2023-11-24T07:43:47.123 INFO:root:teuthology version: 0.0.1.dev205+g2442239 2023-11-24T07:43:47.123 DEBUG:teuthology.run:Teuthology command: teuthology --name rishabh-2023-11-24_07:31:48-fs:volumes-rishabh-mgr-vol-misc-testing-default-smithi --archive /home/teuthworker/archive/rishabh-2023-11-24_07:31:48-fs:volumes-rishabh-mgr-vol-misc-testing-default-smithi/7465575 --verbose --owner scheduled_rishabh@teuthology --description fs:volumes/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{centos_latest} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/volumes/{overrides test/snapshot}} -- /home/teuthworker/archive/rishabh-2023-11-24_07:31:48-fs:volumes-rishabh-mgr-vol-misc-testing-default-smithi/7465575/orig.config.yaml 2023-11-24T07:43:47.190 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-11-24T07:43:47.284 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/rishabh-2023-11-24_07:31:48-fs:volumes-rishabh-mgr-vol-misc-testing-default-smithi/7465575 branch: rishabh-mgr-vol-misc description: fs:volumes/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{centos_latest} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/volumes/{overrides test/snapshot}} email: null first_in_suite: false job_id: '7465575' kernel: client: sha1: distro flavor: default kdb: true sha1: 8247bbb5b470f7e2a1f3eefcc16e24cf1338f606 ktype: distro last_in_suite: false log-rotate: ceph-mds: 10G ceph-osd: 10G machine_type: smithi name: rishabh-2023-11-24_07:31:48-fs:volumes-rishabh-mgr-vol-misc-testing-default-smithi no_nested_subset: false nuke-on-error: true openstack: - volumes: count: 4 size: 20 - machine: disk: 200 os_type: rhel os_version: '8.6' overrides: admin_socket: branch: rishabh-mgr-vol-misc ceph: conf: client: client mount timeout: 600 debug client: 20 debug ms: 1 pid file: '' rados mon op timeout: 900 rados osd op timeout: 900 global: ms die on skipped message: false mds: debug mds: 20 debug mds balancer: 20 debug ms: 1 mds debug frag: true mds debug scatterstat: true mds op complaint time: 180 mds verify scatter: true osd op complaint time: 180 rados mon op timeout: 900 rados osd op timeout: 900 mgr: debug client: 20 debug finisher: 20 debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon op complaint time: 120 osd: bdev async discard: true bdev enable discard: true bluestore allocator: bitmap bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 1/20 debug bluestore: 1/20 debug ms: 1 debug osd: 20 debug rocksdb: 4/10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore osd op complaint time: 180 flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - overall HEALTH_ - \(FS_DEGRADED\) - \(MDS_FAILED\) - \(MDS_DEGRADED\) - \(FS_WITH_FAILED_MDS\) - \(MDS_DAMAGE\) - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - \(FS_INLINE_DATA_DEPRECATED\) - \(POOL_APP_NOT_ENABLED\) - overall HEALTH_ - \(OSD_DOWN\) - \(OSD_ - but it is still running - is not responding - OSD full dropping all updates - OSD near full - pausewr flag - failsafe engaged, dropping updates - failsafe disengaged, no longer dropping - is full \(reached quota - POOL_FULL - POOL_BACKFILLFULL sha1: a13ea5ff00cb4df7a9cc941166ddeb358881144c ceph-deploy: bluestore: true conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} osd: bdev async discard: true bdev enable discard: true bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 1/20 debug bluestore: 1/20 debug rocksdb: 4/10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore fs: xfs install: ceph: flavor: default sha1: a13ea5ff00cb4df7a9cc941166ddeb358881144c kclient: syntax: v2 selinux: whitelist: - scontext=system_u:system_r:logrotate_t:s0 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: mgr-vol-tests sha1: 73843baf45f8a92d8e3fcbea3f024c97e73036da owner: scheduled_rishabh@teuthology priority: 50 repo: https://git.ceph.com/ceph-ci.git roles: - - mon.a - mgr.y - mds.a - mds.c - osd.0 - osd.1 - osd.2 - osd.3 - client.0 - client.2 - - mon.b - mon.c - mgr.x - mds.b - mds.d - osd.4 - osd.5 - osd.6 - osd.7 - client.1 - client.3 seed: 9764 sha1: a13ea5ff00cb4df7a9cc941166ddeb358881144c sleep_before_teardown: 0 suite: fs:volumes suite_branch: mgr-vol-tests suite_path: /home/teuthworker/src/github.com_rishabh-d-dave_ceph_73843baf45f8a92d8e3fcbea3f024c97e73036da/qa suite_relpath: qa suite_repo: https://github.com/rishabh-d-dave/ceph suite_sha1: 73843baf45f8a92d8e3fcbea3f024c97e73036da syslog: ignorelist: - WARNING*.*check_session_state - WARNING*.*__ceph_remove_cap targets: smithi059.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLVvN3+LmzPZ4ixiwyCkS3KZLZ9R6MOPm+kPtaJbl8fExpzbk8TudXCcJWHBL7DGPWFvye2jT6sei9eiX0EUQYs= smithi076.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLVvN3+LmzPZ4ixiwyCkS3KZLZ9R6MOPm+kPtaJbl8fExpzbk8TudXCcJWHBL7DGPWFvye2jT6sei9eiX0EUQYs= tasks: - install: extra_packages: deb: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror rpm: - python3-cephfs - cephfs-top - cephfs-mirror extra_system_packages: deb: - bison - flex - libelf-dev - libssl-dev - network-manager - iproute2 - util-linux - dump - indent - libaio-dev - libtool-bin - uuid-dev - xfslibs-dev - postgresql - postgresql-client - postgresql-common - postgresql-contrib rpm: - bison - flex - elfutils-libelf-devel - openssl-devel - NetworkManager - iproute - util-linux - libacl-devel - libaio-devel - libattr-devel - libtool - libuuid-devel - xfsdump - xfsprogs - xfsprogs-devel - libaio-devel - libtool - libuuid-devel - xfsprogs-devel - postgresql - postgresql-server - postgresql-contrib - ceph: create_rbd_pool: false - kclient: null - cephfs_test_runner: fail_on_skip: false modules: - tasks.cephfs.test_volumes.TestSubvolumeGroupSnapshots - tasks.cephfs.test_volumes.TestSubvolumeSnapshots teuthology: fragments_dropped: [] meta: {} postmerge: - local function is_kupstream() return yaml.ktype == 'upstream' end - local function is_kdistro() return yaml.ktype == 'distro' end - local function is_fuse() return false end - local function syntax_version() return yaml.overrides.kclient.syntax end teuthology_branch: main teuthology_sha1: 2442239d2653456406c25ae0c71b689c8f2657b6 timestamp: 2023-11-24_07:31:48 tube: smithi user: rishabh verbose: false worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.232157 2023-11-24T07:43:47.284 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/github.com_rishabh-d-dave_ceph_73843baf45f8a92d8e3fcbea3f024c97e73036da/qa; will attempt to use it 2023-11-24T07:43:47.285 INFO:teuthology.run:Found tasks at /home/teuthworker/src/github.com_rishabh-d-dave_ceph_73843baf45f8a92d8e3fcbea3f024c97e73036da/qa/tasks 2023-11-24T07:43:47.285 INFO:teuthology.run_tasks:Running task internal.check_packages... 2023-11-24T07:43:47.287 INFO:teuthology.task.internal:Checking packages... 2023-11-24T07:43:47.311 INFO:teuthology.task.internal:Checking packages for os_type 'rhel', flavor 'default' and ceph hash 'a13ea5ff00cb4df7a9cc941166ddeb358881144c' 2023-11-24T07:43:47.311 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2023-11-24T07:43:47.311 INFO:teuthology.packaging:ref: None 2023-11-24T07:43:47.311 INFO:teuthology.packaging:tag: None 2023-11-24T07:43:47.311 INFO:teuthology.packaging:branch: rishabh-mgr-vol-misc 2023-11-24T07:43:47.311 INFO:teuthology.packaging:sha1: a13ea5ff00cb4df7a9cc941166ddeb358881144c 2023-11-24T07:43:47.311 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&ref=rishabh-mgr-vol-misc 2023-11-24T07:43:47.534 INFO:teuthology.task.internal:Found packages for ceph version 18.0.0-7484.ga13ea5ff 2023-11-24T07:43:47.534 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2023-11-24T07:43:47.542 INFO:teuthology.task.internal:no buildpackages task found 2023-11-24T07:43:47.542 INFO:teuthology.run_tasks:Running task internal.save_config... 2023-11-24T07:43:47.648 INFO:teuthology.task.internal:Saving configuration 2023-11-24T07:43:47.666 INFO:teuthology.run_tasks:Running task internal.check_lock... 2023-11-24T07:43:47.672 INFO:teuthology.task.internal.check_lock:Checking locks... 2023-11-24T07:43:47.696 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi059.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/rishabh-2023-11-24_07:31:48-fs:volumes-rishabh-mgr-vol-misc-testing-default-smithi/7465575', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'rhel', 'os_version': '8.6', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-11-24 07:38:24.068042', 'locked_by': 'scheduled_rishabh@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLVvN3+LmzPZ4ixiwyCkS3KZLZ9R6MOPm+kPtaJbl8fExpzbk8TudXCcJWHBL7DGPWFvye2jT6sei9eiX0EUQYs='} 2023-11-24T07:43:47.719 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi076.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/rishabh-2023-11-24_07:31:48-fs:volumes-rishabh-mgr-vol-misc-testing-default-smithi/7465575', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'rhel', 'os_version': '8.6', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-11-24 07:38:24.069710', 'locked_by': 'scheduled_rishabh@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLVvN3+LmzPZ4ixiwyCkS3KZLZ9R6MOPm+kPtaJbl8fExpzbk8TudXCcJWHBL7DGPWFvye2jT6sei9eiX0EUQYs='} 2023-11-24T07:43:47.719 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2023-11-24T07:43:47.726 INFO:teuthology.task.internal:roles: ubuntu@smithi059.front.sepia.ceph.com - ['mon.a', 'mgr.y', 'mds.a', 'mds.c', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'client.2'] 2023-11-24T07:43:47.726 INFO:teuthology.task.internal:roles: ubuntu@smithi076.front.sepia.ceph.com - ['mon.b', 'mon.c', 'mgr.x', 'mds.b', 'mds.d', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'client.3'] 2023-11-24T07:43:47.726 INFO:teuthology.run_tasks:Running task console_log... 2023-11-24T07:43:47.796 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7f2a2ae34b80>, signals=[15]) 2023-11-24T07:43:47.797 INFO:teuthology.run_tasks:Running task internal.connect... 2023-11-24T07:43:47.804 INFO:teuthology.task.internal:Opening connections... 2023-11-24T07:43:47.804 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi059.front.sepia.ceph.com 2023-11-24T07:43:47.806 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi059.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-11-24T07:43:47.878 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi076.front.sepia.ceph.com 2023-11-24T07:43:47.878 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi076.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-11-24T07:43:47.946 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2023-11-24T07:43:47.952 DEBUG:teuthology.orchestra.run.smithi059:> uname -m 2023-11-24T07:43:47.973 INFO:teuthology.orchestra.run.smithi059.stdout:x86_64 2023-11-24T07:43:47.973 DEBUG:teuthology.orchestra.run.smithi059:> cat /etc/os-release 2023-11-24T07:43:48.033 INFO:teuthology.orchestra.run.smithi059.stdout:NAME="Red Hat Enterprise Linux" 2023-11-24T07:43:48.033 INFO:teuthology.orchestra.run.smithi059.stdout:VERSION="8.6 (Ootpa)" 2023-11-24T07:43:48.033 INFO:teuthology.orchestra.run.smithi059.stdout:ID="rhel" 2023-11-24T07:43:48.034 INFO:teuthology.orchestra.run.smithi059.stdout:ID_LIKE="fedora" 2023-11-24T07:43:48.034 INFO:teuthology.orchestra.run.smithi059.stdout:VERSION_ID="8.6" 2023-11-24T07:43:48.034 INFO:teuthology.orchestra.run.smithi059.stdout:PLATFORM_ID="platform:el8" 2023-11-24T07:43:48.034 INFO:teuthology.orchestra.run.smithi059.stdout:PRETTY_NAME="Red Hat Enterprise Linux 8.6 (Ootpa)" 2023-11-24T07:43:48.034 INFO:teuthology.orchestra.run.smithi059.stdout:ANSI_COLOR="0;31" 2023-11-24T07:43:48.034 INFO:teuthology.orchestra.run.smithi059.stdout:CPE_NAME="cpe:/o:redhat:enterprise_linux:8::baseos" 2023-11-24T07:43:48.034 INFO:teuthology.orchestra.run.smithi059.stdout:HOME_URL="https://www.redhat.com/" 2023-11-24T07:43:48.034 INFO:teuthology.orchestra.run.smithi059.stdout:DOCUMENTATION_URL="https://access.redhat.com/documentation/red_hat_enterprise_linux/8/" 2023-11-24T07:43:48.034 INFO:teuthology.orchestra.run.smithi059.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2023-11-24T07:43:48.034 INFO:teuthology.orchestra.run.smithi059.stdout: 2023-11-24T07:43:48.034 INFO:teuthology.orchestra.run.smithi059.stdout:REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 8" 2023-11-24T07:43:48.034 INFO:teuthology.orchestra.run.smithi059.stdout:REDHAT_BUGZILLA_PRODUCT_VERSION=8.6 2023-11-24T07:43:48.035 INFO:teuthology.orchestra.run.smithi059.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux" 2023-11-24T07:43:48.035 INFO:teuthology.orchestra.run.smithi059.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8.6" 2023-11-24T07:43:48.035 INFO:teuthology.lock.ops:Updating smithi059.front.sepia.ceph.com on lock server 2023-11-24T07:43:48.066 DEBUG:teuthology.orchestra.run.smithi076:> uname -m 2023-11-24T07:43:48.086 INFO:teuthology.orchestra.run.smithi076.stdout:x86_64 2023-11-24T07:43:48.086 DEBUG:teuthology.orchestra.run.smithi076:> cat /etc/os-release 2023-11-24T07:43:48.145 INFO:teuthology.orchestra.run.smithi076.stdout:NAME="Red Hat Enterprise Linux" 2023-11-24T07:43:48.146 INFO:teuthology.orchestra.run.smithi076.stdout:VERSION="8.6 (Ootpa)" 2023-11-24T07:43:48.146 INFO:teuthology.orchestra.run.smithi076.stdout:ID="rhel" 2023-11-24T07:43:48.146 INFO:teuthology.orchestra.run.smithi076.stdout:ID_LIKE="fedora" 2023-11-24T07:43:48.146 INFO:teuthology.orchestra.run.smithi076.stdout:VERSION_ID="8.6" 2023-11-24T07:43:48.146 INFO:teuthology.orchestra.run.smithi076.stdout:PLATFORM_ID="platform:el8" 2023-11-24T07:43:48.146 INFO:teuthology.orchestra.run.smithi076.stdout:PRETTY_NAME="Red Hat Enterprise Linux 8.6 (Ootpa)" 2023-11-24T07:43:48.146 INFO:teuthology.orchestra.run.smithi076.stdout:ANSI_COLOR="0;31" 2023-11-24T07:43:48.146 INFO:teuthology.orchestra.run.smithi076.stdout:CPE_NAME="cpe:/o:redhat:enterprise_linux:8::baseos" 2023-11-24T07:43:48.146 INFO:teuthology.orchestra.run.smithi076.stdout:HOME_URL="https://www.redhat.com/" 2023-11-24T07:43:48.147 INFO:teuthology.orchestra.run.smithi076.stdout:DOCUMENTATION_URL="https://access.redhat.com/documentation/red_hat_enterprise_linux/8/" 2023-11-24T07:43:48.147 INFO:teuthology.orchestra.run.smithi076.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2023-11-24T07:43:48.147 INFO:teuthology.orchestra.run.smithi076.stdout: 2023-11-24T07:43:48.147 INFO:teuthology.orchestra.run.smithi076.stdout:REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 8" 2023-11-24T07:43:48.147 INFO:teuthology.orchestra.run.smithi076.stdout:REDHAT_BUGZILLA_PRODUCT_VERSION=8.6 2023-11-24T07:43:48.148 INFO:teuthology.orchestra.run.smithi076.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux" 2023-11-24T07:43:48.148 INFO:teuthology.orchestra.run.smithi076.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8.6" 2023-11-24T07:43:48.148 INFO:teuthology.lock.ops:Updating smithi076.front.sepia.ceph.com on lock server 2023-11-24T07:43:48.166 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2023-11-24T07:43:48.175 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2023-11-24T07:43:48.180 INFO:teuthology.task.internal:Checking for old test directory... 2023-11-24T07:43:48.181 DEBUG:teuthology.orchestra.run.smithi059:> test '!' -e /home/ubuntu/cephtest 2023-11-24T07:43:48.183 DEBUG:teuthology.orchestra.run.smithi076:> test '!' -e /home/ubuntu/cephtest 2023-11-24T07:43:48.203 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2023-11-24T07:43:48.209 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2023-11-24T07:43:48.209 DEBUG:teuthology.orchestra.run.smithi059:> test -z $(ls -A /var/lib/ceph) 2023-11-24T07:43:48.243 DEBUG:teuthology.orchestra.run.smithi076:> test -z $(ls -A /var/lib/ceph) 2023-11-24T07:43:48.271 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2023-11-24T07:43:48.326 INFO:teuthology.run_tasks:Running task kernel... 2023-11-24T07:43:48.342 INFO:teuthology.task.kernel:normalize config orig: {'client': {'sha1': 'distro'}, 'flavor': 'default', 'kdb': True, 'sha1': '8247bbb5b470f7e2a1f3eefcc16e24cf1338f606'} 2023-11-24T07:43:48.343 INFO:teuthology.task.kernel:normalize config final: {'client.0': {'sha1': 'distro'}, 'client.2': {'sha1': 'distro'}, 'client.1': {'sha1': 'distro'}, 'client.3': {'sha1': 'distro'}} 2023-11-24T07:43:48.343 INFO:teuthology.task.kernel:config {'client.0': {'sha1': 'distro'}, 'client.1': {'sha1': 'distro'}}, timeout 300 2023-11-24T07:43:48.343 DEBUG:teuthology.orchestra.run.smithi059:> test -f /run/.containerenv -o -f /.dockerenv 2023-11-24T07:43:48.344 DEBUG:teuthology.orchestra.run.smithi076:> test -f /run/.containerenv -o -f /.dockerenv 2023-11-24T07:43:48.362 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-11-24T07:43:48.363 DEBUG:teuthology.orchestra.run.smithi076:> uname -r 2023-11-24T07:43:48.364 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-11-24T07:43:48.364 DEBUG:teuthology.orchestra.run.smithi059:> uname -r 2023-11-24T07:43:48.423 INFO:teuthology.orchestra.run.smithi076.stdout:4.18.0-372.9.1.el8.x86_64 2023-11-24T07:43:48.423 INFO:teuthology.task.kernel:Running kernel on smithi076: 4.18.0-372.9.1.el8.x86_64 2023-11-24T07:43:48.423 DEBUG:teuthology.orchestra.run.smithi076:> sudo yum install -y kernel 2023-11-24T07:43:48.425 INFO:teuthology.orchestra.run.smithi059.stdout:4.18.0-372.9.1.el8.x86_64 2023-11-24T07:43:48.425 INFO:teuthology.task.kernel:Running kernel on smithi059: 4.18.0-372.9.1.el8.x86_64 2023-11-24T07:43:48.425 DEBUG:teuthology.orchestra.run.smithi059:> sudo yum install -y kernel 2023-11-24T07:43:49.590 INFO:teuthology.orchestra.run.smithi076.stdout:Updating Subscription Management repositories. 2023-11-24T07:43:49.590 INFO:teuthology.orchestra.run.smithi076.stdout:Unable to read consumer identity 2023-11-24T07:43:49.705 INFO:teuthology.orchestra.run.smithi059.stdout:Updating Subscription Management repositories. 2023-11-24T07:43:49.705 INFO:teuthology.orchestra.run.smithi059.stdout:Unable to read consumer identity 2023-11-24T07:43:51.068 INFO:teuthology.orchestra.run.smithi076.stdout:Copr repo for python3-asyncssh owned by ceph 15 kB/s | 3.5 kB 00:00 2023-11-24T07:43:51.285 INFO:teuthology.orchestra.run.smithi059.stdout:Copr repo for python3-asyncssh owned by ceph 15 kB/s | 3.5 kB 00:00 2023-11-24T07:43:51.911 INFO:teuthology.orchestra.run.smithi076.stdout:Extra Packages for Enterprise Linux 21 MB/s | 16 MB 00:00 2023-11-24T07:43:53.066 INFO:teuthology.orchestra.run.smithi059.stdout:Extra Packages for Enterprise Linux 9.1 MB/s | 16 MB 00:01 2023-11-24T07:43:55.402 INFO:teuthology.orchestra.run.smithi076.stdout:lab-extras 415 kB/s | 24 kB 00:00 2023-11-24T07:43:56.097 INFO:teuthology.orchestra.run.smithi076.stdout:Package kernel-4.18.0-372.9.1.el8.x86_64 is already installed. 2023-11-24T07:43:56.144 INFO:teuthology.orchestra.run.smithi076.stdout:Dependencies resolved. 2023-11-24T07:43:56.145 INFO:teuthology.orchestra.run.smithi076.stdout:Nothing to do. 2023-11-24T07:43:56.145 INFO:teuthology.orchestra.run.smithi076.stdout:Complete! 2023-11-24T07:43:56.276 DEBUG:teuthology.orchestra.run.smithi076:> echo no | sudo yum reinstall kernel || true 2023-11-24T07:43:56.608 INFO:teuthology.orchestra.run.smithi076.stdout:Updating Subscription Management repositories. 2023-11-24T07:43:56.609 INFO:teuthology.orchestra.run.smithi076.stdout:Unable to read consumer identity 2023-11-24T07:43:56.784 INFO:teuthology.orchestra.run.smithi059.stdout:lab-extras 275 kB/s | 24 kB 00:00 2023-11-24T07:43:56.866 INFO:teuthology.orchestra.run.smithi076.stdout:Last metadata expiration check: 0:00:01 ago on Fri 24 Nov 2023 07:43:55 AM UTC. 2023-11-24T07:43:56.906 INFO:teuthology.orchestra.run.smithi076.stdout:Installed package kernel-4.18.0-372.9.1.el8.x86_64 (from anaconda) not available. 2023-11-24T07:43:56.956 INFO:teuthology.orchestra.run.smithi076.stderr:Error: No packages marked for reinstall. 2023-11-24T07:43:57.002 DEBUG:teuthology.orchestra.run.smithi076:> sudo yum reinstall -y kernel || true 2023-11-24T07:43:57.334 INFO:teuthology.orchestra.run.smithi076.stdout:Updating Subscription Management repositories. 2023-11-24T07:43:57.334 INFO:teuthology.orchestra.run.smithi076.stdout:Unable to read consumer identity 2023-11-24T07:43:57.511 INFO:teuthology.orchestra.run.smithi059.stdout:Package kernel-4.18.0-372.9.1.el8.x86_64 is already installed. 2023-11-24T07:43:57.563 INFO:teuthology.orchestra.run.smithi059.stdout:Dependencies resolved. 2023-11-24T07:43:57.564 INFO:teuthology.orchestra.run.smithi059.stdout:Nothing to do. 2023-11-24T07:43:57.564 INFO:teuthology.orchestra.run.smithi059.stdout:Complete! 2023-11-24T07:43:57.588 INFO:teuthology.orchestra.run.smithi076.stdout:Last metadata expiration check: 0:00:02 ago on Fri 24 Nov 2023 07:43:55 AM UTC. 2023-11-24T07:43:57.627 INFO:teuthology.orchestra.run.smithi076.stdout:Installed package kernel-4.18.0-372.9.1.el8.x86_64 (from anaconda) not available. 2023-11-24T07:43:57.669 INFO:teuthology.orchestra.run.smithi076.stderr:Error: No packages marked for reinstall. 2023-11-24T07:43:57.686 DEBUG:teuthology.orchestra.run.smithi059:> echo no | sudo yum reinstall kernel || true 2023-11-24T07:43:57.719 DEBUG:teuthology.orchestra.run.smithi076:> rpm -q kernel | sort -rV | head -n 1 2023-11-24T07:43:57.799 INFO:teuthology.orchestra.run.smithi076.stdout:kernel-4.18.0-372.9.1.el8.x86_64 2023-11-24T07:43:57.799 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 4.18.0-372.9.1.el8.x86_64 2023-11-24T07:43:57.799 INFO:teuthology.task.kernel:Newest distro kernel installed and running 2023-11-24T07:43:57.799 DEBUG:teuthology.parallel:result is None 2023-11-24T07:43:58.037 INFO:teuthology.orchestra.run.smithi059.stdout:Updating Subscription Management repositories. 2023-11-24T07:43:58.037 INFO:teuthology.orchestra.run.smithi059.stdout:Unable to read consumer identity 2023-11-24T07:43:58.313 INFO:teuthology.orchestra.run.smithi059.stdout:Last metadata expiration check: 0:00:02 ago on Fri 24 Nov 2023 07:43:56 AM UTC. 2023-11-24T07:43:58.355 INFO:teuthology.orchestra.run.smithi059.stdout:Installed package kernel-4.18.0-372.9.1.el8.x86_64 (from anaconda) not available. 2023-11-24T07:43:58.385 INFO:teuthology.orchestra.run.smithi059.stderr:Error: No packages marked for reinstall. 2023-11-24T07:43:58.433 DEBUG:teuthology.orchestra.run.smithi059:> sudo yum reinstall -y kernel || true 2023-11-24T07:43:58.783 INFO:teuthology.orchestra.run.smithi059.stdout:Updating Subscription Management repositories. 2023-11-24T07:43:58.783 INFO:teuthology.orchestra.run.smithi059.stdout:Unable to read consumer identity 2023-11-24T07:43:59.050 INFO:teuthology.orchestra.run.smithi059.stdout:Last metadata expiration check: 0:00:03 ago on Fri 24 Nov 2023 07:43:56 AM UTC. 2023-11-24T07:43:59.095 INFO:teuthology.orchestra.run.smithi059.stdout:Installed package kernel-4.18.0-372.9.1.el8.x86_64 (from anaconda) not available. 2023-11-24T07:43:59.123 INFO:teuthology.orchestra.run.smithi059.stderr:Error: No packages marked for reinstall. 2023-11-24T07:43:59.175 DEBUG:teuthology.orchestra.run.smithi059:> rpm -q kernel | sort -rV | head -n 1 2023-11-24T07:43:59.256 INFO:teuthology.orchestra.run.smithi059.stdout:kernel-4.18.0-372.9.1.el8.x86_64 2023-11-24T07:43:59.256 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 4.18.0-372.9.1.el8.x86_64 2023-11-24T07:43:59.257 INFO:teuthology.task.kernel:Newest distro kernel installed and running 2023-11-24T07:43:59.257 DEBUG:teuthology.parallel:result is None 2023-11-24T07:43:59.257 INFO:teuthology.run_tasks:Running task internal.base... 2023-11-24T07:43:59.264 INFO:teuthology.task.internal:Creating test directory... 2023-11-24T07:43:59.264 DEBUG:teuthology.orchestra.run.smithi059:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2023-11-24T07:43:59.266 DEBUG:teuthology.orchestra.run.smithi076:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2023-11-24T07:43:59.286 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2023-11-24T07:43:59.293 INFO:teuthology.run_tasks:Running task internal.archive... 2023-11-24T07:43:59.304 INFO:teuthology.task.internal:Creating archive directory... 2023-11-24T07:43:59.304 DEBUG:teuthology.orchestra.run.smithi059:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2023-11-24T07:43:59.323 DEBUG:teuthology.orchestra.run.smithi076:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2023-11-24T07:43:59.352 INFO:teuthology.run_tasks:Running task internal.coredump... 2023-11-24T07:43:59.358 INFO:teuthology.task.internal:Enabling coredump saving... 2023-11-24T07:43:59.358 DEBUG:teuthology.orchestra.run.smithi059:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2023-11-24T07:43:59.379 DEBUG:teuthology.orchestra.run.smithi076:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2023-11-24T07:43:59.408 INFO:teuthology.orchestra.run.smithi059.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-11-24T07:43:59.425 INFO:teuthology.orchestra.run.smithi059.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-11-24T07:43:59.433 INFO:teuthology.orchestra.run.smithi076.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-11-24T07:43:59.455 INFO:teuthology.orchestra.run.smithi076.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-11-24T07:43:59.457 INFO:teuthology.run_tasks:Running task internal.sudo... 2023-11-24T07:43:59.467 INFO:teuthology.task.internal:Configuring sudo... 2023-11-24T07:43:59.467 DEBUG:teuthology.orchestra.run.smithi059:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2023-11-24T07:43:59.470 DEBUG:teuthology.orchestra.run.smithi076:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2023-11-24T07:43:59.503 INFO:teuthology.run_tasks:Running task internal.syslog... 2023-11-24T07:43:59.511 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2023-11-24T07:43:59.511 DEBUG:teuthology.orchestra.run.smithi059:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2023-11-24T07:43:59.538 DEBUG:teuthology.orchestra.run.smithi076:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2023-11-24T07:43:59.563 DEBUG:teuthology.orchestra.run.smithi059:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2023-11-24T07:43:59.636 DEBUG:teuthology.orchestra.run.smithi059:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2023-11-24T07:43:59.685 DEBUG:teuthology.orchestra.run.smithi059:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2023-11-24T07:43:59.770 DEBUG:teuthology.orchestra.run.smithi059:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2023-11-24T07:43:59.813 DEBUG:teuthology.orchestra.run.smithi059:> set -ex 2023-11-24T07:43:59.813 DEBUG:teuthology.orchestra.run.smithi059:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2023-11-24T07:43:59.889 DEBUG:teuthology.orchestra.run.smithi076:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2023-11-24T07:43:59.940 DEBUG:teuthology.orchestra.run.smithi076:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2023-11-24T07:43:59.992 DEBUG:teuthology.orchestra.run.smithi076:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2023-11-24T07:44:00.077 DEBUG:teuthology.orchestra.run.smithi076:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2023-11-24T07:44:00.126 DEBUG:teuthology.orchestra.run.smithi076:> set -ex 2023-11-24T07:44:00.126 DEBUG:teuthology.orchestra.run.smithi076:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2023-11-24T07:44:00.203 DEBUG:teuthology.orchestra.run.smithi059:> sudo service rsyslog restart 2023-11-24T07:44:00.206 DEBUG:teuthology.orchestra.run.smithi076:> sudo service rsyslog restart 2023-11-24T07:44:00.285 INFO:teuthology.orchestra.run.smithi059.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2023-11-24T07:44:00.305 INFO:teuthology.orchestra.run.smithi076.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2023-11-24T07:44:00.772 INFO:teuthology.run_tasks:Running task internal.timer... 2023-11-24T07:44:00.782 INFO:teuthology.task.internal:Starting timer... 2023-11-24T07:44:00.783 INFO:teuthology.run_tasks:Running task pcp... 2023-11-24T07:44:00.819 INFO:teuthology.run_tasks:Running task selinux... 2023-11-24T07:44:00.827 DEBUG:teuthology.task:Applying overrides for task selinux: {'whitelist': ['scontext=system_u:system_r:logrotate_t:s0']} 2023-11-24T07:44:00.828 DEBUG:teuthology.orchestra.run.smithi059:> sudo service auditd rotate 2023-11-24T07:44:00.882 INFO:teuthology.orchestra.run.smithi059.stdout:Rotating logs: 2023-11-24T07:44:00.884 DEBUG:teuthology.orchestra.run.smithi076:> sudo service auditd rotate 2023-11-24T07:44:00.938 INFO:teuthology.orchestra.run.smithi076.stdout:Rotating logs: 2023-11-24T07:44:00.940 DEBUG:teuthology.task.selinux:Getting current SELinux state 2023-11-24T07:44:00.940 DEBUG:teuthology.orchestra.run.smithi059:> /usr/sbin/getenforce 2023-11-24T07:44:00.965 INFO:teuthology.orchestra.run.smithi059.stdout:Permissive 2023-11-24T07:44:00.965 DEBUG:teuthology.orchestra.run.smithi076:> /usr/sbin/getenforce 2023-11-24T07:44:00.986 INFO:teuthology.orchestra.run.smithi076.stdout:Permissive 2023-11-24T07:44:00.986 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@smithi059.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi076.front.sepia.ceph.com': 'permissive'} 2023-11-24T07:44:00.987 DEBUG:teuthology.orchestra.run.smithi059:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2023-11-24T07:44:01.034 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-11-24T07:44:01.035 DEBUG:teuthology.orchestra.run.smithi076:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2023-11-24T07:44:01.063 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-11-24T07:44:01.063 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2023-11-24T07:44:01.063 DEBUG:teuthology.orchestra.run.smithi059:> sudo /usr/sbin/setenforce permissive 2023-11-24T07:44:01.126 DEBUG:teuthology.orchestra.run.smithi076:> sudo /usr/sbin/setenforce permissive 2023-11-24T07:44:01.156 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2023-11-24T07:44:01.165 DEBUG:teuthology.repo_utils:Setting repo remote to https://git.ceph.com/ceph-cm-ansible.git 2023-11-24T07:44:01.170 INFO:teuthology.repo_utils:Fetching main from origin 2023-11-24T07:44:01.297 INFO:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2023-11-24T07:44:01.349 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2023-11-24T07:44:01.350 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi059.front.sepia.ceph.com,smithi076.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2023-11-24T07:53:07.702 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi059.front.sepia.ceph.com'), Remote(name='ubuntu@smithi076.front.sepia.ceph.com')] 2023-11-24T07:53:07.703 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi059.front.sepia.ceph.com' 2023-11-24T07:53:07.704 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi059.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-11-24T07:53:07.775 DEBUG:teuthology.orchestra.run.smithi059:> true 2023-11-24T07:53:07.844 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi059.front.sepia.ceph.com' 2023-11-24T07:53:07.845 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi076.front.sepia.ceph.com' 2023-11-24T07:53:07.845 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi076.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-11-24T07:53:07.913 DEBUG:teuthology.orchestra.run.smithi076:> true 2023-11-24T07:53:07.979 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi076.front.sepia.ceph.com' 2023-11-24T07:53:07.979 INFO:teuthology.run_tasks:Running task clock... 2023-11-24T07:53:07.996 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2023-11-24T07:53:07.997 INFO:teuthology.orchestra.run:Running command with timeout 360 2023-11-24T07:53:07.997 DEBUG:teuthology.orchestra.run.smithi059:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2023-11-24T07:53:08.000 INFO:teuthology.orchestra.run:Running command with timeout 360 2023-11-24T07:53:08.000 DEBUG:teuthology.orchestra.run.smithi076:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2023-11-24T07:53:08.035 INFO:teuthology.orchestra.run.smithi059.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2023-11-24T07:53:08.051 INFO:teuthology.orchestra.run.smithi059.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2023-11-24T07:53:08.052 INFO:teuthology.orchestra.run.smithi076.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2023-11-24T07:53:08.066 INFO:teuthology.orchestra.run.smithi076.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2023-11-24T07:53:08.126 INFO:teuthology.orchestra.run.smithi059.stderr:sudo: ntpd: command not found 2023-11-24T07:53:08.139 INFO:teuthology.orchestra.run.smithi059.stdout:506 Cannot talk to daemon 2023-11-24T07:53:08.148 INFO:teuthology.orchestra.run.smithi076.stderr:sudo: ntpd: command not found 2023-11-24T07:53:08.154 INFO:teuthology.orchestra.run.smithi059.stderr:Failed to start ntp.service: Unit ntp.service not found. 2023-11-24T07:53:08.160 INFO:teuthology.orchestra.run.smithi076.stdout:506 Cannot talk to daemon 2023-11-24T07:53:08.169 INFO:teuthology.orchestra.run.smithi059.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2023-11-24T07:53:08.174 INFO:teuthology.orchestra.run.smithi076.stderr:Failed to start ntp.service: Unit ntp.service not found. 2023-11-24T07:53:08.188 INFO:teuthology.orchestra.run.smithi076.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2023-11-24T07:53:08.236 INFO:teuthology.orchestra.run.smithi059.stderr:bash: ntpq: command not found 2023-11-24T07:53:08.240 INFO:teuthology.orchestra.run.smithi059.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2023-11-24T07:53:08.240 INFO:teuthology.orchestra.run.smithi059.stdout:=============================================================================== 2023-11-24T07:53:08.240 INFO:teuthology.orchestra.run.smithi059.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-11-24T07:53:08.240 INFO:teuthology.orchestra.run.smithi059.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-11-24T07:53:08.240 INFO:teuthology.orchestra.run.smithi059.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-11-24T07:53:08.240 INFO:teuthology.orchestra.run.smithi059.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-11-24T07:53:08.264 INFO:teuthology.orchestra.run.smithi076.stderr:bash: ntpq: command not found 2023-11-24T07:53:08.267 INFO:teuthology.orchestra.run.smithi076.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2023-11-24T07:53:08.268 INFO:teuthology.orchestra.run.smithi076.stdout:=============================================================================== 2023-11-24T07:53:08.268 INFO:teuthology.orchestra.run.smithi076.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-11-24T07:53:08.268 INFO:teuthology.orchestra.run.smithi076.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-11-24T07:53:08.268 INFO:teuthology.orchestra.run.smithi076.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-11-24T07:53:08.268 INFO:teuthology.orchestra.run.smithi076.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-11-24T07:53:08.269 INFO:teuthology.run_tasks:Running task install... 2023-11-24T07:53:08.277 DEBUG:teuthology.task.install:project ceph 2023-11-24T07:53:08.278 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': 'a13ea5ff00cb4df7a9cc941166ddeb358881144c'}} 2023-11-24T07:53:08.278 DEBUG:teuthology.task.install:config {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': 'a13ea5ff00cb4df7a9cc941166ddeb358881144c'} 2023-11-24T07:53:08.278 INFO:teuthology.task.install:Using flavor: default 2023-11-24T07:53:08.284 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2023-11-24T07:53:08.284 INFO:teuthology.task.install:extra packages: {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']} 2023-11-24T07:53:08.284 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': 'a13ea5ff00cb4df7a9cc941166ddeb358881144c', 'tag': None, 'wait_for_package': False} 2023-11-24T07:53:08.285 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=a13ea5ff00cb4df7a9cc941166ddeb358881144c 2023-11-24T07:53:08.286 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': 'a13ea5ff00cb4df7a9cc941166ddeb358881144c', 'tag': None, 'wait_for_package': False} 2023-11-24T07:53:08.286 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=a13ea5ff00cb4df7a9cc941166ddeb358881144c 2023-11-24T07:53:08.508 INFO:teuthology.task.install.rpm:Pulling from https://1.chacra.ceph.com/r/ceph/rishabh-mgr-vol-misc/a13ea5ff00cb4df7a9cc941166ddeb358881144c/centos/8/flavors/default/ 2023-11-24T07:53:08.508 INFO:teuthology.task.install.rpm:Package version is 18.0.0-7484.ga13ea5ff 2023-11-24T07:53:08.513 INFO:teuthology.task.install.rpm:Pulling from https://1.chacra.ceph.com/r/ceph/rishabh-mgr-vol-misc/a13ea5ff00cb4df7a9cc941166ddeb358881144c/centos/8/flavors/default/ 2023-11-24T07:53:08.513 INFO:teuthology.task.install.rpm:Package version is 18.0.0-7484.ga13ea5ff 2023-11-24T07:53:08.635 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://1.chacra.ceph.com/r/ceph/rishabh-mgr-vol-misc/a13ea5ff00cb4df7a9cc941166ddeb358881144c/centos/8/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://1.chacra.ceph.com/r/ceph/rishabh-mgr-vol-misc/a13ea5ff00cb4df7a9cc941166ddeb358881144c/centos/8/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://1.chacra.ceph.com/r/ceph/rishabh-mgr-vol-misc/a13ea5ff00cb4df7a9cc941166ddeb358881144c/centos/8/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2023-11-24T07:53:08.636 DEBUG:teuthology.orchestra.run.smithi076:> set -ex 2023-11-24T07:53:08.636 DEBUG:teuthology.orchestra.run.smithi076:> sudo dd of=/etc/yum.repos.d/ceph.repo 2023-11-24T07:53:08.666 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2023-11-24T07:53:08.667 DEBUG:teuthology.orchestra.run.smithi076:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/a13ea5ff00cb4df7a9cc941166ddeb358881144c/;g' /etc/yum.repos.d/ceph.repo ; fi 2023-11-24T07:53:08.690 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://1.chacra.ceph.com/r/ceph/rishabh-mgr-vol-misc/a13ea5ff00cb4df7a9cc941166ddeb358881144c/centos/8/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://1.chacra.ceph.com/r/ceph/rishabh-mgr-vol-misc/a13ea5ff00cb4df7a9cc941166ddeb358881144c/centos/8/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://1.chacra.ceph.com/r/ceph/rishabh-mgr-vol-misc/a13ea5ff00cb4df7a9cc941166ddeb358881144c/centos/8/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2023-11-24T07:53:08.690 DEBUG:teuthology.orchestra.run.smithi059:> set -ex 2023-11-24T07:53:08.690 DEBUG:teuthology.orchestra.run.smithi059:> sudo dd of=/etc/yum.repos.d/ceph.repo 2023-11-24T07:53:08.723 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2023-11-24T07:53:08.723 DEBUG:teuthology.orchestra.run.smithi059:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/a13ea5ff00cb4df7a9cc941166ddeb358881144c/;g' /etc/yum.repos.d/ceph.repo ; fi 2023-11-24T07:53:08.735 DEBUG:teuthology.orchestra.run.smithi076:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2023-11-24T07:53:08.796 DEBUG:teuthology.orchestra.run.smithi059:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2023-11-24T07:53:08.812 DEBUG:teuthology.orchestra.run.smithi076:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2023-11-24T07:53:08.878 INFO:teuthology.orchestra.run.smithi076.stdout:check_obsoletes = 1 2023-11-24T07:53:08.879 DEBUG:teuthology.orchestra.run.smithi076:> sudo yum clean all 2023-11-24T07:53:08.881 DEBUG:teuthology.orchestra.run.smithi059:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2023-11-24T07:53:08.910 INFO:teuthology.orchestra.run.smithi059.stdout:check_obsoletes = 1 2023-11-24T07:53:08.912 DEBUG:teuthology.orchestra.run.smithi059:> sudo yum clean all 2023-11-24T07:53:09.247 INFO:teuthology.orchestra.run.smithi076.stdout:Updating Subscription Management repositories. 2023-11-24T07:53:09.307 INFO:teuthology.orchestra.run.smithi059.stdout:Updating Subscription Management repositories. 2023-11-24T07:53:09.513 INFO:teuthology.orchestra.run.smithi076.stdout:44 files removed 2023-11-24T07:53:09.559 DEBUG:teuthology.orchestra.run.smithi076:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2023-11-24T07:53:09.590 INFO:teuthology.orchestra.run.smithi059.stdout:44 files removed 2023-11-24T07:53:09.638 DEBUG:teuthology.orchestra.run.smithi059:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2023-11-24T07:53:09.892 INFO:teuthology.orchestra.run.smithi076.stdout:Updating Subscription Management repositories. 2023-11-24T07:53:09.989 INFO:teuthology.orchestra.run.smithi059.stdout:Updating Subscription Management repositories. 2023-11-24T07:53:10.574 INFO:teuthology.orchestra.run.smithi076.stdout:ceph packages for x86_64 261 kB/s | 81 kB 00:00 2023-11-24T07:53:10.726 INFO:teuthology.orchestra.run.smithi059.stdout:ceph packages for x86_64 229 kB/s | 81 kB 00:00 2023-11-24T07:53:10.950 INFO:teuthology.orchestra.run.smithi076.stdout:ceph noarch packages 28 kB/s | 9.5 kB 00:00 2023-11-24T07:53:11.094 INFO:teuthology.orchestra.run.smithi059.stdout:ceph noarch packages 28 kB/s | 9.5 kB 00:00 2023-11-24T07:53:11.259 INFO:teuthology.orchestra.run.smithi076.stdout:ceph source packages 5.2 kB/s | 1.5 kB 00:00 2023-11-24T07:53:11.411 INFO:teuthology.orchestra.run.smithi059.stdout:ceph source packages 5.1 kB/s | 1.5 kB 00:00 2023-11-24T07:53:11.511 INFO:teuthology.orchestra.run.smithi076.stdout:Copr repo for python3-asyncssh owned by ceph 15 kB/s | 3.5 kB 00:00 2023-11-24T07:53:11.629 INFO:teuthology.orchestra.run.smithi059.stdout:Copr repo for python3-asyncssh owned by ceph 18 kB/s | 3.5 kB 00:00 2023-11-24T07:53:12.553 INFO:teuthology.orchestra.run.smithi076.stdout:Extra Packages for Enterprise Linux 16 MB/s | 16 MB 00:01 2023-11-24T07:53:13.582 INFO:teuthology.orchestra.run.smithi059.stdout:Extra Packages for Enterprise Linux 8.3 MB/s | 16 MB 00:01 2023-11-24T07:53:16.078 INFO:teuthology.orchestra.run.smithi076.stdout:lab-extras 559 kB/s | 24 kB 00:00 2023-11-24T07:53:19.648 INFO:teuthology.orchestra.run.smithi076.stdout:Red Hat Enterprise Linux 8 for x86_64 - BaseOS 68 MB/s | 53 MB 00:00 2023-11-24T07:53:19.648 INFO:teuthology.orchestra.run.smithi059.stdout:lab-extras 226 kB/s | 24 kB 00:00 2023-11-24T07:53:19.648 INFO:teuthology.orchestra.run.smithi059.stdout:Red Hat Enterprise Linux 8 for x86_64 - AppStre 54 MB/s | 47 MB 00:00 2023-11-24T07:53:25.115 INFO:teuthology.orchestra.run.smithi076.stdout:Red Hat Enterprise Linux 8 for x86_64 - AppStre 55 MB/s | 47 MB 00:00 2023-11-24T07:53:26.799 INFO:teuthology.orchestra.run.smithi059.stdout:Red Hat Enterprise Linux 8 for x86_64 - BaseOS 56 MB/s | 53 MB 00:00 2023-11-24T07:53:36.013 INFO:teuthology.orchestra.run.smithi076.stdout:Package librados2-1:12.2.7-9.el8.x86_64 is already installed. 2023-11-24T07:53:36.014 INFO:teuthology.orchestra.run.smithi076.stdout:Package librbd1-1:12.2.7-9.el8.x86_64 is already installed. 2023-11-24T07:53:36.028 INFO:teuthology.orchestra.run.smithi076.stdout:Package NetworkManager-1:1.36.0-4.el8.x86_64 is already installed. 2023-11-24T07:53:36.030 INFO:teuthology.orchestra.run.smithi076.stdout:Package iproute-5.15.0-4.el8.x86_64 is already installed. 2023-11-24T07:53:36.031 INFO:teuthology.orchestra.run.smithi076.stdout:Package util-linux-2.32.1-35.el8.x86_64 is already installed. 2023-11-24T07:53:36.032 INFO:teuthology.orchestra.run.smithi076.stdout:Package libacl-devel-2.2.53-1.el8.x86_64 is already installed. 2023-11-24T07:53:36.034 INFO:teuthology.orchestra.run.smithi076.stdout:Package libattr-devel-2.4.48-3.el8.x86_64 is already installed. 2023-11-24T07:53:36.038 INFO:teuthology.orchestra.run.smithi076.stdout:Package xfsdump-3.1.8-4.el8.x86_64 is already installed. 2023-11-24T07:53:36.039 INFO:teuthology.orchestra.run.smithi076.stdout:Package xfsprogs-5.0.0-10.el8.x86_64 is already installed. 2023-11-24T07:53:36.169 INFO:teuthology.orchestra.run.smithi076.stdout:Dependencies resolved. 2023-11-24T07:53:36.183 INFO:teuthology.orchestra.run.smithi076.stdout:=========================================================================================================================== 2023-11-24T07:53:36.184 INFO:teuthology.orchestra.run.smithi076.stdout: Package Arch Version Repository Size 2023-11-24T07:53:36.184 INFO:teuthology.orchestra.run.smithi076.stdout:=========================================================================================================================== 2023-11-24T07:53:36.184 INFO:teuthology.orchestra.run.smithi076.stdout:Installing: 2023-11-24T07:53:36.184 INFO:teuthology.orchestra.run.smithi076.stdout: NetworkManager-initscripts-updown noarch 1:1.36.0-9.el8_6 rhel-8-for-x86_64-baseos-rpms 138 k 2023-11-24T07:53:36.184 INFO:teuthology.orchestra.run.smithi076.stdout: bison x86_64 3.0.4-10.el8 rhel-8-for-x86_64-appstream-rpms 688 k 2023-11-24T07:53:36.184 INFO:teuthology.orchestra.run.smithi076.stdout: ceph x86_64 2:18.0.0-7484.ga13ea5ff.el8 ceph 6.4 k 2023-11-24T07:53:36.184 INFO:teuthology.orchestra.run.smithi076.stdout: ceph-base x86_64 2:18.0.0-7484.ga13ea5ff.el8 ceph 5.4 M 2023-11-24T07:53:36.184 INFO:teuthology.orchestra.run.smithi076.stdout: ceph-fuse x86_64 2:18.0.0-7484.ga13ea5ff.el8 ceph 920 k 2023-11-24T07:53:36.184 INFO:teuthology.orchestra.run.smithi076.stdout: ceph-immutable-object-cache x86_64 2:18.0.0-7484.ga13ea5ff.el8 ceph 171 k 2023-11-24T07:53:36.184 INFO:teuthology.orchestra.run.smithi076.stdout: ceph-mgr x86_64 2:18.0.0-7484.ga13ea5ff.el8 ceph 1.6 M 2023-11-24T07:53:36.184 INFO:teuthology.orchestra.run.smithi076.stdout: ceph-mgr-cephadm noarch 2:18.0.0-7484.ga13ea5ff.el8 ceph-noarch 133 k 2023-11-24T07:53:36.184 INFO:teuthology.orchestra.run.smithi076.stdout: ceph-mgr-dashboard noarch 2:18.0.0-7484.ga13ea5ff.el8 ceph-noarch 1.8 M 2023-11-24T07:53:36.185 INFO:teuthology.orchestra.run.smithi076.stdout: ceph-mgr-diskprediction-local noarch 2:18.0.0-7484.ga13ea5ff.el8 ceph-noarch 7.4 M 2023-11-24T07:53:36.185 INFO:teuthology.orchestra.run.smithi076.stdout: ceph-mgr-rook noarch 2:18.0.0-7484.ga13ea5ff.el8 ceph-noarch 50 k 2023-11-24T07:53:36.185 INFO:teuthology.orchestra.run.smithi076.stdout: ceph-radosgw x86_64 2:18.0.0-7484.ga13ea5ff.el8 ceph 14 M 2023-11-24T07:53:36.185 INFO:teuthology.orchestra.run.smithi076.stdout: ceph-test x86_64 2:18.0.0-7484.ga13ea5ff.el8 ceph 49 M 2023-11-24T07:53:36.185 INFO:teuthology.orchestra.run.smithi076.stdout: ceph-volume noarch 2:18.0.0-7484.ga13ea5ff.el8 ceph-noarch 262 k 2023-11-24T07:53:36.185 INFO:teuthology.orchestra.run.smithi076.stdout: cephadm noarch 2:18.0.0-7484.ga13ea5ff.el8 ceph-noarch 407 k 2023-11-24T07:53:36.185 INFO:teuthology.orchestra.run.smithi076.stdout: cephfs-mirror x86_64 2:18.0.0-7484.ga13ea5ff.el8 ceph 262 k 2023-11-24T07:53:36.185 INFO:teuthology.orchestra.run.smithi076.stdout: cephfs-top noarch 2:18.0.0-7484.ga13ea5ff.el8 ceph-noarch 18 k 2023-11-24T07:53:36.185 INFO:teuthology.orchestra.run.smithi076.stdout: elfutils-libelf-devel x86_64 0.186-1.el8 rhel-8-for-x86_64-baseos-rpms 60 k 2023-11-24T07:53:36.185 INFO:teuthology.orchestra.run.smithi076.stdout: flex x86_64 2.6.1-9.el8 rhel-8-for-x86_64-appstream-rpms 320 k 2023-11-24T07:53:36.185 INFO:teuthology.orchestra.run.smithi076.stdout: libaio-devel x86_64 0.3.112-1.el8 rhel-8-for-x86_64-baseos-rpms 19 k 2023-11-24T07:53:36.185 INFO:teuthology.orchestra.run.smithi076.stdout: libcephfs-devel x86_64 2:18.0.0-7484.ga13ea5ff.el8 ceph 31 k 2023-11-24T07:53:36.186 INFO:teuthology.orchestra.run.smithi076.stdout: libcephfs2 x86_64 2:18.0.0-7484.ga13ea5ff.el8 ceph 839 k 2023-11-24T07:53:36.186 INFO:teuthology.orchestra.run.smithi076.stdout: librados-devel x86_64 2:18.0.0-7484.ga13ea5ff.el8 ceph 139 k 2023-11-24T07:53:36.186 INFO:teuthology.orchestra.run.smithi076.stdout: libtool x86_64 2.4.6-25.el8 rhel-8-for-x86_64-appstream-rpms 709 k 2023-11-24T07:53:36.186 INFO:teuthology.orchestra.run.smithi076.stdout: libuuid-devel x86_64 2.32.1-35.el8 rhel-8-for-x86_64-baseos-rpms 98 k 2023-11-24T07:53:36.186 INFO:teuthology.orchestra.run.smithi076.stdout: openssl-devel x86_64 1:1.1.1k-7.el8_6 rhel-8-for-x86_64-baseos-rpms 2.3 M 2023-11-24T07:53:36.186 INFO:teuthology.orchestra.run.smithi076.stdout: postgresql x86_64 10.21-2.module+el8.6.0+15342+53518fac rhel-8-for-x86_64-appstream-rpms 1.5 M 2023-11-24T07:53:36.186 INFO:teuthology.orchestra.run.smithi076.stdout: postgresql-contrib x86_64 10.21-2.module+el8.6.0+15342+53518fac rhel-8-for-x86_64-appstream-rpms 808 k 2023-11-24T07:53:36.186 INFO:teuthology.orchestra.run.smithi076.stdout: postgresql-server x86_64 10.21-2.module+el8.6.0+15342+53518fac rhel-8-for-x86_64-appstream-rpms 5.1 M 2023-11-24T07:53:36.186 INFO:teuthology.orchestra.run.smithi076.stdout: python3-cephfs x86_64 2:18.0.0-7484.ga13ea5ff.el8 ceph 197 k 2023-11-24T07:53:36.186 INFO:teuthology.orchestra.run.smithi076.stdout: python3-rados x86_64 2:18.0.0-7484.ga13ea5ff.el8 ceph 371 k 2023-11-24T07:53:36.186 INFO:teuthology.orchestra.run.smithi076.stdout: python3-rbd x86_64 2:18.0.0-7484.ga13ea5ff.el8 ceph 364 k 2023-11-24T07:53:36.186 INFO:teuthology.orchestra.run.smithi076.stdout: python3-rgw x86_64 2:18.0.0-7484.ga13ea5ff.el8 ceph 107 k 2023-11-24T07:53:36.187 INFO:teuthology.orchestra.run.smithi076.stdout: rbd-fuse x86_64 2:18.0.0-7484.ga13ea5ff.el8 ceph 94 k 2023-11-24T07:53:36.187 INFO:teuthology.orchestra.run.smithi076.stdout: rbd-mirror x86_64 2:18.0.0-7484.ga13ea5ff.el8 ceph 3.6 M 2023-11-24T07:53:36.187 INFO:teuthology.orchestra.run.smithi076.stdout: rbd-nbd x86_64 2:18.0.0-7484.ga13ea5ff.el8 ceph 181 k 2023-11-24T07:53:36.187 INFO:teuthology.orchestra.run.smithi076.stdout: xfsprogs-devel x86_64 5.0.0-10.el8 rhel-8-for-x86_64-baseos-rpms 72 k 2023-11-24T07:53:36.187 INFO:teuthology.orchestra.run.smithi076.stdout:Upgrading: 2023-11-24T07:53:36.187 INFO:teuthology.orchestra.run.smithi076.stdout: iproute x86_64 5.15.0-4.el8_6.1 rhel-8-for-x86_64-baseos-rpms 798 k 2023-11-24T07:53:36.187 INFO:teuthology.orchestra.run.smithi076.stdout: librados2 x86_64 2:18.0.0-7484.ga13ea5ff.el8 ceph 3.8 M 2023-11-24T07:53:36.187 INFO:teuthology.orchestra.run.smithi076.stdout: librbd1 x86_64 2:18.0.0-7484.ga13ea5ff.el8 ceph 3.7 M 2023-11-24T07:53:36.187 INFO:teuthology.orchestra.run.smithi076.stdout:Installing dependencies: 2023-11-24T07:53:36.187 INFO:teuthology.orchestra.run.smithi076.stdout: automake noarch 1.16.1-7.el8 rhel-8-for-x86_64-appstream-rpms 713 k 2023-11-24T07:53:36.188 INFO:teuthology.orchestra.run.smithi076.stdout: ceph-common x86_64 2:18.0.0-7484.ga13ea5ff.el8 ceph 23 M 2023-11-24T07:53:36.188 INFO:teuthology.orchestra.run.smithi076.stdout: ceph-grafana-dashboards noarch 2:18.0.0-7484.ga13ea5ff.el8 ceph-noarch 28 k 2023-11-24T07:53:36.188 INFO:teuthology.orchestra.run.smithi076.stdout: ceph-mds x86_64 2:18.0.0-7484.ga13ea5ff.el8 ceph 2.3 M 2023-11-24T07:53:36.188 INFO:teuthology.orchestra.run.smithi076.stdout: ceph-mgr-modules-core noarch 2:18.0.0-7484.ga13ea5ff.el8 ceph-noarch 243 k 2023-11-24T07:53:36.188 INFO:teuthology.orchestra.run.smithi076.stdout: ceph-mon x86_64 2:18.0.0-7484.ga13ea5ff.el8 ceph 4.6 M 2023-11-24T07:53:36.188 INFO:teuthology.orchestra.run.smithi076.stdout: ceph-osd x86_64 2:18.0.0-7484.ga13ea5ff.el8 ceph 18 M 2023-11-24T07:53:36.188 INFO:teuthology.orchestra.run.smithi076.stdout: ceph-prometheus-alerts noarch 2:18.0.0-7484.ga13ea5ff.el8 ceph-noarch 15 k 2023-11-24T07:53:36.188 INFO:teuthology.orchestra.run.smithi076.stdout: ceph-selinux x86_64 2:18.0.0-7484.ga13ea5ff.el8 ceph 24 k 2023-11-24T07:53:36.188 INFO:teuthology.orchestra.run.smithi076.stdout: fuse x86_64 2.9.7-15.el8 rhel-8-for-x86_64-baseos-rpms 83 k 2023-11-24T07:53:36.188 INFO:teuthology.orchestra.run.smithi076.stdout: gperftools-libs x86_64 1:2.7-9.el8 epel 306 k 2023-11-24T07:53:36.188 INFO:teuthology.orchestra.run.smithi076.stdout: jq x86_64 1.6-3.el8 rhel-8-for-x86_64-appstream-rpms 202 k 2023-11-24T07:53:36.188 INFO:teuthology.orchestra.run.smithi076.stdout: keyutils-libs-devel x86_64 1.5.10-9.el8 rhel-8-for-x86_64-baseos-rpms 48 k 2023-11-24T07:53:36.188 INFO:teuthology.orchestra.run.smithi076.stdout: krb5-devel x86_64 1.18.2-14.el8 rhel-8-for-x86_64-baseos-rpms 560 k 2023-11-24T07:53:36.189 INFO:teuthology.orchestra.run.smithi076.stdout: libcephsqlite x86_64 2:18.0.0-7484.ga13ea5ff.el8 ceph 183 k 2023-11-24T07:53:36.189 INFO:teuthology.orchestra.run.smithi076.stdout: libcom_err-devel x86_64 1.45.6-4.el8 rhel-8-for-x86_64-baseos-rpms 39 k 2023-11-24T07:53:36.189 INFO:teuthology.orchestra.run.smithi076.stdout: libgfortran x86_64 8.5.0-10.1.el8_6 rhel-8-for-x86_64-baseos-rpms 644 k 2023-11-24T07:53:36.189 INFO:teuthology.orchestra.run.smithi076.stdout: liboath x86_64 2.6.2-3.el8 epel 59 k 2023-11-24T07:53:36.189 INFO:teuthology.orchestra.run.smithi076.stdout: libpq x86_64 13.5-1.el8 rhel-8-for-x86_64-appstream-rpms 198 k 2023-11-24T07:53:36.189 INFO:teuthology.orchestra.run.smithi076.stdout: libquadmath x86_64 8.5.0-10.1.el8_6 rhel-8-for-x86_64-baseos-rpms 171 k 2023-11-24T07:53:36.189 INFO:teuthology.orchestra.run.smithi076.stdout: librabbitmq x86_64 0.9.0-3.el8 rhel-8-for-x86_64-baseos-rpms 47 k 2023-11-24T07:53:36.189 INFO:teuthology.orchestra.run.smithi076.stdout: libradosstriper1 x86_64 2:18.0.0-7484.ga13ea5ff.el8 ceph 517 k 2023-11-24T07:53:36.189 INFO:teuthology.orchestra.run.smithi076.stdout: librdkafka x86_64 0.11.4-3.el8 rhel-8-for-x86_64-appstream-rpms 354 k 2023-11-24T07:53:36.189 INFO:teuthology.orchestra.run.smithi076.stdout: librgw2 x86_64 2:18.0.0-7484.ga13ea5ff.el8 ceph 6.9 M 2023-11-24T07:53:36.189 INFO:teuthology.orchestra.run.smithi076.stdout: libselinux-devel x86_64 2.9-5.el8 rhel-8-for-x86_64-baseos-rpms 200 k 2023-11-24T07:53:36.189 INFO:teuthology.orchestra.run.smithi076.stdout: libsepol-devel x86_64 2.9-3.el8 rhel-8-for-x86_64-baseos-rpms 87 k 2023-11-24T07:53:36.190 INFO:teuthology.orchestra.run.smithi076.stdout: libunwind x86_64 1.3.1-3.el8 epel 75 k 2023-11-24T07:53:36.190 INFO:teuthology.orchestra.run.smithi076.stdout: libverto-devel x86_64 0.3.0-5.el8 rhel-8-for-x86_64-baseos-rpms 18 k 2023-11-24T07:53:36.190 INFO:teuthology.orchestra.run.smithi076.stdout: lttng-ust x86_64 2.8.1-11.el8 rhel-8-for-x86_64-appstream-rpms 259 k 2023-11-24T07:53:36.190 INFO:teuthology.orchestra.run.smithi076.stdout: oniguruma x86_64 6.8.2-2.el8 rhel-8-for-x86_64-appstream-rpms 187 k 2023-11-24T07:53:36.190 INFO:teuthology.orchestra.run.smithi076.stdout: openblas x86_64 0.3.15-3.el8 rhel-8-for-x86_64-appstream-rpms 4.7 M 2023-11-24T07:53:36.190 INFO:teuthology.orchestra.run.smithi076.stdout: openblas-threads x86_64 0.3.15-3.el8 rhel-8-for-x86_64-appstream-rpms 4.9 M 2023-11-24T07:53:36.190 INFO:teuthology.orchestra.run.smithi076.stdout: pcre2-devel x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 605 k 2023-11-24T07:53:36.190 INFO:teuthology.orchestra.run.smithi076.stdout: pcre2-utf16 x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 229 k 2023-11-24T07:53:36.190 INFO:teuthology.orchestra.run.smithi076.stdout: pcre2-utf32 x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 220 k 2023-11-24T07:53:36.190 INFO:teuthology.orchestra.run.smithi076.stdout: python3-asyncssh noarch 2.7.0-2.el8 epel 476 k 2023-11-24T07:53:36.190 INFO:teuthology.orchestra.run.smithi076.stdout: python3-babel noarch 2.5.1-7.el8 rhel-8-for-x86_64-appstream-rpms 4.8 M 2023-11-24T07:53:36.190 INFO:teuthology.orchestra.run.smithi076.stdout: python3-bcrypt x86_64 3.1.6-2.el8.1 epel 44 k 2023-11-24T07:53:36.190 INFO:teuthology.orchestra.run.smithi076.stdout: python3-beautifulsoup4 noarch 4.6.3-2.el8.1 epel 185 k 2023-11-24T07:53:36.191 INFO:teuthology.orchestra.run.smithi076.stdout: python3-cachetools noarch 3.1.1-4.el8 epel 33 k 2023-11-24T07:53:36.191 INFO:teuthology.orchestra.run.smithi076.stdout: python3-ceph-argparse x86_64 2:18.0.0-7484.ga13ea5ff.el8 ceph 45 k 2023-11-24T07:53:36.191 INFO:teuthology.orchestra.run.smithi076.stdout: python3-ceph-common x86_64 2:18.0.0-7484.ga13ea5ff.el8 ceph 128 k 2023-11-24T07:53:36.191 INFO:teuthology.orchestra.run.smithi076.stdout: python3-certifi noarch 2018.10.15-7.el8 epel 16 k 2023-11-24T07:53:36.191 INFO:teuthology.orchestra.run.smithi076.stdout: python3-cffi x86_64 1.11.5-5.el8 rhel-8-for-x86_64-baseos-rpms 238 k 2023-11-24T07:53:36.191 INFO:teuthology.orchestra.run.smithi076.stdout: python3-cheroot noarch 8.5.2-1.el8 epel 173 k 2023-11-24T07:53:36.191 INFO:teuthology.orchestra.run.smithi076.stdout: python3-cherrypy noarch 18.4.0-1.el8 epel 384 k 2023-11-24T07:53:36.191 INFO:teuthology.orchestra.run.smithi076.stdout: python3-cryptography x86_64 3.2.1-5.el8 rhel-8-for-x86_64-baseos-rpms 559 k 2023-11-24T07:53:36.191 INFO:teuthology.orchestra.run.smithi076.stdout: python3-google-auth noarch 1:1.1.1-10.el8 epel 81 k 2023-11-24T07:53:36.191 INFO:teuthology.orchestra.run.smithi076.stdout: python3-jaraco noarch 6.2-6.el8 epel 11 k 2023-11-24T07:53:36.191 INFO:teuthology.orchestra.run.smithi076.stdout: python3-jaraco-functools noarch 2.0-4.el8 epel 18 k 2023-11-24T07:53:36.191 INFO:teuthology.orchestra.run.smithi076.stdout: python3-jinja2 noarch 2.10.1-3.el8 rhel-8-for-x86_64-appstream-rpms 538 k 2023-11-24T07:53:36.191 INFO:teuthology.orchestra.run.smithi076.stdout: python3-jsonpatch noarch 1.21-2.el8 rhel-8-for-x86_64-appstream-rpms 27 k 2023-11-24T07:53:36.192 INFO:teuthology.orchestra.run.smithi076.stdout: python3-jsonpointer noarch 1.10-11.el8 rhel-8-for-x86_64-appstream-rpms 20 k 2023-11-24T07:53:36.192 INFO:teuthology.orchestra.run.smithi076.stdout: python3-jwt noarch 1.6.1-2.el8 rhel-8-for-x86_64-baseos-rpms 43 k 2023-11-24T07:53:36.192 INFO:teuthology.orchestra.run.smithi076.stdout: python3-kubernetes noarch 1:11.0.0-6.el8 epel 1.2 M 2023-11-24T07:53:36.192 INFO:teuthology.orchestra.run.smithi076.stdout: python3-logutils noarch 0.3.5-11.el8 epel 49 k 2023-11-24T07:53:36.192 INFO:teuthology.orchestra.run.smithi076.stdout: python3-mako noarch 1.0.6-13.el8 rhel-8-for-x86_64-appstream-rpms 157 k 2023-11-24T07:53:36.192 INFO:teuthology.orchestra.run.smithi076.stdout: python3-markupsafe x86_64 0.23-19.el8 rhel-8-for-x86_64-appstream-rpms 39 k 2023-11-24T07:53:36.192 INFO:teuthology.orchestra.run.smithi076.stdout: python3-more-itertools noarch 7.2.0-3.el8 epel 59 k 2023-11-24T07:53:36.192 INFO:teuthology.orchestra.run.smithi076.stdout: python3-natsort noarch 7.1.1-2.el8 epel 60 k 2023-11-24T07:53:36.192 INFO:teuthology.orchestra.run.smithi076.stdout: python3-numpy x86_64 1:1.14.3-10.el8 rhel-8-for-x86_64-appstream-rpms 3.7 M 2023-11-24T07:53:36.192 INFO:teuthology.orchestra.run.smithi076.stdout: python3-numpy-f2py x86_64 1:1.14.3-10.el8 rhel-8-for-x86_64-appstream-rpms 225 k 2023-11-24T07:53:36.192 INFO:teuthology.orchestra.run.smithi076.stdout: python3-oauthlib noarch 2.1.0-1.el8 rhel-8-for-x86_64-baseos-rpms 155 k 2023-11-24T07:53:36.192 INFO:teuthology.orchestra.run.smithi076.stdout: python3-pecan noarch 1.3.2-9.el8 epel 283 k 2023-11-24T07:53:36.193 INFO:teuthology.orchestra.run.smithi076.stdout: python3-portend noarch 2.6-1.el8 epel 16 k 2023-11-24T07:53:36.193 INFO:teuthology.orchestra.run.smithi076.stdout: python3-prettytable noarch 0.7.2-14.el8 rhel-8-for-x86_64-appstream-rpms 44 k 2023-11-24T07:53:36.193 INFO:teuthology.orchestra.run.smithi076.stdout: python3-pyOpenSSL noarch 19.0.0-1.el8 rhel-8-for-x86_64-appstream-rpms 103 k 2023-11-24T07:53:36.193 INFO:teuthology.orchestra.run.smithi076.stdout: python3-pyasn1 noarch 0.3.7-6.el8 rhel-8-for-x86_64-appstream-rpms 126 k 2023-11-24T07:53:36.193 INFO:teuthology.orchestra.run.smithi076.stdout: python3-pyasn1-modules noarch 0.3.7-6.el8 rhel-8-for-x86_64-appstream-rpms 110 k 2023-11-24T07:53:36.193 INFO:teuthology.orchestra.run.smithi076.stdout: python3-pycparser noarch 2.14-14.el8 rhel-8-for-x86_64-baseos-rpms 109 k 2023-11-24T07:53:36.193 INFO:teuthology.orchestra.run.smithi076.stdout: python3-pytz noarch 2017.2-9.el8 rhel-8-for-x86_64-appstream-rpms 54 k 2023-11-24T07:53:36.193 INFO:teuthology.orchestra.run.smithi076.stdout: python3-repoze-lru noarch 0.7-6.el8 epel 34 k 2023-11-24T07:53:36.193 INFO:teuthology.orchestra.run.smithi076.stdout: python3-requests-oauthlib noarch 1.0.0-1.el8 rhel-8-for-x86_64-baseos-rpms 43 k 2023-11-24T07:53:36.193 INFO:teuthology.orchestra.run.smithi076.stdout: python3-routes noarch 2.4.1-12.el8 epel 196 k 2023-11-24T07:53:36.193 INFO:teuthology.orchestra.run.smithi076.stdout: python3-rsa noarch 4.9-2.el8 epel 64 k 2023-11-24T07:53:36.193 INFO:teuthology.orchestra.run.smithi076.stdout: python3-scipy x86_64 1.0.0-21.module+el8.5.0+10916+41bd434d rhel-8-for-x86_64-appstream-rpms 14 M 2023-11-24T07:53:36.193 INFO:teuthology.orchestra.run.smithi076.stdout: python3-simplegeneric noarch 0.8.1-17.el8 epel 19 k 2023-11-24T07:53:36.194 INFO:teuthology.orchestra.run.smithi076.stdout: python3-singledispatch noarch 3.4.0.3-18.el8 epel 24 k 2023-11-24T07:53:36.194 INFO:teuthology.orchestra.run.smithi076.stdout: python3-tempora noarch 1.14.1-5.el8 epel 29 k 2023-11-24T07:53:36.194 INFO:teuthology.orchestra.run.smithi076.stdout: python3-trustme noarch 0.6.0-4.el8 epel 27 k 2023-11-24T07:53:36.194 INFO:teuthology.orchestra.run.smithi076.stdout: python3-waitress noarch 1.4.3-1.el8 epel 255 k 2023-11-24T07:53:36.194 INFO:teuthology.orchestra.run.smithi076.stdout: python3-webob noarch 1.8.5-1.el8.1 epel 251 k 2023-11-24T07:53:36.194 INFO:teuthology.orchestra.run.smithi076.stdout: python3-websocket-client noarch 0.56.0-5.el8 epel 61 k 2023-11-24T07:53:36.194 INFO:teuthology.orchestra.run.smithi076.stdout: python3-webtest noarch 2.0.33-1.el8 epel 85 k 2023-11-24T07:53:36.194 INFO:teuthology.orchestra.run.smithi076.stdout: python3-werkzeug noarch 0.12.2-4.el8 rhel-8-for-x86_64-appstream-rpms 457 k 2023-11-24T07:53:36.194 INFO:teuthology.orchestra.run.smithi076.stdout: python3-zc-lockfile noarch 2.0-2.el8 epel 23 k 2023-11-24T07:53:36.194 INFO:teuthology.orchestra.run.smithi076.stdout: socat x86_64 1.7.4.1-1.el8 rhel-8-for-x86_64-appstream-rpms 323 k 2023-11-24T07:53:36.194 INFO:teuthology.orchestra.run.smithi076.stdout: thrift x86_64 0.13.0-2.el8 epel 1.7 M 2023-11-24T07:53:36.194 INFO:teuthology.orchestra.run.smithi076.stdout: uuid x86_64 1.6.2-43.el8 rhel-8-for-x86_64-appstream-rpms 64 k 2023-11-24T07:53:36.195 INFO:teuthology.orchestra.run.smithi076.stdout: xmlstarlet x86_64 1.6.1-20.el8 lab-extras 70 k 2023-11-24T07:53:36.195 INFO:teuthology.orchestra.run.smithi076.stdout: zlib-devel x86_64 1.2.11-18.el8_5 rhel-8-for-x86_64-baseos-rpms 58 k 2023-11-24T07:53:36.195 INFO:teuthology.orchestra.run.smithi076.stdout:Enabling module streams: 2023-11-24T07:53:36.195 INFO:teuthology.orchestra.run.smithi076.stdout: postgresql 10 2023-11-24T07:53:36.195 INFO:teuthology.orchestra.run.smithi076.stdout: 2023-11-24T07:53:36.195 INFO:teuthology.orchestra.run.smithi076.stdout:Transaction Summary 2023-11-24T07:53:36.195 INFO:teuthology.orchestra.run.smithi076.stdout:=========================================================================================================================== 2023-11-24T07:53:36.195 INFO:teuthology.orchestra.run.smithi076.stdout:Install 128 Packages 2023-11-24T07:53:36.195 INFO:teuthology.orchestra.run.smithi076.stdout:Upgrade 3 Packages 2023-11-24T07:53:36.195 INFO:teuthology.orchestra.run.smithi076.stdout: 2023-11-24T07:53:36.196 INFO:teuthology.orchestra.run.smithi076.stdout:Total download size: 209 M 2023-11-24T07:53:36.196 INFO:teuthology.orchestra.run.smithi076.stdout:Downloading Packages: 2023-11-24T07:53:36.349 INFO:teuthology.orchestra.run.smithi076.stdout:(1/131): ceph-18.0.0-7484.ga13ea5ff.el8.x86_64. 52 kB/s | 6.4 kB 00:00 2023-11-24T07:53:36.691 INFO:teuthology.orchestra.run.smithi076.stdout:(2/131): ceph-fuse-18.0.0-7484.ga13ea5ff.el8.x8 2.6 MB/s | 920 kB 00:00 2023-11-24T07:53:36.775 INFO:teuthology.orchestra.run.smithi076.stdout:(3/131): ceph-immutable-object-cache-18.0.0-748 2.0 MB/s | 171 kB 00:00 2023-11-24T07:53:36.875 INFO:teuthology.orchestra.run.smithi076.stdout:(4/131): ceph-base-18.0.0-7484.ga13ea5ff.el8.x8 8.3 MB/s | 5.4 MB 00:00 2023-11-24T07:53:37.092 INFO:teuthology.orchestra.run.smithi076.stdout:(5/131): ceph-mgr-18.0.0-7484.ga13ea5ff.el8.x86 7.5 MB/s | 1.6 MB 00:00 2023-11-24T07:53:37.393 INFO:teuthology.orchestra.run.smithi076.stdout:(6/131): ceph-mds-18.0.0-7484.ga13ea5ff.el8.x86 3.7 MB/s | 2.3 MB 00:00 2023-11-24T07:53:37.643 INFO:teuthology.orchestra.run.smithi076.stdout:(7/131): ceph-mon-18.0.0-7484.ga13ea5ff.el8.x86 8.4 MB/s | 4.6 MB 00:00 2023-11-24T07:53:38.010 INFO:teuthology.orchestra.run.smithi076.stdout:(8/131): ceph-common-18.0.0-7484.ga13ea5ff.el8. 13 MB/s | 23 MB 00:01 2023-11-24T07:53:38.061 INFO:teuthology.orchestra.run.smithi076.stdout:(9/131): ceph-selinux-18.0.0-7484.ga13ea5ff.el8 487 kB/s | 24 kB 00:00 2023-11-24T07:53:38.716 INFO:teuthology.orchestra.run.smithi059.stdout:Package librados2-1:12.2.7-9.el8.x86_64 is already installed. 2023-11-24T07:53:38.717 INFO:teuthology.orchestra.run.smithi059.stdout:Package librbd1-1:12.2.7-9.el8.x86_64 is already installed. 2023-11-24T07:53:38.734 INFO:teuthology.orchestra.run.smithi059.stdout:Package NetworkManager-1:1.36.0-4.el8.x86_64 is already installed. 2023-11-24T07:53:38.735 INFO:teuthology.orchestra.run.smithi059.stdout:Package iproute-5.15.0-4.el8.x86_64 is already installed. 2023-11-24T07:53:38.736 INFO:teuthology.orchestra.run.smithi059.stdout:Package util-linux-2.32.1-35.el8.x86_64 is already installed. 2023-11-24T07:53:38.738 INFO:teuthology.orchestra.run.smithi059.stdout:Package libacl-devel-2.2.53-1.el8.x86_64 is already installed. 2023-11-24T07:53:38.740 INFO:teuthology.orchestra.run.smithi059.stdout:Package libattr-devel-2.4.48-3.el8.x86_64 is already installed. 2023-11-24T07:53:38.743 INFO:teuthology.orchestra.run.smithi059.stdout:Package xfsdump-3.1.8-4.el8.x86_64 is already installed. 2023-11-24T07:53:38.745 INFO:teuthology.orchestra.run.smithi059.stdout:Package xfsprogs-5.0.0-10.el8.x86_64 is already installed. 2023-11-24T07:53:38.887 INFO:teuthology.orchestra.run.smithi059.stdout:Dependencies resolved. 2023-11-24T07:53:38.900 INFO:teuthology.orchestra.run.smithi059.stdout:=========================================================================================================================== 2023-11-24T07:53:38.900 INFO:teuthology.orchestra.run.smithi059.stdout: Package Arch Version Repository Size 2023-11-24T07:53:38.901 INFO:teuthology.orchestra.run.smithi059.stdout:=========================================================================================================================== 2023-11-24T07:53:38.901 INFO:teuthology.orchestra.run.smithi059.stdout:Installing: 2023-11-24T07:53:38.901 INFO:teuthology.orchestra.run.smithi059.stdout: NetworkManager-initscripts-updown noarch 1:1.36.0-9.el8_6 rhel-8-for-x86_64-baseos-rpms 138 k 2023-11-24T07:53:38.901 INFO:teuthology.orchestra.run.smithi059.stdout: bison x86_64 3.0.4-10.el8 rhel-8-for-x86_64-appstream-rpms 688 k 2023-11-24T07:53:38.901 INFO:teuthology.orchestra.run.smithi059.stdout: ceph x86_64 2:18.0.0-7484.ga13ea5ff.el8 ceph 6.4 k 2023-11-24T07:53:38.901 INFO:teuthology.orchestra.run.smithi059.stdout: ceph-base x86_64 2:18.0.0-7484.ga13ea5ff.el8 ceph 5.4 M 2023-11-24T07:53:38.901 INFO:teuthology.orchestra.run.smithi059.stdout: ceph-fuse x86_64 2:18.0.0-7484.ga13ea5ff.el8 ceph 920 k 2023-11-24T07:53:38.901 INFO:teuthology.orchestra.run.smithi059.stdout: ceph-immutable-object-cache x86_64 2:18.0.0-7484.ga13ea5ff.el8 ceph 171 k 2023-11-24T07:53:38.901 INFO:teuthology.orchestra.run.smithi059.stdout: ceph-mgr x86_64 2:18.0.0-7484.ga13ea5ff.el8 ceph 1.6 M 2023-11-24T07:53:38.901 INFO:teuthology.orchestra.run.smithi059.stdout: ceph-mgr-cephadm noarch 2:18.0.0-7484.ga13ea5ff.el8 ceph-noarch 133 k 2023-11-24T07:53:38.901 INFO:teuthology.orchestra.run.smithi059.stdout: ceph-mgr-dashboard noarch 2:18.0.0-7484.ga13ea5ff.el8 ceph-noarch 1.8 M 2023-11-24T07:53:38.902 INFO:teuthology.orchestra.run.smithi059.stdout: ceph-mgr-diskprediction-local noarch 2:18.0.0-7484.ga13ea5ff.el8 ceph-noarch 7.4 M 2023-11-24T07:53:38.902 INFO:teuthology.orchestra.run.smithi059.stdout: ceph-mgr-rook noarch 2:18.0.0-7484.ga13ea5ff.el8 ceph-noarch 50 k 2023-11-24T07:53:38.902 INFO:teuthology.orchestra.run.smithi059.stdout: ceph-radosgw x86_64 2:18.0.0-7484.ga13ea5ff.el8 ceph 14 M 2023-11-24T07:53:38.902 INFO:teuthology.orchestra.run.smithi059.stdout: ceph-test x86_64 2:18.0.0-7484.ga13ea5ff.el8 ceph 49 M 2023-11-24T07:53:38.902 INFO:teuthology.orchestra.run.smithi059.stdout: ceph-volume noarch 2:18.0.0-7484.ga13ea5ff.el8 ceph-noarch 262 k 2023-11-24T07:53:38.902 INFO:teuthology.orchestra.run.smithi059.stdout: cephadm noarch 2:18.0.0-7484.ga13ea5ff.el8 ceph-noarch 407 k 2023-11-24T07:53:38.902 INFO:teuthology.orchestra.run.smithi059.stdout: cephfs-mirror x86_64 2:18.0.0-7484.ga13ea5ff.el8 ceph 262 k 2023-11-24T07:53:38.902 INFO:teuthology.orchestra.run.smithi059.stdout: cephfs-top noarch 2:18.0.0-7484.ga13ea5ff.el8 ceph-noarch 18 k 2023-11-24T07:53:38.902 INFO:teuthology.orchestra.run.smithi059.stdout: elfutils-libelf-devel x86_64 0.186-1.el8 rhel-8-for-x86_64-baseos-rpms 60 k 2023-11-24T07:53:38.902 INFO:teuthology.orchestra.run.smithi059.stdout: flex x86_64 2.6.1-9.el8 rhel-8-for-x86_64-appstream-rpms 320 k 2023-11-24T07:53:38.902 INFO:teuthology.orchestra.run.smithi059.stdout: libaio-devel x86_64 0.3.112-1.el8 rhel-8-for-x86_64-baseos-rpms 19 k 2023-11-24T07:53:38.903 INFO:teuthology.orchestra.run.smithi059.stdout: libcephfs-devel x86_64 2:18.0.0-7484.ga13ea5ff.el8 ceph 31 k 2023-11-24T07:53:38.903 INFO:teuthology.orchestra.run.smithi059.stdout: libcephfs2 x86_64 2:18.0.0-7484.ga13ea5ff.el8 ceph 839 k 2023-11-24T07:53:38.903 INFO:teuthology.orchestra.run.smithi059.stdout: librados-devel x86_64 2:18.0.0-7484.ga13ea5ff.el8 ceph 139 k 2023-11-24T07:53:38.903 INFO:teuthology.orchestra.run.smithi059.stdout: libtool x86_64 2.4.6-25.el8 rhel-8-for-x86_64-appstream-rpms 709 k 2023-11-24T07:53:38.903 INFO:teuthology.orchestra.run.smithi059.stdout: libuuid-devel x86_64 2.32.1-35.el8 rhel-8-for-x86_64-baseos-rpms 98 k 2023-11-24T07:53:38.903 INFO:teuthology.orchestra.run.smithi059.stdout: openssl-devel x86_64 1:1.1.1k-7.el8_6 rhel-8-for-x86_64-baseos-rpms 2.3 M 2023-11-24T07:53:38.903 INFO:teuthology.orchestra.run.smithi059.stdout: postgresql x86_64 10.21-2.module+el8.6.0+15342+53518fac rhel-8-for-x86_64-appstream-rpms 1.5 M 2023-11-24T07:53:38.903 INFO:teuthology.orchestra.run.smithi059.stdout: postgresql-contrib x86_64 10.21-2.module+el8.6.0+15342+53518fac rhel-8-for-x86_64-appstream-rpms 808 k 2023-11-24T07:53:38.903 INFO:teuthology.orchestra.run.smithi059.stdout: postgresql-server x86_64 10.21-2.module+el8.6.0+15342+53518fac rhel-8-for-x86_64-appstream-rpms 5.1 M 2023-11-24T07:53:38.903 INFO:teuthology.orchestra.run.smithi059.stdout: python3-cephfs x86_64 2:18.0.0-7484.ga13ea5ff.el8 ceph 197 k 2023-11-24T07:53:38.903 INFO:teuthology.orchestra.run.smithi059.stdout: python3-rados x86_64 2:18.0.0-7484.ga13ea5ff.el8 ceph 371 k 2023-11-24T07:53:38.903 INFO:teuthology.orchestra.run.smithi059.stdout: python3-rbd x86_64 2:18.0.0-7484.ga13ea5ff.el8 ceph 364 k 2023-11-24T07:53:38.904 INFO:teuthology.orchestra.run.smithi059.stdout: python3-rgw x86_64 2:18.0.0-7484.ga13ea5ff.el8 ceph 107 k 2023-11-24T07:53:38.904 INFO:teuthology.orchestra.run.smithi059.stdout: rbd-fuse x86_64 2:18.0.0-7484.ga13ea5ff.el8 ceph 94 k 2023-11-24T07:53:38.904 INFO:teuthology.orchestra.run.smithi059.stdout: rbd-mirror x86_64 2:18.0.0-7484.ga13ea5ff.el8 ceph 3.6 M 2023-11-24T07:53:38.904 INFO:teuthology.orchestra.run.smithi059.stdout: rbd-nbd x86_64 2:18.0.0-7484.ga13ea5ff.el8 ceph 181 k 2023-11-24T07:53:38.904 INFO:teuthology.orchestra.run.smithi059.stdout: xfsprogs-devel x86_64 5.0.0-10.el8 rhel-8-for-x86_64-baseos-rpms 72 k 2023-11-24T07:53:38.904 INFO:teuthology.orchestra.run.smithi059.stdout:Upgrading: 2023-11-24T07:53:38.904 INFO:teuthology.orchestra.run.smithi059.stdout: iproute x86_64 5.15.0-4.el8_6.1 rhel-8-for-x86_64-baseos-rpms 798 k 2023-11-24T07:53:38.904 INFO:teuthology.orchestra.run.smithi059.stdout: librados2 x86_64 2:18.0.0-7484.ga13ea5ff.el8 ceph 3.8 M 2023-11-24T07:53:38.904 INFO:teuthology.orchestra.run.smithi059.stdout: librbd1 x86_64 2:18.0.0-7484.ga13ea5ff.el8 ceph 3.7 M 2023-11-24T07:53:38.904 INFO:teuthology.orchestra.run.smithi059.stdout:Installing dependencies: 2023-11-24T07:53:38.905 INFO:teuthology.orchestra.run.smithi059.stdout: automake noarch 1.16.1-7.el8 rhel-8-for-x86_64-appstream-rpms 713 k 2023-11-24T07:53:38.905 INFO:teuthology.orchestra.run.smithi059.stdout: ceph-common x86_64 2:18.0.0-7484.ga13ea5ff.el8 ceph 23 M 2023-11-24T07:53:38.906 INFO:teuthology.orchestra.run.smithi059.stdout: ceph-grafana-dashboards noarch 2:18.0.0-7484.ga13ea5ff.el8 ceph-noarch 28 k 2023-11-24T07:53:38.906 INFO:teuthology.orchestra.run.smithi059.stdout: ceph-mds x86_64 2:18.0.0-7484.ga13ea5ff.el8 ceph 2.3 M 2023-11-24T07:53:38.906 INFO:teuthology.orchestra.run.smithi059.stdout: ceph-mgr-modules-core noarch 2:18.0.0-7484.ga13ea5ff.el8 ceph-noarch 243 k 2023-11-24T07:53:38.906 INFO:teuthology.orchestra.run.smithi059.stdout: ceph-mon x86_64 2:18.0.0-7484.ga13ea5ff.el8 ceph 4.6 M 2023-11-24T07:53:38.906 INFO:teuthology.orchestra.run.smithi059.stdout: ceph-osd x86_64 2:18.0.0-7484.ga13ea5ff.el8 ceph 18 M 2023-11-24T07:53:38.906 INFO:teuthology.orchestra.run.smithi059.stdout: ceph-prometheus-alerts noarch 2:18.0.0-7484.ga13ea5ff.el8 ceph-noarch 15 k 2023-11-24T07:53:38.906 INFO:teuthology.orchestra.run.smithi059.stdout: ceph-selinux x86_64 2:18.0.0-7484.ga13ea5ff.el8 ceph 24 k 2023-11-24T07:53:38.906 INFO:teuthology.orchestra.run.smithi059.stdout: fuse x86_64 2.9.7-15.el8 rhel-8-for-x86_64-baseos-rpms 83 k 2023-11-24T07:53:38.906 INFO:teuthology.orchestra.run.smithi059.stdout: gperftools-libs x86_64 1:2.7-9.el8 epel 306 k 2023-11-24T07:53:38.907 INFO:teuthology.orchestra.run.smithi059.stdout: jq x86_64 1.6-3.el8 rhel-8-for-x86_64-appstream-rpms 202 k 2023-11-24T07:53:38.907 INFO:teuthology.orchestra.run.smithi059.stdout: keyutils-libs-devel x86_64 1.5.10-9.el8 rhel-8-for-x86_64-baseos-rpms 48 k 2023-11-24T07:53:38.907 INFO:teuthology.orchestra.run.smithi059.stdout: krb5-devel x86_64 1.18.2-14.el8 rhel-8-for-x86_64-baseos-rpms 560 k 2023-11-24T07:53:38.907 INFO:teuthology.orchestra.run.smithi059.stdout: libcephsqlite x86_64 2:18.0.0-7484.ga13ea5ff.el8 ceph 183 k 2023-11-24T07:53:38.907 INFO:teuthology.orchestra.run.smithi059.stdout: libcom_err-devel x86_64 1.45.6-4.el8 rhel-8-for-x86_64-baseos-rpms 39 k 2023-11-24T07:53:38.907 INFO:teuthology.orchestra.run.smithi059.stdout: libgfortran x86_64 8.5.0-10.1.el8_6 rhel-8-for-x86_64-baseos-rpms 644 k 2023-11-24T07:53:38.907 INFO:teuthology.orchestra.run.smithi059.stdout: liboath x86_64 2.6.2-3.el8 epel 59 k 2023-11-24T07:53:38.907 INFO:teuthology.orchestra.run.smithi059.stdout: libpq x86_64 13.5-1.el8 rhel-8-for-x86_64-appstream-rpms 198 k 2023-11-24T07:53:38.907 INFO:teuthology.orchestra.run.smithi059.stdout: libquadmath x86_64 8.5.0-10.1.el8_6 rhel-8-for-x86_64-baseos-rpms 171 k 2023-11-24T07:53:38.907 INFO:teuthology.orchestra.run.smithi059.stdout: librabbitmq x86_64 0.9.0-3.el8 rhel-8-for-x86_64-baseos-rpms 47 k 2023-11-24T07:53:38.907 INFO:teuthology.orchestra.run.smithi059.stdout: libradosstriper1 x86_64 2:18.0.0-7484.ga13ea5ff.el8 ceph 517 k 2023-11-24T07:53:38.907 INFO:teuthology.orchestra.run.smithi059.stdout: librdkafka x86_64 0.11.4-3.el8 rhel-8-for-x86_64-appstream-rpms 354 k 2023-11-24T07:53:38.908 INFO:teuthology.orchestra.run.smithi059.stdout: librgw2 x86_64 2:18.0.0-7484.ga13ea5ff.el8 ceph 6.9 M 2023-11-24T07:53:38.908 INFO:teuthology.orchestra.run.smithi059.stdout: libselinux-devel x86_64 2.9-5.el8 rhel-8-for-x86_64-baseos-rpms 200 k 2023-11-24T07:53:38.908 INFO:teuthology.orchestra.run.smithi059.stdout: libsepol-devel x86_64 2.9-3.el8 rhel-8-for-x86_64-baseos-rpms 87 k 2023-11-24T07:53:38.908 INFO:teuthology.orchestra.run.smithi059.stdout: libunwind x86_64 1.3.1-3.el8 epel 75 k 2023-11-24T07:53:38.908 INFO:teuthology.orchestra.run.smithi059.stdout: libverto-devel x86_64 0.3.0-5.el8 rhel-8-for-x86_64-baseos-rpms 18 k 2023-11-24T07:53:38.908 INFO:teuthology.orchestra.run.smithi059.stdout: lttng-ust x86_64 2.8.1-11.el8 rhel-8-for-x86_64-appstream-rpms 259 k 2023-11-24T07:53:38.908 INFO:teuthology.orchestra.run.smithi059.stdout: oniguruma x86_64 6.8.2-2.el8 rhel-8-for-x86_64-appstream-rpms 187 k 2023-11-24T07:53:38.908 INFO:teuthology.orchestra.run.smithi059.stdout: openblas x86_64 0.3.15-3.el8 rhel-8-for-x86_64-appstream-rpms 4.7 M 2023-11-24T07:53:38.908 INFO:teuthology.orchestra.run.smithi059.stdout: openblas-threads x86_64 0.3.15-3.el8 rhel-8-for-x86_64-appstream-rpms 4.9 M 2023-11-24T07:53:38.908 INFO:teuthology.orchestra.run.smithi059.stdout: pcre2-devel x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 605 k 2023-11-24T07:53:38.908 INFO:teuthology.orchestra.run.smithi059.stdout: pcre2-utf16 x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 229 k 2023-11-24T07:53:38.908 INFO:teuthology.orchestra.run.smithi059.stdout: pcre2-utf32 x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 220 k 2023-11-24T07:53:38.909 INFO:teuthology.orchestra.run.smithi059.stdout: python3-asyncssh noarch 2.7.0-2.el8 epel 476 k 2023-11-24T07:53:38.909 INFO:teuthology.orchestra.run.smithi059.stdout: python3-babel noarch 2.5.1-7.el8 rhel-8-for-x86_64-appstream-rpms 4.8 M 2023-11-24T07:53:38.909 INFO:teuthology.orchestra.run.smithi059.stdout: python3-bcrypt x86_64 3.1.6-2.el8.1 epel 44 k 2023-11-24T07:53:38.909 INFO:teuthology.orchestra.run.smithi059.stdout: python3-beautifulsoup4 noarch 4.6.3-2.el8.1 epel 185 k 2023-11-24T07:53:38.909 INFO:teuthology.orchestra.run.smithi059.stdout: python3-cachetools noarch 3.1.1-4.el8 epel 33 k 2023-11-24T07:53:38.909 INFO:teuthology.orchestra.run.smithi059.stdout: python3-ceph-argparse x86_64 2:18.0.0-7484.ga13ea5ff.el8 ceph 45 k 2023-11-24T07:53:38.909 INFO:teuthology.orchestra.run.smithi059.stdout: python3-ceph-common x86_64 2:18.0.0-7484.ga13ea5ff.el8 ceph 128 k 2023-11-24T07:53:38.909 INFO:teuthology.orchestra.run.smithi059.stdout: python3-certifi noarch 2018.10.15-7.el8 epel 16 k 2023-11-24T07:53:38.909 INFO:teuthology.orchestra.run.smithi059.stdout: python3-cffi x86_64 1.11.5-5.el8 rhel-8-for-x86_64-baseos-rpms 238 k 2023-11-24T07:53:38.909 INFO:teuthology.orchestra.run.smithi059.stdout: python3-cheroot noarch 8.5.2-1.el8 epel 173 k 2023-11-24T07:53:38.909 INFO:teuthology.orchestra.run.smithi059.stdout: python3-cherrypy noarch 18.4.0-1.el8 epel 384 k 2023-11-24T07:53:38.909 INFO:teuthology.orchestra.run.smithi059.stdout: python3-cryptography x86_64 3.2.1-5.el8 rhel-8-for-x86_64-baseos-rpms 559 k 2023-11-24T07:53:38.909 INFO:teuthology.orchestra.run.smithi059.stdout: python3-google-auth noarch 1:1.1.1-10.el8 epel 81 k 2023-11-24T07:53:38.910 INFO:teuthology.orchestra.run.smithi059.stdout: python3-jaraco noarch 6.2-6.el8 epel 11 k 2023-11-24T07:53:38.910 INFO:teuthology.orchestra.run.smithi059.stdout: python3-jaraco-functools noarch 2.0-4.el8 epel 18 k 2023-11-24T07:53:38.910 INFO:teuthology.orchestra.run.smithi059.stdout: python3-jinja2 noarch 2.10.1-3.el8 rhel-8-for-x86_64-appstream-rpms 538 k 2023-11-24T07:53:38.910 INFO:teuthology.orchestra.run.smithi059.stdout: python3-jsonpatch noarch 1.21-2.el8 rhel-8-for-x86_64-appstream-rpms 27 k 2023-11-24T07:53:38.910 INFO:teuthology.orchestra.run.smithi059.stdout: python3-jsonpointer noarch 1.10-11.el8 rhel-8-for-x86_64-appstream-rpms 20 k 2023-11-24T07:53:38.910 INFO:teuthology.orchestra.run.smithi059.stdout: python3-jwt noarch 1.6.1-2.el8 rhel-8-for-x86_64-baseos-rpms 43 k 2023-11-24T07:53:38.910 INFO:teuthology.orchestra.run.smithi059.stdout: python3-kubernetes noarch 1:11.0.0-6.el8 epel 1.2 M 2023-11-24T07:53:38.910 INFO:teuthology.orchestra.run.smithi059.stdout: python3-logutils noarch 0.3.5-11.el8 epel 49 k 2023-11-24T07:53:38.910 INFO:teuthology.orchestra.run.smithi059.stdout: python3-mako noarch 1.0.6-13.el8 rhel-8-for-x86_64-appstream-rpms 157 k 2023-11-24T07:53:38.910 INFO:teuthology.orchestra.run.smithi059.stdout: python3-markupsafe x86_64 0.23-19.el8 rhel-8-for-x86_64-appstream-rpms 39 k 2023-11-24T07:53:38.910 INFO:teuthology.orchestra.run.smithi059.stdout: python3-more-itertools noarch 7.2.0-3.el8 epel 59 k 2023-11-24T07:53:38.911 INFO:teuthology.orchestra.run.smithi059.stdout: python3-natsort noarch 7.1.1-2.el8 epel 60 k 2023-11-24T07:53:38.911 INFO:teuthology.orchestra.run.smithi059.stdout: python3-numpy x86_64 1:1.14.3-10.el8 rhel-8-for-x86_64-appstream-rpms 3.7 M 2023-11-24T07:53:38.911 INFO:teuthology.orchestra.run.smithi059.stdout: python3-numpy-f2py x86_64 1:1.14.3-10.el8 rhel-8-for-x86_64-appstream-rpms 225 k 2023-11-24T07:53:38.911 INFO:teuthology.orchestra.run.smithi059.stdout: python3-oauthlib noarch 2.1.0-1.el8 rhel-8-for-x86_64-baseos-rpms 155 k 2023-11-24T07:53:38.911 INFO:teuthology.orchestra.run.smithi059.stdout: python3-pecan noarch 1.3.2-9.el8 epel 283 k 2023-11-24T07:53:38.911 INFO:teuthology.orchestra.run.smithi059.stdout: python3-portend noarch 2.6-1.el8 epel 16 k 2023-11-24T07:53:38.911 INFO:teuthology.orchestra.run.smithi059.stdout: python3-prettytable noarch 0.7.2-14.el8 rhel-8-for-x86_64-appstream-rpms 44 k 2023-11-24T07:53:38.911 INFO:teuthology.orchestra.run.smithi059.stdout: python3-pyOpenSSL noarch 19.0.0-1.el8 rhel-8-for-x86_64-appstream-rpms 103 k 2023-11-24T07:53:38.911 INFO:teuthology.orchestra.run.smithi059.stdout: python3-pyasn1 noarch 0.3.7-6.el8 rhel-8-for-x86_64-appstream-rpms 126 k 2023-11-24T07:53:38.911 INFO:teuthology.orchestra.run.smithi059.stdout: python3-pyasn1-modules noarch 0.3.7-6.el8 rhel-8-for-x86_64-appstream-rpms 110 k 2023-11-24T07:53:38.911 INFO:teuthology.orchestra.run.smithi059.stdout: python3-pycparser noarch 2.14-14.el8 rhel-8-for-x86_64-baseos-rpms 109 k 2023-11-24T07:53:38.911 INFO:teuthology.orchestra.run.smithi059.stdout: python3-pytz noarch 2017.2-9.el8 rhel-8-for-x86_64-appstream-rpms 54 k 2023-11-24T07:53:38.911 INFO:teuthology.orchestra.run.smithi059.stdout: python3-repoze-lru noarch 0.7-6.el8 epel 34 k 2023-11-24T07:53:38.912 INFO:teuthology.orchestra.run.smithi059.stdout: python3-requests-oauthlib noarch 1.0.0-1.el8 rhel-8-for-x86_64-baseos-rpms 43 k 2023-11-24T07:53:38.912 INFO:teuthology.orchestra.run.smithi059.stdout: python3-routes noarch 2.4.1-12.el8 epel 196 k 2023-11-24T07:53:38.912 INFO:teuthology.orchestra.run.smithi059.stdout: python3-rsa noarch 4.9-2.el8 epel 64 k 2023-11-24T07:53:38.912 INFO:teuthology.orchestra.run.smithi059.stdout: python3-scipy x86_64 1.0.0-21.module+el8.5.0+10916+41bd434d rhel-8-for-x86_64-appstream-rpms 14 M 2023-11-24T07:53:38.912 INFO:teuthology.orchestra.run.smithi059.stdout: python3-simplegeneric noarch 0.8.1-17.el8 epel 19 k 2023-11-24T07:53:38.912 INFO:teuthology.orchestra.run.smithi059.stdout: python3-singledispatch noarch 3.4.0.3-18.el8 epel 24 k 2023-11-24T07:53:38.912 INFO:teuthology.orchestra.run.smithi059.stdout: python3-tempora noarch 1.14.1-5.el8 epel 29 k 2023-11-24T07:53:38.912 INFO:teuthology.orchestra.run.smithi059.stdout: python3-trustme noarch 0.6.0-4.el8 epel 27 k 2023-11-24T07:53:38.912 INFO:teuthology.orchestra.run.smithi059.stdout: python3-waitress noarch 1.4.3-1.el8 epel 255 k 2023-11-24T07:53:38.912 INFO:teuthology.orchestra.run.smithi059.stdout: python3-webob noarch 1.8.5-1.el8.1 epel 251 k 2023-11-24T07:53:38.912 INFO:teuthology.orchestra.run.smithi059.stdout: python3-websocket-client noarch 0.56.0-5.el8 epel 61 k 2023-11-24T07:53:38.912 INFO:teuthology.orchestra.run.smithi059.stdout: python3-webtest noarch 2.0.33-1.el8 epel 85 k 2023-11-24T07:53:38.913 INFO:teuthology.orchestra.run.smithi059.stdout: python3-werkzeug noarch 0.12.2-4.el8 rhel-8-for-x86_64-appstream-rpms 457 k 2023-11-24T07:53:38.913 INFO:teuthology.orchestra.run.smithi059.stdout: python3-zc-lockfile noarch 2.0-2.el8 epel 23 k 2023-11-24T07:53:38.913 INFO:teuthology.orchestra.run.smithi059.stdout: socat x86_64 1.7.4.1-1.el8 rhel-8-for-x86_64-appstream-rpms 323 k 2023-11-24T07:53:38.913 INFO:teuthology.orchestra.run.smithi059.stdout: thrift x86_64 0.13.0-2.el8 epel 1.7 M 2023-11-24T07:53:38.913 INFO:teuthology.orchestra.run.smithi059.stdout: uuid x86_64 1.6.2-43.el8 rhel-8-for-x86_64-appstream-rpms 64 k 2023-11-24T07:53:38.913 INFO:teuthology.orchestra.run.smithi059.stdout: xmlstarlet x86_64 1.6.1-20.el8 lab-extras 70 k 2023-11-24T07:53:38.913 INFO:teuthology.orchestra.run.smithi059.stdout: zlib-devel x86_64 1.2.11-18.el8_5 rhel-8-for-x86_64-baseos-rpms 58 k 2023-11-24T07:53:38.913 INFO:teuthology.orchestra.run.smithi059.stdout:Enabling module streams: 2023-11-24T07:53:38.913 INFO:teuthology.orchestra.run.smithi059.stdout: postgresql 10 2023-11-24T07:53:38.913 INFO:teuthology.orchestra.run.smithi059.stdout: 2023-11-24T07:53:38.913 INFO:teuthology.orchestra.run.smithi059.stdout:Transaction Summary 2023-11-24T07:53:38.913 INFO:teuthology.orchestra.run.smithi059.stdout:=========================================================================================================================== 2023-11-24T07:53:38.914 INFO:teuthology.orchestra.run.smithi059.stdout:Install 128 Packages 2023-11-24T07:53:38.914 INFO:teuthology.orchestra.run.smithi059.stdout:Upgrade 3 Packages 2023-11-24T07:53:38.914 INFO:teuthology.orchestra.run.smithi059.stdout: 2023-11-24T07:53:38.914 INFO:teuthology.orchestra.run.smithi059.stdout:Total download size: 209 M 2023-11-24T07:53:38.914 INFO:teuthology.orchestra.run.smithi059.stdout:Downloading Packages: 2023-11-24T07:53:39.208 INFO:teuthology.orchestra.run.smithi059.stdout:(1/131): ceph-18.0.0-7484.ga13ea5ff.el8.x86_64. 25 kB/s | 6.4 kB 00:00 2023-11-24T07:53:40.308 INFO:teuthology.orchestra.run.smithi059.stdout:(2/131): ceph-fuse-18.0.0-7484.ga13ea5ff.el8.x8 836 kB/s | 920 kB 00:01 2023-11-24T07:53:40.394 INFO:teuthology.orchestra.run.smithi076.stdout:(10/131): ceph-radosgw-18.0.0-7484.ga13ea5ff.el 5.1 MB/s | 14 MB 00:02 2023-11-24T07:53:40.400 INFO:teuthology.orchestra.run.smithi059.stdout:(3/131): ceph-immutable-object-cache-18.0.0-748 1.8 MB/s | 171 kB 00:00 2023-11-24T07:53:40.469 INFO:teuthology.orchestra.run.smithi076.stdout:(11/131): cephfs-mirror-18.0.0-7484.ga13ea5ff.e 3.4 MB/s | 262 kB 00:00 2023-11-24T07:53:40.553 INFO:teuthology.orchestra.run.smithi076.stdout:(12/131): libcephfs-devel-18.0.0-7484.ga13ea5ff 372 kB/s | 31 kB 00:00 2023-11-24T07:53:40.728 INFO:teuthology.orchestra.run.smithi076.stdout:(13/131): libcephfs2-18.0.0-7484.ga13ea5ff.el8. 4.7 MB/s | 839 kB 00:00 2023-11-24T07:53:40.837 INFO:teuthology.orchestra.run.smithi076.stdout:(14/131): libcephsqlite-18.0.0-7484.ga13ea5ff.e 1.7 MB/s | 183 kB 00:00 2023-11-24T07:53:40.928 INFO:teuthology.orchestra.run.smithi076.stdout:(15/131): librados-devel-18.0.0-7484.ga13ea5ff. 1.5 MB/s | 139 kB 00:00 2023-11-24T07:53:41.045 INFO:teuthology.orchestra.run.smithi076.stdout:(16/131): libradosstriper1-18.0.0-7484.ga13ea5f 4.3 MB/s | 517 kB 00:00 2023-11-24T07:53:41.417 INFO:teuthology.orchestra.run.smithi059.stdout:(4/131): ceph-mds-18.0.0-7484.ga13ea5ff.el8.x86 2.3 MB/s | 2.3 MB 00:01 2023-11-24T07:53:41.437 INFO:teuthology.orchestra.run.smithi076.stdout:(17/131): ceph-osd-18.0.0-7484.ga13ea5ff.el8.x8 4.5 MB/s | 18 MB 00:04 2023-11-24T07:53:41.684 INFO:teuthology.orchestra.run.smithi059.stdout:(5/131): ceph-mgr-18.0.0-7484.ga13ea5ff.el8.x86 6.1 MB/s | 1.6 MB 00:00 2023-11-24T07:53:41.905 INFO:teuthology.orchestra.run.smithi076.stdout:(18/131): ceph-test-18.0.0-7484.ga13ea5ff.el8.x 13 MB/s | 49 MB 00:03 2023-11-24T07:53:41.930 INFO:teuthology.orchestra.run.smithi076.stdout:(19/131): python3-ceph-argparse-18.0.0-7484.ga1 91 kB/s | 45 kB 00:00 2023-11-24T07:53:41.955 INFO:teuthology.orchestra.run.smithi076.stdout:(20/131): python3-ceph-common-18.0.0-7484.ga13e 2.5 MB/s | 128 kB 00:00 2023-11-24T07:53:41.997 INFO:teuthology.orchestra.run.smithi076.stdout:(21/131): python3-cephfs-18.0.0-7484.ga13ea5ff. 2.9 MB/s | 197 kB 00:00 2023-11-24T07:53:42.018 INFO:teuthology.orchestra.run.smithi059.stdout:(6/131): ceph-base-18.0.0-7484.ga13ea5ff.el8.x8 1.7 MB/s | 5.4 MB 00:03 2023-11-24T07:53:42.081 INFO:teuthology.orchestra.run.smithi076.stdout:(22/131): librgw2-18.0.0-7484.ga13ea5ff.el8.x86 6.7 MB/s | 6.9 MB 00:01 2023-11-24T07:53:42.106 INFO:teuthology.orchestra.run.smithi076.stdout:(23/131): python3-rados-18.0.0-7484.ga13ea5ff.e 2.4 MB/s | 371 kB 00:00 2023-11-24T07:53:42.132 INFO:teuthology.orchestra.run.smithi076.stdout:(24/131): python3-rbd-18.0.0-7484.ga13ea5ff.el8 2.7 MB/s | 364 kB 00:00 2023-11-24T07:53:42.149 INFO:teuthology.orchestra.run.smithi076.stdout:(25/131): python3-rgw-18.0.0-7484.ga13ea5ff.el8 1.6 MB/s | 107 kB 00:00 2023-11-24T07:53:42.174 INFO:teuthology.orchestra.run.smithi076.stdout:(26/131): rbd-fuse-18.0.0-7484.ga13ea5ff.el8.x8 1.4 MB/s | 94 kB 00:00 2023-11-24T07:53:42.199 INFO:teuthology.orchestra.run.smithi076.stdout:(27/131): rbd-nbd-18.0.0-7484.ga13ea5ff.el8.x86 3.5 MB/s | 181 kB 00:00 2023-11-24T07:53:42.211 INFO:teuthology.orchestra.run.smithi059.stdout:(7/131): ceph-mon-18.0.0-7484.ga13ea5ff.el8.x86 8.8 MB/s | 4.6 MB 00:00 2023-11-24T07:53:42.225 INFO:teuthology.orchestra.run.smithi076.stdout:(28/131): ceph-grafana-dashboards-18.0.0-7484.g 556 kB/s | 28 kB 00:00 2023-11-24T07:53:42.250 INFO:teuthology.orchestra.run.smithi076.stdout:(29/131): ceph-mgr-cephadm-18.0.0-7484.ga13ea5f 2.6 MB/s | 133 kB 00:00 2023-11-24T07:53:42.617 INFO:teuthology.orchestra.run.smithi076.stdout:(30/131): ceph-mgr-dashboard-18.0.0-7484.ga13ea 4.5 MB/s | 1.8 MB 00:00 2023-11-24T07:53:42.709 INFO:teuthology.orchestra.run.smithi076.stdout:(31/131): ceph-mgr-modules-core-18.0.0-7484.ga1 2.6 MB/s | 243 kB 00:00 2023-11-24T07:53:42.801 INFO:teuthology.orchestra.run.smithi076.stdout:(32/131): ceph-mgr-rook-18.0.0-7484.ga13ea5ff.e 547 kB/s | 50 kB 00:00 2023-11-24T07:53:42.960 INFO:teuthology.orchestra.run.smithi076.stdout:(33/131): ceph-prometheus-alerts-18.0.0-7484.ga 94 kB/s | 15 kB 00:00 2023-11-24T07:53:43.035 INFO:teuthology.orchestra.run.smithi076.stdout:(34/131): rbd-mirror-18.0.0-7484.ga13ea5ff.el8. 4.0 MB/s | 3.6 MB 00:00 2023-11-24T07:53:43.061 INFO:teuthology.orchestra.run.smithi076.stdout:(35/131): ceph-volume-18.0.0-7484.ga13ea5ff.el8 2.5 MB/s | 262 kB 00:00 2023-11-24T07:53:43.153 INFO:teuthology.orchestra.run.smithi076.stdout:(36/131): cephfs-top-18.0.0-7484.ga13ea5ff.el8. 197 kB/s | 18 kB 00:00 2023-11-24T07:53:43.195 INFO:teuthology.orchestra.run.smithi076.stdout:(37/131): cephadm-18.0.0-7484.ga13ea5ff.el8.noa 2.5 MB/s | 407 kB 00:00 2023-11-24T07:53:43.436 INFO:teuthology.orchestra.run.smithi076.stdout:(38/131): ceph-mgr-diskprediction-local-18.0.0- 6.2 MB/s | 7.4 MB 00:01 2023-11-24T07:53:43.502 INFO:teuthology.orchestra.run.smithi059.stdout:(8/131): ceph-radosgw-18.0.0-7484.ga13ea5ff.el8 11 MB/s | 14 MB 00:01 2023-11-24T07:53:43.527 INFO:teuthology.orchestra.run.smithi076.stdout:(39/131): liboath-2.6.2-3.el8.x86_64.rpm 177 kB/s | 59 kB 00:00 2023-11-24T07:53:43.552 INFO:teuthology.orchestra.run.smithi059.stdout:(9/131): ceph-selinux-18.0.0-7484.ga13ea5ff.el8 488 kB/s | 24 kB 00:00 2023-11-24T07:53:43.629 INFO:teuthology.orchestra.run.smithi076.stdout:(40/131): gperftools-libs-2.7-9.el8.x86_64.rpm 643 kB/s | 306 kB 00:00 2023-11-24T07:53:43.721 INFO:teuthology.orchestra.run.smithi076.stdout:(41/131): python3-bcrypt-3.1.6-2.el8.1.x86_64.r 475 kB/s | 44 kB 00:00 2023-11-24T07:53:43.763 INFO:teuthology.orchestra.run.smithi076.stdout:(42/131): libunwind-1.3.1-3.el8.x86_64.rpm 230 kB/s | 75 kB 00:00 2023-11-24T07:53:43.797 INFO:teuthology.orchestra.run.smithi076.stdout:(43/131): python3-asyncssh-2.7.0-2.el8.noarch.r 1.7 MB/s | 476 kB 00:00 2023-11-24T07:53:43.822 INFO:teuthology.orchestra.run.smithi076.stdout:(44/131): python3-beautifulsoup4-4.6.3-2.el8.1. 1.8 MB/s | 185 kB 00:00 2023-11-24T07:53:43.855 INFO:teuthology.orchestra.run.smithi076.stdout:(45/131): python3-cachetools-3.1.1-4.el8.noarch 358 kB/s | 33 kB 00:00 2023-11-24T07:53:43.889 INFO:teuthology.orchestra.run.smithi076.stdout:(46/131): python3-certifi-2018.10.15-7.el8.noar 176 kB/s | 16 kB 00:00 2023-11-24T07:53:43.915 INFO:teuthology.orchestra.run.smithi076.stdout:(47/131): python3-cheroot-8.5.2-1.el8.noarch.rp 1.8 MB/s | 173 kB 00:00 2023-11-24T07:53:43.973 INFO:teuthology.orchestra.run.smithi076.stdout:(48/131): python3-google-auth-1.1.1-10.el8.noar 972 kB/s | 81 kB 00:00 2023-11-24T07:53:44.007 INFO:teuthology.orchestra.run.smithi076.stdout:(49/131): python3-jaraco-6.2-6.el8.noarch.rpm 121 kB/s | 11 kB 00:00 2023-11-24T07:53:44.066 INFO:teuthology.orchestra.run.smithi076.stdout:(50/131): python3-jaraco-functools-2.0-4.el8.no 191 kB/s | 18 kB 00:00 2023-11-24T07:53:44.108 INFO:teuthology.orchestra.run.smithi076.stdout:(51/131): python3-cherrypy-18.4.0-1.el8.noarch. 1.5 MB/s | 384 kB 00:00 2023-11-24T07:53:44.158 INFO:teuthology.orchestra.run.smithi076.stdout:(52/131): python3-logutils-0.3.5-11.el8.noarch. 530 kB/s | 49 kB 00:00 2023-11-24T07:53:44.200 INFO:teuthology.orchestra.run.smithi076.stdout:(53/131): python3-more-itertools-7.2.0-3.el8.no 639 kB/s | 59 kB 00:00 2023-11-24T07:53:44.226 INFO:teuthology.orchestra.run.smithi059.stdout:(10/131): ceph-common-18.0.0-7484.ga13ea5ff.el8 4.3 MB/s | 23 MB 00:05 2023-11-24T07:53:44.250 INFO:teuthology.orchestra.run.smithi076.stdout:(54/131): python3-natsort-7.1.1-2.el8.noarch.rp 655 kB/s | 60 kB 00:00 2023-11-24T07:53:44.284 INFO:teuthology.orchestra.run.smithi076.stdout:(55/131): python3-kubernetes-11.0.0-6.el8.noarc 4.2 MB/s | 1.2 MB 00:00 2023-11-24T07:53:44.309 INFO:teuthology.orchestra.run.smithi076.stdout:(56/131): python3-pecan-1.3.2-9.el8.noarch.rpm 2.5 MB/s | 283 kB 00:00 2023-11-24T07:53:44.336 INFO:teuthology.orchestra.run.smithi059.stdout:(11/131): cephfs-mirror-18.0.0-7484.ga13ea5ff.e 2.3 MB/s | 262 kB 00:00 2023-11-24T07:53:44.368 INFO:teuthology.orchestra.run.smithi076.stdout:(57/131): python3-portend-2.6-1.el8.noarch.rpm 133 kB/s | 16 kB 00:00 2023-11-24T07:53:44.393 INFO:teuthology.orchestra.run.smithi076.stdout:(58/131): python3-repoze-lru-0.7-6.el8.noarch.r 308 kB/s | 34 kB 00:00 2023-11-24T07:53:44.419 INFO:teuthology.orchestra.run.smithi076.stdout:(59/131): python3-routes-2.4.1-12.el8.noarch.rp 1.8 MB/s | 196 kB 00:00 2023-11-24T07:53:44.428 INFO:teuthology.orchestra.run.smithi059.stdout:(12/131): libcephfs-devel-18.0.0-7484.ga13ea5ff 340 kB/s | 31 kB 00:00 2023-11-24T07:53:44.452 INFO:teuthology.orchestra.run.smithi076.stdout:(60/131): python3-rsa-4.9-2.el8.noarch.rpm 765 kB/s | 64 kB 00:00 2023-11-24T07:53:44.486 INFO:teuthology.orchestra.run.smithi076.stdout:(61/131): python3-simplegeneric-0.8.1-17.el8.no 203 kB/s | 19 kB 00:00 2023-11-24T07:53:44.511 INFO:teuthology.orchestra.run.smithi076.stdout:(62/131): python3-singledispatch-3.4.0.3-18.el8 259 kB/s | 24 kB 00:00 2023-11-24T07:53:44.545 INFO:teuthology.orchestra.run.smithi076.stdout:(63/131): python3-tempora-1.14.1-5.el8.noarch.r 317 kB/s | 29 kB 00:00 2023-11-24T07:53:44.578 INFO:teuthology.orchestra.run.smithi076.stdout:(64/131): python3-trustme-0.6.0-4.el8.noarch.rp 298 kB/s | 27 kB 00:00 2023-11-24T07:53:44.612 INFO:teuthology.orchestra.run.smithi076.stdout:(65/131): python3-waitress-1.4.3-1.el8.noarch.r 2.5 MB/s | 255 kB 00:00 2023-11-24T07:53:44.646 INFO:teuthology.orchestra.run.smithi076.stdout:(66/131): python3-webob-1.8.5-1.el8.1.noarch.rp 2.4 MB/s | 251 kB 00:00 2023-11-24T07:53:44.671 INFO:teuthology.orchestra.run.smithi076.stdout:(67/131): python3-websocket-client-0.56.0-5.el8 663 kB/s | 61 kB 00:00 2023-11-24T07:53:44.705 INFO:teuthology.orchestra.run.smithi076.stdout:(68/131): python3-webtest-2.0.33-1.el8.noarch.r 922 kB/s | 85 kB 00:00 2023-11-24T07:53:44.739 INFO:teuthology.orchestra.run.smithi076.stdout:(69/131): python3-zc-lockfile-2.0-2.el8.noarch. 248 kB/s | 23 kB 00:00 2023-11-24T07:53:44.764 INFO:teuthology.orchestra.run.smithi076.stdout:(70/131): xmlstarlet-1.6.1-20.el8.x86_64.rpm 1.2 MB/s | 70 kB 00:00 2023-11-24T07:53:44.778 INFO:teuthology.orchestra.run.smithi059.stdout:(13/131): libcephfs2-18.0.0-7484.ga13ea5ff.el8. 2.3 MB/s | 839 kB 00:00 2023-11-24T07:53:44.806 INFO:teuthology.orchestra.run.smithi076.stdout:(71/131): librabbitmq-0.9.0-3.el8.x86_64.rpm 701 kB/s | 47 kB 00:00 2023-11-24T07:53:44.831 INFO:teuthology.orchestra.run.smithi076.stdout:(72/131): libselinux-devel-2.9-5.el8.x86_64.rpm 3.2 MB/s | 200 kB 00:00 2023-11-24T07:53:44.857 INFO:teuthology.orchestra.run.smithi076.stdout:(73/131): python3-jwt-1.6.1-2.el8.noarch.rpm 848 kB/s | 43 kB 00:00 2023-11-24T07:53:44.898 INFO:teuthology.orchestra.run.smithi076.stdout:(74/131): thrift-0.13.0-2.el8.x86_64.rpm 7.6 MB/s | 1.7 MB 00:00 2023-11-24T07:53:44.915 INFO:teuthology.orchestra.run.smithi076.stdout:(75/131): keyutils-libs-devel-1.5.10-9.el8.x86_ 821 kB/s | 48 kB 00:00 2023-11-24T07:53:44.949 INFO:teuthology.orchestra.run.smithi076.stdout:(76/131): pcre2-utf32-10.32-2.el8.x86_64.rpm 1.8 MB/s | 220 kB 00:00 2023-11-24T07:53:44.952 INFO:teuthology.orchestra.run.smithi059.stdout:(14/131): libcephsqlite-18.0.0-7484.ga13ea5ff.e 1.0 MB/s | 183 kB 00:00 2023-11-24T07:53:44.966 INFO:teuthology.orchestra.run.smithi076.stdout:(77/131): python3-requests-oauthlib-1.0.0-1.el8 954 kB/s | 43 kB 00:00 2023-11-24T07:53:44.991 INFO:teuthology.orchestra.run.smithi076.stdout:(78/131): libsepol-devel-2.9-3.el8.x86_64.rpm 940 kB/s | 87 kB 00:00 2023-11-24T07:53:45.008 INFO:teuthology.orchestra.run.smithi076.stdout:(79/131): python3-pycparser-2.14-14.el8.noarch. 1.8 MB/s | 109 kB 00:00 2023-11-24T07:53:45.042 INFO:teuthology.orchestra.run.smithi076.stdout:(80/131): pcre2-utf16-10.32-2.el8.x86_64.rpm 3.0 MB/s | 229 kB 00:00 2023-11-24T07:53:45.045 INFO:teuthology.orchestra.run.smithi059.stdout:(15/131): librados-devel-18.0.0-7484.ga13ea5ff. 1.5 MB/s | 139 kB 00:00 2023-11-24T07:53:45.067 INFO:teuthology.orchestra.run.smithi076.stdout:(81/131): libaio-devel-0.3.112-1.el8.x86_64.rpm 319 kB/s | 19 kB 00:00 2023-11-24T07:53:45.093 INFO:teuthology.orchestra.run.smithi076.stdout:(82/131): python3-oauthlib-2.1.0-1.el8.noarch.r 3.0 MB/s | 155 kB 00:00 2023-11-24T07:53:45.135 INFO:teuthology.orchestra.run.smithi076.stdout:(83/131): krb5-devel-1.18.2-14.el8.x86_64.rpm 3.8 MB/s | 560 kB 00:00 2023-11-24T07:53:45.192 INFO:teuthology.orchestra.run.smithi076.stdout:(84/131): libverto-devel-0.3.0-5.el8.x86_64.rpm 181 kB/s | 18 kB 00:00 2023-11-24T07:53:45.235 INFO:teuthology.orchestra.run.smithi076.stdout:(85/131): python3-cryptography-3.2.1-5.el8.x86_ 3.3 MB/s | 559 kB 00:00 2023-11-24T07:53:45.260 INFO:teuthology.orchestra.run.smithi076.stdout:(86/131): python3-cffi-1.11.5-5.el8.x86_64.rpm 3.4 MB/s | 238 kB 00:00 2023-11-24T07:53:45.286 INFO:teuthology.orchestra.run.smithi076.stdout:(87/131): pcre2-devel-10.32-2.el8.x86_64.rpm 3.9 MB/s | 605 kB 00:00 2023-11-24T07:53:45.311 INFO:teuthology.orchestra.run.smithi076.stdout:(88/131): xfsprogs-devel-5.0.0-10.el8.x86_64.rp 1.4 MB/s | 72 kB 00:00 2023-11-24T07:53:45.320 INFO:teuthology.orchestra.run.smithi059.stdout:(16/131): libradosstriper1-18.0.0-7484.ga13ea5f 1.8 MB/s | 517 kB 00:00 2023-11-24T07:53:45.337 INFO:teuthology.orchestra.run.smithi076.stdout:(89/131): fuse-2.9.7-15.el8.x86_64.rpm 1.6 MB/s | 83 kB 00:00 2023-11-24T07:53:45.362 INFO:teuthology.orchestra.run.smithi076.stdout:(90/131): elfutils-libelf-devel-0.186-1.el8.x86 1.2 MB/s | 60 kB 00:00 2023-11-24T07:53:45.379 INFO:teuthology.orchestra.run.smithi076.stdout:(91/131): libuuid-devel-2.32.1-35.el8.x86_64.rp 2.3 MB/s | 98 kB 00:00 2023-11-24T07:53:45.404 INFO:teuthology.orchestra.run.smithi076.stdout:(92/131): zlib-devel-1.2.11-18.el8_5.x86_64.rpm 1.3 MB/s | 58 kB 00:00 2023-11-24T07:53:45.430 INFO:teuthology.orchestra.run.smithi076.stdout:(93/131): libcom_err-devel-1.45.6-4.el8.x86_64. 765 kB/s | 39 kB 00:00 2023-11-24T07:53:45.464 INFO:teuthology.orchestra.run.smithi076.stdout:(94/131): libgfortran-8.5.0-10.1.el8_6.x86_64.r 11 MB/s | 644 kB 00:00 2023-11-24T07:53:45.489 INFO:teuthology.orchestra.run.smithi076.stdout:(95/131): libquadmath-8.5.0-10.1.el8_6.x86_64.r 2.8 MB/s | 171 kB 00:00 2023-11-24T07:53:45.514 INFO:teuthology.orchestra.run.smithi076.stdout:(96/131): NetworkManager-initscripts-updown-1.3 2.7 MB/s | 138 kB 00:00 2023-11-24T07:53:45.573 INFO:teuthology.orchestra.run.smithi076.stdout:(97/131): openssl-devel-1.1.1k-7.el8_6.x86_64.r 6.9 MB/s | 2.3 MB 00:00 2023-11-24T07:53:45.598 INFO:teuthology.orchestra.run.smithi076.stdout:(98/131): python3-pyasn1-modules-0.3.7-6.el8.no 1.3 MB/s | 110 kB 00:00 2023-11-24T07:53:45.623 INFO:teuthology.orchestra.run.smithi076.stdout:(99/131): python3-pytz-2017.2-9.el8.noarch.rpm 1.1 MB/s | 54 kB 00:00 2023-11-24T07:53:45.649 INFO:teuthology.orchestra.run.smithi076.stdout:(100/131): lttng-ust-2.8.1-11.el8.x86_64.rpm 5.0 MB/s | 259 kB 00:00 2023-11-24T07:53:45.682 INFO:teuthology.orchestra.run.smithi076.stdout:(101/131): socat-1.7.4.1-1.el8.x86_64.rpm 5.4 MB/s | 323 kB 00:00 2023-11-24T07:53:45.708 INFO:teuthology.orchestra.run.smithi076.stdout:(102/131): python3-markupsafe-0.23-19.el8.x86_6 664 kB/s | 39 kB 00:00 2023-11-24T07:53:45.742 INFO:teuthology.orchestra.run.smithi076.stdout:(103/131): bison-3.0.4-10.el8.x86_64.rpm 11 MB/s | 688 kB 00:00 2023-11-24T07:53:45.783 INFO:teuthology.orchestra.run.smithi076.stdout:(104/131): python3-jinja2-2.10.1-3.el8.noarch.r 7.0 MB/s | 538 kB 00:00 2023-11-24T07:53:45.808 INFO:teuthology.orchestra.run.smithi076.stdout:(105/131): python3-mako-1.0.6-13.el8.noarch.rpm 2.3 MB/s | 157 kB 00:00 2023-11-24T07:53:45.834 INFO:teuthology.orchestra.run.smithi076.stdout:(106/131): oniguruma-6.8.2-2.el8.x86_64.rpm 3.6 MB/s | 187 kB 00:00 2023-11-24T07:53:45.868 INFO:teuthology.orchestra.run.smithi076.stdout:(107/131): automake-1.16.1-7.el8.noarch.rpm 12 MB/s | 713 kB 00:00 2023-11-24T07:53:45.901 INFO:teuthology.orchestra.run.smithi076.stdout:(108/131): python3-jsonpatch-1.21-2.el8.noarch. 796 kB/s | 27 kB 00:00 2023-11-24T07:53:45.943 INFO:teuthology.orchestra.run.smithi076.stdout:(109/131): flex-2.6.1-9.el8.x86_64.rpm 7.5 MB/s | 320 kB 00:00 2023-11-24T07:53:45.977 INFO:teuthology.orchestra.run.smithi076.stdout:(110/131): python3-prettytable-0.7.2-14.el8.noa 1.3 MB/s | 44 kB 00:00 2023-11-24T07:53:46.061 INFO:teuthology.orchestra.run.smithi076.stdout:(111/131): python3-babel-2.5.1-7.el8.noarch.rpm 21 MB/s | 4.8 MB 00:00 2023-11-24T07:53:46.088 INFO:teuthology.orchestra.run.smithi059.stdout:(17/131): ceph-test-18.0.0-7484.ga13ea5ff.el8.x 19 MB/s | 49 MB 00:02 2023-11-24T07:53:46.111 INFO:teuthology.orchestra.run.smithi076.stdout:(112/131): python3-werkzeug-0.12.2-4.el8.noarch 8.9 MB/s | 457 kB 00:00 2023-11-24T07:53:46.138 INFO:teuthology.orchestra.run.smithi059.stdout:(18/131): python3-ceph-argparse-18.0.0-7484.ga1 902 kB/s | 45 kB 00:00 2023-11-24T07:53:46.153 INFO:teuthology.orchestra.run.smithi076.stdout:(113/131): python3-numpy-f2py-1.14.3-10.el8.x86 5.3 MB/s | 225 kB 00:00 2023-11-24T07:53:46.180 INFO:teuthology.orchestra.run.smithi059.stdout:(19/131): python3-ceph-common-18.0.0-7484.ga13e 3.0 MB/s | 128 kB 00:00 2023-11-24T07:53:46.212 INFO:teuthology.orchestra.run.smithi076.stdout:(114/131): python3-numpy-1.14.3-10.el8.x86_64.r 16 MB/s | 3.7 MB 00:00 2023-11-24T07:53:46.222 INFO:teuthology.orchestra.run.smithi059.stdout:(20/131): python3-cephfs-18.0.0-7484.ga13ea5ff. 4.6 MB/s | 197 kB 00:00 2023-11-24T07:53:46.237 INFO:teuthology.orchestra.run.smithi076.stdout:(115/131): uuid-1.6.2-43.el8.x86_64.rpm 765 kB/s | 64 kB 00:00 2023-11-24T07:53:46.262 INFO:teuthology.orchestra.run.smithi076.stdout:(116/131): python3-pyasn1-0.3.7-6.el8.noarch.rp 2.5 MB/s | 126 kB 00:00 2023-11-24T07:53:46.289 INFO:teuthology.orchestra.run.smithi076.stdout:(117/131): python3-jsonpointer-1.10-11.el8.noar 397 kB/s | 20 kB 00:00 2023-11-24T07:53:46.306 INFO:teuthology.orchestra.run.smithi059.stdout:(21/131): python3-rados-18.0.0-7484.ga13ea5ff.e 4.3 MB/s | 371 kB 00:00 2023-11-24T07:53:46.313 INFO:teuthology.orchestra.run.smithi076.stdout:(118/131): python3-pyOpenSSL-19.0.0-1.el8.noarc 2.0 MB/s | 103 kB 00:00 2023-11-24T07:53:46.346 INFO:teuthology.orchestra.run.smithi076.stdout:(119/131): libtool-2.4.6-25.el8.x86_64.rpm 12 MB/s | 709 kB 00:00 2023-11-24T07:53:46.356 INFO:teuthology.orchestra.run.smithi059.stdout:(22/131): python3-rbd-18.0.0-7484.ga13ea5ff.el8 7.1 MB/s | 364 kB 00:00 2023-11-24T07:53:46.398 INFO:teuthology.orchestra.run.smithi059.stdout:(23/131): python3-rgw-18.0.0-7484.ga13ea5ff.el8 2.5 MB/s | 107 kB 00:00 2023-11-24T07:53:46.448 INFO:teuthology.orchestra.run.smithi059.stdout:(24/131): rbd-fuse-18.0.0-7484.ga13ea5ff.el8.x8 1.8 MB/s | 94 kB 00:00 2023-11-24T07:53:46.505 INFO:teuthology.orchestra.run.smithi076.stdout:(120/131): python3-scipy-1.0.0-21.module+el8.5. 13 MB/s | 14 MB 00:01 2023-11-24T07:53:46.530 INFO:teuthology.orchestra.run.smithi076.stdout:(121/131): libpq-13.5-1.el8.x86_64.rpm 1.0 MB/s | 198 kB 00:00 2023-11-24T07:53:46.572 INFO:teuthology.orchestra.run.smithi076.stdout:(122/131): jq-1.6-3.el8.x86_64.rpm 4.7 MB/s | 202 kB 00:00 2023-11-24T07:53:46.615 INFO:teuthology.orchestra.run.smithi076.stdout:(123/131): librdkafka-0.11.4-3.el8.x86_64.rpm 8.3 MB/s | 354 kB 00:00 2023-11-24T07:53:46.673 INFO:teuthology.orchestra.run.smithi059.stdout:(25/131): rbd-mirror-18.0.0-7484.ga13ea5ff.el8. 16 MB/s | 3.6 MB 00:00 2023-11-24T07:53:46.698 INFO:teuthology.orchestra.run.smithi076.stdout:(124/131): openblas-threads-0.3.15-3.el8.x86_64 13 MB/s | 4.9 MB 00:00 2023-11-24T07:53:46.724 INFO:teuthology.orchestra.run.smithi059.stdout:(26/131): rbd-nbd-18.0.0-7484.ga13ea5ff.el8.x86 3.5 MB/s | 181 kB 00:00 2023-11-24T07:53:46.773 INFO:teuthology.orchestra.run.smithi076.stdout:(125/131): openblas-0.3.15-3.el8.x86_64.rpm 18 MB/s | 4.7 MB 00:00 2023-11-24T07:53:46.791 INFO:teuthology.orchestra.run.smithi059.stdout:(27/131): ceph-grafana-dashboards-18.0.0-7484.g 419 kB/s | 28 kB 00:00 2023-11-24T07:53:46.815 INFO:teuthology.orchestra.run.smithi076.stdout:(126/131): postgresql-contrib-10.21-2.module+el 6.7 MB/s | 808 kB 00:00 2023-11-24T07:53:46.833 INFO:teuthology.orchestra.run.smithi059.stdout:(28/131): ceph-mgr-cephadm-18.0.0-7484.ga13ea5f 3.1 MB/s | 133 kB 00:00 2023-11-24T07:53:46.874 INFO:teuthology.orchestra.run.smithi076.stdout:(127/131): postgresql-10.21-2.module+el8.6.0+15 15 MB/s | 1.5 MB 00:00 2023-11-24T07:53:46.958 INFO:teuthology.orchestra.run.smithi076.stdout:(128/131): postgresql-server-10.21-2.module+el8 15 MB/s | 5.1 MB 00:00 2023-11-24T07:53:47.008 INFO:teuthology.orchestra.run.smithi076.stdout:(129/131): iproute-5.15.0-4.el8_6.1.x86_64.rpm 16 MB/s | 798 kB 00:00 2023-11-24T07:53:47.033 INFO:teuthology.orchestra.run.smithi059.stdout:(29/131): librgw2-18.0.0-7484.ga13ea5ff.el8.x86 4.0 MB/s | 6.9 MB 00:01 2023-11-24T07:53:47.075 INFO:teuthology.orchestra.run.smithi059.stdout:(30/131): ceph-mgr-dashboard-18.0.0-7484.ga13ea 7.3 MB/s | 1.8 MB 00:00 2023-11-24T07:53:47.142 INFO:teuthology.orchestra.run.smithi059.stdout:(31/131): ceph-mgr-modules-core-18.0.0-7484.ga1 3.6 MB/s | 243 kB 00:00 2023-11-24T07:53:47.201 INFO:teuthology.orchestra.run.smithi059.stdout:(32/131): ceph-mgr-rook-18.0.0-7484.ga13ea5ff.e 863 kB/s | 50 kB 00:00 2023-11-24T07:53:47.269 INFO:teuthology.orchestra.run.smithi059.stdout:(33/131): ceph-prometheus-alerts-18.0.0-7484.ga 225 kB/s | 15 kB 00:00 2023-11-24T07:53:47.327 INFO:teuthology.orchestra.run.smithi059.stdout:(34/131): ceph-volume-18.0.0-7484.ga13ea5ff.el8 4.4 MB/s | 262 kB 00:00 2023-11-24T07:53:47.350 INFO:teuthology.orchestra.run.smithi076.stdout:(130/131): librbd1-18.0.0-7484.ga13ea5ff.el8.x8 7.8 MB/s | 3.7 MB 00:00 2023-11-24T07:53:47.393 INFO:teuthology.orchestra.run.smithi059.stdout:(35/131): cephadm-18.0.0-7484.ga13ea5ff.el8.noa 6.0 MB/s | 407 kB 00:00 2023-11-24T07:53:47.444 INFO:teuthology.orchestra.run.smithi059.stdout:(36/131): cephfs-top-18.0.0-7484.ga13ea5ff.el8. 364 kB/s | 18 kB 00:00 2023-11-24T07:53:47.508 INFO:teuthology.orchestra.run.smithi076.stdout:(131/131): librados2-18.0.0-7484.ga13ea5ff.el8. 5.4 MB/s | 3.8 MB 00:00 2023-11-24T07:53:47.511 INFO:teuthology.orchestra.run.smithi076.stdout:-------------------------------------------------------------------------------- 2023-11-24T07:53:47.511 INFO:teuthology.orchestra.run.smithi076.stdout:Total 18 MB/s | 209 MB 00:11 2023-11-24T07:53:47.627 INFO:teuthology.orchestra.run.smithi059.stdout:(37/131): gperftools-libs-2.7-9.el8.x86_64.rpm 1.6 MB/s | 306 kB 00:00 2023-11-24T07:53:47.669 INFO:teuthology.orchestra.run.smithi059.stdout:(38/131): liboath-2.6.2-3.el8.x86_64.rpm 1.4 MB/s | 59 kB 00:00 2023-11-24T07:53:47.741 INFO:teuthology.orchestra.run.smithi059.stdout:(39/131): libunwind-1.3.1-3.el8.x86_64.rpm 1.0 MB/s | 75 kB 00:00 2023-11-24T07:53:47.886 INFO:teuthology.orchestra.run.smithi059.stdout:(40/131): python3-asyncssh-2.7.0-2.el8.noarch.r 3.2 MB/s | 476 kB 00:00 2023-11-24T07:53:47.929 INFO:teuthology.orchestra.run.smithi059.stdout:(41/131): python3-bcrypt-3.1.6-2.el8.1.x86_64.r 1.0 MB/s | 44 kB 00:00 2023-11-24T07:53:48.037 INFO:teuthology.orchestra.run.smithi059.stdout:(42/131): python3-beautifulsoup4-4.6.3-2.el8.1. 1.7 MB/s | 185 kB 00:00 2023-11-24T07:53:48.171 INFO:teuthology.orchestra.run.smithi059.stdout:(43/131): ceph-mgr-diskprediction-local-18.0.0- 6.5 MB/s | 7.4 MB 00:01 2023-11-24T07:53:48.222 INFO:teuthology.orchestra.run.smithi059.stdout:(44/131): python3-cachetools-3.1.1-4.el8.noarch 179 kB/s | 33 kB 00:00 2023-11-24T07:53:48.330 INFO:teuthology.orchestra.run.smithi059.stdout:(45/131): python3-cheroot-8.5.2-1.el8.noarch.rp 1.6 MB/s | 173 kB 00:00 2023-11-24T07:53:48.331 INFO:teuthology.orchestra.run.smithi076.stdout:Running transaction check 2023-11-24T07:53:48.355 INFO:teuthology.orchestra.run.smithi059.stdout:(46/131): python3-certifi-2018.10.15-7.el8.noar 89 kB/s | 16 kB 00:00 2023-11-24T07:53:48.414 INFO:teuthology.orchestra.run.smithi059.stdout:(47/131): python3-google-auth-1.1.1-10.el8.noar 1.4 MB/s | 81 kB 00:00 2023-11-24T07:53:48.450 INFO:teuthology.orchestra.run.smithi076.stdout:Transaction check succeeded. 2023-11-24T07:53:48.450 INFO:teuthology.orchestra.run.smithi076.stdout:Running transaction test 2023-11-24T07:53:48.589 INFO:teuthology.orchestra.run.smithi059.stdout:(48/131): python3-jaraco-6.2-6.el8.noarch.rpm 63 kB/s | 11 kB 00:00 2023-11-24T07:53:48.614 INFO:teuthology.orchestra.run.smithi059.stdout:(49/131): python3-cherrypy-18.4.0-1.el8.noarch. 1.3 MB/s | 384 kB 00:00 2023-11-24T07:53:48.631 INFO:teuthology.orchestra.run.smithi059.stdout:(50/131): python3-jaraco-functools-2.0-4.el8.no 420 kB/s | 18 kB 00:00 2023-11-24T07:53:48.665 INFO:teuthology.orchestra.run.smithi059.stdout:(51/131): python3-logutils-0.3.5-11.el8.noarch. 1.4 MB/s | 49 kB 00:00 2023-11-24T07:53:48.715 INFO:teuthology.orchestra.run.smithi059.stdout:(52/131): python3-more-itertools-7.2.0-3.el8.no 1.2 MB/s | 59 kB 00:00 2023-11-24T07:53:48.749 INFO:teuthology.orchestra.run.smithi059.stdout:(53/131): python3-natsort-7.1.1-2.el8.noarch.rp 1.8 MB/s | 60 kB 00:00 2023-11-24T07:53:48.857 INFO:teuthology.orchestra.run.smithi059.stdout:(54/131): python3-pecan-1.3.2-9.el8.noarch.rpm 2.5 MB/s | 283 kB 00:00 2023-11-24T07:53:48.891 INFO:teuthology.orchestra.run.smithi059.stdout:(55/131): python3-portend-2.6-1.el8.noarch.rpm 468 kB/s | 16 kB 00:00 2023-11-24T07:53:48.966 INFO:teuthology.orchestra.run.smithi059.stdout:(56/131): python3-repoze-lru-0.7-6.el8.noarch.r 449 kB/s | 34 kB 00:00 2023-11-24T07:53:49.067 INFO:teuthology.orchestra.run.smithi059.stdout:(57/131): python3-kubernetes-11.0.0-6.el8.noarc 2.6 MB/s | 1.2 MB 00:00 2023-11-24T07:53:49.267 INFO:teuthology.orchestra.run.smithi059.stdout:(58/131): python3-routes-2.4.1-12.el8.noarch.rp 653 kB/s | 196 kB 00:00 2023-11-24T07:53:49.293 INFO:teuthology.orchestra.run.smithi059.stdout:(59/131): python3-rsa-4.9-2.el8.noarch.rpm 285 kB/s | 64 kB 00:00 2023-11-24T07:53:49.318 INFO:teuthology.orchestra.run.smithi059.stdout:(60/131): python3-simplegeneric-0.8.1-17.el8.no 372 kB/s | 19 kB 00:00 2023-11-24T07:53:49.343 INFO:teuthology.orchestra.run.smithi059.stdout:(61/131): python3-singledispatch-3.4.0.3-18.el8 475 kB/s | 24 kB 00:00 2023-11-24T07:53:49.369 INFO:teuthology.orchestra.run.smithi059.stdout:(62/131): python3-tempora-1.14.1-5.el8.noarch.r 583 kB/s | 29 kB 00:00 2023-11-24T07:53:49.394 INFO:teuthology.orchestra.run.smithi059.stdout:(63/131): python3-trustme-0.6.0-4.el8.noarch.rp 548 kB/s | 27 kB 00:00 2023-11-24T07:53:49.497 INFO:teuthology.orchestra.run.smithi059.stdout:(64/131): python3-waitress-1.4.3-1.el8.noarch.r 2.0 MB/s | 255 kB 00:00 2023-11-24T07:53:49.545 INFO:teuthology.orchestra.run.smithi059.stdout:(65/131): python3-webob-1.8.5-1.el8.1.noarch.rp 1.6 MB/s | 251 kB 00:00 2023-11-24T07:53:49.570 INFO:teuthology.orchestra.run.smithi059.stdout:(66/131): python3-websocket-client-0.56.0-5.el8 844 kB/s | 61 kB 00:00 2023-11-24T07:53:49.577 INFO:teuthology.orchestra.run.smithi076.stdout:Transaction test succeeded. 2023-11-24T07:53:49.581 INFO:teuthology.orchestra.run.smithi076.stdout:Running transaction 2023-11-24T07:53:49.603 INFO:teuthology.orchestra.run.smithi059.stdout:(67/131): python3-webtest-2.0.33-1.el8.noarch.r 1.4 MB/s | 85 kB 00:00 2023-11-24T07:53:49.629 INFO:teuthology.orchestra.run.smithi059.stdout:(68/131): python3-zc-lockfile-2.0-2.el8.noarch. 391 kB/s | 23 kB 00:00 2023-11-24T07:53:49.654 INFO:teuthology.orchestra.run.smithi059.stdout:(69/131): xmlstarlet-1.6.1-20.el8.x86_64.rpm 2.7 MB/s | 70 kB 00:00 2023-11-24T07:53:50.013 INFO:teuthology.orchestra.run.smithi059.stdout:(70/131): python3-scipy-1.0.0-21.module+el8.5.0 38 MB/s | 14 MB 00:00 2023-11-24T07:53:50.046 INFO:teuthology.orchestra.run.smithi059.stdout:(71/131): python3-pyasn1-modules-0.3.7-6.el8.no 3.3 MB/s | 110 kB 00:00 2023-11-24T07:53:50.248 INFO:teuthology.orchestra.run.smithi059.stdout:(72/131): ceph-osd-18.0.0-7484.ga13ea5ff.el8.x8 2.2 MB/s | 18 MB 00:08 2023-11-24T07:53:50.280 INFO:teuthology.orchestra.run.smithi059.stdout:(73/131): python3-pytz-2017.2-9.el8.noarch.rpm 232 kB/s | 54 kB 00:00 2023-11-24T07:53:50.330 INFO:teuthology.orchestra.run.smithi059.stdout:(74/131): socat-1.7.4.1-1.el8.x86_64.rpm 7.6 MB/s | 323 kB 00:00 2023-11-24T07:53:50.356 INFO:teuthology.orchestra.run.smithi059.stdout:(75/131): lttng-ust-2.8.1-11.el8.x86_64.rpm 2.4 MB/s | 259 kB 00:00 2023-11-24T07:53:50.381 INFO:teuthology.orchestra.run.smithi059.stdout:(76/131): python3-markupsafe-0.23-19.el8.x86_64 773 kB/s | 39 kB 00:00 2023-11-24T07:53:50.423 INFO:teuthology.orchestra.run.smithi059.stdout:(77/131): bison-3.0.4-10.el8.x86_64.rpm 10 MB/s | 688 kB 00:00 2023-11-24T07:53:50.457 INFO:teuthology.orchestra.run.smithi059.stdout:(78/131): python3-jinja2-2.10.1-3.el8.noarch.rp 7.0 MB/s | 538 kB 00:00 2023-11-24T07:53:50.523 INFO:teuthology.orchestra.run.smithi059.stdout:(79/131): thrift-0.13.0-2.el8.x86_64.rpm 1.9 MB/s | 1.7 MB 00:00 2023-11-24T07:53:50.549 INFO:teuthology.orchestra.run.smithi059.stdout:(80/131): python3-mako-1.0.6-13.el8.noarch.rpm 1.2 MB/s | 157 kB 00:00 2023-11-24T07:53:50.582 INFO:teuthology.orchestra.run.smithi059.stdout:(81/131): oniguruma-6.8.2-2.el8.x86_64.rpm 1.5 MB/s | 187 kB 00:00 2023-11-24T07:53:50.633 INFO:teuthology.orchestra.run.smithi059.stdout:(82/131): automake-1.16.1-7.el8.noarch.rpm 6.4 MB/s | 713 kB 00:00 2023-11-24T07:53:50.650 INFO:teuthology.orchestra.run.smithi059.stdout:(83/131): python3-jsonpatch-1.21-2.el8.noarch.r 397 kB/s | 27 kB 00:00 2023-11-24T07:53:50.684 INFO:teuthology.orchestra.run.smithi059.stdout:(84/131): flex-2.6.1-9.el8.x86_64.rpm 6.2 MB/s | 320 kB 00:00 2023-11-24T07:53:50.729 INFO:teuthology.orchestra.run.smithi059.stdout:(85/131): python3-prettytable-0.7.2-14.el8.noar 553 kB/s | 44 kB 00:00 2023-11-24T07:53:50.792 INFO:teuthology.orchestra.run.smithi059.stdout:(86/131): python3-werkzeug-0.12.2-4.el8.noarch. 7.1 MB/s | 457 kB 00:00 2023-11-24T07:53:50.834 INFO:teuthology.orchestra.run.smithi059.stdout:(87/131): python3-numpy-f2py-1.14.3-10.el8.x86_ 5.3 MB/s | 225 kB 00:00 2023-11-24T07:53:50.868 INFO:teuthology.orchestra.run.smithi059.stdout:(88/131): uuid-1.6.2-43.el8.x86_64.rpm 1.9 MB/s | 64 kB 00:00 2023-11-24T07:53:50.966 INFO:teuthology.orchestra.run.smithi059.stdout:(89/131): python3-babel-2.5.1-7.el8.noarch.rpm 12 MB/s | 4.8 MB 00:00 2023-11-24T07:53:51.035 INFO:teuthology.orchestra.run.smithi059.stdout:(90/131): python3-numpy-1.14.3-10.el8.x86_64.rp 11 MB/s | 3.7 MB 00:00 2023-11-24T07:53:51.060 INFO:teuthology.orchestra.run.smithi059.stdout:(91/131): python3-jsonpointer-1.10-11.el8.noarc 213 kB/s | 20 kB 00:00 2023-11-24T07:53:51.086 INFO:teuthology.orchestra.run.smithi059.stdout:(92/131): python3-pyasn1-0.3.7-6.el8.noarch.rpm 581 kB/s | 126 kB 00:00 2023-11-24T07:53:51.111 INFO:teuthology.orchestra.run.smithi059.stdout:(93/131): python3-pyOpenSSL-19.0.0-1.el8.noarch 1.3 MB/s | 103 kB 00:00 2023-11-24T07:53:51.153 INFO:teuthology.orchestra.run.smithi059.stdout:(94/131): libtool-2.4.6-25.el8.x86_64.rpm 7.5 MB/s | 709 kB 00:00 2023-11-24T07:53:51.179 INFO:teuthology.orchestra.run.smithi059.stdout:(95/131): libpq-13.5-1.el8.x86_64.rpm 2.9 MB/s | 198 kB 00:00 2023-11-24T07:53:51.237 INFO:teuthology.orchestra.run.smithi059.stdout:(96/131): jq-1.6-3.el8.x86_64.rpm 3.4 MB/s | 202 kB 00:00 2023-11-24T07:53:51.287 INFO:teuthology.orchestra.run.smithi059.stdout:(97/131): librdkafka-0.11.4-3.el8.x86_64.rpm 6.9 MB/s | 354 kB 00:00 2023-11-24T07:53:51.388 INFO:teuthology.orchestra.run.smithi059.stdout:(98/131): openblas-threads-0.3.15-3.el8.x86_64. 16 MB/s | 4.9 MB 00:00 2023-11-24T07:53:51.488 INFO:teuthology.orchestra.run.smithi059.stdout:(99/131): openblas-0.3.15-3.el8.x86_64.rpm 14 MB/s | 4.7 MB 00:00 2023-11-24T07:53:51.538 INFO:teuthology.orchestra.run.smithi059.stdout:(100/131): postgresql-contrib-10.21-2.module+el 5.3 MB/s | 808 kB 00:00 2023-11-24T07:53:51.580 INFO:teuthology.orchestra.run.smithi059.stdout:(101/131): librabbitmq-0.9.0-3.el8.x86_64.rpm 1.1 MB/s | 47 kB 00:00 2023-11-24T07:53:51.622 INFO:teuthology.orchestra.run.smithi059.stdout:(102/131): postgresql-10.21-2.module+el8.6.0+15 11 MB/s | 1.5 MB 00:00 2023-11-24T07:53:51.639 INFO:teuthology.orchestra.run.smithi059.stdout:(103/131): libselinux-devel-2.9-5.el8.x86_64.rp 3.3 MB/s | 200 kB 00:00 2023-11-24T07:53:51.664 INFO:teuthology.orchestra.run.smithi059.stdout:(104/131): python3-jwt-1.6.1-2.el8.noarch.rpm 1.0 MB/s | 43 kB 00:00 2023-11-24T07:53:51.715 INFO:teuthology.orchestra.run.smithi059.stdout:(105/131): pcre2-utf32-10.32-2.el8.x86_64.rpm 2.8 MB/s | 220 kB 00:00 2023-11-24T07:53:51.748 INFO:teuthology.orchestra.run.smithi059.stdout:(106/131): keyutils-libs-devel-1.5.10-9.el8.x86 575 kB/s | 48 kB 00:00 2023-11-24T07:53:51.773 INFO:teuthology.orchestra.run.smithi059.stdout:(107/131): libsepol-devel-2.9-3.el8.x86_64.rpm 1.5 MB/s | 87 kB 00:00 2023-11-24T07:53:51.799 INFO:teuthology.orchestra.run.smithi059.stdout:(108/131): python3-requests-oauthlib-1.0.0-1.el 851 kB/s | 43 kB 00:00 2023-11-24T07:53:51.816 INFO:teuthology.orchestra.run.smithi059.stdout:(109/131): python3-pycparser-2.14-14.el8.noarch 2.5 MB/s | 109 kB 00:00 2023-11-24T07:53:51.883 INFO:teuthology.orchestra.run.smithi059.stdout:(110/131): postgresql-server-10.21-2.module+el8 8.5 MB/s | 5.1 MB 00:00 2023-11-24T07:53:51.908 INFO:teuthology.orchestra.run.smithi059.stdout:(111/131): pcre2-utf16-10.32-2.el8.x86_64.rpm 2.1 MB/s | 229 kB 00:00 2023-11-24T07:53:51.960 INFO:teuthology.orchestra.run.smithi059.stdout:(112/131): libaio-devel-0.3.112-1.el8.x86_64.rp 242 kB/s | 19 kB 00:00 2023-11-24T07:53:51.992 INFO:teuthology.orchestra.run.smithi059.stdout:(113/131): python3-oauthlib-2.1.0-1.el8.noarch. 1.8 MB/s | 155 kB 00:00 2023-11-24T07:53:52.025 INFO:teuthology.orchestra.run.smithi059.stdout:(114/131): krb5-devel-1.18.2-14.el8.x86_64.rpm 2.6 MB/s | 560 kB 00:00 2023-11-24T07:53:52.051 INFO:teuthology.orchestra.run.smithi059.stdout:(115/131): libverto-devel-0.3.0-5.el8.x86_64.rp 306 kB/s | 18 kB 00:00 2023-11-24T07:53:52.084 INFO:teuthology.orchestra.run.smithi059.stdout:(116/131): python3-cryptography-3.2.1-5.el8.x86 4.4 MB/s | 559 kB 00:00 2023-11-24T07:53:52.110 INFO:teuthology.orchestra.run.smithi059.stdout:(117/131): python3-cffi-1.11.5-5.el8.x86_64.rpm 4.0 MB/s | 238 kB 00:00 2023-11-24T07:53:52.152 INFO:teuthology.orchestra.run.smithi059.stdout:(118/131): pcre2-devel-10.32-2.el8.x86_64.rpm 4.7 MB/s | 605 kB 00:00 2023-11-24T07:53:52.210 INFO:teuthology.orchestra.run.smithi059.stdout:(119/131): xfsprogs-devel-5.0.0-10.el8.x86_64.r 722 kB/s | 72 kB 00:00 2023-11-24T07:53:52.236 INFO:teuthology.orchestra.run.smithi059.stdout:(120/131): fuse-2.9.7-15.el8.x86_64.rpm 990 kB/s | 83 kB 00:00 2023-11-24T07:53:52.261 INFO:teuthology.orchestra.run.smithi059.stdout:(121/131): elfutils-libelf-devel-0.186-1.el8.x8 1.2 MB/s | 60 kB 00:00 2023-11-24T07:53:52.286 INFO:teuthology.orchestra.run.smithi059.stdout:(122/131): libuuid-devel-2.32.1-35.el8.x86_64.r 1.9 MB/s | 98 kB 00:00 2023-11-24T07:53:52.311 INFO:teuthology.orchestra.run.smithi059.stdout:(123/131): zlib-devel-1.2.11-18.el8_5.x86_64.rp 1.1 MB/s | 58 kB 00:00 2023-11-24T07:53:52.337 INFO:teuthology.orchestra.run.smithi059.stdout:(124/131): libcom_err-devel-1.45.6-4.el8.x86_64 770 kB/s | 39 kB 00:00 2023-11-24T07:53:52.379 INFO:teuthology.orchestra.run.smithi059.stdout:(125/131): libgfortran-8.5.0-10.1.el8_6.x86_64. 9.4 MB/s | 644 kB 00:00 2023-11-24T07:53:52.404 INFO:teuthology.orchestra.run.smithi059.stdout:(126/131): libquadmath-8.5.0-10.1.el8_6.x86_64. 2.5 MB/s | 171 kB 00:00 2023-11-24T07:53:52.451 INFO:teuthology.orchestra.run.smithi059.stdout:(127/131): NetworkManager-initscripts-updown-1. 1.9 MB/s | 138 kB 00:00 2023-11-24T07:53:52.513 INFO:teuthology.orchestra.run.smithi059.stdout:(128/131): openssl-devel-1.1.1k-7.el8_6.x86_64. 5.4 MB/s | 2.3 MB 00:00 2023-11-24T07:53:52.563 INFO:teuthology.orchestra.run.smithi059.stdout:(129/131): iproute-5.15.0-4.el8_6.1.x86_64.rpm 16 MB/s | 798 kB 00:00 2023-11-24T07:53:52.822 INFO:teuthology.orchestra.run.smithi059.stdout:(130/131): librados2-18.0.0-7484.ga13ea5ff.el8. 9.0 MB/s | 3.8 MB 00:00 2023-11-24T07:53:53.564 INFO:teuthology.orchestra.run.smithi059.stdout:(131/131): librbd1-18.0.0-7484.ga13ea5ff.el8.x8 3.3 MB/s | 3.7 MB 00:01 2023-11-24T07:53:53.568 INFO:teuthology.orchestra.run.smithi059.stdout:-------------------------------------------------------------------------------- 2023-11-24T07:53:53.568 INFO:teuthology.orchestra.run.smithi059.stdout:Total 14 MB/s | 209 MB 00:14 2023-11-24T07:53:54.436 INFO:teuthology.orchestra.run.smithi059.stdout:Running transaction check 2023-11-24T07:53:54.448 INFO:teuthology.orchestra.run.smithi076.stdout: Preparing : 1/1 2023-11-24T07:53:54.568 INFO:teuthology.orchestra.run.smithi059.stdout:Transaction check succeeded. 2023-11-24T07:53:54.568 INFO:teuthology.orchestra.run.smithi059.stdout:Running transaction test 2023-11-24T07:53:54.617 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : thrift-0.13.0-2.el8.x86_64 1/134 2023-11-24T07:53:54.800 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : liboath-2.6.2-3.el8.x86_64 2/134 2023-11-24T07:53:54.986 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : librdkafka-0.11.4-3.el8.x86_64 3/134 2023-11-24T07:53:55.115 INFO:teuthology.orchestra.run.smithi076.stdout: Running scriptlet: librdkafka-0.11.4-3.el8.x86_64 3/134 2023-11-24T07:53:55.344 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : lttng-ust-2.8.1-11.el8.x86_64 4/134 2023-11-24T07:53:55.756 INFO:teuthology.orchestra.run.smithi076.stdout: Running scriptlet: lttng-ust-2.8.1-11.el8.x86_64 4/134 2023-11-24T07:53:55.779 INFO:teuthology.orchestra.run.smithi059.stdout:Transaction test succeeded. 2023-11-24T07:53:55.783 INFO:teuthology.orchestra.run.smithi059.stdout:Running transaction 2023-11-24T07:53:56.010 INFO:teuthology.orchestra.run.smithi076.stdout: Upgrading : librados2-2:18.0.0-7484.ga13ea5ff.el8.x86_64 5/134 2023-11-24T07:53:56.416 INFO:teuthology.orchestra.run.smithi076.stdout: Running scriptlet: librados2-2:18.0.0-7484.ga13ea5ff.el8.x86_64 5/134 2023-11-24T07:53:56.610 INFO:teuthology.orchestra.run.smithi076.stdout: Upgrading : librbd1-2:18.0.0-7484.ga13ea5ff.el8.x86_64 6/134 2023-11-24T07:53:56.740 INFO:teuthology.orchestra.run.smithi076.stdout: Running scriptlet: librbd1-2:18.0.0-7484.ga13ea5ff.el8.x86_64 6/134 2023-11-24T07:53:56.985 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : libcephfs2-2:18.0.0-7484.ga13ea5ff.el8.x86_64 7/134 2023-11-24T07:53:57.083 INFO:teuthology.orchestra.run.smithi076.stdout: Running scriptlet: libcephfs2-2:18.0.0-7484.ga13ea5ff.el8.x86_64 7/134 2023-11-24T07:53:57.291 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : python3-rados-2:18.0.0-7484.ga13ea5ff.el8.x86_64 8/134 2023-11-24T07:53:57.486 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : librabbitmq-0.9.0-3.el8.x86_64 9/134 2023-11-24T07:53:57.764 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : libpq-13.5-1.el8.x86_64 10/134 2023-11-24T07:53:57.988 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : python3-pyasn1-0.3.7-6.el8.noarch 11/134 2023-11-24T07:53:58.402 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : python3-more-itertools-7.2.0-3.el8.noarch 12/134 2023-11-24T07:53:58.684 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : postgresql-10.21-2.module+el8.6.0+15342+53518fac 13/134 2023-11-24T07:53:58.860 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : libcephsqlite-2:18.0.0-7484.ga13ea5ff.el8.x86_64 14/134 2023-11-24T07:53:58.976 INFO:teuthology.orchestra.run.smithi076.stdout: Running scriptlet: libcephsqlite-2:18.0.0-7484.ga13ea5ff.el8.x86_64 14/134 2023-11-24T07:53:59.160 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : libradosstriper1-2:18.0.0-7484.ga13ea5ff.el8.x86 15/134 2023-11-24T07:53:59.309 INFO:teuthology.orchestra.run.smithi076.stdout: Running scriptlet: libradosstriper1-2:18.0.0-7484.ga13ea5ff.el8.x86 15/134 2023-11-24T07:53:59.530 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : python3-werkzeug-0.12.2-4.el8.noarch 16/134 2023-11-24T07:53:59.724 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : python3-markupsafe-0.23-19.el8.x86_64 17/134 2023-11-24T07:53:59.912 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : python3-pytz-2017.2-9.el8.noarch 18/134 2023-11-24T07:54:00.060 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : libquadmath-8.5.0-10.1.el8_6.x86_64 19/134 2023-11-24T07:54:00.189 INFO:teuthology.orchestra.run.smithi076.stdout: Running scriptlet: libquadmath-8.5.0-10.1.el8_6.x86_64 19/134 2023-11-24T07:54:00.351 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : libgfortran-8.5.0-10.1.el8_6.x86_64 20/134 2023-11-24T07:54:00.965 INFO:teuthology.orchestra.run.smithi076.stdout: Running scriptlet: libgfortran-8.5.0-10.1.el8_6.x86_64 20/134 2023-11-24T07:54:01.118 INFO:teuthology.orchestra.run.smithi059.stdout: Preparing : 1/1 2023-11-24T07:54:01.176 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : openblas-threads-0.3.15-3.el8.x86_64 21/134 2023-11-24T07:54:01.337 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : thrift-0.13.0-2.el8.x86_64 1/134 2023-11-24T07:54:01.569 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : liboath-2.6.2-3.el8.x86_64 2/134 2023-11-24T07:54:01.774 INFO:teuthology.orchestra.run.smithi076.stdout: Running scriptlet: openblas-threads-0.3.15-3.el8.x86_64 21/134 2023-11-24T07:54:01.798 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : librabbitmq-0.9.0-3.el8.x86_64 3/134 2023-11-24T07:54:01.984 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : openblas-0.3.15-3.el8.x86_64 22/134 2023-11-24T07:54:01.997 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : librdkafka-0.11.4-3.el8.x86_64 4/134 2023-11-24T07:54:02.110 INFO:teuthology.orchestra.run.smithi059.stdout: Running scriptlet: librdkafka-0.11.4-3.el8.x86_64 4/134 2023-11-24T07:54:02.331 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : lttng-ust-2.8.1-11.el8.x86_64 5/134 2023-11-24T07:54:02.696 INFO:teuthology.orchestra.run.smithi076.stdout: Running scriptlet: openblas-0.3.15-3.el8.x86_64 22/134 2023-11-24T07:54:02.751 INFO:teuthology.orchestra.run.smithi059.stdout: Running scriptlet: lttng-ust-2.8.1-11.el8.x86_64 5/134 2023-11-24T07:54:02.949 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : python3-numpy-1:1.14.3-10.el8.x86_64 23/134 2023-11-24T07:54:02.997 INFO:teuthology.orchestra.run.smithi059.stdout: Upgrading : librados2-2:18.0.0-7484.ga13ea5ff.el8.x86_64 6/134 2023-11-24T07:54:03.141 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : zlib-devel-1.2.11-18.el8_5.x86_64 24/134 2023-11-24T07:54:03.379 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : python3-webob-1.8.5-1.el8.1.noarch 25/134 2023-11-24T07:54:03.433 INFO:teuthology.orchestra.run.smithi059.stdout: Running scriptlet: librados2-2:18.0.0-7484.ga13ea5ff.el8.x86_64 6/134 2023-11-24T07:54:03.574 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : python3-ceph-common-2:18.0.0-7484.ga13ea5ff.el8. 26/134 2023-11-24T07:54:03.648 INFO:teuthology.orchestra.run.smithi059.stdout: Upgrading : librbd1-2:18.0.0-7484.ga13ea5ff.el8.x86_64 7/134 2023-11-24T07:54:03.720 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : python3-ceph-argparse-2:18.0.0-7484.ga13ea5ff.el 27/134 2023-11-24T07:54:03.782 INFO:teuthology.orchestra.run.smithi059.stdout: Running scriptlet: librbd1-2:18.0.0-7484.ga13ea5ff.el8.x86_64 7/134 2023-11-24T07:54:03.964 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : python3-cephfs-2:18.0.0-7484.ga13ea5ff.el8.x86_6 28/134 2023-11-24T07:54:03.989 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : libcephfs2-2:18.0.0-7484.ga13ea5ff.el8.x86_64 8/134 2023-11-24T07:54:04.079 INFO:teuthology.orchestra.run.smithi059.stdout: Running scriptlet: libcephfs2-2:18.0.0-7484.ga13ea5ff.el8.x86_64 8/134 2023-11-24T07:54:04.159 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : python3-numpy-f2py-1:1.14.3-10.el8.x86_64 29/134 2023-11-24T07:54:04.313 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : python3-rados-2:18.0.0-7484.ga13ea5ff.el8.x86_64 9/134 2023-11-24T07:54:04.615 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : libpq-13.5-1.el8.x86_64 10/134 2023-11-24T07:54:04.848 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : python3-pyasn1-0.3.7-6.el8.noarch 11/134 2023-11-24T07:54:05.278 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : python3-more-itertools-7.2.0-3.el8.noarch 12/134 2023-11-24T07:54:05.747 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : postgresql-10.21-2.module+el8.6.0+15342+53518fac 13/134 2023-11-24T07:54:05.928 INFO:teuthology.orchestra.run.smithi076.stdout: Running scriptlet: python3-numpy-f2py-1:1.14.3-10.el8.x86_64 29/134 2023-11-24T07:54:05.938 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : libcephsqlite-2:18.0.0-7484.ga13ea5ff.el8.x86_64 14/134 2023-11-24T07:54:06.050 INFO:teuthology.orchestra.run.smithi059.stdout: Running scriptlet: libcephsqlite-2:18.0.0-7484.ga13ea5ff.el8.x86_64 14/134 2023-11-24T07:54:06.247 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : libradosstriper1-2:18.0.0-7484.ga13ea5ff.el8.x86 15/134 2023-11-24T07:54:06.309 INFO:teuthology.orchestra.run.smithi059.stdout: Running scriptlet: libradosstriper1-2:18.0.0-7484.ga13ea5ff.el8.x86 15/134 2023-11-24T07:54:06.480 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : libquadmath-8.5.0-10.1.el8_6.x86_64 16/134 2023-11-24T07:54:06.610 INFO:teuthology.orchestra.run.smithi059.stdout: Running scriptlet: libquadmath-8.5.0-10.1.el8_6.x86_64 16/134 2023-11-24T07:54:06.772 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : libgfortran-8.5.0-10.1.el8_6.x86_64 17/134 2023-11-24T07:54:06.974 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : python3-scipy-1.0.0-21.module+el8.5.0+10916+41bd 30/134 2023-11-24T07:54:07.282 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : python3-babel-2.5.1-7.el8.noarch 31/134 2023-11-24T07:54:07.466 INFO:teuthology.orchestra.run.smithi059.stdout: Running scriptlet: libgfortran-8.5.0-10.1.el8_6.x86_64 17/134 2023-11-24T07:54:07.544 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : python3-jinja2-2.10.1-3.el8.noarch 32/134 2023-11-24T07:54:07.680 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : openblas-threads-0.3.15-3.el8.x86_64 18/134 2023-11-24T07:54:07.788 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : python3-mako-1.0.6-13.el8.noarch 33/134 2023-11-24T07:54:08.012 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : python3-rsa-4.9-2.el8.noarch 34/134 2023-11-24T07:54:08.188 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : python3-pyasn1-modules-0.3.7-6.el8.noarch 35/134 2023-11-24T07:54:08.327 INFO:teuthology.orchestra.run.smithi059.stdout: Running scriptlet: openblas-threads-0.3.15-3.el8.x86_64 18/134 2023-11-24T07:54:08.381 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : python3-rbd-2:18.0.0-7484.ga13ea5ff.el8.x86_64 36/134 2023-11-24T07:54:08.556 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : openblas-0.3.15-3.el8.x86_64 19/134 2023-11-24T07:54:08.574 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : librados-devel-2:18.0.0-7484.ga13ea5ff.el8.x86_6 37/134 2023-11-24T07:54:08.741 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : python3-jsonpointer-1.10-11.el8.noarch 38/134 2023-11-24T07:54:08.910 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : python3-jsonpatch-1.21-2.el8.noarch 39/134 2023-11-24T07:54:09.092 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : uuid-1.6.2-43.el8.x86_64 40/134 2023-11-24T07:54:09.172 INFO:teuthology.orchestra.run.smithi076.stdout: Running scriptlet: uuid-1.6.2-43.el8.x86_64 40/134 2023-11-24T07:54:09.325 INFO:teuthology.orchestra.run.smithi059.stdout: Running scriptlet: openblas-0.3.15-3.el8.x86_64 19/134 2023-11-24T07:54:09.424 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : python3-prettytable-0.7.2-14.el8.noarch 41/134 2023-11-24T07:54:09.626 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : python3-numpy-1:1.14.3-10.el8.x86_64 20/134 2023-11-24T07:54:09.645 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : automake-1.16.1-7.el8.noarch 42/134 2023-11-24T07:54:09.792 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : oniguruma-6.8.2-2.el8.x86_64 43/134 2023-11-24T07:54:09.859 INFO:teuthology.orchestra.run.smithi076.stdout: Running scriptlet: oniguruma-6.8.2-2.el8.x86_64 43/134 2023-11-24T07:54:09.878 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : zlib-devel-1.2.11-18.el8_5.x86_64 21/134 2023-11-24T07:54:10.117 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : python3-werkzeug-0.12.2-4.el8.noarch 22/134 2023-11-24T07:54:10.329 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : python3-markupsafe-0.23-19.el8.x86_64 23/134 2023-11-24T07:54:10.455 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : jq-1.6-3.el8.x86_64 44/134 2023-11-24T07:54:10.556 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : python3-pytz-2017.2-9.el8.noarch 24/134 2023-11-24T07:54:10.638 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : socat-1.7.4.1-1.el8.x86_64 45/134 2023-11-24T07:54:10.816 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : libcom_err-devel-1.45.6-4.el8.x86_64 46/134 2023-11-24T07:54:10.980 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : python3-webob-1.8.5-1.el8.1.noarch 25/134 2023-11-24T07:54:11.019 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : libuuid-devel-2.32.1-35.el8.x86_64 47/134 2023-11-24T07:54:11.178 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : fuse-2.9.7-15.el8.x86_64 48/134 2023-11-24T07:54:11.178 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : python3-ceph-common-2:18.0.0-7484.ga13ea5ff.el8. 26/134 2023-11-24T07:54:11.304 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : libverto-devel-0.3.0-5.el8.x86_64 49/134 2023-11-24T07:54:11.368 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : python3-ceph-argparse-2:18.0.0-7484.ga13ea5ff.el 27/134 2023-11-24T07:54:11.507 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : pcre2-utf16-10.32-2.el8.x86_64 50/134 2023-11-24T07:54:11.705 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : python3-pycparser-2.14-14.el8.noarch 51/134 2023-11-24T07:54:12.095 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : python3-cffi-1.11.5-5.el8.x86_64 52/134 2023-11-24T07:54:12.350 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : python3-cephfs-2:18.0.0-7484.ga13ea5ff.el8.x86_6 28/134 2023-11-24T07:54:12.385 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : python3-cryptography-3.2.1-5.el8.x86_64 53/134 2023-11-24T07:54:12.603 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : python3-jwt-1.6.1-2.el8.noarch 54/134 2023-11-24T07:54:12.684 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : python3-babel-2.5.1-7.el8.noarch 29/134 2023-11-24T07:54:12.871 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : python3-pyOpenSSL-19.0.0-1.el8.noarch 55/134 2023-11-24T07:54:12.969 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : python3-jinja2-2.10.1-3.el8.noarch 30/134 2023-11-24T07:54:13.104 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : python3-oauthlib-2.1.0-1.el8.noarch 56/134 2023-11-24T07:54:13.226 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : python3-mako-1.0.6-13.el8.noarch 31/134 2023-11-24T07:54:13.408 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : python3-requests-oauthlib-1.0.0-1.el8.noarch 57/134 2023-11-24T07:54:13.412 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : python3-numpy-f2py-1:1.14.3-10.el8.x86_64 32/134 2023-11-24T07:54:13.657 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : python3-asyncssh-2.7.0-2.el8.noarch 58/134 2023-11-24T07:54:13.850 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : python3-trustme-0.6.0-4.el8.noarch 59/134 2023-11-24T07:54:14.071 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : python3-bcrypt-3.1.6-2.el8.1.x86_64 60/134 2023-11-24T07:54:14.264 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : libsepol-devel-2.9-3.el8.x86_64 61/134 2023-11-24T07:54:14.469 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : keyutils-libs-devel-1.5.10-9.el8.x86_64 62/134 2023-11-24T07:54:14.764 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : pcre2-utf32-10.32-2.el8.x86_64 63/134 2023-11-24T07:54:15.021 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : pcre2-devel-10.32-2.el8.x86_64 64/134 2023-11-24T07:54:15.286 INFO:teuthology.orchestra.run.smithi059.stdout: Running scriptlet: python3-numpy-f2py-1:1.14.3-10.el8.x86_64 32/134 2023-11-24T07:54:15.296 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : libselinux-devel-2.9-5.el8.x86_64 65/134 2023-11-24T07:54:15.517 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : krb5-devel-1.18.2-14.el8.x86_64 66/134 2023-11-24T07:54:15.624 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : python3-scipy-1.0.0-21.module+el8.5.0+10916+41bd 33/134 2023-11-24T07:54:15.682 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : xmlstarlet-1.6.1-20.el8.x86_64 67/134 2023-11-24T07:54:15.844 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : python3-rsa-4.9-2.el8.noarch 34/134 2023-11-24T07:54:15.869 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : python3-zc-lockfile-2.0-2.el8.noarch 68/134 2023-11-24T07:54:16.050 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : python3-pyasn1-modules-0.3.7-6.el8.noarch 35/134 2023-11-24T07:54:16.150 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : python3-websocket-client-0.56.0-5.el8.noarch 69/134 2023-11-24T07:54:16.261 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : python3-rbd-2:18.0.0-7484.ga13ea5ff.el8.x86_64 36/134 2023-11-24T07:54:16.389 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : python3-waitress-1.4.3-1.el8.noarch 70/134 2023-11-24T07:54:16.434 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : librados-devel-2:18.0.0-7484.ga13ea5ff.el8.x86_6 37/134 2023-11-24T07:54:16.554 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : python3-singledispatch-3.4.0.3-18.el8.noarch 71/134 2023-11-24T07:54:16.617 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : libcom_err-devel-1.45.6-4.el8.x86_64 38/134 2023-11-24T07:54:16.693 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : python3-simplegeneric-0.8.1-17.el8.noarch 72/134 2023-11-24T07:54:16.840 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : libuuid-devel-2.32.1-35.el8.x86_64 39/134 2023-11-24T07:54:16.899 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : python3-repoze-lru-0.7-6.el8.noarch 73/134 2023-11-24T07:54:17.040 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : fuse-2.9.7-15.el8.x86_64 40/134 2023-11-24T07:54:17.102 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : python3-routes-2.4.1-12.el8.noarch 74/134 2023-11-24T07:54:17.201 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : libverto-devel-0.3.0-5.el8.x86_64 41/134 2023-11-24T07:54:17.276 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : python3-natsort-7.1.1-2.el8.noarch 75/134 2023-11-24T07:54:17.427 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : python3-logutils-0.3.5-11.el8.noarch 76/134 2023-11-24T07:54:17.431 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : pcre2-utf16-10.32-2.el8.x86_64 42/134 2023-11-24T07:54:17.572 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : python3-jaraco-6.2-6.el8.noarch 77/134 2023-11-24T07:54:17.673 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : python3-pycparser-2.14-14.el8.noarch 43/134 2023-11-24T07:54:17.845 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : python3-jaraco-functools-2.0-4.el8.noarch 78/134 2023-11-24T07:54:18.012 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : python3-cffi-1.11.5-5.el8.x86_64 44/134 2023-11-24T07:54:18.103 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : python3-cheroot-8.5.2-1.el8.noarch 79/134 2023-11-24T07:54:18.305 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : python3-tempora-1.14.1-5.el8.noarch 80/134 2023-11-24T07:54:18.375 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : python3-cryptography-3.2.1-5.el8.x86_64 45/134 2023-11-24T07:54:18.566 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : python3-pyOpenSSL-19.0.0-1.el8.noarch 46/134 2023-11-24T07:54:18.603 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : python3-portend-2.6-1.el8.noarch 81/134 2023-11-24T07:54:18.826 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : python3-jwt-1.6.1-2.el8.noarch 47/134 2023-11-24T07:54:18.856 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : python3-cherrypy-18.4.0-1.el8.noarch 82/134 2023-11-24T07:54:19.039 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : python3-certifi-2018.10.15-7.el8.noarch 83/134 2023-11-24T07:54:19.061 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : python3-oauthlib-2.1.0-1.el8.noarch 48/134 2023-11-24T07:54:19.257 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : python3-cachetools-3.1.1-4.el8.noarch 84/134 2023-11-24T07:54:19.390 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : python3-requests-oauthlib-1.0.0-1.el8.noarch 49/134 2023-11-24T07:54:19.670 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : python3-asyncssh-2.7.0-2.el8.noarch 50/134 2023-11-24T07:54:19.900 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : python3-trustme-0.6.0-4.el8.noarch 51/134 2023-11-24T07:54:20.174 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : python3-bcrypt-3.1.6-2.el8.1.x86_64 52/134 2023-11-24T07:54:20.358 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : python3-google-auth-1:1.1.1-10.el8.noarch 85/134 2023-11-24T07:54:20.370 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : libsepol-devel-2.9-3.el8.x86_64 53/134 2023-11-24T07:54:20.579 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : keyutils-libs-devel-1.5.10-9.el8.x86_64 54/134 2023-11-24T07:54:20.620 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : python3-kubernetes-1:11.0.0-6.el8.noarch 86/134 2023-11-24T07:54:20.882 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 87/134 2023-11-24T07:54:20.893 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : pcre2-utf32-10.32-2.el8.x86_64 55/134 2023-11-24T07:54:21.184 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : pcre2-devel-10.32-2.el8.x86_64 56/134 2023-11-24T07:54:21.193 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : python3-webtest-2.0.33-1.el8.noarch 88/134 2023-11-24T07:54:21.452 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : python3-pecan-1.3.2-9.el8.noarch 89/134 2023-11-24T07:54:21.461 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : libselinux-devel-2.9-5.el8.x86_64 57/134 2023-11-24T07:54:21.646 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : libunwind-1.3.1-3.el8.x86_64 90/134 2023-11-24T07:54:21.719 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : krb5-devel-1.18.2-14.el8.x86_64 58/134 2023-11-24T07:54:21.920 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : python3-jsonpointer-1.10-11.el8.noarch 59/134 2023-11-24T07:54:22.091 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : python3-jsonpatch-1.21-2.el8.noarch 60/134 2023-11-24T07:54:22.286 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : uuid-1.6.2-43.el8.x86_64 61/134 2023-11-24T07:54:22.376 INFO:teuthology.orchestra.run.smithi059.stdout: Running scriptlet: uuid-1.6.2-43.el8.x86_64 61/134 2023-11-24T07:54:22.499 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : gperftools-libs-1:2.7-9.el8.x86_64 91/134 2023-11-24T07:54:22.621 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : python3-prettytable-0.7.2-14.el8.noarch 62/134 2023-11-24T07:54:22.707 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : librgw2-2:18.0.0-7484.ga13ea5ff.el8.x86_64 92/134 2023-11-24T07:54:22.784 INFO:teuthology.orchestra.run.smithi076.stdout: Running scriptlet: librgw2-2:18.0.0-7484.ga13ea5ff.el8.x86_64 92/134 2023-11-24T07:54:22.850 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : automake-1.16.1-7.el8.noarch 63/134 2023-11-24T07:54:22.957 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : python3-rgw-2:18.0.0-7484.ga13ea5ff.el8.x86_64 93/134 2023-11-24T07:54:23.051 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : oniguruma-6.8.2-2.el8.x86_64 64/134 2023-11-24T07:54:23.138 INFO:teuthology.orchestra.run.smithi059.stdout: Running scriptlet: oniguruma-6.8.2-2.el8.x86_64 64/134 2023-11-24T07:54:23.403 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : jq-1.6-3.el8.x86_64 65/134 2023-11-24T07:54:23.614 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : socat-1.7.4.1-1.el8.x86_64 66/134 2023-11-24T07:54:23.787 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : xmlstarlet-1.6.1-20.el8.x86_64 67/134 2023-11-24T07:54:23.961 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : python3-zc-lockfile-2.0-2.el8.noarch 68/134 2023-11-24T07:54:24.261 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : python3-websocket-client-0.56.0-5.el8.noarch 69/134 2023-11-24T07:54:24.469 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : python3-waitress-1.4.3-1.el8.noarch 70/134 2023-11-24T07:54:24.667 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : python3-singledispatch-3.4.0.3-18.el8.noarch 71/134 2023-11-24T07:54:24.866 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : python3-simplegeneric-0.8.1-17.el8.noarch 72/134 2023-11-24T07:54:25.099 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : python3-repoze-lru-0.7-6.el8.noarch 73/134 2023-11-24T07:54:25.359 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : python3-routes-2.4.1-12.el8.noarch 74/134 2023-11-24T07:54:25.575 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : python3-natsort-7.1.1-2.el8.noarch 75/134 2023-11-24T07:54:25.764 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : python3-logutils-0.3.5-11.el8.noarch 76/134 2023-11-24T07:54:25.943 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : python3-jaraco-6.2-6.el8.noarch 77/134 2023-11-24T07:54:26.195 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : python3-jaraco-functools-2.0-4.el8.noarch 78/134 2023-11-24T07:54:26.433 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : python3-cheroot-8.5.2-1.el8.noarch 79/134 2023-11-24T07:54:26.642 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : python3-tempora-1.14.1-5.el8.noarch 80/134 2023-11-24T07:54:26.663 INFO:teuthology.orchestra.run.smithi076.stdout: Running scriptlet: ceph-common-2:18.0.0-7484.ga13ea5ff.el8.x86_64 94/134 2023-11-24T07:54:26.915 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : python3-portend-2.6-1.el8.noarch 81/134 2023-11-24T07:54:26.923 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : ceph-common-2:18.0.0-7484.ga13ea5ff.el8.x86_64 94/134 2023-11-24T07:54:27.220 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : python3-cherrypy-18.4.0-1.el8.noarch 82/134 2023-11-24T07:54:27.395 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : python3-certifi-2018.10.15-7.el8.noarch 83/134 2023-11-24T07:54:27.506 INFO:teuthology.orchestra.run.smithi076.stdout: Running scriptlet: ceph-common-2:18.0.0-7484.ga13ea5ff.el8.x86_64 94/134 2023-11-24T07:54:27.651 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : python3-cachetools-3.1.1-4.el8.noarch 84/134 2023-11-24T07:54:27.819 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : ceph-base-2:18.0.0-7484.ga13ea5ff.el8.x86_64 95/134 2023-11-24T07:54:27.888 INFO:teuthology.orchestra.run.smithi076.stdout: Running scriptlet: ceph-base-2:18.0.0-7484.ga13ea5ff.el8.x86_64 95/134 2023-11-24T07:54:28.040 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : ceph-selinux-2:18.0.0-7484.ga13ea5ff.el8.x86_64 96/134 2023-11-24T07:54:28.814 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : python3-google-auth-1:1.1.1-10.el8.noarch 85/134 2023-11-24T07:54:29.109 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : python3-kubernetes-1:11.0.0-6.el8.noarch 86/134 2023-11-24T07:54:29.400 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 87/134 2023-11-24T07:54:29.705 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : python3-webtest-2.0.33-1.el8.noarch 88/134 2023-11-24T07:54:30.018 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : python3-pecan-1.3.2-9.el8.noarch 89/134 2023-11-24T07:54:30.235 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : libunwind-1.3.1-3.el8.x86_64 90/134 2023-11-24T07:54:31.136 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : gperftools-libs-1:2.7-9.el8.x86_64 91/134 2023-11-24T07:54:31.336 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : librgw2-2:18.0.0-7484.ga13ea5ff.el8.x86_64 92/134 2023-11-24T07:54:31.415 INFO:teuthology.orchestra.run.smithi059.stdout: Running scriptlet: librgw2-2:18.0.0-7484.ga13ea5ff.el8.x86_64 92/134 2023-11-24T07:54:31.594 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : python3-rgw-2:18.0.0-7484.ga13ea5ff.el8.x86_64 93/134 2023-11-24T07:54:35.563 INFO:teuthology.orchestra.run.smithi059.stdout: Running scriptlet: ceph-common-2:18.0.0-7484.ga13ea5ff.el8.x86_64 94/134 2023-11-24T07:54:35.810 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : ceph-common-2:18.0.0-7484.ga13ea5ff.el8.x86_64 94/134 2023-11-24T07:54:36.501 INFO:teuthology.orchestra.run.smithi059.stdout: Running scriptlet: ceph-common-2:18.0.0-7484.ga13ea5ff.el8.x86_64 94/134 2023-11-24T07:54:36.907 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : ceph-base-2:18.0.0-7484.ga13ea5ff.el8.x86_64 95/134 2023-11-24T07:54:37.020 INFO:teuthology.orchestra.run.smithi059.stdout: Running scriptlet: ceph-base-2:18.0.0-7484.ga13ea5ff.el8.x86_64 95/134 2023-11-24T07:54:37.210 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : ceph-selinux-2:18.0.0-7484.ga13ea5ff.el8.x86_64 96/134 2023-11-24T07:54:46.647 INFO:teuthology.orchestra.run.smithi076.stdout: Running scriptlet: ceph-selinux-2:18.0.0-7484.ga13ea5ff.el8.x86_64 96/134 2023-11-24T07:54:46.648 INFO:teuthology.orchestra.run.smithi076.stdout:skipping the directory /sys 2023-11-24T07:54:46.648 INFO:teuthology.orchestra.run.smithi076.stdout:skipping the directory /proc 2023-11-24T07:54:46.648 INFO:teuthology.orchestra.run.smithi076.stdout:skipping the directory /dev 2023-11-24T07:54:46.648 INFO:teuthology.orchestra.run.smithi076.stdout:skipping the directory /run 2023-11-24T07:54:46.648 INFO:teuthology.orchestra.run.smithi076.stdout:skipping the directory /mnt 2023-11-24T07:54:46.648 INFO:teuthology.orchestra.run.smithi076.stdout:skipping the directory /var/tmp 2023-11-24T07:54:46.648 INFO:teuthology.orchestra.run.smithi076.stdout:skipping the directory /home 2023-11-24T07:54:46.648 INFO:teuthology.orchestra.run.smithi076.stdout:skipping the directory /tmp 2023-11-24T07:54:46.648 INFO:teuthology.orchestra.run.smithi076.stdout:skipping the directory /dev 2023-11-24T07:54:46.648 INFO:teuthology.orchestra.run.smithi076.stdout: 2023-11-24T07:54:46.996 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : ceph-volume-2:18.0.0-7484.ga13ea5ff.el8.noarch 97/134 2023-11-24T07:54:48.596 INFO:teuthology.orchestra.run.smithi076.stdout: Running scriptlet: ceph-volume-2:18.0.0-7484.ga13ea5ff.el8.noarch 97/134 2023-11-24T07:54:48.816 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : ceph-osd-2:18.0.0-7484.ga13ea5ff.el8.x86_64 98/134 2023-11-24T07:54:49.042 INFO:teuthology.orchestra.run.smithi076.stdout: Running scriptlet: ceph-osd-2:18.0.0-7484.ga13ea5ff.el8.x86_64 98/134 2023-11-24T07:54:49.204 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : ceph-mds-2:18.0.0-7484.ga13ea5ff.el8.x86_64 99/134 2023-11-24T07:54:49.661 INFO:teuthology.orchestra.run.smithi076.stdout: Running scriptlet: ceph-mds-2:18.0.0-7484.ga13ea5ff.el8.x86_64 99/134 2023-11-24T07:54:49.853 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : ceph-mon-2:18.0.0-7484.ga13ea5ff.el8.x86_64 100/134 2023-11-24T07:54:49.870 INFO:teuthology.orchestra.run.smithi076.stdout: Running scriptlet: ceph-mon-2:18.0.0-7484.ga13ea5ff.el8.x86_64 100/134 2023-11-24T07:54:51.577 INFO:teuthology.orchestra.run.smithi076.stdout: Running scriptlet: cephadm-2:18.0.0-7484.ga13ea5ff.el8.noarch 101/134 2023-11-24T07:54:51.785 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : cephadm-2:18.0.0-7484.ga13ea5ff.el8.noarch 101/134 2023-11-24T07:54:51.903 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : ceph-prometheus-alerts-2:18.0.0-7484.ga13ea5ff.e 102/134 2023-11-24T07:54:52.078 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : ceph-grafana-dashboards-2:18.0.0-7484.ga13ea5ff. 103/134 2023-11-24T07:54:52.286 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : ceph-mgr-cephadm-2:18.0.0-7484.ga13ea5ff.el8.noa 104/134 2023-11-24T07:54:52.509 INFO:teuthology.orchestra.run.smithi076.stdout: Running scriptlet: ceph-mgr-cephadm-2:18.0.0-7484.ga13ea5ff.el8.noa 104/134 2023-11-24T07:54:52.823 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : ceph-mgr-dashboard-2:18.0.0-7484.ga13ea5ff.el8.n 105/134 2023-11-24T07:54:53.785 INFO:teuthology.orchestra.run.smithi076.stdout: Running scriptlet: ceph-mgr-dashboard-2:18.0.0-7484.ga13ea5ff.el8.n 105/134 2023-11-24T07:54:53.961 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : ceph-mgr-diskprediction-local-2:18.0.0-7484.ga13 106/134 2023-11-24T07:54:54.052 INFO:teuthology.orchestra.run.smithi076.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:18.0.0-7484.ga13 106/134 2023-11-24T07:54:54.416 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : ceph-mgr-modules-core-2:18.0.0-7484.ga13ea5ff.el 107/134 2023-11-24T07:54:54.615 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : ceph-mgr-2:18.0.0-7484.ga13ea5ff.el8.x86_64 108/134 2023-11-24T07:54:54.651 INFO:teuthology.orchestra.run.smithi076.stdout: Running scriptlet: ceph-mgr-2:18.0.0-7484.ga13ea5ff.el8.x86_64 108/134 2023-11-24T07:54:54.869 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : ceph-mgr-rook-2:18.0.0-7484.ga13ea5ff.el8.noarch 109/134 2023-11-24T07:54:54.880 INFO:teuthology.orchestra.run.smithi076.stdout: Running scriptlet: ceph-mgr-rook-2:18.0.0-7484.ga13ea5ff.el8.noarch 109/134 2023-11-24T07:54:56.209 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : ceph-2:18.0.0-7484.ga13ea5ff.el8.x86_64 110/134 2023-11-24T07:54:56.489 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : ceph-radosgw-2:18.0.0-7484.ga13ea5ff.el8.x86_64 111/134 2023-11-24T07:54:56.523 INFO:teuthology.orchestra.run.smithi076.stdout: Running scriptlet: ceph-radosgw-2:18.0.0-7484.ga13ea5ff.el8.x86_64 111/134 2023-11-24T07:54:56.679 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : ceph-immutable-object-cache-2:18.0.0-7484.ga13ea 112/134 2023-11-24T07:54:56.687 INFO:teuthology.orchestra.run.smithi059.stdout: Running scriptlet: ceph-selinux-2:18.0.0-7484.ga13ea5ff.el8.x86_64 96/134 2023-11-24T07:54:56.687 INFO:teuthology.orchestra.run.smithi059.stdout:skipping the directory /sys 2023-11-24T07:54:56.687 INFO:teuthology.orchestra.run.smithi059.stdout:skipping the directory /proc 2023-11-24T07:54:56.687 INFO:teuthology.orchestra.run.smithi059.stdout:skipping the directory /dev 2023-11-24T07:54:56.687 INFO:teuthology.orchestra.run.smithi059.stdout:skipping the directory /run 2023-11-24T07:54:56.687 INFO:teuthology.orchestra.run.smithi059.stdout:skipping the directory /mnt 2023-11-24T07:54:56.687 INFO:teuthology.orchestra.run.smithi059.stdout:skipping the directory /var/tmp 2023-11-24T07:54:56.687 INFO:teuthology.orchestra.run.smithi059.stdout:skipping the directory /home 2023-11-24T07:54:56.687 INFO:teuthology.orchestra.run.smithi059.stdout:skipping the directory /tmp 2023-11-24T07:54:56.687 INFO:teuthology.orchestra.run.smithi059.stdout:skipping the directory /dev 2023-11-24T07:54:56.687 INFO:teuthology.orchestra.run.smithi059.stdout: 2023-11-24T07:54:56.719 INFO:teuthology.orchestra.run.smithi076.stdout: Running scriptlet: ceph-immutable-object-cache-2:18.0.0-7484.ga13ea 112/134 2023-11-24T07:54:56.869 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : cephfs-mirror-2:18.0.0-7484.ga13ea5ff.el8.x86_64 113/134 2023-11-24T07:54:57.040 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : ceph-volume-2:18.0.0-7484.ga13ea5ff.el8.noarch 97/134 2023-11-24T07:54:57.228 INFO:teuthology.orchestra.run.smithi076.stdout: Running scriptlet: cephfs-mirror-2:18.0.0-7484.ga13ea5ff.el8.x86_64 113/134 2023-11-24T07:54:57.427 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : rbd-mirror-2:18.0.0-7484.ga13ea5ff.el8.x86_64 114/134 2023-11-24T07:54:58.761 INFO:teuthology.orchestra.run.smithi059.stdout: Running scriptlet: ceph-volume-2:18.0.0-7484.ga13ea5ff.el8.noarch 97/134 2023-11-24T07:54:58.970 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : ceph-osd-2:18.0.0-7484.ga13ea5ff.el8.x86_64 98/134 2023-11-24T07:54:59.219 INFO:teuthology.orchestra.run.smithi059.stdout: Running scriptlet: ceph-osd-2:18.0.0-7484.ga13ea5ff.el8.x86_64 98/134 2023-11-24T07:54:59.374 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : ceph-mds-2:18.0.0-7484.ga13ea5ff.el8.x86_64 99/134 2023-11-24T07:54:59.860 INFO:teuthology.orchestra.run.smithi059.stdout: Running scriptlet: ceph-mds-2:18.0.0-7484.ga13ea5ff.el8.x86_64 99/134 2023-11-24T07:55:00.082 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : ceph-mon-2:18.0.0-7484.ga13ea5ff.el8.x86_64 100/134 2023-11-24T07:55:00.099 INFO:teuthology.orchestra.run.smithi059.stdout: Running scriptlet: ceph-mon-2:18.0.0-7484.ga13ea5ff.el8.x86_64 100/134 2023-11-24T07:55:02.062 INFO:teuthology.orchestra.run.smithi059.stdout: Running scriptlet: cephadm-2:18.0.0-7484.ga13ea5ff.el8.noarch 101/134 2023-11-24T07:55:02.100 INFO:teuthology.orchestra.run.smithi076.stdout: Running scriptlet: rbd-mirror-2:18.0.0-7484.ga13ea5ff.el8.x86_64 114/134 2023-11-24T07:55:02.281 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : cephadm-2:18.0.0-7484.ga13ea5ff.el8.noarch 101/134 2023-11-24T07:55:02.421 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : ceph-prometheus-alerts-2:18.0.0-7484.ga13ea5ff.e 102/134 2023-11-24T07:55:02.607 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : ceph-grafana-dashboards-2:18.0.0-7484.ga13ea5ff. 103/134 2023-11-24T07:55:02.806 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : ceph-mgr-cephadm-2:18.0.0-7484.ga13ea5ff.el8.noa 104/134 2023-11-24T07:55:03.033 INFO:teuthology.orchestra.run.smithi059.stdout: Running scriptlet: ceph-mgr-cephadm-2:18.0.0-7484.ga13ea5ff.el8.noa 104/134 2023-11-24T07:55:03.038 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : ceph-test-2:18.0.0-7484.ga13ea5ff.el8.x86_64 115/134 2023-11-24T07:55:03.303 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : ceph-mgr-dashboard-2:18.0.0-7484.ga13ea5ff.el8.n 105/134 2023-11-24T07:55:03.381 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : openssl-devel-1:1.1.1k-7.el8_6.x86_64 116/134 2023-11-24T07:55:03.564 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : ceph-fuse-2:18.0.0-7484.ga13ea5ff.el8.x86_64 117/134 2023-11-24T07:55:03.818 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : xfsprogs-devel-5.0.0-10.el8.x86_64 118/134 2023-11-24T07:55:04.001 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : libtool-2.4.6-25.el8.x86_64 119/134 2023-11-24T07:55:04.198 INFO:teuthology.orchestra.run.smithi076.stdout: Running scriptlet: libtool-2.4.6-25.el8.x86_64 119/134 2023-11-24T07:55:04.326 INFO:teuthology.orchestra.run.smithi059.stdout: Running scriptlet: ceph-mgr-dashboard-2:18.0.0-7484.ga13ea5ff.el8.n 105/134 2023-11-24T07:55:04.456 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : postgresql-contrib-10.21-2.module+el8.6.0+15342+ 120/134 2023-11-24T07:55:04.497 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : ceph-mgr-diskprediction-local-2:18.0.0-7484.ga13 106/134 2023-11-24T07:55:04.594 INFO:teuthology.orchestra.run.smithi059.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:18.0.0-7484.ga13 106/134 2023-11-24T07:55:04.639 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : libcephfs-devel-2:18.0.0-7484.ga13ea5ff.el8.x86_ 121/134 2023-11-24T07:55:04.828 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : elfutils-libelf-devel-0.186-1.el8.x86_64 122/134 2023-11-24T07:55:04.989 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : ceph-mgr-modules-core-2:18.0.0-7484.ga13ea5ff.el 107/134 2023-11-24T07:55:05.203 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : ceph-mgr-2:18.0.0-7484.ga13ea5ff.el8.x86_64 108/134 2023-11-24T07:55:05.241 INFO:teuthology.orchestra.run.smithi059.stdout: Running scriptlet: ceph-mgr-2:18.0.0-7484.ga13ea5ff.el8.x86_64 108/134 2023-11-24T07:55:05.422 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : ceph-mgr-rook-2:18.0.0-7484.ga13ea5ff.el8.noarch 109/134 2023-11-24T07:55:05.434 INFO:teuthology.orchestra.run.smithi059.stdout: Running scriptlet: ceph-mgr-rook-2:18.0.0-7484.ga13ea5ff.el8.noarch 109/134 2023-11-24T07:55:06.887 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : ceph-2:18.0.0-7484.ga13ea5ff.el8.x86_64 110/134 2023-11-24T07:55:07.035 INFO:teuthology.orchestra.run.smithi076.stdout: Running scriptlet: postgresql-server-10.21-2.module+el8.6.0+15342+5 123/134 2023-11-24T07:55:07.128 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : ceph-radosgw-2:18.0.0-7484.ga13ea5ff.el8.x86_64 111/134 2023-11-24T07:55:07.164 INFO:teuthology.orchestra.run.smithi059.stdout: Running scriptlet: ceph-radosgw-2:18.0.0-7484.ga13ea5ff.el8.x86_64 111/134 2023-11-24T07:55:07.302 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : postgresql-server-10.21-2.module+el8.6.0+15342+5 123/134 2023-11-24T07:55:07.318 INFO:teuthology.orchestra.run.smithi076.stdout: Running scriptlet: postgresql-server-10.21-2.module+el8.6.0+15342+5 123/134 2023-11-24T07:55:07.339 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : ceph-immutable-object-cache-2:18.0.0-7484.ga13ea 112/134 2023-11-24T07:55:07.383 INFO:teuthology.orchestra.run.smithi059.stdout: Running scriptlet: ceph-immutable-object-cache-2:18.0.0-7484.ga13ea 112/134 2023-11-24T07:55:07.478 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : cephfs-top-2:18.0.0-7484.ga13ea5ff.el8.noarch 124/134 2023-11-24T07:55:07.564 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : cephfs-mirror-2:18.0.0-7484.ga13ea5ff.el8.x86_64 113/134 2023-11-24T07:55:07.759 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : rbd-fuse-2:18.0.0-7484.ga13ea5ff.el8.x86_64 125/134 2023-11-24T07:55:07.947 INFO:teuthology.orchestra.run.smithi059.stdout: Running scriptlet: cephfs-mirror-2:18.0.0-7484.ga13ea5ff.el8.x86_64 113/134 2023-11-24T07:55:08.069 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : rbd-nbd-2:18.0.0-7484.ga13ea5ff.el8.x86_64 126/134 2023-11-24T07:55:08.122 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : rbd-mirror-2:18.0.0-7484.ga13ea5ff.el8.x86_64 114/134 2023-11-24T07:55:08.463 INFO:teuthology.orchestra.run.smithi076.stdout: Upgrading : iproute-5.15.0-4.el8_6.1.x86_64 127/134 2023-11-24T07:55:08.634 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : flex-2.6.1-9.el8.x86_64 128/134 2023-11-24T07:55:08.718 INFO:teuthology.orchestra.run.smithi076.stdout: Running scriptlet: flex-2.6.1-9.el8.x86_64 128/134 2023-11-24T07:55:08.967 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : bison-3.0.4-10.el8.x86_64 129/134 2023-11-24T07:55:08.980 INFO:teuthology.orchestra.run.smithi076.stdout: Running scriptlet: bison-3.0.4-10.el8.x86_64 129/134 2023-11-24T07:55:09.134 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : NetworkManager-initscripts-updown-1:1.36.0-9.el8 130/134 2023-11-24T07:55:09.142 INFO:teuthology.orchestra.run.smithi076.stdout: Running scriptlet: NetworkManager-initscripts-updown-1:1.36.0-9.el8 130/134 2023-11-24T07:55:09.278 INFO:teuthology.orchestra.run.smithi076.stdout: Installing : libaio-devel-0.3.112-1.el8.x86_64 131/134 2023-11-24T07:55:09.278 INFO:teuthology.orchestra.run.smithi076.stdout: Cleanup : librbd1-1:12.2.7-9.el8.x86_64 132/134 2023-11-24T07:55:09.487 INFO:teuthology.orchestra.run.smithi076.stdout: Running scriptlet: librbd1-1:12.2.7-9.el8.x86_64 132/134 2023-11-24T07:55:09.487 INFO:teuthology.orchestra.run.smithi076.stdout: Cleanup : librados2-1:12.2.7-9.el8.x86_64 133/134 2023-11-24T07:55:09.748 INFO:teuthology.orchestra.run.smithi076.stdout: Running scriptlet: librados2-1:12.2.7-9.el8.x86_64 133/134 2023-11-24T07:55:09.941 INFO:teuthology.orchestra.run.smithi076.stdout: Cleanup : iproute-5.15.0-4.el8.x86_64 134/134 2023-11-24T07:55:10.455 INFO:teuthology.orchestra.run.smithi076.stdout: Running scriptlet: iproute-5.15.0-4.el8.x86_64 134/134 2023-11-24T07:55:10.455 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : ceph-2:18.0.0-7484.ga13ea5ff.el8.x86_64 1/134 2023-11-24T07:55:10.455 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : ceph-base-2:18.0.0-7484.ga13ea5ff.el8.x86_64 2/134 2023-11-24T07:55:10.455 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : ceph-common-2:18.0.0-7484.ga13ea5ff.el8.x86_64 3/134 2023-11-24T07:55:10.455 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : ceph-fuse-2:18.0.0-7484.ga13ea5ff.el8.x86_64 4/134 2023-11-24T07:55:10.455 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : ceph-immutable-object-cache-2:18.0.0-7484.ga13ea 5/134 2023-11-24T07:55:10.455 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : ceph-mds-2:18.0.0-7484.ga13ea5ff.el8.x86_64 6/134 2023-11-24T07:55:10.456 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : ceph-mgr-2:18.0.0-7484.ga13ea5ff.el8.x86_64 7/134 2023-11-24T07:55:10.456 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : ceph-mon-2:18.0.0-7484.ga13ea5ff.el8.x86_64 8/134 2023-11-24T07:55:10.456 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : ceph-osd-2:18.0.0-7484.ga13ea5ff.el8.x86_64 9/134 2023-11-24T07:55:10.456 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : ceph-radosgw-2:18.0.0-7484.ga13ea5ff.el8.x86_64 10/134 2023-11-24T07:55:10.456 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : ceph-selinux-2:18.0.0-7484.ga13ea5ff.el8.x86_64 11/134 2023-11-24T07:55:10.456 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : ceph-test-2:18.0.0-7484.ga13ea5ff.el8.x86_64 12/134 2023-11-24T07:55:10.456 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : cephfs-mirror-2:18.0.0-7484.ga13ea5ff.el8.x86_64 13/134 2023-11-24T07:55:10.456 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : libcephfs-devel-2:18.0.0-7484.ga13ea5ff.el8.x86_ 14/134 2023-11-24T07:55:10.456 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : libcephfs2-2:18.0.0-7484.ga13ea5ff.el8.x86_64 15/134 2023-11-24T07:55:10.456 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : libcephsqlite-2:18.0.0-7484.ga13ea5ff.el8.x86_64 16/134 2023-11-24T07:55:10.456 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : librados-devel-2:18.0.0-7484.ga13ea5ff.el8.x86_6 17/134 2023-11-24T07:55:10.456 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : libradosstriper1-2:18.0.0-7484.ga13ea5ff.el8.x86 18/134 2023-11-24T07:55:10.456 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : librgw2-2:18.0.0-7484.ga13ea5ff.el8.x86_64 19/134 2023-11-24T07:55:10.457 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : python3-ceph-argparse-2:18.0.0-7484.ga13ea5ff.el 20/134 2023-11-24T07:55:10.457 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : python3-ceph-common-2:18.0.0-7484.ga13ea5ff.el8. 21/134 2023-11-24T07:55:10.457 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : python3-cephfs-2:18.0.0-7484.ga13ea5ff.el8.x86_6 22/134 2023-11-24T07:55:10.457 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : python3-rados-2:18.0.0-7484.ga13ea5ff.el8.x86_64 23/134 2023-11-24T07:55:10.457 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : python3-rbd-2:18.0.0-7484.ga13ea5ff.el8.x86_64 24/134 2023-11-24T07:55:10.457 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : python3-rgw-2:18.0.0-7484.ga13ea5ff.el8.x86_64 25/134 2023-11-24T07:55:10.457 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : rbd-fuse-2:18.0.0-7484.ga13ea5ff.el8.x86_64 26/134 2023-11-24T07:55:10.457 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : rbd-mirror-2:18.0.0-7484.ga13ea5ff.el8.x86_64 27/134 2023-11-24T07:55:10.457 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : rbd-nbd-2:18.0.0-7484.ga13ea5ff.el8.x86_64 28/134 2023-11-24T07:55:10.457 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : ceph-grafana-dashboards-2:18.0.0-7484.ga13ea5ff. 29/134 2023-11-24T07:55:10.457 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : ceph-mgr-cephadm-2:18.0.0-7484.ga13ea5ff.el8.noa 30/134 2023-11-24T07:55:10.457 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : ceph-mgr-dashboard-2:18.0.0-7484.ga13ea5ff.el8.n 31/134 2023-11-24T07:55:10.457 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : ceph-mgr-diskprediction-local-2:18.0.0-7484.ga13 32/134 2023-11-24T07:55:10.457 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : ceph-mgr-modules-core-2:18.0.0-7484.ga13ea5ff.el 33/134 2023-11-24T07:55:10.458 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : ceph-mgr-rook-2:18.0.0-7484.ga13ea5ff.el8.noarch 34/134 2023-11-24T07:55:10.458 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : ceph-prometheus-alerts-2:18.0.0-7484.ga13ea5ff.e 35/134 2023-11-24T07:55:10.458 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : ceph-volume-2:18.0.0-7484.ga13ea5ff.el8.noarch 36/134 2023-11-24T07:55:10.458 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : cephadm-2:18.0.0-7484.ga13ea5ff.el8.noarch 37/134 2023-11-24T07:55:10.458 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : cephfs-top-2:18.0.0-7484.ga13ea5ff.el8.noarch 38/134 2023-11-24T07:55:10.458 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : gperftools-libs-1:2.7-9.el8.x86_64 39/134 2023-11-24T07:55:10.458 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : liboath-2.6.2-3.el8.x86_64 40/134 2023-11-24T07:55:10.458 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : libunwind-1.3.1-3.el8.x86_64 41/134 2023-11-24T07:55:10.458 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : python3-asyncssh-2.7.0-2.el8.noarch 42/134 2023-11-24T07:55:10.458 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : python3-bcrypt-3.1.6-2.el8.1.x86_64 43/134 2023-11-24T07:55:10.458 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 44/134 2023-11-24T07:55:10.459 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : python3-cachetools-3.1.1-4.el8.noarch 45/134 2023-11-24T07:55:10.459 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : python3-certifi-2018.10.15-7.el8.noarch 46/134 2023-11-24T07:55:10.459 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : python3-cheroot-8.5.2-1.el8.noarch 47/134 2023-11-24T07:55:10.459 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : python3-cherrypy-18.4.0-1.el8.noarch 48/134 2023-11-24T07:55:10.459 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : python3-google-auth-1:1.1.1-10.el8.noarch 49/134 2023-11-24T07:55:10.459 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : python3-jaraco-6.2-6.el8.noarch 50/134 2023-11-24T07:55:10.460 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : python3-jaraco-functools-2.0-4.el8.noarch 51/134 2023-11-24T07:55:10.460 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : python3-kubernetes-1:11.0.0-6.el8.noarch 52/134 2023-11-24T07:55:10.460 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : python3-logutils-0.3.5-11.el8.noarch 53/134 2023-11-24T07:55:10.461 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : python3-more-itertools-7.2.0-3.el8.noarch 54/134 2023-11-24T07:55:10.461 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : python3-natsort-7.1.1-2.el8.noarch 55/134 2023-11-24T07:55:10.461 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : python3-pecan-1.3.2-9.el8.noarch 56/134 2023-11-24T07:55:10.461 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : python3-portend-2.6-1.el8.noarch 57/134 2023-11-24T07:55:10.461 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : python3-repoze-lru-0.7-6.el8.noarch 58/134 2023-11-24T07:55:10.462 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : python3-routes-2.4.1-12.el8.noarch 59/134 2023-11-24T07:55:10.462 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : python3-rsa-4.9-2.el8.noarch 60/134 2023-11-24T07:55:10.462 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : python3-simplegeneric-0.8.1-17.el8.noarch 61/134 2023-11-24T07:55:10.462 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : python3-singledispatch-3.4.0.3-18.el8.noarch 62/134 2023-11-24T07:55:10.462 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : python3-tempora-1.14.1-5.el8.noarch 63/134 2023-11-24T07:55:10.462 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : python3-trustme-0.6.0-4.el8.noarch 64/134 2023-11-24T07:55:10.463 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : python3-waitress-1.4.3-1.el8.noarch 65/134 2023-11-24T07:55:10.463 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : python3-webob-1.8.5-1.el8.1.noarch 66/134 2023-11-24T07:55:10.463 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : python3-websocket-client-0.56.0-5.el8.noarch 67/134 2023-11-24T07:55:10.463 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : python3-webtest-2.0.33-1.el8.noarch 68/134 2023-11-24T07:55:10.463 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : python3-zc-lockfile-2.0-2.el8.noarch 69/134 2023-11-24T07:55:10.463 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : thrift-0.13.0-2.el8.x86_64 70/134 2023-11-24T07:55:10.463 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : xmlstarlet-1.6.1-20.el8.x86_64 71/134 2023-11-24T07:55:10.464 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : librabbitmq-0.9.0-3.el8.x86_64 72/134 2023-11-24T07:55:10.464 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : libselinux-devel-2.9-5.el8.x86_64 73/134 2023-11-24T07:55:10.464 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : python3-jwt-1.6.1-2.el8.noarch 74/134 2023-11-24T07:55:10.464 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : pcre2-utf32-10.32-2.el8.x86_64 75/134 2023-11-24T07:55:10.465 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : keyutils-libs-devel-1.5.10-9.el8.x86_64 76/134 2023-11-24T07:55:10.465 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : libsepol-devel-2.9-3.el8.x86_64 77/134 2023-11-24T07:55:10.465 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : python3-requests-oauthlib-1.0.0-1.el8.noarch 78/134 2023-11-24T07:55:10.465 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : python3-pycparser-2.14-14.el8.noarch 79/134 2023-11-24T07:55:10.465 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : pcre2-utf16-10.32-2.el8.x86_64 80/134 2023-11-24T07:55:10.465 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : krb5-devel-1.18.2-14.el8.x86_64 81/134 2023-11-24T07:55:10.465 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : libaio-devel-0.3.112-1.el8.x86_64 82/134 2023-11-24T07:55:10.466 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : python3-oauthlib-2.1.0-1.el8.noarch 83/134 2023-11-24T07:55:10.466 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : python3-cryptography-3.2.1-5.el8.x86_64 84/134 2023-11-24T07:55:10.466 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : libverto-devel-0.3.0-5.el8.x86_64 85/134 2023-11-24T07:55:10.466 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : pcre2-devel-10.32-2.el8.x86_64 86/134 2023-11-24T07:55:10.466 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : python3-cffi-1.11.5-5.el8.x86_64 87/134 2023-11-24T07:55:10.466 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : openssl-devel-1:1.1.1k-7.el8_6.x86_64 88/134 2023-11-24T07:55:10.466 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : xfsprogs-devel-5.0.0-10.el8.x86_64 89/134 2023-11-24T07:55:10.467 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : fuse-2.9.7-15.el8.x86_64 90/134 2023-11-24T07:55:10.467 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : elfutils-libelf-devel-0.186-1.el8.x86_64 91/134 2023-11-24T07:55:10.467 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : libuuid-devel-2.32.1-35.el8.x86_64 92/134 2023-11-24T07:55:10.467 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : zlib-devel-1.2.11-18.el8_5.x86_64 93/134 2023-11-24T07:55:10.467 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : libcom_err-devel-1.45.6-4.el8.x86_64 94/134 2023-11-24T07:55:10.467 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : libgfortran-8.5.0-10.1.el8_6.x86_64 95/134 2023-11-24T07:55:10.467 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : libquadmath-8.5.0-10.1.el8_6.x86_64 96/134 2023-11-24T07:55:10.467 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : NetworkManager-initscripts-updown-1:1.36.0-9.el8 97/134 2023-11-24T07:55:10.468 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : python3-scipy-1.0.0-21.module+el8.5.0+10916+41bd 98/134 2023-11-24T07:55:10.468 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : python3-pyasn1-modules-0.3.7-6.el8.noarch 99/134 2023-11-24T07:55:10.468 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : python3-pytz-2017.2-9.el8.noarch 100/134 2023-11-24T07:55:10.468 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : lttng-ust-2.8.1-11.el8.x86_64 101/134 2023-11-24T07:55:10.468 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : socat-1.7.4.1-1.el8.x86_64 102/134 2023-11-24T07:55:10.468 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : python3-markupsafe-0.23-19.el8.x86_64 103/134 2023-11-24T07:55:10.468 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : bison-3.0.4-10.el8.x86_64 104/134 2023-11-24T07:55:10.468 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : python3-jinja2-2.10.1-3.el8.noarch 105/134 2023-11-24T07:55:10.468 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : python3-mako-1.0.6-13.el8.noarch 106/134 2023-11-24T07:55:10.468 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : oniguruma-6.8.2-2.el8.x86_64 107/134 2023-11-24T07:55:10.468 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : automake-1.16.1-7.el8.noarch 108/134 2023-11-24T07:55:10.469 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : python3-babel-2.5.1-7.el8.noarch 109/134 2023-11-24T07:55:10.469 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : python3-jsonpatch-1.21-2.el8.noarch 110/134 2023-11-24T07:55:10.469 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : flex-2.6.1-9.el8.x86_64 111/134 2023-11-24T07:55:10.469 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : python3-prettytable-0.7.2-14.el8.noarch 112/134 2023-11-24T07:55:10.469 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : python3-numpy-1:1.14.3-10.el8.x86_64 113/134 2023-11-24T07:55:10.469 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : python3-werkzeug-0.12.2-4.el8.noarch 114/134 2023-11-24T07:55:10.469 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : python3-numpy-f2py-1:1.14.3-10.el8.x86_64 115/134 2023-11-24T07:55:10.469 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : uuid-1.6.2-43.el8.x86_64 116/134 2023-11-24T07:55:10.469 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : python3-pyasn1-0.3.7-6.el8.noarch 117/134 2023-11-24T07:55:10.469 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : python3-jsonpointer-1.10-11.el8.noarch 118/134 2023-11-24T07:55:10.469 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : python3-pyOpenSSL-19.0.0-1.el8.noarch 119/134 2023-11-24T07:55:10.469 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : libtool-2.4.6-25.el8.x86_64 120/134 2023-11-24T07:55:10.470 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : openblas-threads-0.3.15-3.el8.x86_64 121/134 2023-11-24T07:55:10.470 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : libpq-13.5-1.el8.x86_64 122/134 2023-11-24T07:55:10.470 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : openblas-0.3.15-3.el8.x86_64 123/134 2023-11-24T07:55:10.470 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : jq-1.6-3.el8.x86_64 124/134 2023-11-24T07:55:10.470 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : librdkafka-0.11.4-3.el8.x86_64 125/134 2023-11-24T07:55:10.470 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : postgresql-server-10.21-2.module+el8.6.0+15342+5 126/134 2023-11-24T07:55:10.470 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : postgresql-contrib-10.21-2.module+el8.6.0+15342+ 127/134 2023-11-24T07:55:10.470 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : postgresql-10.21-2.module+el8.6.0+15342+53518fac 128/134 2023-11-24T07:55:10.470 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : librados2-2:18.0.0-7484.ga13ea5ff.el8.x86_64 129/134 2023-11-24T07:55:10.470 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : librados2-1:12.2.7-9.el8.x86_64 130/134 2023-11-24T07:55:10.470 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : librbd1-2:18.0.0-7484.ga13ea5ff.el8.x86_64 131/134 2023-11-24T07:55:10.471 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : librbd1-1:12.2.7-9.el8.x86_64 132/134 2023-11-24T07:55:10.471 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : iproute-5.15.0-4.el8_6.1.x86_64 133/134 2023-11-24T07:55:13.114 INFO:teuthology.orchestra.run.smithi059.stdout: Running scriptlet: rbd-mirror-2:18.0.0-7484.ga13ea5ff.el8.x86_64 114/134 2023-11-24T07:55:13.158 INFO:teuthology.orchestra.run.smithi076.stdout: Verifying : iproute-5.15.0-4.el8.x86_64 134/134 2023-11-24T07:55:13.158 INFO:teuthology.orchestra.run.smithi076.stdout:Last metadata expiration check: 0:01:49 ago on Fri 24 Nov 2023 07:53:24 AM UTC. 2023-11-24T07:55:13.419 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : ceph-test-2:18.0.0-7484.ga13ea5ff.el8.x86_64 115/134 2023-11-24T07:55:13.604 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : libtool-2.4.6-25.el8.x86_64 116/134 2023-11-24T07:55:13.809 INFO:teuthology.orchestra.run.smithi059.stdout: Running scriptlet: libtool-2.4.6-25.el8.x86_64 116/134 2023-11-24T07:55:14.764 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : postgresql-contrib-10.21-2.module+el8.6.0+15342+ 117/134 2023-11-24T07:55:15.122 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : openssl-devel-1:1.1.1k-7.el8_6.x86_64 118/134 2023-11-24T07:55:15.328 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : ceph-fuse-2:18.0.0-7484.ga13ea5ff.el8.x86_64 119/134 2023-11-24T07:55:15.584 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : xfsprogs-devel-5.0.0-10.el8.x86_64 120/134 2023-11-24T07:55:15.793 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : libcephfs-devel-2:18.0.0-7484.ga13ea5ff.el8.x86_ 121/134 2023-11-24T07:55:15.859 INFO:teuthology.orchestra.run.smithi076.stdout:Installed products updated. 2023-11-24T07:55:15.939 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : elfutils-libelf-devel-0.186-1.el8.x86_64 122/134 2023-11-24T07:55:16.076 INFO:teuthology.orchestra.run.smithi076.stdout: 2023-11-24T07:55:16.077 INFO:teuthology.orchestra.run.smithi076.stdout:Upgraded: 2023-11-24T07:55:16.077 INFO:teuthology.orchestra.run.smithi076.stdout: iproute-5.15.0-4.el8_6.1.x86_64 2023-11-24T07:55:16.077 INFO:teuthology.orchestra.run.smithi076.stdout: librados2-2:18.0.0-7484.ga13ea5ff.el8.x86_64 2023-11-24T07:55:16.077 INFO:teuthology.orchestra.run.smithi076.stdout: librbd1-2:18.0.0-7484.ga13ea5ff.el8.x86_64 2023-11-24T07:55:16.077 INFO:teuthology.orchestra.run.smithi076.stdout:Installed: 2023-11-24T07:55:16.077 INFO:teuthology.orchestra.run.smithi076.stdout: NetworkManager-initscripts-updown-1:1.36.0-9.el8_6.noarch 2023-11-24T07:55:16.077 INFO:teuthology.orchestra.run.smithi076.stdout: automake-1.16.1-7.el8.noarch 2023-11-24T07:55:16.077 INFO:teuthology.orchestra.run.smithi076.stdout: bison-3.0.4-10.el8.x86_64 2023-11-24T07:55:16.078 INFO:teuthology.orchestra.run.smithi076.stdout: ceph-2:18.0.0-7484.ga13ea5ff.el8.x86_64 2023-11-24T07:55:16.078 INFO:teuthology.orchestra.run.smithi076.stdout: ceph-base-2:18.0.0-7484.ga13ea5ff.el8.x86_64 2023-11-24T07:55:16.078 INFO:teuthology.orchestra.run.smithi076.stdout: ceph-common-2:18.0.0-7484.ga13ea5ff.el8.x86_64 2023-11-24T07:55:16.078 INFO:teuthology.orchestra.run.smithi076.stdout: ceph-fuse-2:18.0.0-7484.ga13ea5ff.el8.x86_64 2023-11-24T07:55:16.078 INFO:teuthology.orchestra.run.smithi076.stdout: ceph-grafana-dashboards-2:18.0.0-7484.ga13ea5ff.el8.noarch 2023-11-24T07:55:16.078 INFO:teuthology.orchestra.run.smithi076.stdout: ceph-immutable-object-cache-2:18.0.0-7484.ga13ea5ff.el8.x86_64 2023-11-24T07:55:16.079 INFO:teuthology.orchestra.run.smithi076.stdout: ceph-mds-2:18.0.0-7484.ga13ea5ff.el8.x86_64 2023-11-24T07:55:16.079 INFO:teuthology.orchestra.run.smithi076.stdout: ceph-mgr-2:18.0.0-7484.ga13ea5ff.el8.x86_64 2023-11-24T07:55:16.079 INFO:teuthology.orchestra.run.smithi076.stdout: ceph-mgr-cephadm-2:18.0.0-7484.ga13ea5ff.el8.noarch 2023-11-24T07:55:16.079 INFO:teuthology.orchestra.run.smithi076.stdout: ceph-mgr-dashboard-2:18.0.0-7484.ga13ea5ff.el8.noarch 2023-11-24T07:55:16.079 INFO:teuthology.orchestra.run.smithi076.stdout: ceph-mgr-diskprediction-local-2:18.0.0-7484.ga13ea5ff.el8.noarch 2023-11-24T07:55:16.079 INFO:teuthology.orchestra.run.smithi076.stdout: ceph-mgr-modules-core-2:18.0.0-7484.ga13ea5ff.el8.noarch 2023-11-24T07:55:16.079 INFO:teuthology.orchestra.run.smithi076.stdout: ceph-mgr-rook-2:18.0.0-7484.ga13ea5ff.el8.noarch 2023-11-24T07:55:16.080 INFO:teuthology.orchestra.run.smithi076.stdout: ceph-mon-2:18.0.0-7484.ga13ea5ff.el8.x86_64 2023-11-24T07:55:16.080 INFO:teuthology.orchestra.run.smithi076.stdout: ceph-osd-2:18.0.0-7484.ga13ea5ff.el8.x86_64 2023-11-24T07:55:16.080 INFO:teuthology.orchestra.run.smithi076.stdout: ceph-prometheus-alerts-2:18.0.0-7484.ga13ea5ff.el8.noarch 2023-11-24T07:55:16.080 INFO:teuthology.orchestra.run.smithi076.stdout: ceph-radosgw-2:18.0.0-7484.ga13ea5ff.el8.x86_64 2023-11-24T07:55:16.080 INFO:teuthology.orchestra.run.smithi076.stdout: ceph-selinux-2:18.0.0-7484.ga13ea5ff.el8.x86_64 2023-11-24T07:55:16.080 INFO:teuthology.orchestra.run.smithi076.stdout: ceph-test-2:18.0.0-7484.ga13ea5ff.el8.x86_64 2023-11-24T07:55:16.080 INFO:teuthology.orchestra.run.smithi076.stdout: ceph-volume-2:18.0.0-7484.ga13ea5ff.el8.noarch 2023-11-24T07:55:16.081 INFO:teuthology.orchestra.run.smithi076.stdout: cephadm-2:18.0.0-7484.ga13ea5ff.el8.noarch 2023-11-24T07:55:16.081 INFO:teuthology.orchestra.run.smithi076.stdout: cephfs-mirror-2:18.0.0-7484.ga13ea5ff.el8.x86_64 2023-11-24T07:55:16.081 INFO:teuthology.orchestra.run.smithi076.stdout: cephfs-top-2:18.0.0-7484.ga13ea5ff.el8.noarch 2023-11-24T07:55:16.081 INFO:teuthology.orchestra.run.smithi076.stdout: elfutils-libelf-devel-0.186-1.el8.x86_64 2023-11-24T07:55:16.081 INFO:teuthology.orchestra.run.smithi076.stdout: flex-2.6.1-9.el8.x86_64 2023-11-24T07:55:16.081 INFO:teuthology.orchestra.run.smithi076.stdout: fuse-2.9.7-15.el8.x86_64 2023-11-24T07:55:16.081 INFO:teuthology.orchestra.run.smithi076.stdout: gperftools-libs-1:2.7-9.el8.x86_64 2023-11-24T07:55:16.082 INFO:teuthology.orchestra.run.smithi076.stdout: jq-1.6-3.el8.x86_64 2023-11-24T07:55:16.082 INFO:teuthology.orchestra.run.smithi076.stdout: keyutils-libs-devel-1.5.10-9.el8.x86_64 2023-11-24T07:55:16.082 INFO:teuthology.orchestra.run.smithi076.stdout: krb5-devel-1.18.2-14.el8.x86_64 2023-11-24T07:55:16.082 INFO:teuthology.orchestra.run.smithi076.stdout: libaio-devel-0.3.112-1.el8.x86_64 2023-11-24T07:55:16.082 INFO:teuthology.orchestra.run.smithi076.stdout: libcephfs-devel-2:18.0.0-7484.ga13ea5ff.el8.x86_64 2023-11-24T07:55:16.082 INFO:teuthology.orchestra.run.smithi076.stdout: libcephfs2-2:18.0.0-7484.ga13ea5ff.el8.x86_64 2023-11-24T07:55:16.082 INFO:teuthology.orchestra.run.smithi076.stdout: libcephsqlite-2:18.0.0-7484.ga13ea5ff.el8.x86_64 2023-11-24T07:55:16.083 INFO:teuthology.orchestra.run.smithi076.stdout: libcom_err-devel-1.45.6-4.el8.x86_64 2023-11-24T07:55:16.083 INFO:teuthology.orchestra.run.smithi076.stdout: libgfortran-8.5.0-10.1.el8_6.x86_64 2023-11-24T07:55:16.083 INFO:teuthology.orchestra.run.smithi076.stdout: liboath-2.6.2-3.el8.x86_64 2023-11-24T07:55:16.083 INFO:teuthology.orchestra.run.smithi076.stdout: libpq-13.5-1.el8.x86_64 2023-11-24T07:55:16.083 INFO:teuthology.orchestra.run.smithi076.stdout: libquadmath-8.5.0-10.1.el8_6.x86_64 2023-11-24T07:55:16.083 INFO:teuthology.orchestra.run.smithi076.stdout: librabbitmq-0.9.0-3.el8.x86_64 2023-11-24T07:55:16.083 INFO:teuthology.orchestra.run.smithi076.stdout: librados-devel-2:18.0.0-7484.ga13ea5ff.el8.x86_64 2023-11-24T07:55:16.083 INFO:teuthology.orchestra.run.smithi076.stdout: libradosstriper1-2:18.0.0-7484.ga13ea5ff.el8.x86_64 2023-11-24T07:55:16.083 INFO:teuthology.orchestra.run.smithi076.stdout: librdkafka-0.11.4-3.el8.x86_64 2023-11-24T07:55:16.085 INFO:teuthology.orchestra.run.smithi076.stdout: librgw2-2:18.0.0-7484.ga13ea5ff.el8.x86_64 2023-11-24T07:55:16.085 INFO:teuthology.orchestra.run.smithi076.stdout: libselinux-devel-2.9-5.el8.x86_64 2023-11-24T07:55:16.085 INFO:teuthology.orchestra.run.smithi076.stdout: libsepol-devel-2.9-3.el8.x86_64 2023-11-24T07:55:16.085 INFO:teuthology.orchestra.run.smithi076.stdout: libtool-2.4.6-25.el8.x86_64 2023-11-24T07:55:16.086 INFO:teuthology.orchestra.run.smithi076.stdout: libunwind-1.3.1-3.el8.x86_64 2023-11-24T07:55:16.086 INFO:teuthology.orchestra.run.smithi076.stdout: libuuid-devel-2.32.1-35.el8.x86_64 2023-11-24T07:55:16.086 INFO:teuthology.orchestra.run.smithi076.stdout: libverto-devel-0.3.0-5.el8.x86_64 2023-11-24T07:55:16.086 INFO:teuthology.orchestra.run.smithi076.stdout: lttng-ust-2.8.1-11.el8.x86_64 2023-11-24T07:55:16.086 INFO:teuthology.orchestra.run.smithi076.stdout: oniguruma-6.8.2-2.el8.x86_64 2023-11-24T07:55:16.086 INFO:teuthology.orchestra.run.smithi076.stdout: openblas-0.3.15-3.el8.x86_64 2023-11-24T07:55:16.086 INFO:teuthology.orchestra.run.smithi076.stdout: openblas-threads-0.3.15-3.el8.x86_64 2023-11-24T07:55:16.087 INFO:teuthology.orchestra.run.smithi076.stdout: openssl-devel-1:1.1.1k-7.el8_6.x86_64 2023-11-24T07:55:16.087 INFO:teuthology.orchestra.run.smithi076.stdout: pcre2-devel-10.32-2.el8.x86_64 2023-11-24T07:55:16.087 INFO:teuthology.orchestra.run.smithi076.stdout: pcre2-utf16-10.32-2.el8.x86_64 2023-11-24T07:55:16.087 INFO:teuthology.orchestra.run.smithi076.stdout: pcre2-utf32-10.32-2.el8.x86_64 2023-11-24T07:55:16.087 INFO:teuthology.orchestra.run.smithi076.stdout: postgresql-10.21-2.module+el8.6.0+15342+53518fac.x86_64 2023-11-24T07:55:16.087 INFO:teuthology.orchestra.run.smithi076.stdout: postgresql-contrib-10.21-2.module+el8.6.0+15342+53518fac.x86_64 2023-11-24T07:55:16.087 INFO:teuthology.orchestra.run.smithi076.stdout: postgresql-server-10.21-2.module+el8.6.0+15342+53518fac.x86_64 2023-11-24T07:55:16.088 INFO:teuthology.orchestra.run.smithi076.stdout: python3-asyncssh-2.7.0-2.el8.noarch 2023-11-24T07:55:16.088 INFO:teuthology.orchestra.run.smithi076.stdout: python3-babel-2.5.1-7.el8.noarch 2023-11-24T07:55:16.088 INFO:teuthology.orchestra.run.smithi076.stdout: python3-bcrypt-3.1.6-2.el8.1.x86_64 2023-11-24T07:55:16.088 INFO:teuthology.orchestra.run.smithi076.stdout: python3-beautifulsoup4-4.6.3-2.el8.1.noarch 2023-11-24T07:55:16.088 INFO:teuthology.orchestra.run.smithi076.stdout: python3-cachetools-3.1.1-4.el8.noarch 2023-11-24T07:55:16.088 INFO:teuthology.orchestra.run.smithi076.stdout: python3-ceph-argparse-2:18.0.0-7484.ga13ea5ff.el8.x86_64 2023-11-24T07:55:16.088 INFO:teuthology.orchestra.run.smithi076.stdout: python3-ceph-common-2:18.0.0-7484.ga13ea5ff.el8.x86_64 2023-11-24T07:55:16.088 INFO:teuthology.orchestra.run.smithi076.stdout: python3-cephfs-2:18.0.0-7484.ga13ea5ff.el8.x86_64 2023-11-24T07:55:16.089 INFO:teuthology.orchestra.run.smithi076.stdout: python3-certifi-2018.10.15-7.el8.noarch 2023-11-24T07:55:16.089 INFO:teuthology.orchestra.run.smithi076.stdout: python3-cffi-1.11.5-5.el8.x86_64 2023-11-24T07:55:16.089 INFO:teuthology.orchestra.run.smithi076.stdout: python3-cheroot-8.5.2-1.el8.noarch 2023-11-24T07:55:16.089 INFO:teuthology.orchestra.run.smithi076.stdout: python3-cherrypy-18.4.0-1.el8.noarch 2023-11-24T07:55:16.089 INFO:teuthology.orchestra.run.smithi076.stdout: python3-cryptography-3.2.1-5.el8.x86_64 2023-11-24T07:55:16.089 INFO:teuthology.orchestra.run.smithi076.stdout: python3-google-auth-1:1.1.1-10.el8.noarch 2023-11-24T07:55:16.089 INFO:teuthology.orchestra.run.smithi076.stdout: python3-jaraco-6.2-6.el8.noarch 2023-11-24T07:55:16.090 INFO:teuthology.orchestra.run.smithi076.stdout: python3-jaraco-functools-2.0-4.el8.noarch 2023-11-24T07:55:16.090 INFO:teuthology.orchestra.run.smithi076.stdout: python3-jinja2-2.10.1-3.el8.noarch 2023-11-24T07:55:16.090 INFO:teuthology.orchestra.run.smithi076.stdout: python3-jsonpatch-1.21-2.el8.noarch 2023-11-24T07:55:16.090 INFO:teuthology.orchestra.run.smithi076.stdout: python3-jsonpointer-1.10-11.el8.noarch 2023-11-24T07:55:16.090 INFO:teuthology.orchestra.run.smithi076.stdout: python3-jwt-1.6.1-2.el8.noarch 2023-11-24T07:55:16.090 INFO:teuthology.orchestra.run.smithi076.stdout: python3-kubernetes-1:11.0.0-6.el8.noarch 2023-11-24T07:55:16.090 INFO:teuthology.orchestra.run.smithi076.stdout: python3-logutils-0.3.5-11.el8.noarch 2023-11-24T07:55:16.091 INFO:teuthology.orchestra.run.smithi076.stdout: python3-mako-1.0.6-13.el8.noarch 2023-11-24T07:55:16.091 INFO:teuthology.orchestra.run.smithi076.stdout: python3-markupsafe-0.23-19.el8.x86_64 2023-11-24T07:55:16.091 INFO:teuthology.orchestra.run.smithi076.stdout: python3-more-itertools-7.2.0-3.el8.noarch 2023-11-24T07:55:16.091 INFO:teuthology.orchestra.run.smithi076.stdout: python3-natsort-7.1.1-2.el8.noarch 2023-11-24T07:55:16.091 INFO:teuthology.orchestra.run.smithi076.stdout: python3-numpy-1:1.14.3-10.el8.x86_64 2023-11-24T07:55:16.091 INFO:teuthology.orchestra.run.smithi076.stdout: python3-numpy-f2py-1:1.14.3-10.el8.x86_64 2023-11-24T07:55:16.091 INFO:teuthology.orchestra.run.smithi076.stdout: python3-oauthlib-2.1.0-1.el8.noarch 2023-11-24T07:55:16.092 INFO:teuthology.orchestra.run.smithi076.stdout: python3-pecan-1.3.2-9.el8.noarch 2023-11-24T07:55:16.092 INFO:teuthology.orchestra.run.smithi076.stdout: python3-portend-2.6-1.el8.noarch 2023-11-24T07:55:16.092 INFO:teuthology.orchestra.run.smithi076.stdout: python3-prettytable-0.7.2-14.el8.noarch 2023-11-24T07:55:16.092 INFO:teuthology.orchestra.run.smithi076.stdout: python3-pyOpenSSL-19.0.0-1.el8.noarch 2023-11-24T07:55:16.092 INFO:teuthology.orchestra.run.smithi076.stdout: python3-pyasn1-0.3.7-6.el8.noarch 2023-11-24T07:55:16.092 INFO:teuthology.orchestra.run.smithi076.stdout: python3-pyasn1-modules-0.3.7-6.el8.noarch 2023-11-24T07:55:16.092 INFO:teuthology.orchestra.run.smithi076.stdout: python3-pycparser-2.14-14.el8.noarch 2023-11-24T07:55:16.093 INFO:teuthology.orchestra.run.smithi076.stdout: python3-pytz-2017.2-9.el8.noarch 2023-11-24T07:55:16.093 INFO:teuthology.orchestra.run.smithi076.stdout: python3-rados-2:18.0.0-7484.ga13ea5ff.el8.x86_64 2023-11-24T07:55:16.093 INFO:teuthology.orchestra.run.smithi076.stdout: python3-rbd-2:18.0.0-7484.ga13ea5ff.el8.x86_64 2023-11-24T07:55:16.093 INFO:teuthology.orchestra.run.smithi076.stdout: python3-repoze-lru-0.7-6.el8.noarch 2023-11-24T07:55:16.093 INFO:teuthology.orchestra.run.smithi076.stdout: python3-requests-oauthlib-1.0.0-1.el8.noarch 2023-11-24T07:55:16.093 INFO:teuthology.orchestra.run.smithi076.stdout: python3-rgw-2:18.0.0-7484.ga13ea5ff.el8.x86_64 2023-11-24T07:55:16.093 INFO:teuthology.orchestra.run.smithi076.stdout: python3-routes-2.4.1-12.el8.noarch 2023-11-24T07:55:16.093 INFO:teuthology.orchestra.run.smithi076.stdout: python3-rsa-4.9-2.el8.noarch 2023-11-24T07:55:16.094 INFO:teuthology.orchestra.run.smithi076.stdout: python3-scipy-1.0.0-21.module+el8.5.0+10916+41bd434d.x86_64 2023-11-24T07:55:16.094 INFO:teuthology.orchestra.run.smithi076.stdout: python3-simplegeneric-0.8.1-17.el8.noarch 2023-11-24T07:55:16.094 INFO:teuthology.orchestra.run.smithi076.stdout: python3-singledispatch-3.4.0.3-18.el8.noarch 2023-11-24T07:55:16.094 INFO:teuthology.orchestra.run.smithi076.stdout: python3-tempora-1.14.1-5.el8.noarch 2023-11-24T07:55:16.094 INFO:teuthology.orchestra.run.smithi076.stdout: python3-trustme-0.6.0-4.el8.noarch 2023-11-24T07:55:16.094 INFO:teuthology.orchestra.run.smithi076.stdout: python3-waitress-1.4.3-1.el8.noarch 2023-11-24T07:55:16.094 INFO:teuthology.orchestra.run.smithi076.stdout: python3-webob-1.8.5-1.el8.1.noarch 2023-11-24T07:55:16.095 INFO:teuthology.orchestra.run.smithi076.stdout: python3-websocket-client-0.56.0-5.el8.noarch 2023-11-24T07:55:16.095 INFO:teuthology.orchestra.run.smithi076.stdout: python3-webtest-2.0.33-1.el8.noarch 2023-11-24T07:55:16.095 INFO:teuthology.orchestra.run.smithi076.stdout: python3-werkzeug-0.12.2-4.el8.noarch 2023-11-24T07:55:16.095 INFO:teuthology.orchestra.run.smithi076.stdout: python3-zc-lockfile-2.0-2.el8.noarch 2023-11-24T07:55:16.095 INFO:teuthology.orchestra.run.smithi076.stdout: rbd-fuse-2:18.0.0-7484.ga13ea5ff.el8.x86_64 2023-11-24T07:55:16.095 INFO:teuthology.orchestra.run.smithi076.stdout: rbd-mirror-2:18.0.0-7484.ga13ea5ff.el8.x86_64 2023-11-24T07:55:16.096 INFO:teuthology.orchestra.run.smithi076.stdout: rbd-nbd-2:18.0.0-7484.ga13ea5ff.el8.x86_64 2023-11-24T07:55:16.096 INFO:teuthology.orchestra.run.smithi076.stdout: socat-1.7.4.1-1.el8.x86_64 2023-11-24T07:55:16.096 INFO:teuthology.orchestra.run.smithi076.stdout: thrift-0.13.0-2.el8.x86_64 2023-11-24T07:55:16.096 INFO:teuthology.orchestra.run.smithi076.stdout: uuid-1.6.2-43.el8.x86_64 2023-11-24T07:55:16.096 INFO:teuthology.orchestra.run.smithi076.stdout: xfsprogs-devel-5.0.0-10.el8.x86_64 2023-11-24T07:55:16.096 INFO:teuthology.orchestra.run.smithi076.stdout: xmlstarlet-1.6.1-20.el8.x86_64 2023-11-24T07:55:16.097 INFO:teuthology.orchestra.run.smithi076.stdout: zlib-devel-1.2.11-18.el8_5.x86_64 2023-11-24T07:55:16.097 INFO:teuthology.orchestra.run.smithi076.stdout: 2023-11-24T07:55:16.097 INFO:teuthology.orchestra.run.smithi076.stdout:Complete! 2023-11-24T07:55:16.278 DEBUG:teuthology.parallel:result is None 2023-11-24T07:55:18.414 INFO:teuthology.orchestra.run.smithi059.stdout: Running scriptlet: postgresql-server-10.21-2.module+el8.6.0+15342+5 123/134 2023-11-24T07:55:18.701 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : postgresql-server-10.21-2.module+el8.6.0+15342+5 123/134 2023-11-24T07:55:18.719 INFO:teuthology.orchestra.run.smithi059.stdout: Running scriptlet: postgresql-server-10.21-2.module+el8.6.0+15342+5 123/134 2023-11-24T07:55:19.022 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : cephfs-top-2:18.0.0-7484.ga13ea5ff.el8.noarch 124/134 2023-11-24T07:55:19.247 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : rbd-fuse-2:18.0.0-7484.ga13ea5ff.el8.x86_64 125/134 2023-11-24T07:55:19.577 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : rbd-nbd-2:18.0.0-7484.ga13ea5ff.el8.x86_64 126/134 2023-11-24T07:55:19.948 INFO:teuthology.orchestra.run.smithi059.stdout: Upgrading : iproute-5.15.0-4.el8_6.1.x86_64 127/134 2023-11-24T07:55:20.087 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : NetworkManager-initscripts-updown-1:1.36.0-9.el8 128/134 2023-11-24T07:55:20.098 INFO:teuthology.orchestra.run.smithi059.stdout: Running scriptlet: NetworkManager-initscripts-updown-1:1.36.0-9.el8 128/134 2023-11-24T07:55:20.314 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : libaio-devel-0.3.112-1.el8.x86_64 129/134 2023-11-24T07:55:20.520 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : flex-2.6.1-9.el8.x86_64 130/134 2023-11-24T07:55:20.612 INFO:teuthology.orchestra.run.smithi059.stdout: Running scriptlet: flex-2.6.1-9.el8.x86_64 130/134 2023-11-24T07:55:20.845 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : bison-3.0.4-10.el8.x86_64 131/134 2023-11-24T07:55:20.860 INFO:teuthology.orchestra.run.smithi059.stdout: Running scriptlet: bison-3.0.4-10.el8.x86_64 131/134 2023-11-24T07:55:20.860 INFO:teuthology.orchestra.run.smithi059.stdout: Cleanup : librbd1-1:12.2.7-9.el8.x86_64 132/134 2023-11-24T07:55:21.074 INFO:teuthology.orchestra.run.smithi059.stdout: Running scriptlet: librbd1-1:12.2.7-9.el8.x86_64 132/134 2023-11-24T07:55:21.074 INFO:teuthology.orchestra.run.smithi059.stdout: Cleanup : librados2-1:12.2.7-9.el8.x86_64 133/134 2023-11-24T07:55:21.327 INFO:teuthology.orchestra.run.smithi059.stdout: Running scriptlet: librados2-1:12.2.7-9.el8.x86_64 133/134 2023-11-24T07:55:21.537 INFO:teuthology.orchestra.run.smithi059.stdout: Cleanup : iproute-5.15.0-4.el8.x86_64 134/134 2023-11-24T07:55:22.073 INFO:teuthology.orchestra.run.smithi059.stdout: Running scriptlet: iproute-5.15.0-4.el8.x86_64 134/134 2023-11-24T07:55:22.073 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : ceph-2:18.0.0-7484.ga13ea5ff.el8.x86_64 1/134 2023-11-24T07:55:22.073 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : ceph-base-2:18.0.0-7484.ga13ea5ff.el8.x86_64 2/134 2023-11-24T07:55:22.073 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : ceph-common-2:18.0.0-7484.ga13ea5ff.el8.x86_64 3/134 2023-11-24T07:55:22.073 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : ceph-fuse-2:18.0.0-7484.ga13ea5ff.el8.x86_64 4/134 2023-11-24T07:55:22.073 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : ceph-immutable-object-cache-2:18.0.0-7484.ga13ea 5/134 2023-11-24T07:55:22.073 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : ceph-mds-2:18.0.0-7484.ga13ea5ff.el8.x86_64 6/134 2023-11-24T07:55:22.073 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : ceph-mgr-2:18.0.0-7484.ga13ea5ff.el8.x86_64 7/134 2023-11-24T07:55:22.073 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : ceph-mon-2:18.0.0-7484.ga13ea5ff.el8.x86_64 8/134 2023-11-24T07:55:22.074 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : ceph-osd-2:18.0.0-7484.ga13ea5ff.el8.x86_64 9/134 2023-11-24T07:55:22.074 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : ceph-radosgw-2:18.0.0-7484.ga13ea5ff.el8.x86_64 10/134 2023-11-24T07:55:22.074 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : ceph-selinux-2:18.0.0-7484.ga13ea5ff.el8.x86_64 11/134 2023-11-24T07:55:22.074 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : ceph-test-2:18.0.0-7484.ga13ea5ff.el8.x86_64 12/134 2023-11-24T07:55:22.074 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : cephfs-mirror-2:18.0.0-7484.ga13ea5ff.el8.x86_64 13/134 2023-11-24T07:55:22.074 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : libcephfs-devel-2:18.0.0-7484.ga13ea5ff.el8.x86_ 14/134 2023-11-24T07:55:22.074 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : libcephfs2-2:18.0.0-7484.ga13ea5ff.el8.x86_64 15/134 2023-11-24T07:55:22.074 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : libcephsqlite-2:18.0.0-7484.ga13ea5ff.el8.x86_64 16/134 2023-11-24T07:55:22.074 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : librados-devel-2:18.0.0-7484.ga13ea5ff.el8.x86_6 17/134 2023-11-24T07:55:22.074 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : libradosstriper1-2:18.0.0-7484.ga13ea5ff.el8.x86 18/134 2023-11-24T07:55:22.074 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : librgw2-2:18.0.0-7484.ga13ea5ff.el8.x86_64 19/134 2023-11-24T07:55:22.074 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : python3-ceph-argparse-2:18.0.0-7484.ga13ea5ff.el 20/134 2023-11-24T07:55:22.074 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : python3-ceph-common-2:18.0.0-7484.ga13ea5ff.el8. 21/134 2023-11-24T07:55:22.075 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : python3-cephfs-2:18.0.0-7484.ga13ea5ff.el8.x86_6 22/134 2023-11-24T07:55:22.075 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : python3-rados-2:18.0.0-7484.ga13ea5ff.el8.x86_64 23/134 2023-11-24T07:55:22.075 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : python3-rbd-2:18.0.0-7484.ga13ea5ff.el8.x86_64 24/134 2023-11-24T07:55:22.075 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : python3-rgw-2:18.0.0-7484.ga13ea5ff.el8.x86_64 25/134 2023-11-24T07:55:22.075 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : rbd-fuse-2:18.0.0-7484.ga13ea5ff.el8.x86_64 26/134 2023-11-24T07:55:22.075 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : rbd-mirror-2:18.0.0-7484.ga13ea5ff.el8.x86_64 27/134 2023-11-24T07:55:22.075 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : rbd-nbd-2:18.0.0-7484.ga13ea5ff.el8.x86_64 28/134 2023-11-24T07:55:22.075 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : ceph-grafana-dashboards-2:18.0.0-7484.ga13ea5ff. 29/134 2023-11-24T07:55:22.075 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : ceph-mgr-cephadm-2:18.0.0-7484.ga13ea5ff.el8.noa 30/134 2023-11-24T07:55:22.075 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : ceph-mgr-dashboard-2:18.0.0-7484.ga13ea5ff.el8.n 31/134 2023-11-24T07:55:22.075 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : ceph-mgr-diskprediction-local-2:18.0.0-7484.ga13 32/134 2023-11-24T07:55:22.075 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : ceph-mgr-modules-core-2:18.0.0-7484.ga13ea5ff.el 33/134 2023-11-24T07:55:22.075 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : ceph-mgr-rook-2:18.0.0-7484.ga13ea5ff.el8.noarch 34/134 2023-11-24T07:55:22.076 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : ceph-prometheus-alerts-2:18.0.0-7484.ga13ea5ff.e 35/134 2023-11-24T07:55:22.076 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : ceph-volume-2:18.0.0-7484.ga13ea5ff.el8.noarch 36/134 2023-11-24T07:55:22.076 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : cephadm-2:18.0.0-7484.ga13ea5ff.el8.noarch 37/134 2023-11-24T07:55:22.076 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : cephfs-top-2:18.0.0-7484.ga13ea5ff.el8.noarch 38/134 2023-11-24T07:55:22.076 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : gperftools-libs-1:2.7-9.el8.x86_64 39/134 2023-11-24T07:55:22.076 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : liboath-2.6.2-3.el8.x86_64 40/134 2023-11-24T07:55:22.076 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : libunwind-1.3.1-3.el8.x86_64 41/134 2023-11-24T07:55:22.076 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : python3-asyncssh-2.7.0-2.el8.noarch 42/134 2023-11-24T07:55:22.076 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : python3-bcrypt-3.1.6-2.el8.1.x86_64 43/134 2023-11-24T07:55:22.076 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 44/134 2023-11-24T07:55:22.076 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : python3-cachetools-3.1.1-4.el8.noarch 45/134 2023-11-24T07:55:22.076 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : python3-certifi-2018.10.15-7.el8.noarch 46/134 2023-11-24T07:55:22.076 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : python3-cheroot-8.5.2-1.el8.noarch 47/134 2023-11-24T07:55:22.077 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : python3-cherrypy-18.4.0-1.el8.noarch 48/134 2023-11-24T07:55:22.077 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : python3-google-auth-1:1.1.1-10.el8.noarch 49/134 2023-11-24T07:55:22.077 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : python3-jaraco-6.2-6.el8.noarch 50/134 2023-11-24T07:55:22.077 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : python3-jaraco-functools-2.0-4.el8.noarch 51/134 2023-11-24T07:55:22.077 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : python3-kubernetes-1:11.0.0-6.el8.noarch 52/134 2023-11-24T07:55:22.077 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : python3-logutils-0.3.5-11.el8.noarch 53/134 2023-11-24T07:55:22.077 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : python3-more-itertools-7.2.0-3.el8.noarch 54/134 2023-11-24T07:55:22.077 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : python3-natsort-7.1.1-2.el8.noarch 55/134 2023-11-24T07:55:22.077 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : python3-pecan-1.3.2-9.el8.noarch 56/134 2023-11-24T07:55:22.077 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : python3-portend-2.6-1.el8.noarch 57/134 2023-11-24T07:55:22.078 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : python3-repoze-lru-0.7-6.el8.noarch 58/134 2023-11-24T07:55:22.078 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : python3-routes-2.4.1-12.el8.noarch 59/134 2023-11-24T07:55:22.078 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : python3-rsa-4.9-2.el8.noarch 60/134 2023-11-24T07:55:22.078 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : python3-simplegeneric-0.8.1-17.el8.noarch 61/134 2023-11-24T07:55:22.078 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : python3-singledispatch-3.4.0.3-18.el8.noarch 62/134 2023-11-24T07:55:22.078 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : python3-tempora-1.14.1-5.el8.noarch 63/134 2023-11-24T07:55:22.078 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : python3-trustme-0.6.0-4.el8.noarch 64/134 2023-11-24T07:55:22.078 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : python3-waitress-1.4.3-1.el8.noarch 65/134 2023-11-24T07:55:22.078 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : python3-webob-1.8.5-1.el8.1.noarch 66/134 2023-11-24T07:55:22.078 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : python3-websocket-client-0.56.0-5.el8.noarch 67/134 2023-11-24T07:55:22.078 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : python3-webtest-2.0.33-1.el8.noarch 68/134 2023-11-24T07:55:22.078 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : python3-zc-lockfile-2.0-2.el8.noarch 69/134 2023-11-24T07:55:22.078 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : thrift-0.13.0-2.el8.x86_64 70/134 2023-11-24T07:55:22.079 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : xmlstarlet-1.6.1-20.el8.x86_64 71/134 2023-11-24T07:55:22.079 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : python3-scipy-1.0.0-21.module+el8.5.0+10916+41bd 72/134 2023-11-24T07:55:22.079 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : python3-pyasn1-modules-0.3.7-6.el8.noarch 73/134 2023-11-24T07:55:22.079 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : python3-pytz-2017.2-9.el8.noarch 74/134 2023-11-24T07:55:22.079 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : lttng-ust-2.8.1-11.el8.x86_64 75/134 2023-11-24T07:55:22.079 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : socat-1.7.4.1-1.el8.x86_64 76/134 2023-11-24T07:55:22.079 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : python3-markupsafe-0.23-19.el8.x86_64 77/134 2023-11-24T07:55:22.079 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : bison-3.0.4-10.el8.x86_64 78/134 2023-11-24T07:55:22.079 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : python3-jinja2-2.10.1-3.el8.noarch 79/134 2023-11-24T07:55:22.079 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : python3-mako-1.0.6-13.el8.noarch 80/134 2023-11-24T07:55:22.079 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : oniguruma-6.8.2-2.el8.x86_64 81/134 2023-11-24T07:55:22.079 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : automake-1.16.1-7.el8.noarch 82/134 2023-11-24T07:55:22.079 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : python3-babel-2.5.1-7.el8.noarch 83/134 2023-11-24T07:55:22.080 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : python3-jsonpatch-1.21-2.el8.noarch 84/134 2023-11-24T07:55:22.080 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : flex-2.6.1-9.el8.x86_64 85/134 2023-11-24T07:55:22.080 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : python3-prettytable-0.7.2-14.el8.noarch 86/134 2023-11-24T07:55:22.080 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : python3-numpy-1:1.14.3-10.el8.x86_64 87/134 2023-11-24T07:55:22.080 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : python3-werkzeug-0.12.2-4.el8.noarch 88/134 2023-11-24T07:55:22.080 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : python3-numpy-f2py-1:1.14.3-10.el8.x86_64 89/134 2023-11-24T07:55:22.080 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : uuid-1.6.2-43.el8.x86_64 90/134 2023-11-24T07:55:22.080 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : python3-pyasn1-0.3.7-6.el8.noarch 91/134 2023-11-24T07:55:22.080 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : python3-jsonpointer-1.10-11.el8.noarch 92/134 2023-11-24T07:55:22.080 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : python3-pyOpenSSL-19.0.0-1.el8.noarch 93/134 2023-11-24T07:55:22.080 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : libtool-2.4.6-25.el8.x86_64 94/134 2023-11-24T07:55:22.080 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : openblas-threads-0.3.15-3.el8.x86_64 95/134 2023-11-24T07:55:22.081 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : libpq-13.5-1.el8.x86_64 96/134 2023-11-24T07:55:22.081 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : openblas-0.3.15-3.el8.x86_64 97/134 2023-11-24T07:55:22.081 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : jq-1.6-3.el8.x86_64 98/134 2023-11-24T07:55:22.081 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : librdkafka-0.11.4-3.el8.x86_64 99/134 2023-11-24T07:55:22.081 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : postgresql-server-10.21-2.module+el8.6.0+15342+5 100/134 2023-11-24T07:55:22.081 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : postgresql-contrib-10.21-2.module+el8.6.0+15342+ 101/134 2023-11-24T07:55:22.081 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : postgresql-10.21-2.module+el8.6.0+15342+53518fac 102/134 2023-11-24T07:55:22.081 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : librabbitmq-0.9.0-3.el8.x86_64 103/134 2023-11-24T07:55:22.081 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : libselinux-devel-2.9-5.el8.x86_64 104/134 2023-11-24T07:55:22.081 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : python3-jwt-1.6.1-2.el8.noarch 105/134 2023-11-24T07:55:22.081 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : pcre2-utf32-10.32-2.el8.x86_64 106/134 2023-11-24T07:55:22.081 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : keyutils-libs-devel-1.5.10-9.el8.x86_64 107/134 2023-11-24T07:55:22.082 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : libsepol-devel-2.9-3.el8.x86_64 108/134 2023-11-24T07:55:22.082 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : python3-requests-oauthlib-1.0.0-1.el8.noarch 109/134 2023-11-24T07:55:22.082 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : python3-pycparser-2.14-14.el8.noarch 110/134 2023-11-24T07:55:22.082 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : pcre2-utf16-10.32-2.el8.x86_64 111/134 2023-11-24T07:55:22.082 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : krb5-devel-1.18.2-14.el8.x86_64 112/134 2023-11-24T07:55:22.082 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : libaio-devel-0.3.112-1.el8.x86_64 113/134 2023-11-24T07:55:22.082 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : python3-oauthlib-2.1.0-1.el8.noarch 114/134 2023-11-24T07:55:22.082 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : python3-cryptography-3.2.1-5.el8.x86_64 115/134 2023-11-24T07:55:22.082 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : libverto-devel-0.3.0-5.el8.x86_64 116/134 2023-11-24T07:55:22.082 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : pcre2-devel-10.32-2.el8.x86_64 117/134 2023-11-24T07:55:22.083 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : python3-cffi-1.11.5-5.el8.x86_64 118/134 2023-11-24T07:55:22.083 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : openssl-devel-1:1.1.1k-7.el8_6.x86_64 119/134 2023-11-24T07:55:22.083 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : xfsprogs-devel-5.0.0-10.el8.x86_64 120/134 2023-11-24T07:55:22.083 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : fuse-2.9.7-15.el8.x86_64 121/134 2023-11-24T07:55:22.083 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : elfutils-libelf-devel-0.186-1.el8.x86_64 122/134 2023-11-24T07:55:22.083 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : libuuid-devel-2.32.1-35.el8.x86_64 123/134 2023-11-24T07:55:22.083 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : zlib-devel-1.2.11-18.el8_5.x86_64 124/134 2023-11-24T07:55:22.083 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : libcom_err-devel-1.45.6-4.el8.x86_64 125/134 2023-11-24T07:55:22.083 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : libgfortran-8.5.0-10.1.el8_6.x86_64 126/134 2023-11-24T07:55:22.083 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : libquadmath-8.5.0-10.1.el8_6.x86_64 127/134 2023-11-24T07:55:22.083 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : NetworkManager-initscripts-updown-1:1.36.0-9.el8 128/134 2023-11-24T07:55:22.084 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : librados2-2:18.0.0-7484.ga13ea5ff.el8.x86_64 129/134 2023-11-24T07:55:22.084 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : librados2-1:12.2.7-9.el8.x86_64 130/134 2023-11-24T07:55:22.084 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : librbd1-2:18.0.0-7484.ga13ea5ff.el8.x86_64 131/134 2023-11-24T07:55:22.084 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : librbd1-1:12.2.7-9.el8.x86_64 132/134 2023-11-24T07:55:22.084 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : iproute-5.15.0-4.el8_6.1.x86_64 133/134 2023-11-24T07:55:24.971 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : iproute-5.15.0-4.el8.x86_64 134/134 2023-11-24T07:55:24.971 INFO:teuthology.orchestra.run.smithi059.stdout:Last metadata expiration check: 0:01:58 ago on Fri 24 Nov 2023 07:53:26 AM UTC. 2023-11-24T07:55:27.696 INFO:teuthology.orchestra.run.smithi059.stdout:Installed products updated. 2023-11-24T07:55:27.938 INFO:teuthology.orchestra.run.smithi059.stdout: 2023-11-24T07:55:27.938 INFO:teuthology.orchestra.run.smithi059.stdout:Upgraded: 2023-11-24T07:55:27.939 INFO:teuthology.orchestra.run.smithi059.stdout: iproute-5.15.0-4.el8_6.1.x86_64 2023-11-24T07:55:27.939 INFO:teuthology.orchestra.run.smithi059.stdout: librados2-2:18.0.0-7484.ga13ea5ff.el8.x86_64 2023-11-24T07:55:27.939 INFO:teuthology.orchestra.run.smithi059.stdout: librbd1-2:18.0.0-7484.ga13ea5ff.el8.x86_64 2023-11-24T07:55:27.939 INFO:teuthology.orchestra.run.smithi059.stdout:Installed: 2023-11-24T07:55:27.939 INFO:teuthology.orchestra.run.smithi059.stdout: NetworkManager-initscripts-updown-1:1.36.0-9.el8_6.noarch 2023-11-24T07:55:27.939 INFO:teuthology.orchestra.run.smithi059.stdout: automake-1.16.1-7.el8.noarch 2023-11-24T07:55:27.939 INFO:teuthology.orchestra.run.smithi059.stdout: bison-3.0.4-10.el8.x86_64 2023-11-24T07:55:27.939 INFO:teuthology.orchestra.run.smithi059.stdout: ceph-2:18.0.0-7484.ga13ea5ff.el8.x86_64 2023-11-24T07:55:27.939 INFO:teuthology.orchestra.run.smithi059.stdout: ceph-base-2:18.0.0-7484.ga13ea5ff.el8.x86_64 2023-11-24T07:55:27.939 INFO:teuthology.orchestra.run.smithi059.stdout: ceph-common-2:18.0.0-7484.ga13ea5ff.el8.x86_64 2023-11-24T07:55:27.939 INFO:teuthology.orchestra.run.smithi059.stdout: ceph-fuse-2:18.0.0-7484.ga13ea5ff.el8.x86_64 2023-11-24T07:55:27.939 INFO:teuthology.orchestra.run.smithi059.stdout: ceph-grafana-dashboards-2:18.0.0-7484.ga13ea5ff.el8.noarch 2023-11-24T07:55:27.940 INFO:teuthology.orchestra.run.smithi059.stdout: ceph-immutable-object-cache-2:18.0.0-7484.ga13ea5ff.el8.x86_64 2023-11-24T07:55:27.940 INFO:teuthology.orchestra.run.smithi059.stdout: ceph-mds-2:18.0.0-7484.ga13ea5ff.el8.x86_64 2023-11-24T07:55:27.940 INFO:teuthology.orchestra.run.smithi059.stdout: ceph-mgr-2:18.0.0-7484.ga13ea5ff.el8.x86_64 2023-11-24T07:55:27.940 INFO:teuthology.orchestra.run.smithi059.stdout: ceph-mgr-cephadm-2:18.0.0-7484.ga13ea5ff.el8.noarch 2023-11-24T07:55:27.940 INFO:teuthology.orchestra.run.smithi059.stdout: ceph-mgr-dashboard-2:18.0.0-7484.ga13ea5ff.el8.noarch 2023-11-24T07:55:27.940 INFO:teuthology.orchestra.run.smithi059.stdout: ceph-mgr-diskprediction-local-2:18.0.0-7484.ga13ea5ff.el8.noarch 2023-11-24T07:55:27.940 INFO:teuthology.orchestra.run.smithi059.stdout: ceph-mgr-modules-core-2:18.0.0-7484.ga13ea5ff.el8.noarch 2023-11-24T07:55:27.940 INFO:teuthology.orchestra.run.smithi059.stdout: ceph-mgr-rook-2:18.0.0-7484.ga13ea5ff.el8.noarch 2023-11-24T07:55:27.940 INFO:teuthology.orchestra.run.smithi059.stdout: ceph-mon-2:18.0.0-7484.ga13ea5ff.el8.x86_64 2023-11-24T07:55:27.940 INFO:teuthology.orchestra.run.smithi059.stdout: ceph-osd-2:18.0.0-7484.ga13ea5ff.el8.x86_64 2023-11-24T07:55:27.940 INFO:teuthology.orchestra.run.smithi059.stdout: ceph-prometheus-alerts-2:18.0.0-7484.ga13ea5ff.el8.noarch 2023-11-24T07:55:27.940 INFO:teuthology.orchestra.run.smithi059.stdout: ceph-radosgw-2:18.0.0-7484.ga13ea5ff.el8.x86_64 2023-11-24T07:55:27.940 INFO:teuthology.orchestra.run.smithi059.stdout: ceph-selinux-2:18.0.0-7484.ga13ea5ff.el8.x86_64 2023-11-24T07:55:27.941 INFO:teuthology.orchestra.run.smithi059.stdout: ceph-test-2:18.0.0-7484.ga13ea5ff.el8.x86_64 2023-11-24T07:55:27.941 INFO:teuthology.orchestra.run.smithi059.stdout: ceph-volume-2:18.0.0-7484.ga13ea5ff.el8.noarch 2023-11-24T07:55:27.941 INFO:teuthology.orchestra.run.smithi059.stdout: cephadm-2:18.0.0-7484.ga13ea5ff.el8.noarch 2023-11-24T07:55:27.941 INFO:teuthology.orchestra.run.smithi059.stdout: cephfs-mirror-2:18.0.0-7484.ga13ea5ff.el8.x86_64 2023-11-24T07:55:27.941 INFO:teuthology.orchestra.run.smithi059.stdout: cephfs-top-2:18.0.0-7484.ga13ea5ff.el8.noarch 2023-11-24T07:55:27.941 INFO:teuthology.orchestra.run.smithi059.stdout: elfutils-libelf-devel-0.186-1.el8.x86_64 2023-11-24T07:55:27.941 INFO:teuthology.orchestra.run.smithi059.stdout: flex-2.6.1-9.el8.x86_64 2023-11-24T07:55:27.941 INFO:teuthology.orchestra.run.smithi059.stdout: fuse-2.9.7-15.el8.x86_64 2023-11-24T07:55:27.941 INFO:teuthology.orchestra.run.smithi059.stdout: gperftools-libs-1:2.7-9.el8.x86_64 2023-11-24T07:55:27.941 INFO:teuthology.orchestra.run.smithi059.stdout: jq-1.6-3.el8.x86_64 2023-11-24T07:55:27.941 INFO:teuthology.orchestra.run.smithi059.stdout: keyutils-libs-devel-1.5.10-9.el8.x86_64 2023-11-24T07:55:27.941 INFO:teuthology.orchestra.run.smithi059.stdout: krb5-devel-1.18.2-14.el8.x86_64 2023-11-24T07:55:27.941 INFO:teuthology.orchestra.run.smithi059.stdout: libaio-devel-0.3.112-1.el8.x86_64 2023-11-24T07:55:27.942 INFO:teuthology.orchestra.run.smithi059.stdout: libcephfs-devel-2:18.0.0-7484.ga13ea5ff.el8.x86_64 2023-11-24T07:55:27.942 INFO:teuthology.orchestra.run.smithi059.stdout: libcephfs2-2:18.0.0-7484.ga13ea5ff.el8.x86_64 2023-11-24T07:55:27.942 INFO:teuthology.orchestra.run.smithi059.stdout: libcephsqlite-2:18.0.0-7484.ga13ea5ff.el8.x86_64 2023-11-24T07:55:27.942 INFO:teuthology.orchestra.run.smithi059.stdout: libcom_err-devel-1.45.6-4.el8.x86_64 2023-11-24T07:55:27.942 INFO:teuthology.orchestra.run.smithi059.stdout: libgfortran-8.5.0-10.1.el8_6.x86_64 2023-11-24T07:55:27.942 INFO:teuthology.orchestra.run.smithi059.stdout: liboath-2.6.2-3.el8.x86_64 2023-11-24T07:55:27.942 INFO:teuthology.orchestra.run.smithi059.stdout: libpq-13.5-1.el8.x86_64 2023-11-24T07:55:27.942 INFO:teuthology.orchestra.run.smithi059.stdout: libquadmath-8.5.0-10.1.el8_6.x86_64 2023-11-24T07:55:27.942 INFO:teuthology.orchestra.run.smithi059.stdout: librabbitmq-0.9.0-3.el8.x86_64 2023-11-24T07:55:27.942 INFO:teuthology.orchestra.run.smithi059.stdout: librados-devel-2:18.0.0-7484.ga13ea5ff.el8.x86_64 2023-11-24T07:55:27.942 INFO:teuthology.orchestra.run.smithi059.stdout: libradosstriper1-2:18.0.0-7484.ga13ea5ff.el8.x86_64 2023-11-24T07:55:27.942 INFO:teuthology.orchestra.run.smithi059.stdout: librdkafka-0.11.4-3.el8.x86_64 2023-11-24T07:55:27.942 INFO:teuthology.orchestra.run.smithi059.stdout: librgw2-2:18.0.0-7484.ga13ea5ff.el8.x86_64 2023-11-24T07:55:27.943 INFO:teuthology.orchestra.run.smithi059.stdout: libselinux-devel-2.9-5.el8.x86_64 2023-11-24T07:55:27.943 INFO:teuthology.orchestra.run.smithi059.stdout: libsepol-devel-2.9-3.el8.x86_64 2023-11-24T07:55:27.943 INFO:teuthology.orchestra.run.smithi059.stdout: libtool-2.4.6-25.el8.x86_64 2023-11-24T07:55:27.943 INFO:teuthology.orchestra.run.smithi059.stdout: libunwind-1.3.1-3.el8.x86_64 2023-11-24T07:55:27.943 INFO:teuthology.orchestra.run.smithi059.stdout: libuuid-devel-2.32.1-35.el8.x86_64 2023-11-24T07:55:27.943 INFO:teuthology.orchestra.run.smithi059.stdout: libverto-devel-0.3.0-5.el8.x86_64 2023-11-24T07:55:27.943 INFO:teuthology.orchestra.run.smithi059.stdout: lttng-ust-2.8.1-11.el8.x86_64 2023-11-24T07:55:27.943 INFO:teuthology.orchestra.run.smithi059.stdout: oniguruma-6.8.2-2.el8.x86_64 2023-11-24T07:55:27.943 INFO:teuthology.orchestra.run.smithi059.stdout: openblas-0.3.15-3.el8.x86_64 2023-11-24T07:55:27.943 INFO:teuthology.orchestra.run.smithi059.stdout: openblas-threads-0.3.15-3.el8.x86_64 2023-11-24T07:55:27.943 INFO:teuthology.orchestra.run.smithi059.stdout: openssl-devel-1:1.1.1k-7.el8_6.x86_64 2023-11-24T07:55:27.943 INFO:teuthology.orchestra.run.smithi059.stdout: pcre2-devel-10.32-2.el8.x86_64 2023-11-24T07:55:27.943 INFO:teuthology.orchestra.run.smithi059.stdout: pcre2-utf16-10.32-2.el8.x86_64 2023-11-24T07:55:27.944 INFO:teuthology.orchestra.run.smithi059.stdout: pcre2-utf32-10.32-2.el8.x86_64 2023-11-24T07:55:27.944 INFO:teuthology.orchestra.run.smithi059.stdout: postgresql-10.21-2.module+el8.6.0+15342+53518fac.x86_64 2023-11-24T07:55:27.944 INFO:teuthology.orchestra.run.smithi059.stdout: postgresql-contrib-10.21-2.module+el8.6.0+15342+53518fac.x86_64 2023-11-24T07:55:27.944 INFO:teuthology.orchestra.run.smithi059.stdout: postgresql-server-10.21-2.module+el8.6.0+15342+53518fac.x86_64 2023-11-24T07:55:27.944 INFO:teuthology.orchestra.run.smithi059.stdout: python3-asyncssh-2.7.0-2.el8.noarch 2023-11-24T07:55:27.944 INFO:teuthology.orchestra.run.smithi059.stdout: python3-babel-2.5.1-7.el8.noarch 2023-11-24T07:55:27.944 INFO:teuthology.orchestra.run.smithi059.stdout: python3-bcrypt-3.1.6-2.el8.1.x86_64 2023-11-24T07:55:27.944 INFO:teuthology.orchestra.run.smithi059.stdout: python3-beautifulsoup4-4.6.3-2.el8.1.noarch 2023-11-24T07:55:27.944 INFO:teuthology.orchestra.run.smithi059.stdout: python3-cachetools-3.1.1-4.el8.noarch 2023-11-24T07:55:27.944 INFO:teuthology.orchestra.run.smithi059.stdout: python3-ceph-argparse-2:18.0.0-7484.ga13ea5ff.el8.x86_64 2023-11-24T07:55:27.944 INFO:teuthology.orchestra.run.smithi059.stdout: python3-ceph-common-2:18.0.0-7484.ga13ea5ff.el8.x86_64 2023-11-24T07:55:27.944 INFO:teuthology.orchestra.run.smithi059.stdout: python3-cephfs-2:18.0.0-7484.ga13ea5ff.el8.x86_64 2023-11-24T07:55:27.945 INFO:teuthology.orchestra.run.smithi059.stdout: python3-certifi-2018.10.15-7.el8.noarch 2023-11-24T07:55:27.945 INFO:teuthology.orchestra.run.smithi059.stdout: python3-cffi-1.11.5-5.el8.x86_64 2023-11-24T07:55:27.945 INFO:teuthology.orchestra.run.smithi059.stdout: python3-cheroot-8.5.2-1.el8.noarch 2023-11-24T07:55:27.945 INFO:teuthology.orchestra.run.smithi059.stdout: python3-cherrypy-18.4.0-1.el8.noarch 2023-11-24T07:55:27.945 INFO:teuthology.orchestra.run.smithi059.stdout: python3-cryptography-3.2.1-5.el8.x86_64 2023-11-24T07:55:27.945 INFO:teuthology.orchestra.run.smithi059.stdout: python3-google-auth-1:1.1.1-10.el8.noarch 2023-11-24T07:55:27.945 INFO:teuthology.orchestra.run.smithi059.stdout: python3-jaraco-6.2-6.el8.noarch 2023-11-24T07:55:27.945 INFO:teuthology.orchestra.run.smithi059.stdout: python3-jaraco-functools-2.0-4.el8.noarch 2023-11-24T07:55:27.945 INFO:teuthology.orchestra.run.smithi059.stdout: python3-jinja2-2.10.1-3.el8.noarch 2023-11-24T07:55:27.945 INFO:teuthology.orchestra.run.smithi059.stdout: python3-jsonpatch-1.21-2.el8.noarch 2023-11-24T07:55:27.945 INFO:teuthology.orchestra.run.smithi059.stdout: python3-jsonpointer-1.10-11.el8.noarch 2023-11-24T07:55:27.945 INFO:teuthology.orchestra.run.smithi059.stdout: python3-jwt-1.6.1-2.el8.noarch 2023-11-24T07:55:27.945 INFO:teuthology.orchestra.run.smithi059.stdout: python3-kubernetes-1:11.0.0-6.el8.noarch 2023-11-24T07:55:27.946 INFO:teuthology.orchestra.run.smithi059.stdout: python3-logutils-0.3.5-11.el8.noarch 2023-11-24T07:55:27.946 INFO:teuthology.orchestra.run.smithi059.stdout: python3-mako-1.0.6-13.el8.noarch 2023-11-24T07:55:27.946 INFO:teuthology.orchestra.run.smithi059.stdout: python3-markupsafe-0.23-19.el8.x86_64 2023-11-24T07:55:27.946 INFO:teuthology.orchestra.run.smithi059.stdout: python3-more-itertools-7.2.0-3.el8.noarch 2023-11-24T07:55:27.946 INFO:teuthology.orchestra.run.smithi059.stdout: python3-natsort-7.1.1-2.el8.noarch 2023-11-24T07:55:27.946 INFO:teuthology.orchestra.run.smithi059.stdout: python3-numpy-1:1.14.3-10.el8.x86_64 2023-11-24T07:55:27.946 INFO:teuthology.orchestra.run.smithi059.stdout: python3-numpy-f2py-1:1.14.3-10.el8.x86_64 2023-11-24T07:55:27.946 INFO:teuthology.orchestra.run.smithi059.stdout: python3-oauthlib-2.1.0-1.el8.noarch 2023-11-24T07:55:27.946 INFO:teuthology.orchestra.run.smithi059.stdout: python3-pecan-1.3.2-9.el8.noarch 2023-11-24T07:55:27.946 INFO:teuthology.orchestra.run.smithi059.stdout: python3-portend-2.6-1.el8.noarch 2023-11-24T07:55:27.946 INFO:teuthology.orchestra.run.smithi059.stdout: python3-prettytable-0.7.2-14.el8.noarch 2023-11-24T07:55:27.946 INFO:teuthology.orchestra.run.smithi059.stdout: python3-pyOpenSSL-19.0.0-1.el8.noarch 2023-11-24T07:55:27.946 INFO:teuthology.orchestra.run.smithi059.stdout: python3-pyasn1-0.3.7-6.el8.noarch 2023-11-24T07:55:27.947 INFO:teuthology.orchestra.run.smithi059.stdout: python3-pyasn1-modules-0.3.7-6.el8.noarch 2023-11-24T07:55:27.947 INFO:teuthology.orchestra.run.smithi059.stdout: python3-pycparser-2.14-14.el8.noarch 2023-11-24T07:55:27.947 INFO:teuthology.orchestra.run.smithi059.stdout: python3-pytz-2017.2-9.el8.noarch 2023-11-24T07:55:27.947 INFO:teuthology.orchestra.run.smithi059.stdout: python3-rados-2:18.0.0-7484.ga13ea5ff.el8.x86_64 2023-11-24T07:55:27.947 INFO:teuthology.orchestra.run.smithi059.stdout: python3-rbd-2:18.0.0-7484.ga13ea5ff.el8.x86_64 2023-11-24T07:55:27.947 INFO:teuthology.orchestra.run.smithi059.stdout: python3-repoze-lru-0.7-6.el8.noarch 2023-11-24T07:55:27.947 INFO:teuthology.orchestra.run.smithi059.stdout: python3-requests-oauthlib-1.0.0-1.el8.noarch 2023-11-24T07:55:27.947 INFO:teuthology.orchestra.run.smithi059.stdout: python3-rgw-2:18.0.0-7484.ga13ea5ff.el8.x86_64 2023-11-24T07:55:27.947 INFO:teuthology.orchestra.run.smithi059.stdout: python3-routes-2.4.1-12.el8.noarch 2023-11-24T07:55:27.947 INFO:teuthology.orchestra.run.smithi059.stdout: python3-rsa-4.9-2.el8.noarch 2023-11-24T07:55:27.947 INFO:teuthology.orchestra.run.smithi059.stdout: python3-scipy-1.0.0-21.module+el8.5.0+10916+41bd434d.x86_64 2023-11-24T07:55:27.947 INFO:teuthology.orchestra.run.smithi059.stdout: python3-simplegeneric-0.8.1-17.el8.noarch 2023-11-24T07:55:27.948 INFO:teuthology.orchestra.run.smithi059.stdout: python3-singledispatch-3.4.0.3-18.el8.noarch 2023-11-24T07:55:27.948 INFO:teuthology.orchestra.run.smithi059.stdout: python3-tempora-1.14.1-5.el8.noarch 2023-11-24T07:55:27.948 INFO:teuthology.orchestra.run.smithi059.stdout: python3-trustme-0.6.0-4.el8.noarch 2023-11-24T07:55:27.948 INFO:teuthology.orchestra.run.smithi059.stdout: python3-waitress-1.4.3-1.el8.noarch 2023-11-24T07:55:27.948 INFO:teuthology.orchestra.run.smithi059.stdout: python3-webob-1.8.5-1.el8.1.noarch 2023-11-24T07:55:27.948 INFO:teuthology.orchestra.run.smithi059.stdout: python3-websocket-client-0.56.0-5.el8.noarch 2023-11-24T07:55:27.948 INFO:teuthology.orchestra.run.smithi059.stdout: python3-webtest-2.0.33-1.el8.noarch 2023-11-24T07:55:27.948 INFO:teuthology.orchestra.run.smithi059.stdout: python3-werkzeug-0.12.2-4.el8.noarch 2023-11-24T07:55:27.948 INFO:teuthology.orchestra.run.smithi059.stdout: python3-zc-lockfile-2.0-2.el8.noarch 2023-11-24T07:55:27.948 INFO:teuthology.orchestra.run.smithi059.stdout: rbd-fuse-2:18.0.0-7484.ga13ea5ff.el8.x86_64 2023-11-24T07:55:27.948 INFO:teuthology.orchestra.run.smithi059.stdout: rbd-mirror-2:18.0.0-7484.ga13ea5ff.el8.x86_64 2023-11-24T07:55:27.948 INFO:teuthology.orchestra.run.smithi059.stdout: rbd-nbd-2:18.0.0-7484.ga13ea5ff.el8.x86_64 2023-11-24T07:55:27.948 INFO:teuthology.orchestra.run.smithi059.stdout: socat-1.7.4.1-1.el8.x86_64 2023-11-24T07:55:27.948 INFO:teuthology.orchestra.run.smithi059.stdout: thrift-0.13.0-2.el8.x86_64 2023-11-24T07:55:27.949 INFO:teuthology.orchestra.run.smithi059.stdout: uuid-1.6.2-43.el8.x86_64 2023-11-24T07:55:27.949 INFO:teuthology.orchestra.run.smithi059.stdout: xfsprogs-devel-5.0.0-10.el8.x86_64 2023-11-24T07:55:27.949 INFO:teuthology.orchestra.run.smithi059.stdout: xmlstarlet-1.6.1-20.el8.x86_64 2023-11-24T07:55:27.949 INFO:teuthology.orchestra.run.smithi059.stdout: zlib-devel-1.2.11-18.el8_5.x86_64 2023-11-24T07:55:27.949 INFO:teuthology.orchestra.run.smithi059.stdout: 2023-11-24T07:55:27.949 INFO:teuthology.orchestra.run.smithi059.stdout:Complete! 2023-11-24T07:55:28.194 DEBUG:teuthology.parallel:result is None 2023-11-24T07:55:28.195 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=a13ea5ff00cb4df7a9cc941166ddeb358881144c 2023-11-24T07:55:28.336 DEBUG:teuthology.orchestra.run.smithi059:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2023-11-24T07:55:28.360 INFO:teuthology.orchestra.run.smithi059.stdout:18.0.0-7484.ga13ea5ff.el8 2023-11-24T07:55:28.360 INFO:teuthology.packaging:The installed version of ceph is 18.0.0-7484.ga13ea5ff.el8 2023-11-24T07:55:28.361 INFO:teuthology.task.install:The correct ceph version 18.0.0-7484.ga13ea5ff is installed. 2023-11-24T07:55:28.361 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=a13ea5ff00cb4df7a9cc941166ddeb358881144c 2023-11-24T07:55:28.496 DEBUG:teuthology.orchestra.run.smithi076:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2023-11-24T07:55:28.524 INFO:teuthology.orchestra.run.smithi076.stdout:18.0.0-7484.ga13ea5ff.el8 2023-11-24T07:55:28.525 INFO:teuthology.packaging:The installed version of ceph is 18.0.0-7484.ga13ea5ff.el8 2023-11-24T07:55:28.525 INFO:teuthology.task.install:The correct ceph version 18.0.0-7484.ga13ea5ff is installed. 2023-11-24T07:55:28.525 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2023-11-24T07:55:28.525 DEBUG:teuthology.orchestra.run.smithi059:> set -ex 2023-11-24T07:55:28.525 DEBUG:teuthology.orchestra.run.smithi059:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2023-11-24T07:55:28.567 DEBUG:teuthology.orchestra.run.smithi076:> set -ex 2023-11-24T07:55:28.567 DEBUG:teuthology.orchestra.run.smithi076:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2023-11-24T07:55:28.616 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2023-11-24T07:55:28.617 DEBUG:teuthology.orchestra.run.smithi059:> set -ex 2023-11-24T07:55:28.617 DEBUG:teuthology.orchestra.run.smithi059:> sudo dd of=/usr/bin/daemon-helper 2023-11-24T07:55:28.644 DEBUG:teuthology.orchestra.run.smithi059:> sudo chmod a=rx -- /usr/bin/daemon-helper 2023-11-24T07:55:28.711 DEBUG:teuthology.orchestra.run.smithi076:> set -ex 2023-11-24T07:55:28.711 DEBUG:teuthology.orchestra.run.smithi076:> sudo dd of=/usr/bin/daemon-helper 2023-11-24T07:55:28.743 DEBUG:teuthology.orchestra.run.smithi076:> sudo chmod a=rx -- /usr/bin/daemon-helper 2023-11-24T07:55:28.813 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2023-11-24T07:55:28.813 DEBUG:teuthology.orchestra.run.smithi059:> set -ex 2023-11-24T07:55:28.814 DEBUG:teuthology.orchestra.run.smithi059:> sudo dd of=/usr/bin/adjust-ulimits 2023-11-24T07:55:28.840 DEBUG:teuthology.orchestra.run.smithi059:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2023-11-24T07:55:28.908 DEBUG:teuthology.orchestra.run.smithi076:> set -ex 2023-11-24T07:55:28.908 DEBUG:teuthology.orchestra.run.smithi076:> sudo dd of=/usr/bin/adjust-ulimits 2023-11-24T07:55:28.940 DEBUG:teuthology.orchestra.run.smithi076:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2023-11-24T07:55:29.009 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2023-11-24T07:55:29.009 DEBUG:teuthology.orchestra.run.smithi059:> set -ex 2023-11-24T07:55:29.009 DEBUG:teuthology.orchestra.run.smithi059:> sudo dd of=/usr/bin/stdin-killer 2023-11-24T07:55:29.036 DEBUG:teuthology.orchestra.run.smithi059:> sudo chmod a=rx -- /usr/bin/stdin-killer 2023-11-24T07:55:29.103 DEBUG:teuthology.orchestra.run.smithi076:> set -ex 2023-11-24T07:55:29.103 DEBUG:teuthology.orchestra.run.smithi076:> sudo dd of=/usr/bin/stdin-killer 2023-11-24T07:55:29.131 DEBUG:teuthology.orchestra.run.smithi076:> sudo chmod a=rx -- /usr/bin/stdin-killer 2023-11-24T07:55:29.199 INFO:teuthology.run_tasks:Running task ceph... 2023-11-24T07:55:29.273 INFO:tasks.ceph:Making ceph log dir writeable by non-root... 2023-11-24T07:55:29.274 DEBUG:teuthology.orchestra.run.smithi059:> sudo chmod 777 /var/log/ceph 2023-11-24T07:55:29.276 DEBUG:teuthology.orchestra.run.smithi076:> sudo chmod 777 /var/log/ceph 2023-11-24T07:55:29.303 INFO:tasks.ceph:Disabling ceph logrotate... 2023-11-24T07:55:29.304 DEBUG:teuthology.orchestra.run.smithi059:> sudo rm -f -- /etc/logrotate.d/ceph 2023-11-24T07:55:29.344 DEBUG:teuthology.orchestra.run.smithi076:> sudo rm -f -- /etc/logrotate.d/ceph 2023-11-24T07:55:29.370 INFO:tasks.ceph:Creating extra log directories... 2023-11-24T07:55:29.370 DEBUG:teuthology.orchestra.run.smithi059:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2023-11-24T07:55:29.411 DEBUG:teuthology.orchestra.run.smithi076:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2023-11-24T07:55:29.444 INFO:tasks.ceph:Setting up log rotation with {'ceph-mds': '10G', 'ceph-osd': '10G'} 2023-11-24T07:55:29.444 INFO:tasks.ceph:writing logrotate stanza for ceph-mds 2023-11-24T07:55:29.444 INFO:tasks.ceph:writing logrotate stanza for ceph-osd 2023-11-24T07:55:29.444 DEBUG:teuthology.orchestra.run.smithi059:> set -ex 2023-11-24T07:55:29.444 DEBUG:teuthology.orchestra.run.smithi059:> dd of=/home/ubuntu/cephtest/logrotate.ceph-test.conf 2023-11-24T07:55:29.493 DEBUG:teuthology.orchestra.run.smithi059:> sudo mv /home/ubuntu/cephtest/logrotate.ceph-test.conf /etc/logrotate.d/ceph-test.conf && sudo chmod 0644 /etc/logrotate.d/ceph-test.conf && sudo chown root.root /etc/logrotate.d/ceph-test.conf 2023-11-24T07:55:29.707 DEBUG:teuthology.orchestra.run.smithi059:> sudo chcon system_u:object_r:etc_t:s0 /etc/logrotate.d/ceph-test.conf 2023-11-24T07:55:29.735 DEBUG:teuthology.orchestra.run.smithi076:> set -ex 2023-11-24T07:55:29.736 DEBUG:teuthology.orchestra.run.smithi076:> dd of=/home/ubuntu/cephtest/logrotate.ceph-test.conf 2023-11-24T07:55:29.755 DEBUG:teuthology.orchestra.run.smithi076:> sudo mv /home/ubuntu/cephtest/logrotate.ceph-test.conf /etc/logrotate.d/ceph-test.conf && sudo chmod 0644 /etc/logrotate.d/ceph-test.conf && sudo chown root.root /etc/logrotate.d/ceph-test.conf 2023-11-24T07:55:29.893 DEBUG:teuthology.orchestra.run.smithi076:> sudo chcon system_u:object_r:etc_t:s0 /etc/logrotate.d/ceph-test.conf 2023-11-24T07:55:29.928 INFO:tasks.ceph:Creating ceph cluster ceph... 2023-11-24T07:55:29.928 INFO:tasks.ceph:config {'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'global': {'ms die on skipped message': False}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug client': 20, 'debug finisher': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon op complaint time': 120}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}}, 'fs': 'xfs', 'mkfs_options': None, 'mount_options': None, 'skip_mgr_daemons': False, 'log_ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'overall HEALTH_', '\\(FS_DEGRADED\\)', '\\(MDS_FAILED\\)', '\\(MDS_DEGRADED\\)', '\\(FS_WITH_FAILED_MDS\\)', '\\(MDS_DAMAGE\\)', '\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(FS_INLINE_DATA_DEPRECATED\\)', '\\(POOL_APP_NOT_ENABLED\\)', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'OSD full dropping all updates', 'OSD near full', 'pausewr flag', 'failsafe engaged, dropping updates', 'failsafe disengaged, no longer dropping', 'is full \\(reached quota', 'POOL_FULL', 'POOL_BACKFILLFULL'], 'cpu_profile': set(), 'cluster': 'ceph', 'mon_bind_msgr2': True, 'mon_bind_addrvec': True} 2023-11-24T07:55:29.928 INFO:tasks.ceph:ctx.config {'archive_path': '/home/teuthworker/archive/rishabh-2023-11-24_07:31:48-fs:volumes-rishabh-mgr-vol-misc-testing-default-smithi/7465575', 'branch': 'rishabh-mgr-vol-misc', 'description': 'fs:volumes/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{centos_latest} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/volumes/{overrides test/snapshot}}', 'email': None, 'first_in_suite': False, 'job_id': '7465575', 'kernel': {'client': {'sha1': 'distro'}, 'flavor': 'default', 'kdb': True, 'sha1': '8247bbb5b470f7e2a1f3eefcc16e24cf1338f606'}, 'ktype': 'distro', 'last_in_suite': False, 'log-rotate': {'ceph-mds': '10G', 'ceph-osd': '10G'}, 'machine_type': 'smithi', 'name': 'rishabh-2023-11-24_07:31:48-fs:volumes-rishabh-mgr-vol-misc-testing-default-smithi', 'no_nested_subset': False, 'nuke-on-error': True, 'openstack': [{'volumes': {'count': 4, 'size': 20}}, {'machine': {'disk': 200}}], 'os_type': 'rhel', 'os_version': '8.6', 'overrides': {'admin_socket': {'branch': 'rishabh-mgr-vol-misc'}, 'ceph': {'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'global': {'ms die on skipped message': False}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug client': 20, 'debug finisher': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon op complaint time': 120}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'overall HEALTH_', '\\(FS_DEGRADED\\)', '\\(MDS_FAILED\\)', '\\(MDS_DEGRADED\\)', '\\(FS_WITH_FAILED_MDS\\)', '\\(MDS_DAMAGE\\)', '\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(FS_INLINE_DATA_DEPRECATED\\)', '\\(POOL_APP_NOT_ENABLED\\)', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'OSD full dropping all updates', 'OSD near full', 'pausewr flag', 'failsafe engaged, dropping updates', 'failsafe disengaged, no longer dropping', 'is full \\(reached quota', 'POOL_FULL', 'POOL_BACKFILLFULL'], 'sha1': 'a13ea5ff00cb4df7a9cc941166ddeb358881144c'}, 'ceph-deploy': {'bluestore': True, 'conf': {'client': {'log file': '/var/log/ceph/ceph-$name.$pid.log'}, 'mon': {}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore'}}, 'fs': 'xfs'}, 'install': {'ceph': {'flavor': 'default', 'sha1': 'a13ea5ff00cb4df7a9cc941166ddeb358881144c'}}, 'kclient': {'syntax': 'v2'}, 'selinux': {'whitelist': ['scontext=system_u:system_r:logrotate_t:s0']}, 'thrashosds': {'bdev_inject_crash': 2, 'bdev_inject_crash_probability': 0.5}, 'workunit': {'branch': 'mgr-vol-tests', 'sha1': '73843baf45f8a92d8e3fcbea3f024c97e73036da'}}, 'owner': 'scheduled_rishabh@teuthology', 'priority': 50, 'repo': 'https://git.ceph.com/ceph-ci.git', 'roles': [['mon.a', 'mgr.y', 'mds.a', 'mds.c', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'client.2'], ['mon.b', 'mon.c', 'mgr.x', 'mds.b', 'mds.d', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'client.3']], 'seed': 9764, 'sha1': 'a13ea5ff00cb4df7a9cc941166ddeb358881144c', 'sleep_before_teardown': 0, 'suite': 'fs:volumes', 'suite_branch': 'mgr-vol-tests', 'suite_path': '/home/teuthworker/src/github.com_rishabh-d-dave_ceph_73843baf45f8a92d8e3fcbea3f024c97e73036da/qa', 'suite_relpath': 'qa', 'suite_repo': 'https://github.com/rishabh-d-dave/ceph', 'suite_sha1': '73843baf45f8a92d8e3fcbea3f024c97e73036da', 'syslog': {'ignorelist': ['WARNING*.*check_session_state', 'WARNING*.*__ceph_remove_cap']}, 'targets': {'smithi059.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLVvN3+LmzPZ4ixiwyCkS3KZLZ9R6MOPm+kPtaJbl8fExpzbk8TudXCcJWHBL7DGPWFvye2jT6sei9eiX0EUQYs=', 'smithi076.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLVvN3+LmzPZ4ixiwyCkS3KZLZ9R6MOPm+kPtaJbl8fExpzbk8TudXCcJWHBL7DGPWFvye2jT6sei9eiX0EUQYs='}, 'tasks': [{'internal.check_packages': None}, {'internal.buildpackages_prep': None}, {'internal.save_config': None}, {'internal.check_lock': None}, {'internal.add_remotes': None}, {'console_log': None}, {'internal.connect': None}, {'internal.push_inventory': None}, {'internal.serialize_remote_roles': None}, {'internal.check_conflict': None}, {'internal.check_ceph_data': None}, {'internal.vm_setup': None}, {'kernel': {'client': {'sha1': 'distro'}, 'flavor': 'default', 'kdb': True, 'sha1': '8247bbb5b470f7e2a1f3eefcc16e24cf1338f606'}}, {'internal.base': None}, {'internal.archive_upload': None}, {'internal.archive': None}, {'internal.coredump': None}, {'internal.sudo': None}, {'internal.syslog': None}, {'internal.timer': None}, {'pcp': None}, {'selinux': None}, {'ansible.cephlab': None}, {'clock': None}, {'install': {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': 'a13ea5ff00cb4df7a9cc941166ddeb358881144c'}}, {'ceph': {'create_rbd_pool': False, 'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'global': {'ms die on skipped message': False}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug client': 20, 'debug finisher': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon op complaint time': 120}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'overall HEALTH_', '\\(FS_DEGRADED\\)', '\\(MDS_FAILED\\)', '\\(MDS_DEGRADED\\)', '\\(FS_WITH_FAILED_MDS\\)', '\\(MDS_DAMAGE\\)', '\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(FS_INLINE_DATA_DEPRECATED\\)', '\\(POOL_APP_NOT_ENABLED\\)', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'OSD full dropping all updates', 'OSD near full', 'pausewr flag', 'failsafe engaged, dropping updates', 'failsafe disengaged, no longer dropping', 'is full \\(reached quota', 'POOL_FULL', 'POOL_BACKFILLFULL'], 'sha1': 'a13ea5ff00cb4df7a9cc941166ddeb358881144c', 'cluster': 'ceph'}}, {'kclient': None}, {'cephfs_test_runner': {'fail_on_skip': False, 'modules': ['tasks.cephfs.test_volumes.TestSubvolumeGroupSnapshots', 'tasks.cephfs.test_volumes.TestSubvolumeSnapshots']}}], 'teuthology': {'fragments_dropped': [], 'meta': {}, 'postmerge': ["local function is_kupstream() return yaml.ktype == 'upstream' end", "local function is_kdistro() return yaml.ktype == 'distro' end", 'local function is_fuse() return false end', 'local function syntax_version() return yaml.overrides.kclient.syntax end']}, 'teuthology_branch': 'main', 'teuthology_sha1': '2442239d2653456406c25ae0c71b689c8f2657b6', 'timestamp': '2023-11-24_07:31:48', 'tube': 'smithi', 'user': 'rishabh', 'verbose': False, 'worker_log': '/home/teuthworker/archive/worker_logs/dispatcher.smithi.232157'} 2023-11-24T07:55:29.929 DEBUG:teuthology.orchestra.run.smithi059:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2023-11-24T07:55:29.931 DEBUG:teuthology.orchestra.run.smithi076:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2023-11-24T07:55:29.979 DEBUG:teuthology.orchestra.run.smithi059:> sudo install -d -m0777 -- /var/run/ceph 2023-11-24T07:55:29.989 DEBUG:teuthology.orchestra.run.smithi076:> sudo install -d -m0777 -- /var/run/ceph 2023-11-24T07:55:30.047 DEBUG:teuthology.orchestra.run.smithi059:> set -ex 2023-11-24T07:55:30.048 DEBUG:teuthology.orchestra.run.smithi059:> dd if=/scratch_devs of=/dev/stdout 2023-11-24T07:55:30.072 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2023-11-24T07:55:30.072 DEBUG:teuthology.orchestra.run.smithi059:> stat /dev/vg_nvme/lv_1 2023-11-24T07:55:30.130 INFO:teuthology.orchestra.run.smithi059.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2023-11-24T07:55:30.131 INFO:teuthology.orchestra.run.smithi059.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-11-24T07:55:30.131 INFO:teuthology.orchestra.run.smithi059.stdout:Device: 6h/6d Inode: 388465 Links: 1 2023-11-24T07:55:30.131 INFO:teuthology.orchestra.run.smithi059.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-11-24T07:55:30.131 INFO:teuthology.orchestra.run.smithi059.stdout:Context: system_u:object_r:device_t:s0 2023-11-24T07:55:30.131 INFO:teuthology.orchestra.run.smithi059.stdout:Access: 2023-11-24 07:55:21.743814457 +0000 2023-11-24T07:55:30.131 INFO:teuthology.orchestra.run.smithi059.stdout:Modify: 2023-11-24 07:52:39.308492376 +0000 2023-11-24T07:55:30.132 INFO:teuthology.orchestra.run.smithi059.stdout:Change: 2023-11-24 07:52:39.308492376 +0000 2023-11-24T07:55:30.132 INFO:teuthology.orchestra.run.smithi059.stdout: Birth: - 2023-11-24T07:55:30.132 DEBUG:teuthology.orchestra.run.smithi059:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2023-11-24T07:55:30.197 INFO:teuthology.orchestra.run.smithi059.stderr:1+0 records in 2023-11-24T07:55:30.197 INFO:teuthology.orchestra.run.smithi059.stderr:1+0 records out 2023-11-24T07:55:30.197 INFO:teuthology.orchestra.run.smithi059.stderr:512 bytes copied, 0.000248402 s, 2.1 MB/s 2023-11-24T07:55:30.198 DEBUG:teuthology.orchestra.run.smithi059:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2023-11-24T07:55:30.257 DEBUG:teuthology.orchestra.run.smithi059:> stat /dev/vg_nvme/lv_2 2023-11-24T07:55:30.314 INFO:teuthology.orchestra.run.smithi059.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2023-11-24T07:55:30.314 INFO:teuthology.orchestra.run.smithi059.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-11-24T07:55:30.314 INFO:teuthology.orchestra.run.smithi059.stdout:Device: 6h/6d Inode: 388514 Links: 1 2023-11-24T07:55:30.314 INFO:teuthology.orchestra.run.smithi059.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-11-24T07:55:30.314 INFO:teuthology.orchestra.run.smithi059.stdout:Context: system_u:object_r:device_t:s0 2023-11-24T07:55:30.314 INFO:teuthology.orchestra.run.smithi059.stdout:Access: 2023-11-24 07:55:21.743814457 +0000 2023-11-24T07:55:30.314 INFO:teuthology.orchestra.run.smithi059.stdout:Modify: 2023-11-24 07:52:39.863485150 +0000 2023-11-24T07:55:30.314 INFO:teuthology.orchestra.run.smithi059.stdout:Change: 2023-11-24 07:52:39.863485150 +0000 2023-11-24T07:55:30.314 INFO:teuthology.orchestra.run.smithi059.stdout: Birth: - 2023-11-24T07:55:30.315 DEBUG:teuthology.orchestra.run.smithi059:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2023-11-24T07:55:30.380 INFO:teuthology.orchestra.run.smithi059.stderr:1+0 records in 2023-11-24T07:55:30.380 INFO:teuthology.orchestra.run.smithi059.stderr:1+0 records out 2023-11-24T07:55:30.380 INFO:teuthology.orchestra.run.smithi059.stderr:512 bytes copied, 0.000218754 s, 2.3 MB/s 2023-11-24T07:55:30.381 DEBUG:teuthology.orchestra.run.smithi059:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2023-11-24T07:55:30.439 DEBUG:teuthology.orchestra.run.smithi059:> stat /dev/vg_nvme/lv_3 2023-11-24T07:55:30.496 INFO:teuthology.orchestra.run.smithi059.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2023-11-24T07:55:30.496 INFO:teuthology.orchestra.run.smithi059.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-11-24T07:55:30.497 INFO:teuthology.orchestra.run.smithi059.stdout:Device: 6h/6d Inode: 388562 Links: 1 2023-11-24T07:55:30.497 INFO:teuthology.orchestra.run.smithi059.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-11-24T07:55:30.497 INFO:teuthology.orchestra.run.smithi059.stdout:Context: system_u:object_r:device_t:s0 2023-11-24T07:55:30.497 INFO:teuthology.orchestra.run.smithi059.stdout:Access: 2023-11-24 07:55:21.743814457 +0000 2023-11-24T07:55:30.497 INFO:teuthology.orchestra.run.smithi059.stdout:Modify: 2023-11-24 07:52:40.398478185 +0000 2023-11-24T07:55:30.497 INFO:teuthology.orchestra.run.smithi059.stdout:Change: 2023-11-24 07:52:40.398478185 +0000 2023-11-24T07:55:30.497 INFO:teuthology.orchestra.run.smithi059.stdout: Birth: - 2023-11-24T07:55:30.497 DEBUG:teuthology.orchestra.run.smithi059:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2023-11-24T07:55:30.562 INFO:teuthology.orchestra.run.smithi059.stderr:1+0 records in 2023-11-24T07:55:30.562 INFO:teuthology.orchestra.run.smithi059.stderr:1+0 records out 2023-11-24T07:55:30.562 INFO:teuthology.orchestra.run.smithi059.stderr:512 bytes copied, 0.000235035 s, 2.2 MB/s 2023-11-24T07:55:30.563 DEBUG:teuthology.orchestra.run.smithi059:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2023-11-24T07:55:30.621 DEBUG:teuthology.orchestra.run.smithi059:> stat /dev/vg_nvme/lv_4 2023-11-24T07:55:30.678 INFO:teuthology.orchestra.run.smithi059.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2023-11-24T07:55:30.679 INFO:teuthology.orchestra.run.smithi059.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-11-24T07:55:30.679 INFO:teuthology.orchestra.run.smithi059.stdout:Device: 6h/6d Inode: 388589 Links: 1 2023-11-24T07:55:30.679 INFO:teuthology.orchestra.run.smithi059.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-11-24T07:55:30.679 INFO:teuthology.orchestra.run.smithi059.stdout:Context: system_u:object_r:device_t:s0 2023-11-24T07:55:30.679 INFO:teuthology.orchestra.run.smithi059.stdout:Access: 2023-11-24 07:55:21.743814457 +0000 2023-11-24T07:55:30.679 INFO:teuthology.orchestra.run.smithi059.stdout:Modify: 2023-11-24 07:52:40.931471246 +0000 2023-11-24T07:55:30.679 INFO:teuthology.orchestra.run.smithi059.stdout:Change: 2023-11-24 07:52:40.931471246 +0000 2023-11-24T07:55:30.679 INFO:teuthology.orchestra.run.smithi059.stdout: Birth: - 2023-11-24T07:55:30.679 DEBUG:teuthology.orchestra.run.smithi059:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2023-11-24T07:55:30.744 INFO:teuthology.orchestra.run.smithi059.stderr:1+0 records in 2023-11-24T07:55:30.744 INFO:teuthology.orchestra.run.smithi059.stderr:1+0 records out 2023-11-24T07:55:30.744 INFO:teuthology.orchestra.run.smithi059.stderr:512 bytes copied, 0.000199003 s, 2.6 MB/s 2023-11-24T07:55:30.745 DEBUG:teuthology.orchestra.run.smithi059:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2023-11-24T07:55:30.803 INFO:tasks.ceph:osd dev map: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2023-11-24T07:55:30.803 DEBUG:teuthology.orchestra.run.smithi076:> set -ex 2023-11-24T07:55:30.803 DEBUG:teuthology.orchestra.run.smithi076:> dd if=/scratch_devs of=/dev/stdout 2023-11-24T07:55:30.824 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2023-11-24T07:55:30.824 DEBUG:teuthology.orchestra.run.smithi076:> stat /dev/vg_nvme/lv_1 2023-11-24T07:55:30.884 INFO:teuthology.orchestra.run.smithi076.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2023-11-24T07:55:30.884 INFO:teuthology.orchestra.run.smithi076.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-11-24T07:55:30.884 INFO:teuthology.orchestra.run.smithi076.stdout:Device: 6h/6d Inode: 392461 Links: 1 2023-11-24T07:55:30.885 INFO:teuthology.orchestra.run.smithi076.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-11-24T07:55:30.885 INFO:teuthology.orchestra.run.smithi076.stdout:Context: system_u:object_r:device_t:s0 2023-11-24T07:55:30.885 INFO:teuthology.orchestra.run.smithi076.stdout:Access: 2023-11-24 07:55:10.143172558 +0000 2023-11-24T07:55:30.885 INFO:teuthology.orchestra.run.smithi076.stdout:Modify: 2023-11-24 07:52:31.076346467 +0000 2023-11-24T07:55:30.885 INFO:teuthology.orchestra.run.smithi076.stdout:Change: 2023-11-24 07:52:31.076346467 +0000 2023-11-24T07:55:30.885 INFO:teuthology.orchestra.run.smithi076.stdout: Birth: - 2023-11-24T07:55:30.885 DEBUG:teuthology.orchestra.run.smithi076:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2023-11-24T07:55:30.955 INFO:teuthology.orchestra.run.smithi076.stderr:1+0 records in 2023-11-24T07:55:30.955 INFO:teuthology.orchestra.run.smithi076.stderr:1+0 records out 2023-11-24T07:55:30.955 INFO:teuthology.orchestra.run.smithi076.stderr:512 bytes copied, 0.000304946 s, 1.7 MB/s 2023-11-24T07:55:30.956 DEBUG:teuthology.orchestra.run.smithi076:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2023-11-24T07:55:31.018 DEBUG:teuthology.orchestra.run.smithi076:> stat /dev/vg_nvme/lv_2 2023-11-24T07:55:31.079 INFO:teuthology.orchestra.run.smithi076.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2023-11-24T07:55:31.079 INFO:teuthology.orchestra.run.smithi076.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-11-24T07:55:31.080 INFO:teuthology.orchestra.run.smithi076.stdout:Device: 6h/6d Inode: 385351 Links: 1 2023-11-24T07:55:31.080 INFO:teuthology.orchestra.run.smithi076.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-11-24T07:55:31.080 INFO:teuthology.orchestra.run.smithi076.stdout:Context: system_u:object_r:device_t:s0 2023-11-24T07:55:31.080 INFO:teuthology.orchestra.run.smithi076.stdout:Access: 2023-11-24 07:55:10.143172558 +0000 2023-11-24T07:55:31.080 INFO:teuthology.orchestra.run.smithi076.stdout:Modify: 2023-11-24 07:52:31.585330702 +0000 2023-11-24T07:55:31.080 INFO:teuthology.orchestra.run.smithi076.stdout:Change: 2023-11-24 07:52:31.585330702 +0000 2023-11-24T07:55:31.080 INFO:teuthology.orchestra.run.smithi076.stdout: Birth: - 2023-11-24T07:55:31.080 DEBUG:teuthology.orchestra.run.smithi076:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2023-11-24T07:55:31.150 INFO:teuthology.orchestra.run.smithi076.stderr:1+0 records in 2023-11-24T07:55:31.150 INFO:teuthology.orchestra.run.smithi076.stderr:1+0 records out 2023-11-24T07:55:31.150 INFO:teuthology.orchestra.run.smithi076.stderr:512 bytes copied, 0.000306011 s, 1.7 MB/s 2023-11-24T07:55:31.151 DEBUG:teuthology.orchestra.run.smithi076:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2023-11-24T07:55:31.213 DEBUG:teuthology.orchestra.run.smithi076:> stat /dev/vg_nvme/lv_3 2023-11-24T07:55:31.274 INFO:teuthology.orchestra.run.smithi076.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2023-11-24T07:55:31.274 INFO:teuthology.orchestra.run.smithi076.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-11-24T07:55:31.274 INFO:teuthology.orchestra.run.smithi076.stdout:Device: 6h/6d Inode: 391070 Links: 1 2023-11-24T07:55:31.274 INFO:teuthology.orchestra.run.smithi076.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-11-24T07:55:31.274 INFO:teuthology.orchestra.run.smithi076.stdout:Context: system_u:object_r:device_t:s0 2023-11-24T07:55:31.274 INFO:teuthology.orchestra.run.smithi076.stdout:Access: 2023-11-24 07:55:10.143172558 +0000 2023-11-24T07:55:31.274 INFO:teuthology.orchestra.run.smithi076.stdout:Modify: 2023-11-24 07:52:32.099314781 +0000 2023-11-24T07:55:31.275 INFO:teuthology.orchestra.run.smithi076.stdout:Change: 2023-11-24 07:52:32.099314781 +0000 2023-11-24T07:55:31.275 INFO:teuthology.orchestra.run.smithi076.stdout: Birth: - 2023-11-24T07:55:31.275 DEBUG:teuthology.orchestra.run.smithi076:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2023-11-24T07:55:31.345 INFO:teuthology.orchestra.run.smithi076.stderr:1+0 records in 2023-11-24T07:55:31.345 INFO:teuthology.orchestra.run.smithi076.stderr:1+0 records out 2023-11-24T07:55:31.345 INFO:teuthology.orchestra.run.smithi076.stderr:512 bytes copied, 0.00028666 s, 1.8 MB/s 2023-11-24T07:55:31.347 DEBUG:teuthology.orchestra.run.smithi076:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2023-11-24T07:55:31.408 DEBUG:teuthology.orchestra.run.smithi076:> stat /dev/vg_nvme/lv_4 2023-11-24T07:55:31.469 INFO:teuthology.orchestra.run.smithi076.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2023-11-24T07:55:31.469 INFO:teuthology.orchestra.run.smithi076.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-11-24T07:55:31.469 INFO:teuthology.orchestra.run.smithi076.stdout:Device: 6h/6d Inode: 388511 Links: 1 2023-11-24T07:55:31.470 INFO:teuthology.orchestra.run.smithi076.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-11-24T07:55:31.470 INFO:teuthology.orchestra.run.smithi076.stdout:Context: system_u:object_r:device_t:s0 2023-11-24T07:55:31.470 INFO:teuthology.orchestra.run.smithi076.stdout:Access: 2023-11-24 07:55:10.143172558 +0000 2023-11-24T07:55:31.470 INFO:teuthology.orchestra.run.smithi076.stdout:Modify: 2023-11-24 07:52:32.592299513 +0000 2023-11-24T07:55:31.470 INFO:teuthology.orchestra.run.smithi076.stdout:Change: 2023-11-24 07:52:32.592299513 +0000 2023-11-24T07:55:31.470 INFO:teuthology.orchestra.run.smithi076.stdout: Birth: - 2023-11-24T07:55:31.471 DEBUG:teuthology.orchestra.run.smithi076:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2023-11-24T07:55:31.540 INFO:teuthology.orchestra.run.smithi076.stderr:1+0 records in 2023-11-24T07:55:31.540 INFO:teuthology.orchestra.run.smithi076.stderr:1+0 records out 2023-11-24T07:55:31.540 INFO:teuthology.orchestra.run.smithi076.stderr:512 bytes copied, 0.000206173 s, 2.5 MB/s 2023-11-24T07:55:31.541 DEBUG:teuthology.orchestra.run.smithi076:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2023-11-24T07:55:31.602 INFO:tasks.ceph:osd dev map: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2023-11-24T07:55:31.602 INFO:tasks.ceph:remote_to_roles_to_devs: {Remote(name='ubuntu@smithi059.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi076.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}} 2023-11-24T07:55:31.602 INFO:tasks.ceph:Generating config... 2023-11-24T07:55:31.603 INFO:tasks.ceph:[client] client mount timeout = 600 2023-11-24T07:55:31.603 INFO:tasks.ceph:[client] debug client = 20 2023-11-24T07:55:31.603 INFO:tasks.ceph:[client] debug ms = 1 2023-11-24T07:55:31.603 INFO:tasks.ceph:[client] pid file = 2023-11-24T07:55:31.604 INFO:tasks.ceph:[client] rados mon op timeout = 900 2023-11-24T07:55:31.604 INFO:tasks.ceph:[client] rados osd op timeout = 900 2023-11-24T07:55:31.604 INFO:tasks.ceph:[global] ms die on skipped message = False 2023-11-24T07:55:31.604 INFO:tasks.ceph:[mds] debug mds = 20 2023-11-24T07:55:31.604 INFO:tasks.ceph:[mds] debug mds balancer = 20 2023-11-24T07:55:31.604 INFO:tasks.ceph:[mds] debug ms = 1 2023-11-24T07:55:31.604 INFO:tasks.ceph:[mds] mds debug frag = True 2023-11-24T07:55:31.604 INFO:tasks.ceph:[mds] mds debug scatterstat = True 2023-11-24T07:55:31.604 INFO:tasks.ceph:[mds] mds op complaint time = 180 2023-11-24T07:55:31.604 INFO:tasks.ceph:[mds] mds verify scatter = True 2023-11-24T07:55:31.604 INFO:tasks.ceph:[mds] osd op complaint time = 180 2023-11-24T07:55:31.604 INFO:tasks.ceph:[mds] rados mon op timeout = 900 2023-11-24T07:55:31.604 INFO:tasks.ceph:[mds] rados osd op timeout = 900 2023-11-24T07:55:31.605 INFO:tasks.ceph:[mgr] debug client = 20 2023-11-24T07:55:31.605 INFO:tasks.ceph:[mgr] debug finisher = 20 2023-11-24T07:55:31.605 INFO:tasks.ceph:[mgr] debug mgr = 20 2023-11-24T07:55:31.605 INFO:tasks.ceph:[mgr] debug ms = 1 2023-11-24T07:55:31.605 INFO:tasks.ceph:[mon] debug mon = 20 2023-11-24T07:55:31.605 INFO:tasks.ceph:[mon] debug ms = 1 2023-11-24T07:55:31.605 INFO:tasks.ceph:[mon] debug paxos = 20 2023-11-24T07:55:31.605 INFO:tasks.ceph:[mon] mon op complaint time = 120 2023-11-24T07:55:31.605 INFO:tasks.ceph:[osd] bdev async discard = True 2023-11-24T07:55:31.605 INFO:tasks.ceph:[osd] bdev enable discard = True 2023-11-24T07:55:31.605 INFO:tasks.ceph:[osd] bluestore allocator = bitmap 2023-11-24T07:55:31.605 INFO:tasks.ceph:[osd] bluestore block size = 96636764160 2023-11-24T07:55:31.605 INFO:tasks.ceph:[osd] bluestore fsck on mount = True 2023-11-24T07:55:31.605 INFO:tasks.ceph:[osd] debug bluefs = 1/20 2023-11-24T07:55:31.606 INFO:tasks.ceph:[osd] debug bluestore = 1/20 2023-11-24T07:55:31.606 INFO:tasks.ceph:[osd] debug ms = 1 2023-11-24T07:55:31.606 INFO:tasks.ceph:[osd] debug osd = 20 2023-11-24T07:55:31.606 INFO:tasks.ceph:[osd] debug rocksdb = 4/10 2023-11-24T07:55:31.606 INFO:tasks.ceph:[osd] mon osd backfillfull_ratio = 0.85 2023-11-24T07:55:31.606 INFO:tasks.ceph:[osd] mon osd full ratio = 0.9 2023-11-24T07:55:31.606 INFO:tasks.ceph:[osd] mon osd nearfull ratio = 0.8 2023-11-24T07:55:31.606 INFO:tasks.ceph:[osd] osd failsafe full ratio = 0.95 2023-11-24T07:55:31.606 INFO:tasks.ceph:[osd] osd objectstore = bluestore 2023-11-24T07:55:31.606 INFO:tasks.ceph:[osd] osd op complaint time = 180 2023-11-24T07:55:31.606 INFO:tasks.ceph:Setting up mon.a... 2023-11-24T07:55:31.607 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring /etc/ceph/ceph.keyring 2023-11-24T07:55:31.695 INFO:teuthology.orchestra.run.smithi059.stdout:creating /etc/ceph/ceph.keyring 2023-11-24T07:55:31.698 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=mon. /etc/ceph/ceph.keyring 2023-11-24T07:55:31.751 DEBUG:teuthology.orchestra.run.smithi059:> sudo chmod 0644 /etc/ceph/ceph.keyring 2023-11-24T07:55:31.779 DEBUG:tasks.ceph:Ceph mon addresses: [('mon.a', '172.21.15.59'), ('mon.b', '172.21.15.76'), ('mon.c', '[v2:172.21.15.76:3301,v1:172.21.15.76:6790]')] 2023-11-24T07:55:31.779 DEBUG:tasks.ceph:writing out conf {'global': {'chdir': '', 'pid file': '/var/run/ceph/$cluster-$name.pid', 'auth supported': 'cephx', 'filestore xattr use omap': 'true', 'mon clock drift allowed': '1.000', 'osd crush chooseleaf type': '0', 'auth debug': 'true', 'ms die on old message': 'true', 'ms die on bug': 'true', 'mon max pg per osd': '10000', 'mon pg warn max object skew': '0', 'osd_pool_default_pg_autoscale_mode': 'off', 'osd pool default size': '2', 'mon osd allow primary affinity': 'true', 'mon osd allow pg remap': 'true', 'mon warn on legacy crush tunables': 'false', 'mon warn on crush straw calc version zero': 'false', 'mon warn on no sortbitwise': 'false', 'mon warn on osd down out interval zero': 'false', 'mon warn on too few osds': 'false', 'mon_warn_on_pool_pg_num_not_power_of_two': 'false', 'mon_warn_on_pool_no_redundancy': 'false', 'mon_allow_pool_size_one': 'true', 'osd pool default erasure code profile': 'plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd', 'osd default data pool replay window': '5', 'mon allow pool delete': 'true', 'mon cluster log file level': 'debug', 'debug asserts on shutdown': 'true', 'mon health detail to clog': 'false', 'mon host': '172.21.15.59,172.21.15.76,[v2:172.21.15.76:3301,v1:172.21.15.76:6790]', 'ms die on skipped message': False}, 'osd': {'osd journal size': '100', 'osd scrub load threshold': '5.0', 'osd scrub max interval': '600', 'osd mclock profile': 'high_recovery_ops', 'osd recover clone overlap': 'true', 'osd recovery max chunk': '1048576', 'osd debug shutdown': 'true', 'osd debug op order': 'true', 'osd debug verify stray on activate': 'true', 'osd debug trim objects': 'true', 'osd open classes on start': 'true', 'osd debug pg log writeout': 'true', 'osd deep scrub update digest min age': '30', 'osd map max advance': '10', 'journal zero on create': 'true', 'filestore ondisk finisher threads': '3', 'filestore apply finisher threads': '3', 'bdev debug aio': 'true', 'osd debug misdirected ops': 'true', 'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}, 'mgr': {'debug ms': 1, 'debug mgr': 20, 'debug mon': '20', 'debug auth': '20', 'mon reweight min pgs per osd': '4', 'mon reweight min bytes per osd': '10', 'mgr/telemetry/nag': 'false', 'debug client': 20, 'debug finisher': 20}, 'mon': {'debug ms': 1, 'debug mon': 20, 'debug paxos': 20, 'debug auth': '20', 'mon data avail warn': '5', 'mon mgr mkfs grace': '240', 'mon reweight min pgs per osd': '4', 'mon osd reporter subtree level': 'osd', 'mon osd prime pg temp': 'true', 'mon reweight min bytes per osd': '10', 'auth mon ticket ttl': '660', 'auth service ticket ttl': '240', 'mon_warn_on_insecure_global_id_reclaim': 'false', 'mon_warn_on_insecure_global_id_reclaim_allowed': 'false', 'mon_down_mkfs_grace': '2m', 'mon_warn_on_filestore_osds': 'false', 'mon op complaint time': 120}, 'client': {'rgw cache enabled': 'true', 'rgw enable ops log': 'true', 'rgw enable usage log': 'true', 'log file': '/var/log/ceph/$cluster-$name.$pid.log', 'admin socket': '/var/run/ceph/$cluster-$name.$pid.asok', 'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mon.a': {}, 'mon.b': {}, 'mon.c': {}, 'mds.a': {}, 'mds.c': {}, 'mds.b': {}, 'mds.d': {}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}} 2023-11-24T07:55:31.780 DEBUG:teuthology.orchestra.run.smithi059:> set -ex 2023-11-24T07:55:31.780 DEBUG:teuthology.orchestra.run.smithi059:> dd of=/home/ubuntu/cephtest/ceph.tmp.conf 2023-11-24T07:55:31.837 DEBUG:teuthology.orchestra.run.smithi059:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage monmaptool -c /home/ubuntu/cephtest/ceph.tmp.conf --create --clobber --enable-all-features --add a 172.21.15.59 --add b 172.21.15.76 --addv c '[v2:172.21.15.76:3301,v1:172.21.15.76:6790]' --print /home/ubuntu/cephtest/ceph.monmap 2023-11-24T07:55:31.928 INFO:teuthology.orchestra.run.smithi059.stdout:monmaptool: monmap file /home/ubuntu/cephtest/ceph.monmap 2023-11-24T07:55:31.928 INFO:teuthology.orchestra.run.smithi059.stdout:monmaptool: generated fsid 9f0e8741-b8f9-48d5-a2c3-09f45903303d 2023-11-24T07:55:31.928 INFO:teuthology.orchestra.run.smithi059.stdout:setting min_mon_release = pacific 2023-11-24T07:55:31.929 INFO:teuthology.orchestra.run.smithi059.stdout:epoch 0 2023-11-24T07:55:31.929 INFO:teuthology.orchestra.run.smithi059.stdout:fsid 9f0e8741-b8f9-48d5-a2c3-09f45903303d 2023-11-24T07:55:31.929 INFO:teuthology.orchestra.run.smithi059.stdout:last_changed 2023-11-24T07:55:31.927891+0000 2023-11-24T07:55:31.929 INFO:teuthology.orchestra.run.smithi059.stdout:created 2023-11-24T07:55:31.927891+0000 2023-11-24T07:55:31.929 INFO:teuthology.orchestra.run.smithi059.stdout:min_mon_release 16 (pacific) 2023-11-24T07:55:31.929 INFO:teuthology.orchestra.run.smithi059.stdout:election_strategy: 1 2023-11-24T07:55:31.929 INFO:teuthology.orchestra.run.smithi059.stdout:0: [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] mon.a 2023-11-24T07:55:31.929 INFO:teuthology.orchestra.run.smithi059.stdout:1: [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] mon.b 2023-11-24T07:55:31.929 INFO:teuthology.orchestra.run.smithi059.stdout:2: [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] mon.c 2023-11-24T07:55:31.930 INFO:teuthology.orchestra.run.smithi059.stdout:monmaptool: writing epoch 0 to /home/ubuntu/cephtest/ceph.monmap (3 monitors) 2023-11-24T07:55:31.931 DEBUG:teuthology.orchestra.run.smithi059:> rm -- /home/ubuntu/cephtest/ceph.tmp.conf 2023-11-24T07:55:31.948 INFO:tasks.ceph:Writing /etc/ceph/ceph.conf for FSID 9f0e8741-b8f9-48d5-a2c3-09f45903303d... 2023-11-24T07:55:31.949 DEBUG:teuthology.orchestra.run.smithi059:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2023-11-24T07:55:31.991 DEBUG:teuthology.orchestra.run.smithi076:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2023-11-24T07:55:32.035 INFO:teuthology.orchestra.run.smithi059.stdout:[global] 2023-11-24T07:55:32.035 INFO:teuthology.orchestra.run.smithi059.stdout: chdir = "" 2023-11-24T07:55:32.035 INFO:teuthology.orchestra.run.smithi059.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2023-11-24T07:55:32.035 INFO:teuthology.orchestra.run.smithi059.stdout: auth supported = cephx 2023-11-24T07:55:32.035 INFO:teuthology.orchestra.run.smithi059.stdout: 2023-11-24T07:55:32.037 INFO:teuthology.orchestra.run.smithi059.stdout: filestore xattr use omap = true 2023-11-24T07:55:32.037 INFO:teuthology.orchestra.run.smithi059.stdout: 2023-11-24T07:55:32.037 INFO:teuthology.orchestra.run.smithi059.stdout: mon clock drift allowed = 1.000 2023-11-24T07:55:32.037 INFO:teuthology.orchestra.run.smithi059.stdout: 2023-11-24T07:55:32.037 INFO:teuthology.orchestra.run.smithi059.stdout: osd crush chooseleaf type = 0 2023-11-24T07:55:32.037 INFO:teuthology.orchestra.run.smithi059.stdout: auth debug = true 2023-11-24T07:55:32.037 INFO:teuthology.orchestra.run.smithi059.stdout: 2023-11-24T07:55:32.037 INFO:teuthology.orchestra.run.smithi059.stdout: ms die on old message = true 2023-11-24T07:55:32.037 INFO:teuthology.orchestra.run.smithi059.stdout: ms die on bug = true 2023-11-24T07:55:32.038 INFO:teuthology.orchestra.run.smithi059.stdout: 2023-11-24T07:55:32.038 INFO:teuthology.orchestra.run.smithi059.stdout: mon max pg per osd = 10000 # >= luminous 2023-11-24T07:55:32.038 INFO:teuthology.orchestra.run.smithi059.stdout: mon pg warn max object skew = 0 2023-11-24T07:55:32.038 INFO:teuthology.orchestra.run.smithi059.stdout: 2023-11-24T07:55:32.038 INFO:teuthology.orchestra.run.smithi059.stdout: # disable pg_autoscaler by default for new pools 2023-11-24T07:55:32.038 INFO:teuthology.orchestra.run.smithi059.stdout: osd_pool_default_pg_autoscale_mode = off 2023-11-24T07:55:32.038 INFO:teuthology.orchestra.run.smithi059.stdout: 2023-11-24T07:55:32.038 INFO:teuthology.orchestra.run.smithi059.stdout: osd pool default size = 2 2023-11-24T07:55:32.038 INFO:teuthology.orchestra.run.smithi059.stdout: 2023-11-24T07:55:32.038 INFO:teuthology.orchestra.run.smithi059.stdout: mon osd allow primary affinity = true 2023-11-24T07:55:32.038 INFO:teuthology.orchestra.run.smithi059.stdout: mon osd allow pg remap = true 2023-11-24T07:55:32.039 INFO:teuthology.orchestra.run.smithi059.stdout: mon warn on legacy crush tunables = false 2023-11-24T07:55:32.039 INFO:teuthology.orchestra.run.smithi059.stdout: mon warn on crush straw calc version zero = false 2023-11-24T07:55:32.039 INFO:teuthology.orchestra.run.smithi059.stdout: mon warn on no sortbitwise = false 2023-11-24T07:55:32.039 INFO:teuthology.orchestra.run.smithi059.stdout: mon warn on osd down out interval zero = false 2023-11-24T07:55:32.039 INFO:teuthology.orchestra.run.smithi059.stdout: mon warn on too few osds = false 2023-11-24T07:55:32.039 INFO:teuthology.orchestra.run.smithi059.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2023-11-24T07:55:32.039 INFO:teuthology.orchestra.run.smithi059.stdout: mon_warn_on_pool_no_redundancy = false 2023-11-24T07:55:32.039 INFO:teuthology.orchestra.run.smithi059.stdout: mon_allow_pool_size_one = true 2023-11-24T07:55:32.039 INFO:teuthology.orchestra.run.smithi059.stdout: 2023-11-24T07:55:32.039 INFO:teuthology.orchestra.run.smithi059.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2023-11-24T07:55:32.039 INFO:teuthology.orchestra.run.smithi059.stdout: 2023-11-24T07:55:32.039 INFO:teuthology.orchestra.run.smithi059.stdout: osd default data pool replay window = 5 2023-11-24T07:55:32.040 INFO:teuthology.orchestra.run.smithi059.stdout: 2023-11-24T07:55:32.040 INFO:teuthology.orchestra.run.smithi059.stdout: mon allow pool delete = true 2023-11-24T07:55:32.040 INFO:teuthology.orchestra.run.smithi059.stdout: 2023-11-24T07:55:32.040 INFO:teuthology.orchestra.run.smithi059.stdout: mon cluster log file level = debug 2023-11-24T07:55:32.040 INFO:teuthology.orchestra.run.smithi059.stdout: debug asserts on shutdown = true 2023-11-24T07:55:32.040 INFO:teuthology.orchestra.run.smithi059.stdout: mon health detail to clog = false 2023-11-24T07:55:32.040 INFO:teuthology.orchestra.run.smithi059.stdout: mon host = "172.21.15.59,172.21.15.76,[v2:172.21.15.76:3301,v1:172.21.15.76:6790]" 2023-11-24T07:55:32.040 INFO:teuthology.orchestra.run.smithi059.stdout: ms die on skipped message = False 2023-11-24T07:55:32.040 INFO:teuthology.orchestra.run.smithi059.stdout: fsid = 9f0e8741-b8f9-48d5-a2c3-09f45903303d 2023-11-24T07:55:32.040 INFO:teuthology.orchestra.run.smithi059.stdout: 2023-11-24T07:55:32.040 INFO:teuthology.orchestra.run.smithi059.stdout:[osd] 2023-11-24T07:55:32.041 INFO:teuthology.orchestra.run.smithi059.stdout: osd journal size = 100 2023-11-24T07:55:32.041 INFO:teuthology.orchestra.run.smithi059.stdout: 2023-11-24T07:55:32.041 INFO:teuthology.orchestra.run.smithi059.stdout: osd scrub load threshold = 5.0 2023-11-24T07:55:32.041 INFO:teuthology.orchestra.run.smithi059.stdout: osd scrub max interval = 600 2023-11-24T07:55:32.041 INFO:teuthology.orchestra.run.smithi059.stdout: osd mclock profile = high_recovery_ops 2023-11-24T07:55:32.041 INFO:teuthology.orchestra.run.smithi059.stdout: 2023-11-24T07:55:32.041 INFO:teuthology.orchestra.run.smithi059.stdout: osd recover clone overlap = true 2023-11-24T07:55:32.041 INFO:teuthology.orchestra.run.smithi059.stdout: osd recovery max chunk = 1048576 2023-11-24T07:55:32.041 INFO:teuthology.orchestra.run.smithi059.stdout: 2023-11-24T07:55:32.041 INFO:teuthology.orchestra.run.smithi059.stdout: osd debug shutdown = true 2023-11-24T07:55:32.041 INFO:teuthology.orchestra.run.smithi059.stdout: osd debug op order = true 2023-11-24T07:55:32.041 INFO:teuthology.orchestra.run.smithi059.stdout: osd debug verify stray on activate = true 2023-11-24T07:55:32.041 INFO:teuthology.orchestra.run.smithi059.stdout: osd debug trim objects = true 2023-11-24T07:55:32.042 INFO:teuthology.orchestra.run.smithi059.stdout: 2023-11-24T07:55:32.042 INFO:teuthology.orchestra.run.smithi059.stdout: osd open classes on start = true 2023-11-24T07:55:32.042 INFO:teuthology.orchestra.run.smithi059.stdout: osd debug pg log writeout = true 2023-11-24T07:55:32.042 INFO:teuthology.orchestra.run.smithi059.stdout: 2023-11-24T07:55:32.042 INFO:teuthology.orchestra.run.smithi059.stdout: osd deep scrub update digest min age = 30 2023-11-24T07:55:32.042 INFO:teuthology.orchestra.run.smithi059.stdout: 2023-11-24T07:55:32.042 INFO:teuthology.orchestra.run.smithi059.stdout: osd map max advance = 10 2023-11-24T07:55:32.042 INFO:teuthology.orchestra.run.smithi059.stdout: 2023-11-24T07:55:32.043 INFO:teuthology.orchestra.run.smithi059.stdout: journal zero on create = true 2023-11-24T07:55:32.043 INFO:teuthology.orchestra.run.smithi059.stdout: 2023-11-24T07:55:32.043 INFO:teuthology.orchestra.run.smithi059.stdout: filestore ondisk finisher threads = 3 2023-11-24T07:55:32.043 INFO:teuthology.orchestra.run.smithi059.stdout: filestore apply finisher threads = 3 2023-11-24T07:55:32.043 INFO:teuthology.orchestra.run.smithi059.stdout: 2023-11-24T07:55:32.043 INFO:teuthology.orchestra.run.smithi059.stdout: bdev debug aio = true 2023-11-24T07:55:32.043 INFO:teuthology.orchestra.run.smithi059.stdout: osd debug misdirected ops = true 2023-11-24T07:55:32.043 INFO:teuthology.orchestra.run.smithi059.stdout: bdev async discard = True 2023-11-24T07:55:32.043 INFO:teuthology.orchestra.run.smithi059.stdout: bdev enable discard = True 2023-11-24T07:55:32.043 INFO:teuthology.orchestra.run.smithi059.stdout: bluestore allocator = bitmap 2023-11-24T07:55:32.044 INFO:teuthology.orchestra.run.smithi059.stdout: bluestore block size = 96636764160 2023-11-24T07:55:32.044 INFO:teuthology.orchestra.run.smithi059.stdout: bluestore fsck on mount = True 2023-11-24T07:55:32.044 INFO:teuthology.orchestra.run.smithi059.stdout: debug bluefs = 1/20 2023-11-24T07:55:32.044 INFO:teuthology.orchestra.run.smithi059.stdout: debug bluestore = 1/20 2023-11-24T07:55:32.044 INFO:teuthology.orchestra.run.smithi059.stdout: debug ms = 1 2023-11-24T07:55:32.044 INFO:teuthology.orchestra.run.smithi059.stdout: debug osd = 20 2023-11-24T07:55:32.044 INFO:teuthology.orchestra.run.smithi059.stdout: debug rocksdb = 4/10 2023-11-24T07:55:32.044 INFO:teuthology.orchestra.run.smithi059.stdout: mon osd backfillfull_ratio = 0.85 2023-11-24T07:55:32.044 INFO:teuthology.orchestra.run.smithi059.stdout: mon osd full ratio = 0.9 2023-11-24T07:55:32.044 INFO:teuthology.orchestra.run.smithi059.stdout: mon osd nearfull ratio = 0.8 2023-11-24T07:55:32.045 INFO:teuthology.orchestra.run.smithi059.stdout: osd failsafe full ratio = 0.95 2023-11-24T07:55:32.045 INFO:teuthology.orchestra.run.smithi059.stdout: osd objectstore = bluestore 2023-11-24T07:55:32.045 INFO:teuthology.orchestra.run.smithi059.stdout: osd op complaint time = 180 2023-11-24T07:55:32.045 INFO:teuthology.orchestra.run.smithi059.stdout: 2023-11-24T07:55:32.045 INFO:teuthology.orchestra.run.smithi059.stdout:[mgr] 2023-11-24T07:55:32.045 INFO:teuthology.orchestra.run.smithi059.stdout: debug ms = 1 2023-11-24T07:55:32.045 INFO:teuthology.orchestra.run.smithi059.stdout: debug mgr = 20 2023-11-24T07:55:32.045 INFO:teuthology.orchestra.run.smithi059.stdout: debug mon = 20 2023-11-24T07:55:32.045 INFO:teuthology.orchestra.run.smithi059.stdout: debug auth = 20 2023-11-24T07:55:32.045 INFO:teuthology.orchestra.run.smithi059.stdout: mon reweight min pgs per osd = 4 2023-11-24T07:55:32.045 INFO:teuthology.orchestra.run.smithi059.stdout: mon reweight min bytes per osd = 10 2023-11-24T07:55:32.046 INFO:teuthology.orchestra.run.smithi059.stdout: mgr/telemetry/nag = false 2023-11-24T07:55:32.046 INFO:teuthology.orchestra.run.smithi059.stdout: debug client = 20 2023-11-24T07:55:32.046 INFO:teuthology.orchestra.run.smithi059.stdout: debug finisher = 20 2023-11-24T07:55:32.046 INFO:teuthology.orchestra.run.smithi059.stdout: 2023-11-24T07:55:32.046 INFO:teuthology.orchestra.run.smithi059.stdout:[mon] 2023-11-24T07:55:32.046 INFO:teuthology.orchestra.run.smithi059.stdout: debug ms = 1 2023-11-24T07:55:32.046 INFO:teuthology.orchestra.run.smithi059.stdout: debug mon = 20 2023-11-24T07:55:32.046 INFO:teuthology.orchestra.run.smithi059.stdout: debug paxos = 20 2023-11-24T07:55:32.046 INFO:teuthology.orchestra.run.smithi059.stdout: debug auth = 20 2023-11-24T07:55:32.047 INFO:teuthology.orchestra.run.smithi059.stdout: mon data avail warn = 5 2023-11-24T07:55:32.047 INFO:teuthology.orchestra.run.smithi059.stdout: mon mgr mkfs grace = 240 2023-11-24T07:55:32.047 INFO:teuthology.orchestra.run.smithi059.stdout: mon reweight min pgs per osd = 4 2023-11-24T07:55:32.047 INFO:teuthology.orchestra.run.smithi059.stdout: mon osd reporter subtree level = osd 2023-11-24T07:55:32.047 INFO:teuthology.orchestra.run.smithi059.stdout: mon osd prime pg temp = true 2023-11-24T07:55:32.047 INFO:teuthology.orchestra.run.smithi059.stdout: mon reweight min bytes per osd = 10 2023-11-24T07:55:32.047 INFO:teuthology.orchestra.run.smithi059.stdout: 2023-11-24T07:55:32.047 INFO:teuthology.orchestra.run.smithi059.stdout: # rotate auth tickets quickly to exercise renewal paths 2023-11-24T07:55:32.047 INFO:teuthology.orchestra.run.smithi059.stdout: auth mon ticket ttl = 660 # 11m 2023-11-24T07:55:32.047 INFO:teuthology.orchestra.run.smithi059.stdout: auth service ticket ttl = 240 # 4m 2023-11-24T07:55:32.047 INFO:teuthology.orchestra.run.smithi059.stdout: 2023-11-24T07:55:32.047 INFO:teuthology.orchestra.run.smithi059.stdout: # don't complain about insecure global_id in the test suite 2023-11-24T07:55:32.047 INFO:teuthology.orchestra.run.smithi059.stdout: mon_warn_on_insecure_global_id_reclaim = false 2023-11-24T07:55:32.048 INFO:teuthology.orchestra.run.smithi059.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2023-11-24T07:55:32.048 INFO:teuthology.orchestra.run.smithi059.stdout: 2023-11-24T07:55:32.048 INFO:teuthology.orchestra.run.smithi059.stdout: # 1m isn't quite enough 2023-11-24T07:55:32.048 INFO:teuthology.orchestra.run.smithi059.stdout: mon_down_mkfs_grace = 2m 2023-11-24T07:55:32.048 INFO:teuthology.orchestra.run.smithi059.stdout: 2023-11-24T07:55:32.048 INFO:teuthology.orchestra.run.smithi059.stdout: mon_warn_on_filestore_osds = false 2023-11-24T07:55:32.048 INFO:teuthology.orchestra.run.smithi059.stdout: mon op complaint time = 120 2023-11-24T07:55:32.048 INFO:teuthology.orchestra.run.smithi059.stdout: 2023-11-24T07:55:32.048 INFO:teuthology.orchestra.run.smithi059.stdout:[client] 2023-11-24T07:55:32.048 INFO:teuthology.orchestra.run.smithi059.stdout: rgw cache enabled = true 2023-11-24T07:55:32.048 INFO:teuthology.orchestra.run.smithi059.stdout: rgw enable ops log = true 2023-11-24T07:55:32.048 INFO:teuthology.orchestra.run.smithi059.stdout: rgw enable usage log = true 2023-11-24T07:55:32.049 INFO:teuthology.orchestra.run.smithi059.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2023-11-24T07:55:32.049 INFO:teuthology.orchestra.run.smithi059.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2023-11-24T07:55:32.049 INFO:teuthology.orchestra.run.smithi059.stdout: client mount timeout = 600 2023-11-24T07:55:32.049 INFO:teuthology.orchestra.run.smithi059.stdout: debug client = 20 2023-11-24T07:55:32.049 INFO:teuthology.orchestra.run.smithi059.stdout: debug ms = 1 2023-11-24T07:55:32.049 INFO:teuthology.orchestra.run.smithi059.stdout: pid file = "" 2023-11-24T07:55:32.049 INFO:teuthology.orchestra.run.smithi059.stdout: rados mon op timeout = 900 2023-11-24T07:55:32.049 INFO:teuthology.orchestra.run.smithi059.stdout: rados osd op timeout = 900 2023-11-24T07:55:32.049 INFO:teuthology.orchestra.run.smithi059.stdout:[mon.a] 2023-11-24T07:55:32.049 INFO:teuthology.orchestra.run.smithi059.stdout:[mon.b] 2023-11-24T07:55:32.049 INFO:teuthology.orchestra.run.smithi059.stdout:[mon.c] 2023-11-24T07:55:32.049 INFO:teuthology.orchestra.run.smithi059.stdout:[mds.a] 2023-11-24T07:55:32.050 INFO:teuthology.orchestra.run.smithi059.stdout:[mds.c] 2023-11-24T07:55:32.050 INFO:teuthology.orchestra.run.smithi059.stdout:[mds.b] 2023-11-24T07:55:32.050 INFO:teuthology.orchestra.run.smithi059.stdout:[mds.d] 2023-11-24T07:55:32.050 INFO:teuthology.orchestra.run.smithi059.stdout:[mds] 2023-11-24T07:55:32.050 INFO:teuthology.orchestra.run.smithi059.stdout: debug mds = 20 2023-11-24T07:55:32.050 INFO:teuthology.orchestra.run.smithi059.stdout: debug mds balancer = 20 2023-11-24T07:55:32.050 INFO:teuthology.orchestra.run.smithi059.stdout: debug ms = 1 2023-11-24T07:55:32.050 INFO:teuthology.orchestra.run.smithi059.stdout: mds debug frag = True 2023-11-24T07:55:32.050 INFO:teuthology.orchestra.run.smithi059.stdout: mds debug scatterstat = True 2023-11-24T07:55:32.050 INFO:teuthology.orchestra.run.smithi059.stdout: mds op complaint time = 180 2023-11-24T07:55:32.050 INFO:teuthology.orchestra.run.smithi059.stdout: mds verify scatter = True 2023-11-24T07:55:32.051 INFO:teuthology.orchestra.run.smithi059.stdout: osd op complaint time = 180 2023-11-24T07:55:32.051 INFO:teuthology.orchestra.run.smithi059.stdout: rados mon op timeout = 900 2023-11-24T07:55:32.051 INFO:teuthology.orchestra.run.smithi059.stdout: rados osd op timeout = 900 2023-11-24T07:55:32.051 INFO:teuthology.orchestra.run.smithi076.stdout:[global] 2023-11-24T07:55:32.051 INFO:teuthology.orchestra.run.smithi076.stdout: chdir = "" 2023-11-24T07:55:32.052 INFO:teuthology.orchestra.run.smithi076.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2023-11-24T07:55:32.052 INFO:teuthology.orchestra.run.smithi076.stdout: auth supported = cephx 2023-11-24T07:55:32.052 INFO:teuthology.orchestra.run.smithi076.stdout: 2023-11-24T07:55:32.052 INFO:teuthology.orchestra.run.smithi076.stdout: filestore xattr use omap = true 2023-11-24T07:55:32.052 INFO:teuthology.orchestra.run.smithi076.stdout: 2023-11-24T07:55:32.052 INFO:teuthology.orchestra.run.smithi076.stdout: mon clock drift allowed = 1.000 2023-11-24T07:55:32.052 INFO:teuthology.orchestra.run.smithi076.stdout: 2023-11-24T07:55:32.052 INFO:teuthology.orchestra.run.smithi076.stdout: osd crush chooseleaf type = 0 2023-11-24T07:55:32.052 INFO:teuthology.orchestra.run.smithi076.stdout: auth debug = true 2023-11-24T07:55:32.052 INFO:teuthology.orchestra.run.smithi076.stdout: 2023-11-24T07:55:32.052 INFO:teuthology.orchestra.run.smithi076.stdout: ms die on old message = true 2023-11-24T07:55:32.052 INFO:teuthology.orchestra.run.smithi076.stdout: ms die on bug = true 2023-11-24T07:55:32.052 INFO:teuthology.orchestra.run.smithi076.stdout: 2023-11-24T07:55:32.053 INFO:teuthology.orchestra.run.smithi076.stdout: mon max pg per osd = 10000 # >= luminous 2023-11-24T07:55:32.053 INFO:teuthology.orchestra.run.smithi076.stdout: mon pg warn max object skew = 0 2023-11-24T07:55:32.053 INFO:teuthology.orchestra.run.smithi076.stdout: 2023-11-24T07:55:32.053 INFO:teuthology.orchestra.run.smithi076.stdout: # disable pg_autoscaler by default for new pools 2023-11-24T07:55:32.053 INFO:teuthology.orchestra.run.smithi076.stdout: osd_pool_default_pg_autoscale_mode = off 2023-11-24T07:55:32.053 INFO:teuthology.orchestra.run.smithi076.stdout: 2023-11-24T07:55:32.053 INFO:teuthology.orchestra.run.smithi076.stdout: osd pool default size = 2 2023-11-24T07:55:32.053 INFO:teuthology.orchestra.run.smithi076.stdout: 2023-11-24T07:55:32.053 INFO:teuthology.orchestra.run.smithi076.stdout: mon osd allow primary affinity = true 2023-11-24T07:55:32.053 INFO:teuthology.orchestra.run.smithi076.stdout: mon osd allow pg remap = true 2023-11-24T07:55:32.054 INFO:teuthology.orchestra.run.smithi076.stdout: mon warn on legacy crush tunables = false 2023-11-24T07:55:32.054 INFO:teuthology.orchestra.run.smithi076.stdout: mon warn on crush straw calc version zero = false 2023-11-24T07:55:32.054 INFO:teuthology.orchestra.run.smithi076.stdout: mon warn on no sortbitwise = false 2023-11-24T07:55:32.054 INFO:teuthology.orchestra.run.smithi076.stdout: mon warn on osd down out interval zero = false 2023-11-24T07:55:32.054 INFO:teuthology.orchestra.run.smithi076.stdout: mon warn on too few osds = false 2023-11-24T07:55:32.054 INFO:teuthology.orchestra.run.smithi076.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2023-11-24T07:55:32.054 INFO:teuthology.orchestra.run.smithi076.stdout: mon_warn_on_pool_no_redundancy = false 2023-11-24T07:55:32.054 INFO:teuthology.orchestra.run.smithi076.stdout: mon_allow_pool_size_one = true 2023-11-24T07:55:32.054 INFO:teuthology.orchestra.run.smithi076.stdout: 2023-11-24T07:55:32.054 INFO:teuthology.orchestra.run.smithi076.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2023-11-24T07:55:32.054 INFO:teuthology.orchestra.run.smithi076.stdout: 2023-11-24T07:55:32.054 INFO:teuthology.orchestra.run.smithi076.stdout: osd default data pool replay window = 5 2023-11-24T07:55:32.055 INFO:teuthology.orchestra.run.smithi076.stdout: 2023-11-24T07:55:32.055 INFO:teuthology.orchestra.run.smithi076.stdout: mon allow pool delete = true 2023-11-24T07:55:32.055 INFO:teuthology.orchestra.run.smithi076.stdout: 2023-11-24T07:55:32.055 INFO:teuthology.orchestra.run.smithi076.stdout: mon cluster log file level = debug 2023-11-24T07:55:32.055 INFO:teuthology.orchestra.run.smithi076.stdout: debug asserts on shutdown = true 2023-11-24T07:55:32.055 INFO:teuthology.orchestra.run.smithi076.stdout: mon health detail to clog = false 2023-11-24T07:55:32.055 INFO:teuthology.orchestra.run.smithi076.stdout: mon host = "172.21.15.59,172.21.15.76,[v2:172.21.15.76:3301,v1:172.21.15.76:6790]" 2023-11-24T07:55:32.055 INFO:teuthology.orchestra.run.smithi076.stdout: ms die on skipped message = False 2023-11-24T07:55:32.055 INFO:teuthology.orchestra.run.smithi076.stdout: fsid = 9f0e8741-b8f9-48d5-a2c3-09f45903303d 2023-11-24T07:55:32.055 INFO:teuthology.orchestra.run.smithi076.stdout: 2023-11-24T07:55:32.056 INFO:teuthology.orchestra.run.smithi076.stdout:[osd] 2023-11-24T07:55:32.056 INFO:teuthology.orchestra.run.smithi076.stdout: osd journal size = 100 2023-11-24T07:55:32.056 INFO:teuthology.orchestra.run.smithi076.stdout: 2023-11-24T07:55:32.056 INFO:teuthology.orchestra.run.smithi076.stdout: osd scrub load threshold = 5.0 2023-11-24T07:55:32.056 INFO:teuthology.orchestra.run.smithi076.stdout: osd scrub max interval = 600 2023-11-24T07:55:32.056 INFO:teuthology.orchestra.run.smithi076.stdout: osd mclock profile = high_recovery_ops 2023-11-24T07:55:32.056 INFO:teuthology.orchestra.run.smithi076.stdout: 2023-11-24T07:55:32.056 INFO:teuthology.orchestra.run.smithi076.stdout: osd recover clone overlap = true 2023-11-24T07:55:32.056 INFO:teuthology.orchestra.run.smithi076.stdout: osd recovery max chunk = 1048576 2023-11-24T07:55:32.056 INFO:teuthology.orchestra.run.smithi076.stdout: 2023-11-24T07:55:32.056 INFO:teuthology.orchestra.run.smithi076.stdout: osd debug shutdown = true 2023-11-24T07:55:32.056 INFO:teuthology.orchestra.run.smithi076.stdout: osd debug op order = true 2023-11-24T07:55:32.056 INFO:teuthology.orchestra.run.smithi076.stdout: osd debug verify stray on activate = true 2023-11-24T07:55:32.057 INFO:teuthology.orchestra.run.smithi076.stdout: osd debug trim objects = true 2023-11-24T07:55:32.057 INFO:teuthology.orchestra.run.smithi076.stdout: 2023-11-24T07:55:32.057 INFO:teuthology.orchestra.run.smithi076.stdout: osd open classes on start = true 2023-11-24T07:55:32.057 INFO:teuthology.orchestra.run.smithi076.stdout: osd debug pg log writeout = true 2023-11-24T07:55:32.057 INFO:teuthology.orchestra.run.smithi076.stdout: 2023-11-24T07:55:32.057 INFO:teuthology.orchestra.run.smithi076.stdout: osd deep scrub update digest min age = 30 2023-11-24T07:55:32.057 INFO:teuthology.orchestra.run.smithi076.stdout: 2023-11-24T07:55:32.057 INFO:teuthology.orchestra.run.smithi076.stdout: osd map max advance = 10 2023-11-24T07:55:32.057 INFO:teuthology.orchestra.run.smithi076.stdout: 2023-11-24T07:55:32.057 INFO:teuthology.orchestra.run.smithi076.stdout: journal zero on create = true 2023-11-24T07:55:32.057 INFO:teuthology.orchestra.run.smithi076.stdout: 2023-11-24T07:55:32.057 INFO:teuthology.orchestra.run.smithi076.stdout: filestore ondisk finisher threads = 3 2023-11-24T07:55:32.057 INFO:teuthology.orchestra.run.smithi076.stdout: filestore apply finisher threads = 3 2023-11-24T07:55:32.058 INFO:teuthology.orchestra.run.smithi076.stdout: 2023-11-24T07:55:32.058 INFO:teuthology.orchestra.run.smithi076.stdout: bdev debug aio = true 2023-11-24T07:55:32.058 INFO:teuthology.orchestra.run.smithi076.stdout: osd debug misdirected ops = true 2023-11-24T07:55:32.058 INFO:teuthology.orchestra.run.smithi076.stdout: bdev async discard = True 2023-11-24T07:55:32.058 INFO:teuthology.orchestra.run.smithi076.stdout: bdev enable discard = True 2023-11-24T07:55:32.058 INFO:teuthology.orchestra.run.smithi076.stdout: bluestore allocator = bitmap 2023-11-24T07:55:32.058 INFO:teuthology.orchestra.run.smithi076.stdout: bluestore block size = 96636764160 2023-11-24T07:55:32.058 INFO:teuthology.orchestra.run.smithi076.stdout: bluestore fsck on mount = True 2023-11-24T07:55:32.058 INFO:teuthology.orchestra.run.smithi076.stdout: debug bluefs = 1/20 2023-11-24T07:55:32.058 INFO:teuthology.orchestra.run.smithi076.stdout: debug bluestore = 1/20 2023-11-24T07:55:32.058 INFO:teuthology.orchestra.run.smithi076.stdout: debug ms = 1 2023-11-24T07:55:32.059 INFO:teuthology.orchestra.run.smithi076.stdout: debug osd = 20 2023-11-24T07:55:32.059 INFO:teuthology.orchestra.run.smithi076.stdout: debug rocksdb = 4/10 2023-11-24T07:55:32.059 INFO:teuthology.orchestra.run.smithi076.stdout: mon osd backfillfull_ratio = 0.85 2023-11-24T07:55:32.059 INFO:teuthology.orchestra.run.smithi076.stdout: mon osd full ratio = 0.9 2023-11-24T07:55:32.059 INFO:teuthology.orchestra.run.smithi076.stdout: mon osd nearfull ratio = 0.8 2023-11-24T07:55:32.059 INFO:teuthology.orchestra.run.smithi076.stdout: osd failsafe full ratio = 0.95 2023-11-24T07:55:32.059 INFO:teuthology.orchestra.run.smithi076.stdout: osd objectstore = bluestore 2023-11-24T07:55:32.059 INFO:teuthology.orchestra.run.smithi076.stdout: osd op complaint time = 180 2023-11-24T07:55:32.059 INFO:teuthology.orchestra.run.smithi076.stdout: 2023-11-24T07:55:32.059 INFO:teuthology.orchestra.run.smithi076.stdout:[mgr] 2023-11-24T07:55:32.059 INFO:teuthology.orchestra.run.smithi076.stdout: debug ms = 1 2023-11-24T07:55:32.059 INFO:teuthology.orchestra.run.smithi076.stdout: debug mgr = 20 2023-11-24T07:55:32.060 INFO:teuthology.orchestra.run.smithi076.stdout: debug mon = 20 2023-11-24T07:55:32.060 INFO:teuthology.orchestra.run.smithi076.stdout: debug auth = 20 2023-11-24T07:55:32.060 INFO:teuthology.orchestra.run.smithi076.stdout: mon reweight min pgs per osd = 4 2023-11-24T07:55:32.060 INFO:teuthology.orchestra.run.smithi076.stdout: mon reweight min bytes per osd = 10 2023-11-24T07:55:32.060 INFO:teuthology.orchestra.run.smithi076.stdout: mgr/telemetry/nag = false 2023-11-24T07:55:32.060 INFO:teuthology.orchestra.run.smithi076.stdout: debug client = 20 2023-11-24T07:55:32.060 INFO:teuthology.orchestra.run.smithi076.stdout: debug finisher = 20 2023-11-24T07:55:32.060 INFO:teuthology.orchestra.run.smithi076.stdout: 2023-11-24T07:55:32.060 INFO:teuthology.orchestra.run.smithi076.stdout:[mon] 2023-11-24T07:55:32.060 INFO:teuthology.orchestra.run.smithi076.stdout: debug ms = 1 2023-11-24T07:55:32.060 INFO:teuthology.orchestra.run.smithi076.stdout: debug mon = 20 2023-11-24T07:55:32.060 INFO:teuthology.orchestra.run.smithi076.stdout: debug paxos = 20 2023-11-24T07:55:32.060 INFO:teuthology.orchestra.run.smithi076.stdout: debug auth = 20 2023-11-24T07:55:32.061 INFO:teuthology.orchestra.run.smithi076.stdout: mon data avail warn = 5 2023-11-24T07:55:32.061 INFO:teuthology.orchestra.run.smithi076.stdout: mon mgr mkfs grace = 240 2023-11-24T07:55:32.061 INFO:teuthology.orchestra.run.smithi076.stdout: mon reweight min pgs per osd = 4 2023-11-24T07:55:32.061 INFO:teuthology.orchestra.run.smithi076.stdout: mon osd reporter subtree level = osd 2023-11-24T07:55:32.061 INFO:teuthology.orchestra.run.smithi076.stdout: mon osd prime pg temp = true 2023-11-24T07:55:32.061 INFO:teuthology.orchestra.run.smithi076.stdout: mon reweight min bytes per osd = 10 2023-11-24T07:55:32.061 INFO:teuthology.orchestra.run.smithi076.stdout: 2023-11-24T07:55:32.061 INFO:teuthology.orchestra.run.smithi076.stdout: # rotate auth tickets quickly to exercise renewal paths 2023-11-24T07:55:32.061 INFO:teuthology.orchestra.run.smithi076.stdout: auth mon ticket ttl = 660 # 11m 2023-11-24T07:55:32.061 INFO:teuthology.orchestra.run.smithi076.stdout: auth service ticket ttl = 240 # 4m 2023-11-24T07:55:32.061 INFO:teuthology.orchestra.run.smithi076.stdout: 2023-11-24T07:55:32.061 INFO:teuthology.orchestra.run.smithi076.stdout: # don't complain about insecure global_id in the test suite 2023-11-24T07:55:32.061 INFO:teuthology.orchestra.run.smithi076.stdout: mon_warn_on_insecure_global_id_reclaim = false 2023-11-24T07:55:32.062 INFO:teuthology.orchestra.run.smithi076.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2023-11-24T07:55:32.062 INFO:teuthology.orchestra.run.smithi076.stdout: 2023-11-24T07:55:32.062 INFO:teuthology.orchestra.run.smithi076.stdout: # 1m isn't quite enough 2023-11-24T07:55:32.062 INFO:teuthology.orchestra.run.smithi076.stdout: mon_down_mkfs_grace = 2m 2023-11-24T07:55:32.062 INFO:teuthology.orchestra.run.smithi076.stdout: 2023-11-24T07:55:32.062 INFO:teuthology.orchestra.run.smithi076.stdout: mon_warn_on_filestore_osds = false 2023-11-24T07:55:32.062 INFO:teuthology.orchestra.run.smithi076.stdout: mon op complaint time = 120 2023-11-24T07:55:32.062 INFO:teuthology.orchestra.run.smithi076.stdout: 2023-11-24T07:55:32.063 INFO:teuthology.orchestra.run.smithi076.stdout:[client] 2023-11-24T07:55:32.063 INFO:teuthology.orchestra.run.smithi076.stdout: rgw cache enabled = true 2023-11-24T07:55:32.063 INFO:teuthology.orchestra.run.smithi076.stdout: rgw enable ops log = true 2023-11-24T07:55:32.063 INFO:teuthology.orchestra.run.smithi076.stdout: rgw enable usage log = true 2023-11-24T07:55:32.063 INFO:teuthology.orchestra.run.smithi076.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2023-11-24T07:55:32.063 INFO:teuthology.orchestra.run.smithi076.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2023-11-24T07:55:32.063 INFO:teuthology.orchestra.run.smithi076.stdout: client mount timeout = 600 2023-11-24T07:55:32.063 INFO:teuthology.orchestra.run.smithi076.stdout: debug client = 20 2023-11-24T07:55:32.063 INFO:teuthology.orchestra.run.smithi076.stdout: debug ms = 1 2023-11-24T07:55:32.063 INFO:teuthology.orchestra.run.smithi076.stdout: pid file = "" 2023-11-24T07:55:32.063 INFO:teuthology.orchestra.run.smithi076.stdout: rados mon op timeout = 900 2023-11-24T07:55:32.063 INFO:teuthology.orchestra.run.smithi076.stdout: rados osd op timeout = 900 2023-11-24T07:55:32.064 INFO:teuthology.orchestra.run.smithi076.stdout:[mon.a] 2023-11-24T07:55:32.064 INFO:teuthology.orchestra.run.smithi076.stdout:[mon.b] 2023-11-24T07:55:32.064 INFO:teuthology.orchestra.run.smithi076.stdout:[mon.c] 2023-11-24T07:55:32.064 INFO:teuthology.orchestra.run.smithi076.stdout:[mds.a] 2023-11-24T07:55:32.064 INFO:teuthology.orchestra.run.smithi076.stdout:[mds.c] 2023-11-24T07:55:32.064 INFO:teuthology.orchestra.run.smithi076.stdout:[mds.b] 2023-11-24T07:55:32.064 INFO:teuthology.orchestra.run.smithi076.stdout:[mds.d] 2023-11-24T07:55:32.064 INFO:teuthology.orchestra.run.smithi076.stdout:[mds] 2023-11-24T07:55:32.064 INFO:teuthology.orchestra.run.smithi076.stdout: debug mds = 20 2023-11-24T07:55:32.064 INFO:teuthology.orchestra.run.smithi076.stdout: debug mds balancer = 20 2023-11-24T07:55:32.064 INFO:teuthology.orchestra.run.smithi076.stdout: debug ms = 1 2023-11-24T07:55:32.064 INFO:teuthology.orchestra.run.smithi076.stdout: mds debug frag = True 2023-11-24T07:55:32.064 INFO:teuthology.orchestra.run.smithi076.stdout: mds debug scatterstat = True 2023-11-24T07:55:32.065 INFO:teuthology.orchestra.run.smithi076.stdout: mds op complaint time = 180 2023-11-24T07:55:32.065 INFO:teuthology.orchestra.run.smithi076.stdout: mds verify scatter = True 2023-11-24T07:55:32.065 INFO:teuthology.orchestra.run.smithi076.stdout: osd op complaint time = 180 2023-11-24T07:55:32.065 INFO:teuthology.orchestra.run.smithi076.stdout: rados mon op timeout = 900 2023-11-24T07:55:32.065 INFO:teuthology.orchestra.run.smithi076.stdout: rados osd op timeout = 900 2023-11-24T07:55:32.066 INFO:tasks.ceph:Creating admin key on mon.a... 2023-11-24T07:55:32.067 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=client.admin --cap mon 'allow *' --cap osd 'allow *' --cap mds 'allow *' --cap mgr 'allow *' /etc/ceph/ceph.keyring 2023-11-24T07:55:32.165 INFO:tasks.ceph:Copying monmap to all nodes... 2023-11-24T07:55:32.165 DEBUG:teuthology.orchestra.run.smithi059:> set -ex 2023-11-24T07:55:32.165 DEBUG:teuthology.orchestra.run.smithi059:> dd if=/etc/ceph/ceph.keyring of=/dev/stdout 2023-11-24T07:55:32.181 DEBUG:teuthology.orchestra.run.smithi059:> set -ex 2023-11-24T07:55:32.182 DEBUG:teuthology.orchestra.run.smithi059:> dd if=/home/ubuntu/cephtest/ceph.monmap of=/dev/stdout 2023-11-24T07:55:32.238 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi059.front.sepia.ceph.com 2023-11-24T07:55:32.238 DEBUG:teuthology.orchestra.run.smithi059:> set -ex 2023-11-24T07:55:32.239 DEBUG:teuthology.orchestra.run.smithi059:> sudo dd of=/etc/ceph/ceph.keyring 2023-11-24T07:55:32.239 DEBUG:teuthology.orchestra.run.smithi059:> sudo chmod 0644 /etc/ceph/ceph.keyring 2023-11-24T07:55:32.316 DEBUG:teuthology.orchestra.run.smithi059:> set -ex 2023-11-24T07:55:32.316 DEBUG:teuthology.orchestra.run.smithi059:> dd of=/home/ubuntu/cephtest/ceph.monmap 2023-11-24T07:55:32.373 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi076.front.sepia.ceph.com 2023-11-24T07:55:32.374 DEBUG:teuthology.orchestra.run.smithi076:> set -ex 2023-11-24T07:55:32.374 DEBUG:teuthology.orchestra.run.smithi076:> sudo dd of=/etc/ceph/ceph.keyring 2023-11-24T07:55:32.374 DEBUG:teuthology.orchestra.run.smithi076:> sudo chmod 0644 /etc/ceph/ceph.keyring 2023-11-24T07:55:32.418 DEBUG:teuthology.orchestra.run.smithi076:> set -ex 2023-11-24T07:55:32.418 DEBUG:teuthology.orchestra.run.smithi076:> dd of=/home/ubuntu/cephtest/ceph.monmap 2023-11-24T07:55:32.479 INFO:tasks.ceph:Setting up mon nodes... 2023-11-24T07:55:32.479 INFO:tasks.ceph:Setting up mgr nodes... 2023-11-24T07:55:32.479 DEBUG:teuthology.orchestra.run.smithi059:> sudo mkdir -p /var/lib/ceph/mgr/ceph-y && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.y /var/lib/ceph/mgr/ceph-y/keyring 2023-11-24T07:55:32.535 INFO:teuthology.orchestra.run.smithi059.stdout:creating /var/lib/ceph/mgr/ceph-y/keyring 2023-11-24T07:55:32.537 DEBUG:teuthology.orchestra.run.smithi076:> sudo mkdir -p /var/lib/ceph/mgr/ceph-x && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.x /var/lib/ceph/mgr/ceph-x/keyring 2023-11-24T07:55:32.625 INFO:teuthology.orchestra.run.smithi076.stdout:creating /var/lib/ceph/mgr/ceph-x/keyring 2023-11-24T07:55:32.628 INFO:tasks.ceph:Setting up mds nodes... 2023-11-24T07:55:32.628 DEBUG:teuthology.orchestra.run.smithi059:> sudo mkdir -p /var/lib/ceph/mds/ceph-a && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.a /var/lib/ceph/mds/ceph-a/keyring 2023-11-24T07:55:32.684 INFO:teuthology.orchestra.run.smithi059.stdout:creating /var/lib/ceph/mds/ceph-a/keyring 2023-11-24T07:55:32.686 DEBUG:teuthology.orchestra.run.smithi059:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-a 2023-11-24T07:55:32.713 DEBUG:teuthology.orchestra.run.smithi059:> sudo mkdir -p /var/lib/ceph/mds/ceph-c && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.c /var/lib/ceph/mds/ceph-c/keyring 2023-11-24T07:55:32.808 INFO:teuthology.orchestra.run.smithi059.stdout:creating /var/lib/ceph/mds/ceph-c/keyring 2023-11-24T07:55:32.811 DEBUG:teuthology.orchestra.run.smithi059:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-c 2023-11-24T07:55:32.838 DEBUG:teuthology.orchestra.run.smithi076:> sudo mkdir -p /var/lib/ceph/mds/ceph-b && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.b /var/lib/ceph/mds/ceph-b/keyring 2023-11-24T07:55:32.901 INFO:teuthology.orchestra.run.smithi076.stdout:creating /var/lib/ceph/mds/ceph-b/keyring 2023-11-24T07:55:32.904 DEBUG:teuthology.orchestra.run.smithi076:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-b 2023-11-24T07:55:32.933 DEBUG:teuthology.orchestra.run.smithi076:> sudo mkdir -p /var/lib/ceph/mds/ceph-d && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.d /var/lib/ceph/mds/ceph-d/keyring 2023-11-24T07:55:33.029 INFO:teuthology.orchestra.run.smithi076.stdout:creating /var/lib/ceph/mds/ceph-d/keyring 2023-11-24T07:55:33.031 DEBUG:teuthology.orchestra.run.smithi076:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-d 2023-11-24T07:55:33.058 INFO:tasks.ceph_client:Setting up client nodes... 2023-11-24T07:55:33.058 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.0 /etc/ceph/ceph.client.0.keyring && sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2023-11-24T07:55:33.103 INFO:teuthology.orchestra.run.smithi059.stdout:creating /etc/ceph/ceph.client.0.keyring 2023-11-24T07:55:33.116 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.2 /etc/ceph/ceph.client.2.keyring && sudo chmod 0644 /etc/ceph/ceph.client.2.keyring 2023-11-24T07:55:33.161 INFO:teuthology.orchestra.run.smithi059.stdout:creating /etc/ceph/ceph.client.2.keyring 2023-11-24T07:55:33.174 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.1 /etc/ceph/ceph.client.1.keyring && sudo chmod 0644 /etc/ceph/ceph.client.1.keyring 2023-11-24T07:55:33.217 INFO:teuthology.orchestra.run.smithi076.stdout:creating /etc/ceph/ceph.client.1.keyring 2023-11-24T07:55:33.229 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.3 /etc/ceph/ceph.client.3.keyring && sudo chmod 0644 /etc/ceph/ceph.client.3.keyring 2023-11-24T07:55:33.313 INFO:teuthology.orchestra.run.smithi076.stdout:creating /etc/ceph/ceph.client.3.keyring 2023-11-24T07:55:33.325 INFO:tasks.ceph:Running mkfs on osd nodes... 2023-11-24T07:55:33.325 INFO:tasks.ceph:ctx.disk_config.remote_to_roles_to_dev: {Remote(name='ubuntu@smithi059.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi076.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}} 2023-11-24T07:55:33.326 DEBUG:teuthology.orchestra.run.smithi059:> sudo mkdir -p /var/lib/ceph/osd/ceph-0 2023-11-24T07:55:33.354 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2023-11-24T07:55:33.354 INFO:tasks.ceph:role: osd.0 2023-11-24T07:55:33.354 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi059.front.sepia.ceph.com 2023-11-24T07:55:33.354 DEBUG:teuthology.orchestra.run.smithi059:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2023-11-24T07:55:33.429 INFO:teuthology.orchestra.run.smithi059.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2023-11-24T07:55:33.429 INFO:teuthology.orchestra.run.smithi059.stdout: = sectsz=512 attr=2, projid32bit=1 2023-11-24T07:55:33.430 INFO:teuthology.orchestra.run.smithi059.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-11-24T07:55:33.430 INFO:teuthology.orchestra.run.smithi059.stdout: = reflink=1 bigtime=0 inobtcount=0 2023-11-24T07:55:33.430 INFO:teuthology.orchestra.run.smithi059.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-11-24T07:55:33.430 INFO:teuthology.orchestra.run.smithi059.stdout: = sunit=0 swidth=0 blks 2023-11-24T07:55:33.430 INFO:teuthology.orchestra.run.smithi059.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-11-24T07:55:33.431 INFO:teuthology.orchestra.run.smithi059.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-11-24T07:55:33.431 INFO:teuthology.orchestra.run.smithi059.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-11-24T07:55:33.431 INFO:teuthology.orchestra.run.smithi059.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-11-24T07:55:33.866 INFO:teuthology.orchestra.run.smithi059.stdout:Discarding blocks...Done. 2023-11-24T07:55:33.868 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi059.front.sepia.ceph.com -o noatime 2023-11-24T07:55:33.869 DEBUG:teuthology.orchestra.run.smithi059:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-0 2023-11-24T07:55:34.047 DEBUG:teuthology.orchestra.run.smithi059:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-0 2023-11-24T07:55:34.082 DEBUG:teuthology.orchestra.run.smithi059:> sudo mkdir -p /var/lib/ceph/osd/ceph-1 2023-11-24T07:55:34.153 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2023-11-24T07:55:34.153 INFO:tasks.ceph:role: osd.1 2023-11-24T07:55:34.153 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi059.front.sepia.ceph.com 2023-11-24T07:55:34.153 DEBUG:teuthology.orchestra.run.smithi059:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2023-11-24T07:55:34.229 INFO:teuthology.orchestra.run.smithi059.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2023-11-24T07:55:34.229 INFO:teuthology.orchestra.run.smithi059.stdout: = sectsz=512 attr=2, projid32bit=1 2023-11-24T07:55:34.229 INFO:teuthology.orchestra.run.smithi059.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-11-24T07:55:34.229 INFO:teuthology.orchestra.run.smithi059.stdout: = reflink=1 bigtime=0 inobtcount=0 2023-11-24T07:55:34.229 INFO:teuthology.orchestra.run.smithi059.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-11-24T07:55:34.230 INFO:teuthology.orchestra.run.smithi059.stdout: = sunit=0 swidth=0 blks 2023-11-24T07:55:34.230 INFO:teuthology.orchestra.run.smithi059.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-11-24T07:55:34.230 INFO:teuthology.orchestra.run.smithi059.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-11-24T07:55:34.230 INFO:teuthology.orchestra.run.smithi059.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-11-24T07:55:34.230 INFO:teuthology.orchestra.run.smithi059.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-11-24T07:55:34.649 INFO:teuthology.orchestra.run.smithi059.stdout:Discarding blocks...Done. 2023-11-24T07:55:34.651 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi059.front.sepia.ceph.com -o noatime 2023-11-24T07:55:34.652 DEBUG:teuthology.orchestra.run.smithi059:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-1 2023-11-24T07:55:34.759 DEBUG:teuthology.orchestra.run.smithi059:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-1 2023-11-24T07:55:34.795 DEBUG:teuthology.orchestra.run.smithi059:> sudo mkdir -p /var/lib/ceph/osd/ceph-2 2023-11-24T07:55:34.865 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2023-11-24T07:55:34.866 INFO:tasks.ceph:role: osd.2 2023-11-24T07:55:34.866 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi059.front.sepia.ceph.com 2023-11-24T07:55:34.866 DEBUG:teuthology.orchestra.run.smithi059:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2023-11-24T07:55:34.942 INFO:teuthology.orchestra.run.smithi059.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2023-11-24T07:55:34.943 INFO:teuthology.orchestra.run.smithi059.stdout: = sectsz=512 attr=2, projid32bit=1 2023-11-24T07:55:34.943 INFO:teuthology.orchestra.run.smithi059.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-11-24T07:55:34.943 INFO:teuthology.orchestra.run.smithi059.stdout: = reflink=1 bigtime=0 inobtcount=0 2023-11-24T07:55:34.943 INFO:teuthology.orchestra.run.smithi059.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-11-24T07:55:34.943 INFO:teuthology.orchestra.run.smithi059.stdout: = sunit=0 swidth=0 blks 2023-11-24T07:55:34.943 INFO:teuthology.orchestra.run.smithi059.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-11-24T07:55:34.944 INFO:teuthology.orchestra.run.smithi059.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-11-24T07:55:34.944 INFO:teuthology.orchestra.run.smithi059.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-11-24T07:55:34.944 INFO:teuthology.orchestra.run.smithi059.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-11-24T07:55:35.385 INFO:teuthology.orchestra.run.smithi059.stdout:Discarding blocks...Done. 2023-11-24T07:55:35.386 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi059.front.sepia.ceph.com -o noatime 2023-11-24T07:55:35.386 DEBUG:teuthology.orchestra.run.smithi059:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-2 2023-11-24T07:55:35.539 DEBUG:teuthology.orchestra.run.smithi059:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-2 2023-11-24T07:55:35.576 DEBUG:teuthology.orchestra.run.smithi059:> sudo mkdir -p /var/lib/ceph/osd/ceph-3 2023-11-24T07:55:35.646 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2023-11-24T07:55:35.646 INFO:tasks.ceph:role: osd.3 2023-11-24T07:55:35.646 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi059.front.sepia.ceph.com 2023-11-24T07:55:35.647 DEBUG:teuthology.orchestra.run.smithi059:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2023-11-24T07:55:35.723 INFO:teuthology.orchestra.run.smithi059.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2023-11-24T07:55:35.723 INFO:teuthology.orchestra.run.smithi059.stdout: = sectsz=512 attr=2, projid32bit=1 2023-11-24T07:55:35.724 INFO:teuthology.orchestra.run.smithi059.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-11-24T07:55:35.724 INFO:teuthology.orchestra.run.smithi059.stdout: = reflink=1 bigtime=0 inobtcount=0 2023-11-24T07:55:35.724 INFO:teuthology.orchestra.run.smithi059.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-11-24T07:55:35.724 INFO:teuthology.orchestra.run.smithi059.stdout: = sunit=0 swidth=0 blks 2023-11-24T07:55:35.724 INFO:teuthology.orchestra.run.smithi059.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-11-24T07:55:35.724 INFO:teuthology.orchestra.run.smithi059.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-11-24T07:55:35.724 INFO:teuthology.orchestra.run.smithi059.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-11-24T07:55:35.725 INFO:teuthology.orchestra.run.smithi059.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-11-24T07:55:36.164 INFO:teuthology.orchestra.run.smithi059.stdout:Discarding blocks...Done. 2023-11-24T07:55:36.166 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi059.front.sepia.ceph.com -o noatime 2023-11-24T07:55:36.166 DEBUG:teuthology.orchestra.run.smithi059:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-3 2023-11-24T07:55:36.327 DEBUG:teuthology.orchestra.run.smithi059:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-3 2023-11-24T07:55:36.362 DEBUG:teuthology.orchestra.run.smithi059:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 0 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-11-24T07:55:36.491 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:55:36.489+0000 7fac4c014640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-0/keyring: can't open /var/lib/ceph/osd/ceph-0/keyring: (2) No such file or directory 2023-11-24T07:55:36.492 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:55:36.489+0000 7fac4c014640 -1 created new key in keyring /var/lib/ceph/osd/ceph-0/keyring 2023-11-24T07:55:36.492 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:55:36.489+0000 7fac4c014640 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2023-11-24T07:55:36.492 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:55:36.489+0000 7fac4c014640 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2023-11-24T07:55:36.492 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:55:36.489+0000 7fac4c014640 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2023-11-24T07:55:36.492 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:55:36.490+0000 7fac4c014640 -1 bluestore(/var/lib/ceph/osd/ceph-0) _read_fsid unparsable uuid 2023-11-24T07:55:39.361 DEBUG:teuthology.orchestra.run.smithi059:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-0 2023-11-24T07:55:39.394 DEBUG:teuthology.orchestra.run.smithi059:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 1 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-11-24T07:55:39.486 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:55:39.484+0000 7f8e37b0a640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-1/keyring: can't open /var/lib/ceph/osd/ceph-1/keyring: (2) No such file or directory 2023-11-24T07:55:39.486 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:55:39.484+0000 7f8e37b0a640 -1 created new key in keyring /var/lib/ceph/osd/ceph-1/keyring 2023-11-24T07:55:39.487 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:55:39.484+0000 7f8e37b0a640 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2023-11-24T07:55:39.487 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:55:39.484+0000 7f8e37b0a640 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2023-11-24T07:55:39.487 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:55:39.484+0000 7f8e37b0a640 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2023-11-24T07:55:39.487 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:55:39.485+0000 7f8e37b0a640 -1 bluestore(/var/lib/ceph/osd/ceph-1) _read_fsid unparsable uuid 2023-11-24T07:55:42.356 DEBUG:teuthology.orchestra.run.smithi059:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-1 2023-11-24T07:55:42.389 DEBUG:teuthology.orchestra.run.smithi059:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 2 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-11-24T07:55:42.481 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:55:42.479+0000 7f3f490b8640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-2/keyring: can't open /var/lib/ceph/osd/ceph-2/keyring: (2) No such file or directory 2023-11-24T07:55:42.481 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:55:42.479+0000 7f3f490b8640 -1 created new key in keyring /var/lib/ceph/osd/ceph-2/keyring 2023-11-24T07:55:42.481 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:55:42.479+0000 7f3f490b8640 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2023-11-24T07:55:42.481 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:55:42.479+0000 7f3f490b8640 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2023-11-24T07:55:42.482 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:55:42.479+0000 7f3f490b8640 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2023-11-24T07:55:42.482 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:55:42.480+0000 7f3f490b8640 -1 bluestore(/var/lib/ceph/osd/ceph-2) _read_fsid unparsable uuid 2023-11-24T07:55:45.349 DEBUG:teuthology.orchestra.run.smithi059:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-2 2023-11-24T07:55:45.381 DEBUG:teuthology.orchestra.run.smithi059:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 3 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-11-24T07:55:45.474 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:55:45.472+0000 7faa0f017640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-3/keyring: can't open /var/lib/ceph/osd/ceph-3/keyring: (2) No such file or directory 2023-11-24T07:55:45.475 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:55:45.473+0000 7faa0f017640 -1 created new key in keyring /var/lib/ceph/osd/ceph-3/keyring 2023-11-24T07:55:45.475 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:55:45.473+0000 7faa0f017640 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2023-11-24T07:55:45.475 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:55:45.473+0000 7faa0f017640 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2023-11-24T07:55:45.475 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:55:45.473+0000 7faa0f017640 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2023-11-24T07:55:45.475 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:55:45.473+0000 7faa0f017640 -1 bluestore(/var/lib/ceph/osd/ceph-3) _read_fsid unparsable uuid 2023-11-24T07:55:48.343 DEBUG:teuthology.orchestra.run.smithi059:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-3 2023-11-24T07:55:48.376 DEBUG:teuthology.orchestra.run.smithi076:> sudo mkdir -p /var/lib/ceph/osd/ceph-4 2023-11-24T07:55:48.409 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2023-11-24T07:55:48.409 INFO:tasks.ceph:role: osd.4 2023-11-24T07:55:48.409 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi076.front.sepia.ceph.com 2023-11-24T07:55:48.410 DEBUG:teuthology.orchestra.run.smithi076:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2023-11-24T07:55:48.489 INFO:teuthology.orchestra.run.smithi076.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2023-11-24T07:55:48.489 INFO:teuthology.orchestra.run.smithi076.stdout: = sectsz=512 attr=2, projid32bit=1 2023-11-24T07:55:48.489 INFO:teuthology.orchestra.run.smithi076.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-11-24T07:55:48.489 INFO:teuthology.orchestra.run.smithi076.stdout: = reflink=1 bigtime=0 inobtcount=0 2023-11-24T07:55:48.489 INFO:teuthology.orchestra.run.smithi076.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-11-24T07:55:48.490 INFO:teuthology.orchestra.run.smithi076.stdout: = sunit=0 swidth=0 blks 2023-11-24T07:55:48.490 INFO:teuthology.orchestra.run.smithi076.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-11-24T07:55:48.490 INFO:teuthology.orchestra.run.smithi076.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-11-24T07:55:48.490 INFO:teuthology.orchestra.run.smithi076.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-11-24T07:55:48.490 INFO:teuthology.orchestra.run.smithi076.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-11-24T07:55:48.917 INFO:teuthology.orchestra.run.smithi076.stdout:Discarding blocks...Done. 2023-11-24T07:55:48.919 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi076.front.sepia.ceph.com -o noatime 2023-11-24T07:55:48.919 DEBUG:teuthology.orchestra.run.smithi076:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-4 2023-11-24T07:55:49.094 DEBUG:teuthology.orchestra.run.smithi076:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-4 2023-11-24T07:55:49.128 DEBUG:teuthology.orchestra.run.smithi076:> sudo mkdir -p /var/lib/ceph/osd/ceph-5 2023-11-24T07:55:49.199 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2023-11-24T07:55:49.199 INFO:tasks.ceph:role: osd.5 2023-11-24T07:55:49.199 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi076.front.sepia.ceph.com 2023-11-24T07:55:49.199 DEBUG:teuthology.orchestra.run.smithi076:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2023-11-24T07:55:49.274 INFO:teuthology.orchestra.run.smithi076.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2023-11-24T07:55:49.274 INFO:teuthology.orchestra.run.smithi076.stdout: = sectsz=512 attr=2, projid32bit=1 2023-11-24T07:55:49.275 INFO:teuthology.orchestra.run.smithi076.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-11-24T07:55:49.275 INFO:teuthology.orchestra.run.smithi076.stdout: = reflink=1 bigtime=0 inobtcount=0 2023-11-24T07:55:49.275 INFO:teuthology.orchestra.run.smithi076.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-11-24T07:55:49.275 INFO:teuthology.orchestra.run.smithi076.stdout: = sunit=0 swidth=0 blks 2023-11-24T07:55:49.276 INFO:teuthology.orchestra.run.smithi076.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-11-24T07:55:49.276 INFO:teuthology.orchestra.run.smithi076.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-11-24T07:55:49.276 INFO:teuthology.orchestra.run.smithi076.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-11-24T07:55:49.276 INFO:teuthology.orchestra.run.smithi076.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-11-24T07:55:49.702 INFO:teuthology.orchestra.run.smithi076.stdout:Discarding blocks...Done. 2023-11-24T07:55:49.704 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi076.front.sepia.ceph.com -o noatime 2023-11-24T07:55:49.704 DEBUG:teuthology.orchestra.run.smithi076:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-5 2023-11-24T07:55:49.814 DEBUG:teuthology.orchestra.run.smithi076:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-5 2023-11-24T07:55:49.847 DEBUG:teuthology.orchestra.run.smithi076:> sudo mkdir -p /var/lib/ceph/osd/ceph-6 2023-11-24T07:55:49.915 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2023-11-24T07:55:49.915 INFO:tasks.ceph:role: osd.6 2023-11-24T07:55:49.916 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi076.front.sepia.ceph.com 2023-11-24T07:55:49.916 DEBUG:teuthology.orchestra.run.smithi076:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2023-11-24T07:55:49.991 INFO:teuthology.orchestra.run.smithi076.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2023-11-24T07:55:49.991 INFO:teuthology.orchestra.run.smithi076.stdout: = sectsz=512 attr=2, projid32bit=1 2023-11-24T07:55:49.991 INFO:teuthology.orchestra.run.smithi076.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-11-24T07:55:49.991 INFO:teuthology.orchestra.run.smithi076.stdout: = reflink=1 bigtime=0 inobtcount=0 2023-11-24T07:55:49.991 INFO:teuthology.orchestra.run.smithi076.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-11-24T07:55:49.991 INFO:teuthology.orchestra.run.smithi076.stdout: = sunit=0 swidth=0 blks 2023-11-24T07:55:49.991 INFO:teuthology.orchestra.run.smithi076.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-11-24T07:55:49.991 INFO:teuthology.orchestra.run.smithi076.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-11-24T07:55:49.992 INFO:teuthology.orchestra.run.smithi076.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-11-24T07:55:49.992 INFO:teuthology.orchestra.run.smithi076.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-11-24T07:55:50.438 INFO:teuthology.orchestra.run.smithi076.stdout:Discarding blocks...Done. 2023-11-24T07:55:50.440 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi076.front.sepia.ceph.com -o noatime 2023-11-24T07:55:50.441 DEBUG:teuthology.orchestra.run.smithi076:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-6 2023-11-24T07:55:50.594 DEBUG:teuthology.orchestra.run.smithi076:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-6 2023-11-24T07:55:50.629 DEBUG:teuthology.orchestra.run.smithi076:> sudo mkdir -p /var/lib/ceph/osd/ceph-7 2023-11-24T07:55:50.697 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2023-11-24T07:55:50.697 INFO:tasks.ceph:role: osd.7 2023-11-24T07:55:50.697 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi076.front.sepia.ceph.com 2023-11-24T07:55:50.698 DEBUG:teuthology.orchestra.run.smithi076:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2023-11-24T07:55:50.771 INFO:teuthology.orchestra.run.smithi076.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2023-11-24T07:55:50.772 INFO:teuthology.orchestra.run.smithi076.stdout: = sectsz=512 attr=2, projid32bit=1 2023-11-24T07:55:50.772 INFO:teuthology.orchestra.run.smithi076.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-11-24T07:55:50.772 INFO:teuthology.orchestra.run.smithi076.stdout: = reflink=1 bigtime=0 inobtcount=0 2023-11-24T07:55:50.772 INFO:teuthology.orchestra.run.smithi076.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-11-24T07:55:50.772 INFO:teuthology.orchestra.run.smithi076.stdout: = sunit=0 swidth=0 blks 2023-11-24T07:55:50.772 INFO:teuthology.orchestra.run.smithi076.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-11-24T07:55:50.772 INFO:teuthology.orchestra.run.smithi076.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-11-24T07:55:50.772 INFO:teuthology.orchestra.run.smithi076.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-11-24T07:55:50.772 INFO:teuthology.orchestra.run.smithi076.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-11-24T07:55:51.212 INFO:teuthology.orchestra.run.smithi076.stdout:Discarding blocks...Done. 2023-11-24T07:55:51.214 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi076.front.sepia.ceph.com -o noatime 2023-11-24T07:55:51.214 DEBUG:teuthology.orchestra.run.smithi076:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-7 2023-11-24T07:55:51.390 DEBUG:teuthology.orchestra.run.smithi076:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-7 2023-11-24T07:55:51.424 DEBUG:teuthology.orchestra.run.smithi076:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 4 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-11-24T07:55:51.536 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-24T07:55:51.534+0000 7f50461ae640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-4/keyring: can't open /var/lib/ceph/osd/ceph-4/keyring: (2) No such file or directory 2023-11-24T07:55:51.536 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-24T07:55:51.534+0000 7f50461ae640 -1 created new key in keyring /var/lib/ceph/osd/ceph-4/keyring 2023-11-24T07:55:51.536 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-24T07:55:51.534+0000 7f50461ae640 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2023-11-24T07:55:51.537 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-24T07:55:51.534+0000 7f50461ae640 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2023-11-24T07:55:51.537 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-24T07:55:51.534+0000 7f50461ae640 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2023-11-24T07:55:51.537 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-24T07:55:51.534+0000 7f50461ae640 -1 bluestore(/var/lib/ceph/osd/ceph-4) _read_fsid unparsable uuid 2023-11-24T07:55:54.407 DEBUG:teuthology.orchestra.run.smithi076:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-4 2023-11-24T07:55:54.439 DEBUG:teuthology.orchestra.run.smithi076:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 5 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-11-24T07:55:54.535 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-24T07:55:54.533+0000 7fb5f2f0c640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-5/keyring: can't open /var/lib/ceph/osd/ceph-5/keyring: (2) No such file or directory 2023-11-24T07:55:54.535 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-24T07:55:54.533+0000 7fb5f2f0c640 -1 created new key in keyring /var/lib/ceph/osd/ceph-5/keyring 2023-11-24T07:55:54.535 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-24T07:55:54.533+0000 7fb5f2f0c640 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2023-11-24T07:55:54.535 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-24T07:55:54.533+0000 7fb5f2f0c640 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2023-11-24T07:55:54.535 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-24T07:55:54.533+0000 7fb5f2f0c640 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2023-11-24T07:55:54.536 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-24T07:55:54.534+0000 7fb5f2f0c640 -1 bluestore(/var/lib/ceph/osd/ceph-5) _read_fsid unparsable uuid 2023-11-24T07:55:57.395 DEBUG:teuthology.orchestra.run.smithi076:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-5 2023-11-24T07:55:57.427 DEBUG:teuthology.orchestra.run.smithi076:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 6 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-11-24T07:55:57.524 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-24T07:55:57.521+0000 7fb333eb1640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-6/keyring: can't open /var/lib/ceph/osd/ceph-6/keyring: (2) No such file or directory 2023-11-24T07:55:57.524 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-24T07:55:57.521+0000 7fb333eb1640 -1 created new key in keyring /var/lib/ceph/osd/ceph-6/keyring 2023-11-24T07:55:57.524 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-24T07:55:57.521+0000 7fb333eb1640 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2023-11-24T07:55:57.524 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-24T07:55:57.521+0000 7fb333eb1640 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2023-11-24T07:55:57.524 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-24T07:55:57.521+0000 7fb333eb1640 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2023-11-24T07:55:57.524 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-24T07:55:57.522+0000 7fb333eb1640 -1 bluestore(/var/lib/ceph/osd/ceph-6) _read_fsid unparsable uuid 2023-11-24T07:55:59.931 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-24T07:55:59.933 DEBUG:teuthology.orchestra.run.smithi076:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-24T07:56:00.395 DEBUG:teuthology.orchestra.run.smithi076:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-6 2023-11-24T07:56:00.427 DEBUG:teuthology.orchestra.run.smithi076:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 7 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-11-24T07:56:00.524 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-24T07:56:00.521+0000 7ff7815ba640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-7/keyring: can't open /var/lib/ceph/osd/ceph-7/keyring: (2) No such file or directory 2023-11-24T07:56:00.524 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-24T07:56:00.521+0000 7ff7815ba640 -1 created new key in keyring /var/lib/ceph/osd/ceph-7/keyring 2023-11-24T07:56:00.524 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-24T07:56:00.521+0000 7ff7815ba640 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2023-11-24T07:56:00.524 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-24T07:56:00.521+0000 7ff7815ba640 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2023-11-24T07:56:00.525 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-24T07:56:00.521+0000 7ff7815ba640 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2023-11-24T07:56:00.525 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-24T07:56:00.522+0000 7ff7815ba640 -1 bluestore(/var/lib/ceph/osd/ceph-7) _read_fsid unparsable uuid 2023-11-24T07:56:03.388 DEBUG:teuthology.orchestra.run.smithi076:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-7 2023-11-24T07:56:03.418 INFO:tasks.ceph:Reading keys from all nodes... 2023-11-24T07:56:03.419 DEBUG:teuthology.orchestra.run.smithi059:> set -ex 2023-11-24T07:56:03.419 DEBUG:teuthology.orchestra.run.smithi059:> sudo dd if=/var/lib/ceph/mgr/ceph-y/keyring of=/dev/stdout 2023-11-24T07:56:03.452 DEBUG:teuthology.orchestra.run.smithi059:> set -ex 2023-11-24T07:56:03.452 DEBUG:teuthology.orchestra.run.smithi059:> sudo dd if=/var/lib/ceph/mds/ceph-a/keyring of=/dev/stdout 2023-11-24T07:56:03.523 DEBUG:teuthology.orchestra.run.smithi059:> set -ex 2023-11-24T07:56:03.523 DEBUG:teuthology.orchestra.run.smithi059:> sudo dd if=/var/lib/ceph/mds/ceph-c/keyring of=/dev/stdout 2023-11-24T07:56:03.594 DEBUG:teuthology.orchestra.run.smithi059:> set -ex 2023-11-24T07:56:03.594 DEBUG:teuthology.orchestra.run.smithi059:> sudo dd if=/var/lib/ceph/osd/ceph-0/keyring of=/dev/stdout 2023-11-24T07:56:03.662 DEBUG:teuthology.orchestra.run.smithi059:> set -ex 2023-11-24T07:56:03.662 DEBUG:teuthology.orchestra.run.smithi059:> sudo dd if=/var/lib/ceph/osd/ceph-1/keyring of=/dev/stdout 2023-11-24T07:56:03.731 DEBUG:teuthology.orchestra.run.smithi059:> set -ex 2023-11-24T07:56:03.731 DEBUG:teuthology.orchestra.run.smithi059:> sudo dd if=/var/lib/ceph/osd/ceph-2/keyring of=/dev/stdout 2023-11-24T07:56:03.799 DEBUG:teuthology.orchestra.run.smithi059:> set -ex 2023-11-24T07:56:03.799 DEBUG:teuthology.orchestra.run.smithi059:> sudo dd if=/var/lib/ceph/osd/ceph-3/keyring of=/dev/stdout 2023-11-24T07:56:03.868 DEBUG:teuthology.orchestra.run.smithi076:> set -ex 2023-11-24T07:56:03.868 DEBUG:teuthology.orchestra.run.smithi076:> sudo dd if=/var/lib/ceph/mgr/ceph-x/keyring of=/dev/stdout 2023-11-24T07:56:03.900 DEBUG:teuthology.orchestra.run.smithi076:> set -ex 2023-11-24T07:56:03.900 DEBUG:teuthology.orchestra.run.smithi076:> sudo dd if=/var/lib/ceph/mds/ceph-b/keyring of=/dev/stdout 2023-11-24T07:56:03.971 DEBUG:teuthology.orchestra.run.smithi076:> set -ex 2023-11-24T07:56:03.971 DEBUG:teuthology.orchestra.run.smithi076:> sudo dd if=/var/lib/ceph/mds/ceph-d/keyring of=/dev/stdout 2023-11-24T07:56:04.043 DEBUG:teuthology.orchestra.run.smithi076:> set -ex 2023-11-24T07:56:04.044 DEBUG:teuthology.orchestra.run.smithi076:> sudo dd if=/var/lib/ceph/osd/ceph-4/keyring of=/dev/stdout 2023-11-24T07:56:04.115 DEBUG:teuthology.orchestra.run.smithi076:> set -ex 2023-11-24T07:56:04.116 DEBUG:teuthology.orchestra.run.smithi076:> sudo dd if=/var/lib/ceph/osd/ceph-5/keyring of=/dev/stdout 2023-11-24T07:56:04.187 DEBUG:teuthology.orchestra.run.smithi076:> set -ex 2023-11-24T07:56:04.187 DEBUG:teuthology.orchestra.run.smithi076:> sudo dd if=/var/lib/ceph/osd/ceph-6/keyring of=/dev/stdout 2023-11-24T07:56:04.259 DEBUG:teuthology.orchestra.run.smithi076:> set -ex 2023-11-24T07:56:04.259 DEBUG:teuthology.orchestra.run.smithi076:> sudo dd if=/var/lib/ceph/osd/ceph-7/keyring of=/dev/stdout 2023-11-24T07:56:04.332 DEBUG:teuthology.orchestra.run.smithi059:> set -ex 2023-11-24T07:56:04.332 DEBUG:teuthology.orchestra.run.smithi059:> dd if=/etc/ceph/ceph.client.0.keyring of=/dev/stdout 2023-11-24T07:56:04.352 DEBUG:teuthology.orchestra.run.smithi059:> set -ex 2023-11-24T07:56:04.352 DEBUG:teuthology.orchestra.run.smithi059:> dd if=/etc/ceph/ceph.client.2.keyring of=/dev/stdout 2023-11-24T07:56:04.411 DEBUG:teuthology.orchestra.run.smithi076:> set -ex 2023-11-24T07:56:04.412 DEBUG:teuthology.orchestra.run.smithi076:> dd if=/etc/ceph/ceph.client.1.keyring of=/dev/stdout 2023-11-24T07:56:04.432 DEBUG:teuthology.orchestra.run.smithi076:> set -ex 2023-11-24T07:56:04.432 DEBUG:teuthology.orchestra.run.smithi076:> dd if=/etc/ceph/ceph.client.3.keyring of=/dev/stdout 2023-11-24T07:56:04.492 INFO:tasks.ceph:Adding keys to all mons... 2023-11-24T07:56:04.492 DEBUG:teuthology.orchestra.run.smithi059:> sudo tee -a /etc/ceph/ceph.keyring 2023-11-24T07:56:04.494 DEBUG:teuthology.orchestra.run.smithi076:> sudo tee -a /etc/ceph/ceph.keyring 2023-11-24T07:56:04.535 INFO:teuthology.orchestra.run.smithi059.stdout:[mgr.y] 2023-11-24T07:56:04.535 INFO:teuthology.orchestra.run.smithi059.stdout: key = AQD0VmBlpXXaHxAAJMwwht1n+zRbRZuGOD5WTQ== 2023-11-24T07:56:04.535 INFO:teuthology.orchestra.run.smithi059.stdout:[mds.a] 2023-11-24T07:56:04.535 INFO:teuthology.orchestra.run.smithi059.stdout: key = AQD0VmBlWGLCKBAASbEQYAkExr8TButtK4KHQQ== 2023-11-24T07:56:04.535 INFO:teuthology.orchestra.run.smithi059.stdout:[mds.c] 2023-11-24T07:56:04.535 INFO:teuthology.orchestra.run.smithi059.stdout: key = AQD0VmBlvpsrMBAAr5lh7nb6B1xRfk0Umm7Xig== 2023-11-24T07:56:04.536 INFO:teuthology.orchestra.run.smithi059.stdout:[osd.0] 2023-11-24T07:56:04.536 INFO:teuthology.orchestra.run.smithi059.stdout: key = AQD4VmBlXjdAHRAAjnK2Y/rQhg4a97faNqVuoA== 2023-11-24T07:56:04.536 INFO:teuthology.orchestra.run.smithi059.stdout:[osd.1] 2023-11-24T07:56:04.536 INFO:teuthology.orchestra.run.smithi059.stdout: key = AQD7VmBlBwTwHBAAbpMH9rzS95ifrq/gVHnYRA== 2023-11-24T07:56:04.536 INFO:teuthology.orchestra.run.smithi059.stdout:[osd.2] 2023-11-24T07:56:04.536 INFO:teuthology.orchestra.run.smithi059.stdout: key = AQD+VmBl0tijHBAAWF2PsV3iH5adOTvHVPFIKw== 2023-11-24T07:56:04.536 INFO:teuthology.orchestra.run.smithi059.stdout:[osd.3] 2023-11-24T07:56:04.537 INFO:teuthology.orchestra.run.smithi059.stdout: key = AQABV2Bl3i8/HBAA4waLTD/PSLuNEg1VBmhEmg== 2023-11-24T07:56:04.537 INFO:teuthology.orchestra.run.smithi059.stdout:[mgr.x] 2023-11-24T07:56:04.537 INFO:teuthology.orchestra.run.smithi059.stdout: key = AQD0VmBlryM+JRAAf72PlDWEHZOwNxhW4k/8Ig== 2023-11-24T07:56:04.537 INFO:teuthology.orchestra.run.smithi059.stdout:[mds.b] 2023-11-24T07:56:04.537 INFO:teuthology.orchestra.run.smithi059.stdout: key = AQD0VmBlDIS0NRAAYQ3wTunU7m/Qk1qCM6Mcvg== 2023-11-24T07:56:04.537 INFO:teuthology.orchestra.run.smithi059.stdout:[mds.d] 2023-11-24T07:56:04.537 INFO:teuthology.orchestra.run.smithi059.stdout: key = AQD1VmBlPTG7ARAAvC60Rk2GarnNtRJ6MMRIMw== 2023-11-24T07:56:04.537 INFO:teuthology.orchestra.run.smithi059.stdout:[osd.4] 2023-11-24T07:56:04.538 INFO:teuthology.orchestra.run.smithi059.stdout: key = AQAHV2Blr2zrHxAALOr+kFGdOTr8uQ9+/G2WUg== 2023-11-24T07:56:04.538 INFO:teuthology.orchestra.run.smithi059.stdout:[osd.5] 2023-11-24T07:56:04.538 INFO:teuthology.orchestra.run.smithi059.stdout: key = AQAKV2BlDZ3bHxAA7ZPPoxi4zVN0VECYzTsAsg== 2023-11-24T07:56:04.538 INFO:teuthology.orchestra.run.smithi059.stdout:[osd.6] 2023-11-24T07:56:04.538 INFO:teuthology.orchestra.run.smithi059.stdout: key = AQANV2BlfDMrHxAAVtm8+kh5FNTd1mJOKg02gg== 2023-11-24T07:56:04.538 INFO:teuthology.orchestra.run.smithi059.stdout:[osd.7] 2023-11-24T07:56:04.538 INFO:teuthology.orchestra.run.smithi059.stdout: key = AQAQV2Bli+slHxAAHY4sa84El5NC4KVgPKHs6w== 2023-11-24T07:56:04.538 INFO:teuthology.orchestra.run.smithi059.stdout:[client.0] 2023-11-24T07:56:04.539 INFO:teuthology.orchestra.run.smithi059.stdout: key = AQD1VmBlxPkpBhAAR841hRsF6QtmzFgJIrVZ9w== 2023-11-24T07:56:04.539 INFO:teuthology.orchestra.run.smithi059.stdout:[client.2] 2023-11-24T07:56:04.539 INFO:teuthology.orchestra.run.smithi059.stdout: key = AQD1VmBlgQSRCRAAcHS1eQCYMJ3YP21jUEPJ0A== 2023-11-24T07:56:04.539 INFO:teuthology.orchestra.run.smithi059.stdout:[client.1] 2023-11-24T07:56:04.539 INFO:teuthology.orchestra.run.smithi059.stdout: key = AQD1VmBl46HpDBAAnO9MGWrJczW0epSp0eCkLg== 2023-11-24T07:56:04.539 INFO:teuthology.orchestra.run.smithi059.stdout:[client.3] 2023-11-24T07:56:04.539 INFO:teuthology.orchestra.run.smithi059.stdout: key = AQD1VmBl7gmhEhAA6uWs7XAbOQwJItA6To549w== 2023-11-24T07:56:04.561 INFO:teuthology.orchestra.run.smithi076.stdout:[mgr.y] 2023-11-24T07:56:04.561 INFO:teuthology.orchestra.run.smithi076.stdout: key = AQD0VmBlpXXaHxAAJMwwht1n+zRbRZuGOD5WTQ== 2023-11-24T07:56:04.562 INFO:teuthology.orchestra.run.smithi076.stdout:[mds.a] 2023-11-24T07:56:04.562 INFO:teuthology.orchestra.run.smithi076.stdout: key = AQD0VmBlWGLCKBAASbEQYAkExr8TButtK4KHQQ== 2023-11-24T07:56:04.562 INFO:teuthology.orchestra.run.smithi076.stdout:[mds.c] 2023-11-24T07:56:04.563 INFO:teuthology.orchestra.run.smithi076.stdout: key = AQD0VmBlvpsrMBAAr5lh7nb6B1xRfk0Umm7Xig== 2023-11-24T07:56:04.563 INFO:teuthology.orchestra.run.smithi076.stdout:[osd.0] 2023-11-24T07:56:04.563 INFO:teuthology.orchestra.run.smithi076.stdout: key = AQD4VmBlXjdAHRAAjnK2Y/rQhg4a97faNqVuoA== 2023-11-24T07:56:04.563 INFO:teuthology.orchestra.run.smithi076.stdout:[osd.1] 2023-11-24T07:56:04.564 INFO:teuthology.orchestra.run.smithi076.stdout: key = AQD7VmBlBwTwHBAAbpMH9rzS95ifrq/gVHnYRA== 2023-11-24T07:56:04.564 INFO:teuthology.orchestra.run.smithi076.stdout:[osd.2] 2023-11-24T07:56:04.564 INFO:teuthology.orchestra.run.smithi076.stdout: key = AQD+VmBl0tijHBAAWF2PsV3iH5adOTvHVPFIKw== 2023-11-24T07:56:04.565 INFO:teuthology.orchestra.run.smithi076.stdout:[osd.3] 2023-11-24T07:56:04.565 INFO:teuthology.orchestra.run.smithi076.stdout: key = AQABV2Bl3i8/HBAA4waLTD/PSLuNEg1VBmhEmg== 2023-11-24T07:56:04.565 INFO:teuthology.orchestra.run.smithi076.stdout:[mgr.x] 2023-11-24T07:56:04.565 INFO:teuthology.orchestra.run.smithi076.stdout: key = AQD0VmBlryM+JRAAf72PlDWEHZOwNxhW4k/8Ig== 2023-11-24T07:56:04.566 INFO:teuthology.orchestra.run.smithi076.stdout:[mds.b] 2023-11-24T07:56:04.566 INFO:teuthology.orchestra.run.smithi076.stdout: key = AQD0VmBlDIS0NRAAYQ3wTunU7m/Qk1qCM6Mcvg== 2023-11-24T07:56:04.566 INFO:teuthology.orchestra.run.smithi076.stdout:[mds.d] 2023-11-24T07:56:04.566 INFO:teuthology.orchestra.run.smithi076.stdout: key = AQD1VmBlPTG7ARAAvC60Rk2GarnNtRJ6MMRIMw== 2023-11-24T07:56:04.567 INFO:teuthology.orchestra.run.smithi076.stdout:[osd.4] 2023-11-24T07:56:04.567 INFO:teuthology.orchestra.run.smithi076.stdout: key = AQAHV2Blr2zrHxAALOr+kFGdOTr8uQ9+/G2WUg== 2023-11-24T07:56:04.567 INFO:teuthology.orchestra.run.smithi076.stdout:[osd.5] 2023-11-24T07:56:04.567 INFO:teuthology.orchestra.run.smithi076.stdout: key = AQAKV2BlDZ3bHxAA7ZPPoxi4zVN0VECYzTsAsg== 2023-11-24T07:56:04.568 INFO:teuthology.orchestra.run.smithi076.stdout:[osd.6] 2023-11-24T07:56:04.568 INFO:teuthology.orchestra.run.smithi076.stdout: key = AQANV2BlfDMrHxAAVtm8+kh5FNTd1mJOKg02gg== 2023-11-24T07:56:04.568 INFO:teuthology.orchestra.run.smithi076.stdout:[osd.7] 2023-11-24T07:56:04.568 INFO:teuthology.orchestra.run.smithi076.stdout: key = AQAQV2Bli+slHxAAHY4sa84El5NC4KVgPKHs6w== 2023-11-24T07:56:04.568 INFO:teuthology.orchestra.run.smithi076.stdout:[client.0] 2023-11-24T07:56:04.568 INFO:teuthology.orchestra.run.smithi076.stdout: key = AQD1VmBlxPkpBhAAR841hRsF6QtmzFgJIrVZ9w== 2023-11-24T07:56:04.568 INFO:teuthology.orchestra.run.smithi076.stdout:[client.2] 2023-11-24T07:56:04.569 INFO:teuthology.orchestra.run.smithi076.stdout: key = AQD1VmBlgQSRCRAAcHS1eQCYMJ3YP21jUEPJ0A== 2023-11-24T07:56:04.569 INFO:teuthology.orchestra.run.smithi076.stdout:[client.1] 2023-11-24T07:56:04.569 INFO:teuthology.orchestra.run.smithi076.stdout: key = AQD1VmBl46HpDBAAnO9MGWrJczW0epSp0eCkLg== 2023-11-24T07:56:04.569 INFO:teuthology.orchestra.run.smithi076.stdout:[client.3] 2023-11-24T07:56:04.569 INFO:teuthology.orchestra.run.smithi076.stdout: key = AQD1VmBl7gmhEhAA6uWs7XAbOQwJItA6To549w== 2023-11-24T07:56:04.570 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2023-11-24T07:56:04.583 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2023-11-24T07:56:04.672 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.a --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-11-24T07:56:04.675 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.a --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-11-24T07:56:04.735 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.c --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-11-24T07:56:04.738 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.c --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-11-24T07:56:04.795 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-11-24T07:56:04.797 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-11-24T07:56:04.851 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-11-24T07:56:04.853 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-11-24T07:56:04.906 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-11-24T07:56:04.908 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-11-24T07:56:04.962 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-11-24T07:56:04.964 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-11-24T07:56:05.018 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2023-11-24T07:56:05.020 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2023-11-24T07:56:05.073 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.b --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-11-24T07:56:05.075 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.b --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-11-24T07:56:05.131 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.d --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-11-24T07:56:05.134 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.d --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-11-24T07:56:05.190 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-11-24T07:56:05.192 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-11-24T07:56:05.246 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-11-24T07:56:05.248 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-11-24T07:56:05.301 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-11-24T07:56:05.303 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-11-24T07:56:05.357 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-11-24T07:56:05.359 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-11-24T07:56:05.412 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-11-24T07:56:05.414 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-11-24T07:56:05.470 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.2 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-11-24T07:56:05.473 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.2 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-11-24T07:56:05.529 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.1 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-11-24T07:56:05.531 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.1 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-11-24T07:56:05.587 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.3 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-11-24T07:56:05.590 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.3 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-11-24T07:56:05.646 INFO:tasks.ceph:Running mkfs on mon nodes... 2023-11-24T07:56:05.646 DEBUG:teuthology.orchestra.run.smithi059:> sudo mkdir -p /var/lib/ceph/mon/ceph-a 2023-11-24T07:56:05.673 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i a --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2023-11-24T07:56:05.773 DEBUG:teuthology.orchestra.run.smithi059:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-a 2023-11-24T07:56:05.800 DEBUG:teuthology.orchestra.run.smithi076:> sudo mkdir -p /var/lib/ceph/mon/ceph-b 2023-11-24T07:56:05.827 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i b --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2023-11-24T07:56:05.925 DEBUG:teuthology.orchestra.run.smithi076:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-b 2023-11-24T07:56:05.951 DEBUG:teuthology.orchestra.run.smithi076:> sudo mkdir -p /var/lib/ceph/mon/ceph-c 2023-11-24T07:56:06.018 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i c --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2023-11-24T07:56:06.117 DEBUG:teuthology.orchestra.run.smithi076:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-c 2023-11-24T07:56:06.143 DEBUG:teuthology.orchestra.run.smithi059:> rm -- /home/ubuntu/cephtest/ceph.monmap 2023-11-24T07:56:06.145 DEBUG:teuthology.orchestra.run.smithi076:> rm -- /home/ubuntu/cephtest/ceph.monmap 2023-11-24T07:56:06.200 INFO:tasks.ceph:Starting mon daemons in cluster ceph... 2023-11-24T07:56:06.200 INFO:tasks.ceph.mon.a:Restarting daemon 2023-11-24T07:56:06.200 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i a 2023-11-24T07:56:06.202 INFO:tasks.ceph.mon.a:Started 2023-11-24T07:56:06.203 INFO:tasks.ceph.mon.b:Restarting daemon 2023-11-24T07:56:06.203 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i b 2023-11-24T07:56:06.242 INFO:tasks.ceph.mon.b:Started 2023-11-24T07:56:06.242 INFO:tasks.ceph.mon.c:Restarting daemon 2023-11-24T07:56:06.242 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i c 2023-11-24T07:56:06.244 INFO:tasks.ceph.mon.c:Started 2023-11-24T07:56:06.244 INFO:tasks.ceph:Starting mgr daemons in cluster ceph... 2023-11-24T07:56:06.244 INFO:tasks.ceph.mgr.y:Restarting daemon 2023-11-24T07:56:06.244 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i y 2023-11-24T07:56:06.246 INFO:tasks.ceph.mgr.y:Started 2023-11-24T07:56:06.247 INFO:tasks.ceph.mgr.x:Restarting daemon 2023-11-24T07:56:06.247 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i x 2023-11-24T07:56:06.248 INFO:tasks.ceph.mgr.x:Started 2023-11-24T07:56:06.249 INFO:tasks.ceph:Setting crush tunables to default 2023-11-24T07:56:06.249 DEBUG:teuthology.orchestra.run.smithi059:> sudo ceph --cluster ceph osd crush tunables default 2023-11-24T07:56:06.375 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:06.373+0000 7f98dbf33700 1 Processor -- start 2023-11-24T07:56:06.375 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:06.374+0000 7f98dbf33700 1 -- start start 2023-11-24T07:56:06.377 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:06.374+0000 7f98dbf33700 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f98d4143ae0 0x7f98d40b51b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:56:06.377 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:06.374+0000 7f98dbf33700 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f98d4143ec0 con 0x7f98d41440f0 2023-11-24T07:56:06.377 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:06.375+0000 7f98dbf33700 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f98d40b3ae0 con 0x7f98d40722c0 2023-11-24T07:56:06.377 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:06.375+0000 7f98dbf33700 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f98d40b40a0 con 0x7f98d4143ae0 2023-11-24T07:56:06.377 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:06.375+0000 7f98d94ce700 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7f98d41440f0 0x7f98d4071bb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:52644/0 (socket says 172.21.15.59:52644) 2023-11-24T07:56:06.377 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:06.375+0000 7f98d94ce700 1 -- 172.21.15.59:0/3776018446 learned_addr learned my addr 172.21.15.59:0/3776018446 (peer_addr_for_me v1:172.21.15.59:0/0) 2023-11-24T07:56:06.377 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:06.375+0000 7f98d9ccf700 1 --2- 172.21.15.59:0/3776018446 >> v2:172.21.15.76:3300/0 conn(0x7f98d4143ae0 0x7f98d40b51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:56:06.378 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:06.375+0000 7f98d9ccf700 1 -- 172.21.15.59:0/3776018446 >> v2:172.21.15.76:3300/0 conn(0x7f98d4143ae0 msgr2=0x7f98d40b51b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2023-11-24T07:56:06.378 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:06.375+0000 7f98d9ccf700 1 -- 172.21.15.59:0/3776018446 >> v2:172.21.15.76:3300/0 conn(0x7f98d4143ae0 msgr2=0x7f98d40b51b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2023-11-24T07:56:06.378 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:06.375+0000 7f98d9ccf700 1 --2- 172.21.15.59:0/3776018446 >> v2:172.21.15.76:3300/0 conn(0x7f98d4143ae0 0x7f98d40b51b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2023-11-24T07:56:06.378 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:06.375+0000 7f98d9ccf700 1 --2- 172.21.15.59:0/3776018446 >> v2:172.21.15.76:3300/0 conn(0x7f98d4143ae0 0x7f98d40b51b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-11-24T07:56:06.578 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:06.576+0000 7f98d9ccf700 1 --2- 172.21.15.59:0/3776018446 >> v2:172.21.15.76:3300/0 conn(0x7f98d4143ae0 0x7f98d40b51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:56:06.578 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:06.576+0000 7f98d9ccf700 1 -- 172.21.15.59:0/3776018446 >> v2:172.21.15.76:3300/0 conn(0x7f98d4143ae0 msgr2=0x7f98d40b51b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2023-11-24T07:56:06.578 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:06.576+0000 7f98d9ccf700 1 -- 172.21.15.59:0/3776018446 >> v2:172.21.15.76:3300/0 conn(0x7f98d4143ae0 msgr2=0x7f98d40b51b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2023-11-24T07:56:06.578 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:06.576+0000 7f98d9ccf700 1 --2- 172.21.15.59:0/3776018446 >> v2:172.21.15.76:3300/0 conn(0x7f98d4143ae0 0x7f98d40b51b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2023-11-24T07:56:06.578 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:06.576+0000 7f98d9ccf700 1 --2- 172.21.15.59:0/3776018446 >> v2:172.21.15.76:3300/0 conn(0x7f98d4143ae0 0x7f98d40b51b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-11-24T07:56:06.581 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:06.576+0000 7f98d8ccd700 1 -- 172.21.15.59:0/3776018446 <== mon.0 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3500700640 0 0) 0x7f98d4143ec0 con 0x7f98d41440f0 2023-11-24T07:56:06.581 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:06.576+0000 7f98d8ccd700 1 -- 172.21.15.59:0/3776018446 --> v1:172.21.15.59:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f98c00035b0 con 0x7f98d41440f0 2023-11-24T07:56:06.581 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:06.577+0000 7f98d8ccd700 1 -- 172.21.15.59:0/3776018446 <== mon.0 v1:172.21.15.59:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3498710878 0 0) 0x7f98c00035b0 con 0x7f98d41440f0 2023-11-24T07:56:06.581 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:06.577+0000 7f98d8ccd700 1 -- 172.21.15.59:0/3776018446 >> v2:172.21.15.76:3300/0 conn(0x7f98d4143ae0 msgr2=0x7f98d40b51b0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-11-24T07:56:06.581 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:06.577+0000 7f98d8ccd700 1 --2- 172.21.15.59:0/3776018446 >> v2:172.21.15.76:3300/0 conn(0x7f98d4143ae0 0x7f98d40b51b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:56:06.581 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:06.577+0000 7f98d8ccd700 1 -- 172.21.15.59:0/3776018446 >> v1:172.21.15.76:6789/0 conn(0x7f98d40722c0 legacy=0x7f98d406e510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:56:06.581 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:06.577+0000 7f98d8ccd700 1 -- 172.21.15.59:0/3776018446 --> v1:172.21.15.59:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f98d40b3dc0 con 0x7f98d41440f0 2023-11-24T07:56:06.581 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:06.577+0000 7f98d8ccd700 1 -- 172.21.15.59:0/3776018446 <== mon.0 v1:172.21.15.59:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1312333566 0 0) 0x7f98d0001670 con 0x7f98d41440f0 2023-11-24T07:56:06.582 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:06.577+0000 7f98d8ccd700 1 -- 172.21.15.59:0/3776018446 >> v1:172.21.15.59:6789/0 conn(0x7f98d41440f0 legacy=0x7f98d4071bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:56:06.582 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:06.577+0000 7f98d8ccd700 1 --2- 172.21.15.59:0/3776018446 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f98c0003a00 0x7f98c0003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:56:06.582 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:06.577+0000 7f98d8ccd700 1 --2- 172.21.15.59:0/3776018446 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f98c0004850 0x7f98c0004cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:56:06.582 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:06.577+0000 7f98d8ccd700 1 --2- 172.21.15.59:0/3776018446 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f98c0005210 0x7f98c000ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:56:06.582 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:06.577+0000 7f98d8ccd700 1 -- 172.21.15.59:0/3776018446 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98d40b40a0 con 0x7f98c0005210 2023-11-24T07:56:06.582 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:06.577+0000 7f98d8ccd700 1 -- 172.21.15.59:0/3776018446 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98c000c160 con 0x7f98c0003a00 2023-11-24T07:56:06.582 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:06.577+0000 7f98d8ccd700 1 -- 172.21.15.59:0/3776018446 --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f98c000c320 con 0x7f98c0004850 2023-11-24T07:56:06.582 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:06.578+0000 7f98da4d0700 1 --2- 172.21.15.59:0/3776018446 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f98c0005210 0x7f98c000ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:56:06.582 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:06.578+0000 7f98d94ce700 1 --2- 172.21.15.59:0/3776018446 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f98c0004850 0x7f98c0004cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:56:06.582 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:06.578+0000 7f98d9ccf700 1 --2- 172.21.15.59:0/3776018446 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f98c0003a00 0x7f98c0003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:56:06.582 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:06.578+0000 7f98da4d0700 1 -- 172.21.15.59:0/3776018446 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f98c0004850 msgr2=0x7f98c0004cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T07:56:06.582 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:06.578+0000 7f98da4d0700 1 --2- 172.21.15.59:0/3776018446 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f98c0004850 0x7f98c0004cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:56:06.583 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:06.578+0000 7f98da4d0700 1 -- 172.21.15.59:0/3776018446 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f98c0003a00 msgr2=0x7f98c0003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T07:56:06.583 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:06.578+0000 7f98da4d0700 1 --2- 172.21.15.59:0/3776018446 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f98c0003a00 0x7f98c0003e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:56:06.583 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:06.578+0000 7f98da4d0700 1 -- 172.21.15.59:0/3776018446 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f98c00035b0 con 0x7f98c0005210 2023-11-24T07:56:06.583 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:06.578+0000 7f98da4d0700 1 -- 172.21.15.59:0/3776018446 reap_dead start 2023-11-24T07:56:06.583 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:06.578+0000 7f98da4d0700 1 --2- 172.21.15.59:0/3776018446 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f98c0005210 0x7f98c000ba10 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f98d415f970 tx=0x7f98c0005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dda59f01d241706c server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:56:06.583 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:06.578+0000 7f98d8ccd700 1 -- 172.21.15.59:0/3776018446 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98cc001db0 con 0x7f98c0005210 2023-11-24T07:56:06.583 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:06.578+0000 7f98d8ccd700 1 -- 172.21.15.59:0/3776018446 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f98cc00cc90 con 0x7f98c0005210 2023-11-24T07:56:06.583 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:06.578+0000 7f98d8ccd700 1 -- 172.21.15.59:0/3776018446 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98cc003a00 con 0x7f98c0005210 2023-11-24T07:56:06.583 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:06.579+0000 7f98dbf33700 1 -- 172.21.15.59:0/3776018446 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f98c0005210 msgr2=0x7f98c000ba10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:56:06.583 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:06.579+0000 7f98dbf33700 1 --2- 172.21.15.59:0/3776018446 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f98c0005210 0x7f98c000ba10 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f98d415f970 tx=0x7f98c0005880 comp rx=0 tx=0).stop 2023-11-24T07:56:06.583 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:06.579+0000 7f98dbf33700 1 -- 172.21.15.59:0/3776018446 shutdown_connections 2023-11-24T07:56:06.583 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:06.579+0000 7f98dbf33700 1 --2- 172.21.15.59:0/3776018446 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f98c0005210 0x7f98c000ba10 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:56:06.584 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:06.579+0000 7f98dbf33700 1 -- 172.21.15.59:0/3776018446 >> 172.21.15.59:0/3776018446 conn(0x7f98d40b2740 msgr2=0x7f98d40b2b20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:56:06.584 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:06.579+0000 7f98dbf33700 1 -- 172.21.15.59:0/3776018446 shutdown_connections 2023-11-24T07:56:06.584 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:06.579+0000 7f98dbf33700 1 -- 172.21.15.59:0/3776018446 wait complete. 2023-11-24T07:56:06.586 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:06.580+0000 7f98dbf33700 1 Processor -- start 2023-11-24T07:56:06.586 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:06.580+0000 7f98dbf33700 1 -- start start 2023-11-24T07:56:06.586 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:06.581+0000 7f98dbf33700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f98c0005210 0x7f98d413d980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:56:06.586 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:06.581+0000 7f98dbf33700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f98d414c440 0x7f98d414c8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:56:06.586 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:06.581+0000 7f98dbf33700 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f98d41410d0 0x7f98d4141550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:56:06.587 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:06.581+0000 7f98dbf33700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98d414ce00 con 0x7f98c0005210 2023-11-24T07:56:06.587 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:06.581+0000 7f98dbf33700 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98d4141e20 con 0x7f98d414c440 2023-11-24T07:56:06.587 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:06.581+0000 7f98dbf33700 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f98d4142100 con 0x7f98d41410d0 2023-11-24T07:56:06.587 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:06.582+0000 7f98d9ccf700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f98c0005210 0x7f98d413d980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:56:06.587 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:06.582+0000 7f98d9ccf700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f98c0005210 0x7f98d413d980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:49194/0 (socket says 172.21.15.59:49194) 2023-11-24T07:56:06.587 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:06.582+0000 7f98d9ccf700 1 -- 172.21.15.59:0/2609152742 learned_addr learned my addr 172.21.15.59:0/2609152742 (peer_addr_for_me v2:172.21.15.59:0/0) 2023-11-24T07:56:06.587 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:06.582+0000 7f98d94ce700 1 --2- 172.21.15.59:0/2609152742 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f98d414c440 0x7f98d414c8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:56:06.587 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:06.582+0000 7f98da4d0700 1 --2- 172.21.15.59:0/2609152742 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f98d41410d0 0x7f98d4141550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:56:06.587 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:06.582+0000 7f98d9ccf700 1 -- 172.21.15.59:0/2609152742 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f98d41410d0 msgr2=0x7f98d4141550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:56:06.587 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:06.582+0000 7f98d9ccf700 1 --2- 172.21.15.59:0/2609152742 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f98d41410d0 0x7f98d4141550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:56:06.587 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:06.582+0000 7f98d9ccf700 1 -- 172.21.15.59:0/2609152742 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f98d414c440 msgr2=0x7f98d414c8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:56:06.587 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:06.582+0000 7f98d9ccf700 1 --2- 172.21.15.59:0/2609152742 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f98d414c440 0x7f98d414c8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:56:06.587 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:06.582+0000 7f98d9ccf700 1 -- 172.21.15.59:0/2609152742 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f98d41423e0 con 0x7f98c0005210 2023-11-24T07:56:06.588 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:06.582+0000 7f98d9ccf700 1 --2- 172.21.15.59:0/2609152742 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f98c0005210 0x7f98d413d980 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f98c400e860 tx=0x7f98c400eb70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:56:06.588 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:06.583+0000 7f98ca7fc700 1 -- 172.21.15.59:0/2609152742 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98c40154b0 con 0x7f98c0005210 2023-11-24T07:56:06.588 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:06.583+0000 7f98ca7fc700 1 -- 172.21.15.59:0/2609152742 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f98c400c040 con 0x7f98c0005210 2023-11-24T07:56:06.588 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:06.583+0000 7f98dbf33700 1 -- 172.21.15.59:0/2609152742 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f98d41426c0 con 0x7f98c0005210 2023-11-24T07:56:06.588 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:06.583+0000 7f98ca7fc700 1 -- 172.21.15.59:0/2609152742 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98c4015850 con 0x7f98c0005210 2023-11-24T07:56:06.588 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:06.583+0000 7f98dbf33700 1 -- 172.21.15.59:0/2609152742 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f98d4141a90 con 0x7f98c0005210 2023-11-24T07:56:06.588 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:06.583+0000 7f98ca7fc700 1 -- 172.21.15.59:0/2609152742 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f98c4005d20 con 0x7f98c0005210 2023-11-24T07:56:06.588 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:06.583+0000 7f98ca7fc700 1 -- 172.21.15.59:0/2609152742 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f98c40067b0 con 0x7f98c0005210 2023-11-24T07:56:06.588 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:06.584+0000 7f98dbf33700 1 -- 172.21.15.59:0/2609152742 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f98d414a600 con 0x7f98c0005210 2023-11-24T07:56:06.589 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:06.587+0000 7f98ca7fc700 1 -- 172.21.15.59:0/2609152742 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72898 (secure 0 0 0) 0x7f98c4023060 con 0x7f98c0005210 2023-11-24T07:56:06.669 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:06.668+0000 7f98dbf33700 1 -- 172.21.15.59:0/2609152742 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd crush tunables", "profile": "default"} v 0) v1 -- 0x7f98d414a3f0 con 0x7f98c0005210 2023-11-24T07:56:06.670 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:06.669+0000 7f98ca7fc700 1 -- 172.21.15.59:0/2609152742 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush tunables", "profile": "default"}]=0 adjusted tunables profile to default v2) v1 ==== 124+0+0 (secure 0 0 0) 0x7f98c402c090 con 0x7f98c0005210 2023-11-24T07:56:06.672 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:06.670+0000 7f98dbf33700 1 -- 172.21.15.59:0/2609152742 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f98c0005210 msgr2=0x7f98d413d980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:56:06.672 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:06.670+0000 7f98dbf33700 1 --2- 172.21.15.59:0/2609152742 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f98c0005210 0x7f98d413d980 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f98c400e860 tx=0x7f98c400eb70 comp rx=0 tx=0).stop 2023-11-24T07:56:06.672 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:06.670+0000 7f98dbf33700 1 -- 172.21.15.59:0/2609152742 shutdown_connections 2023-11-24T07:56:06.672 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:06.670+0000 7f98dbf33700 1 --2- 172.21.15.59:0/2609152742 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f98d41410d0 0x7f98d4141550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:56:06.672 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:06.670+0000 7f98dbf33700 1 --2- 172.21.15.59:0/2609152742 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f98c0005210 0x7f98d413d980 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:56:06.672 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:06.670+0000 7f98dbf33700 1 --2- 172.21.15.59:0/2609152742 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f98d414c440 0x7f98d414c8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:56:06.673 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:06.670+0000 7f98dbf33700 1 -- 172.21.15.59:0/2609152742 >> 172.21.15.59:0/2609152742 conn(0x7f98d40b2740 msgr2=0x7f98d415e0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:56:06.673 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:06.670+0000 7f98dbf33700 1 -- 172.21.15.59:0/2609152742 shutdown_connections 2023-11-24T07:56:06.673 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:06.670+0000 7f98dbf33700 1 -- 172.21.15.59:0/2609152742 wait complete. 2023-11-24T07:56:06.673 INFO:teuthology.orchestra.run.smithi059.stderr:adjusted tunables profile to default 2023-11-24T07:56:06.684 INFO:tasks.ceph:check_enable_crimson: False 2023-11-24T07:56:06.684 INFO:tasks.ceph:Starting osd daemons in cluster ceph... 2023-11-24T07:56:06.684 DEBUG:teuthology.orchestra.run.smithi059:> set -ex 2023-11-24T07:56:06.684 DEBUG:teuthology.orchestra.run.smithi059:> sudo dd if=/var/lib/ceph/osd/ceph-0/fsid of=/dev/stdout 2023-11-24T07:56:06.695 INFO:tasks.ceph.mgr.x.smithi076.stderr:2023-11-24T07:56:06.693+0000 7ff220fa5200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-11-24T07:56:06.708 INFO:tasks.ceph.mgr.y.smithi059.stderr:2023-11-24T07:56:06.707+0000 7eff6f463200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-11-24T07:56:06.711 DEBUG:teuthology.orchestra.run.smithi059:> set -ex 2023-11-24T07:56:06.711 DEBUG:teuthology.orchestra.run.smithi059:> sudo dd if=/var/lib/ceph/osd/ceph-1/fsid of=/dev/stdout 2023-11-24T07:56:06.761 INFO:tasks.ceph.mgr.x.smithi076.stderr:2023-11-24T07:56:06.759+0000 7ff220fa5200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-11-24T07:56:06.779 DEBUG:teuthology.orchestra.run.smithi059:> set -ex 2023-11-24T07:56:06.779 DEBUG:teuthology.orchestra.run.smithi059:> sudo dd if=/var/lib/ceph/osd/ceph-2/fsid of=/dev/stdout 2023-11-24T07:56:06.780 INFO:tasks.ceph.mgr.y.smithi059.stderr:2023-11-24T07:56:06.777+0000 7eff6f463200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-11-24T07:56:06.837 INFO:tasks.ceph.mgr.x.smithi076.stderr:2023-11-24T07:56:06.836+0000 7ff220fa5200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-11-24T07:56:06.847 DEBUG:teuthology.orchestra.run.smithi059:> set -ex 2023-11-24T07:56:06.847 DEBUG:teuthology.orchestra.run.smithi059:> sudo dd if=/var/lib/ceph/osd/ceph-3/fsid of=/dev/stdout 2023-11-24T07:56:06.862 INFO:tasks.ceph.mgr.y.smithi059.stderr:2023-11-24T07:56:06.860+0000 7eff6f463200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-11-24T07:56:06.890 DEBUG:teuthology.orchestra.run.smithi076:> set -ex 2023-11-24T07:56:06.890 DEBUG:teuthology.orchestra.run.smithi076:> sudo dd if=/var/lib/ceph/osd/ceph-4/fsid of=/dev/stdout 2023-11-24T07:56:06.916 DEBUG:teuthology.orchestra.run.smithi076:> set -ex 2023-11-24T07:56:06.916 DEBUG:teuthology.orchestra.run.smithi076:> sudo dd if=/var/lib/ceph/osd/ceph-5/fsid of=/dev/stdout 2023-11-24T07:56:06.918 INFO:tasks.ceph.mgr.x.smithi076.stderr:2023-11-24T07:56:06.916+0000 7ff220fa5200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-11-24T07:56:06.944 DEBUG:teuthology.orchestra.run.smithi076:> set -ex 2023-11-24T07:56:06.945 DEBUG:teuthology.orchestra.run.smithi076:> sudo dd if=/var/lib/ceph/osd/ceph-6/fsid of=/dev/stdout 2023-11-24T07:56:06.948 INFO:tasks.ceph.mgr.y.smithi059.stderr:2023-11-24T07:56:06.946+0000 7eff6f463200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-11-24T07:56:07.011 DEBUG:teuthology.orchestra.run.smithi076:> set -ex 2023-11-24T07:56:07.012 DEBUG:teuthology.orchestra.run.smithi076:> sudo dd if=/var/lib/ceph/osd/ceph-7/fsid of=/dev/stdout 2023-11-24T07:56:07.070 INFO:tasks.ceph.mgr.x.smithi076.stderr:2023-11-24T07:56:07.069+0000 7ff220fa5200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-11-24T07:56:07.077 DEBUG:teuthology.orchestra.run.smithi076:> sudo ceph --cluster ceph osd new 6d8f9e9b-1782-4e41-928b-a39f2a8ebc06 0 2023-11-24T07:56:07.097 INFO:tasks.ceph.mgr.y.smithi059.stderr:2023-11-24T07:56:07.095+0000 7eff6f463200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-11-24T07:56:07.239 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-24T07:56:07.237+0000 7fb329c97700 1 Processor -- start 2023-11-24T07:56:07.239 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-24T07:56:07.237+0000 7fb329c97700 1 -- start start 2023-11-24T07:56:07.242 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-24T07:56:07.238+0000 7fb329c97700 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fb3240b4150 0x7fb32406de50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:56:07.242 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-24T07:56:07.238+0000 7fb329c97700 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb324071f50 con 0x7fb3240b5ab0 2023-11-24T07:56:07.242 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-24T07:56:07.238+0000 7fb329c97700 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb324072230 con 0x7fb32406b620 2023-11-24T07:56:07.242 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-24T07:56:07.238+0000 7fb329c97700 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb324072510 con 0x7fb3240b4150 2023-11-24T07:56:07.243 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-24T07:56:07.238+0000 7fb322ffd700 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fb3240b4150 0x7fb32406de50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:56:07.243 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-24T07:56:07.238+0000 7fb322ffd700 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fb3240b4150 0x7fb32406de50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:34914/0 (socket says 172.21.15.76:34914) 2023-11-24T07:56:07.243 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-24T07:56:07.238+0000 7fb3237fe700 1 --1- >> v1:172.21.15.76:6789/0 conn(0x7fb32406b620 0x7fb3240b3a40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.76:53858/0 (socket says 172.21.15.76:53858) 2023-11-24T07:56:07.243 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-24T07:56:07.238+0000 7fb322ffd700 1 -- 172.21.15.76:0/180031508 learned_addr learned my addr 172.21.15.76:0/180031508 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-24T07:56:07.243 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-24T07:56:07.238+0000 7fb322ffd700 1 -- 172.21.15.76:0/180031508 >> v1:172.21.15.76:6789/0 conn(0x7fb32406b620 legacy=0x7fb3240b3a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:56:07.243 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-24T07:56:07.238+0000 7fb3227fc700 1 -- 172.21.15.76:0/180031508 <== mon.0 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3668545588 0 0) 0x7fb324071f50 con 0x7fb3240b5ab0 2023-11-24T07:56:07.243 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-24T07:56:07.238+0000 7fb322ffd700 1 -- 172.21.15.76:0/180031508 >> v1:172.21.15.59:6789/0 conn(0x7fb3240b5ab0 legacy=0x7fb32406e390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:56:07.243 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-24T07:56:07.238+0000 7fb322ffd700 1 -- 172.21.15.76:0/180031508 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb3240727f0 con 0x7fb3240b4150 2023-11-24T07:56:07.244 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-24T07:56:07.238+0000 7fb322ffd700 1 --2- 172.21.15.76:0/180031508 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb3240b4150 0x7fb32406de50 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fb318009870 tx=0x7fb318009b80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8f30426f15fb4d server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:56:07.244 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-24T07:56:07.239+0000 7fb3227fc700 1 -- 172.21.15.76:0/180031508 <== mon.1 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb318004230 con 0x7fb3240b4150 2023-11-24T07:56:07.244 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-24T07:56:07.239+0000 7fb3227fc700 1 -- 172.21.15.76:0/180031508 <== mon.1 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb318004390 con 0x7fb3240b4150 2023-11-24T07:56:07.244 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-24T07:56:07.239+0000 7fb3227fc700 1 -- 172.21.15.76:0/180031508 <== mon.1 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3180036a0 con 0x7fb3240b4150 2023-11-24T07:56:07.244 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-24T07:56:07.239+0000 7fb329c97700 1 -- 172.21.15.76:0/180031508 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb3240b4150 msgr2=0x7fb32406de50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:56:07.244 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-24T07:56:07.239+0000 7fb329c97700 1 --2- 172.21.15.76:0/180031508 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb3240b4150 0x7fb32406de50 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fb318009870 tx=0x7fb318009b80 comp rx=0 tx=0).stop 2023-11-24T07:56:07.244 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-24T07:56:07.239+0000 7fb329c97700 1 -- 172.21.15.76:0/180031508 shutdown_connections 2023-11-24T07:56:07.244 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-24T07:56:07.239+0000 7fb329c97700 1 --2- 172.21.15.76:0/180031508 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb3240b4150 0x7fb32406de50 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:56:07.244 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-24T07:56:07.239+0000 7fb329c97700 1 -- 172.21.15.76:0/180031508 >> 172.21.15.76:0/180031508 conn(0x7fb3240b2740 msgr2=0x7fb3240b2b20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:56:07.244 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-24T07:56:07.239+0000 7fb329c97700 1 -- 172.21.15.76:0/180031508 shutdown_connections 2023-11-24T07:56:07.244 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-24T07:56:07.239+0000 7fb329c97700 1 -- 172.21.15.76:0/180031508 wait complete. 2023-11-24T07:56:07.248 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-24T07:56:07.241+0000 7fb329c97700 1 Processor -- start 2023-11-24T07:56:07.248 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-24T07:56:07.241+0000 7fb329c97700 1 -- start start 2023-11-24T07:56:07.248 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-24T07:56:07.243+0000 7fb329c97700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb32406b620 0x7fb3241335b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:56:07.248 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-24T07:56:07.243+0000 7fb329c97700 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fb3240b4150 0x7fb3241e4f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:56:07.249 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-24T07:56:07.243+0000 7fb329c97700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb3240b5ab0 0x7fb3240b0150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:56:07.249 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-24T07:56:07.243+0000 7fb329c97700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb324134000 con 0x7fb3240b5ab0 2023-11-24T07:56:07.249 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-24T07:56:07.243+0000 7fb329c97700 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3240c68f0 con 0x7fb32406b620 2023-11-24T07:56:07.249 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-24T07:56:07.243+0000 7fb329c97700 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb3241f3d70 con 0x7fb3240b4150 2023-11-24T07:56:07.249 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-24T07:56:07.243+0000 7fb3237fe700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb32406b620 0x7fb3241335b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:56:07.249 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-24T07:56:07.243+0000 7fb322ffd700 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fb3240b4150 0x7fb3241e4f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:56:07.249 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-24T07:56:07.243+0000 7fb3237fe700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb32406b620 0x7fb3241335b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:34918/0 (socket says 172.21.15.76:34918) 2023-11-24T07:56:07.249 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-24T07:56:07.243+0000 7fb3237fe700 1 -- 172.21.15.76:0/62541582 learned_addr learned my addr 172.21.15.76:0/62541582 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-24T07:56:07.249 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-24T07:56:07.243+0000 7fb323fff700 1 --2- 172.21.15.76:0/62541582 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb3240b5ab0 0x7fb3240b0150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:56:07.249 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-24T07:56:07.243+0000 7fb3237fe700 1 -- 172.21.15.76:0/62541582 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fb3240b4150 msgr2=0x7fb3241e4f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:56:07.249 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-24T07:56:07.243+0000 7fb3237fe700 1 --2- 172.21.15.76:0/62541582 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fb3240b4150 0x7fb3241e4f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:56:07.250 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-24T07:56:07.243+0000 7fb3237fe700 1 -- 172.21.15.76:0/62541582 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb3240b5ab0 msgr2=0x7fb3240b0150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:56:07.250 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-24T07:56:07.243+0000 7fb3237fe700 1 --2- 172.21.15.76:0/62541582 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb3240b5ab0 0x7fb3240b0150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:56:07.250 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-24T07:56:07.243+0000 7fb3237fe700 1 -- 172.21.15.76:0/62541582 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb318009550 con 0x7fb32406b620 2023-11-24T07:56:07.250 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-24T07:56:07.243+0000 7fb322ffd700 1 --2- 172.21.15.76:0/62541582 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fb3240b4150 0x7fb3241e4f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-24T07:56:07.250 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-24T07:56:07.243+0000 7fb3237fe700 1 --2- 172.21.15.76:0/62541582 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb32406b620 0x7fb3241335b0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fb31400c780 tx=0x7fb31400ca90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:56:07.251 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-24T07:56:07.243+0000 7fb30bfff700 1 -- 172.21.15.76:0/62541582 <== mon.1 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb314004c80 con 0x7fb32406b620 2023-11-24T07:56:07.251 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-24T07:56:07.243+0000 7fb30bfff700 1 -- 172.21.15.76:0/62541582 <== mon.1 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb314004de0 con 0x7fb32406b620 2023-11-24T07:56:07.251 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-24T07:56:07.243+0000 7fb329c97700 1 -- 172.21.15.76:0/62541582 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb324133af0 con 0x7fb32406b620 2023-11-24T07:56:07.251 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-24T07:56:07.243+0000 7fb30bfff700 1 -- 172.21.15.76:0/62541582 <== mon.1 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb314007310 con 0x7fb32406b620 2023-11-24T07:56:07.251 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-24T07:56:07.243+0000 7fb329c97700 1 -- 172.21.15.76:0/62541582 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb3240c5d90 con 0x7fb32406b620 2023-11-24T07:56:07.251 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-24T07:56:07.244+0000 7fb30bfff700 1 -- 172.21.15.76:0/62541582 <== mon.1 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7fb31401d3f0 con 0x7fb32406b620 2023-11-24T07:56:07.251 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-24T07:56:07.245+0000 7fb30bfff700 1 -- 172.21.15.76:0/62541582 <== mon.1 v2:172.21.15.76:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 725+0+0 (secure 0 0 0) 0x7fb314013700 con 0x7fb32406b620 2023-11-24T07:56:07.251 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-24T07:56:07.244+0000 7fb329c97700 1 -- 172.21.15.76:0/62541582 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb324068180 con 0x7fb32406b620 2023-11-24T07:56:07.252 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-24T07:56:07.247+0000 7fb30bfff700 1 -- 172.21.15.76:0/62541582 <== mon.1 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72898 (secure 0 0 0) 0x7fb3140139c0 con 0x7fb32406b620 2023-11-24T07:56:07.333 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-24T07:56:07.331+0000 7fb329c97700 1 -- 172.21.15.76:0/62541582 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "osd new", "uuid": "6d8f9e9b-1782-4e41-928b-a39f2a8ebc06", "id": 0} v 0) v1 -- 0x7fb324075480 con 0x7fb32406b620 2023-11-24T07:56:07.336 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-24T07:56:07.334+0000 7fb30bfff700 1 -- 172.21.15.76:0/62541582 <== mon.1 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "6d8f9e9b-1782-4e41-928b-a39f2a8ebc06", "id": 0}]=0 v3) v1 ==== 112+0+2 (secure 0 0 0) 0x7fb314013c30 con 0x7fb32406b620 2023-11-24T07:56:07.336 INFO:teuthology.orchestra.run.smithi076.stdout:0 2023-11-24T07:56:07.337 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-24T07:56:07.335+0000 7fb329c97700 1 -- 172.21.15.76:0/62541582 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb32406b620 msgr2=0x7fb3241335b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:56:07.338 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-24T07:56:07.335+0000 7fb329c97700 1 --2- 172.21.15.76:0/62541582 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb32406b620 0x7fb3241335b0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fb31400c780 tx=0x7fb31400ca90 comp rx=0 tx=0).stop 2023-11-24T07:56:07.338 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-24T07:56:07.335+0000 7fb329c97700 1 -- 172.21.15.76:0/62541582 shutdown_connections 2023-11-24T07:56:07.338 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-24T07:56:07.335+0000 7fb329c97700 1 --2- 172.21.15.76:0/62541582 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb3240b5ab0 0x7fb3240b0150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:56:07.338 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-24T07:56:07.335+0000 7fb329c97700 1 --2- 172.21.15.76:0/62541582 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb32406b620 0x7fb3241335b0 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:56:07.338 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-24T07:56:07.335+0000 7fb329c97700 1 --2- 172.21.15.76:0/62541582 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fb3240b4150 0x7fb3241e4f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:56:07.338 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-24T07:56:07.335+0000 7fb329c97700 1 -- 172.21.15.76:0/62541582 >> 172.21.15.76:0/62541582 conn(0x7fb3240b2740 msgr2=0x7fb324146340 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:56:07.338 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-24T07:56:07.336+0000 7fb329c97700 1 -- 172.21.15.76:0/62541582 shutdown_connections 2023-11-24T07:56:07.338 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-24T07:56:07.336+0000 7fb329c97700 1 -- 172.21.15.76:0/62541582 wait complete. 2023-11-24T07:56:07.350 DEBUG:teuthology.orchestra.run.smithi076:> sudo ceph --cluster ceph osd new e8d882aa-52e1-4c04-9e1e-c44ef9204a5f 1 2023-11-24T07:56:07.471 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-24T07:56:07.469+0000 7f52db125700 1 Processor -- start 2023-11-24T07:56:07.471 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-24T07:56:07.469+0000 7f52db125700 1 -- start start 2023-11-24T07:56:07.476 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-24T07:56:07.469+0000 7f52db125700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f52d40b03b0 0x7f52d40b5e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:56:07.476 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-24T07:56:07.469+0000 7f52db125700 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f52d40adda0 0x7f52d406b970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:56:07.476 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-24T07:56:07.469+0000 7f52db125700 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f52d40b0ec0 con 0x7f52d415ae00 2023-11-24T07:56:07.477 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-24T07:56:07.469+0000 7f52db125700 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f52d4073610 con 0x7f52d40b03b0 2023-11-24T07:56:07.477 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-24T07:56:07.469+0000 7f52db125700 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f52d415b1e0 con 0x7f52d40adda0 2023-11-24T07:56:07.477 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-24T07:56:07.470+0000 7f52d96c2700 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f52d40adda0 0x7f52d406b970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:56:07.477 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-24T07:56:07.470+0000 7f52d96c2700 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f52d40adda0 0x7f52d406b970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:34928/0 (socket says 172.21.15.76:34928) 2023-11-24T07:56:07.477 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-24T07:56:07.470+0000 7f52d96c2700 1 -- 172.21.15.76:0/60977004 learned_addr learned my addr 172.21.15.76:0/60977004 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-24T07:56:07.477 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-24T07:56:07.470+0000 7f52d8ec1700 1 --1- 172.21.15.76:0/60977004 >> v1:172.21.15.76:6789/0 conn(0x7f52d415ae00 0x7f52d40ad690 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.76:53870/0 (socket says 172.21.15.76:53870) 2023-11-24T07:56:07.477 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-24T07:56:07.470+0000 7f52d3fff700 1 --2- 172.21.15.76:0/60977004 >> v2:172.21.15.59:3300/0 conn(0x7f52d40b03b0 0x7f52d40b5e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:56:07.477 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-24T07:56:07.470+0000 7f52d96c2700 1 -- 172.21.15.76:0/60977004 >> v2:172.21.15.59:3300/0 conn(0x7f52d40b03b0 msgr2=0x7f52d40b5e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T07:56:07.477 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-24T07:56:07.470+0000 7f52d96c2700 1 --2- 172.21.15.76:0/60977004 >> v2:172.21.15.59:3300/0 conn(0x7f52d40b03b0 0x7f52d40b5e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:56:07.477 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-24T07:56:07.470+0000 7f52d96c2700 1 -- 172.21.15.76:0/60977004 >> v1:172.21.15.76:6789/0 conn(0x7f52d415ae00 legacy=0x7f52d40ad690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:56:07.477 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-24T07:56:07.470+0000 7f52d96c2700 1 -- 172.21.15.76:0/60977004 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f52d40b0900 con 0x7f52d40adda0 2023-11-24T07:56:07.477 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-24T07:56:07.470+0000 7f52d96c2700 1 --2- 172.21.15.76:0/60977004 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f52d40adda0 0x7f52d406b970 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f52d40706d0 tx=0x7f52cc00b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fe85a81877aaf7cb server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:56:07.478 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-24T07:56:07.470+0000 7f52d37fe700 1 -- 172.21.15.76:0/60977004 <== mon.1 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52cc00e070 con 0x7f52d40adda0 2023-11-24T07:56:07.478 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-24T07:56:07.470+0000 7f52d37fe700 1 -- 172.21.15.76:0/60977004 <== mon.1 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f52cc00b910 con 0x7f52d40adda0 2023-11-24T07:56:07.478 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-24T07:56:07.470+0000 7f52d37fe700 1 -- 172.21.15.76:0/60977004 <== mon.1 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52cc00bcc0 con 0x7f52d40adda0 2023-11-24T07:56:07.478 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-24T07:56:07.471+0000 7f52db125700 1 -- 172.21.15.76:0/60977004 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f52d40adda0 msgr2=0x7f52d406b970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:56:07.478 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-24T07:56:07.471+0000 7f52db125700 1 --2- 172.21.15.76:0/60977004 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f52d40adda0 0x7f52d406b970 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f52d40706d0 tx=0x7f52cc00b330 comp rx=0 tx=0).stop 2023-11-24T07:56:07.478 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-24T07:56:07.471+0000 7f52db125700 1 -- 172.21.15.76:0/60977004 shutdown_connections 2023-11-24T07:56:07.478 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-24T07:56:07.471+0000 7f52db125700 1 --2- 172.21.15.76:0/60977004 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f52d40adda0 0x7f52d406b970 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:56:07.478 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-24T07:56:07.471+0000 7f52db125700 1 --2- 172.21.15.76:0/60977004 >> v2:172.21.15.59:3300/0 conn(0x7f52d40b03b0 0x7f52d40b5e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:56:07.478 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-24T07:56:07.471+0000 7f52db125700 1 -- 172.21.15.76:0/60977004 >> 172.21.15.76:0/60977004 conn(0x7f52d40b5070 msgr2=0x7f52d40b5730 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:56:07.478 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-24T07:56:07.471+0000 7f52db125700 1 -- 172.21.15.76:0/60977004 shutdown_connections 2023-11-24T07:56:07.478 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-24T07:56:07.471+0000 7f52db125700 1 -- 172.21.15.76:0/60977004 wait complete. 2023-11-24T07:56:07.478 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-24T07:56:07.472+0000 7f52db125700 1 Processor -- start 2023-11-24T07:56:07.478 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-24T07:56:07.472+0000 7f52db125700 1 -- start start 2023-11-24T07:56:07.479 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-24T07:56:07.473+0000 7f52db125700 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f52d40adda0 0x7f52d41583f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:56:07.479 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-24T07:56:07.473+0000 7f52db125700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f52d40b03b0 0x7f52d4158e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:56:07.479 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-24T07:56:07.473+0000 7f52db125700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f52d415ae00 0x7f52d414d970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:56:07.479 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-24T07:56:07.473+0000 7f52db125700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52d414f770 con 0x7f52d40b03b0 2023-11-24T07:56:07.479 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-24T07:56:07.473+0000 7f52db125700 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52d4159340 con 0x7f52d415ae00 2023-11-24T07:56:07.479 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-24T07:56:07.473+0000 7f52db125700 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f52d41589d0 con 0x7f52d40adda0 2023-11-24T07:56:07.479 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-24T07:56:07.473+0000 7f52d8ec1700 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f52d40adda0 0x7f52d41583f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:56:07.479 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-24T07:56:07.473+0000 7f52d96c2700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f52d415ae00 0x7f52d414d970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:56:07.479 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-24T07:56:07.473+0000 7f52d96c2700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f52d415ae00 0x7f52d414d970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:34934/0 (socket says 172.21.15.76:34934) 2023-11-24T07:56:07.479 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-24T07:56:07.473+0000 7f52d8ec1700 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f52d40adda0 0x7f52d41583f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3301/0 says I am v2:172.21.15.76:54394/0 (socket says 172.21.15.76:54394) 2023-11-24T07:56:07.479 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-24T07:56:07.473+0000 7f52d96c2700 1 -- 172.21.15.76:0/4291321960 learned_addr learned my addr 172.21.15.76:0/4291321960 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-24T07:56:07.479 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-24T07:56:07.473+0000 7f52d3fff700 1 --2- 172.21.15.76:0/4291321960 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f52d40b03b0 0x7f52d4158e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:56:07.479 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-24T07:56:07.473+0000 7f52d8ec1700 1 -- 172.21.15.76:0/4291321960 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f52d415ae00 msgr2=0x7f52d414d970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:56:07.480 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-24T07:56:07.473+0000 7f52d8ec1700 1 --2- 172.21.15.76:0/4291321960 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f52d415ae00 0x7f52d414d970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:56:07.480 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-24T07:56:07.473+0000 7f52d8ec1700 1 -- 172.21.15.76:0/4291321960 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f52d40b03b0 msgr2=0x7f52d4158e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:56:07.480 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-24T07:56:07.473+0000 7f52d8ec1700 1 --2- 172.21.15.76:0/4291321960 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f52d40b03b0 0x7f52d4158e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:56:07.480 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-24T07:56:07.473+0000 7f52d8ec1700 1 -- 172.21.15.76:0/4291321960 --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f52cc009b30 con 0x7f52d40adda0 2023-11-24T07:56:07.480 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-24T07:56:07.474+0000 7f52d8ec1700 1 --2- 172.21.15.76:0/4291321960 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f52d40adda0 0x7f52d41583f0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f52c800c480 tx=0x7f52c800c790 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:56:07.480 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-24T07:56:07.474+0000 7f52d17fa700 1 -- 172.21.15.76:0/4291321960 <== mon.2 v2:172.21.15.76:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52c8005cb0 con 0x7f52d40adda0 2023-11-24T07:56:07.480 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-24T07:56:07.474+0000 7f52d17fa700 1 -- 172.21.15.76:0/4291321960 <== mon.2 v2:172.21.15.76:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f52c8005e10 con 0x7f52d40adda0 2023-11-24T07:56:07.480 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-24T07:56:07.474+0000 7f52db125700 1 -- 172.21.15.76:0/4291321960 --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f52d414edd0 con 0x7f52d40adda0 2023-11-24T07:56:07.480 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-24T07:56:07.474+0000 7f52d17fa700 1 -- 172.21.15.76:0/4291321960 <== mon.2 v2:172.21.15.76:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52c80072d0 con 0x7f52d40adda0 2023-11-24T07:56:07.480 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-24T07:56:07.474+0000 7f52db125700 1 -- 172.21.15.76:0/4291321960 --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f52d414f390 con 0x7f52d40adda0 2023-11-24T07:56:07.480 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-24T07:56:07.474+0000 7f52d17fa700 1 -- 172.21.15.76:0/4291321960 <== mon.2 v2:172.21.15.76:3301/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f52c801c440 con 0x7f52d40adda0 2023-11-24T07:56:07.480 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-24T07:56:07.474+0000 7f52d17fa700 1 -- 172.21.15.76:0/4291321960 <== mon.2 v2:172.21.15.76:3301/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 840+0+0 (secure 0 0 0) 0x7f52c80127b0 con 0x7f52d40adda0 2023-11-24T07:56:07.480 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-24T07:56:07.475+0000 7f52db125700 1 -- 172.21.15.76:0/4291321960 --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f52d4075480 con 0x7f52d40adda0 2023-11-24T07:56:07.481 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-24T07:56:07.477+0000 7f52d17fa700 1 -- 172.21.15.76:0/4291321960 <== mon.2 v2:172.21.15.76:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72898 (secure 0 0 0) 0x7f52c8021060 con 0x7f52d40adda0 2023-11-24T07:56:07.490 INFO:tasks.ceph.mgr.x.smithi076.stderr:2023-11-24T07:56:07.489+0000 7ff220fa5200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-11-24T07:56:07.518 INFO:tasks.ceph.mgr.y.smithi059.stderr:2023-11-24T07:56:07.516+0000 7eff6f463200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-11-24T07:56:07.565 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-24T07:56:07.563+0000 7f52db125700 1 -- 172.21.15.76:0/4291321960 --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_command({"prefix": "osd new", "uuid": "e8d882aa-52e1-4c04-9e1e-c44ef9204a5f", "id": 1} v 0) v1 -- 0x7f52d4151fd0 con 0x7f52d40adda0 2023-11-24T07:56:07.567 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-24T07:56:07.566+0000 7f52d17fa700 1 -- 172.21.15.76:0/4291321960 <== mon.2 v2:172.21.15.76:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "e8d882aa-52e1-4c04-9e1e-c44ef9204a5f", "id": 1}]=0 v4) v1 ==== 112+0+2 (secure 0 0 0) 0x7f52d4151fd0 con 0x7f52d40adda0 2023-11-24T07:56:07.568 INFO:teuthology.orchestra.run.smithi076.stdout:1 2023-11-24T07:56:07.569 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-24T07:56:07.567+0000 7f52db125700 1 -- 172.21.15.76:0/4291321960 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f52d40adda0 msgr2=0x7f52d41583f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:56:07.569 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-24T07:56:07.567+0000 7f52db125700 1 --2- 172.21.15.76:0/4291321960 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f52d40adda0 0x7f52d41583f0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f52c800c480 tx=0x7f52c800c790 comp rx=0 tx=0).stop 2023-11-24T07:56:07.569 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-24T07:56:07.567+0000 7f52db125700 1 -- 172.21.15.76:0/4291321960 shutdown_connections 2023-11-24T07:56:07.569 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-24T07:56:07.567+0000 7f52db125700 1 --2- 172.21.15.76:0/4291321960 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f52d415ae00 0x7f52d414d970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:56:07.569 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-24T07:56:07.567+0000 7f52db125700 1 --2- 172.21.15.76:0/4291321960 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f52d40adda0 0x7f52d41583f0 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:56:07.569 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-24T07:56:07.567+0000 7f52db125700 1 --2- 172.21.15.76:0/4291321960 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f52d40b03b0 0x7f52d4158e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:56:07.569 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-24T07:56:07.567+0000 7f52db125700 1 -- 172.21.15.76:0/4291321960 >> 172.21.15.76:0/4291321960 conn(0x7f52d40b5070 msgr2=0x7f52d40ad690 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:56:07.570 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-24T07:56:07.567+0000 7f52db125700 1 -- 172.21.15.76:0/4291321960 shutdown_connections 2023-11-24T07:56:07.570 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-24T07:56:07.567+0000 7f52db125700 1 -- 172.21.15.76:0/4291321960 wait complete. 2023-11-24T07:56:07.583 DEBUG:teuthology.orchestra.run.smithi076:> sudo ceph --cluster ceph osd new 69eebfa0-695b-4b21-90b9-8a3537168a01 2 2023-11-24T07:56:07.615 INFO:tasks.ceph.mgr.x.smithi076.stderr:2023-11-24T07:56:07.614+0000 7ff220fa5200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-11-24T07:56:07.639 INFO:tasks.ceph.mgr.y.smithi059.stderr:2023-11-24T07:56:07.637+0000 7eff6f463200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-11-24T07:56:07.705 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-24T07:56:07.703+0000 7fae11408700 1 Processor -- start 2023-11-24T07:56:07.706 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-24T07:56:07.704+0000 7fae11408700 1 -- start start 2023-11-24T07:56:07.708 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-24T07:56:07.704+0000 7fae11408700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fae0c0b5ab0 0x7fae0c06e560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:56:07.708 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-24T07:56:07.704+0000 7fae11408700 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fae0c071f50 con 0x7fae0c0b4150 2023-11-24T07:56:07.708 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-24T07:56:07.704+0000 7fae11408700 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fae0c072230 con 0x7fae0c06b620 2023-11-24T07:56:07.708 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-24T07:56:07.704+0000 7fae11408700 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fae0c072510 con 0x7fae0c0b5ab0 2023-11-24T07:56:07.708 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-24T07:56:07.705+0000 7fae0affd700 1 --1- >> v1:172.21.15.76:6789/0 conn(0x7fae0c06b620 0x7fae0c0b3a40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.76:53882/0 (socket says 172.21.15.76:53882) 2023-11-24T07:56:07.708 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-24T07:56:07.705+0000 7fae0affd700 1 -- 172.21.15.76:0/2344763336 learned_addr learned my addr 172.21.15.76:0/2344763336 (peer_addr_for_me v1:172.21.15.76:0/0) 2023-11-24T07:56:07.708 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-24T07:56:07.705+0000 7fae0b7fe700 1 --2- 172.21.15.76:0/2344763336 >> v2:172.21.15.59:3300/0 conn(0x7fae0c0b5ab0 0x7fae0c06e560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:56:07.709 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-24T07:56:07.705+0000 7fae09ffb700 1 -- 172.21.15.76:0/2344763336 <== mon.1 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 303175472 0 0) 0x7fae0c072230 con 0x7fae0c06b620 2023-11-24T07:56:07.709 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-24T07:56:07.705+0000 7fae09ffb700 1 -- 172.21.15.76:0/2344763336 --> v1:172.21.15.76:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fadf40035b0 con 0x7fae0c06b620 2023-11-24T07:56:07.709 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-24T07:56:07.705+0000 7fae09ffb700 1 -- 172.21.15.76:0/2344763336 <== mon.0 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 726994998 0 0) 0x7fae0c071f50 con 0x7fae0c0b4150 2023-11-24T07:56:07.709 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-24T07:56:07.705+0000 7fae09ffb700 1 -- 172.21.15.76:0/2344763336 --> v1:172.21.15.59:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fae0c072230 con 0x7fae0c0b4150 2023-11-24T07:56:07.709 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-24T07:56:07.705+0000 7fae0b7fe700 1 -- 172.21.15.76:0/2344763336 >> v1:172.21.15.76:6789/0 conn(0x7fae0c06b620 legacy=0x7fae0c0b3a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:56:07.709 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-24T07:56:07.705+0000 7fae0b7fe700 1 -- 172.21.15.76:0/2344763336 >> v1:172.21.15.59:6789/0 conn(0x7fae0c0b4150 legacy=0x7fae0c06de50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:56:07.709 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-24T07:56:07.705+0000 7fae0b7fe700 1 -- 172.21.15.76:0/2344763336 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fae0c0727f0 con 0x7fae0c0b5ab0 2023-11-24T07:56:07.709 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-24T07:56:07.705+0000 7fae0b7fe700 1 --2- 172.21.15.76:0/2344763336 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fae0c0b5ab0 0x7fae0c06e560 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7fae04009d90 tx=0x7fae0400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a943c9449d98f61d server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:56:07.709 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-24T07:56:07.705+0000 7fae09ffb700 1 -- 172.21.15.76:0/2344763336 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae0400e070 con 0x7fae0c0b5ab0 2023-11-24T07:56:07.709 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-24T07:56:07.705+0000 7fae09ffb700 1 -- 172.21.15.76:0/2344763336 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fae0400bb20 con 0x7fae0c0b5ab0 2023-11-24T07:56:07.709 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-24T07:56:07.705+0000 7fae09ffb700 1 -- 172.21.15.76:0/2344763336 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae04012430 con 0x7fae0c0b5ab0 2023-11-24T07:56:07.709 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-24T07:56:07.706+0000 7fae11408700 1 -- 172.21.15.76:0/2344763336 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fae0c0b5ab0 msgr2=0x7fae0c06e560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:56:07.710 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-24T07:56:07.706+0000 7fae11408700 1 --2- 172.21.15.76:0/2344763336 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fae0c0b5ab0 0x7fae0c06e560 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7fae04009d90 tx=0x7fae0400b330 comp rx=0 tx=0).stop 2023-11-24T07:56:07.710 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-24T07:56:07.706+0000 7fae11408700 1 -- 172.21.15.76:0/2344763336 shutdown_connections 2023-11-24T07:56:07.710 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-24T07:56:07.706+0000 7fae11408700 1 --2- 172.21.15.76:0/2344763336 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fae0c0b5ab0 0x7fae0c06e560 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:56:07.710 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-24T07:56:07.706+0000 7fae11408700 1 -- 172.21.15.76:0/2344763336 >> 172.21.15.76:0/2344763336 conn(0x7fae0c0b2740 msgr2=0x7fae0c0b2b20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:56:07.710 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-24T07:56:07.706+0000 7fae11408700 1 -- 172.21.15.76:0/2344763336 shutdown_connections 2023-11-24T07:56:07.710 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-24T07:56:07.706+0000 7fae11408700 1 -- 172.21.15.76:0/2344763336 wait complete. 2023-11-24T07:56:07.712 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-24T07:56:07.707+0000 7fae11408700 1 Processor -- start 2023-11-24T07:56:07.712 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-24T07:56:07.707+0000 7fae11408700 1 -- start start 2023-11-24T07:56:07.712 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-24T07:56:07.708+0000 7fae11408700 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fae0c06b620 0x7fae0c1d9cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:56:07.712 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-24T07:56:07.708+0000 7fae11408700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fae0c0b4150 0x7fae0c1f3130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:56:07.712 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-24T07:56:07.708+0000 7fae11408700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fae0c0b5ab0 0x7fae0c0c63c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:56:07.713 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-24T07:56:07.708+0000 7fae11408700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae0c0c6e80 con 0x7fae0c0b5ab0 2023-11-24T07:56:07.713 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-24T07:56:07.708+0000 7fae11408700 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae0c13fb30 con 0x7fae0c0b4150 2023-11-24T07:56:07.713 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-24T07:56:07.708+0000 7fae11408700 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fae0c1f3670 con 0x7fae0c06b620 2023-11-24T07:56:07.713 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-24T07:56:07.708+0000 7fae0a7fc700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fae0c0b4150 0x7fae0c1f3130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:56:07.713 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-24T07:56:07.708+0000 7fae0affd700 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fae0c06b620 0x7fae0c1d9cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:56:07.713 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-24T07:56:07.708+0000 7fae0a7fc700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fae0c0b4150 0x7fae0c1f3130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:34946/0 (socket says 172.21.15.76:34946) 2023-11-24T07:56:07.713 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-24T07:56:07.708+0000 7fae0a7fc700 1 -- 172.21.15.76:0/1508746824 learned_addr learned my addr 172.21.15.76:0/1508746824 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-24T07:56:07.713 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-24T07:56:07.708+0000 7fae0affd700 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fae0c06b620 0x7fae0c1d9cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3301/0 says I am v2:172.21.15.76:54406/0 (socket says 172.21.15.76:54406) 2023-11-24T07:56:07.713 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-24T07:56:07.708+0000 7fae0b7fe700 1 --2- 172.21.15.76:0/1508746824 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fae0c0b5ab0 0x7fae0c0c63c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:56:07.713 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-24T07:56:07.708+0000 7fae0a7fc700 1 -- 172.21.15.76:0/1508746824 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fae0c06b620 msgr2=0x7fae0c1d9cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:56:07.713 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-24T07:56:07.708+0000 7fae0a7fc700 1 --2- 172.21.15.76:0/1508746824 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fae0c06b620 0x7fae0c1d9cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:56:07.713 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-24T07:56:07.708+0000 7fae0a7fc700 1 -- 172.21.15.76:0/1508746824 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fae0c0b5ab0 msgr2=0x7fae0c0c63c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:56:07.714 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-24T07:56:07.708+0000 7fae0a7fc700 1 --2- 172.21.15.76:0/1508746824 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fae0c0b5ab0 0x7fae0c0c63c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:56:07.714 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-24T07:56:07.708+0000 7fae0a7fc700 1 -- 172.21.15.76:0/1508746824 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fae04009a70 con 0x7fae0c0b4150 2023-11-24T07:56:07.714 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-24T07:56:07.709+0000 7fae0a7fc700 1 --2- 172.21.15.76:0/1508746824 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fae0c0b4150 0x7fae0c1f3130 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7fae0000c520 tx=0x7fae0000c830 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:56:07.714 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-24T07:56:07.709+0000 7fadf37fe700 1 -- 172.21.15.76:0/1508746824 <== mon.1 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae000052d0 con 0x7fae0c0b4150 2023-11-24T07:56:07.714 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-24T07:56:07.709+0000 7fadf37fe700 1 -- 172.21.15.76:0/1508746824 <== mon.1 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fae00005430 con 0x7fae0c0b4150 2023-11-24T07:56:07.714 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-24T07:56:07.709+0000 7fae11408700 1 -- 172.21.15.76:0/1508746824 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fae0c13f200 con 0x7fae0c0b4150 2023-11-24T07:56:07.714 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-24T07:56:07.709+0000 7fadf37fe700 1 -- 172.21.15.76:0/1508746824 <== mon.1 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae000038c0 con 0x7fae0c0b4150 2023-11-24T07:56:07.714 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-24T07:56:07.709+0000 7fae11408700 1 -- 172.21.15.76:0/1508746824 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fae0c13f7c0 con 0x7fae0c0b4150 2023-11-24T07:56:07.714 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-24T07:56:07.709+0000 7fadf37fe700 1 -- 172.21.15.76:0/1508746824 <== mon.1 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7fae00003d20 con 0x7fae0c0b4150 2023-11-24T07:56:07.714 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-24T07:56:07.709+0000 7fadf37fe700 1 -- 172.21.15.76:0/1508746824 <== mon.1 v2:172.21.15.76:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 955+0+0 (secure 0 0 0) 0x7fae00012840 con 0x7fae0c0b4150 2023-11-24T07:56:07.714 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-24T07:56:07.710+0000 7fae11408700 1 -- 172.21.15.76:0/1508746824 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fae0c075480 con 0x7fae0c0b4150 2023-11-24T07:56:07.715 INFO:tasks.ceph.mgr.x.smithi076.stderr:2023-11-24T07:56:07.710+0000 7ff220fa5200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-11-24T07:56:07.715 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-24T07:56:07.712+0000 7fadf37fe700 1 -- 172.21.15.76:0/1508746824 <== mon.1 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72898 (secure 0 0 0) 0x7fae00017070 con 0x7fae0c0b4150 2023-11-24T07:56:07.729 INFO:tasks.ceph.mgr.y.smithi059.stderr:2023-11-24T07:56:07.728+0000 7eff6f463200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-11-24T07:56:07.797 INFO:tasks.ceph.mgr.x.smithi076.stderr:2023-11-24T07:56:07.795+0000 7ff220fa5200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-11-24T07:56:07.799 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-24T07:56:07.797+0000 7fae11408700 1 -- 172.21.15.76:0/1508746824 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "osd new", "uuid": "69eebfa0-695b-4b21-90b9-8a3537168a01", "id": 2} v 0) v1 -- 0x7fae0c06bd40 con 0x7fae0c0b4150 2023-11-24T07:56:07.804 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-24T07:56:07.803+0000 7fadf37fe700 1 -- 172.21.15.76:0/1508746824 <== mon.1 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "69eebfa0-695b-4b21-90b9-8a3537168a01", "id": 2}]=0 v5) v1 ==== 112+0+2 (secure 0 0 0) 0x7fae00011e50 con 0x7fae0c0b4150 2023-11-24T07:56:07.805 INFO:teuthology.orchestra.run.smithi076.stdout:2 2023-11-24T07:56:07.805 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-24T07:56:07.803+0000 7fae11408700 1 -- 172.21.15.76:0/1508746824 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fae0c0b4150 msgr2=0x7fae0c1f3130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:56:07.806 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-24T07:56:07.803+0000 7fae11408700 1 --2- 172.21.15.76:0/1508746824 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fae0c0b4150 0x7fae0c1f3130 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7fae0000c520 tx=0x7fae0000c830 comp rx=0 tx=0).stop 2023-11-24T07:56:07.806 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-24T07:56:07.803+0000 7fae11408700 1 -- 172.21.15.76:0/1508746824 shutdown_connections 2023-11-24T07:56:07.806 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-24T07:56:07.803+0000 7fae11408700 1 --2- 172.21.15.76:0/1508746824 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fae0c0b5ab0 0x7fae0c0c63c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:56:07.806 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-24T07:56:07.803+0000 7fae11408700 1 --2- 172.21.15.76:0/1508746824 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fae0c06b620 0x7fae0c1d9cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:56:07.806 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-24T07:56:07.803+0000 7fae11408700 1 --2- 172.21.15.76:0/1508746824 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fae0c0b4150 0x7fae0c1f3130 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:56:07.806 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-24T07:56:07.803+0000 7fae11408700 1 -- 172.21.15.76:0/1508746824 >> 172.21.15.76:0/1508746824 conn(0x7fae0c0b2740 msgr2=0x7fae0c0aefa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:56:07.806 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-24T07:56:07.804+0000 7fae11408700 1 -- 172.21.15.76:0/1508746824 shutdown_connections 2023-11-24T07:56:07.806 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-24T07:56:07.804+0000 7fae11408700 1 -- 172.21.15.76:0/1508746824 wait complete. 2023-11-24T07:56:07.815 INFO:tasks.ceph.mgr.y.smithi059.stderr:2023-11-24T07:56:07.813+0000 7eff6f463200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-11-24T07:56:07.818 DEBUG:teuthology.orchestra.run.smithi076:> sudo ceph --cluster ceph osd new 96fd77fc-f911-46ff-9445-6f0be2daa8f4 3 2023-11-24T07:56:07.981 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-24T07:56:07.979+0000 7f6741b23700 1 Processor -- start 2023-11-24T07:56:07.981 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-24T07:56:07.979+0000 7f6741b23700 1 -- start start 2023-11-24T07:56:07.982 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-24T07:56:07.980+0000 7f6741b23700 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f673c06f540 0x7f673c0b4880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:56:07.982 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-24T07:56:07.980+0000 7f6741b23700 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f673c0ce5b0 con 0x7f673c0b3ab0 2023-11-24T07:56:07.983 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-24T07:56:07.980+0000 7f6741b23700 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f673c072d70 con 0x7f673c15add0 2023-11-24T07:56:07.983 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-24T07:56:07.980+0000 7f6741b23700 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f673c0b4f10 con 0x7f673c06f540 2023-11-24T07:56:07.983 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-24T07:56:07.980+0000 7f673affd700 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f673c06f540 0x7f673c0b4880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:56:07.983 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-24T07:56:07.980+0000 7f673affd700 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f673c06f540 0x7f673c0b4880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:34952/0 (socket says 172.21.15.76:34952) 2023-11-24T07:56:07.983 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-24T07:56:07.980+0000 7f673affd700 1 -- 172.21.15.76:0/3224967666 learned_addr learned my addr 172.21.15.76:0/3224967666 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-24T07:56:07.983 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-24T07:56:07.980+0000 7f673b7fe700 1 --1- 172.21.15.76:0/3224967666 >> v1:172.21.15.76:6789/0 conn(0x7f673c15add0 0x7f673c06ee30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.76:53894/0 (socket says 172.21.15.76:53894) 2023-11-24T07:56:07.983 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-24T07:56:07.980+0000 7f673affd700 1 -- 172.21.15.76:0/3224967666 >> v1:172.21.15.76:6789/0 conn(0x7f673c15add0 legacy=0x7f673c06ee30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:56:07.983 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-24T07:56:07.980+0000 7f673affd700 1 -- 172.21.15.76:0/3224967666 >> v1:172.21.15.59:6789/0 conn(0x7f673c0b3ab0 legacy=0x7f673c0cea40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:56:07.983 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-24T07:56:07.980+0000 7f673affd700 1 -- 172.21.15.76:0/3224967666 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f673c0b51f0 con 0x7f673c06f540 2023-11-24T07:56:07.983 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-24T07:56:07.980+0000 7f673affd700 1 --2- 172.21.15.76:0/3224967666 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f673c06f540 0x7f673c0b4880 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f67300098f0 tx=0x7f6730009c00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c02eee76919ae760 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:56:07.983 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-24T07:56:07.980+0000 7f673a7fc700 1 -- 172.21.15.76:0/3224967666 <== mon.1 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f673000f8a0 con 0x7f673c06f540 2023-11-24T07:56:07.983 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-24T07:56:07.980+0000 7f673a7fc700 1 -- 172.21.15.76:0/3224967666 <== mon.1 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f673000fa00 con 0x7f673c06f540 2023-11-24T07:56:07.983 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-24T07:56:07.980+0000 7f673a7fc700 1 -- 172.21.15.76:0/3224967666 <== mon.1 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f673000fd90 con 0x7f673c06f540 2023-11-24T07:56:07.984 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-24T07:56:07.981+0000 7f6741b23700 1 -- 172.21.15.76:0/3224967666 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f673c06f540 msgr2=0x7f673c0b4880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:56:07.985 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-24T07:56:07.981+0000 7f6741b23700 1 --2- 172.21.15.76:0/3224967666 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f673c06f540 0x7f673c0b4880 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f67300098f0 tx=0x7f6730009c00 comp rx=0 tx=0).stop 2023-11-24T07:56:07.985 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-24T07:56:07.981+0000 7f6741b23700 1 -- 172.21.15.76:0/3224967666 shutdown_connections 2023-11-24T07:56:07.985 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-24T07:56:07.981+0000 7f6741b23700 1 --2- 172.21.15.76:0/3224967666 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f673c06f540 0x7f673c0b4880 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:56:07.985 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-24T07:56:07.981+0000 7f6741b23700 1 -- 172.21.15.76:0/3224967666 >> 172.21.15.76:0/3224967666 conn(0x7f673c0b27b0 msgr2=0x7f673c0b2b90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:56:07.985 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-24T07:56:07.981+0000 7f6741b23700 1 -- 172.21.15.76:0/3224967666 shutdown_connections 2023-11-24T07:56:07.985 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-24T07:56:07.981+0000 7f6741b23700 1 -- 172.21.15.76:0/3224967666 wait complete. 2023-11-24T07:56:07.985 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-24T07:56:07.982+0000 7f6741b23700 1 Processor -- start 2023-11-24T07:56:07.985 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-24T07:56:07.983+0000 7f6741b23700 1 -- start start 2023-11-24T07:56:07.987 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-24T07:56:07.983+0000 7f6741b23700 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f673c06f540 0x7f673c142ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:56:07.987 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-24T07:56:07.983+0000 7f6741b23700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f673c0b3ab0 0x7f673c1556e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:56:07.987 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-24T07:56:07.983+0000 7f6741b23700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f673c15add0 0x7f673c1528d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:56:07.988 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-24T07:56:07.983+0000 7f6741b23700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f673c15cd10 con 0x7f673c15add0 2023-11-24T07:56:07.988 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-24T07:56:07.983+0000 7f6741b23700 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f673c1467f0 con 0x7f673c0b3ab0 2023-11-24T07:56:07.988 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-24T07:56:07.983+0000 7f6741b23700 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f673c143400 con 0x7f673c06f540 2023-11-24T07:56:07.988 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-24T07:56:07.983+0000 7f673affd700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f673c0b3ab0 0x7f673c1556e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:56:07.988 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-24T07:56:07.983+0000 7f673b7fe700 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f673c06f540 0x7f673c142ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:56:07.988 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-24T07:56:07.983+0000 7f673affd700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f673c0b3ab0 0x7f673c1556e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:34956/0 (socket says 172.21.15.76:34956) 2023-11-24T07:56:07.988 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-24T07:56:07.984+0000 7f673affd700 1 -- 172.21.15.76:0/1627835111 learned_addr learned my addr 172.21.15.76:0/1627835111 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-24T07:56:07.988 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-24T07:56:07.984+0000 7f673bfff700 1 --2- 172.21.15.76:0/1627835111 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f673c15add0 0x7f673c1528d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:56:07.988 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-24T07:56:07.984+0000 7f673affd700 1 -- 172.21.15.76:0/1627835111 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f673c06f540 msgr2=0x7f673c142ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:56:07.988 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-24T07:56:07.984+0000 7f673affd700 1 --2- 172.21.15.76:0/1627835111 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f673c06f540 0x7f673c142ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:56:07.988 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-24T07:56:07.984+0000 7f673affd700 1 -- 172.21.15.76:0/1627835111 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f673c15add0 msgr2=0x7f673c1528d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:56:07.988 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-24T07:56:07.984+0000 7f673affd700 1 --2- 172.21.15.76:0/1627835111 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f673c15add0 0x7f673c1528d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:56:07.989 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-24T07:56:07.984+0000 7f673affd700 1 -- 172.21.15.76:0/1627835111 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6730009550 con 0x7f673c0b3ab0 2023-11-24T07:56:07.989 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-24T07:56:07.984+0000 7f673b7fe700 1 --2- 172.21.15.76:0/1627835111 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f673c06f540 0x7f673c142ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-24T07:56:07.989 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-24T07:56:07.984+0000 7f673affd700 1 --2- 172.21.15.76:0/1627835111 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f673c0b3ab0 0x7f673c1556e0 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f673c070640 tx=0x7f6730004750 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:56:07.989 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-24T07:56:07.984+0000 7f6723fff700 1 -- 172.21.15.76:0/1627835111 <== mon.1 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f673001e070 con 0x7f673c0b3ab0 2023-11-24T07:56:07.989 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-24T07:56:07.984+0000 7f6723fff700 1 -- 172.21.15.76:0/1627835111 <== mon.1 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6730004430 con 0x7f673c0b3ab0 2023-11-24T07:56:07.989 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-24T07:56:07.984+0000 7f6723fff700 1 -- 172.21.15.76:0/1627835111 <== mon.1 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67300038e0 con 0x7f673c0b3ab0 2023-11-24T07:56:07.989 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-24T07:56:07.984+0000 7f6741b23700 1 -- 172.21.15.76:0/1627835111 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f673c1f3220 con 0x7f673c0b3ab0 2023-11-24T07:56:07.989 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-24T07:56:07.984+0000 7f6741b23700 1 -- 172.21.15.76:0/1627835111 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f673c1f37e0 con 0x7f673c0b3ab0 2023-11-24T07:56:07.989 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-24T07:56:07.984+0000 7f6723fff700 1 -- 172.21.15.76:0/1627835111 <== mon.1 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f6730022030 con 0x7f673c0b3ab0 2023-11-24T07:56:07.989 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-24T07:56:07.984+0000 7f6723fff700 1 -- 172.21.15.76:0/1627835111 <== mon.1 v2:172.21.15.76:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1070+0+0 (secure 0 0 0) 0x7f673002e430 con 0x7f673c0b3ab0 2023-11-24T07:56:07.989 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-24T07:56:07.985+0000 7f6741b23700 1 -- 172.21.15.76:0/1627835111 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f673c075480 con 0x7f673c0b3ab0 2023-11-24T07:56:07.990 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-24T07:56:07.987+0000 7f6723fff700 1 -- 172.21.15.76:0/1627835111 <== mon.1 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72898 (secure 0 0 0) 0x7f6730010470 con 0x7f673c0b3ab0 2023-11-24T07:56:08.087 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-24T07:56:08.086+0000 7f6741b23700 1 -- 172.21.15.76:0/1627835111 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "osd new", "uuid": "96fd77fc-f911-46ff-9445-6f0be2daa8f4", "id": 3} v 0) v1 -- 0x7f673c15cec0 con 0x7f673c0b3ab0 2023-11-24T07:56:08.093 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-24T07:56:08.091+0000 7f6723fff700 1 -- 172.21.15.76:0/1627835111 <== mon.1 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "96fd77fc-f911-46ff-9445-6f0be2daa8f4", "id": 3}]=0 v6) v1 ==== 112+0+2 (secure 0 0 0) 0x7f673c15cec0 con 0x7f673c0b3ab0 2023-11-24T07:56:08.093 INFO:teuthology.orchestra.run.smithi076.stdout:3 2023-11-24T07:56:08.094 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-24T07:56:08.092+0000 7f6741b23700 1 -- 172.21.15.76:0/1627835111 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f673c0b3ab0 msgr2=0x7f673c1556e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:56:08.094 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-24T07:56:08.092+0000 7f6741b23700 1 --2- 172.21.15.76:0/1627835111 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f673c0b3ab0 0x7f673c1556e0 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f673c070640 tx=0x7f6730004750 comp rx=0 tx=0).stop 2023-11-24T07:56:08.094 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-24T07:56:08.092+0000 7f6741b23700 1 -- 172.21.15.76:0/1627835111 shutdown_connections 2023-11-24T07:56:08.094 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-24T07:56:08.092+0000 7f6741b23700 1 --2- 172.21.15.76:0/1627835111 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f673c15add0 0x7f673c1528d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:56:08.094 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-24T07:56:08.092+0000 7f6741b23700 1 --2- 172.21.15.76:0/1627835111 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f673c06f540 0x7f673c142ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:56:08.095 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-24T07:56:08.092+0000 7f6741b23700 1 --2- 172.21.15.76:0/1627835111 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f673c0b3ab0 0x7f673c1556e0 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:56:08.095 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-24T07:56:08.092+0000 7f6741b23700 1 -- 172.21.15.76:0/1627835111 >> 172.21.15.76:0/1627835111 conn(0x7f673c0b27b0 msgr2=0x7f673c0b1960 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:56:08.095 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-24T07:56:08.092+0000 7f6741b23700 1 -- 172.21.15.76:0/1627835111 shutdown_connections 2023-11-24T07:56:08.095 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-24T07:56:08.092+0000 7f6741b23700 1 -- 172.21.15.76:0/1627835111 wait complete. 2023-11-24T07:56:08.105 DEBUG:teuthology.orchestra.run.smithi076:> sudo ceph --cluster ceph osd new 7ac29fca-b5b2-4c95-9d69-2f164cbbc708 4 2023-11-24T07:56:08.226 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-24T07:56:08.224+0000 7ff9ab810700 1 Processor -- start 2023-11-24T07:56:08.226 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-24T07:56:08.224+0000 7ff9ab810700 1 -- start start 2023-11-24T07:56:08.227 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-24T07:56:08.225+0000 7ff9ab810700 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7ff9a40b0c40 0x7ff9a40d0220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:56:08.228 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-24T07:56:08.225+0000 7ff9ab810700 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff9a40aeb30 con 0x7ff9a40aefc0 2023-11-24T07:56:08.228 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-24T07:56:08.225+0000 7ff9ab810700 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff9a406d9e0 con 0x7ff9a40ceb00 2023-11-24T07:56:08.228 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-24T07:56:08.225+0000 7ff9ab810700 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff9a406d420 con 0x7ff9a40b0c40 2023-11-24T07:56:08.228 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-24T07:56:08.225+0000 7ff9a8dab700 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7ff9a40b0c40 0x7ff9a40d0220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:56:08.228 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-24T07:56:08.225+0000 7ff9a8dab700 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7ff9a40b0c40 0x7ff9a40d0220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3301/0 says I am v2:172.21.15.76:54426/0 (socket says 172.21.15.76:54426) 2023-11-24T07:56:08.228 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-24T07:56:08.225+0000 7ff9a8dab700 1 -- 172.21.15.76:0/1451274573 learned_addr learned my addr 172.21.15.76:0/1451274573 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-24T07:56:08.228 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-24T07:56:08.225+0000 7ff9a8dab700 1 -- 172.21.15.76:0/1451274573 >> v1:172.21.15.76:6789/0 conn(0x7ff9a40ceb00 legacy=0x7ff9a40cfb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:56:08.228 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-24T07:56:08.225+0000 7ff99bfff700 1 -- 172.21.15.76:0/1451274573 <== mon.1 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3301911434 0 0) 0x7ff9a406d9e0 con 0x7ff9a40ceb00 2023-11-24T07:56:08.228 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-24T07:56:08.225+0000 7ff9a8dab700 1 -- 172.21.15.76:0/1451274573 >> v1:172.21.15.59:6789/0 conn(0x7ff9a40aefc0 legacy=0x7ff9a40af450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:56:08.228 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-24T07:56:08.225+0000 7ff9a8dab700 1 -- 172.21.15.76:0/1451274573 --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff9a406d700 con 0x7ff9a40b0c40 2023-11-24T07:56:08.228 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-24T07:56:08.225+0000 7ff9a8dab700 1 --2- 172.21.15.76:0/1451274573 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7ff9a40b0c40 0x7ff9a40d0220 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7ff994009920 tx=0x7ff994009c30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d13304b21429d887 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:56:08.228 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-24T07:56:08.226+0000 7ff99bfff700 1 -- 172.21.15.76:0/1451274573 <== mon.2 v2:172.21.15.76:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff99400ba60 con 0x7ff9a40b0c40 2023-11-24T07:56:08.229 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-24T07:56:08.226+0000 7ff99bfff700 1 -- 172.21.15.76:0/1451274573 <== mon.2 v2:172.21.15.76:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff99400bbc0 con 0x7ff9a40b0c40 2023-11-24T07:56:08.229 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-24T07:56:08.226+0000 7ff99bfff700 1 -- 172.21.15.76:0/1451274573 <== mon.2 v2:172.21.15.76:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff994009e70 con 0x7ff9a40b0c40 2023-11-24T07:56:08.230 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-24T07:56:08.226+0000 7ff9ab810700 1 -- 172.21.15.76:0/1451274573 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7ff9a40b0c40 msgr2=0x7ff9a40d0220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:56:08.230 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-24T07:56:08.226+0000 7ff9ab810700 1 --2- 172.21.15.76:0/1451274573 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7ff9a40b0c40 0x7ff9a40d0220 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7ff994009920 tx=0x7ff994009c30 comp rx=0 tx=0).stop 2023-11-24T07:56:08.230 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-24T07:56:08.226+0000 7ff9ab810700 1 -- 172.21.15.76:0/1451274573 shutdown_connections 2023-11-24T07:56:08.230 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-24T07:56:08.226+0000 7ff9ab810700 1 --2- 172.21.15.76:0/1451274573 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7ff9a40b0c40 0x7ff9a40d0220 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:56:08.230 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-24T07:56:08.226+0000 7ff9ab810700 1 -- 172.21.15.76:0/1451274573 >> 172.21.15.76:0/1451274573 conn(0x7ff9a40b3720 msgr2=0x7ff9a40b3b30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:56:08.230 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-24T07:56:08.226+0000 7ff9ab810700 1 -- 172.21.15.76:0/1451274573 shutdown_connections 2023-11-24T07:56:08.230 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-24T07:56:08.226+0000 7ff9ab810700 1 -- 172.21.15.76:0/1451274573 wait complete. 2023-11-24T07:56:08.230 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-24T07:56:08.228+0000 7ff9ab810700 1 Processor -- start 2023-11-24T07:56:08.230 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-24T07:56:08.228+0000 7ff9ab810700 1 -- start start 2023-11-24T07:56:08.233 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-24T07:56:08.229+0000 7ff9ab810700 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7ff9a40aefc0 0x7ff9a41f3530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:56:08.233 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-24T07:56:08.229+0000 7ff9ab810700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff9a40b0c40 0x7ff9a415d470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:56:08.233 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-24T07:56:08.229+0000 7ff9ab810700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff9a40ceb00 0x7ff9a415d9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:56:08.233 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-24T07:56:08.229+0000 7ff9ab810700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9a40c61d0 con 0x7ff9a40b0c40 2023-11-24T07:56:08.233 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-24T07:56:08.229+0000 7ff9ab810700 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9a415f6a0 con 0x7ff9a40ceb00 2023-11-24T07:56:08.233 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-24T07:56:08.229+0000 7ff9ab810700 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff9a40c58a0 con 0x7ff9a40aefc0 2023-11-24T07:56:08.233 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-24T07:56:08.229+0000 7ff9a95ac700 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7ff9a40aefc0 0x7ff9a41f3530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:56:08.233 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-24T07:56:08.229+0000 7ff9a9dad700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff9a40ceb00 0x7ff9a415d9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:56:08.233 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-24T07:56:08.229+0000 7ff9a9dad700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff9a40ceb00 0x7ff9a415d9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:34970/0 (socket says 172.21.15.76:34970) 2023-11-24T07:56:08.233 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-24T07:56:08.229+0000 7ff9a9dad700 1 -- 172.21.15.76:0/3322366936 learned_addr learned my addr 172.21.15.76:0/3322366936 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-24T07:56:08.234 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-24T07:56:08.229+0000 7ff9a95ac700 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7ff9a40aefc0 0x7ff9a41f3530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3301/0 says I am v2:172.21.15.76:54430/0 (socket says 172.21.15.76:54430) 2023-11-24T07:56:08.234 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-24T07:56:08.229+0000 7ff9a8dab700 1 --2- 172.21.15.76:0/3322366936 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff9a40b0c40 0x7ff9a415d470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:56:08.234 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-24T07:56:08.229+0000 7ff9a95ac700 1 -- 172.21.15.76:0/3322366936 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff9a40ceb00 msgr2=0x7ff9a415d9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:56:08.234 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-24T07:56:08.229+0000 7ff9a95ac700 1 --2- 172.21.15.76:0/3322366936 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff9a40ceb00 0x7ff9a415d9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:56:08.234 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-24T07:56:08.229+0000 7ff9a95ac700 1 -- 172.21.15.76:0/3322366936 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff9a40b0c40 msgr2=0x7ff9a415d470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:56:08.234 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-24T07:56:08.229+0000 7ff9a95ac700 1 --2- 172.21.15.76:0/3322366936 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff9a40b0c40 0x7ff9a415d470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:56:08.234 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-24T07:56:08.229+0000 7ff9a95ac700 1 -- 172.21.15.76:0/3322366936 --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff994009550 con 0x7ff9a40aefc0 2023-11-24T07:56:08.234 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-24T07:56:08.229+0000 7ff9a9dad700 1 --2- 172.21.15.76:0/3322366936 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff9a40ceb00 0x7ff9a415d9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-24T07:56:08.234 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-24T07:56:08.229+0000 7ff9a8dab700 1 --2- 172.21.15.76:0/3322366936 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff9a40b0c40 0x7ff9a415d470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-24T07:56:08.234 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-24T07:56:08.229+0000 7ff9a95ac700 1 --2- 172.21.15.76:0/3322366936 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7ff9a40aefc0 0x7ff9a41f3530 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7ff9a0000f60 tx=0x7ff9a000c6a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:56:08.234 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-24T07:56:08.230+0000 7ff999ffb700 1 -- 172.21.15.76:0/3322366936 <== mon.2 v2:172.21.15.76:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9a00052d0 con 0x7ff9a40aefc0 2023-11-24T07:56:08.234 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-24T07:56:08.230+0000 7ff999ffb700 1 -- 172.21.15.76:0/3322366936 <== mon.2 v2:172.21.15.76:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff9a0005450 con 0x7ff9a40aefc0 2023-11-24T07:56:08.235 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-24T07:56:08.230+0000 7ff9ab810700 1 -- 172.21.15.76:0/3322366936 --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff9a40c5b80 con 0x7ff9a40aefc0 2023-11-24T07:56:08.235 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-24T07:56:08.230+0000 7ff999ffb700 1 -- 172.21.15.76:0/3322366936 <== mon.2 v2:172.21.15.76:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9a00038c0 con 0x7ff9a40aefc0 2023-11-24T07:56:08.235 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-24T07:56:08.230+0000 7ff9ab810700 1 -- 172.21.15.76:0/3322366936 --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff9a41f3a70 con 0x7ff9a40aefc0 2023-11-24T07:56:08.235 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-24T07:56:08.230+0000 7ff999ffb700 1 -- 172.21.15.76:0/3322366936 <== mon.2 v2:172.21.15.76:3301/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7ff9a0003d20 con 0x7ff9a40aefc0 2023-11-24T07:56:08.235 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-24T07:56:08.230+0000 7ff999ffb700 1 -- 172.21.15.76:0/3322366936 <== mon.2 v2:172.21.15.76:3301/0 5 ==== osd_map(6..6 src has 1..6) v4 ==== 1185+0+0 (secure 0 0 0) 0x7ff9a0012910 con 0x7ff9a40aefc0 2023-11-24T07:56:08.235 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-24T07:56:08.231+0000 7ff9ab810700 1 -- 172.21.15.76:0/3322366936 --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff9a4075480 con 0x7ff9a40aefc0 2023-11-24T07:56:08.235 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-24T07:56:08.233+0000 7ff999ffb700 1 -- 172.21.15.76:0/3322366936 <== mon.2 v2:172.21.15.76:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72898 (secure 0 0 0) 0x7ff9a0017070 con 0x7ff9a40aefc0 2023-11-24T07:56:08.265 INFO:tasks.ceph.mgr.y.smithi059.stderr:2023-11-24T07:56:08.263+0000 7eff6f463200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-11-24T07:56:08.276 INFO:tasks.ceph.mgr.x.smithi076.stderr:2023-11-24T07:56:08.275+0000 7ff220fa5200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-11-24T07:56:08.321 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-24T07:56:08.319+0000 7ff9ab810700 1 -- 172.21.15.76:0/3322366936 --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_command({"prefix": "osd new", "uuid": "7ac29fca-b5b2-4c95-9d69-2f164cbbc708", "id": 4} v 0) v1 -- 0x7ff9a406b340 con 0x7ff9a40aefc0 2023-11-24T07:56:08.326 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-24T07:56:08.324+0000 7ff999ffb700 1 -- 172.21.15.76:0/3322366936 <== mon.2 v2:172.21.15.76:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "7ac29fca-b5b2-4c95-9d69-2f164cbbc708", "id": 4}]=0 v7) v1 ==== 112+0+2 (secure 0 0 0) 0x7ff9a0012ce0 con 0x7ff9a40aefc0 2023-11-24T07:56:08.326 INFO:teuthology.orchestra.run.smithi076.stdout:4 2023-11-24T07:56:08.327 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-24T07:56:08.325+0000 7ff9ab810700 1 -- 172.21.15.76:0/3322366936 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7ff9a40aefc0 msgr2=0x7ff9a41f3530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:56:08.327 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-24T07:56:08.325+0000 7ff9ab810700 1 --2- 172.21.15.76:0/3322366936 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7ff9a40aefc0 0x7ff9a41f3530 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7ff9a0000f60 tx=0x7ff9a000c6a0 comp rx=0 tx=0).stop 2023-11-24T07:56:08.328 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-24T07:56:08.325+0000 7ff9ab810700 1 -- 172.21.15.76:0/3322366936 shutdown_connections 2023-11-24T07:56:08.328 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-24T07:56:08.325+0000 7ff9ab810700 1 --2- 172.21.15.76:0/3322366936 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff9a40ceb00 0x7ff9a415d9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:56:08.328 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-24T07:56:08.325+0000 7ff9ab810700 1 --2- 172.21.15.76:0/3322366936 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7ff9a40aefc0 0x7ff9a41f3530 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:56:08.328 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-24T07:56:08.325+0000 7ff9ab810700 1 --2- 172.21.15.76:0/3322366936 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff9a40b0c40 0x7ff9a415d470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:56:08.328 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-24T07:56:08.325+0000 7ff9ab810700 1 -- 172.21.15.76:0/3322366936 >> 172.21.15.76:0/3322366936 conn(0x7ff9a40b3720 msgr2=0x7ff9a406efc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:56:08.328 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-24T07:56:08.325+0000 7ff9ab810700 1 -- 172.21.15.76:0/3322366936 shutdown_connections 2023-11-24T07:56:08.328 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-24T07:56:08.325+0000 7ff9ab810700 1 -- 172.21.15.76:0/3322366936 wait complete. 2023-11-24T07:56:08.339 DEBUG:teuthology.orchestra.run.smithi076:> sudo ceph --cluster ceph osd new 485bf711-3851-4238-a4f4-51d52e0a9c58 5 2023-11-24T07:56:08.463 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-24T07:56:08.461+0000 7f0cbb4c9700 1 Processor -- start 2023-11-24T07:56:08.463 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-24T07:56:08.461+0000 7f0cbb4c9700 1 -- start start 2023-11-24T07:56:08.465 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-24T07:56:08.462+0000 7f0cbb4c9700 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f0cb415f570 0x7f0cb40d0220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:56:08.465 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-24T07:56:08.462+0000 7f0cbb4c9700 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f0cb40cf040 0x7f0cb40b56d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:56:08.465 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-24T07:56:08.462+0000 7f0cbb4c9700 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0cb40b4c30 con 0x7f0cb40ce4b0 2023-11-24T07:56:08.466 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-24T07:56:08.462+0000 7f0cbb4c9700 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0cb40d0760 con 0x7f0cb40cf040 2023-11-24T07:56:08.466 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-24T07:56:08.462+0000 7f0cbb4c9700 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0cb40cdbc0 con 0x7f0cb415f570 2023-11-24T07:56:08.466 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-24T07:56:08.462+0000 7f0cb9265700 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f0cb415f570 0x7f0cb40d0220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:56:08.466 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-24T07:56:08.462+0000 7f0cb9a66700 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f0cb40cf040 0x7f0cb40b56d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:56:08.466 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-24T07:56:08.462+0000 7f0cb9265700 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f0cb415f570 0x7f0cb40d0220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3301/0 says I am v2:172.21.15.76:54436/0 (socket says 172.21.15.76:54436) 2023-11-24T07:56:08.466 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-24T07:56:08.462+0000 7f0cb9265700 1 -- 172.21.15.76:0/4288717701 learned_addr learned my addr 172.21.15.76:0/4288717701 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-24T07:56:08.466 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-24T07:56:08.462+0000 7f0cb9a66700 1 -- 172.21.15.76:0/4288717701 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f0cb415f570 msgr2=0x7f0cb40d0220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T07:56:08.466 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-24T07:56:08.462+0000 7f0cb9a66700 1 --2- 172.21.15.76:0/4288717701 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f0cb415f570 0x7f0cb40d0220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:56:08.466 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-24T07:56:08.462+0000 7f0cb9a66700 1 -- 172.21.15.76:0/4288717701 >> v1:172.21.15.59:6789/0 conn(0x7f0cb40ce4b0 legacy=0x7f0cb40ce930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:56:08.466 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-24T07:56:08.462+0000 7f0cb9a66700 1 -- 172.21.15.76:0/4288717701 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0cb40d0950 con 0x7f0cb40cf040 2023-11-24T07:56:08.466 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-24T07:56:08.463+0000 7f0cb9a66700 1 --2- 172.21.15.76:0/4288717701 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0cb40cf040 0x7f0cb40b56d0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f0cac009d90 tx=0x7f0cac00b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b0a51e298e2cb6e1 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:56:08.467 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-24T07:56:08.463+0000 7f0cabfff700 1 -- 172.21.15.76:0/4288717701 <== mon.1 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0cac00e070 con 0x7f0cb40cf040 2023-11-24T07:56:08.467 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-24T07:56:08.463+0000 7f0cabfff700 1 -- 172.21.15.76:0/4288717701 <== mon.1 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0cac00bb20 con 0x7f0cb40cf040 2023-11-24T07:56:08.467 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-24T07:56:08.463+0000 7f0cabfff700 1 -- 172.21.15.76:0/4288717701 <== mon.1 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0cac012430 con 0x7f0cb40cf040 2023-11-24T07:56:08.467 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-24T07:56:08.463+0000 7f0cbb4c9700 1 -- 172.21.15.76:0/4288717701 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0cb40cf040 msgr2=0x7f0cb40b56d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:56:08.467 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-24T07:56:08.463+0000 7f0cbb4c9700 1 --2- 172.21.15.76:0/4288717701 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0cb40cf040 0x7f0cb40b56d0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f0cac009d90 tx=0x7f0cac00b330 comp rx=0 tx=0).stop 2023-11-24T07:56:08.467 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-24T07:56:08.463+0000 7f0cbb4c9700 1 -- 172.21.15.76:0/4288717701 shutdown_connections 2023-11-24T07:56:08.467 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-24T07:56:08.463+0000 7f0cbb4c9700 1 --2- 172.21.15.76:0/4288717701 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0cb40cf040 0x7f0cb40b56d0 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:56:08.467 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-24T07:56:08.463+0000 7f0cbb4c9700 1 --2- 172.21.15.76:0/4288717701 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f0cb415f570 0x7f0cb40d0220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:56:08.467 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-24T07:56:08.463+0000 7f0cbb4c9700 1 -- 172.21.15.76:0/4288717701 >> 172.21.15.76:0/4288717701 conn(0x7f0cb40b3800 msgr2=0x7f0cb40b3c10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:56:08.468 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-24T07:56:08.463+0000 7f0cbb4c9700 1 -- 172.21.15.76:0/4288717701 shutdown_connections 2023-11-24T07:56:08.468 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-24T07:56:08.463+0000 7f0cbb4c9700 1 -- 172.21.15.76:0/4288717701 wait complete. 2023-11-24T07:56:08.470 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-24T07:56:08.465+0000 7f0cbb4c9700 1 Processor -- start 2023-11-24T07:56:08.470 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-24T07:56:08.465+0000 7f0cbb4c9700 1 -- start start 2023-11-24T07:56:08.470 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-24T07:56:08.466+0000 7f0cbb4c9700 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f0cb40ce4b0 0x7f0cb41f1420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:56:08.470 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-24T07:56:08.466+0000 7f0cbb4c9700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0cb40cf040 0x7f0cb40c61a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:56:08.471 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-24T07:56:08.466+0000 7f0cbb4c9700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0cb415f570 0x7f0cb40c8e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:56:08.471 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-24T07:56:08.466+0000 7f0cbb4c9700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0cb41f1960 con 0x7f0cb40cf040 2023-11-24T07:56:08.471 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-24T07:56:08.466+0000 7f0cbb4c9700 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0cb40c6bf0 con 0x7f0cb415f570 2023-11-24T07:56:08.471 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-24T07:56:08.466+0000 7f0cbb4c9700 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0cb40c6ed0 con 0x7f0cb40ce4b0 2023-11-24T07:56:08.471 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-24T07:56:08.466+0000 7f0cb9265700 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f0cb40ce4b0 0x7f0cb41f1420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:56:08.471 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-24T07:56:08.466+0000 7f0cb9265700 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f0cb40ce4b0 0x7f0cb41f1420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3301/0 says I am v2:172.21.15.76:54442/0 (socket says 172.21.15.76:54442) 2023-11-24T07:56:08.471 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-24T07:56:08.466+0000 7f0cb9265700 1 -- 172.21.15.76:0/478469240 learned_addr learned my addr 172.21.15.76:0/478469240 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-24T07:56:08.471 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-24T07:56:08.466+0000 7f0cb9a66700 1 --2- 172.21.15.76:0/478469240 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0cb415f570 0x7f0cb40c8e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:56:08.471 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-24T07:56:08.466+0000 7f0cb8a64700 1 --2- 172.21.15.76:0/478469240 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0cb40cf040 0x7f0cb40c61a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:56:08.471 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-24T07:56:08.466+0000 7f0cb9a66700 1 -- 172.21.15.76:0/478469240 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f0cb40ce4b0 msgr2=0x7f0cb41f1420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:56:08.471 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-24T07:56:08.466+0000 7f0cb9a66700 1 --2- 172.21.15.76:0/478469240 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f0cb40ce4b0 0x7f0cb41f1420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:56:08.471 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-24T07:56:08.466+0000 7f0cb9a66700 1 -- 172.21.15.76:0/478469240 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0cb40cf040 msgr2=0x7f0cb40c61a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:56:08.471 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-24T07:56:08.466+0000 7f0cb9a66700 1 --2- 172.21.15.76:0/478469240 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0cb40cf040 0x7f0cb40c61a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:56:08.472 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-24T07:56:08.466+0000 7f0cb9a66700 1 -- 172.21.15.76:0/478469240 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0cac009a70 con 0x7f0cb415f570 2023-11-24T07:56:08.472 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-24T07:56:08.466+0000 7f0cb9265700 1 --2- 172.21.15.76:0/478469240 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f0cb40ce4b0 0x7f0cb41f1420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-24T07:56:08.472 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-24T07:56:08.467+0000 7f0cb9a66700 1 --2- 172.21.15.76:0/478469240 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0cb415f570 0x7f0cb40c8e90 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f0cac016000 tx=0x7f0cac004ad0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:56:08.472 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-24T07:56:08.467+0000 7f0ca9ffb700 1 -- 172.21.15.76:0/478469240 <== mon.1 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0cac012430 con 0x7f0cb415f570 2023-11-24T07:56:08.472 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-24T07:56:08.467+0000 7f0ca9ffb700 1 -- 172.21.15.76:0/478469240 <== mon.1 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0cac020070 con 0x7f0cb415f570 2023-11-24T07:56:08.472 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-24T07:56:08.467+0000 7f0cbb4c9700 1 -- 172.21.15.76:0/478469240 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0cb40c71b0 con 0x7f0cb415f570 2023-11-24T07:56:08.472 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-24T07:56:08.467+0000 7f0ca9ffb700 1 -- 172.21.15.76:0/478469240 <== mon.1 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0cac0098f0 con 0x7f0cb415f570 2023-11-24T07:56:08.472 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-24T07:56:08.467+0000 7f0cbb4c9700 1 -- 172.21.15.76:0/478469240 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0cb40c7770 con 0x7f0cb415f570 2023-11-24T07:56:08.472 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-24T07:56:08.467+0000 7f0ca9ffb700 1 -- 172.21.15.76:0/478469240 <== mon.1 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f0cac004340 con 0x7f0cb415f570 2023-11-24T07:56:08.472 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-24T07:56:08.467+0000 7f0ca9ffb700 1 -- 172.21.15.76:0/478469240 <== mon.1 v2:172.21.15.76:3300/0 5 ==== osd_map(7..7 src has 1..7) v4 ==== 1300+0+0 (secure 0 0 0) 0x7f0cac022070 con 0x7f0cb415f570 2023-11-24T07:56:08.472 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-24T07:56:08.468+0000 7f0cbb4c9700 1 -- 172.21.15.76:0/478469240 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0cb40c7010 con 0x7f0cb415f570 2023-11-24T07:56:08.473 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-24T07:56:08.470+0000 7f0ca9ffb700 1 -- 172.21.15.76:0/478469240 <== mon.1 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72898 (secure 0 0 0) 0x7f0cb40c7010 con 0x7f0cb415f570 2023-11-24T07:56:08.556 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-24T07:56:08.555+0000 7f0cbb4c9700 1 -- 172.21.15.76:0/478469240 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "osd new", "uuid": "485bf711-3851-4238-a4f4-51d52e0a9c58", "id": 5} v 0) v1 -- 0x7f0cb40c7900 con 0x7f0cb415f570 2023-11-24T07:56:08.561 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-24T07:56:08.560+0000 7f0ca9ffb700 1 -- 172.21.15.76:0/478469240 <== mon.1 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "485bf711-3851-4238-a4f4-51d52e0a9c58", "id": 5}]=0 v8) v1 ==== 112+0+2 (secure 0 0 0) 0x7f0cac0048a0 con 0x7f0cb415f570 2023-11-24T07:56:08.562 INFO:teuthology.orchestra.run.smithi076.stdout:5 2023-11-24T07:56:08.563 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-24T07:56:08.561+0000 7f0cbb4c9700 1 -- 172.21.15.76:0/478469240 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0cb415f570 msgr2=0x7f0cb40c8e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:56:08.563 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-24T07:56:08.561+0000 7f0cbb4c9700 1 --2- 172.21.15.76:0/478469240 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0cb415f570 0x7f0cb40c8e90 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f0cac016000 tx=0x7f0cac004ad0 comp rx=0 tx=0).stop 2023-11-24T07:56:08.563 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-24T07:56:08.561+0000 7f0cbb4c9700 1 -- 172.21.15.76:0/478469240 shutdown_connections 2023-11-24T07:56:08.563 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-24T07:56:08.561+0000 7f0cbb4c9700 1 --2- 172.21.15.76:0/478469240 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0cb415f570 0x7f0cb40c8e90 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:56:08.563 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-24T07:56:08.561+0000 7f0cbb4c9700 1 --2- 172.21.15.76:0/478469240 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f0cb40ce4b0 0x7f0cb41f1420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:56:08.563 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-24T07:56:08.561+0000 7f0cbb4c9700 1 --2- 172.21.15.76:0/478469240 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0cb40cf040 0x7f0cb40c61a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:56:08.563 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-24T07:56:08.561+0000 7f0cbb4c9700 1 -- 172.21.15.76:0/478469240 >> 172.21.15.76:0/478469240 conn(0x7f0cb40b3800 msgr2=0x7f0cb40d1440 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:56:08.563 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-24T07:56:08.561+0000 7f0cbb4c9700 1 -- 172.21.15.76:0/478469240 shutdown_connections 2023-11-24T07:56:08.563 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-24T07:56:08.561+0000 7f0cbb4c9700 1 -- 172.21.15.76:0/478469240 wait complete. 2023-11-24T07:56:08.574 DEBUG:teuthology.orchestra.run.smithi076:> sudo ceph --cluster ceph osd new 72b3f2fc-1e4a-4e32-968d-003647a0286f 6 2023-11-24T07:56:08.732 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-24T07:56:08.731+0000 7f8c962de700 1 Processor -- start 2023-11-24T07:56:08.735 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-24T07:56:08.731+0000 7f8c962de700 1 -- start start 2023-11-24T07:56:08.736 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-24T07:56:08.732+0000 7f8c962de700 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f8c9006de20 0x7f8c9006bfe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:56:08.736 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-24T07:56:08.732+0000 7f8c962de700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f8c9006c520 0x7f8c900ce010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:56:08.736 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-24T07:56:08.732+0000 7f8c962de700 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8c9006c970 con 0x7f8c9015a2d0 2023-11-24T07:56:08.736 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-24T07:56:08.732+0000 7f8c962de700 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8c9006d3d0 con 0x7f8c9006c520 2023-11-24T07:56:08.736 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-24T07:56:08.732+0000 7f8c962de700 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8c900b3e00 con 0x7f8c9006de20 2023-11-24T07:56:08.736 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-24T07:56:08.732+0000 7f8c8f7fe700 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f8c9006de20 0x7f8c9006bfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:56:08.736 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-24T07:56:08.732+0000 7f8c8f7fe700 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f8c9006de20 0x7f8c9006bfe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3301/0 says I am v2:172.21.15.76:54450/0 (socket says 172.21.15.76:54450) 2023-11-24T07:56:08.736 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-24T07:56:08.732+0000 7f8c8f7fe700 1 -- 172.21.15.76:0/1247433917 learned_addr learned my addr 172.21.15.76:0/1247433917 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-24T07:56:08.736 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-24T07:56:08.732+0000 7f8c9487b700 1 --2- 172.21.15.76:0/1247433917 >> v2:172.21.15.59:3300/0 conn(0x7f8c9006c520 0x7f8c900ce010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:56:08.736 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-24T07:56:08.733+0000 7f8c8f7fe700 1 -- 172.21.15.76:0/1247433917 >> v2:172.21.15.59:3300/0 conn(0x7f8c9006c520 msgr2=0x7f8c900ce010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T07:56:08.736 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-24T07:56:08.733+0000 7f8c8f7fe700 1 --2- 172.21.15.76:0/1247433917 >> v2:172.21.15.59:3300/0 conn(0x7f8c9006c520 0x7f8c900ce010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:56:08.736 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-24T07:56:08.733+0000 7f8c8f7fe700 1 -- 172.21.15.76:0/1247433917 >> v1:172.21.15.59:6789/0 conn(0x7f8c9015a2d0 legacy=0x7f8c900b3340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:56:08.737 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-24T07:56:08.733+0000 7f8c8f7fe700 1 -- 172.21.15.76:0/1247433917 --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8c900b40e0 con 0x7f8c9006de20 2023-11-24T07:56:08.737 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-24T07:56:08.733+0000 7f8c8f7fe700 1 --2- 172.21.15.76:0/1247433917 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f8c9006de20 0x7f8c9006bfe0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f8c8400bc00 tx=0x7f8c8400bf10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=793149e941139e8e server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:56:08.737 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-24T07:56:08.733+0000 7f8c8effd700 1 -- 172.21.15.76:0/1247433917 <== mon.2 v2:172.21.15.76:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c84009030 con 0x7f8c9006de20 2023-11-24T07:56:08.737 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-24T07:56:08.733+0000 7f8c8effd700 1 -- 172.21.15.76:0/1247433917 <== mon.2 v2:172.21.15.76:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8c8400cbf0 con 0x7f8c9006de20 2023-11-24T07:56:08.737 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-24T07:56:08.733+0000 7f8c8effd700 1 -- 172.21.15.76:0/1247433917 <== mon.2 v2:172.21.15.76:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c8400d450 con 0x7f8c9006de20 2023-11-24T07:56:08.738 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-24T07:56:08.734+0000 7f8c962de700 1 -- 172.21.15.76:0/1247433917 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f8c9006de20 msgr2=0x7f8c9006bfe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:56:08.738 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-24T07:56:08.734+0000 7f8c962de700 1 --2- 172.21.15.76:0/1247433917 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f8c9006de20 0x7f8c9006bfe0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f8c8400bc00 tx=0x7f8c8400bf10 comp rx=0 tx=0).stop 2023-11-24T07:56:08.738 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-24T07:56:08.734+0000 7f8c962de700 1 -- 172.21.15.76:0/1247433917 shutdown_connections 2023-11-24T07:56:08.738 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-24T07:56:08.734+0000 7f8c962de700 1 --2- 172.21.15.76:0/1247433917 >> v2:172.21.15.59:3300/0 conn(0x7f8c9006c520 0x7f8c900ce010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:56:08.738 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-24T07:56:08.734+0000 7f8c962de700 1 --2- 172.21.15.76:0/1247433917 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f8c9006de20 0x7f8c9006bfe0 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:56:08.738 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-24T07:56:08.734+0000 7f8c962de700 1 -- 172.21.15.76:0/1247433917 >> 172.21.15.76:0/1247433917 conn(0x7f8c90071390 msgr2=0x7f8c900717a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:56:08.738 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-24T07:56:08.735+0000 7f8c962de700 1 -- 172.21.15.76:0/1247433917 shutdown_connections 2023-11-24T07:56:08.739 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-24T07:56:08.735+0000 7f8c962de700 1 -- 172.21.15.76:0/1247433917 wait complete. 2023-11-24T07:56:08.739 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-24T07:56:08.736+0000 7f8c962de700 1 Processor -- start 2023-11-24T07:56:08.742 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-24T07:56:08.737+0000 7f8c962de700 1 -- start start 2023-11-24T07:56:08.742 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-24T07:56:08.738+0000 7f8c962de700 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f8c9006c520 0x7f8c90142750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:56:08.742 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-24T07:56:08.738+0000 7f8c962de700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8c9006de20 0x7f8c901458a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:56:08.742 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-24T07:56:08.738+0000 7f8c962de700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8c9015a2d0 0x7f8c901f4840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:56:08.742 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-24T07:56:08.738+0000 7f8c962de700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c90145de0 con 0x7f8c9015a2d0 2023-11-24T07:56:08.742 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-24T07:56:08.738+0000 7f8c962de700 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c90143180 con 0x7f8c9006de20 2023-11-24T07:56:08.742 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-24T07:56:08.738+0000 7f8c962de700 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8c90143460 con 0x7f8c9006c520 2023-11-24T07:56:08.742 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-24T07:56:08.738+0000 7f8c8f7fe700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8c9006de20 0x7f8c901458a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:56:08.742 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-24T07:56:08.738+0000 7f8c8ffff700 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f8c9006c520 0x7f8c90142750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:56:08.742 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-24T07:56:08.738+0000 7f8c8f7fe700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8c9006de20 0x7f8c901458a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:34992/0 (socket says 172.21.15.76:34992) 2023-11-24T07:56:08.742 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-24T07:56:08.738+0000 7f8c9487b700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8c9015a2d0 0x7f8c901f4840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:56:08.743 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-24T07:56:08.738+0000 7f8c8f7fe700 1 -- 172.21.15.76:0/4185291331 learned_addr learned my addr 172.21.15.76:0/4185291331 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-24T07:56:08.743 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-24T07:56:08.738+0000 7f8c8f7fe700 1 -- 172.21.15.76:0/4185291331 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f8c9006c520 msgr2=0x7f8c90142750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:56:08.743 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-24T07:56:08.738+0000 7f8c8f7fe700 1 --2- 172.21.15.76:0/4185291331 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f8c9006c520 0x7f8c90142750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:56:08.743 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-24T07:56:08.738+0000 7f8c8f7fe700 1 -- 172.21.15.76:0/4185291331 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8c9015a2d0 msgr2=0x7f8c901f4840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:56:08.743 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-24T07:56:08.738+0000 7f8c8f7fe700 1 --2- 172.21.15.76:0/4185291331 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8c9015a2d0 0x7f8c901f4840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:56:08.743 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-24T07:56:08.738+0000 7f8c8f7fe700 1 -- 172.21.15.76:0/4185291331 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8c8400b600 con 0x7f8c9006de20 2023-11-24T07:56:08.743 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-24T07:56:08.738+0000 7f8c9487b700 1 --2- 172.21.15.76:0/4185291331 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8c9015a2d0 0x7f8c901f4840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-24T07:56:08.743 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-24T07:56:08.739+0000 7f8c8f7fe700 1 --2- 172.21.15.76:0/4185291331 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8c9006de20 0x7f8c901458a0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f8c84018040 tx=0x7f8c84004750 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:56:08.743 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-24T07:56:08.739+0000 7f8c8cff9700 1 -- 172.21.15.76:0/4185291331 <== mon.1 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c84009030 con 0x7f8c9006de20 2023-11-24T07:56:08.743 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-24T07:56:08.739+0000 7f8c8cff9700 1 -- 172.21.15.76:0/4185291331 <== mon.1 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8c840038a0 con 0x7f8c9006de20 2023-11-24T07:56:08.743 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-24T07:56:08.739+0000 7f8c8cff9700 1 -- 172.21.15.76:0/4185291331 <== mon.1 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c84003c90 con 0x7f8c9006de20 2023-11-24T07:56:08.743 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-24T07:56:08.739+0000 7f8c962de700 1 -- 172.21.15.76:0/4185291331 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8c90143740 con 0x7f8c9006de20 2023-11-24T07:56:08.744 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-24T07:56:08.739+0000 7f8c962de700 1 -- 172.21.15.76:0/4185291331 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8c90143d00 con 0x7f8c9006de20 2023-11-24T07:56:08.744 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-24T07:56:08.740+0000 7f8c962de700 1 -- 172.21.15.76:0/4185291331 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8c90143ec0 con 0x7f8c9006de20 2023-11-24T07:56:08.744 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-24T07:56:08.740+0000 7f8c8cff9700 1 -- 172.21.15.76:0/4185291331 <== mon.1 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f8c8401c040 con 0x7f8c9006de20 2023-11-24T07:56:08.744 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-24T07:56:08.740+0000 7f8c8cff9700 1 -- 172.21.15.76:0/4185291331 <== mon.1 v2:172.21.15.76:3300/0 5 ==== osd_map(8..8 src has 1..8) v4 ==== 1415+0+0 (secure 0 0 0) 0x7f8c8400dd40 con 0x7f8c9006de20 2023-11-24T07:56:08.745 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-24T07:56:08.744+0000 7f8c8cff9700 1 -- 172.21.15.76:0/4185291331 <== mon.1 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72898 (secure 0 0 0) 0x7f8c8402b020 con 0x7f8c9006de20 2023-11-24T07:56:08.833 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-24T07:56:08.831+0000 7f8c962de700 1 -- 172.21.15.76:0/4185291331 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "osd new", "uuid": "72b3f2fc-1e4a-4e32-968d-003647a0286f", "id": 6} v 0) v1 -- 0x7f8c900cc9d0 con 0x7f8c9006de20 2023-11-24T07:56:08.840 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-24T07:56:08.839+0000 7f8c8cff9700 1 -- 172.21.15.76:0/4185291331 <== mon.1 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "72b3f2fc-1e4a-4e32-968d-003647a0286f", "id": 6}]=0 v9) v1 ==== 112+0+2 (secure 0 0 0) 0x7f8c840350b0 con 0x7f8c9006de20 2023-11-24T07:56:08.841 INFO:teuthology.orchestra.run.smithi076.stdout:6 2023-11-24T07:56:08.842 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-24T07:56:08.840+0000 7f8c962de700 1 -- 172.21.15.76:0/4185291331 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8c9006de20 msgr2=0x7f8c901458a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:56:08.842 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-24T07:56:08.840+0000 7f8c962de700 1 --2- 172.21.15.76:0/4185291331 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8c9006de20 0x7f8c901458a0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f8c84018040 tx=0x7f8c84004750 comp rx=0 tx=0).stop 2023-11-24T07:56:08.842 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-24T07:56:08.840+0000 7f8c962de700 1 -- 172.21.15.76:0/4185291331 shutdown_connections 2023-11-24T07:56:08.842 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-24T07:56:08.840+0000 7f8c962de700 1 --2- 172.21.15.76:0/4185291331 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8c9015a2d0 0x7f8c901f4840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:56:08.842 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-24T07:56:08.840+0000 7f8c962de700 1 --2- 172.21.15.76:0/4185291331 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f8c9006c520 0x7f8c90142750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:56:08.842 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-24T07:56:08.840+0000 7f8c962de700 1 --2- 172.21.15.76:0/4185291331 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8c9006de20 0x7f8c901458a0 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:56:08.842 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-24T07:56:08.840+0000 7f8c962de700 1 -- 172.21.15.76:0/4185291331 >> 172.21.15.76:0/4185291331 conn(0x7f8c90071390 msgr2=0x7f8c9006bdc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:56:08.842 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-24T07:56:08.840+0000 7f8c962de700 1 -- 172.21.15.76:0/4185291331 shutdown_connections 2023-11-24T07:56:08.842 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-24T07:56:08.840+0000 7f8c962de700 1 -- 172.21.15.76:0/4185291331 wait complete. 2023-11-24T07:56:08.854 DEBUG:teuthology.orchestra.run.smithi076:> sudo ceph --cluster ceph osd new 1ddceb90-6c5d-4d1a-ad56-99cb877f8c70 7 2023-11-24T07:56:08.937 INFO:tasks.ceph.mgr.x.smithi076.stderr:2023-11-24T07:56:08.935+0000 7ff220fa5200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-11-24T07:56:08.942 INFO:tasks.ceph.mgr.y.smithi059.stderr:2023-11-24T07:56:08.940+0000 7eff6f463200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-11-24T07:56:08.979 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-24T07:56:08.977+0000 7ff882712700 1 Processor -- start 2023-11-24T07:56:08.980 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-24T07:56:08.977+0000 7ff882712700 1 -- start start 2023-11-24T07:56:08.986 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-24T07:56:08.978+0000 7ff882712700 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7ff87c06f540 0x7ff87c0b4880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:56:08.987 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-24T07:56:08.978+0000 7ff882712700 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff87c0ce5b0 con 0x7ff87c15add0 2023-11-24T07:56:08.987 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-24T07:56:08.978+0000 7ff882712700 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff87c072d70 con 0x7ff87c0b3ab0 2023-11-24T07:56:08.987 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-24T07:56:08.978+0000 7ff882712700 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff87c0b4f10 con 0x7ff87c06f540 2023-11-24T07:56:08.987 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-24T07:56:08.978+0000 7ff87b7fe700 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7ff87c06f540 0x7ff87c0b4880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:56:08.987 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-24T07:56:08.978+0000 7ff87b7fe700 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7ff87c06f540 0x7ff87c0b4880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:34998/0 (socket says 172.21.15.76:34998) 2023-11-24T07:56:08.988 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-24T07:56:08.978+0000 7ff87b7fe700 1 -- 172.21.15.76:0/3282827896 learned_addr learned my addr 172.21.15.76:0/3282827896 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-24T07:56:08.988 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-24T07:56:08.978+0000 7ff880caf700 1 --1- 172.21.15.76:0/3282827896 >> v1:172.21.15.76:6789/0 conn(0x7ff87c0b3ab0 0x7ff87c0cea40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.76:53946/0 (socket says 172.21.15.76:53946) 2023-11-24T07:56:08.989 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-24T07:56:08.979+0000 7ff87b7fe700 1 -- 172.21.15.76:0/3282827896 >> v1:172.21.15.76:6789/0 conn(0x7ff87c0b3ab0 legacy=0x7ff87c0cea40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:56:08.989 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-24T07:56:08.979+0000 7ff87b7fe700 1 -- 172.21.15.76:0/3282827896 >> v1:172.21.15.59:6789/0 conn(0x7ff87c15add0 legacy=0x7ff87c06ee30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:56:08.989 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-24T07:56:08.979+0000 7ff87affd700 1 -- 172.21.15.76:0/3282827896 <== mon.1 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2781962836 0 0) 0x7ff87c072d70 con 0x7ff87c0b3ab0 2023-11-24T07:56:08.989 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-24T07:56:08.979+0000 7ff87b7fe700 1 -- 172.21.15.76:0/3282827896 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff87c0b51f0 con 0x7ff87c06f540 2023-11-24T07:56:08.989 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-24T07:56:08.979+0000 7ff87b7fe700 1 --2- 172.21.15.76:0/3282827896 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff87c06f540 0x7ff87c0b4880 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7ff8700098f0 tx=0x7ff870009c00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9c632216ef1c0b9b server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:56:08.990 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-24T07:56:08.979+0000 7ff87affd700 1 -- 172.21.15.76:0/3282827896 <== mon.1 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff870004230 con 0x7ff87c06f540 2023-11-24T07:56:08.990 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-24T07:56:08.979+0000 7ff87affd700 1 -- 172.21.15.76:0/3282827896 <== mon.1 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff870004390 con 0x7ff87c06f540 2023-11-24T07:56:08.990 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-24T07:56:08.979+0000 7ff87affd700 1 -- 172.21.15.76:0/3282827896 <== mon.1 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8700036a0 con 0x7ff87c06f540 2023-11-24T07:56:08.990 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-24T07:56:08.980+0000 7ff882712700 1 -- 172.21.15.76:0/3282827896 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff87c06f540 msgr2=0x7ff87c0b4880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:56:08.991 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-24T07:56:08.980+0000 7ff882712700 1 --2- 172.21.15.76:0/3282827896 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff87c06f540 0x7ff87c0b4880 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7ff8700098f0 tx=0x7ff870009c00 comp rx=0 tx=0).stop 2023-11-24T07:56:08.991 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-24T07:56:08.980+0000 7ff882712700 1 -- 172.21.15.76:0/3282827896 shutdown_connections 2023-11-24T07:56:08.991 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-24T07:56:08.980+0000 7ff882712700 1 --2- 172.21.15.76:0/3282827896 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff87c06f540 0x7ff87c0b4880 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:56:08.991 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-24T07:56:08.980+0000 7ff882712700 1 -- 172.21.15.76:0/3282827896 >> 172.21.15.76:0/3282827896 conn(0x7ff87c0b27b0 msgr2=0x7ff87c0b2b90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:56:08.991 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-24T07:56:08.980+0000 7ff882712700 1 -- 172.21.15.76:0/3282827896 shutdown_connections 2023-11-24T07:56:08.992 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-24T07:56:08.980+0000 7ff882712700 1 -- 172.21.15.76:0/3282827896 wait complete. 2023-11-24T07:56:08.992 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-24T07:56:08.981+0000 7ff882712700 1 Processor -- start 2023-11-24T07:56:08.992 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-24T07:56:08.981+0000 7ff882712700 1 -- start start 2023-11-24T07:56:08.992 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-24T07:56:08.982+0000 7ff882712700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff87c06f540 0x7ff87c146380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:56:08.993 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-24T07:56:08.982+0000 7ff882712700 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7ff87c0b3ab0 0x7ff87c0cb260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:56:08.993 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-24T07:56:08.982+0000 7ff882712700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff87c15add0 0x7ff87c1f5670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:56:08.993 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-24T07:56:08.982+0000 7ff882712700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff87c0cb7a0 con 0x7ff87c15add0 2023-11-24T07:56:08.993 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-24T07:56:08.982+0000 7ff882712700 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff87c1f5bb0 con 0x7ff87c06f540 2023-11-24T07:56:08.993 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-24T07:56:08.982+0000 7ff882712700 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff87c0cba00 con 0x7ff87c0b3ab0 2023-11-24T07:56:08.994 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-24T07:56:08.982+0000 7ff87bfff700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff87c06f540 0x7ff87c146380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:56:08.994 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-24T07:56:08.982+0000 7ff87bfff700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff87c06f540 0x7ff87c146380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:35002/0 (socket says 172.21.15.76:35002) 2023-11-24T07:56:08.994 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-24T07:56:08.982+0000 7ff87bfff700 1 -- 172.21.15.76:0/1051082695 learned_addr learned my addr 172.21.15.76:0/1051082695 (peer_addr_for_me v2:172.21.15.76:0/0) 2023-11-24T07:56:08.994 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-24T07:56:08.982+0000 7ff87b7fe700 1 --2- 172.21.15.76:0/1051082695 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7ff87c0b3ab0 0x7ff87c0cb260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:56:08.994 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-24T07:56:08.982+0000 7ff880caf700 1 --2- 172.21.15.76:0/1051082695 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff87c15add0 0x7ff87c1f5670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:56:08.994 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-24T07:56:08.982+0000 7ff87bfff700 1 -- 172.21.15.76:0/1051082695 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7ff87c0b3ab0 msgr2=0x7ff87c0cb260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:56:08.994 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-24T07:56:08.982+0000 7ff87bfff700 1 --2- 172.21.15.76:0/1051082695 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7ff87c0b3ab0 0x7ff87c0cb260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:56:08.995 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-24T07:56:08.982+0000 7ff87bfff700 1 -- 172.21.15.76:0/1051082695 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff87c15add0 msgr2=0x7ff87c1f5670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:56:08.995 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-24T07:56:08.982+0000 7ff87bfff700 1 --2- 172.21.15.76:0/1051082695 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff87c15add0 0x7ff87c1f5670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:56:08.995 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-24T07:56:08.982+0000 7ff87bfff700 1 -- 172.21.15.76:0/1051082695 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff870009550 con 0x7ff87c06f540 2023-11-24T07:56:08.995 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-24T07:56:08.982+0000 7ff87bfff700 1 --2- 172.21.15.76:0/1051082695 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff87c06f540 0x7ff87c146380 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7ff86c00c650 tx=0x7ff86c00c960 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:56:08.995 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-24T07:56:08.982+0000 7ff878ff9700 1 -- 172.21.15.76:0/1051082695 <== mon.1 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff86c0054e0 con 0x7ff87c06f540 2023-11-24T07:56:08.995 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-24T07:56:08.983+0000 7ff878ff9700 1 -- 172.21.15.76:0/1051082695 <== mon.1 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff86c003680 con 0x7ff87c06f540 2023-11-24T07:56:08.995 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-24T07:56:08.983+0000 7ff882712700 1 -- 172.21.15.76:0/1051082695 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff87c0cbce0 con 0x7ff87c06f540 2023-11-24T07:56:08.996 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-24T07:56:08.983+0000 7ff878ff9700 1 -- 172.21.15.76:0/1051082695 <== mon.1 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff86c003a30 con 0x7ff87c06f540 2023-11-24T07:56:08.996 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-24T07:56:08.983+0000 7ff882712700 1 -- 172.21.15.76:0/1051082695 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff87c0cc2a0 con 0x7ff87c06f540 2023-11-24T07:56:08.996 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-24T07:56:08.983+0000 7ff878ff9700 1 -- 172.21.15.76:0/1051082695 <== mon.1 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7ff86c01c440 con 0x7ff87c06f540 2023-11-24T07:56:08.996 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-24T07:56:08.983+0000 7ff878ff9700 1 -- 172.21.15.76:0/1051082695 <== mon.1 v2:172.21.15.76:3300/0 5 ==== osd_map(9..9 src has 1..9) v4 ==== 1530+0+0 (secure 0 0 0) 0x7ff86c012a60 con 0x7ff87c06f540 2023-11-24T07:56:08.996 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-24T07:56:08.984+0000 7ff882712700 1 -- 172.21.15.76:0/1051082695 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff87c075480 con 0x7ff87c06f540 2023-11-24T07:56:08.997 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-24T07:56:08.986+0000 7ff878ff9700 1 -- 172.21.15.76:0/1051082695 <== mon.1 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72898 (secure 0 0 0) 0x7ff86c017070 con 0x7ff87c06f540 2023-11-24T07:56:09.010 INFO:tasks.ceph.mgr.x.smithi076.stderr:2023-11-24T07:56:09.008+0000 7ff220fa5200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-11-24T07:56:09.014 INFO:tasks.ceph.mgr.y.smithi059.stderr:2023-11-24T07:56:09.012+0000 7eff6f463200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-11-24T07:56:09.073 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-24T07:56:09.071+0000 7ff882712700 1 -- 172.21.15.76:0/1051082695 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "osd new", "uuid": "1ddceb90-6c5d-4d1a-ad56-99cb877f8c70", "id": 7} v 0) v1 -- 0x7ff87c0cee90 con 0x7ff87c06f540 2023-11-24T07:56:09.080 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-24T07:56:09.078+0000 7ff878ff9700 1 -- 172.21.15.76:0/1051082695 <== mon.1 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "1ddceb90-6c5d-4d1a-ad56-99cb877f8c70", "id": 7}]=0 v10) v1 ==== 112+0+2 (secure 0 0 0) 0x7ff86c012400 con 0x7ff87c06f540 2023-11-24T07:56:09.081 INFO:teuthology.orchestra.run.smithi076.stdout:7 2023-11-24T07:56:09.082 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-24T07:56:09.079+0000 7ff882712700 1 -- 172.21.15.76:0/1051082695 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff87c06f540 msgr2=0x7ff87c146380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:56:09.082 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-24T07:56:09.079+0000 7ff882712700 1 --2- 172.21.15.76:0/1051082695 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff87c06f540 0x7ff87c146380 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7ff86c00c650 tx=0x7ff86c00c960 comp rx=0 tx=0).stop 2023-11-24T07:56:09.083 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-24T07:56:09.079+0000 7ff882712700 1 -- 172.21.15.76:0/1051082695 shutdown_connections 2023-11-24T07:56:09.083 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-24T07:56:09.079+0000 7ff882712700 1 --2- 172.21.15.76:0/1051082695 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff87c15add0 0x7ff87c1f5670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:56:09.083 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-24T07:56:09.079+0000 7ff882712700 1 --2- 172.21.15.76:0/1051082695 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff87c06f540 0x7ff87c146380 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:56:09.083 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-24T07:56:09.079+0000 7ff882712700 1 --2- 172.21.15.76:0/1051082695 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7ff87c0b3ab0 0x7ff87c0cb260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:56:09.083 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-24T07:56:09.079+0000 7ff882712700 1 -- 172.21.15.76:0/1051082695 >> 172.21.15.76:0/1051082695 conn(0x7ff87c0b27b0 msgr2=0x7ff87c06daf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:56:09.084 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-24T07:56:09.080+0000 7ff882712700 1 -- 172.21.15.76:0/1051082695 shutdown_connections 2023-11-24T07:56:09.084 INFO:teuthology.orchestra.run.smithi076.stderr:2023-11-24T07:56:09.080+0000 7ff882712700 1 -- 172.21.15.76:0/1051082695 wait complete. 2023-11-24T07:56:09.094 INFO:tasks.ceph.osd.0:Restarting daemon 2023-11-24T07:56:09.094 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 0 2023-11-24T07:56:09.096 INFO:tasks.ceph.osd.0:Started 2023-11-24T07:56:09.097 INFO:tasks.ceph.osd.1:Restarting daemon 2023-11-24T07:56:09.097 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 1 2023-11-24T07:56:09.099 INFO:tasks.ceph.osd.1:Started 2023-11-24T07:56:09.099 INFO:tasks.ceph.osd.2:Restarting daemon 2023-11-24T07:56:09.099 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 2 2023-11-24T07:56:09.101 INFO:tasks.ceph.osd.2:Started 2023-11-24T07:56:09.102 INFO:tasks.ceph.osd.3:Restarting daemon 2023-11-24T07:56:09.102 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 3 2023-11-24T07:56:09.104 INFO:tasks.ceph.osd.3:Started 2023-11-24T07:56:09.105 INFO:tasks.ceph.osd.4:Restarting daemon 2023-11-24T07:56:09.105 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 4 2023-11-24T07:56:09.108 INFO:tasks.ceph.osd.4:Started 2023-11-24T07:56:09.108 INFO:tasks.ceph.osd.5:Restarting daemon 2023-11-24T07:56:09.108 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 5 2023-11-24T07:56:09.110 INFO:tasks.ceph.osd.5:Started 2023-11-24T07:56:09.110 INFO:tasks.ceph.osd.6:Restarting daemon 2023-11-24T07:56:09.110 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 6 2023-11-24T07:56:09.112 INFO:tasks.ceph.osd.6:Started 2023-11-24T07:56:09.113 INFO:tasks.ceph.osd.7:Restarting daemon 2023-11-24T07:56:09.113 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 7 2023-11-24T07:56:09.115 INFO:tasks.ceph.osd.7:Started 2023-11-24T07:56:09.115 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-24T07:56:09.282 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:09.280+0000 7f172a676700 1 Processor -- start 2023-11-24T07:56:09.283 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:09.281+0000 7f172a676700 1 -- start start 2023-11-24T07:56:09.289 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:09.281+0000 7f172a676700 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f1724144610 0x7f172406e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:56:09.289 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:09.281+0000 7f172a676700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f17240b5220 0x7f1724071b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:56:09.290 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:09.281+0000 7f172a676700 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f17240b4610 con 0x7f17240b4da0 2023-11-24T07:56:09.290 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:09.281+0000 7f172a676700 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f17240b56b0 con 0x7f17240b5220 2023-11-24T07:56:09.290 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:09.281+0000 7f172a676700 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f17241449f0 con 0x7f1724144610 2023-11-24T07:56:09.290 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:09.282+0000 7f1728c13700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f17240b5220 0x7f1724071b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:56:09.290 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:09.282+0000 7f17237fe700 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7f17240b4da0 0x7f172415c340 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:52710/0 (socket says 172.21.15.59:52710) 2023-11-24T07:56:09.290 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:09.282+0000 7f17237fe700 1 -- 172.21.15.59:0/3851357157 learned_addr learned my addr 172.21.15.59:0/3851357157 (peer_addr_for_me v1:172.21.15.59:0/0) 2023-11-24T07:56:09.290 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:09.282+0000 7f1723fff700 1 --2- 172.21.15.59:0/3851357157 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f1724144610 0x7f172406e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:56:09.290 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:09.282+0000 7f1728c13700 1 -- 172.21.15.59:0/3851357157 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f1724144610 msgr2=0x7f172406e9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T07:56:09.290 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:09.282+0000 7f1728c13700 1 --2- 172.21.15.59:0/3851357157 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f1724144610 0x7f172406e9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:56:09.290 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:09.282+0000 7f1728c13700 1 -- 172.21.15.59:0/3851357157 >> v1:172.21.15.59:6789/0 conn(0x7f17240b4da0 legacy=0x7f172415c340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:56:09.290 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:09.282+0000 7f1722ffd700 1 -- 172.21.15.59:0/3851357157 <== mon.0 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3416901189 0 0) 0x7f17240b4610 con 0x7f17240b4da0 2023-11-24T07:56:09.291 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:09.282+0000 7f1728c13700 1 -- 172.21.15.59:0/3851357157 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f172406f390 con 0x7f17240b5220 2023-11-24T07:56:09.291 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:09.282+0000 7f1728c13700 1 --2- 172.21.15.59:0/3851357157 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f17240b5220 0x7f1724071b40 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f171c009d90 tx=0x7f171c00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dff767dc57ac8c9 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:56:09.291 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:09.282+0000 7f1722ffd700 1 -- 172.21.15.59:0/3851357157 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f171c00e070 con 0x7f17240b5220 2023-11-24T07:56:09.291 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:09.282+0000 7f1722ffd700 1 -- 172.21.15.59:0/3851357157 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f171c00bb20 con 0x7f17240b5220 2023-11-24T07:56:09.291 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:09.282+0000 7f1722ffd700 1 -- 172.21.15.59:0/3851357157 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f171c012430 con 0x7f17240b5220 2023-11-24T07:56:09.291 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:09.283+0000 7f172a676700 1 -- 172.21.15.59:0/3851357157 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f17240b5220 msgr2=0x7f1724071b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:56:09.291 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:09.283+0000 7f172a676700 1 --2- 172.21.15.59:0/3851357157 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f17240b5220 0x7f1724071b40 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f171c009d90 tx=0x7f171c00b330 comp rx=0 tx=0).stop 2023-11-24T07:56:09.291 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:09.283+0000 7f172a676700 1 -- 172.21.15.59:0/3851357157 shutdown_connections 2023-11-24T07:56:09.291 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:09.283+0000 7f172a676700 1 --2- 172.21.15.59:0/3851357157 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f17240b5220 0x7f1724071b40 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:56:09.291 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:09.283+0000 7f172a676700 1 --2- 172.21.15.59:0/3851357157 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f1724144610 0x7f172406e9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:56:09.291 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:09.283+0000 7f172a676700 1 -- 172.21.15.59:0/3851357157 >> 172.21.15.59:0/3851357157 conn(0x7f172406c0e0 msgr2=0x7f172406c4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:56:09.292 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:09.283+0000 7f172a676700 1 -- 172.21.15.59:0/3851357157 shutdown_connections 2023-11-24T07:56:09.292 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:09.283+0000 7f172a676700 1 -- 172.21.15.59:0/3851357157 wait complete. 2023-11-24T07:56:09.292 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:09.284+0000 7f172a676700 1 Processor -- start 2023-11-24T07:56:09.292 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:09.284+0000 7f172a676700 1 -- start start 2023-11-24T07:56:09.292 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:09.285+0000 7f172a676700 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f17240b4da0 0x7f172413fcc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:56:09.292 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:09.285+0000 7f172a676700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f17240b5220 0x7f17240d1400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:56:09.292 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:09.285+0000 7f172a676700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1724144610 0x7f17240b06b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:56:09.292 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:09.285+0000 7f172a676700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17240b35c0 con 0x7f1724144610 2023-11-24T07:56:09.292 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:09.285+0000 7f172a676700 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17240b0bf0 con 0x7f17240b5220 2023-11-24T07:56:09.292 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:09.285+0000 7f172a676700 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f17240b3000 con 0x7f17240b4da0 2023-11-24T07:56:09.292 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:09.285+0000 7f1728c13700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1724144610 0x7f17240b06b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:56:09.292 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:09.285+0000 7f1728c13700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1724144610 0x7f17240b06b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:49246/0 (socket says 172.21.15.59:49246) 2023-11-24T07:56:09.292 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:09.285+0000 7f1723fff700 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f17240b4da0 0x7f172413fcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:56:09.293 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:09.285+0000 7f17237fe700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f17240b5220 0x7f17240d1400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:56:09.293 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:09.285+0000 7f1728c13700 1 -- 172.21.15.59:0/1062582888 learned_addr learned my addr 172.21.15.59:0/1062582888 (peer_addr_for_me v2:172.21.15.59:0/0) 2023-11-24T07:56:09.293 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:09.285+0000 7f1728c13700 1 -- 172.21.15.59:0/1062582888 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f17240b4da0 msgr2=0x7f172413fcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:56:09.293 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:09.285+0000 7f1728c13700 1 --2- 172.21.15.59:0/1062582888 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f17240b4da0 0x7f172413fcc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:56:09.293 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:09.286+0000 7f1728c13700 1 -- 172.21.15.59:0/1062582888 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f17240b5220 msgr2=0x7f17240d1400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:56:09.293 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:09.286+0000 7f1728c13700 1 --2- 172.21.15.59:0/1062582888 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f17240b5220 0x7f17240d1400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:56:09.293 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:09.286+0000 7f1728c13700 1 -- 172.21.15.59:0/1062582888 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f171c009a70 con 0x7f1724144610 2023-11-24T07:56:09.293 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:09.286+0000 7f1728c13700 1 --2- 172.21.15.59:0/1062582888 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1724144610 0x7f17240b06b0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f17240b3800 tx=0x7f171c004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:56:09.294 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:09.286+0000 7f1720ff9700 1 -- 172.21.15.59:0/1062582888 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f171c012430 con 0x7f1724144610 2023-11-24T07:56:09.294 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:09.286+0000 7f1720ff9700 1 -- 172.21.15.59:0/1062582888 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f171c020070 con 0x7f1724144610 2023-11-24T07:56:09.294 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:09.286+0000 7f172a676700 1 -- 172.21.15.59:0/1062582888 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f17240b32e0 con 0x7f1724144610 2023-11-24T07:56:09.295 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:09.286+0000 7f1720ff9700 1 -- 172.21.15.59:0/1062582888 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f171c004280 con 0x7f1724144610 2023-11-24T07:56:09.295 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:09.286+0000 7f172a676700 1 -- 172.21.15.59:0/1062582888 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f17240b2cf0 con 0x7f1724144610 2023-11-24T07:56:09.295 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:09.287+0000 7f1720ff9700 1 -- 172.21.15.59:0/1062582888 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f171c0046e0 con 0x7f1724144610 2023-11-24T07:56:09.295 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:09.287+0000 7f1720ff9700 1 -- 172.21.15.59:0/1062582888 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(10..10 src has 1..10) v4 ==== 1645+0+0 (secure 0 0 0) 0x7f171c022070 con 0x7f1724144610 2023-11-24T07:56:09.296 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:09.287+0000 7f172a676700 1 -- 172.21.15.59:0/1062582888 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f17240b1040 con 0x7f1724144610 2023-11-24T07:56:09.296 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:09.289+0000 7f1720ff9700 1 -- 172.21.15.59:0/1062582888 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72898 (secure 0 0 0) 0x7f17240b1040 con 0x7f1724144610 2023-11-24T07:56:09.362 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:09.361+0000 7f172a676700 1 -- 172.21.15.59:0/1062582888 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f17240b1420 con 0x7f1724144610 2023-11-24T07:56:09.363 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:09.361+0000 7f1720ff9700 1 -- 172.21.15.59:0/1062582888 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v10) v1 ==== 74+0+7057 (secure 0 0 0) 0x7f17240b1420 con 0x7f1724144610 2023-11-24T07:56:09.363 INFO:teuthology.orchestra.run.smithi059.stdout: 2023-11-24T07:56:09.363 INFO:teuthology.orchestra.run.smithi059.stdout:{"epoch":10,"fsid":"9f0e8741-b8f9-48d5-a2c3-09f45903303d","created":"2023-11-24T07:56:06.552729+0000","modified":"2023-11-24T07:56:09.073250+0000","last_up_change":"0.000000","last_in_change":"2023-11-24T07:56:09.073250+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"6d8f9e9b-1782-4e41-928b-a39f2a8ebc06","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"e8d882aa-52e1-4c04-9e1e-c44ef9204a5f","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"69eebfa0-695b-4b21-90b9-8a3537168a01","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"96fd77fc-f911-46ff-9445-6f0be2daa8f4","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"7ac29fca-b5b2-4c95-9d69-2f164cbbc708","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"485bf711-3851-4238-a4f4-51d52e0a9c58","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"72b3f2fc-1e4a-4e32-968d-003647a0286f","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"1ddceb90-6c5d-4d1a-ad56-99cb877f8c70","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-24T07:56:09.364 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:09.362+0000 7f172a676700 1 -- 172.21.15.59:0/1062582888 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1724144610 msgr2=0x7f17240b06b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:56:09.364 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:09.362+0000 7f172a676700 1 --2- 172.21.15.59:0/1062582888 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1724144610 0x7f17240b06b0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f17240b3800 tx=0x7f171c004ad0 comp rx=0 tx=0).stop 2023-11-24T07:56:09.364 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:09.362+0000 7f172a676700 1 -- 172.21.15.59:0/1062582888 shutdown_connections 2023-11-24T07:56:09.364 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:09.362+0000 7f172a676700 1 --2- 172.21.15.59:0/1062582888 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1724144610 0x7f17240b06b0 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:56:09.364 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:09.362+0000 7f172a676700 1 --2- 172.21.15.59:0/1062582888 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f17240b4da0 0x7f172413fcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:56:09.364 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:09.362+0000 7f172a676700 1 --2- 172.21.15.59:0/1062582888 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f17240b5220 0x7f17240d1400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:56:09.365 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:09.362+0000 7f172a676700 1 -- 172.21.15.59:0/1062582888 >> 172.21.15.59:0/1062582888 conn(0x7f172406c0e0 msgr2=0x7f1724071b40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:56:09.365 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:09.362+0000 7f172a676700 1 -- 172.21.15.59:0/1062582888 shutdown_connections 2023-11-24T07:56:09.365 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:09.363+0000 7f172a676700 1 -- 172.21.15.59:0/1062582888 wait complete. 2023-11-24T07:56:09.376 INFO:tasks.ceph.ceph_manager.ceph:[] 2023-11-24T07:56:09.376 INFO:tasks.ceph:Waiting for OSDs to come up 2023-11-24T07:56:09.718 INFO:tasks.ceph.osd.0.smithi059.stderr:2023-11-24T07:56:09.716+0000 7feb56291640 -1 Falling back to public interface 2023-11-24T07:56:09.720 INFO:tasks.ceph.osd.5.smithi076.stderr:2023-11-24T07:56:09.719+0000 7f1992056640 -1 Falling back to public interface 2023-11-24T07:56:09.721 INFO:tasks.ceph.osd.6.smithi076.stderr:2023-11-24T07:56:09.719+0000 7f77508a6640 -1 Falling back to public interface 2023-11-24T07:56:09.721 INFO:tasks.ceph.osd.6.smithi076.stderr:2023-11-24T07:56:09.719+0000 7f7747c64700 -1 PosixStack listen unable to listen on v2:0.0.0.0:6800/0: (98) Address already in use 2023-11-24T07:56:09.721 INFO:tasks.ceph.osd.5.smithi076.stderr:2023-11-24T07:56:09.719+0000 7f1989414700 -1 PosixStack listen unable to listen on v2:0.0.0.0:6804/0: (98) Address already in use 2023-11-24T07:56:09.721 INFO:tasks.ceph.osd.2.smithi059.stderr:2023-11-24T07:56:09.719+0000 7fbe0b2dc640 -1 Falling back to public interface 2023-11-24T07:56:09.725 INFO:tasks.ceph.osd.3.smithi059.stderr:2023-11-24T07:56:09.723+0000 7ff189efb640 -1 Falling back to public interface 2023-11-24T07:56:09.740 INFO:tasks.ceph.mgr.x.smithi076.stderr:2023-11-24T07:56:09.738+0000 7ff220fa5200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-11-24T07:56:09.740 INFO:tasks.ceph.osd.1.smithi059.stderr:2023-11-24T07:56:09.738+0000 7febe3ae4640 -1 Falling back to public interface 2023-11-24T07:56:09.742 INFO:tasks.ceph.osd.4.smithi076.stderr:2023-11-24T07:56:09.741+0000 7f5387ce4640 -1 Falling back to public interface 2023-11-24T07:56:09.743 INFO:tasks.ceph.osd.7.smithi076.stderr:2023-11-24T07:56:09.741+0000 7fe5a213a640 -1 Falling back to public interface 2023-11-24T07:56:09.743 INFO:tasks.ceph.osd.7.smithi076.stderr:2023-11-24T07:56:09.741+0000 7fe5994f8700 -1 PosixStack listen unable to listen on v2:0.0.0.0:6816/0: (98) Address already in use 2023-11-24T07:56:09.787 INFO:tasks.ceph.mgr.y.smithi059.stderr:2023-11-24T07:56:09.785+0000 7eff6f463200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-11-24T07:56:09.821 INFO:tasks.ceph.mgr.x.smithi076.stderr:2023-11-24T07:56:09.819+0000 7ff220fa5200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-11-24T07:56:09.867 INFO:tasks.ceph.mgr.y.smithi059.stderr:2023-11-24T07:56:09.866+0000 7eff6f463200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-11-24T07:56:09.964 INFO:tasks.ceph.mgr.x.smithi076.stderr:2023-11-24T07:56:09.962+0000 7ff220fa5200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-11-24T07:56:10.015 INFO:tasks.ceph.mgr.y.smithi059.stderr:2023-11-24T07:56:10.013+0000 7eff6f463200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-11-24T07:56:10.182 DEBUG:teuthology.orchestra.run.smithi059:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2023-11-24T07:56:10.203 INFO:tasks.ceph.mgr.x.smithi076.stderr:2023-11-24T07:56:10.201+0000 7ff220fa5200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-11-24T07:56:10.263 INFO:tasks.ceph.mgr.y.smithi059.stderr:2023-11-24T07:56:10.262+0000 7eff6f463200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-11-24T07:56:10.297 INFO:teuthology.misc.health.smithi059.stderr:2023-11-24T07:56:10.296+0000 7fc85f5e2700 1 Processor -- start 2023-11-24T07:56:10.298 INFO:teuthology.misc.health.smithi059.stderr:2023-11-24T07:56:10.296+0000 7fc85f5e2700 1 -- start start 2023-11-24T07:56:10.299 INFO:teuthology.misc.health.smithi059.stderr:2023-11-24T07:56:10.297+0000 7fc85f5e2700 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fc85815e1e0 0x7fc8580b12a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:56:10.299 INFO:teuthology.misc.health.smithi059.stderr:2023-11-24T07:56:10.297+0000 7fc85f5e2700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fc8580b5e40 0x7fc85815f5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:56:10.300 INFO:teuthology.misc.health.smithi059.stderr:2023-11-24T07:56:10.297+0000 7fc85f5e2700 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc85806bb80 con 0x7fc85806b400 2023-11-24T07:56:10.300 INFO:teuthology.misc.health.smithi059.stderr:2023-11-24T07:56:10.297+0000 7fc85f5e2700 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc85806be20 con 0x7fc8580b5e40 2023-11-24T07:56:10.300 INFO:teuthology.misc.health.smithi059.stderr:2023-11-24T07:56:10.297+0000 7fc85f5e2700 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc85806c100 con 0x7fc85815e1e0 2023-11-24T07:56:10.300 INFO:teuthology.misc.health.smithi059.stderr:2023-11-24T07:56:10.297+0000 7fc85db7f700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fc8580b5e40 0x7fc85815f5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:56:10.300 INFO:teuthology.misc.health.smithi059.stderr:2023-11-24T07:56:10.297+0000 7fc85cb7d700 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7fc85806b400 0x7fc8580726a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:52724/0 (socket says 172.21.15.59:52724) 2023-11-24T07:56:10.300 INFO:teuthology.misc.health.smithi059.stderr:2023-11-24T07:56:10.297+0000 7fc85cb7d700 1 -- 172.21.15.59:0/54732785 learned_addr learned my addr 172.21.15.59:0/54732785 (peer_addr_for_me v1:172.21.15.59:0/0) 2023-11-24T07:56:10.300 INFO:teuthology.misc.health.smithi059.stderr:2023-11-24T07:56:10.297+0000 7fc85d37e700 1 --2- 172.21.15.59:0/54732785 >> v2:172.21.15.76:3300/0 conn(0x7fc85815e1e0 0x7fc8580b12a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:56:10.300 INFO:teuthology.misc.health.smithi059.stderr:2023-11-24T07:56:10.297+0000 7fc85db7f700 1 -- 172.21.15.59:0/54732785 >> v2:172.21.15.76:3300/0 conn(0x7fc85815e1e0 msgr2=0x7fc8580b12a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T07:56:10.300 INFO:teuthology.misc.health.smithi059.stderr:2023-11-24T07:56:10.297+0000 7fc85db7f700 1 --2- 172.21.15.59:0/54732785 >> v2:172.21.15.76:3300/0 conn(0x7fc85815e1e0 0x7fc8580b12a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:56:10.300 INFO:teuthology.misc.health.smithi059.stderr:2023-11-24T07:56:10.297+0000 7fc85db7f700 1 -- 172.21.15.59:0/54732785 >> v1:172.21.15.59:6789/0 conn(0x7fc85806b400 legacy=0x7fc8580726a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:56:10.300 INFO:teuthology.misc.health.smithi059.stderr:2023-11-24T07:56:10.297+0000 7fc85db7f700 1 -- 172.21.15.59:0/54732785 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc85806c3e0 con 0x7fc8580b5e40 2023-11-24T07:56:10.300 INFO:teuthology.misc.health.smithi059.stderr:2023-11-24T07:56:10.297+0000 7fc84ffff700 1 -- 172.21.15.59:0/54732785 <== mon.0 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 958792548 0 0) 0x7fc85806bb80 con 0x7fc85806b400 2023-11-24T07:56:10.301 INFO:teuthology.misc.health.smithi059.stderr:2023-11-24T07:56:10.298+0000 7fc85db7f700 1 --2- 172.21.15.59:0/54732785 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc8580b5e40 0x7fc85815f5d0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fc8580737c0 tx=0x7fc85000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3dd1fc9852f5d04b server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:56:10.301 INFO:teuthology.misc.health.smithi059.stderr:2023-11-24T07:56:10.298+0000 7fc84ffff700 1 -- 172.21.15.59:0/54732785 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc85000e070 con 0x7fc8580b5e40 2023-11-24T07:56:10.301 INFO:teuthology.misc.health.smithi059.stderr:2023-11-24T07:56:10.298+0000 7fc84ffff700 1 -- 172.21.15.59:0/54732785 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc85000bb20 con 0x7fc8580b5e40 2023-11-24T07:56:10.301 INFO:teuthology.misc.health.smithi059.stderr:2023-11-24T07:56:10.298+0000 7fc84ffff700 1 -- 172.21.15.59:0/54732785 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc850012430 con 0x7fc8580b5e40 2023-11-24T07:56:10.302 INFO:teuthology.misc.health.smithi059.stderr:2023-11-24T07:56:10.299+0000 7fc85f5e2700 1 -- 172.21.15.59:0/54732785 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc8580b5e40 msgr2=0x7fc85815f5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:56:10.302 INFO:teuthology.misc.health.smithi059.stderr:2023-11-24T07:56:10.299+0000 7fc85f5e2700 1 --2- 172.21.15.59:0/54732785 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc8580b5e40 0x7fc85815f5d0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fc8580737c0 tx=0x7fc85000b330 comp rx=0 tx=0).stop 2023-11-24T07:56:10.302 INFO:teuthology.misc.health.smithi059.stderr:2023-11-24T07:56:10.299+0000 7fc85f5e2700 1 -- 172.21.15.59:0/54732785 shutdown_connections 2023-11-24T07:56:10.302 INFO:teuthology.misc.health.smithi059.stderr:2023-11-24T07:56:10.299+0000 7fc85f5e2700 1 --2- 172.21.15.59:0/54732785 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc8580b5e40 0x7fc85815f5d0 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:56:10.302 INFO:teuthology.misc.health.smithi059.stderr:2023-11-24T07:56:10.299+0000 7fc85f5e2700 1 --2- 172.21.15.59:0/54732785 >> v2:172.21.15.76:3300/0 conn(0x7fc85815e1e0 0x7fc8580b12a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:56:10.302 INFO:teuthology.misc.health.smithi059.stderr:2023-11-24T07:56:10.299+0000 7fc85f5e2700 1 -- 172.21.15.59:0/54732785 >> 172.21.15.59:0/54732785 conn(0x7fc8580b5070 msgr2=0x7fc8580b5730 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:56:10.302 INFO:teuthology.misc.health.smithi059.stderr:2023-11-24T07:56:10.299+0000 7fc85f5e2700 1 -- 172.21.15.59:0/54732785 shutdown_connections 2023-11-24T07:56:10.303 INFO:teuthology.misc.health.smithi059.stderr:2023-11-24T07:56:10.299+0000 7fc85f5e2700 1 -- 172.21.15.59:0/54732785 wait complete. 2023-11-24T07:56:10.303 INFO:teuthology.misc.health.smithi059.stderr:2023-11-24T07:56:10.300+0000 7fc85f5e2700 1 Processor -- start 2023-11-24T07:56:10.303 INFO:teuthology.misc.health.smithi059.stderr:2023-11-24T07:56:10.301+0000 7fc85f5e2700 1 -- start start 2023-11-24T07:56:10.305 INFO:teuthology.misc.health.smithi059.stderr:2023-11-24T07:56:10.301+0000 7fc85f5e2700 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fc85806b400 0x7fc8580998d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:56:10.305 INFO:teuthology.misc.health.smithi059.stderr:2023-11-24T07:56:10.301+0000 7fc85f5e2700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc8580b5e40 0x7fc858158850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:56:10.305 INFO:teuthology.misc.health.smithi059.stderr:2023-11-24T07:56:10.301+0000 7fc85f5e2700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc85815e1e0 0x7fc85815c090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:56:10.305 INFO:teuthology.misc.health.smithi059.stderr:2023-11-24T07:56:10.301+0000 7fc85f5e2700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8581566c0 con 0x7fc8580b5e40 2023-11-24T07:56:10.305 INFO:teuthology.misc.health.smithi059.stderr:2023-11-24T07:56:10.301+0000 7fc85f5e2700 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc858155db0 con 0x7fc85815e1e0 2023-11-24T07:56:10.306 INFO:teuthology.misc.health.smithi059.stderr:2023-11-24T07:56:10.301+0000 7fc85f5e2700 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc858159180 con 0x7fc85806b400 2023-11-24T07:56:10.306 INFO:teuthology.misc.health.smithi059.stderr:2023-11-24T07:56:10.301+0000 7fc85cb7d700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc8580b5e40 0x7fc858158850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:56:10.306 INFO:teuthology.misc.health.smithi059.stderr:2023-11-24T07:56:10.301+0000 7fc85cb7d700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc8580b5e40 0x7fc858158850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:49256/0 (socket says 172.21.15.59:49256) 2023-11-24T07:56:10.306 INFO:teuthology.misc.health.smithi059.stderr:2023-11-24T07:56:10.301+0000 7fc85cb7d700 1 -- 172.21.15.59:0/273926527 learned_addr learned my addr 172.21.15.59:0/273926527 (peer_addr_for_me v2:172.21.15.59:0/0) 2023-11-24T07:56:10.306 INFO:teuthology.misc.health.smithi059.stderr:2023-11-24T07:56:10.302+0000 7fc85d37e700 1 --2- 172.21.15.59:0/273926527 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fc85806b400 0x7fc8580998d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:56:10.306 INFO:teuthology.misc.health.smithi059.stderr:2023-11-24T07:56:10.302+0000 7fc85db7f700 1 --2- 172.21.15.59:0/273926527 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc85815e1e0 0x7fc85815c090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:56:10.306 INFO:teuthology.misc.health.smithi059.stderr:2023-11-24T07:56:10.302+0000 7fc85cb7d700 1 -- 172.21.15.59:0/273926527 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fc85806b400 msgr2=0x7fc8580998d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:56:10.306 INFO:teuthology.misc.health.smithi059.stderr:2023-11-24T07:56:10.302+0000 7fc85cb7d700 1 --2- 172.21.15.59:0/273926527 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fc85806b400 0x7fc8580998d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:56:10.306 INFO:teuthology.misc.health.smithi059.stderr:2023-11-24T07:56:10.302+0000 7fc85cb7d700 1 -- 172.21.15.59:0/273926527 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc85815e1e0 msgr2=0x7fc85815c090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:56:10.307 INFO:teuthology.misc.health.smithi059.stderr:2023-11-24T07:56:10.302+0000 7fc85cb7d700 1 --2- 172.21.15.59:0/273926527 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc85815e1e0 0x7fc85815c090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:56:10.307 INFO:teuthology.misc.health.smithi059.stderr:2023-11-24T07:56:10.302+0000 7fc85cb7d700 1 -- 172.21.15.59:0/273926527 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc850009a70 con 0x7fc8580b5e40 2023-11-24T07:56:10.307 INFO:teuthology.misc.health.smithi059.stderr:2023-11-24T07:56:10.302+0000 7fc85cb7d700 1 --2- 172.21.15.59:0/273926527 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc8580b5e40 0x7fc858158850 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7fc84800c540 tx=0x7fc84800c850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:56:10.307 INFO:teuthology.misc.health.smithi059.stderr:2023-11-24T07:56:10.302+0000 7fc84dffb700 1 -- 172.21.15.59:0/273926527 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc848004c80 con 0x7fc8580b5e40 2023-11-24T07:56:10.307 INFO:teuthology.misc.health.smithi059.stderr:2023-11-24T07:56:10.302+0000 7fc84dffb700 1 -- 172.21.15.59:0/273926527 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc848004de0 con 0x7fc8580b5e40 2023-11-24T07:56:10.307 INFO:teuthology.misc.health.smithi059.stderr:2023-11-24T07:56:10.302+0000 7fc85f5e2700 1 -- 172.21.15.59:0/273926527 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc858156000 con 0x7fc8580b5e40 2023-11-24T07:56:10.307 INFO:teuthology.misc.health.smithi059.stderr:2023-11-24T07:56:10.302+0000 7fc84dffb700 1 -- 172.21.15.59:0/273926527 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc848007310 con 0x7fc8580b5e40 2023-11-24T07:56:10.307 INFO:teuthology.misc.health.smithi059.stderr:2023-11-24T07:56:10.302+0000 7fc84dffb700 1 -- 172.21.15.59:0/273926527 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7fc8480133c0 con 0x7fc8580b5e40 2023-11-24T07:56:10.307 INFO:teuthology.misc.health.smithi059.stderr:2023-11-24T07:56:10.302+0000 7fc85f5e2700 1 -- 172.21.15.59:0/273926527 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc858099e10 con 0x7fc8580b5e40 2023-11-24T07:56:10.307 INFO:teuthology.misc.health.smithi059.stderr:2023-11-24T07:56:10.303+0000 7fc84dffb700 1 -- 172.21.15.59:0/273926527 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(10..10 src has 1..10) v4 ==== 1645+0+0 (secure 0 0 0) 0x7fc8480123b0 con 0x7fc8580b5e40 2023-11-24T07:56:10.308 INFO:teuthology.misc.health.smithi059.stderr:2023-11-24T07:56:10.303+0000 7fc85f5e2700 1 -- 172.21.15.59:0/273926527 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc858156800 con 0x7fc8580b5e40 2023-11-24T07:56:10.308 INFO:teuthology.misc.health.smithi059.stderr:2023-11-24T07:56:10.305+0000 7fc84dffb700 1 -- 172.21.15.59:0/273926527 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72898 (secure 0 0 0) 0x7fc858156800 con 0x7fc8580b5e40 2023-11-24T07:56:10.379 INFO:teuthology.misc.health.smithi059.stderr:2023-11-24T07:56:10.378+0000 7fc85f5e2700 1 -- 172.21.15.59:0/273926527 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fc858075480 con 0x7fc8580b5e40 2023-11-24T07:56:10.380 INFO:teuthology.misc.health.smithi059.stderr:2023-11-24T07:56:10.378+0000 7fc84dffb700 1 -- 172.21.15.59:0/273926527 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v10) v1 ==== 74+0+7057 (secure 0 0 0) 0x7fc848013a60 con 0x7fc8580b5e40 2023-11-24T07:56:10.380 INFO:teuthology.misc.health.smithi059.stdout: 2023-11-24T07:56:10.380 INFO:teuthology.misc.health.smithi059.stdout:{"epoch":10,"fsid":"9f0e8741-b8f9-48d5-a2c3-09f45903303d","created":"2023-11-24T07:56:06.552729+0000","modified":"2023-11-24T07:56:09.073250+0000","last_up_change":"0.000000","last_in_change":"2023-11-24T07:56:09.073250+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"6d8f9e9b-1782-4e41-928b-a39f2a8ebc06","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"e8d882aa-52e1-4c04-9e1e-c44ef9204a5f","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"69eebfa0-695b-4b21-90b9-8a3537168a01","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"96fd77fc-f911-46ff-9445-6f0be2daa8f4","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"7ac29fca-b5b2-4c95-9d69-2f164cbbc708","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"485bf711-3851-4238-a4f4-51d52e0a9c58","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"72b3f2fc-1e4a-4e32-968d-003647a0286f","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"1ddceb90-6c5d-4d1a-ad56-99cb877f8c70","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-24T07:56:10.382 INFO:teuthology.misc.health.smithi059.stderr:2023-11-24T07:56:10.379+0000 7fc85f5e2700 1 -- 172.21.15.59:0/273926527 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc8580b5e40 msgr2=0x7fc858158850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:56:10.382 INFO:teuthology.misc.health.smithi059.stderr:2023-11-24T07:56:10.379+0000 7fc85f5e2700 1 --2- 172.21.15.59:0/273926527 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc8580b5e40 0x7fc858158850 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7fc84800c540 tx=0x7fc84800c850 comp rx=0 tx=0).stop 2023-11-24T07:56:10.382 INFO:teuthology.misc.health.smithi059.stderr:2023-11-24T07:56:10.379+0000 7fc85f5e2700 1 -- 172.21.15.59:0/273926527 shutdown_connections 2023-11-24T07:56:10.382 INFO:teuthology.misc.health.smithi059.stderr:2023-11-24T07:56:10.379+0000 7fc85f5e2700 1 --2- 172.21.15.59:0/273926527 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc85815e1e0 0x7fc85815c090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:56:10.382 INFO:teuthology.misc.health.smithi059.stderr:2023-11-24T07:56:10.379+0000 7fc85f5e2700 1 --2- 172.21.15.59:0/273926527 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fc85806b400 0x7fc8580998d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:56:10.382 INFO:teuthology.misc.health.smithi059.stderr:2023-11-24T07:56:10.379+0000 7fc85f5e2700 1 --2- 172.21.15.59:0/273926527 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc8580b5e40 0x7fc858158850 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:56:10.382 INFO:teuthology.misc.health.smithi059.stderr:2023-11-24T07:56:10.379+0000 7fc85f5e2700 1 -- 172.21.15.59:0/273926527 >> 172.21.15.59:0/273926527 conn(0x7fc8580b5070 msgr2=0x7fc858160500 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:56:10.382 INFO:teuthology.misc.health.smithi059.stderr:2023-11-24T07:56:10.379+0000 7fc85f5e2700 1 -- 172.21.15.59:0/273926527 shutdown_connections 2023-11-24T07:56:10.382 INFO:teuthology.misc.health.smithi059.stderr:2023-11-24T07:56:10.380+0000 7fc85f5e2700 1 -- 172.21.15.59:0/273926527 wait complete. 2023-11-24T07:56:10.393 DEBUG:teuthology.misc:0 of 8 OSDs are up 2023-11-24T07:56:10.415 INFO:tasks.ceph.mgr.x.smithi076.stderr:2023-11-24T07:56:10.413+0000 7ff220fa5200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-11-24T07:56:10.486 INFO:tasks.ceph.mgr.y.smithi059.stderr:2023-11-24T07:56:10.484+0000 7eff6f463200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-11-24T07:56:10.550 INFO:tasks.ceph.mgr.x.smithi076.stderr:2023-11-24T07:56:10.548+0000 7ff220fa5200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-11-24T07:56:10.626 INFO:tasks.ceph.mgr.y.smithi059.stderr:2023-11-24T07:56:10.624+0000 7eff6f463200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-11-24T07:56:10.773 INFO:tasks.ceph.mgr.x.smithi076.stderr:2023-11-24T07:56:10.771+0000 7ff220fa5200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-11-24T07:56:10.856 INFO:tasks.ceph.mgr.y.smithi059.stderr:2023-11-24T07:56:10.854+0000 7eff6f463200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-11-24T07:56:10.932 INFO:tasks.ceph.mgr.x.smithi076.stderr:2023-11-24T07:56:10.930+0000 7ff220fa5200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-11-24T07:56:11.017 INFO:tasks.ceph.mgr.y.smithi059.stderr:2023-11-24T07:56:11.015+0000 7eff6f463200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-11-24T07:56:11.047 INFO:tasks.ceph.mgr.x.smithi076.stderr:2023-11-24T07:56:11.046+0000 7ff220fa5200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-11-24T07:56:11.137 INFO:tasks.ceph.mgr.y.smithi059.stderr:2023-11-24T07:56:11.136+0000 7eff6f463200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-11-24T07:56:11.371 INFO:tasks.ceph.mgr.x.smithi076.stderr:2023-11-24T07:56:11.369+0000 7ff220fa5200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-11-24T07:56:11.473 INFO:tasks.ceph.mgr.y.smithi059.stderr:2023-11-24T07:56:11.471+0000 7eff6f463200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-11-24T07:56:11.473 INFO:tasks.ceph.mgr.x.smithi076.stderr:2023-11-24T07:56:11.472+0000 7ff220fa5200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-11-24T07:56:11.575 INFO:tasks.ceph.mgr.y.smithi059.stderr:2023-11-24T07:56:11.574+0000 7eff6f463200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-11-24T07:56:11.662 INFO:tasks.ceph.mgr.x.smithi076.stderr:2023-11-24T07:56:11.661+0000 7ff220fa5200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-11-24T07:56:11.770 INFO:tasks.ceph.mgr.y.smithi059.stderr:2023-11-24T07:56:11.768+0000 7eff6f463200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-11-24T07:56:12.277 INFO:tasks.ceph.mgr.x.smithi076.stderr:2023-11-24T07:56:12.275+0000 7ff220fa5200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-11-24T07:56:12.402 INFO:tasks.ceph.mgr.y.smithi059.stderr:2023-11-24T07:56:12.400+0000 7eff6f463200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-11-24T07:56:13.257 INFO:tasks.ceph.osd.1.smithi059.stderr:2023-11-24T07:56:13.256+0000 7febe3ae4640 -1 osd.1 0 log_to_monitors true 2023-11-24T07:56:13.263 INFO:tasks.ceph.osd.4.smithi076.stderr:2023-11-24T07:56:13.263+0000 7f5387ce4640 -1 osd.4 0 log_to_monitors true 2023-11-24T07:56:13.272 INFO:tasks.ceph.osd.6.smithi076.stderr:2023-11-24T07:56:13.272+0000 7f77508a6640 -1 osd.6 0 log_to_monitors true 2023-11-24T07:56:13.273 INFO:tasks.ceph.osd.5.smithi076.stderr:2023-11-24T07:56:13.272+0000 7f1992056640 -1 osd.5 0 log_to_monitors true 2023-11-24T07:56:13.277 INFO:tasks.ceph.osd.2.smithi059.stderr:2023-11-24T07:56:13.276+0000 7fbe0b2dc640 -1 osd.2 0 log_to_monitors true 2023-11-24T07:56:13.286 INFO:tasks.ceph.osd.0.smithi059.stderr:2023-11-24T07:56:13.285+0000 7feb56291640 -1 osd.0 0 log_to_monitors true 2023-11-24T07:56:13.296 INFO:tasks.ceph.osd.3.smithi059.stderr:2023-11-24T07:56:13.295+0000 7ff189efb640 -1 osd.3 0 log_to_monitors true 2023-11-24T07:56:13.307 INFO:tasks.ceph.osd.7.smithi076.stderr:2023-11-24T07:56:13.307+0000 7fe5a213a640 -1 osd.7 0 log_to_monitors true 2023-11-24T07:56:13.313 INFO:tasks.ceph.mgr.x.smithi076.stderr:2023-11-24T07:56:13.312+0000 7ff1d8f33700 -1 mgr.server handle_report got status from non-daemon mon.b 2023-11-24T07:56:13.313 INFO:tasks.ceph.mgr.x.smithi076.stderr:2023-11-24T07:56:13.313+0000 7ff1d8f33700 -1 mgr.server handle_report got status from non-daemon mon.c 2023-11-24T07:56:16.361 INFO:tasks.ceph.osd.6.smithi076.stderr:2023-11-24T07:56:16.360+0000 7f774445d700 -1 osd.6 0 waiting for initial osdmap 2023-11-24T07:56:16.370 INFO:tasks.ceph.osd.2.smithi059.stderr:2023-11-24T07:56:16.369+0000 7fbdfee93700 -1 osd.2 0 waiting for initial osdmap 2023-11-24T07:56:16.374 INFO:tasks.ceph.osd.6.smithi076.stderr:2023-11-24T07:56:16.374+0000 7f773c83d700 -1 osd.6 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-11-24T07:56:16.376 INFO:tasks.ceph.osd.5.smithi076.stderr:2023-11-24T07:56:16.375+0000 7f1985c0d700 -1 osd.5 0 waiting for initial osdmap 2023-11-24T07:56:16.386 INFO:tasks.ceph.osd.2.smithi059.stderr:2023-11-24T07:56:16.385+0000 7fbdf7273700 -1 osd.2 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-11-24T07:56:16.391 INFO:tasks.ceph.osd.5.smithi076.stderr:2023-11-24T07:56:16.391+0000 7f197dfed700 -1 osd.5 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-11-24T07:56:16.418 INFO:tasks.ceph.osd.1.smithi059.stderr:2023-11-24T07:56:16.416+0000 7febd769b700 -1 osd.1 0 waiting for initial osdmap 2023-11-24T07:56:16.421 INFO:tasks.ceph.osd.7.smithi076.stderr:2023-11-24T07:56:16.420+0000 7fe595cf1700 -1 osd.7 0 waiting for initial osdmap 2023-11-24T07:56:16.427 INFO:tasks.ceph.osd.4.smithi076.stderr:2023-11-24T07:56:16.426+0000 7f537b89b700 -1 osd.4 0 waiting for initial osdmap 2023-11-24T07:56:16.427 INFO:tasks.ceph.osd.0.smithi059.stderr:2023-11-24T07:56:16.426+0000 7feb49e48700 -1 osd.0 0 waiting for initial osdmap 2023-11-24T07:56:16.430 INFO:tasks.ceph.osd.1.smithi059.stderr:2023-11-24T07:56:16.429+0000 7febcfa7b700 -1 osd.1 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-11-24T07:56:16.433 INFO:tasks.ceph.osd.7.smithi076.stderr:2023-11-24T07:56:16.433+0000 7fe58e0d1700 -1 osd.7 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-11-24T07:56:16.438 INFO:tasks.ceph.osd.0.smithi059.stderr:2023-11-24T07:56:16.437+0000 7feb42228700 -1 osd.0 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-11-24T07:56:16.439 INFO:tasks.ceph.osd.4.smithi076.stderr:2023-11-24T07:56:16.438+0000 7f5373c7b700 -1 osd.4 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-11-24T07:56:16.495 INFO:tasks.ceph.osd.3.smithi059.stderr:2023-11-24T07:56:16.493+0000 7ff17dab2700 -1 osd.3 0 waiting for initial osdmap 2023-11-24T07:56:16.512 INFO:tasks.ceph.osd.3.smithi059.stderr:2023-11-24T07:56:16.512+0000 7ff175e92700 -1 osd.3 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-11-24T07:56:17.200 DEBUG:teuthology.orchestra.run.smithi059:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2023-11-24T07:56:17.328 INFO:teuthology.misc.health.smithi059.stderr:2023-11-24T07:56:17.326+0000 7f4183bcc700 1 Processor -- start 2023-11-24T07:56:17.328 INFO:teuthology.misc.health.smithi059.stderr:2023-11-24T07:56:17.327+0000 7f4183bcc700 1 -- start start 2023-11-24T07:56:17.330 INFO:teuthology.misc.health.smithi059.stderr:2023-11-24T07:56:17.327+0000 7f4183bcc700 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f417c143ae0 0x7f417c0b51b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:56:17.330 INFO:teuthology.misc.health.smithi059.stderr:2023-11-24T07:56:17.327+0000 7f4183bcc700 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f417c0722c0 0x7f417c06e510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:56:17.330 INFO:teuthology.misc.health.smithi059.stderr:2023-11-24T07:56:17.327+0000 7f4183bcc700 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f417c143ec0 con 0x7f417c1440f0 2023-11-24T07:56:17.330 INFO:teuthology.misc.health.smithi059.stderr:2023-11-24T07:56:17.327+0000 7f4183bcc700 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f417c0b40a0 con 0x7f417c143ae0 2023-11-24T07:56:17.330 INFO:teuthology.misc.health.smithi059.stderr:2023-11-24T07:56:17.327+0000 7f4183bcc700 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f417c0b3ae0 con 0x7f417c0722c0 2023-11-24T07:56:17.331 INFO:teuthology.misc.health.smithi059.stderr:2023-11-24T07:56:17.328+0000 7f4182169700 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f417c0722c0 0x7f417c06e510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:56:17.331 INFO:teuthology.misc.health.smithi059.stderr:2023-11-24T07:56:17.328+0000 7f4182169700 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f417c0722c0 0x7f417c06e510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3301/0 says I am v2:172.21.15.59:48756/0 (socket says 172.21.15.59:48756) 2023-11-24T07:56:17.331 INFO:teuthology.misc.health.smithi059.stderr:2023-11-24T07:56:17.328+0000 7f4182169700 1 -- 172.21.15.59:0/489166122 learned_addr learned my addr 172.21.15.59:0/489166122 (peer_addr_for_me v2:172.21.15.59:0/0) 2023-11-24T07:56:17.331 INFO:teuthology.misc.health.smithi059.stderr:2023-11-24T07:56:17.328+0000 7f4181968700 1 --2- 172.21.15.59:0/489166122 >> v2:172.21.15.76:3300/0 conn(0x7f417c143ae0 0x7f417c0b51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:56:17.331 INFO:teuthology.misc.health.smithi059.stderr:2023-11-24T07:56:17.328+0000 7f4182169700 1 -- 172.21.15.59:0/489166122 >> v2:172.21.15.76:3300/0 conn(0x7f417c143ae0 msgr2=0x7f417c0b51b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T07:56:17.331 INFO:teuthology.misc.health.smithi059.stderr:2023-11-24T07:56:17.328+0000 7f4182169700 1 --2- 172.21.15.59:0/489166122 >> v2:172.21.15.76:3300/0 conn(0x7f417c143ae0 0x7f417c0b51b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:56:17.331 INFO:teuthology.misc.health.smithi059.stderr:2023-11-24T07:56:17.328+0000 7f4182169700 1 -- 172.21.15.59:0/489166122 >> v1:172.21.15.76:6789/0 conn(0x7f417c1440f0 legacy=0x7f417c071bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:56:17.331 INFO:teuthology.misc.health.smithi059.stderr:2023-11-24T07:56:17.328+0000 7f4182169700 1 -- 172.21.15.59:0/489166122 --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f417c0b3dc0 con 0x7f417c0722c0 2023-11-24T07:56:17.331 INFO:teuthology.misc.health.smithi059.stderr:2023-11-24T07:56:17.328+0000 7f4180966700 1 -- 172.21.15.59:0/489166122 <== mon.1 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4092436944 0 0) 0x7f417c143ec0 con 0x7f417c1440f0 2023-11-24T07:56:17.331 INFO:teuthology.misc.health.smithi059.stderr:2023-11-24T07:56:17.328+0000 7f4182169700 1 --2- 172.21.15.59:0/489166122 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f417c0722c0 0x7f417c06e510 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f4174009e50 tx=0x7f417400b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a4c6d1865eb2d8f1 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:56:17.331 INFO:teuthology.misc.health.smithi059.stderr:2023-11-24T07:56:17.328+0000 7f4180966700 1 -- 172.21.15.59:0/489166122 <== mon.2 v2:172.21.15.76:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f417400e070 con 0x7f417c0722c0 2023-11-24T07:56:17.332 INFO:teuthology.misc.health.smithi059.stderr:2023-11-24T07:56:17.328+0000 7f4180966700 1 -- 172.21.15.59:0/489166122 <== mon.2 v2:172.21.15.76:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f417400b910 con 0x7f417c0722c0 2023-11-24T07:56:17.332 INFO:teuthology.misc.health.smithi059.stderr:2023-11-24T07:56:17.328+0000 7f4180966700 1 -- 172.21.15.59:0/489166122 <== mon.2 v2:172.21.15.76:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f417400bcc0 con 0x7f417c0722c0 2023-11-24T07:56:17.334 INFO:teuthology.misc.health.smithi059.stderr:2023-11-24T07:56:17.329+0000 7f4183bcc700 1 -- 172.21.15.59:0/489166122 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f417c0722c0 msgr2=0x7f417c06e510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:56:17.335 INFO:teuthology.misc.health.smithi059.stderr:2023-11-24T07:56:17.329+0000 7f4183bcc700 1 --2- 172.21.15.59:0/489166122 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f417c0722c0 0x7f417c06e510 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f4174009e50 tx=0x7f417400b330 comp rx=0 tx=0).stop 2023-11-24T07:56:17.335 INFO:teuthology.misc.health.smithi059.stderr:2023-11-24T07:56:17.329+0000 7f4183bcc700 1 -- 172.21.15.59:0/489166122 shutdown_connections 2023-11-24T07:56:17.335 INFO:teuthology.misc.health.smithi059.stderr:2023-11-24T07:56:17.329+0000 7f4183bcc700 1 --2- 172.21.15.59:0/489166122 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f417c0722c0 0x7f417c06e510 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:56:17.335 INFO:teuthology.misc.health.smithi059.stderr:2023-11-24T07:56:17.329+0000 7f4183bcc700 1 --2- 172.21.15.59:0/489166122 >> v2:172.21.15.76:3300/0 conn(0x7f417c143ae0 0x7f417c0b51b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:56:17.335 INFO:teuthology.misc.health.smithi059.stderr:2023-11-24T07:56:17.329+0000 7f4183bcc700 1 -- 172.21.15.59:0/489166122 >> 172.21.15.59:0/489166122 conn(0x7f417c0b2740 msgr2=0x7f417c0b2b20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:56:17.336 INFO:teuthology.misc.health.smithi059.stderr:2023-11-24T07:56:17.329+0000 7f4183bcc700 1 -- 172.21.15.59:0/489166122 shutdown_connections 2023-11-24T07:56:17.336 INFO:teuthology.misc.health.smithi059.stderr:2023-11-24T07:56:17.329+0000 7f4183bcc700 1 -- 172.21.15.59:0/489166122 wait complete. 2023-11-24T07:56:17.336 INFO:teuthology.misc.health.smithi059.stderr:2023-11-24T07:56:17.330+0000 7f4183bcc700 1 Processor -- start 2023-11-24T07:56:17.336 INFO:teuthology.misc.health.smithi059.stderr:2023-11-24T07:56:17.330+0000 7f4183bcc700 1 -- start start 2023-11-24T07:56:17.336 INFO:teuthology.misc.health.smithi059.stderr:2023-11-24T07:56:17.331+0000 7f4183bcc700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f417c0722c0 0x7f417c13ee60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:56:17.336 INFO:teuthology.misc.health.smithi059.stderr:2023-11-24T07:56:17.331+0000 7f4183bcc700 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f417c143ae0 0x7f417c0b41e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:56:17.336 INFO:teuthology.misc.health.smithi059.stderr:2023-11-24T07:56:17.331+0000 7f4183bcc700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f417c1440f0 0x7f417c0b4720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:56:17.336 INFO:teuthology.misc.health.smithi059.stderr:2023-11-24T07:56:17.331+0000 7f4183bcc700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f417c15e170 con 0x7f417c0722c0 2023-11-24T07:56:17.336 INFO:teuthology.misc.health.smithi059.stderr:2023-11-24T07:56:17.331+0000 7f4183bcc700 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f417c13f3a0 con 0x7f417c1440f0 2023-11-24T07:56:17.336 INFO:teuthology.misc.health.smithi059.stderr:2023-11-24T07:56:17.331+0000 7f4183bcc700 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f417c15d560 con 0x7f417c143ae0 2023-11-24T07:56:17.336 INFO:teuthology.misc.health.smithi059.stderr:2023-11-24T07:56:17.331+0000 7f4181968700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f417c0722c0 0x7f417c13ee60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:56:17.337 INFO:teuthology.misc.health.smithi059.stderr:2023-11-24T07:56:17.331+0000 7f4181968700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f417c0722c0 0x7f417c13ee60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:49318/0 (socket says 172.21.15.59:49318) 2023-11-24T07:56:17.337 INFO:teuthology.misc.health.smithi059.stderr:2023-11-24T07:56:17.331+0000 7f4181968700 1 -- 172.21.15.59:0/2980770996 learned_addr learned my addr 172.21.15.59:0/2980770996 (peer_addr_for_me v2:172.21.15.59:0/0) 2023-11-24T07:56:17.337 INFO:teuthology.misc.health.smithi059.stderr:2023-11-24T07:56:17.331+0000 7f4182169700 1 --2- 172.21.15.59:0/2980770996 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f417c1440f0 0x7f417c0b4720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:56:17.337 INFO:teuthology.misc.health.smithi059.stderr:2023-11-24T07:56:17.331+0000 7f4181167700 1 --2- 172.21.15.59:0/2980770996 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f417c143ae0 0x7f417c0b41e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:56:17.337 INFO:teuthology.misc.health.smithi059.stderr:2023-11-24T07:56:17.332+0000 7f4181968700 1 -- 172.21.15.59:0/2980770996 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f417c143ae0 msgr2=0x7f417c0b41e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:56:17.337 INFO:teuthology.misc.health.smithi059.stderr:2023-11-24T07:56:17.332+0000 7f4181968700 1 --2- 172.21.15.59:0/2980770996 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f417c143ae0 0x7f417c0b41e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:56:17.337 INFO:teuthology.misc.health.smithi059.stderr:2023-11-24T07:56:17.332+0000 7f4181968700 1 -- 172.21.15.59:0/2980770996 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f417c1440f0 msgr2=0x7f417c0b4720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:56:17.337 INFO:teuthology.misc.health.smithi059.stderr:2023-11-24T07:56:17.332+0000 7f4181968700 1 --2- 172.21.15.59:0/2980770996 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f417c1440f0 0x7f417c0b4720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:56:17.337 INFO:teuthology.misc.health.smithi059.stderr:2023-11-24T07:56:17.332+0000 7f4181968700 1 -- 172.21.15.59:0/2980770996 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4174009b30 con 0x7f417c0722c0 2023-11-24T07:56:17.337 INFO:teuthology.misc.health.smithi059.stderr:2023-11-24T07:56:17.332+0000 7f4181968700 1 --2- 172.21.15.59:0/2980770996 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f417c0722c0 0x7f417c13ee60 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f417800e9b0 tx=0x7f417800ecc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:56:17.337 INFO:teuthology.misc.health.smithi059.stderr:2023-11-24T07:56:17.332+0000 7f416e7fc700 1 -- 172.21.15.59:0/2980770996 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4178009710 con 0x7f417c0722c0 2023-11-24T07:56:17.338 INFO:teuthology.misc.health.smithi059.stderr:2023-11-24T07:56:17.332+0000 7f416e7fc700 1 -- 172.21.15.59:0/2980770996 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f417800ee50 con 0x7f417c0722c0 2023-11-24T07:56:17.338 INFO:teuthology.misc.health.smithi059.stderr:2023-11-24T07:56:17.332+0000 7f416e7fc700 1 -- 172.21.15.59:0/2980770996 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f417800ca50 con 0x7f417c0722c0 2023-11-24T07:56:17.338 INFO:teuthology.misc.health.smithi059.stderr:2023-11-24T07:56:17.332+0000 7f4183bcc700 1 -- 172.21.15.59:0/2980770996 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f417c15d840 con 0x7f417c0722c0 2023-11-24T07:56:17.338 INFO:teuthology.misc.health.smithi059.stderr:2023-11-24T07:56:17.332+0000 7f4183bcc700 1 -- 172.21.15.59:0/2980770996 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f417c15de00 con 0x7f417c0722c0 2023-11-24T07:56:17.338 INFO:teuthology.misc.health.smithi059.stderr:2023-11-24T07:56:17.333+0000 7f4183bcc700 1 -- 172.21.15.59:0/2980770996 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f417c075480 con 0x7f417c0722c0 2023-11-24T07:56:17.339 INFO:teuthology.misc.health.smithi059.stderr:2023-11-24T07:56:17.334+0000 7f416e7fc700 1 -- 172.21.15.59:0/2980770996 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 4) v1 ==== 89175+0+0 (secure 0 0 0) 0x7f417800cbb0 con 0x7f417c0722c0 2023-11-24T07:56:17.339 INFO:teuthology.misc.health.smithi059.stderr:2023-11-24T07:56:17.334+0000 7f416e7fc700 1 --2- 172.21.15.59:0/2980770996 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f416006ae40 0x7f416006d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:56:17.339 INFO:teuthology.misc.health.smithi059.stderr:2023-11-24T07:56:17.334+0000 7f416e7fc700 1 -- 172.21.15.59:0/2980770996 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(12..12 src has 1..12) v4 ==== 2231+0+0 (secure 0 0 0) 0x7f4178088760 con 0x7f417c0722c0 2023-11-24T07:56:17.339 INFO:teuthology.misc.health.smithi059.stderr:2023-11-24T07:56:17.334+0000 7f4181167700 1 --2- 172.21.15.59:0/2980770996 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f416006ae40 0x7f416006d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:56:17.339 INFO:teuthology.misc.health.smithi059.stderr:2023-11-24T07:56:17.335+0000 7f4181167700 1 --2- 172.21.15.59:0/2980770996 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f416006ae40 0x7f416006d300 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f417c143750 tx=0x7f4170009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:56:17.343 INFO:teuthology.misc.health.smithi059.stderr:2023-11-24T07:56:17.339+0000 7f416e7fc700 1 -- 172.21.15.59:0/2980770996 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f4178054680 con 0x7f417c0722c0 2023-11-24T07:56:17.550 INFO:teuthology.misc.health.smithi059.stderr:2023-11-24T07:56:17.548+0000 7f4183bcc700 1 -- 172.21.15.59:0/2980770996 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f417c0b5d20 con 0x7f417c0722c0 2023-11-24T07:56:17.550 INFO:teuthology.misc.health.smithi059.stderr:2023-11-24T07:56:17.549+0000 7f416e7fc700 1 -- 172.21.15.59:0/2980770996 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v13) v1 ==== 74+0+10836 (secure 0 0 0) 0x7f4178021090 con 0x7f417c0722c0 2023-11-24T07:56:17.551 INFO:teuthology.misc.health.smithi059.stdout: 2023-11-24T07:56:17.551 INFO:teuthology.misc.health.smithi059.stdout:{"epoch":13,"fsid":"9f0e8741-b8f9-48d5-a2c3-09f45903303d","created":"2023-11-24T07:56:06.552729+0000","modified":"2023-11-24T07:56:17.372352+0000","last_up_change":"2023-11-24T07:56:17.372352+0000","last_in_change":"2023-11-24T07:56:09.073250+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"6d8f9e9b-1782-4e41-928b-a39f2a8ebc06","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6800","nonce":3373878772},{"type":"v1","addr":"172.21.15.59:6801","nonce":3373878772}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6802","nonce":3373878772},{"type":"v1","addr":"172.21.15.59:6803","nonce":3373878772}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6806","nonce":3373878772},{"type":"v1","addr":"172.21.15.59:6807","nonce":3373878772}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6804","nonce":3373878772},{"type":"v1","addr":"172.21.15.59:6805","nonce":3373878772}]},"public_addr":"172.21.15.59:6801/3373878772","cluster_addr":"172.21.15.59:6803/3373878772","heartbeat_back_addr":"172.21.15.59:6807/3373878772","heartbeat_front_addr":"172.21.15.59:6805/3373878772","state":["exists","up"]},{"osd":1,"uuid":"e8d882aa-52e1-4c04-9e1e-c44ef9204a5f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6824","nonce":648258563},{"type":"v1","addr":"172.21.15.59:6825","nonce":648258563}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6826","nonce":648258563},{"type":"v1","addr":"172.21.15.59:6827","nonce":648258563}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6830","nonce":648258563},{"type":"v1","addr":"172.21.15.59:6831","nonce":648258563}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6828","nonce":648258563},{"type":"v1","addr":"172.21.15.59:6829","nonce":648258563}]},"public_addr":"172.21.15.59:6825/648258563","cluster_addr":"172.21.15.59:6827/648258563","heartbeat_back_addr":"172.21.15.59:6831/648258563","heartbeat_front_addr":"172.21.15.59:6829/648258563","state":["exists","up"]},{"osd":2,"uuid":"69eebfa0-695b-4b21-90b9-8a3537168a01","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6808","nonce":1840430654},{"type":"v1","addr":"172.21.15.59:6809","nonce":1840430654}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6810","nonce":1840430654},{"type":"v1","addr":"172.21.15.59:6811","nonce":1840430654}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6814","nonce":1840430654},{"type":"v1","addr":"172.21.15.59:6815","nonce":1840430654}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6812","nonce":1840430654},{"type":"v1","addr":"172.21.15.59:6813","nonce":1840430654}]},"public_addr":"172.21.15.59:6809/1840430654","cluster_addr":"172.21.15.59:6811/1840430654","heartbeat_back_addr":"172.21.15.59:6815/1840430654","heartbeat_front_addr":"172.21.15.59:6813/1840430654","state":["exists","up"]},{"osd":3,"uuid":"96fd77fc-f911-46ff-9445-6f0be2daa8f4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6816","nonce":2504025850},{"type":"v1","addr":"172.21.15.59:6817","nonce":2504025850}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6818","nonce":2504025850},{"type":"v1","addr":"172.21.15.59:6819","nonce":2504025850}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6822","nonce":2504025850},{"type":"v1","addr":"172.21.15.59:6823","nonce":2504025850}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6820","nonce":2504025850},{"type":"v1","addr":"172.21.15.59:6821","nonce":2504025850}]},"public_addr":"172.21.15.59:6817/2504025850","cluster_addr":"172.21.15.59:6819/2504025850","heartbeat_back_addr":"172.21.15.59:6823/2504025850","heartbeat_front_addr":"172.21.15.59:6821/2504025850","state":["exists","up"]},{"osd":4,"uuid":"7ac29fca-b5b2-4c95-9d69-2f164cbbc708","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6816","nonce":2382463431},{"type":"v1","addr":"172.21.15.76:6818","nonce":2382463431}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6820","nonce":2382463431},{"type":"v1","addr":"172.21.15.76:6822","nonce":2382463431}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6828","nonce":2382463431},{"type":"v1","addr":"172.21.15.76:6830","nonce":2382463431}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6824","nonce":2382463431},{"type":"v1","addr":"172.21.15.76:6826","nonce":2382463431}]},"public_addr":"172.21.15.76:6818/2382463431","cluster_addr":"172.21.15.76:6822/2382463431","heartbeat_back_addr":"172.21.15.76:6830/2382463431","heartbeat_front_addr":"172.21.15.76:6826/2382463431","state":["exists","up"]},{"osd":5,"uuid":"485bf711-3851-4238-a4f4-51d52e0a9c58","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6800","nonce":329334308},{"type":"v1","addr":"172.21.15.76:6802","nonce":329334308}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6805","nonce":329334308},{"type":"v1","addr":"172.21.15.76:6807","nonce":329334308}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6814","nonce":329334308},{"type":"v1","addr":"172.21.15.76:6815","nonce":329334308}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6810","nonce":329334308},{"type":"v1","addr":"172.21.15.76:6812","nonce":329334308}]},"public_addr":"172.21.15.76:6802/329334308","cluster_addr":"172.21.15.76:6807/329334308","heartbeat_back_addr":"172.21.15.76:6815/329334308","heartbeat_front_addr":"172.21.15.76:6812/329334308","state":["exists","up"]},{"osd":6,"uuid":"72b3f2fc-1e4a-4e32-968d-003647a0286f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6801","nonce":3946579410},{"type":"v1","addr":"172.21.15.76:6803","nonce":3946579410}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6804","nonce":3946579410},{"type":"v1","addr":"172.21.15.76:6806","nonce":3946579410}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6811","nonce":3946579410},{"type":"v1","addr":"172.21.15.76:6813","nonce":3946579410}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6808","nonce":3946579410},{"type":"v1","addr":"172.21.15.76:6809","nonce":3946579410}]},"public_addr":"172.21.15.76:6803/3946579410","cluster_addr":"172.21.15.76:6806/3946579410","heartbeat_back_addr":"172.21.15.76:6813/3946579410","heartbeat_front_addr":"172.21.15.76:6809/3946579410","state":["exists","up"]},{"osd":7,"uuid":"1ddceb90-6c5d-4d1a-ad56-99cb877f8c70","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6817","nonce":841761538},{"type":"v1","addr":"172.21.15.76:6819","nonce":841761538}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6821","nonce":841761538},{"type":"v1","addr":"172.21.15.76:6823","nonce":841761538}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6829","nonce":841761538},{"type":"v1","addr":"172.21.15.76:6831","nonce":841761538}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6825","nonce":841761538},{"type":"v1","addr":"172.21.15.76:6827","nonce":841761538}]},"public_addr":"172.21.15.76:6819/841761538","cluster_addr":"172.21.15.76:6823/841761538","heartbeat_back_addr":"172.21.15.76:6831/841761538","heartbeat_front_addr":"172.21.15.76:6827/841761538","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-24T07:56:17.553 INFO:teuthology.misc.health.smithi059.stderr:2023-11-24T07:56:17.551+0000 7f4183bcc700 1 -- 172.21.15.59:0/2980770996 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f416006ae40 msgr2=0x7f416006d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:56:17.553 INFO:teuthology.misc.health.smithi059.stderr:2023-11-24T07:56:17.551+0000 7f4183bcc700 1 --2- 172.21.15.59:0/2980770996 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f416006ae40 0x7f416006d300 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f417c143750 tx=0x7f4170009040 comp rx=0 tx=0).stop 2023-11-24T07:56:17.553 INFO:teuthology.misc.health.smithi059.stderr:2023-11-24T07:56:17.551+0000 7f4183bcc700 1 -- 172.21.15.59:0/2980770996 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f417c0722c0 msgr2=0x7f417c13ee60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:56:17.553 INFO:teuthology.misc.health.smithi059.stderr:2023-11-24T07:56:17.551+0000 7f4183bcc700 1 --2- 172.21.15.59:0/2980770996 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f417c0722c0 0x7f417c13ee60 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f417800e9b0 tx=0x7f417800ecc0 comp rx=0 tx=0).stop 2023-11-24T07:56:17.554 INFO:teuthology.misc.health.smithi059.stderr:2023-11-24T07:56:17.551+0000 7f4183bcc700 1 -- 172.21.15.59:0/2980770996 shutdown_connections 2023-11-24T07:56:17.554 INFO:teuthology.misc.health.smithi059.stderr:2023-11-24T07:56:17.551+0000 7f4183bcc700 1 --2- 172.21.15.59:0/2980770996 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f416006ae40 0x7f416006d300 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:56:17.554 INFO:teuthology.misc.health.smithi059.stderr:2023-11-24T07:56:17.551+0000 7f4183bcc700 1 --2- 172.21.15.59:0/2980770996 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f417c1440f0 0x7f417c0b4720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:56:17.554 INFO:teuthology.misc.health.smithi059.stderr:2023-11-24T07:56:17.551+0000 7f4183bcc700 1 --2- 172.21.15.59:0/2980770996 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f417c0722c0 0x7f417c13ee60 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:56:17.554 INFO:teuthology.misc.health.smithi059.stderr:2023-11-24T07:56:17.551+0000 7f4183bcc700 1 --2- 172.21.15.59:0/2980770996 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f417c143ae0 0x7f417c0b41e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:56:17.554 INFO:teuthology.misc.health.smithi059.stderr:2023-11-24T07:56:17.551+0000 7f4183bcc700 1 -- 172.21.15.59:0/2980770996 >> 172.21.15.59:0/2980770996 conn(0x7f417c0b2740 msgr2=0x7f417c071bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:56:17.554 INFO:teuthology.misc.health.smithi059.stderr:2023-11-24T07:56:17.551+0000 7f4183bcc700 1 -- 172.21.15.59:0/2980770996 shutdown_connections 2023-11-24T07:56:17.554 INFO:teuthology.misc.health.smithi059.stderr:2023-11-24T07:56:17.551+0000 7f4183bcc700 1 -- 172.21.15.59:0/2980770996 wait complete. 2023-11-24T07:56:17.564 DEBUG:teuthology.misc:8 of 8 OSDs are up 2023-11-24T07:56:17.564 INFO:tasks.ceph:Starting mds daemons in cluster ceph... 2023-11-24T07:56:17.565 INFO:tasks.ceph.mds.a:Restarting daemon 2023-11-24T07:56:17.565 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2023-11-24T07:56:17.567 INFO:tasks.ceph.mds.a:Started 2023-11-24T07:56:17.567 INFO:tasks.ceph.mds.c:Restarting daemon 2023-11-24T07:56:17.567 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2023-11-24T07:56:17.569 INFO:tasks.ceph.mds.c:Started 2023-11-24T07:56:17.569 INFO:tasks.ceph.mds.b:Restarting daemon 2023-11-24T07:56:17.569 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2023-11-24T07:56:17.572 INFO:tasks.ceph.mds.b:Started 2023-11-24T07:56:17.572 INFO:tasks.ceph.mds.d:Restarting daemon 2023-11-24T07:56:17.572 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2023-11-24T07:56:17.574 INFO:tasks.ceph.mds.d:Started 2023-11-24T07:56:17.574 INFO:tasks.ceph:Setting up CephFS filesystem(s)... 2023-11-24T07:56:17.575 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-24T07:56:17.652 INFO:tasks.ceph.mds.c.smithi059.stdout:starting mds.c at 2023-11-24T07:56:17.652 INFO:tasks.ceph.mds.a.smithi059.stdout:starting mds.a at 2023-11-24T07:56:17.665 INFO:tasks.ceph.mds.b.smithi076.stdout:starting mds.b at 2023-11-24T07:56:17.665 INFO:tasks.ceph.mds.d.smithi076.stdout:starting mds.d at 2023-11-24T07:56:17.707 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:17.706+0000 7fd247654700 1 Processor -- start 2023-11-24T07:56:17.708 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:17.706+0000 7fd247654700 1 -- start start 2023-11-24T07:56:17.710 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:17.707+0000 7fd247654700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fd24015a440 0x7fd2400cffa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:56:17.710 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:17.707+0000 7fd247654700 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fd24015eb80 0x7fd2400d04e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:56:17.710 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:17.707+0000 7fd247654700 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd2400b05f0 con 0x7fd24015a820 2023-11-24T07:56:17.710 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:17.707+0000 7fd247654700 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd24015f080 con 0x7fd24015a440 2023-11-24T07:56:17.710 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:17.707+0000 7fd247654700 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd2400afd50 con 0x7fd24015eb80 2023-11-24T07:56:17.710 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:17.707+0000 7fd2453f0700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fd24015a440 0x7fd2400cffa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:56:17.710 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:17.707+0000 7fd2453f0700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fd24015a440 0x7fd2400cffa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:49498/0 (socket says 172.21.15.59:49498) 2023-11-24T07:56:17.710 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:17.707+0000 7fd2453f0700 1 -- 172.21.15.59:0/2271202984 learned_addr learned my addr 172.21.15.59:0/2271202984 (peer_addr_for_me v2:172.21.15.59:0/0) 2023-11-24T07:56:17.711 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:17.707+0000 7fd245bf1700 1 --2- 172.21.15.59:0/2271202984 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fd24015eb80 0x7fd2400d04e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:56:17.711 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:17.707+0000 7fd237fff700 1 -- 172.21.15.59:0/2271202984 <== mon.1 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 768439845 0 0) 0x7fd2400b05f0 con 0x7fd24015a820 2023-11-24T07:56:17.711 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:17.707+0000 7fd2453f0700 1 -- 172.21.15.59:0/2271202984 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fd24015eb80 msgr2=0x7fd2400d04e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T07:56:17.711 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:17.707+0000 7fd2453f0700 1 --2- 172.21.15.59:0/2271202984 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fd24015eb80 0x7fd2400d04e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:56:17.711 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:17.707+0000 7fd2453f0700 1 -- 172.21.15.59:0/2271202984 >> v1:172.21.15.76:6789/0 conn(0x7fd24015a820 legacy=0x7fd2400726a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:56:17.711 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:17.707+0000 7fd2453f0700 1 -- 172.21.15.59:0/2271202984 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd2400b0030 con 0x7fd24015a440 2023-11-24T07:56:17.711 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:17.707+0000 7fd245bf1700 1 --2- 172.21.15.59:0/2271202984 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fd24015eb80 0x7fd2400d04e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-24T07:56:17.711 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:17.707+0000 7fd2453f0700 1 --2- 172.21.15.59:0/2271202984 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd24015a440 0x7fd2400cffa0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fd23c0098f0 tx=0x7fd23c009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a6a0413eabb88fac server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:56:17.711 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:17.707+0000 7fd237fff700 1 -- 172.21.15.59:0/2271202984 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd23c011830 con 0x7fd24015a440 2023-11-24T07:56:17.711 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:17.708+0000 7fd237fff700 1 -- 172.21.15.59:0/2271202984 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd23c011990 con 0x7fd24015a440 2023-11-24T07:56:17.711 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:17.708+0000 7fd237fff700 1 -- 172.21.15.59:0/2271202984 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd23c011d20 con 0x7fd24015a440 2023-11-24T07:56:17.712 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:17.708+0000 7fd247654700 1 -- 172.21.15.59:0/2271202984 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd24015a440 msgr2=0x7fd2400cffa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:56:17.712 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:17.708+0000 7fd247654700 1 --2- 172.21.15.59:0/2271202984 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd24015a440 0x7fd2400cffa0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fd23c0098f0 tx=0x7fd23c009c00 comp rx=0 tx=0).stop 2023-11-24T07:56:17.712 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:17.708+0000 7fd247654700 1 -- 172.21.15.59:0/2271202984 shutdown_connections 2023-11-24T07:56:17.712 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:17.708+0000 7fd247654700 1 --2- 172.21.15.59:0/2271202984 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fd24015eb80 0x7fd2400d04e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:56:17.712 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:17.708+0000 7fd247654700 1 --2- 172.21.15.59:0/2271202984 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd24015a440 0x7fd2400cffa0 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:56:17.712 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:17.708+0000 7fd247654700 1 -- 172.21.15.59:0/2271202984 >> 172.21.15.59:0/2271202984 conn(0x7fd2400b5070 msgr2=0x7fd2400b5730 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:56:17.712 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:17.708+0000 7fd247654700 1 -- 172.21.15.59:0/2271202984 shutdown_connections 2023-11-24T07:56:17.712 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:17.708+0000 7fd247654700 1 -- 172.21.15.59:0/2271202984 wait complete. 2023-11-24T07:56:17.715 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:17.709+0000 7fd247654700 1 Processor -- start 2023-11-24T07:56:17.715 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:17.709+0000 7fd247654700 1 -- start start 2023-11-24T07:56:17.715 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:17.710+0000 7fd247654700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd24015a440 0x7fd24014d6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:56:17.715 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:17.710+0000 7fd247654700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd24015eb80 0x7fd2400cf090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:56:17.715 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:17.710+0000 7fd247654700 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fd2400cf5d0 0x7fd24015c900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:56:17.715 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:17.710+0000 7fd247654700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2400cfa80 con 0x7fd24015a440 2023-11-24T07:56:17.715 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:17.710+0000 7fd247654700 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2400ce800 con 0x7fd24015eb80 2023-11-24T07:56:17.715 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:17.710+0000 7fd247654700 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd2400ce370 con 0x7fd2400cf5d0 2023-11-24T07:56:17.715 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:17.710+0000 7fd2453f0700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd24015a440 0x7fd24014d6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:56:17.716 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:17.710+0000 7fd2453f0700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd24015a440 0x7fd24014d6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:49506/0 (socket says 172.21.15.59:49506) 2023-11-24T07:56:17.716 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:17.710+0000 7fd2453f0700 1 -- 172.21.15.59:0/2494433402 learned_addr learned my addr 172.21.15.59:0/2494433402 (peer_addr_for_me v2:172.21.15.59:0/0) 2023-11-24T07:56:17.716 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:17.710+0000 7fd245bf1700 1 --2- 172.21.15.59:0/2494433402 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fd2400cf5d0 0x7fd24015c900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:56:17.716 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:17.710+0000 7fd244bef700 1 --2- 172.21.15.59:0/2494433402 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd24015eb80 0x7fd2400cf090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:56:17.716 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:17.711+0000 7fd2453f0700 1 -- 172.21.15.59:0/2494433402 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fd2400cf5d0 msgr2=0x7fd24015c900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:56:17.716 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:17.711+0000 7fd2453f0700 1 --2- 172.21.15.59:0/2494433402 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fd2400cf5d0 0x7fd24015c900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:56:17.716 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:17.711+0000 7fd2453f0700 1 -- 172.21.15.59:0/2494433402 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd24015eb80 msgr2=0x7fd2400cf090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:56:17.716 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:17.711+0000 7fd2453f0700 1 --2- 172.21.15.59:0/2494433402 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd24015eb80 0x7fd2400cf090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:56:17.716 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:17.711+0000 7fd2453f0700 1 -- 172.21.15.59:0/2494433402 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd238009a70 con 0x7fd24015a440 2023-11-24T07:56:17.716 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:17.711+0000 7fd244bef700 1 --2- 172.21.15.59:0/2494433402 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd24015eb80 0x7fd2400cf090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-24T07:56:17.716 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:17.711+0000 7fd2453f0700 1 --2- 172.21.15.59:0/2494433402 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd24015a440 0x7fd24014d6e0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fd2400737f0 tx=0x7fd23c004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:56:17.716 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:17.711+0000 7fd235ffb700 1 -- 172.21.15.59:0/2494433402 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd23c0043c0 con 0x7fd24015a440 2023-11-24T07:56:17.716 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:17.711+0000 7fd235ffb700 1 -- 172.21.15.59:0/2494433402 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd23c01f070 con 0x7fd24015a440 2023-11-24T07:56:17.717 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:17.711+0000 7fd247654700 1 -- 172.21.15.59:0/2494433402 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd23c009550 con 0x7fd24015a440 2023-11-24T07:56:17.717 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:17.711+0000 7fd235ffb700 1 -- 172.21.15.59:0/2494433402 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd23c0036a0 con 0x7fd24015a440 2023-11-24T07:56:17.717 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:17.711+0000 7fd247654700 1 -- 172.21.15.59:0/2494433402 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd2400cda40 con 0x7fd24015a440 2023-11-24T07:56:17.717 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:17.712+0000 7fd247654700 1 -- 172.21.15.59:0/2494433402 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd240068670 con 0x7fd24015a440 2023-11-24T07:56:17.717 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:17.712+0000 7fd235ffb700 1 -- 172.21.15.59:0/2494433402 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 4) v1 ==== 89175+0+0 (secure 0 0 0) 0x7fd23c021030 con 0x7fd24015a440 2023-11-24T07:56:17.717 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:17.713+0000 7fd235ffb700 1 --2- 172.21.15.59:0/2494433402 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7fd22406ae40 0x7fd22406d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:56:17.717 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:17.713+0000 7fd235ffb700 1 -- 172.21.15.59:0/2494433402 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(13..13 src has 1..13) v4 ==== 4471+0+0 (secure 0 0 0) 0x7fd23c08a4c0 con 0x7fd24015a440 2023-11-24T07:56:17.717 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:17.713+0000 7fd244bef700 1 --2- 172.21.15.59:0/2494433402 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7fd22406ae40 0x7fd22406d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:56:17.717 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:17.713+0000 7fd244bef700 1 --2- 172.21.15.59:0/2494433402 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7fd22406ae40 0x7fd22406d300 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fd24015e6c0 tx=0x7fd230009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:56:17.719 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:17.717+0000 7fd235ffb700 1 -- 172.21.15.59:0/2494433402 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fd23c055cd0 con 0x7fd24015a440 2023-11-24T07:56:17.920 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:17.918+0000 7fd247654700 1 -- 172.21.15.59:0/2494433402 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fd2400b59c0 con 0x7fd24015a440 2023-11-24T07:56:17.921 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:17.919+0000 7fd235ffb700 1 -- 172.21.15.59:0/2494433402 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v13) v1 ==== 74+0+10836 (secure 0 0 0) 0x7fd23c0592f0 con 0x7fd24015a440 2023-11-24T07:56:17.922 INFO:teuthology.orchestra.run.smithi059.stdout: 2023-11-24T07:56:17.922 INFO:teuthology.orchestra.run.smithi059.stdout:{"epoch":13,"fsid":"9f0e8741-b8f9-48d5-a2c3-09f45903303d","created":"2023-11-24T07:56:06.552729+0000","modified":"2023-11-24T07:56:17.372352+0000","last_up_change":"2023-11-24T07:56:17.372352+0000","last_in_change":"2023-11-24T07:56:09.073250+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"6d8f9e9b-1782-4e41-928b-a39f2a8ebc06","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6800","nonce":3373878772},{"type":"v1","addr":"172.21.15.59:6801","nonce":3373878772}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6802","nonce":3373878772},{"type":"v1","addr":"172.21.15.59:6803","nonce":3373878772}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6806","nonce":3373878772},{"type":"v1","addr":"172.21.15.59:6807","nonce":3373878772}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6804","nonce":3373878772},{"type":"v1","addr":"172.21.15.59:6805","nonce":3373878772}]},"public_addr":"172.21.15.59:6801/3373878772","cluster_addr":"172.21.15.59:6803/3373878772","heartbeat_back_addr":"172.21.15.59:6807/3373878772","heartbeat_front_addr":"172.21.15.59:6805/3373878772","state":["exists","up"]},{"osd":1,"uuid":"e8d882aa-52e1-4c04-9e1e-c44ef9204a5f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6824","nonce":648258563},{"type":"v1","addr":"172.21.15.59:6825","nonce":648258563}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6826","nonce":648258563},{"type":"v1","addr":"172.21.15.59:6827","nonce":648258563}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6830","nonce":648258563},{"type":"v1","addr":"172.21.15.59:6831","nonce":648258563}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6828","nonce":648258563},{"type":"v1","addr":"172.21.15.59:6829","nonce":648258563}]},"public_addr":"172.21.15.59:6825/648258563","cluster_addr":"172.21.15.59:6827/648258563","heartbeat_back_addr":"172.21.15.59:6831/648258563","heartbeat_front_addr":"172.21.15.59:6829/648258563","state":["exists","up"]},{"osd":2,"uuid":"69eebfa0-695b-4b21-90b9-8a3537168a01","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6808","nonce":1840430654},{"type":"v1","addr":"172.21.15.59:6809","nonce":1840430654}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6810","nonce":1840430654},{"type":"v1","addr":"172.21.15.59:6811","nonce":1840430654}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6814","nonce":1840430654},{"type":"v1","addr":"172.21.15.59:6815","nonce":1840430654}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6812","nonce":1840430654},{"type":"v1","addr":"172.21.15.59:6813","nonce":1840430654}]},"public_addr":"172.21.15.59:6809/1840430654","cluster_addr":"172.21.15.59:6811/1840430654","heartbeat_back_addr":"172.21.15.59:6815/1840430654","heartbeat_front_addr":"172.21.15.59:6813/1840430654","state":["exists","up"]},{"osd":3,"uuid":"96fd77fc-f911-46ff-9445-6f0be2daa8f4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6816","nonce":2504025850},{"type":"v1","addr":"172.21.15.59:6817","nonce":2504025850}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6818","nonce":2504025850},{"type":"v1","addr":"172.21.15.59:6819","nonce":2504025850}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6822","nonce":2504025850},{"type":"v1","addr":"172.21.15.59:6823","nonce":2504025850}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6820","nonce":2504025850},{"type":"v1","addr":"172.21.15.59:6821","nonce":2504025850}]},"public_addr":"172.21.15.59:6817/2504025850","cluster_addr":"172.21.15.59:6819/2504025850","heartbeat_back_addr":"172.21.15.59:6823/2504025850","heartbeat_front_addr":"172.21.15.59:6821/2504025850","state":["exists","up"]},{"osd":4,"uuid":"7ac29fca-b5b2-4c95-9d69-2f164cbbc708","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6816","nonce":2382463431},{"type":"v1","addr":"172.21.15.76:6818","nonce":2382463431}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6820","nonce":2382463431},{"type":"v1","addr":"172.21.15.76:6822","nonce":2382463431}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6828","nonce":2382463431},{"type":"v1","addr":"172.21.15.76:6830","nonce":2382463431}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6824","nonce":2382463431},{"type":"v1","addr":"172.21.15.76:6826","nonce":2382463431}]},"public_addr":"172.21.15.76:6818/2382463431","cluster_addr":"172.21.15.76:6822/2382463431","heartbeat_back_addr":"172.21.15.76:6830/2382463431","heartbeat_front_addr":"172.21.15.76:6826/2382463431","state":["exists","up"]},{"osd":5,"uuid":"485bf711-3851-4238-a4f4-51d52e0a9c58","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6800","nonce":329334308},{"type":"v1","addr":"172.21.15.76:6802","nonce":329334308}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6805","nonce":329334308},{"type":"v1","addr":"172.21.15.76:6807","nonce":329334308}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6814","nonce":329334308},{"type":"v1","addr":"172.21.15.76:6815","nonce":329334308}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6810","nonce":329334308},{"type":"v1","addr":"172.21.15.76:6812","nonce":329334308}]},"public_addr":"172.21.15.76:6802/329334308","cluster_addr":"172.21.15.76:6807/329334308","heartbeat_back_addr":"172.21.15.76:6815/329334308","heartbeat_front_addr":"172.21.15.76:6812/329334308","state":["exists","up"]},{"osd":6,"uuid":"72b3f2fc-1e4a-4e32-968d-003647a0286f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6801","nonce":3946579410},{"type":"v1","addr":"172.21.15.76:6803","nonce":3946579410}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6804","nonce":3946579410},{"type":"v1","addr":"172.21.15.76:6806","nonce":3946579410}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6811","nonce":3946579410},{"type":"v1","addr":"172.21.15.76:6813","nonce":3946579410}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6808","nonce":3946579410},{"type":"v1","addr":"172.21.15.76:6809","nonce":3946579410}]},"public_addr":"172.21.15.76:6803/3946579410","cluster_addr":"172.21.15.76:6806/3946579410","heartbeat_back_addr":"172.21.15.76:6813/3946579410","heartbeat_front_addr":"172.21.15.76:6809/3946579410","state":["exists","up"]},{"osd":7,"uuid":"1ddceb90-6c5d-4d1a-ad56-99cb877f8c70","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6817","nonce":841761538},{"type":"v1","addr":"172.21.15.76:6819","nonce":841761538}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6821","nonce":841761538},{"type":"v1","addr":"172.21.15.76:6823","nonce":841761538}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6829","nonce":841761538},{"type":"v1","addr":"172.21.15.76:6831","nonce":841761538}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6825","nonce":841761538},{"type":"v1","addr":"172.21.15.76:6827","nonce":841761538}]},"public_addr":"172.21.15.76:6819/841761538","cluster_addr":"172.21.15.76:6823/841761538","heartbeat_back_addr":"172.21.15.76:6831/841761538","heartbeat_front_addr":"172.21.15.76:6827/841761538","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-24T07:56:17.923 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:17.921+0000 7fd247654700 1 -- 172.21.15.59:0/2494433402 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7fd22406ae40 msgr2=0x7fd22406d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:56:17.923 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:17.921+0000 7fd247654700 1 --2- 172.21.15.59:0/2494433402 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7fd22406ae40 0x7fd22406d300 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fd24015e6c0 tx=0x7fd230009040 comp rx=0 tx=0).stop 2023-11-24T07:56:17.923 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:17.921+0000 7fd247654700 1 -- 172.21.15.59:0/2494433402 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd24015a440 msgr2=0x7fd24014d6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:56:17.924 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:17.921+0000 7fd247654700 1 --2- 172.21.15.59:0/2494433402 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd24015a440 0x7fd24014d6e0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fd2400737f0 tx=0x7fd23c004750 comp rx=0 tx=0).stop 2023-11-24T07:56:17.924 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:17.921+0000 7fd247654700 1 -- 172.21.15.59:0/2494433402 shutdown_connections 2023-11-24T07:56:17.924 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:17.921+0000 7fd247654700 1 --2- 172.21.15.59:0/2494433402 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7fd22406ae40 0x7fd22406d300 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:56:17.924 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:17.921+0000 7fd247654700 1 --2- 172.21.15.59:0/2494433402 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fd2400cf5d0 0x7fd24015c900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:56:17.924 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:17.921+0000 7fd247654700 1 --2- 172.21.15.59:0/2494433402 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd24015a440 0x7fd24014d6e0 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:56:17.924 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:17.921+0000 7fd247654700 1 --2- 172.21.15.59:0/2494433402 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd24015eb80 0x7fd2400cf090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:56:17.924 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:17.921+0000 7fd247654700 1 -- 172.21.15.59:0/2494433402 >> 172.21.15.59:0/2494433402 conn(0x7fd2400b5070 msgr2=0x7fd24006d350 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:56:17.924 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:17.921+0000 7fd247654700 1 -- 172.21.15.59:0/2494433402 shutdown_connections 2023-11-24T07:56:17.924 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:17.921+0000 7fd247654700 1 -- 172.21.15.59:0/2494433402 wait complete. 2023-11-24T07:56:17.935 INFO:tasks.cephfs.filesystem.ceph_manager:[] 2023-11-24T07:56:17.936 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-24T07:56:18.060 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:18.058+0000 7f69c04cc700 1 Processor -- start 2023-11-24T07:56:18.060 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:18.058+0000 7f69c04cc700 1 -- start start 2023-11-24T07:56:18.062 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:18.059+0000 7f69c04cc700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f69b815ae00 0x7f69b80b5e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:56:18.063 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:18.059+0000 7f69c04cc700 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f69b80b03b0 0x7f69b80ad690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:56:18.063 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:18.059+0000 7f69c04cc700 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f69b80b0ec0 con 0x7f69b80adbd0 2023-11-24T07:56:18.063 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:18.059+0000 7f69c04cc700 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f69b8073610 con 0x7f69b815ae00 2023-11-24T07:56:18.063 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:18.059+0000 7f69c04cc700 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f69b815b1e0 con 0x7f69b80b03b0 2023-11-24T07:56:18.063 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:18.059+0000 7f69be268700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f69b815ae00 0x7f69b80b5e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:56:18.063 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:18.059+0000 7f69be268700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f69b815ae00 0x7f69b80b5e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:49512/0 (socket says 172.21.15.59:49512) 2023-11-24T07:56:18.063 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:18.059+0000 7f69be268700 1 -- 172.21.15.59:0/383210431 learned_addr learned my addr 172.21.15.59:0/383210431 (peer_addr_for_me v2:172.21.15.59:0/0) 2023-11-24T07:56:18.063 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:18.059+0000 7f69bda67700 1 --2- 172.21.15.59:0/383210431 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f69b80b03b0 0x7f69b80ad690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:56:18.063 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:18.059+0000 7f69bda67700 1 -- 172.21.15.59:0/383210431 >> v2:172.21.15.59:3300/0 conn(0x7f69b815ae00 msgr2=0x7f69b80b5e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T07:56:18.063 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:18.059+0000 7f69bda67700 1 --2- 172.21.15.59:0/383210431 >> v2:172.21.15.59:3300/0 conn(0x7f69b815ae00 0x7f69b80b5e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:56:18.063 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:18.059+0000 7f69bda67700 1 -- 172.21.15.59:0/383210431 >> v1:172.21.15.76:6789/0 conn(0x7f69b80adbd0 legacy=0x7f69b815e850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:56:18.063 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:18.059+0000 7f69bda67700 1 -- 172.21.15.59:0/383210431 --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f69b80b0900 con 0x7f69b80b03b0 2023-11-24T07:56:18.064 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:18.059+0000 7f69bd266700 1 -- 172.21.15.59:0/383210431 <== mon.1 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1599421795 0 0) 0x7f69b80b0ec0 con 0x7f69b80adbd0 2023-11-24T07:56:18.064 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:18.060+0000 7f69bda67700 1 --2- 172.21.15.59:0/383210431 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f69b80b03b0 0x7f69b80ad690 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f69b4009870 tx=0x7f69b4009b80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=77374374bd80215a server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:56:18.064 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:18.060+0000 7f69bd266700 1 -- 172.21.15.59:0/383210431 <== mon.2 v2:172.21.15.76:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69b4011830 con 0x7f69b80b03b0 2023-11-24T07:56:18.064 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:18.060+0000 7f69bd266700 1 -- 172.21.15.59:0/383210431 <== mon.2 v2:172.21.15.76:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f69b4011990 con 0x7f69b80b03b0 2023-11-24T07:56:18.064 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:18.060+0000 7f69bd266700 1 -- 172.21.15.59:0/383210431 <== mon.2 v2:172.21.15.76:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69b4011d20 con 0x7f69b80b03b0 2023-11-24T07:56:18.064 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:18.060+0000 7f69c04cc700 1 -- 172.21.15.59:0/383210431 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f69b80b03b0 msgr2=0x7f69b80ad690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:56:18.064 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:18.060+0000 7f69c04cc700 1 --2- 172.21.15.59:0/383210431 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f69b80b03b0 0x7f69b80ad690 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f69b4009870 tx=0x7f69b4009b80 comp rx=0 tx=0).stop 2023-11-24T07:56:18.064 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:18.060+0000 7f69c04cc700 1 -- 172.21.15.59:0/383210431 shutdown_connections 2023-11-24T07:56:18.064 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:18.060+0000 7f69c04cc700 1 --2- 172.21.15.59:0/383210431 >> v2:172.21.15.59:3300/0 conn(0x7f69b815ae00 0x7f69b80b5e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:56:18.064 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:18.060+0000 7f69c04cc700 1 --2- 172.21.15.59:0/383210431 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f69b80b03b0 0x7f69b80ad690 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:56:18.064 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:18.060+0000 7f69c04cc700 1 -- 172.21.15.59:0/383210431 >> 172.21.15.59:0/383210431 conn(0x7f69b80b5070 msgr2=0x7f69b80b5730 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:56:18.064 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:18.060+0000 7f69c04cc700 1 -- 172.21.15.59:0/383210431 shutdown_connections 2023-11-24T07:56:18.064 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:18.060+0000 7f69c04cc700 1 -- 172.21.15.59:0/383210431 wait complete. 2023-11-24T07:56:18.067 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:18.062+0000 7f69c04cc700 1 Processor -- start 2023-11-24T07:56:18.067 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:18.062+0000 7f69c04cc700 1 -- start start 2023-11-24T07:56:18.067 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:18.062+0000 7f69c04cc700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f69b80adbd0 0x7f69b8151870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:56:18.068 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:18.062+0000 7f69c04cc700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f69b80b03b0 0x7f69b81591b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:56:18.068 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:18.062+0000 7f69c04cc700 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f69b815ae00 0x7f69b8155880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:56:18.068 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:18.062+0000 7f69c04cc700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f69b8153c30 con 0x7f69b80b03b0 2023-11-24T07:56:18.068 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:18.062+0000 7f69c04cc700 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f69b81596f0 con 0x7f69b80adbd0 2023-11-24T07:56:18.068 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:18.062+0000 7f69c04cc700 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f69b815bf80 con 0x7f69b815ae00 2023-11-24T07:56:18.068 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:18.062+0000 7f69bda67700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f69b80b03b0 0x7f69b81591b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:56:18.068 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:18.062+0000 7f69bda67700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f69b80b03b0 0x7f69b81591b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:49520/0 (socket says 172.21.15.59:49520) 2023-11-24T07:56:18.068 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:18.062+0000 7f69bda67700 1 -- 172.21.15.59:0/3592830134 learned_addr learned my addr 172.21.15.59:0/3592830134 (peer_addr_for_me v2:172.21.15.59:0/0) 2023-11-24T07:56:18.068 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:18.062+0000 7f69bea69700 1 --2- 172.21.15.59:0/3592830134 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f69b815ae00 0x7f69b8155880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:56:18.068 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:18.063+0000 7f69be268700 1 --2- 172.21.15.59:0/3592830134 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f69b80adbd0 0x7f69b8151870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:56:18.068 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:18.063+0000 7f69bda67700 1 -- 172.21.15.59:0/3592830134 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f69b815ae00 msgr2=0x7f69b8155880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:56:18.069 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:18.063+0000 7f69bda67700 1 --2- 172.21.15.59:0/3592830134 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f69b815ae00 0x7f69b8155880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:56:18.069 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:18.063+0000 7f69bda67700 1 -- 172.21.15.59:0/3592830134 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f69b80adbd0 msgr2=0x7f69b8151870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:56:18.069 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:18.063+0000 7f69bda67700 1 --2- 172.21.15.59:0/3592830134 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f69b80adbd0 0x7f69b8151870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:56:18.069 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:18.063+0000 7f69bda67700 1 -- 172.21.15.59:0/3592830134 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f69b40094a0 con 0x7f69b80b03b0 2023-11-24T07:56:18.069 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:18.063+0000 7f69bea69700 1 --2- 172.21.15.59:0/3592830134 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f69b815ae00 0x7f69b8155880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-24T07:56:18.069 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:18.063+0000 7f69bda67700 1 --2- 172.21.15.59:0/3592830134 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f69b80b03b0 0x7f69b81591b0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f69b4000c00 tx=0x7f69b4004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:56:18.069 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:18.063+0000 7f69aeffd700 1 -- 172.21.15.59:0/3592830134 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69b40036a0 con 0x7f69b80b03b0 2023-11-24T07:56:18.069 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:18.063+0000 7f69aeffd700 1 -- 172.21.15.59:0/3592830134 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f69b401f070 con 0x7f69b80b03b0 2023-11-24T07:56:18.069 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:18.063+0000 7f69c04cc700 1 -- 172.21.15.59:0/3592830134 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f69b81583f0 con 0x7f69b80b03b0 2023-11-24T07:56:18.069 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:18.063+0000 7f69aeffd700 1 -- 172.21.15.59:0/3592830134 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69b4003a00 con 0x7f69b80b03b0 2023-11-24T07:56:18.069 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:18.063+0000 7f69c04cc700 1 -- 172.21.15.59:0/3592830134 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f69b81589b0 con 0x7f69b80b03b0 2023-11-24T07:56:18.070 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:18.064+0000 7f69c04cc700 1 -- 172.21.15.59:0/3592830134 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f69b8068670 con 0x7f69b80b03b0 2023-11-24T07:56:18.070 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:18.064+0000 7f69aeffd700 1 -- 172.21.15.59:0/3592830134 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 4) v1 ==== 89175+0+0 (secure 0 0 0) 0x7f69b4003b60 con 0x7f69b80b03b0 2023-11-24T07:56:18.070 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:18.065+0000 7f69aeffd700 1 --2- 172.21.15.59:0/3592830134 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f699c06ae40 0x7f699c06d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:56:18.070 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:18.065+0000 7f69aeffd700 1 -- 172.21.15.59:0/3592830134 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(13..13 src has 1..13) v4 ==== 4471+0+0 (secure 0 0 0) 0x7f69b4021070 con 0x7f69b80b03b0 2023-11-24T07:56:18.070 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:18.065+0000 7f69be268700 1 --2- 172.21.15.59:0/3592830134 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f699c06ae40 0x7f699c06d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:56:18.070 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:18.065+0000 7f69be268700 1 --2- 172.21.15.59:0/3592830134 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f699c06ae40 0x7f699c06d300 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f69a800bd60 tx=0x7f69a800b4d0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:56:18.071 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:18.069+0000 7f69aeffd700 1 -- 172.21.15.59:0/3592830134 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f69b4056040 con 0x7f69b80b03b0 2023-11-24T07:56:18.318 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:18.316+0000 7f69c04cc700 1 -- 172.21.15.59:0/3592830134 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f69b80d0010 con 0x7f69b80b03b0 2023-11-24T07:56:18.319 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:18.317+0000 7f69aeffd700 1 -- 172.21.15.59:0/3592830134 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 2 v2) v1 ==== 93+0+1321 (secure 0 0 0) 0x7f69b80d0010 con 0x7f69b80b03b0 2023-11-24T07:56:18.319 INFO:teuthology.orchestra.run.smithi059.stdout: 2023-11-24T07:56:18.319 INFO:teuthology.orchestra.run.smithi059.stdout:{"epoch":2,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4220,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6835/838038775","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6833","nonce":838038775},{"type":"v1","addr":"172.21.15.59:6835","nonce":838038775}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2}],"filesystems":[]} 2023-11-24T07:56:18.321 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:18.319+0000 7f69c04cc700 1 -- 172.21.15.59:0/3592830134 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f699c06ae40 msgr2=0x7f699c06d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:56:18.321 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:18.319+0000 7f69c04cc700 1 --2- 172.21.15.59:0/3592830134 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f699c06ae40 0x7f699c06d300 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f69a800bd60 tx=0x7f69a800b4d0 comp rx=0 tx=0).stop 2023-11-24T07:56:18.321 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:18.319+0000 7f69c04cc700 1 -- 172.21.15.59:0/3592830134 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f69b80b03b0 msgr2=0x7f69b81591b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:56:18.321 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:18.319+0000 7f69c04cc700 1 --2- 172.21.15.59:0/3592830134 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f69b80b03b0 0x7f69b81591b0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f69b4000c00 tx=0x7f69b4004750 comp rx=0 tx=0).stop 2023-11-24T07:56:18.321 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:18.319+0000 7f69c04cc700 1 -- 172.21.15.59:0/3592830134 shutdown_connections 2023-11-24T07:56:18.321 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:18.319+0000 7f69c04cc700 1 --2- 172.21.15.59:0/3592830134 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f699c06ae40 0x7f699c06d300 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:56:18.322 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:18.319+0000 7f69c04cc700 1 --2- 172.21.15.59:0/3592830134 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f69b815ae00 0x7f69b8155880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:56:18.322 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:18.319+0000 7f69c04cc700 1 --2- 172.21.15.59:0/3592830134 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f69b80adbd0 0x7f69b8151870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:56:18.322 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:18.319+0000 7f69c04cc700 1 --2- 172.21.15.59:0/3592830134 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f69b80b03b0 0x7f69b81591b0 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:56:18.322 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:18.319+0000 7f69c04cc700 1 -- 172.21.15.59:0/3592830134 >> 172.21.15.59:0/3592830134 conn(0x7f69b80b5070 msgr2=0x7f69b8156ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:56:18.322 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:18.319+0000 7f69c04cc700 1 -- 172.21.15.59:0/3592830134 shutdown_connections 2023-11-24T07:56:18.322 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:18.319+0000 7f69c04cc700 1 -- 172.21.15.59:0/3592830134 wait complete. 2023-11-24T07:56:18.322 INFO:teuthology.orchestra.run.smithi059.stderr:dumped fsmap epoch 2 2023-11-24T07:56:20.334 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-24T07:56:20.462 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:20.461+0000 7f18469e0700 1 Processor -- start 2023-11-24T07:56:20.462 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:20.461+0000 7f18469e0700 1 -- start start 2023-11-24T07:56:20.465 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:20.463+0000 7f18469e0700 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f18400cecd0 0x7f184015c1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:56:20.465 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:20.463+0000 7f18469e0700 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f18400d0a90 con 0x7f18400cf2b0 2023-11-24T07:56:20.465 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:20.463+0000 7f18469e0700 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f184006ee10 con 0x7f184015bde0 2023-11-24T07:56:20.465 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:20.463+0000 7f18469e0700 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f184006f0f0 con 0x7f18400cecd0 2023-11-24T07:56:20.465 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:20.463+0000 7f183f7fe700 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7f18400cf2b0 0x7f18400ce5c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:53048/0 (socket says 172.21.15.59:53048) 2023-11-24T07:56:20.466 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:20.463+0000 7f183f7fe700 1 -- 172.21.15.59:0/2901377627 learned_addr learned my addr 172.21.15.59:0/2901377627 (peer_addr_for_me v1:172.21.15.59:0/0) 2023-11-24T07:56:20.466 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:20.463+0000 7f1844f7d700 1 --2- 172.21.15.59:0/2901377627 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f18400cecd0 0x7f184015c1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:56:20.466 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:20.463+0000 7f183effd700 1 -- 172.21.15.59:0/2901377627 <== mon.0 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2821298525 0 0) 0x7f18400d0a90 con 0x7f18400cf2b0 2023-11-24T07:56:20.466 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:20.463+0000 7f183effd700 1 -- 172.21.15.59:0/2901377627 --> v1:172.21.15.59:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f18280035b0 con 0x7f18400cf2b0 2023-11-24T07:56:20.466 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:20.463+0000 7f183effd700 1 -- 172.21.15.59:0/2901377627 <== mon.1 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1349349933 0 0) 0x7f184006ee10 con 0x7f184015bde0 2023-11-24T07:56:20.466 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:20.463+0000 7f183effd700 1 -- 172.21.15.59:0/2901377627 --> v1:172.21.15.76:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f18400d0a90 con 0x7f184015bde0 2023-11-24T07:56:20.466 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:20.463+0000 7f1844f7d700 1 -- 172.21.15.59:0/2901377627 >> v1:172.21.15.76:6789/0 conn(0x7f184015bde0 legacy=0x7f18400d01b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:56:20.466 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:20.463+0000 7f1844f7d700 1 -- 172.21.15.59:0/2901377627 >> v1:172.21.15.59:6789/0 conn(0x7f18400cf2b0 legacy=0x7f18400ce5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:56:20.466 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:20.463+0000 7f1844f7d700 1 -- 172.21.15.59:0/2901377627 --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f184006f3d0 con 0x7f18400cecd0 2023-11-24T07:56:20.466 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:20.463+0000 7f1844f7d700 1 --2- 172.21.15.59:0/2901377627 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f18400cecd0 0x7f184015c1c0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f1838009dc0 tx=0x7f183800b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fbccb97b0c1b62f9 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:56:20.466 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:20.463+0000 7f183effd700 1 -- 172.21.15.59:0/2901377627 <== mon.2 v2:172.21.15.76:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f183800e070 con 0x7f18400cecd0 2023-11-24T07:56:20.466 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:20.464+0000 7f183effd700 1 -- 172.21.15.59:0/2901377627 <== mon.2 v2:172.21.15.76:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f183800bb20 con 0x7f18400cecd0 2023-11-24T07:56:20.467 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:20.464+0000 7f183effd700 1 -- 172.21.15.59:0/2901377627 <== mon.2 v2:172.21.15.76:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f183800be90 con 0x7f18400cecd0 2023-11-24T07:56:20.467 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:20.464+0000 7f18469e0700 1 -- 172.21.15.59:0/2901377627 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f18400cecd0 msgr2=0x7f184015c1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:56:20.467 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:20.464+0000 7f18469e0700 1 --2- 172.21.15.59:0/2901377627 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f18400cecd0 0x7f184015c1c0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f1838009dc0 tx=0x7f183800b330 comp rx=0 tx=0).stop 2023-11-24T07:56:20.467 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:20.464+0000 7f18469e0700 1 -- 172.21.15.59:0/2901377627 shutdown_connections 2023-11-24T07:56:20.467 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:20.464+0000 7f18469e0700 1 --2- 172.21.15.59:0/2901377627 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f18400cecd0 0x7f184015c1c0 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:56:20.467 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:20.464+0000 7f18469e0700 1 -- 172.21.15.59:0/2901377627 >> 172.21.15.59:0/2901377627 conn(0x7f18400b3720 msgr2=0x7f18400b3b30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:56:20.467 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:20.464+0000 7f18469e0700 1 -- 172.21.15.59:0/2901377627 shutdown_connections 2023-11-24T07:56:20.467 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:20.464+0000 7f18469e0700 1 -- 172.21.15.59:0/2901377627 wait complete. 2023-11-24T07:56:20.469 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:20.465+0000 7f18469e0700 1 Processor -- start 2023-11-24T07:56:20.470 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:20.465+0000 7f18469e0700 1 -- start start 2023-11-24T07:56:20.470 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:20.466+0000 7f18469e0700 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f18400cecd0 0x7f18400ca9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:56:20.470 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:20.466+0000 7f18469e0700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f18400cf2b0 0x7f1840152400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:56:20.470 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:20.466+0000 7f18469e0700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f184015bde0 0x7f18400b6820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:56:20.470 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:20.466+0000 7f18469e0700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1840152940 con 0x7f184015bde0 2023-11-24T07:56:20.470 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:20.466+0000 7f18469e0700 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18400c9940 con 0x7f18400cf2b0 2023-11-24T07:56:20.470 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:20.466+0000 7f18469e0700 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f18400c9c20 con 0x7f18400cecd0 2023-11-24T07:56:20.470 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:20.466+0000 7f1844f7d700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f184015bde0 0x7f18400b6820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:56:20.470 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:20.466+0000 7f1844f7d700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f184015bde0 0x7f18400b6820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:49580/0 (socket says 172.21.15.59:49580) 2023-11-24T07:56:20.470 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:20.466+0000 7f1844f7d700 1 -- 172.21.15.59:0/839614754 learned_addr learned my addr 172.21.15.59:0/839614754 (peer_addr_for_me v2:172.21.15.59:0/0) 2023-11-24T07:56:20.470 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:20.466+0000 7f183ffff700 1 --2- 172.21.15.59:0/839614754 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f18400cecd0 0x7f18400ca9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:56:20.471 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:20.466+0000 7f183f7fe700 1 --2- 172.21.15.59:0/839614754 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f18400cf2b0 0x7f1840152400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:56:20.471 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:20.467+0000 7f1844f7d700 1 -- 172.21.15.59:0/839614754 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f18400cecd0 msgr2=0x7f18400ca9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:56:20.471 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:20.467+0000 7f1844f7d700 1 --2- 172.21.15.59:0/839614754 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f18400cecd0 0x7f18400ca9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:56:20.471 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:20.467+0000 7f1844f7d700 1 -- 172.21.15.59:0/839614754 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f18400cf2b0 msgr2=0x7f1840152400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:56:20.471 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:20.467+0000 7f1844f7d700 1 --2- 172.21.15.59:0/839614754 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f18400cf2b0 0x7f1840152400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:56:20.471 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:20.467+0000 7f1844f7d700 1 -- 172.21.15.59:0/839614754 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1838009a70 con 0x7f184015bde0 2023-11-24T07:56:20.471 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:20.467+0000 7f1844f7d700 1 --2- 172.21.15.59:0/839614754 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f184015bde0 0x7f18400b6820 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f1838009fd0 tx=0x7f1838004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:56:20.471 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:20.467+0000 7f183cff9700 1 -- 172.21.15.59:0/839614754 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18380098f0 con 0x7f184015bde0 2023-11-24T07:56:20.471 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:20.467+0000 7f183cff9700 1 -- 172.21.15.59:0/839614754 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f183801f070 con 0x7f184015bde0 2023-11-24T07:56:20.471 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:20.467+0000 7f18469e0700 1 -- 172.21.15.59:0/839614754 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f18400c9f00 con 0x7f184015bde0 2023-11-24T07:56:20.471 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:20.467+0000 7f183cff9700 1 -- 172.21.15.59:0/839614754 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1838003c40 con 0x7f184015bde0 2023-11-24T07:56:20.471 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:20.467+0000 7f18469e0700 1 -- 172.21.15.59:0/839614754 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f18400ca4c0 con 0x7f184015bde0 2023-11-24T07:56:20.472 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:20.468+0000 7f18469e0700 1 -- 172.21.15.59:0/839614754 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f18400ca650 con 0x7f184015bde0 2023-11-24T07:56:20.472 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:20.468+0000 7f183cff9700 1 -- 172.21.15.59:0/839614754 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 4) v1 ==== 89175+0+0 (secure 0 0 0) 0x7f1838025450 con 0x7f184015bde0 2023-11-24T07:56:20.472 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:20.469+0000 7f183cff9700 1 --2- 172.21.15.59:0/839614754 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f182c06ae40 0x7f182c06d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:56:20.472 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:20.469+0000 7f183cff9700 1 -- 172.21.15.59:0/839614754 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(15..15 src has 1..15) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f183808a6a0 con 0x7f184015bde0 2023-11-24T07:56:20.472 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:20.469+0000 7f183ffff700 1 --2- 172.21.15.59:0/839614754 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f182c06ae40 0x7f182c06d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:56:20.472 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:20.469+0000 7f183ffff700 1 --2- 172.21.15.59:0/839614754 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f182c06ae40 0x7f182c06d300 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f18401f5070 tx=0x7f1830009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:56:20.474 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:20.474+0000 7f183cff9700 1 -- 172.21.15.59:0/839614754 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f1838055b90 con 0x7f184015bde0 2023-11-24T07:56:20.724 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:20.724+0000 7f18469e0700 1 -- 172.21.15.59:0/839614754 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f1840068670 con 0x7f184015bde0 2023-11-24T07:56:20.725 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:20.724+0000 7f183cff9700 1 -- 172.21.15.59:0/839614754 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 3 v3) v1 ==== 93+0+3691 (secure 0 0 0) 0x7f183802ae80 con 0x7f184015bde0 2023-11-24T07:56:20.725 INFO:teuthology.orchestra.run.smithi059.stdout: 2023-11-24T07:56:20.725 INFO:teuthology.orchestra.run.smithi059.stdout:{"epoch":3,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4220,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6835/838038775","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6833","nonce":838038775},{"type":"v1","addr":"172.21.15.59:6835","nonce":838038775}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4232,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.76:6837/3226690004","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6835","nonce":3226690004},{"type":"v1","addr":"172.21.15.76:6837","nonce":3226690004}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4260,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6834/2986456006","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6832","nonce":2986456006},{"type":"v1","addr":"172.21.15.59:6834","nonce":2986456006}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4264,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.76:6836/3993077463","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6834","nonce":3993077463},{"type":"v1","addr":"172.21.15.76:6836","nonce":3993077463}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[]} 2023-11-24T07:56:20.727 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:20.726+0000 7f18469e0700 1 -- 172.21.15.59:0/839614754 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f182c06ae40 msgr2=0x7f182c06d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:56:20.727 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:20.726+0000 7f18469e0700 1 --2- 172.21.15.59:0/839614754 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f182c06ae40 0x7f182c06d300 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f18401f5070 tx=0x7f1830009040 comp rx=0 tx=0).stop 2023-11-24T07:56:20.727 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:20.726+0000 7f18469e0700 1 -- 172.21.15.59:0/839614754 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f184015bde0 msgr2=0x7f18400b6820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:56:20.727 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:20.726+0000 7f18469e0700 1 --2- 172.21.15.59:0/839614754 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f184015bde0 0x7f18400b6820 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f1838009fd0 tx=0x7f1838004ad0 comp rx=0 tx=0).stop 2023-11-24T07:56:20.727 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:20.726+0000 7f18469e0700 1 -- 172.21.15.59:0/839614754 shutdown_connections 2023-11-24T07:56:20.728 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:20.726+0000 7f18469e0700 1 --2- 172.21.15.59:0/839614754 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f182c06ae40 0x7f182c06d300 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:56:20.728 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:20.726+0000 7f18469e0700 1 --2- 172.21.15.59:0/839614754 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f184015bde0 0x7f18400b6820 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:56:20.728 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:20.726+0000 7f18469e0700 1 --2- 172.21.15.59:0/839614754 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f18400cecd0 0x7f18400ca9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:56:20.728 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:20.726+0000 7f18469e0700 1 --2- 172.21.15.59:0/839614754 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f18400cf2b0 0x7f1840152400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:56:20.728 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:20.726+0000 7f18469e0700 1 -- 172.21.15.59:0/839614754 >> 172.21.15.59:0/839614754 conn(0x7f18400b3720 msgr2=0x7f18400d2490 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:56:20.728 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:20.726+0000 7f18469e0700 1 -- 172.21.15.59:0/839614754 shutdown_connections 2023-11-24T07:56:20.729 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:20.726+0000 7f18469e0700 1 -- 172.21.15.59:0/839614754 wait complete. 2023-11-24T07:56:20.729 INFO:teuthology.orchestra.run.smithi059.stderr:dumped fsmap epoch 3 2023-11-24T07:56:20.739 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-24T07:56:20.866 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:20.866+0000 7ff57408a700 1 Processor -- start 2023-11-24T07:56:20.867 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:20.866+0000 7ff57408a700 1 -- start start 2023-11-24T07:56:20.869 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:20.867+0000 7ff57408a700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7ff56c15bde0 0x7ff56c15c1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:56:20.869 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:20.867+0000 7ff57408a700 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7ff56c0cf2b0 0x7ff56c0d01b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:56:20.869 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:20.867+0000 7ff57408a700 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff56c0d0a90 con 0x7ff56c0ce5c0 2023-11-24T07:56:20.869 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:20.867+0000 7ff57408a700 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff56c06ee10 con 0x7ff56c15bde0 2023-11-24T07:56:20.869 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:20.867+0000 7ff57408a700 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff56c06f0f0 con 0x7ff56c0cf2b0 2023-11-24T07:56:20.869 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:20.867+0000 7ff571e26700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7ff56c15bde0 0x7ff56c15c1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:56:20.869 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:20.867+0000 7ff572627700 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7ff56c0ce5c0 0x7ff56c0cc670 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:53062/0 (socket says 172.21.15.59:53062) 2023-11-24T07:56:20.869 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:20.867+0000 7ff572627700 1 -- 172.21.15.59:0/3141360274 learned_addr learned my addr 172.21.15.59:0/3141360274 (peer_addr_for_me v1:172.21.15.59:0/0) 2023-11-24T07:56:20.869 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:20.867+0000 7ff571625700 1 --2- 172.21.15.59:0/3141360274 >> v2:172.21.15.76:3300/0 conn(0x7ff56c0cf2b0 0x7ff56c0d01b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:56:20.870 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:20.867+0000 7ff570e24700 1 -- 172.21.15.59:0/3141360274 <== mon.0 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 734848855 0 0) 0x7ff56c0d0a90 con 0x7ff56c0ce5c0 2023-11-24T07:56:20.870 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:20.867+0000 7ff570e24700 1 -- 172.21.15.59:0/3141360274 --> v1:172.21.15.59:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff5580035b0 con 0x7ff56c0ce5c0 2023-11-24T07:56:20.870 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:20.867+0000 7ff571e26700 1 -- 172.21.15.59:0/3141360274 >> v2:172.21.15.76:3300/0 conn(0x7ff56c0cf2b0 msgr2=0x7ff56c0d01b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T07:56:20.870 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:20.867+0000 7ff571e26700 1 --2- 172.21.15.59:0/3141360274 >> v2:172.21.15.76:3300/0 conn(0x7ff56c0cf2b0 0x7ff56c0d01b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:56:20.870 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:20.867+0000 7ff571e26700 1 -- 172.21.15.59:0/3141360274 >> v1:172.21.15.59:6789/0 conn(0x7ff56c0ce5c0 legacy=0x7ff56c0cc670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:56:20.870 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:20.867+0000 7ff571e26700 1 -- 172.21.15.59:0/3141360274 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff56c06f3d0 con 0x7ff56c15bde0 2023-11-24T07:56:20.870 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:20.867+0000 7ff571e26700 1 --2- 172.21.15.59:0/3141360274 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff56c15bde0 0x7ff56c15c1c0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7ff56c06e780 tx=0x7ff568009b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6c8727938853a2b4 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:56:20.870 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:20.868+0000 7ff570e24700 1 -- 172.21.15.59:0/3141360274 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff568011830 con 0x7ff56c15bde0 2023-11-24T07:56:20.870 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:20.868+0000 7ff570e24700 1 -- 172.21.15.59:0/3141360274 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff568011990 con 0x7ff56c15bde0 2023-11-24T07:56:20.870 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:20.868+0000 7ff570e24700 1 -- 172.21.15.59:0/3141360274 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff568011d20 con 0x7ff56c15bde0 2023-11-24T07:56:20.870 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:20.868+0000 7ff57408a700 1 -- 172.21.15.59:0/3141360274 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff56c15bde0 msgr2=0x7ff56c15c1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:56:20.871 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:20.868+0000 7ff57408a700 1 --2- 172.21.15.59:0/3141360274 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff56c15bde0 0x7ff56c15c1c0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7ff56c06e780 tx=0x7ff568009b20 comp rx=0 tx=0).stop 2023-11-24T07:56:20.871 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:20.868+0000 7ff57408a700 1 -- 172.21.15.59:0/3141360274 shutdown_connections 2023-11-24T07:56:20.871 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:20.868+0000 7ff57408a700 1 --2- 172.21.15.59:0/3141360274 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff56c15bde0 0x7ff56c15c1c0 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:56:20.871 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:20.868+0000 7ff57408a700 1 --2- 172.21.15.59:0/3141360274 >> v2:172.21.15.76:3300/0 conn(0x7ff56c0cf2b0 0x7ff56c0d01b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:56:20.871 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:20.868+0000 7ff57408a700 1 -- 172.21.15.59:0/3141360274 >> 172.21.15.59:0/3141360274 conn(0x7ff56c0b3720 msgr2=0x7ff56c0b3b30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:56:20.871 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:20.868+0000 7ff57408a700 1 -- 172.21.15.59:0/3141360274 shutdown_connections 2023-11-24T07:56:20.871 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:20.868+0000 7ff57408a700 1 -- 172.21.15.59:0/3141360274 wait complete. 2023-11-24T07:56:20.874 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:20.869+0000 7ff57408a700 1 Processor -- start 2023-11-24T07:56:20.874 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:20.869+0000 7ff57408a700 1 -- start start 2023-11-24T07:56:20.874 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:20.870+0000 7ff57408a700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff56c0ce5c0 0x7ff56c0cb910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:56:20.874 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:20.870+0000 7ff57408a700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff56c0cf2b0 0x7ff56c0b5330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:56:20.875 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:20.870+0000 7ff57408a700 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7ff56c15bde0 0x7ff56c1590c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:56:20.875 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:20.870+0000 7ff57408a700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff56c0b6120 con 0x7ff56c0ce5c0 2023-11-24T07:56:20.875 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:20.870+0000 7ff57408a700 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff56c0b5b60 con 0x7ff56c0cf2b0 2023-11-24T07:56:20.875 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:20.870+0000 7ff57408a700 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff56c0b5e40 con 0x7ff56c15bde0 2023-11-24T07:56:20.875 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:20.870+0000 7ff571e26700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff56c0ce5c0 0x7ff56c0cb910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:56:20.875 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:20.870+0000 7ff571e26700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff56c0ce5c0 0x7ff56c0cb910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:49590/0 (socket says 172.21.15.59:49590) 2023-11-24T07:56:20.875 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:20.870+0000 7ff571e26700 1 -- 172.21.15.59:0/3649931711 learned_addr learned my addr 172.21.15.59:0/3649931711 (peer_addr_for_me v2:172.21.15.59:0/0) 2023-11-24T07:56:20.875 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:20.870+0000 7ff571625700 1 --2- 172.21.15.59:0/3649931711 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff56c0cf2b0 0x7ff56c0b5330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:56:20.875 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:20.870+0000 7ff572627700 1 --2- 172.21.15.59:0/3649931711 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7ff56c15bde0 0x7ff56c1590c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:56:20.875 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:20.871+0000 7ff571e26700 1 -- 172.21.15.59:0/3649931711 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7ff56c15bde0 msgr2=0x7ff56c1590c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:56:20.876 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:20.871+0000 7ff571e26700 1 --2- 172.21.15.59:0/3649931711 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7ff56c15bde0 0x7ff56c1590c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:56:20.876 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:20.871+0000 7ff571e26700 1 -- 172.21.15.59:0/3649931711 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff56c0cf2b0 msgr2=0x7ff56c0b5330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:56:20.876 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:20.871+0000 7ff571e26700 1 --2- 172.21.15.59:0/3649931711 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff56c0cf2b0 0x7ff56c0b5330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:56:20.876 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:20.871+0000 7ff571e26700 1 -- 172.21.15.59:0/3649931711 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff5680094a0 con 0x7ff56c0ce5c0 2023-11-24T07:56:20.876 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:20.871+0000 7ff571e26700 1 --2- 172.21.15.59:0/3649931711 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff56c0ce5c0 0x7ff56c0cb910 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7ff568000c00 tx=0x7ff568004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:56:20.876 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:20.871+0000 7ff5627fc700 1 -- 172.21.15.59:0/3649931711 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5680036a0 con 0x7ff56c0ce5c0 2023-11-24T07:56:20.876 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:20.871+0000 7ff5627fc700 1 -- 172.21.15.59:0/3649931711 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff56801e070 con 0x7ff56c0ce5c0 2023-11-24T07:56:20.876 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:20.871+0000 7ff57408a700 1 -- 172.21.15.59:0/3649931711 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff56c06c210 con 0x7ff56c0ce5c0 2023-11-24T07:56:20.876 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:20.871+0000 7ff5627fc700 1 -- 172.21.15.59:0/3649931711 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff568003a20 con 0x7ff56c0ce5c0 2023-11-24T07:56:20.876 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:20.871+0000 7ff57408a700 1 -- 172.21.15.59:0/3649931711 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff56c0b5870 con 0x7ff56c0ce5c0 2023-11-24T07:56:20.876 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:20.872+0000 7ff57408a700 1 -- 172.21.15.59:0/3649931711 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff56c068670 con 0x7ff56c0ce5c0 2023-11-24T07:56:20.877 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:20.873+0000 7ff5627fc700 1 -- 172.21.15.59:0/3649931711 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 4) v1 ==== 89175+0+0 (secure 0 0 0) 0x7ff568003b80 con 0x7ff56c0ce5c0 2023-11-24T07:56:20.877 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:20.873+0000 7ff5627fc700 1 --2- 172.21.15.59:0/3649931711 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7ff55006ae40 0x7ff55006d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:56:20.877 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:20.873+0000 7ff5627fc700 1 -- 172.21.15.59:0/3649931711 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 4857+0+0 (secure 0 0 0) 0x7ff568089f90 con 0x7ff56c0ce5c0 2023-11-24T07:56:20.877 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:20.873+0000 7ff571625700 1 --2- 172.21.15.59:0/3649931711 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7ff55006ae40 0x7ff55006d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:56:20.877 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:20.874+0000 7ff571625700 1 --2- 172.21.15.59:0/3649931711 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7ff55006ae40 0x7ff55006d300 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7ff56c15f1c0 tx=0x7ff55c009210 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:56:20.878 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:20.878+0000 7ff5627fc700 1 -- 172.21.15.59:0/3649931711 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7ff568055500 con 0x7ff56c0ce5c0 2023-11-24T07:56:20.951 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:20.950+0000 7ff5627fc700 1 -- 172.21.15.59:0/3649931711 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7ff5680558d0 con 0x7ff56c0ce5c0 2023-11-24T07:56:21.083 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:21.082+0000 7ff57408a700 1 -- 172.21.15.59:0/3649931711 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7ff56c0cdfa0 con 0x7ff56c0ce5c0 2023-11-24T07:56:21.083 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:21.083+0000 7ff5627fc700 1 -- 172.21.15.59:0/3649931711 <== mon.0 v2:172.21.15.59:3300/0 8 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v16) v1 ==== 74+0+12778 (secure 0 0 0) 0x7ff56806f620 con 0x7ff56c0ce5c0 2023-11-24T07:56:21.084 INFO:teuthology.orchestra.run.smithi059.stdout: 2023-11-24T07:56:21.084 INFO:teuthology.orchestra.run.smithi059.stdout:{"epoch":16,"fsid":"9f0e8741-b8f9-48d5-a2c3-09f45903303d","created":"2023-11-24T07:56:06.552729+0000","modified":"2023-11-24T07:56:20.675357+0000","last_up_change":"2023-11-24T07:56:17.372352+0000","last_in_change":"2023-11-24T07:56:09.073250+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-24T07:56:18.352823+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"6d8f9e9b-1782-4e41-928b-a39f2a8ebc06","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6800","nonce":3373878772},{"type":"v1","addr":"172.21.15.59:6801","nonce":3373878772}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6802","nonce":3373878772},{"type":"v1","addr":"172.21.15.59:6803","nonce":3373878772}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6806","nonce":3373878772},{"type":"v1","addr":"172.21.15.59:6807","nonce":3373878772}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6804","nonce":3373878772},{"type":"v1","addr":"172.21.15.59:6805","nonce":3373878772}]},"public_addr":"172.21.15.59:6801/3373878772","cluster_addr":"172.21.15.59:6803/3373878772","heartbeat_back_addr":"172.21.15.59:6807/3373878772","heartbeat_front_addr":"172.21.15.59:6805/3373878772","state":["exists","up"]},{"osd":1,"uuid":"e8d882aa-52e1-4c04-9e1e-c44ef9204a5f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6824","nonce":648258563},{"type":"v1","addr":"172.21.15.59:6825","nonce":648258563}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6826","nonce":648258563},{"type":"v1","addr":"172.21.15.59:6827","nonce":648258563}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6830","nonce":648258563},{"type":"v1","addr":"172.21.15.59:6831","nonce":648258563}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6828","nonce":648258563},{"type":"v1","addr":"172.21.15.59:6829","nonce":648258563}]},"public_addr":"172.21.15.59:6825/648258563","cluster_addr":"172.21.15.59:6827/648258563","heartbeat_back_addr":"172.21.15.59:6831/648258563","heartbeat_front_addr":"172.21.15.59:6829/648258563","state":["exists","up"]},{"osd":2,"uuid":"69eebfa0-695b-4b21-90b9-8a3537168a01","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6808","nonce":1840430654},{"type":"v1","addr":"172.21.15.59:6809","nonce":1840430654}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6810","nonce":1840430654},{"type":"v1","addr":"172.21.15.59:6811","nonce":1840430654}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6814","nonce":1840430654},{"type":"v1","addr":"172.21.15.59:6815","nonce":1840430654}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6812","nonce":1840430654},{"type":"v1","addr":"172.21.15.59:6813","nonce":1840430654}]},"public_addr":"172.21.15.59:6809/1840430654","cluster_addr":"172.21.15.59:6811/1840430654","heartbeat_back_addr":"172.21.15.59:6815/1840430654","heartbeat_front_addr":"172.21.15.59:6813/1840430654","state":["exists","up"]},{"osd":3,"uuid":"96fd77fc-f911-46ff-9445-6f0be2daa8f4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6816","nonce":2504025850},{"type":"v1","addr":"172.21.15.59:6817","nonce":2504025850}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6818","nonce":2504025850},{"type":"v1","addr":"172.21.15.59:6819","nonce":2504025850}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6822","nonce":2504025850},{"type":"v1","addr":"172.21.15.59:6823","nonce":2504025850}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6820","nonce":2504025850},{"type":"v1","addr":"172.21.15.59:6821","nonce":2504025850}]},"public_addr":"172.21.15.59:6817/2504025850","cluster_addr":"172.21.15.59:6819/2504025850","heartbeat_back_addr":"172.21.15.59:6823/2504025850","heartbeat_front_addr":"172.21.15.59:6821/2504025850","state":["exists","up"]},{"osd":4,"uuid":"7ac29fca-b5b2-4c95-9d69-2f164cbbc708","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6816","nonce":2382463431},{"type":"v1","addr":"172.21.15.76:6818","nonce":2382463431}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6820","nonce":2382463431},{"type":"v1","addr":"172.21.15.76:6822","nonce":2382463431}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6828","nonce":2382463431},{"type":"v1","addr":"172.21.15.76:6830","nonce":2382463431}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6824","nonce":2382463431},{"type":"v1","addr":"172.21.15.76:6826","nonce":2382463431}]},"public_addr":"172.21.15.76:6818/2382463431","cluster_addr":"172.21.15.76:6822/2382463431","heartbeat_back_addr":"172.21.15.76:6830/2382463431","heartbeat_front_addr":"172.21.15.76:6826/2382463431","state":["exists","up"]},{"osd":5,"uuid":"485bf711-3851-4238-a4f4-51d52e0a9c58","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6800","nonce":329334308},{"type":"v1","addr":"172.21.15.76:6802","nonce":329334308}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6805","nonce":329334308},{"type":"v1","addr":"172.21.15.76:6807","nonce":329334308}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6814","nonce":329334308},{"type":"v1","addr":"172.21.15.76:6815","nonce":329334308}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6810","nonce":329334308},{"type":"v1","addr":"172.21.15.76:6812","nonce":329334308}]},"public_addr":"172.21.15.76:6802/329334308","cluster_addr":"172.21.15.76:6807/329334308","heartbeat_back_addr":"172.21.15.76:6815/329334308","heartbeat_front_addr":"172.21.15.76:6812/329334308","state":["exists","up"]},{"osd":6,"uuid":"72b3f2fc-1e4a-4e32-968d-003647a0286f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6801","nonce":3946579410},{"type":"v1","addr":"172.21.15.76:6803","nonce":3946579410}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6804","nonce":3946579410},{"type":"v1","addr":"172.21.15.76:6806","nonce":3946579410}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6811","nonce":3946579410},{"type":"v1","addr":"172.21.15.76:6813","nonce":3946579410}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6808","nonce":3946579410},{"type":"v1","addr":"172.21.15.76:6809","nonce":3946579410}]},"public_addr":"172.21.15.76:6803/3946579410","cluster_addr":"172.21.15.76:6806/3946579410","heartbeat_back_addr":"172.21.15.76:6813/3946579410","heartbeat_front_addr":"172.21.15.76:6809/3946579410","state":["exists","up"]},{"osd":7,"uuid":"1ddceb90-6c5d-4d1a-ad56-99cb877f8c70","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6817","nonce":841761538},{"type":"v1","addr":"172.21.15.76:6819","nonce":841761538}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6821","nonce":841761538},{"type":"v1","addr":"172.21.15.76:6823","nonce":841761538}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6829","nonce":841761538},{"type":"v1","addr":"172.21.15.76:6831","nonce":841761538}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6825","nonce":841761538},{"type":"v1","addr":"172.21.15.76:6827","nonce":841761538}]},"public_addr":"172.21.15.76:6819/841761538","cluster_addr":"172.21.15.76:6823/841761538","heartbeat_back_addr":"172.21.15.76:6831/841761538","heartbeat_front_addr":"172.21.15.76:6827/841761538","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T07:56:14.294838+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T07:56:14.270779+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T07:56:14.231809+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T07:56:14.260983+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T07:56:14.302997+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T07:56:14.258600+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T07:56:14.251413+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T07:56:14.304727+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-24T07:56:21.086 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:21.085+0000 7ff57408a700 1 -- 172.21.15.59:0/3649931711 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7ff55006ae40 msgr2=0x7ff55006d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:56:21.087 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:21.085+0000 7ff57408a700 1 --2- 172.21.15.59:0/3649931711 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7ff55006ae40 0x7ff55006d300 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7ff56c15f1c0 tx=0x7ff55c009210 comp rx=0 tx=0).stop 2023-11-24T07:56:21.087 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:21.085+0000 7ff57408a700 1 -- 172.21.15.59:0/3649931711 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff56c0ce5c0 msgr2=0x7ff56c0cb910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:56:21.087 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:21.085+0000 7ff57408a700 1 --2- 172.21.15.59:0/3649931711 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff56c0ce5c0 0x7ff56c0cb910 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7ff568000c00 tx=0x7ff568004750 comp rx=0 tx=0).stop 2023-11-24T07:56:21.087 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:21.086+0000 7ff57408a700 1 -- 172.21.15.59:0/3649931711 shutdown_connections 2023-11-24T07:56:21.087 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:21.086+0000 7ff57408a700 1 --2- 172.21.15.59:0/3649931711 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7ff55006ae40 0x7ff55006d300 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:56:21.087 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:21.086+0000 7ff57408a700 1 --2- 172.21.15.59:0/3649931711 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7ff56c15bde0 0x7ff56c1590c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:56:21.087 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:21.086+0000 7ff57408a700 1 --2- 172.21.15.59:0/3649931711 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff56c0ce5c0 0x7ff56c0cb910 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:56:21.087 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:21.086+0000 7ff57408a700 1 --2- 172.21.15.59:0/3649931711 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff56c0cf2b0 0x7ff56c0b5330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:56:21.087 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:21.086+0000 7ff57408a700 1 -- 172.21.15.59:0/3649931711 >> 172.21.15.59:0/3649931711 conn(0x7ff56c0b3720 msgr2=0x7ff56c159fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:56:21.087 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:21.086+0000 7ff57408a700 1 -- 172.21.15.59:0/3649931711 shutdown_connections 2023-11-24T07:56:21.087 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:21.086+0000 7ff57408a700 1 -- 172.21.15.59:0/3649931711 wait complete. 2023-11-24T07:56:21.102 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-24T07:56:18.352823+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-24T07:56:21.102 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-24T07:56:21.230 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:21.229+0000 7f1e3977f700 1 Processor -- start 2023-11-24T07:56:21.230 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:21.229+0000 7f1e3977f700 1 -- start start 2023-11-24T07:56:21.232 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:21.230+0000 7f1e3977f700 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f1e340b4240 0x7f1e34160330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:56:21.232 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:21.230+0000 7f1e3977f700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f1e340ce140 0x7f1e340ce590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:56:21.232 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:21.230+0000 7f1e3977f700 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1e340b14c0 con 0x7f1e3415f410 2023-11-24T07:56:21.232 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:21.230+0000 7f1e3977f700 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1e340b46f0 con 0x7f1e340ce140 2023-11-24T07:56:21.233 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:21.230+0000 7f1e3977f700 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1e340b3070 con 0x7f1e340b4240 2023-11-24T07:56:21.233 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:21.230+0000 7f1e337fe700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f1e340ce140 0x7f1e340ce590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:56:21.233 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:21.230+0000 7f1e32ffd700 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7f1e3415f410 0x7f1e3415fc20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:53072/0 (socket says 172.21.15.59:53072) 2023-11-24T07:56:21.233 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:21.230+0000 7f1e32ffd700 1 -- 172.21.15.59:0/679498583 learned_addr learned my addr 172.21.15.59:0/679498583 (peer_addr_for_me v1:172.21.15.59:0/0) 2023-11-24T07:56:21.233 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:21.230+0000 7f1e327fc700 1 --2- 172.21.15.59:0/679498583 >> v2:172.21.15.76:3300/0 conn(0x7f1e340b4240 0x7f1e34160330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:56:21.233 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:21.230+0000 7f1e337fe700 1 -- 172.21.15.59:0/679498583 >> v2:172.21.15.76:3300/0 conn(0x7f1e340b4240 msgr2=0x7f1e34160330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T07:56:21.233 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:21.230+0000 7f1e337fe700 1 --2- 172.21.15.59:0/679498583 >> v2:172.21.15.76:3300/0 conn(0x7f1e340b4240 0x7f1e34160330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:56:21.233 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:21.230+0000 7f1e337fe700 1 -- 172.21.15.59:0/679498583 >> v1:172.21.15.59:6789/0 conn(0x7f1e3415f410 legacy=0x7f1e3415fc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:56:21.233 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:21.230+0000 7f1e31ffb700 1 -- 172.21.15.59:0/679498583 <== mon.0 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 926965171 0 0) 0x7f1e340b14c0 con 0x7f1e3415f410 2023-11-24T07:56:21.233 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:21.230+0000 7f1e337fe700 1 -- 172.21.15.59:0/679498583 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1e340b3270 con 0x7f1e340ce140 2023-11-24T07:56:21.233 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:21.230+0000 7f1e327fc700 1 --2- 172.21.15.59:0/679498583 >> v2:172.21.15.76:3300/0 conn(0x7f1e340b4240 0x7f1e34160330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T07:56:21.233 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:21.231+0000 7f1e337fe700 1 --2- 172.21.15.59:0/679498583 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1e340ce140 0x7f1e340ce590 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f1e340cdd50 tx=0x7f1e2c00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bbecc8707d081b9 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:56:21.234 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:21.231+0000 7f1e31ffb700 1 -- 172.21.15.59:0/679498583 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e2c00e070 con 0x7f1e340ce140 2023-11-24T07:56:21.234 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:21.231+0000 7f1e31ffb700 1 -- 172.21.15.59:0/679498583 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1e2c00bb90 con 0x7f1e340ce140 2023-11-24T07:56:21.234 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:21.231+0000 7f1e31ffb700 1 -- 172.21.15.59:0/679498583 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e2c012430 con 0x7f1e340ce140 2023-11-24T07:56:21.234 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:21.231+0000 7f1e3977f700 1 -- 172.21.15.59:0/679498583 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1e340ce140 msgr2=0x7f1e340ce590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:56:21.234 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:21.231+0000 7f1e3977f700 1 --2- 172.21.15.59:0/679498583 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1e340ce140 0x7f1e340ce590 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f1e340cdd50 tx=0x7f1e2c00b330 comp rx=0 tx=0).stop 2023-11-24T07:56:21.234 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:21.231+0000 7f1e3977f700 1 -- 172.21.15.59:0/679498583 shutdown_connections 2023-11-24T07:56:21.234 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:21.231+0000 7f1e3977f700 1 --2- 172.21.15.59:0/679498583 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1e340ce140 0x7f1e340ce590 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:56:21.234 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:21.231+0000 7f1e3977f700 1 --2- 172.21.15.59:0/679498583 >> v2:172.21.15.76:3300/0 conn(0x7f1e340b4240 0x7f1e34160330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:56:21.234 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:21.231+0000 7f1e3977f700 1 -- 172.21.15.59:0/679498583 >> 172.21.15.59:0/679498583 conn(0x7f1e340b3720 msgr2=0x7f1e340b3b30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:56:21.234 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:21.231+0000 7f1e3977f700 1 -- 172.21.15.59:0/679498583 shutdown_connections 2023-11-24T07:56:21.234 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:21.231+0000 7f1e3977f700 1 -- 172.21.15.59:0/679498583 wait complete. 2023-11-24T07:56:21.237 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:21.233+0000 7f1e3977f700 1 Processor -- start 2023-11-24T07:56:21.237 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:21.233+0000 7f1e3977f700 1 -- start start 2023-11-24T07:56:21.237 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:21.233+0000 7f1e3977f700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1e340b4240 0x7f1e340c8130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:56:21.237 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:21.233+0000 7f1e3977f700 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f1e340ce140 0x7f1e34159bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:56:21.237 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:21.233+0000 7f1e3977f700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1e3415f410 0x7f1e340c9c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:56:21.237 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:21.233+0000 7f1e3977f700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e3415a100 con 0x7f1e3415f410 2023-11-24T07:56:21.237 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:21.233+0000 7f1e3977f700 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e340cb130 con 0x7f1e340b4240 2023-11-24T07:56:21.237 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:21.233+0000 7f1e3977f700 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1e340ca5b0 con 0x7f1e340ce140 2023-11-24T07:56:21.238 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:21.234+0000 7f1e337fe700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1e3415f410 0x7f1e340c9c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:56:21.238 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:21.234+0000 7f1e337fe700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1e3415f410 0x7f1e340c9c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:49608/0 (socket says 172.21.15.59:49608) 2023-11-24T07:56:21.238 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:21.234+0000 7f1e337fe700 1 -- 172.21.15.59:0/1966911079 learned_addr learned my addr 172.21.15.59:0/1966911079 (peer_addr_for_me v2:172.21.15.59:0/0) 2023-11-24T07:56:21.238 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:21.234+0000 7f1e32ffd700 1 --2- 172.21.15.59:0/1966911079 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1e340b4240 0x7f1e340c8130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:56:21.238 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:21.234+0000 7f1e327fc700 1 --2- 172.21.15.59:0/1966911079 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f1e340ce140 0x7f1e34159bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:56:21.238 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:21.234+0000 7f1e337fe700 1 -- 172.21.15.59:0/1966911079 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f1e340ce140 msgr2=0x7f1e34159bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:56:21.238 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:21.234+0000 7f1e337fe700 1 --2- 172.21.15.59:0/1966911079 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f1e340ce140 0x7f1e34159bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:56:21.238 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:21.234+0000 7f1e337fe700 1 -- 172.21.15.59:0/1966911079 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1e340b4240 msgr2=0x7f1e340c8130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:56:21.238 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:21.234+0000 7f1e337fe700 1 --2- 172.21.15.59:0/1966911079 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1e340b4240 0x7f1e340c8130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:56:21.238 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:21.234+0000 7f1e337fe700 1 -- 172.21.15.59:0/1966911079 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1e2c009a70 con 0x7f1e3415f410 2023-11-24T07:56:21.238 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:21.234+0000 7f1e327fc700 1 --2- 172.21.15.59:0/1966911079 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f1e340ce140 0x7f1e34159bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-24T07:56:21.238 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:21.234+0000 7f1e337fe700 1 --2- 172.21.15.59:0/1966911079 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1e3415f410 0x7f1e340c9c80 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f1e2c014040 tx=0x7f1e2c004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:56:21.239 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:21.234+0000 7f1e1b7fe700 1 -- 172.21.15.59:0/1966911079 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e2c012430 con 0x7f1e3415f410 2023-11-24T07:56:21.239 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:21.234+0000 7f1e1b7fe700 1 -- 172.21.15.59:0/1966911079 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1e2c020070 con 0x7f1e3415f410 2023-11-24T07:56:21.239 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:21.234+0000 7f1e3977f700 1 -- 172.21.15.59:0/1966911079 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1e340ca890 con 0x7f1e3415f410 2023-11-24T07:56:21.239 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:21.234+0000 7f1e1b7fe700 1 -- 172.21.15.59:0/1966911079 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e2c004280 con 0x7f1e3415f410 2023-11-24T07:56:21.239 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:21.234+0000 7f1e3977f700 1 -- 172.21.15.59:0/1966911079 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1e340cae50 con 0x7f1e3415f410 2023-11-24T07:56:21.239 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:21.236+0000 7f1e3977f700 1 -- 172.21.15.59:0/1966911079 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1e34075480 con 0x7f1e3415f410 2023-11-24T07:56:21.239 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:21.236+0000 7f1e1b7fe700 1 -- 172.21.15.59:0/1966911079 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f1e2c0043e0 con 0x7f1e3415f410 2023-11-24T07:56:21.239 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:21.236+0000 7f1e1b7fe700 1 --2- 172.21.15.59:0/1966911079 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f1e2006aed0 0x7f1e2006d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:56:21.239 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:21.236+0000 7f1e1b7fe700 1 -- 172.21.15.59:0/1966911079 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f1e2c022070 con 0x7f1e3415f410 2023-11-24T07:56:21.239 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:21.236+0000 7f1e32ffd700 1 --2- 172.21.15.59:0/1966911079 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f1e2006aed0 0x7f1e2006d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:56:21.240 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:21.237+0000 7f1e32ffd700 1 --2- 172.21.15.59:0/1966911079 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f1e2006aed0 0x7f1e2006d390 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f1e28000f60 tx=0x7f1e28007e70 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:56:21.241 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:21.241+0000 7f1e1b7fe700 1 -- 172.21.15.59:0/1966911079 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f1e2c0564f0 con 0x7f1e3415f410 2023-11-24T07:56:21.442 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:21.441+0000 7f1e3977f700 1 -- 172.21.15.59:0/1966911079 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f1e340cf100 con 0x7f1e3415f410 2023-11-24T07:56:21.442 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:21.441+0000 7f1e1b7fe700 1 -- 172.21.15.59:0/1966911079 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v16) v1 ==== 93+0+10 (secure 0 0 0) 0x7f1e2c02c020 con 0x7f1e3415f410 2023-11-24T07:56:21.442 INFO:teuthology.orchestra.run.smithi059.stdout:pg_num: 1 2023-11-24T07:56:21.445 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:21.444+0000 7f1e3977f700 1 -- 172.21.15.59:0/1966911079 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f1e2006aed0 msgr2=0x7f1e2006d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:56:21.445 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:21.444+0000 7f1e3977f700 1 --2- 172.21.15.59:0/1966911079 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f1e2006aed0 0x7f1e2006d390 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f1e28000f60 tx=0x7f1e28007e70 comp rx=0 tx=0).stop 2023-11-24T07:56:21.445 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:21.444+0000 7f1e3977f700 1 -- 172.21.15.59:0/1966911079 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1e3415f410 msgr2=0x7f1e340c9c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:56:21.445 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:21.444+0000 7f1e3977f700 1 --2- 172.21.15.59:0/1966911079 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1e3415f410 0x7f1e340c9c80 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f1e2c014040 tx=0x7f1e2c004ad0 comp rx=0 tx=0).stop 2023-11-24T07:56:21.445 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:21.444+0000 7f1e3977f700 1 -- 172.21.15.59:0/1966911079 shutdown_connections 2023-11-24T07:56:21.445 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:21.444+0000 7f1e3977f700 1 --2- 172.21.15.59:0/1966911079 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f1e2006aed0 0x7f1e2006d390 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:56:21.445 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:21.444+0000 7f1e3977f700 1 --2- 172.21.15.59:0/1966911079 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1e3415f410 0x7f1e340c9c80 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:56:21.445 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:21.444+0000 7f1e3977f700 1 --2- 172.21.15.59:0/1966911079 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1e340b4240 0x7f1e340c8130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:56:21.445 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:21.444+0000 7f1e3977f700 1 --2- 172.21.15.59:0/1966911079 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f1e340ce140 0x7f1e34159bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:56:21.445 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:21.444+0000 7f1e3977f700 1 -- 172.21.15.59:0/1966911079 >> 172.21.15.59:0/1966911079 conn(0x7f1e340b3720 msgr2=0x7f1e340b3b00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:56:21.446 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:21.444+0000 7f1e3977f700 1 -- 172.21.15.59:0/1966911079 shutdown_connections 2023-11-24T07:56:21.446 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:21.444+0000 7f1e3977f700 1 -- 172.21.15.59:0/1966911079 wait complete. 2023-11-24T07:56:21.456 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2023-11-24T07:56:21.581 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:21.580+0000 7fc6aa378700 1 Processor -- start 2023-11-24T07:56:21.581 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:21.580+0000 7fc6aa378700 1 -- start start 2023-11-24T07:56:21.583 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:21.581+0000 7fc6aa378700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fc6a415a440 0x7fc6a40cffa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:56:21.583 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:21.581+0000 7fc6aa378700 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc6a40b05f0 con 0x7fc6a415eb80 2023-11-24T07:56:21.583 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:21.581+0000 7fc6aa378700 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc6a40734c0 con 0x7fc6a415a820 2023-11-24T07:56:21.583 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:21.581+0000 7fc6aa378700 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc6a415f080 con 0x7fc6a415a440 2023-11-24T07:56:21.583 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:21.581+0000 7fc6a3fff700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fc6a415a440 0x7fc6a40cffa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:56:21.583 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:21.581+0000 7fc6a8915700 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7fc6a415eb80 0x7fc6a4072db0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:53090/0 (socket says 172.21.15.59:53090) 2023-11-24T07:56:21.583 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:21.581+0000 7fc6a3fff700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fc6a415a440 0x7fc6a40cffa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:49612/0 (socket says 172.21.15.59:49612) 2023-11-24T07:56:21.583 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:21.581+0000 7fc6a8915700 1 -- 172.21.15.59:0/1101028511 learned_addr learned my addr 172.21.15.59:0/1101028511 (peer_addr_for_me v1:172.21.15.59:0/0) 2023-11-24T07:56:21.583 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:21.581+0000 7fc6a3fff700 1 -- 172.21.15.59:0/1101028511 >> v1:172.21.15.76:6789/0 conn(0x7fc6a415a820 legacy=0x7fc6a40726a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:56:21.583 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:21.581+0000 7fc6a3fff700 1 -- 172.21.15.59:0/1101028511 >> v1:172.21.15.59:6789/0 conn(0x7fc6a415eb80 legacy=0x7fc6a4072db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:56:21.584 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:21.581+0000 7fc6a3fff700 1 -- 172.21.15.59:0/1101028511 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc6a40afd50 con 0x7fc6a415a440 2023-11-24T07:56:21.584 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:21.581+0000 7fc6a3fff700 1 --2- 172.21.15.59:0/1101028511 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc6a415a440 0x7fc6a40cffa0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fc69400a9e0 tx=0x7fc69400acf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=373379c44c317c39 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:56:21.584 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:21.581+0000 7fc6a2ffd700 1 -- 172.21.15.59:0/1101028511 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc69400cb70 con 0x7fc6a415a440 2023-11-24T07:56:21.584 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:21.581+0000 7fc6a2ffd700 1 -- 172.21.15.59:0/1101028511 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc69400ccd0 con 0x7fc6a415a440 2023-11-24T07:56:21.584 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:21.581+0000 7fc6a2ffd700 1 -- 172.21.15.59:0/1101028511 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc694018620 con 0x7fc6a415a440 2023-11-24T07:56:21.584 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:21.582+0000 7fc6aa378700 1 -- 172.21.15.59:0/1101028511 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc6a415a440 msgr2=0x7fc6a40cffa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:56:21.584 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:21.582+0000 7fc6aa378700 1 --2- 172.21.15.59:0/1101028511 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc6a415a440 0x7fc6a40cffa0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fc69400a9e0 tx=0x7fc69400acf0 comp rx=0 tx=0).stop 2023-11-24T07:56:21.584 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:21.582+0000 7fc6aa378700 1 -- 172.21.15.59:0/1101028511 shutdown_connections 2023-11-24T07:56:21.584 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:21.582+0000 7fc6aa378700 1 --2- 172.21.15.59:0/1101028511 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc6a415a440 0x7fc6a40cffa0 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:56:21.584 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:21.582+0000 7fc6aa378700 1 -- 172.21.15.59:0/1101028511 >> 172.21.15.59:0/1101028511 conn(0x7fc6a40b5070 msgr2=0x7fc6a40b5730 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:56:21.584 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:21.582+0000 7fc6aa378700 1 -- 172.21.15.59:0/1101028511 shutdown_connections 2023-11-24T07:56:21.584 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:21.582+0000 7fc6aa378700 1 -- 172.21.15.59:0/1101028511 wait complete. 2023-11-24T07:56:21.587 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:21.583+0000 7fc6aa378700 1 Processor -- start 2023-11-24T07:56:21.587 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:21.583+0000 7fc6aa378700 1 -- start start 2023-11-24T07:56:21.587 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:21.584+0000 7fc6aa378700 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fc6a415a440 0x7fc6a414ec20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:56:21.587 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:21.584+0000 7fc6aa378700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc6a415eb80 0x7fc6a40cf070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:56:21.587 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:21.584+0000 7fc6aa378700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc6a414fa40 0x7fc6a415b890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:56:21.587 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:21.584+0000 7fc6aa378700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6a40cfc50 con 0x7fc6a414fa40 2023-11-24T07:56:21.588 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:21.584+0000 7fc6aa378700 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6a415b400 con 0x7fc6a415eb80 2023-11-24T07:56:21.588 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:21.584+0000 7fc6aa378700 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc6a40cf970 con 0x7fc6a415a440 2023-11-24T07:56:21.588 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:21.584+0000 7fc6a8915700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc6a414fa40 0x7fc6a415b890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:56:21.588 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:21.584+0000 7fc6a8915700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc6a414fa40 0x7fc6a415b890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:49622/0 (socket says 172.21.15.59:49622) 2023-11-24T07:56:21.588 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:21.584+0000 7fc6a8915700 1 -- 172.21.15.59:0/1000712045 learned_addr learned my addr 172.21.15.59:0/1000712045 (peer_addr_for_me v2:172.21.15.59:0/0) 2023-11-24T07:56:21.588 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:21.584+0000 7fc6a3fff700 1 --2- 172.21.15.59:0/1000712045 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fc6a415a440 0x7fc6a414ec20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:56:21.588 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:21.584+0000 7fc6a37fe700 1 --2- 172.21.15.59:0/1000712045 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc6a415eb80 0x7fc6a40cf070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:56:21.589 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:21.585+0000 7fc6a8915700 1 -- 172.21.15.59:0/1000712045 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fc6a415a440 msgr2=0x7fc6a414ec20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:56:21.589 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:21.585+0000 7fc6a8915700 1 --2- 172.21.15.59:0/1000712045 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fc6a415a440 0x7fc6a414ec20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:56:21.589 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:21.585+0000 7fc6a8915700 1 -- 172.21.15.59:0/1000712045 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc6a415eb80 msgr2=0x7fc6a40cf070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:56:21.589 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:21.585+0000 7fc6a8915700 1 --2- 172.21.15.59:0/1000712045 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc6a415eb80 0x7fc6a40cf070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:56:21.589 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:21.585+0000 7fc6a8915700 1 -- 172.21.15.59:0/1000712045 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc69400a640 con 0x7fc6a414fa40 2023-11-24T07:56:21.589 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:21.585+0000 7fc6a8915700 1 --2- 172.21.15.59:0/1000712045 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc6a414fa40 0x7fc6a415b890 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fc69c00caa0 tx=0x7fc69c00ce60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:56:21.589 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:21.585+0000 7fc6a0ff9700 1 -- 172.21.15.59:0/1000712045 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc69c00aa80 con 0x7fc6a414fa40 2023-11-24T07:56:21.589 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:21.585+0000 7fc6a0ff9700 1 -- 172.21.15.59:0/1000712045 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc69c00abe0 con 0x7fc6a414fa40 2023-11-24T07:56:21.590 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:21.585+0000 7fc6aa378700 1 -- 172.21.15.59:0/1000712045 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc6a415adb0 con 0x7fc6a414fa40 2023-11-24T07:56:21.590 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:21.585+0000 7fc6a0ff9700 1 -- 172.21.15.59:0/1000712045 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc69c012400 con 0x7fc6a414fa40 2023-11-24T07:56:21.590 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:21.585+0000 7fc6aa378700 1 -- 172.21.15.59:0/1000712045 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc6a414f160 con 0x7fc6a414fa40 2023-11-24T07:56:21.590 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:21.586+0000 7fc6aa378700 1 -- 172.21.15.59:0/1000712045 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc6a4068670 con 0x7fc6a414fa40 2023-11-24T07:56:21.590 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:21.586+0000 7fc6a0ff9700 1 -- 172.21.15.59:0/1000712045 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fc69c01b3b0 con 0x7fc6a414fa40 2023-11-24T07:56:21.591 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:21.587+0000 7fc6a0ff9700 1 --2- 172.21.15.59:0/1000712045 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7fc69006aed0 0x7fc69006d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:56:21.591 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:21.587+0000 7fc6a0ff9700 1 -- 172.21.15.59:0/1000712045 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 4857+0+0 (secure 0 0 0) 0x7fc69c088e10 con 0x7fc6a414fa40 2023-11-24T07:56:21.591 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:21.587+0000 7fc6a3fff700 1 --2- 172.21.15.59:0/1000712045 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7fc69006aed0 0x7fc69006d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:56:21.591 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:21.587+0000 7fc6a3fff700 1 --2- 172.21.15.59:0/1000712045 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7fc69006aed0 0x7fc69006d390 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fc69400a1c0 tx=0x7fc69401a040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:56:21.592 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:21.591+0000 7fc6a0ff9700 1 -- 172.21.15.59:0/1000712045 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fc69c0542c0 con 0x7fc6a414fa40 2023-11-24T07:56:21.799 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:21.799+0000 7fc6aa378700 1 -- 172.21.15.59:0/1000712045 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7fc6a415b540 con 0x7fc6a414fa40 2023-11-24T07:56:21.800 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:21.799+0000 7fc6a0ff9700 1 -- 172.21.15.59:0/1000712045 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v16) v1 ==== 84+0+65 (secure 0 0 0) 0x7fc69c017060 con 0x7fc6a414fa40 2023-11-24T07:56:21.800 INFO:teuthology.orchestra.run.smithi059.stdout: 2023-11-24T07:56:21.801 INFO:teuthology.orchestra.run.smithi059.stdout:[ 2023-11-24T07:56:21.801 INFO:teuthology.orchestra.run.smithi059.stdout: { 2023-11-24T07:56:21.801 INFO:teuthology.orchestra.run.smithi059.stdout: "poolnum": 1, 2023-11-24T07:56:21.801 INFO:teuthology.orchestra.run.smithi059.stdout: "poolname": ".mgr" 2023-11-24T07:56:21.802 INFO:teuthology.orchestra.run.smithi059.stdout: } 2023-11-24T07:56:21.802 INFO:teuthology.orchestra.run.smithi059.stdout:] 2023-11-24T07:56:21.803 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:21.801+0000 7fc6aa378700 1 -- 172.21.15.59:0/1000712045 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7fc69006aed0 msgr2=0x7fc69006d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:56:21.803 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:21.801+0000 7fc6aa378700 1 --2- 172.21.15.59:0/1000712045 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7fc69006aed0 0x7fc69006d390 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fc69400a1c0 tx=0x7fc69401a040 comp rx=0 tx=0).stop 2023-11-24T07:56:21.803 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:21.801+0000 7fc6aa378700 1 -- 172.21.15.59:0/1000712045 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc6a414fa40 msgr2=0x7fc6a415b890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:56:21.804 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:21.801+0000 7fc6aa378700 1 --2- 172.21.15.59:0/1000712045 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc6a414fa40 0x7fc6a415b890 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fc69c00caa0 tx=0x7fc69c00ce60 comp rx=0 tx=0).stop 2023-11-24T07:56:21.804 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:21.801+0000 7fc6aa378700 1 -- 172.21.15.59:0/1000712045 shutdown_connections 2023-11-24T07:56:21.804 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:21.801+0000 7fc6aa378700 1 --2- 172.21.15.59:0/1000712045 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7fc69006aed0 0x7fc69006d390 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:56:21.805 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:21.801+0000 7fc6aa378700 1 --2- 172.21.15.59:0/1000712045 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc6a414fa40 0x7fc6a415b890 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:56:21.805 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:21.801+0000 7fc6aa378700 1 --2- 172.21.15.59:0/1000712045 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fc6a415a440 0x7fc6a414ec20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:56:21.805 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:21.801+0000 7fc6aa378700 1 --2- 172.21.15.59:0/1000712045 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc6a415eb80 0x7fc6a40cf070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:56:21.805 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:21.801+0000 7fc6aa378700 1 -- 172.21.15.59:0/1000712045 >> 172.21.15.59:0/1000712045 conn(0x7fc6a40b5070 msgr2=0x7fc6a4156190 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:56:21.806 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:21.801+0000 7fc6aa378700 1 -- 172.21.15.59:0/1000712045 shutdown_connections 2023-11-24T07:56:21.806 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:21.801+0000 7fc6aa378700 1 -- 172.21.15.59:0/1000712045 wait complete. 2023-11-24T07:56:21.814 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2023-11-24T07:56:21.814 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2023-11-24T07:56:21.943 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:21.942+0000 7f3a1b35f700 1 Processor -- start 2023-11-24T07:56:21.943 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:21.942+0000 7f3a1b35f700 1 -- start start 2023-11-24T07:56:21.945 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:21.943+0000 7f3a1b35f700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f3a140b11d0 0x7f3a140cfd60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:56:21.945 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:21.943+0000 7f3a1b35f700 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f3a140cecc0 0x7f3a14167c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:56:21.945 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:21.943+0000 7f3a1b35f700 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3a1406f110 con 0x7f3a1406fcb0 2023-11-24T07:56:21.945 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:21.943+0000 7f3a1b35f700 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3a14161a10 con 0x7f3a140b11d0 2023-11-24T07:56:21.945 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:21.943+0000 7f3a1b35f700 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3a140d1010 con 0x7f3a140cecc0 2023-11-24T07:56:21.945 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:21.943+0000 7f3a190fb700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f3a140b11d0 0x7f3a140cfd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:56:21.945 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:21.943+0000 7f3a190fb700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f3a140b11d0 0x7f3a140cfd60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:49626/0 (socket says 172.21.15.59:49626) 2023-11-24T07:56:21.946 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:21.943+0000 7f3a190fb700 1 -- 172.21.15.59:0/3136976685 learned_addr learned my addr 172.21.15.59:0/3136976685 (peer_addr_for_me v2:172.21.15.59:0/0) 2023-11-24T07:56:21.946 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:21.943+0000 7f3a198fc700 1 --2- 172.21.15.59:0/3136976685 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f3a140cecc0 0x7f3a14167c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:56:21.946 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:21.943+0000 7f3a190fb700 1 -- 172.21.15.59:0/3136976685 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f3a140cecc0 msgr2=0x7f3a14167c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T07:56:21.946 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:21.943+0000 7f3a190fb700 1 --2- 172.21.15.59:0/3136976685 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f3a140cecc0 0x7f3a14167c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:56:21.946 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:21.943+0000 7f3a190fb700 1 -- 172.21.15.59:0/3136976685 >> v1:172.21.15.76:6789/0 conn(0x7f3a1406fcb0 legacy=0x7f3a140ce5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:56:21.946 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:21.943+0000 7f3a190fb700 1 -- 172.21.15.59:0/3136976685 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3a140d0400 con 0x7f3a140b11d0 2023-11-24T07:56:21.946 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:21.944+0000 7f3a190fb700 1 --2- 172.21.15.59:0/3136976685 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3a140b11d0 0x7f3a140cfd60 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f3a040098f0 tx=0x7f3a04009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c2756017c61d253f server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:56:21.946 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:21.944+0000 7f3a0bfff700 1 -- 172.21.15.59:0/3136976685 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a0400b8a0 con 0x7f3a140b11d0 2023-11-24T07:56:21.946 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:21.944+0000 7f3a0bfff700 1 -- 172.21.15.59:0/3136976685 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3a0400ba00 con 0x7f3a140b11d0 2023-11-24T07:56:21.946 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:21.944+0000 7f3a0bfff700 1 -- 172.21.15.59:0/3136976685 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a0400bd90 con 0x7f3a140b11d0 2023-11-24T07:56:21.946 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:21.944+0000 7f3a1b35f700 1 -- 172.21.15.59:0/3136976685 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3a140b11d0 msgr2=0x7f3a140cfd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:56:21.947 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:21.944+0000 7f3a1b35f700 1 --2- 172.21.15.59:0/3136976685 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3a140b11d0 0x7f3a140cfd60 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f3a040098f0 tx=0x7f3a04009c00 comp rx=0 tx=0).stop 2023-11-24T07:56:21.947 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:21.944+0000 7f3a1b35f700 1 -- 172.21.15.59:0/3136976685 shutdown_connections 2023-11-24T07:56:21.947 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:21.944+0000 7f3a1b35f700 1 --2- 172.21.15.59:0/3136976685 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f3a140cecc0 0x7f3a14167c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:56:21.947 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:21.944+0000 7f3a1b35f700 1 --2- 172.21.15.59:0/3136976685 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3a140b11d0 0x7f3a140cfd60 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:56:21.947 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:21.944+0000 7f3a1b35f700 1 -- 172.21.15.59:0/3136976685 >> 172.21.15.59:0/3136976685 conn(0x7f3a140b3700 msgr2=0x7f3a140b3b10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:56:21.947 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:21.944+0000 7f3a1b35f700 1 -- 172.21.15.59:0/3136976685 shutdown_connections 2023-11-24T07:56:21.947 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:21.944+0000 7f3a1b35f700 1 -- 172.21.15.59:0/3136976685 wait complete. 2023-11-24T07:56:21.949 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:21.946+0000 7f3a1b35f700 1 Processor -- start 2023-11-24T07:56:21.949 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:21.946+0000 7f3a1b35f700 1 -- start start 2023-11-24T07:56:21.950 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:21.947+0000 7f3a1b35f700 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f3a1406fcb0 0x7f3a1414ab90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:56:21.950 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:21.947+0000 7f3a1b35f700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3a140b11d0 0x7f3a14159bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:56:21.950 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:21.947+0000 7f3a1b35f700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3a140cecc0 0x7f3a1415e080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:56:21.950 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:21.947+0000 7f3a1b35f700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a1415a4a0 con 0x7f3a140b11d0 2023-11-24T07:56:21.950 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:21.947+0000 7f3a1b35f700 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a1415a780 con 0x7f3a140cecc0 2023-11-24T07:56:21.950 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:21.947+0000 7f3a1b35f700 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3a1415aa60 con 0x7f3a1406fcb0 2023-11-24T07:56:21.950 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:21.947+0000 7f3a188fa700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3a140b11d0 0x7f3a14159bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:56:21.950 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:21.947+0000 7f3a188fa700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3a140b11d0 0x7f3a14159bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:49634/0 (socket says 172.21.15.59:49634) 2023-11-24T07:56:21.950 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:21.947+0000 7f3a188fa700 1 -- 172.21.15.59:0/3641129117 learned_addr learned my addr 172.21.15.59:0/3641129117 (peer_addr_for_me v2:172.21.15.59:0/0) 2023-11-24T07:56:21.950 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:21.947+0000 7f3a198fc700 1 --2- 172.21.15.59:0/3641129117 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3a140cecc0 0x7f3a1415e080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:56:21.950 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:21.947+0000 7f3a190fb700 1 --2- 172.21.15.59:0/3641129117 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f3a1406fcb0 0x7f3a1414ab90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:56:21.951 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:21.947+0000 7f3a188fa700 1 -- 172.21.15.59:0/3641129117 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f3a1406fcb0 msgr2=0x7f3a1414ab90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:56:21.951 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:21.947+0000 7f3a188fa700 1 --2- 172.21.15.59:0/3641129117 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f3a1406fcb0 0x7f3a1414ab90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:56:21.951 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:21.947+0000 7f3a188fa700 1 -- 172.21.15.59:0/3641129117 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3a140cecc0 msgr2=0x7f3a1415e080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:56:21.951 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:21.947+0000 7f3a188fa700 1 --2- 172.21.15.59:0/3641129117 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3a140cecc0 0x7f3a1415e080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:56:21.951 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:21.947+0000 7f3a188fa700 1 -- 172.21.15.59:0/3641129117 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3a04009550 con 0x7f3a140b11d0 2023-11-24T07:56:21.951 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:21.947+0000 7f3a198fc700 1 --2- 172.21.15.59:0/3641129117 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3a140cecc0 0x7f3a1415e080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-24T07:56:21.951 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:21.947+0000 7f3a190fb700 1 --2- 172.21.15.59:0/3641129117 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f3a1406fcb0 0x7f3a1414ab90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-24T07:56:21.951 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:21.948+0000 7f3a188fa700 1 --2- 172.21.15.59:0/3641129117 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3a140b11d0 0x7f3a14159bc0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f3a1000c450 tx=0x7f3a1000c810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:56:21.951 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:21.948+0000 7f3a09ffb700 1 -- 172.21.15.59:0/3641129117 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a100052d0 con 0x7f3a140b11d0 2023-11-24T07:56:21.951 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:21.948+0000 7f3a09ffb700 1 -- 172.21.15.59:0/3641129117 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3a10005430 con 0x7f3a140b11d0 2023-11-24T07:56:21.952 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:21.948+0000 7f3a09ffb700 1 -- 172.21.15.59:0/3641129117 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a100038c0 con 0x7f3a140b11d0 2023-11-24T07:56:21.952 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:21.948+0000 7f3a1b35f700 1 -- 172.21.15.59:0/3641129117 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3a1415ad40 con 0x7f3a140b11d0 2023-11-24T07:56:21.952 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:21.948+0000 7f3a1b35f700 1 -- 172.21.15.59:0/3641129117 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3a1415a100 con 0x7f3a140b11d0 2023-11-24T07:56:21.952 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:21.949+0000 7f3a1b35f700 1 -- 172.21.15.59:0/3641129117 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3a1406bc40 con 0x7f3a140b11d0 2023-11-24T07:56:21.953 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:21.949+0000 7f3a09ffb700 1 -- 172.21.15.59:0/3641129117 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f3a10003a20 con 0x7f3a140b11d0 2023-11-24T07:56:21.953 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:21.949+0000 7f3a09ffb700 1 --2- 172.21.15.59:0/3641129117 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f39f806aed0 0x7f39f806d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:56:21.953 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:21.950+0000 7f3a09ffb700 1 -- 172.21.15.59:0/3641129117 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f3a10088550 con 0x7f3a140b11d0 2023-11-24T07:56:21.953 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:21.950+0000 7f3a190fb700 1 --2- 172.21.15.59:0/3641129117 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f39f806aed0 0x7f39f806d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:56:21.953 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:21.950+0000 7f3a190fb700 1 --2- 172.21.15.59:0/3641129117 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f39f806aed0 0x7f39f806d390 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f3a04009c40 tx=0x7f3a04008030 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:56:21.954 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:21.954+0000 7f3a09ffb700 1 -- 172.21.15.59:0/3641129117 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f3a100539e0 con 0x7f3a140b11d0 2023-11-24T07:56:22.156 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:22.155+0000 7f3a1b35f700 1 -- 172.21.15.59:0/3641129117 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7f3a14068670 con 0x7f3a140b11d0 2023-11-24T07:56:22.684 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:22.683+0000 7f3a09ffb700 1 -- 172.21.15.59:0/3641129117 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v17) v1 ==== 138+0+0 (secure 0 0 0) 0x7f3a10057000 con 0x7f3a140b11d0 2023-11-24T07:56:22.688 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:22.687+0000 7f3a1b35f700 1 -- 172.21.15.59:0/3641129117 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f39f806aed0 msgr2=0x7f39f806d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:56:22.688 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:22.687+0000 7f3a1b35f700 1 --2- 172.21.15.59:0/3641129117 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f39f806aed0 0x7f39f806d390 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f3a04009c40 tx=0x7f3a04008030 comp rx=0 tx=0).stop 2023-11-24T07:56:22.688 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:22.687+0000 7f3a1b35f700 1 -- 172.21.15.59:0/3641129117 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3a140b11d0 msgr2=0x7f3a14159bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:56:22.688 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:22.687+0000 7f3a1b35f700 1 --2- 172.21.15.59:0/3641129117 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3a140b11d0 0x7f3a14159bc0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f3a1000c450 tx=0x7f3a1000c810 comp rx=0 tx=0).stop 2023-11-24T07:56:22.688 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:22.687+0000 7f3a1b35f700 1 -- 172.21.15.59:0/3641129117 shutdown_connections 2023-11-24T07:56:22.688 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:22.687+0000 7f3a1b35f700 1 --2- 172.21.15.59:0/3641129117 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f39f806aed0 0x7f39f806d390 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:56:22.688 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:22.687+0000 7f3a1b35f700 1 --2- 172.21.15.59:0/3641129117 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3a140cecc0 0x7f3a1415e080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:56:22.688 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:22.687+0000 7f3a1b35f700 1 --2- 172.21.15.59:0/3641129117 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f3a1406fcb0 0x7f3a1414ab90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:56:22.688 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:22.687+0000 7f3a1b35f700 1 --2- 172.21.15.59:0/3641129117 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3a140b11d0 0x7f3a14159bc0 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:56:22.688 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:22.687+0000 7f3a1b35f700 1 -- 172.21.15.59:0/3641129117 >> 172.21.15.59:0/3641129117 conn(0x7f3a140b3700 msgr2=0x7f3a140d25c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:56:22.688 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:22.687+0000 7f3a1b35f700 1 -- 172.21.15.59:0/3641129117 shutdown_connections 2023-11-24T07:56:22.689 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:22.687+0000 7f3a1b35f700 1 -- 172.21.15.59:0/3641129117 wait complete. 2023-11-24T07:56:22.689 INFO:teuthology.orchestra.run.smithi059.stderr:pool 'cephfs_metadata' created 2023-11-24T07:56:22.706 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2023-11-24T07:56:22.844 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:22.843+0000 7f2e5eeaa700 1 Processor -- start 2023-11-24T07:56:22.844 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:22.843+0000 7f2e5eeaa700 1 -- start start 2023-11-24T07:56:22.846 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:22.844+0000 7f2e5eeaa700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f2e58143ae0 0x7f2e580b51b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:56:22.846 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:22.844+0000 7f2e5eeaa700 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f2e580722c0 0x7f2e5806e510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:56:22.846 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:22.844+0000 7f2e5eeaa700 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2e58143ec0 con 0x7f2e581440f0 2023-11-24T07:56:22.847 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:22.844+0000 7f2e5eeaa700 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2e580b40a0 con 0x7f2e58143ae0 2023-11-24T07:56:22.847 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:22.844+0000 7f2e5eeaa700 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2e580b3ae0 con 0x7f2e580722c0 2023-11-24T07:56:22.847 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:22.844+0000 7f2e5cc46700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f2e58143ae0 0x7f2e580b51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:56:22.847 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:22.844+0000 7f2e5cc46700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f2e58143ae0 0x7f2e580b51b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:49640/0 (socket says 172.21.15.59:49640) 2023-11-24T07:56:22.847 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:22.844+0000 7f2e5cc46700 1 -- 172.21.15.59:0/239674004 learned_addr learned my addr 172.21.15.59:0/239674004 (peer_addr_for_me v2:172.21.15.59:0/0) 2023-11-24T07:56:22.847 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:22.844+0000 7f2e57fff700 1 --1- 172.21.15.59:0/239674004 >> v1:172.21.15.59:6789/0 conn(0x7f2e581440f0 0x7f2e58071bb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:53114/0 (socket says 172.21.15.59:53114) 2023-11-24T07:56:22.847 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:22.844+0000 7f2e5d447700 1 --2- 172.21.15.59:0/239674004 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f2e580722c0 0x7f2e5806e510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:56:22.847 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:22.844+0000 7f2e5cc46700 1 -- 172.21.15.59:0/239674004 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f2e580722c0 msgr2=0x7f2e5806e510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T07:56:22.847 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:22.844+0000 7f2e5cc46700 1 --2- 172.21.15.59:0/239674004 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f2e580722c0 0x7f2e5806e510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:56:22.847 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:22.844+0000 7f2e5cc46700 1 -- 172.21.15.59:0/239674004 >> v1:172.21.15.59:6789/0 conn(0x7f2e581440f0 legacy=0x7f2e58071bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:56:22.847 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:22.844+0000 7f2e5cc46700 1 -- 172.21.15.59:0/239674004 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2e580b3dc0 con 0x7f2e58143ae0 2023-11-24T07:56:22.847 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:22.844+0000 7f2e577fe700 1 -- 172.21.15.59:0/239674004 <== mon.0 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4269920490 0 0) 0x7f2e58143ec0 con 0x7f2e581440f0 2023-11-24T07:56:22.847 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:22.844+0000 7f2e5d447700 1 --2- 172.21.15.59:0/239674004 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f2e580722c0 0x7f2e5806e510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-24T07:56:22.848 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:22.844+0000 7f2e5cc46700 1 --2- 172.21.15.59:0/239674004 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2e58143ae0 0x7f2e580b51b0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f2e4c0098f0 tx=0x7f2e4c009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3b265ebf3b9421d4 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:56:22.848 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:22.844+0000 7f2e577fe700 1 -- 172.21.15.59:0/239674004 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e4c00b8a0 con 0x7f2e58143ae0 2023-11-24T07:56:22.848 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:22.844+0000 7f2e577fe700 1 -- 172.21.15.59:0/239674004 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2e4c00ba00 con 0x7f2e58143ae0 2023-11-24T07:56:22.848 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:22.844+0000 7f2e577fe700 1 -- 172.21.15.59:0/239674004 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e4c00bd90 con 0x7f2e58143ae0 2023-11-24T07:56:22.848 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:22.845+0000 7f2e5eeaa700 1 -- 172.21.15.59:0/239674004 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2e58143ae0 msgr2=0x7f2e580b51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:56:22.848 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:22.845+0000 7f2e5eeaa700 1 --2- 172.21.15.59:0/239674004 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2e58143ae0 0x7f2e580b51b0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f2e4c0098f0 tx=0x7f2e4c009c00 comp rx=0 tx=0).stop 2023-11-24T07:56:22.848 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:22.845+0000 7f2e5eeaa700 1 -- 172.21.15.59:0/239674004 shutdown_connections 2023-11-24T07:56:22.848 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:22.845+0000 7f2e5eeaa700 1 --2- 172.21.15.59:0/239674004 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f2e580722c0 0x7f2e5806e510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:56:22.848 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:22.845+0000 7f2e5eeaa700 1 --2- 172.21.15.59:0/239674004 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2e58143ae0 0x7f2e580b51b0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:56:22.848 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:22.845+0000 7f2e5eeaa700 1 -- 172.21.15.59:0/239674004 >> 172.21.15.59:0/239674004 conn(0x7f2e580b2740 msgr2=0x7f2e580b2b20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:56:22.848 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:22.845+0000 7f2e5eeaa700 1 -- 172.21.15.59:0/239674004 shutdown_connections 2023-11-24T07:56:22.848 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:22.845+0000 7f2e5eeaa700 1 -- 172.21.15.59:0/239674004 wait complete. 2023-11-24T07:56:22.851 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:22.847+0000 7f2e5eeaa700 1 Processor -- start 2023-11-24T07:56:22.851 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:22.847+0000 7f2e5eeaa700 1 -- start start 2023-11-24T07:56:22.851 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:22.848+0000 7f2e5eeaa700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2e580722c0 0x7f2e581423f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:56:22.851 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:22.848+0000 7f2e5eeaa700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2e58143ae0 0x7f2e58142930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:56:22.851 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:22.848+0000 7f2e5eeaa700 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f2e581440f0 0x7f2e58153c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:56:22.852 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:22.848+0000 7f2e5eeaa700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e58151b60 con 0x7f2e580722c0 2023-11-24T07:56:22.852 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:22.848+0000 7f2e5eeaa700 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e58155910 con 0x7f2e58143ae0 2023-11-24T07:56:22.852 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:22.848+0000 7f2e5eeaa700 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2e58154870 con 0x7f2e581440f0 2023-11-24T07:56:22.852 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:22.848+0000 7f2e5cc46700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2e580722c0 0x7f2e581423f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:56:22.852 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:22.848+0000 7f2e5cc46700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2e580722c0 0x7f2e581423f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:49646/0 (socket says 172.21.15.59:49646) 2023-11-24T07:56:22.852 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:22.848+0000 7f2e5cc46700 1 -- 172.21.15.59:0/2702797932 learned_addr learned my addr 172.21.15.59:0/2702797932 (peer_addr_for_me v2:172.21.15.59:0/0) 2023-11-24T07:56:22.852 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:22.848+0000 7f2e57fff700 1 --2- 172.21.15.59:0/2702797932 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2e58143ae0 0x7f2e58142930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:56:22.852 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:22.848+0000 7f2e5d447700 1 --2- 172.21.15.59:0/2702797932 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f2e581440f0 0x7f2e58153c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:56:22.852 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:22.848+0000 7f2e5cc46700 1 -- 172.21.15.59:0/2702797932 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f2e581440f0 msgr2=0x7f2e58153c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:56:22.852 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:22.848+0000 7f2e5cc46700 1 --2- 172.21.15.59:0/2702797932 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f2e581440f0 0x7f2e58153c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:56:22.852 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:22.848+0000 7f2e5cc46700 1 -- 172.21.15.59:0/2702797932 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2e58143ae0 msgr2=0x7f2e58142930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:56:22.853 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:22.848+0000 7f2e5cc46700 1 --2- 172.21.15.59:0/2702797932 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2e58143ae0 0x7f2e58142930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:56:22.853 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:22.848+0000 7f2e5cc46700 1 -- 172.21.15.59:0/2702797932 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2e4c009550 con 0x7f2e580722c0 2023-11-24T07:56:22.853 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:22.848+0000 7f2e57fff700 1 --2- 172.21.15.59:0/2702797932 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2e58143ae0 0x7f2e58142930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T07:56:22.853 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:22.848+0000 7f2e5cc46700 1 --2- 172.21.15.59:0/2702797932 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2e580722c0 0x7f2e581423f0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f2e4c009c40 tx=0x7f2e4c004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:56:22.853 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:22.848+0000 7f2e557fa700 1 -- 172.21.15.59:0/2702797932 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e4c004590 con 0x7f2e580722c0 2023-11-24T07:56:22.853 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:22.848+0000 7f2e557fa700 1 -- 172.21.15.59:0/2702797932 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2e4c0036a0 con 0x7f2e580722c0 2023-11-24T07:56:22.853 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:22.848+0000 7f2e557fa700 1 -- 172.21.15.59:0/2702797932 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e4c003a30 con 0x7f2e580722c0 2023-11-24T07:56:22.853 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:22.848+0000 7f2e5eeaa700 1 -- 172.21.15.59:0/2702797932 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2e58154b50 con 0x7f2e580722c0 2023-11-24T07:56:22.854 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:22.849+0000 7f2e5eeaa700 1 -- 172.21.15.59:0/2702797932 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2e58155110 con 0x7f2e580722c0 2023-11-24T07:56:22.854 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:22.849+0000 7f2e5eeaa700 1 -- 172.21.15.59:0/2702797932 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2e58075480 con 0x7f2e580722c0 2023-11-24T07:56:22.854 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:22.850+0000 7f2e557fa700 1 -- 172.21.15.59:0/2702797932 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f2e4c003b90 con 0x7f2e580722c0 2023-11-24T07:56:22.854 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:22.850+0000 7f2e557fa700 1 --2- 172.21.15.59:0/2702797932 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f2e3c06aed0 0x7f2e3c06d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:56:22.854 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:22.850+0000 7f2e557fa700 1 -- 172.21.15.59:0/2702797932 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(17..17 src has 1..17) v4 ==== 5227+0+0 (secure 0 0 0) 0x7f2e4c08c510 con 0x7f2e580722c0 2023-11-24T07:56:22.854 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:22.851+0000 7f2e57fff700 1 --2- 172.21.15.59:0/2702797932 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f2e3c06aed0 0x7f2e3c06d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:56:22.854 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:22.851+0000 7f2e57fff700 1 --2- 172.21.15.59:0/2702797932 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f2e3c06aed0 0x7f2e3c06d390 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f2e44007fd0 tx=0x7f2e44009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:56:22.856 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:22.855+0000 7f2e557fa700 1 -- 172.21.15.59:0/2702797932 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f2e4c057830 con 0x7f2e580722c0 2023-11-24T07:56:23.055 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:23.054+0000 7f2e5eeaa700 1 -- 172.21.15.59:0/2702797932 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7f2e58072e60 con 0x7f2e580722c0 2023-11-24T07:56:23.686 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:23.685+0000 7f2e557fa700 1 -- 172.21.15.59:0/2702797932 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v18) v1 ==== 170+0+0 (secure 0 0 0) 0x7f2e4c0150d0 con 0x7f2e580722c0 2023-11-24T07:56:23.694 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:23.689+0000 7f2e5eeaa700 1 -- 172.21.15.59:0/2702797932 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f2e3c06aed0 msgr2=0x7f2e3c06d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:56:23.694 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:23.689+0000 7f2e5eeaa700 1 --2- 172.21.15.59:0/2702797932 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f2e3c06aed0 0x7f2e3c06d390 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f2e44007fd0 tx=0x7f2e44009040 comp rx=0 tx=0).stop 2023-11-24T07:56:23.695 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:23.689+0000 7f2e5eeaa700 1 -- 172.21.15.59:0/2702797932 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2e580722c0 msgr2=0x7f2e581423f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:56:23.695 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:23.689+0000 7f2e5eeaa700 1 --2- 172.21.15.59:0/2702797932 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2e580722c0 0x7f2e581423f0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f2e4c009c40 tx=0x7f2e4c004750 comp rx=0 tx=0).stop 2023-11-24T07:56:23.695 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:23.689+0000 7f2e5eeaa700 1 -- 172.21.15.59:0/2702797932 shutdown_connections 2023-11-24T07:56:23.695 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:23.689+0000 7f2e5eeaa700 1 --2- 172.21.15.59:0/2702797932 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f2e3c06aed0 0x7f2e3c06d390 secure :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f2e44007fd0 tx=0x7f2e44009040 comp rx=0 tx=0).stop 2023-11-24T07:56:23.695 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:23.689+0000 7f2e5eeaa700 1 --2- 172.21.15.59:0/2702797932 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f2e581440f0 0x7f2e58153c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:56:23.695 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:23.689+0000 7f2e5eeaa700 1 --2- 172.21.15.59:0/2702797932 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2e580722c0 0x7f2e581423f0 secure :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f2e4c009c40 tx=0x7f2e4c004750 comp rx=0 tx=0).stop 2023-11-24T07:56:23.696 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:23.689+0000 7f2e5eeaa700 1 --2- 172.21.15.59:0/2702797932 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2e58143ae0 0x7f2e58142930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:56:23.696 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:23.689+0000 7f2e5eeaa700 1 -- 172.21.15.59:0/2702797932 >> 172.21.15.59:0/2702797932 conn(0x7f2e580b2740 msgr2=0x7f2e58159610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:56:23.696 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:23.690+0000 7f2e5eeaa700 1 -- 172.21.15.59:0/2702797932 shutdown_connections 2023-11-24T07:56:23.696 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:23.690+0000 7f2e5eeaa700 1 -- 172.21.15.59:0/2702797932 wait complete. 2023-11-24T07:56:23.703 INFO:teuthology.orchestra.run.smithi059.stderr:pool 'cephfs_data' created 2023-11-24T07:56:23.716 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2023-11-24T07:56:23.847 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:23.844+0000 7f8909e8c700 1 Processor -- start 2023-11-24T07:56:23.848 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:23.844+0000 7f8909e8c700 1 -- start start 2023-11-24T07:56:23.848 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:23.845+0000 7f8909e8c700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f89041440f0 0x7f89040b51b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:56:23.848 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:23.845+0000 7f8909e8c700 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8904143ec0 con 0x7f89040722c0 2023-11-24T07:56:23.848 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:23.845+0000 7f8909e8c700 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f89040b3ae0 con 0x7f8904143ae0 2023-11-24T07:56:23.848 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:23.845+0000 7f8909e8c700 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f89040b40a0 con 0x7f89041440f0 2023-11-24T07:56:23.848 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:23.845+0000 7f8903fff700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f89041440f0 0x7f89040b51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:56:23.849 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:23.845+0000 7f8902ffd700 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7f89040722c0 0x7f890406e510 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:53130/0 (socket says 172.21.15.59:53130) 2023-11-24T07:56:23.849 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:23.845+0000 7f8902ffd700 1 -- 172.21.15.59:0/2366129456 learned_addr learned my addr 172.21.15.59:0/2366129456 (peer_addr_for_me v1:172.21.15.59:0/0) 2023-11-24T07:56:23.849 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:23.845+0000 7f8903fff700 1 -- 172.21.15.59:0/2366129456 >> v1:172.21.15.76:6789/0 conn(0x7f8904143ae0 legacy=0x7f8904071bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:56:23.849 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:23.845+0000 7f8903fff700 1 -- 172.21.15.59:0/2366129456 >> v1:172.21.15.59:6789/0 conn(0x7f89040722c0 legacy=0x7f890406e510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:56:23.849 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:23.845+0000 7f8903fff700 1 -- 172.21.15.59:0/2366129456 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f89040b3dc0 con 0x7f89041440f0 2023-11-24T07:56:23.849 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:23.845+0000 7f89027fc700 1 -- 172.21.15.59:0/2366129456 <== mon.0 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1740181729 0 0) 0x7f8904143ec0 con 0x7f89040722c0 2023-11-24T07:56:23.850 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:23.845+0000 7f8903fff700 1 --2- 172.21.15.59:0/2366129456 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f89041440f0 0x7f89040b51b0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f88fc009d90 tx=0x7f88fc00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=13e6549c9de26ac server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:56:23.850 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:23.846+0000 7f89027fc700 1 -- 172.21.15.59:0/2366129456 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88fc00e070 con 0x7f89041440f0 2023-11-24T07:56:23.850 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:23.846+0000 7f89027fc700 1 -- 172.21.15.59:0/2366129456 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f88fc00bbd0 con 0x7f89041440f0 2023-11-24T07:56:23.850 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:23.846+0000 7f89027fc700 1 -- 172.21.15.59:0/2366129456 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88fc012430 con 0x7f89041440f0 2023-11-24T07:56:23.853 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:23.846+0000 7f8909e8c700 1 -- 172.21.15.59:0/2366129456 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f89041440f0 msgr2=0x7f89040b51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:56:23.853 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:23.846+0000 7f8909e8c700 1 --2- 172.21.15.59:0/2366129456 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f89041440f0 0x7f89040b51b0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f88fc009d90 tx=0x7f88fc00b330 comp rx=0 tx=0).stop 2023-11-24T07:56:23.854 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:23.846+0000 7f8909e8c700 1 -- 172.21.15.59:0/2366129456 shutdown_connections 2023-11-24T07:56:23.854 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:23.846+0000 7f8909e8c700 1 --2- 172.21.15.59:0/2366129456 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f89041440f0 0x7f89040b51b0 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:56:23.854 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:23.846+0000 7f8909e8c700 1 -- 172.21.15.59:0/2366129456 >> 172.21.15.59:0/2366129456 conn(0x7f89040b2740 msgr2=0x7f89040b2b20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:56:23.854 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:23.846+0000 7f8909e8c700 1 -- 172.21.15.59:0/2366129456 shutdown_connections 2023-11-24T07:56:23.854 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:23.846+0000 7f8909e8c700 1 -- 172.21.15.59:0/2366129456 wait complete. 2023-11-24T07:56:23.854 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:23.847+0000 7f8909e8c700 1 Processor -- start 2023-11-24T07:56:23.854 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:23.848+0000 7f8909e8c700 1 -- start start 2023-11-24T07:56:23.855 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:23.848+0000 7f8909e8c700 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f89040722c0 0x7f890415db60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:56:23.855 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:23.848+0000 7f8909e8c700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8904143ae0 0x7f8904155780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:56:23.855 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:23.848+0000 7f8909e8c700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f89041440f0 0x7f890415c340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:56:23.855 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:23.848+0000 7f8909e8c700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89040c9dc0 con 0x7f8904143ae0 2023-11-24T07:56:23.855 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:23.848+0000 7f8909e8c700 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8904155cc0 con 0x7f89041440f0 2023-11-24T07:56:23.855 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:23.848+0000 7f8909e8c700 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f890415cb90 con 0x7f89040722c0 2023-11-24T07:56:23.855 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:23.848+0000 7f8902ffd700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8904143ae0 0x7f8904155780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:56:23.856 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:23.848+0000 7f8902ffd700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8904143ae0 0x7f8904155780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:49662/0 (socket says 172.21.15.59:49662) 2023-11-24T07:56:23.856 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:23.848+0000 7f8902ffd700 1 -- 172.21.15.59:0/1480094543 learned_addr learned my addr 172.21.15.59:0/1480094543 (peer_addr_for_me v2:172.21.15.59:0/0) 2023-11-24T07:56:23.856 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:23.849+0000 7f8903fff700 1 --2- 172.21.15.59:0/1480094543 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f89041440f0 0x7f890415c340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:56:23.856 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:23.849+0000 7f89037fe700 1 --2- 172.21.15.59:0/1480094543 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f89040722c0 0x7f890415db60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:56:23.856 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:23.849+0000 7f8902ffd700 1 -- 172.21.15.59:0/1480094543 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f89040722c0 msgr2=0x7f890415db60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:56:23.856 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:23.849+0000 7f8902ffd700 1 --2- 172.21.15.59:0/1480094543 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f89040722c0 0x7f890415db60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:56:23.857 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:23.849+0000 7f8902ffd700 1 -- 172.21.15.59:0/1480094543 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f89041440f0 msgr2=0x7f890415c340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:56:23.857 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:23.849+0000 7f8902ffd700 1 --2- 172.21.15.59:0/1480094543 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f89041440f0 0x7f890415c340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:56:23.857 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:23.849+0000 7f8902ffd700 1 -- 172.21.15.59:0/1480094543 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f88fc009a70 con 0x7f8904143ae0 2023-11-24T07:56:23.857 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:23.849+0000 7f8903fff700 1 --2- 172.21.15.59:0/1480094543 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f89041440f0 0x7f890415c340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T07:56:23.857 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:23.849+0000 7f8902ffd700 1 --2- 172.21.15.59:0/1480094543 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8904143ae0 0x7f8904155780 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f88ec00c520 tx=0x7f88ec00c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:56:23.857 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:23.849+0000 7f88fbfff700 1 -- 172.21.15.59:0/1480094543 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88ec004c80 con 0x7f8904143ae0 2023-11-24T07:56:23.857 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:23.849+0000 7f88fbfff700 1 -- 172.21.15.59:0/1480094543 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f88ec004e00 con 0x7f8904143ae0 2023-11-24T07:56:23.857 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:23.849+0000 7f8909e8c700 1 -- 172.21.15.59:0/1480094543 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f890415ce70 con 0x7f8904143ae0 2023-11-24T07:56:23.858 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:23.849+0000 7f88fbfff700 1 -- 172.21.15.59:0/1480094543 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88ec007310 con 0x7f8904143ae0 2023-11-24T07:56:23.858 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:23.849+0000 7f8909e8c700 1 -- 172.21.15.59:0/1480094543 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f89040c9fd0 con 0x7f8904143ae0 2023-11-24T07:56:23.858 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:23.850+0000 7f8909e8c700 1 -- 172.21.15.59:0/1480094543 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8904075480 con 0x7f8904143ae0 2023-11-24T07:56:23.858 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:23.851+0000 7f88fbfff700 1 -- 172.21.15.59:0/1480094543 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f88ec007470 con 0x7f8904143ae0 2023-11-24T07:56:23.858 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:23.851+0000 7f88fbfff700 1 --2- 172.21.15.59:0/1480094543 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f88e406aed0 0x7f88e406d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:56:23.858 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:23.851+0000 7f88fbfff700 1 -- 172.21.15.59:0/1480094543 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(18..18 src has 1..18) v4 ==== 5609+0+0 (secure 0 0 0) 0x7f88ec0892f0 con 0x7f8904143ae0 2023-11-24T07:56:23.858 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:23.852+0000 7f89037fe700 1 --2- 172.21.15.59:0/1480094543 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f88e406aed0 0x7f88e406d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:56:23.859 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:23.852+0000 7f89037fe700 1 --2- 172.21.15.59:0/1480094543 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f88e406aed0 0x7f88e406d390 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f88f4000f60 tx=0x7f88f4009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:56:23.859 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:23.856+0000 7f88fbfff700 1 -- 172.21.15.59:0/1480094543 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f88ec054400 con 0x7f8904143ae0 2023-11-24T07:56:24.104 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:24.102+0000 7f8909e8c700 1 -- 172.21.15.59:0/1480094543 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7f89040b5d20 con 0x7f8904143ae0 2023-11-24T07:56:24.723 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:24.722+0000 7f88fbfff700 1 -- 172.21.15.59:0/1480094543 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 2 and data pool 3 v4) v1 ==== 172+0+0 (secure 0 0 0) 0x7f88ec057a20 con 0x7f8904143ae0 2023-11-24T07:56:24.733 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:24.732+0000 7f8909e8c700 1 -- 172.21.15.59:0/1480094543 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f88e406aed0 msgr2=0x7f88e406d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:56:24.733 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:24.732+0000 7f8909e8c700 1 --2- 172.21.15.59:0/1480094543 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f88e406aed0 0x7f88e406d390 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f88f4000f60 tx=0x7f88f4009040 comp rx=0 tx=0).stop 2023-11-24T07:56:24.733 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:24.732+0000 7f8909e8c700 1 -- 172.21.15.59:0/1480094543 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8904143ae0 msgr2=0x7f8904155780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:56:24.734 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:24.732+0000 7f8909e8c700 1 --2- 172.21.15.59:0/1480094543 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8904143ae0 0x7f8904155780 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f88ec00c520 tx=0x7f88ec00c8e0 comp rx=0 tx=0).stop 2023-11-24T07:56:24.737 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:24.735+0000 7f8909e8c700 1 -- 172.21.15.59:0/1480094543 shutdown_connections 2023-11-24T07:56:24.737 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:24.735+0000 7f8909e8c700 1 --2- 172.21.15.59:0/1480094543 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f88e406aed0 0x7f88e406d390 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:56:24.737 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:24.735+0000 7f8909e8c700 1 --2- 172.21.15.59:0/1480094543 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f89041440f0 0x7f890415c340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:56:24.737 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:24.735+0000 7f8909e8c700 1 --2- 172.21.15.59:0/1480094543 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f89040722c0 0x7f890415db60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:56:24.737 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:24.735+0000 7f8909e8c700 1 --2- 172.21.15.59:0/1480094543 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8904143ae0 0x7f8904155780 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:56:24.737 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:24.735+0000 7f8909e8c700 1 -- 172.21.15.59:0/1480094543 >> 172.21.15.59:0/1480094543 conn(0x7f89040b2740 msgr2=0x7f890406e410 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:56:24.738 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:24.736+0000 7f8909e8c700 1 -- 172.21.15.59:0/1480094543 shutdown_connections 2023-11-24T07:56:24.738 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:24.736+0000 7f8909e8c700 1 -- 172.21.15.59:0/1480094543 wait complete. 2023-11-24T07:56:24.740 INFO:teuthology.orchestra.run.smithi059.stderr:new fs with metadata pool 2 and data pool 3 2023-11-24T07:56:24.756 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-24T07:56:24.886 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:24.884+0000 7f5454ee5700 1 Processor -- start 2023-11-24T07:56:24.886 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:24.884+0000 7f5454ee5700 1 -- start start 2023-11-24T07:56:24.888 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:24.885+0000 7f5454ee5700 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f5450143ae0 0x7f54500b51b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:56:24.889 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:24.885+0000 7f5454ee5700 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f54501440f0 0x7f5450071bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:56:24.889 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:24.885+0000 7f5454ee5700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f54500720f0 0x7f545015e8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:56:24.889 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:24.885+0000 7f5454ee5700 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f54500b40a0 con 0x7f54500720f0 2023-11-24T07:56:24.889 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:24.885+0000 7f5454ee5700 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5450072580 con 0x7f54501440f0 2023-11-24T07:56:24.889 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:24.885+0000 7f5454ee5700 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5450143ec0 con 0x7f5450143ae0 2023-11-24T07:56:24.889 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:24.885+0000 7f544ed9d700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f54500720f0 0x7f545015e8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:56:24.889 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:24.885+0000 7f544ed9d700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f54500720f0 0x7f545015e8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:49672/0 (socket says 172.21.15.59:49672) 2023-11-24T07:56:24.889 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:24.885+0000 7f544ed9d700 1 -- 172.21.15.59:0/756010131 learned_addr learned my addr 172.21.15.59:0/756010131 (peer_addr_for_me v2:172.21.15.59:0/0) 2023-11-24T07:56:24.889 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:24.886+0000 7f544dd9b700 1 --2- 172.21.15.59:0/756010131 >> v2:172.21.15.76:3300/0 conn(0x7f54501440f0 0x7f5450071bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:56:24.889 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:24.886+0000 7f544e59c700 1 --2- 172.21.15.59:0/756010131 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f5450143ae0 0x7f54500b51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:56:24.889 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:24.886+0000 7f544ed9d700 1 -- 172.21.15.59:0/756010131 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f5450143ae0 msgr2=0x7f54500b51b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T07:56:24.890 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:24.886+0000 7f544ed9d700 1 --2- 172.21.15.59:0/756010131 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f5450143ae0 0x7f54500b51b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:56:24.890 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:24.886+0000 7f544ed9d700 1 -- 172.21.15.59:0/756010131 >> v2:172.21.15.76:3300/0 conn(0x7f54501440f0 msgr2=0x7f5450071bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T07:56:24.890 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:24.886+0000 7f544ed9d700 1 --2- 172.21.15.59:0/756010131 >> v2:172.21.15.76:3300/0 conn(0x7f54501440f0 0x7f5450071bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:56:24.890 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:24.886+0000 7f544ed9d700 1 -- 172.21.15.59:0/756010131 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f54500b3ae0 con 0x7f54500720f0 2023-11-24T07:56:24.890 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:24.886+0000 7f544e59c700 1 --2- 172.21.15.59:0/756010131 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f5450143ae0 0x7f54500b51b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T07:56:24.890 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:24.886+0000 7f544ed9d700 1 --2- 172.21.15.59:0/756010131 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f54500720f0 0x7f545015e8b0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f545006d070 tx=0x7f544800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=29b4fda72c1e755a server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:56:24.890 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:24.886+0000 7f544d59a700 1 -- 172.21.15.59:0/756010131 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f544800e070 con 0x7f54500720f0 2023-11-24T07:56:24.890 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:24.886+0000 7f544d59a700 1 -- 172.21.15.59:0/756010131 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f544800b9c0 con 0x7f54500720f0 2023-11-24T07:56:24.890 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:24.886+0000 7f544d59a700 1 -- 172.21.15.59:0/756010131 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f544800bd70 con 0x7f54500720f0 2023-11-24T07:56:24.890 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:24.887+0000 7f5454ee5700 1 -- 172.21.15.59:0/756010131 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f54500720f0 msgr2=0x7f545015e8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:56:24.890 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:24.887+0000 7f5454ee5700 1 --2- 172.21.15.59:0/756010131 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f54500720f0 0x7f545015e8b0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f545006d070 tx=0x7f544800b330 comp rx=0 tx=0).stop 2023-11-24T07:56:24.891 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:24.887+0000 7f5454ee5700 1 -- 172.21.15.59:0/756010131 shutdown_connections 2023-11-24T07:56:24.891 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:24.887+0000 7f5454ee5700 1 --2- 172.21.15.59:0/756010131 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f54500720f0 0x7f545015e8b0 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:56:24.891 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:24.887+0000 7f5454ee5700 1 --2- 172.21.15.59:0/756010131 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f5450143ae0 0x7f54500b51b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:56:24.891 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:24.887+0000 7f5454ee5700 1 --2- 172.21.15.59:0/756010131 >> v2:172.21.15.76:3300/0 conn(0x7f54501440f0 0x7f5450071bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:56:24.891 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:24.887+0000 7f5454ee5700 1 -- 172.21.15.59:0/756010131 >> 172.21.15.59:0/756010131 conn(0x7f54500b2740 msgr2=0x7f54500b2b20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:56:24.891 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:24.887+0000 7f5454ee5700 1 -- 172.21.15.59:0/756010131 shutdown_connections 2023-11-24T07:56:24.891 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:24.887+0000 7f5454ee5700 1 -- 172.21.15.59:0/756010131 wait complete. 2023-11-24T07:56:24.894 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:24.888+0000 7f5454ee5700 1 Processor -- start 2023-11-24T07:56:24.894 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:24.888+0000 7f5454ee5700 1 -- start start 2023-11-24T07:56:24.894 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:24.889+0000 7f5454ee5700 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f54500720f0 0x7f545015d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:56:24.894 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:24.889+0000 7f5454ee5700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5450143ae0 0x7f5450138940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:56:24.894 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:24.889+0000 7f5454ee5700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f54501440f0 0x7f545015b570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:56:24.894 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:24.889+0000 7f5454ee5700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5450139240 con 0x7f54501440f0 2023-11-24T07:56:24.894 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:24.889+0000 7f5454ee5700 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54501395a0 con 0x7f5450143ae0 2023-11-24T07:56:24.895 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:24.889+0000 7f5454ee5700 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5450139880 con 0x7f54500720f0 2023-11-24T07:56:24.895 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:24.889+0000 7f544ed9d700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f54501440f0 0x7f545015b570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:56:24.895 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:24.890+0000 7f544ed9d700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f54501440f0 0x7f545015b570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:49678/0 (socket says 172.21.15.59:49678) 2023-11-24T07:56:24.895 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:24.890+0000 7f544ed9d700 1 -- 172.21.15.59:0/1333178855 learned_addr learned my addr 172.21.15.59:0/1333178855 (peer_addr_for_me v2:172.21.15.59:0/0) 2023-11-24T07:56:24.895 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:24.890+0000 7f544dd9b700 1 --2- 172.21.15.59:0/1333178855 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5450143ae0 0x7f5450138940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:56:24.895 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:24.890+0000 7f544e59c700 1 --2- 172.21.15.59:0/1333178855 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f54500720f0 0x7f545015d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:56:24.895 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:24.890+0000 7f544ed9d700 1 -- 172.21.15.59:0/1333178855 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f54500720f0 msgr2=0x7f545015d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:56:24.895 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:24.890+0000 7f544ed9d700 1 --2- 172.21.15.59:0/1333178855 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f54500720f0 0x7f545015d5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:56:24.895 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:24.890+0000 7f544ed9d700 1 -- 172.21.15.59:0/1333178855 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5450143ae0 msgr2=0x7f5450138940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:56:24.895 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:24.890+0000 7f544ed9d700 1 --2- 172.21.15.59:0/1333178855 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5450143ae0 0x7f5450138940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:56:24.895 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:24.890+0000 7f544ed9d700 1 -- 172.21.15.59:0/1333178855 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5448009b30 con 0x7f54501440f0 2023-11-24T07:56:24.895 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:24.890+0000 7f544dd9b700 1 --2- 172.21.15.59:0/1333178855 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5450143ae0 0x7f5450138940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-24T07:56:24.896 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:24.890+0000 7f544e59c700 1 --2- 172.21.15.59:0/1333178855 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f54500720f0 0x7f545015d5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-24T07:56:24.896 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:24.890+0000 7f544ed9d700 1 --2- 172.21.15.59:0/1333178855 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f54501440f0 0x7f545015b570 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f54500d1bd0 tx=0x7f5448004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:56:24.896 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:24.890+0000 7f543effd700 1 -- 172.21.15.59:0/1333178855 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5448004250 con 0x7f54501440f0 2023-11-24T07:56:24.896 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:24.890+0000 7f543effd700 1 -- 172.21.15.59:0/1333178855 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f54480043d0 con 0x7f54501440f0 2023-11-24T07:56:24.896 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:24.890+0000 7f5454ee5700 1 -- 172.21.15.59:0/1333178855 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5450139b60 con 0x7f54501440f0 2023-11-24T07:56:24.896 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:24.890+0000 7f543effd700 1 -- 172.21.15.59:0/1333178855 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5448004780 con 0x7f54501440f0 2023-11-24T07:56:24.896 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:24.890+0000 7f5454ee5700 1 -- 172.21.15.59:0/1333178855 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f545015db10 con 0x7f54501440f0 2023-11-24T07:56:24.896 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:24.891+0000 7f5454ee5700 1 -- 172.21.15.59:0/1333178855 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5450068670 con 0x7f54501440f0 2023-11-24T07:56:24.896 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:24.892+0000 7f543effd700 1 -- 172.21.15.59:0/1333178855 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f544801b430 con 0x7f54501440f0 2023-11-24T07:56:24.896 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:24.892+0000 7f543effd700 1 --2- 172.21.15.59:0/1333178855 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f543006aed0 0x7f543006d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:56:24.896 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:24.892+0000 7f543effd700 1 -- 172.21.15.59:0/1333178855 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(19..19 src has 1..19) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f544808aaa0 con 0x7f54501440f0 2023-11-24T07:56:24.896 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:24.892+0000 7f544e59c700 1 --2- 172.21.15.59:0/1333178855 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f543006aed0 0x7f543006d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:56:24.897 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:24.893+0000 7f544e59c700 1 --2- 172.21.15.59:0/1333178855 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f543006aed0 0x7f543006d390 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f544400aa20 tx=0x7f5444008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:56:24.898 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:24.897+0000 7f543effd700 1 -- 172.21.15.59:0/1333178855 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f5448055be0 con 0x7f54501440f0 2023-11-24T07:56:25.098 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:25.096+0000 7f5454ee5700 1 -- 172.21.15.59:0/1333178855 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f54500b5d20 con 0x7f54501440f0 2023-11-24T07:56:25.099 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:25.097+0000 7f543effd700 1 -- 172.21.15.59:0/1333178855 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v19) v1 ==== 74+0+16450 (secure 0 0 0) 0x7f544801f020 con 0x7f54501440f0 2023-11-24T07:56:25.099 INFO:teuthology.orchestra.run.smithi059.stdout: 2023-11-24T07:56:25.099 INFO:teuthology.orchestra.run.smithi059.stdout:{"epoch":19,"fsid":"9f0e8741-b8f9-48d5-a2c3-09f45903303d","created":"2023-11-24T07:56:06.552729+0000","modified":"2023-11-24T07:56:24.103977+0000","last_up_change":"2023-11-24T07:56:17.372352+0000","last_in_change":"2023-11-24T07:56:09.073250+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-24T07:56:18.352823+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-11-24T07:56:22.185140+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-11-24T07:56:23.084326+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"6d8f9e9b-1782-4e41-928b-a39f2a8ebc06","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6800","nonce":3373878772},{"type":"v1","addr":"172.21.15.59:6801","nonce":3373878772}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6802","nonce":3373878772},{"type":"v1","addr":"172.21.15.59:6803","nonce":3373878772}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6806","nonce":3373878772},{"type":"v1","addr":"172.21.15.59:6807","nonce":3373878772}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6804","nonce":3373878772},{"type":"v1","addr":"172.21.15.59:6805","nonce":3373878772}]},"public_addr":"172.21.15.59:6801/3373878772","cluster_addr":"172.21.15.59:6803/3373878772","heartbeat_back_addr":"172.21.15.59:6807/3373878772","heartbeat_front_addr":"172.21.15.59:6805/3373878772","state":["exists","up"]},{"osd":1,"uuid":"e8d882aa-52e1-4c04-9e1e-c44ef9204a5f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6824","nonce":648258563},{"type":"v1","addr":"172.21.15.59:6825","nonce":648258563}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6826","nonce":648258563},{"type":"v1","addr":"172.21.15.59:6827","nonce":648258563}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6830","nonce":648258563},{"type":"v1","addr":"172.21.15.59:6831","nonce":648258563}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6828","nonce":648258563},{"type":"v1","addr":"172.21.15.59:6829","nonce":648258563}]},"public_addr":"172.21.15.59:6825/648258563","cluster_addr":"172.21.15.59:6827/648258563","heartbeat_back_addr":"172.21.15.59:6831/648258563","heartbeat_front_addr":"172.21.15.59:6829/648258563","state":["exists","up"]},{"osd":2,"uuid":"69eebfa0-695b-4b21-90b9-8a3537168a01","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6808","nonce":1840430654},{"type":"v1","addr":"172.21.15.59:6809","nonce":1840430654}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6810","nonce":1840430654},{"type":"v1","addr":"172.21.15.59:6811","nonce":1840430654}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6814","nonce":1840430654},{"type":"v1","addr":"172.21.15.59:6815","nonce":1840430654}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6812","nonce":1840430654},{"type":"v1","addr":"172.21.15.59:6813","nonce":1840430654}]},"public_addr":"172.21.15.59:6809/1840430654","cluster_addr":"172.21.15.59:6811/1840430654","heartbeat_back_addr":"172.21.15.59:6815/1840430654","heartbeat_front_addr":"172.21.15.59:6813/1840430654","state":["exists","up"]},{"osd":3,"uuid":"96fd77fc-f911-46ff-9445-6f0be2daa8f4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6816","nonce":2504025850},{"type":"v1","addr":"172.21.15.59:6817","nonce":2504025850}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6818","nonce":2504025850},{"type":"v1","addr":"172.21.15.59:6819","nonce":2504025850}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6822","nonce":2504025850},{"type":"v1","addr":"172.21.15.59:6823","nonce":2504025850}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6820","nonce":2504025850},{"type":"v1","addr":"172.21.15.59:6821","nonce":2504025850}]},"public_addr":"172.21.15.59:6817/2504025850","cluster_addr":"172.21.15.59:6819/2504025850","heartbeat_back_addr":"172.21.15.59:6823/2504025850","heartbeat_front_addr":"172.21.15.59:6821/2504025850","state":["exists","up"]},{"osd":4,"uuid":"7ac29fca-b5b2-4c95-9d69-2f164cbbc708","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6816","nonce":2382463431},{"type":"v1","addr":"172.21.15.76:6818","nonce":2382463431}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6820","nonce":2382463431},{"type":"v1","addr":"172.21.15.76:6822","nonce":2382463431}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6828","nonce":2382463431},{"type":"v1","addr":"172.21.15.76:6830","nonce":2382463431}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6824","nonce":2382463431},{"type":"v1","addr":"172.21.15.76:6826","nonce":2382463431}]},"public_addr":"172.21.15.76:6818/2382463431","cluster_addr":"172.21.15.76:6822/2382463431","heartbeat_back_addr":"172.21.15.76:6830/2382463431","heartbeat_front_addr":"172.21.15.76:6826/2382463431","state":["exists","up"]},{"osd":5,"uuid":"485bf711-3851-4238-a4f4-51d52e0a9c58","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6800","nonce":329334308},{"type":"v1","addr":"172.21.15.76:6802","nonce":329334308}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6805","nonce":329334308},{"type":"v1","addr":"172.21.15.76:6807","nonce":329334308}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6814","nonce":329334308},{"type":"v1","addr":"172.21.15.76:6815","nonce":329334308}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6810","nonce":329334308},{"type":"v1","addr":"172.21.15.76:6812","nonce":329334308}]},"public_addr":"172.21.15.76:6802/329334308","cluster_addr":"172.21.15.76:6807/329334308","heartbeat_back_addr":"172.21.15.76:6815/329334308","heartbeat_front_addr":"172.21.15.76:6812/329334308","state":["exists","up"]},{"osd":6,"uuid":"72b3f2fc-1e4a-4e32-968d-003647a0286f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6801","nonce":3946579410},{"type":"v1","addr":"172.21.15.76:6803","nonce":3946579410}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6804","nonce":3946579410},{"type":"v1","addr":"172.21.15.76:6806","nonce":3946579410}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6811","nonce":3946579410},{"type":"v1","addr":"172.21.15.76:6813","nonce":3946579410}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6808","nonce":3946579410},{"type":"v1","addr":"172.21.15.76:6809","nonce":3946579410}]},"public_addr":"172.21.15.76:6803/3946579410","cluster_addr":"172.21.15.76:6806/3946579410","heartbeat_back_addr":"172.21.15.76:6813/3946579410","heartbeat_front_addr":"172.21.15.76:6809/3946579410","state":["exists","up"]},{"osd":7,"uuid":"1ddceb90-6c5d-4d1a-ad56-99cb877f8c70","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6817","nonce":841761538},{"type":"v1","addr":"172.21.15.76:6819","nonce":841761538}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6821","nonce":841761538},{"type":"v1","addr":"172.21.15.76:6823","nonce":841761538}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6829","nonce":841761538},{"type":"v1","addr":"172.21.15.76:6831","nonce":841761538}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6825","nonce":841761538},{"type":"v1","addr":"172.21.15.76:6827","nonce":841761538}]},"public_addr":"172.21.15.76:6819/841761538","cluster_addr":"172.21.15.76:6823/841761538","heartbeat_back_addr":"172.21.15.76:6831/841761538","heartbeat_front_addr":"172.21.15.76:6827/841761538","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T07:56:14.294838+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T07:56:14.270779+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T07:56:14.231809+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T07:56:14.260983+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T07:56:14.302997+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T07:56:14.258600+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T07:56:14.251413+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T07:56:14.304727+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-24T07:56:25.101 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:25.099+0000 7f5454ee5700 1 -- 172.21.15.59:0/1333178855 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f543006aed0 msgr2=0x7f543006d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:56:25.101 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:25.099+0000 7f5454ee5700 1 --2- 172.21.15.59:0/1333178855 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f543006aed0 0x7f543006d390 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f544400aa20 tx=0x7f5444008040 comp rx=0 tx=0).stop 2023-11-24T07:56:25.101 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:25.099+0000 7f5454ee5700 1 -- 172.21.15.59:0/1333178855 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f54501440f0 msgr2=0x7f545015b570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:56:25.102 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:25.099+0000 7f5454ee5700 1 --2- 172.21.15.59:0/1333178855 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f54501440f0 0x7f545015b570 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f54500d1bd0 tx=0x7f5448004ad0 comp rx=0 tx=0).stop 2023-11-24T07:56:25.102 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:25.099+0000 7f5454ee5700 1 -- 172.21.15.59:0/1333178855 shutdown_connections 2023-11-24T07:56:25.102 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:25.099+0000 7f5454ee5700 1 --2- 172.21.15.59:0/1333178855 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f543006aed0 0x7f543006d390 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:56:25.102 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:25.099+0000 7f5454ee5700 1 --2- 172.21.15.59:0/1333178855 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f54501440f0 0x7f545015b570 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:56:25.102 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:25.099+0000 7f5454ee5700 1 --2- 172.21.15.59:0/1333178855 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f54500720f0 0x7f545015d5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:56:25.103 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:25.099+0000 7f5454ee5700 1 --2- 172.21.15.59:0/1333178855 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5450143ae0 0x7f5450138940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:56:25.103 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:25.099+0000 7f5454ee5700 1 -- 172.21.15.59:0/1333178855 >> 172.21.15.59:0/1333178855 conn(0x7f54500b2740 msgr2=0x7f545015e820 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:56:25.103 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:25.100+0000 7f5454ee5700 1 -- 172.21.15.59:0/1333178855 shutdown_connections 2023-11-24T07:56:25.103 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:25.100+0000 7f5454ee5700 1 -- 172.21.15.59:0/1333178855 wait complete. 2023-11-24T07:56:25.114 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-24T07:56:25.238 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:25.236+0000 7fbc24b43700 1 Processor -- start 2023-11-24T07:56:25.238 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:25.236+0000 7fbc24b43700 1 -- start start 2023-11-24T07:56:25.240 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:25.237+0000 7fbc24b43700 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fbc200cf2b0 0x7fbc2015c1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:56:25.240 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:25.237+0000 7fbc24b43700 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbc200d0a90 con 0x7fbc200ce5c0 2023-11-24T07:56:25.240 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:25.237+0000 7fbc24b43700 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbc2006ee10 con 0x7fbc2015bde0 2023-11-24T07:56:25.240 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:25.237+0000 7fbc24b43700 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbc2006f0f0 con 0x7fbc200cf2b0 2023-11-24T07:56:25.240 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:25.237+0000 7fbc1ed9d700 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7fbc200ce5c0 0x7fbc2006fbd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:53160/0 (socket says 172.21.15.59:53160) 2023-11-24T07:56:25.240 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:25.237+0000 7fbc1ed9d700 1 -- 172.21.15.59:0/2479351090 learned_addr learned my addr 172.21.15.59:0/2479351090 (peer_addr_for_me v1:172.21.15.59:0/0) 2023-11-24T07:56:25.240 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:25.238+0000 7fbc1dd9b700 1 --2- 172.21.15.59:0/2479351090 >> v2:172.21.15.76:3300/0 conn(0x7fbc200cf2b0 0x7fbc2015c1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:56:25.240 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:25.238+0000 7fbc1d59a700 1 -- 172.21.15.59:0/2479351090 <== mon.0 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 630917726 0 0) 0x7fbc200d0a90 con 0x7fbc200ce5c0 2023-11-24T07:56:25.241 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:25.238+0000 7fbc1d59a700 1 -- 172.21.15.59:0/2479351090 --> v1:172.21.15.59:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbc080035b0 con 0x7fbc200ce5c0 2023-11-24T07:56:25.241 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:25.238+0000 7fbc1d59a700 1 -- 172.21.15.59:0/2479351090 <== mon.0 v1:172.21.15.59:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3570998860 0 0) 0x7fbc080035b0 con 0x7fbc200ce5c0 2023-11-24T07:56:25.241 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:25.238+0000 7fbc1dd9b700 1 -- 172.21.15.59:0/2479351090 >> v1:172.21.15.76:6789/0 conn(0x7fbc2015bde0 legacy=0x7fbc200d01b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:56:25.241 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:25.238+0000 7fbc1dd9b700 1 -- 172.21.15.59:0/2479351090 >> v1:172.21.15.59:6789/0 conn(0x7fbc200ce5c0 legacy=0x7fbc2006fbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:56:25.241 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:25.238+0000 7fbc1dd9b700 1 -- 172.21.15.59:0/2479351090 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbc2006f3d0 con 0x7fbc200cf2b0 2023-11-24T07:56:25.241 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:25.238+0000 7fbc1dd9b700 1 --2- 172.21.15.59:0/2479351090 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fbc200cf2b0 0x7fbc2015c1c0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fbc10009840 tx=0x7fbc10009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=873a997493794f6e server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:56:25.241 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:25.238+0000 7fbc1d59a700 1 -- 172.21.15.59:0/2479351090 <== mon.1 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc180019f0 con 0x7fbc200cf2b0 2023-11-24T07:56:25.241 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:25.238+0000 7fbc1d59a700 1 -- 172.21.15.59:0/2479351090 <== mon.1 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbc10004030 con 0x7fbc200cf2b0 2023-11-24T07:56:25.241 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:25.238+0000 7fbc1d59a700 1 -- 172.21.15.59:0/2479351090 <== mon.1 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc100043a0 con 0x7fbc200cf2b0 2023-11-24T07:56:25.242 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:25.239+0000 7fbc24b43700 1 -- 172.21.15.59:0/2479351090 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fbc200cf2b0 msgr2=0x7fbc2015c1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:56:25.242 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:25.239+0000 7fbc24b43700 1 --2- 172.21.15.59:0/2479351090 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fbc200cf2b0 0x7fbc2015c1c0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fbc10009840 tx=0x7fbc10009b50 comp rx=0 tx=0).stop 2023-11-24T07:56:25.242 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:25.239+0000 7fbc24b43700 1 -- 172.21.15.59:0/2479351090 shutdown_connections 2023-11-24T07:56:25.243 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:25.239+0000 7fbc24b43700 1 --2- 172.21.15.59:0/2479351090 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fbc200cf2b0 0x7fbc2015c1c0 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:56:25.243 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:25.239+0000 7fbc24b43700 1 -- 172.21.15.59:0/2479351090 >> 172.21.15.59:0/2479351090 conn(0x7fbc200b3720 msgr2=0x7fbc200b3b30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:56:25.243 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:25.239+0000 7fbc24b43700 1 -- 172.21.15.59:0/2479351090 shutdown_connections 2023-11-24T07:56:25.243 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:25.239+0000 7fbc24b43700 1 -- 172.21.15.59:0/2479351090 wait complete. 2023-11-24T07:56:25.243 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:25.240+0000 7fbc24b43700 1 Processor -- start 2023-11-24T07:56:25.243 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:25.240+0000 7fbc24b43700 1 -- start start 2023-11-24T07:56:25.245 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:25.241+0000 7fbc24b43700 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fbc200ce5c0 0x7fbc20159370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:56:25.245 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:25.241+0000 7fbc24b43700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbc200cf2b0 0x7fbc2015cf20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:56:25.246 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:25.241+0000 7fbc24b43700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fbc2015bde0 0x7fbc2015d460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:56:25.246 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:25.241+0000 7fbc24b43700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc2015c4c0 con 0x7fbc200cf2b0 2023-11-24T07:56:25.246 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:25.241+0000 7fbc24b43700 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc201fc260 con 0x7fbc2015bde0 2023-11-24T07:56:25.246 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:25.241+0000 7fbc24b43700 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbc201fb4a0 con 0x7fbc200ce5c0 2023-11-24T07:56:25.246 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:25.241+0000 7fbc1dd9b700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbc200cf2b0 0x7fbc2015cf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:56:25.246 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:25.241+0000 7fbc1dd9b700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbc200cf2b0 0x7fbc2015cf20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:49690/0 (socket says 172.21.15.59:49690) 2023-11-24T07:56:25.246 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:25.241+0000 7fbc1dd9b700 1 -- 172.21.15.59:0/1894935886 learned_addr learned my addr 172.21.15.59:0/1894935886 (peer_addr_for_me v2:172.21.15.59:0/0) 2023-11-24T07:56:25.246 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:25.241+0000 7fbc1e59c700 1 --2- 172.21.15.59:0/1894935886 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fbc200ce5c0 0x7fbc20159370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:56:25.246 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:25.242+0000 7fbc1ed9d700 1 --2- 172.21.15.59:0/1894935886 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fbc2015bde0 0x7fbc2015d460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:56:25.246 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:25.242+0000 7fbc1dd9b700 1 -- 172.21.15.59:0/1894935886 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fbc200ce5c0 msgr2=0x7fbc20159370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:56:25.247 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:25.242+0000 7fbc1dd9b700 1 --2- 172.21.15.59:0/1894935886 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fbc200ce5c0 0x7fbc20159370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:56:25.247 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:25.242+0000 7fbc1dd9b700 1 -- 172.21.15.59:0/1894935886 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fbc2015bde0 msgr2=0x7fbc2015d460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:56:25.247 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:25.242+0000 7fbc1dd9b700 1 --2- 172.21.15.59:0/1894935886 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fbc2015bde0 0x7fbc2015d460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:56:25.247 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:25.242+0000 7fbc1dd9b700 1 -- 172.21.15.59:0/1894935886 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbc100094a0 con 0x7fbc200cf2b0 2023-11-24T07:56:25.247 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:25.242+0000 7fbc1e59c700 1 --2- 172.21.15.59:0/1894935886 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fbc200ce5c0 0x7fbc20159370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-24T07:56:25.247 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:25.242+0000 7fbc1dd9b700 1 --2- 172.21.15.59:0/1894935886 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbc200cf2b0 0x7fbc2015cf20 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fbc100090b0 tx=0x7fbc10004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:56:25.247 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:25.242+0000 7fbc0effd700 1 -- 172.21.15.59:0/1894935886 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc10011630 con 0x7fbc200cf2b0 2023-11-24T07:56:25.247 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:25.242+0000 7fbc0effd700 1 -- 172.21.15.59:0/1894935886 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbc1001f070 con 0x7fbc200cf2b0 2023-11-24T07:56:25.247 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:25.242+0000 7fbc24b43700 1 -- 172.21.15.59:0/1894935886 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbc201fb780 con 0x7fbc200cf2b0 2023-11-24T07:56:25.247 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:25.242+0000 7fbc0effd700 1 -- 172.21.15.59:0/1894935886 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc100119d0 con 0x7fbc200cf2b0 2023-11-24T07:56:25.247 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:25.242+0000 7fbc24b43700 1 -- 172.21.15.59:0/1894935886 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbc201fbd40 con 0x7fbc200cf2b0 2023-11-24T07:56:25.248 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:25.243+0000 7fbc24b43700 1 -- 172.21.15.59:0/1894935886 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbc201fc0b0 con 0x7fbc200cf2b0 2023-11-24T07:56:25.248 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:25.243+0000 7fbc0effd700 1 -- 172.21.15.59:0/1894935886 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fbc10011b30 con 0x7fbc200cf2b0 2023-11-24T07:56:25.248 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:25.244+0000 7fbc0effd700 1 --2- 172.21.15.59:0/1894935886 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7fbc0006aed0 0x7fbc0006d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:56:25.248 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:25.244+0000 7fbc0effd700 1 -- 172.21.15.59:0/1894935886 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(19..19 src has 1..19) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fbc10021070 con 0x7fbc200cf2b0 2023-11-24T07:56:25.248 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:25.244+0000 7fbc1e59c700 1 --2- 172.21.15.59:0/1894935886 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7fbc0006aed0 0x7fbc0006d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:56:25.248 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:25.244+0000 7fbc1e59c700 1 --2- 172.21.15.59:0/1894935886 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7fbc0006aed0 0x7fbc0006d390 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fbc200cc060 tx=0x7fbc14009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:56:25.250 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:25.248+0000 7fbc0effd700 1 -- 172.21.15.59:0/1894935886 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fbc100560b0 con 0x7fbc200cf2b0 2023-11-24T07:56:25.451 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:25.450+0000 7fbc24b43700 1 -- 172.21.15.59:0/1894935886 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fbc201fbed0 con 0x7fbc200cf2b0 2023-11-24T07:56:25.453 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:25.451+0000 7fbc0effd700 1 -- 172.21.15.59:0/1894935886 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v19) v1 ==== 74+0+16450 (secure 0 0 0) 0x7fbc10006cc0 con 0x7fbc200cf2b0 2023-11-24T07:56:25.453 INFO:teuthology.orchestra.run.smithi059.stdout: 2023-11-24T07:56:25.453 INFO:teuthology.orchestra.run.smithi059.stdout:{"epoch":19,"fsid":"9f0e8741-b8f9-48d5-a2c3-09f45903303d","created":"2023-11-24T07:56:06.552729+0000","modified":"2023-11-24T07:56:24.103977+0000","last_up_change":"2023-11-24T07:56:17.372352+0000","last_in_change":"2023-11-24T07:56:09.073250+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-24T07:56:18.352823+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-11-24T07:56:22.185140+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-11-24T07:56:23.084326+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"6d8f9e9b-1782-4e41-928b-a39f2a8ebc06","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6800","nonce":3373878772},{"type":"v1","addr":"172.21.15.59:6801","nonce":3373878772}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6802","nonce":3373878772},{"type":"v1","addr":"172.21.15.59:6803","nonce":3373878772}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6806","nonce":3373878772},{"type":"v1","addr":"172.21.15.59:6807","nonce":3373878772}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6804","nonce":3373878772},{"type":"v1","addr":"172.21.15.59:6805","nonce":3373878772}]},"public_addr":"172.21.15.59:6801/3373878772","cluster_addr":"172.21.15.59:6803/3373878772","heartbeat_back_addr":"172.21.15.59:6807/3373878772","heartbeat_front_addr":"172.21.15.59:6805/3373878772","state":["exists","up"]},{"osd":1,"uuid":"e8d882aa-52e1-4c04-9e1e-c44ef9204a5f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6824","nonce":648258563},{"type":"v1","addr":"172.21.15.59:6825","nonce":648258563}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6826","nonce":648258563},{"type":"v1","addr":"172.21.15.59:6827","nonce":648258563}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6830","nonce":648258563},{"type":"v1","addr":"172.21.15.59:6831","nonce":648258563}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6828","nonce":648258563},{"type":"v1","addr":"172.21.15.59:6829","nonce":648258563}]},"public_addr":"172.21.15.59:6825/648258563","cluster_addr":"172.21.15.59:6827/648258563","heartbeat_back_addr":"172.21.15.59:6831/648258563","heartbeat_front_addr":"172.21.15.59:6829/648258563","state":["exists","up"]},{"osd":2,"uuid":"69eebfa0-695b-4b21-90b9-8a3537168a01","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6808","nonce":1840430654},{"type":"v1","addr":"172.21.15.59:6809","nonce":1840430654}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6810","nonce":1840430654},{"type":"v1","addr":"172.21.15.59:6811","nonce":1840430654}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6814","nonce":1840430654},{"type":"v1","addr":"172.21.15.59:6815","nonce":1840430654}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6812","nonce":1840430654},{"type":"v1","addr":"172.21.15.59:6813","nonce":1840430654}]},"public_addr":"172.21.15.59:6809/1840430654","cluster_addr":"172.21.15.59:6811/1840430654","heartbeat_back_addr":"172.21.15.59:6815/1840430654","heartbeat_front_addr":"172.21.15.59:6813/1840430654","state":["exists","up"]},{"osd":3,"uuid":"96fd77fc-f911-46ff-9445-6f0be2daa8f4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6816","nonce":2504025850},{"type":"v1","addr":"172.21.15.59:6817","nonce":2504025850}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6818","nonce":2504025850},{"type":"v1","addr":"172.21.15.59:6819","nonce":2504025850}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6822","nonce":2504025850},{"type":"v1","addr":"172.21.15.59:6823","nonce":2504025850}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6820","nonce":2504025850},{"type":"v1","addr":"172.21.15.59:6821","nonce":2504025850}]},"public_addr":"172.21.15.59:6817/2504025850","cluster_addr":"172.21.15.59:6819/2504025850","heartbeat_back_addr":"172.21.15.59:6823/2504025850","heartbeat_front_addr":"172.21.15.59:6821/2504025850","state":["exists","up"]},{"osd":4,"uuid":"7ac29fca-b5b2-4c95-9d69-2f164cbbc708","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6816","nonce":2382463431},{"type":"v1","addr":"172.21.15.76:6818","nonce":2382463431}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6820","nonce":2382463431},{"type":"v1","addr":"172.21.15.76:6822","nonce":2382463431}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6828","nonce":2382463431},{"type":"v1","addr":"172.21.15.76:6830","nonce":2382463431}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6824","nonce":2382463431},{"type":"v1","addr":"172.21.15.76:6826","nonce":2382463431}]},"public_addr":"172.21.15.76:6818/2382463431","cluster_addr":"172.21.15.76:6822/2382463431","heartbeat_back_addr":"172.21.15.76:6830/2382463431","heartbeat_front_addr":"172.21.15.76:6826/2382463431","state":["exists","up"]},{"osd":5,"uuid":"485bf711-3851-4238-a4f4-51d52e0a9c58","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6800","nonce":329334308},{"type":"v1","addr":"172.21.15.76:6802","nonce":329334308}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6805","nonce":329334308},{"type":"v1","addr":"172.21.15.76:6807","nonce":329334308}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6814","nonce":329334308},{"type":"v1","addr":"172.21.15.76:6815","nonce":329334308}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6810","nonce":329334308},{"type":"v1","addr":"172.21.15.76:6812","nonce":329334308}]},"public_addr":"172.21.15.76:6802/329334308","cluster_addr":"172.21.15.76:6807/329334308","heartbeat_back_addr":"172.21.15.76:6815/329334308","heartbeat_front_addr":"172.21.15.76:6812/329334308","state":["exists","up"]},{"osd":6,"uuid":"72b3f2fc-1e4a-4e32-968d-003647a0286f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6801","nonce":3946579410},{"type":"v1","addr":"172.21.15.76:6803","nonce":3946579410}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6804","nonce":3946579410},{"type":"v1","addr":"172.21.15.76:6806","nonce":3946579410}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6811","nonce":3946579410},{"type":"v1","addr":"172.21.15.76:6813","nonce":3946579410}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6808","nonce":3946579410},{"type":"v1","addr":"172.21.15.76:6809","nonce":3946579410}]},"public_addr":"172.21.15.76:6803/3946579410","cluster_addr":"172.21.15.76:6806/3946579410","heartbeat_back_addr":"172.21.15.76:6813/3946579410","heartbeat_front_addr":"172.21.15.76:6809/3946579410","state":["exists","up"]},{"osd":7,"uuid":"1ddceb90-6c5d-4d1a-ad56-99cb877f8c70","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6817","nonce":841761538},{"type":"v1","addr":"172.21.15.76:6819","nonce":841761538}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6821","nonce":841761538},{"type":"v1","addr":"172.21.15.76:6823","nonce":841761538}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6829","nonce":841761538},{"type":"v1","addr":"172.21.15.76:6831","nonce":841761538}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6825","nonce":841761538},{"type":"v1","addr":"172.21.15.76:6827","nonce":841761538}]},"public_addr":"172.21.15.76:6819/841761538","cluster_addr":"172.21.15.76:6823/841761538","heartbeat_back_addr":"172.21.15.76:6831/841761538","heartbeat_front_addr":"172.21.15.76:6827/841761538","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T07:56:14.294838+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T07:56:14.270779+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T07:56:14.231809+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T07:56:14.260983+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T07:56:14.302997+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T07:56:14.258600+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T07:56:14.251413+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T07:56:14.304727+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-24T07:56:25.455 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:25.453+0000 7fbc24b43700 1 -- 172.21.15.59:0/1894935886 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7fbc0006aed0 msgr2=0x7fbc0006d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:56:25.455 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:25.453+0000 7fbc24b43700 1 --2- 172.21.15.59:0/1894935886 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7fbc0006aed0 0x7fbc0006d390 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fbc200cc060 tx=0x7fbc14009040 comp rx=0 tx=0).stop 2023-11-24T07:56:25.455 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:25.453+0000 7fbc24b43700 1 -- 172.21.15.59:0/1894935886 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbc200cf2b0 msgr2=0x7fbc2015cf20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:56:25.456 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:25.453+0000 7fbc24b43700 1 --2- 172.21.15.59:0/1894935886 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbc200cf2b0 0x7fbc2015cf20 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fbc100090b0 tx=0x7fbc10004750 comp rx=0 tx=0).stop 2023-11-24T07:56:25.456 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:25.453+0000 7fbc24b43700 1 -- 172.21.15.59:0/1894935886 shutdown_connections 2023-11-24T07:56:25.456 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:25.453+0000 7fbc24b43700 1 --2- 172.21.15.59:0/1894935886 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7fbc0006aed0 0x7fbc0006d390 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:56:25.456 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:25.453+0000 7fbc24b43700 1 --2- 172.21.15.59:0/1894935886 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fbc2015bde0 0x7fbc2015d460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:56:25.456 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:25.453+0000 7fbc24b43700 1 --2- 172.21.15.59:0/1894935886 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fbc200ce5c0 0x7fbc20159370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:56:25.456 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:25.453+0000 7fbc24b43700 1 --2- 172.21.15.59:0/1894935886 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbc200cf2b0 0x7fbc2015cf20 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:56:25.456 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:25.453+0000 7fbc24b43700 1 -- 172.21.15.59:0/1894935886 >> 172.21.15.59:0/1894935886 conn(0x7fbc200b3720 msgr2=0x7fbc2015a130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:56:25.456 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:25.453+0000 7fbc24b43700 1 -- 172.21.15.59:0/1894935886 shutdown_connections 2023-11-24T07:56:25.456 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:25.453+0000 7fbc24b43700 1 -- 172.21.15.59:0/1894935886 wait complete. 2023-11-24T07:56:25.467 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2023-11-24T07:56:25.594 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:25.592+0000 7f64532cb700 1 Processor -- start 2023-11-24T07:56:25.594 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:25.593+0000 7f64532cb700 1 -- start start 2023-11-24T07:56:25.597 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:25.593+0000 7f64532cb700 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f644c0b4150 0x7f644c06de50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:56:25.597 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:25.593+0000 7f64532cb700 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f644c0b5ab0 0x7f644c06e390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:56:25.597 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:25.593+0000 7f64532cb700 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f644c071f50 con 0x7f644c06b620 2023-11-24T07:56:25.597 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:25.593+0000 7f64532cb700 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f644c072230 con 0x7f644c0b5ab0 2023-11-24T07:56:25.597 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:25.593+0000 7f64532cb700 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f644c072510 con 0x7f644c0b4150 2023-11-24T07:56:25.597 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:25.594+0000 7f6451868700 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f644c0b5ab0 0x7f644c06e390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:56:25.597 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:25.594+0000 7f6450866700 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f644c0b4150 0x7f644c06de50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:56:25.597 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:25.594+0000 7f6450866700 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f644c0b4150 0x7f644c06de50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3301/0 says I am v2:172.21.15.59:49138/0 (socket says 172.21.15.59:49138) 2023-11-24T07:56:25.597 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:25.594+0000 7f6451868700 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f644c0b5ab0 0x7f644c06e390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.59:36898/0 (socket says 172.21.15.59:36898) 2023-11-24T07:56:25.598 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:25.594+0000 7f6450866700 1 -- 172.21.15.59:0/276908779 learned_addr learned my addr 172.21.15.59:0/276908779 (peer_addr_for_me v2:172.21.15.59:0/0) 2023-11-24T07:56:25.598 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:25.594+0000 7f6451067700 1 --1- 172.21.15.59:0/276908779 >> v1:172.21.15.76:6789/0 conn(0x7f644c06b620 0x7f644c0b3a40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.59:40448/0 (socket says 172.21.15.59:40448) 2023-11-24T07:56:25.598 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:25.594+0000 7f6450866700 1 -- 172.21.15.59:0/276908779 >> v2:172.21.15.76:3300/0 conn(0x7f644c0b5ab0 msgr2=0x7f644c06e390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T07:56:25.598 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:25.594+0000 7f6450866700 1 --2- 172.21.15.59:0/276908779 >> v2:172.21.15.76:3300/0 conn(0x7f644c0b5ab0 0x7f644c06e390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:56:25.598 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:25.594+0000 7f6450866700 1 -- 172.21.15.59:0/276908779 >> v1:172.21.15.76:6789/0 conn(0x7f644c06b620 legacy=0x7f644c0b3a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:56:25.598 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:25.594+0000 7f6450866700 1 -- 172.21.15.59:0/276908779 --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f644c0727f0 con 0x7f644c0b4150 2023-11-24T07:56:25.598 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:25.594+0000 7f6450866700 1 --2- 172.21.15.59:0/276908779 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f644c0b4150 0x7f644c06de50 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f64480098f0 tx=0x7f6448009c00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ce798a3f60e3d923 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:56:25.598 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:25.595+0000 7f643ffff700 1 -- 172.21.15.59:0/276908779 <== mon.2 v2:172.21.15.76:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6448011830 con 0x7f644c0b4150 2023-11-24T07:56:25.598 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:25.595+0000 7f643ffff700 1 -- 172.21.15.59:0/276908779 <== mon.2 v2:172.21.15.76:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6448011990 con 0x7f644c0b4150 2023-11-24T07:56:25.598 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:25.595+0000 7f643ffff700 1 -- 172.21.15.59:0/276908779 <== mon.2 v2:172.21.15.76:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6448011d20 con 0x7f644c0b4150 2023-11-24T07:56:25.598 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:25.595+0000 7f64532cb700 1 -- 172.21.15.59:0/276908779 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f644c0b4150 msgr2=0x7f644c06de50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:56:25.599 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:25.595+0000 7f64532cb700 1 --2- 172.21.15.59:0/276908779 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f644c0b4150 0x7f644c06de50 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f64480098f0 tx=0x7f6448009c00 comp rx=0 tx=0).stop 2023-11-24T07:56:25.599 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:25.595+0000 7f64532cb700 1 -- 172.21.15.59:0/276908779 shutdown_connections 2023-11-24T07:56:25.599 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:25.595+0000 7f64532cb700 1 --2- 172.21.15.59:0/276908779 >> v2:172.21.15.76:3300/0 conn(0x7f644c0b5ab0 0x7f644c06e390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:56:25.599 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:25.595+0000 7f64532cb700 1 --2- 172.21.15.59:0/276908779 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f644c0b4150 0x7f644c06de50 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:56:25.599 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:25.595+0000 7f64532cb700 1 -- 172.21.15.59:0/276908779 >> 172.21.15.59:0/276908779 conn(0x7f644c0b2740 msgr2=0x7f644c0b2b20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:56:25.599 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:25.595+0000 7f64532cb700 1 -- 172.21.15.59:0/276908779 shutdown_connections 2023-11-24T07:56:25.599 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:25.595+0000 7f64532cb700 1 -- 172.21.15.59:0/276908779 wait complete. 2023-11-24T07:56:25.602 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:25.596+0000 7f64532cb700 1 Processor -- start 2023-11-24T07:56:25.602 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:25.596+0000 7f64532cb700 1 -- start start 2023-11-24T07:56:25.602 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:25.597+0000 7f64532cb700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f644c06b620 0x7f644c1e1300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:56:25.602 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:25.597+0000 7f64532cb700 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f644c0b4150 0x7f644c1e29a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:56:25.602 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:25.597+0000 7f64532cb700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f644c0b5ab0 0x7f644c1dca60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:56:25.602 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:25.597+0000 7f64532cb700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f644c1e2580 con 0x7f644c06b620 2023-11-24T07:56:25.602 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:25.597+0000 7f64532cb700 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f644c1dbda0 con 0x7f644c0b5ab0 2023-11-24T07:56:25.602 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:25.597+0000 7f64532cb700 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f644c1e1900 con 0x7f644c0b4150 2023-11-24T07:56:25.602 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:25.597+0000 7f6451067700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f644c06b620 0x7f644c1e1300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:56:25.602 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:25.597+0000 7f6451067700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f644c06b620 0x7f644c1e1300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:49702/0 (socket says 172.21.15.59:49702) 2023-11-24T07:56:25.602 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:25.597+0000 7f6451067700 1 -- 172.21.15.59:0/2161235006 learned_addr learned my addr 172.21.15.59:0/2161235006 (peer_addr_for_me v2:172.21.15.59:0/0) 2023-11-24T07:56:25.603 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:25.597+0000 7f6451868700 1 --2- 172.21.15.59:0/2161235006 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f644c0b5ab0 0x7f644c1dca60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:56:25.603 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:25.597+0000 7f6450866700 1 --2- 172.21.15.59:0/2161235006 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f644c0b4150 0x7f644c1e29a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:56:25.603 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:25.598+0000 7f6451067700 1 -- 172.21.15.59:0/2161235006 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f644c0b4150 msgr2=0x7f644c1e29a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:56:25.603 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:25.598+0000 7f6451067700 1 --2- 172.21.15.59:0/2161235006 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f644c0b4150 0x7f644c1e29a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:56:25.603 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:25.598+0000 7f6451067700 1 -- 172.21.15.59:0/2161235006 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f644c0b5ab0 msgr2=0x7f644c1dca60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:56:25.603 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:25.598+0000 7f6451067700 1 --2- 172.21.15.59:0/2161235006 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f644c0b5ab0 0x7f644c1dca60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:56:25.603 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:25.598+0000 7f6451067700 1 -- 172.21.15.59:0/2161235006 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6448009550 con 0x7f644c06b620 2023-11-24T07:56:25.603 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:25.598+0000 7f6451067700 1 --2- 172.21.15.59:0/2161235006 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f644c06b620 0x7f644c1e1300 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f644000c520 tx=0x7f644000c830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:56:25.603 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:25.598+0000 7f643dffb700 1 -- 172.21.15.59:0/2161235006 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6440005cb0 con 0x7f644c06b620 2023-11-24T07:56:25.603 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:25.598+0000 7f643dffb700 1 -- 172.21.15.59:0/2161235006 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6440005e10 con 0x7f644c06b620 2023-11-24T07:56:25.603 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:25.598+0000 7f64532cb700 1 -- 172.21.15.59:0/2161235006 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f644c1e1be0 con 0x7f644c06b620 2023-11-24T07:56:25.603 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:25.598+0000 7f643dffb700 1 -- 172.21.15.59:0/2161235006 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64400072d0 con 0x7f644c06b620 2023-11-24T07:56:25.604 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:25.598+0000 7f64532cb700 1 -- 172.21.15.59:0/2161235006 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f644c1e21a0 con 0x7f644c06b620 2023-11-24T07:56:25.604 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:25.599+0000 7f64532cb700 1 -- 172.21.15.59:0/2161235006 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f644c075480 con 0x7f644c06b620 2023-11-24T07:56:25.604 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:25.599+0000 7f643dffb700 1 -- 172.21.15.59:0/2161235006 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f6440007430 con 0x7f644c06b620 2023-11-24T07:56:25.604 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:25.600+0000 7f643dffb700 1 --2- 172.21.15.59:0/2161235006 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f643006aed0 0x7f643006d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:56:25.604 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:25.600+0000 7f643dffb700 1 -- 172.21.15.59:0/2161235006 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(19..19 src has 1..19) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f6440088720 con 0x7f644c06b620 2023-11-24T07:56:25.604 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:25.600+0000 7f6450866700 1 --2- 172.21.15.59:0/2161235006 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f643006aed0 0x7f643006d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:56:25.604 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:25.600+0000 7f6450866700 1 --2- 172.21.15.59:0/2161235006 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f643006aed0 0x7f643006d390 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f6448009c40 tx=0x7f6448008030 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:56:25.606 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:25.604+0000 7f643dffb700 1 -- 172.21.15.59:0/2161235006 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f6440053910 con 0x7f644c06b620 2023-11-24T07:56:25.854 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:25.852+0000 7f64532cb700 1 -- 172.21.15.59:0/2161235006 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7f644c0cfe80 con 0x7f644c06b620 2023-11-24T07:56:26.737 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:26.736+0000 7f643dffb700 1 -- 172.21.15.59:0/2161235006 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v7) v1 ==== 118+0+0 (secure 0 0 0) 0x7f64400170b0 con 0x7f644c06b620 2023-11-24T07:56:26.741 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:26.739+0000 7f64532cb700 1 -- 172.21.15.59:0/2161235006 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f643006aed0 msgr2=0x7f643006d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:56:26.741 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:26.739+0000 7f64532cb700 1 --2- 172.21.15.59:0/2161235006 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f643006aed0 0x7f643006d390 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f6448009c40 tx=0x7f6448008030 comp rx=0 tx=0).stop 2023-11-24T07:56:26.741 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:26.739+0000 7f64532cb700 1 -- 172.21.15.59:0/2161235006 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f644c06b620 msgr2=0x7f644c1e1300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:56:26.741 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:26.739+0000 7f64532cb700 1 --2- 172.21.15.59:0/2161235006 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f644c06b620 0x7f644c1e1300 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f644000c520 tx=0x7f644000c830 comp rx=0 tx=0).stop 2023-11-24T07:56:26.741 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:26.739+0000 7f64532cb700 1 -- 172.21.15.59:0/2161235006 shutdown_connections 2023-11-24T07:56:26.741 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:26.739+0000 7f64532cb700 1 --2- 172.21.15.59:0/2161235006 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f643006aed0 0x7f643006d390 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:56:26.741 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:26.739+0000 7f64532cb700 1 --2- 172.21.15.59:0/2161235006 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f644c0b5ab0 0x7f644c1dca60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:56:26.741 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:26.739+0000 7f64532cb700 1 --2- 172.21.15.59:0/2161235006 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f644c06b620 0x7f644c1e1300 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:56:26.742 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:26.739+0000 7f64532cb700 1 --2- 172.21.15.59:0/2161235006 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f644c0b4150 0x7f644c1e29a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:56:26.742 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:26.739+0000 7f64532cb700 1 -- 172.21.15.59:0/2161235006 >> 172.21.15.59:0/2161235006 conn(0x7f644c0b2740 msgr2=0x7f644c1dc060 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:56:26.742 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:26.739+0000 7f64532cb700 1 -- 172.21.15.59:0/2161235006 shutdown_connections 2023-11-24T07:56:26.742 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:26.739+0000 7f64532cb700 1 -- 172.21.15.59:0/2161235006 wait complete. 2023-11-24T07:56:26.756 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2023-11-24T07:56:26.756 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2023-11-24T07:56:26.893 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:26.891+0000 7fe57e133700 1 Processor -- start 2023-11-24T07:56:26.893 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:26.891+0000 7fe57e133700 1 -- start start 2023-11-24T07:56:26.896 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:26.892+0000 7fe57e133700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fe57806b620 0x7fe5780b3a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:56:26.896 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:26.892+0000 7fe57e133700 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fe57806ee60 0x7fe5780b3f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:56:26.896 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:26.892+0000 7fe57e133700 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe578071f50 con 0x7fe5780b5ab0 2023-11-24T07:56:26.896 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:26.892+0000 7fe57e133700 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe578072230 con 0x7fe57806b620 2023-11-24T07:56:26.896 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:26.892+0000 7fe57e133700 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe578072510 con 0x7fe57806ee60 2023-11-24T07:56:26.896 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:26.892+0000 7fe5777fe700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fe57806b620 0x7fe5780b3a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:56:26.896 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:26.892+0000 7fe5777fe700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fe57806b620 0x7fe5780b3a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:49710/0 (socket says 172.21.15.59:49710) 2023-11-24T07:56:26.896 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:26.892+0000 7fe5777fe700 1 -- 172.21.15.59:0/2233116775 learned_addr learned my addr 172.21.15.59:0/2233116775 (peer_addr_for_me v2:172.21.15.59:0/0) 2023-11-24T07:56:26.896 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:26.892+0000 7fe577fff700 1 --2- 172.21.15.59:0/2233116775 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fe57806ee60 0x7fe5780b3f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:56:26.896 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:26.893+0000 7fe5777fe700 1 -- 172.21.15.59:0/2233116775 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fe57806ee60 msgr2=0x7fe5780b3f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T07:56:26.896 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:26.893+0000 7fe5777fe700 1 --2- 172.21.15.59:0/2233116775 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fe57806ee60 0x7fe5780b3f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:56:26.897 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:26.893+0000 7fe5777fe700 1 -- 172.21.15.59:0/2233116775 >> v1:172.21.15.76:6789/0 conn(0x7fe5780b5ab0 legacy=0x7fe578161ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:56:26.897 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:26.893+0000 7fe5777fe700 1 -- 172.21.15.59:0/2233116775 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe5780727f0 con 0x7fe57806b620 2023-11-24T07:56:26.897 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:26.893+0000 7fe5777fe700 1 --2- 172.21.15.59:0/2233116775 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe57806b620 0x7fe5780b3a40 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fe56c0098f0 tx=0x7fe56c009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=203c2a7d9b232524 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:56:26.897 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:26.893+0000 7fe5767fc700 1 -- 172.21.15.59:0/2233116775 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe56c011830 con 0x7fe57806b620 2023-11-24T07:56:26.897 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:26.893+0000 7fe5767fc700 1 -- 172.21.15.59:0/2233116775 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe56c011990 con 0x7fe57806b620 2023-11-24T07:56:26.897 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:26.893+0000 7fe5767fc700 1 -- 172.21.15.59:0/2233116775 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe56c011d20 con 0x7fe57806b620 2023-11-24T07:56:26.897 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:26.893+0000 7fe57e133700 1 -- 172.21.15.59:0/2233116775 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe57806b620 msgr2=0x7fe5780b3a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:56:26.897 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:26.893+0000 7fe57e133700 1 --2- 172.21.15.59:0/2233116775 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe57806b620 0x7fe5780b3a40 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fe56c0098f0 tx=0x7fe56c009c00 comp rx=0 tx=0).stop 2023-11-24T07:56:26.897 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:26.893+0000 7fe57e133700 1 -- 172.21.15.59:0/2233116775 shutdown_connections 2023-11-24T07:56:26.897 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:26.893+0000 7fe57e133700 1 --2- 172.21.15.59:0/2233116775 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fe57806ee60 0x7fe5780b3f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:56:26.897 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:26.893+0000 7fe57e133700 1 --2- 172.21.15.59:0/2233116775 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe57806b620 0x7fe5780b3a40 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:56:26.897 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:26.893+0000 7fe57e133700 1 -- 172.21.15.59:0/2233116775 >> 172.21.15.59:0/2233116775 conn(0x7fe5780b2740 msgr2=0x7fe5780b2b20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:56:26.898 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:26.894+0000 7fe57e133700 1 -- 172.21.15.59:0/2233116775 shutdown_connections 2023-11-24T07:56:26.898 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:26.894+0000 7fe57e133700 1 -- 172.21.15.59:0/2233116775 wait complete. 2023-11-24T07:56:26.900 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:26.895+0000 7fe57e133700 1 Processor -- start 2023-11-24T07:56:26.900 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:26.895+0000 7fe57e133700 1 -- start start 2023-11-24T07:56:26.900 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:26.896+0000 7fe57e133700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe57806b620 0x7fe5781e85c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:56:26.900 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:26.896+0000 7fe57e133700 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fe57806ee60 0x7fe57813dbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:56:26.900 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:26.896+0000 7fe57e133700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe5780b5ab0 0x7fe5780adb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:56:26.900 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:26.896+0000 7fe57e133700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5781e8b00 con 0x7fe57806b620 2023-11-24T07:56:26.900 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:26.896+0000 7fe57e133700 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe57813f2b0 con 0x7fe5780b5ab0 2023-11-24T07:56:26.900 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:26.896+0000 7fe57e133700 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe57813e3c0 con 0x7fe57806ee60 2023-11-24T07:56:26.900 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:26.896+0000 7fe5777fe700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe57806b620 0x7fe5781e85c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:56:26.900 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:26.896+0000 7fe5777fe700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe57806b620 0x7fe5781e85c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:49714/0 (socket says 172.21.15.59:49714) 2023-11-24T07:56:26.900 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:26.896+0000 7fe5777fe700 1 -- 172.21.15.59:0/2064483361 learned_addr learned my addr 172.21.15.59:0/2064483361 (peer_addr_for_me v2:172.21.15.59:0/0) 2023-11-24T07:56:26.901 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:26.896+0000 7fe576ffd700 1 --2- 172.21.15.59:0/2064483361 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fe57806ee60 0x7fe57813dbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:56:26.901 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:26.896+0000 7fe577fff700 1 --2- 172.21.15.59:0/2064483361 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe5780b5ab0 0x7fe5780adb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:56:26.901 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:26.896+0000 7fe5777fe700 1 -- 172.21.15.59:0/2064483361 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fe57806ee60 msgr2=0x7fe57813dbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:56:26.901 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:26.896+0000 7fe5777fe700 1 --2- 172.21.15.59:0/2064483361 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fe57806ee60 0x7fe57813dbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:56:26.901 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:26.896+0000 7fe5777fe700 1 -- 172.21.15.59:0/2064483361 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe5780b5ab0 msgr2=0x7fe5780adb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:56:26.901 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:26.896+0000 7fe5777fe700 1 --2- 172.21.15.59:0/2064483361 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe5780b5ab0 0x7fe5780adb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:56:26.901 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:26.896+0000 7fe5777fe700 1 -- 172.21.15.59:0/2064483361 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe56c009550 con 0x7fe57806b620 2023-11-24T07:56:26.901 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:26.896+0000 7fe5777fe700 1 --2- 172.21.15.59:0/2064483361 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe57806b620 0x7fe5781e85c0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fe56c000c00 tx=0x7fe56c004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:56:26.901 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:26.897+0000 7fe55ffff700 1 -- 172.21.15.59:0/2064483361 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe56c004400 con 0x7fe57806b620 2023-11-24T07:56:26.901 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:26.897+0000 7fe55ffff700 1 -- 172.21.15.59:0/2064483361 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe56c01f070 con 0x7fe57806b620 2023-11-24T07:56:26.901 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:26.897+0000 7fe57e133700 1 -- 172.21.15.59:0/2064483361 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe57813e6a0 con 0x7fe57806b620 2023-11-24T07:56:26.901 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:26.897+0000 7fe55ffff700 1 -- 172.21.15.59:0/2064483361 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe56c0038e0 con 0x7fe57806b620 2023-11-24T07:56:26.902 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:26.897+0000 7fe57e133700 1 -- 172.21.15.59:0/2064483361 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe57813ec60 con 0x7fe57806b620 2023-11-24T07:56:26.902 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:26.898+0000 7fe57e133700 1 -- 172.21.15.59:0/2064483361 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe578075480 con 0x7fe57806b620 2023-11-24T07:56:26.902 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:26.899+0000 7fe55ffff700 1 -- 172.21.15.59:0/2064483361 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fe56c021030 con 0x7fe57806b620 2023-11-24T07:56:26.903 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:26.899+0000 7fe55ffff700 1 --2- 172.21.15.59:0/2064483361 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7fe56406aed0 0x7fe56406d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:56:26.903 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:26.899+0000 7fe55ffff700 1 -- 172.21.15.59:0/2064483361 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fe56c08ad00 con 0x7fe57806b620 2023-11-24T07:56:26.903 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:26.899+0000 7fe576ffd700 1 --2- 172.21.15.59:0/2064483361 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7fe56406aed0 0x7fe56406d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:56:26.903 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:26.900+0000 7fe576ffd700 1 --2- 172.21.15.59:0/2064483361 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7fe56406aed0 0x7fe56406d390 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fe5780ae850 tx=0x7fe568009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:56:26.904 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:26.903+0000 7fe55ffff700 1 -- 172.21.15.59:0/2064483361 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fe56c055f60 con 0x7fe57806b620 2023-11-24T07:56:27.153 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:27.151+0000 7fe57e133700 1 -- 172.21.15.59:0/2064483361 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7fe57813f3f0 con 0x7fe57806b620 2023-11-24T07:56:27.739 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:27.737+0000 7fe55ffff700 1 -- 172.21.15.59:0/2064483361 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v8) v1 ==== 122+0+0 (secure 0 0 0) 0x7fe57813f3f0 con 0x7fe57806b620 2023-11-24T07:56:27.743 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:27.740+0000 7fe57e133700 1 -- 172.21.15.59:0/2064483361 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7fe56406aed0 msgr2=0x7fe56406d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:56:27.743 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:27.740+0000 7fe57e133700 1 --2- 172.21.15.59:0/2064483361 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7fe56406aed0 0x7fe56406d390 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fe5780ae850 tx=0x7fe568009040 comp rx=0 tx=0).stop 2023-11-24T07:56:27.743 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:27.740+0000 7fe57e133700 1 -- 172.21.15.59:0/2064483361 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe57806b620 msgr2=0x7fe5781e85c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:56:27.743 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:27.740+0000 7fe57e133700 1 --2- 172.21.15.59:0/2064483361 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe57806b620 0x7fe5781e85c0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fe56c000c00 tx=0x7fe56c004750 comp rx=0 tx=0).stop 2023-11-24T07:56:27.744 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:27.740+0000 7fe57e133700 1 -- 172.21.15.59:0/2064483361 shutdown_connections 2023-11-24T07:56:27.744 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:27.740+0000 7fe57e133700 1 --2- 172.21.15.59:0/2064483361 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7fe56406aed0 0x7fe56406d390 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:56:27.744 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:27.740+0000 7fe57e133700 1 --2- 172.21.15.59:0/2064483361 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe5780b5ab0 0x7fe5780adb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:56:27.744 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:27.740+0000 7fe57e133700 1 --2- 172.21.15.59:0/2064483361 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe57806b620 0x7fe5781e85c0 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:56:27.744 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:27.740+0000 7fe57e133700 1 --2- 172.21.15.59:0/2064483361 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fe57806ee60 0x7fe57813dbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:56:27.744 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:27.740+0000 7fe57e133700 1 -- 172.21.15.59:0/2064483361 >> 172.21.15.59:0/2064483361 conn(0x7fe5780b2740 msgr2=0x7fe5780b0720 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:56:27.744 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:27.740+0000 7fe57e133700 1 -- 172.21.15.59:0/2064483361 shutdown_connections 2023-11-24T07:56:27.745 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:27.741+0000 7fe57e133700 1 -- 172.21.15.59:0/2064483361 wait complete. 2023-11-24T07:56:27.757 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2023-11-24T07:56:27.757 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-24T07:56:27.889 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:27.888+0000 7fa64e520700 1 Processor -- start 2023-11-24T07:56:27.890 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:27.888+0000 7fa64e520700 1 -- start start 2023-11-24T07:56:27.896 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:27.888+0000 7fa64e520700 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fa6481440f0 0x7fa6480b51b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:56:27.896 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:27.888+0000 7fa64e520700 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa648143ec0 con 0x7fa648143ae0 2023-11-24T07:56:27.897 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:27.888+0000 7fa64e520700 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa6480b3ae0 con 0x7fa6480722c0 2023-11-24T07:56:27.897 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:27.888+0000 7fa64e520700 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa6480b40a0 con 0x7fa6481440f0 2023-11-24T07:56:27.897 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:27.889+0000 7fa647fff700 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7fa648143ae0 0x7fa648071bb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:53196/0 (socket says 172.21.15.59:53196) 2023-11-24T07:56:27.897 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:27.889+0000 7fa647fff700 1 -- 172.21.15.59:0/42648562 learned_addr learned my addr 172.21.15.59:0/42648562 (peer_addr_for_me v1:172.21.15.59:0/0) 2023-11-24T07:56:27.897 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:27.889+0000 7fa64cabd700 1 --2- 172.21.15.59:0/42648562 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fa6481440f0 0x7fa6480b51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:56:27.897 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:27.889+0000 7fa646ffd700 1 -- 172.21.15.59:0/42648562 <== mon.0 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2184234088 0 0) 0x7fa648143ec0 con 0x7fa648143ae0 2023-11-24T07:56:27.897 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:27.889+0000 7fa646ffd700 1 -- 172.21.15.59:0/42648562 --> v1:172.21.15.59:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa6300035b0 con 0x7fa648143ae0 2023-11-24T07:56:27.898 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:27.889+0000 7fa646ffd700 1 -- 172.21.15.59:0/42648562 <== mon.1 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4068159256 0 0) 0x7fa6480b3ae0 con 0x7fa6480722c0 2023-11-24T07:56:27.898 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:27.889+0000 7fa646ffd700 1 -- 172.21.15.59:0/42648562 --> v1:172.21.15.76:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa648143ec0 con 0x7fa6480722c0 2023-11-24T07:56:27.898 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:27.889+0000 7fa646ffd700 1 -- 172.21.15.59:0/42648562 <== mon.0 v1:172.21.15.59:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3353275295 0 0) 0x7fa6300035b0 con 0x7fa648143ae0 2023-11-24T07:56:27.898 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:27.889+0000 7fa646ffd700 1 -- 172.21.15.59:0/42648562 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fa6481440f0 msgr2=0x7fa6480b51b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T07:56:27.899 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:27.889+0000 7fa646ffd700 1 --2- 172.21.15.59:0/42648562 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fa6481440f0 0x7fa6480b51b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:56:27.899 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:27.889+0000 7fa646ffd700 1 -- 172.21.15.59:0/42648562 >> v1:172.21.15.76:6789/0 conn(0x7fa6480722c0 legacy=0x7fa64806e510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:56:27.899 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:27.889+0000 7fa646ffd700 1 -- 172.21.15.59:0/42648562 --> v1:172.21.15.59:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa6480b3dc0 con 0x7fa648143ae0 2023-11-24T07:56:27.899 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:27.889+0000 7fa646ffd700 1 -- 172.21.15.59:0/42648562 <== mon.0 v1:172.21.15.59:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1312333566 0 0) 0x7fa638001670 con 0x7fa648143ae0 2023-11-24T07:56:27.899 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:27.889+0000 7fa64cabd700 1 --2- 172.21.15.59:0/42648562 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fa6481440f0 0x7fa6480b51b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-24T07:56:27.899 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:27.889+0000 7fa646ffd700 1 -- 172.21.15.59:0/42648562 >> v1:172.21.15.59:6789/0 conn(0x7fa648143ae0 legacy=0x7fa648071bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:56:27.899 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:27.889+0000 7fa646ffd700 1 --2- 172.21.15.59:0/42648562 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa630003bc0 0x7fa630003fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:56:27.899 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:27.889+0000 7fa646ffd700 1 --2- 172.21.15.59:0/42648562 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa630004850 0x7fa630004cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:56:27.899 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:27.889+0000 7fa646ffd700 1 --2- 172.21.15.59:0/42648562 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fa6481440f0 0x7fa630005210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:56:27.900 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:27.889+0000 7fa646ffd700 1 -- 172.21.15.59:0/42648562 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6480b40a0 con 0x7fa630003bc0 2023-11-24T07:56:27.900 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:27.889+0000 7fa646ffd700 1 -- 172.21.15.59:0/42648562 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa63000ba40 con 0x7fa630004850 2023-11-24T07:56:27.900 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:27.889+0000 7fa646ffd700 1 -- 172.21.15.59:0/42648562 --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa63000bc00 con 0x7fa6481440f0 2023-11-24T07:56:27.900 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:27.889+0000 7fa647fff700 1 --2- 172.21.15.59:0/42648562 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa630003bc0 0x7fa630003fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:56:27.900 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:27.889+0000 7fa6477fe700 1 --2- 172.21.15.59:0/42648562 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa630004850 0x7fa630004cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:56:27.900 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:27.889+0000 7fa64cabd700 1 --2- 172.21.15.59:0/42648562 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fa6481440f0 0x7fa630005210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:56:27.900 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:27.890+0000 7fa647fff700 1 -- 172.21.15.59:0/42648562 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fa6481440f0 msgr2=0x7fa630005210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T07:56:27.900 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:27.890+0000 7fa647fff700 1 --2- 172.21.15.59:0/42648562 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fa6481440f0 0x7fa630005210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:56:27.901 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:27.890+0000 7fa647fff700 1 -- 172.21.15.59:0/42648562 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa630004850 msgr2=0x7fa630004cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T07:56:27.901 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:27.890+0000 7fa647fff700 1 --2- 172.21.15.59:0/42648562 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa630004850 0x7fa630004cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:56:27.901 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:27.890+0000 7fa647fff700 1 -- 172.21.15.59:0/42648562 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa640009a70 con 0x7fa630003bc0 2023-11-24T07:56:27.901 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:27.890+0000 7fa64cabd700 1 --2- 172.21.15.59:0/42648562 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fa6481440f0 0x7fa630005210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-24T07:56:27.901 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:27.890+0000 7fa647fff700 1 --2- 172.21.15.59:0/42648562 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa630003bc0 0x7fa630003fc0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fa63800c210 tx=0x7fa630005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f5a33aac0838c7d1 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:56:27.901 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:27.890+0000 7fa646ffd700 1 -- 172.21.15.59:0/42648562 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa63800cd50 con 0x7fa630003bc0 2023-11-24T07:56:27.901 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:27.890+0000 7fa646ffd700 1 -- 172.21.15.59:0/42648562 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa638005960 con 0x7fa630003bc0 2023-11-24T07:56:27.901 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:27.890+0000 7fa646ffd700 1 -- 172.21.15.59:0/42648562 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa638005d10 con 0x7fa630003bc0 2023-11-24T07:56:27.902 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:27.890+0000 7fa64e520700 1 -- 172.21.15.59:0/42648562 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa630003bc0 msgr2=0x7fa630003fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:56:27.902 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:27.890+0000 7fa64e520700 1 --2- 172.21.15.59:0/42648562 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa630003bc0 0x7fa630003fc0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fa63800c210 tx=0x7fa630005880 comp rx=0 tx=0).stop 2023-11-24T07:56:27.902 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:27.890+0000 7fa64cabd700 1 -- 172.21.15.59:0/42648562 reap_dead start 2023-11-24T07:56:27.902 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:27.890+0000 7fa64e520700 1 -- 172.21.15.59:0/42648562 shutdown_connections 2023-11-24T07:56:27.902 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:27.890+0000 7fa64e520700 1 -- 172.21.15.59:0/42648562 >> 172.21.15.59:0/42648562 conn(0x7fa6480b2740 msgr2=0x7fa6480b2b20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:56:27.902 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:27.890+0000 7fa64e520700 1 -- 172.21.15.59:0/42648562 shutdown_connections 2023-11-24T07:56:27.902 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:27.890+0000 7fa64e520700 1 -- 172.21.15.59:0/42648562 wait complete. 2023-11-24T07:56:27.902 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:27.892+0000 7fa64e520700 1 Processor -- start 2023-11-24T07:56:27.902 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:27.892+0000 7fa64e520700 1 -- start start 2023-11-24T07:56:27.902 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:27.893+0000 7fa64e520700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa630003bc0 0x7fa6481341b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:56:27.903 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:27.893+0000 7fa64e520700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa6480c9dc0 0x7fa6480cf8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:56:27.903 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:27.893+0000 7fa64e520700 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fa6480ca220 0x7fa6480ca7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:56:27.903 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:27.893+0000 7fa64e520700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6480ce820 con 0x7fa6480c9dc0 2023-11-24T07:56:27.903 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:27.893+0000 7fa64e520700 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6480ceb00 con 0x7fa630003bc0 2023-11-24T07:56:27.903 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:27.893+0000 7fa64e520700 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa6480cede0 con 0x7fa6480ca220 2023-11-24T07:56:27.903 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:27.893+0000 7fa6477fe700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa6480c9dc0 0x7fa6480cf8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:56:27.903 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:27.893+0000 7fa6477fe700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa6480c9dc0 0x7fa6480cf8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:49736/0 (socket says 172.21.15.59:49736) 2023-11-24T07:56:27.904 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:27.893+0000 7fa6477fe700 1 -- 172.21.15.59:0/3550087338 learned_addr learned my addr 172.21.15.59:0/3550087338 (peer_addr_for_me v2:172.21.15.59:0/0) 2023-11-24T07:56:27.904 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:27.893+0000 7fa64cabd700 1 --2- 172.21.15.59:0/3550087338 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fa6480ca220 0x7fa6480ca7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:56:27.904 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:27.893+0000 7fa647fff700 1 --2- 172.21.15.59:0/3550087338 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa630003bc0 0x7fa6481341b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:56:27.904 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:27.893+0000 7fa6477fe700 1 -- 172.21.15.59:0/3550087338 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fa6480ca220 msgr2=0x7fa6480ca7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:56:27.905 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:27.893+0000 7fa6477fe700 1 --2- 172.21.15.59:0/3550087338 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fa6480ca220 0x7fa6480ca7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:56:27.905 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:27.893+0000 7fa6477fe700 1 -- 172.21.15.59:0/3550087338 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa630003bc0 msgr2=0x7fa6481341b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:56:27.905 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:27.893+0000 7fa6477fe700 1 --2- 172.21.15.59:0/3550087338 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa630003bc0 0x7fa6481341b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:56:27.905 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:27.893+0000 7fa6477fe700 1 -- 172.21.15.59:0/3550087338 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa6480cf0c0 con 0x7fa6480c9dc0 2023-11-24T07:56:27.905 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:27.893+0000 7fa6477fe700 1 --2- 172.21.15.59:0/3550087338 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa6480c9dc0 0x7fa6480cf8c0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fa63c00f7a0 tx=0x7fa63c00fab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:56:27.905 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:27.893+0000 7fa644ff9700 1 -- 172.21.15.59:0/3550087338 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa63c005320 con 0x7fa6480c9dc0 2023-11-24T07:56:27.906 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:27.893+0000 7fa644ff9700 1 -- 172.21.15.59:0/3550087338 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa63c0054a0 con 0x7fa6480c9dc0 2023-11-24T07:56:27.906 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:27.893+0000 7fa64e520700 1 -- 172.21.15.59:0/3550087338 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa6480cf3a0 con 0x7fa6480c9dc0 2023-11-24T07:56:27.906 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:27.893+0000 7fa644ff9700 1 -- 172.21.15.59:0/3550087338 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa63c006aa0 con 0x7fa6480c9dc0 2023-11-24T07:56:27.906 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:27.893+0000 7fa64e520700 1 -- 172.21.15.59:0/3550087338 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa6480cfe00 con 0x7fa6480c9dc0 2023-11-24T07:56:27.906 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:27.894+0000 7fa64e520700 1 -- 172.21.15.59:0/3550087338 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa6480cf710 con 0x7fa6480c9dc0 2023-11-24T07:56:27.907 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:27.895+0000 7fa644ff9700 1 -- 172.21.15.59:0/3550087338 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fa63c006c00 con 0x7fa6480c9dc0 2023-11-24T07:56:27.908 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:27.895+0000 7fa644ff9700 1 --2- 172.21.15.59:0/3550087338 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7fa63406aed0 0x7fa63406d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:56:27.908 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:27.895+0000 7fa644ff9700 1 -- 172.21.15.59:0/3550087338 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fa63c088a80 con 0x7fa6480c9dc0 2023-11-24T07:56:27.908 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:27.895+0000 7fa647fff700 1 --2- 172.21.15.59:0/3550087338 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7fa63406aed0 0x7fa63406d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:56:27.909 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:27.896+0000 7fa647fff700 1 --2- 172.21.15.59:0/3550087338 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7fa63406aed0 0x7fa63406d390 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fa638015040 tx=0x7fa638007f90 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:56:27.909 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:27.900+0000 7fa644ff9700 1 -- 172.21.15.59:0/3550087338 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fa63c053a20 con 0x7fa6480c9dc0 2023-11-24T07:56:28.146 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:28.144+0000 7fa64e520700 1 -- 172.21.15.59:0/3550087338 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fa6481447e0 con 0x7fa6480c9dc0 2023-11-24T07:56:28.147 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:28.145+0000 7fa644ff9700 1 -- 172.21.15.59:0/3550087338 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) v1 ==== 93+0+4904 (secure 0 0 0) 0x7fa63c057040 con 0x7fa6480c9dc0 2023-11-24T07:56:28.147 INFO:teuthology.orchestra.run.smithi059.stdout: 2023-11-24T07:56:28.147 INFO:teuthology.orchestra.run.smithi059.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4220,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6835/838038775","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6833","nonce":838038775},{"type":"v1","addr":"172.21.15.59:6835","nonce":838038775}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4232,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.76:6837/3226690004","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6835","nonce":3226690004},{"type":"v1","addr":"172.21.15.76:6837","nonce":3226690004}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4260,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6834/2986456006","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6832","nonce":2986456006},{"type":"v1","addr":"172.21.15.59:6834","nonce":2986456006}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-24T07:56:24.104295+0000","modified":"2023-11-24T07:56:27.152806+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":4264},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4264":{"gid":4264,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.76:6836/3993077463","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6834","nonce":3993077463},{"type":"v1","addr":"172.21.15.76:6836","nonce":3993077463}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-11-24T07:56:28.149 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:28.147+0000 7fa64e520700 1 -- 172.21.15.59:0/3550087338 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7fa63406aed0 msgr2=0x7fa63406d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:56:28.149 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:28.147+0000 7fa64e520700 1 --2- 172.21.15.59:0/3550087338 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7fa63406aed0 0x7fa63406d390 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fa638015040 tx=0x7fa638007f90 comp rx=0 tx=0).stop 2023-11-24T07:56:28.149 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:28.147+0000 7fa64e520700 1 -- 172.21.15.59:0/3550087338 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa6480c9dc0 msgr2=0x7fa6480cf8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:56:28.149 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:28.147+0000 7fa64e520700 1 --2- 172.21.15.59:0/3550087338 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa6480c9dc0 0x7fa6480cf8c0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fa63c00f7a0 tx=0x7fa63c00fab0 comp rx=0 tx=0).stop 2023-11-24T07:56:28.149 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:28.147+0000 7fa64e520700 1 -- 172.21.15.59:0/3550087338 shutdown_connections 2023-11-24T07:56:28.149 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:28.147+0000 7fa64e520700 1 --2- 172.21.15.59:0/3550087338 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7fa63406aed0 0x7fa63406d390 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:56:28.150 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:28.147+0000 7fa64e520700 1 --2- 172.21.15.59:0/3550087338 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fa6480ca220 0x7fa6480ca7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:56:28.150 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:28.147+0000 7fa64e520700 1 --2- 172.21.15.59:0/3550087338 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa630003bc0 0x7fa6481341b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:56:28.150 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:28.147+0000 7fa64e520700 1 --2- 172.21.15.59:0/3550087338 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa6480c9dc0 0x7fa6480cf8c0 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:56:28.150 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:28.147+0000 7fa64e520700 1 -- 172.21.15.59:0/3550087338 >> 172.21.15.59:0/3550087338 conn(0x7fa6480b2740 msgr2=0x7fa64806e410 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:56:28.150 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:28.147+0000 7fa64e520700 1 -- 172.21.15.59:0/3550087338 shutdown_connections 2023-11-24T07:56:28.150 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:28.147+0000 7fa64e520700 1 -- 172.21.15.59:0/3550087338 wait complete. 2023-11-24T07:56:28.151 INFO:teuthology.orchestra.run.smithi059.stderr:dumped fsmap epoch 8 2023-11-24T07:56:28.161 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-24T07:56:28.286 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:28.284+0000 7f6f1031a700 1 Processor -- start 2023-11-24T07:56:28.286 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:28.284+0000 7f6f1031a700 1 -- start start 2023-11-24T07:56:28.289 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:28.285+0000 7f6f1031a700 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f6f080cf2b0 0x7f6f0815c1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:56:28.289 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:28.285+0000 7f6f1031a700 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6f080d0a90 con 0x7f6f0815bde0 2023-11-24T07:56:28.289 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:28.285+0000 7f6f1031a700 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6f0806ee10 con 0x7f6f080ce5c0 2023-11-24T07:56:28.290 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:28.285+0000 7f6f1031a700 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6f0806f0f0 con 0x7f6f080cf2b0 2023-11-24T07:56:28.290 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:28.285+0000 7f6f0e0b6700 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7f6f0815bde0 0x7f6f080d01b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:53216/0 (socket says 172.21.15.59:53216) 2023-11-24T07:56:28.290 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:28.285+0000 7f6f0e0b6700 1 -- 172.21.15.59:0/3588663668 learned_addr learned my addr 172.21.15.59:0/3588663668 (peer_addr_for_me v1:172.21.15.59:0/0) 2023-11-24T07:56:28.290 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:28.286+0000 7f6f0d8b5700 1 --2- 172.21.15.59:0/3588663668 >> v2:172.21.15.76:3300/0 conn(0x7f6f080cf2b0 0x7f6f0815c1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:56:28.290 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:28.286+0000 7f6f0d0b4700 1 -- 172.21.15.59:0/3588663668 <== mon.0 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1597934936 0 0) 0x7f6f080d0a90 con 0x7f6f0815bde0 2023-11-24T07:56:28.290 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:28.286+0000 7f6f0d0b4700 1 -- 172.21.15.59:0/3588663668 --> v1:172.21.15.59:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6ef40035b0 con 0x7f6f0815bde0 2023-11-24T07:56:28.290 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:28.286+0000 7f6f0d0b4700 1 -- 172.21.15.59:0/3588663668 <== mon.0 v1:172.21.15.59:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2874847238 0 0) 0x7f6ef40035b0 con 0x7f6f0815bde0 2023-11-24T07:56:28.290 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:28.286+0000 7f6f0d0b4700 1 -- 172.21.15.59:0/3588663668 >> v2:172.21.15.76:3300/0 conn(0x7f6f080cf2b0 msgr2=0x7f6f0815c1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T07:56:28.290 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:28.286+0000 7f6f0d0b4700 1 --2- 172.21.15.59:0/3588663668 >> v2:172.21.15.76:3300/0 conn(0x7f6f080cf2b0 0x7f6f0815c1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:56:28.291 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:28.286+0000 7f6f0d0b4700 1 -- 172.21.15.59:0/3588663668 >> v1:172.21.15.76:6789/0 conn(0x7f6f080ce5c0 legacy=0x7f6f0806fbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:56:28.291 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:28.286+0000 7f6f0d0b4700 1 -- 172.21.15.59:0/3588663668 --> v1:172.21.15.59:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6f0806f3d0 con 0x7f6f0815bde0 2023-11-24T07:56:28.291 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:28.286+0000 7f6f0d0b4700 1 -- 172.21.15.59:0/3588663668 <== mon.0 v1:172.21.15.59:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1312333566 0 0) 0x7f6ef8001670 con 0x7f6f0815bde0 2023-11-24T07:56:28.291 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:28.286+0000 7f6f0d0b4700 1 -- 172.21.15.59:0/3588663668 >> v1:172.21.15.59:6789/0 conn(0x7f6f0815bde0 legacy=0x7f6f080d01b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:56:28.291 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:28.286+0000 7f6f0d0b4700 1 --2- 172.21.15.59:0/3588663668 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f6ef4003a00 0x7f6ef4003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:56:28.291 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:28.286+0000 7f6f0d0b4700 1 --2- 172.21.15.59:0/3588663668 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6ef4004850 0x7f6ef4004cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:56:28.291 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:28.286+0000 7f6f0d0b4700 1 --2- 172.21.15.59:0/3588663668 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6ef4005210 0x7f6ef400ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:56:28.291 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:28.286+0000 7f6f0d0b4700 1 -- 172.21.15.59:0/3588663668 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f0806f0f0 con 0x7f6ef4005210 2023-11-24T07:56:28.291 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:28.286+0000 7f6f0d0b4700 1 -- 172.21.15.59:0/3588663668 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ef400c1b0 con 0x7f6ef4004850 2023-11-24T07:56:28.291 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:28.286+0000 7f6f0d0b4700 1 -- 172.21.15.59:0/3588663668 --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6ef400c370 con 0x7f6ef4003a00 2023-11-24T07:56:28.291 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:28.286+0000 7f6f0e8b7700 1 --2- 172.21.15.59:0/3588663668 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6ef4005210 0x7f6ef400ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:56:28.291 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:28.286+0000 7f6f0d8b5700 1 --2- 172.21.15.59:0/3588663668 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6ef4004850 0x7f6ef4004cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:56:28.292 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:28.286+0000 7f6f0e0b6700 1 --2- 172.21.15.59:0/3588663668 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f6ef4003a00 0x7f6ef4003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:56:28.292 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:28.286+0000 7f6f0d8b5700 1 -- 172.21.15.59:0/3588663668 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f6ef4003a00 msgr2=0x7f6ef4003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T07:56:28.292 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:28.286+0000 7f6f0d8b5700 1 --2- 172.21.15.59:0/3588663668 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f6ef4003a00 0x7f6ef4003e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:56:28.292 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:28.286+0000 7f6f0d8b5700 1 -- 172.21.15.59:0/3588663668 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6ef4005210 msgr2=0x7f6ef400ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T07:56:28.292 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:28.286+0000 7f6f0d8b5700 1 --2- 172.21.15.59:0/3588663668 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6ef4005210 0x7f6ef400ba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:56:28.292 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:28.286+0000 7f6f0d8b5700 1 -- 172.21.15.59:0/3588663668 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6ef40035b0 con 0x7f6ef4004850 2023-11-24T07:56:28.292 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:28.286+0000 7f6f0e8b7700 1 --2- 172.21.15.59:0/3588663668 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6ef4005210 0x7f6ef400ba10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-24T07:56:28.292 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:28.286+0000 7f6f0e8b7700 1 -- 172.21.15.59:0/3588663668 reap_dead start 2023-11-24T07:56:28.292 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:28.287+0000 7f6f0d8b5700 1 --2- 172.21.15.59:0/3588663668 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6ef4004850 0x7f6ef4004cd0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f6f04009b30 tx=0x7f6ef4005880 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b944c2a4bd7d7ded server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:56:28.292 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:28.287+0000 7f6f0d0b4700 1 -- 172.21.15.59:0/3588663668 <== mon.1 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f0400e6c0 con 0x7f6ef4004850 2023-11-24T07:56:28.292 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:28.287+0000 7f6f0d0b4700 1 -- 172.21.15.59:0/3588663668 <== mon.1 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6f0400e820 con 0x7f6ef4004850 2023-11-24T07:56:28.292 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:28.287+0000 7f6f0d0b4700 1 -- 172.21.15.59:0/3588663668 <== mon.1 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f0400eb90 con 0x7f6ef4004850 2023-11-24T07:56:28.293 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:28.287+0000 7f6f1031a700 1 -- 172.21.15.59:0/3588663668 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6ef4004850 msgr2=0x7f6ef4004cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:56:28.293 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:28.287+0000 7f6f1031a700 1 --2- 172.21.15.59:0/3588663668 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6ef4004850 0x7f6ef4004cd0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f6f04009b30 tx=0x7f6ef4005880 comp rx=0 tx=0).stop 2023-11-24T07:56:28.293 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:28.287+0000 7f6f1031a700 1 -- 172.21.15.59:0/3588663668 shutdown_connections 2023-11-24T07:56:28.293 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:28.287+0000 7f6f1031a700 1 --2- 172.21.15.59:0/3588663668 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6ef4004850 0x7f6ef4004cd0 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:56:28.293 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:28.287+0000 7f6f1031a700 1 -- 172.21.15.59:0/3588663668 >> 172.21.15.59:0/3588663668 conn(0x7f6f080b3720 msgr2=0x7f6f080b3b30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:56:28.293 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:28.287+0000 7f6f1031a700 1 -- 172.21.15.59:0/3588663668 shutdown_connections 2023-11-24T07:56:28.293 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:28.287+0000 7f6f1031a700 1 -- 172.21.15.59:0/3588663668 wait complete. 2023-11-24T07:56:28.295 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:28.289+0000 7f6f1031a700 1 Processor -- start 2023-11-24T07:56:28.295 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:28.289+0000 7f6f1031a700 1 -- start start 2023-11-24T07:56:28.296 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:28.289+0000 7f6f1031a700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6ef4004850 0x7f6f08150f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:56:28.296 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:28.289+0000 7f6f1031a700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6f0815c550 0x7f6f080b6e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:56:28.296 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:28.289+0000 7f6f1031a700 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f6f080b73d0 0x7f6f080b11a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:56:28.296 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:28.289+0000 7f6f1031a700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f080b2980 con 0x7f6ef4004850 2023-11-24T07:56:28.296 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:28.289+0000 7f6f1031a700 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f080b1d70 con 0x7f6f0815c550 2023-11-24T07:56:28.296 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:28.289+0000 7f6f1031a700 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6f080b2050 con 0x7f6f080b73d0 2023-11-24T07:56:28.296 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:28.289+0000 7f6f0e0b6700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6ef4004850 0x7f6f08150f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:56:28.296 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:28.289+0000 7f6f0e0b6700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6ef4004850 0x7f6f08150f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:49754/0 (socket says 172.21.15.59:49754) 2023-11-24T07:56:28.296 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:28.289+0000 7f6f0e0b6700 1 -- 172.21.15.59:0/2165898700 learned_addr learned my addr 172.21.15.59:0/2165898700 (peer_addr_for_me v2:172.21.15.59:0/0) 2023-11-24T07:56:28.296 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:28.290+0000 7f6f0e8b7700 1 --2- 172.21.15.59:0/2165898700 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f6f080b73d0 0x7f6f080b11a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:56:28.296 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:28.290+0000 7f6f0d8b5700 1 --2- 172.21.15.59:0/2165898700 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6f0815c550 0x7f6f080b6e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:56:28.296 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:28.290+0000 7f6f0e0b6700 1 -- 172.21.15.59:0/2165898700 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f6f080b73d0 msgr2=0x7f6f080b11a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:56:28.297 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:28.290+0000 7f6f0e0b6700 1 --2- 172.21.15.59:0/2165898700 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f6f080b73d0 0x7f6f080b11a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:56:28.297 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:28.290+0000 7f6f0e0b6700 1 -- 172.21.15.59:0/2165898700 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6f0815c550 msgr2=0x7f6f080b6e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:56:28.297 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:28.290+0000 7f6f0e0b6700 1 --2- 172.21.15.59:0/2165898700 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6f0815c550 0x7f6f080b6e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:56:28.297 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:28.290+0000 7f6f0e0b6700 1 -- 172.21.15.59:0/2165898700 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6f080b2330 con 0x7f6ef4004850 2023-11-24T07:56:28.297 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:28.290+0000 7f6f0e8b7700 1 --2- 172.21.15.59:0/2165898700 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f6f080b73d0 0x7f6f080b11a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T07:56:28.297 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:28.290+0000 7f6f0e0b6700 1 --2- 172.21.15.59:0/2165898700 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6ef4004850 0x7f6f08150f30 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f6ef80072c0 tx=0x7f6ef8007680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:56:28.297 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:28.290+0000 7f6efeffd700 1 -- 172.21.15.59:0/2165898700 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ef800f8d0 con 0x7f6ef4004850 2023-11-24T07:56:28.297 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:28.290+0000 7f6efeffd700 1 -- 172.21.15.59:0/2165898700 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6ef800fa30 con 0x7f6ef4004850 2023-11-24T07:56:28.297 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:28.290+0000 7f6efeffd700 1 -- 172.21.15.59:0/2165898700 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ef800fde0 con 0x7f6ef4004850 2023-11-24T07:56:28.297 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:28.290+0000 7f6f1031a700 1 -- 172.21.15.59:0/2165898700 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6f080b2610 con 0x7f6ef4004850 2023-11-24T07:56:28.297 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:28.290+0000 7f6f1031a700 1 -- 172.21.15.59:0/2165898700 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6f08158300 con 0x7f6ef4004850 2023-11-24T07:56:28.298 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:28.291+0000 7f6f1031a700 1 -- 172.21.15.59:0/2165898700 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6f08068670 con 0x7f6ef4004850 2023-11-24T07:56:28.298 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:28.292+0000 7f6efeffd700 1 -- 172.21.15.59:0/2165898700 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f6ef801d460 con 0x7f6ef4004850 2023-11-24T07:56:28.298 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:28.292+0000 7f6efeffd700 1 --2- 172.21.15.59:0/2165898700 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f6eec06aed0 0x7f6eec06d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:56:28.298 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:28.292+0000 7f6efeffd700 1 -- 172.21.15.59:0/2165898700 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f6ef80896c0 con 0x7f6ef4004850 2023-11-24T07:56:28.298 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:28.292+0000 7f6f0d8b5700 1 --2- 172.21.15.59:0/2165898700 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f6eec06aed0 0x7f6eec06d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:56:28.298 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:28.293+0000 7f6f0d8b5700 1 --2- 172.21.15.59:0/2165898700 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f6eec06aed0 0x7f6eec06d390 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f6f081f73a0 tx=0x7f6f0401e040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:56:28.298 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:28.296+0000 7f6efeffd700 1 -- 172.21.15.59:0/2165898700 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f6ef8054770 con 0x7f6ef4004850 2023-11-24T07:56:28.501 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:28.499+0000 7f6f1031a700 1 -- 172.21.15.59:0/2165898700 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f6f08067a50 con 0x7f6ef4004850 2023-11-24T07:56:28.502 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:28.500+0000 7f6efeffd700 1 -- 172.21.15.59:0/2165898700 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) v1 ==== 74+0+16437 (secure 0 0 0) 0x7f6ef8057d90 con 0x7f6ef4004850 2023-11-24T07:56:28.502 INFO:teuthology.orchestra.run.smithi059.stdout: 2023-11-24T07:56:28.502 INFO:teuthology.orchestra.run.smithi059.stdout:{"epoch":20,"fsid":"9f0e8741-b8f9-48d5-a2c3-09f45903303d","created":"2023-11-24T07:56:06.552729+0000","modified":"2023-11-24T07:56:25.705091+0000","last_up_change":"2023-11-24T07:56:17.372352+0000","last_in_change":"2023-11-24T07:56:09.073250+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-24T07:56:18.352823+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-11-24T07:56:22.185140+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-11-24T07:56:23.084326+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"6d8f9e9b-1782-4e41-928b-a39f2a8ebc06","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6800","nonce":3373878772},{"type":"v1","addr":"172.21.15.59:6801","nonce":3373878772}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6802","nonce":3373878772},{"type":"v1","addr":"172.21.15.59:6803","nonce":3373878772}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6806","nonce":3373878772},{"type":"v1","addr":"172.21.15.59:6807","nonce":3373878772}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6804","nonce":3373878772},{"type":"v1","addr":"172.21.15.59:6805","nonce":3373878772}]},"public_addr":"172.21.15.59:6801/3373878772","cluster_addr":"172.21.15.59:6803/3373878772","heartbeat_back_addr":"172.21.15.59:6807/3373878772","heartbeat_front_addr":"172.21.15.59:6805/3373878772","state":["exists","up"]},{"osd":1,"uuid":"e8d882aa-52e1-4c04-9e1e-c44ef9204a5f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6824","nonce":648258563},{"type":"v1","addr":"172.21.15.59:6825","nonce":648258563}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6826","nonce":648258563},{"type":"v1","addr":"172.21.15.59:6827","nonce":648258563}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6830","nonce":648258563},{"type":"v1","addr":"172.21.15.59:6831","nonce":648258563}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6828","nonce":648258563},{"type":"v1","addr":"172.21.15.59:6829","nonce":648258563}]},"public_addr":"172.21.15.59:6825/648258563","cluster_addr":"172.21.15.59:6827/648258563","heartbeat_back_addr":"172.21.15.59:6831/648258563","heartbeat_front_addr":"172.21.15.59:6829/648258563","state":["exists","up"]},{"osd":2,"uuid":"69eebfa0-695b-4b21-90b9-8a3537168a01","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6808","nonce":1840430654},{"type":"v1","addr":"172.21.15.59:6809","nonce":1840430654}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6810","nonce":1840430654},{"type":"v1","addr":"172.21.15.59:6811","nonce":1840430654}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6814","nonce":1840430654},{"type":"v1","addr":"172.21.15.59:6815","nonce":1840430654}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6812","nonce":1840430654},{"type":"v1","addr":"172.21.15.59:6813","nonce":1840430654}]},"public_addr":"172.21.15.59:6809/1840430654","cluster_addr":"172.21.15.59:6811/1840430654","heartbeat_back_addr":"172.21.15.59:6815/1840430654","heartbeat_front_addr":"172.21.15.59:6813/1840430654","state":["exists","up"]},{"osd":3,"uuid":"96fd77fc-f911-46ff-9445-6f0be2daa8f4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6816","nonce":2504025850},{"type":"v1","addr":"172.21.15.59:6817","nonce":2504025850}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6818","nonce":2504025850},{"type":"v1","addr":"172.21.15.59:6819","nonce":2504025850}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6822","nonce":2504025850},{"type":"v1","addr":"172.21.15.59:6823","nonce":2504025850}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6820","nonce":2504025850},{"type":"v1","addr":"172.21.15.59:6821","nonce":2504025850}]},"public_addr":"172.21.15.59:6817/2504025850","cluster_addr":"172.21.15.59:6819/2504025850","heartbeat_back_addr":"172.21.15.59:6823/2504025850","heartbeat_front_addr":"172.21.15.59:6821/2504025850","state":["exists","up"]},{"osd":4,"uuid":"7ac29fca-b5b2-4c95-9d69-2f164cbbc708","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6816","nonce":2382463431},{"type":"v1","addr":"172.21.15.76:6818","nonce":2382463431}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6820","nonce":2382463431},{"type":"v1","addr":"172.21.15.76:6822","nonce":2382463431}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6828","nonce":2382463431},{"type":"v1","addr":"172.21.15.76:6830","nonce":2382463431}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6824","nonce":2382463431},{"type":"v1","addr":"172.21.15.76:6826","nonce":2382463431}]},"public_addr":"172.21.15.76:6818/2382463431","cluster_addr":"172.21.15.76:6822/2382463431","heartbeat_back_addr":"172.21.15.76:6830/2382463431","heartbeat_front_addr":"172.21.15.76:6826/2382463431","state":["exists","up"]},{"osd":5,"uuid":"485bf711-3851-4238-a4f4-51d52e0a9c58","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6800","nonce":329334308},{"type":"v1","addr":"172.21.15.76:6802","nonce":329334308}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6805","nonce":329334308},{"type":"v1","addr":"172.21.15.76:6807","nonce":329334308}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6814","nonce":329334308},{"type":"v1","addr":"172.21.15.76:6815","nonce":329334308}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6810","nonce":329334308},{"type":"v1","addr":"172.21.15.76:6812","nonce":329334308}]},"public_addr":"172.21.15.76:6802/329334308","cluster_addr":"172.21.15.76:6807/329334308","heartbeat_back_addr":"172.21.15.76:6815/329334308","heartbeat_front_addr":"172.21.15.76:6812/329334308","state":["exists","up"]},{"osd":6,"uuid":"72b3f2fc-1e4a-4e32-968d-003647a0286f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6801","nonce":3946579410},{"type":"v1","addr":"172.21.15.76:6803","nonce":3946579410}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6804","nonce":3946579410},{"type":"v1","addr":"172.21.15.76:6806","nonce":3946579410}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6811","nonce":3946579410},{"type":"v1","addr":"172.21.15.76:6813","nonce":3946579410}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6808","nonce":3946579410},{"type":"v1","addr":"172.21.15.76:6809","nonce":3946579410}]},"public_addr":"172.21.15.76:6803/3946579410","cluster_addr":"172.21.15.76:6806/3946579410","heartbeat_back_addr":"172.21.15.76:6813/3946579410","heartbeat_front_addr":"172.21.15.76:6809/3946579410","state":["exists","up"]},{"osd":7,"uuid":"1ddceb90-6c5d-4d1a-ad56-99cb877f8c70","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6817","nonce":841761538},{"type":"v1","addr":"172.21.15.76:6819","nonce":841761538}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6821","nonce":841761538},{"type":"v1","addr":"172.21.15.76:6823","nonce":841761538}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6829","nonce":841761538},{"type":"v1","addr":"172.21.15.76:6831","nonce":841761538}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6825","nonce":841761538},{"type":"v1","addr":"172.21.15.76:6827","nonce":841761538}]},"public_addr":"172.21.15.76:6819/841761538","cluster_addr":"172.21.15.76:6823/841761538","heartbeat_back_addr":"172.21.15.76:6831/841761538","heartbeat_front_addr":"172.21.15.76:6827/841761538","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T07:56:14.294838+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T07:56:14.270779+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T07:56:14.231809+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T07:56:14.260983+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T07:56:14.302997+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T07:56:14.258600+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T07:56:14.251413+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T07:56:14.304727+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-24T07:56:28.505 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:28.503+0000 7f6f1031a700 1 -- 172.21.15.59:0/2165898700 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f6eec06aed0 msgr2=0x7f6eec06d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:56:28.505 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:28.503+0000 7f6f1031a700 1 --2- 172.21.15.59:0/2165898700 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f6eec06aed0 0x7f6eec06d390 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f6f081f73a0 tx=0x7f6f0401e040 comp rx=0 tx=0).stop 2023-11-24T07:56:28.505 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:28.503+0000 7f6f1031a700 1 -- 172.21.15.59:0/2165898700 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6ef4004850 msgr2=0x7f6f08150f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:56:28.505 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:28.503+0000 7f6f1031a700 1 --2- 172.21.15.59:0/2165898700 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6ef4004850 0x7f6f08150f30 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f6ef80072c0 tx=0x7f6ef8007680 comp rx=0 tx=0).stop 2023-11-24T07:56:28.505 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:28.503+0000 7f6f1031a700 1 -- 172.21.15.59:0/2165898700 shutdown_connections 2023-11-24T07:56:28.505 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:28.503+0000 7f6f1031a700 1 --2- 172.21.15.59:0/2165898700 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f6eec06aed0 0x7f6eec06d390 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:56:28.506 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:28.503+0000 7f6f1031a700 1 --2- 172.21.15.59:0/2165898700 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f6f080b73d0 0x7f6f080b11a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:56:28.506 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:28.503+0000 7f6f1031a700 1 --2- 172.21.15.59:0/2165898700 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6ef4004850 0x7f6f08150f30 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:56:28.506 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:28.503+0000 7f6f1031a700 1 --2- 172.21.15.59:0/2165898700 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6f0815c550 0x7f6f080b6e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:56:28.506 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:28.503+0000 7f6f1031a700 1 -- 172.21.15.59:0/2165898700 >> 172.21.15.59:0/2165898700 conn(0x7f6f080b3720 msgr2=0x7f6f08070470 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:56:28.506 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:28.503+0000 7f6f1031a700 1 -- 172.21.15.59:0/2165898700 shutdown_connections 2023-11-24T07:56:28.506 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:28.503+0000 7f6f1031a700 1 -- 172.21.15.59:0/2165898700 wait complete. 2023-11-24T07:56:28.517 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2023-11-24T07:56:28.517 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-11-24T07:56:28.642 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:28.640+0000 7f5683ebd700 1 Processor -- start 2023-11-24T07:56:28.642 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:28.640+0000 7f5683ebd700 1 -- start start 2023-11-24T07:56:28.644 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:28.641+0000 7f5683ebd700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f567c0d0820 0x7f567c166c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:56:28.644 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:28.641+0000 7f5683ebd700 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f567c06f610 con 0x7f567c0cfc60 2023-11-24T07:56:28.644 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:28.641+0000 7f5683ebd700 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f567c0cdac0 con 0x7f567c15c010 2023-11-24T07:56:28.644 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:28.641+0000 7f5683ebd700 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f567c06ece0 con 0x7f567c0d0820 2023-11-24T07:56:28.644 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:28.641+0000 7f568245a700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f567c0d0820 0x7f567c166c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:56:28.644 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:28.641+0000 7f568245a700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f567c0d0820 0x7f567c166c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:49764/0 (socket says 172.21.15.59:49764) 2023-11-24T07:56:28.644 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:28.641+0000 7f568245a700 1 -- 172.21.15.59:0/2025923810 learned_addr learned my addr 172.21.15.59:0/2025923810 (peer_addr_for_me v2:172.21.15.59:0/0) 2023-11-24T07:56:28.644 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:28.641+0000 7f5681458700 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7f567c0cfc60 0x7f567c0d0110 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:53236/0 (socket says 172.21.15.59:53236) 2023-11-24T07:56:28.644 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:28.642+0000 7f568245a700 1 -- 172.21.15.59:0/2025923810 >> v1:172.21.15.76:6789/0 conn(0x7f567c15c010 legacy=0x7f567c0cd2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T07:56:28.644 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:28.642+0000 7f568245a700 1 -- 172.21.15.59:0/2025923810 >> v1:172.21.15.59:6789/0 conn(0x7f567c0cfc60 legacy=0x7f567c0d0110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:56:28.645 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:28.642+0000 7f568245a700 1 -- 172.21.15.59:0/2025923810 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f567c06efc0 con 0x7f567c0d0820 2023-11-24T07:56:28.645 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:28.642+0000 7f5680c57700 1 -- 172.21.15.59:0/2025923810 <== mon.0 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3148783171 0 0) 0x7f567c06f610 con 0x7f567c0cfc60 2023-11-24T07:56:28.645 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:28.642+0000 7f568245a700 1 --2- 172.21.15.59:0/2025923810 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f567c0d0820 0x7f567c166c00 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f5674009e50 tx=0x7f567400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=39a5ec0311bc804c server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:56:28.645 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:28.642+0000 7f5680c57700 1 -- 172.21.15.59:0/2025923810 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f567400e070 con 0x7f567c0d0820 2023-11-24T07:56:28.645 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:28.642+0000 7f5680c57700 1 -- 172.21.15.59:0/2025923810 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f567400b9c0 con 0x7f567c0d0820 2023-11-24T07:56:28.645 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:28.642+0000 7f5680c57700 1 -- 172.21.15.59:0/2025923810 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f567400bd70 con 0x7f567c0d0820 2023-11-24T07:56:28.646 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:28.643+0000 7f5683ebd700 1 -- 172.21.15.59:0/2025923810 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f567c0d0820 msgr2=0x7f567c166c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:56:28.646 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:28.643+0000 7f5683ebd700 1 --2- 172.21.15.59:0/2025923810 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f567c0d0820 0x7f567c166c00 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f5674009e50 tx=0x7f567400b330 comp rx=0 tx=0).stop 2023-11-24T07:56:28.646 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:28.643+0000 7f5683ebd700 1 -- 172.21.15.59:0/2025923810 shutdown_connections 2023-11-24T07:56:28.647 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:28.643+0000 7f5683ebd700 1 --2- 172.21.15.59:0/2025923810 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f567c0d0820 0x7f567c166c00 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:56:28.647 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:28.643+0000 7f5683ebd700 1 -- 172.21.15.59:0/2025923810 >> 172.21.15.59:0/2025923810 conn(0x7f567c0b3790 msgr2=0x7f567c0b3ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:56:28.647 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:28.643+0000 7f5683ebd700 1 -- 172.21.15.59:0/2025923810 shutdown_connections 2023-11-24T07:56:28.647 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:28.643+0000 7f5683ebd700 1 -- 172.21.15.59:0/2025923810 wait complete. 2023-11-24T07:56:28.647 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:28.644+0000 7f5683ebd700 1 Processor -- start 2023-11-24T07:56:28.647 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:28.644+0000 7f5683ebd700 1 -- start start 2023-11-24T07:56:28.650 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:28.645+0000 7f5683ebd700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f567c0cfc60 0x7f567c157380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:56:28.650 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:28.645+0000 7f5683ebd700 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f567c0d0820 0x7f567c159b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:56:28.650 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:28.645+0000 7f5683ebd700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f567c15c010 0x7f567c15a0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:56:28.650 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:28.645+0000 7f5683ebd700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f567c15af30 con 0x7f567c15c010 2023-11-24T07:56:28.650 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:28.645+0000 7f5683ebd700 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f567c0c7740 con 0x7f567c0cfc60 2023-11-24T07:56:28.650 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:28.645+0000 7f5683ebd700 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f567c15ac50 con 0x7f567c0d0820 2023-11-24T07:56:28.650 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:28.645+0000 7f568245a700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f567c15c010 0x7f567c15a0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:56:28.651 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:28.645+0000 7f5681c59700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f567c0cfc60 0x7f567c157380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:56:28.651 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:28.645+0000 7f568245a700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f567c15c010 0x7f567c15a0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:49770/0 (socket says 172.21.15.59:49770) 2023-11-24T07:56:28.651 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:28.645+0000 7f568245a700 1 -- 172.21.15.59:0/1950778504 learned_addr learned my addr 172.21.15.59:0/1950778504 (peer_addr_for_me v2:172.21.15.59:0/0) 2023-11-24T07:56:28.651 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:28.645+0000 7f5681458700 1 --2- 172.21.15.59:0/1950778504 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f567c0d0820 0x7f567c159b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:56:28.651 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:28.646+0000 7f568245a700 1 -- 172.21.15.59:0/1950778504 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f567c0d0820 msgr2=0x7f567c159b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:56:28.651 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:28.646+0000 7f568245a700 1 --2- 172.21.15.59:0/1950778504 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f567c0d0820 0x7f567c159b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:56:28.651 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:28.646+0000 7f568245a700 1 -- 172.21.15.59:0/1950778504 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f567c0cfc60 msgr2=0x7f567c157380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:56:28.651 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:28.646+0000 7f568245a700 1 --2- 172.21.15.59:0/1950778504 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f567c0cfc60 0x7f567c157380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:56:28.651 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:28.646+0000 7f568245a700 1 -- 172.21.15.59:0/1950778504 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5674009b30 con 0x7f567c15c010 2023-11-24T07:56:28.652 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:28.646+0000 7f568245a700 1 --2- 172.21.15.59:0/1950778504 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f567c15c010 0x7f567c15a0d0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f567400b730 tx=0x7f5674004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:56:28.652 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:28.646+0000 7f56727fc700 1 -- 172.21.15.59:0/1950778504 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f567401c640 con 0x7f567c15c010 2023-11-24T07:56:28.652 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:28.646+0000 7f56727fc700 1 -- 172.21.15.59:0/1950778504 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f567401c7c0 con 0x7f567c15c010 2023-11-24T07:56:28.652 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:28.646+0000 7f56727fc700 1 -- 172.21.15.59:0/1950778504 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f567401cb70 con 0x7f567c15c010 2023-11-24T07:56:28.652 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:28.646+0000 7f5683ebd700 1 -- 172.21.15.59:0/1950778504 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f567c0c8e50 con 0x7f567c15c010 2023-11-24T07:56:28.652 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:28.646+0000 7f5683ebd700 1 -- 172.21.15.59:0/1950778504 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f567c0c9410 con 0x7f567c15c010 2023-11-24T07:56:28.652 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:28.647+0000 7f5683ebd700 1 -- 172.21.15.59:0/1950778504 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f567c068670 con 0x7f567c15c010 2023-11-24T07:56:28.652 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:28.647+0000 7f56727fc700 1 -- 172.21.15.59:0/1950778504 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f567401ccd0 con 0x7f567c15c010 2023-11-24T07:56:28.652 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:28.648+0000 7f56727fc700 1 --2- 172.21.15.59:0/1950778504 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f566006aed0 0x7f566006d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:56:28.652 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:28.648+0000 7f56727fc700 1 -- 172.21.15.59:0/1950778504 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f567408b180 con 0x7f567c15c010 2023-11-24T07:56:28.653 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:28.648+0000 7f5681c59700 1 --2- 172.21.15.59:0/1950778504 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f566006aed0 0x7f566006d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:56:28.653 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:28.648+0000 7f5681c59700 1 --2- 172.21.15.59:0/1950778504 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f566006aed0 0x7f566006d390 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f5678007fd0 tx=0x7f5678009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:56:28.654 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:28.652+0000 7f56727fc700 1 -- 172.21.15.59:0/1950778504 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f56740563e0 con 0x7f567c15c010 2023-11-24T07:56:28.850 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:28.849+0000 7f5683ebd700 1 -- 172.21.15.59:0/1950778504 --> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f567c0c8cd0 con 0x7f566006aed0 2023-11-24T07:56:28.857 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:28.856+0000 7f56727fc700 1 -- 172.21.15.59:0/1950778504 <== mgr.4100 v2:172.21.15.76:6832/4264046536 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+252811 (secure 0 0 0) 0x7f5678001cc0 con 0x7f566006aed0 2023-11-24T07:56:28.861 INFO:teuthology.orchestra.run.smithi059.stdout: 2023-11-24T07:56:28.863 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:28.859+0000 7f5683ebd700 1 -- 172.21.15.59:0/1950778504 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f566006aed0 msgr2=0x7f566006d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:56:28.864 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:28.859+0000 7f5683ebd700 1 --2- 172.21.15.59:0/1950778504 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f566006aed0 0x7f566006d390 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f5678007fd0 tx=0x7f5678009040 comp rx=0 tx=0).stop 2023-11-24T07:56:28.864 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:28.859+0000 7f5683ebd700 1 -- 172.21.15.59:0/1950778504 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f567c15c010 msgr2=0x7f567c15a0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:56:28.864 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:28.859+0000 7f5683ebd700 1 --2- 172.21.15.59:0/1950778504 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f567c15c010 0x7f567c15a0d0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f567400b730 tx=0x7f5674004ad0 comp rx=0 tx=0).stop 2023-11-24T07:56:28.864 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:28.859+0000 7f5683ebd700 1 -- 172.21.15.59:0/1950778504 shutdown_connections 2023-11-24T07:56:28.864 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:28.859+0000 7f5683ebd700 1 --2- 172.21.15.59:0/1950778504 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f566006aed0 0x7f566006d390 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:56:28.865 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:28.859+0000 7f5683ebd700 1 --2- 172.21.15.59:0/1950778504 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f567c15c010 0x7f567c15a0d0 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:56:28.865 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:28.859+0000 7f5683ebd700 1 --2- 172.21.15.59:0/1950778504 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f567c0cfc60 0x7f567c157380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:56:28.865 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:28.859+0000 7f5683ebd700 1 --2- 172.21.15.59:0/1950778504 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f567c0d0820 0x7f567c159b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:56:28.865 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:28.859+0000 7f5683ebd700 1 -- 172.21.15.59:0/1950778504 >> 172.21.15.59:0/1950778504 conn(0x7f567c0b3790 msgr2=0x7f567c0cde00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:56:28.865 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:28.859+0000 7f5683ebd700 1 -- 172.21.15.59:0/1950778504 shutdown_connections 2023-11-24T07:56:28.865 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:28.860+0000 7f5683ebd700 1 -- 172.21.15.59:0/1950778504 wait complete. 2023-11-24T07:56:28.865 INFO:teuthology.orchestra.run.smithi059.stderr:dumped all 2023-11-24T07:56:28.873 INFO:teuthology.orchestra.run.smithi059.stdout:{"pg_ready":true,"pg_map":{"version":18,"stamp":"2023-11-24T07:56:28.298177+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":66,"acting":66,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":14,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":218596,"kb_used_data":2436,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754756124,"statfs":{"total":773094113280,"available":772870270976,"internally_reserved":0,"allocated":2494464,"data_stored":1575765,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12745,"internal_metadata":221236791},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":14,"apply_latency_ms":14,"commit_latency_ns":14000000,"apply_latency_ns":14000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.611089"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T07:56:23.683366+0000","last_change":"2023-11-24T07:56:23.683366+0000","last_active":"2023-11-24T07:56:23.683366+0000","last_peered":"2023-11-24T07:56:23.683366+0000","last_clean":"2023-11-24T07:56:23.683366+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T07:56:23.683366+0000","last_undegraded":"2023-11-24T07:56:23.683366+0000","last_fullsized":"2023-11-24T07:56:23.683366+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:56:23.683366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:56:23.683366+0000","last_clean_scrub_stamp":"2023-11-24T07:56:23.683366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T07:56:23.683366+0000","last_change":"2023-11-24T07:56:23.683366+0000","last_active":"2023-11-24T07:56:23.683366+0000","last_peered":"2023-11-24T07:56:23.683366+0000","last_clean":"2023-11-24T07:56:23.683366+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T07:56:23.683366+0000","last_undegraded":"2023-11-24T07:56:23.683366+0000","last_fullsized":"2023-11-24T07:56:23.683366+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:56:23.683366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:56:23.683366+0000","last_clean_scrub_stamp":"2023-11-24T07:56:23.683366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T07:56:23.683366+0000","last_change":"2023-11-24T07:56:23.683366+0000","last_active":"2023-11-24T07:56:23.683366+0000","last_peered":"2023-11-24T07:56:23.683366+0000","last_clean":"2023-11-24T07:56:23.683366+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T07:56:23.683366+0000","last_undegraded":"2023-11-24T07:56:23.683366+0000","last_fullsized":"2023-11-24T07:56:23.683366+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:56:23.683366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:56:23.683366+0000","last_clean_scrub_stamp":"2023-11-24T07:56:23.683366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T07:56:23.683366+0000","last_change":"2023-11-24T07:56:23.683366+0000","last_active":"2023-11-24T07:56:23.683366+0000","last_peered":"2023-11-24T07:56:23.683366+0000","last_clean":"2023-11-24T07:56:23.683366+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T07:56:23.683366+0000","last_undegraded":"2023-11-24T07:56:23.683366+0000","last_fullsized":"2023-11-24T07:56:23.683366+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:56:23.683366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:56:23.683366+0000","last_clean_scrub_stamp":"2023-11-24T07:56:23.683366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T07:56:23.683366+0000","last_change":"2023-11-24T07:56:23.683366+0000","last_active":"2023-11-24T07:56:23.683366+0000","last_peered":"2023-11-24T07:56:23.683366+0000","last_clean":"2023-11-24T07:56:23.683366+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T07:56:23.683366+0000","last_undegraded":"2023-11-24T07:56:23.683366+0000","last_fullsized":"2023-11-24T07:56:23.683366+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:56:23.683366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:56:23.683366+0000","last_clean_scrub_stamp":"2023-11-24T07:56:23.683366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T07:56:23.683366+0000","last_change":"2023-11-24T07:56:23.683366+0000","last_active":"2023-11-24T07:56:23.683366+0000","last_peered":"2023-11-24T07:56:23.683366+0000","last_clean":"2023-11-24T07:56:23.683366+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T07:56:23.683366+0000","last_undegraded":"2023-11-24T07:56:23.683366+0000","last_fullsized":"2023-11-24T07:56:23.683366+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:56:23.683366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:56:23.683366+0000","last_clean_scrub_stamp":"2023-11-24T07:56:23.683366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T07:56:23.683366+0000","last_change":"2023-11-24T07:56:23.683366+0000","last_active":"2023-11-24T07:56:23.683366+0000","last_peered":"2023-11-24T07:56:23.683366+0000","last_clean":"2023-11-24T07:56:23.683366+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T07:56:23.683366+0000","last_undegraded":"2023-11-24T07:56:23.683366+0000","last_fullsized":"2023-11-24T07:56:23.683366+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:56:23.683366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:56:23.683366+0000","last_clean_scrub_stamp":"2023-11-24T07:56:23.683366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T07:56:23.683366+0000","last_change":"2023-11-24T07:56:23.683366+0000","last_active":"2023-11-24T07:56:23.683366+0000","last_peered":"2023-11-24T07:56:23.683366+0000","last_clean":"2023-11-24T07:56:23.683366+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T07:56:23.683366+0000","last_undegraded":"2023-11-24T07:56:23.683366+0000","last_fullsized":"2023-11-24T07:56:23.683366+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:56:23.683366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:56:23.683366+0000","last_clean_scrub_stamp":"2023-11-24T07:56:23.683366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T07:56:23.683366+0000","last_change":"2023-11-24T07:56:23.683366+0000","last_active":"2023-11-24T07:56:23.683366+0000","last_peered":"2023-11-24T07:56:23.683366+0000","last_clean":"2023-11-24T07:56:23.683366+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T07:56:23.683366+0000","last_undegraded":"2023-11-24T07:56:23.683366+0000","last_fullsized":"2023-11-24T07:56:23.683366+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:56:23.683366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:56:23.683366+0000","last_clean_scrub_stamp":"2023-11-24T07:56:23.683366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T07:56:23.683366+0000","last_change":"2023-11-24T07:56:23.683366+0000","last_active":"2023-11-24T07:56:23.683366+0000","last_peered":"2023-11-24T07:56:23.683366+0000","last_clean":"2023-11-24T07:56:23.683366+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T07:56:23.683366+0000","last_undegraded":"2023-11-24T07:56:23.683366+0000","last_fullsized":"2023-11-24T07:56:23.683366+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:56:23.683366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:56:23.683366+0000","last_clean_scrub_stamp":"2023-11-24T07:56:23.683366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T07:56:23.683366+0000","last_change":"2023-11-24T07:56:23.683366+0000","last_active":"2023-11-24T07:56:23.683366+0000","last_peered":"2023-11-24T07:56:23.683366+0000","last_clean":"2023-11-24T07:56:23.683366+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T07:56:23.683366+0000","last_undegraded":"2023-11-24T07:56:23.683366+0000","last_fullsized":"2023-11-24T07:56:23.683366+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:56:23.683366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:56:23.683366+0000","last_clean_scrub_stamp":"2023-11-24T07:56:23.683366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T07:56:23.683366+0000","last_change":"2023-11-24T07:56:23.683366+0000","last_active":"2023-11-24T07:56:23.683366+0000","last_peered":"2023-11-24T07:56:23.683366+0000","last_clean":"2023-11-24T07:56:23.683366+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T07:56:23.683366+0000","last_undegraded":"2023-11-24T07:56:23.683366+0000","last_fullsized":"2023-11-24T07:56:23.683366+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:56:23.683366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:56:23.683366+0000","last_clean_scrub_stamp":"2023-11-24T07:56:23.683366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T07:56:23.683366+0000","last_change":"2023-11-24T07:56:23.683366+0000","last_active":"2023-11-24T07:56:23.683366+0000","last_peered":"2023-11-24T07:56:23.683366+0000","last_clean":"2023-11-24T07:56:23.683366+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T07:56:23.683366+0000","last_undegraded":"2023-11-24T07:56:23.683366+0000","last_fullsized":"2023-11-24T07:56:23.683366+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:56:23.683366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:56:23.683366+0000","last_clean_scrub_stamp":"2023-11-24T07:56:23.683366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T07:56:23.683366+0000","last_change":"2023-11-24T07:56:23.683366+0000","last_active":"2023-11-24T07:56:23.683366+0000","last_peered":"2023-11-24T07:56:23.683366+0000","last_clean":"2023-11-24T07:56:23.683366+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T07:56:23.683366+0000","last_undegraded":"2023-11-24T07:56:23.683366+0000","last_fullsized":"2023-11-24T07:56:23.683366+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:56:23.683366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:56:23.683366+0000","last_clean_scrub_stamp":"2023-11-24T07:56:23.683366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T07:56:23.683366+0000","last_change":"2023-11-24T07:56:23.683366+0000","last_active":"2023-11-24T07:56:23.683366+0000","last_peered":"2023-11-24T07:56:23.683366+0000","last_clean":"2023-11-24T07:56:23.683366+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T07:56:23.683366+0000","last_undegraded":"2023-11-24T07:56:23.683366+0000","last_fullsized":"2023-11-24T07:56:23.683366+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:56:23.683366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:56:23.683366+0000","last_clean_scrub_stamp":"2023-11-24T07:56:23.683366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T07:56:23.683366+0000","last_change":"2023-11-24T07:56:23.683366+0000","last_active":"2023-11-24T07:56:23.683366+0000","last_peered":"2023-11-24T07:56:23.683366+0000","last_clean":"2023-11-24T07:56:23.683366+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T07:56:23.683366+0000","last_undegraded":"2023-11-24T07:56:23.683366+0000","last_fullsized":"2023-11-24T07:56:23.683366+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:56:23.683366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:56:23.683366+0000","last_clean_scrub_stamp":"2023-11-24T07:56:23.683366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T07:56:23.683366+0000","last_change":"2023-11-24T07:56:23.683366+0000","last_active":"2023-11-24T07:56:23.683366+0000","last_peered":"2023-11-24T07:56:23.683366+0000","last_clean":"2023-11-24T07:56:23.683366+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T07:56:23.683366+0000","last_undegraded":"2023-11-24T07:56:23.683366+0000","last_fullsized":"2023-11-24T07:56:23.683366+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:56:23.683366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:56:23.683366+0000","last_clean_scrub_stamp":"2023-11-24T07:56:23.683366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T07:56:23.683366+0000","last_change":"2023-11-24T07:56:23.683366+0000","last_active":"2023-11-24T07:56:23.683366+0000","last_peered":"2023-11-24T07:56:23.683366+0000","last_clean":"2023-11-24T07:56:23.683366+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T07:56:23.683366+0000","last_undegraded":"2023-11-24T07:56:23.683366+0000","last_fullsized":"2023-11-24T07:56:23.683366+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:56:23.683366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:56:23.683366+0000","last_clean_scrub_stamp":"2023-11-24T07:56:23.683366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T07:56:23.683366+0000","last_change":"2023-11-24T07:56:23.683366+0000","last_active":"2023-11-24T07:56:23.683366+0000","last_peered":"2023-11-24T07:56:23.683366+0000","last_clean":"2023-11-24T07:56:23.683366+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T07:56:23.683366+0000","last_undegraded":"2023-11-24T07:56:23.683366+0000","last_fullsized":"2023-11-24T07:56:23.683366+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:56:23.683366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:56:23.683366+0000","last_clean_scrub_stamp":"2023-11-24T07:56:23.683366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1f","version":"0'0","reported_seq":1,"reported_epoch":17,"state":"creating+peering","last_fresh":"2023-11-24T07:56:22.690428+0000","last_change":"2023-11-24T07:56:22.690428+0000","last_active":"2023-11-24T07:56:22.678019+0000","last_peered":"2023-11-24T07:56:22.678019+0000","last_clean":"2023-11-24T07:56:22.678019+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T07:56:22.690428+0000","last_undegraded":"2023-11-24T07:56:22.690428+0000","last_fullsized":"2023-11-24T07:56:22.690428+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:56:22.678019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:56:22.678019+0000","last_clean_scrub_stamp":"2023-11-24T07:56:22.678019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T07:56:23.683366+0000","last_change":"2023-11-24T07:56:23.683366+0000","last_active":"2023-11-24T07:56:23.683366+0000","last_peered":"2023-11-24T07:56:23.683366+0000","last_clean":"2023-11-24T07:56:23.683366+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T07:56:23.683366+0000","last_undegraded":"2023-11-24T07:56:23.683366+0000","last_fullsized":"2023-11-24T07:56:23.683366+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:56:23.683366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:56:23.683366+0000","last_clean_scrub_stamp":"2023-11-24T07:56:23.683366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1e","version":"0'0","reported_seq":1,"reported_epoch":17,"state":"creating+peering","last_fresh":"2023-11-24T07:56:22.686164+0000","last_change":"2023-11-24T07:56:22.686164+0000","last_active":"2023-11-24T07:56:22.678019+0000","last_peered":"2023-11-24T07:56:22.678019+0000","last_clean":"2023-11-24T07:56:22.678019+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T07:56:22.686164+0000","last_undegraded":"2023-11-24T07:56:22.686164+0000","last_fullsized":"2023-11-24T07:56:22.686164+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:56:22.678019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:56:22.678019+0000","last_clean_scrub_stamp":"2023-11-24T07:56:22.678019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T07:56:23.683366+0000","last_change":"2023-11-24T07:56:23.683366+0000","last_active":"2023-11-24T07:56:23.683366+0000","last_peered":"2023-11-24T07:56:23.683366+0000","last_clean":"2023-11-24T07:56:23.683366+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T07:56:23.683366+0000","last_undegraded":"2023-11-24T07:56:23.683366+0000","last_fullsized":"2023-11-24T07:56:23.683366+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:56:23.683366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:56:23.683366+0000","last_clean_scrub_stamp":"2023-11-24T07:56:23.683366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1d","version":"0'0","reported_seq":1,"reported_epoch":17,"state":"creating+peering","last_fresh":"2023-11-24T07:56:22.690762+0000","last_change":"2023-11-24T07:56:22.690762+0000","last_active":"2023-11-24T07:56:22.678019+0000","last_peered":"2023-11-24T07:56:22.678019+0000","last_clean":"2023-11-24T07:56:22.678019+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T07:56:22.690762+0000","last_undegraded":"2023-11-24T07:56:22.690762+0000","last_fullsized":"2023-11-24T07:56:22.690762+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:56:22.678019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:56:22.678019+0000","last_clean_scrub_stamp":"2023-11-24T07:56:22.678019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T07:56:23.683366+0000","last_change":"2023-11-24T07:56:23.683366+0000","last_active":"2023-11-24T07:56:23.683366+0000","last_peered":"2023-11-24T07:56:23.683366+0000","last_clean":"2023-11-24T07:56:23.683366+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T07:56:23.683366+0000","last_undegraded":"2023-11-24T07:56:23.683366+0000","last_fullsized":"2023-11-24T07:56:23.683366+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:56:23.683366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:56:23.683366+0000","last_clean_scrub_stamp":"2023-11-24T07:56:23.683366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1c","version":"0'0","reported_seq":1,"reported_epoch":17,"state":"creating+peering","last_fresh":"2023-11-24T07:56:22.690382+0000","last_change":"2023-11-24T07:56:22.690382+0000","last_active":"2023-11-24T07:56:22.678019+0000","last_peered":"2023-11-24T07:56:22.678019+0000","last_clean":"2023-11-24T07:56:22.678019+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T07:56:22.690382+0000","last_undegraded":"2023-11-24T07:56:22.690382+0000","last_fullsized":"2023-11-24T07:56:22.690382+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:56:22.678019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:56:22.678019+0000","last_clean_scrub_stamp":"2023-11-24T07:56:22.678019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T07:56:23.683366+0000","last_change":"2023-11-24T07:56:23.683366+0000","last_active":"2023-11-24T07:56:23.683366+0000","last_peered":"2023-11-24T07:56:23.683366+0000","last_clean":"2023-11-24T07:56:23.683366+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T07:56:23.683366+0000","last_undegraded":"2023-11-24T07:56:23.683366+0000","last_fullsized":"2023-11-24T07:56:23.683366+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:56:23.683366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:56:23.683366+0000","last_clean_scrub_stamp":"2023-11-24T07:56:23.683366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1b","version":"0'0","reported_seq":1,"reported_epoch":17,"state":"creating+peering","last_fresh":"2023-11-24T07:56:22.686185+0000","last_change":"2023-11-24T07:56:22.686185+0000","last_active":"2023-11-24T07:56:22.678019+0000","last_peered":"2023-11-24T07:56:22.678019+0000","last_clean":"2023-11-24T07:56:22.678019+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T07:56:22.686185+0000","last_undegraded":"2023-11-24T07:56:22.686185+0000","last_fullsized":"2023-11-24T07:56:22.686185+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:56:22.678019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:56:22.678019+0000","last_clean_scrub_stamp":"2023-11-24T07:56:22.678019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T07:56:23.683366+0000","last_change":"2023-11-24T07:56:23.683366+0000","last_active":"2023-11-24T07:56:23.683366+0000","last_peered":"2023-11-24T07:56:23.683366+0000","last_clean":"2023-11-24T07:56:23.683366+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T07:56:23.683366+0000","last_undegraded":"2023-11-24T07:56:23.683366+0000","last_fullsized":"2023-11-24T07:56:23.683366+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:56:23.683366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:56:23.683366+0000","last_clean_scrub_stamp":"2023-11-24T07:56:23.683366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1a","version":"0'0","reported_seq":1,"reported_epoch":17,"state":"creating+peering","last_fresh":"2023-11-24T07:56:22.696543+0000","last_change":"2023-11-24T07:56:22.696543+0000","last_active":"2023-11-24T07:56:22.678019+0000","last_peered":"2023-11-24T07:56:22.678019+0000","last_clean":"2023-11-24T07:56:22.678019+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T07:56:22.696543+0000","last_undegraded":"2023-11-24T07:56:22.696543+0000","last_fullsized":"2023-11-24T07:56:22.696543+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:56:22.678019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:56:22.678019+0000","last_clean_scrub_stamp":"2023-11-24T07:56:22.678019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T07:56:23.683366+0000","last_change":"2023-11-24T07:56:23.683366+0000","last_active":"2023-11-24T07:56:23.683366+0000","last_peered":"2023-11-24T07:56:23.683366+0000","last_clean":"2023-11-24T07:56:23.683366+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T07:56:23.683366+0000","last_undegraded":"2023-11-24T07:56:23.683366+0000","last_fullsized":"2023-11-24T07:56:23.683366+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:56:23.683366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:56:23.683366+0000","last_clean_scrub_stamp":"2023-11-24T07:56:23.683366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":1,"reported_epoch":17,"state":"creating+peering","last_fresh":"2023-11-24T07:56:22.689685+0000","last_change":"2023-11-24T07:56:22.689685+0000","last_active":"2023-11-24T07:56:22.678019+0000","last_peered":"2023-11-24T07:56:22.678019+0000","last_clean":"2023-11-24T07:56:22.678019+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T07:56:22.689685+0000","last_undegraded":"2023-11-24T07:56:22.689685+0000","last_fullsized":"2023-11-24T07:56:22.689685+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:56:22.678019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:56:22.678019+0000","last_clean_scrub_stamp":"2023-11-24T07:56:22.678019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T07:56:23.683366+0000","last_change":"2023-11-24T07:56:23.683366+0000","last_active":"2023-11-24T07:56:23.683366+0000","last_peered":"2023-11-24T07:56:23.683366+0000","last_clean":"2023-11-24T07:56:23.683366+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T07:56:23.683366+0000","last_undegraded":"2023-11-24T07:56:23.683366+0000","last_fullsized":"2023-11-24T07:56:23.683366+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:56:23.683366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:56:23.683366+0000","last_clean_scrub_stamp":"2023-11-24T07:56:23.683366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":1,"reported_epoch":17,"state":"creating+peering","last_fresh":"2023-11-24T07:56:22.695289+0000","last_change":"2023-11-24T07:56:22.695289+0000","last_active":"2023-11-24T07:56:22.678019+0000","last_peered":"2023-11-24T07:56:22.678019+0000","last_clean":"2023-11-24T07:56:22.678019+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T07:56:22.695289+0000","last_undegraded":"2023-11-24T07:56:22.695289+0000","last_fullsized":"2023-11-24T07:56:22.695289+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:56:22.678019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:56:22.678019+0000","last_clean_scrub_stamp":"2023-11-24T07:56:22.678019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T07:56:23.683366+0000","last_change":"2023-11-24T07:56:23.683366+0000","last_active":"2023-11-24T07:56:23.683366+0000","last_peered":"2023-11-24T07:56:23.683366+0000","last_clean":"2023-11-24T07:56:23.683366+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T07:56:23.683366+0000","last_undegraded":"2023-11-24T07:56:23.683366+0000","last_fullsized":"2023-11-24T07:56:23.683366+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:56:23.683366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:56:23.683366+0000","last_clean_scrub_stamp":"2023-11-24T07:56:23.683366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":1,"reported_epoch":17,"state":"creating+peering","last_fresh":"2023-11-24T07:56:22.686064+0000","last_change":"2023-11-24T07:56:22.686064+0000","last_active":"2023-11-24T07:56:22.678019+0000","last_peered":"2023-11-24T07:56:22.678019+0000","last_clean":"2023-11-24T07:56:22.678019+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T07:56:22.686064+0000","last_undegraded":"2023-11-24T07:56:22.686064+0000","last_fullsized":"2023-11-24T07:56:22.686064+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:56:22.678019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:56:22.678019+0000","last_clean_scrub_stamp":"2023-11-24T07:56:22.678019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T07:56:23.683366+0000","last_change":"2023-11-24T07:56:23.683366+0000","last_active":"2023-11-24T07:56:23.683366+0000","last_peered":"2023-11-24T07:56:23.683366+0000","last_clean":"2023-11-24T07:56:23.683366+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T07:56:23.683366+0000","last_undegraded":"2023-11-24T07:56:23.683366+0000","last_fullsized":"2023-11-24T07:56:23.683366+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:56:23.683366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:56:23.683366+0000","last_clean_scrub_stamp":"2023-11-24T07:56:23.683366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":1,"reported_epoch":17,"state":"creating+peering","last_fresh":"2023-11-24T07:56:22.690679+0000","last_change":"2023-11-24T07:56:22.690679+0000","last_active":"2023-11-24T07:56:22.678019+0000","last_peered":"2023-11-24T07:56:22.678019+0000","last_clean":"2023-11-24T07:56:22.678019+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T07:56:22.690679+0000","last_undegraded":"2023-11-24T07:56:22.690679+0000","last_fullsized":"2023-11-24T07:56:22.690679+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:56:22.678019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:56:22.678019+0000","last_clean_scrub_stamp":"2023-11-24T07:56:22.678019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T07:56:23.683366+0000","last_change":"2023-11-24T07:56:23.683366+0000","last_active":"2023-11-24T07:56:23.683366+0000","last_peered":"2023-11-24T07:56:23.683366+0000","last_clean":"2023-11-24T07:56:23.683366+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T07:56:23.683366+0000","last_undegraded":"2023-11-24T07:56:23.683366+0000","last_fullsized":"2023-11-24T07:56:23.683366+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:56:23.683366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:56:23.683366+0000","last_clean_scrub_stamp":"2023-11-24T07:56:23.683366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.15","version":"0'0","reported_seq":1,"reported_epoch":17,"state":"creating+peering","last_fresh":"2023-11-24T07:56:22.688558+0000","last_change":"2023-11-24T07:56:22.688558+0000","last_active":"2023-11-24T07:56:22.678019+0000","last_peered":"2023-11-24T07:56:22.678019+0000","last_clean":"2023-11-24T07:56:22.678019+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T07:56:22.688558+0000","last_undegraded":"2023-11-24T07:56:22.688558+0000","last_fullsized":"2023-11-24T07:56:22.688558+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:56:22.678019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:56:22.678019+0000","last_clean_scrub_stamp":"2023-11-24T07:56:22.678019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T07:56:23.683366+0000","last_change":"2023-11-24T07:56:23.683366+0000","last_active":"2023-11-24T07:56:23.683366+0000","last_peered":"2023-11-24T07:56:23.683366+0000","last_clean":"2023-11-24T07:56:23.683366+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T07:56:23.683366+0000","last_undegraded":"2023-11-24T07:56:23.683366+0000","last_fullsized":"2023-11-24T07:56:23.683366+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:56:23.683366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:56:23.683366+0000","last_clean_scrub_stamp":"2023-11-24T07:56:23.683366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.14","version":"0'0","reported_seq":1,"reported_epoch":17,"state":"creating+peering","last_fresh":"2023-11-24T07:56:22.693669+0000","last_change":"2023-11-24T07:56:22.693669+0000","last_active":"2023-11-24T07:56:22.678019+0000","last_peered":"2023-11-24T07:56:22.678019+0000","last_clean":"2023-11-24T07:56:22.678019+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T07:56:22.693669+0000","last_undegraded":"2023-11-24T07:56:22.693669+0000","last_fullsized":"2023-11-24T07:56:22.693669+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:56:22.678019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:56:22.678019+0000","last_clean_scrub_stamp":"2023-11-24T07:56:22.678019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T07:56:23.683366+0000","last_change":"2023-11-24T07:56:23.683366+0000","last_active":"2023-11-24T07:56:23.683366+0000","last_peered":"2023-11-24T07:56:23.683366+0000","last_clean":"2023-11-24T07:56:23.683366+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T07:56:23.683366+0000","last_undegraded":"2023-11-24T07:56:23.683366+0000","last_fullsized":"2023-11-24T07:56:23.683366+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:56:23.683366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:56:23.683366+0000","last_clean_scrub_stamp":"2023-11-24T07:56:23.683366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.13","version":"0'0","reported_seq":1,"reported_epoch":17,"state":"creating+peering","last_fresh":"2023-11-24T07:56:22.688470+0000","last_change":"2023-11-24T07:56:22.688470+0000","last_active":"2023-11-24T07:56:22.678019+0000","last_peered":"2023-11-24T07:56:22.678019+0000","last_clean":"2023-11-24T07:56:22.678019+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T07:56:22.688470+0000","last_undegraded":"2023-11-24T07:56:22.688470+0000","last_fullsized":"2023-11-24T07:56:22.688470+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:56:22.678019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:56:22.678019+0000","last_clean_scrub_stamp":"2023-11-24T07:56:22.678019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T07:56:23.683366+0000","last_change":"2023-11-24T07:56:23.683366+0000","last_active":"2023-11-24T07:56:23.683366+0000","last_peered":"2023-11-24T07:56:23.683366+0000","last_clean":"2023-11-24T07:56:23.683366+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T07:56:23.683366+0000","last_undegraded":"2023-11-24T07:56:23.683366+0000","last_fullsized":"2023-11-24T07:56:23.683366+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:56:23.683366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:56:23.683366+0000","last_clean_scrub_stamp":"2023-11-24T07:56:23.683366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":1,"reported_epoch":17,"state":"creating+peering","last_fresh":"2023-11-24T07:56:22.694366+0000","last_change":"2023-11-24T07:56:22.694366+0000","last_active":"2023-11-24T07:56:22.678019+0000","last_peered":"2023-11-24T07:56:22.678019+0000","last_clean":"2023-11-24T07:56:22.678019+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T07:56:22.694366+0000","last_undegraded":"2023-11-24T07:56:22.694366+0000","last_fullsized":"2023-11-24T07:56:22.694366+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:56:22.678019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:56:22.678019+0000","last_clean_scrub_stamp":"2023-11-24T07:56:22.678019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T07:56:23.683366+0000","last_change":"2023-11-24T07:56:23.683366+0000","last_active":"2023-11-24T07:56:23.683366+0000","last_peered":"2023-11-24T07:56:23.683366+0000","last_clean":"2023-11-24T07:56:23.683366+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T07:56:23.683366+0000","last_undegraded":"2023-11-24T07:56:23.683366+0000","last_fullsized":"2023-11-24T07:56:23.683366+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:56:23.683366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:56:23.683366+0000","last_clean_scrub_stamp":"2023-11-24T07:56:23.683366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":1,"reported_epoch":17,"state":"creating+peering","last_fresh":"2023-11-24T07:56:22.690305+0000","last_change":"2023-11-24T07:56:22.690305+0000","last_active":"2023-11-24T07:56:22.678019+0000","last_peered":"2023-11-24T07:56:22.678019+0000","last_clean":"2023-11-24T07:56:22.678019+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T07:56:22.690305+0000","last_undegraded":"2023-11-24T07:56:22.690305+0000","last_fullsized":"2023-11-24T07:56:22.690305+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:56:22.678019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:56:22.678019+0000","last_clean_scrub_stamp":"2023-11-24T07:56:22.678019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T07:56:23.683366+0000","last_change":"2023-11-24T07:56:23.683366+0000","last_active":"2023-11-24T07:56:23.683366+0000","last_peered":"2023-11-24T07:56:23.683366+0000","last_clean":"2023-11-24T07:56:23.683366+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T07:56:23.683366+0000","last_undegraded":"2023-11-24T07:56:23.683366+0000","last_fullsized":"2023-11-24T07:56:23.683366+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:56:23.683366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:56:23.683366+0000","last_clean_scrub_stamp":"2023-11-24T07:56:23.683366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.10","version":"0'0","reported_seq":1,"reported_epoch":17,"state":"creating+peering","last_fresh":"2023-11-24T07:56:22.690393+0000","last_change":"2023-11-24T07:56:22.690393+0000","last_active":"2023-11-24T07:56:22.678019+0000","last_peered":"2023-11-24T07:56:22.678019+0000","last_clean":"2023-11-24T07:56:22.678019+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T07:56:22.690393+0000","last_undegraded":"2023-11-24T07:56:22.690393+0000","last_fullsized":"2023-11-24T07:56:22.690393+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:56:22.678019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:56:22.678019+0000","last_clean_scrub_stamp":"2023-11-24T07:56:22.678019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T07:56:23.683366+0000","last_change":"2023-11-24T07:56:23.683366+0000","last_active":"2023-11-24T07:56:23.683366+0000","last_peered":"2023-11-24T07:56:23.683366+0000","last_clean":"2023-11-24T07:56:23.683366+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T07:56:23.683366+0000","last_undegraded":"2023-11-24T07:56:23.683366+0000","last_fullsized":"2023-11-24T07:56:23.683366+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:56:23.683366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:56:23.683366+0000","last_clean_scrub_stamp":"2023-11-24T07:56:23.683366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.f","version":"0'0","reported_seq":1,"reported_epoch":17,"state":"creating+peering","last_fresh":"2023-11-24T07:56:22.690726+0000","last_change":"2023-11-24T07:56:22.690726+0000","last_active":"2023-11-24T07:56:22.678019+0000","last_peered":"2023-11-24T07:56:22.678019+0000","last_clean":"2023-11-24T07:56:22.678019+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T07:56:22.690726+0000","last_undegraded":"2023-11-24T07:56:22.690726+0000","last_fullsized":"2023-11-24T07:56:22.690726+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:56:22.678019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:56:22.678019+0000","last_clean_scrub_stamp":"2023-11-24T07:56:22.678019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T07:56:23.683366+0000","last_change":"2023-11-24T07:56:23.683366+0000","last_active":"2023-11-24T07:56:23.683366+0000","last_peered":"2023-11-24T07:56:23.683366+0000","last_clean":"2023-11-24T07:56:23.683366+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T07:56:23.683366+0000","last_undegraded":"2023-11-24T07:56:23.683366+0000","last_fullsized":"2023-11-24T07:56:23.683366+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:56:23.683366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:56:23.683366+0000","last_clean_scrub_stamp":"2023-11-24T07:56:23.683366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":1,"reported_epoch":17,"state":"creating+peering","last_fresh":"2023-11-24T07:56:22.686164+0000","last_change":"2023-11-24T07:56:22.686164+0000","last_active":"2023-11-24T07:56:22.678019+0000","last_peered":"2023-11-24T07:56:22.678019+0000","last_clean":"2023-11-24T07:56:22.678019+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T07:56:22.686164+0000","last_undegraded":"2023-11-24T07:56:22.686164+0000","last_fullsized":"2023-11-24T07:56:22.686164+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:56:22.678019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:56:22.678019+0000","last_clean_scrub_stamp":"2023-11-24T07:56:22.678019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T07:56:23.683366+0000","last_change":"2023-11-24T07:56:23.683366+0000","last_active":"2023-11-24T07:56:23.683366+0000","last_peered":"2023-11-24T07:56:23.683366+0000","last_clean":"2023-11-24T07:56:23.683366+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T07:56:23.683366+0000","last_undegraded":"2023-11-24T07:56:23.683366+0000","last_fullsized":"2023-11-24T07:56:23.683366+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:56:23.683366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:56:23.683366+0000","last_clean_scrub_stamp":"2023-11-24T07:56:23.683366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":1,"reported_epoch":17,"state":"creating+peering","last_fresh":"2023-11-24T07:56:22.695293+0000","last_change":"2023-11-24T07:56:22.695293+0000","last_active":"2023-11-24T07:56:22.678019+0000","last_peered":"2023-11-24T07:56:22.678019+0000","last_clean":"2023-11-24T07:56:22.678019+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T07:56:22.695293+0000","last_undegraded":"2023-11-24T07:56:22.695293+0000","last_fullsized":"2023-11-24T07:56:22.695293+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:56:22.678019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:56:22.678019+0000","last_clean_scrub_stamp":"2023-11-24T07:56:22.678019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T07:56:23.683366+0000","last_change":"2023-11-24T07:56:23.683366+0000","last_active":"2023-11-24T07:56:23.683366+0000","last_peered":"2023-11-24T07:56:23.683366+0000","last_clean":"2023-11-24T07:56:23.683366+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T07:56:23.683366+0000","last_undegraded":"2023-11-24T07:56:23.683366+0000","last_fullsized":"2023-11-24T07:56:23.683366+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:56:23.683366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:56:23.683366+0000","last_clean_scrub_stamp":"2023-11-24T07:56:23.683366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T07:56:23.683366+0000","last_change":"2023-11-24T07:56:23.683366+0000","last_active":"2023-11-24T07:56:23.683366+0000","last_peered":"2023-11-24T07:56:23.683366+0000","last_clean":"2023-11-24T07:56:23.683366+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T07:56:23.683366+0000","last_undegraded":"2023-11-24T07:56:23.683366+0000","last_fullsized":"2023-11-24T07:56:23.683366+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:56:23.683366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:56:23.683366+0000","last_clean_scrub_stamp":"2023-11-24T07:56:23.683366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":1,"reported_epoch":17,"state":"creating+peering","last_fresh":"2023-11-24T07:56:22.688504+0000","last_change":"2023-11-24T07:56:22.688504+0000","last_active":"2023-11-24T07:56:22.678019+0000","last_peered":"2023-11-24T07:56:22.678019+0000","last_clean":"2023-11-24T07:56:22.678019+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T07:56:22.688504+0000","last_undegraded":"2023-11-24T07:56:22.688504+0000","last_fullsized":"2023-11-24T07:56:22.688504+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:56:22.678019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:56:22.678019+0000","last_clean_scrub_stamp":"2023-11-24T07:56:22.678019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T07:56:23.683366+0000","last_change":"2023-11-24T07:56:23.683366+0000","last_active":"2023-11-24T07:56:23.683366+0000","last_peered":"2023-11-24T07:56:23.683366+0000","last_clean":"2023-11-24T07:56:23.683366+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T07:56:23.683366+0000","last_undegraded":"2023-11-24T07:56:23.683366+0000","last_fullsized":"2023-11-24T07:56:23.683366+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:56:23.683366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:56:23.683366+0000","last_clean_scrub_stamp":"2023-11-24T07:56:23.683366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":1,"reported_epoch":17,"state":"creating+peering","last_fresh":"2023-11-24T07:56:22.694975+0000","last_change":"2023-11-24T07:56:22.694975+0000","last_active":"2023-11-24T07:56:22.678019+0000","last_peered":"2023-11-24T07:56:22.678019+0000","last_clean":"2023-11-24T07:56:22.678019+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T07:56:22.694975+0000","last_undegraded":"2023-11-24T07:56:22.694975+0000","last_fullsized":"2023-11-24T07:56:22.694975+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:56:22.678019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:56:22.678019+0000","last_clean_scrub_stamp":"2023-11-24T07:56:22.678019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":162,"reported_epoch":17,"state":"active+clean","last_fresh":"2023-11-24T07:56:22.685865+0000","last_change":"2023-11-24T07:56:19.668905+0000","last_active":"2023-11-24T07:56:22.685865+0000","last_peered":"2023-11-24T07:56:22.685865+0000","last_clean":"2023-11-24T07:56:22.685865+0000","last_became_active":"2023-11-24T07:56:19.668601+0000","last_became_peered":"2023-11-24T07:56:19.668601+0000","last_unstale":"2023-11-24T07:56:22.685865+0000","last_undegraded":"2023-11-24T07:56:22.685865+0000","last_fullsized":"2023-11-24T07:56:22.685865+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:56:18.657844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:56:18.657844+0000","last_clean_scrub_stamp":"2023-11-24T07:56:18.657844+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T11:51:53.828455+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T07:56:23.683366+0000","last_change":"2023-11-24T07:56:23.683366+0000","last_active":"2023-11-24T07:56:23.683366+0000","last_peered":"2023-11-24T07:56:23.683366+0000","last_clean":"2023-11-24T07:56:23.683366+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T07:56:23.683366+0000","last_undegraded":"2023-11-24T07:56:23.683366+0000","last_fullsized":"2023-11-24T07:56:23.683366+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:56:23.683366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:56:23.683366+0000","last_clean_scrub_stamp":"2023-11-24T07:56:23.683366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":1,"reported_epoch":17,"state":"creating+peering","last_fresh":"2023-11-24T07:56:22.694439+0000","last_change":"2023-11-24T07:56:22.694439+0000","last_active":"2023-11-24T07:56:22.678019+0000","last_peered":"2023-11-24T07:56:22.678019+0000","last_clean":"2023-11-24T07:56:22.678019+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T07:56:22.694439+0000","last_undegraded":"2023-11-24T07:56:22.694439+0000","last_fullsized":"2023-11-24T07:56:22.694439+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:56:22.678019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:56:22.678019+0000","last_clean_scrub_stamp":"2023-11-24T07:56:22.678019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":1,"reported_epoch":17,"state":"creating+peering","last_fresh":"2023-11-24T07:56:22.694430+0000","last_change":"2023-11-24T07:56:22.694430+0000","last_active":"2023-11-24T07:56:22.678019+0000","last_peered":"2023-11-24T07:56:22.678019+0000","last_clean":"2023-11-24T07:56:22.678019+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T07:56:22.694430+0000","last_undegraded":"2023-11-24T07:56:22.694430+0000","last_fullsized":"2023-11-24T07:56:22.694430+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:56:22.678019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:56:22.678019+0000","last_clean_scrub_stamp":"2023-11-24T07:56:22.678019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T07:56:23.683366+0000","last_change":"2023-11-24T07:56:23.683366+0000","last_active":"2023-11-24T07:56:23.683366+0000","last_peered":"2023-11-24T07:56:23.683366+0000","last_clean":"2023-11-24T07:56:23.683366+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T07:56:23.683366+0000","last_undegraded":"2023-11-24T07:56:23.683366+0000","last_fullsized":"2023-11-24T07:56:23.683366+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:56:23.683366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:56:23.683366+0000","last_clean_scrub_stamp":"2023-11-24T07:56:23.683366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":1,"reported_epoch":17,"state":"creating+peering","last_fresh":"2023-11-24T07:56:22.690376+0000","last_change":"2023-11-24T07:56:22.690376+0000","last_active":"2023-11-24T07:56:22.678019+0000","last_peered":"2023-11-24T07:56:22.678019+0000","last_clean":"2023-11-24T07:56:22.678019+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T07:56:22.690376+0000","last_undegraded":"2023-11-24T07:56:22.690376+0000","last_fullsized":"2023-11-24T07:56:22.690376+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:56:22.678019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:56:22.678019+0000","last_clean_scrub_stamp":"2023-11-24T07:56:22.678019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T07:56:23.683366+0000","last_change":"2023-11-24T07:56:23.683366+0000","last_active":"2023-11-24T07:56:23.683366+0000","last_peered":"2023-11-24T07:56:23.683366+0000","last_clean":"2023-11-24T07:56:23.683366+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T07:56:23.683366+0000","last_undegraded":"2023-11-24T07:56:23.683366+0000","last_fullsized":"2023-11-24T07:56:23.683366+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:56:23.683366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:56:23.683366+0000","last_clean_scrub_stamp":"2023-11-24T07:56:23.683366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":1,"reported_epoch":17,"state":"creating+peering","last_fresh":"2023-11-24T07:56:22.695123+0000","last_change":"2023-11-24T07:56:22.695123+0000","last_active":"2023-11-24T07:56:22.678019+0000","last_peered":"2023-11-24T07:56:22.678019+0000","last_clean":"2023-11-24T07:56:22.678019+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T07:56:22.695123+0000","last_undegraded":"2023-11-24T07:56:22.695123+0000","last_fullsized":"2023-11-24T07:56:22.695123+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:56:22.678019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:56:22.678019+0000","last_clean_scrub_stamp":"2023-11-24T07:56:22.678019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T07:56:23.683366+0000","last_change":"2023-11-24T07:56:23.683366+0000","last_active":"2023-11-24T07:56:23.683366+0000","last_peered":"2023-11-24T07:56:23.683366+0000","last_clean":"2023-11-24T07:56:23.683366+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T07:56:23.683366+0000","last_undegraded":"2023-11-24T07:56:23.683366+0000","last_fullsized":"2023-11-24T07:56:23.683366+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:56:23.683366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:56:23.683366+0000","last_clean_scrub_stamp":"2023-11-24T07:56:23.683366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":1,"reported_epoch":17,"state":"creating+peering","last_fresh":"2023-11-24T07:56:22.690282+0000","last_change":"2023-11-24T07:56:22.690282+0000","last_active":"2023-11-24T07:56:22.678019+0000","last_peered":"2023-11-24T07:56:22.678019+0000","last_clean":"2023-11-24T07:56:22.678019+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T07:56:22.690282+0000","last_undegraded":"2023-11-24T07:56:22.690282+0000","last_fullsized":"2023-11-24T07:56:22.690282+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:56:22.678019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:56:22.678019+0000","last_clean_scrub_stamp":"2023-11-24T07:56:22.678019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T07:56:23.683366+0000","last_change":"2023-11-24T07:56:23.683366+0000","last_active":"2023-11-24T07:56:23.683366+0000","last_peered":"2023-11-24T07:56:23.683366+0000","last_clean":"2023-11-24T07:56:23.683366+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T07:56:23.683366+0000","last_undegraded":"2023-11-24T07:56:23.683366+0000","last_fullsized":"2023-11-24T07:56:23.683366+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:56:23.683366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:56:23.683366+0000","last_clean_scrub_stamp":"2023-11-24T07:56:23.683366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":1,"reported_epoch":17,"state":"creating+peering","last_fresh":"2023-11-24T07:56:22.690549+0000","last_change":"2023-11-24T07:56:22.690549+0000","last_active":"2023-11-24T07:56:22.678019+0000","last_peered":"2023-11-24T07:56:22.678019+0000","last_clean":"2023-11-24T07:56:22.678019+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T07:56:22.690549+0000","last_undegraded":"2023-11-24T07:56:22.690549+0000","last_fullsized":"2023-11-24T07:56:22.690549+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:56:22.678019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:56:22.678019+0000","last_clean_scrub_stamp":"2023-11-24T07:56:22.678019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T07:56:23.683366+0000","last_change":"2023-11-24T07:56:23.683366+0000","last_active":"2023-11-24T07:56:23.683366+0000","last_peered":"2023-11-24T07:56:23.683366+0000","last_clean":"2023-11-24T07:56:23.683366+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T07:56:23.683366+0000","last_undegraded":"2023-11-24T07:56:23.683366+0000","last_fullsized":"2023-11-24T07:56:23.683366+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:56:23.683366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:56:23.683366+0000","last_clean_scrub_stamp":"2023-11-24T07:56:23.683366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":1,"reported_epoch":17,"state":"creating+peering","last_fresh":"2023-11-24T07:56:22.689706+0000","last_change":"2023-11-24T07:56:22.689706+0000","last_active":"2023-11-24T07:56:22.678019+0000","last_peered":"2023-11-24T07:56:22.678019+0000","last_clean":"2023-11-24T07:56:22.678019+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T07:56:22.689706+0000","last_undegraded":"2023-11-24T07:56:22.689706+0000","last_fullsized":"2023-11-24T07:56:22.689706+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:56:22.678019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:56:22.678019+0000","last_clean_scrub_stamp":"2023-11-24T07:56:22.678019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T07:56:23.683366+0000","last_change":"2023-11-24T07:56:23.683366+0000","last_active":"2023-11-24T07:56:23.683366+0000","last_peered":"2023-11-24T07:56:23.683366+0000","last_clean":"2023-11-24T07:56:23.683366+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T07:56:23.683366+0000","last_undegraded":"2023-11-24T07:56:23.683366+0000","last_fullsized":"2023-11-24T07:56:23.683366+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:56:23.683366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:56:23.683366+0000","last_clean_scrub_stamp":"2023-11-24T07:56:23.683366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.b","version":"0'0","reported_seq":1,"reported_epoch":17,"state":"creating+peering","last_fresh":"2023-11-24T07:56:22.695809+0000","last_change":"2023-11-24T07:56:22.695809+0000","last_active":"2023-11-24T07:56:22.678019+0000","last_peered":"2023-11-24T07:56:22.678019+0000","last_clean":"2023-11-24T07:56:22.678019+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T07:56:22.695809+0000","last_undegraded":"2023-11-24T07:56:22.695809+0000","last_fullsized":"2023-11-24T07:56:22.695809+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:56:22.678019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:56:22.678019+0000","last_clean_scrub_stamp":"2023-11-24T07:56:22.678019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T07:56:23.683366+0000","last_change":"2023-11-24T07:56:23.683366+0000","last_active":"2023-11-24T07:56:23.683366+0000","last_peered":"2023-11-24T07:56:23.683366+0000","last_clean":"2023-11-24T07:56:23.683366+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T07:56:23.683366+0000","last_undegraded":"2023-11-24T07:56:23.683366+0000","last_fullsized":"2023-11-24T07:56:23.683366+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:56:23.683366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:56:23.683366+0000","last_clean_scrub_stamp":"2023-11-24T07:56:23.683366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":1,"reported_epoch":17,"state":"creating+peering","last_fresh":"2023-11-24T07:56:22.690094+0000","last_change":"2023-11-24T07:56:22.690094+0000","last_active":"2023-11-24T07:56:22.678019+0000","last_peered":"2023-11-24T07:56:22.678019+0000","last_clean":"2023-11-24T07:56:22.678019+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T07:56:22.690094+0000","last_undegraded":"2023-11-24T07:56:22.690094+0000","last_fullsized":"2023-11-24T07:56:22.690094+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:56:22.678019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:56:22.678019+0000","last_clean_scrub_stamp":"2023-11-24T07:56:22.678019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T07:56:23.683366+0000","last_change":"2023-11-24T07:56:23.683366+0000","last_active":"2023-11-24T07:56:23.683366+0000","last_peered":"2023-11-24T07:56:23.683366+0000","last_clean":"2023-11-24T07:56:23.683366+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T07:56:23.683366+0000","last_undegraded":"2023-11-24T07:56:23.683366+0000","last_fullsized":"2023-11-24T07:56:23.683366+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:56:23.683366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:56:23.683366+0000","last_clean_scrub_stamp":"2023-11-24T07:56:23.683366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.d","version":"0'0","reported_seq":1,"reported_epoch":17,"state":"creating+peering","last_fresh":"2023-11-24T07:56:22.688223+0000","last_change":"2023-11-24T07:56:22.688223+0000","last_active":"2023-11-24T07:56:22.678019+0000","last_peered":"2023-11-24T07:56:22.678019+0000","last_clean":"2023-11-24T07:56:22.678019+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T07:56:22.688223+0000","last_undegraded":"2023-11-24T07:56:22.688223+0000","last_fullsized":"2023-11-24T07:56:22.688223+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:56:22.678019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:56:22.678019+0000","last_clean_scrub_stamp":"2023-11-24T07:56:22.678019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T07:56:23.683366+0000","last_change":"2023-11-24T07:56:23.683366+0000","last_active":"2023-11-24T07:56:23.683366+0000","last_peered":"2023-11-24T07:56:23.683366+0000","last_clean":"2023-11-24T07:56:23.683366+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T07:56:23.683366+0000","last_undegraded":"2023-11-24T07:56:23.683366+0000","last_fullsized":"2023-11-24T07:56:23.683366+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:56:23.683366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:56:23.683366+0000","last_clean_scrub_stamp":"2023-11-24T07:56:23.683366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.e","version":"0'0","reported_seq":1,"reported_epoch":17,"state":"creating+peering","last_fresh":"2023-11-24T07:56:22.691566+0000","last_change":"2023-11-24T07:56:22.691566+0000","last_active":"2023-11-24T07:56:22.678019+0000","last_peered":"2023-11-24T07:56:22.678019+0000","last_clean":"2023-11-24T07:56:22.678019+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T07:56:22.691566+0000","last_undegraded":"2023-11-24T07:56:22.691566+0000","last_fullsized":"2023-11-24T07:56:22.691566+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:56:22.678019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:56:22.678019+0000","last_clean_scrub_stamp":"2023-11-24T07:56:22.678019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T07:56:23.683366+0000","last_change":"2023-11-24T07:56:23.683366+0000","last_active":"2023-11-24T07:56:23.683366+0000","last_peered":"2023-11-24T07:56:23.683366+0000","last_clean":"2023-11-24T07:56:23.683366+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T07:56:23.683366+0000","last_undegraded":"2023-11-24T07:56:23.683366+0000","last_fullsized":"2023-11-24T07:56:23.683366+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:56:23.683366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:56:23.683366+0000","last_clean_scrub_stamp":"2023-11-24T07:56:23.683366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T07:56:23.683366+0000","last_change":"2023-11-24T07:56:23.683366+0000","last_active":"2023-11-24T07:56:23.683366+0000","last_peered":"2023-11-24T07:56:23.683366+0000","last_clean":"2023-11-24T07:56:23.683366+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T07:56:23.683366+0000","last_undegraded":"2023-11-24T07:56:23.683366+0000","last_fullsized":"2023-11-24T07:56:23.683366+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:56:23.683366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:56:23.683366+0000","last_clean_scrub_stamp":"2023-11-24T07:56:23.683366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T07:56:23.683366+0000","last_change":"2023-11-24T07:56:23.683366+0000","last_active":"2023-11-24T07:56:23.683366+0000","last_peered":"2023-11-24T07:56:23.683366+0000","last_clean":"2023-11-24T07:56:23.683366+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T07:56:23.683366+0000","last_undegraded":"2023-11-24T07:56:23.683366+0000","last_fullsized":"2023-11-24T07:56:23.683366+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:56:23.683366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:56:23.683366+0000","last_clean_scrub_stamp":"2023-11-24T07:56:23.683366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T07:56:23.683366+0000","last_change":"2023-11-24T07:56:23.683366+0000","last_active":"2023-11-24T07:56:23.683366+0000","last_peered":"2023-11-24T07:56:23.683366+0000","last_clean":"2023-11-24T07:56:23.683366+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T07:56:23.683366+0000","last_undegraded":"2023-11-24T07:56:23.683366+0000","last_fullsized":"2023-11-24T07:56:23.683366+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:56:23.683366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:56:23.683366+0000","last_clean_scrub_stamp":"2023-11-24T07:56:23.683366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T07:56:23.683366+0000","last_change":"2023-11-24T07:56:23.683366+0000","last_active":"2023-11-24T07:56:23.683366+0000","last_peered":"2023-11-24T07:56:23.683366+0000","last_clean":"2023-11-24T07:56:23.683366+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T07:56:23.683366+0000","last_undegraded":"2023-11-24T07:56:23.683366+0000","last_fullsized":"2023-11-24T07:56:23.683366+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:56:23.683366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:56:23.683366+0000","last_clean_scrub_stamp":"2023-11-24T07:56:23.683366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T07:56:23.683366+0000","last_change":"2023-11-24T07:56:23.683366+0000","last_active":"2023-11-24T07:56:23.683366+0000","last_peered":"2023-11-24T07:56:23.683366+0000","last_clean":"2023-11-24T07:56:23.683366+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T07:56:23.683366+0000","last_undegraded":"2023-11-24T07:56:23.683366+0000","last_fullsized":"2023-11-24T07:56:23.683366+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:56:23.683366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:56:23.683366+0000","last_clean_scrub_stamp":"2023-11-24T07:56:23.683366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T07:56:23.683366+0000","last_change":"2023-11-24T07:56:23.683366+0000","last_active":"2023-11-24T07:56:23.683366+0000","last_peered":"2023-11-24T07:56:23.683366+0000","last_clean":"2023-11-24T07:56:23.683366+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T07:56:23.683366+0000","last_undegraded":"2023-11-24T07:56:23.683366+0000","last_fullsized":"2023-11-24T07:56:23.683366+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:56:23.683366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:56:23.683366+0000","last_clean_scrub_stamp":"2023-11-24T07:56:23.683366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T07:56:23.683366+0000","last_change":"2023-11-24T07:56:23.683366+0000","last_active":"2023-11-24T07:56:23.683366+0000","last_peered":"2023-11-24T07:56:23.683366+0000","last_clean":"2023-11-24T07:56:23.683366+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T07:56:23.683366+0000","last_undegraded":"2023-11-24T07:56:23.683366+0000","last_fullsized":"2023-11-24T07:56:23.683366+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:56:23.683366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:56:23.683366+0000","last_clean_scrub_stamp":"2023-11-24T07:56:23.683366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T07:56:23.683366+0000","last_change":"2023-11-24T07:56:23.683366+0000","last_active":"2023-11-24T07:56:23.683366+0000","last_peered":"2023-11-24T07:56:23.683366+0000","last_clean":"2023-11-24T07:56:23.683366+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T07:56:23.683366+0000","last_undegraded":"2023-11-24T07:56:23.683366+0000","last_fullsized":"2023-11-24T07:56:23.683366+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:56:23.683366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:56:23.683366+0000","last_clean_scrub_stamp":"2023-11-24T07:56:23.683366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T07:56:23.683366+0000","last_change":"2023-11-24T07:56:23.683366+0000","last_active":"2023-11-24T07:56:23.683366+0000","last_peered":"2023-11-24T07:56:23.683366+0000","last_clean":"2023-11-24T07:56:23.683366+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T07:56:23.683366+0000","last_undegraded":"2023-11-24T07:56:23.683366+0000","last_fullsized":"2023-11-24T07:56:23.683366+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:56:23.683366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:56:23.683366+0000","last_clean_scrub_stamp":"2023-11-24T07:56:23.683366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T07:56:23.683366+0000","last_change":"2023-11-24T07:56:23.683366+0000","last_active":"2023-11-24T07:56:23.683366+0000","last_peered":"2023-11-24T07:56:23.683366+0000","last_clean":"2023-11-24T07:56:23.683366+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T07:56:23.683366+0000","last_undegraded":"2023-11-24T07:56:23.683366+0000","last_fullsized":"2023-11-24T07:56:23.683366+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:56:23.683366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:56:23.683366+0000","last_clean_scrub_stamp":"2023-11-24T07:56:23.683366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T07:56:23.683366+0000","last_change":"2023-11-24T07:56:23.683366+0000","last_active":"2023-11-24T07:56:23.683366+0000","last_peered":"2023-11-24T07:56:23.683366+0000","last_clean":"2023-11-24T07:56:23.683366+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T07:56:23.683366+0000","last_undegraded":"2023-11-24T07:56:23.683366+0000","last_fullsized":"2023-11-24T07:56:23.683366+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:56:23.683366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:56:23.683366+0000","last_clean_scrub_stamp":"2023-11-24T07:56:23.683366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T07:56:23.683366+0000","last_change":"2023-11-24T07:56:23.683366+0000","last_active":"2023-11-24T07:56:23.683366+0000","last_peered":"2023-11-24T07:56:23.683366+0000","last_clean":"2023-11-24T07:56:23.683366+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T07:56:23.683366+0000","last_undegraded":"2023-11-24T07:56:23.683366+0000","last_fullsized":"2023-11-24T07:56:23.683366+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:56:23.683366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:56:23.683366+0000","last_clean_scrub_stamp":"2023-11-24T07:56:23.683366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T07:56:23.683366+0000","last_change":"2023-11-24T07:56:23.683366+0000","last_active":"2023-11-24T07:56:23.683366+0000","last_peered":"2023-11-24T07:56:23.683366+0000","last_clean":"2023-11-24T07:56:23.683366+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T07:56:23.683366+0000","last_undegraded":"2023-11-24T07:56:23.683366+0000","last_fullsized":"2023-11-24T07:56:23.683366+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:56:23.683366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:56:23.683366+0000","last_clean_scrub_stamp":"2023-11-24T07:56:23.683366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574851,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27172,"kb_used_data":152,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344668,"statfs":{"total":96636764160,"available":96608940032,"internally_reserved":0,"allocated":155648,"data_stored":45743,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.40100000000000002}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61599999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64400000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66300000000000003}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.42599999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.45900000000000002}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55000000000000004}]}]},{"osd":6,"up_from":13,"seq":55834574852,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27172,"kb_used_data":152,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344668,"statfs":{"total":96636764160,"available":96608940032,"internally_reserved":0,"allocated":155648,"data_stored":45743,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65300000000000002}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54100000000000004}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.76600000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.876}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59999999999999998}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.84999999999999998}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58099999999999996}]}]},{"osd":1,"up_from":13,"seq":55834574853,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27764,"kb_used_data":744,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344076,"statfs":{"total":96636764160,"available":96608333824,"internally_reserved":0,"allocated":761856,"data_stored":641928,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.33100000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.308}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.432}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.69699999999999995}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68000000000000005}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.72099999999999997}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.81200000000000006}]}]},{"osd":0,"up_from":13,"seq":55834574853,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27172,"kb_used_data":152,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344668,"statfs":{"total":96636764160,"available":96608940032,"internally_reserved":0,"allocated":155648,"data_stored":45743,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.28799999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.32800000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.30499999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.40500000000000003}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54200000000000004}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48499999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52600000000000002}]}]},{"osd":2,"up_from":13,"seq":55834574853,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27184,"kb_used_data":164,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344656,"statfs":{"total":96636764160,"available":96608927744,"internally_reserved":0,"allocated":167936,"data_stored":51560,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48699999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.44500000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53300000000000003}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.81599999999999995}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.84899999999999998}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.83599999999999997}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.80000000000000004}]}]},{"osd":3,"up_from":13,"seq":55834574853,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27184,"kb_used_data":164,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344656,"statfs":{"total":96636764160,"available":96608927744,"internally_reserved":0,"allocated":167936,"data_stored":51560,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.313}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.39600000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.35899999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.73099999999999998}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57199999999999995}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60899999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.71099999999999997}]}]},{"osd":4,"up_from":13,"seq":55834574852,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27184,"kb_used_data":164,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344656,"statfs":{"total":96636764160,"available":96608927744,"internally_reserved":0,"allocated":167936,"data_stored":51560,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.78400000000000003}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.42699999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.745}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65100000000000002}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47099999999999997}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55100000000000005}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58099999999999996}]}]},{"osd":5,"up_from":13,"seq":55834574853,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27764,"kb_used_data":744,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344076,"statfs":{"total":96636764160,"available":96608333824,"internally_reserved":0,"allocated":761856,"data_stored":641928,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0109999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.98499999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.86599999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.046}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51600000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54800000000000004}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.96599999999999997}]}]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-11-24T07:56:28.878 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-11-24T07:56:29.038 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:29.036+0000 7f896e1c3700 1 Processor -- start 2023-11-24T07:56:29.038 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:29.036+0000 7f896e1c3700 1 -- start start 2023-11-24T07:56:29.041 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:29.037+0000 7f896e1c3700 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f89680722c0 0x7f89680b51b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:56:29.041 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:29.037+0000 7f896e1c3700 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8968143ec0 con 0x7f89681440f0 2023-11-24T07:56:29.041 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:29.037+0000 7f896e1c3700 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f89680b3ae0 con 0x7f8968143ae0 2023-11-24T07:56:29.041 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:29.037+0000 7f896e1c3700 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f89680b40a0 con 0x7f89680722c0 2023-11-24T07:56:29.041 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:29.037+0000 7f8967fff700 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7f89681440f0 0x7f896806e510 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:53252/0 (socket says 172.21.15.59:53252) 2023-11-24T07:56:29.041 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:29.037+0000 7f8967fff700 1 -- 172.21.15.59:0/2306887291 learned_addr learned my addr 172.21.15.59:0/2306887291 (peer_addr_for_me v1:172.21.15.59:0/0) 2023-11-24T07:56:29.042 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:29.037+0000 7f8966ffd700 1 --2- 172.21.15.59:0/2306887291 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f89680722c0 0x7f89680b51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:56:29.042 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:29.037+0000 7f89667fc700 1 -- 172.21.15.59:0/2306887291 <== mon.0 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 934758443 0 0) 0x7f8968143ec0 con 0x7f89681440f0 2023-11-24T07:56:29.042 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:29.037+0000 7f89667fc700 1 -- 172.21.15.59:0/2306887291 --> v1:172.21.15.59:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f89500035b0 con 0x7f89681440f0 2023-11-24T07:56:29.042 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:29.037+0000 7f89667fc700 1 -- 172.21.15.59:0/2306887291 <== mon.1 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1460406529 0 0) 0x7f89680b3ae0 con 0x7f8968143ae0 2023-11-24T07:56:29.042 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:29.037+0000 7f89667fc700 1 -- 172.21.15.59:0/2306887291 --> v1:172.21.15.76:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8968143ec0 con 0x7f8968143ae0 2023-11-24T07:56:29.042 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:29.037+0000 7f89667fc700 1 -- 172.21.15.59:0/2306887291 <== mon.0 v1:172.21.15.59:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2601904541 0 0) 0x7f89500035b0 con 0x7f89681440f0 2023-11-24T07:56:29.042 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:29.037+0000 7f89667fc700 1 -- 172.21.15.59:0/2306887291 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f89680722c0 msgr2=0x7f89680b51b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T07:56:29.043 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:29.037+0000 7f89667fc700 1 --2- 172.21.15.59:0/2306887291 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f89680722c0 0x7f89680b51b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:56:29.044 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:29.037+0000 7f89667fc700 1 -- 172.21.15.59:0/2306887291 >> v1:172.21.15.76:6789/0 conn(0x7f8968143ae0 legacy=0x7f8968071bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:56:29.044 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:29.037+0000 7f89667fc700 1 -- 172.21.15.59:0/2306887291 --> v1:172.21.15.59:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f89680b3dc0 con 0x7f89681440f0 2023-11-24T07:56:29.044 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:29.037+0000 7f89667fc700 1 -- 172.21.15.59:0/2306887291 <== mon.0 v1:172.21.15.59:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1312333566 0 0) 0x7f89600019f0 con 0x7f89681440f0 2023-11-24T07:56:29.044 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:29.037+0000 7f8966ffd700 1 --2- 172.21.15.59:0/2306887291 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f89680722c0 0x7f89680b51b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-24T07:56:29.046 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:29.038+0000 7f89667fc700 1 -- 172.21.15.59:0/2306887291 >> v1:172.21.15.59:6789/0 conn(0x7f89681440f0 legacy=0x7f896806e510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:56:29.046 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:29.038+0000 7f89667fc700 1 --2- 172.21.15.59:0/2306887291 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8950003bc0 0x7f8950003fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:56:29.046 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:29.038+0000 7f89667fc700 1 --2- 172.21.15.59:0/2306887291 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f89680722c0 0x7f8950004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:56:29.046 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:29.038+0000 7f89667fc700 1 --2- 172.21.15.59:0/2306887291 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8950004d90 0x7f8950005210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:56:29.047 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:29.038+0000 7f89667fc700 1 -- 172.21.15.59:0/2306887291 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89680b40a0 con 0x7f8950003bc0 2023-11-24T07:56:29.047 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:29.038+0000 7f89667fc700 1 -- 172.21.15.59:0/2306887291 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f895000dcd0 con 0x7f8950004d90 2023-11-24T07:56:29.047 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:29.038+0000 7f89667fc700 1 -- 172.21.15.59:0/2306887291 --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f895000de10 con 0x7f89680722c0 2023-11-24T07:56:29.047 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:29.038+0000 7f89677fe700 1 --2- 172.21.15.59:0/2306887291 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8950003bc0 0x7f8950003fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:56:29.047 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:29.038+0000 7f8967fff700 1 --2- 172.21.15.59:0/2306887291 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8950004d90 0x7f8950005210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:56:29.047 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:29.038+0000 7f8966ffd700 1 --2- 172.21.15.59:0/2306887291 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f89680722c0 0x7f8950004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:56:29.047 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:29.038+0000 7f89677fe700 1 -- 172.21.15.59:0/2306887291 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f89680722c0 msgr2=0x7f8950004850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T07:56:29.047 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:29.038+0000 7f89677fe700 1 --2- 172.21.15.59:0/2306887291 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f89680722c0 0x7f8950004850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:56:29.047 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:29.038+0000 7f89677fe700 1 -- 172.21.15.59:0/2306887291 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8950004d90 msgr2=0x7f8950005210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T07:56:29.047 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:29.038+0000 7f89677fe700 1 --2- 172.21.15.59:0/2306887291 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8950004d90 0x7f8950005210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:56:29.047 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:29.038+0000 7f89677fe700 1 -- 172.21.15.59:0/2306887291 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f895c0094a0 con 0x7f8950003bc0 2023-11-24T07:56:29.048 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:29.038+0000 7f89677fe700 1 --2- 172.21.15.59:0/2306887291 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8950003bc0 0x7f8950003fc0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f8958001970 tx=0x7f8950005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9b4998418c31f243 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:56:29.048 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:29.038+0000 7f89667fc700 1 -- 172.21.15.59:0/2306887291 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8958001430 con 0x7f8950003bc0 2023-11-24T07:56:29.048 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:29.038+0000 7f89667fc700 1 -- 172.21.15.59:0/2306887291 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f89580052d0 con 0x7f8950003bc0 2023-11-24T07:56:29.048 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:29.038+0000 7f89667fc700 1 -- 172.21.15.59:0/2306887291 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8958003680 con 0x7f8950003bc0 2023-11-24T07:56:29.048 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:29.039+0000 7f896e1c3700 1 -- 172.21.15.59:0/2306887291 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8950003bc0 msgr2=0x7f8950003fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:56:29.048 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:29.039+0000 7f896e1c3700 1 --2- 172.21.15.59:0/2306887291 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8950003bc0 0x7f8950003fc0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f8958001970 tx=0x7f8950005880 comp rx=0 tx=0).stop 2023-11-24T07:56:29.048 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:29.039+0000 7f8967fff700 1 -- 172.21.15.59:0/2306887291 reap_dead start 2023-11-24T07:56:29.048 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:29.039+0000 7f896e1c3700 1 -- 172.21.15.59:0/2306887291 shutdown_connections 2023-11-24T07:56:29.048 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:29.039+0000 7f896e1c3700 1 -- 172.21.15.59:0/2306887291 >> 172.21.15.59:0/2306887291 conn(0x7f89680b2740 msgr2=0x7f89680b2b20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:56:29.048 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:29.039+0000 7f896e1c3700 1 -- 172.21.15.59:0/2306887291 shutdown_connections 2023-11-24T07:56:29.048 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:29.039+0000 7f896e1c3700 1 -- 172.21.15.59:0/2306887291 wait complete. 2023-11-24T07:56:29.051 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:29.040+0000 7f896e1c3700 1 Processor -- start 2023-11-24T07:56:29.051 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:29.040+0000 7f896e1c3700 1 -- start start 2023-11-24T07:56:29.051 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:29.041+0000 7f896e1c3700 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f8950003bc0 0x7f89680c6630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:56:29.051 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:29.041+0000 7f896e1c3700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f896815cb50 0x7f89681ee960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:56:29.051 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:29.041+0000 7f896e1c3700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f896815cfd0 0x7f89681eeea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:56:29.052 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:29.041+0000 7f896e1c3700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89681f3a90 con 0x7f896815cfd0 2023-11-24T07:56:29.052 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:29.041+0000 7f896e1c3700 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89681ef3e0 con 0x7f896815cb50 2023-11-24T07:56:29.052 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:29.041+0000 7f896e1c3700 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f89680c61a0 con 0x7f8950003bc0 2023-11-24T07:56:29.052 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:29.041+0000 7f8967fff700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f896815cfd0 0x7f89681eeea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:56:29.052 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:29.041+0000 7f8967fff700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f896815cfd0 0x7f89681eeea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:49790/0 (socket says 172.21.15.59:49790) 2023-11-24T07:56:29.052 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:29.041+0000 7f8967fff700 1 -- 172.21.15.59:0/3209296463 learned_addr learned my addr 172.21.15.59:0/3209296463 (peer_addr_for_me v2:172.21.15.59:0/0) 2023-11-24T07:56:29.052 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:29.041+0000 7f8966ffd700 1 --2- 172.21.15.59:0/3209296463 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f896815cb50 0x7f89681ee960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:56:29.052 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:29.041+0000 7f89677fe700 1 --2- 172.21.15.59:0/3209296463 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f8950003bc0 0x7f89680c6630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:56:29.052 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:29.041+0000 7f8967fff700 1 -- 172.21.15.59:0/3209296463 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f8950003bc0 msgr2=0x7f89680c6630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:56:29.052 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:29.041+0000 7f8967fff700 1 --2- 172.21.15.59:0/3209296463 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f8950003bc0 0x7f89680c6630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:56:29.052 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:29.041+0000 7f8967fff700 1 -- 172.21.15.59:0/3209296463 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f896815cb50 msgr2=0x7f89681ee960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:56:29.053 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:29.041+0000 7f8967fff700 1 --2- 172.21.15.59:0/3209296463 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f896815cb50 0x7f89681ee960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:56:29.053 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:29.041+0000 7f8967fff700 1 -- 172.21.15.59:0/3209296463 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f89681f37b0 con 0x7f896815cfd0 2023-11-24T07:56:29.053 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:29.042+0000 7f8967fff700 1 --2- 172.21.15.59:0/3209296463 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f896815cfd0 0x7f89681eeea0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f8960000f80 tx=0x7f8960001ae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:56:29.053 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:29.042+0000 7f894ffff700 1 -- 172.21.15.59:0/3209296463 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f896000ade0 con 0x7f896815cfd0 2023-11-24T07:56:29.053 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:29.042+0000 7f894ffff700 1 -- 172.21.15.59:0/3209296463 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f89600133f0 con 0x7f896815cfd0 2023-11-24T07:56:29.053 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:29.042+0000 7f894ffff700 1 -- 172.21.15.59:0/3209296463 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89600137a0 con 0x7f896815cfd0 2023-11-24T07:56:29.053 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:29.042+0000 7f896e1c3700 1 -- 172.21.15.59:0/3209296463 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f89680c5b50 con 0x7f896815cfd0 2023-11-24T07:56:29.053 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:29.042+0000 7f896e1c3700 1 -- 172.21.15.59:0/3209296463 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f89681f2af0 con 0x7f896815cfd0 2023-11-24T07:56:29.053 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:29.043+0000 7f896e1c3700 1 -- 172.21.15.59:0/3209296463 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f89680c59b0 con 0x7f896815cfd0 2023-11-24T07:56:29.053 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:29.043+0000 7f894ffff700 1 -- 172.21.15.59:0/3209296463 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f8960013900 con 0x7f896815cfd0 2023-11-24T07:56:29.053 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:29.044+0000 7f894ffff700 1 --2- 172.21.15.59:0/3209296463 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f895406aed0 0x7f895406d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:56:29.053 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:29.044+0000 7f894ffff700 1 -- 172.21.15.59:0/3209296463 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f8960089f60 con 0x7f896815cfd0 2023-11-24T07:56:29.054 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:29.044+0000 7f89677fe700 1 --2- 172.21.15.59:0/3209296463 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f895406aed0 0x7f895406d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:56:29.054 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:29.044+0000 7f89677fe700 1 --2- 172.21.15.59:0/3209296463 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f895406aed0 0x7f895406d390 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f8958015040 tx=0x7f895800a000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:56:29.054 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:29.048+0000 7f894ffff700 1 -- 172.21.15.59:0/3209296463 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f89680c59b0 con 0x7f896815cfd0 2023-11-24T07:56:29.243 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:29.242+0000 7f896e1c3700 1 -- 172.21.15.59:0/3209296463 --> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f8968144860 con 0x7f895406aed0 2023-11-24T07:56:29.251 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:29.249+0000 7f894ffff700 1 -- 172.21.15.59:0/3209296463 <== mgr.4100 v2:172.21.15.76:6832/4264046536 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+252811 (secure 0 0 0) 0x7f8958005440 con 0x7f895406aed0 2023-11-24T07:56:29.254 INFO:teuthology.orchestra.run.smithi059.stdout: 2023-11-24T07:56:29.258 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:29.252+0000 7f896e1c3700 1 -- 172.21.15.59:0/3209296463 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f895406aed0 msgr2=0x7f895406d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:56:29.258 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:29.252+0000 7f896e1c3700 1 --2- 172.21.15.59:0/3209296463 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f895406aed0 0x7f895406d390 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f8958015040 tx=0x7f895800a000 comp rx=0 tx=0).stop 2023-11-24T07:56:29.259 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:29.252+0000 7f896e1c3700 1 -- 172.21.15.59:0/3209296463 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f896815cfd0 msgr2=0x7f89681eeea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:56:29.259 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:29.252+0000 7f896e1c3700 1 --2- 172.21.15.59:0/3209296463 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f896815cfd0 0x7f89681eeea0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f8960000f80 tx=0x7f8960001ae0 comp rx=0 tx=0).stop 2023-11-24T07:56:29.259 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:29.252+0000 7f896e1c3700 1 -- 172.21.15.59:0/3209296463 shutdown_connections 2023-11-24T07:56:29.259 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:29.252+0000 7f896e1c3700 1 --2- 172.21.15.59:0/3209296463 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f895406aed0 0x7f895406d390 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:56:29.259 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:29.252+0000 7f896e1c3700 1 --2- 172.21.15.59:0/3209296463 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f896815cfd0 0x7f89681eeea0 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:56:29.259 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:29.252+0000 7f896e1c3700 1 --2- 172.21.15.59:0/3209296463 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f8950003bc0 0x7f89680c6630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:56:29.260 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:29.252+0000 7f896e1c3700 1 --2- 172.21.15.59:0/3209296463 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f896815cb50 0x7f89681ee960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:56:29.260 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:29.252+0000 7f896e1c3700 1 -- 172.21.15.59:0/3209296463 >> 172.21.15.59:0/3209296463 conn(0x7f89680b2740 msgr2=0x7f89680cf8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:56:29.260 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:29.252+0000 7f896e1c3700 1 -- 172.21.15.59:0/3209296463 shutdown_connections 2023-11-24T07:56:29.260 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:29.252+0000 7f896e1c3700 1 -- 172.21.15.59:0/3209296463 wait complete. 2023-11-24T07:56:29.260 INFO:teuthology.orchestra.run.smithi059.stderr:dumped all 2023-11-24T07:56:29.266 INFO:teuthology.orchestra.run.smithi059.stdout:{"pg_ready":true,"pg_map":{"version":18,"stamp":"2023-11-24T07:56:28.298177+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":66,"acting":66,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":14,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":218596,"kb_used_data":2436,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754756124,"statfs":{"total":773094113280,"available":772870270976,"internally_reserved":0,"allocated":2494464,"data_stored":1575765,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12745,"internal_metadata":221236791},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":14,"apply_latency_ms":14,"commit_latency_ns":14000000,"apply_latency_ns":14000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.611089"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T07:56:23.683366+0000","last_change":"2023-11-24T07:56:23.683366+0000","last_active":"2023-11-24T07:56:23.683366+0000","last_peered":"2023-11-24T07:56:23.683366+0000","last_clean":"2023-11-24T07:56:23.683366+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T07:56:23.683366+0000","last_undegraded":"2023-11-24T07:56:23.683366+0000","last_fullsized":"2023-11-24T07:56:23.683366+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:56:23.683366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:56:23.683366+0000","last_clean_scrub_stamp":"2023-11-24T07:56:23.683366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T07:56:23.683366+0000","last_change":"2023-11-24T07:56:23.683366+0000","last_active":"2023-11-24T07:56:23.683366+0000","last_peered":"2023-11-24T07:56:23.683366+0000","last_clean":"2023-11-24T07:56:23.683366+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T07:56:23.683366+0000","last_undegraded":"2023-11-24T07:56:23.683366+0000","last_fullsized":"2023-11-24T07:56:23.683366+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:56:23.683366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:56:23.683366+0000","last_clean_scrub_stamp":"2023-11-24T07:56:23.683366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T07:56:23.683366+0000","last_change":"2023-11-24T07:56:23.683366+0000","last_active":"2023-11-24T07:56:23.683366+0000","last_peered":"2023-11-24T07:56:23.683366+0000","last_clean":"2023-11-24T07:56:23.683366+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T07:56:23.683366+0000","last_undegraded":"2023-11-24T07:56:23.683366+0000","last_fullsized":"2023-11-24T07:56:23.683366+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:56:23.683366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:56:23.683366+0000","last_clean_scrub_stamp":"2023-11-24T07:56:23.683366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T07:56:23.683366+0000","last_change":"2023-11-24T07:56:23.683366+0000","last_active":"2023-11-24T07:56:23.683366+0000","last_peered":"2023-11-24T07:56:23.683366+0000","last_clean":"2023-11-24T07:56:23.683366+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T07:56:23.683366+0000","last_undegraded":"2023-11-24T07:56:23.683366+0000","last_fullsized":"2023-11-24T07:56:23.683366+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:56:23.683366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:56:23.683366+0000","last_clean_scrub_stamp":"2023-11-24T07:56:23.683366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T07:56:23.683366+0000","last_change":"2023-11-24T07:56:23.683366+0000","last_active":"2023-11-24T07:56:23.683366+0000","last_peered":"2023-11-24T07:56:23.683366+0000","last_clean":"2023-11-24T07:56:23.683366+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T07:56:23.683366+0000","last_undegraded":"2023-11-24T07:56:23.683366+0000","last_fullsized":"2023-11-24T07:56:23.683366+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:56:23.683366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:56:23.683366+0000","last_clean_scrub_stamp":"2023-11-24T07:56:23.683366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T07:56:23.683366+0000","last_change":"2023-11-24T07:56:23.683366+0000","last_active":"2023-11-24T07:56:23.683366+0000","last_peered":"2023-11-24T07:56:23.683366+0000","last_clean":"2023-11-24T07:56:23.683366+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T07:56:23.683366+0000","last_undegraded":"2023-11-24T07:56:23.683366+0000","last_fullsized":"2023-11-24T07:56:23.683366+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:56:23.683366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:56:23.683366+0000","last_clean_scrub_stamp":"2023-11-24T07:56:23.683366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T07:56:23.683366+0000","last_change":"2023-11-24T07:56:23.683366+0000","last_active":"2023-11-24T07:56:23.683366+0000","last_peered":"2023-11-24T07:56:23.683366+0000","last_clean":"2023-11-24T07:56:23.683366+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T07:56:23.683366+0000","last_undegraded":"2023-11-24T07:56:23.683366+0000","last_fullsized":"2023-11-24T07:56:23.683366+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:56:23.683366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:56:23.683366+0000","last_clean_scrub_stamp":"2023-11-24T07:56:23.683366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T07:56:23.683366+0000","last_change":"2023-11-24T07:56:23.683366+0000","last_active":"2023-11-24T07:56:23.683366+0000","last_peered":"2023-11-24T07:56:23.683366+0000","last_clean":"2023-11-24T07:56:23.683366+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T07:56:23.683366+0000","last_undegraded":"2023-11-24T07:56:23.683366+0000","last_fullsized":"2023-11-24T07:56:23.683366+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:56:23.683366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:56:23.683366+0000","last_clean_scrub_stamp":"2023-11-24T07:56:23.683366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T07:56:23.683366+0000","last_change":"2023-11-24T07:56:23.683366+0000","last_active":"2023-11-24T07:56:23.683366+0000","last_peered":"2023-11-24T07:56:23.683366+0000","last_clean":"2023-11-24T07:56:23.683366+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T07:56:23.683366+0000","last_undegraded":"2023-11-24T07:56:23.683366+0000","last_fullsized":"2023-11-24T07:56:23.683366+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:56:23.683366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:56:23.683366+0000","last_clean_scrub_stamp":"2023-11-24T07:56:23.683366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T07:56:23.683366+0000","last_change":"2023-11-24T07:56:23.683366+0000","last_active":"2023-11-24T07:56:23.683366+0000","last_peered":"2023-11-24T07:56:23.683366+0000","last_clean":"2023-11-24T07:56:23.683366+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T07:56:23.683366+0000","last_undegraded":"2023-11-24T07:56:23.683366+0000","last_fullsized":"2023-11-24T07:56:23.683366+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:56:23.683366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:56:23.683366+0000","last_clean_scrub_stamp":"2023-11-24T07:56:23.683366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T07:56:23.683366+0000","last_change":"2023-11-24T07:56:23.683366+0000","last_active":"2023-11-24T07:56:23.683366+0000","last_peered":"2023-11-24T07:56:23.683366+0000","last_clean":"2023-11-24T07:56:23.683366+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T07:56:23.683366+0000","last_undegraded":"2023-11-24T07:56:23.683366+0000","last_fullsized":"2023-11-24T07:56:23.683366+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:56:23.683366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:56:23.683366+0000","last_clean_scrub_stamp":"2023-11-24T07:56:23.683366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T07:56:23.683366+0000","last_change":"2023-11-24T07:56:23.683366+0000","last_active":"2023-11-24T07:56:23.683366+0000","last_peered":"2023-11-24T07:56:23.683366+0000","last_clean":"2023-11-24T07:56:23.683366+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T07:56:23.683366+0000","last_undegraded":"2023-11-24T07:56:23.683366+0000","last_fullsized":"2023-11-24T07:56:23.683366+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:56:23.683366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:56:23.683366+0000","last_clean_scrub_stamp":"2023-11-24T07:56:23.683366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T07:56:23.683366+0000","last_change":"2023-11-24T07:56:23.683366+0000","last_active":"2023-11-24T07:56:23.683366+0000","last_peered":"2023-11-24T07:56:23.683366+0000","last_clean":"2023-11-24T07:56:23.683366+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T07:56:23.683366+0000","last_undegraded":"2023-11-24T07:56:23.683366+0000","last_fullsized":"2023-11-24T07:56:23.683366+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:56:23.683366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:56:23.683366+0000","last_clean_scrub_stamp":"2023-11-24T07:56:23.683366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T07:56:23.683366+0000","last_change":"2023-11-24T07:56:23.683366+0000","last_active":"2023-11-24T07:56:23.683366+0000","last_peered":"2023-11-24T07:56:23.683366+0000","last_clean":"2023-11-24T07:56:23.683366+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T07:56:23.683366+0000","last_undegraded":"2023-11-24T07:56:23.683366+0000","last_fullsized":"2023-11-24T07:56:23.683366+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:56:23.683366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:56:23.683366+0000","last_clean_scrub_stamp":"2023-11-24T07:56:23.683366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T07:56:23.683366+0000","last_change":"2023-11-24T07:56:23.683366+0000","last_active":"2023-11-24T07:56:23.683366+0000","last_peered":"2023-11-24T07:56:23.683366+0000","last_clean":"2023-11-24T07:56:23.683366+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T07:56:23.683366+0000","last_undegraded":"2023-11-24T07:56:23.683366+0000","last_fullsized":"2023-11-24T07:56:23.683366+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:56:23.683366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:56:23.683366+0000","last_clean_scrub_stamp":"2023-11-24T07:56:23.683366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T07:56:23.683366+0000","last_change":"2023-11-24T07:56:23.683366+0000","last_active":"2023-11-24T07:56:23.683366+0000","last_peered":"2023-11-24T07:56:23.683366+0000","last_clean":"2023-11-24T07:56:23.683366+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T07:56:23.683366+0000","last_undegraded":"2023-11-24T07:56:23.683366+0000","last_fullsized":"2023-11-24T07:56:23.683366+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:56:23.683366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:56:23.683366+0000","last_clean_scrub_stamp":"2023-11-24T07:56:23.683366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T07:56:23.683366+0000","last_change":"2023-11-24T07:56:23.683366+0000","last_active":"2023-11-24T07:56:23.683366+0000","last_peered":"2023-11-24T07:56:23.683366+0000","last_clean":"2023-11-24T07:56:23.683366+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T07:56:23.683366+0000","last_undegraded":"2023-11-24T07:56:23.683366+0000","last_fullsized":"2023-11-24T07:56:23.683366+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:56:23.683366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:56:23.683366+0000","last_clean_scrub_stamp":"2023-11-24T07:56:23.683366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T07:56:23.683366+0000","last_change":"2023-11-24T07:56:23.683366+0000","last_active":"2023-11-24T07:56:23.683366+0000","last_peered":"2023-11-24T07:56:23.683366+0000","last_clean":"2023-11-24T07:56:23.683366+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T07:56:23.683366+0000","last_undegraded":"2023-11-24T07:56:23.683366+0000","last_fullsized":"2023-11-24T07:56:23.683366+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:56:23.683366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:56:23.683366+0000","last_clean_scrub_stamp":"2023-11-24T07:56:23.683366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T07:56:23.683366+0000","last_change":"2023-11-24T07:56:23.683366+0000","last_active":"2023-11-24T07:56:23.683366+0000","last_peered":"2023-11-24T07:56:23.683366+0000","last_clean":"2023-11-24T07:56:23.683366+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T07:56:23.683366+0000","last_undegraded":"2023-11-24T07:56:23.683366+0000","last_fullsized":"2023-11-24T07:56:23.683366+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:56:23.683366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:56:23.683366+0000","last_clean_scrub_stamp":"2023-11-24T07:56:23.683366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1f","version":"0'0","reported_seq":1,"reported_epoch":17,"state":"creating+peering","last_fresh":"2023-11-24T07:56:22.690428+0000","last_change":"2023-11-24T07:56:22.690428+0000","last_active":"2023-11-24T07:56:22.678019+0000","last_peered":"2023-11-24T07:56:22.678019+0000","last_clean":"2023-11-24T07:56:22.678019+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T07:56:22.690428+0000","last_undegraded":"2023-11-24T07:56:22.690428+0000","last_fullsized":"2023-11-24T07:56:22.690428+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:56:22.678019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:56:22.678019+0000","last_clean_scrub_stamp":"2023-11-24T07:56:22.678019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T07:56:23.683366+0000","last_change":"2023-11-24T07:56:23.683366+0000","last_active":"2023-11-24T07:56:23.683366+0000","last_peered":"2023-11-24T07:56:23.683366+0000","last_clean":"2023-11-24T07:56:23.683366+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T07:56:23.683366+0000","last_undegraded":"2023-11-24T07:56:23.683366+0000","last_fullsized":"2023-11-24T07:56:23.683366+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:56:23.683366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:56:23.683366+0000","last_clean_scrub_stamp":"2023-11-24T07:56:23.683366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1e","version":"0'0","reported_seq":1,"reported_epoch":17,"state":"creating+peering","last_fresh":"2023-11-24T07:56:22.686164+0000","last_change":"2023-11-24T07:56:22.686164+0000","last_active":"2023-11-24T07:56:22.678019+0000","last_peered":"2023-11-24T07:56:22.678019+0000","last_clean":"2023-11-24T07:56:22.678019+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T07:56:22.686164+0000","last_undegraded":"2023-11-24T07:56:22.686164+0000","last_fullsized":"2023-11-24T07:56:22.686164+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:56:22.678019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:56:22.678019+0000","last_clean_scrub_stamp":"2023-11-24T07:56:22.678019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T07:56:23.683366+0000","last_change":"2023-11-24T07:56:23.683366+0000","last_active":"2023-11-24T07:56:23.683366+0000","last_peered":"2023-11-24T07:56:23.683366+0000","last_clean":"2023-11-24T07:56:23.683366+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T07:56:23.683366+0000","last_undegraded":"2023-11-24T07:56:23.683366+0000","last_fullsized":"2023-11-24T07:56:23.683366+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:56:23.683366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:56:23.683366+0000","last_clean_scrub_stamp":"2023-11-24T07:56:23.683366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1d","version":"0'0","reported_seq":1,"reported_epoch":17,"state":"creating+peering","last_fresh":"2023-11-24T07:56:22.690762+0000","last_change":"2023-11-24T07:56:22.690762+0000","last_active":"2023-11-24T07:56:22.678019+0000","last_peered":"2023-11-24T07:56:22.678019+0000","last_clean":"2023-11-24T07:56:22.678019+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T07:56:22.690762+0000","last_undegraded":"2023-11-24T07:56:22.690762+0000","last_fullsized":"2023-11-24T07:56:22.690762+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:56:22.678019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:56:22.678019+0000","last_clean_scrub_stamp":"2023-11-24T07:56:22.678019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T07:56:23.683366+0000","last_change":"2023-11-24T07:56:23.683366+0000","last_active":"2023-11-24T07:56:23.683366+0000","last_peered":"2023-11-24T07:56:23.683366+0000","last_clean":"2023-11-24T07:56:23.683366+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T07:56:23.683366+0000","last_undegraded":"2023-11-24T07:56:23.683366+0000","last_fullsized":"2023-11-24T07:56:23.683366+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:56:23.683366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:56:23.683366+0000","last_clean_scrub_stamp":"2023-11-24T07:56:23.683366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1c","version":"0'0","reported_seq":1,"reported_epoch":17,"state":"creating+peering","last_fresh":"2023-11-24T07:56:22.690382+0000","last_change":"2023-11-24T07:56:22.690382+0000","last_active":"2023-11-24T07:56:22.678019+0000","last_peered":"2023-11-24T07:56:22.678019+0000","last_clean":"2023-11-24T07:56:22.678019+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T07:56:22.690382+0000","last_undegraded":"2023-11-24T07:56:22.690382+0000","last_fullsized":"2023-11-24T07:56:22.690382+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:56:22.678019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:56:22.678019+0000","last_clean_scrub_stamp":"2023-11-24T07:56:22.678019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T07:56:23.683366+0000","last_change":"2023-11-24T07:56:23.683366+0000","last_active":"2023-11-24T07:56:23.683366+0000","last_peered":"2023-11-24T07:56:23.683366+0000","last_clean":"2023-11-24T07:56:23.683366+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T07:56:23.683366+0000","last_undegraded":"2023-11-24T07:56:23.683366+0000","last_fullsized":"2023-11-24T07:56:23.683366+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:56:23.683366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:56:23.683366+0000","last_clean_scrub_stamp":"2023-11-24T07:56:23.683366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1b","version":"0'0","reported_seq":1,"reported_epoch":17,"state":"creating+peering","last_fresh":"2023-11-24T07:56:22.686185+0000","last_change":"2023-11-24T07:56:22.686185+0000","last_active":"2023-11-24T07:56:22.678019+0000","last_peered":"2023-11-24T07:56:22.678019+0000","last_clean":"2023-11-24T07:56:22.678019+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T07:56:22.686185+0000","last_undegraded":"2023-11-24T07:56:22.686185+0000","last_fullsized":"2023-11-24T07:56:22.686185+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:56:22.678019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:56:22.678019+0000","last_clean_scrub_stamp":"2023-11-24T07:56:22.678019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T07:56:23.683366+0000","last_change":"2023-11-24T07:56:23.683366+0000","last_active":"2023-11-24T07:56:23.683366+0000","last_peered":"2023-11-24T07:56:23.683366+0000","last_clean":"2023-11-24T07:56:23.683366+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T07:56:23.683366+0000","last_undegraded":"2023-11-24T07:56:23.683366+0000","last_fullsized":"2023-11-24T07:56:23.683366+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:56:23.683366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:56:23.683366+0000","last_clean_scrub_stamp":"2023-11-24T07:56:23.683366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1a","version":"0'0","reported_seq":1,"reported_epoch":17,"state":"creating+peering","last_fresh":"2023-11-24T07:56:22.696543+0000","last_change":"2023-11-24T07:56:22.696543+0000","last_active":"2023-11-24T07:56:22.678019+0000","last_peered":"2023-11-24T07:56:22.678019+0000","last_clean":"2023-11-24T07:56:22.678019+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T07:56:22.696543+0000","last_undegraded":"2023-11-24T07:56:22.696543+0000","last_fullsized":"2023-11-24T07:56:22.696543+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:56:22.678019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:56:22.678019+0000","last_clean_scrub_stamp":"2023-11-24T07:56:22.678019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T07:56:23.683366+0000","last_change":"2023-11-24T07:56:23.683366+0000","last_active":"2023-11-24T07:56:23.683366+0000","last_peered":"2023-11-24T07:56:23.683366+0000","last_clean":"2023-11-24T07:56:23.683366+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T07:56:23.683366+0000","last_undegraded":"2023-11-24T07:56:23.683366+0000","last_fullsized":"2023-11-24T07:56:23.683366+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:56:23.683366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:56:23.683366+0000","last_clean_scrub_stamp":"2023-11-24T07:56:23.683366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":1,"reported_epoch":17,"state":"creating+peering","last_fresh":"2023-11-24T07:56:22.689685+0000","last_change":"2023-11-24T07:56:22.689685+0000","last_active":"2023-11-24T07:56:22.678019+0000","last_peered":"2023-11-24T07:56:22.678019+0000","last_clean":"2023-11-24T07:56:22.678019+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T07:56:22.689685+0000","last_undegraded":"2023-11-24T07:56:22.689685+0000","last_fullsized":"2023-11-24T07:56:22.689685+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:56:22.678019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:56:22.678019+0000","last_clean_scrub_stamp":"2023-11-24T07:56:22.678019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T07:56:23.683366+0000","last_change":"2023-11-24T07:56:23.683366+0000","last_active":"2023-11-24T07:56:23.683366+0000","last_peered":"2023-11-24T07:56:23.683366+0000","last_clean":"2023-11-24T07:56:23.683366+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T07:56:23.683366+0000","last_undegraded":"2023-11-24T07:56:23.683366+0000","last_fullsized":"2023-11-24T07:56:23.683366+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:56:23.683366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:56:23.683366+0000","last_clean_scrub_stamp":"2023-11-24T07:56:23.683366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":1,"reported_epoch":17,"state":"creating+peering","last_fresh":"2023-11-24T07:56:22.695289+0000","last_change":"2023-11-24T07:56:22.695289+0000","last_active":"2023-11-24T07:56:22.678019+0000","last_peered":"2023-11-24T07:56:22.678019+0000","last_clean":"2023-11-24T07:56:22.678019+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T07:56:22.695289+0000","last_undegraded":"2023-11-24T07:56:22.695289+0000","last_fullsized":"2023-11-24T07:56:22.695289+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:56:22.678019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:56:22.678019+0000","last_clean_scrub_stamp":"2023-11-24T07:56:22.678019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T07:56:23.683366+0000","last_change":"2023-11-24T07:56:23.683366+0000","last_active":"2023-11-24T07:56:23.683366+0000","last_peered":"2023-11-24T07:56:23.683366+0000","last_clean":"2023-11-24T07:56:23.683366+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T07:56:23.683366+0000","last_undegraded":"2023-11-24T07:56:23.683366+0000","last_fullsized":"2023-11-24T07:56:23.683366+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:56:23.683366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:56:23.683366+0000","last_clean_scrub_stamp":"2023-11-24T07:56:23.683366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":1,"reported_epoch":17,"state":"creating+peering","last_fresh":"2023-11-24T07:56:22.686064+0000","last_change":"2023-11-24T07:56:22.686064+0000","last_active":"2023-11-24T07:56:22.678019+0000","last_peered":"2023-11-24T07:56:22.678019+0000","last_clean":"2023-11-24T07:56:22.678019+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T07:56:22.686064+0000","last_undegraded":"2023-11-24T07:56:22.686064+0000","last_fullsized":"2023-11-24T07:56:22.686064+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:56:22.678019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:56:22.678019+0000","last_clean_scrub_stamp":"2023-11-24T07:56:22.678019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T07:56:23.683366+0000","last_change":"2023-11-24T07:56:23.683366+0000","last_active":"2023-11-24T07:56:23.683366+0000","last_peered":"2023-11-24T07:56:23.683366+0000","last_clean":"2023-11-24T07:56:23.683366+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T07:56:23.683366+0000","last_undegraded":"2023-11-24T07:56:23.683366+0000","last_fullsized":"2023-11-24T07:56:23.683366+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:56:23.683366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:56:23.683366+0000","last_clean_scrub_stamp":"2023-11-24T07:56:23.683366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":1,"reported_epoch":17,"state":"creating+peering","last_fresh":"2023-11-24T07:56:22.690679+0000","last_change":"2023-11-24T07:56:22.690679+0000","last_active":"2023-11-24T07:56:22.678019+0000","last_peered":"2023-11-24T07:56:22.678019+0000","last_clean":"2023-11-24T07:56:22.678019+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T07:56:22.690679+0000","last_undegraded":"2023-11-24T07:56:22.690679+0000","last_fullsized":"2023-11-24T07:56:22.690679+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:56:22.678019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:56:22.678019+0000","last_clean_scrub_stamp":"2023-11-24T07:56:22.678019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T07:56:23.683366+0000","last_change":"2023-11-24T07:56:23.683366+0000","last_active":"2023-11-24T07:56:23.683366+0000","last_peered":"2023-11-24T07:56:23.683366+0000","last_clean":"2023-11-24T07:56:23.683366+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T07:56:23.683366+0000","last_undegraded":"2023-11-24T07:56:23.683366+0000","last_fullsized":"2023-11-24T07:56:23.683366+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:56:23.683366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:56:23.683366+0000","last_clean_scrub_stamp":"2023-11-24T07:56:23.683366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.15","version":"0'0","reported_seq":1,"reported_epoch":17,"state":"creating+peering","last_fresh":"2023-11-24T07:56:22.688558+0000","last_change":"2023-11-24T07:56:22.688558+0000","last_active":"2023-11-24T07:56:22.678019+0000","last_peered":"2023-11-24T07:56:22.678019+0000","last_clean":"2023-11-24T07:56:22.678019+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T07:56:22.688558+0000","last_undegraded":"2023-11-24T07:56:22.688558+0000","last_fullsized":"2023-11-24T07:56:22.688558+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:56:22.678019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:56:22.678019+0000","last_clean_scrub_stamp":"2023-11-24T07:56:22.678019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T07:56:23.683366+0000","last_change":"2023-11-24T07:56:23.683366+0000","last_active":"2023-11-24T07:56:23.683366+0000","last_peered":"2023-11-24T07:56:23.683366+0000","last_clean":"2023-11-24T07:56:23.683366+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T07:56:23.683366+0000","last_undegraded":"2023-11-24T07:56:23.683366+0000","last_fullsized":"2023-11-24T07:56:23.683366+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:56:23.683366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:56:23.683366+0000","last_clean_scrub_stamp":"2023-11-24T07:56:23.683366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.14","version":"0'0","reported_seq":1,"reported_epoch":17,"state":"creating+peering","last_fresh":"2023-11-24T07:56:22.693669+0000","last_change":"2023-11-24T07:56:22.693669+0000","last_active":"2023-11-24T07:56:22.678019+0000","last_peered":"2023-11-24T07:56:22.678019+0000","last_clean":"2023-11-24T07:56:22.678019+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T07:56:22.693669+0000","last_undegraded":"2023-11-24T07:56:22.693669+0000","last_fullsized":"2023-11-24T07:56:22.693669+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:56:22.678019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:56:22.678019+0000","last_clean_scrub_stamp":"2023-11-24T07:56:22.678019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T07:56:23.683366+0000","last_change":"2023-11-24T07:56:23.683366+0000","last_active":"2023-11-24T07:56:23.683366+0000","last_peered":"2023-11-24T07:56:23.683366+0000","last_clean":"2023-11-24T07:56:23.683366+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T07:56:23.683366+0000","last_undegraded":"2023-11-24T07:56:23.683366+0000","last_fullsized":"2023-11-24T07:56:23.683366+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:56:23.683366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:56:23.683366+0000","last_clean_scrub_stamp":"2023-11-24T07:56:23.683366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.13","version":"0'0","reported_seq":1,"reported_epoch":17,"state":"creating+peering","last_fresh":"2023-11-24T07:56:22.688470+0000","last_change":"2023-11-24T07:56:22.688470+0000","last_active":"2023-11-24T07:56:22.678019+0000","last_peered":"2023-11-24T07:56:22.678019+0000","last_clean":"2023-11-24T07:56:22.678019+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T07:56:22.688470+0000","last_undegraded":"2023-11-24T07:56:22.688470+0000","last_fullsized":"2023-11-24T07:56:22.688470+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:56:22.678019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:56:22.678019+0000","last_clean_scrub_stamp":"2023-11-24T07:56:22.678019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T07:56:23.683366+0000","last_change":"2023-11-24T07:56:23.683366+0000","last_active":"2023-11-24T07:56:23.683366+0000","last_peered":"2023-11-24T07:56:23.683366+0000","last_clean":"2023-11-24T07:56:23.683366+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T07:56:23.683366+0000","last_undegraded":"2023-11-24T07:56:23.683366+0000","last_fullsized":"2023-11-24T07:56:23.683366+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:56:23.683366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:56:23.683366+0000","last_clean_scrub_stamp":"2023-11-24T07:56:23.683366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":1,"reported_epoch":17,"state":"creating+peering","last_fresh":"2023-11-24T07:56:22.694366+0000","last_change":"2023-11-24T07:56:22.694366+0000","last_active":"2023-11-24T07:56:22.678019+0000","last_peered":"2023-11-24T07:56:22.678019+0000","last_clean":"2023-11-24T07:56:22.678019+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T07:56:22.694366+0000","last_undegraded":"2023-11-24T07:56:22.694366+0000","last_fullsized":"2023-11-24T07:56:22.694366+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:56:22.678019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:56:22.678019+0000","last_clean_scrub_stamp":"2023-11-24T07:56:22.678019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T07:56:23.683366+0000","last_change":"2023-11-24T07:56:23.683366+0000","last_active":"2023-11-24T07:56:23.683366+0000","last_peered":"2023-11-24T07:56:23.683366+0000","last_clean":"2023-11-24T07:56:23.683366+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T07:56:23.683366+0000","last_undegraded":"2023-11-24T07:56:23.683366+0000","last_fullsized":"2023-11-24T07:56:23.683366+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:56:23.683366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:56:23.683366+0000","last_clean_scrub_stamp":"2023-11-24T07:56:23.683366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":1,"reported_epoch":17,"state":"creating+peering","last_fresh":"2023-11-24T07:56:22.690305+0000","last_change":"2023-11-24T07:56:22.690305+0000","last_active":"2023-11-24T07:56:22.678019+0000","last_peered":"2023-11-24T07:56:22.678019+0000","last_clean":"2023-11-24T07:56:22.678019+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T07:56:22.690305+0000","last_undegraded":"2023-11-24T07:56:22.690305+0000","last_fullsized":"2023-11-24T07:56:22.690305+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:56:22.678019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:56:22.678019+0000","last_clean_scrub_stamp":"2023-11-24T07:56:22.678019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T07:56:23.683366+0000","last_change":"2023-11-24T07:56:23.683366+0000","last_active":"2023-11-24T07:56:23.683366+0000","last_peered":"2023-11-24T07:56:23.683366+0000","last_clean":"2023-11-24T07:56:23.683366+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T07:56:23.683366+0000","last_undegraded":"2023-11-24T07:56:23.683366+0000","last_fullsized":"2023-11-24T07:56:23.683366+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:56:23.683366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:56:23.683366+0000","last_clean_scrub_stamp":"2023-11-24T07:56:23.683366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.10","version":"0'0","reported_seq":1,"reported_epoch":17,"state":"creating+peering","last_fresh":"2023-11-24T07:56:22.690393+0000","last_change":"2023-11-24T07:56:22.690393+0000","last_active":"2023-11-24T07:56:22.678019+0000","last_peered":"2023-11-24T07:56:22.678019+0000","last_clean":"2023-11-24T07:56:22.678019+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T07:56:22.690393+0000","last_undegraded":"2023-11-24T07:56:22.690393+0000","last_fullsized":"2023-11-24T07:56:22.690393+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:56:22.678019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:56:22.678019+0000","last_clean_scrub_stamp":"2023-11-24T07:56:22.678019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T07:56:23.683366+0000","last_change":"2023-11-24T07:56:23.683366+0000","last_active":"2023-11-24T07:56:23.683366+0000","last_peered":"2023-11-24T07:56:23.683366+0000","last_clean":"2023-11-24T07:56:23.683366+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T07:56:23.683366+0000","last_undegraded":"2023-11-24T07:56:23.683366+0000","last_fullsized":"2023-11-24T07:56:23.683366+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:56:23.683366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:56:23.683366+0000","last_clean_scrub_stamp":"2023-11-24T07:56:23.683366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.f","version":"0'0","reported_seq":1,"reported_epoch":17,"state":"creating+peering","last_fresh":"2023-11-24T07:56:22.690726+0000","last_change":"2023-11-24T07:56:22.690726+0000","last_active":"2023-11-24T07:56:22.678019+0000","last_peered":"2023-11-24T07:56:22.678019+0000","last_clean":"2023-11-24T07:56:22.678019+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T07:56:22.690726+0000","last_undegraded":"2023-11-24T07:56:22.690726+0000","last_fullsized":"2023-11-24T07:56:22.690726+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:56:22.678019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:56:22.678019+0000","last_clean_scrub_stamp":"2023-11-24T07:56:22.678019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T07:56:23.683366+0000","last_change":"2023-11-24T07:56:23.683366+0000","last_active":"2023-11-24T07:56:23.683366+0000","last_peered":"2023-11-24T07:56:23.683366+0000","last_clean":"2023-11-24T07:56:23.683366+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T07:56:23.683366+0000","last_undegraded":"2023-11-24T07:56:23.683366+0000","last_fullsized":"2023-11-24T07:56:23.683366+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:56:23.683366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:56:23.683366+0000","last_clean_scrub_stamp":"2023-11-24T07:56:23.683366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":1,"reported_epoch":17,"state":"creating+peering","last_fresh":"2023-11-24T07:56:22.686164+0000","last_change":"2023-11-24T07:56:22.686164+0000","last_active":"2023-11-24T07:56:22.678019+0000","last_peered":"2023-11-24T07:56:22.678019+0000","last_clean":"2023-11-24T07:56:22.678019+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T07:56:22.686164+0000","last_undegraded":"2023-11-24T07:56:22.686164+0000","last_fullsized":"2023-11-24T07:56:22.686164+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:56:22.678019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:56:22.678019+0000","last_clean_scrub_stamp":"2023-11-24T07:56:22.678019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T07:56:23.683366+0000","last_change":"2023-11-24T07:56:23.683366+0000","last_active":"2023-11-24T07:56:23.683366+0000","last_peered":"2023-11-24T07:56:23.683366+0000","last_clean":"2023-11-24T07:56:23.683366+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T07:56:23.683366+0000","last_undegraded":"2023-11-24T07:56:23.683366+0000","last_fullsized":"2023-11-24T07:56:23.683366+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:56:23.683366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:56:23.683366+0000","last_clean_scrub_stamp":"2023-11-24T07:56:23.683366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":1,"reported_epoch":17,"state":"creating+peering","last_fresh":"2023-11-24T07:56:22.695293+0000","last_change":"2023-11-24T07:56:22.695293+0000","last_active":"2023-11-24T07:56:22.678019+0000","last_peered":"2023-11-24T07:56:22.678019+0000","last_clean":"2023-11-24T07:56:22.678019+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T07:56:22.695293+0000","last_undegraded":"2023-11-24T07:56:22.695293+0000","last_fullsized":"2023-11-24T07:56:22.695293+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:56:22.678019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:56:22.678019+0000","last_clean_scrub_stamp":"2023-11-24T07:56:22.678019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T07:56:23.683366+0000","last_change":"2023-11-24T07:56:23.683366+0000","last_active":"2023-11-24T07:56:23.683366+0000","last_peered":"2023-11-24T07:56:23.683366+0000","last_clean":"2023-11-24T07:56:23.683366+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T07:56:23.683366+0000","last_undegraded":"2023-11-24T07:56:23.683366+0000","last_fullsized":"2023-11-24T07:56:23.683366+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:56:23.683366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:56:23.683366+0000","last_clean_scrub_stamp":"2023-11-24T07:56:23.683366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T07:56:23.683366+0000","last_change":"2023-11-24T07:56:23.683366+0000","last_active":"2023-11-24T07:56:23.683366+0000","last_peered":"2023-11-24T07:56:23.683366+0000","last_clean":"2023-11-24T07:56:23.683366+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T07:56:23.683366+0000","last_undegraded":"2023-11-24T07:56:23.683366+0000","last_fullsized":"2023-11-24T07:56:23.683366+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:56:23.683366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:56:23.683366+0000","last_clean_scrub_stamp":"2023-11-24T07:56:23.683366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":1,"reported_epoch":17,"state":"creating+peering","last_fresh":"2023-11-24T07:56:22.688504+0000","last_change":"2023-11-24T07:56:22.688504+0000","last_active":"2023-11-24T07:56:22.678019+0000","last_peered":"2023-11-24T07:56:22.678019+0000","last_clean":"2023-11-24T07:56:22.678019+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T07:56:22.688504+0000","last_undegraded":"2023-11-24T07:56:22.688504+0000","last_fullsized":"2023-11-24T07:56:22.688504+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:56:22.678019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:56:22.678019+0000","last_clean_scrub_stamp":"2023-11-24T07:56:22.678019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T07:56:23.683366+0000","last_change":"2023-11-24T07:56:23.683366+0000","last_active":"2023-11-24T07:56:23.683366+0000","last_peered":"2023-11-24T07:56:23.683366+0000","last_clean":"2023-11-24T07:56:23.683366+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T07:56:23.683366+0000","last_undegraded":"2023-11-24T07:56:23.683366+0000","last_fullsized":"2023-11-24T07:56:23.683366+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:56:23.683366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:56:23.683366+0000","last_clean_scrub_stamp":"2023-11-24T07:56:23.683366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":1,"reported_epoch":17,"state":"creating+peering","last_fresh":"2023-11-24T07:56:22.694975+0000","last_change":"2023-11-24T07:56:22.694975+0000","last_active":"2023-11-24T07:56:22.678019+0000","last_peered":"2023-11-24T07:56:22.678019+0000","last_clean":"2023-11-24T07:56:22.678019+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T07:56:22.694975+0000","last_undegraded":"2023-11-24T07:56:22.694975+0000","last_fullsized":"2023-11-24T07:56:22.694975+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:56:22.678019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:56:22.678019+0000","last_clean_scrub_stamp":"2023-11-24T07:56:22.678019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":162,"reported_epoch":17,"state":"active+clean","last_fresh":"2023-11-24T07:56:22.685865+0000","last_change":"2023-11-24T07:56:19.668905+0000","last_active":"2023-11-24T07:56:22.685865+0000","last_peered":"2023-11-24T07:56:22.685865+0000","last_clean":"2023-11-24T07:56:22.685865+0000","last_became_active":"2023-11-24T07:56:19.668601+0000","last_became_peered":"2023-11-24T07:56:19.668601+0000","last_unstale":"2023-11-24T07:56:22.685865+0000","last_undegraded":"2023-11-24T07:56:22.685865+0000","last_fullsized":"2023-11-24T07:56:22.685865+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:56:18.657844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:56:18.657844+0000","last_clean_scrub_stamp":"2023-11-24T07:56:18.657844+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T11:51:53.828455+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T07:56:23.683366+0000","last_change":"2023-11-24T07:56:23.683366+0000","last_active":"2023-11-24T07:56:23.683366+0000","last_peered":"2023-11-24T07:56:23.683366+0000","last_clean":"2023-11-24T07:56:23.683366+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T07:56:23.683366+0000","last_undegraded":"2023-11-24T07:56:23.683366+0000","last_fullsized":"2023-11-24T07:56:23.683366+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:56:23.683366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:56:23.683366+0000","last_clean_scrub_stamp":"2023-11-24T07:56:23.683366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":1,"reported_epoch":17,"state":"creating+peering","last_fresh":"2023-11-24T07:56:22.694439+0000","last_change":"2023-11-24T07:56:22.694439+0000","last_active":"2023-11-24T07:56:22.678019+0000","last_peered":"2023-11-24T07:56:22.678019+0000","last_clean":"2023-11-24T07:56:22.678019+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T07:56:22.694439+0000","last_undegraded":"2023-11-24T07:56:22.694439+0000","last_fullsized":"2023-11-24T07:56:22.694439+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:56:22.678019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:56:22.678019+0000","last_clean_scrub_stamp":"2023-11-24T07:56:22.678019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":1,"reported_epoch":17,"state":"creating+peering","last_fresh":"2023-11-24T07:56:22.694430+0000","last_change":"2023-11-24T07:56:22.694430+0000","last_active":"2023-11-24T07:56:22.678019+0000","last_peered":"2023-11-24T07:56:22.678019+0000","last_clean":"2023-11-24T07:56:22.678019+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T07:56:22.694430+0000","last_undegraded":"2023-11-24T07:56:22.694430+0000","last_fullsized":"2023-11-24T07:56:22.694430+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:56:22.678019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:56:22.678019+0000","last_clean_scrub_stamp":"2023-11-24T07:56:22.678019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T07:56:23.683366+0000","last_change":"2023-11-24T07:56:23.683366+0000","last_active":"2023-11-24T07:56:23.683366+0000","last_peered":"2023-11-24T07:56:23.683366+0000","last_clean":"2023-11-24T07:56:23.683366+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T07:56:23.683366+0000","last_undegraded":"2023-11-24T07:56:23.683366+0000","last_fullsized":"2023-11-24T07:56:23.683366+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:56:23.683366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:56:23.683366+0000","last_clean_scrub_stamp":"2023-11-24T07:56:23.683366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":1,"reported_epoch":17,"state":"creating+peering","last_fresh":"2023-11-24T07:56:22.690376+0000","last_change":"2023-11-24T07:56:22.690376+0000","last_active":"2023-11-24T07:56:22.678019+0000","last_peered":"2023-11-24T07:56:22.678019+0000","last_clean":"2023-11-24T07:56:22.678019+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T07:56:22.690376+0000","last_undegraded":"2023-11-24T07:56:22.690376+0000","last_fullsized":"2023-11-24T07:56:22.690376+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:56:22.678019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:56:22.678019+0000","last_clean_scrub_stamp":"2023-11-24T07:56:22.678019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T07:56:23.683366+0000","last_change":"2023-11-24T07:56:23.683366+0000","last_active":"2023-11-24T07:56:23.683366+0000","last_peered":"2023-11-24T07:56:23.683366+0000","last_clean":"2023-11-24T07:56:23.683366+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T07:56:23.683366+0000","last_undegraded":"2023-11-24T07:56:23.683366+0000","last_fullsized":"2023-11-24T07:56:23.683366+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:56:23.683366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:56:23.683366+0000","last_clean_scrub_stamp":"2023-11-24T07:56:23.683366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":1,"reported_epoch":17,"state":"creating+peering","last_fresh":"2023-11-24T07:56:22.695123+0000","last_change":"2023-11-24T07:56:22.695123+0000","last_active":"2023-11-24T07:56:22.678019+0000","last_peered":"2023-11-24T07:56:22.678019+0000","last_clean":"2023-11-24T07:56:22.678019+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T07:56:22.695123+0000","last_undegraded":"2023-11-24T07:56:22.695123+0000","last_fullsized":"2023-11-24T07:56:22.695123+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:56:22.678019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:56:22.678019+0000","last_clean_scrub_stamp":"2023-11-24T07:56:22.678019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T07:56:23.683366+0000","last_change":"2023-11-24T07:56:23.683366+0000","last_active":"2023-11-24T07:56:23.683366+0000","last_peered":"2023-11-24T07:56:23.683366+0000","last_clean":"2023-11-24T07:56:23.683366+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T07:56:23.683366+0000","last_undegraded":"2023-11-24T07:56:23.683366+0000","last_fullsized":"2023-11-24T07:56:23.683366+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:56:23.683366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:56:23.683366+0000","last_clean_scrub_stamp":"2023-11-24T07:56:23.683366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":1,"reported_epoch":17,"state":"creating+peering","last_fresh":"2023-11-24T07:56:22.690282+0000","last_change":"2023-11-24T07:56:22.690282+0000","last_active":"2023-11-24T07:56:22.678019+0000","last_peered":"2023-11-24T07:56:22.678019+0000","last_clean":"2023-11-24T07:56:22.678019+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T07:56:22.690282+0000","last_undegraded":"2023-11-24T07:56:22.690282+0000","last_fullsized":"2023-11-24T07:56:22.690282+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:56:22.678019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:56:22.678019+0000","last_clean_scrub_stamp":"2023-11-24T07:56:22.678019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T07:56:23.683366+0000","last_change":"2023-11-24T07:56:23.683366+0000","last_active":"2023-11-24T07:56:23.683366+0000","last_peered":"2023-11-24T07:56:23.683366+0000","last_clean":"2023-11-24T07:56:23.683366+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T07:56:23.683366+0000","last_undegraded":"2023-11-24T07:56:23.683366+0000","last_fullsized":"2023-11-24T07:56:23.683366+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:56:23.683366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:56:23.683366+0000","last_clean_scrub_stamp":"2023-11-24T07:56:23.683366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":1,"reported_epoch":17,"state":"creating+peering","last_fresh":"2023-11-24T07:56:22.690549+0000","last_change":"2023-11-24T07:56:22.690549+0000","last_active":"2023-11-24T07:56:22.678019+0000","last_peered":"2023-11-24T07:56:22.678019+0000","last_clean":"2023-11-24T07:56:22.678019+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T07:56:22.690549+0000","last_undegraded":"2023-11-24T07:56:22.690549+0000","last_fullsized":"2023-11-24T07:56:22.690549+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:56:22.678019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:56:22.678019+0000","last_clean_scrub_stamp":"2023-11-24T07:56:22.678019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T07:56:23.683366+0000","last_change":"2023-11-24T07:56:23.683366+0000","last_active":"2023-11-24T07:56:23.683366+0000","last_peered":"2023-11-24T07:56:23.683366+0000","last_clean":"2023-11-24T07:56:23.683366+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T07:56:23.683366+0000","last_undegraded":"2023-11-24T07:56:23.683366+0000","last_fullsized":"2023-11-24T07:56:23.683366+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:56:23.683366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:56:23.683366+0000","last_clean_scrub_stamp":"2023-11-24T07:56:23.683366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":1,"reported_epoch":17,"state":"creating+peering","last_fresh":"2023-11-24T07:56:22.689706+0000","last_change":"2023-11-24T07:56:22.689706+0000","last_active":"2023-11-24T07:56:22.678019+0000","last_peered":"2023-11-24T07:56:22.678019+0000","last_clean":"2023-11-24T07:56:22.678019+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T07:56:22.689706+0000","last_undegraded":"2023-11-24T07:56:22.689706+0000","last_fullsized":"2023-11-24T07:56:22.689706+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:56:22.678019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:56:22.678019+0000","last_clean_scrub_stamp":"2023-11-24T07:56:22.678019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T07:56:23.683366+0000","last_change":"2023-11-24T07:56:23.683366+0000","last_active":"2023-11-24T07:56:23.683366+0000","last_peered":"2023-11-24T07:56:23.683366+0000","last_clean":"2023-11-24T07:56:23.683366+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T07:56:23.683366+0000","last_undegraded":"2023-11-24T07:56:23.683366+0000","last_fullsized":"2023-11-24T07:56:23.683366+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:56:23.683366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:56:23.683366+0000","last_clean_scrub_stamp":"2023-11-24T07:56:23.683366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.b","version":"0'0","reported_seq":1,"reported_epoch":17,"state":"creating+peering","last_fresh":"2023-11-24T07:56:22.695809+0000","last_change":"2023-11-24T07:56:22.695809+0000","last_active":"2023-11-24T07:56:22.678019+0000","last_peered":"2023-11-24T07:56:22.678019+0000","last_clean":"2023-11-24T07:56:22.678019+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T07:56:22.695809+0000","last_undegraded":"2023-11-24T07:56:22.695809+0000","last_fullsized":"2023-11-24T07:56:22.695809+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:56:22.678019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:56:22.678019+0000","last_clean_scrub_stamp":"2023-11-24T07:56:22.678019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T07:56:23.683366+0000","last_change":"2023-11-24T07:56:23.683366+0000","last_active":"2023-11-24T07:56:23.683366+0000","last_peered":"2023-11-24T07:56:23.683366+0000","last_clean":"2023-11-24T07:56:23.683366+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T07:56:23.683366+0000","last_undegraded":"2023-11-24T07:56:23.683366+0000","last_fullsized":"2023-11-24T07:56:23.683366+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:56:23.683366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:56:23.683366+0000","last_clean_scrub_stamp":"2023-11-24T07:56:23.683366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":1,"reported_epoch":17,"state":"creating+peering","last_fresh":"2023-11-24T07:56:22.690094+0000","last_change":"2023-11-24T07:56:22.690094+0000","last_active":"2023-11-24T07:56:22.678019+0000","last_peered":"2023-11-24T07:56:22.678019+0000","last_clean":"2023-11-24T07:56:22.678019+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T07:56:22.690094+0000","last_undegraded":"2023-11-24T07:56:22.690094+0000","last_fullsized":"2023-11-24T07:56:22.690094+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:56:22.678019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:56:22.678019+0000","last_clean_scrub_stamp":"2023-11-24T07:56:22.678019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T07:56:23.683366+0000","last_change":"2023-11-24T07:56:23.683366+0000","last_active":"2023-11-24T07:56:23.683366+0000","last_peered":"2023-11-24T07:56:23.683366+0000","last_clean":"2023-11-24T07:56:23.683366+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T07:56:23.683366+0000","last_undegraded":"2023-11-24T07:56:23.683366+0000","last_fullsized":"2023-11-24T07:56:23.683366+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:56:23.683366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:56:23.683366+0000","last_clean_scrub_stamp":"2023-11-24T07:56:23.683366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.d","version":"0'0","reported_seq":1,"reported_epoch":17,"state":"creating+peering","last_fresh":"2023-11-24T07:56:22.688223+0000","last_change":"2023-11-24T07:56:22.688223+0000","last_active":"2023-11-24T07:56:22.678019+0000","last_peered":"2023-11-24T07:56:22.678019+0000","last_clean":"2023-11-24T07:56:22.678019+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T07:56:22.688223+0000","last_undegraded":"2023-11-24T07:56:22.688223+0000","last_fullsized":"2023-11-24T07:56:22.688223+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:56:22.678019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:56:22.678019+0000","last_clean_scrub_stamp":"2023-11-24T07:56:22.678019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T07:56:23.683366+0000","last_change":"2023-11-24T07:56:23.683366+0000","last_active":"2023-11-24T07:56:23.683366+0000","last_peered":"2023-11-24T07:56:23.683366+0000","last_clean":"2023-11-24T07:56:23.683366+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T07:56:23.683366+0000","last_undegraded":"2023-11-24T07:56:23.683366+0000","last_fullsized":"2023-11-24T07:56:23.683366+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:56:23.683366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:56:23.683366+0000","last_clean_scrub_stamp":"2023-11-24T07:56:23.683366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.e","version":"0'0","reported_seq":1,"reported_epoch":17,"state":"creating+peering","last_fresh":"2023-11-24T07:56:22.691566+0000","last_change":"2023-11-24T07:56:22.691566+0000","last_active":"2023-11-24T07:56:22.678019+0000","last_peered":"2023-11-24T07:56:22.678019+0000","last_clean":"2023-11-24T07:56:22.678019+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T07:56:22.691566+0000","last_undegraded":"2023-11-24T07:56:22.691566+0000","last_fullsized":"2023-11-24T07:56:22.691566+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:56:22.678019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:56:22.678019+0000","last_clean_scrub_stamp":"2023-11-24T07:56:22.678019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T07:56:23.683366+0000","last_change":"2023-11-24T07:56:23.683366+0000","last_active":"2023-11-24T07:56:23.683366+0000","last_peered":"2023-11-24T07:56:23.683366+0000","last_clean":"2023-11-24T07:56:23.683366+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T07:56:23.683366+0000","last_undegraded":"2023-11-24T07:56:23.683366+0000","last_fullsized":"2023-11-24T07:56:23.683366+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:56:23.683366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:56:23.683366+0000","last_clean_scrub_stamp":"2023-11-24T07:56:23.683366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T07:56:23.683366+0000","last_change":"2023-11-24T07:56:23.683366+0000","last_active":"2023-11-24T07:56:23.683366+0000","last_peered":"2023-11-24T07:56:23.683366+0000","last_clean":"2023-11-24T07:56:23.683366+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T07:56:23.683366+0000","last_undegraded":"2023-11-24T07:56:23.683366+0000","last_fullsized":"2023-11-24T07:56:23.683366+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:56:23.683366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:56:23.683366+0000","last_clean_scrub_stamp":"2023-11-24T07:56:23.683366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T07:56:23.683366+0000","last_change":"2023-11-24T07:56:23.683366+0000","last_active":"2023-11-24T07:56:23.683366+0000","last_peered":"2023-11-24T07:56:23.683366+0000","last_clean":"2023-11-24T07:56:23.683366+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T07:56:23.683366+0000","last_undegraded":"2023-11-24T07:56:23.683366+0000","last_fullsized":"2023-11-24T07:56:23.683366+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:56:23.683366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:56:23.683366+0000","last_clean_scrub_stamp":"2023-11-24T07:56:23.683366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T07:56:23.683366+0000","last_change":"2023-11-24T07:56:23.683366+0000","last_active":"2023-11-24T07:56:23.683366+0000","last_peered":"2023-11-24T07:56:23.683366+0000","last_clean":"2023-11-24T07:56:23.683366+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T07:56:23.683366+0000","last_undegraded":"2023-11-24T07:56:23.683366+0000","last_fullsized":"2023-11-24T07:56:23.683366+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:56:23.683366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:56:23.683366+0000","last_clean_scrub_stamp":"2023-11-24T07:56:23.683366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T07:56:23.683366+0000","last_change":"2023-11-24T07:56:23.683366+0000","last_active":"2023-11-24T07:56:23.683366+0000","last_peered":"2023-11-24T07:56:23.683366+0000","last_clean":"2023-11-24T07:56:23.683366+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T07:56:23.683366+0000","last_undegraded":"2023-11-24T07:56:23.683366+0000","last_fullsized":"2023-11-24T07:56:23.683366+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:56:23.683366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:56:23.683366+0000","last_clean_scrub_stamp":"2023-11-24T07:56:23.683366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T07:56:23.683366+0000","last_change":"2023-11-24T07:56:23.683366+0000","last_active":"2023-11-24T07:56:23.683366+0000","last_peered":"2023-11-24T07:56:23.683366+0000","last_clean":"2023-11-24T07:56:23.683366+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T07:56:23.683366+0000","last_undegraded":"2023-11-24T07:56:23.683366+0000","last_fullsized":"2023-11-24T07:56:23.683366+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:56:23.683366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:56:23.683366+0000","last_clean_scrub_stamp":"2023-11-24T07:56:23.683366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T07:56:23.683366+0000","last_change":"2023-11-24T07:56:23.683366+0000","last_active":"2023-11-24T07:56:23.683366+0000","last_peered":"2023-11-24T07:56:23.683366+0000","last_clean":"2023-11-24T07:56:23.683366+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T07:56:23.683366+0000","last_undegraded":"2023-11-24T07:56:23.683366+0000","last_fullsized":"2023-11-24T07:56:23.683366+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:56:23.683366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:56:23.683366+0000","last_clean_scrub_stamp":"2023-11-24T07:56:23.683366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T07:56:23.683366+0000","last_change":"2023-11-24T07:56:23.683366+0000","last_active":"2023-11-24T07:56:23.683366+0000","last_peered":"2023-11-24T07:56:23.683366+0000","last_clean":"2023-11-24T07:56:23.683366+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T07:56:23.683366+0000","last_undegraded":"2023-11-24T07:56:23.683366+0000","last_fullsized":"2023-11-24T07:56:23.683366+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:56:23.683366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:56:23.683366+0000","last_clean_scrub_stamp":"2023-11-24T07:56:23.683366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T07:56:23.683366+0000","last_change":"2023-11-24T07:56:23.683366+0000","last_active":"2023-11-24T07:56:23.683366+0000","last_peered":"2023-11-24T07:56:23.683366+0000","last_clean":"2023-11-24T07:56:23.683366+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T07:56:23.683366+0000","last_undegraded":"2023-11-24T07:56:23.683366+0000","last_fullsized":"2023-11-24T07:56:23.683366+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:56:23.683366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:56:23.683366+0000","last_clean_scrub_stamp":"2023-11-24T07:56:23.683366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T07:56:23.683366+0000","last_change":"2023-11-24T07:56:23.683366+0000","last_active":"2023-11-24T07:56:23.683366+0000","last_peered":"2023-11-24T07:56:23.683366+0000","last_clean":"2023-11-24T07:56:23.683366+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T07:56:23.683366+0000","last_undegraded":"2023-11-24T07:56:23.683366+0000","last_fullsized":"2023-11-24T07:56:23.683366+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:56:23.683366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:56:23.683366+0000","last_clean_scrub_stamp":"2023-11-24T07:56:23.683366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T07:56:23.683366+0000","last_change":"2023-11-24T07:56:23.683366+0000","last_active":"2023-11-24T07:56:23.683366+0000","last_peered":"2023-11-24T07:56:23.683366+0000","last_clean":"2023-11-24T07:56:23.683366+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T07:56:23.683366+0000","last_undegraded":"2023-11-24T07:56:23.683366+0000","last_fullsized":"2023-11-24T07:56:23.683366+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:56:23.683366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:56:23.683366+0000","last_clean_scrub_stamp":"2023-11-24T07:56:23.683366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T07:56:23.683366+0000","last_change":"2023-11-24T07:56:23.683366+0000","last_active":"2023-11-24T07:56:23.683366+0000","last_peered":"2023-11-24T07:56:23.683366+0000","last_clean":"2023-11-24T07:56:23.683366+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T07:56:23.683366+0000","last_undegraded":"2023-11-24T07:56:23.683366+0000","last_fullsized":"2023-11-24T07:56:23.683366+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:56:23.683366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:56:23.683366+0000","last_clean_scrub_stamp":"2023-11-24T07:56:23.683366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T07:56:23.683366+0000","last_change":"2023-11-24T07:56:23.683366+0000","last_active":"2023-11-24T07:56:23.683366+0000","last_peered":"2023-11-24T07:56:23.683366+0000","last_clean":"2023-11-24T07:56:23.683366+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T07:56:23.683366+0000","last_undegraded":"2023-11-24T07:56:23.683366+0000","last_fullsized":"2023-11-24T07:56:23.683366+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:56:23.683366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:56:23.683366+0000","last_clean_scrub_stamp":"2023-11-24T07:56:23.683366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T07:56:23.683366+0000","last_change":"2023-11-24T07:56:23.683366+0000","last_active":"2023-11-24T07:56:23.683366+0000","last_peered":"2023-11-24T07:56:23.683366+0000","last_clean":"2023-11-24T07:56:23.683366+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T07:56:23.683366+0000","last_undegraded":"2023-11-24T07:56:23.683366+0000","last_fullsized":"2023-11-24T07:56:23.683366+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:56:23.683366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:56:23.683366+0000","last_clean_scrub_stamp":"2023-11-24T07:56:23.683366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574851,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27172,"kb_used_data":152,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344668,"statfs":{"total":96636764160,"available":96608940032,"internally_reserved":0,"allocated":155648,"data_stored":45743,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.40100000000000002}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61599999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64400000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66300000000000003}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.42599999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.45900000000000002}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55000000000000004}]}]},{"osd":6,"up_from":13,"seq":55834574852,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27172,"kb_used_data":152,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344668,"statfs":{"total":96636764160,"available":96608940032,"internally_reserved":0,"allocated":155648,"data_stored":45743,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65300000000000002}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54100000000000004}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.76600000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.876}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59999999999999998}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.84999999999999998}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58099999999999996}]}]},{"osd":1,"up_from":13,"seq":55834574853,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27764,"kb_used_data":744,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344076,"statfs":{"total":96636764160,"available":96608333824,"internally_reserved":0,"allocated":761856,"data_stored":641928,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.33100000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.308}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.432}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.69699999999999995}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68000000000000005}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.72099999999999997}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.81200000000000006}]}]},{"osd":0,"up_from":13,"seq":55834574853,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27172,"kb_used_data":152,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344668,"statfs":{"total":96636764160,"available":96608940032,"internally_reserved":0,"allocated":155648,"data_stored":45743,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.28799999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.32800000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.30499999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.40500000000000003}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54200000000000004}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48499999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52600000000000002}]}]},{"osd":2,"up_from":13,"seq":55834574853,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27184,"kb_used_data":164,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344656,"statfs":{"total":96636764160,"available":96608927744,"internally_reserved":0,"allocated":167936,"data_stored":51560,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48699999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.44500000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53300000000000003}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.81599999999999995}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.84899999999999998}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.83599999999999997}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.80000000000000004}]}]},{"osd":3,"up_from":13,"seq":55834574853,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27184,"kb_used_data":164,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344656,"statfs":{"total":96636764160,"available":96608927744,"internally_reserved":0,"allocated":167936,"data_stored":51560,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.313}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.39600000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.35899999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.73099999999999998}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57199999999999995}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60899999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.71099999999999997}]}]},{"osd":4,"up_from":13,"seq":55834574852,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27184,"kb_used_data":164,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344656,"statfs":{"total":96636764160,"available":96608927744,"internally_reserved":0,"allocated":167936,"data_stored":51560,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.78400000000000003}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.42699999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.745}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65100000000000002}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47099999999999997}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55100000000000005}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58099999999999996}]}]},{"osd":5,"up_from":13,"seq":55834574853,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27764,"kb_used_data":744,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344076,"statfs":{"total":96636764160,"available":96608333824,"internally_reserved":0,"allocated":761856,"data_stored":641928,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0109999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.98499999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.86599999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.046}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51600000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54800000000000004}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.96599999999999997}]}]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-11-24T07:56:29.270 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-11-24T07:56:29.431 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:29.429+0000 7f1e1a0a9700 1 Processor -- start 2023-11-24T07:56:29.431 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:29.429+0000 7f1e1a0a9700 1 -- start start 2023-11-24T07:56:29.435 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:29.430+0000 7f1e1a0a9700 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f1e1406b620 0x7f1e140b3a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:56:29.435 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:29.430+0000 7f1e1a0a9700 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f1e1406ee60 0x7f1e140b3f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:56:29.435 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:29.430+0000 7f1e1a0a9700 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1e14071f50 con 0x7f1e140b5ab0 2023-11-24T07:56:29.435 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:29.430+0000 7f1e1a0a9700 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1e14072230 con 0x7f1e1406b620 2023-11-24T07:56:29.435 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:29.430+0000 7f1e1a0a9700 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1e14072510 con 0x7f1e1406ee60 2023-11-24T07:56:29.435 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:29.430+0000 7f1e12ffd700 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7f1e140b5ab0 0x7f1e14161ba0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:53270/0 (socket says 172.21.15.59:53270) 2023-11-24T07:56:29.436 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:29.430+0000 7f1e12ffd700 1 -- 172.21.15.59:0/3365480297 learned_addr learned my addr 172.21.15.59:0/3365480297 (peer_addr_for_me v1:172.21.15.59:0/0) 2023-11-24T07:56:29.436 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:29.430+0000 7f1e137fe700 1 --2- 172.21.15.59:0/3365480297 >> v2:172.21.15.76:3300/0 conn(0x7f1e1406b620 0x7f1e140b3a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:56:29.436 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:29.430+0000 7f1e13fff700 1 --2- 172.21.15.59:0/3365480297 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f1e1406ee60 0x7f1e140b3f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:56:29.436 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:29.430+0000 7f1e127fc700 1 -- 172.21.15.59:0/3365480297 <== mon.0 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 658395444 0 0) 0x7f1e14071f50 con 0x7f1e140b5ab0 2023-11-24T07:56:29.436 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:29.430+0000 7f1e127fc700 1 -- 172.21.15.59:0/3365480297 --> v1:172.21.15.59:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1e000035b0 con 0x7f1e140b5ab0 2023-11-24T07:56:29.436 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:29.430+0000 7f1e127fc700 1 -- 172.21.15.59:0/3365480297 <== mon.0 v1:172.21.15.59:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4173408987 0 0) 0x7f1e000035b0 con 0x7f1e140b5ab0 2023-11-24T07:56:29.436 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:29.431+0000 7f1e127fc700 1 -- 172.21.15.59:0/3365480297 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f1e1406ee60 msgr2=0x7f1e140b3f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T07:56:29.436 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:29.431+0000 7f1e127fc700 1 --2- 172.21.15.59:0/3365480297 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f1e1406ee60 0x7f1e140b3f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:56:29.437 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:29.431+0000 7f1e127fc700 1 -- 172.21.15.59:0/3365480297 >> v2:172.21.15.76:3300/0 conn(0x7f1e1406b620 msgr2=0x7f1e140b3a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T07:56:29.437 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:29.431+0000 7f1e127fc700 1 --2- 172.21.15.59:0/3365480297 >> v2:172.21.15.76:3300/0 conn(0x7f1e1406b620 0x7f1e140b3a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:56:29.437 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:29.431+0000 7f1e127fc700 1 -- 172.21.15.59:0/3365480297 --> v1:172.21.15.59:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1e140727f0 con 0x7f1e140b5ab0 2023-11-24T07:56:29.437 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:29.431+0000 7f1e127fc700 1 -- 172.21.15.59:0/3365480297 <== mon.0 v1:172.21.15.59:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1312333566 0 0) 0x7f1dfc001670 con 0x7f1e140b5ab0 2023-11-24T07:56:29.437 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:29.431+0000 7f1e127fc700 1 -- 172.21.15.59:0/3365480297 >> v1:172.21.15.59:6789/0 conn(0x7f1e140b5ab0 legacy=0x7f1e14161ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:56:29.437 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:29.431+0000 7f1e127fc700 1 --2- 172.21.15.59:0/3365480297 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1e00003a00 0x7f1e00003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:56:29.437 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:29.431+0000 7f1e127fc700 1 --2- 172.21.15.59:0/3365480297 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1e00004850 0x7f1e00004cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:56:29.437 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:29.431+0000 7f1e127fc700 1 --2- 172.21.15.59:0/3365480297 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f1e1406ee60 0x7f1e00005210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:56:29.437 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:29.431+0000 7f1e127fc700 1 -- 172.21.15.59:0/3365480297 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e14072230 con 0x7f1e00003a00 2023-11-24T07:56:29.437 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:29.431+0000 7f1e127fc700 1 -- 172.21.15.59:0/3365480297 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e14072510 con 0x7f1e00004850 2023-11-24T07:56:29.438 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:29.431+0000 7f1e127fc700 1 -- 172.21.15.59:0/3365480297 --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1e0000ba10 con 0x7f1e1406ee60 2023-11-24T07:56:29.438 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:29.431+0000 7f1e137fe700 1 --2- 172.21.15.59:0/3365480297 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1e00003a00 0x7f1e00003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:56:29.438 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:29.431+0000 7f1e12ffd700 1 --2- 172.21.15.59:0/3365480297 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1e00004850 0x7f1e00004cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:56:29.438 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:29.431+0000 7f1e13fff700 1 --2- 172.21.15.59:0/3365480297 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f1e1406ee60 0x7f1e00005210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:56:29.438 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:29.431+0000 7f1e137fe700 1 -- 172.21.15.59:0/3365480297 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f1e1406ee60 msgr2=0x7f1e00005210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T07:56:29.438 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:29.431+0000 7f1e137fe700 1 --2- 172.21.15.59:0/3365480297 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f1e1406ee60 0x7f1e00005210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:56:29.438 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:29.431+0000 7f1e137fe700 1 -- 172.21.15.59:0/3365480297 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1e00004850 msgr2=0x7f1e00004cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T07:56:29.438 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:29.431+0000 7f1e137fe700 1 --2- 172.21.15.59:0/3365480297 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1e00004850 0x7f1e00004cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:56:29.438 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:29.431+0000 7f1e137fe700 1 -- 172.21.15.59:0/3365480297 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1e000035b0 con 0x7f1e00003a00 2023-11-24T07:56:29.439 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:29.431+0000 7f1e12ffd700 1 --2- 172.21.15.59:0/3365480297 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1e00004850 0x7f1e00004cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-24T07:56:29.439 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:29.431+0000 7f1e137fe700 1 --2- 172.21.15.59:0/3365480297 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1e00003a00 0x7f1e00003e20 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f1e04009b60 tx=0x7f1e00005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a3f4162d4bea255f server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:56:29.439 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:29.431+0000 7f1e127fc700 1 -- 172.21.15.59:0/3365480297 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e0400e6c0 con 0x7f1e00003a00 2023-11-24T07:56:29.439 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:29.431+0000 7f1e127fc700 1 -- 172.21.15.59:0/3365480297 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1e0400e820 con 0x7f1e00003a00 2023-11-24T07:56:29.439 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:29.431+0000 7f1e127fc700 1 -- 172.21.15.59:0/3365480297 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e0400eb90 con 0x7f1e00003a00 2023-11-24T07:56:29.439 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:29.432+0000 7f1e1a0a9700 1 -- 172.21.15.59:0/3365480297 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1e00003a00 msgr2=0x7f1e00003e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:56:29.439 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:29.432+0000 7f1e1a0a9700 1 --2- 172.21.15.59:0/3365480297 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1e00003a00 0x7f1e00003e20 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f1e04009b60 tx=0x7f1e00005880 comp rx=0 tx=0).stop 2023-11-24T07:56:29.439 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:29.432+0000 7f1e13fff700 1 -- 172.21.15.59:0/3365480297 reap_dead start 2023-11-24T07:56:29.439 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:29.432+0000 7f1e1a0a9700 1 -- 172.21.15.59:0/3365480297 shutdown_connections 2023-11-24T07:56:29.439 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:29.432+0000 7f1e1a0a9700 1 -- 172.21.15.59:0/3365480297 >> 172.21.15.59:0/3365480297 conn(0x7f1e140b2740 msgr2=0x7f1e140b2b20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:56:29.439 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:29.432+0000 7f1e1a0a9700 1 -- 172.21.15.59:0/3365480297 shutdown_connections 2023-11-24T07:56:29.440 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:29.432+0000 7f1e1a0a9700 1 -- 172.21.15.59:0/3365480297 wait complete. 2023-11-24T07:56:29.440 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:29.433+0000 7f1e1a0a9700 1 Processor -- start 2023-11-24T07:56:29.440 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:29.433+0000 7f1e1a0a9700 1 -- start start 2023-11-24T07:56:29.442 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:29.434+0000 7f1e1a0a9700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1e00003a00 0x7f1e14143520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:56:29.443 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:29.434+0000 7f1e1a0a9700 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f1e140c59b0 0x7f1e140b16b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:56:29.443 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:29.434+0000 7f1e1a0a9700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1e140c5e30 0x7f1e14143e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:56:29.443 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:29.434+0000 7f1e1a0a9700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e141f7800 con 0x7f1e00003a00 2023-11-24T07:56:29.443 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:29.434+0000 7f1e1a0a9700 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e141f7ae0 con 0x7f1e140c5e30 2023-11-24T07:56:29.443 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:29.434+0000 7f1e1a0a9700 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1e141f7dc0 con 0x7f1e140c59b0 2023-11-24T07:56:29.443 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:29.434+0000 7f1e137fe700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1e00003a00 0x7f1e14143520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:56:29.443 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:29.434+0000 7f1e137fe700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1e00003a00 0x7f1e14143520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:49806/0 (socket says 172.21.15.59:49806) 2023-11-24T07:56:29.443 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:29.434+0000 7f1e137fe700 1 -- 172.21.15.59:0/1908909277 learned_addr learned my addr 172.21.15.59:0/1908909277 (peer_addr_for_me v2:172.21.15.59:0/0) 2023-11-24T07:56:29.443 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:29.434+0000 7f1e12ffd700 1 --2- 172.21.15.59:0/1908909277 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f1e140c59b0 0x7f1e140b16b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:56:29.443 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:29.434+0000 7f1e13fff700 1 --2- 172.21.15.59:0/1908909277 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1e140c5e30 0x7f1e14143e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:56:29.444 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:29.434+0000 7f1e137fe700 1 -- 172.21.15.59:0/1908909277 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f1e140c59b0 msgr2=0x7f1e140b16b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:56:29.444 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:29.434+0000 7f1e137fe700 1 --2- 172.21.15.59:0/1908909277 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f1e140c59b0 0x7f1e140b16b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:56:29.444 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:29.435+0000 7f1e137fe700 1 -- 172.21.15.59:0/1908909277 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1e140c5e30 msgr2=0x7f1e14143e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:56:29.444 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:29.435+0000 7f1e137fe700 1 --2- 172.21.15.59:0/1908909277 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1e140c5e30 0x7f1e14143e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:56:29.444 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:29.435+0000 7f1e137fe700 1 -- 172.21.15.59:0/1908909277 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1e141f80a0 con 0x7f1e00003a00 2023-11-24T07:56:29.444 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:29.435+0000 7f1e137fe700 1 --2- 172.21.15.59:0/1908909277 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1e00003a00 0x7f1e14143520 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f1e000046a0 tx=0x7f1e040047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:56:29.444 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:29.435+0000 7f1e0bfff700 1 -- 172.21.15.59:0/1908909277 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e0400e6c0 con 0x7f1e00003a00 2023-11-24T07:56:29.444 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:29.435+0000 7f1e0bfff700 1 -- 172.21.15.59:0/1908909277 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1e04003ab0 con 0x7f1e00003a00 2023-11-24T07:56:29.444 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:29.435+0000 7f1e1a0a9700 1 -- 172.21.15.59:0/1908909277 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1e14143a60 con 0x7f1e00003a00 2023-11-24T07:56:29.444 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:29.435+0000 7f1e0bfff700 1 -- 172.21.15.59:0/1908909277 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e04003e60 con 0x7f1e00003a00 2023-11-24T07:56:29.445 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:29.435+0000 7f1e1a0a9700 1 -- 172.21.15.59:0/1908909277 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1e140b1dd0 con 0x7f1e00003a00 2023-11-24T07:56:29.445 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:29.436+0000 7f1e1a0a9700 1 -- 172.21.15.59:0/1908909277 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1e141f40d0 con 0x7f1e00003a00 2023-11-24T07:56:29.445 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:29.436+0000 7f1e0bfff700 1 -- 172.21.15.59:0/1908909277 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f1e04013470 con 0x7f1e00003a00 2023-11-24T07:56:29.445 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:29.437+0000 7f1e0bfff700 1 --2- 172.21.15.59:0/1908909277 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f1df406aed0 0x7f1df406d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:56:29.445 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:29.437+0000 7f1e0bfff700 1 -- 172.21.15.59:0/1908909277 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f1e0408add0 con 0x7f1e00003a00 2023-11-24T07:56:29.445 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:29.437+0000 7f1e12ffd700 1 --2- 172.21.15.59:0/1908909277 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f1df406aed0 0x7f1df406d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:56:29.445 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:29.437+0000 7f1e12ffd700 1 --2- 172.21.15.59:0/1908909277 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f1df406aed0 0x7f1df406d390 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f1e140cfc60 tx=0x7f1dfc007040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:56:29.446 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:29.441+0000 7f1e0bfff700 1 -- 172.21.15.59:0/1908909277 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f1e04056030 con 0x7f1e00003a00 2023-11-24T07:56:29.637 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:29.635+0000 7f1e1a0a9700 1 -- 172.21.15.59:0/1908909277 --> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f1e141f7800 con 0x7f1df406aed0 2023-11-24T07:56:29.644 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:29.643+0000 7f1e0bfff700 1 -- 172.21.15.59:0/1908909277 <== mgr.4100 v2:172.21.15.76:6832/4264046536 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+252811 (secure 0 0 0) 0x7f1e141f7800 con 0x7f1df406aed0 2023-11-24T07:56:29.647 INFO:teuthology.orchestra.run.smithi059.stdout: 2023-11-24T07:56:29.649 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:29.645+0000 7f1e1a0a9700 1 -- 172.21.15.59:0/1908909277 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f1df406aed0 msgr2=0x7f1df406d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:56:29.649 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:29.645+0000 7f1e1a0a9700 1 --2- 172.21.15.59:0/1908909277 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f1df406aed0 0x7f1df406d390 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f1e140cfc60 tx=0x7f1dfc007040 comp rx=0 tx=0).stop 2023-11-24T07:56:29.649 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:29.645+0000 7f1e1a0a9700 1 -- 172.21.15.59:0/1908909277 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1e00003a00 msgr2=0x7f1e14143520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:56:29.649 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:29.645+0000 7f1e1a0a9700 1 --2- 172.21.15.59:0/1908909277 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1e00003a00 0x7f1e14143520 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f1e000046a0 tx=0x7f1e040047c0 comp rx=0 tx=0).stop 2023-11-24T07:56:29.650 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:29.645+0000 7f1e1a0a9700 1 -- 172.21.15.59:0/1908909277 shutdown_connections 2023-11-24T07:56:29.650 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:29.645+0000 7f1e1a0a9700 1 --2- 172.21.15.59:0/1908909277 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f1df406aed0 0x7f1df406d390 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:56:29.650 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:29.646+0000 7f1e1a0a9700 1 --2- 172.21.15.59:0/1908909277 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1e140c5e30 0x7f1e14143e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:56:29.650 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:29.646+0000 7f1e1a0a9700 1 --2- 172.21.15.59:0/1908909277 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1e00003a00 0x7f1e14143520 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:56:29.650 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:29.646+0000 7f1e1a0a9700 1 --2- 172.21.15.59:0/1908909277 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f1e140c59b0 0x7f1e140b16b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:56:29.650 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:29.646+0000 7f1e1a0a9700 1 -- 172.21.15.59:0/1908909277 >> 172.21.15.59:0/1908909277 conn(0x7f1e140b2740 msgr2=0x7f1e1406f320 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:56:29.650 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:29.646+0000 7f1e1a0a9700 1 -- 172.21.15.59:0/1908909277 shutdown_connections 2023-11-24T07:56:29.651 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:29.646+0000 7f1e1a0a9700 1 -- 172.21.15.59:0/1908909277 wait complete. 2023-11-24T07:56:29.651 INFO:teuthology.orchestra.run.smithi059.stderr:dumped all 2023-11-24T07:56:29.660 INFO:teuthology.orchestra.run.smithi059.stdout:{"pg_ready":true,"pg_map":{"version":18,"stamp":"2023-11-24T07:56:28.298177+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":66,"acting":66,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":14,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":218596,"kb_used_data":2436,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754756124,"statfs":{"total":773094113280,"available":772870270976,"internally_reserved":0,"allocated":2494464,"data_stored":1575765,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12745,"internal_metadata":221236791},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":14,"apply_latency_ms":14,"commit_latency_ns":14000000,"apply_latency_ns":14000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.611089"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T07:56:23.683366+0000","last_change":"2023-11-24T07:56:23.683366+0000","last_active":"2023-11-24T07:56:23.683366+0000","last_peered":"2023-11-24T07:56:23.683366+0000","last_clean":"2023-11-24T07:56:23.683366+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T07:56:23.683366+0000","last_undegraded":"2023-11-24T07:56:23.683366+0000","last_fullsized":"2023-11-24T07:56:23.683366+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:56:23.683366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:56:23.683366+0000","last_clean_scrub_stamp":"2023-11-24T07:56:23.683366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T07:56:23.683366+0000","last_change":"2023-11-24T07:56:23.683366+0000","last_active":"2023-11-24T07:56:23.683366+0000","last_peered":"2023-11-24T07:56:23.683366+0000","last_clean":"2023-11-24T07:56:23.683366+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T07:56:23.683366+0000","last_undegraded":"2023-11-24T07:56:23.683366+0000","last_fullsized":"2023-11-24T07:56:23.683366+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:56:23.683366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:56:23.683366+0000","last_clean_scrub_stamp":"2023-11-24T07:56:23.683366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T07:56:23.683366+0000","last_change":"2023-11-24T07:56:23.683366+0000","last_active":"2023-11-24T07:56:23.683366+0000","last_peered":"2023-11-24T07:56:23.683366+0000","last_clean":"2023-11-24T07:56:23.683366+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T07:56:23.683366+0000","last_undegraded":"2023-11-24T07:56:23.683366+0000","last_fullsized":"2023-11-24T07:56:23.683366+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:56:23.683366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:56:23.683366+0000","last_clean_scrub_stamp":"2023-11-24T07:56:23.683366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T07:56:23.683366+0000","last_change":"2023-11-24T07:56:23.683366+0000","last_active":"2023-11-24T07:56:23.683366+0000","last_peered":"2023-11-24T07:56:23.683366+0000","last_clean":"2023-11-24T07:56:23.683366+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T07:56:23.683366+0000","last_undegraded":"2023-11-24T07:56:23.683366+0000","last_fullsized":"2023-11-24T07:56:23.683366+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:56:23.683366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:56:23.683366+0000","last_clean_scrub_stamp":"2023-11-24T07:56:23.683366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T07:56:23.683366+0000","last_change":"2023-11-24T07:56:23.683366+0000","last_active":"2023-11-24T07:56:23.683366+0000","last_peered":"2023-11-24T07:56:23.683366+0000","last_clean":"2023-11-24T07:56:23.683366+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T07:56:23.683366+0000","last_undegraded":"2023-11-24T07:56:23.683366+0000","last_fullsized":"2023-11-24T07:56:23.683366+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:56:23.683366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:56:23.683366+0000","last_clean_scrub_stamp":"2023-11-24T07:56:23.683366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T07:56:23.683366+0000","last_change":"2023-11-24T07:56:23.683366+0000","last_active":"2023-11-24T07:56:23.683366+0000","last_peered":"2023-11-24T07:56:23.683366+0000","last_clean":"2023-11-24T07:56:23.683366+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T07:56:23.683366+0000","last_undegraded":"2023-11-24T07:56:23.683366+0000","last_fullsized":"2023-11-24T07:56:23.683366+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:56:23.683366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:56:23.683366+0000","last_clean_scrub_stamp":"2023-11-24T07:56:23.683366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T07:56:23.683366+0000","last_change":"2023-11-24T07:56:23.683366+0000","last_active":"2023-11-24T07:56:23.683366+0000","last_peered":"2023-11-24T07:56:23.683366+0000","last_clean":"2023-11-24T07:56:23.683366+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T07:56:23.683366+0000","last_undegraded":"2023-11-24T07:56:23.683366+0000","last_fullsized":"2023-11-24T07:56:23.683366+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:56:23.683366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:56:23.683366+0000","last_clean_scrub_stamp":"2023-11-24T07:56:23.683366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T07:56:23.683366+0000","last_change":"2023-11-24T07:56:23.683366+0000","last_active":"2023-11-24T07:56:23.683366+0000","last_peered":"2023-11-24T07:56:23.683366+0000","last_clean":"2023-11-24T07:56:23.683366+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T07:56:23.683366+0000","last_undegraded":"2023-11-24T07:56:23.683366+0000","last_fullsized":"2023-11-24T07:56:23.683366+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:56:23.683366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:56:23.683366+0000","last_clean_scrub_stamp":"2023-11-24T07:56:23.683366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T07:56:23.683366+0000","last_change":"2023-11-24T07:56:23.683366+0000","last_active":"2023-11-24T07:56:23.683366+0000","last_peered":"2023-11-24T07:56:23.683366+0000","last_clean":"2023-11-24T07:56:23.683366+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T07:56:23.683366+0000","last_undegraded":"2023-11-24T07:56:23.683366+0000","last_fullsized":"2023-11-24T07:56:23.683366+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:56:23.683366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:56:23.683366+0000","last_clean_scrub_stamp":"2023-11-24T07:56:23.683366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T07:56:23.683366+0000","last_change":"2023-11-24T07:56:23.683366+0000","last_active":"2023-11-24T07:56:23.683366+0000","last_peered":"2023-11-24T07:56:23.683366+0000","last_clean":"2023-11-24T07:56:23.683366+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T07:56:23.683366+0000","last_undegraded":"2023-11-24T07:56:23.683366+0000","last_fullsized":"2023-11-24T07:56:23.683366+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:56:23.683366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:56:23.683366+0000","last_clean_scrub_stamp":"2023-11-24T07:56:23.683366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T07:56:23.683366+0000","last_change":"2023-11-24T07:56:23.683366+0000","last_active":"2023-11-24T07:56:23.683366+0000","last_peered":"2023-11-24T07:56:23.683366+0000","last_clean":"2023-11-24T07:56:23.683366+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T07:56:23.683366+0000","last_undegraded":"2023-11-24T07:56:23.683366+0000","last_fullsized":"2023-11-24T07:56:23.683366+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:56:23.683366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:56:23.683366+0000","last_clean_scrub_stamp":"2023-11-24T07:56:23.683366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T07:56:23.683366+0000","last_change":"2023-11-24T07:56:23.683366+0000","last_active":"2023-11-24T07:56:23.683366+0000","last_peered":"2023-11-24T07:56:23.683366+0000","last_clean":"2023-11-24T07:56:23.683366+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T07:56:23.683366+0000","last_undegraded":"2023-11-24T07:56:23.683366+0000","last_fullsized":"2023-11-24T07:56:23.683366+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:56:23.683366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:56:23.683366+0000","last_clean_scrub_stamp":"2023-11-24T07:56:23.683366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T07:56:23.683366+0000","last_change":"2023-11-24T07:56:23.683366+0000","last_active":"2023-11-24T07:56:23.683366+0000","last_peered":"2023-11-24T07:56:23.683366+0000","last_clean":"2023-11-24T07:56:23.683366+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T07:56:23.683366+0000","last_undegraded":"2023-11-24T07:56:23.683366+0000","last_fullsized":"2023-11-24T07:56:23.683366+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:56:23.683366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:56:23.683366+0000","last_clean_scrub_stamp":"2023-11-24T07:56:23.683366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T07:56:23.683366+0000","last_change":"2023-11-24T07:56:23.683366+0000","last_active":"2023-11-24T07:56:23.683366+0000","last_peered":"2023-11-24T07:56:23.683366+0000","last_clean":"2023-11-24T07:56:23.683366+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T07:56:23.683366+0000","last_undegraded":"2023-11-24T07:56:23.683366+0000","last_fullsized":"2023-11-24T07:56:23.683366+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:56:23.683366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:56:23.683366+0000","last_clean_scrub_stamp":"2023-11-24T07:56:23.683366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T07:56:23.683366+0000","last_change":"2023-11-24T07:56:23.683366+0000","last_active":"2023-11-24T07:56:23.683366+0000","last_peered":"2023-11-24T07:56:23.683366+0000","last_clean":"2023-11-24T07:56:23.683366+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T07:56:23.683366+0000","last_undegraded":"2023-11-24T07:56:23.683366+0000","last_fullsized":"2023-11-24T07:56:23.683366+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:56:23.683366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:56:23.683366+0000","last_clean_scrub_stamp":"2023-11-24T07:56:23.683366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T07:56:23.683366+0000","last_change":"2023-11-24T07:56:23.683366+0000","last_active":"2023-11-24T07:56:23.683366+0000","last_peered":"2023-11-24T07:56:23.683366+0000","last_clean":"2023-11-24T07:56:23.683366+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T07:56:23.683366+0000","last_undegraded":"2023-11-24T07:56:23.683366+0000","last_fullsized":"2023-11-24T07:56:23.683366+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:56:23.683366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:56:23.683366+0000","last_clean_scrub_stamp":"2023-11-24T07:56:23.683366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T07:56:23.683366+0000","last_change":"2023-11-24T07:56:23.683366+0000","last_active":"2023-11-24T07:56:23.683366+0000","last_peered":"2023-11-24T07:56:23.683366+0000","last_clean":"2023-11-24T07:56:23.683366+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T07:56:23.683366+0000","last_undegraded":"2023-11-24T07:56:23.683366+0000","last_fullsized":"2023-11-24T07:56:23.683366+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:56:23.683366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:56:23.683366+0000","last_clean_scrub_stamp":"2023-11-24T07:56:23.683366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T07:56:23.683366+0000","last_change":"2023-11-24T07:56:23.683366+0000","last_active":"2023-11-24T07:56:23.683366+0000","last_peered":"2023-11-24T07:56:23.683366+0000","last_clean":"2023-11-24T07:56:23.683366+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T07:56:23.683366+0000","last_undegraded":"2023-11-24T07:56:23.683366+0000","last_fullsized":"2023-11-24T07:56:23.683366+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:56:23.683366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:56:23.683366+0000","last_clean_scrub_stamp":"2023-11-24T07:56:23.683366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T07:56:23.683366+0000","last_change":"2023-11-24T07:56:23.683366+0000","last_active":"2023-11-24T07:56:23.683366+0000","last_peered":"2023-11-24T07:56:23.683366+0000","last_clean":"2023-11-24T07:56:23.683366+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T07:56:23.683366+0000","last_undegraded":"2023-11-24T07:56:23.683366+0000","last_fullsized":"2023-11-24T07:56:23.683366+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:56:23.683366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:56:23.683366+0000","last_clean_scrub_stamp":"2023-11-24T07:56:23.683366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1f","version":"0'0","reported_seq":1,"reported_epoch":17,"state":"creating+peering","last_fresh":"2023-11-24T07:56:22.690428+0000","last_change":"2023-11-24T07:56:22.690428+0000","last_active":"2023-11-24T07:56:22.678019+0000","last_peered":"2023-11-24T07:56:22.678019+0000","last_clean":"2023-11-24T07:56:22.678019+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T07:56:22.690428+0000","last_undegraded":"2023-11-24T07:56:22.690428+0000","last_fullsized":"2023-11-24T07:56:22.690428+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:56:22.678019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:56:22.678019+0000","last_clean_scrub_stamp":"2023-11-24T07:56:22.678019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T07:56:23.683366+0000","last_change":"2023-11-24T07:56:23.683366+0000","last_active":"2023-11-24T07:56:23.683366+0000","last_peered":"2023-11-24T07:56:23.683366+0000","last_clean":"2023-11-24T07:56:23.683366+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T07:56:23.683366+0000","last_undegraded":"2023-11-24T07:56:23.683366+0000","last_fullsized":"2023-11-24T07:56:23.683366+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:56:23.683366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:56:23.683366+0000","last_clean_scrub_stamp":"2023-11-24T07:56:23.683366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1e","version":"0'0","reported_seq":1,"reported_epoch":17,"state":"creating+peering","last_fresh":"2023-11-24T07:56:22.686164+0000","last_change":"2023-11-24T07:56:22.686164+0000","last_active":"2023-11-24T07:56:22.678019+0000","last_peered":"2023-11-24T07:56:22.678019+0000","last_clean":"2023-11-24T07:56:22.678019+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T07:56:22.686164+0000","last_undegraded":"2023-11-24T07:56:22.686164+0000","last_fullsized":"2023-11-24T07:56:22.686164+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:56:22.678019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:56:22.678019+0000","last_clean_scrub_stamp":"2023-11-24T07:56:22.678019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T07:56:23.683366+0000","last_change":"2023-11-24T07:56:23.683366+0000","last_active":"2023-11-24T07:56:23.683366+0000","last_peered":"2023-11-24T07:56:23.683366+0000","last_clean":"2023-11-24T07:56:23.683366+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T07:56:23.683366+0000","last_undegraded":"2023-11-24T07:56:23.683366+0000","last_fullsized":"2023-11-24T07:56:23.683366+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:56:23.683366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:56:23.683366+0000","last_clean_scrub_stamp":"2023-11-24T07:56:23.683366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1d","version":"0'0","reported_seq":1,"reported_epoch":17,"state":"creating+peering","last_fresh":"2023-11-24T07:56:22.690762+0000","last_change":"2023-11-24T07:56:22.690762+0000","last_active":"2023-11-24T07:56:22.678019+0000","last_peered":"2023-11-24T07:56:22.678019+0000","last_clean":"2023-11-24T07:56:22.678019+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T07:56:22.690762+0000","last_undegraded":"2023-11-24T07:56:22.690762+0000","last_fullsized":"2023-11-24T07:56:22.690762+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:56:22.678019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:56:22.678019+0000","last_clean_scrub_stamp":"2023-11-24T07:56:22.678019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T07:56:23.683366+0000","last_change":"2023-11-24T07:56:23.683366+0000","last_active":"2023-11-24T07:56:23.683366+0000","last_peered":"2023-11-24T07:56:23.683366+0000","last_clean":"2023-11-24T07:56:23.683366+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T07:56:23.683366+0000","last_undegraded":"2023-11-24T07:56:23.683366+0000","last_fullsized":"2023-11-24T07:56:23.683366+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:56:23.683366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:56:23.683366+0000","last_clean_scrub_stamp":"2023-11-24T07:56:23.683366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1c","version":"0'0","reported_seq":1,"reported_epoch":17,"state":"creating+peering","last_fresh":"2023-11-24T07:56:22.690382+0000","last_change":"2023-11-24T07:56:22.690382+0000","last_active":"2023-11-24T07:56:22.678019+0000","last_peered":"2023-11-24T07:56:22.678019+0000","last_clean":"2023-11-24T07:56:22.678019+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T07:56:22.690382+0000","last_undegraded":"2023-11-24T07:56:22.690382+0000","last_fullsized":"2023-11-24T07:56:22.690382+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:56:22.678019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:56:22.678019+0000","last_clean_scrub_stamp":"2023-11-24T07:56:22.678019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T07:56:23.683366+0000","last_change":"2023-11-24T07:56:23.683366+0000","last_active":"2023-11-24T07:56:23.683366+0000","last_peered":"2023-11-24T07:56:23.683366+0000","last_clean":"2023-11-24T07:56:23.683366+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T07:56:23.683366+0000","last_undegraded":"2023-11-24T07:56:23.683366+0000","last_fullsized":"2023-11-24T07:56:23.683366+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:56:23.683366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:56:23.683366+0000","last_clean_scrub_stamp":"2023-11-24T07:56:23.683366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1b","version":"0'0","reported_seq":1,"reported_epoch":17,"state":"creating+peering","last_fresh":"2023-11-24T07:56:22.686185+0000","last_change":"2023-11-24T07:56:22.686185+0000","last_active":"2023-11-24T07:56:22.678019+0000","last_peered":"2023-11-24T07:56:22.678019+0000","last_clean":"2023-11-24T07:56:22.678019+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T07:56:22.686185+0000","last_undegraded":"2023-11-24T07:56:22.686185+0000","last_fullsized":"2023-11-24T07:56:22.686185+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:56:22.678019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:56:22.678019+0000","last_clean_scrub_stamp":"2023-11-24T07:56:22.678019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T07:56:23.683366+0000","last_change":"2023-11-24T07:56:23.683366+0000","last_active":"2023-11-24T07:56:23.683366+0000","last_peered":"2023-11-24T07:56:23.683366+0000","last_clean":"2023-11-24T07:56:23.683366+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T07:56:23.683366+0000","last_undegraded":"2023-11-24T07:56:23.683366+0000","last_fullsized":"2023-11-24T07:56:23.683366+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:56:23.683366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:56:23.683366+0000","last_clean_scrub_stamp":"2023-11-24T07:56:23.683366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1a","version":"0'0","reported_seq":1,"reported_epoch":17,"state":"creating+peering","last_fresh":"2023-11-24T07:56:22.696543+0000","last_change":"2023-11-24T07:56:22.696543+0000","last_active":"2023-11-24T07:56:22.678019+0000","last_peered":"2023-11-24T07:56:22.678019+0000","last_clean":"2023-11-24T07:56:22.678019+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T07:56:22.696543+0000","last_undegraded":"2023-11-24T07:56:22.696543+0000","last_fullsized":"2023-11-24T07:56:22.696543+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:56:22.678019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:56:22.678019+0000","last_clean_scrub_stamp":"2023-11-24T07:56:22.678019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T07:56:23.683366+0000","last_change":"2023-11-24T07:56:23.683366+0000","last_active":"2023-11-24T07:56:23.683366+0000","last_peered":"2023-11-24T07:56:23.683366+0000","last_clean":"2023-11-24T07:56:23.683366+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T07:56:23.683366+0000","last_undegraded":"2023-11-24T07:56:23.683366+0000","last_fullsized":"2023-11-24T07:56:23.683366+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:56:23.683366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:56:23.683366+0000","last_clean_scrub_stamp":"2023-11-24T07:56:23.683366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":1,"reported_epoch":17,"state":"creating+peering","last_fresh":"2023-11-24T07:56:22.689685+0000","last_change":"2023-11-24T07:56:22.689685+0000","last_active":"2023-11-24T07:56:22.678019+0000","last_peered":"2023-11-24T07:56:22.678019+0000","last_clean":"2023-11-24T07:56:22.678019+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T07:56:22.689685+0000","last_undegraded":"2023-11-24T07:56:22.689685+0000","last_fullsized":"2023-11-24T07:56:22.689685+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:56:22.678019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:56:22.678019+0000","last_clean_scrub_stamp":"2023-11-24T07:56:22.678019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T07:56:23.683366+0000","last_change":"2023-11-24T07:56:23.683366+0000","last_active":"2023-11-24T07:56:23.683366+0000","last_peered":"2023-11-24T07:56:23.683366+0000","last_clean":"2023-11-24T07:56:23.683366+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T07:56:23.683366+0000","last_undegraded":"2023-11-24T07:56:23.683366+0000","last_fullsized":"2023-11-24T07:56:23.683366+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:56:23.683366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:56:23.683366+0000","last_clean_scrub_stamp":"2023-11-24T07:56:23.683366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":1,"reported_epoch":17,"state":"creating+peering","last_fresh":"2023-11-24T07:56:22.695289+0000","last_change":"2023-11-24T07:56:22.695289+0000","last_active":"2023-11-24T07:56:22.678019+0000","last_peered":"2023-11-24T07:56:22.678019+0000","last_clean":"2023-11-24T07:56:22.678019+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T07:56:22.695289+0000","last_undegraded":"2023-11-24T07:56:22.695289+0000","last_fullsized":"2023-11-24T07:56:22.695289+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:56:22.678019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:56:22.678019+0000","last_clean_scrub_stamp":"2023-11-24T07:56:22.678019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T07:56:23.683366+0000","last_change":"2023-11-24T07:56:23.683366+0000","last_active":"2023-11-24T07:56:23.683366+0000","last_peered":"2023-11-24T07:56:23.683366+0000","last_clean":"2023-11-24T07:56:23.683366+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T07:56:23.683366+0000","last_undegraded":"2023-11-24T07:56:23.683366+0000","last_fullsized":"2023-11-24T07:56:23.683366+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:56:23.683366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:56:23.683366+0000","last_clean_scrub_stamp":"2023-11-24T07:56:23.683366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":1,"reported_epoch":17,"state":"creating+peering","last_fresh":"2023-11-24T07:56:22.686064+0000","last_change":"2023-11-24T07:56:22.686064+0000","last_active":"2023-11-24T07:56:22.678019+0000","last_peered":"2023-11-24T07:56:22.678019+0000","last_clean":"2023-11-24T07:56:22.678019+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T07:56:22.686064+0000","last_undegraded":"2023-11-24T07:56:22.686064+0000","last_fullsized":"2023-11-24T07:56:22.686064+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:56:22.678019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:56:22.678019+0000","last_clean_scrub_stamp":"2023-11-24T07:56:22.678019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T07:56:23.683366+0000","last_change":"2023-11-24T07:56:23.683366+0000","last_active":"2023-11-24T07:56:23.683366+0000","last_peered":"2023-11-24T07:56:23.683366+0000","last_clean":"2023-11-24T07:56:23.683366+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T07:56:23.683366+0000","last_undegraded":"2023-11-24T07:56:23.683366+0000","last_fullsized":"2023-11-24T07:56:23.683366+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:56:23.683366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:56:23.683366+0000","last_clean_scrub_stamp":"2023-11-24T07:56:23.683366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":1,"reported_epoch":17,"state":"creating+peering","last_fresh":"2023-11-24T07:56:22.690679+0000","last_change":"2023-11-24T07:56:22.690679+0000","last_active":"2023-11-24T07:56:22.678019+0000","last_peered":"2023-11-24T07:56:22.678019+0000","last_clean":"2023-11-24T07:56:22.678019+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T07:56:22.690679+0000","last_undegraded":"2023-11-24T07:56:22.690679+0000","last_fullsized":"2023-11-24T07:56:22.690679+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:56:22.678019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:56:22.678019+0000","last_clean_scrub_stamp":"2023-11-24T07:56:22.678019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T07:56:23.683366+0000","last_change":"2023-11-24T07:56:23.683366+0000","last_active":"2023-11-24T07:56:23.683366+0000","last_peered":"2023-11-24T07:56:23.683366+0000","last_clean":"2023-11-24T07:56:23.683366+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T07:56:23.683366+0000","last_undegraded":"2023-11-24T07:56:23.683366+0000","last_fullsized":"2023-11-24T07:56:23.683366+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:56:23.683366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:56:23.683366+0000","last_clean_scrub_stamp":"2023-11-24T07:56:23.683366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.15","version":"0'0","reported_seq":1,"reported_epoch":17,"state":"creating+peering","last_fresh":"2023-11-24T07:56:22.688558+0000","last_change":"2023-11-24T07:56:22.688558+0000","last_active":"2023-11-24T07:56:22.678019+0000","last_peered":"2023-11-24T07:56:22.678019+0000","last_clean":"2023-11-24T07:56:22.678019+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T07:56:22.688558+0000","last_undegraded":"2023-11-24T07:56:22.688558+0000","last_fullsized":"2023-11-24T07:56:22.688558+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:56:22.678019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:56:22.678019+0000","last_clean_scrub_stamp":"2023-11-24T07:56:22.678019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T07:56:23.683366+0000","last_change":"2023-11-24T07:56:23.683366+0000","last_active":"2023-11-24T07:56:23.683366+0000","last_peered":"2023-11-24T07:56:23.683366+0000","last_clean":"2023-11-24T07:56:23.683366+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T07:56:23.683366+0000","last_undegraded":"2023-11-24T07:56:23.683366+0000","last_fullsized":"2023-11-24T07:56:23.683366+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:56:23.683366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:56:23.683366+0000","last_clean_scrub_stamp":"2023-11-24T07:56:23.683366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.14","version":"0'0","reported_seq":1,"reported_epoch":17,"state":"creating+peering","last_fresh":"2023-11-24T07:56:22.693669+0000","last_change":"2023-11-24T07:56:22.693669+0000","last_active":"2023-11-24T07:56:22.678019+0000","last_peered":"2023-11-24T07:56:22.678019+0000","last_clean":"2023-11-24T07:56:22.678019+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T07:56:22.693669+0000","last_undegraded":"2023-11-24T07:56:22.693669+0000","last_fullsized":"2023-11-24T07:56:22.693669+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:56:22.678019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:56:22.678019+0000","last_clean_scrub_stamp":"2023-11-24T07:56:22.678019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T07:56:23.683366+0000","last_change":"2023-11-24T07:56:23.683366+0000","last_active":"2023-11-24T07:56:23.683366+0000","last_peered":"2023-11-24T07:56:23.683366+0000","last_clean":"2023-11-24T07:56:23.683366+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T07:56:23.683366+0000","last_undegraded":"2023-11-24T07:56:23.683366+0000","last_fullsized":"2023-11-24T07:56:23.683366+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:56:23.683366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:56:23.683366+0000","last_clean_scrub_stamp":"2023-11-24T07:56:23.683366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.13","version":"0'0","reported_seq":1,"reported_epoch":17,"state":"creating+peering","last_fresh":"2023-11-24T07:56:22.688470+0000","last_change":"2023-11-24T07:56:22.688470+0000","last_active":"2023-11-24T07:56:22.678019+0000","last_peered":"2023-11-24T07:56:22.678019+0000","last_clean":"2023-11-24T07:56:22.678019+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T07:56:22.688470+0000","last_undegraded":"2023-11-24T07:56:22.688470+0000","last_fullsized":"2023-11-24T07:56:22.688470+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:56:22.678019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:56:22.678019+0000","last_clean_scrub_stamp":"2023-11-24T07:56:22.678019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T07:56:23.683366+0000","last_change":"2023-11-24T07:56:23.683366+0000","last_active":"2023-11-24T07:56:23.683366+0000","last_peered":"2023-11-24T07:56:23.683366+0000","last_clean":"2023-11-24T07:56:23.683366+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T07:56:23.683366+0000","last_undegraded":"2023-11-24T07:56:23.683366+0000","last_fullsized":"2023-11-24T07:56:23.683366+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:56:23.683366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:56:23.683366+0000","last_clean_scrub_stamp":"2023-11-24T07:56:23.683366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":1,"reported_epoch":17,"state":"creating+peering","last_fresh":"2023-11-24T07:56:22.694366+0000","last_change":"2023-11-24T07:56:22.694366+0000","last_active":"2023-11-24T07:56:22.678019+0000","last_peered":"2023-11-24T07:56:22.678019+0000","last_clean":"2023-11-24T07:56:22.678019+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T07:56:22.694366+0000","last_undegraded":"2023-11-24T07:56:22.694366+0000","last_fullsized":"2023-11-24T07:56:22.694366+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:56:22.678019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:56:22.678019+0000","last_clean_scrub_stamp":"2023-11-24T07:56:22.678019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T07:56:23.683366+0000","last_change":"2023-11-24T07:56:23.683366+0000","last_active":"2023-11-24T07:56:23.683366+0000","last_peered":"2023-11-24T07:56:23.683366+0000","last_clean":"2023-11-24T07:56:23.683366+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T07:56:23.683366+0000","last_undegraded":"2023-11-24T07:56:23.683366+0000","last_fullsized":"2023-11-24T07:56:23.683366+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:56:23.683366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:56:23.683366+0000","last_clean_scrub_stamp":"2023-11-24T07:56:23.683366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":1,"reported_epoch":17,"state":"creating+peering","last_fresh":"2023-11-24T07:56:22.690305+0000","last_change":"2023-11-24T07:56:22.690305+0000","last_active":"2023-11-24T07:56:22.678019+0000","last_peered":"2023-11-24T07:56:22.678019+0000","last_clean":"2023-11-24T07:56:22.678019+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T07:56:22.690305+0000","last_undegraded":"2023-11-24T07:56:22.690305+0000","last_fullsized":"2023-11-24T07:56:22.690305+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:56:22.678019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:56:22.678019+0000","last_clean_scrub_stamp":"2023-11-24T07:56:22.678019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T07:56:23.683366+0000","last_change":"2023-11-24T07:56:23.683366+0000","last_active":"2023-11-24T07:56:23.683366+0000","last_peered":"2023-11-24T07:56:23.683366+0000","last_clean":"2023-11-24T07:56:23.683366+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T07:56:23.683366+0000","last_undegraded":"2023-11-24T07:56:23.683366+0000","last_fullsized":"2023-11-24T07:56:23.683366+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:56:23.683366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:56:23.683366+0000","last_clean_scrub_stamp":"2023-11-24T07:56:23.683366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.10","version":"0'0","reported_seq":1,"reported_epoch":17,"state":"creating+peering","last_fresh":"2023-11-24T07:56:22.690393+0000","last_change":"2023-11-24T07:56:22.690393+0000","last_active":"2023-11-24T07:56:22.678019+0000","last_peered":"2023-11-24T07:56:22.678019+0000","last_clean":"2023-11-24T07:56:22.678019+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T07:56:22.690393+0000","last_undegraded":"2023-11-24T07:56:22.690393+0000","last_fullsized":"2023-11-24T07:56:22.690393+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:56:22.678019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:56:22.678019+0000","last_clean_scrub_stamp":"2023-11-24T07:56:22.678019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T07:56:23.683366+0000","last_change":"2023-11-24T07:56:23.683366+0000","last_active":"2023-11-24T07:56:23.683366+0000","last_peered":"2023-11-24T07:56:23.683366+0000","last_clean":"2023-11-24T07:56:23.683366+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T07:56:23.683366+0000","last_undegraded":"2023-11-24T07:56:23.683366+0000","last_fullsized":"2023-11-24T07:56:23.683366+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:56:23.683366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:56:23.683366+0000","last_clean_scrub_stamp":"2023-11-24T07:56:23.683366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.f","version":"0'0","reported_seq":1,"reported_epoch":17,"state":"creating+peering","last_fresh":"2023-11-24T07:56:22.690726+0000","last_change":"2023-11-24T07:56:22.690726+0000","last_active":"2023-11-24T07:56:22.678019+0000","last_peered":"2023-11-24T07:56:22.678019+0000","last_clean":"2023-11-24T07:56:22.678019+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T07:56:22.690726+0000","last_undegraded":"2023-11-24T07:56:22.690726+0000","last_fullsized":"2023-11-24T07:56:22.690726+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:56:22.678019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:56:22.678019+0000","last_clean_scrub_stamp":"2023-11-24T07:56:22.678019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T07:56:23.683366+0000","last_change":"2023-11-24T07:56:23.683366+0000","last_active":"2023-11-24T07:56:23.683366+0000","last_peered":"2023-11-24T07:56:23.683366+0000","last_clean":"2023-11-24T07:56:23.683366+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T07:56:23.683366+0000","last_undegraded":"2023-11-24T07:56:23.683366+0000","last_fullsized":"2023-11-24T07:56:23.683366+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:56:23.683366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:56:23.683366+0000","last_clean_scrub_stamp":"2023-11-24T07:56:23.683366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":1,"reported_epoch":17,"state":"creating+peering","last_fresh":"2023-11-24T07:56:22.686164+0000","last_change":"2023-11-24T07:56:22.686164+0000","last_active":"2023-11-24T07:56:22.678019+0000","last_peered":"2023-11-24T07:56:22.678019+0000","last_clean":"2023-11-24T07:56:22.678019+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T07:56:22.686164+0000","last_undegraded":"2023-11-24T07:56:22.686164+0000","last_fullsized":"2023-11-24T07:56:22.686164+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:56:22.678019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:56:22.678019+0000","last_clean_scrub_stamp":"2023-11-24T07:56:22.678019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T07:56:23.683366+0000","last_change":"2023-11-24T07:56:23.683366+0000","last_active":"2023-11-24T07:56:23.683366+0000","last_peered":"2023-11-24T07:56:23.683366+0000","last_clean":"2023-11-24T07:56:23.683366+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T07:56:23.683366+0000","last_undegraded":"2023-11-24T07:56:23.683366+0000","last_fullsized":"2023-11-24T07:56:23.683366+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:56:23.683366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:56:23.683366+0000","last_clean_scrub_stamp":"2023-11-24T07:56:23.683366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":1,"reported_epoch":17,"state":"creating+peering","last_fresh":"2023-11-24T07:56:22.695293+0000","last_change":"2023-11-24T07:56:22.695293+0000","last_active":"2023-11-24T07:56:22.678019+0000","last_peered":"2023-11-24T07:56:22.678019+0000","last_clean":"2023-11-24T07:56:22.678019+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T07:56:22.695293+0000","last_undegraded":"2023-11-24T07:56:22.695293+0000","last_fullsized":"2023-11-24T07:56:22.695293+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:56:22.678019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:56:22.678019+0000","last_clean_scrub_stamp":"2023-11-24T07:56:22.678019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T07:56:23.683366+0000","last_change":"2023-11-24T07:56:23.683366+0000","last_active":"2023-11-24T07:56:23.683366+0000","last_peered":"2023-11-24T07:56:23.683366+0000","last_clean":"2023-11-24T07:56:23.683366+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T07:56:23.683366+0000","last_undegraded":"2023-11-24T07:56:23.683366+0000","last_fullsized":"2023-11-24T07:56:23.683366+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:56:23.683366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:56:23.683366+0000","last_clean_scrub_stamp":"2023-11-24T07:56:23.683366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T07:56:23.683366+0000","last_change":"2023-11-24T07:56:23.683366+0000","last_active":"2023-11-24T07:56:23.683366+0000","last_peered":"2023-11-24T07:56:23.683366+0000","last_clean":"2023-11-24T07:56:23.683366+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T07:56:23.683366+0000","last_undegraded":"2023-11-24T07:56:23.683366+0000","last_fullsized":"2023-11-24T07:56:23.683366+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:56:23.683366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:56:23.683366+0000","last_clean_scrub_stamp":"2023-11-24T07:56:23.683366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":1,"reported_epoch":17,"state":"creating+peering","last_fresh":"2023-11-24T07:56:22.688504+0000","last_change":"2023-11-24T07:56:22.688504+0000","last_active":"2023-11-24T07:56:22.678019+0000","last_peered":"2023-11-24T07:56:22.678019+0000","last_clean":"2023-11-24T07:56:22.678019+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T07:56:22.688504+0000","last_undegraded":"2023-11-24T07:56:22.688504+0000","last_fullsized":"2023-11-24T07:56:22.688504+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:56:22.678019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:56:22.678019+0000","last_clean_scrub_stamp":"2023-11-24T07:56:22.678019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T07:56:23.683366+0000","last_change":"2023-11-24T07:56:23.683366+0000","last_active":"2023-11-24T07:56:23.683366+0000","last_peered":"2023-11-24T07:56:23.683366+0000","last_clean":"2023-11-24T07:56:23.683366+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T07:56:23.683366+0000","last_undegraded":"2023-11-24T07:56:23.683366+0000","last_fullsized":"2023-11-24T07:56:23.683366+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:56:23.683366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:56:23.683366+0000","last_clean_scrub_stamp":"2023-11-24T07:56:23.683366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":1,"reported_epoch":17,"state":"creating+peering","last_fresh":"2023-11-24T07:56:22.694975+0000","last_change":"2023-11-24T07:56:22.694975+0000","last_active":"2023-11-24T07:56:22.678019+0000","last_peered":"2023-11-24T07:56:22.678019+0000","last_clean":"2023-11-24T07:56:22.678019+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T07:56:22.694975+0000","last_undegraded":"2023-11-24T07:56:22.694975+0000","last_fullsized":"2023-11-24T07:56:22.694975+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:56:22.678019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:56:22.678019+0000","last_clean_scrub_stamp":"2023-11-24T07:56:22.678019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":162,"reported_epoch":17,"state":"active+clean","last_fresh":"2023-11-24T07:56:22.685865+0000","last_change":"2023-11-24T07:56:19.668905+0000","last_active":"2023-11-24T07:56:22.685865+0000","last_peered":"2023-11-24T07:56:22.685865+0000","last_clean":"2023-11-24T07:56:22.685865+0000","last_became_active":"2023-11-24T07:56:19.668601+0000","last_became_peered":"2023-11-24T07:56:19.668601+0000","last_unstale":"2023-11-24T07:56:22.685865+0000","last_undegraded":"2023-11-24T07:56:22.685865+0000","last_fullsized":"2023-11-24T07:56:22.685865+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:56:18.657844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:56:18.657844+0000","last_clean_scrub_stamp":"2023-11-24T07:56:18.657844+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T11:51:53.828455+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T07:56:23.683366+0000","last_change":"2023-11-24T07:56:23.683366+0000","last_active":"2023-11-24T07:56:23.683366+0000","last_peered":"2023-11-24T07:56:23.683366+0000","last_clean":"2023-11-24T07:56:23.683366+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T07:56:23.683366+0000","last_undegraded":"2023-11-24T07:56:23.683366+0000","last_fullsized":"2023-11-24T07:56:23.683366+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:56:23.683366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:56:23.683366+0000","last_clean_scrub_stamp":"2023-11-24T07:56:23.683366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":1,"reported_epoch":17,"state":"creating+peering","last_fresh":"2023-11-24T07:56:22.694439+0000","last_change":"2023-11-24T07:56:22.694439+0000","last_active":"2023-11-24T07:56:22.678019+0000","last_peered":"2023-11-24T07:56:22.678019+0000","last_clean":"2023-11-24T07:56:22.678019+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T07:56:22.694439+0000","last_undegraded":"2023-11-24T07:56:22.694439+0000","last_fullsized":"2023-11-24T07:56:22.694439+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:56:22.678019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:56:22.678019+0000","last_clean_scrub_stamp":"2023-11-24T07:56:22.678019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":1,"reported_epoch":17,"state":"creating+peering","last_fresh":"2023-11-24T07:56:22.694430+0000","last_change":"2023-11-24T07:56:22.694430+0000","last_active":"2023-11-24T07:56:22.678019+0000","last_peered":"2023-11-24T07:56:22.678019+0000","last_clean":"2023-11-24T07:56:22.678019+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T07:56:22.694430+0000","last_undegraded":"2023-11-24T07:56:22.694430+0000","last_fullsized":"2023-11-24T07:56:22.694430+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:56:22.678019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:56:22.678019+0000","last_clean_scrub_stamp":"2023-11-24T07:56:22.678019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T07:56:23.683366+0000","last_change":"2023-11-24T07:56:23.683366+0000","last_active":"2023-11-24T07:56:23.683366+0000","last_peered":"2023-11-24T07:56:23.683366+0000","last_clean":"2023-11-24T07:56:23.683366+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T07:56:23.683366+0000","last_undegraded":"2023-11-24T07:56:23.683366+0000","last_fullsized":"2023-11-24T07:56:23.683366+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:56:23.683366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:56:23.683366+0000","last_clean_scrub_stamp":"2023-11-24T07:56:23.683366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":1,"reported_epoch":17,"state":"creating+peering","last_fresh":"2023-11-24T07:56:22.690376+0000","last_change":"2023-11-24T07:56:22.690376+0000","last_active":"2023-11-24T07:56:22.678019+0000","last_peered":"2023-11-24T07:56:22.678019+0000","last_clean":"2023-11-24T07:56:22.678019+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T07:56:22.690376+0000","last_undegraded":"2023-11-24T07:56:22.690376+0000","last_fullsized":"2023-11-24T07:56:22.690376+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:56:22.678019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:56:22.678019+0000","last_clean_scrub_stamp":"2023-11-24T07:56:22.678019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T07:56:23.683366+0000","last_change":"2023-11-24T07:56:23.683366+0000","last_active":"2023-11-24T07:56:23.683366+0000","last_peered":"2023-11-24T07:56:23.683366+0000","last_clean":"2023-11-24T07:56:23.683366+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T07:56:23.683366+0000","last_undegraded":"2023-11-24T07:56:23.683366+0000","last_fullsized":"2023-11-24T07:56:23.683366+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:56:23.683366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:56:23.683366+0000","last_clean_scrub_stamp":"2023-11-24T07:56:23.683366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":1,"reported_epoch":17,"state":"creating+peering","last_fresh":"2023-11-24T07:56:22.695123+0000","last_change":"2023-11-24T07:56:22.695123+0000","last_active":"2023-11-24T07:56:22.678019+0000","last_peered":"2023-11-24T07:56:22.678019+0000","last_clean":"2023-11-24T07:56:22.678019+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T07:56:22.695123+0000","last_undegraded":"2023-11-24T07:56:22.695123+0000","last_fullsized":"2023-11-24T07:56:22.695123+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:56:22.678019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:56:22.678019+0000","last_clean_scrub_stamp":"2023-11-24T07:56:22.678019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T07:56:23.683366+0000","last_change":"2023-11-24T07:56:23.683366+0000","last_active":"2023-11-24T07:56:23.683366+0000","last_peered":"2023-11-24T07:56:23.683366+0000","last_clean":"2023-11-24T07:56:23.683366+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T07:56:23.683366+0000","last_undegraded":"2023-11-24T07:56:23.683366+0000","last_fullsized":"2023-11-24T07:56:23.683366+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:56:23.683366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:56:23.683366+0000","last_clean_scrub_stamp":"2023-11-24T07:56:23.683366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":1,"reported_epoch":17,"state":"creating+peering","last_fresh":"2023-11-24T07:56:22.690282+0000","last_change":"2023-11-24T07:56:22.690282+0000","last_active":"2023-11-24T07:56:22.678019+0000","last_peered":"2023-11-24T07:56:22.678019+0000","last_clean":"2023-11-24T07:56:22.678019+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T07:56:22.690282+0000","last_undegraded":"2023-11-24T07:56:22.690282+0000","last_fullsized":"2023-11-24T07:56:22.690282+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:56:22.678019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:56:22.678019+0000","last_clean_scrub_stamp":"2023-11-24T07:56:22.678019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T07:56:23.683366+0000","last_change":"2023-11-24T07:56:23.683366+0000","last_active":"2023-11-24T07:56:23.683366+0000","last_peered":"2023-11-24T07:56:23.683366+0000","last_clean":"2023-11-24T07:56:23.683366+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T07:56:23.683366+0000","last_undegraded":"2023-11-24T07:56:23.683366+0000","last_fullsized":"2023-11-24T07:56:23.683366+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:56:23.683366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:56:23.683366+0000","last_clean_scrub_stamp":"2023-11-24T07:56:23.683366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":1,"reported_epoch":17,"state":"creating+peering","last_fresh":"2023-11-24T07:56:22.690549+0000","last_change":"2023-11-24T07:56:22.690549+0000","last_active":"2023-11-24T07:56:22.678019+0000","last_peered":"2023-11-24T07:56:22.678019+0000","last_clean":"2023-11-24T07:56:22.678019+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T07:56:22.690549+0000","last_undegraded":"2023-11-24T07:56:22.690549+0000","last_fullsized":"2023-11-24T07:56:22.690549+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:56:22.678019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:56:22.678019+0000","last_clean_scrub_stamp":"2023-11-24T07:56:22.678019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T07:56:23.683366+0000","last_change":"2023-11-24T07:56:23.683366+0000","last_active":"2023-11-24T07:56:23.683366+0000","last_peered":"2023-11-24T07:56:23.683366+0000","last_clean":"2023-11-24T07:56:23.683366+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T07:56:23.683366+0000","last_undegraded":"2023-11-24T07:56:23.683366+0000","last_fullsized":"2023-11-24T07:56:23.683366+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:56:23.683366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:56:23.683366+0000","last_clean_scrub_stamp":"2023-11-24T07:56:23.683366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":1,"reported_epoch":17,"state":"creating+peering","last_fresh":"2023-11-24T07:56:22.689706+0000","last_change":"2023-11-24T07:56:22.689706+0000","last_active":"2023-11-24T07:56:22.678019+0000","last_peered":"2023-11-24T07:56:22.678019+0000","last_clean":"2023-11-24T07:56:22.678019+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T07:56:22.689706+0000","last_undegraded":"2023-11-24T07:56:22.689706+0000","last_fullsized":"2023-11-24T07:56:22.689706+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:56:22.678019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:56:22.678019+0000","last_clean_scrub_stamp":"2023-11-24T07:56:22.678019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T07:56:23.683366+0000","last_change":"2023-11-24T07:56:23.683366+0000","last_active":"2023-11-24T07:56:23.683366+0000","last_peered":"2023-11-24T07:56:23.683366+0000","last_clean":"2023-11-24T07:56:23.683366+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T07:56:23.683366+0000","last_undegraded":"2023-11-24T07:56:23.683366+0000","last_fullsized":"2023-11-24T07:56:23.683366+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:56:23.683366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:56:23.683366+0000","last_clean_scrub_stamp":"2023-11-24T07:56:23.683366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.b","version":"0'0","reported_seq":1,"reported_epoch":17,"state":"creating+peering","last_fresh":"2023-11-24T07:56:22.695809+0000","last_change":"2023-11-24T07:56:22.695809+0000","last_active":"2023-11-24T07:56:22.678019+0000","last_peered":"2023-11-24T07:56:22.678019+0000","last_clean":"2023-11-24T07:56:22.678019+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T07:56:22.695809+0000","last_undegraded":"2023-11-24T07:56:22.695809+0000","last_fullsized":"2023-11-24T07:56:22.695809+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:56:22.678019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:56:22.678019+0000","last_clean_scrub_stamp":"2023-11-24T07:56:22.678019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T07:56:23.683366+0000","last_change":"2023-11-24T07:56:23.683366+0000","last_active":"2023-11-24T07:56:23.683366+0000","last_peered":"2023-11-24T07:56:23.683366+0000","last_clean":"2023-11-24T07:56:23.683366+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T07:56:23.683366+0000","last_undegraded":"2023-11-24T07:56:23.683366+0000","last_fullsized":"2023-11-24T07:56:23.683366+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:56:23.683366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:56:23.683366+0000","last_clean_scrub_stamp":"2023-11-24T07:56:23.683366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":1,"reported_epoch":17,"state":"creating+peering","last_fresh":"2023-11-24T07:56:22.690094+0000","last_change":"2023-11-24T07:56:22.690094+0000","last_active":"2023-11-24T07:56:22.678019+0000","last_peered":"2023-11-24T07:56:22.678019+0000","last_clean":"2023-11-24T07:56:22.678019+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T07:56:22.690094+0000","last_undegraded":"2023-11-24T07:56:22.690094+0000","last_fullsized":"2023-11-24T07:56:22.690094+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:56:22.678019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:56:22.678019+0000","last_clean_scrub_stamp":"2023-11-24T07:56:22.678019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T07:56:23.683366+0000","last_change":"2023-11-24T07:56:23.683366+0000","last_active":"2023-11-24T07:56:23.683366+0000","last_peered":"2023-11-24T07:56:23.683366+0000","last_clean":"2023-11-24T07:56:23.683366+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T07:56:23.683366+0000","last_undegraded":"2023-11-24T07:56:23.683366+0000","last_fullsized":"2023-11-24T07:56:23.683366+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:56:23.683366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:56:23.683366+0000","last_clean_scrub_stamp":"2023-11-24T07:56:23.683366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.d","version":"0'0","reported_seq":1,"reported_epoch":17,"state":"creating+peering","last_fresh":"2023-11-24T07:56:22.688223+0000","last_change":"2023-11-24T07:56:22.688223+0000","last_active":"2023-11-24T07:56:22.678019+0000","last_peered":"2023-11-24T07:56:22.678019+0000","last_clean":"2023-11-24T07:56:22.678019+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T07:56:22.688223+0000","last_undegraded":"2023-11-24T07:56:22.688223+0000","last_fullsized":"2023-11-24T07:56:22.688223+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:56:22.678019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:56:22.678019+0000","last_clean_scrub_stamp":"2023-11-24T07:56:22.678019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T07:56:23.683366+0000","last_change":"2023-11-24T07:56:23.683366+0000","last_active":"2023-11-24T07:56:23.683366+0000","last_peered":"2023-11-24T07:56:23.683366+0000","last_clean":"2023-11-24T07:56:23.683366+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T07:56:23.683366+0000","last_undegraded":"2023-11-24T07:56:23.683366+0000","last_fullsized":"2023-11-24T07:56:23.683366+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:56:23.683366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:56:23.683366+0000","last_clean_scrub_stamp":"2023-11-24T07:56:23.683366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.e","version":"0'0","reported_seq":1,"reported_epoch":17,"state":"creating+peering","last_fresh":"2023-11-24T07:56:22.691566+0000","last_change":"2023-11-24T07:56:22.691566+0000","last_active":"2023-11-24T07:56:22.678019+0000","last_peered":"2023-11-24T07:56:22.678019+0000","last_clean":"2023-11-24T07:56:22.678019+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T07:56:22.691566+0000","last_undegraded":"2023-11-24T07:56:22.691566+0000","last_fullsized":"2023-11-24T07:56:22.691566+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:56:22.678019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:56:22.678019+0000","last_clean_scrub_stamp":"2023-11-24T07:56:22.678019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T07:56:23.683366+0000","last_change":"2023-11-24T07:56:23.683366+0000","last_active":"2023-11-24T07:56:23.683366+0000","last_peered":"2023-11-24T07:56:23.683366+0000","last_clean":"2023-11-24T07:56:23.683366+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T07:56:23.683366+0000","last_undegraded":"2023-11-24T07:56:23.683366+0000","last_fullsized":"2023-11-24T07:56:23.683366+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:56:23.683366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:56:23.683366+0000","last_clean_scrub_stamp":"2023-11-24T07:56:23.683366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T07:56:23.683366+0000","last_change":"2023-11-24T07:56:23.683366+0000","last_active":"2023-11-24T07:56:23.683366+0000","last_peered":"2023-11-24T07:56:23.683366+0000","last_clean":"2023-11-24T07:56:23.683366+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T07:56:23.683366+0000","last_undegraded":"2023-11-24T07:56:23.683366+0000","last_fullsized":"2023-11-24T07:56:23.683366+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:56:23.683366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:56:23.683366+0000","last_clean_scrub_stamp":"2023-11-24T07:56:23.683366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T07:56:23.683366+0000","last_change":"2023-11-24T07:56:23.683366+0000","last_active":"2023-11-24T07:56:23.683366+0000","last_peered":"2023-11-24T07:56:23.683366+0000","last_clean":"2023-11-24T07:56:23.683366+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T07:56:23.683366+0000","last_undegraded":"2023-11-24T07:56:23.683366+0000","last_fullsized":"2023-11-24T07:56:23.683366+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:56:23.683366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:56:23.683366+0000","last_clean_scrub_stamp":"2023-11-24T07:56:23.683366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T07:56:23.683366+0000","last_change":"2023-11-24T07:56:23.683366+0000","last_active":"2023-11-24T07:56:23.683366+0000","last_peered":"2023-11-24T07:56:23.683366+0000","last_clean":"2023-11-24T07:56:23.683366+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T07:56:23.683366+0000","last_undegraded":"2023-11-24T07:56:23.683366+0000","last_fullsized":"2023-11-24T07:56:23.683366+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:56:23.683366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:56:23.683366+0000","last_clean_scrub_stamp":"2023-11-24T07:56:23.683366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T07:56:23.683366+0000","last_change":"2023-11-24T07:56:23.683366+0000","last_active":"2023-11-24T07:56:23.683366+0000","last_peered":"2023-11-24T07:56:23.683366+0000","last_clean":"2023-11-24T07:56:23.683366+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T07:56:23.683366+0000","last_undegraded":"2023-11-24T07:56:23.683366+0000","last_fullsized":"2023-11-24T07:56:23.683366+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:56:23.683366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:56:23.683366+0000","last_clean_scrub_stamp":"2023-11-24T07:56:23.683366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T07:56:23.683366+0000","last_change":"2023-11-24T07:56:23.683366+0000","last_active":"2023-11-24T07:56:23.683366+0000","last_peered":"2023-11-24T07:56:23.683366+0000","last_clean":"2023-11-24T07:56:23.683366+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T07:56:23.683366+0000","last_undegraded":"2023-11-24T07:56:23.683366+0000","last_fullsized":"2023-11-24T07:56:23.683366+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:56:23.683366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:56:23.683366+0000","last_clean_scrub_stamp":"2023-11-24T07:56:23.683366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T07:56:23.683366+0000","last_change":"2023-11-24T07:56:23.683366+0000","last_active":"2023-11-24T07:56:23.683366+0000","last_peered":"2023-11-24T07:56:23.683366+0000","last_clean":"2023-11-24T07:56:23.683366+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T07:56:23.683366+0000","last_undegraded":"2023-11-24T07:56:23.683366+0000","last_fullsized":"2023-11-24T07:56:23.683366+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:56:23.683366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:56:23.683366+0000","last_clean_scrub_stamp":"2023-11-24T07:56:23.683366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T07:56:23.683366+0000","last_change":"2023-11-24T07:56:23.683366+0000","last_active":"2023-11-24T07:56:23.683366+0000","last_peered":"2023-11-24T07:56:23.683366+0000","last_clean":"2023-11-24T07:56:23.683366+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T07:56:23.683366+0000","last_undegraded":"2023-11-24T07:56:23.683366+0000","last_fullsized":"2023-11-24T07:56:23.683366+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:56:23.683366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:56:23.683366+0000","last_clean_scrub_stamp":"2023-11-24T07:56:23.683366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T07:56:23.683366+0000","last_change":"2023-11-24T07:56:23.683366+0000","last_active":"2023-11-24T07:56:23.683366+0000","last_peered":"2023-11-24T07:56:23.683366+0000","last_clean":"2023-11-24T07:56:23.683366+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T07:56:23.683366+0000","last_undegraded":"2023-11-24T07:56:23.683366+0000","last_fullsized":"2023-11-24T07:56:23.683366+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:56:23.683366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:56:23.683366+0000","last_clean_scrub_stamp":"2023-11-24T07:56:23.683366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T07:56:23.683366+0000","last_change":"2023-11-24T07:56:23.683366+0000","last_active":"2023-11-24T07:56:23.683366+0000","last_peered":"2023-11-24T07:56:23.683366+0000","last_clean":"2023-11-24T07:56:23.683366+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T07:56:23.683366+0000","last_undegraded":"2023-11-24T07:56:23.683366+0000","last_fullsized":"2023-11-24T07:56:23.683366+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:56:23.683366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:56:23.683366+0000","last_clean_scrub_stamp":"2023-11-24T07:56:23.683366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T07:56:23.683366+0000","last_change":"2023-11-24T07:56:23.683366+0000","last_active":"2023-11-24T07:56:23.683366+0000","last_peered":"2023-11-24T07:56:23.683366+0000","last_clean":"2023-11-24T07:56:23.683366+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T07:56:23.683366+0000","last_undegraded":"2023-11-24T07:56:23.683366+0000","last_fullsized":"2023-11-24T07:56:23.683366+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:56:23.683366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:56:23.683366+0000","last_clean_scrub_stamp":"2023-11-24T07:56:23.683366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T07:56:23.683366+0000","last_change":"2023-11-24T07:56:23.683366+0000","last_active":"2023-11-24T07:56:23.683366+0000","last_peered":"2023-11-24T07:56:23.683366+0000","last_clean":"2023-11-24T07:56:23.683366+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T07:56:23.683366+0000","last_undegraded":"2023-11-24T07:56:23.683366+0000","last_fullsized":"2023-11-24T07:56:23.683366+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:56:23.683366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:56:23.683366+0000","last_clean_scrub_stamp":"2023-11-24T07:56:23.683366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T07:56:23.683366+0000","last_change":"2023-11-24T07:56:23.683366+0000","last_active":"2023-11-24T07:56:23.683366+0000","last_peered":"2023-11-24T07:56:23.683366+0000","last_clean":"2023-11-24T07:56:23.683366+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T07:56:23.683366+0000","last_undegraded":"2023-11-24T07:56:23.683366+0000","last_fullsized":"2023-11-24T07:56:23.683366+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:56:23.683366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:56:23.683366+0000","last_clean_scrub_stamp":"2023-11-24T07:56:23.683366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T07:56:23.683366+0000","last_change":"2023-11-24T07:56:23.683366+0000","last_active":"2023-11-24T07:56:23.683366+0000","last_peered":"2023-11-24T07:56:23.683366+0000","last_clean":"2023-11-24T07:56:23.683366+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T07:56:23.683366+0000","last_undegraded":"2023-11-24T07:56:23.683366+0000","last_fullsized":"2023-11-24T07:56:23.683366+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:56:23.683366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:56:23.683366+0000","last_clean_scrub_stamp":"2023-11-24T07:56:23.683366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574851,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27172,"kb_used_data":152,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344668,"statfs":{"total":96636764160,"available":96608940032,"internally_reserved":0,"allocated":155648,"data_stored":45743,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.40100000000000002}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61599999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64400000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66300000000000003}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.42599999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.45900000000000002}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55000000000000004}]}]},{"osd":6,"up_from":13,"seq":55834574852,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27172,"kb_used_data":152,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344668,"statfs":{"total":96636764160,"available":96608940032,"internally_reserved":0,"allocated":155648,"data_stored":45743,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65300000000000002}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54100000000000004}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.76600000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.876}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59999999999999998}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.84999999999999998}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58099999999999996}]}]},{"osd":1,"up_from":13,"seq":55834574853,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27764,"kb_used_data":744,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344076,"statfs":{"total":96636764160,"available":96608333824,"internally_reserved":0,"allocated":761856,"data_stored":641928,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.33100000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.308}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.432}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.69699999999999995}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68000000000000005}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.72099999999999997}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.81200000000000006}]}]},{"osd":0,"up_from":13,"seq":55834574853,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27172,"kb_used_data":152,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344668,"statfs":{"total":96636764160,"available":96608940032,"internally_reserved":0,"allocated":155648,"data_stored":45743,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.28799999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.32800000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.30499999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.40500000000000003}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54200000000000004}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48499999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52600000000000002}]}]},{"osd":2,"up_from":13,"seq":55834574853,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27184,"kb_used_data":164,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344656,"statfs":{"total":96636764160,"available":96608927744,"internally_reserved":0,"allocated":167936,"data_stored":51560,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48699999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.44500000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53300000000000003}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.81599999999999995}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.84899999999999998}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.83599999999999997}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.80000000000000004}]}]},{"osd":3,"up_from":13,"seq":55834574853,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27184,"kb_used_data":164,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344656,"statfs":{"total":96636764160,"available":96608927744,"internally_reserved":0,"allocated":167936,"data_stored":51560,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.313}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.39600000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.35899999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.73099999999999998}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57199999999999995}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60899999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.71099999999999997}]}]},{"osd":4,"up_from":13,"seq":55834574852,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27184,"kb_used_data":164,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344656,"statfs":{"total":96636764160,"available":96608927744,"internally_reserved":0,"allocated":167936,"data_stored":51560,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.78400000000000003}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.42699999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.745}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65100000000000002}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47099999999999997}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55100000000000005}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58099999999999996}]}]},{"osd":5,"up_from":13,"seq":55834574853,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27764,"kb_used_data":744,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344076,"statfs":{"total":96636764160,"available":96608333824,"internally_reserved":0,"allocated":761856,"data_stored":641928,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0109999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.98499999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.86599999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.046}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51600000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54800000000000004}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.96599999999999997}]}]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-11-24T07:56:29.664 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.3f is not active+clean 2023-11-24T07:56:29.664 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.3f', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-24T07:56:23.683366+0000', 'last_change': '2023-11-24T07:56:23.683366+0000', 'last_active': '2023-11-24T07:56:23.683366+0000', 'last_peered': '2023-11-24T07:56:23.683366+0000', 'last_clean': '2023-11-24T07:56:23.683366+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-24T07:56:23.683366+0000', 'last_undegraded': '2023-11-24T07:56:23.683366+0000', 'last_fullsized': '2023-11-24T07:56:23.683366+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-24T07:56:23.683366+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-24T07:56:23.683366+0000', 'last_clean_scrub_stamp': '2023-11-24T07:56:23.683366+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-24T07:56:29.664 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.3e is not active+clean 2023-11-24T07:56:29.665 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.3e', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-24T07:56:23.683366+0000', 'last_change': '2023-11-24T07:56:23.683366+0000', 'last_active': '2023-11-24T07:56:23.683366+0000', 'last_peered': '2023-11-24T07:56:23.683366+0000', 'last_clean': '2023-11-24T07:56:23.683366+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-24T07:56:23.683366+0000', 'last_undegraded': '2023-11-24T07:56:23.683366+0000', 'last_fullsized': '2023-11-24T07:56:23.683366+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-24T07:56:23.683366+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-24T07:56:23.683366+0000', 'last_clean_scrub_stamp': '2023-11-24T07:56:23.683366+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-24T07:56:29.665 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.3d is not active+clean 2023-11-24T07:56:29.665 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.3d', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-24T07:56:23.683366+0000', 'last_change': '2023-11-24T07:56:23.683366+0000', 'last_active': '2023-11-24T07:56:23.683366+0000', 'last_peered': '2023-11-24T07:56:23.683366+0000', 'last_clean': '2023-11-24T07:56:23.683366+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-24T07:56:23.683366+0000', 'last_undegraded': '2023-11-24T07:56:23.683366+0000', 'last_fullsized': '2023-11-24T07:56:23.683366+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-24T07:56:23.683366+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-24T07:56:23.683366+0000', 'last_clean_scrub_stamp': '2023-11-24T07:56:23.683366+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-24T07:56:29.665 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.3c is not active+clean 2023-11-24T07:56:29.665 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.3c', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-24T07:56:23.683366+0000', 'last_change': '2023-11-24T07:56:23.683366+0000', 'last_active': '2023-11-24T07:56:23.683366+0000', 'last_peered': '2023-11-24T07:56:23.683366+0000', 'last_clean': '2023-11-24T07:56:23.683366+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-24T07:56:23.683366+0000', 'last_undegraded': '2023-11-24T07:56:23.683366+0000', 'last_fullsized': '2023-11-24T07:56:23.683366+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-24T07:56:23.683366+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-24T07:56:23.683366+0000', 'last_clean_scrub_stamp': '2023-11-24T07:56:23.683366+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-24T07:56:29.665 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.3b is not active+clean 2023-11-24T07:56:29.665 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.3b', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-24T07:56:23.683366+0000', 'last_change': '2023-11-24T07:56:23.683366+0000', 'last_active': '2023-11-24T07:56:23.683366+0000', 'last_peered': '2023-11-24T07:56:23.683366+0000', 'last_clean': '2023-11-24T07:56:23.683366+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-24T07:56:23.683366+0000', 'last_undegraded': '2023-11-24T07:56:23.683366+0000', 'last_fullsized': '2023-11-24T07:56:23.683366+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-24T07:56:23.683366+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-24T07:56:23.683366+0000', 'last_clean_scrub_stamp': '2023-11-24T07:56:23.683366+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-24T07:56:29.665 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.3a is not active+clean 2023-11-24T07:56:29.665 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.3a', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-24T07:56:23.683366+0000', 'last_change': '2023-11-24T07:56:23.683366+0000', 'last_active': '2023-11-24T07:56:23.683366+0000', 'last_peered': '2023-11-24T07:56:23.683366+0000', 'last_clean': '2023-11-24T07:56:23.683366+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-24T07:56:23.683366+0000', 'last_undegraded': '2023-11-24T07:56:23.683366+0000', 'last_fullsized': '2023-11-24T07:56:23.683366+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-24T07:56:23.683366+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-24T07:56:23.683366+0000', 'last_clean_scrub_stamp': '2023-11-24T07:56:23.683366+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-24T07:56:29.665 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.39 is not active+clean 2023-11-24T07:56:29.666 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.39', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-24T07:56:23.683366+0000', 'last_change': '2023-11-24T07:56:23.683366+0000', 'last_active': '2023-11-24T07:56:23.683366+0000', 'last_peered': '2023-11-24T07:56:23.683366+0000', 'last_clean': '2023-11-24T07:56:23.683366+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-24T07:56:23.683366+0000', 'last_undegraded': '2023-11-24T07:56:23.683366+0000', 'last_fullsized': '2023-11-24T07:56:23.683366+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-24T07:56:23.683366+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-24T07:56:23.683366+0000', 'last_clean_scrub_stamp': '2023-11-24T07:56:23.683366+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-24T07:56:29.666 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.38 is not active+clean 2023-11-24T07:56:29.666 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.38', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-24T07:56:23.683366+0000', 'last_change': '2023-11-24T07:56:23.683366+0000', 'last_active': '2023-11-24T07:56:23.683366+0000', 'last_peered': '2023-11-24T07:56:23.683366+0000', 'last_clean': '2023-11-24T07:56:23.683366+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-24T07:56:23.683366+0000', 'last_undegraded': '2023-11-24T07:56:23.683366+0000', 'last_fullsized': '2023-11-24T07:56:23.683366+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-24T07:56:23.683366+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-24T07:56:23.683366+0000', 'last_clean_scrub_stamp': '2023-11-24T07:56:23.683366+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-24T07:56:29.666 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.37 is not active+clean 2023-11-24T07:56:29.666 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.37', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-24T07:56:23.683366+0000', 'last_change': '2023-11-24T07:56:23.683366+0000', 'last_active': '2023-11-24T07:56:23.683366+0000', 'last_peered': '2023-11-24T07:56:23.683366+0000', 'last_clean': '2023-11-24T07:56:23.683366+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-24T07:56:23.683366+0000', 'last_undegraded': '2023-11-24T07:56:23.683366+0000', 'last_fullsized': '2023-11-24T07:56:23.683366+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-24T07:56:23.683366+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-24T07:56:23.683366+0000', 'last_clean_scrub_stamp': '2023-11-24T07:56:23.683366+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-24T07:56:29.666 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.36 is not active+clean 2023-11-24T07:56:29.666 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.36', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-24T07:56:23.683366+0000', 'last_change': '2023-11-24T07:56:23.683366+0000', 'last_active': '2023-11-24T07:56:23.683366+0000', 'last_peered': '2023-11-24T07:56:23.683366+0000', 'last_clean': '2023-11-24T07:56:23.683366+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-24T07:56:23.683366+0000', 'last_undegraded': '2023-11-24T07:56:23.683366+0000', 'last_fullsized': '2023-11-24T07:56:23.683366+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-24T07:56:23.683366+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-24T07:56:23.683366+0000', 'last_clean_scrub_stamp': '2023-11-24T07:56:23.683366+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-24T07:56:29.666 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.35 is not active+clean 2023-11-24T07:56:29.666 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.35', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-24T07:56:23.683366+0000', 'last_change': '2023-11-24T07:56:23.683366+0000', 'last_active': '2023-11-24T07:56:23.683366+0000', 'last_peered': '2023-11-24T07:56:23.683366+0000', 'last_clean': '2023-11-24T07:56:23.683366+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-24T07:56:23.683366+0000', 'last_undegraded': '2023-11-24T07:56:23.683366+0000', 'last_fullsized': '2023-11-24T07:56:23.683366+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-24T07:56:23.683366+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-24T07:56:23.683366+0000', 'last_clean_scrub_stamp': '2023-11-24T07:56:23.683366+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-24T07:56:29.666 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.34 is not active+clean 2023-11-24T07:56:29.667 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.34', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-24T07:56:23.683366+0000', 'last_change': '2023-11-24T07:56:23.683366+0000', 'last_active': '2023-11-24T07:56:23.683366+0000', 'last_peered': '2023-11-24T07:56:23.683366+0000', 'last_clean': '2023-11-24T07:56:23.683366+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-24T07:56:23.683366+0000', 'last_undegraded': '2023-11-24T07:56:23.683366+0000', 'last_fullsized': '2023-11-24T07:56:23.683366+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-24T07:56:23.683366+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-24T07:56:23.683366+0000', 'last_clean_scrub_stamp': '2023-11-24T07:56:23.683366+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-24T07:56:29.667 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.33 is not active+clean 2023-11-24T07:56:29.667 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.33', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-24T07:56:23.683366+0000', 'last_change': '2023-11-24T07:56:23.683366+0000', 'last_active': '2023-11-24T07:56:23.683366+0000', 'last_peered': '2023-11-24T07:56:23.683366+0000', 'last_clean': '2023-11-24T07:56:23.683366+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-24T07:56:23.683366+0000', 'last_undegraded': '2023-11-24T07:56:23.683366+0000', 'last_fullsized': '2023-11-24T07:56:23.683366+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-24T07:56:23.683366+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-24T07:56:23.683366+0000', 'last_clean_scrub_stamp': '2023-11-24T07:56:23.683366+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-24T07:56:29.667 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.32 is not active+clean 2023-11-24T07:56:29.667 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.32', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-24T07:56:23.683366+0000', 'last_change': '2023-11-24T07:56:23.683366+0000', 'last_active': '2023-11-24T07:56:23.683366+0000', 'last_peered': '2023-11-24T07:56:23.683366+0000', 'last_clean': '2023-11-24T07:56:23.683366+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-24T07:56:23.683366+0000', 'last_undegraded': '2023-11-24T07:56:23.683366+0000', 'last_fullsized': '2023-11-24T07:56:23.683366+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-24T07:56:23.683366+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-24T07:56:23.683366+0000', 'last_clean_scrub_stamp': '2023-11-24T07:56:23.683366+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-24T07:56:29.667 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.31 is not active+clean 2023-11-24T07:56:29.667 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.31', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-24T07:56:23.683366+0000', 'last_change': '2023-11-24T07:56:23.683366+0000', 'last_active': '2023-11-24T07:56:23.683366+0000', 'last_peered': '2023-11-24T07:56:23.683366+0000', 'last_clean': '2023-11-24T07:56:23.683366+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-24T07:56:23.683366+0000', 'last_undegraded': '2023-11-24T07:56:23.683366+0000', 'last_fullsized': '2023-11-24T07:56:23.683366+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-24T07:56:23.683366+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-24T07:56:23.683366+0000', 'last_clean_scrub_stamp': '2023-11-24T07:56:23.683366+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-24T07:56:29.667 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.30 is not active+clean 2023-11-24T07:56:29.667 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.30', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-24T07:56:23.683366+0000', 'last_change': '2023-11-24T07:56:23.683366+0000', 'last_active': '2023-11-24T07:56:23.683366+0000', 'last_peered': '2023-11-24T07:56:23.683366+0000', 'last_clean': '2023-11-24T07:56:23.683366+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-24T07:56:23.683366+0000', 'last_undegraded': '2023-11-24T07:56:23.683366+0000', 'last_fullsized': '2023-11-24T07:56:23.683366+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-24T07:56:23.683366+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-24T07:56:23.683366+0000', 'last_clean_scrub_stamp': '2023-11-24T07:56:23.683366+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-24T07:56:29.667 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.2f is not active+clean 2023-11-24T07:56:29.667 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.2f', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-24T07:56:23.683366+0000', 'last_change': '2023-11-24T07:56:23.683366+0000', 'last_active': '2023-11-24T07:56:23.683366+0000', 'last_peered': '2023-11-24T07:56:23.683366+0000', 'last_clean': '2023-11-24T07:56:23.683366+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-24T07:56:23.683366+0000', 'last_undegraded': '2023-11-24T07:56:23.683366+0000', 'last_fullsized': '2023-11-24T07:56:23.683366+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-24T07:56:23.683366+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-24T07:56:23.683366+0000', 'last_clean_scrub_stamp': '2023-11-24T07:56:23.683366+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-24T07:56:29.668 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.2e is not active+clean 2023-11-24T07:56:29.668 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.2e', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-24T07:56:23.683366+0000', 'last_change': '2023-11-24T07:56:23.683366+0000', 'last_active': '2023-11-24T07:56:23.683366+0000', 'last_peered': '2023-11-24T07:56:23.683366+0000', 'last_clean': '2023-11-24T07:56:23.683366+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-24T07:56:23.683366+0000', 'last_undegraded': '2023-11-24T07:56:23.683366+0000', 'last_fullsized': '2023-11-24T07:56:23.683366+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-24T07:56:23.683366+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-24T07:56:23.683366+0000', 'last_clean_scrub_stamp': '2023-11-24T07:56:23.683366+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-24T07:56:29.668 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.2d is not active+clean 2023-11-24T07:56:29.668 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.2d', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-24T07:56:23.683366+0000', 'last_change': '2023-11-24T07:56:23.683366+0000', 'last_active': '2023-11-24T07:56:23.683366+0000', 'last_peered': '2023-11-24T07:56:23.683366+0000', 'last_clean': '2023-11-24T07:56:23.683366+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-24T07:56:23.683366+0000', 'last_undegraded': '2023-11-24T07:56:23.683366+0000', 'last_fullsized': '2023-11-24T07:56:23.683366+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-24T07:56:23.683366+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-24T07:56:23.683366+0000', 'last_clean_scrub_stamp': '2023-11-24T07:56:23.683366+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-24T07:56:29.668 INFO:tasks.cephfs.filesystem.ceph_manager:PG 2.1f is not active+clean 2023-11-24T07:56:29.668 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '2.1f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 17, 'state': 'creating+peering', 'last_fresh': '2023-11-24T07:56:22.690428+0000', 'last_change': '2023-11-24T07:56:22.690428+0000', 'last_active': '2023-11-24T07:56:22.678019+0000', 'last_peered': '2023-11-24T07:56:22.678019+0000', 'last_clean': '2023-11-24T07:56:22.678019+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-24T07:56:22.690428+0000', 'last_undegraded': '2023-11-24T07:56:22.690428+0000', 'last_fullsized': '2023-11-24T07:56:22.690428+0000', 'mapping_epoch': 17, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-24T07:56:22.678019+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-24T07:56:22.678019+0000', 'last_clean_scrub_stamp': '2023-11-24T07:56:22.678019+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 3], 'acting': [4, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2023-11-24T07:56:29.668 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.1e is not active+clean 2023-11-24T07:56:29.668 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.1e', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-24T07:56:23.683366+0000', 'last_change': '2023-11-24T07:56:23.683366+0000', 'last_active': '2023-11-24T07:56:23.683366+0000', 'last_peered': '2023-11-24T07:56:23.683366+0000', 'last_clean': '2023-11-24T07:56:23.683366+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-24T07:56:23.683366+0000', 'last_undegraded': '2023-11-24T07:56:23.683366+0000', 'last_fullsized': '2023-11-24T07:56:23.683366+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-24T07:56:23.683366+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-24T07:56:23.683366+0000', 'last_clean_scrub_stamp': '2023-11-24T07:56:23.683366+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-24T07:56:29.668 INFO:tasks.cephfs.filesystem.ceph_manager:PG 2.1e is not active+clean 2023-11-24T07:56:29.668 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '2.1e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 17, 'state': 'creating+peering', 'last_fresh': '2023-11-24T07:56:22.686164+0000', 'last_change': '2023-11-24T07:56:22.686164+0000', 'last_active': '2023-11-24T07:56:22.678019+0000', 'last_peered': '2023-11-24T07:56:22.678019+0000', 'last_clean': '2023-11-24T07:56:22.678019+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-24T07:56:22.686164+0000', 'last_undegraded': '2023-11-24T07:56:22.686164+0000', 'last_fullsized': '2023-11-24T07:56:22.686164+0000', 'mapping_epoch': 17, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-24T07:56:22.678019+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-24T07:56:22.678019+0000', 'last_clean_scrub_stamp': '2023-11-24T07:56:22.678019+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 7], 'acting': [5, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2023-11-24T07:56:29.669 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.1f is not active+clean 2023-11-24T07:56:29.669 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.1f', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-24T07:56:23.683366+0000', 'last_change': '2023-11-24T07:56:23.683366+0000', 'last_active': '2023-11-24T07:56:23.683366+0000', 'last_peered': '2023-11-24T07:56:23.683366+0000', 'last_clean': '2023-11-24T07:56:23.683366+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-24T07:56:23.683366+0000', 'last_undegraded': '2023-11-24T07:56:23.683366+0000', 'last_fullsized': '2023-11-24T07:56:23.683366+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-24T07:56:23.683366+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-24T07:56:23.683366+0000', 'last_clean_scrub_stamp': '2023-11-24T07:56:23.683366+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-24T07:56:29.669 INFO:tasks.cephfs.filesystem.ceph_manager:PG 2.1d is not active+clean 2023-11-24T07:56:29.669 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '2.1d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 17, 'state': 'creating+peering', 'last_fresh': '2023-11-24T07:56:22.690762+0000', 'last_change': '2023-11-24T07:56:22.690762+0000', 'last_active': '2023-11-24T07:56:22.678019+0000', 'last_peered': '2023-11-24T07:56:22.678019+0000', 'last_clean': '2023-11-24T07:56:22.678019+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-24T07:56:22.690762+0000', 'last_undegraded': '2023-11-24T07:56:22.690762+0000', 'last_fullsized': '2023-11-24T07:56:22.690762+0000', 'mapping_epoch': 17, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-24T07:56:22.678019+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-24T07:56:22.678019+0000', 'last_clean_scrub_stamp': '2023-11-24T07:56:22.678019+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2023-11-24T07:56:29.669 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.1c is not active+clean 2023-11-24T07:56:29.669 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.1c', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-24T07:56:23.683366+0000', 'last_change': '2023-11-24T07:56:23.683366+0000', 'last_active': '2023-11-24T07:56:23.683366+0000', 'last_peered': '2023-11-24T07:56:23.683366+0000', 'last_clean': '2023-11-24T07:56:23.683366+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-24T07:56:23.683366+0000', 'last_undegraded': '2023-11-24T07:56:23.683366+0000', 'last_fullsized': '2023-11-24T07:56:23.683366+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-24T07:56:23.683366+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-24T07:56:23.683366+0000', 'last_clean_scrub_stamp': '2023-11-24T07:56:23.683366+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-24T07:56:29.669 INFO:tasks.cephfs.filesystem.ceph_manager:PG 2.1c is not active+clean 2023-11-24T07:56:29.669 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '2.1c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 17, 'state': 'creating+peering', 'last_fresh': '2023-11-24T07:56:22.690382+0000', 'last_change': '2023-11-24T07:56:22.690382+0000', 'last_active': '2023-11-24T07:56:22.678019+0000', 'last_peered': '2023-11-24T07:56:22.678019+0000', 'last_clean': '2023-11-24T07:56:22.678019+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-24T07:56:22.690382+0000', 'last_undegraded': '2023-11-24T07:56:22.690382+0000', 'last_fullsized': '2023-11-24T07:56:22.690382+0000', 'mapping_epoch': 17, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-24T07:56:22.678019+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-24T07:56:22.678019+0000', 'last_clean_scrub_stamp': '2023-11-24T07:56:22.678019+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2023-11-24T07:56:29.669 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.1d is not active+clean 2023-11-24T07:56:29.669 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.1d', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-24T07:56:23.683366+0000', 'last_change': '2023-11-24T07:56:23.683366+0000', 'last_active': '2023-11-24T07:56:23.683366+0000', 'last_peered': '2023-11-24T07:56:23.683366+0000', 'last_clean': '2023-11-24T07:56:23.683366+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-24T07:56:23.683366+0000', 'last_undegraded': '2023-11-24T07:56:23.683366+0000', 'last_fullsized': '2023-11-24T07:56:23.683366+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-24T07:56:23.683366+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-24T07:56:23.683366+0000', 'last_clean_scrub_stamp': '2023-11-24T07:56:23.683366+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-24T07:56:29.670 INFO:tasks.cephfs.filesystem.ceph_manager:PG 2.1b is not active+clean 2023-11-24T07:56:29.670 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '2.1b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 17, 'state': 'creating+peering', 'last_fresh': '2023-11-24T07:56:22.686185+0000', 'last_change': '2023-11-24T07:56:22.686185+0000', 'last_active': '2023-11-24T07:56:22.678019+0000', 'last_peered': '2023-11-24T07:56:22.678019+0000', 'last_clean': '2023-11-24T07:56:22.678019+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-24T07:56:22.686185+0000', 'last_undegraded': '2023-11-24T07:56:22.686185+0000', 'last_fullsized': '2023-11-24T07:56:22.686185+0000', 'mapping_epoch': 17, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-24T07:56:22.678019+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-24T07:56:22.678019+0000', 'last_clean_scrub_stamp': '2023-11-24T07:56:22.678019+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 6], 'acting': [5, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2023-11-24T07:56:29.670 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.1a is not active+clean 2023-11-24T07:56:29.670 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.1a', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-24T07:56:23.683366+0000', 'last_change': '2023-11-24T07:56:23.683366+0000', 'last_active': '2023-11-24T07:56:23.683366+0000', 'last_peered': '2023-11-24T07:56:23.683366+0000', 'last_clean': '2023-11-24T07:56:23.683366+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-24T07:56:23.683366+0000', 'last_undegraded': '2023-11-24T07:56:23.683366+0000', 'last_fullsized': '2023-11-24T07:56:23.683366+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-24T07:56:23.683366+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-24T07:56:23.683366+0000', 'last_clean_scrub_stamp': '2023-11-24T07:56:23.683366+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-24T07:56:29.670 INFO:tasks.cephfs.filesystem.ceph_manager:PG 2.1a is not active+clean 2023-11-24T07:56:29.670 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '2.1a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 17, 'state': 'creating+peering', 'last_fresh': '2023-11-24T07:56:22.696543+0000', 'last_change': '2023-11-24T07:56:22.696543+0000', 'last_active': '2023-11-24T07:56:22.678019+0000', 'last_peered': '2023-11-24T07:56:22.678019+0000', 'last_clean': '2023-11-24T07:56:22.678019+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-24T07:56:22.696543+0000', 'last_undegraded': '2023-11-24T07:56:22.696543+0000', 'last_fullsized': '2023-11-24T07:56:22.696543+0000', 'mapping_epoch': 17, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-24T07:56:22.678019+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-24T07:56:22.678019+0000', 'last_clean_scrub_stamp': '2023-11-24T07:56:22.678019+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 0], 'acting': [3, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2023-11-24T07:56:29.670 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.1b is not active+clean 2023-11-24T07:56:29.670 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.1b', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-24T07:56:23.683366+0000', 'last_change': '2023-11-24T07:56:23.683366+0000', 'last_active': '2023-11-24T07:56:23.683366+0000', 'last_peered': '2023-11-24T07:56:23.683366+0000', 'last_clean': '2023-11-24T07:56:23.683366+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-24T07:56:23.683366+0000', 'last_undegraded': '2023-11-24T07:56:23.683366+0000', 'last_fullsized': '2023-11-24T07:56:23.683366+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-24T07:56:23.683366+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-24T07:56:23.683366+0000', 'last_clean_scrub_stamp': '2023-11-24T07:56:23.683366+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-24T07:56:29.670 INFO:tasks.cephfs.filesystem.ceph_manager:PG 2.19 is not active+clean 2023-11-24T07:56:29.670 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '2.19', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 17, 'state': 'creating+peering', 'last_fresh': '2023-11-24T07:56:22.689685+0000', 'last_change': '2023-11-24T07:56:22.689685+0000', 'last_active': '2023-11-24T07:56:22.678019+0000', 'last_peered': '2023-11-24T07:56:22.678019+0000', 'last_clean': '2023-11-24T07:56:22.678019+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-24T07:56:22.689685+0000', 'last_undegraded': '2023-11-24T07:56:22.689685+0000', 'last_fullsized': '2023-11-24T07:56:22.689685+0000', 'mapping_epoch': 17, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-24T07:56:22.678019+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-24T07:56:22.678019+0000', 'last_clean_scrub_stamp': '2023-11-24T07:56:22.678019+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 0], 'acting': [6, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2023-11-24T07:56:29.671 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.18 is not active+clean 2023-11-24T07:56:29.671 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.18', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-24T07:56:23.683366+0000', 'last_change': '2023-11-24T07:56:23.683366+0000', 'last_active': '2023-11-24T07:56:23.683366+0000', 'last_peered': '2023-11-24T07:56:23.683366+0000', 'last_clean': '2023-11-24T07:56:23.683366+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-24T07:56:23.683366+0000', 'last_undegraded': '2023-11-24T07:56:23.683366+0000', 'last_fullsized': '2023-11-24T07:56:23.683366+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-24T07:56:23.683366+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-24T07:56:23.683366+0000', 'last_clean_scrub_stamp': '2023-11-24T07:56:23.683366+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-24T07:56:29.671 INFO:tasks.cephfs.filesystem.ceph_manager:PG 2.18 is not active+clean 2023-11-24T07:56:29.671 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '2.18', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 17, 'state': 'creating+peering', 'last_fresh': '2023-11-24T07:56:22.695289+0000', 'last_change': '2023-11-24T07:56:22.695289+0000', 'last_active': '2023-11-24T07:56:22.678019+0000', 'last_peered': '2023-11-24T07:56:22.678019+0000', 'last_clean': '2023-11-24T07:56:22.678019+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-24T07:56:22.695289+0000', 'last_undegraded': '2023-11-24T07:56:22.695289+0000', 'last_fullsized': '2023-11-24T07:56:22.695289+0000', 'mapping_epoch': 17, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-24T07:56:22.678019+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-24T07:56:22.678019+0000', 'last_clean_scrub_stamp': '2023-11-24T07:56:22.678019+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 1], 'acting': [3, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2023-11-24T07:56:29.671 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.19 is not active+clean 2023-11-24T07:56:29.671 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.19', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-24T07:56:23.683366+0000', 'last_change': '2023-11-24T07:56:23.683366+0000', 'last_active': '2023-11-24T07:56:23.683366+0000', 'last_peered': '2023-11-24T07:56:23.683366+0000', 'last_clean': '2023-11-24T07:56:23.683366+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-24T07:56:23.683366+0000', 'last_undegraded': '2023-11-24T07:56:23.683366+0000', 'last_fullsized': '2023-11-24T07:56:23.683366+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-24T07:56:23.683366+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-24T07:56:23.683366+0000', 'last_clean_scrub_stamp': '2023-11-24T07:56:23.683366+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-24T07:56:29.671 INFO:tasks.cephfs.filesystem.ceph_manager:PG 2.17 is not active+clean 2023-11-24T07:56:29.671 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '2.17', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 17, 'state': 'creating+peering', 'last_fresh': '2023-11-24T07:56:22.686064+0000', 'last_change': '2023-11-24T07:56:22.686064+0000', 'last_active': '2023-11-24T07:56:22.678019+0000', 'last_peered': '2023-11-24T07:56:22.678019+0000', 'last_clean': '2023-11-24T07:56:22.678019+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-24T07:56:22.686064+0000', 'last_undegraded': '2023-11-24T07:56:22.686064+0000', 'last_fullsized': '2023-11-24T07:56:22.686064+0000', 'mapping_epoch': 17, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-24T07:56:22.678019+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-24T07:56:22.678019+0000', 'last_clean_scrub_stamp': '2023-11-24T07:56:22.678019+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 4], 'acting': [1, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2023-11-24T07:56:29.671 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.16 is not active+clean 2023-11-24T07:56:29.671 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.16', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-24T07:56:23.683366+0000', 'last_change': '2023-11-24T07:56:23.683366+0000', 'last_active': '2023-11-24T07:56:23.683366+0000', 'last_peered': '2023-11-24T07:56:23.683366+0000', 'last_clean': '2023-11-24T07:56:23.683366+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-24T07:56:23.683366+0000', 'last_undegraded': '2023-11-24T07:56:23.683366+0000', 'last_fullsized': '2023-11-24T07:56:23.683366+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-24T07:56:23.683366+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-24T07:56:23.683366+0000', 'last_clean_scrub_stamp': '2023-11-24T07:56:23.683366+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-24T07:56:29.671 INFO:tasks.cephfs.filesystem.ceph_manager:PG 2.16 is not active+clean 2023-11-24T07:56:29.672 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '2.16', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 17, 'state': 'creating+peering', 'last_fresh': '2023-11-24T07:56:22.690679+0000', 'last_change': '2023-11-24T07:56:22.690679+0000', 'last_active': '2023-11-24T07:56:22.678019+0000', 'last_peered': '2023-11-24T07:56:22.678019+0000', 'last_clean': '2023-11-24T07:56:22.678019+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-24T07:56:22.690679+0000', 'last_undegraded': '2023-11-24T07:56:22.690679+0000', 'last_fullsized': '2023-11-24T07:56:22.690679+0000', 'mapping_epoch': 17, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-24T07:56:22.678019+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-24T07:56:22.678019+0000', 'last_clean_scrub_stamp': '2023-11-24T07:56:22.678019+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2023-11-24T07:56:29.672 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.17 is not active+clean 2023-11-24T07:56:29.672 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.17', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-24T07:56:23.683366+0000', 'last_change': '2023-11-24T07:56:23.683366+0000', 'last_active': '2023-11-24T07:56:23.683366+0000', 'last_peered': '2023-11-24T07:56:23.683366+0000', 'last_clean': '2023-11-24T07:56:23.683366+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-24T07:56:23.683366+0000', 'last_undegraded': '2023-11-24T07:56:23.683366+0000', 'last_fullsized': '2023-11-24T07:56:23.683366+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-24T07:56:23.683366+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-24T07:56:23.683366+0000', 'last_clean_scrub_stamp': '2023-11-24T07:56:23.683366+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-24T07:56:29.672 INFO:tasks.cephfs.filesystem.ceph_manager:PG 2.15 is not active+clean 2023-11-24T07:56:29.672 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '2.15', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 17, 'state': 'creating+peering', 'last_fresh': '2023-11-24T07:56:22.688558+0000', 'last_change': '2023-11-24T07:56:22.688558+0000', 'last_active': '2023-11-24T07:56:22.678019+0000', 'last_peered': '2023-11-24T07:56:22.678019+0000', 'last_clean': '2023-11-24T07:56:22.678019+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-24T07:56:22.688558+0000', 'last_undegraded': '2023-11-24T07:56:22.688558+0000', 'last_fullsized': '2023-11-24T07:56:22.688558+0000', 'mapping_epoch': 17, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-24T07:56:22.678019+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-24T07:56:22.678019+0000', 'last_clean_scrub_stamp': '2023-11-24T07:56:22.678019+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 4], 'acting': [7, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2023-11-24T07:56:29.672 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.14 is not active+clean 2023-11-24T07:56:29.672 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.14', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-24T07:56:23.683366+0000', 'last_change': '2023-11-24T07:56:23.683366+0000', 'last_active': '2023-11-24T07:56:23.683366+0000', 'last_peered': '2023-11-24T07:56:23.683366+0000', 'last_clean': '2023-11-24T07:56:23.683366+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-24T07:56:23.683366+0000', 'last_undegraded': '2023-11-24T07:56:23.683366+0000', 'last_fullsized': '2023-11-24T07:56:23.683366+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-24T07:56:23.683366+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-24T07:56:23.683366+0000', 'last_clean_scrub_stamp': '2023-11-24T07:56:23.683366+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-24T07:56:29.672 INFO:tasks.cephfs.filesystem.ceph_manager:PG 2.14 is not active+clean 2023-11-24T07:56:29.672 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '2.14', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 17, 'state': 'creating+peering', 'last_fresh': '2023-11-24T07:56:22.693669+0000', 'last_change': '2023-11-24T07:56:22.693669+0000', 'last_active': '2023-11-24T07:56:22.678019+0000', 'last_peered': '2023-11-24T07:56:22.678019+0000', 'last_clean': '2023-11-24T07:56:22.678019+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-24T07:56:22.693669+0000', 'last_undegraded': '2023-11-24T07:56:22.693669+0000', 'last_fullsized': '2023-11-24T07:56:22.693669+0000', 'mapping_epoch': 17, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-24T07:56:22.678019+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-24T07:56:22.678019+0000', 'last_clean_scrub_stamp': '2023-11-24T07:56:22.678019+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 4], 'acting': [2, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2023-11-24T07:56:29.672 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.15 is not active+clean 2023-11-24T07:56:29.673 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.15', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-24T07:56:23.683366+0000', 'last_change': '2023-11-24T07:56:23.683366+0000', 'last_active': '2023-11-24T07:56:23.683366+0000', 'last_peered': '2023-11-24T07:56:23.683366+0000', 'last_clean': '2023-11-24T07:56:23.683366+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-24T07:56:23.683366+0000', 'last_undegraded': '2023-11-24T07:56:23.683366+0000', 'last_fullsized': '2023-11-24T07:56:23.683366+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-24T07:56:23.683366+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-24T07:56:23.683366+0000', 'last_clean_scrub_stamp': '2023-11-24T07:56:23.683366+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-24T07:56:29.673 INFO:tasks.cephfs.filesystem.ceph_manager:PG 2.13 is not active+clean 2023-11-24T07:56:29.673 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '2.13', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 17, 'state': 'creating+peering', 'last_fresh': '2023-11-24T07:56:22.688470+0000', 'last_change': '2023-11-24T07:56:22.688470+0000', 'last_active': '2023-11-24T07:56:22.678019+0000', 'last_peered': '2023-11-24T07:56:22.678019+0000', 'last_clean': '2023-11-24T07:56:22.678019+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-24T07:56:22.688470+0000', 'last_undegraded': '2023-11-24T07:56:22.688470+0000', 'last_fullsized': '2023-11-24T07:56:22.688470+0000', 'mapping_epoch': 17, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-24T07:56:22.678019+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-24T07:56:22.678019+0000', 'last_clean_scrub_stamp': '2023-11-24T07:56:22.678019+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 0], 'acting': [7, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2023-11-24T07:56:29.673 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.12 is not active+clean 2023-11-24T07:56:29.673 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.12', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-24T07:56:23.683366+0000', 'last_change': '2023-11-24T07:56:23.683366+0000', 'last_active': '2023-11-24T07:56:23.683366+0000', 'last_peered': '2023-11-24T07:56:23.683366+0000', 'last_clean': '2023-11-24T07:56:23.683366+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-24T07:56:23.683366+0000', 'last_undegraded': '2023-11-24T07:56:23.683366+0000', 'last_fullsized': '2023-11-24T07:56:23.683366+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-24T07:56:23.683366+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-24T07:56:23.683366+0000', 'last_clean_scrub_stamp': '2023-11-24T07:56:23.683366+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-24T07:56:29.673 INFO:tasks.cephfs.filesystem.ceph_manager:PG 2.12 is not active+clean 2023-11-24T07:56:29.673 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '2.12', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 17, 'state': 'creating+peering', 'last_fresh': '2023-11-24T07:56:22.694366+0000', 'last_change': '2023-11-24T07:56:22.694366+0000', 'last_active': '2023-11-24T07:56:22.678019+0000', 'last_peered': '2023-11-24T07:56:22.678019+0000', 'last_clean': '2023-11-24T07:56:22.678019+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-24T07:56:22.694366+0000', 'last_undegraded': '2023-11-24T07:56:22.694366+0000', 'last_fullsized': '2023-11-24T07:56:22.694366+0000', 'mapping_epoch': 17, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-24T07:56:22.678019+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-24T07:56:22.678019+0000', 'last_clean_scrub_stamp': '2023-11-24T07:56:22.678019+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 7], 'acting': [3, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2023-11-24T07:56:29.673 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.13 is not active+clean 2023-11-24T07:56:29.674 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.13', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-24T07:56:23.683366+0000', 'last_change': '2023-11-24T07:56:23.683366+0000', 'last_active': '2023-11-24T07:56:23.683366+0000', 'last_peered': '2023-11-24T07:56:23.683366+0000', 'last_clean': '2023-11-24T07:56:23.683366+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-24T07:56:23.683366+0000', 'last_undegraded': '2023-11-24T07:56:23.683366+0000', 'last_fullsized': '2023-11-24T07:56:23.683366+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-24T07:56:23.683366+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-24T07:56:23.683366+0000', 'last_clean_scrub_stamp': '2023-11-24T07:56:23.683366+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-24T07:56:29.674 INFO:tasks.cephfs.filesystem.ceph_manager:PG 2.11 is not active+clean 2023-11-24T07:56:29.674 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '2.11', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 17, 'state': 'creating+peering', 'last_fresh': '2023-11-24T07:56:22.690305+0000', 'last_change': '2023-11-24T07:56:22.690305+0000', 'last_active': '2023-11-24T07:56:22.678019+0000', 'last_peered': '2023-11-24T07:56:22.678019+0000', 'last_clean': '2023-11-24T07:56:22.678019+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-24T07:56:22.690305+0000', 'last_undegraded': '2023-11-24T07:56:22.690305+0000', 'last_fullsized': '2023-11-24T07:56:22.690305+0000', 'mapping_epoch': 17, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-24T07:56:22.678019+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-24T07:56:22.678019+0000', 'last_clean_scrub_stamp': '2023-11-24T07:56:22.678019+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 3], 'acting': [0, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2023-11-24T07:56:29.674 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.10 is not active+clean 2023-11-24T07:56:29.674 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.10', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-24T07:56:23.683366+0000', 'last_change': '2023-11-24T07:56:23.683366+0000', 'last_active': '2023-11-24T07:56:23.683366+0000', 'last_peered': '2023-11-24T07:56:23.683366+0000', 'last_clean': '2023-11-24T07:56:23.683366+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-24T07:56:23.683366+0000', 'last_undegraded': '2023-11-24T07:56:23.683366+0000', 'last_fullsized': '2023-11-24T07:56:23.683366+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-24T07:56:23.683366+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-24T07:56:23.683366+0000', 'last_clean_scrub_stamp': '2023-11-24T07:56:23.683366+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-24T07:56:29.674 INFO:tasks.cephfs.filesystem.ceph_manager:PG 2.10 is not active+clean 2023-11-24T07:56:29.674 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '2.10', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 17, 'state': 'creating+peering', 'last_fresh': '2023-11-24T07:56:22.690393+0000', 'last_change': '2023-11-24T07:56:22.690393+0000', 'last_active': '2023-11-24T07:56:22.678019+0000', 'last_peered': '2023-11-24T07:56:22.678019+0000', 'last_clean': '2023-11-24T07:56:22.678019+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-24T07:56:22.690393+0000', 'last_undegraded': '2023-11-24T07:56:22.690393+0000', 'last_fullsized': '2023-11-24T07:56:22.690393+0000', 'mapping_epoch': 17, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-24T07:56:22.678019+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-24T07:56:22.678019+0000', 'last_clean_scrub_stamp': '2023-11-24T07:56:22.678019+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 7], 'acting': [4, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2023-11-24T07:56:29.674 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.11 is not active+clean 2023-11-24T07:56:29.674 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.11', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-24T07:56:23.683366+0000', 'last_change': '2023-11-24T07:56:23.683366+0000', 'last_active': '2023-11-24T07:56:23.683366+0000', 'last_peered': '2023-11-24T07:56:23.683366+0000', 'last_clean': '2023-11-24T07:56:23.683366+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-24T07:56:23.683366+0000', 'last_undegraded': '2023-11-24T07:56:23.683366+0000', 'last_fullsized': '2023-11-24T07:56:23.683366+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-24T07:56:23.683366+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-24T07:56:23.683366+0000', 'last_clean_scrub_stamp': '2023-11-24T07:56:23.683366+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-24T07:56:29.675 INFO:tasks.cephfs.filesystem.ceph_manager:PG 2.f is not active+clean 2023-11-24T07:56:29.675 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '2.f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 17, 'state': 'creating+peering', 'last_fresh': '2023-11-24T07:56:22.690726+0000', 'last_change': '2023-11-24T07:56:22.690726+0000', 'last_active': '2023-11-24T07:56:22.678019+0000', 'last_peered': '2023-11-24T07:56:22.678019+0000', 'last_clean': '2023-11-24T07:56:22.678019+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-24T07:56:22.690726+0000', 'last_undegraded': '2023-11-24T07:56:22.690726+0000', 'last_fullsized': '2023-11-24T07:56:22.690726+0000', 'mapping_epoch': 17, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-24T07:56:22.678019+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-24T07:56:22.678019+0000', 'last_clean_scrub_stamp': '2023-11-24T07:56:22.678019+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 5], 'acting': [0, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2023-11-24T07:56:29.675 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.e is not active+clean 2023-11-24T07:56:29.675 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.e', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-24T07:56:23.683366+0000', 'last_change': '2023-11-24T07:56:23.683366+0000', 'last_active': '2023-11-24T07:56:23.683366+0000', 'last_peered': '2023-11-24T07:56:23.683366+0000', 'last_clean': '2023-11-24T07:56:23.683366+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-24T07:56:23.683366+0000', 'last_undegraded': '2023-11-24T07:56:23.683366+0000', 'last_fullsized': '2023-11-24T07:56:23.683366+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-24T07:56:23.683366+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-24T07:56:23.683366+0000', 'last_clean_scrub_stamp': '2023-11-24T07:56:23.683366+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-24T07:56:29.675 INFO:tasks.cephfs.filesystem.ceph_manager:PG 2.4 is not active+clean 2023-11-24T07:56:29.675 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '2.4', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 17, 'state': 'creating+peering', 'last_fresh': '2023-11-24T07:56:22.686164+0000', 'last_change': '2023-11-24T07:56:22.686164+0000', 'last_active': '2023-11-24T07:56:22.678019+0000', 'last_peered': '2023-11-24T07:56:22.678019+0000', 'last_clean': '2023-11-24T07:56:22.678019+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-24T07:56:22.686164+0000', 'last_undegraded': '2023-11-24T07:56:22.686164+0000', 'last_fullsized': '2023-11-24T07:56:22.686164+0000', 'mapping_epoch': 17, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-24T07:56:22.678019+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-24T07:56:22.678019+0000', 'last_clean_scrub_stamp': '2023-11-24T07:56:22.678019+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 6], 'acting': [5, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2023-11-24T07:56:29.675 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.5 is not active+clean 2023-11-24T07:56:29.675 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.5', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-24T07:56:23.683366+0000', 'last_change': '2023-11-24T07:56:23.683366+0000', 'last_active': '2023-11-24T07:56:23.683366+0000', 'last_peered': '2023-11-24T07:56:23.683366+0000', 'last_clean': '2023-11-24T07:56:23.683366+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-24T07:56:23.683366+0000', 'last_undegraded': '2023-11-24T07:56:23.683366+0000', 'last_fullsized': '2023-11-24T07:56:23.683366+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-24T07:56:23.683366+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-24T07:56:23.683366+0000', 'last_clean_scrub_stamp': '2023-11-24T07:56:23.683366+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-24T07:56:29.675 INFO:tasks.cephfs.filesystem.ceph_manager:PG 2.2 is not active+clean 2023-11-24T07:56:29.675 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '2.2', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 17, 'state': 'creating+peering', 'last_fresh': '2023-11-24T07:56:22.695293+0000', 'last_change': '2023-11-24T07:56:22.695293+0000', 'last_active': '2023-11-24T07:56:22.678019+0000', 'last_peered': '2023-11-24T07:56:22.678019+0000', 'last_clean': '2023-11-24T07:56:22.678019+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-24T07:56:22.695293+0000', 'last_undegraded': '2023-11-24T07:56:22.695293+0000', 'last_fullsized': '2023-11-24T07:56:22.695293+0000', 'mapping_epoch': 17, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-24T07:56:22.678019+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-24T07:56:22.678019+0000', 'last_clean_scrub_stamp': '2023-11-24T07:56:22.678019+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 6], 'acting': [3, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2023-11-24T07:56:29.675 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.3 is not active+clean 2023-11-24T07:56:29.676 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.3', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-24T07:56:23.683366+0000', 'last_change': '2023-11-24T07:56:23.683366+0000', 'last_active': '2023-11-24T07:56:23.683366+0000', 'last_peered': '2023-11-24T07:56:23.683366+0000', 'last_clean': '2023-11-24T07:56:23.683366+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-24T07:56:23.683366+0000', 'last_undegraded': '2023-11-24T07:56:23.683366+0000', 'last_fullsized': '2023-11-24T07:56:23.683366+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-24T07:56:23.683366+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-24T07:56:23.683366+0000', 'last_clean_scrub_stamp': '2023-11-24T07:56:23.683366+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-24T07:56:29.676 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.0 is not active+clean 2023-11-24T07:56:29.676 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.0', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-24T07:56:23.683366+0000', 'last_change': '2023-11-24T07:56:23.683366+0000', 'last_active': '2023-11-24T07:56:23.683366+0000', 'last_peered': '2023-11-24T07:56:23.683366+0000', 'last_clean': '2023-11-24T07:56:23.683366+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-24T07:56:23.683366+0000', 'last_undegraded': '2023-11-24T07:56:23.683366+0000', 'last_fullsized': '2023-11-24T07:56:23.683366+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-24T07:56:23.683366+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-24T07:56:23.683366+0000', 'last_clean_scrub_stamp': '2023-11-24T07:56:23.683366+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-24T07:56:29.676 INFO:tasks.cephfs.filesystem.ceph_manager:PG 2.1 is not active+clean 2023-11-24T07:56:29.676 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '2.1', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 17, 'state': 'creating+peering', 'last_fresh': '2023-11-24T07:56:22.688504+0000', 'last_change': '2023-11-24T07:56:22.688504+0000', 'last_active': '2023-11-24T07:56:22.678019+0000', 'last_peered': '2023-11-24T07:56:22.678019+0000', 'last_clean': '2023-11-24T07:56:22.678019+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-24T07:56:22.688504+0000', 'last_undegraded': '2023-11-24T07:56:22.688504+0000', 'last_fullsized': '2023-11-24T07:56:22.688504+0000', 'mapping_epoch': 17, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-24T07:56:22.678019+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-24T07:56:22.678019+0000', 'last_clean_scrub_stamp': '2023-11-24T07:56:22.678019+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 5], 'acting': [7, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2023-11-24T07:56:29.676 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.2 is not active+clean 2023-11-24T07:56:29.676 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.2', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-24T07:56:23.683366+0000', 'last_change': '2023-11-24T07:56:23.683366+0000', 'last_active': '2023-11-24T07:56:23.683366+0000', 'last_peered': '2023-11-24T07:56:23.683366+0000', 'last_clean': '2023-11-24T07:56:23.683366+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-24T07:56:23.683366+0000', 'last_undegraded': '2023-11-24T07:56:23.683366+0000', 'last_fullsized': '2023-11-24T07:56:23.683366+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-24T07:56:23.683366+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-24T07:56:23.683366+0000', 'last_clean_scrub_stamp': '2023-11-24T07:56:23.683366+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-24T07:56:29.676 INFO:tasks.cephfs.filesystem.ceph_manager:PG 2.3 is not active+clean 2023-11-24T07:56:29.676 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '2.3', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 17, 'state': 'creating+peering', 'last_fresh': '2023-11-24T07:56:22.694975+0000', 'last_change': '2023-11-24T07:56:22.694975+0000', 'last_active': '2023-11-24T07:56:22.678019+0000', 'last_peered': '2023-11-24T07:56:22.678019+0000', 'last_clean': '2023-11-24T07:56:22.678019+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-24T07:56:22.694975+0000', 'last_undegraded': '2023-11-24T07:56:22.694975+0000', 'last_fullsized': '2023-11-24T07:56:22.694975+0000', 'mapping_epoch': 17, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-24T07:56:22.678019+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-24T07:56:22.678019+0000', 'last_clean_scrub_stamp': '2023-11-24T07:56:22.678019+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 2], 'acting': [3, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2023-11-24T07:56:29.676 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.1 is not active+clean 2023-11-24T07:56:29.677 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.1', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-24T07:56:23.683366+0000', 'last_change': '2023-11-24T07:56:23.683366+0000', 'last_active': '2023-11-24T07:56:23.683366+0000', 'last_peered': '2023-11-24T07:56:23.683366+0000', 'last_clean': '2023-11-24T07:56:23.683366+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-24T07:56:23.683366+0000', 'last_undegraded': '2023-11-24T07:56:23.683366+0000', 'last_fullsized': '2023-11-24T07:56:23.683366+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-24T07:56:23.683366+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-24T07:56:23.683366+0000', 'last_clean_scrub_stamp': '2023-11-24T07:56:23.683366+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-24T07:56:29.677 INFO:tasks.cephfs.filesystem.ceph_manager:PG 2.0 is not active+clean 2023-11-24T07:56:29.677 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '2.0', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 17, 'state': 'creating+peering', 'last_fresh': '2023-11-24T07:56:22.694439+0000', 'last_change': '2023-11-24T07:56:22.694439+0000', 'last_active': '2023-11-24T07:56:22.678019+0000', 'last_peered': '2023-11-24T07:56:22.678019+0000', 'last_clean': '2023-11-24T07:56:22.678019+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-24T07:56:22.694439+0000', 'last_undegraded': '2023-11-24T07:56:22.694439+0000', 'last_fullsized': '2023-11-24T07:56:22.694439+0000', 'mapping_epoch': 17, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-24T07:56:22.678019+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-24T07:56:22.678019+0000', 'last_clean_scrub_stamp': '2023-11-24T07:56:22.678019+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 5], 'acting': [3, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2023-11-24T07:56:29.677 INFO:tasks.cephfs.filesystem.ceph_manager:PG 2.5 is not active+clean 2023-11-24T07:56:29.677 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '2.5', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 17, 'state': 'creating+peering', 'last_fresh': '2023-11-24T07:56:22.694430+0000', 'last_change': '2023-11-24T07:56:22.694430+0000', 'last_active': '2023-11-24T07:56:22.678019+0000', 'last_peered': '2023-11-24T07:56:22.678019+0000', 'last_clean': '2023-11-24T07:56:22.678019+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-24T07:56:22.694430+0000', 'last_undegraded': '2023-11-24T07:56:22.694430+0000', 'last_fullsized': '2023-11-24T07:56:22.694430+0000', 'mapping_epoch': 17, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-24T07:56:22.678019+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-24T07:56:22.678019+0000', 'last_clean_scrub_stamp': '2023-11-24T07:56:22.678019+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 4], 'acting': [3, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2023-11-24T07:56:29.677 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.4 is not active+clean 2023-11-24T07:56:29.677 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.4', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-24T07:56:23.683366+0000', 'last_change': '2023-11-24T07:56:23.683366+0000', 'last_active': '2023-11-24T07:56:23.683366+0000', 'last_peered': '2023-11-24T07:56:23.683366+0000', 'last_clean': '2023-11-24T07:56:23.683366+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-24T07:56:23.683366+0000', 'last_undegraded': '2023-11-24T07:56:23.683366+0000', 'last_fullsized': '2023-11-24T07:56:23.683366+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-24T07:56:23.683366+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-24T07:56:23.683366+0000', 'last_clean_scrub_stamp': '2023-11-24T07:56:23.683366+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-24T07:56:29.677 INFO:tasks.cephfs.filesystem.ceph_manager:PG 2.6 is not active+clean 2023-11-24T07:56:29.677 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '2.6', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 17, 'state': 'creating+peering', 'last_fresh': '2023-11-24T07:56:22.690376+0000', 'last_change': '2023-11-24T07:56:22.690376+0000', 'last_active': '2023-11-24T07:56:22.678019+0000', 'last_peered': '2023-11-24T07:56:22.678019+0000', 'last_clean': '2023-11-24T07:56:22.678019+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-24T07:56:22.690376+0000', 'last_undegraded': '2023-11-24T07:56:22.690376+0000', 'last_fullsized': '2023-11-24T07:56:22.690376+0000', 'mapping_epoch': 17, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-24T07:56:22.678019+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-24T07:56:22.678019+0000', 'last_clean_scrub_stamp': '2023-11-24T07:56:22.678019+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 0], 'acting': [4, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2023-11-24T07:56:29.677 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.7 is not active+clean 2023-11-24T07:56:29.678 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.7', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-24T07:56:23.683366+0000', 'last_change': '2023-11-24T07:56:23.683366+0000', 'last_active': '2023-11-24T07:56:23.683366+0000', 'last_peered': '2023-11-24T07:56:23.683366+0000', 'last_clean': '2023-11-24T07:56:23.683366+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-24T07:56:23.683366+0000', 'last_undegraded': '2023-11-24T07:56:23.683366+0000', 'last_fullsized': '2023-11-24T07:56:23.683366+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-24T07:56:23.683366+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-24T07:56:23.683366+0000', 'last_clean_scrub_stamp': '2023-11-24T07:56:23.683366+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-24T07:56:29.678 INFO:tasks.cephfs.filesystem.ceph_manager:PG 2.7 is not active+clean 2023-11-24T07:56:29.678 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '2.7', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 17, 'state': 'creating+peering', 'last_fresh': '2023-11-24T07:56:22.695123+0000', 'last_change': '2023-11-24T07:56:22.695123+0000', 'last_active': '2023-11-24T07:56:22.678019+0000', 'last_peered': '2023-11-24T07:56:22.678019+0000', 'last_clean': '2023-11-24T07:56:22.678019+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-24T07:56:22.695123+0000', 'last_undegraded': '2023-11-24T07:56:22.695123+0000', 'last_fullsized': '2023-11-24T07:56:22.695123+0000', 'mapping_epoch': 17, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-24T07:56:22.678019+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-24T07:56:22.678019+0000', 'last_clean_scrub_stamp': '2023-11-24T07:56:22.678019+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 0], 'acting': [3, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2023-11-24T07:56:29.678 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.6 is not active+clean 2023-11-24T07:56:29.678 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.6', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-24T07:56:23.683366+0000', 'last_change': '2023-11-24T07:56:23.683366+0000', 'last_active': '2023-11-24T07:56:23.683366+0000', 'last_peered': '2023-11-24T07:56:23.683366+0000', 'last_clean': '2023-11-24T07:56:23.683366+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-24T07:56:23.683366+0000', 'last_undegraded': '2023-11-24T07:56:23.683366+0000', 'last_fullsized': '2023-11-24T07:56:23.683366+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-24T07:56:23.683366+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-24T07:56:23.683366+0000', 'last_clean_scrub_stamp': '2023-11-24T07:56:23.683366+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-24T07:56:29.678 INFO:tasks.cephfs.filesystem.ceph_manager:PG 2.8 is not active+clean 2023-11-24T07:56:29.678 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '2.8', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 17, 'state': 'creating+peering', 'last_fresh': '2023-11-24T07:56:22.690282+0000', 'last_change': '2023-11-24T07:56:22.690282+0000', 'last_active': '2023-11-24T07:56:22.678019+0000', 'last_peered': '2023-11-24T07:56:22.678019+0000', 'last_clean': '2023-11-24T07:56:22.678019+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-24T07:56:22.690282+0000', 'last_undegraded': '2023-11-24T07:56:22.690282+0000', 'last_fullsized': '2023-11-24T07:56:22.690282+0000', 'mapping_epoch': 17, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-24T07:56:22.678019+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-24T07:56:22.678019+0000', 'last_clean_scrub_stamp': '2023-11-24T07:56:22.678019+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 2], 'acting': [0, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2023-11-24T07:56:29.679 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.9 is not active+clean 2023-11-24T07:56:29.679 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.9', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-24T07:56:23.683366+0000', 'last_change': '2023-11-24T07:56:23.683366+0000', 'last_active': '2023-11-24T07:56:23.683366+0000', 'last_peered': '2023-11-24T07:56:23.683366+0000', 'last_clean': '2023-11-24T07:56:23.683366+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-24T07:56:23.683366+0000', 'last_undegraded': '2023-11-24T07:56:23.683366+0000', 'last_fullsized': '2023-11-24T07:56:23.683366+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-24T07:56:23.683366+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-24T07:56:23.683366+0000', 'last_clean_scrub_stamp': '2023-11-24T07:56:23.683366+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-24T07:56:29.679 INFO:tasks.cephfs.filesystem.ceph_manager:PG 2.9 is not active+clean 2023-11-24T07:56:29.679 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '2.9', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 17, 'state': 'creating+peering', 'last_fresh': '2023-11-24T07:56:22.690549+0000', 'last_change': '2023-11-24T07:56:22.690549+0000', 'last_active': '2023-11-24T07:56:22.678019+0000', 'last_peered': '2023-11-24T07:56:22.678019+0000', 'last_clean': '2023-11-24T07:56:22.678019+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-24T07:56:22.690549+0000', 'last_undegraded': '2023-11-24T07:56:22.690549+0000', 'last_fullsized': '2023-11-24T07:56:22.690549+0000', 'mapping_epoch': 17, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-24T07:56:22.678019+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-24T07:56:22.678019+0000', 'last_clean_scrub_stamp': '2023-11-24T07:56:22.678019+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 0], 'acting': [4, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2023-11-24T07:56:29.679 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.8 is not active+clean 2023-11-24T07:56:29.679 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.8', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-24T07:56:23.683366+0000', 'last_change': '2023-11-24T07:56:23.683366+0000', 'last_active': '2023-11-24T07:56:23.683366+0000', 'last_peered': '2023-11-24T07:56:23.683366+0000', 'last_clean': '2023-11-24T07:56:23.683366+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-24T07:56:23.683366+0000', 'last_undegraded': '2023-11-24T07:56:23.683366+0000', 'last_fullsized': '2023-11-24T07:56:23.683366+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-24T07:56:23.683366+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-24T07:56:23.683366+0000', 'last_clean_scrub_stamp': '2023-11-24T07:56:23.683366+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-24T07:56:29.679 INFO:tasks.cephfs.filesystem.ceph_manager:PG 2.a is not active+clean 2023-11-24T07:56:29.679 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '2.a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 17, 'state': 'creating+peering', 'last_fresh': '2023-11-24T07:56:22.689706+0000', 'last_change': '2023-11-24T07:56:22.689706+0000', 'last_active': '2023-11-24T07:56:22.678019+0000', 'last_peered': '2023-11-24T07:56:22.678019+0000', 'last_clean': '2023-11-24T07:56:22.678019+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-24T07:56:22.689706+0000', 'last_undegraded': '2023-11-24T07:56:22.689706+0000', 'last_fullsized': '2023-11-24T07:56:22.689706+0000', 'mapping_epoch': 17, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-24T07:56:22.678019+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-24T07:56:22.678019+0000', 'last_clean_scrub_stamp': '2023-11-24T07:56:22.678019+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 5], 'acting': [6, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2023-11-24T07:56:29.679 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.b is not active+clean 2023-11-24T07:56:29.679 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.b', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-24T07:56:23.683366+0000', 'last_change': '2023-11-24T07:56:23.683366+0000', 'last_active': '2023-11-24T07:56:23.683366+0000', 'last_peered': '2023-11-24T07:56:23.683366+0000', 'last_clean': '2023-11-24T07:56:23.683366+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-24T07:56:23.683366+0000', 'last_undegraded': '2023-11-24T07:56:23.683366+0000', 'last_fullsized': '2023-11-24T07:56:23.683366+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-24T07:56:23.683366+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-24T07:56:23.683366+0000', 'last_clean_scrub_stamp': '2023-11-24T07:56:23.683366+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-24T07:56:29.680 INFO:tasks.cephfs.filesystem.ceph_manager:PG 2.b is not active+clean 2023-11-24T07:56:29.680 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '2.b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 17, 'state': 'creating+peering', 'last_fresh': '2023-11-24T07:56:22.695809+0000', 'last_change': '2023-11-24T07:56:22.695809+0000', 'last_active': '2023-11-24T07:56:22.678019+0000', 'last_peered': '2023-11-24T07:56:22.678019+0000', 'last_clean': '2023-11-24T07:56:22.678019+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-24T07:56:22.695809+0000', 'last_undegraded': '2023-11-24T07:56:22.695809+0000', 'last_fullsized': '2023-11-24T07:56:22.695809+0000', 'mapping_epoch': 17, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-24T07:56:22.678019+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-24T07:56:22.678019+0000', 'last_clean_scrub_stamp': '2023-11-24T07:56:22.678019+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 2], 'acting': [3, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2023-11-24T07:56:29.680 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.a is not active+clean 2023-11-24T07:56:29.680 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.a', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-24T07:56:23.683366+0000', 'last_change': '2023-11-24T07:56:23.683366+0000', 'last_active': '2023-11-24T07:56:23.683366+0000', 'last_peered': '2023-11-24T07:56:23.683366+0000', 'last_clean': '2023-11-24T07:56:23.683366+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-24T07:56:23.683366+0000', 'last_undegraded': '2023-11-24T07:56:23.683366+0000', 'last_fullsized': '2023-11-24T07:56:23.683366+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-24T07:56:23.683366+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-24T07:56:23.683366+0000', 'last_clean_scrub_stamp': '2023-11-24T07:56:23.683366+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-24T07:56:29.680 INFO:tasks.cephfs.filesystem.ceph_manager:PG 2.c is not active+clean 2023-11-24T07:56:29.680 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '2.c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 17, 'state': 'creating+peering', 'last_fresh': '2023-11-24T07:56:22.690094+0000', 'last_change': '2023-11-24T07:56:22.690094+0000', 'last_active': '2023-11-24T07:56:22.678019+0000', 'last_peered': '2023-11-24T07:56:22.678019+0000', 'last_clean': '2023-11-24T07:56:22.678019+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-24T07:56:22.690094+0000', 'last_undegraded': '2023-11-24T07:56:22.690094+0000', 'last_fullsized': '2023-11-24T07:56:22.690094+0000', 'mapping_epoch': 17, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-24T07:56:22.678019+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-24T07:56:22.678019+0000', 'last_clean_scrub_stamp': '2023-11-24T07:56:22.678019+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 0], 'acting': [6, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2023-11-24T07:56:29.680 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.d is not active+clean 2023-11-24T07:56:29.680 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.d', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-24T07:56:23.683366+0000', 'last_change': '2023-11-24T07:56:23.683366+0000', 'last_active': '2023-11-24T07:56:23.683366+0000', 'last_peered': '2023-11-24T07:56:23.683366+0000', 'last_clean': '2023-11-24T07:56:23.683366+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-24T07:56:23.683366+0000', 'last_undegraded': '2023-11-24T07:56:23.683366+0000', 'last_fullsized': '2023-11-24T07:56:23.683366+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-24T07:56:23.683366+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-24T07:56:23.683366+0000', 'last_clean_scrub_stamp': '2023-11-24T07:56:23.683366+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-24T07:56:29.680 INFO:tasks.cephfs.filesystem.ceph_manager:PG 2.d is not active+clean 2023-11-24T07:56:29.680 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '2.d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 17, 'state': 'creating+peering', 'last_fresh': '2023-11-24T07:56:22.688223+0000', 'last_change': '2023-11-24T07:56:22.688223+0000', 'last_active': '2023-11-24T07:56:22.678019+0000', 'last_peered': '2023-11-24T07:56:22.678019+0000', 'last_clean': '2023-11-24T07:56:22.678019+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-24T07:56:22.688223+0000', 'last_undegraded': '2023-11-24T07:56:22.688223+0000', 'last_fullsized': '2023-11-24T07:56:22.688223+0000', 'mapping_epoch': 17, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-24T07:56:22.678019+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-24T07:56:22.678019+0000', 'last_clean_scrub_stamp': '2023-11-24T07:56:22.678019+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 1], 'acting': [5, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2023-11-24T07:56:29.681 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.c is not active+clean 2023-11-24T07:56:29.681 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.c', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-24T07:56:23.683366+0000', 'last_change': '2023-11-24T07:56:23.683366+0000', 'last_active': '2023-11-24T07:56:23.683366+0000', 'last_peered': '2023-11-24T07:56:23.683366+0000', 'last_clean': '2023-11-24T07:56:23.683366+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-24T07:56:23.683366+0000', 'last_undegraded': '2023-11-24T07:56:23.683366+0000', 'last_fullsized': '2023-11-24T07:56:23.683366+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-24T07:56:23.683366+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-24T07:56:23.683366+0000', 'last_clean_scrub_stamp': '2023-11-24T07:56:23.683366+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-24T07:56:29.681 INFO:tasks.cephfs.filesystem.ceph_manager:PG 2.e is not active+clean 2023-11-24T07:56:29.681 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '2.e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 17, 'state': 'creating+peering', 'last_fresh': '2023-11-24T07:56:22.691566+0000', 'last_change': '2023-11-24T07:56:22.691566+0000', 'last_active': '2023-11-24T07:56:22.678019+0000', 'last_peered': '2023-11-24T07:56:22.678019+0000', 'last_clean': '2023-11-24T07:56:22.678019+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-24T07:56:22.691566+0000', 'last_undegraded': '2023-11-24T07:56:22.691566+0000', 'last_fullsized': '2023-11-24T07:56:22.691566+0000', 'mapping_epoch': 17, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-24T07:56:22.678019+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-24T07:56:22.678019+0000', 'last_clean_scrub_stamp': '2023-11-24T07:56:22.678019+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 5], 'acting': [4, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2023-11-24T07:56:29.681 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.f is not active+clean 2023-11-24T07:56:29.681 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.f', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-24T07:56:23.683366+0000', 'last_change': '2023-11-24T07:56:23.683366+0000', 'last_active': '2023-11-24T07:56:23.683366+0000', 'last_peered': '2023-11-24T07:56:23.683366+0000', 'last_clean': '2023-11-24T07:56:23.683366+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-24T07:56:23.683366+0000', 'last_undegraded': '2023-11-24T07:56:23.683366+0000', 'last_fullsized': '2023-11-24T07:56:23.683366+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-24T07:56:23.683366+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-24T07:56:23.683366+0000', 'last_clean_scrub_stamp': '2023-11-24T07:56:23.683366+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-24T07:56:29.681 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.20 is not active+clean 2023-11-24T07:56:29.681 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.20', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-24T07:56:23.683366+0000', 'last_change': '2023-11-24T07:56:23.683366+0000', 'last_active': '2023-11-24T07:56:23.683366+0000', 'last_peered': '2023-11-24T07:56:23.683366+0000', 'last_clean': '2023-11-24T07:56:23.683366+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-24T07:56:23.683366+0000', 'last_undegraded': '2023-11-24T07:56:23.683366+0000', 'last_fullsized': '2023-11-24T07:56:23.683366+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-24T07:56:23.683366+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-24T07:56:23.683366+0000', 'last_clean_scrub_stamp': '2023-11-24T07:56:23.683366+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-24T07:56:29.681 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.21 is not active+clean 2023-11-24T07:56:29.681 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.21', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-24T07:56:23.683366+0000', 'last_change': '2023-11-24T07:56:23.683366+0000', 'last_active': '2023-11-24T07:56:23.683366+0000', 'last_peered': '2023-11-24T07:56:23.683366+0000', 'last_clean': '2023-11-24T07:56:23.683366+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-24T07:56:23.683366+0000', 'last_undegraded': '2023-11-24T07:56:23.683366+0000', 'last_fullsized': '2023-11-24T07:56:23.683366+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-24T07:56:23.683366+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-24T07:56:23.683366+0000', 'last_clean_scrub_stamp': '2023-11-24T07:56:23.683366+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-24T07:56:29.682 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.22 is not active+clean 2023-11-24T07:56:29.682 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.22', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-24T07:56:23.683366+0000', 'last_change': '2023-11-24T07:56:23.683366+0000', 'last_active': '2023-11-24T07:56:23.683366+0000', 'last_peered': '2023-11-24T07:56:23.683366+0000', 'last_clean': '2023-11-24T07:56:23.683366+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-24T07:56:23.683366+0000', 'last_undegraded': '2023-11-24T07:56:23.683366+0000', 'last_fullsized': '2023-11-24T07:56:23.683366+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-24T07:56:23.683366+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-24T07:56:23.683366+0000', 'last_clean_scrub_stamp': '2023-11-24T07:56:23.683366+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-24T07:56:29.682 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.23 is not active+clean 2023-11-24T07:56:29.682 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.23', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-24T07:56:23.683366+0000', 'last_change': '2023-11-24T07:56:23.683366+0000', 'last_active': '2023-11-24T07:56:23.683366+0000', 'last_peered': '2023-11-24T07:56:23.683366+0000', 'last_clean': '2023-11-24T07:56:23.683366+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-24T07:56:23.683366+0000', 'last_undegraded': '2023-11-24T07:56:23.683366+0000', 'last_fullsized': '2023-11-24T07:56:23.683366+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-24T07:56:23.683366+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-24T07:56:23.683366+0000', 'last_clean_scrub_stamp': '2023-11-24T07:56:23.683366+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-24T07:56:29.682 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.24 is not active+clean 2023-11-24T07:56:29.682 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.24', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-24T07:56:23.683366+0000', 'last_change': '2023-11-24T07:56:23.683366+0000', 'last_active': '2023-11-24T07:56:23.683366+0000', 'last_peered': '2023-11-24T07:56:23.683366+0000', 'last_clean': '2023-11-24T07:56:23.683366+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-24T07:56:23.683366+0000', 'last_undegraded': '2023-11-24T07:56:23.683366+0000', 'last_fullsized': '2023-11-24T07:56:23.683366+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-24T07:56:23.683366+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-24T07:56:23.683366+0000', 'last_clean_scrub_stamp': '2023-11-24T07:56:23.683366+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-24T07:56:29.682 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.25 is not active+clean 2023-11-24T07:56:29.682 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.25', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-24T07:56:23.683366+0000', 'last_change': '2023-11-24T07:56:23.683366+0000', 'last_active': '2023-11-24T07:56:23.683366+0000', 'last_peered': '2023-11-24T07:56:23.683366+0000', 'last_clean': '2023-11-24T07:56:23.683366+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-24T07:56:23.683366+0000', 'last_undegraded': '2023-11-24T07:56:23.683366+0000', 'last_fullsized': '2023-11-24T07:56:23.683366+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-24T07:56:23.683366+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-24T07:56:23.683366+0000', 'last_clean_scrub_stamp': '2023-11-24T07:56:23.683366+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-24T07:56:29.683 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.26 is not active+clean 2023-11-24T07:56:29.683 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.26', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-24T07:56:23.683366+0000', 'last_change': '2023-11-24T07:56:23.683366+0000', 'last_active': '2023-11-24T07:56:23.683366+0000', 'last_peered': '2023-11-24T07:56:23.683366+0000', 'last_clean': '2023-11-24T07:56:23.683366+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-24T07:56:23.683366+0000', 'last_undegraded': '2023-11-24T07:56:23.683366+0000', 'last_fullsized': '2023-11-24T07:56:23.683366+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-24T07:56:23.683366+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-24T07:56:23.683366+0000', 'last_clean_scrub_stamp': '2023-11-24T07:56:23.683366+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-24T07:56:29.683 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.27 is not active+clean 2023-11-24T07:56:29.683 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.27', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-24T07:56:23.683366+0000', 'last_change': '2023-11-24T07:56:23.683366+0000', 'last_active': '2023-11-24T07:56:23.683366+0000', 'last_peered': '2023-11-24T07:56:23.683366+0000', 'last_clean': '2023-11-24T07:56:23.683366+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-24T07:56:23.683366+0000', 'last_undegraded': '2023-11-24T07:56:23.683366+0000', 'last_fullsized': '2023-11-24T07:56:23.683366+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-24T07:56:23.683366+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-24T07:56:23.683366+0000', 'last_clean_scrub_stamp': '2023-11-24T07:56:23.683366+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-24T07:56:29.683 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.28 is not active+clean 2023-11-24T07:56:29.683 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.28', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-24T07:56:23.683366+0000', 'last_change': '2023-11-24T07:56:23.683366+0000', 'last_active': '2023-11-24T07:56:23.683366+0000', 'last_peered': '2023-11-24T07:56:23.683366+0000', 'last_clean': '2023-11-24T07:56:23.683366+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-24T07:56:23.683366+0000', 'last_undegraded': '2023-11-24T07:56:23.683366+0000', 'last_fullsized': '2023-11-24T07:56:23.683366+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-24T07:56:23.683366+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-24T07:56:23.683366+0000', 'last_clean_scrub_stamp': '2023-11-24T07:56:23.683366+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-24T07:56:29.683 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.29 is not active+clean 2023-11-24T07:56:29.683 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.29', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-24T07:56:23.683366+0000', 'last_change': '2023-11-24T07:56:23.683366+0000', 'last_active': '2023-11-24T07:56:23.683366+0000', 'last_peered': '2023-11-24T07:56:23.683366+0000', 'last_clean': '2023-11-24T07:56:23.683366+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-24T07:56:23.683366+0000', 'last_undegraded': '2023-11-24T07:56:23.683366+0000', 'last_fullsized': '2023-11-24T07:56:23.683366+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-24T07:56:23.683366+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-24T07:56:23.683366+0000', 'last_clean_scrub_stamp': '2023-11-24T07:56:23.683366+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-24T07:56:29.683 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.2a is not active+clean 2023-11-24T07:56:29.684 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.2a', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-24T07:56:23.683366+0000', 'last_change': '2023-11-24T07:56:23.683366+0000', 'last_active': '2023-11-24T07:56:23.683366+0000', 'last_peered': '2023-11-24T07:56:23.683366+0000', 'last_clean': '2023-11-24T07:56:23.683366+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-24T07:56:23.683366+0000', 'last_undegraded': '2023-11-24T07:56:23.683366+0000', 'last_fullsized': '2023-11-24T07:56:23.683366+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-24T07:56:23.683366+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-24T07:56:23.683366+0000', 'last_clean_scrub_stamp': '2023-11-24T07:56:23.683366+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-24T07:56:29.684 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.2b is not active+clean 2023-11-24T07:56:29.684 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.2b', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-24T07:56:23.683366+0000', 'last_change': '2023-11-24T07:56:23.683366+0000', 'last_active': '2023-11-24T07:56:23.683366+0000', 'last_peered': '2023-11-24T07:56:23.683366+0000', 'last_clean': '2023-11-24T07:56:23.683366+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-24T07:56:23.683366+0000', 'last_undegraded': '2023-11-24T07:56:23.683366+0000', 'last_fullsized': '2023-11-24T07:56:23.683366+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-24T07:56:23.683366+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-24T07:56:23.683366+0000', 'last_clean_scrub_stamp': '2023-11-24T07:56:23.683366+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-24T07:56:29.684 INFO:tasks.cephfs.filesystem.ceph_manager:PG 3.2c is not active+clean 2023-11-24T07:56:29.684 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '3.2c', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-24T07:56:23.683366+0000', 'last_change': '2023-11-24T07:56:23.683366+0000', 'last_active': '2023-11-24T07:56:23.683366+0000', 'last_peered': '2023-11-24T07:56:23.683366+0000', 'last_clean': '2023-11-24T07:56:23.683366+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-24T07:56:23.683366+0000', 'last_undegraded': '2023-11-24T07:56:23.683366+0000', 'last_fullsized': '2023-11-24T07:56:23.683366+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-24T07:56:23.683366+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-24T07:56:23.683366+0000', 'last_clean_scrub_stamp': '2023-11-24T07:56:23.683366+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-24T07:56:29.684 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2023-11-24T07:56:29.685 INFO:tasks.ceph.mds.b.smithi076.stderr:2023-11-24T07:56:29.664+0000 7f2eff170700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-11-24T07:56:29.824 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:29.822+0000 7f4e13683700 1 Processor -- start 2023-11-24T07:56:29.825 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:29.822+0000 7f4e13683700 1 -- start start 2023-11-24T07:56:29.833 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:29.823+0000 7f4e13683700 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f4e0c144d10 0x7f4e0c06f720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:56:29.834 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:29.823+0000 7f4e13683700 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f4e0c06b6e0 0x7f4e0c1450f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:56:29.834 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:29.823+0000 7f4e13683700 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4e0c06e580 con 0x7f4e0c15c460 2023-11-24T07:56:29.834 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:29.823+0000 7f4e13683700 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4e0c0d0960 con 0x7f4e0c06b6e0 2023-11-24T07:56:29.834 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:29.823+0000 7f4e13683700 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4e0c06e7d0 con 0x7f4e0c144d10 2023-11-24T07:56:29.834 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:29.823+0000 7f4e10c1e700 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7f4e0c15c460 0x7f4e0c06afd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:53288/0 (socket says 172.21.15.59:53288) 2023-11-24T07:56:29.834 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:29.823+0000 7f4e10c1e700 1 -- 172.21.15.59:0/3802791733 learned_addr learned my addr 172.21.15.59:0/3802791733 (peer_addr_for_me v1:172.21.15.59:0/0) 2023-11-24T07:56:29.834 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:29.823+0000 7f4e11c20700 1 --2- 172.21.15.59:0/3802791733 >> v2:172.21.15.76:3300/0 conn(0x7f4e0c06b6e0 0x7f4e0c1450f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:56:29.834 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:29.823+0000 7f4e1141f700 1 --2- 172.21.15.59:0/3802791733 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f4e0c144d10 0x7f4e0c06f720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:56:29.834 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:29.823+0000 7f4e03fff700 1 -- 172.21.15.59:0/3802791733 <== mon.0 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3783091563 0 0) 0x7f4e0c06e580 con 0x7f4e0c15c460 2023-11-24T07:56:29.834 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:29.824+0000 7f4e03fff700 1 -- 172.21.15.59:0/3802791733 --> v1:172.21.15.59:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4df80035b0 con 0x7f4e0c15c460 2023-11-24T07:56:29.834 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:29.824+0000 7f4e03fff700 1 -- 172.21.15.59:0/3802791733 <== mon.0 v1:172.21.15.59:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2761851155 0 0) 0x7f4df80035b0 con 0x7f4e0c15c460 2023-11-24T07:56:29.835 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:29.824+0000 7f4e03fff700 1 -- 172.21.15.59:0/3802791733 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f4e0c144d10 msgr2=0x7f4e0c06f720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T07:56:29.835 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:29.824+0000 7f4e03fff700 1 --2- 172.21.15.59:0/3802791733 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f4e0c144d10 0x7f4e0c06f720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:56:29.835 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:29.824+0000 7f4e03fff700 1 -- 172.21.15.59:0/3802791733 >> v2:172.21.15.76:3300/0 conn(0x7f4e0c06b6e0 msgr2=0x7f4e0c1450f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T07:56:29.835 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:29.824+0000 7f4e03fff700 1 --2- 172.21.15.59:0/3802791733 >> v2:172.21.15.76:3300/0 conn(0x7f4e0c06b6e0 0x7f4e0c1450f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:56:29.835 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:29.824+0000 7f4e03fff700 1 -- 172.21.15.59:0/3802791733 --> v1:172.21.15.59:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4e0c06eab0 con 0x7f4e0c15c460 2023-11-24T07:56:29.835 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:29.824+0000 7f4e03fff700 1 -- 172.21.15.59:0/3802791733 <== mon.0 v1:172.21.15.59:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1312333566 0 0) 0x7f4dfc001670 con 0x7f4e0c15c460 2023-11-24T07:56:29.836 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:29.824+0000 7f4e03fff700 1 -- 172.21.15.59:0/3802791733 >> v1:172.21.15.59:6789/0 conn(0x7f4e0c15c460 legacy=0x7f4e0c06afd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:56:29.836 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:29.824+0000 7f4e03fff700 1 --2- 172.21.15.59:0/3802791733 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f4e0c144d10 0x7f4df8003a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:56:29.836 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:29.824+0000 7f4e03fff700 1 --2- 172.21.15.59:0/3802791733 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4df8003f80 0x7f4df8004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:56:29.836 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:29.824+0000 7f4e03fff700 1 --2- 172.21.15.59:0/3802791733 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4df8004d90 0x7f4df8005210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:56:29.836 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:29.824+0000 7f4e03fff700 1 -- 172.21.15.59:0/3802791733 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e0c0d0960 con 0x7f4df8004d90 2023-11-24T07:56:29.836 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:29.824+0000 7f4e03fff700 1 -- 172.21.15.59:0/3802791733 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e0c06e7d0 con 0x7f4df8003f80 2023-11-24T07:56:29.836 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:29.824+0000 7f4e03fff700 1 -- 172.21.15.59:0/3802791733 --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4df800ba60 con 0x7f4e0c144d10 2023-11-24T07:56:29.837 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:29.824+0000 7f4e11c20700 1 --2- 172.21.15.59:0/3802791733 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4df8004d90 0x7f4df8005210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:56:29.837 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:29.824+0000 7f4e1141f700 1 --2- 172.21.15.59:0/3802791733 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f4e0c144d10 0x7f4df8003a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:56:29.837 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:29.824+0000 7f4e10c1e700 1 --2- 172.21.15.59:0/3802791733 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4df8003f80 0x7f4df8004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:56:29.837 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:29.824+0000 7f4e11c20700 1 -- 172.21.15.59:0/3802791733 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f4e0c144d10 msgr2=0x7f4df8003a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T07:56:29.837 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:29.824+0000 7f4e11c20700 1 --2- 172.21.15.59:0/3802791733 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f4e0c144d10 0x7f4df8003a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:56:29.837 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:29.824+0000 7f4e11c20700 1 -- 172.21.15.59:0/3802791733 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4df8003f80 msgr2=0x7f4df8004850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T07:56:29.838 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:29.824+0000 7f4e11c20700 1 --2- 172.21.15.59:0/3802791733 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4df8003f80 0x7f4df8004850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:56:29.838 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:29.824+0000 7f4e11c20700 1 -- 172.21.15.59:0/3802791733 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4df80035b0 con 0x7f4df8004d90 2023-11-24T07:56:29.838 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:29.824+0000 7f4e11c20700 1 --2- 172.21.15.59:0/3802791733 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4df8004d90 0x7f4df8005210 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f4e04009fc0 tx=0x7f4df8005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3f1c0f1b19d8347c server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:56:29.838 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:29.825+0000 7f4e03fff700 1 -- 172.21.15.59:0/3802791733 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e0400ec30 con 0x7f4df8004d90 2023-11-24T07:56:29.838 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:29.825+0000 7f4e03fff700 1 -- 172.21.15.59:0/3802791733 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4e0400f040 con 0x7f4df8004d90 2023-11-24T07:56:29.839 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:29.825+0000 7f4e03fff700 1 -- 172.21.15.59:0/3802791733 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e04014480 con 0x7f4df8004d90 2023-11-24T07:56:29.839 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:29.825+0000 7f4e13683700 1 -- 172.21.15.59:0/3802791733 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4df8004d90 msgr2=0x7f4df8005210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:56:29.839 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:29.825+0000 7f4e13683700 1 --2- 172.21.15.59:0/3802791733 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4df8004d90 0x7f4df8005210 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f4e04009fc0 tx=0x7f4df8005880 comp rx=0 tx=0).stop 2023-11-24T07:56:29.839 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:29.825+0000 7f4e11c20700 1 -- 172.21.15.59:0/3802791733 reap_dead start 2023-11-24T07:56:29.839 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:29.825+0000 7f4e13683700 1 -- 172.21.15.59:0/3802791733 shutdown_connections 2023-11-24T07:56:29.839 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:29.825+0000 7f4e13683700 1 -- 172.21.15.59:0/3802791733 >> 172.21.15.59:0/3802791733 conn(0x7f4e0c0b2740 msgr2=0x7f4e0c0b2b20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:56:29.839 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:29.825+0000 7f4e13683700 1 -- 172.21.15.59:0/3802791733 shutdown_connections 2023-11-24T07:56:29.840 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:29.825+0000 7f4e13683700 1 -- 172.21.15.59:0/3802791733 wait complete. 2023-11-24T07:56:29.840 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:29.826+0000 7f4e13683700 1 Processor -- start 2023-11-24T07:56:29.840 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:29.827+0000 7f4e13683700 1 -- start start 2023-11-24T07:56:29.840 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:29.827+0000 7f4e13683700 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f4df8004d90 0x7f4e0c0b5120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:56:29.840 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:29.827+0000 7f4e13683700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4e0c1e6270 0x7f4e0c1f6b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:56:29.840 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:29.827+0000 7f4e13683700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4e0c1f3d40 0x7f4e0c1e37c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:56:29.840 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:29.827+0000 7f4e13683700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e0c1f7350 con 0x7f4e0c1e6270 2023-11-24T07:56:29.841 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:29.827+0000 7f4e13683700 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e0c1f7630 con 0x7f4e0c1f3d40 2023-11-24T07:56:29.841 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:29.827+0000 7f4e13683700 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4e0c1f7910 con 0x7f4df8004d90 2023-11-24T07:56:29.841 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:29.827+0000 7f4e10c1e700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4e0c1e6270 0x7f4e0c1f6b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:56:29.841 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:29.827+0000 7f4e10c1e700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4e0c1e6270 0x7f4e0c1f6b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:49826/0 (socket says 172.21.15.59:49826) 2023-11-24T07:56:29.841 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:29.827+0000 7f4e10c1e700 1 -- 172.21.15.59:0/2101765776 learned_addr learned my addr 172.21.15.59:0/2101765776 (peer_addr_for_me v2:172.21.15.59:0/0) 2023-11-24T07:56:29.841 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:29.828+0000 7f4e1141f700 1 --2- 172.21.15.59:0/2101765776 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f4df8004d90 0x7f4e0c0b5120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:56:29.841 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:29.828+0000 7f4e11c20700 1 --2- 172.21.15.59:0/2101765776 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4e0c1f3d40 0x7f4e0c1e37c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:56:29.842 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:29.828+0000 7f4e10c1e700 1 -- 172.21.15.59:0/2101765776 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f4df8004d90 msgr2=0x7f4e0c0b5120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:56:29.842 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:29.828+0000 7f4e10c1e700 1 --2- 172.21.15.59:0/2101765776 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f4df8004d90 0x7f4e0c0b5120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:56:29.842 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:29.828+0000 7f4e10c1e700 1 -- 172.21.15.59:0/2101765776 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4e0c1f3d40 msgr2=0x7f4e0c1e37c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:56:29.842 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:29.828+0000 7f4e10c1e700 1 --2- 172.21.15.59:0/2101765776 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4e0c1f3d40 0x7f4e0c1e37c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:56:29.842 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:29.828+0000 7f4e10c1e700 1 -- 172.21.15.59:0/2101765776 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4e0c1f3820 con 0x7f4e0c1e6270 2023-11-24T07:56:29.842 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:29.828+0000 7f4e10c1e700 1 --2- 172.21.15.59:0/2101765776 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4e0c1e6270 0x7f4e0c1f6b00 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f4dfc001aa0 tx=0x7f4dfc0076a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:56:29.843 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:29.828+0000 7f4e01ffb700 1 -- 172.21.15.59:0/2101765776 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4dfc00f8a0 con 0x7f4e0c1e6270 2023-11-24T07:56:29.843 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:29.828+0000 7f4e01ffb700 1 -- 172.21.15.59:0/2101765776 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4dfc00fa20 con 0x7f4e0c1e6270 2023-11-24T07:56:29.843 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:29.828+0000 7f4e13683700 1 -- 172.21.15.59:0/2101765776 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4e0c1f7040 con 0x7f4e0c1e6270 2023-11-24T07:56:29.843 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:29.828+0000 7f4e01ffb700 1 -- 172.21.15.59:0/2101765776 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4dfc00fdd0 con 0x7f4e0c1e6270 2023-11-24T07:56:29.843 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:29.828+0000 7f4e13683700 1 -- 172.21.15.59:0/2101765776 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4e0c1f41c0 con 0x7f4e0c1e6270 2023-11-24T07:56:29.844 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:29.829+0000 7f4e13683700 1 -- 172.21.15.59:0/2101765776 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4e0c0cf0a0 con 0x7f4e0c1e6270 2023-11-24T07:56:29.844 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:29.830+0000 7f4e01ffb700 1 -- 172.21.15.59:0/2101765776 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f4dfc01d460 con 0x7f4e0c1e6270 2023-11-24T07:56:29.844 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:29.830+0000 7f4e01ffb700 1 --2- 172.21.15.59:0/2101765776 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f4df006af00 0x7f4df006d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:56:29.844 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:29.830+0000 7f4e01ffb700 1 -- 172.21.15.59:0/2101765776 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f4dfc0896a0 con 0x7f4e0c1e6270 2023-11-24T07:56:29.844 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:29.830+0000 7f4e1141f700 1 --2- 172.21.15.59:0/2101765776 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f4df006af00 0x7f4df006d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:56:29.844 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:29.831+0000 7f4e1141f700 1 --2- 172.21.15.59:0/2101765776 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f4df006af00 0x7f4df006d3c0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f4e08009aa0 tx=0x7f4e08008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:56:29.845 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:29.834+0000 7f4e01ffb700 1 -- 172.21.15.59:0/2101765776 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f4dfc054750 con 0x7f4e0c1e6270 2023-11-24T07:56:30.035 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-24T07:56:30.038 DEBUG:teuthology.orchestra.run.smithi076:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-24T07:56:30.125 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:30.123+0000 7f4e13683700 1 -- 172.21.15.59:0/2101765776 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) v1 -- 0x7f4e0c068670 con 0x7f4e0c1e6270 2023-11-24T07:56:30.125 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:30.123+0000 7f4e01ffb700 1 -- 172.21.15.59:0/2101765776 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) v1 ==== 72+0+1106 (secure 0 0 0) 0x7f4dfc057d70 con 0x7f4e0c1e6270 2023-11-24T07:56:30.125 INFO:teuthology.orchestra.run.smithi059.stdout: 2023-11-24T07:56:30.126 INFO:teuthology.orchestra.run.smithi059.stdout:{"fsid":"9f0e8741-b8f9-48d5-a2c3-09f45903303d","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":23,"monmap":{"epoch":1,"min_mon_release_name":"reef","num_mons":3},"osdmap":{"epoch":20,"num_osds":8,"num_up_osds":8,"osd_up_since":1700812577,"num_in_osds":8,"osd_in_since":1700812569,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"unknown","count":64},{"state_name":"creating+peering","count":32},{"state_name":"active+clean","count":1}],"num_pgs":97,"num_pools":3,"num_objects":2,"data_bytes":590368,"bytes_used":223842304,"bytes_avail":772870270976,"bytes_total":773094113280,"unknown_pgs_ratio":0.65979379415512085,"inactive_pgs_ratio":0.32989689707756042},"fsmap":{"epoch":8,"id":1,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":1,"rank":0,"name":"b","status":"up:active","gid":4264}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":1,"modified":"2023-11-24T07:56:06.553915+0000","services":{}},"progress_events":{}} 2023-11-24T07:56:30.127 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:30.125+0000 7f4e13683700 1 -- 172.21.15.59:0/2101765776 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f4df006af00 msgr2=0x7f4df006d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:56:30.128 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:30.125+0000 7f4e13683700 1 --2- 172.21.15.59:0/2101765776 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f4df006af00 0x7f4df006d3c0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f4e08009aa0 tx=0x7f4e08008040 comp rx=0 tx=0).stop 2023-11-24T07:56:30.128 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:30.125+0000 7f4e13683700 1 -- 172.21.15.59:0/2101765776 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4e0c1e6270 msgr2=0x7f4e0c1f6b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:56:30.128 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:30.125+0000 7f4e13683700 1 --2- 172.21.15.59:0/2101765776 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4e0c1e6270 0x7f4e0c1f6b00 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f4dfc001aa0 tx=0x7f4dfc0076a0 comp rx=0 tx=0).stop 2023-11-24T07:56:30.128 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:30.125+0000 7f4e13683700 1 -- 172.21.15.59:0/2101765776 shutdown_connections 2023-11-24T07:56:30.128 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:30.125+0000 7f4e13683700 1 --2- 172.21.15.59:0/2101765776 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f4df006af00 0x7f4df006d3c0 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:56:30.129 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:30.125+0000 7f4e13683700 1 --2- 172.21.15.59:0/2101765776 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4e0c1f3d40 0x7f4e0c1e37c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:56:30.129 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:30.125+0000 7f4e13683700 1 --2- 172.21.15.59:0/2101765776 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f4df8004d90 0x7f4e0c0b5120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:56:30.129 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:30.125+0000 7f4e13683700 1 --2- 172.21.15.59:0/2101765776 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4e0c1e6270 0x7f4e0c1f6b00 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:56:30.129 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:30.125+0000 7f4e13683700 1 -- 172.21.15.59:0/2101765776 >> 172.21.15.59:0/2101765776 conn(0x7f4e0c0b2740 msgr2=0x7f4e0c141f40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:56:30.129 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:30.126+0000 7f4e13683700 1 -- 172.21.15.59:0/2101765776 shutdown_connections 2023-11-24T07:56:30.130 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:30.126+0000 7f4e13683700 1 -- 172.21.15.59:0/2101765776 wait complete. 2023-11-24T07:56:30.140 INFO:tasks.cephfs.filesystem.ceph_manager:no progress seen, keeping timeout for now 2023-11-24T07:56:30.140 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-11-24T07:56:30.265 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:30.263+0000 7f81323bd700 1 Processor -- start 2023-11-24T07:56:30.265 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:30.263+0000 7f81323bd700 1 -- start start 2023-11-24T07:56:30.271 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:30.264+0000 7f81323bd700 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f812c06fc50 0x7f812c06eef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:56:30.271 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:30.264+0000 7f81323bd700 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f812c0d1230 con 0x7f812c0b0e90 2023-11-24T07:56:30.271 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:30.264+0000 7f81323bd700 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f812c0d0c70 con 0x7f812c15e1c0 2023-11-24T07:56:30.271 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:30.264+0000 7f81323bd700 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f812c0d0f50 con 0x7f812c06fc50 2023-11-24T07:56:30.272 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:30.264+0000 7f813095a700 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7f812c0b0e90 0x7f812c06ca10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:53312/0 (socket says 172.21.15.59:53312) 2023-11-24T07:56:30.272 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:30.264+0000 7f813095a700 1 -- 172.21.15.59:0/810188506 learned_addr learned my addr 172.21.15.59:0/810188506 (peer_addr_for_me v1:172.21.15.59:0/0) 2023-11-24T07:56:30.272 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:30.264+0000 7f812b7fe700 1 --2- 172.21.15.59:0/810188506 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f812c06fc50 0x7f812c06eef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:56:30.272 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:30.264+0000 7f812affd700 1 -- 172.21.15.59:0/810188506 <== mon.0 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3542898173 0 0) 0x7f812c0d1230 con 0x7f812c0b0e90 2023-11-24T07:56:30.272 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:30.264+0000 7f812affd700 1 -- 172.21.15.59:0/810188506 --> v1:172.21.15.59:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f81180035b0 con 0x7f812c0b0e90 2023-11-24T07:56:30.272 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:30.264+0000 7f812affd700 1 -- 172.21.15.59:0/810188506 <== mon.0 v1:172.21.15.59:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2975100602 0 0) 0x7f81180035b0 con 0x7f812c0b0e90 2023-11-24T07:56:30.272 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:30.264+0000 7f812affd700 1 -- 172.21.15.59:0/810188506 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f812c06fc50 msgr2=0x7f812c06eef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T07:56:30.272 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:30.264+0000 7f812affd700 1 --2- 172.21.15.59:0/810188506 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f812c06fc50 0x7f812c06eef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:56:30.272 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:30.264+0000 7f812affd700 1 -- 172.21.15.59:0/810188506 >> v1:172.21.15.76:6789/0 conn(0x7f812c15e1c0 legacy=0x7f812c06f540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:56:30.272 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:30.264+0000 7f812affd700 1 -- 172.21.15.59:0/810188506 --> v1:172.21.15.59:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f812c06c210 con 0x7f812c0b0e90 2023-11-24T07:56:30.273 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:30.265+0000 7f812affd700 1 -- 172.21.15.59:0/810188506 <== mon.0 v1:172.21.15.59:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1312333566 0 0) 0x7f81240019f0 con 0x7f812c0b0e90 2023-11-24T07:56:30.273 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:30.265+0000 7f812affd700 1 -- 172.21.15.59:0/810188506 >> v1:172.21.15.59:6789/0 conn(0x7f812c0b0e90 legacy=0x7f812c06ca10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:56:30.273 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:30.265+0000 7f812affd700 1 --2- 172.21.15.59:0/810188506 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8118003a30 0x7f8118003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:56:30.273 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:30.265+0000 7f812affd700 1 --2- 172.21.15.59:0/810188506 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f812c06fc50 0x7f8118004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:56:30.273 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:30.265+0000 7f812affd700 1 --2- 172.21.15.59:0/810188506 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8118004d90 0x7f8118005210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:56:30.273 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:30.265+0000 7f812affd700 1 -- 172.21.15.59:0/810188506 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f812c0d0f50 con 0x7f8118003a30 2023-11-24T07:56:30.273 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:30.265+0000 7f812affd700 1 -- 172.21.15.59:0/810188506 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f811800bac0 con 0x7f8118004d90 2023-11-24T07:56:30.273 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:30.265+0000 7f812affd700 1 -- 172.21.15.59:0/810188506 --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f811800bc80 con 0x7f812c06fc50 2023-11-24T07:56:30.273 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:30.265+0000 7f812bfff700 1 --2- 172.21.15.59:0/810188506 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8118003a30 0x7f8118003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:56:30.273 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:30.265+0000 7f812b7fe700 1 --2- 172.21.15.59:0/810188506 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f812c06fc50 0x7f8118004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:56:30.273 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:30.265+0000 7f813095a700 1 --2- 172.21.15.59:0/810188506 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8118004d90 0x7f8118005210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:56:30.273 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:30.265+0000 7f812bfff700 1 -- 172.21.15.59:0/810188506 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f812c06fc50 msgr2=0x7f8118004850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T07:56:30.274 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:30.265+0000 7f812bfff700 1 --2- 172.21.15.59:0/810188506 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f812c06fc50 0x7f8118004850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:56:30.274 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:30.265+0000 7f812bfff700 1 -- 172.21.15.59:0/810188506 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8118004d90 msgr2=0x7f8118005210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T07:56:30.274 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:30.265+0000 7f812bfff700 1 --2- 172.21.15.59:0/810188506 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8118004d90 0x7f8118005210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:56:30.274 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:30.265+0000 7f812bfff700 1 -- 172.21.15.59:0/810188506 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f81180035b0 con 0x7f8118003a30 2023-11-24T07:56:30.274 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:30.265+0000 7f812bfff700 1 --2- 172.21.15.59:0/810188506 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8118003a30 0x7f8118003e10 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f811c0018b0 tx=0x7f8118005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=33c4c2c0a689858b server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:56:30.274 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:30.265+0000 7f812affd700 1 -- 172.21.15.59:0/810188506 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f811c001370 con 0x7f8118003a30 2023-11-24T07:56:30.274 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:30.265+0000 7f812affd700 1 -- 172.21.15.59:0/810188506 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f811c00c6b0 con 0x7f8118003a30 2023-11-24T07:56:30.274 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:30.265+0000 7f812affd700 1 -- 172.21.15.59:0/810188506 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f811c00ca60 con 0x7f8118003a30 2023-11-24T07:56:30.275 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:30.266+0000 7f81323bd700 1 -- 172.21.15.59:0/810188506 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8118003a30 msgr2=0x7f8118003e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:56:30.275 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:30.266+0000 7f81323bd700 1 --2- 172.21.15.59:0/810188506 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8118003a30 0x7f8118003e10 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f811c0018b0 tx=0x7f8118005880 comp rx=0 tx=0).stop 2023-11-24T07:56:30.275 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:30.266+0000 7f813095a700 1 -- 172.21.15.59:0/810188506 reap_dead start 2023-11-24T07:56:30.275 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:30.266+0000 7f81323bd700 1 -- 172.21.15.59:0/810188506 shutdown_connections 2023-11-24T07:56:30.275 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:30.266+0000 7f81323bd700 1 -- 172.21.15.59:0/810188506 >> 172.21.15.59:0/810188506 conn(0x7f812c0b3720 msgr2=0x7f812c0b3b30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:56:30.275 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:30.266+0000 7f81323bd700 1 -- 172.21.15.59:0/810188506 shutdown_connections 2023-11-24T07:56:30.275 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:30.266+0000 7f81323bd700 1 -- 172.21.15.59:0/810188506 wait complete. 2023-11-24T07:56:30.276 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:30.267+0000 7f81323bd700 1 Processor -- start 2023-11-24T07:56:30.276 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:30.267+0000 7f81323bd700 1 -- start start 2023-11-24T07:56:30.276 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:30.268+0000 7f81323bd700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8118003a30 0x7f812c1f7f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:56:30.276 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:30.268+0000 7f81323bd700 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f812c09f200 0x7f812c15cba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:56:30.276 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:30.268+0000 7f81323bd700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f812c15d0e0 0x7f812c161660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:56:30.276 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:30.268+0000 7f81323bd700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f812c1f7b20 con 0x7f8118003a30 2023-11-24T07:56:30.277 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:30.268+0000 7f81323bd700 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f812c15d560 con 0x7f812c15d0e0 2023-11-24T07:56:30.277 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:30.268+0000 7f81323bd700 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f812c15f110 con 0x7f812c09f200 2023-11-24T07:56:30.277 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:30.268+0000 7f812bfff700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8118003a30 0x7f812c1f7f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:56:30.277 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:30.268+0000 7f812bfff700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8118003a30 0x7f812c1f7f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:49846/0 (socket says 172.21.15.59:49846) 2023-11-24T07:56:30.277 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:30.268+0000 7f812bfff700 1 -- 172.21.15.59:0/1724824655 learned_addr learned my addr 172.21.15.59:0/1724824655 (peer_addr_for_me v2:172.21.15.59:0/0) 2023-11-24T07:56:30.277 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:30.268+0000 7f812b7fe700 1 --2- 172.21.15.59:0/1724824655 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f812c09f200 0x7f812c15cba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:56:30.277 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:30.268+0000 7f812bfff700 1 -- 172.21.15.59:0/1724824655 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f812c09f200 msgr2=0x7f812c15cba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:56:30.277 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:30.268+0000 7f813095a700 1 --2- 172.21.15.59:0/1724824655 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f812c15d0e0 0x7f812c161660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:56:30.277 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:30.268+0000 7f812bfff700 1 --2- 172.21.15.59:0/1724824655 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f812c09f200 0x7f812c15cba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:56:30.277 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:30.268+0000 7f812bfff700 1 -- 172.21.15.59:0/1724824655 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f812c15d0e0 msgr2=0x7f812c161660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:56:30.277 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:30.268+0000 7f812bfff700 1 --2- 172.21.15.59:0/1724824655 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f812c15d0e0 0x7f812c161660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:56:30.278 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:30.268+0000 7f812bfff700 1 -- 172.21.15.59:0/1724824655 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f812c162340 con 0x7f8118003a30 2023-11-24T07:56:30.278 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:30.268+0000 7f812b7fe700 1 --2- 172.21.15.59:0/1724824655 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f812c09f200 0x7f812c15cba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T07:56:30.278 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:30.269+0000 7f812bfff700 1 --2- 172.21.15.59:0/1724824655 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8118003a30 0x7f812c1f7f40 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f811c0011a0 tx=0x7f811c001110 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:56:30.278 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:30.269+0000 7f8128ff9700 1 -- 172.21.15.59:0/1724824655 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f811c001370 con 0x7f8118003a30 2023-11-24T07:56:30.278 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:30.269+0000 7f8128ff9700 1 -- 172.21.15.59:0/1724824655 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f811c00cc30 con 0x7f8118003a30 2023-11-24T07:56:30.278 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:30.269+0000 7f8128ff9700 1 -- 172.21.15.59:0/1724824655 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f811c003680 con 0x7f8118003a30 2023-11-24T07:56:30.278 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:30.269+0000 7f81323bd700 1 -- 172.21.15.59:0/1724824655 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f812c162620 con 0x7f8118003a30 2023-11-24T07:56:30.278 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:30.269+0000 7f81323bd700 1 -- 172.21.15.59:0/1724824655 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f812c1f7740 con 0x7f8118003a30 2023-11-24T07:56:30.279 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:30.270+0000 7f81323bd700 1 -- 172.21.15.59:0/1724824655 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f812c0c8930 con 0x7f8118003a30 2023-11-24T07:56:30.279 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:30.270+0000 7f8128ff9700 1 -- 172.21.15.59:0/1724824655 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f811c023070 con 0x7f8118003a30 2023-11-24T07:56:30.280 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:30.270+0000 7f8128ff9700 1 --2- 172.21.15.59:0/1724824655 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f810c06aed0 0x7f810c06d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:56:30.280 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:30.271+0000 7f8128ff9700 1 -- 172.21.15.59:0/1724824655 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f811c08aad0 con 0x7f8118003a30 2023-11-24T07:56:30.280 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:30.271+0000 7f812b7fe700 1 --2- 172.21.15.59:0/1724824655 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f810c06aed0 0x7f810c06d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:56:30.280 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:30.271+0000 7f812b7fe700 1 --2- 172.21.15.59:0/1724824655 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f810c06aed0 0x7f810c06d390 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f812c0b7a40 tx=0x7f8114008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:56:30.280 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:30.275+0000 7f8128ff9700 1 -- 172.21.15.59:0/1724824655 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f811c055cd0 con 0x7f8118003a30 2023-11-24T07:56:30.472 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:30.470+0000 7f81323bd700 1 -- 172.21.15.59:0/1724824655 --> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f812c15e5a0 con 0x7f810c06aed0 2023-11-24T07:56:30.480 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:30.478+0000 7f8128ff9700 1 -- 172.21.15.59:0/1724824655 <== mgr.4100 v2:172.21.15.76:6832/4264046536 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+264685 (secure 0 0 0) 0x7f81140137d0 con 0x7f810c06aed0 2023-11-24T07:56:30.484 INFO:teuthology.orchestra.run.smithi059.stdout: 2023-11-24T07:56:30.486 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:30.481+0000 7f81323bd700 1 -- 172.21.15.59:0/1724824655 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f810c06aed0 msgr2=0x7f810c06d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:56:30.486 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:30.481+0000 7f81323bd700 1 --2- 172.21.15.59:0/1724824655 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f810c06aed0 0x7f810c06d390 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f812c0b7a40 tx=0x7f8114008040 comp rx=0 tx=0).stop 2023-11-24T07:56:30.486 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:30.481+0000 7f81323bd700 1 -- 172.21.15.59:0/1724824655 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8118003a30 msgr2=0x7f812c1f7f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:56:30.486 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:30.481+0000 7f81323bd700 1 --2- 172.21.15.59:0/1724824655 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8118003a30 0x7f812c1f7f40 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f811c0011a0 tx=0x7f811c001110 comp rx=0 tx=0).stop 2023-11-24T07:56:30.487 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:30.481+0000 7f81323bd700 1 -- 172.21.15.59:0/1724824655 shutdown_connections 2023-11-24T07:56:30.487 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:30.481+0000 7f81323bd700 1 --2- 172.21.15.59:0/1724824655 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f810c06aed0 0x7f810c06d390 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:56:30.487 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:30.481+0000 7f81323bd700 1 --2- 172.21.15.59:0/1724824655 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f812c15d0e0 0x7f812c161660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:56:30.487 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:30.481+0000 7f81323bd700 1 --2- 172.21.15.59:0/1724824655 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8118003a30 0x7f812c1f7f40 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:56:30.487 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:30.481+0000 7f81323bd700 1 --2- 172.21.15.59:0/1724824655 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f812c09f200 0x7f812c15cba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:56:30.487 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:30.481+0000 7f81323bd700 1 -- 172.21.15.59:0/1724824655 >> 172.21.15.59:0/1724824655 conn(0x7f812c0b3720 msgr2=0x7f812c0b2730 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:56:30.487 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:30.482+0000 7f81323bd700 1 -- 172.21.15.59:0/1724824655 shutdown_connections 2023-11-24T07:56:30.487 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:30.482+0000 7f81323bd700 1 -- 172.21.15.59:0/1724824655 wait complete. 2023-11-24T07:56:30.487 INFO:teuthology.orchestra.run.smithi059.stderr:dumped all 2023-11-24T07:56:30.496 INFO:teuthology.orchestra.run.smithi059.stdout:{"pg_ready":true,"pg_map":{"version":19,"stamp":"2023-11-24T07:56:30.298587+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":168,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":218976,"kb_used_data":2800,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754755744,"statfs":{"total":773094113280,"available":772869881856,"internally_reserved":0,"allocated":2867200,"data_stored":1740236,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12745,"internal_metadata":221236791},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":33,"apply_latency_ms":33,"commit_latency_ns":33000000,"apply_latency_ns":33000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.611499"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T07:56:27.672077+0000","last_change":"2023-11-24T07:56:24.732130+0000","last_active":"2023-11-24T07:56:27.672077+0000","last_peered":"2023-11-24T07:56:27.672077+0000","last_clean":"2023-11-24T07:56:27.672077+0000","last_became_active":"2023-11-24T07:56:24.731726+0000","last_became_peered":"2023-11-24T07:56:24.731726+0000","last_unstale":"2023-11-24T07:56:27.672077+0000","last_undegraded":"2023-11-24T07:56:27.672077+0000","last_fullsized":"2023-11-24T07:56:27.672077+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:56:23.683366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:56:23.683366+0000","last_clean_scrub_stamp":"2023-11-24T07:56:23.683366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:52:22.451306+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T07:56:25.712721+0000","last_change":"2023-11-24T07:56:24.721147+0000","last_active":"2023-11-24T07:56:25.712721+0000","last_peered":"2023-11-24T07:56:25.712721+0000","last_clean":"2023-11-24T07:56:25.712721+0000","last_became_active":"2023-11-24T07:56:24.720857+0000","last_became_peered":"2023-11-24T07:56:24.720857+0000","last_unstale":"2023-11-24T07:56:25.712721+0000","last_undegraded":"2023-11-24T07:56:25.712721+0000","last_fullsized":"2023-11-24T07:56:25.712721+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:56:23.683366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:56:23.683366+0000","last_clean_scrub_stamp":"2023-11-24T07:56:23.683366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T19:48:41.181670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T07:56:25.711756+0000","last_change":"2023-11-24T07:56:24.726027+0000","last_active":"2023-11-24T07:56:25.711756+0000","last_peered":"2023-11-24T07:56:25.711756+0000","last_clean":"2023-11-24T07:56:25.711756+0000","last_became_active":"2023-11-24T07:56:24.725339+0000","last_became_peered":"2023-11-24T07:56:24.725339+0000","last_unstale":"2023-11-24T07:56:25.711756+0000","last_undegraded":"2023-11-24T07:56:25.711756+0000","last_fullsized":"2023-11-24T07:56:25.711756+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:56:23.683366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:56:23.683366+0000","last_clean_scrub_stamp":"2023-11-24T07:56:23.683366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T19:26:21.558298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T07:56:25.709012+0000","last_change":"2023-11-24T07:56:24.725132+0000","last_active":"2023-11-24T07:56:25.709012+0000","last_peered":"2023-11-24T07:56:25.709012+0000","last_clean":"2023-11-24T07:56:25.709012+0000","last_became_active":"2023-11-24T07:56:24.724817+0000","last_became_peered":"2023-11-24T07:56:24.724817+0000","last_unstale":"2023-11-24T07:56:25.709012+0000","last_undegraded":"2023-11-24T07:56:25.709012+0000","last_fullsized":"2023-11-24T07:56:25.709012+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:56:23.683366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:56:23.683366+0000","last_clean_scrub_stamp":"2023-11-24T07:56:23.683366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T19:32:53.988497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T07:56:27.671602+0000","last_change":"2023-11-24T07:56:24.722870+0000","last_active":"2023-11-24T07:56:27.671602+0000","last_peered":"2023-11-24T07:56:27.671602+0000","last_clean":"2023-11-24T07:56:27.671602+0000","last_became_active":"2023-11-24T07:56:24.722572+0000","last_became_peered":"2023-11-24T07:56:24.722572+0000","last_unstale":"2023-11-24T07:56:27.671602+0000","last_undegraded":"2023-11-24T07:56:27.671602+0000","last_fullsized":"2023-11-24T07:56:27.671602+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:56:23.683366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:56:23.683366+0000","last_clean_scrub_stamp":"2023-11-24T07:56:23.683366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T15:28:38.799061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T07:56:25.708767+0000","last_change":"2023-11-24T07:56:24.726061+0000","last_active":"2023-11-24T07:56:25.708767+0000","last_peered":"2023-11-24T07:56:25.708767+0000","last_clean":"2023-11-24T07:56:25.708767+0000","last_became_active":"2023-11-24T07:56:24.725750+0000","last_became_peered":"2023-11-24T07:56:24.725750+0000","last_unstale":"2023-11-24T07:56:25.708767+0000","last_undegraded":"2023-11-24T07:56:25.708767+0000","last_fullsized":"2023-11-24T07:56:25.708767+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:56:23.683366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:56:23.683366+0000","last_clean_scrub_stamp":"2023-11-24T07:56:23.683366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T16:22:33.714226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T07:56:27.671397+0000","last_change":"2023-11-24T07:56:24.725406+0000","last_active":"2023-11-24T07:56:27.671397+0000","last_peered":"2023-11-24T07:56:27.671397+0000","last_clean":"2023-11-24T07:56:27.671397+0000","last_became_active":"2023-11-24T07:56:24.725080+0000","last_became_peered":"2023-11-24T07:56:24.725080+0000","last_unstale":"2023-11-24T07:56:27.671397+0000","last_undegraded":"2023-11-24T07:56:27.671397+0000","last_fullsized":"2023-11-24T07:56:27.671397+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:56:23.683366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:56:23.683366+0000","last_clean_scrub_stamp":"2023-11-24T07:56:23.683366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T18:48:33.280136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T07:56:25.711711+0000","last_change":"2023-11-24T07:56:24.719072+0000","last_active":"2023-11-24T07:56:25.711711+0000","last_peered":"2023-11-24T07:56:25.711711+0000","last_clean":"2023-11-24T07:56:25.711711+0000","last_became_active":"2023-11-24T07:56:24.718791+0000","last_became_peered":"2023-11-24T07:56:24.718791+0000","last_unstale":"2023-11-24T07:56:25.711711+0000","last_undegraded":"2023-11-24T07:56:25.711711+0000","last_fullsized":"2023-11-24T07:56:25.711711+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:56:23.683366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:56:23.683366+0000","last_clean_scrub_stamp":"2023-11-24T07:56:23.683366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:13:16.865275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T07:56:27.783015+0000","last_change":"2023-11-24T07:56:24.743409+0000","last_active":"2023-11-24T07:56:27.783015+0000","last_peered":"2023-11-24T07:56:27.783015+0000","last_clean":"2023-11-24T07:56:27.783015+0000","last_became_active":"2023-11-24T07:56:24.742988+0000","last_became_peered":"2023-11-24T07:56:24.742988+0000","last_unstale":"2023-11-24T07:56:27.783015+0000","last_undegraded":"2023-11-24T07:56:27.783015+0000","last_fullsized":"2023-11-24T07:56:27.783015+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:56:23.683366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:56:23.683366+0000","last_clean_scrub_stamp":"2023-11-24T07:56:23.683366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T09:01:29.634179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T07:56:27.672028+0000","last_change":"2023-11-24T07:56:24.718066+0000","last_active":"2023-11-24T07:56:27.672028+0000","last_peered":"2023-11-24T07:56:27.672028+0000","last_clean":"2023-11-24T07:56:27.672028+0000","last_became_active":"2023-11-24T07:56:24.715725+0000","last_became_peered":"2023-11-24T07:56:24.715725+0000","last_unstale":"2023-11-24T07:56:27.672028+0000","last_undegraded":"2023-11-24T07:56:27.672028+0000","last_fullsized":"2023-11-24T07:56:27.672028+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:56:23.683366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:56:23.683366+0000","last_clean_scrub_stamp":"2023-11-24T07:56:23.683366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T13:56:07.967730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T07:56:27.671765+0000","last_change":"2023-11-24T07:56:24.730488+0000","last_active":"2023-11-24T07:56:27.671765+0000","last_peered":"2023-11-24T07:56:27.671765+0000","last_clean":"2023-11-24T07:56:27.671765+0000","last_became_active":"2023-11-24T07:56:24.730171+0000","last_became_peered":"2023-11-24T07:56:24.730171+0000","last_unstale":"2023-11-24T07:56:27.671765+0000","last_undegraded":"2023-11-24T07:56:27.671765+0000","last_fullsized":"2023-11-24T07:56:27.671765+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:56:23.683366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:56:23.683366+0000","last_clean_scrub_stamp":"2023-11-24T07:56:23.683366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T19:29:50.373581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T07:56:27.671956+0000","last_change":"2023-11-24T07:56:24.734314+0000","last_active":"2023-11-24T07:56:27.671956+0000","last_peered":"2023-11-24T07:56:27.671956+0000","last_clean":"2023-11-24T07:56:27.671956+0000","last_became_active":"2023-11-24T07:56:24.733966+0000","last_became_peered":"2023-11-24T07:56:24.733966+0000","last_unstale":"2023-11-24T07:56:27.671956+0000","last_undegraded":"2023-11-24T07:56:27.671956+0000","last_fullsized":"2023-11-24T07:56:27.671956+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:56:23.683366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:56:23.683366+0000","last_clean_scrub_stamp":"2023-11-24T07:56:23.683366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T19:39:29.828696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T07:56:27.782981+0000","last_change":"2023-11-24T07:56:24.717337+0000","last_active":"2023-11-24T07:56:27.782981+0000","last_peered":"2023-11-24T07:56:27.782981+0000","last_clean":"2023-11-24T07:56:27.782981+0000","last_became_active":"2023-11-24T07:56:24.716929+0000","last_became_peered":"2023-11-24T07:56:24.716929+0000","last_unstale":"2023-11-24T07:56:27.782981+0000","last_undegraded":"2023-11-24T07:56:27.782981+0000","last_fullsized":"2023-11-24T07:56:27.782981+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:56:23.683366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:56:23.683366+0000","last_clean_scrub_stamp":"2023-11-24T07:56:23.683366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T17:36:14.367398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T07:56:27.783087+0000","last_change":"2023-11-24T07:56:24.720081+0000","last_active":"2023-11-24T07:56:27.783087+0000","last_peered":"2023-11-24T07:56:27.783087+0000","last_clean":"2023-11-24T07:56:27.783087+0000","last_became_active":"2023-11-24T07:56:24.719776+0000","last_became_peered":"2023-11-24T07:56:24.719776+0000","last_unstale":"2023-11-24T07:56:27.783087+0000","last_undegraded":"2023-11-24T07:56:27.783087+0000","last_fullsized":"2023-11-24T07:56:27.783087+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:56:23.683366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:56:23.683366+0000","last_clean_scrub_stamp":"2023-11-24T07:56:23.683366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T19:43:40.193280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T07:56:25.712435+0000","last_change":"2023-11-24T07:56:24.719400+0000","last_active":"2023-11-24T07:56:25.712435+0000","last_peered":"2023-11-24T07:56:25.712435+0000","last_clean":"2023-11-24T07:56:25.712435+0000","last_became_active":"2023-11-24T07:56:24.719095+0000","last_became_peered":"2023-11-24T07:56:24.719095+0000","last_unstale":"2023-11-24T07:56:25.712435+0000","last_undegraded":"2023-11-24T07:56:25.712435+0000","last_fullsized":"2023-11-24T07:56:25.712435+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:56:23.683366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:56:23.683366+0000","last_clean_scrub_stamp":"2023-11-24T07:56:23.683366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T19:29:50.373581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T07:56:26.100997+0000","last_change":"2023-11-24T07:56:24.725927+0000","last_active":"2023-11-24T07:56:26.100997+0000","last_peered":"2023-11-24T07:56:26.100997+0000","last_clean":"2023-11-24T07:56:26.100997+0000","last_became_active":"2023-11-24T07:56:24.725568+0000","last_became_peered":"2023-11-24T07:56:24.725568+0000","last_unstale":"2023-11-24T07:56:26.100997+0000","last_undegraded":"2023-11-24T07:56:26.100997+0000","last_fullsized":"2023-11-24T07:56:26.100997+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:56:23.683366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:56:23.683366+0000","last_clean_scrub_stamp":"2023-11-24T07:56:23.683366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T19:39:29.828696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T07:56:26.101717+0000","last_change":"2023-11-24T07:56:24.730986+0000","last_active":"2023-11-24T07:56:26.101717+0000","last_peered":"2023-11-24T07:56:26.101717+0000","last_clean":"2023-11-24T07:56:26.101717+0000","last_became_active":"2023-11-24T07:56:24.730564+0000","last_became_peered":"2023-11-24T07:56:24.730564+0000","last_unstale":"2023-11-24T07:56:26.101717+0000","last_undegraded":"2023-11-24T07:56:26.101717+0000","last_fullsized":"2023-11-24T07:56:26.101717+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:56:23.683366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:56:23.683366+0000","last_clean_scrub_stamp":"2023-11-24T07:56:23.683366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T14:48:19.320894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T07:56:27.783028+0000","last_change":"2023-11-24T07:56:24.716131+0000","last_active":"2023-11-24T07:56:27.783028+0000","last_peered":"2023-11-24T07:56:27.783028+0000","last_clean":"2023-11-24T07:56:27.783028+0000","last_became_active":"2023-11-24T07:56:24.715770+0000","last_became_peered":"2023-11-24T07:56:24.715770+0000","last_unstale":"2023-11-24T07:56:27.783028+0000","last_undegraded":"2023-11-24T07:56:27.783028+0000","last_fullsized":"2023-11-24T07:56:27.783028+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:56:23.683366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:56:23.683366+0000","last_clean_scrub_stamp":"2023-11-24T07:56:23.683366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T16:22:33.714226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T07:56:26.101413+0000","last_change":"2023-11-24T07:56:24.726104+0000","last_active":"2023-11-24T07:56:26.101413+0000","last_peered":"2023-11-24T07:56:26.101413+0000","last_clean":"2023-11-24T07:56:26.101413+0000","last_became_active":"2023-11-24T07:56:24.725718+0000","last_became_peered":"2023-11-24T07:56:24.725718+0000","last_unstale":"2023-11-24T07:56:26.101413+0000","last_undegraded":"2023-11-24T07:56:26.101413+0000","last_fullsized":"2023-11-24T07:56:26.101413+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:56:23.683366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:56:23.683366+0000","last_clean_scrub_stamp":"2023-11-24T07:56:23.683366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T15:24:09.050735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"19'4","reported_seq":16,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-24T07:56:24.743886+0000","last_change":"2023-11-24T07:56:23.714550+0000","last_active":"2023-11-24T07:56:24.743886+0000","last_peered":"2023-11-24T07:56:24.743886+0000","last_clean":"2023-11-24T07:56:24.743886+0000","last_became_active":"2023-11-24T07:56:23.714275+0000","last_became_peered":"2023-11-24T07:56:23.714275+0000","last_unstale":"2023-11-24T07:56:24.743886+0000","last_undegraded":"2023-11-24T07:56:24.743886+0000","last_fullsized":"2023-11-24T07:56:24.743886+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:56:22.678019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:56:22.678019+0000","last_clean_scrub_stamp":"2023-11-24T07:56:22.678019+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T19:05:26.056275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T07:56:25.712605+0000","last_change":"2023-11-24T07:56:24.722991+0000","last_active":"2023-11-24T07:56:25.712605+0000","last_peered":"2023-11-24T07:56:25.712605+0000","last_clean":"2023-11-24T07:56:25.712605+0000","last_became_active":"2023-11-24T07:56:24.722711+0000","last_became_peered":"2023-11-24T07:56:24.722711+0000","last_unstale":"2023-11-24T07:56:25.712605+0000","last_undegraded":"2023-11-24T07:56:25.712605+0000","last_fullsized":"2023-11-24T07:56:25.712605+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:56:23.683366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:56:23.683366+0000","last_clean_scrub_stamp":"2023-11-24T07:56:23.683366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T12:41:22.436958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1e","version":"19'1","reported_seq":13,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-24T07:56:24.739921+0000","last_change":"2023-11-24T07:56:23.702219+0000","last_active":"2023-11-24T07:56:24.739921+0000","last_peered":"2023-11-24T07:56:24.739921+0000","last_clean":"2023-11-24T07:56:24.739921+0000","last_became_active":"2023-11-24T07:56:23.701922+0000","last_became_peered":"2023-11-24T07:56:23.701922+0000","last_unstale":"2023-11-24T07:56:24.739921+0000","last_undegraded":"2023-11-24T07:56:24.739921+0000","last_fullsized":"2023-11-24T07:56:24.739921+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:56:22.678019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:56:22.678019+0000","last_clean_scrub_stamp":"2023-11-24T07:56:22.678019+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T09:01:28.628833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T07:56:25.712243+0000","last_change":"2023-11-24T07:56:24.733713+0000","last_active":"2023-11-24T07:56:25.712243+0000","last_peered":"2023-11-24T07:56:25.712243+0000","last_clean":"2023-11-24T07:56:25.712243+0000","last_became_active":"2023-11-24T07:56:24.733302+0000","last_became_peered":"2023-11-24T07:56:24.733302+0000","last_unstale":"2023-11-24T07:56:25.712243+0000","last_undegraded":"2023-11-24T07:56:25.712243+0000","last_fullsized":"2023-11-24T07:56:25.712243+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:56:23.683366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:56:23.683366+0000","last_clean_scrub_stamp":"2023-11-24T07:56:23.683366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T15:28:38.799061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1d","version":"19'2","reported_seq":14,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-24T07:56:24.740403+0000","last_change":"2023-11-24T07:56:23.707439+0000","last_active":"2023-11-24T07:56:24.740403+0000","last_peered":"2023-11-24T07:56:24.740403+0000","last_clean":"2023-11-24T07:56:24.740403+0000","last_became_active":"2023-11-24T07:56:23.707136+0000","last_became_peered":"2023-11-24T07:56:23.707136+0000","last_unstale":"2023-11-24T07:56:24.740403+0000","last_undegraded":"2023-11-24T07:56:24.740403+0000","last_fullsized":"2023-11-24T07:56:24.740403+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:56:22.678019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:56:22.678019+0000","last_clean_scrub_stamp":"2023-11-24T07:56:22.678019+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T19:43:39.187934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T07:56:27.782857+0000","last_change":"2023-11-24T07:56:24.733325+0000","last_active":"2023-11-24T07:56:27.782857+0000","last_peered":"2023-11-24T07:56:27.782857+0000","last_clean":"2023-11-24T07:56:27.782857+0000","last_became_active":"2023-11-24T07:56:24.733014+0000","last_became_peered":"2023-11-24T07:56:24.733014+0000","last_unstale":"2023-11-24T07:56:27.782857+0000","last_undegraded":"2023-11-24T07:56:27.782857+0000","last_fullsized":"2023-11-24T07:56:27.782857+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:56:23.683366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:56:23.683366+0000","last_clean_scrub_stamp":"2023-11-24T07:56:23.683366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T19:43:40.193280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"19'2","reported_seq":14,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-24T07:56:24.740415+0000","last_change":"2023-11-24T07:56:23.709116+0000","last_active":"2023-11-24T07:56:24.740415+0000","last_peered":"2023-11-24T07:56:24.740415+0000","last_clean":"2023-11-24T07:56:24.740415+0000","last_became_active":"2023-11-24T07:56:23.708703+0000","last_became_peered":"2023-11-24T07:56:23.708703+0000","last_unstale":"2023-11-24T07:56:24.740415+0000","last_undegraded":"2023-11-24T07:56:24.740415+0000","last_fullsized":"2023-11-24T07:56:24.740415+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:56:22.678019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:56:22.678019+0000","last_clean_scrub_stamp":"2023-11-24T07:56:22.678019+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T09:01:28.628833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T07:56:26.101403+0000","last_change":"2023-11-24T07:56:24.731750+0000","last_active":"2023-11-24T07:56:26.101403+0000","last_peered":"2023-11-24T07:56:26.101403+0000","last_clean":"2023-11-24T07:56:26.101403+0000","last_became_active":"2023-11-24T07:56:24.730636+0000","last_became_peered":"2023-11-24T07:56:24.730636+0000","last_unstale":"2023-11-24T07:56:26.101403+0000","last_undegraded":"2023-11-24T07:56:26.101403+0000","last_fullsized":"2023-11-24T07:56:26.101403+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:56:23.683366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:56:23.683366+0000","last_clean_scrub_stamp":"2023-11-24T07:56:23.683366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T15:11:49.015783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1b","version":"19'1","reported_seq":13,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-24T07:56:24.737950+0000","last_change":"2023-11-24T07:56:23.702767+0000","last_active":"2023-11-24T07:56:24.737950+0000","last_peered":"2023-11-24T07:56:24.737950+0000","last_clean":"2023-11-24T07:56:24.737950+0000","last_became_active":"2023-11-24T07:56:23.702499+0000","last_became_peered":"2023-11-24T07:56:23.702499+0000","last_unstale":"2023-11-24T07:56:24.737950+0000","last_undegraded":"2023-11-24T07:56:24.737950+0000","last_fullsized":"2023-11-24T07:56:24.737950+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:56:22.678019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:56:22.678019+0000","last_clean_scrub_stamp":"2023-11-24T07:56:22.678019+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T17:36:13.362051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T07:56:27.782679+0000","last_change":"2023-11-24T07:56:24.735158+0000","last_active":"2023-11-24T07:56:27.782679+0000","last_peered":"2023-11-24T07:56:27.782679+0000","last_clean":"2023-11-24T07:56:27.782679+0000","last_became_active":"2023-11-24T07:56:24.734600+0000","last_became_peered":"2023-11-24T07:56:24.734600+0000","last_unstale":"2023-11-24T07:56:27.782679+0000","last_undegraded":"2023-11-24T07:56:27.782679+0000","last_fullsized":"2023-11-24T07:56:27.782679+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:56:23.683366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:56:23.683366+0000","last_clean_scrub_stamp":"2023-11-24T07:56:23.683366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T17:36:14.367398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1a","version":"19'1","reported_seq":13,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-24T07:56:24.743591+0000","last_change":"2023-11-24T07:56:23.712864+0000","last_active":"2023-11-24T07:56:24.743591+0000","last_peered":"2023-11-24T07:56:24.743591+0000","last_clean":"2023-11-24T07:56:24.743591+0000","last_became_active":"2023-11-24T07:56:23.712559+0000","last_became_peered":"2023-11-24T07:56:23.712559+0000","last_unstale":"2023-11-24T07:56:24.743591+0000","last_undegraded":"2023-11-24T07:56:24.743591+0000","last_fullsized":"2023-11-24T07:56:24.743591+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:56:22.678019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:56:22.678019+0000","last_clean_scrub_stamp":"2023-11-24T07:56:22.678019+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:19:21.093151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T07:56:25.712365+0000","last_change":"2023-11-24T07:56:24.727143+0000","last_active":"2023-11-24T07:56:25.712365+0000","last_peered":"2023-11-24T07:56:25.712365+0000","last_clean":"2023-11-24T07:56:25.712365+0000","last_became_active":"2023-11-24T07:56:24.726637+0000","last_became_peered":"2023-11-24T07:56:24.726637+0000","last_unstale":"2023-11-24T07:56:25.712365+0000","last_undegraded":"2023-11-24T07:56:25.712365+0000","last_fullsized":"2023-11-24T07:56:25.712365+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:56:23.683366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:56:23.683366+0000","last_clean_scrub_stamp":"2023-11-24T07:56:23.683366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T15:28:38.799061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":12,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-24T07:56:24.710275+0000","last_change":"2023-11-24T07:56:23.707863+0000","last_active":"2023-11-24T07:56:24.710275+0000","last_peered":"2023-11-24T07:56:24.710275+0000","last_clean":"2023-11-24T07:56:24.710275+0000","last_became_active":"2023-11-24T07:56:23.707595+0000","last_became_peered":"2023-11-24T07:56:23.707595+0000","last_unstale":"2023-11-24T07:56:24.710275+0000","last_undegraded":"2023-11-24T07:56:24.710275+0000","last_fullsized":"2023-11-24T07:56:24.710275+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:56:22.678019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:56:22.678019+0000","last_clean_scrub_stamp":"2023-11-24T07:56:22.678019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T11:00:03.947675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T07:56:25.712360+0000","last_change":"2023-11-24T07:56:24.721105+0000","last_active":"2023-11-24T07:56:25.712360+0000","last_peered":"2023-11-24T07:56:25.712360+0000","last_clean":"2023-11-24T07:56:25.712360+0000","last_became_active":"2023-11-24T07:56:24.720813+0000","last_became_peered":"2023-11-24T07:56:24.720813+0000","last_unstale":"2023-11-24T07:56:25.712360+0000","last_undegraded":"2023-11-24T07:56:25.712360+0000","last_fullsized":"2023-11-24T07:56:25.712360+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:56:23.683366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:56:23.683366+0000","last_clean_scrub_stamp":"2023-11-24T07:56:23.683366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T17:43:47.155869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":12,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-24T07:56:24.719929+0000","last_change":"2023-11-24T07:56:23.704771+0000","last_active":"2023-11-24T07:56:24.719929+0000","last_peered":"2023-11-24T07:56:24.719929+0000","last_clean":"2023-11-24T07:56:24.719929+0000","last_became_active":"2023-11-24T07:56:23.704433+0000","last_became_peered":"2023-11-24T07:56:23.704433+0000","last_unstale":"2023-11-24T07:56:24.719929+0000","last_undegraded":"2023-11-24T07:56:24.719929+0000","last_fullsized":"2023-11-24T07:56:24.719929+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:56:22.678019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:56:22.678019+0000","last_clean_scrub_stamp":"2023-11-24T07:56:22.678019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:14:32.121393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T07:56:25.711836+0000","last_change":"2023-11-24T07:56:24.734742+0000","last_active":"2023-11-24T07:56:25.711836+0000","last_peered":"2023-11-24T07:56:25.711836+0000","last_clean":"2023-11-24T07:56:25.711836+0000","last_became_active":"2023-11-24T07:56:24.734447+0000","last_became_peered":"2023-11-24T07:56:24.734447+0000","last_unstale":"2023-11-24T07:56:25.711836+0000","last_undegraded":"2023-11-24T07:56:25.711836+0000","last_fullsized":"2023-11-24T07:56:25.711836+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:56:23.683366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:56:23.683366+0000","last_clean_scrub_stamp":"2023-11-24T07:56:23.683366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T19:26:21.558298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":12,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-24T07:56:24.729505+0000","last_change":"2023-11-24T07:56:23.701888+0000","last_active":"2023-11-24T07:56:24.729505+0000","last_peered":"2023-11-24T07:56:24.729505+0000","last_clean":"2023-11-24T07:56:24.729505+0000","last_became_active":"2023-11-24T07:56:23.701611+0000","last_became_peered":"2023-11-24T07:56:23.701611+0000","last_unstale":"2023-11-24T07:56:24.729505+0000","last_undegraded":"2023-11-24T07:56:24.729505+0000","last_fullsized":"2023-11-24T07:56:24.729505+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:56:22.678019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:56:22.678019+0000","last_clean_scrub_stamp":"2023-11-24T07:56:22.678019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T18:53:13.845460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T07:56:25.709371+0000","last_change":"2023-11-24T07:56:24.731116+0000","last_active":"2023-11-24T07:56:25.709371+0000","last_peered":"2023-11-24T07:56:25.709371+0000","last_clean":"2023-11-24T07:56:25.709371+0000","last_became_active":"2023-11-24T07:56:24.730805+0000","last_became_peered":"2023-11-24T07:56:24.730805+0000","last_unstale":"2023-11-24T07:56:25.709371+0000","last_undegraded":"2023-11-24T07:56:25.709371+0000","last_fullsized":"2023-11-24T07:56:25.709371+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:56:23.683366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:56:23.683366+0000","last_clean_scrub_stamp":"2023-11-24T07:56:23.683366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T19:05:27.061622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":12,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-24T07:56:24.712027+0000","last_change":"2023-11-24T07:56:23.707470+0000","last_active":"2023-11-24T07:56:24.712027+0000","last_peered":"2023-11-24T07:56:24.712027+0000","last_clean":"2023-11-24T07:56:24.712027+0000","last_became_active":"2023-11-24T07:56:23.707194+0000","last_became_peered":"2023-11-24T07:56:23.707194+0000","last_unstale":"2023-11-24T07:56:24.712027+0000","last_undegraded":"2023-11-24T07:56:24.712027+0000","last_fullsized":"2023-11-24T07:56:24.712027+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:56:22.678019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:56:22.678019+0000","last_clean_scrub_stamp":"2023-11-24T07:56:22.678019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T19:26:20.552951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T07:56:25.712327+0000","last_change":"2023-11-24T07:56:24.733042+0000","last_active":"2023-11-24T07:56:25.712327+0000","last_peered":"2023-11-24T07:56:25.712327+0000","last_clean":"2023-11-24T07:56:25.712327+0000","last_became_active":"2023-11-24T07:56:24.732746+0000","last_became_peered":"2023-11-24T07:56:24.732746+0000","last_unstale":"2023-11-24T07:56:25.712327+0000","last_undegraded":"2023-11-24T07:56:25.712327+0000","last_fullsized":"2023-11-24T07:56:25.712327+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:56:23.683366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:56:23.683366+0000","last_clean_scrub_stamp":"2023-11-24T07:56:23.683366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:19:22.098498+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.15","version":"19'3","reported_seq":15,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-24T07:56:24.739732+0000","last_change":"2023-11-24T07:56:23.703208+0000","last_active":"2023-11-24T07:56:24.739732+0000","last_peered":"2023-11-24T07:56:24.739732+0000","last_clean":"2023-11-24T07:56:24.739732+0000","last_became_active":"2023-11-24T07:56:23.702891+0000","last_became_peered":"2023-11-24T07:56:23.702891+0000","last_unstale":"2023-11-24T07:56:24.739732+0000","last_undegraded":"2023-11-24T07:56:24.739732+0000","last_fullsized":"2023-11-24T07:56:24.739732+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:56:22.678019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:56:22.678019+0000","last_clean_scrub_stamp":"2023-11-24T07:56:22.678019+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T08:42:21.735516+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T07:56:26.101263+0000","last_change":"2023-11-24T07:56:24.729010+0000","last_active":"2023-11-24T07:56:26.101263+0000","last_peered":"2023-11-24T07:56:26.101263+0000","last_clean":"2023-11-24T07:56:26.101263+0000","last_became_active":"2023-11-24T07:56:24.728711+0000","last_became_peered":"2023-11-24T07:56:24.728711+0000","last_unstale":"2023-11-24T07:56:26.101263+0000","last_undegraded":"2023-11-24T07:56:26.101263+0000","last_fullsized":"2023-11-24T07:56:26.101263+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:56:23.683366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:56:23.683366+0000","last_clean_scrub_stamp":"2023-11-24T07:56:23.683366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T17:48:28.079331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.14","version":"19'6","reported_seq":18,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-24T07:56:24.743066+0000","last_change":"2023-11-24T07:56:23.706638+0000","last_active":"2023-11-24T07:56:24.743066+0000","last_peered":"2023-11-24T07:56:24.743066+0000","last_clean":"2023-11-24T07:56:24.743066+0000","last_became_active":"2023-11-24T07:56:23.706357+0000","last_became_peered":"2023-11-24T07:56:23.706357+0000","last_unstale":"2023-11-24T07:56:24.743066+0000","last_undegraded":"2023-11-24T07:56:24.743066+0000","last_fullsized":"2023-11-24T07:56:24.743066+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:56:22.678019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:56:22.678019+0000","last_clean_scrub_stamp":"2023-11-24T07:56:22.678019+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T14:40:38.179620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T07:56:27.671894+0000","last_change":"2023-11-24T07:56:24.717150+0000","last_active":"2023-11-24T07:56:27.671894+0000","last_peered":"2023-11-24T07:56:27.671894+0000","last_clean":"2023-11-24T07:56:27.671894+0000","last_became_active":"2023-11-24T07:56:24.716723+0000","last_became_peered":"2023-11-24T07:56:24.716723+0000","last_unstale":"2023-11-24T07:56:27.671894+0000","last_undegraded":"2023-11-24T07:56:27.671894+0000","last_fullsized":"2023-11-24T07:56:27.671894+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:56:23.683366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:56:23.683366+0000","last_clean_scrub_stamp":"2023-11-24T07:56:23.683366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T12:41:22.436958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.13","version":"19'2","reported_seq":14,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-24T07:56:24.739597+0000","last_change":"2023-11-24T07:56:23.707817+0000","last_active":"2023-11-24T07:56:24.739597+0000","last_peered":"2023-11-24T07:56:24.739597+0000","last_clean":"2023-11-24T07:56:24.739597+0000","last_became_active":"2023-11-24T07:56:23.707511+0000","last_became_peered":"2023-11-24T07:56:23.707511+0000","last_unstale":"2023-11-24T07:56:24.739597+0000","last_undegraded":"2023-11-24T07:56:24.739597+0000","last_fullsized":"2023-11-24T07:56:24.739597+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:56:22.678019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:56:22.678019+0000","last_clean_scrub_stamp":"2023-11-24T07:56:22.678019+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T15:03:38.755757+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T07:56:27.783173+0000","last_change":"2023-11-24T07:56:24.730834+0000","last_active":"2023-11-24T07:56:27.783173+0000","last_peered":"2023-11-24T07:56:27.783173+0000","last_clean":"2023-11-24T07:56:27.783173+0000","last_became_active":"2023-11-24T07:56:24.730545+0000","last_became_peered":"2023-11-24T07:56:24.730545+0000","last_unstale":"2023-11-24T07:56:27.783173+0000","last_undegraded":"2023-11-24T07:56:27.783173+0000","last_fullsized":"2023-11-24T07:56:27.783173+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:56:23.683366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:56:23.683366+0000","last_clean_scrub_stamp":"2023-11-24T07:56:23.683366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T18:20:49.373891+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":12,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-24T07:56:24.723510+0000","last_change":"2023-11-24T07:56:23.712182+0000","last_active":"2023-11-24T07:56:24.723510+0000","last_peered":"2023-11-24T07:56:24.723510+0000","last_clean":"2023-11-24T07:56:24.723510+0000","last_became_active":"2023-11-24T07:56:23.711814+0000","last_became_peered":"2023-11-24T07:56:23.711814+0000","last_unstale":"2023-11-24T07:56:24.723510+0000","last_undegraded":"2023-11-24T07:56:24.723510+0000","last_fullsized":"2023-11-24T07:56:24.723510+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:56:22.678019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:56:22.678019+0000","last_clean_scrub_stamp":"2023-11-24T07:56:22.678019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T15:43:53.652738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T07:56:26.101323+0000","last_change":"2023-11-24T07:56:24.730497+0000","last_active":"2023-11-24T07:56:26.101323+0000","last_peered":"2023-11-24T07:56:26.101323+0000","last_clean":"2023-11-24T07:56:26.101323+0000","last_became_active":"2023-11-24T07:56:24.730178+0000","last_became_peered":"2023-11-24T07:56:24.730178+0000","last_unstale":"2023-11-24T07:56:26.101323+0000","last_undegraded":"2023-11-24T07:56:26.101323+0000","last_fullsized":"2023-11-24T07:56:26.101323+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:56:23.683366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:56:23.683366+0000","last_clean_scrub_stamp":"2023-11-24T07:56:23.683366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T16:27:17.883314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":12,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-24T07:56:24.711899+0000","last_change":"2023-11-24T07:56:23.713877+0000","last_active":"2023-11-24T07:56:24.711899+0000","last_peered":"2023-11-24T07:56:24.711899+0000","last_clean":"2023-11-24T07:56:24.711899+0000","last_became_active":"2023-11-24T07:56:23.713603+0000","last_became_peered":"2023-11-24T07:56:23.713603+0000","last_unstale":"2023-11-24T07:56:24.711899+0000","last_undegraded":"2023-11-24T07:56:24.711899+0000","last_fullsized":"2023-11-24T07:56:24.711899+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:56:22.678019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:56:22.678019+0000","last_clean_scrub_stamp":"2023-11-24T07:56:22.678019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:13:15.859928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T07:56:27.782480+0000","last_change":"2023-11-24T07:56:24.733613+0000","last_active":"2023-11-24T07:56:27.782480+0000","last_peered":"2023-11-24T07:56:27.782480+0000","last_clean":"2023-11-24T07:56:27.782480+0000","last_became_active":"2023-11-24T07:56:24.733361+0000","last_became_peered":"2023-11-24T07:56:24.733361+0000","last_unstale":"2023-11-24T07:56:27.782480+0000","last_undegraded":"2023-11-24T07:56:27.782480+0000","last_fullsized":"2023-11-24T07:56:27.782480+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:56:23.683366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:56:23.683366+0000","last_clean_scrub_stamp":"2023-11-24T07:56:23.683366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T16:22:33.714226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.10","version":"19'8","reported_seq":20,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-24T07:56:24.740763+0000","last_change":"2023-11-24T07:56:23.701319+0000","last_active":"2023-11-24T07:56:24.740763+0000","last_peered":"2023-11-24T07:56:24.740763+0000","last_clean":"2023-11-24T07:56:24.740763+0000","last_became_active":"2023-11-24T07:56:23.700995+0000","last_became_peered":"2023-11-24T07:56:23.700995+0000","last_unstale":"2023-11-24T07:56:24.740763+0000","last_undegraded":"2023-11-24T07:56:24.740763+0000","last_fullsized":"2023-11-24T07:56:24.740763+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:56:22.678019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:56:22.678019+0000","last_clean_scrub_stamp":"2023-11-24T07:56:22.678019+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T19:43:39.187934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T07:56:26.101104+0000","last_change":"2023-11-24T07:56:24.730591+0000","last_active":"2023-11-24T07:56:26.101104+0000","last_peered":"2023-11-24T07:56:26.101104+0000","last_clean":"2023-11-24T07:56:26.101104+0000","last_became_active":"2023-11-24T07:56:24.730281+0000","last_became_peered":"2023-11-24T07:56:24.730281+0000","last_unstale":"2023-11-24T07:56:26.101104+0000","last_undegraded":"2023-11-24T07:56:26.101104+0000","last_fullsized":"2023-11-24T07:56:26.101104+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:56:23.683366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:56:23.683366+0000","last_clean_scrub_stamp":"2023-11-24T07:56:23.683366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:52:22.451306+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.f","version":"19'1","reported_seq":13,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-24T07:56:24.739965+0000","last_change":"2023-11-24T07:56:23.708274+0000","last_active":"2023-11-24T07:56:24.739965+0000","last_peered":"2023-11-24T07:56:24.739965+0000","last_clean":"2023-11-24T07:56:24.739965+0000","last_became_active":"2023-11-24T07:56:23.708001+0000","last_became_peered":"2023-11-24T07:56:23.708001+0000","last_unstale":"2023-11-24T07:56:24.739965+0000","last_undegraded":"2023-11-24T07:56:24.739965+0000","last_fullsized":"2023-11-24T07:56:24.739965+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:56:22.678019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:56:22.678019+0000","last_clean_scrub_stamp":"2023-11-24T07:56:22.678019+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T09:29:56.578230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T07:56:25.709280+0000","last_change":"2023-11-24T07:56:24.720989+0000","last_active":"2023-11-24T07:56:25.709280+0000","last_peered":"2023-11-24T07:56:25.709280+0000","last_clean":"2023-11-24T07:56:25.709280+0000","last_became_active":"2023-11-24T07:56:24.720500+0000","last_became_peered":"2023-11-24T07:56:24.720500+0000","last_unstale":"2023-11-24T07:56:25.709280+0000","last_undegraded":"2023-11-24T07:56:25.709280+0000","last_fullsized":"2023-11-24T07:56:25.709280+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:56:23.683366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:56:23.683366+0000","last_clean_scrub_stamp":"2023-11-24T07:56:23.683366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T09:01:29.634179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":12,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-24T07:56:24.714465+0000","last_change":"2023-11-24T07:56:23.704585+0000","last_active":"2023-11-24T07:56:24.714465+0000","last_peered":"2023-11-24T07:56:24.714465+0000","last_clean":"2023-11-24T07:56:24.714465+0000","last_became_active":"2023-11-24T07:56:23.704316+0000","last_became_peered":"2023-11-24T07:56:23.704316+0000","last_unstale":"2023-11-24T07:56:24.714465+0000","last_undegraded":"2023-11-24T07:56:24.714465+0000","last_fullsized":"2023-11-24T07:56:24.714465+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:56:22.678019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:56:22.678019+0000","last_clean_scrub_stamp":"2023-11-24T07:56:22.678019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T18:53:13.845460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T07:56:25.709031+0000","last_change":"2023-11-24T07:56:24.718689+0000","last_active":"2023-11-24T07:56:25.709031+0000","last_peered":"2023-11-24T07:56:25.709031+0000","last_clean":"2023-11-24T07:56:25.709031+0000","last_became_active":"2023-11-24T07:56:24.717862+0000","last_became_peered":"2023-11-24T07:56:24.717862+0000","last_unstale":"2023-11-24T07:56:25.709031+0000","last_undegraded":"2023-11-24T07:56:25.709031+0000","last_fullsized":"2023-11-24T07:56:25.709031+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:56:23.683366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:56:23.683366+0000","last_clean_scrub_stamp":"2023-11-24T07:56:23.683366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T17:36:14.367398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":12,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-24T07:56:24.723355+0000","last_change":"2023-11-24T07:56:23.712764+0000","last_active":"2023-11-24T07:56:24.723355+0000","last_peered":"2023-11-24T07:56:24.723355+0000","last_clean":"2023-11-24T07:56:24.723355+0000","last_became_active":"2023-11-24T07:56:23.712448+0000","last_became_peered":"2023-11-24T07:56:23.712448+0000","last_unstale":"2023-11-24T07:56:24.723355+0000","last_undegraded":"2023-11-24T07:56:24.723355+0000","last_fullsized":"2023-11-24T07:56:24.723355+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:56:22.678019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:56:22.678019+0000","last_clean_scrub_stamp":"2023-11-24T07:56:22.678019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T19:29:49.368234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T07:56:27.671634+0000","last_change":"2023-11-24T07:56:24.716237+0000","last_active":"2023-11-24T07:56:27.671634+0000","last_peered":"2023-11-24T07:56:27.671634+0000","last_clean":"2023-11-24T07:56:27.671634+0000","last_became_active":"2023-11-24T07:56:24.715897+0000","last_became_peered":"2023-11-24T07:56:24.715897+0000","last_unstale":"2023-11-24T07:56:27.671634+0000","last_undegraded":"2023-11-24T07:56:27.671634+0000","last_fullsized":"2023-11-24T07:56:27.671634+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:56:23.683366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:56:23.683366+0000","last_clean_scrub_stamp":"2023-11-24T07:56:23.683366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:19:22.098498+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T07:56:25.711564+0000","last_change":"2023-11-24T07:56:24.723045+0000","last_active":"2023-11-24T07:56:25.711564+0000","last_peered":"2023-11-24T07:56:25.711564+0000","last_clean":"2023-11-24T07:56:25.711564+0000","last_became_active":"2023-11-24T07:56:24.722669+0000","last_became_peered":"2023-11-24T07:56:24.722669+0000","last_unstale":"2023-11-24T07:56:25.711564+0000","last_undegraded":"2023-11-24T07:56:25.711564+0000","last_fullsized":"2023-11-24T07:56:25.711564+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:56:23.683366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:56:23.683366+0000","last_clean_scrub_stamp":"2023-11-24T07:56:23.683366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:14:33.126740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":12,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-24T07:56:24.710112+0000","last_change":"2023-11-24T07:56:23.702360+0000","last_active":"2023-11-24T07:56:24.710112+0000","last_peered":"2023-11-24T07:56:24.710112+0000","last_clean":"2023-11-24T07:56:24.710112+0000","last_became_active":"2023-11-24T07:56:23.701979+0000","last_became_peered":"2023-11-24T07:56:23.701979+0000","last_unstale":"2023-11-24T07:56:24.710112+0000","last_undegraded":"2023-11-24T07:56:24.710112+0000","last_fullsized":"2023-11-24T07:56:24.710112+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:56:22.678019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:56:22.678019+0000","last_clean_scrub_stamp":"2023-11-24T07:56:22.678019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T14:52:28.306044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T07:56:25.711765+0000","last_change":"2023-11-24T07:56:24.727041+0000","last_active":"2023-11-24T07:56:25.711765+0000","last_peered":"2023-11-24T07:56:25.711765+0000","last_clean":"2023-11-24T07:56:25.711765+0000","last_became_active":"2023-11-24T07:56:24.726344+0000","last_became_peered":"2023-11-24T07:56:24.726344+0000","last_unstale":"2023-11-24T07:56:25.711765+0000","last_undegraded":"2023-11-24T07:56:25.711765+0000","last_fullsized":"2023-11-24T07:56:25.711765+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:56:23.683366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:56:23.683366+0000","last_clean_scrub_stamp":"2023-11-24T07:56:23.683366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:13:16.865275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":12,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-24T07:56:24.718243+0000","last_change":"2023-11-24T07:56:23.712776+0000","last_active":"2023-11-24T07:56:24.718243+0000","last_peered":"2023-11-24T07:56:24.718243+0000","last_clean":"2023-11-24T07:56:24.718243+0000","last_became_active":"2023-11-24T07:56:23.712504+0000","last_became_peered":"2023-11-24T07:56:23.712504+0000","last_unstale":"2023-11-24T07:56:24.718243+0000","last_undegraded":"2023-11-24T07:56:24.718243+0000","last_fullsized":"2023-11-24T07:56:24.718243+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:56:22.678019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:56:22.678019+0000","last_clean_scrub_stamp":"2023-11-24T07:56:22.678019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T19:05:26.056275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":165,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-24T07:56:24.728424+0000","last_change":"2023-11-24T07:56:19.668905+0000","last_active":"2023-11-24T07:56:24.728424+0000","last_peered":"2023-11-24T07:56:24.728424+0000","last_clean":"2023-11-24T07:56:24.728424+0000","last_became_active":"2023-11-24T07:56:19.668601+0000","last_became_peered":"2023-11-24T07:56:19.668601+0000","last_unstale":"2023-11-24T07:56:24.728424+0000","last_undegraded":"2023-11-24T07:56:24.728424+0000","last_fullsized":"2023-11-24T07:56:24.728424+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:56:18.657844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:56:18.657844+0000","last_clean_scrub_stamp":"2023-11-24T07:56:18.657844+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T11:51:53.828455+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T07:56:27.783117+0000","last_change":"2023-11-24T07:56:24.716904+0000","last_active":"2023-11-24T07:56:27.783117+0000","last_peered":"2023-11-24T07:56:27.783117+0000","last_clean":"2023-11-24T07:56:27.783117+0000","last_became_active":"2023-11-24T07:56:24.716599+0000","last_became_peered":"2023-11-24T07:56:24.716599+0000","last_unstale":"2023-11-24T07:56:27.783117+0000","last_undegraded":"2023-11-24T07:56:27.783117+0000","last_fullsized":"2023-11-24T07:56:27.783117+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:56:23.683366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:56:23.683366+0000","last_clean_scrub_stamp":"2023-11-24T07:56:23.683366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T09:01:29.634179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":12,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-24T07:56:24.719998+0000","last_change":"2023-11-24T07:56:23.712140+0000","last_active":"2023-11-24T07:56:24.719998+0000","last_peered":"2023-11-24T07:56:24.719998+0000","last_clean":"2023-11-24T07:56:24.719998+0000","last_became_active":"2023-11-24T07:56:23.711855+0000","last_became_peered":"2023-11-24T07:56:23.711855+0000","last_unstale":"2023-11-24T07:56:24.719998+0000","last_undegraded":"2023-11-24T07:56:24.719998+0000","last_fullsized":"2023-11-24T07:56:24.719998+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:56:22.678019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:56:22.678019+0000","last_clean_scrub_stamp":"2023-11-24T07:56:22.678019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:13:15.859928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":12,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-24T07:56:24.722016+0000","last_change":"2023-11-24T07:56:23.712415+0000","last_active":"2023-11-24T07:56:24.722016+0000","last_peered":"2023-11-24T07:56:24.722016+0000","last_clean":"2023-11-24T07:56:24.722016+0000","last_became_active":"2023-11-24T07:56:23.712104+0000","last_became_peered":"2023-11-24T07:56:23.712104+0000","last_unstale":"2023-11-24T07:56:24.722016+0000","last_undegraded":"2023-11-24T07:56:24.722016+0000","last_fullsized":"2023-11-24T07:56:24.722016+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:56:22.678019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:56:22.678019+0000","last_clean_scrub_stamp":"2023-11-24T07:56:22.678019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T18:48:32.274789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T07:56:27.783252+0000","last_change":"2023-11-24T07:56:24.718851+0000","last_active":"2023-11-24T07:56:27.783252+0000","last_peered":"2023-11-24T07:56:27.783252+0000","last_clean":"2023-11-24T07:56:27.783252+0000","last_became_active":"2023-11-24T07:56:24.718574+0000","last_became_peered":"2023-11-24T07:56:24.718574+0000","last_unstale":"2023-11-24T07:56:27.783252+0000","last_undegraded":"2023-11-24T07:56:27.783252+0000","last_fullsized":"2023-11-24T07:56:27.783252+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:56:23.683366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:56:23.683366+0000","last_clean_scrub_stamp":"2023-11-24T07:56:23.683366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T14:36:27.615721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"20'1","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T07:56:25.711989+0000","last_change":"2023-11-24T07:56:23.708583+0000","last_active":"2023-11-24T07:56:25.711989+0000","last_peered":"2023-11-24T07:56:25.711989+0000","last_clean":"2023-11-24T07:56:25.711989+0000","last_became_active":"2023-11-24T07:56:23.708291+0000","last_became_peered":"2023-11-24T07:56:23.708291+0000","last_unstale":"2023-11-24T07:56:25.711989+0000","last_undegraded":"2023-11-24T07:56:25.711989+0000","last_fullsized":"2023-11-24T07:56:25.711989+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:56:22.678019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:56:22.678019+0000","last_clean_scrub_stamp":"2023-11-24T07:56:22.678019+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T18:20:48.368545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T07:56:25.712005+0000","last_change":"2023-11-24T07:56:24.725877+0000","last_active":"2023-11-24T07:56:25.712005+0000","last_peered":"2023-11-24T07:56:25.712005+0000","last_clean":"2023-11-24T07:56:25.712005+0000","last_became_active":"2023-11-24T07:56:24.725622+0000","last_became_peered":"2023-11-24T07:56:24.725622+0000","last_unstale":"2023-11-24T07:56:25.712005+0000","last_undegraded":"2023-11-24T07:56:25.712005+0000","last_fullsized":"2023-11-24T07:56:25.712005+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:56:23.683366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:56:23.683366+0000","last_clean_scrub_stamp":"2023-11-24T07:56:23.683366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T17:43:47.155869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"19'2","reported_seq":14,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-24T07:56:24.745752+0000","last_change":"2023-11-24T07:56:23.713967+0000","last_active":"2023-11-24T07:56:24.745752+0000","last_peered":"2023-11-24T07:56:24.745752+0000","last_clean":"2023-11-24T07:56:24.745752+0000","last_became_active":"2023-11-24T07:56:23.713711+0000","last_became_peered":"2023-11-24T07:56:23.713711+0000","last_unstale":"2023-11-24T07:56:24.745752+0000","last_undegraded":"2023-11-24T07:56:24.745752+0000","last_fullsized":"2023-11-24T07:56:24.745752+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:56:22.678019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:56:22.678019+0000","last_clean_scrub_stamp":"2023-11-24T07:56:22.678019+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T09:29:56.578230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T07:56:25.712076+0000","last_change":"2023-11-24T07:56:24.724396+0000","last_active":"2023-11-24T07:56:25.712076+0000","last_peered":"2023-11-24T07:56:25.712076+0000","last_clean":"2023-11-24T07:56:25.712076+0000","last_became_active":"2023-11-24T07:56:24.724093+0000","last_became_peered":"2023-11-24T07:56:24.724093+0000","last_unstale":"2023-11-24T07:56:25.712076+0000","last_undegraded":"2023-11-24T07:56:25.712076+0000","last_fullsized":"2023-11-24T07:56:25.712076+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:56:23.683366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:56:23.683366+0000","last_clean_scrub_stamp":"2023-11-24T07:56:23.683366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T18:48:33.280136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":12,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-24T07:56:24.709839+0000","last_change":"2023-11-24T07:56:23.712459+0000","last_active":"2023-11-24T07:56:24.709839+0000","last_peered":"2023-11-24T07:56:24.709839+0000","last_clean":"2023-11-24T07:56:24.709839+0000","last_became_active":"2023-11-24T07:56:23.712194+0000","last_became_peered":"2023-11-24T07:56:23.712194+0000","last_unstale":"2023-11-24T07:56:24.709839+0000","last_undegraded":"2023-11-24T07:56:24.709839+0000","last_fullsized":"2023-11-24T07:56:24.709839+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:56:22.678019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:56:22.678019+0000","last_clean_scrub_stamp":"2023-11-24T07:56:22.678019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T17:19:43.614472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T07:56:26.101174+0000","last_change":"2023-11-24T07:56:24.725437+0000","last_active":"2023-11-24T07:56:26.101174+0000","last_peered":"2023-11-24T07:56:26.101174+0000","last_clean":"2023-11-24T07:56:26.101174+0000","last_became_active":"2023-11-24T07:56:24.724898+0000","last_became_peered":"2023-11-24T07:56:24.724898+0000","last_unstale":"2023-11-24T07:56:26.101174+0000","last_undegraded":"2023-11-24T07:56:26.101174+0000","last_fullsized":"2023-11-24T07:56:26.101174+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:56:23.683366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:56:23.683366+0000","last_clean_scrub_stamp":"2023-11-24T07:56:23.683366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T14:42:07.025247+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":12,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-24T07:56:24.721117+0000","last_change":"2023-11-24T07:56:23.708117+0000","last_active":"2023-11-24T07:56:24.721117+0000","last_peered":"2023-11-24T07:56:24.721117+0000","last_clean":"2023-11-24T07:56:24.721117+0000","last_became_active":"2023-11-24T07:56:23.707774+0000","last_became_peered":"2023-11-24T07:56:23.707774+0000","last_unstale":"2023-11-24T07:56:24.721117+0000","last_undegraded":"2023-11-24T07:56:24.721117+0000","last_fullsized":"2023-11-24T07:56:24.721117+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:56:22.678019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:56:22.678019+0000","last_clean_scrub_stamp":"2023-11-24T07:56:22.678019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T09:01:28.628833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T07:56:25.711408+0000","last_change":"2023-11-24T07:56:24.723772+0000","last_active":"2023-11-24T07:56:25.711408+0000","last_peered":"2023-11-24T07:56:25.711408+0000","last_clean":"2023-11-24T07:56:25.711408+0000","last_became_active":"2023-11-24T07:56:24.723497+0000","last_became_peered":"2023-11-24T07:56:24.723497+0000","last_unstale":"2023-11-24T07:56:25.711408+0000","last_undegraded":"2023-11-24T07:56:25.711408+0000","last_fullsized":"2023-11-24T07:56:25.711408+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:56:23.683366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:56:23.683366+0000","last_clean_scrub_stamp":"2023-11-24T07:56:23.683366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:14:33.126740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":12,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-24T07:56:24.711586+0000","last_change":"2023-11-24T07:56:23.704407+0000","last_active":"2023-11-24T07:56:24.711586+0000","last_peered":"2023-11-24T07:56:24.711586+0000","last_clean":"2023-11-24T07:56:24.711586+0000","last_became_active":"2023-11-24T07:56:23.704097+0000","last_became_peered":"2023-11-24T07:56:23.704097+0000","last_unstale":"2023-11-24T07:56:24.711586+0000","last_undegraded":"2023-11-24T07:56:24.711586+0000","last_fullsized":"2023-11-24T07:56:24.711586+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:56:22.678019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:56:22.678019+0000","last_clean_scrub_stamp":"2023-11-24T07:56:22.678019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T14:40:38.179620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T07:56:27.783059+0000","last_change":"2023-11-24T07:56:24.716998+0000","last_active":"2023-11-24T07:56:27.783059+0000","last_peered":"2023-11-24T07:56:27.783059+0000","last_clean":"2023-11-24T07:56:27.783059+0000","last_became_active":"2023-11-24T07:56:24.716713+0000","last_became_peered":"2023-11-24T07:56:24.716713+0000","last_unstale":"2023-11-24T07:56:27.783059+0000","last_undegraded":"2023-11-24T07:56:27.783059+0000","last_fullsized":"2023-11-24T07:56:27.783059+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:56:23.683366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:56:23.683366+0000","last_clean_scrub_stamp":"2023-11-24T07:56:23.683366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T19:32:53.988497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.b","version":"19'2","reported_seq":14,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-24T07:56:24.745886+0000","last_change":"2023-11-24T07:56:23.712730+0000","last_active":"2023-11-24T07:56:24.745886+0000","last_peered":"2023-11-24T07:56:24.745886+0000","last_clean":"2023-11-24T07:56:24.745886+0000","last_became_active":"2023-11-24T07:56:23.712423+0000","last_became_peered":"2023-11-24T07:56:23.712423+0000","last_unstale":"2023-11-24T07:56:24.745886+0000","last_undegraded":"2023-11-24T07:56:24.745886+0000","last_fullsized":"2023-11-24T07:56:24.745886+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:56:22.678019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:56:22.678019+0000","last_clean_scrub_stamp":"2023-11-24T07:56:22.678019+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T14:36:26.610375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T07:56:26.101381+0000","last_change":"2023-11-24T07:56:24.739297+0000","last_active":"2023-11-24T07:56:26.101381+0000","last_peered":"2023-11-24T07:56:26.101381+0000","last_clean":"2023-11-24T07:56:26.101381+0000","last_became_active":"2023-11-24T07:56:24.738991+0000","last_became_peered":"2023-11-24T07:56:24.738991+0000","last_unstale":"2023-11-24T07:56:26.101381+0000","last_undegraded":"2023-11-24T07:56:26.101381+0000","last_fullsized":"2023-11-24T07:56:26.101381+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:56:23.683366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:56:23.683366+0000","last_clean_scrub_stamp":"2023-11-24T07:56:23.683366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T09:26:22.526827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":12,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-24T07:56:24.712584+0000","last_change":"2023-11-24T07:56:23.709703+0000","last_active":"2023-11-24T07:56:24.712584+0000","last_peered":"2023-11-24T07:56:24.712584+0000","last_clean":"2023-11-24T07:56:24.712584+0000","last_became_active":"2023-11-24T07:56:23.709360+0000","last_became_peered":"2023-11-24T07:56:23.709360+0000","last_unstale":"2023-11-24T07:56:24.712584+0000","last_undegraded":"2023-11-24T07:56:24.712584+0000","last_fullsized":"2023-11-24T07:56:24.712584+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:56:22.678019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:56:22.678019+0000","last_clean_scrub_stamp":"2023-11-24T07:56:22.678019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T16:22:32.708879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T07:56:26.101579+0000","last_change":"2023-11-24T07:56:24.739181+0000","last_active":"2023-11-24T07:56:26.101579+0000","last_peered":"2023-11-24T07:56:26.101579+0000","last_clean":"2023-11-24T07:56:26.101579+0000","last_became_active":"2023-11-24T07:56:24.738864+0000","last_became_peered":"2023-11-24T07:56:24.738864+0000","last_unstale":"2023-11-24T07:56:26.101579+0000","last_undegraded":"2023-11-24T07:56:26.101579+0000","last_fullsized":"2023-11-24T07:56:26.101579+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:56:23.683366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:56:23.683366+0000","last_clean_scrub_stamp":"2023-11-24T07:56:23.683366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T17:52:32.557707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.d","version":"19'1","reported_seq":13,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-24T07:56:24.740226+0000","last_change":"2023-11-24T07:56:23.696616+0000","last_active":"2023-11-24T07:56:24.740226+0000","last_peered":"2023-11-24T07:56:24.740226+0000","last_clean":"2023-11-24T07:56:24.740226+0000","last_became_active":"2023-11-24T07:56:23.696320+0000","last_became_peered":"2023-11-24T07:56:23.696320+0000","last_unstale":"2023-11-24T07:56:24.740226+0000","last_undegraded":"2023-11-24T07:56:24.740226+0000","last_fullsized":"2023-11-24T07:56:24.740226+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:56:22.678019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:56:22.678019+0000","last_clean_scrub_stamp":"2023-11-24T07:56:22.678019+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T17:19:43.614472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T07:56:27.671768+0000","last_change":"2023-11-24T07:56:24.716634+0000","last_active":"2023-11-24T07:56:27.671768+0000","last_peered":"2023-11-24T07:56:27.671768+0000","last_clean":"2023-11-24T07:56:27.671768+0000","last_became_active":"2023-11-24T07:56:24.716211+0000","last_became_peered":"2023-11-24T07:56:24.716211+0000","last_unstale":"2023-11-24T07:56:27.671768+0000","last_undegraded":"2023-11-24T07:56:27.671768+0000","last_fullsized":"2023-11-24T07:56:27.671768+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:56:23.683366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:56:23.683366+0000","last_clean_scrub_stamp":"2023-11-24T07:56:23.683366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T15:43:54.658085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"19'2","reported_seq":14,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-24T07:56:24.736924+0000","last_change":"2023-11-24T07:56:23.703304+0000","last_active":"2023-11-24T07:56:24.736924+0000","last_peered":"2023-11-24T07:56:24.736924+0000","last_clean":"2023-11-24T07:56:24.736924+0000","last_became_active":"2023-11-24T07:56:23.703016+0000","last_became_peered":"2023-11-24T07:56:23.703016+0000","last_unstale":"2023-11-24T07:56:24.736924+0000","last_undegraded":"2023-11-24T07:56:24.736924+0000","last_fullsized":"2023-11-24T07:56:24.736924+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:56:22.678019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:56:22.678019+0000","last_clean_scrub_stamp":"2023-11-24T07:56:22.678019+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T14:36:26.610375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T07:56:27.672202+0000","last_change":"2023-11-24T07:56:24.735087+0000","last_active":"2023-11-24T07:56:27.672202+0000","last_peered":"2023-11-24T07:56:27.672202+0000","last_clean":"2023-11-24T07:56:27.672202+0000","last_became_active":"2023-11-24T07:56:24.734770+0000","last_became_peered":"2023-11-24T07:56:24.734770+0000","last_unstale":"2023-11-24T07:56:27.672202+0000","last_undegraded":"2023-11-24T07:56:27.672202+0000","last_fullsized":"2023-11-24T07:56:27.672202+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:56:23.683366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:56:23.683366+0000","last_clean_scrub_stamp":"2023-11-24T07:56:23.683366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T14:46:45.906414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T07:56:25.708692+0000","last_change":"2023-11-24T07:56:24.719171+0000","last_active":"2023-11-24T07:56:25.708692+0000","last_peered":"2023-11-24T07:56:25.708692+0000","last_clean":"2023-11-24T07:56:25.708692+0000","last_became_active":"2023-11-24T07:56:24.718062+0000","last_became_peered":"2023-11-24T07:56:24.718062+0000","last_unstale":"2023-11-24T07:56:25.708692+0000","last_undegraded":"2023-11-24T07:56:25.708692+0000","last_fullsized":"2023-11-24T07:56:25.708692+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:56:23.683366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:56:23.683366+0000","last_clean_scrub_stamp":"2023-11-24T07:56:23.683366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T18:53:14.850807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T07:56:27.782576+0000","last_change":"2023-11-24T07:56:24.734652+0000","last_active":"2023-11-24T07:56:27.782576+0000","last_peered":"2023-11-24T07:56:27.782576+0000","last_clean":"2023-11-24T07:56:27.782576+0000","last_became_active":"2023-11-24T07:56:24.734348+0000","last_became_peered":"2023-11-24T07:56:24.734348+0000","last_unstale":"2023-11-24T07:56:27.782576+0000","last_undegraded":"2023-11-24T07:56:27.782576+0000","last_fullsized":"2023-11-24T07:56:27.782576+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:56:23.683366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:56:23.683366+0000","last_clean_scrub_stamp":"2023-11-24T07:56:23.683366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T18:53:14.850807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T07:56:26.101190+0000","last_change":"2023-11-24T07:56:24.731378+0000","last_active":"2023-11-24T07:56:26.101190+0000","last_peered":"2023-11-24T07:56:26.101190+0000","last_clean":"2023-11-24T07:56:26.101190+0000","last_became_active":"2023-11-24T07:56:24.731096+0000","last_became_peered":"2023-11-24T07:56:24.731096+0000","last_unstale":"2023-11-24T07:56:26.101190+0000","last_undegraded":"2023-11-24T07:56:26.101190+0000","last_fullsized":"2023-11-24T07:56:26.101190+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:56:23.683366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:56:23.683366+0000","last_clean_scrub_stamp":"2023-11-24T07:56:23.683366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T14:46:45.906414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T07:56:25.712218+0000","last_change":"2023-11-24T07:56:24.723023+0000","last_active":"2023-11-24T07:56:25.712218+0000","last_peered":"2023-11-24T07:56:25.712218+0000","last_clean":"2023-11-24T07:56:25.712218+0000","last_became_active":"2023-11-24T07:56:24.722725+0000","last_became_peered":"2023-11-24T07:56:24.722725+0000","last_unstale":"2023-11-24T07:56:25.712218+0000","last_undegraded":"2023-11-24T07:56:25.712218+0000","last_fullsized":"2023-11-24T07:56:25.712218+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:56:23.683366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:56:23.683366+0000","last_clean_scrub_stamp":"2023-11-24T07:56:23.683366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T18:48:33.280136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T07:56:27.783164+0000","last_change":"2023-11-24T07:56:24.717012+0000","last_active":"2023-11-24T07:56:27.783164+0000","last_peered":"2023-11-24T07:56:27.783164+0000","last_clean":"2023-11-24T07:56:27.783164+0000","last_became_active":"2023-11-24T07:56:24.716377+0000","last_became_peered":"2023-11-24T07:56:24.716377+0000","last_unstale":"2023-11-24T07:56:27.783164+0000","last_undegraded":"2023-11-24T07:56:27.783164+0000","last_fullsized":"2023-11-24T07:56:27.783164+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:56:23.683366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:56:23.683366+0000","last_clean_scrub_stamp":"2023-11-24T07:56:23.683366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T19:05:27.061622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T07:56:27.783295+0000","last_change":"2023-11-24T07:56:24.718395+0000","last_active":"2023-11-24T07:56:27.783295+0000","last_peered":"2023-11-24T07:56:27.783295+0000","last_clean":"2023-11-24T07:56:27.783295+0000","last_became_active":"2023-11-24T07:56:24.718132+0000","last_became_peered":"2023-11-24T07:56:24.718132+0000","last_unstale":"2023-11-24T07:56:27.783295+0000","last_undegraded":"2023-11-24T07:56:27.783295+0000","last_fullsized":"2023-11-24T07:56:27.783295+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:56:23.683366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:56:23.683366+0000","last_clean_scrub_stamp":"2023-11-24T07:56:23.683366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:45:41.378607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T07:56:25.709271+0000","last_change":"2023-11-24T07:56:24.731485+0000","last_active":"2023-11-24T07:56:25.709271+0000","last_peered":"2023-11-24T07:56:25.709271+0000","last_clean":"2023-11-24T07:56:25.709271+0000","last_became_active":"2023-11-24T07:56:24.731163+0000","last_became_peered":"2023-11-24T07:56:24.731163+0000","last_unstale":"2023-11-24T07:56:25.709271+0000","last_undegraded":"2023-11-24T07:56:25.709271+0000","last_fullsized":"2023-11-24T07:56:25.709271+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:56:23.683366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:56:23.683366+0000","last_clean_scrub_stamp":"2023-11-24T07:56:23.683366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T19:43:40.193280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T07:56:25.712117+0000","last_change":"2023-11-24T07:56:24.733823+0000","last_active":"2023-11-24T07:56:25.712117+0000","last_peered":"2023-11-24T07:56:25.712117+0000","last_clean":"2023-11-24T07:56:25.712117+0000","last_became_active":"2023-11-24T07:56:24.733550+0000","last_became_peered":"2023-11-24T07:56:24.733550+0000","last_unstale":"2023-11-24T07:56:25.712117+0000","last_undegraded":"2023-11-24T07:56:25.712117+0000","last_fullsized":"2023-11-24T07:56:25.712117+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:56:23.683366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:56:23.683366+0000","last_clean_scrub_stamp":"2023-11-24T07:56:23.683366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T09:29:57.583576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T07:56:26.101006+0000","last_change":"2023-11-24T07:56:24.728674+0000","last_active":"2023-11-24T07:56:26.101006+0000","last_peered":"2023-11-24T07:56:26.101006+0000","last_clean":"2023-11-24T07:56:26.101006+0000","last_became_active":"2023-11-24T07:56:24.728157+0000","last_became_peered":"2023-11-24T07:56:24.728157+0000","last_unstale":"2023-11-24T07:56:26.101006+0000","last_undegraded":"2023-11-24T07:56:26.101006+0000","last_fullsized":"2023-11-24T07:56:26.101006+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:56:23.683366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:56:23.683366+0000","last_clean_scrub_stamp":"2023-11-24T07:56:23.683366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T13:56:07.967730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T07:56:25.712536+0000","last_change":"2023-11-24T07:56:24.722224+0000","last_active":"2023-11-24T07:56:25.712536+0000","last_peered":"2023-11-24T07:56:25.712536+0000","last_clean":"2023-11-24T07:56:25.712536+0000","last_became_active":"2023-11-24T07:56:24.721954+0000","last_became_peered":"2023-11-24T07:56:24.721954+0000","last_unstale":"2023-11-24T07:56:25.712536+0000","last_undegraded":"2023-11-24T07:56:25.712536+0000","last_fullsized":"2023-11-24T07:56:25.712536+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:56:23.683366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:56:23.683366+0000","last_clean_scrub_stamp":"2023-11-24T07:56:23.683366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T15:43:54.658085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T07:56:27.782867+0000","last_change":"2023-11-24T07:56:24.734996+0000","last_active":"2023-11-24T07:56:27.782867+0000","last_peered":"2023-11-24T07:56:27.782867+0000","last_clean":"2023-11-24T07:56:27.782867+0000","last_became_active":"2023-11-24T07:56:24.734198+0000","last_became_peered":"2023-11-24T07:56:24.734198+0000","last_unstale":"2023-11-24T07:56:27.782867+0000","last_undegraded":"2023-11-24T07:56:27.782867+0000","last_fullsized":"2023-11-24T07:56:27.782867+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:56:23.683366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:56:23.683366+0000","last_clean_scrub_stamp":"2023-11-24T07:56:23.683366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T19:32:53.988497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T07:56:25.712025+0000","last_change":"2023-11-24T07:56:24.727463+0000","last_active":"2023-11-24T07:56:25.712025+0000","last_peered":"2023-11-24T07:56:25.712025+0000","last_clean":"2023-11-24T07:56:25.712025+0000","last_became_active":"2023-11-24T07:56:24.727033+0000","last_became_peered":"2023-11-24T07:56:24.727033+0000","last_unstale":"2023-11-24T07:56:25.712025+0000","last_undegraded":"2023-11-24T07:56:25.712025+0000","last_fullsized":"2023-11-24T07:56:25.712025+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:56:23.683366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:56:23.683366+0000","last_clean_scrub_stamp":"2023-11-24T07:56:23.683366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T09:29:57.583576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T07:56:27.671926+0000","last_change":"2023-11-24T07:56:24.734645+0000","last_active":"2023-11-24T07:56:27.671926+0000","last_peered":"2023-11-24T07:56:27.671926+0000","last_clean":"2023-11-24T07:56:27.671926+0000","last_became_active":"2023-11-24T07:56:24.734367+0000","last_became_peered":"2023-11-24T07:56:24.734367+0000","last_unstale":"2023-11-24T07:56:27.671926+0000","last_undegraded":"2023-11-24T07:56:27.671926+0000","last_fullsized":"2023-11-24T07:56:27.671926+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:56:23.683366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:56:23.683366+0000","last_clean_scrub_stamp":"2023-11-24T07:56:23.683366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T19:48:41.181670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574852,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27216,"kb_used_data":196,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344624,"statfs":{"total":96636764160,"available":96608894976,"internally_reserved":0,"allocated":200704,"data_stored":64734,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48899999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.33600000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.35799999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.38500000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52100000000000002}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55000000000000004}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59299999999999997}]}]},{"osd":6,"up_from":13,"seq":55834574853,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27208,"kb_used_data":188,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344632,"statfs":{"total":96636764160,"available":96608903168,"internally_reserved":0,"allocated":192512,"data_stored":64610,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.45200000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.43099999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58999999999999997}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57099999999999995}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67800000000000005}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64500000000000002}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66400000000000003}]}]},{"osd":1,"up_from":13,"seq":55834574854,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27804,"kb_used_data":784,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344036,"statfs":{"total":96636764160,"available":96608292864,"internally_reserved":0,"allocated":802816,"data_stored":661348,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.433}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.36699999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46000000000000002}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64500000000000002}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66800000000000004}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61299999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.627}]}]},{"osd":0,"up_from":13,"seq":55834574854,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27228,"kb_used_data":208,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344612,"statfs":{"total":96636764160,"available":96608882688,"internally_reserved":0,"allocated":212992,"data_stored":70966,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.439}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.35799999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.373}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.69799999999999995}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.70999999999999996}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.72799999999999998}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.74399999999999999}]}]},{"osd":2,"up_from":13,"seq":55834574854,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27236,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344604,"statfs":{"total":96636764160,"available":96608874496,"internally_reserved":0,"allocated":217088,"data_stored":72522,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46000000000000002}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.35999999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.442}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.626}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55500000000000005}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.76500000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58899999999999997}]}]},{"osd":3,"up_from":13,"seq":55834574854,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27228,"kb_used_data":204,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344612,"statfs":{"total":96636764160,"available":96608882688,"internally_reserved":0,"allocated":208896,"data_stored":71492,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54600000000000004}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.50600000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.78700000000000003}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.80600000000000005}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.81899999999999995}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.74099999999999999}]}]},{"osd":4,"up_from":13,"seq":55834574853,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27244,"kb_used_data":220,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344596,"statfs":{"total":96636764160,"available":96608866304,"internally_reserved":0,"allocated":225280,"data_stored":73194,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.72099999999999997}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59199999999999997}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68899999999999995}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.79600000000000004}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65100000000000002}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.76800000000000002}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.89300000000000002}]}]},{"osd":5,"up_from":13,"seq":55834574854,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27812,"kb_used_data":788,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344028,"statfs":{"total":96636764160,"available":96608284672,"internally_reserved":0,"allocated":806912,"data_stored":661370,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60199999999999998}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51400000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66400000000000003}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64500000000000002}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.38200000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.41599999999999998}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53200000000000003}]}]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":2284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":124,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-11-24T07:56:33.503 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-11-24T07:56:33.641 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:33.639+0000 7f1040212700 1 Processor -- start 2023-11-24T07:56:33.641 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:33.640+0000 7f1040212700 1 -- start start 2023-11-24T07:56:33.643 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:33.640+0000 7f1040212700 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f103815bde0 0x7f103815c1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:56:33.644 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:33.640+0000 7f1040212700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f10380ce5c0 0x7f10380cea30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:56:33.644 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:33.640+0000 7f1040212700 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f10380d0a90 con 0x7f10380cf2b0 2023-11-24T07:56:33.644 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:33.640+0000 7f1040212700 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f10380cef70 con 0x7f10380ce5c0 2023-11-24T07:56:33.644 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:33.640+0000 7f1040212700 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f103806ee10 con 0x7f103815bde0 2023-11-24T07:56:33.644 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:33.640+0000 7f103e7af700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f10380ce5c0 0x7f10380cea30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:56:33.644 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:33.641+0000 7f103e7af700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f10380ce5c0 0x7f10380cea30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:49858/0 (socket says 172.21.15.59:49858) 2023-11-24T07:56:33.644 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:33.641+0000 7f103e7af700 1 -- 172.21.15.59:0/617519048 learned_addr learned my addr 172.21.15.59:0/617519048 (peer_addr_for_me v2:172.21.15.59:0/0) 2023-11-24T07:56:33.644 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:33.641+0000 7f103dfae700 1 --2- 172.21.15.59:0/617519048 >> v2:172.21.15.76:3300/0 conn(0x7f103815bde0 0x7f103815c1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:56:33.645 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:33.641+0000 7f103e7af700 1 -- 172.21.15.59:0/617519048 >> v2:172.21.15.76:3300/0 conn(0x7f103815bde0 msgr2=0x7f103815c1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T07:56:33.645 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:33.641+0000 7f103e7af700 1 --2- 172.21.15.59:0/617519048 >> v2:172.21.15.76:3300/0 conn(0x7f103815bde0 0x7f103815c1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:56:33.645 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:33.641+0000 7f103e7af700 1 -- 172.21.15.59:0/617519048 >> v1:172.21.15.76:6789/0 conn(0x7f10380cf2b0 legacy=0x7f10380d01b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:56:33.645 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:33.641+0000 7f103e7af700 1 -- 172.21.15.59:0/617519048 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f103806f0f0 con 0x7f10380ce5c0 2023-11-24T07:56:33.645 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:33.641+0000 7f103e7af700 1 --2- 172.21.15.59:0/617519048 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f10380ce5c0 0x7f10380cea30 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f1038072370 tx=0x7f103000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5102ddfc626dba2f server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:56:33.645 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:33.641+0000 7f103cfac700 1 -- 172.21.15.59:0/617519048 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f103000e070 con 0x7f10380ce5c0 2023-11-24T07:56:33.645 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:33.641+0000 7f103cfac700 1 -- 172.21.15.59:0/617519048 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f103000b9c0 con 0x7f10380ce5c0 2023-11-24T07:56:33.645 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:33.641+0000 7f103cfac700 1 -- 172.21.15.59:0/617519048 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f103000bd70 con 0x7f10380ce5c0 2023-11-24T07:56:33.645 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:33.642+0000 7f1040212700 1 -- 172.21.15.59:0/617519048 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f10380ce5c0 msgr2=0x7f10380cea30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:56:33.645 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:33.642+0000 7f1040212700 1 --2- 172.21.15.59:0/617519048 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f10380ce5c0 0x7f10380cea30 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f1038072370 tx=0x7f103000b330 comp rx=0 tx=0).stop 2023-11-24T07:56:33.645 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:33.642+0000 7f1040212700 1 -- 172.21.15.59:0/617519048 shutdown_connections 2023-11-24T07:56:33.646 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:33.642+0000 7f1040212700 1 --2- 172.21.15.59:0/617519048 >> v2:172.21.15.76:3300/0 conn(0x7f103815bde0 0x7f103815c1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:56:33.646 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:33.642+0000 7f1040212700 1 --2- 172.21.15.59:0/617519048 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f10380ce5c0 0x7f10380cea30 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:56:33.646 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:33.642+0000 7f1040212700 1 -- 172.21.15.59:0/617519048 >> 172.21.15.59:0/617519048 conn(0x7f10380b3720 msgr2=0x7f10380b3b30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:56:33.646 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:33.642+0000 7f1040212700 1 -- 172.21.15.59:0/617519048 shutdown_connections 2023-11-24T07:56:33.646 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:33.642+0000 7f1040212700 1 -- 172.21.15.59:0/617519048 wait complete. 2023-11-24T07:56:33.648 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:33.643+0000 7f1040212700 1 Processor -- start 2023-11-24T07:56:33.648 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:33.643+0000 7f1040212700 1 -- start start 2023-11-24T07:56:33.648 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:33.644+0000 7f1040212700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f10380ce5c0 0x7f10380c8ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:56:33.649 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:33.644+0000 7f1040212700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f10380cf2b0 0x7f103814f3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:56:33.649 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:33.644+0000 7f1040212700 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f103815bde0 0x7f10380b7020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:56:33.649 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:33.644+0000 7f1040212700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f103815d420 con 0x7f10380cf2b0 2023-11-24T07:56:33.649 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:33.644+0000 7f1040212700 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f103815d700 con 0x7f10380ce5c0 2023-11-24T07:56:33.649 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:33.644+0000 7f1040212700 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f103815d9e0 con 0x7f103815bde0 2023-11-24T07:56:33.649 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:33.644+0000 7f103d7ad700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f10380cf2b0 0x7f103814f3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:56:33.649 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:33.644+0000 7f103dfae700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f10380ce5c0 0x7f10380c8ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:56:33.649 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:33.644+0000 7f103d7ad700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f10380cf2b0 0x7f103814f3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:49862/0 (socket says 172.21.15.59:49862) 2023-11-24T07:56:33.649 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:33.644+0000 7f103d7ad700 1 -- 172.21.15.59:0/3537196935 learned_addr learned my addr 172.21.15.59:0/3537196935 (peer_addr_for_me v2:172.21.15.59:0/0) 2023-11-24T07:56:33.649 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:33.644+0000 7f103e7af700 1 --2- 172.21.15.59:0/3537196935 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f103815bde0 0x7f10380b7020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:56:33.649 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:33.644+0000 7f103d7ad700 1 -- 172.21.15.59:0/3537196935 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f103815bde0 msgr2=0x7f10380b7020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:56:33.650 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:33.644+0000 7f103d7ad700 1 --2- 172.21.15.59:0/3537196935 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f103815bde0 0x7f10380b7020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:56:33.650 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:33.644+0000 7f103d7ad700 1 -- 172.21.15.59:0/3537196935 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f10380ce5c0 msgr2=0x7f10380c8ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:56:33.650 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:33.644+0000 7f103d7ad700 1 --2- 172.21.15.59:0/3537196935 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f10380ce5c0 0x7f10380c8ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:56:33.650 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:33.644+0000 7f103d7ad700 1 -- 172.21.15.59:0/3537196935 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1030009b30 con 0x7f10380cf2b0 2023-11-24T07:56:33.650 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:33.644+0000 7f103d7ad700 1 --2- 172.21.15.59:0/3537196935 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f10380cf2b0 0x7f103814f3d0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f103400c4b0 tx=0x7f103400c7c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:56:33.650 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:33.645+0000 7f102e7fc700 1 -- 172.21.15.59:0/3537196935 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10340052d0 con 0x7f10380cf2b0 2023-11-24T07:56:33.650 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:33.645+0000 7f102e7fc700 1 -- 172.21.15.59:0/3537196935 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1034005430 con 0x7f10380cf2b0 2023-11-24T07:56:33.650 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:33.645+0000 7f1040212700 1 -- 172.21.15.59:0/3537196935 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f103815dcc0 con 0x7f10380cf2b0 2023-11-24T07:56:33.650 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:33.645+0000 7f102e7fc700 1 -- 172.21.15.59:0/3537196935 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10340038c0 con 0x7f10380cf2b0 2023-11-24T07:56:33.650 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:33.645+0000 7f1040212700 1 -- 172.21.15.59:0/3537196935 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f10380c91e0 con 0x7f10380cf2b0 2023-11-24T07:56:33.650 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:33.646+0000 7f1040212700 1 -- 172.21.15.59:0/3537196935 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1038068670 con 0x7f10380cf2b0 2023-11-24T07:56:33.650 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:33.646+0000 7f102e7fc700 1 -- 172.21.15.59:0/3537196935 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f1034003a20 con 0x7f10380cf2b0 2023-11-24T07:56:33.651 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:33.647+0000 7f102e7fc700 1 --2- 172.21.15.59:0/3537196935 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f101c06aed0 0x7f101c06d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:56:33.651 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:33.647+0000 7f102e7fc700 1 -- 172.21.15.59:0/3537196935 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f1034088920 con 0x7f10380cf2b0 2023-11-24T07:56:33.651 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:33.647+0000 7f103dfae700 1 --2- 172.21.15.59:0/3537196935 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f101c06aed0 0x7f101c06d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:56:33.651 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:33.647+0000 7f103dfae700 1 --2- 172.21.15.59:0/3537196935 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f101c06aed0 0x7f101c06d390 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f1028007a30 tx=0x7f1028008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:56:33.653 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:33.651+0000 7f102e7fc700 1 -- 172.21.15.59:0/3537196935 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f10340539d0 con 0x7f10380cf2b0 2023-11-24T07:56:33.850 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:33.849+0000 7f1040212700 1 -- 172.21.15.59:0/3537196935 --> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f103806f6b0 con 0x7f101c06aed0 2023-11-24T07:56:33.858 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:33.857+0000 7f102e7fc700 1 -- 172.21.15.59:0/3537196935 <== mgr.4100 v2:172.21.15.76:6832/4264046536 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+264685 (secure 0 0 0) 0x7f1028010740 con 0x7f101c06aed0 2023-11-24T07:56:33.861 INFO:teuthology.orchestra.run.smithi059.stdout: 2023-11-24T07:56:33.863 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:33.860+0000 7f1040212700 1 -- 172.21.15.59:0/3537196935 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f101c06aed0 msgr2=0x7f101c06d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:56:33.863 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:33.860+0000 7f1040212700 1 --2- 172.21.15.59:0/3537196935 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f101c06aed0 0x7f101c06d390 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f1028007a30 tx=0x7f1028008040 comp rx=0 tx=0).stop 2023-11-24T07:56:33.864 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:33.860+0000 7f1040212700 1 -- 172.21.15.59:0/3537196935 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f10380cf2b0 msgr2=0x7f103814f3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:56:33.864 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:33.860+0000 7f1040212700 1 --2- 172.21.15.59:0/3537196935 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f10380cf2b0 0x7f103814f3d0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f103400c4b0 tx=0x7f103400c7c0 comp rx=0 tx=0).stop 2023-11-24T07:56:33.864 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:33.860+0000 7f1040212700 1 -- 172.21.15.59:0/3537196935 shutdown_connections 2023-11-24T07:56:33.864 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:33.860+0000 7f1040212700 1 --2- 172.21.15.59:0/3537196935 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f101c06aed0 0x7f101c06d390 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:56:33.864 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:33.860+0000 7f1040212700 1 --2- 172.21.15.59:0/3537196935 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f103815bde0 0x7f10380b7020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:56:33.864 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:33.860+0000 7f1040212700 1 --2- 172.21.15.59:0/3537196935 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f10380ce5c0 0x7f10380c8ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:56:33.864 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:33.860+0000 7f1040212700 1 --2- 172.21.15.59:0/3537196935 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f10380cf2b0 0x7f103814f3d0 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:56:33.864 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:33.860+0000 7f1040212700 1 -- 172.21.15.59:0/3537196935 >> 172.21.15.59:0/3537196935 conn(0x7f10380b3720 msgr2=0x7f10380b3b00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:56:33.864 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:33.860+0000 7f1040212700 1 -- 172.21.15.59:0/3537196935 shutdown_connections 2023-11-24T07:56:33.864 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:33.860+0000 7f1040212700 1 -- 172.21.15.59:0/3537196935 wait complete. 2023-11-24T07:56:33.864 INFO:teuthology.orchestra.run.smithi059.stderr:dumped all 2023-11-24T07:56:33.874 INFO:teuthology.orchestra.run.smithi059.stdout:{"pg_ready":true,"pg_map":{"version":20,"stamp":"2023-11-24T07:56:32.299351+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":168,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":218976,"kb_used_data":2800,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754755744,"statfs":{"total":773094113280,"available":772869881856,"internally_reserved":0,"allocated":2867200,"data_stored":1740236,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12745,"internal_metadata":221236791},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":33,"apply_latency_ms":33,"commit_latency_ns":33000000,"apply_latency_ns":33000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.002207"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T07:56:27.672077+0000","last_change":"2023-11-24T07:56:24.732130+0000","last_active":"2023-11-24T07:56:27.672077+0000","last_peered":"2023-11-24T07:56:27.672077+0000","last_clean":"2023-11-24T07:56:27.672077+0000","last_became_active":"2023-11-24T07:56:24.731726+0000","last_became_peered":"2023-11-24T07:56:24.731726+0000","last_unstale":"2023-11-24T07:56:27.672077+0000","last_undegraded":"2023-11-24T07:56:27.672077+0000","last_fullsized":"2023-11-24T07:56:27.672077+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:56:23.683366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:56:23.683366+0000","last_clean_scrub_stamp":"2023-11-24T07:56:23.683366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:52:22.451306+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T07:56:25.712721+0000","last_change":"2023-11-24T07:56:24.721147+0000","last_active":"2023-11-24T07:56:25.712721+0000","last_peered":"2023-11-24T07:56:25.712721+0000","last_clean":"2023-11-24T07:56:25.712721+0000","last_became_active":"2023-11-24T07:56:24.720857+0000","last_became_peered":"2023-11-24T07:56:24.720857+0000","last_unstale":"2023-11-24T07:56:25.712721+0000","last_undegraded":"2023-11-24T07:56:25.712721+0000","last_fullsized":"2023-11-24T07:56:25.712721+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:56:23.683366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:56:23.683366+0000","last_clean_scrub_stamp":"2023-11-24T07:56:23.683366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T19:48:41.181670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T07:56:25.711756+0000","last_change":"2023-11-24T07:56:24.726027+0000","last_active":"2023-11-24T07:56:25.711756+0000","last_peered":"2023-11-24T07:56:25.711756+0000","last_clean":"2023-11-24T07:56:25.711756+0000","last_became_active":"2023-11-24T07:56:24.725339+0000","last_became_peered":"2023-11-24T07:56:24.725339+0000","last_unstale":"2023-11-24T07:56:25.711756+0000","last_undegraded":"2023-11-24T07:56:25.711756+0000","last_fullsized":"2023-11-24T07:56:25.711756+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:56:23.683366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:56:23.683366+0000","last_clean_scrub_stamp":"2023-11-24T07:56:23.683366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T19:26:21.558298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T07:56:25.709012+0000","last_change":"2023-11-24T07:56:24.725132+0000","last_active":"2023-11-24T07:56:25.709012+0000","last_peered":"2023-11-24T07:56:25.709012+0000","last_clean":"2023-11-24T07:56:25.709012+0000","last_became_active":"2023-11-24T07:56:24.724817+0000","last_became_peered":"2023-11-24T07:56:24.724817+0000","last_unstale":"2023-11-24T07:56:25.709012+0000","last_undegraded":"2023-11-24T07:56:25.709012+0000","last_fullsized":"2023-11-24T07:56:25.709012+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:56:23.683366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:56:23.683366+0000","last_clean_scrub_stamp":"2023-11-24T07:56:23.683366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T19:32:53.988497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T07:56:27.671602+0000","last_change":"2023-11-24T07:56:24.722870+0000","last_active":"2023-11-24T07:56:27.671602+0000","last_peered":"2023-11-24T07:56:27.671602+0000","last_clean":"2023-11-24T07:56:27.671602+0000","last_became_active":"2023-11-24T07:56:24.722572+0000","last_became_peered":"2023-11-24T07:56:24.722572+0000","last_unstale":"2023-11-24T07:56:27.671602+0000","last_undegraded":"2023-11-24T07:56:27.671602+0000","last_fullsized":"2023-11-24T07:56:27.671602+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:56:23.683366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:56:23.683366+0000","last_clean_scrub_stamp":"2023-11-24T07:56:23.683366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T15:28:38.799061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T07:56:25.708767+0000","last_change":"2023-11-24T07:56:24.726061+0000","last_active":"2023-11-24T07:56:25.708767+0000","last_peered":"2023-11-24T07:56:25.708767+0000","last_clean":"2023-11-24T07:56:25.708767+0000","last_became_active":"2023-11-24T07:56:24.725750+0000","last_became_peered":"2023-11-24T07:56:24.725750+0000","last_unstale":"2023-11-24T07:56:25.708767+0000","last_undegraded":"2023-11-24T07:56:25.708767+0000","last_fullsized":"2023-11-24T07:56:25.708767+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:56:23.683366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:56:23.683366+0000","last_clean_scrub_stamp":"2023-11-24T07:56:23.683366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T16:22:33.714226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T07:56:27.671397+0000","last_change":"2023-11-24T07:56:24.725406+0000","last_active":"2023-11-24T07:56:27.671397+0000","last_peered":"2023-11-24T07:56:27.671397+0000","last_clean":"2023-11-24T07:56:27.671397+0000","last_became_active":"2023-11-24T07:56:24.725080+0000","last_became_peered":"2023-11-24T07:56:24.725080+0000","last_unstale":"2023-11-24T07:56:27.671397+0000","last_undegraded":"2023-11-24T07:56:27.671397+0000","last_fullsized":"2023-11-24T07:56:27.671397+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:56:23.683366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:56:23.683366+0000","last_clean_scrub_stamp":"2023-11-24T07:56:23.683366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T18:48:33.280136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T07:56:25.711711+0000","last_change":"2023-11-24T07:56:24.719072+0000","last_active":"2023-11-24T07:56:25.711711+0000","last_peered":"2023-11-24T07:56:25.711711+0000","last_clean":"2023-11-24T07:56:25.711711+0000","last_became_active":"2023-11-24T07:56:24.718791+0000","last_became_peered":"2023-11-24T07:56:24.718791+0000","last_unstale":"2023-11-24T07:56:25.711711+0000","last_undegraded":"2023-11-24T07:56:25.711711+0000","last_fullsized":"2023-11-24T07:56:25.711711+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:56:23.683366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:56:23.683366+0000","last_clean_scrub_stamp":"2023-11-24T07:56:23.683366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:13:16.865275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T07:56:27.783015+0000","last_change":"2023-11-24T07:56:24.743409+0000","last_active":"2023-11-24T07:56:27.783015+0000","last_peered":"2023-11-24T07:56:27.783015+0000","last_clean":"2023-11-24T07:56:27.783015+0000","last_became_active":"2023-11-24T07:56:24.742988+0000","last_became_peered":"2023-11-24T07:56:24.742988+0000","last_unstale":"2023-11-24T07:56:27.783015+0000","last_undegraded":"2023-11-24T07:56:27.783015+0000","last_fullsized":"2023-11-24T07:56:27.783015+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:56:23.683366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:56:23.683366+0000","last_clean_scrub_stamp":"2023-11-24T07:56:23.683366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T09:01:29.634179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T07:56:27.672028+0000","last_change":"2023-11-24T07:56:24.718066+0000","last_active":"2023-11-24T07:56:27.672028+0000","last_peered":"2023-11-24T07:56:27.672028+0000","last_clean":"2023-11-24T07:56:27.672028+0000","last_became_active":"2023-11-24T07:56:24.715725+0000","last_became_peered":"2023-11-24T07:56:24.715725+0000","last_unstale":"2023-11-24T07:56:27.672028+0000","last_undegraded":"2023-11-24T07:56:27.672028+0000","last_fullsized":"2023-11-24T07:56:27.672028+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:56:23.683366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:56:23.683366+0000","last_clean_scrub_stamp":"2023-11-24T07:56:23.683366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T13:56:07.967730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T07:56:27.671765+0000","last_change":"2023-11-24T07:56:24.730488+0000","last_active":"2023-11-24T07:56:27.671765+0000","last_peered":"2023-11-24T07:56:27.671765+0000","last_clean":"2023-11-24T07:56:27.671765+0000","last_became_active":"2023-11-24T07:56:24.730171+0000","last_became_peered":"2023-11-24T07:56:24.730171+0000","last_unstale":"2023-11-24T07:56:27.671765+0000","last_undegraded":"2023-11-24T07:56:27.671765+0000","last_fullsized":"2023-11-24T07:56:27.671765+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:56:23.683366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:56:23.683366+0000","last_clean_scrub_stamp":"2023-11-24T07:56:23.683366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T19:29:50.373581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T07:56:27.671956+0000","last_change":"2023-11-24T07:56:24.734314+0000","last_active":"2023-11-24T07:56:27.671956+0000","last_peered":"2023-11-24T07:56:27.671956+0000","last_clean":"2023-11-24T07:56:27.671956+0000","last_became_active":"2023-11-24T07:56:24.733966+0000","last_became_peered":"2023-11-24T07:56:24.733966+0000","last_unstale":"2023-11-24T07:56:27.671956+0000","last_undegraded":"2023-11-24T07:56:27.671956+0000","last_fullsized":"2023-11-24T07:56:27.671956+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:56:23.683366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:56:23.683366+0000","last_clean_scrub_stamp":"2023-11-24T07:56:23.683366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T19:39:29.828696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T07:56:27.782981+0000","last_change":"2023-11-24T07:56:24.717337+0000","last_active":"2023-11-24T07:56:27.782981+0000","last_peered":"2023-11-24T07:56:27.782981+0000","last_clean":"2023-11-24T07:56:27.782981+0000","last_became_active":"2023-11-24T07:56:24.716929+0000","last_became_peered":"2023-11-24T07:56:24.716929+0000","last_unstale":"2023-11-24T07:56:27.782981+0000","last_undegraded":"2023-11-24T07:56:27.782981+0000","last_fullsized":"2023-11-24T07:56:27.782981+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:56:23.683366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:56:23.683366+0000","last_clean_scrub_stamp":"2023-11-24T07:56:23.683366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T17:36:14.367398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T07:56:27.783087+0000","last_change":"2023-11-24T07:56:24.720081+0000","last_active":"2023-11-24T07:56:27.783087+0000","last_peered":"2023-11-24T07:56:27.783087+0000","last_clean":"2023-11-24T07:56:27.783087+0000","last_became_active":"2023-11-24T07:56:24.719776+0000","last_became_peered":"2023-11-24T07:56:24.719776+0000","last_unstale":"2023-11-24T07:56:27.783087+0000","last_undegraded":"2023-11-24T07:56:27.783087+0000","last_fullsized":"2023-11-24T07:56:27.783087+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:56:23.683366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:56:23.683366+0000","last_clean_scrub_stamp":"2023-11-24T07:56:23.683366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T19:43:40.193280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T07:56:25.712435+0000","last_change":"2023-11-24T07:56:24.719400+0000","last_active":"2023-11-24T07:56:25.712435+0000","last_peered":"2023-11-24T07:56:25.712435+0000","last_clean":"2023-11-24T07:56:25.712435+0000","last_became_active":"2023-11-24T07:56:24.719095+0000","last_became_peered":"2023-11-24T07:56:24.719095+0000","last_unstale":"2023-11-24T07:56:25.712435+0000","last_undegraded":"2023-11-24T07:56:25.712435+0000","last_fullsized":"2023-11-24T07:56:25.712435+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:56:23.683366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:56:23.683366+0000","last_clean_scrub_stamp":"2023-11-24T07:56:23.683366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T19:29:50.373581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T07:56:26.100997+0000","last_change":"2023-11-24T07:56:24.725927+0000","last_active":"2023-11-24T07:56:26.100997+0000","last_peered":"2023-11-24T07:56:26.100997+0000","last_clean":"2023-11-24T07:56:26.100997+0000","last_became_active":"2023-11-24T07:56:24.725568+0000","last_became_peered":"2023-11-24T07:56:24.725568+0000","last_unstale":"2023-11-24T07:56:26.100997+0000","last_undegraded":"2023-11-24T07:56:26.100997+0000","last_fullsized":"2023-11-24T07:56:26.100997+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:56:23.683366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:56:23.683366+0000","last_clean_scrub_stamp":"2023-11-24T07:56:23.683366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T19:39:29.828696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T07:56:26.101717+0000","last_change":"2023-11-24T07:56:24.730986+0000","last_active":"2023-11-24T07:56:26.101717+0000","last_peered":"2023-11-24T07:56:26.101717+0000","last_clean":"2023-11-24T07:56:26.101717+0000","last_became_active":"2023-11-24T07:56:24.730564+0000","last_became_peered":"2023-11-24T07:56:24.730564+0000","last_unstale":"2023-11-24T07:56:26.101717+0000","last_undegraded":"2023-11-24T07:56:26.101717+0000","last_fullsized":"2023-11-24T07:56:26.101717+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:56:23.683366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:56:23.683366+0000","last_clean_scrub_stamp":"2023-11-24T07:56:23.683366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T14:48:19.320894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T07:56:27.783028+0000","last_change":"2023-11-24T07:56:24.716131+0000","last_active":"2023-11-24T07:56:27.783028+0000","last_peered":"2023-11-24T07:56:27.783028+0000","last_clean":"2023-11-24T07:56:27.783028+0000","last_became_active":"2023-11-24T07:56:24.715770+0000","last_became_peered":"2023-11-24T07:56:24.715770+0000","last_unstale":"2023-11-24T07:56:27.783028+0000","last_undegraded":"2023-11-24T07:56:27.783028+0000","last_fullsized":"2023-11-24T07:56:27.783028+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:56:23.683366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:56:23.683366+0000","last_clean_scrub_stamp":"2023-11-24T07:56:23.683366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T16:22:33.714226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T07:56:26.101413+0000","last_change":"2023-11-24T07:56:24.726104+0000","last_active":"2023-11-24T07:56:26.101413+0000","last_peered":"2023-11-24T07:56:26.101413+0000","last_clean":"2023-11-24T07:56:26.101413+0000","last_became_active":"2023-11-24T07:56:24.725718+0000","last_became_peered":"2023-11-24T07:56:24.725718+0000","last_unstale":"2023-11-24T07:56:26.101413+0000","last_undegraded":"2023-11-24T07:56:26.101413+0000","last_fullsized":"2023-11-24T07:56:26.101413+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:56:23.683366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:56:23.683366+0000","last_clean_scrub_stamp":"2023-11-24T07:56:23.683366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T15:24:09.050735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"19'4","reported_seq":16,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-24T07:56:24.743886+0000","last_change":"2023-11-24T07:56:23.714550+0000","last_active":"2023-11-24T07:56:24.743886+0000","last_peered":"2023-11-24T07:56:24.743886+0000","last_clean":"2023-11-24T07:56:24.743886+0000","last_became_active":"2023-11-24T07:56:23.714275+0000","last_became_peered":"2023-11-24T07:56:23.714275+0000","last_unstale":"2023-11-24T07:56:24.743886+0000","last_undegraded":"2023-11-24T07:56:24.743886+0000","last_fullsized":"2023-11-24T07:56:24.743886+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:56:22.678019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:56:22.678019+0000","last_clean_scrub_stamp":"2023-11-24T07:56:22.678019+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T19:05:26.056275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T07:56:25.712605+0000","last_change":"2023-11-24T07:56:24.722991+0000","last_active":"2023-11-24T07:56:25.712605+0000","last_peered":"2023-11-24T07:56:25.712605+0000","last_clean":"2023-11-24T07:56:25.712605+0000","last_became_active":"2023-11-24T07:56:24.722711+0000","last_became_peered":"2023-11-24T07:56:24.722711+0000","last_unstale":"2023-11-24T07:56:25.712605+0000","last_undegraded":"2023-11-24T07:56:25.712605+0000","last_fullsized":"2023-11-24T07:56:25.712605+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:56:23.683366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:56:23.683366+0000","last_clean_scrub_stamp":"2023-11-24T07:56:23.683366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T12:41:22.436958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1e","version":"19'1","reported_seq":13,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-24T07:56:24.739921+0000","last_change":"2023-11-24T07:56:23.702219+0000","last_active":"2023-11-24T07:56:24.739921+0000","last_peered":"2023-11-24T07:56:24.739921+0000","last_clean":"2023-11-24T07:56:24.739921+0000","last_became_active":"2023-11-24T07:56:23.701922+0000","last_became_peered":"2023-11-24T07:56:23.701922+0000","last_unstale":"2023-11-24T07:56:24.739921+0000","last_undegraded":"2023-11-24T07:56:24.739921+0000","last_fullsized":"2023-11-24T07:56:24.739921+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:56:22.678019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:56:22.678019+0000","last_clean_scrub_stamp":"2023-11-24T07:56:22.678019+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T09:01:28.628833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T07:56:25.712243+0000","last_change":"2023-11-24T07:56:24.733713+0000","last_active":"2023-11-24T07:56:25.712243+0000","last_peered":"2023-11-24T07:56:25.712243+0000","last_clean":"2023-11-24T07:56:25.712243+0000","last_became_active":"2023-11-24T07:56:24.733302+0000","last_became_peered":"2023-11-24T07:56:24.733302+0000","last_unstale":"2023-11-24T07:56:25.712243+0000","last_undegraded":"2023-11-24T07:56:25.712243+0000","last_fullsized":"2023-11-24T07:56:25.712243+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:56:23.683366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:56:23.683366+0000","last_clean_scrub_stamp":"2023-11-24T07:56:23.683366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T15:28:38.799061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1d","version":"19'2","reported_seq":14,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-24T07:56:24.740403+0000","last_change":"2023-11-24T07:56:23.707439+0000","last_active":"2023-11-24T07:56:24.740403+0000","last_peered":"2023-11-24T07:56:24.740403+0000","last_clean":"2023-11-24T07:56:24.740403+0000","last_became_active":"2023-11-24T07:56:23.707136+0000","last_became_peered":"2023-11-24T07:56:23.707136+0000","last_unstale":"2023-11-24T07:56:24.740403+0000","last_undegraded":"2023-11-24T07:56:24.740403+0000","last_fullsized":"2023-11-24T07:56:24.740403+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:56:22.678019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:56:22.678019+0000","last_clean_scrub_stamp":"2023-11-24T07:56:22.678019+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T19:43:39.187934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T07:56:27.782857+0000","last_change":"2023-11-24T07:56:24.733325+0000","last_active":"2023-11-24T07:56:27.782857+0000","last_peered":"2023-11-24T07:56:27.782857+0000","last_clean":"2023-11-24T07:56:27.782857+0000","last_became_active":"2023-11-24T07:56:24.733014+0000","last_became_peered":"2023-11-24T07:56:24.733014+0000","last_unstale":"2023-11-24T07:56:27.782857+0000","last_undegraded":"2023-11-24T07:56:27.782857+0000","last_fullsized":"2023-11-24T07:56:27.782857+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:56:23.683366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:56:23.683366+0000","last_clean_scrub_stamp":"2023-11-24T07:56:23.683366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T19:43:40.193280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"19'2","reported_seq":14,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-24T07:56:24.740415+0000","last_change":"2023-11-24T07:56:23.709116+0000","last_active":"2023-11-24T07:56:24.740415+0000","last_peered":"2023-11-24T07:56:24.740415+0000","last_clean":"2023-11-24T07:56:24.740415+0000","last_became_active":"2023-11-24T07:56:23.708703+0000","last_became_peered":"2023-11-24T07:56:23.708703+0000","last_unstale":"2023-11-24T07:56:24.740415+0000","last_undegraded":"2023-11-24T07:56:24.740415+0000","last_fullsized":"2023-11-24T07:56:24.740415+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:56:22.678019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:56:22.678019+0000","last_clean_scrub_stamp":"2023-11-24T07:56:22.678019+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T09:01:28.628833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T07:56:26.101403+0000","last_change":"2023-11-24T07:56:24.731750+0000","last_active":"2023-11-24T07:56:26.101403+0000","last_peered":"2023-11-24T07:56:26.101403+0000","last_clean":"2023-11-24T07:56:26.101403+0000","last_became_active":"2023-11-24T07:56:24.730636+0000","last_became_peered":"2023-11-24T07:56:24.730636+0000","last_unstale":"2023-11-24T07:56:26.101403+0000","last_undegraded":"2023-11-24T07:56:26.101403+0000","last_fullsized":"2023-11-24T07:56:26.101403+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:56:23.683366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:56:23.683366+0000","last_clean_scrub_stamp":"2023-11-24T07:56:23.683366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T15:11:49.015783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1b","version":"19'1","reported_seq":13,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-24T07:56:24.737950+0000","last_change":"2023-11-24T07:56:23.702767+0000","last_active":"2023-11-24T07:56:24.737950+0000","last_peered":"2023-11-24T07:56:24.737950+0000","last_clean":"2023-11-24T07:56:24.737950+0000","last_became_active":"2023-11-24T07:56:23.702499+0000","last_became_peered":"2023-11-24T07:56:23.702499+0000","last_unstale":"2023-11-24T07:56:24.737950+0000","last_undegraded":"2023-11-24T07:56:24.737950+0000","last_fullsized":"2023-11-24T07:56:24.737950+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:56:22.678019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:56:22.678019+0000","last_clean_scrub_stamp":"2023-11-24T07:56:22.678019+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T17:36:13.362051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T07:56:27.782679+0000","last_change":"2023-11-24T07:56:24.735158+0000","last_active":"2023-11-24T07:56:27.782679+0000","last_peered":"2023-11-24T07:56:27.782679+0000","last_clean":"2023-11-24T07:56:27.782679+0000","last_became_active":"2023-11-24T07:56:24.734600+0000","last_became_peered":"2023-11-24T07:56:24.734600+0000","last_unstale":"2023-11-24T07:56:27.782679+0000","last_undegraded":"2023-11-24T07:56:27.782679+0000","last_fullsized":"2023-11-24T07:56:27.782679+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:56:23.683366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:56:23.683366+0000","last_clean_scrub_stamp":"2023-11-24T07:56:23.683366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T17:36:14.367398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1a","version":"19'1","reported_seq":13,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-24T07:56:24.743591+0000","last_change":"2023-11-24T07:56:23.712864+0000","last_active":"2023-11-24T07:56:24.743591+0000","last_peered":"2023-11-24T07:56:24.743591+0000","last_clean":"2023-11-24T07:56:24.743591+0000","last_became_active":"2023-11-24T07:56:23.712559+0000","last_became_peered":"2023-11-24T07:56:23.712559+0000","last_unstale":"2023-11-24T07:56:24.743591+0000","last_undegraded":"2023-11-24T07:56:24.743591+0000","last_fullsized":"2023-11-24T07:56:24.743591+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:56:22.678019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:56:22.678019+0000","last_clean_scrub_stamp":"2023-11-24T07:56:22.678019+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:19:21.093151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T07:56:25.712365+0000","last_change":"2023-11-24T07:56:24.727143+0000","last_active":"2023-11-24T07:56:25.712365+0000","last_peered":"2023-11-24T07:56:25.712365+0000","last_clean":"2023-11-24T07:56:25.712365+0000","last_became_active":"2023-11-24T07:56:24.726637+0000","last_became_peered":"2023-11-24T07:56:24.726637+0000","last_unstale":"2023-11-24T07:56:25.712365+0000","last_undegraded":"2023-11-24T07:56:25.712365+0000","last_fullsized":"2023-11-24T07:56:25.712365+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:56:23.683366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:56:23.683366+0000","last_clean_scrub_stamp":"2023-11-24T07:56:23.683366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T15:28:38.799061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":12,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-24T07:56:24.710275+0000","last_change":"2023-11-24T07:56:23.707863+0000","last_active":"2023-11-24T07:56:24.710275+0000","last_peered":"2023-11-24T07:56:24.710275+0000","last_clean":"2023-11-24T07:56:24.710275+0000","last_became_active":"2023-11-24T07:56:23.707595+0000","last_became_peered":"2023-11-24T07:56:23.707595+0000","last_unstale":"2023-11-24T07:56:24.710275+0000","last_undegraded":"2023-11-24T07:56:24.710275+0000","last_fullsized":"2023-11-24T07:56:24.710275+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:56:22.678019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:56:22.678019+0000","last_clean_scrub_stamp":"2023-11-24T07:56:22.678019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T11:00:03.947675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T07:56:25.712360+0000","last_change":"2023-11-24T07:56:24.721105+0000","last_active":"2023-11-24T07:56:25.712360+0000","last_peered":"2023-11-24T07:56:25.712360+0000","last_clean":"2023-11-24T07:56:25.712360+0000","last_became_active":"2023-11-24T07:56:24.720813+0000","last_became_peered":"2023-11-24T07:56:24.720813+0000","last_unstale":"2023-11-24T07:56:25.712360+0000","last_undegraded":"2023-11-24T07:56:25.712360+0000","last_fullsized":"2023-11-24T07:56:25.712360+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:56:23.683366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:56:23.683366+0000","last_clean_scrub_stamp":"2023-11-24T07:56:23.683366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T17:43:47.155869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":12,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-24T07:56:24.719929+0000","last_change":"2023-11-24T07:56:23.704771+0000","last_active":"2023-11-24T07:56:24.719929+0000","last_peered":"2023-11-24T07:56:24.719929+0000","last_clean":"2023-11-24T07:56:24.719929+0000","last_became_active":"2023-11-24T07:56:23.704433+0000","last_became_peered":"2023-11-24T07:56:23.704433+0000","last_unstale":"2023-11-24T07:56:24.719929+0000","last_undegraded":"2023-11-24T07:56:24.719929+0000","last_fullsized":"2023-11-24T07:56:24.719929+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:56:22.678019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:56:22.678019+0000","last_clean_scrub_stamp":"2023-11-24T07:56:22.678019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:14:32.121393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T07:56:25.711836+0000","last_change":"2023-11-24T07:56:24.734742+0000","last_active":"2023-11-24T07:56:25.711836+0000","last_peered":"2023-11-24T07:56:25.711836+0000","last_clean":"2023-11-24T07:56:25.711836+0000","last_became_active":"2023-11-24T07:56:24.734447+0000","last_became_peered":"2023-11-24T07:56:24.734447+0000","last_unstale":"2023-11-24T07:56:25.711836+0000","last_undegraded":"2023-11-24T07:56:25.711836+0000","last_fullsized":"2023-11-24T07:56:25.711836+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:56:23.683366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:56:23.683366+0000","last_clean_scrub_stamp":"2023-11-24T07:56:23.683366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T19:26:21.558298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":12,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-24T07:56:24.729505+0000","last_change":"2023-11-24T07:56:23.701888+0000","last_active":"2023-11-24T07:56:24.729505+0000","last_peered":"2023-11-24T07:56:24.729505+0000","last_clean":"2023-11-24T07:56:24.729505+0000","last_became_active":"2023-11-24T07:56:23.701611+0000","last_became_peered":"2023-11-24T07:56:23.701611+0000","last_unstale":"2023-11-24T07:56:24.729505+0000","last_undegraded":"2023-11-24T07:56:24.729505+0000","last_fullsized":"2023-11-24T07:56:24.729505+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:56:22.678019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:56:22.678019+0000","last_clean_scrub_stamp":"2023-11-24T07:56:22.678019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T18:53:13.845460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T07:56:25.709371+0000","last_change":"2023-11-24T07:56:24.731116+0000","last_active":"2023-11-24T07:56:25.709371+0000","last_peered":"2023-11-24T07:56:25.709371+0000","last_clean":"2023-11-24T07:56:25.709371+0000","last_became_active":"2023-11-24T07:56:24.730805+0000","last_became_peered":"2023-11-24T07:56:24.730805+0000","last_unstale":"2023-11-24T07:56:25.709371+0000","last_undegraded":"2023-11-24T07:56:25.709371+0000","last_fullsized":"2023-11-24T07:56:25.709371+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:56:23.683366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:56:23.683366+0000","last_clean_scrub_stamp":"2023-11-24T07:56:23.683366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T19:05:27.061622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":12,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-24T07:56:24.712027+0000","last_change":"2023-11-24T07:56:23.707470+0000","last_active":"2023-11-24T07:56:24.712027+0000","last_peered":"2023-11-24T07:56:24.712027+0000","last_clean":"2023-11-24T07:56:24.712027+0000","last_became_active":"2023-11-24T07:56:23.707194+0000","last_became_peered":"2023-11-24T07:56:23.707194+0000","last_unstale":"2023-11-24T07:56:24.712027+0000","last_undegraded":"2023-11-24T07:56:24.712027+0000","last_fullsized":"2023-11-24T07:56:24.712027+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:56:22.678019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:56:22.678019+0000","last_clean_scrub_stamp":"2023-11-24T07:56:22.678019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T19:26:20.552951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T07:56:25.712327+0000","last_change":"2023-11-24T07:56:24.733042+0000","last_active":"2023-11-24T07:56:25.712327+0000","last_peered":"2023-11-24T07:56:25.712327+0000","last_clean":"2023-11-24T07:56:25.712327+0000","last_became_active":"2023-11-24T07:56:24.732746+0000","last_became_peered":"2023-11-24T07:56:24.732746+0000","last_unstale":"2023-11-24T07:56:25.712327+0000","last_undegraded":"2023-11-24T07:56:25.712327+0000","last_fullsized":"2023-11-24T07:56:25.712327+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:56:23.683366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:56:23.683366+0000","last_clean_scrub_stamp":"2023-11-24T07:56:23.683366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:19:22.098498+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.15","version":"19'3","reported_seq":15,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-24T07:56:24.739732+0000","last_change":"2023-11-24T07:56:23.703208+0000","last_active":"2023-11-24T07:56:24.739732+0000","last_peered":"2023-11-24T07:56:24.739732+0000","last_clean":"2023-11-24T07:56:24.739732+0000","last_became_active":"2023-11-24T07:56:23.702891+0000","last_became_peered":"2023-11-24T07:56:23.702891+0000","last_unstale":"2023-11-24T07:56:24.739732+0000","last_undegraded":"2023-11-24T07:56:24.739732+0000","last_fullsized":"2023-11-24T07:56:24.739732+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:56:22.678019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:56:22.678019+0000","last_clean_scrub_stamp":"2023-11-24T07:56:22.678019+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T08:42:21.735516+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T07:56:26.101263+0000","last_change":"2023-11-24T07:56:24.729010+0000","last_active":"2023-11-24T07:56:26.101263+0000","last_peered":"2023-11-24T07:56:26.101263+0000","last_clean":"2023-11-24T07:56:26.101263+0000","last_became_active":"2023-11-24T07:56:24.728711+0000","last_became_peered":"2023-11-24T07:56:24.728711+0000","last_unstale":"2023-11-24T07:56:26.101263+0000","last_undegraded":"2023-11-24T07:56:26.101263+0000","last_fullsized":"2023-11-24T07:56:26.101263+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:56:23.683366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:56:23.683366+0000","last_clean_scrub_stamp":"2023-11-24T07:56:23.683366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T17:48:28.079331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.14","version":"19'6","reported_seq":18,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-24T07:56:24.743066+0000","last_change":"2023-11-24T07:56:23.706638+0000","last_active":"2023-11-24T07:56:24.743066+0000","last_peered":"2023-11-24T07:56:24.743066+0000","last_clean":"2023-11-24T07:56:24.743066+0000","last_became_active":"2023-11-24T07:56:23.706357+0000","last_became_peered":"2023-11-24T07:56:23.706357+0000","last_unstale":"2023-11-24T07:56:24.743066+0000","last_undegraded":"2023-11-24T07:56:24.743066+0000","last_fullsized":"2023-11-24T07:56:24.743066+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:56:22.678019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:56:22.678019+0000","last_clean_scrub_stamp":"2023-11-24T07:56:22.678019+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T14:40:38.179620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T07:56:27.671894+0000","last_change":"2023-11-24T07:56:24.717150+0000","last_active":"2023-11-24T07:56:27.671894+0000","last_peered":"2023-11-24T07:56:27.671894+0000","last_clean":"2023-11-24T07:56:27.671894+0000","last_became_active":"2023-11-24T07:56:24.716723+0000","last_became_peered":"2023-11-24T07:56:24.716723+0000","last_unstale":"2023-11-24T07:56:27.671894+0000","last_undegraded":"2023-11-24T07:56:27.671894+0000","last_fullsized":"2023-11-24T07:56:27.671894+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:56:23.683366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:56:23.683366+0000","last_clean_scrub_stamp":"2023-11-24T07:56:23.683366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T12:41:22.436958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.13","version":"19'2","reported_seq":14,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-24T07:56:24.739597+0000","last_change":"2023-11-24T07:56:23.707817+0000","last_active":"2023-11-24T07:56:24.739597+0000","last_peered":"2023-11-24T07:56:24.739597+0000","last_clean":"2023-11-24T07:56:24.739597+0000","last_became_active":"2023-11-24T07:56:23.707511+0000","last_became_peered":"2023-11-24T07:56:23.707511+0000","last_unstale":"2023-11-24T07:56:24.739597+0000","last_undegraded":"2023-11-24T07:56:24.739597+0000","last_fullsized":"2023-11-24T07:56:24.739597+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:56:22.678019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:56:22.678019+0000","last_clean_scrub_stamp":"2023-11-24T07:56:22.678019+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T15:03:38.755757+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T07:56:27.783173+0000","last_change":"2023-11-24T07:56:24.730834+0000","last_active":"2023-11-24T07:56:27.783173+0000","last_peered":"2023-11-24T07:56:27.783173+0000","last_clean":"2023-11-24T07:56:27.783173+0000","last_became_active":"2023-11-24T07:56:24.730545+0000","last_became_peered":"2023-11-24T07:56:24.730545+0000","last_unstale":"2023-11-24T07:56:27.783173+0000","last_undegraded":"2023-11-24T07:56:27.783173+0000","last_fullsized":"2023-11-24T07:56:27.783173+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:56:23.683366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:56:23.683366+0000","last_clean_scrub_stamp":"2023-11-24T07:56:23.683366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T18:20:49.373891+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":12,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-24T07:56:24.723510+0000","last_change":"2023-11-24T07:56:23.712182+0000","last_active":"2023-11-24T07:56:24.723510+0000","last_peered":"2023-11-24T07:56:24.723510+0000","last_clean":"2023-11-24T07:56:24.723510+0000","last_became_active":"2023-11-24T07:56:23.711814+0000","last_became_peered":"2023-11-24T07:56:23.711814+0000","last_unstale":"2023-11-24T07:56:24.723510+0000","last_undegraded":"2023-11-24T07:56:24.723510+0000","last_fullsized":"2023-11-24T07:56:24.723510+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:56:22.678019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:56:22.678019+0000","last_clean_scrub_stamp":"2023-11-24T07:56:22.678019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T15:43:53.652738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T07:56:26.101323+0000","last_change":"2023-11-24T07:56:24.730497+0000","last_active":"2023-11-24T07:56:26.101323+0000","last_peered":"2023-11-24T07:56:26.101323+0000","last_clean":"2023-11-24T07:56:26.101323+0000","last_became_active":"2023-11-24T07:56:24.730178+0000","last_became_peered":"2023-11-24T07:56:24.730178+0000","last_unstale":"2023-11-24T07:56:26.101323+0000","last_undegraded":"2023-11-24T07:56:26.101323+0000","last_fullsized":"2023-11-24T07:56:26.101323+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:56:23.683366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:56:23.683366+0000","last_clean_scrub_stamp":"2023-11-24T07:56:23.683366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T16:27:17.883314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":12,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-24T07:56:24.711899+0000","last_change":"2023-11-24T07:56:23.713877+0000","last_active":"2023-11-24T07:56:24.711899+0000","last_peered":"2023-11-24T07:56:24.711899+0000","last_clean":"2023-11-24T07:56:24.711899+0000","last_became_active":"2023-11-24T07:56:23.713603+0000","last_became_peered":"2023-11-24T07:56:23.713603+0000","last_unstale":"2023-11-24T07:56:24.711899+0000","last_undegraded":"2023-11-24T07:56:24.711899+0000","last_fullsized":"2023-11-24T07:56:24.711899+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:56:22.678019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:56:22.678019+0000","last_clean_scrub_stamp":"2023-11-24T07:56:22.678019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:13:15.859928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T07:56:27.782480+0000","last_change":"2023-11-24T07:56:24.733613+0000","last_active":"2023-11-24T07:56:27.782480+0000","last_peered":"2023-11-24T07:56:27.782480+0000","last_clean":"2023-11-24T07:56:27.782480+0000","last_became_active":"2023-11-24T07:56:24.733361+0000","last_became_peered":"2023-11-24T07:56:24.733361+0000","last_unstale":"2023-11-24T07:56:27.782480+0000","last_undegraded":"2023-11-24T07:56:27.782480+0000","last_fullsized":"2023-11-24T07:56:27.782480+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:56:23.683366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:56:23.683366+0000","last_clean_scrub_stamp":"2023-11-24T07:56:23.683366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T16:22:33.714226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.10","version":"19'8","reported_seq":20,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-24T07:56:24.740763+0000","last_change":"2023-11-24T07:56:23.701319+0000","last_active":"2023-11-24T07:56:24.740763+0000","last_peered":"2023-11-24T07:56:24.740763+0000","last_clean":"2023-11-24T07:56:24.740763+0000","last_became_active":"2023-11-24T07:56:23.700995+0000","last_became_peered":"2023-11-24T07:56:23.700995+0000","last_unstale":"2023-11-24T07:56:24.740763+0000","last_undegraded":"2023-11-24T07:56:24.740763+0000","last_fullsized":"2023-11-24T07:56:24.740763+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:56:22.678019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:56:22.678019+0000","last_clean_scrub_stamp":"2023-11-24T07:56:22.678019+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T19:43:39.187934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T07:56:26.101104+0000","last_change":"2023-11-24T07:56:24.730591+0000","last_active":"2023-11-24T07:56:26.101104+0000","last_peered":"2023-11-24T07:56:26.101104+0000","last_clean":"2023-11-24T07:56:26.101104+0000","last_became_active":"2023-11-24T07:56:24.730281+0000","last_became_peered":"2023-11-24T07:56:24.730281+0000","last_unstale":"2023-11-24T07:56:26.101104+0000","last_undegraded":"2023-11-24T07:56:26.101104+0000","last_fullsized":"2023-11-24T07:56:26.101104+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:56:23.683366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:56:23.683366+0000","last_clean_scrub_stamp":"2023-11-24T07:56:23.683366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:52:22.451306+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.f","version":"19'1","reported_seq":13,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-24T07:56:24.739965+0000","last_change":"2023-11-24T07:56:23.708274+0000","last_active":"2023-11-24T07:56:24.739965+0000","last_peered":"2023-11-24T07:56:24.739965+0000","last_clean":"2023-11-24T07:56:24.739965+0000","last_became_active":"2023-11-24T07:56:23.708001+0000","last_became_peered":"2023-11-24T07:56:23.708001+0000","last_unstale":"2023-11-24T07:56:24.739965+0000","last_undegraded":"2023-11-24T07:56:24.739965+0000","last_fullsized":"2023-11-24T07:56:24.739965+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:56:22.678019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:56:22.678019+0000","last_clean_scrub_stamp":"2023-11-24T07:56:22.678019+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T09:29:56.578230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T07:56:25.709280+0000","last_change":"2023-11-24T07:56:24.720989+0000","last_active":"2023-11-24T07:56:25.709280+0000","last_peered":"2023-11-24T07:56:25.709280+0000","last_clean":"2023-11-24T07:56:25.709280+0000","last_became_active":"2023-11-24T07:56:24.720500+0000","last_became_peered":"2023-11-24T07:56:24.720500+0000","last_unstale":"2023-11-24T07:56:25.709280+0000","last_undegraded":"2023-11-24T07:56:25.709280+0000","last_fullsized":"2023-11-24T07:56:25.709280+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:56:23.683366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:56:23.683366+0000","last_clean_scrub_stamp":"2023-11-24T07:56:23.683366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T09:01:29.634179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":12,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-24T07:56:24.714465+0000","last_change":"2023-11-24T07:56:23.704585+0000","last_active":"2023-11-24T07:56:24.714465+0000","last_peered":"2023-11-24T07:56:24.714465+0000","last_clean":"2023-11-24T07:56:24.714465+0000","last_became_active":"2023-11-24T07:56:23.704316+0000","last_became_peered":"2023-11-24T07:56:23.704316+0000","last_unstale":"2023-11-24T07:56:24.714465+0000","last_undegraded":"2023-11-24T07:56:24.714465+0000","last_fullsized":"2023-11-24T07:56:24.714465+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:56:22.678019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:56:22.678019+0000","last_clean_scrub_stamp":"2023-11-24T07:56:22.678019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T18:53:13.845460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T07:56:25.709031+0000","last_change":"2023-11-24T07:56:24.718689+0000","last_active":"2023-11-24T07:56:25.709031+0000","last_peered":"2023-11-24T07:56:25.709031+0000","last_clean":"2023-11-24T07:56:25.709031+0000","last_became_active":"2023-11-24T07:56:24.717862+0000","last_became_peered":"2023-11-24T07:56:24.717862+0000","last_unstale":"2023-11-24T07:56:25.709031+0000","last_undegraded":"2023-11-24T07:56:25.709031+0000","last_fullsized":"2023-11-24T07:56:25.709031+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:56:23.683366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:56:23.683366+0000","last_clean_scrub_stamp":"2023-11-24T07:56:23.683366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T17:36:14.367398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":12,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-24T07:56:24.723355+0000","last_change":"2023-11-24T07:56:23.712764+0000","last_active":"2023-11-24T07:56:24.723355+0000","last_peered":"2023-11-24T07:56:24.723355+0000","last_clean":"2023-11-24T07:56:24.723355+0000","last_became_active":"2023-11-24T07:56:23.712448+0000","last_became_peered":"2023-11-24T07:56:23.712448+0000","last_unstale":"2023-11-24T07:56:24.723355+0000","last_undegraded":"2023-11-24T07:56:24.723355+0000","last_fullsized":"2023-11-24T07:56:24.723355+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:56:22.678019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:56:22.678019+0000","last_clean_scrub_stamp":"2023-11-24T07:56:22.678019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T19:29:49.368234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T07:56:27.671634+0000","last_change":"2023-11-24T07:56:24.716237+0000","last_active":"2023-11-24T07:56:27.671634+0000","last_peered":"2023-11-24T07:56:27.671634+0000","last_clean":"2023-11-24T07:56:27.671634+0000","last_became_active":"2023-11-24T07:56:24.715897+0000","last_became_peered":"2023-11-24T07:56:24.715897+0000","last_unstale":"2023-11-24T07:56:27.671634+0000","last_undegraded":"2023-11-24T07:56:27.671634+0000","last_fullsized":"2023-11-24T07:56:27.671634+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:56:23.683366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:56:23.683366+0000","last_clean_scrub_stamp":"2023-11-24T07:56:23.683366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:19:22.098498+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T07:56:25.711564+0000","last_change":"2023-11-24T07:56:24.723045+0000","last_active":"2023-11-24T07:56:25.711564+0000","last_peered":"2023-11-24T07:56:25.711564+0000","last_clean":"2023-11-24T07:56:25.711564+0000","last_became_active":"2023-11-24T07:56:24.722669+0000","last_became_peered":"2023-11-24T07:56:24.722669+0000","last_unstale":"2023-11-24T07:56:25.711564+0000","last_undegraded":"2023-11-24T07:56:25.711564+0000","last_fullsized":"2023-11-24T07:56:25.711564+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:56:23.683366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:56:23.683366+0000","last_clean_scrub_stamp":"2023-11-24T07:56:23.683366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:14:33.126740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":12,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-24T07:56:24.710112+0000","last_change":"2023-11-24T07:56:23.702360+0000","last_active":"2023-11-24T07:56:24.710112+0000","last_peered":"2023-11-24T07:56:24.710112+0000","last_clean":"2023-11-24T07:56:24.710112+0000","last_became_active":"2023-11-24T07:56:23.701979+0000","last_became_peered":"2023-11-24T07:56:23.701979+0000","last_unstale":"2023-11-24T07:56:24.710112+0000","last_undegraded":"2023-11-24T07:56:24.710112+0000","last_fullsized":"2023-11-24T07:56:24.710112+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:56:22.678019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:56:22.678019+0000","last_clean_scrub_stamp":"2023-11-24T07:56:22.678019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T14:52:28.306044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T07:56:25.711765+0000","last_change":"2023-11-24T07:56:24.727041+0000","last_active":"2023-11-24T07:56:25.711765+0000","last_peered":"2023-11-24T07:56:25.711765+0000","last_clean":"2023-11-24T07:56:25.711765+0000","last_became_active":"2023-11-24T07:56:24.726344+0000","last_became_peered":"2023-11-24T07:56:24.726344+0000","last_unstale":"2023-11-24T07:56:25.711765+0000","last_undegraded":"2023-11-24T07:56:25.711765+0000","last_fullsized":"2023-11-24T07:56:25.711765+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:56:23.683366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:56:23.683366+0000","last_clean_scrub_stamp":"2023-11-24T07:56:23.683366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:13:16.865275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":12,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-24T07:56:24.718243+0000","last_change":"2023-11-24T07:56:23.712776+0000","last_active":"2023-11-24T07:56:24.718243+0000","last_peered":"2023-11-24T07:56:24.718243+0000","last_clean":"2023-11-24T07:56:24.718243+0000","last_became_active":"2023-11-24T07:56:23.712504+0000","last_became_peered":"2023-11-24T07:56:23.712504+0000","last_unstale":"2023-11-24T07:56:24.718243+0000","last_undegraded":"2023-11-24T07:56:24.718243+0000","last_fullsized":"2023-11-24T07:56:24.718243+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:56:22.678019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:56:22.678019+0000","last_clean_scrub_stamp":"2023-11-24T07:56:22.678019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T19:05:26.056275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":165,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-24T07:56:24.728424+0000","last_change":"2023-11-24T07:56:19.668905+0000","last_active":"2023-11-24T07:56:24.728424+0000","last_peered":"2023-11-24T07:56:24.728424+0000","last_clean":"2023-11-24T07:56:24.728424+0000","last_became_active":"2023-11-24T07:56:19.668601+0000","last_became_peered":"2023-11-24T07:56:19.668601+0000","last_unstale":"2023-11-24T07:56:24.728424+0000","last_undegraded":"2023-11-24T07:56:24.728424+0000","last_fullsized":"2023-11-24T07:56:24.728424+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:56:18.657844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:56:18.657844+0000","last_clean_scrub_stamp":"2023-11-24T07:56:18.657844+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T11:51:53.828455+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T07:56:27.783117+0000","last_change":"2023-11-24T07:56:24.716904+0000","last_active":"2023-11-24T07:56:27.783117+0000","last_peered":"2023-11-24T07:56:27.783117+0000","last_clean":"2023-11-24T07:56:27.783117+0000","last_became_active":"2023-11-24T07:56:24.716599+0000","last_became_peered":"2023-11-24T07:56:24.716599+0000","last_unstale":"2023-11-24T07:56:27.783117+0000","last_undegraded":"2023-11-24T07:56:27.783117+0000","last_fullsized":"2023-11-24T07:56:27.783117+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:56:23.683366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:56:23.683366+0000","last_clean_scrub_stamp":"2023-11-24T07:56:23.683366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T09:01:29.634179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":12,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-24T07:56:24.719998+0000","last_change":"2023-11-24T07:56:23.712140+0000","last_active":"2023-11-24T07:56:24.719998+0000","last_peered":"2023-11-24T07:56:24.719998+0000","last_clean":"2023-11-24T07:56:24.719998+0000","last_became_active":"2023-11-24T07:56:23.711855+0000","last_became_peered":"2023-11-24T07:56:23.711855+0000","last_unstale":"2023-11-24T07:56:24.719998+0000","last_undegraded":"2023-11-24T07:56:24.719998+0000","last_fullsized":"2023-11-24T07:56:24.719998+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:56:22.678019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:56:22.678019+0000","last_clean_scrub_stamp":"2023-11-24T07:56:22.678019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:13:15.859928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":12,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-24T07:56:24.722016+0000","last_change":"2023-11-24T07:56:23.712415+0000","last_active":"2023-11-24T07:56:24.722016+0000","last_peered":"2023-11-24T07:56:24.722016+0000","last_clean":"2023-11-24T07:56:24.722016+0000","last_became_active":"2023-11-24T07:56:23.712104+0000","last_became_peered":"2023-11-24T07:56:23.712104+0000","last_unstale":"2023-11-24T07:56:24.722016+0000","last_undegraded":"2023-11-24T07:56:24.722016+0000","last_fullsized":"2023-11-24T07:56:24.722016+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:56:22.678019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:56:22.678019+0000","last_clean_scrub_stamp":"2023-11-24T07:56:22.678019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T18:48:32.274789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T07:56:27.783252+0000","last_change":"2023-11-24T07:56:24.718851+0000","last_active":"2023-11-24T07:56:27.783252+0000","last_peered":"2023-11-24T07:56:27.783252+0000","last_clean":"2023-11-24T07:56:27.783252+0000","last_became_active":"2023-11-24T07:56:24.718574+0000","last_became_peered":"2023-11-24T07:56:24.718574+0000","last_unstale":"2023-11-24T07:56:27.783252+0000","last_undegraded":"2023-11-24T07:56:27.783252+0000","last_fullsized":"2023-11-24T07:56:27.783252+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:56:23.683366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:56:23.683366+0000","last_clean_scrub_stamp":"2023-11-24T07:56:23.683366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T14:36:27.615721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"20'1","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T07:56:25.711989+0000","last_change":"2023-11-24T07:56:23.708583+0000","last_active":"2023-11-24T07:56:25.711989+0000","last_peered":"2023-11-24T07:56:25.711989+0000","last_clean":"2023-11-24T07:56:25.711989+0000","last_became_active":"2023-11-24T07:56:23.708291+0000","last_became_peered":"2023-11-24T07:56:23.708291+0000","last_unstale":"2023-11-24T07:56:25.711989+0000","last_undegraded":"2023-11-24T07:56:25.711989+0000","last_fullsized":"2023-11-24T07:56:25.711989+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:56:22.678019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:56:22.678019+0000","last_clean_scrub_stamp":"2023-11-24T07:56:22.678019+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T18:20:48.368545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T07:56:25.712005+0000","last_change":"2023-11-24T07:56:24.725877+0000","last_active":"2023-11-24T07:56:25.712005+0000","last_peered":"2023-11-24T07:56:25.712005+0000","last_clean":"2023-11-24T07:56:25.712005+0000","last_became_active":"2023-11-24T07:56:24.725622+0000","last_became_peered":"2023-11-24T07:56:24.725622+0000","last_unstale":"2023-11-24T07:56:25.712005+0000","last_undegraded":"2023-11-24T07:56:25.712005+0000","last_fullsized":"2023-11-24T07:56:25.712005+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:56:23.683366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:56:23.683366+0000","last_clean_scrub_stamp":"2023-11-24T07:56:23.683366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T17:43:47.155869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"19'2","reported_seq":14,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-24T07:56:24.745752+0000","last_change":"2023-11-24T07:56:23.713967+0000","last_active":"2023-11-24T07:56:24.745752+0000","last_peered":"2023-11-24T07:56:24.745752+0000","last_clean":"2023-11-24T07:56:24.745752+0000","last_became_active":"2023-11-24T07:56:23.713711+0000","last_became_peered":"2023-11-24T07:56:23.713711+0000","last_unstale":"2023-11-24T07:56:24.745752+0000","last_undegraded":"2023-11-24T07:56:24.745752+0000","last_fullsized":"2023-11-24T07:56:24.745752+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:56:22.678019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:56:22.678019+0000","last_clean_scrub_stamp":"2023-11-24T07:56:22.678019+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T09:29:56.578230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T07:56:25.712076+0000","last_change":"2023-11-24T07:56:24.724396+0000","last_active":"2023-11-24T07:56:25.712076+0000","last_peered":"2023-11-24T07:56:25.712076+0000","last_clean":"2023-11-24T07:56:25.712076+0000","last_became_active":"2023-11-24T07:56:24.724093+0000","last_became_peered":"2023-11-24T07:56:24.724093+0000","last_unstale":"2023-11-24T07:56:25.712076+0000","last_undegraded":"2023-11-24T07:56:25.712076+0000","last_fullsized":"2023-11-24T07:56:25.712076+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:56:23.683366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:56:23.683366+0000","last_clean_scrub_stamp":"2023-11-24T07:56:23.683366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T18:48:33.280136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":12,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-24T07:56:24.709839+0000","last_change":"2023-11-24T07:56:23.712459+0000","last_active":"2023-11-24T07:56:24.709839+0000","last_peered":"2023-11-24T07:56:24.709839+0000","last_clean":"2023-11-24T07:56:24.709839+0000","last_became_active":"2023-11-24T07:56:23.712194+0000","last_became_peered":"2023-11-24T07:56:23.712194+0000","last_unstale":"2023-11-24T07:56:24.709839+0000","last_undegraded":"2023-11-24T07:56:24.709839+0000","last_fullsized":"2023-11-24T07:56:24.709839+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:56:22.678019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:56:22.678019+0000","last_clean_scrub_stamp":"2023-11-24T07:56:22.678019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T17:19:43.614472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T07:56:26.101174+0000","last_change":"2023-11-24T07:56:24.725437+0000","last_active":"2023-11-24T07:56:26.101174+0000","last_peered":"2023-11-24T07:56:26.101174+0000","last_clean":"2023-11-24T07:56:26.101174+0000","last_became_active":"2023-11-24T07:56:24.724898+0000","last_became_peered":"2023-11-24T07:56:24.724898+0000","last_unstale":"2023-11-24T07:56:26.101174+0000","last_undegraded":"2023-11-24T07:56:26.101174+0000","last_fullsized":"2023-11-24T07:56:26.101174+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:56:23.683366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:56:23.683366+0000","last_clean_scrub_stamp":"2023-11-24T07:56:23.683366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T14:42:07.025247+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":12,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-24T07:56:24.721117+0000","last_change":"2023-11-24T07:56:23.708117+0000","last_active":"2023-11-24T07:56:24.721117+0000","last_peered":"2023-11-24T07:56:24.721117+0000","last_clean":"2023-11-24T07:56:24.721117+0000","last_became_active":"2023-11-24T07:56:23.707774+0000","last_became_peered":"2023-11-24T07:56:23.707774+0000","last_unstale":"2023-11-24T07:56:24.721117+0000","last_undegraded":"2023-11-24T07:56:24.721117+0000","last_fullsized":"2023-11-24T07:56:24.721117+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:56:22.678019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:56:22.678019+0000","last_clean_scrub_stamp":"2023-11-24T07:56:22.678019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T09:01:28.628833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T07:56:25.711408+0000","last_change":"2023-11-24T07:56:24.723772+0000","last_active":"2023-11-24T07:56:25.711408+0000","last_peered":"2023-11-24T07:56:25.711408+0000","last_clean":"2023-11-24T07:56:25.711408+0000","last_became_active":"2023-11-24T07:56:24.723497+0000","last_became_peered":"2023-11-24T07:56:24.723497+0000","last_unstale":"2023-11-24T07:56:25.711408+0000","last_undegraded":"2023-11-24T07:56:25.711408+0000","last_fullsized":"2023-11-24T07:56:25.711408+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:56:23.683366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:56:23.683366+0000","last_clean_scrub_stamp":"2023-11-24T07:56:23.683366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:14:33.126740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":12,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-24T07:56:24.711586+0000","last_change":"2023-11-24T07:56:23.704407+0000","last_active":"2023-11-24T07:56:24.711586+0000","last_peered":"2023-11-24T07:56:24.711586+0000","last_clean":"2023-11-24T07:56:24.711586+0000","last_became_active":"2023-11-24T07:56:23.704097+0000","last_became_peered":"2023-11-24T07:56:23.704097+0000","last_unstale":"2023-11-24T07:56:24.711586+0000","last_undegraded":"2023-11-24T07:56:24.711586+0000","last_fullsized":"2023-11-24T07:56:24.711586+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:56:22.678019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:56:22.678019+0000","last_clean_scrub_stamp":"2023-11-24T07:56:22.678019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T14:40:38.179620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T07:56:27.783059+0000","last_change":"2023-11-24T07:56:24.716998+0000","last_active":"2023-11-24T07:56:27.783059+0000","last_peered":"2023-11-24T07:56:27.783059+0000","last_clean":"2023-11-24T07:56:27.783059+0000","last_became_active":"2023-11-24T07:56:24.716713+0000","last_became_peered":"2023-11-24T07:56:24.716713+0000","last_unstale":"2023-11-24T07:56:27.783059+0000","last_undegraded":"2023-11-24T07:56:27.783059+0000","last_fullsized":"2023-11-24T07:56:27.783059+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:56:23.683366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:56:23.683366+0000","last_clean_scrub_stamp":"2023-11-24T07:56:23.683366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T19:32:53.988497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.b","version":"19'2","reported_seq":14,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-24T07:56:24.745886+0000","last_change":"2023-11-24T07:56:23.712730+0000","last_active":"2023-11-24T07:56:24.745886+0000","last_peered":"2023-11-24T07:56:24.745886+0000","last_clean":"2023-11-24T07:56:24.745886+0000","last_became_active":"2023-11-24T07:56:23.712423+0000","last_became_peered":"2023-11-24T07:56:23.712423+0000","last_unstale":"2023-11-24T07:56:24.745886+0000","last_undegraded":"2023-11-24T07:56:24.745886+0000","last_fullsized":"2023-11-24T07:56:24.745886+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:56:22.678019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:56:22.678019+0000","last_clean_scrub_stamp":"2023-11-24T07:56:22.678019+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T14:36:26.610375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T07:56:26.101381+0000","last_change":"2023-11-24T07:56:24.739297+0000","last_active":"2023-11-24T07:56:26.101381+0000","last_peered":"2023-11-24T07:56:26.101381+0000","last_clean":"2023-11-24T07:56:26.101381+0000","last_became_active":"2023-11-24T07:56:24.738991+0000","last_became_peered":"2023-11-24T07:56:24.738991+0000","last_unstale":"2023-11-24T07:56:26.101381+0000","last_undegraded":"2023-11-24T07:56:26.101381+0000","last_fullsized":"2023-11-24T07:56:26.101381+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:56:23.683366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:56:23.683366+0000","last_clean_scrub_stamp":"2023-11-24T07:56:23.683366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T09:26:22.526827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":12,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-24T07:56:24.712584+0000","last_change":"2023-11-24T07:56:23.709703+0000","last_active":"2023-11-24T07:56:24.712584+0000","last_peered":"2023-11-24T07:56:24.712584+0000","last_clean":"2023-11-24T07:56:24.712584+0000","last_became_active":"2023-11-24T07:56:23.709360+0000","last_became_peered":"2023-11-24T07:56:23.709360+0000","last_unstale":"2023-11-24T07:56:24.712584+0000","last_undegraded":"2023-11-24T07:56:24.712584+0000","last_fullsized":"2023-11-24T07:56:24.712584+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:56:22.678019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:56:22.678019+0000","last_clean_scrub_stamp":"2023-11-24T07:56:22.678019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T16:22:32.708879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T07:56:26.101579+0000","last_change":"2023-11-24T07:56:24.739181+0000","last_active":"2023-11-24T07:56:26.101579+0000","last_peered":"2023-11-24T07:56:26.101579+0000","last_clean":"2023-11-24T07:56:26.101579+0000","last_became_active":"2023-11-24T07:56:24.738864+0000","last_became_peered":"2023-11-24T07:56:24.738864+0000","last_unstale":"2023-11-24T07:56:26.101579+0000","last_undegraded":"2023-11-24T07:56:26.101579+0000","last_fullsized":"2023-11-24T07:56:26.101579+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:56:23.683366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:56:23.683366+0000","last_clean_scrub_stamp":"2023-11-24T07:56:23.683366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T17:52:32.557707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.d","version":"19'1","reported_seq":13,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-24T07:56:24.740226+0000","last_change":"2023-11-24T07:56:23.696616+0000","last_active":"2023-11-24T07:56:24.740226+0000","last_peered":"2023-11-24T07:56:24.740226+0000","last_clean":"2023-11-24T07:56:24.740226+0000","last_became_active":"2023-11-24T07:56:23.696320+0000","last_became_peered":"2023-11-24T07:56:23.696320+0000","last_unstale":"2023-11-24T07:56:24.740226+0000","last_undegraded":"2023-11-24T07:56:24.740226+0000","last_fullsized":"2023-11-24T07:56:24.740226+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:56:22.678019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:56:22.678019+0000","last_clean_scrub_stamp":"2023-11-24T07:56:22.678019+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T17:19:43.614472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T07:56:27.671768+0000","last_change":"2023-11-24T07:56:24.716634+0000","last_active":"2023-11-24T07:56:27.671768+0000","last_peered":"2023-11-24T07:56:27.671768+0000","last_clean":"2023-11-24T07:56:27.671768+0000","last_became_active":"2023-11-24T07:56:24.716211+0000","last_became_peered":"2023-11-24T07:56:24.716211+0000","last_unstale":"2023-11-24T07:56:27.671768+0000","last_undegraded":"2023-11-24T07:56:27.671768+0000","last_fullsized":"2023-11-24T07:56:27.671768+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:56:23.683366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:56:23.683366+0000","last_clean_scrub_stamp":"2023-11-24T07:56:23.683366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T15:43:54.658085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"19'2","reported_seq":14,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-24T07:56:24.736924+0000","last_change":"2023-11-24T07:56:23.703304+0000","last_active":"2023-11-24T07:56:24.736924+0000","last_peered":"2023-11-24T07:56:24.736924+0000","last_clean":"2023-11-24T07:56:24.736924+0000","last_became_active":"2023-11-24T07:56:23.703016+0000","last_became_peered":"2023-11-24T07:56:23.703016+0000","last_unstale":"2023-11-24T07:56:24.736924+0000","last_undegraded":"2023-11-24T07:56:24.736924+0000","last_fullsized":"2023-11-24T07:56:24.736924+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:56:22.678019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:56:22.678019+0000","last_clean_scrub_stamp":"2023-11-24T07:56:22.678019+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T14:36:26.610375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T07:56:27.672202+0000","last_change":"2023-11-24T07:56:24.735087+0000","last_active":"2023-11-24T07:56:27.672202+0000","last_peered":"2023-11-24T07:56:27.672202+0000","last_clean":"2023-11-24T07:56:27.672202+0000","last_became_active":"2023-11-24T07:56:24.734770+0000","last_became_peered":"2023-11-24T07:56:24.734770+0000","last_unstale":"2023-11-24T07:56:27.672202+0000","last_undegraded":"2023-11-24T07:56:27.672202+0000","last_fullsized":"2023-11-24T07:56:27.672202+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:56:23.683366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:56:23.683366+0000","last_clean_scrub_stamp":"2023-11-24T07:56:23.683366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T14:46:45.906414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T07:56:25.708692+0000","last_change":"2023-11-24T07:56:24.719171+0000","last_active":"2023-11-24T07:56:25.708692+0000","last_peered":"2023-11-24T07:56:25.708692+0000","last_clean":"2023-11-24T07:56:25.708692+0000","last_became_active":"2023-11-24T07:56:24.718062+0000","last_became_peered":"2023-11-24T07:56:24.718062+0000","last_unstale":"2023-11-24T07:56:25.708692+0000","last_undegraded":"2023-11-24T07:56:25.708692+0000","last_fullsized":"2023-11-24T07:56:25.708692+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:56:23.683366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:56:23.683366+0000","last_clean_scrub_stamp":"2023-11-24T07:56:23.683366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T18:53:14.850807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T07:56:27.782576+0000","last_change":"2023-11-24T07:56:24.734652+0000","last_active":"2023-11-24T07:56:27.782576+0000","last_peered":"2023-11-24T07:56:27.782576+0000","last_clean":"2023-11-24T07:56:27.782576+0000","last_became_active":"2023-11-24T07:56:24.734348+0000","last_became_peered":"2023-11-24T07:56:24.734348+0000","last_unstale":"2023-11-24T07:56:27.782576+0000","last_undegraded":"2023-11-24T07:56:27.782576+0000","last_fullsized":"2023-11-24T07:56:27.782576+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:56:23.683366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:56:23.683366+0000","last_clean_scrub_stamp":"2023-11-24T07:56:23.683366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T18:53:14.850807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T07:56:26.101190+0000","last_change":"2023-11-24T07:56:24.731378+0000","last_active":"2023-11-24T07:56:26.101190+0000","last_peered":"2023-11-24T07:56:26.101190+0000","last_clean":"2023-11-24T07:56:26.101190+0000","last_became_active":"2023-11-24T07:56:24.731096+0000","last_became_peered":"2023-11-24T07:56:24.731096+0000","last_unstale":"2023-11-24T07:56:26.101190+0000","last_undegraded":"2023-11-24T07:56:26.101190+0000","last_fullsized":"2023-11-24T07:56:26.101190+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:56:23.683366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:56:23.683366+0000","last_clean_scrub_stamp":"2023-11-24T07:56:23.683366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T14:46:45.906414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T07:56:25.712218+0000","last_change":"2023-11-24T07:56:24.723023+0000","last_active":"2023-11-24T07:56:25.712218+0000","last_peered":"2023-11-24T07:56:25.712218+0000","last_clean":"2023-11-24T07:56:25.712218+0000","last_became_active":"2023-11-24T07:56:24.722725+0000","last_became_peered":"2023-11-24T07:56:24.722725+0000","last_unstale":"2023-11-24T07:56:25.712218+0000","last_undegraded":"2023-11-24T07:56:25.712218+0000","last_fullsized":"2023-11-24T07:56:25.712218+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:56:23.683366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:56:23.683366+0000","last_clean_scrub_stamp":"2023-11-24T07:56:23.683366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T18:48:33.280136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T07:56:27.783164+0000","last_change":"2023-11-24T07:56:24.717012+0000","last_active":"2023-11-24T07:56:27.783164+0000","last_peered":"2023-11-24T07:56:27.783164+0000","last_clean":"2023-11-24T07:56:27.783164+0000","last_became_active":"2023-11-24T07:56:24.716377+0000","last_became_peered":"2023-11-24T07:56:24.716377+0000","last_unstale":"2023-11-24T07:56:27.783164+0000","last_undegraded":"2023-11-24T07:56:27.783164+0000","last_fullsized":"2023-11-24T07:56:27.783164+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:56:23.683366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:56:23.683366+0000","last_clean_scrub_stamp":"2023-11-24T07:56:23.683366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T19:05:27.061622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T07:56:27.783295+0000","last_change":"2023-11-24T07:56:24.718395+0000","last_active":"2023-11-24T07:56:27.783295+0000","last_peered":"2023-11-24T07:56:27.783295+0000","last_clean":"2023-11-24T07:56:27.783295+0000","last_became_active":"2023-11-24T07:56:24.718132+0000","last_became_peered":"2023-11-24T07:56:24.718132+0000","last_unstale":"2023-11-24T07:56:27.783295+0000","last_undegraded":"2023-11-24T07:56:27.783295+0000","last_fullsized":"2023-11-24T07:56:27.783295+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:56:23.683366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:56:23.683366+0000","last_clean_scrub_stamp":"2023-11-24T07:56:23.683366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:45:41.378607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T07:56:25.709271+0000","last_change":"2023-11-24T07:56:24.731485+0000","last_active":"2023-11-24T07:56:25.709271+0000","last_peered":"2023-11-24T07:56:25.709271+0000","last_clean":"2023-11-24T07:56:25.709271+0000","last_became_active":"2023-11-24T07:56:24.731163+0000","last_became_peered":"2023-11-24T07:56:24.731163+0000","last_unstale":"2023-11-24T07:56:25.709271+0000","last_undegraded":"2023-11-24T07:56:25.709271+0000","last_fullsized":"2023-11-24T07:56:25.709271+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:56:23.683366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:56:23.683366+0000","last_clean_scrub_stamp":"2023-11-24T07:56:23.683366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T19:43:40.193280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T07:56:25.712117+0000","last_change":"2023-11-24T07:56:24.733823+0000","last_active":"2023-11-24T07:56:25.712117+0000","last_peered":"2023-11-24T07:56:25.712117+0000","last_clean":"2023-11-24T07:56:25.712117+0000","last_became_active":"2023-11-24T07:56:24.733550+0000","last_became_peered":"2023-11-24T07:56:24.733550+0000","last_unstale":"2023-11-24T07:56:25.712117+0000","last_undegraded":"2023-11-24T07:56:25.712117+0000","last_fullsized":"2023-11-24T07:56:25.712117+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:56:23.683366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:56:23.683366+0000","last_clean_scrub_stamp":"2023-11-24T07:56:23.683366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T09:29:57.583576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T07:56:26.101006+0000","last_change":"2023-11-24T07:56:24.728674+0000","last_active":"2023-11-24T07:56:26.101006+0000","last_peered":"2023-11-24T07:56:26.101006+0000","last_clean":"2023-11-24T07:56:26.101006+0000","last_became_active":"2023-11-24T07:56:24.728157+0000","last_became_peered":"2023-11-24T07:56:24.728157+0000","last_unstale":"2023-11-24T07:56:26.101006+0000","last_undegraded":"2023-11-24T07:56:26.101006+0000","last_fullsized":"2023-11-24T07:56:26.101006+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:56:23.683366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:56:23.683366+0000","last_clean_scrub_stamp":"2023-11-24T07:56:23.683366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T13:56:07.967730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T07:56:25.712536+0000","last_change":"2023-11-24T07:56:24.722224+0000","last_active":"2023-11-24T07:56:25.712536+0000","last_peered":"2023-11-24T07:56:25.712536+0000","last_clean":"2023-11-24T07:56:25.712536+0000","last_became_active":"2023-11-24T07:56:24.721954+0000","last_became_peered":"2023-11-24T07:56:24.721954+0000","last_unstale":"2023-11-24T07:56:25.712536+0000","last_undegraded":"2023-11-24T07:56:25.712536+0000","last_fullsized":"2023-11-24T07:56:25.712536+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:56:23.683366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:56:23.683366+0000","last_clean_scrub_stamp":"2023-11-24T07:56:23.683366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T15:43:54.658085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T07:56:27.782867+0000","last_change":"2023-11-24T07:56:24.734996+0000","last_active":"2023-11-24T07:56:27.782867+0000","last_peered":"2023-11-24T07:56:27.782867+0000","last_clean":"2023-11-24T07:56:27.782867+0000","last_became_active":"2023-11-24T07:56:24.734198+0000","last_became_peered":"2023-11-24T07:56:24.734198+0000","last_unstale":"2023-11-24T07:56:27.782867+0000","last_undegraded":"2023-11-24T07:56:27.782867+0000","last_fullsized":"2023-11-24T07:56:27.782867+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:56:23.683366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:56:23.683366+0000","last_clean_scrub_stamp":"2023-11-24T07:56:23.683366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T19:32:53.988497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T07:56:25.712025+0000","last_change":"2023-11-24T07:56:24.727463+0000","last_active":"2023-11-24T07:56:25.712025+0000","last_peered":"2023-11-24T07:56:25.712025+0000","last_clean":"2023-11-24T07:56:25.712025+0000","last_became_active":"2023-11-24T07:56:24.727033+0000","last_became_peered":"2023-11-24T07:56:24.727033+0000","last_unstale":"2023-11-24T07:56:25.712025+0000","last_undegraded":"2023-11-24T07:56:25.712025+0000","last_fullsized":"2023-11-24T07:56:25.712025+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:56:23.683366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:56:23.683366+0000","last_clean_scrub_stamp":"2023-11-24T07:56:23.683366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T09:29:57.583576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T07:56:27.671926+0000","last_change":"2023-11-24T07:56:24.734645+0000","last_active":"2023-11-24T07:56:27.671926+0000","last_peered":"2023-11-24T07:56:27.671926+0000","last_clean":"2023-11-24T07:56:27.671926+0000","last_became_active":"2023-11-24T07:56:24.734367+0000","last_became_peered":"2023-11-24T07:56:24.734367+0000","last_unstale":"2023-11-24T07:56:27.671926+0000","last_undegraded":"2023-11-24T07:56:27.671926+0000","last_fullsized":"2023-11-24T07:56:27.671926+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:56:23.683366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:56:23.683366+0000","last_clean_scrub_stamp":"2023-11-24T07:56:23.683366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T19:48:41.181670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574852,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27216,"kb_used_data":196,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344624,"statfs":{"total":96636764160,"available":96608894976,"internally_reserved":0,"allocated":200704,"data_stored":64734,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48899999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.33600000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.35799999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.38500000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52100000000000002}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55000000000000004}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59299999999999997}]}]},{"osd":6,"up_from":13,"seq":55834574853,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27208,"kb_used_data":188,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344632,"statfs":{"total":96636764160,"available":96608903168,"internally_reserved":0,"allocated":192512,"data_stored":64610,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.45200000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.43099999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58999999999999997}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57099999999999995}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67800000000000005}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64500000000000002}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66400000000000003}]}]},{"osd":1,"up_from":13,"seq":55834574854,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27804,"kb_used_data":784,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344036,"statfs":{"total":96636764160,"available":96608292864,"internally_reserved":0,"allocated":802816,"data_stored":661348,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.433}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.36699999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46000000000000002}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64500000000000002}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66800000000000004}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61299999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.627}]}]},{"osd":0,"up_from":13,"seq":55834574854,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27228,"kb_used_data":208,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344612,"statfs":{"total":96636764160,"available":96608882688,"internally_reserved":0,"allocated":212992,"data_stored":70966,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.439}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.35799999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.373}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.69799999999999995}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.70999999999999996}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.72799999999999998}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.74399999999999999}]}]},{"osd":2,"up_from":13,"seq":55834574854,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27236,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344604,"statfs":{"total":96636764160,"available":96608874496,"internally_reserved":0,"allocated":217088,"data_stored":72522,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46000000000000002}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.35999999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.442}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.626}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55500000000000005}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.76500000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58899999999999997}]}]},{"osd":3,"up_from":13,"seq":55834574854,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27228,"kb_used_data":204,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344612,"statfs":{"total":96636764160,"available":96608882688,"internally_reserved":0,"allocated":208896,"data_stored":71492,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54600000000000004}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.50600000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.78700000000000003}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.80600000000000005}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.81899999999999995}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.74099999999999999}]}]},{"osd":4,"up_from":13,"seq":55834574853,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27244,"kb_used_data":220,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344596,"statfs":{"total":96636764160,"available":96608866304,"internally_reserved":0,"allocated":225280,"data_stored":73194,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.72099999999999997}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59199999999999997}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68899999999999995}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.79600000000000004}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65100000000000002}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.76800000000000002}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.89300000000000002}]}]},{"osd":5,"up_from":13,"seq":55834574854,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27812,"kb_used_data":788,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344028,"statfs":{"total":96636764160,"available":96608284672,"internally_reserved":0,"allocated":806912,"data_stored":661370,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60199999999999998}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51400000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66400000000000003}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64500000000000002}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.38200000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.41599999999999998}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53200000000000003}]}]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":2284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":124,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-11-24T07:56:33.878 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2023-11-24T07:56:33.878 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2023-11-24T07:56:33.878 INFO:tasks.ceph.ceph_manager.ceph:waiting for mgr available 2023-11-24T07:56:33.879 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json 2023-11-24T07:56:33.879 INFO:tasks.daemonwatchdog.daemon_watchdog:watchdog starting 2023-11-24T07:56:34.040 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:34.038+0000 7f563ada7700 1 Processor -- start 2023-11-24T07:56:34.040 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:34.038+0000 7f563ada7700 1 -- start start 2023-11-24T07:56:34.044 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:34.039+0000 7f563ada7700 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f56340cf2b0 0x7f563415c1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:56:34.044 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:34.039+0000 7f563ada7700 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f56340ce5c0 0x7f56340cea70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:56:34.045 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:34.039+0000 7f563ada7700 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f56340d0a90 con 0x7f563415bde0 2023-11-24T07:56:34.045 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:34.039+0000 7f563ada7700 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f56340cefb0 con 0x7f56340ce5c0 2023-11-24T07:56:34.045 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:34.039+0000 7f563ada7700 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f563406ee10 con 0x7f56340cf2b0 2023-11-24T07:56:34.045 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:34.039+0000 7f5638b43700 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7f563415bde0 0x7f56340d01b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:53342/0 (socket says 172.21.15.59:53342) 2023-11-24T07:56:34.046 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:34.039+0000 7f5638b43700 1 -- 172.21.15.59:0/3621728268 learned_addr learned my addr 172.21.15.59:0/3621728268 (peer_addr_for_me v1:172.21.15.59:0/0) 2023-11-24T07:56:34.046 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:34.040+0000 7f5633fff700 1 --2- 172.21.15.59:0/3621728268 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f56340cf2b0 0x7f563415c1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:56:34.047 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:34.040+0000 7f5639344700 1 --2- 172.21.15.59:0/3621728268 >> v2:172.21.15.76:3300/0 conn(0x7f56340ce5c0 0x7f56340cea70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:56:34.047 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:34.040+0000 7f56337fe700 1 -- 172.21.15.59:0/3621728268 <== mon.0 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1948399911 0 0) 0x7f56340d0a90 con 0x7f563415bde0 2023-11-24T07:56:34.047 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:34.040+0000 7f56337fe700 1 -- 172.21.15.59:0/3621728268 --> v1:172.21.15.59:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f56240035b0 con 0x7f563415bde0 2023-11-24T07:56:34.047 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:34.040+0000 7f56337fe700 1 -- 172.21.15.59:0/3621728268 <== mon.0 v1:172.21.15.59:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3078623247 0 0) 0x7f56240035b0 con 0x7f563415bde0 2023-11-24T07:56:34.048 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:34.040+0000 7f56337fe700 1 -- 172.21.15.59:0/3621728268 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f56340cf2b0 msgr2=0x7f563415c1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T07:56:34.048 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:34.040+0000 7f56337fe700 1 --2- 172.21.15.59:0/3621728268 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f56340cf2b0 0x7f563415c1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:56:34.048 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:34.040+0000 7f56337fe700 1 -- 172.21.15.59:0/3621728268 >> v2:172.21.15.76:3300/0 conn(0x7f56340ce5c0 msgr2=0x7f56340cea70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T07:56:34.048 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:34.040+0000 7f56337fe700 1 --2- 172.21.15.59:0/3621728268 >> v2:172.21.15.76:3300/0 conn(0x7f56340ce5c0 0x7f56340cea70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:56:34.048 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:34.040+0000 7f56337fe700 1 -- 172.21.15.59:0/3621728268 --> v1:172.21.15.59:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f563406f0f0 con 0x7f563415bde0 2023-11-24T07:56:34.048 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:34.040+0000 7f56337fe700 1 -- 172.21.15.59:0/3621728268 <== mon.0 v1:172.21.15.59:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1312333566 0 0) 0x7f5628001670 con 0x7f563415bde0 2023-11-24T07:56:34.048 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:34.040+0000 7f56337fe700 1 -- 172.21.15.59:0/3621728268 >> v1:172.21.15.59:6789/0 conn(0x7f563415bde0 legacy=0x7f56340d01b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:56:34.049 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:34.040+0000 7f56337fe700 1 --2- 172.21.15.59:0/3621728268 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f56340cf2b0 0x7f5624003a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:56:34.049 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:34.040+0000 7f56337fe700 1 --2- 172.21.15.59:0/3621728268 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5624003f80 0x7f5624004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:56:34.049 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:34.040+0000 7f56337fe700 1 --2- 172.21.15.59:0/3621728268 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5624004d90 0x7f5624005210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:56:34.049 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:34.040+0000 7f56337fe700 1 -- 172.21.15.59:0/3621728268 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56340cefb0 con 0x7f5624003f80 2023-11-24T07:56:34.049 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:34.040+0000 7f56337fe700 1 -- 172.21.15.59:0/3621728268 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f563406ee10 con 0x7f5624004d90 2023-11-24T07:56:34.049 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:34.040+0000 7f56337fe700 1 -- 172.21.15.59:0/3621728268 --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f562400ba60 con 0x7f56340cf2b0 2023-11-24T07:56:34.049 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:34.040+0000 7f5633fff700 1 --2- 172.21.15.59:0/3621728268 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5624003f80 0x7f5624004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:56:34.049 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:34.040+0000 7f5638b43700 1 --2- 172.21.15.59:0/3621728268 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f56340cf2b0 0x7f5624003a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:56:34.049 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:34.040+0000 7f5639344700 1 --2- 172.21.15.59:0/3621728268 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5624004d90 0x7f5624005210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:56:34.049 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:34.040+0000 7f5633fff700 1 -- 172.21.15.59:0/3621728268 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f56340cf2b0 msgr2=0x7f5624003a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T07:56:34.049 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:34.040+0000 7f5633fff700 1 --2- 172.21.15.59:0/3621728268 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f56340cf2b0 0x7f5624003a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:56:34.050 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:34.040+0000 7f5633fff700 1 -- 172.21.15.59:0/3621728268 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5624004d90 msgr2=0x7f5624005210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T07:56:34.050 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:34.040+0000 7f5633fff700 1 --2- 172.21.15.59:0/3621728268 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5624004d90 0x7f5624005210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:56:34.050 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:34.040+0000 7f5633fff700 1 -- 172.21.15.59:0/3621728268 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f56240035b0 con 0x7f5624003f80 2023-11-24T07:56:34.050 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:34.041+0000 7f5633fff700 1 --2- 172.21.15.59:0/3621728268 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5624003f80 0x7f5624004850 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f5620009b60 tx=0x7f5624005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6d69e244cf666f6c server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:56:34.050 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:34.041+0000 7f56337fe700 1 -- 172.21.15.59:0/3621728268 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f562000e6c0 con 0x7f5624003f80 2023-11-24T07:56:34.050 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:34.041+0000 7f56337fe700 1 -- 172.21.15.59:0/3621728268 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f562000e820 con 0x7f5624003f80 2023-11-24T07:56:34.050 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:34.041+0000 7f56337fe700 1 -- 172.21.15.59:0/3621728268 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f562000eb90 con 0x7f5624003f80 2023-11-24T07:56:34.050 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:34.041+0000 7f563ada7700 1 -- 172.21.15.59:0/3621728268 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5624003f80 msgr2=0x7f5624004850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:56:34.050 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:34.041+0000 7f563ada7700 1 --2- 172.21.15.59:0/3621728268 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5624003f80 0x7f5624004850 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f5620009b60 tx=0x7f5624005880 comp rx=0 tx=0).stop 2023-11-24T07:56:34.050 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:34.041+0000 7f5639344700 1 -- 172.21.15.59:0/3621728268 reap_dead start 2023-11-24T07:56:34.050 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:34.041+0000 7f563ada7700 1 -- 172.21.15.59:0/3621728268 shutdown_connections 2023-11-24T07:56:34.050 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:34.041+0000 7f563ada7700 1 -- 172.21.15.59:0/3621728268 >> 172.21.15.59:0/3621728268 conn(0x7f56340b3720 msgr2=0x7f56340b3b30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:56:34.051 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:34.041+0000 7f563ada7700 1 -- 172.21.15.59:0/3621728268 shutdown_connections 2023-11-24T07:56:34.051 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:34.041+0000 7f563ada7700 1 -- 172.21.15.59:0/3621728268 wait complete. 2023-11-24T07:56:34.051 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:34.042+0000 7f563ada7700 1 Processor -- start 2023-11-24T07:56:34.051 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:34.042+0000 7f563ada7700 1 -- start start 2023-11-24T07:56:34.053 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:34.043+0000 7f563ada7700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5624003f80 0x7f56340b1e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:56:34.053 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:34.043+0000 7f563ada7700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f56341f4e40 0x7f563415c6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:56:34.053 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:34.043+0000 7f563ada7700 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f56341492b0 0x7f56340b6820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:56:34.053 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:34.043+0000 7f563ada7700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56341f52c0 con 0x7f56341f4e40 2023-11-24T07:56:34.054 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:34.043+0000 7f563ada7700 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56340b2470 con 0x7f5624003f80 2023-11-24T07:56:34.054 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:34.043+0000 7f563ada7700 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f56340cab00 con 0x7f56341492b0 2023-11-24T07:56:34.054 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:34.043+0000 7f5633fff700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f56341f4e40 0x7f563415c6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:56:34.054 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:34.043+0000 7f5633fff700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f56341f4e40 0x7f563415c6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:49882/0 (socket says 172.21.15.59:49882) 2023-11-24T07:56:34.054 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:34.043+0000 7f5633fff700 1 -- 172.21.15.59:0/2447016445 learned_addr learned my addr 172.21.15.59:0/2447016445 (peer_addr_for_me v2:172.21.15.59:0/0) 2023-11-24T07:56:34.054 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:34.044+0000 7f5638b43700 1 --2- 172.21.15.59:0/2447016445 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5624003f80 0x7f56340b1e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:56:34.054 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:34.044+0000 7f5639344700 1 --2- 172.21.15.59:0/2447016445 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f56341492b0 0x7f56340b6820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:56:34.054 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:34.044+0000 7f5633fff700 1 -- 172.21.15.59:0/2447016445 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f56341492b0 msgr2=0x7f56340b6820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:56:34.054 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:34.044+0000 7f5633fff700 1 --2- 172.21.15.59:0/2447016445 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f56341492b0 0x7f56340b6820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:56:34.054 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:34.044+0000 7f5633fff700 1 -- 172.21.15.59:0/2447016445 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5624003f80 msgr2=0x7f56340b1e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:56:34.054 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:34.044+0000 7f5633fff700 1 --2- 172.21.15.59:0/2447016445 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5624003f80 0x7f56340b1e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:56:34.055 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:34.044+0000 7f5633fff700 1 -- 172.21.15.59:0/2447016445 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f56340cade0 con 0x7f56341f4e40 2023-11-24T07:56:34.055 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:34.044+0000 7f5633fff700 1 --2- 172.21.15.59:0/2447016445 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f56341f4e40 0x7f563415c6f0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f5624004790 tx=0x7f56200047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:56:34.055 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:34.044+0000 7f56317fa700 1 -- 172.21.15.59:0/2447016445 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f562000e6c0 con 0x7f56341f4e40 2023-11-24T07:56:34.055 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:34.044+0000 7f56317fa700 1 -- 172.21.15.59:0/2447016445 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5620013cf0 con 0x7f56341f4e40 2023-11-24T07:56:34.055 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:34.044+0000 7f563ada7700 1 -- 172.21.15.59:0/2447016445 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f56340cb0c0 con 0x7f56341f4e40 2023-11-24T07:56:34.055 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:34.044+0000 7f56317fa700 1 -- 172.21.15.59:0/2447016445 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5620028690 con 0x7f56341f4e40 2023-11-24T07:56:34.055 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:34.044+0000 7f563ada7700 1 -- 172.21.15.59:0/2447016445 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f56340ca720 con 0x7f56341f4e40 2023-11-24T07:56:34.055 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:34.045+0000 7f563ada7700 1 -- 172.21.15.59:0/2447016445 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f56340ca940 con 0x7f56341f4e40 2023-11-24T07:56:34.055 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:34.045+0000 7f56317fa700 1 -- 172.21.15.59:0/2447016445 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f562001c450 con 0x7f56341f4e40 2023-11-24T07:56:34.055 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:34.046+0000 7f56317fa700 1 --2- 172.21.15.59:0/2447016445 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f561806aed0 0x7f561806d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:56:34.055 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:34.046+0000 7f56317fa700 1 -- 172.21.15.59:0/2447016445 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f562008ada0 con 0x7f56341f4e40 2023-11-24T07:56:34.055 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:34.046+0000 7f5638b43700 1 --2- 172.21.15.59:0/2447016445 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f561806aed0 0x7f561806d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:56:34.055 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:34.046+0000 7f5638b43700 1 --2- 172.21.15.59:0/2447016445 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f561806aed0 0x7f561806d390 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f5628000c00 tx=0x7f5628007980 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:56:34.056 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:34.051+0000 7f56317fa700 1 -- 172.21.15.59:0/2447016445 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f56200560b0 con 0x7f56341f4e40 2023-11-24T07:56:34.316 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:34.314+0000 7f563ada7700 1 -- 172.21.15.59:0/2447016445 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json"} v 0) v1 -- 0x7f5634072f10 con 0x7f56341f4e40 2023-11-24T07:56:34.318 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:34.317+0000 7f56317fa700 1 -- 172.21.15.59:0/2447016445 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json"}]=0 v5) v1 ==== 74+0+171032 (secure 0 0 0) 0x7f5620013e60 con 0x7f56341f4e40 2023-11-24T07:56:34.321 INFO:teuthology.orchestra.run.smithi059.stdout: 2023-11-24T07:56:34.322 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:34.320+0000 7f563ada7700 1 -- 172.21.15.59:0/2447016445 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f561806aed0 msgr2=0x7f561806d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:56:34.322 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:34.320+0000 7f563ada7700 1 --2- 172.21.15.59:0/2447016445 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f561806aed0 0x7f561806d390 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f5628000c00 tx=0x7f5628007980 comp rx=0 tx=0).stop 2023-11-24T07:56:34.322 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:34.320+0000 7f563ada7700 1 -- 172.21.15.59:0/2447016445 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f56341f4e40 msgr2=0x7f563415c6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:56:34.323 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:34.320+0000 7f563ada7700 1 --2- 172.21.15.59:0/2447016445 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f56341f4e40 0x7f563415c6f0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f5624004790 tx=0x7f56200047c0 comp rx=0 tx=0).stop 2023-11-24T07:56:34.323 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:34.320+0000 7f563ada7700 1 -- 172.21.15.59:0/2447016445 shutdown_connections 2023-11-24T07:56:34.323 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:34.320+0000 7f563ada7700 1 --2- 172.21.15.59:0/2447016445 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f561806aed0 0x7f561806d390 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:56:34.323 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:34.320+0000 7f563ada7700 1 --2- 172.21.15.59:0/2447016445 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f56341492b0 0x7f56340b6820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:56:34.323 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:34.320+0000 7f563ada7700 1 --2- 172.21.15.59:0/2447016445 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5624003f80 0x7f56340b1e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:56:34.323 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:34.320+0000 7f563ada7700 1 --2- 172.21.15.59:0/2447016445 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f56341f4e40 0x7f563415c6f0 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:56:34.323 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:34.320+0000 7f563ada7700 1 -- 172.21.15.59:0/2447016445 >> 172.21.15.59:0/2447016445 conn(0x7f56340b3720 msgr2=0x7f56340b0580 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:56:34.323 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:34.320+0000 7f563ada7700 1 -- 172.21.15.59:0/2447016445 shutdown_connections 2023-11-24T07:56:34.323 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:34.320+0000 7f563ada7700 1 -- 172.21.15.59:0/2447016445 wait complete. 2023-11-24T07:56:34.334 INFO:teuthology.orchestra.run.smithi059.stdout:{"epoch":5,"active_gid":4100,"active_name":"x","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6832","nonce":4264046536},{"type":"v1","addr":"172.21.15.76:6833","nonce":4264046536}]},"active_addr":"172.21.15.76:6833/4264046536","active_change":"2023-11-24T07:56:12.289885+0000","active_mgr_features":4540138322906710015,"available":true,"standbys":[{"gid":4110,"name":"y","mgr_features":4540138322906710015,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.7","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:latest","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"secs","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"drive_group_interval":{"name":"drive_group_interval","type":"float","level":"advanced","flags":0,"default_value":"300.0","min":"","max":"","enum_allowed":[],"desc":"interval in seconds between re-application of applied drive_groups","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.7","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:latest","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"secs","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"drive_group_interval":{"name":"drive_group_interval","type":"float","level":"advanced","flags":0,"default_value":"300.0","min":"","max":"","enum_allowed":[],"desc":"interval in seconds between re-application of applied drive_groups","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":0,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.76:0","nonce":3487617030}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.76:0","nonce":1101961311}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.76:0","nonce":3400921206}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.76:0","nonce":2748852627}]}]} 2023-11-24T07:56:34.337 INFO:tasks.ceph.ceph_manager.ceph:mgr available! 2023-11-24T07:56:34.337 INFO:tasks.ceph.ceph_manager.ceph:waiting for all up 2023-11-24T07:56:34.337 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-24T07:56:34.465 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:34.461+0000 7f2874916700 1 Processor -- start 2023-11-24T07:56:34.465 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:34.461+0000 7f2874916700 1 -- start start 2023-11-24T07:56:34.465 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:34.462+0000 7f2874916700 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f287006fc50 0x7f287006eef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:56:34.465 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:34.462+0000 7f2874916700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f28700b0e90 0x7f287006ca10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:56:34.465 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:34.462+0000 7f2874916700 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f28700d1230 con 0x7f287015e1c0 2023-11-24T07:56:34.465 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:34.462+0000 7f2874916700 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f28700d0c70 con 0x7f28700b0e90 2023-11-24T07:56:34.466 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:34.462+0000 7f2874916700 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f28700d0f50 con 0x7f287006fc50 2023-11-24T07:56:34.466 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:34.462+0000 7f286ed9d700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f28700b0e90 0x7f287006ca10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:56:34.466 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:34.462+0000 7f286e59c700 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7f287015e1c0 0x7f287006f540 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:53362/0 (socket says 172.21.15.59:53362) 2023-11-24T07:56:34.466 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:34.462+0000 7f286e59c700 1 -- 172.21.15.59:0/2249246027 learned_addr learned my addr 172.21.15.59:0/2249246027 (peer_addr_for_me v1:172.21.15.59:0/0) 2023-11-24T07:56:34.466 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:34.462+0000 7f286ed9d700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f28700b0e90 0x7f287006ca10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:49892/0 (socket says 172.21.15.59:49892) 2023-11-24T07:56:34.466 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:34.462+0000 7f286dd9b700 1 --2- 172.21.15.59:0/2249246027 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f287006fc50 0x7f287006eef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:56:34.466 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:34.462+0000 7f286ed9d700 1 -- 172.21.15.59:0/2249246027 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f287006fc50 msgr2=0x7f287006eef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T07:56:34.466 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:34.462+0000 7f286ed9d700 1 --2- 172.21.15.59:0/2249246027 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f287006fc50 0x7f287006eef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:56:34.466 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:34.462+0000 7f286ed9d700 1 -- 172.21.15.59:0/2249246027 >> v1:172.21.15.59:6789/0 conn(0x7f287015e1c0 legacy=0x7f287006f540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:56:34.466 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:34.462+0000 7f286ed9d700 1 -- 172.21.15.59:0/2249246027 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f287006c210 con 0x7f28700b0e90 2023-11-24T07:56:34.466 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:34.462+0000 7f286d59a700 1 -- 172.21.15.59:0/2249246027 <== mon.0 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3262667771 0 0) 0x7f28700d1230 con 0x7f287015e1c0 2023-11-24T07:56:34.466 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:34.463+0000 7f286ed9d700 1 --2- 172.21.15.59:0/2249246027 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f28700b0e90 0x7f287006ca10 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f2868009d90 tx=0x7f286800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3f5a645b117c993c server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:56:34.467 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:34.463+0000 7f286d59a700 1 -- 172.21.15.59:0/2249246027 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f286800e070 con 0x7f28700b0e90 2023-11-24T07:56:34.467 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:34.463+0000 7f286d59a700 1 -- 172.21.15.59:0/2249246027 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f286800bbd0 con 0x7f28700b0e90 2023-11-24T07:56:34.467 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:34.463+0000 7f286d59a700 1 -- 172.21.15.59:0/2249246027 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2868012430 con 0x7f28700b0e90 2023-11-24T07:56:34.467 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:34.463+0000 7f2874916700 1 -- 172.21.15.59:0/2249246027 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f28700b0e90 msgr2=0x7f287006ca10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:56:34.467 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:34.463+0000 7f2874916700 1 --2- 172.21.15.59:0/2249246027 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f28700b0e90 0x7f287006ca10 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f2868009d90 tx=0x7f286800b330 comp rx=0 tx=0).stop 2023-11-24T07:56:34.467 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:34.463+0000 7f2874916700 1 -- 172.21.15.59:0/2249246027 shutdown_connections 2023-11-24T07:56:34.467 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:34.463+0000 7f2874916700 1 --2- 172.21.15.59:0/2249246027 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f28700b0e90 0x7f287006ca10 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:56:34.468 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:34.463+0000 7f2874916700 1 --2- 172.21.15.59:0/2249246027 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f287006fc50 0x7f287006eef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:56:34.468 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:34.463+0000 7f2874916700 1 -- 172.21.15.59:0/2249246027 >> 172.21.15.59:0/2249246027 conn(0x7f28700b3720 msgr2=0x7f28700b3b30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:56:34.468 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:34.463+0000 7f2874916700 1 -- 172.21.15.59:0/2249246027 shutdown_connections 2023-11-24T07:56:34.468 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:34.463+0000 7f2874916700 1 -- 172.21.15.59:0/2249246027 wait complete. 2023-11-24T07:56:34.471 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:34.465+0000 7f2874916700 1 Processor -- start 2023-11-24T07:56:34.471 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:34.465+0000 7f2874916700 1 -- start start 2023-11-24T07:56:34.471 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:34.465+0000 7f2874916700 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f287006fc50 0x7f287009df30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:56:34.472 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:34.465+0000 7f2874916700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f28700b0e90 0x7f28700b6c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:56:34.472 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:34.465+0000 7f2874916700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f287015e1c0 0x7f2870157ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:56:34.472 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:34.465+0000 7f2874916700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f287009bc00 con 0x7f287015e1c0 2023-11-24T07:56:34.472 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:34.465+0000 7f2874916700 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f287009bee0 con 0x7f28700b0e90 2023-11-24T07:56:34.472 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:34.465+0000 7f2874916700 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f287009c1c0 con 0x7f287006fc50 2023-11-24T07:56:34.472 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:34.466+0000 7f286ed9d700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f287015e1c0 0x7f2870157ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:56:34.472 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:34.466+0000 7f286ed9d700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f287015e1c0 0x7f2870157ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:49896/0 (socket says 172.21.15.59:49896) 2023-11-24T07:56:34.472 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:34.466+0000 7f286ed9d700 1 -- 172.21.15.59:0/1370086304 learned_addr learned my addr 172.21.15.59:0/1370086304 (peer_addr_for_me v2:172.21.15.59:0/0) 2023-11-24T07:56:34.472 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:34.466+0000 7f286e59c700 1 --2- 172.21.15.59:0/1370086304 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f287006fc50 0x7f287009df30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:56:34.473 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:34.466+0000 7f286dd9b700 1 --2- 172.21.15.59:0/1370086304 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f28700b0e90 0x7f28700b6c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:56:34.473 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:34.466+0000 7f286ed9d700 1 -- 172.21.15.59:0/1370086304 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f287006fc50 msgr2=0x7f287009df30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:56:34.473 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:34.466+0000 7f286ed9d700 1 --2- 172.21.15.59:0/1370086304 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f287006fc50 0x7f287009df30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:56:34.481 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:34.466+0000 7f286ed9d700 1 -- 172.21.15.59:0/1370086304 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f28700b0e90 msgr2=0x7f28700b6c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:56:34.481 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:34.466+0000 7f286ed9d700 1 --2- 172.21.15.59:0/1370086304 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f28700b0e90 0x7f28700b6c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:56:34.481 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:34.466+0000 7f286ed9d700 1 -- 172.21.15.59:0/1370086304 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2868009a70 con 0x7f287015e1c0 2023-11-24T07:56:34.481 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:34.466+0000 7f286ed9d700 1 --2- 172.21.15.59:0/1370086304 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f287015e1c0 0x7f2870157ea0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f2868013000 tx=0x7f2868004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:56:34.481 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:34.466+0000 7f285effd700 1 -- 172.21.15.59:0/1370086304 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2868004880 con 0x7f287015e1c0 2023-11-24T07:56:34.481 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:34.466+0000 7f285effd700 1 -- 172.21.15.59:0/1370086304 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2868020070 con 0x7f287015e1c0 2023-11-24T07:56:34.481 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:34.466+0000 7f2874916700 1 -- 172.21.15.59:0/1370086304 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f287009c4a0 con 0x7f287015e1c0 2023-11-24T07:56:34.482 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:34.466+0000 7f285effd700 1 -- 172.21.15.59:0/1370086304 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28680276c0 con 0x7f287015e1c0 2023-11-24T07:56:34.482 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:34.466+0000 7f2874916700 1 -- 172.21.15.59:0/1370086304 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f287009c9c0 con 0x7f287015e1c0 2023-11-24T07:56:34.482 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:34.467+0000 7f2874916700 1 -- 172.21.15.59:0/1370086304 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2870068670 con 0x7f287015e1c0 2023-11-24T07:56:34.482 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:34.468+0000 7f285effd700 1 -- 172.21.15.59:0/1370086304 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f2868012430 con 0x7f287015e1c0 2023-11-24T07:56:34.482 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:34.468+0000 7f285effd700 1 --2- 172.21.15.59:0/1370086304 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f285006aed0 0x7f285006d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:56:34.482 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:34.468+0000 7f285effd700 1 -- 172.21.15.59:0/1370086304 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f2868022070 con 0x7f287015e1c0 2023-11-24T07:56:34.482 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:34.468+0000 7f286e59c700 1 --2- 172.21.15.59:0/1370086304 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f285006aed0 0x7f285006d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:56:34.483 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:34.469+0000 7f286e59c700 1 --2- 172.21.15.59:0/1370086304 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f285006aed0 0x7f285006d390 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f2864000f60 tx=0x7f2864007e70 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:56:34.483 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:34.473+0000 7f285effd700 1 -- 172.21.15.59:0/1370086304 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f2868056810 con 0x7f287015e1c0 2023-11-24T07:56:34.679 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:34.677+0000 7f2874916700 1 -- 172.21.15.59:0/1370086304 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f28700c88f0 con 0x7f287015e1c0 2023-11-24T07:56:34.680 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:34.679+0000 7f285effd700 1 -- 172.21.15.59:0/1370086304 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) v1 ==== 74+0+16437 (secure 0 0 0) 0x7f2868059e30 con 0x7f287015e1c0 2023-11-24T07:56:34.681 INFO:teuthology.orchestra.run.smithi059.stdout: 2023-11-24T07:56:34.681 INFO:teuthology.orchestra.run.smithi059.stdout:{"epoch":20,"fsid":"9f0e8741-b8f9-48d5-a2c3-09f45903303d","created":"2023-11-24T07:56:06.552729+0000","modified":"2023-11-24T07:56:25.705091+0000","last_up_change":"2023-11-24T07:56:17.372352+0000","last_in_change":"2023-11-24T07:56:09.073250+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-24T07:56:18.352823+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-11-24T07:56:22.185140+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-11-24T07:56:23.084326+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"6d8f9e9b-1782-4e41-928b-a39f2a8ebc06","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6800","nonce":3373878772},{"type":"v1","addr":"172.21.15.59:6801","nonce":3373878772}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6802","nonce":3373878772},{"type":"v1","addr":"172.21.15.59:6803","nonce":3373878772}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6806","nonce":3373878772},{"type":"v1","addr":"172.21.15.59:6807","nonce":3373878772}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6804","nonce":3373878772},{"type":"v1","addr":"172.21.15.59:6805","nonce":3373878772}]},"public_addr":"172.21.15.59:6801/3373878772","cluster_addr":"172.21.15.59:6803/3373878772","heartbeat_back_addr":"172.21.15.59:6807/3373878772","heartbeat_front_addr":"172.21.15.59:6805/3373878772","state":["exists","up"]},{"osd":1,"uuid":"e8d882aa-52e1-4c04-9e1e-c44ef9204a5f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6824","nonce":648258563},{"type":"v1","addr":"172.21.15.59:6825","nonce":648258563}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6826","nonce":648258563},{"type":"v1","addr":"172.21.15.59:6827","nonce":648258563}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6830","nonce":648258563},{"type":"v1","addr":"172.21.15.59:6831","nonce":648258563}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6828","nonce":648258563},{"type":"v1","addr":"172.21.15.59:6829","nonce":648258563}]},"public_addr":"172.21.15.59:6825/648258563","cluster_addr":"172.21.15.59:6827/648258563","heartbeat_back_addr":"172.21.15.59:6831/648258563","heartbeat_front_addr":"172.21.15.59:6829/648258563","state":["exists","up"]},{"osd":2,"uuid":"69eebfa0-695b-4b21-90b9-8a3537168a01","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6808","nonce":1840430654},{"type":"v1","addr":"172.21.15.59:6809","nonce":1840430654}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6810","nonce":1840430654},{"type":"v1","addr":"172.21.15.59:6811","nonce":1840430654}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6814","nonce":1840430654},{"type":"v1","addr":"172.21.15.59:6815","nonce":1840430654}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6812","nonce":1840430654},{"type":"v1","addr":"172.21.15.59:6813","nonce":1840430654}]},"public_addr":"172.21.15.59:6809/1840430654","cluster_addr":"172.21.15.59:6811/1840430654","heartbeat_back_addr":"172.21.15.59:6815/1840430654","heartbeat_front_addr":"172.21.15.59:6813/1840430654","state":["exists","up"]},{"osd":3,"uuid":"96fd77fc-f911-46ff-9445-6f0be2daa8f4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6816","nonce":2504025850},{"type":"v1","addr":"172.21.15.59:6817","nonce":2504025850}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6818","nonce":2504025850},{"type":"v1","addr":"172.21.15.59:6819","nonce":2504025850}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6822","nonce":2504025850},{"type":"v1","addr":"172.21.15.59:6823","nonce":2504025850}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6820","nonce":2504025850},{"type":"v1","addr":"172.21.15.59:6821","nonce":2504025850}]},"public_addr":"172.21.15.59:6817/2504025850","cluster_addr":"172.21.15.59:6819/2504025850","heartbeat_back_addr":"172.21.15.59:6823/2504025850","heartbeat_front_addr":"172.21.15.59:6821/2504025850","state":["exists","up"]},{"osd":4,"uuid":"7ac29fca-b5b2-4c95-9d69-2f164cbbc708","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6816","nonce":2382463431},{"type":"v1","addr":"172.21.15.76:6818","nonce":2382463431}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6820","nonce":2382463431},{"type":"v1","addr":"172.21.15.76:6822","nonce":2382463431}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6828","nonce":2382463431},{"type":"v1","addr":"172.21.15.76:6830","nonce":2382463431}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6824","nonce":2382463431},{"type":"v1","addr":"172.21.15.76:6826","nonce":2382463431}]},"public_addr":"172.21.15.76:6818/2382463431","cluster_addr":"172.21.15.76:6822/2382463431","heartbeat_back_addr":"172.21.15.76:6830/2382463431","heartbeat_front_addr":"172.21.15.76:6826/2382463431","state":["exists","up"]},{"osd":5,"uuid":"485bf711-3851-4238-a4f4-51d52e0a9c58","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6800","nonce":329334308},{"type":"v1","addr":"172.21.15.76:6802","nonce":329334308}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6805","nonce":329334308},{"type":"v1","addr":"172.21.15.76:6807","nonce":329334308}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6814","nonce":329334308},{"type":"v1","addr":"172.21.15.76:6815","nonce":329334308}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6810","nonce":329334308},{"type":"v1","addr":"172.21.15.76:6812","nonce":329334308}]},"public_addr":"172.21.15.76:6802/329334308","cluster_addr":"172.21.15.76:6807/329334308","heartbeat_back_addr":"172.21.15.76:6815/329334308","heartbeat_front_addr":"172.21.15.76:6812/329334308","state":["exists","up"]},{"osd":6,"uuid":"72b3f2fc-1e4a-4e32-968d-003647a0286f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6801","nonce":3946579410},{"type":"v1","addr":"172.21.15.76:6803","nonce":3946579410}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6804","nonce":3946579410},{"type":"v1","addr":"172.21.15.76:6806","nonce":3946579410}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6811","nonce":3946579410},{"type":"v1","addr":"172.21.15.76:6813","nonce":3946579410}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6808","nonce":3946579410},{"type":"v1","addr":"172.21.15.76:6809","nonce":3946579410}]},"public_addr":"172.21.15.76:6803/3946579410","cluster_addr":"172.21.15.76:6806/3946579410","heartbeat_back_addr":"172.21.15.76:6813/3946579410","heartbeat_front_addr":"172.21.15.76:6809/3946579410","state":["exists","up"]},{"osd":7,"uuid":"1ddceb90-6c5d-4d1a-ad56-99cb877f8c70","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6817","nonce":841761538},{"type":"v1","addr":"172.21.15.76:6819","nonce":841761538}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6821","nonce":841761538},{"type":"v1","addr":"172.21.15.76:6823","nonce":841761538}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6829","nonce":841761538},{"type":"v1","addr":"172.21.15.76:6831","nonce":841761538}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6825","nonce":841761538},{"type":"v1","addr":"172.21.15.76:6827","nonce":841761538}]},"public_addr":"172.21.15.76:6819/841761538","cluster_addr":"172.21.15.76:6823/841761538","heartbeat_back_addr":"172.21.15.76:6831/841761538","heartbeat_front_addr":"172.21.15.76:6827/841761538","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T07:56:14.294838+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T07:56:14.270779+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T07:56:14.231809+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T07:56:14.260983+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T07:56:14.302997+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T07:56:14.258600+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T07:56:14.251413+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T07:56:14.304727+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-24T07:56:34.683 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:34.681+0000 7f2874916700 1 -- 172.21.15.59:0/1370086304 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f285006aed0 msgr2=0x7f285006d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:56:34.683 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:34.681+0000 7f2874916700 1 --2- 172.21.15.59:0/1370086304 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f285006aed0 0x7f285006d390 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f2864000f60 tx=0x7f2864007e70 comp rx=0 tx=0).stop 2023-11-24T07:56:34.683 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:34.681+0000 7f2874916700 1 -- 172.21.15.59:0/1370086304 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f287015e1c0 msgr2=0x7f2870157ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:56:34.683 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:34.681+0000 7f2874916700 1 --2- 172.21.15.59:0/1370086304 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f287015e1c0 0x7f2870157ea0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f2868013000 tx=0x7f2868004ad0 comp rx=0 tx=0).stop 2023-11-24T07:56:34.683 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:34.681+0000 7f2874916700 1 -- 172.21.15.59:0/1370086304 shutdown_connections 2023-11-24T07:56:34.683 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:34.681+0000 7f2874916700 1 --2- 172.21.15.59:0/1370086304 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f285006aed0 0x7f285006d390 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:56:34.683 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:34.681+0000 7f2874916700 1 --2- 172.21.15.59:0/1370086304 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f287015e1c0 0x7f2870157ea0 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:56:34.684 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:34.681+0000 7f2874916700 1 --2- 172.21.15.59:0/1370086304 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f287006fc50 0x7f287009df30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:56:34.684 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:34.681+0000 7f2874916700 1 --2- 172.21.15.59:0/1370086304 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f28700b0e90 0x7f28700b6c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:56:34.684 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:34.681+0000 7f2874916700 1 -- 172.21.15.59:0/1370086304 >> 172.21.15.59:0/1370086304 conn(0x7f28700b3720 msgr2=0x7f28700b3b00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:56:34.684 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:34.681+0000 7f2874916700 1 -- 172.21.15.59:0/1370086304 shutdown_connections 2023-11-24T07:56:34.684 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:34.681+0000 7f2874916700 1 -- 172.21.15.59:0/1370086304 wait complete. 2023-11-24T07:56:34.695 INFO:tasks.ceph.ceph_manager.ceph:all up! 2023-11-24T07:56:34.695 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-24T07:56:34.819 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:34.818+0000 7f13b7b7f700 1 Processor -- start 2023-11-24T07:56:34.820 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:34.818+0000 7f13b7b7f700 1 -- start start 2023-11-24T07:56:34.823 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:34.819+0000 7f13b7b7f700 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f13b0143ae0 0x7f13b00b51b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:56:34.823 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:34.819+0000 7f13b7b7f700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f13b01440f0 0x7f13b0071bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:56:34.823 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:34.819+0000 7f13b7b7f700 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f13b0143ec0 con 0x7f13b00720f0 2023-11-24T07:56:34.823 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:34.819+0000 7f13b7b7f700 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f13b00b40a0 con 0x7f13b01440f0 2023-11-24T07:56:34.823 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:34.819+0000 7f13b7b7f700 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f13b0072560 con 0x7f13b0143ae0 2023-11-24T07:56:34.823 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:34.819+0000 7f13b511a700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f13b01440f0 0x7f13b0071bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:56:34.823 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:34.819+0000 7f13b611c700 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7f13b00720f0 0x7f13b015e8b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:53380/0 (socket says 172.21.15.59:53380) 2023-11-24T07:56:34.823 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:34.819+0000 7f13b611c700 1 -- 172.21.15.59:0/3761831490 learned_addr learned my addr 172.21.15.59:0/3761831490 (peer_addr_for_me v1:172.21.15.59:0/0) 2023-11-24T07:56:34.824 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:34.819+0000 7f13b591b700 1 --2- 172.21.15.59:0/3761831490 >> v2:172.21.15.76:3300/0 conn(0x7f13b0143ae0 0x7f13b00b51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:56:34.824 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:34.819+0000 7f13b511a700 1 -- 172.21.15.59:0/3761831490 >> v2:172.21.15.76:3300/0 conn(0x7f13b0143ae0 msgr2=0x7f13b00b51b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T07:56:34.824 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:34.819+0000 7f13b511a700 1 --2- 172.21.15.59:0/3761831490 >> v2:172.21.15.76:3300/0 conn(0x7f13b0143ae0 0x7f13b00b51b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:56:34.824 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:34.819+0000 7f13b511a700 1 -- 172.21.15.59:0/3761831490 >> v1:172.21.15.59:6789/0 conn(0x7f13b00720f0 legacy=0x7f13b015e8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:56:34.824 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:34.819+0000 7f13b511a700 1 -- 172.21.15.59:0/3761831490 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f13b00b3ae0 con 0x7f13b01440f0 2023-11-24T07:56:34.824 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:34.819+0000 7f13b4919700 1 -- 172.21.15.59:0/3761831490 <== mon.0 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1492004432 0 0) 0x7f13b0143ec0 con 0x7f13b00720f0 2023-11-24T07:56:34.824 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:34.819+0000 7f13b511a700 1 --2- 172.21.15.59:0/3761831490 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f13b01440f0 0x7f13b0071bb0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f13b006d2e0 tx=0x7f13a0009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=327c049e0e36e6c1 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:56:34.824 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:34.819+0000 7f13b4919700 1 -- 172.21.15.59:0/3761831490 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13a0011830 con 0x7f13b01440f0 2023-11-24T07:56:34.824 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:34.820+0000 7f13b4919700 1 -- 172.21.15.59:0/3761831490 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f13a0011990 con 0x7f13b01440f0 2023-11-24T07:56:34.824 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:34.820+0000 7f13b4919700 1 -- 172.21.15.59:0/3761831490 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13a0011d20 con 0x7f13b01440f0 2023-11-24T07:56:34.824 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:34.820+0000 7f13b7b7f700 1 -- 172.21.15.59:0/3761831490 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f13b01440f0 msgr2=0x7f13b0071bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:56:34.824 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:34.820+0000 7f13b7b7f700 1 --2- 172.21.15.59:0/3761831490 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f13b01440f0 0x7f13b0071bb0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f13b006d2e0 tx=0x7f13a0009b50 comp rx=0 tx=0).stop 2023-11-24T07:56:34.825 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:34.820+0000 7f13b7b7f700 1 -- 172.21.15.59:0/3761831490 shutdown_connections 2023-11-24T07:56:34.825 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:34.820+0000 7f13b7b7f700 1 --2- 172.21.15.59:0/3761831490 >> v2:172.21.15.76:3300/0 conn(0x7f13b0143ae0 0x7f13b00b51b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:56:34.825 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:34.820+0000 7f13b7b7f700 1 --2- 172.21.15.59:0/3761831490 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f13b01440f0 0x7f13b0071bb0 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:56:34.825 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:34.820+0000 7f13b7b7f700 1 -- 172.21.15.59:0/3761831490 >> 172.21.15.59:0/3761831490 conn(0x7f13b00b2740 msgr2=0x7f13b00b2b20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:56:34.825 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:34.820+0000 7f13b7b7f700 1 -- 172.21.15.59:0/3761831490 shutdown_connections 2023-11-24T07:56:34.825 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:34.820+0000 7f13b7b7f700 1 -- 172.21.15.59:0/3761831490 wait complete. 2023-11-24T07:56:34.825 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:34.821+0000 7f13b7b7f700 1 Processor -- start 2023-11-24T07:56:34.825 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:34.821+0000 7f13b7b7f700 1 -- start start 2023-11-24T07:56:34.828 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:34.822+0000 7f13b7b7f700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f13b00720f0 0x7f13b01f6c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:56:34.828 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:34.822+0000 7f13b7b7f700 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f13b0143ae0 0x7f13b00cbcf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:56:34.828 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:34.822+0000 7f13b7b7f700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f13b01440f0 0x7f13b01e1a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:56:34.828 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:34.822+0000 7f13b7b7f700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13b00cd250 con 0x7f13b01440f0 2023-11-24T07:56:34.828 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:34.822+0000 7f13b7b7f700 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13b00cc640 con 0x7f13b00720f0 2023-11-24T07:56:34.828 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:34.822+0000 7f13b7b7f700 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f13b00cc920 con 0x7f13b0143ae0 2023-11-24T07:56:34.829 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:34.822+0000 7f13b611c700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f13b01440f0 0x7f13b01e1a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:56:34.829 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:34.822+0000 7f13b611c700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f13b01440f0 0x7f13b01e1a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:49912/0 (socket says 172.21.15.59:49912) 2023-11-24T07:56:34.829 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:34.822+0000 7f13b611c700 1 -- 172.21.15.59:0/791198714 learned_addr learned my addr 172.21.15.59:0/791198714 (peer_addr_for_me v2:172.21.15.59:0/0) 2023-11-24T07:56:34.829 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:34.822+0000 7f13b511a700 1 --2- 172.21.15.59:0/791198714 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f13b0143ae0 0x7f13b00cbcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:56:34.829 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:34.822+0000 7f13b611c700 1 -- 172.21.15.59:0/791198714 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f13b0143ae0 msgr2=0x7f13b00cbcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:56:34.829 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:34.822+0000 7f13b591b700 1 --2- 172.21.15.59:0/791198714 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f13b00720f0 0x7f13b01f6c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:56:34.829 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:34.822+0000 7f13b611c700 1 --2- 172.21.15.59:0/791198714 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f13b0143ae0 0x7f13b00cbcf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:56:34.829 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:34.822+0000 7f13b611c700 1 -- 172.21.15.59:0/791198714 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f13b00720f0 msgr2=0x7f13b01f6c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:56:34.830 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:34.822+0000 7f13b611c700 1 --2- 172.21.15.59:0/791198714 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f13b00720f0 0x7f13b01f6c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:56:34.830 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:34.822+0000 7f13b611c700 1 -- 172.21.15.59:0/791198714 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f13a00094a0 con 0x7f13b01440f0 2023-11-24T07:56:34.830 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:34.822+0000 7f13b511a700 1 --2- 172.21.15.59:0/791198714 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f13b0143ae0 0x7f13b00cbcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T07:56:34.830 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:34.823+0000 7f13b591b700 1 --2- 172.21.15.59:0/791198714 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f13b00720f0 0x7f13b01f6c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T07:56:34.830 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:34.823+0000 7f13b611c700 1 --2- 172.21.15.59:0/791198714 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f13b01440f0 0x7f13b01e1a20 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f13a800cb60 tx=0x7f13a800cf20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:56:34.830 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:34.823+0000 7f13a67fc700 1 -- 172.21.15.59:0/791198714 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13a800ab50 con 0x7f13b01440f0 2023-11-24T07:56:34.830 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:34.823+0000 7f13a67fc700 1 -- 172.21.15.59:0/791198714 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f13a800acb0 con 0x7f13b01440f0 2023-11-24T07:56:34.830 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:34.823+0000 7f13b7b7f700 1 -- 172.21.15.59:0/791198714 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f13b00ccc00 con 0x7f13b01440f0 2023-11-24T07:56:34.830 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:34.823+0000 7f13a67fc700 1 -- 172.21.15.59:0/791198714 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13a8007310 con 0x7f13b01440f0 2023-11-24T07:56:34.831 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:34.823+0000 7f13b7b7f700 1 -- 172.21.15.59:0/791198714 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f13b00cc230 con 0x7f13b01440f0 2023-11-24T07:56:34.831 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:34.824+0000 7f13b7b7f700 1 -- 172.21.15.59:0/791198714 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f13b0068670 con 0x7f13b01440f0 2023-11-24T07:56:34.831 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:34.824+0000 7f13a67fc700 1 -- 172.21.15.59:0/791198714 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f13a8007470 con 0x7f13b01440f0 2023-11-24T07:56:34.831 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:34.825+0000 7f13a67fc700 1 --2- 172.21.15.59:0/791198714 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f139406aed0 0x7f139406d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:56:34.831 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:34.825+0000 7f13a67fc700 1 -- 172.21.15.59:0/791198714 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f13a8089150 con 0x7f13b01440f0 2023-11-24T07:56:34.831 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:34.825+0000 7f13b591b700 1 --2- 172.21.15.59:0/791198714 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f139406aed0 0x7f139406d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:56:34.833 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:34.825+0000 7f13b591b700 1 --2- 172.21.15.59:0/791198714 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f139406aed0 0x7f139406d390 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f13ac006fd0 tx=0x7f13ac008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:56:34.833 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:34.829+0000 7f13a67fc700 1 -- 172.21.15.59:0/791198714 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f13a8054200 con 0x7f13b01440f0 2023-11-24T07:56:35.032 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.031+0000 7f13b7b7f700 1 -- 172.21.15.59:0/791198714 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f13b0144830 con 0x7f13b01440f0 2023-11-24T07:56:35.034 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.032+0000 7f13a67fc700 1 -- 172.21.15.59:0/791198714 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) v1 ==== 74+0+16437 (secure 0 0 0) 0x7f13a8057820 con 0x7f13b01440f0 2023-11-24T07:56:35.034 INFO:teuthology.orchestra.run.smithi059.stdout: 2023-11-24T07:56:35.034 INFO:teuthology.orchestra.run.smithi059.stdout:{"epoch":20,"fsid":"9f0e8741-b8f9-48d5-a2c3-09f45903303d","created":"2023-11-24T07:56:06.552729+0000","modified":"2023-11-24T07:56:25.705091+0000","last_up_change":"2023-11-24T07:56:17.372352+0000","last_in_change":"2023-11-24T07:56:09.073250+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-24T07:56:18.352823+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-11-24T07:56:22.185140+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-11-24T07:56:23.084326+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"6d8f9e9b-1782-4e41-928b-a39f2a8ebc06","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6800","nonce":3373878772},{"type":"v1","addr":"172.21.15.59:6801","nonce":3373878772}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6802","nonce":3373878772},{"type":"v1","addr":"172.21.15.59:6803","nonce":3373878772}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6806","nonce":3373878772},{"type":"v1","addr":"172.21.15.59:6807","nonce":3373878772}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6804","nonce":3373878772},{"type":"v1","addr":"172.21.15.59:6805","nonce":3373878772}]},"public_addr":"172.21.15.59:6801/3373878772","cluster_addr":"172.21.15.59:6803/3373878772","heartbeat_back_addr":"172.21.15.59:6807/3373878772","heartbeat_front_addr":"172.21.15.59:6805/3373878772","state":["exists","up"]},{"osd":1,"uuid":"e8d882aa-52e1-4c04-9e1e-c44ef9204a5f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6824","nonce":648258563},{"type":"v1","addr":"172.21.15.59:6825","nonce":648258563}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6826","nonce":648258563},{"type":"v1","addr":"172.21.15.59:6827","nonce":648258563}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6830","nonce":648258563},{"type":"v1","addr":"172.21.15.59:6831","nonce":648258563}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6828","nonce":648258563},{"type":"v1","addr":"172.21.15.59:6829","nonce":648258563}]},"public_addr":"172.21.15.59:6825/648258563","cluster_addr":"172.21.15.59:6827/648258563","heartbeat_back_addr":"172.21.15.59:6831/648258563","heartbeat_front_addr":"172.21.15.59:6829/648258563","state":["exists","up"]},{"osd":2,"uuid":"69eebfa0-695b-4b21-90b9-8a3537168a01","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6808","nonce":1840430654},{"type":"v1","addr":"172.21.15.59:6809","nonce":1840430654}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6810","nonce":1840430654},{"type":"v1","addr":"172.21.15.59:6811","nonce":1840430654}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6814","nonce":1840430654},{"type":"v1","addr":"172.21.15.59:6815","nonce":1840430654}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6812","nonce":1840430654},{"type":"v1","addr":"172.21.15.59:6813","nonce":1840430654}]},"public_addr":"172.21.15.59:6809/1840430654","cluster_addr":"172.21.15.59:6811/1840430654","heartbeat_back_addr":"172.21.15.59:6815/1840430654","heartbeat_front_addr":"172.21.15.59:6813/1840430654","state":["exists","up"]},{"osd":3,"uuid":"96fd77fc-f911-46ff-9445-6f0be2daa8f4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6816","nonce":2504025850},{"type":"v1","addr":"172.21.15.59:6817","nonce":2504025850}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6818","nonce":2504025850},{"type":"v1","addr":"172.21.15.59:6819","nonce":2504025850}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6822","nonce":2504025850},{"type":"v1","addr":"172.21.15.59:6823","nonce":2504025850}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6820","nonce":2504025850},{"type":"v1","addr":"172.21.15.59:6821","nonce":2504025850}]},"public_addr":"172.21.15.59:6817/2504025850","cluster_addr":"172.21.15.59:6819/2504025850","heartbeat_back_addr":"172.21.15.59:6823/2504025850","heartbeat_front_addr":"172.21.15.59:6821/2504025850","state":["exists","up"]},{"osd":4,"uuid":"7ac29fca-b5b2-4c95-9d69-2f164cbbc708","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6816","nonce":2382463431},{"type":"v1","addr":"172.21.15.76:6818","nonce":2382463431}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6820","nonce":2382463431},{"type":"v1","addr":"172.21.15.76:6822","nonce":2382463431}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6828","nonce":2382463431},{"type":"v1","addr":"172.21.15.76:6830","nonce":2382463431}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6824","nonce":2382463431},{"type":"v1","addr":"172.21.15.76:6826","nonce":2382463431}]},"public_addr":"172.21.15.76:6818/2382463431","cluster_addr":"172.21.15.76:6822/2382463431","heartbeat_back_addr":"172.21.15.76:6830/2382463431","heartbeat_front_addr":"172.21.15.76:6826/2382463431","state":["exists","up"]},{"osd":5,"uuid":"485bf711-3851-4238-a4f4-51d52e0a9c58","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6800","nonce":329334308},{"type":"v1","addr":"172.21.15.76:6802","nonce":329334308}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6805","nonce":329334308},{"type":"v1","addr":"172.21.15.76:6807","nonce":329334308}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6814","nonce":329334308},{"type":"v1","addr":"172.21.15.76:6815","nonce":329334308}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6810","nonce":329334308},{"type":"v1","addr":"172.21.15.76:6812","nonce":329334308}]},"public_addr":"172.21.15.76:6802/329334308","cluster_addr":"172.21.15.76:6807/329334308","heartbeat_back_addr":"172.21.15.76:6815/329334308","heartbeat_front_addr":"172.21.15.76:6812/329334308","state":["exists","up"]},{"osd":6,"uuid":"72b3f2fc-1e4a-4e32-968d-003647a0286f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6801","nonce":3946579410},{"type":"v1","addr":"172.21.15.76:6803","nonce":3946579410}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6804","nonce":3946579410},{"type":"v1","addr":"172.21.15.76:6806","nonce":3946579410}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6811","nonce":3946579410},{"type":"v1","addr":"172.21.15.76:6813","nonce":3946579410}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6808","nonce":3946579410},{"type":"v1","addr":"172.21.15.76:6809","nonce":3946579410}]},"public_addr":"172.21.15.76:6803/3946579410","cluster_addr":"172.21.15.76:6806/3946579410","heartbeat_back_addr":"172.21.15.76:6813/3946579410","heartbeat_front_addr":"172.21.15.76:6809/3946579410","state":["exists","up"]},{"osd":7,"uuid":"1ddceb90-6c5d-4d1a-ad56-99cb877f8c70","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6817","nonce":841761538},{"type":"v1","addr":"172.21.15.76:6819","nonce":841761538}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6821","nonce":841761538},{"type":"v1","addr":"172.21.15.76:6823","nonce":841761538}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6829","nonce":841761538},{"type":"v1","addr":"172.21.15.76:6831","nonce":841761538}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6825","nonce":841761538},{"type":"v1","addr":"172.21.15.76:6827","nonce":841761538}]},"public_addr":"172.21.15.76:6819/841761538","cluster_addr":"172.21.15.76:6823/841761538","heartbeat_back_addr":"172.21.15.76:6831/841761538","heartbeat_front_addr":"172.21.15.76:6827/841761538","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T07:56:14.294838+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T07:56:14.270779+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T07:56:14.231809+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T07:56:14.260983+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T07:56:14.302997+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T07:56:14.258600+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T07:56:14.251413+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T07:56:14.304727+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-24T07:56:35.036 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.034+0000 7f13b7b7f700 1 -- 172.21.15.59:0/791198714 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f139406aed0 msgr2=0x7f139406d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:56:35.037 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.034+0000 7f13b7b7f700 1 --2- 172.21.15.59:0/791198714 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f139406aed0 0x7f139406d390 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f13ac006fd0 tx=0x7f13ac008040 comp rx=0 tx=0).stop 2023-11-24T07:56:35.037 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.034+0000 7f13b7b7f700 1 -- 172.21.15.59:0/791198714 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f13b01440f0 msgr2=0x7f13b01e1a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:56:35.037 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.034+0000 7f13b7b7f700 1 --2- 172.21.15.59:0/791198714 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f13b01440f0 0x7f13b01e1a20 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f13a800cb60 tx=0x7f13a800cf20 comp rx=0 tx=0).stop 2023-11-24T07:56:35.037 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.034+0000 7f13b7b7f700 1 -- 172.21.15.59:0/791198714 shutdown_connections 2023-11-24T07:56:35.037 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.034+0000 7f13b7b7f700 1 --2- 172.21.15.59:0/791198714 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f139406aed0 0x7f139406d390 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:56:35.037 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.034+0000 7f13b7b7f700 1 --2- 172.21.15.59:0/791198714 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f13b01440f0 0x7f13b01e1a20 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:56:35.038 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.034+0000 7f13b7b7f700 1 --2- 172.21.15.59:0/791198714 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f13b00720f0 0x7f13b01f6c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:56:35.038 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.034+0000 7f13b7b7f700 1 --2- 172.21.15.59:0/791198714 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f13b0143ae0 0x7f13b00cbcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:56:35.038 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.034+0000 7f13b7b7f700 1 -- 172.21.15.59:0/791198714 >> 172.21.15.59:0/791198714 conn(0x7f13b00b2740 msgr2=0x7f13b006e660 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:56:35.038 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.034+0000 7f13b7b7f700 1 -- 172.21.15.59:0/791198714 shutdown_connections 2023-11-24T07:56:35.038 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.034+0000 7f13b7b7f700 1 -- 172.21.15.59:0/791198714 wait complete. 2023-11-24T07:56:35.048 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 flush_pg_stats 2023-11-24T07:56:35.049 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 flush_pg_stats 2023-11-24T07:56:35.049 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 flush_pg_stats 2023-11-24T07:56:35.050 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 flush_pg_stats 2023-11-24T07:56:35.050 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 flush_pg_stats 2023-11-24T07:56:35.051 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 flush_pg_stats 2023-11-24T07:56:35.051 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 flush_pg_stats 2023-11-24T07:56:35.051 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 flush_pg_stats 2023-11-24T07:56:35.262 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.260+0000 7f57c5603700 1 Processor -- start 2023-11-24T07:56:35.262 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.260+0000 7f57c5603700 1 -- start start 2023-11-24T07:56:35.263 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.261+0000 7f57c5603700 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f57c01440f0 0x7f57c00b51b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:56:35.263 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.261+0000 7f57c5603700 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f57c0143ec0 con 0x7f57c00722c0 2023-11-24T07:56:35.263 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.261+0000 7f57c5603700 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f57c00b3ae0 con 0x7f57c0143ae0 2023-11-24T07:56:35.263 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.261+0000 7f57c5603700 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f57c00b40a0 con 0x7f57c01440f0 2023-11-24T07:56:35.263 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.261+0000 7f57be7fc700 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7f57c00722c0 0x7f57c006e510 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:53392/0 (socket says 172.21.15.59:53392) 2023-11-24T07:56:35.263 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.261+0000 7f57be7fc700 1 -- 172.21.15.59:0/534545651 learned_addr learned my addr 172.21.15.59:0/534545651 (peer_addr_for_me v1:172.21.15.59:0/0) 2023-11-24T07:56:35.265 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.262+0000 7f57bdffb700 1 -- 172.21.15.59:0/534545651 <== mon.0 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3519308231 0 0) 0x7f57c0143ec0 con 0x7f57c00722c0 2023-11-24T07:56:35.265 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.262+0000 7f57bdffb700 1 -- 172.21.15.59:0/534545651 --> v1:172.21.15.59:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f57a80035b0 con 0x7f57c00722c0 2023-11-24T07:56:35.266 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.262+0000 7f57bdffb700 1 -- 172.21.15.59:0/534545651 <== mon.1 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4250412156 0 0) 0x7f57c00b3ae0 con 0x7f57c0143ae0 2023-11-24T07:56:35.266 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.262+0000 7f57bdffb700 1 -- 172.21.15.59:0/534545651 --> v1:172.21.15.76:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f57c0143ec0 con 0x7f57c0143ae0 2023-11-24T07:56:35.266 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.262+0000 7f57bdffb700 1 -- 172.21.15.59:0/534545651 <== mon.0 v1:172.21.15.59:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1512840470 0 0) 0x7f57a80035b0 con 0x7f57c00722c0 2023-11-24T07:56:35.266 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.262+0000 7f57bdffb700 1 -- 172.21.15.59:0/534545651 >> v2:172.21.15.76:3300/0 conn(0x7f57c01440f0 msgr2=0x7f57c00b51b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T07:56:35.266 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.262+0000 7f57bdffb700 1 --2- 172.21.15.59:0/534545651 >> v2:172.21.15.76:3300/0 conn(0x7f57c01440f0 0x7f57c00b51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:56:35.266 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.262+0000 7f57bdffb700 1 -- 172.21.15.59:0/534545651 >> v1:172.21.15.76:6789/0 conn(0x7f57c0143ae0 legacy=0x7f57c0071bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:56:35.266 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.262+0000 7f57bdffb700 1 -- 172.21.15.59:0/534545651 --> v1:172.21.15.59:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f57c00b3dc0 con 0x7f57c00722c0 2023-11-24T07:56:35.267 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.262+0000 7f57bdffb700 1 -- 172.21.15.59:0/534545651 <== mon.0 v1:172.21.15.59:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1312333566 0 0) 0x7f57b4001670 con 0x7f57c00722c0 2023-11-24T07:56:35.267 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.262+0000 7f57bdffb700 1 -- 172.21.15.59:0/534545651 >> v1:172.21.15.59:6789/0 conn(0x7f57c00722c0 legacy=0x7f57c006e510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:56:35.267 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.262+0000 7f57bdffb700 1 --2- 172.21.15.59:0/534545651 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f57a8004bf0 0x7f57a8004ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:56:35.267 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.262+0000 7f57bdffb700 1 --2- 172.21.15.59:0/534545651 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f57a80037f0 0x7f57a8003c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:56:35.267 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.262+0000 7f57bdffb700 1 --2- 172.21.15.59:0/534545651 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f57a80042d0 0x7f57a800ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:56:35.267 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.262+0000 7f57bdffb700 1 -- 172.21.15.59:0/534545651 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57c00b40a0 con 0x7f57a80042d0 2023-11-24T07:56:35.267 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.262+0000 7f57bdffb700 1 -- 172.21.15.59:0/534545651 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57a800c230 con 0x7f57a80037f0 2023-11-24T07:56:35.267 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.262+0000 7f57bdffb700 1 -- 172.21.15.59:0/534545651 --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f57a800c3f0 con 0x7f57a8004bf0 2023-11-24T07:56:35.267 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.262+0000 7f57be7fc700 1 --2- 172.21.15.59:0/534545651 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f57a80037f0 0x7f57a8003c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:56:35.267 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.263+0000 7f57be7fc700 1 -- 172.21.15.59:0/534545651 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f57a8004bf0 msgr2=0x7f57a8004ff0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-11-24T07:56:35.267 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.263+0000 7f57be7fc700 1 --2- 172.21.15.59:0/534545651 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f57a8004bf0 0x7f57a8004ff0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:56:35.267 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.263+0000 7f57be7fc700 1 -- 172.21.15.59:0/534545651 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f57a80042d0 msgr2=0x7f57a800ba10 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-11-24T07:56:35.268 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.263+0000 7f57be7fc700 1 --2- 172.21.15.59:0/534545651 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f57a80042d0 0x7f57a800ba10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:56:35.268 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.263+0000 7f57be7fc700 1 -- 172.21.15.59:0/534545651 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f57a80035b0 con 0x7f57a80037f0 2023-11-24T07:56:35.268 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.263+0000 7f57be7fc700 1 --2- 172.21.15.59:0/534545651 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f57a80037f0 0x7f57a8003c70 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f57b4001eb0 tx=0x7f57a8005880 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c721e85de9b8a1ec server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:56:35.268 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.263+0000 7f57bdffb700 1 -- 172.21.15.59:0/534545651 <== mon.1 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57b400cd80 con 0x7f57a80037f0 2023-11-24T07:56:35.268 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.263+0000 7f57bdffb700 1 -- 172.21.15.59:0/534545651 <== mon.1 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f57b4005960 con 0x7f57a80037f0 2023-11-24T07:56:35.268 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.263+0000 7f57bdffb700 1 -- 172.21.15.59:0/534545651 <== mon.1 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57b4005cf0 con 0x7f57a80037f0 2023-11-24T07:56:35.268 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.264+0000 7f57c5603700 1 -- 172.21.15.59:0/534545651 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f57a80037f0 msgr2=0x7f57a8003c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:56:35.269 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.264+0000 7f57c5603700 1 --2- 172.21.15.59:0/534545651 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f57a80037f0 0x7f57a8003c70 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f57b4001eb0 tx=0x7f57a8005880 comp rx=0 tx=0).stop 2023-11-24T07:56:35.269 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.265+0000 7f57bf7fe700 1 -- 172.21.15.59:0/534545651 reap_dead start 2023-11-24T07:56:35.269 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.265+0000 7f57bf7fe700 1 -- 172.21.15.59:0/534545651 reap_dead start 2023-11-24T07:56:35.269 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.266+0000 7f57c5603700 1 -- 172.21.15.59:0/534545651 shutdown_connections 2023-11-24T07:56:35.269 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.266+0000 7f57c5603700 1 -- 172.21.15.59:0/534545651 >> 172.21.15.59:0/534545651 conn(0x7f57c00b2740 msgr2=0x7f57c00b2b20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:56:35.273 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.271+0000 7f57c5603700 1 -- 172.21.15.59:0/534545651 shutdown_connections 2023-11-24T07:56:35.274 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.271+0000 7f57c5603700 1 -- 172.21.15.59:0/534545651 wait complete. 2023-11-24T07:56:35.275 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.273+0000 7f57c5603700 1 Processor -- start 2023-11-24T07:56:35.275 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.273+0000 7f57c5603700 1 -- start start 2023-11-24T07:56:35.276 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.274+0000 7f57c5603700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f57a80037f0 0x7f57c00c59b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:56:35.277 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.274+0000 7f57c5603700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f57c0141c00 0x7f57c016a3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:56:35.277 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.274+0000 7f57c5603700 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f57c0137010 0x7f57c015c710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:56:35.277 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.275+0000 7f57c5603700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57c016a900 con 0x7f57c0141c00 2023-11-24T07:56:35.277 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.275+0000 7f57c5603700 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57c00c6020 con 0x7f57a80037f0 2023-11-24T07:56:35.277 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.275+0000 7f57c5603700 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f57c00c6300 con 0x7f57c0137010 2023-11-24T07:56:35.277 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.275+0000 7f57be7fc700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f57c0141c00 0x7f57c016a3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:56:35.277 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.275+0000 7f57be7fc700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f57c0141c00 0x7f57c016a3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:49930/0 (socket says 172.21.15.59:49930) 2023-11-24T07:56:35.278 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.275+0000 7f57be7fc700 1 -- 172.21.15.59:0/2354079022 learned_addr learned my addr 172.21.15.59:0/2354079022 (peer_addr_for_me v2:172.21.15.59:0/0) 2023-11-24T07:56:35.278 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.275+0000 7f93625eb700 1 Processor -- start 2023-11-24T07:56:35.278 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.275+0000 7f57beffd700 1 --2- 172.21.15.59:0/2354079022 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f57a80037f0 0x7f57c00c59b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:56:35.279 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.275+0000 7f93625eb700 1 -- start start 2023-11-24T07:56:35.279 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.275+0000 7f57bf7fe700 1 --2- 172.21.15.59:0/2354079022 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f57c0137010 0x7f57c015c710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:56:35.279 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.275+0000 7f57be7fc700 1 -- 172.21.15.59:0/2354079022 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f57c0137010 msgr2=0x7f57c015c710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:56:35.279 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.275+0000 7f57be7fc700 1 --2- 172.21.15.59:0/2354079022 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f57c0137010 0x7f57c015c710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:56:35.280 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.275+0000 7f57be7fc700 1 -- 172.21.15.59:0/2354079022 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f57a80037f0 msgr2=0x7f57c00c59b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:56:35.280 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.275+0000 7f57be7fc700 1 --2- 172.21.15.59:0/2354079022 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f57a80037f0 0x7f57c00c59b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:56:35.280 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.275+0000 7f57be7fc700 1 -- 172.21.15.59:0/2354079022 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f57c00c65e0 con 0x7f57c0141c00 2023-11-24T07:56:35.280 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.271+0000 7f9e45547700 1 Processor -- start 2023-11-24T07:56:35.280 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.272+0000 7f9e45547700 1 -- start start 2023-11-24T07:56:35.280 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.273+0000 7f9e45547700 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f9e4006cae0 0x7f9e400ce440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:56:35.280 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.273+0000 7f9e45547700 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f9e400ce980 0x7f9e400cafa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:56:35.280 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.273+0000 7f9e45547700 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9e400cab10 con 0x7f9e40070a80 2023-11-24T07:56:35.281 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.273+0000 7f9e45547700 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9e400c9fd0 con 0x7f9e4006cae0 2023-11-24T07:56:35.281 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.273+0000 7f9e45547700 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9e400cd7e0 con 0x7f9e400ce980 2023-11-24T07:56:35.281 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.273+0000 7f9e3effd700 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7f9e40070a80 0x7f9e40070e60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:53400/0 (socket says 172.21.15.59:53400) 2023-11-24T07:56:35.281 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.273+0000 7f9e3effd700 1 -- 172.21.15.59:0/4070954130 learned_addr learned my addr 172.21.15.59:0/4070954130 (peer_addr_for_me v1:172.21.15.59:0/0) 2023-11-24T07:56:35.281 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.273+0000 7f9e3f7fe700 1 --2- 172.21.15.59:0/4070954130 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f9e400ce980 0x7f9e400cafa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:56:35.281 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.273+0000 7f9e3e7fc700 1 --2- 172.21.15.59:0/4070954130 >> v2:172.21.15.76:3300/0 conn(0x7f9e4006cae0 0x7f9e400ce440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:56:35.281 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.276+0000 7f57be7fc700 1 --2- 172.21.15.59:0/2354079022 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f57c0141c00 0x7f57c016a3c0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f57b40018b0 tx=0x7f57b4001980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:56:35.282 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.276+0000 7f7da4d6d700 1 Processor -- start 2023-11-24T07:56:35.282 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.273+0000 7f9e3dffb700 1 -- 172.21.15.59:0/4070954130 <== mon.0 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 808683239 0 0) 0x7f9e400cab10 con 0x7f9e40070a80 2023-11-24T07:56:35.282 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.276+0000 7f9e3dffb700 1 -- 172.21.15.59:0/4070954130 --> v1:172.21.15.59:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9e280035b0 con 0x7f9e40070a80 2023-11-24T07:56:35.283 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.276+0000 7f9e3f7fe700 1 -- 172.21.15.59:0/4070954130 >> v2:172.21.15.76:3300/0 conn(0x7f9e4006cae0 msgr2=0x7f9e400ce440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T07:56:35.283 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.276+0000 7f9e3f7fe700 1 --2- 172.21.15.59:0/4070954130 >> v2:172.21.15.76:3300/0 conn(0x7f9e4006cae0 0x7f9e400ce440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:56:35.283 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.276+0000 7f9e3f7fe700 1 -- 172.21.15.59:0/4070954130 >> v1:172.21.15.59:6789/0 conn(0x7f9e40070a80 legacy=0x7f9e40070e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:56:35.283 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.276+0000 7f9e3f7fe700 1 -- 172.21.15.59:0/4070954130 --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9e400ca1e0 con 0x7f9e400ce980 2023-11-24T07:56:35.283 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.276+0000 7f9e3f7fe700 1 --2- 172.21.15.59:0/4070954130 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f9e400ce980 0x7f9e400cafa0 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f9e3800c2e0 tx=0x7f9e3800c5f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b585a217da95ea53 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:56:35.283 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.276+0000 7f93625eb700 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f935c06b620 0x7f935c0b3a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:56:35.283 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.276+0000 7f93625eb700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f935c0b5ab0 0x7f935c0b3f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:56:35.283 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.276+0000 7f93625eb700 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f935c071f50 con 0x7f935c1478c0 2023-11-24T07:56:35.284 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.276+0000 7f93625eb700 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f935c072230 con 0x7f935c0b5ab0 2023-11-24T07:56:35.284 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.276+0000 7f93625eb700 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f935c072510 con 0x7f935c06b620 2023-11-24T07:56:35.284 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.276+0000 7f57a77fe700 1 -- 172.21.15.59:0/2354079022 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57b4004a80 con 0x7f57c0141c00 2023-11-24T07:56:35.284 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.276+0000 7f57a77fe700 1 -- 172.21.15.59:0/2354079022 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f57b4004be0 con 0x7f57c0141c00 2023-11-24T07:56:35.284 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.277+0000 7f57a77fe700 1 -- 172.21.15.59:0/2354079022 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57b40070d0 con 0x7f57c0141c00 2023-11-24T07:56:35.285 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.277+0000 7f9e3dffb700 1 -- 172.21.15.59:0/4070954130 <== mon.2 v2:172.21.15.76:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e38007ce0 con 0x7f9e400ce980 2023-11-24T07:56:35.285 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.277+0000 7f935b7fe700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f935c0b5ab0 0x7f935c0b3f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:56:35.285 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.277+0000 7f935b7fe700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f935c0b5ab0 0x7f935c0b3f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:49938/0 (socket says 172.21.15.59:49938) 2023-11-24T07:56:35.285 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.277+0000 7f935b7fe700 1 -- 172.21.15.59:0/3161612114 learned_addr learned my addr 172.21.15.59:0/3161612114 (peer_addr_for_me v2:172.21.15.59:0/0) 2023-11-24T07:56:35.285 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.277+0000 7f935b7fe700 1 -- 172.21.15.59:0/3161612114 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f935c06b620 msgr2=0x7f935c0b3a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T07:56:35.285 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.277+0000 7f935bfff700 1 --2- 172.21.15.59:0/3161612114 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f935c06b620 0x7f935c0b3a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:56:35.286 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.277+0000 7f935b7fe700 1 --2- 172.21.15.59:0/3161612114 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f935c06b620 0x7f935c0b3a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:56:35.286 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.277+0000 7f935b7fe700 1 -- 172.21.15.59:0/3161612114 >> v1:172.21.15.59:6789/0 conn(0x7f935c1478c0 legacy=0x7f935c161ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:56:35.286 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.277+0000 7f935b7fe700 1 -- 172.21.15.59:0/3161612114 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f935c0727f0 con 0x7f935c0b5ab0 2023-11-24T07:56:35.286 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.276+0000 7f7da4d6d700 1 -- start start 2023-11-24T07:56:35.286 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.277+0000 7f7da4d6d700 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f7da015db10 0x7f7da015def0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:56:35.287 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.277+0000 7f7da4d6d700 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f7da00ca2c0 0x7f7da00ca750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:56:35.287 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.277+0000 7f7da4d6d700 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7da00ce000 con 0x7f7da0071390 2023-11-24T07:56:35.287 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.277+0000 7f7da4d6d700 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7da00ce2c0 con 0x7f7da00ca2c0 2023-11-24T07:56:35.287 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.277+0000 7f7da4d6d700 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7da00ce5a0 con 0x7f7da015db10 2023-11-24T07:56:35.287 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.278+0000 7f935bfff700 1 --2- 172.21.15.59:0/3161612114 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f935c06b620 0x7f935c0b3a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T07:56:35.288 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.278+0000 7f7d9e59c700 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f7da015db10 0x7f7da015def0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:56:35.288 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.278+0000 7f57c5603700 1 -- 172.21.15.59:0/2354079022 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f57c016a900 con 0x7f57c0141c00 2023-11-24T07:56:35.288 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.278+0000 7f7d9dd9b700 1 --1- >> v1:172.21.15.76:6789/0 conn(0x7f7da0071390 0x7f7da015e430 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.59:40702/0 (socket says 172.21.15.59:40702) 2023-11-24T07:56:35.288 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.278+0000 7f7d9dd9b700 1 -- 172.21.15.59:0/1836430178 learned_addr learned my addr 172.21.15.59:0/1836430178 (peer_addr_for_me v1:172.21.15.59:0/0) 2023-11-24T07:56:35.289 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.278+0000 7f57c5603700 1 -- 172.21.15.59:0/2354079022 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f57c00c68c0 con 0x7f57c0141c00 2023-11-24T07:56:35.289 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.278+0000 7f7d9ed9d700 1 --2- 172.21.15.59:0/1836430178 >> v2:172.21.15.76:3300/0 conn(0x7f7da00ca2c0 0x7f7da00ca750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:56:35.290 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.277+0000 7f9e3dffb700 1 -- 172.21.15.59:0/4070954130 <== mon.2 v2:172.21.15.76:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9e38007e40 con 0x7f9e400ce980 2023-11-24T07:56:35.290 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.277+0000 7f9e3dffb700 1 -- 172.21.15.59:0/4070954130 <== mon.2 v2:172.21.15.76:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e3800a360 con 0x7f9e400ce980 2023-11-24T07:56:35.290 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.278+0000 7f9e45547700 1 -- 172.21.15.59:0/4070954130 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f9e400ce980 msgr2=0x7f9e400cafa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:56:35.290 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.278+0000 7f9e45547700 1 --2- 172.21.15.59:0/4070954130 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f9e400ce980 0x7f9e400cafa0 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f9e3800c2e0 tx=0x7f9e3800c5f0 comp rx=0 tx=0).stop 2023-11-24T07:56:35.290 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.278+0000 7f9e45547700 1 -- 172.21.15.59:0/4070954130 shutdown_connections 2023-11-24T07:56:35.290 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.278+0000 7f9e45547700 1 --2- 172.21.15.59:0/4070954130 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f9e400ce980 0x7f9e400cafa0 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:56:35.290 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.278+0000 7f9e45547700 1 --2- 172.21.15.59:0/4070954130 >> v2:172.21.15.76:3300/0 conn(0x7f9e4006cae0 0x7f9e400ce440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:56:35.290 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.278+0000 7f9e45547700 1 -- 172.21.15.59:0/4070954130 >> 172.21.15.59:0/4070954130 conn(0x7f9e400cf980 msgr2=0x7f9e400cfd90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:56:35.291 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.278+0000 7f9e45547700 1 -- 172.21.15.59:0/4070954130 shutdown_connections 2023-11-24T07:56:35.291 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.278+0000 7f9e45547700 1 -- 172.21.15.59:0/4070954130 wait complete. 2023-11-24T07:56:35.292 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.275+0000 7fba2bf5c700 1 Processor -- start 2023-11-24T07:56:35.292 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.275+0000 7fba2bf5c700 1 -- start start 2023-11-24T07:56:35.292 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.277+0000 7fba2bf5c700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fba2406cae0 0x7fba240ce440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:56:35.292 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.277+0000 7fba2bf5c700 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fba240cab10 con 0x7fba24070a80 2023-11-24T07:56:35.292 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.277+0000 7fba2bf5c700 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fba240cd7e0 con 0x7fba240ce980 2023-11-24T07:56:35.293 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.277+0000 7fba2bf5c700 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fba240c9fd0 con 0x7fba2406cae0 2023-11-24T07:56:35.293 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.277+0000 7fba2a4f9700 1 --1- >> v1:172.21.15.76:6789/0 conn(0x7fba240ce980 0x7fba240cafa0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.59:40698/0 (socket says 172.21.15.59:40698) 2023-11-24T07:56:35.293 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.277+0000 7fba2a4f9700 1 -- 172.21.15.59:0/582293757 learned_addr learned my addr 172.21.15.59:0/582293757 (peer_addr_for_me v1:172.21.15.59:0/0) 2023-11-24T07:56:35.293 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.278+0000 7fba28cf6700 1 -- 172.21.15.59:0/582293757 <== mon.1 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1022892136 0 0) 0x7fba240cd7e0 con 0x7fba240ce980 2023-11-24T07:56:35.293 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.278+0000 7fba28cf6700 1 -- 172.21.15.59:0/582293757 --> v1:172.21.15.76:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fba10003a00 con 0x7fba240ce980 2023-11-24T07:56:35.294 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.278+0000 7f935b7fe700 1 --2- 172.21.15.59:0/3161612114 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f935c0b5ab0 0x7f935c0b3f80 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f93500098f0 tx=0x7f9350009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a011eb53f09c7204 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:56:35.294 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.279+0000 7fba444a3700 1 Processor -- start 2023-11-24T07:56:35.295 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.279+0000 7f7d9e59c700 1 -- 172.21.15.59:0/1836430178 >> v2:172.21.15.76:3300/0 conn(0x7f7da00ca2c0 msgr2=0x7f7da00ca750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T07:56:35.295 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.279+0000 7f7d9e59c700 1 --2- 172.21.15.59:0/1836430178 >> v2:172.21.15.76:3300/0 conn(0x7f7da00ca2c0 0x7f7da00ca750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:56:35.295 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.279+0000 7f7d9e59c700 1 -- 172.21.15.59:0/1836430178 >> v1:172.21.15.76:6789/0 conn(0x7f7da0071390 legacy=0x7f7da015e430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:56:35.295 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.279+0000 7f7d9e59c700 1 -- 172.21.15.59:0/1836430178 --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7da00ce880 con 0x7f7da015db10 2023-11-24T07:56:35.296 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.279+0000 7f7d9e59c700 1 --2- 172.21.15.59:0/1836430178 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f7da015db10 0x7f7da015def0 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f7d9800c2e0 tx=0x7f7d9800c5f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=96cfb8b8e39476b7 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:56:35.296 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.279+0000 7fba444a3700 1 -- start start 2023-11-24T07:56:35.296 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.279+0000 7f7d9d59a700 1 -- 172.21.15.59:0/1836430178 <== mon.2 v2:172.21.15.76:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d9800f070 con 0x7f7da015db10 2023-11-24T07:56:35.296 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.280+0000 7f7d9d59a700 1 -- 172.21.15.59:0/1836430178 <== mon.2 v2:172.21.15.76:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7d9800ce00 con 0x7f7da015db10 2023-11-24T07:56:35.296 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.280+0000 7f7d9d59a700 1 -- 172.21.15.59:0/1836430178 <== mon.2 v2:172.21.15.76:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d98003c40 con 0x7f7da015db10 2023-11-24T07:56:35.297 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.280+0000 7f57c5603700 1 -- 172.21.15.59:0/2354079022 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f57c00681f0 con 0x7f57c0141c00 2023-11-24T07:56:35.297 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.278+0000 7fba294f7700 1 --2- 172.21.15.59:0/582293757 >> v2:172.21.15.59:3300/0 conn(0x7fba2406cae0 0x7fba240ce440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:56:35.297 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.279+0000 7fba28cf6700 1 -- 172.21.15.59:0/582293757 <== mon.1 v1:172.21.15.76:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1804495373 0 0) 0x7fba10003a00 con 0x7fba240ce980 2023-11-24T07:56:35.297 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.279+0000 7fba28cf6700 1 -- 172.21.15.59:0/582293757 >> v2:172.21.15.59:3300/0 conn(0x7fba2406cae0 msgr2=0x7fba240ce440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T07:56:35.297 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.279+0000 7fba28cf6700 1 --2- 172.21.15.59:0/582293757 >> v2:172.21.15.59:3300/0 conn(0x7fba2406cae0 0x7fba240ce440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:56:35.297 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.279+0000 7fba28cf6700 1 -- 172.21.15.59:0/582293757 >> v1:172.21.15.59:6789/0 conn(0x7fba24070a80 legacy=0x7fba24070e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T07:56:35.297 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.279+0000 7fba28cf6700 1 -- 172.21.15.59:0/582293757 --> v1:172.21.15.76:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fba240ca1e0 con 0x7fba240ce980 2023-11-24T07:56:35.297 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.279+0000 7fba28cf6700 1 -- 172.21.15.59:0/582293757 <== mon.1 v1:172.21.15.76:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1312333566 0 0) 0x7fba1c0019f0 con 0x7fba240ce980 2023-11-24T07:56:35.298 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.279+0000 7fba28cf6700 1 -- 172.21.15.59:0/582293757 >> v1:172.21.15.76:6789/0 conn(0x7fba240ce980 legacy=0x7fba240cafa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:56:35.298 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.279+0000 7fba28cf6700 1 --2- 172.21.15.59:0/582293757 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fba10005530 0x7fba10003c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:56:35.298 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.279+0000 7fba28cf6700 1 --2- 172.21.15.59:0/582293757 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fba10004180 0x7fba10004600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:56:35.298 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.279+0000 7fba28cf6700 1 --2- 172.21.15.59:0/582293757 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fba10009e50 0x7fba1000c240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:56:35.298 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.279+0000 7fba28cf6700 1 -- 172.21.15.59:0/582293757 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba240c9fd0 con 0x7fba10004180 2023-11-24T07:56:35.298 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.279+0000 7fba28cf6700 1 -- 172.21.15.59:0/582293757 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba1000c970 con 0x7fba10009e50 2023-11-24T07:56:35.298 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.279+0000 7fba28cf6700 1 -- 172.21.15.59:0/582293757 --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fba1000cb30 con 0x7fba10005530 2023-11-24T07:56:35.298 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.279+0000 7fba29cf8700 1 --2- 172.21.15.59:0/582293757 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fba10005530 0x7fba10003c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:56:35.298 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.279+0000 7fba2a4f9700 1 --2- 172.21.15.59:0/582293757 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fba10009e50 0x7fba1000c240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:56:35.298 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.280+0000 7fba2a4f9700 1 -- 172.21.15.59:0/582293757 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fba10005530 msgr2=0x7fba10003c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T07:56:35.298 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.280+0000 7fba2a4f9700 1 --2- 172.21.15.59:0/582293757 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fba10005530 0x7fba10003c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:56:35.298 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.280+0000 7fba2a4f9700 1 -- 172.21.15.59:0/582293757 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fba10004180 msgr2=0x7fba10004600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T07:56:35.299 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.280+0000 7fba2a4f9700 1 --2- 172.21.15.59:0/582293757 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fba10004180 0x7fba10004600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:56:35.299 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.280+0000 7fba2a4f9700 1 -- 172.21.15.59:0/582293757 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fba10003a00 con 0x7fba10009e50 2023-11-24T07:56:35.299 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.280+0000 7fba2a4f9700 1 -- 172.21.15.59:0/582293757 reap_dead start 2023-11-24T07:56:35.299 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.280+0000 7fba2a4f9700 1 --2- 172.21.15.59:0/582293757 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fba10009e50 0x7fba1000c240 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fba1c00a7e0 tx=0x7fba10005cd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c4bd5cd7c5427716 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:56:35.299 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.280+0000 7f7da4d6d700 1 -- 172.21.15.59:0/1836430178 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f7da015db10 msgr2=0x7f7da015def0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:56:35.299 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.280+0000 7f7da4d6d700 1 --2- 172.21.15.59:0/1836430178 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f7da015db10 0x7f7da015def0 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f7d9800c2e0 tx=0x7f7d9800c5f0 comp rx=0 tx=0).stop 2023-11-24T07:56:35.300 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.281+0000 7fba444a3700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fba3c143ae0 0x7fba3c0b51b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:56:35.300 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.281+0000 7fba444a3700 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fba3c1440f0 0x7fba3c071bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:56:35.300 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.281+0000 7fba444a3700 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fba3c143ec0 con 0x7fba3c0720f0 2023-11-24T07:56:35.300 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.281+0000 7fba28cf6700 1 -- 172.21.15.59:0/582293757 <== mon.1 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba1c012600 con 0x7fba10009e50 2023-11-24T07:56:35.300 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.281+0000 7fba444a3700 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fba3c0b40a0 con 0x7fba3c143ae0 2023-11-24T07:56:35.300 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.281+0000 7fba444a3700 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fba3c072580 con 0x7fba3c1440f0 2023-11-24T07:56:35.301 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.281+0000 7fba28cf6700 1 -- 172.21.15.59:0/582293757 <== mon.1 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fba1c012760 con 0x7fba10009e50 2023-11-24T07:56:35.302 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.280+0000 7f9e45547700 1 Processor -- start 2023-11-24T07:56:35.302 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.280+0000 7f9e45547700 1 -- start start 2023-11-24T07:56:35.302 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.281+0000 7fba28cf6700 1 -- 172.21.15.59:0/582293757 <== mon.1 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba1c012ad0 con 0x7fba10009e50 2023-11-24T07:56:35.303 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.281+0000 7f7da4d6d700 1 -- 172.21.15.59:0/1836430178 shutdown_connections 2023-11-24T07:56:35.303 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.281+0000 7f7da4d6d700 1 --2- 172.21.15.59:0/1836430178 >> v2:172.21.15.76:3300/0 conn(0x7f7da00ca2c0 0x7f7da00ca750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:56:35.303 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.281+0000 7f7da4d6d700 1 --2- 172.21.15.59:0/1836430178 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f7da015db10 0x7f7da015def0 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:56:35.303 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.281+0000 7f7da4d6d700 1 -- 172.21.15.59:0/1836430178 >> 172.21.15.59:0/1836430178 conn(0x7f7da00cf980 msgr2=0x7f7da00cfd90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:56:35.303 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.281+0000 7f935affd700 1 -- 172.21.15.59:0/3161612114 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f935000b8a0 con 0x7f935c0b5ab0 2023-11-24T07:56:35.303 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.281+0000 7fba4223f700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fba3c143ae0 0x7fba3c0b51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:56:35.303 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.281+0000 7fba4223f700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fba3c143ae0 0x7fba3c0b51b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:49960/0 (socket says 172.21.15.59:49960) 2023-11-24T07:56:35.304 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.281+0000 7fba4223f700 1 -- 172.21.15.59:0/4255191709 learned_addr learned my addr 172.21.15.59:0/4255191709 (peer_addr_for_me v2:172.21.15.59:0/0) 2023-11-24T07:56:35.304 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.281+0000 7f935affd700 1 -- 172.21.15.59:0/3161612114 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f935000ba00 con 0x7f935c0b5ab0 2023-11-24T07:56:35.304 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.281+0000 7f935affd700 1 -- 172.21.15.59:0/3161612114 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f935000bd90 con 0x7f935c0b5ab0 2023-11-24T07:56:35.304 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.282+0000 7f9e45547700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9e4006cae0 0x7f9e4015c360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:56:35.305 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.282+0000 7f9e45547700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9e40070a80 0x7f9e4006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:56:35.305 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.282+0000 7f9e45547700 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f9e400ce980 0x7f9e400b0670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:56:35.305 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.282+0000 7f9e45547700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e4016bb80 con 0x7f9e4006cae0 2023-11-24T07:56:35.305 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.282+0000 7f9e45547700 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e4015c8a0 con 0x7f9e40070a80 2023-11-24T07:56:35.305 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.282+0000 7f9e45547700 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9e4016af00 con 0x7f9e400ce980 2023-11-24T07:56:35.305 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.282+0000 7f9e3e7fc700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9e40070a80 0x7f9e4006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:56:35.305 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.282+0000 7f9e3e7fc700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9e40070a80 0x7f9e4006d990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.59:37164/0 (socket says 172.21.15.59:37164) 2023-11-24T07:56:35.305 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.282+0000 7f9e3e7fc700 1 -- 172.21.15.59:0/1031333834 learned_addr learned my addr 172.21.15.59:0/1031333834 (peer_addr_for_me v2:172.21.15.59:0/0) 2023-11-24T07:56:35.306 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.281+0000 7f57a77fe700 1 -- 172.21.15.59:0/2354079022 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f57b4007230 con 0x7f57c0141c00 2023-11-24T07:56:35.306 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.282+0000 7f57a77fe700 1 --2- 172.21.15.59:0/2354079022 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f57ac06afa0 0x7f57ac06d460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:56:35.306 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.282+0000 7f57a77fe700 1 -- 172.21.15.59:0/2354079022 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f57b408ab90 con 0x7f57c0141c00 2023-11-24T07:56:35.306 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.282+0000 7f57a77fe700 1 --2- 172.21.15.59:0/2354079022 >> [v2:172.21.15.76:6801/3946579410,v1:172.21.15.76:6803/3946579410] conn(0x7f57ac073140 0x7f57ac075560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:56:35.306 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.282+0000 7f57a77fe700 1 -- 172.21.15.59:0/2354079022 --> [v2:172.21.15.76:6801/3946579410,v1:172.21.15.76:6803/3946579410] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f57ac075c10 con 0x7f57ac073140 2023-11-24T07:56:35.306 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.282+0000 7f9e3f7fe700 1 --2- 172.21.15.59:0/1031333834 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f9e400ce980 0x7f9e400b0670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:56:35.306 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.282+0000 7f93625eb700 1 -- 172.21.15.59:0/3161612114 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f935c0b5ab0 msgr2=0x7f935c0b3f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:56:35.307 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.282+0000 7f93625eb700 1 --2- 172.21.15.59:0/3161612114 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f935c0b5ab0 0x7f935c0b3f80 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f93500098f0 tx=0x7f9350009c00 comp rx=0 tx=0).stop 2023-11-24T07:56:35.307 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.282+0000 7f7da4d6d700 1 -- 172.21.15.59:0/1836430178 shutdown_connections 2023-11-24T07:56:35.307 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.282+0000 7f9e3e7fc700 1 -- 172.21.15.59:0/1031333834 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f9e400ce980 msgr2=0x7f9e400b0670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:56:35.307 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.282+0000 7f9e3e7fc700 1 --2- 172.21.15.59:0/1031333834 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f9e400ce980 0x7f9e400b0670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:56:35.307 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.282+0000 7f9e3e7fc700 1 -- 172.21.15.59:0/1031333834 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9e4006cae0 msgr2=0x7f9e4015c360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:56:35.307 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.282+0000 7f9e3e7fc700 1 --2- 172.21.15.59:0/1031333834 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9e4006cae0 0x7f9e4015c360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:56:35.308 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.282+0000 7fba4223f700 1 -- 172.21.15.59:0/4255191709 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fba3c1440f0 msgr2=0x7fba3c071bb0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-11-24T07:56:35.308 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.282+0000 7fba4223f700 1 --2- 172.21.15.59:0/4255191709 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fba3c1440f0 0x7fba3c071bb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:56:35.308 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.282+0000 7fba4223f700 1 -- 172.21.15.59:0/4255191709 >> v1:172.21.15.76:6789/0 conn(0x7fba3c0720f0 legacy=0x7fba3c15e8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T07:56:35.308 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.282+0000 7fba4223f700 1 -- 172.21.15.59:0/4255191709 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fba3c0b3ae0 con 0x7fba3c143ae0 2023-11-24T07:56:35.308 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.282+0000 7fba4223f700 1 --2- 172.21.15.59:0/4255191709 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fba3c143ae0 0x7fba3c0b51b0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fba2c00bb90 tx=0x7fba2c00bea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=841e11589393a025 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:56:35.308 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.282+0000 7fba4123d700 1 -- 172.21.15.59:0/4255191709 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba2c00dd10 con 0x7fba3c143ae0 2023-11-24T07:56:35.308 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.282+0000 7f9e3e7fc700 1 -- 172.21.15.59:0/1031333834 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9e3800c050 con 0x7f9e40070a80 2023-11-24T07:56:35.309 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.282+0000 7fba2bf5c700 1 -- 172.21.15.59:0/582293757 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fba10009e50 msgr2=0x7fba1000c240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:56:35.309 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.282+0000 7fba2bf5c700 1 --2- 172.21.15.59:0/582293757 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fba10009e50 0x7fba1000c240 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fba1c00a7e0 tx=0x7fba10005cd0 comp rx=0 tx=0).stop 2023-11-24T07:56:35.309 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.282+0000 7fba2bf5c700 1 -- 172.21.15.59:0/582293757 shutdown_connections 2023-11-24T07:56:35.309 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.282+0000 7fba2bf5c700 1 --2- 172.21.15.59:0/582293757 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fba10009e50 0x7fba1000c240 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:56:35.309 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.282+0000 7fba2bf5c700 1 -- 172.21.15.59:0/582293757 >> 172.21.15.59:0/582293757 conn(0x7fba240cf980 msgr2=0x7fba240cfd90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:56:35.309 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.282+0000 7fba4123d700 1 -- 172.21.15.59:0/4255191709 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fba2c00de70 con 0x7fba3c143ae0 2023-11-24T07:56:35.309 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.282+0000 7fba4123d700 1 -- 172.21.15.59:0/4255191709 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba2c004230 con 0x7fba3c143ae0 2023-11-24T07:56:35.310 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.282+0000 7f57bf7fe700 1 --2- 172.21.15.59:0/2354079022 >> [v2:172.21.15.76:6801/3946579410,v1:172.21.15.76:6803/3946579410] conn(0x7f57ac073140 0x7f57ac075560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:56:35.310 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.282+0000 7f9e3f7fe700 1 --2- 172.21.15.59:0/1031333834 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f9e400ce980 0x7f9e400b0670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-24T07:56:35.310 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.283+0000 7f7da4d6d700 1 -- 172.21.15.59:0/1836430178 wait complete. 2023-11-24T07:56:35.311 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.282+0000 7f57beffd700 1 --2- 172.21.15.59:0/2354079022 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f57ac06afa0 0x7f57ac06d460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:56:35.311 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.283+0000 7f93625eb700 1 -- 172.21.15.59:0/3161612114 shutdown_connections 2023-11-24T07:56:35.312 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.283+0000 7f93625eb700 1 --2- 172.21.15.59:0/3161612114 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f935c06b620 0x7f935c0b3a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:56:35.312 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.283+0000 7f93625eb700 1 --2- 172.21.15.59:0/3161612114 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f935c0b5ab0 0x7f935c0b3f80 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:56:35.312 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.283+0000 7f93625eb700 1 -- 172.21.15.59:0/3161612114 >> 172.21.15.59:0/3161612114 conn(0x7f935c0b2740 msgr2=0x7f935c0b2b20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:56:35.312 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.283+0000 7f9e3e7fc700 1 --2- 172.21.15.59:0/1031333834 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9e40070a80 0x7f9e4006d990 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f9e3000dae0 tx=0x7f9e3000dbc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:56:35.312 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.283+0000 7f93625eb700 1 -- 172.21.15.59:0/3161612114 shutdown_connections 2023-11-24T07:56:35.313 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.283+0000 7f93625eb700 1 -- 172.21.15.59:0/3161612114 wait complete. 2023-11-24T07:56:35.313 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.283+0000 7f57bf7fe700 1 --2- 172.21.15.59:0/2354079022 >> [v2:172.21.15.76:6801/3946579410,v1:172.21.15.76:6803/3946579410] conn(0x7f57ac073140 0x7f57ac075560 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:56:35.313 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.283+0000 7f57beffd700 1 --2- 172.21.15.59:0/2354079022 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f57ac06afa0 0x7f57ac06d460 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f57b0007fd0 tx=0x7f57b0009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:56:35.314 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.283+0000 7fba444a3700 1 -- 172.21.15.59:0/4255191709 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fba3c143ae0 msgr2=0x7fba3c0b51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:56:35.314 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.283+0000 7fba444a3700 1 --2- 172.21.15.59:0/4255191709 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fba3c143ae0 0x7fba3c0b51b0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fba2c00bb90 tx=0x7fba2c00bea0 comp rx=0 tx=0).stop 2023-11-24T07:56:35.314 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.283+0000 7f9e277fe700 1 -- 172.21.15.59:0/1031333834 <== mon.1 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e300099b0 con 0x7f9e40070a80 2023-11-24T07:56:35.314 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.283+0000 7f9e45547700 1 -- 172.21.15.59:0/1031333834 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9e4016b1e0 con 0x7f9e40070a80 2023-11-24T07:56:35.314 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.283+0000 7f9e45547700 1 -- 172.21.15.59:0/1031333834 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9e4016b7a0 con 0x7f9e40070a80 2023-11-24T07:56:35.315 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.283+0000 7f9e277fe700 1 -- 172.21.15.59:0/1031333834 <== mon.1 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9e30009b10 con 0x7f9e40070a80 2023-11-24T07:56:35.315 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.283+0000 7f9e277fe700 1 -- 172.21.15.59:0/1031333834 <== mon.1 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e30009ea0 con 0x7f9e40070a80 2023-11-24T07:56:35.315 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.283+0000 7fba444a3700 1 -- 172.21.15.59:0/4255191709 shutdown_connections 2023-11-24T07:56:35.315 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.283+0000 7fba444a3700 1 --2- 172.21.15.59:0/4255191709 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fba3c143ae0 0x7fba3c0b51b0 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:56:35.315 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.283+0000 7fba444a3700 1 --2- 172.21.15.59:0/4255191709 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fba3c1440f0 0x7fba3c071bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:56:35.315 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.283+0000 7fba444a3700 1 -- 172.21.15.59:0/4255191709 >> 172.21.15.59:0/4255191709 conn(0x7fba3c0b2740 msgr2=0x7fba3c0b2b20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:56:35.315 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.284+0000 7f57a77fe700 1 -- 172.21.15.59:0/2354079022 <== osd.6 v2:172.21.15.76:6801/3946579410 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+26965 (crc 0 0 0) 0x7f57ac075c10 con 0x7f57ac073140 2023-11-24T07:56:35.316 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.284+0000 7f9e45547700 1 -- 172.21.15.59:0/1031333834 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f9e400ca7a0 con 0x7f9e40070a80 2023-11-24T07:56:35.316 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.285+0000 7fba2bf5c700 1 -- 172.21.15.59:0/582293757 shutdown_connections 2023-11-24T07:56:35.316 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.285+0000 7f93625eb700 1 Processor -- start 2023-11-24T07:56:35.317 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.285+0000 7f9e277fe700 1 -- 172.21.15.59:0/1031333834 <== mon.1 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f9e30019410 con 0x7f9e40070a80 2023-11-24T07:56:35.317 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.286+0000 7f9e277fe700 1 --2- 172.21.15.59:0/1031333834 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f9e2c06af20 0x7f9e2c06d3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:56:35.317 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.286+0000 7f9e277fe700 1 -- 172.21.15.59:0/1031333834 <== mon.1 v2:172.21.15.76:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f9e3008a1e0 con 0x7f9e40070a80 2023-11-24T07:56:35.317 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.286+0000 7f9e3effd700 1 --2- 172.21.15.59:0/1031333834 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f9e2c06af20 0x7f9e2c06d3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:56:35.317 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.286+0000 7f9e277fe700 1 --2- 172.21.15.59:0/1031333834 >> [v2:172.21.15.59:6800/3373878772,v1:172.21.15.59:6801/3373878772] conn(0x7f9e2c073140 0x7f9e2c075560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:56:35.317 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.286+0000 7f9e277fe700 1 -- 172.21.15.59:0/1031333834 --> [v2:172.21.15.59:6800/3373878772,v1:172.21.15.59:6801/3373878772] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f9e2c075c10 con 0x7f9e2c073140 2023-11-24T07:56:35.317 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.286+0000 7f9e277fe700 1 -- 172.21.15.59:0/1031333834 <== mon.1 v2:172.21.15.76:3300/0 6 ==== mon_get_version_reply(handle=1 version=20) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9e3008a680 con 0x7f9e40070a80 2023-11-24T07:56:35.317 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.285+0000 7fba2bf5c700 1 -- 172.21.15.59:0/582293757 wait complete. 2023-11-24T07:56:35.318 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.286+0000 7fbe80f28700 1 Processor -- start 2023-11-24T07:56:35.318 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.286+0000 7fbe80f28700 1 -- start start 2023-11-24T07:56:35.318 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.285+0000 7f93625eb700 1 -- start start 2023-11-24T07:56:35.318 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.286+0000 7f93625eb700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f935c06b620 0x7f935c0aa590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:56:35.318 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.286+0000 7f93625eb700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f935c0b5ab0 0x7f935c0aaad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:56:35.318 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.286+0000 7f93625eb700 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f935c1478c0 0x7f935c13d6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:56:35.318 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.286+0000 7f93625eb700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f935c0acd20 con 0x7f935c0b5ab0 2023-11-24T07:56:35.318 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.286+0000 7f93625eb700 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f935c0af5a0 con 0x7f935c06b620 2023-11-24T07:56:35.319 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.286+0000 7f93625eb700 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f935c15bb90 con 0x7f935c1478c0 2023-11-24T07:56:35.319 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.286+0000 7fba2bf5c700 1 Processor -- start 2023-11-24T07:56:35.319 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.286+0000 7fba2bf5c700 1 -- start start 2023-11-24T07:56:35.319 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.287+0000 7f9e3f7fe700 1 --2- 172.21.15.59:0/1031333834 >> [v2:172.21.15.59:6800/3373878772,v1:172.21.15.59:6801/3373878772] conn(0x7f9e2c073140 0x7f9e2c075560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:56:35.319 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.288+0000 7f9e3f7fe700 1 --2- 172.21.15.59:0/1031333834 >> [v2:172.21.15.59:6800/3373878772,v1:172.21.15.59:6801/3373878772] conn(0x7f9e2c073140 0x7f9e2c075560 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:56:35.320 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.288+0000 7fba2bf5c700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fba10009e50 0x7fba2408a820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:56:35.320 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.288+0000 7fba2bf5c700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fba240c7e90 0x7fba240c8310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:56:35.320 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.288+0000 7fba2bf5c700 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fba2415a130 0x7fba2415a5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:56:35.320 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.288+0000 7fba2bf5c700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba240c8850 con 0x7fba10009e50 2023-11-24T07:56:35.320 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.288+0000 7fba2bf5c700 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba24207070 con 0x7fba240c7e90 2023-11-24T07:56:35.320 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.288+0000 7fba2bf5c700 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fba2415aaf0 con 0x7fba2415a130 2023-11-24T07:56:35.320 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.288+0000 7fba29cf8700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fba10009e50 0x7fba2408a820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:56:35.320 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.288+0000 7fba29cf8700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fba10009e50 0x7fba2408a820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:49988/0 (socket says 172.21.15.59:49988) 2023-11-24T07:56:35.320 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.288+0000 7fba29cf8700 1 -- 172.21.15.59:0/2248999922 learned_addr learned my addr 172.21.15.59:0/2248999922 (peer_addr_for_me v2:172.21.15.59:0/0) 2023-11-24T07:56:35.320 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.286+0000 7f935bfff700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f935c06b620 0x7f935c0aa590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:56:35.321 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.286+0000 7f935bfff700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f935c06b620 0x7f935c0aa590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.59:37178/0 (socket says 172.21.15.59:37178) 2023-11-24T07:56:35.321 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.286+0000 7f935bfff700 1 -- 172.21.15.59:0/1617312151 learned_addr learned my addr 172.21.15.59:0/1617312151 (peer_addr_for_me v2:172.21.15.59:0/0) 2023-11-24T07:56:35.321 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.287+0000 7f9360b88700 1 --2- 172.21.15.59:0/1617312151 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f935c1478c0 0x7f935c13d6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:56:35.321 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.287+0000 7f935b7fe700 1 --2- 172.21.15.59:0/1617312151 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f935c0b5ab0 0x7f935c0aaad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:56:35.321 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.287+0000 7f935b7fe700 1 -- 172.21.15.59:0/1617312151 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f935c1478c0 msgr2=0x7f935c13d6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:56:35.321 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.287+0000 7f935b7fe700 1 --2- 172.21.15.59:0/1617312151 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f935c1478c0 0x7f935c13d6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:56:35.321 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.287+0000 7f935b7fe700 1 -- 172.21.15.59:0/1617312151 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f935c06b620 msgr2=0x7f935c0aa590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:56:35.321 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.287+0000 7f935b7fe700 1 --2- 172.21.15.59:0/1617312151 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f935c06b620 0x7f935c0aa590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:56:35.321 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.287+0000 7f935b7fe700 1 -- 172.21.15.59:0/1617312151 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9350009550 con 0x7f935c0b5ab0 2023-11-24T07:56:35.321 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.288+0000 7f935b7fe700 1 --2- 172.21.15.59:0/1617312151 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f935c0b5ab0 0x7f935c0aaad0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f9350016000 tx=0x7f9350004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:56:35.322 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.285+0000 7f7da4d6d700 1 Processor -- start 2023-11-24T07:56:35.322 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.287+0000 7f7da4d6d700 1 -- start start 2023-11-24T07:56:35.322 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.288+0000 7f7da4d6d700 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f7da0071390 0x7f7da00b4600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:56:35.322 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.288+0000 7f7da4d6d700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7da00ca2c0 0x7f7da01ef390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:56:35.322 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.288+0000 7f7da4d6d700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7da015db10 0x7f7da013d6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:56:35.322 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.288+0000 7f7da4d6d700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7da00b3560 con 0x7f7da00ca2c0 2023-11-24T07:56:35.322 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.288+0000 7f7da4d6d700 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7da00b3840 con 0x7f7da015db10 2023-11-24T07:56:35.322 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.288+0000 7f7da4d6d700 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7da00b3b20 con 0x7f7da0071390 2023-11-24T07:56:35.322 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.289+0000 7f7d9dd9b700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7da00ca2c0 0x7f7da01ef390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:56:35.322 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.289+0000 7f7d9dd9b700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7da00ca2c0 0x7f7da01ef390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:49996/0 (socket says 172.21.15.59:49996) 2023-11-24T07:56:35.322 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.289+0000 7f7d9dd9b700 1 -- 172.21.15.59:0/4922869 learned_addr learned my addr 172.21.15.59:0/4922869 (peer_addr_for_me v2:172.21.15.59:0/0) 2023-11-24T07:56:35.323 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.287+0000 7fba444a3700 1 -- 172.21.15.59:0/4255191709 shutdown_connections 2023-11-24T07:56:35.323 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.287+0000 7fba444a3700 1 -- 172.21.15.59:0/4255191709 wait complete. 2023-11-24T07:56:35.323 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.288+0000 7fba444a3700 1 Processor -- start 2023-11-24T07:56:35.323 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.287+0000 7f57a77fe700 1 -- 172.21.15.59:0/2354079022 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_get_version_reply(handle=1 version=20) v2 ==== 24+0+0 (secure 0 0 0) 0x7f57b4055d70 con 0x7f57c0141c00 2023-11-24T07:56:35.323 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.289+0000 7fba2a4f9700 1 --2- 172.21.15.59:0/2248999922 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fba2415a130 0x7fba2415a5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:56:35.324 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.289+0000 7fba29cf8700 1 -- 172.21.15.59:0/2248999922 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fba2415a130 msgr2=0x7fba2415a5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:56:35.324 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.289+0000 7f7d9ed9d700 1 --2- 172.21.15.59:0/4922869 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7da015db10 0x7f7da013d6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:56:35.324 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.289+0000 7f7d9e59c700 1 --2- 172.21.15.59:0/4922869 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f7da0071390 0x7f7da00b4600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:56:35.325 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.287+0000 7fbe80f28700 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fbe7c1668c0 0x7fbe7c0b49e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:56:35.325 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.287+0000 7fbe80f28700 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fbe7c14b730 0x7fbe7c0cdf50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:56:35.325 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.287+0000 7fbe80f28700 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbe7c0cd520 con 0x7fbe7c14b350 2023-11-24T07:56:35.325 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.287+0000 7fbe80f28700 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbe7c0cdb30 con 0x7fbe7c1668c0 2023-11-24T07:56:35.325 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.287+0000 7fbe80f28700 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbe7c14bf50 con 0x7fbe7c14b730 2023-11-24T07:56:35.325 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.288+0000 7fbe7ad9d700 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fbe7c14b730 0x7fbe7c0cdf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:56:35.325 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.288+0000 7fbe7ad9d700 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fbe7c14b730 0x7fbe7c0cdf50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3301/0 says I am v2:172.21.15.59:49432/0 (socket says 172.21.15.59:49432) 2023-11-24T07:56:35.325 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.288+0000 7fbe7ad9d700 1 -- 172.21.15.59:0/2616764326 learned_addr learned my addr 172.21.15.59:0/2616764326 (peer_addr_for_me v2:172.21.15.59:0/0) 2023-11-24T07:56:35.325 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.288+0000 7fbe7ad9d700 1 -- 172.21.15.59:0/2616764326 >> v2:172.21.15.76:3300/0 conn(0x7fbe7c1668c0 msgr2=0x7fbe7c0b49e0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-11-24T07:56:35.325 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.288+0000 7fbe7ad9d700 1 --2- 172.21.15.59:0/2616764326 >> v2:172.21.15.76:3300/0 conn(0x7fbe7c1668c0 0x7fbe7c0b49e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:56:35.326 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.289+0000 7fbe7ad9d700 1 -- 172.21.15.59:0/2616764326 >> v1:172.21.15.59:6789/0 conn(0x7fbe7c14b350 legacy=0x7fbe7c1661b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T07:56:35.326 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.289+0000 7fbe7ad9d700 1 -- 172.21.15.59:0/2616764326 --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbe7c1670f0 con 0x7fbe7c14b730 2023-11-24T07:56:35.326 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.289+0000 7fbe7ad9d700 1 --2- 172.21.15.59:0/2616764326 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fbe7c14b730 0x7fbe7c0cdf50 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7fbe6c0098f0 tx=0x7fbe6c009c00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=593e29de1b6657f5 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:56:35.327 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.289+0000 7f7d9dd9b700 1 -- 172.21.15.59:0/4922869 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f7da0071390 msgr2=0x7f7da00b4600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:56:35.327 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.289+0000 7f7d9dd9b700 1 --2- 172.21.15.59:0/4922869 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f7da0071390 0x7f7da00b4600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:56:35.327 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.289+0000 7f7d9dd9b700 1 -- 172.21.15.59:0/4922869 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7da015db10 msgr2=0x7f7da013d6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:56:35.327 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.289+0000 7f7d9dd9b700 1 --2- 172.21.15.59:0/4922869 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7da015db10 0x7f7da013d6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:56:35.327 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.289+0000 7f7d9dd9b700 1 -- 172.21.15.59:0/4922869 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7d9800c050 con 0x7f7da00ca2c0 2023-11-24T07:56:35.327 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.289+0000 7fba29cf8700 1 --2- 172.21.15.59:0/2248999922 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fba2415a130 0x7fba2415a5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:56:35.328 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.289+0000 7fba29cf8700 1 -- 172.21.15.59:0/2248999922 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fba240c7e90 msgr2=0x7fba240c8310 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-11-24T07:56:35.328 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.289+0000 7fba29cf8700 1 --2- 172.21.15.59:0/2248999922 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fba240c7e90 0x7fba240c8310 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:56:35.328 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.289+0000 7fba29cf8700 1 -- 172.21.15.59:0/2248999922 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fba2415adb0 con 0x7fba10009e50 2023-11-24T07:56:35.328 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.289+0000 7f9358ff9700 1 -- 172.21.15.59:0/1617312151 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9350004510 con 0x7f935c0b5ab0 2023-11-24T07:56:35.328 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.289+0000 7fbe7959a700 1 -- 172.21.15.59:0/2616764326 <== mon.2 v2:172.21.15.76:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe6c004230 con 0x7fbe7c14b730 2023-11-24T07:56:35.329 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.289+0000 7fbe7959a700 1 -- 172.21.15.59:0/2616764326 <== mon.2 v2:172.21.15.76:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbe6c004390 con 0x7fbe7c14b730 2023-11-24T07:56:35.329 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.289+0000 7fbe7959a700 1 -- 172.21.15.59:0/2616764326 <== mon.2 v2:172.21.15.76:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe6c0036a0 con 0x7fbe7c14b730 2023-11-24T07:56:35.329 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.289+0000 7f9e3effd700 1 --2- 172.21.15.59:0/1031333834 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f9e2c06af20 0x7f9e2c06d3e0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f9e340016b0 tx=0x7f9e340079c0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:56:35.329 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.289+0000 7f7d9dd9b700 1 --2- 172.21.15.59:0/4922869 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7da00ca2c0 0x7f7da01ef390 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f7d9000c780 tx=0x7f7d9000ca90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:56:35.330 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.289+0000 7fba2a4f9700 1 --2- 172.21.15.59:0/2248999922 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fba2415a130 0x7fba2415a5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T07:56:35.330 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.290+0000 7fba29cf8700 1 --2- 172.21.15.59:0/2248999922 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fba10009e50 0x7fba2408a820 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fba2000f7a0 tx=0x7fba2000fb60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:56:35.330 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.290+0000 7fba1a7fc700 1 -- 172.21.15.59:0/2248999922 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba20014680 con 0x7fba10009e50 2023-11-24T07:56:35.330 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.290+0000 7fba2bf5c700 1 -- 172.21.15.59:0/2248999922 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fba2415b090 con 0x7fba10009e50 2023-11-24T07:56:35.330 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.290+0000 7fba2bf5c700 1 -- 172.21.15.59:0/2248999922 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fba2415b650 con 0x7fba10009e50 2023-11-24T07:56:35.330 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.290+0000 7fba1a7fc700 1 -- 172.21.15.59:0/2248999922 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fba200147e0 con 0x7fba10009e50 2023-11-24T07:56:35.330 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.290+0000 7fba1a7fc700 1 -- 172.21.15.59:0/2248999922 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba20014b70 con 0x7fba10009e50 2023-11-24T07:56:35.331 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.289+0000 7f9358ff9700 1 -- 172.21.15.59:0/1617312151 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f93500036a0 con 0x7f935c0b5ab0 2023-11-24T07:56:35.331 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.289+0000 7f9358ff9700 1 -- 172.21.15.59:0/1617312151 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9350003a10 con 0x7f935c0b5ab0 2023-11-24T07:56:35.331 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.290+0000 7f9e277fe700 1 -- 172.21.15.59:0/1031333834 <== osd.0 v2:172.21.15.59:6800/3373878772 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+26965 (crc 0 0 0) 0x7f9e2c075c10 con 0x7f9e2c073140 2023-11-24T07:56:35.332 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.290+0000 7fbe80f28700 1 -- 172.21.15.59:0/2616764326 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fbe7c14b730 msgr2=0x7fbe7c0cdf50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:56:35.332 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.290+0000 7fbe80f28700 1 --2- 172.21.15.59:0/2616764326 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fbe7c14b730 0x7fbe7c0cdf50 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7fbe6c0098f0 tx=0x7fbe6c009c00 comp rx=0 tx=0).stop 2023-11-24T07:56:35.332 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.291+0000 7fbe80f28700 1 -- 172.21.15.59:0/2616764326 shutdown_connections 2023-11-24T07:56:35.332 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.291+0000 7fbe80f28700 1 --2- 172.21.15.59:0/2616764326 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fbe7c14b730 0x7fbe7c0cdf50 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:56:35.332 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.291+0000 7fbe80f28700 1 --2- 172.21.15.59:0/2616764326 >> v2:172.21.15.76:3300/0 conn(0x7fbe7c1668c0 0x7fbe7c0b49e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:56:35.332 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.291+0000 7fbe80f28700 1 -- 172.21.15.59:0/2616764326 >> 172.21.15.59:0/2616764326 conn(0x7fbe7c0b1cc0 msgr2=0x7fbe7c0b20a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:56:35.332 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.291+0000 7f7d8effd700 1 -- 172.21.15.59:0/4922869 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d90004c80 con 0x7f7da00ca2c0 2023-11-24T07:56:35.332 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.291+0000 7f7d8effd700 1 -- 172.21.15.59:0/4922869 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7d90004de0 con 0x7f7da00ca2c0 2023-11-24T07:56:35.333 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.291+0000 7f7d8effd700 1 -- 172.21.15.59:0/4922869 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d90007310 con 0x7f7da00ca2c0 2023-11-24T07:56:35.333 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.291+0000 7fbe80f28700 1 -- 172.21.15.59:0/2616764326 shutdown_connections 2023-11-24T07:56:35.333 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.291+0000 7fbe80f28700 1 -- 172.21.15.59:0/2616764326 wait complete. 2023-11-24T07:56:35.333 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.290+0000 7fba444a3700 1 -- start start 2023-11-24T07:56:35.334 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.292+0000 7f7da4d6d700 1 -- 172.21.15.59:0/4922869 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7da00b3e00 con 0x7f7da00ca2c0 2023-11-24T07:56:35.334 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.292+0000 7f7da4d6d700 1 -- 172.21.15.59:0/4922869 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7da01ef8d0 con 0x7f7da00ca2c0 2023-11-24T07:56:35.334 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.293+0000 7fbe80f28700 1 Processor -- start 2023-11-24T07:56:35.334 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.292+0000 7fba444a3700 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fba3c0720f0 0x7fba3c15c360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:56:35.334 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.292+0000 7fba444a3700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fba3c143ae0 0x7fba3c15c8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:56:35.335 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.292+0000 7fba444a3700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fba3c1440f0 0x7fba3c15cde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:56:35.335 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.292+0000 7fba444a3700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba3c203080 con 0x7fba3c143ae0 2023-11-24T07:56:35.335 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.292+0000 7fba444a3700 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba3c203640 con 0x7fba3c1440f0 2023-11-24T07:56:35.335 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.292+0000 7fba444a3700 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fba3c157770 con 0x7fba3c0720f0 2023-11-24T07:56:35.335 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.293+0000 7fba4223f700 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fba3c0720f0 0x7fba3c15c360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:56:35.335 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.293+0000 7fba4223f700 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fba3c0720f0 0x7fba3c15c360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3301/0 says I am v2:172.21.15.59:49442/0 (socket says 172.21.15.59:49442) 2023-11-24T07:56:35.335 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.293+0000 7fba4223f700 1 -- 172.21.15.59:0/29556039 learned_addr learned my addr 172.21.15.59:0/29556039 (peer_addr_for_me v2:172.21.15.59:0/0) 2023-11-24T07:56:35.335 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.293+0000 7fba42a40700 1 --2- 172.21.15.59:0/29556039 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fba3c1440f0 0x7fba3c15cde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:56:35.335 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.292+0000 7f93625eb700 1 -- 172.21.15.59:0/1617312151 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f935c15be70 con 0x7f935c0b5ab0 2023-11-24T07:56:35.336 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.292+0000 7f93625eb700 1 -- 172.21.15.59:0/1617312151 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f935c0ac5f0 con 0x7f935c0b5ab0 2023-11-24T07:56:35.336 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.293+0000 7fba4223f700 1 -- 172.21.15.59:0/29556039 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fba3c1440f0 msgr2=0x7fba3c15cde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:56:35.336 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.293+0000 7fba4223f700 1 --2- 172.21.15.59:0/29556039 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fba3c1440f0 0x7fba3c15cde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:56:35.336 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.293+0000 7fba4223f700 1 -- 172.21.15.59:0/29556039 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fba3c143ae0 msgr2=0x7fba3c15c8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:56:35.336 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.293+0000 7fba4223f700 1 --2- 172.21.15.59:0/29556039 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fba3c143ae0 0x7fba3c15c8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:56:35.336 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.293+0000 7fba4223f700 1 -- 172.21.15.59:0/29556039 --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fba2c00b7f0 con 0x7fba3c0720f0 2023-11-24T07:56:35.336 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.293+0000 7fba1a7fc700 1 -- 172.21.15.59:0/2248999922 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fba20014cd0 con 0x7fba10009e50 2023-11-24T07:56:35.337 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.293+0000 7fba1a7fc700 1 --2- 172.21.15.59:0/2248999922 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7fba0806aec0 0x7fba0806d380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:56:35.337 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.293+0000 7f93625eb700 1 -- 172.21.15.59:0/1617312151 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f935c0b3ae0 con 0x7f935c0b5ab0 2023-11-24T07:56:35.337 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.293+0000 7fba1a7fc700 1 -- 172.21.15.59:0/2248999922 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fba200896a0 con 0x7fba10009e50 2023-11-24T07:56:35.337 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.294+0000 7f7da4d6d700 1 -- 172.21.15.59:0/4922869 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f7da0071850 con 0x7f7da00ca2c0 2023-11-24T07:56:35.338 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.294+0000 7fba2bf5c700 1 --2- 172.21.15.59:0/2248999922 >> [v2:172.21.15.59:6824/648258563,v1:172.21.15.59:6825/648258563] conn(0x7fba0c0015d0 0x7fba0c003a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:56:35.338 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.294+0000 7fba2bf5c700 1 -- 172.21.15.59:0/2248999922 --> [v2:172.21.15.59:6824/648258563,v1:172.21.15.59:6825/648258563] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fba0c006bc0 con 0x7fba0c0015d0 2023-11-24T07:56:35.338 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.293+0000 7fbe80f28700 1 -- start start 2023-11-24T07:56:35.338 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.294+0000 7fbe80f28700 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fbe7c14b350 0x7fbe7c161ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:56:35.338 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.294+0000 7fbe80f28700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fbe7c14b730 0x7fbe7c164050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:56:35.338 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.294+0000 7fbe80f28700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbe7c1668c0 0x7fbe7c140f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:56:35.338 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.294+0000 7fbe80f28700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe7c165a40 con 0x7fbe7c1668c0 2023-11-24T07:56:35.339 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.294+0000 7fbe80f28700 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe7c162530 con 0x7fbe7c14b730 2023-11-24T07:56:35.339 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.294+0000 7fbe80f28700 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbe7c1649a0 con 0x7fbe7c14b350 2023-11-24T07:56:35.339 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.294+0000 7fbe7ad9d700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbe7c1668c0 0x7fbe7c140f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:56:35.339 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.294+0000 7fbe7ad9d700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbe7c1668c0 0x7fbe7c140f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:50010/0 (socket says 172.21.15.59:50010) 2023-11-24T07:56:35.339 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.294+0000 7fbe7ad9d700 1 -- 172.21.15.59:0/2838155481 learned_addr learned my addr 172.21.15.59:0/2838155481 (peer_addr_for_me v2:172.21.15.59:0/0) 2023-11-24T07:56:35.339 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.294+0000 7fbe7a59c700 1 --2- 172.21.15.59:0/2838155481 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fbe7c14b350 0x7fbe7c161ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:56:35.339 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.293+0000 7fba4223f700 1 --2- 172.21.15.59:0/29556039 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fba3c0720f0 0x7fba3c15c360 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7fba3c0d1bd0 tx=0x7fba2c004750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:56:35.339 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.294+0000 7fba32ffd700 1 -- 172.21.15.59:0/29556039 <== mon.2 v2:172.21.15.76:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba2c010670 con 0x7fba3c0720f0 2023-11-24T07:56:35.340 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.295+0000 7fbe79d9b700 1 --2- 172.21.15.59:0/2838155481 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fbe7c14b730 0x7fbe7c164050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:56:35.340 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.294+0000 7fba32ffd700 1 -- 172.21.15.59:0/29556039 <== mon.2 v2:172.21.15.76:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fba2c0107f0 con 0x7fba3c0720f0 2023-11-24T07:56:35.340 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.294+0000 7fba32ffd700 1 -- 172.21.15.59:0/29556039 <== mon.2 v2:172.21.15.76:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba2c010ba0 con 0x7fba3c0720f0 2023-11-24T07:56:35.340 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.294+0000 7fba444a3700 1 -- 172.21.15.59:0/29556039 --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fba3c0c5b50 con 0x7fba3c0720f0 2023-11-24T07:56:35.340 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.294+0000 7fba444a3700 1 -- 172.21.15.59:0/29556039 --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fba3c2032d0 con 0x7fba3c0720f0 2023-11-24T07:56:35.341 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.295+0000 7fba2a4f9700 1 --2- 172.21.15.59:0/2248999922 >> [v2:172.21.15.59:6824/648258563,v1:172.21.15.59:6825/648258563] conn(0x7fba0c0015d0 0x7fba0c003a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:56:35.341 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.295+0000 7fbe7ad9d700 1 -- 172.21.15.59:0/2838155481 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fbe7c14b350 msgr2=0x7fbe7c161ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:56:35.342 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.295+0000 7fbe7ad9d700 1 --2- 172.21.15.59:0/2838155481 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fbe7c14b350 0x7fbe7c161ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:56:35.342 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.295+0000 7fbe7ad9d700 1 -- 172.21.15.59:0/2838155481 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fbe7c14b730 msgr2=0x7fbe7c164050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:56:35.342 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.295+0000 7fbe7ad9d700 1 --2- 172.21.15.59:0/2838155481 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fbe7c14b730 0x7fbe7c164050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:56:35.342 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.295+0000 7fbe7ad9d700 1 -- 172.21.15.59:0/2838155481 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbe6c009550 con 0x7fbe7c1668c0 2023-11-24T07:56:35.342 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.295+0000 7fbe7ad9d700 1 --2- 172.21.15.59:0/2838155481 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbe7c1668c0 0x7fbe7c140f90 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fbe7c14c090 tx=0x7fbe6c004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:56:35.342 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.295+0000 7fbe6affd700 1 -- 172.21.15.59:0/2838155481 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe6c003e90 con 0x7fbe7c1668c0 2023-11-24T07:56:35.342 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.295+0000 7fbe6affd700 1 -- 172.21.15.59:0/2838155481 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbe6c00b660 con 0x7fbe7c1668c0 2023-11-24T07:56:35.343 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.295+0000 7fba2a4f9700 1 --2- 172.21.15.59:0/2248999922 >> [v2:172.21.15.59:6824/648258563,v1:172.21.15.59:6825/648258563] conn(0x7fba0c0015d0 0x7fba0c003a90 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:56:35.343 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.295+0000 7fbe6affd700 1 -- 172.21.15.59:0/2838155481 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe6c00ba10 con 0x7fbe7c1668c0 2023-11-24T07:56:35.343 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.296+0000 7fbe80f28700 1 -- 172.21.15.59:0/2838155481 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbe7c164c80 con 0x7fbe7c1668c0 2023-11-24T07:56:35.343 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.296+0000 7fbe80f28700 1 -- 172.21.15.59:0/2838155481 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbe7c165240 con 0x7fbe7c1668c0 2023-11-24T07:56:35.344 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.296+0000 7f7d8effd700 1 -- 172.21.15.59:0/4922869 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f7d90007470 con 0x7f7da00ca2c0 2023-11-24T07:56:35.344 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.297+0000 7f7d8effd700 1 --2- 172.21.15.59:0/4922869 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f7d8006afa0 0x7f7d8006d460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:56:35.344 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.297+0000 7f7d8effd700 1 -- 172.21.15.59:0/4922869 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f7d90089b30 con 0x7f7da00ca2c0 2023-11-24T07:56:35.344 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.297+0000 7f7d8effd700 1 --2- 172.21.15.59:0/4922869 >> [v2:172.21.15.76:6816/2382463431,v1:172.21.15.76:6818/2382463431] conn(0x7f7d800731c0 0x7f7d800755e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:56:35.344 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.297+0000 7f7d8effd700 1 -- 172.21.15.59:0/4922869 --> [v2:172.21.15.76:6816/2382463431,v1:172.21.15.76:6818/2382463431] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f7d80075c90 con 0x7f7d800731c0 2023-11-24T07:56:35.344 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.297+0000 7f7d8effd700 1 -- 172.21.15.59:0/4922869 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_get_version_reply(handle=1 version=20) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7d90089ee0 con 0x7f7da00ca2c0 2023-11-24T07:56:35.345 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.296+0000 7f9358ff9700 1 -- 172.21.15.59:0/1617312151 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f9350003b70 con 0x7f935c0b5ab0 2023-11-24T07:56:35.345 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.297+0000 7f9358ff9700 1 --2- 172.21.15.59:0/1617312151 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f934806aed0 0x7f934806d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:56:35.345 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.297+0000 7f9358ff9700 1 -- 172.21.15.59:0/1617312151 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f935008ae30 con 0x7f935c0b5ab0 2023-11-24T07:56:35.345 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.297+0000 7f9358ff9700 1 --2- 172.21.15.59:0/1617312151 >> [v2:172.21.15.59:6816/2504025850,v1:172.21.15.59:6817/2504025850] conn(0x7f93480730f0 0x7f9348075510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:56:35.345 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.297+0000 7f9358ff9700 1 -- 172.21.15.59:0/1617312151 --> [v2:172.21.15.59:6816/2504025850,v1:172.21.15.59:6817/2504025850] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f9348075bc0 con 0x7f93480730f0 2023-11-24T07:56:35.345 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.297+0000 7f9358ff9700 1 -- 172.21.15.59:0/1617312151 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_get_version_reply(handle=1 version=20) v2 ==== 24+0+0 (secure 0 0 0) 0x7f935008b1e0 con 0x7f935c0b5ab0 2023-11-24T07:56:35.345 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.297+0000 7f7d9e59c700 1 --2- 172.21.15.59:0/4922869 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f7d8006afa0 0x7f7d8006d460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:56:35.346 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.297+0000 7f935bfff700 1 --2- 172.21.15.59:0/1617312151 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f934806aed0 0x7f934806d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:56:35.346 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.297+0000 7f7d9ed9d700 1 --2- 172.21.15.59:0/4922869 >> [v2:172.21.15.76:6816/2382463431,v1:172.21.15.76:6818/2382463431] conn(0x7f7d800731c0 0x7f7d800755e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:56:35.346 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.297+0000 7fba32ffd700 1 -- 172.21.15.59:0/29556039 <== mon.2 v2:172.21.15.76:3301/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fba2c02c430 con 0x7fba3c0720f0 2023-11-24T07:56:35.346 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.298+0000 7f9360b88700 1 --2- 172.21.15.59:0/1617312151 >> [v2:172.21.15.59:6816/2504025850,v1:172.21.15.59:6817/2504025850] conn(0x7f93480730f0 0x7f9348075510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:56:35.347 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.298+0000 7f935bfff700 1 --2- 172.21.15.59:0/1617312151 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f934806aed0 0x7f934806d390 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f934c006fd0 tx=0x7f934c008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:56:35.347 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.298+0000 7fba294f7700 1 --2- 172.21.15.59:0/2248999922 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7fba0806aec0 0x7fba0806d380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:56:35.347 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.298+0000 7fba1a7fc700 1 -- 172.21.15.59:0/2248999922 <== osd.1 v2:172.21.15.59:6824/648258563 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+26965 (crc 0 0 0) 0x7fba0c006bc0 con 0x7fba0c0015d0 2023-11-24T07:56:35.347 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.298+0000 7fbe6affd700 1 -- 172.21.15.59:0/2838155481 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fbe6c022070 con 0x7fbe7c1668c0 2023-11-24T07:56:35.347 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.298+0000 7fba32ffd700 1 --2- 172.21.15.59:0/29556039 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7fba2006afa0 0x7fba2006d460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:56:35.348 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.298+0000 7fba32ffd700 1 -- 172.21.15.59:0/29556039 <== mon.2 v2:172.21.15.76:3301/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fba2c08b870 con 0x7fba3c0720f0 2023-11-24T07:56:35.348 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.298+0000 7fba41a3e700 1 --2- 172.21.15.59:0/29556039 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7fba2006afa0 0x7fba2006d460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:56:35.348 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.298+0000 7f9360b88700 1 --2- 172.21.15.59:0/1617312151 >> [v2:172.21.15.59:6816/2504025850,v1:172.21.15.59:6817/2504025850] conn(0x7f93480730f0 0x7f9348075510 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:56:35.348 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.299+0000 7fbe6affd700 1 --2- 172.21.15.59:0/2838155481 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7fbe5c06aed0 0x7fbe5c06d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:56:35.348 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.299+0000 7f7d9ed9d700 1 --2- 172.21.15.59:0/4922869 >> [v2:172.21.15.76:6816/2382463431,v1:172.21.15.76:6818/2382463431] conn(0x7f7d800731c0 0x7f7d800755e0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:56:35.349 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.299+0000 7f7d9e59c700 1 --2- 172.21.15.59:0/4922869 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f7d8006afa0 0x7f7d8006d460 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f7d98000f80 tx=0x7f7d9801d040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:56:35.349 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.299+0000 7fbe6affd700 1 -- 172.21.15.59:0/2838155481 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fbe6c08b0f0 con 0x7fbe7c1668c0 2023-11-24T07:56:35.349 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.299+0000 7f7d8effd700 1 -- 172.21.15.59:0/4922869 <== osd.4 v2:172.21.15.76:6816/2382463431 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+26965 (crc 0 0 0) 0x7f7d80075c90 con 0x7f7d800731c0 2023-11-24T07:56:35.349 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.300+0000 7fbe7a59c700 1 --2- 172.21.15.59:0/2838155481 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7fbe5c06aed0 0x7fbe5c06d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:56:35.350 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.300+0000 7f9358ff9700 1 -- 172.21.15.59:0/1617312151 <== osd.3 v2:172.21.15.59:6816/2504025850 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+26965 (crc 0 0 0) 0x7f9348075bc0 con 0x7f93480730f0 2023-11-24T07:56:35.350 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.300+0000 7fbe80f28700 1 --2- 172.21.15.59:0/2838155481 >> [v2:172.21.15.59:6808/1840430654,v1:172.21.15.59:6809/1840430654] conn(0x7fbe7c143230 0x7fbe7c0afc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:56:35.350 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.300+0000 7fbe80f28700 1 -- 172.21.15.59:0/2838155481 --> [v2:172.21.15.59:6808/1840430654,v1:172.21.15.59:6809/1840430654] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fbe7c0cd750 con 0x7fbe7c143230 2023-11-24T07:56:35.350 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.300+0000 7fbe7a59c700 1 --2- 172.21.15.59:0/2838155481 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7fbe5c06aed0 0x7fbe5c06d390 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fbe70007fd0 tx=0x7fbe70009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:56:35.350 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.300+0000 7fbe79d9b700 1 --2- 172.21.15.59:0/2838155481 >> [v2:172.21.15.59:6808/1840430654,v1:172.21.15.59:6809/1840430654] conn(0x7fbe7c143230 0x7fbe7c0afc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:56:35.351 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.301+0000 7fbe79d9b700 1 --2- 172.21.15.59:0/2838155481 >> [v2:172.21.15.59:6808/1840430654,v1:172.21.15.59:6809/1840430654] conn(0x7fbe7c143230 0x7fbe7c0afc30 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:56:35.351 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.302+0000 7fbe6affd700 1 -- 172.21.15.59:0/2838155481 <== osd.2 v2:172.21.15.59:6808/1840430654 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+26965 (crc 0 0 0) 0x7fbe74009b80 con 0x7fbe7c143230 2023-11-24T07:56:35.351 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.300+0000 7fba294f7700 1 --2- 172.21.15.59:0/2248999922 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7fba0806aec0 0x7fba0806d380 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fba2406c9c0 tx=0x7fba14009210 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:56:35.351 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.300+0000 7fba41a3e700 1 --2- 172.21.15.59:0/29556039 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7fba2006afa0 0x7fba2006d460 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fba3c0ceda0 tx=0x7fba38006cd0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:56:35.351 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.301+0000 7fba444a3700 1 --2- 172.21.15.59:0/29556039 >> [v2:172.21.15.76:6800/329334308,v1:172.21.15.76:6802/329334308] conn(0x7fba240015d0 0x7fba24003a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:56:35.351 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.301+0000 7fba444a3700 1 -- 172.21.15.59:0/29556039 --> [v2:172.21.15.76:6800/329334308,v1:172.21.15.76:6802/329334308] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fba24006bc0 con 0x7fba240015d0 2023-11-24T07:56:35.351 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.303+0000 7fba42a40700 1 --2- 172.21.15.59:0/29556039 >> [v2:172.21.15.76:6800/329334308,v1:172.21.15.76:6802/329334308] conn(0x7fba240015d0 0x7fba24003a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:56:35.352 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.303+0000 7fba42a40700 1 --2- 172.21.15.59:0/29556039 >> [v2:172.21.15.76:6800/329334308,v1:172.21.15.76:6802/329334308] conn(0x7fba240015d0 0x7fba24003a90 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:56:35.352 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.307+0000 7fba32ffd700 1 -- 172.21.15.59:0/29556039 <== osd.5 v2:172.21.15.76:6800/329334308 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+26965 (crc 0 0 0) 0x7fba24006bc0 con 0x7fba240015d0 2023-11-24T07:56:35.352 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.312+0000 7f57c5603700 1 -- 172.21.15.59:0/2354079022 --> [v2:172.21.15.76:6801/3946579410,v1:172.21.15.76:6803/3946579410] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f57c0144860 con 0x7f57ac073140 2023-11-24T07:56:35.353 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.312+0000 7f769c0c0700 1 Processor -- start 2023-11-24T07:56:35.353 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.312+0000 7f769c0c0700 1 -- start start 2023-11-24T07:56:35.353 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.313+0000 7f769c0c0700 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f7694070a80 0x7f7694070e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:56:35.353 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.313+0000 7f769c0c0700 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f7694145870 0x7f7694165ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:56:35.353 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.313+0000 7f769c0c0700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f7694166430 0x7f76940cad70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:56:35.353 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.313+0000 7f769c0c0700 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f76940ca8e0 con 0x7f7694166430 2023-11-24T07:56:35.353 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.313+0000 7f769c0c0700 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f76940713a0 con 0x7f7694145870 2023-11-24T07:56:35.354 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.313+0000 7f769c0c0700 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f76940c9fd0 con 0x7f7694070a80 2023-11-24T07:56:35.354 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.314+0000 7f7699e5c700 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f7694070a80 0x7f7694070e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:56:35.354 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.314+0000 7f7699e5c700 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f7694070a80 0x7f7694070e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3301/0 says I am v2:172.21.15.59:49472/0 (socket says 172.21.15.59:49472) 2023-11-24T07:56:35.354 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.314+0000 7f7699e5c700 1 -- 172.21.15.59:0/508085679 learned_addr learned my addr 172.21.15.59:0/508085679 (peer_addr_for_me v2:172.21.15.59:0/0) 2023-11-24T07:56:35.354 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.314+0000 7f7699e5c700 1 -- 172.21.15.59:0/508085679 >> v2:172.21.15.76:3300/0 conn(0x7f7694145870 msgr2=0x7f7694165ef0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-11-24T07:56:35.354 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.314+0000 7f7699e5c700 1 --2- 172.21.15.59:0/508085679 >> v2:172.21.15.76:3300/0 conn(0x7f7694145870 0x7f7694165ef0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:56:35.354 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.314+0000 7f7699e5c700 1 -- 172.21.15.59:0/508085679 >> v2:172.21.15.59:3300/0 conn(0x7f7694166430 msgr2=0x7f76940cad70 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-11-24T07:56:35.354 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.314+0000 7f7699e5c700 1 --2- 172.21.15.59:0/508085679 >> v2:172.21.15.59:3300/0 conn(0x7f7694166430 0x7f76940cad70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:56:35.354 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.314+0000 7f7699e5c700 1 -- 172.21.15.59:0/508085679 --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7694166c60 con 0x7f7694070a80 2023-11-24T07:56:35.354 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.314+0000 7f7699e5c700 1 --2- 172.21.15.59:0/508085679 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f7694070a80 0x7f7694070e60 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f7694069580 tx=0x7f7690009bd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8cee8bd4ce84c6ad server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:56:35.354 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.315+0000 7f7698e5a700 1 -- 172.21.15.59:0/508085679 <== mon.2 v2:172.21.15.76:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f769000b870 con 0x7f7694070a80 2023-11-24T07:56:35.354 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.315+0000 7f7698e5a700 1 -- 172.21.15.59:0/508085679 <== mon.2 v2:172.21.15.76:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f769000b9d0 con 0x7f7694070a80 2023-11-24T07:56:35.354 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.315+0000 7f7698e5a700 1 -- 172.21.15.59:0/508085679 <== mon.2 v2:172.21.15.76:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f769000bd60 con 0x7f7694070a80 2023-11-24T07:56:35.355 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.315+0000 7f769c0c0700 1 -- 172.21.15.59:0/508085679 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f7694070a80 msgr2=0x7f7694070e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:56:35.355 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.315+0000 7f769c0c0700 1 --2- 172.21.15.59:0/508085679 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f7694070a80 0x7f7694070e60 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f7694069580 tx=0x7f7690009bd0 comp rx=0 tx=0).stop 2023-11-24T07:56:35.355 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.319+0000 7f57a77fe700 1 -- 172.21.15.59:0/2354079022 <== osd.6 v2:172.21.15.76:6801/3946579410 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f57b80020f0 con 0x7f57ac073140 2023-11-24T07:56:35.355 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.319+0000 7f769c0c0700 1 -- 172.21.15.59:0/508085679 shutdown_connections 2023-11-24T07:56:35.355 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.319+0000 7f769c0c0700 1 --2- 172.21.15.59:0/508085679 >> v2:172.21.15.59:3300/0 conn(0x7f7694166430 0x7f76940cad70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:56:35.356 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.319+0000 7f769c0c0700 1 --2- 172.21.15.59:0/508085679 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f7694070a80 0x7f7694070e60 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:56:35.356 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.319+0000 7f769c0c0700 1 --2- 172.21.15.59:0/508085679 >> v2:172.21.15.76:3300/0 conn(0x7f7694145870 0x7f7694165ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:56:35.356 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.319+0000 7f769c0c0700 1 -- 172.21.15.59:0/508085679 >> 172.21.15.59:0/508085679 conn(0x7f76940cf980 msgr2=0x7f76940cfd90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:56:35.356 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.322+0000 7fba2bf5c700 1 -- 172.21.15.59:0/2248999922 --> [v2:172.21.15.59:6824/648258563,v1:172.21.15.59:6825/648258563] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fba0c005c40 con 0x7fba0c0015d0 2023-11-24T07:56:35.357 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.322+0000 7f57c5603700 1 -- 172.21.15.59:0/2354079022 >> [v2:172.21.15.76:6801/3946579410,v1:172.21.15.76:6803/3946579410] conn(0x7f57ac073140 msgr2=0x7f57ac075560 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:56:35.357 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.322+0000 7f57c5603700 1 --2- 172.21.15.59:0/2354079022 >> [v2:172.21.15.76:6801/3946579410,v1:172.21.15.76:6803/3946579410] conn(0x7f57ac073140 0x7f57ac075560 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:56:35.357 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.322+0000 7f57c5603700 1 -- 172.21.15.59:0/2354079022 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f57ac06afa0 msgr2=0x7f57ac06d460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:56:35.357 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.322+0000 7f57c5603700 1 --2- 172.21.15.59:0/2354079022 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f57ac06afa0 0x7f57ac06d460 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f57b0007fd0 tx=0x7f57b0009040 comp rx=0 tx=0).stop 2023-11-24T07:56:35.357 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.322+0000 7f57c5603700 1 -- 172.21.15.59:0/2354079022 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f57c0141c00 msgr2=0x7f57c016a3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:56:35.357 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.322+0000 7f57c5603700 1 --2- 172.21.15.59:0/2354079022 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f57c0141c00 0x7f57c016a3c0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f57b40018b0 tx=0x7f57b4001980 comp rx=0 tx=0).stop 2023-11-24T07:56:35.357 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.322+0000 7f57bf7fe700 1 -- 172.21.15.59:0/2354079022 reap_dead start 2023-11-24T07:56:35.357 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.323+0000 7f57c5603700 1 -- 172.21.15.59:0/2354079022 shutdown_connections 2023-11-24T07:56:35.357 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.323+0000 7f57c5603700 1 -- 172.21.15.59:0/2354079022 >> 172.21.15.59:0/2354079022 conn(0x7f57c00b2740 msgr2=0x7f57c00b4ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:56:35.358 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.323+0000 7f57c5603700 1 -- 172.21.15.59:0/2354079022 shutdown_connections 2023-11-24T07:56:35.358 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.323+0000 7f57c5603700 1 -- 172.21.15.59:0/2354079022 wait complete. 2023-11-24T07:56:35.358 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.324+0000 7fba1a7fc700 1 -- 172.21.15.59:0/2248999922 <== osd.1 v2:172.21.15.59:6824/648258563 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7fba0c005c40 con 0x7fba0c0015d0 2023-11-24T07:56:35.359 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.324+0000 7fba2bf5c700 1 -- 172.21.15.59:0/2248999922 >> [v2:172.21.15.59:6824/648258563,v1:172.21.15.59:6825/648258563] conn(0x7fba0c0015d0 msgr2=0x7fba0c003a90 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:56:35.359 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.324+0000 7fba2bf5c700 1 --2- 172.21.15.59:0/2248999922 >> [v2:172.21.15.59:6824/648258563,v1:172.21.15.59:6825/648258563] conn(0x7fba0c0015d0 0x7fba0c003a90 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:56:35.359 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.325+0000 7fba2bf5c700 1 -- 172.21.15.59:0/2248999922 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7fba0806aec0 msgr2=0x7fba0806d380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:56:35.359 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.325+0000 7fba2bf5c700 1 --2- 172.21.15.59:0/2248999922 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7fba0806aec0 0x7fba0806d380 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fba2406c9c0 tx=0x7fba14009210 comp rx=0 tx=0).stop 2023-11-24T07:56:35.359 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.325+0000 7fba2bf5c700 1 -- 172.21.15.59:0/2248999922 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fba10009e50 msgr2=0x7fba2408a820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:56:35.359 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.325+0000 7fba2bf5c700 1 --2- 172.21.15.59:0/2248999922 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fba10009e50 0x7fba2408a820 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fba2000f7a0 tx=0x7fba2000fb60 comp rx=0 tx=0).stop 2023-11-24T07:56:35.359 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.325+0000 7fba2a4f9700 1 -- 172.21.15.59:0/2248999922 reap_dead start 2023-11-24T07:56:35.359 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.327+0000 7fba2bf5c700 1 -- 172.21.15.59:0/2248999922 shutdown_connections 2023-11-24T07:56:35.359 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.327+0000 7fba2bf5c700 1 -- 172.21.15.59:0/2248999922 >> 172.21.15.59:0/2248999922 conn(0x7fba240cf980 msgr2=0x7fba240cb970 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:56:35.360 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.328+0000 7f9e45547700 1 -- 172.21.15.59:0/1031333834 --> [v2:172.21.15.59:6800/3373878772,v1:172.21.15.59:6801/3373878772] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f9e4016bcc0 con 0x7f9e2c073140 2023-11-24T07:56:35.360 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.329+0000 7f9e277fe700 1 -- 172.21.15.59:0/1031333834 <== osd.0 v2:172.21.15.59:6800/3373878772 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f9e4016bcc0 con 0x7f9e2c073140 2023-11-24T07:56:35.360 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.332+0000 7f93625eb700 1 -- 172.21.15.59:0/1617312151 --> [v2:172.21.15.59:6816/2504025850,v1:172.21.15.59:6817/2504025850] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f935c06ba00 con 0x7f93480730f0 2023-11-24T07:56:35.360 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.333+0000 7f9358ff9700 1 -- 172.21.15.59:0/1617312151 <== osd.3 v2:172.21.15.59:6816/2504025850 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f9354001ec0 con 0x7f93480730f0 2023-11-24T07:56:35.361 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.333+0000 7f93625eb700 1 -- 172.21.15.59:0/1617312151 >> [v2:172.21.15.59:6816/2504025850,v1:172.21.15.59:6817/2504025850] conn(0x7f93480730f0 msgr2=0x7f9348075510 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:56:35.361 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.333+0000 7f93625eb700 1 --2- 172.21.15.59:0/1617312151 >> [v2:172.21.15.59:6816/2504025850,v1:172.21.15.59:6817/2504025850] conn(0x7f93480730f0 0x7f9348075510 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:56:35.361 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.333+0000 7f93625eb700 1 -- 172.21.15.59:0/1617312151 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f934806aed0 msgr2=0x7f934806d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:56:35.361 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.333+0000 7f93625eb700 1 --2- 172.21.15.59:0/1617312151 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f934806aed0 0x7f934806d390 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f934c006fd0 tx=0x7f934c008040 comp rx=0 tx=0).stop 2023-11-24T07:56:35.362 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.333+0000 7f93625eb700 1 -- 172.21.15.59:0/1617312151 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f935c0b5ab0 msgr2=0x7f935c0aaad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:56:35.362 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.333+0000 7f93625eb700 1 --2- 172.21.15.59:0/1617312151 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f935c0b5ab0 0x7f935c0aaad0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f9350016000 tx=0x7f9350004750 comp rx=0 tx=0).stop 2023-11-24T07:56:35.362 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.333+0000 7f9360b88700 1 -- 172.21.15.59:0/1617312151 reap_dead start 2023-11-24T07:56:35.362 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.333+0000 7f93625eb700 1 -- 172.21.15.59:0/1617312151 shutdown_connections 2023-11-24T07:56:35.362 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.333+0000 7f93625eb700 1 -- 172.21.15.59:0/1617312151 >> 172.21.15.59:0/1617312151 conn(0x7f935c0b2740 msgr2=0x7f935c15ca60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:56:35.362 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.333+0000 7f93625eb700 1 -- 172.21.15.59:0/1617312151 shutdown_connections 2023-11-24T07:56:35.362 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.333+0000 7f93625eb700 1 -- 172.21.15.59:0/1617312151 wait complete. 2023-11-24T07:56:35.362 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.334+0000 7f7da4d6d700 1 -- 172.21.15.59:0/4922869 --> [v2:172.21.15.76:6816/2382463431,v1:172.21.15.76:6818/2382463431] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f7da00b4320 con 0x7f7d800731c0 2023-11-24T07:56:35.363 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.335+0000 7f7d8effd700 1 -- 172.21.15.59:0/4922869 <== osd.4 v2:172.21.15.76:6816/2382463431 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f7da00b4320 con 0x7f7d800731c0 2023-11-24T07:56:35.364 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.335+0000 7f7da4d6d700 1 -- 172.21.15.59:0/4922869 >> [v2:172.21.15.76:6816/2382463431,v1:172.21.15.76:6818/2382463431] conn(0x7f7d800731c0 msgr2=0x7f7d800755e0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:56:35.364 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.335+0000 7f7da4d6d700 1 --2- 172.21.15.59:0/4922869 >> [v2:172.21.15.76:6816/2382463431,v1:172.21.15.76:6818/2382463431] conn(0x7f7d800731c0 0x7f7d800755e0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:56:35.364 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.336+0000 7f7da4d6d700 1 -- 172.21.15.59:0/4922869 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f7d8006afa0 msgr2=0x7f7d8006d460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:56:35.364 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.336+0000 7f7da4d6d700 1 --2- 172.21.15.59:0/4922869 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f7d8006afa0 0x7f7d8006d460 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f7d98000f80 tx=0x7f7d9801d040 comp rx=0 tx=0).stop 2023-11-24T07:56:35.364 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.336+0000 7f7da4d6d700 1 -- 172.21.15.59:0/4922869 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7da00ca2c0 msgr2=0x7f7da01ef390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:56:35.364 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.336+0000 7f7da4d6d700 1 --2- 172.21.15.59:0/4922869 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7da00ca2c0 0x7f7da01ef390 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f7d9000c780 tx=0x7f7d9000ca90 comp rx=0 tx=0).stop 2023-11-24T07:56:35.364 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.336+0000 7f7d9ed9d700 1 -- 172.21.15.59:0/4922869 reap_dead start 2023-11-24T07:56:35.364 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.337+0000 7fba2bf5c700 1 -- 172.21.15.59:0/2248999922 shutdown_connections 2023-11-24T07:56:35.365 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.337+0000 7fbe80f28700 1 -- 172.21.15.59:0/2838155481 --> [v2:172.21.15.59:6808/1840430654,v1:172.21.15.59:6809/1840430654] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fbe7c14c150 con 0x7fbe7c143230 2023-11-24T07:56:35.365 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.337+0000 7fba2bf5c700 1 -- 172.21.15.59:0/2248999922 wait complete. 2023-11-24T07:56:35.365 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.337+0000 7fbe6affd700 1 -- 172.21.15.59:0/2838155481 <== osd.2 v2:172.21.15.59:6808/1840430654 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7fbe7400b410 con 0x7fbe7c143230 2023-11-24T07:56:35.366 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.337+0000 7f7da4d6d700 1 -- 172.21.15.59:0/4922869 shutdown_connections 2023-11-24T07:56:35.366 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.337+0000 7f7da4d6d700 1 -- 172.21.15.59:0/4922869 >> 172.21.15.59:0/4922869 conn(0x7f7da00cf980 msgr2=0x7f7da006d120 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:56:35.366 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.337+0000 7fba444a3700 1 -- 172.21.15.59:0/29556039 --> [v2:172.21.15.76:6800/329334308,v1:172.21.15.76:6802/329334308] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fba24005ca0 con 0x7fba240015d0 2023-11-24T07:56:35.366 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.337+0000 7f7da4d6d700 1 -- 172.21.15.59:0/4922869 shutdown_connections 2023-11-24T07:56:35.366 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.338+0000 7f7da4d6d700 1 -- 172.21.15.59:0/4922869 wait complete. 2023-11-24T07:56:35.367 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.337+0000 7f9e45547700 1 -- 172.21.15.59:0/1031333834 >> [v2:172.21.15.59:6800/3373878772,v1:172.21.15.59:6801/3373878772] conn(0x7f9e2c073140 msgr2=0x7f9e2c075560 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:56:35.367 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.337+0000 7f9e45547700 1 --2- 172.21.15.59:0/1031333834 >> [v2:172.21.15.59:6800/3373878772,v1:172.21.15.59:6801/3373878772] conn(0x7f9e2c073140 0x7f9e2c075560 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:56:35.367 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.338+0000 7f769c0c0700 1 -- 172.21.15.59:0/508085679 shutdown_connections 2023-11-24T07:56:35.368 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.338+0000 7f9e45547700 1 -- 172.21.15.59:0/1031333834 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f9e2c06af20 msgr2=0x7f9e2c06d3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:56:35.368 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.338+0000 7f9e45547700 1 --2- 172.21.15.59:0/1031333834 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f9e2c06af20 0x7f9e2c06d3e0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f9e340016b0 tx=0x7f9e340079c0 comp rx=0 tx=0).stop 2023-11-24T07:56:35.368 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.338+0000 7f9e45547700 1 -- 172.21.15.59:0/1031333834 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9e40070a80 msgr2=0x7f9e4006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:56:35.368 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.338+0000 7f9e45547700 1 --2- 172.21.15.59:0/1031333834 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9e40070a80 0x7f9e4006d990 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f9e3000dae0 tx=0x7f9e3000dbc0 comp rx=0 tx=0).stop 2023-11-24T07:56:35.368 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.338+0000 7f9e3f7fe700 1 -- 172.21.15.59:0/1031333834 reap_dead start 2023-11-24T07:56:35.368 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.338+0000 7f9e45547700 1 -- 172.21.15.59:0/1031333834 shutdown_connections 2023-11-24T07:56:35.368 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.338+0000 7f9e45547700 1 -- 172.21.15.59:0/1031333834 >> 172.21.15.59:0/1031333834 conn(0x7f9e400cf980 msgr2=0x7f9e4006e690 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:56:35.368 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.338+0000 7f9e45547700 1 -- 172.21.15.59:0/1031333834 shutdown_connections 2023-11-24T07:56:35.368 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.338+0000 7f9e45547700 1 -- 172.21.15.59:0/1031333834 wait complete. 2023-11-24T07:56:35.369 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.338+0000 7fba32ffd700 1 -- 172.21.15.59:0/29556039 <== osd.5 v2:172.21.15.76:6800/329334308 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7fba24005ca0 con 0x7fba240015d0 2023-11-24T07:56:35.369 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.339+0000 7fba444a3700 1 -- 172.21.15.59:0/29556039 >> [v2:172.21.15.76:6800/329334308,v1:172.21.15.76:6802/329334308] conn(0x7fba240015d0 msgr2=0x7fba24003a90 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:56:35.369 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.339+0000 7fba444a3700 1 --2- 172.21.15.59:0/29556039 >> [v2:172.21.15.76:6800/329334308,v1:172.21.15.76:6802/329334308] conn(0x7fba240015d0 0x7fba24003a90 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:56:35.369 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.339+0000 7fba444a3700 1 -- 172.21.15.59:0/29556039 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7fba2006afa0 msgr2=0x7fba2006d460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:56:35.370 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.339+0000 7fba444a3700 1 --2- 172.21.15.59:0/29556039 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7fba2006afa0 0x7fba2006d460 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fba3c0ceda0 tx=0x7fba38006cd0 comp rx=0 tx=0).stop 2023-11-24T07:56:35.370 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.339+0000 7fba444a3700 1 -- 172.21.15.59:0/29556039 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fba3c0720f0 msgr2=0x7fba3c15c360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:56:35.370 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.339+0000 7fba444a3700 1 --2- 172.21.15.59:0/29556039 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fba3c0720f0 0x7fba3c15c360 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7fba3c0d1bd0 tx=0x7fba2c004750 comp rx=0 tx=0).stop 2023-11-24T07:56:35.370 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.339+0000 7fba42a40700 1 -- 172.21.15.59:0/29556039 reap_dead start 2023-11-24T07:56:35.370 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.339+0000 7fba444a3700 1 -- 172.21.15.59:0/29556039 shutdown_connections 2023-11-24T07:56:35.370 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.339+0000 7fba444a3700 1 -- 172.21.15.59:0/29556039 >> 172.21.15.59:0/29556039 conn(0x7fba3c0b2740 msgr2=0x7fba3c06ea50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:56:35.370 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.340+0000 7fbe80f28700 1 -- 172.21.15.59:0/2838155481 >> [v2:172.21.15.59:6808/1840430654,v1:172.21.15.59:6809/1840430654] conn(0x7fbe7c143230 msgr2=0x7fbe7c0afc30 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:56:35.370 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.340+0000 7fbe80f28700 1 --2- 172.21.15.59:0/2838155481 >> [v2:172.21.15.59:6808/1840430654,v1:172.21.15.59:6809/1840430654] conn(0x7fbe7c143230 0x7fbe7c0afc30 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:56:35.371 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.338+0000 7f769c0c0700 1 -- 172.21.15.59:0/508085679 wait complete. 2023-11-24T07:56:35.371 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.340+0000 7f769c0c0700 1 Processor -- start 2023-11-24T07:56:35.371 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.340+0000 7fbe80f28700 1 -- 172.21.15.59:0/2838155481 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7fbe5c06aed0 msgr2=0x7fbe5c06d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:56:35.371 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.340+0000 7fbe80f28700 1 --2- 172.21.15.59:0/2838155481 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7fbe5c06aed0 0x7fbe5c06d390 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fbe70007fd0 tx=0x7fbe70009040 comp rx=0 tx=0).stop 2023-11-24T07:56:35.371 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.340+0000 7fbe80f28700 1 -- 172.21.15.59:0/2838155481 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbe7c1668c0 msgr2=0x7fbe7c140f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:56:35.371 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.340+0000 7fbe80f28700 1 --2- 172.21.15.59:0/2838155481 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbe7c1668c0 0x7fbe7c140f90 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fbe7c14c090 tx=0x7fbe6c004750 comp rx=0 tx=0).stop 2023-11-24T07:56:35.371 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.340+0000 7fbe7ad9d700 1 -- 172.21.15.59:0/2838155481 reap_dead start 2023-11-24T07:56:35.372 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.341+0000 7f769c0c0700 1 -- start start 2023-11-24T07:56:35.372 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.341+0000 7fbe80f28700 1 -- 172.21.15.59:0/2838155481 shutdown_connections 2023-11-24T07:56:35.372 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.341+0000 7fbe80f28700 1 -- 172.21.15.59:0/2838155481 >> 172.21.15.59:0/2838155481 conn(0x7fbe7c0b1cc0 msgr2=0x7fbe7c0b0c10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:56:35.373 INFO:teuthology.orchestra.run.smithi059.stdout:55834574855 2023-11-24T07:56:35.373 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2023-11-24T07:56:35.374 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.342+0000 7f769c0c0700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7694070a80 0x7f76941fc010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:56:35.374 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.342+0000 7f769c0c0700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7694145870 0x7f7694144b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:56:35.374 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.342+0000 7f769c0c0700 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f7694166430 0x7f76941f7e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:56:35.374 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.342+0000 7f769c0c0700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76941f8a10 con 0x7f7694145870 2023-11-24T07:56:35.374 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.342+0000 7f769c0c0700 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76941fb530 con 0x7f7694070a80 2023-11-24T07:56:35.375 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.342+0000 7f769c0c0700 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f76941fb810 con 0x7f7694166430 2023-11-24T07:56:35.375 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.342+0000 7f7699e5c700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7694070a80 0x7f76941fc010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:56:35.375 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.342+0000 7f7699e5c700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7694070a80 0x7f76941fc010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.59:37234/0 (socket says 172.21.15.59:37234) 2023-11-24T07:56:35.375 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.342+0000 7f7699e5c700 1 -- 172.21.15.59:0/822549232 learned_addr learned my addr 172.21.15.59:0/822549232 (peer_addr_for_me v2:172.21.15.59:0/0) 2023-11-24T07:56:35.375 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.342+0000 7f769a65d700 1 --2- 172.21.15.59:0/822549232 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f7694166430 0x7f76941f7e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:56:35.375 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.343+0000 7fbe80f28700 1 -- 172.21.15.59:0/2838155481 shutdown_connections 2023-11-24T07:56:35.375 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.343+0000 7fbe80f28700 1 -- 172.21.15.59:0/2838155481 wait complete. 2023-11-24T07:56:35.376 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.343+0000 7f7699e5c700 1 -- 172.21.15.59:0/822549232 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f7694166430 msgr2=0x7f76941f7e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:56:35.376 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.343+0000 7f7699e5c700 1 --2- 172.21.15.59:0/822549232 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f7694166430 0x7f76941f7e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:56:35.376 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.343+0000 7f7699e5c700 1 -- 172.21.15.59:0/822549232 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7694145870 msgr2=0x7f7694144b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:56:35.376 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.343+0000 7f7699e5c700 1 --2- 172.21.15.59:0/822549232 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7694145870 0x7f7694144b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:56:35.376 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.343+0000 7f7699e5c700 1 -- 172.21.15.59:0/822549232 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7690009550 con 0x7f7694070a80 2023-11-24T07:56:35.376 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.343+0000 7fba444a3700 1 -- 172.21.15.59:0/29556039 shutdown_connections 2023-11-24T07:56:35.377 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.343+0000 7fba444a3700 1 -- 172.21.15.59:0/29556039 wait complete. 2023-11-24T07:56:35.378 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.343+0000 7f7699e5c700 1 --2- 172.21.15.59:0/822549232 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7694070a80 0x7f76941fc010 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f7690014040 tx=0x7f7690004750 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:56:35.378 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.343+0000 7f768a7fc700 1 -- 172.21.15.59:0/822549232 <== mon.1 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7690004460 con 0x7f7694070a80 2023-11-24T07:56:35.378 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.343+0000 7f768a7fc700 1 -- 172.21.15.59:0/822549232 <== mon.1 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f769001e070 con 0x7f7694070a80 2023-11-24T07:56:35.378 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.343+0000 7f769c0c0700 1 -- 172.21.15.59:0/822549232 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f76941fbaf0 con 0x7f7694070a80 2023-11-24T07:56:35.379 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.343+0000 7f768a7fc700 1 -- 172.21.15.59:0/822549232 <== mon.1 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76900038e0 con 0x7f7694070a80 2023-11-24T07:56:35.379 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.343+0000 7f769c0c0700 1 -- 172.21.15.59:0/822549232 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f769415f570 con 0x7f7694070a80 2023-11-24T07:56:35.379 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.344+0000 7f769c0c0700 1 -- 172.21.15.59:0/822549232 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f76940ca570 con 0x7f7694070a80 2023-11-24T07:56:35.379 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.346+0000 7f768a7fc700 1 -- 172.21.15.59:0/822549232 <== mon.1 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f7690003a40 con 0x7f7694070a80 2023-11-24T07:56:35.379 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.346+0000 7f768a7fc700 1 --2- 172.21.15.59:0/822549232 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f767806af20 0x7f767806d3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:56:35.379 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.346+0000 7f768a7fc700 1 -- 172.21.15.59:0/822549232 <== mon.1 v2:172.21.15.76:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f7690020070 con 0x7f7694070a80 2023-11-24T07:56:35.379 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.346+0000 7f768a7fc700 1 --2- 172.21.15.59:0/822549232 >> [v2:172.21.15.76:6817/841761538,v1:172.21.15.76:6819/841761538] conn(0x7f7678073140 0x7f7678075560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:56:35.379 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.346+0000 7f768a7fc700 1 -- 172.21.15.59:0/822549232 --> [v2:172.21.15.76:6817/841761538,v1:172.21.15.76:6819/841761538] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f7678075c10 con 0x7f7678073140 2023-11-24T07:56:35.379 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.346+0000 7f768a7fc700 1 -- 172.21.15.59:0/822549232 <== mon.1 v2:172.21.15.76:3300/0 6 ==== mon_get_version_reply(handle=1 version=20) v2 ==== 24+0+0 (secure 0 0 0) 0x7f769008b220 con 0x7f7694070a80 2023-11-24T07:56:35.379 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.346+0000 7f769965b700 1 --2- 172.21.15.59:0/822549232 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f767806af20 0x7f767806d3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:56:35.379 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.347+0000 7f769a65d700 1 --2- 172.21.15.59:0/822549232 >> [v2:172.21.15.76:6817/841761538,v1:172.21.15.76:6819/841761538] conn(0x7f7678073140 0x7f7678075560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:56:35.379 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.347+0000 7f769965b700 1 --2- 172.21.15.59:0/822549232 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f767806af20 0x7f767806d3e0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f76940d07c0 tx=0x7f768c009250 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:56:35.379 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.347+0000 7f769a65d700 1 --2- 172.21.15.59:0/822549232 >> [v2:172.21.15.76:6817/841761538,v1:172.21.15.76:6819/841761538] conn(0x7f7678073140 0x7f7678075560 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:56:35.380 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.348+0000 7f768a7fc700 1 -- 172.21.15.59:0/822549232 <== osd.7 v2:172.21.15.76:6817/841761538 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+26965 (crc 0 0 0) 0x7f7678075c10 con 0x7f7678073140 2023-11-24T07:56:35.380 INFO:teuthology.orchestra.run.smithi059.stdout:55834574856 2023-11-24T07:56:35.380 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-11-24T07:56:35.381 INFO:teuthology.orchestra.run.smithi059.stdout:55834574856 2023-11-24T07:56:35.382 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-11-24T07:56:35.382 INFO:teuthology.orchestra.run.smithi059.stdout:55834574856 2023-11-24T07:56:35.382 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-11-24T07:56:35.384 INFO:teuthology.orchestra.run.smithi059.stdout:55834574855 2023-11-24T07:56:35.384 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2023-11-24T07:56:35.385 INFO:teuthology.orchestra.run.smithi059.stdout:55834574856 2023-11-24T07:56:35.385 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-11-24T07:56:35.386 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.367+0000 7f769c0c0700 1 -- 172.21.15.59:0/822549232 --> [v2:172.21.15.76:6817/841761538,v1:172.21.15.76:6819/841761538] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f7694067f60 con 0x7f7678073140 2023-11-24T07:56:35.386 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.368+0000 7f768a7fc700 1 -- 172.21.15.59:0/822549232 <== osd.7 v2:172.21.15.76:6817/841761538 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f7694067f60 con 0x7f7678073140 2023-11-24T07:56:35.387 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.368+0000 7f769c0c0700 1 -- 172.21.15.59:0/822549232 >> [v2:172.21.15.76:6817/841761538,v1:172.21.15.76:6819/841761538] conn(0x7f7678073140 msgr2=0x7f7678075560 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:56:35.387 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.368+0000 7f769c0c0700 1 --2- 172.21.15.59:0/822549232 >> [v2:172.21.15.76:6817/841761538,v1:172.21.15.76:6819/841761538] conn(0x7f7678073140 0x7f7678075560 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:56:35.387 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.368+0000 7f769c0c0700 1 -- 172.21.15.59:0/822549232 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f767806af20 msgr2=0x7f767806d3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:56:35.387 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.368+0000 7f769c0c0700 1 --2- 172.21.15.59:0/822549232 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f767806af20 0x7f767806d3e0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f76940d07c0 tx=0x7f768c009250 comp rx=0 tx=0).stop 2023-11-24T07:56:35.387 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.368+0000 7f769c0c0700 1 -- 172.21.15.59:0/822549232 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7694070a80 msgr2=0x7f76941fc010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:56:35.387 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.368+0000 7f769c0c0700 1 --2- 172.21.15.59:0/822549232 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7694070a80 0x7f76941fc010 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f7690014040 tx=0x7f7690004750 comp rx=0 tx=0).stop 2023-11-24T07:56:35.387 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.368+0000 7f769a65d700 1 -- 172.21.15.59:0/822549232 reap_dead start 2023-11-24T07:56:35.387 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.368+0000 7f769c0c0700 1 -- 172.21.15.59:0/822549232 shutdown_connections 2023-11-24T07:56:35.388 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.368+0000 7f769c0c0700 1 -- 172.21.15.59:0/822549232 >> 172.21.15.59:0/822549232 conn(0x7f76940cf980 msgr2=0x7f7694146890 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:56:35.388 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.368+0000 7f769c0c0700 1 -- 172.21.15.59:0/822549232 shutdown_connections 2023-11-24T07:56:35.388 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.368+0000 7f769c0c0700 1 -- 172.21.15.59:0/822549232 wait complete. 2023-11-24T07:56:35.388 INFO:teuthology.orchestra.run.smithi059.stdout:55834574856 2023-11-24T07:56:35.389 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2023-11-24T07:56:35.389 INFO:teuthology.orchestra.run.smithi059.stdout:55834574854 2023-11-24T07:56:35.390 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2023-11-24T07:56:35.640 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.638+0000 7f6e9fa82700 1 Processor -- start 2023-11-24T07:56:35.640 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.638+0000 7f6e9fa82700 1 -- start start 2023-11-24T07:56:35.641 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.639+0000 7f6e9fa82700 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f6e9815db10 0x7f6e9815def0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:56:35.641 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.639+0000 7f6e9fa82700 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6e980ce000 con 0x7f6e980ca2c0 2023-11-24T07:56:35.641 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.639+0000 7f6e9fa82700 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6e980ce2c0 con 0x7f6e98071390 2023-11-24T07:56:35.642 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.639+0000 7f6e9fa82700 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6e980ce5a0 con 0x7f6e9815db10 2023-11-24T07:56:35.642 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.640+0000 7fc4ef6a4700 1 Processor -- start 2023-11-24T07:56:35.643 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.640+0000 7f6e9d81e700 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f6e9815db10 0x7f6e9815def0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:56:35.643 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.640+0000 7f6e9d81e700 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f6e9815db10 0x7f6e9815def0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3301/0 says I am v2:172.21.15.59:49486/0 (socket says 172.21.15.59:49486) 2023-11-24T07:56:35.643 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.640+0000 7f6e9d81e700 1 -- 172.21.15.59:0/854767074 learned_addr learned my addr 172.21.15.59:0/854767074 (peer_addr_for_me v2:172.21.15.59:0/0) 2023-11-24T07:56:35.643 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.640+0000 7f6e9d81e700 1 -- 172.21.15.59:0/854767074 >> v1:172.21.15.76:6789/0 conn(0x7f6e98071390 legacy=0x7f6e9815e430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:56:35.643 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.640+0000 7f6e9d81e700 1 -- 172.21.15.59:0/854767074 >> v1:172.21.15.59:6789/0 conn(0x7f6e980ca2c0 legacy=0x7f6e980ca750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:56:35.644 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.640+0000 7f6e9d81e700 1 -- 172.21.15.59:0/854767074 --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6e980ce880 con 0x7f6e9815db10 2023-11-24T07:56:35.644 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.640+0000 7fc4ef6a4700 1 -- start start 2023-11-24T07:56:35.645 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.641+0000 7f6e9d81e700 1 --2- 172.21.15.59:0/854767074 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f6e9815db10 0x7f6e9815def0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f6e94008920 tx=0x7f6e9400d330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=25884d7c190d83a4 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:56:35.645 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.641+0000 7f6e9c81c700 1 -- 172.21.15.59:0/854767074 <== mon.2 v2:172.21.15.76:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e94010070 con 0x7f6e9815db10 2023-11-24T07:56:35.646 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.641+0000 7f6e9c81c700 1 -- 172.21.15.59:0/854767074 <== mon.2 v2:172.21.15.76:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6e9400da80 con 0x7f6e9815db10 2023-11-24T07:56:35.646 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.641+0000 7f6e9c81c700 1 -- 172.21.15.59:0/854767074 <== mon.2 v2:172.21.15.76:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e9400de10 con 0x7f6e9815db10 2023-11-24T07:56:35.646 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.642+0000 7fc4ef6a4700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fc4e815db10 0x7fc4e815def0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:56:35.647 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.642+0000 7fc4ef6a4700 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fc4e8071390 0x7fc4e815e430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:56:35.647 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.642+0000 7fc4ef6a4700 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fc4e815a520 0x7fc4e815a970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:56:35.647 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.642+0000 7fc4ef6a4700 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc4e80ce000 con 0x7fc4e815db10 2023-11-24T07:56:35.647 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.642+0000 7fc4ef6a4700 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc4e80ce2c0 con 0x7fc4e8071390 2023-11-24T07:56:35.647 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.642+0000 7fc4ef6a4700 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc4e80ce5a0 con 0x7fc4e815a520 2023-11-24T07:56:35.648 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.642+0000 7fc4ecc3f700 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fc4e8071390 0x7fc4e815e430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:56:35.648 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.642+0000 7fc4ecc3f700 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fc4e8071390 0x7fc4e815e430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.59:37252/0 (socket says 172.21.15.59:37252) 2023-11-24T07:56:35.648 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.642+0000 7fc4ecc3f700 1 -- 172.21.15.59:0/511209987 learned_addr learned my addr 172.21.15.59:0/511209987 (peer_addr_for_me v2:172.21.15.59:0/0) 2023-11-24T07:56:35.648 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.642+0000 7f6e9fa82700 1 -- 172.21.15.59:0/854767074 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f6e9815db10 msgr2=0x7f6e9815def0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:56:35.648 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.642+0000 7f6e9fa82700 1 --2- 172.21.15.59:0/854767074 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f6e9815db10 0x7f6e9815def0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f6e94008920 tx=0x7f6e9400d330 comp rx=0 tx=0).stop 2023-11-24T07:56:35.649 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.642+0000 7fc4edc41700 1 --2- 172.21.15.59:0/511209987 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fc4e815a520 0x7fc4e815a970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:56:35.649 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.642+0000 7fc4ecc3f700 1 -- 172.21.15.59:0/511209987 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fc4e815a520 msgr2=0x7fc4e815a970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T07:56:35.650 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.642+0000 7fc4ecc3f700 1 --2- 172.21.15.59:0/511209987 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fc4e815a520 0x7fc4e815a970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:56:35.650 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.642+0000 7fc4ecc3f700 1 -- 172.21.15.59:0/511209987 >> v2:172.21.15.59:3300/0 conn(0x7fc4e815db10 msgr2=0x7fc4e815def0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T07:56:35.650 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.642+0000 7fc4ecc3f700 1 --2- 172.21.15.59:0/511209987 >> v2:172.21.15.59:3300/0 conn(0x7fc4e815db10 0x7fc4e815def0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:56:35.650 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.642+0000 7fc4ecc3f700 1 -- 172.21.15.59:0/511209987 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc4e80ce880 con 0x7fc4e8071390 2023-11-24T07:56:35.650 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.643+0000 7fc4ecc3f700 1 --2- 172.21.15.59:0/511209987 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc4e8071390 0x7fc4e815e430 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fc4e8072e00 tx=0x7fc4d8008bd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8a7ed775f0ee03b7 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:56:35.653 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.643+0000 7fadf54a8700 1 Processor -- start 2023-11-24T07:56:35.653 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.643+0000 7fadf54a8700 1 -- start start 2023-11-24T07:56:35.654 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.644+0000 7fc4dffff700 1 -- 172.21.15.59:0/511209987 <== mon.1 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4d800b820 con 0x7fc4e8071390 2023-11-24T07:56:35.654 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.644+0000 7fadf54a8700 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fadf00ceff0 0x7fadf00c8cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:56:35.654 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.644+0000 7fadf54a8700 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fadf00c9690 con 0x7fadf00c9210 2023-11-24T07:56:35.655 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.644+0000 7fadf54a8700 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fadf00718e0 con 0x7fadf00cdc60 2023-11-24T07:56:35.655 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.644+0000 7fadf54a8700 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fadf00cf3d0 con 0x7fadf00ceff0 2023-11-24T07:56:35.656 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.644+0000 7fc4dffff700 1 -- 172.21.15.59:0/511209987 <== mon.1 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc4d800b980 con 0x7fc4e8071390 2023-11-24T07:56:35.656 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.644+0000 7fc4dffff700 1 -- 172.21.15.59:0/511209987 <== mon.1 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4d80126f0 con 0x7fc4e8071390 2023-11-24T07:56:35.656 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.644+0000 7fc4ef6a4700 1 -- 172.21.15.59:0/511209987 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc4e8071390 msgr2=0x7fc4e815e430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:56:35.656 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.644+0000 7fc4ef6a4700 1 --2- 172.21.15.59:0/511209987 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc4e8071390 0x7fc4e815e430 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fc4e8072e00 tx=0x7fc4d8008bd0 comp rx=0 tx=0).stop 2023-11-24T07:56:35.656 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.645+0000 7fc4ef6a4700 1 -- 172.21.15.59:0/511209987 shutdown_connections 2023-11-24T07:56:35.657 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.645+0000 7fc4ef6a4700 1 --2- 172.21.15.59:0/511209987 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fc4e815a520 0x7fc4e815a970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:56:35.657 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.645+0000 7fc4ef6a4700 1 --2- 172.21.15.59:0/511209987 >> v2:172.21.15.59:3300/0 conn(0x7fc4e815db10 0x7fc4e815def0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:56:35.657 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.645+0000 7fc4ef6a4700 1 --2- 172.21.15.59:0/511209987 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc4e8071390 0x7fc4e815e430 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:56:35.657 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.645+0000 7fc4ef6a4700 1 -- 172.21.15.59:0/511209987 >> 172.21.15.59:0/511209987 conn(0x7fc4e80cf980 msgr2=0x7fc4e80cfd90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:56:35.657 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.645+0000 7fc4ef6a4700 1 -- 172.21.15.59:0/511209987 shutdown_connections 2023-11-24T07:56:35.658 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.645+0000 7fc4ef6a4700 1 -- 172.21.15.59:0/511209987 wait complete. 2023-11-24T07:56:35.658 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.645+0000 7fadef7fe700 1 --1- >> v1:172.21.15.76:6789/0 conn(0x7fadf00cdc60 0x7fadf0161a90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.59:40810/0 (socket says 172.21.15.59:40810) 2023-11-24T07:56:35.658 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.645+0000 7fadef7fe700 1 -- 172.21.15.59:0/3155326323 learned_addr learned my addr 172.21.15.59:0/3155326323 (peer_addr_for_me v1:172.21.15.59:0/0) 2023-11-24T07:56:35.659 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.645+0000 7fadeeffd700 1 --2- 172.21.15.59:0/3155326323 >> v2:172.21.15.76:3300/0 conn(0x7fadf00ceff0 0x7fadf00c8cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:56:35.659 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.645+0000 7fadedffb700 1 -- 172.21.15.59:0/3155326323 <== mon.0 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 263461237 0 0) 0x7fadf00c9690 con 0x7fadf00c9210 2023-11-24T07:56:35.659 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.645+0000 7fadedffb700 1 -- 172.21.15.59:0/3155326323 --> v1:172.21.15.59:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fadd80035b0 con 0x7fadf00c9210 2023-11-24T07:56:35.659 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.645+0000 7f7577216700 1 Processor -- start 2023-11-24T07:56:35.660 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.645+0000 7fadeeffd700 1 -- 172.21.15.59:0/3155326323 >> v1:172.21.15.76:6789/0 conn(0x7fadf00cdc60 legacy=0x7fadf0161a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:56:35.660 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.645+0000 7fadeeffd700 1 -- 172.21.15.59:0/3155326323 >> v1:172.21.15.59:6789/0 conn(0x7fadf00c9210 legacy=0x7fadf00cd550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:56:35.660 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.645+0000 7fadeeffd700 1 -- 172.21.15.59:0/3155326323 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fadf015e4a0 con 0x7fadf00ceff0 2023-11-24T07:56:35.660 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.646+0000 7f7577216700 1 -- start start 2023-11-24T07:56:35.660 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.646+0000 7fadeeffd700 1 --2- 172.21.15.59:0/3155326323 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fadf00ceff0 0x7fadf00c8cd0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fade0009840 tx=0x7fade0009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2b9c258602571ac4 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:56:35.661 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.646+0000 7fadedffb700 1 -- 172.21.15.59:0/3155326323 <== mon.1 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fade0011830 con 0x7fadf00ceff0 2023-11-24T07:56:35.661 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.646+0000 7fadedffb700 1 -- 172.21.15.59:0/3155326323 <== mon.1 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fade0011990 con 0x7fadf00ceff0 2023-11-24T07:56:35.661 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.646+0000 7fadedffb700 1 -- 172.21.15.59:0/3155326323 <== mon.1 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fade0011d20 con 0x7fadf00ceff0 2023-11-24T07:56:35.661 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.646+0000 7fc4ef6a4700 1 Processor -- start 2023-11-24T07:56:35.662 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.647+0000 7fadf54a8700 1 -- 172.21.15.59:0/3155326323 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fadf00ceff0 msgr2=0x7fadf00c8cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:56:35.662 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.647+0000 7fadf54a8700 1 --2- 172.21.15.59:0/3155326323 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fadf00ceff0 0x7fadf00c8cd0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fade0009840 tx=0x7fade0009b50 comp rx=0 tx=0).stop 2023-11-24T07:56:35.662 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.647+0000 7fadf54a8700 1 -- 172.21.15.59:0/3155326323 shutdown_connections 2023-11-24T07:56:35.662 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.647+0000 7fadf54a8700 1 --2- 172.21.15.59:0/3155326323 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fadf00ceff0 0x7fadf00c8cd0 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:56:35.662 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.647+0000 7fadf54a8700 1 -- 172.21.15.59:0/3155326323 >> 172.21.15.59:0/3155326323 conn(0x7fadf00cf980 msgr2=0x7fadf00cfd90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:56:35.663 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.647+0000 7f7577216700 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f75700ceff0 0x7f75700c8cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:56:35.663 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.647+0000 7f7577216700 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f75700c9210 0x7f75700cd550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:56:35.663 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.647+0000 7f7577216700 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f75700c9690 con 0x7f75700cda90 2023-11-24T07:56:35.663 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.647+0000 7f7577216700 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f75700cf3d0 con 0x7f75700ceff0 2023-11-24T07:56:35.663 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.647+0000 7f7577216700 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f75700718e0 con 0x7f75700c9210 2023-11-24T07:56:35.663 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.647+0000 7f75757b3700 1 --1- >> v1:172.21.15.76:6789/0 conn(0x7f75700cda90 0x7f75700c9fd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.59:40818/0 (socket says 172.21.15.59:40818) 2023-11-24T07:56:35.664 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.647+0000 7f75757b3700 1 -- 172.21.15.59:0/2023645943 learned_addr learned my addr 172.21.15.59:0/2023645943 (peer_addr_for_me v1:172.21.15.59:0/0) 2023-11-24T07:56:35.664 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.647+0000 7f756ffff700 1 --2- 172.21.15.59:0/2023645943 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f75700c9210 0x7f75700cd550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:56:35.664 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.647+0000 7f7574fb2700 1 --2- 172.21.15.59:0/2023645943 >> v2:172.21.15.76:3300/0 conn(0x7f75700ceff0 0x7f75700c8cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:56:35.664 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.647+0000 7f756f7fe700 1 -- 172.21.15.59:0/2023645943 <== mon.1 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3133404481 0 0) 0x7f75700c9690 con 0x7f75700cda90 2023-11-24T07:56:35.664 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.648+0000 7f756f7fe700 1 -- 172.21.15.59:0/2023645943 --> v1:172.21.15.76:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f75580035b0 con 0x7f75700cda90 2023-11-24T07:56:35.664 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.647+0000 7fbfcda41700 1 Processor -- start 2023-11-24T07:56:35.665 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.647+0000 7f32eee79700 1 Processor -- start 2023-11-24T07:56:35.665 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.648+0000 7f7574fb2700 1 -- 172.21.15.59:0/2023645943 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f75700c9210 msgr2=0x7f75700cd550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T07:56:35.665 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.648+0000 7f7574fb2700 1 --2- 172.21.15.59:0/2023645943 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f75700c9210 0x7f75700cd550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:56:35.665 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.648+0000 7f7574fb2700 1 -- 172.21.15.59:0/2023645943 >> v1:172.21.15.76:6789/0 conn(0x7f75700cda90 legacy=0x7f75700c9fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:56:35.665 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.648+0000 7f7574fb2700 1 -- 172.21.15.59:0/2023645943 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f757015e4a0 con 0x7f75700ceff0 2023-11-24T07:56:35.666 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.648+0000 7f7574fb2700 1 --2- 172.21.15.59:0/2023645943 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f75700ceff0 0x7f75700c8cd0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f7568009d90 tx=0x7f756800b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f4161d1f5f4f561d server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:56:35.666 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.648+0000 7f756f7fe700 1 -- 172.21.15.59:0/2023645943 <== mon.1 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f756800e070 con 0x7f75700ceff0 2023-11-24T07:56:35.666 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.648+0000 7f756f7fe700 1 -- 172.21.15.59:0/2023645943 <== mon.1 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f756800bb20 con 0x7f75700ceff0 2023-11-24T07:56:35.666 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.648+0000 7f756f7fe700 1 -- 172.21.15.59:0/2023645943 <== mon.1 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7568012430 con 0x7f75700ceff0 2023-11-24T07:56:35.666 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.649+0000 7f6e9fa82700 1 -- 172.21.15.59:0/854767074 shutdown_connections 2023-11-24T07:56:35.666 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.649+0000 7f6e9fa82700 1 --2- 172.21.15.59:0/854767074 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f6e9815db10 0x7f6e9815def0 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:56:35.666 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.649+0000 7f6e9fa82700 1 -- 172.21.15.59:0/854767074 >> 172.21.15.59:0/854767074 conn(0x7f6e980cf980 msgr2=0x7f6e980cfd90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:56:35.667 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.649+0000 7f6e9fa82700 1 -- 172.21.15.59:0/854767074 shutdown_connections 2023-11-24T07:56:35.667 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.647+0000 7fc4ef6a4700 1 -- start start 2023-11-24T07:56:35.667 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.648+0000 7fc4ef6a4700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc4e8071390 0x7fc4e8159780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:56:35.667 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.648+0000 7fc4ef6a4700 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fc4e815a520 0x7fc4e81f79d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:56:35.667 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.648+0000 7fc4ef6a4700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc4e815db10 0x7fc4e81fb800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:56:35.667 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.648+0000 7fc4ef6a4700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4e81592f0 con 0x7fc4e815db10 2023-11-24T07:56:35.667 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.648+0000 7fc4ef6a4700 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4e80b57f0 con 0x7fc4e8071390 2023-11-24T07:56:35.667 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.648+0000 7fc4ef6a4700 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc4e80b5510 con 0x7fc4e815a520 2023-11-24T07:56:35.667 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.648+0000 7fc4edc41700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc4e815db10 0x7fc4e81fb800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:56:35.668 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.648+0000 7fc4edc41700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc4e815db10 0x7fc4e81fb800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:50070/0 (socket says 172.21.15.59:50070) 2023-11-24T07:56:35.668 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.648+0000 7fc4edc41700 1 -- 172.21.15.59:0/768754559 learned_addr learned my addr 172.21.15.59:0/768754559 (peer_addr_for_me v2:172.21.15.59:0/0) 2023-11-24T07:56:35.668 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.648+0000 7fc4ed440700 1 --2- 172.21.15.59:0/768754559 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc4e8071390 0x7fc4e8159780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:56:35.668 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.649+0000 7fc4ed440700 1 -- 172.21.15.59:0/768754559 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fc4e815a520 msgr2=0x7fc4e81f79d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:56:35.668 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.649+0000 7fc4ed440700 1 --2- 172.21.15.59:0/768754559 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fc4e815a520 0x7fc4e81f79d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:56:35.668 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.649+0000 7fc4ed440700 1 -- 172.21.15.59:0/768754559 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc4e815db10 msgr2=0x7fc4e81fb800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:56:35.668 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.649+0000 7fc4ed440700 1 --2- 172.21.15.59:0/768754559 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc4e815db10 0x7fc4e81fb800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:56:35.668 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.649+0000 7fc4ed440700 1 -- 172.21.15.59:0/768754559 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc4d80085d0 con 0x7fc4e8071390 2023-11-24T07:56:35.668 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.649+0000 7fc4ed440700 1 --2- 172.21.15.59:0/768754559 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc4e8071390 0x7fc4e8159780 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fc4e000bd70 tx=0x7fc4e000bda0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:56:35.669 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.647+0000 7fbfcda41700 1 -- start start 2023-11-24T07:56:35.669 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.649+0000 7fbfcda41700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fbfc80b7050 0x7fbfc80b15f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:56:35.669 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.649+0000 7fbfcda41700 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fbfc809d160 0x7fbfc80b58c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:56:35.669 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.649+0000 7fbfcda41700 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbfc80b7730 con 0x7fbfc80b5e00 2023-11-24T07:56:35.669 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.649+0000 7fbfcda41700 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbfc80b6290 con 0x7fbfc80b7050 2023-11-24T07:56:35.669 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.649+0000 7fbfcda41700 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbfc80b7a10 con 0x7fbfc809d160 2023-11-24T07:56:35.669 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.649+0000 7fbfc6ffd700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fbfc80b7050 0x7fbfc80b15f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:56:35.670 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.649+0000 7fbfc6ffd700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fbfc80b7050 0x7fbfc80b15f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:50078/0 (socket says 172.21.15.59:50078) 2023-11-24T07:56:35.670 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.649+0000 7fbfc6ffd700 1 -- 172.21.15.59:0/1589925849 learned_addr learned my addr 172.21.15.59:0/1589925849 (peer_addr_for_me v2:172.21.15.59:0/0) 2023-11-24T07:56:35.670 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.647+0000 7f32eee79700 1 -- start start 2023-11-24T07:56:35.670 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.649+0000 7f32eee79700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f32e80722c0 0x7f32e80b51b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:56:35.670 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.649+0000 7f32eee79700 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f32e81440f0 0x7f32e806e510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:56:35.670 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.649+0000 7f32eee79700 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f32e8143ec0 con 0x7f32e8143ae0 2023-11-24T07:56:35.670 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.649+0000 7f32eee79700 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f32e80b40a0 con 0x7f32e80722c0 2023-11-24T07:56:35.670 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.649+0000 7f32eee79700 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f32e80b3ae0 con 0x7f32e81440f0 2023-11-24T07:56:35.671 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.649+0000 7f32ed416700 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f32e81440f0 0x7f32e806e510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:56:35.671 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.649+0000 7f32ed416700 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f32e81440f0 0x7f32e806e510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.59:37274/0 (socket says 172.21.15.59:37274) 2023-11-24T07:56:35.671 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.649+0000 7f32ed416700 1 -- 172.21.15.59:0/29173670 learned_addr learned my addr 172.21.15.59:0/29173670 (peer_addr_for_me v2:172.21.15.59:0/0) 2023-11-24T07:56:35.671 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.649+0000 7fbfc67fc700 1 --2- 172.21.15.59:0/1589925849 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fbfc809d160 0x7fbfc80b58c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:56:35.671 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.649+0000 7f6e9fa82700 1 -- 172.21.15.59:0/854767074 wait complete. 2023-11-24T07:56:35.672 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.649+0000 7f7577216700 1 -- 172.21.15.59:0/2023645943 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f75700ceff0 msgr2=0x7f75700c8cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:56:35.672 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.649+0000 7f7577216700 1 --2- 172.21.15.59:0/2023645943 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f75700ceff0 0x7f75700c8cd0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f7568009d90 tx=0x7f756800b330 comp rx=0 tx=0).stop 2023-11-24T07:56:35.672 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.649+0000 7f7577216700 1 -- 172.21.15.59:0/2023645943 shutdown_connections 2023-11-24T07:56:35.672 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.649+0000 7f7577216700 1 --2- 172.21.15.59:0/2023645943 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f75700ceff0 0x7f75700c8cd0 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:56:35.672 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.649+0000 7f7577216700 1 --2- 172.21.15.59:0/2023645943 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f75700c9210 0x7f75700cd550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:56:35.672 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.649+0000 7f7577216700 1 -- 172.21.15.59:0/2023645943 >> 172.21.15.59:0/2023645943 conn(0x7f75700cf980 msgr2=0x7f75700cfd90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:56:35.673 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.650+0000 7fbfc67fc700 1 -- 172.21.15.59:0/1589925849 >> v2:172.21.15.59:3300/0 conn(0x7fbfc80b7050 msgr2=0x7fbfc80b15f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T07:56:35.673 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.650+0000 7fbfc67fc700 1 --2- 172.21.15.59:0/1589925849 >> v2:172.21.15.59:3300/0 conn(0x7fbfc80b7050 0x7fbfc80b15f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:56:35.673 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.650+0000 7fbfc67fc700 1 -- 172.21.15.59:0/1589925849 >> v1:172.21.15.59:6789/0 conn(0x7fbfc80b5e00 legacy=0x7fbfc809bd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T07:56:35.673 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.650+0000 7fbfc67fc700 1 -- 172.21.15.59:0/1589925849 --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbfc80b7cf0 con 0x7fbfc809d160 2023-11-24T07:56:35.673 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.650+0000 7fbfc67fc700 1 --2- 172.21.15.59:0/1589925849 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fbfc809d160 0x7fbfc80b58c0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fbfbc009840 tx=0x7fbfbc009b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=da12bf07212f7ed9 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:56:35.674 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.649+0000 7f32dffff700 1 --2- 172.21.15.59:0/29173670 >> v2:172.21.15.59:3300/0 conn(0x7f32e80722c0 0x7f32e80b51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:56:35.674 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.649+0000 7f32ed416700 1 -- 172.21.15.59:0/29173670 >> v2:172.21.15.59:3300/0 conn(0x7f32e80722c0 msgr2=0x7f32e80b51b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T07:56:35.674 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.650+0000 7f32ed416700 1 --2- 172.21.15.59:0/29173670 >> v2:172.21.15.59:3300/0 conn(0x7f32e80722c0 0x7f32e80b51b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:56:35.674 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.650+0000 7f32ed416700 1 -- 172.21.15.59:0/29173670 >> v1:172.21.15.59:6789/0 conn(0x7f32e8143ae0 legacy=0x7f32e8071bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:56:35.674 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.650+0000 7f32ed416700 1 -- 172.21.15.59:0/29173670 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f32e80b3dc0 con 0x7f32e81440f0 2023-11-24T07:56:35.674 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.650+0000 7f32ed416700 1 --2- 172.21.15.59:0/29173670 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f32e81440f0 0x7f32e806e510 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f32e815f950 tx=0x7f32e000b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cd985d0865aae9a server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:56:35.675 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.650+0000 7fadf54a8700 1 -- 172.21.15.59:0/3155326323 shutdown_connections 2023-11-24T07:56:35.675 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.650+0000 7fbfc5ffb700 1 -- 172.21.15.59:0/1589925849 <== mon.2 v2:172.21.15.76:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbfbc004230 con 0x7fbfc809d160 2023-11-24T07:56:35.675 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.650+0000 7fbfc5ffb700 1 -- 172.21.15.59:0/1589925849 <== mon.2 v2:172.21.15.76:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbfbc004390 con 0x7fbfc809d160 2023-11-24T07:56:35.675 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.650+0000 7fbfc5ffb700 1 -- 172.21.15.59:0/1589925849 <== mon.2 v2:172.21.15.76:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbfbc0036a0 con 0x7fbfc809d160 2023-11-24T07:56:35.675 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.650+0000 7f32df7fe700 1 -- 172.21.15.59:0/29173670 <== mon.1 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32e000bbc0 con 0x7f32e81440f0 2023-11-24T07:56:35.675 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.650+0000 7f32df7fe700 1 -- 172.21.15.59:0/29173670 <== mon.1 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f32e000bd20 con 0x7f32e81440f0 2023-11-24T07:56:35.676 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.650+0000 7f32df7fe700 1 -- 172.21.15.59:0/29173670 <== mon.1 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32e0010710 con 0x7f32e81440f0 2023-11-24T07:56:35.676 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.651+0000 7fadf54a8700 1 -- 172.21.15.59:0/3155326323 wait complete. 2023-11-24T07:56:35.676 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.651+0000 7f6e9fa82700 1 Processor -- start 2023-11-24T07:56:35.676 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.651+0000 7f6e9fa82700 1 -- start start 2023-11-24T07:56:35.677 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.651+0000 7f32eee79700 1 -- 172.21.15.59:0/29173670 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f32e81440f0 msgr2=0x7f32e806e510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:56:35.677 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.651+0000 7f32eee79700 1 --2- 172.21.15.59:0/29173670 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f32e81440f0 0x7f32e806e510 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f32e815f950 tx=0x7f32e000b330 comp rx=0 tx=0).stop 2023-11-24T07:56:35.677 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.651+0000 7fbfcda41700 1 -- 172.21.15.59:0/1589925849 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fbfc809d160 msgr2=0x7fbfc80b58c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:56:35.677 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.651+0000 7fbfcda41700 1 --2- 172.21.15.59:0/1589925849 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fbfc809d160 0x7fbfc80b58c0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fbfbc009840 tx=0x7fbfbc009b50 comp rx=0 tx=0).stop 2023-11-24T07:56:35.677 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.651+0000 7fbfcda41700 1 -- 172.21.15.59:0/1589925849 shutdown_connections 2023-11-24T07:56:35.677 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.651+0000 7fbfcda41700 1 --2- 172.21.15.59:0/1589925849 >> v2:172.21.15.59:3300/0 conn(0x7fbfc80b7050 0x7fbfc80b15f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:56:35.678 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.651+0000 7fbfcda41700 1 --2- 172.21.15.59:0/1589925849 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fbfc809d160 0x7fbfc80b58c0 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:56:35.678 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.651+0000 7fbfcda41700 1 -- 172.21.15.59:0/1589925849 >> 172.21.15.59:0/1589925849 conn(0x7fbfc80d08c0 msgr2=0x7fbfc80d0cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:56:35.678 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.651+0000 7f32eee79700 1 -- 172.21.15.59:0/29173670 shutdown_connections 2023-11-24T07:56:35.678 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.651+0000 7f32eee79700 1 --2- 172.21.15.59:0/29173670 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f32e81440f0 0x7f32e806e510 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:56:35.678 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.651+0000 7f32eee79700 1 --2- 172.21.15.59:0/29173670 >> v2:172.21.15.59:3300/0 conn(0x7f32e80722c0 0x7f32e80b51b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:56:35.678 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.651+0000 7f32eee79700 1 -- 172.21.15.59:0/29173670 >> 172.21.15.59:0/29173670 conn(0x7f32e80b2740 msgr2=0x7f32e80b2b20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:56:35.679 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.651+0000 7fbfcda41700 1 -- 172.21.15.59:0/1589925849 shutdown_connections 2023-11-24T07:56:35.679 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.651+0000 7f1cf9685700 1 Processor -- start 2023-11-24T07:56:35.679 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.651+0000 7f32eee79700 1 -- 172.21.15.59:0/29173670 shutdown_connections 2023-11-24T07:56:35.680 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.652+0000 7fc4ddffb700 1 -- 172.21.15.59:0/768754559 <== mon.1 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4e000cc90 con 0x7fc4e8071390 2023-11-24T07:56:35.680 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.652+0000 7fc4ddffb700 1 -- 172.21.15.59:0/768754559 <== mon.1 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc4e000cdf0 con 0x7fc4e8071390 2023-11-24T07:56:35.680 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.652+0000 7fc4ddffb700 1 -- 172.21.15.59:0/768754559 <== mon.1 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4e0014680 con 0x7fc4e8071390 2023-11-24T07:56:35.680 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.652+0000 7fc4ef6a4700 1 -- 172.21.15.59:0/768754559 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc4e8158ca0 con 0x7fc4e8071390 2023-11-24T07:56:35.680 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.652+0000 7fc4ef6a4700 1 -- 172.21.15.59:0/768754559 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc4e81fc530 con 0x7fc4e8071390 2023-11-24T07:56:35.681 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.652+0000 7f6e9fa82700 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f6e98071390 0x7f6e9806cf00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:56:35.681 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.652+0000 7f6e9fa82700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6e980ca2c0 0x7f6e98159280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:56:35.681 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.652+0000 7f6e9fa82700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6e9815db10 0x7f6e981597c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:56:35.681 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.652+0000 7f6e9fa82700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e98159d00 con 0x7f6e9815db10 2023-11-24T07:56:35.681 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.652+0000 7f6e9fa82700 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e981f8510 con 0x7f6e980ca2c0 2023-11-24T07:56:35.681 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.652+0000 7f6e9fa82700 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6e9815a450 con 0x7f6e98071390 2023-11-24T07:56:35.681 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.652+0000 7f6e9d81e700 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f6e98071390 0x7f6e9806cf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:56:35.681 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.653+0000 7f6e9d01d700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6e980ca2c0 0x7f6e98159280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:56:35.681 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.653+0000 7f6e9d01d700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6e980ca2c0 0x7f6e98159280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.59:37286/0 (socket says 172.21.15.59:37286) 2023-11-24T07:56:35.681 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.653+0000 7f6e9d01d700 1 -- 172.21.15.59:0/3903790008 learned_addr learned my addr 172.21.15.59:0/3903790008 (peer_addr_for_me v2:172.21.15.59:0/0) 2023-11-24T07:56:35.682 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.652+0000 7f7577216700 1 -- 172.21.15.59:0/2023645943 shutdown_connections 2023-11-24T07:56:35.682 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.653+0000 7f7577216700 1 -- 172.21.15.59:0/2023645943 wait complete. 2023-11-24T07:56:35.682 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.653+0000 7fadf54a8700 1 Processor -- start 2023-11-24T07:56:35.683 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.652+0000 7f1cf9685700 1 -- start start 2023-11-24T07:56:35.683 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.653+0000 7f1cf9685700 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f1cf4166680 0x7f1cf4070a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:56:35.683 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.653+0000 7f1cf9685700 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1cf406d1f0 con 0x7f1cf40cbb80 2023-11-24T07:56:35.683 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.653+0000 7f1cf9685700 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1cf40cc080 con 0x7f1cf414bf10 2023-11-24T07:56:35.683 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.653+0000 7f1cf9685700 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1cf40cdf60 con 0x7f1cf4166680 2023-11-24T07:56:35.683 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.653+0000 7f1cf2ffd700 1 --1- >> v1:172.21.15.76:6789/0 conn(0x7f1cf414bf10 0x7f1cf406cae0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.59:40846/0 (socket says 172.21.15.59:40846) 2023-11-24T07:56:35.683 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.653+0000 7f1cf2ffd700 1 -- 172.21.15.59:0/1910015220 learned_addr learned my addr 172.21.15.59:0/1910015220 (peer_addr_for_me v1:172.21.15.59:0/0) 2023-11-24T07:56:35.684 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.654+0000 7f1cf1ffb700 1 -- 172.21.15.59:0/1910015220 <== mon.1 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1223320190 0 0) 0x7f1cf40cc080 con 0x7f1cf414bf10 2023-11-24T07:56:35.684 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.654+0000 7f1cf1ffb700 1 -- 172.21.15.59:0/1910015220 --> v1:172.21.15.76:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1cdc0035b0 con 0x7f1cf414bf10 2023-11-24T07:56:35.684 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.654+0000 7f1cf1ffb700 1 -- 172.21.15.59:0/1910015220 <== mon.1 v1:172.21.15.76:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2866516101 0 0) 0x7f1cdc0035b0 con 0x7f1cf414bf10 2023-11-24T07:56:35.684 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.654+0000 7f1cf1ffb700 1 -- 172.21.15.59:0/1910015220 >> v2:172.21.15.76:3300/0 conn(0x7f1cf4166680 msgr2=0x7f1cf4070a80 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-11-24T07:56:35.684 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.654+0000 7f1cf1ffb700 1 --2- 172.21.15.59:0/1910015220 >> v2:172.21.15.76:3300/0 conn(0x7f1cf4166680 0x7f1cf4070a80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:56:35.684 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.654+0000 7f1cf1ffb700 1 -- 172.21.15.59:0/1910015220 >> v1:172.21.15.59:6789/0 conn(0x7f1cf40cbb80 legacy=0x7f1cf4165f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T07:56:35.684 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.654+0000 7f1cf1ffb700 1 -- 172.21.15.59:0/1910015220 --> v1:172.21.15.76:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1cf406e950 con 0x7f1cf414bf10 2023-11-24T07:56:35.684 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.654+0000 7f1cf1ffb700 1 -- 172.21.15.59:0/1910015220 <== mon.1 v1:172.21.15.76:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1312333566 0 0) 0x7f1ce40019f0 con 0x7f1cf414bf10 2023-11-24T07:56:35.684 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.654+0000 7f1cf1ffb700 1 -- 172.21.15.59:0/1910015220 >> v1:172.21.15.76:6789/0 conn(0x7f1cf414bf10 legacy=0x7f1cf406cae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:56:35.684 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.654+0000 7f1cf1ffb700 1 --2- 172.21.15.59:0/1910015220 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1cdc003a00 0x7f1cdc003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:56:35.685 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.654+0000 7f1cf1ffb700 1 --2- 172.21.15.59:0/1910015220 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f1cdc004850 0x7f1cdc004cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:56:35.685 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.654+0000 7f1cf1ffb700 1 --2- 172.21.15.59:0/1910015220 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1cdc005330 0x7f1cdc00ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:56:35.685 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.654+0000 7f1cf1ffb700 1 -- 172.21.15.59:0/1910015220 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1cdc00c200 con 0x7f1cdc003a00 2023-11-24T07:56:35.685 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.654+0000 7f1cf1ffb700 1 -- 172.21.15.59:0/1910015220 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1cdc00c370 con 0x7f1cdc005330 2023-11-24T07:56:35.685 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.654+0000 7f1cf1ffb700 1 -- 172.21.15.59:0/1910015220 --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1cdc00c530 con 0x7f1cdc004850 2023-11-24T07:56:35.685 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.651+0000 7fbfcda41700 1 -- 172.21.15.59:0/1589925849 wait complete. 2023-11-24T07:56:35.686 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.653+0000 7fbfcda41700 1 Processor -- start 2023-11-24T07:56:35.686 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.654+0000 7fbfcda41700 1 -- start start 2023-11-24T07:56:35.686 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.653+0000 7f32eee79700 1 -- 172.21.15.59:0/29173670 wait complete. 2023-11-24T07:56:35.686 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.654+0000 7f32eee79700 1 Processor -- start 2023-11-24T07:56:35.686 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.655+0000 7f32eee79700 1 -- start start 2023-11-24T07:56:35.687 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.655+0000 7f7577216700 1 Processor -- start 2023-11-24T07:56:35.687 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.655+0000 7fbfcda41700 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fbfc809d160 0x7fbfc80ca440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:56:35.687 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.655+0000 7fbfcda41700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbfc80b5e00 0x7fbfc80c7890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:56:35.687 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.655+0000 7fbfcda41700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fbfc80b7050 0x7fbfc81571a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:56:35.687 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.655+0000 7fbfcda41700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbfc80c7dd0 con 0x7fbfc80b5e00 2023-11-24T07:56:35.687 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.655+0000 7fbfcda41700 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbfc80c8000 con 0x7fbfc80b7050 2023-11-24T07:56:35.687 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.655+0000 7fbfcda41700 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbfc80c82e0 con 0x7fbfc809d160 2023-11-24T07:56:35.688 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.655+0000 7f1cf37fe700 1 --2- 172.21.15.59:0/1910015220 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1cdc005330 0x7f1cdc00ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:56:35.688 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.655+0000 7f6e9d01d700 1 -- 172.21.15.59:0/3903790008 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f6e98071390 msgr2=0x7f6e9806cf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:56:35.688 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.655+0000 7f6e9d01d700 1 --2- 172.21.15.59:0/3903790008 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f6e98071390 0x7f6e9806cf00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:56:35.688 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.655+0000 7f6e9d01d700 1 -- 172.21.15.59:0/3903790008 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6e9815db10 msgr2=0x7f6e981597c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:56:35.688 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.655+0000 7f6e9d01d700 1 --2- 172.21.15.59:0/3903790008 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6e9815db10 0x7f6e981597c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:56:35.689 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.655+0000 7f6e9d01d700 1 -- 172.21.15.59:0/3903790008 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6e940085d0 con 0x7f6e980ca2c0 2023-11-24T07:56:35.689 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.653+0000 7fadf54a8700 1 -- start start 2023-11-24T07:56:35.689 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.654+0000 7fadf54a8700 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fadf00c9210 0x7fadf0159bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:56:35.689 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.654+0000 7fadf54a8700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fadf00cdc60 0x7fadf0157720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:56:35.689 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.654+0000 7fadf54a8700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fadf0144e30 0x7fadf013bfd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:56:35.690 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.654+0000 7fadf54a8700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fadf0157e20 con 0x7fadf0144e30 2023-11-24T07:56:35.690 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.654+0000 7fadf54a8700 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fadf01fb7b0 con 0x7fadf00cdc60 2023-11-24T07:56:35.690 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.654+0000 7fadf54a8700 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fadf01fba90 con 0x7fadf00c9210 2023-11-24T07:56:35.690 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.655+0000 7fadeeffd700 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fadf00c9210 0x7fadf0159bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:56:35.690 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.655+0000 7fadeeffd700 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fadf00c9210 0x7fadf0159bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3301/0 says I am v2:172.21.15.59:49542/0 (socket says 172.21.15.59:49542) 2023-11-24T07:56:35.691 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.655+0000 7fadeeffd700 1 -- 172.21.15.59:0/1833136323 learned_addr learned my addr 172.21.15.59:0/1833136323 (peer_addr_for_me v2:172.21.15.59:0/0) 2023-11-24T07:56:35.691 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.655+0000 7fadee7fc700 1 --2- 172.21.15.59:0/1833136323 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fadf00cdc60 0x7fadf0157720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:56:35.691 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.655+0000 7fadef7fe700 1 --2- 172.21.15.59:0/1833136323 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fadf0144e30 0x7fadf013bfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:56:35.692 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.655+0000 7f7577216700 1 -- start start 2023-11-24T07:56:35.692 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.655+0000 7fbfc6ffd700 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fbfc809d160 0x7fbfc80ca440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:56:35.692 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.655+0000 7fbfc6ffd700 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fbfc809d160 0x7fbfc80ca440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3301/0 says I am v2:172.21.15.59:49550/0 (socket says 172.21.15.59:49550) 2023-11-24T07:56:35.693 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.655+0000 7fbfc6ffd700 1 -- 172.21.15.59:0/1812054286 learned_addr learned my addr 172.21.15.59:0/1812054286 (peer_addr_for_me v2:172.21.15.59:0/0) 2023-11-24T07:56:35.693 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.655+0000 7fbfc77fe700 1 --2- 172.21.15.59:0/1812054286 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fbfc80b7050 0x7fbfc81571a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:56:35.693 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.655+0000 7f1cf2ffd700 1 --2- 172.21.15.59:0/1910015220 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1cdc003a00 0x7f1cdc003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:56:35.694 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.655+0000 7f1cf37fe700 1 -- 172.21.15.59:0/1910015220 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f1cdc004850 msgr2=0x7f1cdc004cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T07:56:35.694 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.655+0000 7f1cf37fe700 1 --2- 172.21.15.59:0/1910015220 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f1cdc004850 0x7f1cdc004cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:56:35.695 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.655+0000 7f1cf37fe700 1 -- 172.21.15.59:0/1910015220 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1cdc003a00 msgr2=0x7f1cdc003e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T07:56:35.695 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.655+0000 7f1cf37fe700 1 --2- 172.21.15.59:0/1910015220 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1cdc003a00 0x7f1cdc003e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:56:35.695 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.655+0000 7f1cf37fe700 1 -- 172.21.15.59:0/1910015220 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1cf406e950 con 0x7f1cdc005330 2023-11-24T07:56:35.695 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.655+0000 7f1cf37fe700 1 -- 172.21.15.59:0/1910015220 reap_dead start 2023-11-24T07:56:35.696 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.655+0000 7fc4ddffb700 1 -- 172.21.15.59:0/768754559 <== mon.1 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fc4e00147e0 con 0x7fc4e8071390 2023-11-24T07:56:35.696 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.655+0000 7fc4ef6a4700 1 -- 172.21.15.59:0/768754559 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc4e8067f60 con 0x7fc4e8071390 2023-11-24T07:56:35.696 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.656+0000 7fbfc6ffd700 1 -- 172.21.15.59:0/1812054286 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fbfc80b7050 msgr2=0x7fbfc81571a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:56:35.696 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.656+0000 7fbfc6ffd700 1 --2- 172.21.15.59:0/1812054286 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fbfc80b7050 0x7fbfc81571a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:56:35.696 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.656+0000 7fbfc6ffd700 1 -- 172.21.15.59:0/1812054286 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbfc80b5e00 msgr2=0x7fbfc80c7890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:56:35.696 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.656+0000 7fbfc6ffd700 1 --2- 172.21.15.59:0/1812054286 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbfc80b5e00 0x7fbfc80c7890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:56:35.696 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.656+0000 7fbfc6ffd700 1 -- 172.21.15.59:0/1812054286 --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbfbc0094a0 con 0x7fbfc809d160 2023-11-24T07:56:35.697 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.655+0000 7fadeeffd700 1 -- 172.21.15.59:0/1833136323 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fadf00cdc60 msgr2=0x7fadf0157720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:56:35.697 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.655+0000 7fadeeffd700 1 --2- 172.21.15.59:0/1833136323 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fadf00cdc60 0x7fadf0157720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:56:35.697 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.655+0000 7fadeeffd700 1 -- 172.21.15.59:0/1833136323 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fadf0144e30 msgr2=0x7fadf013bfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:56:35.697 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.655+0000 7fadeeffd700 1 --2- 172.21.15.59:0/1833136323 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fadf0144e30 0x7fadf013bfd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:56:35.697 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.655+0000 7fadeeffd700 1 -- 172.21.15.59:0/1833136323 --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fade00094a0 con 0x7fadf00c9210 2023-11-24T07:56:35.697 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.655+0000 7fadef7fe700 1 --2- 172.21.15.59:0/1833136323 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fadf0144e30 0x7fadf013bfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-24T07:56:35.697 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.655+0000 7f1cf2ffd700 1 --2- 172.21.15.59:0/1910015220 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1cdc003a00 0x7f1cdc003e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T07:56:35.698 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.656+0000 7f1cf37fe700 1 --2- 172.21.15.59:0/1910015220 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1cdc005330 0x7f1cdc00ba10 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f1cec006fd0 tx=0x7f1cdc005880 comp rx=0 tx=0).ready entity=mon.1 client_cookie=57245c2ccbaf0bfe server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:56:35.698 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.656+0000 7fc4ddffb700 1 --2- 172.21.15.59:0/768754559 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7fc4cc06afa0 0x7fc4cc06d460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:56:35.698 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.656+0000 7fc4ecc3f700 1 --2- 172.21.15.59:0/768754559 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7fc4cc06afa0 0x7fc4cc06d460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:56:35.698 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.656+0000 7fadeeffd700 1 --2- 172.21.15.59:0/1833136323 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fadf00c9210 0x7fadf0159bf0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fade0009fd0 tx=0x7fade0004750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:56:35.699 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.656+0000 7fc4ddffb700 1 -- 172.21.15.59:0/768754559 <== mon.1 v2:172.21.15.76:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fc4e0089440 con 0x7fc4e8071390 2023-11-24T07:56:35.699 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.656+0000 7fadd77fe700 1 -- 172.21.15.59:0/1833136323 <== mon.2 v2:172.21.15.76:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fade00036a0 con 0x7fadf00c9210 2023-11-24T07:56:35.699 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.656+0000 7f32eee79700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f32e80722c0 0x7f32e8155250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:56:35.699 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.656+0000 7f32eee79700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f32e8143ae0 0x7f32e8153d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:56:35.700 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.656+0000 7f32eee79700 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f32e81440f0 0x7f32e815da40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:56:35.700 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.656+0000 7f32eee79700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32e8134a40 con 0x7f32e8143ae0 2023-11-24T07:56:35.700 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.656+0000 7f32eee79700 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32e8152cf0 con 0x7f32e80722c0 2023-11-24T07:56:35.700 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.656+0000 7f32eee79700 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f32e8152fd0 con 0x7f32e81440f0 2023-11-24T07:56:35.700 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.656+0000 7f32dffff700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f32e8143ae0 0x7f32e8153d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:56:35.700 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.656+0000 7f1cf1ffb700 1 -- 172.21.15.59:0/1910015220 <== mon.1 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1cec015070 con 0x7f1cdc005330 2023-11-24T07:56:35.700 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.656+0000 7f32dffff700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f32e8143ae0 0x7f32e8153d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:50120/0 (socket says 172.21.15.59:50120) 2023-11-24T07:56:35.701 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.656+0000 7f32dffff700 1 -- 172.21.15.59:0/3739815000 learned_addr learned my addr 172.21.15.59:0/3739815000 (peer_addr_for_me v2:172.21.15.59:0/0) 2023-11-24T07:56:35.701 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.656+0000 7f1cf1ffb700 1 -- 172.21.15.59:0/1910015220 <== mon.1 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1cec011420 con 0x7f1cdc005330 2023-11-24T07:56:35.701 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.656+0000 7f1cf1ffb700 1 -- 172.21.15.59:0/1910015220 <== mon.1 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1cec0117b0 con 0x7f1cdc005330 2023-11-24T07:56:35.701 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.656+0000 7fadf54a8700 1 -- 172.21.15.59:0/1833136323 --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fadf01fbd70 con 0x7fadf00c9210 2023-11-24T07:56:35.701 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.656+0000 7fadf54a8700 1 -- 172.21.15.59:0/1833136323 --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fadf01fc570 con 0x7fadf00c9210 2023-11-24T07:56:35.702 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.656+0000 7f32ed416700 1 --2- 172.21.15.59:0/3739815000 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f32e81440f0 0x7f32e815da40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:56:35.702 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.657+0000 7f32ecc15700 1 --2- 172.21.15.59:0/3739815000 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f32e80722c0 0x7f32e8155250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:56:35.702 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.657+0000 7fc4ecc3f700 1 --2- 172.21.15.59:0/768754559 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7fc4cc06afa0 0x7fc4cc06d460 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fc4d8008c10 tx=0x7fc4d8013000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:56:35.703 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.657+0000 7f7577216700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f75700c9210 0x7f7570158b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:56:35.703 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.657+0000 7f7577216700 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f75700cda90 0x7f757013f200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:56:35.703 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.657+0000 7f7577216700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f75700ceff0 0x7f75700720f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:56:35.703 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.657+0000 7f7577216700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75701590a0 con 0x7f75700ceff0 2023-11-24T07:56:35.703 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.657+0000 7f7577216700 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f757013f740 con 0x7f75700c9210 2023-11-24T07:56:35.703 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.657+0000 7f7577216700 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f75701592d0 con 0x7f75700cda90 2023-11-24T07:56:35.703 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.657+0000 7f75757b3700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f75700ceff0 0x7f75700720f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:56:35.703 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.657+0000 7f75757b3700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f75700ceff0 0x7f75700720f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:50126/0 (socket says 172.21.15.59:50126) 2023-11-24T07:56:35.703 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.657+0000 7f75757b3700 1 -- 172.21.15.59:0/409551634 learned_addr learned my addr 172.21.15.59:0/409551634 (peer_addr_for_me v2:172.21.15.59:0/0) 2023-11-24T07:56:35.703 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.657+0000 7f7574fb2700 1 --2- 172.21.15.59:0/409551634 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f75700c9210 0x7f7570158b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:56:35.704 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.657+0000 7f756ffff700 1 --2- 172.21.15.59:0/409551634 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f75700cda90 0x7f757013f200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:56:35.704 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.657+0000 7f75757b3700 1 -- 172.21.15.59:0/409551634 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f75700cda90 msgr2=0x7f757013f200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:56:35.704 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.657+0000 7f75757b3700 1 --2- 172.21.15.59:0/409551634 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f75700cda90 0x7f757013f200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:56:35.704 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.657+0000 7f75757b3700 1 -- 172.21.15.59:0/409551634 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f75700c9210 msgr2=0x7f7570158b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:56:35.704 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.657+0000 7f75757b3700 1 --2- 172.21.15.59:0/409551634 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f75700c9210 0x7f7570158b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:56:35.704 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.657+0000 7f75757b3700 1 -- 172.21.15.59:0/409551634 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7568009a70 con 0x7f75700ceff0 2023-11-24T07:56:35.704 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.658+0000 7f75757b3700 1 --2- 172.21.15.59:0/409551634 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f75700ceff0 0x7f75700720f0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f756400c420 tx=0x7f756400c730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:56:35.704 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.657+0000 7f6e9d01d700 1 --2- 172.21.15.59:0/3903790008 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6e980ca2c0 0x7f6e98159280 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f6e90001ed0 tx=0x7f6e9000a340 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:56:35.705 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.657+0000 7fbfc6ffd700 1 --2- 172.21.15.59:0/1812054286 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fbfc809d160 0x7fbfc80ca440 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fbfb800e7e0 tx=0x7fbfb800eaf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:56:35.705 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.657+0000 7fbfaffff700 1 -- 172.21.15.59:0/1812054286 <== mon.2 v2:172.21.15.76:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbfb8013610 con 0x7fbfc809d160 2023-11-24T07:56:35.705 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.657+0000 7fbfaffff700 1 -- 172.21.15.59:0/1812054286 <== mon.2 v2:172.21.15.76:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbfb8013770 con 0x7fbfc809d160 2023-11-24T07:56:35.705 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.657+0000 7fbfaffff700 1 -- 172.21.15.59:0/1812054286 <== mon.2 v2:172.21.15.76:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbfb8013b20 con 0x7fbfc809d160 2023-11-24T07:56:35.705 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.657+0000 7fbfcda41700 1 -- 172.21.15.59:0/1812054286 --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbfc80c85c0 con 0x7fbfc809d160 2023-11-24T07:56:35.705 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.657+0000 7fbfcda41700 1 -- 172.21.15.59:0/1812054286 --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbfc80c8b80 con 0x7fbfc809d160 2023-11-24T07:56:35.705 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.658+0000 7fadf54a8700 1 -- 172.21.15.59:0/1833136323 --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fadf01fc290 con 0x7fadf00c9210 2023-11-24T07:56:35.706 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.657+0000 7f32dffff700 1 -- 172.21.15.59:0/3739815000 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f32e81440f0 msgr2=0x7f32e815da40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:56:35.706 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.657+0000 7f32dffff700 1 --2- 172.21.15.59:0/3739815000 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f32e81440f0 0x7f32e815da40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:56:35.706 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.657+0000 7f32dffff700 1 -- 172.21.15.59:0/3739815000 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f32e80722c0 msgr2=0x7f32e8155250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:56:35.706 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.657+0000 7f32dffff700 1 --2- 172.21.15.59:0/3739815000 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f32e80722c0 0x7f32e8155250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:56:35.706 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.657+0000 7f32dffff700 1 -- 172.21.15.59:0/3739815000 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f32e0009b30 con 0x7f32e8143ae0 2023-11-24T07:56:35.706 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.657+0000 7f32dffff700 1 --2- 172.21.15.59:0/3739815000 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f32e8143ae0 0x7f32e8153d90 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f32d800e810 tx=0x7f32d800ebd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:56:35.706 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.657+0000 7f1cf9685700 1 -- 172.21.15.59:0/1910015220 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1cdc005330 msgr2=0x7f1cdc00ba10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:56:35.707 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.657+0000 7f1cf9685700 1 --2- 172.21.15.59:0/1910015220 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1cdc005330 0x7f1cdc00ba10 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f1cec006fd0 tx=0x7f1cdc005880 comp rx=0 tx=0).stop 2023-11-24T07:56:35.707 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.658+0000 7f1cf9685700 1 -- 172.21.15.59:0/1910015220 shutdown_connections 2023-11-24T07:56:35.707 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.658+0000 7f1cf9685700 1 --2- 172.21.15.59:0/1910015220 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1cdc005330 0x7f1cdc00ba10 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:56:35.707 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.658+0000 7f1cf9685700 1 -- 172.21.15.59:0/1910015220 >> 172.21.15.59:0/1910015220 conn(0x7f1cf40cf980 msgr2=0x7f1cf40cfd60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:56:35.707 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.658+0000 7f1cf9685700 1 -- 172.21.15.59:0/1910015220 shutdown_connections 2023-11-24T07:56:35.707 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.658+0000 7f756d7fa700 1 -- 172.21.15.59:0/409551634 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7564005d70 con 0x7f75700ceff0 2023-11-24T07:56:35.707 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.658+0000 7f756d7fa700 1 -- 172.21.15.59:0/409551634 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f756400ce30 con 0x7f75700ceff0 2023-11-24T07:56:35.707 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.658+0000 7f756d7fa700 1 -- 172.21.15.59:0/409551634 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75640072d0 con 0x7f75700ceff0 2023-11-24T07:56:35.708 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.658+0000 7f7577216700 1 -- 172.21.15.59:0/409551634 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f75701595b0 con 0x7f75700ceff0 2023-11-24T07:56:35.708 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.658+0000 7f7577216700 1 -- 172.21.15.59:0/409551634 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7570159b70 con 0x7f75700ceff0 2023-11-24T07:56:35.708 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.658+0000 7fadd77fe700 1 -- 172.21.15.59:0/1833136323 <== mon.2 v2:172.21.15.76:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fade001f070 con 0x7fadf00c9210 2023-11-24T07:56:35.708 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.658+0000 7fadd77fe700 1 -- 172.21.15.59:0/1833136323 <== mon.2 v2:172.21.15.76:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fade0003a40 con 0x7fadf00c9210 2023-11-24T07:56:35.708 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.658+0000 7f32dd7fa700 1 -- 172.21.15.59:0/3739815000 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32d8013610 con 0x7f32e8143ae0 2023-11-24T07:56:35.709 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.658+0000 7f32dd7fa700 1 -- 172.21.15.59:0/3739815000 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f32d8013770 con 0x7f32e8143ae0 2023-11-24T07:56:35.709 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.658+0000 7f32dd7fa700 1 -- 172.21.15.59:0/3739815000 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32d8013b00 con 0x7f32e8143ae0 2023-11-24T07:56:35.709 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.659+0000 7f32eee79700 1 -- 172.21.15.59:0/3739815000 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f32e81532b0 con 0x7f32e8143ae0 2023-11-24T07:56:35.709 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.659+0000 7f32eee79700 1 -- 172.21.15.59:0/3739815000 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f32e8153870 con 0x7f32e8143ae0 2023-11-24T07:56:35.709 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.660+0000 7f7577216700 1 -- 172.21.15.59:0/409551634 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f75500052e0 con 0x7f75700ceff0 2023-11-24T07:56:35.709 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.658+0000 7f1cf9685700 1 -- 172.21.15.59:0/1910015220 wait complete. 2023-11-24T07:56:35.710 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.660+0000 7f1cf9685700 1 Processor -- start 2023-11-24T07:56:35.710 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.660+0000 7f1cf9685700 1 -- start start 2023-11-24T07:56:35.710 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.660+0000 7fbfcda41700 1 -- 172.21.15.59:0/1812054286 --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbfc80c8da0 con 0x7fbfc809d160 2023-11-24T07:56:35.710 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.659+0000 7fadd77fe700 1 -- 172.21.15.59:0/1833136323 <== mon.2 v2:172.21.15.76:3301/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fade0003be0 con 0x7fadf00c9210 2023-11-24T07:56:35.710 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.660+0000 7fadd77fe700 1 --2- 172.21.15.59:0/1833136323 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7faddc06aed0 0x7faddc06d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:56:35.711 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.660+0000 7fadd77fe700 1 -- 172.21.15.59:0/1833136323 <== mon.2 v2:172.21.15.76:3301/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fade0021070 con 0x7fadf00c9210 2023-11-24T07:56:35.711 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.660+0000 7fadee7fc700 1 --2- 172.21.15.59:0/1833136323 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7faddc06aed0 0x7faddc06d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:56:35.711 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.660+0000 7f32eee79700 1 -- 172.21.15.59:0/3739815000 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f32e8068670 con 0x7f32e8143ae0 2023-11-24T07:56:35.711 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.661+0000 7fadee7fc700 1 --2- 172.21.15.59:0/1833136323 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7faddc06aed0 0x7faddc06d390 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fadf00b59e0 tx=0x7fade4009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:56:35.712 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.661+0000 7f1cf9685700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1cdc005330 0x7f1cf413b580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:56:35.712 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.661+0000 7f1cf9685700 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f1cf415bd40 0x7f1cf415ae70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:56:35.712 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.661+0000 7f1cf9685700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1cf414a9b0 0x7f1cf414ae30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:56:35.712 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.661+0000 7f1cf9685700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1cf414b370 con 0x7f1cf414a9b0 2023-11-24T07:56:35.712 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.661+0000 7f1cf9685700 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1cf413bac0 con 0x7f1cdc005330 2023-11-24T07:56:35.712 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.661+0000 7f1cf9685700 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1cf413bd40 con 0x7f1cf415bd40 2023-11-24T07:56:35.712 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.661+0000 7f1cf37fe700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1cf414a9b0 0x7f1cf414ae30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:56:35.712 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.661+0000 7f1cf37fe700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1cf414a9b0 0x7f1cf414ae30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:50136/0 (socket says 172.21.15.59:50136) 2023-11-24T07:56:35.712 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.661+0000 7f1cf37fe700 1 -- 172.21.15.59:0/3385067550 learned_addr learned my addr 172.21.15.59:0/3385067550 (peer_addr_for_me v2:172.21.15.59:0/0) 2023-11-24T07:56:35.712 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.661+0000 7f1cf27fc700 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f1cf415bd40 0x7f1cf415ae70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:56:35.712 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.662+0000 7f1cf37fe700 1 -- 172.21.15.59:0/3385067550 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f1cf415bd40 msgr2=0x7f1cf415ae70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:56:35.712 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.662+0000 7f1cf37fe700 1 --2- 172.21.15.59:0/3385067550 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f1cf415bd40 0x7f1cf415ae70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:56:35.713 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.662+0000 7f1cf37fe700 1 -- 172.21.15.59:0/3385067550 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1cdc005330 msgr2=0x7f1cf413b580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:56:35.713 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.662+0000 7f1cf37fe700 1 --2- 172.21.15.59:0/3385067550 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1cdc005330 0x7f1cf413b580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:56:35.713 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.662+0000 7f1cf37fe700 1 -- 172.21.15.59:0/3385067550 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1cf413c020 con 0x7f1cf414a9b0 2023-11-24T07:56:35.713 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.661+0000 7f35d8723700 1 Processor -- start 2023-11-24T07:56:35.713 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.662+0000 7fc4ddffb700 1 -- 172.21.15.59:0/768754559 <== mon.1 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fc4e0054590 con 0x7fc4e8071390 2023-11-24T07:56:35.714 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.661+0000 7f32dd7fa700 1 -- 172.21.15.59:0/3739815000 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f32d8013c60 con 0x7f32e8143ae0 2023-11-24T07:56:35.714 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.662+0000 7f32dd7fa700 1 --2- 172.21.15.59:0/3739815000 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f32d406af20 0x7f32d406d3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:56:35.714 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.662+0000 7f32dd7fa700 1 -- 172.21.15.59:0/3739815000 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f32d8089280 con 0x7f32e8143ae0 2023-11-24T07:56:35.714 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.662+0000 7f756d7fa700 1 -- 172.21.15.59:0/409551634 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f7564007430 con 0x7f75700ceff0 2023-11-24T07:56:35.714 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.662+0000 7f1cf37fe700 1 --2- 172.21.15.59:0/3385067550 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1cf414a9b0 0x7f1cf414ae30 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f1cf40b5c10 tx=0x7f1cec004790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:56:35.715 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.662+0000 7f756d7fa700 1 --2- 172.21.15.59:0/409551634 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f755c06afa0 0x7f755c06d460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:56:35.715 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.662+0000 7f756d7fa700 1 -- 172.21.15.59:0/409551634 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f7564088c20 con 0x7f75700ceff0 2023-11-24T07:56:35.715 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.663+0000 7f7574fb2700 1 --2- 172.21.15.59:0/409551634 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f755c06afa0 0x7f755c06d460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:56:35.715 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.663+0000 7f32ecc15700 1 --2- 172.21.15.59:0/3739815000 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f32d406af20 0x7f32d406d3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:56:35.716 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.663+0000 7f6e8e7fc700 1 -- 172.21.15.59:0/3903790008 <== mon.1 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e9000ade0 con 0x7f6e980ca2c0 2023-11-24T07:56:35.716 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.663+0000 7f7574fb2700 1 --2- 172.21.15.59:0/409551634 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f755c06afa0 0x7f755c06d460 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f7568000f80 tx=0x7f7568011000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:56:35.716 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.661+0000 7f35d8723700 1 -- start start 2023-11-24T07:56:35.716 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.663+0000 7f35d8723700 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f35d0071390 0x7f35d015e600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:56:35.717 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.663+0000 7f32ecc15700 1 --2- 172.21.15.59:0/3739815000 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f32d406af20 0x7f32d406d3e0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f32e4000f60 tx=0x7f32e4009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:56:35.717 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.663+0000 7f35d8723700 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f35d00ca2c0 0x7f35d00ca750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:56:35.717 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.663+0000 7f35d8723700 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f35d00ce000 con 0x7f35d015db10 2023-11-24T07:56:35.717 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.663+0000 7f35d8723700 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f35d00ce2c0 con 0x7f35d00ca2c0 2023-11-24T07:56:35.717 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.663+0000 7f35d8723700 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f35d00ce5a0 con 0x7f35d0071390 2023-11-24T07:56:35.717 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.663+0000 7f6e9fa82700 1 -- 172.21.15.59:0/3903790008 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6e981fc860 con 0x7f6e980ca2c0 2023-11-24T07:56:35.718 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.663+0000 7f35d5cbe700 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f35d0071390 0x7f35d015e600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:56:35.718 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.663+0000 7f35d5cbe700 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f35d0071390 0x7f35d015e600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3301/0 says I am v2:172.21.15.59:49584/0 (socket says 172.21.15.59:49584) 2023-11-24T07:56:35.718 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.663+0000 7f35d5cbe700 1 -- 172.21.15.59:0/894582538 learned_addr learned my addr 172.21.15.59:0/894582538 (peer_addr_for_me v2:172.21.15.59:0/0) 2023-11-24T07:56:35.718 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.663+0000 7f35d6cc0700 1 --2- 172.21.15.59:0/894582538 >> v2:172.21.15.76:3300/0 conn(0x7f35d00ca2c0 0x7f35d00ca750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:56:35.718 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.663+0000 7f6e9fa82700 1 -- 172.21.15.59:0/3903790008 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6e9815a9f0 con 0x7f6e980ca2c0 2023-11-24T07:56:35.719 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.663+0000 7f35d5cbe700 1 -- 172.21.15.59:0/894582538 >> v2:172.21.15.76:3300/0 conn(0x7f35d00ca2c0 msgr2=0x7f35d00ca750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T07:56:35.719 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.663+0000 7f35d5cbe700 1 --2- 172.21.15.59:0/894582538 >> v2:172.21.15.76:3300/0 conn(0x7f35d00ca2c0 0x7f35d00ca750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:56:35.719 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.663+0000 7f35d5cbe700 1 -- 172.21.15.59:0/894582538 >> v1:172.21.15.76:6789/0 conn(0x7f35d015db10 legacy=0x7f35d015def0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T07:56:35.719 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.663+0000 7f35d5cbe700 1 -- 172.21.15.59:0/894582538 --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f35d00ce880 con 0x7f35d0071390 2023-11-24T07:56:35.719 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.664+0000 7f35d5cbe700 1 --2- 172.21.15.59:0/894582538 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f35d0071390 0x7f35d015e600 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f35c00098f0 tx=0x7f35c0009c00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5b873eeaaa4bd56a server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:56:35.719 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.664+0000 7fadd77fe700 1 -- 172.21.15.59:0/1833136323 <== mon.2 v2:172.21.15.76:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fade00563a0 con 0x7fadf00c9210 2023-11-24T07:56:35.720 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.663+0000 7f6e8e7fc700 1 -- 172.21.15.59:0/3903790008 <== mon.1 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6e900020d0 con 0x7f6e980ca2c0 2023-11-24T07:56:35.720 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.663+0000 7f6e8e7fc700 1 -- 172.21.15.59:0/3903790008 <== mon.1 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e90003c00 con 0x7f6e980ca2c0 2023-11-24T07:56:35.720 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.664+0000 7f35d54bd700 1 -- 172.21.15.59:0/894582538 <== mon.2 v2:172.21.15.76:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35c0004230 con 0x7f35d0071390 2023-11-24T07:56:35.721 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.664+0000 7f35d54bd700 1 -- 172.21.15.59:0/894582538 <== mon.2 v2:172.21.15.76:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f35c0004390 con 0x7f35d0071390 2023-11-24T07:56:35.721 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.664+0000 7f35d54bd700 1 -- 172.21.15.59:0/894582538 <== mon.2 v2:172.21.15.76:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35c00036a0 con 0x7f35d0071390 2023-11-24T07:56:35.721 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.665+0000 7f35d8723700 1 -- 172.21.15.59:0/894582538 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f35d0071390 msgr2=0x7f35d015e600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:56:35.721 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.665+0000 7f35d8723700 1 --2- 172.21.15.59:0/894582538 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f35d0071390 0x7f35d015e600 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f35c00098f0 tx=0x7f35c0009c00 comp rx=0 tx=0).stop 2023-11-24T07:56:35.721 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.666+0000 7fbfaffff700 1 -- 172.21.15.59:0/1812054286 <== mon.2 v2:172.21.15.76:3301/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fbfb8013c80 con 0x7fbfc809d160 2023-11-24T07:56:35.721 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.664+0000 7f6e9fa82700 1 -- 172.21.15.59:0/3903790008 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6e800052e0 con 0x7f6e980ca2c0 2023-11-24T07:56:35.722 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.666+0000 7f35d8723700 1 -- 172.21.15.59:0/894582538 shutdown_connections 2023-11-24T07:56:35.722 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.666+0000 7f35d8723700 1 --2- 172.21.15.59:0/894582538 >> v2:172.21.15.76:3300/0 conn(0x7f35d00ca2c0 0x7f35d00ca750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:56:35.722 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.666+0000 7f35d8723700 1 --2- 172.21.15.59:0/894582538 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f35d0071390 0x7f35d015e600 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:56:35.722 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.666+0000 7f35d8723700 1 -- 172.21.15.59:0/894582538 >> 172.21.15.59:0/894582538 conn(0x7f35d00cf980 msgr2=0x7f35d00cfd90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:56:35.722 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.666+0000 7f35d8723700 1 -- 172.21.15.59:0/894582538 shutdown_connections 2023-11-24T07:56:35.722 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.666+0000 7f35d8723700 1 -- 172.21.15.59:0/894582538 wait complete. 2023-11-24T07:56:35.723 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.666+0000 7fbfaffff700 1 --2- 172.21.15.59:0/1812054286 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7fbfb406aed0 0x7fbfb406d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:56:35.723 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.667+0000 7f6e8e7fc700 1 -- 172.21.15.59:0/3903790008 <== mon.1 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f6e900050d0 con 0x7f6e980ca2c0 2023-11-24T07:56:35.723 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.667+0000 7f6e8e7fc700 1 --2- 172.21.15.59:0/3903790008 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f6e7c06afa0 0x7f6e7c06d460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:56:35.723 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.667+0000 7f6e8e7fc700 1 -- 172.21.15.59:0/3903790008 <== mon.1 v2:172.21.15.76:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f6e90089710 con 0x7f6e980ca2c0 2023-11-24T07:56:35.724 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.668+0000 7f6e9d81e700 1 --2- 172.21.15.59:0/3903790008 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f6e7c06afa0 0x7f6e7c06d460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:56:35.724 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.666+0000 7fbfaffff700 1 -- 172.21.15.59:0/1812054286 <== mon.2 v2:172.21.15.76:3301/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fbfb8089650 con 0x7fbfc809d160 2023-11-24T07:56:35.724 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.666+0000 7fbfc67fc700 1 --2- 172.21.15.59:0/1812054286 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7fbfb406aed0 0x7fbfb406d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:56:35.724 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.667+0000 7fbfc67fc700 1 --2- 172.21.15.59:0/1812054286 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7fbfb406aed0 0x7fbfb406d390 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fbfc806d110 tx=0x7fbfbc013040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:56:35.724 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.668+0000 7f35d8723700 1 Processor -- start 2023-11-24T07:56:35.725 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.668+0000 7f6e9d81e700 1 --2- 172.21.15.59:0/3903790008 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f6e7c06afa0 0x7f6e7c06d460 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f6e9400d370 tx=0x7f6e94012000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:56:35.726 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.668+0000 7f35d8723700 1 -- start start 2023-11-24T07:56:35.726 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.669+0000 7f35d8723700 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f35d0071390 0x7f35d013b5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:56:35.726 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.669+0000 7f35d8723700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f35d00ca2c0 0x7f35d01e5740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:56:35.726 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.669+0000 7f35d8723700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f35d015db10 0x7f35d00b09e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:56:35.726 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.669+0000 7f35d8723700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35d013bb30 con 0x7f35d00ca2c0 2023-11-24T07:56:35.726 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.669+0000 7f35d8723700 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35d01e2d90 con 0x7f35d015db10 2023-11-24T07:56:35.727 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.669+0000 7f35d8723700 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f35d01e5c80 con 0x7f35d0071390 2023-11-24T07:56:35.727 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.669+0000 7f35d64bf700 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f35d0071390 0x7f35d013b5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:56:35.727 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.669+0000 7f35d64bf700 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f35d0071390 0x7f35d013b5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3301/0 says I am v2:172.21.15.59:49592/0 (socket says 172.21.15.59:49592) 2023-11-24T07:56:35.727 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.669+0000 7f35d64bf700 1 -- 172.21.15.59:0/3363610734 learned_addr learned my addr 172.21.15.59:0/3363610734 (peer_addr_for_me v2:172.21.15.59:0/0) 2023-11-24T07:56:35.727 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.669+0000 7f35d6cc0700 1 --2- 172.21.15.59:0/3363610734 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f35d015db10 0x7f35d00b09e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:56:35.727 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.669+0000 7f35d64bf700 1 -- 172.21.15.59:0/3363610734 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f35d015db10 msgr2=0x7f35d00b09e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:56:35.727 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.669+0000 7f35d64bf700 1 --2- 172.21.15.59:0/3363610734 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f35d015db10 0x7f35d00b09e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:56:35.727 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.669+0000 7f35d64bf700 1 -- 172.21.15.59:0/3363610734 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f35d00ca2c0 msgr2=0x7f35d01e5740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:56:35.727 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.669+0000 7f35d64bf700 1 --2- 172.21.15.59:0/3363610734 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f35d00ca2c0 0x7f35d01e5740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:56:35.727 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.669+0000 7f35d64bf700 1 -- 172.21.15.59:0/3363610734 --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f35c0009550 con 0x7f35d0071390 2023-11-24T07:56:35.727 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.670+0000 7f35d64bf700 1 --2- 172.21.15.59:0/3363610734 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f35d0071390 0x7f35d013b5f0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f35c800ba80 tx=0x7f35c800bd90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:56:35.728 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.670+0000 7fbfaffff700 1 -- 172.21.15.59:0/1812054286 <== mon.2 v2:172.21.15.76:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fbfb8054790 con 0x7fbfc809d160 2023-11-24T07:56:35.728 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.670+0000 7f35c6ffd700 1 -- 172.21.15.59:0/3363610734 <== mon.2 v2:172.21.15.76:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35c800c9e0 con 0x7f35d0071390 2023-11-24T07:56:35.729 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.670+0000 7f35c6ffd700 1 -- 172.21.15.59:0/3363610734 <== mon.2 v2:172.21.15.76:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f35c800cb40 con 0x7f35d0071390 2023-11-24T07:56:35.729 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.670+0000 7f35d8723700 1 -- 172.21.15.59:0/3363610734 --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f35d014e110 con 0x7f35d0071390 2023-11-24T07:56:35.729 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.670+0000 7f35c6ffd700 1 -- 172.21.15.59:0/3363610734 <== mon.2 v2:172.21.15.76:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35c8014440 con 0x7f35d0071390 2023-11-24T07:56:35.729 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.670+0000 7f35d8723700 1 -- 172.21.15.59:0/3363610734 --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f35d01e32c0 con 0x7f35d0071390 2023-11-24T07:56:35.729 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.671+0000 7f35d8723700 1 -- 172.21.15.59:0/3363610734 --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f35d00b4410 con 0x7f35d0071390 2023-11-24T07:56:35.729 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.672+0000 7f35c6ffd700 1 -- 172.21.15.59:0/3363610734 <== mon.2 v2:172.21.15.76:3301/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f35c8006d50 con 0x7f35d0071390 2023-11-24T07:56:35.729 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.672+0000 7f6e8e7fc700 1 -- 172.21.15.59:0/3903790008 <== mon.1 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f6e900547a0 con 0x7f6e980ca2c0 2023-11-24T07:56:35.730 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.672+0000 7f35c6ffd700 1 --2- 172.21.15.59:0/3363610734 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f35b406aed0 0x7f35b406d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:56:35.730 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.672+0000 7f35c6ffd700 1 -- 172.21.15.59:0/3363610734 <== mon.2 v2:172.21.15.76:3301/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f35c8089120 con 0x7f35d0071390 2023-11-24T07:56:35.730 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.672+0000 7f35d5cbe700 1 --2- 172.21.15.59:0/3363610734 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f35b406aed0 0x7f35b406d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:56:35.730 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.673+0000 7f756d7fa700 1 -- 172.21.15.59:0/409551634 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f7564053df0 con 0x7f75700ceff0 2023-11-24T07:56:35.731 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.673+0000 7f35d5cbe700 1 --2- 172.21.15.59:0/3363610734 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f35b406aed0 0x7f35b406d390 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f35d0159cb0 tx=0x7f35c0010000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:56:35.731 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.674+0000 7f32dd7fa700 1 -- 172.21.15.59:0/3739815000 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f32d8054330 con 0x7f32e8143ae0 2023-11-24T07:56:35.731 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.674+0000 7f1cdb7fe700 1 -- 172.21.15.59:0/3385067550 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1cec015040 con 0x7f1cf414a9b0 2023-11-24T07:56:35.731 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.674+0000 7f1cdb7fe700 1 -- 172.21.15.59:0/3385067550 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1cec01d450 con 0x7f1cf414a9b0 2023-11-24T07:56:35.732 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.674+0000 7f1cdb7fe700 1 -- 172.21.15.59:0/3385067550 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1cec01d800 con 0x7f1cf414a9b0 2023-11-24T07:56:35.732 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.674+0000 7f1cf9685700 1 -- 172.21.15.59:0/3385067550 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1cf413c300 con 0x7f1cf414a9b0 2023-11-24T07:56:35.732 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.674+0000 7f1cf9685700 1 -- 172.21.15.59:0/3385067550 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1cf413c8c0 con 0x7f1cf414a9b0 2023-11-24T07:56:35.732 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.675+0000 7f1cf9685700 1 -- 172.21.15.59:0/3385067550 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1cf413d980 con 0x7f1cf414a9b0 2023-11-24T07:56:35.732 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.677+0000 7f35c6ffd700 1 -- 172.21.15.59:0/3363610734 <== mon.2 v2:172.21.15.76:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f35c80541e0 con 0x7f35d0071390 2023-11-24T07:56:35.733 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.677+0000 7f1cdb7fe700 1 -- 172.21.15.59:0/3385067550 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f1cec01d960 con 0x7f1cf414a9b0 2023-11-24T07:56:35.733 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.678+0000 7f1cdb7fe700 1 --2- 172.21.15.59:0/3385067550 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f1ce006af20 0x7f1ce006d3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:56:35.733 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.678+0000 7f1cdb7fe700 1 -- 172.21.15.59:0/3385067550 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f1cec089760 con 0x7f1cf414a9b0 2023-11-24T07:56:35.733 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.678+0000 7f1cf2ffd700 1 --2- 172.21.15.59:0/3385067550 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f1ce006af20 0x7f1ce006d3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:56:35.733 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.678+0000 7f1cf2ffd700 1 --2- 172.21.15.59:0/3385067550 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f1ce006af20 0x7f1ce006d3e0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f1cf4069c90 tx=0x7f1ce401a040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:56:35.733 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:35.681+0000 7f1cdb7fe700 1 -- 172.21.15.59:0/3385067550 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f1cec054af0 con 0x7f1cf414a9b0 2023-11-24T07:56:36.086 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:36.085+0000 7fbfcda41700 1 -- 172.21.15.59:0/1812054286 --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7fbfc80c8420 con 0x7fbfc809d160 2023-11-24T07:56:36.087 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:36.085+0000 7fadf54a8700 1 -- 172.21.15.59:0/1833136323 --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) v1 -- 0x7fadf01fc290 con 0x7fadf00c9210 2023-11-24T07:56:36.087 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:36.085+0000 7fadd77fe700 1 -- 172.21.15.59:0/1833136323 <== mon.2 v2:172.21.15.76:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fade0015020 con 0x7fadf00c9210 2023-11-24T07:56:36.090 INFO:teuthology.orchestra.run.smithi059.stdout:55834574854 2023-11-24T07:56:36.090 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:36.086+0000 7fc4ef6a4700 1 -- 172.21.15.59:0/768754559 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) v1 -- 0x7fc4e80ca0a0 con 0x7fc4e8071390 2023-11-24T07:56:36.090 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:36.087+0000 7fc4ddffb700 1 -- 172.21.15.59:0/768754559 <== mon.1 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fc4e0057bb0 con 0x7fc4e8071390 2023-11-24T07:56:36.091 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:36.086+0000 7fbfaffff700 1 -- 172.21.15.59:0/1812054286 <== mon.2 v2:172.21.15.76:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fbfc80c8420 con 0x7fbfc809d160 2023-11-24T07:56:36.091 INFO:teuthology.orchestra.run.smithi059.stdout:55834574854 2023-11-24T07:56:36.091 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:36.089+0000 7f7577216700 1 -- 172.21.15.59:0/409551634 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f7550005860 con 0x7f75700ceff0 2023-11-24T07:56:36.091 INFO:teuthology.orchestra.run.smithi059.stdout:55834574855 2023-11-24T07:56:36.093 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:36.091+0000 7fbfcda41700 1 -- 172.21.15.59:0/1812054286 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7fbfb406aed0 msgr2=0x7fbfb406d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:56:36.093 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:36.091+0000 7fbfcda41700 1 --2- 172.21.15.59:0/1812054286 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7fbfb406aed0 0x7fbfb406d390 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fbfc806d110 tx=0x7fbfbc013040 comp rx=0 tx=0).stop 2023-11-24T07:56:36.093 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:36.091+0000 7fbfcda41700 1 -- 172.21.15.59:0/1812054286 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fbfc809d160 msgr2=0x7fbfc80ca440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:56:36.093 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:36.091+0000 7fbfcda41700 1 --2- 172.21.15.59:0/1812054286 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fbfc809d160 0x7fbfc80ca440 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fbfb800e7e0 tx=0x7fbfb800eaf0 comp rx=0 tx=0).stop 2023-11-24T07:56:36.093 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:36.091+0000 7f35d8723700 1 -- 172.21.15.59:0/3363610734 --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) v1 -- 0x7f35d0067f60 con 0x7f35d0071390 2023-11-24T07:56:36.093 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:36.091+0000 7f35c6ffd700 1 -- 172.21.15.59:0/3363610734 <== mon.2 v2:172.21.15.76:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f35c8057800 con 0x7f35d0071390 2023-11-24T07:56:36.094 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:36.092+0000 7fbfcda41700 1 -- 172.21.15.59:0/1812054286 shutdown_connections 2023-11-24T07:56:36.094 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:36.092+0000 7fbfcda41700 1 --2- 172.21.15.59:0/1812054286 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7fbfb406aed0 0x7fbfb406d390 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:56:36.094 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:36.092+0000 7fbfcda41700 1 --2- 172.21.15.59:0/1812054286 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fbfc80b7050 0x7fbfc81571a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:56:36.094 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:36.092+0000 7fbfcda41700 1 --2- 172.21.15.59:0/1812054286 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fbfc809d160 0x7fbfc80ca440 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:56:36.094 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:36.092+0000 7fbfcda41700 1 --2- 172.21.15.59:0/1812054286 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbfc80b5e00 0x7fbfc80c7890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:56:36.095 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:36.092+0000 7fbfcda41700 1 -- 172.21.15.59:0/1812054286 >> 172.21.15.59:0/1812054286 conn(0x7fbfc80d08c0 msgr2=0x7fbfc80d20d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:56:36.095 INFO:teuthology.orchestra.run.smithi059.stdout:55834574853 2023-11-24T07:56:36.095 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:36.092+0000 7fadf54a8700 1 -- 172.21.15.59:0/1833136323 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7faddc06aed0 msgr2=0x7faddc06d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:56:36.096 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:36.092+0000 7fadf54a8700 1 --2- 172.21.15.59:0/1833136323 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7faddc06aed0 0x7faddc06d390 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fadf00b59e0 tx=0x7fade4009040 comp rx=0 tx=0).stop 2023-11-24T07:56:36.096 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:36.092+0000 7fadf54a8700 1 -- 172.21.15.59:0/1833136323 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fadf00c9210 msgr2=0x7fadf0159bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:56:36.096 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:36.092+0000 7fadf54a8700 1 --2- 172.21.15.59:0/1833136323 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fadf00c9210 0x7fadf0159bf0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fade0009fd0 tx=0x7fade0004750 comp rx=0 tx=0).stop 2023-11-24T07:56:36.096 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:36.093+0000 7f6e9fa82700 1 -- 172.21.15.59:0/3903790008 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f6e80005590 con 0x7f6e980ca2c0 2023-11-24T07:56:36.097 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:36.094+0000 7fadf54a8700 1 -- 172.21.15.59:0/1833136323 shutdown_connections 2023-11-24T07:56:36.097 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:36.094+0000 7fadf54a8700 1 --2- 172.21.15.59:0/1833136323 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7faddc06aed0 0x7faddc06d390 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:56:36.097 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:36.094+0000 7fadf54a8700 1 --2- 172.21.15.59:0/1833136323 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fadf0144e30 0x7fadf013bfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:56:36.097 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:36.094+0000 7fadf54a8700 1 --2- 172.21.15.59:0/1833136323 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fadf00c9210 0x7fadf0159bf0 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:56:36.097 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:36.094+0000 7fadf54a8700 1 --2- 172.21.15.59:0/1833136323 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fadf00cdc60 0x7fadf0157720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:56:36.097 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:36.094+0000 7fadf54a8700 1 -- 172.21.15.59:0/1833136323 >> 172.21.15.59:0/1833136323 conn(0x7fadf00cf980 msgr2=0x7fadf0146150 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:56:36.097 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:36.094+0000 7fc4ef6a4700 1 -- 172.21.15.59:0/768754559 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7fc4cc06afa0 msgr2=0x7fc4cc06d460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:56:36.098 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:36.094+0000 7fc4ef6a4700 1 --2- 172.21.15.59:0/768754559 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7fc4cc06afa0 0x7fc4cc06d460 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fc4d8008c10 tx=0x7fc4d8013000 comp rx=0 tx=0).stop 2023-11-24T07:56:36.098 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:36.094+0000 7fc4ef6a4700 1 -- 172.21.15.59:0/768754559 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc4e8071390 msgr2=0x7fc4e8159780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:56:36.098 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:36.094+0000 7fc4ef6a4700 1 --2- 172.21.15.59:0/768754559 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc4e8071390 0x7fc4e8159780 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fc4e000bd70 tx=0x7fc4e000bda0 comp rx=0 tx=0).stop 2023-11-24T07:56:36.098 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:36.094+0000 7f756d7fa700 1 -- 172.21.15.59:0/409551634 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f7564021090 con 0x7f75700ceff0 2023-11-24T07:56:36.099 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:36.094+0000 7fc4ef6a4700 1 -- 172.21.15.59:0/768754559 shutdown_connections 2023-11-24T07:56:36.099 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:36.094+0000 7fc4ef6a4700 1 --2- 172.21.15.59:0/768754559 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7fc4cc06afa0 0x7fc4cc06d460 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:56:36.099 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:36.094+0000 7fc4ef6a4700 1 --2- 172.21.15.59:0/768754559 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc4e815db10 0x7fc4e81fb800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:56:36.099 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:36.094+0000 7fc4ef6a4700 1 --2- 172.21.15.59:0/768754559 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc4e8071390 0x7fc4e8159780 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:56:36.099 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:36.094+0000 7fc4ef6a4700 1 --2- 172.21.15.59:0/768754559 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fc4e815a520 0x7fc4e81f79d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:56:36.100 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:36.094+0000 7fc4ef6a4700 1 -- 172.21.15.59:0/768754559 >> 172.21.15.59:0/768754559 conn(0x7fc4e80cf980 msgr2=0x7fc4e8141ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:56:36.100 INFO:teuthology.orchestra.run.smithi059.stdout:55834574855 2023-11-24T07:56:36.101 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:36.094+0000 7fc4ef6a4700 1 -- 172.21.15.59:0/768754559 shutdown_connections 2023-11-24T07:56:36.101 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:36.094+0000 7f35d8723700 1 -- 172.21.15.59:0/3363610734 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f35b406aed0 msgr2=0x7f35b406d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:56:36.101 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:36.094+0000 7f35d8723700 1 --2- 172.21.15.59:0/3363610734 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f35b406aed0 0x7f35b406d390 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f35d0159cb0 tx=0x7f35c0010000 comp rx=0 tx=0).stop 2023-11-24T07:56:36.102 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:36.094+0000 7f35d8723700 1 -- 172.21.15.59:0/3363610734 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f35d0071390 msgr2=0x7f35d013b5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:56:36.102 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:36.094+0000 7f35d8723700 1 --2- 172.21.15.59:0/3363610734 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f35d0071390 0x7f35d013b5f0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f35c800ba80 tx=0x7f35c800bd90 comp rx=0 tx=0).stop 2023-11-24T07:56:36.102 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:36.094+0000 7fc4ef6a4700 1 -- 172.21.15.59:0/768754559 wait complete. 2023-11-24T07:56:36.103 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:36.094+0000 7fadf54a8700 1 -- 172.21.15.59:0/1833136323 shutdown_connections 2023-11-24T07:56:36.103 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:36.094+0000 7fadf54a8700 1 -- 172.21.15.59:0/1833136323 wait complete. 2023-11-24T07:56:36.103 INFO:teuthology.orchestra.run.smithi059.stdout:55834574855 2023-11-24T07:56:36.104 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:36.094+0000 7f6e8e7fc700 1 -- 172.21.15.59:0/3903790008 <== mon.1 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f6e90057dc0 con 0x7f6e980ca2c0 2023-11-24T07:56:36.104 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:36.094+0000 7f35d8723700 1 -- 172.21.15.59:0/3363610734 shutdown_connections 2023-11-24T07:56:36.105 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:36.094+0000 7f35d8723700 1 --2- 172.21.15.59:0/3363610734 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f35b406aed0 0x7f35b406d390 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:56:36.105 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:36.094+0000 7f35d8723700 1 --2- 172.21.15.59:0/3363610734 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f35d015db10 0x7f35d00b09e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:56:36.105 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:36.094+0000 7f35d8723700 1 --2- 172.21.15.59:0/3363610734 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f35d0071390 0x7f35d013b5f0 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:56:36.105 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:36.094+0000 7f35d8723700 1 --2- 172.21.15.59:0/3363610734 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f35d00ca2c0 0x7f35d01e5740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:56:36.105 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:36.094+0000 7f35d8723700 1 -- 172.21.15.59:0/3363610734 >> 172.21.15.59:0/3363610734 conn(0x7f35d00cf980 msgr2=0x7f35d00c8cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:56:36.106 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:36.095+0000 7fbfcda41700 1 -- 172.21.15.59:0/1812054286 shutdown_connections 2023-11-24T07:56:36.107 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:36.095+0000 7f35d8723700 1 -- 172.21.15.59:0/3363610734 shutdown_connections 2023-11-24T07:56:36.107 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:36.095+0000 7f35d8723700 1 -- 172.21.15.59:0/3363610734 wait complete. 2023-11-24T07:56:36.107 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:36.096+0000 7fbfcda41700 1 -- 172.21.15.59:0/1812054286 wait complete. 2023-11-24T07:56:36.108 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:36.097+0000 7f32eee79700 1 -- 172.21.15.59:0/3739815000 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f32e813fc90 con 0x7f32e8143ae0 2023-11-24T07:56:36.108 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:36.097+0000 7f7577216700 1 -- 172.21.15.59:0/409551634 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f755c06afa0 msgr2=0x7f755c06d460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:56:36.109 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:36.097+0000 7f7577216700 1 --2- 172.21.15.59:0/409551634 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f755c06afa0 0x7f755c06d460 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f7568000f80 tx=0x7f7568011000 comp rx=0 tx=0).stop 2023-11-24T07:56:36.109 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:36.097+0000 7f7577216700 1 -- 172.21.15.59:0/409551634 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f75700ceff0 msgr2=0x7f75700720f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:56:36.109 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:36.097+0000 7f7577216700 1 --2- 172.21.15.59:0/409551634 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f75700ceff0 0x7f75700720f0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f756400c420 tx=0x7f756400c730 comp rx=0 tx=0).stop 2023-11-24T07:56:36.109 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:36.098+0000 7f32dd7fa700 1 -- 172.21.15.59:0/3739815000 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f32d8057950 con 0x7f32e8143ae0 2023-11-24T07:56:36.110 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:36.098+0000 7f7577216700 1 -- 172.21.15.59:0/409551634 shutdown_connections 2023-11-24T07:56:36.110 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:36.098+0000 7f7577216700 1 --2- 172.21.15.59:0/409551634 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f755c06afa0 0x7f755c06d460 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:56:36.110 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:36.098+0000 7f7577216700 1 --2- 172.21.15.59:0/409551634 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f75700ceff0 0x7f75700720f0 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:56:36.110 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:36.098+0000 7f7577216700 1 --2- 172.21.15.59:0/409551634 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f75700c9210 0x7f7570158b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:56:36.110 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:36.098+0000 7f7577216700 1 --2- 172.21.15.59:0/409551634 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f75700cda90 0x7f757013f200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:56:36.110 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:36.098+0000 7f7577216700 1 -- 172.21.15.59:0/409551634 >> 172.21.15.59:0/409551634 conn(0x7f75700cf980 msgr2=0x7f757015c0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:56:36.110 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:36.098+0000 7f7577216700 1 -- 172.21.15.59:0/409551634 shutdown_connections 2023-11-24T07:56:36.111 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:36.098+0000 7f7577216700 1 -- 172.21.15.59:0/409551634 wait complete. 2023-11-24T07:56:36.111 INFO:teuthology.orchestra.run.smithi059.stdout:55834574855 2023-11-24T07:56:36.111 INFO:teuthology.orchestra.run.smithi059.stdout:55834574855 2023-11-24T07:56:36.111 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:36.098+0000 7f1cf9685700 1 -- 172.21.15.59:0/3385067550 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) v1 -- 0x7f1cf413cc30 con 0x7f1cf414a9b0 2023-11-24T07:56:36.112 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:36.098+0000 7f1cdb7fe700 1 -- 172.21.15.59:0/3385067550 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f1cec058110 con 0x7f1cf414a9b0 2023-11-24T07:56:36.112 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:36.101+0000 7f1cf9685700 1 -- 172.21.15.59:0/3385067550 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f1ce006af20 msgr2=0x7f1ce006d3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:56:36.112 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:36.101+0000 7f1cf9685700 1 --2- 172.21.15.59:0/3385067550 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f1ce006af20 0x7f1ce006d3e0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f1cf4069c90 tx=0x7f1ce401a040 comp rx=0 tx=0).stop 2023-11-24T07:56:36.112 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:36.101+0000 7f1cf9685700 1 -- 172.21.15.59:0/3385067550 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1cf414a9b0 msgr2=0x7f1cf414ae30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:56:36.112 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:36.101+0000 7f1cf9685700 1 --2- 172.21.15.59:0/3385067550 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1cf414a9b0 0x7f1cf414ae30 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f1cf40b5c10 tx=0x7f1cec004790 comp rx=0 tx=0).stop 2023-11-24T07:56:36.112 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:36.102+0000 7f32eee79700 1 -- 172.21.15.59:0/3739815000 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f32d406af20 msgr2=0x7f32d406d3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:56:36.113 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:36.102+0000 7f32eee79700 1 --2- 172.21.15.59:0/3739815000 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f32d406af20 0x7f32d406d3e0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f32e4000f60 tx=0x7f32e4009040 comp rx=0 tx=0).stop 2023-11-24T07:56:36.113 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:36.102+0000 7f32eee79700 1 -- 172.21.15.59:0/3739815000 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f32e8143ae0 msgr2=0x7f32e8153d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:56:36.113 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:36.102+0000 7f32eee79700 1 --2- 172.21.15.59:0/3739815000 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f32e8143ae0 0x7f32e8153d90 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f32d800e810 tx=0x7f32d800ebd0 comp rx=0 tx=0).stop 2023-11-24T07:56:36.113 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:36.102+0000 7f32eee79700 1 -- 172.21.15.59:0/3739815000 shutdown_connections 2023-11-24T07:56:36.113 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:36.102+0000 7f32eee79700 1 --2- 172.21.15.59:0/3739815000 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f32d406af20 0x7f32d406d3e0 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:56:36.113 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:36.102+0000 7f32eee79700 1 --2- 172.21.15.59:0/3739815000 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f32e81440f0 0x7f32e815da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:56:36.114 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:36.102+0000 7f32eee79700 1 --2- 172.21.15.59:0/3739815000 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f32e80722c0 0x7f32e8155250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:56:36.114 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:36.102+0000 7f32eee79700 1 --2- 172.21.15.59:0/3739815000 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f32e8143ae0 0x7f32e8153d90 secure :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f32d800e810 tx=0x7f32d800ebd0 comp rx=0 tx=0).stop 2023-11-24T07:56:36.114 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:36.102+0000 7f32eee79700 1 -- 172.21.15.59:0/3739815000 >> 172.21.15.59:0/3739815000 conn(0x7f32e80b2740 msgr2=0x7f32e80b51b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:56:36.114 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:36.104+0000 7f6e9fa82700 1 -- 172.21.15.59:0/3903790008 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f6e7c06afa0 msgr2=0x7f6e7c06d460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:56:36.114 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:36.104+0000 7f6e9fa82700 1 --2- 172.21.15.59:0/3903790008 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f6e7c06afa0 0x7f6e7c06d460 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f6e9400d370 tx=0x7f6e94012000 comp rx=0 tx=0).stop 2023-11-24T07:56:36.114 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:36.104+0000 7f6e9fa82700 1 -- 172.21.15.59:0/3903790008 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6e980ca2c0 msgr2=0x7f6e98159280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:56:36.114 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:36.104+0000 7f6e9fa82700 1 --2- 172.21.15.59:0/3903790008 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6e980ca2c0 0x7f6e98159280 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f6e90001ed0 tx=0x7f6e9000a340 comp rx=0 tx=0).stop 2023-11-24T07:56:36.115 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:36.105+0000 7f32eee79700 1 -- 172.21.15.59:0/3739815000 shutdown_connections 2023-11-24T07:56:36.115 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:36.105+0000 7f32eee79700 1 -- 172.21.15.59:0/3739815000 wait complete. 2023-11-24T07:56:36.115 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:36.107+0000 7f1cf9685700 1 -- 172.21.15.59:0/3385067550 shutdown_connections 2023-11-24T07:56:36.115 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:36.107+0000 7f1cf9685700 1 --2- 172.21.15.59:0/3385067550 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f1ce006af20 0x7f1ce006d3e0 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:56:36.115 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:36.107+0000 7f1cf9685700 1 --2- 172.21.15.59:0/3385067550 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1cf414a9b0 0x7f1cf414ae30 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:56:36.115 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:36.107+0000 7f1cf9685700 1 --2- 172.21.15.59:0/3385067550 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1cdc005330 0x7f1cf413b580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:56:36.116 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:36.107+0000 7f1cf9685700 1 --2- 172.21.15.59:0/3385067550 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f1cf415bd40 0x7f1cf415ae70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:56:36.116 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:36.107+0000 7f1cf9685700 1 -- 172.21.15.59:0/3385067550 >> 172.21.15.59:0/3385067550 conn(0x7f1cf40cf980 msgr2=0x7f1cf40cd4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:56:36.116 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:36.108+0000 7f6e9fa82700 1 -- 172.21.15.59:0/3903790008 shutdown_connections 2023-11-24T07:56:36.116 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:36.108+0000 7f6e9fa82700 1 --2- 172.21.15.59:0/3903790008 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f6e7c06afa0 0x7f6e7c06d460 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:56:36.116 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:36.108+0000 7f6e9fa82700 1 --2- 172.21.15.59:0/3903790008 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6e9815db10 0x7f6e981597c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:56:36.116 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:36.108+0000 7f6e9fa82700 1 --2- 172.21.15.59:0/3903790008 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f6e98071390 0x7f6e9806cf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:56:36.116 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:36.108+0000 7f6e9fa82700 1 --2- 172.21.15.59:0/3903790008 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6e980ca2c0 0x7f6e98159280 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:56:36.116 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:36.108+0000 7f6e9fa82700 1 -- 172.21.15.59:0/3903790008 >> 172.21.15.59:0/3903790008 conn(0x7f6e980cf980 msgr2=0x7f6e980cfd60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:56:36.117 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:36.112+0000 7f1cf9685700 1 -- 172.21.15.59:0/3385067550 shutdown_connections 2023-11-24T07:56:36.117 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:36.112+0000 7f1cf9685700 1 -- 172.21.15.59:0/3385067550 wait complete. 2023-11-24T07:56:36.118 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:36.115+0000 7f6e9fa82700 1 -- 172.21.15.59:0/3903790008 shutdown_connections 2023-11-24T07:56:36.118 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:36.115+0000 7f6e9fa82700 1 -- 172.21.15.59:0/3903790008 wait complete. 2023-11-24T07:56:36.119 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574855 got 55834574854 for osd.4 2023-11-24T07:56:36.126 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574854 got 55834574853 for osd.7 2023-11-24T07:56:36.128 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574856 got 55834574855 for osd.1 2023-11-24T07:56:36.129 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574855 got 55834574854 for osd.6 2023-11-24T07:56:36.129 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574856 got 55834574855 for osd.0 2023-11-24T07:56:36.132 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574856 got 55834574855 for osd.2 2023-11-24T07:56:36.136 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574856 got 55834574855 for osd.5 2023-11-24T07:56:36.136 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574856 got 55834574855 for osd.3 2023-11-24T07:56:37.120 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2023-11-24T07:56:37.126 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2023-11-24T07:56:37.129 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-11-24T07:56:37.129 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2023-11-24T07:56:37.130 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-11-24T07:56:37.132 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-11-24T07:56:37.137 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2023-11-24T07:56:37.138 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-11-24T07:56:37.325 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:37.323+0000 7fb9a33c8700 1 Processor -- start 2023-11-24T07:56:37.327 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:37.324+0000 7fb9a33c8700 1 -- start start 2023-11-24T07:56:37.327 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:37.325+0000 7fb9a33c8700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fb99c071390 0x7fb99c15e600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:56:37.328 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:37.325+0000 7fb9a33c8700 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb99c0ce000 con 0x7fb99c0ca2c0 2023-11-24T07:56:37.328 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:37.325+0000 7fb9a33c8700 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb99c0ce2c0 con 0x7fb99c15db10 2023-11-24T07:56:37.328 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:37.325+0000 7fb9a33c8700 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb99c0ce5a0 con 0x7fb99c071390 2023-11-24T07:56:37.329 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:37.326+0000 7fb9a1965700 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7fb99c0ca2c0 0x7fb99c0ca6d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:53638/0 (socket says 172.21.15.59:53638) 2023-11-24T07:56:37.329 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:37.326+0000 7fb9a0963700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fb99c071390 0x7fb99c15e600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:56:37.329 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:37.326+0000 7fb9a1965700 1 -- 172.21.15.59:0/2606904936 learned_addr learned my addr 172.21.15.59:0/2606904936 (peer_addr_for_me v1:172.21.15.59:0/0) 2023-11-24T07:56:37.329 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:37.326+0000 7fb9a1164700 1 --1- 172.21.15.59:0/2606904936 >> v1:172.21.15.76:6789/0 conn(0x7fb99c15db10 0x7fb99c15def0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.59:40918/0 (socket says 172.21.15.59:40918) 2023-11-24T07:56:37.329 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:37.327+0000 7fb9a0963700 1 -- 172.21.15.59:0/2606904936 >> v1:172.21.15.76:6789/0 conn(0x7fb99c15db10 legacy=0x7fb99c15def0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T07:56:37.330 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:37.327+0000 7fb9a0963700 1 -- 172.21.15.59:0/2606904936 >> v1:172.21.15.59:6789/0 conn(0x7fb99c0ca2c0 legacy=0x7fb99c0ca6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:56:37.330 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:37.327+0000 7fb9a0963700 1 -- 172.21.15.59:0/2606904936 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb99c0ce880 con 0x7fb99c071390 2023-11-24T07:56:37.330 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:37.328+0000 7fb9a0963700 1 --2- 172.21.15.59:0/2606904936 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb99c071390 0x7fb99c15e600 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fb994006600 tx=0x7fb994007e60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ff8c3f80a30d368e server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:56:37.331 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:37.329+0000 7fb993fff700 1 -- 172.21.15.59:0/2606904936 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb994003c20 con 0x7fb99c071390 2023-11-24T07:56:37.331 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:37.330+0000 7fb993fff700 1 -- 172.21.15.59:0/2606904936 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb994003d80 con 0x7fb99c071390 2023-11-24T07:56:37.331 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:37.330+0000 7fb993fff700 1 -- 172.21.15.59:0/2606904936 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb99400cb00 con 0x7fb99c071390 2023-11-24T07:56:37.332 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:37.331+0000 7fb9a33c8700 1 -- 172.21.15.59:0/2606904936 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb99c071390 msgr2=0x7fb99c15e600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:56:37.333 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:37.331+0000 7fb9a33c8700 1 --2- 172.21.15.59:0/2606904936 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb99c071390 0x7fb99c15e600 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fb994006600 tx=0x7fb994007e60 comp rx=0 tx=0).stop 2023-11-24T07:56:37.337 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:37.335+0000 7fb9a33c8700 1 -- 172.21.15.59:0/2606904936 shutdown_connections 2023-11-24T07:56:37.337 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:37.335+0000 7fb9a33c8700 1 --2- 172.21.15.59:0/2606904936 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb99c071390 0x7fb99c15e600 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:56:37.337 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:37.335+0000 7fb9a33c8700 1 -- 172.21.15.59:0/2606904936 >> 172.21.15.59:0/2606904936 conn(0x7fb99c0cf980 msgr2=0x7fb99c0cfd90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:56:37.337 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:37.335+0000 7fb9a33c8700 1 -- 172.21.15.59:0/2606904936 shutdown_connections 2023-11-24T07:56:37.339 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:37.335+0000 7fb9a33c8700 1 -- 172.21.15.59:0/2606904936 wait complete. 2023-11-24T07:56:37.339 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:37.337+0000 7fb9a33c8700 1 Processor -- start 2023-11-24T07:56:37.340 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:37.338+0000 7fb9a33c8700 1 -- start start 2023-11-24T07:56:37.341 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:37.339+0000 7fb9a33c8700 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fb99c071390 0x7fb99c1f33c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:56:37.341 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:37.339+0000 7fb9a33c8700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb99c0ca2c0 0x7fb99c0b3360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:56:37.342 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:37.339+0000 7fb9a33c8700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb99c15db10 0x7fb99c0b38a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:56:37.342 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:37.339+0000 7fb9a33c8700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb99c0b5630 con 0x7fb99c15db10 2023-11-24T07:56:37.342 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:37.339+0000 7fb9a33c8700 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb99c0b3f20 con 0x7fb99c0ca2c0 2023-11-24T07:56:37.342 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:37.339+0000 7fb9a33c8700 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb99c0b4200 con 0x7fb99c071390 2023-11-24T07:56:37.342 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:37.340+0000 7fb9a1164700 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fb99c071390 0x7fb99c1f33c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:56:37.343 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:37.340+0000 7fb9a1965700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb99c15db10 0x7fb99c0b38a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:56:37.343 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:37.340+0000 7fb9a1965700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb99c15db10 0x7fb99c0b38a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:50172/0 (socket says 172.21.15.59:50172) 2023-11-24T07:56:37.343 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:37.340+0000 7fb9a1965700 1 -- 172.21.15.59:0/3407824262 learned_addr learned my addr 172.21.15.59:0/3407824262 (peer_addr_for_me v2:172.21.15.59:0/0) 2023-11-24T07:56:37.344 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:37.340+0000 7fb9a0963700 1 --2- 172.21.15.59:0/3407824262 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb99c0ca2c0 0x7fb99c0b3360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:56:37.344 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:37.341+0000 7fb9a1965700 1 -- 172.21.15.59:0/3407824262 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fb99c071390 msgr2=0x7fb99c1f33c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:56:37.344 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:37.341+0000 7fb9a1965700 1 --2- 172.21.15.59:0/3407824262 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fb99c071390 0x7fb99c1f33c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:56:37.344 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:37.341+0000 7fb9a1965700 1 -- 172.21.15.59:0/3407824262 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb99c0ca2c0 msgr2=0x7fb99c0b3360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:56:37.344 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:37.341+0000 7fb9a1965700 1 --2- 172.21.15.59:0/3407824262 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb99c0ca2c0 0x7fb99c0b3360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:56:37.344 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:37.341+0000 7fb9a1965700 1 -- 172.21.15.59:0/3407824262 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb9940077e0 con 0x7fb99c15db10 2023-11-24T07:56:37.344 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:37.341+0000 7fb9a1965700 1 --2- 172.21.15.59:0/3407824262 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb99c15db10 0x7fb99c0b38a0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fb99800c420 tx=0x7fb99800c7e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:56:37.345 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:37.342+0000 7fb991ffb700 1 -- 172.21.15.59:0/3407824262 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb998004c80 con 0x7fb99c15db10 2023-11-24T07:56:37.345 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:37.342+0000 7fb991ffb700 1 -- 172.21.15.59:0/3407824262 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb998004de0 con 0x7fb99c15db10 2023-11-24T07:56:37.345 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:37.342+0000 7fb991ffb700 1 -- 172.21.15.59:0/3407824262 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9980062f0 con 0x7fb99c15db10 2023-11-24T07:56:37.345 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:37.342+0000 7fb9a33c8700 1 -- 172.21.15.59:0/3407824262 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb99c0b44e0 con 0x7fb99c15db10 2023-11-24T07:56:37.345 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:37.342+0000 7fb9a33c8700 1 -- 172.21.15.59:0/3407824262 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb99c0b4aa0 con 0x7fb99c15db10 2023-11-24T07:56:37.347 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:37.345+0000 7fb991ffb700 1 -- 172.21.15.59:0/3407824262 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fb998006450 con 0x7fb99c15db10 2023-11-24T07:56:37.347 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:37.345+0000 7fb991ffb700 1 --2- 172.21.15.59:0/3407824262 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7fb98806b150 0x7fb98806d610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:56:37.347 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:37.345+0000 7fb991ffb700 1 -- 172.21.15.59:0/3407824262 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fb998088ad0 con 0x7fb99c15db10 2023-11-24T07:56:37.351 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:37.350+0000 7fb9a33c8700 1 -- 172.21.15.59:0/3407824262 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb97c0052e0 con 0x7fb99c15db10 2023-11-24T07:56:37.356 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:37.355+0000 7fb9a1164700 1 --2- 172.21.15.59:0/3407824262 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7fb98806b150 0x7fb98806d610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:56:37.361 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:37.356+0000 7fb9a1164700 1 --2- 172.21.15.59:0/3407824262 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7fb98806b150 0x7fb98806d610 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fb98c00a7f0 tx=0x7fb98c008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:56:37.361 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:37.359+0000 7fb991ffb700 1 -- 172.21.15.59:0/3407824262 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fb998053b60 con 0x7fb99c15db10 2023-11-24T07:56:37.393 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:37.391+0000 7f2d11ccc700 1 Processor -- start 2023-11-24T07:56:37.393 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:37.391+0000 7f814597a700 1 Processor -- start 2023-11-24T07:56:37.394 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:37.391+0000 7f307814f700 1 Processor -- start 2023-11-24T07:56:37.394 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:37.391+0000 7f814597a700 1 -- start start 2023-11-24T07:56:37.395 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:37.392+0000 7f307814f700 1 -- start start 2023-11-24T07:56:37.395 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:37.393+0000 7f814597a700 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f814014bf10 0x7f8140070a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:56:37.395 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:37.393+0000 7f814597a700 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f814006d1f0 con 0x7f81400cbb80 2023-11-24T07:56:37.395 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:37.393+0000 7f814597a700 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f81400cc080 con 0x7f8140165f70 2023-11-24T07:56:37.396 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:37.393+0000 7f814597a700 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f81400cdf60 con 0x7f814014bf10 2023-11-24T07:56:37.396 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:37.391+0000 7f2d11ccc700 1 -- start start 2023-11-24T07:56:37.397 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:37.393+0000 7f2d11ccc700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f2d0c14bf10 0x7f2d0c070a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:56:37.397 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:37.393+0000 7f2d11ccc700 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f2d0c0cbb80 0x7f2d0c06cae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:56:37.397 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:37.393+0000 7f2d11ccc700 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2d0c0cc080 con 0x7f2d0c165f70 2023-11-24T07:56:37.397 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:37.393+0000 7f2d11ccc700 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2d0c0cdf60 con 0x7f2d0c14bf10 2023-11-24T07:56:37.397 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:37.393+0000 7f2d11ccc700 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2d0c06e950 con 0x7f2d0c0cbb80 2023-11-24T07:56:37.397 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:37.393+0000 7f2d0affd700 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f2d0c0cbb80 0x7f2d0c06cae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:56:37.398 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:37.393+0000 7f2d0affd700 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f2d0c0cbb80 0x7f2d0c06cae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3301/0 says I am v2:172.21.15.59:49616/0 (socket says 172.21.15.59:49616) 2023-11-24T07:56:37.398 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:37.393+0000 7f2d0affd700 1 -- 172.21.15.59:0/3946484523 learned_addr learned my addr 172.21.15.59:0/3946484523 (peer_addr_for_me v2:172.21.15.59:0/0) 2023-11-24T07:56:37.399 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:37.393+0000 7f813f7fe700 1 --1- >> v1:172.21.15.76:6789/0 conn(0x7f8140165f70 0x7f81400ce3b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.59:40928/0 (socket says 172.21.15.59:40928) 2023-11-24T07:56:37.399 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:37.393+0000 7f813f7fe700 1 -- 172.21.15.59:0/3830462728 learned_addr learned my addr 172.21.15.59:0/3830462728 (peer_addr_for_me v1:172.21.15.59:0/0) 2023-11-24T07:56:37.399 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:37.393+0000 7f307814f700 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f307014c040 0x7f30701629b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:56:37.399 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:37.393+0000 7f307814f700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f307006e520 0x7f307006afd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:56:37.400 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:37.393+0000 7f307814f700 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f307006e9a0 0x7f307006b510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:56:37.400 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:37.393+0000 7f307814f700 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f30700b48d0 con 0x7f307006e520 2023-11-24T07:56:37.400 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:37.393+0000 7f307814f700 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3070162ef0 con 0x7f307006e9a0 2023-11-24T07:56:37.400 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:37.393+0000 7f307814f700 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f307006e250 con 0x7f307014c040 2023-11-24T07:56:37.400 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:37.394+0000 7f30766ec700 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f307006e9a0 0x7f307006b510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:56:37.401 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:37.394+0000 7f2d0affd700 1 -- 172.21.15.59:0/3946484523 >> v2:172.21.15.59:3300/0 conn(0x7f2d0c14bf10 msgr2=0x7f2d0c070a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T07:56:37.401 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:37.394+0000 7f2d0affd700 1 --2- 172.21.15.59:0/3946484523 >> v2:172.21.15.59:3300/0 conn(0x7f2d0c14bf10 0x7f2d0c070a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:56:37.401 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:37.394+0000 7f2d0affd700 1 -- 172.21.15.59:0/3946484523 >> v1:172.21.15.59:6789/0 conn(0x7f2d0c165f70 legacy=0x7f2d0c0ce3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T07:56:37.401 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:37.394+0000 7f2d0affd700 1 -- 172.21.15.59:0/3946484523 --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2d0c06ec30 con 0x7f2d0c0cbb80 2023-11-24T07:56:37.402 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:37.394+0000 7f3075eeb700 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f307014c040 0x7f30701629b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:56:37.402 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:37.394+0000 7f30766ec700 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f307006e9a0 0x7f307006b510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.59:37386/0 (socket says 172.21.15.59:37386) 2023-11-24T07:56:37.402 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:37.394+0000 7f30766ec700 1 -- 172.21.15.59:0/2982989120 learned_addr learned my addr 172.21.15.59:0/2982989120 (peer_addr_for_me v2:172.21.15.59:0/0) 2023-11-24T07:56:37.402 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:37.394+0000 7f2d0affd700 1 --2- 172.21.15.59:0/3946484523 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f2d0c0cbb80 0x7f2d0c06cae0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f2d04009e50 tx=0x7f2d0400b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=386d7b5455c2a11b server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:56:37.403 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:37.394+0000 7f30766ec700 1 -- 172.21.15.59:0/2982989120 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f307014c040 msgr2=0x7f30701629b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T07:56:37.403 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:37.394+0000 7f30766ec700 1 --2- 172.21.15.59:0/2982989120 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f307014c040 0x7f30701629b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:56:37.403 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:37.394+0000 7f30766ec700 1 -- 172.21.15.59:0/2982989120 >> v2:172.21.15.59:3300/0 conn(0x7f307006e520 msgr2=0x7f307006afd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T07:56:37.403 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:37.394+0000 7f30766ec700 1 --2- 172.21.15.59:0/2982989120 >> v2:172.21.15.59:3300/0 conn(0x7f307006e520 0x7f307006afd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:56:37.404 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:37.394+0000 7f30766ec700 1 -- 172.21.15.59:0/2982989120 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f307006dc90 con 0x7f307006e9a0 2023-11-24T07:56:37.404 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:37.394+0000 7f2d0a7fc700 1 -- 172.21.15.59:0/3946484523 <== mon.2 v2:172.21.15.76:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d0400e070 con 0x7f2d0c0cbb80 2023-11-24T07:56:37.404 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:37.395+0000 7f2d0a7fc700 1 -- 172.21.15.59:0/3946484523 <== mon.2 v2:172.21.15.76:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2d04003a20 con 0x7f2d0c0cbb80 2023-11-24T07:56:37.404 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:37.395+0000 7f2d0a7fc700 1 -- 172.21.15.59:0/3946484523 <== mon.2 v2:172.21.15.76:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d04003dd0 con 0x7f2d0c0cbb80 2023-11-24T07:56:37.405 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:37.395+0000 7f30766ec700 1 --2- 172.21.15.59:0/2982989120 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f307006e9a0 0x7f307006b510 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f30700734c0 tx=0x7f306800b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=48a8862cbd60df7d server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:56:37.405 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:37.395+0000 7f3074ee9700 1 -- 172.21.15.59:0/2982989120 <== mon.1 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f306800e070 con 0x7f307006e9a0 2023-11-24T07:56:37.405 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:37.395+0000 7f3074ee9700 1 -- 172.21.15.59:0/2982989120 <== mon.1 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f306800b910 con 0x7f307006e9a0 2023-11-24T07:56:37.405 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:37.395+0000 7f3074ee9700 1 -- 172.21.15.59:0/2982989120 <== mon.1 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f306800bcc0 con 0x7f307006e9a0 2023-11-24T07:56:37.406 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:37.396+0000 7f2d11ccc700 1 -- 172.21.15.59:0/3946484523 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f2d0c0cbb80 msgr2=0x7f2d0c06cae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:56:37.406 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:37.396+0000 7f2d11ccc700 1 --2- 172.21.15.59:0/3946484523 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f2d0c0cbb80 0x7f2d0c06cae0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f2d04009e50 tx=0x7f2d0400b330 comp rx=0 tx=0).stop 2023-11-24T07:56:37.406 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:37.396+0000 7f2d11ccc700 1 -- 172.21.15.59:0/3946484523 shutdown_connections 2023-11-24T07:56:37.406 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:37.396+0000 7f2d11ccc700 1 --2- 172.21.15.59:0/3946484523 >> v2:172.21.15.59:3300/0 conn(0x7f2d0c14bf10 0x7f2d0c070a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:56:37.406 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:37.396+0000 7f2d11ccc700 1 --2- 172.21.15.59:0/3946484523 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f2d0c0cbb80 0x7f2d0c06cae0 secure :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f2d04009e50 tx=0x7f2d0400b330 comp rx=0 tx=0).stop 2023-11-24T07:56:37.407 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:37.396+0000 7f2d11ccc700 1 -- 172.21.15.59:0/3946484523 >> 172.21.15.59:0/3946484523 conn(0x7f2d0c0cf980 msgr2=0x7f2d0c0cfd60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:56:37.407 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:37.396+0000 7f307814f700 1 -- 172.21.15.59:0/2982989120 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f307006e9a0 msgr2=0x7f307006b510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:56:37.407 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:37.396+0000 7f307814f700 1 --2- 172.21.15.59:0/2982989120 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f307006e9a0 0x7f307006b510 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f30700734c0 tx=0x7f306800b330 comp rx=0 tx=0).stop 2023-11-24T07:56:37.408 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:37.394+0000 7f813effd700 1 --2- 172.21.15.59:0/3830462728 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f814014bf10 0x7f8140070a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:56:37.408 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:37.395+0000 7f813effd700 1 -- 172.21.15.59:0/3830462728 >> v1:172.21.15.76:6789/0 conn(0x7f8140165f70 legacy=0x7f81400ce3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:56:37.408 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:37.395+0000 7f813effd700 1 -- 172.21.15.59:0/3830462728 >> v1:172.21.15.59:6789/0 conn(0x7f81400cbb80 legacy=0x7f814006cae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T07:56:37.408 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:37.395+0000 7f813effd700 1 -- 172.21.15.59:0/3830462728 --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f814006e950 con 0x7f814014bf10 2023-11-24T07:56:37.408 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:37.395+0000 7f813effd700 1 --2- 172.21.15.59:0/3830462728 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f814014bf10 0x7f8140070a80 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f81340097f0 tx=0x7f8134009b00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bec485e13553e88e server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:56:37.408 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:37.396+0000 7f813dffb700 1 -- 172.21.15.59:0/3830462728 <== mon.2 v2:172.21.15.76:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8134004230 con 0x7f814014bf10 2023-11-24T07:56:37.409 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:37.396+0000 7f307814f700 1 -- 172.21.15.59:0/2982989120 shutdown_connections 2023-11-24T07:56:37.409 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:37.396+0000 7f307814f700 1 --2- 172.21.15.59:0/2982989120 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f307006e9a0 0x7f307006b510 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:56:37.409 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:37.396+0000 7f307814f700 1 --2- 172.21.15.59:0/2982989120 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f307014c040 0x7f30701629b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:56:37.409 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:37.396+0000 7f307814f700 1 --2- 172.21.15.59:0/2982989120 >> v2:172.21.15.59:3300/0 conn(0x7f307006e520 0x7f307006afd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:56:37.409 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:37.396+0000 7f307814f700 1 -- 172.21.15.59:0/2982989120 >> 172.21.15.59:0/2982989120 conn(0x7f30700d0bc0 msgr2=0x7f30700d0fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:56:37.410 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:37.396+0000 7f813dffb700 1 -- 172.21.15.59:0/3830462728 <== mon.2 v2:172.21.15.76:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8134004390 con 0x7f814014bf10 2023-11-24T07:56:37.410 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:37.396+0000 7f813dffb700 1 -- 172.21.15.59:0/3830462728 <== mon.2 v2:172.21.15.76:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81340036a0 con 0x7f814014bf10 2023-11-24T07:56:37.410 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:37.397+0000 7f814597a700 1 -- 172.21.15.59:0/3830462728 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f814014bf10 msgr2=0x7f8140070a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:56:37.411 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:37.397+0000 7f814597a700 1 --2- 172.21.15.59:0/3830462728 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f814014bf10 0x7f8140070a80 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f81340097f0 tx=0x7f8134009b00 comp rx=0 tx=0).stop 2023-11-24T07:56:37.411 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:37.397+0000 7f814597a700 1 -- 172.21.15.59:0/3830462728 shutdown_connections 2023-11-24T07:56:37.411 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:37.397+0000 7f814597a700 1 --2- 172.21.15.59:0/3830462728 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f814014bf10 0x7f8140070a80 secure :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f81340097f0 tx=0x7f8134009b00 comp rx=0 tx=0).stop 2023-11-24T07:56:37.411 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:37.397+0000 7f814597a700 1 -- 172.21.15.59:0/3830462728 >> 172.21.15.59:0/3830462728 conn(0x7f81400cf980 msgr2=0x7f81400cfd60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:56:37.411 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:37.398+0000 7f307814f700 1 -- 172.21.15.59:0/2982989120 shutdown_connections 2023-11-24T07:56:37.412 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:37.398+0000 7f814597a700 1 -- 172.21.15.59:0/3830462728 shutdown_connections 2023-11-24T07:56:37.412 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:37.398+0000 7f814597a700 1 -- 172.21.15.59:0/3830462728 wait complete. 2023-11-24T07:56:37.412 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:37.398+0000 7f4178ff8700 1 Processor -- start 2023-11-24T07:56:37.413 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:37.398+0000 7f307814f700 1 -- 172.21.15.59:0/2982989120 wait complete. 2023-11-24T07:56:37.413 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:37.400+0000 7f307814f700 1 Processor -- start 2023-11-24T07:56:37.413 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:37.400+0000 7f814597a700 1 Processor -- start 2023-11-24T07:56:37.413 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:37.400+0000 7f814597a700 1 -- start start 2023-11-24T07:56:37.413 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:37.401+0000 7f814597a700 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f81400cbb80 0x7f8140143910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:56:37.414 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:37.401+0000 7f814597a700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8140165f70 0x7f8140145060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:56:37.414 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:37.401+0000 7f814597a700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f81401455a0 0x7f8140162890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:56:37.414 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:37.401+0000 7f814597a700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8140141d80 con 0x7f8140165f70 2023-11-24T07:56:37.414 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:37.401+0000 7f814597a700 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8140145bd0 con 0x7f81401455a0 2023-11-24T07:56:37.414 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:37.401+0000 7f814597a700 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8140145eb0 con 0x7f81400cbb80 2023-11-24T07:56:37.415 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:37.398+0000 7f4178ff8700 1 -- start start 2023-11-24T07:56:37.415 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:37.400+0000 7f4178ff8700 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f417415db10 0x7f417415def0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:56:37.415 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:37.400+0000 7f4178ff8700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f4174071390 0x7f417415e430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:56:37.415 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:37.400+0000 7f4178ff8700 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f41740ce000 con 0x7f417415a520 2023-11-24T07:56:37.415 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:37.400+0000 7f4178ff8700 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f41740cb2d0 con 0x7f4174071390 2023-11-24T07:56:37.415 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:37.400+0000 7f4178ff8700 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f41740ce2c0 con 0x7f417415db10 2023-11-24T07:56:37.415 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:37.400+0000 7f4172d9d700 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7f417415a520 0x7f417415a9b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:53672/0 (socket says 172.21.15.59:53672) 2023-11-24T07:56:37.416 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:37.400+0000 7f4172d9d700 1 -- 172.21.15.59:0/1578126407 learned_addr learned my addr 172.21.15.59:0/1578126407 (peer_addr_for_me v1:172.21.15.59:0/0) 2023-11-24T07:56:37.416 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:37.400+0000 7f417259c700 1 --2- 172.21.15.59:0/1578126407 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f417415db10 0x7f417415def0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:56:37.416 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:37.400+0000 7f4171d9b700 1 --2- 172.21.15.59:0/1578126407 >> v2:172.21.15.59:3300/0 conn(0x7f4174071390 0x7f417415e430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:56:37.416 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:37.401+0000 7f417259c700 1 -- 172.21.15.59:0/1578126407 >> v2:172.21.15.59:3300/0 conn(0x7f4174071390 msgr2=0x7f417415e430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T07:56:37.416 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:37.401+0000 7f417259c700 1 --2- 172.21.15.59:0/1578126407 >> v2:172.21.15.59:3300/0 conn(0x7f4174071390 0x7f417415e430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:56:37.416 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:37.401+0000 7f417259c700 1 -- 172.21.15.59:0/1578126407 >> v1:172.21.15.59:6789/0 conn(0x7f417415a520 legacy=0x7f417415a9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:56:37.416 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:37.401+0000 7f417259c700 1 -- 172.21.15.59:0/1578126407 --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f41740ce5a0 con 0x7f417415db10 2023-11-24T07:56:37.417 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:37.401+0000 7f4171d9b700 1 --2- 172.21.15.59:0/1578126407 >> v2:172.21.15.59:3300/0 conn(0x7f4174071390 0x7f417415e430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-24T07:56:37.417 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:37.400+0000 7f307814f700 1 -- start start 2023-11-24T07:56:37.417 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:37.401+0000 7f307814f700 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f307006e520 0x7f30701ffba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:56:37.417 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:37.401+0000 7f307814f700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f307006e9a0 0x7f30700ac010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:56:37.417 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:37.401+0000 7f307814f700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f307014c040 0x7f30700aded0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:56:37.418 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:37.401+0000 7f307814f700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f30700ac9f0 con 0x7f307014c040 2023-11-24T07:56:37.418 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:37.401+0000 7f307814f700 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f30700accd0 con 0x7f307006e9a0 2023-11-24T07:56:37.418 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:37.401+0000 7f307814f700 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f30700acfb0 con 0x7f307006e520 2023-11-24T07:56:37.418 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:37.401+0000 7f3075eeb700 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f307006e520 0x7f30701ffba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:56:37.418 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:37.401+0000 7f3075eeb700 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f307006e520 0x7f30701ffba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3301/0 says I am v2:172.21.15.59:49640/0 (socket says 172.21.15.59:49640) 2023-11-24T07:56:37.418 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:37.401+0000 7f3075eeb700 1 -- 172.21.15.59:0/213583033 learned_addr learned my addr 172.21.15.59:0/213583033 (peer_addr_for_me v2:172.21.15.59:0/0) 2023-11-24T07:56:37.418 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:37.401+0000 7f30756ea700 1 --2- 172.21.15.59:0/213583033 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f307006e9a0 0x7f30700ac010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:56:37.419 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:37.401+0000 7f30766ec700 1 --2- 172.21.15.59:0/213583033 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f307014c040 0x7f30700aded0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:56:37.419 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:37.401+0000 7f813e7fc700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8140165f70 0x7f8140145060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:56:37.419 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:37.401+0000 7f813e7fc700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8140165f70 0x7f8140145060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:50202/0 (socket says 172.21.15.59:50202) 2023-11-24T07:56:37.419 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:37.401+0000 7f813e7fc700 1 -- 172.21.15.59:0/2652776422 learned_addr learned my addr 172.21.15.59:0/2652776422 (peer_addr_for_me v2:172.21.15.59:0/0) 2023-11-24T07:56:37.419 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:37.402+0000 7f813effd700 1 --2- 172.21.15.59:0/2652776422 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f81400cbb80 0x7f8140143910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:56:37.420 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:37.401+0000 7f417259c700 1 --2- 172.21.15.59:0/1578126407 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f417415db10 0x7f417415def0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f416400bad0 tx=0x7f416400bde0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5fc367c62c97fbb5 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:56:37.420 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:37.401+0000 7f417159a700 1 -- 172.21.15.59:0/1578126407 <== mon.2 v2:172.21.15.76:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f416400dd20 con 0x7f417415db10 2023-11-24T07:56:37.420 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:37.402+0000 7f417159a700 1 -- 172.21.15.59:0/1578126407 <== mon.2 v2:172.21.15.76:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f416400de80 con 0x7f417415db10 2023-11-24T07:56:37.420 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:37.402+0000 7f417159a700 1 -- 172.21.15.59:0/1578126407 <== mon.2 v2:172.21.15.76:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4164006d50 con 0x7f417415db10 2023-11-24T07:56:37.420 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:37.402+0000 7f813e7fc700 1 -- 172.21.15.59:0/2652776422 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f81400cbb80 msgr2=0x7f8140143910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:56:37.421 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:37.402+0000 7f813e7fc700 1 --2- 172.21.15.59:0/2652776422 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f81400cbb80 0x7f8140143910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:56:37.421 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:37.402+0000 7f813e7fc700 1 -- 172.21.15.59:0/2652776422 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f81401455a0 msgr2=0x7f8140162890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:56:37.421 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:37.402+0000 7f813e7fc700 1 --2- 172.21.15.59:0/2652776422 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f81401455a0 0x7f8140162890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:56:37.421 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:37.402+0000 7f813e7fc700 1 -- 172.21.15.59:0/2652776422 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f81340094a0 con 0x7f8140165f70 2023-11-24T07:56:37.421 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:37.402+0000 7f3075eeb700 1 -- 172.21.15.59:0/213583033 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f307006e9a0 msgr2=0x7f30700ac010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:56:37.422 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:37.402+0000 7f3075eeb700 1 --2- 172.21.15.59:0/213583033 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f307006e9a0 0x7f30700ac010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:56:37.422 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:37.402+0000 7f3075eeb700 1 -- 172.21.15.59:0/213583033 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f307014c040 msgr2=0x7f30700aded0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:56:37.422 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:37.402+0000 7f3075eeb700 1 --2- 172.21.15.59:0/213583033 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f307014c040 0x7f30700aded0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:56:37.422 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:37.402+0000 7f3075eeb700 1 -- 172.21.15.59:0/213583033 --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3068009b30 con 0x7f307006e520 2023-11-24T07:56:37.423 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:37.403+0000 7f4178ff8700 1 -- 172.21.15.59:0/1578126407 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f417415db10 msgr2=0x7f417415def0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:56:37.423 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:37.403+0000 7f4178ff8700 1 --2- 172.21.15.59:0/1578126407 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f417415db10 0x7f417415def0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f416400bad0 tx=0x7f416400bde0 comp rx=0 tx=0).stop 2023-11-24T07:56:37.423 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:37.403+0000 7f4178ff8700 1 -- 172.21.15.59:0/1578126407 shutdown_connections 2023-11-24T07:56:37.423 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:37.403+0000 7f4178ff8700 1 --2- 172.21.15.59:0/1578126407 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f417415db10 0x7f417415def0 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:56:37.423 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:37.403+0000 7f4178ff8700 1 --2- 172.21.15.59:0/1578126407 >> v2:172.21.15.59:3300/0 conn(0x7f4174071390 0x7f417415e430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:56:37.423 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:37.403+0000 7f4178ff8700 1 -- 172.21.15.59:0/1578126407 >> 172.21.15.59:0/1578126407 conn(0x7f41740cf980 msgr2=0x7f41740cfd90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:56:37.424 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:37.402+0000 7f2d11ccc700 1 -- 172.21.15.59:0/3946484523 shutdown_connections 2023-11-24T07:56:37.424 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:37.402+0000 7f2d11ccc700 1 -- 172.21.15.59:0/3946484523 wait complete. 2023-11-24T07:56:37.424 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:37.403+0000 7f3075eeb700 1 --2- 172.21.15.59:0/213583033 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f307006e520 0x7f30701ffba0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f306000e870 tx=0x7f306000eb80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:56:37.425 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:37.403+0000 7f4178ff8700 1 -- 172.21.15.59:0/1578126407 shutdown_connections 2023-11-24T07:56:37.425 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:37.403+0000 7f30667fc700 1 -- 172.21.15.59:0/213583033 <== mon.2 v2:172.21.15.76:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3060013680 con 0x7f307006e520 2023-11-24T07:56:37.425 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:37.403+0000 7f4178ff8700 1 -- 172.21.15.59:0/1578126407 wait complete. 2023-11-24T07:56:37.426 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:37.403+0000 7f30667fc700 1 -- 172.21.15.59:0/213583033 <== mon.2 v2:172.21.15.76:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f30600137e0 con 0x7f307006e520 2023-11-24T07:56:37.426 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:37.403+0000 7f30667fc700 1 -- 172.21.15.59:0/213583033 <== mon.2 v2:172.21.15.76:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3060013b50 con 0x7f307006e520 2023-11-24T07:56:37.426 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:37.404+0000 7f2d11ccc700 1 Processor -- start 2023-11-24T07:56:37.427 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:37.404+0000 7f813e7fc700 1 --2- 172.21.15.59:0/2652776422 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8140165f70 0x7f8140145060 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f813000c4a0 tx=0x7f813000c7b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:56:37.427 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:37.403+0000 7f307814f700 1 -- 172.21.15.59:0/213583033 --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f30700ad290 con 0x7f307006e520 2023-11-24T07:56:37.427 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:37.404+0000 7f307814f700 1 -- 172.21.15.59:0/213583033 --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f30702000e0 con 0x7f307006e520 2023-11-24T07:56:37.427 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:37.405+0000 7f2d11ccc700 1 -- start start 2023-11-24T07:56:37.428 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:37.405+0000 7f4178ff8700 1 Processor -- start 2023-11-24T07:56:37.428 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:37.405+0000 7f829dc71700 1 Processor -- start 2023-11-24T07:56:37.428 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:37.405+0000 7f4178ff8700 1 -- start start 2023-11-24T07:56:37.429 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:37.405+0000 7f829dc71700 1 -- start start 2023-11-24T07:56:37.429 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:37.406+0000 7f4178ff8700 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f4174071390 0x7f4174158740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:56:37.429 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:37.406+0000 7f4178ff8700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f417415a520 0x7f41740b47a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:56:37.429 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:37.406+0000 7f4178ff8700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f417415db10 0x7f417413ebf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:56:37.429 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:37.406+0000 7f4178ff8700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41740b4ce0 con 0x7f417415a520 2023-11-24T07:56:37.430 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:37.406+0000 7f4178ff8700 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41741576a0 con 0x7f417415db10 2023-11-24T07:56:37.430 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:37.406+0000 7f4178ff8700 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4174157980 con 0x7f4174071390 2023-11-24T07:56:37.430 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:37.407+0000 7f829dc71700 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f82980cbd90 0x7f82980ca9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:56:37.430 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:37.407+0000 7f829dc71700 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f82980ce2c0 con 0x7f82980d04a0 2023-11-24T07:56:37.430 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:37.407+0000 7f829dc71700 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f82980ce5a0 con 0x7f8298071390 2023-11-24T07:56:37.431 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:37.407+0000 7f829dc71700 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f82980ce880 con 0x7f82980cbd90 2023-11-24T07:56:37.431 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:37.406+0000 7f2d11ccc700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2d0c14bf10 0x7f2d0c1606a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:56:37.431 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:37.406+0000 7f2d11ccc700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2d0c165f70 0x7f2d0c15f4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:56:37.431 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:37.406+0000 7f2d11ccc700 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f2d0c160be0 0x7f2d0c15fa10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:56:37.431 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:37.406+0000 7f2d11ccc700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d0c15ff50 con 0x7f2d0c165f70 2023-11-24T07:56:37.432 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:37.406+0000 7f2d11ccc700 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d0c1638e0 con 0x7f2d0c14bf10 2023-11-24T07:56:37.432 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:37.406+0000 7f2d11ccc700 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2d0c138910 con 0x7f2d0c160be0 2023-11-24T07:56:37.432 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:37.406+0000 7f2d0affd700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2d0c165f70 0x7f2d0c15f4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:56:37.432 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:37.407+0000 7f2d0affd700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2d0c165f70 0x7f2d0c15f4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:50218/0 (socket says 172.21.15.59:50218) 2023-11-24T07:56:37.432 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:37.407+0000 7f2d0affd700 1 -- 172.21.15.59:0/584783546 learned_addr learned my addr 172.21.15.59:0/584783546 (peer_addr_for_me v2:172.21.15.59:0/0) 2023-11-24T07:56:37.433 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:37.406+0000 7f4172d9d700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f417415db10 0x7f417413ebf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:56:37.433 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:37.407+0000 7f307814f700 1 -- 172.21.15.59:0/213583033 --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f30700691e0 con 0x7f307006e520 2023-11-24T07:56:37.433 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:37.407+0000 7f4171d9b700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f417415a520 0x7f41740b47a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:56:37.433 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:37.407+0000 7f4171d9b700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f417415a520 0x7f41740b47a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:50222/0 (socket says 172.21.15.59:50222) 2023-11-24T07:56:37.434 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:37.407+0000 7f4171d9b700 1 -- 172.21.15.59:0/3179878697 learned_addr learned my addr 172.21.15.59:0/3179878697 (peer_addr_for_me v2:172.21.15.59:0/0) 2023-11-24T07:56:37.434 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:37.407+0000 7f82977fe700 1 --1- >> v1:172.21.15.76:6789/0 conn(0x7f8298071390 0x7f829815adb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.59:40978/0 (socket says 172.21.15.59:40978) 2023-11-24T07:56:37.434 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:37.407+0000 7f82977fe700 1 -- 172.21.15.59:0/4124760166 learned_addr learned my addr 172.21.15.59:0/4124760166 (peer_addr_for_me v1:172.21.15.59:0/0) 2023-11-24T07:56:37.434 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:37.407+0000 7f417259c700 1 --2- 172.21.15.59:0/3179878697 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f4174071390 0x7f4174158740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:56:37.435 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:37.407+0000 7f30667fc700 1 -- 172.21.15.59:0/213583033 <== mon.2 v2:172.21.15.76:3301/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f3060013cb0 con 0x7f307006e520 2023-11-24T07:56:37.435 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:37.407+0000 7f8127fff700 1 -- 172.21.15.59:0/2652776422 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81300054e0 con 0x7f8140165f70 2023-11-24T07:56:37.436 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:37.407+0000 7f8297fff700 1 --2- 172.21.15.59:0/4124760166 >> v2:172.21.15.76:3300/0 conn(0x7f82980cbd90 0x7f82980ca9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:56:37.436 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:37.408+0000 7f8297fff700 1 -- 172.21.15.59:0/4124760166 >> v1:172.21.15.76:6789/0 conn(0x7f8298071390 legacy=0x7f829815adb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:56:37.436 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:37.408+0000 7f8297fff700 1 -- 172.21.15.59:0/4124760166 >> v1:172.21.15.59:6789/0 conn(0x7f82980d04a0 legacy=0x7f82980ca2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:56:37.436 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:37.408+0000 7f8297fff700 1 -- 172.21.15.59:0/4124760166 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f82980ceda0 con 0x7f82980cbd90 2023-11-24T07:56:37.436 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:37.408+0000 7f2d0affd700 1 -- 172.21.15.59:0/584783546 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f2d0c160be0 msgr2=0x7f2d0c15fa10 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2023-11-24T07:56:37.437 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:37.408+0000 7f2d0affd700 1 --2- 172.21.15.59:0/584783546 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f2d0c160be0 0x7f2d0c15fa10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:56:37.437 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:37.408+0000 7f2d0affd700 1 -- 172.21.15.59:0/584783546 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2d0c14bf10 msgr2=0x7f2d0c1606a0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2023-11-24T07:56:37.437 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:37.408+0000 7f2d0affd700 1 --2- 172.21.15.59:0/584783546 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2d0c14bf10 0x7f2d0c1606a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:56:37.437 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:37.408+0000 7f2d0affd700 1 -- 172.21.15.59:0/584783546 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2d04009b30 con 0x7f2d0c165f70 2023-11-24T07:56:37.437 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:37.407+0000 7f30667fc700 1 --2- 172.21.15.59:0/213583033 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f305406afa0 0x7f305406d460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:56:37.438 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:37.408+0000 7f30667fc700 1 -- 172.21.15.59:0/213583033 <== mon.2 v2:172.21.15.76:3301/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f3060089490 con 0x7f307006e520 2023-11-24T07:56:37.438 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:37.408+0000 7f30756ea700 1 --2- 172.21.15.59:0/213583033 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f305406afa0 0x7f305406d460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:56:37.438 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:37.407+0000 7f8127fff700 1 -- 172.21.15.59:0/2652776422 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8130003680 con 0x7f8140165f70 2023-11-24T07:56:37.438 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:37.407+0000 7f8127fff700 1 -- 172.21.15.59:0/2652776422 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8130003a30 con 0x7f8140165f70 2023-11-24T07:56:37.438 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:37.407+0000 7f4171d9b700 1 -- 172.21.15.59:0/3179878697 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f4174071390 msgr2=0x7f4174158740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:56:37.439 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:37.407+0000 7f4171d9b700 1 --2- 172.21.15.59:0/3179878697 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f4174071390 0x7f4174158740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:56:37.439 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:37.407+0000 7f4171d9b700 1 -- 172.21.15.59:0/3179878697 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f417415db10 msgr2=0x7f417413ebf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:56:37.439 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:37.407+0000 7f4171d9b700 1 --2- 172.21.15.59:0/3179878697 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f417415db10 0x7f417413ebf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:56:37.439 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:37.407+0000 7f4171d9b700 1 -- 172.21.15.59:0/3179878697 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f416400b730 con 0x7f417415a520 2023-11-24T07:56:37.440 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:37.408+0000 7f8297fff700 1 --2- 172.21.15.59:0/4124760166 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f82980cbd90 0x7f82980ca9d0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f8290009dc0 tx=0x7f829000b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9c8aa91f34a26db9 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:56:37.440 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:37.408+0000 7f4171d9b700 1 --2- 172.21.15.59:0/3179878697 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f417415a520 0x7f41740b47a0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f416800e810 tx=0x7f416800eb20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:56:37.440 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:37.408+0000 7f814597a700 1 -- 172.21.15.59:0/2652776422 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8140141730 con 0x7f8140165f70 2023-11-24T07:56:37.440 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:37.408+0000 7f814597a700 1 -- 172.21.15.59:0/2652776422 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f81401620f0 con 0x7f8140165f70 2023-11-24T07:56:37.441 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:37.408+0000 7f2d0affd700 1 --2- 172.21.15.59:0/584783546 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2d0c165f70 0x7f2d0c15f4d0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f2d04004000 tx=0x7f2d04004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:56:37.441 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:37.409+0000 7f30756ea700 1 --2- 172.21.15.59:0/213583033 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f305406afa0 0x7f305406d460 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f30700ab590 tx=0x7f306c009190 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:56:37.441 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:37.408+0000 7f82967fc700 1 -- 172.21.15.59:0/4124760166 <== mon.1 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f829000bb90 con 0x7f82980cbd90 2023-11-24T07:56:37.442 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:37.409+0000 7f82967fc700 1 -- 172.21.15.59:0/4124760166 <== mon.1 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f829000bcf0 con 0x7f82980cbd90 2023-11-24T07:56:37.442 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:37.409+0000 7f82967fc700 1 -- 172.21.15.59:0/4124760166 <== mon.1 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8290011640 con 0x7f82980cbd90 2023-11-24T07:56:37.442 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:37.409+0000 7f829dc71700 1 -- 172.21.15.59:0/4124760166 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f82980cbd90 msgr2=0x7f82980ca9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:56:37.442 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:37.409+0000 7f829dc71700 1 --2- 172.21.15.59:0/4124760166 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f82980cbd90 0x7f82980ca9d0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f8290009dc0 tx=0x7f829000b330 comp rx=0 tx=0).stop 2023-11-24T07:56:37.442 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:37.410+0000 7f829dc71700 1 -- 172.21.15.59:0/4124760166 shutdown_connections 2023-11-24T07:56:37.442 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:37.410+0000 7f829dc71700 1 --2- 172.21.15.59:0/4124760166 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f82980cbd90 0x7f82980ca9d0 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:56:37.442 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:37.410+0000 7f829dc71700 1 -- 172.21.15.59:0/4124760166 >> 172.21.15.59:0/4124760166 conn(0x7f82980cf980 msgr2=0x7f82980cfd90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:56:37.443 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:37.409+0000 7f4162ffd700 1 -- 172.21.15.59:0/3179878697 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f416800c020 con 0x7f417415a520 2023-11-24T07:56:37.443 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:37.409+0000 7f4162ffd700 1 -- 172.21.15.59:0/3179878697 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4168004250 con 0x7f417415a520 2023-11-24T07:56:37.443 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:37.409+0000 7f4162ffd700 1 -- 172.21.15.59:0/3179878697 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41680036a0 con 0x7f417415a520 2023-11-24T07:56:37.443 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:37.409+0000 7f4178ff8700 1 -- 172.21.15.59:0/3179878697 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4174157c60 con 0x7f417415a520 2023-11-24T07:56:37.443 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:37.409+0000 7f4178ff8700 1 -- 172.21.15.59:0/3179878697 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4174158220 con 0x7f417415a520 2023-11-24T07:56:37.444 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:37.410+0000 7f829dc71700 1 -- 172.21.15.59:0/4124760166 shutdown_connections 2023-11-24T07:56:37.444 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:37.410+0000 7f829dc71700 1 -- 172.21.15.59:0/4124760166 wait complete. 2023-11-24T07:56:37.444 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:37.411+0000 7f2cf3fff700 1 -- 172.21.15.59:0/584783546 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d04004440 con 0x7f2d0c165f70 2023-11-24T07:56:37.445 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:37.411+0000 7f2cf3fff700 1 -- 172.21.15.59:0/584783546 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2d040045a0 con 0x7f2d0c165f70 2023-11-24T07:56:37.445 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:37.411+0000 7f2cf3fff700 1 -- 172.21.15.59:0/584783546 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d04004910 con 0x7f2d0c165f70 2023-11-24T07:56:37.445 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:37.411+0000 7f8127fff700 1 -- 172.21.15.59:0/2652776422 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f8130003b90 con 0x7f8140165f70 2023-11-24T07:56:37.445 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:37.411+0000 7f4178ff8700 1 -- 172.21.15.59:0/3179878697 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4174158440 con 0x7f417415a520 2023-11-24T07:56:37.446 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:37.411+0000 7f2d11ccc700 1 -- 172.21.15.59:0/584783546 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2d0c1610f0 con 0x7f2d0c165f70 2023-11-24T07:56:37.446 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:37.411+0000 7f2d11ccc700 1 -- 172.21.15.59:0/584783546 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2d0c1616b0 con 0x7f2d0c165f70 2023-11-24T07:56:37.446 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:37.411+0000 7f8127fff700 1 --2- 172.21.15.59:0/2652776422 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f812c06af20 0x7f812c06d3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:56:37.447 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:37.411+0000 7f8127fff700 1 -- 172.21.15.59:0/2652776422 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f8130088840 con 0x7f8140165f70 2023-11-24T07:56:37.447 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:37.412+0000 7f4162ffd700 1 -- 172.21.15.59:0/3179878697 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f4168003800 con 0x7f417415a520 2023-11-24T07:56:37.447 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:37.412+0000 7f2d11ccc700 1 -- 172.21.15.59:0/584783546 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2d0c1618d0 con 0x7f2d0c165f70 2023-11-24T07:56:37.448 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:37.412+0000 7f30667fc700 1 -- 172.21.15.59:0/213583033 <== mon.2 v2:172.21.15.76:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f3060054490 con 0x7f307006e520 2023-11-24T07:56:37.448 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:37.412+0000 7f4162ffd700 1 --2- 172.21.15.59:0/3179878697 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f415406afa0 0x7f415406d460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:56:37.448 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:37.413+0000 7f4162ffd700 1 -- 172.21.15.59:0/3179878697 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f4168089140 con 0x7f417415a520 2023-11-24T07:56:37.448 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:37.412+0000 7f829dc71700 1 Processor -- start 2023-11-24T07:56:37.449 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:37.412+0000 7f829dc71700 1 -- start start 2023-11-24T07:56:37.449 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:37.413+0000 7f813effd700 1 --2- 172.21.15.59:0/2652776422 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f812c06af20 0x7f812c06d3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:56:37.450 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:37.413+0000 7f829dc71700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8298071390 0x7f82980c6320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:56:37.450 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:37.413+0000 7f829dc71700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f82980cbd90 0x7f82982002c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:56:37.451 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:37.413+0000 7f829dc71700 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f82980d04a0 0x7f82980a9df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:56:37.451 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:37.413+0000 7f829dc71700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82981ea080 con 0x7f82980cbd90 2023-11-24T07:56:37.451 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:37.413+0000 7f829dc71700 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82980c7a90 con 0x7f8298071390 2023-11-24T07:56:37.451 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:37.413+0000 7f829dc71700 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f82980c72a0 con 0x7f82980d04a0 2023-11-24T07:56:37.451 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:37.413+0000 7f8296ffd700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f82980cbd90 0x7f82982002c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:56:37.451 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:37.413+0000 7f8296ffd700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f82980cbd90 0x7f82982002c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:50236/0 (socket says 172.21.15.59:50236) 2023-11-24T07:56:37.451 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:37.413+0000 7f8296ffd700 1 -- 172.21.15.59:0/397815871 learned_addr learned my addr 172.21.15.59:0/397815871 (peer_addr_for_me v2:172.21.15.59:0/0) 2023-11-24T07:56:37.452 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:37.413+0000 7f8297fff700 1 --2- 172.21.15.59:0/397815871 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f82980d04a0 0x7f82980a9df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:56:37.452 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:37.413+0000 7f82977fe700 1 --2- 172.21.15.59:0/397815871 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8298071390 0x7f82980c6320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:56:37.452 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:37.413+0000 7f417259c700 1 --2- 172.21.15.59:0/3179878697 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f415406afa0 0x7f415406d460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:56:37.453 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:37.414+0000 7f8296ffd700 1 -- 172.21.15.59:0/397815871 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f82980d04a0 msgr2=0x7f82980a9df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:56:37.453 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:37.414+0000 7f8296ffd700 1 --2- 172.21.15.59:0/397815871 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f82980d04a0 0x7f82980a9df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:56:37.453 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:37.414+0000 7f8296ffd700 1 -- 172.21.15.59:0/397815871 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8298071390 msgr2=0x7f82980c6320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:56:37.453 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:37.414+0000 7f8296ffd700 1 --2- 172.21.15.59:0/397815871 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8298071390 0x7f82980c6320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:56:37.453 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:37.414+0000 7f8296ffd700 1 -- 172.21.15.59:0/397815871 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8290009a70 con 0x7f82980cbd90 2023-11-24T07:56:37.454 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:37.414+0000 7f8297fff700 1 --2- 172.21.15.59:0/397815871 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f82980d04a0 0x7f82980a9df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-24T07:56:37.454 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:37.414+0000 7f813effd700 1 --2- 172.21.15.59:0/2652776422 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f812c06af20 0x7f812c06d3e0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f8140067df0 tx=0x7f8134008030 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:56:37.454 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:37.414+0000 7f8296ffd700 1 --2- 172.21.15.59:0/397815871 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f82980cbd90 0x7f82982002c0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f828c00c450 tx=0x7f828c00c810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:56:37.455 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:37.414+0000 7f417259c700 1 --2- 172.21.15.59:0/3179878697 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f415406afa0 0x7f415406d460 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f4164000c00 tx=0x7f4164012000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:56:37.455 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:37.414+0000 7f67b7761700 1 Processor -- start 2023-11-24T07:56:37.455 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:37.415+0000 7f67b7761700 1 -- start start 2023-11-24T07:56:37.456 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:37.415+0000 7f814597a700 1 -- 172.21.15.59:0/2652776422 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8140067f60 con 0x7f8140165f70 2023-11-24T07:56:37.457 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:37.416+0000 7f67b7761700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f67b00ceff0 0x7f67b00c8cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:56:37.457 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:37.416+0000 7f67b7761700 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f67b00c9210 0x7f67b00cd550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:56:37.457 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:37.416+0000 7f67b7761700 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f67b00c9690 con 0x7f67b00cda90 2023-11-24T07:56:37.457 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:37.416+0000 7f67b7761700 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f67b00cf3d0 con 0x7f67b00ceff0 2023-11-24T07:56:37.457 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:37.416+0000 7f67b7761700 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f67b00718e0 con 0x7f67b00c9210 2023-11-24T07:56:37.457 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:37.416+0000 7f67b54fd700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f67b00ceff0 0x7f67b00c8cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:56:37.458 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:37.416+0000 7f67b5cfe700 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7f67b00cda90 0x7f67b00c9fd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:53722/0 (socket says 172.21.15.59:53722) 2023-11-24T07:56:37.458 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:37.416+0000 7f67b5cfe700 1 -- 172.21.15.59:0/48805878 learned_addr learned my addr 172.21.15.59:0/48805878 (peer_addr_for_me v1:172.21.15.59:0/0) 2023-11-24T07:56:37.458 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:37.416+0000 7f67b54fd700 1 --2- 172.21.15.59:0/48805878 >> v2:172.21.15.59:3300/0 conn(0x7f67b00ceff0 0x7f67b00c8cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:50244/0 (socket says 172.21.15.59:50244) 2023-11-24T07:56:37.458 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:37.416+0000 7f67b4cfc700 1 --2- 172.21.15.59:0/48805878 >> v2:172.21.15.76:3300/0 conn(0x7f67b00c9210 0x7f67b00cd550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:56:37.458 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:37.416+0000 7f67b54fd700 1 -- 172.21.15.59:0/48805878 >> v2:172.21.15.76:3300/0 conn(0x7f67b00c9210 msgr2=0x7f67b00cd550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T07:56:37.458 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:37.416+0000 7f67b54fd700 1 --2- 172.21.15.59:0/48805878 >> v2:172.21.15.76:3300/0 conn(0x7f67b00c9210 0x7f67b00cd550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:56:37.458 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:37.416+0000 7f67b54fd700 1 -- 172.21.15.59:0/48805878 >> v1:172.21.15.59:6789/0 conn(0x7f67b00cda90 legacy=0x7f67b00c9fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:56:37.459 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:37.417+0000 7f67b54fd700 1 -- 172.21.15.59:0/48805878 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f67b015e4a0 con 0x7f67b00ceff0 2023-11-24T07:56:37.459 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:37.417+0000 7f67b4cfc700 1 --2- 172.21.15.59:0/48805878 >> v2:172.21.15.76:3300/0 conn(0x7f67b00c9210 0x7f67b00cd550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-24T07:56:37.460 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:37.417+0000 7f68974af700 1 Processor -- start 2023-11-24T07:56:37.460 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:37.417+0000 7f68974af700 1 -- start start 2023-11-24T07:56:37.460 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:37.418+0000 7f68974af700 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f68900cdc60 0x7f6890168e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:56:37.460 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:37.418+0000 7f68974af700 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f68900718e0 con 0x7f68900ceff0 2023-11-24T07:56:37.460 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:37.418+0000 7f68974af700 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f689015e4a0 con 0x7f68900c93e0 2023-11-24T07:56:37.460 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:37.418+0000 7f68974af700 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f68900cf3d0 con 0x7f68900cdc60 2023-11-24T07:56:37.460 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:37.418+0000 7f689524b700 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7f68900ceff0 0x7f68900c8cd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:53724/0 (socket says 172.21.15.59:53724) 2023-11-24T07:56:37.461 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:37.418+0000 7f689524b700 1 -- 172.21.15.59:0/3909754213 learned_addr learned my addr 172.21.15.59:0/3909754213 (peer_addr_for_me v1:172.21.15.59:0/0) 2023-11-24T07:56:37.461 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:37.418+0000 7f6895a4c700 1 --2- 172.21.15.59:0/3909754213 >> v2:172.21.15.76:3300/0 conn(0x7f68900cdc60 0x7f6890168e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:56:37.461 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:37.418+0000 7f67b54fd700 1 --2- 172.21.15.59:0/48805878 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f67b00ceff0 0x7f67b00c8cd0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f67b00b5c10 tx=0x7f67ac009b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d04caedf73aa5142 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:56:37.461 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:37.418+0000 7f4162ffd700 1 -- 172.21.15.59:0/3179878697 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f4168054140 con 0x7f417415a520 2023-11-24T07:56:37.462 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:37.418+0000 7f6895a4c700 1 -- 172.21.15.59:0/3909754213 >> v1:172.21.15.76:6789/0 conn(0x7f68900c93e0 legacy=0x7f68900cd550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T07:56:37.462 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:37.418+0000 7f6895a4c700 1 -- 172.21.15.59:0/3909754213 >> v1:172.21.15.59:6789/0 conn(0x7f68900ceff0 legacy=0x7f68900c8cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:56:37.462 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:37.418+0000 7f6895a4c700 1 -- 172.21.15.59:0/3909754213 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f68900ce280 con 0x7f68900cdc60 2023-11-24T07:56:37.463 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:37.419+0000 7f8287fff700 1 -- 172.21.15.59:0/397815871 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f828c0052d0 con 0x7f82980cbd90 2023-11-24T07:56:37.463 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:37.419+0000 7f829dc71700 1 -- 172.21.15.59:0/397815871 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f82980c6a00 con 0x7f82980cbd90 2023-11-24T07:56:37.463 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:37.419+0000 7f829dc71700 1 -- 172.21.15.59:0/397815871 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f82980c6fc0 con 0x7f82980cbd90 2023-11-24T07:56:37.463 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:37.420+0000 7f2cf3fff700 1 -- 172.21.15.59:0/584783546 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f2d0402f430 con 0x7f2d0c165f70 2023-11-24T07:56:37.464 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:37.420+0000 7f2cf3fff700 1 --2- 172.21.15.59:0/584783546 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f2cf806aed0 0x7f2cf806d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:56:37.464 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:37.421+0000 7f829dc71700 1 -- 172.21.15.59:0/397815871 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8298067f60 con 0x7f82980cbd90 2023-11-24T07:56:37.464 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:37.421+0000 7f2d0b7fe700 1 --2- 172.21.15.59:0/584783546 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f2cf806aed0 0x7f2cf806d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:56:37.465 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:37.421+0000 7f2d0b7fe700 1 --2- 172.21.15.59:0/584783546 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f2cf806aed0 0x7f2cf806d390 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f2cfc009730 tx=0x7f2cfc006cb0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:56:37.465 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:37.423+0000 7f67a7fff700 1 -- 172.21.15.59:0/48805878 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67ac011830 con 0x7f67b00ceff0 2023-11-24T07:56:37.465 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:37.419+0000 7f6895a4c700 1 --2- 172.21.15.59:0/3909754213 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f68900cdc60 0x7f6890168e20 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f6888009d90 tx=0x7f688800b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=75e9eb94c92d1944 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:56:37.465 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:37.419+0000 7f6887fff700 1 -- 172.21.15.59:0/3909754213 <== mon.1 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f688800e070 con 0x7f68900cdc60 2023-11-24T07:56:37.466 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:37.419+0000 7f6887fff700 1 -- 172.21.15.59:0/3909754213 <== mon.1 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f688800bbd0 con 0x7f68900cdc60 2023-11-24T07:56:37.466 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:37.419+0000 7f6887fff700 1 -- 172.21.15.59:0/3909754213 <== mon.1 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6888003a20 con 0x7f68900cdc60 2023-11-24T07:56:37.466 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:37.420+0000 7f68974af700 1 -- 172.21.15.59:0/3909754213 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f68900cdc60 msgr2=0x7f6890168e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:56:37.466 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:37.420+0000 7f68974af700 1 --2- 172.21.15.59:0/3909754213 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f68900cdc60 0x7f6890168e20 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f6888009d90 tx=0x7f688800b330 comp rx=0 tx=0).stop 2023-11-24T07:56:37.466 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:37.420+0000 7f68974af700 1 -- 172.21.15.59:0/3909754213 shutdown_connections 2023-11-24T07:56:37.466 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:37.420+0000 7f68974af700 1 --2- 172.21.15.59:0/3909754213 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f68900cdc60 0x7f6890168e20 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:56:37.466 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:37.420+0000 7f68974af700 1 -- 172.21.15.59:0/3909754213 >> 172.21.15.59:0/3909754213 conn(0x7f68900cf980 msgr2=0x7f68900cfd90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:56:37.467 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:37.424+0000 7f8287fff700 1 -- 172.21.15.59:0/397815871 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f828c005430 con 0x7f82980cbd90 2023-11-24T07:56:37.467 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:37.424+0000 7f8287fff700 1 -- 172.21.15.59:0/397815871 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f828c003880 con 0x7f82980cbd90 2023-11-24T07:56:37.467 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:37.424+0000 7f67a7fff700 1 -- 172.21.15.59:0/48805878 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f67ac011990 con 0x7f67b00ceff0 2023-11-24T07:56:37.468 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:37.424+0000 7f67a7fff700 1 -- 172.21.15.59:0/48805878 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67ac011d20 con 0x7f67b00ceff0 2023-11-24T07:56:37.468 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:37.424+0000 7f2cf3fff700 1 -- 172.21.15.59:0/584783546 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f2d0402e390 con 0x7f2d0c165f70 2023-11-24T07:56:37.468 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:37.425+0000 7f67b7761700 1 -- 172.21.15.59:0/48805878 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f67b00ceff0 msgr2=0x7f67b00c8cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:56:37.469 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:37.425+0000 7f67b7761700 1 --2- 172.21.15.59:0/48805878 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f67b00ceff0 0x7f67b00c8cd0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f67b00b5c10 tx=0x7f67ac009b20 comp rx=0 tx=0).stop 2023-11-24T07:56:37.469 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:37.425+0000 7f67b7761700 1 -- 172.21.15.59:0/48805878 shutdown_connections 2023-11-24T07:56:37.469 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:37.425+0000 7f67b7761700 1 --2- 172.21.15.59:0/48805878 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f67b00ceff0 0x7f67b00c8cd0 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:56:37.469 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:37.425+0000 7f67b7761700 1 --2- 172.21.15.59:0/48805878 >> v2:172.21.15.76:3300/0 conn(0x7f67b00c9210 0x7f67b00cd550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:56:37.469 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:37.425+0000 7f67b7761700 1 -- 172.21.15.59:0/48805878 >> 172.21.15.59:0/48805878 conn(0x7f67b00cf980 msgr2=0x7f67b00cfd90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:56:37.470 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:37.424+0000 7f8127fff700 1 -- 172.21.15.59:0/2652776422 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f8130053b10 con 0x7f8140165f70 2023-11-24T07:56:37.470 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:37.428+0000 7f68974af700 1 -- 172.21.15.59:0/3909754213 shutdown_connections 2023-11-24T07:56:37.471 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:37.428+0000 7f67b7761700 1 -- 172.21.15.59:0/48805878 shutdown_connections 2023-11-24T07:56:37.471 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:37.428+0000 7f67b7761700 1 -- 172.21.15.59:0/48805878 wait complete. 2023-11-24T07:56:37.472 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:37.429+0000 7f67b7761700 1 Processor -- start 2023-11-24T07:56:37.472 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:37.429+0000 7f67b7761700 1 -- start start 2023-11-24T07:56:37.472 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:37.430+0000 7f67b7761700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f67b00c9210 0x7f67b013d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:56:37.472 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:37.430+0000 7f67b7761700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f67b00cda90 0x7f67b0144da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:56:37.472 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:37.430+0000 7f67b7761700 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f67b00ceff0 0x7f67b01452e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:56:37.472 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:37.430+0000 7f67b7761700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67b013d930 con 0x7f67b00c9210 2023-11-24T07:56:37.472 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:37.430+0000 7f67b7761700 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67b0141370 con 0x7f67b00cda90 2023-11-24T07:56:37.473 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:37.430+0000 7f67b7761700 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f67b0160810 con 0x7f67b00ceff0 2023-11-24T07:56:37.473 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:37.430+0000 7f67b5cfe700 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f67b00ceff0 0x7f67b01452e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:56:37.473 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:37.430+0000 7f67b54fd700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f67b00c9210 0x7f67b013d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:56:37.473 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:37.430+0000 7f67b54fd700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f67b00c9210 0x7f67b013d3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:50258/0 (socket says 172.21.15.59:50258) 2023-11-24T07:56:37.473 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:37.430+0000 7f67b5cfe700 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f67b00ceff0 0x7f67b01452e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3301/0 says I am v2:172.21.15.59:49700/0 (socket says 172.21.15.59:49700) 2023-11-24T07:56:37.473 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:37.430+0000 7f67b54fd700 1 -- 172.21.15.59:0/1858778710 learned_addr learned my addr 172.21.15.59:0/1858778710 (peer_addr_for_me v2:172.21.15.59:0/0) 2023-11-24T07:56:37.474 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:37.431+0000 7f67b5cfe700 1 -- 172.21.15.59:0/1858778710 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f67b00cda90 msgr2=0x7f67b0144da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:56:37.474 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:37.431+0000 7f67b5cfe700 1 --2- 172.21.15.59:0/1858778710 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f67b00cda90 0x7f67b0144da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:56:37.474 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:37.431+0000 7f67b5cfe700 1 -- 172.21.15.59:0/1858778710 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f67b00c9210 msgr2=0x7f67b013d3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:56:37.474 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:37.431+0000 7f67b5cfe700 1 --2- 172.21.15.59:0/1858778710 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f67b00c9210 0x7f67b013d3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:56:37.474 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:37.431+0000 7f67b5cfe700 1 -- 172.21.15.59:0/1858778710 --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f67a00094a0 con 0x7f67b00ceff0 2023-11-24T07:56:37.474 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:37.431+0000 7f67b54fd700 1 --2- 172.21.15.59:0/1858778710 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f67b00c9210 0x7f67b013d3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-24T07:56:37.475 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:37.431+0000 7f67b5cfe700 1 --2- 172.21.15.59:0/1858778710 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f67b00ceff0 0x7f67b01452e0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f67a800cad0 tx=0x7f67a800cde0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:56:37.475 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:37.433+0000 7f2cf3fff700 1 -- 172.21.15.59:0/584783546 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f2d04059a10 con 0x7f2d0c165f70 2023-11-24T07:56:37.476 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:37.434+0000 7f8287fff700 1 -- 172.21.15.59:0/397815871 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f828c0039e0 con 0x7f82980cbd90 2023-11-24T07:56:37.476 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:37.435+0000 7f8287fff700 1 --2- 172.21.15.59:0/397815871 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f828006b080 0x7f828006d540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:56:37.476 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:37.435+0000 7f8287fff700 1 -- 172.21.15.59:0/397815871 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f828c089050 con 0x7f82980cbd90 2023-11-24T07:56:37.476 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:37.435+0000 7f82977fe700 1 --2- 172.21.15.59:0/397815871 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f828006b080 0x7f828006d540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:56:37.476 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:37.435+0000 7f82977fe700 1 --2- 172.21.15.59:0/397815871 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f828006b080 0x7f828006d540 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f8288005910 tx=0x7f8288009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:56:37.477 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:37.439+0000 7f67a5ffb700 1 -- 172.21.15.59:0/1858778710 <== mon.2 v2:172.21.15.76:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67a800a9a0 con 0x7f67b00ceff0 2023-11-24T07:56:37.477 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:37.439+0000 7f67a5ffb700 1 -- 172.21.15.59:0/1858778710 <== mon.2 v2:172.21.15.76:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f67a800ab00 con 0x7f67b00ceff0 2023-11-24T07:56:37.477 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:37.439+0000 7f67a5ffb700 1 -- 172.21.15.59:0/1858778710 <== mon.2 v2:172.21.15.76:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67a800ae70 con 0x7f67b00ceff0 2023-11-24T07:56:37.477 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:37.439+0000 7f67b7761700 1 -- 172.21.15.59:0/1858778710 --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f67ac0094a0 con 0x7f67b00ceff0 2023-11-24T07:56:37.477 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:37.439+0000 7f67b7761700 1 -- 172.21.15.59:0/1858778710 --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f67b0160dd0 con 0x7f67b00ceff0 2023-11-24T07:56:37.478 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:37.439+0000 7f68974af700 1 -- 172.21.15.59:0/3909754213 wait complete. 2023-11-24T07:56:37.478 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:37.441+0000 7f68974af700 1 Processor -- start 2023-11-24T07:56:37.479 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:37.441+0000 7f67a5ffb700 1 -- 172.21.15.59:0/1858778710 <== mon.2 v2:172.21.15.76:3301/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f67a8012400 con 0x7f67b00ceff0 2023-11-24T07:56:37.479 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:37.441+0000 7f67a5ffb700 1 --2- 172.21.15.59:0/1858778710 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f679406af20 0x7f679406d3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:56:37.479 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:37.442+0000 7f67a5ffb700 1 -- 172.21.15.59:0/1858778710 <== mon.2 v2:172.21.15.76:3301/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f67a8088f90 con 0x7f67b00ceff0 2023-11-24T07:56:37.479 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:37.442+0000 7f67b54fd700 1 --2- 172.21.15.59:0/1858778710 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f679406af20 0x7f679406d3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:56:37.480 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:37.441+0000 7f68974af700 1 -- start start 2023-11-24T07:56:37.480 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:37.442+0000 7f68974af700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f68900c93e0 0x7f689015b330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:56:37.480 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:37.442+0000 7f68974af700 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f68900cdc60 0x7f68900cfd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:56:37.480 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:37.442+0000 7f68974af700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f68900ceff0 0x7f68900d02a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:56:37.480 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:37.442+0000 7f68974af700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68900b3850 con 0x7f68900c93e0 2023-11-24T07:56:37.480 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:37.442+0000 7f68974af700 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68900b2bd0 con 0x7f68900ceff0 2023-11-24T07:56:37.481 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:37.442+0000 7f68974af700 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f68900b2eb0 con 0x7f68900cdc60 2023-11-24T07:56:37.481 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:37.442+0000 7f689524b700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f68900c93e0 0x7f689015b330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:56:37.481 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:37.442+0000 7f689524b700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f68900c93e0 0x7f689015b330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:50268/0 (socket says 172.21.15.59:50268) 2023-11-24T07:56:37.481 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:37.442+0000 7f689524b700 1 -- 172.21.15.59:0/560923472 learned_addr learned my addr 172.21.15.59:0/560923472 (peer_addr_for_me v2:172.21.15.59:0/0) 2023-11-24T07:56:37.481 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:37.442+0000 7f6894a4a700 1 --2- 172.21.15.59:0/560923472 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f68900cdc60 0x7f68900cfd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:56:37.481 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:37.442+0000 7f6895a4c700 1 --2- 172.21.15.59:0/560923472 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f68900ceff0 0x7f68900d02a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:56:37.482 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:37.442+0000 7f67b54fd700 1 --2- 172.21.15.59:0/1858778710 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f679406af20 0x7f679406d3e0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f67ac014040 tx=0x7f67ac01c040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:56:37.482 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:37.443+0000 7f8287fff700 1 -- 172.21.15.59:0/397815871 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f828c054320 con 0x7f82980cbd90 2023-11-24T07:56:37.482 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:37.448+0000 7f67b7761700 1 -- 172.21.15.59:0/1858778710 --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f67b0141530 con 0x7f67b00ceff0 2023-11-24T07:56:37.484 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:37.443+0000 7f6894a4a700 1 -- 172.21.15.59:0/560923472 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f68900ceff0 msgr2=0x7f68900d02a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:56:37.484 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:37.443+0000 7f6894a4a700 1 --2- 172.21.15.59:0/560923472 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f68900ceff0 0x7f68900d02a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:56:37.484 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:37.443+0000 7f6894a4a700 1 -- 172.21.15.59:0/560923472 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f68900c93e0 msgr2=0x7f689015b330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:56:37.484 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:37.443+0000 7f6894a4a700 1 --2- 172.21.15.59:0/560923472 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f68900c93e0 0x7f689015b330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:56:37.484 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:37.443+0000 7f6894a4a700 1 -- 172.21.15.59:0/560923472 --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6888009a70 con 0x7f68900cdc60 2023-11-24T07:56:37.485 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:37.443+0000 7f689524b700 1 --2- 172.21.15.59:0/560923472 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f68900c93e0 0x7f689015b330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-24T07:56:37.485 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:37.443+0000 7f6894a4a700 1 --2- 172.21.15.59:0/560923472 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f68900cdc60 0x7f68900cfd60 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f688000b3f0 tx=0x7f688000b700 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:56:37.485 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:37.443+0000 7f6885ffb700 1 -- 172.21.15.59:0/560923472 <== mon.2 v2:172.21.15.76:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6880011830 con 0x7f68900cdc60 2023-11-24T07:56:37.485 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:37.443+0000 7f6885ffb700 1 -- 172.21.15.59:0/560923472 <== mon.2 v2:172.21.15.76:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6880011990 con 0x7f68900cdc60 2023-11-24T07:56:37.485 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:37.443+0000 7f68974af700 1 -- 172.21.15.59:0/560923472 --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f68900b3190 con 0x7f68900cdc60 2023-11-24T07:56:37.485 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:37.443+0000 7f6885ffb700 1 -- 172.21.15.59:0/560923472 <== mon.2 v2:172.21.15.76:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6880011d40 con 0x7f68900cdc60 2023-11-24T07:56:37.485 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:37.443+0000 7f68974af700 1 -- 172.21.15.59:0/560923472 --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f689015b870 con 0x7f68900cdc60 2023-11-24T07:56:37.486 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:37.445+0000 7f68974af700 1 -- 172.21.15.59:0/560923472 --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f68900b3990 con 0x7f68900cdc60 2023-11-24T07:56:37.486 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:37.445+0000 7f6885ffb700 1 -- 172.21.15.59:0/560923472 <== mon.2 v2:172.21.15.76:3301/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f688001a400 con 0x7f68900cdc60 2023-11-24T07:56:37.486 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:37.445+0000 7f6885ffb700 1 --2- 172.21.15.59:0/560923472 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f687406aed0 0x7f687406d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:56:37.489 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:37.445+0000 7f6885ffb700 1 -- 172.21.15.59:0/560923472 <== mon.2 v2:172.21.15.76:3301/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f68800891a0 con 0x7f68900cdc60 2023-11-24T07:56:37.489 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:37.445+0000 7f689524b700 1 --2- 172.21.15.59:0/560923472 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f687406aed0 0x7f687406d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:56:37.489 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:37.446+0000 7f689524b700 1 --2- 172.21.15.59:0/560923472 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f687406aed0 0x7f687406d390 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f688c005910 tx=0x7f688c009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:56:37.489 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:37.450+0000 7f6885ffb700 1 -- 172.21.15.59:0/560923472 <== mon.2 v2:172.21.15.76:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f68900b3990 con 0x7f68900cdc60 2023-11-24T07:56:37.490 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:37.455+0000 7f67a5ffb700 1 -- 172.21.15.59:0/1858778710 <== mon.2 v2:172.21.15.76:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f67a8054140 con 0x7f67b00ceff0 2023-11-24T07:56:37.790 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:37.789+0000 7fb9a33c8700 1 -- 172.21.15.59:0/3407824262 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) v1 -- 0x7fb97c005590 con 0x7fb99c15db10 2023-11-24T07:56:37.791 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:37.789+0000 7fb991ffb700 1 -- 172.21.15.59:0/3407824262 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fb998017030 con 0x7fb99c15db10 2023-11-24T07:56:37.794 INFO:teuthology.orchestra.run.smithi059.stdout:55834574855 2023-11-24T07:56:37.795 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:37.793+0000 7fb9a33c8700 1 -- 172.21.15.59:0/3407824262 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7fb98806b150 msgr2=0x7fb98806d610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:56:37.795 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:37.793+0000 7fb9a33c8700 1 --2- 172.21.15.59:0/3407824262 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7fb98806b150 0x7fb98806d610 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fb98c00a7f0 tx=0x7fb98c008040 comp rx=0 tx=0).stop 2023-11-24T07:56:37.795 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:37.793+0000 7fb9a33c8700 1 -- 172.21.15.59:0/3407824262 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb99c15db10 msgr2=0x7fb99c0b38a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:56:37.795 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:37.793+0000 7fb9a33c8700 1 --2- 172.21.15.59:0/3407824262 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb99c15db10 0x7fb99c0b38a0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fb99800c420 tx=0x7fb99800c7e0 comp rx=0 tx=0).stop 2023-11-24T07:56:37.795 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:37.794+0000 7fb9a33c8700 1 -- 172.21.15.59:0/3407824262 shutdown_connections 2023-11-24T07:56:37.795 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:37.794+0000 7fb9a33c8700 1 --2- 172.21.15.59:0/3407824262 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7fb98806b150 0x7fb98806d610 secure :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fb98c00a7f0 tx=0x7fb98c008040 comp rx=0 tx=0).stop 2023-11-24T07:56:37.796 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:37.794+0000 7fb9a33c8700 1 --2- 172.21.15.59:0/3407824262 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb99c15db10 0x7fb99c0b38a0 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:56:37.796 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:37.794+0000 7fb9a33c8700 1 --2- 172.21.15.59:0/3407824262 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fb99c071390 0x7fb99c1f33c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:56:37.796 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:37.794+0000 7fb9a33c8700 1 --2- 172.21.15.59:0/3407824262 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb99c0ca2c0 0x7fb99c0b3360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:56:37.796 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:37.794+0000 7fb9a33c8700 1 -- 172.21.15.59:0/3407824262 >> 172.21.15.59:0/3407824262 conn(0x7fb99c0cf980 msgr2=0x7fb99c0af5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:56:37.796 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:37.795+0000 7fb9a33c8700 1 -- 172.21.15.59:0/3407824262 shutdown_connections 2023-11-24T07:56:37.798 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:37.796+0000 7fb9a33c8700 1 -- 172.21.15.59:0/3407824262 wait complete. 2023-11-24T07:56:37.816 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574855 got 55834574855 for osd.4 2023-11-24T07:56:37.817 DEBUG:teuthology.parallel:result is None 2023-11-24T07:56:37.826 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:37.825+0000 7f307814f700 1 -- 172.21.15.59:0/213583033 --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f30700ad7b0 con 0x7f307006e520 2023-11-24T07:56:37.827 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:37.825+0000 7f30667fc700 1 -- 172.21.15.59:0/213583033 <== mon.2 v2:172.21.15.76:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f3060057ab0 con 0x7f307006e520 2023-11-24T07:56:37.828 INFO:teuthology.orchestra.run.smithi059.stdout:55834574856 2023-11-24T07:56:37.830 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:37.829+0000 7f307814f700 1 -- 172.21.15.59:0/213583033 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f305406afa0 msgr2=0x7f305406d460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:56:37.830 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:37.829+0000 7f307814f700 1 --2- 172.21.15.59:0/213583033 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f305406afa0 0x7f305406d460 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f30700ab590 tx=0x7f306c009190 comp rx=0 tx=0).stop 2023-11-24T07:56:37.830 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:37.829+0000 7f307814f700 1 -- 172.21.15.59:0/213583033 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f307006e520 msgr2=0x7f30701ffba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:56:37.831 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:37.829+0000 7f307814f700 1 --2- 172.21.15.59:0/213583033 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f307006e520 0x7f30701ffba0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f306000e870 tx=0x7f306000eb80 comp rx=0 tx=0).stop 2023-11-24T07:56:37.831 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:37.829+0000 7f307814f700 1 -- 172.21.15.59:0/213583033 shutdown_connections 2023-11-24T07:56:37.831 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:37.829+0000 7f307814f700 1 --2- 172.21.15.59:0/213583033 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f305406afa0 0x7f305406d460 secure :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f30700ab590 tx=0x7f306c009190 comp rx=0 tx=0).stop 2023-11-24T07:56:37.831 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:37.829+0000 7f307814f700 1 --2- 172.21.15.59:0/213583033 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f307014c040 0x7f30700aded0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:56:37.831 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:37.829+0000 7f307814f700 1 --2- 172.21.15.59:0/213583033 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f307006e520 0x7f30701ffba0 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:56:37.831 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:37.829+0000 7f307814f700 1 --2- 172.21.15.59:0/213583033 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f307006e9a0 0x7f30700ac010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:56:37.831 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:37.829+0000 7f307814f700 1 -- 172.21.15.59:0/213583033 >> 172.21.15.59:0/213583033 conn(0x7f30700d0bc0 msgr2=0x7f30700b4d60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:56:37.832 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:37.830+0000 7f307814f700 1 -- 172.21.15.59:0/213583033 shutdown_connections 2023-11-24T07:56:37.832 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:37.830+0000 7f307814f700 1 -- 172.21.15.59:0/213583033 wait complete. 2023-11-24T07:56:37.837 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:37.836+0000 7f814597a700 1 -- 172.21.15.59:0/2652776422 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) v1 -- 0x7f8140141ec0 con 0x7f8140165f70 2023-11-24T07:56:37.838 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:37.837+0000 7f68974af700 1 -- 172.21.15.59:0/560923472 --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f68900b3990 con 0x7f68900cdc60 2023-11-24T07:56:37.839 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:37.837+0000 7f8127fff700 1 -- 172.21.15.59:0/2652776422 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f8130017070 con 0x7f8140165f70 2023-11-24T07:56:37.839 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:37.837+0000 7f6885ffb700 1 -- 172.21.15.59:0/560923472 <== mon.2 v2:172.21.15.76:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f68900b3990 con 0x7f68900cdc60 2023-11-24T07:56:37.840 INFO:teuthology.orchestra.run.smithi059.stdout:55834574856 2023-11-24T07:56:37.840 INFO:teuthology.orchestra.run.smithi059.stdout:55834574856 2023-11-24T07:56:37.842 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:37.840+0000 7f68974af700 1 -- 172.21.15.59:0/560923472 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f687406aed0 msgr2=0x7f687406d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:56:37.842 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:37.840+0000 7f68974af700 1 --2- 172.21.15.59:0/560923472 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f687406aed0 0x7f687406d390 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f688c005910 tx=0x7f688c009040 comp rx=0 tx=0).stop 2023-11-24T07:56:37.842 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:37.840+0000 7f68974af700 1 -- 172.21.15.59:0/560923472 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f68900cdc60 msgr2=0x7f68900cfd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:56:37.842 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:37.840+0000 7f68974af700 1 --2- 172.21.15.59:0/560923472 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f68900cdc60 0x7f68900cfd60 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f688000b3f0 tx=0x7f688000b700 comp rx=0 tx=0).stop 2023-11-24T07:56:37.843 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:37.840+0000 7f68974af700 1 -- 172.21.15.59:0/560923472 shutdown_connections 2023-11-24T07:56:37.843 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:37.840+0000 7f68974af700 1 --2- 172.21.15.59:0/560923472 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f687406aed0 0x7f687406d390 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:56:37.843 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:37.840+0000 7f68974af700 1 --2- 172.21.15.59:0/560923472 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f68900ceff0 0x7f68900d02a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:56:37.843 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:37.840+0000 7f68974af700 1 --2- 172.21.15.59:0/560923472 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f68900c93e0 0x7f689015b330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:56:37.843 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:37.840+0000 7f68974af700 1 --2- 172.21.15.59:0/560923472 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f68900cdc60 0x7f68900cfd60 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:56:37.843 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:37.840+0000 7f68974af700 1 -- 172.21.15.59:0/560923472 >> 172.21.15.59:0/560923472 conn(0x7f68900cf980 msgr2=0x7f689015d780 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:56:37.843 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:37.840+0000 7f68974af700 1 -- 172.21.15.59:0/560923472 shutdown_connections 2023-11-24T07:56:37.843 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:37.840+0000 7f68974af700 1 -- 172.21.15.59:0/560923472 wait complete. 2023-11-24T07:56:37.844 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:37.841+0000 7f814597a700 1 -- 172.21.15.59:0/2652776422 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f812c06af20 msgr2=0x7f812c06d3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:56:37.844 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:37.841+0000 7f814597a700 1 --2- 172.21.15.59:0/2652776422 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f812c06af20 0x7f812c06d3e0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f8140067df0 tx=0x7f8134008030 comp rx=0 tx=0).stop 2023-11-24T07:56:37.844 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:37.841+0000 7f814597a700 1 -- 172.21.15.59:0/2652776422 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8140165f70 msgr2=0x7f8140145060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:56:37.844 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:37.841+0000 7f814597a700 1 --2- 172.21.15.59:0/2652776422 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8140165f70 0x7f8140145060 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f813000c4a0 tx=0x7f813000c7b0 comp rx=0 tx=0).stop 2023-11-24T07:56:37.845 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:37.841+0000 7f814597a700 1 -- 172.21.15.59:0/2652776422 shutdown_connections 2023-11-24T07:56:37.845 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:37.841+0000 7f814597a700 1 --2- 172.21.15.59:0/2652776422 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f812c06af20 0x7f812c06d3e0 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:56:37.845 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:37.841+0000 7f814597a700 1 --2- 172.21.15.59:0/2652776422 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f81401455a0 0x7f8140162890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:56:37.845 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:37.841+0000 7f814597a700 1 --2- 172.21.15.59:0/2652776422 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f81400cbb80 0x7f8140143910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:56:37.845 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:37.841+0000 7f814597a700 1 --2- 172.21.15.59:0/2652776422 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8140165f70 0x7f8140145060 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:56:37.845 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:37.841+0000 7f814597a700 1 -- 172.21.15.59:0/2652776422 >> 172.21.15.59:0/2652776422 conn(0x7f81400cf980 msgr2=0x7f81400af5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:56:37.845 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:37.842+0000 7f814597a700 1 -- 172.21.15.59:0/2652776422 shutdown_connections 2023-11-24T07:56:37.845 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:37.842+0000 7f814597a700 1 -- 172.21.15.59:0/2652776422 wait complete. 2023-11-24T07:56:37.845 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:37.844+0000 7f4178ff8700 1 -- 172.21.15.59:0/3179878697 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f4174067f60 con 0x7f417415a520 2023-11-24T07:56:37.846 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:37.845+0000 7f4162ffd700 1 -- 172.21.15.59:0/3179878697 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f4168057760 con 0x7f417415a520 2023-11-24T07:56:37.852 INFO:teuthology.orchestra.run.smithi059.stdout:55834574856 2023-11-24T07:56:37.854 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574856 got 55834574856 for osd.2 2023-11-24T07:56:37.854 DEBUG:teuthology.parallel:result is None 2023-11-24T07:56:37.855 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:37.853+0000 7f4178ff8700 1 -- 172.21.15.59:0/3179878697 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f415406afa0 msgr2=0x7f415406d460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:56:37.855 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:37.853+0000 7f4178ff8700 1 --2- 172.21.15.59:0/3179878697 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f415406afa0 0x7f415406d460 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f4164000c00 tx=0x7f4164012000 comp rx=0 tx=0).stop 2023-11-24T07:56:37.855 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:37.853+0000 7f4178ff8700 1 -- 172.21.15.59:0/3179878697 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f417415a520 msgr2=0x7f41740b47a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:56:37.856 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:37.853+0000 7f4178ff8700 1 --2- 172.21.15.59:0/3179878697 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f417415a520 0x7f41740b47a0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f416800e810 tx=0x7f416800eb20 comp rx=0 tx=0).stop 2023-11-24T07:56:37.857 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:37.854+0000 7f4178ff8700 1 -- 172.21.15.59:0/3179878697 shutdown_connections 2023-11-24T07:56:37.857 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:37.854+0000 7f4178ff8700 1 --2- 172.21.15.59:0/3179878697 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f415406afa0 0x7f415406d460 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:56:37.857 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:37.854+0000 7f4178ff8700 1 --2- 172.21.15.59:0/3179878697 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f417415db10 0x7f417413ebf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:56:37.857 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:37.854+0000 7f4178ff8700 1 --2- 172.21.15.59:0/3179878697 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f4174071390 0x7f4174158740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:56:37.857 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:37.854+0000 7f4178ff8700 1 --2- 172.21.15.59:0/3179878697 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f417415a520 0x7f41740b47a0 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:56:37.857 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:37.854+0000 7f4178ff8700 1 -- 172.21.15.59:0/3179878697 >> 172.21.15.59:0/3179878697 conn(0x7f41740cf980 msgr2=0x7f417406eac0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:56:37.857 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:37.854+0000 7f4178ff8700 1 -- 172.21.15.59:0/3179878697 shutdown_connections 2023-11-24T07:56:37.858 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:37.854+0000 7f4178ff8700 1 -- 172.21.15.59:0/3179878697 wait complete. 2023-11-24T07:56:37.858 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:37.855+0000 7f2d11ccc700 1 -- 172.21.15.59:0/584783546 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f2d0c1618d0 con 0x7f2d0c165f70 2023-11-24T07:56:37.858 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:37.856+0000 7f2cf3fff700 1 -- 172.21.15.59:0/584783546 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f2d0c1618d0 con 0x7f2d0c165f70 2023-11-24T07:56:37.858 INFO:teuthology.orchestra.run.smithi059.stdout:55834574856 2023-11-24T07:56:37.859 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:37.858+0000 7f2d11ccc700 1 -- 172.21.15.59:0/584783546 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f2cf806aed0 msgr2=0x7f2cf806d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:56:37.860 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:37.858+0000 7f2d11ccc700 1 --2- 172.21.15.59:0/584783546 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f2cf806aed0 0x7f2cf806d390 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f2cfc009730 tx=0x7f2cfc006cb0 comp rx=0 tx=0).stop 2023-11-24T07:56:37.860 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:37.858+0000 7f2d11ccc700 1 -- 172.21.15.59:0/584783546 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2d0c165f70 msgr2=0x7f2d0c15f4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:56:37.860 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:37.858+0000 7f2d11ccc700 1 --2- 172.21.15.59:0/584783546 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2d0c165f70 0x7f2d0c15f4d0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f2d04004000 tx=0x7f2d04004ad0 comp rx=0 tx=0).stop 2023-11-24T07:56:37.860 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:37.859+0000 7f829dc71700 1 -- 172.21.15.59:0/397815871 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) v1 -- 0x7f82980cb2d0 con 0x7f82980cbd90 2023-11-24T07:56:37.861 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:37.859+0000 7f8287fff700 1 -- 172.21.15.59:0/397815871 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f828c057940 con 0x7f82980cbd90 2023-11-24T07:56:37.861 INFO:teuthology.orchestra.run.smithi059.stdout:55834574854 2023-11-24T07:56:37.863 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574856 got 55834574856 for osd.0 2023-11-24T07:56:37.864 DEBUG:teuthology.parallel:result is None 2023-11-24T07:56:37.864 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:37.861+0000 7f2d11ccc700 1 -- 172.21.15.59:0/584783546 shutdown_connections 2023-11-24T07:56:37.864 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:37.861+0000 7f2d11ccc700 1 --2- 172.21.15.59:0/584783546 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f2cf806aed0 0x7f2cf806d390 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:56:37.864 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:37.861+0000 7f2d11ccc700 1 --2- 172.21.15.59:0/584783546 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f2d0c160be0 0x7f2d0c15fa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:56:37.864 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:37.861+0000 7f2d11ccc700 1 --2- 172.21.15.59:0/584783546 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2d0c14bf10 0x7f2d0c1606a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:56:37.865 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:37.861+0000 7f2d11ccc700 1 --2- 172.21.15.59:0/584783546 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2d0c165f70 0x7f2d0c15f4d0 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:56:37.865 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:37.861+0000 7f2d11ccc700 1 -- 172.21.15.59:0/584783546 >> 172.21.15.59:0/584783546 conn(0x7f2d0c0cf980 msgr2=0x7f2d0c0cc360 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:56:37.865 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:37.862+0000 7f2d11ccc700 1 -- 172.21.15.59:0/584783546 shutdown_connections 2023-11-24T07:56:37.865 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:37.862+0000 7f2d11ccc700 1 -- 172.21.15.59:0/584783546 wait complete. 2023-11-24T07:56:37.866 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:37.862+0000 7f829dc71700 1 -- 172.21.15.59:0/397815871 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f828006b080 msgr2=0x7f828006d540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:56:37.866 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:37.862+0000 7f829dc71700 1 --2- 172.21.15.59:0/397815871 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f828006b080 0x7f828006d540 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f8288005910 tx=0x7f8288009040 comp rx=0 tx=0).stop 2023-11-24T07:56:37.866 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:37.862+0000 7f829dc71700 1 -- 172.21.15.59:0/397815871 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f82980cbd90 msgr2=0x7f82982002c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:56:37.866 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:37.862+0000 7f829dc71700 1 --2- 172.21.15.59:0/397815871 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f82980cbd90 0x7f82982002c0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f828c00c450 tx=0x7f828c00c810 comp rx=0 tx=0).stop 2023-11-24T07:56:37.867 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:37.863+0000 7f829dc71700 1 -- 172.21.15.59:0/397815871 shutdown_connections 2023-11-24T07:56:37.867 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:37.863+0000 7f829dc71700 1 --2- 172.21.15.59:0/397815871 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f828006b080 0x7f828006d540 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:56:37.867 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:37.863+0000 7f829dc71700 1 --2- 172.21.15.59:0/397815871 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f82980d04a0 0x7f82980a9df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:56:37.867 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:37.863+0000 7f829dc71700 1 --2- 172.21.15.59:0/397815871 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8298071390 0x7f82980c6320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:56:37.867 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:37.863+0000 7f829dc71700 1 --2- 172.21.15.59:0/397815871 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f82980cbd90 0x7f82982002c0 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:56:37.867 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:37.863+0000 7f829dc71700 1 -- 172.21.15.59:0/397815871 >> 172.21.15.59:0/397815871 conn(0x7f82980cf980 msgr2=0x7f82980b07e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:56:37.867 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:37.863+0000 7f829dc71700 1 -- 172.21.15.59:0/397815871 shutdown_connections 2023-11-24T07:56:37.868 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:37.863+0000 7f829dc71700 1 -- 172.21.15.59:0/397815871 wait complete. 2023-11-24T07:56:37.868 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574856 got 55834574856 for osd.5 2023-11-24T07:56:37.869 DEBUG:teuthology.parallel:result is None 2023-11-24T07:56:37.870 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:37.868+0000 7f67b7761700 1 -- 172.21.15.59:0/1858778710 --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) v1 -- 0x7f67b0067f60 con 0x7f67b00ceff0 2023-11-24T07:56:37.870 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:37.868+0000 7f67a5ffb700 1 -- 172.21.15.59:0/1858778710 <== mon.2 v2:172.21.15.76:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f67a8017020 con 0x7f67b00ceff0 2023-11-24T07:56:37.870 INFO:teuthology.orchestra.run.smithi059.stdout:55834574855 2023-11-24T07:56:37.875 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:37.872+0000 7f67b7761700 1 -- 172.21.15.59:0/1858778710 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f679406af20 msgr2=0x7f679406d3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:56:37.875 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:37.872+0000 7f67b7761700 1 --2- 172.21.15.59:0/1858778710 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f679406af20 0x7f679406d3e0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f67ac014040 tx=0x7f67ac01c040 comp rx=0 tx=0).stop 2023-11-24T07:56:37.875 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:37.872+0000 7f67b7761700 1 -- 172.21.15.59:0/1858778710 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f67b00ceff0 msgr2=0x7f67b01452e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:56:37.875 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:37.872+0000 7f67b7761700 1 --2- 172.21.15.59:0/1858778710 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f67b00ceff0 0x7f67b01452e0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f67a800cad0 tx=0x7f67a800cde0 comp rx=0 tx=0).stop 2023-11-24T07:56:37.875 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:37.872+0000 7f67b7761700 1 -- 172.21.15.59:0/1858778710 shutdown_connections 2023-11-24T07:56:37.876 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:37.872+0000 7f67b7761700 1 --2- 172.21.15.59:0/1858778710 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f679406af20 0x7f679406d3e0 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:56:37.876 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:37.872+0000 7f67b7761700 1 --2- 172.21.15.59:0/1858778710 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f67b00ceff0 0x7f67b01452e0 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:56:37.876 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:37.872+0000 7f67b7761700 1 --2- 172.21.15.59:0/1858778710 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f67b00c9210 0x7f67b013d3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:56:37.876 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:37.872+0000 7f67b7761700 1 --2- 172.21.15.59:0/1858778710 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f67b00cda90 0x7f67b0144da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:56:37.877 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:37.872+0000 7f67b7761700 1 -- 172.21.15.59:0/1858778710 >> 172.21.15.59:0/1858778710 conn(0x7f67b00cf980 msgr2=0x7f67b00cc3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:56:37.877 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:37.872+0000 7f67b7761700 1 -- 172.21.15.59:0/1858778710 shutdown_connections 2023-11-24T07:56:37.877 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:37.872+0000 7f67b7761700 1 -- 172.21.15.59:0/1858778710 wait complete. 2023-11-24T07:56:37.878 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574856 got 55834574856 for osd.3 2023-11-24T07:56:37.879 DEBUG:teuthology.parallel:result is None 2023-11-24T07:56:37.880 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574856 got 55834574856 for osd.1 2023-11-24T07:56:37.880 DEBUG:teuthology.parallel:result is None 2023-11-24T07:56:37.881 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574854 got 55834574854 for osd.7 2023-11-24T07:56:37.881 DEBUG:teuthology.parallel:result is None 2023-11-24T07:56:37.889 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574855 got 55834574855 for osd.6 2023-11-24T07:56:37.889 DEBUG:teuthology.parallel:result is None 2023-11-24T07:56:37.889 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2023-11-24T07:56:37.889 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-11-24T07:56:38.053 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:38.051+0000 7ffaf3102700 1 Processor -- start 2023-11-24T07:56:38.053 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:38.051+0000 7ffaf3102700 1 -- start start 2023-11-24T07:56:38.061 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:38.052+0000 7ffaf3102700 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7ffaec1626d0 0x7ffaec06dd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:56:38.062 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:38.052+0000 7ffaf3102700 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffaec162240 con 0x7ffaec06afd0 2023-11-24T07:56:38.062 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:38.052+0000 7ffaf3102700 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffaec06e810 con 0x7ffaec06e2b0 2023-11-24T07:56:38.062 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:38.052+0000 7ffaf3102700 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffaec161bf0 con 0x7ffaec1626d0 2023-11-24T07:56:38.062 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:38.052+0000 7ffaf0e9e700 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7ffaec06afd0 0x7ffaec06b3b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:53750/0 (socket says 172.21.15.59:53750) 2023-11-24T07:56:38.062 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:38.052+0000 7ffaf0e9e700 1 -- 172.21.15.59:0/1511249089 learned_addr learned my addr 172.21.15.59:0/1511249089 (peer_addr_for_me v1:172.21.15.59:0/0) 2023-11-24T07:56:38.062 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:38.052+0000 7ffaebfff700 1 --2- 172.21.15.59:0/1511249089 >> v2:172.21.15.76:3300/0 conn(0x7ffaec1626d0 0x7ffaec06dd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:56:38.063 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:38.052+0000 7ffaeb7fe700 1 -- 172.21.15.59:0/1511249089 <== mon.0 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1485743157 0 0) 0x7ffaec162240 con 0x7ffaec06afd0 2023-11-24T07:56:38.063 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:38.052+0000 7ffaeb7fe700 1 -- 172.21.15.59:0/1511249089 --> v1:172.21.15.59:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ffadc0035b0 con 0x7ffaec06afd0 2023-11-24T07:56:38.063 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:38.052+0000 7ffaeb7fe700 1 -- 172.21.15.59:0/1511249089 <== mon.1 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3154645034 0 0) 0x7ffaec06e810 con 0x7ffaec06e2b0 2023-11-24T07:56:38.063 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:38.052+0000 7ffaeb7fe700 1 -- 172.21.15.59:0/1511249089 --> v1:172.21.15.76:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ffaec162240 con 0x7ffaec06e2b0 2023-11-24T07:56:38.063 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:38.052+0000 7ffaebfff700 1 -- 172.21.15.59:0/1511249089 >> v1:172.21.15.76:6789/0 conn(0x7ffaec06e2b0 legacy=0x7ffaec15e4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:56:38.064 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:38.052+0000 7ffaebfff700 1 -- 172.21.15.59:0/1511249089 >> v1:172.21.15.59:6789/0 conn(0x7ffaec06afd0 legacy=0x7ffaec06b3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:56:38.064 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:38.052+0000 7ffaeb7fe700 1 -- 172.21.15.59:0/1511249089 <== mon.0 v1:172.21.15.59:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2902138235 0 0) 0x7ffadc0035b0 con 0x7ffaec06afd0 2023-11-24T07:56:38.064 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:38.052+0000 7ffaebfff700 1 -- 172.21.15.59:0/1511249089 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffaec161ed0 con 0x7ffaec1626d0 2023-11-24T07:56:38.064 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:38.053+0000 7ffaebfff700 1 --2- 172.21.15.59:0/1511249089 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ffaec1626d0 0x7ffaec06dd70 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7ffad8009840 tx=0x7ffad8009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a45e62cbcfd9671d server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:56:38.064 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:38.053+0000 7ffaeb7fe700 1 -- 172.21.15.59:0/1511249089 <== mon.1 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffae0001670 con 0x7ffaec1626d0 2023-11-24T07:56:38.064 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:38.053+0000 7ffaeb7fe700 1 -- 172.21.15.59:0/1511249089 <== mon.1 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffad8004030 con 0x7ffaec1626d0 2023-11-24T07:56:38.064 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:38.053+0000 7ffaeb7fe700 1 -- 172.21.15.59:0/1511249089 <== mon.1 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffad80043c0 con 0x7ffaec1626d0 2023-11-24T07:56:38.065 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:38.053+0000 7ffaf3102700 1 -- 172.21.15.59:0/1511249089 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ffaec1626d0 msgr2=0x7ffaec06dd70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:56:38.065 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:38.053+0000 7ffaf3102700 1 --2- 172.21.15.59:0/1511249089 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ffaec1626d0 0x7ffaec06dd70 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7ffad8009840 tx=0x7ffad8009b50 comp rx=0 tx=0).stop 2023-11-24T07:56:38.065 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:38.053+0000 7ffaf3102700 1 -- 172.21.15.59:0/1511249089 shutdown_connections 2023-11-24T07:56:38.065 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:38.053+0000 7ffaf3102700 1 --2- 172.21.15.59:0/1511249089 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ffaec1626d0 0x7ffaec06dd70 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:56:38.065 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:38.053+0000 7ffaf3102700 1 -- 172.21.15.59:0/1511249089 >> 172.21.15.59:0/1511249089 conn(0x7ffaec0b2740 msgr2=0x7ffaec0b2b50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:56:38.065 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:38.053+0000 7ffaf3102700 1 -- 172.21.15.59:0/1511249089 shutdown_connections 2023-11-24T07:56:38.065 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:38.053+0000 7ffaf3102700 1 -- 172.21.15.59:0/1511249089 wait complete. 2023-11-24T07:56:38.066 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:38.055+0000 7ffaf3102700 1 Processor -- start 2023-11-24T07:56:38.066 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:38.055+0000 7ffaf3102700 1 -- start start 2023-11-24T07:56:38.066 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:38.055+0000 7ffaf3102700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ffaec06afd0 0x7ffaec14b430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:56:38.066 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:38.056+0000 7ffaf3102700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ffaec06e2b0 0x7ffaec0c63c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:56:38.066 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:38.056+0000 7ffaf3102700 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7ffaec1626d0 0x7ffaec0c76c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:56:38.066 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:38.056+0000 7ffaf3102700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffaec15e570 con 0x7ffaec06e2b0 2023-11-24T07:56:38.066 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:38.056+0000 7ffaf3102700 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffaec0c6e40 con 0x7ffaec06afd0 2023-11-24T07:56:38.067 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:38.056+0000 7ffaf3102700 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffaec15e810 con 0x7ffaec1626d0 2023-11-24T07:56:38.067 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:38.056+0000 7ffaebfff700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ffaec06e2b0 0x7ffaec0c63c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:56:38.067 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:38.056+0000 7ffaebfff700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ffaec06e2b0 0x7ffaec0c63c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:50284/0 (socket says 172.21.15.59:50284) 2023-11-24T07:56:38.067 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:38.056+0000 7ffaebfff700 1 -- 172.21.15.59:0/2169704841 learned_addr learned my addr 172.21.15.59:0/2169704841 (peer_addr_for_me v2:172.21.15.59:0/0) 2023-11-24T07:56:38.067 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:38.056+0000 7ffaf0e9e700 1 --2- 172.21.15.59:0/2169704841 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ffaec06afd0 0x7ffaec14b430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:56:38.067 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:38.056+0000 7ffaf169f700 1 --2- 172.21.15.59:0/2169704841 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7ffaec1626d0 0x7ffaec0c76c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:56:38.067 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:38.056+0000 7ffaebfff700 1 -- 172.21.15.59:0/2169704841 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7ffaec1626d0 msgr2=0x7ffaec0c76c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:56:38.068 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:38.056+0000 7ffaebfff700 1 --2- 172.21.15.59:0/2169704841 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7ffaec1626d0 0x7ffaec0c76c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:56:38.068 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:38.056+0000 7ffaebfff700 1 -- 172.21.15.59:0/2169704841 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ffaec06afd0 msgr2=0x7ffaec14b430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:56:38.068 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:38.056+0000 7ffaebfff700 1 --2- 172.21.15.59:0/2169704841 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ffaec06afd0 0x7ffaec14b430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:56:38.068 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:38.056+0000 7ffaebfff700 1 -- 172.21.15.59:0/2169704841 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffad80094a0 con 0x7ffaec06e2b0 2023-11-24T07:56:38.068 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:38.056+0000 7ffaf0e9e700 1 --2- 172.21.15.59:0/2169704841 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ffaec06afd0 0x7ffaec14b430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-24T07:56:38.068 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:38.056+0000 7ffaebfff700 1 --2- 172.21.15.59:0/2169704841 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ffaec06e2b0 0x7ffaec0c63c0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7ffad80090b0 tx=0x7ffad8004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:56:38.068 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:38.056+0000 7ffae97fa700 1 -- 172.21.15.59:0/2169704841 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffad8011830 con 0x7ffaec06e2b0 2023-11-24T07:56:38.069 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:38.056+0000 7ffae97fa700 1 -- 172.21.15.59:0/2169704841 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffad801f070 con 0x7ffaec06e2b0 2023-11-24T07:56:38.069 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:38.056+0000 7ffaf3102700 1 -- 172.21.15.59:0/2169704841 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffaec15eaf0 con 0x7ffaec06e2b0 2023-11-24T07:56:38.069 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:38.056+0000 7ffae97fa700 1 -- 172.21.15.59:0/2169704841 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffad8011bf0 con 0x7ffaec06e2b0 2023-11-24T07:56:38.069 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:38.056+0000 7ffaf3102700 1 -- 172.21.15.59:0/2169704841 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffaec14b970 con 0x7ffaec06e2b0 2023-11-24T07:56:38.069 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:38.057+0000 7ffaf3102700 1 -- 172.21.15.59:0/2169704841 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffaec15ed00 con 0x7ffaec06e2b0 2023-11-24T07:56:38.069 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:38.058+0000 7ffae97fa700 1 -- 172.21.15.59:0/2169704841 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7ffad8011d50 con 0x7ffaec06e2b0 2023-11-24T07:56:38.070 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:38.058+0000 7ffae97fa700 1 --2- 172.21.15.59:0/2169704841 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7ffad006aed0 0x7ffad006d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:56:38.070 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:38.058+0000 7ffae97fa700 1 -- 172.21.15.59:0/2169704841 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7ffad8021070 con 0x7ffaec06e2b0 2023-11-24T07:56:38.070 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:38.059+0000 7ffaf0e9e700 1 --2- 172.21.15.59:0/2169704841 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7ffad006aed0 0x7ffad006d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:56:38.070 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:38.059+0000 7ffaf0e9e700 1 --2- 172.21.15.59:0/2169704841 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7ffad006aed0 0x7ffad006d390 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7ffae00017a0 tx=0x7ffae0001570 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:56:38.071 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:38.063+0000 7ffae97fa700 1 -- 172.21.15.59:0/2169704841 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7ffad8056010 con 0x7ffaec06e2b0 2023-11-24T07:56:38.260 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:38.259+0000 7ffaf3102700 1 -- 172.21.15.59:0/2169704841 --> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7ffaec070320 con 0x7ffad006aed0 2023-11-24T07:56:38.268 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:38.266+0000 7ffae97fa700 1 -- 172.21.15.59:0/2169704841 <== mgr.4100 v2:172.21.15.76:6832/4264046536 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+264560 (secure 0 0 0) 0x7ffae0001d60 con 0x7ffad006aed0 2023-11-24T07:56:38.274 INFO:teuthology.orchestra.run.smithi059.stdout: 2023-11-24T07:56:38.277 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:38.269+0000 7ffaf3102700 1 -- 172.21.15.59:0/2169704841 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7ffad006aed0 msgr2=0x7ffad006d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:56:38.277 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:38.269+0000 7ffaf3102700 1 --2- 172.21.15.59:0/2169704841 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7ffad006aed0 0x7ffad006d390 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7ffae00017a0 tx=0x7ffae0001570 comp rx=0 tx=0).stop 2023-11-24T07:56:38.277 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:38.269+0000 7ffaf3102700 1 -- 172.21.15.59:0/2169704841 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ffaec06e2b0 msgr2=0x7ffaec0c63c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:56:38.277 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:38.269+0000 7ffaf3102700 1 --2- 172.21.15.59:0/2169704841 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ffaec06e2b0 0x7ffaec0c63c0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7ffad80090b0 tx=0x7ffad8004750 comp rx=0 tx=0).stop 2023-11-24T07:56:38.277 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:38.269+0000 7ffaf3102700 1 -- 172.21.15.59:0/2169704841 shutdown_connections 2023-11-24T07:56:38.277 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:38.269+0000 7ffaf3102700 1 --2- 172.21.15.59:0/2169704841 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7ffad006aed0 0x7ffad006d390 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:56:38.278 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:38.269+0000 7ffaf3102700 1 --2- 172.21.15.59:0/2169704841 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7ffaec1626d0 0x7ffaec0c76c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:56:38.278 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:38.269+0000 7ffaf3102700 1 --2- 172.21.15.59:0/2169704841 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ffaec06afd0 0x7ffaec14b430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:56:38.278 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:38.269+0000 7ffaf3102700 1 --2- 172.21.15.59:0/2169704841 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ffaec06e2b0 0x7ffaec0c63c0 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:56:38.278 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:38.269+0000 7ffaf3102700 1 -- 172.21.15.59:0/2169704841 >> 172.21.15.59:0/2169704841 conn(0x7ffaec0b2740 msgr2=0x7ffaec0ae970 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:56:38.278 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:38.269+0000 7ffaf3102700 1 -- 172.21.15.59:0/2169704841 shutdown_connections 2023-11-24T07:56:38.278 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:38.270+0000 7ffaf3102700 1 -- 172.21.15.59:0/2169704841 wait complete. 2023-11-24T07:56:38.279 INFO:teuthology.orchestra.run.smithi059.stderr:dumped all 2023-11-24T07:56:38.283 INFO:teuthology.orchestra.run.smithi059.stdout:{"pg_ready":true,"pg_map":{"version":22,"stamp":"2023-11-24T07:56:36.300499+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":219000,"kb_used_data":2824,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754755720,"statfs":{"total":773094113280,"available":772869857280,"internally_reserved":0,"allocated":2891776,"data_stored":1752836,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12745,"internal_metadata":221236791},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"10.590900"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T07:56:27.672077+0000","last_change":"2023-11-24T07:56:24.732130+0000","last_active":"2023-11-24T07:56:27.672077+0000","last_peered":"2023-11-24T07:56:27.672077+0000","last_clean":"2023-11-24T07:56:27.672077+0000","last_became_active":"2023-11-24T07:56:24.731726+0000","last_became_peered":"2023-11-24T07:56:24.731726+0000","last_unstale":"2023-11-24T07:56:27.672077+0000","last_undegraded":"2023-11-24T07:56:27.672077+0000","last_fullsized":"2023-11-24T07:56:27.672077+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:56:23.683366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:56:23.683366+0000","last_clean_scrub_stamp":"2023-11-24T07:56:23.683366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:52:22.451306+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T07:56:25.712721+0000","last_change":"2023-11-24T07:56:24.721147+0000","last_active":"2023-11-24T07:56:25.712721+0000","last_peered":"2023-11-24T07:56:25.712721+0000","last_clean":"2023-11-24T07:56:25.712721+0000","last_became_active":"2023-11-24T07:56:24.720857+0000","last_became_peered":"2023-11-24T07:56:24.720857+0000","last_unstale":"2023-11-24T07:56:25.712721+0000","last_undegraded":"2023-11-24T07:56:25.712721+0000","last_fullsized":"2023-11-24T07:56:25.712721+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:56:23.683366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:56:23.683366+0000","last_clean_scrub_stamp":"2023-11-24T07:56:23.683366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T19:48:41.181670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T07:56:25.711756+0000","last_change":"2023-11-24T07:56:24.726027+0000","last_active":"2023-11-24T07:56:25.711756+0000","last_peered":"2023-11-24T07:56:25.711756+0000","last_clean":"2023-11-24T07:56:25.711756+0000","last_became_active":"2023-11-24T07:56:24.725339+0000","last_became_peered":"2023-11-24T07:56:24.725339+0000","last_unstale":"2023-11-24T07:56:25.711756+0000","last_undegraded":"2023-11-24T07:56:25.711756+0000","last_fullsized":"2023-11-24T07:56:25.711756+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:56:23.683366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:56:23.683366+0000","last_clean_scrub_stamp":"2023-11-24T07:56:23.683366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T19:26:21.558298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T07:56:25.709012+0000","last_change":"2023-11-24T07:56:24.725132+0000","last_active":"2023-11-24T07:56:25.709012+0000","last_peered":"2023-11-24T07:56:25.709012+0000","last_clean":"2023-11-24T07:56:25.709012+0000","last_became_active":"2023-11-24T07:56:24.724817+0000","last_became_peered":"2023-11-24T07:56:24.724817+0000","last_unstale":"2023-11-24T07:56:25.709012+0000","last_undegraded":"2023-11-24T07:56:25.709012+0000","last_fullsized":"2023-11-24T07:56:25.709012+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:56:23.683366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:56:23.683366+0000","last_clean_scrub_stamp":"2023-11-24T07:56:23.683366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T19:32:53.988497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T07:56:27.671602+0000","last_change":"2023-11-24T07:56:24.722870+0000","last_active":"2023-11-24T07:56:27.671602+0000","last_peered":"2023-11-24T07:56:27.671602+0000","last_clean":"2023-11-24T07:56:27.671602+0000","last_became_active":"2023-11-24T07:56:24.722572+0000","last_became_peered":"2023-11-24T07:56:24.722572+0000","last_unstale":"2023-11-24T07:56:27.671602+0000","last_undegraded":"2023-11-24T07:56:27.671602+0000","last_fullsized":"2023-11-24T07:56:27.671602+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:56:23.683366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:56:23.683366+0000","last_clean_scrub_stamp":"2023-11-24T07:56:23.683366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T15:28:38.799061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T07:56:25.708767+0000","last_change":"2023-11-24T07:56:24.726061+0000","last_active":"2023-11-24T07:56:25.708767+0000","last_peered":"2023-11-24T07:56:25.708767+0000","last_clean":"2023-11-24T07:56:25.708767+0000","last_became_active":"2023-11-24T07:56:24.725750+0000","last_became_peered":"2023-11-24T07:56:24.725750+0000","last_unstale":"2023-11-24T07:56:25.708767+0000","last_undegraded":"2023-11-24T07:56:25.708767+0000","last_fullsized":"2023-11-24T07:56:25.708767+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:56:23.683366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:56:23.683366+0000","last_clean_scrub_stamp":"2023-11-24T07:56:23.683366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T16:22:33.714226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T07:56:27.671397+0000","last_change":"2023-11-24T07:56:24.725406+0000","last_active":"2023-11-24T07:56:27.671397+0000","last_peered":"2023-11-24T07:56:27.671397+0000","last_clean":"2023-11-24T07:56:27.671397+0000","last_became_active":"2023-11-24T07:56:24.725080+0000","last_became_peered":"2023-11-24T07:56:24.725080+0000","last_unstale":"2023-11-24T07:56:27.671397+0000","last_undegraded":"2023-11-24T07:56:27.671397+0000","last_fullsized":"2023-11-24T07:56:27.671397+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:56:23.683366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:56:23.683366+0000","last_clean_scrub_stamp":"2023-11-24T07:56:23.683366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T18:48:33.280136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T07:56:25.711711+0000","last_change":"2023-11-24T07:56:24.719072+0000","last_active":"2023-11-24T07:56:25.711711+0000","last_peered":"2023-11-24T07:56:25.711711+0000","last_clean":"2023-11-24T07:56:25.711711+0000","last_became_active":"2023-11-24T07:56:24.718791+0000","last_became_peered":"2023-11-24T07:56:24.718791+0000","last_unstale":"2023-11-24T07:56:25.711711+0000","last_undegraded":"2023-11-24T07:56:25.711711+0000","last_fullsized":"2023-11-24T07:56:25.711711+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:56:23.683366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:56:23.683366+0000","last_clean_scrub_stamp":"2023-11-24T07:56:23.683366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:13:16.865275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T07:56:27.783015+0000","last_change":"2023-11-24T07:56:24.743409+0000","last_active":"2023-11-24T07:56:27.783015+0000","last_peered":"2023-11-24T07:56:27.783015+0000","last_clean":"2023-11-24T07:56:27.783015+0000","last_became_active":"2023-11-24T07:56:24.742988+0000","last_became_peered":"2023-11-24T07:56:24.742988+0000","last_unstale":"2023-11-24T07:56:27.783015+0000","last_undegraded":"2023-11-24T07:56:27.783015+0000","last_fullsized":"2023-11-24T07:56:27.783015+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:56:23.683366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:56:23.683366+0000","last_clean_scrub_stamp":"2023-11-24T07:56:23.683366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T09:01:29.634179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T07:56:27.672028+0000","last_change":"2023-11-24T07:56:24.718066+0000","last_active":"2023-11-24T07:56:27.672028+0000","last_peered":"2023-11-24T07:56:27.672028+0000","last_clean":"2023-11-24T07:56:27.672028+0000","last_became_active":"2023-11-24T07:56:24.715725+0000","last_became_peered":"2023-11-24T07:56:24.715725+0000","last_unstale":"2023-11-24T07:56:27.672028+0000","last_undegraded":"2023-11-24T07:56:27.672028+0000","last_fullsized":"2023-11-24T07:56:27.672028+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:56:23.683366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:56:23.683366+0000","last_clean_scrub_stamp":"2023-11-24T07:56:23.683366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T13:56:07.967730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T07:56:27.671765+0000","last_change":"2023-11-24T07:56:24.730488+0000","last_active":"2023-11-24T07:56:27.671765+0000","last_peered":"2023-11-24T07:56:27.671765+0000","last_clean":"2023-11-24T07:56:27.671765+0000","last_became_active":"2023-11-24T07:56:24.730171+0000","last_became_peered":"2023-11-24T07:56:24.730171+0000","last_unstale":"2023-11-24T07:56:27.671765+0000","last_undegraded":"2023-11-24T07:56:27.671765+0000","last_fullsized":"2023-11-24T07:56:27.671765+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:56:23.683366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:56:23.683366+0000","last_clean_scrub_stamp":"2023-11-24T07:56:23.683366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T19:29:50.373581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T07:56:27.671956+0000","last_change":"2023-11-24T07:56:24.734314+0000","last_active":"2023-11-24T07:56:27.671956+0000","last_peered":"2023-11-24T07:56:27.671956+0000","last_clean":"2023-11-24T07:56:27.671956+0000","last_became_active":"2023-11-24T07:56:24.733966+0000","last_became_peered":"2023-11-24T07:56:24.733966+0000","last_unstale":"2023-11-24T07:56:27.671956+0000","last_undegraded":"2023-11-24T07:56:27.671956+0000","last_fullsized":"2023-11-24T07:56:27.671956+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:56:23.683366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:56:23.683366+0000","last_clean_scrub_stamp":"2023-11-24T07:56:23.683366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T19:39:29.828696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T07:56:27.782981+0000","last_change":"2023-11-24T07:56:24.717337+0000","last_active":"2023-11-24T07:56:27.782981+0000","last_peered":"2023-11-24T07:56:27.782981+0000","last_clean":"2023-11-24T07:56:27.782981+0000","last_became_active":"2023-11-24T07:56:24.716929+0000","last_became_peered":"2023-11-24T07:56:24.716929+0000","last_unstale":"2023-11-24T07:56:27.782981+0000","last_undegraded":"2023-11-24T07:56:27.782981+0000","last_fullsized":"2023-11-24T07:56:27.782981+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:56:23.683366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:56:23.683366+0000","last_clean_scrub_stamp":"2023-11-24T07:56:23.683366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T17:36:14.367398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T07:56:27.783087+0000","last_change":"2023-11-24T07:56:24.720081+0000","last_active":"2023-11-24T07:56:27.783087+0000","last_peered":"2023-11-24T07:56:27.783087+0000","last_clean":"2023-11-24T07:56:27.783087+0000","last_became_active":"2023-11-24T07:56:24.719776+0000","last_became_peered":"2023-11-24T07:56:24.719776+0000","last_unstale":"2023-11-24T07:56:27.783087+0000","last_undegraded":"2023-11-24T07:56:27.783087+0000","last_fullsized":"2023-11-24T07:56:27.783087+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:56:23.683366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:56:23.683366+0000","last_clean_scrub_stamp":"2023-11-24T07:56:23.683366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T19:43:40.193280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T07:56:25.712435+0000","last_change":"2023-11-24T07:56:24.719400+0000","last_active":"2023-11-24T07:56:25.712435+0000","last_peered":"2023-11-24T07:56:25.712435+0000","last_clean":"2023-11-24T07:56:25.712435+0000","last_became_active":"2023-11-24T07:56:24.719095+0000","last_became_peered":"2023-11-24T07:56:24.719095+0000","last_unstale":"2023-11-24T07:56:25.712435+0000","last_undegraded":"2023-11-24T07:56:25.712435+0000","last_fullsized":"2023-11-24T07:56:25.712435+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:56:23.683366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:56:23.683366+0000","last_clean_scrub_stamp":"2023-11-24T07:56:23.683366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T19:29:50.373581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T07:56:26.100997+0000","last_change":"2023-11-24T07:56:24.725927+0000","last_active":"2023-11-24T07:56:26.100997+0000","last_peered":"2023-11-24T07:56:26.100997+0000","last_clean":"2023-11-24T07:56:26.100997+0000","last_became_active":"2023-11-24T07:56:24.725568+0000","last_became_peered":"2023-11-24T07:56:24.725568+0000","last_unstale":"2023-11-24T07:56:26.100997+0000","last_undegraded":"2023-11-24T07:56:26.100997+0000","last_fullsized":"2023-11-24T07:56:26.100997+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:56:23.683366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:56:23.683366+0000","last_clean_scrub_stamp":"2023-11-24T07:56:23.683366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T19:39:29.828696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T07:56:26.101717+0000","last_change":"2023-11-24T07:56:24.730986+0000","last_active":"2023-11-24T07:56:26.101717+0000","last_peered":"2023-11-24T07:56:26.101717+0000","last_clean":"2023-11-24T07:56:26.101717+0000","last_became_active":"2023-11-24T07:56:24.730564+0000","last_became_peered":"2023-11-24T07:56:24.730564+0000","last_unstale":"2023-11-24T07:56:26.101717+0000","last_undegraded":"2023-11-24T07:56:26.101717+0000","last_fullsized":"2023-11-24T07:56:26.101717+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:56:23.683366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:56:23.683366+0000","last_clean_scrub_stamp":"2023-11-24T07:56:23.683366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T14:48:19.320894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T07:56:27.783028+0000","last_change":"2023-11-24T07:56:24.716131+0000","last_active":"2023-11-24T07:56:27.783028+0000","last_peered":"2023-11-24T07:56:27.783028+0000","last_clean":"2023-11-24T07:56:27.783028+0000","last_became_active":"2023-11-24T07:56:24.715770+0000","last_became_peered":"2023-11-24T07:56:24.715770+0000","last_unstale":"2023-11-24T07:56:27.783028+0000","last_undegraded":"2023-11-24T07:56:27.783028+0000","last_fullsized":"2023-11-24T07:56:27.783028+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:56:23.683366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:56:23.683366+0000","last_clean_scrub_stamp":"2023-11-24T07:56:23.683366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T16:22:33.714226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T07:56:26.101413+0000","last_change":"2023-11-24T07:56:24.726104+0000","last_active":"2023-11-24T07:56:26.101413+0000","last_peered":"2023-11-24T07:56:26.101413+0000","last_clean":"2023-11-24T07:56:26.101413+0000","last_became_active":"2023-11-24T07:56:24.725718+0000","last_became_peered":"2023-11-24T07:56:24.725718+0000","last_unstale":"2023-11-24T07:56:26.101413+0000","last_undegraded":"2023-11-24T07:56:26.101413+0000","last_fullsized":"2023-11-24T07:56:26.101413+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:56:23.683366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:56:23.683366+0000","last_clean_scrub_stamp":"2023-11-24T07:56:23.683366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T15:24:09.050735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"19'4","reported_seq":16,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-24T07:56:24.743886+0000","last_change":"2023-11-24T07:56:23.714550+0000","last_active":"2023-11-24T07:56:24.743886+0000","last_peered":"2023-11-24T07:56:24.743886+0000","last_clean":"2023-11-24T07:56:24.743886+0000","last_became_active":"2023-11-24T07:56:23.714275+0000","last_became_peered":"2023-11-24T07:56:23.714275+0000","last_unstale":"2023-11-24T07:56:24.743886+0000","last_undegraded":"2023-11-24T07:56:24.743886+0000","last_fullsized":"2023-11-24T07:56:24.743886+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:56:22.678019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:56:22.678019+0000","last_clean_scrub_stamp":"2023-11-24T07:56:22.678019+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T19:05:26.056275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T07:56:25.712605+0000","last_change":"2023-11-24T07:56:24.722991+0000","last_active":"2023-11-24T07:56:25.712605+0000","last_peered":"2023-11-24T07:56:25.712605+0000","last_clean":"2023-11-24T07:56:25.712605+0000","last_became_active":"2023-11-24T07:56:24.722711+0000","last_became_peered":"2023-11-24T07:56:24.722711+0000","last_unstale":"2023-11-24T07:56:25.712605+0000","last_undegraded":"2023-11-24T07:56:25.712605+0000","last_fullsized":"2023-11-24T07:56:25.712605+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:56:23.683366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:56:23.683366+0000","last_clean_scrub_stamp":"2023-11-24T07:56:23.683366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T12:41:22.436958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1e","version":"19'1","reported_seq":13,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-24T07:56:24.739921+0000","last_change":"2023-11-24T07:56:23.702219+0000","last_active":"2023-11-24T07:56:24.739921+0000","last_peered":"2023-11-24T07:56:24.739921+0000","last_clean":"2023-11-24T07:56:24.739921+0000","last_became_active":"2023-11-24T07:56:23.701922+0000","last_became_peered":"2023-11-24T07:56:23.701922+0000","last_unstale":"2023-11-24T07:56:24.739921+0000","last_undegraded":"2023-11-24T07:56:24.739921+0000","last_fullsized":"2023-11-24T07:56:24.739921+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:56:22.678019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:56:22.678019+0000","last_clean_scrub_stamp":"2023-11-24T07:56:22.678019+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T09:01:28.628833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T07:56:25.712243+0000","last_change":"2023-11-24T07:56:24.733713+0000","last_active":"2023-11-24T07:56:25.712243+0000","last_peered":"2023-11-24T07:56:25.712243+0000","last_clean":"2023-11-24T07:56:25.712243+0000","last_became_active":"2023-11-24T07:56:24.733302+0000","last_became_peered":"2023-11-24T07:56:24.733302+0000","last_unstale":"2023-11-24T07:56:25.712243+0000","last_undegraded":"2023-11-24T07:56:25.712243+0000","last_fullsized":"2023-11-24T07:56:25.712243+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:56:23.683366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:56:23.683366+0000","last_clean_scrub_stamp":"2023-11-24T07:56:23.683366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T15:28:38.799061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1d","version":"19'2","reported_seq":14,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-24T07:56:24.740403+0000","last_change":"2023-11-24T07:56:23.707439+0000","last_active":"2023-11-24T07:56:24.740403+0000","last_peered":"2023-11-24T07:56:24.740403+0000","last_clean":"2023-11-24T07:56:24.740403+0000","last_became_active":"2023-11-24T07:56:23.707136+0000","last_became_peered":"2023-11-24T07:56:23.707136+0000","last_unstale":"2023-11-24T07:56:24.740403+0000","last_undegraded":"2023-11-24T07:56:24.740403+0000","last_fullsized":"2023-11-24T07:56:24.740403+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:56:22.678019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:56:22.678019+0000","last_clean_scrub_stamp":"2023-11-24T07:56:22.678019+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T19:43:39.187934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T07:56:27.782857+0000","last_change":"2023-11-24T07:56:24.733325+0000","last_active":"2023-11-24T07:56:27.782857+0000","last_peered":"2023-11-24T07:56:27.782857+0000","last_clean":"2023-11-24T07:56:27.782857+0000","last_became_active":"2023-11-24T07:56:24.733014+0000","last_became_peered":"2023-11-24T07:56:24.733014+0000","last_unstale":"2023-11-24T07:56:27.782857+0000","last_undegraded":"2023-11-24T07:56:27.782857+0000","last_fullsized":"2023-11-24T07:56:27.782857+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:56:23.683366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:56:23.683366+0000","last_clean_scrub_stamp":"2023-11-24T07:56:23.683366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T19:43:40.193280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"19'2","reported_seq":14,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-24T07:56:24.740415+0000","last_change":"2023-11-24T07:56:23.709116+0000","last_active":"2023-11-24T07:56:24.740415+0000","last_peered":"2023-11-24T07:56:24.740415+0000","last_clean":"2023-11-24T07:56:24.740415+0000","last_became_active":"2023-11-24T07:56:23.708703+0000","last_became_peered":"2023-11-24T07:56:23.708703+0000","last_unstale":"2023-11-24T07:56:24.740415+0000","last_undegraded":"2023-11-24T07:56:24.740415+0000","last_fullsized":"2023-11-24T07:56:24.740415+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:56:22.678019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:56:22.678019+0000","last_clean_scrub_stamp":"2023-11-24T07:56:22.678019+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T09:01:28.628833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T07:56:26.101403+0000","last_change":"2023-11-24T07:56:24.731750+0000","last_active":"2023-11-24T07:56:26.101403+0000","last_peered":"2023-11-24T07:56:26.101403+0000","last_clean":"2023-11-24T07:56:26.101403+0000","last_became_active":"2023-11-24T07:56:24.730636+0000","last_became_peered":"2023-11-24T07:56:24.730636+0000","last_unstale":"2023-11-24T07:56:26.101403+0000","last_undegraded":"2023-11-24T07:56:26.101403+0000","last_fullsized":"2023-11-24T07:56:26.101403+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:56:23.683366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:56:23.683366+0000","last_clean_scrub_stamp":"2023-11-24T07:56:23.683366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T15:11:49.015783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1b","version":"19'1","reported_seq":13,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-24T07:56:24.737950+0000","last_change":"2023-11-24T07:56:23.702767+0000","last_active":"2023-11-24T07:56:24.737950+0000","last_peered":"2023-11-24T07:56:24.737950+0000","last_clean":"2023-11-24T07:56:24.737950+0000","last_became_active":"2023-11-24T07:56:23.702499+0000","last_became_peered":"2023-11-24T07:56:23.702499+0000","last_unstale":"2023-11-24T07:56:24.737950+0000","last_undegraded":"2023-11-24T07:56:24.737950+0000","last_fullsized":"2023-11-24T07:56:24.737950+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:56:22.678019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:56:22.678019+0000","last_clean_scrub_stamp":"2023-11-24T07:56:22.678019+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T17:36:13.362051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T07:56:27.782679+0000","last_change":"2023-11-24T07:56:24.735158+0000","last_active":"2023-11-24T07:56:27.782679+0000","last_peered":"2023-11-24T07:56:27.782679+0000","last_clean":"2023-11-24T07:56:27.782679+0000","last_became_active":"2023-11-24T07:56:24.734600+0000","last_became_peered":"2023-11-24T07:56:24.734600+0000","last_unstale":"2023-11-24T07:56:27.782679+0000","last_undegraded":"2023-11-24T07:56:27.782679+0000","last_fullsized":"2023-11-24T07:56:27.782679+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:56:23.683366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:56:23.683366+0000","last_clean_scrub_stamp":"2023-11-24T07:56:23.683366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T17:36:14.367398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1a","version":"19'1","reported_seq":13,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-24T07:56:24.743591+0000","last_change":"2023-11-24T07:56:23.712864+0000","last_active":"2023-11-24T07:56:24.743591+0000","last_peered":"2023-11-24T07:56:24.743591+0000","last_clean":"2023-11-24T07:56:24.743591+0000","last_became_active":"2023-11-24T07:56:23.712559+0000","last_became_peered":"2023-11-24T07:56:23.712559+0000","last_unstale":"2023-11-24T07:56:24.743591+0000","last_undegraded":"2023-11-24T07:56:24.743591+0000","last_fullsized":"2023-11-24T07:56:24.743591+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:56:22.678019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:56:22.678019+0000","last_clean_scrub_stamp":"2023-11-24T07:56:22.678019+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:19:21.093151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T07:56:25.712365+0000","last_change":"2023-11-24T07:56:24.727143+0000","last_active":"2023-11-24T07:56:25.712365+0000","last_peered":"2023-11-24T07:56:25.712365+0000","last_clean":"2023-11-24T07:56:25.712365+0000","last_became_active":"2023-11-24T07:56:24.726637+0000","last_became_peered":"2023-11-24T07:56:24.726637+0000","last_unstale":"2023-11-24T07:56:25.712365+0000","last_undegraded":"2023-11-24T07:56:25.712365+0000","last_fullsized":"2023-11-24T07:56:25.712365+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:56:23.683366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:56:23.683366+0000","last_clean_scrub_stamp":"2023-11-24T07:56:23.683366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T15:28:38.799061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":12,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-24T07:56:24.710275+0000","last_change":"2023-11-24T07:56:23.707863+0000","last_active":"2023-11-24T07:56:24.710275+0000","last_peered":"2023-11-24T07:56:24.710275+0000","last_clean":"2023-11-24T07:56:24.710275+0000","last_became_active":"2023-11-24T07:56:23.707595+0000","last_became_peered":"2023-11-24T07:56:23.707595+0000","last_unstale":"2023-11-24T07:56:24.710275+0000","last_undegraded":"2023-11-24T07:56:24.710275+0000","last_fullsized":"2023-11-24T07:56:24.710275+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:56:22.678019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:56:22.678019+0000","last_clean_scrub_stamp":"2023-11-24T07:56:22.678019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T11:00:03.947675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T07:56:25.712360+0000","last_change":"2023-11-24T07:56:24.721105+0000","last_active":"2023-11-24T07:56:25.712360+0000","last_peered":"2023-11-24T07:56:25.712360+0000","last_clean":"2023-11-24T07:56:25.712360+0000","last_became_active":"2023-11-24T07:56:24.720813+0000","last_became_peered":"2023-11-24T07:56:24.720813+0000","last_unstale":"2023-11-24T07:56:25.712360+0000","last_undegraded":"2023-11-24T07:56:25.712360+0000","last_fullsized":"2023-11-24T07:56:25.712360+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:56:23.683366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:56:23.683366+0000","last_clean_scrub_stamp":"2023-11-24T07:56:23.683366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T17:43:47.155869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":12,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-24T07:56:24.719929+0000","last_change":"2023-11-24T07:56:23.704771+0000","last_active":"2023-11-24T07:56:24.719929+0000","last_peered":"2023-11-24T07:56:24.719929+0000","last_clean":"2023-11-24T07:56:24.719929+0000","last_became_active":"2023-11-24T07:56:23.704433+0000","last_became_peered":"2023-11-24T07:56:23.704433+0000","last_unstale":"2023-11-24T07:56:24.719929+0000","last_undegraded":"2023-11-24T07:56:24.719929+0000","last_fullsized":"2023-11-24T07:56:24.719929+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:56:22.678019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:56:22.678019+0000","last_clean_scrub_stamp":"2023-11-24T07:56:22.678019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:14:32.121393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T07:56:25.711836+0000","last_change":"2023-11-24T07:56:24.734742+0000","last_active":"2023-11-24T07:56:25.711836+0000","last_peered":"2023-11-24T07:56:25.711836+0000","last_clean":"2023-11-24T07:56:25.711836+0000","last_became_active":"2023-11-24T07:56:24.734447+0000","last_became_peered":"2023-11-24T07:56:24.734447+0000","last_unstale":"2023-11-24T07:56:25.711836+0000","last_undegraded":"2023-11-24T07:56:25.711836+0000","last_fullsized":"2023-11-24T07:56:25.711836+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:56:23.683366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:56:23.683366+0000","last_clean_scrub_stamp":"2023-11-24T07:56:23.683366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T19:26:21.558298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":12,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-24T07:56:24.729505+0000","last_change":"2023-11-24T07:56:23.701888+0000","last_active":"2023-11-24T07:56:24.729505+0000","last_peered":"2023-11-24T07:56:24.729505+0000","last_clean":"2023-11-24T07:56:24.729505+0000","last_became_active":"2023-11-24T07:56:23.701611+0000","last_became_peered":"2023-11-24T07:56:23.701611+0000","last_unstale":"2023-11-24T07:56:24.729505+0000","last_undegraded":"2023-11-24T07:56:24.729505+0000","last_fullsized":"2023-11-24T07:56:24.729505+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:56:22.678019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:56:22.678019+0000","last_clean_scrub_stamp":"2023-11-24T07:56:22.678019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T18:53:13.845460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T07:56:25.709371+0000","last_change":"2023-11-24T07:56:24.731116+0000","last_active":"2023-11-24T07:56:25.709371+0000","last_peered":"2023-11-24T07:56:25.709371+0000","last_clean":"2023-11-24T07:56:25.709371+0000","last_became_active":"2023-11-24T07:56:24.730805+0000","last_became_peered":"2023-11-24T07:56:24.730805+0000","last_unstale":"2023-11-24T07:56:25.709371+0000","last_undegraded":"2023-11-24T07:56:25.709371+0000","last_fullsized":"2023-11-24T07:56:25.709371+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:56:23.683366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:56:23.683366+0000","last_clean_scrub_stamp":"2023-11-24T07:56:23.683366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T19:05:27.061622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":12,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-24T07:56:24.712027+0000","last_change":"2023-11-24T07:56:23.707470+0000","last_active":"2023-11-24T07:56:24.712027+0000","last_peered":"2023-11-24T07:56:24.712027+0000","last_clean":"2023-11-24T07:56:24.712027+0000","last_became_active":"2023-11-24T07:56:23.707194+0000","last_became_peered":"2023-11-24T07:56:23.707194+0000","last_unstale":"2023-11-24T07:56:24.712027+0000","last_undegraded":"2023-11-24T07:56:24.712027+0000","last_fullsized":"2023-11-24T07:56:24.712027+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:56:22.678019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:56:22.678019+0000","last_clean_scrub_stamp":"2023-11-24T07:56:22.678019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T19:26:20.552951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T07:56:25.712327+0000","last_change":"2023-11-24T07:56:24.733042+0000","last_active":"2023-11-24T07:56:25.712327+0000","last_peered":"2023-11-24T07:56:25.712327+0000","last_clean":"2023-11-24T07:56:25.712327+0000","last_became_active":"2023-11-24T07:56:24.732746+0000","last_became_peered":"2023-11-24T07:56:24.732746+0000","last_unstale":"2023-11-24T07:56:25.712327+0000","last_undegraded":"2023-11-24T07:56:25.712327+0000","last_fullsized":"2023-11-24T07:56:25.712327+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:56:23.683366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:56:23.683366+0000","last_clean_scrub_stamp":"2023-11-24T07:56:23.683366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:19:22.098498+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.15","version":"19'3","reported_seq":15,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-24T07:56:24.739732+0000","last_change":"2023-11-24T07:56:23.703208+0000","last_active":"2023-11-24T07:56:24.739732+0000","last_peered":"2023-11-24T07:56:24.739732+0000","last_clean":"2023-11-24T07:56:24.739732+0000","last_became_active":"2023-11-24T07:56:23.702891+0000","last_became_peered":"2023-11-24T07:56:23.702891+0000","last_unstale":"2023-11-24T07:56:24.739732+0000","last_undegraded":"2023-11-24T07:56:24.739732+0000","last_fullsized":"2023-11-24T07:56:24.739732+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:56:22.678019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:56:22.678019+0000","last_clean_scrub_stamp":"2023-11-24T07:56:22.678019+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T08:42:21.735516+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T07:56:26.101263+0000","last_change":"2023-11-24T07:56:24.729010+0000","last_active":"2023-11-24T07:56:26.101263+0000","last_peered":"2023-11-24T07:56:26.101263+0000","last_clean":"2023-11-24T07:56:26.101263+0000","last_became_active":"2023-11-24T07:56:24.728711+0000","last_became_peered":"2023-11-24T07:56:24.728711+0000","last_unstale":"2023-11-24T07:56:26.101263+0000","last_undegraded":"2023-11-24T07:56:26.101263+0000","last_fullsized":"2023-11-24T07:56:26.101263+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:56:23.683366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:56:23.683366+0000","last_clean_scrub_stamp":"2023-11-24T07:56:23.683366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T17:48:28.079331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.14","version":"19'6","reported_seq":18,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-24T07:56:24.743066+0000","last_change":"2023-11-24T07:56:23.706638+0000","last_active":"2023-11-24T07:56:24.743066+0000","last_peered":"2023-11-24T07:56:24.743066+0000","last_clean":"2023-11-24T07:56:24.743066+0000","last_became_active":"2023-11-24T07:56:23.706357+0000","last_became_peered":"2023-11-24T07:56:23.706357+0000","last_unstale":"2023-11-24T07:56:24.743066+0000","last_undegraded":"2023-11-24T07:56:24.743066+0000","last_fullsized":"2023-11-24T07:56:24.743066+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:56:22.678019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:56:22.678019+0000","last_clean_scrub_stamp":"2023-11-24T07:56:22.678019+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T14:40:38.179620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T07:56:27.671894+0000","last_change":"2023-11-24T07:56:24.717150+0000","last_active":"2023-11-24T07:56:27.671894+0000","last_peered":"2023-11-24T07:56:27.671894+0000","last_clean":"2023-11-24T07:56:27.671894+0000","last_became_active":"2023-11-24T07:56:24.716723+0000","last_became_peered":"2023-11-24T07:56:24.716723+0000","last_unstale":"2023-11-24T07:56:27.671894+0000","last_undegraded":"2023-11-24T07:56:27.671894+0000","last_fullsized":"2023-11-24T07:56:27.671894+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:56:23.683366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:56:23.683366+0000","last_clean_scrub_stamp":"2023-11-24T07:56:23.683366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T12:41:22.436958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.13","version":"19'2","reported_seq":14,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-24T07:56:24.739597+0000","last_change":"2023-11-24T07:56:23.707817+0000","last_active":"2023-11-24T07:56:24.739597+0000","last_peered":"2023-11-24T07:56:24.739597+0000","last_clean":"2023-11-24T07:56:24.739597+0000","last_became_active":"2023-11-24T07:56:23.707511+0000","last_became_peered":"2023-11-24T07:56:23.707511+0000","last_unstale":"2023-11-24T07:56:24.739597+0000","last_undegraded":"2023-11-24T07:56:24.739597+0000","last_fullsized":"2023-11-24T07:56:24.739597+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:56:22.678019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:56:22.678019+0000","last_clean_scrub_stamp":"2023-11-24T07:56:22.678019+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T15:03:38.755757+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T07:56:27.783173+0000","last_change":"2023-11-24T07:56:24.730834+0000","last_active":"2023-11-24T07:56:27.783173+0000","last_peered":"2023-11-24T07:56:27.783173+0000","last_clean":"2023-11-24T07:56:27.783173+0000","last_became_active":"2023-11-24T07:56:24.730545+0000","last_became_peered":"2023-11-24T07:56:24.730545+0000","last_unstale":"2023-11-24T07:56:27.783173+0000","last_undegraded":"2023-11-24T07:56:27.783173+0000","last_fullsized":"2023-11-24T07:56:27.783173+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:56:23.683366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:56:23.683366+0000","last_clean_scrub_stamp":"2023-11-24T07:56:23.683366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T18:20:49.373891+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":12,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-24T07:56:24.723510+0000","last_change":"2023-11-24T07:56:23.712182+0000","last_active":"2023-11-24T07:56:24.723510+0000","last_peered":"2023-11-24T07:56:24.723510+0000","last_clean":"2023-11-24T07:56:24.723510+0000","last_became_active":"2023-11-24T07:56:23.711814+0000","last_became_peered":"2023-11-24T07:56:23.711814+0000","last_unstale":"2023-11-24T07:56:24.723510+0000","last_undegraded":"2023-11-24T07:56:24.723510+0000","last_fullsized":"2023-11-24T07:56:24.723510+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:56:22.678019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:56:22.678019+0000","last_clean_scrub_stamp":"2023-11-24T07:56:22.678019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T15:43:53.652738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T07:56:26.101323+0000","last_change":"2023-11-24T07:56:24.730497+0000","last_active":"2023-11-24T07:56:26.101323+0000","last_peered":"2023-11-24T07:56:26.101323+0000","last_clean":"2023-11-24T07:56:26.101323+0000","last_became_active":"2023-11-24T07:56:24.730178+0000","last_became_peered":"2023-11-24T07:56:24.730178+0000","last_unstale":"2023-11-24T07:56:26.101323+0000","last_undegraded":"2023-11-24T07:56:26.101323+0000","last_fullsized":"2023-11-24T07:56:26.101323+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:56:23.683366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:56:23.683366+0000","last_clean_scrub_stamp":"2023-11-24T07:56:23.683366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T16:27:17.883314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":12,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-24T07:56:24.711899+0000","last_change":"2023-11-24T07:56:23.713877+0000","last_active":"2023-11-24T07:56:24.711899+0000","last_peered":"2023-11-24T07:56:24.711899+0000","last_clean":"2023-11-24T07:56:24.711899+0000","last_became_active":"2023-11-24T07:56:23.713603+0000","last_became_peered":"2023-11-24T07:56:23.713603+0000","last_unstale":"2023-11-24T07:56:24.711899+0000","last_undegraded":"2023-11-24T07:56:24.711899+0000","last_fullsized":"2023-11-24T07:56:24.711899+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:56:22.678019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:56:22.678019+0000","last_clean_scrub_stamp":"2023-11-24T07:56:22.678019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:13:15.859928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T07:56:27.782480+0000","last_change":"2023-11-24T07:56:24.733613+0000","last_active":"2023-11-24T07:56:27.782480+0000","last_peered":"2023-11-24T07:56:27.782480+0000","last_clean":"2023-11-24T07:56:27.782480+0000","last_became_active":"2023-11-24T07:56:24.733361+0000","last_became_peered":"2023-11-24T07:56:24.733361+0000","last_unstale":"2023-11-24T07:56:27.782480+0000","last_undegraded":"2023-11-24T07:56:27.782480+0000","last_fullsized":"2023-11-24T07:56:27.782480+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:56:23.683366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:56:23.683366+0000","last_clean_scrub_stamp":"2023-11-24T07:56:23.683366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T16:22:33.714226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.10","version":"19'8","reported_seq":20,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-24T07:56:24.740763+0000","last_change":"2023-11-24T07:56:23.701319+0000","last_active":"2023-11-24T07:56:24.740763+0000","last_peered":"2023-11-24T07:56:24.740763+0000","last_clean":"2023-11-24T07:56:24.740763+0000","last_became_active":"2023-11-24T07:56:23.700995+0000","last_became_peered":"2023-11-24T07:56:23.700995+0000","last_unstale":"2023-11-24T07:56:24.740763+0000","last_undegraded":"2023-11-24T07:56:24.740763+0000","last_fullsized":"2023-11-24T07:56:24.740763+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:56:22.678019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:56:22.678019+0000","last_clean_scrub_stamp":"2023-11-24T07:56:22.678019+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T19:43:39.187934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T07:56:26.101104+0000","last_change":"2023-11-24T07:56:24.730591+0000","last_active":"2023-11-24T07:56:26.101104+0000","last_peered":"2023-11-24T07:56:26.101104+0000","last_clean":"2023-11-24T07:56:26.101104+0000","last_became_active":"2023-11-24T07:56:24.730281+0000","last_became_peered":"2023-11-24T07:56:24.730281+0000","last_unstale":"2023-11-24T07:56:26.101104+0000","last_undegraded":"2023-11-24T07:56:26.101104+0000","last_fullsized":"2023-11-24T07:56:26.101104+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:56:23.683366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:56:23.683366+0000","last_clean_scrub_stamp":"2023-11-24T07:56:23.683366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:52:22.451306+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.f","version":"19'1","reported_seq":13,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-24T07:56:24.739965+0000","last_change":"2023-11-24T07:56:23.708274+0000","last_active":"2023-11-24T07:56:24.739965+0000","last_peered":"2023-11-24T07:56:24.739965+0000","last_clean":"2023-11-24T07:56:24.739965+0000","last_became_active":"2023-11-24T07:56:23.708001+0000","last_became_peered":"2023-11-24T07:56:23.708001+0000","last_unstale":"2023-11-24T07:56:24.739965+0000","last_undegraded":"2023-11-24T07:56:24.739965+0000","last_fullsized":"2023-11-24T07:56:24.739965+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:56:22.678019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:56:22.678019+0000","last_clean_scrub_stamp":"2023-11-24T07:56:22.678019+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T09:29:56.578230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T07:56:25.709280+0000","last_change":"2023-11-24T07:56:24.720989+0000","last_active":"2023-11-24T07:56:25.709280+0000","last_peered":"2023-11-24T07:56:25.709280+0000","last_clean":"2023-11-24T07:56:25.709280+0000","last_became_active":"2023-11-24T07:56:24.720500+0000","last_became_peered":"2023-11-24T07:56:24.720500+0000","last_unstale":"2023-11-24T07:56:25.709280+0000","last_undegraded":"2023-11-24T07:56:25.709280+0000","last_fullsized":"2023-11-24T07:56:25.709280+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:56:23.683366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:56:23.683366+0000","last_clean_scrub_stamp":"2023-11-24T07:56:23.683366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T09:01:29.634179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":12,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-24T07:56:24.714465+0000","last_change":"2023-11-24T07:56:23.704585+0000","last_active":"2023-11-24T07:56:24.714465+0000","last_peered":"2023-11-24T07:56:24.714465+0000","last_clean":"2023-11-24T07:56:24.714465+0000","last_became_active":"2023-11-24T07:56:23.704316+0000","last_became_peered":"2023-11-24T07:56:23.704316+0000","last_unstale":"2023-11-24T07:56:24.714465+0000","last_undegraded":"2023-11-24T07:56:24.714465+0000","last_fullsized":"2023-11-24T07:56:24.714465+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:56:22.678019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:56:22.678019+0000","last_clean_scrub_stamp":"2023-11-24T07:56:22.678019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T18:53:13.845460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T07:56:25.709031+0000","last_change":"2023-11-24T07:56:24.718689+0000","last_active":"2023-11-24T07:56:25.709031+0000","last_peered":"2023-11-24T07:56:25.709031+0000","last_clean":"2023-11-24T07:56:25.709031+0000","last_became_active":"2023-11-24T07:56:24.717862+0000","last_became_peered":"2023-11-24T07:56:24.717862+0000","last_unstale":"2023-11-24T07:56:25.709031+0000","last_undegraded":"2023-11-24T07:56:25.709031+0000","last_fullsized":"2023-11-24T07:56:25.709031+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:56:23.683366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:56:23.683366+0000","last_clean_scrub_stamp":"2023-11-24T07:56:23.683366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T17:36:14.367398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":12,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-24T07:56:24.723355+0000","last_change":"2023-11-24T07:56:23.712764+0000","last_active":"2023-11-24T07:56:24.723355+0000","last_peered":"2023-11-24T07:56:24.723355+0000","last_clean":"2023-11-24T07:56:24.723355+0000","last_became_active":"2023-11-24T07:56:23.712448+0000","last_became_peered":"2023-11-24T07:56:23.712448+0000","last_unstale":"2023-11-24T07:56:24.723355+0000","last_undegraded":"2023-11-24T07:56:24.723355+0000","last_fullsized":"2023-11-24T07:56:24.723355+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:56:22.678019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:56:22.678019+0000","last_clean_scrub_stamp":"2023-11-24T07:56:22.678019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T19:29:49.368234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T07:56:27.671634+0000","last_change":"2023-11-24T07:56:24.716237+0000","last_active":"2023-11-24T07:56:27.671634+0000","last_peered":"2023-11-24T07:56:27.671634+0000","last_clean":"2023-11-24T07:56:27.671634+0000","last_became_active":"2023-11-24T07:56:24.715897+0000","last_became_peered":"2023-11-24T07:56:24.715897+0000","last_unstale":"2023-11-24T07:56:27.671634+0000","last_undegraded":"2023-11-24T07:56:27.671634+0000","last_fullsized":"2023-11-24T07:56:27.671634+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:56:23.683366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:56:23.683366+0000","last_clean_scrub_stamp":"2023-11-24T07:56:23.683366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:19:22.098498+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T07:56:25.711564+0000","last_change":"2023-11-24T07:56:24.723045+0000","last_active":"2023-11-24T07:56:25.711564+0000","last_peered":"2023-11-24T07:56:25.711564+0000","last_clean":"2023-11-24T07:56:25.711564+0000","last_became_active":"2023-11-24T07:56:24.722669+0000","last_became_peered":"2023-11-24T07:56:24.722669+0000","last_unstale":"2023-11-24T07:56:25.711564+0000","last_undegraded":"2023-11-24T07:56:25.711564+0000","last_fullsized":"2023-11-24T07:56:25.711564+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:56:23.683366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:56:23.683366+0000","last_clean_scrub_stamp":"2023-11-24T07:56:23.683366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:14:33.126740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":12,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-24T07:56:24.710112+0000","last_change":"2023-11-24T07:56:23.702360+0000","last_active":"2023-11-24T07:56:24.710112+0000","last_peered":"2023-11-24T07:56:24.710112+0000","last_clean":"2023-11-24T07:56:24.710112+0000","last_became_active":"2023-11-24T07:56:23.701979+0000","last_became_peered":"2023-11-24T07:56:23.701979+0000","last_unstale":"2023-11-24T07:56:24.710112+0000","last_undegraded":"2023-11-24T07:56:24.710112+0000","last_fullsized":"2023-11-24T07:56:24.710112+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:56:22.678019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:56:22.678019+0000","last_clean_scrub_stamp":"2023-11-24T07:56:22.678019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T14:52:28.306044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T07:56:25.711765+0000","last_change":"2023-11-24T07:56:24.727041+0000","last_active":"2023-11-24T07:56:25.711765+0000","last_peered":"2023-11-24T07:56:25.711765+0000","last_clean":"2023-11-24T07:56:25.711765+0000","last_became_active":"2023-11-24T07:56:24.726344+0000","last_became_peered":"2023-11-24T07:56:24.726344+0000","last_unstale":"2023-11-24T07:56:25.711765+0000","last_undegraded":"2023-11-24T07:56:25.711765+0000","last_fullsized":"2023-11-24T07:56:25.711765+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:56:23.683366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:56:23.683366+0000","last_clean_scrub_stamp":"2023-11-24T07:56:23.683366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:13:16.865275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":12,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-24T07:56:24.718243+0000","last_change":"2023-11-24T07:56:23.712776+0000","last_active":"2023-11-24T07:56:24.718243+0000","last_peered":"2023-11-24T07:56:24.718243+0000","last_clean":"2023-11-24T07:56:24.718243+0000","last_became_active":"2023-11-24T07:56:23.712504+0000","last_became_peered":"2023-11-24T07:56:23.712504+0000","last_unstale":"2023-11-24T07:56:24.718243+0000","last_undegraded":"2023-11-24T07:56:24.718243+0000","last_fullsized":"2023-11-24T07:56:24.718243+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:56:22.678019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:56:22.678019+0000","last_clean_scrub_stamp":"2023-11-24T07:56:22.678019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T19:05:26.056275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":165,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-24T07:56:24.728424+0000","last_change":"2023-11-24T07:56:19.668905+0000","last_active":"2023-11-24T07:56:24.728424+0000","last_peered":"2023-11-24T07:56:24.728424+0000","last_clean":"2023-11-24T07:56:24.728424+0000","last_became_active":"2023-11-24T07:56:19.668601+0000","last_became_peered":"2023-11-24T07:56:19.668601+0000","last_unstale":"2023-11-24T07:56:24.728424+0000","last_undegraded":"2023-11-24T07:56:24.728424+0000","last_fullsized":"2023-11-24T07:56:24.728424+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:56:18.657844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:56:18.657844+0000","last_clean_scrub_stamp":"2023-11-24T07:56:18.657844+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T11:51:53.828455+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T07:56:27.783117+0000","last_change":"2023-11-24T07:56:24.716904+0000","last_active":"2023-11-24T07:56:27.783117+0000","last_peered":"2023-11-24T07:56:27.783117+0000","last_clean":"2023-11-24T07:56:27.783117+0000","last_became_active":"2023-11-24T07:56:24.716599+0000","last_became_peered":"2023-11-24T07:56:24.716599+0000","last_unstale":"2023-11-24T07:56:27.783117+0000","last_undegraded":"2023-11-24T07:56:27.783117+0000","last_fullsized":"2023-11-24T07:56:27.783117+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:56:23.683366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:56:23.683366+0000","last_clean_scrub_stamp":"2023-11-24T07:56:23.683366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T09:01:29.634179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":12,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-24T07:56:24.719998+0000","last_change":"2023-11-24T07:56:23.712140+0000","last_active":"2023-11-24T07:56:24.719998+0000","last_peered":"2023-11-24T07:56:24.719998+0000","last_clean":"2023-11-24T07:56:24.719998+0000","last_became_active":"2023-11-24T07:56:23.711855+0000","last_became_peered":"2023-11-24T07:56:23.711855+0000","last_unstale":"2023-11-24T07:56:24.719998+0000","last_undegraded":"2023-11-24T07:56:24.719998+0000","last_fullsized":"2023-11-24T07:56:24.719998+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:56:22.678019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:56:22.678019+0000","last_clean_scrub_stamp":"2023-11-24T07:56:22.678019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:13:15.859928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":12,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-24T07:56:24.722016+0000","last_change":"2023-11-24T07:56:23.712415+0000","last_active":"2023-11-24T07:56:24.722016+0000","last_peered":"2023-11-24T07:56:24.722016+0000","last_clean":"2023-11-24T07:56:24.722016+0000","last_became_active":"2023-11-24T07:56:23.712104+0000","last_became_peered":"2023-11-24T07:56:23.712104+0000","last_unstale":"2023-11-24T07:56:24.722016+0000","last_undegraded":"2023-11-24T07:56:24.722016+0000","last_fullsized":"2023-11-24T07:56:24.722016+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:56:22.678019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:56:22.678019+0000","last_clean_scrub_stamp":"2023-11-24T07:56:22.678019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T18:48:32.274789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T07:56:27.783252+0000","last_change":"2023-11-24T07:56:24.718851+0000","last_active":"2023-11-24T07:56:27.783252+0000","last_peered":"2023-11-24T07:56:27.783252+0000","last_clean":"2023-11-24T07:56:27.783252+0000","last_became_active":"2023-11-24T07:56:24.718574+0000","last_became_peered":"2023-11-24T07:56:24.718574+0000","last_unstale":"2023-11-24T07:56:27.783252+0000","last_undegraded":"2023-11-24T07:56:27.783252+0000","last_fullsized":"2023-11-24T07:56:27.783252+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:56:23.683366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:56:23.683366+0000","last_clean_scrub_stamp":"2023-11-24T07:56:23.683366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T14:36:27.615721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"20'1","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T07:56:25.711989+0000","last_change":"2023-11-24T07:56:23.708583+0000","last_active":"2023-11-24T07:56:25.711989+0000","last_peered":"2023-11-24T07:56:25.711989+0000","last_clean":"2023-11-24T07:56:25.711989+0000","last_became_active":"2023-11-24T07:56:23.708291+0000","last_became_peered":"2023-11-24T07:56:23.708291+0000","last_unstale":"2023-11-24T07:56:25.711989+0000","last_undegraded":"2023-11-24T07:56:25.711989+0000","last_fullsized":"2023-11-24T07:56:25.711989+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:56:22.678019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:56:22.678019+0000","last_clean_scrub_stamp":"2023-11-24T07:56:22.678019+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T18:20:48.368545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T07:56:25.712005+0000","last_change":"2023-11-24T07:56:24.725877+0000","last_active":"2023-11-24T07:56:25.712005+0000","last_peered":"2023-11-24T07:56:25.712005+0000","last_clean":"2023-11-24T07:56:25.712005+0000","last_became_active":"2023-11-24T07:56:24.725622+0000","last_became_peered":"2023-11-24T07:56:24.725622+0000","last_unstale":"2023-11-24T07:56:25.712005+0000","last_undegraded":"2023-11-24T07:56:25.712005+0000","last_fullsized":"2023-11-24T07:56:25.712005+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:56:23.683366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:56:23.683366+0000","last_clean_scrub_stamp":"2023-11-24T07:56:23.683366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T17:43:47.155869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"19'2","reported_seq":14,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-24T07:56:24.745752+0000","last_change":"2023-11-24T07:56:23.713967+0000","last_active":"2023-11-24T07:56:24.745752+0000","last_peered":"2023-11-24T07:56:24.745752+0000","last_clean":"2023-11-24T07:56:24.745752+0000","last_became_active":"2023-11-24T07:56:23.713711+0000","last_became_peered":"2023-11-24T07:56:23.713711+0000","last_unstale":"2023-11-24T07:56:24.745752+0000","last_undegraded":"2023-11-24T07:56:24.745752+0000","last_fullsized":"2023-11-24T07:56:24.745752+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:56:22.678019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:56:22.678019+0000","last_clean_scrub_stamp":"2023-11-24T07:56:22.678019+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T09:29:56.578230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T07:56:25.712076+0000","last_change":"2023-11-24T07:56:24.724396+0000","last_active":"2023-11-24T07:56:25.712076+0000","last_peered":"2023-11-24T07:56:25.712076+0000","last_clean":"2023-11-24T07:56:25.712076+0000","last_became_active":"2023-11-24T07:56:24.724093+0000","last_became_peered":"2023-11-24T07:56:24.724093+0000","last_unstale":"2023-11-24T07:56:25.712076+0000","last_undegraded":"2023-11-24T07:56:25.712076+0000","last_fullsized":"2023-11-24T07:56:25.712076+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:56:23.683366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:56:23.683366+0000","last_clean_scrub_stamp":"2023-11-24T07:56:23.683366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T18:48:33.280136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":12,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-24T07:56:24.709839+0000","last_change":"2023-11-24T07:56:23.712459+0000","last_active":"2023-11-24T07:56:24.709839+0000","last_peered":"2023-11-24T07:56:24.709839+0000","last_clean":"2023-11-24T07:56:24.709839+0000","last_became_active":"2023-11-24T07:56:23.712194+0000","last_became_peered":"2023-11-24T07:56:23.712194+0000","last_unstale":"2023-11-24T07:56:24.709839+0000","last_undegraded":"2023-11-24T07:56:24.709839+0000","last_fullsized":"2023-11-24T07:56:24.709839+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:56:22.678019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:56:22.678019+0000","last_clean_scrub_stamp":"2023-11-24T07:56:22.678019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T17:19:43.614472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T07:56:26.101174+0000","last_change":"2023-11-24T07:56:24.725437+0000","last_active":"2023-11-24T07:56:26.101174+0000","last_peered":"2023-11-24T07:56:26.101174+0000","last_clean":"2023-11-24T07:56:26.101174+0000","last_became_active":"2023-11-24T07:56:24.724898+0000","last_became_peered":"2023-11-24T07:56:24.724898+0000","last_unstale":"2023-11-24T07:56:26.101174+0000","last_undegraded":"2023-11-24T07:56:26.101174+0000","last_fullsized":"2023-11-24T07:56:26.101174+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:56:23.683366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:56:23.683366+0000","last_clean_scrub_stamp":"2023-11-24T07:56:23.683366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T14:42:07.025247+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":12,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-24T07:56:24.721117+0000","last_change":"2023-11-24T07:56:23.708117+0000","last_active":"2023-11-24T07:56:24.721117+0000","last_peered":"2023-11-24T07:56:24.721117+0000","last_clean":"2023-11-24T07:56:24.721117+0000","last_became_active":"2023-11-24T07:56:23.707774+0000","last_became_peered":"2023-11-24T07:56:23.707774+0000","last_unstale":"2023-11-24T07:56:24.721117+0000","last_undegraded":"2023-11-24T07:56:24.721117+0000","last_fullsized":"2023-11-24T07:56:24.721117+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:56:22.678019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:56:22.678019+0000","last_clean_scrub_stamp":"2023-11-24T07:56:22.678019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T09:01:28.628833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T07:56:25.711408+0000","last_change":"2023-11-24T07:56:24.723772+0000","last_active":"2023-11-24T07:56:25.711408+0000","last_peered":"2023-11-24T07:56:25.711408+0000","last_clean":"2023-11-24T07:56:25.711408+0000","last_became_active":"2023-11-24T07:56:24.723497+0000","last_became_peered":"2023-11-24T07:56:24.723497+0000","last_unstale":"2023-11-24T07:56:25.711408+0000","last_undegraded":"2023-11-24T07:56:25.711408+0000","last_fullsized":"2023-11-24T07:56:25.711408+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:56:23.683366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:56:23.683366+0000","last_clean_scrub_stamp":"2023-11-24T07:56:23.683366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:14:33.126740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":12,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-24T07:56:24.711586+0000","last_change":"2023-11-24T07:56:23.704407+0000","last_active":"2023-11-24T07:56:24.711586+0000","last_peered":"2023-11-24T07:56:24.711586+0000","last_clean":"2023-11-24T07:56:24.711586+0000","last_became_active":"2023-11-24T07:56:23.704097+0000","last_became_peered":"2023-11-24T07:56:23.704097+0000","last_unstale":"2023-11-24T07:56:24.711586+0000","last_undegraded":"2023-11-24T07:56:24.711586+0000","last_fullsized":"2023-11-24T07:56:24.711586+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:56:22.678019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:56:22.678019+0000","last_clean_scrub_stamp":"2023-11-24T07:56:22.678019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T14:40:38.179620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T07:56:27.783059+0000","last_change":"2023-11-24T07:56:24.716998+0000","last_active":"2023-11-24T07:56:27.783059+0000","last_peered":"2023-11-24T07:56:27.783059+0000","last_clean":"2023-11-24T07:56:27.783059+0000","last_became_active":"2023-11-24T07:56:24.716713+0000","last_became_peered":"2023-11-24T07:56:24.716713+0000","last_unstale":"2023-11-24T07:56:27.783059+0000","last_undegraded":"2023-11-24T07:56:27.783059+0000","last_fullsized":"2023-11-24T07:56:27.783059+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:56:23.683366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:56:23.683366+0000","last_clean_scrub_stamp":"2023-11-24T07:56:23.683366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T19:32:53.988497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.b","version":"19'2","reported_seq":14,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-24T07:56:24.745886+0000","last_change":"2023-11-24T07:56:23.712730+0000","last_active":"2023-11-24T07:56:24.745886+0000","last_peered":"2023-11-24T07:56:24.745886+0000","last_clean":"2023-11-24T07:56:24.745886+0000","last_became_active":"2023-11-24T07:56:23.712423+0000","last_became_peered":"2023-11-24T07:56:23.712423+0000","last_unstale":"2023-11-24T07:56:24.745886+0000","last_undegraded":"2023-11-24T07:56:24.745886+0000","last_fullsized":"2023-11-24T07:56:24.745886+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:56:22.678019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:56:22.678019+0000","last_clean_scrub_stamp":"2023-11-24T07:56:22.678019+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T14:36:26.610375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T07:56:26.101381+0000","last_change":"2023-11-24T07:56:24.739297+0000","last_active":"2023-11-24T07:56:26.101381+0000","last_peered":"2023-11-24T07:56:26.101381+0000","last_clean":"2023-11-24T07:56:26.101381+0000","last_became_active":"2023-11-24T07:56:24.738991+0000","last_became_peered":"2023-11-24T07:56:24.738991+0000","last_unstale":"2023-11-24T07:56:26.101381+0000","last_undegraded":"2023-11-24T07:56:26.101381+0000","last_fullsized":"2023-11-24T07:56:26.101381+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:56:23.683366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:56:23.683366+0000","last_clean_scrub_stamp":"2023-11-24T07:56:23.683366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T09:26:22.526827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":12,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-24T07:56:24.712584+0000","last_change":"2023-11-24T07:56:23.709703+0000","last_active":"2023-11-24T07:56:24.712584+0000","last_peered":"2023-11-24T07:56:24.712584+0000","last_clean":"2023-11-24T07:56:24.712584+0000","last_became_active":"2023-11-24T07:56:23.709360+0000","last_became_peered":"2023-11-24T07:56:23.709360+0000","last_unstale":"2023-11-24T07:56:24.712584+0000","last_undegraded":"2023-11-24T07:56:24.712584+0000","last_fullsized":"2023-11-24T07:56:24.712584+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:56:22.678019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:56:22.678019+0000","last_clean_scrub_stamp":"2023-11-24T07:56:22.678019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T16:22:32.708879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T07:56:26.101579+0000","last_change":"2023-11-24T07:56:24.739181+0000","last_active":"2023-11-24T07:56:26.101579+0000","last_peered":"2023-11-24T07:56:26.101579+0000","last_clean":"2023-11-24T07:56:26.101579+0000","last_became_active":"2023-11-24T07:56:24.738864+0000","last_became_peered":"2023-11-24T07:56:24.738864+0000","last_unstale":"2023-11-24T07:56:26.101579+0000","last_undegraded":"2023-11-24T07:56:26.101579+0000","last_fullsized":"2023-11-24T07:56:26.101579+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:56:23.683366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:56:23.683366+0000","last_clean_scrub_stamp":"2023-11-24T07:56:23.683366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T17:52:32.557707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.d","version":"19'1","reported_seq":13,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-24T07:56:24.740226+0000","last_change":"2023-11-24T07:56:23.696616+0000","last_active":"2023-11-24T07:56:24.740226+0000","last_peered":"2023-11-24T07:56:24.740226+0000","last_clean":"2023-11-24T07:56:24.740226+0000","last_became_active":"2023-11-24T07:56:23.696320+0000","last_became_peered":"2023-11-24T07:56:23.696320+0000","last_unstale":"2023-11-24T07:56:24.740226+0000","last_undegraded":"2023-11-24T07:56:24.740226+0000","last_fullsized":"2023-11-24T07:56:24.740226+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:56:22.678019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:56:22.678019+0000","last_clean_scrub_stamp":"2023-11-24T07:56:22.678019+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T17:19:43.614472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T07:56:27.671768+0000","last_change":"2023-11-24T07:56:24.716634+0000","last_active":"2023-11-24T07:56:27.671768+0000","last_peered":"2023-11-24T07:56:27.671768+0000","last_clean":"2023-11-24T07:56:27.671768+0000","last_became_active":"2023-11-24T07:56:24.716211+0000","last_became_peered":"2023-11-24T07:56:24.716211+0000","last_unstale":"2023-11-24T07:56:27.671768+0000","last_undegraded":"2023-11-24T07:56:27.671768+0000","last_fullsized":"2023-11-24T07:56:27.671768+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:56:23.683366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:56:23.683366+0000","last_clean_scrub_stamp":"2023-11-24T07:56:23.683366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T15:43:54.658085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"19'2","reported_seq":14,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-24T07:56:24.736924+0000","last_change":"2023-11-24T07:56:23.703304+0000","last_active":"2023-11-24T07:56:24.736924+0000","last_peered":"2023-11-24T07:56:24.736924+0000","last_clean":"2023-11-24T07:56:24.736924+0000","last_became_active":"2023-11-24T07:56:23.703016+0000","last_became_peered":"2023-11-24T07:56:23.703016+0000","last_unstale":"2023-11-24T07:56:24.736924+0000","last_undegraded":"2023-11-24T07:56:24.736924+0000","last_fullsized":"2023-11-24T07:56:24.736924+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:56:22.678019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:56:22.678019+0000","last_clean_scrub_stamp":"2023-11-24T07:56:22.678019+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T14:36:26.610375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T07:56:27.672202+0000","last_change":"2023-11-24T07:56:24.735087+0000","last_active":"2023-11-24T07:56:27.672202+0000","last_peered":"2023-11-24T07:56:27.672202+0000","last_clean":"2023-11-24T07:56:27.672202+0000","last_became_active":"2023-11-24T07:56:24.734770+0000","last_became_peered":"2023-11-24T07:56:24.734770+0000","last_unstale":"2023-11-24T07:56:27.672202+0000","last_undegraded":"2023-11-24T07:56:27.672202+0000","last_fullsized":"2023-11-24T07:56:27.672202+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:56:23.683366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:56:23.683366+0000","last_clean_scrub_stamp":"2023-11-24T07:56:23.683366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T14:46:45.906414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T07:56:25.708692+0000","last_change":"2023-11-24T07:56:24.719171+0000","last_active":"2023-11-24T07:56:25.708692+0000","last_peered":"2023-11-24T07:56:25.708692+0000","last_clean":"2023-11-24T07:56:25.708692+0000","last_became_active":"2023-11-24T07:56:24.718062+0000","last_became_peered":"2023-11-24T07:56:24.718062+0000","last_unstale":"2023-11-24T07:56:25.708692+0000","last_undegraded":"2023-11-24T07:56:25.708692+0000","last_fullsized":"2023-11-24T07:56:25.708692+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:56:23.683366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:56:23.683366+0000","last_clean_scrub_stamp":"2023-11-24T07:56:23.683366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T18:53:14.850807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T07:56:27.782576+0000","last_change":"2023-11-24T07:56:24.734652+0000","last_active":"2023-11-24T07:56:27.782576+0000","last_peered":"2023-11-24T07:56:27.782576+0000","last_clean":"2023-11-24T07:56:27.782576+0000","last_became_active":"2023-11-24T07:56:24.734348+0000","last_became_peered":"2023-11-24T07:56:24.734348+0000","last_unstale":"2023-11-24T07:56:27.782576+0000","last_undegraded":"2023-11-24T07:56:27.782576+0000","last_fullsized":"2023-11-24T07:56:27.782576+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:56:23.683366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:56:23.683366+0000","last_clean_scrub_stamp":"2023-11-24T07:56:23.683366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T18:53:14.850807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T07:56:26.101190+0000","last_change":"2023-11-24T07:56:24.731378+0000","last_active":"2023-11-24T07:56:26.101190+0000","last_peered":"2023-11-24T07:56:26.101190+0000","last_clean":"2023-11-24T07:56:26.101190+0000","last_became_active":"2023-11-24T07:56:24.731096+0000","last_became_peered":"2023-11-24T07:56:24.731096+0000","last_unstale":"2023-11-24T07:56:26.101190+0000","last_undegraded":"2023-11-24T07:56:26.101190+0000","last_fullsized":"2023-11-24T07:56:26.101190+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:56:23.683366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:56:23.683366+0000","last_clean_scrub_stamp":"2023-11-24T07:56:23.683366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T14:46:45.906414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T07:56:25.712218+0000","last_change":"2023-11-24T07:56:24.723023+0000","last_active":"2023-11-24T07:56:25.712218+0000","last_peered":"2023-11-24T07:56:25.712218+0000","last_clean":"2023-11-24T07:56:25.712218+0000","last_became_active":"2023-11-24T07:56:24.722725+0000","last_became_peered":"2023-11-24T07:56:24.722725+0000","last_unstale":"2023-11-24T07:56:25.712218+0000","last_undegraded":"2023-11-24T07:56:25.712218+0000","last_fullsized":"2023-11-24T07:56:25.712218+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:56:23.683366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:56:23.683366+0000","last_clean_scrub_stamp":"2023-11-24T07:56:23.683366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T18:48:33.280136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T07:56:27.783164+0000","last_change":"2023-11-24T07:56:24.717012+0000","last_active":"2023-11-24T07:56:27.783164+0000","last_peered":"2023-11-24T07:56:27.783164+0000","last_clean":"2023-11-24T07:56:27.783164+0000","last_became_active":"2023-11-24T07:56:24.716377+0000","last_became_peered":"2023-11-24T07:56:24.716377+0000","last_unstale":"2023-11-24T07:56:27.783164+0000","last_undegraded":"2023-11-24T07:56:27.783164+0000","last_fullsized":"2023-11-24T07:56:27.783164+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:56:23.683366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:56:23.683366+0000","last_clean_scrub_stamp":"2023-11-24T07:56:23.683366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T19:05:27.061622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T07:56:27.783295+0000","last_change":"2023-11-24T07:56:24.718395+0000","last_active":"2023-11-24T07:56:27.783295+0000","last_peered":"2023-11-24T07:56:27.783295+0000","last_clean":"2023-11-24T07:56:27.783295+0000","last_became_active":"2023-11-24T07:56:24.718132+0000","last_became_peered":"2023-11-24T07:56:24.718132+0000","last_unstale":"2023-11-24T07:56:27.783295+0000","last_undegraded":"2023-11-24T07:56:27.783295+0000","last_fullsized":"2023-11-24T07:56:27.783295+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:56:23.683366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:56:23.683366+0000","last_clean_scrub_stamp":"2023-11-24T07:56:23.683366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:45:41.378607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T07:56:25.709271+0000","last_change":"2023-11-24T07:56:24.731485+0000","last_active":"2023-11-24T07:56:25.709271+0000","last_peered":"2023-11-24T07:56:25.709271+0000","last_clean":"2023-11-24T07:56:25.709271+0000","last_became_active":"2023-11-24T07:56:24.731163+0000","last_became_peered":"2023-11-24T07:56:24.731163+0000","last_unstale":"2023-11-24T07:56:25.709271+0000","last_undegraded":"2023-11-24T07:56:25.709271+0000","last_fullsized":"2023-11-24T07:56:25.709271+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:56:23.683366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:56:23.683366+0000","last_clean_scrub_stamp":"2023-11-24T07:56:23.683366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T19:43:40.193280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T07:56:25.712117+0000","last_change":"2023-11-24T07:56:24.733823+0000","last_active":"2023-11-24T07:56:25.712117+0000","last_peered":"2023-11-24T07:56:25.712117+0000","last_clean":"2023-11-24T07:56:25.712117+0000","last_became_active":"2023-11-24T07:56:24.733550+0000","last_became_peered":"2023-11-24T07:56:24.733550+0000","last_unstale":"2023-11-24T07:56:25.712117+0000","last_undegraded":"2023-11-24T07:56:25.712117+0000","last_fullsized":"2023-11-24T07:56:25.712117+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:56:23.683366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:56:23.683366+0000","last_clean_scrub_stamp":"2023-11-24T07:56:23.683366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T09:29:57.583576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T07:56:26.101006+0000","last_change":"2023-11-24T07:56:24.728674+0000","last_active":"2023-11-24T07:56:26.101006+0000","last_peered":"2023-11-24T07:56:26.101006+0000","last_clean":"2023-11-24T07:56:26.101006+0000","last_became_active":"2023-11-24T07:56:24.728157+0000","last_became_peered":"2023-11-24T07:56:24.728157+0000","last_unstale":"2023-11-24T07:56:26.101006+0000","last_undegraded":"2023-11-24T07:56:26.101006+0000","last_fullsized":"2023-11-24T07:56:26.101006+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:56:23.683366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:56:23.683366+0000","last_clean_scrub_stamp":"2023-11-24T07:56:23.683366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T13:56:07.967730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T07:56:25.712536+0000","last_change":"2023-11-24T07:56:24.722224+0000","last_active":"2023-11-24T07:56:25.712536+0000","last_peered":"2023-11-24T07:56:25.712536+0000","last_clean":"2023-11-24T07:56:25.712536+0000","last_became_active":"2023-11-24T07:56:24.721954+0000","last_became_peered":"2023-11-24T07:56:24.721954+0000","last_unstale":"2023-11-24T07:56:25.712536+0000","last_undegraded":"2023-11-24T07:56:25.712536+0000","last_fullsized":"2023-11-24T07:56:25.712536+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:56:23.683366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:56:23.683366+0000","last_clean_scrub_stamp":"2023-11-24T07:56:23.683366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T15:43:54.658085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T07:56:27.782867+0000","last_change":"2023-11-24T07:56:24.734996+0000","last_active":"2023-11-24T07:56:27.782867+0000","last_peered":"2023-11-24T07:56:27.782867+0000","last_clean":"2023-11-24T07:56:27.782867+0000","last_became_active":"2023-11-24T07:56:24.734198+0000","last_became_peered":"2023-11-24T07:56:24.734198+0000","last_unstale":"2023-11-24T07:56:27.782867+0000","last_undegraded":"2023-11-24T07:56:27.782867+0000","last_fullsized":"2023-11-24T07:56:27.782867+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:56:23.683366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:56:23.683366+0000","last_clean_scrub_stamp":"2023-11-24T07:56:23.683366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T19:32:53.988497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T07:56:25.712025+0000","last_change":"2023-11-24T07:56:24.727463+0000","last_active":"2023-11-24T07:56:25.712025+0000","last_peered":"2023-11-24T07:56:25.712025+0000","last_clean":"2023-11-24T07:56:25.712025+0000","last_became_active":"2023-11-24T07:56:24.727033+0000","last_became_peered":"2023-11-24T07:56:24.727033+0000","last_unstale":"2023-11-24T07:56:25.712025+0000","last_undegraded":"2023-11-24T07:56:25.712025+0000","last_fullsized":"2023-11-24T07:56:25.712025+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:56:23.683366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:56:23.683366+0000","last_clean_scrub_stamp":"2023-11-24T07:56:23.683366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T09:29:57.583576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T07:56:27.671926+0000","last_change":"2023-11-24T07:56:24.734645+0000","last_active":"2023-11-24T07:56:27.671926+0000","last_peered":"2023-11-24T07:56:27.671926+0000","last_clean":"2023-11-24T07:56:27.671926+0000","last_became_active":"2023-11-24T07:56:24.734367+0000","last_became_peered":"2023-11-24T07:56:24.734367+0000","last_unstale":"2023-11-24T07:56:27.671926+0000","last_undegraded":"2023-11-24T07:56:27.671926+0000","last_fullsized":"2023-11-24T07:56:27.671926+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:56:23.683366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:56:23.683366+0000","last_clean_scrub_stamp":"2023-11-24T07:56:23.683366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T19:48:41.181670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574854,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27228,"kb_used_data":208,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344612,"statfs":{"total":96636764160,"available":96608882688,"internally_reserved":0,"allocated":212992,"data_stored":71034,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47499999999999998}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59499999999999997}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65100000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.44400000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55400000000000005}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.39500000000000002}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54300000000000004}]}]},{"osd":6,"up_from":13,"seq":55834574855,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27220,"kb_used_data":200,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344620,"statfs":{"total":96636764160,"available":96608890880,"internally_reserved":0,"allocated":204800,"data_stored":70910,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59699999999999998}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.69599999999999995}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.505}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55900000000000005}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53300000000000003}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58199999999999996}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.72199999999999998}]}]},{"osd":1,"up_from":13,"seq":55834574856,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27804,"kb_used_data":784,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344036,"statfs":{"total":96636764160,"available":96608292864,"internally_reserved":0,"allocated":802816,"data_stored":661348,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.70899999999999996}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60999999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66500000000000004}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.755}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.73899999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.81299999999999994}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.85399999999999998}]}]},{"osd":0,"up_from":13,"seq":55834574856,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27228,"kb_used_data":208,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344612,"statfs":{"total":96636764160,"available":96608882688,"internally_reserved":0,"allocated":212992,"data_stored":70966,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.29199999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.251}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.32100000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55200000000000005}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67000000000000004}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58599999999999997}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65900000000000003}]}]},{"osd":2,"up_from":13,"seq":55834574856,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27236,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344604,"statfs":{"total":96636764160,"available":96608874496,"internally_reserved":0,"allocated":217088,"data_stored":72522,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.39700000000000002}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59199999999999997}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.45400000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57499999999999996}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.70199999999999996}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.72699999999999998}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.76500000000000001}]}]},{"osd":3,"up_from":13,"seq":55834574856,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27228,"kb_used_data":204,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344612,"statfs":{"total":96636764160,"available":96608882688,"internally_reserved":0,"allocated":208896,"data_stored":71492,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.31}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.35899999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.27500000000000002}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58999999999999997}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53100000000000003}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.747}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.70899999999999996}]}]},{"osd":4,"up_from":13,"seq":55834574855,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27244,"kb_used_data":220,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344596,"statfs":{"total":96636764160,"available":96608866304,"internally_reserved":0,"allocated":225280,"data_stored":73194,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64800000000000002}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.69399999999999995}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59599999999999997}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.70899999999999996}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.39700000000000002}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.377}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.498}]}]},{"osd":5,"up_from":13,"seq":55834574856,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27812,"kb_used_data":788,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344028,"statfs":{"total":96636764160,"available":96608284672,"internally_reserved":0,"allocated":806912,"data_stored":661370,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.45300000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53300000000000003}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.622}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60499999999999998}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68799999999999994}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.77100000000000002}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.78500000000000003}]}]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":2284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":124,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-11-24T07:56:38.288 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-11-24T07:56:38.450 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:38.448+0000 7f102f635700 1 Processor -- start 2023-11-24T07:56:38.450 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:38.448+0000 7f102f635700 1 -- start start 2023-11-24T07:56:38.453 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:38.449+0000 7f102f635700 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f10280722c0 0x7f10280b51b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:56:38.453 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:38.449+0000 7f102f635700 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f10281440f0 0x7f102806e510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:56:38.454 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:38.449+0000 7f102f635700 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1028143ec0 con 0x7f1028143ae0 2023-11-24T07:56:38.454 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:38.449+0000 7f102f635700 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f10280b40a0 con 0x7f10281440f0 2023-11-24T07:56:38.454 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:38.449+0000 7f102f635700 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f10280b3ae0 con 0x7f10280722c0 2023-11-24T07:56:38.454 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:38.449+0000 7f102d3d1700 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7f1028143ae0 0x7f1028071bb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:53764/0 (socket says 172.21.15.59:53764) 2023-11-24T07:56:38.454 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:38.449+0000 7f102d3d1700 1 -- 172.21.15.59:0/3192035966 learned_addr learned my addr 172.21.15.59:0/3192035966 (peer_addr_for_me v1:172.21.15.59:0/0) 2023-11-24T07:56:38.454 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:38.449+0000 7f102dbd2700 1 --2- 172.21.15.59:0/3192035966 >> v2:172.21.15.76:3300/0 conn(0x7f10281440f0 0x7f102806e510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:56:38.454 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:38.449+0000 7f102cbd0700 1 --2- 172.21.15.59:0/3192035966 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f10280722c0 0x7f10280b51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:56:38.454 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:38.449+0000 7f101bfff700 1 -- 172.21.15.59:0/3192035966 <== mon.0 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1474231456 0 0) 0x7f1028143ec0 con 0x7f1028143ae0 2023-11-24T07:56:38.455 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:38.449+0000 7f101bfff700 1 -- 172.21.15.59:0/3192035966 --> v1:172.21.15.59:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f10140035b0 con 0x7f1028143ae0 2023-11-24T07:56:38.455 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:38.449+0000 7f101bfff700 1 -- 172.21.15.59:0/3192035966 <== mon.0 v1:172.21.15.59:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1905922211 0 0) 0x7f10140035b0 con 0x7f1028143ae0 2023-11-24T07:56:38.455 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:38.450+0000 7f101bfff700 1 -- 172.21.15.59:0/3192035966 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f10280722c0 msgr2=0x7f10280b51b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T07:56:38.455 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:38.450+0000 7f101bfff700 1 --2- 172.21.15.59:0/3192035966 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f10280722c0 0x7f10280b51b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:56:38.455 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:38.450+0000 7f101bfff700 1 -- 172.21.15.59:0/3192035966 >> v2:172.21.15.76:3300/0 conn(0x7f10281440f0 msgr2=0x7f102806e510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T07:56:38.455 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:38.450+0000 7f101bfff700 1 --2- 172.21.15.59:0/3192035966 >> v2:172.21.15.76:3300/0 conn(0x7f10281440f0 0x7f102806e510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:56:38.455 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:38.450+0000 7f101bfff700 1 -- 172.21.15.59:0/3192035966 --> v1:172.21.15.59:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f10280b3dc0 con 0x7f1028143ae0 2023-11-24T07:56:38.455 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:38.450+0000 7f101bfff700 1 -- 172.21.15.59:0/3192035966 <== mon.0 v1:172.21.15.59:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1312333566 0 0) 0x7f1024001670 con 0x7f1028143ae0 2023-11-24T07:56:38.455 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:38.450+0000 7f102dbd2700 1 --2- 172.21.15.59:0/3192035966 >> v2:172.21.15.76:3300/0 conn(0x7f10281440f0 0x7f102806e510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-24T07:56:38.455 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:38.450+0000 7f101bfff700 1 -- 172.21.15.59:0/3192035966 >> v1:172.21.15.59:6789/0 conn(0x7f1028143ae0 legacy=0x7f1028071bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:56:38.456 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:38.450+0000 7f101bfff700 1 --2- 172.21.15.59:0/3192035966 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1014003a00 0x7f1014003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:56:38.456 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:38.450+0000 7f101bfff700 1 --2- 172.21.15.59:0/3192035966 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f10280722c0 0x7f1014004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:56:38.456 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:38.450+0000 7f101bfff700 1 --2- 172.21.15.59:0/3192035966 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1014004d90 0x7f1014005210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:56:38.456 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:38.450+0000 7f101bfff700 1 -- 172.21.15.59:0/3192035966 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10280b40a0 con 0x7f1014003a00 2023-11-24T07:56:38.456 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:38.450+0000 7f101bfff700 1 -- 172.21.15.59:0/3192035966 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10280b3ae0 con 0x7f1014004d90 2023-11-24T07:56:38.456 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:38.450+0000 7f101bfff700 1 -- 172.21.15.59:0/3192035966 --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f101400ba10 con 0x7f10280722c0 2023-11-24T07:56:38.457 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:38.450+0000 7f102d3d1700 1 --2- 172.21.15.59:0/3192035966 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1014003a00 0x7f1014003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:56:38.457 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:38.450+0000 7f102cbd0700 1 --2- 172.21.15.59:0/3192035966 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f10280722c0 0x7f1014004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:56:38.457 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:38.450+0000 7f102dbd2700 1 --2- 172.21.15.59:0/3192035966 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1014004d90 0x7f1014005210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:56:38.457 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:38.450+0000 7f102d3d1700 1 -- 172.21.15.59:0/3192035966 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f10280722c0 msgr2=0x7f1014004850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T07:56:38.457 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:38.450+0000 7f102d3d1700 1 --2- 172.21.15.59:0/3192035966 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f10280722c0 0x7f1014004850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:56:38.457 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:38.450+0000 7f102d3d1700 1 -- 172.21.15.59:0/3192035966 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1014004d90 msgr2=0x7f1014005210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T07:56:38.457 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:38.450+0000 7f102d3d1700 1 --2- 172.21.15.59:0/3192035966 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1014004d90 0x7f1014005210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:56:38.457 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:38.450+0000 7f102d3d1700 1 -- 172.21.15.59:0/3192035966 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f10140035b0 con 0x7f1014003a00 2023-11-24T07:56:38.457 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:38.450+0000 7f102dbd2700 1 --2- 172.21.15.59:0/3192035966 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1014004d90 0x7f1014005210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-24T07:56:38.457 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:38.450+0000 7f102d3d1700 1 --2- 172.21.15.59:0/3192035966 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1014003a00 0x7f1014003e20 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f1024007f40 tx=0x7f1014005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=62052cecfb7df04c server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:56:38.457 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:38.450+0000 7f101bfff700 1 -- 172.21.15.59:0/3192035966 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f102400ce00 con 0x7f1014003a00 2023-11-24T07:56:38.458 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:38.450+0000 7f101bfff700 1 -- 172.21.15.59:0/3192035966 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1024005960 con 0x7f1014003a00 2023-11-24T07:56:38.458 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:38.450+0000 7f101bfff700 1 -- 172.21.15.59:0/3192035966 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1024005d10 con 0x7f1014003a00 2023-11-24T07:56:38.458 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:38.451+0000 7f102f635700 1 -- 172.21.15.59:0/3192035966 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1014003a00 msgr2=0x7f1014003e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:56:38.458 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:38.451+0000 7f102f635700 1 --2- 172.21.15.59:0/3192035966 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1014003a00 0x7f1014003e20 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f1024007f40 tx=0x7f1014005880 comp rx=0 tx=0).stop 2023-11-24T07:56:38.458 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:38.451+0000 7f102dbd2700 1 -- 172.21.15.59:0/3192035966 reap_dead start 2023-11-24T07:56:38.458 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:38.451+0000 7f102f635700 1 -- 172.21.15.59:0/3192035966 shutdown_connections 2023-11-24T07:56:38.458 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:38.451+0000 7f102f635700 1 -- 172.21.15.59:0/3192035966 >> 172.21.15.59:0/3192035966 conn(0x7f10280b2740 msgr2=0x7f10280b2b20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:56:38.458 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:38.451+0000 7f102f635700 1 -- 172.21.15.59:0/3192035966 shutdown_connections 2023-11-24T07:56:38.458 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:38.451+0000 7f102f635700 1 -- 172.21.15.59:0/3192035966 wait complete. 2023-11-24T07:56:38.461 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:38.452+0000 7f102f635700 1 Processor -- start 2023-11-24T07:56:38.461 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:38.452+0000 7f102f635700 1 -- start start 2023-11-24T07:56:38.461 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:38.453+0000 7f102f635700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1014003a00 0x7f102815a9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:56:38.461 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:38.453+0000 7f102f635700 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f10280cafc0 0x7f1028142370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:56:38.461 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:38.453+0000 7f102f635700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f10281493c0 0x7f1028149870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:56:38.461 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:38.453+0000 7f102f635700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10280cb8a0 con 0x7f1014003a00 2023-11-24T07:56:38.461 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:38.453+0000 7f102f635700 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10280cbb80 con 0x7f10281493c0 2023-11-24T07:56:38.462 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:38.453+0000 7f102f635700 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f10280cbe60 con 0x7f10280cafc0 2023-11-24T07:56:38.462 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:38.453+0000 7f102d3d1700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1014003a00 0x7f102815a9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:56:38.462 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:38.453+0000 7f102d3d1700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1014003a00 0x7f102815a9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:50302/0 (socket says 172.21.15.59:50302) 2023-11-24T07:56:38.462 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:38.453+0000 7f102d3d1700 1 -- 172.21.15.59:0/2654087045 learned_addr learned my addr 172.21.15.59:0/2654087045 (peer_addr_for_me v2:172.21.15.59:0/0) 2023-11-24T07:56:38.462 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:38.453+0000 7f102cbd0700 1 --2- 172.21.15.59:0/2654087045 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f10280cafc0 0x7f1028142370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:56:38.462 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:38.453+0000 7f102dbd2700 1 --2- 172.21.15.59:0/2654087045 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f10281493c0 0x7f1028149870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:56:38.462 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:38.453+0000 7f102d3d1700 1 -- 172.21.15.59:0/2654087045 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f10280cafc0 msgr2=0x7f1028142370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:56:38.462 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:38.453+0000 7f102d3d1700 1 --2- 172.21.15.59:0/2654087045 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f10280cafc0 0x7f1028142370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:56:38.462 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:38.453+0000 7f102d3d1700 1 -- 172.21.15.59:0/2654087045 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f10281493c0 msgr2=0x7f1028149870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:56:38.463 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:38.453+0000 7f102d3d1700 1 --2- 172.21.15.59:0/2654087045 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f10281493c0 0x7f1028149870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:56:38.463 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:38.453+0000 7f102d3d1700 1 -- 172.21.15.59:0/2654087045 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f10280cc140 con 0x7f1014003a00 2023-11-24T07:56:38.463 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:38.454+0000 7f102d3d1700 1 --2- 172.21.15.59:0/2654087045 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1014003a00 0x7f102815a9f0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f10140046e0 tx=0x7f10280b56f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:56:38.463 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:38.454+0000 7f1019ffb700 1 -- 172.21.15.59:0/2654087045 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1024024070 con 0x7f1014003a00 2023-11-24T07:56:38.463 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:38.454+0000 7f1019ffb700 1 -- 172.21.15.59:0/2654087045 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1024007ba0 con 0x7f1014003a00 2023-11-24T07:56:38.463 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:38.454+0000 7f102f635700 1 -- 172.21.15.59:0/2654087045 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f10280cc420 con 0x7f1014003a00 2023-11-24T07:56:38.463 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:38.454+0000 7f1019ffb700 1 -- 172.21.15.59:0/2654087045 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1024015960 con 0x7f1014003a00 2023-11-24T07:56:38.464 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:38.454+0000 7f102f635700 1 -- 172.21.15.59:0/2654087045 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f10281428b0 con 0x7f1014003a00 2023-11-24T07:56:38.464 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:38.455+0000 7f102f635700 1 -- 172.21.15.59:0/2654087045 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f10280cc790 con 0x7f1014003a00 2023-11-24T07:56:38.464 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:38.455+0000 7f1019ffb700 1 -- 172.21.15.59:0/2654087045 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f102401b020 con 0x7f1014003a00 2023-11-24T07:56:38.464 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:38.456+0000 7f1019ffb700 1 --2- 172.21.15.59:0/2654087045 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f100c06aed0 0x7f100c06d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:56:38.464 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:38.456+0000 7f1019ffb700 1 -- 172.21.15.59:0/2654087045 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f1024092270 con 0x7f1014003a00 2023-11-24T07:56:38.465 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:38.456+0000 7f102cbd0700 1 --2- 172.21.15.59:0/2654087045 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f100c06aed0 0x7f100c06d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:56:38.465 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:38.456+0000 7f102cbd0700 1 --2- 172.21.15.59:0/2654087045 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f100c06aed0 0x7f100c06d390 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f102806b7a0 tx=0x7f101c008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:56:38.465 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:38.460+0000 7f1019ffb700 1 -- 172.21.15.59:0/2654087045 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f102405d310 con 0x7f1014003a00 2023-11-24T07:56:38.659 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:38.657+0000 7f102f635700 1 -- 172.21.15.59:0/2654087045 --> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f1028144860 con 0x7f100c06aed0 2023-11-24T07:56:38.666 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:38.665+0000 7f1019ffb700 1 -- 172.21.15.59:0/2654087045 <== mgr.4100 v2:172.21.15.76:6832/4264046536 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+264560 (secure 0 0 0) 0x7f101c013830 con 0x7f100c06aed0 2023-11-24T07:56:38.669 INFO:teuthology.orchestra.run.smithi059.stdout: 2023-11-24T07:56:38.671 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:38.668+0000 7f102f635700 1 -- 172.21.15.59:0/2654087045 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f100c06aed0 msgr2=0x7f100c06d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:56:38.671 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:38.668+0000 7f102f635700 1 --2- 172.21.15.59:0/2654087045 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f100c06aed0 0x7f100c06d390 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f102806b7a0 tx=0x7f101c008040 comp rx=0 tx=0).stop 2023-11-24T07:56:38.671 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:38.668+0000 7f102f635700 1 -- 172.21.15.59:0/2654087045 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1014003a00 msgr2=0x7f102815a9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:56:38.672 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:38.668+0000 7f102f635700 1 --2- 172.21.15.59:0/2654087045 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1014003a00 0x7f102815a9f0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f10140046e0 tx=0x7f10280b56f0 comp rx=0 tx=0).stop 2023-11-24T07:56:38.672 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:38.668+0000 7f102f635700 1 -- 172.21.15.59:0/2654087045 shutdown_connections 2023-11-24T07:56:38.672 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:38.668+0000 7f102f635700 1 --2- 172.21.15.59:0/2654087045 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f100c06aed0 0x7f100c06d390 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:56:38.672 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:38.668+0000 7f102f635700 1 --2- 172.21.15.59:0/2654087045 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f10281493c0 0x7f1028149870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:56:38.672 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:38.668+0000 7f102f635700 1 --2- 172.21.15.59:0/2654087045 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1014003a00 0x7f102815a9f0 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:56:38.672 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:38.668+0000 7f102f635700 1 --2- 172.21.15.59:0/2654087045 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f10280cafc0 0x7f1028142370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:56:38.672 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:38.668+0000 7f102f635700 1 -- 172.21.15.59:0/2654087045 >> 172.21.15.59:0/2654087045 conn(0x7f10280b2740 msgr2=0x7f10280cdf90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:56:38.672 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:38.668+0000 7f102f635700 1 -- 172.21.15.59:0/2654087045 shutdown_connections 2023-11-24T07:56:38.673 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:38.668+0000 7f102f635700 1 -- 172.21.15.59:0/2654087045 wait complete. 2023-11-24T07:56:38.673 INFO:teuthology.orchestra.run.smithi059.stderr:dumped all 2023-11-24T07:56:38.682 INFO:teuthology.orchestra.run.smithi059.stdout:{"pg_ready":true,"pg_map":{"version":23,"stamp":"2023-11-24T07:56:38.301163+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":219000,"kb_used_data":2824,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754755720,"statfs":{"total":773094113280,"available":772869857280,"internally_reserved":0,"allocated":2891776,"data_stored":1752836,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12745,"internal_metadata":221236791},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003532"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T07:56:27.672077+0000","last_change":"2023-11-24T07:56:24.732130+0000","last_active":"2023-11-24T07:56:27.672077+0000","last_peered":"2023-11-24T07:56:27.672077+0000","last_clean":"2023-11-24T07:56:27.672077+0000","last_became_active":"2023-11-24T07:56:24.731726+0000","last_became_peered":"2023-11-24T07:56:24.731726+0000","last_unstale":"2023-11-24T07:56:27.672077+0000","last_undegraded":"2023-11-24T07:56:27.672077+0000","last_fullsized":"2023-11-24T07:56:27.672077+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:56:23.683366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:56:23.683366+0000","last_clean_scrub_stamp":"2023-11-24T07:56:23.683366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:52:22.451306+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T07:56:25.712721+0000","last_change":"2023-11-24T07:56:24.721147+0000","last_active":"2023-11-24T07:56:25.712721+0000","last_peered":"2023-11-24T07:56:25.712721+0000","last_clean":"2023-11-24T07:56:25.712721+0000","last_became_active":"2023-11-24T07:56:24.720857+0000","last_became_peered":"2023-11-24T07:56:24.720857+0000","last_unstale":"2023-11-24T07:56:25.712721+0000","last_undegraded":"2023-11-24T07:56:25.712721+0000","last_fullsized":"2023-11-24T07:56:25.712721+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:56:23.683366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:56:23.683366+0000","last_clean_scrub_stamp":"2023-11-24T07:56:23.683366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T19:48:41.181670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T07:56:25.711756+0000","last_change":"2023-11-24T07:56:24.726027+0000","last_active":"2023-11-24T07:56:25.711756+0000","last_peered":"2023-11-24T07:56:25.711756+0000","last_clean":"2023-11-24T07:56:25.711756+0000","last_became_active":"2023-11-24T07:56:24.725339+0000","last_became_peered":"2023-11-24T07:56:24.725339+0000","last_unstale":"2023-11-24T07:56:25.711756+0000","last_undegraded":"2023-11-24T07:56:25.711756+0000","last_fullsized":"2023-11-24T07:56:25.711756+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:56:23.683366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:56:23.683366+0000","last_clean_scrub_stamp":"2023-11-24T07:56:23.683366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T19:26:21.558298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T07:56:25.709012+0000","last_change":"2023-11-24T07:56:24.725132+0000","last_active":"2023-11-24T07:56:25.709012+0000","last_peered":"2023-11-24T07:56:25.709012+0000","last_clean":"2023-11-24T07:56:25.709012+0000","last_became_active":"2023-11-24T07:56:24.724817+0000","last_became_peered":"2023-11-24T07:56:24.724817+0000","last_unstale":"2023-11-24T07:56:25.709012+0000","last_undegraded":"2023-11-24T07:56:25.709012+0000","last_fullsized":"2023-11-24T07:56:25.709012+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:56:23.683366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:56:23.683366+0000","last_clean_scrub_stamp":"2023-11-24T07:56:23.683366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T19:32:53.988497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T07:56:27.671602+0000","last_change":"2023-11-24T07:56:24.722870+0000","last_active":"2023-11-24T07:56:27.671602+0000","last_peered":"2023-11-24T07:56:27.671602+0000","last_clean":"2023-11-24T07:56:27.671602+0000","last_became_active":"2023-11-24T07:56:24.722572+0000","last_became_peered":"2023-11-24T07:56:24.722572+0000","last_unstale":"2023-11-24T07:56:27.671602+0000","last_undegraded":"2023-11-24T07:56:27.671602+0000","last_fullsized":"2023-11-24T07:56:27.671602+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:56:23.683366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:56:23.683366+0000","last_clean_scrub_stamp":"2023-11-24T07:56:23.683366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T15:28:38.799061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T07:56:25.708767+0000","last_change":"2023-11-24T07:56:24.726061+0000","last_active":"2023-11-24T07:56:25.708767+0000","last_peered":"2023-11-24T07:56:25.708767+0000","last_clean":"2023-11-24T07:56:25.708767+0000","last_became_active":"2023-11-24T07:56:24.725750+0000","last_became_peered":"2023-11-24T07:56:24.725750+0000","last_unstale":"2023-11-24T07:56:25.708767+0000","last_undegraded":"2023-11-24T07:56:25.708767+0000","last_fullsized":"2023-11-24T07:56:25.708767+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:56:23.683366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:56:23.683366+0000","last_clean_scrub_stamp":"2023-11-24T07:56:23.683366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T16:22:33.714226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T07:56:27.671397+0000","last_change":"2023-11-24T07:56:24.725406+0000","last_active":"2023-11-24T07:56:27.671397+0000","last_peered":"2023-11-24T07:56:27.671397+0000","last_clean":"2023-11-24T07:56:27.671397+0000","last_became_active":"2023-11-24T07:56:24.725080+0000","last_became_peered":"2023-11-24T07:56:24.725080+0000","last_unstale":"2023-11-24T07:56:27.671397+0000","last_undegraded":"2023-11-24T07:56:27.671397+0000","last_fullsized":"2023-11-24T07:56:27.671397+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:56:23.683366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:56:23.683366+0000","last_clean_scrub_stamp":"2023-11-24T07:56:23.683366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T18:48:33.280136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T07:56:25.711711+0000","last_change":"2023-11-24T07:56:24.719072+0000","last_active":"2023-11-24T07:56:25.711711+0000","last_peered":"2023-11-24T07:56:25.711711+0000","last_clean":"2023-11-24T07:56:25.711711+0000","last_became_active":"2023-11-24T07:56:24.718791+0000","last_became_peered":"2023-11-24T07:56:24.718791+0000","last_unstale":"2023-11-24T07:56:25.711711+0000","last_undegraded":"2023-11-24T07:56:25.711711+0000","last_fullsized":"2023-11-24T07:56:25.711711+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:56:23.683366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:56:23.683366+0000","last_clean_scrub_stamp":"2023-11-24T07:56:23.683366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:13:16.865275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T07:56:27.783015+0000","last_change":"2023-11-24T07:56:24.743409+0000","last_active":"2023-11-24T07:56:27.783015+0000","last_peered":"2023-11-24T07:56:27.783015+0000","last_clean":"2023-11-24T07:56:27.783015+0000","last_became_active":"2023-11-24T07:56:24.742988+0000","last_became_peered":"2023-11-24T07:56:24.742988+0000","last_unstale":"2023-11-24T07:56:27.783015+0000","last_undegraded":"2023-11-24T07:56:27.783015+0000","last_fullsized":"2023-11-24T07:56:27.783015+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:56:23.683366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:56:23.683366+0000","last_clean_scrub_stamp":"2023-11-24T07:56:23.683366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T09:01:29.634179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T07:56:27.672028+0000","last_change":"2023-11-24T07:56:24.718066+0000","last_active":"2023-11-24T07:56:27.672028+0000","last_peered":"2023-11-24T07:56:27.672028+0000","last_clean":"2023-11-24T07:56:27.672028+0000","last_became_active":"2023-11-24T07:56:24.715725+0000","last_became_peered":"2023-11-24T07:56:24.715725+0000","last_unstale":"2023-11-24T07:56:27.672028+0000","last_undegraded":"2023-11-24T07:56:27.672028+0000","last_fullsized":"2023-11-24T07:56:27.672028+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:56:23.683366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:56:23.683366+0000","last_clean_scrub_stamp":"2023-11-24T07:56:23.683366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T13:56:07.967730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T07:56:27.671765+0000","last_change":"2023-11-24T07:56:24.730488+0000","last_active":"2023-11-24T07:56:27.671765+0000","last_peered":"2023-11-24T07:56:27.671765+0000","last_clean":"2023-11-24T07:56:27.671765+0000","last_became_active":"2023-11-24T07:56:24.730171+0000","last_became_peered":"2023-11-24T07:56:24.730171+0000","last_unstale":"2023-11-24T07:56:27.671765+0000","last_undegraded":"2023-11-24T07:56:27.671765+0000","last_fullsized":"2023-11-24T07:56:27.671765+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:56:23.683366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:56:23.683366+0000","last_clean_scrub_stamp":"2023-11-24T07:56:23.683366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T19:29:50.373581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T07:56:27.671956+0000","last_change":"2023-11-24T07:56:24.734314+0000","last_active":"2023-11-24T07:56:27.671956+0000","last_peered":"2023-11-24T07:56:27.671956+0000","last_clean":"2023-11-24T07:56:27.671956+0000","last_became_active":"2023-11-24T07:56:24.733966+0000","last_became_peered":"2023-11-24T07:56:24.733966+0000","last_unstale":"2023-11-24T07:56:27.671956+0000","last_undegraded":"2023-11-24T07:56:27.671956+0000","last_fullsized":"2023-11-24T07:56:27.671956+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:56:23.683366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:56:23.683366+0000","last_clean_scrub_stamp":"2023-11-24T07:56:23.683366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T19:39:29.828696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T07:56:27.782981+0000","last_change":"2023-11-24T07:56:24.717337+0000","last_active":"2023-11-24T07:56:27.782981+0000","last_peered":"2023-11-24T07:56:27.782981+0000","last_clean":"2023-11-24T07:56:27.782981+0000","last_became_active":"2023-11-24T07:56:24.716929+0000","last_became_peered":"2023-11-24T07:56:24.716929+0000","last_unstale":"2023-11-24T07:56:27.782981+0000","last_undegraded":"2023-11-24T07:56:27.782981+0000","last_fullsized":"2023-11-24T07:56:27.782981+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:56:23.683366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:56:23.683366+0000","last_clean_scrub_stamp":"2023-11-24T07:56:23.683366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T17:36:14.367398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T07:56:27.783087+0000","last_change":"2023-11-24T07:56:24.720081+0000","last_active":"2023-11-24T07:56:27.783087+0000","last_peered":"2023-11-24T07:56:27.783087+0000","last_clean":"2023-11-24T07:56:27.783087+0000","last_became_active":"2023-11-24T07:56:24.719776+0000","last_became_peered":"2023-11-24T07:56:24.719776+0000","last_unstale":"2023-11-24T07:56:27.783087+0000","last_undegraded":"2023-11-24T07:56:27.783087+0000","last_fullsized":"2023-11-24T07:56:27.783087+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:56:23.683366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:56:23.683366+0000","last_clean_scrub_stamp":"2023-11-24T07:56:23.683366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T19:43:40.193280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T07:56:25.712435+0000","last_change":"2023-11-24T07:56:24.719400+0000","last_active":"2023-11-24T07:56:25.712435+0000","last_peered":"2023-11-24T07:56:25.712435+0000","last_clean":"2023-11-24T07:56:25.712435+0000","last_became_active":"2023-11-24T07:56:24.719095+0000","last_became_peered":"2023-11-24T07:56:24.719095+0000","last_unstale":"2023-11-24T07:56:25.712435+0000","last_undegraded":"2023-11-24T07:56:25.712435+0000","last_fullsized":"2023-11-24T07:56:25.712435+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:56:23.683366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:56:23.683366+0000","last_clean_scrub_stamp":"2023-11-24T07:56:23.683366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T19:29:50.373581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T07:56:26.100997+0000","last_change":"2023-11-24T07:56:24.725927+0000","last_active":"2023-11-24T07:56:26.100997+0000","last_peered":"2023-11-24T07:56:26.100997+0000","last_clean":"2023-11-24T07:56:26.100997+0000","last_became_active":"2023-11-24T07:56:24.725568+0000","last_became_peered":"2023-11-24T07:56:24.725568+0000","last_unstale":"2023-11-24T07:56:26.100997+0000","last_undegraded":"2023-11-24T07:56:26.100997+0000","last_fullsized":"2023-11-24T07:56:26.100997+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:56:23.683366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:56:23.683366+0000","last_clean_scrub_stamp":"2023-11-24T07:56:23.683366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T19:39:29.828696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T07:56:26.101717+0000","last_change":"2023-11-24T07:56:24.730986+0000","last_active":"2023-11-24T07:56:26.101717+0000","last_peered":"2023-11-24T07:56:26.101717+0000","last_clean":"2023-11-24T07:56:26.101717+0000","last_became_active":"2023-11-24T07:56:24.730564+0000","last_became_peered":"2023-11-24T07:56:24.730564+0000","last_unstale":"2023-11-24T07:56:26.101717+0000","last_undegraded":"2023-11-24T07:56:26.101717+0000","last_fullsized":"2023-11-24T07:56:26.101717+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:56:23.683366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:56:23.683366+0000","last_clean_scrub_stamp":"2023-11-24T07:56:23.683366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T14:48:19.320894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T07:56:27.783028+0000","last_change":"2023-11-24T07:56:24.716131+0000","last_active":"2023-11-24T07:56:27.783028+0000","last_peered":"2023-11-24T07:56:27.783028+0000","last_clean":"2023-11-24T07:56:27.783028+0000","last_became_active":"2023-11-24T07:56:24.715770+0000","last_became_peered":"2023-11-24T07:56:24.715770+0000","last_unstale":"2023-11-24T07:56:27.783028+0000","last_undegraded":"2023-11-24T07:56:27.783028+0000","last_fullsized":"2023-11-24T07:56:27.783028+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:56:23.683366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:56:23.683366+0000","last_clean_scrub_stamp":"2023-11-24T07:56:23.683366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T16:22:33.714226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T07:56:26.101413+0000","last_change":"2023-11-24T07:56:24.726104+0000","last_active":"2023-11-24T07:56:26.101413+0000","last_peered":"2023-11-24T07:56:26.101413+0000","last_clean":"2023-11-24T07:56:26.101413+0000","last_became_active":"2023-11-24T07:56:24.725718+0000","last_became_peered":"2023-11-24T07:56:24.725718+0000","last_unstale":"2023-11-24T07:56:26.101413+0000","last_undegraded":"2023-11-24T07:56:26.101413+0000","last_fullsized":"2023-11-24T07:56:26.101413+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:56:23.683366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:56:23.683366+0000","last_clean_scrub_stamp":"2023-11-24T07:56:23.683366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T15:24:09.050735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"19'4","reported_seq":16,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-24T07:56:24.743886+0000","last_change":"2023-11-24T07:56:23.714550+0000","last_active":"2023-11-24T07:56:24.743886+0000","last_peered":"2023-11-24T07:56:24.743886+0000","last_clean":"2023-11-24T07:56:24.743886+0000","last_became_active":"2023-11-24T07:56:23.714275+0000","last_became_peered":"2023-11-24T07:56:23.714275+0000","last_unstale":"2023-11-24T07:56:24.743886+0000","last_undegraded":"2023-11-24T07:56:24.743886+0000","last_fullsized":"2023-11-24T07:56:24.743886+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:56:22.678019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:56:22.678019+0000","last_clean_scrub_stamp":"2023-11-24T07:56:22.678019+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T19:05:26.056275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T07:56:25.712605+0000","last_change":"2023-11-24T07:56:24.722991+0000","last_active":"2023-11-24T07:56:25.712605+0000","last_peered":"2023-11-24T07:56:25.712605+0000","last_clean":"2023-11-24T07:56:25.712605+0000","last_became_active":"2023-11-24T07:56:24.722711+0000","last_became_peered":"2023-11-24T07:56:24.722711+0000","last_unstale":"2023-11-24T07:56:25.712605+0000","last_undegraded":"2023-11-24T07:56:25.712605+0000","last_fullsized":"2023-11-24T07:56:25.712605+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:56:23.683366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:56:23.683366+0000","last_clean_scrub_stamp":"2023-11-24T07:56:23.683366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T12:41:22.436958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1e","version":"19'1","reported_seq":13,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-24T07:56:24.739921+0000","last_change":"2023-11-24T07:56:23.702219+0000","last_active":"2023-11-24T07:56:24.739921+0000","last_peered":"2023-11-24T07:56:24.739921+0000","last_clean":"2023-11-24T07:56:24.739921+0000","last_became_active":"2023-11-24T07:56:23.701922+0000","last_became_peered":"2023-11-24T07:56:23.701922+0000","last_unstale":"2023-11-24T07:56:24.739921+0000","last_undegraded":"2023-11-24T07:56:24.739921+0000","last_fullsized":"2023-11-24T07:56:24.739921+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:56:22.678019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:56:22.678019+0000","last_clean_scrub_stamp":"2023-11-24T07:56:22.678019+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T09:01:28.628833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T07:56:25.712243+0000","last_change":"2023-11-24T07:56:24.733713+0000","last_active":"2023-11-24T07:56:25.712243+0000","last_peered":"2023-11-24T07:56:25.712243+0000","last_clean":"2023-11-24T07:56:25.712243+0000","last_became_active":"2023-11-24T07:56:24.733302+0000","last_became_peered":"2023-11-24T07:56:24.733302+0000","last_unstale":"2023-11-24T07:56:25.712243+0000","last_undegraded":"2023-11-24T07:56:25.712243+0000","last_fullsized":"2023-11-24T07:56:25.712243+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:56:23.683366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:56:23.683366+0000","last_clean_scrub_stamp":"2023-11-24T07:56:23.683366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T15:28:38.799061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1d","version":"19'2","reported_seq":14,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-24T07:56:24.740403+0000","last_change":"2023-11-24T07:56:23.707439+0000","last_active":"2023-11-24T07:56:24.740403+0000","last_peered":"2023-11-24T07:56:24.740403+0000","last_clean":"2023-11-24T07:56:24.740403+0000","last_became_active":"2023-11-24T07:56:23.707136+0000","last_became_peered":"2023-11-24T07:56:23.707136+0000","last_unstale":"2023-11-24T07:56:24.740403+0000","last_undegraded":"2023-11-24T07:56:24.740403+0000","last_fullsized":"2023-11-24T07:56:24.740403+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:56:22.678019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:56:22.678019+0000","last_clean_scrub_stamp":"2023-11-24T07:56:22.678019+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T19:43:39.187934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T07:56:27.782857+0000","last_change":"2023-11-24T07:56:24.733325+0000","last_active":"2023-11-24T07:56:27.782857+0000","last_peered":"2023-11-24T07:56:27.782857+0000","last_clean":"2023-11-24T07:56:27.782857+0000","last_became_active":"2023-11-24T07:56:24.733014+0000","last_became_peered":"2023-11-24T07:56:24.733014+0000","last_unstale":"2023-11-24T07:56:27.782857+0000","last_undegraded":"2023-11-24T07:56:27.782857+0000","last_fullsized":"2023-11-24T07:56:27.782857+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:56:23.683366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:56:23.683366+0000","last_clean_scrub_stamp":"2023-11-24T07:56:23.683366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T19:43:40.193280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"19'2","reported_seq":14,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-24T07:56:24.740415+0000","last_change":"2023-11-24T07:56:23.709116+0000","last_active":"2023-11-24T07:56:24.740415+0000","last_peered":"2023-11-24T07:56:24.740415+0000","last_clean":"2023-11-24T07:56:24.740415+0000","last_became_active":"2023-11-24T07:56:23.708703+0000","last_became_peered":"2023-11-24T07:56:23.708703+0000","last_unstale":"2023-11-24T07:56:24.740415+0000","last_undegraded":"2023-11-24T07:56:24.740415+0000","last_fullsized":"2023-11-24T07:56:24.740415+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:56:22.678019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:56:22.678019+0000","last_clean_scrub_stamp":"2023-11-24T07:56:22.678019+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T09:01:28.628833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T07:56:26.101403+0000","last_change":"2023-11-24T07:56:24.731750+0000","last_active":"2023-11-24T07:56:26.101403+0000","last_peered":"2023-11-24T07:56:26.101403+0000","last_clean":"2023-11-24T07:56:26.101403+0000","last_became_active":"2023-11-24T07:56:24.730636+0000","last_became_peered":"2023-11-24T07:56:24.730636+0000","last_unstale":"2023-11-24T07:56:26.101403+0000","last_undegraded":"2023-11-24T07:56:26.101403+0000","last_fullsized":"2023-11-24T07:56:26.101403+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:56:23.683366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:56:23.683366+0000","last_clean_scrub_stamp":"2023-11-24T07:56:23.683366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T15:11:49.015783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1b","version":"19'1","reported_seq":13,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-24T07:56:24.737950+0000","last_change":"2023-11-24T07:56:23.702767+0000","last_active":"2023-11-24T07:56:24.737950+0000","last_peered":"2023-11-24T07:56:24.737950+0000","last_clean":"2023-11-24T07:56:24.737950+0000","last_became_active":"2023-11-24T07:56:23.702499+0000","last_became_peered":"2023-11-24T07:56:23.702499+0000","last_unstale":"2023-11-24T07:56:24.737950+0000","last_undegraded":"2023-11-24T07:56:24.737950+0000","last_fullsized":"2023-11-24T07:56:24.737950+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:56:22.678019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:56:22.678019+0000","last_clean_scrub_stamp":"2023-11-24T07:56:22.678019+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T17:36:13.362051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T07:56:27.782679+0000","last_change":"2023-11-24T07:56:24.735158+0000","last_active":"2023-11-24T07:56:27.782679+0000","last_peered":"2023-11-24T07:56:27.782679+0000","last_clean":"2023-11-24T07:56:27.782679+0000","last_became_active":"2023-11-24T07:56:24.734600+0000","last_became_peered":"2023-11-24T07:56:24.734600+0000","last_unstale":"2023-11-24T07:56:27.782679+0000","last_undegraded":"2023-11-24T07:56:27.782679+0000","last_fullsized":"2023-11-24T07:56:27.782679+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:56:23.683366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:56:23.683366+0000","last_clean_scrub_stamp":"2023-11-24T07:56:23.683366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T17:36:14.367398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1a","version":"19'1","reported_seq":13,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-24T07:56:24.743591+0000","last_change":"2023-11-24T07:56:23.712864+0000","last_active":"2023-11-24T07:56:24.743591+0000","last_peered":"2023-11-24T07:56:24.743591+0000","last_clean":"2023-11-24T07:56:24.743591+0000","last_became_active":"2023-11-24T07:56:23.712559+0000","last_became_peered":"2023-11-24T07:56:23.712559+0000","last_unstale":"2023-11-24T07:56:24.743591+0000","last_undegraded":"2023-11-24T07:56:24.743591+0000","last_fullsized":"2023-11-24T07:56:24.743591+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:56:22.678019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:56:22.678019+0000","last_clean_scrub_stamp":"2023-11-24T07:56:22.678019+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:19:21.093151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T07:56:25.712365+0000","last_change":"2023-11-24T07:56:24.727143+0000","last_active":"2023-11-24T07:56:25.712365+0000","last_peered":"2023-11-24T07:56:25.712365+0000","last_clean":"2023-11-24T07:56:25.712365+0000","last_became_active":"2023-11-24T07:56:24.726637+0000","last_became_peered":"2023-11-24T07:56:24.726637+0000","last_unstale":"2023-11-24T07:56:25.712365+0000","last_undegraded":"2023-11-24T07:56:25.712365+0000","last_fullsized":"2023-11-24T07:56:25.712365+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:56:23.683366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:56:23.683366+0000","last_clean_scrub_stamp":"2023-11-24T07:56:23.683366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T15:28:38.799061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":12,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-24T07:56:24.710275+0000","last_change":"2023-11-24T07:56:23.707863+0000","last_active":"2023-11-24T07:56:24.710275+0000","last_peered":"2023-11-24T07:56:24.710275+0000","last_clean":"2023-11-24T07:56:24.710275+0000","last_became_active":"2023-11-24T07:56:23.707595+0000","last_became_peered":"2023-11-24T07:56:23.707595+0000","last_unstale":"2023-11-24T07:56:24.710275+0000","last_undegraded":"2023-11-24T07:56:24.710275+0000","last_fullsized":"2023-11-24T07:56:24.710275+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:56:22.678019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:56:22.678019+0000","last_clean_scrub_stamp":"2023-11-24T07:56:22.678019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T11:00:03.947675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T07:56:25.712360+0000","last_change":"2023-11-24T07:56:24.721105+0000","last_active":"2023-11-24T07:56:25.712360+0000","last_peered":"2023-11-24T07:56:25.712360+0000","last_clean":"2023-11-24T07:56:25.712360+0000","last_became_active":"2023-11-24T07:56:24.720813+0000","last_became_peered":"2023-11-24T07:56:24.720813+0000","last_unstale":"2023-11-24T07:56:25.712360+0000","last_undegraded":"2023-11-24T07:56:25.712360+0000","last_fullsized":"2023-11-24T07:56:25.712360+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:56:23.683366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:56:23.683366+0000","last_clean_scrub_stamp":"2023-11-24T07:56:23.683366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T17:43:47.155869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":12,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-24T07:56:24.719929+0000","last_change":"2023-11-24T07:56:23.704771+0000","last_active":"2023-11-24T07:56:24.719929+0000","last_peered":"2023-11-24T07:56:24.719929+0000","last_clean":"2023-11-24T07:56:24.719929+0000","last_became_active":"2023-11-24T07:56:23.704433+0000","last_became_peered":"2023-11-24T07:56:23.704433+0000","last_unstale":"2023-11-24T07:56:24.719929+0000","last_undegraded":"2023-11-24T07:56:24.719929+0000","last_fullsized":"2023-11-24T07:56:24.719929+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:56:22.678019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:56:22.678019+0000","last_clean_scrub_stamp":"2023-11-24T07:56:22.678019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:14:32.121393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T07:56:25.711836+0000","last_change":"2023-11-24T07:56:24.734742+0000","last_active":"2023-11-24T07:56:25.711836+0000","last_peered":"2023-11-24T07:56:25.711836+0000","last_clean":"2023-11-24T07:56:25.711836+0000","last_became_active":"2023-11-24T07:56:24.734447+0000","last_became_peered":"2023-11-24T07:56:24.734447+0000","last_unstale":"2023-11-24T07:56:25.711836+0000","last_undegraded":"2023-11-24T07:56:25.711836+0000","last_fullsized":"2023-11-24T07:56:25.711836+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:56:23.683366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:56:23.683366+0000","last_clean_scrub_stamp":"2023-11-24T07:56:23.683366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T19:26:21.558298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":12,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-24T07:56:24.729505+0000","last_change":"2023-11-24T07:56:23.701888+0000","last_active":"2023-11-24T07:56:24.729505+0000","last_peered":"2023-11-24T07:56:24.729505+0000","last_clean":"2023-11-24T07:56:24.729505+0000","last_became_active":"2023-11-24T07:56:23.701611+0000","last_became_peered":"2023-11-24T07:56:23.701611+0000","last_unstale":"2023-11-24T07:56:24.729505+0000","last_undegraded":"2023-11-24T07:56:24.729505+0000","last_fullsized":"2023-11-24T07:56:24.729505+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:56:22.678019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:56:22.678019+0000","last_clean_scrub_stamp":"2023-11-24T07:56:22.678019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T18:53:13.845460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T07:56:25.709371+0000","last_change":"2023-11-24T07:56:24.731116+0000","last_active":"2023-11-24T07:56:25.709371+0000","last_peered":"2023-11-24T07:56:25.709371+0000","last_clean":"2023-11-24T07:56:25.709371+0000","last_became_active":"2023-11-24T07:56:24.730805+0000","last_became_peered":"2023-11-24T07:56:24.730805+0000","last_unstale":"2023-11-24T07:56:25.709371+0000","last_undegraded":"2023-11-24T07:56:25.709371+0000","last_fullsized":"2023-11-24T07:56:25.709371+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:56:23.683366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:56:23.683366+0000","last_clean_scrub_stamp":"2023-11-24T07:56:23.683366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T19:05:27.061622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":12,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-24T07:56:24.712027+0000","last_change":"2023-11-24T07:56:23.707470+0000","last_active":"2023-11-24T07:56:24.712027+0000","last_peered":"2023-11-24T07:56:24.712027+0000","last_clean":"2023-11-24T07:56:24.712027+0000","last_became_active":"2023-11-24T07:56:23.707194+0000","last_became_peered":"2023-11-24T07:56:23.707194+0000","last_unstale":"2023-11-24T07:56:24.712027+0000","last_undegraded":"2023-11-24T07:56:24.712027+0000","last_fullsized":"2023-11-24T07:56:24.712027+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:56:22.678019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:56:22.678019+0000","last_clean_scrub_stamp":"2023-11-24T07:56:22.678019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T19:26:20.552951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T07:56:25.712327+0000","last_change":"2023-11-24T07:56:24.733042+0000","last_active":"2023-11-24T07:56:25.712327+0000","last_peered":"2023-11-24T07:56:25.712327+0000","last_clean":"2023-11-24T07:56:25.712327+0000","last_became_active":"2023-11-24T07:56:24.732746+0000","last_became_peered":"2023-11-24T07:56:24.732746+0000","last_unstale":"2023-11-24T07:56:25.712327+0000","last_undegraded":"2023-11-24T07:56:25.712327+0000","last_fullsized":"2023-11-24T07:56:25.712327+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:56:23.683366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:56:23.683366+0000","last_clean_scrub_stamp":"2023-11-24T07:56:23.683366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:19:22.098498+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.15","version":"19'3","reported_seq":15,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-24T07:56:24.739732+0000","last_change":"2023-11-24T07:56:23.703208+0000","last_active":"2023-11-24T07:56:24.739732+0000","last_peered":"2023-11-24T07:56:24.739732+0000","last_clean":"2023-11-24T07:56:24.739732+0000","last_became_active":"2023-11-24T07:56:23.702891+0000","last_became_peered":"2023-11-24T07:56:23.702891+0000","last_unstale":"2023-11-24T07:56:24.739732+0000","last_undegraded":"2023-11-24T07:56:24.739732+0000","last_fullsized":"2023-11-24T07:56:24.739732+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:56:22.678019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:56:22.678019+0000","last_clean_scrub_stamp":"2023-11-24T07:56:22.678019+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T08:42:21.735516+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T07:56:26.101263+0000","last_change":"2023-11-24T07:56:24.729010+0000","last_active":"2023-11-24T07:56:26.101263+0000","last_peered":"2023-11-24T07:56:26.101263+0000","last_clean":"2023-11-24T07:56:26.101263+0000","last_became_active":"2023-11-24T07:56:24.728711+0000","last_became_peered":"2023-11-24T07:56:24.728711+0000","last_unstale":"2023-11-24T07:56:26.101263+0000","last_undegraded":"2023-11-24T07:56:26.101263+0000","last_fullsized":"2023-11-24T07:56:26.101263+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:56:23.683366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:56:23.683366+0000","last_clean_scrub_stamp":"2023-11-24T07:56:23.683366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T17:48:28.079331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.14","version":"19'6","reported_seq":18,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-24T07:56:24.743066+0000","last_change":"2023-11-24T07:56:23.706638+0000","last_active":"2023-11-24T07:56:24.743066+0000","last_peered":"2023-11-24T07:56:24.743066+0000","last_clean":"2023-11-24T07:56:24.743066+0000","last_became_active":"2023-11-24T07:56:23.706357+0000","last_became_peered":"2023-11-24T07:56:23.706357+0000","last_unstale":"2023-11-24T07:56:24.743066+0000","last_undegraded":"2023-11-24T07:56:24.743066+0000","last_fullsized":"2023-11-24T07:56:24.743066+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:56:22.678019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:56:22.678019+0000","last_clean_scrub_stamp":"2023-11-24T07:56:22.678019+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T14:40:38.179620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T07:56:27.671894+0000","last_change":"2023-11-24T07:56:24.717150+0000","last_active":"2023-11-24T07:56:27.671894+0000","last_peered":"2023-11-24T07:56:27.671894+0000","last_clean":"2023-11-24T07:56:27.671894+0000","last_became_active":"2023-11-24T07:56:24.716723+0000","last_became_peered":"2023-11-24T07:56:24.716723+0000","last_unstale":"2023-11-24T07:56:27.671894+0000","last_undegraded":"2023-11-24T07:56:27.671894+0000","last_fullsized":"2023-11-24T07:56:27.671894+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:56:23.683366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:56:23.683366+0000","last_clean_scrub_stamp":"2023-11-24T07:56:23.683366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T12:41:22.436958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.13","version":"19'2","reported_seq":14,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-24T07:56:24.739597+0000","last_change":"2023-11-24T07:56:23.707817+0000","last_active":"2023-11-24T07:56:24.739597+0000","last_peered":"2023-11-24T07:56:24.739597+0000","last_clean":"2023-11-24T07:56:24.739597+0000","last_became_active":"2023-11-24T07:56:23.707511+0000","last_became_peered":"2023-11-24T07:56:23.707511+0000","last_unstale":"2023-11-24T07:56:24.739597+0000","last_undegraded":"2023-11-24T07:56:24.739597+0000","last_fullsized":"2023-11-24T07:56:24.739597+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:56:22.678019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:56:22.678019+0000","last_clean_scrub_stamp":"2023-11-24T07:56:22.678019+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T15:03:38.755757+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T07:56:27.783173+0000","last_change":"2023-11-24T07:56:24.730834+0000","last_active":"2023-11-24T07:56:27.783173+0000","last_peered":"2023-11-24T07:56:27.783173+0000","last_clean":"2023-11-24T07:56:27.783173+0000","last_became_active":"2023-11-24T07:56:24.730545+0000","last_became_peered":"2023-11-24T07:56:24.730545+0000","last_unstale":"2023-11-24T07:56:27.783173+0000","last_undegraded":"2023-11-24T07:56:27.783173+0000","last_fullsized":"2023-11-24T07:56:27.783173+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:56:23.683366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:56:23.683366+0000","last_clean_scrub_stamp":"2023-11-24T07:56:23.683366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T18:20:49.373891+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":12,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-24T07:56:24.723510+0000","last_change":"2023-11-24T07:56:23.712182+0000","last_active":"2023-11-24T07:56:24.723510+0000","last_peered":"2023-11-24T07:56:24.723510+0000","last_clean":"2023-11-24T07:56:24.723510+0000","last_became_active":"2023-11-24T07:56:23.711814+0000","last_became_peered":"2023-11-24T07:56:23.711814+0000","last_unstale":"2023-11-24T07:56:24.723510+0000","last_undegraded":"2023-11-24T07:56:24.723510+0000","last_fullsized":"2023-11-24T07:56:24.723510+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:56:22.678019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:56:22.678019+0000","last_clean_scrub_stamp":"2023-11-24T07:56:22.678019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T15:43:53.652738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T07:56:26.101323+0000","last_change":"2023-11-24T07:56:24.730497+0000","last_active":"2023-11-24T07:56:26.101323+0000","last_peered":"2023-11-24T07:56:26.101323+0000","last_clean":"2023-11-24T07:56:26.101323+0000","last_became_active":"2023-11-24T07:56:24.730178+0000","last_became_peered":"2023-11-24T07:56:24.730178+0000","last_unstale":"2023-11-24T07:56:26.101323+0000","last_undegraded":"2023-11-24T07:56:26.101323+0000","last_fullsized":"2023-11-24T07:56:26.101323+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:56:23.683366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:56:23.683366+0000","last_clean_scrub_stamp":"2023-11-24T07:56:23.683366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T16:27:17.883314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":12,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-24T07:56:24.711899+0000","last_change":"2023-11-24T07:56:23.713877+0000","last_active":"2023-11-24T07:56:24.711899+0000","last_peered":"2023-11-24T07:56:24.711899+0000","last_clean":"2023-11-24T07:56:24.711899+0000","last_became_active":"2023-11-24T07:56:23.713603+0000","last_became_peered":"2023-11-24T07:56:23.713603+0000","last_unstale":"2023-11-24T07:56:24.711899+0000","last_undegraded":"2023-11-24T07:56:24.711899+0000","last_fullsized":"2023-11-24T07:56:24.711899+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:56:22.678019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:56:22.678019+0000","last_clean_scrub_stamp":"2023-11-24T07:56:22.678019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:13:15.859928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T07:56:27.782480+0000","last_change":"2023-11-24T07:56:24.733613+0000","last_active":"2023-11-24T07:56:27.782480+0000","last_peered":"2023-11-24T07:56:27.782480+0000","last_clean":"2023-11-24T07:56:27.782480+0000","last_became_active":"2023-11-24T07:56:24.733361+0000","last_became_peered":"2023-11-24T07:56:24.733361+0000","last_unstale":"2023-11-24T07:56:27.782480+0000","last_undegraded":"2023-11-24T07:56:27.782480+0000","last_fullsized":"2023-11-24T07:56:27.782480+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:56:23.683366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:56:23.683366+0000","last_clean_scrub_stamp":"2023-11-24T07:56:23.683366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T16:22:33.714226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.10","version":"19'8","reported_seq":20,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-24T07:56:24.740763+0000","last_change":"2023-11-24T07:56:23.701319+0000","last_active":"2023-11-24T07:56:24.740763+0000","last_peered":"2023-11-24T07:56:24.740763+0000","last_clean":"2023-11-24T07:56:24.740763+0000","last_became_active":"2023-11-24T07:56:23.700995+0000","last_became_peered":"2023-11-24T07:56:23.700995+0000","last_unstale":"2023-11-24T07:56:24.740763+0000","last_undegraded":"2023-11-24T07:56:24.740763+0000","last_fullsized":"2023-11-24T07:56:24.740763+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:56:22.678019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:56:22.678019+0000","last_clean_scrub_stamp":"2023-11-24T07:56:22.678019+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T19:43:39.187934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T07:56:26.101104+0000","last_change":"2023-11-24T07:56:24.730591+0000","last_active":"2023-11-24T07:56:26.101104+0000","last_peered":"2023-11-24T07:56:26.101104+0000","last_clean":"2023-11-24T07:56:26.101104+0000","last_became_active":"2023-11-24T07:56:24.730281+0000","last_became_peered":"2023-11-24T07:56:24.730281+0000","last_unstale":"2023-11-24T07:56:26.101104+0000","last_undegraded":"2023-11-24T07:56:26.101104+0000","last_fullsized":"2023-11-24T07:56:26.101104+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:56:23.683366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:56:23.683366+0000","last_clean_scrub_stamp":"2023-11-24T07:56:23.683366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:52:22.451306+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.f","version":"19'1","reported_seq":13,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-24T07:56:24.739965+0000","last_change":"2023-11-24T07:56:23.708274+0000","last_active":"2023-11-24T07:56:24.739965+0000","last_peered":"2023-11-24T07:56:24.739965+0000","last_clean":"2023-11-24T07:56:24.739965+0000","last_became_active":"2023-11-24T07:56:23.708001+0000","last_became_peered":"2023-11-24T07:56:23.708001+0000","last_unstale":"2023-11-24T07:56:24.739965+0000","last_undegraded":"2023-11-24T07:56:24.739965+0000","last_fullsized":"2023-11-24T07:56:24.739965+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:56:22.678019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:56:22.678019+0000","last_clean_scrub_stamp":"2023-11-24T07:56:22.678019+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T09:29:56.578230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T07:56:25.709280+0000","last_change":"2023-11-24T07:56:24.720989+0000","last_active":"2023-11-24T07:56:25.709280+0000","last_peered":"2023-11-24T07:56:25.709280+0000","last_clean":"2023-11-24T07:56:25.709280+0000","last_became_active":"2023-11-24T07:56:24.720500+0000","last_became_peered":"2023-11-24T07:56:24.720500+0000","last_unstale":"2023-11-24T07:56:25.709280+0000","last_undegraded":"2023-11-24T07:56:25.709280+0000","last_fullsized":"2023-11-24T07:56:25.709280+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:56:23.683366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:56:23.683366+0000","last_clean_scrub_stamp":"2023-11-24T07:56:23.683366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T09:01:29.634179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":12,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-24T07:56:24.714465+0000","last_change":"2023-11-24T07:56:23.704585+0000","last_active":"2023-11-24T07:56:24.714465+0000","last_peered":"2023-11-24T07:56:24.714465+0000","last_clean":"2023-11-24T07:56:24.714465+0000","last_became_active":"2023-11-24T07:56:23.704316+0000","last_became_peered":"2023-11-24T07:56:23.704316+0000","last_unstale":"2023-11-24T07:56:24.714465+0000","last_undegraded":"2023-11-24T07:56:24.714465+0000","last_fullsized":"2023-11-24T07:56:24.714465+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:56:22.678019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:56:22.678019+0000","last_clean_scrub_stamp":"2023-11-24T07:56:22.678019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T18:53:13.845460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T07:56:25.709031+0000","last_change":"2023-11-24T07:56:24.718689+0000","last_active":"2023-11-24T07:56:25.709031+0000","last_peered":"2023-11-24T07:56:25.709031+0000","last_clean":"2023-11-24T07:56:25.709031+0000","last_became_active":"2023-11-24T07:56:24.717862+0000","last_became_peered":"2023-11-24T07:56:24.717862+0000","last_unstale":"2023-11-24T07:56:25.709031+0000","last_undegraded":"2023-11-24T07:56:25.709031+0000","last_fullsized":"2023-11-24T07:56:25.709031+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:56:23.683366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:56:23.683366+0000","last_clean_scrub_stamp":"2023-11-24T07:56:23.683366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T17:36:14.367398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":12,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-24T07:56:24.723355+0000","last_change":"2023-11-24T07:56:23.712764+0000","last_active":"2023-11-24T07:56:24.723355+0000","last_peered":"2023-11-24T07:56:24.723355+0000","last_clean":"2023-11-24T07:56:24.723355+0000","last_became_active":"2023-11-24T07:56:23.712448+0000","last_became_peered":"2023-11-24T07:56:23.712448+0000","last_unstale":"2023-11-24T07:56:24.723355+0000","last_undegraded":"2023-11-24T07:56:24.723355+0000","last_fullsized":"2023-11-24T07:56:24.723355+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:56:22.678019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:56:22.678019+0000","last_clean_scrub_stamp":"2023-11-24T07:56:22.678019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T19:29:49.368234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T07:56:27.671634+0000","last_change":"2023-11-24T07:56:24.716237+0000","last_active":"2023-11-24T07:56:27.671634+0000","last_peered":"2023-11-24T07:56:27.671634+0000","last_clean":"2023-11-24T07:56:27.671634+0000","last_became_active":"2023-11-24T07:56:24.715897+0000","last_became_peered":"2023-11-24T07:56:24.715897+0000","last_unstale":"2023-11-24T07:56:27.671634+0000","last_undegraded":"2023-11-24T07:56:27.671634+0000","last_fullsized":"2023-11-24T07:56:27.671634+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:56:23.683366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:56:23.683366+0000","last_clean_scrub_stamp":"2023-11-24T07:56:23.683366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:19:22.098498+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T07:56:25.711564+0000","last_change":"2023-11-24T07:56:24.723045+0000","last_active":"2023-11-24T07:56:25.711564+0000","last_peered":"2023-11-24T07:56:25.711564+0000","last_clean":"2023-11-24T07:56:25.711564+0000","last_became_active":"2023-11-24T07:56:24.722669+0000","last_became_peered":"2023-11-24T07:56:24.722669+0000","last_unstale":"2023-11-24T07:56:25.711564+0000","last_undegraded":"2023-11-24T07:56:25.711564+0000","last_fullsized":"2023-11-24T07:56:25.711564+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:56:23.683366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:56:23.683366+0000","last_clean_scrub_stamp":"2023-11-24T07:56:23.683366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:14:33.126740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":12,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-24T07:56:24.710112+0000","last_change":"2023-11-24T07:56:23.702360+0000","last_active":"2023-11-24T07:56:24.710112+0000","last_peered":"2023-11-24T07:56:24.710112+0000","last_clean":"2023-11-24T07:56:24.710112+0000","last_became_active":"2023-11-24T07:56:23.701979+0000","last_became_peered":"2023-11-24T07:56:23.701979+0000","last_unstale":"2023-11-24T07:56:24.710112+0000","last_undegraded":"2023-11-24T07:56:24.710112+0000","last_fullsized":"2023-11-24T07:56:24.710112+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:56:22.678019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:56:22.678019+0000","last_clean_scrub_stamp":"2023-11-24T07:56:22.678019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T14:52:28.306044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T07:56:25.711765+0000","last_change":"2023-11-24T07:56:24.727041+0000","last_active":"2023-11-24T07:56:25.711765+0000","last_peered":"2023-11-24T07:56:25.711765+0000","last_clean":"2023-11-24T07:56:25.711765+0000","last_became_active":"2023-11-24T07:56:24.726344+0000","last_became_peered":"2023-11-24T07:56:24.726344+0000","last_unstale":"2023-11-24T07:56:25.711765+0000","last_undegraded":"2023-11-24T07:56:25.711765+0000","last_fullsized":"2023-11-24T07:56:25.711765+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:56:23.683366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:56:23.683366+0000","last_clean_scrub_stamp":"2023-11-24T07:56:23.683366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:13:16.865275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":12,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-24T07:56:24.718243+0000","last_change":"2023-11-24T07:56:23.712776+0000","last_active":"2023-11-24T07:56:24.718243+0000","last_peered":"2023-11-24T07:56:24.718243+0000","last_clean":"2023-11-24T07:56:24.718243+0000","last_became_active":"2023-11-24T07:56:23.712504+0000","last_became_peered":"2023-11-24T07:56:23.712504+0000","last_unstale":"2023-11-24T07:56:24.718243+0000","last_undegraded":"2023-11-24T07:56:24.718243+0000","last_fullsized":"2023-11-24T07:56:24.718243+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:56:22.678019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:56:22.678019+0000","last_clean_scrub_stamp":"2023-11-24T07:56:22.678019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T19:05:26.056275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":165,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-24T07:56:24.728424+0000","last_change":"2023-11-24T07:56:19.668905+0000","last_active":"2023-11-24T07:56:24.728424+0000","last_peered":"2023-11-24T07:56:24.728424+0000","last_clean":"2023-11-24T07:56:24.728424+0000","last_became_active":"2023-11-24T07:56:19.668601+0000","last_became_peered":"2023-11-24T07:56:19.668601+0000","last_unstale":"2023-11-24T07:56:24.728424+0000","last_undegraded":"2023-11-24T07:56:24.728424+0000","last_fullsized":"2023-11-24T07:56:24.728424+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:56:18.657844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:56:18.657844+0000","last_clean_scrub_stamp":"2023-11-24T07:56:18.657844+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T11:51:53.828455+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T07:56:27.783117+0000","last_change":"2023-11-24T07:56:24.716904+0000","last_active":"2023-11-24T07:56:27.783117+0000","last_peered":"2023-11-24T07:56:27.783117+0000","last_clean":"2023-11-24T07:56:27.783117+0000","last_became_active":"2023-11-24T07:56:24.716599+0000","last_became_peered":"2023-11-24T07:56:24.716599+0000","last_unstale":"2023-11-24T07:56:27.783117+0000","last_undegraded":"2023-11-24T07:56:27.783117+0000","last_fullsized":"2023-11-24T07:56:27.783117+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:56:23.683366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:56:23.683366+0000","last_clean_scrub_stamp":"2023-11-24T07:56:23.683366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T09:01:29.634179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":12,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-24T07:56:24.719998+0000","last_change":"2023-11-24T07:56:23.712140+0000","last_active":"2023-11-24T07:56:24.719998+0000","last_peered":"2023-11-24T07:56:24.719998+0000","last_clean":"2023-11-24T07:56:24.719998+0000","last_became_active":"2023-11-24T07:56:23.711855+0000","last_became_peered":"2023-11-24T07:56:23.711855+0000","last_unstale":"2023-11-24T07:56:24.719998+0000","last_undegraded":"2023-11-24T07:56:24.719998+0000","last_fullsized":"2023-11-24T07:56:24.719998+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:56:22.678019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:56:22.678019+0000","last_clean_scrub_stamp":"2023-11-24T07:56:22.678019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:13:15.859928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":12,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-24T07:56:24.722016+0000","last_change":"2023-11-24T07:56:23.712415+0000","last_active":"2023-11-24T07:56:24.722016+0000","last_peered":"2023-11-24T07:56:24.722016+0000","last_clean":"2023-11-24T07:56:24.722016+0000","last_became_active":"2023-11-24T07:56:23.712104+0000","last_became_peered":"2023-11-24T07:56:23.712104+0000","last_unstale":"2023-11-24T07:56:24.722016+0000","last_undegraded":"2023-11-24T07:56:24.722016+0000","last_fullsized":"2023-11-24T07:56:24.722016+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:56:22.678019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:56:22.678019+0000","last_clean_scrub_stamp":"2023-11-24T07:56:22.678019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T18:48:32.274789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T07:56:27.783252+0000","last_change":"2023-11-24T07:56:24.718851+0000","last_active":"2023-11-24T07:56:27.783252+0000","last_peered":"2023-11-24T07:56:27.783252+0000","last_clean":"2023-11-24T07:56:27.783252+0000","last_became_active":"2023-11-24T07:56:24.718574+0000","last_became_peered":"2023-11-24T07:56:24.718574+0000","last_unstale":"2023-11-24T07:56:27.783252+0000","last_undegraded":"2023-11-24T07:56:27.783252+0000","last_fullsized":"2023-11-24T07:56:27.783252+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:56:23.683366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:56:23.683366+0000","last_clean_scrub_stamp":"2023-11-24T07:56:23.683366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T14:36:27.615721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"20'1","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T07:56:25.711989+0000","last_change":"2023-11-24T07:56:23.708583+0000","last_active":"2023-11-24T07:56:25.711989+0000","last_peered":"2023-11-24T07:56:25.711989+0000","last_clean":"2023-11-24T07:56:25.711989+0000","last_became_active":"2023-11-24T07:56:23.708291+0000","last_became_peered":"2023-11-24T07:56:23.708291+0000","last_unstale":"2023-11-24T07:56:25.711989+0000","last_undegraded":"2023-11-24T07:56:25.711989+0000","last_fullsized":"2023-11-24T07:56:25.711989+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:56:22.678019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:56:22.678019+0000","last_clean_scrub_stamp":"2023-11-24T07:56:22.678019+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T18:20:48.368545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T07:56:25.712005+0000","last_change":"2023-11-24T07:56:24.725877+0000","last_active":"2023-11-24T07:56:25.712005+0000","last_peered":"2023-11-24T07:56:25.712005+0000","last_clean":"2023-11-24T07:56:25.712005+0000","last_became_active":"2023-11-24T07:56:24.725622+0000","last_became_peered":"2023-11-24T07:56:24.725622+0000","last_unstale":"2023-11-24T07:56:25.712005+0000","last_undegraded":"2023-11-24T07:56:25.712005+0000","last_fullsized":"2023-11-24T07:56:25.712005+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:56:23.683366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:56:23.683366+0000","last_clean_scrub_stamp":"2023-11-24T07:56:23.683366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T17:43:47.155869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"19'2","reported_seq":14,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-24T07:56:24.745752+0000","last_change":"2023-11-24T07:56:23.713967+0000","last_active":"2023-11-24T07:56:24.745752+0000","last_peered":"2023-11-24T07:56:24.745752+0000","last_clean":"2023-11-24T07:56:24.745752+0000","last_became_active":"2023-11-24T07:56:23.713711+0000","last_became_peered":"2023-11-24T07:56:23.713711+0000","last_unstale":"2023-11-24T07:56:24.745752+0000","last_undegraded":"2023-11-24T07:56:24.745752+0000","last_fullsized":"2023-11-24T07:56:24.745752+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:56:22.678019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:56:22.678019+0000","last_clean_scrub_stamp":"2023-11-24T07:56:22.678019+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T09:29:56.578230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T07:56:25.712076+0000","last_change":"2023-11-24T07:56:24.724396+0000","last_active":"2023-11-24T07:56:25.712076+0000","last_peered":"2023-11-24T07:56:25.712076+0000","last_clean":"2023-11-24T07:56:25.712076+0000","last_became_active":"2023-11-24T07:56:24.724093+0000","last_became_peered":"2023-11-24T07:56:24.724093+0000","last_unstale":"2023-11-24T07:56:25.712076+0000","last_undegraded":"2023-11-24T07:56:25.712076+0000","last_fullsized":"2023-11-24T07:56:25.712076+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:56:23.683366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:56:23.683366+0000","last_clean_scrub_stamp":"2023-11-24T07:56:23.683366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T18:48:33.280136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":12,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-24T07:56:24.709839+0000","last_change":"2023-11-24T07:56:23.712459+0000","last_active":"2023-11-24T07:56:24.709839+0000","last_peered":"2023-11-24T07:56:24.709839+0000","last_clean":"2023-11-24T07:56:24.709839+0000","last_became_active":"2023-11-24T07:56:23.712194+0000","last_became_peered":"2023-11-24T07:56:23.712194+0000","last_unstale":"2023-11-24T07:56:24.709839+0000","last_undegraded":"2023-11-24T07:56:24.709839+0000","last_fullsized":"2023-11-24T07:56:24.709839+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:56:22.678019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:56:22.678019+0000","last_clean_scrub_stamp":"2023-11-24T07:56:22.678019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T17:19:43.614472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T07:56:26.101174+0000","last_change":"2023-11-24T07:56:24.725437+0000","last_active":"2023-11-24T07:56:26.101174+0000","last_peered":"2023-11-24T07:56:26.101174+0000","last_clean":"2023-11-24T07:56:26.101174+0000","last_became_active":"2023-11-24T07:56:24.724898+0000","last_became_peered":"2023-11-24T07:56:24.724898+0000","last_unstale":"2023-11-24T07:56:26.101174+0000","last_undegraded":"2023-11-24T07:56:26.101174+0000","last_fullsized":"2023-11-24T07:56:26.101174+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:56:23.683366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:56:23.683366+0000","last_clean_scrub_stamp":"2023-11-24T07:56:23.683366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T14:42:07.025247+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":12,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-24T07:56:24.721117+0000","last_change":"2023-11-24T07:56:23.708117+0000","last_active":"2023-11-24T07:56:24.721117+0000","last_peered":"2023-11-24T07:56:24.721117+0000","last_clean":"2023-11-24T07:56:24.721117+0000","last_became_active":"2023-11-24T07:56:23.707774+0000","last_became_peered":"2023-11-24T07:56:23.707774+0000","last_unstale":"2023-11-24T07:56:24.721117+0000","last_undegraded":"2023-11-24T07:56:24.721117+0000","last_fullsized":"2023-11-24T07:56:24.721117+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:56:22.678019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:56:22.678019+0000","last_clean_scrub_stamp":"2023-11-24T07:56:22.678019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T09:01:28.628833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T07:56:25.711408+0000","last_change":"2023-11-24T07:56:24.723772+0000","last_active":"2023-11-24T07:56:25.711408+0000","last_peered":"2023-11-24T07:56:25.711408+0000","last_clean":"2023-11-24T07:56:25.711408+0000","last_became_active":"2023-11-24T07:56:24.723497+0000","last_became_peered":"2023-11-24T07:56:24.723497+0000","last_unstale":"2023-11-24T07:56:25.711408+0000","last_undegraded":"2023-11-24T07:56:25.711408+0000","last_fullsized":"2023-11-24T07:56:25.711408+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:56:23.683366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:56:23.683366+0000","last_clean_scrub_stamp":"2023-11-24T07:56:23.683366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:14:33.126740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":12,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-24T07:56:24.711586+0000","last_change":"2023-11-24T07:56:23.704407+0000","last_active":"2023-11-24T07:56:24.711586+0000","last_peered":"2023-11-24T07:56:24.711586+0000","last_clean":"2023-11-24T07:56:24.711586+0000","last_became_active":"2023-11-24T07:56:23.704097+0000","last_became_peered":"2023-11-24T07:56:23.704097+0000","last_unstale":"2023-11-24T07:56:24.711586+0000","last_undegraded":"2023-11-24T07:56:24.711586+0000","last_fullsized":"2023-11-24T07:56:24.711586+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:56:22.678019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:56:22.678019+0000","last_clean_scrub_stamp":"2023-11-24T07:56:22.678019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T14:40:38.179620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T07:56:27.783059+0000","last_change":"2023-11-24T07:56:24.716998+0000","last_active":"2023-11-24T07:56:27.783059+0000","last_peered":"2023-11-24T07:56:27.783059+0000","last_clean":"2023-11-24T07:56:27.783059+0000","last_became_active":"2023-11-24T07:56:24.716713+0000","last_became_peered":"2023-11-24T07:56:24.716713+0000","last_unstale":"2023-11-24T07:56:27.783059+0000","last_undegraded":"2023-11-24T07:56:27.783059+0000","last_fullsized":"2023-11-24T07:56:27.783059+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:56:23.683366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:56:23.683366+0000","last_clean_scrub_stamp":"2023-11-24T07:56:23.683366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T19:32:53.988497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.b","version":"19'2","reported_seq":14,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-24T07:56:24.745886+0000","last_change":"2023-11-24T07:56:23.712730+0000","last_active":"2023-11-24T07:56:24.745886+0000","last_peered":"2023-11-24T07:56:24.745886+0000","last_clean":"2023-11-24T07:56:24.745886+0000","last_became_active":"2023-11-24T07:56:23.712423+0000","last_became_peered":"2023-11-24T07:56:23.712423+0000","last_unstale":"2023-11-24T07:56:24.745886+0000","last_undegraded":"2023-11-24T07:56:24.745886+0000","last_fullsized":"2023-11-24T07:56:24.745886+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:56:22.678019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:56:22.678019+0000","last_clean_scrub_stamp":"2023-11-24T07:56:22.678019+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T14:36:26.610375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T07:56:26.101381+0000","last_change":"2023-11-24T07:56:24.739297+0000","last_active":"2023-11-24T07:56:26.101381+0000","last_peered":"2023-11-24T07:56:26.101381+0000","last_clean":"2023-11-24T07:56:26.101381+0000","last_became_active":"2023-11-24T07:56:24.738991+0000","last_became_peered":"2023-11-24T07:56:24.738991+0000","last_unstale":"2023-11-24T07:56:26.101381+0000","last_undegraded":"2023-11-24T07:56:26.101381+0000","last_fullsized":"2023-11-24T07:56:26.101381+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:56:23.683366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:56:23.683366+0000","last_clean_scrub_stamp":"2023-11-24T07:56:23.683366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T09:26:22.526827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":12,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-24T07:56:24.712584+0000","last_change":"2023-11-24T07:56:23.709703+0000","last_active":"2023-11-24T07:56:24.712584+0000","last_peered":"2023-11-24T07:56:24.712584+0000","last_clean":"2023-11-24T07:56:24.712584+0000","last_became_active":"2023-11-24T07:56:23.709360+0000","last_became_peered":"2023-11-24T07:56:23.709360+0000","last_unstale":"2023-11-24T07:56:24.712584+0000","last_undegraded":"2023-11-24T07:56:24.712584+0000","last_fullsized":"2023-11-24T07:56:24.712584+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:56:22.678019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:56:22.678019+0000","last_clean_scrub_stamp":"2023-11-24T07:56:22.678019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T16:22:32.708879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T07:56:26.101579+0000","last_change":"2023-11-24T07:56:24.739181+0000","last_active":"2023-11-24T07:56:26.101579+0000","last_peered":"2023-11-24T07:56:26.101579+0000","last_clean":"2023-11-24T07:56:26.101579+0000","last_became_active":"2023-11-24T07:56:24.738864+0000","last_became_peered":"2023-11-24T07:56:24.738864+0000","last_unstale":"2023-11-24T07:56:26.101579+0000","last_undegraded":"2023-11-24T07:56:26.101579+0000","last_fullsized":"2023-11-24T07:56:26.101579+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:56:23.683366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:56:23.683366+0000","last_clean_scrub_stamp":"2023-11-24T07:56:23.683366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T17:52:32.557707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.d","version":"19'1","reported_seq":13,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-24T07:56:24.740226+0000","last_change":"2023-11-24T07:56:23.696616+0000","last_active":"2023-11-24T07:56:24.740226+0000","last_peered":"2023-11-24T07:56:24.740226+0000","last_clean":"2023-11-24T07:56:24.740226+0000","last_became_active":"2023-11-24T07:56:23.696320+0000","last_became_peered":"2023-11-24T07:56:23.696320+0000","last_unstale":"2023-11-24T07:56:24.740226+0000","last_undegraded":"2023-11-24T07:56:24.740226+0000","last_fullsized":"2023-11-24T07:56:24.740226+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:56:22.678019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:56:22.678019+0000","last_clean_scrub_stamp":"2023-11-24T07:56:22.678019+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T17:19:43.614472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T07:56:27.671768+0000","last_change":"2023-11-24T07:56:24.716634+0000","last_active":"2023-11-24T07:56:27.671768+0000","last_peered":"2023-11-24T07:56:27.671768+0000","last_clean":"2023-11-24T07:56:27.671768+0000","last_became_active":"2023-11-24T07:56:24.716211+0000","last_became_peered":"2023-11-24T07:56:24.716211+0000","last_unstale":"2023-11-24T07:56:27.671768+0000","last_undegraded":"2023-11-24T07:56:27.671768+0000","last_fullsized":"2023-11-24T07:56:27.671768+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:56:23.683366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:56:23.683366+0000","last_clean_scrub_stamp":"2023-11-24T07:56:23.683366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T15:43:54.658085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"19'2","reported_seq":14,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-11-24T07:56:24.736924+0000","last_change":"2023-11-24T07:56:23.703304+0000","last_active":"2023-11-24T07:56:24.736924+0000","last_peered":"2023-11-24T07:56:24.736924+0000","last_clean":"2023-11-24T07:56:24.736924+0000","last_became_active":"2023-11-24T07:56:23.703016+0000","last_became_peered":"2023-11-24T07:56:23.703016+0000","last_unstale":"2023-11-24T07:56:24.736924+0000","last_undegraded":"2023-11-24T07:56:24.736924+0000","last_fullsized":"2023-11-24T07:56:24.736924+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:56:22.678019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:56:22.678019+0000","last_clean_scrub_stamp":"2023-11-24T07:56:22.678019+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T14:36:26.610375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T07:56:27.672202+0000","last_change":"2023-11-24T07:56:24.735087+0000","last_active":"2023-11-24T07:56:27.672202+0000","last_peered":"2023-11-24T07:56:27.672202+0000","last_clean":"2023-11-24T07:56:27.672202+0000","last_became_active":"2023-11-24T07:56:24.734770+0000","last_became_peered":"2023-11-24T07:56:24.734770+0000","last_unstale":"2023-11-24T07:56:27.672202+0000","last_undegraded":"2023-11-24T07:56:27.672202+0000","last_fullsized":"2023-11-24T07:56:27.672202+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:56:23.683366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:56:23.683366+0000","last_clean_scrub_stamp":"2023-11-24T07:56:23.683366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T14:46:45.906414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T07:56:25.708692+0000","last_change":"2023-11-24T07:56:24.719171+0000","last_active":"2023-11-24T07:56:25.708692+0000","last_peered":"2023-11-24T07:56:25.708692+0000","last_clean":"2023-11-24T07:56:25.708692+0000","last_became_active":"2023-11-24T07:56:24.718062+0000","last_became_peered":"2023-11-24T07:56:24.718062+0000","last_unstale":"2023-11-24T07:56:25.708692+0000","last_undegraded":"2023-11-24T07:56:25.708692+0000","last_fullsized":"2023-11-24T07:56:25.708692+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:56:23.683366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:56:23.683366+0000","last_clean_scrub_stamp":"2023-11-24T07:56:23.683366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T18:53:14.850807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T07:56:27.782576+0000","last_change":"2023-11-24T07:56:24.734652+0000","last_active":"2023-11-24T07:56:27.782576+0000","last_peered":"2023-11-24T07:56:27.782576+0000","last_clean":"2023-11-24T07:56:27.782576+0000","last_became_active":"2023-11-24T07:56:24.734348+0000","last_became_peered":"2023-11-24T07:56:24.734348+0000","last_unstale":"2023-11-24T07:56:27.782576+0000","last_undegraded":"2023-11-24T07:56:27.782576+0000","last_fullsized":"2023-11-24T07:56:27.782576+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:56:23.683366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:56:23.683366+0000","last_clean_scrub_stamp":"2023-11-24T07:56:23.683366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T18:53:14.850807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T07:56:26.101190+0000","last_change":"2023-11-24T07:56:24.731378+0000","last_active":"2023-11-24T07:56:26.101190+0000","last_peered":"2023-11-24T07:56:26.101190+0000","last_clean":"2023-11-24T07:56:26.101190+0000","last_became_active":"2023-11-24T07:56:24.731096+0000","last_became_peered":"2023-11-24T07:56:24.731096+0000","last_unstale":"2023-11-24T07:56:26.101190+0000","last_undegraded":"2023-11-24T07:56:26.101190+0000","last_fullsized":"2023-11-24T07:56:26.101190+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:56:23.683366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:56:23.683366+0000","last_clean_scrub_stamp":"2023-11-24T07:56:23.683366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T14:46:45.906414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T07:56:25.712218+0000","last_change":"2023-11-24T07:56:24.723023+0000","last_active":"2023-11-24T07:56:25.712218+0000","last_peered":"2023-11-24T07:56:25.712218+0000","last_clean":"2023-11-24T07:56:25.712218+0000","last_became_active":"2023-11-24T07:56:24.722725+0000","last_became_peered":"2023-11-24T07:56:24.722725+0000","last_unstale":"2023-11-24T07:56:25.712218+0000","last_undegraded":"2023-11-24T07:56:25.712218+0000","last_fullsized":"2023-11-24T07:56:25.712218+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:56:23.683366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:56:23.683366+0000","last_clean_scrub_stamp":"2023-11-24T07:56:23.683366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T18:48:33.280136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T07:56:27.783164+0000","last_change":"2023-11-24T07:56:24.717012+0000","last_active":"2023-11-24T07:56:27.783164+0000","last_peered":"2023-11-24T07:56:27.783164+0000","last_clean":"2023-11-24T07:56:27.783164+0000","last_became_active":"2023-11-24T07:56:24.716377+0000","last_became_peered":"2023-11-24T07:56:24.716377+0000","last_unstale":"2023-11-24T07:56:27.783164+0000","last_undegraded":"2023-11-24T07:56:27.783164+0000","last_fullsized":"2023-11-24T07:56:27.783164+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:56:23.683366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:56:23.683366+0000","last_clean_scrub_stamp":"2023-11-24T07:56:23.683366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T19:05:27.061622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T07:56:27.783295+0000","last_change":"2023-11-24T07:56:24.718395+0000","last_active":"2023-11-24T07:56:27.783295+0000","last_peered":"2023-11-24T07:56:27.783295+0000","last_clean":"2023-11-24T07:56:27.783295+0000","last_became_active":"2023-11-24T07:56:24.718132+0000","last_became_peered":"2023-11-24T07:56:24.718132+0000","last_unstale":"2023-11-24T07:56:27.783295+0000","last_undegraded":"2023-11-24T07:56:27.783295+0000","last_fullsized":"2023-11-24T07:56:27.783295+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:56:23.683366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:56:23.683366+0000","last_clean_scrub_stamp":"2023-11-24T07:56:23.683366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:45:41.378607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T07:56:25.709271+0000","last_change":"2023-11-24T07:56:24.731485+0000","last_active":"2023-11-24T07:56:25.709271+0000","last_peered":"2023-11-24T07:56:25.709271+0000","last_clean":"2023-11-24T07:56:25.709271+0000","last_became_active":"2023-11-24T07:56:24.731163+0000","last_became_peered":"2023-11-24T07:56:24.731163+0000","last_unstale":"2023-11-24T07:56:25.709271+0000","last_undegraded":"2023-11-24T07:56:25.709271+0000","last_fullsized":"2023-11-24T07:56:25.709271+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:56:23.683366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:56:23.683366+0000","last_clean_scrub_stamp":"2023-11-24T07:56:23.683366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T19:43:40.193280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T07:56:25.712117+0000","last_change":"2023-11-24T07:56:24.733823+0000","last_active":"2023-11-24T07:56:25.712117+0000","last_peered":"2023-11-24T07:56:25.712117+0000","last_clean":"2023-11-24T07:56:25.712117+0000","last_became_active":"2023-11-24T07:56:24.733550+0000","last_became_peered":"2023-11-24T07:56:24.733550+0000","last_unstale":"2023-11-24T07:56:25.712117+0000","last_undegraded":"2023-11-24T07:56:25.712117+0000","last_fullsized":"2023-11-24T07:56:25.712117+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:56:23.683366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:56:23.683366+0000","last_clean_scrub_stamp":"2023-11-24T07:56:23.683366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T09:29:57.583576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T07:56:26.101006+0000","last_change":"2023-11-24T07:56:24.728674+0000","last_active":"2023-11-24T07:56:26.101006+0000","last_peered":"2023-11-24T07:56:26.101006+0000","last_clean":"2023-11-24T07:56:26.101006+0000","last_became_active":"2023-11-24T07:56:24.728157+0000","last_became_peered":"2023-11-24T07:56:24.728157+0000","last_unstale":"2023-11-24T07:56:26.101006+0000","last_undegraded":"2023-11-24T07:56:26.101006+0000","last_fullsized":"2023-11-24T07:56:26.101006+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:56:23.683366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:56:23.683366+0000","last_clean_scrub_stamp":"2023-11-24T07:56:23.683366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T13:56:07.967730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T07:56:25.712536+0000","last_change":"2023-11-24T07:56:24.722224+0000","last_active":"2023-11-24T07:56:25.712536+0000","last_peered":"2023-11-24T07:56:25.712536+0000","last_clean":"2023-11-24T07:56:25.712536+0000","last_became_active":"2023-11-24T07:56:24.721954+0000","last_became_peered":"2023-11-24T07:56:24.721954+0000","last_unstale":"2023-11-24T07:56:25.712536+0000","last_undegraded":"2023-11-24T07:56:25.712536+0000","last_fullsized":"2023-11-24T07:56:25.712536+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:56:23.683366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:56:23.683366+0000","last_clean_scrub_stamp":"2023-11-24T07:56:23.683366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T15:43:54.658085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T07:56:27.782867+0000","last_change":"2023-11-24T07:56:24.734996+0000","last_active":"2023-11-24T07:56:27.782867+0000","last_peered":"2023-11-24T07:56:27.782867+0000","last_clean":"2023-11-24T07:56:27.782867+0000","last_became_active":"2023-11-24T07:56:24.734198+0000","last_became_peered":"2023-11-24T07:56:24.734198+0000","last_unstale":"2023-11-24T07:56:27.782867+0000","last_undegraded":"2023-11-24T07:56:27.782867+0000","last_fullsized":"2023-11-24T07:56:27.782867+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:56:23.683366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:56:23.683366+0000","last_clean_scrub_stamp":"2023-11-24T07:56:23.683366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T19:32:53.988497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T07:56:25.712025+0000","last_change":"2023-11-24T07:56:24.727463+0000","last_active":"2023-11-24T07:56:25.712025+0000","last_peered":"2023-11-24T07:56:25.712025+0000","last_clean":"2023-11-24T07:56:25.712025+0000","last_became_active":"2023-11-24T07:56:24.727033+0000","last_became_peered":"2023-11-24T07:56:24.727033+0000","last_unstale":"2023-11-24T07:56:25.712025+0000","last_undegraded":"2023-11-24T07:56:25.712025+0000","last_fullsized":"2023-11-24T07:56:25.712025+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:56:23.683366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:56:23.683366+0000","last_clean_scrub_stamp":"2023-11-24T07:56:23.683366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T09:29:57.583576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-24T07:56:27.671926+0000","last_change":"2023-11-24T07:56:24.734645+0000","last_active":"2023-11-24T07:56:27.671926+0000","last_peered":"2023-11-24T07:56:27.671926+0000","last_clean":"2023-11-24T07:56:27.671926+0000","last_became_active":"2023-11-24T07:56:24.734367+0000","last_became_peered":"2023-11-24T07:56:24.734367+0000","last_unstale":"2023-11-24T07:56:27.671926+0000","last_undegraded":"2023-11-24T07:56:27.671926+0000","last_fullsized":"2023-11-24T07:56:27.671926+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:56:23.683366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:56:23.683366+0000","last_clean_scrub_stamp":"2023-11-24T07:56:23.683366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T19:48:41.181670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574854,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27228,"kb_used_data":208,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344612,"statfs":{"total":96636764160,"available":96608882688,"internally_reserved":0,"allocated":212992,"data_stored":71034,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47499999999999998}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59499999999999997}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65100000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.44400000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55400000000000005}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.39500000000000002}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54300000000000004}]}]},{"osd":6,"up_from":13,"seq":55834574855,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27220,"kb_used_data":200,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344620,"statfs":{"total":96636764160,"available":96608890880,"internally_reserved":0,"allocated":204800,"data_stored":70910,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59699999999999998}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.69599999999999995}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.505}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55900000000000005}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53300000000000003}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58199999999999996}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.72199999999999998}]}]},{"osd":1,"up_from":13,"seq":55834574856,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27804,"kb_used_data":784,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344036,"statfs":{"total":96636764160,"available":96608292864,"internally_reserved":0,"allocated":802816,"data_stored":661348,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.70899999999999996}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60999999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66500000000000004}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.755}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.73899999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.81299999999999994}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.85399999999999998}]}]},{"osd":0,"up_from":13,"seq":55834574856,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27228,"kb_used_data":208,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344612,"statfs":{"total":96636764160,"available":96608882688,"internally_reserved":0,"allocated":212992,"data_stored":70966,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.29199999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.251}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.32100000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55200000000000005}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67000000000000004}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58599999999999997}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65900000000000003}]}]},{"osd":2,"up_from":13,"seq":55834574856,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27236,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344604,"statfs":{"total":96636764160,"available":96608874496,"internally_reserved":0,"allocated":217088,"data_stored":72522,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.39700000000000002}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59199999999999997}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.45400000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57499999999999996}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.70199999999999996}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.72699999999999998}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.76500000000000001}]}]},{"osd":3,"up_from":13,"seq":55834574856,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27228,"kb_used_data":204,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344612,"statfs":{"total":96636764160,"available":96608882688,"internally_reserved":0,"allocated":208896,"data_stored":71492,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.31}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.35899999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.27500000000000002}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58999999999999997}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53100000000000003}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.747}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.70899999999999996}]}]},{"osd":4,"up_from":13,"seq":55834574855,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27244,"kb_used_data":220,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344596,"statfs":{"total":96636764160,"available":96608866304,"internally_reserved":0,"allocated":225280,"data_stored":73194,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64800000000000002}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.69399999999999995}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59599999999999997}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.70899999999999996}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.39700000000000002}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.377}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.498}]}]},{"osd":5,"up_from":13,"seq":55834574856,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27812,"kb_used_data":788,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344028,"statfs":{"total":96636764160,"available":96608284672,"internally_reserved":0,"allocated":806912,"data_stored":661370,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.45300000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53300000000000003}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.622}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60499999999999998}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68799999999999994}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.77100000000000002}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.78500000000000003}]}]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":2284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":124,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-11-24T07:56:38.687 INFO:tasks.ceph.ceph_manager.ceph:clean! 2023-11-24T07:56:38.687 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2023-11-24T07:56:38.688 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy 2023-11-24T07:56:38.688 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2023-11-24T07:56:38.846 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:38.844+0000 7f8db3592700 1 Processor -- start 2023-11-24T07:56:38.846 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:38.844+0000 7f8db3592700 1 -- start start 2023-11-24T07:56:38.849 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:38.845+0000 7f8db3592700 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f8dac143ae0 0x7f8dac0b51b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:56:38.850 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:38.845+0000 7f8db3592700 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f8dac1440f0 0x7f8dac071bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:56:38.850 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:38.845+0000 7f8db3592700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f8dac0720f0 0x7f8dac15e8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:56:38.850 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:38.845+0000 7f8db3592700 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8dac0b40a0 con 0x7f8dac0720f0 2023-11-24T07:56:38.850 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:38.845+0000 7f8db3592700 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8dac072580 con 0x7f8dac1440f0 2023-11-24T07:56:38.850 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:38.845+0000 7f8db3592700 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8dac143ec0 con 0x7f8dac143ae0 2023-11-24T07:56:38.850 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:38.845+0000 7f8db1b2f700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f8dac0720f0 0x7f8dac15e8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:56:38.850 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:38.845+0000 7f8db1b2f700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f8dac0720f0 0x7f8dac15e8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:50312/0 (socket says 172.21.15.59:50312) 2023-11-24T07:56:38.850 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:38.845+0000 7f8db1b2f700 1 -- 172.21.15.59:0/3570136001 learned_addr learned my addr 172.21.15.59:0/3570136001 (peer_addr_for_me v2:172.21.15.59:0/0) 2023-11-24T07:56:38.850 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:38.846+0000 7f8db0b2d700 1 --2- 172.21.15.59:0/3570136001 >> v2:172.21.15.76:3300/0 conn(0x7f8dac1440f0 0x7f8dac071bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:56:38.850 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:38.846+0000 7f8db132e700 1 --2- 172.21.15.59:0/3570136001 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f8dac143ae0 0x7f8dac0b51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:56:38.850 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:38.846+0000 7f8db1b2f700 1 -- 172.21.15.59:0/3570136001 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f8dac143ae0 msgr2=0x7f8dac0b51b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T07:56:38.851 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:38.846+0000 7f8db1b2f700 1 --2- 172.21.15.59:0/3570136001 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f8dac143ae0 0x7f8dac0b51b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:56:38.851 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:38.846+0000 7f8db1b2f700 1 -- 172.21.15.59:0/3570136001 >> v2:172.21.15.76:3300/0 conn(0x7f8dac1440f0 msgr2=0x7f8dac071bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T07:56:38.851 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:38.846+0000 7f8db1b2f700 1 --2- 172.21.15.59:0/3570136001 >> v2:172.21.15.76:3300/0 conn(0x7f8dac1440f0 0x7f8dac071bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:56:38.851 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:38.846+0000 7f8db1b2f700 1 -- 172.21.15.59:0/3570136001 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8dac0b3ae0 con 0x7f8dac0720f0 2023-11-24T07:56:38.851 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:38.846+0000 7f8db1b2f700 1 --2- 172.21.15.59:0/3570136001 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8dac0720f0 0x7f8dac15e8b0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f8dac06d070 tx=0x7f8da400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=16145a324c0a4266 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:56:38.851 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:38.846+0000 7f8da3fff700 1 -- 172.21.15.59:0/3570136001 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8da400e070 con 0x7f8dac0720f0 2023-11-24T07:56:38.851 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:38.846+0000 7f8da3fff700 1 -- 172.21.15.59:0/3570136001 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8da400b910 con 0x7f8dac0720f0 2023-11-24T07:56:38.851 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:38.846+0000 7f8da3fff700 1 -- 172.21.15.59:0/3570136001 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8da400bcc0 con 0x7f8dac0720f0 2023-11-24T07:56:38.851 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:38.847+0000 7f8db3592700 1 -- 172.21.15.59:0/3570136001 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8dac0720f0 msgr2=0x7f8dac15e8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:56:38.851 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:38.847+0000 7f8db3592700 1 --2- 172.21.15.59:0/3570136001 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8dac0720f0 0x7f8dac15e8b0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f8dac06d070 tx=0x7f8da400b330 comp rx=0 tx=0).stop 2023-11-24T07:56:38.851 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:38.847+0000 7f8db3592700 1 -- 172.21.15.59:0/3570136001 shutdown_connections 2023-11-24T07:56:38.852 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:38.847+0000 7f8db3592700 1 --2- 172.21.15.59:0/3570136001 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8dac0720f0 0x7f8dac15e8b0 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:56:38.852 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:38.847+0000 7f8db3592700 1 --2- 172.21.15.59:0/3570136001 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f8dac143ae0 0x7f8dac0b51b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:56:38.852 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:38.847+0000 7f8db3592700 1 --2- 172.21.15.59:0/3570136001 >> v2:172.21.15.76:3300/0 conn(0x7f8dac1440f0 0x7f8dac071bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:56:38.852 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:38.847+0000 7f8db3592700 1 -- 172.21.15.59:0/3570136001 >> 172.21.15.59:0/3570136001 conn(0x7f8dac0b2740 msgr2=0x7f8dac0b2b20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:56:38.852 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:38.847+0000 7f8db3592700 1 -- 172.21.15.59:0/3570136001 shutdown_connections 2023-11-24T07:56:38.852 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:38.847+0000 7f8db3592700 1 -- 172.21.15.59:0/3570136001 wait complete. 2023-11-24T07:56:38.855 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:38.848+0000 7f8db3592700 1 Processor -- start 2023-11-24T07:56:38.856 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:38.848+0000 7f8db3592700 1 -- start start 2023-11-24T07:56:38.856 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:38.849+0000 7f8db3592700 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f8dac0720f0 0x7f8dac1f1250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:56:38.856 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:38.849+0000 7f8db3592700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8dac143ae0 0x7f8dac1ee460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:56:38.856 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:38.849+0000 7f8db3592700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8dac1440f0 0x7f8dac15c8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:56:38.856 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:38.849+0000 7f8db3592700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8dac1f1790 con 0x7f8dac1440f0 2023-11-24T07:56:38.856 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:38.849+0000 7f8db3592700 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8dac1f2080 con 0x7f8dac143ae0 2023-11-24T07:56:38.856 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:38.849+0000 7f8db3592700 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8dac1f2360 con 0x7f8dac0720f0 2023-11-24T07:56:38.856 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:38.849+0000 7f8db1b2f700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8dac1440f0 0x7f8dac15c8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:56:38.856 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:38.849+0000 7f8db1b2f700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8dac1440f0 0x7f8dac15c8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:50318/0 (socket says 172.21.15.59:50318) 2023-11-24T07:56:38.856 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:38.849+0000 7f8db1b2f700 1 -- 172.21.15.59:0/2204295795 learned_addr learned my addr 172.21.15.59:0/2204295795 (peer_addr_for_me v2:172.21.15.59:0/0) 2023-11-24T07:56:38.856 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:38.850+0000 7f8db0b2d700 1 --2- 172.21.15.59:0/2204295795 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8dac143ae0 0x7f8dac1ee460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:56:38.857 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:38.850+0000 7f8db132e700 1 --2- 172.21.15.59:0/2204295795 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f8dac0720f0 0x7f8dac1f1250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:56:38.857 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:38.850+0000 7f8db1b2f700 1 -- 172.21.15.59:0/2204295795 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f8dac0720f0 msgr2=0x7f8dac1f1250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:56:38.857 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:38.850+0000 7f8db1b2f700 1 --2- 172.21.15.59:0/2204295795 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f8dac0720f0 0x7f8dac1f1250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:56:38.857 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:38.850+0000 7f8db1b2f700 1 -- 172.21.15.59:0/2204295795 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8dac143ae0 msgr2=0x7f8dac1ee460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:56:38.857 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:38.850+0000 7f8db1b2f700 1 --2- 172.21.15.59:0/2204295795 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8dac143ae0 0x7f8dac1ee460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:56:38.857 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:38.850+0000 7f8db1b2f700 1 -- 172.21.15.59:0/2204295795 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8da4009b30 con 0x7f8dac1440f0 2023-11-24T07:56:38.857 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:38.850+0000 7f8db0b2d700 1 --2- 172.21.15.59:0/2204295795 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8dac143ae0 0x7f8dac1ee460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-24T07:56:38.857 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:38.850+0000 7f8db132e700 1 --2- 172.21.15.59:0/2204295795 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f8dac0720f0 0x7f8dac1f1250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T07:56:38.857 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:38.850+0000 7f8db1b2f700 1 --2- 172.21.15.59:0/2204295795 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8dac1440f0 0x7f8dac15c8b0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f8dac0d1bd0 tx=0x7f8da4004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:56:38.857 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:38.850+0000 7f8da1ffb700 1 -- 172.21.15.59:0/2204295795 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8da400ce80 con 0x7f8dac1440f0 2023-11-24T07:56:38.857 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:38.850+0000 7f8da1ffb700 1 -- 172.21.15.59:0/2204295795 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8da4004250 con 0x7f8dac1440f0 2023-11-24T07:56:38.857 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:38.850+0000 7f8da1ffb700 1 -- 172.21.15.59:0/2204295795 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8da4004600 con 0x7f8dac1440f0 2023-11-24T07:56:38.858 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:38.850+0000 7f8db3592700 1 -- 172.21.15.59:0/2204295795 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8dac1f2640 con 0x7f8dac1440f0 2023-11-24T07:56:38.858 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:38.850+0000 7f8db3592700 1 -- 172.21.15.59:0/2204295795 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8dac1f2c00 con 0x7f8dac1440f0 2023-11-24T07:56:38.858 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:38.851+0000 7f8db3592700 1 -- 172.21.15.59:0/2204295795 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8dac068670 con 0x7f8dac1440f0 2023-11-24T07:56:38.858 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:38.852+0000 7f8da1ffb700 1 -- 172.21.15.59:0/2204295795 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f8da4004760 con 0x7f8dac1440f0 2023-11-24T07:56:38.858 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:38.852+0000 7f8da1ffb700 1 --2- 172.21.15.59:0/2204295795 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f8d9006aed0 0x7f8d9006d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:56:38.858 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:38.852+0000 7f8da1ffb700 1 -- 172.21.15.59:0/2204295795 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f8da408a690 con 0x7f8dac1440f0 2023-11-24T07:56:38.858 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:38.852+0000 7f8db132e700 1 --2- 172.21.15.59:0/2204295795 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f8d9006aed0 0x7f8d9006d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:56:38.858 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:38.853+0000 7f8db132e700 1 --2- 172.21.15.59:0/2204295795 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f8d9006aed0 0x7f8d9006d390 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f8dac15da20 tx=0x7f8d9c008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:56:38.859 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:38.856+0000 7f8da1ffb700 1 -- 172.21.15.59:0/2204295795 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f8da40557d0 con 0x7f8dac1440f0 2023-11-24T07:56:39.104 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:39.102+0000 7f8db3592700 1 -- 172.21.15.59:0/2204295795 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7f8dac144830 con 0x7f8dac1440f0 2023-11-24T07:56:39.104 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:39.102+0000 7f8da1ffb700 1 -- 172.21.15.59:0/2204295795 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+46 (secure 0 0 0) 0x7f8da4058df0 con 0x7f8dac1440f0 2023-11-24T07:56:39.104 INFO:teuthology.orchestra.run.smithi059.stdout: 2023-11-24T07:56:39.104 INFO:teuthology.orchestra.run.smithi059.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2023-11-24T07:56:39.107 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:39.104+0000 7f8db3592700 1 -- 172.21.15.59:0/2204295795 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f8d9006aed0 msgr2=0x7f8d9006d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:56:39.107 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:39.104+0000 7f8db3592700 1 --2- 172.21.15.59:0/2204295795 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f8d9006aed0 0x7f8d9006d390 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f8dac15da20 tx=0x7f8d9c008040 comp rx=0 tx=0).stop 2023-11-24T07:56:39.107 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:39.104+0000 7f8db3592700 1 -- 172.21.15.59:0/2204295795 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8dac1440f0 msgr2=0x7f8dac15c8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:56:39.107 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:39.104+0000 7f8db3592700 1 --2- 172.21.15.59:0/2204295795 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8dac1440f0 0x7f8dac15c8b0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f8dac0d1bd0 tx=0x7f8da4004ad0 comp rx=0 tx=0).stop 2023-11-24T07:56:39.107 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:39.104+0000 7f8db3592700 1 -- 172.21.15.59:0/2204295795 shutdown_connections 2023-11-24T07:56:39.107 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:39.105+0000 7f8db3592700 1 --2- 172.21.15.59:0/2204295795 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f8d9006aed0 0x7f8d9006d390 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:56:39.107 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:39.105+0000 7f8db3592700 1 --2- 172.21.15.59:0/2204295795 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8dac1440f0 0x7f8dac15c8b0 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:56:39.107 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:39.105+0000 7f8db3592700 1 --2- 172.21.15.59:0/2204295795 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f8dac0720f0 0x7f8dac1f1250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:56:39.107 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:39.105+0000 7f8db3592700 1 --2- 172.21.15.59:0/2204295795 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8dac143ae0 0x7f8dac1ee460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:56:39.107 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:39.105+0000 7f8db3592700 1 -- 172.21.15.59:0/2204295795 >> 172.21.15.59:0/2204295795 conn(0x7f8dac0b2740 msgr2=0x7f8dac1595a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:56:39.107 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:39.105+0000 7f8db3592700 1 -- 172.21.15.59:0/2204295795 shutdown_connections 2023-11-24T07:56:39.108 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:39.105+0000 7f8db3592700 1 -- 172.21.15.59:0/2204295795 wait complete. 2023-11-24T07:56:39.118 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy done 2023-11-24T07:56:39.119 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-24T07:56:39.243 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:39.241+0000 7f35d982c700 1 Processor -- start 2023-11-24T07:56:39.243 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:39.241+0000 7f35d982c700 1 -- start start 2023-11-24T07:56:39.245 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:39.242+0000 7f35d982c700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f35d415e460 0x7f35d40d49d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:56:39.245 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:39.242+0000 7f35d982c700 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f35d40d2ba0 0x7f35d40b40a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:56:39.245 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:39.242+0000 7f35d982c700 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f35d40b3370 con 0x7f35d40cf480 2023-11-24T07:56:39.246 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:39.242+0000 7f35d982c700 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f35d40cfc40 con 0x7f35d415e460 2023-11-24T07:56:39.246 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:39.242+0000 7f35d982c700 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f35d40b35c0 con 0x7f35d40d2ba0 2023-11-24T07:56:39.246 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:39.242+0000 7f35d2ffd700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f35d415e460 0x7f35d40d49d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:56:39.246 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:39.242+0000 7f35d2ffd700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f35d415e460 0x7f35d40d49d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:50322/0 (socket says 172.21.15.59:50322) 2023-11-24T07:56:39.246 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:39.242+0000 7f35d2ffd700 1 -- 172.21.15.59:0/197021228 learned_addr learned my addr 172.21.15.59:0/197021228 (peer_addr_for_me v2:172.21.15.59:0/0) 2023-11-24T07:56:39.246 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:39.242+0000 7f35d37fe700 1 --2- 172.21.15.59:0/197021228 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f35d40d2ba0 0x7f35d40b40a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:56:39.246 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:39.242+0000 7f35d2ffd700 1 -- 172.21.15.59:0/197021228 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f35d40d2ba0 msgr2=0x7f35d40b40a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T07:56:39.246 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:39.242+0000 7f35d2ffd700 1 --2- 172.21.15.59:0/197021228 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f35d40d2ba0 0x7f35d40b40a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:56:39.246 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:39.242+0000 7f35d2ffd700 1 -- 172.21.15.59:0/197021228 >> v1:172.21.15.76:6789/0 conn(0x7f35d40cf480 legacy=0x7f35d40d2490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:56:39.247 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:39.242+0000 7f35d2ffd700 1 -- 172.21.15.59:0/197021228 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f35d40b38a0 con 0x7f35d415e460 2023-11-24T07:56:39.247 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:39.242+0000 7f35d2ffd700 1 --2- 172.21.15.59:0/197021228 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f35d415e460 0x7f35d40d49d0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f35bc0098f0 tx=0x7f35bc009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bccc1631825c8da8 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:56:39.247 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:39.243+0000 7f35d1ffb700 1 -- 172.21.15.59:0/197021228 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35bc011830 con 0x7f35d415e460 2023-11-24T07:56:39.247 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:39.243+0000 7f35d1ffb700 1 -- 172.21.15.59:0/197021228 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f35bc011990 con 0x7f35d415e460 2023-11-24T07:56:39.247 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:39.243+0000 7f35d1ffb700 1 -- 172.21.15.59:0/197021228 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35bc011d20 con 0x7f35d415e460 2023-11-24T07:56:39.247 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:39.243+0000 7f35d982c700 1 -- 172.21.15.59:0/197021228 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f35d415e460 msgr2=0x7f35d40d49d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:56:39.247 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:39.243+0000 7f35d982c700 1 --2- 172.21.15.59:0/197021228 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f35d415e460 0x7f35d40d49d0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f35bc0098f0 tx=0x7f35bc009c00 comp rx=0 tx=0).stop 2023-11-24T07:56:39.247 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:39.243+0000 7f35d982c700 1 -- 172.21.15.59:0/197021228 shutdown_connections 2023-11-24T07:56:39.247 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:39.243+0000 7f35d982c700 1 --2- 172.21.15.59:0/197021228 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f35d40d2ba0 0x7f35d40b40a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:56:39.247 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:39.243+0000 7f35d982c700 1 --2- 172.21.15.59:0/197021228 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f35d415e460 0x7f35d40d49d0 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:56:39.248 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:39.243+0000 7f35d982c700 1 -- 172.21.15.59:0/197021228 >> 172.21.15.59:0/197021228 conn(0x7f35d40d3eb0 msgr2=0x7f35d40d42c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:56:39.248 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:39.243+0000 7f35d982c700 1 -- 172.21.15.59:0/197021228 shutdown_connections 2023-11-24T07:56:39.248 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:39.243+0000 7f35d982c700 1 -- 172.21.15.59:0/197021228 wait complete. 2023-11-24T07:56:39.250 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:39.244+0000 7f35d982c700 1 Processor -- start 2023-11-24T07:56:39.250 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:39.245+0000 7f35d982c700 1 -- start start 2023-11-24T07:56:39.251 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:39.245+0000 7f35d982c700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f35d40cf480 0x7f35d41f3c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:56:39.251 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:39.245+0000 7f35d982c700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f35d40d2ba0 0x7f35d414e360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:56:39.251 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:39.245+0000 7f35d982c700 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f35d415e460 0x7f35d40b9520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:56:39.251 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:39.245+0000 7f35d982c700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35d414e8a0 con 0x7f35d40d2ba0 2023-11-24T07:56:39.251 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:39.245+0000 7f35d982c700 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35d415bd50 con 0x7f35d40cf480 2023-11-24T07:56:39.251 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:39.245+0000 7f35d982c700 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f35d40cc0b0 con 0x7f35d415e460 2023-11-24T07:56:39.251 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:39.246+0000 7f35d27fc700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f35d40d2ba0 0x7f35d414e360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:56:39.251 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:39.246+0000 7f35d27fc700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f35d40d2ba0 0x7f35d414e360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:50330/0 (socket says 172.21.15.59:50330) 2023-11-24T07:56:39.251 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:39.246+0000 7f35d27fc700 1 -- 172.21.15.59:0/3616737188 learned_addr learned my addr 172.21.15.59:0/3616737188 (peer_addr_for_me v2:172.21.15.59:0/0) 2023-11-24T07:56:39.251 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:39.246+0000 7f35d2ffd700 1 --2- 172.21.15.59:0/3616737188 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f35d40cf480 0x7f35d41f3c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:56:39.251 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:39.246+0000 7f35d37fe700 1 --2- 172.21.15.59:0/3616737188 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f35d415e460 0x7f35d40b9520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:56:39.251 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:39.246+0000 7f35d27fc700 1 -- 172.21.15.59:0/3616737188 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f35d415e460 msgr2=0x7f35d40b9520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:56:39.252 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:39.246+0000 7f35d27fc700 1 --2- 172.21.15.59:0/3616737188 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f35d415e460 0x7f35d40b9520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:56:39.252 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:39.246+0000 7f35d27fc700 1 -- 172.21.15.59:0/3616737188 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f35d40cf480 msgr2=0x7f35d41f3c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:56:39.252 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:39.246+0000 7f35d27fc700 1 --2- 172.21.15.59:0/3616737188 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f35d40cf480 0x7f35d41f3c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:56:39.252 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:39.246+0000 7f35d27fc700 1 -- 172.21.15.59:0/3616737188 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f35bc009550 con 0x7f35d40d2ba0 2023-11-24T07:56:39.252 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:39.246+0000 7f35d37fe700 1 --2- 172.21.15.59:0/3616737188 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f35d415e460 0x7f35d40b9520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T07:56:39.252 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:39.246+0000 7f35d2ffd700 1 --2- 172.21.15.59:0/3616737188 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f35d40cf480 0x7f35d41f3c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T07:56:39.252 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:39.246+0000 7f35d27fc700 1 --2- 172.21.15.59:0/3616737188 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f35d40d2ba0 0x7f35d414e360 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f35c400c4e0 tx=0x7f35c400c8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:56:39.252 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:39.246+0000 7f35cbfff700 1 -- 172.21.15.59:0/3616737188 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35c40052d0 con 0x7f35d40d2ba0 2023-11-24T07:56:39.252 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:39.246+0000 7f35cbfff700 1 -- 172.21.15.59:0/3616737188 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f35c4005430 con 0x7f35d40d2ba0 2023-11-24T07:56:39.252 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:39.246+0000 7f35cbfff700 1 -- 172.21.15.59:0/3616737188 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35c40038c0 con 0x7f35d40d2ba0 2023-11-24T07:56:39.252 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:39.246+0000 7f35d982c700 1 -- 172.21.15.59:0/3616737188 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f35d40cc330 con 0x7f35d40d2ba0 2023-11-24T07:56:39.253 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:39.246+0000 7f35d982c700 1 -- 172.21.15.59:0/3616737188 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f35d40cc8f0 con 0x7f35d40d2ba0 2023-11-24T07:56:39.253 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:39.247+0000 7f35d982c700 1 -- 172.21.15.59:0/3616737188 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f35d4067f60 con 0x7f35d40d2ba0 2023-11-24T07:56:39.253 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:39.248+0000 7f35cbfff700 1 -- 172.21.15.59:0/3616737188 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f35c4003a20 con 0x7f35d40d2ba0 2023-11-24T07:56:39.253 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:39.248+0000 7f35cbfff700 1 --2- 172.21.15.59:0/3616737188 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f35b406aed0 0x7f35b406d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:56:39.253 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:39.248+0000 7f35cbfff700 1 -- 172.21.15.59:0/3616737188 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f35c4088ab0 con 0x7f35d40d2ba0 2023-11-24T07:56:39.253 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:39.249+0000 7f35d2ffd700 1 --2- 172.21.15.59:0/3616737188 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f35b406aed0 0x7f35b406d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:56:39.253 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:39.249+0000 7f35d2ffd700 1 --2- 172.21.15.59:0/3616737188 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f35b406aed0 0x7f35b406d390 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f35bc014040 tx=0x7f35bc008030 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:56:39.254 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:39.253+0000 7f35cbfff700 1 -- 172.21.15.59:0/3616737188 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f35c4053b60 con 0x7f35d40d2ba0 2023-11-24T07:56:39.456 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:39.454+0000 7f35d982c700 1 -- 172.21.15.59:0/3616737188 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f35d40d2270 con 0x7f35d40d2ba0 2023-11-24T07:56:39.457 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:39.456+0000 7f35cbfff700 1 -- 172.21.15.59:0/3616737188 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) v1 ==== 74+0+16437 (secure 0 0 0) 0x7f35c4017090 con 0x7f35d40d2ba0 2023-11-24T07:56:39.457 INFO:teuthology.orchestra.run.smithi059.stdout: 2023-11-24T07:56:39.458 INFO:teuthology.orchestra.run.smithi059.stdout:{"epoch":20,"fsid":"9f0e8741-b8f9-48d5-a2c3-09f45903303d","created":"2023-11-24T07:56:06.552729+0000","modified":"2023-11-24T07:56:25.705091+0000","last_up_change":"2023-11-24T07:56:17.372352+0000","last_in_change":"2023-11-24T07:56:09.073250+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-24T07:56:18.352823+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-11-24T07:56:22.185140+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-11-24T07:56:23.084326+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"6d8f9e9b-1782-4e41-928b-a39f2a8ebc06","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6800","nonce":3373878772},{"type":"v1","addr":"172.21.15.59:6801","nonce":3373878772}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6802","nonce":3373878772},{"type":"v1","addr":"172.21.15.59:6803","nonce":3373878772}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6806","nonce":3373878772},{"type":"v1","addr":"172.21.15.59:6807","nonce":3373878772}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6804","nonce":3373878772},{"type":"v1","addr":"172.21.15.59:6805","nonce":3373878772}]},"public_addr":"172.21.15.59:6801/3373878772","cluster_addr":"172.21.15.59:6803/3373878772","heartbeat_back_addr":"172.21.15.59:6807/3373878772","heartbeat_front_addr":"172.21.15.59:6805/3373878772","state":["exists","up"]},{"osd":1,"uuid":"e8d882aa-52e1-4c04-9e1e-c44ef9204a5f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6824","nonce":648258563},{"type":"v1","addr":"172.21.15.59:6825","nonce":648258563}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6826","nonce":648258563},{"type":"v1","addr":"172.21.15.59:6827","nonce":648258563}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6830","nonce":648258563},{"type":"v1","addr":"172.21.15.59:6831","nonce":648258563}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6828","nonce":648258563},{"type":"v1","addr":"172.21.15.59:6829","nonce":648258563}]},"public_addr":"172.21.15.59:6825/648258563","cluster_addr":"172.21.15.59:6827/648258563","heartbeat_back_addr":"172.21.15.59:6831/648258563","heartbeat_front_addr":"172.21.15.59:6829/648258563","state":["exists","up"]},{"osd":2,"uuid":"69eebfa0-695b-4b21-90b9-8a3537168a01","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6808","nonce":1840430654},{"type":"v1","addr":"172.21.15.59:6809","nonce":1840430654}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6810","nonce":1840430654},{"type":"v1","addr":"172.21.15.59:6811","nonce":1840430654}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6814","nonce":1840430654},{"type":"v1","addr":"172.21.15.59:6815","nonce":1840430654}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6812","nonce":1840430654},{"type":"v1","addr":"172.21.15.59:6813","nonce":1840430654}]},"public_addr":"172.21.15.59:6809/1840430654","cluster_addr":"172.21.15.59:6811/1840430654","heartbeat_back_addr":"172.21.15.59:6815/1840430654","heartbeat_front_addr":"172.21.15.59:6813/1840430654","state":["exists","up"]},{"osd":3,"uuid":"96fd77fc-f911-46ff-9445-6f0be2daa8f4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6816","nonce":2504025850},{"type":"v1","addr":"172.21.15.59:6817","nonce":2504025850}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6818","nonce":2504025850},{"type":"v1","addr":"172.21.15.59:6819","nonce":2504025850}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6822","nonce":2504025850},{"type":"v1","addr":"172.21.15.59:6823","nonce":2504025850}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6820","nonce":2504025850},{"type":"v1","addr":"172.21.15.59:6821","nonce":2504025850}]},"public_addr":"172.21.15.59:6817/2504025850","cluster_addr":"172.21.15.59:6819/2504025850","heartbeat_back_addr":"172.21.15.59:6823/2504025850","heartbeat_front_addr":"172.21.15.59:6821/2504025850","state":["exists","up"]},{"osd":4,"uuid":"7ac29fca-b5b2-4c95-9d69-2f164cbbc708","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6816","nonce":2382463431},{"type":"v1","addr":"172.21.15.76:6818","nonce":2382463431}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6820","nonce":2382463431},{"type":"v1","addr":"172.21.15.76:6822","nonce":2382463431}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6828","nonce":2382463431},{"type":"v1","addr":"172.21.15.76:6830","nonce":2382463431}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6824","nonce":2382463431},{"type":"v1","addr":"172.21.15.76:6826","nonce":2382463431}]},"public_addr":"172.21.15.76:6818/2382463431","cluster_addr":"172.21.15.76:6822/2382463431","heartbeat_back_addr":"172.21.15.76:6830/2382463431","heartbeat_front_addr":"172.21.15.76:6826/2382463431","state":["exists","up"]},{"osd":5,"uuid":"485bf711-3851-4238-a4f4-51d52e0a9c58","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6800","nonce":329334308},{"type":"v1","addr":"172.21.15.76:6802","nonce":329334308}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6805","nonce":329334308},{"type":"v1","addr":"172.21.15.76:6807","nonce":329334308}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6814","nonce":329334308},{"type":"v1","addr":"172.21.15.76:6815","nonce":329334308}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6810","nonce":329334308},{"type":"v1","addr":"172.21.15.76:6812","nonce":329334308}]},"public_addr":"172.21.15.76:6802/329334308","cluster_addr":"172.21.15.76:6807/329334308","heartbeat_back_addr":"172.21.15.76:6815/329334308","heartbeat_front_addr":"172.21.15.76:6812/329334308","state":["exists","up"]},{"osd":6,"uuid":"72b3f2fc-1e4a-4e32-968d-003647a0286f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6801","nonce":3946579410},{"type":"v1","addr":"172.21.15.76:6803","nonce":3946579410}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6804","nonce":3946579410},{"type":"v1","addr":"172.21.15.76:6806","nonce":3946579410}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6811","nonce":3946579410},{"type":"v1","addr":"172.21.15.76:6813","nonce":3946579410}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6808","nonce":3946579410},{"type":"v1","addr":"172.21.15.76:6809","nonce":3946579410}]},"public_addr":"172.21.15.76:6803/3946579410","cluster_addr":"172.21.15.76:6806/3946579410","heartbeat_back_addr":"172.21.15.76:6813/3946579410","heartbeat_front_addr":"172.21.15.76:6809/3946579410","state":["exists","up"]},{"osd":7,"uuid":"1ddceb90-6c5d-4d1a-ad56-99cb877f8c70","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6817","nonce":841761538},{"type":"v1","addr":"172.21.15.76:6819","nonce":841761538}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6821","nonce":841761538},{"type":"v1","addr":"172.21.15.76:6823","nonce":841761538}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6829","nonce":841761538},{"type":"v1","addr":"172.21.15.76:6831","nonce":841761538}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6825","nonce":841761538},{"type":"v1","addr":"172.21.15.76:6827","nonce":841761538}]},"public_addr":"172.21.15.76:6819/841761538","cluster_addr":"172.21.15.76:6823/841761538","heartbeat_back_addr":"172.21.15.76:6831/841761538","heartbeat_front_addr":"172.21.15.76:6827/841761538","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T07:56:14.294838+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T07:56:14.270779+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T07:56:14.231809+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T07:56:14.260983+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T07:56:14.302997+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T07:56:14.258600+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T07:56:14.251413+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T07:56:14.304727+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-24T07:56:39.461 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:39.458+0000 7f35d982c700 1 -- 172.21.15.59:0/3616737188 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f35b406aed0 msgr2=0x7f35b406d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:56:39.461 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:39.458+0000 7f35d982c700 1 --2- 172.21.15.59:0/3616737188 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f35b406aed0 0x7f35b406d390 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f35bc014040 tx=0x7f35bc008030 comp rx=0 tx=0).stop 2023-11-24T07:56:39.461 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:39.458+0000 7f35d982c700 1 -- 172.21.15.59:0/3616737188 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f35d40d2ba0 msgr2=0x7f35d414e360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:56:39.461 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:39.458+0000 7f35d982c700 1 --2- 172.21.15.59:0/3616737188 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f35d40d2ba0 0x7f35d414e360 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f35c400c4e0 tx=0x7f35c400c8a0 comp rx=0 tx=0).stop 2023-11-24T07:56:39.461 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:39.458+0000 7f35d982c700 1 -- 172.21.15.59:0/3616737188 shutdown_connections 2023-11-24T07:56:39.461 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:39.458+0000 7f35d982c700 1 --2- 172.21.15.59:0/3616737188 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f35b406aed0 0x7f35b406d390 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:56:39.461 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:39.458+0000 7f35d982c700 1 --2- 172.21.15.59:0/3616737188 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f35d415e460 0x7f35d40b9520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:56:39.461 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:39.458+0000 7f35d982c700 1 --2- 172.21.15.59:0/3616737188 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f35d40cf480 0x7f35d41f3c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:56:39.461 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:39.458+0000 7f35d982c700 1 --2- 172.21.15.59:0/3616737188 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f35d40d2ba0 0x7f35d414e360 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:56:39.462 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:39.458+0000 7f35d982c700 1 -- 172.21.15.59:0/3616737188 >> 172.21.15.59:0/3616737188 conn(0x7f35d40d3eb0 msgr2=0x7f35d40cfd80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:56:39.462 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:39.458+0000 7f35d982c700 1 -- 172.21.15.59:0/3616737188 shutdown_connections 2023-11-24T07:56:39.462 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:39.458+0000 7f35d982c700 1 -- 172.21.15.59:0/3616737188 wait complete. 2023-11-24T07:56:39.472 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-24T07:56:18.352823+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-24T07:56:22.185140+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-11-24T07:56:23.084326+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-24T07:56:39.472 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-24T07:56:39.599 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:39.598+0000 7fe85fff6700 1 Processor -- start 2023-11-24T07:56:39.600 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:39.598+0000 7fe85fff6700 1 -- start start 2023-11-24T07:56:39.602 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:39.599+0000 7fe85fff6700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fe858143ae0 0x7fe8580b51b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:56:39.602 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:39.599+0000 7fe85fff6700 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fe8581440f0 0x7fe858071bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:56:39.603 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:39.599+0000 7fe85fff6700 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe858143ec0 con 0x7fe8580720f0 2023-11-24T07:56:39.603 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:39.599+0000 7fe85fff6700 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe8580b40a0 con 0x7fe858143ae0 2023-11-24T07:56:39.603 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:39.599+0000 7fe85fff6700 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe858072580 con 0x7fe8581440f0 2023-11-24T07:56:39.603 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:39.599+0000 7fe85dd92700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fe858143ae0 0x7fe8580b51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:56:39.603 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:39.599+0000 7fe85dd92700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fe858143ae0 0x7fe8580b51b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:50336/0 (socket says 172.21.15.59:50336) 2023-11-24T07:56:39.603 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:39.599+0000 7fe85dd92700 1 -- 172.21.15.59:0/640733254 learned_addr learned my addr 172.21.15.59:0/640733254 (peer_addr_for_me v2:172.21.15.59:0/0) 2023-11-24T07:56:39.603 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:39.599+0000 7fe85d591700 1 --2- 172.21.15.59:0/640733254 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fe8581440f0 0x7fe858071bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:56:39.604 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:39.599+0000 7fe85dd92700 1 -- 172.21.15.59:0/640733254 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fe8581440f0 msgr2=0x7fe858071bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T07:56:39.604 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:39.599+0000 7fe85dd92700 1 --2- 172.21.15.59:0/640733254 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fe8581440f0 0x7fe858071bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:56:39.604 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:39.599+0000 7fe85dd92700 1 -- 172.21.15.59:0/640733254 >> v1:172.21.15.76:6789/0 conn(0x7fe8580720f0 legacy=0x7fe85815e8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:56:39.604 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:39.599+0000 7fe85dd92700 1 -- 172.21.15.59:0/640733254 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe8580b3ae0 con 0x7fe858143ae0 2023-11-24T07:56:39.604 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:39.599+0000 7fe85dd92700 1 --2- 172.21.15.59:0/640733254 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe858143ae0 0x7fe8580b51b0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fe85400bb90 tx=0x7fe85400bea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=df0331ae995c5ed5 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:56:39.604 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:39.599+0000 7fe85cd90700 1 -- 172.21.15.59:0/640733254 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe85400dd50 con 0x7fe858143ae0 2023-11-24T07:56:39.604 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:39.600+0000 7fe85cd90700 1 -- 172.21.15.59:0/640733254 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe854010470 con 0x7fe858143ae0 2023-11-24T07:56:39.605 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:39.600+0000 7fe85cd90700 1 -- 172.21.15.59:0/640733254 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe854010800 con 0x7fe858143ae0 2023-11-24T07:56:39.605 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:39.600+0000 7fe85fff6700 1 -- 172.21.15.59:0/640733254 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe858143ae0 msgr2=0x7fe8580b51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:56:39.605 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:39.600+0000 7fe85fff6700 1 --2- 172.21.15.59:0/640733254 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe858143ae0 0x7fe8580b51b0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fe85400bb90 tx=0x7fe85400bea0 comp rx=0 tx=0).stop 2023-11-24T07:56:39.605 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:39.600+0000 7fe85fff6700 1 -- 172.21.15.59:0/640733254 shutdown_connections 2023-11-24T07:56:39.605 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:39.600+0000 7fe85fff6700 1 --2- 172.21.15.59:0/640733254 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe858143ae0 0x7fe8580b51b0 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:56:39.605 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:39.600+0000 7fe85fff6700 1 --2- 172.21.15.59:0/640733254 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fe8581440f0 0x7fe858071bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:56:39.605 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:39.600+0000 7fe85fff6700 1 -- 172.21.15.59:0/640733254 >> 172.21.15.59:0/640733254 conn(0x7fe8580b2740 msgr2=0x7fe8580b2b20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:56:39.605 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:39.600+0000 7fe85fff6700 1 -- 172.21.15.59:0/640733254 shutdown_connections 2023-11-24T07:56:39.606 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:39.600+0000 7fe85fff6700 1 -- 172.21.15.59:0/640733254 wait complete. 2023-11-24T07:56:39.608 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:39.601+0000 7fe85fff6700 1 Processor -- start 2023-11-24T07:56:39.608 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:39.601+0000 7fe85fff6700 1 -- start start 2023-11-24T07:56:39.608 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:39.602+0000 7fe85fff6700 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fe8580720f0 0x7fe8581f3560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:56:39.609 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:39.602+0000 7fe85fff6700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe858143ae0 0x7fe858152200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:56:39.609 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:39.602+0000 7fe85fff6700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe8581440f0 0x7fe85814f840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:56:39.609 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:39.602+0000 7fe85fff6700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe858142b90 con 0x7fe858143ae0 2023-11-24T07:56:39.609 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:39.602+0000 7fe85fff6700 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe858154d80 con 0x7fe8581440f0 2023-11-24T07:56:39.609 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:39.602+0000 7fe85fff6700 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe858155050 con 0x7fe8580720f0 2023-11-24T07:56:39.610 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:39.602+0000 7fe85d591700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe858143ae0 0x7fe858152200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:56:39.610 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:39.602+0000 7fe85d591700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe858143ae0 0x7fe858152200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:50344/0 (socket says 172.21.15.59:50344) 2023-11-24T07:56:39.610 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:39.602+0000 7fe85d591700 1 -- 172.21.15.59:0/2687076608 learned_addr learned my addr 172.21.15.59:0/2687076608 (peer_addr_for_me v2:172.21.15.59:0/0) 2023-11-24T07:56:39.610 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:39.602+0000 7fe85dd92700 1 --2- 172.21.15.59:0/2687076608 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fe8580720f0 0x7fe8581f3560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:56:39.610 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:39.602+0000 7fe85e593700 1 --2- 172.21.15.59:0/2687076608 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe8581440f0 0x7fe85814f840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:56:39.610 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:39.602+0000 7fe85d591700 1 -- 172.21.15.59:0/2687076608 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fe8580720f0 msgr2=0x7fe8581f3560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:56:39.611 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:39.602+0000 7fe85d591700 1 --2- 172.21.15.59:0/2687076608 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fe8580720f0 0x7fe8581f3560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:56:39.611 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:39.602+0000 7fe85d591700 1 -- 172.21.15.59:0/2687076608 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe8581440f0 msgr2=0x7fe85814f840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:56:39.611 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:39.602+0000 7fe85d591700 1 --2- 172.21.15.59:0/2687076608 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe8581440f0 0x7fe85814f840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:56:39.611 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:39.602+0000 7fe85d591700 1 -- 172.21.15.59:0/2687076608 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe85400b7f0 con 0x7fe858143ae0 2023-11-24T07:56:39.611 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:39.602+0000 7fe85e593700 1 --2- 172.21.15.59:0/2687076608 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe8581440f0 0x7fe85814f840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-24T07:56:39.611 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:39.603+0000 7fe85d591700 1 --2- 172.21.15.59:0/2687076608 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe858143ae0 0x7fe858152200 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fe84800d800 tx=0x7fe84800db10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:56:39.611 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:39.603+0000 7fe84e7fc700 1 -- 172.21.15.59:0/2687076608 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8480125f0 con 0x7fe858143ae0 2023-11-24T07:56:39.611 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:39.603+0000 7fe84e7fc700 1 -- 172.21.15.59:0/2687076608 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe848012750 con 0x7fe858143ae0 2023-11-24T07:56:39.612 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:39.603+0000 7fe85fff6700 1 -- 172.21.15.59:0/2687076608 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe858155330 con 0x7fe858143ae0 2023-11-24T07:56:39.612 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:39.603+0000 7fe84e7fc700 1 -- 172.21.15.59:0/2687076608 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe848012b00 con 0x7fe858143ae0 2023-11-24T07:56:39.612 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:39.603+0000 7fe85fff6700 1 -- 172.21.15.59:0/2687076608 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe8581558f0 con 0x7fe858143ae0 2023-11-24T07:56:39.612 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:39.604+0000 7fe85fff6700 1 -- 172.21.15.59:0/2687076608 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe858075480 con 0x7fe858143ae0 2023-11-24T07:56:39.612 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:39.604+0000 7fe84e7fc700 1 -- 172.21.15.59:0/2687076608 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fe848012c60 con 0x7fe858143ae0 2023-11-24T07:56:39.612 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:39.604+0000 7fe84e7fc700 1 --2- 172.21.15.59:0/2687076608 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7fe83c06aed0 0x7fe83c06d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:56:39.612 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:39.605+0000 7fe84e7fc700 1 -- 172.21.15.59:0/2687076608 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fe8480898a0 con 0x7fe858143ae0 2023-11-24T07:56:39.612 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:39.605+0000 7fe85dd92700 1 --2- 172.21.15.59:0/2687076608 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7fe83c06aed0 0x7fe83c06d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:56:39.613 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:39.605+0000 7fe85dd92700 1 --2- 172.21.15.59:0/2687076608 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7fe83c06aed0 0x7fe83c06d390 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fe85400bee0 tx=0x7fe85401c040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:56:39.613 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:39.609+0000 7fe84e7fc700 1 -- 172.21.15.59:0/2687076608 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fe848054950 con 0x7fe858143ae0 2023-11-24T07:56:39.812 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:39.811+0000 7fe85fff6700 1 -- 172.21.15.59:0/2687076608 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fe858144830 con 0x7fe858143ae0 2023-11-24T07:56:39.813 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:39.811+0000 7fe84e7fc700 1 -- 172.21.15.59:0/2687076608 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v20) v1 ==== 93+0+10 (secure 0 0 0) 0x7fe848057f70 con 0x7fe858143ae0 2023-11-24T07:56:39.813 INFO:teuthology.orchestra.run.smithi059.stdout:pg_num: 1 2023-11-24T07:56:39.815 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:39.813+0000 7fe85fff6700 1 -- 172.21.15.59:0/2687076608 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7fe83c06aed0 msgr2=0x7fe83c06d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:56:39.815 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:39.813+0000 7fe85fff6700 1 --2- 172.21.15.59:0/2687076608 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7fe83c06aed0 0x7fe83c06d390 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fe85400bee0 tx=0x7fe85401c040 comp rx=0 tx=0).stop 2023-11-24T07:56:39.815 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:39.813+0000 7fe85fff6700 1 -- 172.21.15.59:0/2687076608 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe858143ae0 msgr2=0x7fe858152200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:56:39.816 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:39.813+0000 7fe85fff6700 1 --2- 172.21.15.59:0/2687076608 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe858143ae0 0x7fe858152200 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fe84800d800 tx=0x7fe84800db10 comp rx=0 tx=0).stop 2023-11-24T07:56:39.816 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:39.813+0000 7fe85fff6700 1 -- 172.21.15.59:0/2687076608 shutdown_connections 2023-11-24T07:56:39.816 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:39.813+0000 7fe85fff6700 1 --2- 172.21.15.59:0/2687076608 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7fe83c06aed0 0x7fe83c06d390 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:56:39.816 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:39.813+0000 7fe85fff6700 1 --2- 172.21.15.59:0/2687076608 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe8581440f0 0x7fe85814f840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:56:39.816 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:39.813+0000 7fe85fff6700 1 --2- 172.21.15.59:0/2687076608 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fe8580720f0 0x7fe8581f3560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:56:39.816 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:39.813+0000 7fe85fff6700 1 --2- 172.21.15.59:0/2687076608 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe858143ae0 0x7fe858152200 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:56:39.816 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:39.813+0000 7fe85fff6700 1 -- 172.21.15.59:0/2687076608 >> 172.21.15.59:0/2687076608 conn(0x7fe8580b2740 msgr2=0x7fe8580ce9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:56:39.816 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:39.813+0000 7fe85fff6700 1 -- 172.21.15.59:0/2687076608 shutdown_connections 2023-11-24T07:56:39.816 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:39.813+0000 7fe85fff6700 1 -- 172.21.15.59:0/2687076608 wait complete. 2023-11-24T07:56:39.827 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-24T07:56:39.956 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:39.952+0000 7fd1ac4ab700 1 Processor -- start 2023-11-24T07:56:39.956 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:39.952+0000 7fd1ac4ab700 1 -- start start 2023-11-24T07:56:39.957 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:39.953+0000 7fd1ac4ab700 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fd1a4143ae0 0x7fd1a40b51b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:56:39.957 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:39.953+0000 7fd1ac4ab700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fd1a40722c0 0x7fd1a406e510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:56:39.957 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:39.953+0000 7fd1ac4ab700 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd1a4143ec0 con 0x7fd1a41440f0 2023-11-24T07:56:39.957 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:39.953+0000 7fd1ac4ab700 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd1a40b40a0 con 0x7fd1a40722c0 2023-11-24T07:56:39.957 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:39.953+0000 7fd1ac4ab700 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd1a40b3ae0 con 0x7fd1a4143ae0 2023-11-24T07:56:39.957 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:39.953+0000 7fd1aaa48700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fd1a40722c0 0x7fd1a406e510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:56:39.957 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:39.953+0000 7fd1aaa48700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fd1a40722c0 0x7fd1a406e510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:50354/0 (socket says 172.21.15.59:50354) 2023-11-24T07:56:39.957 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:39.953+0000 7fd1aaa48700 1 -- 172.21.15.59:0/2607003713 learned_addr learned my addr 172.21.15.59:0/2607003713 (peer_addr_for_me v2:172.21.15.59:0/0) 2023-11-24T07:56:39.957 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:39.953+0000 7fd1aa247700 1 --2- 172.21.15.59:0/2607003713 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fd1a4143ae0 0x7fd1a40b51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:56:39.957 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:39.954+0000 7fd1aaa48700 1 -- 172.21.15.59:0/2607003713 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fd1a4143ae0 msgr2=0x7fd1a40b51b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T07:56:39.957 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:39.954+0000 7fd1aaa48700 1 --2- 172.21.15.59:0/2607003713 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fd1a4143ae0 0x7fd1a40b51b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:56:39.958 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:39.954+0000 7fd1aaa48700 1 -- 172.21.15.59:0/2607003713 >> v1:172.21.15.76:6789/0 conn(0x7fd1a41440f0 legacy=0x7fd1a4071bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:56:39.958 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:39.954+0000 7fd1aaa48700 1 -- 172.21.15.59:0/2607003713 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd1a40b3dc0 con 0x7fd1a40722c0 2023-11-24T07:56:39.958 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:39.954+0000 7fd1aaa48700 1 --2- 172.21.15.59:0/2607003713 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd1a40722c0 0x7fd1a406e510 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fd19c00d150 tx=0x7fd19c00d460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b3a677b51d62fec9 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:56:39.958 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:39.954+0000 7fd1a9245700 1 -- 172.21.15.59:0/2607003713 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd19c010070 con 0x7fd1a40722c0 2023-11-24T07:56:39.958 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:39.954+0000 7fd1a9245700 1 -- 172.21.15.59:0/2607003713 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd19c00dd10 con 0x7fd1a40722c0 2023-11-24T07:56:39.958 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:39.954+0000 7fd1a9245700 1 -- 172.21.15.59:0/2607003713 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd19c00b310 con 0x7fd1a40722c0 2023-11-24T07:56:39.958 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:39.954+0000 7fd1ac4ab700 1 -- 172.21.15.59:0/2607003713 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd1a40722c0 msgr2=0x7fd1a406e510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:56:39.958 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:39.954+0000 7fd1ac4ab700 1 --2- 172.21.15.59:0/2607003713 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd1a40722c0 0x7fd1a406e510 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fd19c00d150 tx=0x7fd19c00d460 comp rx=0 tx=0).stop 2023-11-24T07:56:39.958 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:39.954+0000 7fd1ac4ab700 1 -- 172.21.15.59:0/2607003713 shutdown_connections 2023-11-24T07:56:39.959 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:39.954+0000 7fd1ac4ab700 1 --2- 172.21.15.59:0/2607003713 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fd1a4143ae0 0x7fd1a40b51b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:56:39.959 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:39.954+0000 7fd1ac4ab700 1 --2- 172.21.15.59:0/2607003713 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd1a40722c0 0x7fd1a406e510 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:56:39.959 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:39.955+0000 7fd1ac4ab700 1 -- 172.21.15.59:0/2607003713 >> 172.21.15.59:0/2607003713 conn(0x7fd1a40b2740 msgr2=0x7fd1a40b2b20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:56:39.959 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:39.955+0000 7fd1ac4ab700 1 -- 172.21.15.59:0/2607003713 shutdown_connections 2023-11-24T07:56:39.959 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:39.955+0000 7fd1ac4ab700 1 -- 172.21.15.59:0/2607003713 wait complete. 2023-11-24T07:56:39.961 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:39.956+0000 7fd1ac4ab700 1 Processor -- start 2023-11-24T07:56:39.961 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:39.956+0000 7fd1ac4ab700 1 -- start start 2023-11-24T07:56:39.961 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:39.956+0000 7fd1ac4ab700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd1a40722c0 0x7fd1a413f670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:56:39.962 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:39.956+0000 7fd1ac4ab700 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fd1a4143ae0 0x7fd1a4154fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:56:39.962 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:39.956+0000 7fd1ac4ab700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd1a41440f0 0x7fd1a41581c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:56:39.962 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:39.957+0000 7fd1ac4ab700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1a41f48f0 con 0x7fd1a41440f0 2023-11-24T07:56:39.962 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:39.957+0000 7fd1ac4ab700 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1a4153d20 con 0x7fd1a40722c0 2023-11-24T07:56:39.962 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:39.957+0000 7fd1ac4ab700 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd1a4153fc0 con 0x7fd1a4143ae0 2023-11-24T07:56:39.962 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:39.957+0000 7fd1aaa48700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd1a41440f0 0x7fd1a41581c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:56:39.962 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:39.957+0000 7fd1aaa48700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd1a41440f0 0x7fd1a41581c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:50360/0 (socket says 172.21.15.59:50360) 2023-11-24T07:56:39.962 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:39.957+0000 7fd1aaa48700 1 -- 172.21.15.59:0/4228308790 learned_addr learned my addr 172.21.15.59:0/4228308790 (peer_addr_for_me v2:172.21.15.59:0/0) 2023-11-24T07:56:39.962 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:39.957+0000 7fd1aa247700 1 --2- 172.21.15.59:0/4228308790 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd1a40722c0 0x7fd1a413f670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:56:39.962 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:39.957+0000 7fd1a9a46700 1 --2- 172.21.15.59:0/4228308790 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fd1a4143ae0 0x7fd1a4154fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:56:39.962 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:39.957+0000 7fd1aaa48700 1 -- 172.21.15.59:0/4228308790 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fd1a4143ae0 msgr2=0x7fd1a4154fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:56:39.963 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:39.957+0000 7fd1aaa48700 1 --2- 172.21.15.59:0/4228308790 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fd1a4143ae0 0x7fd1a4154fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:56:39.963 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:39.957+0000 7fd1aaa48700 1 -- 172.21.15.59:0/4228308790 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd1a40722c0 msgr2=0x7fd1a413f670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:56:39.963 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:39.957+0000 7fd1aaa48700 1 --2- 172.21.15.59:0/4228308790 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd1a40722c0 0x7fd1a413f670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:56:39.963 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:39.957+0000 7fd1aaa48700 1 -- 172.21.15.59:0/4228308790 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd19c00bd40 con 0x7fd1a41440f0 2023-11-24T07:56:39.963 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:39.957+0000 7fd1aaa48700 1 --2- 172.21.15.59:0/4228308790 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd1a41440f0 0x7fd1a41581c0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fd19c014000 tx=0x7fd19c004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:56:39.963 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:39.957+0000 7fd19affd700 1 -- 172.21.15.59:0/4228308790 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd19c009a10 con 0x7fd1a41440f0 2023-11-24T07:56:39.963 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:39.957+0000 7fd19affd700 1 -- 172.21.15.59:0/4228308790 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd19c009b90 con 0x7fd1a41440f0 2023-11-24T07:56:39.963 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:39.957+0000 7fd1ac4ab700 1 -- 172.21.15.59:0/4228308790 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd1a41542a0 con 0x7fd1a41440f0 2023-11-24T07:56:39.963 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:39.957+0000 7fd19affd700 1 -- 172.21.15.59:0/4228308790 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd19c004040 con 0x7fd1a41440f0 2023-11-24T07:56:39.963 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:39.957+0000 7fd1ac4ab700 1 -- 172.21.15.59:0/4228308790 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd1a4154860 con 0x7fd1a41440f0 2023-11-24T07:56:39.963 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:39.958+0000 7fd1ac4ab700 1 -- 172.21.15.59:0/4228308790 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd1a41549f0 con 0x7fd1a41440f0 2023-11-24T07:56:39.964 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:39.959+0000 7fd19affd700 1 -- 172.21.15.59:0/4228308790 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fd19c0041a0 con 0x7fd1a41440f0 2023-11-24T07:56:39.964 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:39.959+0000 7fd19affd700 1 --2- 172.21.15.59:0/4228308790 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7fd18806aed0 0x7fd18806d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:56:39.964 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:39.959+0000 7fd19affd700 1 -- 172.21.15.59:0/4228308790 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fd19c08af60 con 0x7fd1a41440f0 2023-11-24T07:56:39.964 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:39.959+0000 7fd1aa247700 1 --2- 172.21.15.59:0/4228308790 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7fd18806aed0 0x7fd18806d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:56:39.964 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:39.960+0000 7fd1aa247700 1 --2- 172.21.15.59:0/4228308790 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7fd18806aed0 0x7fd18806d390 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fd194009be0 tx=0x7fd194009210 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:56:39.965 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:39.963+0000 7fd19affd700 1 -- 172.21.15.59:0/4228308790 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fd19c0560a0 con 0x7fd1a41440f0 2023-11-24T07:56:40.166 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:40.164+0000 7fd1ac4ab700 1 -- 172.21.15.59:0/4228308790 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fd1a4075480 con 0x7fd1a41440f0 2023-11-24T07:56:40.166 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:40.164+0000 7fd19affd700 1 -- 172.21.15.59:0/4228308790 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v20) v1 ==== 104+0+11 (secure 0 0 0) 0x7fd19c0596c0 con 0x7fd1a41440f0 2023-11-24T07:56:40.167 INFO:teuthology.orchestra.run.smithi059.stdout:pg_num: 32 2023-11-24T07:56:40.169 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:40.167+0000 7fd1ac4ab700 1 -- 172.21.15.59:0/4228308790 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7fd18806aed0 msgr2=0x7fd18806d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:56:40.169 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:40.167+0000 7fd1ac4ab700 1 --2- 172.21.15.59:0/4228308790 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7fd18806aed0 0x7fd18806d390 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fd194009be0 tx=0x7fd194009210 comp rx=0 tx=0).stop 2023-11-24T07:56:40.169 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:40.167+0000 7fd1ac4ab700 1 -- 172.21.15.59:0/4228308790 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd1a41440f0 msgr2=0x7fd1a41581c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:56:40.169 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:40.167+0000 7fd1ac4ab700 1 --2- 172.21.15.59:0/4228308790 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd1a41440f0 0x7fd1a41581c0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fd19c014000 tx=0x7fd19c004ad0 comp rx=0 tx=0).stop 2023-11-24T07:56:40.169 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:40.167+0000 7fd1ac4ab700 1 -- 172.21.15.59:0/4228308790 shutdown_connections 2023-11-24T07:56:40.169 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:40.167+0000 7fd1ac4ab700 1 --2- 172.21.15.59:0/4228308790 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7fd18806aed0 0x7fd18806d390 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:56:40.170 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:40.167+0000 7fd1ac4ab700 1 --2- 172.21.15.59:0/4228308790 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd1a41440f0 0x7fd1a41581c0 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:56:40.170 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:40.167+0000 7fd1ac4ab700 1 --2- 172.21.15.59:0/4228308790 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd1a40722c0 0x7fd1a413f670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:56:40.170 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:40.167+0000 7fd1ac4ab700 1 --2- 172.21.15.59:0/4228308790 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fd1a4143ae0 0x7fd1a4154fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:56:40.170 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:40.167+0000 7fd1ac4ab700 1 -- 172.21.15.59:0/4228308790 >> 172.21.15.59:0/4228308790 conn(0x7fd1a40b2740 msgr2=0x7fd1a40cf7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:56:40.170 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:40.167+0000 7fd1ac4ab700 1 -- 172.21.15.59:0/4228308790 shutdown_connections 2023-11-24T07:56:40.171 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:40.167+0000 7fd1ac4ab700 1 -- 172.21.15.59:0/4228308790 wait complete. 2023-11-24T07:56:40.180 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-24T07:56:40.309 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:40.307+0000 7f02af9ab700 1 Processor -- start 2023-11-24T07:56:40.309 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:40.307+0000 7f02af9ab700 1 -- start start 2023-11-24T07:56:40.312 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:40.308+0000 7f02af9ab700 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f02a815add0 0x7f02a80b4880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:56:40.312 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:40.308+0000 7f02af9ab700 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f02a80b3ab0 0x7f02a806ee30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:56:40.312 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:40.308+0000 7f02af9ab700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f02a806f370 0x7f02a806daf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:56:40.312 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:40.308+0000 7f02af9ab700 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f02a806f800 con 0x7f02a806f370 2023-11-24T07:56:40.312 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:40.308+0000 7f02af9ab700 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f02a80ce5b0 con 0x7f02a80b3ab0 2023-11-24T07:56:40.312 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:40.308+0000 7f02af9ab700 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f02a8072d70 con 0x7f02a815add0 2023-11-24T07:56:40.312 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:40.308+0000 7f02adf48700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f02a806f370 0x7f02a806daf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:56:40.312 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:40.308+0000 7f02adf48700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f02a806f370 0x7f02a806daf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:50368/0 (socket says 172.21.15.59:50368) 2023-11-24T07:56:40.312 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:40.308+0000 7f02adf48700 1 -- 172.21.15.59:0/2021825760 learned_addr learned my addr 172.21.15.59:0/2021825760 (peer_addr_for_me v2:172.21.15.59:0/0) 2023-11-24T07:56:40.312 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:40.309+0000 7f02ad747700 1 --2- 172.21.15.59:0/2021825760 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f02a815add0 0x7f02a80b4880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:56:40.312 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:40.309+0000 7f02acf46700 1 --2- 172.21.15.59:0/2021825760 >> v2:172.21.15.76:3300/0 conn(0x7f02a80b3ab0 0x7f02a806ee30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:56:40.312 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:40.309+0000 7f02adf48700 1 -- 172.21.15.59:0/2021825760 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f02a815add0 msgr2=0x7f02a80b4880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T07:56:40.313 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:40.309+0000 7f02adf48700 1 --2- 172.21.15.59:0/2021825760 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f02a815add0 0x7f02a80b4880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:56:40.313 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:40.309+0000 7f02adf48700 1 -- 172.21.15.59:0/2021825760 >> v2:172.21.15.76:3300/0 conn(0x7f02a80b3ab0 msgr2=0x7f02a806ee30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T07:56:40.313 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:40.309+0000 7f02adf48700 1 --2- 172.21.15.59:0/2021825760 >> v2:172.21.15.76:3300/0 conn(0x7f02a80b3ab0 0x7f02a806ee30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:56:40.313 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:40.309+0000 7f02adf48700 1 -- 172.21.15.59:0/2021825760 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f02a80b4f10 con 0x7f02a806f370 2023-11-24T07:56:40.313 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:40.309+0000 7f02ad747700 1 --2- 172.21.15.59:0/2021825760 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f02a815add0 0x7f02a80b4880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T07:56:40.313 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:40.309+0000 7f02acf46700 1 --2- 172.21.15.59:0/2021825760 >> v2:172.21.15.76:3300/0 conn(0x7f02a80b3ab0 0x7f02a806ee30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T07:56:40.313 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:40.309+0000 7f02adf48700 1 --2- 172.21.15.59:0/2021825760 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f02a806f370 0x7f02a806daf0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f02a80691b0 tx=0x7f02a000d430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9d6fc8e93621fd08 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:56:40.313 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:40.309+0000 7f029ffff700 1 -- 172.21.15.59:0/2021825760 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02a0010070 con 0x7f02a806f370 2023-11-24T07:56:40.313 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:40.309+0000 7f029ffff700 1 -- 172.21.15.59:0/2021825760 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f02a000dd90 con 0x7f02a806f370 2023-11-24T07:56:40.313 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:40.309+0000 7f029ffff700 1 -- 172.21.15.59:0/2021825760 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02a000b310 con 0x7f02a806f370 2023-11-24T07:56:40.313 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:40.310+0000 7f02af9ab700 1 -- 172.21.15.59:0/2021825760 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f02a806f370 msgr2=0x7f02a806daf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:56:40.313 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:40.310+0000 7f02af9ab700 1 --2- 172.21.15.59:0/2021825760 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f02a806f370 0x7f02a806daf0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f02a80691b0 tx=0x7f02a000d430 comp rx=0 tx=0).stop 2023-11-24T07:56:40.313 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:40.310+0000 7f02af9ab700 1 -- 172.21.15.59:0/2021825760 shutdown_connections 2023-11-24T07:56:40.314 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:40.310+0000 7f02af9ab700 1 --2- 172.21.15.59:0/2021825760 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f02a806f370 0x7f02a806daf0 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:56:40.314 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:40.310+0000 7f02af9ab700 1 --2- 172.21.15.59:0/2021825760 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f02a815add0 0x7f02a80b4880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:56:40.314 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:40.310+0000 7f02af9ab700 1 --2- 172.21.15.59:0/2021825760 >> v2:172.21.15.76:3300/0 conn(0x7f02a80b3ab0 0x7f02a806ee30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:56:40.314 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:40.310+0000 7f02af9ab700 1 -- 172.21.15.59:0/2021825760 >> 172.21.15.59:0/2021825760 conn(0x7f02a80b27b0 msgr2=0x7f02a80b2b90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:56:40.314 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:40.310+0000 7f02af9ab700 1 -- 172.21.15.59:0/2021825760 shutdown_connections 2023-11-24T07:56:40.314 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:40.310+0000 7f02af9ab700 1 -- 172.21.15.59:0/2021825760 wait complete. 2023-11-24T07:56:40.316 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:40.311+0000 7f02af9ab700 1 Processor -- start 2023-11-24T07:56:40.316 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:40.311+0000 7f02af9ab700 1 -- start start 2023-11-24T07:56:40.317 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:40.312+0000 7f02af9ab700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f02a806f370 0x7f02a8150070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:56:40.317 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:40.312+0000 7f02af9ab700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f02a80b3ab0 0x7f02a815d530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:56:40.317 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:40.312+0000 7f02af9ab700 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f02a815add0 0x7f02a815da70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:56:40.317 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:40.312+0000 7f02af9ab700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02a814e190 con 0x7f02a80b3ab0 2023-11-24T07:56:40.317 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:40.312+0000 7f02af9ab700 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02a81505b0 con 0x7f02a806f370 2023-11-24T07:56:40.317 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:40.312+0000 7f02af9ab700 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f02a80caed0 con 0x7f02a815add0 2023-11-24T07:56:40.317 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:40.312+0000 7f02acf46700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f02a80b3ab0 0x7f02a815d530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:56:40.317 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:40.312+0000 7f02acf46700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f02a80b3ab0 0x7f02a815d530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:50372/0 (socket says 172.21.15.59:50372) 2023-11-24T07:56:40.317 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:40.312+0000 7f02acf46700 1 -- 172.21.15.59:0/3475541210 learned_addr learned my addr 172.21.15.59:0/3475541210 (peer_addr_for_me v2:172.21.15.59:0/0) 2023-11-24T07:56:40.317 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:40.312+0000 7f02adf48700 1 --2- 172.21.15.59:0/3475541210 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f02a815add0 0x7f02a815da70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:56:40.317 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:40.312+0000 7f02ad747700 1 --2- 172.21.15.59:0/3475541210 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f02a806f370 0x7f02a8150070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:56:40.317 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:40.312+0000 7f02acf46700 1 -- 172.21.15.59:0/3475541210 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f02a815add0 msgr2=0x7f02a815da70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:56:40.317 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:40.312+0000 7f02acf46700 1 --2- 172.21.15.59:0/3475541210 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f02a815add0 0x7f02a815da70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:56:40.318 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:40.312+0000 7f02acf46700 1 -- 172.21.15.59:0/3475541210 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f02a806f370 msgr2=0x7f02a8150070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:56:40.318 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:40.312+0000 7f02acf46700 1 --2- 172.21.15.59:0/3475541210 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f02a806f370 0x7f02a8150070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:56:40.318 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:40.312+0000 7f02acf46700 1 -- 172.21.15.59:0/3475541210 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f02a000bd40 con 0x7f02a80b3ab0 2023-11-24T07:56:40.318 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:40.312+0000 7f02ad747700 1 --2- 172.21.15.59:0/3475541210 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f02a806f370 0x7f02a8150070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-24T07:56:40.318 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:40.312+0000 7f02acf46700 1 --2- 172.21.15.59:0/3475541210 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f02a80b3ab0 0x7f02a815d530 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f02a400c740 tx=0x7f02a400ca50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:56:40.318 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:40.312+0000 7f029dffb700 1 -- 172.21.15.59:0/3475541210 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02a4011800 con 0x7f02a80b3ab0 2023-11-24T07:56:40.318 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:40.312+0000 7f029dffb700 1 -- 172.21.15.59:0/3475541210 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f02a4011960 con 0x7f02a80b3ab0 2023-11-24T07:56:40.318 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:40.312+0000 7f02af9ab700 1 -- 172.21.15.59:0/3475541210 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f02a80cb1b0 con 0x7f02a80b3ab0 2023-11-24T07:56:40.318 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:40.312+0000 7f029dffb700 1 -- 172.21.15.59:0/3475541210 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02a4011cd0 con 0x7f02a80b3ab0 2023-11-24T07:56:40.318 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:40.313+0000 7f02af9ab700 1 -- 172.21.15.59:0/3475541210 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f02a80cb770 con 0x7f02a80b3ab0 2023-11-24T07:56:40.318 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:40.313+0000 7f02af9ab700 1 -- 172.21.15.59:0/3475541210 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f02a80cb960 con 0x7f02a80b3ab0 2023-11-24T07:56:40.318 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:40.314+0000 7f029dffb700 1 -- 172.21.15.59:0/3475541210 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f02a4021350 con 0x7f02a80b3ab0 2023-11-24T07:56:40.319 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:40.314+0000 7f029dffb700 1 --2- 172.21.15.59:0/3475541210 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f028c06aed0 0x7f028c06d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:56:40.319 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:40.314+0000 7f029dffb700 1 -- 172.21.15.59:0/3475541210 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f02a40893a0 con 0x7f02a80b3ab0 2023-11-24T07:56:40.319 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:40.314+0000 7f02ad747700 1 --2- 172.21.15.59:0/3475541210 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f028c06aed0 0x7f028c06d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:56:40.319 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:40.315+0000 7f02ad747700 1 --2- 172.21.15.59:0/3475541210 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f028c06aed0 0x7f028c06d390 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f029800aa20 tx=0x7f0298008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:56:40.320 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:40.319+0000 7f029dffb700 1 -- 172.21.15.59:0/3475541210 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f02a40544e0 con 0x7f02a80b3ab0 2023-11-24T07:56:40.520 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:40.518+0000 7f02af9ab700 1 -- 172.21.15.59:0/3475541210 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f02a814e2d0 con 0x7f02a80b3ab0 2023-11-24T07:56:40.520 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:40.518+0000 7f029dffb700 1 -- 172.21.15.59:0/3475541210 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v20) v1 ==== 100+0+11 (secure 0 0 0) 0x7f02a4057b00 con 0x7f02a80b3ab0 2023-11-24T07:56:40.520 INFO:teuthology.orchestra.run.smithi059.stdout:pg_num: 64 2023-11-24T07:56:40.523 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:40.521+0000 7f02af9ab700 1 -- 172.21.15.59:0/3475541210 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f028c06aed0 msgr2=0x7f028c06d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:56:40.523 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:40.521+0000 7f02af9ab700 1 --2- 172.21.15.59:0/3475541210 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f028c06aed0 0x7f028c06d390 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f029800aa20 tx=0x7f0298008040 comp rx=0 tx=0).stop 2023-11-24T07:56:40.523 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:40.521+0000 7f02af9ab700 1 -- 172.21.15.59:0/3475541210 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f02a80b3ab0 msgr2=0x7f02a815d530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:56:40.523 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:40.521+0000 7f02af9ab700 1 --2- 172.21.15.59:0/3475541210 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f02a80b3ab0 0x7f02a815d530 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f02a400c740 tx=0x7f02a400ca50 comp rx=0 tx=0).stop 2023-11-24T07:56:40.523 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:40.521+0000 7f02af9ab700 1 -- 172.21.15.59:0/3475541210 shutdown_connections 2023-11-24T07:56:40.523 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:40.521+0000 7f02af9ab700 1 --2- 172.21.15.59:0/3475541210 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f028c06aed0 0x7f028c06d390 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:56:40.523 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:40.521+0000 7f02af9ab700 1 --2- 172.21.15.59:0/3475541210 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f02a815add0 0x7f02a815da70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:56:40.523 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:40.521+0000 7f02af9ab700 1 --2- 172.21.15.59:0/3475541210 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f02a806f370 0x7f02a8150070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:56:40.523 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:40.521+0000 7f02af9ab700 1 --2- 172.21.15.59:0/3475541210 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f02a80b3ab0 0x7f02a815d530 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:56:40.523 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:40.521+0000 7f02af9ab700 1 -- 172.21.15.59:0/3475541210 >> 172.21.15.59:0/3475541210 conn(0x7f02a80b27b0 msgr2=0x7f02a806daf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:56:40.523 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:40.521+0000 7f02af9ab700 1 -- 172.21.15.59:0/3475541210 shutdown_connections 2023-11-24T07:56:40.524 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:40.521+0000 7f02af9ab700 1 -- 172.21.15.59:0/3475541210 wait complete. 2023-11-24T07:56:40.534 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-24T07:56:40.659 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:40.657+0000 7f2f89906700 1 Processor -- start 2023-11-24T07:56:40.659 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:40.657+0000 7f2f89906700 1 -- start start 2023-11-24T07:56:40.661 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:40.658+0000 7f2f89906700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f2f8406b620 0x7f2f840b3a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:56:40.661 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:40.658+0000 7f2f89906700 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f2f8406ee60 0x7f2f840b3f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:56:40.662 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:40.658+0000 7f2f89906700 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2f84071f50 con 0x7f2f840b5ab0 2023-11-24T07:56:40.662 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:40.658+0000 7f2f89906700 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2f84072230 con 0x7f2f8406b620 2023-11-24T07:56:40.662 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:40.658+0000 7f2f89906700 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2f84072510 con 0x7f2f8406ee60 2023-11-24T07:56:40.662 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:40.658+0000 7f2f82ffd700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f2f8406b620 0x7f2f840b3a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:56:40.662 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:40.658+0000 7f2f827fc700 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7f2f840b5ab0 0x7f2f84161ba0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:53852/0 (socket says 172.21.15.59:53852) 2023-11-24T07:56:40.662 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:40.658+0000 7f2f827fc700 1 -- 172.21.15.59:0/475890537 learned_addr learned my addr 172.21.15.59:0/475890537 (peer_addr_for_me v1:172.21.15.59:0/0) 2023-11-24T07:56:40.662 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:40.658+0000 7f2f837fe700 1 --2- 172.21.15.59:0/475890537 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f2f8406ee60 0x7f2f840b3f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:56:40.662 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:40.659+0000 7f2f81ffb700 1 -- 172.21.15.59:0/475890537 <== mon.0 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 791339010 0 0) 0x7f2f84071f50 con 0x7f2f840b5ab0 2023-11-24T07:56:40.662 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:40.659+0000 7f2f82ffd700 1 -- 172.21.15.59:0/475890537 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f2f8406ee60 msgr2=0x7f2f840b3f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T07:56:40.662 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:40.659+0000 7f2f82ffd700 1 --2- 172.21.15.59:0/475890537 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f2f8406ee60 0x7f2f840b3f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:56:40.662 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:40.659+0000 7f2f82ffd700 1 -- 172.21.15.59:0/475890537 >> v1:172.21.15.59:6789/0 conn(0x7f2f840b5ab0 legacy=0x7f2f84161ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:56:40.663 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:40.659+0000 7f2f82ffd700 1 -- 172.21.15.59:0/475890537 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2f840727f0 con 0x7f2f8406b620 2023-11-24T07:56:40.663 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:40.659+0000 7f2f82ffd700 1 --2- 172.21.15.59:0/475890537 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2f8406b620 0x7f2f840b3a40 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f2f78009840 tx=0x7f2f78009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=71f2d8aab5139de9 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:56:40.663 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:40.659+0000 7f2f81ffb700 1 -- 172.21.15.59:0/475890537 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f78011830 con 0x7f2f8406b620 2023-11-24T07:56:40.663 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:40.659+0000 7f2f81ffb700 1 -- 172.21.15.59:0/475890537 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2f78011990 con 0x7f2f8406b620 2023-11-24T07:56:40.663 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:40.659+0000 7f2f81ffb700 1 -- 172.21.15.59:0/475890537 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f78011d20 con 0x7f2f8406b620 2023-11-24T07:56:40.663 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:40.659+0000 7f2f89906700 1 -- 172.21.15.59:0/475890537 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2f8406b620 msgr2=0x7f2f840b3a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:56:40.663 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:40.659+0000 7f2f89906700 1 --2- 172.21.15.59:0/475890537 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2f8406b620 0x7f2f840b3a40 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f2f78009840 tx=0x7f2f78009b50 comp rx=0 tx=0).stop 2023-11-24T07:56:40.663 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:40.659+0000 7f2f89906700 1 -- 172.21.15.59:0/475890537 shutdown_connections 2023-11-24T07:56:40.663 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:40.659+0000 7f2f89906700 1 --2- 172.21.15.59:0/475890537 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f2f8406ee60 0x7f2f840b3f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:56:40.663 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:40.659+0000 7f2f89906700 1 --2- 172.21.15.59:0/475890537 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2f8406b620 0x7f2f840b3a40 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:56:40.663 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:40.659+0000 7f2f89906700 1 -- 172.21.15.59:0/475890537 >> 172.21.15.59:0/475890537 conn(0x7f2f840b2740 msgr2=0x7f2f840b2b20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:56:40.663 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:40.660+0000 7f2f89906700 1 -- 172.21.15.59:0/475890537 shutdown_connections 2023-11-24T07:56:40.663 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:40.660+0000 7f2f89906700 1 -- 172.21.15.59:0/475890537 wait complete. 2023-11-24T07:56:40.666 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:40.661+0000 7f2f89906700 1 Processor -- start 2023-11-24T07:56:40.666 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:40.661+0000 7f2f89906700 1 -- start start 2023-11-24T07:56:40.666 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:40.662+0000 7f2f89906700 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f2f8406b620 0x7f2f841e6490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:56:40.666 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:40.662+0000 7f2f89906700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2f8406ee60 0x7f2f841f39a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:56:40.666 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:40.662+0000 7f2f89906700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2f840b5ab0 0x7f2f841f3ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:56:40.666 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:40.662+0000 7f2f89906700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f84144000 con 0x7f2f8406ee60 2023-11-24T07:56:40.667 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:40.662+0000 7f2f89906700 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f841f44d0 con 0x7f2f840b5ab0 2023-11-24T07:56:40.667 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:40.662+0000 7f2f89906700 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2f841f47b0 con 0x7f2f8406b620 2023-11-24T07:56:40.667 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:40.662+0000 7f2f827fc700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2f8406ee60 0x7f2f841f39a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:56:40.667 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:40.662+0000 7f2f827fc700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2f8406ee60 0x7f2f841f39a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:50384/0 (socket says 172.21.15.59:50384) 2023-11-24T07:56:40.667 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:40.662+0000 7f2f827fc700 1 -- 172.21.15.59:0/3686334342 learned_addr learned my addr 172.21.15.59:0/3686334342 (peer_addr_for_me v2:172.21.15.59:0/0) 2023-11-24T07:56:40.667 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:40.662+0000 7f2f82ffd700 1 --2- 172.21.15.59:0/3686334342 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f2f8406b620 0x7f2f841e6490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:56:40.667 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:40.662+0000 7f2f837fe700 1 --2- 172.21.15.59:0/3686334342 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2f840b5ab0 0x7f2f841f3ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:56:40.667 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:40.662+0000 7f2f827fc700 1 -- 172.21.15.59:0/3686334342 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f2f8406b620 msgr2=0x7f2f841e6490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:56:40.667 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:40.662+0000 7f2f827fc700 1 --2- 172.21.15.59:0/3686334342 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f2f8406b620 0x7f2f841e6490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:56:40.668 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:40.662+0000 7f2f827fc700 1 -- 172.21.15.59:0/3686334342 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2f840b5ab0 msgr2=0x7f2f841f3ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:56:40.668 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:40.662+0000 7f2f827fc700 1 --2- 172.21.15.59:0/3686334342 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2f840b5ab0 0x7f2f841f3ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:56:40.668 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:40.662+0000 7f2f827fc700 1 -- 172.21.15.59:0/3686334342 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2f780094a0 con 0x7f2f8406ee60 2023-11-24T07:56:40.668 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:40.662+0000 7f2f827fc700 1 --2- 172.21.15.59:0/3686334342 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2f8406ee60 0x7f2f841f39a0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f2f7400c520 tx=0x7f2f7400c830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:56:40.668 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:40.662+0000 7f2f6bfff700 1 -- 172.21.15.59:0/3686334342 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f74004c80 con 0x7f2f8406ee60 2023-11-24T07:56:40.668 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:40.663+0000 7f2f6bfff700 1 -- 172.21.15.59:0/3686334342 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2f74004de0 con 0x7f2f8406ee60 2023-11-24T07:56:40.668 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:40.663+0000 7f2f89906700 1 -- 172.21.15.59:0/3686334342 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2f841f4a90 con 0x7f2f8406ee60 2023-11-24T07:56:40.668 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:40.663+0000 7f2f6bfff700 1 -- 172.21.15.59:0/3686334342 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f740072d0 con 0x7f2f8406ee60 2023-11-24T07:56:40.668 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:40.663+0000 7f2f89906700 1 -- 172.21.15.59:0/3686334342 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2f841f5050 con 0x7f2f8406ee60 2023-11-24T07:56:40.668 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:40.663+0000 7f2f89906700 1 -- 172.21.15.59:0/3686334342 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2f84068670 con 0x7f2f8406ee60 2023-11-24T07:56:40.668 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:40.664+0000 7f2f6bfff700 1 -- 172.21.15.59:0/3686334342 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f2f74007430 con 0x7f2f8406ee60 2023-11-24T07:56:40.669 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:40.664+0000 7f2f6bfff700 1 --2- 172.21.15.59:0/3686334342 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f2f7006aed0 0x7f2f7006d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:56:40.669 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:40.664+0000 7f2f6bfff700 1 -- 172.21.15.59:0/3686334342 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f2f74089100 con 0x7f2f8406ee60 2023-11-24T07:56:40.669 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:40.665+0000 7f2f82ffd700 1 --2- 172.21.15.59:0/3686334342 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f2f7006aed0 0x7f2f7006d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:56:40.669 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:40.665+0000 7f2f82ffd700 1 --2- 172.21.15.59:0/3686334342 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f2f7006aed0 0x7f2f7006d390 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f2f780090b0 tx=0x7f2f78008030 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:56:40.670 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:40.669+0000 7f2f6bfff700 1 -- 172.21.15.59:0/3686334342 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f2f74054240 con 0x7f2f8406ee60 2023-11-24T07:56:40.919 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:40.918+0000 7f2f89906700 1 -- 172.21.15.59:0/3686334342 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f2f8406bd40 con 0x7f2f8406ee60 2023-11-24T07:56:40.920 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:40.918+0000 7f2f6bfff700 1 -- 172.21.15.59:0/3686334342 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) v1 ==== 93+0+4904 (secure 0 0 0) 0x7f2f74057860 con 0x7f2f8406ee60 2023-11-24T07:56:40.920 INFO:teuthology.orchestra.run.smithi059.stdout: 2023-11-24T07:56:40.920 INFO:teuthology.orchestra.run.smithi059.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4220,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6835/838038775","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6833","nonce":838038775},{"type":"v1","addr":"172.21.15.59:6835","nonce":838038775}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4232,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.76:6837/3226690004","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6835","nonce":3226690004},{"type":"v1","addr":"172.21.15.76:6837","nonce":3226690004}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4260,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6834/2986456006","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6832","nonce":2986456006},{"type":"v1","addr":"172.21.15.59:6834","nonce":2986456006}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-24T07:56:24.104295+0000","modified":"2023-11-24T07:56:27.152806+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":4264},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4264":{"gid":4264,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.76:6836/3993077463","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6834","nonce":3993077463},{"type":"v1","addr":"172.21.15.76:6836","nonce":3993077463}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-11-24T07:56:40.922 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:40.920+0000 7f2f89906700 1 -- 172.21.15.59:0/3686334342 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f2f7006aed0 msgr2=0x7f2f7006d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:56:40.923 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:40.920+0000 7f2f89906700 1 --2- 172.21.15.59:0/3686334342 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f2f7006aed0 0x7f2f7006d390 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f2f780090b0 tx=0x7f2f78008030 comp rx=0 tx=0).stop 2023-11-24T07:56:40.923 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:40.920+0000 7f2f89906700 1 -- 172.21.15.59:0/3686334342 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2f8406ee60 msgr2=0x7f2f841f39a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:56:40.923 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:40.920+0000 7f2f89906700 1 --2- 172.21.15.59:0/3686334342 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2f8406ee60 0x7f2f841f39a0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f2f7400c520 tx=0x7f2f7400c830 comp rx=0 tx=0).stop 2023-11-24T07:56:40.923 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:40.921+0000 7f2f89906700 1 -- 172.21.15.59:0/3686334342 shutdown_connections 2023-11-24T07:56:40.923 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:40.921+0000 7f2f89906700 1 --2- 172.21.15.59:0/3686334342 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f2f7006aed0 0x7f2f7006d390 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:56:40.923 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:40.921+0000 7f2f89906700 1 --2- 172.21.15.59:0/3686334342 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2f840b5ab0 0x7f2f841f3ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:56:40.923 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:40.921+0000 7f2f89906700 1 --2- 172.21.15.59:0/3686334342 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f2f8406b620 0x7f2f841e6490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:56:40.923 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:40.921+0000 7f2f89906700 1 --2- 172.21.15.59:0/3686334342 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2f8406ee60 0x7f2f841f39a0 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:56:40.923 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:40.921+0000 7f2f89906700 1 -- 172.21.15.59:0/3686334342 >> 172.21.15.59:0/3686334342 conn(0x7f2f840b2740 msgr2=0x7f2f840b0690 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:56:40.923 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:40.921+0000 7f2f89906700 1 -- 172.21.15.59:0/3686334342 shutdown_connections 2023-11-24T07:56:40.923 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:40.921+0000 7f2f89906700 1 -- 172.21.15.59:0/3686334342 wait complete. 2023-11-24T07:56:40.924 INFO:teuthology.orchestra.run.smithi059.stderr:dumped fsmap epoch 8 2023-11-24T07:56:40.935 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-24T07:56:41.059 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:41.057+0000 7f8b3a924700 1 Processor -- start 2023-11-24T07:56:41.059 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:41.058+0000 7f8b3a924700 1 -- start start 2023-11-24T07:56:41.062 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:41.059+0000 7f8b3a924700 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f8b341459a0 0x7f8b340cfa80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:56:41.062 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:41.059+0000 7f8b3a924700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f8b3406b6e0 0x7f8b340cffc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:56:41.062 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:41.059+0000 7f8b3a924700 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8b340cf5f0 con 0x7f8b34142810 2023-11-24T07:56:41.062 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:41.059+0000 7f8b3a924700 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8b3406ea80 con 0x7f8b3406b6e0 2023-11-24T07:56:41.062 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:41.059+0000 7f8b3a924700 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8b3406ed60 con 0x7f8b341459a0 2023-11-24T07:56:41.062 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:41.059+0000 7f8b38ec1700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f8b3406b6e0 0x7f8b340cffc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:56:41.063 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:41.059+0000 7f8b38ec1700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f8b3406b6e0 0x7f8b340cffc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:50394/0 (socket says 172.21.15.59:50394) 2023-11-24T07:56:41.063 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:41.059+0000 7f8b38ec1700 1 -- 172.21.15.59:0/3678016194 learned_addr learned my addr 172.21.15.59:0/3678016194 (peer_addr_for_me v2:172.21.15.59:0/0) 2023-11-24T07:56:41.063 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:41.059+0000 7f8b337fe700 1 --2- 172.21.15.59:0/3678016194 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f8b341459a0 0x7f8b340cfa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:56:41.063 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:41.059+0000 7f8b38ec1700 1 -- 172.21.15.59:0/3678016194 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f8b341459a0 msgr2=0x7f8b340cfa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T07:56:41.063 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:41.059+0000 7f8b38ec1700 1 --2- 172.21.15.59:0/3678016194 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f8b341459a0 0x7f8b340cfa80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:56:41.063 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:41.059+0000 7f8b38ec1700 1 -- 172.21.15.59:0/3678016194 >> v1:172.21.15.76:6789/0 conn(0x7f8b34142810 legacy=0x7f8b3406afd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T07:56:41.063 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:41.059+0000 7f8b38ec1700 1 -- 172.21.15.59:0/3678016194 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8b3406f040 con 0x7f8b3406b6e0 2023-11-24T07:56:41.063 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:41.059+0000 7f8b38ec1700 1 --2- 172.21.15.59:0/3678016194 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8b3406b6e0 0x7f8b340cffc0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f8b2c009e50 tx=0x7f8b2c00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4056ec48940aaf0d server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:56:41.063 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:41.059+0000 7f8b32ffd700 1 -- 172.21.15.59:0/3678016194 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b2c00e070 con 0x7f8b3406b6e0 2023-11-24T07:56:41.063 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:41.060+0000 7f8b32ffd700 1 -- 172.21.15.59:0/3678016194 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8b2c003a20 con 0x7f8b3406b6e0 2023-11-24T07:56:41.063 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:41.060+0000 7f8b32ffd700 1 -- 172.21.15.59:0/3678016194 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b2c003dd0 con 0x7f8b3406b6e0 2023-11-24T07:56:41.063 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:41.060+0000 7f8b3a924700 1 -- 172.21.15.59:0/3678016194 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8b3406b6e0 msgr2=0x7f8b340cffc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:56:41.064 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:41.060+0000 7f8b3a924700 1 --2- 172.21.15.59:0/3678016194 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8b3406b6e0 0x7f8b340cffc0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f8b2c009e50 tx=0x7f8b2c00b330 comp rx=0 tx=0).stop 2023-11-24T07:56:41.064 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:41.060+0000 7f8b3a924700 1 -- 172.21.15.59:0/3678016194 shutdown_connections 2023-11-24T07:56:41.064 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:41.060+0000 7f8b3a924700 1 --2- 172.21.15.59:0/3678016194 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8b3406b6e0 0x7f8b340cffc0 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:56:41.064 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:41.060+0000 7f8b3a924700 1 --2- 172.21.15.59:0/3678016194 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f8b341459a0 0x7f8b340cfa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:56:41.064 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:41.060+0000 7f8b3a924700 1 -- 172.21.15.59:0/3678016194 >> 172.21.15.59:0/3678016194 conn(0x7f8b340b2740 msgr2=0x7f8b340b2b20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:56:41.064 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:41.060+0000 7f8b3a924700 1 -- 172.21.15.59:0/3678016194 shutdown_connections 2023-11-24T07:56:41.064 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:41.060+0000 7f8b3a924700 1 -- 172.21.15.59:0/3678016194 wait complete. 2023-11-24T07:56:41.067 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:41.061+0000 7f8b3a924700 1 Processor -- start 2023-11-24T07:56:41.067 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:41.062+0000 7f8b3a924700 1 -- start start 2023-11-24T07:56:41.067 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:41.062+0000 7f8b3a924700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8b3406b6e0 0x7f8b340cd330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:56:41.067 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:41.062+0000 7f8b3a924700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8b34142810 0x7f8b3415cf30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:56:41.067 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:41.062+0000 7f8b3a924700 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f8b341459a0 0x7f8b340ae8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:56:41.067 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:41.062+0000 7f8b3a924700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b3415d940 con 0x7f8b34142810 2023-11-24T07:56:41.067 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:41.062+0000 7f8b3a924700 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b3415dc20 con 0x7f8b3406b6e0 2023-11-24T07:56:41.067 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:41.062+0000 7f8b3a924700 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8b3415df00 con 0x7f8b341459a0 2023-11-24T07:56:41.067 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:41.062+0000 7f8b337fe700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8b34142810 0x7f8b3415cf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:56:41.067 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:41.062+0000 7f8b337fe700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8b34142810 0x7f8b3415cf30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:50398/0 (socket says 172.21.15.59:50398) 2023-11-24T07:56:41.067 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:41.062+0000 7f8b337fe700 1 -- 172.21.15.59:0/1562176941 learned_addr learned my addr 172.21.15.59:0/1562176941 (peer_addr_for_me v2:172.21.15.59:0/0) 2023-11-24T07:56:41.067 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:41.063+0000 7f8b38ec1700 1 --2- 172.21.15.59:0/1562176941 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f8b341459a0 0x7f8b340ae8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:56:41.068 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:41.063+0000 7f8b33fff700 1 --2- 172.21.15.59:0/1562176941 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8b3406b6e0 0x7f8b340cd330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:56:41.068 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:41.063+0000 7f8b337fe700 1 -- 172.21.15.59:0/1562176941 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f8b341459a0 msgr2=0x7f8b340ae8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:56:41.068 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:41.063+0000 7f8b337fe700 1 --2- 172.21.15.59:0/1562176941 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f8b341459a0 0x7f8b340ae8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:56:41.068 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:41.063+0000 7f8b337fe700 1 -- 172.21.15.59:0/1562176941 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8b3406b6e0 msgr2=0x7f8b340cd330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:56:41.068 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:41.063+0000 7f8b337fe700 1 --2- 172.21.15.59:0/1562176941 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8b3406b6e0 0x7f8b340cd330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:56:41.068 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:41.063+0000 7f8b337fe700 1 -- 172.21.15.59:0/1562176941 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8b2c009b30 con 0x7f8b34142810 2023-11-24T07:56:41.068 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:41.063+0000 7f8b337fe700 1 --2- 172.21.15.59:0/1562176941 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8b34142810 0x7f8b3415cf30 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f8b2400d800 tx=0x7f8b2400dbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:56:41.068 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:41.063+0000 7f8b30ff9700 1 -- 172.21.15.59:0/1562176941 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b240125f0 con 0x7f8b34142810 2023-11-24T07:56:41.068 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:41.063+0000 7f8b30ff9700 1 -- 172.21.15.59:0/1562176941 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8b24012750 con 0x7f8b34142810 2023-11-24T07:56:41.068 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:41.063+0000 7f8b3a924700 1 -- 172.21.15.59:0/1562176941 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8b3415e1e0 con 0x7f8b34142810 2023-11-24T07:56:41.068 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:41.063+0000 7f8b30ff9700 1 -- 172.21.15.59:0/1562176941 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b24012b00 con 0x7f8b34142810 2023-11-24T07:56:41.068 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:41.063+0000 7f8b3a924700 1 -- 172.21.15.59:0/1562176941 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8b340cd870 con 0x7f8b34142810 2023-11-24T07:56:41.069 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:41.064+0000 7f8b3a924700 1 -- 172.21.15.59:0/1562176941 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8b3406be80 con 0x7f8b34142810 2023-11-24T07:56:41.069 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:41.065+0000 7f8b30ff9700 1 -- 172.21.15.59:0/1562176941 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f8b24012c60 con 0x7f8b34142810 2023-11-24T07:56:41.069 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:41.065+0000 7f8b30ff9700 1 --2- 172.21.15.59:0/1562176941 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f8b1406aed0 0x7f8b1406d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:56:41.069 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:41.065+0000 7f8b30ff9700 1 -- 172.21.15.59:0/1562176941 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f8b240899d0 con 0x7f8b34142810 2023-11-24T07:56:41.069 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:41.065+0000 7f8b33fff700 1 --2- 172.21.15.59:0/1562176941 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f8b1406aed0 0x7f8b1406d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:56:41.069 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:41.066+0000 7f8b33fff700 1 --2- 172.21.15.59:0/1562176941 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f8b1406aed0 0x7f8b1406d390 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f8b1c004f30 tx=0x7f8b1c009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:56:41.071 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:41.070+0000 7f8b30ff9700 1 -- 172.21.15.59:0/1562176941 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f8b24054a80 con 0x7f8b34142810 2023-11-24T07:56:41.275 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:41.274+0000 7f8b3a924700 1 -- 172.21.15.59:0/1562176941 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f8b340b3440 con 0x7f8b34142810 2023-11-24T07:56:41.276 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:41.275+0000 7f8b30ff9700 1 -- 172.21.15.59:0/1562176941 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) v1 ==== 74+0+16437 (secure 0 0 0) 0x7f8b240580a0 con 0x7f8b34142810 2023-11-24T07:56:41.277 INFO:teuthology.orchestra.run.smithi059.stdout: 2023-11-24T07:56:41.277 INFO:teuthology.orchestra.run.smithi059.stdout:{"epoch":20,"fsid":"9f0e8741-b8f9-48d5-a2c3-09f45903303d","created":"2023-11-24T07:56:06.552729+0000","modified":"2023-11-24T07:56:25.705091+0000","last_up_change":"2023-11-24T07:56:17.372352+0000","last_in_change":"2023-11-24T07:56:09.073250+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-24T07:56:18.352823+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-11-24T07:56:22.185140+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-11-24T07:56:23.084326+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"6d8f9e9b-1782-4e41-928b-a39f2a8ebc06","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6800","nonce":3373878772},{"type":"v1","addr":"172.21.15.59:6801","nonce":3373878772}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6802","nonce":3373878772},{"type":"v1","addr":"172.21.15.59:6803","nonce":3373878772}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6806","nonce":3373878772},{"type":"v1","addr":"172.21.15.59:6807","nonce":3373878772}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6804","nonce":3373878772},{"type":"v1","addr":"172.21.15.59:6805","nonce":3373878772}]},"public_addr":"172.21.15.59:6801/3373878772","cluster_addr":"172.21.15.59:6803/3373878772","heartbeat_back_addr":"172.21.15.59:6807/3373878772","heartbeat_front_addr":"172.21.15.59:6805/3373878772","state":["exists","up"]},{"osd":1,"uuid":"e8d882aa-52e1-4c04-9e1e-c44ef9204a5f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6824","nonce":648258563},{"type":"v1","addr":"172.21.15.59:6825","nonce":648258563}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6826","nonce":648258563},{"type":"v1","addr":"172.21.15.59:6827","nonce":648258563}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6830","nonce":648258563},{"type":"v1","addr":"172.21.15.59:6831","nonce":648258563}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6828","nonce":648258563},{"type":"v1","addr":"172.21.15.59:6829","nonce":648258563}]},"public_addr":"172.21.15.59:6825/648258563","cluster_addr":"172.21.15.59:6827/648258563","heartbeat_back_addr":"172.21.15.59:6831/648258563","heartbeat_front_addr":"172.21.15.59:6829/648258563","state":["exists","up"]},{"osd":2,"uuid":"69eebfa0-695b-4b21-90b9-8a3537168a01","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6808","nonce":1840430654},{"type":"v1","addr":"172.21.15.59:6809","nonce":1840430654}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6810","nonce":1840430654},{"type":"v1","addr":"172.21.15.59:6811","nonce":1840430654}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6814","nonce":1840430654},{"type":"v1","addr":"172.21.15.59:6815","nonce":1840430654}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6812","nonce":1840430654},{"type":"v1","addr":"172.21.15.59:6813","nonce":1840430654}]},"public_addr":"172.21.15.59:6809/1840430654","cluster_addr":"172.21.15.59:6811/1840430654","heartbeat_back_addr":"172.21.15.59:6815/1840430654","heartbeat_front_addr":"172.21.15.59:6813/1840430654","state":["exists","up"]},{"osd":3,"uuid":"96fd77fc-f911-46ff-9445-6f0be2daa8f4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6816","nonce":2504025850},{"type":"v1","addr":"172.21.15.59:6817","nonce":2504025850}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6818","nonce":2504025850},{"type":"v1","addr":"172.21.15.59:6819","nonce":2504025850}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6822","nonce":2504025850},{"type":"v1","addr":"172.21.15.59:6823","nonce":2504025850}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6820","nonce":2504025850},{"type":"v1","addr":"172.21.15.59:6821","nonce":2504025850}]},"public_addr":"172.21.15.59:6817/2504025850","cluster_addr":"172.21.15.59:6819/2504025850","heartbeat_back_addr":"172.21.15.59:6823/2504025850","heartbeat_front_addr":"172.21.15.59:6821/2504025850","state":["exists","up"]},{"osd":4,"uuid":"7ac29fca-b5b2-4c95-9d69-2f164cbbc708","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6816","nonce":2382463431},{"type":"v1","addr":"172.21.15.76:6818","nonce":2382463431}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6820","nonce":2382463431},{"type":"v1","addr":"172.21.15.76:6822","nonce":2382463431}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6828","nonce":2382463431},{"type":"v1","addr":"172.21.15.76:6830","nonce":2382463431}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6824","nonce":2382463431},{"type":"v1","addr":"172.21.15.76:6826","nonce":2382463431}]},"public_addr":"172.21.15.76:6818/2382463431","cluster_addr":"172.21.15.76:6822/2382463431","heartbeat_back_addr":"172.21.15.76:6830/2382463431","heartbeat_front_addr":"172.21.15.76:6826/2382463431","state":["exists","up"]},{"osd":5,"uuid":"485bf711-3851-4238-a4f4-51d52e0a9c58","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6800","nonce":329334308},{"type":"v1","addr":"172.21.15.76:6802","nonce":329334308}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6805","nonce":329334308},{"type":"v1","addr":"172.21.15.76:6807","nonce":329334308}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6814","nonce":329334308},{"type":"v1","addr":"172.21.15.76:6815","nonce":329334308}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6810","nonce":329334308},{"type":"v1","addr":"172.21.15.76:6812","nonce":329334308}]},"public_addr":"172.21.15.76:6802/329334308","cluster_addr":"172.21.15.76:6807/329334308","heartbeat_back_addr":"172.21.15.76:6815/329334308","heartbeat_front_addr":"172.21.15.76:6812/329334308","state":["exists","up"]},{"osd":6,"uuid":"72b3f2fc-1e4a-4e32-968d-003647a0286f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6801","nonce":3946579410},{"type":"v1","addr":"172.21.15.76:6803","nonce":3946579410}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6804","nonce":3946579410},{"type":"v1","addr":"172.21.15.76:6806","nonce":3946579410}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6811","nonce":3946579410},{"type":"v1","addr":"172.21.15.76:6813","nonce":3946579410}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6808","nonce":3946579410},{"type":"v1","addr":"172.21.15.76:6809","nonce":3946579410}]},"public_addr":"172.21.15.76:6803/3946579410","cluster_addr":"172.21.15.76:6806/3946579410","heartbeat_back_addr":"172.21.15.76:6813/3946579410","heartbeat_front_addr":"172.21.15.76:6809/3946579410","state":["exists","up"]},{"osd":7,"uuid":"1ddceb90-6c5d-4d1a-ad56-99cb877f8c70","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6817","nonce":841761538},{"type":"v1","addr":"172.21.15.76:6819","nonce":841761538}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6821","nonce":841761538},{"type":"v1","addr":"172.21.15.76:6823","nonce":841761538}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6829","nonce":841761538},{"type":"v1","addr":"172.21.15.76:6831","nonce":841761538}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6825","nonce":841761538},{"type":"v1","addr":"172.21.15.76:6827","nonce":841761538}]},"public_addr":"172.21.15.76:6819/841761538","cluster_addr":"172.21.15.76:6823/841761538","heartbeat_back_addr":"172.21.15.76:6831/841761538","heartbeat_front_addr":"172.21.15.76:6827/841761538","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T07:56:14.294838+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T07:56:14.270779+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T07:56:14.231809+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T07:56:14.260983+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T07:56:14.302997+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T07:56:14.258600+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T07:56:14.251413+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T07:56:14.304727+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-24T07:56:41.279 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:41.277+0000 7f8b3a924700 1 -- 172.21.15.59:0/1562176941 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f8b1406aed0 msgr2=0x7f8b1406d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:56:41.279 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:41.277+0000 7f8b3a924700 1 --2- 172.21.15.59:0/1562176941 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f8b1406aed0 0x7f8b1406d390 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f8b1c004f30 tx=0x7f8b1c009040 comp rx=0 tx=0).stop 2023-11-24T07:56:41.279 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:41.277+0000 7f8b3a924700 1 -- 172.21.15.59:0/1562176941 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8b34142810 msgr2=0x7f8b3415cf30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:56:41.279 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:41.277+0000 7f8b3a924700 1 --2- 172.21.15.59:0/1562176941 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8b34142810 0x7f8b3415cf30 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f8b2400d800 tx=0x7f8b2400dbc0 comp rx=0 tx=0).stop 2023-11-24T07:56:41.279 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:41.277+0000 7f8b3a924700 1 -- 172.21.15.59:0/1562176941 shutdown_connections 2023-11-24T07:56:41.279 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:41.277+0000 7f8b3a924700 1 --2- 172.21.15.59:0/1562176941 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f8b1406aed0 0x7f8b1406d390 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:56:41.279 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:41.277+0000 7f8b3a924700 1 --2- 172.21.15.59:0/1562176941 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f8b341459a0 0x7f8b340ae8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:56:41.279 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:41.277+0000 7f8b3a924700 1 --2- 172.21.15.59:0/1562176941 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8b3406b6e0 0x7f8b340cd330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:56:41.280 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:41.277+0000 7f8b3a924700 1 --2- 172.21.15.59:0/1562176941 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8b34142810 0x7f8b3415cf30 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:56:41.280 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:41.277+0000 7f8b3a924700 1 -- 172.21.15.59:0/1562176941 >> 172.21.15.59:0/1562176941 conn(0x7f8b340b2740 msgr2=0x7f8b34144e60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:56:41.280 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:41.277+0000 7f8b3a924700 1 -- 172.21.15.59:0/1562176941 shutdown_connections 2023-11-24T07:56:41.280 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:41.277+0000 7f8b3a924700 1 -- 172.21.15.59:0/1562176941 wait complete. 2023-11-24T07:56:41.291 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-24T07:56:18.352823+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-24T07:56:22.185140+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-11-24T07:56:23.084326+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-24T07:56:41.292 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-24T07:56:41.420 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:41.419+0000 7f8a2624f700 1 Processor -- start 2023-11-24T07:56:41.420 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:41.419+0000 7f8a2624f700 1 -- start start 2023-11-24T07:56:41.423 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:41.419+0000 7f8a2624f700 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f8a2015bde0 0x7f8a2015c1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:56:41.423 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:41.419+0000 7f8a2624f700 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f8a200cf2b0 0x7f8a200d01b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:56:41.423 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:41.419+0000 7f8a2624f700 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8a200d0a90 con 0x7f8a200ce5c0 2023-11-24T07:56:41.423 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:41.419+0000 7f8a2624f700 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8a2006ee10 con 0x7f8a2015bde0 2023-11-24T07:56:41.423 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:41.419+0000 7f8a2624f700 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8a2006f0f0 con 0x7f8a200cf2b0 2023-11-24T07:56:41.423 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:41.420+0000 7f8a1f7fe700 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f8a2015bde0 0x7f8a2015c1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:56:41.423 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:41.420+0000 7f8a1effd700 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f8a200cf2b0 0x7f8a200d01b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:56:41.423 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:41.420+0000 7f8a1ffff700 1 --1- >> v1:172.21.15.76:6789/0 conn(0x7f8a200ce5c0 0x7f8a200cc670 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.59:41160/0 (socket says 172.21.15.59:41160) 2023-11-24T07:56:41.424 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:41.420+0000 7f8a1ffff700 1 -- 172.21.15.59:0/2475042182 learned_addr learned my addr 172.21.15.59:0/2475042182 (peer_addr_for_me v1:172.21.15.59:0/0) 2023-11-24T07:56:41.424 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:41.420+0000 7f8a1f7fe700 1 -- 172.21.15.59:0/2475042182 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f8a200cf2b0 msgr2=0x7f8a200d01b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T07:56:41.424 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:41.420+0000 7f8a1f7fe700 1 --2- 172.21.15.59:0/2475042182 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f8a200cf2b0 0x7f8a200d01b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:56:41.424 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:41.420+0000 7f8a1f7fe700 1 -- 172.21.15.59:0/2475042182 >> v1:172.21.15.76:6789/0 conn(0x7f8a200ce5c0 legacy=0x7f8a200cc670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:56:41.424 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:41.420+0000 7f8a1f7fe700 1 -- 172.21.15.59:0/2475042182 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8a2006f3d0 con 0x7f8a2015bde0 2023-11-24T07:56:41.424 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:41.420+0000 7f8a1e7fc700 1 -- 172.21.15.59:0/2475042182 <== mon.1 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1473143419 0 0) 0x7f8a200d0a90 con 0x7f8a200ce5c0 2023-11-24T07:56:41.424 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:41.420+0000 7f8a1effd700 1 --2- 172.21.15.59:0/2475042182 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f8a200cf2b0 0x7f8a200d01b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-24T07:56:41.424 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:41.420+0000 7f8a1f7fe700 1 --2- 172.21.15.59:0/2475042182 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8a2015bde0 0x7f8a2015c1c0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f8a10009840 tx=0x7f8a10009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ef2ec79093a868bf server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:56:41.424 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:41.421+0000 7f8a1e7fc700 1 -- 172.21.15.59:0/2475042182 <== mon.1 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a10011830 con 0x7f8a2015bde0 2023-11-24T07:56:41.424 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:41.421+0000 7f8a1e7fc700 1 -- 172.21.15.59:0/2475042182 <== mon.1 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8a10011990 con 0x7f8a2015bde0 2023-11-24T07:56:41.424 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:41.421+0000 7f8a1e7fc700 1 -- 172.21.15.59:0/2475042182 <== mon.1 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a10011d20 con 0x7f8a2015bde0 2023-11-24T07:56:41.424 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:41.421+0000 7f8a2624f700 1 -- 172.21.15.59:0/2475042182 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8a2015bde0 msgr2=0x7f8a2015c1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:56:41.424 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:41.421+0000 7f8a2624f700 1 --2- 172.21.15.59:0/2475042182 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8a2015bde0 0x7f8a2015c1c0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f8a10009840 tx=0x7f8a10009b50 comp rx=0 tx=0).stop 2023-11-24T07:56:41.425 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:41.421+0000 7f8a2624f700 1 -- 172.21.15.59:0/2475042182 shutdown_connections 2023-11-24T07:56:41.425 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:41.421+0000 7f8a2624f700 1 --2- 172.21.15.59:0/2475042182 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8a2015bde0 0x7f8a2015c1c0 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:56:41.425 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:41.421+0000 7f8a2624f700 1 --2- 172.21.15.59:0/2475042182 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f8a200cf2b0 0x7f8a200d01b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:56:41.425 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:41.421+0000 7f8a2624f700 1 -- 172.21.15.59:0/2475042182 >> 172.21.15.59:0/2475042182 conn(0x7f8a200b3720 msgr2=0x7f8a200b3b30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:56:41.425 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:41.421+0000 7f8a2624f700 1 -- 172.21.15.59:0/2475042182 shutdown_connections 2023-11-24T07:56:41.425 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:41.421+0000 7f8a2624f700 1 -- 172.21.15.59:0/2475042182 wait complete. 2023-11-24T07:56:41.429 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:41.422+0000 7f8a2624f700 1 Processor -- start 2023-11-24T07:56:41.429 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:41.423+0000 7f8a2624f700 1 -- start start 2023-11-24T07:56:41.429 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:41.423+0000 7f8a2624f700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8a200ce5c0 0x7f8a201595a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:56:41.429 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:41.423+0000 7f8a2624f700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8a200cf2b0 0x7f8a2006fbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:56:41.429 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:41.423+0000 7f8a2624f700 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f8a2015bde0 0x7f8a20070110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:56:41.429 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:41.423+0000 7f8a2624f700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a200b6860 con 0x7f8a200cf2b0 2023-11-24T07:56:41.429 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:41.423+0000 7f8a2624f700 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a200b5fc0 con 0x7f8a200ce5c0 2023-11-24T07:56:41.429 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:41.423+0000 7f8a2624f700 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8a200b62a0 con 0x7f8a2015bde0 2023-11-24T07:56:41.429 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:41.423+0000 7f8a1effd700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8a200cf2b0 0x7f8a2006fbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:56:41.430 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:41.424+0000 7f8a1effd700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8a200cf2b0 0x7f8a2006fbd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:50410/0 (socket says 172.21.15.59:50410) 2023-11-24T07:56:41.430 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:41.424+0000 7f8a1effd700 1 -- 172.21.15.59:0/139048557 learned_addr learned my addr 172.21.15.59:0/139048557 (peer_addr_for_me v2:172.21.15.59:0/0) 2023-11-24T07:56:41.430 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:41.424+0000 7f8a1ffff700 1 --2- 172.21.15.59:0/139048557 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f8a2015bde0 0x7f8a20070110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:56:41.430 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:41.424+0000 7f8a1f7fe700 1 --2- 172.21.15.59:0/139048557 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8a200ce5c0 0x7f8a201595a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:56:41.430 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:41.424+0000 7f8a1effd700 1 -- 172.21.15.59:0/139048557 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f8a2015bde0 msgr2=0x7f8a20070110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:56:41.430 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:41.424+0000 7f8a1effd700 1 --2- 172.21.15.59:0/139048557 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f8a2015bde0 0x7f8a20070110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:56:41.430 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:41.424+0000 7f8a1effd700 1 -- 172.21.15.59:0/139048557 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8a200ce5c0 msgr2=0x7f8a201595a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:56:41.430 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:41.424+0000 7f8a1effd700 1 --2- 172.21.15.59:0/139048557 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8a200ce5c0 0x7f8a201595a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:56:41.430 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:41.424+0000 7f8a1effd700 1 -- 172.21.15.59:0/139048557 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8a140094a0 con 0x7f8a200cf2b0 2023-11-24T07:56:41.430 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:41.424+0000 7f8a1ffff700 1 --2- 172.21.15.59:0/139048557 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f8a2015bde0 0x7f8a20070110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T07:56:41.430 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:41.424+0000 7f8a1f7fe700 1 --2- 172.21.15.59:0/139048557 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8a200ce5c0 0x7f8a201595a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T07:56:41.430 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:41.424+0000 7f8a1effd700 1 --2- 172.21.15.59:0/139048557 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8a200cf2b0 0x7f8a2006fbd0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f8a1400ebb0 tx=0x7f8a1400ef70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:56:41.431 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:41.424+0000 7f8a07fff700 1 -- 172.21.15.59:0/139048557 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a1400c910 con 0x7f8a200cf2b0 2023-11-24T07:56:41.431 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:41.424+0000 7f8a07fff700 1 -- 172.21.15.59:0/139048557 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8a1400ca70 con 0x7f8a200cf2b0 2023-11-24T07:56:41.431 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:41.424+0000 7f8a07fff700 1 -- 172.21.15.59:0/139048557 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a1400ce00 con 0x7f8a200cf2b0 2023-11-24T07:56:41.431 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:41.424+0000 7f8a2624f700 1 -- 172.21.15.59:0/139048557 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8a100094a0 con 0x7f8a200cf2b0 2023-11-24T07:56:41.431 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:41.424+0000 7f8a2624f700 1 -- 172.21.15.59:0/139048557 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8a200c9370 con 0x7f8a200cf2b0 2023-11-24T07:56:41.431 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:41.425+0000 7f8a2624f700 1 -- 172.21.15.59:0/139048557 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8a200c9530 con 0x7f8a200cf2b0 2023-11-24T07:56:41.431 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:41.426+0000 7f8a07fff700 1 -- 172.21.15.59:0/139048557 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f8a14013470 con 0x7f8a200cf2b0 2023-11-24T07:56:41.431 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:41.426+0000 7f8a07fff700 1 --2- 172.21.15.59:0/139048557 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f8a0c06af00 0x7f8a0c06d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:56:41.431 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:41.426+0000 7f8a07fff700 1 -- 172.21.15.59:0/139048557 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f8a14089720 con 0x7f8a200cf2b0 2023-11-24T07:56:41.431 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:41.426+0000 7f8a1f7fe700 1 --2- 172.21.15.59:0/139048557 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f8a0c06af00 0x7f8a0c06d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:56:41.431 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:41.427+0000 7f8a1f7fe700 1 --2- 172.21.15.59:0/139048557 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f8a0c06af00 0x7f8a0c06d3c0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f8a100090b0 tx=0x7f8a10008030 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:56:41.432 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:41.431+0000 7f8a07fff700 1 -- 172.21.15.59:0/139048557 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f8a140547d0 con 0x7f8a200cf2b0 2023-11-24T07:56:41.635 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:41.633+0000 7f8a2624f700 1 -- 172.21.15.59:0/139048557 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f8a20075480 con 0x7f8a200cf2b0 2023-11-24T07:56:41.635 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:41.633+0000 7f8a07fff700 1 -- 172.21.15.59:0/139048557 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v20) v1 ==== 93+0+10 (secure 0 0 0) 0x7f8a14057df0 con 0x7f8a200cf2b0 2023-11-24T07:56:41.635 INFO:teuthology.orchestra.run.smithi059.stdout:pg_num: 1 2023-11-24T07:56:41.638 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:41.636+0000 7f8a2624f700 1 -- 172.21.15.59:0/139048557 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f8a0c06af00 msgr2=0x7f8a0c06d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:56:41.638 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:41.636+0000 7f8a2624f700 1 --2- 172.21.15.59:0/139048557 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f8a0c06af00 0x7f8a0c06d3c0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f8a100090b0 tx=0x7f8a10008030 comp rx=0 tx=0).stop 2023-11-24T07:56:41.638 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:41.636+0000 7f8a2624f700 1 -- 172.21.15.59:0/139048557 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8a200cf2b0 msgr2=0x7f8a2006fbd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:56:41.638 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:41.636+0000 7f8a2624f700 1 --2- 172.21.15.59:0/139048557 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8a200cf2b0 0x7f8a2006fbd0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f8a1400ebb0 tx=0x7f8a1400ef70 comp rx=0 tx=0).stop 2023-11-24T07:56:41.638 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:41.636+0000 7f8a2624f700 1 -- 172.21.15.59:0/139048557 shutdown_connections 2023-11-24T07:56:41.638 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:41.636+0000 7f8a2624f700 1 --2- 172.21.15.59:0/139048557 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f8a0c06af00 0x7f8a0c06d3c0 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:56:41.639 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:41.636+0000 7f8a2624f700 1 --2- 172.21.15.59:0/139048557 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f8a2015bde0 0x7f8a20070110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:56:41.639 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:41.636+0000 7f8a2624f700 1 --2- 172.21.15.59:0/139048557 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8a200ce5c0 0x7f8a201595a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:56:41.639 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:41.636+0000 7f8a2624f700 1 --2- 172.21.15.59:0/139048557 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8a200cf2b0 0x7f8a2006fbd0 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:56:41.639 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:41.636+0000 7f8a2624f700 1 -- 172.21.15.59:0/139048557 >> 172.21.15.59:0/139048557 conn(0x7f8a200b3720 msgr2=0x7f8a2015af80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:56:41.639 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:41.636+0000 7f8a2624f700 1 -- 172.21.15.59:0/139048557 shutdown_connections 2023-11-24T07:56:41.639 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:41.636+0000 7f8a2624f700 1 -- 172.21.15.59:0/139048557 wait complete. 2023-11-24T07:56:41.650 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-24T07:56:41.777 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:41.776+0000 7f415f60e700 1 Processor -- start 2023-11-24T07:56:41.778 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:41.776+0000 7f415f60e700 1 -- start start 2023-11-24T07:56:41.780 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:41.777+0000 7f415f60e700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f415815bde0 0x7f415815c1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:56:41.780 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:41.777+0000 7f415f60e700 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f41580d0a90 con 0x7f41580ce5c0 2023-11-24T07:56:41.780 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:41.777+0000 7f415f60e700 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f415806ee10 con 0x7f41580cf2b0 2023-11-24T07:56:41.780 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:41.777+0000 7f415f60e700 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f415806f0f0 con 0x7f415815bde0 2023-11-24T07:56:41.781 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:41.777+0000 7f415d3aa700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f415815bde0 0x7f415815c1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:56:41.781 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:41.777+0000 7f415d3aa700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f415815bde0 0x7f415815c1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:50418/0 (socket says 172.21.15.59:50418) 2023-11-24T07:56:41.781 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:41.777+0000 7f415d3aa700 1 -- 172.21.15.59:0/2234845584 learned_addr learned my addr 172.21.15.59:0/2234845584 (peer_addr_for_me v2:172.21.15.59:0/0) 2023-11-24T07:56:41.781 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:41.777+0000 7f415dbab700 1 --1- 172.21.15.59:0/2234845584 >> v1:172.21.15.59:6789/0 conn(0x7f41580ce5c0 0x7f415806fbd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:53896/0 (socket says 172.21.15.59:53896) 2023-11-24T07:56:41.781 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:41.777+0000 7f415d3aa700 1 -- 172.21.15.59:0/2234845584 >> v1:172.21.15.76:6789/0 conn(0x7f41580cf2b0 legacy=0x7f41580d01b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T07:56:41.781 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:41.777+0000 7f415d3aa700 1 -- 172.21.15.59:0/2234845584 >> v1:172.21.15.59:6789/0 conn(0x7f41580ce5c0 legacy=0x7f415806fbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:56:41.781 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:41.777+0000 7f415d3aa700 1 -- 172.21.15.59:0/2234845584 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f415806f3d0 con 0x7f415815bde0 2023-11-24T07:56:41.781 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:41.777+0000 7f415d3aa700 1 --2- 172.21.15.59:0/2234845584 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f415815bde0 0x7f415815c1c0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f41540098f0 tx=0x7f4154009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=57ebd2da1711e1cc server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:56:41.782 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:41.777+0000 7f414ffff700 1 -- 172.21.15.59:0/2234845584 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4154004230 con 0x7f415815bde0 2023-11-24T07:56:41.782 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:41.777+0000 7f414ffff700 1 -- 172.21.15.59:0/2234845584 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4154004390 con 0x7f415815bde0 2023-11-24T07:56:41.782 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:41.777+0000 7f414ffff700 1 -- 172.21.15.59:0/2234845584 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41540036a0 con 0x7f415815bde0 2023-11-24T07:56:41.782 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:41.778+0000 7f415f60e700 1 -- 172.21.15.59:0/2234845584 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f415815bde0 msgr2=0x7f415815c1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:56:41.782 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:41.778+0000 7f415f60e700 1 --2- 172.21.15.59:0/2234845584 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f415815bde0 0x7f415815c1c0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f41540098f0 tx=0x7f4154009c00 comp rx=0 tx=0).stop 2023-11-24T07:56:41.782 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:41.778+0000 7f415f60e700 1 -- 172.21.15.59:0/2234845584 shutdown_connections 2023-11-24T07:56:41.782 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:41.778+0000 7f415f60e700 1 --2- 172.21.15.59:0/2234845584 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f415815bde0 0x7f415815c1c0 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:56:41.782 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:41.778+0000 7f415f60e700 1 -- 172.21.15.59:0/2234845584 >> 172.21.15.59:0/2234845584 conn(0x7f41580b3720 msgr2=0x7f41580b3b30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:56:41.782 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:41.778+0000 7f415f60e700 1 -- 172.21.15.59:0/2234845584 shutdown_connections 2023-11-24T07:56:41.782 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:41.778+0000 7f415f60e700 1 -- 172.21.15.59:0/2234845584 wait complete. 2023-11-24T07:56:41.785 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:41.779+0000 7f415f60e700 1 Processor -- start 2023-11-24T07:56:41.785 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:41.779+0000 7f415f60e700 1 -- start start 2023-11-24T07:56:41.785 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:41.780+0000 7f415f60e700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f41580ce5c0 0x7f415814ba40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:56:41.785 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:41.780+0000 7f415f60e700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f41580cf2b0 0x7f41580c7c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:56:41.785 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:41.780+0000 7f415f60e700 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f415815bde0 0x7f4158149000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:56:41.785 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:41.780+0000 7f415f60e700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41580ca720 con 0x7f41580cf2b0 2023-11-24T07:56:41.785 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:41.780+0000 7f415f60e700 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41580c8fd0 con 0x7f41580ce5c0 2023-11-24T07:56:41.785 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:41.780+0000 7f415f60e700 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f41580ca9c0 con 0x7f415815bde0 2023-11-24T07:56:41.785 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:41.780+0000 7f415cba9700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f41580cf2b0 0x7f41580c7c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:56:41.786 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:41.780+0000 7f415cba9700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f41580cf2b0 0x7f41580c7c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:50424/0 (socket says 172.21.15.59:50424) 2023-11-24T07:56:41.786 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:41.780+0000 7f415cba9700 1 -- 172.21.15.59:0/44743392 learned_addr learned my addr 172.21.15.59:0/44743392 (peer_addr_for_me v2:172.21.15.59:0/0) 2023-11-24T07:56:41.787 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:41.780+0000 7f415dbab700 1 --2- 172.21.15.59:0/44743392 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f415815bde0 0x7f4158149000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:56:41.787 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:41.780+0000 7f415d3aa700 1 --2- 172.21.15.59:0/44743392 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f41580ce5c0 0x7f415814ba40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:56:41.787 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:41.780+0000 7f415cba9700 1 -- 172.21.15.59:0/44743392 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f415815bde0 msgr2=0x7f4158149000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:56:41.787 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:41.780+0000 7f415cba9700 1 --2- 172.21.15.59:0/44743392 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f415815bde0 0x7f4158149000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:56:41.787 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:41.780+0000 7f415cba9700 1 -- 172.21.15.59:0/44743392 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f41580ce5c0 msgr2=0x7f415814ba40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:56:41.787 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:41.780+0000 7f415cba9700 1 --2- 172.21.15.59:0/44743392 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f41580ce5c0 0x7f415814ba40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:56:41.787 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:41.780+0000 7f415cba9700 1 -- 172.21.15.59:0/44743392 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4154009550 con 0x7f41580cf2b0 2023-11-24T07:56:41.787 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:41.781+0000 7f415cba9700 1 --2- 172.21.15.59:0/44743392 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f41580cf2b0 0x7f41580c7c10 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f414800c4a0 tx=0x7f414800c860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:56:41.787 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:41.781+0000 7f414dffb700 1 -- 172.21.15.59:0/44743392 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4148011790 con 0x7f41580cf2b0 2023-11-24T07:56:41.788 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:41.781+0000 7f414dffb700 1 -- 172.21.15.59:0/44743392 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f41480118f0 con 0x7f41580cf2b0 2023-11-24T07:56:41.788 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:41.781+0000 7f414dffb700 1 -- 172.21.15.59:0/44743392 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4148011ca0 con 0x7f41580cf2b0 2023-11-24T07:56:41.788 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:41.781+0000 7f415f60e700 1 -- 172.21.15.59:0/44743392 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f41580caca0 con 0x7f41580cf2b0 2023-11-24T07:56:41.788 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:41.781+0000 7f415f60e700 1 -- 172.21.15.59:0/44743392 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f415814bf80 con 0x7f41580cf2b0 2023-11-24T07:56:41.788 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:41.782+0000 7f415f60e700 1 -- 172.21.15.59:0/44743392 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4158075480 con 0x7f41580cf2b0 2023-11-24T07:56:41.788 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:41.782+0000 7f414dffb700 1 -- 172.21.15.59:0/44743392 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f414801c460 con 0x7f41580cf2b0 2023-11-24T07:56:41.788 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:41.783+0000 7f414dffb700 1 --2- 172.21.15.59:0/44743392 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f413c06aed0 0x7f413c06d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:56:41.788 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:41.783+0000 7f414dffb700 1 -- 172.21.15.59:0/44743392 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f4148089500 con 0x7f41580cf2b0 2023-11-24T07:56:41.789 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:41.783+0000 7f415d3aa700 1 --2- 172.21.15.59:0/44743392 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f413c06aed0 0x7f413c06d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:56:41.789 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:41.783+0000 7f415d3aa700 1 --2- 172.21.15.59:0/44743392 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f413c06aed0 0x7f413c06d390 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f4154009130 tx=0x7f4154008030 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:56:41.789 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:41.787+0000 7f414dffb700 1 -- 172.21.15.59:0/44743392 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f4148054480 con 0x7f41580cf2b0 2023-11-24T07:56:41.989 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:41.986+0000 7f415f60e700 1 -- 172.21.15.59:0/44743392 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f41580cb1c0 con 0x7f41580cf2b0 2023-11-24T07:56:41.990 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:41.987+0000 7f414dffb700 1 -- 172.21.15.59:0/44743392 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v20) v1 ==== 104+0+11 (secure 0 0 0) 0x7f41580cb1c0 con 0x7f41580cf2b0 2023-11-24T07:56:41.990 INFO:teuthology.orchestra.run.smithi059.stdout:pg_num: 32 2023-11-24T07:56:41.991 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:41.989+0000 7f415f60e700 1 -- 172.21.15.59:0/44743392 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f413c06aed0 msgr2=0x7f413c06d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:56:41.991 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:41.989+0000 7f415f60e700 1 --2- 172.21.15.59:0/44743392 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f413c06aed0 0x7f413c06d390 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f4154009130 tx=0x7f4154008030 comp rx=0 tx=0).stop 2023-11-24T07:56:41.991 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:41.989+0000 7f415f60e700 1 -- 172.21.15.59:0/44743392 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f41580cf2b0 msgr2=0x7f41580c7c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:56:41.991 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:41.989+0000 7f415f60e700 1 --2- 172.21.15.59:0/44743392 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f41580cf2b0 0x7f41580c7c10 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f414800c4a0 tx=0x7f414800c860 comp rx=0 tx=0).stop 2023-11-24T07:56:41.992 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:41.989+0000 7f415f60e700 1 -- 172.21.15.59:0/44743392 shutdown_connections 2023-11-24T07:56:41.992 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:41.989+0000 7f415f60e700 1 --2- 172.21.15.59:0/44743392 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f413c06aed0 0x7f413c06d390 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:56:41.992 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:41.989+0000 7f415f60e700 1 --2- 172.21.15.59:0/44743392 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f415815bde0 0x7f4158149000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:56:41.992 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:41.989+0000 7f415f60e700 1 --2- 172.21.15.59:0/44743392 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f41580ce5c0 0x7f415814ba40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:56:41.993 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:41.989+0000 7f415f60e700 1 --2- 172.21.15.59:0/44743392 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f41580cf2b0 0x7f41580c7c10 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:56:41.993 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:41.989+0000 7f415f60e700 1 -- 172.21.15.59:0/44743392 >> 172.21.15.59:0/44743392 conn(0x7f41580b3720 msgr2=0x7f41580b3b00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:56:41.993 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:41.989+0000 7f415f60e700 1 -- 172.21.15.59:0/44743392 shutdown_connections 2023-11-24T07:56:41.993 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:41.989+0000 7f415f60e700 1 -- 172.21.15.59:0/44743392 wait complete. 2023-11-24T07:56:42.003 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-24T07:56:42.131 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:42.129+0000 7f0fdb565700 1 Processor -- start 2023-11-24T07:56:42.131 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:42.129+0000 7f0fdb565700 1 -- start start 2023-11-24T07:56:42.133 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:42.130+0000 7f0fdb565700 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f0fd4144c10 0x7f0fd406afd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:56:42.133 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:42.130+0000 7f0fdb565700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f0fd4145100 0x7f0fd406b510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:56:42.133 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:42.130+0000 7f0fdb565700 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f0fd4145650 0x7f0fd4071bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:56:42.133 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:42.130+0000 7f0fdb565700 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0fd406f500 con 0x7f0fd4145100 2023-11-24T07:56:42.134 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:42.130+0000 7f0fdb565700 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0fd40cd610 con 0x7f0fd4144c10 2023-11-24T07:56:42.134 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:42.130+0000 7f0fdb565700 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0fd4160720 con 0x7f0fd4145650 2023-11-24T07:56:42.134 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:42.130+0000 7f0fd8b00700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f0fd4145100 0x7f0fd406b510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:56:42.134 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:42.130+0000 7f0fd8b00700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f0fd4145100 0x7f0fd406b510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:50434/0 (socket says 172.21.15.59:50434) 2023-11-24T07:56:42.134 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:42.130+0000 7f0fd8b00700 1 -- 172.21.15.59:0/325801964 learned_addr learned my addr 172.21.15.59:0/325801964 (peer_addr_for_me v2:172.21.15.59:0/0) 2023-11-24T07:56:42.134 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:42.130+0000 7f0fd9301700 1 --2- 172.21.15.59:0/325801964 >> v2:172.21.15.76:3300/0 conn(0x7f0fd4144c10 0x7f0fd406afd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:56:42.134 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:42.130+0000 7f0fd9b02700 1 --2- 172.21.15.59:0/325801964 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f0fd4145650 0x7f0fd4071bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:56:42.134 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:42.130+0000 7f0fd8b00700 1 -- 172.21.15.59:0/325801964 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f0fd4145650 msgr2=0x7f0fd4071bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T07:56:42.134 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:42.130+0000 7f0fd8b00700 1 --2- 172.21.15.59:0/325801964 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f0fd4145650 0x7f0fd4071bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:56:42.134 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:42.130+0000 7f0fd8b00700 1 -- 172.21.15.59:0/325801964 >> v2:172.21.15.76:3300/0 conn(0x7f0fd4144c10 msgr2=0x7f0fd406afd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T07:56:42.134 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:42.130+0000 7f0fd8b00700 1 --2- 172.21.15.59:0/325801964 >> v2:172.21.15.76:3300/0 conn(0x7f0fd4144c10 0x7f0fd406afd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:56:42.134 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:42.130+0000 7f0fd8b00700 1 -- 172.21.15.59:0/325801964 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0fd406f710 con 0x7f0fd4145100 2023-11-24T07:56:42.134 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:42.130+0000 7f0fd9b02700 1 --2- 172.21.15.59:0/325801964 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f0fd4145650 0x7f0fd4071bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T07:56:42.135 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:42.131+0000 7f0fd8b00700 1 --2- 172.21.15.59:0/325801964 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0fd4145100 0x7f0fd406b510 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f0fd40706e0 tx=0x7f0fd0009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bff84fd38ea36163 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:56:42.135 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:42.131+0000 7f0fcbfff700 1 -- 172.21.15.59:0/325801964 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0fd000b870 con 0x7f0fd4145100 2023-11-24T07:56:42.135 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:42.131+0000 7f0fcbfff700 1 -- 172.21.15.59:0/325801964 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0fd000b9d0 con 0x7f0fd4145100 2023-11-24T07:56:42.135 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:42.131+0000 7f0fcbfff700 1 -- 172.21.15.59:0/325801964 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0fd000bd60 con 0x7f0fd4145100 2023-11-24T07:56:42.135 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:42.131+0000 7f0fdb565700 1 -- 172.21.15.59:0/325801964 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0fd4145100 msgr2=0x7f0fd406b510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:56:42.135 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:42.131+0000 7f0fdb565700 1 --2- 172.21.15.59:0/325801964 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0fd4145100 0x7f0fd406b510 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f0fd40706e0 tx=0x7f0fd0009bd0 comp rx=0 tx=0).stop 2023-11-24T07:56:42.135 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:42.131+0000 7f0fdb565700 1 -- 172.21.15.59:0/325801964 shutdown_connections 2023-11-24T07:56:42.135 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:42.131+0000 7f0fdb565700 1 --2- 172.21.15.59:0/325801964 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f0fd4145650 0x7f0fd4071bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:56:42.135 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:42.131+0000 7f0fdb565700 1 --2- 172.21.15.59:0/325801964 >> v2:172.21.15.76:3300/0 conn(0x7f0fd4144c10 0x7f0fd406afd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:56:42.135 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:42.131+0000 7f0fdb565700 1 --2- 172.21.15.59:0/325801964 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0fd4145100 0x7f0fd406b510 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:56:42.135 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:42.131+0000 7f0fdb565700 1 -- 172.21.15.59:0/325801964 >> 172.21.15.59:0/325801964 conn(0x7f0fd40b2740 msgr2=0x7f0fd40b2b20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:56:42.135 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:42.131+0000 7f0fdb565700 1 -- 172.21.15.59:0/325801964 shutdown_connections 2023-11-24T07:56:42.136 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:42.131+0000 7f0fdb565700 1 -- 172.21.15.59:0/325801964 wait complete. 2023-11-24T07:56:42.138 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:42.133+0000 7f0fdb565700 1 Processor -- start 2023-11-24T07:56:42.139 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:42.133+0000 7f0fdb565700 1 -- start start 2023-11-24T07:56:42.139 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:42.133+0000 7f0fdb565700 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f0fd4144c10 0x7f0fd415ae90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:56:42.139 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:42.133+0000 7f0fdb565700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0fd4145100 0x7f0fd40ad250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:56:42.139 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:42.133+0000 7f0fdb565700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0fd4145650 0x7f0fd40a9450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:56:42.139 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:42.133+0000 7f0fdb565700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0fd40aa3b0 con 0x7f0fd4145100 2023-11-24T07:56:42.139 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:42.133+0000 7f0fdb565700 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0fd40aac10 con 0x7f0fd4145650 2023-11-24T07:56:42.139 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:42.133+0000 7f0fdb565700 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0fd40ad790 con 0x7f0fd4144c10 2023-11-24T07:56:42.139 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:42.133+0000 7f0fd8b00700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0fd4145100 0x7f0fd40ad250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:56:42.139 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:42.133+0000 7f0fd8b00700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0fd4145100 0x7f0fd40ad250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:50440/0 (socket says 172.21.15.59:50440) 2023-11-24T07:56:42.139 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:42.133+0000 7f0fd8b00700 1 -- 172.21.15.59:0/1450082574 learned_addr learned my addr 172.21.15.59:0/1450082574 (peer_addr_for_me v2:172.21.15.59:0/0) 2023-11-24T07:56:42.139 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:42.134+0000 7f0fd9301700 1 --2- 172.21.15.59:0/1450082574 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f0fd4144c10 0x7f0fd415ae90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:56:42.139 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:42.134+0000 7f0fd9b02700 1 --2- 172.21.15.59:0/1450082574 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0fd4145650 0x7f0fd40a9450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:56:42.140 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:42.134+0000 7f0fd8b00700 1 -- 172.21.15.59:0/1450082574 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f0fd4144c10 msgr2=0x7f0fd415ae90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:56:42.140 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:42.134+0000 7f0fd8b00700 1 --2- 172.21.15.59:0/1450082574 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f0fd4144c10 0x7f0fd415ae90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:56:42.140 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:42.134+0000 7f0fd8b00700 1 -- 172.21.15.59:0/1450082574 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0fd4145650 msgr2=0x7f0fd40a9450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:56:42.140 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:42.134+0000 7f0fd8b00700 1 --2- 172.21.15.59:0/1450082574 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0fd4145650 0x7f0fd40a9450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:56:42.140 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:42.134+0000 7f0fd8b00700 1 -- 172.21.15.59:0/1450082574 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0fd0009550 con 0x7f0fd4145100 2023-11-24T07:56:42.140 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:42.134+0000 7f0fd9b02700 1 --2- 172.21.15.59:0/1450082574 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0fd4145650 0x7f0fd40a9450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T07:56:42.140 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:42.134+0000 7f0fd8b00700 1 --2- 172.21.15.59:0/1450082574 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0fd4145100 0x7f0fd40ad250 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f0fd0000c00 tx=0x7f0fd0004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:56:42.140 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:42.134+0000 7f0fc9ffb700 1 -- 172.21.15.59:0/1450082574 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0fd0004310 con 0x7f0fd4145100 2023-11-24T07:56:42.140 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:42.134+0000 7f0fc9ffb700 1 -- 172.21.15.59:0/1450082574 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0fd001f070 con 0x7f0fd4145100 2023-11-24T07:56:42.140 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:42.134+0000 7f0fdb565700 1 -- 172.21.15.59:0/1450082574 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0fd40a9df0 con 0x7f0fd4145100 2023-11-24T07:56:42.140 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:42.134+0000 7f0fc9ffb700 1 -- 172.21.15.59:0/1450082574 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0fd00036a0 con 0x7f0fd4145100 2023-11-24T07:56:42.140 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:42.134+0000 7f0fdb565700 1 -- 172.21.15.59:0/1450082574 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0fd40aa8a0 con 0x7f0fd4145100 2023-11-24T07:56:42.140 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:42.135+0000 7f0fdb565700 1 -- 172.21.15.59:0/1450082574 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0fd40aa700 con 0x7f0fd4145100 2023-11-24T07:56:42.140 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:42.136+0000 7f0fc9ffb700 1 -- 172.21.15.59:0/1450082574 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f0fd0021030 con 0x7f0fd4145100 2023-11-24T07:56:42.141 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:42.137+0000 7f0fc9ffb700 1 --2- 172.21.15.59:0/1450082574 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f0fb806aed0 0x7f0fb806d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:56:42.141 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:42.137+0000 7f0fc9ffb700 1 -- 172.21.15.59:0/1450082574 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f0fd008ac90 con 0x7f0fd4145100 2023-11-24T07:56:42.141 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:42.137+0000 7f0fd9301700 1 --2- 172.21.15.59:0/1450082574 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f0fb806aed0 0x7f0fb806d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:56:42.141 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:42.137+0000 7f0fd9301700 1 --2- 172.21.15.59:0/1450082574 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f0fb806aed0 0x7f0fb806d390 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f0fc400c900 tx=0x7f0fc4008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:56:42.142 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:42.141+0000 7f0fc9ffb700 1 -- 172.21.15.59:0/1450082574 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f0fd0055fd0 con 0x7f0fd4145100 2023-11-24T07:56:42.344 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:42.342+0000 7f0fdb565700 1 -- 172.21.15.59:0/1450082574 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f0fd40aa700 con 0x7f0fd4145100 2023-11-24T07:56:42.344 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:42.343+0000 7f0fc9ffb700 1 -- 172.21.15.59:0/1450082574 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v20) v1 ==== 100+0+11 (secure 0 0 0) 0x7f0fd40aa700 con 0x7f0fd4145100 2023-11-24T07:56:42.344 INFO:teuthology.orchestra.run.smithi059.stdout:pg_num: 64 2023-11-24T07:56:42.347 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:42.345+0000 7f0fdb565700 1 -- 172.21.15.59:0/1450082574 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f0fb806aed0 msgr2=0x7f0fb806d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:56:42.347 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:42.345+0000 7f0fdb565700 1 --2- 172.21.15.59:0/1450082574 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f0fb806aed0 0x7f0fb806d390 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f0fc400c900 tx=0x7f0fc4008040 comp rx=0 tx=0).stop 2023-11-24T07:56:42.347 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:42.345+0000 7f0fdb565700 1 -- 172.21.15.59:0/1450082574 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0fd4145100 msgr2=0x7f0fd40ad250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:56:42.347 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:42.345+0000 7f0fdb565700 1 --2- 172.21.15.59:0/1450082574 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0fd4145100 0x7f0fd40ad250 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f0fd0000c00 tx=0x7f0fd0004750 comp rx=0 tx=0).stop 2023-11-24T07:56:42.347 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:42.345+0000 7f0fdb565700 1 -- 172.21.15.59:0/1450082574 shutdown_connections 2023-11-24T07:56:42.347 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:42.345+0000 7f0fdb565700 1 --2- 172.21.15.59:0/1450082574 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f0fb806aed0 0x7f0fb806d390 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:56:42.347 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:42.345+0000 7f0fdb565700 1 --2- 172.21.15.59:0/1450082574 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0fd4145650 0x7f0fd40a9450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:56:42.347 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:42.345+0000 7f0fdb565700 1 --2- 172.21.15.59:0/1450082574 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f0fd4144c10 0x7f0fd415ae90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:56:42.347 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:42.345+0000 7f0fdb565700 1 --2- 172.21.15.59:0/1450082574 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0fd4145100 0x7f0fd40ad250 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:56:42.347 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:42.345+0000 7f0fdb565700 1 -- 172.21.15.59:0/1450082574 >> 172.21.15.59:0/1450082574 conn(0x7f0fd40b2740 msgr2=0x7f0fd40b0e40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:56:42.347 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:42.345+0000 7f0fdb565700 1 -- 172.21.15.59:0/1450082574 shutdown_connections 2023-11-24T07:56:42.348 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:42.345+0000 7f0fdb565700 1 -- 172.21.15.59:0/1450082574 wait complete. 2023-11-24T07:56:42.359 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-24T07:56:42.483 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:42.482+0000 7f4c76519700 1 Processor -- start 2023-11-24T07:56:42.484 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:42.482+0000 7f4c76519700 1 -- start start 2023-11-24T07:56:42.486 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:42.483+0000 7f4c76519700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f4c701459a0 0x7f4c700cfa80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:56:42.486 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:42.483+0000 7f4c76519700 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f4c7006b6e0 0x7f4c700cffc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:56:42.486 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:42.483+0000 7f4c76519700 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4c700cf5f0 con 0x7f4c70142810 2023-11-24T07:56:42.486 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:42.483+0000 7f4c76519700 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4c7006ea80 con 0x7f4c701459a0 2023-11-24T07:56:42.486 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:42.483+0000 7f4c76519700 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4c7006ed60 con 0x7f4c7006b6e0 2023-11-24T07:56:42.486 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:42.483+0000 7f4c6f7fe700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f4c701459a0 0x7f4c700cfa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:56:42.486 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:42.483+0000 7f4c6f7fe700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f4c701459a0 0x7f4c700cfa80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:50448/0 (socket says 172.21.15.59:50448) 2023-11-24T07:56:42.486 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:42.483+0000 7f4c6f7fe700 1 -- 172.21.15.59:0/4022009259 learned_addr learned my addr 172.21.15.59:0/4022009259 (peer_addr_for_me v2:172.21.15.59:0/0) 2023-11-24T07:56:42.486 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:42.483+0000 7f4c74ab6700 1 --2- 172.21.15.59:0/4022009259 >> v2:172.21.15.76:3300/0 conn(0x7f4c7006b6e0 0x7f4c700cffc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:56:42.487 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:42.483+0000 7f4c6f7fe700 1 -- 172.21.15.59:0/4022009259 >> v2:172.21.15.76:3300/0 conn(0x7f4c7006b6e0 msgr2=0x7f4c700cffc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T07:56:42.487 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:42.483+0000 7f4c6f7fe700 1 --2- 172.21.15.59:0/4022009259 >> v2:172.21.15.76:3300/0 conn(0x7f4c7006b6e0 0x7f4c700cffc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:56:42.487 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:42.483+0000 7f4c6f7fe700 1 -- 172.21.15.59:0/4022009259 >> v1:172.21.15.76:6789/0 conn(0x7f4c70142810 legacy=0x7f4c7006afd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:56:42.487 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:42.483+0000 7f4c6f7fe700 1 -- 172.21.15.59:0/4022009259 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4c7006f040 con 0x7f4c701459a0 2023-11-24T07:56:42.487 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:42.483+0000 7f4c6f7fe700 1 --2- 172.21.15.59:0/4022009259 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4c701459a0 0x7f4c700cfa80 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f4c700747c0 tx=0x7f4c64009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6dcc4a284701a06e server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:56:42.487 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:42.483+0000 7f4c6effd700 1 -- 172.21.15.59:0/4022009259 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c64011830 con 0x7f4c701459a0 2023-11-24T07:56:42.487 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:42.483+0000 7f4c6effd700 1 -- 172.21.15.59:0/4022009259 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4c64011990 con 0x7f4c701459a0 2023-11-24T07:56:42.487 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:42.483+0000 7f4c6effd700 1 -- 172.21.15.59:0/4022009259 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c64011d20 con 0x7f4c701459a0 2023-11-24T07:56:42.487 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:42.484+0000 7f4c76519700 1 -- 172.21.15.59:0/4022009259 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4c701459a0 msgr2=0x7f4c700cfa80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:56:42.487 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:42.484+0000 7f4c76519700 1 --2- 172.21.15.59:0/4022009259 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4c701459a0 0x7f4c700cfa80 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f4c700747c0 tx=0x7f4c64009bd0 comp rx=0 tx=0).stop 2023-11-24T07:56:42.487 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:42.484+0000 7f4c76519700 1 -- 172.21.15.59:0/4022009259 shutdown_connections 2023-11-24T07:56:42.487 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:42.484+0000 7f4c76519700 1 --2- 172.21.15.59:0/4022009259 >> v2:172.21.15.76:3300/0 conn(0x7f4c7006b6e0 0x7f4c700cffc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:56:42.487 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:42.484+0000 7f4c76519700 1 --2- 172.21.15.59:0/4022009259 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4c701459a0 0x7f4c700cfa80 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:56:42.487 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:42.484+0000 7f4c76519700 1 -- 172.21.15.59:0/4022009259 >> 172.21.15.59:0/4022009259 conn(0x7f4c700b2740 msgr2=0x7f4c700b2b20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:56:42.488 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:42.484+0000 7f4c76519700 1 -- 172.21.15.59:0/4022009259 shutdown_connections 2023-11-24T07:56:42.488 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:42.484+0000 7f4c76519700 1 -- 172.21.15.59:0/4022009259 wait complete. 2023-11-24T07:56:42.490 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:42.485+0000 7f4c76519700 1 Processor -- start 2023-11-24T07:56:42.490 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:42.485+0000 7f4c76519700 1 -- start start 2023-11-24T07:56:42.490 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:42.486+0000 7f4c76519700 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f4c7006b6e0 0x7f4c7013f8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:56:42.490 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:42.486+0000 7f4c76519700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4c70142810 0x7f4c701f6f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:56:42.491 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:42.486+0000 7f4c76519700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4c701459a0 0x7f4c701f74a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:56:42.491 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:42.486+0000 7f4c76519700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c700ad860 con 0x7f4c70142810 2023-11-24T07:56:42.491 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:42.486+0000 7f4c76519700 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c7015e390 con 0x7f4c701459a0 2023-11-24T07:56:42.491 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:42.486+0000 7f4c76519700 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4c7015e670 con 0x7f4c7006b6e0 2023-11-24T07:56:42.491 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:42.486+0000 7f4c6f7fe700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4c70142810 0x7f4c701f6f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:56:42.491 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:42.486+0000 7f4c6f7fe700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4c70142810 0x7f4c701f6f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:50454/0 (socket says 172.21.15.59:50454) 2023-11-24T07:56:42.491 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:42.486+0000 7f4c6f7fe700 1 -- 172.21.15.59:0/1613374145 learned_addr learned my addr 172.21.15.59:0/1613374145 (peer_addr_for_me v2:172.21.15.59:0/0) 2023-11-24T07:56:42.491 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:42.486+0000 7f4c74ab6700 1 --2- 172.21.15.59:0/1613374145 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4c701459a0 0x7f4c701f74a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:56:42.491 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:42.486+0000 7f4c6ffff700 1 --2- 172.21.15.59:0/1613374145 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f4c7006b6e0 0x7f4c7013f8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:56:42.491 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:42.487+0000 7f4c6f7fe700 1 -- 172.21.15.59:0/1613374145 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f4c7006b6e0 msgr2=0x7f4c7013f8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:56:42.491 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:42.487+0000 7f4c6f7fe700 1 --2- 172.21.15.59:0/1613374145 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f4c7006b6e0 0x7f4c7013f8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:56:42.491 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:42.487+0000 7f4c6f7fe700 1 -- 172.21.15.59:0/1613374145 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4c701459a0 msgr2=0x7f4c701f74a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:56:42.491 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:42.487+0000 7f4c6f7fe700 1 --2- 172.21.15.59:0/1613374145 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4c701459a0 0x7f4c701f74a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:56:42.491 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:42.487+0000 7f4c6f7fe700 1 -- 172.21.15.59:0/1613374145 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4c64009550 con 0x7f4c70142810 2023-11-24T07:56:42.492 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:42.487+0000 7f4c74ab6700 1 --2- 172.21.15.59:0/1613374145 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4c701459a0 0x7f4c701f74a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T07:56:42.492 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:42.487+0000 7f4c6f7fe700 1 --2- 172.21.15.59:0/1613374145 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4c70142810 0x7f4c701f6f60 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f4c64009c10 tx=0x7f4c64004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:56:42.492 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:42.487+0000 7f4c6cff9700 1 -- 172.21.15.59:0/1613374145 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c640036a0 con 0x7f4c70142810 2023-11-24T07:56:42.492 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:42.487+0000 7f4c6cff9700 1 -- 172.21.15.59:0/1613374145 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4c6401e070 con 0x7f4c70142810 2023-11-24T07:56:42.492 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:42.487+0000 7f4c76519700 1 -- 172.21.15.59:0/1613374145 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4c7015e950 con 0x7f4c70142810 2023-11-24T07:56:42.492 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:42.487+0000 7f4c6cff9700 1 -- 172.21.15.59:0/1613374145 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c64003a60 con 0x7f4c70142810 2023-11-24T07:56:42.492 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:42.487+0000 7f4c76519700 1 -- 172.21.15.59:0/1613374145 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4c7013fe00 con 0x7f4c70142810 2023-11-24T07:56:42.492 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:42.488+0000 7f4c76519700 1 -- 172.21.15.59:0/1613374145 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4c700ad9d0 con 0x7f4c70142810 2023-11-24T07:56:42.492 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:42.488+0000 7f4c6cff9700 1 -- 172.21.15.59:0/1613374145 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f4c64003bc0 con 0x7f4c70142810 2023-11-24T07:56:42.492 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:42.489+0000 7f4c6cff9700 1 --2- 172.21.15.59:0/1613374145 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f4c5c06aed0 0x7f4c5c06d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:56:42.492 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:42.489+0000 7f4c6cff9700 1 -- 172.21.15.59:0/1613374145 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f4c6408a4a0 con 0x7f4c70142810 2023-11-24T07:56:42.492 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:42.489+0000 7f4c6ffff700 1 --2- 172.21.15.59:0/1613374145 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f4c5c06aed0 0x7f4c5c06d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:56:42.493 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:42.489+0000 7f4c6ffff700 1 --2- 172.21.15.59:0/1613374145 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f4c5c06aed0 0x7f4c5c06d390 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f4c60007fd0 tx=0x7f4c60009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:56:42.495 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:42.493+0000 7f4c6cff9700 1 -- 172.21.15.59:0/1613374145 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f4c700ad9d0 con 0x7f4c70142810 2023-11-24T07:56:42.743 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:42.742+0000 7f4c76519700 1 -- 172.21.15.59:0/1613374145 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f4c700ad9d0 con 0x7f4c70142810 2023-11-24T07:56:42.744 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:42.742+0000 7f4c6cff9700 1 -- 172.21.15.59:0/1613374145 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) v1 ==== 93+0+4904 (secure 0 0 0) 0x7f4c700ad9d0 con 0x7f4c70142810 2023-11-24T07:56:42.744 INFO:teuthology.orchestra.run.smithi059.stdout: 2023-11-24T07:56:42.744 INFO:teuthology.orchestra.run.smithi059.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4220,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6835/838038775","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6833","nonce":838038775},{"type":"v1","addr":"172.21.15.59:6835","nonce":838038775}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4232,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.76:6837/3226690004","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6835","nonce":3226690004},{"type":"v1","addr":"172.21.15.76:6837","nonce":3226690004}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4260,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6834/2986456006","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6832","nonce":2986456006},{"type":"v1","addr":"172.21.15.59:6834","nonce":2986456006}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-24T07:56:24.104295+0000","modified":"2023-11-24T07:56:27.152806+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":4264},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4264":{"gid":4264,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.76:6836/3993077463","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6834","nonce":3993077463},{"type":"v1","addr":"172.21.15.76:6836","nonce":3993077463}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-11-24T07:56:42.746 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:42.744+0000 7f4c76519700 1 -- 172.21.15.59:0/1613374145 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f4c5c06aed0 msgr2=0x7f4c5c06d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:56:42.746 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:42.744+0000 7f4c76519700 1 --2- 172.21.15.59:0/1613374145 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f4c5c06aed0 0x7f4c5c06d390 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f4c60007fd0 tx=0x7f4c60009040 comp rx=0 tx=0).stop 2023-11-24T07:56:42.746 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:42.744+0000 7f4c76519700 1 -- 172.21.15.59:0/1613374145 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4c70142810 msgr2=0x7f4c701f6f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:56:42.746 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:42.744+0000 7f4c76519700 1 --2- 172.21.15.59:0/1613374145 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4c70142810 0x7f4c701f6f60 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f4c64009c10 tx=0x7f4c64004750 comp rx=0 tx=0).stop 2023-11-24T07:56:42.746 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:42.744+0000 7f4c76519700 1 -- 172.21.15.59:0/1613374145 shutdown_connections 2023-11-24T07:56:42.747 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:42.744+0000 7f4c76519700 1 --2- 172.21.15.59:0/1613374145 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f4c5c06aed0 0x7f4c5c06d390 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:56:42.747 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:42.744+0000 7f4c76519700 1 --2- 172.21.15.59:0/1613374145 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4c701459a0 0x7f4c701f74a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:56:42.747 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:42.744+0000 7f4c76519700 1 --2- 172.21.15.59:0/1613374145 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f4c7006b6e0 0x7f4c7013f8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:56:42.747 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:42.744+0000 7f4c76519700 1 --2- 172.21.15.59:0/1613374145 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4c70142810 0x7f4c701f6f60 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:56:42.747 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:42.744+0000 7f4c76519700 1 -- 172.21.15.59:0/1613374145 >> 172.21.15.59:0/1613374145 conn(0x7f4c700b2740 msgr2=0x7f4c70145250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:56:42.747 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:42.744+0000 7f4c76519700 1 -- 172.21.15.59:0/1613374145 shutdown_connections 2023-11-24T07:56:42.747 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:42.744+0000 7f4c76519700 1 -- 172.21.15.59:0/1613374145 wait complete. 2023-11-24T07:56:42.747 INFO:teuthology.orchestra.run.smithi059.stderr:dumped fsmap epoch 8 2023-11-24T07:56:42.758 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-24T07:56:42.882 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:42.880+0000 7fe7c8aba700 1 Processor -- start 2023-11-24T07:56:42.882 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:42.881+0000 7fe7c8aba700 1 -- start start 2023-11-24T07:56:42.885 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:42.881+0000 7fe7c8aba700 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fe7c415bde0 0x7fe7c415c1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:56:42.885 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:42.881+0000 7fe7c8aba700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fe7c40ce5c0 0x7fe7c40cea50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:56:42.885 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:42.881+0000 7fe7c8aba700 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe7c40d0a90 con 0x7fe7c40cf2b0 2023-11-24T07:56:42.885 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:42.881+0000 7fe7c8aba700 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe7c40cef90 con 0x7fe7c40ce5c0 2023-11-24T07:56:42.885 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:42.881+0000 7fe7c8aba700 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe7c406ee10 con 0x7fe7c415bde0 2023-11-24T07:56:42.885 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:42.882+0000 7fe7c2d9d700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fe7c40ce5c0 0x7fe7c40cea50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:56:42.885 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:42.882+0000 7fe7c1d9b700 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7fe7c40cf2b0 0x7fe7c40d01b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:53934/0 (socket says 172.21.15.59:53934) 2023-11-24T07:56:42.885 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:42.882+0000 7fe7c1d9b700 1 -- 172.21.15.59:0/163005798 learned_addr learned my addr 172.21.15.59:0/163005798 (peer_addr_for_me v1:172.21.15.59:0/0) 2023-11-24T07:56:42.885 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:42.882+0000 7fe7c259c700 1 --2- 172.21.15.59:0/163005798 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fe7c415bde0 0x7fe7c415c1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:56:42.885 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:42.882+0000 7fe7c2d9d700 1 -- 172.21.15.59:0/163005798 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fe7c415bde0 msgr2=0x7fe7c415c1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T07:56:42.886 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:42.882+0000 7fe7c2d9d700 1 --2- 172.21.15.59:0/163005798 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fe7c415bde0 0x7fe7c415c1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:56:42.886 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:42.882+0000 7fe7c2d9d700 1 -- 172.21.15.59:0/163005798 >> v1:172.21.15.59:6789/0 conn(0x7fe7c40cf2b0 legacy=0x7fe7c40d01b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:56:42.886 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:42.882+0000 7fe7c2d9d700 1 -- 172.21.15.59:0/163005798 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe7c406f0f0 con 0x7fe7c40ce5c0 2023-11-24T07:56:42.886 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:42.882+0000 7fe7c159a700 1 -- 172.21.15.59:0/163005798 <== mon.0 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 518053886 0 0) 0x7fe7c40d0a90 con 0x7fe7c40cf2b0 2023-11-24T07:56:42.886 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:42.882+0000 7fe7c2d9d700 1 --2- 172.21.15.59:0/163005798 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe7c40ce5c0 0x7fe7c40cea50 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fe7bc009d90 tx=0x7fe7bc00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=df9c56ba2395f745 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:56:42.886 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:42.882+0000 7fe7c159a700 1 -- 172.21.15.59:0/163005798 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7bc00e070 con 0x7fe7c40ce5c0 2023-11-24T07:56:42.886 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:42.882+0000 7fe7c159a700 1 -- 172.21.15.59:0/163005798 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe7bc00bb20 con 0x7fe7c40ce5c0 2023-11-24T07:56:42.887 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:42.882+0000 7fe7c159a700 1 -- 172.21.15.59:0/163005798 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7bc012430 con 0x7fe7c40ce5c0 2023-11-24T07:56:42.887 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:42.883+0000 7fe7c8aba700 1 -- 172.21.15.59:0/163005798 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe7c40ce5c0 msgr2=0x7fe7c40cea50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:56:42.887 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:42.883+0000 7fe7c8aba700 1 --2- 172.21.15.59:0/163005798 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe7c40ce5c0 0x7fe7c40cea50 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fe7bc009d90 tx=0x7fe7bc00b330 comp rx=0 tx=0).stop 2023-11-24T07:56:42.887 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:42.883+0000 7fe7c8aba700 1 -- 172.21.15.59:0/163005798 shutdown_connections 2023-11-24T07:56:42.887 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:42.883+0000 7fe7c8aba700 1 --2- 172.21.15.59:0/163005798 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe7c40ce5c0 0x7fe7c40cea50 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:56:42.887 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:42.883+0000 7fe7c8aba700 1 --2- 172.21.15.59:0/163005798 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fe7c415bde0 0x7fe7c415c1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:56:42.887 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:42.883+0000 7fe7c8aba700 1 -- 172.21.15.59:0/163005798 >> 172.21.15.59:0/163005798 conn(0x7fe7c40b3720 msgr2=0x7fe7c40b3b30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:56:42.887 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:42.883+0000 7fe7c8aba700 1 -- 172.21.15.59:0/163005798 shutdown_connections 2023-11-24T07:56:42.888 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:42.883+0000 7fe7c8aba700 1 -- 172.21.15.59:0/163005798 wait complete. 2023-11-24T07:56:42.890 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:42.884+0000 7fe7c8aba700 1 Processor -- start 2023-11-24T07:56:42.890 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:42.884+0000 7fe7c8aba700 1 -- start start 2023-11-24T07:56:42.890 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:42.885+0000 7fe7c8aba700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe7c40ce5c0 0x7fe7c415a510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:56:42.890 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:42.885+0000 7fe7c8aba700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe7c40cf2b0 0x7fe7c406caf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:56:42.891 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:42.885+0000 7fe7c8aba700 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fe7c415bde0 0x7fe7c40c7700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:56:42.891 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:42.885+0000 7fe7c8aba700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7c415aa50 con 0x7fe7c40ce5c0 2023-11-24T07:56:42.891 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:42.885+0000 7fe7c8aba700 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7c40cb570 con 0x7fe7c40cf2b0 2023-11-24T07:56:42.891 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:42.885+0000 7fe7c8aba700 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe7c406da30 con 0x7fe7c415bde0 2023-11-24T07:56:42.891 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:42.885+0000 7fe7c259c700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe7c40ce5c0 0x7fe7c415a510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:56:42.891 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:42.885+0000 7fe7c259c700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe7c40ce5c0 0x7fe7c415a510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:50464/0 (socket says 172.21.15.59:50464) 2023-11-24T07:56:42.891 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:42.885+0000 7fe7c259c700 1 -- 172.21.15.59:0/2112730005 learned_addr learned my addr 172.21.15.59:0/2112730005 (peer_addr_for_me v2:172.21.15.59:0/0) 2023-11-24T07:56:42.891 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:42.885+0000 7fe7c2d9d700 1 --2- 172.21.15.59:0/2112730005 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fe7c415bde0 0x7fe7c40c7700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:56:42.892 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:42.885+0000 7fe7c1d9b700 1 --2- 172.21.15.59:0/2112730005 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe7c40cf2b0 0x7fe7c406caf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:56:42.892 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:42.885+0000 7fe7c259c700 1 -- 172.21.15.59:0/2112730005 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fe7c415bde0 msgr2=0x7fe7c40c7700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:56:42.892 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:42.885+0000 7fe7c259c700 1 --2- 172.21.15.59:0/2112730005 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fe7c415bde0 0x7fe7c40c7700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:56:42.892 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:42.885+0000 7fe7c259c700 1 -- 172.21.15.59:0/2112730005 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe7c40cf2b0 msgr2=0x7fe7c406caf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:56:42.892 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:42.885+0000 7fe7c259c700 1 --2- 172.21.15.59:0/2112730005 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe7c40cf2b0 0x7fe7c406caf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:56:42.892 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:42.885+0000 7fe7c259c700 1 -- 172.21.15.59:0/2112730005 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe7bc009a70 con 0x7fe7c40ce5c0 2023-11-24T07:56:42.892 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:42.886+0000 7fe7c259c700 1 --2- 172.21.15.59:0/2112730005 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe7c40ce5c0 0x7fe7c415a510 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fe7b400e9b0 tx=0x7fe7b400ed70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:56:42.893 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:42.886+0000 7fe7baffd700 1 -- 172.21.15.59:0/2112730005 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7b4009710 con 0x7fe7c40ce5c0 2023-11-24T07:56:42.893 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:42.886+0000 7fe7baffd700 1 -- 172.21.15.59:0/2112730005 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe7b400c850 con 0x7fe7c40ce5c0 2023-11-24T07:56:42.893 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:42.886+0000 7fe7c8aba700 1 -- 172.21.15.59:0/2112730005 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe7c40ca960 con 0x7fe7c40ce5c0 2023-11-24T07:56:42.893 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:42.886+0000 7fe7baffd700 1 -- 172.21.15.59:0/2112730005 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7b400cbc0 con 0x7fe7c40ce5c0 2023-11-24T07:56:42.893 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:42.886+0000 7fe7c8aba700 1 -- 172.21.15.59:0/2112730005 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe7c40caf20 con 0x7fe7c40ce5c0 2023-11-24T07:56:42.893 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:42.887+0000 7fe7c8aba700 1 -- 172.21.15.59:0/2112730005 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe7c415d020 con 0x7fe7c40ce5c0 2023-11-24T07:56:42.893 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:42.887+0000 7fe7baffd700 1 -- 172.21.15.59:0/2112730005 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fe7b400cd20 con 0x7fe7c40ce5c0 2023-11-24T07:56:42.893 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:42.888+0000 7fe7baffd700 1 --2- 172.21.15.59:0/2112730005 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7fe7a406aed0 0x7fe7a406d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:56:42.893 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:42.888+0000 7fe7baffd700 1 -- 172.21.15.59:0/2112730005 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fe7b4089730 con 0x7fe7c40ce5c0 2023-11-24T07:56:42.893 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:42.888+0000 7fe7c1d9b700 1 --2- 172.21.15.59:0/2112730005 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7fe7a406aed0 0x7fe7a406d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:56:42.893 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:42.888+0000 7fe7c1d9b700 1 --2- 172.21.15.59:0/2112730005 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7fe7a406aed0 0x7fe7a406d390 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fe7c406e2c0 tx=0x7fe7ac007c00 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:56:42.894 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:42.892+0000 7fe7baffd700 1 -- 172.21.15.59:0/2112730005 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fe7b40547e0 con 0x7fe7c40ce5c0 2023-11-24T07:56:43.095 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:43.093+0000 7fe7c8aba700 1 -- 172.21.15.59:0/2112730005 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fe7c4068670 con 0x7fe7c40ce5c0 2023-11-24T07:56:43.096 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:43.095+0000 7fe7baffd700 1 -- 172.21.15.59:0/2112730005 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) v1 ==== 74+0+16437 (secure 0 0 0) 0x7fe7b4057e00 con 0x7fe7c40ce5c0 2023-11-24T07:56:43.096 INFO:teuthology.orchestra.run.smithi059.stdout: 2023-11-24T07:56:43.097 INFO:teuthology.orchestra.run.smithi059.stdout:{"epoch":20,"fsid":"9f0e8741-b8f9-48d5-a2c3-09f45903303d","created":"2023-11-24T07:56:06.552729+0000","modified":"2023-11-24T07:56:25.705091+0000","last_up_change":"2023-11-24T07:56:17.372352+0000","last_in_change":"2023-11-24T07:56:09.073250+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-24T07:56:18.352823+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-11-24T07:56:22.185140+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-11-24T07:56:23.084326+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"6d8f9e9b-1782-4e41-928b-a39f2a8ebc06","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6800","nonce":3373878772},{"type":"v1","addr":"172.21.15.59:6801","nonce":3373878772}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6802","nonce":3373878772},{"type":"v1","addr":"172.21.15.59:6803","nonce":3373878772}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6806","nonce":3373878772},{"type":"v1","addr":"172.21.15.59:6807","nonce":3373878772}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6804","nonce":3373878772},{"type":"v1","addr":"172.21.15.59:6805","nonce":3373878772}]},"public_addr":"172.21.15.59:6801/3373878772","cluster_addr":"172.21.15.59:6803/3373878772","heartbeat_back_addr":"172.21.15.59:6807/3373878772","heartbeat_front_addr":"172.21.15.59:6805/3373878772","state":["exists","up"]},{"osd":1,"uuid":"e8d882aa-52e1-4c04-9e1e-c44ef9204a5f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6824","nonce":648258563},{"type":"v1","addr":"172.21.15.59:6825","nonce":648258563}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6826","nonce":648258563},{"type":"v1","addr":"172.21.15.59:6827","nonce":648258563}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6830","nonce":648258563},{"type":"v1","addr":"172.21.15.59:6831","nonce":648258563}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6828","nonce":648258563},{"type":"v1","addr":"172.21.15.59:6829","nonce":648258563}]},"public_addr":"172.21.15.59:6825/648258563","cluster_addr":"172.21.15.59:6827/648258563","heartbeat_back_addr":"172.21.15.59:6831/648258563","heartbeat_front_addr":"172.21.15.59:6829/648258563","state":["exists","up"]},{"osd":2,"uuid":"69eebfa0-695b-4b21-90b9-8a3537168a01","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6808","nonce":1840430654},{"type":"v1","addr":"172.21.15.59:6809","nonce":1840430654}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6810","nonce":1840430654},{"type":"v1","addr":"172.21.15.59:6811","nonce":1840430654}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6814","nonce":1840430654},{"type":"v1","addr":"172.21.15.59:6815","nonce":1840430654}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6812","nonce":1840430654},{"type":"v1","addr":"172.21.15.59:6813","nonce":1840430654}]},"public_addr":"172.21.15.59:6809/1840430654","cluster_addr":"172.21.15.59:6811/1840430654","heartbeat_back_addr":"172.21.15.59:6815/1840430654","heartbeat_front_addr":"172.21.15.59:6813/1840430654","state":["exists","up"]},{"osd":3,"uuid":"96fd77fc-f911-46ff-9445-6f0be2daa8f4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6816","nonce":2504025850},{"type":"v1","addr":"172.21.15.59:6817","nonce":2504025850}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6818","nonce":2504025850},{"type":"v1","addr":"172.21.15.59:6819","nonce":2504025850}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6822","nonce":2504025850},{"type":"v1","addr":"172.21.15.59:6823","nonce":2504025850}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6820","nonce":2504025850},{"type":"v1","addr":"172.21.15.59:6821","nonce":2504025850}]},"public_addr":"172.21.15.59:6817/2504025850","cluster_addr":"172.21.15.59:6819/2504025850","heartbeat_back_addr":"172.21.15.59:6823/2504025850","heartbeat_front_addr":"172.21.15.59:6821/2504025850","state":["exists","up"]},{"osd":4,"uuid":"7ac29fca-b5b2-4c95-9d69-2f164cbbc708","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6816","nonce":2382463431},{"type":"v1","addr":"172.21.15.76:6818","nonce":2382463431}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6820","nonce":2382463431},{"type":"v1","addr":"172.21.15.76:6822","nonce":2382463431}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6828","nonce":2382463431},{"type":"v1","addr":"172.21.15.76:6830","nonce":2382463431}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6824","nonce":2382463431},{"type":"v1","addr":"172.21.15.76:6826","nonce":2382463431}]},"public_addr":"172.21.15.76:6818/2382463431","cluster_addr":"172.21.15.76:6822/2382463431","heartbeat_back_addr":"172.21.15.76:6830/2382463431","heartbeat_front_addr":"172.21.15.76:6826/2382463431","state":["exists","up"]},{"osd":5,"uuid":"485bf711-3851-4238-a4f4-51d52e0a9c58","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6800","nonce":329334308},{"type":"v1","addr":"172.21.15.76:6802","nonce":329334308}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6805","nonce":329334308},{"type":"v1","addr":"172.21.15.76:6807","nonce":329334308}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6814","nonce":329334308},{"type":"v1","addr":"172.21.15.76:6815","nonce":329334308}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6810","nonce":329334308},{"type":"v1","addr":"172.21.15.76:6812","nonce":329334308}]},"public_addr":"172.21.15.76:6802/329334308","cluster_addr":"172.21.15.76:6807/329334308","heartbeat_back_addr":"172.21.15.76:6815/329334308","heartbeat_front_addr":"172.21.15.76:6812/329334308","state":["exists","up"]},{"osd":6,"uuid":"72b3f2fc-1e4a-4e32-968d-003647a0286f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6801","nonce":3946579410},{"type":"v1","addr":"172.21.15.76:6803","nonce":3946579410}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6804","nonce":3946579410},{"type":"v1","addr":"172.21.15.76:6806","nonce":3946579410}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6811","nonce":3946579410},{"type":"v1","addr":"172.21.15.76:6813","nonce":3946579410}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6808","nonce":3946579410},{"type":"v1","addr":"172.21.15.76:6809","nonce":3946579410}]},"public_addr":"172.21.15.76:6803/3946579410","cluster_addr":"172.21.15.76:6806/3946579410","heartbeat_back_addr":"172.21.15.76:6813/3946579410","heartbeat_front_addr":"172.21.15.76:6809/3946579410","state":["exists","up"]},{"osd":7,"uuid":"1ddceb90-6c5d-4d1a-ad56-99cb877f8c70","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6817","nonce":841761538},{"type":"v1","addr":"172.21.15.76:6819","nonce":841761538}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6821","nonce":841761538},{"type":"v1","addr":"172.21.15.76:6823","nonce":841761538}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6829","nonce":841761538},{"type":"v1","addr":"172.21.15.76:6831","nonce":841761538}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6825","nonce":841761538},{"type":"v1","addr":"172.21.15.76:6827","nonce":841761538}]},"public_addr":"172.21.15.76:6819/841761538","cluster_addr":"172.21.15.76:6823/841761538","heartbeat_back_addr":"172.21.15.76:6831/841761538","heartbeat_front_addr":"172.21.15.76:6827/841761538","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T07:56:14.294838+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T07:56:14.270779+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T07:56:14.231809+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T07:56:14.260983+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T07:56:14.302997+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T07:56:14.258600+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T07:56:14.251413+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T07:56:14.304727+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-24T07:56:43.099 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:43.097+0000 7fe7c8aba700 1 -- 172.21.15.59:0/2112730005 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7fe7a406aed0 msgr2=0x7fe7a406d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:56:43.099 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:43.097+0000 7fe7c8aba700 1 --2- 172.21.15.59:0/2112730005 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7fe7a406aed0 0x7fe7a406d390 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fe7c406e2c0 tx=0x7fe7ac007c00 comp rx=0 tx=0).stop 2023-11-24T07:56:43.099 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:43.097+0000 7fe7c8aba700 1 -- 172.21.15.59:0/2112730005 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe7c40ce5c0 msgr2=0x7fe7c415a510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:56:43.099 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:43.097+0000 7fe7c8aba700 1 --2- 172.21.15.59:0/2112730005 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe7c40ce5c0 0x7fe7c415a510 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fe7b400e9b0 tx=0x7fe7b400ed70 comp rx=0 tx=0).stop 2023-11-24T07:56:43.099 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:43.097+0000 7fe7c8aba700 1 -- 172.21.15.59:0/2112730005 shutdown_connections 2023-11-24T07:56:43.099 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:43.097+0000 7fe7c8aba700 1 --2- 172.21.15.59:0/2112730005 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7fe7a406aed0 0x7fe7a406d390 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:56:43.099 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:43.097+0000 7fe7c8aba700 1 --2- 172.21.15.59:0/2112730005 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fe7c415bde0 0x7fe7c40c7700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:56:43.099 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:43.097+0000 7fe7c8aba700 1 --2- 172.21.15.59:0/2112730005 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe7c40ce5c0 0x7fe7c415a510 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:56:43.099 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:43.097+0000 7fe7c8aba700 1 --2- 172.21.15.59:0/2112730005 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe7c40cf2b0 0x7fe7c406caf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:56:43.099 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:43.097+0000 7fe7c8aba700 1 -- 172.21.15.59:0/2112730005 >> 172.21.15.59:0/2112730005 conn(0x7fe7c40b3720 msgr2=0x7fe7c40aed00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:56:43.099 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:43.097+0000 7fe7c8aba700 1 -- 172.21.15.59:0/2112730005 shutdown_connections 2023-11-24T07:56:43.100 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:43.097+0000 7fe7c8aba700 1 -- 172.21.15.59:0/2112730005 wait complete. 2023-11-24T07:56:43.111 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-24T07:56:43.235 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:43.234+0000 7f5ffbc84700 1 Processor -- start 2023-11-24T07:56:43.235 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:43.234+0000 7f5ffbc84700 1 -- start start 2023-11-24T07:56:43.238 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:43.235+0000 7f5ffbc84700 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f5ff406b620 0x7f5ff40b3a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:56:43.238 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:43.235+0000 7f5ffbc84700 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f5ff406ee60 0x7f5ff40b3f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:56:43.238 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:43.235+0000 7f5ffbc84700 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5ff4071f50 con 0x7f5ff40b5ab0 2023-11-24T07:56:43.238 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:43.235+0000 7f5ffbc84700 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5ff4072230 con 0x7f5ff406b620 2023-11-24T07:56:43.238 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:43.235+0000 7f5ffbc84700 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5ff4072510 con 0x7f5ff406ee60 2023-11-24T07:56:43.238 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:43.235+0000 7f5ff921f700 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7f5ff40b5ab0 0x7f5ff4161ba0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:53946/0 (socket says 172.21.15.59:53946) 2023-11-24T07:56:43.238 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:43.235+0000 7f5ff921f700 1 -- 172.21.15.59:0/2838923755 learned_addr learned my addr 172.21.15.59:0/2838923755 (peer_addr_for_me v1:172.21.15.59:0/0) 2023-11-24T07:56:43.239 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:43.235+0000 7f5ff9a20700 1 --2- 172.21.15.59:0/2838923755 >> v2:172.21.15.76:3300/0 conn(0x7f5ff406b620 0x7f5ff40b3a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:56:43.239 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:43.235+0000 7f5ffa221700 1 --2- 172.21.15.59:0/2838923755 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f5ff406ee60 0x7f5ff40b3f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:56:43.239 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:43.235+0000 7f5ff8a1e700 1 -- 172.21.15.59:0/2838923755 <== mon.0 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 206098144 0 0) 0x7f5ff4071f50 con 0x7f5ff40b5ab0 2023-11-24T07:56:43.239 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:43.235+0000 7f5ff8a1e700 1 -- 172.21.15.59:0/2838923755 --> v1:172.21.15.59:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5fe00035b0 con 0x7f5ff40b5ab0 2023-11-24T07:56:43.239 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:43.235+0000 7f5ff9a20700 1 -- 172.21.15.59:0/2838923755 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f5ff406ee60 msgr2=0x7f5ff40b3f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T07:56:43.239 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:43.235+0000 7f5ff9a20700 1 --2- 172.21.15.59:0/2838923755 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f5ff406ee60 0x7f5ff40b3f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:56:43.239 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:43.235+0000 7f5ff9a20700 1 -- 172.21.15.59:0/2838923755 >> v1:172.21.15.59:6789/0 conn(0x7f5ff40b5ab0 legacy=0x7f5ff4161ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:56:43.239 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:43.235+0000 7f5ff9a20700 1 -- 172.21.15.59:0/2838923755 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5ff40727f0 con 0x7f5ff406b620 2023-11-24T07:56:43.239 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:43.236+0000 7f5ff9a20700 1 --2- 172.21.15.59:0/2838923755 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5ff406b620 0x7f5ff40b3a40 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f5ff0009840 tx=0x7f5ff0009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=81328e2ab75f27da server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:56:43.239 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:43.236+0000 7f5ff8a1e700 1 -- 172.21.15.59:0/2838923755 <== mon.1 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ff0011830 con 0x7f5ff406b620 2023-11-24T07:56:43.239 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:43.236+0000 7f5ff8a1e700 1 -- 172.21.15.59:0/2838923755 <== mon.1 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5ff0011990 con 0x7f5ff406b620 2023-11-24T07:56:43.239 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:43.236+0000 7f5ff8a1e700 1 -- 172.21.15.59:0/2838923755 <== mon.1 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ff0011d20 con 0x7f5ff406b620 2023-11-24T07:56:43.239 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:43.236+0000 7f5ffbc84700 1 -- 172.21.15.59:0/2838923755 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5ff406b620 msgr2=0x7f5ff40b3a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:56:43.240 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:43.236+0000 7f5ffbc84700 1 --2- 172.21.15.59:0/2838923755 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5ff406b620 0x7f5ff40b3a40 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f5ff0009840 tx=0x7f5ff0009b50 comp rx=0 tx=0).stop 2023-11-24T07:56:43.240 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:43.236+0000 7f5ffbc84700 1 -- 172.21.15.59:0/2838923755 shutdown_connections 2023-11-24T07:56:43.240 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:43.236+0000 7f5ffbc84700 1 --2- 172.21.15.59:0/2838923755 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f5ff406ee60 0x7f5ff40b3f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:56:43.240 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:43.236+0000 7f5ffbc84700 1 --2- 172.21.15.59:0/2838923755 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5ff406b620 0x7f5ff40b3a40 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:56:43.240 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:43.236+0000 7f5ffbc84700 1 -- 172.21.15.59:0/2838923755 >> 172.21.15.59:0/2838923755 conn(0x7f5ff40b2740 msgr2=0x7f5ff40b2b20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:56:43.240 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:43.236+0000 7f5ffbc84700 1 -- 172.21.15.59:0/2838923755 shutdown_connections 2023-11-24T07:56:43.240 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:43.236+0000 7f5ffbc84700 1 -- 172.21.15.59:0/2838923755 wait complete. 2023-11-24T07:56:43.243 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:43.237+0000 7f5ffbc84700 1 Processor -- start 2023-11-24T07:56:43.243 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:43.238+0000 7f5ffbc84700 1 -- start start 2023-11-24T07:56:43.243 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:43.238+0000 7f5ffbc84700 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f5ff406b620 0x7f5ff41d6950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:56:43.243 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:43.238+0000 7f5ffbc84700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5ff406ee60 0x7f5ff41d8040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:56:43.243 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:43.238+0000 7f5ffbc84700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5ff40b5ab0 0x7f5ff41d5de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:56:43.243 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:43.238+0000 7f5ffbc84700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ff41d6fa0 con 0x7f5ff406ee60 2023-11-24T07:56:43.243 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:43.238+0000 7f5ffbc84700 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ff41d7280 con 0x7f5ff40b5ab0 2023-11-24T07:56:43.244 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:43.238+0000 7f5ffbc84700 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5ff41d7560 con 0x7f5ff406b620 2023-11-24T07:56:43.244 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:43.238+0000 7f5ff921f700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5ff406ee60 0x7f5ff41d8040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:56:43.244 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:43.238+0000 7f5ff921f700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5ff406ee60 0x7f5ff41d8040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:50480/0 (socket says 172.21.15.59:50480) 2023-11-24T07:56:43.244 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:43.238+0000 7f5ff921f700 1 -- 172.21.15.59:0/11669562 learned_addr learned my addr 172.21.15.59:0/11669562 (peer_addr_for_me v2:172.21.15.59:0/0) 2023-11-24T07:56:43.244 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:43.238+0000 7f5ff9a20700 1 --2- 172.21.15.59:0/11669562 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f5ff406b620 0x7f5ff41d6950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:56:43.244 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:43.239+0000 7f5ffa221700 1 --2- 172.21.15.59:0/11669562 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5ff40b5ab0 0x7f5ff41d5de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:56:43.244 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:43.239+0000 7f5ff921f700 1 -- 172.21.15.59:0/11669562 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f5ff406b620 msgr2=0x7f5ff41d6950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:56:43.244 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:43.239+0000 7f5ff921f700 1 --2- 172.21.15.59:0/11669562 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f5ff406b620 0x7f5ff41d6950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:56:43.245 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:43.239+0000 7f5ff921f700 1 -- 172.21.15.59:0/11669562 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5ff40b5ab0 msgr2=0x7f5ff41d5de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:56:43.245 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:43.239+0000 7f5ff921f700 1 --2- 172.21.15.59:0/11669562 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5ff40b5ab0 0x7f5ff41d5de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:56:43.245 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:43.239+0000 7f5ff921f700 1 -- 172.21.15.59:0/11669562 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5ff00094a0 con 0x7f5ff406ee60 2023-11-24T07:56:43.245 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:43.239+0000 7f5ffa221700 1 --2- 172.21.15.59:0/11669562 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5ff40b5ab0 0x7f5ff41d5de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-24T07:56:43.245 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:43.239+0000 7f5ff921f700 1 --2- 172.21.15.59:0/11669562 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5ff406ee60 0x7f5ff41d8040 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f5fe4005910 tx=0x7f5fe400c7b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:56:43.245 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:43.239+0000 7f5fea7fc700 1 -- 172.21.15.59:0/11669562 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5fe4005d70 con 0x7f5ff406ee60 2023-11-24T07:56:43.245 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:43.239+0000 7f5fea7fc700 1 -- 172.21.15.59:0/11669562 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5fe40070d0 con 0x7f5ff406ee60 2023-11-24T07:56:43.245 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:43.239+0000 7f5ffbc84700 1 -- 172.21.15.59:0/11669562 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5ff41d7840 con 0x7f5ff406ee60 2023-11-24T07:56:43.245 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:43.239+0000 7f5fea7fc700 1 -- 172.21.15.59:0/11669562 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5fe4007460 con 0x7f5ff406ee60 2023-11-24T07:56:43.245 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:43.239+0000 7f5ffbc84700 1 -- 172.21.15.59:0/11669562 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5ff41d8580 con 0x7f5ff406ee60 2023-11-24T07:56:43.245 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:43.240+0000 7f5ffbc84700 1 -- 172.21.15.59:0/11669562 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5ff4068670 con 0x7f5ff406ee60 2023-11-24T07:56:43.245 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:43.241+0000 7f5fea7fc700 1 -- 172.21.15.59:0/11669562 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f5fe40075c0 con 0x7f5ff406ee60 2023-11-24T07:56:43.246 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:43.241+0000 7f5fea7fc700 1 --2- 172.21.15.59:0/11669562 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f5fd806aed0 0x7f5fd806d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:56:43.246 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:43.241+0000 7f5fea7fc700 1 -- 172.21.15.59:0/11669562 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f5fe4088a20 con 0x7f5ff406ee60 2023-11-24T07:56:43.246 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:43.241+0000 7f5ff9a20700 1 --2- 172.21.15.59:0/11669562 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f5fd806aed0 0x7f5fd806d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:56:43.246 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:43.242+0000 7f5ff9a20700 1 --2- 172.21.15.59:0/11669562 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f5fd806aed0 0x7f5fd806d390 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f5ff0000c00 tx=0x7f5ff0008030 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:56:43.247 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:43.245+0000 7f5fea7fc700 1 -- 172.21.15.59:0/11669562 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f5fe4053ae0 con 0x7f5ff406ee60 2023-11-24T07:56:43.498 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:43.496+0000 7f5ffbc84700 1 -- 172.21.15.59:0/11669562 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f5ff40ae2e0 con 0x7f5ff406ee60 2023-11-24T07:56:43.498 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:43.496+0000 7f5fea7fc700 1 -- 172.21.15.59:0/11669562 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) v1 ==== 93+0+4904 (secure 0 0 0) 0x7f5fe4057100 con 0x7f5ff406ee60 2023-11-24T07:56:43.498 INFO:teuthology.orchestra.run.smithi059.stdout: 2023-11-24T07:56:43.498 INFO:teuthology.orchestra.run.smithi059.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4220,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6835/838038775","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6833","nonce":838038775},{"type":"v1","addr":"172.21.15.59:6835","nonce":838038775}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4232,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.76:6837/3226690004","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6835","nonce":3226690004},{"type":"v1","addr":"172.21.15.76:6837","nonce":3226690004}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4260,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6834/2986456006","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6832","nonce":2986456006},{"type":"v1","addr":"172.21.15.59:6834","nonce":2986456006}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-24T07:56:24.104295+0000","modified":"2023-11-24T07:56:27.152806+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":4264},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4264":{"gid":4264,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.76:6836/3993077463","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6834","nonce":3993077463},{"type":"v1","addr":"172.21.15.76:6836","nonce":3993077463}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-11-24T07:56:43.500 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:43.498+0000 7f5ffbc84700 1 -- 172.21.15.59:0/11669562 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f5fd806aed0 msgr2=0x7f5fd806d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:56:43.501 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:43.498+0000 7f5ffbc84700 1 --2- 172.21.15.59:0/11669562 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f5fd806aed0 0x7f5fd806d390 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f5ff0000c00 tx=0x7f5ff0008030 comp rx=0 tx=0).stop 2023-11-24T07:56:43.501 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:43.498+0000 7f5ffbc84700 1 -- 172.21.15.59:0/11669562 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5ff406ee60 msgr2=0x7f5ff41d8040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:56:43.501 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:43.498+0000 7f5ffbc84700 1 --2- 172.21.15.59:0/11669562 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5ff406ee60 0x7f5ff41d8040 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f5fe4005910 tx=0x7f5fe400c7b0 comp rx=0 tx=0).stop 2023-11-24T07:56:43.501 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:43.498+0000 7f5ffbc84700 1 -- 172.21.15.59:0/11669562 shutdown_connections 2023-11-24T07:56:43.501 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:43.498+0000 7f5ffbc84700 1 --2- 172.21.15.59:0/11669562 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f5fd806aed0 0x7f5fd806d390 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:56:43.501 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:43.498+0000 7f5ffbc84700 1 --2- 172.21.15.59:0/11669562 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5ff40b5ab0 0x7f5ff41d5de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:56:43.501 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:43.498+0000 7f5ffbc84700 1 --2- 172.21.15.59:0/11669562 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f5ff406b620 0x7f5ff41d6950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:56:43.501 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:43.498+0000 7f5ffbc84700 1 --2- 172.21.15.59:0/11669562 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5ff406ee60 0x7f5ff41d8040 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:56:43.502 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:43.498+0000 7f5ffbc84700 1 -- 172.21.15.59:0/11669562 >> 172.21.15.59:0/11669562 conn(0x7f5ff40b2740 msgr2=0x7f5ff406f320 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:56:43.502 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:43.499+0000 7f5ffbc84700 1 -- 172.21.15.59:0/11669562 shutdown_connections 2023-11-24T07:56:43.502 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:43.499+0000 7f5ffbc84700 1 -- 172.21.15.59:0/11669562 wait complete. 2023-11-24T07:56:43.502 INFO:teuthology.orchestra.run.smithi059.stderr:dumped fsmap epoch 8 2023-11-24T07:56:43.916 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 8, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-11-24T07:56:24.104295+0000', 'modified': '2023-11-24T07:56:27.152806+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 4264}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_4264': {'gid': 4264, 'name': 'b', 'rank': 0, 'incarnation': 4, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.76:6836/3993077463', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.76:6834', 'nonce': 3993077463}, {'type': 'v1', 'addr': '172.21.15.76:6836', 'nonce': 3993077463}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [3], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2023-11-24T07:56:43.916 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2023-11-24T07:56:43.916 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2023-11-24T07:56:44.054 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:44.052+0000 7ff086a80700 1 Processor -- start 2023-11-24T07:56:44.054 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:44.052+0000 7ff086a80700 1 -- start start 2023-11-24T07:56:44.056 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:44.053+0000 7ff086a80700 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7ff0801448b0 0x7ff08006ec70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:56:44.057 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:44.053+0000 7ff086a80700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7ff08006f990 0x7ff0800ce2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:56:44.057 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:44.053+0000 7ff086a80700 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff080144c90 con 0x7ff0800b44c0 2023-11-24T07:56:44.057 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:44.053+0000 7ff086a80700 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff0800b4c80 con 0x7ff08006f990 2023-11-24T07:56:44.057 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:44.053+0000 7ff086a80700 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff0800b4f60 con 0x7ff0801448b0 2023-11-24T07:56:44.057 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:44.053+0000 7ff08501d700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7ff08006f990 0x7ff0800ce2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:56:44.057 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:44.053+0000 7ff07ffff700 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7ff0800b44c0 0x7ff08006f280 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:53960/0 (socket says 172.21.15.59:53960) 2023-11-24T07:56:44.057 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:44.053+0000 7ff07ffff700 1 -- 172.21.15.59:0/3897250198 learned_addr learned my addr 172.21.15.59:0/3897250198 (peer_addr_for_me v1:172.21.15.59:0/0) 2023-11-24T07:56:44.057 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:44.053+0000 7ff08481c700 1 --2- 172.21.15.59:0/3897250198 >> v2:172.21.15.76:3300/0 conn(0x7ff0801448b0 0x7ff08006ec70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:56:44.057 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:44.053+0000 7ff08501d700 1 -- 172.21.15.59:0/3897250198 >> v2:172.21.15.76:3300/0 conn(0x7ff0801448b0 msgr2=0x7ff08006ec70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T07:56:44.057 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:44.053+0000 7ff08501d700 1 --2- 172.21.15.59:0/3897250198 >> v2:172.21.15.76:3300/0 conn(0x7ff0801448b0 0x7ff08006ec70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:56:44.057 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:44.053+0000 7ff08501d700 1 -- 172.21.15.59:0/3897250198 >> v1:172.21.15.59:6789/0 conn(0x7ff0800b44c0 legacy=0x7ff08006f280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:56:44.057 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:44.053+0000 7ff08501d700 1 -- 172.21.15.59:0/3897250198 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff0800b5240 con 0x7ff08006f990 2023-11-24T07:56:44.058 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:44.053+0000 7ff07f7fe700 1 -- 172.21.15.59:0/3897250198 <== mon.0 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 794957130 0 0) 0x7ff080144c90 con 0x7ff0800b44c0 2023-11-24T07:56:44.058 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:44.053+0000 7ff08481c700 1 --2- 172.21.15.59:0/3897250198 >> v2:172.21.15.76:3300/0 conn(0x7ff0801448b0 0x7ff08006ec70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-24T07:56:44.058 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:44.053+0000 7ff08501d700 1 --2- 172.21.15.59:0/3897250198 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff08006f990 0x7ff0800ce2a0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7ff08006d4b0 tx=0x7ff07800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a3b6dcf1e5ee276d server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:56:44.058 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:44.054+0000 7ff07f7fe700 1 -- 172.21.15.59:0/3897250198 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff07800e070 con 0x7ff08006f990 2023-11-24T07:56:44.058 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:44.054+0000 7ff07f7fe700 1 -- 172.21.15.59:0/3897250198 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff07800bbd0 con 0x7ff08006f990 2023-11-24T07:56:44.058 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:44.054+0000 7ff07f7fe700 1 -- 172.21.15.59:0/3897250198 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff078012430 con 0x7ff08006f990 2023-11-24T07:56:44.058 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:44.054+0000 7ff086a80700 1 -- 172.21.15.59:0/3897250198 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff08006f990 msgr2=0x7ff0800ce2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:56:44.058 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:44.054+0000 7ff086a80700 1 --2- 172.21.15.59:0/3897250198 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff08006f990 0x7ff0800ce2a0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7ff08006d4b0 tx=0x7ff07800b330 comp rx=0 tx=0).stop 2023-11-24T07:56:44.058 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:44.054+0000 7ff086a80700 1 -- 172.21.15.59:0/3897250198 shutdown_connections 2023-11-24T07:56:44.058 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:44.054+0000 7ff086a80700 1 --2- 172.21.15.59:0/3897250198 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff08006f990 0x7ff0800ce2a0 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:56:44.059 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:44.054+0000 7ff086a80700 1 --2- 172.21.15.59:0/3897250198 >> v2:172.21.15.76:3300/0 conn(0x7ff0801448b0 0x7ff08006ec70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:56:44.059 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:44.054+0000 7ff086a80700 1 -- 172.21.15.59:0/3897250198 >> 172.21.15.59:0/3897250198 conn(0x7ff0800d0bc0 msgr2=0x7ff0800d0fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:56:44.059 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:44.054+0000 7ff086a80700 1 -- 172.21.15.59:0/3897250198 shutdown_connections 2023-11-24T07:56:44.059 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:44.054+0000 7ff086a80700 1 -- 172.21.15.59:0/3897250198 wait complete. 2023-11-24T07:56:44.061 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:44.056+0000 7ff086a80700 1 Processor -- start 2023-11-24T07:56:44.061 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:44.056+0000 7ff086a80700 1 -- start start 2023-11-24T07:56:44.062 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:44.056+0000 7ff086a80700 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7ff08006f990 0x7ff08015a4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:56:44.062 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:44.056+0000 7ff086a80700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff0800b44c0 0x7ff0801f3ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:56:44.062 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:44.057+0000 7ff086a80700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff0801448b0 0x7ff080159c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:56:44.062 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:44.057+0000 7ff086a80700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0801f4000 con 0x7ff0801448b0 2023-11-24T07:56:44.062 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:44.057+0000 7ff086a80700 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0800cb1e0 con 0x7ff0800b44c0 2023-11-24T07:56:44.062 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:44.057+0000 7ff086a80700 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff0800cb4c0 con 0x7ff08006f990 2023-11-24T07:56:44.062 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:44.057+0000 7ff08501d700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff0801448b0 0x7ff080159c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:56:44.062 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:44.057+0000 7ff08501d700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff0801448b0 0x7ff080159c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:50494/0 (socket says 172.21.15.59:50494) 2023-11-24T07:56:44.062 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:44.057+0000 7ff08501d700 1 -- 172.21.15.59:0/1638785233 learned_addr learned my addr 172.21.15.59:0/1638785233 (peer_addr_for_me v2:172.21.15.59:0/0) 2023-11-24T07:56:44.063 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:44.057+0000 7ff07ffff700 1 --2- 172.21.15.59:0/1638785233 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff0800b44c0 0x7ff0801f3ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:56:44.063 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:44.057+0000 7ff08481c700 1 --2- 172.21.15.59:0/1638785233 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7ff08006f990 0x7ff08015a4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:56:44.063 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:44.057+0000 7ff08501d700 1 -- 172.21.15.59:0/1638785233 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7ff08006f990 msgr2=0x7ff08015a4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:56:44.063 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:44.057+0000 7ff08501d700 1 --2- 172.21.15.59:0/1638785233 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7ff08006f990 0x7ff08015a4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:56:44.063 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:44.057+0000 7ff08501d700 1 -- 172.21.15.59:0/1638785233 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff0800b44c0 msgr2=0x7ff0801f3ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:56:44.063 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:44.057+0000 7ff08501d700 1 --2- 172.21.15.59:0/1638785233 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff0800b44c0 0x7ff0801f3ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:56:44.063 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:44.057+0000 7ff08501d700 1 -- 172.21.15.59:0/1638785233 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff078009a70 con 0x7ff0801448b0 2023-11-24T07:56:44.063 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:44.057+0000 7ff08501d700 1 --2- 172.21.15.59:0/1638785233 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff0801448b0 0x7ff080159c80 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7ff078013000 tx=0x7ff078004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:56:44.063 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:44.057+0000 7ff07d7fa700 1 -- 172.21.15.59:0/1638785233 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff078012430 con 0x7ff0801448b0 2023-11-24T07:56:44.064 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:44.057+0000 7ff07d7fa700 1 -- 172.21.15.59:0/1638785233 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff078020070 con 0x7ff0801448b0 2023-11-24T07:56:44.064 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:44.057+0000 7ff086a80700 1 -- 172.21.15.59:0/1638785233 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff0800cb7a0 con 0x7ff0801448b0 2023-11-24T07:56:44.064 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:44.057+0000 7ff07d7fa700 1 -- 172.21.15.59:0/1638785233 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff078004280 con 0x7ff0801448b0 2023-11-24T07:56:44.064 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:44.057+0000 7ff086a80700 1 -- 172.21.15.59:0/1638785233 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff0800cbd60 con 0x7ff0801448b0 2023-11-24T07:56:44.064 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:44.059+0000 7ff07d7fa700 1 -- 172.21.15.59:0/1638785233 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7ff0780043e0 con 0x7ff0801448b0 2023-11-24T07:56:44.064 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:44.059+0000 7ff07d7fa700 1 --2- 172.21.15.59:0/1638785233 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7ff06406aed0 0x7ff06406d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:56:44.064 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:44.059+0000 7ff07d7fa700 1 -- 172.21.15.59:0/1638785233 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7ff078022070 con 0x7ff0801448b0 2023-11-24T07:56:44.064 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:44.059+0000 7ff08481c700 1 --2- 172.21.15.59:0/1638785233 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7ff06406aed0 0x7ff06406d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:56:44.064 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:44.060+0000 7ff08481c700 1 --2- 172.21.15.59:0/1638785233 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7ff06406aed0 0x7ff06406d390 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7ff0801f6fe0 tx=0x7ff074008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:56:44.073 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:44.061+0000 7ff092c79b80 1 Processor -- start 2023-11-24T07:56:44.073 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:44.061+0000 7ff092c79b80 1 -- start start 2023-11-24T07:56:44.073 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:44.062+0000 7ff092c79b80 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x561e47b83080 0x561e47bc9840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:56:44.073 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:44.062+0000 7ff092c79b80 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x561e47b98ed0 0x561e47bc9d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:56:44.073 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:44.062+0000 7ff092c79b80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561e47b80490 0x561e47bace70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:56:44.073 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:44.062+0000 7ff092c79b80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x561e47af6770 con 0x561e47b80490 2023-11-24T07:56:44.073 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:44.062+0000 7ff092c79b80 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x561e47add750 con 0x561e47b98ed0 2023-11-24T07:56:44.073 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:44.062+0000 7ff092c79b80 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 v1 -- 0x561e47adb070 con 0x561e47b83080 2023-11-24T07:56:44.073 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:44.062+0000 7ff08501d700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561e47b80490 0x561e47bace70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:56:44.074 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:44.062+0000 7ff08501d700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561e47b80490 0x561e47bace70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:50502/0 (socket says 172.21.15.59:50502) 2023-11-24T07:56:44.074 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:44.062+0000 7ff08501d700 1 -- 172.21.15.59:0/1334566188 learned_addr learned my addr 172.21.15.59:0/1334566188 (peer_addr_for_me v2:172.21.15.59:0/0) 2023-11-24T07:56:44.074 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:44.062+0000 7ff07ffff700 1 --2- 172.21.15.59:0/1334566188 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x561e47b98ed0 0x561e47bc9d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:56:44.074 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:44.062+0000 7ff08481c700 1 --2- 172.21.15.59:0/1334566188 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x561e47b83080 0x561e47bc9840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:56:44.074 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:44.062+0000 7ff08501d700 1 -- 172.21.15.59:0/1334566188 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x561e47b83080 msgr2=0x561e47bc9840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T07:56:44.074 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:44.062+0000 7ff08501d700 1 --2- 172.21.15.59:0/1334566188 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x561e47b83080 0x561e47bc9840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:56:44.074 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:44.062+0000 7ff08501d700 1 -- 172.21.15.59:0/1334566188 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x561e47b98ed0 msgr2=0x561e47bc9d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T07:56:44.074 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:44.062+0000 7ff08501d700 1 --2- 172.21.15.59:0/1334566188 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x561e47b98ed0 0x561e47bc9d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:56:44.075 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:44.062+0000 7ff08501d700 1 -- 172.21.15.59:0/1334566188 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561e479535e0 con 0x561e47b80490 2023-11-24T07:56:44.075 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:44.062+0000 7ff08501d700 1 --2- 172.21.15.59:0/1334566188 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561e47b80490 0x561e47bace70 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7ff07800b730 tx=0x7ff078009990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d0ab4cc9f9d2f6ec server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:56:44.075 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:44.062+0000 7ff06a7fc700 1 -- 172.21.15.59:0/1334566188 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff078059b50 con 0x561e47b80490 2023-11-24T07:56:44.075 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:44.063+0000 7ff06a7fc700 1 -- 172.21.15.59:0/1334566188 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff078012e40 con 0x561e47b80490 2023-11-24T07:56:44.075 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:44.063+0000 7ff06a7fc700 1 -- 172.21.15.59:0/1334566188 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff07805a8f0 con 0x561e47b80490 2023-11-24T07:56:44.075 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:44.063+0000 7ff092c79b80 1 -- 172.21.15.59:0/1334566188 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561e47b80490 msgr2=0x561e47bace70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:56:44.075 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:44.063+0000 7ff092c79b80 1 --2- 172.21.15.59:0/1334566188 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561e47b80490 0x561e47bace70 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7ff07800b730 tx=0x7ff078009990 comp rx=0 tx=0).stop 2023-11-24T07:56:44.075 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:44.063+0000 7ff092c79b80 1 -- 172.21.15.59:0/1334566188 shutdown_connections 2023-11-24T07:56:44.075 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:44.063+0000 7ff092c79b80 1 --2- 172.21.15.59:0/1334566188 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561e47b80490 0x561e47bace70 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:56:44.075 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:44.063+0000 7ff092c79b80 1 --2- 172.21.15.59:0/1334566188 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x561e47b83080 0x561e47bc9840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:56:44.076 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:44.063+0000 7ff092c79b80 1 --2- 172.21.15.59:0/1334566188 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x561e47b98ed0 0x561e47bc9d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:56:44.076 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:44.063+0000 7ff092c79b80 1 -- 172.21.15.59:0/1334566188 >> 172.21.15.59:0/1334566188 conn(0x561e47ab1310 msgr2=0x561e47bc7e70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:56:44.076 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:44.063+0000 7ff092c79b80 1 -- 172.21.15.59:0/1334566188 shutdown_connections 2023-11-24T07:56:44.076 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:44.063+0000 7ff092c79b80 1 -- 172.21.15.59:0/1334566188 wait complete. 2023-11-24T07:56:44.076 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:44.064+0000 7ff092c79b80 1 -- start start 2023-11-24T07:56:44.076 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:44.064+0000 7ff092c79b80 1 Processor -- start 2023-11-24T07:56:44.076 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:44.065+0000 7ff092c79b80 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x561e47b80490 0x561e47b77730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:56:44.077 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:44.065+0000 7ff092c79b80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561e47b83080 0x561e47bc15d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:56:44.077 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:44.065+0000 7ff092c79b80 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x561e47b98ed0 0x561e47bc1b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:56:44.077 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:44.065+0000 7ff092c79b80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x561e47ad6960 con 0x561e47b83080 2023-11-24T07:56:44.077 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:44.065+0000 7ff092c79b80 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x561e47a90510 con 0x561e47b80490 2023-11-24T07:56:44.077 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:44.065+0000 7ff092c79b80 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 v1 -- 0x561e47a901b0 con 0x561e47b98ed0 2023-11-24T07:56:44.077 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:44.065+0000 7ff092c79b80 10 client.0 _ll_register_callbacks cb 0x561e479bba30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-24T07:56:44.077 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:44.065+0000 7ff07ffff700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561e47b83080 0x561e47bc15d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:56:44.077 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:44.065+0000 7ff07ffff700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561e47b83080 0x561e47bc15d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:50506/0 (socket says 172.21.15.59:50506) 2023-11-24T07:56:44.078 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:44.065+0000 7ff07ffff700 1 -- 172.21.15.59:0/4215782372 learned_addr learned my addr 172.21.15.59:0/4215782372 (peer_addr_for_me v2:172.21.15.59:0/0) 2023-11-24T07:56:44.078 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:44.065+0000 7ff08481c700 1 --2- 172.21.15.59:0/4215782372 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x561e47b80490 0x561e47b77730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:56:44.078 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:44.065+0000 7ff08501d700 1 --2- 172.21.15.59:0/4215782372 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x561e47b98ed0 0x561e47bc1b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:56:44.078 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:44.065+0000 7ff07ffff700 1 -- 172.21.15.59:0/4215782372 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x561e47b98ed0 msgr2=0x561e47bc1b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T07:56:44.078 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:44.065+0000 7ff07ffff700 1 --2- 172.21.15.59:0/4215782372 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x561e47b98ed0 0x561e47bc1b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:56:44.078 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:44.065+0000 7ff07ffff700 1 -- 172.21.15.59:0/4215782372 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x561e47b80490 msgr2=0x561e47b77730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T07:56:44.078 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:44.065+0000 7ff07ffff700 1 --2- 172.21.15.59:0/4215782372 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x561e47b80490 0x561e47b77730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:56:44.078 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:44.065+0000 7ff07ffff700 1 -- 172.21.15.59:0/4215782372 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff0800cbd60 con 0x561e47b83080 2023-11-24T07:56:44.078 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:44.065+0000 7ff07ffff700 1 --2- 172.21.15.59:0/4215782372 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561e47b83080 0x561e47bc15d0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7ff06c0011e0 tx=0x7ff06c001650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1d1e0e4dcc026c11 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:56:44.078 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:44.065+0000 7ff05f7fe700 10 client.0 ms_handle_connect on v2:172.21.15.59:3300/0 2023-11-24T07:56:44.079 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:44.066+0000 7ff05f7fe700 1 -- 172.21.15.59:0/4215782372 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff06c001c60 con 0x561e47b83080 2023-11-24T07:56:44.079 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:44.066+0000 7ff05f7fe700 1 -- 172.21.15.59:0/4215782372 <== mon.0 v2:172.21.15.59:3300/0 2 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7ff06c00c780 con 0x561e47b83080 2023-11-24T07:56:44.079 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:44.066+0000 7ff092c79b80 1 -- 172.21.15.59:0/4215782372 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x561e47b75a40 con 0x561e47b83080 2023-11-24T07:56:44.079 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:44.066+0000 7ff05f7fe700 1 -- 172.21.15.59:0/4215782372 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_get_version_reply(handle=1 version=8) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff06c00c940 con 0x561e47b83080 2023-11-24T07:56:44.079 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:44.066+0000 7ff092c79b80 10 client.4656 fetch_fsmap learned FSMap version 8 2023-11-24T07:56:44.079 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:44.066+0000 7ff092c79b80 1 -- 172.21.15.59:0/4215782372 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,fsmap=8,monmap=2+}) v3 -- 0x561e478ffd90 con 0x561e47b83080 2023-11-24T07:56:44.079 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:44.066+0000 7ff05f7fe700 1 -- 172.21.15.59:0/4215782372 <== mon.0 v2:172.21.15.59:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff06c024020 con 0x561e47b83080 2023-11-24T07:56:44.079 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:44.066+0000 7ff05f7fe700 1 -- 172.21.15.59:0/4215782372 <== mon.0 v2:172.21.15.59:3300/0 5 ==== fsmap(e 8) v1 ==== 3294+0+0 (secure 0 0 0) 0x7ff06c012050 con 0x561e47b83080 2023-11-24T07:56:44.079 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:44.066+0000 7ff092c79b80 10 client.4656 fetch_fsmap finished waiting for FSMap version 8 2023-11-24T07:56:44.079 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:44.066+0000 7ff092c79b80 10 client.4656 resolve_mds: resolved name 'b' to daemon mds.b 2023-11-24T07:56:44.079 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:44.066+0000 7ff092c79b80 20 client.4656 populate_metadata read hostname 'smithi059' 2023-11-24T07:56:44.079 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:44.066+0000 7ff092c79b80 1 --2- 172.21.15.59:0/4215782372 >> [v2:172.21.15.76:6834/3993077463,v1:172.21.15.76:6836/3993077463] conn(0x561e47b98410 0x561e47bb0330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:56:44.080 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:44.066+0000 7ff092c79b80 4 client.4656 mds_command: new command op to 4264 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-11-24T07:56:44.080 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:44.066+0000 7ff092c79b80 1 -- 172.21.15.59:0/4215782372 --> [v2:172.21.15.76:6834/3993077463,v1:172.21.15.76:6836/3993077463] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x561e47b51840 con 0x561e47b98410 2023-11-24T07:56:44.080 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:44.066+0000 7ff08481c700 1 --2- 172.21.15.59:0/4215782372 >> [v2:172.21.15.76:6834/3993077463,v1:172.21.15.76:6836/3993077463] conn(0x561e47b98410 0x561e47bb0330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:56:44.080 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:44.067+0000 7ff05f7fe700 10 client.4656 ms_handle_connect on v2:172.21.15.76:6834/3993077463 2023-11-24T07:56:44.080 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:44.067+0000 7ff08481c700 1 --2- 172.21.15.59:0/4215782372 >> [v2:172.21.15.76:6834/3993077463,v1:172.21.15.76:6836/3993077463] conn(0x561e47b98410 0x561e47bb0330 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=576601d97670359e server_cookie=b5a20cf66d7caa41 in_seq=0 out_seq=0 2023-11-24T07:56:44.080 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:44.067+0000 7ff05f7fe700 1 -- 172.21.15.59:0/4215782372 <== mds.0 v2:172.21.15.76:6834/3993077463 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x561e47b51840 con 0x561e47b98410 2023-11-24T07:56:44.080 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:44.067+0000 7ff05f7fe700 10 client.4656 handle_command_reply: tid=0 2023-11-24T07:56:44.080 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:44.067+0000 7ff092c79b80 1 client.4656 shutdown 2023-11-24T07:56:44.080 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:44.068+0000 7ff092c79b80 1 -- 172.21.15.59:0/4215782372 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561e47b83080 msgr2=0x561e47bc15d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:56:44.080 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:44.068+0000 7ff092c79b80 1 --2- 172.21.15.59:0/4215782372 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561e47b83080 0x561e47bc15d0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7ff06c0011e0 tx=0x7ff06c001650 comp rx=0 tx=0).stop 2023-11-24T07:56:44.080 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:44.068+0000 7ff092c79b80 1 -- 172.21.15.59:0/4215782372 shutdown_connections 2023-11-24T07:56:44.080 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:44.068+0000 7ff092c79b80 1 --2- 172.21.15.59:0/4215782372 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x561e47b98ed0 0x561e47bc1b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:56:44.081 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:44.068+0000 7ff092c79b80 1 --2- 172.21.15.59:0/4215782372 >> [v2:172.21.15.76:6834/3993077463,v1:172.21.15.76:6836/3993077463] conn(0x561e47b98410 0x561e47bb0330 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:56:44.081 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:44.068+0000 7ff092c79b80 1 --2- 172.21.15.59:0/4215782372 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x561e47b80490 0x561e47b77730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:56:44.081 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:44.068+0000 7ff092c79b80 1 --2- 172.21.15.59:0/4215782372 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561e47b83080 0x561e47bc15d0 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:56:44.081 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:44.068+0000 7ff05f7fe700 0 client.4656 ms_handle_reset on v2:172.21.15.76:6834/3993077463 2023-11-24T07:56:44.081 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:44.068+0000 7ff092c79b80 1 -- 172.21.15.59:0/4215782372 >> 172.21.15.59:0/4215782372 conn(0x561e47ab1310 msgr2=0x561e47bcbc80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:56:44.081 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:44.068+0000 7ff092c79b80 1 -- 172.21.15.59:0/4215782372 shutdown_connections 2023-11-24T07:56:44.081 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:44.068+0000 7ff092c79b80 1 -- 172.21.15.59:0/4215782372 wait complete. 2023-11-24T07:56:44.081 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:44.068+0000 7ff092c79b80 20 client.4656 trim_cache size 0 max 16384 2023-11-24T07:56:44.088 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:44.086+0000 7ff092c79b80 1 Processor -- start 2023-11-24T07:56:44.088 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:44.086+0000 7ff092c79b80 1 -- start start 2023-11-24T07:56:44.091 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:44.087+0000 7ff092c79b80 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x561e47b80490 0x561e47bca0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:56:44.091 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:44.087+0000 7ff092c79b80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561e47b83080 0x561e47bad760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:56:44.091 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:44.087+0000 7ff092c79b80 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x561e47b98ed0 0x561e47b7e0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:56:44.091 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:44.087+0000 7ff092c79b80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x561e479558d0 con 0x561e47b83080 2023-11-24T07:56:44.091 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:44.087+0000 7ff092c79b80 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x561e47bc3130 con 0x561e47b98ed0 2023-11-24T07:56:44.091 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:44.087+0000 7ff092c79b80 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 v1 -- 0x561e47bac960 con 0x561e47b80490 2023-11-24T07:56:44.091 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:44.087+0000 7ff07ffff700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561e47b83080 0x561e47bad760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:56:44.092 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:44.087+0000 7ff07ffff700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561e47b83080 0x561e47bad760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:50512/0 (socket says 172.21.15.59:50512) 2023-11-24T07:56:44.092 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:44.087+0000 7ff07ffff700 1 -- 172.21.15.59:0/402745279 learned_addr learned my addr 172.21.15.59:0/402745279 (peer_addr_for_me v2:172.21.15.59:0/0) 2023-11-24T07:56:44.092 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:44.087+0000 7ff08501d700 1 --2- 172.21.15.59:0/402745279 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x561e47b98ed0 0x561e47b7e0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:56:44.092 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:44.087+0000 7ff08481c700 1 --2- 172.21.15.59:0/402745279 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x561e47b80490 0x561e47bca0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:56:44.092 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:44.087+0000 7ff07ffff700 1 -- 172.21.15.59:0/402745279 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x561e47b80490 msgr2=0x561e47bca0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T07:56:44.092 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:44.087+0000 7ff07ffff700 1 --2- 172.21.15.59:0/402745279 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x561e47b80490 0x561e47bca0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:56:44.092 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:44.087+0000 7ff07ffff700 1 -- 172.21.15.59:0/402745279 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x561e47b98ed0 msgr2=0x561e47b7e0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T07:56:44.093 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:44.087+0000 7ff07ffff700 1 --2- 172.21.15.59:0/402745279 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x561e47b98ed0 0x561e47b7e0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:56:44.093 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:44.087+0000 7ff07ffff700 1 -- 172.21.15.59:0/402745279 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561e478e8c10 con 0x561e47b83080 2023-11-24T07:56:44.093 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:44.087+0000 7ff08501d700 1 --2- 172.21.15.59:0/402745279 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x561e47b98ed0 0x561e47b7e0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-24T07:56:44.093 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:44.087+0000 7ff07ffff700 1 --2- 172.21.15.59:0/402745279 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561e47b83080 0x561e47bad760 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7ff06c001a20 tx=0x7ff06c00f000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c9e1289c12e0f0a9 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:56:44.093 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:44.088+0000 7ff06affd700 1 -- 172.21.15.59:0/402745279 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff06c00c280 con 0x561e47b83080 2023-11-24T07:56:44.093 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:44.088+0000 7ff06affd700 1 -- 172.21.15.59:0/402745279 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff06c024070 con 0x561e47b83080 2023-11-24T07:56:44.093 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:44.088+0000 7ff06affd700 1 -- 172.21.15.59:0/402745279 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff06c00c620 con 0x561e47b83080 2023-11-24T07:56:44.093 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:44.088+0000 7ff092c79b80 1 -- 172.21.15.59:0/402745279 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561e47b83080 msgr2=0x561e47bad760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:56:44.093 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:44.088+0000 7ff092c79b80 1 --2- 172.21.15.59:0/402745279 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561e47b83080 0x561e47bad760 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7ff06c001a20 tx=0x7ff06c00f000 comp rx=0 tx=0).stop 2023-11-24T07:56:44.093 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:44.088+0000 7ff092c79b80 1 -- 172.21.15.59:0/402745279 shutdown_connections 2023-11-24T07:56:44.094 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:44.088+0000 7ff092c79b80 1 --2- 172.21.15.59:0/402745279 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x561e47b98ed0 0x561e47b7e0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:56:44.094 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:44.088+0000 7ff092c79b80 1 --2- 172.21.15.59:0/402745279 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x561e47b80490 0x561e47bca0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:56:44.094 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:44.088+0000 7ff092c79b80 1 --2- 172.21.15.59:0/402745279 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561e47b83080 0x561e47bad760 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:56:44.094 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:44.088+0000 7ff092c79b80 1 -- 172.21.15.59:0/402745279 >> 172.21.15.59:0/402745279 conn(0x561e47ab1310 msgr2=0x561e47b77540 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:56:44.094 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:44.088+0000 7ff092c79b80 1 -- 172.21.15.59:0/402745279 shutdown_connections 2023-11-24T07:56:44.094 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:44.088+0000 7ff092c79b80 1 -- 172.21.15.59:0/402745279 wait complete. 2023-11-24T07:56:44.094 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:44.089+0000 7ff092c79b80 1 -- start start 2023-11-24T07:56:44.094 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:44.089+0000 7ff092c79b80 1 Processor -- start 2023-11-24T07:56:44.099 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:44.090+0000 7ff092c79b80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561e47b80490 0x561e47bad1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:56:44.099 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:44.090+0000 7ff092c79b80 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x561e47b83080 0x561e47bad700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:56:44.100 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:44.090+0000 7ff092c79b80 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x561e47b98ed0 0x561e47b969e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:56:44.100 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:44.090+0000 7ff092c79b80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x561e47b97890 con 0x561e47b80490 2023-11-24T07:56:44.100 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:44.090+0000 7ff092c79b80 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x561e47bc7520 con 0x561e47b98ed0 2023-11-24T07:56:44.100 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:44.090+0000 7ff092c79b80 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 v1 -- 0x561e47bb0020 con 0x561e47b83080 2023-11-24T07:56:44.100 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:44.090+0000 7ff08481c700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561e47b80490 0x561e47bad1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:56:44.100 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:44.090+0000 7ff092c79b80 10 client.0 _ll_register_callbacks cb 0x561e479bba30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-24T07:56:44.100 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:44.090+0000 7ff08481c700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561e47b80490 0x561e47bad1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:50518/0 (socket says 172.21.15.59:50518) 2023-11-24T07:56:44.100 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:44.090+0000 7ff08481c700 1 -- 172.21.15.59:0/3223105916 learned_addr learned my addr 172.21.15.59:0/3223105916 (peer_addr_for_me v2:172.21.15.59:0/0) 2023-11-24T07:56:44.100 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:44.091+0000 7ff07ffff700 1 --2- 172.21.15.59:0/3223105916 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x561e47b83080 0x561e47bad700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:56:44.100 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:44.091+0000 7ff08501d700 1 --2- 172.21.15.59:0/3223105916 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x561e47b98ed0 0x561e47b969e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:56:44.101 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:44.091+0000 7ff08481c700 1 -- 172.21.15.59:0/3223105916 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x561e47b83080 msgr2=0x561e47bad700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T07:56:44.101 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:44.091+0000 7ff08481c700 1 --2- 172.21.15.59:0/3223105916 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x561e47b83080 0x561e47bad700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:56:44.101 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:44.091+0000 7ff08481c700 1 -- 172.21.15.59:0/3223105916 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x561e47b98ed0 msgr2=0x561e47b969e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T07:56:44.101 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:44.091+0000 7ff08481c700 1 --2- 172.21.15.59:0/3223105916 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x561e47b98ed0 0x561e47b969e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:56:44.101 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:44.091+0000 7ff08481c700 1 -- 172.21.15.59:0/3223105916 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x561e478ffd90 con 0x561e47b80490 2023-11-24T07:56:44.101 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:44.091+0000 7ff07ffff700 1 --2- 172.21.15.59:0/3223105916 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x561e47b83080 0x561e47bad700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T07:56:44.101 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:44.091+0000 7ff08501d700 1 --2- 172.21.15.59:0/3223105916 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x561e47b98ed0 0x561e47b969e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T07:56:44.101 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:44.091+0000 7ff08481c700 1 --2- 172.21.15.59:0/3223105916 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561e47b80490 0x561e47bad1c0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7ff074006f10 tx=0x7ff0740047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2ef4fbd96fb1ccd6 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:56:44.102 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:44.091+0000 7ff05f7fe700 10 client.0 ms_handle_connect on v2:172.21.15.59:3300/0 2023-11-24T07:56:44.102 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:44.091+0000 7ff05f7fe700 1 -- 172.21.15.59:0/3223105916 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff074029040 con 0x561e47b80490 2023-11-24T07:56:44.102 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:44.091+0000 7ff05f7fe700 1 -- 172.21.15.59:0/3223105916 <== mon.0 v2:172.21.15.59:3300/0 2 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7ff074015e20 con 0x561e47b80490 2023-11-24T07:56:44.102 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:44.091+0000 7ff092c79b80 1 -- 172.21.15.59:0/3223105916 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7ff078012e40 con 0x561e47b80490 2023-11-24T07:56:44.102 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:44.091+0000 7ff05f7fe700 1 -- 172.21.15.59:0/3223105916 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_get_version_reply(handle=1 version=8) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff074005c20 con 0x561e47b80490 2023-11-24T07:56:44.102 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:44.091+0000 7ff092c79b80 10 client.4662 fetch_fsmap learned FSMap version 8 2023-11-24T07:56:44.102 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:44.091+0000 7ff092c79b80 1 -- 172.21.15.59:0/3223105916 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,fsmap=8,monmap=2+}) v3 -- 0x561e47926200 con 0x561e47b80490 2023-11-24T07:56:44.102 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:44.091+0000 7ff05f7fe700 1 -- 172.21.15.59:0/3223105916 <== mon.0 v2:172.21.15.59:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff078012e40 con 0x561e47b80490 2023-11-24T07:56:44.102 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:44.091+0000 7ff05f7fe700 1 -- 172.21.15.59:0/3223105916 <== mon.0 v2:172.21.15.59:3300/0 5 ==== fsmap(e 8) v1 ==== 3294+0+0 (secure 0 0 0) 0x7ff07402c070 con 0x561e47b80490 2023-11-24T07:56:44.102 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:44.091+0000 7ff092c79b80 10 client.4662 fetch_fsmap finished waiting for FSMap version 8 2023-11-24T07:56:44.102 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:44.091+0000 7ff092c79b80 10 client.4662 resolve_mds: resolved name 'b' to daemon mds.b 2023-11-24T07:56:44.103 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:44.091+0000 7ff092c79b80 20 client.4662 populate_metadata read hostname 'smithi059' 2023-11-24T07:56:44.103 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:44.092+0000 7ff092c79b80 1 --2- 172.21.15.59:0/3223105916 >> [v2:172.21.15.76:6834/3993077463,v1:172.21.15.76:6836/3993077463] conn(0x561e47bb0da0 0x561e47b96f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:56:44.103 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:44.092+0000 7ff092c79b80 4 client.4662 mds_command: new command op to 4264 tid=0 multi_id=0 [{"prefix": "status"}] 2023-11-24T07:56:44.103 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:44.092+0000 7ff092c79b80 1 -- 172.21.15.59:0/3223105916 --> [v2:172.21.15.76:6834/3993077463,v1:172.21.15.76:6836/3993077463] -- command(tid 0: {"prefix": "status"}) v1 -- 0x561e47a9cf80 con 0x561e47bb0da0 2023-11-24T07:56:44.103 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:44.092+0000 7ff07ffff700 1 --2- 172.21.15.59:0/3223105916 >> [v2:172.21.15.76:6834/3993077463,v1:172.21.15.76:6836/3993077463] conn(0x561e47bb0da0 0x561e47b96f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:56:44.103 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:44.092+0000 7ff05f7fe700 10 client.4662 ms_handle_connect on v2:172.21.15.76:6834/3993077463 2023-11-24T07:56:44.103 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:44.092+0000 7ff07ffff700 1 --2- 172.21.15.59:0/3223105916 >> [v2:172.21.15.76:6834/3993077463,v1:172.21.15.76:6836/3993077463] conn(0x561e47bb0da0 0x561e47b96f20 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=eac92b70cf64e2fc server_cookie=5764fb04efa57eb0 in_seq=0 out_seq=0 2023-11-24T07:56:44.103 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:44.092+0000 7ff05f7fe700 1 -- 172.21.15.59:0/3223105916 <== mds.0 v2:172.21.15.76:6834/3993077463 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+330 (crc 0 0 0) 0x561e47a9cf80 con 0x561e47bb0da0 2023-11-24T07:56:44.103 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:44.092+0000 7ff05f7fe700 10 client.4662 handle_command_reply: tid=0 2023-11-24T07:56:44.104 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:44.092+0000 7ff092c79b80 1 client.4662 shutdown 2023-11-24T07:56:44.104 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:44.092+0000 7ff092c79b80 1 -- 172.21.15.59:0/3223105916 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561e47b80490 msgr2=0x561e47bad1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:56:44.104 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:44.092+0000 7ff092c79b80 1 --2- 172.21.15.59:0/3223105916 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561e47b80490 0x561e47bad1c0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7ff074006f10 tx=0x7ff0740047c0 comp rx=0 tx=0).stop 2023-11-24T07:56:44.104 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:44.092+0000 7ff092c79b80 1 -- 172.21.15.59:0/3223105916 shutdown_connections 2023-11-24T07:56:44.104 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:44.092+0000 7ff092c79b80 1 --2- 172.21.15.59:0/3223105916 >> [v2:172.21.15.76:6834/3993077463,v1:172.21.15.76:6836/3993077463] conn(0x561e47bb0da0 0x561e47b96f20 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:56:44.104 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:44.092+0000 7ff092c79b80 1 --2- 172.21.15.59:0/3223105916 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x561e47b98ed0 0x561e47b969e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:56:44.104 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:44.092+0000 7ff092c79b80 1 --2- 172.21.15.59:0/3223105916 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561e47b80490 0x561e47bad1c0 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:56:44.104 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:44.092+0000 7ff05f7fe700 0 client.4662 ms_handle_reset on v2:172.21.15.76:6834/3993077463 2023-11-24T07:56:44.104 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:44.092+0000 7ff092c79b80 1 --2- 172.21.15.59:0/3223105916 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x561e47b83080 0x561e47bad700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:56:44.104 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:44.093+0000 7ff092c79b80 1 -- 172.21.15.59:0/3223105916 >> 172.21.15.59:0/3223105916 conn(0x561e47ab1310 msgr2=0x561e47ba07d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:56:44.105 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:44.093+0000 7ff092c79b80 1 -- 172.21.15.59:0/3223105916 shutdown_connections 2023-11-24T07:56:44.105 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:44.093+0000 7ff092c79b80 1 -- 172.21.15.59:0/3223105916 wait complete. 2023-11-24T07:56:44.105 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:44.093+0000 7ff092c79b80 20 client.4662 trim_cache size 0 max 16384 2023-11-24T07:56:44.105 INFO:teuthology.orchestra.run.smithi059.stdout:{ 2023-11-24T07:56:44.105 INFO:teuthology.orchestra.run.smithi059.stdout: "cluster_fsid": "9f0e8741-b8f9-48d5-a2c3-09f45903303d", 2023-11-24T07:56:44.105 INFO:teuthology.orchestra.run.smithi059.stdout: "whoami": 0, 2023-11-24T07:56:44.105 INFO:teuthology.orchestra.run.smithi059.stdout: "id": 4264, 2023-11-24T07:56:44.105 INFO:teuthology.orchestra.run.smithi059.stdout: "want_state": "up:active", 2023-11-24T07:56:44.106 INFO:teuthology.orchestra.run.smithi059.stdout: "state": "up:active", 2023-11-24T07:56:44.106 INFO:teuthology.orchestra.run.smithi059.stdout: "fs_name": "cephfs", 2023-11-24T07:56:44.106 INFO:teuthology.orchestra.run.smithi059.stdout: "rank_uptime": 19.369804126999998, 2023-11-24T07:56:44.106 INFO:teuthology.orchestra.run.smithi059.stdout: "mdsmap_epoch": 8, 2023-11-24T07:56:44.106 INFO:teuthology.orchestra.run.smithi059.stdout: "osdmap_epoch": 20, 2023-11-24T07:56:44.106 INFO:teuthology.orchestra.run.smithi059.stdout: "osdmap_epoch_barrier": 20, 2023-11-24T07:56:44.106 INFO:teuthology.orchestra.run.smithi059.stdout: "uptime": 26.427320228999999 2023-11-24T07:56:44.106 INFO:teuthology.orchestra.run.smithi059.stdout:} 2023-11-24T07:56:44.108 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:44.093+0000 7ff086a80700 1 -- 172.21.15.59:0/1638785233 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7ff06406aed0 msgr2=0x7ff06406d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:56:44.108 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:44.093+0000 7ff086a80700 1 --2- 172.21.15.59:0/1638785233 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7ff06406aed0 0x7ff06406d390 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7ff0801f6fe0 tx=0x7ff074008040 comp rx=0 tx=0).stop 2023-11-24T07:56:44.108 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:44.093+0000 7ff086a80700 1 -- 172.21.15.59:0/1638785233 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff0801448b0 msgr2=0x7ff080159c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:56:44.108 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:44.093+0000 7ff086a80700 1 --2- 172.21.15.59:0/1638785233 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff0801448b0 0x7ff080159c80 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7ff078013000 tx=0x7ff078004ad0 comp rx=0 tx=0).stop 2023-11-24T07:56:44.108 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:44.093+0000 7ff086a80700 1 -- 172.21.15.59:0/1638785233 shutdown_connections 2023-11-24T07:56:44.108 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:44.093+0000 7ff086a80700 1 --2- 172.21.15.59:0/1638785233 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7ff06406aed0 0x7ff06406d390 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:56:44.109 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:44.093+0000 7ff086a80700 1 --2- 172.21.15.59:0/1638785233 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff0801448b0 0x7ff080159c80 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:56:44.109 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:44.093+0000 7ff086a80700 1 --2- 172.21.15.59:0/1638785233 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7ff08006f990 0x7ff08015a4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:56:44.109 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:44.093+0000 7ff086a80700 1 --2- 172.21.15.59:0/1638785233 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff0800b44c0 0x7ff0801f3ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:56:44.109 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:44.093+0000 7ff086a80700 1 -- 172.21.15.59:0/1638785233 >> 172.21.15.59:0/1638785233 conn(0x7ff0800d0bc0 msgr2=0x7ff080071df0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:56:44.109 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:44.093+0000 7ff086a80700 1 -- 172.21.15.59:0/1638785233 shutdown_connections 2023-11-24T07:56:44.109 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:44.093+0000 7ff086a80700 1 -- 172.21.15.59:0/1638785233 wait complete. 2023-11-24T07:56:44.109 INFO:teuthology.run_tasks:Running task kclient... 2023-11-24T07:56:44.133 INFO:tasks.kclient:Mounting kernel clients... 2023-11-24T07:56:44.133 INFO:tasks.kclient:config is {'client.0': {}, 'client.2': {}, 'client.1': {}, 'client.3': {}} 2023-11-24T07:56:44.134 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T07:56:44.134 DEBUG:teuthology.orchestra.run.smithi059:> ip netns list 2023-11-24T07:56:44.152 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T07:56:44.152 DEBUG:teuthology.orchestra.run.smithi059:> sudo ip link delete ceph-brx 2023-11-24T07:56:44.221 INFO:teuthology.orchestra.run.smithi059.stderr:Cannot find device "ceph-brx" 2023-11-24T07:56:44.222 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-11-24T07:56:44.223 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T07:56:44.223 DEBUG:teuthology.orchestra.run.smithi059:> ip netns list 2023-11-24T07:56:44.282 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T07:56:44.282 DEBUG:teuthology.orchestra.run.smithi059:> sudo ip link delete ceph-brx 2023-11-24T07:56:44.349 INFO:teuthology.orchestra.run.smithi059.stderr:Cannot find device "ceph-brx" 2023-11-24T07:56:44.350 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-11-24T07:56:44.351 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T07:56:44.351 DEBUG:teuthology.orchestra.run.smithi076:> ip netns list 2023-11-24T07:56:44.372 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T07:56:44.372 DEBUG:teuthology.orchestra.run.smithi076:> sudo ip link delete ceph-brx 2023-11-24T07:56:44.447 INFO:teuthology.orchestra.run.smithi076.stderr:Cannot find device "ceph-brx" 2023-11-24T07:56:44.448 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-11-24T07:56:44.449 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T07:56:44.449 DEBUG:teuthology.orchestra.run.smithi076:> ip netns list 2023-11-24T07:56:44.509 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T07:56:44.509 DEBUG:teuthology.orchestra.run.smithi076:> sudo ip link delete ceph-brx 2023-11-24T07:56:44.582 INFO:teuthology.orchestra.run.smithi076.stderr:Cannot find device "ceph-brx" 2023-11-24T07:56:44.583 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-11-24T07:56:44.584 INFO:tasks.kclient:client.0 config is {'syntax': 'v2'} 2023-11-24T07:56:44.584 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2023-11-24T07:56:44.584 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-11-24T07:56:44.584 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-11-24T07:56:44.710 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:44.708+0000 7f21b97b2700 1 Processor -- start 2023-11-24T07:56:44.710 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:44.708+0000 7f21b97b2700 1 -- start start 2023-11-24T07:56:44.714 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:44.709+0000 7f21b97b2700 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f21b4144610 0x7f21b406e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:56:44.714 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:44.709+0000 7f21b97b2700 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f21b40b5220 0x7f21b4071b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:56:44.714 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:44.709+0000 7f21b97b2700 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f21b40b4610 con 0x7f21b40b4da0 2023-11-24T07:56:44.714 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:44.709+0000 7f21b97b2700 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f21b40b56b0 con 0x7f21b4144610 2023-11-24T07:56:44.714 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:44.709+0000 7f21b97b2700 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f21b41449f0 con 0x7f21b40b5220 2023-11-24T07:56:44.714 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:44.709+0000 7f21b27fc700 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7f21b40b4da0 0x7f21b415c340 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:54000/0 (socket says 172.21.15.59:54000) 2023-11-24T07:56:44.714 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:44.709+0000 7f21b27fc700 1 -- 172.21.15.59:0/813093388 learned_addr learned my addr 172.21.15.59:0/813093388 (peer_addr_for_me v1:172.21.15.59:0/0) 2023-11-24T07:56:44.714 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:44.710+0000 7f21b37fe700 1 --2- 172.21.15.59:0/813093388 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f21b40b5220 0x7f21b4071b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:56:44.715 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:44.710+0000 7f21b2ffd700 1 --2- 172.21.15.59:0/813093388 >> v2:172.21.15.76:3300/0 conn(0x7f21b4144610 0x7f21b406e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:56:44.715 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:44.710+0000 7f21b1ffb700 1 -- 172.21.15.59:0/813093388 <== mon.0 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4156420991 0 0) 0x7f21b40b4610 con 0x7f21b40b4da0 2023-11-24T07:56:44.715 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:44.710+0000 7f21b1ffb700 1 -- 172.21.15.59:0/813093388 --> v1:172.21.15.59:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f219c0035b0 con 0x7f21b40b4da0 2023-11-24T07:56:44.715 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:44.710+0000 7f21b1ffb700 1 -- 172.21.15.59:0/813093388 <== mon.0 v1:172.21.15.59:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1367996968 0 0) 0x7f219c0035b0 con 0x7f21b40b4da0 2023-11-24T07:56:44.715 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:44.710+0000 7f21b1ffb700 1 -- 172.21.15.59:0/813093388 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f21b40b5220 msgr2=0x7f21b4071b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T07:56:44.715 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:44.710+0000 7f21b1ffb700 1 --2- 172.21.15.59:0/813093388 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f21b40b5220 0x7f21b4071b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:56:44.715 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:44.710+0000 7f21b1ffb700 1 -- 172.21.15.59:0/813093388 >> v2:172.21.15.76:3300/0 conn(0x7f21b4144610 msgr2=0x7f21b406e9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T07:56:44.715 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:44.710+0000 7f21b1ffb700 1 --2- 172.21.15.59:0/813093388 >> v2:172.21.15.76:3300/0 conn(0x7f21b4144610 0x7f21b406e9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:56:44.715 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:44.710+0000 7f21b1ffb700 1 -- 172.21.15.59:0/813093388 --> v1:172.21.15.59:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f21b406f390 con 0x7f21b40b4da0 2023-11-24T07:56:44.715 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:44.710+0000 7f21b1ffb700 1 -- 172.21.15.59:0/813093388 <== mon.0 v1:172.21.15.59:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1312333566 0 0) 0x7f21a4001670 con 0x7f21b40b4da0 2023-11-24T07:56:44.715 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:44.710+0000 7f21b1ffb700 1 -- 172.21.15.59:0/813093388 >> v1:172.21.15.59:6789/0 conn(0x7f21b40b4da0 legacy=0x7f21b415c340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:56:44.715 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:44.710+0000 7f21b1ffb700 1 --2- 172.21.15.59:0/813093388 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f219c003a00 0x7f219c003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:56:44.716 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:44.710+0000 7f21b1ffb700 1 --2- 172.21.15.59:0/813093388 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f21b40b5220 0x7f219c004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:56:44.716 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:44.710+0000 7f21b1ffb700 1 --2- 172.21.15.59:0/813093388 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f219c004d90 0x7f219c005210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:56:44.716 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:44.710+0000 7f21b1ffb700 1 -- 172.21.15.59:0/813093388 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f21b40b56b0 con 0x7f219c003a00 2023-11-24T07:56:44.716 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:44.710+0000 7f21b1ffb700 1 -- 172.21.15.59:0/813093388 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f21b41449f0 con 0x7f219c004d90 2023-11-24T07:56:44.716 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:44.710+0000 7f21b1ffb700 1 -- 172.21.15.59:0/813093388 --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f219c00dc20 con 0x7f21b40b5220 2023-11-24T07:56:44.716 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:44.710+0000 7f21b2ffd700 1 --2- 172.21.15.59:0/813093388 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f219c003a00 0x7f219c003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:56:44.716 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:44.710+0000 7f21b37fe700 1 --2- 172.21.15.59:0/813093388 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f219c004d90 0x7f219c005210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:56:44.716 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:44.710+0000 7f21b27fc700 1 --2- 172.21.15.59:0/813093388 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f21b40b5220 0x7f219c004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:56:44.716 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:44.710+0000 7f21b2ffd700 1 -- 172.21.15.59:0/813093388 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f21b40b5220 msgr2=0x7f219c004850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T07:56:44.716 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:44.710+0000 7f21b2ffd700 1 --2- 172.21.15.59:0/813093388 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f21b40b5220 0x7f219c004850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:56:44.716 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:44.710+0000 7f21b2ffd700 1 -- 172.21.15.59:0/813093388 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f219c004d90 msgr2=0x7f219c005210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T07:56:44.716 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:44.710+0000 7f21b2ffd700 1 --2- 172.21.15.59:0/813093388 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f219c004d90 0x7f219c005210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:56:44.716 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:44.710+0000 7f21b2ffd700 1 -- 172.21.15.59:0/813093388 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f219c0035b0 con 0x7f219c003a00 2023-11-24T07:56:44.717 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:44.710+0000 7f21b27fc700 1 --2- 172.21.15.59:0/813093388 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f21b40b5220 0x7f219c004850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-24T07:56:44.717 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:44.711+0000 7f21b2ffd700 1 --2- 172.21.15.59:0/813093388 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f219c003a00 0x7f219c003e20 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f21a8009b60 tx=0x7f219c005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=22809a87c840a85b server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:56:44.717 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:44.711+0000 7f21b1ffb700 1 -- 172.21.15.59:0/813093388 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21a800e6c0 con 0x7f219c003a00 2023-11-24T07:56:44.717 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:44.711+0000 7f21b1ffb700 1 -- 172.21.15.59:0/813093388 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f21a800e820 con 0x7f219c003a00 2023-11-24T07:56:44.717 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:44.711+0000 7f21b1ffb700 1 -- 172.21.15.59:0/813093388 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21a800eb90 con 0x7f219c003a00 2023-11-24T07:56:44.717 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:44.711+0000 7f21b97b2700 1 -- 172.21.15.59:0/813093388 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f219c003a00 msgr2=0x7f219c003e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:56:44.717 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:44.711+0000 7f21b97b2700 1 --2- 172.21.15.59:0/813093388 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f219c003a00 0x7f219c003e20 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f21a8009b60 tx=0x7f219c005880 comp rx=0 tx=0).stop 2023-11-24T07:56:44.717 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:44.711+0000 7f21b37fe700 1 -- 172.21.15.59:0/813093388 reap_dead start 2023-11-24T07:56:44.717 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:44.711+0000 7f21b97b2700 1 -- 172.21.15.59:0/813093388 shutdown_connections 2023-11-24T07:56:44.717 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:44.711+0000 7f21b97b2700 1 -- 172.21.15.59:0/813093388 >> 172.21.15.59:0/813093388 conn(0x7f21b406c0e0 msgr2=0x7f21b406c4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:56:44.717 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:44.711+0000 7f21b97b2700 1 -- 172.21.15.59:0/813093388 shutdown_connections 2023-11-24T07:56:44.717 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:44.711+0000 7f21b97b2700 1 -- 172.21.15.59:0/813093388 wait complete. 2023-11-24T07:56:44.720 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:44.713+0000 7f21b97b2700 1 Processor -- start 2023-11-24T07:56:44.720 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:44.713+0000 7f21b97b2700 1 -- start start 2023-11-24T07:56:44.720 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:44.714+0000 7f21b97b2700 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f219c003a00 0x7f21b4158150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:56:44.720 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:44.714+0000 7f21b97b2700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f21b41572a0 0x7f21b41f3bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:56:44.720 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:44.714+0000 7f21b97b2700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f21b415e040 0x7f21b415e4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:56:44.720 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:44.714+0000 7f21b97b2700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f21b415ea00 con 0x7f21b415e040 2023-11-24T07:56:44.720 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:44.714+0000 7f21b97b2700 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f21b4157d40 con 0x7f21b41572a0 2023-11-24T07:56:44.720 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:44.714+0000 7f21b97b2700 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f21b41f4110 con 0x7f219c003a00 2023-11-24T07:56:44.720 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:44.714+0000 7f21b37fe700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f21b415e040 0x7f21b415e4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:56:44.721 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:44.714+0000 7f21b37fe700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f21b415e040 0x7f21b415e4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:50540/0 (socket says 172.21.15.59:50540) 2023-11-24T07:56:44.721 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:44.714+0000 7f21b37fe700 1 -- 172.21.15.59:0/1795117348 learned_addr learned my addr 172.21.15.59:0/1795117348 (peer_addr_for_me v2:172.21.15.59:0/0) 2023-11-24T07:56:44.721 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:44.714+0000 7f21b2ffd700 1 --2- 172.21.15.59:0/1795117348 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f219c003a00 0x7f21b4158150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:56:44.721 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:44.714+0000 7f21b27fc700 1 --2- 172.21.15.59:0/1795117348 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f21b41572a0 0x7f21b41f3bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:56:44.721 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:44.714+0000 7f21b37fe700 1 -- 172.21.15.59:0/1795117348 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f219c003a00 msgr2=0x7f21b4158150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:56:44.721 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:44.714+0000 7f21b37fe700 1 --2- 172.21.15.59:0/1795117348 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f219c003a00 0x7f21b4158150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:56:44.721 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:44.714+0000 7f21b37fe700 1 -- 172.21.15.59:0/1795117348 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f21b41572a0 msgr2=0x7f21b41f3bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:56:44.721 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:44.714+0000 7f21b37fe700 1 --2- 172.21.15.59:0/1795117348 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f21b41572a0 0x7f21b41f3bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:56:44.721 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:44.714+0000 7f21b37fe700 1 -- 172.21.15.59:0/1795117348 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f21b4157a60 con 0x7f21b415e040 2023-11-24T07:56:44.721 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:44.714+0000 7f21b37fe700 1 --2- 172.21.15.59:0/1795117348 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f21b415e040 0x7f21b415e4c0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f21b40d19e0 tx=0x7f21ac011ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:56:44.721 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:44.714+0000 7f219b7fe700 1 -- 172.21.15.59:0/1795117348 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21ac00e040 con 0x7f21b415e040 2023-11-24T07:56:44.721 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:44.715+0000 7f219b7fe700 1 -- 172.21.15.59:0/1795117348 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f21ac00f040 con 0x7f21b415e040 2023-11-24T07:56:44.722 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:44.715+0000 7f21b97b2700 1 -- 172.21.15.59:0/1795117348 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f21b40c9ef0 con 0x7f21b415e040 2023-11-24T07:56:44.722 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:44.715+0000 7f219b7fe700 1 -- 172.21.15.59:0/1795117348 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21ac00fc40 con 0x7f21b415e040 2023-11-24T07:56:44.722 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:44.715+0000 7f21b97b2700 1 -- 172.21.15.59:0/1795117348 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f21b40ca4b0 con 0x7f21b415e040 2023-11-24T07:56:44.722 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:44.715+0000 7f21b97b2700 1 -- 172.21.15.59:0/1795117348 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f21b4068670 con 0x7f21b415e040 2023-11-24T07:56:44.722 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:44.716+0000 7f219b7fe700 1 -- 172.21.15.59:0/1795117348 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f21ac015430 con 0x7f21b415e040 2023-11-24T07:56:44.722 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:44.716+0000 7f219b7fe700 1 --2- 172.21.15.59:0/1795117348 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f21a006aed0 0x7f21a006d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:56:44.722 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:44.716+0000 7f219b7fe700 1 -- 172.21.15.59:0/1795117348 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f21ac089d40 con 0x7f21b415e040 2023-11-24T07:56:44.722 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:44.717+0000 7f21b2ffd700 1 --2- 172.21.15.59:0/1795117348 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f21a006aed0 0x7f21a006d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:56:44.722 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:44.717+0000 7f21b2ffd700 1 --2- 172.21.15.59:0/1795117348 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f21a006aed0 0x7f21a006d390 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f21a80090b0 tx=0x7f21a801f040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:56:44.723 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:44.721+0000 7f219b7fe700 1 -- 172.21.15.59:0/1795117348 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f21ac054f30 con 0x7f21b415e040 2023-11-24T07:56:44.974 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:44.972+0000 7f21b97b2700 1 -- 172.21.15.59:0/1795117348 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f21b415d9e0 con 0x7f21b415e040 2023-11-24T07:56:44.974 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:44.972+0000 7f219b7fe700 1 -- 172.21.15.59:0/1795117348 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v8) v1 ==== 53+0+73 (secure 0 0 0) 0x7f21ac058550 con 0x7f21b415e040 2023-11-24T07:56:44.974 INFO:teuthology.orchestra.run.smithi059.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-11-24T07:56:44.976 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:44.974+0000 7f21b97b2700 1 -- 172.21.15.59:0/1795117348 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f21a006aed0 msgr2=0x7f21a006d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:56:44.976 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:44.974+0000 7f21b97b2700 1 --2- 172.21.15.59:0/1795117348 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f21a006aed0 0x7f21a006d390 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f21a80090b0 tx=0x7f21a801f040 comp rx=0 tx=0).stop 2023-11-24T07:56:44.976 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:44.974+0000 7f21b97b2700 1 -- 172.21.15.59:0/1795117348 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f21b415e040 msgr2=0x7f21b415e4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:56:44.976 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:44.974+0000 7f21b97b2700 1 --2- 172.21.15.59:0/1795117348 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f21b415e040 0x7f21b415e4c0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f21b40d19e0 tx=0x7f21ac011ed0 comp rx=0 tx=0).stop 2023-11-24T07:56:44.976 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:44.974+0000 7f21b97b2700 1 -- 172.21.15.59:0/1795117348 shutdown_connections 2023-11-24T07:56:44.976 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:44.974+0000 7f21b97b2700 1 --2- 172.21.15.59:0/1795117348 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f21a006aed0 0x7f21a006d390 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:56:44.976 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:44.974+0000 7f21b97b2700 1 --2- 172.21.15.59:0/1795117348 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f21b415e040 0x7f21b415e4c0 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:56:44.976 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:44.974+0000 7f21b97b2700 1 --2- 172.21.15.59:0/1795117348 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f219c003a00 0x7f21b4158150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:56:44.977 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:44.974+0000 7f21b97b2700 1 --2- 172.21.15.59:0/1795117348 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f21b41572a0 0x7f21b41f3bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:56:44.977 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:44.974+0000 7f21b97b2700 1 -- 172.21.15.59:0/1795117348 >> 172.21.15.59:0/1795117348 conn(0x7f21b406c0e0 msgr2=0x7f21b40ccc30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:56:44.977 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:44.974+0000 7f21b97b2700 1 -- 172.21.15.59:0/1795117348 shutdown_connections 2023-11-24T07:56:44.977 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:44.974+0000 7f21b97b2700 1 -- 172.21.15.59:0/1795117348 wait complete. 2023-11-24T07:56:44.988 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2023-11-24T07:56:44.988 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-11-24T07:56:44.988 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi059.front.sepia.ceph.com 2023-11-24T07:56:44.988 INFO:tasks.cephfs.mount:self.client.name = client.0 2023-11-24T07:56:44.988 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-11-24T07:56:44.988 INFO:tasks.cephfs.mount:self.cephfs_name = None 2023-11-24T07:56:44.989 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-11-24T07:56:44.989 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-11-24T07:56:44.989 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.0' 2023-11-24T07:56:44.989 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T07:56:44.989 DEBUG:teuthology.orchestra.run.smithi059:> ip addr 2023-11-24T07:56:45.006 INFO:teuthology.orchestra.run.smithi059.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-11-24T07:56:45.007 INFO:teuthology.orchestra.run.smithi059.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-11-24T07:56:45.007 INFO:teuthology.orchestra.run.smithi059.stdout: inet 127.0.0.1/8 scope host lo 2023-11-24T07:56:45.007 INFO:teuthology.orchestra.run.smithi059.stdout: valid_lft forever preferred_lft forever 2023-11-24T07:56:45.007 INFO:teuthology.orchestra.run.smithi059.stdout: inet6 ::1/128 scope host 2023-11-24T07:56:45.007 INFO:teuthology.orchestra.run.smithi059.stdout: valid_lft forever preferred_lft forever 2023-11-24T07:56:45.007 INFO:teuthology.orchestra.run.smithi059.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-11-24T07:56:45.007 INFO:teuthology.orchestra.run.smithi059.stdout: link/ether 0c:c4:7a:6b:fe:ca brd ff:ff:ff:ff:ff:ff 2023-11-24T07:56:45.007 INFO:teuthology.orchestra.run.smithi059.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-11-24T07:56:45.007 INFO:teuthology.orchestra.run.smithi059.stdout: link/ether 0c:c4:7a:6b:fe:cb brd ff:ff:ff:ff:ff:ff 2023-11-24T07:56:45.007 INFO:teuthology.orchestra.run.smithi059.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2023-11-24T07:56:45.007 INFO:teuthology.orchestra.run.smithi059.stdout: link/ether 0c:c4:7a:bd:19:2a brd ff:ff:ff:ff:ff:ff 2023-11-24T07:56:45.008 INFO:teuthology.orchestra.run.smithi059.stdout: inet 172.21.15.59/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2023-11-24T07:56:45.008 INFO:teuthology.orchestra.run.smithi059.stdout: valid_lft 42362sec preferred_lft 42362sec 2023-11-24T07:56:45.008 INFO:teuthology.orchestra.run.smithi059.stdout: inet6 fe80::ec4:7aff:febd:192a/64 scope link 2023-11-24T07:56:45.008 INFO:teuthology.orchestra.run.smithi059.stdout: valid_lft forever preferred_lft forever 2023-11-24T07:56:45.008 INFO:teuthology.orchestra.run.smithi059.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-11-24T07:56:45.008 INFO:teuthology.orchestra.run.smithi059.stdout: link/ether 0c:c4:7a:bd:19:2b brd ff:ff:ff:ff:ff:ff 2023-11-24T07:56:45.008 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2023-11-24T07:56:45.008 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T07:56:45.009 DEBUG:teuthology.orchestra.run.smithi059:> (cd / && exec bash -c ' 2023-11-24T07:56:45.009 DEBUG:teuthology.orchestra.run.smithi059:> set -e 2023-11-24T07:56:45.009 DEBUG:teuthology.orchestra.run.smithi059:> sudo ip link add name ceph-brx type bridge 2023-11-24T07:56:45.009 DEBUG:teuthology.orchestra.run.smithi059:> sudo ip addr flush dev ceph-brx 2023-11-24T07:56:45.009 DEBUG:teuthology.orchestra.run.smithi059:> sudo ip link set ceph-brx up 2023-11-24T07:56:45.009 DEBUG:teuthology.orchestra.run.smithi059:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2023-11-24T07:56:45.009 DEBUG:teuthology.orchestra.run.smithi059:> ') 2023-11-24T07:56:45.179 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T07:56:45.179 DEBUG:teuthology.orchestra.run.smithi059:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-11-24T07:56:45.207 INFO:teuthology.orchestra.run.smithi059.stdout:1 2023-11-24T07:56:45.208 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T07:56:45.208 DEBUG:teuthology.orchestra.run.smithi059:> route 2023-11-24T07:56:45.276 INFO:teuthology.orchestra.run.smithi059.stdout:Kernel IP routing table 2023-11-24T07:56:45.276 INFO:teuthology.orchestra.run.smithi059.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-11-24T07:56:45.276 INFO:teuthology.orchestra.run.smithi059.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2023-11-24T07:56:45.276 INFO:teuthology.orchestra.run.smithi059.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2023-11-24T07:56:45.276 INFO:teuthology.orchestra.run.smithi059.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-11-24T07:56:45.276 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T07:56:45.277 DEBUG:teuthology.orchestra.run.smithi059:> (cd / && exec bash -c ' 2023-11-24T07:56:45.277 DEBUG:teuthology.orchestra.run.smithi059:> set -e 2023-11-24T07:56:45.277 DEBUG:teuthology.orchestra.run.smithi059:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2023-11-24T07:56:45.277 DEBUG:teuthology.orchestra.run.smithi059:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2023-11-24T07:56:45.277 DEBUG:teuthology.orchestra.run.smithi059:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2023-11-24T07:56:45.277 DEBUG:teuthology.orchestra.run.smithi059:> ') 2023-11-24T07:56:45.561 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T07:56:45.561 DEBUG:teuthology.orchestra.run.smithi059:> ip netns list 2023-11-24T07:56:45.579 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T07:56:45.579 DEBUG:teuthology.orchestra.run.smithi059:> ip netns list-id 2023-11-24T07:56:45.635 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T07:56:45.635 DEBUG:teuthology.orchestra.run.smithi059:> (cd / && exec bash -c ' 2023-11-24T07:56:45.636 DEBUG:teuthology.orchestra.run.smithi059:> set -e 2023-11-24T07:56:45.636 DEBUG:teuthology.orchestra.run.smithi059:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.0 2023-11-24T07:56:45.636 DEBUG:teuthology.orchestra.run.smithi059:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.0 0 2023-11-24T07:56:45.636 DEBUG:teuthology.orchestra.run.smithi059:> ') 2023-11-24T07:56:45.730 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.0' with 192.168.0.1/16 2023-11-24T07:56:45.731 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T07:56:45.731 DEBUG:teuthology.orchestra.run.smithi059:> (cd / && exec bash -c ' 2023-11-24T07:56:45.731 DEBUG:teuthology.orchestra.run.smithi059:> set -e 2023-11-24T07:56:45.731 DEBUG:teuthology.orchestra.run.smithi059:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.0 type veth peer name brx.0 2023-11-24T07:56:45.731 DEBUG:teuthology.orchestra.run.smithi059:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2023-11-24T07:56:45.731 DEBUG:teuthology.orchestra.run.smithi059:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set veth0 up 2023-11-24T07:56:45.731 DEBUG:teuthology.orchestra.run.smithi059:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set lo up 2023-11-24T07:56:45.731 DEBUG:teuthology.orchestra.run.smithi059:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip route add default via 192.168.255.254 2023-11-24T07:56:45.731 DEBUG:teuthology.orchestra.run.smithi059:> ') 2023-11-24T07:56:45.915 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T07:56:45.915 DEBUG:teuthology.orchestra.run.smithi059:> (cd / && exec bash -c ' 2023-11-24T07:56:45.915 DEBUG:teuthology.orchestra.run.smithi059:> set -e 2023-11-24T07:56:45.915 DEBUG:teuthology.orchestra.run.smithi059:> sudo ip link set brx.0 up 2023-11-24T07:56:45.915 DEBUG:teuthology.orchestra.run.smithi059:> sudo ip link set dev brx.0 master ceph-brx 2023-11-24T07:56:45.915 DEBUG:teuthology.orchestra.run.smithi059:> ') 2023-11-24T07:56:46.024 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-11-24T07:56:46.024 DEBUG:teuthology.orchestra.run.smithi059:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2023-11-24T07:56:46.043 INFO:teuthology.orchestra.run.smithi059.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2023-11-24T07:56:46.044 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-11-24T07:56:46.044 DEBUG:teuthology.orchestra.run.smithi059:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2023-11-24T07:56:46.100 INFO:tasks.cephfs.kernel_mount:mounting using device: 0@.cephfs=/ 2023-11-24T07:56:46.100 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T07:56:46.100 DEBUG:teuthology.orchestra.run.smithi059:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 0@.cephfs=/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-11-24T07:56:46.200 INFO:teuthology.orchestra.run.smithi059.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-11-24T07:56:46.200 INFO:teuthology.orchestra.run.smithi059.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-11-24T07:56:46.322 INFO:teuthology.orchestra.run.smithi059.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-11-24T07:56:46.322 INFO:teuthology.orchestra.run.smithi059.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-11-24T07:56:46.322 INFO:teuthology.orchestra.run.smithi059.stdout:mount.ceph: resolved to: "172.21.15.59:3300,172.21.15.76:3300,172.21.15.76:3301" 2023-11-24T07:56:46.322 INFO:teuthology.orchestra.run.smithi059.stdout:mount.ceph: trying mount with new device syntax: 0@9f0e8741-b8f9-48d5-a2c3-09f45903303d.cephfs=/ 2023-11-24T07:56:46.322 INFO:teuthology.orchestra.run.smithi059.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,ms_mode=prefer-crc,key=0,mon_addr=172.21.15.59:3300/172.21.15.76:3300/172.21.15.76:3301" will pass to kernel 2023-11-24T07:56:46.322 INFO:teuthology.orchestra.run.smithi059.stdout:mount.ceph: v2 mount support check returned 2 2023-11-24T07:56:46.322 INFO:teuthology.orchestra.run.smithi059.stdout:mount.ceph: kernel does not support v2 syntax 2023-11-24T07:56:46.323 INFO:teuthology.orchestra.run.smithi059.stdout:mount.ceph: switching to using v1 address with old syntax 2023-11-24T07:56:46.379 INFO:teuthology.orchestra.run.smithi059.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-11-24T07:56:46.379 INFO:teuthology.orchestra.run.smithi059.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-11-24T07:56:46.379 INFO:teuthology.orchestra.run.smithi059.stdout:mount.ceph: resolved to: "172.21.15.59:3300,172.21.15.76:3300,172.21.15.76:3301" 2023-11-24T07:56:46.379 INFO:teuthology.orchestra.run.smithi059.stdout:mount.ceph: trying mount with new device syntax: 0@9f0e8741-b8f9-48d5-a2c3-09f45903303d.cephfs=/ 2023-11-24T07:56:46.379 INFO:teuthology.orchestra.run.smithi059.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,ms_mode=prefer-crc,key=0,mon_addr=172.21.15.59:3300/172.21.15.76:3300/172.21.15.76:3301" will pass to kernel 2023-11-24T07:56:46.379 INFO:teuthology.orchestra.run.smithi059.stdout:mount.ceph: v2 mount support check returned 2 2023-11-24T07:56:46.379 INFO:teuthology.orchestra.run.smithi059.stdout:mount.ceph: kernel does not support v2 syntax 2023-11-24T07:56:46.379 INFO:teuthology.orchestra.run.smithi059.stdout:mount.ceph: switching to using v1 address with old syntax 2023-11-24T07:56:46.379 INFO:teuthology.orchestra.run.smithi059.stdout:mount.ceph: resolved to: "172.21.15.59:6789,172.21.15.76:6789,172.21.15.76:6790" 2023-11-24T07:56:46.380 INFO:teuthology.orchestra.run.smithi059.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.59:6789,172.21.15.76:6789,172.21.15.76:6790:/ 2023-11-24T07:56:46.380 INFO:teuthology.orchestra.run.smithi059.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,key=0,mds_namespace=cephfs,fsid=9f0e8741-b8f9-48d5-a2c3-09f45903303d" will pass to kernel 2023-11-24T07:56:46.385 INFO:tasks.cephfs.kernel_mount:mount command passed 2023-11-24T07:56:46.385 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T07:56:46.386 DEBUG:teuthology.orchestra.run.smithi059:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2023-11-24T07:56:46.414 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T07:56:46.414 DEBUG:teuthology.orchestra.run.smithi059:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-11-24T07:56:46.502 INFO:teuthology.orchestra.run.smithi059.stdout:client4671 2023-11-24T07:56:46.502 DEBUG:teuthology.orchestra.run.smithi059:> set -ex 2023-11-24T07:56:46.503 DEBUG:teuthology.orchestra.run.smithi059:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-24T07:56:46.520 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T07:56:46.520 DEBUG:teuthology.orchestra.run.smithi059:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-11-24T07:56:46.608 INFO:teuthology.orchestra.run.smithi059.stdout:client4671 2023-11-24T07:56:46.609 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T07:56:46.609 DEBUG:teuthology.orchestra.run.smithi059:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-11-24T07:56:46.657 INFO:teuthology.orchestra.run.smithi059.stdout:client4671 2023-11-24T07:56:46.657 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T07:56:46.657 DEBUG:teuthology.orchestra.run.smithi059:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'sudo dd if=/sys/kernel/debug/ceph/9f0e8741-b8f9-48d5-a2c3-09f45903303d.client4671/status') 2023-11-24T07:56:46.711 INFO:teuthology.orchestra.run.smithi059.stdout:instance: client.4671 (3)192.168.0.1:0/1967207726 2023-11-24T07:56:46.712 INFO:teuthology.orchestra.run.smithi059.stdout:blocklisted: false 2023-11-24T07:56:46.712 INFO:teuthology.orchestra.run.smithi059.stderr:0+1 records in 2023-11-24T07:56:46.712 INFO:teuthology.orchestra.run.smithi059.stderr:0+1 records out 2023-11-24T07:56:46.712 INFO:teuthology.orchestra.run.smithi059.stderr:69 bytes copied, 2.5194e-05 s, 2.7 MB/s 2023-11-24T07:56:46.713 INFO:tasks.kclient:client.2 config is {'syntax': 'v2'} 2023-11-24T07:56:46.714 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2023-11-24T07:56:46.714 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.2 2023-11-24T07:56:46.714 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-11-24T07:56:46.838 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:46.837+0000 7fb4e9571700 1 Processor -- start 2023-11-24T07:56:46.839 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:46.837+0000 7fb4e9571700 1 -- start start 2023-11-24T07:56:46.841 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:46.837+0000 7fb4e9571700 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fb4e4143ae0 0x7fb4e40b51b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:56:46.842 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:46.837+0000 7fb4e9571700 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb4e4143ec0 con 0x7fb4e40722c0 2023-11-24T07:56:46.842 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:46.837+0000 7fb4e9571700 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb4e40b3ae0 con 0x7fb4e41440f0 2023-11-24T07:56:46.842 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:46.837+0000 7fb4e9571700 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb4e40b40a0 con 0x7fb4e4143ae0 2023-11-24T07:56:46.842 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:46.838+0000 7fb4e37fe700 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7fb4e40722c0 0x7fb4e406e510 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:54026/0 (socket says 172.21.15.59:54026) 2023-11-24T07:56:46.842 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:46.838+0000 7fb4e37fe700 1 -- 172.21.15.59:0/1707376423 learned_addr learned my addr 172.21.15.59:0/1707376423 (peer_addr_for_me v1:172.21.15.59:0/0) 2023-11-24T07:56:46.842 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:46.838+0000 7fb4e2ffd700 1 --2- 172.21.15.59:0/1707376423 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fb4e4143ae0 0x7fb4e40b51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:56:46.842 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:46.838+0000 7fb4e1ffb700 1 -- 172.21.15.59:0/1707376423 <== mon.0 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3447607134 0 0) 0x7fb4e4143ec0 con 0x7fb4e40722c0 2023-11-24T07:56:46.843 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:46.838+0000 7fb4e1ffb700 1 -- 172.21.15.59:0/1707376423 --> v1:172.21.15.59:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb4cc0035b0 con 0x7fb4e40722c0 2023-11-24T07:56:46.843 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:46.838+0000 7fb4e1ffb700 1 -- 172.21.15.59:0/1707376423 <== mon.1 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2101764396 0 0) 0x7fb4e40b3ae0 con 0x7fb4e41440f0 2023-11-24T07:56:46.843 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:46.838+0000 7fb4e1ffb700 1 -- 172.21.15.59:0/1707376423 --> v1:172.21.15.76:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb4e4143ec0 con 0x7fb4e41440f0 2023-11-24T07:56:46.843 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:46.838+0000 7fb4e2ffd700 1 -- 172.21.15.59:0/1707376423 >> v1:172.21.15.76:6789/0 conn(0x7fb4e41440f0 legacy=0x7fb4e4071bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:56:46.843 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:46.838+0000 7fb4e2ffd700 1 -- 172.21.15.59:0/1707376423 >> v1:172.21.15.59:6789/0 conn(0x7fb4e40722c0 legacy=0x7fb4e406e510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:56:46.843 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:46.838+0000 7fb4e2ffd700 1 -- 172.21.15.59:0/1707376423 --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb4e40b3dc0 con 0x7fb4e4143ae0 2023-11-24T07:56:46.843 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:46.838+0000 7fb4e2ffd700 1 --2- 172.21.15.59:0/1707376423 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fb4e4143ae0 0x7fb4e40b51b0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fb4d4009870 tx=0x7fb4d4009b80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=466348be26ed8fb8 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:56:46.843 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:46.838+0000 7fb4e1ffb700 1 -- 172.21.15.59:0/1707376423 <== mon.2 v2:172.21.15.76:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4d4011830 con 0x7fb4e4143ae0 2023-11-24T07:56:46.843 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:46.838+0000 7fb4e1ffb700 1 -- 172.21.15.59:0/1707376423 <== mon.2 v2:172.21.15.76:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb4d4011990 con 0x7fb4e4143ae0 2023-11-24T07:56:46.843 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:46.838+0000 7fb4e1ffb700 1 -- 172.21.15.59:0/1707376423 <== mon.2 v2:172.21.15.76:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4d4011d20 con 0x7fb4e4143ae0 2023-11-24T07:56:46.844 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:46.839+0000 7fb4e9571700 1 -- 172.21.15.59:0/1707376423 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fb4e4143ae0 msgr2=0x7fb4e40b51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:56:46.844 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:46.839+0000 7fb4e9571700 1 --2- 172.21.15.59:0/1707376423 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fb4e4143ae0 0x7fb4e40b51b0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fb4d4009870 tx=0x7fb4d4009b80 comp rx=0 tx=0).stop 2023-11-24T07:56:46.844 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:46.839+0000 7fb4e9571700 1 -- 172.21.15.59:0/1707376423 shutdown_connections 2023-11-24T07:56:46.844 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:46.839+0000 7fb4e9571700 1 --2- 172.21.15.59:0/1707376423 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fb4e4143ae0 0x7fb4e40b51b0 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:56:46.844 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:46.839+0000 7fb4e9571700 1 -- 172.21.15.59:0/1707376423 >> 172.21.15.59:0/1707376423 conn(0x7fb4e40b2740 msgr2=0x7fb4e40b2b20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:56:46.844 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:46.839+0000 7fb4e9571700 1 -- 172.21.15.59:0/1707376423 shutdown_connections 2023-11-24T07:56:46.844 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:46.839+0000 7fb4e9571700 1 -- 172.21.15.59:0/1707376423 wait complete. 2023-11-24T07:56:46.847 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:46.840+0000 7fb4e9571700 1 Processor -- start 2023-11-24T07:56:46.847 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:46.841+0000 7fb4e9571700 1 -- start start 2023-11-24T07:56:46.848 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:46.841+0000 7fb4e9571700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb4e40722c0 0x7fb4e4150360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:56:46.848 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:46.841+0000 7fb4e9571700 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fb4e4143ae0 0x7fb4e415dc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:56:46.848 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:46.841+0000 7fb4e9571700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb4e41440f0 0x7fb4e4137410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:56:46.848 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:46.841+0000 7fb4e9571700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4e415e6a0 con 0x7fb4e41440f0 2023-11-24T07:56:46.848 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:46.841+0000 7fb4e9571700 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4e415e980 con 0x7fb4e40722c0 2023-11-24T07:56:46.848 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:46.841+0000 7fb4e9571700 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb4e415ec60 con 0x7fb4e4143ae0 2023-11-24T07:56:46.848 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:46.841+0000 7fb4e37fe700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb4e41440f0 0x7fb4e4137410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:56:46.848 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:46.841+0000 7fb4e37fe700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb4e41440f0 0x7fb4e4137410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:50560/0 (socket says 172.21.15.59:50560) 2023-11-24T07:56:46.848 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:46.841+0000 7fb4e37fe700 1 -- 172.21.15.59:0/1402446670 learned_addr learned my addr 172.21.15.59:0/1402446670 (peer_addr_for_me v2:172.21.15.59:0/0) 2023-11-24T07:56:46.849 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:46.842+0000 7fb4e27fc700 1 --2- 172.21.15.59:0/1402446670 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fb4e4143ae0 0x7fb4e415dc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:56:46.849 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:46.842+0000 7fb4e2ffd700 1 --2- 172.21.15.59:0/1402446670 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb4e40722c0 0x7fb4e4150360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:56:46.849 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:46.842+0000 7fb4e37fe700 1 -- 172.21.15.59:0/1402446670 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fb4e4143ae0 msgr2=0x7fb4e415dc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:56:46.849 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:46.842+0000 7fb4e37fe700 1 --2- 172.21.15.59:0/1402446670 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fb4e4143ae0 0x7fb4e415dc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:56:46.849 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:46.842+0000 7fb4e37fe700 1 -- 172.21.15.59:0/1402446670 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb4e40722c0 msgr2=0x7fb4e4150360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:56:46.849 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:46.842+0000 7fb4e37fe700 1 --2- 172.21.15.59:0/1402446670 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb4e40722c0 0x7fb4e4150360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:56:46.849 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:46.842+0000 7fb4e37fe700 1 -- 172.21.15.59:0/1402446670 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb4d40094a0 con 0x7fb4e41440f0 2023-11-24T07:56:46.849 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:46.842+0000 7fb4e37fe700 1 --2- 172.21.15.59:0/1402446670 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb4e41440f0 0x7fb4e4137410 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fb4dc00cb70 tx=0x7fb4dc00cf30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:56:46.849 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:46.842+0000 7fb4cb7fe700 1 -- 172.21.15.59:0/1402446670 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4dc00aaf0 con 0x7fb4e41440f0 2023-11-24T07:56:46.849 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:46.842+0000 7fb4cb7fe700 1 -- 172.21.15.59:0/1402446670 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb4dc00ac50 con 0x7fb4e41440f0 2023-11-24T07:56:46.850 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:46.842+0000 7fb4e9571700 1 -- 172.21.15.59:0/1402446670 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb4e415ef40 con 0x7fb4e41440f0 2023-11-24T07:56:46.850 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:46.842+0000 7fb4cb7fe700 1 -- 172.21.15.59:0/1402446670 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4dc012400 con 0x7fb4e41440f0 2023-11-24T07:56:46.850 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:46.842+0000 7fb4e9571700 1 -- 172.21.15.59:0/1402446670 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb4e413dc80 con 0x7fb4e41440f0 2023-11-24T07:56:46.850 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:46.843+0000 7fb4e9571700 1 -- 172.21.15.59:0/1402446670 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb4e4068670 con 0x7fb4e41440f0 2023-11-24T07:56:46.850 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:46.844+0000 7fb4cb7fe700 1 -- 172.21.15.59:0/1402446670 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fb4dc01b3b0 con 0x7fb4e41440f0 2023-11-24T07:56:46.850 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:46.844+0000 7fb4cb7fe700 1 --2- 172.21.15.59:0/1402446670 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7fb4d006aed0 0x7fb4d006d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:56:46.850 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:46.844+0000 7fb4cb7fe700 1 -- 172.21.15.59:0/1402446670 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fb4dc0893b0 con 0x7fb4e41440f0 2023-11-24T07:56:46.850 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:46.844+0000 7fb4e2ffd700 1 --2- 172.21.15.59:0/1402446670 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7fb4d006aed0 0x7fb4d006d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:56:46.850 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:46.845+0000 7fb4e2ffd700 1 --2- 172.21.15.59:0/1402446670 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7fb4d006aed0 0x7fb4d006d390 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fb4d40097c0 tx=0x7fb4d401c040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:56:46.851 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:46.849+0000 7fb4cb7fe700 1 -- 172.21.15.59:0/1402446670 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fb4dc057f00 con 0x7fb4e41440f0 2023-11-24T07:56:47.100 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:47.098+0000 7fb4e9571700 1 -- 172.21.15.59:0/1402446670 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7fb4e40cf8f0 con 0x7fb4e41440f0 2023-11-24T07:56:47.101 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:47.098+0000 7fb4cb7fe700 1 -- 172.21.15.59:0/1402446670 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v8) v1 ==== 53+0+73 (secure 0 0 0) 0x7fb4dc057a90 con 0x7fb4e41440f0 2023-11-24T07:56:47.101 INFO:teuthology.orchestra.run.smithi059.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-11-24T07:56:47.105 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:47.101+0000 7fb4e9571700 1 -- 172.21.15.59:0/1402446670 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7fb4d006aed0 msgr2=0x7fb4d006d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:56:47.106 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:47.101+0000 7fb4e9571700 1 --2- 172.21.15.59:0/1402446670 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7fb4d006aed0 0x7fb4d006d390 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fb4d40097c0 tx=0x7fb4d401c040 comp rx=0 tx=0).stop 2023-11-24T07:56:47.106 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:47.101+0000 7fb4e9571700 1 -- 172.21.15.59:0/1402446670 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb4e41440f0 msgr2=0x7fb4e4137410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:56:47.106 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:47.101+0000 7fb4e9571700 1 --2- 172.21.15.59:0/1402446670 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb4e41440f0 0x7fb4e4137410 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fb4dc00cb70 tx=0x7fb4dc00cf30 comp rx=0 tx=0).stop 2023-11-24T07:56:47.106 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:47.101+0000 7fb4e9571700 1 -- 172.21.15.59:0/1402446670 shutdown_connections 2023-11-24T07:56:47.106 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:47.101+0000 7fb4e9571700 1 --2- 172.21.15.59:0/1402446670 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7fb4d006aed0 0x7fb4d006d390 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:56:47.106 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:47.101+0000 7fb4e9571700 1 --2- 172.21.15.59:0/1402446670 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb4e41440f0 0x7fb4e4137410 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:56:47.106 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:47.101+0000 7fb4e9571700 1 --2- 172.21.15.59:0/1402446670 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb4e40722c0 0x7fb4e4150360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:56:47.106 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:47.101+0000 7fb4e9571700 1 --2- 172.21.15.59:0/1402446670 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fb4e4143ae0 0x7fb4e415dc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:56:47.106 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:47.101+0000 7fb4e9571700 1 -- 172.21.15.59:0/1402446670 >> 172.21.15.59:0/1402446670 conn(0x7fb4e40b2740 msgr2=0x7fb4e406f4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:56:47.106 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:47.101+0000 7fb4e9571700 1 -- 172.21.15.59:0/1402446670 shutdown_connections 2023-11-24T07:56:47.106 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:47.101+0000 7fb4e9571700 1 -- 172.21.15.59:0/1402446670 wait complete. 2023-11-24T07:56:47.115 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2023-11-24T07:56:47.115 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-11-24T07:56:47.115 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi059.front.sepia.ceph.com 2023-11-24T07:56:47.115 INFO:tasks.cephfs.mount:self.client.name = client.2 2023-11-24T07:56:47.115 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.2 2023-11-24T07:56:47.115 INFO:tasks.cephfs.mount:self.cephfs_name = None 2023-11-24T07:56:47.116 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-11-24T07:56:47.116 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-11-24T07:56:47.116 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.2' 2023-11-24T07:56:47.116 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T07:56:47.116 DEBUG:teuthology.orchestra.run.smithi059:> ip addr 2023-11-24T07:56:47.134 INFO:teuthology.orchestra.run.smithi059.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-11-24T07:56:47.134 INFO:teuthology.orchestra.run.smithi059.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-11-24T07:56:47.134 INFO:teuthology.orchestra.run.smithi059.stdout: inet 127.0.0.1/8 scope host lo 2023-11-24T07:56:47.134 INFO:teuthology.orchestra.run.smithi059.stdout: valid_lft forever preferred_lft forever 2023-11-24T07:56:47.134 INFO:teuthology.orchestra.run.smithi059.stdout: inet6 ::1/128 scope host 2023-11-24T07:56:47.134 INFO:teuthology.orchestra.run.smithi059.stdout: valid_lft forever preferred_lft forever 2023-11-24T07:56:47.134 INFO:teuthology.orchestra.run.smithi059.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-11-24T07:56:47.135 INFO:teuthology.orchestra.run.smithi059.stdout: link/ether 0c:c4:7a:6b:fe:ca brd ff:ff:ff:ff:ff:ff 2023-11-24T07:56:47.135 INFO:teuthology.orchestra.run.smithi059.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-11-24T07:56:47.135 INFO:teuthology.orchestra.run.smithi059.stdout: link/ether 0c:c4:7a:6b:fe:cb brd ff:ff:ff:ff:ff:ff 2023-11-24T07:56:47.135 INFO:teuthology.orchestra.run.smithi059.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2023-11-24T07:56:47.135 INFO:teuthology.orchestra.run.smithi059.stdout: link/ether 0c:c4:7a:bd:19:2a brd ff:ff:ff:ff:ff:ff 2023-11-24T07:56:47.135 INFO:teuthology.orchestra.run.smithi059.stdout: inet 172.21.15.59/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2023-11-24T07:56:47.135 INFO:teuthology.orchestra.run.smithi059.stdout: valid_lft 42360sec preferred_lft 42360sec 2023-11-24T07:56:47.135 INFO:teuthology.orchestra.run.smithi059.stdout: inet6 fe80::ec4:7aff:febd:192a/64 scope link 2023-11-24T07:56:47.135 INFO:teuthology.orchestra.run.smithi059.stdout: valid_lft forever preferred_lft forever 2023-11-24T07:56:47.135 INFO:teuthology.orchestra.run.smithi059.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-11-24T07:56:47.135 INFO:teuthology.orchestra.run.smithi059.stdout: link/ether 0c:c4:7a:bd:19:2b brd ff:ff:ff:ff:ff:ff 2023-11-24T07:56:47.135 INFO:teuthology.orchestra.run.smithi059.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-11-24T07:56:47.135 INFO:teuthology.orchestra.run.smithi059.stdout: link/ether 92:6a:7d:e0:5d:9b brd ff:ff:ff:ff:ff:ff 2023-11-24T07:56:47.136 INFO:teuthology.orchestra.run.smithi059.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-11-24T07:56:47.136 INFO:teuthology.orchestra.run.smithi059.stdout: valid_lft forever preferred_lft forever 2023-11-24T07:56:47.136 INFO:teuthology.orchestra.run.smithi059.stdout: inet6 fe80::64c1:62ff:feb9:5b59/64 scope link 2023-11-24T07:56:47.136 INFO:teuthology.orchestra.run.smithi059.stdout: valid_lft forever preferred_lft forever 2023-11-24T07:56:47.136 INFO:teuthology.orchestra.run.smithi059.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-11-24T07:56:47.136 INFO:teuthology.orchestra.run.smithi059.stdout: link/ether 92:6a:7d:e0:5d:9b brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2023-11-24T07:56:47.136 INFO:teuthology.orchestra.run.smithi059.stdout: inet6 fe80::906a:7dff:fee0:5d9b/64 scope link tentative 2023-11-24T07:56:47.136 INFO:teuthology.orchestra.run.smithi059.stdout: valid_lft forever preferred_lft forever 2023-11-24T07:56:47.136 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T07:56:47.137 DEBUG:teuthology.orchestra.run.smithi059:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-11-24T07:56:47.201 INFO:teuthology.orchestra.run.smithi059.stdout:1 2023-11-24T07:56:47.202 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T07:56:47.202 DEBUG:teuthology.orchestra.run.smithi059:> route 2023-11-24T07:56:47.261 INFO:teuthology.orchestra.run.smithi059.stdout:Kernel IP routing table 2023-11-24T07:56:47.261 INFO:teuthology.orchestra.run.smithi059.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-11-24T07:56:47.261 INFO:teuthology.orchestra.run.smithi059.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2023-11-24T07:56:47.261 INFO:teuthology.orchestra.run.smithi059.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2023-11-24T07:56:47.262 INFO:teuthology.orchestra.run.smithi059.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-11-24T07:56:47.262 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T07:56:47.262 DEBUG:teuthology.orchestra.run.smithi059:> (cd / && exec bash -c ' 2023-11-24T07:56:47.262 DEBUG:teuthology.orchestra.run.smithi059:> set -e 2023-11-24T07:56:47.262 DEBUG:teuthology.orchestra.run.smithi059:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2023-11-24T07:56:47.262 DEBUG:teuthology.orchestra.run.smithi059:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2023-11-24T07:56:47.263 DEBUG:teuthology.orchestra.run.smithi059:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2023-11-24T07:56:47.263 DEBUG:teuthology.orchestra.run.smithi059:> ') 2023-11-24T07:56:47.369 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T07:56:47.370 DEBUG:teuthology.orchestra.run.smithi059:> ip netns list 2023-11-24T07:56:47.387 INFO:teuthology.orchestra.run.smithi059.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-11-24T07:56:47.388 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T07:56:47.388 DEBUG:teuthology.orchestra.run.smithi059:> ip netns list-id 2023-11-24T07:56:47.445 INFO:teuthology.orchestra.run.smithi059.stdout:nsid 0 (iproute2 netns name: ceph-ns--home-ubuntu-cephtest-mnt.0) 2023-11-24T07:56:47.445 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T07:56:47.445 DEBUG:teuthology.orchestra.run.smithi059:> (cd / && exec bash -c ' 2023-11-24T07:56:47.446 DEBUG:teuthology.orchestra.run.smithi059:> set -e 2023-11-24T07:56:47.446 DEBUG:teuthology.orchestra.run.smithi059:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.2 2023-11-24T07:56:47.446 DEBUG:teuthology.orchestra.run.smithi059:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.2 1 2023-11-24T07:56:47.446 DEBUG:teuthology.orchestra.run.smithi059:> ') 2023-11-24T07:56:47.540 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T07:56:47.540 DEBUG:teuthology.orchestra.run.smithi059:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr 2023-11-24T07:56:47.582 INFO:teuthology.orchestra.run.smithi059.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-11-24T07:56:47.582 INFO:teuthology.orchestra.run.smithi059.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-11-24T07:56:47.582 INFO:teuthology.orchestra.run.smithi059.stdout: inet 127.0.0.1/8 scope host lo 2023-11-24T07:56:47.582 INFO:teuthology.orchestra.run.smithi059.stdout: valid_lft forever preferred_lft forever 2023-11-24T07:56:47.583 INFO:teuthology.orchestra.run.smithi059.stdout: inet6 ::1/128 scope host 2023-11-24T07:56:47.583 INFO:teuthology.orchestra.run.smithi059.stdout: valid_lft forever preferred_lft forever 2023-11-24T07:56:47.583 INFO:teuthology.orchestra.run.smithi059.stdout:2: veth0@if7: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-11-24T07:56:47.583 INFO:teuthology.orchestra.run.smithi059.stdout: link/ether 8e:e7:4f:6d:19:c3 brd ff:ff:ff:ff:ff:ff link-netnsid 0 2023-11-24T07:56:47.583 INFO:teuthology.orchestra.run.smithi059.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2023-11-24T07:56:47.583 INFO:teuthology.orchestra.run.smithi059.stdout: valid_lft forever preferred_lft forever 2023-11-24T07:56:47.583 INFO:teuthology.orchestra.run.smithi059.stdout: inet6 fe80::8ce7:4fff:fe6d:19c3/64 scope link tentative 2023-11-24T07:56:47.583 INFO:teuthology.orchestra.run.smithi059.stdout: valid_lft forever preferred_lft forever 2023-11-24T07:56:47.595 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T07:56:47.595 DEBUG:teuthology.orchestra.run.smithi059:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr 2023-11-24T07:56:47.671 INFO:teuthology.orchestra.run.smithi059.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-11-24T07:56:47.671 INFO:teuthology.orchestra.run.smithi059.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-11-24T07:56:47.672 INFO:teuthology.orchestra.run.smithi059.stdout: inet 127.0.0.1/8 scope host lo 2023-11-24T07:56:47.672 INFO:teuthology.orchestra.run.smithi059.stdout: valid_lft forever preferred_lft forever 2023-11-24T07:56:47.672 INFO:teuthology.orchestra.run.smithi059.stdout: inet6 ::1/128 scope host 2023-11-24T07:56:47.672 INFO:teuthology.orchestra.run.smithi059.stdout: valid_lft forever preferred_lft forever 2023-11-24T07:56:47.672 INFO:teuthology.orchestra.run.smithi059.stdout:2: veth0@if7: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-11-24T07:56:47.672 INFO:teuthology.orchestra.run.smithi059.stdout: link/ether 8e:e7:4f:6d:19:c3 brd ff:ff:ff:ff:ff:ff link-netnsid 0 2023-11-24T07:56:47.672 INFO:teuthology.orchestra.run.smithi059.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2023-11-24T07:56:47.672 INFO:teuthology.orchestra.run.smithi059.stdout: valid_lft forever preferred_lft forever 2023-11-24T07:56:47.672 INFO:teuthology.orchestra.run.smithi059.stdout: inet6 fe80::8ce7:4fff:fe6d:19c3/64 scope link tentative 2023-11-24T07:56:47.672 INFO:teuthology.orchestra.run.smithi059.stdout: valid_lft forever preferred_lft forever 2023-11-24T07:56:47.687 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.2' with 192.168.0.2/16 2023-11-24T07:56:47.687 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T07:56:47.687 DEBUG:teuthology.orchestra.run.smithi059:> (cd / && exec bash -c ' 2023-11-24T07:56:47.687 DEBUG:teuthology.orchestra.run.smithi059:> set -e 2023-11-24T07:56:47.687 DEBUG:teuthology.orchestra.run.smithi059:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.2 type veth peer name brx.1 2023-11-24T07:56:47.687 DEBUG:teuthology.orchestra.run.smithi059:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip addr add 192.168.0.2/16 brd 192.168.255.255 dev veth0 2023-11-24T07:56:47.687 DEBUG:teuthology.orchestra.run.smithi059:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip link set veth0 up 2023-11-24T07:56:47.687 DEBUG:teuthology.orchestra.run.smithi059:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip link set lo up 2023-11-24T07:56:47.687 DEBUG:teuthology.orchestra.run.smithi059:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip route add default via 192.168.255.254 2023-11-24T07:56:47.687 DEBUG:teuthology.orchestra.run.smithi059:> ') 2023-11-24T07:56:47.937 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T07:56:47.937 DEBUG:teuthology.orchestra.run.smithi059:> (cd / && exec bash -c ' 2023-11-24T07:56:47.937 DEBUG:teuthology.orchestra.run.smithi059:> set -e 2023-11-24T07:56:47.937 DEBUG:teuthology.orchestra.run.smithi059:> sudo ip link set brx.1 up 2023-11-24T07:56:47.937 DEBUG:teuthology.orchestra.run.smithi059:> sudo ip link set dev brx.1 master ceph-brx 2023-11-24T07:56:47.938 DEBUG:teuthology.orchestra.run.smithi059:> ') 2023-11-24T07:56:48.058 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-11-24T07:56:48.058 DEBUG:teuthology.orchestra.run.smithi059:> mkdir -p -v /home/ubuntu/cephtest/mnt.2 2023-11-24T07:56:48.091 INFO:teuthology.orchestra.run.smithi059.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.2' 2023-11-24T07:56:48.091 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-11-24T07:56:48.091 DEBUG:teuthology.orchestra.run.smithi059:> chmod 0000 /home/ubuntu/cephtest/mnt.2 2023-11-24T07:56:48.148 INFO:tasks.cephfs.kernel_mount:mounting using device: 2@.cephfs=/ 2023-11-24T07:56:48.148 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T07:56:48.148 DEBUG:teuthology.orchestra.run.smithi059:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 2@.cephfs=/ /home/ubuntu/cephtest/mnt.2 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-11-24T07:56:48.235 INFO:teuthology.orchestra.run.smithi059.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-11-24T07:56:48.235 INFO:teuthology.orchestra.run.smithi059.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-11-24T07:56:48.266 INFO:teuthology.orchestra.run.smithi059.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-11-24T07:56:48.266 INFO:teuthology.orchestra.run.smithi059.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-11-24T07:56:48.266 INFO:teuthology.orchestra.run.smithi059.stdout:mount.ceph: resolved to: "172.21.15.59:3300,172.21.15.76:3300,172.21.15.76:3301" 2023-11-24T07:56:48.266 INFO:teuthology.orchestra.run.smithi059.stdout:mount.ceph: trying mount with new device syntax: 2@9f0e8741-b8f9-48d5-a2c3-09f45903303d.cephfs=/ 2023-11-24T07:56:48.266 INFO:teuthology.orchestra.run.smithi059.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=2,ms_mode=prefer-crc,key=2,mon_addr=172.21.15.59:3300/172.21.15.76:3300/172.21.15.76:3301" will pass to kernel 2023-11-24T07:56:48.266 INFO:teuthology.orchestra.run.smithi059.stdout:mount.ceph: v2 mount support check returned 2 2023-11-24T07:56:48.266 INFO:teuthology.orchestra.run.smithi059.stdout:mount.ceph: kernel does not support v2 syntax 2023-11-24T07:56:48.267 INFO:teuthology.orchestra.run.smithi059.stdout:mount.ceph: switching to using v1 address with old syntax 2023-11-24T07:56:48.295 INFO:teuthology.orchestra.run.smithi059.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-11-24T07:56:48.297 INFO:teuthology.orchestra.run.smithi059.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-11-24T07:56:48.298 INFO:teuthology.orchestra.run.smithi059.stdout:mount.ceph: resolved to: "172.21.15.59:3300,172.21.15.76:3300,172.21.15.76:3301" 2023-11-24T07:56:48.298 INFO:teuthology.orchestra.run.smithi059.stdout:mount.ceph: trying mount with new device syntax: 2@9f0e8741-b8f9-48d5-a2c3-09f45903303d.cephfs=/ 2023-11-24T07:56:48.298 INFO:teuthology.orchestra.run.smithi059.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=2,ms_mode=prefer-crc,key=2,mon_addr=172.21.15.59:3300/172.21.15.76:3300/172.21.15.76:3301" will pass to kernel 2023-11-24T07:56:48.299 INFO:teuthology.orchestra.run.smithi059.stdout:mount.ceph: v2 mount support check returned 2 2023-11-24T07:56:48.299 INFO:teuthology.orchestra.run.smithi059.stdout:mount.ceph: kernel does not support v2 syntax 2023-11-24T07:56:48.300 INFO:teuthology.orchestra.run.smithi059.stdout:mount.ceph: switching to using v1 address with old syntax 2023-11-24T07:56:48.300 INFO:teuthology.orchestra.run.smithi059.stdout:mount.ceph: resolved to: "172.21.15.59:6789,172.21.15.76:6789,172.21.15.76:6790" 2023-11-24T07:56:48.300 INFO:teuthology.orchestra.run.smithi059.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.59:6789,172.21.15.76:6789,172.21.15.76:6790:/ 2023-11-24T07:56:48.300 INFO:teuthology.orchestra.run.smithi059.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=2,key=2,mds_namespace=cephfs,fsid=9f0e8741-b8f9-48d5-a2c3-09f45903303d" will pass to kernel 2023-11-24T07:56:48.302 INFO:tasks.cephfs.kernel_mount:mount command passed 2023-11-24T07:56:48.303 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T07:56:48.303 DEBUG:teuthology.orchestra.run.smithi059:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.2 2023-11-24T07:56:48.343 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T07:56:48.343 DEBUG:teuthology.orchestra.run.smithi059:> (cd /home/ubuntu/cephtest/mnt.2 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-11-24T07:56:48.416 INFO:teuthology.orchestra.run.smithi059.stdout:client4680 2023-11-24T07:56:48.417 DEBUG:teuthology.orchestra.run.smithi059:> set -ex 2023-11-24T07:56:48.417 DEBUG:teuthology.orchestra.run.smithi059:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-24T07:56:48.474 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T07:56:48.474 DEBUG:teuthology.orchestra.run.smithi059:> (cd /home/ubuntu/cephtest/mnt.2 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-11-24T07:56:48.547 INFO:teuthology.orchestra.run.smithi059.stdout:client4680 2023-11-24T07:56:48.547 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T07:56:48.547 DEBUG:teuthology.orchestra.run.smithi059:> (cd /home/ubuntu/cephtest/mnt.2 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-11-24T07:56:48.620 INFO:teuthology.orchestra.run.smithi059.stdout:client4680 2023-11-24T07:56:48.621 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T07:56:48.621 DEBUG:teuthology.orchestra.run.smithi059:> (cd /home/ubuntu/cephtest/mnt.2 && exec bash -c 'sudo dd if=/sys/kernel/debug/ceph/9f0e8741-b8f9-48d5-a2c3-09f45903303d.client4680/status') 2023-11-24T07:56:48.702 INFO:teuthology.orchestra.run.smithi059.stdout:instance: client.4680 (3)192.168.0.2:0/1262345806 2023-11-24T07:56:48.702 INFO:teuthology.orchestra.run.smithi059.stdout:blocklisted: false 2023-11-24T07:56:48.703 INFO:teuthology.orchestra.run.smithi059.stderr:0+1 records in 2023-11-24T07:56:48.703 INFO:teuthology.orchestra.run.smithi059.stderr:0+1 records out 2023-11-24T07:56:48.703 INFO:teuthology.orchestra.run.smithi059.stderr:69 bytes copied, 2.3655e-05 s, 2.9 MB/s 2023-11-24T07:56:48.703 INFO:tasks.kclient:client.1 config is {'syntax': 'v2'} 2023-11-24T07:56:48.704 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2023-11-24T07:56:48.704 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2023-11-24T07:56:48.704 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-11-24T07:56:48.868 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:48.866+0000 7ff763f01700 1 Processor -- start 2023-11-24T07:56:48.869 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:48.867+0000 7ff763f01700 1 -- start start 2023-11-24T07:56:48.871 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:48.867+0000 7ff763f01700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7ff75c0b4f20 0x7ff75c071d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:56:48.871 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:48.868+0000 7ff763f01700 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7ff75c0722d0 0x7ff75c0cdee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:56:48.871 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:48.868+0000 7ff763f01700 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff75c072760 con 0x7ff75c15abb0 2023-11-24T07:56:48.871 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:48.868+0000 7ff763f01700 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff75c06e550 con 0x7ff75c0b4f20 2023-11-24T07:56:48.871 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:48.868+0000 7ff763f01700 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff75c06f3b0 con 0x7ff75c0722d0 2023-11-24T07:56:48.871 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:48.868+0000 7ff76149c700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7ff75c0b4f20 0x7ff75c071d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:56:48.872 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:48.868+0000 7ff761c9d700 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7ff75c15abb0 0x7ff75c0b4810 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:54046/0 (socket says 172.21.15.59:54046) 2023-11-24T07:56:48.872 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:48.868+0000 7ff76149c700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7ff75c0b4f20 0x7ff75c071d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:50574/0 (socket says 172.21.15.59:50574) 2023-11-24T07:56:48.872 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:48.868+0000 7ff761c9d700 1 -- 172.21.15.59:0/420055387 learned_addr learned my addr 172.21.15.59:0/420055387 (peer_addr_for_me v1:172.21.15.59:0/0) 2023-11-24T07:56:48.872 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:48.868+0000 7ff76249e700 1 --2- 172.21.15.59:0/420055387 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7ff75c0722d0 0x7ff75c0cdee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:56:48.872 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:48.868+0000 7ff76149c700 1 -- 172.21.15.59:0/420055387 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7ff75c0722d0 msgr2=0x7ff75c0cdee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T07:56:48.872 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:48.868+0000 7ff76149c700 1 --2- 172.21.15.59:0/420055387 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7ff75c0722d0 0x7ff75c0cdee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:56:48.872 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:48.868+0000 7ff76149c700 1 -- 172.21.15.59:0/420055387 >> v1:172.21.15.59:6789/0 conn(0x7ff75c15abb0 legacy=0x7ff75c0b4810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:56:48.872 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:48.868+0000 7ff76149c700 1 -- 172.21.15.59:0/420055387 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff75c06e7a0 con 0x7ff75c0b4f20 2023-11-24T07:56:48.872 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:48.868+0000 7ff760c9b700 1 -- 172.21.15.59:0/420055387 <== mon.0 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2443560812 0 0) 0x7ff750001110 con 0x7ff75c15abb0 2023-11-24T07:56:48.872 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:48.868+0000 7ff76249e700 1 --2- 172.21.15.59:0/420055387 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7ff75c0722d0 0x7ff75c0cdee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T07:56:48.872 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:48.868+0000 7ff76149c700 1 --2- 172.21.15.59:0/420055387 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff75c0b4f20 0x7ff75c071d90 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7ff758009840 tx=0x7ff758009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9144fa844a24985b server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:56:48.872 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:48.868+0000 7ff760c9b700 1 -- 172.21.15.59:0/420055387 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff758011830 con 0x7ff75c0b4f20 2023-11-24T07:56:48.873 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:48.868+0000 7ff760c9b700 1 -- 172.21.15.59:0/420055387 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff758011990 con 0x7ff75c0b4f20 2023-11-24T07:56:48.873 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:48.868+0000 7ff760c9b700 1 -- 172.21.15.59:0/420055387 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff758011d20 con 0x7ff75c0b4f20 2023-11-24T07:56:48.873 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:48.869+0000 7ff763f01700 1 -- 172.21.15.59:0/420055387 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff75c0b4f20 msgr2=0x7ff75c071d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:56:48.873 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:48.869+0000 7ff763f01700 1 --2- 172.21.15.59:0/420055387 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff75c0b4f20 0x7ff75c071d90 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7ff758009840 tx=0x7ff758009b50 comp rx=0 tx=0).stop 2023-11-24T07:56:48.873 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:48.869+0000 7ff763f01700 1 -- 172.21.15.59:0/420055387 shutdown_connections 2023-11-24T07:56:48.873 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:48.869+0000 7ff763f01700 1 --2- 172.21.15.59:0/420055387 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7ff75c0722d0 0x7ff75c0cdee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:56:48.873 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:48.869+0000 7ff763f01700 1 --2- 172.21.15.59:0/420055387 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff75c0b4f20 0x7ff75c071d90 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:56:48.873 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:48.869+0000 7ff763f01700 1 -- 172.21.15.59:0/420055387 >> 172.21.15.59:0/420055387 conn(0x7ff75c0d0aa0 msgr2=0x7ff75c0d0eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:56:48.873 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:48.869+0000 7ff763f01700 1 -- 172.21.15.59:0/420055387 shutdown_connections 2023-11-24T07:56:48.873 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:48.869+0000 7ff763f01700 1 -- 172.21.15.59:0/420055387 wait complete. 2023-11-24T07:56:48.876 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:48.870+0000 7ff763f01700 1 Processor -- start 2023-11-24T07:56:48.876 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:48.870+0000 7ff763f01700 1 -- start start 2023-11-24T07:56:48.876 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:48.871+0000 7ff763f01700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff75c0722d0 0x7ff75c1f74f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:56:48.876 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:48.871+0000 7ff763f01700 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7ff75c0b4f20 0x7ff75c153580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:56:48.877 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:48.871+0000 7ff763f01700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff75c15abb0 0x7ff75c15cb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:56:48.877 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:48.871+0000 7ff763f01700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff75c0b0320 con 0x7ff75c15abb0 2023-11-24T07:56:48.877 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:48.871+0000 7ff763f01700 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff75c0ce6d0 con 0x7ff75c0722d0 2023-11-24T07:56:48.877 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:48.871+0000 7ff763f01700 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff75c1f7a30 con 0x7ff75c0b4f20 2023-11-24T07:56:48.877 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:48.871+0000 7ff76249e700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff75c15abb0 0x7ff75c15cb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:56:48.877 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:48.871+0000 7ff76249e700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff75c15abb0 0x7ff75c15cb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:50582/0 (socket says 172.21.15.59:50582) 2023-11-24T07:56:48.877 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:48.871+0000 7ff76249e700 1 -- 172.21.15.59:0/597621185 learned_addr learned my addr 172.21.15.59:0/597621185 (peer_addr_for_me v2:172.21.15.59:0/0) 2023-11-24T07:56:48.877 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:48.871+0000 7ff76149c700 1 --2- 172.21.15.59:0/597621185 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7ff75c0b4f20 0x7ff75c153580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:56:48.877 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:48.871+0000 7ff761c9d700 1 --2- 172.21.15.59:0/597621185 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff75c0722d0 0x7ff75c1f74f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:56:48.877 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:48.871+0000 7ff76249e700 1 -- 172.21.15.59:0/597621185 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7ff75c0b4f20 msgr2=0x7ff75c153580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:56:48.877 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:48.871+0000 7ff76249e700 1 --2- 172.21.15.59:0/597621185 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7ff75c0b4f20 0x7ff75c153580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:56:48.878 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:48.871+0000 7ff76249e700 1 -- 172.21.15.59:0/597621185 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff75c0722d0 msgr2=0x7ff75c1f74f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:56:48.878 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:48.871+0000 7ff76249e700 1 --2- 172.21.15.59:0/597621185 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff75c0722d0 0x7ff75c1f74f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:56:48.878 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:48.871+0000 7ff76249e700 1 -- 172.21.15.59:0/597621185 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff7580094a0 con 0x7ff75c15abb0 2023-11-24T07:56:48.878 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:48.872+0000 7ff76249e700 1 --2- 172.21.15.59:0/597621185 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff75c15abb0 0x7ff75c15cb80 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7ff75400cd50 tx=0x7ff75400a560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:56:48.878 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:48.872+0000 7ff74e7fc700 1 -- 172.21.15.59:0/597621185 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff75400cea0 con 0x7ff75c15abb0 2023-11-24T07:56:48.878 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:48.872+0000 7ff74e7fc700 1 -- 172.21.15.59:0/597621185 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff75400f480 con 0x7ff75c15abb0 2023-11-24T07:56:48.878 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:48.872+0000 7ff74e7fc700 1 -- 172.21.15.59:0/597621185 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff75400f810 con 0x7ff75c15abb0 2023-11-24T07:56:48.878 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:48.872+0000 7ff763f01700 1 -- 172.21.15.59:0/597621185 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff75c0b0040 con 0x7ff75c15abb0 2023-11-24T07:56:48.878 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:48.872+0000 7ff763f01700 1 -- 172.21.15.59:0/597621185 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff75c0ce360 con 0x7ff75c15abb0 2023-11-24T07:56:48.879 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:48.873+0000 7ff763f01700 1 -- 172.21.15.59:0/597621185 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff75c068670 con 0x7ff75c15abb0 2023-11-24T07:56:48.879 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:48.873+0000 7ff74e7fc700 1 -- 172.21.15.59:0/597621185 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7ff75400b040 con 0x7ff75c15abb0 2023-11-24T07:56:48.879 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:48.874+0000 7ff74e7fc700 1 --2- 172.21.15.59:0/597621185 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7ff74006aed0 0x7ff74006d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:56:48.879 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:48.874+0000 7ff74e7fc700 1 -- 172.21.15.59:0/597621185 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7ff754089dc0 con 0x7ff75c15abb0 2023-11-24T07:56:48.879 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:48.874+0000 7ff761c9d700 1 --2- 172.21.15.59:0/597621185 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7ff74006aed0 0x7ff74006d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:56:48.879 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:48.874+0000 7ff761c9d700 1 --2- 172.21.15.59:0/597621185 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7ff74006aed0 0x7ff74006d390 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7ff750000f60 tx=0x7ff750007a40 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:56:48.880 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:48.878+0000 7ff74e7fc700 1 -- 172.21.15.59:0/597621185 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7ff754054e70 con 0x7ff75c15abb0 2023-11-24T07:56:49.128 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:49.126+0000 7ff763f01700 1 -- 172.21.15.59:0/597621185 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7ff75c0ce810 con 0x7ff75c15abb0 2023-11-24T07:56:49.128 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:49.126+0000 7ff74e7fc700 1 -- 172.21.15.59:0/597621185 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v8) v1 ==== 53+0+73 (secure 0 0 0) 0x7ff75401d020 con 0x7ff75c15abb0 2023-11-24T07:56:49.128 INFO:teuthology.orchestra.run.smithi059.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-11-24T07:56:49.130 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:49.128+0000 7ff763f01700 1 -- 172.21.15.59:0/597621185 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7ff74006aed0 msgr2=0x7ff74006d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:56:49.130 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:49.128+0000 7ff763f01700 1 --2- 172.21.15.59:0/597621185 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7ff74006aed0 0x7ff74006d390 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7ff750000f60 tx=0x7ff750007a40 comp rx=0 tx=0).stop 2023-11-24T07:56:49.130 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:49.128+0000 7ff763f01700 1 -- 172.21.15.59:0/597621185 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff75c15abb0 msgr2=0x7ff75c15cb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:56:49.130 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:49.128+0000 7ff763f01700 1 --2- 172.21.15.59:0/597621185 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff75c15abb0 0x7ff75c15cb80 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7ff75400cd50 tx=0x7ff75400a560 comp rx=0 tx=0).stop 2023-11-24T07:56:49.131 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:49.128+0000 7ff763f01700 1 -- 172.21.15.59:0/597621185 shutdown_connections 2023-11-24T07:56:49.131 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:49.128+0000 7ff763f01700 1 --2- 172.21.15.59:0/597621185 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7ff74006aed0 0x7ff74006d390 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:56:49.131 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:49.128+0000 7ff763f01700 1 --2- 172.21.15.59:0/597621185 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff75c15abb0 0x7ff75c15cb80 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:56:49.131 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:49.128+0000 7ff763f01700 1 --2- 172.21.15.59:0/597621185 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff75c0722d0 0x7ff75c1f74f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:56:49.131 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:49.128+0000 7ff763f01700 1 --2- 172.21.15.59:0/597621185 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7ff75c0b4f20 0x7ff75c153580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:56:49.131 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:49.128+0000 7ff763f01700 1 -- 172.21.15.59:0/597621185 >> 172.21.15.59:0/597621185 conn(0x7ff75c0d0aa0 msgr2=0x7ff75c0b1010 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:56:49.131 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:49.128+0000 7ff763f01700 1 -- 172.21.15.59:0/597621185 shutdown_connections 2023-11-24T07:56:49.131 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:49.128+0000 7ff763f01700 1 -- 172.21.15.59:0/597621185 wait complete. 2023-11-24T07:56:49.142 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2023-11-24T07:56:49.142 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-11-24T07:56:49.142 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi076.front.sepia.ceph.com 2023-11-24T07:56:49.142 INFO:tasks.cephfs.mount:self.client.name = client.1 2023-11-24T07:56:49.142 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2023-11-24T07:56:49.143 INFO:tasks.cephfs.mount:self.cephfs_name = None 2023-11-24T07:56:49.143 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-11-24T07:56:49.143 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-11-24T07:56:49.143 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.1' 2023-11-24T07:56:49.143 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T07:56:49.143 DEBUG:teuthology.orchestra.run.smithi076:> ip addr 2023-11-24T07:56:49.164 INFO:teuthology.orchestra.run.smithi076.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-11-24T07:56:49.164 INFO:teuthology.orchestra.run.smithi076.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-11-24T07:56:49.164 INFO:teuthology.orchestra.run.smithi076.stdout: inet 127.0.0.1/8 scope host lo 2023-11-24T07:56:49.165 INFO:teuthology.orchestra.run.smithi076.stdout: valid_lft forever preferred_lft forever 2023-11-24T07:56:49.165 INFO:teuthology.orchestra.run.smithi076.stdout: inet6 ::1/128 scope host 2023-11-24T07:56:49.165 INFO:teuthology.orchestra.run.smithi076.stdout: valid_lft forever preferred_lft forever 2023-11-24T07:56:49.165 INFO:teuthology.orchestra.run.smithi076.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-11-24T07:56:49.165 INFO:teuthology.orchestra.run.smithi076.stdout: link/ether 0c:c4:7a:d8:da:66 brd ff:ff:ff:ff:ff:ff 2023-11-24T07:56:49.165 INFO:teuthology.orchestra.run.smithi076.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-11-24T07:56:49.165 INFO:teuthology.orchestra.run.smithi076.stdout: link/ether 0c:c4:7a:d8:da:67 brd ff:ff:ff:ff:ff:ff 2023-11-24T07:56:49.165 INFO:teuthology.orchestra.run.smithi076.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-11-24T07:56:49.165 INFO:teuthology.orchestra.run.smithi076.stdout: link/ether 0c:c4:7a:8f:47:e0 brd ff:ff:ff:ff:ff:ff 2023-11-24T07:56:49.165 INFO:teuthology.orchestra.run.smithi076.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-11-24T07:56:49.165 INFO:teuthology.orchestra.run.smithi076.stdout: link/ether 0c:c4:7a:8f:47:e1 brd ff:ff:ff:ff:ff:ff 2023-11-24T07:56:49.165 INFO:teuthology.orchestra.run.smithi076.stdout: inet 172.21.15.76/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2023-11-24T07:56:49.165 INFO:teuthology.orchestra.run.smithi076.stdout: valid_lft 42390sec preferred_lft 42390sec 2023-11-24T07:56:49.165 INFO:teuthology.orchestra.run.smithi076.stdout: inet6 fe80::a34d:4ce5:8360:8f9/64 scope link noprefixroute 2023-11-24T07:56:49.166 INFO:teuthology.orchestra.run.smithi076.stdout: valid_lft forever preferred_lft forever 2023-11-24T07:56:49.167 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2023-11-24T07:56:49.167 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T07:56:49.167 DEBUG:teuthology.orchestra.run.smithi076:> (cd / && exec bash -c ' 2023-11-24T07:56:49.167 DEBUG:teuthology.orchestra.run.smithi076:> set -e 2023-11-24T07:56:49.167 DEBUG:teuthology.orchestra.run.smithi076:> sudo ip link add name ceph-brx type bridge 2023-11-24T07:56:49.167 DEBUG:teuthology.orchestra.run.smithi076:> sudo ip addr flush dev ceph-brx 2023-11-24T07:56:49.167 DEBUG:teuthology.orchestra.run.smithi076:> sudo ip link set ceph-brx up 2023-11-24T07:56:49.167 DEBUG:teuthology.orchestra.run.smithi076:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2023-11-24T07:56:49.167 DEBUG:teuthology.orchestra.run.smithi076:> ') 2023-11-24T07:56:49.355 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T07:56:49.355 DEBUG:teuthology.orchestra.run.smithi076:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-11-24T07:56:49.388 INFO:teuthology.orchestra.run.smithi076.stdout:1 2023-11-24T07:56:49.390 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T07:56:49.390 DEBUG:teuthology.orchestra.run.smithi076:> route 2023-11-24T07:56:49.463 INFO:teuthology.orchestra.run.smithi076.stdout:Kernel IP routing table 2023-11-24T07:56:49.464 INFO:teuthology.orchestra.run.smithi076.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-11-24T07:56:49.464 INFO:teuthology.orchestra.run.smithi076.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-11-24T07:56:49.464 INFO:teuthology.orchestra.run.smithi076.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-11-24T07:56:49.464 INFO:teuthology.orchestra.run.smithi076.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-11-24T07:56:49.464 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T07:56:49.465 DEBUG:teuthology.orchestra.run.smithi076:> (cd / && exec bash -c ' 2023-11-24T07:56:49.465 DEBUG:teuthology.orchestra.run.smithi076:> set -e 2023-11-24T07:56:49.465 DEBUG:teuthology.orchestra.run.smithi076:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-11-24T07:56:49.465 DEBUG:teuthology.orchestra.run.smithi076:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-11-24T07:56:49.465 DEBUG:teuthology.orchestra.run.smithi076:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-11-24T07:56:49.465 DEBUG:teuthology.orchestra.run.smithi076:> ') 2023-11-24T07:56:49.728 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T07:56:49.728 DEBUG:teuthology.orchestra.run.smithi076:> ip netns list 2023-11-24T07:56:49.746 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T07:56:49.746 DEBUG:teuthology.orchestra.run.smithi076:> ip netns list-id 2023-11-24T07:56:49.802 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T07:56:49.802 DEBUG:teuthology.orchestra.run.smithi076:> (cd / && exec bash -c ' 2023-11-24T07:56:49.802 DEBUG:teuthology.orchestra.run.smithi076:> set -e 2023-11-24T07:56:49.802 DEBUG:teuthology.orchestra.run.smithi076:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.1 2023-11-24T07:56:49.803 DEBUG:teuthology.orchestra.run.smithi076:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.1 0 2023-11-24T07:56:49.803 DEBUG:teuthology.orchestra.run.smithi076:> ') 2023-11-24T07:56:49.896 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.1' with 192.168.0.1/16 2023-11-24T07:56:49.896 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T07:56:49.897 DEBUG:teuthology.orchestra.run.smithi076:> (cd / && exec bash -c ' 2023-11-24T07:56:49.897 DEBUG:teuthology.orchestra.run.smithi076:> set -e 2023-11-24T07:56:49.897 DEBUG:teuthology.orchestra.run.smithi076:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.1 type veth peer name brx.0 2023-11-24T07:56:49.897 DEBUG:teuthology.orchestra.run.smithi076:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2023-11-24T07:56:49.897 DEBUG:teuthology.orchestra.run.smithi076:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip link set veth0 up 2023-11-24T07:56:49.897 DEBUG:teuthology.orchestra.run.smithi076:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip link set lo up 2023-11-24T07:56:49.897 DEBUG:teuthology.orchestra.run.smithi076:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip route add default via 192.168.255.254 2023-11-24T07:56:49.897 DEBUG:teuthology.orchestra.run.smithi076:> ') 2023-11-24T07:56:50.102 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T07:56:50.103 DEBUG:teuthology.orchestra.run.smithi076:> (cd / && exec bash -c ' 2023-11-24T07:56:50.103 DEBUG:teuthology.orchestra.run.smithi076:> set -e 2023-11-24T07:56:50.103 DEBUG:teuthology.orchestra.run.smithi076:> sudo ip link set brx.0 up 2023-11-24T07:56:50.103 DEBUG:teuthology.orchestra.run.smithi076:> sudo ip link set dev brx.0 master ceph-brx 2023-11-24T07:56:50.103 DEBUG:teuthology.orchestra.run.smithi076:> ') 2023-11-24T07:56:50.202 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-11-24T07:56:50.203 DEBUG:teuthology.orchestra.run.smithi076:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2023-11-24T07:56:50.223 INFO:teuthology.orchestra.run.smithi076.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2023-11-24T07:56:50.223 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-11-24T07:56:50.223 DEBUG:teuthology.orchestra.run.smithi076:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2023-11-24T07:56:50.279 INFO:tasks.cephfs.kernel_mount:mounting using device: 1@.cephfs=/ 2023-11-24T07:56:50.279 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T07:56:50.279 DEBUG:teuthology.orchestra.run.smithi076:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 1@.cephfs=/ /home/ubuntu/cephtest/mnt.1 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-11-24T07:56:50.378 INFO:teuthology.orchestra.run.smithi076.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-11-24T07:56:50.378 INFO:teuthology.orchestra.run.smithi076.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-11-24T07:56:50.496 INFO:teuthology.orchestra.run.smithi076.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-11-24T07:56:50.496 INFO:teuthology.orchestra.run.smithi076.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-11-24T07:56:50.496 INFO:teuthology.orchestra.run.smithi076.stdout:mount.ceph: resolved to: "172.21.15.59:3300,172.21.15.76:3300,172.21.15.76:3301" 2023-11-24T07:56:50.496 INFO:teuthology.orchestra.run.smithi076.stdout:mount.ceph: trying mount with new device syntax: 1@9f0e8741-b8f9-48d5-a2c3-09f45903303d.cephfs=/ 2023-11-24T07:56:50.496 INFO:teuthology.orchestra.run.smithi076.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,ms_mode=prefer-crc,key=1,mon_addr=172.21.15.59:3300/172.21.15.76:3300/172.21.15.76:3301" will pass to kernel 2023-11-24T07:56:50.497 INFO:teuthology.orchestra.run.smithi076.stdout:mount.ceph: v2 mount support check returned 2 2023-11-24T07:56:50.497 INFO:teuthology.orchestra.run.smithi076.stdout:mount.ceph: kernel does not support v2 syntax 2023-11-24T07:56:50.497 INFO:teuthology.orchestra.run.smithi076.stdout:mount.ceph: switching to using v1 address with old syntax 2023-11-24T07:56:50.530 INFO:teuthology.orchestra.run.smithi076.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-11-24T07:56:50.531 INFO:teuthology.orchestra.run.smithi076.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-11-24T07:56:50.531 INFO:teuthology.orchestra.run.smithi076.stdout:mount.ceph: resolved to: "172.21.15.59:3300,172.21.15.76:3300,172.21.15.76:3301" 2023-11-24T07:56:50.531 INFO:teuthology.orchestra.run.smithi076.stdout:mount.ceph: trying mount with new device syntax: 1@9f0e8741-b8f9-48d5-a2c3-09f45903303d.cephfs=/ 2023-11-24T07:56:50.532 INFO:teuthology.orchestra.run.smithi076.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,ms_mode=prefer-crc,key=1,mon_addr=172.21.15.59:3300/172.21.15.76:3300/172.21.15.76:3301" will pass to kernel 2023-11-24T07:56:50.532 INFO:teuthology.orchestra.run.smithi076.stdout:mount.ceph: v2 mount support check returned 2 2023-11-24T07:56:50.532 INFO:teuthology.orchestra.run.smithi076.stdout:mount.ceph: kernel does not support v2 syntax 2023-11-24T07:56:50.533 INFO:teuthology.orchestra.run.smithi076.stdout:mount.ceph: switching to using v1 address with old syntax 2023-11-24T07:56:50.533 INFO:teuthology.orchestra.run.smithi076.stdout:mount.ceph: resolved to: "172.21.15.59:6789,172.21.15.76:6789,172.21.15.76:6790" 2023-11-24T07:56:50.533 INFO:teuthology.orchestra.run.smithi076.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.59:6789,172.21.15.76:6789,172.21.15.76:6790:/ 2023-11-24T07:56:50.533 INFO:teuthology.orchestra.run.smithi076.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,key=1,mds_namespace=cephfs,fsid=9f0e8741-b8f9-48d5-a2c3-09f45903303d" will pass to kernel 2023-11-24T07:56:50.534 INFO:tasks.cephfs.kernel_mount:mount command passed 2023-11-24T07:56:50.535 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T07:56:50.535 DEBUG:teuthology.orchestra.run.smithi076:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2023-11-24T07:56:50.566 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T07:56:50.566 DEBUG:teuthology.orchestra.run.smithi076:> (cd /home/ubuntu/cephtest/mnt.1 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-11-24T07:56:50.638 INFO:teuthology.orchestra.run.smithi076.stdout:client4612 2023-11-24T07:56:50.638 DEBUG:teuthology.orchestra.run.smithi076:> set -ex 2023-11-24T07:56:50.638 DEBUG:teuthology.orchestra.run.smithi076:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-24T07:56:50.696 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T07:56:50.697 DEBUG:teuthology.orchestra.run.smithi076:> (cd /home/ubuntu/cephtest/mnt.1 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-11-24T07:56:50.768 INFO:teuthology.orchestra.run.smithi076.stdout:client4612 2023-11-24T07:56:50.769 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T07:56:50.769 DEBUG:teuthology.orchestra.run.smithi076:> (cd /home/ubuntu/cephtest/mnt.1 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-11-24T07:56:50.841 INFO:teuthology.orchestra.run.smithi076.stdout:client4612 2023-11-24T07:56:50.841 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T07:56:50.842 DEBUG:teuthology.orchestra.run.smithi076:> (cd /home/ubuntu/cephtest/mnt.1 && exec bash -c 'sudo dd if=/sys/kernel/debug/ceph/9f0e8741-b8f9-48d5-a2c3-09f45903303d.client4612/status') 2023-11-24T07:56:50.922 INFO:teuthology.orchestra.run.smithi076.stdout:instance: client.4612 (3)192.168.0.1:0/494839953 2023-11-24T07:56:50.923 INFO:teuthology.orchestra.run.smithi076.stdout:blocklisted: false 2023-11-24T07:56:50.924 INFO:teuthology.orchestra.run.smithi076.stderr:0+1 records in 2023-11-24T07:56:50.924 INFO:teuthology.orchestra.run.smithi076.stderr:0+1 records out 2023-11-24T07:56:50.924 INFO:teuthology.orchestra.run.smithi076.stderr:68 bytes copied, 2.3439e-05 s, 2.9 MB/s 2023-11-24T07:56:50.924 INFO:tasks.kclient:client.3 config is {'syntax': 'v2'} 2023-11-24T07:56:50.925 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2023-11-24T07:56:50.925 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.3 2023-11-24T07:56:50.925 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-11-24T07:56:51.064 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:51.062+0000 7fb6c1b24700 1 Processor -- start 2023-11-24T07:56:51.064 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:51.062+0000 7fb6c1b24700 1 -- start start 2023-11-24T07:56:51.066 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:51.063+0000 7fb6c1b24700 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fb6bc0b4150 0x7fb6bc06de50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:56:51.066 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:51.063+0000 7fb6c1b24700 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb6bc071f50 con 0x7fb6bc06b620 2023-11-24T07:56:51.066 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:51.063+0000 7fb6c1b24700 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb6bc072230 con 0x7fb6bc0b5ab0 2023-11-24T07:56:51.066 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:51.063+0000 7fb6c1b24700 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb6bc072510 con 0x7fb6bc0b4150 2023-11-24T07:56:51.066 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:51.063+0000 7fb6bb7fe700 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7fb6bc06b620 0x7fb6bc0b3a40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:54060/0 (socket says 172.21.15.59:54060) 2023-11-24T07:56:51.067 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:51.063+0000 7fb6bb7fe700 1 -- 172.21.15.59:0/909342417 learned_addr learned my addr 172.21.15.59:0/909342417 (peer_addr_for_me v1:172.21.15.59:0/0) 2023-11-24T07:56:51.067 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:51.063+0000 7fb6baffd700 1 --2- 172.21.15.59:0/909342417 >> v2:172.21.15.76:3300/0 conn(0x7fb6bc0b4150 0x7fb6bc06de50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:56:51.067 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:51.063+0000 7fb6ba7fc700 1 -- 172.21.15.59:0/909342417 <== mon.0 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3962587038 0 0) 0x7fb6bc071f50 con 0x7fb6bc06b620 2023-11-24T07:56:51.067 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:51.063+0000 7fb6ba7fc700 1 -- 172.21.15.59:0/909342417 --> v1:172.21.15.59:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb6a80035b0 con 0x7fb6bc06b620 2023-11-24T07:56:51.067 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:51.064+0000 7fb6ba7fc700 1 -- 172.21.15.59:0/909342417 <== mon.1 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3444449646 0 0) 0x7fb6bc072230 con 0x7fb6bc0b5ab0 2023-11-24T07:56:51.067 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:51.064+0000 7fb6ba7fc700 1 -- 172.21.15.59:0/909342417 --> v1:172.21.15.76:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb6bc071f50 con 0x7fb6bc0b5ab0 2023-11-24T07:56:51.067 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:51.064+0000 7fb6baffd700 1 -- 172.21.15.59:0/909342417 >> v1:172.21.15.76:6789/0 conn(0x7fb6bc0b5ab0 legacy=0x7fb6bc06e390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:56:51.067 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:51.064+0000 7fb6baffd700 1 -- 172.21.15.59:0/909342417 >> v1:172.21.15.59:6789/0 conn(0x7fb6bc06b620 legacy=0x7fb6bc0b3a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:56:51.067 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:51.064+0000 7fb6baffd700 1 -- 172.21.15.59:0/909342417 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb6bc0727f0 con 0x7fb6bc0b4150 2023-11-24T07:56:51.067 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:51.064+0000 7fb6baffd700 1 --2- 172.21.15.59:0/909342417 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb6bc0b4150 0x7fb6bc06de50 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fb6ac009840 tx=0x7fb6ac009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=983419aaa8345419 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:56:51.067 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:51.064+0000 7fb6ba7fc700 1 -- 172.21.15.59:0/909342417 <== mon.1 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6ac011830 con 0x7fb6bc0b4150 2023-11-24T07:56:51.067 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:51.064+0000 7fb6ba7fc700 1 -- 172.21.15.59:0/909342417 <== mon.1 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb6ac011990 con 0x7fb6bc0b4150 2023-11-24T07:56:51.068 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:51.064+0000 7fb6ba7fc700 1 -- 172.21.15.59:0/909342417 <== mon.1 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6ac011d00 con 0x7fb6bc0b4150 2023-11-24T07:56:51.068 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:51.064+0000 7fb6c1b24700 1 -- 172.21.15.59:0/909342417 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb6bc0b4150 msgr2=0x7fb6bc06de50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:56:51.068 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:51.064+0000 7fb6c1b24700 1 --2- 172.21.15.59:0/909342417 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb6bc0b4150 0x7fb6bc06de50 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fb6ac009840 tx=0x7fb6ac009b50 comp rx=0 tx=0).stop 2023-11-24T07:56:51.068 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:51.065+0000 7fb6c1b24700 1 -- 172.21.15.59:0/909342417 shutdown_connections 2023-11-24T07:56:51.068 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:51.065+0000 7fb6c1b24700 1 --2- 172.21.15.59:0/909342417 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb6bc0b4150 0x7fb6bc06de50 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:56:51.068 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:51.065+0000 7fb6c1b24700 1 -- 172.21.15.59:0/909342417 >> 172.21.15.59:0/909342417 conn(0x7fb6bc0b2740 msgr2=0x7fb6bc0b2b20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:56:51.068 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:51.065+0000 7fb6c1b24700 1 -- 172.21.15.59:0/909342417 shutdown_connections 2023-11-24T07:56:51.068 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:51.065+0000 7fb6c1b24700 1 -- 172.21.15.59:0/909342417 wait complete. 2023-11-24T07:56:51.070 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:51.066+0000 7fb6c1b24700 1 Processor -- start 2023-11-24T07:56:51.070 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:51.066+0000 7fb6c1b24700 1 -- start start 2023-11-24T07:56:51.071 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:51.067+0000 7fb6c1b24700 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fb6bc06b620 0x7fb6bc1e9210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:56:51.071 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:51.067+0000 7fb6c1b24700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb6bc0b4150 0x7fb6bc0c66b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:56:51.071 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:51.067+0000 7fb6c1b24700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb6bc0b5ab0 0x7fb6bc06eda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:56:51.071 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:51.067+0000 7fb6c1b24700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6bc1e9750 con 0x7fb6bc0b4150 2023-11-24T07:56:51.071 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:51.067+0000 7fb6c1b24700 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6bc0c7090 con 0x7fb6bc0b5ab0 2023-11-24T07:56:51.071 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:51.067+0000 7fb6c1b24700 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb6bc1f3c20 con 0x7fb6bc06b620 2023-11-24T07:56:51.071 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:51.067+0000 7fb6baffd700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb6bc0b4150 0x7fb6bc0c66b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:56:51.071 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:51.067+0000 7fb6baffd700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb6bc0b4150 0x7fb6bc0c66b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:50594/0 (socket says 172.21.15.59:50594) 2023-11-24T07:56:51.071 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:51.067+0000 7fb6baffd700 1 -- 172.21.15.59:0/2491137034 learned_addr learned my addr 172.21.15.59:0/2491137034 (peer_addr_for_me v2:172.21.15.59:0/0) 2023-11-24T07:56:51.071 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:51.067+0000 7fb6bbfff700 1 --2- 172.21.15.59:0/2491137034 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb6bc0b5ab0 0x7fb6bc06eda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:56:51.071 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:51.067+0000 7fb6bb7fe700 1 --2- 172.21.15.59:0/2491137034 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fb6bc06b620 0x7fb6bc1e9210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:56:51.072 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:51.067+0000 7fb6baffd700 1 -- 172.21.15.59:0/2491137034 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fb6bc06b620 msgr2=0x7fb6bc1e9210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:56:51.072 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:51.067+0000 7fb6baffd700 1 --2- 172.21.15.59:0/2491137034 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fb6bc06b620 0x7fb6bc1e9210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:56:51.072 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:51.067+0000 7fb6baffd700 1 -- 172.21.15.59:0/2491137034 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb6bc0b5ab0 msgr2=0x7fb6bc06eda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:56:51.072 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:51.067+0000 7fb6baffd700 1 --2- 172.21.15.59:0/2491137034 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb6bc0b5ab0 0x7fb6bc06eda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:56:51.072 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:51.067+0000 7fb6baffd700 1 -- 172.21.15.59:0/2491137034 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb6ac0094a0 con 0x7fb6bc0b4150 2023-11-24T07:56:51.072 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:51.067+0000 7fb6bbfff700 1 --2- 172.21.15.59:0/2491137034 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb6bc0b5ab0 0x7fb6bc06eda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-24T07:56:51.072 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:51.067+0000 7fb6baffd700 1 --2- 172.21.15.59:0/2491137034 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb6bc0b4150 0x7fb6bc0c66b0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fb6ac009fd0 tx=0x7fb6ac004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:56:51.072 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:51.068+0000 7fb6b3fff700 1 -- 172.21.15.59:0/2491137034 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6ac0045d0 con 0x7fb6bc0b4150 2023-11-24T07:56:51.072 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:51.068+0000 7fb6b3fff700 1 -- 172.21.15.59:0/2491137034 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb6ac01f070 con 0x7fb6bc0b4150 2023-11-24T07:56:51.072 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:51.068+0000 7fb6c1b24700 1 -- 172.21.15.59:0/2491137034 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb6bc1f3f00 con 0x7fb6bc0b4150 2023-11-24T07:56:51.072 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:51.068+0000 7fb6b3fff700 1 -- 172.21.15.59:0/2491137034 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6ac0038c0 con 0x7fb6bc0b4150 2023-11-24T07:56:51.072 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:51.068+0000 7fb6c1b24700 1 -- 172.21.15.59:0/2491137034 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb6bc1f3880 con 0x7fb6bc0b4150 2023-11-24T07:56:51.073 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:51.069+0000 7fb6c1b24700 1 -- 172.21.15.59:0/2491137034 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb6bc1f3a10 con 0x7fb6bc0b4150 2023-11-24T07:56:51.073 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:51.069+0000 7fb6b3fff700 1 -- 172.21.15.59:0/2491137034 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fb6ac003a20 con 0x7fb6bc0b4150 2023-11-24T07:56:51.073 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:51.069+0000 7fb6b3fff700 1 --2- 172.21.15.59:0/2491137034 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7fb69c06aed0 0x7fb69c06d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:56:51.073 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:51.069+0000 7fb6b3fff700 1 -- 172.21.15.59:0/2491137034 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fb6ac021070 con 0x7fb6bc0b4150 2023-11-24T07:56:51.073 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:51.070+0000 7fb6bb7fe700 1 --2- 172.21.15.59:0/2491137034 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7fb69c06aed0 0x7fb69c06d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:56:51.074 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:51.070+0000 7fb6bb7fe700 1 --2- 172.21.15.59:0/2491137034 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7fb69c06aed0 0x7fb69c06d390 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fb6a4000f60 tx=0x7fb6a4009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:56:51.075 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:51.074+0000 7fb6b3fff700 1 -- 172.21.15.59:0/2491137034 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fb6ac055fb0 con 0x7fb6bc0b4150 2023-11-24T07:56:51.326 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:51.324+0000 7fb6c1b24700 1 -- 172.21.15.59:0/2491137034 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7fb6bc068670 con 0x7fb6bc0b4150 2023-11-24T07:56:51.326 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:51.324+0000 7fb6b3fff700 1 -- 172.21.15.59:0/2491137034 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v8) v1 ==== 53+0+73 (secure 0 0 0) 0x7fb6ac015020 con 0x7fb6bc0b4150 2023-11-24T07:56:51.326 INFO:teuthology.orchestra.run.smithi059.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-11-24T07:56:51.329 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:51.327+0000 7fb6c1b24700 1 -- 172.21.15.59:0/2491137034 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7fb69c06aed0 msgr2=0x7fb69c06d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:56:51.329 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:51.327+0000 7fb6c1b24700 1 --2- 172.21.15.59:0/2491137034 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7fb69c06aed0 0x7fb69c06d390 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fb6a4000f60 tx=0x7fb6a4009040 comp rx=0 tx=0).stop 2023-11-24T07:56:51.329 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:51.327+0000 7fb6c1b24700 1 -- 172.21.15.59:0/2491137034 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb6bc0b4150 msgr2=0x7fb6bc0c66b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:56:51.329 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:51.327+0000 7fb6c1b24700 1 --2- 172.21.15.59:0/2491137034 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb6bc0b4150 0x7fb6bc0c66b0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fb6ac009fd0 tx=0x7fb6ac004750 comp rx=0 tx=0).stop 2023-11-24T07:56:51.329 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:51.327+0000 7fb6c1b24700 1 -- 172.21.15.59:0/2491137034 shutdown_connections 2023-11-24T07:56:51.329 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:51.327+0000 7fb6c1b24700 1 --2- 172.21.15.59:0/2491137034 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7fb69c06aed0 0x7fb69c06d390 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:56:51.329 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:51.327+0000 7fb6c1b24700 1 --2- 172.21.15.59:0/2491137034 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb6bc0b5ab0 0x7fb6bc06eda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:56:51.329 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:51.327+0000 7fb6c1b24700 1 --2- 172.21.15.59:0/2491137034 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fb6bc06b620 0x7fb6bc1e9210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:56:51.329 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:51.327+0000 7fb6c1b24700 1 --2- 172.21.15.59:0/2491137034 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb6bc0b4150 0x7fb6bc0c66b0 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:56:51.329 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:51.327+0000 7fb6c1b24700 1 -- 172.21.15.59:0/2491137034 >> 172.21.15.59:0/2491137034 conn(0x7fb6bc0b2740 msgr2=0x7fb6bc1475e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:56:51.330 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:51.327+0000 7fb6c1b24700 1 -- 172.21.15.59:0/2491137034 shutdown_connections 2023-11-24T07:56:51.330 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:51.327+0000 7fb6c1b24700 1 -- 172.21.15.59:0/2491137034 wait complete. 2023-11-24T07:56:51.341 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2023-11-24T07:56:51.341 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-11-24T07:56:51.341 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi076.front.sepia.ceph.com 2023-11-24T07:56:51.341 INFO:tasks.cephfs.mount:self.client.name = client.3 2023-11-24T07:56:51.341 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.3 2023-11-24T07:56:51.341 INFO:tasks.cephfs.mount:self.cephfs_name = None 2023-11-24T07:56:51.341 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-11-24T07:56:51.341 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-11-24T07:56:51.341 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.3' 2023-11-24T07:56:51.341 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T07:56:51.342 DEBUG:teuthology.orchestra.run.smithi076:> ip addr 2023-11-24T07:56:51.359 INFO:teuthology.orchestra.run.smithi076.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-11-24T07:56:51.359 INFO:teuthology.orchestra.run.smithi076.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-11-24T07:56:51.359 INFO:teuthology.orchestra.run.smithi076.stdout: inet 127.0.0.1/8 scope host lo 2023-11-24T07:56:51.359 INFO:teuthology.orchestra.run.smithi076.stdout: valid_lft forever preferred_lft forever 2023-11-24T07:56:51.360 INFO:teuthology.orchestra.run.smithi076.stdout: inet6 ::1/128 scope host 2023-11-24T07:56:51.360 INFO:teuthology.orchestra.run.smithi076.stdout: valid_lft forever preferred_lft forever 2023-11-24T07:56:51.360 INFO:teuthology.orchestra.run.smithi076.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-11-24T07:56:51.360 INFO:teuthology.orchestra.run.smithi076.stdout: link/ether 0c:c4:7a:d8:da:66 brd ff:ff:ff:ff:ff:ff 2023-11-24T07:56:51.360 INFO:teuthology.orchestra.run.smithi076.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-11-24T07:56:51.360 INFO:teuthology.orchestra.run.smithi076.stdout: link/ether 0c:c4:7a:d8:da:67 brd ff:ff:ff:ff:ff:ff 2023-11-24T07:56:51.360 INFO:teuthology.orchestra.run.smithi076.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-11-24T07:56:51.360 INFO:teuthology.orchestra.run.smithi076.stdout: link/ether 0c:c4:7a:8f:47:e0 brd ff:ff:ff:ff:ff:ff 2023-11-24T07:56:51.360 INFO:teuthology.orchestra.run.smithi076.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-11-24T07:56:51.360 INFO:teuthology.orchestra.run.smithi076.stdout: link/ether 0c:c4:7a:8f:47:e1 brd ff:ff:ff:ff:ff:ff 2023-11-24T07:56:51.360 INFO:teuthology.orchestra.run.smithi076.stdout: inet 172.21.15.76/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2023-11-24T07:56:51.360 INFO:teuthology.orchestra.run.smithi076.stdout: valid_lft 42388sec preferred_lft 42388sec 2023-11-24T07:56:51.361 INFO:teuthology.orchestra.run.smithi076.stdout: inet6 fe80::a34d:4ce5:8360:8f9/64 scope link noprefixroute 2023-11-24T07:56:51.361 INFO:teuthology.orchestra.run.smithi076.stdout: valid_lft forever preferred_lft forever 2023-11-24T07:56:51.361 INFO:teuthology.orchestra.run.smithi076.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-11-24T07:56:51.361 INFO:teuthology.orchestra.run.smithi076.stdout: link/ether 46:3e:09:af:65:64 brd ff:ff:ff:ff:ff:ff 2023-11-24T07:56:51.361 INFO:teuthology.orchestra.run.smithi076.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-11-24T07:56:51.361 INFO:teuthology.orchestra.run.smithi076.stdout: valid_lft forever preferred_lft forever 2023-11-24T07:56:51.361 INFO:teuthology.orchestra.run.smithi076.stdout: inet6 fe80::4c23:44ff:feb9:d9d4/64 scope link 2023-11-24T07:56:51.361 INFO:teuthology.orchestra.run.smithi076.stdout: valid_lft forever preferred_lft forever 2023-11-24T07:56:51.361 INFO:teuthology.orchestra.run.smithi076.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-11-24T07:56:51.361 INFO:teuthology.orchestra.run.smithi076.stdout: link/ether 46:3e:09:af:65:64 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2023-11-24T07:56:51.361 INFO:teuthology.orchestra.run.smithi076.stdout: inet6 fe80::443e:9ff:feaf:6564/64 scope link tentative 2023-11-24T07:56:51.361 INFO:teuthology.orchestra.run.smithi076.stdout: valid_lft forever preferred_lft forever 2023-11-24T07:56:51.362 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T07:56:51.362 DEBUG:teuthology.orchestra.run.smithi076:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-11-24T07:56:51.426 INFO:teuthology.orchestra.run.smithi076.stdout:1 2023-11-24T07:56:51.428 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T07:56:51.428 DEBUG:teuthology.orchestra.run.smithi076:> route 2023-11-24T07:56:51.486 INFO:teuthology.orchestra.run.smithi076.stdout:Kernel IP routing table 2023-11-24T07:56:51.487 INFO:teuthology.orchestra.run.smithi076.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-11-24T07:56:51.487 INFO:teuthology.orchestra.run.smithi076.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-11-24T07:56:51.487 INFO:teuthology.orchestra.run.smithi076.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-11-24T07:56:51.487 INFO:teuthology.orchestra.run.smithi076.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-11-24T07:56:51.487 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T07:56:51.487 DEBUG:teuthology.orchestra.run.smithi076:> (cd / && exec bash -c ' 2023-11-24T07:56:51.487 DEBUG:teuthology.orchestra.run.smithi076:> set -e 2023-11-24T07:56:51.488 DEBUG:teuthology.orchestra.run.smithi076:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-11-24T07:56:51.488 DEBUG:teuthology.orchestra.run.smithi076:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-11-24T07:56:51.488 DEBUG:teuthology.orchestra.run.smithi076:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-11-24T07:56:51.488 DEBUG:teuthology.orchestra.run.smithi076:> ') 2023-11-24T07:56:51.591 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T07:56:51.591 DEBUG:teuthology.orchestra.run.smithi076:> ip netns list 2023-11-24T07:56:51.607 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2023-11-24T07:56:51.608 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T07:56:51.608 DEBUG:teuthology.orchestra.run.smithi076:> ip netns list-id 2023-11-24T07:56:51.665 INFO:teuthology.orchestra.run.smithi076.stdout:nsid 0 (iproute2 netns name: ceph-ns--home-ubuntu-cephtest-mnt.1) 2023-11-24T07:56:51.665 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T07:56:51.665 DEBUG:teuthology.orchestra.run.smithi076:> (cd / && exec bash -c ' 2023-11-24T07:56:51.665 DEBUG:teuthology.orchestra.run.smithi076:> set -e 2023-11-24T07:56:51.665 DEBUG:teuthology.orchestra.run.smithi076:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.3 2023-11-24T07:56:51.665 DEBUG:teuthology.orchestra.run.smithi076:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.3 1 2023-11-24T07:56:51.666 DEBUG:teuthology.orchestra.run.smithi076:> ') 2023-11-24T07:56:51.758 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T07:56:51.758 DEBUG:teuthology.orchestra.run.smithi076:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr 2023-11-24T07:56:51.790 INFO:teuthology.orchestra.run.smithi076.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-11-24T07:56:51.791 INFO:teuthology.orchestra.run.smithi076.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-11-24T07:56:51.791 INFO:teuthology.orchestra.run.smithi076.stdout: inet 127.0.0.1/8 scope host lo 2023-11-24T07:56:51.791 INFO:teuthology.orchestra.run.smithi076.stdout: valid_lft forever preferred_lft forever 2023-11-24T07:56:51.791 INFO:teuthology.orchestra.run.smithi076.stdout: inet6 ::1/128 scope host 2023-11-24T07:56:51.791 INFO:teuthology.orchestra.run.smithi076.stdout: valid_lft forever preferred_lft forever 2023-11-24T07:56:51.791 INFO:teuthology.orchestra.run.smithi076.stdout:2: veth0@if7: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-11-24T07:56:51.791 INFO:teuthology.orchestra.run.smithi076.stdout: link/ether ce:f2:81:8d:3a:11 brd ff:ff:ff:ff:ff:ff link-netnsid 0 2023-11-24T07:56:51.791 INFO:teuthology.orchestra.run.smithi076.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2023-11-24T07:56:51.791 INFO:teuthology.orchestra.run.smithi076.stdout: valid_lft forever preferred_lft forever 2023-11-24T07:56:51.791 INFO:teuthology.orchestra.run.smithi076.stdout: inet6 fe80::ccf2:81ff:fe8d:3a11/64 scope link tentative 2023-11-24T07:56:51.791 INFO:teuthology.orchestra.run.smithi076.stdout: valid_lft forever preferred_lft forever 2023-11-24T07:56:51.806 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T07:56:51.806 DEBUG:teuthology.orchestra.run.smithi076:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr 2023-11-24T07:56:51.884 INFO:teuthology.orchestra.run.smithi076.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-11-24T07:56:51.884 INFO:teuthology.orchestra.run.smithi076.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-11-24T07:56:51.885 INFO:teuthology.orchestra.run.smithi076.stdout: inet 127.0.0.1/8 scope host lo 2023-11-24T07:56:51.885 INFO:teuthology.orchestra.run.smithi076.stdout: valid_lft forever preferred_lft forever 2023-11-24T07:56:51.885 INFO:teuthology.orchestra.run.smithi076.stdout: inet6 ::1/128 scope host 2023-11-24T07:56:51.885 INFO:teuthology.orchestra.run.smithi076.stdout: valid_lft forever preferred_lft forever 2023-11-24T07:56:51.885 INFO:teuthology.orchestra.run.smithi076.stdout:2: veth0@if7: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-11-24T07:56:51.886 INFO:teuthology.orchestra.run.smithi076.stdout: link/ether ce:f2:81:8d:3a:11 brd ff:ff:ff:ff:ff:ff link-netnsid 0 2023-11-24T07:56:51.886 INFO:teuthology.orchestra.run.smithi076.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2023-11-24T07:56:51.886 INFO:teuthology.orchestra.run.smithi076.stdout: valid_lft forever preferred_lft forever 2023-11-24T07:56:51.886 INFO:teuthology.orchestra.run.smithi076.stdout: inet6 fe80::ccf2:81ff:fe8d:3a11/64 scope link 2023-11-24T07:56:51.886 INFO:teuthology.orchestra.run.smithi076.stdout: valid_lft forever preferred_lft forever 2023-11-24T07:56:51.899 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.3' with 192.168.0.2/16 2023-11-24T07:56:51.900 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T07:56:51.900 DEBUG:teuthology.orchestra.run.smithi076:> (cd / && exec bash -c ' 2023-11-24T07:56:51.900 DEBUG:teuthology.orchestra.run.smithi076:> set -e 2023-11-24T07:56:51.900 DEBUG:teuthology.orchestra.run.smithi076:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.3 type veth peer name brx.1 2023-11-24T07:56:51.900 DEBUG:teuthology.orchestra.run.smithi076:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip addr add 192.168.0.2/16 brd 192.168.255.255 dev veth0 2023-11-24T07:56:51.901 DEBUG:teuthology.orchestra.run.smithi076:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip link set veth0 up 2023-11-24T07:56:51.901 DEBUG:teuthology.orchestra.run.smithi076:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip link set lo up 2023-11-24T07:56:51.901 DEBUG:teuthology.orchestra.run.smithi076:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip route add default via 192.168.255.254 2023-11-24T07:56:51.901 DEBUG:teuthology.orchestra.run.smithi076:> ') 2023-11-24T07:56:52.119 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T07:56:52.119 DEBUG:teuthology.orchestra.run.smithi076:> (cd / && exec bash -c ' 2023-11-24T07:56:52.119 DEBUG:teuthology.orchestra.run.smithi076:> set -e 2023-11-24T07:56:52.119 DEBUG:teuthology.orchestra.run.smithi076:> sudo ip link set brx.1 up 2023-11-24T07:56:52.119 DEBUG:teuthology.orchestra.run.smithi076:> sudo ip link set dev brx.1 master ceph-brx 2023-11-24T07:56:52.119 DEBUG:teuthology.orchestra.run.smithi076:> ') 2023-11-24T07:56:52.225 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-11-24T07:56:52.225 DEBUG:teuthology.orchestra.run.smithi076:> mkdir -p -v /home/ubuntu/cephtest/mnt.3 2023-11-24T07:56:52.247 INFO:teuthology.orchestra.run.smithi076.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.3' 2023-11-24T07:56:52.247 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-11-24T07:56:52.247 DEBUG:teuthology.orchestra.run.smithi076:> chmod 0000 /home/ubuntu/cephtest/mnt.3 2023-11-24T07:56:52.304 INFO:tasks.cephfs.kernel_mount:mounting using device: 3@.cephfs=/ 2023-11-24T07:56:52.305 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T07:56:52.305 DEBUG:teuthology.orchestra.run.smithi076:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 3@.cephfs=/ /home/ubuntu/cephtest/mnt.3 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-11-24T07:56:52.390 INFO:teuthology.orchestra.run.smithi076.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-11-24T07:56:52.390 INFO:teuthology.orchestra.run.smithi076.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-11-24T07:56:52.419 INFO:teuthology.orchestra.run.smithi076.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-11-24T07:56:52.419 INFO:teuthology.orchestra.run.smithi076.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-11-24T07:56:52.419 INFO:teuthology.orchestra.run.smithi076.stdout:mount.ceph: resolved to: "172.21.15.59:3300,172.21.15.76:3300,172.21.15.76:3301" 2023-11-24T07:56:52.419 INFO:teuthology.orchestra.run.smithi076.stdout:mount.ceph: trying mount with new device syntax: 3@9f0e8741-b8f9-48d5-a2c3-09f45903303d.cephfs=/ 2023-11-24T07:56:52.420 INFO:teuthology.orchestra.run.smithi076.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=3,ms_mode=prefer-crc,key=3,mon_addr=172.21.15.59:3300/172.21.15.76:3300/172.21.15.76:3301" will pass to kernel 2023-11-24T07:56:52.420 INFO:teuthology.orchestra.run.smithi076.stdout:mount.ceph: v2 mount support check returned 2 2023-11-24T07:56:52.420 INFO:teuthology.orchestra.run.smithi076.stdout:mount.ceph: kernel does not support v2 syntax 2023-11-24T07:56:52.420 INFO:teuthology.orchestra.run.smithi076.stdout:mount.ceph: switching to using v1 address with old syntax 2023-11-24T07:56:52.494 INFO:teuthology.orchestra.run.smithi076.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-11-24T07:56:52.494 INFO:teuthology.orchestra.run.smithi076.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-11-24T07:56:52.495 INFO:teuthology.orchestra.run.smithi076.stdout:mount.ceph: resolved to: "172.21.15.59:3300,172.21.15.76:3300,172.21.15.76:3301" 2023-11-24T07:56:52.495 INFO:teuthology.orchestra.run.smithi076.stdout:mount.ceph: trying mount with new device syntax: 3@9f0e8741-b8f9-48d5-a2c3-09f45903303d.cephfs=/ 2023-11-24T07:56:52.495 INFO:teuthology.orchestra.run.smithi076.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=3,ms_mode=prefer-crc,key=3,mon_addr=172.21.15.59:3300/172.21.15.76:3300/172.21.15.76:3301" will pass to kernel 2023-11-24T07:56:52.495 INFO:teuthology.orchestra.run.smithi076.stdout:mount.ceph: v2 mount support check returned 2 2023-11-24T07:56:52.495 INFO:teuthology.orchestra.run.smithi076.stdout:mount.ceph: kernel does not support v2 syntax 2023-11-24T07:56:52.495 INFO:teuthology.orchestra.run.smithi076.stdout:mount.ceph: switching to using v1 address with old syntax 2023-11-24T07:56:52.495 INFO:teuthology.orchestra.run.smithi076.stdout:mount.ceph: resolved to: "172.21.15.59:6789,172.21.15.76:6789,172.21.15.76:6790" 2023-11-24T07:56:52.495 INFO:teuthology.orchestra.run.smithi076.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.59:6789,172.21.15.76:6789,172.21.15.76:6790:/ 2023-11-24T07:56:52.495 INFO:teuthology.orchestra.run.smithi076.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=3,key=3,mds_namespace=cephfs,fsid=9f0e8741-b8f9-48d5-a2c3-09f45903303d" will pass to kernel 2023-11-24T07:56:52.497 INFO:tasks.cephfs.kernel_mount:mount command passed 2023-11-24T07:56:52.497 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T07:56:52.498 DEBUG:teuthology.orchestra.run.smithi076:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.3 2023-11-24T07:56:52.524 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T07:56:52.524 DEBUG:teuthology.orchestra.run.smithi076:> (cd /home/ubuntu/cephtest/mnt.3 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-11-24T07:56:52.596 INFO:teuthology.orchestra.run.smithi076.stdout:client4698 2023-11-24T07:56:52.597 DEBUG:teuthology.orchestra.run.smithi076:> set -ex 2023-11-24T07:56:52.597 DEBUG:teuthology.orchestra.run.smithi076:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-24T07:56:52.654 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T07:56:52.654 DEBUG:teuthology.orchestra.run.smithi076:> (cd /home/ubuntu/cephtest/mnt.3 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-11-24T07:56:52.726 INFO:teuthology.orchestra.run.smithi076.stdout:client4698 2023-11-24T07:56:52.726 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T07:56:52.727 DEBUG:teuthology.orchestra.run.smithi076:> (cd /home/ubuntu/cephtest/mnt.3 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-11-24T07:56:52.798 INFO:teuthology.orchestra.run.smithi076.stdout:client4698 2023-11-24T07:56:52.798 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T07:56:52.798 DEBUG:teuthology.orchestra.run.smithi076:> (cd /home/ubuntu/cephtest/mnt.3 && exec bash -c 'sudo dd if=/sys/kernel/debug/ceph/9f0e8741-b8f9-48d5-a2c3-09f45903303d.client4698/status') 2023-11-24T07:56:52.878 INFO:teuthology.orchestra.run.smithi076.stdout:instance: client.4698 (3)172.21.15.76:0/3491119758 2023-11-24T07:56:52.878 INFO:teuthology.orchestra.run.smithi076.stdout:blocklisted: false 2023-11-24T07:56:52.879 INFO:teuthology.orchestra.run.smithi076.stderr:0+1 records in 2023-11-24T07:56:52.879 INFO:teuthology.orchestra.run.smithi076.stderr:0+1 records out 2023-11-24T07:56:52.879 INFO:teuthology.orchestra.run.smithi076.stderr:70 bytes copied, 2.2138e-05 s, 3.2 MB/s 2023-11-24T07:56:52.880 INFO:teuthology.run_tasks:Running task cephfs_test_runner... 2023-11-24T07:56:52.896 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-24T07:56:53.042 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:53.034+0000 7f246cbf1700 1 Processor -- start 2023-11-24T07:56:53.042 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:53.034+0000 7f246cbf1700 1 -- start start 2023-11-24T07:56:53.042 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:53.035+0000 7f246cbf1700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f246815e1c0 0x7f246806eef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:56:53.042 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:53.035+0000 7f246cbf1700 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f24680b0e90 0x7f246806f540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:56:53.042 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:53.035+0000 7f246cbf1700 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f24680d1230 con 0x7f246806fa80 2023-11-24T07:56:53.042 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:53.035+0000 7f246cbf1700 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f246806ff10 con 0x7f246815e1c0 2023-11-24T07:56:53.043 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:53.035+0000 7f246cbf1700 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f24680d0c70 con 0x7f24680b0e90 2023-11-24T07:56:53.043 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:53.035+0000 7f246659c700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f246815e1c0 0x7f246806eef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:56:53.043 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:53.035+0000 7f2466d9d700 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7f246806fa80 0x7f2468161d90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:54078/0 (socket says 172.21.15.59:54078) 2023-11-24T07:56:53.043 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:53.035+0000 7f2466d9d700 1 -- 172.21.15.59:0/1782264644 learned_addr learned my addr 172.21.15.59:0/1782264644 (peer_addr_for_me v1:172.21.15.59:0/0) 2023-11-24T07:56:53.043 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:53.036+0000 7f2465d9b700 1 --2- 172.21.15.59:0/1782264644 >> v2:172.21.15.76:3300/0 conn(0x7f24680b0e90 0x7f246806f540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:56:53.043 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:53.036+0000 7f246659c700 1 -- 172.21.15.59:0/1782264644 >> v2:172.21.15.76:3300/0 conn(0x7f24680b0e90 msgr2=0x7f246806f540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T07:56:53.043 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:53.036+0000 7f246659c700 1 --2- 172.21.15.59:0/1782264644 >> v2:172.21.15.76:3300/0 conn(0x7f24680b0e90 0x7f246806f540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:56:53.043 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:53.036+0000 7f246659c700 1 -- 172.21.15.59:0/1782264644 >> v1:172.21.15.59:6789/0 conn(0x7f246806fa80 legacy=0x7f2468161d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:56:53.043 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:53.036+0000 7f246659c700 1 -- 172.21.15.59:0/1782264644 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f24680d0f50 con 0x7f246815e1c0 2023-11-24T07:56:53.043 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:53.036+0000 7f246559a700 1 -- 172.21.15.59:0/1782264644 <== mon.0 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4220408473 0 0) 0x7f24680d1230 con 0x7f246806fa80 2023-11-24T07:56:53.043 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:53.036+0000 7f2465d9b700 1 --2- 172.21.15.59:0/1782264644 >> v2:172.21.15.76:3300/0 conn(0x7f24680b0e90 0x7f246806f540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-24T07:56:53.043 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:53.036+0000 7f246659c700 1 --2- 172.21.15.59:0/1782264644 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f246815e1c0 0x7f246806eef0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f2468071ba0 tx=0x7f245c009b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5807fda08919d2ca server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:56:53.044 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:53.036+0000 7f246559a700 1 -- 172.21.15.59:0/1782264644 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f245c011830 con 0x7f246815e1c0 2023-11-24T07:56:53.044 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:53.036+0000 7f246559a700 1 -- 172.21.15.59:0/1782264644 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f245c011990 con 0x7f246815e1c0 2023-11-24T07:56:53.044 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:53.036+0000 7f246559a700 1 -- 172.21.15.59:0/1782264644 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f245c011d20 con 0x7f246815e1c0 2023-11-24T07:56:53.044 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:53.037+0000 7f246cbf1700 1 -- 172.21.15.59:0/1782264644 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f246815e1c0 msgr2=0x7f246806eef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:56:53.044 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:53.037+0000 7f246cbf1700 1 --2- 172.21.15.59:0/1782264644 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f246815e1c0 0x7f246806eef0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f2468071ba0 tx=0x7f245c009b20 comp rx=0 tx=0).stop 2023-11-24T07:56:53.044 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:53.037+0000 7f246cbf1700 1 -- 172.21.15.59:0/1782264644 shutdown_connections 2023-11-24T07:56:53.044 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:53.037+0000 7f246cbf1700 1 --2- 172.21.15.59:0/1782264644 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f246815e1c0 0x7f246806eef0 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:56:53.044 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:53.037+0000 7f246cbf1700 1 --2- 172.21.15.59:0/1782264644 >> v2:172.21.15.76:3300/0 conn(0x7f24680b0e90 0x7f246806f540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:56:53.044 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:53.037+0000 7f246cbf1700 1 -- 172.21.15.59:0/1782264644 >> 172.21.15.59:0/1782264644 conn(0x7f24680b3720 msgr2=0x7f24680b3b30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:56:53.044 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:53.037+0000 7f246cbf1700 1 -- 172.21.15.59:0/1782264644 shutdown_connections 2023-11-24T07:56:53.044 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:53.037+0000 7f246cbf1700 1 -- 172.21.15.59:0/1782264644 wait complete. 2023-11-24T07:56:53.044 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:53.038+0000 7f246cbf1700 1 Processor -- start 2023-11-24T07:56:53.044 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:53.038+0000 7f246cbf1700 1 -- start start 2023-11-24T07:56:53.045 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:53.039+0000 7f246cbf1700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f246806fa80 0x7f24681f42a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:56:53.045 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:53.039+0000 7f246cbf1700 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f24680b0e90 0x7f246809eff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:56:53.045 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:53.039+0000 7f246cbf1700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f246815e1c0 0x7f246809f530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:56:53.045 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:53.039+0000 7f246cbf1700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24681f47e0 con 0x7f246815e1c0 2023-11-24T07:56:53.045 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:53.039+0000 7f246cbf1700 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24680b6e20 con 0x7f246806fa80 2023-11-24T07:56:53.045 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:53.039+0000 7f246cbf1700 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f24680b7100 con 0x7f24680b0e90 2023-11-24T07:56:53.045 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:53.039+0000 7f2466d9d700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f246815e1c0 0x7f246809f530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:56:53.045 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:53.039+0000 7f2466d9d700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f246815e1c0 0x7f246809f530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:50610/0 (socket says 172.21.15.59:50610) 2023-11-24T07:56:53.045 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:53.039+0000 7f2466d9d700 1 -- 172.21.15.59:0/3888729698 learned_addr learned my addr 172.21.15.59:0/3888729698 (peer_addr_for_me v2:172.21.15.59:0/0) 2023-11-24T07:56:53.045 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:53.039+0000 7f246659c700 1 --2- 172.21.15.59:0/3888729698 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f246806fa80 0x7f24681f42a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:56:53.045 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:53.039+0000 7f2465d9b700 1 --2- 172.21.15.59:0/3888729698 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f24680b0e90 0x7f246809eff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:56:53.045 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:53.039+0000 7f2466d9d700 1 -- 172.21.15.59:0/3888729698 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f24680b0e90 msgr2=0x7f246809eff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:56:53.046 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:53.039+0000 7f2466d9d700 1 --2- 172.21.15.59:0/3888729698 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f24680b0e90 0x7f246809eff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:56:53.046 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:53.039+0000 7f2466d9d700 1 -- 172.21.15.59:0/3888729698 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f246806fa80 msgr2=0x7f24681f42a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:56:53.046 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:53.039+0000 7f2466d9d700 1 --2- 172.21.15.59:0/3888729698 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f246806fa80 0x7f24681f42a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:56:53.046 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:53.039+0000 7f2466d9d700 1 -- 172.21.15.59:0/3888729698 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f245c0094a0 con 0x7f246815e1c0 2023-11-24T07:56:53.046 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:53.040+0000 7f2466d9d700 1 --2- 172.21.15.59:0/3888729698 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f246815e1c0 0x7f246809f530 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f246000cca0 tx=0x7f246000a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:56:53.046 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:53.040+0000 7f2456ffd700 1 -- 172.21.15.59:0/3888729698 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f246000ab70 con 0x7f246815e1c0 2023-11-24T07:56:53.046 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:53.040+0000 7f2456ffd700 1 -- 172.21.15.59:0/3888729698 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f246000acd0 con 0x7f246815e1c0 2023-11-24T07:56:53.046 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:53.040+0000 7f246cbf1700 1 -- 172.21.15.59:0/3888729698 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f24680b73e0 con 0x7f246815e1c0 2023-11-24T07:56:53.046 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:53.040+0000 7f2456ffd700 1 -- 172.21.15.59:0/3888729698 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2460007310 con 0x7f246815e1c0 2023-11-24T07:56:53.046 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:53.040+0000 7f246cbf1700 1 -- 172.21.15.59:0/3888729698 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f24681f49e0 con 0x7f246815e1c0 2023-11-24T07:56:53.047 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:53.041+0000 7f246cbf1700 1 -- 172.21.15.59:0/3888729698 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2468068670 con 0x7f246815e1c0 2023-11-24T07:56:53.047 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:53.041+0000 7f2456ffd700 1 -- 172.21.15.59:0/3888729698 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f2460007470 con 0x7f246815e1c0 2023-11-24T07:56:53.047 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:53.041+0000 7f2456ffd700 1 --2- 172.21.15.59:0/3888729698 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f244806af00 0x7f244806d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:56:53.047 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:53.042+0000 7f2456ffd700 1 -- 172.21.15.59:0/3888729698 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f2460089370 con 0x7f246815e1c0 2023-11-24T07:56:53.048 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:53.042+0000 7f246659c700 1 --2- 172.21.15.59:0/3888729698 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f244806af00 0x7f244806d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:56:53.048 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:53.042+0000 7f246659c700 1 --2- 172.21.15.59:0/3888729698 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f244806af00 0x7f244806d3c0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f245c000c00 tx=0x7f245c01c040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:56:53.048 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:53.046+0000 7f2456ffd700 1 -- 172.21.15.59:0/3888729698 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f2460054420 con 0x7f246815e1c0 2023-11-24T07:56:53.247 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:53.246+0000 7f246cbf1700 1 -- 172.21.15.59:0/3888729698 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f24681f4c00 con 0x7f246815e1c0 2023-11-24T07:56:53.248 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:53.247+0000 7f2456ffd700 1 -- 172.21.15.59:0/3888729698 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) v1 ==== 74+0+16437 (secure 0 0 0) 0x7f2460057a40 con 0x7f246815e1c0 2023-11-24T07:56:53.249 INFO:teuthology.orchestra.run.smithi059.stdout: 2023-11-24T07:56:53.249 INFO:teuthology.orchestra.run.smithi059.stdout:{"epoch":20,"fsid":"9f0e8741-b8f9-48d5-a2c3-09f45903303d","created":"2023-11-24T07:56:06.552729+0000","modified":"2023-11-24T07:56:25.705091+0000","last_up_change":"2023-11-24T07:56:17.372352+0000","last_in_change":"2023-11-24T07:56:09.073250+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-24T07:56:18.352823+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-11-24T07:56:22.185140+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-11-24T07:56:23.084326+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"6d8f9e9b-1782-4e41-928b-a39f2a8ebc06","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6800","nonce":3373878772},{"type":"v1","addr":"172.21.15.59:6801","nonce":3373878772}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6802","nonce":3373878772},{"type":"v1","addr":"172.21.15.59:6803","nonce":3373878772}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6806","nonce":3373878772},{"type":"v1","addr":"172.21.15.59:6807","nonce":3373878772}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6804","nonce":3373878772},{"type":"v1","addr":"172.21.15.59:6805","nonce":3373878772}]},"public_addr":"172.21.15.59:6801/3373878772","cluster_addr":"172.21.15.59:6803/3373878772","heartbeat_back_addr":"172.21.15.59:6807/3373878772","heartbeat_front_addr":"172.21.15.59:6805/3373878772","state":["exists","up"]},{"osd":1,"uuid":"e8d882aa-52e1-4c04-9e1e-c44ef9204a5f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6824","nonce":648258563},{"type":"v1","addr":"172.21.15.59:6825","nonce":648258563}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6826","nonce":648258563},{"type":"v1","addr":"172.21.15.59:6827","nonce":648258563}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6830","nonce":648258563},{"type":"v1","addr":"172.21.15.59:6831","nonce":648258563}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6828","nonce":648258563},{"type":"v1","addr":"172.21.15.59:6829","nonce":648258563}]},"public_addr":"172.21.15.59:6825/648258563","cluster_addr":"172.21.15.59:6827/648258563","heartbeat_back_addr":"172.21.15.59:6831/648258563","heartbeat_front_addr":"172.21.15.59:6829/648258563","state":["exists","up"]},{"osd":2,"uuid":"69eebfa0-695b-4b21-90b9-8a3537168a01","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6808","nonce":1840430654},{"type":"v1","addr":"172.21.15.59:6809","nonce":1840430654}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6810","nonce":1840430654},{"type":"v1","addr":"172.21.15.59:6811","nonce":1840430654}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6814","nonce":1840430654},{"type":"v1","addr":"172.21.15.59:6815","nonce":1840430654}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6812","nonce":1840430654},{"type":"v1","addr":"172.21.15.59:6813","nonce":1840430654}]},"public_addr":"172.21.15.59:6809/1840430654","cluster_addr":"172.21.15.59:6811/1840430654","heartbeat_back_addr":"172.21.15.59:6815/1840430654","heartbeat_front_addr":"172.21.15.59:6813/1840430654","state":["exists","up"]},{"osd":3,"uuid":"96fd77fc-f911-46ff-9445-6f0be2daa8f4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6816","nonce":2504025850},{"type":"v1","addr":"172.21.15.59:6817","nonce":2504025850}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6818","nonce":2504025850},{"type":"v1","addr":"172.21.15.59:6819","nonce":2504025850}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6822","nonce":2504025850},{"type":"v1","addr":"172.21.15.59:6823","nonce":2504025850}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6820","nonce":2504025850},{"type":"v1","addr":"172.21.15.59:6821","nonce":2504025850}]},"public_addr":"172.21.15.59:6817/2504025850","cluster_addr":"172.21.15.59:6819/2504025850","heartbeat_back_addr":"172.21.15.59:6823/2504025850","heartbeat_front_addr":"172.21.15.59:6821/2504025850","state":["exists","up"]},{"osd":4,"uuid":"7ac29fca-b5b2-4c95-9d69-2f164cbbc708","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6816","nonce":2382463431},{"type":"v1","addr":"172.21.15.76:6818","nonce":2382463431}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6820","nonce":2382463431},{"type":"v1","addr":"172.21.15.76:6822","nonce":2382463431}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6828","nonce":2382463431},{"type":"v1","addr":"172.21.15.76:6830","nonce":2382463431}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6824","nonce":2382463431},{"type":"v1","addr":"172.21.15.76:6826","nonce":2382463431}]},"public_addr":"172.21.15.76:6818/2382463431","cluster_addr":"172.21.15.76:6822/2382463431","heartbeat_back_addr":"172.21.15.76:6830/2382463431","heartbeat_front_addr":"172.21.15.76:6826/2382463431","state":["exists","up"]},{"osd":5,"uuid":"485bf711-3851-4238-a4f4-51d52e0a9c58","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6800","nonce":329334308},{"type":"v1","addr":"172.21.15.76:6802","nonce":329334308}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6805","nonce":329334308},{"type":"v1","addr":"172.21.15.76:6807","nonce":329334308}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6814","nonce":329334308},{"type":"v1","addr":"172.21.15.76:6815","nonce":329334308}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6810","nonce":329334308},{"type":"v1","addr":"172.21.15.76:6812","nonce":329334308}]},"public_addr":"172.21.15.76:6802/329334308","cluster_addr":"172.21.15.76:6807/329334308","heartbeat_back_addr":"172.21.15.76:6815/329334308","heartbeat_front_addr":"172.21.15.76:6812/329334308","state":["exists","up"]},{"osd":6,"uuid":"72b3f2fc-1e4a-4e32-968d-003647a0286f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6801","nonce":3946579410},{"type":"v1","addr":"172.21.15.76:6803","nonce":3946579410}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6804","nonce":3946579410},{"type":"v1","addr":"172.21.15.76:6806","nonce":3946579410}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6811","nonce":3946579410},{"type":"v1","addr":"172.21.15.76:6813","nonce":3946579410}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6808","nonce":3946579410},{"type":"v1","addr":"172.21.15.76:6809","nonce":3946579410}]},"public_addr":"172.21.15.76:6803/3946579410","cluster_addr":"172.21.15.76:6806/3946579410","heartbeat_back_addr":"172.21.15.76:6813/3946579410","heartbeat_front_addr":"172.21.15.76:6809/3946579410","state":["exists","up"]},{"osd":7,"uuid":"1ddceb90-6c5d-4d1a-ad56-99cb877f8c70","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6817","nonce":841761538},{"type":"v1","addr":"172.21.15.76:6819","nonce":841761538}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6821","nonce":841761538},{"type":"v1","addr":"172.21.15.76:6823","nonce":841761538}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6829","nonce":841761538},{"type":"v1","addr":"172.21.15.76:6831","nonce":841761538}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6825","nonce":841761538},{"type":"v1","addr":"172.21.15.76:6827","nonce":841761538}]},"public_addr":"172.21.15.76:6819/841761538","cluster_addr":"172.21.15.76:6823/841761538","heartbeat_back_addr":"172.21.15.76:6831/841761538","heartbeat_front_addr":"172.21.15.76:6827/841761538","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T07:56:14.294838+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T07:56:14.270779+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T07:56:14.231809+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T07:56:14.260983+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T07:56:14.302997+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T07:56:14.258600+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T07:56:14.251413+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T07:56:14.304727+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-24T07:56:53.251 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:53.249+0000 7f246cbf1700 1 -- 172.21.15.59:0/3888729698 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f244806af00 msgr2=0x7f244806d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:56:53.251 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:53.249+0000 7f246cbf1700 1 --2- 172.21.15.59:0/3888729698 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f244806af00 0x7f244806d3c0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f245c000c00 tx=0x7f245c01c040 comp rx=0 tx=0).stop 2023-11-24T07:56:53.251 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:53.249+0000 7f246cbf1700 1 -- 172.21.15.59:0/3888729698 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f246815e1c0 msgr2=0x7f246809f530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:56:53.251 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:53.249+0000 7f246cbf1700 1 --2- 172.21.15.59:0/3888729698 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f246815e1c0 0x7f246809f530 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f246000cca0 tx=0x7f246000a060 comp rx=0 tx=0).stop 2023-11-24T07:56:53.251 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:53.249+0000 7f246cbf1700 1 -- 172.21.15.59:0/3888729698 shutdown_connections 2023-11-24T07:56:53.251 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:53.249+0000 7f246cbf1700 1 --2- 172.21.15.59:0/3888729698 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f244806af00 0x7f244806d3c0 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:56:53.251 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:53.249+0000 7f246cbf1700 1 --2- 172.21.15.59:0/3888729698 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f246815e1c0 0x7f246809f530 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:56:53.251 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:53.249+0000 7f246cbf1700 1 --2- 172.21.15.59:0/3888729698 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f246806fa80 0x7f24681f42a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:56:53.251 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:53.249+0000 7f246cbf1700 1 --2- 172.21.15.59:0/3888729698 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f24680b0e90 0x7f246809eff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:56:53.252 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:53.249+0000 7f246cbf1700 1 -- 172.21.15.59:0/3888729698 >> 172.21.15.59:0/3888729698 conn(0x7f24680b3720 msgr2=0x7f246815bb50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:56:53.252 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:53.249+0000 7f246cbf1700 1 -- 172.21.15.59:0/3888729698 shutdown_connections 2023-11-24T07:56:53.252 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:53.249+0000 7f246cbf1700 1 -- 172.21.15.59:0/3888729698 wait complete. 2023-11-24T07:56:53.263 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-24T07:56:18.352823+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-24T07:56:22.185140+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-11-24T07:56:23.084326+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-24T07:56:53.264 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-24T07:56:53.392 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:53.391+0000 7f68f7211700 1 Processor -- start 2023-11-24T07:56:53.392 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:53.391+0000 7f68f7211700 1 -- start start 2023-11-24T07:56:53.395 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:53.391+0000 7f68f7211700 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f68f006fc50 0x7f68f006eef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:56:53.395 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:53.391+0000 7f68f7211700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f68f00b0e90 0x7f68f006ca10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:56:53.395 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:53.391+0000 7f68f7211700 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f68f00d1230 con 0x7f68f015e1c0 2023-11-24T07:56:53.395 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:53.391+0000 7f68f7211700 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f68f00d0c70 con 0x7f68f00b0e90 2023-11-24T07:56:53.395 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:53.392+0000 7f68f7211700 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f68f00d0f50 con 0x7f68f006fc50 2023-11-24T07:56:53.395 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:53.392+0000 7f68f57ae700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f68f00b0e90 0x7f68f006ca10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:56:53.395 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:53.392+0000 7f68f4fad700 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7f68f015e1c0 0x7f68f006f540 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:54088/0 (socket says 172.21.15.59:54088) 2023-11-24T07:56:53.395 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:53.392+0000 7f68f4fad700 1 -- 172.21.15.59:0/1218715499 learned_addr learned my addr 172.21.15.59:0/1218715499 (peer_addr_for_me v1:172.21.15.59:0/0) 2023-11-24T07:56:53.395 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:53.392+0000 7f68effff700 1 --2- 172.21.15.59:0/1218715499 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f68f006fc50 0x7f68f006eef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:56:53.395 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:53.392+0000 7f68f57ae700 1 -- 172.21.15.59:0/1218715499 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f68f006fc50 msgr2=0x7f68f006eef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T07:56:53.395 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:53.392+0000 7f68f57ae700 1 --2- 172.21.15.59:0/1218715499 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f68f006fc50 0x7f68f006eef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:56:53.396 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:53.392+0000 7f68f57ae700 1 -- 172.21.15.59:0/1218715499 >> v1:172.21.15.59:6789/0 conn(0x7f68f015e1c0 legacy=0x7f68f006f540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:56:53.396 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:53.392+0000 7f68ef7fe700 1 -- 172.21.15.59:0/1218715499 <== mon.0 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 813914308 0 0) 0x7f68f00d1230 con 0x7f68f015e1c0 2023-11-24T07:56:53.396 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:53.392+0000 7f68f57ae700 1 -- 172.21.15.59:0/1218715499 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f68f006c210 con 0x7f68f00b0e90 2023-11-24T07:56:53.396 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:53.392+0000 7f68f57ae700 1 --2- 172.21.15.59:0/1218715499 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f68f00b0e90 0x7f68f006ca10 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f68e8009d90 tx=0x7f68e800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=82aae54338baa4f server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:56:53.396 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:53.392+0000 7f68ef7fe700 1 -- 172.21.15.59:0/1218715499 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68e800e070 con 0x7f68f00b0e90 2023-11-24T07:56:53.396 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:53.392+0000 7f68ef7fe700 1 -- 172.21.15.59:0/1218715499 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f68e800bb20 con 0x7f68f00b0e90 2023-11-24T07:56:53.396 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:53.392+0000 7f68ef7fe700 1 -- 172.21.15.59:0/1218715499 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68e8012430 con 0x7f68f00b0e90 2023-11-24T07:56:53.396 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:53.393+0000 7f68f7211700 1 -- 172.21.15.59:0/1218715499 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f68f00b0e90 msgr2=0x7f68f006ca10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:56:53.396 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:53.393+0000 7f68f7211700 1 --2- 172.21.15.59:0/1218715499 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f68f00b0e90 0x7f68f006ca10 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f68e8009d90 tx=0x7f68e800b330 comp rx=0 tx=0).stop 2023-11-24T07:56:53.396 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:53.393+0000 7f68f7211700 1 -- 172.21.15.59:0/1218715499 shutdown_connections 2023-11-24T07:56:53.396 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:53.393+0000 7f68f7211700 1 --2- 172.21.15.59:0/1218715499 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f68f00b0e90 0x7f68f006ca10 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:56:53.396 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:53.393+0000 7f68f7211700 1 --2- 172.21.15.59:0/1218715499 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f68f006fc50 0x7f68f006eef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:56:53.396 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:53.393+0000 7f68f7211700 1 -- 172.21.15.59:0/1218715499 >> 172.21.15.59:0/1218715499 conn(0x7f68f00b3720 msgr2=0x7f68f00b3b30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:56:53.396 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:53.393+0000 7f68f7211700 1 -- 172.21.15.59:0/1218715499 shutdown_connections 2023-11-24T07:56:53.397 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:53.393+0000 7f68f7211700 1 -- 172.21.15.59:0/1218715499 wait complete. 2023-11-24T07:56:53.399 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:53.394+0000 7f68f7211700 1 Processor -- start 2023-11-24T07:56:53.399 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:53.394+0000 7f68f7211700 1 -- start start 2023-11-24T07:56:53.399 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:53.395+0000 7f68f7211700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f68f006fc50 0x7f68f0157f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:56:53.399 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:53.395+0000 7f68f7211700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f68f00b0e90 0x7f68f009f320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:56:53.399 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:53.395+0000 7f68f7211700 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f68f015e1c0 0x7f68f00d1b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:56:53.400 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:53.395+0000 7f68f7211700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68f009fb70 con 0x7f68f00b0e90 2023-11-24T07:56:53.400 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:53.395+0000 7f68f7211700 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68f00b5a00 con 0x7f68f006fc50 2023-11-24T07:56:53.400 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:53.395+0000 7f68f7211700 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f68f00b5ce0 con 0x7f68f015e1c0 2023-11-24T07:56:53.400 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:53.395+0000 7f68effff700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f68f00b0e90 0x7f68f009f320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:56:53.400 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:53.395+0000 7f68effff700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f68f00b0e90 0x7f68f009f320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:50622/0 (socket says 172.21.15.59:50622) 2023-11-24T07:56:53.400 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:53.395+0000 7f68effff700 1 -- 172.21.15.59:0/2278171010 learned_addr learned my addr 172.21.15.59:0/2278171010 (peer_addr_for_me v2:172.21.15.59:0/0) 2023-11-24T07:56:53.400 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:53.395+0000 7f68f4fad700 1 --2- 172.21.15.59:0/2278171010 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f68f006fc50 0x7f68f0157f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:56:53.400 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:53.395+0000 7f68f57ae700 1 --2- 172.21.15.59:0/2278171010 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f68f015e1c0 0x7f68f00d1b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:56:53.400 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:53.395+0000 7f68effff700 1 -- 172.21.15.59:0/2278171010 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f68f015e1c0 msgr2=0x7f68f00d1b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:56:53.400 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:53.396+0000 7f68effff700 1 --2- 172.21.15.59:0/2278171010 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f68f015e1c0 0x7f68f00d1b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:56:53.400 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:53.396+0000 7f68effff700 1 -- 172.21.15.59:0/2278171010 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f68f006fc50 msgr2=0x7f68f0157f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:56:53.400 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:53.396+0000 7f68effff700 1 --2- 172.21.15.59:0/2278171010 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f68f006fc50 0x7f68f0157f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:56:53.400 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:53.396+0000 7f68effff700 1 -- 172.21.15.59:0/2278171010 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f68e8009a70 con 0x7f68f00b0e90 2023-11-24T07:56:53.400 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:53.396+0000 7f68f57ae700 1 --2- 172.21.15.59:0/2278171010 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f68f015e1c0 0x7f68f00d1b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-24T07:56:53.401 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:53.396+0000 7f68effff700 1 --2- 172.21.15.59:0/2278171010 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f68f00b0e90 0x7f68f009f320 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f68dc00e9b0 tx=0x7f68dc00ecc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:56:53.401 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:53.396+0000 7f68ed7fa700 1 -- 172.21.15.59:0/2278171010 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68dc009710 con 0x7f68f00b0e90 2023-11-24T07:56:53.401 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:53.396+0000 7f68ed7fa700 1 -- 172.21.15.59:0/2278171010 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f68dc00c850 con 0x7f68f00b0e90 2023-11-24T07:56:53.401 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:53.396+0000 7f68ed7fa700 1 -- 172.21.15.59:0/2278171010 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68dc00cbc0 con 0x7f68f00b0e90 2023-11-24T07:56:53.401 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:53.396+0000 7f68f7211700 1 -- 172.21.15.59:0/2278171010 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f68f00b5fc0 con 0x7f68f00b0e90 2023-11-24T07:56:53.401 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:53.396+0000 7f68f7211700 1 -- 172.21.15.59:0/2278171010 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f68f0158450 con 0x7f68f00b0e90 2023-11-24T07:56:53.401 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:53.397+0000 7f68f7211700 1 -- 172.21.15.59:0/2278171010 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f68f0075480 con 0x7f68f00b0e90 2023-11-24T07:56:53.401 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:53.397+0000 7f68ed7fa700 1 -- 172.21.15.59:0/2278171010 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f68dc00cd20 con 0x7f68f00b0e90 2023-11-24T07:56:53.402 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:53.398+0000 7f68ed7fa700 1 --2- 172.21.15.59:0/2278171010 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f68d406aed0 0x7f68d406d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:56:53.402 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:53.398+0000 7f68ed7fa700 1 -- 172.21.15.59:0/2278171010 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f68dc089620 con 0x7f68f00b0e90 2023-11-24T07:56:53.402 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:53.398+0000 7f68f4fad700 1 --2- 172.21.15.59:0/2278171010 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f68d406aed0 0x7f68d406d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:56:53.402 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:53.398+0000 7f68f4fad700 1 --2- 172.21.15.59:0/2278171010 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f68d406aed0 0x7f68d406d390 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f68e4000f60 tx=0x7f68e4007e70 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:56:53.404 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:53.402+0000 7f68ed7fa700 1 -- 172.21.15.59:0/2278171010 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f68dc0546d0 con 0x7f68f00b0e90 2023-11-24T07:56:53.606 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:53.604+0000 7f68f7211700 1 -- 172.21.15.59:0/2278171010 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f68f00b4240 con 0x7f68f00b0e90 2023-11-24T07:56:53.606 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:53.604+0000 7f68ed7fa700 1 -- 172.21.15.59:0/2278171010 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v20) v1 ==== 93+0+10 (secure 0 0 0) 0x7f68dc057cf0 con 0x7f68f00b0e90 2023-11-24T07:56:53.606 INFO:teuthology.orchestra.run.smithi059.stdout:pg_num: 1 2023-11-24T07:56:53.608 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:53.606+0000 7f68f7211700 1 -- 172.21.15.59:0/2278171010 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f68d406aed0 msgr2=0x7f68d406d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:56:53.608 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:53.606+0000 7f68f7211700 1 --2- 172.21.15.59:0/2278171010 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f68d406aed0 0x7f68d406d390 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f68e4000f60 tx=0x7f68e4007e70 comp rx=0 tx=0).stop 2023-11-24T07:56:53.608 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:53.606+0000 7f68f7211700 1 -- 172.21.15.59:0/2278171010 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f68f00b0e90 msgr2=0x7f68f009f320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:56:53.608 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:53.606+0000 7f68f7211700 1 --2- 172.21.15.59:0/2278171010 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f68f00b0e90 0x7f68f009f320 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f68dc00e9b0 tx=0x7f68dc00ecc0 comp rx=0 tx=0).stop 2023-11-24T07:56:53.608 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:53.606+0000 7f68f7211700 1 -- 172.21.15.59:0/2278171010 shutdown_connections 2023-11-24T07:56:53.609 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:53.606+0000 7f68f7211700 1 --2- 172.21.15.59:0/2278171010 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f68d406aed0 0x7f68d406d390 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:56:53.609 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:53.606+0000 7f68f7211700 1 --2- 172.21.15.59:0/2278171010 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f68f015e1c0 0x7f68f00d1b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:56:53.609 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:53.606+0000 7f68f7211700 1 --2- 172.21.15.59:0/2278171010 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f68f006fc50 0x7f68f0157f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:56:53.609 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:53.606+0000 7f68f7211700 1 --2- 172.21.15.59:0/2278171010 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f68f00b0e90 0x7f68f009f320 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:56:53.609 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:53.606+0000 7f68f7211700 1 -- 172.21.15.59:0/2278171010 >> 172.21.15.59:0/2278171010 conn(0x7f68f00b3720 msgr2=0x7f68f0070190 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:56:53.609 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:53.606+0000 7f68f7211700 1 -- 172.21.15.59:0/2278171010 shutdown_connections 2023-11-24T07:56:53.609 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:53.606+0000 7f68f7211700 1 -- 172.21.15.59:0/2278171010 wait complete. 2023-11-24T07:56:53.620 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-24T07:56:53.748 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:53.746+0000 7f812c366700 1 Processor -- start 2023-11-24T07:56:53.748 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:53.746+0000 7f812c366700 1 -- start start 2023-11-24T07:56:53.750 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:53.747+0000 7f812c366700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f8124145490 0x7f812406ed80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:56:53.750 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:53.747+0000 7f812c366700 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f812406b6e0 0x7f81240cef40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:56:53.750 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:53.747+0000 7f812c366700 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f81241601a0 con 0x7f812406f2c0 2023-11-24T07:56:53.750 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:53.747+0000 7f812c366700 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f81240d18c0 con 0x7f8124145490 2023-11-24T07:56:53.750 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:53.747+0000 7f812c366700 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f812406f8d0 con 0x7f812406b6e0 2023-11-24T07:56:53.750 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:53.747+0000 7f812a102700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f8124145490 0x7f812406ed80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:56:53.750 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:53.747+0000 7f8129901700 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7f812406f2c0 0x7f812406afd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:54102/0 (socket says 172.21.15.59:54102) 2023-11-24T07:56:53.750 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:53.747+0000 7f8129901700 1 -- 172.21.15.59:0/383379676 learned_addr learned my addr 172.21.15.59:0/383379676 (peer_addr_for_me v1:172.21.15.59:0/0) 2023-11-24T07:56:53.750 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:53.748+0000 7f812a903700 1 --2- 172.21.15.59:0/383379676 >> v2:172.21.15.76:3300/0 conn(0x7f812406b6e0 0x7f81240cef40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:56:53.750 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:53.748+0000 7f812a102700 1 -- 172.21.15.59:0/383379676 >> v2:172.21.15.76:3300/0 conn(0x7f812406b6e0 msgr2=0x7f81240cef40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T07:56:53.751 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:53.748+0000 7f812a102700 1 --2- 172.21.15.59:0/383379676 >> v2:172.21.15.76:3300/0 conn(0x7f812406b6e0 0x7f81240cef40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:56:53.751 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:53.748+0000 7f812a102700 1 -- 172.21.15.59:0/383379676 >> v1:172.21.15.59:6789/0 conn(0x7f812406f2c0 legacy=0x7f812406afd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:56:53.751 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:53.748+0000 7f8129100700 1 -- 172.21.15.59:0/383379676 <== mon.0 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1242534530 0 0) 0x7f81241601a0 con 0x7f812406f2c0 2023-11-24T07:56:53.751 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:53.748+0000 7f812a102700 1 -- 172.21.15.59:0/383379676 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f812415f870 con 0x7f8124145490 2023-11-24T07:56:53.751 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:53.748+0000 7f812a102700 1 --2- 172.21.15.59:0/383379676 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8124145490 0x7f812406ed80 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f81240b37e0 tx=0x7f8114009b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=63d8d03dcf3d58f0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:56:53.751 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:53.748+0000 7f8129100700 1 -- 172.21.15.59:0/383379676 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8114011830 con 0x7f8124145490 2023-11-24T07:56:53.751 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:53.748+0000 7f8129100700 1 -- 172.21.15.59:0/383379676 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8114011990 con 0x7f8124145490 2023-11-24T07:56:53.751 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:53.748+0000 7f8129100700 1 -- 172.21.15.59:0/383379676 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8114011d20 con 0x7f8124145490 2023-11-24T07:56:53.753 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:53.749+0000 7f812c366700 1 -- 172.21.15.59:0/383379676 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8124145490 msgr2=0x7f812406ed80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:56:53.754 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:53.749+0000 7f812c366700 1 --2- 172.21.15.59:0/383379676 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8124145490 0x7f812406ed80 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f81240b37e0 tx=0x7f8114009b20 comp rx=0 tx=0).stop 2023-11-24T07:56:53.754 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:53.749+0000 7f812c366700 1 -- 172.21.15.59:0/383379676 shutdown_connections 2023-11-24T07:56:53.754 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:53.749+0000 7f812c366700 1 --2- 172.21.15.59:0/383379676 >> v2:172.21.15.76:3300/0 conn(0x7f812406b6e0 0x7f81240cef40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:56:53.754 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:53.749+0000 7f812c366700 1 --2- 172.21.15.59:0/383379676 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8124145490 0x7f812406ed80 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:56:53.754 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:53.749+0000 7f812c366700 1 -- 172.21.15.59:0/383379676 >> 172.21.15.59:0/383379676 conn(0x7f81240b2740 msgr2=0x7f81240b2b50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:56:53.754 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:53.749+0000 7f812c366700 1 -- 172.21.15.59:0/383379676 shutdown_connections 2023-11-24T07:56:53.754 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:53.749+0000 7f812c366700 1 -- 172.21.15.59:0/383379676 wait complete. 2023-11-24T07:56:53.754 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:53.750+0000 7f812c366700 1 Processor -- start 2023-11-24T07:56:53.754 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:53.750+0000 7f812c366700 1 -- start start 2023-11-24T07:56:53.754 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:53.751+0000 7f812c366700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f812406b6e0 0x7f8124139440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:56:53.754 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:53.751+0000 7f812c366700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f812406f2c0 0x7f8124141c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:56:53.755 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:53.751+0000 7f812c366700 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f8124145490 0x7f8124143d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:56:53.755 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:53.751+0000 7f812c366700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f812413ba00 con 0x7f812406f2c0 2023-11-24T07:56:53.755 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:53.751+0000 7f812c366700 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8124139980 con 0x7f812406b6e0 2023-11-24T07:56:53.755 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:53.751+0000 7f812c366700 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8124142880 con 0x7f8124145490 2023-11-24T07:56:53.755 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:53.751+0000 7f8129901700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f812406f2c0 0x7f8124141c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:56:53.755 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:53.751+0000 7f8129901700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f812406f2c0 0x7f8124141c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:50636/0 (socket says 172.21.15.59:50636) 2023-11-24T07:56:53.755 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:53.751+0000 7f8129901700 1 -- 172.21.15.59:0/4084141203 learned_addr learned my addr 172.21.15.59:0/4084141203 (peer_addr_for_me v2:172.21.15.59:0/0) 2023-11-24T07:56:53.755 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:53.751+0000 7f812a102700 1 --2- 172.21.15.59:0/4084141203 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f812406b6e0 0x7f8124139440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:56:53.755 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:53.751+0000 7f812a903700 1 --2- 172.21.15.59:0/4084141203 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f8124145490 0x7f8124143d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:56:53.755 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:53.751+0000 7f8129901700 1 -- 172.21.15.59:0/4084141203 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f8124145490 msgr2=0x7f8124143d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:56:53.755 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:53.751+0000 7f8129901700 1 --2- 172.21.15.59:0/4084141203 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f8124145490 0x7f8124143d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:56:53.755 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:53.751+0000 7f8129901700 1 -- 172.21.15.59:0/4084141203 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f812406b6e0 msgr2=0x7f8124139440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:56:53.755 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:53.751+0000 7f8129901700 1 --2- 172.21.15.59:0/4084141203 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f812406b6e0 0x7f8124139440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:56:53.756 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:53.751+0000 7f8129901700 1 -- 172.21.15.59:0/4084141203 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f81140094a0 con 0x7f812406f2c0 2023-11-24T07:56:53.756 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:53.751+0000 7f8129901700 1 --2- 172.21.15.59:0/4084141203 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f812406f2c0 0x7f8124141c50 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f81200011e0 tx=0x7f812000ca30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:56:53.756 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:53.752+0000 7f811affd700 1 -- 172.21.15.59:0/4084141203 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8120004c80 con 0x7f812406f2c0 2023-11-24T07:56:53.756 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:53.752+0000 7f811affd700 1 -- 172.21.15.59:0/4084141203 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8120004e00 con 0x7f812406f2c0 2023-11-24T07:56:53.756 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:53.752+0000 7f812c366700 1 -- 172.21.15.59:0/4084141203 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f81241607b0 con 0x7f812406f2c0 2023-11-24T07:56:53.756 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:53.752+0000 7f811affd700 1 -- 172.21.15.59:0/4084141203 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8120007310 con 0x7f812406f2c0 2023-11-24T07:56:53.756 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:53.752+0000 7f812c366700 1 -- 172.21.15.59:0/4084141203 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f812413bca0 con 0x7f812406f2c0 2023-11-24T07:56:53.757 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:53.752+0000 7f812c366700 1 -- 172.21.15.59:0/4084141203 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f812413b860 con 0x7f812406f2c0 2023-11-24T07:56:53.757 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:53.753+0000 7f811affd700 1 -- 172.21.15.59:0/4084141203 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f8120007470 con 0x7f812406f2c0 2023-11-24T07:56:53.757 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:53.753+0000 7f811affd700 1 --2- 172.21.15.59:0/4084141203 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f810806aed0 0x7f810806d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:56:53.757 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:53.754+0000 7f811affd700 1 -- 172.21.15.59:0/4084141203 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f8120089290 con 0x7f812406f2c0 2023-11-24T07:56:53.757 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:53.754+0000 7f812a102700 1 --2- 172.21.15.59:0/4084141203 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f810806aed0 0x7f810806d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:56:53.757 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:53.754+0000 7f812a102700 1 --2- 172.21.15.59:0/4084141203 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f810806aed0 0x7f810806d390 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f81140090b0 tx=0x7f811401c040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:56:53.759 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:53.758+0000 7f811affd700 1 -- 172.21.15.59:0/4084141203 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f812413b860 con 0x7f812406f2c0 2023-11-24T07:56:53.957 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:53.956+0000 7f812c366700 1 -- 172.21.15.59:0/4084141203 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f812413b860 con 0x7f812406f2c0 2023-11-24T07:56:53.958 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:53.956+0000 7f811affd700 1 -- 172.21.15.59:0/4084141203 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v20) v1 ==== 104+0+11 (secure 0 0 0) 0x7f812413b860 con 0x7f812406f2c0 2023-11-24T07:56:53.958 INFO:teuthology.orchestra.run.smithi059.stdout:pg_num: 32 2023-11-24T07:56:53.960 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:53.958+0000 7f812c366700 1 -- 172.21.15.59:0/4084141203 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f810806aed0 msgr2=0x7f810806d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:56:53.960 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:53.958+0000 7f812c366700 1 --2- 172.21.15.59:0/4084141203 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f810806aed0 0x7f810806d390 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f81140090b0 tx=0x7f811401c040 comp rx=0 tx=0).stop 2023-11-24T07:56:53.960 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:53.958+0000 7f812c366700 1 -- 172.21.15.59:0/4084141203 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f812406f2c0 msgr2=0x7f8124141c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:56:53.960 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:53.958+0000 7f812c366700 1 --2- 172.21.15.59:0/4084141203 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f812406f2c0 0x7f8124141c50 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f81200011e0 tx=0x7f812000ca30 comp rx=0 tx=0).stop 2023-11-24T07:56:53.960 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:53.958+0000 7f812c366700 1 -- 172.21.15.59:0/4084141203 shutdown_connections 2023-11-24T07:56:53.961 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:53.958+0000 7f812c366700 1 --2- 172.21.15.59:0/4084141203 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f810806aed0 0x7f810806d390 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:56:53.961 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:53.958+0000 7f812c366700 1 --2- 172.21.15.59:0/4084141203 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f8124145490 0x7f8124143d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:56:53.961 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:53.958+0000 7f812c366700 1 --2- 172.21.15.59:0/4084141203 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f812406b6e0 0x7f8124139440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:56:53.961 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:53.958+0000 7f812c366700 1 --2- 172.21.15.59:0/4084141203 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f812406f2c0 0x7f8124141c50 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:56:53.961 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:53.958+0000 7f812c366700 1 -- 172.21.15.59:0/4084141203 >> 172.21.15.59:0/4084141203 conn(0x7f81240b2740 msgr2=0x7f81240b1490 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:56:53.961 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:53.958+0000 7f812c366700 1 -- 172.21.15.59:0/4084141203 shutdown_connections 2023-11-24T07:56:53.961 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:53.958+0000 7f812c366700 1 -- 172.21.15.59:0/4084141203 wait complete. 2023-11-24T07:56:53.972 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-24T07:56:54.100 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:54.098+0000 7fe356b86700 1 Processor -- start 2023-11-24T07:56:54.100 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:54.098+0000 7fe356b86700 1 -- start start 2023-11-24T07:56:54.103 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:54.099+0000 7fe356b86700 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fe3500cf2b0 0x7fe35015c1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:56:54.103 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:54.099+0000 7fe356b86700 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fe3500ce5c0 0x7fe3500cea70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:56:54.103 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:54.099+0000 7fe356b86700 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe3500d0a90 con 0x7fe35015bde0 2023-11-24T07:56:54.103 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:54.099+0000 7fe356b86700 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe3500cefb0 con 0x7fe3500ce5c0 2023-11-24T07:56:54.103 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:54.099+0000 7fe356b86700 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe35006ee10 con 0x7fe3500cf2b0 2023-11-24T07:56:54.103 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:54.099+0000 7fe354922700 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7fe35015bde0 0x7fe3500d01b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:54116/0 (socket says 172.21.15.59:54116) 2023-11-24T07:56:54.103 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:54.099+0000 7fe354922700 1 -- 172.21.15.59:0/1216445400 learned_addr learned my addr 172.21.15.59:0/1216445400 (peer_addr_for_me v1:172.21.15.59:0/0) 2023-11-24T07:56:54.103 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:54.099+0000 7fe355123700 1 --2- 172.21.15.59:0/1216445400 >> v2:172.21.15.76:3300/0 conn(0x7fe3500ce5c0 0x7fe3500cea70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:56:54.103 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:54.099+0000 7fe34ffff700 1 --2- 172.21.15.59:0/1216445400 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fe3500cf2b0 0x7fe35015c1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:56:54.103 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:54.099+0000 7fe34f7fe700 1 -- 172.21.15.59:0/1216445400 <== mon.0 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3456179470 0 0) 0x7fe3500d0a90 con 0x7fe35015bde0 2023-11-24T07:56:54.103 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:54.100+0000 7fe34f7fe700 1 -- 172.21.15.59:0/1216445400 --> v1:172.21.15.59:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe3400035b0 con 0x7fe35015bde0 2023-11-24T07:56:54.103 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:54.100+0000 7fe34f7fe700 1 -- 172.21.15.59:0/1216445400 <== mon.0 v1:172.21.15.59:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1670335067 0 0) 0x7fe3400035b0 con 0x7fe35015bde0 2023-11-24T07:56:54.103 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:54.100+0000 7fe355123700 1 -- 172.21.15.59:0/1216445400 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fe3500cf2b0 msgr2=0x7fe35015c1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T07:56:54.104 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:54.100+0000 7fe355123700 1 --2- 172.21.15.59:0/1216445400 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fe3500cf2b0 0x7fe35015c1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:56:54.104 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:54.100+0000 7fe355123700 1 -- 172.21.15.59:0/1216445400 >> v1:172.21.15.59:6789/0 conn(0x7fe35015bde0 legacy=0x7fe3500d01b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:56:54.104 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:54.100+0000 7fe355123700 1 -- 172.21.15.59:0/1216445400 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe35006f0f0 con 0x7fe3500ce5c0 2023-11-24T07:56:54.104 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:54.100+0000 7fe34ffff700 1 --2- 172.21.15.59:0/1216445400 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fe3500cf2b0 0x7fe35015c1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-24T07:56:54.104 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:54.100+0000 7fe355123700 1 --2- 172.21.15.59:0/1216445400 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe3500ce5c0 0x7fe3500cea70 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fe348009d90 tx=0x7fe34800b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=868dd30e7801c341 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:56:54.104 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:54.100+0000 7fe34f7fe700 1 -- 172.21.15.59:0/1216445400 <== mon.1 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe34800e070 con 0x7fe3500ce5c0 2023-11-24T07:56:54.104 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:54.100+0000 7fe34f7fe700 1 -- 172.21.15.59:0/1216445400 <== mon.1 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe348003a20 con 0x7fe3500ce5c0 2023-11-24T07:56:54.104 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:54.100+0000 7fe34f7fe700 1 -- 172.21.15.59:0/1216445400 <== mon.1 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe344001670 con 0x7fe3500ce5c0 2023-11-24T07:56:54.104 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:54.101+0000 7fe356b86700 1 -- 172.21.15.59:0/1216445400 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe3500ce5c0 msgr2=0x7fe3500cea70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:56:54.104 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:54.101+0000 7fe356b86700 1 --2- 172.21.15.59:0/1216445400 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe3500ce5c0 0x7fe3500cea70 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fe348009d90 tx=0x7fe34800b330 comp rx=0 tx=0).stop 2023-11-24T07:56:54.104 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:54.101+0000 7fe356b86700 1 -- 172.21.15.59:0/1216445400 shutdown_connections 2023-11-24T07:56:54.104 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:54.101+0000 7fe356b86700 1 --2- 172.21.15.59:0/1216445400 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe3500ce5c0 0x7fe3500cea70 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:56:54.104 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:54.101+0000 7fe356b86700 1 --2- 172.21.15.59:0/1216445400 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fe3500cf2b0 0x7fe35015c1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:56:54.104 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:54.101+0000 7fe356b86700 1 -- 172.21.15.59:0/1216445400 >> 172.21.15.59:0/1216445400 conn(0x7fe3500b3720 msgr2=0x7fe3500b3b30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:56:54.105 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:54.101+0000 7fe356b86700 1 -- 172.21.15.59:0/1216445400 shutdown_connections 2023-11-24T07:56:54.105 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:54.101+0000 7fe356b86700 1 -- 172.21.15.59:0/1216445400 wait complete. 2023-11-24T07:56:54.107 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:54.102+0000 7fe356b86700 1 Processor -- start 2023-11-24T07:56:54.107 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:54.102+0000 7fe356b86700 1 -- start start 2023-11-24T07:56:54.107 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:54.103+0000 7fe356b86700 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fe3500ce5c0 0x7fe3501591e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:56:54.107 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:54.103+0000 7fe356b86700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe3500cf2b0 0x7fe3500b7040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:56:54.107 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:54.103+0000 7fe356b86700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe35015bde0 0x7fe3500b4810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:56:54.107 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:54.103+0000 7fe356b86700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3500c73f0 con 0x7fe3500cf2b0 2023-11-24T07:56:54.107 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:54.103+0000 7fe356b86700 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3500c67e0 con 0x7fe35015bde0 2023-11-24T07:56:54.108 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:54.103+0000 7fe356b86700 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe3500c6ac0 con 0x7fe3500ce5c0 2023-11-24T07:56:54.108 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:54.104+0000 7fe34ffff700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe3500cf2b0 0x7fe3500b7040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:56:54.108 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:54.104+0000 7fe34ffff700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe3500cf2b0 0x7fe3500b7040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:50650/0 (socket says 172.21.15.59:50650) 2023-11-24T07:56:54.108 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:54.104+0000 7fe34ffff700 1 -- 172.21.15.59:0/2384627735 learned_addr learned my addr 172.21.15.59:0/2384627735 (peer_addr_for_me v2:172.21.15.59:0/0) 2023-11-24T07:56:54.108 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:54.104+0000 7fe354922700 1 --2- 172.21.15.59:0/2384627735 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fe3500ce5c0 0x7fe3501591e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:56:54.108 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:54.104+0000 7fe355123700 1 --2- 172.21.15.59:0/2384627735 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe35015bde0 0x7fe3500b4810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:56:54.108 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:54.104+0000 7fe34ffff700 1 -- 172.21.15.59:0/2384627735 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fe3500ce5c0 msgr2=0x7fe3501591e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:56:54.108 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:54.104+0000 7fe34ffff700 1 --2- 172.21.15.59:0/2384627735 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fe3500ce5c0 0x7fe3501591e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:56:54.108 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:54.104+0000 7fe34ffff700 1 -- 172.21.15.59:0/2384627735 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe35015bde0 msgr2=0x7fe3500b4810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:56:54.108 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:54.104+0000 7fe34ffff700 1 --2- 172.21.15.59:0/2384627735 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe35015bde0 0x7fe3500b4810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:56:54.108 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:54.104+0000 7fe34ffff700 1 -- 172.21.15.59:0/2384627735 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe33c0094a0 con 0x7fe3500cf2b0 2023-11-24T07:56:54.108 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:54.104+0000 7fe354922700 1 --2- 172.21.15.59:0/2384627735 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fe3500ce5c0 0x7fe3501591e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-24T07:56:54.108 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:54.104+0000 7fe355123700 1 --2- 172.21.15.59:0/2384627735 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe35015bde0 0x7fe3500b4810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-24T07:56:54.109 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:54.104+0000 7fe34ffff700 1 --2- 172.21.15.59:0/2384627735 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe3500cf2b0 0x7fe3500b7040 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fe33c00ebb0 tx=0x7fe33c00ef70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:56:54.109 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:54.104+0000 7fe34d7fa700 1 -- 172.21.15.59:0/2384627735 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe33c00c910 con 0x7fe3500cf2b0 2023-11-24T07:56:54.109 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:54.104+0000 7fe34d7fa700 1 -- 172.21.15.59:0/2384627735 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe33c00ca70 con 0x7fe3500cf2b0 2023-11-24T07:56:54.109 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:54.104+0000 7fe34d7fa700 1 -- 172.21.15.59:0/2384627735 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe33c00ce00 con 0x7fe3500cf2b0 2023-11-24T07:56:54.109 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:54.104+0000 7fe356b86700 1 -- 172.21.15.59:0/2384627735 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe348009a70 con 0x7fe3500cf2b0 2023-11-24T07:56:54.109 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:54.104+0000 7fe356b86700 1 -- 172.21.15.59:0/2384627735 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe3500c7080 con 0x7fe3500cf2b0 2023-11-24T07:56:54.109 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:54.105+0000 7fe356b86700 1 -- 172.21.15.59:0/2384627735 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe35006b4c0 con 0x7fe3500cf2b0 2023-11-24T07:56:54.109 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:54.106+0000 7fe34d7fa700 1 -- 172.21.15.59:0/2384627735 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fe33c013470 con 0x7fe3500cf2b0 2023-11-24T07:56:54.110 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:54.106+0000 7fe34d7fa700 1 --2- 172.21.15.59:0/2384627735 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7fe33406aed0 0x7fe33406d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:56:54.110 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:54.106+0000 7fe34d7fa700 1 -- 172.21.15.59:0/2384627735 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fe33c0897d0 con 0x7fe3500cf2b0 2023-11-24T07:56:54.110 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:54.106+0000 7fe354922700 1 --2- 172.21.15.59:0/2384627735 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7fe33406aed0 0x7fe33406d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:56:54.110 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:54.107+0000 7fe354922700 1 --2- 172.21.15.59:0/2384627735 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7fe33406aed0 0x7fe33406d390 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fe3440011e0 tx=0x7fe344007b00 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:56:54.112 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:54.111+0000 7fe34d7fa700 1 -- 172.21.15.59:0/2384627735 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fe33c054880 con 0x7fe3500cf2b0 2023-11-24T07:56:54.312 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:54.311+0000 7fe356b86700 1 -- 172.21.15.59:0/2384627735 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fe350075480 con 0x7fe3500cf2b0 2023-11-24T07:56:54.313 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:54.311+0000 7fe34d7fa700 1 -- 172.21.15.59:0/2384627735 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v20) v1 ==== 100+0+11 (secure 0 0 0) 0x7fe33c0170a0 con 0x7fe3500cf2b0 2023-11-24T07:56:54.313 INFO:teuthology.orchestra.run.smithi059.stdout:pg_num: 64 2023-11-24T07:56:54.315 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:54.313+0000 7fe356b86700 1 -- 172.21.15.59:0/2384627735 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7fe33406aed0 msgr2=0x7fe33406d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:56:54.315 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:54.313+0000 7fe356b86700 1 --2- 172.21.15.59:0/2384627735 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7fe33406aed0 0x7fe33406d390 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fe3440011e0 tx=0x7fe344007b00 comp rx=0 tx=0).stop 2023-11-24T07:56:54.315 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:54.313+0000 7fe356b86700 1 -- 172.21.15.59:0/2384627735 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe3500cf2b0 msgr2=0x7fe3500b7040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:56:54.315 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:54.313+0000 7fe356b86700 1 --2- 172.21.15.59:0/2384627735 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe3500cf2b0 0x7fe3500b7040 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fe33c00ebb0 tx=0x7fe33c00ef70 comp rx=0 tx=0).stop 2023-11-24T07:56:54.315 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:54.313+0000 7fe356b86700 1 -- 172.21.15.59:0/2384627735 shutdown_connections 2023-11-24T07:56:54.315 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:54.313+0000 7fe356b86700 1 --2- 172.21.15.59:0/2384627735 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7fe33406aed0 0x7fe33406d390 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:56:54.316 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:54.313+0000 7fe356b86700 1 --2- 172.21.15.59:0/2384627735 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe35015bde0 0x7fe3500b4810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:56:54.316 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:54.313+0000 7fe356b86700 1 --2- 172.21.15.59:0/2384627735 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fe3500ce5c0 0x7fe3501591e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:56:54.316 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:54.313+0000 7fe356b86700 1 --2- 172.21.15.59:0/2384627735 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe3500cf2b0 0x7fe3500b7040 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:56:54.316 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:54.313+0000 7fe356b86700 1 -- 172.21.15.59:0/2384627735 >> 172.21.15.59:0/2384627735 conn(0x7fe3500b3720 msgr2=0x7fe3500afcf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:56:54.316 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:54.313+0000 7fe356b86700 1 -- 172.21.15.59:0/2384627735 shutdown_connections 2023-11-24T07:56:54.316 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:54.313+0000 7fe356b86700 1 -- 172.21.15.59:0/2384627735 wait complete. 2023-11-24T07:56:54.327 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-24T07:56:54.453 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:54.451+0000 7fb5d1bab700 1 Processor -- start 2023-11-24T07:56:54.454 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:54.451+0000 7fb5d1bab700 1 -- start start 2023-11-24T07:56:54.456 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:54.452+0000 7fb5d1bab700 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fb5cc142710 0x7fb5cc0724e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:56:54.456 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:54.452+0000 7fb5d1bab700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fb5cc0b33f0 0x7fb5cc06afd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:56:54.456 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:54.452+0000 7fb5d1bab700 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fb5cc06b510 0x7fb5cc15fb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:56:54.456 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:54.452+0000 7fb5d1bab700 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb5cc06b9c0 con 0x7fb5cc0b33f0 2023-11-24T07:56:54.456 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:54.452+0000 7fb5d1bab700 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb5cc142af0 con 0x7fb5cc06b510 2023-11-24T07:56:54.456 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:54.452+0000 7fb5d1bab700 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb5cc0cf190 con 0x7fb5cc142710 2023-11-24T07:56:54.456 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:54.452+0000 7fb5caffd700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fb5cc0b33f0 0x7fb5cc06afd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:56:54.457 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:54.452+0000 7fb5caffd700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fb5cc0b33f0 0x7fb5cc06afd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:50658/0 (socket says 172.21.15.59:50658) 2023-11-24T07:56:54.457 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:54.452+0000 7fb5caffd700 1 -- 172.21.15.59:0/1042317292 learned_addr learned my addr 172.21.15.59:0/1042317292 (peer_addr_for_me v2:172.21.15.59:0/0) 2023-11-24T07:56:54.457 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:54.452+0000 7fb5cb7fe700 1 --2- 172.21.15.59:0/1042317292 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fb5cc142710 0x7fb5cc0724e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:56:54.457 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:54.452+0000 7fb5cbfff700 1 --2- 172.21.15.59:0/1042317292 >> v2:172.21.15.76:3300/0 conn(0x7fb5cc06b510 0x7fb5cc15fb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:56:54.457 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:54.453+0000 7fb5caffd700 1 -- 172.21.15.59:0/1042317292 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fb5cc142710 msgr2=0x7fb5cc0724e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T07:56:54.457 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:54.453+0000 7fb5caffd700 1 --2- 172.21.15.59:0/1042317292 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fb5cc142710 0x7fb5cc0724e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:56:54.457 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:54.453+0000 7fb5caffd700 1 -- 172.21.15.59:0/1042317292 >> v2:172.21.15.76:3300/0 conn(0x7fb5cc06b510 msgr2=0x7fb5cc15fb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T07:56:54.457 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:54.453+0000 7fb5caffd700 1 --2- 172.21.15.59:0/1042317292 >> v2:172.21.15.76:3300/0 conn(0x7fb5cc06b510 0x7fb5cc15fb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:56:54.457 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:54.453+0000 7fb5caffd700 1 -- 172.21.15.59:0/1042317292 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb5cc072a20 con 0x7fb5cc0b33f0 2023-11-24T07:56:54.457 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:54.453+0000 7fb5caffd700 1 --2- 172.21.15.59:0/1042317292 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb5cc0b33f0 0x7fb5cc06afd0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fb5cc06c0b0 tx=0x7fb5b4009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5be2c77c1ba3c457 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:56:54.458 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:54.453+0000 7fb5ca7fc700 1 -- 172.21.15.59:0/1042317292 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5b4011830 con 0x7fb5cc0b33f0 2023-11-24T07:56:54.458 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:54.453+0000 7fb5ca7fc700 1 -- 172.21.15.59:0/1042317292 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb5b4011990 con 0x7fb5cc0b33f0 2023-11-24T07:56:54.458 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:54.453+0000 7fb5ca7fc700 1 -- 172.21.15.59:0/1042317292 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5b4011d20 con 0x7fb5cc0b33f0 2023-11-24T07:56:54.458 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:54.454+0000 7fb5d1bab700 1 -- 172.21.15.59:0/1042317292 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb5cc0b33f0 msgr2=0x7fb5cc06afd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:56:54.458 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:54.454+0000 7fb5d1bab700 1 --2- 172.21.15.59:0/1042317292 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb5cc0b33f0 0x7fb5cc06afd0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fb5cc06c0b0 tx=0x7fb5b4009bd0 comp rx=0 tx=0).stop 2023-11-24T07:56:54.458 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:54.454+0000 7fb5d1bab700 1 -- 172.21.15.59:0/1042317292 shutdown_connections 2023-11-24T07:56:54.458 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:54.454+0000 7fb5d1bab700 1 --2- 172.21.15.59:0/1042317292 >> v2:172.21.15.76:3300/0 conn(0x7fb5cc06b510 0x7fb5cc15fb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:56:54.458 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:54.454+0000 7fb5d1bab700 1 --2- 172.21.15.59:0/1042317292 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fb5cc142710 0x7fb5cc0724e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:56:54.458 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:54.454+0000 7fb5d1bab700 1 --2- 172.21.15.59:0/1042317292 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb5cc0b33f0 0x7fb5cc06afd0 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:56:54.458 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:54.454+0000 7fb5d1bab700 1 -- 172.21.15.59:0/1042317292 >> 172.21.15.59:0/1042317292 conn(0x7fb5cc0b27b0 msgr2=0x7fb5cc0b2b90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:56:54.458 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:54.454+0000 7fb5d1bab700 1 -- 172.21.15.59:0/1042317292 shutdown_connections 2023-11-24T07:56:54.458 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:54.454+0000 7fb5d1bab700 1 -- 172.21.15.59:0/1042317292 wait complete. 2023-11-24T07:56:54.461 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:54.455+0000 7fb5d1bab700 1 Processor -- start 2023-11-24T07:56:54.461 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:54.455+0000 7fb5d1bab700 1 -- start start 2023-11-24T07:56:54.461 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:54.456+0000 7fb5d1bab700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb5cc06b510 0x7fb5cc0a9450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:56:54.461 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:54.456+0000 7fb5d1bab700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb5cc0b33f0 0x7fb5cc15b9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:56:54.461 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:54.456+0000 7fb5d1bab700 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fb5cc142710 0x7fb5cc130d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:56:54.462 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:54.456+0000 7fb5d1bab700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5cc0aa7d0 con 0x7fb5cc0b33f0 2023-11-24T07:56:54.462 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:54.456+0000 7fb5d1bab700 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5cc15bef0 con 0x7fb5cc06b510 2023-11-24T07:56:54.462 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:54.456+0000 7fb5d1bab700 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb5cc1411c0 con 0x7fb5cc142710 2023-11-24T07:56:54.462 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:54.456+0000 7fb5caffd700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb5cc0b33f0 0x7fb5cc15b9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:56:54.462 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:54.456+0000 7fb5caffd700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb5cc0b33f0 0x7fb5cc15b9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:50664/0 (socket says 172.21.15.59:50664) 2023-11-24T07:56:54.462 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:54.456+0000 7fb5caffd700 1 -- 172.21.15.59:0/3640116453 learned_addr learned my addr 172.21.15.59:0/3640116453 (peer_addr_for_me v2:172.21.15.59:0/0) 2023-11-24T07:56:54.462 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:54.456+0000 7fb5cb7fe700 1 --2- 172.21.15.59:0/3640116453 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb5cc06b510 0x7fb5cc0a9450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:56:54.462 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:54.456+0000 7fb5cbfff700 1 --2- 172.21.15.59:0/3640116453 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fb5cc142710 0x7fb5cc130d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:56:54.462 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:54.456+0000 7fb5caffd700 1 -- 172.21.15.59:0/3640116453 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fb5cc142710 msgr2=0x7fb5cc130d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:56:54.462 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:54.456+0000 7fb5caffd700 1 --2- 172.21.15.59:0/3640116453 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fb5cc142710 0x7fb5cc130d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:56:54.463 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:54.456+0000 7fb5caffd700 1 -- 172.21.15.59:0/3640116453 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb5cc06b510 msgr2=0x7fb5cc0a9450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:56:54.463 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:54.456+0000 7fb5caffd700 1 --2- 172.21.15.59:0/3640116453 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb5cc06b510 0x7fb5cc0a9450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:56:54.463 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:54.456+0000 7fb5caffd700 1 -- 172.21.15.59:0/3640116453 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb5b4009550 con 0x7fb5cc0b33f0 2023-11-24T07:56:54.463 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:54.456+0000 7fb5caffd700 1 --2- 172.21.15.59:0/3640116453 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb5cc0b33f0 0x7fb5cc15b9b0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fb5b4000c00 tx=0x7fb5b4004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:56:54.463 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:54.457+0000 7fb5c3fff700 1 -- 172.21.15.59:0/3640116453 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5b4004240 con 0x7fb5cc0b33f0 2023-11-24T07:56:54.463 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:54.457+0000 7fb5c3fff700 1 -- 172.21.15.59:0/3640116453 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb5b401f070 con 0x7fb5cc0b33f0 2023-11-24T07:56:54.463 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:54.457+0000 7fb5d1bab700 1 -- 172.21.15.59:0/3640116453 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb5cc0a9b50 con 0x7fb5cc0b33f0 2023-11-24T07:56:54.463 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:54.457+0000 7fb5c3fff700 1 -- 172.21.15.59:0/3640116453 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5b40036a0 con 0x7fb5cc0b33f0 2023-11-24T07:56:54.463 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:54.457+0000 7fb5d1bab700 1 -- 172.21.15.59:0/3640116453 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb5cc0aa110 con 0x7fb5cc0b33f0 2023-11-24T07:56:54.463 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:54.458+0000 7fb5d1bab700 1 -- 172.21.15.59:0/3640116453 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb5cc075480 con 0x7fb5cc0b33f0 2023-11-24T07:56:54.463 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:54.458+0000 7fb5c3fff700 1 -- 172.21.15.59:0/3640116453 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fb5b4021030 con 0x7fb5cc0b33f0 2023-11-24T07:56:54.463 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:54.458+0000 7fb5c3fff700 1 --2- 172.21.15.59:0/3640116453 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7fb5ac06aed0 0x7fb5ac06d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:56:54.464 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:54.458+0000 7fb5c3fff700 1 -- 172.21.15.59:0/3640116453 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fb5b408a710 con 0x7fb5cc0b33f0 2023-11-24T07:56:54.464 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:54.459+0000 7fb5cb7fe700 1 --2- 172.21.15.59:0/3640116453 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7fb5ac06aed0 0x7fb5ac06d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:56:54.464 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:54.459+0000 7fb5cb7fe700 1 --2- 172.21.15.59:0/3640116453 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7fb5ac06aed0 0x7fb5ac06d390 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fb5bc009b40 tx=0x7fb5bc009210 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:56:54.464 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:54.463+0000 7fb5c3fff700 1 -- 172.21.15.59:0/3640116453 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fb5b40557b0 con 0x7fb5cc0b33f0 2023-11-24T07:56:54.669 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:54.667+0000 7fb5d1bab700 1 -- 172.21.15.59:0/3640116453 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fb5cc0aa910 con 0x7fb5cc0b33f0 2023-11-24T07:56:54.670 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:54.669+0000 7fb5c3fff700 1 -- 172.21.15.59:0/3640116453 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) v1 ==== 74+0+16437 (secure 0 0 0) 0x7fb5cc0aa910 con 0x7fb5cc0b33f0 2023-11-24T07:56:54.670 INFO:teuthology.orchestra.run.smithi059.stdout: 2023-11-24T07:56:54.671 INFO:teuthology.orchestra.run.smithi059.stdout:{"epoch":20,"fsid":"9f0e8741-b8f9-48d5-a2c3-09f45903303d","created":"2023-11-24T07:56:06.552729+0000","modified":"2023-11-24T07:56:25.705091+0000","last_up_change":"2023-11-24T07:56:17.372352+0000","last_in_change":"2023-11-24T07:56:09.073250+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-24T07:56:18.352823+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-11-24T07:56:22.185140+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-11-24T07:56:23.084326+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"6d8f9e9b-1782-4e41-928b-a39f2a8ebc06","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6800","nonce":3373878772},{"type":"v1","addr":"172.21.15.59:6801","nonce":3373878772}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6802","nonce":3373878772},{"type":"v1","addr":"172.21.15.59:6803","nonce":3373878772}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6806","nonce":3373878772},{"type":"v1","addr":"172.21.15.59:6807","nonce":3373878772}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6804","nonce":3373878772},{"type":"v1","addr":"172.21.15.59:6805","nonce":3373878772}]},"public_addr":"172.21.15.59:6801/3373878772","cluster_addr":"172.21.15.59:6803/3373878772","heartbeat_back_addr":"172.21.15.59:6807/3373878772","heartbeat_front_addr":"172.21.15.59:6805/3373878772","state":["exists","up"]},{"osd":1,"uuid":"e8d882aa-52e1-4c04-9e1e-c44ef9204a5f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6824","nonce":648258563},{"type":"v1","addr":"172.21.15.59:6825","nonce":648258563}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6826","nonce":648258563},{"type":"v1","addr":"172.21.15.59:6827","nonce":648258563}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6830","nonce":648258563},{"type":"v1","addr":"172.21.15.59:6831","nonce":648258563}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6828","nonce":648258563},{"type":"v1","addr":"172.21.15.59:6829","nonce":648258563}]},"public_addr":"172.21.15.59:6825/648258563","cluster_addr":"172.21.15.59:6827/648258563","heartbeat_back_addr":"172.21.15.59:6831/648258563","heartbeat_front_addr":"172.21.15.59:6829/648258563","state":["exists","up"]},{"osd":2,"uuid":"69eebfa0-695b-4b21-90b9-8a3537168a01","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6808","nonce":1840430654},{"type":"v1","addr":"172.21.15.59:6809","nonce":1840430654}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6810","nonce":1840430654},{"type":"v1","addr":"172.21.15.59:6811","nonce":1840430654}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6814","nonce":1840430654},{"type":"v1","addr":"172.21.15.59:6815","nonce":1840430654}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6812","nonce":1840430654},{"type":"v1","addr":"172.21.15.59:6813","nonce":1840430654}]},"public_addr":"172.21.15.59:6809/1840430654","cluster_addr":"172.21.15.59:6811/1840430654","heartbeat_back_addr":"172.21.15.59:6815/1840430654","heartbeat_front_addr":"172.21.15.59:6813/1840430654","state":["exists","up"]},{"osd":3,"uuid":"96fd77fc-f911-46ff-9445-6f0be2daa8f4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6816","nonce":2504025850},{"type":"v1","addr":"172.21.15.59:6817","nonce":2504025850}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6818","nonce":2504025850},{"type":"v1","addr":"172.21.15.59:6819","nonce":2504025850}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6822","nonce":2504025850},{"type":"v1","addr":"172.21.15.59:6823","nonce":2504025850}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6820","nonce":2504025850},{"type":"v1","addr":"172.21.15.59:6821","nonce":2504025850}]},"public_addr":"172.21.15.59:6817/2504025850","cluster_addr":"172.21.15.59:6819/2504025850","heartbeat_back_addr":"172.21.15.59:6823/2504025850","heartbeat_front_addr":"172.21.15.59:6821/2504025850","state":["exists","up"]},{"osd":4,"uuid":"7ac29fca-b5b2-4c95-9d69-2f164cbbc708","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6816","nonce":2382463431},{"type":"v1","addr":"172.21.15.76:6818","nonce":2382463431}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6820","nonce":2382463431},{"type":"v1","addr":"172.21.15.76:6822","nonce":2382463431}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6828","nonce":2382463431},{"type":"v1","addr":"172.21.15.76:6830","nonce":2382463431}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6824","nonce":2382463431},{"type":"v1","addr":"172.21.15.76:6826","nonce":2382463431}]},"public_addr":"172.21.15.76:6818/2382463431","cluster_addr":"172.21.15.76:6822/2382463431","heartbeat_back_addr":"172.21.15.76:6830/2382463431","heartbeat_front_addr":"172.21.15.76:6826/2382463431","state":["exists","up"]},{"osd":5,"uuid":"485bf711-3851-4238-a4f4-51d52e0a9c58","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6800","nonce":329334308},{"type":"v1","addr":"172.21.15.76:6802","nonce":329334308}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6805","nonce":329334308},{"type":"v1","addr":"172.21.15.76:6807","nonce":329334308}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6814","nonce":329334308},{"type":"v1","addr":"172.21.15.76:6815","nonce":329334308}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6810","nonce":329334308},{"type":"v1","addr":"172.21.15.76:6812","nonce":329334308}]},"public_addr":"172.21.15.76:6802/329334308","cluster_addr":"172.21.15.76:6807/329334308","heartbeat_back_addr":"172.21.15.76:6815/329334308","heartbeat_front_addr":"172.21.15.76:6812/329334308","state":["exists","up"]},{"osd":6,"uuid":"72b3f2fc-1e4a-4e32-968d-003647a0286f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6801","nonce":3946579410},{"type":"v1","addr":"172.21.15.76:6803","nonce":3946579410}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6804","nonce":3946579410},{"type":"v1","addr":"172.21.15.76:6806","nonce":3946579410}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6811","nonce":3946579410},{"type":"v1","addr":"172.21.15.76:6813","nonce":3946579410}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6808","nonce":3946579410},{"type":"v1","addr":"172.21.15.76:6809","nonce":3946579410}]},"public_addr":"172.21.15.76:6803/3946579410","cluster_addr":"172.21.15.76:6806/3946579410","heartbeat_back_addr":"172.21.15.76:6813/3946579410","heartbeat_front_addr":"172.21.15.76:6809/3946579410","state":["exists","up"]},{"osd":7,"uuid":"1ddceb90-6c5d-4d1a-ad56-99cb877f8c70","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6817","nonce":841761538},{"type":"v1","addr":"172.21.15.76:6819","nonce":841761538}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6821","nonce":841761538},{"type":"v1","addr":"172.21.15.76:6823","nonce":841761538}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6829","nonce":841761538},{"type":"v1","addr":"172.21.15.76:6831","nonce":841761538}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6825","nonce":841761538},{"type":"v1","addr":"172.21.15.76:6827","nonce":841761538}]},"public_addr":"172.21.15.76:6819/841761538","cluster_addr":"172.21.15.76:6823/841761538","heartbeat_back_addr":"172.21.15.76:6831/841761538","heartbeat_front_addr":"172.21.15.76:6827/841761538","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T07:56:14.294838+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T07:56:14.270779+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T07:56:14.231809+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T07:56:14.260983+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T07:56:14.302997+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T07:56:14.258600+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T07:56:14.251413+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T07:56:14.304727+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-24T07:56:54.673 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:54.671+0000 7fb5d1bab700 1 -- 172.21.15.59:0/3640116453 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7fb5ac06aed0 msgr2=0x7fb5ac06d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:56:54.673 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:54.671+0000 7fb5d1bab700 1 --2- 172.21.15.59:0/3640116453 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7fb5ac06aed0 0x7fb5ac06d390 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fb5bc009b40 tx=0x7fb5bc009210 comp rx=0 tx=0).stop 2023-11-24T07:56:54.673 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:54.671+0000 7fb5d1bab700 1 -- 172.21.15.59:0/3640116453 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb5cc0b33f0 msgr2=0x7fb5cc15b9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:56:54.673 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:54.671+0000 7fb5d1bab700 1 --2- 172.21.15.59:0/3640116453 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb5cc0b33f0 0x7fb5cc15b9b0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fb5b4000c00 tx=0x7fb5b4004750 comp rx=0 tx=0).stop 2023-11-24T07:56:54.673 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:54.671+0000 7fb5d1bab700 1 -- 172.21.15.59:0/3640116453 shutdown_connections 2023-11-24T07:56:54.673 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:54.671+0000 7fb5d1bab700 1 --2- 172.21.15.59:0/3640116453 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7fb5ac06aed0 0x7fb5ac06d390 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:56:54.673 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:54.671+0000 7fb5d1bab700 1 --2- 172.21.15.59:0/3640116453 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fb5cc142710 0x7fb5cc130d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:56:54.673 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:54.671+0000 7fb5d1bab700 1 --2- 172.21.15.59:0/3640116453 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb5cc06b510 0x7fb5cc0a9450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:56:54.673 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:54.671+0000 7fb5d1bab700 1 --2- 172.21.15.59:0/3640116453 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb5cc0b33f0 0x7fb5cc15b9b0 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:56:54.673 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:54.671+0000 7fb5d1bab700 1 -- 172.21.15.59:0/3640116453 >> 172.21.15.59:0/3640116453 conn(0x7fb5cc0b27b0 msgr2=0x7fb5cc0b1dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:56:54.674 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:54.671+0000 7fb5d1bab700 1 -- 172.21.15.59:0/3640116453 shutdown_connections 2023-11-24T07:56:54.674 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:54.671+0000 7fb5d1bab700 1 -- 172.21.15.59:0/3640116453 wait complete. 2023-11-24T07:56:54.685 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-24T07:56:18.352823+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-24T07:56:22.185140+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-11-24T07:56:23.084326+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-24T07:56:54.686 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-24T07:56:54.813 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:54.811+0000 7f7d41df8700 1 Processor -- start 2023-11-24T07:56:54.813 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:54.811+0000 7f7d41df8700 1 -- start start 2023-11-24T07:56:54.821 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:54.812+0000 7f7d41df8700 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f7d3c143ae0 0x7f7d3c0b51b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:56:54.821 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:54.812+0000 7f7d41df8700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f7d3c0722c0 0x7f7d3c06e510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:56:54.821 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:54.812+0000 7f7d41df8700 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7d3c143ec0 con 0x7f7d3c1440f0 2023-11-24T07:56:54.821 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:54.812+0000 7f7d41df8700 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7d3c0b40a0 con 0x7f7d3c0722c0 2023-11-24T07:56:54.821 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:54.812+0000 7f7d41df8700 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7d3c0b3ae0 con 0x7f7d3c143ae0 2023-11-24T07:56:54.821 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:54.812+0000 7f7d3bfff700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f7d3c0722c0 0x7f7d3c06e510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:56:54.822 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:54.812+0000 7f7d3affd700 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7f7d3c1440f0 0x7f7d3c071bb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:54144/0 (socket says 172.21.15.59:54144) 2023-11-24T07:56:54.822 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:54.812+0000 7f7d3affd700 1 -- 172.21.15.59:0/4013392454 learned_addr learned my addr 172.21.15.59:0/4013392454 (peer_addr_for_me v1:172.21.15.59:0/0) 2023-11-24T07:56:54.822 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:54.813+0000 7f7d3b7fe700 1 --2- 172.21.15.59:0/4013392454 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f7d3c143ae0 0x7f7d3c0b51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:56:54.822 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:54.813+0000 7f7d3bfff700 1 -- 172.21.15.59:0/4013392454 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f7d3c143ae0 msgr2=0x7f7d3c0b51b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T07:56:54.822 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:54.813+0000 7f7d3bfff700 1 --2- 172.21.15.59:0/4013392454 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f7d3c143ae0 0x7f7d3c0b51b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:56:54.823 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:54.813+0000 7f7d3bfff700 1 -- 172.21.15.59:0/4013392454 >> v1:172.21.15.59:6789/0 conn(0x7f7d3c1440f0 legacy=0x7f7d3c071bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:56:54.823 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:54.813+0000 7f7d3bfff700 1 -- 172.21.15.59:0/4013392454 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7d3c0b3dc0 con 0x7f7d3c0722c0 2023-11-24T07:56:54.823 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:54.813+0000 7f7d3b7fe700 1 --2- 172.21.15.59:0/4013392454 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f7d3c143ae0 0x7f7d3c0b51b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T07:56:54.823 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:54.813+0000 7f7d3bfff700 1 --2- 172.21.15.59:0/4013392454 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7d3c0722c0 0x7f7d3c06e510 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f7d3400bfa0 tx=0x7f7d3400d330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1426500eee0b27a8 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:56:54.823 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:54.813+0000 7f7d3a7fc700 1 -- 172.21.15.59:0/4013392454 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d34010070 con 0x7f7d3c0722c0 2023-11-24T07:56:54.823 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:54.813+0000 7f7d3a7fc700 1 -- 172.21.15.59:0/4013392454 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7d3400b2e0 con 0x7f7d3c0722c0 2023-11-24T07:56:54.823 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:54.813+0000 7f7d3a7fc700 1 -- 172.21.15.59:0/4013392454 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d3400dcf0 con 0x7f7d3c0722c0 2023-11-24T07:56:54.824 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:54.814+0000 7f7d41df8700 1 -- 172.21.15.59:0/4013392454 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7d3c0722c0 msgr2=0x7f7d3c06e510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:56:54.824 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:54.814+0000 7f7d41df8700 1 --2- 172.21.15.59:0/4013392454 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7d3c0722c0 0x7f7d3c06e510 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f7d3400bfa0 tx=0x7f7d3400d330 comp rx=0 tx=0).stop 2023-11-24T07:56:54.824 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:54.814+0000 7f7d41df8700 1 -- 172.21.15.59:0/4013392454 shutdown_connections 2023-11-24T07:56:54.824 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:54.814+0000 7f7d41df8700 1 --2- 172.21.15.59:0/4013392454 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7d3c0722c0 0x7f7d3c06e510 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:56:54.824 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:54.814+0000 7f7d41df8700 1 --2- 172.21.15.59:0/4013392454 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f7d3c143ae0 0x7f7d3c0b51b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:56:54.824 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:54.814+0000 7f7d41df8700 1 -- 172.21.15.59:0/4013392454 >> 172.21.15.59:0/4013392454 conn(0x7f7d3c0b2740 msgr2=0x7f7d3c0b2b20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:56:54.824 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:54.814+0000 7f7d41df8700 1 -- 172.21.15.59:0/4013392454 shutdown_connections 2023-11-24T07:56:54.824 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:54.814+0000 7f7d41df8700 1 -- 172.21.15.59:0/4013392454 wait complete. 2023-11-24T07:56:54.824 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:54.815+0000 7f7d41df8700 1 Processor -- start 2023-11-24T07:56:54.824 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:54.815+0000 7f7d41df8700 1 -- start start 2023-11-24T07:56:54.825 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:54.816+0000 7f7d41df8700 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f7d3c0722c0 0x7f7d3c155b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:56:54.825 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:54.816+0000 7f7d41df8700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7d3c143ae0 0x7f7d3c158760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:56:54.825 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:54.816+0000 7f7d41df8700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7d3c1440f0 0x7f7d3c158ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:56:54.825 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:54.816+0000 7f7d41df8700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d3c1f4930 con 0x7f7d3c143ae0 2023-11-24T07:56:54.825 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:54.816+0000 7f7d41df8700 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d3c1f3cb0 con 0x7f7d3c1440f0 2023-11-24T07:56:54.825 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:54.816+0000 7f7d41df8700 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7d3c1f3f90 con 0x7f7d3c0722c0 2023-11-24T07:56:54.825 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:54.816+0000 7f7d3affd700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7d3c143ae0 0x7f7d3c158760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:56:54.825 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:54.816+0000 7f7d3affd700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7d3c143ae0 0x7f7d3c158760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:50678/0 (socket says 172.21.15.59:50678) 2023-11-24T07:56:54.825 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:54.816+0000 7f7d3affd700 1 -- 172.21.15.59:0/451186138 learned_addr learned my addr 172.21.15.59:0/451186138 (peer_addr_for_me v2:172.21.15.59:0/0) 2023-11-24T07:56:54.825 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:54.816+0000 7f7d3b7fe700 1 --2- 172.21.15.59:0/451186138 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f7d3c0722c0 0x7f7d3c155b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:56:54.825 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:54.816+0000 7f7d3bfff700 1 --2- 172.21.15.59:0/451186138 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7d3c1440f0 0x7f7d3c158ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:56:54.826 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:54.816+0000 7f7d3affd700 1 -- 172.21.15.59:0/451186138 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f7d3c0722c0 msgr2=0x7f7d3c155b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:56:54.826 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:54.816+0000 7f7d3affd700 1 --2- 172.21.15.59:0/451186138 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f7d3c0722c0 0x7f7d3c155b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:56:54.826 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:54.816+0000 7f7d3affd700 1 -- 172.21.15.59:0/451186138 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7d3c1440f0 msgr2=0x7f7d3c158ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:56:54.826 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:54.816+0000 7f7d3affd700 1 --2- 172.21.15.59:0/451186138 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7d3c1440f0 0x7f7d3c158ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:56:54.826 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:54.816+0000 7f7d3affd700 1 -- 172.21.15.59:0/451186138 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7d3400bc80 con 0x7f7d3c143ae0 2023-11-24T07:56:54.826 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:54.816+0000 7f7d3affd700 1 --2- 172.21.15.59:0/451186138 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7d3c143ae0 0x7f7d3c158760 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f7d2400c480 tx=0x7f7d2400c790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:56:54.826 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:54.817+0000 7f7d33fff700 1 -- 172.21.15.59:0/451186138 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d24005d70 con 0x7f7d3c143ae0 2023-11-24T07:56:54.826 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:54.817+0000 7f7d33fff700 1 -- 172.21.15.59:0/451186138 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7d2400ce90 con 0x7f7d3c143ae0 2023-11-24T07:56:54.826 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:54.817+0000 7f7d41df8700 1 -- 172.21.15.59:0/451186138 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7d3c1f4270 con 0x7f7d3c143ae0 2023-11-24T07:56:54.826 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:54.817+0000 7f7d33fff700 1 -- 172.21.15.59:0/451186138 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d240072d0 con 0x7f7d3c143ae0 2023-11-24T07:56:54.827 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:54.817+0000 7f7d41df8700 1 -- 172.21.15.59:0/451186138 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7d3c156040 con 0x7f7d3c143ae0 2023-11-24T07:56:54.827 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:54.818+0000 7f7d41df8700 1 -- 172.21.15.59:0/451186138 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7d3c075480 con 0x7f7d3c143ae0 2023-11-24T07:56:54.827 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:54.818+0000 7f7d33fff700 1 -- 172.21.15.59:0/451186138 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f7d24007430 con 0x7f7d3c143ae0 2023-11-24T07:56:54.827 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:54.818+0000 7f7d33fff700 1 --2- 172.21.15.59:0/451186138 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f7d1c06aed0 0x7f7d1c06d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:56:54.827 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:54.819+0000 7f7d33fff700 1 -- 172.21.15.59:0/451186138 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f7d24088790 con 0x7f7d3c143ae0 2023-11-24T07:56:54.827 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:54.819+0000 7f7d3b7fe700 1 --2- 172.21.15.59:0/451186138 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f7d1c06aed0 0x7f7d1c06d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:56:54.828 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:54.819+0000 7f7d3b7fe700 1 --2- 172.21.15.59:0/451186138 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f7d1c06aed0 0x7f7d1c06d390 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f7d2c006fd0 tx=0x7f7d2c008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:56:54.828 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:54.823+0000 7f7d33fff700 1 -- 172.21.15.59:0/451186138 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f7d240538f0 con 0x7f7d3c143ae0 2023-11-24T07:56:55.022 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:55.021+0000 7f7d41df8700 1 -- 172.21.15.59:0/451186138 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f7d3c1f4a70 con 0x7f7d3c143ae0 2023-11-24T07:56:55.066 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:55.021+0000 7f7d33fff700 1 -- 172.21.15.59:0/451186138 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v20) v1 ==== 93+0+10 (secure 0 0 0) 0x7f7d24017020 con 0x7f7d3c143ae0 2023-11-24T07:56:55.067 INFO:teuthology.orchestra.run.smithi059.stdout:pg_num: 1 2023-11-24T07:56:55.068 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:55.024+0000 7f7d41df8700 1 -- 172.21.15.59:0/451186138 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f7d1c06aed0 msgr2=0x7f7d1c06d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:56:55.069 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:55.024+0000 7f7d41df8700 1 --2- 172.21.15.59:0/451186138 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f7d1c06aed0 0x7f7d1c06d390 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f7d2c006fd0 tx=0x7f7d2c008040 comp rx=0 tx=0).stop 2023-11-24T07:56:55.069 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:55.024+0000 7f7d41df8700 1 -- 172.21.15.59:0/451186138 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7d3c143ae0 msgr2=0x7f7d3c158760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:56:55.069 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:55.024+0000 7f7d41df8700 1 --2- 172.21.15.59:0/451186138 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7d3c143ae0 0x7f7d3c158760 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f7d2400c480 tx=0x7f7d2400c790 comp rx=0 tx=0).stop 2023-11-24T07:56:55.069 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:55.024+0000 7f7d41df8700 1 -- 172.21.15.59:0/451186138 shutdown_connections 2023-11-24T07:56:55.069 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:55.024+0000 7f7d41df8700 1 --2- 172.21.15.59:0/451186138 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f7d1c06aed0 0x7f7d1c06d390 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:56:55.070 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:55.024+0000 7f7d41df8700 1 --2- 172.21.15.59:0/451186138 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7d3c1440f0 0x7f7d3c158ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:56:55.070 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:55.024+0000 7f7d41df8700 1 --2- 172.21.15.59:0/451186138 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f7d3c0722c0 0x7f7d3c155b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:56:55.070 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:55.024+0000 7f7d41df8700 1 --2- 172.21.15.59:0/451186138 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7d3c143ae0 0x7f7d3c158760 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:56:55.070 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:55.024+0000 7f7d41df8700 1 -- 172.21.15.59:0/451186138 >> 172.21.15.59:0/451186138 conn(0x7f7d3c0b2740 msgr2=0x7f7d3c0cf9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:56:55.071 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:55.024+0000 7f7d41df8700 1 -- 172.21.15.59:0/451186138 shutdown_connections 2023-11-24T07:56:55.071 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:55.024+0000 7f7d41df8700 1 -- 172.21.15.59:0/451186138 wait complete. 2023-11-24T07:56:55.071 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-24T07:56:55.199 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:55.198+0000 7f1afae80700 1 Processor -- start 2023-11-24T07:56:55.200 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:55.198+0000 7f1afae80700 1 -- start start 2023-11-24T07:56:55.206 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:55.198+0000 7f1afae80700 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f1af406b620 0x7f1af40b3a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:56:55.206 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:55.198+0000 7f1afae80700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f1af406ee60 0x7f1af40b3f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:56:55.207 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:55.199+0000 7f1afae80700 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1af4071f50 con 0x7f1af40b5ab0 2023-11-24T07:56:55.207 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:55.199+0000 7f1afae80700 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1af4072230 con 0x7f1af406ee60 2023-11-24T07:56:55.207 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:55.199+0000 7f1afae80700 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1af4072510 con 0x7f1af406b620 2023-11-24T07:56:55.207 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:55.199+0000 7f1af941d700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f1af406ee60 0x7f1af40b3f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:56:55.207 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:55.199+0000 7f1af941d700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f1af406ee60 0x7f1af40b3f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:50688/0 (socket says 172.21.15.59:50688) 2023-11-24T07:56:55.207 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:55.199+0000 7f1af941d700 1 -- 172.21.15.59:0/451628419 learned_addr learned my addr 172.21.15.59:0/451628419 (peer_addr_for_me v2:172.21.15.59:0/0) 2023-11-24T07:56:55.207 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:55.199+0000 7f1af8c1c700 1 --2- 172.21.15.59:0/451628419 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f1af406b620 0x7f1af40b3a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:56:55.208 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:55.199+0000 7f1af941d700 1 -- 172.21.15.59:0/451628419 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f1af406b620 msgr2=0x7f1af40b3a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T07:56:55.208 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:55.199+0000 7f1af941d700 1 --2- 172.21.15.59:0/451628419 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f1af406b620 0x7f1af40b3a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:56:55.208 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:55.199+0000 7f1af941d700 1 -- 172.21.15.59:0/451628419 >> v1:172.21.15.76:6789/0 conn(0x7f1af40b5ab0 legacy=0x7f1af4161ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:56:55.208 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:55.199+0000 7f1af941d700 1 -- 172.21.15.59:0/451628419 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1af40727f0 con 0x7f1af406ee60 2023-11-24T07:56:55.208 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:55.199+0000 7f1af941d700 1 --2- 172.21.15.59:0/451628419 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1af406ee60 0x7f1af40b3f80 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f1aec009e50 tx=0x7f1aec00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d879837541688f01 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:56:55.208 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:55.199+0000 7f1af37fe700 1 -- 172.21.15.59:0/451628419 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1aec00e070 con 0x7f1af406ee60 2023-11-24T07:56:55.208 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:55.199+0000 7f1af37fe700 1 -- 172.21.15.59:0/451628419 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1aec00b910 con 0x7f1af406ee60 2023-11-24T07:56:55.208 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:55.199+0000 7f1af37fe700 1 -- 172.21.15.59:0/451628419 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1aec00bcc0 con 0x7f1af406ee60 2023-11-24T07:56:55.209 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:55.200+0000 7f1afae80700 1 -- 172.21.15.59:0/451628419 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1af406ee60 msgr2=0x7f1af40b3f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:56:55.209 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:55.200+0000 7f1afae80700 1 --2- 172.21.15.59:0/451628419 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1af406ee60 0x7f1af40b3f80 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f1aec009e50 tx=0x7f1aec00b330 comp rx=0 tx=0).stop 2023-11-24T07:56:55.209 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:55.200+0000 7f1afae80700 1 -- 172.21.15.59:0/451628419 shutdown_connections 2023-11-24T07:56:55.209 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:55.200+0000 7f1afae80700 1 --2- 172.21.15.59:0/451628419 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f1af406b620 0x7f1af40b3a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:56:55.209 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:55.200+0000 7f1afae80700 1 --2- 172.21.15.59:0/451628419 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1af406ee60 0x7f1af40b3f80 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:56:55.209 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:55.200+0000 7f1afae80700 1 -- 172.21.15.59:0/451628419 >> 172.21.15.59:0/451628419 conn(0x7f1af40b2740 msgr2=0x7f1af40b2b20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:56:55.209 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:55.200+0000 7f1afae80700 1 -- 172.21.15.59:0/451628419 shutdown_connections 2023-11-24T07:56:55.210 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:55.200+0000 7f1afae80700 1 -- 172.21.15.59:0/451628419 wait complete. 2023-11-24T07:56:55.210 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:55.201+0000 7f1afae80700 1 Processor -- start 2023-11-24T07:56:55.210 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:55.201+0000 7f1afae80700 1 -- start start 2023-11-24T07:56:55.210 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:55.202+0000 7f1afae80700 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f1af406b620 0x7f1af40b1af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:56:55.210 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:55.202+0000 7f1afae80700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1af406ee60 0x7f1af4145f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:56:55.210 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:55.202+0000 7f1afae80700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1af40b5ab0 0x7f1af41464d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:56:55.210 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:55.202+0000 7f1afae80700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1af41f7310 con 0x7f1af406ee60 2023-11-24T07:56:55.211 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:55.202+0000 7f1afae80700 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1af41f4a80 con 0x7f1af40b5ab0 2023-11-24T07:56:55.211 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:55.202+0000 7f1afae80700 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1af41f7520 con 0x7f1af406b620 2023-11-24T07:56:55.211 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:55.202+0000 7f1af3fff700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1af406ee60 0x7f1af4145f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:56:55.211 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:55.202+0000 7f1af3fff700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1af406ee60 0x7f1af4145f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:50692/0 (socket says 172.21.15.59:50692) 2023-11-24T07:56:55.211 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:55.202+0000 7f1af3fff700 1 -- 172.21.15.59:0/1110885848 learned_addr learned my addr 172.21.15.59:0/1110885848 (peer_addr_for_me v2:172.21.15.59:0/0) 2023-11-24T07:56:55.211 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:55.202+0000 7f1af8c1c700 1 --2- 172.21.15.59:0/1110885848 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f1af406b620 0x7f1af40b1af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:56:55.211 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:55.202+0000 7f1af941d700 1 --2- 172.21.15.59:0/1110885848 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1af40b5ab0 0x7f1af41464d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:56:55.212 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:55.203+0000 7f1af3fff700 1 -- 172.21.15.59:0/1110885848 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f1af406b620 msgr2=0x7f1af40b1af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:56:55.212 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:55.203+0000 7f1af3fff700 1 --2- 172.21.15.59:0/1110885848 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f1af406b620 0x7f1af40b1af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:56:55.212 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:55.203+0000 7f1af3fff700 1 -- 172.21.15.59:0/1110885848 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1af40b5ab0 msgr2=0x7f1af41464d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:56:55.212 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:55.203+0000 7f1af3fff700 1 --2- 172.21.15.59:0/1110885848 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1af40b5ab0 0x7f1af41464d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:56:55.212 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:55.203+0000 7f1af3fff700 1 -- 172.21.15.59:0/1110885848 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1aec009b30 con 0x7f1af406ee60 2023-11-24T07:56:55.212 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:55.203+0000 7f1af3fff700 1 --2- 172.21.15.59:0/1110885848 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1af406ee60 0x7f1af4145f90 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f1ae000c420 tx=0x7f1ae000c730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:56:55.212 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:55.203+0000 7f1af17fa700 1 -- 172.21.15.59:0/1110885848 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ae00052d0 con 0x7f1af406ee60 2023-11-24T07:56:55.213 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:55.203+0000 7f1af17fa700 1 -- 172.21.15.59:0/1110885848 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1ae0005430 con 0x7f1af406ee60 2023-11-24T07:56:55.213 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:55.203+0000 7f1afae80700 1 -- 172.21.15.59:0/1110885848 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1af41f7800 con 0x7f1af406ee60 2023-11-24T07:56:55.213 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:55.203+0000 7f1af17fa700 1 -- 172.21.15.59:0/1110885848 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ae00038c0 con 0x7f1af406ee60 2023-11-24T07:56:55.213 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:55.203+0000 7f1afae80700 1 -- 172.21.15.59:0/1110885848 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1af40b2030 con 0x7f1af406ee60 2023-11-24T07:56:55.213 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:55.204+0000 7f1afae80700 1 -- 172.21.15.59:0/1110885848 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1af41f7660 con 0x7f1af406ee60 2023-11-24T07:56:55.213 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:55.204+0000 7f1af17fa700 1 -- 172.21.15.59:0/1110885848 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f1ae0003a20 con 0x7f1af406ee60 2023-11-24T07:56:55.213 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:55.205+0000 7f1af17fa700 1 --2- 172.21.15.59:0/1110885848 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f1ad806aed0 0x7f1ad806d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:56:55.213 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:55.205+0000 7f1af17fa700 1 -- 172.21.15.59:0/1110885848 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f1ae00888a0 con 0x7f1af406ee60 2023-11-24T07:56:55.214 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:55.205+0000 7f1af8c1c700 1 --2- 172.21.15.59:0/1110885848 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f1ad806aed0 0x7f1ad806d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:56:55.214 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:55.206+0000 7f1af8c1c700 1 --2- 172.21.15.59:0/1110885848 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f1ad806aed0 0x7f1ad806d390 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f1af4146c70 tx=0x7f1ae8009210 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:56:55.214 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:55.210+0000 7f1af17fa700 1 -- 172.21.15.59:0/1110885848 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f1af41f7660 con 0x7f1af406ee60 2023-11-24T07:56:55.412 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:55.410+0000 7f1afae80700 1 -- 172.21.15.59:0/1110885848 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f1af41f7660 con 0x7f1af406ee60 2023-11-24T07:56:55.412 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:55.410+0000 7f1af17fa700 1 -- 172.21.15.59:0/1110885848 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v20) v1 ==== 104+0+11 (secure 0 0 0) 0x7f1ae0017020 con 0x7f1af406ee60 2023-11-24T07:56:55.412 INFO:teuthology.orchestra.run.smithi059.stdout:pg_num: 32 2023-11-24T07:56:55.414 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:55.412+0000 7f1afae80700 1 -- 172.21.15.59:0/1110885848 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f1ad806aed0 msgr2=0x7f1ad806d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:56:55.415 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:55.412+0000 7f1afae80700 1 --2- 172.21.15.59:0/1110885848 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f1ad806aed0 0x7f1ad806d390 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f1af4146c70 tx=0x7f1ae8009210 comp rx=0 tx=0).stop 2023-11-24T07:56:55.415 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:55.412+0000 7f1afae80700 1 -- 172.21.15.59:0/1110885848 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1af406ee60 msgr2=0x7f1af4145f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:56:55.415 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:55.412+0000 7f1afae80700 1 --2- 172.21.15.59:0/1110885848 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1af406ee60 0x7f1af4145f90 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f1ae000c420 tx=0x7f1ae000c730 comp rx=0 tx=0).stop 2023-11-24T07:56:55.415 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:55.413+0000 7f1afae80700 1 -- 172.21.15.59:0/1110885848 shutdown_connections 2023-11-24T07:56:55.415 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:55.413+0000 7f1afae80700 1 --2- 172.21.15.59:0/1110885848 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f1ad806aed0 0x7f1ad806d390 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:56:55.415 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:55.413+0000 7f1afae80700 1 --2- 172.21.15.59:0/1110885848 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1af40b5ab0 0x7f1af41464d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:56:55.415 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:55.413+0000 7f1afae80700 1 --2- 172.21.15.59:0/1110885848 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f1af406b620 0x7f1af40b1af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:56:55.415 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:55.413+0000 7f1afae80700 1 --2- 172.21.15.59:0/1110885848 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1af406ee60 0x7f1af4145f90 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:56:55.415 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:55.413+0000 7f1afae80700 1 -- 172.21.15.59:0/1110885848 >> 172.21.15.59:0/1110885848 conn(0x7f1af40b2740 msgr2=0x7f1af4147660 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:56:55.415 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:55.413+0000 7f1afae80700 1 -- 172.21.15.59:0/1110885848 shutdown_connections 2023-11-24T07:56:55.415 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:55.413+0000 7f1afae80700 1 -- 172.21.15.59:0/1110885848 wait complete. 2023-11-24T07:56:55.427 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-24T07:56:55.555 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:55.554+0000 7fdda9609700 1 Processor -- start 2023-11-24T07:56:55.556 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:55.554+0000 7fdda9609700 1 -- start start 2023-11-24T07:56:55.558 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:55.555+0000 7fdda9609700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fdda41459c0 0x7fdda415c540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:56:55.558 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:55.555+0000 7fdda9609700 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fdda40cd5d0 0x7fdda406afd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:56:55.558 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:55.555+0000 7fdda9609700 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdda406daf0 con 0x7fdda40cda50 2023-11-24T07:56:55.558 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:55.555+0000 7fdda9609700 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdda40cfe60 con 0x7fdda41459c0 2023-11-24T07:56:55.558 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:55.555+0000 7fdda9609700 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdda40d0140 con 0x7fdda40cd5d0 2023-11-24T07:56:55.558 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:55.555+0000 7fdda2ffd700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fdda41459c0 0x7fdda415c540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:56:55.558 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:55.555+0000 7fdda2ffd700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fdda41459c0 0x7fdda415c540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:50696/0 (socket says 172.21.15.59:50696) 2023-11-24T07:56:55.559 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:55.555+0000 7fdda37fe700 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7fdda40cda50 0x7fdda406e260 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:54170/0 (socket says 172.21.15.59:54170) 2023-11-24T07:56:55.559 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:55.555+0000 7fdda2ffd700 1 -- 172.21.15.59:0/94016220 learned_addr learned my addr 172.21.15.59:0/94016220 (peer_addr_for_me v2:172.21.15.59:0/0) 2023-11-24T07:56:55.559 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:55.555+0000 7fdda27fc700 1 --2- 172.21.15.59:0/94016220 >> v2:172.21.15.76:3300/0 conn(0x7fdda40cd5d0 0x7fdda406afd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:56:55.559 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:55.555+0000 7fdda2ffd700 1 -- 172.21.15.59:0/94016220 >> v2:172.21.15.76:3300/0 conn(0x7fdda40cd5d0 msgr2=0x7fdda406afd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T07:56:55.559 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:55.555+0000 7fdda2ffd700 1 --2- 172.21.15.59:0/94016220 >> v2:172.21.15.76:3300/0 conn(0x7fdda40cd5d0 0x7fdda406afd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:56:55.559 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:55.555+0000 7fdda2ffd700 1 -- 172.21.15.59:0/94016220 >> v1:172.21.15.59:6789/0 conn(0x7fdda40cda50 legacy=0x7fdda406e260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:56:55.559 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:55.555+0000 7fdda1ffb700 1 -- 172.21.15.59:0/94016220 <== mon.0 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4002618451 0 0) 0x7fdda406daf0 con 0x7fdda40cda50 2023-11-24T07:56:55.559 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:55.555+0000 7fdda2ffd700 1 -- 172.21.15.59:0/94016220 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdda40d0420 con 0x7fdda41459c0 2023-11-24T07:56:55.559 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:55.555+0000 7fdda2ffd700 1 --2- 172.21.15.59:0/94016220 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fdda41459c0 0x7fdda415c540 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fdda40d0b90 tx=0x7fdd94009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c5ac63a25ad40e14 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:56:55.559 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:55.555+0000 7fdda1ffb700 1 -- 172.21.15.59:0/94016220 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd9400b870 con 0x7fdda41459c0 2023-11-24T07:56:55.560 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:55.555+0000 7fdda1ffb700 1 -- 172.21.15.59:0/94016220 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdd9400b9d0 con 0x7fdda41459c0 2023-11-24T07:56:55.560 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:55.556+0000 7fdda1ffb700 1 -- 172.21.15.59:0/94016220 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd9400bd60 con 0x7fdda41459c0 2023-11-24T07:56:55.560 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:55.556+0000 7fdda9609700 1 -- 172.21.15.59:0/94016220 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fdda41459c0 msgr2=0x7fdda415c540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:56:55.560 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:55.556+0000 7fdda9609700 1 --2- 172.21.15.59:0/94016220 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fdda41459c0 0x7fdda415c540 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fdda40d0b90 tx=0x7fdd94009bd0 comp rx=0 tx=0).stop 2023-11-24T07:56:55.560 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:55.556+0000 7fdda9609700 1 -- 172.21.15.59:0/94016220 shutdown_connections 2023-11-24T07:56:55.560 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:55.556+0000 7fdda9609700 1 --2- 172.21.15.59:0/94016220 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fdda41459c0 0x7fdda415c540 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:56:55.560 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:55.556+0000 7fdda9609700 1 --2- 172.21.15.59:0/94016220 >> v2:172.21.15.76:3300/0 conn(0x7fdda40cd5d0 0x7fdda406afd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:56:55.560 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:55.556+0000 7fdda9609700 1 -- 172.21.15.59:0/94016220 >> 172.21.15.59:0/94016220 conn(0x7fdda40b27b0 msgr2=0x7fdda40b2b90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:56:55.560 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:55.556+0000 7fdda9609700 1 -- 172.21.15.59:0/94016220 shutdown_connections 2023-11-24T07:56:55.560 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:55.556+0000 7fdda9609700 1 -- 172.21.15.59:0/94016220 wait complete. 2023-11-24T07:56:55.563 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:55.557+0000 7fdda9609700 1 Processor -- start 2023-11-24T07:56:55.563 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:55.557+0000 7fdda9609700 1 -- start start 2023-11-24T07:56:55.563 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:55.558+0000 7fdda9609700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fdda40cd5d0 0x7fdda413cdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:56:55.563 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:55.558+0000 7fdda9609700 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fdda40cda50 0x7fdda4152190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:56:55.563 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:55.558+0000 7fdda9609700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fdda41459c0 0x7fdda413d4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:56:55.563 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:55.558+0000 7fdda9609700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdda413ddb0 con 0x7fdda41459c0 2023-11-24T07:56:55.564 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:55.558+0000 7fdda9609700 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdda40b3bb0 con 0x7fdda40cd5d0 2023-11-24T07:56:55.564 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:55.558+0000 7fdda9609700 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdda40b4410 con 0x7fdda40cda50 2023-11-24T07:56:55.564 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:55.558+0000 7fdda37fe700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fdda41459c0 0x7fdda413d4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:56:55.564 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:55.558+0000 7fdda37fe700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fdda41459c0 0x7fdda413d4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:50706/0 (socket says 172.21.15.59:50706) 2023-11-24T07:56:55.564 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:55.558+0000 7fdda37fe700 1 -- 172.21.15.59:0/2967371310 learned_addr learned my addr 172.21.15.59:0/2967371310 (peer_addr_for_me v2:172.21.15.59:0/0) 2023-11-24T07:56:55.564 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:55.558+0000 7fdda27fc700 1 --2- 172.21.15.59:0/2967371310 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fdda40cda50 0x7fdda4152190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:56:55.564 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:55.558+0000 7fdda2ffd700 1 --2- 172.21.15.59:0/2967371310 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fdda40cd5d0 0x7fdda413cdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:56:55.564 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:55.558+0000 7fdda37fe700 1 -- 172.21.15.59:0/2967371310 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fdda40cda50 msgr2=0x7fdda4152190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:56:55.564 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:55.558+0000 7fdda37fe700 1 --2- 172.21.15.59:0/2967371310 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fdda40cda50 0x7fdda4152190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:56:55.564 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:55.558+0000 7fdda37fe700 1 -- 172.21.15.59:0/2967371310 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fdda40cd5d0 msgr2=0x7fdda413cdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:56:55.564 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:55.559+0000 7fdda37fe700 1 --2- 172.21.15.59:0/2967371310 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fdda40cd5d0 0x7fdda413cdc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:56:55.565 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:55.559+0000 7fdda37fe700 1 -- 172.21.15.59:0/2967371310 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdd94009550 con 0x7fdda41459c0 2023-11-24T07:56:55.565 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:55.559+0000 7fdda27fc700 1 --2- 172.21.15.59:0/2967371310 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fdda40cda50 0x7fdda4152190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T07:56:55.565 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:55.559+0000 7fdda2ffd700 1 --2- 172.21.15.59:0/2967371310 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fdda40cd5d0 0x7fdda413cdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T07:56:55.565 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:55.559+0000 7fdda37fe700 1 --2- 172.21.15.59:0/2967371310 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fdda41459c0 0x7fdda413d4b0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fdd9c00cc10 tx=0x7fdd9c00a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:56:55.565 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:55.559+0000 7fdd8b7fe700 1 -- 172.21.15.59:0/2967371310 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd9c00abb0 con 0x7fdda41459c0 2023-11-24T07:56:55.565 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:55.559+0000 7fdd8b7fe700 1 -- 172.21.15.59:0/2967371310 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdd9c00ad10 con 0x7fdda41459c0 2023-11-24T07:56:55.565 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:55.559+0000 7fdda9609700 1 -- 172.21.15.59:0/2967371310 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdda413dad0 con 0x7fdda41459c0 2023-11-24T07:56:55.565 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:55.559+0000 7fdd8b7fe700 1 -- 172.21.15.59:0/2967371310 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd9c007310 con 0x7fdda41459c0 2023-11-24T07:56:55.565 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:55.559+0000 7fdda9609700 1 -- 172.21.15.59:0/2967371310 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdda40b3e70 con 0x7fdda41459c0 2023-11-24T07:56:55.565 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:55.560+0000 7fdda9609700 1 -- 172.21.15.59:0/2967371310 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdda40b5b80 con 0x7fdda41459c0 2023-11-24T07:56:55.565 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:55.560+0000 7fdd8b7fe700 1 -- 172.21.15.59:0/2967371310 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fdd9c007470 con 0x7fdda41459c0 2023-11-24T07:56:55.565 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:55.561+0000 7fdd8b7fe700 1 --2- 172.21.15.59:0/2967371310 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7fdd9006aed0 0x7fdd9006d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:56:55.566 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:55.561+0000 7fdd8b7fe700 1 -- 172.21.15.59:0/2967371310 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fdd9c0893d0 con 0x7fdda41459c0 2023-11-24T07:56:55.566 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:55.561+0000 7fdda2ffd700 1 --2- 172.21.15.59:0/2967371310 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7fdd9006aed0 0x7fdd9006d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:56:55.566 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:55.561+0000 7fdda2ffd700 1 --2- 172.21.15.59:0/2967371310 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7fdd9006aed0 0x7fdd9006d390 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fdd94009130 tx=0x7fdd94008030 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:56:55.566 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:55.565+0000 7fdd8b7fe700 1 -- 172.21.15.59:0/2967371310 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fdd9c054480 con 0x7fdda41459c0 2023-11-24T07:56:55.770 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:55.768+0000 7fdda9609700 1 -- 172.21.15.59:0/2967371310 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fdda40b4580 con 0x7fdda41459c0 2023-11-24T07:56:55.770 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:55.768+0000 7fdd8b7fe700 1 -- 172.21.15.59:0/2967371310 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v20) v1 ==== 100+0+11 (secure 0 0 0) 0x7fdd9c057aa0 con 0x7fdda41459c0 2023-11-24T07:56:55.770 INFO:teuthology.orchestra.run.smithi059.stdout:pg_num: 64 2023-11-24T07:56:55.772 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:55.770+0000 7fdda9609700 1 -- 172.21.15.59:0/2967371310 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7fdd9006aed0 msgr2=0x7fdd9006d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:56:55.772 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:55.770+0000 7fdda9609700 1 --2- 172.21.15.59:0/2967371310 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7fdd9006aed0 0x7fdd9006d390 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fdd94009130 tx=0x7fdd94008030 comp rx=0 tx=0).stop 2023-11-24T07:56:55.772 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:55.770+0000 7fdda9609700 1 -- 172.21.15.59:0/2967371310 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fdda41459c0 msgr2=0x7fdda413d4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:56:55.772 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:55.770+0000 7fdda9609700 1 --2- 172.21.15.59:0/2967371310 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fdda41459c0 0x7fdda413d4b0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fdd9c00cc10 tx=0x7fdd9c00a060 comp rx=0 tx=0).stop 2023-11-24T07:56:55.772 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:55.770+0000 7fdda9609700 1 -- 172.21.15.59:0/2967371310 shutdown_connections 2023-11-24T07:56:55.773 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:55.770+0000 7fdda9609700 1 --2- 172.21.15.59:0/2967371310 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7fdd9006aed0 0x7fdd9006d390 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:56:55.773 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:55.770+0000 7fdda9609700 1 --2- 172.21.15.59:0/2967371310 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fdda41459c0 0x7fdda413d4b0 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:56:55.773 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:55.770+0000 7fdda9609700 1 --2- 172.21.15.59:0/2967371310 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fdda40cd5d0 0x7fdda413cdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:56:55.773 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:55.770+0000 7fdda9609700 1 --2- 172.21.15.59:0/2967371310 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fdda40cda50 0x7fdda4152190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:56:55.773 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:55.770+0000 7fdda9609700 1 -- 172.21.15.59:0/2967371310 >> 172.21.15.59:0/2967371310 conn(0x7fdda40b27b0 msgr2=0x7fdda4144cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:56:55.773 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:55.770+0000 7fdda9609700 1 -- 172.21.15.59:0/2967371310 shutdown_connections 2023-11-24T07:56:55.773 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:55.770+0000 7fdda9609700 1 -- 172.21.15.59:0/2967371310 wait complete. 2023-11-24T07:56:55.784 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-24T07:56:55.908 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:55.906+0000 7fd8879e6700 1 Processor -- start 2023-11-24T07:56:55.908 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:55.907+0000 7fd8879e6700 1 -- start start 2023-11-24T07:56:55.911 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:55.907+0000 7fd8879e6700 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fd8800722c0 0x7fd8800b51b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:56:55.911 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:55.907+0000 7fd8879e6700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fd8801440f0 0x7fd88006e510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:56:55.911 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:55.908+0000 7fd8879e6700 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd880143ec0 con 0x7fd880143ae0 2023-11-24T07:56:55.911 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:55.908+0000 7fd8879e6700 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd8800b40a0 con 0x7fd8801440f0 2023-11-24T07:56:55.911 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:55.908+0000 7fd8879e6700 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd8800b3ae0 con 0x7fd8800722c0 2023-11-24T07:56:55.911 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:55.908+0000 7fd885f83700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fd8801440f0 0x7fd88006e510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:56:55.911 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:55.908+0000 7fd885782700 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7fd880143ae0 0x7fd880071bb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:54184/0 (socket says 172.21.15.59:54184) 2023-11-24T07:56:55.911 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:55.908+0000 7fd885782700 1 -- 172.21.15.59:0/507126201 learned_addr learned my addr 172.21.15.59:0/507126201 (peer_addr_for_me v1:172.21.15.59:0/0) 2023-11-24T07:56:55.912 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:55.908+0000 7fd884f81700 1 --2- 172.21.15.59:0/507126201 >> v2:172.21.15.76:3300/0 conn(0x7fd8800722c0 0x7fd8800b51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:56:55.912 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:55.908+0000 7fd885f83700 1 -- 172.21.15.59:0/507126201 >> v2:172.21.15.76:3300/0 conn(0x7fd8800722c0 msgr2=0x7fd8800b51b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T07:56:55.912 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:55.908+0000 7fd885f83700 1 --2- 172.21.15.59:0/507126201 >> v2:172.21.15.76:3300/0 conn(0x7fd8800722c0 0x7fd8800b51b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:56:55.912 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:55.908+0000 7fd885f83700 1 -- 172.21.15.59:0/507126201 >> v1:172.21.15.59:6789/0 conn(0x7fd880143ae0 legacy=0x7fd880071bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:56:55.912 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:55.908+0000 7fd877fff700 1 -- 172.21.15.59:0/507126201 <== mon.0 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1258077457 0 0) 0x7fd880143ec0 con 0x7fd880143ae0 2023-11-24T07:56:55.912 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:55.908+0000 7fd885f83700 1 -- 172.21.15.59:0/507126201 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd8800b3dc0 con 0x7fd8801440f0 2023-11-24T07:56:55.912 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:55.908+0000 7fd884f81700 1 --2- 172.21.15.59:0/507126201 >> v2:172.21.15.76:3300/0 conn(0x7fd8800722c0 0x7fd8800b51b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T07:56:55.912 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:55.908+0000 7fd885f83700 1 --2- 172.21.15.59:0/507126201 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd8801440f0 0x7fd88006e510 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fd8800d1b00 tx=0x7fd87800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4c6d5b2a8d446ec7 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:56:55.912 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:55.908+0000 7fd877fff700 1 -- 172.21.15.59:0/507126201 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd87800e070 con 0x7fd8801440f0 2023-11-24T07:56:55.912 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:55.908+0000 7fd877fff700 1 -- 172.21.15.59:0/507126201 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd87800bbd0 con 0x7fd8801440f0 2023-11-24T07:56:55.912 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:55.908+0000 7fd877fff700 1 -- 172.21.15.59:0/507126201 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd878012430 con 0x7fd8801440f0 2023-11-24T07:56:55.912 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:55.909+0000 7fd8879e6700 1 -- 172.21.15.59:0/507126201 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd8801440f0 msgr2=0x7fd88006e510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:56:55.913 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:55.909+0000 7fd8879e6700 1 --2- 172.21.15.59:0/507126201 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd8801440f0 0x7fd88006e510 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fd8800d1b00 tx=0x7fd87800b330 comp rx=0 tx=0).stop 2023-11-24T07:56:55.913 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:55.909+0000 7fd8879e6700 1 -- 172.21.15.59:0/507126201 shutdown_connections 2023-11-24T07:56:55.913 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:55.909+0000 7fd8879e6700 1 --2- 172.21.15.59:0/507126201 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd8801440f0 0x7fd88006e510 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:56:55.913 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:55.909+0000 7fd8879e6700 1 --2- 172.21.15.59:0/507126201 >> v2:172.21.15.76:3300/0 conn(0x7fd8800722c0 0x7fd8800b51b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:56:55.913 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:55.909+0000 7fd8879e6700 1 -- 172.21.15.59:0/507126201 >> 172.21.15.59:0/507126201 conn(0x7fd8800b2740 msgr2=0x7fd8800b2b20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:56:55.913 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:55.909+0000 7fd8879e6700 1 -- 172.21.15.59:0/507126201 shutdown_connections 2023-11-24T07:56:55.913 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:55.909+0000 7fd8879e6700 1 -- 172.21.15.59:0/507126201 wait complete. 2023-11-24T07:56:55.915 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:55.910+0000 7fd8879e6700 1 Processor -- start 2023-11-24T07:56:55.916 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:55.910+0000 7fd8879e6700 1 -- start start 2023-11-24T07:56:55.916 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:55.911+0000 7fd8879e6700 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fd8800722c0 0x7fd88013b4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:56:55.916 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:55.911+0000 7fd8879e6700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd880143ae0 0x7fd880151670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:56:55.916 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:55.911+0000 7fd8879e6700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd8801440f0 0x7fd880151bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:56:55.916 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:55.911+0000 7fd8879e6700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd88015deb0 con 0x7fd8801440f0 2023-11-24T07:56:55.916 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:55.911+0000 7fd8879e6700 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd88013ba30 con 0x7fd880143ae0 2023-11-24T07:56:55.916 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:55.911+0000 7fd8879e6700 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd8801546c0 con 0x7fd8800722c0 2023-11-24T07:56:55.916 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:55.911+0000 7fd885f83700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd8801440f0 0x7fd880151bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:56:55.916 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:55.911+0000 7fd885f83700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd8801440f0 0x7fd880151bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:50720/0 (socket says 172.21.15.59:50720) 2023-11-24T07:56:55.916 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:55.911+0000 7fd885f83700 1 -- 172.21.15.59:0/501077118 learned_addr learned my addr 172.21.15.59:0/501077118 (peer_addr_for_me v2:172.21.15.59:0/0) 2023-11-24T07:56:55.917 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:55.911+0000 7fd885782700 1 --2- 172.21.15.59:0/501077118 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fd8800722c0 0x7fd88013b4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:56:55.917 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:55.911+0000 7fd884f81700 1 --2- 172.21.15.59:0/501077118 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd880143ae0 0x7fd880151670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:56:55.917 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:55.911+0000 7fd885f83700 1 -- 172.21.15.59:0/501077118 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fd8800722c0 msgr2=0x7fd88013b4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:56:55.917 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:55.911+0000 7fd885f83700 1 --2- 172.21.15.59:0/501077118 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fd8800722c0 0x7fd88013b4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:56:55.917 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:55.911+0000 7fd885f83700 1 -- 172.21.15.59:0/501077118 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd880143ae0 msgr2=0x7fd880151670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:56:55.918 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:55.911+0000 7fd885f83700 1 --2- 172.21.15.59:0/501077118 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd880143ae0 0x7fd880151670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:56:55.918 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:55.911+0000 7fd885f83700 1 -- 172.21.15.59:0/501077118 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd878009a70 con 0x7fd8801440f0 2023-11-24T07:56:55.919 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:55.912+0000 7fd885f83700 1 --2- 172.21.15.59:0/501077118 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd8801440f0 0x7fd880151bb0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fd8800d1bd0 tx=0x7fd878004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:56:55.919 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:55.912+0000 7fd875ffb700 1 -- 172.21.15.59:0/501077118 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd878012430 con 0x7fd8801440f0 2023-11-24T07:56:55.919 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:55.912+0000 7fd875ffb700 1 -- 172.21.15.59:0/501077118 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd878020070 con 0x7fd8801440f0 2023-11-24T07:56:55.919 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:55.912+0000 7fd875ffb700 1 -- 172.21.15.59:0/501077118 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd878004280 con 0x7fd8801440f0 2023-11-24T07:56:55.920 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:55.912+0000 7fd8879e6700 1 -- 172.21.15.59:0/501077118 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd880153620 con 0x7fd8801440f0 2023-11-24T07:56:55.920 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:55.912+0000 7fd8879e6700 1 -- 172.21.15.59:0/501077118 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd880153be0 con 0x7fd8801440f0 2023-11-24T07:56:55.920 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:55.913+0000 7fd8879e6700 1 -- 172.21.15.59:0/501077118 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd880068670 con 0x7fd8801440f0 2023-11-24T07:56:55.921 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:55.913+0000 7fd875ffb700 1 -- 172.21.15.59:0/501077118 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fd8780043e0 con 0x7fd8801440f0 2023-11-24T07:56:55.921 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:55.914+0000 7fd875ffb700 1 --2- 172.21.15.59:0/501077118 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7fd86406aed0 0x7fd86406d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:56:55.921 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:55.914+0000 7fd875ffb700 1 -- 172.21.15.59:0/501077118 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fd878022070 con 0x7fd8801440f0 2023-11-24T07:56:55.922 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:55.914+0000 7fd885782700 1 --2- 172.21.15.59:0/501077118 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7fd86406aed0 0x7fd86406d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:56:55.923 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:55.915+0000 7fd885782700 1 --2- 172.21.15.59:0/501077118 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7fd86406aed0 0x7fd86406d390 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fd870007fd0 tx=0x7fd870007de0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:56:55.923 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:55.918+0000 7fd875ffb700 1 -- 172.21.15.59:0/501077118 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fd878056480 con 0x7fd8801440f0 2023-11-24T07:56:56.119 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:56.118+0000 7fd8879e6700 1 -- 172.21.15.59:0/501077118 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fd8800b5d20 con 0x7fd8801440f0 2023-11-24T07:56:56.120 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:56.119+0000 7fd875ffb700 1 -- 172.21.15.59:0/501077118 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) v1 ==== 74+0+16437 (secure 0 0 0) 0x7fd878012e60 con 0x7fd8801440f0 2023-11-24T07:56:56.121 INFO:teuthology.orchestra.run.smithi059.stdout: 2023-11-24T07:56:56.121 INFO:teuthology.orchestra.run.smithi059.stdout:{"epoch":20,"fsid":"9f0e8741-b8f9-48d5-a2c3-09f45903303d","created":"2023-11-24T07:56:06.552729+0000","modified":"2023-11-24T07:56:25.705091+0000","last_up_change":"2023-11-24T07:56:17.372352+0000","last_in_change":"2023-11-24T07:56:09.073250+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-24T07:56:18.352823+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-11-24T07:56:22.185140+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-11-24T07:56:23.084326+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"6d8f9e9b-1782-4e41-928b-a39f2a8ebc06","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6800","nonce":3373878772},{"type":"v1","addr":"172.21.15.59:6801","nonce":3373878772}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6802","nonce":3373878772},{"type":"v1","addr":"172.21.15.59:6803","nonce":3373878772}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6806","nonce":3373878772},{"type":"v1","addr":"172.21.15.59:6807","nonce":3373878772}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6804","nonce":3373878772},{"type":"v1","addr":"172.21.15.59:6805","nonce":3373878772}]},"public_addr":"172.21.15.59:6801/3373878772","cluster_addr":"172.21.15.59:6803/3373878772","heartbeat_back_addr":"172.21.15.59:6807/3373878772","heartbeat_front_addr":"172.21.15.59:6805/3373878772","state":["exists","up"]},{"osd":1,"uuid":"e8d882aa-52e1-4c04-9e1e-c44ef9204a5f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6824","nonce":648258563},{"type":"v1","addr":"172.21.15.59:6825","nonce":648258563}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6826","nonce":648258563},{"type":"v1","addr":"172.21.15.59:6827","nonce":648258563}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6830","nonce":648258563},{"type":"v1","addr":"172.21.15.59:6831","nonce":648258563}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6828","nonce":648258563},{"type":"v1","addr":"172.21.15.59:6829","nonce":648258563}]},"public_addr":"172.21.15.59:6825/648258563","cluster_addr":"172.21.15.59:6827/648258563","heartbeat_back_addr":"172.21.15.59:6831/648258563","heartbeat_front_addr":"172.21.15.59:6829/648258563","state":["exists","up"]},{"osd":2,"uuid":"69eebfa0-695b-4b21-90b9-8a3537168a01","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6808","nonce":1840430654},{"type":"v1","addr":"172.21.15.59:6809","nonce":1840430654}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6810","nonce":1840430654},{"type":"v1","addr":"172.21.15.59:6811","nonce":1840430654}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6814","nonce":1840430654},{"type":"v1","addr":"172.21.15.59:6815","nonce":1840430654}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6812","nonce":1840430654},{"type":"v1","addr":"172.21.15.59:6813","nonce":1840430654}]},"public_addr":"172.21.15.59:6809/1840430654","cluster_addr":"172.21.15.59:6811/1840430654","heartbeat_back_addr":"172.21.15.59:6815/1840430654","heartbeat_front_addr":"172.21.15.59:6813/1840430654","state":["exists","up"]},{"osd":3,"uuid":"96fd77fc-f911-46ff-9445-6f0be2daa8f4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6816","nonce":2504025850},{"type":"v1","addr":"172.21.15.59:6817","nonce":2504025850}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6818","nonce":2504025850},{"type":"v1","addr":"172.21.15.59:6819","nonce":2504025850}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6822","nonce":2504025850},{"type":"v1","addr":"172.21.15.59:6823","nonce":2504025850}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6820","nonce":2504025850},{"type":"v1","addr":"172.21.15.59:6821","nonce":2504025850}]},"public_addr":"172.21.15.59:6817/2504025850","cluster_addr":"172.21.15.59:6819/2504025850","heartbeat_back_addr":"172.21.15.59:6823/2504025850","heartbeat_front_addr":"172.21.15.59:6821/2504025850","state":["exists","up"]},{"osd":4,"uuid":"7ac29fca-b5b2-4c95-9d69-2f164cbbc708","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6816","nonce":2382463431},{"type":"v1","addr":"172.21.15.76:6818","nonce":2382463431}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6820","nonce":2382463431},{"type":"v1","addr":"172.21.15.76:6822","nonce":2382463431}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6828","nonce":2382463431},{"type":"v1","addr":"172.21.15.76:6830","nonce":2382463431}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6824","nonce":2382463431},{"type":"v1","addr":"172.21.15.76:6826","nonce":2382463431}]},"public_addr":"172.21.15.76:6818/2382463431","cluster_addr":"172.21.15.76:6822/2382463431","heartbeat_back_addr":"172.21.15.76:6830/2382463431","heartbeat_front_addr":"172.21.15.76:6826/2382463431","state":["exists","up"]},{"osd":5,"uuid":"485bf711-3851-4238-a4f4-51d52e0a9c58","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6800","nonce":329334308},{"type":"v1","addr":"172.21.15.76:6802","nonce":329334308}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6805","nonce":329334308},{"type":"v1","addr":"172.21.15.76:6807","nonce":329334308}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6814","nonce":329334308},{"type":"v1","addr":"172.21.15.76:6815","nonce":329334308}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6810","nonce":329334308},{"type":"v1","addr":"172.21.15.76:6812","nonce":329334308}]},"public_addr":"172.21.15.76:6802/329334308","cluster_addr":"172.21.15.76:6807/329334308","heartbeat_back_addr":"172.21.15.76:6815/329334308","heartbeat_front_addr":"172.21.15.76:6812/329334308","state":["exists","up"]},{"osd":6,"uuid":"72b3f2fc-1e4a-4e32-968d-003647a0286f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6801","nonce":3946579410},{"type":"v1","addr":"172.21.15.76:6803","nonce":3946579410}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6804","nonce":3946579410},{"type":"v1","addr":"172.21.15.76:6806","nonce":3946579410}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6811","nonce":3946579410},{"type":"v1","addr":"172.21.15.76:6813","nonce":3946579410}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6808","nonce":3946579410},{"type":"v1","addr":"172.21.15.76:6809","nonce":3946579410}]},"public_addr":"172.21.15.76:6803/3946579410","cluster_addr":"172.21.15.76:6806/3946579410","heartbeat_back_addr":"172.21.15.76:6813/3946579410","heartbeat_front_addr":"172.21.15.76:6809/3946579410","state":["exists","up"]},{"osd":7,"uuid":"1ddceb90-6c5d-4d1a-ad56-99cb877f8c70","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6817","nonce":841761538},{"type":"v1","addr":"172.21.15.76:6819","nonce":841761538}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6821","nonce":841761538},{"type":"v1","addr":"172.21.15.76:6823","nonce":841761538}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6829","nonce":841761538},{"type":"v1","addr":"172.21.15.76:6831","nonce":841761538}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6825","nonce":841761538},{"type":"v1","addr":"172.21.15.76:6827","nonce":841761538}]},"public_addr":"172.21.15.76:6819/841761538","cluster_addr":"172.21.15.76:6823/841761538","heartbeat_back_addr":"172.21.15.76:6831/841761538","heartbeat_front_addr":"172.21.15.76:6827/841761538","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T07:56:14.294838+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T07:56:14.270779+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T07:56:14.231809+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T07:56:14.260983+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T07:56:14.302997+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T07:56:14.258600+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T07:56:14.251413+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T07:56:14.304727+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-24T07:56:56.123 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:56.121+0000 7fd8879e6700 1 -- 172.21.15.59:0/501077118 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7fd86406aed0 msgr2=0x7fd86406d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:56:56.123 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:56.121+0000 7fd8879e6700 1 --2- 172.21.15.59:0/501077118 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7fd86406aed0 0x7fd86406d390 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fd870007fd0 tx=0x7fd870007de0 comp rx=0 tx=0).stop 2023-11-24T07:56:56.123 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:56.121+0000 7fd8879e6700 1 -- 172.21.15.59:0/501077118 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd8801440f0 msgr2=0x7fd880151bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:56:56.123 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:56.121+0000 7fd8879e6700 1 --2- 172.21.15.59:0/501077118 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd8801440f0 0x7fd880151bb0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fd8800d1bd0 tx=0x7fd878004ad0 comp rx=0 tx=0).stop 2023-11-24T07:56:56.123 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:56.121+0000 7fd8879e6700 1 -- 172.21.15.59:0/501077118 shutdown_connections 2023-11-24T07:56:56.123 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:56.121+0000 7fd8879e6700 1 --2- 172.21.15.59:0/501077118 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7fd86406aed0 0x7fd86406d390 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:56:56.123 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:56.121+0000 7fd8879e6700 1 --2- 172.21.15.59:0/501077118 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd8801440f0 0x7fd880151bb0 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:56:56.123 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:56.121+0000 7fd8879e6700 1 --2- 172.21.15.59:0/501077118 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fd8800722c0 0x7fd88013b4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:56:56.123 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:56.121+0000 7fd8879e6700 1 --2- 172.21.15.59:0/501077118 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd880143ae0 0x7fd880151670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:56:56.124 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:56.121+0000 7fd8879e6700 1 -- 172.21.15.59:0/501077118 >> 172.21.15.59:0/501077118 conn(0x7fd8800b2740 msgr2=0x7fd8801410d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:56:56.124 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:56.121+0000 7fd8879e6700 1 -- 172.21.15.59:0/501077118 shutdown_connections 2023-11-24T07:56:56.124 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:56.121+0000 7fd8879e6700 1 -- 172.21.15.59:0/501077118 wait complete. 2023-11-24T07:56:56.135 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-24T07:56:18.352823+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-24T07:56:22.185140+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-11-24T07:56:23.084326+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-24T07:56:56.135 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-24T07:56:56.263 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:56.261+0000 7f6ecc9db700 1 Processor -- start 2023-11-24T07:56:56.263 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:56.261+0000 7f6ecc9db700 1 -- start start 2023-11-24T07:56:56.265 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:56.262+0000 7f6ecc9db700 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f6ec806b620 0x7f6ec80b3a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:56:56.265 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:56.262+0000 7f6ecc9db700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f6ec806ee60 0x7f6ec80b3f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:56:56.266 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:56.262+0000 7f6ecc9db700 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6ec8071f50 con 0x7f6ec80b5ab0 2023-11-24T07:56:56.266 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:56.262+0000 7f6ecc9db700 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6ec8072230 con 0x7f6ec806ee60 2023-11-24T07:56:56.266 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:56.262+0000 7f6ecc9db700 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6ec8072510 con 0x7f6ec806b620 2023-11-24T07:56:56.266 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:56.262+0000 7f6ec6d9d700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f6ec806ee60 0x7f6ec80b3f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:56:56.266 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:56.262+0000 7f6ec6d9d700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f6ec806ee60 0x7f6ec80b3f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:50728/0 (socket says 172.21.15.59:50728) 2023-11-24T07:56:56.266 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:56.262+0000 7f6ec6d9d700 1 -- 172.21.15.59:0/1072524424 learned_addr learned my addr 172.21.15.59:0/1072524424 (peer_addr_for_me v2:172.21.15.59:0/0) 2023-11-24T07:56:56.266 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:56.262+0000 7f6ec659c700 1 --2- 172.21.15.59:0/1072524424 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f6ec806b620 0x7f6ec80b3a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:56:56.266 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:56.262+0000 7f6ec6d9d700 1 -- 172.21.15.59:0/1072524424 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f6ec806b620 msgr2=0x7f6ec80b3a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T07:56:56.266 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:56.262+0000 7f6ec6d9d700 1 --2- 172.21.15.59:0/1072524424 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f6ec806b620 0x7f6ec80b3a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:56:56.267 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:56.262+0000 7f6ec6d9d700 1 -- 172.21.15.59:0/1072524424 >> v1:172.21.15.76:6789/0 conn(0x7f6ec80b5ab0 legacy=0x7f6ec8161ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T07:56:56.267 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:56.262+0000 7f6ec6d9d700 1 -- 172.21.15.59:0/1072524424 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6ec80727f0 con 0x7f6ec806ee60 2023-11-24T07:56:56.267 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:56.263+0000 7f6ec6d9d700 1 --2- 172.21.15.59:0/1072524424 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6ec806ee60 0x7f6ec80b3f80 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f6ec0009e50 tx=0x7f6ec000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aa106ad08f6a4d67 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:56:56.267 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:56.263+0000 7f6ec559a700 1 -- 172.21.15.59:0/1072524424 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ec000e070 con 0x7f6ec806ee60 2023-11-24T07:56:56.267 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:56.263+0000 7f6ec559a700 1 -- 172.21.15.59:0/1072524424 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6ec0003a20 con 0x7f6ec806ee60 2023-11-24T07:56:56.267 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:56.263+0000 7f6ec559a700 1 -- 172.21.15.59:0/1072524424 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ec0003dd0 con 0x7f6ec806ee60 2023-11-24T07:56:56.267 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:56.263+0000 7f6ecc9db700 1 -- 172.21.15.59:0/1072524424 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6ec806ee60 msgr2=0x7f6ec80b3f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:56:56.267 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:56.263+0000 7f6ecc9db700 1 --2- 172.21.15.59:0/1072524424 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6ec806ee60 0x7f6ec80b3f80 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f6ec0009e50 tx=0x7f6ec000b330 comp rx=0 tx=0).stop 2023-11-24T07:56:56.267 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:56.263+0000 7f6ecc9db700 1 -- 172.21.15.59:0/1072524424 shutdown_connections 2023-11-24T07:56:56.267 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:56.263+0000 7f6ecc9db700 1 --2- 172.21.15.59:0/1072524424 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f6ec806b620 0x7f6ec80b3a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:56:56.268 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:56.263+0000 7f6ecc9db700 1 --2- 172.21.15.59:0/1072524424 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6ec806ee60 0x7f6ec80b3f80 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:56:56.268 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:56.263+0000 7f6ecc9db700 1 -- 172.21.15.59:0/1072524424 >> 172.21.15.59:0/1072524424 conn(0x7f6ec80b2740 msgr2=0x7f6ec80b2b20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:56:56.268 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:56.263+0000 7f6ecc9db700 1 -- 172.21.15.59:0/1072524424 shutdown_connections 2023-11-24T07:56:56.268 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:56.263+0000 7f6ecc9db700 1 -- 172.21.15.59:0/1072524424 wait complete. 2023-11-24T07:56:56.270 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:56.265+0000 7f6ecc9db700 1 Processor -- start 2023-11-24T07:56:56.270 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:56.265+0000 7f6ecc9db700 1 -- start start 2023-11-24T07:56:56.271 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:56.265+0000 7f6ecc9db700 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f6ec806b620 0x7f6ec81df7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:56:56.271 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:56.265+0000 7f6ecc9db700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6ec806ee60 0x7f6ec81e47f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:56:56.271 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:56.266+0000 7f6ecc9db700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6ec80b5ab0 0x7f6ec8134c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:56:56.271 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:56.266+0000 7f6ecc9db700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ec8133b60 con 0x7f6ec806ee60 2023-11-24T07:56:56.271 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:56.266+0000 7f6ecc9db700 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ec8133e40 con 0x7f6ec80b5ab0 2023-11-24T07:56:56.271 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:56.266+0000 7f6ecc9db700 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6ec8134120 con 0x7f6ec806b620 2023-11-24T07:56:56.271 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:56.266+0000 7f6ec5d9b700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6ec806ee60 0x7f6ec81e47f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:56:56.271 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:56.266+0000 7f6ec5d9b700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6ec806ee60 0x7f6ec81e47f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:50732/0 (socket says 172.21.15.59:50732) 2023-11-24T07:56:56.271 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:56.266+0000 7f6ec5d9b700 1 -- 172.21.15.59:0/3108164294 learned_addr learned my addr 172.21.15.59:0/3108164294 (peer_addr_for_me v2:172.21.15.59:0/0) 2023-11-24T07:56:56.272 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:56.266+0000 7f6ec659c700 1 --2- 172.21.15.59:0/3108164294 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f6ec806b620 0x7f6ec81df7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:56:56.272 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:56.266+0000 7f6ec6d9d700 1 --2- 172.21.15.59:0/3108164294 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6ec80b5ab0 0x7f6ec8134c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:56:56.272 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:56.266+0000 7f6ec5d9b700 1 -- 172.21.15.59:0/3108164294 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f6ec806b620 msgr2=0x7f6ec81df7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:56:56.272 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:56.266+0000 7f6ec5d9b700 1 --2- 172.21.15.59:0/3108164294 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f6ec806b620 0x7f6ec81df7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:56:56.272 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:56.266+0000 7f6ec5d9b700 1 -- 172.21.15.59:0/3108164294 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6ec80b5ab0 msgr2=0x7f6ec8134c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:56:56.272 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:56.266+0000 7f6ec5d9b700 1 --2- 172.21.15.59:0/3108164294 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6ec80b5ab0 0x7f6ec8134c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:56:56.272 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:56.266+0000 7f6ec5d9b700 1 -- 172.21.15.59:0/3108164294 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6ec0009b30 con 0x7f6ec806ee60 2023-11-24T07:56:56.272 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:56.266+0000 7f6ec5d9b700 1 --2- 172.21.15.59:0/3108164294 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6ec806ee60 0x7f6ec81e47f0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f6ebc00b450 tx=0x7f6ebc00b760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:56:56.272 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:56.266+0000 7f6eb6ffd700 1 -- 172.21.15.59:0/3108164294 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ebc011830 con 0x7f6ec806ee60 2023-11-24T07:56:56.272 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:56.266+0000 7f6eb6ffd700 1 -- 172.21.15.59:0/3108164294 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6ebc011990 con 0x7f6ec806ee60 2023-11-24T07:56:56.272 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:56.266+0000 7f6ecc9db700 1 -- 172.21.15.59:0/3108164294 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6ec8134400 con 0x7f6ec806ee60 2023-11-24T07:56:56.273 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:56.266+0000 7f6eb6ffd700 1 -- 172.21.15.59:0/3108164294 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ebc011d40 con 0x7f6ec806ee60 2023-11-24T07:56:56.273 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:56.266+0000 7f6ecc9db700 1 -- 172.21.15.59:0/3108164294 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6ec81dfd20 con 0x7f6ec806ee60 2023-11-24T07:56:56.273 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:56.267+0000 7f6ecc9db700 1 -- 172.21.15.59:0/3108164294 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6ec8075480 con 0x7f6ec806ee60 2023-11-24T07:56:56.273 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:56.268+0000 7f6eb6ffd700 1 -- 172.21.15.59:0/3108164294 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f6ebc01a400 con 0x7f6ec806ee60 2023-11-24T07:56:56.273 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:56.268+0000 7f6eb6ffd700 1 --2- 172.21.15.59:0/3108164294 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f6ea806aed0 0x7f6ea806d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:56:56.273 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:56.268+0000 7f6eb6ffd700 1 -- 172.21.15.59:0/3108164294 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f6ebc089150 con 0x7f6ec806ee60 2023-11-24T07:56:56.273 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:56.268+0000 7f6ec659c700 1 --2- 172.21.15.59:0/3108164294 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f6ea806aed0 0x7f6ea806d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:56:56.273 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:56.269+0000 7f6ec659c700 1 --2- 172.21.15.59:0/3108164294 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f6ea806aed0 0x7f6ea806d390 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f6eb8006b20 tx=0x7f6eb8008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:56:56.274 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:56.273+0000 7f6eb6ffd700 1 -- 172.21.15.59:0/3108164294 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f6ebc054200 con 0x7f6ec806ee60 2023-11-24T07:56:56.475 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:56.474+0000 7f6ecc9db700 1 -- 172.21.15.59:0/3108164294 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f6ec806bd40 con 0x7f6ec806ee60 2023-11-24T07:56:56.476 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:56.474+0000 7f6eb6ffd700 1 -- 172.21.15.59:0/3108164294 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v20) v1 ==== 93+0+10 (secure 0 0 0) 0x7f6ebc00be60 con 0x7f6ec806ee60 2023-11-24T07:56:56.476 INFO:teuthology.orchestra.run.smithi059.stdout:pg_num: 1 2023-11-24T07:56:56.478 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:56.476+0000 7f6ecc9db700 1 -- 172.21.15.59:0/3108164294 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f6ea806aed0 msgr2=0x7f6ea806d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:56:56.478 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:56.476+0000 7f6ecc9db700 1 --2- 172.21.15.59:0/3108164294 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f6ea806aed0 0x7f6ea806d390 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f6eb8006b20 tx=0x7f6eb8008040 comp rx=0 tx=0).stop 2023-11-24T07:56:56.478 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:56.476+0000 7f6ecc9db700 1 -- 172.21.15.59:0/3108164294 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6ec806ee60 msgr2=0x7f6ec81e47f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:56:56.478 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:56.476+0000 7f6ecc9db700 1 --2- 172.21.15.59:0/3108164294 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6ec806ee60 0x7f6ec81e47f0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f6ebc00b450 tx=0x7f6ebc00b760 comp rx=0 tx=0).stop 2023-11-24T07:56:56.479 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:56.476+0000 7f6ecc9db700 1 -- 172.21.15.59:0/3108164294 shutdown_connections 2023-11-24T07:56:56.479 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:56.476+0000 7f6ecc9db700 1 --2- 172.21.15.59:0/3108164294 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f6ea806aed0 0x7f6ea806d390 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:56:56.479 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:56.476+0000 7f6ecc9db700 1 --2- 172.21.15.59:0/3108164294 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6ec80b5ab0 0x7f6ec8134c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:56:56.479 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:56.476+0000 7f6ecc9db700 1 --2- 172.21.15.59:0/3108164294 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f6ec806b620 0x7f6ec81df7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:56:56.479 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:56.476+0000 7f6ecc9db700 1 --2- 172.21.15.59:0/3108164294 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6ec806ee60 0x7f6ec81e47f0 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:56:56.479 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:56.476+0000 7f6ecc9db700 1 -- 172.21.15.59:0/3108164294 >> 172.21.15.59:0/3108164294 conn(0x7f6ec80b2740 msgr2=0x7f6ec81471a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:56:56.479 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:56.476+0000 7f6ecc9db700 1 -- 172.21.15.59:0/3108164294 shutdown_connections 2023-11-24T07:56:56.479 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:56.476+0000 7f6ecc9db700 1 -- 172.21.15.59:0/3108164294 wait complete. 2023-11-24T07:56:56.490 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-24T07:56:56.619 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:56.617+0000 7f2f79a5f700 1 Processor -- start 2023-11-24T07:56:56.619 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:56.617+0000 7f2f79a5f700 1 -- start start 2023-11-24T07:56:56.622 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:56.618+0000 7f2f79a5f700 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f2f740d0c50 0x7f2f740d26c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:56:56.622 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:56.618+0000 7f2f79a5f700 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f2f740d49f0 0x7f2f740b6d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:56:56.623 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:56.618+0000 7f2f79a5f700 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2f740b5370 con 0x7f2f740d0870 2023-11-24T07:56:56.623 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:56.618+0000 7f2f79a5f700 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2f740d2f80 con 0x7f2f740d0c50 2023-11-24T07:56:56.623 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:56.618+0000 7f2f79a5f700 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2f740cff70 con 0x7f2f740d49f0 2023-11-24T07:56:56.623 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:56.618+0000 7f2f72ffd700 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7f2f740d0870 0x7f2f740cf860 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:54212/0 (socket says 172.21.15.59:54212) 2023-11-24T07:56:56.623 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:56.618+0000 7f2f72ffd700 1 -- 172.21.15.59:0/2013119469 learned_addr learned my addr 172.21.15.59:0/2013119469 (peer_addr_for_me v1:172.21.15.59:0/0) 2023-11-24T07:56:56.623 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:56.618+0000 7f2f71ffb700 1 -- 172.21.15.59:0/2013119469 <== mon.0 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3329016348 0 0) 0x7f2f740b5370 con 0x7f2f740d0870 2023-11-24T07:56:56.623 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:56.618+0000 7f2f727fc700 1 --2- 172.21.15.59:0/2013119469 >> v2:172.21.15.76:3300/0 conn(0x7f2f740d0c50 0x7f2f740d26c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:56:56.623 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:56.618+0000 7f2f71ffb700 1 -- 172.21.15.59:0/2013119469 --> v1:172.21.15.59:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2f5c0035b0 con 0x7f2f740d0870 2023-11-24T07:56:56.623 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:56.618+0000 7f2f737fe700 1 --2- 172.21.15.59:0/2013119469 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f2f740d49f0 0x7f2f740b6d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:56:56.623 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:56.619+0000 7f2f71ffb700 1 -- 172.21.15.59:0/2013119469 <== mon.0 v1:172.21.15.59:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4150845621 0 0) 0x7f2f5c0035b0 con 0x7f2f740d0870 2023-11-24T07:56:56.623 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:56.619+0000 7f2f71ffb700 1 -- 172.21.15.59:0/2013119469 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f2f740d49f0 msgr2=0x7f2f740b6d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T07:56:56.624 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:56.619+0000 7f2f71ffb700 1 --2- 172.21.15.59:0/2013119469 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f2f740d49f0 0x7f2f740b6d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:56:56.624 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:56.619+0000 7f2f71ffb700 1 -- 172.21.15.59:0/2013119469 >> v2:172.21.15.76:3300/0 conn(0x7f2f740d0c50 msgr2=0x7f2f740d26c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T07:56:56.624 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:56.619+0000 7f2f71ffb700 1 --2- 172.21.15.59:0/2013119469 >> v2:172.21.15.76:3300/0 conn(0x7f2f740d0c50 0x7f2f740d26c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:56:56.624 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:56.619+0000 7f2f71ffb700 1 -- 172.21.15.59:0/2013119469 --> v1:172.21.15.59:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2f740ceae0 con 0x7f2f740d0870 2023-11-24T07:56:56.624 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:56.619+0000 7f2f71ffb700 1 -- 172.21.15.59:0/2013119469 <== mon.0 v1:172.21.15.59:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1312333566 0 0) 0x7f2f68001670 con 0x7f2f740d0870 2023-11-24T07:56:56.624 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:56.619+0000 7f2f71ffb700 1 -- 172.21.15.59:0/2013119469 >> v1:172.21.15.59:6789/0 conn(0x7f2f740d0870 legacy=0x7f2f740cf860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:56:56.624 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:56.619+0000 7f2f71ffb700 1 --2- 172.21.15.59:0/2013119469 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2f5c003a00 0x7f2f5c003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:56:56.624 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:56.619+0000 7f2f71ffb700 1 --2- 172.21.15.59:0/2013119469 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2f5c004850 0x7f2f5c004cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:56:56.624 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:56.619+0000 7f2f71ffb700 1 --2- 172.21.15.59:0/2013119469 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f2f740d49f0 0x7f2f5c005210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:56:56.624 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:56.619+0000 7f2f71ffb700 1 -- 172.21.15.59:0/2013119469 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f740d2f80 con 0x7f2f5c003a00 2023-11-24T07:56:56.624 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:56.619+0000 7f2f71ffb700 1 -- 172.21.15.59:0/2013119469 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f740cff70 con 0x7f2f5c004850 2023-11-24T07:56:56.624 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:56.619+0000 7f2f71ffb700 1 -- 172.21.15.59:0/2013119469 --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2f5c00ba10 con 0x7f2f740d49f0 2023-11-24T07:56:56.624 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:56.619+0000 7f2f72ffd700 1 --2- 172.21.15.59:0/2013119469 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2f5c003a00 0x7f2f5c003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:56:56.625 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:56.619+0000 7f2f727fc700 1 --2- 172.21.15.59:0/2013119469 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2f5c004850 0x7f2f5c004cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:56:56.625 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:56.619+0000 7f2f737fe700 1 --2- 172.21.15.59:0/2013119469 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f2f740d49f0 0x7f2f5c005210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:56:56.625 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:56.619+0000 7f2f72ffd700 1 -- 172.21.15.59:0/2013119469 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f2f740d49f0 msgr2=0x7f2f5c005210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T07:56:56.625 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:56.619+0000 7f2f72ffd700 1 --2- 172.21.15.59:0/2013119469 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f2f740d49f0 0x7f2f5c005210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:56:56.625 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:56.619+0000 7f2f72ffd700 1 -- 172.21.15.59:0/2013119469 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2f5c004850 msgr2=0x7f2f5c004cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T07:56:56.625 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:56.619+0000 7f2f72ffd700 1 --2- 172.21.15.59:0/2013119469 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2f5c004850 0x7f2f5c004cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:56:56.625 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:56.619+0000 7f2f72ffd700 1 -- 172.21.15.59:0/2013119469 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2f5c0035b0 con 0x7f2f5c003a00 2023-11-24T07:56:56.625 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:56.619+0000 7f2f72ffd700 1 --2- 172.21.15.59:0/2013119469 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2f5c003a00 0x7f2f5c003e20 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f2f6800c2f0 tx=0x7f2f5c005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=41cf7103acccbf9d server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:56:56.625 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:56.620+0000 7f2f71ffb700 1 -- 172.21.15.59:0/2013119469 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f6800ce30 con 0x7f2f5c003a00 2023-11-24T07:56:56.625 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:56.620+0000 7f2f71ffb700 1 -- 172.21.15.59:0/2013119469 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2f68005960 con 0x7f2f5c003a00 2023-11-24T07:56:56.625 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:56.620+0000 7f2f71ffb700 1 -- 172.21.15.59:0/2013119469 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f68005d10 con 0x7f2f5c003a00 2023-11-24T07:56:56.625 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:56.620+0000 7f2f79a5f700 1 -- 172.21.15.59:0/2013119469 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2f5c003a00 msgr2=0x7f2f5c003e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:56:56.625 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:56.620+0000 7f2f79a5f700 1 --2- 172.21.15.59:0/2013119469 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2f5c003a00 0x7f2f5c003e20 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f2f6800c2f0 tx=0x7f2f5c005880 comp rx=0 tx=0).stop 2023-11-24T07:56:56.626 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:56.620+0000 7f2f737fe700 1 -- 172.21.15.59:0/2013119469 reap_dead start 2023-11-24T07:56:56.626 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:56.620+0000 7f2f79a5f700 1 -- 172.21.15.59:0/2013119469 shutdown_connections 2023-11-24T07:56:56.626 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:56.620+0000 7f2f79a5f700 1 -- 172.21.15.59:0/2013119469 >> 172.21.15.59:0/2013119469 conn(0x7f2f740d3ed0 msgr2=0x7f2f740d42e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:56:56.626 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:56.620+0000 7f2f79a5f700 1 -- 172.21.15.59:0/2013119469 shutdown_connections 2023-11-24T07:56:56.626 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:56.620+0000 7f2f79a5f700 1 -- 172.21.15.59:0/2013119469 wait complete. 2023-11-24T07:56:56.628 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:56.621+0000 7f2f79a5f700 1 Processor -- start 2023-11-24T07:56:56.628 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:56.622+0000 7f2f79a5f700 1 -- start start 2023-11-24T07:56:56.628 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:56.622+0000 7f2f79a5f700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2f5c003a00 0x7f2f741533f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:56:56.629 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:56.622+0000 7f2f79a5f700 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f2f740cc910 0x7f2f74158f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:56:56.629 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:56.622+0000 7f2f79a5f700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2f7415a000 0x7f2f74156200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:56:56.629 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:56.622+0000 7f2f79a5f700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f740cc4f0 con 0x7f2f7415a000 2023-11-24T07:56:56.629 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:56.622+0000 7f2f79a5f700 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f74159790 con 0x7f2f5c003a00 2023-11-24T07:56:56.629 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:56.622+0000 7f2f79a5f700 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2f74153930 con 0x7f2f740cc910 2023-11-24T07:56:56.629 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:56.622+0000 7f2f737fe700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2f7415a000 0x7f2f74156200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:56:56.629 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:56.622+0000 7f2f737fe700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2f7415a000 0x7f2f74156200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:50754/0 (socket says 172.21.15.59:50754) 2023-11-24T07:56:56.629 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:56.622+0000 7f2f737fe700 1 -- 172.21.15.59:0/1029994501 learned_addr learned my addr 172.21.15.59:0/1029994501 (peer_addr_for_me v2:172.21.15.59:0/0) 2023-11-24T07:56:56.629 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:56.623+0000 7f2f727fc700 1 --2- 172.21.15.59:0/1029994501 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f2f740cc910 0x7f2f74158f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:56:56.629 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:56.623+0000 7f2f72ffd700 1 --2- 172.21.15.59:0/1029994501 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2f5c003a00 0x7f2f741533f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:56:56.629 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:56.623+0000 7f2f737fe700 1 -- 172.21.15.59:0/1029994501 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f2f740cc910 msgr2=0x7f2f74158f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:56:56.630 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:56.623+0000 7f2f737fe700 1 --2- 172.21.15.59:0/1029994501 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f2f740cc910 0x7f2f74158f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:56:56.630 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:56.623+0000 7f2f737fe700 1 -- 172.21.15.59:0/1029994501 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2f5c003a00 msgr2=0x7f2f741533f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:56:56.630 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:56.623+0000 7f2f737fe700 1 --2- 172.21.15.59:0/1029994501 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2f5c003a00 0x7f2f741533f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:56:56.630 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:56.623+0000 7f2f737fe700 1 -- 172.21.15.59:0/1029994501 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2f740cbb50 con 0x7f2f7415a000 2023-11-24T07:56:56.630 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:56.623+0000 7f2f72ffd700 1 --2- 172.21.15.59:0/1029994501 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2f5c003a00 0x7f2f741533f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-24T07:56:56.630 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:56.623+0000 7f2f737fe700 1 --2- 172.21.15.59:0/1029994501 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2f7415a000 0x7f2f74156200 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f2f74067c90 tx=0x7f2f6c00f490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:56:56.630 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:56.623+0000 7f2f5bfff700 1 -- 172.21.15.59:0/1029994501 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f6c00e040 con 0x7f2f7415a000 2023-11-24T07:56:56.630 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:56.623+0000 7f2f5bfff700 1 -- 172.21.15.59:0/1029994501 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2f6c00f040 con 0x7f2f7415a000 2023-11-24T07:56:56.630 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:56.623+0000 7f2f79a5f700 1 -- 172.21.15.59:0/1029994501 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2f740cbe30 con 0x7f2f7415a000 2023-11-24T07:56:56.630 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:56.623+0000 7f2f5bfff700 1 -- 172.21.15.59:0/1029994501 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f6c011e50 con 0x7f2f7415a000 2023-11-24T07:56:56.630 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:56.623+0000 7f2f79a5f700 1 -- 172.21.15.59:0/1029994501 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2f741594a0 con 0x7f2f7415a000 2023-11-24T07:56:56.630 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:56.624+0000 7f2f79a5f700 1 -- 172.21.15.59:0/1029994501 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2f740b57c0 con 0x7f2f7415a000 2023-11-24T07:56:56.631 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:56.625+0000 7f2f5bfff700 1 -- 172.21.15.59:0/1029994501 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f2f6c022460 con 0x7f2f7415a000 2023-11-24T07:56:56.631 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:56.625+0000 7f2f5bfff700 1 --2- 172.21.15.59:0/1029994501 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f2f6006aed0 0x7f2f6006d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:56:56.631 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:56.625+0000 7f2f5bfff700 1 -- 172.21.15.59:0/1029994501 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f2f6c089ab0 con 0x7f2f7415a000 2023-11-24T07:56:56.631 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:56.625+0000 7f2f72ffd700 1 --2- 172.21.15.59:0/1029994501 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f2f6006aed0 0x7f2f6006d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:56:56.631 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:56.626+0000 7f2f72ffd700 1 --2- 172.21.15.59:0/1029994501 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f2f6006aed0 0x7f2f6006d390 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f2f68000c00 tx=0x7f2f6800a000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:56:56.631 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:56.630+0000 7f2f5bfff700 1 -- 172.21.15.59:0/1029994501 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f2f6c054b60 con 0x7f2f7415a000 2023-11-24T07:56:56.836 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:56.835+0000 7f2f79a5f700 1 -- 172.21.15.59:0/1029994501 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f2f740cc630 con 0x7f2f7415a000 2023-11-24T07:56:56.837 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:56.835+0000 7f2f5bfff700 1 -- 172.21.15.59:0/1029994501 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v20) v1 ==== 104+0+11 (secure 0 0 0) 0x7f2f740cc630 con 0x7f2f7415a000 2023-11-24T07:56:56.837 INFO:teuthology.orchestra.run.smithi059.stdout:pg_num: 32 2023-11-24T07:56:56.839 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:56.837+0000 7f2f79a5f700 1 -- 172.21.15.59:0/1029994501 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f2f6006aed0 msgr2=0x7f2f6006d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:56:56.839 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:56.837+0000 7f2f79a5f700 1 --2- 172.21.15.59:0/1029994501 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f2f6006aed0 0x7f2f6006d390 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f2f68000c00 tx=0x7f2f6800a000 comp rx=0 tx=0).stop 2023-11-24T07:56:56.839 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:56.837+0000 7f2f79a5f700 1 -- 172.21.15.59:0/1029994501 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2f7415a000 msgr2=0x7f2f74156200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:56:56.840 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:56.837+0000 7f2f79a5f700 1 --2- 172.21.15.59:0/1029994501 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2f7415a000 0x7f2f74156200 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f2f74067c90 tx=0x7f2f6c00f490 comp rx=0 tx=0).stop 2023-11-24T07:56:56.840 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:56.837+0000 7f2f79a5f700 1 -- 172.21.15.59:0/1029994501 shutdown_connections 2023-11-24T07:56:56.840 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:56.837+0000 7f2f79a5f700 1 --2- 172.21.15.59:0/1029994501 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f2f6006aed0 0x7f2f6006d390 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:56:56.840 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:56.837+0000 7f2f79a5f700 1 --2- 172.21.15.59:0/1029994501 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2f7415a000 0x7f2f74156200 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:56:56.840 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:56.837+0000 7f2f79a5f700 1 --2- 172.21.15.59:0/1029994501 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2f5c003a00 0x7f2f741533f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:56:56.840 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:56.837+0000 7f2f79a5f700 1 --2- 172.21.15.59:0/1029994501 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f2f740cc910 0x7f2f74158f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:56:56.840 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:56.837+0000 7f2f79a5f700 1 -- 172.21.15.59:0/1029994501 >> 172.21.15.59:0/1029994501 conn(0x7f2f740d3ed0 msgr2=0x7f2f7406bd40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:56:56.840 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:56.838+0000 7f2f79a5f700 1 -- 172.21.15.59:0/1029994501 shutdown_connections 2023-11-24T07:56:56.840 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:56.838+0000 7f2f79a5f700 1 -- 172.21.15.59:0/1029994501 wait complete. 2023-11-24T07:56:56.852 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-24T07:56:56.978 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:56.977+0000 7f205c0a2700 1 Processor -- start 2023-11-24T07:56:56.979 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:56.977+0000 7f205c0a2700 1 -- start start 2023-11-24T07:56:56.981 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:56.978+0000 7f205c0a2700 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f205415e090 0x7f205406afd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:56:56.981 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:56.978+0000 7f205c0a2700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f205406b510 0x7f2054161cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:56:56.982 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:56.978+0000 7f205c0a2700 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f205415e470 con 0x7f2054162200 2023-11-24T07:56:56.982 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:56.978+0000 7f205c0a2700 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f20540c6900 con 0x7f205406b510 2023-11-24T07:56:56.982 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:56.978+0000 7f205c0a2700 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f205406b990 con 0x7f205415e090 2023-11-24T07:56:56.982 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:56.978+0000 7f205963d700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f205406b510 0x7f2054161cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:56:56.982 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:56.978+0000 7f205a63f700 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7f2054162200 0x7f2054071b40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:54236/0 (socket says 172.21.15.59:54236) 2023-11-24T07:56:56.982 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:56.978+0000 7f205a63f700 1 -- 172.21.15.59:0/4023746362 learned_addr learned my addr 172.21.15.59:0/4023746362 (peer_addr_for_me v1:172.21.15.59:0/0) 2023-11-24T07:56:56.982 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:56.978+0000 7f2059e3e700 1 --2- 172.21.15.59:0/4023746362 >> v2:172.21.15.76:3300/0 conn(0x7f205415e090 0x7f205406afd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:56:56.982 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:56.978+0000 7f205963d700 1 -- 172.21.15.59:0/4023746362 >> v2:172.21.15.76:3300/0 conn(0x7f205415e090 msgr2=0x7f205406afd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T07:56:56.982 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:56.978+0000 7f205963d700 1 --2- 172.21.15.59:0/4023746362 >> v2:172.21.15.76:3300/0 conn(0x7f205415e090 0x7f205406afd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:56:56.982 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:56.978+0000 7f205963d700 1 -- 172.21.15.59:0/4023746362 >> v1:172.21.15.59:6789/0 conn(0x7f2054162200 legacy=0x7f2054071b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:56:56.983 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:56.978+0000 7f205963d700 1 -- 172.21.15.59:0/4023746362 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f20541629f0 con 0x7f205406b510 2023-11-24T07:56:56.983 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:56.978+0000 7f2058e3c700 1 -- 172.21.15.59:0/4023746362 <== mon.0 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1677544724 0 0) 0x7f205415e470 con 0x7f2054162200 2023-11-24T07:56:56.983 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:56.978+0000 7f205963d700 1 --2- 172.21.15.59:0/4023746362 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f205406b510 0x7f2054161cc0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f2054072e00 tx=0x7f2050009b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e9e07453766c1cac server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:56:56.983 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:56.979+0000 7f2058e3c700 1 -- 172.21.15.59:0/4023746362 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2050011830 con 0x7f205406b510 2023-11-24T07:56:56.983 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:56.979+0000 7f2058e3c700 1 -- 172.21.15.59:0/4023746362 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2050011990 con 0x7f205406b510 2023-11-24T07:56:56.983 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:56.979+0000 7f2058e3c700 1 -- 172.21.15.59:0/4023746362 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2050011d20 con 0x7f205406b510 2023-11-24T07:56:56.983 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:56.979+0000 7f205c0a2700 1 -- 172.21.15.59:0/4023746362 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f205406b510 msgr2=0x7f2054161cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:56:56.984 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:56.979+0000 7f205c0a2700 1 --2- 172.21.15.59:0/4023746362 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f205406b510 0x7f2054161cc0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f2054072e00 tx=0x7f2050009b20 comp rx=0 tx=0).stop 2023-11-24T07:56:56.984 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:56.979+0000 7f205c0a2700 1 -- 172.21.15.59:0/4023746362 shutdown_connections 2023-11-24T07:56:56.984 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:56.979+0000 7f205c0a2700 1 --2- 172.21.15.59:0/4023746362 >> v2:172.21.15.76:3300/0 conn(0x7f205415e090 0x7f205406afd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:56:56.984 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:56.979+0000 7f205c0a2700 1 --2- 172.21.15.59:0/4023746362 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f205406b510 0x7f2054161cc0 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:56:56.984 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:56.979+0000 7f205c0a2700 1 -- 172.21.15.59:0/4023746362 >> 172.21.15.59:0/4023746362 conn(0x7f20540d0b10 msgr2=0x7f20540d0f20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:56:56.984 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:56.979+0000 7f205c0a2700 1 -- 172.21.15.59:0/4023746362 shutdown_connections 2023-11-24T07:56:56.984 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:56.979+0000 7f205c0a2700 1 -- 172.21.15.59:0/4023746362 wait complete. 2023-11-24T07:56:56.987 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:56.980+0000 7f205c0a2700 1 Processor -- start 2023-11-24T07:56:56.987 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:56.980+0000 7f205c0a2700 1 -- start start 2023-11-24T07:56:56.987 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:56.981+0000 7f205c0a2700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f205406b510 0x7f2054161040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:56:56.987 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:56.981+0000 7f205c0a2700 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f205415e090 0x7f2054159330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:56:56.987 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:56.981+0000 7f205c0a2700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2054162200 0x7f20540b1800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:56:56.987 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:56.981+0000 7f205c0a2700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2054158ea0 con 0x7f205406b510 2023-11-24T07:56:56.987 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:56.981+0000 7f205c0a2700 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2054161580 con 0x7f2054162200 2023-11-24T07:56:56.987 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:56.981+0000 7f205c0a2700 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f205406fae0 con 0x7f205415e090 2023-11-24T07:56:56.988 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:56.981+0000 7f2059e3e700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f205406b510 0x7f2054161040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:56:56.988 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:56.981+0000 7f2059e3e700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f205406b510 0x7f2054161040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:50764/0 (socket says 172.21.15.59:50764) 2023-11-24T07:56:56.988 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:56.981+0000 7f2059e3e700 1 -- 172.21.15.59:0/4159244936 learned_addr learned my addr 172.21.15.59:0/4159244936 (peer_addr_for_me v2:172.21.15.59:0/0) 2023-11-24T07:56:56.988 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:56.981+0000 7f2059e3e700 1 -- 172.21.15.59:0/4159244936 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f205415e090 msgr2=0x7f2054159330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:56:56.988 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:56.981+0000 7f2059e3e700 1 --2- 172.21.15.59:0/4159244936 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f205415e090 0x7f2054159330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:56:56.988 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:56.982+0000 7f2059e3e700 1 -- 172.21.15.59:0/4159244936 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2054162200 msgr2=0x7f20540b1800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:56:56.988 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:56.981+0000 7f205a63f700 1 --2- 172.21.15.59:0/4159244936 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2054162200 0x7f20540b1800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:56:56.988 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:56.982+0000 7f2059e3e700 1 --2- 172.21.15.59:0/4159244936 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2054162200 0x7f20540b1800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:56:56.988 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:56.982+0000 7f2059e3e700 1 -- 172.21.15.59:0/4159244936 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f20500094a0 con 0x7f205406b510 2023-11-24T07:56:56.988 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:56.982+0000 7f2059e3e700 1 --2- 172.21.15.59:0/4159244936 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f205406b510 0x7f2054161040 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f204400e810 tx=0x7f204400ebd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:56:56.988 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:56.982+0000 7f204a7fc700 1 -- 172.21.15.59:0/4159244936 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2044013610 con 0x7f205406b510 2023-11-24T07:56:56.988 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:56.982+0000 7f204a7fc700 1 -- 172.21.15.59:0/4159244936 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2044013770 con 0x7f205406b510 2023-11-24T07:56:56.989 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:56.982+0000 7f205c0a2700 1 -- 172.21.15.59:0/4159244936 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2054158570 con 0x7f205406b510 2023-11-24T07:56:56.989 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:56.982+0000 7f204a7fc700 1 -- 172.21.15.59:0/4159244936 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2044013b20 con 0x7f205406b510 2023-11-24T07:56:56.989 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:56.982+0000 7f205c0a2700 1 -- 172.21.15.59:0/4159244936 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2054158b30 con 0x7f205406b510 2023-11-24T07:56:56.989 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:56.983+0000 7f205c0a2700 1 -- 172.21.15.59:0/4159244936 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2054075480 con 0x7f205406b510 2023-11-24T07:56:56.989 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:56.983+0000 7f204a7fc700 1 -- 172.21.15.59:0/4159244936 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f2044013c80 con 0x7f205406b510 2023-11-24T07:56:56.989 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:56.984+0000 7f204a7fc700 1 --2- 172.21.15.59:0/4159244936 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f203806aed0 0x7f203806d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:56:56.989 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:56.984+0000 7f204a7fc700 1 -- 172.21.15.59:0/4159244936 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f2044089320 con 0x7f205406b510 2023-11-24T07:56:56.989 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:56.984+0000 7f205963d700 1 --2- 172.21.15.59:0/4159244936 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f203806aed0 0x7f203806d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:56:56.989 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:56.984+0000 7f205963d700 1 --2- 172.21.15.59:0/4159244936 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f203806aed0 0x7f203806d390 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f2054148450 tx=0x7f2050010000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:56:56.990 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:56.988+0000 7f204a7fc700 1 -- 172.21.15.59:0/4159244936 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f2044054340 con 0x7f205406b510 2023-11-24T07:56:57.188 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:57.187+0000 7f205c0a2700 1 -- 172.21.15.59:0/4159244936 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f2054159180 con 0x7f205406b510 2023-11-24T07:56:57.189 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:57.187+0000 7f204a7fc700 1 -- 172.21.15.59:0/4159244936 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v20) v1 ==== 100+0+11 (secure 0 0 0) 0x7f2044057960 con 0x7f205406b510 2023-11-24T07:56:57.189 INFO:teuthology.orchestra.run.smithi059.stdout:pg_num: 64 2023-11-24T07:56:57.191 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:57.189+0000 7f205c0a2700 1 -- 172.21.15.59:0/4159244936 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f203806aed0 msgr2=0x7f203806d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:56:57.192 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:57.189+0000 7f205c0a2700 1 --2- 172.21.15.59:0/4159244936 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f203806aed0 0x7f203806d390 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f2054148450 tx=0x7f2050010000 comp rx=0 tx=0).stop 2023-11-24T07:56:57.192 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:57.189+0000 7f205c0a2700 1 -- 172.21.15.59:0/4159244936 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f205406b510 msgr2=0x7f2054161040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:56:57.192 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:57.189+0000 7f205c0a2700 1 --2- 172.21.15.59:0/4159244936 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f205406b510 0x7f2054161040 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f204400e810 tx=0x7f204400ebd0 comp rx=0 tx=0).stop 2023-11-24T07:56:57.192 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:57.190+0000 7f205c0a2700 1 -- 172.21.15.59:0/4159244936 shutdown_connections 2023-11-24T07:56:57.192 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:57.190+0000 7f205c0a2700 1 --2- 172.21.15.59:0/4159244936 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f203806aed0 0x7f203806d390 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:56:57.192 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:57.190+0000 7f205c0a2700 1 --2- 172.21.15.59:0/4159244936 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2054162200 0x7f20540b1800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:56:57.192 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:57.190+0000 7f205c0a2700 1 --2- 172.21.15.59:0/4159244936 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f205406b510 0x7f2054161040 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:56:57.192 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:57.190+0000 7f205c0a2700 1 --2- 172.21.15.59:0/4159244936 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f205415e090 0x7f2054159330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:56:57.192 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:57.190+0000 7f205c0a2700 1 -- 172.21.15.59:0/4159244936 >> 172.21.15.59:0/4159244936 conn(0x7f20540d0b10 msgr2=0x7f20540ada10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:56:57.192 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:57.190+0000 7f205c0a2700 1 -- 172.21.15.59:0/4159244936 shutdown_connections 2023-11-24T07:56:57.192 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:57.190+0000 7f205c0a2700 1 -- 172.21.15.59:0/4159244936 wait complete. 2023-11-24T07:56:57.203 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-24T07:56:57.328 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:57.327+0000 7fbc27a27700 1 Processor -- start 2023-11-24T07:56:57.328 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:57.327+0000 7fbc27a27700 1 -- start start 2023-11-24T07:56:57.331 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:57.327+0000 7fbc27a27700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fbc20143ae0 0x7fbc200b51b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:56:57.331 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:57.328+0000 7fbc27a27700 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fbc201440f0 0x7fbc20071bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:56:57.331 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:57.328+0000 7fbc27a27700 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbc20143ec0 con 0x7fbc200720f0 2023-11-24T07:56:57.332 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:57.328+0000 7fbc27a27700 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbc200b40a0 con 0x7fbc20143ae0 2023-11-24T07:56:57.332 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:57.328+0000 7fbc27a27700 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbc20072560 con 0x7fbc201440f0 2023-11-24T07:56:57.332 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:57.328+0000 7fbc257c3700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fbc20143ae0 0x7fbc200b51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:56:57.332 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:57.328+0000 7fbc257c3700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fbc20143ae0 0x7fbc200b51b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:50774/0 (socket says 172.21.15.59:50774) 2023-11-24T07:56:57.332 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:57.328+0000 7fbc257c3700 1 -- 172.21.15.59:0/350132498 learned_addr learned my addr 172.21.15.59:0/350132498 (peer_addr_for_me v2:172.21.15.59:0/0) 2023-11-24T07:56:57.332 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:57.328+0000 7fbc24fc2700 1 --2- 172.21.15.59:0/350132498 >> v2:172.21.15.76:3300/0 conn(0x7fbc201440f0 0x7fbc20071bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:56:57.332 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:57.328+0000 7fbc257c3700 1 -- 172.21.15.59:0/350132498 >> v2:172.21.15.76:3300/0 conn(0x7fbc201440f0 msgr2=0x7fbc20071bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T07:56:57.332 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:57.328+0000 7fbc257c3700 1 --2- 172.21.15.59:0/350132498 >> v2:172.21.15.76:3300/0 conn(0x7fbc201440f0 0x7fbc20071bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:56:57.332 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:57.328+0000 7fbc257c3700 1 -- 172.21.15.59:0/350132498 >> v1:172.21.15.76:6789/0 conn(0x7fbc200720f0 legacy=0x7fbc2015e8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:56:57.332 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:57.328+0000 7fbc257c3700 1 -- 172.21.15.59:0/350132498 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbc200b3ae0 con 0x7fbc20143ae0 2023-11-24T07:56:57.332 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:57.328+0000 7fbc257c3700 1 --2- 172.21.15.59:0/350132498 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbc20143ae0 0x7fbc200b51b0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fbc200d1b00 tx=0x7fbc1c009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aecfd81fe3af99c1 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:56:57.332 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:57.328+0000 7fbc13fff700 1 -- 172.21.15.59:0/350132498 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc1c011830 con 0x7fbc20143ae0 2023-11-24T07:56:57.333 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:57.328+0000 7fbc13fff700 1 -- 172.21.15.59:0/350132498 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbc1c011990 con 0x7fbc20143ae0 2023-11-24T07:56:57.333 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:57.328+0000 7fbc13fff700 1 -- 172.21.15.59:0/350132498 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc1c011d20 con 0x7fbc20143ae0 2023-11-24T07:56:57.333 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:57.329+0000 7fbc27a27700 1 -- 172.21.15.59:0/350132498 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbc20143ae0 msgr2=0x7fbc200b51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:56:57.333 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:57.329+0000 7fbc27a27700 1 --2- 172.21.15.59:0/350132498 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbc20143ae0 0x7fbc200b51b0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fbc200d1b00 tx=0x7fbc1c009c00 comp rx=0 tx=0).stop 2023-11-24T07:56:57.333 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:57.329+0000 7fbc27a27700 1 -- 172.21.15.59:0/350132498 shutdown_connections 2023-11-24T07:56:57.333 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:57.329+0000 7fbc27a27700 1 --2- 172.21.15.59:0/350132498 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbc20143ae0 0x7fbc200b51b0 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:56:57.333 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:57.329+0000 7fbc27a27700 1 --2- 172.21.15.59:0/350132498 >> v2:172.21.15.76:3300/0 conn(0x7fbc201440f0 0x7fbc20071bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:56:57.333 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:57.329+0000 7fbc27a27700 1 -- 172.21.15.59:0/350132498 >> 172.21.15.59:0/350132498 conn(0x7fbc200b2740 msgr2=0x7fbc200b2b20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:56:57.333 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:57.329+0000 7fbc27a27700 1 -- 172.21.15.59:0/350132498 shutdown_connections 2023-11-24T07:56:57.334 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:57.329+0000 7fbc27a27700 1 -- 172.21.15.59:0/350132498 wait complete. 2023-11-24T07:56:57.336 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:57.330+0000 7fbc27a27700 1 Processor -- start 2023-11-24T07:56:57.336 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:57.330+0000 7fbc27a27700 1 -- start start 2023-11-24T07:56:57.336 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:57.331+0000 7fbc27a27700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbc200720f0 0x7fbc20167230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:56:57.336 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:57.331+0000 7fbc27a27700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fbc20143ae0 0x7fbc20159f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:56:57.337 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:57.331+0000 7fbc27a27700 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fbc201440f0 0x7fbc2015a450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:56:57.337 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:57.331+0000 7fbc27a27700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc2013b0d0 con 0x7fbc200720f0 2023-11-24T07:56:57.337 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:57.331+0000 7fbc27a27700 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc2013b3b0 con 0x7fbc20143ae0 2023-11-24T07:56:57.337 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:57.331+0000 7fbc27a27700 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbc2013b690 con 0x7fbc201440f0 2023-11-24T07:56:57.337 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:57.331+0000 7fbc257c3700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbc200720f0 0x7fbc20167230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:56:57.337 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:57.331+0000 7fbc257c3700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbc200720f0 0x7fbc20167230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:50778/0 (socket says 172.21.15.59:50778) 2023-11-24T07:56:57.337 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:57.331+0000 7fbc257c3700 1 -- 172.21.15.59:0/347449815 learned_addr learned my addr 172.21.15.59:0/347449815 (peer_addr_for_me v2:172.21.15.59:0/0) 2023-11-24T07:56:57.337 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:57.331+0000 7fbc24fc2700 1 --2- 172.21.15.59:0/347449815 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fbc20143ae0 0x7fbc20159f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:56:57.337 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:57.331+0000 7fbc25fc4700 1 --2- 172.21.15.59:0/347449815 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fbc201440f0 0x7fbc2015a450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:56:57.337 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:57.331+0000 7fbc257c3700 1 -- 172.21.15.59:0/347449815 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fbc201440f0 msgr2=0x7fbc2015a450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:56:57.337 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:57.331+0000 7fbc257c3700 1 --2- 172.21.15.59:0/347449815 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fbc201440f0 0x7fbc2015a450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:56:57.338 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:57.331+0000 7fbc257c3700 1 -- 172.21.15.59:0/347449815 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fbc20143ae0 msgr2=0x7fbc20159f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:56:57.338 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:57.331+0000 7fbc257c3700 1 --2- 172.21.15.59:0/347449815 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fbc20143ae0 0x7fbc20159f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:56:57.338 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:57.331+0000 7fbc257c3700 1 -- 172.21.15.59:0/347449815 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbc1c009550 con 0x7fbc200720f0 2023-11-24T07:56:57.338 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:57.331+0000 7fbc25fc4700 1 --2- 172.21.15.59:0/347449815 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fbc201440f0 0x7fbc2015a450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T07:56:57.338 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:57.332+0000 7fbc257c3700 1 --2- 172.21.15.59:0/347449815 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbc200720f0 0x7fbc20167230 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fbc1c009c40 tx=0x7fbc1c004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:56:57.338 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:57.332+0000 7fbc11ffb700 1 -- 172.21.15.59:0/347449815 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc1c0036a0 con 0x7fbc200720f0 2023-11-24T07:56:57.338 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:57.332+0000 7fbc11ffb700 1 -- 172.21.15.59:0/347449815 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbc1c01e070 con 0x7fbc200720f0 2023-11-24T07:56:57.338 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:57.332+0000 7fbc27a27700 1 -- 172.21.15.59:0/347449815 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbc20152670 con 0x7fbc200720f0 2023-11-24T07:56:57.338 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:57.332+0000 7fbc11ffb700 1 -- 172.21.15.59:0/347449815 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc1c003a00 con 0x7fbc200720f0 2023-11-24T07:56:57.338 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:57.332+0000 7fbc27a27700 1 -- 172.21.15.59:0/347449815 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbc2013a130 con 0x7fbc200720f0 2023-11-24T07:56:57.339 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:57.333+0000 7fbc27a27700 1 -- 172.21.15.59:0/347449815 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbc20068670 con 0x7fbc200720f0 2023-11-24T07:56:57.339 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:57.334+0000 7fbc11ffb700 1 -- 172.21.15.59:0/347449815 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fbc1c003b60 con 0x7fbc200720f0 2023-11-24T07:56:57.339 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:57.334+0000 7fbc11ffb700 1 --2- 172.21.15.59:0/347449815 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7fbc0406aed0 0x7fbc0406d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:56:57.339 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:57.334+0000 7fbc11ffb700 1 -- 172.21.15.59:0/347449815 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fbc1c08a980 con 0x7fbc200720f0 2023-11-24T07:56:57.339 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:57.335+0000 7fbc24fc2700 1 --2- 172.21.15.59:0/347449815 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7fbc0406aed0 0x7fbc0406d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:56:57.339 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:57.335+0000 7fbc24fc2700 1 --2- 172.21.15.59:0/347449815 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7fbc0406aed0 0x7fbc0406d390 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fbc201f7610 tx=0x7fbc14008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:56:57.339 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:57.338+0000 7fbc11ffb700 1 -- 172.21.15.59:0/347449815 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fbc1c055aa0 con 0x7fbc200720f0 2023-11-24T07:56:57.542 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:57.540+0000 7fbc27a27700 1 -- 172.21.15.59:0/347449815 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fbc200b5d20 con 0x7fbc200720f0 2023-11-24T07:56:57.543 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:57.541+0000 7fbc11ffb700 1 -- 172.21.15.59:0/347449815 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) v1 ==== 74+0+16437 (secure 0 0 0) 0x7fbc1c006d80 con 0x7fbc200720f0 2023-11-24T07:56:57.543 INFO:teuthology.orchestra.run.smithi059.stdout: 2023-11-24T07:56:57.543 INFO:teuthology.orchestra.run.smithi059.stdout:{"epoch":20,"fsid":"9f0e8741-b8f9-48d5-a2c3-09f45903303d","created":"2023-11-24T07:56:06.552729+0000","modified":"2023-11-24T07:56:25.705091+0000","last_up_change":"2023-11-24T07:56:17.372352+0000","last_in_change":"2023-11-24T07:56:09.073250+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-24T07:56:18.352823+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-11-24T07:56:22.185140+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-11-24T07:56:23.084326+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"6d8f9e9b-1782-4e41-928b-a39f2a8ebc06","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6800","nonce":3373878772},{"type":"v1","addr":"172.21.15.59:6801","nonce":3373878772}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6802","nonce":3373878772},{"type":"v1","addr":"172.21.15.59:6803","nonce":3373878772}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6806","nonce":3373878772},{"type":"v1","addr":"172.21.15.59:6807","nonce":3373878772}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6804","nonce":3373878772},{"type":"v1","addr":"172.21.15.59:6805","nonce":3373878772}]},"public_addr":"172.21.15.59:6801/3373878772","cluster_addr":"172.21.15.59:6803/3373878772","heartbeat_back_addr":"172.21.15.59:6807/3373878772","heartbeat_front_addr":"172.21.15.59:6805/3373878772","state":["exists","up"]},{"osd":1,"uuid":"e8d882aa-52e1-4c04-9e1e-c44ef9204a5f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6824","nonce":648258563},{"type":"v1","addr":"172.21.15.59:6825","nonce":648258563}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6826","nonce":648258563},{"type":"v1","addr":"172.21.15.59:6827","nonce":648258563}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6830","nonce":648258563},{"type":"v1","addr":"172.21.15.59:6831","nonce":648258563}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6828","nonce":648258563},{"type":"v1","addr":"172.21.15.59:6829","nonce":648258563}]},"public_addr":"172.21.15.59:6825/648258563","cluster_addr":"172.21.15.59:6827/648258563","heartbeat_back_addr":"172.21.15.59:6831/648258563","heartbeat_front_addr":"172.21.15.59:6829/648258563","state":["exists","up"]},{"osd":2,"uuid":"69eebfa0-695b-4b21-90b9-8a3537168a01","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6808","nonce":1840430654},{"type":"v1","addr":"172.21.15.59:6809","nonce":1840430654}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6810","nonce":1840430654},{"type":"v1","addr":"172.21.15.59:6811","nonce":1840430654}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6814","nonce":1840430654},{"type":"v1","addr":"172.21.15.59:6815","nonce":1840430654}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6812","nonce":1840430654},{"type":"v1","addr":"172.21.15.59:6813","nonce":1840430654}]},"public_addr":"172.21.15.59:6809/1840430654","cluster_addr":"172.21.15.59:6811/1840430654","heartbeat_back_addr":"172.21.15.59:6815/1840430654","heartbeat_front_addr":"172.21.15.59:6813/1840430654","state":["exists","up"]},{"osd":3,"uuid":"96fd77fc-f911-46ff-9445-6f0be2daa8f4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6816","nonce":2504025850},{"type":"v1","addr":"172.21.15.59:6817","nonce":2504025850}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6818","nonce":2504025850},{"type":"v1","addr":"172.21.15.59:6819","nonce":2504025850}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6822","nonce":2504025850},{"type":"v1","addr":"172.21.15.59:6823","nonce":2504025850}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6820","nonce":2504025850},{"type":"v1","addr":"172.21.15.59:6821","nonce":2504025850}]},"public_addr":"172.21.15.59:6817/2504025850","cluster_addr":"172.21.15.59:6819/2504025850","heartbeat_back_addr":"172.21.15.59:6823/2504025850","heartbeat_front_addr":"172.21.15.59:6821/2504025850","state":["exists","up"]},{"osd":4,"uuid":"7ac29fca-b5b2-4c95-9d69-2f164cbbc708","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6816","nonce":2382463431},{"type":"v1","addr":"172.21.15.76:6818","nonce":2382463431}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6820","nonce":2382463431},{"type":"v1","addr":"172.21.15.76:6822","nonce":2382463431}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6828","nonce":2382463431},{"type":"v1","addr":"172.21.15.76:6830","nonce":2382463431}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6824","nonce":2382463431},{"type":"v1","addr":"172.21.15.76:6826","nonce":2382463431}]},"public_addr":"172.21.15.76:6818/2382463431","cluster_addr":"172.21.15.76:6822/2382463431","heartbeat_back_addr":"172.21.15.76:6830/2382463431","heartbeat_front_addr":"172.21.15.76:6826/2382463431","state":["exists","up"]},{"osd":5,"uuid":"485bf711-3851-4238-a4f4-51d52e0a9c58","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6800","nonce":329334308},{"type":"v1","addr":"172.21.15.76:6802","nonce":329334308}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6805","nonce":329334308},{"type":"v1","addr":"172.21.15.76:6807","nonce":329334308}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6814","nonce":329334308},{"type":"v1","addr":"172.21.15.76:6815","nonce":329334308}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6810","nonce":329334308},{"type":"v1","addr":"172.21.15.76:6812","nonce":329334308}]},"public_addr":"172.21.15.76:6802/329334308","cluster_addr":"172.21.15.76:6807/329334308","heartbeat_back_addr":"172.21.15.76:6815/329334308","heartbeat_front_addr":"172.21.15.76:6812/329334308","state":["exists","up"]},{"osd":6,"uuid":"72b3f2fc-1e4a-4e32-968d-003647a0286f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6801","nonce":3946579410},{"type":"v1","addr":"172.21.15.76:6803","nonce":3946579410}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6804","nonce":3946579410},{"type":"v1","addr":"172.21.15.76:6806","nonce":3946579410}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6811","nonce":3946579410},{"type":"v1","addr":"172.21.15.76:6813","nonce":3946579410}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6808","nonce":3946579410},{"type":"v1","addr":"172.21.15.76:6809","nonce":3946579410}]},"public_addr":"172.21.15.76:6803/3946579410","cluster_addr":"172.21.15.76:6806/3946579410","heartbeat_back_addr":"172.21.15.76:6813/3946579410","heartbeat_front_addr":"172.21.15.76:6809/3946579410","state":["exists","up"]},{"osd":7,"uuid":"1ddceb90-6c5d-4d1a-ad56-99cb877f8c70","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6817","nonce":841761538},{"type":"v1","addr":"172.21.15.76:6819","nonce":841761538}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6821","nonce":841761538},{"type":"v1","addr":"172.21.15.76:6823","nonce":841761538}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6829","nonce":841761538},{"type":"v1","addr":"172.21.15.76:6831","nonce":841761538}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6825","nonce":841761538},{"type":"v1","addr":"172.21.15.76:6827","nonce":841761538}]},"public_addr":"172.21.15.76:6819/841761538","cluster_addr":"172.21.15.76:6823/841761538","heartbeat_back_addr":"172.21.15.76:6831/841761538","heartbeat_front_addr":"172.21.15.76:6827/841761538","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T07:56:14.294838+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T07:56:14.270779+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T07:56:14.231809+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T07:56:14.260983+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T07:56:14.302997+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T07:56:14.258600+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T07:56:14.251413+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T07:56:14.304727+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-24T07:56:57.546 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:57.544+0000 7fbc27a27700 1 -- 172.21.15.59:0/347449815 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7fbc0406aed0 msgr2=0x7fbc0406d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:56:57.546 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:57.544+0000 7fbc27a27700 1 --2- 172.21.15.59:0/347449815 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7fbc0406aed0 0x7fbc0406d390 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fbc201f7610 tx=0x7fbc14008040 comp rx=0 tx=0).stop 2023-11-24T07:56:57.546 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:57.544+0000 7fbc27a27700 1 -- 172.21.15.59:0/347449815 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbc200720f0 msgr2=0x7fbc20167230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:56:57.546 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:57.544+0000 7fbc27a27700 1 --2- 172.21.15.59:0/347449815 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbc200720f0 0x7fbc20167230 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fbc1c009c40 tx=0x7fbc1c004750 comp rx=0 tx=0).stop 2023-11-24T07:56:57.546 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:57.544+0000 7fbc27a27700 1 -- 172.21.15.59:0/347449815 shutdown_connections 2023-11-24T07:56:57.546 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:57.544+0000 7fbc27a27700 1 --2- 172.21.15.59:0/347449815 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7fbc0406aed0 0x7fbc0406d390 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:56:57.546 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:57.544+0000 7fbc27a27700 1 --2- 172.21.15.59:0/347449815 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fbc201440f0 0x7fbc2015a450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:56:57.546 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:57.544+0000 7fbc27a27700 1 --2- 172.21.15.59:0/347449815 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbc200720f0 0x7fbc20167230 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:56:57.546 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:57.544+0000 7fbc27a27700 1 --2- 172.21.15.59:0/347449815 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fbc20143ae0 0x7fbc20159f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:56:57.546 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:57.544+0000 7fbc27a27700 1 -- 172.21.15.59:0/347449815 >> 172.21.15.59:0/347449815 conn(0x7fbc200b2740 msgr2=0x7fbc200cdf90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:56:57.547 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:57.544+0000 7fbc27a27700 1 -- 172.21.15.59:0/347449815 shutdown_connections 2023-11-24T07:56:57.547 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:57.544+0000 7fbc27a27700 1 -- 172.21.15.59:0/347449815 wait complete. 2023-11-24T07:56:57.558 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-24T07:56:18.352823+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-24T07:56:22.185140+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-11-24T07:56:23.084326+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-24T07:56:57.558 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-24T07:56:57.687 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:57.685+0000 7f6a72819700 1 Processor -- start 2023-11-24T07:56:57.687 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:57.685+0000 7f6a72819700 1 -- start start 2023-11-24T07:56:57.688 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:57.686+0000 7f6a72819700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f6a6c0cf2b0 0x7f6a6c15c1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:56:57.689 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:57.686+0000 7f6a72819700 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6a6c0d0a90 con 0x7f6a6c0ce5c0 2023-11-24T07:56:57.689 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:57.686+0000 7f6a72819700 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6a6c06ee10 con 0x7f6a6c15bde0 2023-11-24T07:56:57.689 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:57.686+0000 7f6a72819700 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6a6c06f0f0 con 0x7f6a6c0cf2b0 2023-11-24T07:56:57.689 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:57.686+0000 7f6a6b7fe700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f6a6c0cf2b0 0x7f6a6c15c1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:56:57.689 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:57.686+0000 7f6a70db6700 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7f6a6c0ce5c0 0x7f6a6c06fbd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:54264/0 (socket says 172.21.15.59:54264) 2023-11-24T07:56:57.689 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:57.686+0000 7f6a6b7fe700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f6a6c0cf2b0 0x7f6a6c15c1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:50788/0 (socket says 172.21.15.59:50788) 2023-11-24T07:56:57.689 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:57.686+0000 7f6a70db6700 1 -- 172.21.15.59:0/573204039 learned_addr learned my addr 172.21.15.59:0/573204039 (peer_addr_for_me v1:172.21.15.59:0/0) 2023-11-24T07:56:57.689 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:57.686+0000 7f6a6b7fe700 1 -- 172.21.15.59:0/573204039 >> v1:172.21.15.76:6789/0 conn(0x7f6a6c15bde0 legacy=0x7f6a6c0d01b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:56:57.689 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:57.686+0000 7f6a6b7fe700 1 -- 172.21.15.59:0/573204039 >> v1:172.21.15.59:6789/0 conn(0x7f6a6c0ce5c0 legacy=0x7f6a6c06fbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:56:57.689 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:57.686+0000 7f6a6b7fe700 1 -- 172.21.15.59:0/573204039 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6a6c06f3d0 con 0x7f6a6c0cf2b0 2023-11-24T07:56:57.689 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:57.687+0000 7f6a6b7fe700 1 --2- 172.21.15.59:0/573204039 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6a6c0cf2b0 0x7f6a6c15c1c0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f6a5c009840 tx=0x7f6a5c009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4fd29f46da64b800 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:56:57.689 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:57.687+0000 7f6a6affd700 1 -- 172.21.15.59:0/573204039 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a5c011830 con 0x7f6a6c0cf2b0 2023-11-24T07:56:57.690 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:57.687+0000 7f6a6affd700 1 -- 172.21.15.59:0/573204039 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6a5c011990 con 0x7f6a6c0cf2b0 2023-11-24T07:56:57.690 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:57.687+0000 7f6a6affd700 1 -- 172.21.15.59:0/573204039 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a5c011d20 con 0x7f6a6c0cf2b0 2023-11-24T07:56:57.690 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:57.687+0000 7f6a72819700 1 -- 172.21.15.59:0/573204039 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6a6c0cf2b0 msgr2=0x7f6a6c15c1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:56:57.690 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:57.687+0000 7f6a72819700 1 --2- 172.21.15.59:0/573204039 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6a6c0cf2b0 0x7f6a6c15c1c0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f6a5c009840 tx=0x7f6a5c009b50 comp rx=0 tx=0).stop 2023-11-24T07:56:57.691 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:57.687+0000 7f6a72819700 1 -- 172.21.15.59:0/573204039 shutdown_connections 2023-11-24T07:56:57.691 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:57.687+0000 7f6a72819700 1 --2- 172.21.15.59:0/573204039 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6a6c0cf2b0 0x7f6a6c15c1c0 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:56:57.691 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:57.687+0000 7f6a72819700 1 -- 172.21.15.59:0/573204039 >> 172.21.15.59:0/573204039 conn(0x7f6a6c0b3720 msgr2=0x7f6a6c0b3b30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:56:57.691 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:57.687+0000 7f6a72819700 1 -- 172.21.15.59:0/573204039 shutdown_connections 2023-11-24T07:56:57.691 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:57.687+0000 7f6a72819700 1 -- 172.21.15.59:0/573204039 wait complete. 2023-11-24T07:56:57.691 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:57.689+0000 7f6a72819700 1 Processor -- start 2023-11-24T07:56:57.691 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:57.689+0000 7f6a72819700 1 -- start start 2023-11-24T07:56:57.693 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:57.690+0000 7f6a72819700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6a6c0ce5c0 0x7f6a6c15e6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:56:57.693 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:57.690+0000 7f6a72819700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6a6c0cf2b0 0x7f6a6c0c8ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:56:57.694 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:57.690+0000 7f6a72819700 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f6a6c15bde0 0x7f6a6c1f4ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:56:57.694 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:57.690+0000 7f6a72819700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a6c15ec00 con 0x7f6a6c0ce5c0 2023-11-24T07:56:57.694 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:57.690+0000 7f6a72819700 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a6c0b6570 con 0x7f6a6c0cf2b0 2023-11-24T07:56:57.694 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:57.690+0000 7f6a72819700 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6a6c0b4e70 con 0x7f6a6c15bde0 2023-11-24T07:56:57.694 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:57.690+0000 7f6a6bfff700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6a6c0ce5c0 0x7f6a6c15e6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:56:57.694 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:57.690+0000 7f6a6bfff700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6a6c0ce5c0 0x7f6a6c15e6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:50792/0 (socket says 172.21.15.59:50792) 2023-11-24T07:56:57.694 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:57.690+0000 7f6a6bfff700 1 -- 172.21.15.59:0/607420342 learned_addr learned my addr 172.21.15.59:0/607420342 (peer_addr_for_me v2:172.21.15.59:0/0) 2023-11-24T07:56:57.694 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:57.690+0000 7f6a70db6700 1 --2- 172.21.15.59:0/607420342 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f6a6c15bde0 0x7f6a6c1f4ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:56:57.694 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:57.690+0000 7f6a6b7fe700 1 --2- 172.21.15.59:0/607420342 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6a6c0cf2b0 0x7f6a6c0c8ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:56:57.694 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:57.690+0000 7f6a6bfff700 1 -- 172.21.15.59:0/607420342 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f6a6c15bde0 msgr2=0x7f6a6c1f4ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:56:57.694 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:57.690+0000 7f6a6bfff700 1 --2- 172.21.15.59:0/607420342 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f6a6c15bde0 0x7f6a6c1f4ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:56:57.695 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:57.690+0000 7f6a6bfff700 1 -- 172.21.15.59:0/607420342 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6a6c0cf2b0 msgr2=0x7f6a6c0c8ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:56:57.695 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:57.690+0000 7f6a6bfff700 1 --2- 172.21.15.59:0/607420342 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6a6c0cf2b0 0x7f6a6c0c8ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:56:57.695 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:57.690+0000 7f6a6bfff700 1 -- 172.21.15.59:0/607420342 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6a5c0094a0 con 0x7f6a6c0ce5c0 2023-11-24T07:56:57.695 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:57.690+0000 7f6a6bfff700 1 --2- 172.21.15.59:0/607420342 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6a6c0ce5c0 0x7f6a6c15e6c0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f6a5400c450 tx=0x7f6a5400c760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:56:57.695 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:57.690+0000 7f6a68ff9700 1 -- 172.21.15.59:0/607420342 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a540052d0 con 0x7f6a6c0ce5c0 2023-11-24T07:56:57.695 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:57.690+0000 7f6a68ff9700 1 -- 172.21.15.59:0/607420342 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6a54005430 con 0x7f6a6c0ce5c0 2023-11-24T07:56:57.695 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:57.690+0000 7f6a72819700 1 -- 172.21.15.59:0/607420342 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6a6c0b5f20 con 0x7f6a6c0ce5c0 2023-11-24T07:56:57.695 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:57.691+0000 7f6a68ff9700 1 -- 172.21.15.59:0/607420342 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a540038c0 con 0x7f6a6c0ce5c0 2023-11-24T07:56:57.695 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:57.691+0000 7f6a72819700 1 -- 172.21.15.59:0/607420342 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6a6c0c91e0 con 0x7f6a6c0ce5c0 2023-11-24T07:56:57.695 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:57.692+0000 7f6a72819700 1 -- 172.21.15.59:0/607420342 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6a6c075480 con 0x7f6a6c0ce5c0 2023-11-24T07:56:57.695 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:57.692+0000 7f6a68ff9700 1 -- 172.21.15.59:0/607420342 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f6a54003a20 con 0x7f6a6c0ce5c0 2023-11-24T07:56:57.696 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:57.693+0000 7f6a68ff9700 1 --2- 172.21.15.59:0/607420342 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f6a4c06aed0 0x7f6a4c06d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:56:57.696 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:57.693+0000 7f6a68ff9700 1 -- 172.21.15.59:0/607420342 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f6a54088900 con 0x7f6a6c0ce5c0 2023-11-24T07:56:57.696 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:57.693+0000 7f6a6b7fe700 1 --2- 172.21.15.59:0/607420342 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f6a4c06aed0 0x7f6a4c06d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:56:57.696 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:57.693+0000 7f6a6b7fe700 1 --2- 172.21.15.59:0/607420342 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f6a4c06aed0 0x7f6a4c06d390 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f6a5c009840 tx=0x7f6a5c008030 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:56:57.699 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:57.697+0000 7f6a68ff9700 1 -- 172.21.15.59:0/607420342 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f6a540539a0 con 0x7f6a6c0ce5c0 2023-11-24T07:56:57.901 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:57.899+0000 7f6a72819700 1 -- 172.21.15.59:0/607420342 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f6a6c0b5150 con 0x7f6a6c0ce5c0 2023-11-24T07:56:57.902 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:57.900+0000 7f6a68ff9700 1 -- 172.21.15.59:0/607420342 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v20) v1 ==== 93+0+10 (secure 0 0 0) 0x7f6a540170a0 con 0x7f6a6c0ce5c0 2023-11-24T07:56:57.902 INFO:teuthology.orchestra.run.smithi059.stdout:pg_num: 1 2023-11-24T07:56:57.904 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:57.902+0000 7f6a72819700 1 -- 172.21.15.59:0/607420342 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f6a4c06aed0 msgr2=0x7f6a4c06d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:56:57.904 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:57.902+0000 7f6a72819700 1 --2- 172.21.15.59:0/607420342 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f6a4c06aed0 0x7f6a4c06d390 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f6a5c009840 tx=0x7f6a5c008030 comp rx=0 tx=0).stop 2023-11-24T07:56:57.904 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:57.902+0000 7f6a72819700 1 -- 172.21.15.59:0/607420342 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6a6c0ce5c0 msgr2=0x7f6a6c15e6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:56:57.904 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:57.902+0000 7f6a72819700 1 --2- 172.21.15.59:0/607420342 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6a6c0ce5c0 0x7f6a6c15e6c0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f6a5400c450 tx=0x7f6a5400c760 comp rx=0 tx=0).stop 2023-11-24T07:56:57.904 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:57.902+0000 7f6a72819700 1 -- 172.21.15.59:0/607420342 shutdown_connections 2023-11-24T07:56:57.904 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:57.902+0000 7f6a72819700 1 --2- 172.21.15.59:0/607420342 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f6a4c06aed0 0x7f6a4c06d390 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:56:57.904 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:57.902+0000 7f6a72819700 1 --2- 172.21.15.59:0/607420342 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f6a6c15bde0 0x7f6a6c1f4ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:56:57.904 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:57.902+0000 7f6a72819700 1 --2- 172.21.15.59:0/607420342 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6a6c0ce5c0 0x7f6a6c15e6c0 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:56:57.905 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:57.902+0000 7f6a72819700 1 --2- 172.21.15.59:0/607420342 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6a6c0cf2b0 0x7f6a6c0c8ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:56:57.905 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:57.902+0000 7f6a72819700 1 -- 172.21.15.59:0/607420342 >> 172.21.15.59:0/607420342 conn(0x7f6a6c0b3720 msgr2=0x7f6a6c070340 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:56:57.905 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:57.902+0000 7f6a72819700 1 -- 172.21.15.59:0/607420342 shutdown_connections 2023-11-24T07:56:57.905 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:57.902+0000 7f6a72819700 1 -- 172.21.15.59:0/607420342 wait complete. 2023-11-24T07:56:57.916 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-24T07:56:58.043 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:58.041+0000 7fa656620700 1 Processor -- start 2023-11-24T07:56:58.043 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:58.041+0000 7fa656620700 1 -- start start 2023-11-24T07:56:58.045 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:58.042+0000 7fa656620700 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fa650143ae0 0x7fa6500b51b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:56:58.045 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:58.042+0000 7fa656620700 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fa6501440f0 0x7fa650071bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:56:58.045 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:58.042+0000 7fa656620700 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa650143ec0 con 0x7fa6500720f0 2023-11-24T07:56:58.045 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:58.042+0000 7fa656620700 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa6500b40a0 con 0x7fa6501440f0 2023-11-24T07:56:58.045 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:58.042+0000 7fa656620700 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa650072580 con 0x7fa650143ae0 2023-11-24T07:56:58.046 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:58.042+0000 7fa654bbd700 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7fa6500720f0 0x7fa65015e8b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:54278/0 (socket says 172.21.15.59:54278) 2023-11-24T07:56:58.046 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:58.042+0000 7fa654bbd700 1 -- 172.21.15.59:0/1338023131 learned_addr learned my addr 172.21.15.59:0/1338023131 (peer_addr_for_me v1:172.21.15.59:0/0) 2023-11-24T07:56:58.047 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:58.042+0000 7fa64ffff700 1 --2- 172.21.15.59:0/1338023131 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fa650143ae0 0x7fa6500b51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:56:58.047 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:58.042+0000 7fa64f7fe700 1 --2- 172.21.15.59:0/1338023131 >> v2:172.21.15.76:3300/0 conn(0x7fa6501440f0 0x7fa650071bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:56:58.047 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:58.042+0000 7fa64effd700 1 -- 172.21.15.59:0/1338023131 <== mon.0 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 761523367 0 0) 0x7fa650143ec0 con 0x7fa6500720f0 2023-11-24T07:56:58.047 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:58.042+0000 7fa64effd700 1 -- 172.21.15.59:0/1338023131 --> v1:172.21.15.59:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa63c0035b0 con 0x7fa6500720f0 2023-11-24T07:56:58.048 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:58.042+0000 7fa64ffff700 1 -- 172.21.15.59:0/1338023131 >> v2:172.21.15.76:3300/0 conn(0x7fa6501440f0 msgr2=0x7fa650071bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T07:56:58.048 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:58.042+0000 7fa64ffff700 1 --2- 172.21.15.59:0/1338023131 >> v2:172.21.15.76:3300/0 conn(0x7fa6501440f0 0x7fa650071bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:56:58.048 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:58.042+0000 7fa64ffff700 1 -- 172.21.15.59:0/1338023131 >> v1:172.21.15.59:6789/0 conn(0x7fa6500720f0 legacy=0x7fa65015e8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:56:58.048 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:58.043+0000 7fa64ffff700 1 -- 172.21.15.59:0/1338023131 --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa6500b3ae0 con 0x7fa650143ae0 2023-11-24T07:56:58.048 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:58.043+0000 7fa64ffff700 1 --2- 172.21.15.59:0/1338023131 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fa650143ae0 0x7fa6500b51b0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fa640009840 tx=0x7fa640009b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=eb804587b6ac335e server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:56:58.049 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:58.043+0000 7fa64effd700 1 -- 172.21.15.59:0/1338023131 <== mon.2 v2:172.21.15.76:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa640011830 con 0x7fa650143ae0 2023-11-24T07:56:58.049 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:58.043+0000 7fa64effd700 1 -- 172.21.15.59:0/1338023131 <== mon.2 v2:172.21.15.76:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa640011990 con 0x7fa650143ae0 2023-11-24T07:56:58.049 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:58.043+0000 7fa64effd700 1 -- 172.21.15.59:0/1338023131 <== mon.2 v2:172.21.15.76:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa640011d20 con 0x7fa650143ae0 2023-11-24T07:56:58.049 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:58.043+0000 7fa656620700 1 -- 172.21.15.59:0/1338023131 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fa650143ae0 msgr2=0x7fa6500b51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:56:58.049 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:58.043+0000 7fa656620700 1 --2- 172.21.15.59:0/1338023131 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fa650143ae0 0x7fa6500b51b0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fa640009840 tx=0x7fa640009b50 comp rx=0 tx=0).stop 2023-11-24T07:56:58.049 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:58.043+0000 7fa656620700 1 -- 172.21.15.59:0/1338023131 shutdown_connections 2023-11-24T07:56:58.049 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:58.043+0000 7fa656620700 1 --2- 172.21.15.59:0/1338023131 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fa650143ae0 0x7fa6500b51b0 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:56:58.050 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:58.043+0000 7fa656620700 1 --2- 172.21.15.59:0/1338023131 >> v2:172.21.15.76:3300/0 conn(0x7fa6501440f0 0x7fa650071bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:56:58.050 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:58.043+0000 7fa656620700 1 -- 172.21.15.59:0/1338023131 >> 172.21.15.59:0/1338023131 conn(0x7fa6500b2740 msgr2=0x7fa6500b2b20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:56:58.050 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:58.043+0000 7fa656620700 1 -- 172.21.15.59:0/1338023131 shutdown_connections 2023-11-24T07:56:58.050 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:58.044+0000 7fa656620700 1 -- 172.21.15.59:0/1338023131 wait complete. 2023-11-24T07:56:58.053 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:58.045+0000 7fa656620700 1 Processor -- start 2023-11-24T07:56:58.053 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:58.045+0000 7fa656620700 1 -- start start 2023-11-24T07:56:58.053 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:58.046+0000 7fa656620700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa6500720f0 0x7fa650142d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:56:58.053 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:58.046+0000 7fa656620700 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fa650143ae0 0x7fa650155990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:56:58.053 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:58.046+0000 7fa656620700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa6501440f0 0x7fa650155ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:56:58.053 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:58.046+0000 7fa656620700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa650142070 con 0x7fa6500720f0 2023-11-24T07:56:58.053 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:58.046+0000 7fa656620700 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa650135440 con 0x7fa6501440f0 2023-11-24T07:56:58.053 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:58.046+0000 7fa656620700 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa650135160 con 0x7fa650143ae0 2023-11-24T07:56:58.053 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:58.046+0000 7fa64ffff700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa6500720f0 0x7fa650142d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:56:58.054 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:58.046+0000 7fa64ffff700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa6500720f0 0x7fa650142d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:50806/0 (socket says 172.21.15.59:50806) 2023-11-24T07:56:58.054 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:58.046+0000 7fa64ffff700 1 -- 172.21.15.59:0/2201977054 learned_addr learned my addr 172.21.15.59:0/2201977054 (peer_addr_for_me v2:172.21.15.59:0/0) 2023-11-24T07:56:58.054 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:58.046+0000 7fa654bbd700 1 --2- 172.21.15.59:0/2201977054 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa6501440f0 0x7fa650155ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:56:58.054 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:58.046+0000 7fa64f7fe700 1 --2- 172.21.15.59:0/2201977054 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fa650143ae0 0x7fa650155990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:56:58.054 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:58.046+0000 7fa64ffff700 1 -- 172.21.15.59:0/2201977054 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fa650143ae0 msgr2=0x7fa650155990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:56:58.054 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:58.046+0000 7fa64ffff700 1 --2- 172.21.15.59:0/2201977054 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fa650143ae0 0x7fa650155990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:56:58.054 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:58.046+0000 7fa64ffff700 1 -- 172.21.15.59:0/2201977054 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa6501440f0 msgr2=0x7fa650155ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:56:58.054 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:58.046+0000 7fa64ffff700 1 --2- 172.21.15.59:0/2201977054 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa6501440f0 0x7fa650155ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:56:58.054 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:58.046+0000 7fa64ffff700 1 -- 172.21.15.59:0/2201977054 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa6400094a0 con 0x7fa6500720f0 2023-11-24T07:56:58.054 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:58.046+0000 7fa654bbd700 1 --2- 172.21.15.59:0/2201977054 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa6501440f0 0x7fa650155ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-24T07:56:58.054 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:58.046+0000 7fa64ffff700 1 --2- 172.21.15.59:0/2201977054 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa6500720f0 0x7fa650142d60 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fa640000c00 tx=0x7fa640004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:56:58.054 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:58.046+0000 7fa64cff9700 1 -- 172.21.15.59:0/2201977054 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6400036a0 con 0x7fa6500720f0 2023-11-24T07:56:58.055 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:58.046+0000 7fa64cff9700 1 -- 172.21.15.59:0/2201977054 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa64001f070 con 0x7fa6500720f0 2023-11-24T07:56:58.055 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:58.046+0000 7fa656620700 1 -- 172.21.15.59:0/2201977054 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa650142280 con 0x7fa6500720f0 2023-11-24T07:56:58.055 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:58.046+0000 7fa64cff9700 1 -- 172.21.15.59:0/2201977054 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa640003a40 con 0x7fa6500720f0 2023-11-24T07:56:58.055 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:58.046+0000 7fa656620700 1 -- 172.21.15.59:0/2201977054 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa650142840 con 0x7fa6500720f0 2023-11-24T07:56:58.055 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:58.047+0000 7fa656620700 1 -- 172.21.15.59:0/2201977054 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa6500ccde0 con 0x7fa6500720f0 2023-11-24T07:56:58.055 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:58.048+0000 7fa64cff9700 1 -- 172.21.15.59:0/2201977054 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fa640003ba0 con 0x7fa6500720f0 2023-11-24T07:56:58.055 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:58.048+0000 7fa64cff9700 1 --2- 172.21.15.59:0/2201977054 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7fa63006aed0 0x7fa63006d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:56:58.055 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:58.048+0000 7fa64cff9700 1 -- 172.21.15.59:0/2201977054 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fa640021070 con 0x7fa6500720f0 2023-11-24T07:56:58.055 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:58.048+0000 7fa64f7fe700 1 --2- 172.21.15.59:0/2201977054 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7fa63006aed0 0x7fa63006d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:56:58.055 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:58.049+0000 7fa64f7fe700 1 --2- 172.21.15.59:0/2201977054 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7fa63006aed0 0x7fa63006d390 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fa6500b2710 tx=0x7fa638008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:56:58.056 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:58.053+0000 7fa64cff9700 1 -- 172.21.15.59:0/2201977054 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fa640055f50 con 0x7fa6500720f0 2023-11-24T07:56:58.255 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:58.254+0000 7fa656620700 1 -- 172.21.15.59:0/2201977054 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fa650075480 con 0x7fa6500720f0 2023-11-24T07:56:58.256 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:58.254+0000 7fa64cff9700 1 -- 172.21.15.59:0/2201977054 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v20) v1 ==== 104+0+11 (secure 0 0 0) 0x7fa640015020 con 0x7fa6500720f0 2023-11-24T07:56:58.256 INFO:teuthology.orchestra.run.smithi059.stdout:pg_num: 32 2023-11-24T07:56:58.258 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:58.256+0000 7fa656620700 1 -- 172.21.15.59:0/2201977054 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7fa63006aed0 msgr2=0x7fa63006d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:56:58.258 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:58.256+0000 7fa656620700 1 --2- 172.21.15.59:0/2201977054 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7fa63006aed0 0x7fa63006d390 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fa6500b2710 tx=0x7fa638008040 comp rx=0 tx=0).stop 2023-11-24T07:56:58.258 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:58.256+0000 7fa656620700 1 -- 172.21.15.59:0/2201977054 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa6500720f0 msgr2=0x7fa650142d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:56:58.258 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:58.256+0000 7fa656620700 1 --2- 172.21.15.59:0/2201977054 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa6500720f0 0x7fa650142d60 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fa640000c00 tx=0x7fa640004750 comp rx=0 tx=0).stop 2023-11-24T07:56:58.258 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:58.256+0000 7fa656620700 1 -- 172.21.15.59:0/2201977054 shutdown_connections 2023-11-24T07:56:58.258 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:58.256+0000 7fa656620700 1 --2- 172.21.15.59:0/2201977054 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7fa63006aed0 0x7fa63006d390 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:56:58.258 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:58.256+0000 7fa656620700 1 --2- 172.21.15.59:0/2201977054 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa6501440f0 0x7fa650155ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:56:58.259 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:58.256+0000 7fa656620700 1 --2- 172.21.15.59:0/2201977054 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa6500720f0 0x7fa650142d60 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:56:58.259 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:58.256+0000 7fa656620700 1 --2- 172.21.15.59:0/2201977054 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fa650143ae0 0x7fa650155990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:56:58.259 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:58.256+0000 7fa656620700 1 -- 172.21.15.59:0/2201977054 >> 172.21.15.59:0/2201977054 conn(0x7fa6500b2740 msgr2=0x7fa65006afd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:56:58.259 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:58.256+0000 7fa656620700 1 -- 172.21.15.59:0/2201977054 shutdown_connections 2023-11-24T07:56:58.259 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:58.256+0000 7fa656620700 1 -- 172.21.15.59:0/2201977054 wait complete. 2023-11-24T07:56:58.270 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-24T07:56:58.399 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:58.398+0000 7fba1443f700 1 Processor -- start 2023-11-24T07:56:58.400 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:58.398+0000 7fba1443f700 1 -- start start 2023-11-24T07:56:58.402 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:58.399+0000 7fba1443f700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fba0c0b4150 0x7fba0c06de50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:56:58.402 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:58.399+0000 7fba1443f700 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fba0c0b5ab0 0x7fba0c06e390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:56:58.402 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:58.399+0000 7fba1443f700 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fba0c071f50 con 0x7fba0c06b620 2023-11-24T07:56:58.402 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:58.399+0000 7fba1443f700 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fba0c072230 con 0x7fba0c0b4150 2023-11-24T07:56:58.403 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:58.399+0000 7fba1443f700 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fba0c072510 con 0x7fba0c0b5ab0 2023-11-24T07:56:58.403 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:58.399+0000 7fba119da700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fba0c0b4150 0x7fba0c06de50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:56:58.403 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:58.399+0000 7fba121db700 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7fba0c06b620 0x7fba0c0b3a40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:54288/0 (socket says 172.21.15.59:54288) 2023-11-24T07:56:58.403 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:58.399+0000 7fba121db700 1 -- 172.21.15.59:0/3240100209 learned_addr learned my addr 172.21.15.59:0/3240100209 (peer_addr_for_me v1:172.21.15.59:0/0) 2023-11-24T07:56:58.403 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:58.399+0000 7fba129dc700 1 --2- 172.21.15.59:0/3240100209 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fba0c0b5ab0 0x7fba0c06e390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:56:58.403 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:58.399+0000 7fba111d9700 1 -- 172.21.15.59:0/3240100209 <== mon.0 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3176846283 0 0) 0x7fba0c071f50 con 0x7fba0c06b620 2023-11-24T07:56:58.404 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:58.399+0000 7fba111d9700 1 -- 172.21.15.59:0/3240100209 --> v1:172.21.15.59:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb9f80035b0 con 0x7fba0c06b620 2023-11-24T07:56:58.404 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:58.399+0000 7fba119da700 1 -- 172.21.15.59:0/3240100209 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fba0c0b5ab0 msgr2=0x7fba0c06e390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T07:56:58.404 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:58.399+0000 7fba119da700 1 --2- 172.21.15.59:0/3240100209 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fba0c0b5ab0 0x7fba0c06e390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:56:58.404 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:58.399+0000 7fba119da700 1 -- 172.21.15.59:0/3240100209 >> v1:172.21.15.59:6789/0 conn(0x7fba0c06b620 legacy=0x7fba0c0b3a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:56:58.404 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:58.399+0000 7fba119da700 1 -- 172.21.15.59:0/3240100209 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fba0c0727f0 con 0x7fba0c0b4150 2023-11-24T07:56:58.404 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:58.399+0000 7fba119da700 1 --2- 172.21.15.59:0/3240100209 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fba0c0b4150 0x7fba0c06de50 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fba08009840 tx=0x7fba08009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=feb150b15b8fff32 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:56:58.405 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:58.399+0000 7fba111d9700 1 -- 172.21.15.59:0/3240100209 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba08011830 con 0x7fba0c0b4150 2023-11-24T07:56:58.405 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:58.399+0000 7fba111d9700 1 -- 172.21.15.59:0/3240100209 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fba08011990 con 0x7fba0c0b4150 2023-11-24T07:56:58.405 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:58.400+0000 7fba111d9700 1 -- 172.21.15.59:0/3240100209 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba08011d20 con 0x7fba0c0b4150 2023-11-24T07:56:58.405 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:58.400+0000 7fba1443f700 1 -- 172.21.15.59:0/3240100209 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fba0c0b4150 msgr2=0x7fba0c06de50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:56:58.405 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:58.400+0000 7fba1443f700 1 --2- 172.21.15.59:0/3240100209 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fba0c0b4150 0x7fba0c06de50 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fba08009840 tx=0x7fba08009b50 comp rx=0 tx=0).stop 2023-11-24T07:56:58.405 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:58.400+0000 7fba1443f700 1 -- 172.21.15.59:0/3240100209 shutdown_connections 2023-11-24T07:56:58.405 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:58.400+0000 7fba1443f700 1 --2- 172.21.15.59:0/3240100209 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fba0c0b5ab0 0x7fba0c06e390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:56:58.405 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:58.400+0000 7fba1443f700 1 --2- 172.21.15.59:0/3240100209 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fba0c0b4150 0x7fba0c06de50 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:56:58.405 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:58.400+0000 7fba1443f700 1 -- 172.21.15.59:0/3240100209 >> 172.21.15.59:0/3240100209 conn(0x7fba0c0b2740 msgr2=0x7fba0c0b2b20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:56:58.405 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:58.400+0000 7fba1443f700 1 -- 172.21.15.59:0/3240100209 shutdown_connections 2023-11-24T07:56:58.405 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:58.400+0000 7fba1443f700 1 -- 172.21.15.59:0/3240100209 wait complete. 2023-11-24T07:56:58.408 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:58.401+0000 7fba1443f700 1 Processor -- start 2023-11-24T07:56:58.408 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:58.401+0000 7fba1443f700 1 -- start start 2023-11-24T07:56:58.408 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:58.402+0000 7fba1443f700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fba0c06b620 0x7fba0c1453a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:56:58.408 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:58.402+0000 7fba1443f700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fba0c0b4150 0x7fba0c0c9a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:56:58.408 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:58.402+0000 7fba1443f700 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fba0c0b5ab0 0x7fba0c0c9fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:56:58.408 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:58.402+0000 7fba1443f700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba0c0c7840 con 0x7fba0c0b4150 2023-11-24T07:56:58.409 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:58.402+0000 7fba1443f700 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba0c1f71a0 con 0x7fba0c06b620 2023-11-24T07:56:58.409 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:58.402+0000 7fba1443f700 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fba0c0ca5e0 con 0x7fba0c0b5ab0 2023-11-24T07:56:58.409 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:58.402+0000 7fba119da700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fba0c0b4150 0x7fba0c0c9a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:56:58.409 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:58.402+0000 7fba119da700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fba0c0b4150 0x7fba0c0c9a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:50822/0 (socket says 172.21.15.59:50822) 2023-11-24T07:56:58.409 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:58.402+0000 7fba119da700 1 -- 172.21.15.59:0/2215257526 learned_addr learned my addr 172.21.15.59:0/2215257526 (peer_addr_for_me v2:172.21.15.59:0/0) 2023-11-24T07:56:58.409 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:58.403+0000 7fba129dc700 1 --2- 172.21.15.59:0/2215257526 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fba0c0b5ab0 0x7fba0c0c9fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:56:58.409 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:58.403+0000 7fba119da700 1 -- 172.21.15.59:0/2215257526 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fba0c0b5ab0 msgr2=0x7fba0c0c9fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:56:58.409 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:58.403+0000 7fba119da700 1 --2- 172.21.15.59:0/2215257526 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fba0c0b5ab0 0x7fba0c0c9fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:56:58.409 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:58.403+0000 7fba121db700 1 --2- 172.21.15.59:0/2215257526 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fba0c06b620 0x7fba0c1453a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:56:58.409 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:58.403+0000 7fba119da700 1 -- 172.21.15.59:0/2215257526 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fba0c06b620 msgr2=0x7fba0c1453a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:56:58.409 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:58.403+0000 7fba119da700 1 --2- 172.21.15.59:0/2215257526 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fba0c06b620 0x7fba0c1453a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:56:58.409 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:58.403+0000 7fba119da700 1 -- 172.21.15.59:0/2215257526 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fba080094a0 con 0x7fba0c0b4150 2023-11-24T07:56:58.410 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:58.403+0000 7fba129dc700 1 --2- 172.21.15.59:0/2215257526 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fba0c0b5ab0 0x7fba0c0c9fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T07:56:58.410 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:58.403+0000 7fba119da700 1 --2- 172.21.15.59:0/2215257526 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fba0c0b4150 0x7fba0c0c9a70 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fba080090b0 tx=0x7fba08004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:56:58.410 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:58.403+0000 7fba02ffd700 1 -- 172.21.15.59:0/2215257526 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba080036a0 con 0x7fba0c0b4150 2023-11-24T07:56:58.410 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:58.403+0000 7fba02ffd700 1 -- 172.21.15.59:0/2215257526 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fba0801f070 con 0x7fba0c0b4150 2023-11-24T07:56:58.410 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:58.403+0000 7fba1443f700 1 -- 172.21.15.59:0/2215257526 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fba0c0ca8c0 con 0x7fba0c0b4150 2023-11-24T07:56:58.410 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:58.403+0000 7fba02ffd700 1 -- 172.21.15.59:0/2215257526 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba08003a40 con 0x7fba0c0b4150 2023-11-24T07:56:58.410 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:58.403+0000 7fba1443f700 1 -- 172.21.15.59:0/2215257526 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fba0c0caba0 con 0x7fba0c0b4150 2023-11-24T07:56:58.410 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:58.404+0000 7fba1443f700 1 -- 172.21.15.59:0/2215257526 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fba0c0cad30 con 0x7fba0c0b4150 2023-11-24T07:56:58.410 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:58.404+0000 7fba02ffd700 1 -- 172.21.15.59:0/2215257526 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fba08003ba0 con 0x7fba0c0b4150 2023-11-24T07:56:58.410 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:58.405+0000 7fba02ffd700 1 --2- 172.21.15.59:0/2215257526 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7fb9f006aed0 0x7fb9f006d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:56:58.410 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:58.405+0000 7fba02ffd700 1 -- 172.21.15.59:0/2215257526 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fba08021070 con 0x7fba0c0b4150 2023-11-24T07:56:58.411 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:58.405+0000 7fba121db700 1 --2- 172.21.15.59:0/2215257526 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7fb9f006aed0 0x7fb9f006d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:56:58.411 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:58.405+0000 7fba121db700 1 --2- 172.21.15.59:0/2215257526 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7fb9f006aed0 0x7fb9f006d390 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fb9fc0011e0 tx=0x7fb9fc009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:56:58.411 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:58.409+0000 7fba02ffd700 1 -- 172.21.15.59:0/2215257526 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fba08055f60 con 0x7fba0c0b4150 2023-11-24T07:56:58.612 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:58.610+0000 7fba1443f700 1 -- 172.21.15.59:0/2215257526 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fba0c075480 con 0x7fba0c0b4150 2023-11-24T07:56:58.612 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:58.610+0000 7fba02ffd700 1 -- 172.21.15.59:0/2215257526 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v20) v1 ==== 100+0+11 (secure 0 0 0) 0x7fba08015020 con 0x7fba0c0b4150 2023-11-24T07:56:58.612 INFO:teuthology.orchestra.run.smithi059.stdout:pg_num: 64 2023-11-24T07:56:58.614 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:58.612+0000 7fba1443f700 1 -- 172.21.15.59:0/2215257526 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7fb9f006aed0 msgr2=0x7fb9f006d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:56:58.615 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:58.613+0000 7fba1443f700 1 --2- 172.21.15.59:0/2215257526 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7fb9f006aed0 0x7fb9f006d390 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fb9fc0011e0 tx=0x7fb9fc009040 comp rx=0 tx=0).stop 2023-11-24T07:56:58.615 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:58.613+0000 7fba1443f700 1 -- 172.21.15.59:0/2215257526 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fba0c0b4150 msgr2=0x7fba0c0c9a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:56:58.615 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:58.613+0000 7fba1443f700 1 --2- 172.21.15.59:0/2215257526 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fba0c0b4150 0x7fba0c0c9a70 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fba080090b0 tx=0x7fba08004750 comp rx=0 tx=0).stop 2023-11-24T07:56:58.615 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:58.613+0000 7fba1443f700 1 -- 172.21.15.59:0/2215257526 shutdown_connections 2023-11-24T07:56:58.615 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:58.613+0000 7fba1443f700 1 --2- 172.21.15.59:0/2215257526 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7fb9f006aed0 0x7fb9f006d390 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:56:58.615 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:58.613+0000 7fba1443f700 1 --2- 172.21.15.59:0/2215257526 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fba0c0b5ab0 0x7fba0c0c9fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:56:58.615 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:58.613+0000 7fba1443f700 1 --2- 172.21.15.59:0/2215257526 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fba0c06b620 0x7fba0c1453a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:56:58.615 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:58.613+0000 7fba1443f700 1 --2- 172.21.15.59:0/2215257526 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fba0c0b4150 0x7fba0c0c9a70 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:56:58.615 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:58.613+0000 7fba1443f700 1 -- 172.21.15.59:0/2215257526 >> 172.21.15.59:0/2215257526 conn(0x7fba0c0b2740 msgr2=0x7fba0c146240 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:56:58.615 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:58.613+0000 7fba1443f700 1 -- 172.21.15.59:0/2215257526 shutdown_connections 2023-11-24T07:56:58.615 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:58.613+0000 7fba1443f700 1 -- 172.21.15.59:0/2215257526 wait complete. 2023-11-24T07:56:58.691 INFO:tasks.cephfs_test_runner:Starting test: test_nonexistent_subvolume_group_snapshot_rm (tasks.cephfs.test_volumes.TestSubvolumeGroupSnapshots) 2023-11-24T07:56:58.691 INFO:tasks.cephfs_test_runner:test_nonexistent_subvolume_group_snapshot_rm (tasks.cephfs.test_volumes.TestSubvolumeGroupSnapshots) ... skipped 'skipping subvolumegroup snapshot tests' 2023-11-24T07:56:58.691 INFO:tasks.cephfs_test_runner:Starting test: test_subvolume_group_snapshot_create_and_rm (tasks.cephfs.test_volumes.TestSubvolumeGroupSnapshots) 2023-11-24T07:56:58.691 INFO:tasks.cephfs_test_runner:test_subvolume_group_snapshot_create_and_rm (tasks.cephfs.test_volumes.TestSubvolumeGroupSnapshots) ... skipped 'skipping subvolumegroup snapshot tests' 2023-11-24T07:56:58.691 INFO:tasks.cephfs_test_runner:Starting test: test_subvolume_group_snapshot_idempotence (tasks.cephfs.test_volumes.TestSubvolumeGroupSnapshots) 2023-11-24T07:56:58.691 INFO:tasks.cephfs_test_runner:test_subvolume_group_snapshot_idempotence (tasks.cephfs.test_volumes.TestSubvolumeGroupSnapshots) ... skipped 'skipping subvolumegroup snapshot tests' 2023-11-24T07:56:58.691 INFO:tasks.cephfs_test_runner:Starting test: test_subvolume_group_snapshot_ls (tasks.cephfs.test_volumes.TestSubvolumeGroupSnapshots) 2023-11-24T07:56:58.691 INFO:tasks.cephfs_test_runner:test_subvolume_group_snapshot_ls (tasks.cephfs.test_volumes.TestSubvolumeGroupSnapshots) ... skipped 'skipping subvolumegroup snapshot tests' 2023-11-24T07:56:58.691 INFO:tasks.cephfs_test_runner:Starting test: test_subvolume_group_snapshot_rm_force (tasks.cephfs.test_volumes.TestSubvolumeGroupSnapshots) 2023-11-24T07:56:58.692 INFO:tasks.cephfs_test_runner:test_subvolume_group_snapshot_rm_force (tasks.cephfs.test_volumes.TestSubvolumeGroupSnapshots) ... skipped 'skipping subvolumegroup snapshot tests' 2023-11-24T07:56:58.692 INFO:tasks.cephfs_test_runner:Starting test: test_subvolume_group_snapshot_unsupported_status (tasks.cephfs.test_volumes.TestSubvolumeGroupSnapshots) 2023-11-24T07:56:58.692 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-24T07:56:58.817 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:58.816+0000 7fefe332e700 1 Processor -- start 2023-11-24T07:56:58.818 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:58.816+0000 7fefe332e700 1 -- start start 2023-11-24T07:56:58.820 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:58.817+0000 7fefe332e700 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fefdc15e1e0 0x7fefdc0b12a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:56:58.821 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:58.817+0000 7fefe332e700 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fefdc06b400 0x7fefdc0b17e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:56:58.821 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:58.817+0000 7fefe332e700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fefdc0b5e40 0x7fefdc0726a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:56:58.821 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:58.817+0000 7fefe332e700 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fefdc0b1d20 con 0x7fefdc0b5e40 2023-11-24T07:56:58.821 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:58.817+0000 7fefe332e700 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fefdc06bb80 con 0x7fefdc06b400 2023-11-24T07:56:58.821 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:58.817+0000 7fefe332e700 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fefdc06be20 con 0x7fefdc15e1e0 2023-11-24T07:56:58.822 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:58.817+0000 7fefe18cb700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fefdc0b5e40 0x7fefdc0726a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:56:58.822 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:58.817+0000 7fefe18cb700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fefdc0b5e40 0x7fefdc0726a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:50832/0 (socket says 172.21.15.59:50832) 2023-11-24T07:56:58.822 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:58.817+0000 7fefe18cb700 1 -- 172.21.15.59:0/3895590081 learned_addr learned my addr 172.21.15.59:0/3895590081 (peer_addr_for_me v2:172.21.15.59:0/0) 2023-11-24T07:56:58.822 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:58.817+0000 7fefe10ca700 1 --2- 172.21.15.59:0/3895590081 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fefdc15e1e0 0x7fefdc0b12a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:56:58.823 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:58.817+0000 7fefe08c9700 1 --2- 172.21.15.59:0/3895590081 >> v2:172.21.15.76:3300/0 conn(0x7fefdc06b400 0x7fefdc0b17e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:56:58.823 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:58.817+0000 7fefe18cb700 1 -- 172.21.15.59:0/3895590081 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fefdc15e1e0 msgr2=0x7fefdc0b12a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T07:56:58.823 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:58.817+0000 7fefe18cb700 1 --2- 172.21.15.59:0/3895590081 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fefdc15e1e0 0x7fefdc0b12a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:56:58.823 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:58.817+0000 7fefe18cb700 1 -- 172.21.15.59:0/3895590081 >> v2:172.21.15.76:3300/0 conn(0x7fefdc06b400 msgr2=0x7fefdc0b17e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T07:56:58.823 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:58.817+0000 7fefe18cb700 1 --2- 172.21.15.59:0/3895590081 >> v2:172.21.15.76:3300/0 conn(0x7fefdc06b400 0x7fefdc0b17e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:56:58.823 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:58.817+0000 7fefe18cb700 1 -- 172.21.15.59:0/3895590081 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fefdc06c100 con 0x7fefdc0b5e40 2023-11-24T07:56:58.824 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:58.817+0000 7fefe08c9700 1 --2- 172.21.15.59:0/3895590081 >> v2:172.21.15.76:3300/0 conn(0x7fefdc06b400 0x7fefdc0b17e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T07:56:58.824 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:58.817+0000 7fefe18cb700 1 --2- 172.21.15.59:0/3895590081 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fefdc0b5e40 0x7fefdc0726a0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7fefdc06d2e0 tx=0x7fefd400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9e696cf5bd3dcf25 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:56:58.824 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:58.818+0000 7fefd3fff700 1 -- 172.21.15.59:0/3895590081 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fefd400e070 con 0x7fefdc0b5e40 2023-11-24T07:56:58.824 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:58.818+0000 7fefd3fff700 1 -- 172.21.15.59:0/3895590081 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fefd400b9c0 con 0x7fefdc0b5e40 2023-11-24T07:56:58.824 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:58.818+0000 7fefd3fff700 1 -- 172.21.15.59:0/3895590081 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fefd400bd70 con 0x7fefdc0b5e40 2023-11-24T07:56:58.824 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:58.818+0000 7fefe332e700 1 -- 172.21.15.59:0/3895590081 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fefdc0b5e40 msgr2=0x7fefdc0726a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:56:58.824 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:58.818+0000 7fefe332e700 1 --2- 172.21.15.59:0/3895590081 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fefdc0b5e40 0x7fefdc0726a0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7fefdc06d2e0 tx=0x7fefd400b330 comp rx=0 tx=0).stop 2023-11-24T07:56:58.824 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:58.818+0000 7fefe332e700 1 -- 172.21.15.59:0/3895590081 shutdown_connections 2023-11-24T07:56:58.824 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:58.818+0000 7fefe332e700 1 --2- 172.21.15.59:0/3895590081 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fefdc0b5e40 0x7fefdc0726a0 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:56:58.824 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:58.818+0000 7fefe332e700 1 --2- 172.21.15.59:0/3895590081 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fefdc15e1e0 0x7fefdc0b12a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:56:58.824 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:58.818+0000 7fefe332e700 1 --2- 172.21.15.59:0/3895590081 >> v2:172.21.15.76:3300/0 conn(0x7fefdc06b400 0x7fefdc0b17e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:56:58.824 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:58.818+0000 7fefe332e700 1 -- 172.21.15.59:0/3895590081 >> 172.21.15.59:0/3895590081 conn(0x7fefdc0b5070 msgr2=0x7fefdc0b5730 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:56:58.825 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:58.818+0000 7fefe332e700 1 -- 172.21.15.59:0/3895590081 shutdown_connections 2023-11-24T07:56:58.825 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:58.818+0000 7fefe332e700 1 -- 172.21.15.59:0/3895590081 wait complete. 2023-11-24T07:56:58.827 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:58.819+0000 7fefe332e700 1 Processor -- start 2023-11-24T07:56:58.827 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:58.819+0000 7fefe332e700 1 -- start start 2023-11-24T07:56:58.827 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:58.820+0000 7fefe332e700 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fefdc06b400 0x7fefdc09d8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:56:58.828 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:58.820+0000 7fefe332e700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fefdc0b5e40 0x7fefdc1f4240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:56:58.828 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:58.820+0000 7fefe332e700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fefdc15e1e0 0x7fefdc15b830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:56:58.828 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:58.820+0000 7fefe332e700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fefdc1f4780 con 0x7fefdc0b5e40 2023-11-24T07:56:58.828 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:58.820+0000 7fefe332e700 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fefdc09b710 con 0x7fefdc15e1e0 2023-11-24T07:56:58.828 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:58.820+0000 7fefe332e700 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fefdc0c9a70 con 0x7fefdc06b400 2023-11-24T07:56:58.828 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:58.820+0000 7fefe08c9700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fefdc0b5e40 0x7fefdc1f4240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:56:58.828 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:58.820+0000 7fefe08c9700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fefdc0b5e40 0x7fefdc1f4240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:50836/0 (socket says 172.21.15.59:50836) 2023-11-24T07:56:58.828 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:58.820+0000 7fefe08c9700 1 -- 172.21.15.59:0/486440960 learned_addr learned my addr 172.21.15.59:0/486440960 (peer_addr_for_me v2:172.21.15.59:0/0) 2023-11-24T07:56:58.828 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:58.821+0000 7fefe18cb700 1 --2- 172.21.15.59:0/486440960 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fefdc15e1e0 0x7fefdc15b830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:56:58.828 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:58.821+0000 7fefe10ca700 1 --2- 172.21.15.59:0/486440960 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fefdc06b400 0x7fefdc09d8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:56:58.829 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:58.821+0000 7fefe08c9700 1 -- 172.21.15.59:0/486440960 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fefdc06b400 msgr2=0x7fefdc09d8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:56:58.829 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:58.821+0000 7fefe08c9700 1 --2- 172.21.15.59:0/486440960 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fefdc06b400 0x7fefdc09d8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:56:58.829 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:58.821+0000 7fefe08c9700 1 -- 172.21.15.59:0/486440960 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fefdc15e1e0 msgr2=0x7fefdc15b830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:56:58.829 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:58.821+0000 7fefe08c9700 1 --2- 172.21.15.59:0/486440960 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fefdc15e1e0 0x7fefdc15b830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:56:58.829 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:58.821+0000 7fefe08c9700 1 -- 172.21.15.59:0/486440960 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fefd4009b30 con 0x7fefdc0b5e40 2023-11-24T07:56:58.829 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:58.821+0000 7fefe08c9700 1 --2- 172.21.15.59:0/486440960 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fefdc0b5e40 0x7fefdc1f4240 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7fefcc00c740 tx=0x7fefcc00cb00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:56:58.829 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:58.821+0000 7fefd1ffb700 1 -- 172.21.15.59:0/486440960 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fefcc011800 con 0x7fefdc0b5e40 2023-11-24T07:56:58.829 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:58.821+0000 7fefd1ffb700 1 -- 172.21.15.59:0/486440960 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fefcc011960 con 0x7fefdc0b5e40 2023-11-24T07:56:58.829 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:58.821+0000 7fefe332e700 1 -- 172.21.15.59:0/486440960 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fefdc14d3e0 con 0x7fefdc0b5e40 2023-11-24T07:56:58.829 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:58.821+0000 7fefd1ffb700 1 -- 172.21.15.59:0/486440960 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fefcc011cd0 con 0x7fefdc0b5e40 2023-11-24T07:56:58.829 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:58.821+0000 7fefe332e700 1 -- 172.21.15.59:0/486440960 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fefdc14d9a0 con 0x7fefdc0b5e40 2023-11-24T07:56:58.830 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:58.822+0000 7fefe332e700 1 -- 172.21.15.59:0/486440960 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fefdc068670 con 0x7fefdc0b5e40 2023-11-24T07:56:58.830 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:58.822+0000 7fefd1ffb700 1 -- 172.21.15.59:0/486440960 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fefcc021350 con 0x7fefdc0b5e40 2023-11-24T07:56:58.830 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:58.823+0000 7fefd1ffb700 1 --2- 172.21.15.59:0/486440960 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7fefc006aed0 0x7fefc006d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:56:58.830 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:58.823+0000 7fefd1ffb700 1 -- 172.21.15.59:0/486440960 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fefcc089580 con 0x7fefdc0b5e40 2023-11-24T07:56:58.830 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:58.823+0000 7fefe10ca700 1 --2- 172.21.15.59:0/486440960 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7fefc006aed0 0x7fefc006d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:56:58.830 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:58.823+0000 7fefe10ca700 1 --2- 172.21.15.59:0/486440960 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7fefc006aed0 0x7fefc006d390 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fefdc09e4f0 tx=0x7fefd8008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:56:58.830 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:58.827+0000 7fefd1ffb700 1 -- 172.21.15.59:0/486440960 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fefcc054640 con 0x7fefdc0b5e40 2023-11-24T07:56:59.028 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:59.026+0000 7fefe332e700 1 -- 172.21.15.59:0/486440960 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fefdc09bba0 con 0x7fefdc0b5e40 2023-11-24T07:56:59.029 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:59.028+0000 7fefd1ffb700 1 -- 172.21.15.59:0/486440960 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) v1 ==== 74+0+16437 (secure 0 0 0) 0x7fefcc057c60 con 0x7fefdc0b5e40 2023-11-24T07:56:59.030 INFO:teuthology.orchestra.run.smithi059.stdout: 2023-11-24T07:56:59.030 INFO:teuthology.orchestra.run.smithi059.stdout:{"epoch":20,"fsid":"9f0e8741-b8f9-48d5-a2c3-09f45903303d","created":"2023-11-24T07:56:06.552729+0000","modified":"2023-11-24T07:56:25.705091+0000","last_up_change":"2023-11-24T07:56:17.372352+0000","last_in_change":"2023-11-24T07:56:09.073250+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-24T07:56:18.352823+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-11-24T07:56:22.185140+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-11-24T07:56:23.084326+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"6d8f9e9b-1782-4e41-928b-a39f2a8ebc06","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6800","nonce":3373878772},{"type":"v1","addr":"172.21.15.59:6801","nonce":3373878772}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6802","nonce":3373878772},{"type":"v1","addr":"172.21.15.59:6803","nonce":3373878772}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6806","nonce":3373878772},{"type":"v1","addr":"172.21.15.59:6807","nonce":3373878772}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6804","nonce":3373878772},{"type":"v1","addr":"172.21.15.59:6805","nonce":3373878772}]},"public_addr":"172.21.15.59:6801/3373878772","cluster_addr":"172.21.15.59:6803/3373878772","heartbeat_back_addr":"172.21.15.59:6807/3373878772","heartbeat_front_addr":"172.21.15.59:6805/3373878772","state":["exists","up"]},{"osd":1,"uuid":"e8d882aa-52e1-4c04-9e1e-c44ef9204a5f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6824","nonce":648258563},{"type":"v1","addr":"172.21.15.59:6825","nonce":648258563}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6826","nonce":648258563},{"type":"v1","addr":"172.21.15.59:6827","nonce":648258563}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6830","nonce":648258563},{"type":"v1","addr":"172.21.15.59:6831","nonce":648258563}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6828","nonce":648258563},{"type":"v1","addr":"172.21.15.59:6829","nonce":648258563}]},"public_addr":"172.21.15.59:6825/648258563","cluster_addr":"172.21.15.59:6827/648258563","heartbeat_back_addr":"172.21.15.59:6831/648258563","heartbeat_front_addr":"172.21.15.59:6829/648258563","state":["exists","up"]},{"osd":2,"uuid":"69eebfa0-695b-4b21-90b9-8a3537168a01","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6808","nonce":1840430654},{"type":"v1","addr":"172.21.15.59:6809","nonce":1840430654}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6810","nonce":1840430654},{"type":"v1","addr":"172.21.15.59:6811","nonce":1840430654}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6814","nonce":1840430654},{"type":"v1","addr":"172.21.15.59:6815","nonce":1840430654}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6812","nonce":1840430654},{"type":"v1","addr":"172.21.15.59:6813","nonce":1840430654}]},"public_addr":"172.21.15.59:6809/1840430654","cluster_addr":"172.21.15.59:6811/1840430654","heartbeat_back_addr":"172.21.15.59:6815/1840430654","heartbeat_front_addr":"172.21.15.59:6813/1840430654","state":["exists","up"]},{"osd":3,"uuid":"96fd77fc-f911-46ff-9445-6f0be2daa8f4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6816","nonce":2504025850},{"type":"v1","addr":"172.21.15.59:6817","nonce":2504025850}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6818","nonce":2504025850},{"type":"v1","addr":"172.21.15.59:6819","nonce":2504025850}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6822","nonce":2504025850},{"type":"v1","addr":"172.21.15.59:6823","nonce":2504025850}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6820","nonce":2504025850},{"type":"v1","addr":"172.21.15.59:6821","nonce":2504025850}]},"public_addr":"172.21.15.59:6817/2504025850","cluster_addr":"172.21.15.59:6819/2504025850","heartbeat_back_addr":"172.21.15.59:6823/2504025850","heartbeat_front_addr":"172.21.15.59:6821/2504025850","state":["exists","up"]},{"osd":4,"uuid":"7ac29fca-b5b2-4c95-9d69-2f164cbbc708","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6816","nonce":2382463431},{"type":"v1","addr":"172.21.15.76:6818","nonce":2382463431}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6820","nonce":2382463431},{"type":"v1","addr":"172.21.15.76:6822","nonce":2382463431}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6828","nonce":2382463431},{"type":"v1","addr":"172.21.15.76:6830","nonce":2382463431}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6824","nonce":2382463431},{"type":"v1","addr":"172.21.15.76:6826","nonce":2382463431}]},"public_addr":"172.21.15.76:6818/2382463431","cluster_addr":"172.21.15.76:6822/2382463431","heartbeat_back_addr":"172.21.15.76:6830/2382463431","heartbeat_front_addr":"172.21.15.76:6826/2382463431","state":["exists","up"]},{"osd":5,"uuid":"485bf711-3851-4238-a4f4-51d52e0a9c58","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6800","nonce":329334308},{"type":"v1","addr":"172.21.15.76:6802","nonce":329334308}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6805","nonce":329334308},{"type":"v1","addr":"172.21.15.76:6807","nonce":329334308}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6814","nonce":329334308},{"type":"v1","addr":"172.21.15.76:6815","nonce":329334308}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6810","nonce":329334308},{"type":"v1","addr":"172.21.15.76:6812","nonce":329334308}]},"public_addr":"172.21.15.76:6802/329334308","cluster_addr":"172.21.15.76:6807/329334308","heartbeat_back_addr":"172.21.15.76:6815/329334308","heartbeat_front_addr":"172.21.15.76:6812/329334308","state":["exists","up"]},{"osd":6,"uuid":"72b3f2fc-1e4a-4e32-968d-003647a0286f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6801","nonce":3946579410},{"type":"v1","addr":"172.21.15.76:6803","nonce":3946579410}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6804","nonce":3946579410},{"type":"v1","addr":"172.21.15.76:6806","nonce":3946579410}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6811","nonce":3946579410},{"type":"v1","addr":"172.21.15.76:6813","nonce":3946579410}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6808","nonce":3946579410},{"type":"v1","addr":"172.21.15.76:6809","nonce":3946579410}]},"public_addr":"172.21.15.76:6803/3946579410","cluster_addr":"172.21.15.76:6806/3946579410","heartbeat_back_addr":"172.21.15.76:6813/3946579410","heartbeat_front_addr":"172.21.15.76:6809/3946579410","state":["exists","up"]},{"osd":7,"uuid":"1ddceb90-6c5d-4d1a-ad56-99cb877f8c70","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6817","nonce":841761538},{"type":"v1","addr":"172.21.15.76:6819","nonce":841761538}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6821","nonce":841761538},{"type":"v1","addr":"172.21.15.76:6823","nonce":841761538}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6829","nonce":841761538},{"type":"v1","addr":"172.21.15.76:6831","nonce":841761538}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6825","nonce":841761538},{"type":"v1","addr":"172.21.15.76:6827","nonce":841761538}]},"public_addr":"172.21.15.76:6819/841761538","cluster_addr":"172.21.15.76:6823/841761538","heartbeat_back_addr":"172.21.15.76:6831/841761538","heartbeat_front_addr":"172.21.15.76:6827/841761538","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T07:56:14.294838+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T07:56:14.270779+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T07:56:14.231809+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T07:56:14.260983+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T07:56:14.302997+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T07:56:14.258600+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T07:56:14.251413+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T07:56:14.304727+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-24T07:56:59.032 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:59.030+0000 7fefe332e700 1 -- 172.21.15.59:0/486440960 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7fefc006aed0 msgr2=0x7fefc006d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:56:59.032 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:59.030+0000 7fefe332e700 1 --2- 172.21.15.59:0/486440960 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7fefc006aed0 0x7fefc006d390 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fefdc09e4f0 tx=0x7fefd8008040 comp rx=0 tx=0).stop 2023-11-24T07:56:59.032 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:59.030+0000 7fefe332e700 1 -- 172.21.15.59:0/486440960 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fefdc0b5e40 msgr2=0x7fefdc1f4240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:56:59.032 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:59.030+0000 7fefe332e700 1 --2- 172.21.15.59:0/486440960 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fefdc0b5e40 0x7fefdc1f4240 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7fefcc00c740 tx=0x7fefcc00cb00 comp rx=0 tx=0).stop 2023-11-24T07:56:59.032 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:59.030+0000 7fefe332e700 1 -- 172.21.15.59:0/486440960 shutdown_connections 2023-11-24T07:56:59.032 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:59.030+0000 7fefe332e700 1 --2- 172.21.15.59:0/486440960 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7fefc006aed0 0x7fefc006d390 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:56:59.033 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:59.030+0000 7fefe332e700 1 --2- 172.21.15.59:0/486440960 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fefdc15e1e0 0x7fefdc15b830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:56:59.033 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:59.030+0000 7fefe332e700 1 --2- 172.21.15.59:0/486440960 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fefdc06b400 0x7fefdc09d8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:56:59.033 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:59.030+0000 7fefe332e700 1 --2- 172.21.15.59:0/486440960 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fefdc0b5e40 0x7fefdc1f4240 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:56:59.033 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:59.030+0000 7fefe332e700 1 -- 172.21.15.59:0/486440960 >> 172.21.15.59:0/486440960 conn(0x7fefdc0b5070 msgr2=0x7fefdc15fc70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:56:59.033 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:59.030+0000 7fefe332e700 1 -- 172.21.15.59:0/486440960 shutdown_connections 2023-11-24T07:56:59.033 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:59.030+0000 7fefe332e700 1 -- 172.21.15.59:0/486440960 wait complete. 2023-11-24T07:56:59.044 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-24T07:56:18.352823+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-24T07:56:22.185140+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-11-24T07:56:23.084326+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-24T07:56:59.044 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-24T07:56:59.173 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:59.171+0000 7f2cb2a0d700 1 Processor -- start 2023-11-24T07:56:59.173 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:59.171+0000 7f2cb2a0d700 1 -- start start 2023-11-24T07:56:59.175 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:59.172+0000 7f2cb2a0d700 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f2cac142810 0x7f2cac06afd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:56:59.176 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:59.172+0000 7f2cb2a0d700 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f2cac1459a0 0x7f2cac06b510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:56:59.176 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:59.172+0000 7f2cb2a0d700 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2cac0cf5f0 con 0x7f2cac06e580 2023-11-24T07:56:59.176 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:59.172+0000 7f2cb2a0d700 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2cac06ea80 con 0x7f2cac1459a0 2023-11-24T07:56:59.176 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:59.172+0000 7f2cb2a0d700 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2cac06ed60 con 0x7f2cac142810 2023-11-24T07:56:59.176 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:59.172+0000 7f2cb0faa700 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7f2cac06e580 0x7f2cac071bb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:54320/0 (socket says 172.21.15.59:54320) 2023-11-24T07:56:59.177 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:59.172+0000 7f2cb0faa700 1 -- 172.21.15.59:0/3047003785 learned_addr learned my addr 172.21.15.59:0/3047003785 (peer_addr_for_me v1:172.21.15.59:0/0) 2023-11-24T07:56:59.177 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:59.172+0000 7f2cabfff700 1 --2- 172.21.15.59:0/3047003785 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f2cac142810 0x7f2cac06afd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:56:59.177 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:59.172+0000 7f2cab7fe700 1 --2- 172.21.15.59:0/3047003785 >> v2:172.21.15.76:3300/0 conn(0x7f2cac1459a0 0x7f2cac06b510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:56:59.177 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:59.172+0000 7f2caaffd700 1 -- 172.21.15.59:0/3047003785 <== mon.0 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3131808736 0 0) 0x7f2cac0cf5f0 con 0x7f2cac06e580 2023-11-24T07:56:59.177 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:59.172+0000 7f2caaffd700 1 -- 172.21.15.59:0/3047003785 --> v1:172.21.15.59:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2c980035b0 con 0x7f2cac06e580 2023-11-24T07:56:59.178 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:59.172+0000 7f2cabfff700 1 -- 172.21.15.59:0/3047003785 >> v2:172.21.15.76:3300/0 conn(0x7f2cac1459a0 msgr2=0x7f2cac06b510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T07:56:59.178 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:59.172+0000 7f2cabfff700 1 --2- 172.21.15.59:0/3047003785 >> v2:172.21.15.76:3300/0 conn(0x7f2cac1459a0 0x7f2cac06b510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:56:59.178 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:59.172+0000 7f2cabfff700 1 -- 172.21.15.59:0/3047003785 >> v1:172.21.15.59:6789/0 conn(0x7f2cac06e580 legacy=0x7f2cac071bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:56:59.178 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:59.172+0000 7f2cabfff700 1 -- 172.21.15.59:0/3047003785 --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2cac06f040 con 0x7f2cac142810 2023-11-24T07:56:59.178 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:59.173+0000 7f2cabfff700 1 --2- 172.21.15.59:0/3047003785 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f2cac142810 0x7f2cac06afd0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f2c94009840 tx=0x7f2c94009b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2abf477b3cdf49b3 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:56:59.179 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:59.173+0000 7f2caaffd700 1 -- 172.21.15.59:0/3047003785 <== mon.2 v2:172.21.15.76:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c94011830 con 0x7f2cac142810 2023-11-24T07:56:59.179 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:59.173+0000 7f2caaffd700 1 -- 172.21.15.59:0/3047003785 <== mon.2 v2:172.21.15.76:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2c94011990 con 0x7f2cac142810 2023-11-24T07:56:59.180 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:59.173+0000 7f2caaffd700 1 -- 172.21.15.59:0/3047003785 <== mon.2 v2:172.21.15.76:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c94011d20 con 0x7f2cac142810 2023-11-24T07:56:59.180 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:59.173+0000 7f2cb2a0d700 1 -- 172.21.15.59:0/3047003785 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f2cac142810 msgr2=0x7f2cac06afd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:56:59.180 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:59.173+0000 7f2cb2a0d700 1 --2- 172.21.15.59:0/3047003785 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f2cac142810 0x7f2cac06afd0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f2c94009840 tx=0x7f2c94009b50 comp rx=0 tx=0).stop 2023-11-24T07:56:59.180 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:59.173+0000 7f2cb2a0d700 1 -- 172.21.15.59:0/3047003785 shutdown_connections 2023-11-24T07:56:59.180 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:59.173+0000 7f2cb2a0d700 1 --2- 172.21.15.59:0/3047003785 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f2cac142810 0x7f2cac06afd0 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:56:59.181 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:59.173+0000 7f2cb2a0d700 1 --2- 172.21.15.59:0/3047003785 >> v2:172.21.15.76:3300/0 conn(0x7f2cac1459a0 0x7f2cac06b510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:56:59.181 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:59.173+0000 7f2cb2a0d700 1 -- 172.21.15.59:0/3047003785 >> 172.21.15.59:0/3047003785 conn(0x7f2cac0b2740 msgr2=0x7f2cac0b2b20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:56:59.181 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:59.173+0000 7f2cb2a0d700 1 -- 172.21.15.59:0/3047003785 shutdown_connections 2023-11-24T07:56:59.181 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:59.173+0000 7f2cb2a0d700 1 -- 172.21.15.59:0/3047003785 wait complete. 2023-11-24T07:56:59.183 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:59.175+0000 7f2cb2a0d700 1 Processor -- start 2023-11-24T07:56:59.184 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:59.175+0000 7f2cb2a0d700 1 -- start start 2023-11-24T07:56:59.184 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:59.176+0000 7f2cb2a0d700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2cac06e580 0x7f2cac136190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:56:59.184 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:59.176+0000 7f2cb2a0d700 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f2cac142810 0x7f2cac13c630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:56:59.184 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:59.176+0000 7f2cb2a0d700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2cac1459a0 0x7f2cac13a5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:56:59.184 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:59.176+0000 7f2cb2a0d700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2cac13d6b0 con 0x7f2cac06e580 2023-11-24T07:56:59.184 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:59.176+0000 7f2cb2a0d700 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2cac1f2300 con 0x7f2cac1459a0 2023-11-24T07:56:59.184 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:59.176+0000 7f2cb2a0d700 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2cac1439c0 con 0x7f2cac142810 2023-11-24T07:56:59.184 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:59.176+0000 7f2cabfff700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2cac06e580 0x7f2cac136190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:56:59.184 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:59.176+0000 7f2cabfff700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2cac06e580 0x7f2cac136190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:50848/0 (socket says 172.21.15.59:50848) 2023-11-24T07:56:59.184 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:59.176+0000 7f2cabfff700 1 -- 172.21.15.59:0/1346787686 learned_addr learned my addr 172.21.15.59:0/1346787686 (peer_addr_for_me v2:172.21.15.59:0/0) 2023-11-24T07:56:59.185 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:59.176+0000 7f2cb0faa700 1 --2- 172.21.15.59:0/1346787686 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2cac1459a0 0x7f2cac13a5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:56:59.185 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:59.176+0000 7f2cab7fe700 1 --2- 172.21.15.59:0/1346787686 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f2cac142810 0x7f2cac13c630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:56:59.185 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:59.176+0000 7f2cabfff700 1 -- 172.21.15.59:0/1346787686 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f2cac142810 msgr2=0x7f2cac13c630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:56:59.185 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:59.176+0000 7f2cabfff700 1 --2- 172.21.15.59:0/1346787686 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f2cac142810 0x7f2cac13c630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:56:59.185 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:59.176+0000 7f2cabfff700 1 -- 172.21.15.59:0/1346787686 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2cac1459a0 msgr2=0x7f2cac13a5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:56:59.185 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:59.176+0000 7f2cabfff700 1 --2- 172.21.15.59:0/1346787686 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2cac1459a0 0x7f2cac13a5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:56:59.185 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:59.176+0000 7f2cabfff700 1 -- 172.21.15.59:0/1346787686 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2c940094a0 con 0x7f2cac06e580 2023-11-24T07:56:59.185 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:59.176+0000 7f2cabfff700 1 --2- 172.21.15.59:0/1346787686 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2cac06e580 0x7f2cac136190 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f2c94000c00 tx=0x7f2c94004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:56:59.185 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:59.176+0000 7f2ca8ff9700 1 -- 172.21.15.59:0/1346787686 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c940036a0 con 0x7f2cac06e580 2023-11-24T07:56:59.185 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:59.176+0000 7f2ca8ff9700 1 -- 172.21.15.59:0/1346787686 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2c9401f070 con 0x7f2cac06e580 2023-11-24T07:56:59.185 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:59.176+0000 7f2cb2a0d700 1 -- 172.21.15.59:0/1346787686 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2cac13d0f0 con 0x7f2cac06e580 2023-11-24T07:56:59.186 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:59.176+0000 7f2ca8ff9700 1 -- 172.21.15.59:0/1346787686 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c94003a00 con 0x7f2cac06e580 2023-11-24T07:56:59.186 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:59.176+0000 7f2cb2a0d700 1 -- 172.21.15.59:0/1346787686 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2cac143650 con 0x7f2cac06e580 2023-11-24T07:56:59.186 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:59.177+0000 7f2cb2a0d700 1 -- 172.21.15.59:0/1346787686 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2cac143ca0 con 0x7f2cac06e580 2023-11-24T07:56:59.186 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:59.178+0000 7f2ca8ff9700 1 -- 172.21.15.59:0/1346787686 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f2c94003b60 con 0x7f2cac06e580 2023-11-24T07:56:59.186 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:59.178+0000 7f2ca8ff9700 1 --2- 172.21.15.59:0/1346787686 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f2c8c06aed0 0x7f2c8c06d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:56:59.186 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:59.178+0000 7f2ca8ff9700 1 -- 172.21.15.59:0/1346787686 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f2c94021070 con 0x7f2cac06e580 2023-11-24T07:56:59.186 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:59.179+0000 7f2cab7fe700 1 --2- 172.21.15.59:0/1346787686 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f2c8c06aed0 0x7f2c8c06d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:56:59.186 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:59.179+0000 7f2cab7fe700 1 --2- 172.21.15.59:0/1346787686 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f2c8c06aed0 0x7f2c8c06d390 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f2cac15ca90 tx=0x7f2c9c008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:56:59.187 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:59.183+0000 7f2ca8ff9700 1 -- 172.21.15.59:0/1346787686 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f2c94055fe0 con 0x7f2cac06e580 2023-11-24T07:56:59.386 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:59.384+0000 7f2cb2a0d700 1 -- 172.21.15.59:0/1346787686 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f2cac1434b0 con 0x7f2cac06e580 2023-11-24T07:56:59.386 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:59.385+0000 7f2ca8ff9700 1 -- 172.21.15.59:0/1346787686 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v20) v1 ==== 93+0+10 (secure 0 0 0) 0x7f2cac1434b0 con 0x7f2cac06e580 2023-11-24T07:56:59.387 INFO:teuthology.orchestra.run.smithi059.stdout:pg_num: 1 2023-11-24T07:56:59.389 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:59.387+0000 7f2cb2a0d700 1 -- 172.21.15.59:0/1346787686 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f2c8c06aed0 msgr2=0x7f2c8c06d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:56:59.389 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:59.387+0000 7f2cb2a0d700 1 --2- 172.21.15.59:0/1346787686 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f2c8c06aed0 0x7f2c8c06d390 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f2cac15ca90 tx=0x7f2c9c008040 comp rx=0 tx=0).stop 2023-11-24T07:56:59.389 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:59.387+0000 7f2cb2a0d700 1 -- 172.21.15.59:0/1346787686 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2cac06e580 msgr2=0x7f2cac136190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:56:59.390 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:59.387+0000 7f2cb2a0d700 1 --2- 172.21.15.59:0/1346787686 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2cac06e580 0x7f2cac136190 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f2c94000c00 tx=0x7f2c94004750 comp rx=0 tx=0).stop 2023-11-24T07:56:59.390 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:59.387+0000 7f2cb2a0d700 1 -- 172.21.15.59:0/1346787686 shutdown_connections 2023-11-24T07:56:59.390 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:59.387+0000 7f2cb2a0d700 1 --2- 172.21.15.59:0/1346787686 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f2c8c06aed0 0x7f2c8c06d390 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:56:59.390 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:59.387+0000 7f2cb2a0d700 1 --2- 172.21.15.59:0/1346787686 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2cac1459a0 0x7f2cac13a5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:56:59.390 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:59.387+0000 7f2cb2a0d700 1 --2- 172.21.15.59:0/1346787686 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2cac06e580 0x7f2cac136190 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:56:59.390 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:59.387+0000 7f2cb2a0d700 1 --2- 172.21.15.59:0/1346787686 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f2cac142810 0x7f2cac13c630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:56:59.390 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:59.387+0000 7f2cb2a0d700 1 -- 172.21.15.59:0/1346787686 >> 172.21.15.59:0/1346787686 conn(0x7f2cac0b2740 msgr2=0x7f2cac141340 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:56:59.391 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:59.387+0000 7f2cb2a0d700 1 -- 172.21.15.59:0/1346787686 shutdown_connections 2023-11-24T07:56:59.391 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:59.387+0000 7f2cb2a0d700 1 -- 172.21.15.59:0/1346787686 wait complete. 2023-11-24T07:56:59.402 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-24T07:56:59.530 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:59.528+0000 7f97a4cf9700 1 Processor -- start 2023-11-24T07:56:59.530 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:59.528+0000 7f97a4cf9700 1 -- start start 2023-11-24T07:56:59.532 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:59.529+0000 7f97a4cf9700 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f97a015e090 0x7f97a006afd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:56:59.532 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:59.529+0000 7f97a4cf9700 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f97a006b510 0x7f97a0161cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:56:59.532 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:59.529+0000 7f97a4cf9700 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f97a015e470 con 0x7f97a0162200 2023-11-24T07:56:59.532 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:59.529+0000 7f97a4cf9700 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f97a00c6900 con 0x7f97a006b510 2023-11-24T07:56:59.532 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:59.529+0000 7f97a4cf9700 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f97a006b990 con 0x7f97a015e090 2023-11-24T07:56:59.532 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:59.529+0000 7f979ed9d700 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7f97a0162200 0x7f97a0071b40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:54334/0 (socket says 172.21.15.59:54334) 2023-11-24T07:56:59.532 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:59.529+0000 7f979ed9d700 1 -- 172.21.15.59:0/3035730594 learned_addr learned my addr 172.21.15.59:0/3035730594 (peer_addr_for_me v1:172.21.15.59:0/0) 2023-11-24T07:56:59.533 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:59.529+0000 7f979e59c700 1 --2- 172.21.15.59:0/3035730594 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f97a015e090 0x7f97a006afd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:56:59.533 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:59.529+0000 7f979dd9b700 1 --2- 172.21.15.59:0/3035730594 >> v2:172.21.15.76:3300/0 conn(0x7f97a006b510 0x7f97a0161cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:56:59.533 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:59.529+0000 7f979d59a700 1 -- 172.21.15.59:0/3035730594 <== mon.0 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 794463459 0 0) 0x7f97a015e470 con 0x7f97a0162200 2023-11-24T07:56:59.533 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:59.529+0000 7f979d59a700 1 -- 172.21.15.59:0/3035730594 --> v1:172.21.15.59:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f97880035b0 con 0x7f97a0162200 2023-11-24T07:56:59.533 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:59.530+0000 7f979e59c700 1 -- 172.21.15.59:0/3035730594 >> v2:172.21.15.76:3300/0 conn(0x7f97a006b510 msgr2=0x7f97a0161cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T07:56:59.533 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:59.530+0000 7f979e59c700 1 --2- 172.21.15.59:0/3035730594 >> v2:172.21.15.76:3300/0 conn(0x7f97a006b510 0x7f97a0161cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:56:59.533 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:59.530+0000 7f979e59c700 1 -- 172.21.15.59:0/3035730594 >> v1:172.21.15.59:6789/0 conn(0x7f97a0162200 legacy=0x7f97a0071b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:56:59.533 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:59.530+0000 7f979d59a700 1 -- 172.21.15.59:0/3035730594 <== mon.0 v1:172.21.15.59:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3538484897 0 0) 0x7f97880035b0 con 0x7f97a0162200 2023-11-24T07:56:59.533 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:59.530+0000 7f979e59c700 1 -- 172.21.15.59:0/3035730594 --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f97a01629f0 con 0x7f97a015e090 2023-11-24T07:56:59.533 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:59.530+0000 7f979e59c700 1 --2- 172.21.15.59:0/3035730594 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f97a015e090 0x7f97a006afd0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f9794009840 tx=0x7f9794009b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d7d90bf3f5debe6d server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:56:59.534 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:59.530+0000 7f979d59a700 1 -- 172.21.15.59:0/3035730594 <== mon.2 v2:172.21.15.76:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97980019f0 con 0x7f97a015e090 2023-11-24T07:56:59.534 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:59.530+0000 7f979d59a700 1 -- 172.21.15.59:0/3035730594 <== mon.2 v2:172.21.15.76:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9794004030 con 0x7f97a015e090 2023-11-24T07:56:59.534 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:59.530+0000 7f979d59a700 1 -- 172.21.15.59:0/3035730594 <== mon.2 v2:172.21.15.76:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97940043a0 con 0x7f97a015e090 2023-11-24T07:56:59.537 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:59.531+0000 7f97a4cf9700 1 -- 172.21.15.59:0/3035730594 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f97a015e090 msgr2=0x7f97a006afd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:56:59.537 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:59.531+0000 7f97a4cf9700 1 --2- 172.21.15.59:0/3035730594 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f97a015e090 0x7f97a006afd0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f9794009840 tx=0x7f9794009b50 comp rx=0 tx=0).stop 2023-11-24T07:56:59.537 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:59.531+0000 7f97a4cf9700 1 -- 172.21.15.59:0/3035730594 shutdown_connections 2023-11-24T07:56:59.537 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:59.531+0000 7f97a4cf9700 1 --2- 172.21.15.59:0/3035730594 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f97a015e090 0x7f97a006afd0 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:56:59.538 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:59.531+0000 7f97a4cf9700 1 --2- 172.21.15.59:0/3035730594 >> v2:172.21.15.76:3300/0 conn(0x7f97a006b510 0x7f97a0161cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:56:59.538 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:59.531+0000 7f97a4cf9700 1 -- 172.21.15.59:0/3035730594 >> 172.21.15.59:0/3035730594 conn(0x7f97a00d0b10 msgr2=0x7f97a00d0f20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:56:59.538 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:59.531+0000 7f97a4cf9700 1 -- 172.21.15.59:0/3035730594 shutdown_connections 2023-11-24T07:56:59.538 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:59.531+0000 7f97a4cf9700 1 -- 172.21.15.59:0/3035730594 wait complete. 2023-11-24T07:56:59.538 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:59.532+0000 7f97a4cf9700 1 Processor -- start 2023-11-24T07:56:59.538 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:59.532+0000 7f97a4cf9700 1 -- start start 2023-11-24T07:56:59.538 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:59.533+0000 7f97a4cf9700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f97a006b510 0x7f97a01525a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:56:59.538 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:59.533+0000 7f97a4cf9700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f97a015e090 0x7f97a00b4ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:56:59.539 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:59.533+0000 7f97a4cf9700 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f97a0162200 0x7f97a00b50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:56:59.539 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:59.533+0000 7f97a4cf9700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97a00b5710 con 0x7f97a006b510 2023-11-24T07:56:59.539 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:59.533+0000 7f97a4cf9700 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97a00b59f0 con 0x7f97a015e090 2023-11-24T07:56:59.539 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:59.533+0000 7f97a4cf9700 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f97a00b5cd0 con 0x7f97a0162200 2023-11-24T07:56:59.539 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:59.533+0000 7f979e59c700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f97a006b510 0x7f97a01525a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:56:59.539 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:59.533+0000 7f979e59c700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f97a006b510 0x7f97a01525a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:50862/0 (socket says 172.21.15.59:50862) 2023-11-24T07:56:59.539 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:59.533+0000 7f979e59c700 1 -- 172.21.15.59:0/2764782864 learned_addr learned my addr 172.21.15.59:0/2764782864 (peer_addr_for_me v2:172.21.15.59:0/0) 2023-11-24T07:56:59.539 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:59.533+0000 7f979ed9d700 1 --2- 172.21.15.59:0/2764782864 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f97a0162200 0x7f97a00b50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:56:59.540 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:59.533+0000 7f979dd9b700 1 --2- 172.21.15.59:0/2764782864 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f97a015e090 0x7f97a00b4ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:56:59.540 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:59.533+0000 7f979e59c700 1 -- 172.21.15.59:0/2764782864 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f97a0162200 msgr2=0x7f97a00b50e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:56:59.540 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:59.533+0000 7f979e59c700 1 --2- 172.21.15.59:0/2764782864 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f97a0162200 0x7f97a00b50e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:56:59.540 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:59.533+0000 7f979e59c700 1 -- 172.21.15.59:0/2764782864 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f97a015e090 msgr2=0x7f97a00b4ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:56:59.540 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:59.533+0000 7f979e59c700 1 --2- 172.21.15.59:0/2764782864 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f97a015e090 0x7f97a00b4ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:56:59.540 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:59.533+0000 7f979e59c700 1 -- 172.21.15.59:0/2764782864 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f97940094a0 con 0x7f97a006b510 2023-11-24T07:56:59.540 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:59.533+0000 7f979ed9d700 1 --2- 172.21.15.59:0/2764782864 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f97a0162200 0x7f97a00b50e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-24T07:56:59.540 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:59.534+0000 7f979e59c700 1 --2- 172.21.15.59:0/2764782864 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f97a006b510 0x7f97a01525a0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f979400a010 tx=0x7f9794004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:56:59.540 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:59.534+0000 7f978effd700 1 -- 172.21.15.59:0/2764782864 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9794004590 con 0x7f97a006b510 2023-11-24T07:56:59.540 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:59.534+0000 7f978effd700 1 -- 172.21.15.59:0/2764782864 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f979401f070 con 0x7f97a006b510 2023-11-24T07:56:59.540 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:59.534+0000 7f97a4cf9700 1 -- 172.21.15.59:0/2764782864 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f97a00b5fb0 con 0x7f97a006b510 2023-11-24T07:56:59.540 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:59.534+0000 7f978effd700 1 -- 172.21.15.59:0/2764782864 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9794011850 con 0x7f97a006b510 2023-11-24T07:56:59.541 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:59.534+0000 7f97a4cf9700 1 -- 172.21.15.59:0/2764782864 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f97a0161370 con 0x7f97a006b510 2023-11-24T07:56:59.541 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:59.535+0000 7f97a4cf9700 1 -- 172.21.15.59:0/2764782864 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f97a00b6600 con 0x7f97a006b510 2023-11-24T07:56:59.541 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:59.535+0000 7f978effd700 1 -- 172.21.15.59:0/2764782864 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f97940119b0 con 0x7f97a006b510 2023-11-24T07:56:59.541 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:59.536+0000 7f978effd700 1 --2- 172.21.15.59:0/2764782864 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f978006aed0 0x7f978006d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:56:59.541 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:59.536+0000 7f978effd700 1 -- 172.21.15.59:0/2764782864 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f9794021070 con 0x7f97a006b510 2023-11-24T07:56:59.541 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:59.536+0000 7f979dd9b700 1 --2- 172.21.15.59:0/2764782864 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f978006aed0 0x7f978006d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:56:59.541 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:59.536+0000 7f979dd9b700 1 --2- 172.21.15.59:0/2764782864 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f978006aed0 0x7f978006d390 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f97a01f6f20 tx=0x7f9790008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:56:59.542 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:59.540+0000 7f978effd700 1 -- 172.21.15.59:0/2764782864 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f9794056080 con 0x7f97a006b510 2023-11-24T07:56:59.744 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:59.743+0000 7f97a4cf9700 1 -- 172.21.15.59:0/2764782864 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f97a0075480 con 0x7f97a006b510 2023-11-24T07:56:59.745 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:59.743+0000 7f978effd700 1 -- 172.21.15.59:0/2764782864 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v20) v1 ==== 104+0+11 (secure 0 0 0) 0x7f97940596a0 con 0x7f97a006b510 2023-11-24T07:56:59.745 INFO:teuthology.orchestra.run.smithi059.stdout:pg_num: 32 2023-11-24T07:56:59.747 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:59.745+0000 7f97a4cf9700 1 -- 172.21.15.59:0/2764782864 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f978006aed0 msgr2=0x7f978006d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:56:59.747 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:59.745+0000 7f97a4cf9700 1 --2- 172.21.15.59:0/2764782864 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f978006aed0 0x7f978006d390 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f97a01f6f20 tx=0x7f9790008040 comp rx=0 tx=0).stop 2023-11-24T07:56:59.748 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:59.745+0000 7f97a4cf9700 1 -- 172.21.15.59:0/2764782864 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f97a006b510 msgr2=0x7f97a01525a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:56:59.748 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:59.745+0000 7f97a4cf9700 1 --2- 172.21.15.59:0/2764782864 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f97a006b510 0x7f97a01525a0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f979400a010 tx=0x7f9794004750 comp rx=0 tx=0).stop 2023-11-24T07:56:59.748 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:59.745+0000 7f97a4cf9700 1 -- 172.21.15.59:0/2764782864 shutdown_connections 2023-11-24T07:56:59.748 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:59.745+0000 7f97a4cf9700 1 --2- 172.21.15.59:0/2764782864 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f978006aed0 0x7f978006d390 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:56:59.748 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:59.745+0000 7f97a4cf9700 1 --2- 172.21.15.59:0/2764782864 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f97a0162200 0x7f97a00b50e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:56:59.748 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:59.745+0000 7f97a4cf9700 1 --2- 172.21.15.59:0/2764782864 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f97a006b510 0x7f97a01525a0 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:56:59.748 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:59.745+0000 7f97a4cf9700 1 --2- 172.21.15.59:0/2764782864 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f97a015e090 0x7f97a00b4ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:56:59.748 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:59.745+0000 7f97a4cf9700 1 -- 172.21.15.59:0/2764782864 >> 172.21.15.59:0/2764782864 conn(0x7f97a00d0b10 msgr2=0x7f97a00aec70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:56:59.748 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:59.745+0000 7f97a4cf9700 1 -- 172.21.15.59:0/2764782864 shutdown_connections 2023-11-24T07:56:59.748 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:59.745+0000 7f97a4cf9700 1 -- 172.21.15.59:0/2764782864 wait complete. 2023-11-24T07:56:59.759 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-24T07:56:59.887 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:59.885+0000 7efca56c3700 1 Processor -- start 2023-11-24T07:56:59.887 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:59.885+0000 7efca56c3700 1 -- start start 2023-11-24T07:56:59.889 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:59.886+0000 7efca56c3700 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7efca006b620 0x7efca00b3a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:56:59.889 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:59.886+0000 7efca56c3700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7efca006ee60 0x7efca00b3f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:56:59.889 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:59.886+0000 7efca56c3700 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efca0071f50 con 0x7efca00b5ab0 2023-11-24T07:56:59.890 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:59.886+0000 7efca56c3700 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efca0072230 con 0x7efca006ee60 2023-11-24T07:56:59.890 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:59.886+0000 7efca56c3700 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efca0072510 con 0x7efca006b620 2023-11-24T07:56:59.890 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:59.886+0000 7efc9f7fe700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7efca006ee60 0x7efca00b3f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:56:59.890 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:59.886+0000 7efc9f7fe700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7efca006ee60 0x7efca00b3f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:50874/0 (socket says 172.21.15.59:50874) 2023-11-24T07:56:59.890 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:59.886+0000 7efc9f7fe700 1 -- 172.21.15.59:0/1278447383 learned_addr learned my addr 172.21.15.59:0/1278447383 (peer_addr_for_me v2:172.21.15.59:0/0) 2023-11-24T07:56:59.890 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:59.886+0000 7efc9effd700 1 --2- 172.21.15.59:0/1278447383 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7efca006b620 0x7efca00b3a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:56:59.890 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:59.887+0000 7efc9f7fe700 1 -- 172.21.15.59:0/1278447383 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7efca006b620 msgr2=0x7efca00b3a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T07:56:59.890 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:59.887+0000 7efc9f7fe700 1 --2- 172.21.15.59:0/1278447383 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7efca006b620 0x7efca00b3a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:56:59.890 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:59.887+0000 7efc9f7fe700 1 -- 172.21.15.59:0/1278447383 >> v1:172.21.15.76:6789/0 conn(0x7efca00b5ab0 legacy=0x7efca0161ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:56:59.890 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:59.887+0000 7efc9f7fe700 1 -- 172.21.15.59:0/1278447383 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efca00727f0 con 0x7efca006ee60 2023-11-24T07:56:59.890 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:59.887+0000 7efc9f7fe700 1 --2- 172.21.15.59:0/1278447383 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7efca006ee60 0x7efca00b3f80 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7efc98009e50 tx=0x7efc9800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=502b914bf7f33ccd server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:56:59.890 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:59.887+0000 7efc9dffb700 1 -- 172.21.15.59:0/1278447383 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc9800e070 con 0x7efca006ee60 2023-11-24T07:56:59.891 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:59.887+0000 7efc9dffb700 1 -- 172.21.15.59:0/1278447383 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efc9800b910 con 0x7efca006ee60 2023-11-24T07:56:59.891 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:59.887+0000 7efc9dffb700 1 -- 172.21.15.59:0/1278447383 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc9800bcc0 con 0x7efca006ee60 2023-11-24T07:56:59.891 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:59.887+0000 7efca56c3700 1 -- 172.21.15.59:0/1278447383 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7efca006ee60 msgr2=0x7efca00b3f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:56:59.891 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:59.887+0000 7efca56c3700 1 --2- 172.21.15.59:0/1278447383 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7efca006ee60 0x7efca00b3f80 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7efc98009e50 tx=0x7efc9800b330 comp rx=0 tx=0).stop 2023-11-24T07:56:59.891 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:59.887+0000 7efca56c3700 1 -- 172.21.15.59:0/1278447383 shutdown_connections 2023-11-24T07:56:59.891 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:59.887+0000 7efca56c3700 1 --2- 172.21.15.59:0/1278447383 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7efca006b620 0x7efca00b3a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:56:59.891 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:59.887+0000 7efca56c3700 1 --2- 172.21.15.59:0/1278447383 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7efca006ee60 0x7efca00b3f80 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:56:59.891 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:59.887+0000 7efca56c3700 1 -- 172.21.15.59:0/1278447383 >> 172.21.15.59:0/1278447383 conn(0x7efca00b2740 msgr2=0x7efca00b2b20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:56:59.891 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:59.888+0000 7efca56c3700 1 -- 172.21.15.59:0/1278447383 shutdown_connections 2023-11-24T07:56:59.891 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:59.888+0000 7efca56c3700 1 -- 172.21.15.59:0/1278447383 wait complete. 2023-11-24T07:56:59.894 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:59.889+0000 7efca56c3700 1 Processor -- start 2023-11-24T07:56:59.894 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:59.889+0000 7efca56c3700 1 -- start start 2023-11-24T07:56:59.894 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:59.890+0000 7efca56c3700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7efca006b620 0x7efca00cc040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:56:59.894 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:59.890+0000 7efca56c3700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7efca006ee60 0x7efca00c7b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:56:59.894 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:59.890+0000 7efca56c3700 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7efca00b5ab0 0x7efca01e5ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:56:59.894 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:59.890+0000 7efca56c3700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efca0134c30 con 0x7efca006ee60 2023-11-24T07:56:59.894 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:59.890+0000 7efca56c3700 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efca00c8b00 con 0x7efca006b620 2023-11-24T07:56:59.894 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:59.890+0000 7efca56c3700 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efca00c8540 con 0x7efca00b5ab0 2023-11-24T07:56:59.894 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:59.890+0000 7efc9e7fc700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7efca006ee60 0x7efca00c7b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:56:59.895 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:59.890+0000 7efc9e7fc700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7efca006ee60 0x7efca00c7b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:50878/0 (socket says 172.21.15.59:50878) 2023-11-24T07:56:59.895 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:59.890+0000 7efc9e7fc700 1 -- 172.21.15.59:0/349261451 learned_addr learned my addr 172.21.15.59:0/349261451 (peer_addr_for_me v2:172.21.15.59:0/0) 2023-11-24T07:56:59.895 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:59.890+0000 7efc9effd700 1 --2- 172.21.15.59:0/349261451 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7efca006b620 0x7efca00cc040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:56:59.895 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:59.890+0000 7efc9f7fe700 1 --2- 172.21.15.59:0/349261451 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7efca00b5ab0 0x7efca01e5ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:56:59.895 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:59.890+0000 7efc9e7fc700 1 -- 172.21.15.59:0/349261451 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7efca00b5ab0 msgr2=0x7efca01e5ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:56:59.895 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:59.890+0000 7efc9e7fc700 1 --2- 172.21.15.59:0/349261451 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7efca00b5ab0 0x7efca01e5ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:56:59.895 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:59.890+0000 7efc9e7fc700 1 -- 172.21.15.59:0/349261451 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7efca006b620 msgr2=0x7efca00cc040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:56:59.895 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:59.890+0000 7efc9e7fc700 1 --2- 172.21.15.59:0/349261451 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7efca006b620 0x7efca00cc040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:56:59.895 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:59.890+0000 7efc9e7fc700 1 -- 172.21.15.59:0/349261451 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efc98009b30 con 0x7efca006ee60 2023-11-24T07:56:59.895 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:59.890+0000 7efc9e7fc700 1 --2- 172.21.15.59:0/349261451 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7efca006ee60 0x7efca00c7b80 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7efc9000c450 tx=0x7efc9000c760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:56:59.895 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:59.890+0000 7efc877fe700 1 -- 172.21.15.59:0/349261451 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc900052d0 con 0x7efca006ee60 2023-11-24T07:56:59.895 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:59.890+0000 7efc877fe700 1 -- 172.21.15.59:0/349261451 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efc90005430 con 0x7efca006ee60 2023-11-24T07:56:59.895 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:59.890+0000 7efc877fe700 1 -- 172.21.15.59:0/349261451 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc900038c0 con 0x7efca006ee60 2023-11-24T07:56:59.896 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:59.890+0000 7efca56c3700 1 -- 172.21.15.59:0/349261451 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efca00c8820 con 0x7efca006ee60 2023-11-24T07:56:59.896 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:59.891+0000 7efca56c3700 1 -- 172.21.15.59:0/349261451 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efca0146f20 con 0x7efca006ee60 2023-11-24T07:56:59.896 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:59.892+0000 7efca56c3700 1 -- 172.21.15.59:0/349261451 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efca01470b0 con 0x7efca006ee60 2023-11-24T07:56:59.896 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:59.892+0000 7efc877fe700 1 -- 172.21.15.59:0/349261451 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7efc90003a20 con 0x7efca006ee60 2023-11-24T07:56:59.896 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:59.892+0000 7efc877fe700 1 --2- 172.21.15.59:0/349261451 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7efc8c06aed0 0x7efc8c06d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:56:59.896 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:59.892+0000 7efc877fe700 1 -- 172.21.15.59:0/349261451 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7efc90088810 con 0x7efca006ee60 2023-11-24T07:56:59.896 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:59.892+0000 7efc9effd700 1 --2- 172.21.15.59:0/349261451 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7efc8c06aed0 0x7efc8c06d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:56:59.896 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:59.893+0000 7efc9effd700 1 --2- 172.21.15.59:0/349261451 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7efc8c06aed0 0x7efc8c06d390 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7efc94009be0 tx=0x7efc94009210 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:56:59.898 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:56:59.897+0000 7efc877fe700 1 -- 172.21.15.59:0/349261451 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7efc90053950 con 0x7efca006ee60 2023-11-24T07:57:00.099 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:00.097+0000 7efca56c3700 1 -- 172.21.15.59:0/349261451 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7efca0134d70 con 0x7efca006ee60 2023-11-24T07:57:00.100 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:00.098+0000 7efc877fe700 1 -- 172.21.15.59:0/349261451 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v20) v1 ==== 100+0+11 (secure 0 0 0) 0x7efca0134d70 con 0x7efca006ee60 2023-11-24T07:57:00.100 INFO:teuthology.orchestra.run.smithi059.stdout:pg_num: 64 2023-11-24T07:57:00.102 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:00.100+0000 7efca56c3700 1 -- 172.21.15.59:0/349261451 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7efc8c06aed0 msgr2=0x7efc8c06d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:57:00.102 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:00.100+0000 7efca56c3700 1 --2- 172.21.15.59:0/349261451 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7efc8c06aed0 0x7efc8c06d390 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7efc94009be0 tx=0x7efc94009210 comp rx=0 tx=0).stop 2023-11-24T07:57:00.103 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:00.100+0000 7efca56c3700 1 -- 172.21.15.59:0/349261451 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7efca006ee60 msgr2=0x7efca00c7b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:57:00.103 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:00.100+0000 7efca56c3700 1 --2- 172.21.15.59:0/349261451 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7efca006ee60 0x7efca00c7b80 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7efc9000c450 tx=0x7efc9000c760 comp rx=0 tx=0).stop 2023-11-24T07:57:00.103 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:00.100+0000 7efca56c3700 1 -- 172.21.15.59:0/349261451 shutdown_connections 2023-11-24T07:57:00.103 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:00.100+0000 7efca56c3700 1 --2- 172.21.15.59:0/349261451 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7efc8c06aed0 0x7efc8c06d390 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:00.103 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:00.100+0000 7efca56c3700 1 --2- 172.21.15.59:0/349261451 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7efca00b5ab0 0x7efca01e5ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:00.103 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:00.100+0000 7efca56c3700 1 --2- 172.21.15.59:0/349261451 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7efca006b620 0x7efca00cc040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:00.103 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:00.100+0000 7efca56c3700 1 --2- 172.21.15.59:0/349261451 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7efca006ee60 0x7efca00c7b80 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:00.103 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:00.100+0000 7efca56c3700 1 -- 172.21.15.59:0/349261451 >> 172.21.15.59:0/349261451 conn(0x7efca00b2740 msgr2=0x7efca00b0840 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:57:00.103 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:00.100+0000 7efca56c3700 1 -- 172.21.15.59:0/349261451 shutdown_connections 2023-11-24T07:57:00.104 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:00.100+0000 7efca56c3700 1 -- 172.21.15.59:0/349261451 wait complete. 2023-11-24T07:57:00.105 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-24T07:57:00.107 DEBUG:teuthology.orchestra.run.smithi076:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-24T07:57:00.114 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_volumes.TestSubvolumeGroupSnapshots.test_subvolume_group_snapshot_unsupported_status' 2023-11-24T07:57:00.279 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:00.277+0000 7f33e75b0700 1 Processor -- start 2023-11-24T07:57:00.279 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:00.278+0000 7f33e75b0700 1 -- start start 2023-11-24T07:57:00.281 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:00.278+0000 7f33e75b0700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f33e015c010 0x7f33e00cd2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:57:00.282 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:00.278+0000 7f33e75b0700 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f33e006f610 con 0x7f33e00cf120 2023-11-24T07:57:00.282 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:00.278+0000 7f33e75b0700 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f33e00cd7f0 con 0x7f33e00ce560 2023-11-24T07:57:00.282 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:00.278+0000 7f33e75b0700 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f33e00cdac0 con 0x7f33e015c010 2023-11-24T07:57:00.282 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:00.279+0000 7f33e534c700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f33e015c010 0x7f33e00cd2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:57:00.282 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:00.279+0000 7f33e5b4d700 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7f33e00cf120 0x7f33e006faa0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:54360/0 (socket says 172.21.15.59:54360) 2023-11-24T07:57:00.282 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:00.279+0000 7f33e5b4d700 1 -- 172.21.15.59:0/1361349801 learned_addr learned my addr 172.21.15.59:0/1361349801 (peer_addr_for_me v1:172.21.15.59:0/0) 2023-11-24T07:57:00.282 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:00.279+0000 7f33e534c700 1 -- 172.21.15.59:0/1361349801 >> v1:172.21.15.76:6789/0 conn(0x7f33e00ce560 legacy=0x7f33e00cea10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:57:00.282 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:00.279+0000 7f33e534c700 1 -- 172.21.15.59:0/1361349801 >> v1:172.21.15.59:6789/0 conn(0x7f33e00cf120 legacy=0x7f33e006faa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:57:00.282 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:00.279+0000 7f33e534c700 1 -- 172.21.15.59:0/1361349801 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f33e006ece0 con 0x7f33e015c010 2023-11-24T07:57:00.282 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:00.279+0000 7f33e534c700 1 --2- 172.21.15.59:0/1361349801 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f33e015c010 0x7f33e00cd2b0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f33dc009840 tx=0x7f33dc009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6e21bab6184cd3d2 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:57:00.283 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:00.279+0000 7f33d7fff700 1 -- 172.21.15.59:0/1361349801 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33dc011830 con 0x7f33e015c010 2023-11-24T07:57:00.283 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:00.279+0000 7f33d7fff700 1 -- 172.21.15.59:0/1361349801 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f33dc011990 con 0x7f33e015c010 2023-11-24T07:57:00.283 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:00.279+0000 7f33d7fff700 1 -- 172.21.15.59:0/1361349801 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33dc011d20 con 0x7f33e015c010 2023-11-24T07:57:00.283 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:00.280+0000 7f33e75b0700 1 -- 172.21.15.59:0/1361349801 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f33e015c010 msgr2=0x7f33e00cd2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:57:00.283 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:00.280+0000 7f33e75b0700 1 --2- 172.21.15.59:0/1361349801 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f33e015c010 0x7f33e00cd2b0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f33dc009840 tx=0x7f33dc009b50 comp rx=0 tx=0).stop 2023-11-24T07:57:00.283 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:00.280+0000 7f33e75b0700 1 -- 172.21.15.59:0/1361349801 shutdown_connections 2023-11-24T07:57:00.283 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:00.280+0000 7f33e75b0700 1 --2- 172.21.15.59:0/1361349801 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f33e015c010 0x7f33e00cd2b0 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:00.283 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:00.280+0000 7f33e75b0700 1 -- 172.21.15.59:0/1361349801 >> 172.21.15.59:0/1361349801 conn(0x7f33e00b3790 msgr2=0x7f33e00b3ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:57:00.283 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:00.280+0000 7f33e75b0700 1 -- 172.21.15.59:0/1361349801 shutdown_connections 2023-11-24T07:57:00.283 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:00.280+0000 7f33e75b0700 1 -- 172.21.15.59:0/1361349801 wait complete. 2023-11-24T07:57:00.286 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:00.281+0000 7f33e75b0700 1 Processor -- start 2023-11-24T07:57:00.286 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:00.281+0000 7f33e75b0700 1 -- start start 2023-11-24T07:57:00.286 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:00.282+0000 7f33e75b0700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f33e00ce560 0x7f33e0158d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:57:00.286 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:00.282+0000 7f33e75b0700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f33e00cf120 0x7f33e015e030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:57:00.286 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:00.282+0000 7f33e75b0700 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f33e015c010 0x7f33e00b4ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:57:00.286 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:00.282+0000 7f33e75b0700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33e01596a0 con 0x7f33e00cf120 2023-11-24T07:57:00.286 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:00.282+0000 7f33e75b0700 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33e00b6470 con 0x7f33e00ce560 2023-11-24T07:57:00.286 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:00.282+0000 7f33e75b0700 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f33e00b5bd0 con 0x7f33e015c010 2023-11-24T07:57:00.286 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:00.282+0000 7f33e4b4b700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f33e00cf120 0x7f33e015e030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:57:00.286 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:00.282+0000 7f33e4b4b700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f33e00cf120 0x7f33e015e030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:50890/0 (socket says 172.21.15.59:50890) 2023-11-24T07:57:00.286 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:00.282+0000 7f33e4b4b700 1 -- 172.21.15.59:0/650051795 learned_addr learned my addr 172.21.15.59:0/650051795 (peer_addr_for_me v2:172.21.15.59:0/0) 2023-11-24T07:57:00.287 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:00.282+0000 7f33e534c700 1 --2- 172.21.15.59:0/650051795 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f33e00ce560 0x7f33e0158d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:57:00.287 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:00.282+0000 7f33e5b4d700 1 --2- 172.21.15.59:0/650051795 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f33e015c010 0x7f33e00b4ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:57:00.287 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:00.282+0000 7f33e4b4b700 1 -- 172.21.15.59:0/650051795 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f33e015c010 msgr2=0x7f33e00b4ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:57:00.287 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:00.282+0000 7f33e4b4b700 1 --2- 172.21.15.59:0/650051795 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f33e015c010 0x7f33e00b4ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:00.287 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:00.282+0000 7f33e4b4b700 1 -- 172.21.15.59:0/650051795 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f33e00ce560 msgr2=0x7f33e0158d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:57:00.287 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:00.282+0000 7f33e4b4b700 1 --2- 172.21.15.59:0/650051795 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f33e00ce560 0x7f33e0158d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:00.287 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:00.282+0000 7f33e4b4b700 1 -- 172.21.15.59:0/650051795 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f33dc0094a0 con 0x7f33e00cf120 2023-11-24T07:57:00.287 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:00.282+0000 7f33e534c700 1 --2- 172.21.15.59:0/650051795 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f33e00ce560 0x7f33e0158d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-24T07:57:00.287 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:00.283+0000 7f33e4b4b700 1 --2- 172.21.15.59:0/650051795 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f33e00cf120 0x7f33e015e030 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f33d000c450 tx=0x7f33d000c810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:57:00.288 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:00.283+0000 7f33d5ffb700 1 -- 172.21.15.59:0/650051795 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33d00052d0 con 0x7f33e00cf120 2023-11-24T07:57:00.288 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:00.283+0000 7f33d5ffb700 1 -- 172.21.15.59:0/650051795 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f33d0005430 con 0x7f33e00cf120 2023-11-24T07:57:00.288 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:00.283+0000 7f33e75b0700 1 -- 172.21.15.59:0/650051795 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f33e00b5eb0 con 0x7f33e00cf120 2023-11-24T07:57:00.288 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:00.283+0000 7f33d5ffb700 1 -- 172.21.15.59:0/650051795 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33d00038c0 con 0x7f33e00cf120 2023-11-24T07:57:00.288 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:00.283+0000 7f33e75b0700 1 -- 172.21.15.59:0/650051795 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f33e00b49b0 con 0x7f33e00cf120 2023-11-24T07:57:00.288 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:00.284+0000 7f33e75b0700 1 -- 172.21.15.59:0/650051795 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f33e00b4b70 con 0x7f33e00cf120 2023-11-24T07:57:00.288 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:00.284+0000 7f33d5ffb700 1 -- 172.21.15.59:0/650051795 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f33d0003a20 con 0x7f33e00cf120 2023-11-24T07:57:00.289 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:00.285+0000 7f33d5ffb700 1 --2- 172.21.15.59:0/650051795 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f33c406aed0 0x7f33c406d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:57:00.289 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:00.285+0000 7f33d5ffb700 1 -- 172.21.15.59:0/650051795 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f33d0088930 con 0x7f33e00cf120 2023-11-24T07:57:00.289 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:00.285+0000 7f33e534c700 1 --2- 172.21.15.59:0/650051795 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f33c406aed0 0x7f33c406d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:57:00.289 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:00.285+0000 7f33e534c700 1 --2- 172.21.15.59:0/650051795 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f33c406aed0 0x7f33c406d390 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f33dc0090b0 tx=0x7f33dc008030 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:57:00.290 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:00.289+0000 7f33d5ffb700 1 -- 172.21.15.59:0/650051795 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f33d00539e0 con 0x7f33e00cf120 2023-11-24T07:57:00.576 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:00.574+0000 7f33e75b0700 1 -- 172.21.15.59:0/650051795 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_volumes.TestSubvolumeGroupSnapshots.test_subvolume_group_snapshot_unsupported_status"]} v 0) v1 -- 0x7f33e00b4810 con 0x7f33e00cf120 2023-11-24T07:57:01.402 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:01.401+0000 7f33d5ffb700 1 -- 172.21.15.59:0/650051795 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_volumes.TestSubvolumeGroupSnapshots.test_subvolume_group_snapshot_unsupported_status"]}]=0 v49) v1 ==== 184+0+0 (secure 0 0 0) 0x7f33e00b4810 con 0x7f33e00cf120 2023-11-24T07:57:01.406 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:01.404+0000 7f33e75b0700 1 -- 172.21.15.59:0/650051795 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f33c406aed0 msgr2=0x7f33c406d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:57:01.407 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:01.404+0000 7f33e75b0700 1 --2- 172.21.15.59:0/650051795 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f33c406aed0 0x7f33c406d390 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f33dc0090b0 tx=0x7f33dc008030 comp rx=0 tx=0).stop 2023-11-24T07:57:01.407 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:01.405+0000 7f33e75b0700 1 -- 172.21.15.59:0/650051795 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f33e00cf120 msgr2=0x7f33e015e030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:57:01.407 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:01.405+0000 7f33e75b0700 1 --2- 172.21.15.59:0/650051795 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f33e00cf120 0x7f33e015e030 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f33d000c450 tx=0x7f33d000c810 comp rx=0 tx=0).stop 2023-11-24T07:57:01.407 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:01.405+0000 7f33e75b0700 1 -- 172.21.15.59:0/650051795 shutdown_connections 2023-11-24T07:57:01.407 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:01.405+0000 7f33e75b0700 1 --2- 172.21.15.59:0/650051795 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f33c406aed0 0x7f33c406d390 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:01.407 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:01.405+0000 7f33e75b0700 1 --2- 172.21.15.59:0/650051795 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f33e015c010 0x7f33e00b4ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:01.407 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:01.405+0000 7f33e75b0700 1 --2- 172.21.15.59:0/650051795 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f33e00ce560 0x7f33e0158d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:01.407 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:01.405+0000 7f33e75b0700 1 --2- 172.21.15.59:0/650051795 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f33e00cf120 0x7f33e015e030 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:01.407 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:01.405+0000 7f33e75b0700 1 -- 172.21.15.59:0/650051795 >> 172.21.15.59:0/650051795 conn(0x7f33e00b3790 msgr2=0x7f33e00cea10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:57:01.407 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:01.405+0000 7f33e75b0700 1 -- 172.21.15.59:0/650051795 shutdown_connections 2023-11-24T07:57:01.408 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:01.405+0000 7f33e75b0700 1 -- 172.21.15.59:0/650051795 wait complete. 2023-11-24T07:57:01.423 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2023-11-24T07:57:01.562 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:01.560+0000 7fca440d4700 1 Processor -- start 2023-11-24T07:57:01.563 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:01.560+0000 7fca440d4700 1 -- start start 2023-11-24T07:57:01.566 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:01.561+0000 7fca440d4700 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fca3c15e1e0 0x7fca3c0b12a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:57:01.566 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:01.561+0000 7fca440d4700 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fca3c06b400 0x7fca3c0b17e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:57:01.566 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:01.561+0000 7fca440d4700 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fca3c0b1d20 con 0x7fca3c0b5e40 2023-11-24T07:57:01.567 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:01.561+0000 7fca440d4700 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fca3c06bb80 con 0x7fca3c06b400 2023-11-24T07:57:01.567 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:01.561+0000 7fca440d4700 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fca3c06be20 con 0x7fca3c15e1e0 2023-11-24T07:57:01.567 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:01.561+0000 7fca42671700 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7fca3c0b5e40 0x7fca3c0726a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:54374/0 (socket says 172.21.15.59:54374) 2023-11-24T07:57:01.567 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:01.561+0000 7fca42671700 1 -- 172.21.15.59:0/2260412844 learned_addr learned my addr 172.21.15.59:0/2260412844 (peer_addr_for_me v1:172.21.15.59:0/0) 2023-11-24T07:57:01.567 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:01.561+0000 7fca41e70700 1 --2- 172.21.15.59:0/2260412844 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fca3c15e1e0 0x7fca3c0b12a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:57:01.567 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:01.561+0000 7fca4166f700 1 --2- 172.21.15.59:0/2260412844 >> v2:172.21.15.76:3300/0 conn(0x7fca3c06b400 0x7fca3c0b17e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:57:01.567 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:01.562+0000 7fca40e6e700 1 -- 172.21.15.59:0/2260412844 <== mon.0 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3112990390 0 0) 0x7fca3c0b1d20 con 0x7fca3c0b5e40 2023-11-24T07:57:01.567 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:01.562+0000 7fca40e6e700 1 -- 172.21.15.59:0/2260412844 --> v1:172.21.15.59:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fca280035b0 con 0x7fca3c0b5e40 2023-11-24T07:57:01.567 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:01.562+0000 7fca40e6e700 1 -- 172.21.15.59:0/2260412844 <== mon.0 v1:172.21.15.59:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1840636425 0 0) 0x7fca280035b0 con 0x7fca3c0b5e40 2023-11-24T07:57:01.567 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:01.562+0000 7fca40e6e700 1 -- 172.21.15.59:0/2260412844 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fca3c15e1e0 msgr2=0x7fca3c0b12a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T07:57:01.567 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:01.562+0000 7fca40e6e700 1 --2- 172.21.15.59:0/2260412844 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fca3c15e1e0 0x7fca3c0b12a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:01.568 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:01.562+0000 7fca40e6e700 1 -- 172.21.15.59:0/2260412844 >> v2:172.21.15.76:3300/0 conn(0x7fca3c06b400 msgr2=0x7fca3c0b17e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T07:57:01.568 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:01.562+0000 7fca40e6e700 1 --2- 172.21.15.59:0/2260412844 >> v2:172.21.15.76:3300/0 conn(0x7fca3c06b400 0x7fca3c0b17e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:01.568 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:01.562+0000 7fca40e6e700 1 -- 172.21.15.59:0/2260412844 --> v1:172.21.15.59:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fca3c06c100 con 0x7fca3c0b5e40 2023-11-24T07:57:01.568 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:01.562+0000 7fca40e6e700 1 -- 172.21.15.59:0/2260412844 <== mon.0 v1:172.21.15.59:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1312333566 0 0) 0x7fca340019f0 con 0x7fca3c0b5e40 2023-11-24T07:57:01.568 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:01.562+0000 7fca41e70700 1 --2- 172.21.15.59:0/2260412844 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fca3c15e1e0 0x7fca3c0b12a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-24T07:57:01.568 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:01.562+0000 7fca40e6e700 1 -- 172.21.15.59:0/2260412844 >> v1:172.21.15.59:6789/0 conn(0x7fca3c0b5e40 legacy=0x7fca3c0726a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:57:01.568 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:01.562+0000 7fca40e6e700 1 --2- 172.21.15.59:0/2260412844 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fca28003a00 0x7fca28003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:57:01.568 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:01.562+0000 7fca40e6e700 1 --2- 172.21.15.59:0/2260412844 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fca28004850 0x7fca28004cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:57:01.568 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:01.562+0000 7fca40e6e700 1 --2- 172.21.15.59:0/2260412844 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fca3c15e1e0 0x7fca28005210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:57:01.569 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:01.562+0000 7fca40e6e700 1 -- 172.21.15.59:0/2260412844 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca3c06bb80 con 0x7fca28003a00 2023-11-24T07:57:01.569 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:01.562+0000 7fca40e6e700 1 -- 172.21.15.59:0/2260412844 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca3c06be20 con 0x7fca28004850 2023-11-24T07:57:01.569 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:01.562+0000 7fca40e6e700 1 -- 172.21.15.59:0/2260412844 --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fca2800ba10 con 0x7fca3c15e1e0 2023-11-24T07:57:01.569 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:01.562+0000 7fca41e70700 1 --2- 172.21.15.59:0/2260412844 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fca28003a00 0x7fca28003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:57:01.569 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:01.562+0000 7fca4166f700 1 --2- 172.21.15.59:0/2260412844 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fca28004850 0x7fca28004cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:57:01.569 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:01.562+0000 7fca42671700 1 --2- 172.21.15.59:0/2260412844 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fca3c15e1e0 0x7fca28005210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:57:01.569 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:01.562+0000 7fca41e70700 1 -- 172.21.15.59:0/2260412844 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fca3c15e1e0 msgr2=0x7fca28005210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T07:57:01.569 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:01.562+0000 7fca41e70700 1 --2- 172.21.15.59:0/2260412844 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fca3c15e1e0 0x7fca28005210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:01.569 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:01.562+0000 7fca41e70700 1 -- 172.21.15.59:0/2260412844 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fca28004850 msgr2=0x7fca28004cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T07:57:01.569 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:01.562+0000 7fca41e70700 1 --2- 172.21.15.59:0/2260412844 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fca28004850 0x7fca28004cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:01.569 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:01.562+0000 7fca41e70700 1 -- 172.21.15.59:0/2260412844 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fca2c0094a0 con 0x7fca28003a00 2023-11-24T07:57:01.570 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:01.562+0000 7fca4166f700 1 --2- 172.21.15.59:0/2260412844 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fca28004850 0x7fca28004cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-24T07:57:01.570 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:01.563+0000 7fca41e70700 1 --2- 172.21.15.59:0/2260412844 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fca28003a00 0x7fca28003e20 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7fca2c009d30 tx=0x7fca28005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=63ea05cea2a007ec server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:57:01.570 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:01.563+0000 7fca40e6e700 1 -- 172.21.15.59:0/2260412844 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca2c009e80 con 0x7fca28003a00 2023-11-24T07:57:01.570 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:01.563+0000 7fca40e6e700 1 -- 172.21.15.59:0/2260412844 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fca2c00e9a0 con 0x7fca28003a00 2023-11-24T07:57:01.570 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:01.563+0000 7fca40e6e700 1 -- 172.21.15.59:0/2260412844 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca2c00ed10 con 0x7fca28003a00 2023-11-24T07:57:01.570 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:01.563+0000 7fca440d4700 1 -- 172.21.15.59:0/2260412844 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fca28003a00 msgr2=0x7fca28003e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:57:01.570 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:01.563+0000 7fca440d4700 1 --2- 172.21.15.59:0/2260412844 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fca28003a00 0x7fca28003e20 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7fca2c009d30 tx=0x7fca28005880 comp rx=0 tx=0).stop 2023-11-24T07:57:01.570 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:01.563+0000 7fca42671700 1 -- 172.21.15.59:0/2260412844 reap_dead start 2023-11-24T07:57:01.570 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:01.563+0000 7fca440d4700 1 -- 172.21.15.59:0/2260412844 shutdown_connections 2023-11-24T07:57:01.570 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:01.563+0000 7fca440d4700 1 -- 172.21.15.59:0/2260412844 >> 172.21.15.59:0/2260412844 conn(0x7fca3c0b5070 msgr2=0x7fca3c0b5730 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:57:01.571 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:01.563+0000 7fca440d4700 1 -- 172.21.15.59:0/2260412844 shutdown_connections 2023-11-24T07:57:01.571 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:01.563+0000 7fca440d4700 1 -- 172.21.15.59:0/2260412844 wait complete. 2023-11-24T07:57:01.571 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:01.565+0000 7fca440d4700 1 Processor -- start 2023-11-24T07:57:01.571 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:01.565+0000 7fca440d4700 1 -- start start 2023-11-24T07:57:01.573 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:01.565+0000 7fca440d4700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fca28003a00 0x7fca3c1567f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:57:01.574 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:01.565+0000 7fca440d4700 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fca3c159d10 0x7fca3c156d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:57:01.574 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:01.565+0000 7fca440d4700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fca3c14b8c0 0x7fca3c1f3d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:57:01.574 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:01.565+0000 7fca440d4700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca3c158f90 con 0x7fca28003a00 2023-11-24T07:57:01.574 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:01.565+0000 7fca440d4700 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca3c15ad00 con 0x7fca3c14b8c0 2023-11-24T07:57:01.574 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:01.565+0000 7fca440d4700 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fca3c159230 con 0x7fca3c159d10 2023-11-24T07:57:01.574 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:01.565+0000 7fca41e70700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fca28003a00 0x7fca3c1567f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:57:01.574 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:01.565+0000 7fca41e70700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fca28003a00 0x7fca3c1567f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:50908/0 (socket says 172.21.15.59:50908) 2023-11-24T07:57:01.574 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:01.565+0000 7fca41e70700 1 -- 172.21.15.59:0/2829000622 learned_addr learned my addr 172.21.15.59:0/2829000622 (peer_addr_for_me v2:172.21.15.59:0/0) 2023-11-24T07:57:01.574 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:01.566+0000 7fca4166f700 1 --2- 172.21.15.59:0/2829000622 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fca3c159d10 0x7fca3c156d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:57:01.575 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:01.566+0000 7fca42671700 1 --2- 172.21.15.59:0/2829000622 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fca3c14b8c0 0x7fca3c1f3d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:57:01.575 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:01.566+0000 7fca41e70700 1 -- 172.21.15.59:0/2829000622 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fca3c159d10 msgr2=0x7fca3c156d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:57:01.575 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:01.566+0000 7fca41e70700 1 --2- 172.21.15.59:0/2829000622 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fca3c159d10 0x7fca3c156d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:01.575 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:01.566+0000 7fca41e70700 1 -- 172.21.15.59:0/2829000622 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fca3c14b8c0 msgr2=0x7fca3c1f3d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:57:01.575 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:01.566+0000 7fca41e70700 1 --2- 172.21.15.59:0/2829000622 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fca3c14b8c0 0x7fca3c1f3d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:01.575 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:01.566+0000 7fca41e70700 1 -- 172.21.15.59:0/2829000622 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fca3c159510 con 0x7fca28003a00 2023-11-24T07:57:01.575 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:01.566+0000 7fca42671700 1 --2- 172.21.15.59:0/2829000622 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fca3c14b8c0 0x7fca3c1f3d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-24T07:57:01.575 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:01.566+0000 7fca41e70700 1 --2- 172.21.15.59:0/2829000622 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fca28003a00 0x7fca3c1567f0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7fca28004660 tx=0x7fca2c0047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:57:01.575 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:01.566+0000 7fca327fc700 1 -- 172.21.15.59:0/2829000622 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca2c003ab0 con 0x7fca28003a00 2023-11-24T07:57:01.575 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:01.566+0000 7fca327fc700 1 -- 172.21.15.59:0/2829000622 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fca2c003c30 con 0x7fca28003a00 2023-11-24T07:57:01.575 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:01.566+0000 7fca327fc700 1 -- 172.21.15.59:0/2829000622 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca2c009e80 con 0x7fca28003a00 2023-11-24T07:57:01.576 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:01.566+0000 7fca440d4700 1 -- 172.21.15.59:0/2829000622 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fca3c1597f0 con 0x7fca28003a00 2023-11-24T07:57:01.576 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:01.566+0000 7fca440d4700 1 -- 172.21.15.59:0/2829000622 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fca3c15a8b0 con 0x7fca28003a00 2023-11-24T07:57:01.576 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:01.567+0000 7fca440d4700 1 -- 172.21.15.59:0/2829000622 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fca3c15aad0 con 0x7fca28003a00 2023-11-24T07:57:01.576 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:01.568+0000 7fca327fc700 1 -- 172.21.15.59:0/2829000622 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fca2c013470 con 0x7fca28003a00 2023-11-24T07:57:01.576 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:01.568+0000 7fca327fc700 1 --2- 172.21.15.59:0/2829000622 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7fca2006af00 0x7fca2006d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:57:01.576 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:01.568+0000 7fca327fc700 1 -- 172.21.15.59:0/2829000622 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fca2c08b050 con 0x7fca28003a00 2023-11-24T07:57:01.576 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:01.568+0000 7fca4166f700 1 --2- 172.21.15.59:0/2829000622 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7fca2006af00 0x7fca2006d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:57:01.576 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:01.569+0000 7fca4166f700 1 --2- 172.21.15.59:0/2829000622 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7fca2006af00 0x7fca2006d3c0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fca3c06eca0 tx=0x7fca38008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:57:01.577 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:01.573+0000 7fca327fc700 1 -- 172.21.15.59:0/2829000622 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fca2c056190 con 0x7fca28003a00 2023-11-24T07:57:01.776 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:01.775+0000 7fca440d4700 1 -- 172.21.15.59:0/2829000622 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7fca3c075480 con 0x7fca28003a00 2023-11-24T07:57:01.779 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:01.778+0000 7fca327fc700 1 -- 172.21.15.59:0/2829000622 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v10)=0 v10) v1 ==== 122+0+0 (secure 0 0 0) 0x7fca2c0597b0 con 0x7fca28003a00 2023-11-24T07:57:01.782 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:01.780+0000 7fca440d4700 1 -- 172.21.15.59:0/2829000622 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7fca2006af00 msgr2=0x7fca2006d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:57:01.782 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:01.780+0000 7fca440d4700 1 --2- 172.21.15.59:0/2829000622 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7fca2006af00 0x7fca2006d3c0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fca3c06eca0 tx=0x7fca38008040 comp rx=0 tx=0).stop 2023-11-24T07:57:01.783 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:01.780+0000 7fca440d4700 1 -- 172.21.15.59:0/2829000622 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fca28003a00 msgr2=0x7fca3c1567f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:57:01.783 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:01.780+0000 7fca440d4700 1 --2- 172.21.15.59:0/2829000622 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fca28003a00 0x7fca3c1567f0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7fca28004660 tx=0x7fca2c0047c0 comp rx=0 tx=0).stop 2023-11-24T07:57:01.783 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:01.780+0000 7fca440d4700 1 -- 172.21.15.59:0/2829000622 shutdown_connections 2023-11-24T07:57:01.783 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:01.780+0000 7fca440d4700 1 --2- 172.21.15.59:0/2829000622 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7fca2006af00 0x7fca2006d3c0 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:01.784 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:01.780+0000 7fca440d4700 1 --2- 172.21.15.59:0/2829000622 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fca3c14b8c0 0x7fca3c1f3d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:01.784 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:01.780+0000 7fca440d4700 1 --2- 172.21.15.59:0/2829000622 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fca28003a00 0x7fca3c1567f0 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:01.784 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:01.780+0000 7fca440d4700 1 --2- 172.21.15.59:0/2829000622 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fca3c159d10 0x7fca3c156d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:01.784 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:01.780+0000 7fca440d4700 1 -- 172.21.15.59:0/2829000622 >> 172.21.15.59:0/2829000622 conn(0x7fca3c0b5070 msgr2=0x7fca3c0b5610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:57:01.784 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:01.780+0000 7fca440d4700 1 -- 172.21.15.59:0/2829000622 shutdown_connections 2023-11-24T07:57:01.784 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:01.780+0000 7fca440d4700 1 -- 172.21.15.59:0/2829000622 wait complete. 2023-11-24T07:57:01.794 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-11-24T07:57:01.795 DEBUG:teuthology.orchestra.run.smithi059:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-11-24T07:57:01.838 DEBUG:teuthology.orchestra.run.smithi076:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-11-24T07:57:01.896 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-11-24T07:57:01.897 DEBUG:teuthology.orchestra.run.smithi059:> set -ex 2023-11-24T07:57:01.897 DEBUG:teuthology.orchestra.run.smithi059:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-24T07:57:01.914 DEBUG:teuthology.orchestra.run.smithi059:> set -ex 2023-11-24T07:57:01.914 DEBUG:teuthology.orchestra.run.smithi059:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-24T07:57:01.972 DEBUG:teuthology.orchestra.run.smithi059:> set -ex 2023-11-24T07:57:01.972 DEBUG:teuthology.orchestra.run.smithi059:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-24T07:57:02.029 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-24T07:57:02.193 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:02.191+0000 7fecfa9c6700 1 Processor -- start 2023-11-24T07:57:02.193 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:02.192+0000 7fecfa9c6700 1 -- start start 2023-11-24T07:57:02.195 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:02.192+0000 7fecfa9c6700 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fecf406b6e0 0x7fecf4168ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:57:02.196 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:02.192+0000 7fecfa9c6700 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fecf4147bb0 con 0x7fecf40d0080 2023-11-24T07:57:02.196 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:02.192+0000 7fecfa9c6700 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fecf40d0460 con 0x7fecf4071bb0 2023-11-24T07:57:02.196 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:02.192+0000 7fecfa9c6700 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fecf4072cd0 con 0x7fecf406b6e0 2023-11-24T07:57:02.196 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:02.192+0000 7fecf3fff700 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7fecf40d0080 0x7fecf40b5890 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:54390/0 (socket says 172.21.15.59:54390) 2023-11-24T07:57:02.196 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:02.193+0000 7fecf3fff700 1 -- 172.21.15.59:0/1554990163 learned_addr learned my addr 172.21.15.59:0/1554990163 (peer_addr_for_me v1:172.21.15.59:0/0) 2023-11-24T07:57:02.196 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:02.193+0000 7fecf8f63700 1 --2- 172.21.15.59:0/1554990163 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fecf406b6e0 0x7fecf4168ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:57:02.196 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:02.193+0000 7fecf2ffd700 1 -- 172.21.15.59:0/1554990163 <== mon.0 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 377543765 0 0) 0x7fecf4147bb0 con 0x7fecf40d0080 2023-11-24T07:57:02.196 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:02.193+0000 7fecf2ffd700 1 -- 172.21.15.59:0/1554990163 --> v1:172.21.15.59:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fecdc0035b0 con 0x7fecf40d0080 2023-11-24T07:57:02.196 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:02.193+0000 7fecf2ffd700 1 -- 172.21.15.59:0/1554990163 <== mon.1 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 626642439 0 0) 0x7fecf40d0460 con 0x7fecf4071bb0 2023-11-24T07:57:02.196 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:02.193+0000 7fecf8f63700 1 -- 172.21.15.59:0/1554990163 >> v1:172.21.15.76:6789/0 conn(0x7fecf4071bb0 legacy=0x7fecf406afd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:57:02.197 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:02.193+0000 7fecf8f63700 1 -- 172.21.15.59:0/1554990163 >> v1:172.21.15.59:6789/0 conn(0x7fecf40d0080 legacy=0x7fecf40b5890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:57:02.197 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:02.193+0000 7fecf8f63700 1 -- 172.21.15.59:0/1554990163 --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fecf4072150 con 0x7fecf406b6e0 2023-11-24T07:57:02.197 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:02.193+0000 7fecf8f63700 1 --2- 172.21.15.59:0/1554990163 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fecf406b6e0 0x7fecf4168ec0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fecec009dc0 tx=0x7fecec00b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=894b831953ec0899 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:57:02.197 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:02.193+0000 7fecf2ffd700 1 -- 172.21.15.59:0/1554990163 <== mon.2 v2:172.21.15.76:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fecec00e070 con 0x7fecf406b6e0 2023-11-24T07:57:02.197 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:02.193+0000 7fecf2ffd700 1 -- 172.21.15.59:0/1554990163 <== mon.2 v2:172.21.15.76:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fecec003a20 con 0x7fecf406b6e0 2023-11-24T07:57:02.197 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:02.193+0000 7fecf2ffd700 1 -- 172.21.15.59:0/1554990163 <== mon.2 v2:172.21.15.76:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fece4001670 con 0x7fecf406b6e0 2023-11-24T07:57:02.197 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:02.194+0000 7fecfa9c6700 1 -- 172.21.15.59:0/1554990163 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fecf406b6e0 msgr2=0x7fecf4168ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:57:02.197 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:02.194+0000 7fecfa9c6700 1 --2- 172.21.15.59:0/1554990163 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fecf406b6e0 0x7fecf4168ec0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fecec009dc0 tx=0x7fecec00b330 comp rx=0 tx=0).stop 2023-11-24T07:57:02.197 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:02.194+0000 7fecfa9c6700 1 -- 172.21.15.59:0/1554990163 shutdown_connections 2023-11-24T07:57:02.197 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:02.194+0000 7fecfa9c6700 1 --2- 172.21.15.59:0/1554990163 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fecf406b6e0 0x7fecf4168ec0 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:02.197 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:02.194+0000 7fecfa9c6700 1 -- 172.21.15.59:0/1554990163 >> 172.21.15.59:0/1554990163 conn(0x7fecf40b27d0 msgr2=0x7fecf40b2be0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:57:02.197 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:02.194+0000 7fecfa9c6700 1 -- 172.21.15.59:0/1554990163 shutdown_connections 2023-11-24T07:57:02.198 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:02.194+0000 7fecfa9c6700 1 -- 172.21.15.59:0/1554990163 wait complete. 2023-11-24T07:57:02.200 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:02.195+0000 7fecfa9c6700 1 Processor -- start 2023-11-24T07:57:02.200 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:02.196+0000 7fecfa9c6700 1 -- start start 2023-11-24T07:57:02.200 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:02.196+0000 7fecfa9c6700 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fecf406b6e0 0x7fecf415dc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:57:02.200 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:02.196+0000 7fecfa9c6700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fecf4071bb0 0x7fecf40c5c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:57:02.200 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:02.196+0000 7fecfa9c6700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fecf40d0080 0x7fecf4136f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:57:02.200 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:02.196+0000 7fecfa9c6700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fecf4142e40 con 0x7fecf40d0080 2023-11-24T07:57:02.200 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:02.196+0000 7fecfa9c6700 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fecf40c6170 con 0x7fecf4071bb0 2023-11-24T07:57:02.200 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:02.196+0000 7fecfa9c6700 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fecf40c6aa0 con 0x7fecf406b6e0 2023-11-24T07:57:02.201 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:02.196+0000 7fecf8f63700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fecf40d0080 0x7fecf4136f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:57:02.201 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:02.196+0000 7fecf8f63700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fecf40d0080 0x7fecf4136f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:50926/0 (socket says 172.21.15.59:50926) 2023-11-24T07:57:02.201 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:02.196+0000 7fecf8f63700 1 -- 172.21.15.59:0/1743893508 learned_addr learned my addr 172.21.15.59:0/1743893508 (peer_addr_for_me v2:172.21.15.59:0/0) 2023-11-24T07:57:02.201 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:02.197+0000 7fecf3fff700 1 --2- 172.21.15.59:0/1743893508 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fecf406b6e0 0x7fecf415dc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:57:02.201 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:02.197+0000 7fecf37fe700 1 --2- 172.21.15.59:0/1743893508 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fecf4071bb0 0x7fecf40c5c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:57:02.201 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:02.197+0000 7fecf8f63700 1 -- 172.21.15.59:0/1743893508 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fecf406b6e0 msgr2=0x7fecf415dc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:57:02.201 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:02.197+0000 7fecf8f63700 1 --2- 172.21.15.59:0/1743893508 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fecf406b6e0 0x7fecf415dc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:02.201 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:02.197+0000 7fecf8f63700 1 -- 172.21.15.59:0/1743893508 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fecf4071bb0 msgr2=0x7fecf40c5c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:57:02.201 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:02.197+0000 7fecf8f63700 1 --2- 172.21.15.59:0/1743893508 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fecf4071bb0 0x7fecf40c5c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:02.201 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:02.197+0000 7fecf8f63700 1 -- 172.21.15.59:0/1743893508 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fecec009a70 con 0x7fecf40d0080 2023-11-24T07:57:02.201 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:02.197+0000 7fecf8f63700 1 --2- 172.21.15.59:0/1743893508 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fecf40d0080 0x7fecf4136f20 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7fecec015040 tx=0x7fecec004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:57:02.202 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:02.197+0000 7fecf0ff9700 1 -- 172.21.15.59:0/1743893508 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fecec004540 con 0x7fecf40d0080 2023-11-24T07:57:02.202 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:02.197+0000 7fecf0ff9700 1 -- 172.21.15.59:0/1743893508 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fecec0046e0 con 0x7fecf40d0080 2023-11-24T07:57:02.202 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:02.197+0000 7fecfa9c6700 1 -- 172.21.15.59:0/1743893508 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fecf40c6d80 con 0x7fecf40d0080 2023-11-24T07:57:02.202 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:02.197+0000 7fecf0ff9700 1 -- 172.21.15.59:0/1743893508 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fecec009410 con 0x7fecf40d0080 2023-11-24T07:57:02.202 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:02.197+0000 7fecfa9c6700 1 -- 172.21.15.59:0/1743893508 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fecf40c7340 con 0x7fecf40d0080 2023-11-24T07:57:02.202 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:02.198+0000 7fecfa9c6700 1 -- 172.21.15.59:0/1743893508 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fecf4068670 con 0x7fecf40d0080 2023-11-24T07:57:02.202 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:02.199+0000 7fecf0ff9700 1 -- 172.21.15.59:0/1743893508 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fecec012450 con 0x7fecf40d0080 2023-11-24T07:57:02.203 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:02.199+0000 7fecf0ff9700 1 --2- 172.21.15.59:0/1743893508 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7fece006af00 0x7fece006d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:57:02.203 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:02.199+0000 7fecf0ff9700 1 -- 172.21.15.59:0/1743893508 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fecec08ae80 con 0x7fecf40d0080 2023-11-24T07:57:02.203 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:02.199+0000 7fecf3fff700 1 --2- 172.21.15.59:0/1743893508 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7fece006af00 0x7fece006d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:57:02.203 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:02.200+0000 7fecf3fff700 1 --2- 172.21.15.59:0/1743893508 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7fece006af00 0x7fece006d3c0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fece4004aa0 tx=0x7fece4001890 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:57:02.205 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:02.203+0000 7fecf0ff9700 1 -- 172.21.15.59:0/1743893508 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fecec0599d0 con 0x7fecf40d0080 2023-11-24T07:57:02.408 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:02.406+0000 7fecfa9c6700 1 -- 172.21.15.59:0/1743893508 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fecf40b3510 con 0x7fecf40d0080 2023-11-24T07:57:02.409 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:02.407+0000 7fecf0ff9700 1 -- 172.21.15.59:0/1743893508 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) v1 ==== 74+0+16437 (secure 0 0 0) 0x7fecec059560 con 0x7fecf40d0080 2023-11-24T07:57:02.409 INFO:teuthology.orchestra.run.smithi059.stdout: 2023-11-24T07:57:02.409 INFO:teuthology.orchestra.run.smithi059.stdout:{"epoch":20,"fsid":"9f0e8741-b8f9-48d5-a2c3-09f45903303d","created":"2023-11-24T07:56:06.552729+0000","modified":"2023-11-24T07:56:25.705091+0000","last_up_change":"2023-11-24T07:56:17.372352+0000","last_in_change":"2023-11-24T07:56:09.073250+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-24T07:56:18.352823+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-11-24T07:56:22.185140+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-11-24T07:56:23.084326+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"6d8f9e9b-1782-4e41-928b-a39f2a8ebc06","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6800","nonce":3373878772},{"type":"v1","addr":"172.21.15.59:6801","nonce":3373878772}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6802","nonce":3373878772},{"type":"v1","addr":"172.21.15.59:6803","nonce":3373878772}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6806","nonce":3373878772},{"type":"v1","addr":"172.21.15.59:6807","nonce":3373878772}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6804","nonce":3373878772},{"type":"v1","addr":"172.21.15.59:6805","nonce":3373878772}]},"public_addr":"172.21.15.59:6801/3373878772","cluster_addr":"172.21.15.59:6803/3373878772","heartbeat_back_addr":"172.21.15.59:6807/3373878772","heartbeat_front_addr":"172.21.15.59:6805/3373878772","state":["exists","up"]},{"osd":1,"uuid":"e8d882aa-52e1-4c04-9e1e-c44ef9204a5f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6824","nonce":648258563},{"type":"v1","addr":"172.21.15.59:6825","nonce":648258563}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6826","nonce":648258563},{"type":"v1","addr":"172.21.15.59:6827","nonce":648258563}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6830","nonce":648258563},{"type":"v1","addr":"172.21.15.59:6831","nonce":648258563}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6828","nonce":648258563},{"type":"v1","addr":"172.21.15.59:6829","nonce":648258563}]},"public_addr":"172.21.15.59:6825/648258563","cluster_addr":"172.21.15.59:6827/648258563","heartbeat_back_addr":"172.21.15.59:6831/648258563","heartbeat_front_addr":"172.21.15.59:6829/648258563","state":["exists","up"]},{"osd":2,"uuid":"69eebfa0-695b-4b21-90b9-8a3537168a01","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6808","nonce":1840430654},{"type":"v1","addr":"172.21.15.59:6809","nonce":1840430654}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6810","nonce":1840430654},{"type":"v1","addr":"172.21.15.59:6811","nonce":1840430654}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6814","nonce":1840430654},{"type":"v1","addr":"172.21.15.59:6815","nonce":1840430654}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6812","nonce":1840430654},{"type":"v1","addr":"172.21.15.59:6813","nonce":1840430654}]},"public_addr":"172.21.15.59:6809/1840430654","cluster_addr":"172.21.15.59:6811/1840430654","heartbeat_back_addr":"172.21.15.59:6815/1840430654","heartbeat_front_addr":"172.21.15.59:6813/1840430654","state":["exists","up"]},{"osd":3,"uuid":"96fd77fc-f911-46ff-9445-6f0be2daa8f4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6816","nonce":2504025850},{"type":"v1","addr":"172.21.15.59:6817","nonce":2504025850}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6818","nonce":2504025850},{"type":"v1","addr":"172.21.15.59:6819","nonce":2504025850}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6822","nonce":2504025850},{"type":"v1","addr":"172.21.15.59:6823","nonce":2504025850}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6820","nonce":2504025850},{"type":"v1","addr":"172.21.15.59:6821","nonce":2504025850}]},"public_addr":"172.21.15.59:6817/2504025850","cluster_addr":"172.21.15.59:6819/2504025850","heartbeat_back_addr":"172.21.15.59:6823/2504025850","heartbeat_front_addr":"172.21.15.59:6821/2504025850","state":["exists","up"]},{"osd":4,"uuid":"7ac29fca-b5b2-4c95-9d69-2f164cbbc708","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6816","nonce":2382463431},{"type":"v1","addr":"172.21.15.76:6818","nonce":2382463431}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6820","nonce":2382463431},{"type":"v1","addr":"172.21.15.76:6822","nonce":2382463431}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6828","nonce":2382463431},{"type":"v1","addr":"172.21.15.76:6830","nonce":2382463431}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6824","nonce":2382463431},{"type":"v1","addr":"172.21.15.76:6826","nonce":2382463431}]},"public_addr":"172.21.15.76:6818/2382463431","cluster_addr":"172.21.15.76:6822/2382463431","heartbeat_back_addr":"172.21.15.76:6830/2382463431","heartbeat_front_addr":"172.21.15.76:6826/2382463431","state":["exists","up"]},{"osd":5,"uuid":"485bf711-3851-4238-a4f4-51d52e0a9c58","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6800","nonce":329334308},{"type":"v1","addr":"172.21.15.76:6802","nonce":329334308}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6805","nonce":329334308},{"type":"v1","addr":"172.21.15.76:6807","nonce":329334308}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6814","nonce":329334308},{"type":"v1","addr":"172.21.15.76:6815","nonce":329334308}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6810","nonce":329334308},{"type":"v1","addr":"172.21.15.76:6812","nonce":329334308}]},"public_addr":"172.21.15.76:6802/329334308","cluster_addr":"172.21.15.76:6807/329334308","heartbeat_back_addr":"172.21.15.76:6815/329334308","heartbeat_front_addr":"172.21.15.76:6812/329334308","state":["exists","up"]},{"osd":6,"uuid":"72b3f2fc-1e4a-4e32-968d-003647a0286f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6801","nonce":3946579410},{"type":"v1","addr":"172.21.15.76:6803","nonce":3946579410}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6804","nonce":3946579410},{"type":"v1","addr":"172.21.15.76:6806","nonce":3946579410}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6811","nonce":3946579410},{"type":"v1","addr":"172.21.15.76:6813","nonce":3946579410}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6808","nonce":3946579410},{"type":"v1","addr":"172.21.15.76:6809","nonce":3946579410}]},"public_addr":"172.21.15.76:6803/3946579410","cluster_addr":"172.21.15.76:6806/3946579410","heartbeat_back_addr":"172.21.15.76:6813/3946579410","heartbeat_front_addr":"172.21.15.76:6809/3946579410","state":["exists","up"]},{"osd":7,"uuid":"1ddceb90-6c5d-4d1a-ad56-99cb877f8c70","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6817","nonce":841761538},{"type":"v1","addr":"172.21.15.76:6819","nonce":841761538}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6821","nonce":841761538},{"type":"v1","addr":"172.21.15.76:6823","nonce":841761538}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6829","nonce":841761538},{"type":"v1","addr":"172.21.15.76:6831","nonce":841761538}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6825","nonce":841761538},{"type":"v1","addr":"172.21.15.76:6827","nonce":841761538}]},"public_addr":"172.21.15.76:6819/841761538","cluster_addr":"172.21.15.76:6823/841761538","heartbeat_back_addr":"172.21.15.76:6831/841761538","heartbeat_front_addr":"172.21.15.76:6827/841761538","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T07:56:14.294838+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T07:56:14.270779+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T07:56:14.231809+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T07:56:14.260983+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T07:56:14.302997+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T07:56:14.258600+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T07:56:14.251413+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T07:56:14.304727+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-24T07:57:02.411 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:02.409+0000 7fecfa9c6700 1 -- 172.21.15.59:0/1743893508 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7fece006af00 msgr2=0x7fece006d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:57:02.411 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:02.409+0000 7fecfa9c6700 1 --2- 172.21.15.59:0/1743893508 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7fece006af00 0x7fece006d3c0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fece4004aa0 tx=0x7fece4001890 comp rx=0 tx=0).stop 2023-11-24T07:57:02.411 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:02.409+0000 7fecfa9c6700 1 -- 172.21.15.59:0/1743893508 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fecf40d0080 msgr2=0x7fecf4136f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:57:02.411 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:02.409+0000 7fecfa9c6700 1 --2- 172.21.15.59:0/1743893508 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fecf40d0080 0x7fecf4136f20 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7fecec015040 tx=0x7fecec004ad0 comp rx=0 tx=0).stop 2023-11-24T07:57:02.411 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:02.409+0000 7fecfa9c6700 1 -- 172.21.15.59:0/1743893508 shutdown_connections 2023-11-24T07:57:02.412 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:02.409+0000 7fecfa9c6700 1 --2- 172.21.15.59:0/1743893508 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7fece006af00 0x7fece006d3c0 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:02.412 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:02.409+0000 7fecfa9c6700 1 --2- 172.21.15.59:0/1743893508 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fecf40d0080 0x7fecf4136f20 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:02.412 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:02.409+0000 7fecfa9c6700 1 --2- 172.21.15.59:0/1743893508 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fecf406b6e0 0x7fecf415dc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:02.412 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:02.409+0000 7fecfa9c6700 1 --2- 172.21.15.59:0/1743893508 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fecf4071bb0 0x7fecf40c5c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:02.412 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:02.409+0000 7fecfa9c6700 1 -- 172.21.15.59:0/1743893508 >> 172.21.15.59:0/1743893508 conn(0x7fecf40b27d0 msgr2=0x7fecf4146780 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:57:02.412 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:02.409+0000 7fecfa9c6700 1 -- 172.21.15.59:0/1743893508 shutdown_connections 2023-11-24T07:57:02.412 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:02.409+0000 7fecfa9c6700 1 -- 172.21.15.59:0/1743893508 wait complete. 2023-11-24T07:57:02.423 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-24T07:56:18.352823+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-24T07:56:22.185140+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-11-24T07:56:23.084326+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-24T07:57:02.424 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-24T07:57:02.552 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:02.550+0000 7fc81db9b700 1 Processor -- start 2023-11-24T07:57:02.552 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:02.550+0000 7fc81db9b700 1 -- start start 2023-11-24T07:57:02.554 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:02.551+0000 7fc81db9b700 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fc8181459c0 0x7fc81815c540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:57:02.555 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:02.551+0000 7fc81db9b700 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fc81806b6e0 0x7fc81806e260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:57:02.555 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:02.551+0000 7fc81db9b700 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc81806daf0 con 0x7fc8180cd5d0 2023-11-24T07:57:02.555 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:02.551+0000 7fc81db9b700 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc8180cfe60 con 0x7fc81806b6e0 2023-11-24T07:57:02.555 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:02.551+0000 7fc81db9b700 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc8180d0140 con 0x7fc8181459c0 2023-11-24T07:57:02.555 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:02.551+0000 7fc816ffd700 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7fc8180cd5d0 0x7fc81806afd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:54406/0 (socket says 172.21.15.59:54406) 2023-11-24T07:57:02.555 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:02.551+0000 7fc816ffd700 1 -- 172.21.15.59:0/430511074 learned_addr learned my addr 172.21.15.59:0/430511074 (peer_addr_for_me v1:172.21.15.59:0/0) 2023-11-24T07:57:02.555 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:02.552+0000 7fc8177fe700 1 --2- 172.21.15.59:0/430511074 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fc8181459c0 0x7fc81815c540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:57:02.556 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:02.552+0000 7fc817fff700 1 --2- 172.21.15.59:0/430511074 >> v2:172.21.15.76:3300/0 conn(0x7fc81806b6e0 0x7fc81806e260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:57:02.556 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:02.552+0000 7fc8167fc700 1 -- 172.21.15.59:0/430511074 <== mon.0 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1425110467 0 0) 0x7fc81806daf0 con 0x7fc8180cd5d0 2023-11-24T07:57:02.556 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:02.552+0000 7fc8167fc700 1 -- 172.21.15.59:0/430511074 --> v1:172.21.15.59:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc8000035b0 con 0x7fc8180cd5d0 2023-11-24T07:57:02.556 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:02.552+0000 7fc8177fe700 1 -- 172.21.15.59:0/430511074 >> v2:172.21.15.76:3300/0 conn(0x7fc81806b6e0 msgr2=0x7fc81806e260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T07:57:02.556 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:02.552+0000 7fc8177fe700 1 --2- 172.21.15.59:0/430511074 >> v2:172.21.15.76:3300/0 conn(0x7fc81806b6e0 0x7fc81806e260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:02.556 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:02.552+0000 7fc8177fe700 1 -- 172.21.15.59:0/430511074 >> v1:172.21.15.59:6789/0 conn(0x7fc8180cd5d0 legacy=0x7fc81806afd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:57:02.556 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:02.552+0000 7fc8177fe700 1 -- 172.21.15.59:0/430511074 --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc8180d0420 con 0x7fc8181459c0 2023-11-24T07:57:02.556 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:02.552+0000 7fc8177fe700 1 --2- 172.21.15.59:0/430511074 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fc8181459c0 0x7fc81815c540 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fc808009840 tx=0x7fc808009b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=94a0312b6f79dd12 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:57:02.556 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:02.552+0000 7fc8167fc700 1 -- 172.21.15.59:0/430511074 <== mon.2 v2:172.21.15.76:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc808011830 con 0x7fc8181459c0 2023-11-24T07:57:02.556 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:02.552+0000 7fc8167fc700 1 -- 172.21.15.59:0/430511074 <== mon.2 v2:172.21.15.76:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc808011990 con 0x7fc8181459c0 2023-11-24T07:57:02.556 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:02.552+0000 7fc8167fc700 1 -- 172.21.15.59:0/430511074 <== mon.2 v2:172.21.15.76:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc808011d20 con 0x7fc8181459c0 2023-11-24T07:57:02.557 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:02.553+0000 7fc81db9b700 1 -- 172.21.15.59:0/430511074 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fc8181459c0 msgr2=0x7fc81815c540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:57:02.557 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:02.553+0000 7fc81db9b700 1 --2- 172.21.15.59:0/430511074 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fc8181459c0 0x7fc81815c540 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fc808009840 tx=0x7fc808009b50 comp rx=0 tx=0).stop 2023-11-24T07:57:02.557 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:02.553+0000 7fc81db9b700 1 -- 172.21.15.59:0/430511074 shutdown_connections 2023-11-24T07:57:02.557 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:02.553+0000 7fc81db9b700 1 --2- 172.21.15.59:0/430511074 >> v2:172.21.15.76:3300/0 conn(0x7fc81806b6e0 0x7fc81806e260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:02.557 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:02.553+0000 7fc81db9b700 1 --2- 172.21.15.59:0/430511074 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fc8181459c0 0x7fc81815c540 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:02.557 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:02.553+0000 7fc81db9b700 1 -- 172.21.15.59:0/430511074 >> 172.21.15.59:0/430511074 conn(0x7fc8180b27b0 msgr2=0x7fc8180b2b90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:57:02.557 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:02.553+0000 7fc81db9b700 1 -- 172.21.15.59:0/430511074 shutdown_connections 2023-11-24T07:57:02.557 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:02.553+0000 7fc81db9b700 1 -- 172.21.15.59:0/430511074 wait complete. 2023-11-24T07:57:02.560 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:02.554+0000 7fc81db9b700 1 Processor -- start 2023-11-24T07:57:02.560 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:02.554+0000 7fc81db9b700 1 -- start start 2023-11-24T07:57:02.560 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:02.555+0000 7fc81db9b700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc81806b6e0 0x7fc81813aad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:57:02.560 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:02.555+0000 7fc81db9b700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc8180cd5d0 0x7fc8180b51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:57:02.561 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:02.555+0000 7fc81db9b700 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fc8181459c0 0x7fc818143cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:57:02.561 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:02.555+0000 7fc81db9b700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8180b56f0 con 0x7fc81806b6e0 2023-11-24T07:57:02.561 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:02.555+0000 7fc81db9b700 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8180c9e30 con 0x7fc8180cd5d0 2023-11-24T07:57:02.561 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:02.555+0000 7fc81db9b700 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc8180ca110 con 0x7fc8181459c0 2023-11-24T07:57:02.561 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:02.555+0000 7fc8177fe700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc81806b6e0 0x7fc81813aad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:57:02.561 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:02.555+0000 7fc8177fe700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc81806b6e0 0x7fc81813aad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:50936/0 (socket says 172.21.15.59:50936) 2023-11-24T07:57:02.561 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:02.555+0000 7fc8177fe700 1 -- 172.21.15.59:0/1098403162 learned_addr learned my addr 172.21.15.59:0/1098403162 (peer_addr_for_me v2:172.21.15.59:0/0) 2023-11-24T07:57:02.562 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:02.555+0000 7fc817fff700 1 --2- 172.21.15.59:0/1098403162 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fc8181459c0 0x7fc818143cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:57:02.562 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:02.555+0000 7fc816ffd700 1 --2- 172.21.15.59:0/1098403162 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc8180cd5d0 0x7fc8180b51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:57:02.562 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:02.555+0000 7fc8177fe700 1 -- 172.21.15.59:0/1098403162 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fc8181459c0 msgr2=0x7fc818143cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:57:02.562 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:02.555+0000 7fc8177fe700 1 --2- 172.21.15.59:0/1098403162 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fc8181459c0 0x7fc818143cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:02.562 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:02.555+0000 7fc8177fe700 1 -- 172.21.15.59:0/1098403162 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc8180cd5d0 msgr2=0x7fc8180b51b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:57:02.562 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:02.555+0000 7fc8177fe700 1 --2- 172.21.15.59:0/1098403162 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc8180cd5d0 0x7fc8180b51b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:02.562 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:02.555+0000 7fc8177fe700 1 -- 172.21.15.59:0/1098403162 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc8080094a0 con 0x7fc81806b6e0 2023-11-24T07:57:02.562 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:02.555+0000 7fc817fff700 1 --2- 172.21.15.59:0/1098403162 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fc8181459c0 0x7fc818143cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-24T07:57:02.562 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:02.555+0000 7fc816ffd700 1 --2- 172.21.15.59:0/1098403162 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc8180cd5d0 0x7fc8180b51b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-24T07:57:02.562 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:02.556+0000 7fc8177fe700 1 --2- 172.21.15.59:0/1098403162 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc81806b6e0 0x7fc81813aad0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fc808009fd0 tx=0x7fc808004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:57:02.563 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:02.556+0000 7fc7fffff700 1 -- 172.21.15.59:0/1098403162 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8080036a0 con 0x7fc81806b6e0 2023-11-24T07:57:02.563 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:02.556+0000 7fc7fffff700 1 -- 172.21.15.59:0/1098403162 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc80801e070 con 0x7fc81806b6e0 2023-11-24T07:57:02.563 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:02.556+0000 7fc81db9b700 1 -- 172.21.15.59:0/1098403162 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc8180ca3f0 con 0x7fc81806b6e0 2023-11-24T07:57:02.563 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:02.556+0000 7fc7fffff700 1 -- 172.21.15.59:0/1098403162 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc808003a60 con 0x7fc81806b6e0 2023-11-24T07:57:02.563 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:02.556+0000 7fc81db9b700 1 -- 172.21.15.59:0/1098403162 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc8180ca9b0 con 0x7fc81806b6e0 2023-11-24T07:57:02.563 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:02.557+0000 7fc81db9b700 1 -- 172.21.15.59:0/1098403162 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc81815f0f0 con 0x7fc81806b6e0 2023-11-24T07:57:02.563 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:02.557+0000 7fc7fffff700 1 -- 172.21.15.59:0/1098403162 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fc808003bc0 con 0x7fc81806b6e0 2023-11-24T07:57:02.563 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:02.558+0000 7fc7fffff700 1 --2- 172.21.15.59:0/1098403162 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7fc80406aed0 0x7fc80406d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:57:02.563 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:02.558+0000 7fc7fffff700 1 -- 172.21.15.59:0/1098403162 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fc80808a4e0 con 0x7fc81806b6e0 2023-11-24T07:57:02.563 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:02.558+0000 7fc816ffd700 1 --2- 172.21.15.59:0/1098403162 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7fc80406aed0 0x7fc80406d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:57:02.563 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:02.558+0000 7fc816ffd700 1 --2- 172.21.15.59:0/1098403162 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7fc80406aed0 0x7fc80406d390 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fc81815bed0 tx=0x7fc80c009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:57:02.564 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:02.562+0000 7fc7fffff700 1 -- 172.21.15.59:0/1098403162 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fc8080555a0 con 0x7fc81806b6e0 2023-11-24T07:57:02.765 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:02.763+0000 7fc81db9b700 1 -- 172.21.15.59:0/1098403162 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fc818075480 con 0x7fc81806b6e0 2023-11-24T07:57:02.765 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:02.763+0000 7fc7fffff700 1 -- 172.21.15.59:0/1098403162 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v20) v1 ==== 93+0+10 (secure 0 0 0) 0x7fc808058bc0 con 0x7fc81806b6e0 2023-11-24T07:57:02.765 INFO:teuthology.orchestra.run.smithi059.stdout:pg_num: 1 2023-11-24T07:57:02.768 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:02.766+0000 7fc81db9b700 1 -- 172.21.15.59:0/1098403162 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7fc80406aed0 msgr2=0x7fc80406d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:57:02.768 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:02.766+0000 7fc81db9b700 1 --2- 172.21.15.59:0/1098403162 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7fc80406aed0 0x7fc80406d390 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fc81815bed0 tx=0x7fc80c009040 comp rx=0 tx=0).stop 2023-11-24T07:57:02.768 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:02.766+0000 7fc81db9b700 1 -- 172.21.15.59:0/1098403162 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc81806b6e0 msgr2=0x7fc81813aad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:57:02.768 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:02.766+0000 7fc81db9b700 1 --2- 172.21.15.59:0/1098403162 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc81806b6e0 0x7fc81813aad0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fc808009fd0 tx=0x7fc808004750 comp rx=0 tx=0).stop 2023-11-24T07:57:02.768 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:02.766+0000 7fc81db9b700 1 -- 172.21.15.59:0/1098403162 shutdown_connections 2023-11-24T07:57:02.768 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:02.766+0000 7fc81db9b700 1 --2- 172.21.15.59:0/1098403162 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7fc80406aed0 0x7fc80406d390 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:02.768 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:02.766+0000 7fc81db9b700 1 --2- 172.21.15.59:0/1098403162 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fc8181459c0 0x7fc818143cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:02.768 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:02.766+0000 7fc81db9b700 1 --2- 172.21.15.59:0/1098403162 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc81806b6e0 0x7fc81813aad0 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:02.769 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:02.766+0000 7fc81db9b700 1 --2- 172.21.15.59:0/1098403162 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc8180cd5d0 0x7fc8180b51b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:02.769 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:02.766+0000 7fc81db9b700 1 -- 172.21.15.59:0/1098403162 >> 172.21.15.59:0/1098403162 conn(0x7fc8180b27b0 msgr2=0x7fc81815d610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:57:02.769 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:02.766+0000 7fc81db9b700 1 -- 172.21.15.59:0/1098403162 shutdown_connections 2023-11-24T07:57:02.769 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:02.766+0000 7fc81db9b700 1 -- 172.21.15.59:0/1098403162 wait complete. 2023-11-24T07:57:02.780 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-24T07:57:02.908 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:02.906+0000 7fb7198cc700 1 Processor -- start 2023-11-24T07:57:02.908 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:02.906+0000 7fb7198cc700 1 -- start start 2023-11-24T07:57:02.910 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:02.907+0000 7fb7198cc700 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fb7140d0870 0x7fb7140d26c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:57:02.910 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:02.907+0000 7fb7198cc700 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fb7140d49f0 0x7fb7140b6d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:57:02.910 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:02.907+0000 7fb7198cc700 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb7140b5370 con 0x7fb7140d0c50 2023-11-24T07:57:02.910 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:02.907+0000 7fb7198cc700 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb7140d2f80 con 0x7fb7140d0870 2023-11-24T07:57:02.910 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:02.907+0000 7fb7198cc700 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb7140cff70 con 0x7fb7140d49f0 2023-11-24T07:57:02.911 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:02.907+0000 7fb7127fc700 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7fb7140d0c50 0x7fb7140cf860 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:54418/0 (socket says 172.21.15.59:54418) 2023-11-24T07:57:02.911 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:02.907+0000 7fb7127fc700 1 -- 172.21.15.59:0/593082271 learned_addr learned my addr 172.21.15.59:0/593082271 (peer_addr_for_me v1:172.21.15.59:0/0) 2023-11-24T07:57:02.911 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:02.907+0000 7fb7137fe700 1 --2- 172.21.15.59:0/593082271 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fb7140d49f0 0x7fb7140b6d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:57:02.911 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:02.907+0000 7fb712ffd700 1 --2- 172.21.15.59:0/593082271 >> v2:172.21.15.76:3300/0 conn(0x7fb7140d0870 0x7fb7140d26c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:57:02.911 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:02.907+0000 7fb711ffb700 1 -- 172.21.15.59:0/593082271 <== mon.0 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1273254060 0 0) 0x7fb7140b5370 con 0x7fb7140d0c50 2023-11-24T07:57:02.911 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:02.907+0000 7fb711ffb700 1 -- 172.21.15.59:0/593082271 --> v1:172.21.15.59:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb7000035b0 con 0x7fb7140d0c50 2023-11-24T07:57:02.911 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:02.908+0000 7fb711ffb700 1 -- 172.21.15.59:0/593082271 <== mon.0 v1:172.21.15.59:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1575427460 0 0) 0x7fb7000035b0 con 0x7fb7140d0c50 2023-11-24T07:57:02.911 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:02.908+0000 7fb7137fe700 1 -- 172.21.15.59:0/593082271 >> v2:172.21.15.76:3300/0 conn(0x7fb7140d0870 msgr2=0x7fb7140d26c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T07:57:02.911 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:02.908+0000 7fb7137fe700 1 --2- 172.21.15.59:0/593082271 >> v2:172.21.15.76:3300/0 conn(0x7fb7140d0870 0x7fb7140d26c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:02.911 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:02.908+0000 7fb7137fe700 1 -- 172.21.15.59:0/593082271 >> v1:172.21.15.59:6789/0 conn(0x7fb7140d0c50 legacy=0x7fb7140cf860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:57:02.911 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:02.908+0000 7fb7137fe700 1 -- 172.21.15.59:0/593082271 --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb7140ceae0 con 0x7fb7140d49f0 2023-11-24T07:57:02.911 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:02.908+0000 7fb7137fe700 1 --2- 172.21.15.59:0/593082271 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fb7140d49f0 0x7fb7140b6d20 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fb70c009d90 tx=0x7fb70c00b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8a9bbaba4193817c server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:57:02.912 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:02.908+0000 7fb711ffb700 1 -- 172.21.15.59:0/593082271 <== mon.2 v2:172.21.15.76:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb70c00e070 con 0x7fb7140d49f0 2023-11-24T07:57:02.912 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:02.908+0000 7fb711ffb700 1 -- 172.21.15.59:0/593082271 <== mon.2 v2:172.21.15.76:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb70c003a20 con 0x7fb7140d49f0 2023-11-24T07:57:02.912 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:02.908+0000 7fb711ffb700 1 -- 172.21.15.59:0/593082271 <== mon.2 v2:172.21.15.76:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb704001670 con 0x7fb7140d49f0 2023-11-24T07:57:02.912 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:02.908+0000 7fb7198cc700 1 -- 172.21.15.59:0/593082271 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fb7140d49f0 msgr2=0x7fb7140b6d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:57:02.912 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:02.908+0000 7fb7198cc700 1 --2- 172.21.15.59:0/593082271 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fb7140d49f0 0x7fb7140b6d20 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fb70c009d90 tx=0x7fb70c00b330 comp rx=0 tx=0).stop 2023-11-24T07:57:02.912 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:02.908+0000 7fb7198cc700 1 -- 172.21.15.59:0/593082271 shutdown_connections 2023-11-24T07:57:02.912 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:02.908+0000 7fb7198cc700 1 --2- 172.21.15.59:0/593082271 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fb7140d49f0 0x7fb7140b6d20 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:02.912 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:02.908+0000 7fb7198cc700 1 --2- 172.21.15.59:0/593082271 >> v2:172.21.15.76:3300/0 conn(0x7fb7140d0870 0x7fb7140d26c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:02.912 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:02.908+0000 7fb7198cc700 1 -- 172.21.15.59:0/593082271 >> 172.21.15.59:0/593082271 conn(0x7fb7140d3ed0 msgr2=0x7fb7140d42e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:57:02.912 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:02.909+0000 7fb7198cc700 1 -- 172.21.15.59:0/593082271 shutdown_connections 2023-11-24T07:57:02.912 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:02.909+0000 7fb7198cc700 1 -- 172.21.15.59:0/593082271 wait complete. 2023-11-24T07:57:02.915 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:02.910+0000 7fb7198cc700 1 Processor -- start 2023-11-24T07:57:02.915 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:02.910+0000 7fb7198cc700 1 -- start start 2023-11-24T07:57:02.915 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:02.911+0000 7fb7198cc700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb7140d0870 0x7fb7141f6de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:57:02.916 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:02.911+0000 7fb7198cc700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb7140d0c50 0x7fb7140ccc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:57:02.916 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:02.911+0000 7fb7198cc700 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fb7140d49f0 0x7fb7140cd1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:57:02.916 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:02.911+0000 7fb7198cc700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7141f7320 con 0x7fb7140d0c50 2023-11-24T07:57:02.916 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:02.911+0000 7fb7198cc700 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb71409ecd0 con 0x7fb7140d0870 2023-11-24T07:57:02.916 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:02.911+0000 7fb7198cc700 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb71409efb0 con 0x7fb7140d49f0 2023-11-24T07:57:02.916 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:02.911+0000 7fb7127fc700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb7140d0c50 0x7fb7140ccc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:57:02.917 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:02.911+0000 7fb7127fc700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb7140d0c50 0x7fb7140ccc80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:50952/0 (socket says 172.21.15.59:50952) 2023-11-24T07:57:02.917 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:02.911+0000 7fb7127fc700 1 -- 172.21.15.59:0/2433178814 learned_addr learned my addr 172.21.15.59:0/2433178814 (peer_addr_for_me v2:172.21.15.59:0/0) 2023-11-24T07:57:02.917 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:02.911+0000 7fb712ffd700 1 --2- 172.21.15.59:0/2433178814 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb7140d0870 0x7fb7141f6de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:57:02.917 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:02.911+0000 7fb7137fe700 1 --2- 172.21.15.59:0/2433178814 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fb7140d49f0 0x7fb7140cd1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:57:02.918 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:02.911+0000 7fb7127fc700 1 -- 172.21.15.59:0/2433178814 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fb7140d49f0 msgr2=0x7fb7140cd1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:57:02.918 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:02.911+0000 7fb7127fc700 1 --2- 172.21.15.59:0/2433178814 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fb7140d49f0 0x7fb7140cd1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:02.918 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:02.911+0000 7fb7127fc700 1 -- 172.21.15.59:0/2433178814 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb7140d0870 msgr2=0x7fb7141f6de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:57:02.918 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:02.911+0000 7fb7127fc700 1 --2- 172.21.15.59:0/2433178814 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb7140d0870 0x7fb7141f6de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:02.918 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:02.911+0000 7fb7127fc700 1 -- 172.21.15.59:0/2433178814 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb70c009a70 con 0x7fb7140d0c50 2023-11-24T07:57:02.918 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:02.911+0000 7fb712ffd700 1 --2- 172.21.15.59:0/2433178814 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb7140d0870 0x7fb7141f6de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T07:57:02.918 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:02.911+0000 7fb7127fc700 1 --2- 172.21.15.59:0/2433178814 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb7140d0c50 0x7fb7140ccc80 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7fb70400c780 tx=0x7fb70400cb40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:57:02.918 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:02.911+0000 7fb70bfff700 1 -- 172.21.15.59:0/2433178814 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb704004c80 con 0x7fb7140d0c50 2023-11-24T07:57:02.919 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:02.911+0000 7fb70bfff700 1 -- 172.21.15.59:0/2433178814 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb704004de0 con 0x7fb7140d0c50 2023-11-24T07:57:02.919 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:02.912+0000 7fb7198cc700 1 -- 172.21.15.59:0/2433178814 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb71409f290 con 0x7fb7140d0c50 2023-11-24T07:57:02.919 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:02.912+0000 7fb70bfff700 1 -- 172.21.15.59:0/2433178814 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7040072d0 con 0x7fb7140d0c50 2023-11-24T07:57:02.919 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:02.912+0000 7fb7198cc700 1 -- 172.21.15.59:0/2433178814 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb71409f850 con 0x7fb7140d0c50 2023-11-24T07:57:02.919 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:02.912+0000 7fb7198cc700 1 -- 172.21.15.59:0/2433178814 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb71409fa10 con 0x7fb7140d0c50 2023-11-24T07:57:02.919 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:02.913+0000 7fb70bfff700 1 -- 172.21.15.59:0/2433178814 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fb704007430 con 0x7fb7140d0c50 2023-11-24T07:57:02.919 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:02.913+0000 7fb70bfff700 1 --2- 172.21.15.59:0/2433178814 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7fb6f406aed0 0x7fb6f406d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:57:02.919 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:02.913+0000 7fb70bfff700 1 -- 172.21.15.59:0/2433178814 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fb704088c50 con 0x7fb7140d0c50 2023-11-24T07:57:02.919 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:02.914+0000 7fb712ffd700 1 --2- 172.21.15.59:0/2433178814 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7fb6f406aed0 0x7fb6f406d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:57:02.920 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:02.914+0000 7fb712ffd700 1 --2- 172.21.15.59:0/2433178814 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7fb6f406aed0 0x7fb6f406d390 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fb6fc009b60 tx=0x7fb6fc008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:57:02.920 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:02.918+0000 7fb70bfff700 1 -- 172.21.15.59:0/2433178814 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fb70408e020 con 0x7fb7140d0c50 2023-11-24T07:57:03.119 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:03.117+0000 7fb7198cc700 1 -- 172.21.15.59:0/2433178814 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fb714067a20 con 0x7fb7140d0c50 2023-11-24T07:57:03.119 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:03.117+0000 7fb70bfff700 1 -- 172.21.15.59:0/2433178814 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v20) v1 ==== 104+0+11 (secure 0 0 0) 0x7fb704018020 con 0x7fb7140d0c50 2023-11-24T07:57:03.119 INFO:teuthology.orchestra.run.smithi059.stdout:pg_num: 32 2023-11-24T07:57:03.122 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:03.120+0000 7fb7198cc700 1 -- 172.21.15.59:0/2433178814 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7fb6f406aed0 msgr2=0x7fb6f406d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:57:03.122 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:03.120+0000 7fb7198cc700 1 --2- 172.21.15.59:0/2433178814 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7fb6f406aed0 0x7fb6f406d390 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fb6fc009b60 tx=0x7fb6fc008040 comp rx=0 tx=0).stop 2023-11-24T07:57:03.122 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:03.120+0000 7fb7198cc700 1 -- 172.21.15.59:0/2433178814 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb7140d0c50 msgr2=0x7fb7140ccc80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:57:03.122 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:03.120+0000 7fb7198cc700 1 --2- 172.21.15.59:0/2433178814 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb7140d0c50 0x7fb7140ccc80 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7fb70400c780 tx=0x7fb70400cb40 comp rx=0 tx=0).stop 2023-11-24T07:57:03.122 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:03.120+0000 7fb7198cc700 1 -- 172.21.15.59:0/2433178814 shutdown_connections 2023-11-24T07:57:03.122 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:03.120+0000 7fb7198cc700 1 --2- 172.21.15.59:0/2433178814 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7fb6f406aed0 0x7fb6f406d390 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:03.122 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:03.120+0000 7fb7198cc700 1 --2- 172.21.15.59:0/2433178814 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fb7140d49f0 0x7fb7140cd1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:03.122 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:03.120+0000 7fb7198cc700 1 --2- 172.21.15.59:0/2433178814 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb7140d0870 0x7fb7141f6de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:03.122 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:03.120+0000 7fb7198cc700 1 --2- 172.21.15.59:0/2433178814 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb7140d0c50 0x7fb7140ccc80 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:03.123 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:03.120+0000 7fb7198cc700 1 -- 172.21.15.59:0/2433178814 >> 172.21.15.59:0/2433178814 conn(0x7fb7140d3ed0 msgr2=0x7fb71406c8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:57:03.123 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:03.120+0000 7fb7198cc700 1 -- 172.21.15.59:0/2433178814 shutdown_connections 2023-11-24T07:57:03.123 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:03.120+0000 7fb7198cc700 1 -- 172.21.15.59:0/2433178814 wait complete. 2023-11-24T07:57:03.133 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-24T07:57:03.261 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:03.259+0000 7efd04193700 1 Processor -- start 2023-11-24T07:57:03.261 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:03.259+0000 7efd04193700 1 -- start start 2023-11-24T07:57:03.263 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:03.260+0000 7efd04193700 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7efcfc143ae0 0x7efcfc0b51b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:57:03.264 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:03.260+0000 7efd04193700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7efcfc0722c0 0x7efcfc06e510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:57:03.264 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:03.260+0000 7efd04193700 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efcfc143ec0 con 0x7efcfc1440f0 2023-11-24T07:57:03.264 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:03.260+0000 7efd04193700 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efcfc0b40a0 con 0x7efcfc0722c0 2023-11-24T07:57:03.264 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:03.260+0000 7efd04193700 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efcfc0b3ae0 con 0x7efcfc143ae0 2023-11-24T07:57:03.264 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:03.260+0000 7efd02730700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7efcfc0722c0 0x7efcfc06e510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:57:03.264 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:03.260+0000 7efd0172e700 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7efcfc1440f0 0x7efcfc071bb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:54432/0 (socket says 172.21.15.59:54432) 2023-11-24T07:57:03.264 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:03.260+0000 7efd0172e700 1 -- 172.21.15.59:0/1116343073 learned_addr learned my addr 172.21.15.59:0/1116343073 (peer_addr_for_me v1:172.21.15.59:0/0) 2023-11-24T07:57:03.264 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:03.261+0000 7efd01f2f700 1 --2- 172.21.15.59:0/1116343073 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7efcfc143ae0 0x7efcfc0b51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:57:03.264 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:03.261+0000 7efd02730700 1 -- 172.21.15.59:0/1116343073 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7efcfc143ae0 msgr2=0x7efcfc0b51b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T07:57:03.265 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:03.261+0000 7efd02730700 1 --2- 172.21.15.59:0/1116343073 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7efcfc143ae0 0x7efcfc0b51b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:03.265 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:03.261+0000 7efd02730700 1 -- 172.21.15.59:0/1116343073 >> v1:172.21.15.59:6789/0 conn(0x7efcfc1440f0 legacy=0x7efcfc071bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:57:03.265 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:03.261+0000 7efd02730700 1 -- 172.21.15.59:0/1116343073 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efcfc0b3dc0 con 0x7efcfc0722c0 2023-11-24T07:57:03.265 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:03.261+0000 7efd02730700 1 --2- 172.21.15.59:0/1116343073 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7efcfc0722c0 0x7efcfc06e510 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7efcf4009d90 tx=0x7efcf400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=64a258c72a5d2dfe server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:57:03.265 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:03.261+0000 7efd00f2d700 1 -- 172.21.15.59:0/1116343073 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efcf400e070 con 0x7efcfc0722c0 2023-11-24T07:57:03.265 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:03.261+0000 7efd00f2d700 1 -- 172.21.15.59:0/1116343073 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efcf400bbd0 con 0x7efcfc0722c0 2023-11-24T07:57:03.265 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:03.261+0000 7efd00f2d700 1 -- 172.21.15.59:0/1116343073 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efcf4012430 con 0x7efcfc0722c0 2023-11-24T07:57:03.265 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:03.262+0000 7efd04193700 1 -- 172.21.15.59:0/1116343073 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7efcfc0722c0 msgr2=0x7efcfc06e510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:57:03.265 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:03.262+0000 7efd04193700 1 --2- 172.21.15.59:0/1116343073 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7efcfc0722c0 0x7efcfc06e510 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7efcf4009d90 tx=0x7efcf400b330 comp rx=0 tx=0).stop 2023-11-24T07:57:03.266 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:03.262+0000 7efd04193700 1 -- 172.21.15.59:0/1116343073 shutdown_connections 2023-11-24T07:57:03.266 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:03.262+0000 7efd04193700 1 --2- 172.21.15.59:0/1116343073 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7efcfc0722c0 0x7efcfc06e510 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:03.266 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:03.262+0000 7efd04193700 1 --2- 172.21.15.59:0/1116343073 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7efcfc143ae0 0x7efcfc0b51b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:03.266 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:03.262+0000 7efd04193700 1 -- 172.21.15.59:0/1116343073 >> 172.21.15.59:0/1116343073 conn(0x7efcfc0b2740 msgr2=0x7efcfc0b2b20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:57:03.266 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:03.262+0000 7efd04193700 1 -- 172.21.15.59:0/1116343073 shutdown_connections 2023-11-24T07:57:03.266 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:03.262+0000 7efd04193700 1 -- 172.21.15.59:0/1116343073 wait complete. 2023-11-24T07:57:03.269 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:03.263+0000 7efd04193700 1 Processor -- start 2023-11-24T07:57:03.269 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:03.263+0000 7efd04193700 1 -- start start 2023-11-24T07:57:03.269 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:03.264+0000 7efd04193700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7efcfc0722c0 0x7efcfc13b130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:57:03.269 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:03.264+0000 7efd04193700 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7efcfc143ae0 0x7efcfc141f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:57:03.269 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:03.264+0000 7efd04193700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7efcfc1440f0 0x7efcfc142480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:57:03.269 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:03.264+0000 7efd04193700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efcfc1f5ba0 con 0x7efcfc1440f0 2023-11-24T07:57:03.269 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:03.264+0000 7efd04193700 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efcfc158fe0 con 0x7efcfc0722c0 2023-11-24T07:57:03.269 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:03.264+0000 7efd04193700 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efcfc1f54e0 con 0x7efcfc143ae0 2023-11-24T07:57:03.269 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:03.264+0000 7efd02730700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7efcfc1440f0 0x7efcfc142480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:57:03.270 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:03.264+0000 7efd02730700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7efcfc1440f0 0x7efcfc142480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:50968/0 (socket says 172.21.15.59:50968) 2023-11-24T07:57:03.270 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:03.264+0000 7efd02730700 1 -- 172.21.15.59:0/2267380064 learned_addr learned my addr 172.21.15.59:0/2267380064 (peer_addr_for_me v2:172.21.15.59:0/0) 2023-11-24T07:57:03.270 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:03.264+0000 7efd0172e700 1 --2- 172.21.15.59:0/2267380064 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7efcfc143ae0 0x7efcfc141f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:57:03.270 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:03.264+0000 7efd01f2f700 1 --2- 172.21.15.59:0/2267380064 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7efcfc0722c0 0x7efcfc13b130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:57:03.270 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:03.264+0000 7efd02730700 1 -- 172.21.15.59:0/2267380064 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7efcfc143ae0 msgr2=0x7efcfc141f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:57:03.270 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:03.264+0000 7efd02730700 1 --2- 172.21.15.59:0/2267380064 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7efcfc143ae0 0x7efcfc141f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:03.270 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:03.264+0000 7efd02730700 1 -- 172.21.15.59:0/2267380064 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7efcfc0722c0 msgr2=0x7efcfc13b130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:57:03.270 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:03.264+0000 7efd02730700 1 --2- 172.21.15.59:0/2267380064 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7efcfc0722c0 0x7efcfc13b130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:03.270 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:03.264+0000 7efd02730700 1 -- 172.21.15.59:0/2267380064 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efcf4009a70 con 0x7efcfc1440f0 2023-11-24T07:57:03.270 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:03.264+0000 7efd0172e700 1 --2- 172.21.15.59:0/2267380064 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7efcfc143ae0 0x7efcfc141f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-24T07:57:03.270 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:03.264+0000 7efd02730700 1 --2- 172.21.15.59:0/2267380064 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7efcfc1440f0 0x7efcfc142480 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7efcf4014040 tx=0x7efcf4004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:57:03.271 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:03.264+0000 7efcf27fc700 1 -- 172.21.15.59:0/2267380064 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efcf4012430 con 0x7efcfc1440f0 2023-11-24T07:57:03.271 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:03.264+0000 7efcf27fc700 1 -- 172.21.15.59:0/2267380064 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efcf4020070 con 0x7efcfc1440f0 2023-11-24T07:57:03.271 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:03.264+0000 7efd04193700 1 -- 172.21.15.59:0/2267380064 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efcfc1f57c0 con 0x7efcfc1440f0 2023-11-24T07:57:03.271 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:03.264+0000 7efcf27fc700 1 -- 172.21.15.59:0/2267380064 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efcf4003ea0 con 0x7efcfc1440f0 2023-11-24T07:57:03.271 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:03.264+0000 7efd04193700 1 -- 172.21.15.59:0/2267380064 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efcfc1f5fc0 con 0x7efcfc1440f0 2023-11-24T07:57:03.271 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:03.265+0000 7efd04193700 1 -- 172.21.15.59:0/2267380064 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efcfc075480 con 0x7efcfc1440f0 2023-11-24T07:57:03.271 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:03.266+0000 7efcf27fc700 1 -- 172.21.15.59:0/2267380064 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7efcf4004280 con 0x7efcfc1440f0 2023-11-24T07:57:03.271 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:03.266+0000 7efcf27fc700 1 --2- 172.21.15.59:0/2267380064 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7efce006aed0 0x7efce006d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:57:03.271 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:03.266+0000 7efcf27fc700 1 -- 172.21.15.59:0/2267380064 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7efcf4022070 con 0x7efcfc1440f0 2023-11-24T07:57:03.271 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:03.266+0000 7efd01f2f700 1 --2- 172.21.15.59:0/2267380064 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7efce006aed0 0x7efce006d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:57:03.271 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:03.267+0000 7efd01f2f700 1 --2- 172.21.15.59:0/2267380064 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7efce006aed0 0x7efce006d390 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7efcec009930 tx=0x7efcec008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:57:03.272 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:03.271+0000 7efcf27fc700 1 -- 172.21.15.59:0/2267380064 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7efcf4056990 con 0x7efcfc1440f0 2023-11-24T07:57:03.473 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:03.472+0000 7efd04193700 1 -- 172.21.15.59:0/2267380064 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7efcfc1f5ce0 con 0x7efcfc1440f0 2023-11-24T07:57:03.474 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:03.472+0000 7efcf27fc700 1 -- 172.21.15.59:0/2267380064 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v20) v1 ==== 100+0+11 (secure 0 0 0) 0x7efcfc1f5ce0 con 0x7efcfc1440f0 2023-11-24T07:57:03.474 INFO:teuthology.orchestra.run.smithi059.stdout:pg_num: 64 2023-11-24T07:57:03.476 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:03.474+0000 7efd04193700 1 -- 172.21.15.59:0/2267380064 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7efce006aed0 msgr2=0x7efce006d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:57:03.476 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:03.474+0000 7efd04193700 1 --2- 172.21.15.59:0/2267380064 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7efce006aed0 0x7efce006d390 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7efcec009930 tx=0x7efcec008040 comp rx=0 tx=0).stop 2023-11-24T07:57:03.477 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:03.474+0000 7efd04193700 1 -- 172.21.15.59:0/2267380064 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7efcfc1440f0 msgr2=0x7efcfc142480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:57:03.477 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:03.474+0000 7efd04193700 1 --2- 172.21.15.59:0/2267380064 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7efcfc1440f0 0x7efcfc142480 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7efcf4014040 tx=0x7efcf4004ad0 comp rx=0 tx=0).stop 2023-11-24T07:57:03.477 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:03.474+0000 7efd04193700 1 -- 172.21.15.59:0/2267380064 shutdown_connections 2023-11-24T07:57:03.477 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:03.474+0000 7efd04193700 1 --2- 172.21.15.59:0/2267380064 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7efce006aed0 0x7efce006d390 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:03.477 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:03.474+0000 7efd04193700 1 --2- 172.21.15.59:0/2267380064 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7efcfc1440f0 0x7efcfc142480 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:03.477 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:03.474+0000 7efd04193700 1 --2- 172.21.15.59:0/2267380064 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7efcfc0722c0 0x7efcfc13b130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:03.477 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:03.474+0000 7efd04193700 1 --2- 172.21.15.59:0/2267380064 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7efcfc143ae0 0x7efcfc141f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:03.477 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:03.474+0000 7efd04193700 1 -- 172.21.15.59:0/2267380064 >> 172.21.15.59:0/2267380064 conn(0x7efcfc0b2740 msgr2=0x7efcfc15bf90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:57:03.477 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:03.474+0000 7efd04193700 1 -- 172.21.15.59:0/2267380064 shutdown_connections 2023-11-24T07:57:03.477 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:03.474+0000 7efd04193700 1 -- 172.21.15.59:0/2267380064 wait complete. 2023-11-24T07:57:03.488 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-11-24T07:57:03.614 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:03.612+0000 7fe5f4f0a700 1 Processor -- start 2023-11-24T07:57:03.614 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:03.612+0000 7fe5f4f0a700 1 -- start start 2023-11-24T07:57:03.616 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:03.613+0000 7fe5f4f0a700 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fe5f00d0080 0x7fe5f00b5890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:57:03.617 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:03.613+0000 7fe5f4f0a700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fe5f006b6e0 0x7fe5f006dfb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:57:03.617 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:03.613+0000 7fe5f4f0a700 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe5f0147bb0 con 0x7fe5f0071bb0 2023-11-24T07:57:03.617 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:03.613+0000 7fe5f4f0a700 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe5f0072cd0 con 0x7fe5f006b6e0 2023-11-24T07:57:03.617 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:03.613+0000 7fe5f4f0a700 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe5f00b5dd0 con 0x7fe5f00d0080 2023-11-24T07:57:03.618 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:03.613+0000 7fe5eed9d700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fe5f006b6e0 0x7fe5f006dfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:57:03.618 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:03.613+0000 7fe5eed9d700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fe5f006b6e0 0x7fe5f006dfb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:50976/0 (socket says 172.21.15.59:50976) 2023-11-24T07:57:03.618 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:03.613+0000 7fe5eed9d700 1 -- 172.21.15.59:0/3557300604 learned_addr learned my addr 172.21.15.59:0/3557300604 (peer_addr_for_me v2:172.21.15.59:0/0) 2023-11-24T07:57:03.618 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:03.614+0000 7fe5ee59c700 1 --2- 172.21.15.59:0/3557300604 >> v2:172.21.15.76:3300/0 conn(0x7fe5f00d0080 0x7fe5f00b5890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:57:03.618 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:03.614+0000 7fe5eed9d700 1 -- 172.21.15.59:0/3557300604 >> v2:172.21.15.76:3300/0 conn(0x7fe5f00d0080 msgr2=0x7fe5f00b5890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T07:57:03.618 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:03.614+0000 7fe5eed9d700 1 --2- 172.21.15.59:0/3557300604 >> v2:172.21.15.76:3300/0 conn(0x7fe5f00d0080 0x7fe5f00b5890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:03.619 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:03.614+0000 7fe5eed9d700 1 -- 172.21.15.59:0/3557300604 >> v1:172.21.15.76:6789/0 conn(0x7fe5f0071bb0 legacy=0x7fe5f006afd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T07:57:03.619 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:03.614+0000 7fe5eed9d700 1 -- 172.21.15.59:0/3557300604 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe5f00d0460 con 0x7fe5f006b6e0 2023-11-24T07:57:03.619 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:03.614+0000 7fe5eed9d700 1 --2- 172.21.15.59:0/3557300604 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe5f006b6e0 0x7fe5f006dfb0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7fe5f0062d30 tx=0x7fe5e800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d27582be77e2aeef server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:57:03.619 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:03.614+0000 7fe5ed59a700 1 -- 172.21.15.59:0/3557300604 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5e800e070 con 0x7fe5f006b6e0 2023-11-24T07:57:03.619 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:03.614+0000 7fe5ed59a700 1 -- 172.21.15.59:0/3557300604 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe5e8003a20 con 0x7fe5f006b6e0 2023-11-24T07:57:03.619 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:03.614+0000 7fe5ed59a700 1 -- 172.21.15.59:0/3557300604 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5e8003dd0 con 0x7fe5f006b6e0 2023-11-24T07:57:03.619 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:03.615+0000 7fe5f4f0a700 1 -- 172.21.15.59:0/3557300604 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe5f006b6e0 msgr2=0x7fe5f006dfb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:57:03.619 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:03.615+0000 7fe5f4f0a700 1 --2- 172.21.15.59:0/3557300604 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe5f006b6e0 0x7fe5f006dfb0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7fe5f0062d30 tx=0x7fe5e800b330 comp rx=0 tx=0).stop 2023-11-24T07:57:03.620 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:03.615+0000 7fe5f4f0a700 1 -- 172.21.15.59:0/3557300604 shutdown_connections 2023-11-24T07:57:03.620 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:03.615+0000 7fe5f4f0a700 1 --2- 172.21.15.59:0/3557300604 >> v2:172.21.15.76:3300/0 conn(0x7fe5f00d0080 0x7fe5f00b5890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:03.620 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:03.615+0000 7fe5f4f0a700 1 --2- 172.21.15.59:0/3557300604 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe5f006b6e0 0x7fe5f006dfb0 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:03.620 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:03.615+0000 7fe5f4f0a700 1 -- 172.21.15.59:0/3557300604 >> 172.21.15.59:0/3557300604 conn(0x7fe5f00b27d0 msgr2=0x7fe5f00b2be0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:57:03.620 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:03.615+0000 7fe5f4f0a700 1 -- 172.21.15.59:0/3557300604 shutdown_connections 2023-11-24T07:57:03.620 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:03.615+0000 7fe5f4f0a700 1 -- 172.21.15.59:0/3557300604 wait complete. 2023-11-24T07:57:03.623 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:03.616+0000 7fe5f4f0a700 1 Processor -- start 2023-11-24T07:57:03.623 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:03.616+0000 7fe5f4f0a700 1 -- start start 2023-11-24T07:57:03.623 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:03.617+0000 7fe5f4f0a700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe5f006b6e0 0x7fe5f01f36e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:57:03.623 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:03.617+0000 7fe5f4f0a700 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fe5f0071bb0 0x7fe5f013ea50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:57:03.623 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:03.617+0000 7fe5f4f0a700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe5f00d0080 0x7fe5f013ef90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:57:03.623 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:03.617+0000 7fe5f4f0a700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5f013e2c0 con 0x7fe5f00d0080 2023-11-24T07:57:03.623 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:03.617+0000 7fe5f4f0a700 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5f013c520 con 0x7fe5f006b6e0 2023-11-24T07:57:03.624 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:03.617+0000 7fe5f4f0a700 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe5f013d640 con 0x7fe5f0071bb0 2023-11-24T07:57:03.624 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:03.617+0000 7fe5eed9d700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe5f00d0080 0x7fe5f013ef90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:57:03.624 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:03.617+0000 7fe5eed9d700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe5f00d0080 0x7fe5f013ef90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:50982/0 (socket says 172.21.15.59:50982) 2023-11-24T07:57:03.624 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:03.617+0000 7fe5eed9d700 1 -- 172.21.15.59:0/3641974376 learned_addr learned my addr 172.21.15.59:0/3641974376 (peer_addr_for_me v2:172.21.15.59:0/0) 2023-11-24T07:57:03.624 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:03.617+0000 7fe5ee59c700 1 --2- 172.21.15.59:0/3641974376 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe5f006b6e0 0x7fe5f01f36e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:57:03.624 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:03.617+0000 7fe5edd9b700 1 --2- 172.21.15.59:0/3641974376 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fe5f0071bb0 0x7fe5f013ea50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:57:03.624 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:03.617+0000 7fe5eed9d700 1 -- 172.21.15.59:0/3641974376 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fe5f0071bb0 msgr2=0x7fe5f013ea50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:57:03.624 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:03.617+0000 7fe5eed9d700 1 --2- 172.21.15.59:0/3641974376 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fe5f0071bb0 0x7fe5f013ea50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:03.625 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:03.617+0000 7fe5eed9d700 1 -- 172.21.15.59:0/3641974376 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe5f006b6e0 msgr2=0x7fe5f01f36e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:57:03.625 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:03.617+0000 7fe5eed9d700 1 --2- 172.21.15.59:0/3641974376 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe5f006b6e0 0x7fe5f01f36e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:03.625 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:03.617+0000 7fe5eed9d700 1 -- 172.21.15.59:0/3641974376 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe5e8009b30 con 0x7fe5f00d0080 2023-11-24T07:57:03.625 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:03.618+0000 7fe5eed9d700 1 --2- 172.21.15.59:0/3641974376 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe5f00d0080 0x7fe5f013ef90 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7fe5e800b370 tx=0x7fe5e8004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:57:03.625 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:03.618+0000 7fe5deffd700 1 -- 172.21.15.59:0/3641974376 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5e8004440 con 0x7fe5f00d0080 2023-11-24T07:57:03.625 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:03.618+0000 7fe5deffd700 1 -- 172.21.15.59:0/3641974376 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe5e80092e0 con 0x7fe5f00d0080 2023-11-24T07:57:03.625 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:03.618+0000 7fe5f4f0a700 1 -- 172.21.15.59:0/3641974376 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe5f013d920 con 0x7fe5f00d0080 2023-11-24T07:57:03.625 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:03.618+0000 7fe5deffd700 1 -- 172.21.15.59:0/3641974376 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5e8004800 con 0x7fe5f00d0080 2023-11-24T07:57:03.625 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:03.618+0000 7fe5f4f0a700 1 -- 172.21.15.59:0/3641974376 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe5f013dee0 con 0x7fe5f00d0080 2023-11-24T07:57:03.626 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:03.619+0000 7fe5f4f0a700 1 -- 172.21.15.59:0/3641974376 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe5f0068670 con 0x7fe5f00d0080 2023-11-24T07:57:03.626 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:03.619+0000 7fe5deffd700 1 -- 172.21.15.59:0/3641974376 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fe5e8012430 con 0x7fe5f00d0080 2023-11-24T07:57:03.626 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:03.620+0000 7fe5deffd700 1 --2- 172.21.15.59:0/3641974376 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7fe5d006aed0 0x7fe5d006d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:57:03.626 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:03.620+0000 7fe5deffd700 1 -- 172.21.15.59:0/3641974376 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fe5e808a810 con 0x7fe5f00d0080 2023-11-24T07:57:03.626 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:03.620+0000 7fe5ee59c700 1 --2- 172.21.15.59:0/3641974376 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7fe5d006aed0 0x7fe5d006d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:57:03.626 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:03.620+0000 7fe5ee59c700 1 --2- 172.21.15.59:0/3641974376 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7fe5d006aed0 0x7fe5d006d390 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fe5e4006b20 tx=0x7fe5e4008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:57:03.626 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:03.624+0000 7fe5deffd700 1 -- 172.21.15.59:0/3641974376 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fe5e8055950 con 0x7fe5f00d0080 2023-11-24T07:57:03.843 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:03.842+0000 7fe5f4f0a700 1 -- 172.21.15.59:0/3641974376 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7fe5f0139fc0 con 0x7fe5f00d0080 2023-11-24T07:57:03.843 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:03.842+0000 7fe5deffd700 1 -- 172.21.15.59:0/3641974376 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v20) v1 ==== 80+0+0 (secure 0 0 0) 0x7fe5e8017080 con 0x7fe5f00d0080 2023-11-24T07:57:03.846 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:03.844+0000 7fe5f4f0a700 1 -- 172.21.15.59:0/3641974376 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7fe5d006aed0 msgr2=0x7fe5d006d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:57:03.846 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:03.844+0000 7fe5f4f0a700 1 --2- 172.21.15.59:0/3641974376 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7fe5d006aed0 0x7fe5d006d390 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fe5e4006b20 tx=0x7fe5e4008040 comp rx=0 tx=0).stop 2023-11-24T07:57:03.847 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:03.844+0000 7fe5f4f0a700 1 -- 172.21.15.59:0/3641974376 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe5f00d0080 msgr2=0x7fe5f013ef90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:57:03.847 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:03.844+0000 7fe5f4f0a700 1 --2- 172.21.15.59:0/3641974376 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe5f00d0080 0x7fe5f013ef90 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7fe5e800b370 tx=0x7fe5e8004ad0 comp rx=0 tx=0).stop 2023-11-24T07:57:03.847 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:03.844+0000 7fe5f4f0a700 1 -- 172.21.15.59:0/3641974376 shutdown_connections 2023-11-24T07:57:03.847 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:03.844+0000 7fe5f4f0a700 1 --2- 172.21.15.59:0/3641974376 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7fe5d006aed0 0x7fe5d006d390 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:03.847 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:03.844+0000 7fe5f4f0a700 1 --2- 172.21.15.59:0/3641974376 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe5f00d0080 0x7fe5f013ef90 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:03.847 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:03.844+0000 7fe5f4f0a700 1 --2- 172.21.15.59:0/3641974376 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe5f006b6e0 0x7fe5f01f36e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:03.847 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:03.844+0000 7fe5f4f0a700 1 --2- 172.21.15.59:0/3641974376 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fe5f0071bb0 0x7fe5f013ea50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:03.847 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:03.844+0000 7fe5f4f0a700 1 -- 172.21.15.59:0/3641974376 >> 172.21.15.59:0/3641974376 conn(0x7fe5f00b27d0 msgr2=0x7fe5f0143730 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:57:03.847 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:03.844+0000 7fe5f4f0a700 1 -- 172.21.15.59:0/3641974376 shutdown_connections 2023-11-24T07:57:03.848 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:03.844+0000 7fe5f4f0a700 1 -- 172.21.15.59:0/3641974376 wait complete. 2023-11-24T07:57:03.848 INFO:teuthology.orchestra.run.smithi059.stderr:listed 0 entries 2023-11-24T07:57:03.858 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2023-11-24T07:57:03.858 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T07:57:03.858 DEBUG:teuthology.orchestra.run.smithi059:> sudo umount /home/ubuntu/cephtest/mnt.0 -f 2023-11-24T07:57:03.906 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi059.front.sepia.ceph.com 2023-11-24T07:57:03.906 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T07:57:03.906 DEBUG:teuthology.orchestra.run.smithi059:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-11-24T07:57:03.951 DEBUG:teuthology.orchestra.run.smithi076:> set -ex 2023-11-24T07:57:03.951 DEBUG:teuthology.orchestra.run.smithi076:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-24T07:57:03.972 DEBUG:teuthology.orchestra.run.smithi076:> set -ex 2023-11-24T07:57:03.973 DEBUG:teuthology.orchestra.run.smithi076:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-24T07:57:04.033 DEBUG:teuthology.orchestra.run.smithi076:> set -ex 2023-11-24T07:57:04.034 DEBUG:teuthology.orchestra.run.smithi076:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-24T07:57:04.095 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-24T07:57:04.220 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:04.218+0000 7fc3d3ecd700 1 Processor -- start 2023-11-24T07:57:04.220 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:04.218+0000 7fc3d3ecd700 1 -- start start 2023-11-24T07:57:04.222 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:04.219+0000 7fc3d3ecd700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fc3cc0cf2b0 0x7fc3cc15c1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:57:04.222 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:04.219+0000 7fc3d3ecd700 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc3cc0d0a90 con 0x7fc3cc15bde0 2023-11-24T07:57:04.222 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:04.219+0000 7fc3d3ecd700 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc3cc06ee10 con 0x7fc3cc0ce5c0 2023-11-24T07:57:04.222 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:04.219+0000 7fc3d3ecd700 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc3cc06f0f0 con 0x7fc3cc0cf2b0 2023-11-24T07:57:04.222 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:04.219+0000 7fc3d1468700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fc3cc0cf2b0 0x7fc3cc15c1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:57:04.222 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:04.219+0000 7fc3d1468700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fc3cc0cf2b0 0x7fc3cc15c1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:50988/0 (socket says 172.21.15.59:50988) 2023-11-24T07:57:04.223 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:04.219+0000 7fc3d1468700 1 -- 172.21.15.59:0/146303139 learned_addr learned my addr 172.21.15.59:0/146303139 (peer_addr_for_me v2:172.21.15.59:0/0) 2023-11-24T07:57:04.223 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:04.219+0000 7fc3d1468700 1 -- 172.21.15.59:0/146303139 >> v1:172.21.15.76:6789/0 conn(0x7fc3cc0ce5c0 legacy=0x7fc3cc06fbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:57:04.223 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:04.219+0000 7fc3d1468700 1 -- 172.21.15.59:0/146303139 >> v1:172.21.15.59:6789/0 conn(0x7fc3cc15bde0 legacy=0x7fc3cc0d01b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:57:04.223 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:04.219+0000 7fc3d1468700 1 -- 172.21.15.59:0/146303139 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc3cc06f3d0 con 0x7fc3cc0cf2b0 2023-11-24T07:57:04.223 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:04.220+0000 7fc3d1468700 1 --2- 172.21.15.59:0/146303139 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc3cc0cf2b0 0x7fc3cc15c1c0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7fc3c80098f0 tx=0x7fc3c8009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a4c3508d92b06799 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:57:04.223 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:04.220+0000 7fc3d0c67700 1 -- 172.21.15.59:0/146303139 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3c800b8a0 con 0x7fc3cc0cf2b0 2023-11-24T07:57:04.223 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:04.220+0000 7fc3d0c67700 1 -- 172.21.15.59:0/146303139 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc3c800ba00 con 0x7fc3cc0cf2b0 2023-11-24T07:57:04.223 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:04.220+0000 7fc3d0c67700 1 -- 172.21.15.59:0/146303139 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3c800bd90 con 0x7fc3cc0cf2b0 2023-11-24T07:57:04.223 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:04.220+0000 7fc3d3ecd700 1 -- 172.21.15.59:0/146303139 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc3cc0cf2b0 msgr2=0x7fc3cc15c1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:57:04.223 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:04.220+0000 7fc3d3ecd700 1 --2- 172.21.15.59:0/146303139 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc3cc0cf2b0 0x7fc3cc15c1c0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7fc3c80098f0 tx=0x7fc3c8009c00 comp rx=0 tx=0).stop 2023-11-24T07:57:04.223 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:04.220+0000 7fc3d3ecd700 1 -- 172.21.15.59:0/146303139 shutdown_connections 2023-11-24T07:57:04.224 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:04.220+0000 7fc3d3ecd700 1 --2- 172.21.15.59:0/146303139 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc3cc0cf2b0 0x7fc3cc15c1c0 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:04.224 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:04.220+0000 7fc3d3ecd700 1 -- 172.21.15.59:0/146303139 >> 172.21.15.59:0/146303139 conn(0x7fc3cc0b3720 msgr2=0x7fc3cc0b3b30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:57:04.224 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:04.220+0000 7fc3d3ecd700 1 -- 172.21.15.59:0/146303139 shutdown_connections 2023-11-24T07:57:04.224 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:04.220+0000 7fc3d3ecd700 1 -- 172.21.15.59:0/146303139 wait complete. 2023-11-24T07:57:04.226 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:04.222+0000 7fc3d3ecd700 1 Processor -- start 2023-11-24T07:57:04.226 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:04.222+0000 7fc3d3ecd700 1 -- start start 2023-11-24T07:57:04.226 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:04.222+0000 7fc3d3ecd700 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fc3cc0ce5c0 0x7fc3cc159a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:57:04.226 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:04.222+0000 7fc3d3ecd700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc3cc0cf2b0 0x7fc3cc0b4810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:57:04.226 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:04.222+0000 7fc3d3ecd700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc3cc15bde0 0x7fc3cc0b4d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:57:04.226 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:04.222+0000 7fc3d3ecd700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3cc159fb0 con 0x7fc3cc15bde0 2023-11-24T07:57:04.226 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:04.222+0000 7fc3d3ecd700 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3cc0c9820 con 0x7fc3cc0cf2b0 2023-11-24T07:57:04.226 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:04.222+0000 7fc3d3ecd700 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc3cc0b7100 con 0x7fc3cc0ce5c0 2023-11-24T07:57:04.227 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:04.222+0000 7fc3d246a700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc3cc15bde0 0x7fc3cc0b4d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:57:04.227 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:04.222+0000 7fc3d246a700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc3cc15bde0 0x7fc3cc0b4d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:50996/0 (socket says 172.21.15.59:50996) 2023-11-24T07:57:04.227 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:04.222+0000 7fc3d246a700 1 -- 172.21.15.59:0/3707919168 learned_addr learned my addr 172.21.15.59:0/3707919168 (peer_addr_for_me v2:172.21.15.59:0/0) 2023-11-24T07:57:04.227 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:04.223+0000 7fc3d1468700 1 --2- 172.21.15.59:0/3707919168 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc3cc0cf2b0 0x7fc3cc0b4810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:57:04.227 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:04.223+0000 7fc3d1c69700 1 --2- 172.21.15.59:0/3707919168 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fc3cc0ce5c0 0x7fc3cc159a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:57:04.227 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:04.223+0000 7fc3d246a700 1 -- 172.21.15.59:0/3707919168 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fc3cc0ce5c0 msgr2=0x7fc3cc159a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:57:04.227 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:04.223+0000 7fc3d246a700 1 --2- 172.21.15.59:0/3707919168 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fc3cc0ce5c0 0x7fc3cc159a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:04.227 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:04.223+0000 7fc3d246a700 1 -- 172.21.15.59:0/3707919168 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc3cc0cf2b0 msgr2=0x7fc3cc0b4810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:57:04.227 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:04.223+0000 7fc3d246a700 1 --2- 172.21.15.59:0/3707919168 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc3cc0cf2b0 0x7fc3cc0b4810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:04.227 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:04.223+0000 7fc3d246a700 1 -- 172.21.15.59:0/3707919168 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc3c8009550 con 0x7fc3cc15bde0 2023-11-24T07:57:04.227 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:04.223+0000 7fc3d246a700 1 --2- 172.21.15.59:0/3707919168 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc3cc15bde0 0x7fc3cc0b4d50 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7fc3c400cc50 tx=0x7fc3c400a340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:57:04.228 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:04.223+0000 7fc3c27fc700 1 -- 172.21.15.59:0/3707919168 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3c400ad80 con 0x7fc3cc15bde0 2023-11-24T07:57:04.228 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:04.223+0000 7fc3c27fc700 1 -- 172.21.15.59:0/3707919168 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc3c400ce50 con 0x7fc3cc15bde0 2023-11-24T07:57:04.228 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:04.223+0000 7fc3d3ecd700 1 -- 172.21.15.59:0/3707919168 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc3cc0b6b40 con 0x7fc3cc15bde0 2023-11-24T07:57:04.228 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:04.223+0000 7fc3c27fc700 1 -- 172.21.15.59:0/3707919168 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3c4003c40 con 0x7fc3cc15bde0 2023-11-24T07:57:04.228 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:04.223+0000 7fc3d3ecd700 1 -- 172.21.15.59:0/3707919168 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc3cc06c260 con 0x7fc3cc15bde0 2023-11-24T07:57:04.228 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:04.224+0000 7fc3d3ecd700 1 -- 172.21.15.59:0/3707919168 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc3cc068670 con 0x7fc3cc15bde0 2023-11-24T07:57:04.228 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:04.225+0000 7fc3c27fc700 1 -- 172.21.15.59:0/3707919168 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fc3c40050d0 con 0x7fc3cc15bde0 2023-11-24T07:57:04.229 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:04.225+0000 7fc3c27fc700 1 --2- 172.21.15.59:0/3707919168 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7fc3b006aed0 0x7fc3b006d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:57:04.229 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:04.225+0000 7fc3c27fc700 1 -- 172.21.15.59:0/3707919168 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fc3c4089660 con 0x7fc3cc15bde0 2023-11-24T07:57:04.229 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:04.225+0000 7fc3d1c69700 1 --2- 172.21.15.59:0/3707919168 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7fc3b006aed0 0x7fc3b006d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:57:04.229 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:04.226+0000 7fc3d1c69700 1 --2- 172.21.15.59:0/3707919168 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7fc3b006aed0 0x7fc3b006d390 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fc3bc000f60 tx=0x7fc3bc009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:57:04.231 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:04.230+0000 7fc3c27fc700 1 -- 172.21.15.59:0/3707919168 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fc3c40547a0 con 0x7fc3cc15bde0 2023-11-24T07:57:04.436 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:04.435+0000 7fc3d3ecd700 1 -- 172.21.15.59:0/3707919168 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fc3cc0af820 con 0x7fc3cc15bde0 2023-11-24T07:57:04.437 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:04.436+0000 7fc3c27fc700 1 -- 172.21.15.59:0/3707919168 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) v1 ==== 74+0+16437 (secure 0 0 0) 0x7fc3c4057dc0 con 0x7fc3cc15bde0 2023-11-24T07:57:04.438 INFO:teuthology.orchestra.run.smithi059.stdout: 2023-11-24T07:57:04.438 INFO:teuthology.orchestra.run.smithi059.stdout:{"epoch":20,"fsid":"9f0e8741-b8f9-48d5-a2c3-09f45903303d","created":"2023-11-24T07:56:06.552729+0000","modified":"2023-11-24T07:56:25.705091+0000","last_up_change":"2023-11-24T07:56:17.372352+0000","last_in_change":"2023-11-24T07:56:09.073250+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-24T07:56:18.352823+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-11-24T07:56:22.185140+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-11-24T07:56:23.084326+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"6d8f9e9b-1782-4e41-928b-a39f2a8ebc06","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6800","nonce":3373878772},{"type":"v1","addr":"172.21.15.59:6801","nonce":3373878772}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6802","nonce":3373878772},{"type":"v1","addr":"172.21.15.59:6803","nonce":3373878772}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6806","nonce":3373878772},{"type":"v1","addr":"172.21.15.59:6807","nonce":3373878772}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6804","nonce":3373878772},{"type":"v1","addr":"172.21.15.59:6805","nonce":3373878772}]},"public_addr":"172.21.15.59:6801/3373878772","cluster_addr":"172.21.15.59:6803/3373878772","heartbeat_back_addr":"172.21.15.59:6807/3373878772","heartbeat_front_addr":"172.21.15.59:6805/3373878772","state":["exists","up"]},{"osd":1,"uuid":"e8d882aa-52e1-4c04-9e1e-c44ef9204a5f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6824","nonce":648258563},{"type":"v1","addr":"172.21.15.59:6825","nonce":648258563}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6826","nonce":648258563},{"type":"v1","addr":"172.21.15.59:6827","nonce":648258563}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6830","nonce":648258563},{"type":"v1","addr":"172.21.15.59:6831","nonce":648258563}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6828","nonce":648258563},{"type":"v1","addr":"172.21.15.59:6829","nonce":648258563}]},"public_addr":"172.21.15.59:6825/648258563","cluster_addr":"172.21.15.59:6827/648258563","heartbeat_back_addr":"172.21.15.59:6831/648258563","heartbeat_front_addr":"172.21.15.59:6829/648258563","state":["exists","up"]},{"osd":2,"uuid":"69eebfa0-695b-4b21-90b9-8a3537168a01","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6808","nonce":1840430654},{"type":"v1","addr":"172.21.15.59:6809","nonce":1840430654}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6810","nonce":1840430654},{"type":"v1","addr":"172.21.15.59:6811","nonce":1840430654}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6814","nonce":1840430654},{"type":"v1","addr":"172.21.15.59:6815","nonce":1840430654}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6812","nonce":1840430654},{"type":"v1","addr":"172.21.15.59:6813","nonce":1840430654}]},"public_addr":"172.21.15.59:6809/1840430654","cluster_addr":"172.21.15.59:6811/1840430654","heartbeat_back_addr":"172.21.15.59:6815/1840430654","heartbeat_front_addr":"172.21.15.59:6813/1840430654","state":["exists","up"]},{"osd":3,"uuid":"96fd77fc-f911-46ff-9445-6f0be2daa8f4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6816","nonce":2504025850},{"type":"v1","addr":"172.21.15.59:6817","nonce":2504025850}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6818","nonce":2504025850},{"type":"v1","addr":"172.21.15.59:6819","nonce":2504025850}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6822","nonce":2504025850},{"type":"v1","addr":"172.21.15.59:6823","nonce":2504025850}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6820","nonce":2504025850},{"type":"v1","addr":"172.21.15.59:6821","nonce":2504025850}]},"public_addr":"172.21.15.59:6817/2504025850","cluster_addr":"172.21.15.59:6819/2504025850","heartbeat_back_addr":"172.21.15.59:6823/2504025850","heartbeat_front_addr":"172.21.15.59:6821/2504025850","state":["exists","up"]},{"osd":4,"uuid":"7ac29fca-b5b2-4c95-9d69-2f164cbbc708","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6816","nonce":2382463431},{"type":"v1","addr":"172.21.15.76:6818","nonce":2382463431}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6820","nonce":2382463431},{"type":"v1","addr":"172.21.15.76:6822","nonce":2382463431}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6828","nonce":2382463431},{"type":"v1","addr":"172.21.15.76:6830","nonce":2382463431}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6824","nonce":2382463431},{"type":"v1","addr":"172.21.15.76:6826","nonce":2382463431}]},"public_addr":"172.21.15.76:6818/2382463431","cluster_addr":"172.21.15.76:6822/2382463431","heartbeat_back_addr":"172.21.15.76:6830/2382463431","heartbeat_front_addr":"172.21.15.76:6826/2382463431","state":["exists","up"]},{"osd":5,"uuid":"485bf711-3851-4238-a4f4-51d52e0a9c58","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6800","nonce":329334308},{"type":"v1","addr":"172.21.15.76:6802","nonce":329334308}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6805","nonce":329334308},{"type":"v1","addr":"172.21.15.76:6807","nonce":329334308}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6814","nonce":329334308},{"type":"v1","addr":"172.21.15.76:6815","nonce":329334308}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6810","nonce":329334308},{"type":"v1","addr":"172.21.15.76:6812","nonce":329334308}]},"public_addr":"172.21.15.76:6802/329334308","cluster_addr":"172.21.15.76:6807/329334308","heartbeat_back_addr":"172.21.15.76:6815/329334308","heartbeat_front_addr":"172.21.15.76:6812/329334308","state":["exists","up"]},{"osd":6,"uuid":"72b3f2fc-1e4a-4e32-968d-003647a0286f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6801","nonce":3946579410},{"type":"v1","addr":"172.21.15.76:6803","nonce":3946579410}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6804","nonce":3946579410},{"type":"v1","addr":"172.21.15.76:6806","nonce":3946579410}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6811","nonce":3946579410},{"type":"v1","addr":"172.21.15.76:6813","nonce":3946579410}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6808","nonce":3946579410},{"type":"v1","addr":"172.21.15.76:6809","nonce":3946579410}]},"public_addr":"172.21.15.76:6803/3946579410","cluster_addr":"172.21.15.76:6806/3946579410","heartbeat_back_addr":"172.21.15.76:6813/3946579410","heartbeat_front_addr":"172.21.15.76:6809/3946579410","state":["exists","up"]},{"osd":7,"uuid":"1ddceb90-6c5d-4d1a-ad56-99cb877f8c70","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6817","nonce":841761538},{"type":"v1","addr":"172.21.15.76:6819","nonce":841761538}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6821","nonce":841761538},{"type":"v1","addr":"172.21.15.76:6823","nonce":841761538}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6829","nonce":841761538},{"type":"v1","addr":"172.21.15.76:6831","nonce":841761538}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6825","nonce":841761538},{"type":"v1","addr":"172.21.15.76:6827","nonce":841761538}]},"public_addr":"172.21.15.76:6819/841761538","cluster_addr":"172.21.15.76:6823/841761538","heartbeat_back_addr":"172.21.15.76:6831/841761538","heartbeat_front_addr":"172.21.15.76:6827/841761538","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T07:56:14.294838+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T07:56:14.270779+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T07:56:14.231809+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T07:56:14.260983+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T07:56:14.302997+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T07:56:14.258600+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T07:56:14.251413+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T07:56:14.304727+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-24T07:57:04.440 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:04.438+0000 7fc3d3ecd700 1 -- 172.21.15.59:0/3707919168 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7fc3b006aed0 msgr2=0x7fc3b006d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:57:04.440 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:04.438+0000 7fc3d3ecd700 1 --2- 172.21.15.59:0/3707919168 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7fc3b006aed0 0x7fc3b006d390 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fc3bc000f60 tx=0x7fc3bc009040 comp rx=0 tx=0).stop 2023-11-24T07:57:04.440 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:04.438+0000 7fc3d3ecd700 1 -- 172.21.15.59:0/3707919168 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc3cc15bde0 msgr2=0x7fc3cc0b4d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:57:04.440 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:04.438+0000 7fc3d3ecd700 1 --2- 172.21.15.59:0/3707919168 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc3cc15bde0 0x7fc3cc0b4d50 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7fc3c400cc50 tx=0x7fc3c400a340 comp rx=0 tx=0).stop 2023-11-24T07:57:04.441 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:04.438+0000 7fc3d3ecd700 1 -- 172.21.15.59:0/3707919168 shutdown_connections 2023-11-24T07:57:04.441 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:04.438+0000 7fc3d3ecd700 1 --2- 172.21.15.59:0/3707919168 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7fc3b006aed0 0x7fc3b006d390 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:04.441 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:04.438+0000 7fc3d3ecd700 1 --2- 172.21.15.59:0/3707919168 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc3cc15bde0 0x7fc3cc0b4d50 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:04.441 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:04.438+0000 7fc3d3ecd700 1 --2- 172.21.15.59:0/3707919168 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fc3cc0ce5c0 0x7fc3cc159a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:04.441 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:04.438+0000 7fc3d3ecd700 1 --2- 172.21.15.59:0/3707919168 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc3cc0cf2b0 0x7fc3cc0b4810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:04.441 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:04.438+0000 7fc3d3ecd700 1 -- 172.21.15.59:0/3707919168 >> 172.21.15.59:0/3707919168 conn(0x7fc3cc0b3720 msgr2=0x7fc3cc0b03e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:57:04.441 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:04.438+0000 7fc3d3ecd700 1 -- 172.21.15.59:0/3707919168 shutdown_connections 2023-11-24T07:57:04.442 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:04.438+0000 7fc3d3ecd700 1 -- 172.21.15.59:0/3707919168 wait complete. 2023-11-24T07:57:04.452 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-24T07:56:18.352823+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-24T07:56:22.185140+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-11-24T07:56:23.084326+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-24T07:57:04.453 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-24T07:57:04.580 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:04.579+0000 7f330c888700 1 Processor -- start 2023-11-24T07:57:04.581 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:04.579+0000 7f330c888700 1 -- start start 2023-11-24T07:57:04.583 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:04.580+0000 7f330c888700 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f33080b2c60 0x7f33080cc090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:57:04.583 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:04.580+0000 7f330c888700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f33080b3790 0x7f33080b3c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:57:04.583 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:04.580+0000 7f330c888700 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f330815b130 con 0x7f330815ad50 2023-11-24T07:57:04.583 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:04.580+0000 7f330c888700 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3308072ce0 con 0x7f33080b3790 2023-11-24T07:57:04.583 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:04.580+0000 7f330c888700 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f33080b30e0 con 0x7f33080b2c60 2023-11-24T07:57:04.583 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:04.580+0000 7f3306d9d700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f33080b3790 0x7f33080b3c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:57:04.583 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:04.580+0000 7f3306d9d700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f33080b3790 0x7f33080b3c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:51004/0 (socket says 172.21.15.59:51004) 2023-11-24T07:57:04.584 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:04.580+0000 7f3306d9d700 1 -- 172.21.15.59:0/2319031105 learned_addr learned my addr 172.21.15.59:0/2319031105 (peer_addr_for_me v2:172.21.15.59:0/0) 2023-11-24T07:57:04.584 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:04.580+0000 7f3305d9b700 1 --2- 172.21.15.59:0/2319031105 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f33080b2c60 0x7f33080cc090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:57:04.584 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:04.580+0000 7f3306d9d700 1 -- 172.21.15.59:0/2319031105 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f33080b2c60 msgr2=0x7f33080cc090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T07:57:04.584 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:04.580+0000 7f3306d9d700 1 --2- 172.21.15.59:0/2319031105 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f33080b2c60 0x7f33080cc090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:04.584 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:04.580+0000 7f3306d9d700 1 -- 172.21.15.59:0/2319031105 >> v1:172.21.15.76:6789/0 conn(0x7f330815ad50 legacy=0x7f33080725d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:57:04.584 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:04.580+0000 7f3306d9d700 1 -- 172.21.15.59:0/2319031105 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3308071530 con 0x7f33080b3790 2023-11-24T07:57:04.584 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:04.580+0000 7f3305d9b700 1 --2- 172.21.15.59:0/2319031105 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f33080b2c60 0x7f33080cc090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-24T07:57:04.584 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:04.580+0000 7f3306d9d700 1 --2- 172.21.15.59:0/2319031105 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f33080b3790 0x7f33080b3c20 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f330000d180 tx=0x7f330000d490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d147b0fd2e4e074a server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:57:04.584 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:04.581+0000 7f330559a700 1 -- 172.21.15.59:0/2319031105 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3300010070 con 0x7f33080b3790 2023-11-24T07:57:04.584 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:04.581+0000 7f330559a700 1 -- 172.21.15.59:0/2319031105 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f330000ddb0 con 0x7f33080b3790 2023-11-24T07:57:04.585 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:04.581+0000 7f330559a700 1 -- 172.21.15.59:0/2319031105 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f330000b310 con 0x7f33080b3790 2023-11-24T07:57:04.585 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:04.581+0000 7f330c888700 1 -- 172.21.15.59:0/2319031105 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f33080b3790 msgr2=0x7f33080b3c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:57:04.585 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:04.581+0000 7f330c888700 1 --2- 172.21.15.59:0/2319031105 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f33080b3790 0x7f33080b3c20 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f330000d180 tx=0x7f330000d490 comp rx=0 tx=0).stop 2023-11-24T07:57:04.585 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:04.581+0000 7f330c888700 1 -- 172.21.15.59:0/2319031105 shutdown_connections 2023-11-24T07:57:04.585 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:04.581+0000 7f330c888700 1 --2- 172.21.15.59:0/2319031105 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f33080b3790 0x7f33080b3c20 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:04.585 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:04.581+0000 7f330c888700 1 --2- 172.21.15.59:0/2319031105 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f33080b2c60 0x7f33080cc090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:04.585 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:04.581+0000 7f330c888700 1 -- 172.21.15.59:0/2319031105 >> 172.21.15.59:0/2319031105 conn(0x7f33080b1650 msgr2=0x7f33080b1a30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:57:04.585 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:04.581+0000 7f330c888700 1 -- 172.21.15.59:0/2319031105 shutdown_connections 2023-11-24T07:57:04.585 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:04.581+0000 7f330c888700 1 -- 172.21.15.59:0/2319031105 wait complete. 2023-11-24T07:57:04.588 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:04.583+0000 7f330c888700 1 Processor -- start 2023-11-24T07:57:04.588 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:04.583+0000 7f330c888700 1 -- start start 2023-11-24T07:57:04.588 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:04.583+0000 7f330c888700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f33080b2c60 0x7f33081f7fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:57:04.588 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:04.583+0000 7f330c888700 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f33080b3790 0x7f330814aea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:57:04.588 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:04.583+0000 7f330c888700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f330815ad50 0x7f33081542e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:57:04.589 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:04.583+0000 7f330c888700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3308157160 con 0x7f330815ad50 2023-11-24T07:57:04.589 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:04.583+0000 7f330c888700 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f330815b2c0 con 0x7f33080b2c60 2023-11-24T07:57:04.589 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:04.583+0000 7f330c888700 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f33081f8500 con 0x7f33080b3790 2023-11-24T07:57:04.589 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:04.583+0000 7f3306d9d700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f330815ad50 0x7f33081542e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:57:04.589 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:04.583+0000 7f3306d9d700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f330815ad50 0x7f33081542e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:51010/0 (socket says 172.21.15.59:51010) 2023-11-24T07:57:04.589 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:04.583+0000 7f3306d9d700 1 -- 172.21.15.59:0/4213728820 learned_addr learned my addr 172.21.15.59:0/4213728820 (peer_addr_for_me v2:172.21.15.59:0/0) 2023-11-24T07:57:04.589 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:04.584+0000 7f3305d9b700 1 --2- 172.21.15.59:0/4213728820 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f33080b3790 0x7f330814aea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:57:04.589 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:04.584+0000 7f330659c700 1 --2- 172.21.15.59:0/4213728820 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f33080b2c60 0x7f33081f7fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:57:04.589 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:04.584+0000 7f3306d9d700 1 -- 172.21.15.59:0/4213728820 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f33080b3790 msgr2=0x7f330814aea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:57:04.589 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:04.584+0000 7f3306d9d700 1 --2- 172.21.15.59:0/4213728820 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f33080b3790 0x7f330814aea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:04.589 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:04.584+0000 7f3306d9d700 1 -- 172.21.15.59:0/4213728820 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f33080b2c60 msgr2=0x7f33081f7fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:57:04.589 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:04.584+0000 7f3306d9d700 1 --2- 172.21.15.59:0/4213728820 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f33080b2c60 0x7f33081f7fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:04.590 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:04.584+0000 7f3306d9d700 1 -- 172.21.15.59:0/4213728820 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f330000bd40 con 0x7f330815ad50 2023-11-24T07:57:04.590 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:04.584+0000 7f3306d9d700 1 --2- 172.21.15.59:0/4213728820 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f330815ad50 0x7f33081542e0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f330000d890 tx=0x7f3300004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:57:04.590 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:04.584+0000 7f32f6ffd700 1 -- 172.21.15.59:0/4213728820 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3300009a10 con 0x7f330815ad50 2023-11-24T07:57:04.590 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:04.584+0000 7f32f6ffd700 1 -- 172.21.15.59:0/4213728820 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3300009b70 con 0x7f330815ad50 2023-11-24T07:57:04.590 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:04.584+0000 7f32f6ffd700 1 -- 172.21.15.59:0/4213728820 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f330000b4c0 con 0x7f330815ad50 2023-11-24T07:57:04.590 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:04.584+0000 7f330c888700 1 -- 172.21.15.59:0/4213728820 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f33081573b0 con 0x7f330815ad50 2023-11-24T07:57:04.590 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:04.584+0000 7f330c888700 1 -- 172.21.15.59:0/4213728820 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f330815b510 con 0x7f330815ad50 2023-11-24T07:57:04.590 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:04.585+0000 7f330c888700 1 -- 172.21.15.59:0/4213728820 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3308075480 con 0x7f330815ad50 2023-11-24T07:57:04.590 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:04.586+0000 7f32f6ffd700 1 -- 172.21.15.59:0/4213728820 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f3300004040 con 0x7f330815ad50 2023-11-24T07:57:04.590 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:04.586+0000 7f32f6ffd700 1 --2- 172.21.15.59:0/4213728820 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f32e806af00 0x7f32e806d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:57:04.590 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:04.586+0000 7f32f6ffd700 1 -- 172.21.15.59:0/4213728820 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f330008aae0 con 0x7f330815ad50 2023-11-24T07:57:04.591 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:04.586+0000 7f330659c700 1 --2- 172.21.15.59:0/4213728820 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f32e806af00 0x7f32e806d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:57:04.591 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:04.587+0000 7f330659c700 1 --2- 172.21.15.59:0/4213728820 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f32e806af00 0x7f32e806d3c0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f32f8007fd0 tx=0x7f32f8009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:57:04.592 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:04.591+0000 7f32f6ffd700 1 -- 172.21.15.59:0/4213728820 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f3300055d40 con 0x7f330815ad50 2023-11-24T07:57:04.796 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:04.795+0000 7f330c888700 1 -- 172.21.15.59:0/4213728820 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f330806e120 con 0x7f330815ad50 2023-11-24T07:57:04.797 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:04.795+0000 7f32f6ffd700 1 -- 172.21.15.59:0/4213728820 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v20) v1 ==== 93+0+10 (secure 0 0 0) 0x7f3300059360 con 0x7f330815ad50 2023-11-24T07:57:04.797 INFO:teuthology.orchestra.run.smithi059.stdout:pg_num: 1 2023-11-24T07:57:04.799 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:04.797+0000 7f330c888700 1 -- 172.21.15.59:0/4213728820 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f32e806af00 msgr2=0x7f32e806d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:57:04.799 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:04.797+0000 7f330c888700 1 --2- 172.21.15.59:0/4213728820 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f32e806af00 0x7f32e806d3c0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f32f8007fd0 tx=0x7f32f8009040 comp rx=0 tx=0).stop 2023-11-24T07:57:04.799 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:04.797+0000 7f330c888700 1 -- 172.21.15.59:0/4213728820 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f330815ad50 msgr2=0x7f33081542e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:57:04.799 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:04.797+0000 7f330c888700 1 --2- 172.21.15.59:0/4213728820 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f330815ad50 0x7f33081542e0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f330000d890 tx=0x7f3300004ad0 comp rx=0 tx=0).stop 2023-11-24T07:57:04.800 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:04.797+0000 7f330c888700 1 -- 172.21.15.59:0/4213728820 shutdown_connections 2023-11-24T07:57:04.800 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:04.797+0000 7f330c888700 1 --2- 172.21.15.59:0/4213728820 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f32e806af00 0x7f32e806d3c0 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:04.800 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:04.797+0000 7f330c888700 1 --2- 172.21.15.59:0/4213728820 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f330815ad50 0x7f33081542e0 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:04.800 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:04.797+0000 7f330c888700 1 --2- 172.21.15.59:0/4213728820 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f33080b2c60 0x7f33081f7fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:04.800 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:04.797+0000 7f330c888700 1 --2- 172.21.15.59:0/4213728820 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f33080b3790 0x7f330814aea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:04.800 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:04.797+0000 7f330c888700 1 -- 172.21.15.59:0/4213728820 >> 172.21.15.59:0/4213728820 conn(0x7f33080b1650 msgr2=0x7f33080cbe80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:57:04.800 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:04.797+0000 7f330c888700 1 -- 172.21.15.59:0/4213728820 shutdown_connections 2023-11-24T07:57:04.800 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:04.797+0000 7f330c888700 1 -- 172.21.15.59:0/4213728820 wait complete. 2023-11-24T07:57:04.811 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-24T07:57:04.938 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:04.937+0000 7f63855e3700 1 Processor -- start 2023-11-24T07:57:04.939 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:04.937+0000 7f63855e3700 1 -- start start 2023-11-24T07:57:04.940 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:04.938+0000 7f63855e3700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f63800d0870 0x7f63800d26c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:57:04.940 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:04.938+0000 7f63855e3700 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f63800d4e20 con 0x7f63800d49f0 2023-11-24T07:57:04.940 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:04.938+0000 7f63855e3700 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f63800b5370 con 0x7f63800d0c50 2023-11-24T07:57:04.940 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:04.938+0000 7f63855e3700 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f63800d2f80 con 0x7f63800d0870 2023-11-24T07:57:04.940 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:04.938+0000 7f637effd700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f63800d0870 0x7f63800d26c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:57:04.941 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:04.938+0000 7f637effd700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f63800d0870 0x7f63800d26c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:51014/0 (socket says 172.21.15.59:51014) 2023-11-24T07:57:04.941 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:04.938+0000 7f637effd700 1 -- 172.21.15.59:0/3434130151 learned_addr learned my addr 172.21.15.59:0/3434130151 (peer_addr_for_me v2:172.21.15.59:0/0) 2023-11-24T07:57:04.941 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:04.938+0000 7f637f7fe700 1 --1- 172.21.15.59:0/3434130151 >> v1:172.21.15.59:6789/0 conn(0x7f63800d49f0 0x7f63800b6d20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:54492/0 (socket says 172.21.15.59:54492) 2023-11-24T07:57:04.941 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:04.938+0000 7f637effd700 1 -- 172.21.15.59:0/3434130151 >> v1:172.21.15.76:6789/0 conn(0x7f63800d0c50 legacy=0x7f63800cf860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:57:04.941 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:04.938+0000 7f637effd700 1 -- 172.21.15.59:0/3434130151 >> v1:172.21.15.59:6789/0 conn(0x7f63800d49f0 legacy=0x7f63800b6d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:57:04.941 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:04.938+0000 7f637dffb700 1 -- 172.21.15.59:0/3434130151 <== mon.0 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1902711013 0 0) 0x7f63800d4e20 con 0x7f63800d49f0 2023-11-24T07:57:04.941 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:04.938+0000 7f637effd700 1 -- 172.21.15.59:0/3434130151 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f63800ceae0 con 0x7f63800d0870 2023-11-24T07:57:04.941 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:04.938+0000 7f637effd700 1 --2- 172.21.15.59:0/3434130151 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f63800d0870 0x7f63800d26c0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f63700098f0 tx=0x7f6370009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f02d1e7b03a7890d server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:57:04.941 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:04.938+0000 7f637dffb700 1 -- 172.21.15.59:0/3434130151 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f637000b8a0 con 0x7f63800d0870 2023-11-24T07:57:04.941 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:04.939+0000 7f637dffb700 1 -- 172.21.15.59:0/3434130151 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f637000ba00 con 0x7f63800d0870 2023-11-24T07:57:04.941 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:04.939+0000 7f637dffb700 1 -- 172.21.15.59:0/3434130151 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f637000bd90 con 0x7f63800d0870 2023-11-24T07:57:04.944 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:04.939+0000 7f63855e3700 1 -- 172.21.15.59:0/3434130151 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f63800d0870 msgr2=0x7f63800d26c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:57:04.944 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:04.939+0000 7f63855e3700 1 --2- 172.21.15.59:0/3434130151 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f63800d0870 0x7f63800d26c0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f63700098f0 tx=0x7f6370009c00 comp rx=0 tx=0).stop 2023-11-24T07:57:04.944 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:04.939+0000 7f63855e3700 1 -- 172.21.15.59:0/3434130151 shutdown_connections 2023-11-24T07:57:04.944 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:04.939+0000 7f63855e3700 1 --2- 172.21.15.59:0/3434130151 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f63800d0870 0x7f63800d26c0 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:04.945 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:04.939+0000 7f63855e3700 1 -- 172.21.15.59:0/3434130151 >> 172.21.15.59:0/3434130151 conn(0x7f63800d3ed0 msgr2=0x7f63800d42e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:57:04.945 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:04.939+0000 7f63855e3700 1 -- 172.21.15.59:0/3434130151 shutdown_connections 2023-11-24T07:57:04.945 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:04.939+0000 7f63855e3700 1 -- 172.21.15.59:0/3434130151 wait complete. 2023-11-24T07:57:04.945 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:04.940+0000 7f63855e3700 1 Processor -- start 2023-11-24T07:57:04.945 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:04.941+0000 7f63855e3700 1 -- start start 2023-11-24T07:57:04.945 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:04.941+0000 7f63855e3700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f63800d0870 0x7f63800ca660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:57:04.945 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:04.941+0000 7f63855e3700 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f63800d0c50 0x7f63800cc170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:57:04.945 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:04.941+0000 7f63855e3700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f63800d49f0 0x7f63800cc6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:57:04.945 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:04.941+0000 7f63855e3700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63800cff70 con 0x7f63800d49f0 2023-11-24T07:57:04.946 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:04.941+0000 7f63855e3700 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6380147ed0 con 0x7f63800d0870 2023-11-24T07:57:04.946 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:04.941+0000 7f63855e3700 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6380147bf0 con 0x7f63800d0c50 2023-11-24T07:57:04.946 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:04.941+0000 7f637f7fe700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f63800d49f0 0x7f63800cc6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:57:04.946 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:04.941+0000 7f637f7fe700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f63800d49f0 0x7f63800cc6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:51024/0 (socket says 172.21.15.59:51024) 2023-11-24T07:57:04.946 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:04.941+0000 7f637f7fe700 1 -- 172.21.15.59:0/3495155478 learned_addr learned my addr 172.21.15.59:0/3495155478 (peer_addr_for_me v2:172.21.15.59:0/0) 2023-11-24T07:57:04.946 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:04.942+0000 7f637effd700 1 --2- 172.21.15.59:0/3495155478 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f63800d0870 0x7f63800ca660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:57:04.946 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:04.942+0000 7f637f7fe700 1 -- 172.21.15.59:0/3495155478 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f63800d0c50 msgr2=0x7f63800cc170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:57:04.946 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:04.942+0000 7f637e7fc700 1 --2- 172.21.15.59:0/3495155478 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f63800d0c50 0x7f63800cc170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:57:04.947 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:04.942+0000 7f637f7fe700 1 --2- 172.21.15.59:0/3495155478 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f63800d0c50 0x7f63800cc170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:04.947 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:04.942+0000 7f637f7fe700 1 -- 172.21.15.59:0/3495155478 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f63800d0870 msgr2=0x7f63800ca660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:57:04.947 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:04.942+0000 7f637f7fe700 1 --2- 172.21.15.59:0/3495155478 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f63800d0870 0x7f63800ca660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:04.947 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:04.942+0000 7f637f7fe700 1 -- 172.21.15.59:0/3495155478 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6370009550 con 0x7f63800d49f0 2023-11-24T07:57:04.947 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:04.942+0000 7f637effd700 1 --2- 172.21.15.59:0/3495155478 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f63800d0870 0x7f63800ca660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T07:57:04.947 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:04.942+0000 7f637e7fc700 1 --2- 172.21.15.59:0/3495155478 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f63800d0c50 0x7f63800cc170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T07:57:04.947 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:04.942+0000 7f637f7fe700 1 --2- 172.21.15.59:0/3495155478 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f63800d49f0 0x7f63800cc6b0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f637800cba0 tx=0x7f637800cf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:57:04.947 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:04.942+0000 7f63677fe700 1 -- 172.21.15.59:0/3495155478 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f637800ab80 con 0x7f63800d49f0 2023-11-24T07:57:04.947 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:04.942+0000 7f63677fe700 1 -- 172.21.15.59:0/3495155478 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f637800ad00 con 0x7f63800d49f0 2023-11-24T07:57:04.948 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:04.942+0000 7f63855e3700 1 -- 172.21.15.59:0/3495155478 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6380144ff0 con 0x7f63800d49f0 2023-11-24T07:57:04.948 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:04.942+0000 7f63677fe700 1 -- 172.21.15.59:0/3495155478 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6378007310 con 0x7f63800d49f0 2023-11-24T07:57:04.948 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:04.942+0000 7f63855e3700 1 -- 172.21.15.59:0/3495155478 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f63801455b0 con 0x7f63800d49f0 2023-11-24T07:57:04.949 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:04.943+0000 7f63855e3700 1 -- 172.21.15.59:0/3495155478 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f63800b57c0 con 0x7f63800d49f0 2023-11-24T07:57:04.949 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:04.943+0000 7f63677fe700 1 -- 172.21.15.59:0/3495155478 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f6378007470 con 0x7f63800d49f0 2023-11-24T07:57:04.949 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:04.944+0000 7f63677fe700 1 --2- 172.21.15.59:0/3495155478 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f636c06aed0 0x7f636c06d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:57:04.949 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:04.944+0000 7f63677fe700 1 -- 172.21.15.59:0/3495155478 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f63780894c0 con 0x7f63800d49f0 2023-11-24T07:57:04.949 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:04.944+0000 7f637effd700 1 --2- 172.21.15.59:0/3495155478 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f636c06aed0 0x7f636c06d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:57:04.950 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:04.944+0000 7f637effd700 1 --2- 172.21.15.59:0/3495155478 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f636c06aed0 0x7f636c06d390 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f6370009c40 tx=0x7f6370008030 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:57:04.950 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:04.948+0000 7f63677fe700 1 -- 172.21.15.59:0/3495155478 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f63780544e0 con 0x7f63800d49f0 2023-11-24T07:57:05.149 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:05.148+0000 7f63855e3700 1 -- 172.21.15.59:0/3495155478 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f6380075480 con 0x7f63800d49f0 2023-11-24T07:57:05.150 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:05.148+0000 7f63677fe700 1 -- 172.21.15.59:0/3495155478 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v20) v1 ==== 104+0+11 (secure 0 0 0) 0x7f6378057b00 con 0x7f63800d49f0 2023-11-24T07:57:05.150 INFO:teuthology.orchestra.run.smithi059.stdout:pg_num: 32 2023-11-24T07:57:05.152 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:05.150+0000 7f63855e3700 1 -- 172.21.15.59:0/3495155478 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f636c06aed0 msgr2=0x7f636c06d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:57:05.152 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:05.150+0000 7f63855e3700 1 --2- 172.21.15.59:0/3495155478 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f636c06aed0 0x7f636c06d390 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f6370009c40 tx=0x7f6370008030 comp rx=0 tx=0).stop 2023-11-24T07:57:05.152 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:05.150+0000 7f63855e3700 1 -- 172.21.15.59:0/3495155478 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f63800d49f0 msgr2=0x7f63800cc6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:57:05.153 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:05.150+0000 7f63855e3700 1 --2- 172.21.15.59:0/3495155478 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f63800d49f0 0x7f63800cc6b0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f637800cba0 tx=0x7f637800cf60 comp rx=0 tx=0).stop 2023-11-24T07:57:05.153 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:05.150+0000 7f63855e3700 1 -- 172.21.15.59:0/3495155478 shutdown_connections 2023-11-24T07:57:05.153 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:05.150+0000 7f63855e3700 1 --2- 172.21.15.59:0/3495155478 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f636c06aed0 0x7f636c06d390 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:05.153 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:05.150+0000 7f63855e3700 1 --2- 172.21.15.59:0/3495155478 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f63800d49f0 0x7f63800cc6b0 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:05.153 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:05.150+0000 7f63855e3700 1 --2- 172.21.15.59:0/3495155478 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f63800d0870 0x7f63800ca660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:05.153 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:05.150+0000 7f63855e3700 1 --2- 172.21.15.59:0/3495155478 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f63800d0c50 0x7f63800cc170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:05.153 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:05.150+0000 7f63855e3700 1 -- 172.21.15.59:0/3495155478 >> 172.21.15.59:0/3495155478 conn(0x7f63800d3ed0 msgr2=0x7f638015b560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:57:05.153 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:05.150+0000 7f63855e3700 1 -- 172.21.15.59:0/3495155478 shutdown_connections 2023-11-24T07:57:05.153 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:05.151+0000 7f63855e3700 1 -- 172.21.15.59:0/3495155478 wait complete. 2023-11-24T07:57:05.164 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-24T07:57:05.292 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:05.290+0000 7f15f07b0700 1 Processor -- start 2023-11-24T07:57:05.292 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:05.290+0000 7f15f07b0700 1 -- start start 2023-11-24T07:57:05.294 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:05.291+0000 7f15f07b0700 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f15e806b620 0x7f15e80b3a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:57:05.294 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:05.291+0000 7f15f07b0700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f15e80b5ab0 0x7f15e80b3f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:57:05.294 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:05.291+0000 7f15f07b0700 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f15e8071f50 con 0x7f15e81478c0 2023-11-24T07:57:05.294 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:05.291+0000 7f15f07b0700 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f15e8072230 con 0x7f15e80b5ab0 2023-11-24T07:57:05.294 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:05.291+0000 7f15f07b0700 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f15e8072510 con 0x7f15e806b620 2023-11-24T07:57:05.294 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:05.291+0000 7f15edd4b700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f15e80b5ab0 0x7f15e80b3f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:57:05.295 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:05.291+0000 7f15edd4b700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f15e80b5ab0 0x7f15e80b3f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:51028/0 (socket says 172.21.15.59:51028) 2023-11-24T07:57:05.295 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:05.291+0000 7f15eed4d700 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7f15e81478c0 0x7f15e8161ba0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:54504/0 (socket says 172.21.15.59:54504) 2023-11-24T07:57:05.295 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:05.291+0000 7f15edd4b700 1 -- 172.21.15.59:0/2355487341 learned_addr learned my addr 172.21.15.59:0/2355487341 (peer_addr_for_me v2:172.21.15.59:0/0) 2023-11-24T07:57:05.295 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:05.291+0000 7f15ee54c700 1 --2- 172.21.15.59:0/2355487341 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f15e806b620 0x7f15e80b3a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:57:05.295 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:05.291+0000 7f15edd4b700 1 -- 172.21.15.59:0/2355487341 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f15e806b620 msgr2=0x7f15e80b3a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T07:57:05.295 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:05.291+0000 7f15edd4b700 1 --2- 172.21.15.59:0/2355487341 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f15e806b620 0x7f15e80b3a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:05.295 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:05.291+0000 7f15edd4b700 1 -- 172.21.15.59:0/2355487341 >> v1:172.21.15.59:6789/0 conn(0x7f15e81478c0 legacy=0x7f15e8161ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:57:05.295 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:05.291+0000 7f15edd4b700 1 -- 172.21.15.59:0/2355487341 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f15e80727f0 con 0x7f15e80b5ab0 2023-11-24T07:57:05.295 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:05.291+0000 7f15ed54a700 1 -- 172.21.15.59:0/2355487341 <== mon.0 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 806852936 0 0) 0x7f15e8071f50 con 0x7f15e81478c0 2023-11-24T07:57:05.295 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:05.292+0000 7f15edd4b700 1 --2- 172.21.15.59:0/2355487341 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f15e80b5ab0 0x7f15e80b3f80 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f15e40098f0 tx=0x7f15e4009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ba1408808a2702b4 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:57:05.295 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:05.292+0000 7f15ed54a700 1 -- 172.21.15.59:0/2355487341 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15e4011830 con 0x7f15e80b5ab0 2023-11-24T07:57:05.295 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:05.292+0000 7f15ed54a700 1 -- 172.21.15.59:0/2355487341 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f15e4011990 con 0x7f15e80b5ab0 2023-11-24T07:57:05.296 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:05.292+0000 7f15ed54a700 1 -- 172.21.15.59:0/2355487341 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15e4011d20 con 0x7f15e80b5ab0 2023-11-24T07:57:05.296 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:05.292+0000 7f15f07b0700 1 -- 172.21.15.59:0/2355487341 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f15e80b5ab0 msgr2=0x7f15e80b3f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:57:05.296 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:05.292+0000 7f15f07b0700 1 --2- 172.21.15.59:0/2355487341 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f15e80b5ab0 0x7f15e80b3f80 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f15e40098f0 tx=0x7f15e4009c00 comp rx=0 tx=0).stop 2023-11-24T07:57:05.296 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:05.292+0000 7f15f07b0700 1 -- 172.21.15.59:0/2355487341 shutdown_connections 2023-11-24T07:57:05.296 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:05.292+0000 7f15f07b0700 1 --2- 172.21.15.59:0/2355487341 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f15e806b620 0x7f15e80b3a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:05.296 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:05.292+0000 7f15f07b0700 1 --2- 172.21.15.59:0/2355487341 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f15e80b5ab0 0x7f15e80b3f80 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:05.296 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:05.292+0000 7f15f07b0700 1 -- 172.21.15.59:0/2355487341 >> 172.21.15.59:0/2355487341 conn(0x7f15e80b2740 msgr2=0x7f15e80b2b20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:57:05.296 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:05.292+0000 7f15f07b0700 1 -- 172.21.15.59:0/2355487341 shutdown_connections 2023-11-24T07:57:05.296 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:05.292+0000 7f15f07b0700 1 -- 172.21.15.59:0/2355487341 wait complete. 2023-11-24T07:57:05.299 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:05.294+0000 7f15f07b0700 1 Processor -- start 2023-11-24T07:57:05.299 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:05.294+0000 7f15f07b0700 1 -- start start 2023-11-24T07:57:05.299 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:05.294+0000 7f15f07b0700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f15e806b620 0x7f15e81e1220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:57:05.299 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:05.294+0000 7f15f07b0700 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f15e80b5ab0 0x7f15e81dece0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:57:05.299 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:05.294+0000 7f15f07b0700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f15e81478c0 0x7f15e81df220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:57:05.299 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:05.294+0000 7f15f07b0700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15e81ddae0 con 0x7f15e806b620 2023-11-24T07:57:05.299 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:05.294+0000 7f15f07b0700 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15e81e1760 con 0x7f15e81478c0 2023-11-24T07:57:05.299 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:05.294+0000 7f15f07b0700 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f15e81e5530 con 0x7f15e80b5ab0 2023-11-24T07:57:05.299 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:05.295+0000 7f15ee54c700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f15e806b620 0x7f15e81e1220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:57:05.299 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:05.295+0000 7f15ee54c700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f15e806b620 0x7f15e81e1220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:51032/0 (socket says 172.21.15.59:51032) 2023-11-24T07:57:05.300 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:05.295+0000 7f15ee54c700 1 -- 172.21.15.59:0/1383307160 learned_addr learned my addr 172.21.15.59:0/1383307160 (peer_addr_for_me v2:172.21.15.59:0/0) 2023-11-24T07:57:05.300 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:05.295+0000 7f15edd4b700 1 --2- 172.21.15.59:0/1383307160 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f15e80b5ab0 0x7f15e81dece0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:57:05.300 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:05.295+0000 7f15eed4d700 1 --2- 172.21.15.59:0/1383307160 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f15e81478c0 0x7f15e81df220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:57:05.300 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:05.295+0000 7f15ee54c700 1 -- 172.21.15.59:0/1383307160 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f15e80b5ab0 msgr2=0x7f15e81dece0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:57:05.300 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:05.295+0000 7f15ee54c700 1 --2- 172.21.15.59:0/1383307160 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f15e80b5ab0 0x7f15e81dece0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:05.300 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:05.295+0000 7f15ee54c700 1 -- 172.21.15.59:0/1383307160 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f15e81478c0 msgr2=0x7f15e81df220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:57:05.300 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:05.295+0000 7f15ee54c700 1 --2- 172.21.15.59:0/1383307160 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f15e81478c0 0x7f15e81df220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:05.300 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:05.295+0000 7f15ee54c700 1 -- 172.21.15.59:0/1383307160 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f15e4009550 con 0x7f15e806b620 2023-11-24T07:57:05.300 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:05.295+0000 7f15eed4d700 1 --2- 172.21.15.59:0/1383307160 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f15e81478c0 0x7f15e81df220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-24T07:57:05.300 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:05.295+0000 7f15ee54c700 1 --2- 172.21.15.59:0/1383307160 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f15e806b620 0x7f15e81e1220 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f15d800e9b0 tx=0x7f15d800ecc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:57:05.301 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:05.295+0000 7f15deffd700 1 -- 172.21.15.59:0/1383307160 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15d8009710 con 0x7f15e806b620 2023-11-24T07:57:05.301 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:05.295+0000 7f15deffd700 1 -- 172.21.15.59:0/1383307160 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f15d800c850 con 0x7f15e806b620 2023-11-24T07:57:05.301 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:05.295+0000 7f15f07b0700 1 -- 172.21.15.59:0/1383307160 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f15e81dced0 con 0x7f15e806b620 2023-11-24T07:57:05.301 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:05.295+0000 7f15deffd700 1 -- 172.21.15.59:0/1383307160 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15d800cbc0 con 0x7f15e806b620 2023-11-24T07:57:05.301 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:05.295+0000 7f15f07b0700 1 -- 172.21.15.59:0/1383307160 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f15e81dd490 con 0x7f15e806b620 2023-11-24T07:57:05.301 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:05.296+0000 7f15f07b0700 1 -- 172.21.15.59:0/1383307160 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f15e8075480 con 0x7f15e806b620 2023-11-24T07:57:05.301 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:05.297+0000 7f15deffd700 1 -- 172.21.15.59:0/1383307160 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f15d800cd20 con 0x7f15e806b620 2023-11-24T07:57:05.302 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:05.297+0000 7f15deffd700 1 --2- 172.21.15.59:0/1383307160 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f15cc06aed0 0x7f15cc06d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:57:05.302 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:05.297+0000 7f15deffd700 1 -- 172.21.15.59:0/1383307160 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f15d80895f0 con 0x7f15e806b620 2023-11-24T07:57:05.302 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:05.297+0000 7f15edd4b700 1 --2- 172.21.15.59:0/1383307160 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f15cc06aed0 0x7f15cc06d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:57:05.302 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:05.298+0000 7f15edd4b700 1 --2- 172.21.15.59:0/1383307160 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f15cc06aed0 0x7f15cc06d390 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f15e80b5f10 tx=0x7f15e4008030 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:57:05.303 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:05.302+0000 7f15deffd700 1 -- 172.21.15.59:0/1383307160 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f15d8054730 con 0x7f15e806b620 2023-11-24T07:57:05.503 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:05.501+0000 7f15f07b0700 1 -- 172.21.15.59:0/1383307160 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f15e81deac0 con 0x7f15e806b620 2023-11-24T07:57:05.504 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:05.502+0000 7f15deffd700 1 -- 172.21.15.59:0/1383307160 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v20) v1 ==== 100+0+11 (secure 0 0 0) 0x7f15d8057d50 con 0x7f15e806b620 2023-11-24T07:57:05.504 INFO:teuthology.orchestra.run.smithi059.stdout:pg_num: 64 2023-11-24T07:57:05.506 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:05.504+0000 7f15f07b0700 1 -- 172.21.15.59:0/1383307160 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f15cc06aed0 msgr2=0x7f15cc06d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:57:05.506 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:05.504+0000 7f15f07b0700 1 --2- 172.21.15.59:0/1383307160 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f15cc06aed0 0x7f15cc06d390 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f15e80b5f10 tx=0x7f15e4008030 comp rx=0 tx=0).stop 2023-11-24T07:57:05.506 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:05.504+0000 7f15f07b0700 1 -- 172.21.15.59:0/1383307160 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f15e806b620 msgr2=0x7f15e81e1220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:57:05.507 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:05.504+0000 7f15f07b0700 1 --2- 172.21.15.59:0/1383307160 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f15e806b620 0x7f15e81e1220 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f15d800e9b0 tx=0x7f15d800ecc0 comp rx=0 tx=0).stop 2023-11-24T07:57:05.507 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:05.504+0000 7f15f07b0700 1 -- 172.21.15.59:0/1383307160 shutdown_connections 2023-11-24T07:57:05.507 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:05.504+0000 7f15f07b0700 1 --2- 172.21.15.59:0/1383307160 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f15cc06aed0 0x7f15cc06d390 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:05.507 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:05.504+0000 7f15f07b0700 1 --2- 172.21.15.59:0/1383307160 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f15e81478c0 0x7f15e81df220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:05.507 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:05.504+0000 7f15f07b0700 1 --2- 172.21.15.59:0/1383307160 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f15e806b620 0x7f15e81e1220 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:05.507 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:05.504+0000 7f15f07b0700 1 --2- 172.21.15.59:0/1383307160 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f15e80b5ab0 0x7f15e81dece0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:05.507 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:05.504+0000 7f15f07b0700 1 -- 172.21.15.59:0/1383307160 >> 172.21.15.59:0/1383307160 conn(0x7f15e80b2740 msgr2=0x7f15e80b1b40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:57:05.507 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:05.504+0000 7f15f07b0700 1 -- 172.21.15.59:0/1383307160 shutdown_connections 2023-11-24T07:57:05.507 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:05.504+0000 7f15f07b0700 1 -- 172.21.15.59:0/1383307160 wait complete. 2023-11-24T07:57:05.518 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-11-24T07:57:05.644 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:05.642+0000 7f5ed2d18700 1 Processor -- start 2023-11-24T07:57:05.644 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:05.643+0000 7f5ed2d18700 1 -- start start 2023-11-24T07:57:05.646 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:05.643+0000 7f5ed2d18700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f5ecc15e1c0 0x7f5ecc06eef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:57:05.646 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:05.643+0000 7f5ed2d18700 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5ecc0d1230 con 0x7f5ecc0b0e90 2023-11-24T07:57:05.647 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:05.643+0000 7f5ed2d18700 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5ecc0d0c70 con 0x7f5ecc06fc50 2023-11-24T07:57:05.647 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:05.643+0000 7f5ed2d18700 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5ecc0d0f50 con 0x7f5ecc15e1c0 2023-11-24T07:57:05.647 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:05.644+0000 7f5ed0ab4700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f5ecc15e1c0 0x7f5ecc06eef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:57:05.647 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:05.644+0000 7f5ed0ab4700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f5ecc15e1c0 0x7f5ecc06eef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:51040/0 (socket says 172.21.15.59:51040) 2023-11-24T07:57:05.647 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:05.644+0000 7f5ed0ab4700 1 -- 172.21.15.59:0/3064193936 learned_addr learned my addr 172.21.15.59:0/3064193936 (peer_addr_for_me v2:172.21.15.59:0/0) 2023-11-24T07:57:05.647 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:05.644+0000 7f5ed0ab4700 1 -- 172.21.15.59:0/3064193936 >> v1:172.21.15.76:6789/0 conn(0x7f5ecc06fc50 legacy=0x7f5ecc06ca10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T07:57:05.647 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:05.644+0000 7f5ed0ab4700 1 -- 172.21.15.59:0/3064193936 >> v1:172.21.15.59:6789/0 conn(0x7f5ecc0b0e90 legacy=0x7f5ecc06f540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:57:05.647 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:05.644+0000 7f5ed0ab4700 1 -- 172.21.15.59:0/3064193936 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5ecc06c210 con 0x7f5ecc15e1c0 2023-11-24T07:57:05.647 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:05.644+0000 7f5ed0ab4700 1 --2- 172.21.15.59:0/3064193936 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5ecc15e1c0 0x7f5ecc06eef0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f5ec00098f0 tx=0x7f5ec0009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=88d1dfdb723a5fd6 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:57:05.647 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:05.644+0000 7f5ecb7fe700 1 -- 172.21.15.59:0/3064193936 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ec0004230 con 0x7f5ecc15e1c0 2023-11-24T07:57:05.647 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:05.644+0000 7f5ecb7fe700 1 -- 172.21.15.59:0/3064193936 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5ec0004390 con 0x7f5ecc15e1c0 2023-11-24T07:57:05.647 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:05.644+0000 7f5ecb7fe700 1 -- 172.21.15.59:0/3064193936 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ec00036a0 con 0x7f5ecc15e1c0 2023-11-24T07:57:05.647 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:05.645+0000 7f5ed2d18700 1 -- 172.21.15.59:0/3064193936 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5ecc15e1c0 msgr2=0x7f5ecc06eef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:57:05.648 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:05.645+0000 7f5ed2d18700 1 --2- 172.21.15.59:0/3064193936 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5ecc15e1c0 0x7f5ecc06eef0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f5ec00098f0 tx=0x7f5ec0009c00 comp rx=0 tx=0).stop 2023-11-24T07:57:05.648 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:05.645+0000 7f5ed2d18700 1 -- 172.21.15.59:0/3064193936 shutdown_connections 2023-11-24T07:57:05.648 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:05.645+0000 7f5ed2d18700 1 --2- 172.21.15.59:0/3064193936 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5ecc15e1c0 0x7f5ecc06eef0 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:05.648 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:05.645+0000 7f5ed2d18700 1 -- 172.21.15.59:0/3064193936 >> 172.21.15.59:0/3064193936 conn(0x7f5ecc0b3720 msgr2=0x7f5ecc0b3b30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:57:05.648 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:05.645+0000 7f5ed2d18700 1 -- 172.21.15.59:0/3064193936 shutdown_connections 2023-11-24T07:57:05.648 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:05.645+0000 7f5ed2d18700 1 -- 172.21.15.59:0/3064193936 wait complete. 2023-11-24T07:57:05.650 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:05.646+0000 7f5ed2d18700 1 Processor -- start 2023-11-24T07:57:05.650 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:05.646+0000 7f5ed2d18700 1 -- start start 2023-11-24T07:57:05.650 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:05.647+0000 7f5ed2d18700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5ecc06fc50 0x7f5ecc157d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:57:05.650 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:05.647+0000 7f5ed2d18700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5ecc0b0e90 0x7f5ecc1f3e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:57:05.651 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:05.647+0000 7f5ed2d18700 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f5ecc15e1c0 0x7f5ecc15ccd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:57:05.651 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:05.647+0000 7f5ed2d18700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ecc1f71c0 con 0x7f5ecc06fc50 2023-11-24T07:57:05.651 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:05.647+0000 7f5ed2d18700 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ecc09ceb0 con 0x7f5ecc0b0e90 2023-11-24T07:57:05.651 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:05.647+0000 7f5ed2d18700 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5ecc09d190 con 0x7f5ecc15e1c0 2023-11-24T07:57:05.651 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:05.647+0000 7f5ed0ab4700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5ecc06fc50 0x7f5ecc157d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:57:05.651 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:05.647+0000 7f5ed0ab4700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5ecc06fc50 0x7f5ecc157d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:51046/0 (socket says 172.21.15.59:51046) 2023-11-24T07:57:05.651 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:05.647+0000 7f5ed0ab4700 1 -- 172.21.15.59:0/2651078117 learned_addr learned my addr 172.21.15.59:0/2651078117 (peer_addr_for_me v2:172.21.15.59:0/0) 2023-11-24T07:57:05.651 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:05.647+0000 7f5ecbfff700 1 --2- 172.21.15.59:0/2651078117 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5ecc0b0e90 0x7f5ecc1f3e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:57:05.651 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:05.647+0000 7f5ed12b5700 1 --2- 172.21.15.59:0/2651078117 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f5ecc15e1c0 0x7f5ecc15ccd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:57:05.651 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:05.647+0000 7f5ed0ab4700 1 -- 172.21.15.59:0/2651078117 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f5ecc15e1c0 msgr2=0x7f5ecc15ccd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:57:05.651 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:05.647+0000 7f5ed0ab4700 1 --2- 172.21.15.59:0/2651078117 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f5ecc15e1c0 0x7f5ecc15ccd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:05.651 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:05.647+0000 7f5ed0ab4700 1 -- 172.21.15.59:0/2651078117 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5ecc0b0e90 msgr2=0x7f5ecc1f3e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:57:05.652 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:05.647+0000 7f5ed0ab4700 1 --2- 172.21.15.59:0/2651078117 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5ecc0b0e90 0x7f5ecc1f3e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:05.652 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:05.647+0000 7f5ed0ab4700 1 -- 172.21.15.59:0/2651078117 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5ec0009550 con 0x7f5ecc06fc50 2023-11-24T07:57:05.652 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:05.647+0000 7f5ed0ab4700 1 --2- 172.21.15.59:0/2651078117 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5ecc06fc50 0x7f5ecc157d80 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f5ec0000c00 tx=0x7f5ec0004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:57:05.652 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:05.648+0000 7f5ec97fa700 1 -- 172.21.15.59:0/2651078117 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ec000b660 con 0x7f5ecc06fc50 2023-11-24T07:57:05.652 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:05.648+0000 7f5ec97fa700 1 -- 172.21.15.59:0/2651078117 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5ec000b7e0 con 0x7f5ecc06fc50 2023-11-24T07:57:05.652 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:05.648+0000 7f5ed2d18700 1 -- 172.21.15.59:0/2651078117 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5ecc09d470 con 0x7f5ecc06fc50 2023-11-24T07:57:05.652 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:05.648+0000 7f5ec97fa700 1 -- 172.21.15.59:0/2651078117 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ec000bb90 con 0x7f5ecc06fc50 2023-11-24T07:57:05.652 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:05.648+0000 7f5ed2d18700 1 -- 172.21.15.59:0/2651078117 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5ecc09da30 con 0x7f5ecc06fc50 2023-11-24T07:57:05.652 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:05.649+0000 7f5ed2d18700 1 -- 172.21.15.59:0/2651078117 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5ecc09dbc0 con 0x7f5ecc06fc50 2023-11-24T07:57:05.653 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:05.649+0000 7f5ec97fa700 1 -- 172.21.15.59:0/2651078117 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f5ec000bcf0 con 0x7f5ecc06fc50 2023-11-24T07:57:05.653 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:05.650+0000 7f5ec97fa700 1 --2- 172.21.15.59:0/2651078117 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f5eb006af00 0x7f5eb006d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:57:05.653 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:05.650+0000 7f5ec97fa700 1 -- 172.21.15.59:0/2651078117 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f5ec008b080 con 0x7f5ecc06fc50 2023-11-24T07:57:05.653 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:05.650+0000 7f5ecbfff700 1 --2- 172.21.15.59:0/2651078117 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f5eb006af00 0x7f5eb006d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:57:05.653 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:05.650+0000 7f5ecbfff700 1 --2- 172.21.15.59:0/2651078117 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f5eb006af00 0x7f5eb006d3c0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f5ecc06cbf0 tx=0x7f5eb8009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:57:05.656 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:05.654+0000 7f5ec97fa700 1 -- 172.21.15.59:0/2651078117 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f5ec00bde90 con 0x7f5ecc06fc50 2023-11-24T07:57:05.873 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:05.871+0000 7f5ed2d18700 1 -- 172.21.15.59:0/2651078117 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f5ecc0cf4a0 con 0x7f5ecc06fc50 2023-11-24T07:57:05.873 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:05.872+0000 7f5ec97fa700 1 -- 172.21.15.59:0/2651078117 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v20) v1 ==== 80+0+0 (secure 0 0 0) 0x7f5ecc0cf4a0 con 0x7f5ecc06fc50 2023-11-24T07:57:05.875 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:05.873+0000 7f5ed2d18700 1 -- 172.21.15.59:0/2651078117 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f5eb006af00 msgr2=0x7f5eb006d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:57:05.875 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:05.873+0000 7f5ed2d18700 1 --2- 172.21.15.59:0/2651078117 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f5eb006af00 0x7f5eb006d3c0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f5ecc06cbf0 tx=0x7f5eb8009040 comp rx=0 tx=0).stop 2023-11-24T07:57:05.876 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:05.873+0000 7f5ed2d18700 1 -- 172.21.15.59:0/2651078117 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5ecc06fc50 msgr2=0x7f5ecc157d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:57:05.876 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:05.873+0000 7f5ed2d18700 1 --2- 172.21.15.59:0/2651078117 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5ecc06fc50 0x7f5ecc157d80 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f5ec0000c00 tx=0x7f5ec0004750 comp rx=0 tx=0).stop 2023-11-24T07:57:05.876 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:05.874+0000 7f5ed2d18700 1 -- 172.21.15.59:0/2651078117 shutdown_connections 2023-11-24T07:57:05.876 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:05.874+0000 7f5ed2d18700 1 --2- 172.21.15.59:0/2651078117 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f5eb006af00 0x7f5eb006d3c0 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:05.876 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:05.874+0000 7f5ed2d18700 1 --2- 172.21.15.59:0/2651078117 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f5ecc15e1c0 0x7f5ecc15ccd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:05.876 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:05.874+0000 7f5ed2d18700 1 --2- 172.21.15.59:0/2651078117 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5ecc06fc50 0x7f5ecc157d80 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:05.876 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:05.874+0000 7f5ed2d18700 1 --2- 172.21.15.59:0/2651078117 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5ecc0b0e90 0x7f5ecc1f3e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:05.876 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:05.874+0000 7f5ed2d18700 1 -- 172.21.15.59:0/2651078117 >> 172.21.15.59:0/2651078117 conn(0x7f5ecc0b3720 msgr2=0x7f5ecc0b1a50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:57:05.876 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:05.874+0000 7f5ed2d18700 1 -- 172.21.15.59:0/2651078117 shutdown_connections 2023-11-24T07:57:05.876 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:05.874+0000 7f5ed2d18700 1 -- 172.21.15.59:0/2651078117 wait complete. 2023-11-24T07:57:05.877 INFO:teuthology.orchestra.run.smithi059.stderr:listed 0 entries 2023-11-24T07:57:05.887 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.1... 2023-11-24T07:57:05.888 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T07:57:05.888 DEBUG:teuthology.orchestra.run.smithi076:> sudo umount /home/ubuntu/cephtest/mnt.1 -f 2023-11-24T07:57:05.929 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi076.front.sepia.ceph.com 2023-11-24T07:57:05.929 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T07:57:05.929 DEBUG:teuthology.orchestra.run.smithi076:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2023-11-24T07:57:05.995 DEBUG:teuthology.orchestra.run.smithi059:> set -ex 2023-11-24T07:57:05.996 DEBUG:teuthology.orchestra.run.smithi059:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-24T07:57:06.013 DEBUG:teuthology.orchestra.run.smithi059:> set -ex 2023-11-24T07:57:06.014 DEBUG:teuthology.orchestra.run.smithi059:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-24T07:57:06.071 DEBUG:teuthology.orchestra.run.smithi059:> set -ex 2023-11-24T07:57:06.071 DEBUG:teuthology.orchestra.run.smithi059:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-24T07:57:06.129 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-24T07:57:06.294 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:06.293+0000 7f5503caa700 1 Processor -- start 2023-11-24T07:57:06.294 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:06.293+0000 7f5503caa700 1 -- start start 2023-11-24T07:57:06.298 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:06.294+0000 7f5503caa700 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f54fc0d16e0 0x7f54fc0affb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:57:06.298 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:06.294+0000 7f5503caa700 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f54fc0d1b40 con 0x7f54fc06afd0 2023-11-24T07:57:06.298 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:06.294+0000 7f5503caa700 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f54fc070490 con 0x7f54fc0ada10 2023-11-24T07:57:06.298 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:06.294+0000 7f5503caa700 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f54fc0b04f0 con 0x7f54fc0d16e0 2023-11-24T07:57:06.298 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:06.294+0000 7f5501a46700 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7f54fc06afd0 0x7f54fc06b3b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:54528/0 (socket says 172.21.15.59:54528) 2023-11-24T07:57:06.298 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:06.294+0000 7f5501a46700 1 -- 172.21.15.59:0/3257110852 learned_addr learned my addr 172.21.15.59:0/3257110852 (peer_addr_for_me v1:172.21.15.59:0/0) 2023-11-24T07:57:06.299 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:06.294+0000 7f5501245700 1 --2- 172.21.15.59:0/3257110852 >> v2:172.21.15.76:3300/0 conn(0x7f54fc0d16e0 0x7f54fc0affb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:57:06.299 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:06.294+0000 7f5500a44700 1 -- 172.21.15.59:0/3257110852 <== mon.0 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1424608866 0 0) 0x7f54fc0d1b40 con 0x7f54fc06afd0 2023-11-24T07:57:06.299 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:06.294+0000 7f5500a44700 1 -- 172.21.15.59:0/3257110852 --> v1:172.21.15.59:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f54e80035b0 con 0x7f54fc06afd0 2023-11-24T07:57:06.299 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:06.294+0000 7f5500a44700 1 -- 172.21.15.59:0/3257110852 <== mon.0 v1:172.21.15.59:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3683117740 0 0) 0x7f54e80035b0 con 0x7f54fc06afd0 2023-11-24T07:57:06.299 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:06.294+0000 7f5500a44700 1 -- 172.21.15.59:0/3257110852 >> v2:172.21.15.76:3300/0 conn(0x7f54fc0d16e0 msgr2=0x7f54fc0affb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T07:57:06.299 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:06.294+0000 7f5500a44700 1 --2- 172.21.15.59:0/3257110852 >> v2:172.21.15.76:3300/0 conn(0x7f54fc0d16e0 0x7f54fc0affb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:06.299 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:06.294+0000 7f5500a44700 1 -- 172.21.15.59:0/3257110852 >> v1:172.21.15.76:6789/0 conn(0x7f54fc0ada10 legacy=0x7f54fc0adea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:57:06.299 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:06.294+0000 7f5500a44700 1 -- 172.21.15.59:0/3257110852 --> v1:172.21.15.59:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f54fc0af9f0 con 0x7f54fc06afd0 2023-11-24T07:57:06.299 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:06.294+0000 7f5500a44700 1 -- 172.21.15.59:0/3257110852 <== mon.0 v1:172.21.15.59:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1312333566 0 0) 0x7f54f8001670 con 0x7f54fc06afd0 2023-11-24T07:57:06.299 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:06.294+0000 7f5500a44700 1 -- 172.21.15.59:0/3257110852 >> v1:172.21.15.59:6789/0 conn(0x7f54fc06afd0 legacy=0x7f54fc06b3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:57:06.299 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:06.294+0000 7f5500a44700 1 --2- 172.21.15.59:0/3257110852 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f54e8003a00 0x7f54e8003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:57:06.300 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:06.294+0000 7f5500a44700 1 --2- 172.21.15.59:0/3257110852 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f54e8004850 0x7f54e8004cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:57:06.300 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:06.295+0000 7f5500a44700 1 --2- 172.21.15.59:0/3257110852 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f54e8005210 0x7f54e800ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:57:06.300 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:06.295+0000 7f5500a44700 1 -- 172.21.15.59:0/3257110852 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54fc0b04f0 con 0x7f54e8004850 2023-11-24T07:57:06.300 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:06.295+0000 7f5500a44700 1 -- 172.21.15.59:0/3257110852 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54e800c0b0 con 0x7f54e8005210 2023-11-24T07:57:06.300 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:06.295+0000 7f5500a44700 1 -- 172.21.15.59:0/3257110852 --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f54e800c270 con 0x7f54e8003a00 2023-11-24T07:57:06.300 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:06.295+0000 7f5501245700 1 --2- 172.21.15.59:0/3257110852 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f54e8004850 0x7f54e8004cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:57:06.300 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:06.295+0000 7f5501a46700 1 --2- 172.21.15.59:0/3257110852 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f54e8003a00 0x7f54e8003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:57:06.300 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:06.295+0000 7f5502247700 1 --2- 172.21.15.59:0/3257110852 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f54e8005210 0x7f54e800ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:57:06.300 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:06.295+0000 7f5501245700 1 -- 172.21.15.59:0/3257110852 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f54e8003a00 msgr2=0x7f54e8003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T07:57:06.300 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:06.295+0000 7f5501245700 1 --2- 172.21.15.59:0/3257110852 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f54e8003a00 0x7f54e8003e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:06.300 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:06.295+0000 7f5501245700 1 -- 172.21.15.59:0/3257110852 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f54e8005210 msgr2=0x7f54e800ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T07:57:06.300 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:06.295+0000 7f5501245700 1 --2- 172.21.15.59:0/3257110852 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f54e8005210 0x7f54e800ba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:06.301 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:06.295+0000 7f5502247700 1 -- 172.21.15.59:0/3257110852 reap_dead start 2023-11-24T07:57:06.301 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:06.295+0000 7f5501245700 1 -- 172.21.15.59:0/3257110852 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f54e80035b0 con 0x7f54e8004850 2023-11-24T07:57:06.301 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:06.295+0000 7f5501245700 1 --2- 172.21.15.59:0/3257110852 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f54e8004850 0x7f54e8004cd0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f54ec009b60 tx=0x7f54e8005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8ad94dc7d5a02dc8 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:57:06.301 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:06.295+0000 7f5500a44700 1 -- 172.21.15.59:0/3257110852 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54ec00e6c0 con 0x7f54e8004850 2023-11-24T07:57:06.301 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:06.295+0000 7f5500a44700 1 -- 172.21.15.59:0/3257110852 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f54ec00e820 con 0x7f54e8004850 2023-11-24T07:57:06.301 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:06.295+0000 7f5500a44700 1 -- 172.21.15.59:0/3257110852 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54ec00eb90 con 0x7f54e8004850 2023-11-24T07:57:06.301 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:06.296+0000 7f5503caa700 1 -- 172.21.15.59:0/3257110852 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f54e8004850 msgr2=0x7f54e8004cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:57:06.301 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:06.296+0000 7f5503caa700 1 --2- 172.21.15.59:0/3257110852 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f54e8004850 0x7f54e8004cd0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f54ec009b60 tx=0x7f54e8005880 comp rx=0 tx=0).stop 2023-11-24T07:57:06.301 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:06.296+0000 7f5503caa700 1 -- 172.21.15.59:0/3257110852 shutdown_connections 2023-11-24T07:57:06.301 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:06.296+0000 7f5503caa700 1 --2- 172.21.15.59:0/3257110852 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f54e8004850 0x7f54e8004cd0 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:06.301 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:06.296+0000 7f5503caa700 1 -- 172.21.15.59:0/3257110852 >> 172.21.15.59:0/3257110852 conn(0x7f54fc0d0bc0 msgr2=0x7f54fc0d0fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:57:06.301 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:06.296+0000 7f5503caa700 1 -- 172.21.15.59:0/3257110852 shutdown_connections 2023-11-24T07:57:06.302 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:06.296+0000 7f5503caa700 1 -- 172.21.15.59:0/3257110852 wait complete. 2023-11-24T07:57:06.304 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:06.297+0000 7f5503caa700 1 Processor -- start 2023-11-24T07:57:06.304 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:06.297+0000 7f5503caa700 1 -- start start 2023-11-24T07:57:06.304 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:06.298+0000 7f5503caa700 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f54e8004850 0x7f54fc15e2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:57:06.304 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:06.298+0000 7f5503caa700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f54fc14f8f0 0x7f54fc15e820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:57:06.304 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:06.298+0000 7f5503caa700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f54fc0ce620 0x7f54fc0ceaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:57:06.305 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:06.298+0000 7f5503caa700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54fc0cf330 con 0x7f54fc14f8f0 2023-11-24T07:57:06.305 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:06.298+0000 7f5503caa700 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54fc0cf610 con 0x7f54fc0ce620 2023-11-24T07:57:06.305 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:06.298+0000 7f5503caa700 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f54fc0c60c0 con 0x7f54e8004850 2023-11-24T07:57:06.305 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:06.298+0000 7f5501245700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f54fc14f8f0 0x7f54fc15e820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:57:06.305 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:06.298+0000 7f5501245700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f54fc14f8f0 0x7f54fc15e820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:51068/0 (socket says 172.21.15.59:51068) 2023-11-24T07:57:06.305 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:06.298+0000 7f5501245700 1 -- 172.21.15.59:0/894039002 learned_addr learned my addr 172.21.15.59:0/894039002 (peer_addr_for_me v2:172.21.15.59:0/0) 2023-11-24T07:57:06.305 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:06.298+0000 7f5501a46700 1 --2- 172.21.15.59:0/894039002 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f54e8004850 0x7f54fc15e2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:57:06.305 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:06.298+0000 7f5502247700 1 --2- 172.21.15.59:0/894039002 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f54fc0ce620 0x7f54fc0ceaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:57:06.305 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:06.298+0000 7f5501245700 1 -- 172.21.15.59:0/894039002 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f54e8004850 msgr2=0x7f54fc15e2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:57:06.305 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:06.298+0000 7f5501245700 1 --2- 172.21.15.59:0/894039002 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f54e8004850 0x7f54fc15e2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:06.305 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:06.298+0000 7f5501245700 1 -- 172.21.15.59:0/894039002 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f54fc0ce620 msgr2=0x7f54fc0ceaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:57:06.306 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:06.298+0000 7f5501245700 1 --2- 172.21.15.59:0/894039002 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f54fc0ce620 0x7f54fc0ceaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:06.306 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:06.298+0000 7f5501245700 1 -- 172.21.15.59:0/894039002 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f54fc0c63a0 con 0x7f54fc14f8f0 2023-11-24T07:57:06.306 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:06.298+0000 7f5502247700 1 --2- 172.21.15.59:0/894039002 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f54fc0ce620 0x7f54fc0ceaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-24T07:57:06.306 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:06.299+0000 7f5501245700 1 --2- 172.21.15.59:0/894039002 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f54fc14f8f0 0x7f54fc15e820 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f54e800c080 tx=0x7f54ec0047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:57:06.306 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:06.299+0000 7f54f27fc700 1 -- 172.21.15.59:0/894039002 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54ec00e6c0 con 0x7f54fc14f8f0 2023-11-24T07:57:06.306 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:06.299+0000 7f54f27fc700 1 -- 172.21.15.59:0/894039002 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f54ec003ad0 con 0x7f54fc14f8f0 2023-11-24T07:57:06.306 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:06.299+0000 7f5503caa700 1 -- 172.21.15.59:0/894039002 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f54fc0c6680 con 0x7f54fc14f8f0 2023-11-24T07:57:06.306 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:06.299+0000 7f54f27fc700 1 -- 172.21.15.59:0/894039002 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54ec003e80 con 0x7f54fc14f8f0 2023-11-24T07:57:06.306 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:06.299+0000 7f5503caa700 1 -- 172.21.15.59:0/894039002 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f54fc0ca0d0 con 0x7f54fc14f8f0 2023-11-24T07:57:06.306 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:06.300+0000 7f5503caa700 1 -- 172.21.15.59:0/894039002 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f54fc0ca2c0 con 0x7f54fc14f8f0 2023-11-24T07:57:06.306 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:06.300+0000 7f54f27fc700 1 -- 172.21.15.59:0/894039002 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f54ec013470 con 0x7f54fc14f8f0 2023-11-24T07:57:06.306 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:06.301+0000 7f54f27fc700 1 --2- 172.21.15.59:0/894039002 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f54e006aed0 0x7f54e006d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:57:06.307 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:06.301+0000 7f54f27fc700 1 -- 172.21.15.59:0/894039002 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f54ec08b060 con 0x7f54fc14f8f0 2023-11-24T07:57:06.307 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:06.301+0000 7f5501a46700 1 --2- 172.21.15.59:0/894039002 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f54e006aed0 0x7f54e006d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:57:06.307 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:06.301+0000 7f5501a46700 1 --2- 172.21.15.59:0/894039002 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f54e006aed0 0x7f54e006d390 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f54f8000c00 tx=0x7f54f8000f80 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:57:06.307 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:06.305+0000 7f54f27fc700 1 -- 172.21.15.59:0/894039002 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f54ec0562c0 con 0x7f54fc14f8f0 2023-11-24T07:57:06.510 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:06.509+0000 7f5503caa700 1 -- 172.21.15.59:0/894039002 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f54fc0c6810 con 0x7f54fc14f8f0 2023-11-24T07:57:06.512 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:06.510+0000 7f54f27fc700 1 -- 172.21.15.59:0/894039002 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) v1 ==== 74+0+16437 (secure 0 0 0) 0x7f54ec0598e0 con 0x7f54fc14f8f0 2023-11-24T07:57:06.512 INFO:teuthology.orchestra.run.smithi059.stdout: 2023-11-24T07:57:06.512 INFO:teuthology.orchestra.run.smithi059.stdout:{"epoch":20,"fsid":"9f0e8741-b8f9-48d5-a2c3-09f45903303d","created":"2023-11-24T07:56:06.552729+0000","modified":"2023-11-24T07:56:25.705091+0000","last_up_change":"2023-11-24T07:56:17.372352+0000","last_in_change":"2023-11-24T07:56:09.073250+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-24T07:56:18.352823+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-11-24T07:56:22.185140+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-11-24T07:56:23.084326+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"6d8f9e9b-1782-4e41-928b-a39f2a8ebc06","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6800","nonce":3373878772},{"type":"v1","addr":"172.21.15.59:6801","nonce":3373878772}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6802","nonce":3373878772},{"type":"v1","addr":"172.21.15.59:6803","nonce":3373878772}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6806","nonce":3373878772},{"type":"v1","addr":"172.21.15.59:6807","nonce":3373878772}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6804","nonce":3373878772},{"type":"v1","addr":"172.21.15.59:6805","nonce":3373878772}]},"public_addr":"172.21.15.59:6801/3373878772","cluster_addr":"172.21.15.59:6803/3373878772","heartbeat_back_addr":"172.21.15.59:6807/3373878772","heartbeat_front_addr":"172.21.15.59:6805/3373878772","state":["exists","up"]},{"osd":1,"uuid":"e8d882aa-52e1-4c04-9e1e-c44ef9204a5f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6824","nonce":648258563},{"type":"v1","addr":"172.21.15.59:6825","nonce":648258563}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6826","nonce":648258563},{"type":"v1","addr":"172.21.15.59:6827","nonce":648258563}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6830","nonce":648258563},{"type":"v1","addr":"172.21.15.59:6831","nonce":648258563}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6828","nonce":648258563},{"type":"v1","addr":"172.21.15.59:6829","nonce":648258563}]},"public_addr":"172.21.15.59:6825/648258563","cluster_addr":"172.21.15.59:6827/648258563","heartbeat_back_addr":"172.21.15.59:6831/648258563","heartbeat_front_addr":"172.21.15.59:6829/648258563","state":["exists","up"]},{"osd":2,"uuid":"69eebfa0-695b-4b21-90b9-8a3537168a01","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6808","nonce":1840430654},{"type":"v1","addr":"172.21.15.59:6809","nonce":1840430654}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6810","nonce":1840430654},{"type":"v1","addr":"172.21.15.59:6811","nonce":1840430654}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6814","nonce":1840430654},{"type":"v1","addr":"172.21.15.59:6815","nonce":1840430654}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6812","nonce":1840430654},{"type":"v1","addr":"172.21.15.59:6813","nonce":1840430654}]},"public_addr":"172.21.15.59:6809/1840430654","cluster_addr":"172.21.15.59:6811/1840430654","heartbeat_back_addr":"172.21.15.59:6815/1840430654","heartbeat_front_addr":"172.21.15.59:6813/1840430654","state":["exists","up"]},{"osd":3,"uuid":"96fd77fc-f911-46ff-9445-6f0be2daa8f4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6816","nonce":2504025850},{"type":"v1","addr":"172.21.15.59:6817","nonce":2504025850}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6818","nonce":2504025850},{"type":"v1","addr":"172.21.15.59:6819","nonce":2504025850}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6822","nonce":2504025850},{"type":"v1","addr":"172.21.15.59:6823","nonce":2504025850}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6820","nonce":2504025850},{"type":"v1","addr":"172.21.15.59:6821","nonce":2504025850}]},"public_addr":"172.21.15.59:6817/2504025850","cluster_addr":"172.21.15.59:6819/2504025850","heartbeat_back_addr":"172.21.15.59:6823/2504025850","heartbeat_front_addr":"172.21.15.59:6821/2504025850","state":["exists","up"]},{"osd":4,"uuid":"7ac29fca-b5b2-4c95-9d69-2f164cbbc708","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6816","nonce":2382463431},{"type":"v1","addr":"172.21.15.76:6818","nonce":2382463431}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6820","nonce":2382463431},{"type":"v1","addr":"172.21.15.76:6822","nonce":2382463431}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6828","nonce":2382463431},{"type":"v1","addr":"172.21.15.76:6830","nonce":2382463431}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6824","nonce":2382463431},{"type":"v1","addr":"172.21.15.76:6826","nonce":2382463431}]},"public_addr":"172.21.15.76:6818/2382463431","cluster_addr":"172.21.15.76:6822/2382463431","heartbeat_back_addr":"172.21.15.76:6830/2382463431","heartbeat_front_addr":"172.21.15.76:6826/2382463431","state":["exists","up"]},{"osd":5,"uuid":"485bf711-3851-4238-a4f4-51d52e0a9c58","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6800","nonce":329334308},{"type":"v1","addr":"172.21.15.76:6802","nonce":329334308}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6805","nonce":329334308},{"type":"v1","addr":"172.21.15.76:6807","nonce":329334308}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6814","nonce":329334308},{"type":"v1","addr":"172.21.15.76:6815","nonce":329334308}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6810","nonce":329334308},{"type":"v1","addr":"172.21.15.76:6812","nonce":329334308}]},"public_addr":"172.21.15.76:6802/329334308","cluster_addr":"172.21.15.76:6807/329334308","heartbeat_back_addr":"172.21.15.76:6815/329334308","heartbeat_front_addr":"172.21.15.76:6812/329334308","state":["exists","up"]},{"osd":6,"uuid":"72b3f2fc-1e4a-4e32-968d-003647a0286f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6801","nonce":3946579410},{"type":"v1","addr":"172.21.15.76:6803","nonce":3946579410}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6804","nonce":3946579410},{"type":"v1","addr":"172.21.15.76:6806","nonce":3946579410}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6811","nonce":3946579410},{"type":"v1","addr":"172.21.15.76:6813","nonce":3946579410}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6808","nonce":3946579410},{"type":"v1","addr":"172.21.15.76:6809","nonce":3946579410}]},"public_addr":"172.21.15.76:6803/3946579410","cluster_addr":"172.21.15.76:6806/3946579410","heartbeat_back_addr":"172.21.15.76:6813/3946579410","heartbeat_front_addr":"172.21.15.76:6809/3946579410","state":["exists","up"]},{"osd":7,"uuid":"1ddceb90-6c5d-4d1a-ad56-99cb877f8c70","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6817","nonce":841761538},{"type":"v1","addr":"172.21.15.76:6819","nonce":841761538}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6821","nonce":841761538},{"type":"v1","addr":"172.21.15.76:6823","nonce":841761538}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6829","nonce":841761538},{"type":"v1","addr":"172.21.15.76:6831","nonce":841761538}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6825","nonce":841761538},{"type":"v1","addr":"172.21.15.76:6827","nonce":841761538}]},"public_addr":"172.21.15.76:6819/841761538","cluster_addr":"172.21.15.76:6823/841761538","heartbeat_back_addr":"172.21.15.76:6831/841761538","heartbeat_front_addr":"172.21.15.76:6827/841761538","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T07:56:14.294838+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T07:56:14.270779+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T07:56:14.231809+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T07:56:14.260983+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T07:56:14.302997+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T07:56:14.258600+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T07:56:14.251413+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T07:56:14.304727+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-24T07:57:06.514 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:06.512+0000 7f5503caa700 1 -- 172.21.15.59:0/894039002 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f54e006aed0 msgr2=0x7f54e006d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:57:06.514 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:06.512+0000 7f5503caa700 1 --2- 172.21.15.59:0/894039002 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f54e006aed0 0x7f54e006d390 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f54f8000c00 tx=0x7f54f8000f80 comp rx=0 tx=0).stop 2023-11-24T07:57:06.514 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:06.512+0000 7f5503caa700 1 -- 172.21.15.59:0/894039002 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f54fc14f8f0 msgr2=0x7f54fc15e820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:57:06.514 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:06.512+0000 7f5503caa700 1 --2- 172.21.15.59:0/894039002 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f54fc14f8f0 0x7f54fc15e820 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f54e800c080 tx=0x7f54ec0047c0 comp rx=0 tx=0).stop 2023-11-24T07:57:06.514 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:06.512+0000 7f5503caa700 1 -- 172.21.15.59:0/894039002 shutdown_connections 2023-11-24T07:57:06.515 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:06.512+0000 7f5503caa700 1 --2- 172.21.15.59:0/894039002 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f54e006aed0 0x7f54e006d390 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:06.515 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:06.512+0000 7f5503caa700 1 --2- 172.21.15.59:0/894039002 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f54fc0ce620 0x7f54fc0ceaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:06.515 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:06.512+0000 7f5503caa700 1 --2- 172.21.15.59:0/894039002 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f54e8004850 0x7f54fc15e2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:06.515 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:06.512+0000 7f5503caa700 1 --2- 172.21.15.59:0/894039002 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f54fc14f8f0 0x7f54fc15e820 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:06.515 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:06.512+0000 7f5503caa700 1 -- 172.21.15.59:0/894039002 >> 172.21.15.59:0/894039002 conn(0x7f54fc0d0bc0 msgr2=0x7f54fc15d2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:57:06.515 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:06.512+0000 7f5503caa700 1 -- 172.21.15.59:0/894039002 shutdown_connections 2023-11-24T07:57:06.515 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:06.512+0000 7f5503caa700 1 -- 172.21.15.59:0/894039002 wait complete. 2023-11-24T07:57:06.526 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-24T07:56:18.352823+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-24T07:56:22.185140+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-11-24T07:56:23.084326+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-24T07:57:06.527 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-24T07:57:06.654 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:06.653+0000 7fcf8444f700 1 Processor -- start 2023-11-24T07:57:06.655 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:06.653+0000 7fcf8444f700 1 -- start start 2023-11-24T07:57:06.657 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:06.654+0000 7fcf8444f700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fcf7c143ae0 0x7fcf7c0b51b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:57:06.657 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:06.654+0000 7fcf8444f700 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fcf7c1440f0 0x7fcf7c071bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:57:06.657 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:06.654+0000 7fcf8444f700 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcf7c143ec0 con 0x7fcf7c0720f0 2023-11-24T07:57:06.657 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:06.654+0000 7fcf8444f700 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcf7c0b40a0 con 0x7fcf7c143ae0 2023-11-24T07:57:06.657 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:06.654+0000 7fcf8444f700 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcf7c072580 con 0x7fcf7c1440f0 2023-11-24T07:57:06.657 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:06.654+0000 7fcf821eb700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fcf7c143ae0 0x7fcf7c0b51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:57:06.657 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:06.654+0000 7fcf821eb700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fcf7c143ae0 0x7fcf7c0b51b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:51074/0 (socket says 172.21.15.59:51074) 2023-11-24T07:57:06.658 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:06.654+0000 7fcf821eb700 1 -- 172.21.15.59:0/2598535530 learned_addr learned my addr 172.21.15.59:0/2598535530 (peer_addr_for_me v2:172.21.15.59:0/0) 2023-11-24T07:57:06.658 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:06.654+0000 7fcf819ea700 1 --2- 172.21.15.59:0/2598535530 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fcf7c1440f0 0x7fcf7c071bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:57:06.658 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:06.654+0000 7fcf821eb700 1 -- 172.21.15.59:0/2598535530 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fcf7c1440f0 msgr2=0x7fcf7c071bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T07:57:06.658 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:06.654+0000 7fcf821eb700 1 --2- 172.21.15.59:0/2598535530 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fcf7c1440f0 0x7fcf7c071bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:06.658 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:06.654+0000 7fcf821eb700 1 -- 172.21.15.59:0/2598535530 >> v1:172.21.15.76:6789/0 conn(0x7fcf7c0720f0 legacy=0x7fcf7c15e8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:57:06.658 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:06.654+0000 7fcf821eb700 1 -- 172.21.15.59:0/2598535530 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcf7c0b3ae0 con 0x7fcf7c143ae0 2023-11-24T07:57:06.658 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:06.654+0000 7fcf819ea700 1 --2- 172.21.15.59:0/2598535530 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fcf7c1440f0 0x7fcf7c071bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-24T07:57:06.658 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:06.654+0000 7fcf821eb700 1 --2- 172.21.15.59:0/2598535530 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fcf7c143ae0 0x7fcf7c0b51b0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7fcf7800bb90 tx=0x7fcf7800bea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2cbc4695c68b98e6 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:57:06.658 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:06.655+0000 7fcf811e9700 1 -- 172.21.15.59:0/2598535530 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf7800dd50 con 0x7fcf7c143ae0 2023-11-24T07:57:06.658 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:06.655+0000 7fcf811e9700 1 -- 172.21.15.59:0/2598535530 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcf78010470 con 0x7fcf7c143ae0 2023-11-24T07:57:06.658 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:06.655+0000 7fcf811e9700 1 -- 172.21.15.59:0/2598535530 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf78010800 con 0x7fcf7c143ae0 2023-11-24T07:57:06.659 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:06.655+0000 7fcf8444f700 1 -- 172.21.15.59:0/2598535530 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fcf7c143ae0 msgr2=0x7fcf7c0b51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:57:06.659 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:06.655+0000 7fcf8444f700 1 --2- 172.21.15.59:0/2598535530 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fcf7c143ae0 0x7fcf7c0b51b0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7fcf7800bb90 tx=0x7fcf7800bea0 comp rx=0 tx=0).stop 2023-11-24T07:57:06.659 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:06.655+0000 7fcf8444f700 1 -- 172.21.15.59:0/2598535530 shutdown_connections 2023-11-24T07:57:06.659 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:06.655+0000 7fcf8444f700 1 --2- 172.21.15.59:0/2598535530 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fcf7c143ae0 0x7fcf7c0b51b0 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:06.659 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:06.655+0000 7fcf8444f700 1 --2- 172.21.15.59:0/2598535530 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fcf7c1440f0 0x7fcf7c071bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:06.659 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:06.655+0000 7fcf8444f700 1 -- 172.21.15.59:0/2598535530 >> 172.21.15.59:0/2598535530 conn(0x7fcf7c0b2740 msgr2=0x7fcf7c0b2b20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:57:06.659 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:06.655+0000 7fcf8444f700 1 -- 172.21.15.59:0/2598535530 shutdown_connections 2023-11-24T07:57:06.659 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:06.655+0000 7fcf8444f700 1 -- 172.21.15.59:0/2598535530 wait complete. 2023-11-24T07:57:06.662 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:06.656+0000 7fcf8444f700 1 Processor -- start 2023-11-24T07:57:06.662 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:06.657+0000 7fcf8444f700 1 -- start start 2023-11-24T07:57:06.662 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:06.657+0000 7fcf8444f700 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fcf7c0720f0 0x7fcf7c156510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:57:06.663 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:06.657+0000 7fcf8444f700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fcf7c143ae0 0x7fcf7c142580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:57:06.663 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:06.657+0000 7fcf8444f700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fcf7c1440f0 0x7fcf7c152b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:57:06.663 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:06.657+0000 7fcf8444f700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf7c159420 con 0x7fcf7c1440f0 2023-11-24T07:57:06.663 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:06.657+0000 7fcf8444f700 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf7c142ac0 con 0x7fcf7c143ae0 2023-11-24T07:57:06.663 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:06.657+0000 7fcf8444f700 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcf7c156bd0 con 0x7fcf7c0720f0 2023-11-24T07:57:06.663 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:06.657+0000 7fcf829ec700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fcf7c1440f0 0x7fcf7c152b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:57:06.663 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:06.657+0000 7fcf829ec700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fcf7c1440f0 0x7fcf7c152b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:51084/0 (socket says 172.21.15.59:51084) 2023-11-24T07:57:06.663 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:06.657+0000 7fcf829ec700 1 -- 172.21.15.59:0/2248010801 learned_addr learned my addr 172.21.15.59:0/2248010801 (peer_addr_for_me v2:172.21.15.59:0/0) 2023-11-24T07:57:06.663 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:06.658+0000 7fcf819ea700 1 --2- 172.21.15.59:0/2248010801 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fcf7c143ae0 0x7fcf7c142580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:57:06.664 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:06.658+0000 7fcf829ec700 1 -- 172.21.15.59:0/2248010801 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fcf7c0720f0 msgr2=0x7fcf7c156510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:57:06.664 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:06.658+0000 7fcf821eb700 1 --2- 172.21.15.59:0/2248010801 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fcf7c0720f0 0x7fcf7c156510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:57:06.664 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:06.658+0000 7fcf829ec700 1 --2- 172.21.15.59:0/2248010801 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fcf7c0720f0 0x7fcf7c156510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:06.664 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:06.658+0000 7fcf829ec700 1 -- 172.21.15.59:0/2248010801 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fcf7c143ae0 msgr2=0x7fcf7c142580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:57:06.664 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:06.658+0000 7fcf829ec700 1 --2- 172.21.15.59:0/2248010801 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fcf7c143ae0 0x7fcf7c142580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:06.664 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:06.658+0000 7fcf829ec700 1 -- 172.21.15.59:0/2248010801 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcf7800b7f0 con 0x7fcf7c1440f0 2023-11-24T07:57:06.664 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:06.658+0000 7fcf819ea700 1 --2- 172.21.15.59:0/2248010801 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fcf7c143ae0 0x7fcf7c142580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T07:57:06.664 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:06.658+0000 7fcf821eb700 1 --2- 172.21.15.59:0/2248010801 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fcf7c0720f0 0x7fcf7c156510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T07:57:06.664 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:06.658+0000 7fcf829ec700 1 --2- 172.21.15.59:0/2248010801 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fcf7c1440f0 0x7fcf7c152b50 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7fcf7400cc50 tx=0x7fcf7400a340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:57:06.665 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:06.658+0000 7fcf72ffd700 1 -- 172.21.15.59:0/2248010801 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf7400ad80 con 0x7fcf7c1440f0 2023-11-24T07:57:06.665 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:06.658+0000 7fcf72ffd700 1 -- 172.21.15.59:0/2248010801 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcf7400ce50 con 0x7fcf7c1440f0 2023-11-24T07:57:06.665 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:06.658+0000 7fcf8444f700 1 -- 172.21.15.59:0/2248010801 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcf7c156eb0 con 0x7fcf7c1440f0 2023-11-24T07:57:06.665 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:06.658+0000 7fcf72ffd700 1 -- 172.21.15.59:0/2248010801 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf74003c40 con 0x7fcf7c1440f0 2023-11-24T07:57:06.666 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:06.658+0000 7fcf8444f700 1 -- 172.21.15.59:0/2248010801 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcf7c157470 con 0x7fcf7c1440f0 2023-11-24T07:57:06.666 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:06.659+0000 7fcf8444f700 1 -- 172.21.15.59:0/2248010801 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcf7c075480 con 0x7fcf7c1440f0 2023-11-24T07:57:06.666 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:06.660+0000 7fcf72ffd700 1 -- 172.21.15.59:0/2248010801 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fcf740050d0 con 0x7fcf7c1440f0 2023-11-24T07:57:06.666 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:06.660+0000 7fcf72ffd700 1 --2- 172.21.15.59:0/2248010801 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7fcf6006aed0 0x7fcf6006d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:57:06.666 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:06.660+0000 7fcf72ffd700 1 -- 172.21.15.59:0/2248010801 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fcf7408a110 con 0x7fcf7c1440f0 2023-11-24T07:57:06.666 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:06.660+0000 7fcf821eb700 1 --2- 172.21.15.59:0/2248010801 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7fcf6006aed0 0x7fcf6006d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:57:06.667 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:06.661+0000 7fcf821eb700 1 --2- 172.21.15.59:0/2248010801 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7fcf6006aed0 0x7fcf6006d390 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fcf7800b340 tx=0x7fcf78013000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:57:06.667 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:06.664+0000 7fcf72ffd700 1 -- 172.21.15.59:0/2248010801 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fcf740552a0 con 0x7fcf7c1440f0 2023-11-24T07:57:06.863 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:06.861+0000 7fcf8444f700 1 -- 172.21.15.59:0/2248010801 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fcf7c144830 con 0x7fcf7c1440f0 2023-11-24T07:57:06.863 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:06.862+0000 7fcf72ffd700 1 -- 172.21.15.59:0/2248010801 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v20) v1 ==== 93+0+10 (secure 0 0 0) 0x7fcf740588c0 con 0x7fcf7c1440f0 2023-11-24T07:57:06.864 INFO:teuthology.orchestra.run.smithi059.stdout:pg_num: 1 2023-11-24T07:57:06.866 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:06.864+0000 7fcf8444f700 1 -- 172.21.15.59:0/2248010801 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7fcf6006aed0 msgr2=0x7fcf6006d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:57:06.866 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:06.864+0000 7fcf8444f700 1 --2- 172.21.15.59:0/2248010801 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7fcf6006aed0 0x7fcf6006d390 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fcf7800b340 tx=0x7fcf78013000 comp rx=0 tx=0).stop 2023-11-24T07:57:06.866 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:06.864+0000 7fcf8444f700 1 -- 172.21.15.59:0/2248010801 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fcf7c1440f0 msgr2=0x7fcf7c152b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:57:06.866 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:06.864+0000 7fcf8444f700 1 --2- 172.21.15.59:0/2248010801 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fcf7c1440f0 0x7fcf7c152b50 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7fcf7400cc50 tx=0x7fcf7400a340 comp rx=0 tx=0).stop 2023-11-24T07:57:06.866 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:06.864+0000 7fcf8444f700 1 -- 172.21.15.59:0/2248010801 shutdown_connections 2023-11-24T07:57:06.866 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:06.864+0000 7fcf8444f700 1 --2- 172.21.15.59:0/2248010801 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7fcf6006aed0 0x7fcf6006d390 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:06.867 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:06.864+0000 7fcf8444f700 1 --2- 172.21.15.59:0/2248010801 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fcf7c1440f0 0x7fcf7c152b50 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:06.867 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:06.864+0000 7fcf8444f700 1 --2- 172.21.15.59:0/2248010801 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fcf7c0720f0 0x7fcf7c156510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:06.867 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:06.864+0000 7fcf8444f700 1 --2- 172.21.15.59:0/2248010801 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fcf7c143ae0 0x7fcf7c142580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:06.867 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:06.864+0000 7fcf8444f700 1 -- 172.21.15.59:0/2248010801 >> 172.21.15.59:0/2248010801 conn(0x7fcf7c0b2740 msgr2=0x7fcf7c0cdf90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:57:06.867 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:06.864+0000 7fcf8444f700 1 -- 172.21.15.59:0/2248010801 shutdown_connections 2023-11-24T07:57:06.867 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:06.864+0000 7fcf8444f700 1 -- 172.21.15.59:0/2248010801 wait complete. 2023-11-24T07:57:06.878 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-24T07:57:07.006 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:07.004+0000 7fbfcaba1700 1 Processor -- start 2023-11-24T07:57:07.006 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:07.004+0000 7fbfcaba1700 1 -- start start 2023-11-24T07:57:07.009 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:07.006+0000 7fbfcaba1700 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fbfc415ad70 0x7fbfc415e860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:57:07.009 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:07.006+0000 7fbfcaba1700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fbfc4066ce0 0x7fbfc415eda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:57:07.009 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:07.006+0000 7fbfcaba1700 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbfc406d2e0 con 0x7fbfc415f2e0 2023-11-24T07:57:07.009 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:07.006+0000 7fbfcaba1700 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbfc415b150 con 0x7fbfc4066ce0 2023-11-24T07:57:07.009 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:07.006+0000 7fbfcaba1700 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbfc40ce160 con 0x7fbfc415ad70 2023-11-24T07:57:07.009 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:07.006+0000 7fbfc939e700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fbfc4066ce0 0x7fbfc415eda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:57:07.009 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:07.006+0000 7fbfc939e700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fbfc4066ce0 0x7fbfc415eda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:51090/0 (socket says 172.21.15.59:51090) 2023-11-24T07:57:07.009 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:07.006+0000 7fbfc939e700 1 -- 172.21.15.59:0/439053577 learned_addr learned my addr 172.21.15.59:0/439053577 (peer_addr_for_me v2:172.21.15.59:0/0) 2023-11-24T07:57:07.009 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:07.006+0000 7fbfc9b9f700 1 --2- 172.21.15.59:0/439053577 >> v2:172.21.15.76:3300/0 conn(0x7fbfc415ad70 0x7fbfc415e860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:57:07.010 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:07.006+0000 7fbfc939e700 1 -- 172.21.15.59:0/439053577 >> v2:172.21.15.76:3300/0 conn(0x7fbfc415ad70 msgr2=0x7fbfc415e860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T07:57:07.010 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:07.006+0000 7fbfc939e700 1 --2- 172.21.15.59:0/439053577 >> v2:172.21.15.76:3300/0 conn(0x7fbfc415ad70 0x7fbfc415e860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:07.010 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:07.006+0000 7fbfc939e700 1 -- 172.21.15.59:0/439053577 >> v1:172.21.15.76:6789/0 conn(0x7fbfc415f2e0 legacy=0x7fbfc406afd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:57:07.010 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:07.006+0000 7fbfc939e700 1 -- 172.21.15.59:0/439053577 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbfc40ce440 con 0x7fbfc4066ce0 2023-11-24T07:57:07.010 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:07.006+0000 7fbfc939e700 1 --2- 172.21.15.59:0/439053577 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbfc4066ce0 0x7fbfc415eda0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7fbfc4069310 tx=0x7fbfc0009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a0e8ceb0a512c2c3 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:57:07.010 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:07.006+0000 7fbfc8b9d700 1 -- 172.21.15.59:0/439053577 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbfc0011830 con 0x7fbfc4066ce0 2023-11-24T07:57:07.010 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:07.006+0000 7fbfc8b9d700 1 -- 172.21.15.59:0/439053577 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbfc0011990 con 0x7fbfc4066ce0 2023-11-24T07:57:07.010 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:07.007+0000 7fbfc8b9d700 1 -- 172.21.15.59:0/439053577 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbfc0011d20 con 0x7fbfc4066ce0 2023-11-24T07:57:07.010 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:07.007+0000 7fbfcaba1700 1 -- 172.21.15.59:0/439053577 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbfc4066ce0 msgr2=0x7fbfc415eda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:57:07.010 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:07.007+0000 7fbfcaba1700 1 --2- 172.21.15.59:0/439053577 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbfc4066ce0 0x7fbfc415eda0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7fbfc4069310 tx=0x7fbfc0009bd0 comp rx=0 tx=0).stop 2023-11-24T07:57:07.011 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:07.007+0000 7fbfcaba1700 1 -- 172.21.15.59:0/439053577 shutdown_connections 2023-11-24T07:57:07.011 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:07.007+0000 7fbfcaba1700 1 --2- 172.21.15.59:0/439053577 >> v2:172.21.15.76:3300/0 conn(0x7fbfc415ad70 0x7fbfc415e860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:07.011 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:07.007+0000 7fbfcaba1700 1 --2- 172.21.15.59:0/439053577 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbfc4066ce0 0x7fbfc415eda0 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:07.011 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:07.007+0000 7fbfcaba1700 1 -- 172.21.15.59:0/439053577 >> 172.21.15.59:0/439053577 conn(0x7fbfc40d00c0 msgr2=0x7fbfc40d04a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:57:07.011 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:07.007+0000 7fbfcaba1700 1 -- 172.21.15.59:0/439053577 shutdown_connections 2023-11-24T07:57:07.011 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:07.007+0000 7fbfcaba1700 1 -- 172.21.15.59:0/439053577 wait complete. 2023-11-24T07:57:07.014 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:07.008+0000 7fbfcaba1700 1 Processor -- start 2023-11-24T07:57:07.014 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:07.008+0000 7fbfcaba1700 1 -- start start 2023-11-24T07:57:07.015 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:07.009+0000 7fbfcaba1700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbfc4066ce0 0x7fbfc409d1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:57:07.015 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:07.009+0000 7fbfcaba1700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fbfc415ad70 0x7fbfc41685b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:57:07.015 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:07.009+0000 7fbfcaba1700 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fbfc415d570 0x7fbfc4153b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:57:07.015 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:07.009+0000 7fbfcaba1700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbfc409a9f0 con 0x7fbfc4066ce0 2023-11-24T07:57:07.015 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:07.009+0000 7fbfcaba1700 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbfc409acd0 con 0x7fbfc415ad70 2023-11-24T07:57:07.015 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:07.009+0000 7fbfcaba1700 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbfc409afb0 con 0x7fbfc415d570 2023-11-24T07:57:07.015 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:07.009+0000 7fbfc9b9f700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbfc4066ce0 0x7fbfc409d1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:57:07.015 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:07.009+0000 7fbfc9b9f700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbfc4066ce0 0x7fbfc409d1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:51094/0 (socket says 172.21.15.59:51094) 2023-11-24T07:57:07.015 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:07.009+0000 7fbfc9b9f700 1 -- 172.21.15.59:0/1523680307 learned_addr learned my addr 172.21.15.59:0/1523680307 (peer_addr_for_me v2:172.21.15.59:0/0) 2023-11-24T07:57:07.015 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:07.009+0000 7fbfc9b9f700 1 -- 172.21.15.59:0/1523680307 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fbfc415d570 msgr2=0x7fbfc4153b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:57:07.016 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:07.009+0000 7fbfc9b9f700 1 --2- 172.21.15.59:0/1523680307 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fbfc415d570 0x7fbfc4153b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:07.016 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:07.009+0000 7fbfc9b9f700 1 -- 172.21.15.59:0/1523680307 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fbfc415ad70 msgr2=0x7fbfc41685b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:57:07.016 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:07.009+0000 7fbfc9b9f700 1 --2- 172.21.15.59:0/1523680307 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fbfc415ad70 0x7fbfc41685b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:07.016 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:07.009+0000 7fbfc9b9f700 1 -- 172.21.15.59:0/1523680307 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbfc0009550 con 0x7fbfc4066ce0 2023-11-24T07:57:07.016 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:07.010+0000 7fbfc9b9f700 1 --2- 172.21.15.59:0/1523680307 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbfc4066ce0 0x7fbfc409d1c0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7fbfb400e9b0 tx=0x7fbfb400ed70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:57:07.016 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:07.010+0000 7fbfba7fc700 1 -- 172.21.15.59:0/1523680307 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbfb4009710 con 0x7fbfc4066ce0 2023-11-24T07:57:07.016 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:07.010+0000 7fbfba7fc700 1 -- 172.21.15.59:0/1523680307 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbfb400c850 con 0x7fbfc4066ce0 2023-11-24T07:57:07.016 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:07.010+0000 7fbfcaba1700 1 -- 172.21.15.59:0/1523680307 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbfc409b290 con 0x7fbfc4066ce0 2023-11-24T07:57:07.016 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:07.010+0000 7fbfba7fc700 1 -- 172.21.15.59:0/1523680307 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbfb400cbc0 con 0x7fbfc4066ce0 2023-11-24T07:57:07.017 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:07.010+0000 7fbfcaba1700 1 -- 172.21.15.59:0/1523680307 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbfc4168af0 con 0x7fbfc4066ce0 2023-11-24T07:57:07.017 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:07.011+0000 7fbfcaba1700 1 -- 172.21.15.59:0/1523680307 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbfc4075300 con 0x7fbfc4066ce0 2023-11-24T07:57:07.017 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:07.011+0000 7fbfba7fc700 1 -- 172.21.15.59:0/1523680307 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fbfb400cd20 con 0x7fbfc4066ce0 2023-11-24T07:57:07.017 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:07.012+0000 7fbfba7fc700 1 --2- 172.21.15.59:0/1523680307 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7fbfa806aed0 0x7fbfa806d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:57:07.017 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:07.012+0000 7fbfba7fc700 1 -- 172.21.15.59:0/1523680307 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fbfb40896d0 con 0x7fbfc4066ce0 2023-11-24T07:57:07.017 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:07.012+0000 7fbfc939e700 1 --2- 172.21.15.59:0/1523680307 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7fbfa806aed0 0x7fbfa806d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:57:07.017 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:07.012+0000 7fbfc939e700 1 --2- 172.21.15.59:0/1523680307 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7fbfa806aed0 0x7fbfa806d390 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fbfc409e2a0 tx=0x7fbfc0010000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:57:07.018 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:07.016+0000 7fbfba7fc700 1 -- 172.21.15.59:0/1523680307 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fbfb4054780 con 0x7fbfc4066ce0 2023-11-24T07:57:07.216 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:07.214+0000 7fbfcaba1700 1 -- 172.21.15.59:0/1523680307 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fbfc40ae570 con 0x7fbfc4066ce0 2023-11-24T07:57:07.216 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:07.215+0000 7fbfba7fc700 1 -- 172.21.15.59:0/1523680307 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v20) v1 ==== 104+0+11 (secure 0 0 0) 0x7fbfb4057da0 con 0x7fbfc4066ce0 2023-11-24T07:57:07.217 INFO:teuthology.orchestra.run.smithi059.stdout:pg_num: 32 2023-11-24T07:57:07.219 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:07.217+0000 7fbfcaba1700 1 -- 172.21.15.59:0/1523680307 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7fbfa806aed0 msgr2=0x7fbfa806d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:57:07.219 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:07.217+0000 7fbfcaba1700 1 --2- 172.21.15.59:0/1523680307 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7fbfa806aed0 0x7fbfa806d390 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fbfc409e2a0 tx=0x7fbfc0010000 comp rx=0 tx=0).stop 2023-11-24T07:57:07.219 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:07.217+0000 7fbfcaba1700 1 -- 172.21.15.59:0/1523680307 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbfc4066ce0 msgr2=0x7fbfc409d1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:57:07.219 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:07.217+0000 7fbfcaba1700 1 --2- 172.21.15.59:0/1523680307 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbfc4066ce0 0x7fbfc409d1c0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7fbfb400e9b0 tx=0x7fbfb400ed70 comp rx=0 tx=0).stop 2023-11-24T07:57:07.219 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:07.217+0000 7fbfcaba1700 1 -- 172.21.15.59:0/1523680307 shutdown_connections 2023-11-24T07:57:07.219 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:07.217+0000 7fbfcaba1700 1 --2- 172.21.15.59:0/1523680307 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7fbfa806aed0 0x7fbfa806d390 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:07.219 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:07.217+0000 7fbfcaba1700 1 --2- 172.21.15.59:0/1523680307 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fbfc415d570 0x7fbfc4153b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:07.220 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:07.217+0000 7fbfcaba1700 1 --2- 172.21.15.59:0/1523680307 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbfc4066ce0 0x7fbfc409d1c0 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:07.220 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:07.217+0000 7fbfcaba1700 1 --2- 172.21.15.59:0/1523680307 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fbfc415ad70 0x7fbfc41685b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:07.220 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:07.217+0000 7fbfcaba1700 1 -- 172.21.15.59:0/1523680307 >> 172.21.15.59:0/1523680307 conn(0x7fbfc40d00c0 msgr2=0x7fbfc406b970 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:57:07.220 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:07.217+0000 7fbfcaba1700 1 -- 172.21.15.59:0/1523680307 shutdown_connections 2023-11-24T07:57:07.220 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:07.217+0000 7fbfcaba1700 1 -- 172.21.15.59:0/1523680307 wait complete. 2023-11-24T07:57:07.231 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-24T07:57:07.358 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:07.356+0000 7fc3aee69700 1 Processor -- start 2023-11-24T07:57:07.358 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:07.356+0000 7fc3aee69700 1 -- start start 2023-11-24T07:57:07.362 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:07.358+0000 7fc3aee69700 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fc3a80d1100 0x7fc3a815e7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:57:07.363 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:07.358+0000 7fc3aee69700 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fc3a806f700 0x7fc3a815ed00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:57:07.363 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:07.358+0000 7fc3aee69700 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc3a806e9d0 con 0x7fc3a8144610 2023-11-24T07:57:07.363 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:07.358+0000 7fc3aee69700 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc3a806f270 con 0x7fc3a806f700 2023-11-24T07:57:07.363 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:07.358+0000 7fc3aee69700 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc3a81449f0 con 0x7fc3a80d1100 2023-11-24T07:57:07.363 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:07.358+0000 7fc3acc05700 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7fc3a8144610 0x7fc3a80d09f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:54576/0 (socket says 172.21.15.59:54576) 2023-11-24T07:57:07.363 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:07.358+0000 7fc3acc05700 1 -- 172.21.15.59:0/3005829735 learned_addr learned my addr 172.21.15.59:0/3005829735 (peer_addr_for_me v1:172.21.15.59:0/0) 2023-11-24T07:57:07.363 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:07.358+0000 7fc3a7fff700 1 --2- 172.21.15.59:0/3005829735 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fc3a80d1100 0x7fc3a815e7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:57:07.364 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:07.358+0000 7fc3ad406700 1 --2- 172.21.15.59:0/3005829735 >> v2:172.21.15.76:3300/0 conn(0x7fc3a806f700 0x7fc3a815ed00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:57:07.364 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:07.358+0000 7fc3a77fe700 1 -- 172.21.15.59:0/3005829735 <== mon.0 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2595948230 0 0) 0x7fc3a806e9d0 con 0x7fc3a8144610 2023-11-24T07:57:07.364 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:07.358+0000 7fc3a77fe700 1 -- 172.21.15.59:0/3005829735 --> v1:172.21.15.59:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc3980035b0 con 0x7fc3a8144610 2023-11-24T07:57:07.364 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:07.358+0000 7fc3a7fff700 1 -- 172.21.15.59:0/3005829735 >> v2:172.21.15.76:3300/0 conn(0x7fc3a806f700 msgr2=0x7fc3a815ed00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T07:57:07.364 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:07.358+0000 7fc3a7fff700 1 --2- 172.21.15.59:0/3005829735 >> v2:172.21.15.76:3300/0 conn(0x7fc3a806f700 0x7fc3a815ed00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:07.364 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:07.358+0000 7fc3a7fff700 1 -- 172.21.15.59:0/3005829735 >> v1:172.21.15.59:6789/0 conn(0x7fc3a8144610 legacy=0x7fc3a80d09f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:57:07.364 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:07.358+0000 7fc3a7fff700 1 -- 172.21.15.59:0/3005829735 --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc3a815f530 con 0x7fc3a80d1100 2023-11-24T07:57:07.364 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:07.358+0000 7fc3ad406700 1 --2- 172.21.15.59:0/3005829735 >> v2:172.21.15.76:3300/0 conn(0x7fc3a806f700 0x7fc3a815ed00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-24T07:57:07.364 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:07.359+0000 7fc3a7fff700 1 --2- 172.21.15.59:0/3005829735 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fc3a80d1100 0x7fc3a815e7c0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fc394009840 tx=0x7fc394009b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=58806d454d0b87f8 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:57:07.365 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:07.359+0000 7fc3a77fe700 1 -- 172.21.15.59:0/3005829735 <== mon.2 v2:172.21.15.76:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc394011830 con 0x7fc3a80d1100 2023-11-24T07:57:07.365 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:07.359+0000 7fc3a77fe700 1 -- 172.21.15.59:0/3005829735 <== mon.2 v2:172.21.15.76:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc394011990 con 0x7fc3a80d1100 2023-11-24T07:57:07.365 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:07.359+0000 7fc3a77fe700 1 -- 172.21.15.59:0/3005829735 <== mon.2 v2:172.21.15.76:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc394011d20 con 0x7fc3a80d1100 2023-11-24T07:57:07.365 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:07.359+0000 7fc3aee69700 1 -- 172.21.15.59:0/3005829735 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fc3a80d1100 msgr2=0x7fc3a815e7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:57:07.365 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:07.359+0000 7fc3aee69700 1 --2- 172.21.15.59:0/3005829735 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fc3a80d1100 0x7fc3a815e7c0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fc394009840 tx=0x7fc394009b50 comp rx=0 tx=0).stop 2023-11-24T07:57:07.366 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:07.359+0000 7fc3aee69700 1 -- 172.21.15.59:0/3005829735 shutdown_connections 2023-11-24T07:57:07.366 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:07.359+0000 7fc3aee69700 1 --2- 172.21.15.59:0/3005829735 >> v2:172.21.15.76:3300/0 conn(0x7fc3a806f700 0x7fc3a815ed00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:07.366 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:07.359+0000 7fc3aee69700 1 --2- 172.21.15.59:0/3005829735 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fc3a80d1100 0x7fc3a815e7c0 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:07.366 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:07.359+0000 7fc3aee69700 1 -- 172.21.15.59:0/3005829735 >> 172.21.15.59:0/3005829735 conn(0x7fc3a80b2740 msgr2=0x7fc3a80b2b50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:57:07.366 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:07.359+0000 7fc3aee69700 1 -- 172.21.15.59:0/3005829735 shutdown_connections 2023-11-24T07:57:07.366 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:07.359+0000 7fc3aee69700 1 -- 172.21.15.59:0/3005829735 wait complete. 2023-11-24T07:57:07.366 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:07.360+0000 7fc3aee69700 1 Processor -- start 2023-11-24T07:57:07.367 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:07.361+0000 7fc3aee69700 1 -- start start 2023-11-24T07:57:07.371 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:07.361+0000 7fc3aee69700 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fc3a806f700 0x7fc3a815e3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:57:07.371 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:07.361+0000 7fc3aee69700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc3a80d1100 0x7fc3a813d7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:57:07.371 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:07.361+0000 7fc3aee69700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc3a8144610 0x7fc3a81398e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:57:07.371 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:07.361+0000 7fc3aee69700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3a815f020 con 0x7fc3a8144610 2023-11-24T07:57:07.371 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:07.361+0000 7fc3aee69700 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3a8132170 con 0x7fc3a80d1100 2023-11-24T07:57:07.371 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:07.361+0000 7fc3aee69700 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc3a8132450 con 0x7fc3a806f700 2023-11-24T07:57:07.372 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:07.362+0000 7fc3ad406700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc3a8144610 0x7fc3a81398e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:57:07.372 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:07.362+0000 7fc3ad406700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc3a8144610 0x7fc3a81398e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:51112/0 (socket says 172.21.15.59:51112) 2023-11-24T07:57:07.372 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:07.362+0000 7fc3ad406700 1 -- 172.21.15.59:0/2241986920 learned_addr learned my addr 172.21.15.59:0/2241986920 (peer_addr_for_me v2:172.21.15.59:0/0) 2023-11-24T07:57:07.372 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:07.362+0000 7fc3a7fff700 1 --2- 172.21.15.59:0/2241986920 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc3a80d1100 0x7fc3a813d7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:57:07.372 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:07.362+0000 7fc3acc05700 1 --2- 172.21.15.59:0/2241986920 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fc3a806f700 0x7fc3a815e3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:57:07.372 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:07.362+0000 7fc3ad406700 1 -- 172.21.15.59:0/2241986920 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fc3a806f700 msgr2=0x7fc3a815e3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:57:07.372 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:07.362+0000 7fc3ad406700 1 --2- 172.21.15.59:0/2241986920 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fc3a806f700 0x7fc3a815e3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:07.373 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:07.362+0000 7fc3ad406700 1 -- 172.21.15.59:0/2241986920 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc3a80d1100 msgr2=0x7fc3a813d7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:57:07.373 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:07.362+0000 7fc3ad406700 1 --2- 172.21.15.59:0/2241986920 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc3a80d1100 0x7fc3a813d7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:07.373 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:07.362+0000 7fc3ad406700 1 -- 172.21.15.59:0/2241986920 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc3a0009a70 con 0x7fc3a8144610 2023-11-24T07:57:07.373 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:07.362+0000 7fc3acc05700 1 --2- 172.21.15.59:0/2241986920 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fc3a806f700 0x7fc3a815e3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T07:57:07.373 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:07.362+0000 7fc3ad406700 1 --2- 172.21.15.59:0/2241986920 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc3a8144610 0x7fc3a81398e0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7fc3a000efc0 tx=0x7fc3a000c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:57:07.373 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:07.362+0000 7fc3a57fa700 1 -- 172.21.15.59:0/2241986920 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3a0009d70 con 0x7fc3a8144610 2023-11-24T07:57:07.373 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:07.362+0000 7fc3a57fa700 1 -- 172.21.15.59:0/2241986920 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc3a000f040 con 0x7fc3a8144610 2023-11-24T07:57:07.374 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:07.362+0000 7fc3aee69700 1 -- 172.21.15.59:0/2241986920 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc3940094a0 con 0x7fc3a8144610 2023-11-24T07:57:07.374 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:07.362+0000 7fc3a57fa700 1 -- 172.21.15.59:0/2241986920 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3a000ce00 con 0x7fc3a8144610 2023-11-24T07:57:07.374 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:07.362+0000 7fc3aee69700 1 -- 172.21.15.59:0/2241986920 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc3a8132a10 con 0x7fc3a8144610 2023-11-24T07:57:07.375 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:07.364+0000 7fc3aee69700 1 -- 172.21.15.59:0/2241986920 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc3a8075480 con 0x7fc3a8144610 2023-11-24T07:57:07.375 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:07.364+0000 7fc3a57fa700 1 -- 172.21.15.59:0/2241986920 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fc3a0014480 con 0x7fc3a8144610 2023-11-24T07:57:07.375 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:07.364+0000 7fc3a57fa700 1 --2- 172.21.15.59:0/2241986920 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7fc38c06aed0 0x7fc38c06d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:57:07.375 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:07.364+0000 7fc3a57fa700 1 -- 172.21.15.59:0/2241986920 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fc3a0089dd0 con 0x7fc3a8144610 2023-11-24T07:57:07.376 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:07.364+0000 7fc3acc05700 1 --2- 172.21.15.59:0/2241986920 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7fc38c06aed0 0x7fc38c06d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:57:07.376 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:07.365+0000 7fc3acc05700 1 --2- 172.21.15.59:0/2241986920 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7fc38c06aed0 0x7fc38c06d390 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fc39c0011e0 tx=0x7fc39c009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:57:07.376 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:07.369+0000 7fc3a57fa700 1 -- 172.21.15.59:0/2241986920 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fc3a0054e80 con 0x7fc3a8144610 2023-11-24T07:57:07.570 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:07.568+0000 7fc3aee69700 1 -- 172.21.15.59:0/2241986920 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fc3a806e2a0 con 0x7fc3a8144610 2023-11-24T07:57:07.570 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:07.569+0000 7fc3a57fa700 1 -- 172.21.15.59:0/2241986920 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v20) v1 ==== 100+0+11 (secure 0 0 0) 0x7fc3a00584a0 con 0x7fc3a8144610 2023-11-24T07:57:07.571 INFO:teuthology.orchestra.run.smithi059.stdout:pg_num: 64 2023-11-24T07:57:07.573 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:07.571+0000 7fc3aee69700 1 -- 172.21.15.59:0/2241986920 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7fc38c06aed0 msgr2=0x7fc38c06d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:57:07.573 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:07.571+0000 7fc3aee69700 1 --2- 172.21.15.59:0/2241986920 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7fc38c06aed0 0x7fc38c06d390 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fc39c0011e0 tx=0x7fc39c009040 comp rx=0 tx=0).stop 2023-11-24T07:57:07.573 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:07.571+0000 7fc3aee69700 1 -- 172.21.15.59:0/2241986920 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc3a8144610 msgr2=0x7fc3a81398e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:57:07.573 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:07.571+0000 7fc3aee69700 1 --2- 172.21.15.59:0/2241986920 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc3a8144610 0x7fc3a81398e0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7fc3a000efc0 tx=0x7fc3a000c3a0 comp rx=0 tx=0).stop 2023-11-24T07:57:07.574 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:07.571+0000 7fc3aee69700 1 -- 172.21.15.59:0/2241986920 shutdown_connections 2023-11-24T07:57:07.574 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:07.571+0000 7fc3aee69700 1 --2- 172.21.15.59:0/2241986920 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7fc38c06aed0 0x7fc38c06d390 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:07.574 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:07.571+0000 7fc3aee69700 1 --2- 172.21.15.59:0/2241986920 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc3a8144610 0x7fc3a81398e0 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:07.574 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:07.571+0000 7fc3aee69700 1 --2- 172.21.15.59:0/2241986920 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fc3a806f700 0x7fc3a815e3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:07.574 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:07.571+0000 7fc3aee69700 1 --2- 172.21.15.59:0/2241986920 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc3a80d1100 0x7fc3a813d7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:07.574 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:07.571+0000 7fc3aee69700 1 -- 172.21.15.59:0/2241986920 >> 172.21.15.59:0/2241986920 conn(0x7fc3a80b2740 msgr2=0x7fc3a80ce2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:57:07.574 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:07.571+0000 7fc3aee69700 1 -- 172.21.15.59:0/2241986920 shutdown_connections 2023-11-24T07:57:07.574 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:07.571+0000 7fc3aee69700 1 -- 172.21.15.59:0/2241986920 wait complete. 2023-11-24T07:57:07.585 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-11-24T07:57:07.711 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:07.709+0000 7f6d28bd4700 1 Processor -- start 2023-11-24T07:57:07.711 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:07.709+0000 7f6d28bd4700 1 -- start start 2023-11-24T07:57:07.713 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:07.710+0000 7f6d28bd4700 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f6d241440f0 0x7f6d240b51b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:57:07.713 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:07.710+0000 7f6d28bd4700 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6d24143ec0 con 0x7f6d240722c0 2023-11-24T07:57:07.713 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:07.710+0000 7f6d28bd4700 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6d240b3ae0 con 0x7f6d24143ae0 2023-11-24T07:57:07.713 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:07.710+0000 7f6d28bd4700 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6d240b40a0 con 0x7f6d241440f0 2023-11-24T07:57:07.713 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:07.710+0000 7f6d21d9b700 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7f6d240722c0 0x7f6d2406e510 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:54590/0 (socket says 172.21.15.59:54590) 2023-11-24T07:57:07.713 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:07.710+0000 7f6d21d9b700 1 -- 172.21.15.59:0/800010280 learned_addr learned my addr 172.21.15.59:0/800010280 (peer_addr_for_me v1:172.21.15.59:0/0) 2023-11-24T07:57:07.713 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:07.711+0000 7f6d22d9d700 1 --2- 172.21.15.59:0/800010280 >> v2:172.21.15.76:3300/0 conn(0x7f6d241440f0 0x7f6d240b51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:57:07.713 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:07.711+0000 7f6d2159a700 1 -- 172.21.15.59:0/800010280 <== mon.0 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2955467075 0 0) 0x7f6d24143ec0 con 0x7f6d240722c0 2023-11-24T07:57:07.714 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:07.711+0000 7f6d2159a700 1 -- 172.21.15.59:0/800010280 --> v1:172.21.15.59:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6d100035b0 con 0x7f6d240722c0 2023-11-24T07:57:07.714 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:07.711+0000 7f6d2159a700 1 -- 172.21.15.59:0/800010280 <== mon.0 v1:172.21.15.59:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 582145737 0 0) 0x7f6d100035b0 con 0x7f6d240722c0 2023-11-24T07:57:07.714 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:07.711+0000 7f6d22d9d700 1 -- 172.21.15.59:0/800010280 >> v1:172.21.15.76:6789/0 conn(0x7f6d24143ae0 legacy=0x7f6d24071bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:57:07.714 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:07.711+0000 7f6d22d9d700 1 -- 172.21.15.59:0/800010280 >> v1:172.21.15.59:6789/0 conn(0x7f6d240722c0 legacy=0x7f6d2406e510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:57:07.714 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:07.711+0000 7f6d22d9d700 1 -- 172.21.15.59:0/800010280 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6d240b3dc0 con 0x7f6d241440f0 2023-11-24T07:57:07.714 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:07.711+0000 7f6d22d9d700 1 --2- 172.21.15.59:0/800010280 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6d241440f0 0x7f6d240b51b0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f6d1c009d90 tx=0x7f6d1c00b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9b03d7a3c677abf5 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:57:07.714 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:07.711+0000 7f6d2159a700 1 -- 172.21.15.59:0/800010280 <== mon.1 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d1c00e070 con 0x7f6d241440f0 2023-11-24T07:57:07.714 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:07.711+0000 7f6d2159a700 1 -- 172.21.15.59:0/800010280 <== mon.1 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6d1c003a20 con 0x7f6d241440f0 2023-11-24T07:57:07.714 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:07.711+0000 7f6d2159a700 1 -- 172.21.15.59:0/800010280 <== mon.1 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d0c001670 con 0x7f6d241440f0 2023-11-24T07:57:07.717 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:07.712+0000 7f6d28bd4700 1 -- 172.21.15.59:0/800010280 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6d241440f0 msgr2=0x7f6d240b51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:57:07.717 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:07.712+0000 7f6d28bd4700 1 --2- 172.21.15.59:0/800010280 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6d241440f0 0x7f6d240b51b0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f6d1c009d90 tx=0x7f6d1c00b330 comp rx=0 tx=0).stop 2023-11-24T07:57:07.717 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:07.712+0000 7f6d28bd4700 1 -- 172.21.15.59:0/800010280 shutdown_connections 2023-11-24T07:57:07.717 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:07.712+0000 7f6d28bd4700 1 --2- 172.21.15.59:0/800010280 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6d241440f0 0x7f6d240b51b0 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:07.717 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:07.712+0000 7f6d28bd4700 1 -- 172.21.15.59:0/800010280 >> 172.21.15.59:0/800010280 conn(0x7f6d240b2740 msgr2=0x7f6d240b2b20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:57:07.718 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:07.712+0000 7f6d28bd4700 1 -- 172.21.15.59:0/800010280 shutdown_connections 2023-11-24T07:57:07.718 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:07.712+0000 7f6d28bd4700 1 -- 172.21.15.59:0/800010280 wait complete. 2023-11-24T07:57:07.718 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:07.713+0000 7f6d28bd4700 1 Processor -- start 2023-11-24T07:57:07.718 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:07.713+0000 7f6d28bd4700 1 -- start start 2023-11-24T07:57:07.718 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:07.714+0000 7f6d28bd4700 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f6d240722c0 0x7f6d241555f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:57:07.718 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:07.714+0000 7f6d28bd4700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6d24143ae0 0x7f6d241e6ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:57:07.718 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:07.714+0000 7f6d28bd4700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6d241440f0 0x7f6d241e7530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:57:07.718 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:07.714+0000 7f6d28bd4700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d241e7b40 con 0x7f6d241440f0 2023-11-24T07:57:07.718 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:07.714+0000 7f6d28bd4700 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d241e7e20 con 0x7f6d24143ae0 2023-11-24T07:57:07.718 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:07.714+0000 7f6d28bd4700 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6d241e8100 con 0x7f6d240722c0 2023-11-24T07:57:07.718 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:07.714+0000 7f6d22d9d700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6d241440f0 0x7f6d241e7530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:57:07.719 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:07.714+0000 7f6d22d9d700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6d241440f0 0x7f6d241e7530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:51126/0 (socket says 172.21.15.59:51126) 2023-11-24T07:57:07.719 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:07.714+0000 7f6d22d9d700 1 -- 172.21.15.59:0/4010486071 learned_addr learned my addr 172.21.15.59:0/4010486071 (peer_addr_for_me v2:172.21.15.59:0/0) 2023-11-24T07:57:07.719 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:07.714+0000 7f6d21d9b700 1 --2- 172.21.15.59:0/4010486071 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6d24143ae0 0x7f6d241e6ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:57:07.719 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:07.714+0000 7f6d2259c700 1 --2- 172.21.15.59:0/4010486071 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f6d240722c0 0x7f6d241555f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:57:07.719 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:07.715+0000 7f6d22d9d700 1 -- 172.21.15.59:0/4010486071 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f6d240722c0 msgr2=0x7f6d241555f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:57:07.719 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:07.715+0000 7f6d22d9d700 1 --2- 172.21.15.59:0/4010486071 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f6d240722c0 0x7f6d241555f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:07.719 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:07.715+0000 7f6d22d9d700 1 -- 172.21.15.59:0/4010486071 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6d24143ae0 msgr2=0x7f6d241e6ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:57:07.719 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:07.715+0000 7f6d22d9d700 1 --2- 172.21.15.59:0/4010486071 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6d24143ae0 0x7f6d241e6ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:07.719 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:07.715+0000 7f6d22d9d700 1 -- 172.21.15.59:0/4010486071 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6d1c009a70 con 0x7f6d241440f0 2023-11-24T07:57:07.719 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:07.715+0000 7f6d2259c700 1 --2- 172.21.15.59:0/4010486071 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f6d240722c0 0x7f6d241555f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T07:57:07.719 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:07.715+0000 7f6d21d9b700 1 --2- 172.21.15.59:0/4010486071 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6d24143ae0 0x7f6d241e6ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T07:57:07.720 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:07.715+0000 7f6d22d9d700 1 --2- 172.21.15.59:0/4010486071 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6d241440f0 0x7f6d241e7530 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f6d1c00b370 tx=0x7f6d1c004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:57:07.720 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:07.715+0000 7f6d1affd700 1 -- 172.21.15.59:0/4010486071 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d1c0046f0 con 0x7f6d241440f0 2023-11-24T07:57:07.720 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:07.715+0000 7f6d1affd700 1 -- 172.21.15.59:0/4010486071 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6d1c004850 con 0x7f6d241440f0 2023-11-24T07:57:07.720 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:07.715+0000 7f6d28bd4700 1 -- 172.21.15.59:0/4010486071 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6d241e83e0 con 0x7f6d241440f0 2023-11-24T07:57:07.720 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:07.715+0000 7f6d1affd700 1 -- 172.21.15.59:0/4010486071 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d1c012670 con 0x7f6d241440f0 2023-11-24T07:57:07.720 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:07.715+0000 7f6d28bd4700 1 -- 172.21.15.59:0/4010486071 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6d2413f730 con 0x7f6d241440f0 2023-11-24T07:57:07.721 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:07.716+0000 7f6d28bd4700 1 -- 172.21.15.59:0/4010486071 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6d240cdbb0 con 0x7f6d241440f0 2023-11-24T07:57:07.721 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:07.716+0000 7f6d1affd700 1 -- 172.21.15.59:0/4010486071 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f6d1c0127d0 con 0x7f6d241440f0 2023-11-24T07:57:07.721 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:07.717+0000 7f6d1affd700 1 --2- 172.21.15.59:0/4010486071 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f6d0406aed0 0x7f6d0406d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:57:07.722 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:07.717+0000 7f6d1affd700 1 -- 172.21.15.59:0/4010486071 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f6d1c08abc0 con 0x7f6d241440f0 2023-11-24T07:57:07.722 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:07.717+0000 7f6d2259c700 1 --2- 172.21.15.59:0/4010486071 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f6d0406aed0 0x7f6d0406d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:57:07.722 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:07.718+0000 7f6d2259c700 1 --2- 172.21.15.59:0/4010486071 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f6d0406aed0 0x7f6d0406d390 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f6d14000f60 tx=0x7f6d14009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:57:07.723 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:07.721+0000 7f6d1affd700 1 -- 172.21.15.59:0/4010486071 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f6d1c055d00 con 0x7f6d241440f0 2023-11-24T07:57:07.942 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:07.940+0000 7f6d28bd4700 1 -- 172.21.15.59:0/4010486071 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f6d241562b0 con 0x7f6d241440f0 2023-11-24T07:57:07.942 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:07.941+0000 7f6d1affd700 1 -- 172.21.15.59:0/4010486071 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v20) v1 ==== 80+0+0 (secure 0 0 0) 0x7f6d241562b0 con 0x7f6d241440f0 2023-11-24T07:57:07.944 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:07.942+0000 7f6d28bd4700 1 -- 172.21.15.59:0/4010486071 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f6d0406aed0 msgr2=0x7f6d0406d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:57:07.945 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:07.943+0000 7f6d28bd4700 1 --2- 172.21.15.59:0/4010486071 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f6d0406aed0 0x7f6d0406d390 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f6d14000f60 tx=0x7f6d14009040 comp rx=0 tx=0).stop 2023-11-24T07:57:07.945 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:07.943+0000 7f6d28bd4700 1 -- 172.21.15.59:0/4010486071 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6d241440f0 msgr2=0x7f6d241e7530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:57:07.945 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:07.943+0000 7f6d28bd4700 1 --2- 172.21.15.59:0/4010486071 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6d241440f0 0x7f6d241e7530 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f6d1c00b370 tx=0x7f6d1c004ad0 comp rx=0 tx=0).stop 2023-11-24T07:57:07.945 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:07.943+0000 7f6d28bd4700 1 -- 172.21.15.59:0/4010486071 shutdown_connections 2023-11-24T07:57:07.945 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:07.943+0000 7f6d28bd4700 1 --2- 172.21.15.59:0/4010486071 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f6d0406aed0 0x7f6d0406d390 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:07.945 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:07.943+0000 7f6d28bd4700 1 --2- 172.21.15.59:0/4010486071 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6d241440f0 0x7f6d241e7530 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:07.945 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:07.943+0000 7f6d28bd4700 1 --2- 172.21.15.59:0/4010486071 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f6d240722c0 0x7f6d241555f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:07.945 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:07.943+0000 7f6d28bd4700 1 --2- 172.21.15.59:0/4010486071 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6d24143ae0 0x7f6d241e6ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:07.945 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:07.943+0000 7f6d28bd4700 1 -- 172.21.15.59:0/4010486071 >> 172.21.15.59:0/4010486071 conn(0x7f6d240b2740 msgr2=0x7f6d24159df0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:57:07.945 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:07.943+0000 7f6d28bd4700 1 -- 172.21.15.59:0/4010486071 shutdown_connections 2023-11-24T07:57:07.945 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:07.943+0000 7f6d28bd4700 1 -- 172.21.15.59:0/4010486071 wait complete. 2023-11-24T07:57:07.946 INFO:teuthology.orchestra.run.smithi059.stderr:listed 0 entries 2023-11-24T07:57:07.956 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.2... 2023-11-24T07:57:07.957 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T07:57:07.957 DEBUG:teuthology.orchestra.run.smithi059:> sudo umount /home/ubuntu/cephtest/mnt.2 -f 2023-11-24T07:57:08.002 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi059.front.sepia.ceph.com 2023-11-24T07:57:08.002 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T07:57:08.002 DEBUG:teuthology.orchestra.run.smithi059:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.2) 2023-11-24T07:57:08.020 DEBUG:teuthology.orchestra.run.smithi076:> set -ex 2023-11-24T07:57:08.020 DEBUG:teuthology.orchestra.run.smithi076:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-24T07:57:08.040 DEBUG:teuthology.orchestra.run.smithi076:> set -ex 2023-11-24T07:57:08.040 DEBUG:teuthology.orchestra.run.smithi076:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-24T07:57:08.100 DEBUG:teuthology.orchestra.run.smithi076:> set -ex 2023-11-24T07:57:08.100 DEBUG:teuthology.orchestra.run.smithi076:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-24T07:57:08.160 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-24T07:57:08.286 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:08.284+0000 7fad450d9700 1 Processor -- start 2023-11-24T07:57:08.286 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:08.284+0000 7fad450d9700 1 -- start start 2023-11-24T07:57:08.288 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:08.285+0000 7fad450d9700 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fad401448b0 0x7fad4006ec70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:57:08.288 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:08.285+0000 7fad450d9700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fad4006f990 0x7fad400ce2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:57:08.288 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:08.285+0000 7fad450d9700 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fad40144c90 con 0x7fad400b44c0 2023-11-24T07:57:08.289 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:08.285+0000 7fad450d9700 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fad400b4c80 con 0x7fad4006f990 2023-11-24T07:57:08.289 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:08.285+0000 7fad450d9700 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fad400b4f60 con 0x7fad401448b0 2023-11-24T07:57:08.289 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:08.285+0000 7fad3f59e700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fad4006f990 0x7fad400ce2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:57:08.289 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:08.285+0000 7fad3f59e700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fad4006f990 0x7fad400ce2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:51132/0 (socket says 172.21.15.59:51132) 2023-11-24T07:57:08.289 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:08.285+0000 7fad3e59c700 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7fad400b44c0 0x7fad4006f280 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:54604/0 (socket says 172.21.15.59:54604) 2023-11-24T07:57:08.289 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:08.285+0000 7fad3f59e700 1 -- 172.21.15.59:0/2396362417 learned_addr learned my addr 172.21.15.59:0/2396362417 (peer_addr_for_me v2:172.21.15.59:0/0) 2023-11-24T07:57:08.289 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:08.286+0000 7fad3f59e700 1 -- 172.21.15.59:0/2396362417 >> v2:172.21.15.76:3300/0 conn(0x7fad401448b0 msgr2=0x7fad4006ec70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T07:57:08.289 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:08.286+0000 7fad3f59e700 1 --2- 172.21.15.59:0/2396362417 >> v2:172.21.15.76:3300/0 conn(0x7fad401448b0 0x7fad4006ec70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:08.289 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:08.286+0000 7fad3f59e700 1 -- 172.21.15.59:0/2396362417 >> v1:172.21.15.59:6789/0 conn(0x7fad400b44c0 legacy=0x7fad4006f280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:57:08.289 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:08.286+0000 7fad3f59e700 1 -- 172.21.15.59:0/2396362417 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fad400b5240 con 0x7fad4006f990 2023-11-24T07:57:08.290 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:08.286+0000 7fad3dd9b700 1 -- 172.21.15.59:0/2396362417 <== mon.0 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1188204071 0 0) 0x7fad40144c90 con 0x7fad400b44c0 2023-11-24T07:57:08.290 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:08.286+0000 7fad3f59e700 1 --2- 172.21.15.59:0/2396362417 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fad4006f990 0x7fad400ce2a0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7fad4006d4b0 tx=0x7fad3800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f7a610a010867e69 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:57:08.290 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:08.286+0000 7fad3dd9b700 1 -- 172.21.15.59:0/2396362417 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad3800e070 con 0x7fad4006f990 2023-11-24T07:57:08.290 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:08.286+0000 7fad3dd9b700 1 -- 172.21.15.59:0/2396362417 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fad3800b9c0 con 0x7fad4006f990 2023-11-24T07:57:08.290 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:08.286+0000 7fad3dd9b700 1 -- 172.21.15.59:0/2396362417 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad3800bd70 con 0x7fad4006f990 2023-11-24T07:57:08.290 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:08.286+0000 7fad450d9700 1 -- 172.21.15.59:0/2396362417 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fad4006f990 msgr2=0x7fad400ce2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:57:08.290 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:08.286+0000 7fad450d9700 1 --2- 172.21.15.59:0/2396362417 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fad4006f990 0x7fad400ce2a0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7fad4006d4b0 tx=0x7fad3800b330 comp rx=0 tx=0).stop 2023-11-24T07:57:08.290 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:08.286+0000 7fad450d9700 1 -- 172.21.15.59:0/2396362417 shutdown_connections 2023-11-24T07:57:08.291 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:08.286+0000 7fad450d9700 1 --2- 172.21.15.59:0/2396362417 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fad4006f990 0x7fad400ce2a0 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:08.291 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:08.286+0000 7fad450d9700 1 --2- 172.21.15.59:0/2396362417 >> v2:172.21.15.76:3300/0 conn(0x7fad401448b0 0x7fad4006ec70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:08.291 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:08.286+0000 7fad450d9700 1 -- 172.21.15.59:0/2396362417 >> 172.21.15.59:0/2396362417 conn(0x7fad400d0bc0 msgr2=0x7fad400d0fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:57:08.291 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:08.287+0000 7fad450d9700 1 -- 172.21.15.59:0/2396362417 shutdown_connections 2023-11-24T07:57:08.291 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:08.287+0000 7fad450d9700 1 -- 172.21.15.59:0/2396362417 wait complete. 2023-11-24T07:57:08.294 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:08.288+0000 7fad450d9700 1 Processor -- start 2023-11-24T07:57:08.294 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:08.288+0000 7fad450d9700 1 -- start start 2023-11-24T07:57:08.294 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:08.289+0000 7fad450d9700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fad4006f990 0x7fad401f4ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:57:08.294 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:08.289+0000 7fad450d9700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fad400b44c0 0x7fad400c63e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:57:08.294 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:08.289+0000 7fad450d9700 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fad401448b0 0x7fad4015d600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:57:08.294 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:08.289+0000 7fad450d9700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad400c6e10 con 0x7fad4006f990 2023-11-24T07:57:08.294 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:08.289+0000 7fad450d9700 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad400c70f0 con 0x7fad400b44c0 2023-11-24T07:57:08.294 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:08.289+0000 7fad450d9700 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fad400c73d0 con 0x7fad401448b0 2023-11-24T07:57:08.295 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:08.289+0000 7fad3ed9d700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fad4006f990 0x7fad401f4ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:57:08.295 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:08.289+0000 7fad3ed9d700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fad4006f990 0x7fad401f4ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:51134/0 (socket says 172.21.15.59:51134) 2023-11-24T07:57:08.295 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:08.289+0000 7fad3ed9d700 1 -- 172.21.15.59:0/1368592088 learned_addr learned my addr 172.21.15.59:0/1368592088 (peer_addr_for_me v2:172.21.15.59:0/0) 2023-11-24T07:57:08.295 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:08.290+0000 7fad3f59e700 1 --2- 172.21.15.59:0/1368592088 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fad401448b0 0x7fad4015d600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:57:08.295 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:08.290+0000 7fad3e59c700 1 --2- 172.21.15.59:0/1368592088 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fad400b44c0 0x7fad400c63e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:57:08.295 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:08.290+0000 7fad3ed9d700 1 -- 172.21.15.59:0/1368592088 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fad401448b0 msgr2=0x7fad4015d600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:57:08.295 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:08.290+0000 7fad3ed9d700 1 --2- 172.21.15.59:0/1368592088 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fad401448b0 0x7fad4015d600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:08.295 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:08.290+0000 7fad3ed9d700 1 -- 172.21.15.59:0/1368592088 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fad400b44c0 msgr2=0x7fad400c63e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:57:08.296 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:08.290+0000 7fad3ed9d700 1 --2- 172.21.15.59:0/1368592088 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fad400b44c0 0x7fad400c63e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:08.296 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:08.290+0000 7fad3ed9d700 1 -- 172.21.15.59:0/1368592088 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fad38009b30 con 0x7fad4006f990 2023-11-24T07:57:08.296 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:08.290+0000 7fad3e59c700 1 --2- 172.21.15.59:0/1368592088 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fad400b44c0 0x7fad400c63e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T07:57:08.296 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:08.290+0000 7fad3ed9d700 1 --2- 172.21.15.59:0/1368592088 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fad4006f990 0x7fad401f4ba0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7fad3400b4a0 tx=0x7fad3400b860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:57:08.296 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:08.290+0000 7fad277fe700 1 -- 172.21.15.59:0/1368592088 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad34011830 con 0x7fad4006f990 2023-11-24T07:57:08.296 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:08.290+0000 7fad277fe700 1 -- 172.21.15.59:0/1368592088 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fad34011990 con 0x7fad4006f990 2023-11-24T07:57:08.296 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:08.290+0000 7fad450d9700 1 -- 172.21.15.59:0/1368592088 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fad400c76b0 con 0x7fad4006f990 2023-11-24T07:57:08.296 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:08.290+0000 7fad277fe700 1 -- 172.21.15.59:0/1368592088 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad34011d40 con 0x7fad4006f990 2023-11-24T07:57:08.296 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:08.290+0000 7fad450d9700 1 -- 172.21.15.59:0/1368592088 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fad401f50e0 con 0x7fad4006f990 2023-11-24T07:57:08.296 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:08.291+0000 7fad450d9700 1 -- 172.21.15.59:0/1368592088 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fad40068670 con 0x7fad4006f990 2023-11-24T07:57:08.297 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:08.292+0000 7fad277fe700 1 -- 172.21.15.59:0/1368592088 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fad3401a400 con 0x7fad4006f990 2023-11-24T07:57:08.297 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:08.292+0000 7fad277fe700 1 --2- 172.21.15.59:0/1368592088 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7fad2c06aed0 0x7fad2c06d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:57:08.297 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:08.292+0000 7fad277fe700 1 -- 172.21.15.59:0/1368592088 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fad340895d0 con 0x7fad4006f990 2023-11-24T07:57:08.297 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:08.292+0000 7fad3e59c700 1 --2- 172.21.15.59:0/1368592088 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7fad2c06aed0 0x7fad2c06d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:57:08.297 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:08.293+0000 7fad3e59c700 1 --2- 172.21.15.59:0/1368592088 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7fad2c06aed0 0x7fad2c06d390 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fad30000f60 tx=0x7fad30009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:57:08.298 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:08.296+0000 7fad277fe700 1 -- 172.21.15.59:0/1368592088 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fad34054690 con 0x7fad4006f990 2023-11-24T07:57:08.502 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:08.500+0000 7fad450d9700 1 -- 172.21.15.59:0/1368592088 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fad400b1e10 con 0x7fad4006f990 2023-11-24T07:57:08.503 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:08.501+0000 7fad277fe700 1 -- 172.21.15.59:0/1368592088 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) v1 ==== 74+0+16437 (secure 0 0 0) 0x7fad34057cb0 con 0x7fad4006f990 2023-11-24T07:57:08.503 INFO:teuthology.orchestra.run.smithi059.stdout: 2023-11-24T07:57:08.504 INFO:teuthology.orchestra.run.smithi059.stdout:{"epoch":20,"fsid":"9f0e8741-b8f9-48d5-a2c3-09f45903303d","created":"2023-11-24T07:56:06.552729+0000","modified":"2023-11-24T07:56:25.705091+0000","last_up_change":"2023-11-24T07:56:17.372352+0000","last_in_change":"2023-11-24T07:56:09.073250+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-24T07:56:18.352823+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-11-24T07:56:22.185140+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-11-24T07:56:23.084326+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"6d8f9e9b-1782-4e41-928b-a39f2a8ebc06","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6800","nonce":3373878772},{"type":"v1","addr":"172.21.15.59:6801","nonce":3373878772}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6802","nonce":3373878772},{"type":"v1","addr":"172.21.15.59:6803","nonce":3373878772}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6806","nonce":3373878772},{"type":"v1","addr":"172.21.15.59:6807","nonce":3373878772}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6804","nonce":3373878772},{"type":"v1","addr":"172.21.15.59:6805","nonce":3373878772}]},"public_addr":"172.21.15.59:6801/3373878772","cluster_addr":"172.21.15.59:6803/3373878772","heartbeat_back_addr":"172.21.15.59:6807/3373878772","heartbeat_front_addr":"172.21.15.59:6805/3373878772","state":["exists","up"]},{"osd":1,"uuid":"e8d882aa-52e1-4c04-9e1e-c44ef9204a5f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6824","nonce":648258563},{"type":"v1","addr":"172.21.15.59:6825","nonce":648258563}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6826","nonce":648258563},{"type":"v1","addr":"172.21.15.59:6827","nonce":648258563}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6830","nonce":648258563},{"type":"v1","addr":"172.21.15.59:6831","nonce":648258563}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6828","nonce":648258563},{"type":"v1","addr":"172.21.15.59:6829","nonce":648258563}]},"public_addr":"172.21.15.59:6825/648258563","cluster_addr":"172.21.15.59:6827/648258563","heartbeat_back_addr":"172.21.15.59:6831/648258563","heartbeat_front_addr":"172.21.15.59:6829/648258563","state":["exists","up"]},{"osd":2,"uuid":"69eebfa0-695b-4b21-90b9-8a3537168a01","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6808","nonce":1840430654},{"type":"v1","addr":"172.21.15.59:6809","nonce":1840430654}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6810","nonce":1840430654},{"type":"v1","addr":"172.21.15.59:6811","nonce":1840430654}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6814","nonce":1840430654},{"type":"v1","addr":"172.21.15.59:6815","nonce":1840430654}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6812","nonce":1840430654},{"type":"v1","addr":"172.21.15.59:6813","nonce":1840430654}]},"public_addr":"172.21.15.59:6809/1840430654","cluster_addr":"172.21.15.59:6811/1840430654","heartbeat_back_addr":"172.21.15.59:6815/1840430654","heartbeat_front_addr":"172.21.15.59:6813/1840430654","state":["exists","up"]},{"osd":3,"uuid":"96fd77fc-f911-46ff-9445-6f0be2daa8f4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6816","nonce":2504025850},{"type":"v1","addr":"172.21.15.59:6817","nonce":2504025850}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6818","nonce":2504025850},{"type":"v1","addr":"172.21.15.59:6819","nonce":2504025850}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6822","nonce":2504025850},{"type":"v1","addr":"172.21.15.59:6823","nonce":2504025850}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6820","nonce":2504025850},{"type":"v1","addr":"172.21.15.59:6821","nonce":2504025850}]},"public_addr":"172.21.15.59:6817/2504025850","cluster_addr":"172.21.15.59:6819/2504025850","heartbeat_back_addr":"172.21.15.59:6823/2504025850","heartbeat_front_addr":"172.21.15.59:6821/2504025850","state":["exists","up"]},{"osd":4,"uuid":"7ac29fca-b5b2-4c95-9d69-2f164cbbc708","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6816","nonce":2382463431},{"type":"v1","addr":"172.21.15.76:6818","nonce":2382463431}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6820","nonce":2382463431},{"type":"v1","addr":"172.21.15.76:6822","nonce":2382463431}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6828","nonce":2382463431},{"type":"v1","addr":"172.21.15.76:6830","nonce":2382463431}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6824","nonce":2382463431},{"type":"v1","addr":"172.21.15.76:6826","nonce":2382463431}]},"public_addr":"172.21.15.76:6818/2382463431","cluster_addr":"172.21.15.76:6822/2382463431","heartbeat_back_addr":"172.21.15.76:6830/2382463431","heartbeat_front_addr":"172.21.15.76:6826/2382463431","state":["exists","up"]},{"osd":5,"uuid":"485bf711-3851-4238-a4f4-51d52e0a9c58","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6800","nonce":329334308},{"type":"v1","addr":"172.21.15.76:6802","nonce":329334308}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6805","nonce":329334308},{"type":"v1","addr":"172.21.15.76:6807","nonce":329334308}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6814","nonce":329334308},{"type":"v1","addr":"172.21.15.76:6815","nonce":329334308}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6810","nonce":329334308},{"type":"v1","addr":"172.21.15.76:6812","nonce":329334308}]},"public_addr":"172.21.15.76:6802/329334308","cluster_addr":"172.21.15.76:6807/329334308","heartbeat_back_addr":"172.21.15.76:6815/329334308","heartbeat_front_addr":"172.21.15.76:6812/329334308","state":["exists","up"]},{"osd":6,"uuid":"72b3f2fc-1e4a-4e32-968d-003647a0286f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6801","nonce":3946579410},{"type":"v1","addr":"172.21.15.76:6803","nonce":3946579410}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6804","nonce":3946579410},{"type":"v1","addr":"172.21.15.76:6806","nonce":3946579410}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6811","nonce":3946579410},{"type":"v1","addr":"172.21.15.76:6813","nonce":3946579410}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6808","nonce":3946579410},{"type":"v1","addr":"172.21.15.76:6809","nonce":3946579410}]},"public_addr":"172.21.15.76:6803/3946579410","cluster_addr":"172.21.15.76:6806/3946579410","heartbeat_back_addr":"172.21.15.76:6813/3946579410","heartbeat_front_addr":"172.21.15.76:6809/3946579410","state":["exists","up"]},{"osd":7,"uuid":"1ddceb90-6c5d-4d1a-ad56-99cb877f8c70","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6817","nonce":841761538},{"type":"v1","addr":"172.21.15.76:6819","nonce":841761538}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6821","nonce":841761538},{"type":"v1","addr":"172.21.15.76:6823","nonce":841761538}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6829","nonce":841761538},{"type":"v1","addr":"172.21.15.76:6831","nonce":841761538}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6825","nonce":841761538},{"type":"v1","addr":"172.21.15.76:6827","nonce":841761538}]},"public_addr":"172.21.15.76:6819/841761538","cluster_addr":"172.21.15.76:6823/841761538","heartbeat_back_addr":"172.21.15.76:6831/841761538","heartbeat_front_addr":"172.21.15.76:6827/841761538","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T07:56:14.294838+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T07:56:14.270779+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T07:56:14.231809+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T07:56:14.260983+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T07:56:14.302997+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T07:56:14.258600+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T07:56:14.251413+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T07:56:14.304727+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-24T07:57:08.506 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:08.504+0000 7fad450d9700 1 -- 172.21.15.59:0/1368592088 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7fad2c06aed0 msgr2=0x7fad2c06d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:57:08.506 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:08.504+0000 7fad450d9700 1 --2- 172.21.15.59:0/1368592088 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7fad2c06aed0 0x7fad2c06d390 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fad30000f60 tx=0x7fad30009040 comp rx=0 tx=0).stop 2023-11-24T07:57:08.506 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:08.504+0000 7fad450d9700 1 -- 172.21.15.59:0/1368592088 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fad4006f990 msgr2=0x7fad401f4ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:57:08.506 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:08.504+0000 7fad450d9700 1 --2- 172.21.15.59:0/1368592088 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fad4006f990 0x7fad401f4ba0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7fad3400b4a0 tx=0x7fad3400b860 comp rx=0 tx=0).stop 2023-11-24T07:57:08.506 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:08.504+0000 7fad450d9700 1 -- 172.21.15.59:0/1368592088 shutdown_connections 2023-11-24T07:57:08.506 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:08.504+0000 7fad450d9700 1 --2- 172.21.15.59:0/1368592088 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7fad2c06aed0 0x7fad2c06d390 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:08.506 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:08.504+0000 7fad450d9700 1 --2- 172.21.15.59:0/1368592088 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fad401448b0 0x7fad4015d600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:08.506 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:08.504+0000 7fad450d9700 1 --2- 172.21.15.59:0/1368592088 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fad4006f990 0x7fad401f4ba0 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:08.506 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:08.504+0000 7fad450d9700 1 --2- 172.21.15.59:0/1368592088 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fad400b44c0 0x7fad400c63e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:08.506 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:08.504+0000 7fad450d9700 1 -- 172.21.15.59:0/1368592088 >> 172.21.15.59:0/1368592088 conn(0x7fad400d0bc0 msgr2=0x7fad400d0fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:57:08.507 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:08.504+0000 7fad450d9700 1 -- 172.21.15.59:0/1368592088 shutdown_connections 2023-11-24T07:57:08.507 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:08.504+0000 7fad450d9700 1 -- 172.21.15.59:0/1368592088 wait complete. 2023-11-24T07:57:08.518 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-24T07:56:18.352823+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-24T07:56:22.185140+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-11-24T07:56:23.084326+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-24T07:57:08.518 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-24T07:57:08.647 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:08.645+0000 7f263e196700 1 Processor -- start 2023-11-24T07:57:08.647 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:08.645+0000 7f263e196700 1 -- start start 2023-11-24T07:57:08.654 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:08.645+0000 7f263e196700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f263815bf10 0x7f2638066ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:57:08.654 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:08.645+0000 7f263e196700 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f26380af980 0x7f26380b1550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:57:08.654 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:08.645+0000 7f263e196700 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f26380b10c0 con 0x7f26380b1a90 2023-11-24T07:57:08.654 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:08.645+0000 7f263e196700 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f26380af730 con 0x7f263815bf10 2023-11-24T07:57:08.655 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:08.645+0000 7f263e196700 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f26380b5ee0 con 0x7f26380af980 2023-11-24T07:57:08.655 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:08.646+0000 7f26377fe700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f263815bf10 0x7f2638066ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:57:08.655 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:08.646+0000 7f26377fe700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f263815bf10 0x7f2638066ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:51142/0 (socket says 172.21.15.59:51142) 2023-11-24T07:57:08.655 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:08.646+0000 7f26377fe700 1 -- 172.21.15.59:0/2021878671 learned_addr learned my addr 172.21.15.59:0/2021878671 (peer_addr_for_me v2:172.21.15.59:0/0) 2023-11-24T07:57:08.655 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:08.646+0000 7f2636ffd700 1 --2- 172.21.15.59:0/2021878671 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f26380af980 0x7f26380b1550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:57:08.655 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:08.646+0000 7f26377fe700 1 -- 172.21.15.59:0/2021878671 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f26380af980 msgr2=0x7f26380b1550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T07:57:08.655 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:08.646+0000 7f26377fe700 1 --2- 172.21.15.59:0/2021878671 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f26380af980 0x7f26380b1550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:08.656 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:08.646+0000 7f26377fe700 1 -- 172.21.15.59:0/2021878671 >> v1:172.21.15.76:6789/0 conn(0x7f26380b1a90 legacy=0x7f263806b7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T07:57:08.656 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:08.646+0000 7f26377fe700 1 -- 172.21.15.59:0/2021878671 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f26380b61c0 con 0x7f263815bf10 2023-11-24T07:57:08.656 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:08.646+0000 7f26377fe700 1 --2- 172.21.15.59:0/2021878671 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f263815bf10 0x7f2638066ce0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f26280098f0 tx=0x7f2628009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f12c03433c684588 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:57:08.656 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:08.646+0000 7f26367fc700 1 -- 172.21.15.59:0/2021878671 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2628004230 con 0x7f263815bf10 2023-11-24T07:57:08.656 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:08.646+0000 7f26367fc700 1 -- 172.21.15.59:0/2021878671 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2628004390 con 0x7f263815bf10 2023-11-24T07:57:08.656 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:08.646+0000 7f26367fc700 1 -- 172.21.15.59:0/2021878671 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26280036a0 con 0x7f263815bf10 2023-11-24T07:57:08.656 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:08.647+0000 7f263e196700 1 -- 172.21.15.59:0/2021878671 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f263815bf10 msgr2=0x7f2638066ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:57:08.656 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:08.647+0000 7f263e196700 1 --2- 172.21.15.59:0/2021878671 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f263815bf10 0x7f2638066ce0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f26280098f0 tx=0x7f2628009c00 comp rx=0 tx=0).stop 2023-11-24T07:57:08.656 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:08.647+0000 7f263e196700 1 -- 172.21.15.59:0/2021878671 shutdown_connections 2023-11-24T07:57:08.656 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:08.647+0000 7f263e196700 1 --2- 172.21.15.59:0/2021878671 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f263815bf10 0x7f2638066ce0 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:08.656 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:08.647+0000 7f263e196700 1 --2- 172.21.15.59:0/2021878671 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f26380af980 0x7f26380b1550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:08.656 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:08.647+0000 7f263e196700 1 -- 172.21.15.59:0/2021878671 >> 172.21.15.59:0/2021878671 conn(0x7f26380b5070 msgr2=0x7f26380b5730 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:57:08.657 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:08.647+0000 7f263e196700 1 -- 172.21.15.59:0/2021878671 shutdown_connections 2023-11-24T07:57:08.657 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:08.647+0000 7f263e196700 1 -- 172.21.15.59:0/2021878671 wait complete. 2023-11-24T07:57:08.657 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:08.648+0000 7f263e196700 1 Processor -- start 2023-11-24T07:57:08.657 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:08.648+0000 7f263e196700 1 -- start start 2023-11-24T07:57:08.657 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:08.649+0000 7f263e196700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f26380af980 0x7f26380aee80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:57:08.657 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:08.649+0000 7f263e196700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f263815bf10 0x7f26380adff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:57:08.657 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:08.649+0000 7f263e196700 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f26381ee8a0 0x7f26381fb570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:57:08.657 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:08.649+0000 7f263e196700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26380ae7e0 con 0x7f263815bf10 2023-11-24T07:57:08.657 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:08.649+0000 7f263e196700 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f263815ef90 con 0x7f26380af980 2023-11-24T07:57:08.657 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:08.649+0000 7f263e196700 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f26380ae530 con 0x7f26381ee8a0 2023-11-24T07:57:08.657 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:08.649+0000 7f2636ffd700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f263815bf10 0x7f26380adff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:57:08.657 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:08.649+0000 7f2636ffd700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f263815bf10 0x7f26380adff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:51150/0 (socket says 172.21.15.59:51150) 2023-11-24T07:57:08.657 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:08.649+0000 7f2636ffd700 1 -- 172.21.15.59:0/2804915919 learned_addr learned my addr 172.21.15.59:0/2804915919 (peer_addr_for_me v2:172.21.15.59:0/0) 2023-11-24T07:57:08.658 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:08.649+0000 7f2637fff700 1 --2- 172.21.15.59:0/2804915919 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f26381ee8a0 0x7f26381fb570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:57:08.658 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:08.649+0000 7f26377fe700 1 --2- 172.21.15.59:0/2804915919 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f26380af980 0x7f26380aee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:57:08.658 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:08.649+0000 7f2636ffd700 1 -- 172.21.15.59:0/2804915919 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f26381ee8a0 msgr2=0x7f26381fb570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:57:08.658 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:08.649+0000 7f2636ffd700 1 --2- 172.21.15.59:0/2804915919 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f26381ee8a0 0x7f26381fb570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:08.658 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:08.649+0000 7f2636ffd700 1 -- 172.21.15.59:0/2804915919 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f26380af980 msgr2=0x7f26380aee80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:57:08.658 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:08.649+0000 7f2636ffd700 1 --2- 172.21.15.59:0/2804915919 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f26380af980 0x7f26380aee80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:08.658 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:08.649+0000 7f2636ffd700 1 -- 172.21.15.59:0/2804915919 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2628009550 con 0x7f263815bf10 2023-11-24T07:57:08.658 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:08.650+0000 7f2636ffd700 1 --2- 172.21.15.59:0/2804915919 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f263815bf10 0x7f26380adff0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f262c00c7f0 tx=0x7f262c00cb00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:57:08.658 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:08.650+0000 7f261ffff700 1 -- 172.21.15.59:0/2804915919 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f262c004220 con 0x7f263815bf10 2023-11-24T07:57:08.658 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:08.650+0000 7f261ffff700 1 -- 172.21.15.59:0/2804915919 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f262c004380 con 0x7f263815bf10 2023-11-24T07:57:08.658 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:08.650+0000 7f263e196700 1 -- 172.21.15.59:0/2804915919 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f263815e1d0 con 0x7f263815bf10 2023-11-24T07:57:08.659 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:08.650+0000 7f261ffff700 1 -- 172.21.15.59:0/2804915919 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f262c0036a0 con 0x7f263815bf10 2023-11-24T07:57:08.659 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:08.650+0000 7f263e196700 1 -- 172.21.15.59:0/2804915919 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f263815e790 con 0x7f263815bf10 2023-11-24T07:57:08.659 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:08.651+0000 7f263e196700 1 -- 172.21.15.59:0/2804915919 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f263815f490 con 0x7f263815bf10 2023-11-24T07:57:08.659 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:08.651+0000 7f261ffff700 1 -- 172.21.15.59:0/2804915919 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f262c003800 con 0x7f263815bf10 2023-11-24T07:57:08.659 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:08.652+0000 7f261ffff700 1 --2- 172.21.15.59:0/2804915919 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f262406aed0 0x7f262406d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:57:08.659 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:08.652+0000 7f261ffff700 1 -- 172.21.15.59:0/2804915919 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f262c089020 con 0x7f263815bf10 2023-11-24T07:57:08.659 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:08.652+0000 7f26377fe700 1 --2- 172.21.15.59:0/2804915919 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f262406aed0 0x7f262406d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:57:08.659 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:08.652+0000 7f26377fe700 1 --2- 172.21.15.59:0/2804915919 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f262406aed0 0x7f262406d390 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f2628009130 tx=0x7f2628008030 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:57:08.660 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:08.656+0000 7f261ffff700 1 -- 172.21.15.59:0/2804915919 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f262c054160 con 0x7f263815bf10 2023-11-24T07:57:08.860 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:08.858+0000 7f263e196700 1 -- 172.21.15.59:0/2804915919 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f2638075480 con 0x7f263815bf10 2023-11-24T07:57:08.860 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:08.859+0000 7f261ffff700 1 -- 172.21.15.59:0/2804915919 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v20) v1 ==== 93+0+10 (secure 0 0 0) 0x7f262c057780 con 0x7f263815bf10 2023-11-24T07:57:08.861 INFO:teuthology.orchestra.run.smithi059.stdout:pg_num: 1 2023-11-24T07:57:08.863 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:08.861+0000 7f263e196700 1 -- 172.21.15.59:0/2804915919 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f262406aed0 msgr2=0x7f262406d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:57:08.863 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:08.861+0000 7f263e196700 1 --2- 172.21.15.59:0/2804915919 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f262406aed0 0x7f262406d390 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f2628009130 tx=0x7f2628008030 comp rx=0 tx=0).stop 2023-11-24T07:57:08.863 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:08.861+0000 7f263e196700 1 -- 172.21.15.59:0/2804915919 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f263815bf10 msgr2=0x7f26380adff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:57:08.863 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:08.861+0000 7f263e196700 1 --2- 172.21.15.59:0/2804915919 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f263815bf10 0x7f26380adff0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f262c00c7f0 tx=0x7f262c00cb00 comp rx=0 tx=0).stop 2023-11-24T07:57:08.863 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:08.861+0000 7f263e196700 1 -- 172.21.15.59:0/2804915919 shutdown_connections 2023-11-24T07:57:08.864 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:08.861+0000 7f263e196700 1 --2- 172.21.15.59:0/2804915919 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f262406aed0 0x7f262406d390 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:08.864 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:08.861+0000 7f263e196700 1 --2- 172.21.15.59:0/2804915919 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f26381ee8a0 0x7f26381fb570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:08.864 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:08.861+0000 7f263e196700 1 --2- 172.21.15.59:0/2804915919 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f26380af980 0x7f26380aee80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:08.864 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:08.861+0000 7f263e196700 1 --2- 172.21.15.59:0/2804915919 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f263815bf10 0x7f26380adff0 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:08.864 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:08.861+0000 7f263e196700 1 -- 172.21.15.59:0/2804915919 >> 172.21.15.59:0/2804915919 conn(0x7f26380b5070 msgr2=0x7f263815ad30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:57:08.864 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:08.861+0000 7f263e196700 1 -- 172.21.15.59:0/2804915919 shutdown_connections 2023-11-24T07:57:08.864 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:08.861+0000 7f263e196700 1 -- 172.21.15.59:0/2804915919 wait complete. 2023-11-24T07:57:08.875 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-24T07:57:09.003 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:09.001+0000 7fbd220ff700 1 Processor -- start 2023-11-24T07:57:09.003 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:09.001+0000 7fbd220ff700 1 -- start start 2023-11-24T07:57:09.011 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:09.002+0000 7fbd220ff700 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fbd1c06f680 0x7fbd1c0b5940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:57:09.011 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:09.002+0000 7fbd220ff700 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fbd1c1472f0 0x7fbd1c06afd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:57:09.011 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:09.002+0000 7fbd220ff700 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbd1c147120 con 0x7fbd1c06b510 2023-11-24T07:57:09.012 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:09.002+0000 7fbd220ff700 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbd1c06b9c0 con 0x7fbd1c06f680 2023-11-24T07:57:09.012 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:09.002+0000 7fbd220ff700 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbd1c071b30 con 0x7fbd1c1472f0 2023-11-24T07:57:09.012 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:09.002+0000 7fbd1affd700 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fbd1c1472f0 0x7fbd1c06afd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:57:09.012 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:09.002+0000 7fbd1b7fe700 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fbd1c06f680 0x7fbd1c0b5940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:57:09.012 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:09.002+0000 7fbd1affd700 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fbd1c1472f0 0x7fbd1c06afd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3301/0 says I am v2:172.21.15.59:50598/0 (socket says 172.21.15.59:50598) 2023-11-24T07:57:09.012 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:09.002+0000 7fbd1affd700 1 -- 172.21.15.59:0/907491177 learned_addr learned my addr 172.21.15.59:0/907491177 (peer_addr_for_me v2:172.21.15.59:0/0) 2023-11-24T07:57:09.012 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:09.002+0000 7fbd1affd700 1 -- 172.21.15.59:0/907491177 >> v2:172.21.15.76:3300/0 conn(0x7fbd1c06f680 msgr2=0x7fbd1c0b5940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T07:57:09.012 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:09.002+0000 7fbd1affd700 1 --2- 172.21.15.59:0/907491177 >> v2:172.21.15.76:3300/0 conn(0x7fbd1c06f680 0x7fbd1c0b5940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:09.012 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:09.002+0000 7fbd1affd700 1 -- 172.21.15.59:0/907491177 >> v1:172.21.15.76:6789/0 conn(0x7fbd1c06b510 legacy=0x7fbd1c161c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:57:09.012 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:09.002+0000 7fbd1affd700 1 -- 172.21.15.59:0/907491177 --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbd1c071e10 con 0x7fbd1c1472f0 2023-11-24T07:57:09.012 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:09.003+0000 7fbd1affd700 1 --2- 172.21.15.59:0/907491177 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fbd1c1472f0 0x7fbd1c06afd0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fbd040098f0 tx=0x7fbd04009c00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6028a02d15b6fee3 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:57:09.013 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:09.003+0000 7fbd1a7fc700 1 -- 172.21.15.59:0/907491177 <== mon.2 v2:172.21.15.76:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd04011830 con 0x7fbd1c1472f0 2023-11-24T07:57:09.013 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:09.003+0000 7fbd1a7fc700 1 -- 172.21.15.59:0/907491177 <== mon.2 v2:172.21.15.76:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbd04011990 con 0x7fbd1c1472f0 2023-11-24T07:57:09.013 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:09.003+0000 7fbd1a7fc700 1 -- 172.21.15.59:0/907491177 <== mon.2 v2:172.21.15.76:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd04011d20 con 0x7fbd1c1472f0 2023-11-24T07:57:09.013 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:09.003+0000 7fbd220ff700 1 -- 172.21.15.59:0/907491177 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fbd1c1472f0 msgr2=0x7fbd1c06afd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:57:09.013 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:09.003+0000 7fbd220ff700 1 --2- 172.21.15.59:0/907491177 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fbd1c1472f0 0x7fbd1c06afd0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fbd040098f0 tx=0x7fbd04009c00 comp rx=0 tx=0).stop 2023-11-24T07:57:09.013 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:09.003+0000 7fbd220ff700 1 -- 172.21.15.59:0/907491177 shutdown_connections 2023-11-24T07:57:09.013 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:09.003+0000 7fbd220ff700 1 --2- 172.21.15.59:0/907491177 >> v2:172.21.15.76:3300/0 conn(0x7fbd1c06f680 0x7fbd1c0b5940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:09.013 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:09.003+0000 7fbd220ff700 1 --2- 172.21.15.59:0/907491177 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fbd1c1472f0 0x7fbd1c06afd0 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:09.013 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:09.003+0000 7fbd220ff700 1 -- 172.21.15.59:0/907491177 >> 172.21.15.59:0/907491177 conn(0x7fbd1c0b27b0 msgr2=0x7fbd1c0b2b90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:57:09.013 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:09.004+0000 7fbd220ff700 1 -- 172.21.15.59:0/907491177 shutdown_connections 2023-11-24T07:57:09.013 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:09.004+0000 7fbd220ff700 1 -- 172.21.15.59:0/907491177 wait complete. 2023-11-24T07:57:09.013 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:09.005+0000 7fbd220ff700 1 Processor -- start 2023-11-24T07:57:09.014 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:09.005+0000 7fbd220ff700 1 -- start start 2023-11-24T07:57:09.014 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:09.006+0000 7fbd220ff700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fbd1c06b510 0x7fbd1c1f3140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:57:09.014 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:09.006+0000 7fbd220ff700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbd1c06f680 0x7fbd1c0af6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:57:09.014 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:09.006+0000 7fbd220ff700 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fbd1c1472f0 0x7fbd1c0afc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:57:09.014 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:09.006+0000 7fbd220ff700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd1c136ef0 con 0x7fbd1c06f680 2023-11-24T07:57:09.014 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:09.006+0000 7fbd220ff700 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd1c15e400 con 0x7fbd1c06b510 2023-11-24T07:57:09.014 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:09.006+0000 7fbd220ff700 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbd1c134da0 con 0x7fbd1c1472f0 2023-11-24T07:57:09.014 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:09.006+0000 7fbd1affd700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbd1c06f680 0x7fbd1c0af6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:57:09.014 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:09.006+0000 7fbd1affd700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbd1c06f680 0x7fbd1c0af6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:51164/0 (socket says 172.21.15.59:51164) 2023-11-24T07:57:09.015 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:09.006+0000 7fbd1affd700 1 -- 172.21.15.59:0/1974437195 learned_addr learned my addr 172.21.15.59:0/1974437195 (peer_addr_for_me v2:172.21.15.59:0/0) 2023-11-24T07:57:09.015 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:09.006+0000 7fbd1b7fe700 1 --2- 172.21.15.59:0/1974437195 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fbd1c06b510 0x7fbd1c1f3140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:57:09.015 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:09.006+0000 7fbd1bfff700 1 --2- 172.21.15.59:0/1974437195 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fbd1c1472f0 0x7fbd1c0afc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:57:09.015 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:09.006+0000 7fbd1affd700 1 -- 172.21.15.59:0/1974437195 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fbd1c1472f0 msgr2=0x7fbd1c0afc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:57:09.015 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:09.006+0000 7fbd1affd700 1 --2- 172.21.15.59:0/1974437195 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fbd1c1472f0 0x7fbd1c0afc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:09.015 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:09.006+0000 7fbd1affd700 1 -- 172.21.15.59:0/1974437195 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fbd1c06b510 msgr2=0x7fbd1c1f3140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:57:09.015 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:09.006+0000 7fbd1affd700 1 --2- 172.21.15.59:0/1974437195 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fbd1c06b510 0x7fbd1c1f3140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:09.015 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:09.006+0000 7fbd1affd700 1 -- 172.21.15.59:0/1974437195 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbd04009550 con 0x7fbd1c06f680 2023-11-24T07:57:09.015 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:09.006+0000 7fbd1affd700 1 --2- 172.21.15.59:0/1974437195 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbd1c06f680 0x7fbd1c0af6d0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7fbd04014040 tx=0x7fbd04004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:57:09.016 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:09.006+0000 7fbd13fff700 1 -- 172.21.15.59:0/1974437195 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd04004400 con 0x7fbd1c06f680 2023-11-24T07:57:09.016 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:09.006+0000 7fbd13fff700 1 -- 172.21.15.59:0/1974437195 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbd0401f070 con 0x7fbd1c06f680 2023-11-24T07:57:09.016 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:09.006+0000 7fbd220ff700 1 -- 172.21.15.59:0/1974437195 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbd1c1365c0 con 0x7fbd1c06f680 2023-11-24T07:57:09.016 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:09.006+0000 7fbd13fff700 1 -- 172.21.15.59:0/1974437195 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd040038e0 con 0x7fbd1c06f680 2023-11-24T07:57:09.016 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:09.007+0000 7fbd220ff700 1 -- 172.21.15.59:0/1974437195 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbd1c136b80 con 0x7fbd1c06f680 2023-11-24T07:57:09.016 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:09.008+0000 7fbd220ff700 1 -- 172.21.15.59:0/1974437195 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbd1c075480 con 0x7fbd1c06f680 2023-11-24T07:57:09.016 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:09.008+0000 7fbd13fff700 1 -- 172.21.15.59:0/1974437195 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fbd04021030 con 0x7fbd1c06f680 2023-11-24T07:57:09.017 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:09.008+0000 7fbd13fff700 1 --2- 172.21.15.59:0/1974437195 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7fbcfc06aed0 0x7fbcfc06d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:57:09.017 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:09.008+0000 7fbd13fff700 1 -- 172.21.15.59:0/1974437195 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fbd0408a8b0 con 0x7fbd1c06f680 2023-11-24T07:57:09.017 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:09.008+0000 7fbd1b7fe700 1 --2- 172.21.15.59:0/1974437195 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7fbcfc06aed0 0x7fbcfc06d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:57:09.017 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:09.009+0000 7fbd1b7fe700 1 --2- 172.21.15.59:0/1974437195 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7fbcfc06aed0 0x7fbcfc06d390 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fbd0c009be0 tx=0x7fbd0c009210 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:57:09.017 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:09.013+0000 7fbd13fff700 1 -- 172.21.15.59:0/1974437195 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fbd040559f0 con 0x7fbd1c06f680 2023-11-24T07:57:09.212 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:09.211+0000 7fbd220ff700 1 -- 172.21.15.59:0/1974437195 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fbd1c06afd0 con 0x7fbd1c06f680 2023-11-24T07:57:09.213 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:09.211+0000 7fbd13fff700 1 -- 172.21.15.59:0/1974437195 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v20) v1 ==== 104+0+11 (secure 0 0 0) 0x7fbd04015020 con 0x7fbd1c06f680 2023-11-24T07:57:09.213 INFO:teuthology.orchestra.run.smithi059.stdout:pg_num: 32 2023-11-24T07:57:09.215 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:09.213+0000 7fbd220ff700 1 -- 172.21.15.59:0/1974437195 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7fbcfc06aed0 msgr2=0x7fbcfc06d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:57:09.215 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:09.213+0000 7fbd220ff700 1 --2- 172.21.15.59:0/1974437195 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7fbcfc06aed0 0x7fbcfc06d390 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fbd0c009be0 tx=0x7fbd0c009210 comp rx=0 tx=0).stop 2023-11-24T07:57:09.215 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:09.213+0000 7fbd220ff700 1 -- 172.21.15.59:0/1974437195 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbd1c06f680 msgr2=0x7fbd1c0af6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:57:09.215 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:09.213+0000 7fbd220ff700 1 --2- 172.21.15.59:0/1974437195 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbd1c06f680 0x7fbd1c0af6d0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7fbd04014040 tx=0x7fbd04004750 comp rx=0 tx=0).stop 2023-11-24T07:57:09.216 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:09.213+0000 7fbd220ff700 1 -- 172.21.15.59:0/1974437195 shutdown_connections 2023-11-24T07:57:09.216 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:09.213+0000 7fbd220ff700 1 --2- 172.21.15.59:0/1974437195 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7fbcfc06aed0 0x7fbcfc06d390 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:09.216 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:09.213+0000 7fbd220ff700 1 --2- 172.21.15.59:0/1974437195 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fbd1c1472f0 0x7fbd1c0afc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:09.216 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:09.213+0000 7fbd220ff700 1 --2- 172.21.15.59:0/1974437195 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fbd1c06b510 0x7fbd1c1f3140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:09.216 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:09.213+0000 7fbd220ff700 1 --2- 172.21.15.59:0/1974437195 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbd1c06f680 0x7fbd1c0af6d0 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:09.216 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:09.213+0000 7fbd220ff700 1 -- 172.21.15.59:0/1974437195 >> 172.21.15.59:0/1974437195 conn(0x7fbd1c0b27b0 msgr2=0x7fbd1c1465c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:57:09.216 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:09.213+0000 7fbd220ff700 1 -- 172.21.15.59:0/1974437195 shutdown_connections 2023-11-24T07:57:09.216 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:09.213+0000 7fbd220ff700 1 -- 172.21.15.59:0/1974437195 wait complete. 2023-11-24T07:57:09.227 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-24T07:57:09.355 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:09.354+0000 7f4d83a95700 1 Processor -- start 2023-11-24T07:57:09.355 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:09.354+0000 7f4d83a95700 1 -- start start 2023-11-24T07:57:09.358 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:09.355+0000 7f4d83a95700 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f4d7c0d1100 0x7f4d7c15e7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:57:09.359 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:09.355+0000 7f4d83a95700 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4d7c06e9d0 con 0x7f4d7c144610 2023-11-24T07:57:09.359 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:09.355+0000 7f4d83a95700 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4d7c06f270 con 0x7f4d7c06f700 2023-11-24T07:57:09.359 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:09.355+0000 7f4d83a95700 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4d7c1449f0 con 0x7f4d7c0d1100 2023-11-24T07:57:09.359 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:09.355+0000 7f4d81831700 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7f4d7c144610 0x7f4d7c0d09f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:54644/0 (socket says 172.21.15.59:54644) 2023-11-24T07:57:09.359 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:09.355+0000 7f4d81831700 1 -- 172.21.15.59:0/3168370679 learned_addr learned my addr 172.21.15.59:0/3168370679 (peer_addr_for_me v1:172.21.15.59:0/0) 2023-11-24T07:57:09.360 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:09.355+0000 7f4d81030700 1 --2- 172.21.15.59:0/3168370679 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f4d7c0d1100 0x7f4d7c15e7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:57:09.360 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:09.355+0000 7f4d8082f700 1 -- 172.21.15.59:0/3168370679 <== mon.0 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 886426478 0 0) 0x7f4d7c06e9d0 con 0x7f4d7c144610 2023-11-24T07:57:09.360 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:09.355+0000 7f4d8082f700 1 -- 172.21.15.59:0/3168370679 --> v1:172.21.15.59:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4d680035b0 con 0x7f4d7c144610 2023-11-24T07:57:09.360 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:09.355+0000 7f4d8082f700 1 -- 172.21.15.59:0/3168370679 <== mon.1 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1713507052 0 0) 0x7f4d7c06f270 con 0x7f4d7c06f700 2023-11-24T07:57:09.360 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:09.355+0000 7f4d8082f700 1 -- 172.21.15.59:0/3168370679 --> v1:172.21.15.76:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4d7c06e9d0 con 0x7f4d7c06f700 2023-11-24T07:57:09.360 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:09.355+0000 7f4d8082f700 1 -- 172.21.15.59:0/3168370679 <== mon.0 v1:172.21.15.59:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3868026251 0 0) 0x7f4d680035b0 con 0x7f4d7c144610 2023-11-24T07:57:09.360 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:09.355+0000 7f4d81030700 1 -- 172.21.15.59:0/3168370679 >> v1:172.21.15.76:6789/0 conn(0x7f4d7c06f700 legacy=0x7f4d7c15ed00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:57:09.361 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:09.355+0000 7f4d81030700 1 -- 172.21.15.59:0/3168370679 >> v1:172.21.15.59:6789/0 conn(0x7f4d7c144610 legacy=0x7f4d7c0d09f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:57:09.361 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:09.355+0000 7f4d81030700 1 -- 172.21.15.59:0/3168370679 --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4d7c15f530 con 0x7f4d7c0d1100 2023-11-24T07:57:09.361 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:09.355+0000 7f4d81030700 1 --2- 172.21.15.59:0/3168370679 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f4d7c0d1100 0x7f4d7c15e7c0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f4d78009870 tx=0x7f4d78009b80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9873d43dd7197bd8 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:57:09.361 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:09.356+0000 7f4d8082f700 1 -- 172.21.15.59:0/3168370679 <== mon.2 v2:172.21.15.76:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d6c001670 con 0x7f4d7c0d1100 2023-11-24T07:57:09.361 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:09.356+0000 7f4d8082f700 1 -- 172.21.15.59:0/3168370679 <== mon.2 v2:172.21.15.76:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4d78004030 con 0x7f4d7c0d1100 2023-11-24T07:57:09.361 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:09.356+0000 7f4d8082f700 1 -- 172.21.15.59:0/3168370679 <== mon.2 v2:172.21.15.76:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d780043c0 con 0x7f4d7c0d1100 2023-11-24T07:57:09.362 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:09.356+0000 7f4d83a95700 1 -- 172.21.15.59:0/3168370679 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f4d7c0d1100 msgr2=0x7f4d7c15e7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:57:09.362 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:09.356+0000 7f4d83a95700 1 --2- 172.21.15.59:0/3168370679 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f4d7c0d1100 0x7f4d7c15e7c0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f4d78009870 tx=0x7f4d78009b80 comp rx=0 tx=0).stop 2023-11-24T07:57:09.362 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:09.356+0000 7f4d83a95700 1 -- 172.21.15.59:0/3168370679 shutdown_connections 2023-11-24T07:57:09.362 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:09.356+0000 7f4d83a95700 1 --2- 172.21.15.59:0/3168370679 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f4d7c0d1100 0x7f4d7c15e7c0 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:09.362 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:09.356+0000 7f4d83a95700 1 -- 172.21.15.59:0/3168370679 >> 172.21.15.59:0/3168370679 conn(0x7f4d7c0b2740 msgr2=0x7f4d7c0b2b50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:57:09.362 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:09.356+0000 7f4d83a95700 1 -- 172.21.15.59:0/3168370679 shutdown_connections 2023-11-24T07:57:09.363 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:09.356+0000 7f4d83a95700 1 -- 172.21.15.59:0/3168370679 wait complete. 2023-11-24T07:57:09.367 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:09.357+0000 7f4d83a95700 1 Processor -- start 2023-11-24T07:57:09.367 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:09.358+0000 7f4d83a95700 1 -- start start 2023-11-24T07:57:09.368 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:09.358+0000 7f4d83a95700 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f4d7c06f700 0x7f4d7c15c2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:57:09.368 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:09.358+0000 7f4d83a95700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4d7c0d1100 0x7f4d7c157d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:57:09.368 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:09.358+0000 7f4d83a95700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4d7c144610 0x7f4d7c1582a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:57:09.368 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:09.358+0000 7f4d83a95700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d7c15bb40 con 0x7f4d7c144610 2023-11-24T07:57:09.368 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:09.358+0000 7f4d83a95700 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d7c15db30 con 0x7f4d7c0d1100 2023-11-24T07:57:09.368 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:09.358+0000 7f4d83a95700 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4d7c15de10 con 0x7f4d7c06f700 2023-11-24T07:57:09.369 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:09.358+0000 7f4d82032700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4d7c144610 0x7f4d7c1582a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:57:09.369 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:09.358+0000 7f4d82032700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4d7c144610 0x7f4d7c1582a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:51180/0 (socket says 172.21.15.59:51180) 2023-11-24T07:57:09.369 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:09.358+0000 7f4d82032700 1 -- 172.21.15.59:0/917904637 learned_addr learned my addr 172.21.15.59:0/917904637 (peer_addr_for_me v2:172.21.15.59:0/0) 2023-11-24T07:57:09.369 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:09.359+0000 7f4d81831700 1 --2- 172.21.15.59:0/917904637 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f4d7c06f700 0x7f4d7c15c2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:57:09.369 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:09.359+0000 7f4d81030700 1 --2- 172.21.15.59:0/917904637 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4d7c0d1100 0x7f4d7c157d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:57:09.369 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:09.359+0000 7f4d82032700 1 -- 172.21.15.59:0/917904637 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f4d7c06f700 msgr2=0x7f4d7c15c2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:57:09.370 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:09.359+0000 7f4d82032700 1 --2- 172.21.15.59:0/917904637 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f4d7c06f700 0x7f4d7c15c2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:09.370 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:09.359+0000 7f4d82032700 1 -- 172.21.15.59:0/917904637 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4d7c0d1100 msgr2=0x7f4d7c157d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:57:09.370 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:09.359+0000 7f4d82032700 1 --2- 172.21.15.59:0/917904637 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4d7c0d1100 0x7f4d7c157d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:09.370 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:09.359+0000 7f4d82032700 1 -- 172.21.15.59:0/917904637 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4d780094a0 con 0x7f4d7c144610 2023-11-24T07:57:09.370 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:09.359+0000 7f4d81831700 1 --2- 172.21.15.59:0/917904637 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f4d7c06f700 0x7f4d7c15c2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-24T07:57:09.370 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:09.359+0000 7f4d82032700 1 --2- 172.21.15.59:0/917904637 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4d7c144610 0x7f4d7c1582a0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f4d7400ee60 tx=0x7f4d7400c550 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:57:09.370 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:09.359+0000 7f4d727fc700 1 -- 172.21.15.59:0/917904637 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d74003c00 con 0x7f4d7c144610 2023-11-24T07:57:09.371 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:09.359+0000 7f4d727fc700 1 -- 172.21.15.59:0/917904637 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4d74003d60 con 0x7f4d7c144610 2023-11-24T07:57:09.371 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:09.359+0000 7f4d83a95700 1 -- 172.21.15.59:0/917904637 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4d7c15e0f0 con 0x7f4d7c144610 2023-11-24T07:57:09.371 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:09.359+0000 7f4d727fc700 1 -- 172.21.15.59:0/917904637 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d74005310 con 0x7f4d7c144610 2023-11-24T07:57:09.371 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:09.359+0000 7f4d83a95700 1 -- 172.21.15.59:0/917904637 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4d7c15c7f0 con 0x7f4d7c144610 2023-11-24T07:57:09.371 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:09.360+0000 7f4d83a95700 1 -- 172.21.15.59:0/917904637 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4d7c075480 con 0x7f4d7c144610 2023-11-24T07:57:09.371 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:09.361+0000 7f4d727fc700 1 -- 172.21.15.59:0/917904637 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f4d74005470 con 0x7f4d7c144610 2023-11-24T07:57:09.371 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:09.361+0000 7f4d727fc700 1 --2- 172.21.15.59:0/917904637 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f4d6006aed0 0x7f4d6006d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:57:09.371 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:09.361+0000 7f4d727fc700 1 -- 172.21.15.59:0/917904637 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f4d7408bda0 con 0x7f4d7c144610 2023-11-24T07:57:09.372 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:09.361+0000 7f4d81831700 1 --2- 172.21.15.59:0/917904637 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f4d6006aed0 0x7f4d6006d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:57:09.372 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:09.362+0000 7f4d81831700 1 --2- 172.21.15.59:0/917904637 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f4d6006aed0 0x7f4d6006d390 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f4d6c001860 tx=0x7f4d6c0016d0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:57:09.372 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:09.366+0000 7f4d727fc700 1 -- 172.21.15.59:0/917904637 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f4d74056ee0 con 0x7f4d7c144610 2023-11-24T07:57:09.569 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:09.567+0000 7f4d83a95700 1 -- 172.21.15.59:0/917904637 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f4d7c06e2a0 con 0x7f4d7c144610 2023-11-24T07:57:09.569 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:09.568+0000 7f4d727fc700 1 -- 172.21.15.59:0/917904637 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v20) v1 ==== 100+0+11 (secure 0 0 0) 0x7f4d7401f070 con 0x7f4d7c144610 2023-11-24T07:57:09.570 INFO:teuthology.orchestra.run.smithi059.stdout:pg_num: 64 2023-11-24T07:57:09.572 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:09.570+0000 7f4d83a95700 1 -- 172.21.15.59:0/917904637 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f4d6006aed0 msgr2=0x7f4d6006d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:57:09.572 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:09.570+0000 7f4d83a95700 1 --2- 172.21.15.59:0/917904637 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f4d6006aed0 0x7f4d6006d390 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f4d6c001860 tx=0x7f4d6c0016d0 comp rx=0 tx=0).stop 2023-11-24T07:57:09.572 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:09.570+0000 7f4d83a95700 1 -- 172.21.15.59:0/917904637 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4d7c144610 msgr2=0x7f4d7c1582a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:57:09.573 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:09.570+0000 7f4d83a95700 1 --2- 172.21.15.59:0/917904637 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4d7c144610 0x7f4d7c1582a0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f4d7400ee60 tx=0x7f4d7400c550 comp rx=0 tx=0).stop 2023-11-24T07:57:09.573 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:09.570+0000 7f4d83a95700 1 -- 172.21.15.59:0/917904637 shutdown_connections 2023-11-24T07:57:09.573 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:09.570+0000 7f4d83a95700 1 --2- 172.21.15.59:0/917904637 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f4d6006aed0 0x7f4d6006d390 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:09.573 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:09.570+0000 7f4d83a95700 1 --2- 172.21.15.59:0/917904637 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4d7c144610 0x7f4d7c1582a0 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:09.573 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:09.570+0000 7f4d83a95700 1 --2- 172.21.15.59:0/917904637 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f4d7c06f700 0x7f4d7c15c2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:09.574 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:09.570+0000 7f4d83a95700 1 --2- 172.21.15.59:0/917904637 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4d7c0d1100 0x7f4d7c157d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:09.574 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:09.570+0000 7f4d83a95700 1 -- 172.21.15.59:0/917904637 >> 172.21.15.59:0/917904637 conn(0x7f4d7c0b2740 msgr2=0x7f4d7c13f590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:57:09.574 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:09.570+0000 7f4d83a95700 1 -- 172.21.15.59:0/917904637 shutdown_connections 2023-11-24T07:57:09.574 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:09.570+0000 7f4d83a95700 1 -- 172.21.15.59:0/917904637 wait complete. 2023-11-24T07:57:09.584 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-11-24T07:57:09.710 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:09.708+0000 7f11cbbc1700 1 Processor -- start 2023-11-24T07:57:09.710 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:09.708+0000 7f11cbbc1700 1 -- start start 2023-11-24T07:57:09.712 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:09.709+0000 7f11cbbc1700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f11c4143ae0 0x7f11c40b51b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:57:09.712 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:09.709+0000 7f11cbbc1700 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f11c41440f0 0x7f11c4071bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:57:09.712 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:09.709+0000 7f11cbbc1700 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f11c40720f0 0x7f11c415e8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:57:09.713 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:09.709+0000 7f11cbbc1700 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f11c40b40a0 con 0x7f11c4143ae0 2023-11-24T07:57:09.713 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:09.709+0000 7f11cbbc1700 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f11c4072580 con 0x7f11c40720f0 2023-11-24T07:57:09.713 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:09.709+0000 7f11cbbc1700 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f11c4143ec0 con 0x7f11c41440f0 2023-11-24T07:57:09.713 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:09.709+0000 7f11c995d700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f11c4143ae0 0x7f11c40b51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:57:09.713 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:09.709+0000 7f11c995d700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f11c4143ae0 0x7f11c40b51b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:51184/0 (socket says 172.21.15.59:51184) 2023-11-24T07:57:09.713 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:09.709+0000 7f11c995d700 1 -- 172.21.15.59:0/3553785925 learned_addr learned my addr 172.21.15.59:0/3553785925 (peer_addr_for_me v2:172.21.15.59:0/0) 2023-11-24T07:57:09.713 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:09.709+0000 7f11ca15e700 1 --2- 172.21.15.59:0/3553785925 >> v2:172.21.15.76:3300/0 conn(0x7f11c40720f0 0x7f11c415e8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:57:09.713 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:09.709+0000 7f11c915c700 1 --2- 172.21.15.59:0/3553785925 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f11c41440f0 0x7f11c4071bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:57:09.713 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:09.709+0000 7f11c995d700 1 -- 172.21.15.59:0/3553785925 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f11c41440f0 msgr2=0x7f11c4071bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T07:57:09.713 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:09.709+0000 7f11c995d700 1 --2- 172.21.15.59:0/3553785925 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f11c41440f0 0x7f11c4071bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:09.713 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:09.709+0000 7f11c995d700 1 -- 172.21.15.59:0/3553785925 >> v2:172.21.15.76:3300/0 conn(0x7f11c40720f0 msgr2=0x7f11c415e8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T07:57:09.714 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:09.709+0000 7f11c995d700 1 --2- 172.21.15.59:0/3553785925 >> v2:172.21.15.76:3300/0 conn(0x7f11c40720f0 0x7f11c415e8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:09.714 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:09.709+0000 7f11c995d700 1 -- 172.21.15.59:0/3553785925 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f11c40b3ae0 con 0x7f11c4143ae0 2023-11-24T07:57:09.714 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:09.710+0000 7f11c995d700 1 --2- 172.21.15.59:0/3553785925 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f11c4143ae0 0x7f11c40b51b0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f11c40706c0 tx=0x7f11c0009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=caed4f7d184951c6 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:57:09.714 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:09.710+0000 7f11c895b700 1 -- 172.21.15.59:0/3553785925 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11c000b870 con 0x7f11c4143ae0 2023-11-24T07:57:09.714 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:09.710+0000 7f11c895b700 1 -- 172.21.15.59:0/3553785925 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f11c000b9d0 con 0x7f11c4143ae0 2023-11-24T07:57:09.714 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:09.710+0000 7f11c895b700 1 -- 172.21.15.59:0/3553785925 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11c000bd60 con 0x7f11c4143ae0 2023-11-24T07:57:09.714 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:09.710+0000 7f11cbbc1700 1 -- 172.21.15.59:0/3553785925 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f11c4143ae0 msgr2=0x7f11c40b51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:57:09.714 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:09.710+0000 7f11cbbc1700 1 --2- 172.21.15.59:0/3553785925 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f11c4143ae0 0x7f11c40b51b0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f11c40706c0 tx=0x7f11c0009bd0 comp rx=0 tx=0).stop 2023-11-24T07:57:09.714 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:09.710+0000 7f11cbbc1700 1 -- 172.21.15.59:0/3553785925 shutdown_connections 2023-11-24T07:57:09.714 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:09.710+0000 7f11cbbc1700 1 --2- 172.21.15.59:0/3553785925 >> v2:172.21.15.76:3300/0 conn(0x7f11c40720f0 0x7f11c415e8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:09.714 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:09.710+0000 7f11cbbc1700 1 --2- 172.21.15.59:0/3553785925 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f11c4143ae0 0x7f11c40b51b0 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:09.714 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:09.710+0000 7f11cbbc1700 1 --2- 172.21.15.59:0/3553785925 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f11c41440f0 0x7f11c4071bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:09.715 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:09.710+0000 7f11cbbc1700 1 -- 172.21.15.59:0/3553785925 >> 172.21.15.59:0/3553785925 conn(0x7f11c40b2740 msgr2=0x7f11c40b2b20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:57:09.715 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:09.711+0000 7f11cbbc1700 1 -- 172.21.15.59:0/3553785925 shutdown_connections 2023-11-24T07:57:09.715 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:09.711+0000 7f11cbbc1700 1 -- 172.21.15.59:0/3553785925 wait complete. 2023-11-24T07:57:09.717 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:09.712+0000 7f11cbbc1700 1 Processor -- start 2023-11-24T07:57:09.717 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:09.712+0000 7f11cbbc1700 1 -- start start 2023-11-24T07:57:09.717 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:09.712+0000 7f11cbbc1700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f11c40720f0 0x7f11c4141ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:57:09.717 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:09.712+0000 7f11cbbc1700 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f11c4143ae0 0x7f11c41efa80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:57:09.718 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:09.712+0000 7f11cbbc1700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f11c41440f0 0x7f11c41422f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:57:09.718 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:09.712+0000 7f11cbbc1700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11c415d670 con 0x7f11c40720f0 2023-11-24T07:57:09.718 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:09.712+0000 7f11cbbc1700 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11c415d1a0 con 0x7f11c41440f0 2023-11-24T07:57:09.718 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:09.712+0000 7f11cbbc1700 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f11c415cbe0 con 0x7f11c4143ae0 2023-11-24T07:57:09.718 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:09.713+0000 7f11c995d700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f11c40720f0 0x7f11c4141ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:57:09.718 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:09.713+0000 7f11c995d700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f11c40720f0 0x7f11c4141ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:51190/0 (socket says 172.21.15.59:51190) 2023-11-24T07:57:09.718 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:09.713+0000 7f11c995d700 1 -- 172.21.15.59:0/1775875803 learned_addr learned my addr 172.21.15.59:0/1775875803 (peer_addr_for_me v2:172.21.15.59:0/0) 2023-11-24T07:57:09.718 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:09.713+0000 7f11ca15e700 1 --2- 172.21.15.59:0/1775875803 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f11c41440f0 0x7f11c41422f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:57:09.718 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:09.713+0000 7f11c915c700 1 --2- 172.21.15.59:0/1775875803 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f11c4143ae0 0x7f11c41efa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:57:09.718 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:09.713+0000 7f11c995d700 1 -- 172.21.15.59:0/1775875803 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f11c4143ae0 msgr2=0x7f11c41efa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:57:09.718 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:09.713+0000 7f11c995d700 1 --2- 172.21.15.59:0/1775875803 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f11c4143ae0 0x7f11c41efa80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:09.718 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:09.713+0000 7f11c995d700 1 -- 172.21.15.59:0/1775875803 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f11c41440f0 msgr2=0x7f11c41422f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:57:09.719 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:09.713+0000 7f11c995d700 1 --2- 172.21.15.59:0/1775875803 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f11c41440f0 0x7f11c41422f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:09.719 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:09.713+0000 7f11c995d700 1 -- 172.21.15.59:0/1775875803 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f11c0009550 con 0x7f11c40720f0 2023-11-24T07:57:09.719 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:09.713+0000 7f11c995d700 1 --2- 172.21.15.59:0/1775875803 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f11c40720f0 0x7f11c4141ce0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f11c0000c00 tx=0x7f11c0004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:57:09.719 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:09.713+0000 7f11ba7fc700 1 -- 172.21.15.59:0/1775875803 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11c00129c0 con 0x7f11c40720f0 2023-11-24T07:57:09.719 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:09.713+0000 7f11ba7fc700 1 -- 172.21.15.59:0/1775875803 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f11c001e070 con 0x7f11c40720f0 2023-11-24T07:57:09.719 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:09.713+0000 7f11ba7fc700 1 -- 172.21.15.59:0/1775875803 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11c0012d40 con 0x7f11c40720f0 2023-11-24T07:57:09.719 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:09.713+0000 7f11cbbc1700 1 -- 172.21.15.59:0/1775875803 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f11c415cec0 con 0x7f11c40720f0 2023-11-24T07:57:09.719 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:09.713+0000 7f11cbbc1700 1 -- 172.21.15.59:0/1775875803 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f11c41558d0 con 0x7f11c40720f0 2023-11-24T07:57:09.719 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:09.714+0000 7f11cbbc1700 1 -- 172.21.15.59:0/1775875803 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f11c415d8e0 con 0x7f11c40720f0 2023-11-24T07:57:09.719 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:09.715+0000 7f11ba7fc700 1 -- 172.21.15.59:0/1775875803 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f11c002d450 con 0x7f11c40720f0 2023-11-24T07:57:09.719 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:09.715+0000 7f11ba7fc700 1 --2- 172.21.15.59:0/1775875803 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f11a806aed0 0x7f11a806d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:57:09.719 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:09.715+0000 7f11ba7fc700 1 -- 172.21.15.59:0/1775875803 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f11c008a6a0 con 0x7f11c40720f0 2023-11-24T07:57:09.719 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:09.715+0000 7f11c915c700 1 --2- 172.21.15.59:0/1775875803 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f11a806aed0 0x7f11a806d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:57:09.720 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:09.716+0000 7f11c915c700 1 --2- 172.21.15.59:0/1775875803 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f11a806aed0 0x7f11a806d390 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f11b40098b0 tx=0x7f11b4008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:57:09.721 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:09.719+0000 7f11ba7fc700 1 -- 172.21.15.59:0/1775875803 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f11c00557e0 con 0x7f11c40720f0 2023-11-24T07:57:09.935 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:09.933+0000 7f11cbbc1700 1 -- 172.21.15.59:0/1775875803 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f11c4068670 con 0x7f11c40720f0 2023-11-24T07:57:09.935 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:09.934+0000 7f11ba7fc700 1 -- 172.21.15.59:0/1775875803 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v20) v1 ==== 80+0+0 (secure 0 0 0) 0x7f11c0058e00 con 0x7f11c40720f0 2023-11-24T07:57:09.938 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:09.936+0000 7f11cbbc1700 1 -- 172.21.15.59:0/1775875803 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f11a806aed0 msgr2=0x7f11a806d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:57:09.938 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:09.936+0000 7f11cbbc1700 1 --2- 172.21.15.59:0/1775875803 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f11a806aed0 0x7f11a806d390 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f11b40098b0 tx=0x7f11b4008040 comp rx=0 tx=0).stop 2023-11-24T07:57:09.938 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:09.936+0000 7f11cbbc1700 1 -- 172.21.15.59:0/1775875803 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f11c40720f0 msgr2=0x7f11c4141ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:57:09.938 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:09.936+0000 7f11cbbc1700 1 --2- 172.21.15.59:0/1775875803 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f11c40720f0 0x7f11c4141ce0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f11c0000c00 tx=0x7f11c0004750 comp rx=0 tx=0).stop 2023-11-24T07:57:09.938 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:09.936+0000 7f11cbbc1700 1 -- 172.21.15.59:0/1775875803 shutdown_connections 2023-11-24T07:57:09.938 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:09.936+0000 7f11cbbc1700 1 --2- 172.21.15.59:0/1775875803 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f11a806aed0 0x7f11a806d390 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:09.938 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:09.936+0000 7f11cbbc1700 1 --2- 172.21.15.59:0/1775875803 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f11c41440f0 0x7f11c41422f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:09.938 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:09.936+0000 7f11cbbc1700 1 --2- 172.21.15.59:0/1775875803 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f11c40720f0 0x7f11c4141ce0 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:09.938 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:09.936+0000 7f11cbbc1700 1 --2- 172.21.15.59:0/1775875803 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f11c4143ae0 0x7f11c41efa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:09.939 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:09.936+0000 7f11cbbc1700 1 -- 172.21.15.59:0/1775875803 >> 172.21.15.59:0/1775875803 conn(0x7f11c40b2740 msgr2=0x7f11c40ce620 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:57:09.939 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:09.936+0000 7f11cbbc1700 1 -- 172.21.15.59:0/1775875803 shutdown_connections 2023-11-24T07:57:09.939 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:09.936+0000 7f11cbbc1700 1 -- 172.21.15.59:0/1775875803 wait complete. 2023-11-24T07:57:09.939 INFO:teuthology.orchestra.run.smithi059.stderr:listed 0 entries 2023-11-24T07:57:09.950 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.3... 2023-11-24T07:57:09.950 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T07:57:09.950 DEBUG:teuthology.orchestra.run.smithi076:> sudo umount /home/ubuntu/cephtest/mnt.3 -f 2023-11-24T07:57:09.990 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi076.front.sepia.ceph.com 2023-11-24T07:57:09.991 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T07:57:09.991 DEBUG:teuthology.orchestra.run.smithi076:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.3) 2023-11-24T07:57:10.050 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2023-11-24T07:57:10.050 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-24T07:57:10.175 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:10.173+0000 7f52d162c700 1 Processor -- start 2023-11-24T07:57:10.175 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:10.174+0000 7f52d162c700 1 -- start start 2023-11-24T07:57:10.178 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:10.174+0000 7f52d162c700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f52cc15e460 0x7f52cc0d49d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:57:10.178 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:10.174+0000 7f52d162c700 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f52cc0d2ba0 0x7f52cc0b40a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:57:10.178 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:10.174+0000 7f52d162c700 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f52cc0b3370 con 0x7f52cc0cf480 2023-11-24T07:57:10.178 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:10.174+0000 7f52d162c700 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f52cc0cfc40 con 0x7f52cc15e460 2023-11-24T07:57:10.179 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:10.174+0000 7f52d162c700 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f52cc0b35c0 con 0x7f52cc0d2ba0 2023-11-24T07:57:10.179 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:10.174+0000 7f52caffd700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f52cc15e460 0x7f52cc0d49d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:57:10.179 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:10.175+0000 7f52caffd700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f52cc15e460 0x7f52cc0d49d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:51198/0 (socket says 172.21.15.59:51198) 2023-11-24T07:57:10.179 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:10.175+0000 7f52caffd700 1 -- 172.21.15.59:0/3129693438 learned_addr learned my addr 172.21.15.59:0/3129693438 (peer_addr_for_me v2:172.21.15.59:0/0) 2023-11-24T07:57:10.179 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:10.175+0000 7f52cb7fe700 1 --2- 172.21.15.59:0/3129693438 >> v2:172.21.15.76:3300/0 conn(0x7f52cc0d2ba0 0x7f52cc0b40a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:57:10.179 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:10.175+0000 7f52caffd700 1 -- 172.21.15.59:0/3129693438 >> v2:172.21.15.76:3300/0 conn(0x7f52cc0d2ba0 msgr2=0x7f52cc0b40a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T07:57:10.180 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:10.175+0000 7f52caffd700 1 --2- 172.21.15.59:0/3129693438 >> v2:172.21.15.76:3300/0 conn(0x7f52cc0d2ba0 0x7f52cc0b40a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:10.180 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:10.175+0000 7f52caffd700 1 -- 172.21.15.59:0/3129693438 >> v1:172.21.15.76:6789/0 conn(0x7f52cc0cf480 legacy=0x7f52cc0d2490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:57:10.180 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:10.175+0000 7f52caffd700 1 -- 172.21.15.59:0/3129693438 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f52cc0b38a0 con 0x7f52cc15e460 2023-11-24T07:57:10.180 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:10.175+0000 7f52caffd700 1 --2- 172.21.15.59:0/3129693438 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f52cc15e460 0x7f52cc0d49d0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f52cc08d240 tx=0x7f52bc009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=42625ae504f2bfe3 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:57:10.181 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:10.175+0000 7f52c9ffb700 1 -- 172.21.15.59:0/3129693438 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52bc00b870 con 0x7f52cc15e460 2023-11-24T07:57:10.181 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:10.175+0000 7f52c9ffb700 1 -- 172.21.15.59:0/3129693438 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f52bc00b9d0 con 0x7f52cc15e460 2023-11-24T07:57:10.181 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:10.175+0000 7f52c9ffb700 1 -- 172.21.15.59:0/3129693438 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52bc00bd60 con 0x7f52cc15e460 2023-11-24T07:57:10.181 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:10.176+0000 7f52d162c700 1 -- 172.21.15.59:0/3129693438 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f52cc15e460 msgr2=0x7f52cc0d49d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:57:10.182 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:10.176+0000 7f52d162c700 1 --2- 172.21.15.59:0/3129693438 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f52cc15e460 0x7f52cc0d49d0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f52cc08d240 tx=0x7f52bc009bd0 comp rx=0 tx=0).stop 2023-11-24T07:57:10.182 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:10.176+0000 7f52d162c700 1 -- 172.21.15.59:0/3129693438 shutdown_connections 2023-11-24T07:57:10.182 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:10.176+0000 7f52d162c700 1 --2- 172.21.15.59:0/3129693438 >> v2:172.21.15.76:3300/0 conn(0x7f52cc0d2ba0 0x7f52cc0b40a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:10.182 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:10.176+0000 7f52d162c700 1 --2- 172.21.15.59:0/3129693438 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f52cc15e460 0x7f52cc0d49d0 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:10.182 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:10.176+0000 7f52d162c700 1 -- 172.21.15.59:0/3129693438 >> 172.21.15.59:0/3129693438 conn(0x7f52cc0d3eb0 msgr2=0x7f52cc0d42c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:57:10.182 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:10.176+0000 7f52d162c700 1 -- 172.21.15.59:0/3129693438 shutdown_connections 2023-11-24T07:57:10.183 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:10.176+0000 7f52d162c700 1 -- 172.21.15.59:0/3129693438 wait complete. 2023-11-24T07:57:10.185 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:10.177+0000 7f52d162c700 1 Processor -- start 2023-11-24T07:57:10.185 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:10.177+0000 7f52d162c700 1 -- start start 2023-11-24T07:57:10.185 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:10.178+0000 7f52d162c700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f52cc0cf480 0x7f52cc15e840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:57:10.185 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:10.178+0000 7f52d162c700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f52cc0d2ba0 0x7f52cc0cdc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:57:10.185 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:10.178+0000 7f52d162c700 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f52cc15e460 0x7f52cc0b1cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:57:10.186 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:10.178+0000 7f52d162c700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52cc0ce160 con 0x7f52cc0cf480 2023-11-24T07:57:10.186 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:10.178+0000 7f52d162c700 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52cc15ee20 con 0x7f52cc0d2ba0 2023-11-24T07:57:10.186 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:10.178+0000 7f52d162c700 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f52cc15f100 con 0x7f52cc15e460 2023-11-24T07:57:10.186 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:10.178+0000 7f52caffd700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f52cc0cf480 0x7f52cc15e840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:57:10.186 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:10.178+0000 7f52caffd700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f52cc0cf480 0x7f52cc15e840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:51206/0 (socket says 172.21.15.59:51206) 2023-11-24T07:57:10.186 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:10.178+0000 7f52caffd700 1 -- 172.21.15.59:0/2175823437 learned_addr learned my addr 172.21.15.59:0/2175823437 (peer_addr_for_me v2:172.21.15.59:0/0) 2023-11-24T07:57:10.186 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:10.178+0000 7f52ca7fc700 1 --2- 172.21.15.59:0/2175823437 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f52cc0d2ba0 0x7f52cc0cdc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:57:10.186 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:10.178+0000 7f52cb7fe700 1 --2- 172.21.15.59:0/2175823437 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f52cc15e460 0x7f52cc0b1cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:57:10.186 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:10.178+0000 7f52caffd700 1 -- 172.21.15.59:0/2175823437 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f52cc15e460 msgr2=0x7f52cc0b1cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:57:10.186 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:10.178+0000 7f52caffd700 1 --2- 172.21.15.59:0/2175823437 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f52cc15e460 0x7f52cc0b1cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:10.186 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:10.178+0000 7f52caffd700 1 -- 172.21.15.59:0/2175823437 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f52cc0d2ba0 msgr2=0x7f52cc0cdc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:57:10.187 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:10.178+0000 7f52caffd700 1 --2- 172.21.15.59:0/2175823437 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f52cc0d2ba0 0x7f52cc0cdc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:10.187 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:10.178+0000 7f52caffd700 1 -- 172.21.15.59:0/2175823437 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f52bc009550 con 0x7f52cc0cf480 2023-11-24T07:57:10.187 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:10.179+0000 7f52caffd700 1 --2- 172.21.15.59:0/2175823437 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f52cc0cf480 0x7f52cc15e840 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f52bc000c00 tx=0x7f52bc004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:57:10.187 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:10.179+0000 7f52b37fe700 1 -- 172.21.15.59:0/2175823437 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52bc004520 con 0x7f52cc0cf480 2023-11-24T07:57:10.187 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:10.179+0000 7f52b37fe700 1 -- 172.21.15.59:0/2175823437 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f52bc01e070 con 0x7f52cc0cf480 2023-11-24T07:57:10.187 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:10.179+0000 7f52d162c700 1 -- 172.21.15.59:0/2175823437 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f52cc15f3e0 con 0x7f52cc0cf480 2023-11-24T07:57:10.187 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:10.179+0000 7f52b37fe700 1 -- 172.21.15.59:0/2175823437 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52bc0038a0 con 0x7f52cc0cf480 2023-11-24T07:57:10.187 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:10.179+0000 7f52d162c700 1 -- 172.21.15.59:0/2175823437 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f52cc15f9a0 con 0x7f52cc0cf480 2023-11-24T07:57:10.187 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:10.180+0000 7f52d162c700 1 -- 172.21.15.59:0/2175823437 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f52cc15fd10 con 0x7f52cc0cf480 2023-11-24T07:57:10.187 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:10.180+0000 7f52b37fe700 1 -- 172.21.15.59:0/2175823437 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f52bc003a00 con 0x7f52cc0cf480 2023-11-24T07:57:10.187 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:10.180+0000 7f52b37fe700 1 --2- 172.21.15.59:0/2175823437 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f52b806aed0 0x7f52b806d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:57:10.187 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:10.181+0000 7f52b37fe700 1 -- 172.21.15.59:0/2175823437 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f52bc020070 con 0x7f52cc0cf480 2023-11-24T07:57:10.188 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:10.181+0000 7f52ca7fc700 1 --2- 172.21.15.59:0/2175823437 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f52b806aed0 0x7f52b806d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:57:10.188 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:10.181+0000 7f52ca7fc700 1 --2- 172.21.15.59:0/2175823437 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f52b806aed0 0x7f52b806d390 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f52cc067300 tx=0x7f52c0009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:57:10.188 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:10.185+0000 7f52b37fe700 1 -- 172.21.15.59:0/2175823437 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f52bc055c70 con 0x7f52cc0cf480 2023-11-24T07:57:10.433 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:10.430+0000 7f52d162c700 1 -- 172.21.15.59:0/2175823437 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f52cc067f60 con 0x7f52cc0cf480 2023-11-24T07:57:10.514 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:10.431+0000 7f52b37fe700 1 -- 172.21.15.59:0/2175823437 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) v1 ==== 93+0+4904 (secure 0 0 0) 0x7f52bc003cb0 con 0x7f52cc0cf480 2023-11-24T07:57:10.515 INFO:teuthology.orchestra.run.smithi059.stdout: 2023-11-24T07:57:10.515 INFO:teuthology.orchestra.run.smithi059.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4220,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6835/838038775","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6833","nonce":838038775},{"type":"v1","addr":"172.21.15.59:6835","nonce":838038775}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4232,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.76:6837/3226690004","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6835","nonce":3226690004},{"type":"v1","addr":"172.21.15.76:6837","nonce":3226690004}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4260,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6834/2986456006","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6832","nonce":2986456006},{"type":"v1","addr":"172.21.15.59:6834","nonce":2986456006}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-24T07:56:24.104295+0000","modified":"2023-11-24T07:56:27.152806+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":4264},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4264":{"gid":4264,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.76:6836/3993077463","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6834","nonce":3993077463},{"type":"v1","addr":"172.21.15.76:6836","nonce":3993077463}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-11-24T07:57:10.516 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:10.433+0000 7f52d162c700 1 -- 172.21.15.59:0/2175823437 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f52b806aed0 msgr2=0x7f52b806d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:57:10.517 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:10.433+0000 7f52d162c700 1 --2- 172.21.15.59:0/2175823437 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f52b806aed0 0x7f52b806d390 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f52cc067300 tx=0x7f52c0009040 comp rx=0 tx=0).stop 2023-11-24T07:57:10.517 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:10.433+0000 7f52d162c700 1 -- 172.21.15.59:0/2175823437 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f52cc0cf480 msgr2=0x7f52cc15e840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:57:10.517 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:10.433+0000 7f52d162c700 1 --2- 172.21.15.59:0/2175823437 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f52cc0cf480 0x7f52cc15e840 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f52bc000c00 tx=0x7f52bc004750 comp rx=0 tx=0).stop 2023-11-24T07:57:10.517 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:10.433+0000 7f52d162c700 1 -- 172.21.15.59:0/2175823437 shutdown_connections 2023-11-24T07:57:10.517 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:10.433+0000 7f52d162c700 1 --2- 172.21.15.59:0/2175823437 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f52b806aed0 0x7f52b806d390 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:10.517 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:10.433+0000 7f52d162c700 1 --2- 172.21.15.59:0/2175823437 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f52cc15e460 0x7f52cc0b1cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:10.517 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:10.433+0000 7f52d162c700 1 --2- 172.21.15.59:0/2175823437 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f52cc0cf480 0x7f52cc15e840 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:10.517 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:10.433+0000 7f52d162c700 1 --2- 172.21.15.59:0/2175823437 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f52cc0d2ba0 0x7f52cc0cdc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:10.517 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:10.433+0000 7f52d162c700 1 -- 172.21.15.59:0/2175823437 >> 172.21.15.59:0/2175823437 conn(0x7f52cc0d3eb0 msgr2=0x7f52cc06d140 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:57:10.517 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:10.433+0000 7f52d162c700 1 -- 172.21.15.59:0/2175823437 shutdown_connections 2023-11-24T07:57:10.517 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:10.433+0000 7f52d162c700 1 -- 172.21.15.59:0/2175823437 wait complete. 2023-11-24T07:57:10.517 INFO:teuthology.orchestra.run.smithi059.stderr:dumped fsmap epoch 8 2023-11-24T07:57:10.518 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-24T07:57:10.649 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:10.648+0000 7f1e43483700 1 Processor -- start 2023-11-24T07:57:10.650 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:10.648+0000 7f1e43483700 1 -- start start 2023-11-24T07:57:10.653 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:10.649+0000 7f1e43483700 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f1e3c1440f0 0x7f1e3c0b51b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:57:10.653 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:10.649+0000 7f1e43483700 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1e3c143ec0 con 0x7f1e3c143ae0 2023-11-24T07:57:10.653 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:10.649+0000 7f1e43483700 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1e3c0b3ae0 con 0x7f1e3c0722c0 2023-11-24T07:57:10.653 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:10.649+0000 7f1e43483700 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1e3c0b40a0 con 0x7f1e3c1440f0 2023-11-24T07:57:10.653 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:10.649+0000 7f1e4121f700 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7f1e3c143ae0 0x7f1e3c071bb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:54686/0 (socket says 172.21.15.59:54686) 2023-11-24T07:57:10.654 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:10.649+0000 7f1e4121f700 1 -- 172.21.15.59:0/282236932 learned_addr learned my addr 172.21.15.59:0/282236932 (peer_addr_for_me v1:172.21.15.59:0/0) 2023-11-24T07:57:10.654 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:10.649+0000 7f1e41a20700 1 --2- 172.21.15.59:0/282236932 >> v2:172.21.15.76:3300/0 conn(0x7f1e3c1440f0 0x7f1e3c0b51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:57:10.654 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:10.649+0000 7f1e33fff700 1 -- 172.21.15.59:0/282236932 <== mon.0 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 698554217 0 0) 0x7f1e3c143ec0 con 0x7f1e3c143ae0 2023-11-24T07:57:10.654 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:10.649+0000 7f1e33fff700 1 -- 172.21.15.59:0/282236932 --> v1:172.21.15.59:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1e280035b0 con 0x7f1e3c143ae0 2023-11-24T07:57:10.654 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:10.649+0000 7f1e33fff700 1 -- 172.21.15.59:0/282236932 <== mon.0 v1:172.21.15.59:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 978937101 0 0) 0x7f1e280035b0 con 0x7f1e3c143ae0 2023-11-24T07:57:10.654 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:10.649+0000 7f1e33fff700 1 -- 172.21.15.59:0/282236932 >> v2:172.21.15.76:3300/0 conn(0x7f1e3c1440f0 msgr2=0x7f1e3c0b51b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T07:57:10.654 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:10.649+0000 7f1e33fff700 1 --2- 172.21.15.59:0/282236932 >> v2:172.21.15.76:3300/0 conn(0x7f1e3c1440f0 0x7f1e3c0b51b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:10.654 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:10.649+0000 7f1e33fff700 1 -- 172.21.15.59:0/282236932 >> v1:172.21.15.76:6789/0 conn(0x7f1e3c0722c0 legacy=0x7f1e3c06e510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:57:10.654 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:10.649+0000 7f1e33fff700 1 -- 172.21.15.59:0/282236932 --> v1:172.21.15.59:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1e3c0b3dc0 con 0x7f1e3c143ae0 2023-11-24T07:57:10.654 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:10.649+0000 7f1e33fff700 1 -- 172.21.15.59:0/282236932 <== mon.0 v1:172.21.15.59:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1312333566 0 0) 0x7f1e38001670 con 0x7f1e3c143ae0 2023-11-24T07:57:10.654 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:10.649+0000 7f1e33fff700 1 -- 172.21.15.59:0/282236932 >> v1:172.21.15.59:6789/0 conn(0x7f1e3c143ae0 legacy=0x7f1e3c071bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:57:10.654 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:10.649+0000 7f1e33fff700 1 --2- 172.21.15.59:0/282236932 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1e28003a00 0x7f1e28003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:57:10.655 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:10.649+0000 7f1e33fff700 1 --2- 172.21.15.59:0/282236932 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1e28004850 0x7f1e28004cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:57:10.655 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:10.649+0000 7f1e33fff700 1 --2- 172.21.15.59:0/282236932 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f1e28005210 0x7f1e2800ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:57:10.655 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:10.649+0000 7f1e33fff700 1 -- 172.21.15.59:0/282236932 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e3c0b40a0 con 0x7f1e28003a00 2023-11-24T07:57:10.655 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:10.649+0000 7f1e33fff700 1 -- 172.21.15.59:0/282236932 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e2800c0b0 con 0x7f1e28004850 2023-11-24T07:57:10.655 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:10.649+0000 7f1e33fff700 1 -- 172.21.15.59:0/282236932 --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1e2800c270 con 0x7f1e28005210 2023-11-24T07:57:10.655 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:10.650+0000 7f1e4121f700 1 --2- 172.21.15.59:0/282236932 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1e28003a00 0x7f1e28003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:57:10.655 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:10.650+0000 7f1e40a1e700 1 --2- 172.21.15.59:0/282236932 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1e28004850 0x7f1e28004cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:57:10.655 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:10.650+0000 7f1e41a20700 1 --2- 172.21.15.59:0/282236932 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f1e28005210 0x7f1e2800ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:57:10.655 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:10.650+0000 7f1e4121f700 1 -- 172.21.15.59:0/282236932 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f1e28005210 msgr2=0x7f1e2800ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T07:57:10.655 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:10.650+0000 7f1e4121f700 1 --2- 172.21.15.59:0/282236932 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f1e28005210 0x7f1e2800ba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:10.655 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:10.650+0000 7f1e4121f700 1 -- 172.21.15.59:0/282236932 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1e28004850 msgr2=0x7f1e28004cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T07:57:10.655 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:10.650+0000 7f1e4121f700 1 --2- 172.21.15.59:0/282236932 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1e28004850 0x7f1e28004cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:10.655 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:10.650+0000 7f1e4121f700 1 -- 172.21.15.59:0/282236932 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1e280035b0 con 0x7f1e28003a00 2023-11-24T07:57:10.656 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:10.650+0000 7f1e40a1e700 1 --2- 172.21.15.59:0/282236932 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1e28004850 0x7f1e28004cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-24T07:57:10.656 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:10.650+0000 7f1e41a20700 1 -- 172.21.15.59:0/282236932 reap_dead start 2023-11-24T07:57:10.656 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:10.650+0000 7f1e4121f700 1 --2- 172.21.15.59:0/282236932 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1e28003a00 0x7f1e28003e20 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f1e3800c240 tx=0x7f1e28005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3f4600e31cb17077 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:57:10.656 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:10.650+0000 7f1e33fff700 1 -- 172.21.15.59:0/282236932 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e38007c10 con 0x7f1e28003a00 2023-11-24T07:57:10.656 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:10.650+0000 7f1e33fff700 1 -- 172.21.15.59:0/282236932 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1e3800cde0 con 0x7f1e28003a00 2023-11-24T07:57:10.656 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:10.650+0000 7f1e33fff700 1 -- 172.21.15.59:0/282236932 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e38005ba0 con 0x7f1e28003a00 2023-11-24T07:57:10.656 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:10.651+0000 7f1e43483700 1 -- 172.21.15.59:0/282236932 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1e28003a00 msgr2=0x7f1e28003e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:57:10.656 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:10.651+0000 7f1e43483700 1 --2- 172.21.15.59:0/282236932 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1e28003a00 0x7f1e28003e20 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f1e3800c240 tx=0x7f1e28005880 comp rx=0 tx=0).stop 2023-11-24T07:57:10.656 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:10.651+0000 7f1e43483700 1 -- 172.21.15.59:0/282236932 shutdown_connections 2023-11-24T07:57:10.656 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:10.651+0000 7f1e43483700 1 --2- 172.21.15.59:0/282236932 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1e28003a00 0x7f1e28003e20 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:10.656 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:10.651+0000 7f1e43483700 1 -- 172.21.15.59:0/282236932 >> 172.21.15.59:0/282236932 conn(0x7f1e3c0b2740 msgr2=0x7f1e3c0b2b20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:57:10.656 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:10.651+0000 7f1e43483700 1 -- 172.21.15.59:0/282236932 shutdown_connections 2023-11-24T07:57:10.656 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:10.651+0000 7f1e43483700 1 -- 172.21.15.59:0/282236932 wait complete. 2023-11-24T07:57:10.659 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:10.652+0000 7f1e43483700 1 Processor -- start 2023-11-24T07:57:10.659 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:10.652+0000 7f1e43483700 1 -- start start 2023-11-24T07:57:10.659 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:10.653+0000 7f1e43483700 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f1e28003a00 0x7f1e3c13aa50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:57:10.660 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:10.653+0000 7f1e43483700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1e3c15dbc0 0x7f1e3c13e5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:57:10.660 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:10.653+0000 7f1e43483700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1e3c148df0 0x7f1e3c149270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:57:10.660 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:10.653+0000 7f1e43483700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e3c1497b0 con 0x7f1e3c15dbc0 2023-11-24T07:57:10.660 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:10.653+0000 7f1e43483700 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e3c15b2f0 con 0x7f1e3c148df0 2023-11-24T07:57:10.660 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:10.653+0000 7f1e43483700 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1e3c1409d0 con 0x7f1e28003a00 2023-11-24T07:57:10.660 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:10.653+0000 7f1e40a1e700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1e3c15dbc0 0x7f1e3c13e5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:57:10.660 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:10.653+0000 7f1e40a1e700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1e3c15dbc0 0x7f1e3c13e5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:51226/0 (socket says 172.21.15.59:51226) 2023-11-24T07:57:10.660 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:10.653+0000 7f1e40a1e700 1 -- 172.21.15.59:0/3018429478 learned_addr learned my addr 172.21.15.59:0/3018429478 (peer_addr_for_me v2:172.21.15.59:0/0) 2023-11-24T07:57:10.660 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:10.653+0000 7f1e4121f700 1 --2- 172.21.15.59:0/3018429478 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f1e28003a00 0x7f1e3c13aa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:57:10.660 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:10.653+0000 7f1e41a20700 1 --2- 172.21.15.59:0/3018429478 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1e3c148df0 0x7f1e3c149270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:57:10.660 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:10.653+0000 7f1e40a1e700 1 -- 172.21.15.59:0/3018429478 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f1e28003a00 msgr2=0x7f1e3c13aa50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:57:10.660 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:10.653+0000 7f1e40a1e700 1 --2- 172.21.15.59:0/3018429478 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f1e28003a00 0x7f1e3c13aa50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:10.660 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:10.653+0000 7f1e40a1e700 1 -- 172.21.15.59:0/3018429478 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1e3c148df0 msgr2=0x7f1e3c149270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:57:10.661 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:10.653+0000 7f1e40a1e700 1 --2- 172.21.15.59:0/3018429478 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1e3c148df0 0x7f1e3c149270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:10.661 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:10.653+0000 7f1e40a1e700 1 -- 172.21.15.59:0/3018429478 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1e3c15e040 con 0x7f1e3c15dbc0 2023-11-24T07:57:10.661 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:10.653+0000 7f1e41a20700 1 --2- 172.21.15.59:0/3018429478 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1e3c148df0 0x7f1e3c149270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-24T07:57:10.661 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:10.653+0000 7f1e40a1e700 1 --2- 172.21.15.59:0/3018429478 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1e3c15dbc0 0x7f1e3c13e5f0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f1e2c00f7f0 tx=0x7f1e2c00fb00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:57:10.661 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:10.653+0000 7f1e31ffb700 1 -- 172.21.15.59:0/3018429478 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e2c0052e0 con 0x7f1e3c15dbc0 2023-11-24T07:57:10.661 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:10.653+0000 7f1e31ffb700 1 -- 172.21.15.59:0/3018429478 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1e2c005440 con 0x7f1e3c15dbc0 2023-11-24T07:57:10.661 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:10.653+0000 7f1e31ffb700 1 -- 172.21.15.59:0/3018429478 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e2c0038a0 con 0x7f1e3c15dbc0 2023-11-24T07:57:10.661 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:10.654+0000 7f1e43483700 1 -- 172.21.15.59:0/3018429478 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1e3c13ee20 con 0x7f1e3c15dbc0 2023-11-24T07:57:10.661 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:10.654+0000 7f1e43483700 1 -- 172.21.15.59:0/3018429478 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1e3c13f3e0 con 0x7f1e3c15dbc0 2023-11-24T07:57:10.661 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:10.655+0000 7f1e43483700 1 -- 172.21.15.59:0/3018429478 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1e3c15b5b0 con 0x7f1e3c15dbc0 2023-11-24T07:57:10.661 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:10.655+0000 7f1e31ffb700 1 -- 172.21.15.59:0/3018429478 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f1e2c003a00 con 0x7f1e3c15dbc0 2023-11-24T07:57:10.661 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:10.655+0000 7f1e31ffb700 1 --2- 172.21.15.59:0/3018429478 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f1e2006aed0 0x7f1e2006d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:57:10.662 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:10.656+0000 7f1e31ffb700 1 -- 172.21.15.59:0/3018429478 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f1e2c089440 con 0x7f1e3c15dbc0 2023-11-24T07:57:10.662 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:10.656+0000 7f1e4121f700 1 --2- 172.21.15.59:0/3018429478 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f1e2006aed0 0x7f1e2006d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:57:10.662 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:10.656+0000 7f1e4121f700 1 --2- 172.21.15.59:0/3018429478 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f1e2006aed0 0x7f1e2006d390 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f1e38000c00 tx=0x7f1e38007000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:57:10.663 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:10.660+0000 7f1e31ffb700 1 -- 172.21.15.59:0/3018429478 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f1e2c0544f0 con 0x7f1e3c15dbc0 2023-11-24T07:57:10.867 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:10.865+0000 7f1e43483700 1 -- 172.21.15.59:0/3018429478 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f1e3c068670 con 0x7f1e3c15dbc0 2023-11-24T07:57:10.868 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:10.867+0000 7f1e31ffb700 1 -- 172.21.15.59:0/3018429478 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) v1 ==== 74+0+16437 (secure 0 0 0) 0x7f1e2c003cb0 con 0x7f1e3c15dbc0 2023-11-24T07:57:10.868 INFO:teuthology.orchestra.run.smithi059.stdout: 2023-11-24T07:57:10.869 INFO:teuthology.orchestra.run.smithi059.stdout:{"epoch":20,"fsid":"9f0e8741-b8f9-48d5-a2c3-09f45903303d","created":"2023-11-24T07:56:06.552729+0000","modified":"2023-11-24T07:56:25.705091+0000","last_up_change":"2023-11-24T07:56:17.372352+0000","last_in_change":"2023-11-24T07:56:09.073250+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-24T07:56:18.352823+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-11-24T07:56:22.185140+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-11-24T07:56:23.084326+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"6d8f9e9b-1782-4e41-928b-a39f2a8ebc06","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6800","nonce":3373878772},{"type":"v1","addr":"172.21.15.59:6801","nonce":3373878772}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6802","nonce":3373878772},{"type":"v1","addr":"172.21.15.59:6803","nonce":3373878772}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6806","nonce":3373878772},{"type":"v1","addr":"172.21.15.59:6807","nonce":3373878772}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6804","nonce":3373878772},{"type":"v1","addr":"172.21.15.59:6805","nonce":3373878772}]},"public_addr":"172.21.15.59:6801/3373878772","cluster_addr":"172.21.15.59:6803/3373878772","heartbeat_back_addr":"172.21.15.59:6807/3373878772","heartbeat_front_addr":"172.21.15.59:6805/3373878772","state":["exists","up"]},{"osd":1,"uuid":"e8d882aa-52e1-4c04-9e1e-c44ef9204a5f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6824","nonce":648258563},{"type":"v1","addr":"172.21.15.59:6825","nonce":648258563}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6826","nonce":648258563},{"type":"v1","addr":"172.21.15.59:6827","nonce":648258563}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6830","nonce":648258563},{"type":"v1","addr":"172.21.15.59:6831","nonce":648258563}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6828","nonce":648258563},{"type":"v1","addr":"172.21.15.59:6829","nonce":648258563}]},"public_addr":"172.21.15.59:6825/648258563","cluster_addr":"172.21.15.59:6827/648258563","heartbeat_back_addr":"172.21.15.59:6831/648258563","heartbeat_front_addr":"172.21.15.59:6829/648258563","state":["exists","up"]},{"osd":2,"uuid":"69eebfa0-695b-4b21-90b9-8a3537168a01","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6808","nonce":1840430654},{"type":"v1","addr":"172.21.15.59:6809","nonce":1840430654}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6810","nonce":1840430654},{"type":"v1","addr":"172.21.15.59:6811","nonce":1840430654}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6814","nonce":1840430654},{"type":"v1","addr":"172.21.15.59:6815","nonce":1840430654}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6812","nonce":1840430654},{"type":"v1","addr":"172.21.15.59:6813","nonce":1840430654}]},"public_addr":"172.21.15.59:6809/1840430654","cluster_addr":"172.21.15.59:6811/1840430654","heartbeat_back_addr":"172.21.15.59:6815/1840430654","heartbeat_front_addr":"172.21.15.59:6813/1840430654","state":["exists","up"]},{"osd":3,"uuid":"96fd77fc-f911-46ff-9445-6f0be2daa8f4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6816","nonce":2504025850},{"type":"v1","addr":"172.21.15.59:6817","nonce":2504025850}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6818","nonce":2504025850},{"type":"v1","addr":"172.21.15.59:6819","nonce":2504025850}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6822","nonce":2504025850},{"type":"v1","addr":"172.21.15.59:6823","nonce":2504025850}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6820","nonce":2504025850},{"type":"v1","addr":"172.21.15.59:6821","nonce":2504025850}]},"public_addr":"172.21.15.59:6817/2504025850","cluster_addr":"172.21.15.59:6819/2504025850","heartbeat_back_addr":"172.21.15.59:6823/2504025850","heartbeat_front_addr":"172.21.15.59:6821/2504025850","state":["exists","up"]},{"osd":4,"uuid":"7ac29fca-b5b2-4c95-9d69-2f164cbbc708","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6816","nonce":2382463431},{"type":"v1","addr":"172.21.15.76:6818","nonce":2382463431}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6820","nonce":2382463431},{"type":"v1","addr":"172.21.15.76:6822","nonce":2382463431}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6828","nonce":2382463431},{"type":"v1","addr":"172.21.15.76:6830","nonce":2382463431}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6824","nonce":2382463431},{"type":"v1","addr":"172.21.15.76:6826","nonce":2382463431}]},"public_addr":"172.21.15.76:6818/2382463431","cluster_addr":"172.21.15.76:6822/2382463431","heartbeat_back_addr":"172.21.15.76:6830/2382463431","heartbeat_front_addr":"172.21.15.76:6826/2382463431","state":["exists","up"]},{"osd":5,"uuid":"485bf711-3851-4238-a4f4-51d52e0a9c58","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6800","nonce":329334308},{"type":"v1","addr":"172.21.15.76:6802","nonce":329334308}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6805","nonce":329334308},{"type":"v1","addr":"172.21.15.76:6807","nonce":329334308}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6814","nonce":329334308},{"type":"v1","addr":"172.21.15.76:6815","nonce":329334308}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6810","nonce":329334308},{"type":"v1","addr":"172.21.15.76:6812","nonce":329334308}]},"public_addr":"172.21.15.76:6802/329334308","cluster_addr":"172.21.15.76:6807/329334308","heartbeat_back_addr":"172.21.15.76:6815/329334308","heartbeat_front_addr":"172.21.15.76:6812/329334308","state":["exists","up"]},{"osd":6,"uuid":"72b3f2fc-1e4a-4e32-968d-003647a0286f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6801","nonce":3946579410},{"type":"v1","addr":"172.21.15.76:6803","nonce":3946579410}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6804","nonce":3946579410},{"type":"v1","addr":"172.21.15.76:6806","nonce":3946579410}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6811","nonce":3946579410},{"type":"v1","addr":"172.21.15.76:6813","nonce":3946579410}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6808","nonce":3946579410},{"type":"v1","addr":"172.21.15.76:6809","nonce":3946579410}]},"public_addr":"172.21.15.76:6803/3946579410","cluster_addr":"172.21.15.76:6806/3946579410","heartbeat_back_addr":"172.21.15.76:6813/3946579410","heartbeat_front_addr":"172.21.15.76:6809/3946579410","state":["exists","up"]},{"osd":7,"uuid":"1ddceb90-6c5d-4d1a-ad56-99cb877f8c70","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6817","nonce":841761538},{"type":"v1","addr":"172.21.15.76:6819","nonce":841761538}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6821","nonce":841761538},{"type":"v1","addr":"172.21.15.76:6823","nonce":841761538}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6829","nonce":841761538},{"type":"v1","addr":"172.21.15.76:6831","nonce":841761538}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6825","nonce":841761538},{"type":"v1","addr":"172.21.15.76:6827","nonce":841761538}]},"public_addr":"172.21.15.76:6819/841761538","cluster_addr":"172.21.15.76:6823/841761538","heartbeat_back_addr":"172.21.15.76:6831/841761538","heartbeat_front_addr":"172.21.15.76:6827/841761538","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T07:56:14.294838+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T07:56:14.270779+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T07:56:14.231809+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T07:56:14.260983+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T07:56:14.302997+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T07:56:14.258600+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T07:56:14.251413+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T07:56:14.304727+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-24T07:57:10.871 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:10.869+0000 7f1e43483700 1 -- 172.21.15.59:0/3018429478 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f1e2006aed0 msgr2=0x7f1e2006d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:57:10.871 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:10.869+0000 7f1e43483700 1 --2- 172.21.15.59:0/3018429478 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f1e2006aed0 0x7f1e2006d390 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f1e38000c00 tx=0x7f1e38007000 comp rx=0 tx=0).stop 2023-11-24T07:57:10.871 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:10.869+0000 7f1e43483700 1 -- 172.21.15.59:0/3018429478 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1e3c15dbc0 msgr2=0x7f1e3c13e5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:57:10.871 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:10.869+0000 7f1e43483700 1 --2- 172.21.15.59:0/3018429478 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1e3c15dbc0 0x7f1e3c13e5f0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f1e2c00f7f0 tx=0x7f1e2c00fb00 comp rx=0 tx=0).stop 2023-11-24T07:57:10.871 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:10.869+0000 7f1e43483700 1 -- 172.21.15.59:0/3018429478 shutdown_connections 2023-11-24T07:57:10.871 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:10.869+0000 7f1e43483700 1 --2- 172.21.15.59:0/3018429478 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f1e2006aed0 0x7f1e2006d390 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:10.871 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:10.869+0000 7f1e43483700 1 --2- 172.21.15.59:0/3018429478 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1e3c148df0 0x7f1e3c149270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:10.871 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:10.869+0000 7f1e43483700 1 --2- 172.21.15.59:0/3018429478 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f1e28003a00 0x7f1e3c13aa50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:10.871 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:10.869+0000 7f1e43483700 1 --2- 172.21.15.59:0/3018429478 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1e3c15dbc0 0x7f1e3c13e5f0 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:10.871 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:10.869+0000 7f1e43483700 1 -- 172.21.15.59:0/3018429478 >> 172.21.15.59:0/3018429478 conn(0x7f1e3c0b2740 msgr2=0x7f1e3c0ce2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:57:10.872 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:10.869+0000 7f1e43483700 1 -- 172.21.15.59:0/3018429478 shutdown_connections 2023-11-24T07:57:10.872 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:10.869+0000 7f1e43483700 1 -- 172.21.15.59:0/3018429478 wait complete. 2023-11-24T07:57:10.883 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-24T07:56:18.352823+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-24T07:56:22.185140+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-11-24T07:56:23.084326+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-24T07:57:10.883 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-24T07:57:11.011 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:11.009+0000 7fd1305f6700 1 Processor -- start 2023-11-24T07:57:11.011 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:11.009+0000 7fd1305f6700 1 -- start start 2023-11-24T07:57:11.013 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:11.010+0000 7fd1305f6700 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fd128144610 0x7fd1280d09f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:57:11.013 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:11.010+0000 7fd1305f6700 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd12806e9d0 con 0x7fd12806f700 2023-11-24T07:57:11.013 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:11.010+0000 7fd1305f6700 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd12806f270 con 0x7fd12815eed0 2023-11-24T07:57:11.013 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:11.010+0000 7fd1305f6700 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd1281449f0 con 0x7fd128144610 2023-11-24T07:57:11.013 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:11.010+0000 7fd12db91700 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7fd12806f700 0x7fd12815e7c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:54706/0 (socket says 172.21.15.59:54706) 2023-11-24T07:57:11.013 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:11.010+0000 7fd12db91700 1 -- 172.21.15.59:0/1500838085 learned_addr learned my addr 172.21.15.59:0/1500838085 (peer_addr_for_me v1:172.21.15.59:0/0) 2023-11-24T07:57:11.013 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:11.011+0000 7fd12e392700 1 --2- 172.21.15.59:0/1500838085 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fd128144610 0x7fd1280d09f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:57:11.013 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:11.011+0000 7fd12d390700 1 -- 172.21.15.59:0/1500838085 <== mon.0 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 904782046 0 0) 0x7fd12806e9d0 con 0x7fd12806f700 2023-11-24T07:57:11.014 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:11.011+0000 7fd12d390700 1 -- 172.21.15.59:0/1500838085 --> v1:172.21.15.59:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd1140035b0 con 0x7fd12806f700 2023-11-24T07:57:11.014 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:11.011+0000 7fd12d390700 1 -- 172.21.15.59:0/1500838085 <== mon.1 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1891975804 0 0) 0x7fd12806f270 con 0x7fd12815eed0 2023-11-24T07:57:11.014 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:11.011+0000 7fd12e392700 1 -- 172.21.15.59:0/1500838085 >> v1:172.21.15.76:6789/0 conn(0x7fd12815eed0 legacy=0x7fd1280cde80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:57:11.014 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:11.011+0000 7fd12e392700 1 -- 172.21.15.59:0/1500838085 >> v1:172.21.15.59:6789/0 conn(0x7fd12806f700 legacy=0x7fd12815e7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:57:11.014 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:11.011+0000 7fd12e392700 1 -- 172.21.15.59:0/1500838085 --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd12815f530 con 0x7fd128144610 2023-11-24T07:57:11.014 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:11.011+0000 7fd12e392700 1 --2- 172.21.15.59:0/1500838085 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fd128144610 0x7fd1280d09f0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fd1180098a0 tx=0x7fd118009bb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d49e759e0c014441 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:57:11.014 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:11.011+0000 7fd12d390700 1 -- 172.21.15.59:0/1500838085 <== mon.2 v2:172.21.15.76:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd124001670 con 0x7fd128144610 2023-11-24T07:57:11.014 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:11.011+0000 7fd12d390700 1 -- 172.21.15.59:0/1500838085 <== mon.2 v2:172.21.15.76:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd118004030 con 0x7fd128144610 2023-11-24T07:57:11.014 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:11.011+0000 7fd12d390700 1 -- 172.21.15.59:0/1500838085 <== mon.2 v2:172.21.15.76:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1180043c0 con 0x7fd128144610 2023-11-24T07:57:11.015 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:11.012+0000 7fd1305f6700 1 -- 172.21.15.59:0/1500838085 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fd128144610 msgr2=0x7fd1280d09f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:57:11.015 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:11.012+0000 7fd1305f6700 1 --2- 172.21.15.59:0/1500838085 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fd128144610 0x7fd1280d09f0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fd1180098a0 tx=0x7fd118009bb0 comp rx=0 tx=0).stop 2023-11-24T07:57:11.015 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:11.012+0000 7fd1305f6700 1 -- 172.21.15.59:0/1500838085 shutdown_connections 2023-11-24T07:57:11.015 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:11.012+0000 7fd1305f6700 1 --2- 172.21.15.59:0/1500838085 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fd128144610 0x7fd1280d09f0 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:11.015 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:11.012+0000 7fd1305f6700 1 -- 172.21.15.59:0/1500838085 >> 172.21.15.59:0/1500838085 conn(0x7fd1280b2740 msgr2=0x7fd1280b2b50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:57:11.015 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:11.012+0000 7fd1305f6700 1 -- 172.21.15.59:0/1500838085 shutdown_connections 2023-11-24T07:57:11.016 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:11.012+0000 7fd1305f6700 1 -- 172.21.15.59:0/1500838085 wait complete. 2023-11-24T07:57:11.016 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:11.013+0000 7fd1305f6700 1 Processor -- start 2023-11-24T07:57:11.016 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:11.013+0000 7fd1305f6700 1 -- start start 2023-11-24T07:57:11.017 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:11.014+0000 7fd1305f6700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd12806f700 0x7fd128130b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:57:11.018 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:11.014+0000 7fd1305f6700 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fd128144610 0x7fd12815b190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:57:11.018 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:11.014+0000 7fd1305f6700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd12815eed0 0x7fd1280afd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:57:11.018 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:11.014+0000 7fd1305f6700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd12815c210 con 0x7fd12806f700 2023-11-24T07:57:11.018 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:11.014+0000 7fd1305f6700 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1280a9450 con 0x7fd12815eed0 2023-11-24T07:57:11.018 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:11.014+0000 7fd1305f6700 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd12815b6d0 con 0x7fd128144610 2023-11-24T07:57:11.018 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:11.014+0000 7fd12e392700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd12806f700 0x7fd128130b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:57:11.018 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:11.014+0000 7fd12e392700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd12806f700 0x7fd128130b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:51238/0 (socket says 172.21.15.59:51238) 2023-11-24T07:57:11.018 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:11.014+0000 7fd12e392700 1 -- 172.21.15.59:0/607492932 learned_addr learned my addr 172.21.15.59:0/607492932 (peer_addr_for_me v2:172.21.15.59:0/0) 2023-11-24T07:57:11.018 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:11.014+0000 7fd12eb93700 1 --2- 172.21.15.59:0/607492932 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd12815eed0 0x7fd1280afd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:57:11.018 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:11.014+0000 7fd12e392700 1 -- 172.21.15.59:0/607492932 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fd128144610 msgr2=0x7fd12815b190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:57:11.018 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:11.014+0000 7fd12db91700 1 --2- 172.21.15.59:0/607492932 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fd128144610 0x7fd12815b190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:57:11.018 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:11.014+0000 7fd12e392700 1 --2- 172.21.15.59:0/607492932 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fd128144610 0x7fd12815b190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:11.018 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:11.014+0000 7fd12e392700 1 -- 172.21.15.59:0/607492932 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd12815eed0 msgr2=0x7fd1280afd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:57:11.019 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:11.014+0000 7fd12e392700 1 --2- 172.21.15.59:0/607492932 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd12815eed0 0x7fd1280afd90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:11.019 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:11.014+0000 7fd12e392700 1 -- 172.21.15.59:0/607492932 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd1180094a0 con 0x7fd12806f700 2023-11-24T07:57:11.019 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:11.015+0000 7fd12e392700 1 --2- 172.21.15.59:0/607492932 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd12806f700 0x7fd128130b00 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7fd118009870 tx=0x7fd118004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:57:11.019 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:11.015+0000 7fd11effd700 1 -- 172.21.15.59:0/607492932 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd118010790 con 0x7fd12806f700 2023-11-24T07:57:11.019 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:11.015+0000 7fd11effd700 1 -- 172.21.15.59:0/607492932 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd1180108f0 con 0x7fd12806f700 2023-11-24T07:57:11.019 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:11.015+0000 7fd11effd700 1 -- 172.21.15.59:0/607492932 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd118010c80 con 0x7fd12806f700 2023-11-24T07:57:11.019 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:11.015+0000 7fd1305f6700 1 -- 172.21.15.59:0/607492932 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd12815b970 con 0x7fd12806f700 2023-11-24T07:57:11.019 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:11.015+0000 7fd1305f6700 1 -- 172.21.15.59:0/607492932 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd12815bf30 con 0x7fd12806f700 2023-11-24T07:57:11.019 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:11.016+0000 7fd1305f6700 1 -- 172.21.15.59:0/607492932 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd128075480 con 0x7fd12806f700 2023-11-24T07:57:11.020 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:11.017+0000 7fd11effd700 1 -- 172.21.15.59:0/607492932 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fd11802f430 con 0x7fd12806f700 2023-11-24T07:57:11.020 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:11.018+0000 7fd11effd700 1 --2- 172.21.15.59:0/607492932 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7fd10c06aed0 0x7fd10c06d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:57:11.020 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:11.018+0000 7fd11effd700 1 -- 172.21.15.59:0/607492932 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fd11808add0 con 0x7fd12806f700 2023-11-24T07:57:11.020 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:11.018+0000 7fd12db91700 1 --2- 172.21.15.59:0/607492932 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7fd10c06aed0 0x7fd10c06d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:57:11.020 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:11.018+0000 7fd12db91700 1 --2- 172.21.15.59:0/607492932 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7fd10c06aed0 0x7fd10c06d390 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fd124000c00 tx=0x7fd124009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:57:11.023 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:11.021+0000 7fd11effd700 1 -- 172.21.15.59:0/607492932 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fd118055ff0 con 0x7fd12806f700 2023-11-24T07:57:11.222 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:11.220+0000 7fd1305f6700 1 -- 172.21.15.59:0/607492932 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fd12806e2a0 con 0x7fd12806f700 2023-11-24T07:57:11.222 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:11.221+0000 7fd11effd700 1 -- 172.21.15.59:0/607492932 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v20) v1 ==== 93+0+10 (secure 0 0 0) 0x7fd1180045b0 con 0x7fd12806f700 2023-11-24T07:57:11.223 INFO:teuthology.orchestra.run.smithi059.stdout:pg_num: 1 2023-11-24T07:57:11.225 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:11.223+0000 7fd1305f6700 1 -- 172.21.15.59:0/607492932 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7fd10c06aed0 msgr2=0x7fd10c06d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:57:11.225 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:11.223+0000 7fd1305f6700 1 --2- 172.21.15.59:0/607492932 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7fd10c06aed0 0x7fd10c06d390 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fd124000c00 tx=0x7fd124009040 comp rx=0 tx=0).stop 2023-11-24T07:57:11.225 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:11.223+0000 7fd1305f6700 1 -- 172.21.15.59:0/607492932 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd12806f700 msgr2=0x7fd128130b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:57:11.225 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:11.223+0000 7fd1305f6700 1 --2- 172.21.15.59:0/607492932 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd12806f700 0x7fd128130b00 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7fd118009870 tx=0x7fd118004750 comp rx=0 tx=0).stop 2023-11-24T07:57:11.225 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:11.223+0000 7fd1305f6700 1 -- 172.21.15.59:0/607492932 shutdown_connections 2023-11-24T07:57:11.225 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:11.223+0000 7fd1305f6700 1 --2- 172.21.15.59:0/607492932 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7fd10c06aed0 0x7fd10c06d390 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:11.225 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:11.223+0000 7fd1305f6700 1 --2- 172.21.15.59:0/607492932 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd12815eed0 0x7fd1280afd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:11.225 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:11.223+0000 7fd1305f6700 1 --2- 172.21.15.59:0/607492932 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd12806f700 0x7fd128130b00 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:11.225 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:11.223+0000 7fd1305f6700 1 --2- 172.21.15.59:0/607492932 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fd128144610 0x7fd12815b190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:11.225 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:11.223+0000 7fd1305f6700 1 -- 172.21.15.59:0/607492932 >> 172.21.15.59:0/607492932 conn(0x7fd1280b2740 msgr2=0x7fd1280cdf00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:57:11.225 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:11.223+0000 7fd1305f6700 1 -- 172.21.15.59:0/607492932 shutdown_connections 2023-11-24T07:57:11.225 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:11.223+0000 7fd1305f6700 1 -- 172.21.15.59:0/607492932 wait complete. 2023-11-24T07:57:11.237 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-24T07:57:11.365 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:11.363+0000 7fb41c488700 1 Processor -- start 2023-11-24T07:57:11.365 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:11.363+0000 7fb41c488700 1 -- start start 2023-11-24T07:57:11.367 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:11.364+0000 7fb41c488700 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fb4140722c0 0x7fb4140b51b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:57:11.367 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:11.364+0000 7fb41c488700 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fb4141440f0 0x7fb41406e510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:57:11.367 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:11.364+0000 7fb41c488700 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb414143ec0 con 0x7fb414143ae0 2023-11-24T07:57:11.367 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:11.364+0000 7fb41c488700 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb4140b40a0 con 0x7fb4140722c0 2023-11-24T07:57:11.367 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:11.364+0000 7fb41c488700 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb4140b3ae0 con 0x7fb4141440f0 2023-11-24T07:57:11.367 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:11.365+0000 7fb419a23700 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fb4140722c0 0x7fb4140b51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:57:11.367 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:11.365+0000 7fb41aa25700 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fb4141440f0 0x7fb41406e510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:57:11.367 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:11.365+0000 7fb41aa25700 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fb4141440f0 0x7fb41406e510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3301/0 says I am v2:172.21.15.59:50690/0 (socket says 172.21.15.59:50690) 2023-11-24T07:57:11.368 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:11.365+0000 7fb41aa25700 1 -- 172.21.15.59:0/1576448015 learned_addr learned my addr 172.21.15.59:0/1576448015 (peer_addr_for_me v2:172.21.15.59:0/0) 2023-11-24T07:57:11.368 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:11.365+0000 7fb41aa25700 1 -- 172.21.15.59:0/1576448015 >> v2:172.21.15.76:3300/0 conn(0x7fb4140722c0 msgr2=0x7fb4140b51b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T07:57:11.368 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:11.365+0000 7fb41aa25700 1 --2- 172.21.15.59:0/1576448015 >> v2:172.21.15.76:3300/0 conn(0x7fb4140722c0 0x7fb4140b51b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:11.368 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:11.365+0000 7fb41aa25700 1 -- 172.21.15.59:0/1576448015 >> v1:172.21.15.76:6789/0 conn(0x7fb414143ae0 legacy=0x7fb414071bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:57:11.368 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:11.365+0000 7fb41aa25700 1 -- 172.21.15.59:0/1576448015 --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb4140b3dc0 con 0x7fb4141440f0 2023-11-24T07:57:11.368 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:11.365+0000 7fb419a23700 1 --2- 172.21.15.59:0/1576448015 >> v2:172.21.15.76:3300/0 conn(0x7fb4140722c0 0x7fb4140b51b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-24T07:57:11.368 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:11.365+0000 7fb41aa25700 1 --2- 172.21.15.59:0/1576448015 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fb4141440f0 0x7fb41406e510 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fb40c009e50 tx=0x7fb40c00b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c569e2093f494c96 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:57:11.368 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:11.365+0000 7fb419222700 1 -- 172.21.15.59:0/1576448015 <== mon.2 v2:172.21.15.76:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb40c00e070 con 0x7fb4141440f0 2023-11-24T07:57:11.368 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:11.365+0000 7fb419222700 1 -- 172.21.15.59:0/1576448015 <== mon.2 v2:172.21.15.76:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb40c00b910 con 0x7fb4141440f0 2023-11-24T07:57:11.368 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:11.365+0000 7fb419222700 1 -- 172.21.15.59:0/1576448015 <== mon.2 v2:172.21.15.76:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb40c00bcc0 con 0x7fb4141440f0 2023-11-24T07:57:11.369 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:11.366+0000 7fb41c488700 1 -- 172.21.15.59:0/1576448015 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fb4141440f0 msgr2=0x7fb41406e510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:57:11.369 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:11.366+0000 7fb41c488700 1 --2- 172.21.15.59:0/1576448015 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fb4141440f0 0x7fb41406e510 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fb40c009e50 tx=0x7fb40c00b330 comp rx=0 tx=0).stop 2023-11-24T07:57:11.369 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:11.366+0000 7fb41c488700 1 -- 172.21.15.59:0/1576448015 shutdown_connections 2023-11-24T07:57:11.369 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:11.366+0000 7fb41c488700 1 --2- 172.21.15.59:0/1576448015 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fb4141440f0 0x7fb41406e510 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:11.370 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:11.366+0000 7fb41c488700 1 --2- 172.21.15.59:0/1576448015 >> v2:172.21.15.76:3300/0 conn(0x7fb4140722c0 0x7fb4140b51b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:11.370 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:11.366+0000 7fb41c488700 1 -- 172.21.15.59:0/1576448015 >> 172.21.15.59:0/1576448015 conn(0x7fb4140b2740 msgr2=0x7fb4140b2b20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:57:11.370 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:11.366+0000 7fb41c488700 1 -- 172.21.15.59:0/1576448015 shutdown_connections 2023-11-24T07:57:11.370 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:11.366+0000 7fb41c488700 1 -- 172.21.15.59:0/1576448015 wait complete. 2023-11-24T07:57:11.370 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:11.367+0000 7fb41c488700 1 Processor -- start 2023-11-24T07:57:11.370 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:11.367+0000 7fb41c488700 1 -- start start 2023-11-24T07:57:11.372 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:11.368+0000 7fb41c488700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb4140722c0 0x7fb4140cacb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:57:11.372 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:11.368+0000 7fb41c488700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb414143ae0 0x7fb4141f4ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:57:11.372 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:11.368+0000 7fb41c488700 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fb4141440f0 0x7fb414158b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:57:11.372 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:11.368+0000 7fb41c488700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb41413fac0 con 0x7fb4140722c0 2023-11-24T07:57:11.372 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:11.368+0000 7fb41c488700 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4141f5380 con 0x7fb414143ae0 2023-11-24T07:57:11.372 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:11.368+0000 7fb41c488700 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb4140c9ef0 con 0x7fb4141440f0 2023-11-24T07:57:11.372 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:11.368+0000 7fb41a224700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb4140722c0 0x7fb4140cacb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:57:11.372 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:11.368+0000 7fb41a224700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb4140722c0 0x7fb4140cacb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:51252/0 (socket says 172.21.15.59:51252) 2023-11-24T07:57:11.373 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:11.368+0000 7fb41a224700 1 -- 172.21.15.59:0/3692213878 learned_addr learned my addr 172.21.15.59:0/3692213878 (peer_addr_for_me v2:172.21.15.59:0/0) 2023-11-24T07:57:11.373 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:11.368+0000 7fb41aa25700 1 --2- 172.21.15.59:0/3692213878 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fb4141440f0 0x7fb414158b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:57:11.373 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:11.368+0000 7fb419a23700 1 --2- 172.21.15.59:0/3692213878 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb414143ae0 0x7fb4141f4ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:57:11.373 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:11.369+0000 7fb41a224700 1 -- 172.21.15.59:0/3692213878 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fb4141440f0 msgr2=0x7fb414158b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:57:11.373 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:11.369+0000 7fb41a224700 1 --2- 172.21.15.59:0/3692213878 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fb4141440f0 0x7fb414158b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:11.373 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:11.369+0000 7fb41a224700 1 -- 172.21.15.59:0/3692213878 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb414143ae0 msgr2=0x7fb4141f4ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:57:11.373 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:11.369+0000 7fb41a224700 1 --2- 172.21.15.59:0/3692213878 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb414143ae0 0x7fb4141f4ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:11.373 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:11.369+0000 7fb41a224700 1 -- 172.21.15.59:0/3692213878 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb4100094a0 con 0x7fb4140722c0 2023-11-24T07:57:11.373 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:11.369+0000 7fb419a23700 1 --2- 172.21.15.59:0/3692213878 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb414143ae0 0x7fb4141f4ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T07:57:11.374 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:11.369+0000 7fb41a224700 1 --2- 172.21.15.59:0/3692213878 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb4140722c0 0x7fb4140cacb0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7fb40400c4e0 tx=0x7fb40400c8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:57:11.374 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:11.369+0000 7fb40affd700 1 -- 172.21.15.59:0/3692213878 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4040052d0 con 0x7fb4140722c0 2023-11-24T07:57:11.374 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:11.369+0000 7fb40affd700 1 -- 172.21.15.59:0/3692213878 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb404005430 con 0x7fb4140722c0 2023-11-24T07:57:11.374 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:11.369+0000 7fb41c488700 1 -- 172.21.15.59:0/3692213878 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb40c009b30 con 0x7fb4140722c0 2023-11-24T07:57:11.374 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:11.369+0000 7fb40affd700 1 -- 172.21.15.59:0/3692213878 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4040038c0 con 0x7fb4140722c0 2023-11-24T07:57:11.374 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:11.369+0000 7fb41c488700 1 -- 172.21.15.59:0/3692213878 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb4140ca4b0 con 0x7fb4140722c0 2023-11-24T07:57:11.374 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:11.370+0000 7fb41c488700 1 -- 172.21.15.59:0/3692213878 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb414075480 con 0x7fb4140722c0 2023-11-24T07:57:11.374 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:11.370+0000 7fb40affd700 1 -- 172.21.15.59:0/3692213878 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fb404003a20 con 0x7fb4140722c0 2023-11-24T07:57:11.375 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:11.371+0000 7fb40affd700 1 --2- 172.21.15.59:0/3692213878 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7fb3f806aed0 0x7fb3f806d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:57:11.375 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:11.371+0000 7fb40affd700 1 -- 172.21.15.59:0/3692213878 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fb404088a20 con 0x7fb4140722c0 2023-11-24T07:57:11.375 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:11.371+0000 7fb419a23700 1 --2- 172.21.15.59:0/3692213878 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7fb3f806aed0 0x7fb3f806d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:57:11.375 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:11.372+0000 7fb419a23700 1 --2- 172.21.15.59:0/3692213878 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7fb3f806aed0 0x7fb3f806d390 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fb41406c040 tx=0x7fb410008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:57:11.377 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:11.376+0000 7fb40affd700 1 -- 172.21.15.59:0/3692213878 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fb404053b60 con 0x7fb4140722c0 2023-11-24T07:57:11.583 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:11.581+0000 7fb41c488700 1 -- 172.21.15.59:0/3692213878 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fb4140ccc50 con 0x7fb4140722c0 2023-11-24T07:57:11.583 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:11.582+0000 7fb40affd700 1 -- 172.21.15.59:0/3692213878 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v20) v1 ==== 104+0+11 (secure 0 0 0) 0x7fb404017020 con 0x7fb4140722c0 2023-11-24T07:57:11.583 INFO:teuthology.orchestra.run.smithi059.stdout:pg_num: 32 2023-11-24T07:57:11.586 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:11.584+0000 7fb41c488700 1 -- 172.21.15.59:0/3692213878 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7fb3f806aed0 msgr2=0x7fb3f806d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:57:11.586 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:11.584+0000 7fb41c488700 1 --2- 172.21.15.59:0/3692213878 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7fb3f806aed0 0x7fb3f806d390 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fb41406c040 tx=0x7fb410008040 comp rx=0 tx=0).stop 2023-11-24T07:57:11.586 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:11.584+0000 7fb41c488700 1 -- 172.21.15.59:0/3692213878 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb4140722c0 msgr2=0x7fb4140cacb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:57:11.586 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:11.584+0000 7fb41c488700 1 --2- 172.21.15.59:0/3692213878 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb4140722c0 0x7fb4140cacb0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7fb40400c4e0 tx=0x7fb40400c8a0 comp rx=0 tx=0).stop 2023-11-24T07:57:11.586 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:11.584+0000 7fb41c488700 1 -- 172.21.15.59:0/3692213878 shutdown_connections 2023-11-24T07:57:11.586 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:11.584+0000 7fb41c488700 1 --2- 172.21.15.59:0/3692213878 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7fb3f806aed0 0x7fb3f806d390 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:11.586 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:11.584+0000 7fb41c488700 1 --2- 172.21.15.59:0/3692213878 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fb4141440f0 0x7fb414158b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:11.586 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:11.584+0000 7fb41c488700 1 --2- 172.21.15.59:0/3692213878 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb4140722c0 0x7fb4140cacb0 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:11.586 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:11.584+0000 7fb41c488700 1 --2- 172.21.15.59:0/3692213878 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb414143ae0 0x7fb4141f4ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:11.587 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:11.584+0000 7fb41c488700 1 -- 172.21.15.59:0/3692213878 >> 172.21.15.59:0/3692213878 conn(0x7fb4140b2740 msgr2=0x7fb414159f80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:57:11.587 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:11.584+0000 7fb41c488700 1 -- 172.21.15.59:0/3692213878 shutdown_connections 2023-11-24T07:57:11.587 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:11.584+0000 7fb41c488700 1 -- 172.21.15.59:0/3692213878 wait complete. 2023-11-24T07:57:11.598 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-24T07:57:11.727 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:11.725+0000 7f4a4bdd5700 1 Processor -- start 2023-11-24T07:57:11.727 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:11.726+0000 7f4a4bdd5700 1 -- start start 2023-11-24T07:57:11.731 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:11.726+0000 7f4a4bdd5700 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f4a4406fc50 0x7f4a4406eef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:57:11.731 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:11.726+0000 7f4a4bdd5700 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4a440d1230 con 0x7f4a440b0e90 2023-11-24T07:57:11.731 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:11.726+0000 7f4a4bdd5700 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4a440d0c70 con 0x7f4a4415e1c0 2023-11-24T07:57:11.731 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:11.726+0000 7f4a4bdd5700 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4a440d0f50 con 0x7f4a4406fc50 2023-11-24T07:57:11.731 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:11.727+0000 7f4a4a372700 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7f4a440b0e90 0x7f4a4406ca10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:54738/0 (socket says 172.21.15.59:54738) 2023-11-24T07:57:11.731 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:11.727+0000 7f4a4a372700 1 -- 172.21.15.59:0/3035103471 learned_addr learned my addr 172.21.15.59:0/3035103471 (peer_addr_for_me v1:172.21.15.59:0/0) 2023-11-24T07:57:11.731 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:11.727+0000 7f4a49370700 1 --2- 172.21.15.59:0/3035103471 >> v2:172.21.15.76:3300/0 conn(0x7f4a4406fc50 0x7f4a4406eef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:57:11.732 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:11.727+0000 7f4a48b6f700 1 -- 172.21.15.59:0/3035103471 <== mon.0 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 100321701 0 0) 0x7f4a440d1230 con 0x7f4a440b0e90 2023-11-24T07:57:11.732 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:11.727+0000 7f4a48b6f700 1 -- 172.21.15.59:0/3035103471 --> v1:172.21.15.59:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4a300035b0 con 0x7f4a440b0e90 2023-11-24T07:57:11.732 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:11.727+0000 7f4a48b6f700 1 -- 172.21.15.59:0/3035103471 <== mon.1 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 42898169 0 0) 0x7f4a440d0c70 con 0x7f4a4415e1c0 2023-11-24T07:57:11.732 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:11.727+0000 7f4a48b6f700 1 -- 172.21.15.59:0/3035103471 --> v1:172.21.15.76:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4a440d1230 con 0x7f4a4415e1c0 2023-11-24T07:57:11.732 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:11.727+0000 7f4a48b6f700 1 -- 172.21.15.59:0/3035103471 <== mon.0 v1:172.21.15.59:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 600289749 0 0) 0x7f4a300035b0 con 0x7f4a440b0e90 2023-11-24T07:57:11.732 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:11.727+0000 7f4a48b6f700 1 -- 172.21.15.59:0/3035103471 >> v2:172.21.15.76:3300/0 conn(0x7f4a4406fc50 msgr2=0x7f4a4406eef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T07:57:11.732 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:11.727+0000 7f4a48b6f700 1 --2- 172.21.15.59:0/3035103471 >> v2:172.21.15.76:3300/0 conn(0x7f4a4406fc50 0x7f4a4406eef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:11.732 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:11.727+0000 7f4a48b6f700 1 -- 172.21.15.59:0/3035103471 >> v1:172.21.15.76:6789/0 conn(0x7f4a4415e1c0 legacy=0x7f4a4406f540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:57:11.732 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:11.727+0000 7f4a48b6f700 1 -- 172.21.15.59:0/3035103471 --> v1:172.21.15.59:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4a4406c210 con 0x7f4a440b0e90 2023-11-24T07:57:11.732 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:11.727+0000 7f4a48b6f700 1 -- 172.21.15.59:0/3035103471 <== mon.0 v1:172.21.15.59:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1312333566 0 0) 0x7f4a3c0019f0 con 0x7f4a440b0e90 2023-11-24T07:57:11.732 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:11.727+0000 7f4a49370700 1 --2- 172.21.15.59:0/3035103471 >> v2:172.21.15.76:3300/0 conn(0x7f4a4406fc50 0x7f4a4406eef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-24T07:57:11.732 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:11.727+0000 7f4a48b6f700 1 -- 172.21.15.59:0/3035103471 >> v1:172.21.15.59:6789/0 conn(0x7f4a440b0e90 legacy=0x7f4a4406ca10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:57:11.733 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:11.727+0000 7f4a48b6f700 1 --2- 172.21.15.59:0/3035103471 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4a30003b90 0x7f4a30003f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:57:11.733 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:11.727+0000 7f4a48b6f700 1 --2- 172.21.15.59:0/3035103471 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4a30004850 0x7f4a30004cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:57:11.733 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:11.727+0000 7f4a48b6f700 1 --2- 172.21.15.59:0/3035103471 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f4a30005210 0x7f4a3000ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:57:11.733 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:11.727+0000 7f4a48b6f700 1 -- 172.21.15.59:0/3035103471 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a440d0f50 con 0x7f4a30003b90 2023-11-24T07:57:11.733 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:11.727+0000 7f4a48b6f700 1 -- 172.21.15.59:0/3035103471 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a3000c140 con 0x7f4a30004850 2023-11-24T07:57:11.733 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:11.727+0000 7f4a48b6f700 1 -- 172.21.15.59:0/3035103471 --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4a3000c300 con 0x7f4a30005210 2023-11-24T07:57:11.733 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:11.727+0000 7f4a49b71700 1 --2- 172.21.15.59:0/3035103471 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4a30003b90 0x7f4a30003f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:57:11.733 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:11.728+0000 7f4a49370700 1 --2- 172.21.15.59:0/3035103471 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4a30004850 0x7f4a30004cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:57:11.733 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:11.728+0000 7f4a4a372700 1 --2- 172.21.15.59:0/3035103471 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f4a30005210 0x7f4a3000ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:57:11.733 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:11.728+0000 7f4a49b71700 1 -- 172.21.15.59:0/3035103471 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f4a30005210 msgr2=0x7f4a3000ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T07:57:11.733 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:11.728+0000 7f4a49b71700 1 --2- 172.21.15.59:0/3035103471 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f4a30005210 0x7f4a3000ba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:11.734 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:11.728+0000 7f4a49b71700 1 -- 172.21.15.59:0/3035103471 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4a30004850 msgr2=0x7f4a30004cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T07:57:11.734 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:11.728+0000 7f4a49b71700 1 --2- 172.21.15.59:0/3035103471 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4a30004850 0x7f4a30004cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:11.734 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:11.728+0000 7f4a4a372700 1 -- 172.21.15.59:0/3035103471 reap_dead start 2023-11-24T07:57:11.734 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:11.728+0000 7f4a49b71700 1 -- 172.21.15.59:0/3035103471 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4a300035b0 con 0x7f4a30003b90 2023-11-24T07:57:11.734 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:11.728+0000 7f4a49b71700 1 --2- 172.21.15.59:0/3035103471 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4a30003b90 0x7f4a30003f90 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f4a40001940 tx=0x7f4a30005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8174d5bfb5291830 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:57:11.734 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:11.728+0000 7f4a48b6f700 1 -- 172.21.15.59:0/3035103471 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a40001430 con 0x7f4a30003b90 2023-11-24T07:57:11.734 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:11.728+0000 7f4a48b6f700 1 -- 172.21.15.59:0/3035103471 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4a40001d70 con 0x7f4a30003b90 2023-11-24T07:57:11.734 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:11.728+0000 7f4a48b6f700 1 -- 172.21.15.59:0/3035103471 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a40005510 con 0x7f4a30003b90 2023-11-24T07:57:11.734 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:11.729+0000 7f4a4bdd5700 1 -- 172.21.15.59:0/3035103471 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4a30003b90 msgr2=0x7f4a30003f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:57:11.734 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:11.729+0000 7f4a4bdd5700 1 --2- 172.21.15.59:0/3035103471 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4a30003b90 0x7f4a30003f90 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f4a40001940 tx=0x7f4a30005880 comp rx=0 tx=0).stop 2023-11-24T07:57:11.734 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:11.729+0000 7f4a4bdd5700 1 -- 172.21.15.59:0/3035103471 shutdown_connections 2023-11-24T07:57:11.734 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:11.729+0000 7f4a4bdd5700 1 --2- 172.21.15.59:0/3035103471 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4a30003b90 0x7f4a30003f90 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:11.735 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:11.729+0000 7f4a4bdd5700 1 -- 172.21.15.59:0/3035103471 >> 172.21.15.59:0/3035103471 conn(0x7f4a440b3720 msgr2=0x7f4a440b3b30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:57:11.735 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:11.729+0000 7f4a4bdd5700 1 -- 172.21.15.59:0/3035103471 shutdown_connections 2023-11-24T07:57:11.735 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:11.729+0000 7f4a4bdd5700 1 -- 172.21.15.59:0/3035103471 wait complete. 2023-11-24T07:57:11.737 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:11.730+0000 7f4a4bdd5700 1 Processor -- start 2023-11-24T07:57:11.737 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:11.730+0000 7f4a4bdd5700 1 -- start start 2023-11-24T07:57:11.737 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:11.731+0000 7f4a4bdd5700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4a30003b90 0x7f4a44161c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:57:11.737 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:11.731+0000 7f4a4bdd5700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4a440b7360 0x7f4a4415ccd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:57:11.738 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:11.731+0000 7f4a4bdd5700 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f4a440b77e0 0x7f4a4415d210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:57:11.738 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:11.731+0000 7f4a4bdd5700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a440b4b40 con 0x7f4a440b7360 2023-11-24T07:57:11.738 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:11.731+0000 7f4a4bdd5700 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a440c6060 con 0x7f4a30003b90 2023-11-24T07:57:11.738 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:11.731+0000 7f4a4bdd5700 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4a440c6340 con 0x7f4a440b77e0 2023-11-24T07:57:11.738 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:11.731+0000 7f4a49370700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4a440b7360 0x7f4a4415ccd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:57:11.738 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:11.731+0000 7f4a49370700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4a440b7360 0x7f4a4415ccd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:51274/0 (socket says 172.21.15.59:51274) 2023-11-24T07:57:11.738 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:11.731+0000 7f4a49370700 1 -- 172.21.15.59:0/2804238046 learned_addr learned my addr 172.21.15.59:0/2804238046 (peer_addr_for_me v2:172.21.15.59:0/0) 2023-11-24T07:57:11.738 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:11.731+0000 7f4a4a372700 1 --2- 172.21.15.59:0/2804238046 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f4a440b77e0 0x7f4a4415d210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:57:11.738 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:11.731+0000 7f4a49b71700 1 --2- 172.21.15.59:0/2804238046 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4a30003b90 0x7f4a44161c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:57:11.738 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:11.731+0000 7f4a49370700 1 -- 172.21.15.59:0/2804238046 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f4a440b77e0 msgr2=0x7f4a4415d210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:57:11.738 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:11.731+0000 7f4a49370700 1 --2- 172.21.15.59:0/2804238046 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f4a440b77e0 0x7f4a4415d210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:11.738 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:11.731+0000 7f4a49370700 1 -- 172.21.15.59:0/2804238046 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4a30003b90 msgr2=0x7f4a44161c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:57:11.739 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:11.731+0000 7f4a49370700 1 --2- 172.21.15.59:0/2804238046 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4a30003b90 0x7f4a44161c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:11.739 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:11.731+0000 7f4a49370700 1 -- 172.21.15.59:0/2804238046 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4a440c6620 con 0x7f4a440b7360 2023-11-24T07:57:11.739 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:11.731+0000 7f4a49370700 1 --2- 172.21.15.59:0/2804238046 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4a440b7360 0x7f4a4415ccd0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f4a38011c00 tx=0x7f4a38009eb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:57:11.739 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:11.732+0000 7f4a367fc700 1 -- 172.21.15.59:0/2804238046 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a3800faf0 con 0x7f4a440b7360 2023-11-24T07:57:11.739 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:11.732+0000 7f4a367fc700 1 -- 172.21.15.59:0/2804238046 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4a3800fc50 con 0x7f4a440b7360 2023-11-24T07:57:11.739 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:11.732+0000 7f4a367fc700 1 -- 172.21.15.59:0/2804238046 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a38004030 con 0x7f4a440b7360 2023-11-24T07:57:11.739 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:11.732+0000 7f4a4bdd5700 1 -- 172.21.15.59:0/2804238046 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4a440c6900 con 0x7f4a440b7360 2023-11-24T07:57:11.739 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:11.732+0000 7f4a4bdd5700 1 -- 172.21.15.59:0/2804238046 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4a440c6ec0 con 0x7f4a440b7360 2023-11-24T07:57:11.739 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:11.733+0000 7f4a4bdd5700 1 -- 172.21.15.59:0/2804238046 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4a440c7230 con 0x7f4a440b7360 2023-11-24T07:57:11.739 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:11.733+0000 7f4a367fc700 1 -- 172.21.15.59:0/2804238046 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f4a38004190 con 0x7f4a440b7360 2023-11-24T07:57:11.739 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:11.734+0000 7f4a367fc700 1 --2- 172.21.15.59:0/2804238046 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f4a2806aed0 0x7f4a2806d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:57:11.739 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:11.734+0000 7f4a367fc700 1 -- 172.21.15.59:0/2804238046 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f4a38089c30 con 0x7f4a440b7360 2023-11-24T07:57:11.740 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:11.734+0000 7f4a49b71700 1 --2- 172.21.15.59:0/2804238046 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f4a2806aed0 0x7f4a2806d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:57:11.740 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:11.734+0000 7f4a49b71700 1 --2- 172.21.15.59:0/2804238046 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f4a2806aed0 0x7f4a2806d390 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f4a40005040 tx=0x7f4a4000a000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:57:11.740 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:11.738+0000 7f4a367fc700 1 -- 172.21.15.59:0/2804238046 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f4a38054c40 con 0x7f4a440b7360 2023-11-24T07:57:11.941 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:11.939+0000 7f4a4bdd5700 1 -- 172.21.15.59:0/2804238046 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f4a44075480 con 0x7f4a440b7360 2023-11-24T07:57:11.941 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:11.940+0000 7f4a367fc700 1 -- 172.21.15.59:0/2804238046 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v20) v1 ==== 100+0+11 (secure 0 0 0) 0x7f4a38022070 con 0x7f4a440b7360 2023-11-24T07:57:11.942 INFO:teuthology.orchestra.run.smithi059.stdout:pg_num: 64 2023-11-24T07:57:11.944 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:11.942+0000 7f4a4bdd5700 1 -- 172.21.15.59:0/2804238046 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f4a2806aed0 msgr2=0x7f4a2806d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:57:11.944 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:11.942+0000 7f4a4bdd5700 1 --2- 172.21.15.59:0/2804238046 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f4a2806aed0 0x7f4a2806d390 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f4a40005040 tx=0x7f4a4000a000 comp rx=0 tx=0).stop 2023-11-24T07:57:11.945 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:11.942+0000 7f4a4bdd5700 1 -- 172.21.15.59:0/2804238046 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4a440b7360 msgr2=0x7f4a4415ccd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:57:11.945 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:11.942+0000 7f4a4bdd5700 1 --2- 172.21.15.59:0/2804238046 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4a440b7360 0x7f4a4415ccd0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f4a38011c00 tx=0x7f4a38009eb0 comp rx=0 tx=0).stop 2023-11-24T07:57:11.945 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:11.942+0000 7f4a4bdd5700 1 -- 172.21.15.59:0/2804238046 shutdown_connections 2023-11-24T07:57:11.946 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:11.942+0000 7f4a4bdd5700 1 --2- 172.21.15.59:0/2804238046 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f4a2806aed0 0x7f4a2806d390 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:11.946 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:11.942+0000 7f4a4bdd5700 1 --2- 172.21.15.59:0/2804238046 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f4a440b77e0 0x7f4a4415d210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:11.946 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:11.942+0000 7f4a4bdd5700 1 --2- 172.21.15.59:0/2804238046 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4a30003b90 0x7f4a44161c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:11.946 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:11.942+0000 7f4a4bdd5700 1 --2- 172.21.15.59:0/2804238046 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4a440b7360 0x7f4a4415ccd0 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:11.946 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:11.942+0000 7f4a4bdd5700 1 -- 172.21.15.59:0/2804238046 >> 172.21.15.59:0/2804238046 conn(0x7f4a440b3720 msgr2=0x7f4a440b2ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:57:11.946 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:11.942+0000 7f4a4bdd5700 1 -- 172.21.15.59:0/2804238046 shutdown_connections 2023-11-24T07:57:11.947 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:11.942+0000 7f4a4bdd5700 1 -- 172.21.15.59:0/2804238046 wait complete. 2023-11-24T07:57:11.956 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-24T07:57:12.081 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:12.079+0000 7f2cd24e8700 1 Processor -- start 2023-11-24T07:57:12.081 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:12.079+0000 7f2cd24e8700 1 -- start start 2023-11-24T07:57:12.083 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:12.080+0000 7f2cd24e8700 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f2ccc0b4150 0x7f2ccc06de50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:57:12.083 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:12.080+0000 7f2cd24e8700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f2ccc0b5ab0 0x7f2ccc06e390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:57:12.083 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:12.080+0000 7f2cd24e8700 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2ccc071f50 con 0x7f2ccc06b620 2023-11-24T07:57:12.084 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:12.080+0000 7f2cd24e8700 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2ccc072230 con 0x7f2ccc0b5ab0 2023-11-24T07:57:12.084 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:12.080+0000 7f2cd24e8700 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2ccc072510 con 0x7f2ccc0b4150 2023-11-24T07:57:12.084 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:12.080+0000 7f2cd0a85700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f2ccc0b5ab0 0x7f2ccc06e390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:57:12.084 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:12.080+0000 7f2ccbfff700 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7f2ccc06b620 0x7f2ccc0b3a40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:54754/0 (socket says 172.21.15.59:54754) 2023-11-24T07:57:12.084 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:12.080+0000 7f2ccbfff700 1 -- 172.21.15.59:0/2723265269 learned_addr learned my addr 172.21.15.59:0/2723265269 (peer_addr_for_me v1:172.21.15.59:0/0) 2023-11-24T07:57:12.084 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:12.080+0000 7f2ccb7fe700 1 --2- 172.21.15.59:0/2723265269 >> v2:172.21.15.76:3300/0 conn(0x7f2ccc0b4150 0x7f2ccc06de50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:57:12.084 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:12.081+0000 7f2cd0a85700 1 -- 172.21.15.59:0/2723265269 >> v2:172.21.15.76:3300/0 conn(0x7f2ccc0b4150 msgr2=0x7f2ccc06de50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T07:57:12.084 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:12.081+0000 7f2cd0a85700 1 --2- 172.21.15.59:0/2723265269 >> v2:172.21.15.76:3300/0 conn(0x7f2ccc0b4150 0x7f2ccc06de50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:12.084 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:12.081+0000 7f2cd0a85700 1 -- 172.21.15.59:0/2723265269 >> v1:172.21.15.59:6789/0 conn(0x7f2ccc06b620 legacy=0x7f2ccc0b3a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:57:12.084 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:12.081+0000 7f2cd0a85700 1 -- 172.21.15.59:0/2723265269 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2ccc0727f0 con 0x7f2ccc0b5ab0 2023-11-24T07:57:12.084 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:12.081+0000 7f2ccaffd700 1 -- 172.21.15.59:0/2723265269 <== mon.0 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1718906893 0 0) 0x7f2ccc071f50 con 0x7f2ccc06b620 2023-11-24T07:57:12.084 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:12.081+0000 7f2cd0a85700 1 --2- 172.21.15.59:0/2723265269 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2ccc0b5ab0 0x7f2ccc06e390 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f2ccc06d6f0 tx=0x7f2cc400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e77be2bf4edb07ca server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:57:12.084 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:12.081+0000 7f2ccaffd700 1 -- 172.21.15.59:0/2723265269 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2cc400e070 con 0x7f2ccc0b5ab0 2023-11-24T07:57:12.085 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:12.081+0000 7f2ccaffd700 1 -- 172.21.15.59:0/2723265269 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2cc400bbd0 con 0x7f2ccc0b5ab0 2023-11-24T07:57:12.085 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:12.081+0000 7f2ccaffd700 1 -- 172.21.15.59:0/2723265269 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2cc4012430 con 0x7f2ccc0b5ab0 2023-11-24T07:57:12.085 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:12.081+0000 7f2cd24e8700 1 -- 172.21.15.59:0/2723265269 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2ccc0b5ab0 msgr2=0x7f2ccc06e390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:57:12.085 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:12.081+0000 7f2cd24e8700 1 --2- 172.21.15.59:0/2723265269 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2ccc0b5ab0 0x7f2ccc06e390 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f2ccc06d6f0 tx=0x7f2cc400b330 comp rx=0 tx=0).stop 2023-11-24T07:57:12.085 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:12.081+0000 7f2cd24e8700 1 -- 172.21.15.59:0/2723265269 shutdown_connections 2023-11-24T07:57:12.085 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:12.081+0000 7f2cd24e8700 1 --2- 172.21.15.59:0/2723265269 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2ccc0b5ab0 0x7f2ccc06e390 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:12.085 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:12.081+0000 7f2cd24e8700 1 --2- 172.21.15.59:0/2723265269 >> v2:172.21.15.76:3300/0 conn(0x7f2ccc0b4150 0x7f2ccc06de50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:12.085 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:12.081+0000 7f2cd24e8700 1 -- 172.21.15.59:0/2723265269 >> 172.21.15.59:0/2723265269 conn(0x7f2ccc0b2740 msgr2=0x7f2ccc0b2b20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:57:12.085 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:12.082+0000 7f2cd24e8700 1 -- 172.21.15.59:0/2723265269 shutdown_connections 2023-11-24T07:57:12.085 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:12.082+0000 7f2cd24e8700 1 -- 172.21.15.59:0/2723265269 wait complete. 2023-11-24T07:57:12.087 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:12.083+0000 7f2cd24e8700 1 Processor -- start 2023-11-24T07:57:12.087 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:12.083+0000 7f2cd24e8700 1 -- start start 2023-11-24T07:57:12.087 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:12.084+0000 7f2cd24e8700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2ccc06b620 0x7f2ccc1dd4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:57:12.088 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:12.084+0000 7f2cd24e8700 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f2ccc0b4150 0x7f2ccc1f7610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:57:12.088 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:12.084+0000 7f2cd24e8700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2ccc0b5ab0 0x7f2ccc13f450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:57:12.088 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:12.084+0000 7f2cd24e8700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ccc1ddae0 con 0x7f2ccc0b5ab0 2023-11-24T07:57:12.088 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:12.084+0000 7f2cd24e8700 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ccc1f7b50 con 0x7f2ccc06b620 2023-11-24T07:57:12.088 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:12.084+0000 7f2cd24e8700 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2ccc1e5b60 con 0x7f2ccc0b4150 2023-11-24T07:57:12.088 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:12.084+0000 7f2cd0a85700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2ccc0b5ab0 0x7f2ccc13f450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:57:12.088 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:12.084+0000 7f2cd0a85700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2ccc0b5ab0 0x7f2ccc13f450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:51290/0 (socket says 172.21.15.59:51290) 2023-11-24T07:57:12.088 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:12.084+0000 7f2cd0a85700 1 -- 172.21.15.59:0/4258175630 learned_addr learned my addr 172.21.15.59:0/4258175630 (peer_addr_for_me v2:172.21.15.59:0/0) 2023-11-24T07:57:12.088 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:12.084+0000 7f2ccbfff700 1 --2- 172.21.15.59:0/4258175630 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2ccc06b620 0x7f2ccc1dd4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:57:12.088 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:12.084+0000 7f2ccb7fe700 1 --2- 172.21.15.59:0/4258175630 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f2ccc0b4150 0x7f2ccc1f7610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:57:12.088 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:12.084+0000 7f2cd0a85700 1 -- 172.21.15.59:0/4258175630 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f2ccc0b4150 msgr2=0x7f2ccc1f7610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:57:12.088 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:12.084+0000 7f2cd0a85700 1 --2- 172.21.15.59:0/4258175630 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f2ccc0b4150 0x7f2ccc1f7610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:12.088 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:12.084+0000 7f2cd0a85700 1 -- 172.21.15.59:0/4258175630 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2ccc06b620 msgr2=0x7f2ccc1dd4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:57:12.089 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:12.084+0000 7f2cd0a85700 1 --2- 172.21.15.59:0/4258175630 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2ccc06b620 0x7f2ccc1dd4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:12.089 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:12.084+0000 7f2cd0a85700 1 -- 172.21.15.59:0/4258175630 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2cc4009a70 con 0x7f2ccc0b5ab0 2023-11-24T07:57:12.089 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:12.084+0000 7f2ccb7fe700 1 --2- 172.21.15.59:0/4258175630 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f2ccc0b4150 0x7f2ccc1f7610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-24T07:57:12.089 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:12.084+0000 7f2cd0a85700 1 --2- 172.21.15.59:0/4258175630 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2ccc0b5ab0 0x7f2ccc13f450 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f2ccc0d1bd0 tx=0x7f2cc4004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:57:12.089 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:12.084+0000 7f2cc8ff9700 1 -- 172.21.15.59:0/4258175630 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2cc4012430 con 0x7f2ccc0b5ab0 2023-11-24T07:57:12.089 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:12.084+0000 7f2cc8ff9700 1 -- 172.21.15.59:0/4258175630 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2cc4020070 con 0x7f2ccc0b5ab0 2023-11-24T07:57:12.089 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:12.084+0000 7f2cd24e8700 1 -- 172.21.15.59:0/4258175630 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2ccc1e5e40 con 0x7f2ccc0b5ab0 2023-11-24T07:57:12.089 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:12.084+0000 7f2cc8ff9700 1 -- 172.21.15.59:0/4258175630 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2cc4004280 con 0x7f2ccc0b5ab0 2023-11-24T07:57:12.089 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:12.085+0000 7f2cd24e8700 1 -- 172.21.15.59:0/4258175630 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2ccc1e6400 con 0x7f2ccc0b5ab0 2023-11-24T07:57:12.089 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:12.085+0000 7f2cd24e8700 1 -- 172.21.15.59:0/4258175630 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2ccc1e65c0 con 0x7f2ccc0b5ab0 2023-11-24T07:57:12.089 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:12.086+0000 7f2cc8ff9700 1 -- 172.21.15.59:0/4258175630 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f2cc40043e0 con 0x7f2ccc0b5ab0 2023-11-24T07:57:12.090 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:12.086+0000 7f2cc8ff9700 1 --2- 172.21.15.59:0/4258175630 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f2cb806aed0 0x7f2cb806d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:57:12.090 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:12.086+0000 7f2cc8ff9700 1 -- 172.21.15.59:0/4258175630 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f2cc4022070 con 0x7f2ccc0b5ab0 2023-11-24T07:57:12.090 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:12.086+0000 7f2ccbfff700 1 --2- 172.21.15.59:0/4258175630 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f2cb806aed0 0x7f2cb806d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:57:12.090 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:12.087+0000 7f2ccbfff700 1 --2- 172.21.15.59:0/4258175630 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f2cb806aed0 0x7f2cb806d390 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f2cbc007fd0 tx=0x7f2cbc007de0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:57:12.092 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:12.091+0000 7f2cc8ff9700 1 -- 172.21.15.59:0/4258175630 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f2cc4056610 con 0x7f2ccc0b5ab0 2023-11-24T07:57:12.337 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:12.336+0000 7f2cd24e8700 1 -- 172.21.15.59:0/4258175630 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f2ccc068670 con 0x7f2ccc0b5ab0 2023-11-24T07:57:12.338 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:12.336+0000 7f2cc8ff9700 1 -- 172.21.15.59:0/4258175630 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) v1 ==== 93+0+4904 (secure 0 0 0) 0x7f2cc4059c30 con 0x7f2ccc0b5ab0 2023-11-24T07:57:12.338 INFO:teuthology.orchestra.run.smithi059.stdout: 2023-11-24T07:57:12.338 INFO:teuthology.orchestra.run.smithi059.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4220,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6835/838038775","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6833","nonce":838038775},{"type":"v1","addr":"172.21.15.59:6835","nonce":838038775}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4232,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.76:6837/3226690004","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6835","nonce":3226690004},{"type":"v1","addr":"172.21.15.76:6837","nonce":3226690004}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4260,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6834/2986456006","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6832","nonce":2986456006},{"type":"v1","addr":"172.21.15.59:6834","nonce":2986456006}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-24T07:56:24.104295+0000","modified":"2023-11-24T07:56:27.152806+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":4264},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4264":{"gid":4264,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.76:6836/3993077463","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6834","nonce":3993077463},{"type":"v1","addr":"172.21.15.76:6836","nonce":3993077463}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-11-24T07:57:12.340 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:12.338+0000 7f2cd24e8700 1 -- 172.21.15.59:0/4258175630 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f2cb806aed0 msgr2=0x7f2cb806d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:57:12.340 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:12.338+0000 7f2cd24e8700 1 --2- 172.21.15.59:0/4258175630 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f2cb806aed0 0x7f2cb806d390 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f2cbc007fd0 tx=0x7f2cbc007de0 comp rx=0 tx=0).stop 2023-11-24T07:57:12.340 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:12.338+0000 7f2cd24e8700 1 -- 172.21.15.59:0/4258175630 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2ccc0b5ab0 msgr2=0x7f2ccc13f450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:57:12.340 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:12.338+0000 7f2cd24e8700 1 --2- 172.21.15.59:0/4258175630 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2ccc0b5ab0 0x7f2ccc13f450 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f2ccc0d1bd0 tx=0x7f2cc4004ad0 comp rx=0 tx=0).stop 2023-11-24T07:57:12.340 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:12.338+0000 7f2cd24e8700 1 -- 172.21.15.59:0/4258175630 shutdown_connections 2023-11-24T07:57:12.341 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:12.338+0000 7f2cd24e8700 1 --2- 172.21.15.59:0/4258175630 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f2cb806aed0 0x7f2cb806d390 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:12.341 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:12.338+0000 7f2cd24e8700 1 --2- 172.21.15.59:0/4258175630 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2ccc0b5ab0 0x7f2ccc13f450 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:12.341 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:12.338+0000 7f2cd24e8700 1 --2- 172.21.15.59:0/4258175630 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2ccc06b620 0x7f2ccc1dd4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:12.341 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:12.338+0000 7f2cd24e8700 1 --2- 172.21.15.59:0/4258175630 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f2ccc0b4150 0x7f2ccc1f7610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:12.341 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:12.338+0000 7f2cd24e8700 1 -- 172.21.15.59:0/4258175630 >> 172.21.15.59:0/4258175630 conn(0x7f2ccc0b2740 msgr2=0x7f2ccc06dd50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:57:12.341 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:12.338+0000 7f2cd24e8700 1 -- 172.21.15.59:0/4258175630 shutdown_connections 2023-11-24T07:57:12.341 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:12.338+0000 7f2cd24e8700 1 -- 172.21.15.59:0/4258175630 wait complete. 2023-11-24T07:57:12.341 INFO:teuthology.orchestra.run.smithi059.stderr:dumped fsmap epoch 8 2023-11-24T07:57:12.352 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-24T07:57:12.480 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:12.478+0000 7f535af33700 1 Processor -- start 2023-11-24T07:57:12.481 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:12.478+0000 7f535af33700 1 -- start start 2023-11-24T07:57:12.483 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:12.479+0000 7f535af33700 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f535406b6e0 0x7f535406db70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:57:12.483 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:12.479+0000 7f535af33700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f5354145100 0x7f535406e0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:57:12.483 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:12.479+0000 7f535af33700 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5354160720 con 0x7f5354144c10 2023-11-24T07:57:12.483 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:12.479+0000 7f535af33700 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f535406f500 con 0x7f5354145100 2023-11-24T07:57:12.483 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:12.479+0000 7f535af33700 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f53540cd610 con 0x7f535406b6e0 2023-11-24T07:57:12.483 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:12.479+0000 7f53594d0700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f5354145100 0x7f535406e0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:57:12.483 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:12.479+0000 7f5358ccf700 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7f5354144c10 0x7f535406afd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:54768/0 (socket says 172.21.15.59:54768) 2023-11-24T07:57:12.484 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:12.479+0000 7f5358ccf700 1 -- 172.21.15.59:0/1486977920 learned_addr learned my addr 172.21.15.59:0/1486977920 (peer_addr_for_me v1:172.21.15.59:0/0) 2023-11-24T07:57:12.484 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:12.480+0000 7f5353fff700 1 --2- 172.21.15.59:0/1486977920 >> v2:172.21.15.76:3300/0 conn(0x7f535406b6e0 0x7f535406db70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:57:12.484 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:12.480+0000 7f53594d0700 1 -- 172.21.15.59:0/1486977920 >> v2:172.21.15.76:3300/0 conn(0x7f535406b6e0 msgr2=0x7f535406db70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T07:57:12.484 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:12.480+0000 7f53594d0700 1 --2- 172.21.15.59:0/1486977920 >> v2:172.21.15.76:3300/0 conn(0x7f535406b6e0 0x7f535406db70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:12.484 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:12.480+0000 7f53594d0700 1 -- 172.21.15.59:0/1486977920 >> v1:172.21.15.59:6789/0 conn(0x7f5354144c10 legacy=0x7f535406afd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:57:12.484 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:12.480+0000 7f53594d0700 1 -- 172.21.15.59:0/1486977920 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f535406f710 con 0x7f5354145100 2023-11-24T07:57:12.484 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:12.480+0000 7f5353fff700 1 --2- 172.21.15.59:0/1486977920 >> v2:172.21.15.76:3300/0 conn(0x7f535406b6e0 0x7f535406db70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T07:57:12.484 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:12.480+0000 7f53594d0700 1 --2- 172.21.15.59:0/1486977920 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5354145100 0x7f535406e0b0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f5354071b60 tx=0x7f534c00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=69fbf95a9372a29 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:57:12.484 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:12.480+0000 7f53537fe700 1 -- 172.21.15.59:0/1486977920 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f534c00e070 con 0x7f5354145100 2023-11-24T07:57:12.484 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:12.480+0000 7f53537fe700 1 -- 172.21.15.59:0/1486977920 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f534c00bbd0 con 0x7f5354145100 2023-11-24T07:57:12.484 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:12.480+0000 7f53537fe700 1 -- 172.21.15.59:0/1486977920 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f534c012430 con 0x7f5354145100 2023-11-24T07:57:12.485 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:12.480+0000 7f535af33700 1 -- 172.21.15.59:0/1486977920 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5354145100 msgr2=0x7f535406e0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:57:12.485 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:12.480+0000 7f535af33700 1 --2- 172.21.15.59:0/1486977920 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5354145100 0x7f535406e0b0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f5354071b60 tx=0x7f534c00b330 comp rx=0 tx=0).stop 2023-11-24T07:57:12.485 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:12.481+0000 7f535af33700 1 -- 172.21.15.59:0/1486977920 shutdown_connections 2023-11-24T07:57:12.485 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:12.481+0000 7f535af33700 1 --2- 172.21.15.59:0/1486977920 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5354145100 0x7f535406e0b0 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:12.485 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:12.481+0000 7f535af33700 1 --2- 172.21.15.59:0/1486977920 >> v2:172.21.15.76:3300/0 conn(0x7f535406b6e0 0x7f535406db70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:12.485 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:12.481+0000 7f535af33700 1 -- 172.21.15.59:0/1486977920 >> 172.21.15.59:0/1486977920 conn(0x7f53540b2740 msgr2=0x7f53540b2b20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:57:12.485 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:12.481+0000 7f535af33700 1 -- 172.21.15.59:0/1486977920 shutdown_connections 2023-11-24T07:57:12.485 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:12.481+0000 7f535af33700 1 -- 172.21.15.59:0/1486977920 wait complete. 2023-11-24T07:57:12.488 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:12.482+0000 7f535af33700 1 Processor -- start 2023-11-24T07:57:12.488 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:12.482+0000 7f535af33700 1 -- start start 2023-11-24T07:57:12.488 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:12.483+0000 7f535af33700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f535406b6e0 0x7f535413ab70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:57:12.488 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:12.483+0000 7f535af33700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5354144c10 0x7f53541f0e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:57:12.488 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:12.483+0000 7f535af33700 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f5354145100 0x7f535413cf50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:57:12.488 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:12.483+0000 7f535af33700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5354142c50 con 0x7f535406b6e0 2023-11-24T07:57:12.488 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:12.483+0000 7f535af33700 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f535413b0b0 con 0x7f5354144c10 2023-11-24T07:57:12.488 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:12.483+0000 7f535af33700 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5354143490 con 0x7f5354145100 2023-11-24T07:57:12.488 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:12.483+0000 7f5358ccf700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f535406b6e0 0x7f535413ab70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:57:12.489 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:12.483+0000 7f5358ccf700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f535406b6e0 0x7f535413ab70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:51300/0 (socket says 172.21.15.59:51300) 2023-11-24T07:57:12.489 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:12.483+0000 7f5358ccf700 1 -- 172.21.15.59:0/1440647908 learned_addr learned my addr 172.21.15.59:0/1440647908 (peer_addr_for_me v2:172.21.15.59:0/0) 2023-11-24T07:57:12.489 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:12.484+0000 7f53594d0700 1 --2- 172.21.15.59:0/1440647908 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f5354145100 0x7f535413cf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:57:12.489 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:12.484+0000 7f5353fff700 1 --2- 172.21.15.59:0/1440647908 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5354144c10 0x7f53541f0e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:57:12.489 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:12.484+0000 7f5358ccf700 1 -- 172.21.15.59:0/1440647908 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f5354145100 msgr2=0x7f535413cf50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:57:12.489 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:12.484+0000 7f5358ccf700 1 --2- 172.21.15.59:0/1440647908 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f5354145100 0x7f535413cf50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:12.489 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:12.484+0000 7f5358ccf700 1 -- 172.21.15.59:0/1440647908 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5354144c10 msgr2=0x7f53541f0e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:57:12.489 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:12.484+0000 7f5358ccf700 1 --2- 172.21.15.59:0/1440647908 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5354144c10 0x7f53541f0e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:12.489 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:12.484+0000 7f5358ccf700 1 -- 172.21.15.59:0/1440647908 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f534c009a70 con 0x7f535406b6e0 2023-11-24T07:57:12.489 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:12.484+0000 7f5358ccf700 1 --2- 172.21.15.59:0/1440647908 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f535406b6e0 0x7f535413ab70 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f534800c520 tx=0x7f534800c830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:57:12.489 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:12.484+0000 7f53517fa700 1 -- 172.21.15.59:0/1440647908 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5348005d70 con 0x7f535406b6e0 2023-11-24T07:57:12.490 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:12.484+0000 7f53517fa700 1 -- 172.21.15.59:0/1440647908 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f53480070d0 con 0x7f535406b6e0 2023-11-24T07:57:12.490 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:12.484+0000 7f53517fa700 1 -- 172.21.15.59:0/1440647908 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5348007440 con 0x7f535406b6e0 2023-11-24T07:57:12.490 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:12.484+0000 7f535af33700 1 -- 172.21.15.59:0/1440647908 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f535415e110 con 0x7f535406b6e0 2023-11-24T07:57:12.490 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:12.484+0000 7f535af33700 1 -- 172.21.15.59:0/1440647908 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f535413b860 con 0x7f535406b6e0 2023-11-24T07:57:12.490 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:12.485+0000 7f535af33700 1 -- 172.21.15.59:0/1440647908 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5354068670 con 0x7f535406b6e0 2023-11-24T07:57:12.490 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:12.486+0000 7f53517fa700 1 -- 172.21.15.59:0/1440647908 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f53480075a0 con 0x7f535406b6e0 2023-11-24T07:57:12.490 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:12.486+0000 7f53517fa700 1 --2- 172.21.15.59:0/1440647908 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f533806aed0 0x7f533806d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:57:12.490 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:12.486+0000 7f53517fa700 1 -- 172.21.15.59:0/1440647908 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5738+0+0 (secure 0 0 0) 0x7f5348088770 con 0x7f535406b6e0 2023-11-24T07:57:12.490 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:12.486+0000 7f5353fff700 1 --2- 172.21.15.59:0/1440647908 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f533806aed0 0x7f533806d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:57:12.491 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:12.487+0000 7f5353fff700 1 --2- 172.21.15.59:0/1440647908 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f533806aed0 0x7f533806d390 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f53540b3b10 tx=0x7f5340008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:57:12.492 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:12.490+0000 7f53517fa700 1 -- 172.21.15.59:0/1440647908 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f5348053940 con 0x7f535406b6e0 2023-11-24T07:57:12.696 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:12.695+0000 7f535af33700 1 -- 172.21.15.59:0/1440647908 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f53540d0840 con 0x7f535406b6e0 2023-11-24T07:57:12.697 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:12.696+0000 7f53517fa700 1 -- 172.21.15.59:0/1440647908 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v21) v1 ==== 74+0+16486 (secure 0 0 0) 0x7f5348056f60 con 0x7f535406b6e0 2023-11-24T07:57:12.698 INFO:teuthology.orchestra.run.smithi059.stdout: 2023-11-24T07:57:12.698 INFO:teuthology.orchestra.run.smithi059.stdout:{"epoch":21,"fsid":"9f0e8741-b8f9-48d5-a2c3-09f45903303d","created":"2023-11-24T07:56:06.552729+0000","modified":"2023-11-24T07:57:12.429639+0000","last_up_change":"2023-11-24T07:56:17.372352+0000","last_in_change":"2023-11-24T07:56:09.073250+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-24T07:56:18.352823+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-11-24T07:56:22.185140+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-11-24T07:56:23.084326+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"6d8f9e9b-1782-4e41-928b-a39f2a8ebc06","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6800","nonce":3373878772},{"type":"v1","addr":"172.21.15.59:6801","nonce":3373878772}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6802","nonce":3373878772},{"type":"v1","addr":"172.21.15.59:6803","nonce":3373878772}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6806","nonce":3373878772},{"type":"v1","addr":"172.21.15.59:6807","nonce":3373878772}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6804","nonce":3373878772},{"type":"v1","addr":"172.21.15.59:6805","nonce":3373878772}]},"public_addr":"172.21.15.59:6801/3373878772","cluster_addr":"172.21.15.59:6803/3373878772","heartbeat_back_addr":"172.21.15.59:6807/3373878772","heartbeat_front_addr":"172.21.15.59:6805/3373878772","state":["exists","up"]},{"osd":1,"uuid":"e8d882aa-52e1-4c04-9e1e-c44ef9204a5f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6824","nonce":648258563},{"type":"v1","addr":"172.21.15.59:6825","nonce":648258563}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6826","nonce":648258563},{"type":"v1","addr":"172.21.15.59:6827","nonce":648258563}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6830","nonce":648258563},{"type":"v1","addr":"172.21.15.59:6831","nonce":648258563}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6828","nonce":648258563},{"type":"v1","addr":"172.21.15.59:6829","nonce":648258563}]},"public_addr":"172.21.15.59:6825/648258563","cluster_addr":"172.21.15.59:6827/648258563","heartbeat_back_addr":"172.21.15.59:6831/648258563","heartbeat_front_addr":"172.21.15.59:6829/648258563","state":["exists","up"]},{"osd":2,"uuid":"69eebfa0-695b-4b21-90b9-8a3537168a01","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6808","nonce":1840430654},{"type":"v1","addr":"172.21.15.59:6809","nonce":1840430654}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6810","nonce":1840430654},{"type":"v1","addr":"172.21.15.59:6811","nonce":1840430654}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6814","nonce":1840430654},{"type":"v1","addr":"172.21.15.59:6815","nonce":1840430654}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6812","nonce":1840430654},{"type":"v1","addr":"172.21.15.59:6813","nonce":1840430654}]},"public_addr":"172.21.15.59:6809/1840430654","cluster_addr":"172.21.15.59:6811/1840430654","heartbeat_back_addr":"172.21.15.59:6815/1840430654","heartbeat_front_addr":"172.21.15.59:6813/1840430654","state":["exists","up"]},{"osd":3,"uuid":"96fd77fc-f911-46ff-9445-6f0be2daa8f4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6816","nonce":2504025850},{"type":"v1","addr":"172.21.15.59:6817","nonce":2504025850}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6818","nonce":2504025850},{"type":"v1","addr":"172.21.15.59:6819","nonce":2504025850}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6822","nonce":2504025850},{"type":"v1","addr":"172.21.15.59:6823","nonce":2504025850}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6820","nonce":2504025850},{"type":"v1","addr":"172.21.15.59:6821","nonce":2504025850}]},"public_addr":"172.21.15.59:6817/2504025850","cluster_addr":"172.21.15.59:6819/2504025850","heartbeat_back_addr":"172.21.15.59:6823/2504025850","heartbeat_front_addr":"172.21.15.59:6821/2504025850","state":["exists","up"]},{"osd":4,"uuid":"7ac29fca-b5b2-4c95-9d69-2f164cbbc708","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6816","nonce":2382463431},{"type":"v1","addr":"172.21.15.76:6818","nonce":2382463431}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6820","nonce":2382463431},{"type":"v1","addr":"172.21.15.76:6822","nonce":2382463431}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6828","nonce":2382463431},{"type":"v1","addr":"172.21.15.76:6830","nonce":2382463431}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6824","nonce":2382463431},{"type":"v1","addr":"172.21.15.76:6826","nonce":2382463431}]},"public_addr":"172.21.15.76:6818/2382463431","cluster_addr":"172.21.15.76:6822/2382463431","heartbeat_back_addr":"172.21.15.76:6830/2382463431","heartbeat_front_addr":"172.21.15.76:6826/2382463431","state":["exists","up"]},{"osd":5,"uuid":"485bf711-3851-4238-a4f4-51d52e0a9c58","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6800","nonce":329334308},{"type":"v1","addr":"172.21.15.76:6802","nonce":329334308}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6805","nonce":329334308},{"type":"v1","addr":"172.21.15.76:6807","nonce":329334308}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6814","nonce":329334308},{"type":"v1","addr":"172.21.15.76:6815","nonce":329334308}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6810","nonce":329334308},{"type":"v1","addr":"172.21.15.76:6812","nonce":329334308}]},"public_addr":"172.21.15.76:6802/329334308","cluster_addr":"172.21.15.76:6807/329334308","heartbeat_back_addr":"172.21.15.76:6815/329334308","heartbeat_front_addr":"172.21.15.76:6812/329334308","state":["exists","up"]},{"osd":6,"uuid":"72b3f2fc-1e4a-4e32-968d-003647a0286f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6801","nonce":3946579410},{"type":"v1","addr":"172.21.15.76:6803","nonce":3946579410}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6804","nonce":3946579410},{"type":"v1","addr":"172.21.15.76:6806","nonce":3946579410}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6811","nonce":3946579410},{"type":"v1","addr":"172.21.15.76:6813","nonce":3946579410}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6808","nonce":3946579410},{"type":"v1","addr":"172.21.15.76:6809","nonce":3946579410}]},"public_addr":"172.21.15.76:6803/3946579410","cluster_addr":"172.21.15.76:6806/3946579410","heartbeat_back_addr":"172.21.15.76:6813/3946579410","heartbeat_front_addr":"172.21.15.76:6809/3946579410","state":["exists","up"]},{"osd":7,"uuid":"1ddceb90-6c5d-4d1a-ad56-99cb877f8c70","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6817","nonce":841761538},{"type":"v1","addr":"172.21.15.76:6819","nonce":841761538}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6821","nonce":841761538},{"type":"v1","addr":"172.21.15.76:6823","nonce":841761538}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6829","nonce":841761538},{"type":"v1","addr":"172.21.15.76:6831","nonce":841761538}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6825","nonce":841761538},{"type":"v1","addr":"172.21.15.76:6827","nonce":841761538}]},"public_addr":"172.21.15.76:6819/841761538","cluster_addr":"172.21.15.76:6823/841761538","heartbeat_back_addr":"172.21.15.76:6831/841761538","heartbeat_front_addr":"172.21.15.76:6827/841761538","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T07:56:14.294838+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T07:56:14.270779+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T07:56:14.231809+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T07:56:14.260983+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T07:56:14.302997+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T07:56:14.258600+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T07:56:14.251413+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T07:56:14.304727+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.31","mappings":[{"from":0,"to":7}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-24T07:57:12.700 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:12.698+0000 7f535af33700 1 -- 172.21.15.59:0/1440647908 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f533806aed0 msgr2=0x7f533806d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:57:12.700 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:12.698+0000 7f535af33700 1 --2- 172.21.15.59:0/1440647908 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f533806aed0 0x7f533806d390 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f53540b3b10 tx=0x7f5340008040 comp rx=0 tx=0).stop 2023-11-24T07:57:12.700 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:12.698+0000 7f535af33700 1 -- 172.21.15.59:0/1440647908 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f535406b6e0 msgr2=0x7f535413ab70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:57:12.701 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:12.698+0000 7f535af33700 1 --2- 172.21.15.59:0/1440647908 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f535406b6e0 0x7f535413ab70 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f534800c520 tx=0x7f534800c830 comp rx=0 tx=0).stop 2023-11-24T07:57:12.701 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:12.698+0000 7f535af33700 1 -- 172.21.15.59:0/1440647908 shutdown_connections 2023-11-24T07:57:12.701 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:12.698+0000 7f535af33700 1 --2- 172.21.15.59:0/1440647908 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f533806aed0 0x7f533806d390 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:12.701 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:12.698+0000 7f535af33700 1 --2- 172.21.15.59:0/1440647908 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f5354145100 0x7f535413cf50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:12.701 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:12.698+0000 7f535af33700 1 --2- 172.21.15.59:0/1440647908 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f535406b6e0 0x7f535413ab70 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:12.701 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:12.698+0000 7f535af33700 1 --2- 172.21.15.59:0/1440647908 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5354144c10 0x7f53541f0e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:12.701 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:12.698+0000 7f535af33700 1 -- 172.21.15.59:0/1440647908 >> 172.21.15.59:0/1440647908 conn(0x7f53540b2740 msgr2=0x7f5354072510 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:57:12.701 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:12.698+0000 7f535af33700 1 -- 172.21.15.59:0/1440647908 shutdown_connections 2023-11-24T07:57:12.701 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:12.698+0000 7f535af33700 1 -- 172.21.15.59:0/1440647908 wait complete. 2023-11-24T07:57:12.712 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2023-11-24T07:57:12.712 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-24T07:57:12.839 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:12.837+0000 7f89fb034700 1 Processor -- start 2023-11-24T07:57:12.839 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:12.837+0000 7f89fb034700 1 -- start start 2023-11-24T07:57:12.842 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:12.838+0000 7f89fb034700 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f89f415bf10 0x7f89f4066ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:57:12.842 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:12.838+0000 7f89fb034700 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f89f406b7c0 0x7f89f406bc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:57:12.842 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:12.838+0000 7f89fb034700 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f89f40b10c0 con 0x7f89f40af980 2023-11-24T07:57:12.842 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:12.838+0000 7f89fb034700 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f89f406c190 con 0x7f89f415bf10 2023-11-24T07:57:12.843 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:12.838+0000 7f89fb034700 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f89f40af730 con 0x7f89f406b7c0 2023-11-24T07:57:12.843 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:12.838+0000 7f89f3fff700 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7f89f40af980 0x7f89f40b1550 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:54782/0 (socket says 172.21.15.59:54782) 2023-11-24T07:57:12.843 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:12.838+0000 7f89f3fff700 1 -- 172.21.15.59:0/4287651108 learned_addr learned my addr 172.21.15.59:0/4287651108 (peer_addr_for_me v1:172.21.15.59:0/0) 2023-11-24T07:57:12.843 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:12.838+0000 7f89f8dd0700 1 --2- 172.21.15.59:0/4287651108 >> v2:172.21.15.76:3300/0 conn(0x7f89f415bf10 0x7f89f4066ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:57:12.843 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:12.838+0000 7f89f95d1700 1 --2- 172.21.15.59:0/4287651108 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f89f406b7c0 0x7f89f406bc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:57:12.843 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:12.838+0000 7f89f37fe700 1 -- 172.21.15.59:0/4287651108 <== mon.0 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3032578378 0 0) 0x7f89f40b10c0 con 0x7f89f40af980 2023-11-24T07:57:12.843 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:12.838+0000 7f89f37fe700 1 -- 172.21.15.59:0/4287651108 --> v1:172.21.15.59:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f89e40035b0 con 0x7f89f40af980 2023-11-24T07:57:12.843 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:12.839+0000 7f89f37fe700 1 -- 172.21.15.59:0/4287651108 <== mon.0 v1:172.21.15.59:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2446530406 0 0) 0x7f89e40035b0 con 0x7f89f40af980 2023-11-24T07:57:12.843 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:12.839+0000 7f89f37fe700 1 -- 172.21.15.59:0/4287651108 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f89f406b7c0 msgr2=0x7f89f406bc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T07:57:12.843 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:12.839+0000 7f89f37fe700 1 --2- 172.21.15.59:0/4287651108 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f89f406b7c0 0x7f89f406bc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:12.843 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:12.839+0000 7f89f37fe700 1 -- 172.21.15.59:0/4287651108 >> v2:172.21.15.76:3300/0 conn(0x7f89f415bf10 msgr2=0x7f89f4066ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T07:57:12.844 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:12.839+0000 7f89f37fe700 1 --2- 172.21.15.59:0/4287651108 >> v2:172.21.15.76:3300/0 conn(0x7f89f415bf10 0x7f89f4066ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:12.844 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:12.839+0000 7f89f37fe700 1 -- 172.21.15.59:0/4287651108 --> v1:172.21.15.59:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f89f40b1c60 con 0x7f89f40af980 2023-11-24T07:57:12.844 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:12.839+0000 7f89f37fe700 1 -- 172.21.15.59:0/4287651108 <== mon.0 v1:172.21.15.59:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1312333566 0 0) 0x7f89e0001670 con 0x7f89f40af980 2023-11-24T07:57:12.844 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:12.839+0000 7f89f37fe700 1 -- 172.21.15.59:0/4287651108 >> v1:172.21.15.59:6789/0 conn(0x7f89f40af980 legacy=0x7f89f40b1550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:57:12.844 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:12.839+0000 7f89f37fe700 1 --2- 172.21.15.59:0/4287651108 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f89e4003a00 0x7f89e4003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:57:12.844 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:12.839+0000 7f89f37fe700 1 --2- 172.21.15.59:0/4287651108 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f89e4004850 0x7f89e4004cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:57:12.844 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:12.839+0000 7f89f37fe700 1 --2- 172.21.15.59:0/4287651108 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f89f406b7c0 0x7f89e4005210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:57:12.844 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:12.839+0000 7f89f37fe700 1 -- 172.21.15.59:0/4287651108 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89f406c190 con 0x7f89e4004850 2023-11-24T07:57:12.844 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:12.839+0000 7f89f37fe700 1 -- 172.21.15.59:0/4287651108 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89f40af730 con 0x7f89e4003a00 2023-11-24T07:57:12.845 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:12.839+0000 7f89f37fe700 1 -- 172.21.15.59:0/4287651108 --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f89e400ba10 con 0x7f89f406b7c0 2023-11-24T07:57:12.845 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:12.839+0000 7f89f3fff700 1 --2- 172.21.15.59:0/4287651108 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f89e4004850 0x7f89e4004cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:57:12.845 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:12.839+0000 7f89f8dd0700 1 --2- 172.21.15.59:0/4287651108 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f89e4003a00 0x7f89e4003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:57:12.845 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:12.839+0000 7f89f95d1700 1 --2- 172.21.15.59:0/4287651108 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f89f406b7c0 0x7f89e4005210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:57:12.845 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:12.839+0000 7f89f3fff700 1 -- 172.21.15.59:0/4287651108 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f89f406b7c0 msgr2=0x7f89e4005210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T07:57:12.845 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:12.839+0000 7f89f3fff700 1 --2- 172.21.15.59:0/4287651108 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f89f406b7c0 0x7f89e4005210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:12.845 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:12.839+0000 7f89f3fff700 1 -- 172.21.15.59:0/4287651108 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f89e4003a00 msgr2=0x7f89e4003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T07:57:12.845 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:12.839+0000 7f89f3fff700 1 --2- 172.21.15.59:0/4287651108 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f89e4003a00 0x7f89e4003e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:12.845 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:12.839+0000 7f89f3fff700 1 -- 172.21.15.59:0/4287651108 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f89e40035b0 con 0x7f89e4004850 2023-11-24T07:57:12.846 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:12.839+0000 7f89f8dd0700 1 --2- 172.21.15.59:0/4287651108 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f89e4003a00 0x7f89e4003e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-24T07:57:12.846 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:12.839+0000 7f89f3fff700 1 --2- 172.21.15.59:0/4287651108 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f89e4004850 0x7f89e4004cd0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f89e000c190 tx=0x7f89e4005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e256d0cb59f3f5fb server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:57:12.846 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:12.839+0000 7f89f37fe700 1 -- 172.21.15.59:0/4287651108 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89e0007b70 con 0x7f89e4004850 2023-11-24T07:57:12.846 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:12.839+0000 7f89f37fe700 1 -- 172.21.15.59:0/4287651108 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f89e000ccd0 con 0x7f89e4004850 2023-11-24T07:57:12.846 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:12.839+0000 7f89f37fe700 1 -- 172.21.15.59:0/4287651108 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89e0005ba0 con 0x7f89e4004850 2023-11-24T07:57:12.846 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:12.840+0000 7f89fb034700 1 -- 172.21.15.59:0/4287651108 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f89e4004850 msgr2=0x7f89e4004cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:57:12.846 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:12.840+0000 7f89fb034700 1 --2- 172.21.15.59:0/4287651108 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f89e4004850 0x7f89e4004cd0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f89e000c190 tx=0x7f89e4005880 comp rx=0 tx=0).stop 2023-11-24T07:57:12.846 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:12.840+0000 7f89f95d1700 1 -- 172.21.15.59:0/4287651108 reap_dead start 2023-11-24T07:57:12.846 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:12.840+0000 7f89fb034700 1 -- 172.21.15.59:0/4287651108 shutdown_connections 2023-11-24T07:57:12.846 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:12.840+0000 7f89fb034700 1 -- 172.21.15.59:0/4287651108 >> 172.21.15.59:0/4287651108 conn(0x7f89f40b5070 msgr2=0x7f89f40b5730 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:57:12.847 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:12.840+0000 7f89fb034700 1 -- 172.21.15.59:0/4287651108 shutdown_connections 2023-11-24T07:57:12.847 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:12.840+0000 7f89fb034700 1 -- 172.21.15.59:0/4287651108 wait complete. 2023-11-24T07:57:12.849 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:12.841+0000 7f89fb034700 1 Processor -- start 2023-11-24T07:57:12.849 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:12.841+0000 7f89fb034700 1 -- start start 2023-11-24T07:57:12.850 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:12.842+0000 7f89fb034700 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f89e4004850 0x7f89f414c8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:57:12.850 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:12.842+0000 7f89fb034700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f89f41f58f0 0x7f89f415e410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:57:12.850 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:12.842+0000 7f89fb034700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f89f41582b0 0x7f89f4158730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:57:12.850 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:12.842+0000 7f89fb034700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89f4158c70 con 0x7f89f41582b0 2023-11-24T07:57:12.850 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:12.842+0000 7f89fb034700 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89f415f040 con 0x7f89f41f58f0 2023-11-24T07:57:12.850 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:12.842+0000 7f89fb034700 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f89f415ed60 con 0x7f89e4004850 2023-11-24T07:57:12.850 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:12.842+0000 7f89f95d1700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f89f41582b0 0x7f89f4158730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:57:12.850 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:12.842+0000 7f89f95d1700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f89f41582b0 0x7f89f4158730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:51324/0 (socket says 172.21.15.59:51324) 2023-11-24T07:57:12.850 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:12.842+0000 7f89f95d1700 1 -- 172.21.15.59:0/3502532850 learned_addr learned my addr 172.21.15.59:0/3502532850 (peer_addr_for_me v2:172.21.15.59:0/0) 2023-11-24T07:57:12.850 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:12.843+0000 7f89f8dd0700 1 --2- 172.21.15.59:0/3502532850 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f89e4004850 0x7f89f414c8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:57:12.850 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:12.843+0000 7f89f95d1700 1 -- 172.21.15.59:0/3502532850 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f89e4004850 msgr2=0x7f89f414c8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:57:12.851 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:12.843+0000 7f89f95d1700 1 --2- 172.21.15.59:0/3502532850 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f89e4004850 0x7f89f414c8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:12.851 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:12.843+0000 7f89f3fff700 1 --2- 172.21.15.59:0/3502532850 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f89f41f58f0 0x7f89f415e410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:57:12.851 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:12.843+0000 7f89f95d1700 1 -- 172.21.15.59:0/3502532850 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f89f41f58f0 msgr2=0x7f89f415e410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:57:12.851 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:12.843+0000 7f89f95d1700 1 --2- 172.21.15.59:0/3502532850 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f89f41f58f0 0x7f89f415e410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:12.851 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:12.843+0000 7f89f95d1700 1 -- 172.21.15.59:0/3502532850 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f89f414f490 con 0x7f89f41582b0 2023-11-24T07:57:12.851 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:12.843+0000 7f89f95d1700 1 --2- 172.21.15.59:0/3502532850 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f89f41582b0 0x7f89f4158730 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f89f406cc60 tx=0x7f89ec011f80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:57:12.851 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:12.843+0000 7f89f17fa700 1 -- 172.21.15.59:0/3502532850 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89ec00e040 con 0x7f89f41582b0 2023-11-24T07:57:12.851 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:12.843+0000 7f89f17fa700 1 -- 172.21.15.59:0/3502532850 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f89ec00f040 con 0x7f89f41582b0 2023-11-24T07:57:12.851 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:12.843+0000 7f89fb034700 1 -- 172.21.15.59:0/3502532850 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f89f414f770 con 0x7f89f41582b0 2023-11-24T07:57:12.851 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:12.843+0000 7f89f17fa700 1 -- 172.21.15.59:0/3502532850 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89ec00fd80 con 0x7f89f41582b0 2023-11-24T07:57:12.851 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:12.843+0000 7f89fb034700 1 -- 172.21.15.59:0/3502532850 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f89f414cdf0 con 0x7f89f41582b0 2023-11-24T07:57:12.852 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:12.844+0000 7f89fb034700 1 -- 172.21.15.59:0/3502532850 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f89f4068670 con 0x7f89f41582b0 2023-11-24T07:57:12.852 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:12.844+0000 7f89f17fa700 1 -- 172.21.15.59:0/3502532850 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f89ec015430 con 0x7f89f41582b0 2023-11-24T07:57:12.852 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:12.845+0000 7f89f17fa700 1 --2- 172.21.15.59:0/3502532850 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f89d806aed0 0x7f89d806d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:57:12.852 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:12.845+0000 7f89f17fa700 1 -- 172.21.15.59:0/3502532850 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5738+0+0 (secure 0 0 0) 0x7f89ec089bc0 con 0x7f89f41582b0 2023-11-24T07:57:12.852 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:12.845+0000 7f89f8dd0700 1 --2- 172.21.15.59:0/3502532850 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f89d806aed0 0x7f89d806d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:57:12.852 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:12.845+0000 7f89f8dd0700 1 --2- 172.21.15.59:0/3502532850 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f89d806aed0 0x7f89d806d390 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f89e8009ba0 tx=0x7f89e8008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:57:12.852 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:12.849+0000 7f89f17fa700 1 -- 172.21.15.59:0/3502532850 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f89ec054ce0 con 0x7f89f41582b0 2023-11-24T07:57:13.098 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:13.096+0000 7f89fb034700 1 -- 172.21.15.59:0/3502532850 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f89f414fc90 con 0x7f89f41582b0 2023-11-24T07:57:13.098 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:13.097+0000 7f89f17fa700 1 -- 172.21.15.59:0/3502532850 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) v1 ==== 93+0+4904 (secure 0 0 0) 0x7f89f414fc90 con 0x7f89f41582b0 2023-11-24T07:57:13.099 INFO:teuthology.orchestra.run.smithi059.stdout: 2023-11-24T07:57:13.099 INFO:teuthology.orchestra.run.smithi059.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4220,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6835/838038775","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6833","nonce":838038775},{"type":"v1","addr":"172.21.15.59:6835","nonce":838038775}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4232,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.76:6837/3226690004","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6835","nonce":3226690004},{"type":"v1","addr":"172.21.15.76:6837","nonce":3226690004}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4260,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6834/2986456006","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6832","nonce":2986456006},{"type":"v1","addr":"172.21.15.59:6834","nonce":2986456006}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-24T07:56:24.104295+0000","modified":"2023-11-24T07:56:27.152806+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":4264},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4264":{"gid":4264,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.76:6836/3993077463","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6834","nonce":3993077463},{"type":"v1","addr":"172.21.15.76:6836","nonce":3993077463}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-11-24T07:57:13.101 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:13.099+0000 7f89fb034700 1 -- 172.21.15.59:0/3502532850 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f89d806aed0 msgr2=0x7f89d806d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:57:13.101 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:13.099+0000 7f89fb034700 1 --2- 172.21.15.59:0/3502532850 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f89d806aed0 0x7f89d806d390 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f89e8009ba0 tx=0x7f89e8008040 comp rx=0 tx=0).stop 2023-11-24T07:57:13.101 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:13.099+0000 7f89fb034700 1 -- 172.21.15.59:0/3502532850 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f89f41582b0 msgr2=0x7f89f4158730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:57:13.101 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:13.099+0000 7f89fb034700 1 --2- 172.21.15.59:0/3502532850 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f89f41582b0 0x7f89f4158730 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f89f406cc60 tx=0x7f89ec011f80 comp rx=0 tx=0).stop 2023-11-24T07:57:13.101 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:13.099+0000 7f89fb034700 1 -- 172.21.15.59:0/3502532850 shutdown_connections 2023-11-24T07:57:13.102 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:13.099+0000 7f89fb034700 1 --2- 172.21.15.59:0/3502532850 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f89d806aed0 0x7f89d806d390 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:13.102 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:13.099+0000 7f89fb034700 1 --2- 172.21.15.59:0/3502532850 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f89f41582b0 0x7f89f4158730 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:13.102 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:13.099+0000 7f89fb034700 1 --2- 172.21.15.59:0/3502532850 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f89e4004850 0x7f89f414c8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:13.102 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:13.099+0000 7f89fb034700 1 --2- 172.21.15.59:0/3502532850 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f89f41f58f0 0x7f89f415e410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:13.102 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:13.099+0000 7f89fb034700 1 -- 172.21.15.59:0/3502532850 >> 172.21.15.59:0/3502532850 conn(0x7f89f40b5070 msgr2=0x7f89f4159e30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:57:13.102 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:13.099+0000 7f89fb034700 1 -- 172.21.15.59:0/3502532850 shutdown_connections 2023-11-24T07:57:13.102 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:13.099+0000 7f89fb034700 1 -- 172.21.15.59:0/3502532850 wait complete. 2023-11-24T07:57:13.103 INFO:teuthology.orchestra.run.smithi059.stderr:dumped fsmap epoch 8 2023-11-24T07:57:13.113 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-24T07:57:13.239 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:13.237+0000 7f9e857b4700 1 Processor -- start 2023-11-24T07:57:13.239 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:13.237+0000 7f9e857b4700 1 -- start start 2023-11-24T07:57:13.241 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:13.238+0000 7f9e857b4700 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f9e800d49d0 0x7f9e800d2490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:57:13.241 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:13.238+0000 7f9e857b4700 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f9e800d29d0 0x7f9e800b40a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:57:13.241 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:13.238+0000 7f9e857b4700 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9e800d4e50 con 0x7f9e8015e460 2023-11-24T07:57:13.241 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:13.238+0000 7f9e857b4700 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9e800d2e60 con 0x7f9e800d29d0 2023-11-24T07:57:13.241 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:13.238+0000 7f9e857b4700 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9e800b3370 con 0x7f9e800d49d0 2023-11-24T07:57:13.242 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:13.238+0000 7f9e7effd700 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7f9e8015e460 0x7f9e800cf480 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:54802/0 (socket says 172.21.15.59:54802) 2023-11-24T07:57:13.242 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:13.238+0000 7f9e7effd700 1 -- 172.21.15.59:0/3542065397 learned_addr learned my addr 172.21.15.59:0/3542065397 (peer_addr_for_me v1:172.21.15.59:0/0) 2023-11-24T07:57:13.242 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:13.238+0000 7f9e7e7fc700 1 --2- 172.21.15.59:0/3542065397 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f9e800d49d0 0x7f9e800d2490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:57:13.242 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:13.238+0000 7f9e7f7fe700 1 --2- 172.21.15.59:0/3542065397 >> v2:172.21.15.76:3300/0 conn(0x7f9e800d29d0 0x7f9e800b40a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:57:13.242 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:13.238+0000 7f9e7dffb700 1 -- 172.21.15.59:0/3542065397 <== mon.0 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3940104730 0 0) 0x7f9e800d4e50 con 0x7f9e8015e460 2023-11-24T07:57:13.242 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:13.238+0000 7f9e7dffb700 1 -- 172.21.15.59:0/3542065397 --> v1:172.21.15.59:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9e680035b0 con 0x7f9e8015e460 2023-11-24T07:57:13.242 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:13.238+0000 7f9e7e7fc700 1 -- 172.21.15.59:0/3542065397 >> v2:172.21.15.76:3300/0 conn(0x7f9e800d29d0 msgr2=0x7f9e800b40a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T07:57:13.245 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:13.238+0000 7f9e7dffb700 1 -- 172.21.15.59:0/3542065397 <== mon.0 v1:172.21.15.59:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1487339018 0 0) 0x7f9e680035b0 con 0x7f9e8015e460 2023-11-24T07:57:13.245 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:13.238+0000 7f9e7e7fc700 1 --2- 172.21.15.59:0/3542065397 >> v2:172.21.15.76:3300/0 conn(0x7f9e800d29d0 0x7f9e800b40a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:13.245 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:13.238+0000 7f9e7e7fc700 1 -- 172.21.15.59:0/3542065397 >> v1:172.21.15.59:6789/0 conn(0x7f9e8015e460 legacy=0x7f9e800cf480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:57:13.245 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:13.238+0000 7f9e7e7fc700 1 -- 172.21.15.59:0/3542065397 --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9e800cfc40 con 0x7f9e800d49d0 2023-11-24T07:57:13.245 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:13.239+0000 7f9e7e7fc700 1 --2- 172.21.15.59:0/3542065397 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f9e800d49d0 0x7f9e800d2490 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f9e70009840 tx=0x7f9e70009b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=736d70175e002bf0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:57:13.246 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:13.239+0000 7f9e7dffb700 1 -- 172.21.15.59:0/3542065397 <== mon.2 v2:172.21.15.76:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e74001670 con 0x7f9e800d49d0 2023-11-24T07:57:13.246 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:13.239+0000 7f9e7dffb700 1 -- 172.21.15.59:0/3542065397 <== mon.2 v2:172.21.15.76:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9e70004030 con 0x7f9e800d49d0 2023-11-24T07:57:13.246 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:13.239+0000 7f9e7dffb700 1 -- 172.21.15.59:0/3542065397 <== mon.2 v2:172.21.15.76:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e700043c0 con 0x7f9e800d49d0 2023-11-24T07:57:13.246 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:13.239+0000 7f9e857b4700 1 -- 172.21.15.59:0/3542065397 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f9e800d49d0 msgr2=0x7f9e800d2490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:57:13.246 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:13.239+0000 7f9e857b4700 1 --2- 172.21.15.59:0/3542065397 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f9e800d49d0 0x7f9e800d2490 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f9e70009840 tx=0x7f9e70009b50 comp rx=0 tx=0).stop 2023-11-24T07:57:13.246 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:13.239+0000 7f9e857b4700 1 -- 172.21.15.59:0/3542065397 shutdown_connections 2023-11-24T07:57:13.247 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:13.239+0000 7f9e857b4700 1 --2- 172.21.15.59:0/3542065397 >> v2:172.21.15.76:3300/0 conn(0x7f9e800d29d0 0x7f9e800b40a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:13.247 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:13.239+0000 7f9e857b4700 1 --2- 172.21.15.59:0/3542065397 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f9e800d49d0 0x7f9e800d2490 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:13.247 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:13.239+0000 7f9e857b4700 1 -- 172.21.15.59:0/3542065397 >> 172.21.15.59:0/3542065397 conn(0x7f9e800d3eb0 msgr2=0x7f9e800d42c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:57:13.247 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:13.239+0000 7f9e857b4700 1 -- 172.21.15.59:0/3542065397 shutdown_connections 2023-11-24T07:57:13.247 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:13.239+0000 7f9e857b4700 1 -- 172.21.15.59:0/3542065397 wait complete. 2023-11-24T07:57:13.250 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:13.241+0000 7f9e857b4700 1 Processor -- start 2023-11-24T07:57:13.250 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:13.241+0000 7f9e857b4700 1 -- start start 2023-11-24T07:57:13.250 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:13.241+0000 7f9e857b4700 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f9e800d29d0 0x7f9e8015b550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:57:13.250 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:13.241+0000 7f9e857b4700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9e800d49d0 0x7f9e800b6a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:57:13.250 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:13.241+0000 7f9e857b4700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9e8015e460 0x7f9e800b0ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:57:13.250 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:13.241+0000 7f9e857b4700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e800b7360 con 0x7f9e800d49d0 2023-11-24T07:57:13.250 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:13.241+0000 7f9e857b4700 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e800b7640 con 0x7f9e8015e460 2023-11-24T07:57:13.250 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:13.241+0000 7f9e857b4700 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9e800b7920 con 0x7f9e800d29d0 2023-11-24T07:57:13.250 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:13.242+0000 7f9e7e7fc700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9e800d49d0 0x7f9e800b6a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:57:13.250 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:13.242+0000 7f9e7e7fc700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9e800d49d0 0x7f9e800b6a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:51336/0 (socket says 172.21.15.59:51336) 2023-11-24T07:57:13.251 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:13.242+0000 7f9e7e7fc700 1 -- 172.21.15.59:0/2677538511 learned_addr learned my addr 172.21.15.59:0/2677538511 (peer_addr_for_me v2:172.21.15.59:0/0) 2023-11-24T07:57:13.251 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:13.242+0000 7f9e7f7fe700 1 --2- 172.21.15.59:0/2677538511 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9e8015e460 0x7f9e800b0ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:57:13.251 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:13.242+0000 7f9e7effd700 1 --2- 172.21.15.59:0/2677538511 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f9e800d29d0 0x7f9e8015b550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:57:13.251 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:13.242+0000 7f9e7e7fc700 1 -- 172.21.15.59:0/2677538511 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f9e800d29d0 msgr2=0x7f9e8015b550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:57:13.251 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:13.242+0000 7f9e7e7fc700 1 --2- 172.21.15.59:0/2677538511 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f9e800d29d0 0x7f9e8015b550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:13.251 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:13.242+0000 7f9e7e7fc700 1 -- 172.21.15.59:0/2677538511 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9e8015e460 msgr2=0x7f9e800b0ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:57:13.251 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:13.242+0000 7f9e7e7fc700 1 --2- 172.21.15.59:0/2677538511 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9e8015e460 0x7f9e800b0ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:13.251 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:13.242+0000 7f9e7e7fc700 1 -- 172.21.15.59:0/2677538511 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9e700094a0 con 0x7f9e800d49d0 2023-11-24T07:57:13.251 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:13.242+0000 7f9e7f7fe700 1 --2- 172.21.15.59:0/2677538511 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9e8015e460 0x7f9e800b0ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T07:57:13.251 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:13.242+0000 7f9e7e7fc700 1 --2- 172.21.15.59:0/2677538511 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9e800d49d0 0x7f9e800b6a60 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f9e7000a010 tx=0x7f9e70004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:57:13.251 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:13.242+0000 7f9e677fe700 1 -- 172.21.15.59:0/2677538511 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e70003e50 con 0x7f9e800d49d0 2023-11-24T07:57:13.251 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:13.242+0000 7f9e677fe700 1 -- 172.21.15.59:0/2677538511 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9e7001f070 con 0x7f9e800d49d0 2023-11-24T07:57:13.251 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:13.242+0000 7f9e677fe700 1 -- 172.21.15.59:0/2677538511 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e70011870 con 0x7f9e800d49d0 2023-11-24T07:57:13.252 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:13.242+0000 7f9e857b4700 1 -- 172.21.15.59:0/2677538511 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9e800b7c00 con 0x7f9e800d49d0 2023-11-24T07:57:13.252 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:13.242+0000 7f9e857b4700 1 -- 172.21.15.59:0/2677538511 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9e800b7ee0 con 0x7f9e800d49d0 2023-11-24T07:57:13.252 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:13.243+0000 7f9e857b4700 1 -- 172.21.15.59:0/2677538511 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9e800b8070 con 0x7f9e800d49d0 2023-11-24T07:57:13.252 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:13.244+0000 7f9e677fe700 1 -- 172.21.15.59:0/2677538511 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f9e700119d0 con 0x7f9e800d49d0 2023-11-24T07:57:13.252 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:13.244+0000 7f9e677fe700 1 --2- 172.21.15.59:0/2677538511 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f9e6c06aed0 0x7f9e6c06d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:57:13.252 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:13.244+0000 7f9e677fe700 1 -- 172.21.15.59:0/2677538511 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5738+0+0 (secure 0 0 0) 0x7f9e70021070 con 0x7f9e800d49d0 2023-11-24T07:57:13.252 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:13.244+0000 7f9e7effd700 1 --2- 172.21.15.59:0/2677538511 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f9e6c06aed0 0x7f9e6c06d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:57:13.252 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:13.245+0000 7f9e7effd700 1 --2- 172.21.15.59:0/2677538511 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f9e6c06aed0 0x7f9e6c06d390 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f9e740011e0 tx=0x7f9e74009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:57:13.252 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:13.249+0000 7f9e677fe700 1 -- 172.21.15.59:0/2677538511 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f9e70055d90 con 0x7f9e800d49d0 2023-11-24T07:57:13.499 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:13.497+0000 7f9e857b4700 1 -- 172.21.15.59:0/2677538511 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f9e80067f60 con 0x7f9e800d49d0 2023-11-24T07:57:13.500 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:13.498+0000 7f9e677fe700 1 -- 172.21.15.59:0/2677538511 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) v1 ==== 93+0+4904 (secure 0 0 0) 0x7f9e700593b0 con 0x7f9e800d49d0 2023-11-24T07:57:13.500 INFO:teuthology.orchestra.run.smithi059.stdout: 2023-11-24T07:57:13.500 INFO:teuthology.orchestra.run.smithi059.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4220,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6835/838038775","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6833","nonce":838038775},{"type":"v1","addr":"172.21.15.59:6835","nonce":838038775}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4232,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.76:6837/3226690004","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6835","nonce":3226690004},{"type":"v1","addr":"172.21.15.76:6837","nonce":3226690004}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4260,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6834/2986456006","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6832","nonce":2986456006},{"type":"v1","addr":"172.21.15.59:6834","nonce":2986456006}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-24T07:56:24.104295+0000","modified":"2023-11-24T07:56:27.152806+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":4264},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4264":{"gid":4264,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.76:6836/3993077463","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6834","nonce":3993077463},{"type":"v1","addr":"172.21.15.76:6836","nonce":3993077463}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-11-24T07:57:13.502 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:13.500+0000 7f9e857b4700 1 -- 172.21.15.59:0/2677538511 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f9e6c06aed0 msgr2=0x7f9e6c06d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:57:13.502 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:13.500+0000 7f9e857b4700 1 --2- 172.21.15.59:0/2677538511 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f9e6c06aed0 0x7f9e6c06d390 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f9e740011e0 tx=0x7f9e74009040 comp rx=0 tx=0).stop 2023-11-24T07:57:13.502 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:13.500+0000 7f9e857b4700 1 -- 172.21.15.59:0/2677538511 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9e800d49d0 msgr2=0x7f9e800b6a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:57:13.503 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:13.500+0000 7f9e857b4700 1 --2- 172.21.15.59:0/2677538511 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9e800d49d0 0x7f9e800b6a60 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f9e7000a010 tx=0x7f9e70004750 comp rx=0 tx=0).stop 2023-11-24T07:57:13.503 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:13.500+0000 7f9e857b4700 1 -- 172.21.15.59:0/2677538511 shutdown_connections 2023-11-24T07:57:13.503 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:13.500+0000 7f9e857b4700 1 --2- 172.21.15.59:0/2677538511 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f9e6c06aed0 0x7f9e6c06d390 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:13.503 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:13.500+0000 7f9e857b4700 1 --2- 172.21.15.59:0/2677538511 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9e8015e460 0x7f9e800b0ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:13.503 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:13.500+0000 7f9e857b4700 1 --2- 172.21.15.59:0/2677538511 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f9e800d29d0 0x7f9e8015b550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:13.503 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:13.500+0000 7f9e857b4700 1 --2- 172.21.15.59:0/2677538511 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9e800d49d0 0x7f9e800b6a60 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:13.503 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:13.500+0000 7f9e857b4700 1 -- 172.21.15.59:0/2677538511 >> 172.21.15.59:0/2677538511 conn(0x7f9e800d3eb0 msgr2=0x7f9e800d4290 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:57:13.503 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:13.500+0000 7f9e857b4700 1 -- 172.21.15.59:0/2677538511 shutdown_connections 2023-11-24T07:57:13.503 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:13.500+0000 7f9e857b4700 1 -- 172.21.15.59:0/2677538511 wait complete. 2023-11-24T07:57:13.504 INFO:teuthology.orchestra.run.smithi059.stderr:dumped fsmap epoch 8 2023-11-24T07:57:13.514 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-24T07:57:13.640 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:13.638+0000 7f99b1daa700 1 Processor -- start 2023-11-24T07:57:13.640 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:13.638+0000 7f99b1daa700 1 -- start start 2023-11-24T07:57:13.642 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:13.639+0000 7f99b1daa700 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f99ac1457a0 0x7f99ac0cff70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:57:13.642 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:13.639+0000 7f99b1daa700 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f99ac0cd470 0x7f99ac160330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:57:13.642 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:13.639+0000 7f99b1daa700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f99ac06afd0 0x7f99ac06b400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:57:13.643 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:13.639+0000 7f99b1daa700 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f99ac0d04b0 con 0x7f99ac06afd0 2023-11-24T07:57:13.643 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:13.639+0000 7f99b1daa700 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f99ac06b940 con 0x7f99ac1457a0 2023-11-24T07:57:13.643 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:13.639+0000 7f99b1daa700 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f99ac160870 con 0x7f99ac0cd470 2023-11-24T07:57:13.643 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:13.639+0000 7f99abfff700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f99ac06afd0 0x7f99ac06b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:57:13.643 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:13.639+0000 7f99abfff700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f99ac06afd0 0x7f99ac06b400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:51346/0 (socket says 172.21.15.59:51346) 2023-11-24T07:57:13.643 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:13.639+0000 7f99abfff700 1 -- 172.21.15.59:0/2403064618 learned_addr learned my addr 172.21.15.59:0/2403064618 (peer_addr_for_me v2:172.21.15.59:0/0) 2023-11-24T07:57:13.643 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:13.639+0000 7f99ab7fe700 1 --2- 172.21.15.59:0/2403064618 >> v2:172.21.15.76:3300/0 conn(0x7f99ac1457a0 0x7f99ac0cff70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:57:13.643 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:13.639+0000 7f99aaffd700 1 --2- 172.21.15.59:0/2403064618 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f99ac0cd470 0x7f99ac160330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:57:13.643 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:13.639+0000 7f99abfff700 1 -- 172.21.15.59:0/2403064618 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f99ac0cd470 msgr2=0x7f99ac160330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T07:57:13.644 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:13.639+0000 7f99abfff700 1 --2- 172.21.15.59:0/2403064618 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f99ac0cd470 0x7f99ac160330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:13.644 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:13.639+0000 7f99abfff700 1 -- 172.21.15.59:0/2403064618 >> v2:172.21.15.76:3300/0 conn(0x7f99ac1457a0 msgr2=0x7f99ac0cff70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T07:57:13.644 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:13.639+0000 7f99abfff700 1 --2- 172.21.15.59:0/2403064618 >> v2:172.21.15.76:3300/0 conn(0x7f99ac1457a0 0x7f99ac0cff70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:13.644 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:13.639+0000 7f99abfff700 1 -- 172.21.15.59:0/2403064618 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f99ac0cf1b0 con 0x7f99ac06afd0 2023-11-24T07:57:13.644 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:13.639+0000 7f99abfff700 1 --2- 172.21.15.59:0/2403064618 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f99ac06afd0 0x7f99ac06b400 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f99ac0b5ea0 tx=0x7f99a400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=42b8ad3dca09d750 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:57:13.644 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:13.639+0000 7f99aa7fc700 1 -- 172.21.15.59:0/2403064618 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99a400e070 con 0x7f99ac06afd0 2023-11-24T07:57:13.644 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:13.639+0000 7f99aa7fc700 1 -- 172.21.15.59:0/2403064618 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f99a400b9c0 con 0x7f99ac06afd0 2023-11-24T07:57:13.644 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:13.639+0000 7f99aa7fc700 1 -- 172.21.15.59:0/2403064618 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99a400bd70 con 0x7f99ac06afd0 2023-11-24T07:57:13.644 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:13.640+0000 7f99b1daa700 1 -- 172.21.15.59:0/2403064618 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f99ac06afd0 msgr2=0x7f99ac06b400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:57:13.644 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:13.640+0000 7f99b1daa700 1 --2- 172.21.15.59:0/2403064618 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f99ac06afd0 0x7f99ac06b400 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f99ac0b5ea0 tx=0x7f99a400b330 comp rx=0 tx=0).stop 2023-11-24T07:57:13.644 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:13.640+0000 7f99b1daa700 1 -- 172.21.15.59:0/2403064618 shutdown_connections 2023-11-24T07:57:13.644 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:13.640+0000 7f99b1daa700 1 --2- 172.21.15.59:0/2403064618 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f99ac06afd0 0x7f99ac06b400 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:13.644 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:13.640+0000 7f99b1daa700 1 --2- 172.21.15.59:0/2403064618 >> v2:172.21.15.76:3300/0 conn(0x7f99ac1457a0 0x7f99ac0cff70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:13.645 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:13.640+0000 7f99b1daa700 1 --2- 172.21.15.59:0/2403064618 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f99ac0cd470 0x7f99ac160330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:13.645 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:13.640+0000 7f99b1daa700 1 -- 172.21.15.59:0/2403064618 >> 172.21.15.59:0/2403064618 conn(0x7f99ac0d0aa0 msgr2=0x7f99ac0d0eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:57:13.645 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:13.640+0000 7f99b1daa700 1 -- 172.21.15.59:0/2403064618 shutdown_connections 2023-11-24T07:57:13.645 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:13.640+0000 7f99b1daa700 1 -- 172.21.15.59:0/2403064618 wait complete. 2023-11-24T07:57:13.650 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:13.641+0000 7f99b1daa700 1 Processor -- start 2023-11-24T07:57:13.650 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:13.641+0000 7f99b1daa700 1 -- start start 2023-11-24T07:57:13.650 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:13.642+0000 7f99b1daa700 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f99ac06afd0 0x7f99ac1432a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:57:13.650 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:13.642+0000 7f99b1daa700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f99ac0cd470 0x7f99ac144ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:57:13.650 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:13.642+0000 7f99b1daa700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f99ac1457a0 0x7f99ac157080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:57:13.650 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:13.642+0000 7f99b1daa700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99ac13bbb0 con 0x7f99ac0cd470 2023-11-24T07:57:13.651 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:13.642+0000 7f99b1daa700 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99ac1437e0 con 0x7f99ac1457a0 2023-11-24T07:57:13.651 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:13.642+0000 7f99b1daa700 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f99ac144620 con 0x7f99ac06afd0 2023-11-24T07:57:13.651 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:13.642+0000 7f99aaffd700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f99ac0cd470 0x7f99ac144ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:57:13.651 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:13.642+0000 7f99aaffd700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f99ac0cd470 0x7f99ac144ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:51350/0 (socket says 172.21.15.59:51350) 2023-11-24T07:57:13.651 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:13.642+0000 7f99aaffd700 1 -- 172.21.15.59:0/919474759 learned_addr learned my addr 172.21.15.59:0/919474759 (peer_addr_for_me v2:172.21.15.59:0/0) 2023-11-24T07:57:13.651 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:13.643+0000 7f99abfff700 1 --2- 172.21.15.59:0/919474759 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f99ac1457a0 0x7f99ac157080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:57:13.651 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:13.643+0000 7f99ab7fe700 1 --2- 172.21.15.59:0/919474759 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f99ac06afd0 0x7f99ac1432a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:57:13.652 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:13.643+0000 7f99aaffd700 1 -- 172.21.15.59:0/919474759 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f99ac06afd0 msgr2=0x7f99ac1432a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:57:13.652 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:13.643+0000 7f99aaffd700 1 --2- 172.21.15.59:0/919474759 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f99ac06afd0 0x7f99ac1432a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:13.652 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:13.643+0000 7f99aaffd700 1 -- 172.21.15.59:0/919474759 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f99ac1457a0 msgr2=0x7f99ac157080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:57:13.652 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:13.643+0000 7f99aaffd700 1 --2- 172.21.15.59:0/919474759 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f99ac1457a0 0x7f99ac157080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:13.652 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:13.643+0000 7f99aaffd700 1 -- 172.21.15.59:0/919474759 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f99a4009b30 con 0x7f99ac0cd470 2023-11-24T07:57:13.652 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:13.643+0000 7f99abfff700 1 --2- 172.21.15.59:0/919474759 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f99ac1457a0 0x7f99ac157080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-24T07:57:13.652 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:13.643+0000 7f99aaffd700 1 --2- 172.21.15.59:0/919474759 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f99ac0cd470 0x7f99ac144ab0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f99a000d800 tx=0x7f99a000dbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:57:13.652 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:13.643+0000 7f9993fff700 1 -- 172.21.15.59:0/919474759 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99a00125f0 con 0x7f99ac0cd470 2023-11-24T07:57:13.653 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:13.643+0000 7f9993fff700 1 -- 172.21.15.59:0/919474759 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f99a0012750 con 0x7f99ac0cd470 2023-11-24T07:57:13.653 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:13.643+0000 7f9993fff700 1 -- 172.21.15.59:0/919474759 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99a0012ae0 con 0x7f99ac0cd470 2023-11-24T07:57:13.653 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:13.643+0000 7f99b1daa700 1 -- 172.21.15.59:0/919474759 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f99ac144ff0 con 0x7f99ac0cd470 2023-11-24T07:57:13.653 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:13.643+0000 7f99b1daa700 1 -- 172.21.15.59:0/919474759 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f99ac143cf0 con 0x7f99ac0cd470 2023-11-24T07:57:13.653 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:13.644+0000 7f99b1daa700 1 -- 172.21.15.59:0/919474759 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f99ac144790 con 0x7f99ac0cd470 2023-11-24T07:57:13.653 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:13.644+0000 7f9993fff700 1 -- 172.21.15.59:0/919474759 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f99a0012c40 con 0x7f99ac0cd470 2023-11-24T07:57:13.653 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:13.645+0000 7f9993fff700 1 --2- 172.21.15.59:0/919474759 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f999806aed0 0x7f999806d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:57:13.653 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:13.645+0000 7f9993fff700 1 -- 172.21.15.59:0/919474759 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5738+0+0 (secure 0 0 0) 0x7f99a0089880 con 0x7f99ac0cd470 2023-11-24T07:57:13.653 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:13.645+0000 7f99ab7fe700 1 --2- 172.21.15.59:0/919474759 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f999806aed0 0x7f999806d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:57:13.653 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:13.646+0000 7f99ab7fe700 1 --2- 172.21.15.59:0/919474759 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f999806aed0 0x7f999806d390 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f999c009930 tx=0x7f999c008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:57:13.655 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:13.649+0000 7f9993fff700 1 -- 172.21.15.59:0/919474759 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f99a00549a0 con 0x7f99ac0cd470 2023-11-24T07:57:13.852 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:13.850+0000 7f99b1daa700 1 -- 172.21.15.59:0/919474759 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f99ac068670 con 0x7f99ac0cd470 2023-11-24T07:57:13.853 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:13.851+0000 7f9993fff700 1 -- 172.21.15.59:0/919474759 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v22) v1 ==== 74+0+16486 (secure 0 0 0) 0x7f99a0057fc0 con 0x7f99ac0cd470 2023-11-24T07:57:13.853 INFO:teuthology.orchestra.run.smithi059.stdout: 2023-11-24T07:57:13.853 INFO:teuthology.orchestra.run.smithi059.stdout:{"epoch":22,"fsid":"9f0e8741-b8f9-48d5-a2c3-09f45903303d","created":"2023-11-24T07:56:06.552729+0000","modified":"2023-11-24T07:57:13.434013+0000","last_up_change":"2023-11-24T07:56:17.372352+0000","last_in_change":"2023-11-24T07:56:09.073250+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-24T07:56:18.352823+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-11-24T07:56:22.185140+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-11-24T07:56:23.084326+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"6d8f9e9b-1782-4e41-928b-a39f2a8ebc06","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6800","nonce":3373878772},{"type":"v1","addr":"172.21.15.59:6801","nonce":3373878772}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6802","nonce":3373878772},{"type":"v1","addr":"172.21.15.59:6803","nonce":3373878772}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6806","nonce":3373878772},{"type":"v1","addr":"172.21.15.59:6807","nonce":3373878772}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6804","nonce":3373878772},{"type":"v1","addr":"172.21.15.59:6805","nonce":3373878772}]},"public_addr":"172.21.15.59:6801/3373878772","cluster_addr":"172.21.15.59:6803/3373878772","heartbeat_back_addr":"172.21.15.59:6807/3373878772","heartbeat_front_addr":"172.21.15.59:6805/3373878772","state":["exists","up"]},{"osd":1,"uuid":"e8d882aa-52e1-4c04-9e1e-c44ef9204a5f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6824","nonce":648258563},{"type":"v1","addr":"172.21.15.59:6825","nonce":648258563}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6826","nonce":648258563},{"type":"v1","addr":"172.21.15.59:6827","nonce":648258563}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6830","nonce":648258563},{"type":"v1","addr":"172.21.15.59:6831","nonce":648258563}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6828","nonce":648258563},{"type":"v1","addr":"172.21.15.59:6829","nonce":648258563}]},"public_addr":"172.21.15.59:6825/648258563","cluster_addr":"172.21.15.59:6827/648258563","heartbeat_back_addr":"172.21.15.59:6831/648258563","heartbeat_front_addr":"172.21.15.59:6829/648258563","state":["exists","up"]},{"osd":2,"uuid":"69eebfa0-695b-4b21-90b9-8a3537168a01","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6808","nonce":1840430654},{"type":"v1","addr":"172.21.15.59:6809","nonce":1840430654}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6810","nonce":1840430654},{"type":"v1","addr":"172.21.15.59:6811","nonce":1840430654}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6814","nonce":1840430654},{"type":"v1","addr":"172.21.15.59:6815","nonce":1840430654}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6812","nonce":1840430654},{"type":"v1","addr":"172.21.15.59:6813","nonce":1840430654}]},"public_addr":"172.21.15.59:6809/1840430654","cluster_addr":"172.21.15.59:6811/1840430654","heartbeat_back_addr":"172.21.15.59:6815/1840430654","heartbeat_front_addr":"172.21.15.59:6813/1840430654","state":["exists","up"]},{"osd":3,"uuid":"96fd77fc-f911-46ff-9445-6f0be2daa8f4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6816","nonce":2504025850},{"type":"v1","addr":"172.21.15.59:6817","nonce":2504025850}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6818","nonce":2504025850},{"type":"v1","addr":"172.21.15.59:6819","nonce":2504025850}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6822","nonce":2504025850},{"type":"v1","addr":"172.21.15.59:6823","nonce":2504025850}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6820","nonce":2504025850},{"type":"v1","addr":"172.21.15.59:6821","nonce":2504025850}]},"public_addr":"172.21.15.59:6817/2504025850","cluster_addr":"172.21.15.59:6819/2504025850","heartbeat_back_addr":"172.21.15.59:6823/2504025850","heartbeat_front_addr":"172.21.15.59:6821/2504025850","state":["exists","up"]},{"osd":4,"uuid":"7ac29fca-b5b2-4c95-9d69-2f164cbbc708","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6816","nonce":2382463431},{"type":"v1","addr":"172.21.15.76:6818","nonce":2382463431}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6820","nonce":2382463431},{"type":"v1","addr":"172.21.15.76:6822","nonce":2382463431}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6828","nonce":2382463431},{"type":"v1","addr":"172.21.15.76:6830","nonce":2382463431}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6824","nonce":2382463431},{"type":"v1","addr":"172.21.15.76:6826","nonce":2382463431}]},"public_addr":"172.21.15.76:6818/2382463431","cluster_addr":"172.21.15.76:6822/2382463431","heartbeat_back_addr":"172.21.15.76:6830/2382463431","heartbeat_front_addr":"172.21.15.76:6826/2382463431","state":["exists","up"]},{"osd":5,"uuid":"485bf711-3851-4238-a4f4-51d52e0a9c58","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6800","nonce":329334308},{"type":"v1","addr":"172.21.15.76:6802","nonce":329334308}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6805","nonce":329334308},{"type":"v1","addr":"172.21.15.76:6807","nonce":329334308}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6814","nonce":329334308},{"type":"v1","addr":"172.21.15.76:6815","nonce":329334308}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6810","nonce":329334308},{"type":"v1","addr":"172.21.15.76:6812","nonce":329334308}]},"public_addr":"172.21.15.76:6802/329334308","cluster_addr":"172.21.15.76:6807/329334308","heartbeat_back_addr":"172.21.15.76:6815/329334308","heartbeat_front_addr":"172.21.15.76:6812/329334308","state":["exists","up"]},{"osd":6,"uuid":"72b3f2fc-1e4a-4e32-968d-003647a0286f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6801","nonce":3946579410},{"type":"v1","addr":"172.21.15.76:6803","nonce":3946579410}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6804","nonce":3946579410},{"type":"v1","addr":"172.21.15.76:6806","nonce":3946579410}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6811","nonce":3946579410},{"type":"v1","addr":"172.21.15.76:6813","nonce":3946579410}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6808","nonce":3946579410},{"type":"v1","addr":"172.21.15.76:6809","nonce":3946579410}]},"public_addr":"172.21.15.76:6803/3946579410","cluster_addr":"172.21.15.76:6806/3946579410","heartbeat_back_addr":"172.21.15.76:6813/3946579410","heartbeat_front_addr":"172.21.15.76:6809/3946579410","state":["exists","up"]},{"osd":7,"uuid":"1ddceb90-6c5d-4d1a-ad56-99cb877f8c70","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6817","nonce":841761538},{"type":"v1","addr":"172.21.15.76:6819","nonce":841761538}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6821","nonce":841761538},{"type":"v1","addr":"172.21.15.76:6823","nonce":841761538}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6829","nonce":841761538},{"type":"v1","addr":"172.21.15.76:6831","nonce":841761538}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6825","nonce":841761538},{"type":"v1","addr":"172.21.15.76:6827","nonce":841761538}]},"public_addr":"172.21.15.76:6819/841761538","cluster_addr":"172.21.15.76:6823/841761538","heartbeat_back_addr":"172.21.15.76:6831/841761538","heartbeat_front_addr":"172.21.15.76:6827/841761538","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T07:56:14.294838+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T07:56:14.270779+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T07:56:14.231809+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T07:56:14.260983+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T07:56:14.302997+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T07:56:14.258600+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T07:56:14.251413+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T07:56:14.304727+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.31","mappings":[{"from":0,"to":7}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-24T07:57:13.855 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:13.853+0000 7f99b1daa700 1 -- 172.21.15.59:0/919474759 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f999806aed0 msgr2=0x7f999806d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:57:13.855 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:13.853+0000 7f99b1daa700 1 --2- 172.21.15.59:0/919474759 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f999806aed0 0x7f999806d390 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f999c009930 tx=0x7f999c008040 comp rx=0 tx=0).stop 2023-11-24T07:57:13.856 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:13.853+0000 7f99b1daa700 1 -- 172.21.15.59:0/919474759 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f99ac0cd470 msgr2=0x7f99ac144ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:57:13.856 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:13.853+0000 7f99b1daa700 1 --2- 172.21.15.59:0/919474759 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f99ac0cd470 0x7f99ac144ab0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f99a000d800 tx=0x7f99a000dbc0 comp rx=0 tx=0).stop 2023-11-24T07:57:13.856 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:13.853+0000 7f99b1daa700 1 -- 172.21.15.59:0/919474759 shutdown_connections 2023-11-24T07:57:13.856 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:13.853+0000 7f99b1daa700 1 --2- 172.21.15.59:0/919474759 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f999806aed0 0x7f999806d390 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:13.856 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:13.853+0000 7f99b1daa700 1 --2- 172.21.15.59:0/919474759 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f99ac1457a0 0x7f99ac157080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:13.856 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:13.853+0000 7f99b1daa700 1 --2- 172.21.15.59:0/919474759 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f99ac06afd0 0x7f99ac1432a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:13.856 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:13.853+0000 7f99b1daa700 1 --2- 172.21.15.59:0/919474759 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f99ac0cd470 0x7f99ac144ab0 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:13.856 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:13.853+0000 7f99b1daa700 1 -- 172.21.15.59:0/919474759 >> 172.21.15.59:0/919474759 conn(0x7f99ac0d0aa0 msgr2=0x7f99ac15a3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:57:13.856 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:13.854+0000 7f99b1daa700 1 -- 172.21.15.59:0/919474759 shutdown_connections 2023-11-24T07:57:13.856 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:13.854+0000 7f99b1daa700 1 -- 172.21.15.59:0/919474759 wait complete. 2023-11-24T07:57:13.868 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2023-11-24T07:57:13.994 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:13.992+0000 7efdadce2700 1 Processor -- start 2023-11-24T07:57:13.994 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:13.992+0000 7efdadce2700 1 -- start start 2023-11-24T07:57:14.001 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:13.992+0000 7efdadce2700 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7efda815bde0 0x7efda815c1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:57:14.001 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:13.993+0000 7efdadce2700 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7efda80ce5c0 0x7efda80cea50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:57:14.001 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:13.993+0000 7efdadce2700 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efda80d0a90 con 0x7efda80cf2b0 2023-11-24T07:57:14.002 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:13.993+0000 7efdadce2700 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efda80cef90 con 0x7efda80ce5c0 2023-11-24T07:57:14.002 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:13.993+0000 7efdadce2700 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efda806ee10 con 0x7efda815bde0 2023-11-24T07:57:14.002 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:13.993+0000 7efda6ffd700 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7efda80cf2b0 0x7efda80d01b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:54830/0 (socket says 172.21.15.59:54830) 2023-11-24T07:57:14.002 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:13.993+0000 7efda6ffd700 1 -- 172.21.15.59:0/1237401476 learned_addr learned my addr 172.21.15.59:0/1237401476 (peer_addr_for_me v1:172.21.15.59:0/0) 2023-11-24T07:57:14.002 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:13.993+0000 7efda7fff700 1 --2- 172.21.15.59:0/1237401476 >> v2:172.21.15.76:3300/0 conn(0x7efda80ce5c0 0x7efda80cea50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:57:14.002 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:13.993+0000 7efda77fe700 1 --2- 172.21.15.59:0/1237401476 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7efda815bde0 0x7efda815c1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:57:14.002 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:13.993+0000 7efda67fc700 1 -- 172.21.15.59:0/1237401476 <== mon.0 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4172431527 0 0) 0x7efda80d0a90 con 0x7efda80cf2b0 2023-11-24T07:57:14.003 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:13.993+0000 7efda67fc700 1 -- 172.21.15.59:0/1237401476 --> v1:172.21.15.59:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7efd900035b0 con 0x7efda80cf2b0 2023-11-24T07:57:14.003 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:13.993+0000 7efda67fc700 1 -- 172.21.15.59:0/1237401476 <== mon.0 v1:172.21.15.59:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1156854602 0 0) 0x7efd900035b0 con 0x7efda80cf2b0 2023-11-24T07:57:14.003 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:13.993+0000 7efda67fc700 1 -- 172.21.15.59:0/1237401476 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7efda815bde0 msgr2=0x7efda815c1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T07:57:14.003 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:13.993+0000 7efda67fc700 1 --2- 172.21.15.59:0/1237401476 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7efda815bde0 0x7efda815c1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:14.003 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:13.993+0000 7efda67fc700 1 -- 172.21.15.59:0/1237401476 >> v2:172.21.15.76:3300/0 conn(0x7efda80ce5c0 msgr2=0x7efda80cea50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T07:57:14.003 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:13.993+0000 7efda67fc700 1 --2- 172.21.15.59:0/1237401476 >> v2:172.21.15.76:3300/0 conn(0x7efda80ce5c0 0x7efda80cea50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:14.003 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:13.993+0000 7efda67fc700 1 -- 172.21.15.59:0/1237401476 --> v1:172.21.15.59:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efda806f0f0 con 0x7efda80cf2b0 2023-11-24T07:57:14.004 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:13.993+0000 7efda67fc700 1 -- 172.21.15.59:0/1237401476 <== mon.0 v1:172.21.15.59:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1312333566 0 0) 0x7efd9c001670 con 0x7efda80cf2b0 2023-11-24T07:57:14.004 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:13.993+0000 7efda67fc700 1 -- 172.21.15.59:0/1237401476 >> v1:172.21.15.59:6789/0 conn(0x7efda80cf2b0 legacy=0x7efda80d01b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:57:14.004 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:13.993+0000 7efda67fc700 1 --2- 172.21.15.59:0/1237401476 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7efd90003a00 0x7efd90003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:57:14.004 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:13.993+0000 7efda67fc700 1 --2- 172.21.15.59:0/1237401476 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7efd90004850 0x7efd90004cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:57:14.004 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:13.993+0000 7efda67fc700 1 --2- 172.21.15.59:0/1237401476 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7efda815bde0 0x7efd90005210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:57:14.005 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:13.993+0000 7efda67fc700 1 -- 172.21.15.59:0/1237401476 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efda80cef90 con 0x7efd90003a00 2023-11-24T07:57:14.005 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:13.993+0000 7efda67fc700 1 -- 172.21.15.59:0/1237401476 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efda806ee10 con 0x7efd90004850 2023-11-24T07:57:14.005 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:13.993+0000 7efda67fc700 1 -- 172.21.15.59:0/1237401476 --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efd9000ba10 con 0x7efda815bde0 2023-11-24T07:57:14.005 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:13.994+0000 7efda77fe700 1 --2- 172.21.15.59:0/1237401476 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7efd90003a00 0x7efd90003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:57:14.006 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:13.994+0000 7efda6ffd700 1 --2- 172.21.15.59:0/1237401476 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7efd90004850 0x7efd90004cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:57:14.006 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:13.994+0000 7efda7fff700 1 --2- 172.21.15.59:0/1237401476 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7efda815bde0 0x7efd90005210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:57:14.006 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:13.994+0000 7efda77fe700 1 -- 172.21.15.59:0/1237401476 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7efda815bde0 msgr2=0x7efd90005210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T07:57:14.006 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:13.994+0000 7efda77fe700 1 --2- 172.21.15.59:0/1237401476 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7efda815bde0 0x7efd90005210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:14.006 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:13.994+0000 7efda77fe700 1 -- 172.21.15.59:0/1237401476 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7efd90004850 msgr2=0x7efd90004cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T07:57:14.007 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:13.994+0000 7efda77fe700 1 --2- 172.21.15.59:0/1237401476 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7efd90004850 0x7efd90004cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:14.007 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:13.994+0000 7efda77fe700 1 -- 172.21.15.59:0/1237401476 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efd900035b0 con 0x7efd90003a00 2023-11-24T07:57:14.007 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:13.994+0000 7efda7fff700 1 --2- 172.21.15.59:0/1237401476 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7efda815bde0 0x7efd90005210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-24T07:57:14.007 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:13.994+0000 7efda6ffd700 1 --2- 172.21.15.59:0/1237401476 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7efd90004850 0x7efd90004cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-24T07:57:14.007 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:13.994+0000 7efda77fe700 1 --2- 172.21.15.59:0/1237401476 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7efd90003a00 0x7efd90003e20 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7efd98009b90 tx=0x7efd90005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=231f9821d1fbc582 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:57:14.007 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:13.994+0000 7efda67fc700 1 -- 172.21.15.59:0/1237401476 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd9800e6c0 con 0x7efd90003a00 2023-11-24T07:57:14.008 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:13.994+0000 7efda67fc700 1 -- 172.21.15.59:0/1237401476 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efd9800e820 con 0x7efd90003a00 2023-11-24T07:57:14.008 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:13.994+0000 7efda67fc700 1 -- 172.21.15.59:0/1237401476 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd9800eb90 con 0x7efd90003a00 2023-11-24T07:57:14.008 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:13.995+0000 7efdadce2700 1 -- 172.21.15.59:0/1237401476 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7efd90003a00 msgr2=0x7efd90003e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:57:14.008 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:13.995+0000 7efdadce2700 1 --2- 172.21.15.59:0/1237401476 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7efd90003a00 0x7efd90003e20 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7efd98009b90 tx=0x7efd90005880 comp rx=0 tx=0).stop 2023-11-24T07:57:14.008 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:13.995+0000 7efda7fff700 1 -- 172.21.15.59:0/1237401476 reap_dead start 2023-11-24T07:57:14.008 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:13.995+0000 7efdadce2700 1 -- 172.21.15.59:0/1237401476 shutdown_connections 2023-11-24T07:57:14.009 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:13.995+0000 7efdadce2700 1 -- 172.21.15.59:0/1237401476 >> 172.21.15.59:0/1237401476 conn(0x7efda80b3720 msgr2=0x7efda80b3b30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:57:14.009 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:13.995+0000 7efdadce2700 1 -- 172.21.15.59:0/1237401476 shutdown_connections 2023-11-24T07:57:14.009 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:13.995+0000 7efdadce2700 1 -- 172.21.15.59:0/1237401476 wait complete. 2023-11-24T07:57:14.009 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:13.996+0000 7efdadce2700 1 Processor -- start 2023-11-24T07:57:14.009 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:13.996+0000 7efdadce2700 1 -- start start 2023-11-24T07:57:14.009 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:13.997+0000 7efdadce2700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7efd90003a00 0x7efda815a0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:57:14.010 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:13.997+0000 7efdadce2700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7efda80afa40 0x7efda80afec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:57:14.010 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:13.997+0000 7efdadce2700 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7efda81f39c0 0x7efda81f3e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:57:14.010 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:13.997+0000 7efdadce2700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efda80ca0c0 con 0x7efda80afa40 2023-11-24T07:57:14.010 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:13.997+0000 7efdadce2700 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efda81f7350 con 0x7efd90003a00 2023-11-24T07:57:14.010 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:13.997+0000 7efdadce2700 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efda80b0400 con 0x7efda81f39c0 2023-11-24T07:57:14.011 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:13.997+0000 7efda6ffd700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7efda80afa40 0x7efda80afec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:57:14.011 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:13.997+0000 7efda6ffd700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7efda80afa40 0x7efda80afec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:51370/0 (socket says 172.21.15.59:51370) 2023-11-24T07:57:14.011 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:13.997+0000 7efda6ffd700 1 -- 172.21.15.59:0/3534610195 learned_addr learned my addr 172.21.15.59:0/3534610195 (peer_addr_for_me v2:172.21.15.59:0/0) 2023-11-24T07:57:14.011 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:13.997+0000 7efda77fe700 1 --2- 172.21.15.59:0/3534610195 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7efd90003a00 0x7efda815a0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:57:14.011 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:13.997+0000 7efda7fff700 1 --2- 172.21.15.59:0/3534610195 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7efda81f39c0 0x7efda81f3e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:57:14.011 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:13.997+0000 7efda6ffd700 1 -- 172.21.15.59:0/3534610195 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7efda81f39c0 msgr2=0x7efda81f3e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:57:14.011 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:13.997+0000 7efda6ffd700 1 --2- 172.21.15.59:0/3534610195 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7efda81f39c0 0x7efda81f3e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:14.011 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:13.997+0000 7efda6ffd700 1 -- 172.21.15.59:0/3534610195 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7efd90003a00 msgr2=0x7efda815a0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:57:14.012 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:13.997+0000 7efda6ffd700 1 --2- 172.21.15.59:0/3534610195 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7efd90003a00 0x7efda815a0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:14.012 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:13.997+0000 7efda6ffd700 1 -- 172.21.15.59:0/3534610195 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efda81f4960 con 0x7efda80afa40 2023-11-24T07:57:14.012 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:13.997+0000 7efda7fff700 1 --2- 172.21.15.59:0/3534610195 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7efda81f39c0 0x7efda81f3e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-24T07:57:14.012 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:13.997+0000 7efda6ffd700 1 --2- 172.21.15.59:0/3534610195 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7efda80afa40 0x7efda80afec0 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7efd9c000fb0 tx=0x7efd9c00ead0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:57:14.012 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:13.997+0000 7efd8ffff700 1 -- 172.21.15.59:0/3534610195 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd9c012590 con 0x7efda80afa40 2023-11-24T07:57:14.012 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:13.998+0000 7efd8ffff700 1 -- 172.21.15.59:0/3534610195 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efd9c0126f0 con 0x7efda80afa40 2023-11-24T07:57:14.012 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:13.998+0000 7efdadce2700 1 -- 172.21.15.59:0/3534610195 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efda81f4680 con 0x7efda80afa40 2023-11-24T07:57:14.012 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:13.998+0000 7efd8ffff700 1 -- 172.21.15.59:0/3534610195 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd9c012a60 con 0x7efda80afa40 2023-11-24T07:57:14.013 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:13.998+0000 7efdadce2700 1 -- 172.21.15.59:0/3534610195 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efda80ca5b0 con 0x7efda80afa40 2023-11-24T07:57:14.013 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:13.998+0000 7efdadce2700 1 -- 172.21.15.59:0/3534610195 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efda8068670 con 0x7efda80afa40 2023-11-24T07:57:14.013 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:13.999+0000 7efd8ffff700 1 -- 172.21.15.59:0/3534610195 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7efd9c012bc0 con 0x7efda80afa40 2023-11-24T07:57:14.013 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:13.999+0000 7efd8ffff700 1 --2- 172.21.15.59:0/3534610195 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7efd9406aed0 0x7efd9406d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:57:14.013 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:13.999+0000 7efd8ffff700 1 -- 172.21.15.59:0/3534610195 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5738+0+0 (secure 0 0 0) 0x7efd9c089340 con 0x7efda80afa40 2023-11-24T07:57:14.013 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:14.000+0000 7efda77fe700 1 --2- 172.21.15.59:0/3534610195 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7efd9406aed0 0x7efd9406d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:57:14.014 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:14.000+0000 7efda77fe700 1 --2- 172.21.15.59:0/3534610195 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7efd9406aed0 0x7efd9406d390 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7efd980090b0 tx=0x7efd9801f040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:57:14.014 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:14.004+0000 7efd8ffff700 1 -- 172.21.15.59:0/3534610195 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7efd9c057e60 con 0x7efda80afa40 2023-11-24T07:57:14.254 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:14.252+0000 7efdadce2700 1 -- 172.21.15.59:0/3534610195 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7efda80cfa20 con 0x7efda80afa40 2023-11-24T07:57:14.254 INFO:tasks.ceph.mon.a.smithi059.stderr:2023-11-24T07:57:14.253+0000 7f5560821700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-11-24T07:57:14.255 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:14.254+0000 7efd8ffff700 1 -- 172.21.15.59:0/3534610195 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v9) v1 ==== 161+0+0 (secure 0 0 0) 0x7efd9c0579f0 con 0x7efda80afa40 2023-11-24T07:57:14.259 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:14.256+0000 7efdadce2700 1 -- 172.21.15.59:0/3534610195 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7efd9406aed0 msgr2=0x7efd9406d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:57:14.259 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:14.256+0000 7efdadce2700 1 --2- 172.21.15.59:0/3534610195 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7efd9406aed0 0x7efd9406d390 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7efd980090b0 tx=0x7efd9801f040 comp rx=0 tx=0).stop 2023-11-24T07:57:14.260 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:14.256+0000 7efdadce2700 1 -- 172.21.15.59:0/3534610195 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7efda80afa40 msgr2=0x7efda80afec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:57:14.260 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:14.256+0000 7efdadce2700 1 --2- 172.21.15.59:0/3534610195 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7efda80afa40 0x7efda80afec0 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7efd9c000fb0 tx=0x7efd9c00ead0 comp rx=0 tx=0).stop 2023-11-24T07:57:14.260 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:14.256+0000 7efdadce2700 1 -- 172.21.15.59:0/3534610195 shutdown_connections 2023-11-24T07:57:14.260 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:14.256+0000 7efdadce2700 1 --2- 172.21.15.59:0/3534610195 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7efd9406aed0 0x7efd9406d390 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:14.261 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:14.256+0000 7efdadce2700 1 --2- 172.21.15.59:0/3534610195 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7efda81f39c0 0x7efda81f3e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:14.261 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:14.256+0000 7efdadce2700 1 --2- 172.21.15.59:0/3534610195 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7efd90003a00 0x7efda815a0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:14.261 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:14.256+0000 7efdadce2700 1 --2- 172.21.15.59:0/3534610195 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7efda80afa40 0x7efda80afec0 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:14.261 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:14.256+0000 7efdadce2700 1 -- 172.21.15.59:0/3534610195 >> 172.21.15.59:0/3534610195 conn(0x7efda80b3720 msgr2=0x7efda80b2b90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:57:14.262 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:14.257+0000 7efdadce2700 1 -- 172.21.15.59:0/3534610195 shutdown_connections 2023-11-24T07:57:14.262 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:14.257+0000 7efdadce2700 1 -- 172.21.15.59:0/3534610195 wait complete. 2023-11-24T07:57:14.262 INFO:teuthology.orchestra.run.smithi059.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2023-11-24T07:57:14.271 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2023-11-24T07:57:14.300 INFO:tasks.ceph.mds.b.smithi076.stdout:starting mds.b at 2023-11-24T07:57:14.399 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:14.397+0000 7f1ffffb6700 1 Processor -- start 2023-11-24T07:57:14.399 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:14.397+0000 7f1ffffb6700 1 -- start start 2023-11-24T07:57:14.400 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:14.398+0000 7f1ffffb6700 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f1ff815e1e0 0x7f1ff80b12a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:57:14.401 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:14.398+0000 7f1ffffb6700 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f1ff80b5e40 0x7f1ff815f5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:57:14.401 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:14.398+0000 7f1ffffb6700 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1ff806bb80 con 0x7f1ff806b400 2023-11-24T07:57:14.401 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:14.398+0000 7f1ffffb6700 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1ff806be20 con 0x7f1ff80b5e40 2023-11-24T07:57:14.401 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:14.398+0000 7f1ffffb6700 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1ff806c100 con 0x7f1ff815e1e0 2023-11-24T07:57:14.401 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:14.398+0000 7f1ffdd52700 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f1ff815e1e0 0x7f1ff80b12a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:57:14.401 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:14.398+0000 7f1ffe553700 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f1ff80b5e40 0x7f1ff815f5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:57:14.401 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:14.398+0000 7f1ffdd52700 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f1ff815e1e0 0x7f1ff80b12a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3301/0 says I am v2:172.21.15.59:50816/0 (socket says 172.21.15.59:50816) 2023-11-24T07:57:14.401 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:14.398+0000 7f1ffdd52700 1 -- 172.21.15.59:0/114919008 learned_addr learned my addr 172.21.15.59:0/114919008 (peer_addr_for_me v2:172.21.15.59:0/0) 2023-11-24T07:57:14.401 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:14.399+0000 7f1ffdd52700 1 -- 172.21.15.59:0/114919008 >> v2:172.21.15.76:3300/0 conn(0x7f1ff80b5e40 msgr2=0x7f1ff815f5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T07:57:14.401 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:14.399+0000 7f1ffdd52700 1 --2- 172.21.15.59:0/114919008 >> v2:172.21.15.76:3300/0 conn(0x7f1ff80b5e40 0x7f1ff815f5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:14.401 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:14.399+0000 7f1ffdd52700 1 -- 172.21.15.59:0/114919008 >> v1:172.21.15.76:6789/0 conn(0x7f1ff806b400 legacy=0x7f1ff80726a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:57:14.402 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:14.399+0000 7f1ffdd52700 1 -- 172.21.15.59:0/114919008 --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1ff806c3e0 con 0x7f1ff815e1e0 2023-11-24T07:57:14.402 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:14.399+0000 7f1ffdd52700 1 --2- 172.21.15.59:0/114919008 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f1ff815e1e0 0x7f1ff80b12a0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f1fe80098f0 tx=0x7f1fe8009c00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=484b3ebce6dc2a63 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:57:14.402 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:14.399+0000 7f1ffcd50700 1 -- 172.21.15.59:0/114919008 <== mon.2 v2:172.21.15.76:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1fe8011830 con 0x7f1ff815e1e0 2023-11-24T07:57:14.402 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:14.399+0000 7f1ffcd50700 1 -- 172.21.15.59:0/114919008 <== mon.2 v2:172.21.15.76:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1fe8011990 con 0x7f1ff815e1e0 2023-11-24T07:57:14.402 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:14.399+0000 7f1ffcd50700 1 -- 172.21.15.59:0/114919008 <== mon.2 v2:172.21.15.76:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1fe8011d20 con 0x7f1ff815e1e0 2023-11-24T07:57:14.405 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:14.399+0000 7f1ffffb6700 1 -- 172.21.15.59:0/114919008 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f1ff815e1e0 msgr2=0x7f1ff80b12a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:57:14.405 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:14.399+0000 7f1ffffb6700 1 --2- 172.21.15.59:0/114919008 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f1ff815e1e0 0x7f1ff80b12a0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f1fe80098f0 tx=0x7f1fe8009c00 comp rx=0 tx=0).stop 2023-11-24T07:57:14.405 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:14.400+0000 7f1ffffb6700 1 -- 172.21.15.59:0/114919008 shutdown_connections 2023-11-24T07:57:14.405 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:14.400+0000 7f1ffffb6700 1 --2- 172.21.15.59:0/114919008 >> v2:172.21.15.76:3300/0 conn(0x7f1ff80b5e40 0x7f1ff815f5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:14.405 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:14.400+0000 7f1ffffb6700 1 --2- 172.21.15.59:0/114919008 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f1ff815e1e0 0x7f1ff80b12a0 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:14.406 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:14.400+0000 7f1ffffb6700 1 -- 172.21.15.59:0/114919008 >> 172.21.15.59:0/114919008 conn(0x7f1ff80b5070 msgr2=0x7f1ff80b5730 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:57:14.406 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:14.400+0000 7f1ffffb6700 1 -- 172.21.15.59:0/114919008 shutdown_connections 2023-11-24T07:57:14.406 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:14.400+0000 7f1ffffb6700 1 -- 172.21.15.59:0/114919008 wait complete. 2023-11-24T07:57:14.406 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:14.401+0000 7f1ffffb6700 1 Processor -- start 2023-11-24T07:57:14.406 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:14.401+0000 7f1ffffb6700 1 -- start start 2023-11-24T07:57:14.406 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:14.402+0000 7f1ffffb6700 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f1ff806b400 0x7f1ff8145d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:57:14.406 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:14.402+0000 7f1ffffb6700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1ff80b5e40 0x7f1ff80adc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:57:14.406 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:14.402+0000 7f1ffffb6700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1ff815e1e0 0x7f1ff80ae140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:57:14.406 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:14.402+0000 7f1ffffb6700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ff8159200 con 0x7f1ff80b5e40 2023-11-24T07:57:14.407 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:14.402+0000 7f1ffffb6700 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ff8146260 con 0x7f1ff815e1e0 2023-11-24T07:57:14.407 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:14.402+0000 7f1ffffb6700 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1ff81469e0 con 0x7f1ff806b400 2023-11-24T07:57:14.407 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:14.402+0000 7f1ffd551700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1ff80b5e40 0x7f1ff80adc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:57:14.407 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:14.402+0000 7f1ffdd52700 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f1ff806b400 0x7f1ff8145d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:57:14.407 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:14.402+0000 7f1ffd551700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1ff80b5e40 0x7f1ff80adc00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:51384/0 (socket says 172.21.15.59:51384) 2023-11-24T07:57:14.407 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:14.402+0000 7f1ffd551700 1 -- 172.21.15.59:0/3291108173 learned_addr learned my addr 172.21.15.59:0/3291108173 (peer_addr_for_me v2:172.21.15.59:0/0) 2023-11-24T07:57:14.407 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:14.402+0000 7f1ffe553700 1 --2- 172.21.15.59:0/3291108173 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1ff815e1e0 0x7f1ff80ae140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:57:14.407 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:14.402+0000 7f1ffdd52700 1 -- 172.21.15.59:0/3291108173 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1ff815e1e0 msgr2=0x7f1ff80ae140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:57:14.407 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:14.402+0000 7f1ffdd52700 1 --2- 172.21.15.59:0/3291108173 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1ff815e1e0 0x7f1ff80ae140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:14.407 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:14.402+0000 7f1ffdd52700 1 -- 172.21.15.59:0/3291108173 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1ff80b5e40 msgr2=0x7f1ff80adc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:57:14.407 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:14.402+0000 7f1ffdd52700 1 --2- 172.21.15.59:0/3291108173 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1ff80b5e40 0x7f1ff80adc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:14.407 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:14.402+0000 7f1ffdd52700 1 -- 172.21.15.59:0/3291108173 --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1fe8009550 con 0x7f1ff806b400 2023-11-24T07:57:14.408 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:14.402+0000 7f1ffd551700 1 --2- 172.21.15.59:0/3291108173 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1ff80b5e40 0x7f1ff80adc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-24T07:57:14.408 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:14.402+0000 7f1ffdd52700 1 --2- 172.21.15.59:0/3291108173 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f1ff806b400 0x7f1ff8145d20 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f1fe8016000 tx=0x7f1fe8004750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:57:14.408 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:14.403+0000 7f1fee7fc700 1 -- 172.21.15.59:0/3291108173 <== mon.2 v2:172.21.15.76:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1fe8004590 con 0x7f1ff806b400 2023-11-24T07:57:14.408 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:14.403+0000 7f1fee7fc700 1 -- 172.21.15.59:0/3291108173 <== mon.2 v2:172.21.15.76:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1fe801e070 con 0x7f1ff806b400 2023-11-24T07:57:14.408 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:14.403+0000 7f1ffffb6700 1 -- 172.21.15.59:0/3291108173 --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1ff80ae770 con 0x7f1ff806b400 2023-11-24T07:57:14.408 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:14.403+0000 7f1fee7fc700 1 -- 172.21.15.59:0/3291108173 <== mon.2 v2:172.21.15.76:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1fe80038a0 con 0x7f1ff806b400 2023-11-24T07:57:14.408 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:14.403+0000 7f1ffffb6700 1 -- 172.21.15.59:0/3291108173 --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1ff8146670 con 0x7f1ff806b400 2023-11-24T07:57:14.409 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:14.404+0000 7f1ffffb6700 1 -- 172.21.15.59:0/3291108173 --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1ff80aec90 con 0x7f1ff806b400 2023-11-24T07:57:14.409 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:14.404+0000 7f1fee7fc700 1 -- 172.21.15.59:0/3291108173 <== mon.2 v2:172.21.15.76:3301/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f1fe8003a00 con 0x7f1ff806b400 2023-11-24T07:57:14.409 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:14.405+0000 7f1fee7fc700 1 --2- 172.21.15.59:0/3291108173 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f1fdc06aed0 0x7f1fdc06d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:57:14.409 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:14.405+0000 7f1fee7fc700 1 -- 172.21.15.59:0/3291108173 <== mon.2 v2:172.21.15.76:3301/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5824+0+0 (secure 0 0 0) 0x7f1fe808a610 con 0x7f1ff806b400 2023-11-24T07:57:14.410 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:14.405+0000 7f1ffd551700 1 --2- 172.21.15.59:0/3291108173 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f1fdc06aed0 0x7f1fdc06d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:57:14.410 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:14.405+0000 7f1ffd551700 1 --2- 172.21.15.59:0/3291108173 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f1fdc06aed0 0x7f1fdc06d390 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f1ff4001d10 tx=0x7f1ff4009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:57:14.411 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:14.410+0000 7f1fee7fc700 1 -- 172.21.15.59:0/3291108173 <== mon.2 v2:172.21.15.76:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f1ff80aec90 con 0x7f1ff806b400 2023-11-24T07:57:14.660 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:14.658+0000 7f1ffffb6700 1 -- 172.21.15.59:0/3291108173 --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f1ff81464d0 con 0x7f1ff806b400 2023-11-24T07:57:15.265 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:15.263+0000 7f1fee7fc700 1 -- 172.21.15.59:0/3291108173 <== mon.2 v2:172.21.15.76:3301/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v10) v1 ==== 104+0+0 (secure 0 0 0) 0x7f1ff81464d0 con 0x7f1ff806b400 2023-11-24T07:57:15.268 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:15.266+0000 7f1ffffb6700 1 -- 172.21.15.59:0/3291108173 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f1fdc06aed0 msgr2=0x7f1fdc06d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:57:15.269 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:15.266+0000 7f1ffffb6700 1 --2- 172.21.15.59:0/3291108173 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f1fdc06aed0 0x7f1fdc06d390 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f1ff4001d10 tx=0x7f1ff4009040 comp rx=0 tx=0).stop 2023-11-24T07:57:15.269 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:15.266+0000 7f1ffffb6700 1 -- 172.21.15.59:0/3291108173 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f1ff806b400 msgr2=0x7f1ff8145d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:57:15.269 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:15.266+0000 7f1ffffb6700 1 --2- 172.21.15.59:0/3291108173 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f1ff806b400 0x7f1ff8145d20 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f1fe8016000 tx=0x7f1fe8004750 comp rx=0 tx=0).stop 2023-11-24T07:57:15.269 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:15.266+0000 7f1ffffb6700 1 -- 172.21.15.59:0/3291108173 shutdown_connections 2023-11-24T07:57:15.269 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:15.267+0000 7f1ffffb6700 1 --2- 172.21.15.59:0/3291108173 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f1fdc06aed0 0x7f1fdc06d390 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:15.270 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:15.267+0000 7f1ffffb6700 1 --2- 172.21.15.59:0/3291108173 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1ff815e1e0 0x7f1ff80ae140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:15.270 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:15.267+0000 7f1ffffb6700 1 --2- 172.21.15.59:0/3291108173 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f1ff806b400 0x7f1ff8145d20 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:15.270 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:15.267+0000 7f1ffffb6700 1 --2- 172.21.15.59:0/3291108173 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1ff80b5e40 0x7f1ff80adc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:15.270 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:15.267+0000 7f1ffffb6700 1 -- 172.21.15.59:0/3291108173 >> 172.21.15.59:0/3291108173 conn(0x7f1ff80b5070 msgr2=0x7f1ff815e5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:57:15.270 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:15.267+0000 7f1ffffb6700 1 -- 172.21.15.59:0/3291108173 shutdown_connections 2023-11-24T07:57:15.271 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:15.267+0000 7f1ffffb6700 1 -- 172.21.15.59:0/3291108173 wait complete. 2023-11-24T07:57:15.283 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2023-11-24T07:57:15.419 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:15.417+0000 7f4a61ee9700 1 Processor -- start 2023-11-24T07:57:15.419 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:15.417+0000 7f4a61ee9700 1 -- start start 2023-11-24T07:57:15.421 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:15.418+0000 7f4a61ee9700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f4a5c0722c0 0x7f4a5c0b51b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:57:15.422 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:15.418+0000 7f4a61ee9700 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f4a5c1440f0 0x7f4a5c06e510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:57:15.422 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:15.418+0000 7f4a61ee9700 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4a5c143ec0 con 0x7f4a5c143ae0 2023-11-24T07:57:15.422 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:15.418+0000 7f4a61ee9700 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4a5c0b40a0 con 0x7f4a5c0722c0 2023-11-24T07:57:15.422 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:15.418+0000 7f4a61ee9700 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4a5c0b3ae0 con 0x7f4a5c1440f0 2023-11-24T07:57:15.422 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:15.418+0000 7f4a5affd700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f4a5c0722c0 0x7f4a5c0b51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:57:15.422 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:15.418+0000 7f4a5b7fe700 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7f4a5c143ae0 0x7f4a5c071bb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:54864/0 (socket says 172.21.15.59:54864) 2023-11-24T07:57:15.422 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:15.418+0000 7f4a5b7fe700 1 -- 172.21.15.59:0/354120497 learned_addr learned my addr 172.21.15.59:0/354120497 (peer_addr_for_me v1:172.21.15.59:0/0) 2023-11-24T07:57:15.422 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:15.418+0000 7f4a5bfff700 1 --2- 172.21.15.59:0/354120497 >> v2:172.21.15.76:3300/0 conn(0x7f4a5c1440f0 0x7f4a5c06e510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:57:15.423 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:15.419+0000 7f4a5affd700 1 -- 172.21.15.59:0/354120497 >> v2:172.21.15.76:3300/0 conn(0x7f4a5c1440f0 msgr2=0x7f4a5c06e510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T07:57:15.423 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:15.419+0000 7f4a5affd700 1 --2- 172.21.15.59:0/354120497 >> v2:172.21.15.76:3300/0 conn(0x7f4a5c1440f0 0x7f4a5c06e510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:15.423 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:15.419+0000 7f4a5affd700 1 -- 172.21.15.59:0/354120497 >> v1:172.21.15.59:6789/0 conn(0x7f4a5c143ae0 legacy=0x7f4a5c071bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:57:15.423 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:15.419+0000 7f4a5affd700 1 -- 172.21.15.59:0/354120497 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4a5c0b3dc0 con 0x7f4a5c0722c0 2023-11-24T07:57:15.423 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:15.419+0000 7f4a5a7fc700 1 -- 172.21.15.59:0/354120497 <== mon.0 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1971509816 0 0) 0x7f4a5c143ec0 con 0x7f4a5c143ae0 2023-11-24T07:57:15.423 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:15.419+0000 7f4a5bfff700 1 --2- 172.21.15.59:0/354120497 >> v2:172.21.15.76:3300/0 conn(0x7f4a5c1440f0 0x7f4a5c06e510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-24T07:57:15.423 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:15.419+0000 7f4a5affd700 1 --2- 172.21.15.59:0/354120497 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4a5c0722c0 0x7f4a5c0b51b0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f4a5c15f950 tx=0x7f4a4400bdc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=face3e4f0984433c server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:57:15.423 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:15.419+0000 7f4a5a7fc700 1 -- 172.21.15.59:0/354120497 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a4400dc70 con 0x7f4a5c0722c0 2023-11-24T07:57:15.423 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:15.419+0000 7f4a5a7fc700 1 -- 172.21.15.59:0/354120497 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4a4400ddd0 con 0x7f4a5c0722c0 2023-11-24T07:57:15.423 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:15.419+0000 7f4a5a7fc700 1 -- 172.21.15.59:0/354120497 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a44010690 con 0x7f4a5c0722c0 2023-11-24T07:57:15.423 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:15.419+0000 7f4a61ee9700 1 -- 172.21.15.59:0/354120497 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4a5c0722c0 msgr2=0x7f4a5c0b51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:57:15.424 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:15.420+0000 7f4a61ee9700 1 --2- 172.21.15.59:0/354120497 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4a5c0722c0 0x7f4a5c0b51b0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f4a5c15f950 tx=0x7f4a4400bdc0 comp rx=0 tx=0).stop 2023-11-24T07:57:15.424 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:15.420+0000 7f4a61ee9700 1 -- 172.21.15.59:0/354120497 shutdown_connections 2023-11-24T07:57:15.424 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:15.420+0000 7f4a61ee9700 1 --2- 172.21.15.59:0/354120497 >> v2:172.21.15.76:3300/0 conn(0x7f4a5c1440f0 0x7f4a5c06e510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:15.424 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:15.420+0000 7f4a61ee9700 1 --2- 172.21.15.59:0/354120497 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4a5c0722c0 0x7f4a5c0b51b0 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:15.424 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:15.420+0000 7f4a61ee9700 1 -- 172.21.15.59:0/354120497 >> 172.21.15.59:0/354120497 conn(0x7f4a5c0b2740 msgr2=0x7f4a5c0b2b20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:57:15.424 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:15.420+0000 7f4a61ee9700 1 -- 172.21.15.59:0/354120497 shutdown_connections 2023-11-24T07:57:15.424 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:15.420+0000 7f4a61ee9700 1 -- 172.21.15.59:0/354120497 wait complete. 2023-11-24T07:57:15.427 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:15.421+0000 7f4a61ee9700 1 Processor -- start 2023-11-24T07:57:15.427 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:15.421+0000 7f4a61ee9700 1 -- start start 2023-11-24T07:57:15.427 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:15.422+0000 7f4a61ee9700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4a5c0722c0 0x7f4a5c1f16b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:57:15.427 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:15.422+0000 7f4a61ee9700 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f4a5c143ae0 0x7f4a5c15be70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:57:15.427 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:15.422+0000 7f4a61ee9700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4a5c1440f0 0x7f4a5c13f470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:57:15.427 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:15.422+0000 7f4a61ee9700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a5c15c3b0 con 0x7f4a5c0722c0 2023-11-24T07:57:15.427 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:15.422+0000 7f4a61ee9700 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a5c1f1dc0 con 0x7f4a5c1440f0 2023-11-24T07:57:15.427 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:15.422+0000 7f4a61ee9700 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4a5c1f20a0 con 0x7f4a5c143ae0 2023-11-24T07:57:15.427 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:15.422+0000 7f4a5b7fe700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4a5c0722c0 0x7f4a5c1f16b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:57:15.427 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:15.423+0000 7f4a5b7fe700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4a5c0722c0 0x7f4a5c1f16b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:51396/0 (socket says 172.21.15.59:51396) 2023-11-24T07:57:15.427 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:15.423+0000 7f4a5b7fe700 1 -- 172.21.15.59:0/502249554 learned_addr learned my addr 172.21.15.59:0/502249554 (peer_addr_for_me v2:172.21.15.59:0/0) 2023-11-24T07:57:15.428 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:15.423+0000 7f4a5affd700 1 --2- 172.21.15.59:0/502249554 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f4a5c143ae0 0x7f4a5c15be70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:57:15.428 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:15.423+0000 7f4a5bfff700 1 --2- 172.21.15.59:0/502249554 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4a5c1440f0 0x7f4a5c13f470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:57:15.428 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:15.423+0000 7f4a5b7fe700 1 -- 172.21.15.59:0/502249554 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f4a5c143ae0 msgr2=0x7f4a5c15be70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:57:15.428 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:15.423+0000 7f4a5b7fe700 1 --2- 172.21.15.59:0/502249554 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f4a5c143ae0 0x7f4a5c15be70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:15.428 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:15.423+0000 7f4a5b7fe700 1 -- 172.21.15.59:0/502249554 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4a5c1440f0 msgr2=0x7f4a5c13f470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:57:15.428 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:15.423+0000 7f4a5b7fe700 1 --2- 172.21.15.59:0/502249554 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4a5c1440f0 0x7f4a5c13f470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:15.428 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:15.423+0000 7f4a5b7fe700 1 -- 172.21.15.59:0/502249554 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4a4400b740 con 0x7f4a5c0722c0 2023-11-24T07:57:15.428 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:15.423+0000 7f4a5affd700 1 --2- 172.21.15.59:0/502249554 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f4a5c143ae0 0x7f4a5c15be70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-24T07:57:15.428 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:15.423+0000 7f4a5b7fe700 1 --2- 172.21.15.59:0/502249554 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4a5c0722c0 0x7f4a5c1f16b0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f4a4c0011e0 tx=0x7f4a4c00cae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:57:15.428 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:15.423+0000 7f4a53fff700 1 -- 172.21.15.59:0/502249554 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a4c004c80 con 0x7f4a5c0722c0 2023-11-24T07:57:15.428 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:15.423+0000 7f4a53fff700 1 -- 172.21.15.59:0/502249554 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4a4c004e00 con 0x7f4a5c0722c0 2023-11-24T07:57:15.428 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:15.423+0000 7f4a61ee9700 1 -- 172.21.15.59:0/502249554 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4a5c1f2380 con 0x7f4a5c0722c0 2023-11-24T07:57:15.428 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:15.423+0000 7f4a53fff700 1 -- 172.21.15.59:0/502249554 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a4c007310 con 0x7f4a5c0722c0 2023-11-24T07:57:15.428 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:15.423+0000 7f4a61ee9700 1 -- 172.21.15.59:0/502249554 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4a5c1f2940 con 0x7f4a5c0722c0 2023-11-24T07:57:15.429 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:15.424+0000 7f4a61ee9700 1 -- 172.21.15.59:0/502249554 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4a5c068670 con 0x7f4a5c0722c0 2023-11-24T07:57:15.429 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:15.425+0000 7f4a53fff700 1 -- 172.21.15.59:0/502249554 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f4a4c007470 con 0x7f4a5c0722c0 2023-11-24T07:57:15.429 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:15.425+0000 7f4a53fff700 1 --2- 172.21.15.59:0/502249554 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f4a3c06aed0 0x7f4a3c06d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:57:15.429 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:15.425+0000 7f4a53fff700 1 -- 172.21.15.59:0/502249554 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5824+0+0 (secure 0 0 0) 0x7f4a4c0893d0 con 0x7f4a5c0722c0 2023-11-24T07:57:15.429 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:15.425+0000 7f4a5affd700 1 --2- 172.21.15.59:0/502249554 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f4a3c06aed0 0x7f4a3c06d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:57:15.429 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:15.426+0000 7f4a5affd700 1 --2- 172.21.15.59:0/502249554 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f4a3c06aed0 0x7f4a3c06d390 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f4a4400be00 tx=0x7f4a44013000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:57:15.431 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:15.430+0000 7f4a53fff700 1 -- 172.21.15.59:0/502249554 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f4a4c0544a0 con 0x7f4a5c0722c0 2023-11-24T07:57:15.632 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:15.631+0000 7f4a61ee9700 1 -- 172.21.15.59:0/502249554 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f4a5c0b5d20 con 0x7f4a5c0722c0 2023-11-24T07:57:16.451 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:16.449+0000 7f4a53fff700 1 -- 172.21.15.59:0/502249554 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v24) v1 ==== 181+0+0 (secure 0 0 0) 0x7f4a4c057ac0 con 0x7f4a5c0722c0 2023-11-24T07:57:16.455 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:16.453+0000 7f4a61ee9700 1 -- 172.21.15.59:0/502249554 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f4a3c06aed0 msgr2=0x7f4a3c06d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:57:16.455 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:16.453+0000 7f4a61ee9700 1 --2- 172.21.15.59:0/502249554 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f4a3c06aed0 0x7f4a3c06d390 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f4a4400be00 tx=0x7f4a44013000 comp rx=0 tx=0).stop 2023-11-24T07:57:16.455 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:16.453+0000 7f4a61ee9700 1 -- 172.21.15.59:0/502249554 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4a5c0722c0 msgr2=0x7f4a5c1f16b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:57:16.455 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:16.453+0000 7f4a61ee9700 1 --2- 172.21.15.59:0/502249554 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4a5c0722c0 0x7f4a5c1f16b0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f4a4c0011e0 tx=0x7f4a4c00cae0 comp rx=0 tx=0).stop 2023-11-24T07:57:16.456 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:16.454+0000 7f4a61ee9700 1 -- 172.21.15.59:0/502249554 shutdown_connections 2023-11-24T07:57:16.456 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:16.454+0000 7f4a61ee9700 1 --2- 172.21.15.59:0/502249554 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f4a3c06aed0 0x7f4a3c06d390 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:16.456 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:16.454+0000 7f4a61ee9700 1 --2- 172.21.15.59:0/502249554 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4a5c1440f0 0x7f4a5c13f470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:16.456 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:16.454+0000 7f4a61ee9700 1 --2- 172.21.15.59:0/502249554 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4a5c0722c0 0x7f4a5c1f16b0 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:16.456 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:16.454+0000 7f4a61ee9700 1 --2- 172.21.15.59:0/502249554 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f4a5c143ae0 0x7f4a5c15be70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:16.456 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:16.454+0000 7f4a61ee9700 1 -- 172.21.15.59:0/502249554 >> 172.21.15.59:0/502249554 conn(0x7f4a5c0b2740 msgr2=0x7f4a5c0b4430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:57:16.456 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:16.454+0000 7f4a61ee9700 1 -- 172.21.15.59:0/502249554 shutdown_connections 2023-11-24T07:57:16.456 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:16.454+0000 7f4a61ee9700 1 -- 172.21.15.59:0/502249554 wait complete. 2023-11-24T07:57:16.457 INFO:teuthology.orchestra.run.smithi059.stderr:pool 'cephfs_metadata' removed 2023-11-24T07:57:16.473 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2023-11-24T07:57:16.612 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:16.610+0000 7fe2348e9700 1 Processor -- start 2023-11-24T07:57:16.612 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:16.610+0000 7fe2348e9700 1 -- start start 2023-11-24T07:57:16.614 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:16.611+0000 7fe2348e9700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fe23006fc50 0x7fe23006eef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:57:16.614 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:16.611+0000 7fe2348e9700 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fe2300b0e90 0x7fe23006ca10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:57:16.614 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:16.611+0000 7fe2348e9700 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe2300d1230 con 0x7fe23015e1c0 2023-11-24T07:57:16.614 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:16.611+0000 7fe2348e9700 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe2300d0c70 con 0x7fe23006fc50 2023-11-24T07:57:16.615 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:16.611+0000 7fe2348e9700 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe2300d0f50 con 0x7fe2300b0e90 2023-11-24T07:57:16.615 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:16.611+0000 7fe22dd9b700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fe23006fc50 0x7fe23006eef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:57:16.615 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:16.611+0000 7fe22dd9b700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fe23006fc50 0x7fe23006eef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:51404/0 (socket says 172.21.15.59:51404) 2023-11-24T07:57:16.615 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:16.611+0000 7fe22dd9b700 1 -- 172.21.15.59:0/3279052131 learned_addr learned my addr 172.21.15.59:0/3279052131 (peer_addr_for_me v2:172.21.15.59:0/0) 2023-11-24T07:57:16.615 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:16.611+0000 7fe22e59c700 1 --1- 172.21.15.59:0/3279052131 >> v1:172.21.15.59:6789/0 conn(0x7fe23015e1c0 0x7fe23006f540 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:54878/0 (socket says 172.21.15.59:54878) 2023-11-24T07:57:16.615 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:16.612+0000 7fe22ed9d700 1 --2- 172.21.15.59:0/3279052131 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fe2300b0e90 0x7fe23006ca10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:57:16.615 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:16.612+0000 7fe22dd9b700 1 -- 172.21.15.59:0/3279052131 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fe2300b0e90 msgr2=0x7fe23006ca10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T07:57:16.615 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:16.612+0000 7fe22dd9b700 1 --2- 172.21.15.59:0/3279052131 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fe2300b0e90 0x7fe23006ca10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:16.615 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:16.612+0000 7fe22dd9b700 1 -- 172.21.15.59:0/3279052131 >> v1:172.21.15.59:6789/0 conn(0x7fe23015e1c0 legacy=0x7fe23006f540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:57:16.615 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:16.612+0000 7fe22dd9b700 1 -- 172.21.15.59:0/3279052131 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe23006c210 con 0x7fe23006fc50 2023-11-24T07:57:16.616 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:16.612+0000 7fe22d59a700 1 -- 172.21.15.59:0/3279052131 <== mon.0 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2625683667 0 0) 0x7fe2300d1230 con 0x7fe23015e1c0 2023-11-24T07:57:16.616 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:16.612+0000 7fe22ed9d700 1 --2- 172.21.15.59:0/3279052131 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fe2300b0e90 0x7fe23006ca10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T07:57:16.616 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:16.612+0000 7fe22dd9b700 1 --2- 172.21.15.59:0/3279052131 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe23006fc50 0x7fe23006eef0 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7fe2200098f0 tx=0x7fe220009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7d527fdef149dc41 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:57:16.616 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:16.612+0000 7fe22d59a700 1 -- 172.21.15.59:0/3279052131 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe22000b8a0 con 0x7fe23006fc50 2023-11-24T07:57:16.616 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:16.612+0000 7fe22d59a700 1 -- 172.21.15.59:0/3279052131 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe22000ba00 con 0x7fe23006fc50 2023-11-24T07:57:16.616 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:16.612+0000 7fe22d59a700 1 -- 172.21.15.59:0/3279052131 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe22000bd90 con 0x7fe23006fc50 2023-11-24T07:57:16.616 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:16.612+0000 7fe2348e9700 1 -- 172.21.15.59:0/3279052131 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe23006fc50 msgr2=0x7fe23006eef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:57:16.616 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:16.612+0000 7fe2348e9700 1 --2- 172.21.15.59:0/3279052131 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe23006fc50 0x7fe23006eef0 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7fe2200098f0 tx=0x7fe220009c00 comp rx=0 tx=0).stop 2023-11-24T07:57:16.616 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:16.613+0000 7fe2348e9700 1 -- 172.21.15.59:0/3279052131 shutdown_connections 2023-11-24T07:57:16.616 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:16.613+0000 7fe2348e9700 1 --2- 172.21.15.59:0/3279052131 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fe2300b0e90 0x7fe23006ca10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:16.616 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:16.613+0000 7fe2348e9700 1 --2- 172.21.15.59:0/3279052131 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe23006fc50 0x7fe23006eef0 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:16.617 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:16.613+0000 7fe2348e9700 1 -- 172.21.15.59:0/3279052131 >> 172.21.15.59:0/3279052131 conn(0x7fe2300b3720 msgr2=0x7fe2300b3b30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:57:16.617 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:16.613+0000 7fe2348e9700 1 -- 172.21.15.59:0/3279052131 shutdown_connections 2023-11-24T07:57:16.617 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:16.613+0000 7fe2348e9700 1 -- 172.21.15.59:0/3279052131 wait complete. 2023-11-24T07:57:16.619 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:16.614+0000 7fe2348e9700 1 Processor -- start 2023-11-24T07:57:16.619 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:16.614+0000 7fe2348e9700 1 -- start start 2023-11-24T07:57:16.619 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:16.615+0000 7fe2348e9700 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fe23006fc50 0x7fe2300b7600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:57:16.619 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:16.615+0000 7fe2348e9700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe2300b0e90 0x7fe2301f4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:57:16.620 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:16.615+0000 7fe2348e9700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe23015e1c0 0x7fe2300b17e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:57:16.620 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:16.615+0000 7fe2348e9700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2300b4810 con 0x7fe2300b0e90 2023-11-24T07:57:16.620 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:16.615+0000 7fe2348e9700 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2301f49d0 con 0x7fe23015e1c0 2023-11-24T07:57:16.620 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:16.615+0000 7fe2348e9700 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe2300b4a60 con 0x7fe23006fc50 2023-11-24T07:57:16.620 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:16.615+0000 7fe22dd9b700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe2300b0e90 0x7fe2301f4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:57:16.620 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:16.615+0000 7fe22dd9b700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe2300b0e90 0x7fe2301f4d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:51410/0 (socket says 172.21.15.59:51410) 2023-11-24T07:57:16.620 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:16.615+0000 7fe22dd9b700 1 -- 172.21.15.59:0/821438444 learned_addr learned my addr 172.21.15.59:0/821438444 (peer_addr_for_me v2:172.21.15.59:0/0) 2023-11-24T07:57:16.620 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:16.615+0000 7fe22e59c700 1 --2- 172.21.15.59:0/821438444 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fe23006fc50 0x7fe2300b7600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:57:16.620 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:16.615+0000 7fe22ed9d700 1 --2- 172.21.15.59:0/821438444 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe23015e1c0 0x7fe2300b17e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:57:16.620 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:16.615+0000 7fe22dd9b700 1 -- 172.21.15.59:0/821438444 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fe23006fc50 msgr2=0x7fe2300b7600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:57:16.620 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:16.615+0000 7fe22dd9b700 1 --2- 172.21.15.59:0/821438444 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fe23006fc50 0x7fe2300b7600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:16.621 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:16.615+0000 7fe22dd9b700 1 -- 172.21.15.59:0/821438444 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe23015e1c0 msgr2=0x7fe2300b17e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:57:16.621 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:16.615+0000 7fe22dd9b700 1 --2- 172.21.15.59:0/821438444 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe23015e1c0 0x7fe2300b17e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:16.621 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:16.615+0000 7fe22dd9b700 1 -- 172.21.15.59:0/821438444 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe220009550 con 0x7fe2300b0e90 2023-11-24T07:57:16.621 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:16.615+0000 7fe22e59c700 1 --2- 172.21.15.59:0/821438444 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fe23006fc50 0x7fe2300b7600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-24T07:57:16.621 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:16.615+0000 7fe22ed9d700 1 --2- 172.21.15.59:0/821438444 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe23015e1c0 0x7fe2300b17e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-24T07:57:16.621 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:16.615+0000 7fe22dd9b700 1 --2- 172.21.15.59:0/821438444 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe2300b0e90 0x7fe2301f4d40 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7fe220000c00 tx=0x7fe220004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:57:16.621 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:16.616+0000 7fe21effd700 1 -- 172.21.15.59:0/821438444 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2200036a0 con 0x7fe2300b0e90 2023-11-24T07:57:16.621 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:16.616+0000 7fe21effd700 1 -- 172.21.15.59:0/821438444 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe22001f070 con 0x7fe2300b0e90 2023-11-24T07:57:16.621 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:16.616+0000 7fe2348e9700 1 -- 172.21.15.59:0/821438444 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe2300b4d40 con 0x7fe2300b0e90 2023-11-24T07:57:16.621 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:16.616+0000 7fe21effd700 1 -- 172.21.15.59:0/821438444 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe220003a20 con 0x7fe2300b0e90 2023-11-24T07:57:16.622 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:16.616+0000 7fe2348e9700 1 -- 172.21.15.59:0/821438444 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe2300b1d20 con 0x7fe2300b0e90 2023-11-24T07:57:16.622 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:16.617+0000 7fe2348e9700 1 -- 172.21.15.59:0/821438444 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe230068670 con 0x7fe2300b0e90 2023-11-24T07:57:16.622 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:16.617+0000 7fe21effd700 1 -- 172.21.15.59:0/821438444 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fe220003b80 con 0x7fe2300b0e90 2023-11-24T07:57:16.622 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:16.617+0000 7fe21effd700 1 --2- 172.21.15.59:0/821438444 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7fe21006aed0 0x7fe21006d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:57:16.622 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:16.618+0000 7fe21effd700 1 -- 172.21.15.59:0/821438444 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 5386+0+0 (secure 0 0 0) 0x7fe220021070 con 0x7fe2300b0e90 2023-11-24T07:57:16.622 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:16.618+0000 7fe22e59c700 1 --2- 172.21.15.59:0/821438444 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7fe21006aed0 0x7fe21006d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:57:16.622 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:16.618+0000 7fe22e59c700 1 --2- 172.21.15.59:0/821438444 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7fe21006aed0 0x7fe21006d390 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fe224000f60 tx=0x7fe224007d50 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:57:16.624 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:16.622+0000 7fe21effd700 1 -- 172.21.15.59:0/821438444 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fe220056050 con 0x7fe2300b0e90 2023-11-24T07:57:16.838 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:16.835+0000 7fe2348e9700 1 -- 172.21.15.59:0/821438444 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fe23015a800 con 0x7fe2300b0e90 2023-11-24T07:57:17.453 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:17.451+0000 7fe21effd700 1 -- 172.21.15.59:0/821438444 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v25) v1 ==== 169+0+0 (secure 0 0 0) 0x7fe220059670 con 0x7fe2300b0e90 2023-11-24T07:57:17.456 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:17.454+0000 7fe2348e9700 1 -- 172.21.15.59:0/821438444 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7fe21006aed0 msgr2=0x7fe21006d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:57:17.456 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:17.454+0000 7fe2348e9700 1 --2- 172.21.15.59:0/821438444 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7fe21006aed0 0x7fe21006d390 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fe224000f60 tx=0x7fe224007d50 comp rx=0 tx=0).stop 2023-11-24T07:57:17.456 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:17.454+0000 7fe2348e9700 1 -- 172.21.15.59:0/821438444 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe2300b0e90 msgr2=0x7fe2301f4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:57:17.456 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:17.454+0000 7fe2348e9700 1 --2- 172.21.15.59:0/821438444 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe2300b0e90 0x7fe2301f4d40 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7fe220000c00 tx=0x7fe220004750 comp rx=0 tx=0).stop 2023-11-24T07:57:17.456 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:17.454+0000 7fe2348e9700 1 -- 172.21.15.59:0/821438444 shutdown_connections 2023-11-24T07:57:17.457 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:17.454+0000 7fe2348e9700 1 --2- 172.21.15.59:0/821438444 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7fe21006aed0 0x7fe21006d390 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:17.457 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:17.454+0000 7fe2348e9700 1 --2- 172.21.15.59:0/821438444 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe23015e1c0 0x7fe2300b17e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:17.457 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:17.454+0000 7fe2348e9700 1 --2- 172.21.15.59:0/821438444 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fe23006fc50 0x7fe2300b7600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:17.457 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:17.454+0000 7fe2348e9700 1 --2- 172.21.15.59:0/821438444 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe2300b0e90 0x7fe2301f4d40 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:17.457 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:17.454+0000 7fe2348e9700 1 -- 172.21.15.59:0/821438444 >> 172.21.15.59:0/821438444 conn(0x7fe2300b3720 msgr2=0x7fe23015b220 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:57:17.457 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:17.454+0000 7fe2348e9700 1 -- 172.21.15.59:0/821438444 shutdown_connections 2023-11-24T07:57:17.457 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:17.454+0000 7fe2348e9700 1 -- 172.21.15.59:0/821438444 wait complete. 2023-11-24T07:57:17.457 INFO:teuthology.orchestra.run.smithi059.stderr:pool 'cephfs_data' removed 2023-11-24T07:57:17.472 INFO:tasks.ceph.mds.a:Restarting daemon 2023-11-24T07:57:17.472 INFO:tasks.ceph.mds.a:Stopping old one... 2023-11-24T07:57:17.472 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2023-11-24T07:57:17.472 INFO:teuthology.orchestra.run:waiting for 300 2023-11-24T07:57:17.472 INFO:tasks.ceph.mds.c:Restarting daemon 2023-11-24T07:57:17.472 INFO:tasks.ceph.mds.c:Stopping old one... 2023-11-24T07:57:17.473 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2023-11-24T07:57:17.473 INFO:teuthology.orchestra.run:waiting for 300 2023-11-24T07:57:17.473 INFO:tasks.ceph.mds.b:Restarting daemon 2023-11-24T07:57:17.473 INFO:tasks.ceph.mds.b:Stopping old one... 2023-11-24T07:57:17.473 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2023-11-24T07:57:17.473 INFO:teuthology.orchestra.run:waiting for 300 2023-11-24T07:57:17.473 INFO:tasks.ceph.mds.d:Restarting daemon 2023-11-24T07:57:17.474 INFO:tasks.ceph.mds.d:Stopping old one... 2023-11-24T07:57:17.474 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2023-11-24T07:57:17.474 INFO:teuthology.orchestra.run:waiting for 300 2023-11-24T07:57:17.483 INFO:tasks.ceph.mds.a:Stopped 2023-11-24T07:57:17.483 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2023-11-24T07:57:17.484 INFO:tasks.ceph.mds.c:Stopped 2023-11-24T07:57:17.484 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2023-11-24T07:57:17.486 INFO:tasks.ceph.mds.b:Stopped 2023-11-24T07:57:17.486 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2023-11-24T07:57:17.486 INFO:tasks.ceph.mds.d:Stopped 2023-11-24T07:57:17.487 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2023-11-24T07:57:17.526 INFO:tasks.ceph.mds.a:Started 2023-11-24T07:57:17.526 DEBUG:teuthology.parallel:result is None 2023-11-24T07:57:17.527 INFO:tasks.ceph.mds.c:Started 2023-11-24T07:57:17.527 DEBUG:teuthology.parallel:result is None 2023-11-24T07:57:17.529 INFO:tasks.ceph.mds.b:Started 2023-11-24T07:57:17.529 DEBUG:teuthology.parallel:result is None 2023-11-24T07:57:17.530 INFO:tasks.ceph.mds.d:Started 2023-11-24T07:57:17.530 DEBUG:teuthology.parallel:result is None 2023-11-24T07:57:17.530 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2023-11-24T07:57:17.613 INFO:tasks.ceph.mds.a.smithi059.stdout:starting mds.a at 2023-11-24T07:57:17.613 INFO:tasks.ceph.mds.c.smithi059.stdout:starting mds.c at 2023-11-24T07:57:17.613 INFO:tasks.ceph.mds.d.smithi076.stdout:starting mds.d at 2023-11-24T07:57:17.614 INFO:tasks.ceph.mds.b.smithi076.stdout:starting mds.b at 2023-11-24T07:57:17.664 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:17.662+0000 7fa9e949b700 1 Processor -- start 2023-11-24T07:57:17.664 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:17.662+0000 7fa9e949b700 1 -- start start 2023-11-24T07:57:17.666 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:17.663+0000 7fa9e949b700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fa9e41448b0 0x7fa9e406ec70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:57:17.666 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:17.663+0000 7fa9e949b700 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fa9e406f990 0x7fa9e40ce2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:57:17.666 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:17.663+0000 7fa9e949b700 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa9e4144c90 con 0x7fa9e40b44c0 2023-11-24T07:57:17.666 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:17.663+0000 7fa9e949b700 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa9e40b4c80 con 0x7fa9e41448b0 2023-11-24T07:57:17.667 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:17.663+0000 7fa9e949b700 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa9e40b4f60 con 0x7fa9e406f990 2023-11-24T07:57:17.667 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:17.663+0000 7fa9e2ffd700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fa9e41448b0 0x7fa9e406ec70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:57:17.667 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:17.663+0000 7fa9e2ffd700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fa9e41448b0 0x7fa9e406ec70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:51440/0 (socket says 172.21.15.59:51440) 2023-11-24T07:57:17.667 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:17.663+0000 7fa9e2ffd700 1 -- 172.21.15.59:0/1832802885 learned_addr learned my addr 172.21.15.59:0/1832802885 (peer_addr_for_me v2:172.21.15.59:0/0) 2023-11-24T07:57:17.667 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:17.663+0000 7fa9e37fe700 1 --2- 172.21.15.59:0/1832802885 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fa9e406f990 0x7fa9e40ce2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:57:17.667 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:17.663+0000 7fa9e2ffd700 1 -- 172.21.15.59:0/1832802885 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fa9e406f990 msgr2=0x7fa9e40ce2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T07:57:17.667 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:17.663+0000 7fa9e2ffd700 1 --2- 172.21.15.59:0/1832802885 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fa9e406f990 0x7fa9e40ce2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:17.667 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:17.663+0000 7fa9e2ffd700 1 -- 172.21.15.59:0/1832802885 >> v1:172.21.15.76:6789/0 conn(0x7fa9e40b44c0 legacy=0x7fa9e406f280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T07:57:17.667 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:17.663+0000 7fa9e2ffd700 1 -- 172.21.15.59:0/1832802885 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa9e40b5240 con 0x7fa9e41448b0 2023-11-24T07:57:17.667 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:17.664+0000 7fa9e2ffd700 1 --2- 172.21.15.59:0/1832802885 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa9e41448b0 0x7fa9e406ec70 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7fa9dc00bb90 tx=0x7fa9dc00bea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bc73ddc643fd7d60 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:57:17.667 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:17.664+0000 7fa9e1ffb700 1 -- 172.21.15.59:0/1832802885 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9dc00dd10 con 0x7fa9e41448b0 2023-11-24T07:57:17.667 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:17.664+0000 7fa9e1ffb700 1 -- 172.21.15.59:0/1832802885 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa9dc00de70 con 0x7fa9e41448b0 2023-11-24T07:57:17.668 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:17.664+0000 7fa9e1ffb700 1 -- 172.21.15.59:0/1832802885 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9dc004230 con 0x7fa9e41448b0 2023-11-24T07:57:17.668 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:17.664+0000 7fa9e949b700 1 -- 172.21.15.59:0/1832802885 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa9e41448b0 msgr2=0x7fa9e406ec70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:57:17.668 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:17.664+0000 7fa9e949b700 1 --2- 172.21.15.59:0/1832802885 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa9e41448b0 0x7fa9e406ec70 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7fa9dc00bb90 tx=0x7fa9dc00bea0 comp rx=0 tx=0).stop 2023-11-24T07:57:17.668 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:17.664+0000 7fa9e949b700 1 -- 172.21.15.59:0/1832802885 shutdown_connections 2023-11-24T07:57:17.668 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:17.664+0000 7fa9e949b700 1 --2- 172.21.15.59:0/1832802885 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fa9e406f990 0x7fa9e40ce2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:17.668 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:17.664+0000 7fa9e949b700 1 --2- 172.21.15.59:0/1832802885 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa9e41448b0 0x7fa9e406ec70 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:17.668 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:17.664+0000 7fa9e949b700 1 -- 172.21.15.59:0/1832802885 >> 172.21.15.59:0/1832802885 conn(0x7fa9e40d0bc0 msgr2=0x7fa9e40d0fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:57:17.668 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:17.664+0000 7fa9e949b700 1 -- 172.21.15.59:0/1832802885 shutdown_connections 2023-11-24T07:57:17.668 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:17.664+0000 7fa9e949b700 1 -- 172.21.15.59:0/1832802885 wait complete. 2023-11-24T07:57:17.671 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:17.666+0000 7fa9e949b700 1 Processor -- start 2023-11-24T07:57:17.671 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:17.666+0000 7fa9e949b700 1 -- start start 2023-11-24T07:57:17.671 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:17.667+0000 7fa9e949b700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa9e406f990 0x7fa9e41f6bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:57:17.672 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:17.667+0000 7fa9e949b700 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fa9e40b44c0 0x7fa9e40caf60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:57:17.672 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:17.667+0000 7fa9e949b700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa9e41448b0 0x7fa9e4140530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:57:17.672 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:17.667+0000 7fa9e949b700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9e41f7110 con 0x7fa9e41448b0 2023-11-24T07:57:17.672 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:17.667+0000 7fa9e949b700 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9e41f3c20 con 0x7fa9e406f990 2023-11-24T07:57:17.672 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:17.667+0000 7fa9e949b700 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa9e40cba20 con 0x7fa9e40b44c0 2023-11-24T07:57:17.672 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:17.667+0000 7fa9e37fe700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa9e41448b0 0x7fa9e4140530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:57:17.672 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:17.667+0000 7fa9e37fe700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa9e41448b0 0x7fa9e4140530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:51450/0 (socket says 172.21.15.59:51450) 2023-11-24T07:57:17.672 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:17.667+0000 7fa9e37fe700 1 -- 172.21.15.59:0/1995443497 learned_addr learned my addr 172.21.15.59:0/1995443497 (peer_addr_for_me v2:172.21.15.59:0/0) 2023-11-24T07:57:17.672 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:17.667+0000 7fa9e2ffd700 1 --2- 172.21.15.59:0/1995443497 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa9e406f990 0x7fa9e41f6bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:57:17.672 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:17.667+0000 7fa9e27fc700 1 --2- 172.21.15.59:0/1995443497 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fa9e40b44c0 0x7fa9e40caf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:57:17.673 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:17.667+0000 7fa9e37fe700 1 -- 172.21.15.59:0/1995443497 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fa9e40b44c0 msgr2=0x7fa9e40caf60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:57:17.673 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:17.667+0000 7fa9e37fe700 1 --2- 172.21.15.59:0/1995443497 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fa9e40b44c0 0x7fa9e40caf60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:17.673 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:17.667+0000 7fa9e37fe700 1 -- 172.21.15.59:0/1995443497 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa9e406f990 msgr2=0x7fa9e41f6bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:57:17.673 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:17.667+0000 7fa9e37fe700 1 --2- 172.21.15.59:0/1995443497 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa9e406f990 0x7fa9e41f6bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:17.673 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:17.667+0000 7fa9e37fe700 1 -- 172.21.15.59:0/1995443497 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa9dc00b7f0 con 0x7fa9e41448b0 2023-11-24T07:57:17.673 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:17.667+0000 7fa9e2ffd700 1 --2- 172.21.15.59:0/1995443497 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa9e406f990 0x7fa9e41f6bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-24T07:57:17.673 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:17.667+0000 7fa9e37fe700 1 --2- 172.21.15.59:0/1995443497 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa9e41448b0 0x7fa9e4140530 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7fa9d4007de0 tx=0x7fa9d400e3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:57:17.673 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:17.667+0000 7fa9cb7fe700 1 -- 172.21.15.59:0/1995443497 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9d400ede0 con 0x7fa9e41448b0 2023-11-24T07:57:17.673 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:17.667+0000 7fa9cb7fe700 1 -- 172.21.15.59:0/1995443497 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa9d4014420 con 0x7fa9e41448b0 2023-11-24T07:57:17.674 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:17.667+0000 7fa9cb7fe700 1 -- 172.21.15.59:0/1995443497 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9d40147d0 con 0x7fa9e41448b0 2023-11-24T07:57:17.674 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:17.667+0000 7fa9e949b700 1 -- 172.21.15.59:0/1995443497 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa9e40cbd00 con 0x7fa9e41448b0 2023-11-24T07:57:17.674 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:17.667+0000 7fa9e949b700 1 -- 172.21.15.59:0/1995443497 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa9e40cc2c0 con 0x7fa9e41448b0 2023-11-24T07:57:17.674 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:17.668+0000 7fa9e949b700 1 -- 172.21.15.59:0/1995443497 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa9e40b4920 con 0x7fa9e41448b0 2023-11-24T07:57:17.674 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:17.669+0000 7fa9cb7fe700 1 -- 172.21.15.59:0/1995443497 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fa9d4014930 con 0x7fa9e41448b0 2023-11-24T07:57:17.674 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:17.669+0000 7fa9cb7fe700 1 --2- 172.21.15.59:0/1995443497 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7fa9d006aed0 0x7fa9d006d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:57:17.674 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:17.669+0000 7fa9cb7fe700 1 -- 172.21.15.59:0/1995443497 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 4943+0+0 (secure 0 0 0) 0x7fa9d4089ae0 con 0x7fa9e41448b0 2023-11-24T07:57:17.674 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:17.669+0000 7fa9e2ffd700 1 --2- 172.21.15.59:0/1995443497 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7fa9d006aed0 0x7fa9d006d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:57:17.674 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:17.670+0000 7fa9e2ffd700 1 --2- 172.21.15.59:0/1995443497 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7fa9d006aed0 0x7fa9d006d390 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fa9dc00b340 tx=0x7fa9dc013000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:57:17.675 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:17.674+0000 7fa9cb7fe700 1 -- 172.21.15.59:0/1995443497 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fa9e40b4920 con 0x7fa9e41448b0 2023-11-24T07:57:17.892 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:17.891+0000 7fa9e949b700 1 -- 172.21.15.59:0/1995443497 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7fa9e40b4920 con 0x7fa9e41448b0 2023-11-24T07:57:18.473 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:18.472+0000 7fa9cb7fe700 1 -- 172.21.15.59:0/1995443497 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v27) v1 ==== 97+0+0 (secure 0 0 0) 0x7fa9e40b4920 con 0x7fa9e41448b0 2023-11-24T07:57:18.476 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:18.474+0000 7fa9e949b700 1 -- 172.21.15.59:0/1995443497 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7fa9d006aed0 msgr2=0x7fa9d006d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:57:18.476 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:18.474+0000 7fa9e949b700 1 --2- 172.21.15.59:0/1995443497 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7fa9d006aed0 0x7fa9d006d390 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fa9dc00b340 tx=0x7fa9dc013000 comp rx=0 tx=0).stop 2023-11-24T07:57:18.477 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:18.474+0000 7fa9e949b700 1 -- 172.21.15.59:0/1995443497 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa9e41448b0 msgr2=0x7fa9e4140530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:57:18.477 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:18.474+0000 7fa9e949b700 1 --2- 172.21.15.59:0/1995443497 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa9e41448b0 0x7fa9e4140530 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7fa9d4007de0 tx=0x7fa9d400e3c0 comp rx=0 tx=0).stop 2023-11-24T07:57:18.477 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:18.474+0000 7fa9e949b700 1 -- 172.21.15.59:0/1995443497 shutdown_connections 2023-11-24T07:57:18.477 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:18.474+0000 7fa9e949b700 1 --2- 172.21.15.59:0/1995443497 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7fa9d006aed0 0x7fa9d006d390 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:18.478 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:18.474+0000 7fa9e949b700 1 --2- 172.21.15.59:0/1995443497 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa9e41448b0 0x7fa9e4140530 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:18.478 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:18.474+0000 7fa9e949b700 1 --2- 172.21.15.59:0/1995443497 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa9e406f990 0x7fa9e41f6bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:18.478 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:18.474+0000 7fa9e949b700 1 --2- 172.21.15.59:0/1995443497 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fa9e40b44c0 0x7fa9e40caf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:18.478 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:18.474+0000 7fa9e949b700 1 -- 172.21.15.59:0/1995443497 >> 172.21.15.59:0/1995443497 conn(0x7fa9e40d0bc0 msgr2=0x7fa9e406dbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:57:18.479 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:18.474+0000 7fa9e949b700 1 -- 172.21.15.59:0/1995443497 shutdown_connections 2023-11-24T07:57:18.479 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:18.474+0000 7fa9e949b700 1 -- 172.21.15.59:0/1995443497 wait complete. 2023-11-24T07:57:18.479 INFO:teuthology.orchestra.run.smithi059.stderr: removed all blocklist entries 2023-11-24T07:57:18.492 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2023-11-24T07:57:18.619 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:18.617+0000 7fe77fef5700 1 Processor -- start 2023-11-24T07:57:18.620 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:18.618+0000 7fe77fef5700 1 -- start start 2023-11-24T07:57:18.628 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:18.618+0000 7fe77fef5700 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fe778144c10 0x7fe77806afd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:57:18.628 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:18.618+0000 7fe77fef5700 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fe778145650 0x7fe77806b510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:57:18.628 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:18.618+0000 7fe77fef5700 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe77806eb80 con 0x7fe778145100 2023-11-24T07:57:18.628 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:18.618+0000 7fe77fef5700 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe77806f500 con 0x7fe778144c10 2023-11-24T07:57:18.628 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:18.618+0000 7fe77fef5700 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe7780cd610 con 0x7fe778145650 2023-11-24T07:57:18.628 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:18.619+0000 7fe77d490700 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7fe778145100 0x7fe778071bb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:54928/0 (socket says 172.21.15.59:54928) 2023-11-24T07:57:18.628 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:18.619+0000 7fe77d490700 1 -- 172.21.15.59:0/1031474219 learned_addr learned my addr 172.21.15.59:0/1031474219 (peer_addr_for_me v1:172.21.15.59:0/0) 2023-11-24T07:57:18.629 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:18.619+0000 7fe77dc91700 1 --2- 172.21.15.59:0/1031474219 >> v2:172.21.15.76:3300/0 conn(0x7fe778144c10 0x7fe77806afd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:57:18.629 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:18.619+0000 7fe77e492700 1 --2- 172.21.15.59:0/1031474219 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fe778145650 0x7fe77806b510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:57:18.629 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:18.619+0000 7fe77cc8f700 1 -- 172.21.15.59:0/1031474219 <== mon.0 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2413301900 0 0) 0x7fe77806eb80 con 0x7fe778145100 2023-11-24T07:57:18.629 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:18.619+0000 7fe77cc8f700 1 -- 172.21.15.59:0/1031474219 --> v1:172.21.15.59:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe7640035b0 con 0x7fe778145100 2023-11-24T07:57:18.629 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:18.619+0000 7fe77cc8f700 1 -- 172.21.15.59:0/1031474219 <== mon.0 v1:172.21.15.59:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1835392215 0 0) 0x7fe7640035b0 con 0x7fe778145100 2023-11-24T07:57:18.629 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:18.619+0000 7fe77cc8f700 1 -- 172.21.15.59:0/1031474219 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fe778145650 msgr2=0x7fe77806b510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T07:57:18.629 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:18.619+0000 7fe77cc8f700 1 --2- 172.21.15.59:0/1031474219 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fe778145650 0x7fe77806b510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:18.629 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:18.619+0000 7fe77cc8f700 1 -- 172.21.15.59:0/1031474219 >> v2:172.21.15.76:3300/0 conn(0x7fe778144c10 msgr2=0x7fe77806afd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T07:57:18.629 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:18.619+0000 7fe77cc8f700 1 --2- 172.21.15.59:0/1031474219 >> v2:172.21.15.76:3300/0 conn(0x7fe778144c10 0x7fe77806afd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:18.629 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:18.619+0000 7fe77cc8f700 1 -- 172.21.15.59:0/1031474219 --> v1:172.21.15.59:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe778160720 con 0x7fe778145100 2023-11-24T07:57:18.629 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:18.619+0000 7fe77cc8f700 1 -- 172.21.15.59:0/1031474219 <== mon.0 v1:172.21.15.59:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1312333566 0 0) 0x7fe774001670 con 0x7fe778145100 2023-11-24T07:57:18.630 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:18.619+0000 7fe77cc8f700 1 -- 172.21.15.59:0/1031474219 >> v1:172.21.15.59:6789/0 conn(0x7fe778145100 legacy=0x7fe778071bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:57:18.630 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:18.619+0000 7fe77cc8f700 1 --2- 172.21.15.59:0/1031474219 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe764003a00 0x7fe764003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:57:18.630 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:18.619+0000 7fe77cc8f700 1 --2- 172.21.15.59:0/1031474219 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe764004850 0x7fe764004cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:57:18.630 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:18.619+0000 7fe77cc8f700 1 --2- 172.21.15.59:0/1031474219 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fe778145650 0x7fe764005210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:57:18.631 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:18.619+0000 7fe77cc8f700 1 -- 172.21.15.59:0/1031474219 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe77806f500 con 0x7fe764003a00 2023-11-24T07:57:18.631 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:18.619+0000 7fe77cc8f700 1 -- 172.21.15.59:0/1031474219 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7780cd610 con 0x7fe764004850 2023-11-24T07:57:18.632 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:18.619+0000 7fe77cc8f700 1 -- 172.21.15.59:0/1031474219 --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe76400ba10 con 0x7fe778145650 2023-11-24T07:57:18.632 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:18.619+0000 7fe77dc91700 1 --2- 172.21.15.59:0/1031474219 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe764003a00 0x7fe764003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:57:18.632 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:18.619+0000 7fe77d490700 1 --2- 172.21.15.59:0/1031474219 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe764004850 0x7fe764004cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:57:18.632 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:18.619+0000 7fe77e492700 1 --2- 172.21.15.59:0/1031474219 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fe778145650 0x7fe764005210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:57:18.633 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:18.620+0000 7fe77dc91700 1 -- 172.21.15.59:0/1031474219 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fe778145650 msgr2=0x7fe764005210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T07:57:18.633 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:18.620+0000 7fe77dc91700 1 --2- 172.21.15.59:0/1031474219 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fe778145650 0x7fe764005210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:18.633 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:18.620+0000 7fe77dc91700 1 -- 172.21.15.59:0/1031474219 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe764004850 msgr2=0x7fe764004cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T07:57:18.633 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:18.620+0000 7fe77dc91700 1 --2- 172.21.15.59:0/1031474219 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe764004850 0x7fe764004cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:18.633 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:18.620+0000 7fe77dc91700 1 -- 172.21.15.59:0/1031474219 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe7640035b0 con 0x7fe764003a00 2023-11-24T07:57:18.634 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:18.620+0000 7fe77dc91700 1 --2- 172.21.15.59:0/1031474219 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe764003a00 0x7fe764003e20 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7fe768009b60 tx=0x7fe764005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=32068e75d0cc46b9 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:57:18.634 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:18.620+0000 7fe77cc8f700 1 -- 172.21.15.59:0/1031474219 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe76800e6c0 con 0x7fe764003a00 2023-11-24T07:57:18.634 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:18.620+0000 7fe77cc8f700 1 -- 172.21.15.59:0/1031474219 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe76800e820 con 0x7fe764003a00 2023-11-24T07:57:18.634 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:18.620+0000 7fe77cc8f700 1 -- 172.21.15.59:0/1031474219 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe76800eb90 con 0x7fe764003a00 2023-11-24T07:57:18.634 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:18.620+0000 7fe77fef5700 1 -- 172.21.15.59:0/1031474219 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe764003a00 msgr2=0x7fe764003e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:57:18.634 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:18.620+0000 7fe77fef5700 1 --2- 172.21.15.59:0/1031474219 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe764003a00 0x7fe764003e20 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7fe768009b60 tx=0x7fe764005880 comp rx=0 tx=0).stop 2023-11-24T07:57:18.635 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:18.620+0000 7fe77e492700 1 -- 172.21.15.59:0/1031474219 reap_dead start 2023-11-24T07:57:18.635 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:18.620+0000 7fe77fef5700 1 -- 172.21.15.59:0/1031474219 shutdown_connections 2023-11-24T07:57:18.635 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:18.620+0000 7fe77fef5700 1 -- 172.21.15.59:0/1031474219 >> 172.21.15.59:0/1031474219 conn(0x7fe7780b2740 msgr2=0x7fe7780b2b20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:57:18.635 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:18.621+0000 7fe77fef5700 1 -- 172.21.15.59:0/1031474219 shutdown_connections 2023-11-24T07:57:18.635 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:18.621+0000 7fe77fef5700 1 -- 172.21.15.59:0/1031474219 wait complete. 2023-11-24T07:57:18.635 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:18.622+0000 7fe77fef5700 1 Processor -- start 2023-11-24T07:57:18.636 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:18.622+0000 7fe77fef5700 1 -- start start 2023-11-24T07:57:18.636 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:18.622+0000 7fe77fef5700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe764003a00 0x7fe778141b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:57:18.636 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:18.622+0000 7fe77fef5700 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fe7781401a0 0x7fe7780b0c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:57:18.636 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:18.622+0000 7fe77fef5700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe7781420a0 0x7fe7780b11b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:57:18.636 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:18.622+0000 7fe77fef5700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe778140e30 con 0x7fe7781420a0 2023-11-24T07:57:18.636 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:18.622+0000 7fe77fef5700 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe77815f0d0 con 0x7fe764003a00 2023-11-24T07:57:18.637 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:18.622+0000 7fe77fef5700 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe778141080 con 0x7fe7781401a0 2023-11-24T07:57:18.637 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:18.623+0000 7fe77e492700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe7781420a0 0x7fe7780b11b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:57:18.637 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:18.623+0000 7fe77e492700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe7781420a0 0x7fe7780b11b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:51468/0 (socket says 172.21.15.59:51468) 2023-11-24T07:57:18.637 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:18.623+0000 7fe77e492700 1 -- 172.21.15.59:0/10502821 learned_addr learned my addr 172.21.15.59:0/10502821 (peer_addr_for_me v2:172.21.15.59:0/0) 2023-11-24T07:57:18.637 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:18.623+0000 7fe77d490700 1 --2- 172.21.15.59:0/10502821 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fe7781401a0 0x7fe7780b0c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:57:18.638 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:18.623+0000 7fe77dc91700 1 --2- 172.21.15.59:0/10502821 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe764003a00 0x7fe778141b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:57:18.638 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:18.623+0000 7fe77e492700 1 -- 172.21.15.59:0/10502821 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fe7781401a0 msgr2=0x7fe7780b0c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:57:18.638 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:18.623+0000 7fe77e492700 1 --2- 172.21.15.59:0/10502821 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fe7781401a0 0x7fe7780b0c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:18.638 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:18.623+0000 7fe77e492700 1 -- 172.21.15.59:0/10502821 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe764003a00 msgr2=0x7fe778141b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:57:18.638 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:18.623+0000 7fe77e492700 1 --2- 172.21.15.59:0/10502821 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe764003a00 0x7fe778141b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:18.639 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:18.623+0000 7fe77e492700 1 -- 172.21.15.59:0/10502821 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe778141360 con 0x7fe7781420a0 2023-11-24T07:57:18.639 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:18.623+0000 7fe77dc91700 1 --2- 172.21.15.59:0/10502821 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe764003a00 0x7fe778141b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-24T07:57:18.639 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:18.623+0000 7fe77e492700 1 --2- 172.21.15.59:0/10502821 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe7781420a0 0x7fe7780b11b0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7fe778072d90 tx=0x7fe770011f60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:57:18.639 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:18.623+0000 7fe76e7fc700 1 -- 172.21.15.59:0/10502821 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe77000e040 con 0x7fe7781420a0 2023-11-24T07:57:18.639 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:18.623+0000 7fe76e7fc700 1 -- 172.21.15.59:0/10502821 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe77000f040 con 0x7fe7781420a0 2023-11-24T07:57:18.639 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:18.623+0000 7fe77fef5700 1 -- 172.21.15.59:0/10502821 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe778141640 con 0x7fe7781420a0 2023-11-24T07:57:18.639 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:18.623+0000 7fe76e7fc700 1 -- 172.21.15.59:0/10502821 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe77000fa00 con 0x7fe7781420a0 2023-11-24T07:57:18.639 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:18.623+0000 7fe77fef5700 1 -- 172.21.15.59:0/10502821 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe7781f3e00 con 0x7fe7781420a0 2023-11-24T07:57:18.639 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:18.624+0000 7fe77fef5700 1 -- 172.21.15.59:0/10502821 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe778068670 con 0x7fe7781420a0 2023-11-24T07:57:18.639 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:18.625+0000 7fe76e7fc700 1 -- 172.21.15.59:0/10502821 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fe77000fb60 con 0x7fe7781420a0 2023-11-24T07:57:18.639 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:18.625+0000 7fe76e7fc700 1 --2- 172.21.15.59:0/10502821 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7fe75c06af00 0x7fe75c06d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:57:18.639 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:18.625+0000 7fe76e7fc700 1 -- 172.21.15.59:0/10502821 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 4857+0+0 (secure 0 0 0) 0x7fe770089890 con 0x7fe7781420a0 2023-11-24T07:57:18.640 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:18.625+0000 7fe77dc91700 1 --2- 172.21.15.59:0/10502821 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7fe75c06af00 0x7fe75c06d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:57:18.640 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:18.626+0000 7fe77dc91700 1 --2- 172.21.15.59:0/10502821 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7fe75c06af00 0x7fe75c06d3c0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fe7680090b0 tx=0x7fe768008030 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:57:18.640 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:18.630+0000 7fe76e7fc700 1 -- 172.21.15.59:0/10502821 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fe770054c00 con 0x7fe7781420a0 2023-11-24T07:57:18.912 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:18.911+0000 7fe77fef5700 1 -- 172.21.15.59:0/10502821 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7fe7781ea6b0 con 0x7fe7781420a0 2023-11-24T07:57:18.913 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:18.911+0000 7fe76e7fc700 1 -- 172.21.15.59:0/10502821 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v6) v1 ==== 80+0+6645 (secure 0 0 0) 0x7fe770058220 con 0x7fe7781420a0 2023-11-24T07:57:18.913 INFO:teuthology.orchestra.run.smithi059.stdout: 2023-11-24T07:57:18.913 INFO:teuthology.orchestra.run.smithi059.stdout:{ 2023-11-24T07:57:18.914 INFO:teuthology.orchestra.run.smithi059.stdout: "auth_dump": [ 2023-11-24T07:57:18.914 INFO:teuthology.orchestra.run.smithi059.stdout: { 2023-11-24T07:57:18.914 INFO:teuthology.orchestra.run.smithi059.stdout: "entity": "mds.a", 2023-11-24T07:57:18.914 INFO:teuthology.orchestra.run.smithi059.stdout: "key": "AQD0VmBlWGLCKBAASbEQYAkExr8TButtK4KHQQ==", 2023-11-24T07:57:18.914 INFO:teuthology.orchestra.run.smithi059.stdout: "caps": { 2023-11-24T07:57:18.914 INFO:teuthology.orchestra.run.smithi059.stdout: "mds": "allow", 2023-11-24T07:57:18.914 INFO:teuthology.orchestra.run.smithi059.stdout: "mgr": "allow *", 2023-11-24T07:57:18.914 INFO:teuthology.orchestra.run.smithi059.stdout: "mon": "allow *", 2023-11-24T07:57:18.914 INFO:teuthology.orchestra.run.smithi059.stdout: "osd": "allow *" 2023-11-24T07:57:18.915 INFO:teuthology.orchestra.run.smithi059.stdout: } 2023-11-24T07:57:18.915 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-11-24T07:57:18.915 INFO:teuthology.orchestra.run.smithi059.stdout: { 2023-11-24T07:57:18.915 INFO:teuthology.orchestra.run.smithi059.stdout: "entity": "mds.b", 2023-11-24T07:57:18.915 INFO:teuthology.orchestra.run.smithi059.stdout: "key": "AQD0VmBlDIS0NRAAYQ3wTunU7m/Qk1qCM6Mcvg==", 2023-11-24T07:57:18.915 INFO:teuthology.orchestra.run.smithi059.stdout: "caps": { 2023-11-24T07:57:18.915 INFO:teuthology.orchestra.run.smithi059.stdout: "mds": "allow", 2023-11-24T07:57:18.915 INFO:teuthology.orchestra.run.smithi059.stdout: "mgr": "allow *", 2023-11-24T07:57:18.915 INFO:teuthology.orchestra.run.smithi059.stdout: "mon": "allow *", 2023-11-24T07:57:18.915 INFO:teuthology.orchestra.run.smithi059.stdout: "osd": "allow *" 2023-11-24T07:57:18.915 INFO:teuthology.orchestra.run.smithi059.stdout: } 2023-11-24T07:57:18.916 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-11-24T07:57:18.916 INFO:teuthology.orchestra.run.smithi059.stdout: { 2023-11-24T07:57:18.916 INFO:teuthology.orchestra.run.smithi059.stdout: "entity": "mds.c", 2023-11-24T07:57:18.916 INFO:teuthology.orchestra.run.smithi059.stdout: "key": "AQD0VmBlvpsrMBAAr5lh7nb6B1xRfk0Umm7Xig==", 2023-11-24T07:57:18.916 INFO:teuthology.orchestra.run.smithi059.stdout: "caps": { 2023-11-24T07:57:18.916 INFO:teuthology.orchestra.run.smithi059.stdout: "mds": "allow", 2023-11-24T07:57:18.916 INFO:teuthology.orchestra.run.smithi059.stdout: "mgr": "allow *", 2023-11-24T07:57:18.916 INFO:teuthology.orchestra.run.smithi059.stdout: "mon": "allow *", 2023-11-24T07:57:18.916 INFO:teuthology.orchestra.run.smithi059.stdout: "osd": "allow *" 2023-11-24T07:57:18.917 INFO:teuthology.orchestra.run.smithi059.stdout: } 2023-11-24T07:57:18.917 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-11-24T07:57:18.917 INFO:teuthology.orchestra.run.smithi059.stdout: { 2023-11-24T07:57:18.917 INFO:teuthology.orchestra.run.smithi059.stdout: "entity": "mds.d", 2023-11-24T07:57:18.917 INFO:teuthology.orchestra.run.smithi059.stdout: "key": "AQD1VmBlPTG7ARAAvC60Rk2GarnNtRJ6MMRIMw==", 2023-11-24T07:57:18.917 INFO:teuthology.orchestra.run.smithi059.stdout: "caps": { 2023-11-24T07:57:18.917 INFO:teuthology.orchestra.run.smithi059.stdout: "mds": "allow", 2023-11-24T07:57:18.917 INFO:teuthology.orchestra.run.smithi059.stdout: "mgr": "allow *", 2023-11-24T07:57:18.917 INFO:teuthology.orchestra.run.smithi059.stdout: "mon": "allow *", 2023-11-24T07:57:18.917 INFO:teuthology.orchestra.run.smithi059.stdout: "osd": "allow *" 2023-11-24T07:57:18.917 INFO:teuthology.orchestra.run.smithi059.stdout: } 2023-11-24T07:57:18.917 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-11-24T07:57:18.917 INFO:teuthology.orchestra.run.smithi059.stdout: { 2023-11-24T07:57:18.918 INFO:teuthology.orchestra.run.smithi059.stdout: "entity": "osd.0", 2023-11-24T07:57:18.918 INFO:teuthology.orchestra.run.smithi059.stdout: "key": "AQD4VmBlXjdAHRAAjnK2Y/rQhg4a97faNqVuoA==", 2023-11-24T07:57:18.918 INFO:teuthology.orchestra.run.smithi059.stdout: "caps": { 2023-11-24T07:57:18.918 INFO:teuthology.orchestra.run.smithi059.stdout: "mgr": "allow profile osd", 2023-11-24T07:57:18.918 INFO:teuthology.orchestra.run.smithi059.stdout: "mon": "allow profile osd", 2023-11-24T07:57:18.918 INFO:teuthology.orchestra.run.smithi059.stdout: "osd": "allow *" 2023-11-24T07:57:18.918 INFO:teuthology.orchestra.run.smithi059.stdout: } 2023-11-24T07:57:18.918 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-11-24T07:57:18.919 INFO:teuthology.orchestra.run.smithi059.stdout: { 2023-11-24T07:57:18.919 INFO:teuthology.orchestra.run.smithi059.stdout: "entity": "osd.1", 2023-11-24T07:57:18.919 INFO:teuthology.orchestra.run.smithi059.stdout: "key": "AQD7VmBlBwTwHBAAbpMH9rzS95ifrq/gVHnYRA==", 2023-11-24T07:57:18.919 INFO:teuthology.orchestra.run.smithi059.stdout: "caps": { 2023-11-24T07:57:18.920 INFO:teuthology.orchestra.run.smithi059.stdout: "mgr": "allow profile osd", 2023-11-24T07:57:18.920 INFO:teuthology.orchestra.run.smithi059.stdout: "mon": "allow profile osd", 2023-11-24T07:57:18.920 INFO:teuthology.orchestra.run.smithi059.stdout: "osd": "allow *" 2023-11-24T07:57:18.920 INFO:teuthology.orchestra.run.smithi059.stdout: } 2023-11-24T07:57:18.920 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-11-24T07:57:18.920 INFO:teuthology.orchestra.run.smithi059.stdout: { 2023-11-24T07:57:18.920 INFO:teuthology.orchestra.run.smithi059.stdout: "entity": "osd.2", 2023-11-24T07:57:18.920 INFO:teuthology.orchestra.run.smithi059.stdout: "key": "AQD+VmBl0tijHBAAWF2PsV3iH5adOTvHVPFIKw==", 2023-11-24T07:57:18.920 INFO:teuthology.orchestra.run.smithi059.stdout: "caps": { 2023-11-24T07:57:18.920 INFO:teuthology.orchestra.run.smithi059.stdout: "mgr": "allow profile osd", 2023-11-24T07:57:18.920 INFO:teuthology.orchestra.run.smithi059.stdout: "mon": "allow profile osd", 2023-11-24T07:57:18.920 INFO:teuthology.orchestra.run.smithi059.stdout: "osd": "allow *" 2023-11-24T07:57:18.921 INFO:teuthology.orchestra.run.smithi059.stdout: } 2023-11-24T07:57:18.921 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-11-24T07:57:18.921 INFO:teuthology.orchestra.run.smithi059.stdout: { 2023-11-24T07:57:18.921 INFO:teuthology.orchestra.run.smithi059.stdout: "entity": "osd.3", 2023-11-24T07:57:18.921 INFO:teuthology.orchestra.run.smithi059.stdout: "key": "AQABV2Bl3i8/HBAA4waLTD/PSLuNEg1VBmhEmg==", 2023-11-24T07:57:18.921 INFO:teuthology.orchestra.run.smithi059.stdout: "caps": { 2023-11-24T07:57:18.921 INFO:teuthology.orchestra.run.smithi059.stdout: "mgr": "allow profile osd", 2023-11-24T07:57:18.921 INFO:teuthology.orchestra.run.smithi059.stdout: "mon": "allow profile osd", 2023-11-24T07:57:18.921 INFO:teuthology.orchestra.run.smithi059.stdout: "osd": "allow *" 2023-11-24T07:57:18.921 INFO:teuthology.orchestra.run.smithi059.stdout: } 2023-11-24T07:57:18.921 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-11-24T07:57:18.921 INFO:teuthology.orchestra.run.smithi059.stdout: { 2023-11-24T07:57:18.922 INFO:teuthology.orchestra.run.smithi059.stdout: "entity": "osd.4", 2023-11-24T07:57:18.922 INFO:teuthology.orchestra.run.smithi059.stdout: "key": "AQAHV2Blr2zrHxAALOr+kFGdOTr8uQ9+/G2WUg==", 2023-11-24T07:57:18.922 INFO:teuthology.orchestra.run.smithi059.stdout: "caps": { 2023-11-24T07:57:18.922 INFO:teuthology.orchestra.run.smithi059.stdout: "mgr": "allow profile osd", 2023-11-24T07:57:18.922 INFO:teuthology.orchestra.run.smithi059.stdout: "mon": "allow profile osd", 2023-11-24T07:57:18.922 INFO:teuthology.orchestra.run.smithi059.stdout: "osd": "allow *" 2023-11-24T07:57:18.922 INFO:teuthology.orchestra.run.smithi059.stdout: } 2023-11-24T07:57:18.922 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-11-24T07:57:18.922 INFO:teuthology.orchestra.run.smithi059.stdout: { 2023-11-24T07:57:18.922 INFO:teuthology.orchestra.run.smithi059.stdout: "entity": "osd.5", 2023-11-24T07:57:18.922 INFO:teuthology.orchestra.run.smithi059.stdout: "key": "AQAKV2BlDZ3bHxAA7ZPPoxi4zVN0VECYzTsAsg==", 2023-11-24T07:57:18.922 INFO:teuthology.orchestra.run.smithi059.stdout: "caps": { 2023-11-24T07:57:18.922 INFO:teuthology.orchestra.run.smithi059.stdout: "mgr": "allow profile osd", 2023-11-24T07:57:18.923 INFO:teuthology.orchestra.run.smithi059.stdout: "mon": "allow profile osd", 2023-11-24T07:57:18.923 INFO:teuthology.orchestra.run.smithi059.stdout: "osd": "allow *" 2023-11-24T07:57:18.923 INFO:teuthology.orchestra.run.smithi059.stdout: } 2023-11-24T07:57:18.923 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-11-24T07:57:18.923 INFO:teuthology.orchestra.run.smithi059.stdout: { 2023-11-24T07:57:18.923 INFO:teuthology.orchestra.run.smithi059.stdout: "entity": "osd.6", 2023-11-24T07:57:18.923 INFO:teuthology.orchestra.run.smithi059.stdout: "key": "AQANV2BlfDMrHxAAVtm8+kh5FNTd1mJOKg02gg==", 2023-11-24T07:57:18.923 INFO:teuthology.orchestra.run.smithi059.stdout: "caps": { 2023-11-24T07:57:18.923 INFO:teuthology.orchestra.run.smithi059.stdout: "mgr": "allow profile osd", 2023-11-24T07:57:18.923 INFO:teuthology.orchestra.run.smithi059.stdout: "mon": "allow profile osd", 2023-11-24T07:57:18.923 INFO:teuthology.orchestra.run.smithi059.stdout: "osd": "allow *" 2023-11-24T07:57:18.923 INFO:teuthology.orchestra.run.smithi059.stdout: } 2023-11-24T07:57:18.923 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-11-24T07:57:18.924 INFO:teuthology.orchestra.run.smithi059.stdout: { 2023-11-24T07:57:18.924 INFO:teuthology.orchestra.run.smithi059.stdout: "entity": "osd.7", 2023-11-24T07:57:18.924 INFO:teuthology.orchestra.run.smithi059.stdout: "key": "AQAQV2Bli+slHxAAHY4sa84El5NC4KVgPKHs6w==", 2023-11-24T07:57:18.924 INFO:teuthology.orchestra.run.smithi059.stdout: "caps": { 2023-11-24T07:57:18.924 INFO:teuthology.orchestra.run.smithi059.stdout: "mgr": "allow profile osd", 2023-11-24T07:57:18.924 INFO:teuthology.orchestra.run.smithi059.stdout: "mon": "allow profile osd", 2023-11-24T07:57:18.924 INFO:teuthology.orchestra.run.smithi059.stdout: "osd": "allow *" 2023-11-24T07:57:18.924 INFO:teuthology.orchestra.run.smithi059.stdout: } 2023-11-24T07:57:18.924 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-11-24T07:57:18.924 INFO:teuthology.orchestra.run.smithi059.stdout: { 2023-11-24T07:57:18.924 INFO:teuthology.orchestra.run.smithi059.stdout: "entity": "client.0", 2023-11-24T07:57:18.924 INFO:teuthology.orchestra.run.smithi059.stdout: "key": "AQD1VmBlxPkpBhAAR841hRsF6QtmzFgJIrVZ9w==", 2023-11-24T07:57:18.924 INFO:teuthology.orchestra.run.smithi059.stdout: "caps": { 2023-11-24T07:57:18.925 INFO:teuthology.orchestra.run.smithi059.stdout: "mds": "allow", 2023-11-24T07:57:18.925 INFO:teuthology.orchestra.run.smithi059.stdout: "mgr": "allow r", 2023-11-24T07:57:18.925 INFO:teuthology.orchestra.run.smithi059.stdout: "mon": "allow rw", 2023-11-24T07:57:18.925 INFO:teuthology.orchestra.run.smithi059.stdout: "osd": "allow rwx" 2023-11-24T07:57:18.925 INFO:teuthology.orchestra.run.smithi059.stdout: } 2023-11-24T07:57:18.925 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-11-24T07:57:18.925 INFO:teuthology.orchestra.run.smithi059.stdout: { 2023-11-24T07:57:18.925 INFO:teuthology.orchestra.run.smithi059.stdout: "entity": "client.1", 2023-11-24T07:57:18.925 INFO:teuthology.orchestra.run.smithi059.stdout: "key": "AQD1VmBl46HpDBAAnO9MGWrJczW0epSp0eCkLg==", 2023-11-24T07:57:18.925 INFO:teuthology.orchestra.run.smithi059.stdout: "caps": { 2023-11-24T07:57:18.925 INFO:teuthology.orchestra.run.smithi059.stdout: "mds": "allow", 2023-11-24T07:57:18.925 INFO:teuthology.orchestra.run.smithi059.stdout: "mgr": "allow r", 2023-11-24T07:57:18.925 INFO:teuthology.orchestra.run.smithi059.stdout: "mon": "allow rw", 2023-11-24T07:57:18.926 INFO:teuthology.orchestra.run.smithi059.stdout: "osd": "allow rwx" 2023-11-24T07:57:18.926 INFO:teuthology.orchestra.run.smithi059.stdout: } 2023-11-24T07:57:18.926 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-11-24T07:57:18.926 INFO:teuthology.orchestra.run.smithi059.stdout: { 2023-11-24T07:57:18.926 INFO:teuthology.orchestra.run.smithi059.stdout: "entity": "client.2", 2023-11-24T07:57:18.926 INFO:teuthology.orchestra.run.smithi059.stdout: "key": "AQD1VmBlgQSRCRAAcHS1eQCYMJ3YP21jUEPJ0A==", 2023-11-24T07:57:18.926 INFO:teuthology.orchestra.run.smithi059.stdout: "caps": { 2023-11-24T07:57:18.926 INFO:teuthology.orchestra.run.smithi059.stdout: "mds": "allow", 2023-11-24T07:57:18.926 INFO:teuthology.orchestra.run.smithi059.stdout: "mgr": "allow r", 2023-11-24T07:57:18.927 INFO:teuthology.orchestra.run.smithi059.stdout: "mon": "allow rw", 2023-11-24T07:57:18.927 INFO:teuthology.orchestra.run.smithi059.stdout: "osd": "allow rwx" 2023-11-24T07:57:18.927 INFO:teuthology.orchestra.run.smithi059.stdout: } 2023-11-24T07:57:18.927 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-11-24T07:57:18.927 INFO:teuthology.orchestra.run.smithi059.stdout: { 2023-11-24T07:57:18.927 INFO:teuthology.orchestra.run.smithi059.stdout: "entity": "client.3", 2023-11-24T07:57:18.927 INFO:teuthology.orchestra.run.smithi059.stdout: "key": "AQD1VmBl7gmhEhAA6uWs7XAbOQwJItA6To549w==", 2023-11-24T07:57:18.927 INFO:teuthology.orchestra.run.smithi059.stdout: "caps": { 2023-11-24T07:57:18.927 INFO:teuthology.orchestra.run.smithi059.stdout: "mds": "allow", 2023-11-24T07:57:18.927 INFO:teuthology.orchestra.run.smithi059.stdout: "mgr": "allow r", 2023-11-24T07:57:18.927 INFO:teuthology.orchestra.run.smithi059.stdout: "mon": "allow rw", 2023-11-24T07:57:18.927 INFO:teuthology.orchestra.run.smithi059.stdout: "osd": "allow rwx" 2023-11-24T07:57:18.927 INFO:teuthology.orchestra.run.smithi059.stdout: } 2023-11-24T07:57:18.928 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-11-24T07:57:18.928 INFO:teuthology.orchestra.run.smithi059.stdout: { 2023-11-24T07:57:18.928 INFO:teuthology.orchestra.run.smithi059.stdout: "entity": "client.admin", 2023-11-24T07:57:18.928 INFO:teuthology.orchestra.run.smithi059.stdout: "key": "AQD0VmBlAECpCRAA1CkLlkyjnOovaNUGDSWhaw==", 2023-11-24T07:57:18.928 INFO:teuthology.orchestra.run.smithi059.stdout: "caps": { 2023-11-24T07:57:18.928 INFO:teuthology.orchestra.run.smithi059.stdout: "mds": "allow *", 2023-11-24T07:57:18.928 INFO:teuthology.orchestra.run.smithi059.stdout: "mgr": "allow *", 2023-11-24T07:57:18.928 INFO:teuthology.orchestra.run.smithi059.stdout: "mon": "allow *", 2023-11-24T07:57:18.928 INFO:teuthology.orchestra.run.smithi059.stdout: "osd": "allow *" 2023-11-24T07:57:18.928 INFO:teuthology.orchestra.run.smithi059.stdout: } 2023-11-24T07:57:18.928 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-11-24T07:57:18.929 INFO:teuthology.orchestra.run.smithi059.stdout: { 2023-11-24T07:57:18.929 INFO:teuthology.orchestra.run.smithi059.stdout: "entity": "client.bootstrap-mds", 2023-11-24T07:57:18.929 INFO:teuthology.orchestra.run.smithi059.stdout: "key": "AQAWV2Blx3j5IBAAsF7qLAonPkGuzpgnoETUUw==", 2023-11-24T07:57:18.929 INFO:teuthology.orchestra.run.smithi059.stdout: "caps": { 2023-11-24T07:57:18.929 INFO:teuthology.orchestra.run.smithi059.stdout: "mon": "allow profile bootstrap-mds" 2023-11-24T07:57:18.929 INFO:teuthology.orchestra.run.smithi059.stdout: } 2023-11-24T07:57:18.929 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-11-24T07:57:18.929 INFO:teuthology.orchestra.run.smithi059.stdout: { 2023-11-24T07:57:18.929 INFO:teuthology.orchestra.run.smithi059.stdout: "entity": "client.bootstrap-mgr", 2023-11-24T07:57:18.929 INFO:teuthology.orchestra.run.smithi059.stdout: "key": "AQAWV2Bl7oP5IBAAH4rjlqBFeVoUwHlqh6MmQA==", 2023-11-24T07:57:18.929 INFO:teuthology.orchestra.run.smithi059.stdout: "caps": { 2023-11-24T07:57:18.929 INFO:teuthology.orchestra.run.smithi059.stdout: "mon": "allow profile bootstrap-mgr" 2023-11-24T07:57:18.930 INFO:teuthology.orchestra.run.smithi059.stdout: } 2023-11-24T07:57:18.930 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-11-24T07:57:18.930 INFO:teuthology.orchestra.run.smithi059.stdout: { 2023-11-24T07:57:18.930 INFO:teuthology.orchestra.run.smithi059.stdout: "entity": "client.bootstrap-osd", 2023-11-24T07:57:18.930 INFO:teuthology.orchestra.run.smithi059.stdout: "key": "AQAWV2BlnY75IBAA7XNdWR7xT4aWQ+VIG3jKGA==", 2023-11-24T07:57:18.930 INFO:teuthology.orchestra.run.smithi059.stdout: "caps": { 2023-11-24T07:57:18.930 INFO:teuthology.orchestra.run.smithi059.stdout: "mon": "allow profile bootstrap-osd" 2023-11-24T07:57:18.930 INFO:teuthology.orchestra.run.smithi059.stdout: } 2023-11-24T07:57:18.930 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-11-24T07:57:18.930 INFO:teuthology.orchestra.run.smithi059.stdout: { 2023-11-24T07:57:18.930 INFO:teuthology.orchestra.run.smithi059.stdout: "entity": "client.bootstrap-rbd", 2023-11-24T07:57:18.930 INFO:teuthology.orchestra.run.smithi059.stdout: "key": "AQAWV2BlMJf5IBAAI5DQvz1RaDeU14rDNHRvug==", 2023-11-24T07:57:18.931 INFO:teuthology.orchestra.run.smithi059.stdout: "caps": { 2023-11-24T07:57:18.931 INFO:teuthology.orchestra.run.smithi059.stdout: "mon": "allow profile bootstrap-rbd" 2023-11-24T07:57:18.931 INFO:teuthology.orchestra.run.smithi059.stdout: } 2023-11-24T07:57:18.931 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-11-24T07:57:18.931 INFO:teuthology.orchestra.run.smithi059.stdout: { 2023-11-24T07:57:18.931 INFO:teuthology.orchestra.run.smithi059.stdout: "entity": "client.bootstrap-rbd-mirror", 2023-11-24T07:57:18.931 INFO:teuthology.orchestra.run.smithi059.stdout: "key": "AQAWV2Blo5/5IBAAtPntoBLts+rLygYRWbCmoA==", 2023-11-24T07:57:18.931 INFO:teuthology.orchestra.run.smithi059.stdout: "caps": { 2023-11-24T07:57:18.931 INFO:teuthology.orchestra.run.smithi059.stdout: "mon": "allow profile bootstrap-rbd-mirror" 2023-11-24T07:57:18.931 INFO:teuthology.orchestra.run.smithi059.stdout: } 2023-11-24T07:57:18.931 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-11-24T07:57:18.931 INFO:teuthology.orchestra.run.smithi059.stdout: { 2023-11-24T07:57:18.931 INFO:teuthology.orchestra.run.smithi059.stdout: "entity": "client.bootstrap-rgw", 2023-11-24T07:57:18.932 INFO:teuthology.orchestra.run.smithi059.stdout: "key": "AQAWV2BlMqj5IBAAugSFGrP+uvDEDYeaa/Vtqg==", 2023-11-24T07:57:18.932 INFO:teuthology.orchestra.run.smithi059.stdout: "caps": { 2023-11-24T07:57:18.932 INFO:teuthology.orchestra.run.smithi059.stdout: "mon": "allow profile bootstrap-rgw" 2023-11-24T07:57:18.932 INFO:teuthology.orchestra.run.smithi059.stdout: } 2023-11-24T07:57:18.932 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-11-24T07:57:18.932 INFO:teuthology.orchestra.run.smithi059.stdout: { 2023-11-24T07:57:18.932 INFO:teuthology.orchestra.run.smithi059.stdout: "entity": "mgr.x", 2023-11-24T07:57:18.932 INFO:teuthology.orchestra.run.smithi059.stdout: "key": "AQD0VmBlryM+JRAAf72PlDWEHZOwNxhW4k/8Ig==", 2023-11-24T07:57:18.932 INFO:teuthology.orchestra.run.smithi059.stdout: "caps": { 2023-11-24T07:57:18.932 INFO:teuthology.orchestra.run.smithi059.stdout: "mds": "allow *", 2023-11-24T07:57:18.932 INFO:teuthology.orchestra.run.smithi059.stdout: "mon": "allow profile mgr", 2023-11-24T07:57:18.932 INFO:teuthology.orchestra.run.smithi059.stdout: "osd": "allow *" 2023-11-24T07:57:18.933 INFO:teuthology.orchestra.run.smithi059.stdout: } 2023-11-24T07:57:18.933 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-11-24T07:57:18.933 INFO:teuthology.orchestra.run.smithi059.stdout: { 2023-11-24T07:57:18.933 INFO:teuthology.orchestra.run.smithi059.stdout: "entity": "mgr.y", 2023-11-24T07:57:18.933 INFO:teuthology.orchestra.run.smithi059.stdout: "key": "AQD0VmBlpXXaHxAAJMwwht1n+zRbRZuGOD5WTQ==", 2023-11-24T07:57:18.933 INFO:teuthology.orchestra.run.smithi059.stdout: "caps": { 2023-11-24T07:57:18.933 INFO:teuthology.orchestra.run.smithi059.stdout: "mds": "allow *", 2023-11-24T07:57:18.933 INFO:teuthology.orchestra.run.smithi059.stdout: "mon": "allow profile mgr", 2023-11-24T07:57:18.933 INFO:teuthology.orchestra.run.smithi059.stdout: "osd": "allow *" 2023-11-24T07:57:18.933 INFO:teuthology.orchestra.run.smithi059.stdout: } 2023-11-24T07:57:18.933 INFO:teuthology.orchestra.run.smithi059.stdout: } 2023-11-24T07:57:18.933 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2023-11-24T07:57:18.933 INFO:teuthology.orchestra.run.smithi059.stdout:} 2023-11-24T07:57:18.935 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:18.913+0000 7fe77fef5700 1 -- 172.21.15.59:0/10502821 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7fe75c06af00 msgr2=0x7fe75c06d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:57:18.935 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:18.913+0000 7fe77fef5700 1 --2- 172.21.15.59:0/10502821 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7fe75c06af00 0x7fe75c06d3c0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fe7680090b0 tx=0x7fe768008030 comp rx=0 tx=0).stop 2023-11-24T07:57:18.935 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:18.913+0000 7fe77fef5700 1 -- 172.21.15.59:0/10502821 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe7781420a0 msgr2=0x7fe7780b11b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:57:18.935 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:18.913+0000 7fe77fef5700 1 --2- 172.21.15.59:0/10502821 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe7781420a0 0x7fe7780b11b0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7fe778072d90 tx=0x7fe770011f60 comp rx=0 tx=0).stop 2023-11-24T07:57:18.935 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:18.913+0000 7fe77fef5700 1 -- 172.21.15.59:0/10502821 shutdown_connections 2023-11-24T07:57:18.935 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:18.913+0000 7fe77fef5700 1 --2- 172.21.15.59:0/10502821 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7fe75c06af00 0x7fe75c06d3c0 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:18.936 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:18.913+0000 7fe77fef5700 1 --2- 172.21.15.59:0/10502821 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe7781420a0 0x7fe7780b11b0 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:18.936 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:18.913+0000 7fe77fef5700 1 --2- 172.21.15.59:0/10502821 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe764003a00 0x7fe778141b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:18.936 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:18.913+0000 7fe77fef5700 1 --2- 172.21.15.59:0/10502821 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fe7781401a0 0x7fe7780b0c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:18.936 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:18.913+0000 7fe77fef5700 1 -- 172.21.15.59:0/10502821 >> 172.21.15.59:0/10502821 conn(0x7fe7780b2740 msgr2=0x7fe7780cf140 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:57:18.936 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:18.913+0000 7fe77fef5700 1 -- 172.21.15.59:0/10502821 shutdown_connections 2023-11-24T07:57:18.936 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:18.913+0000 7fe77fef5700 1 -- 172.21.15.59:0/10502821 wait complete. 2023-11-24T07:57:18.936 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-mds 2023-11-24T07:57:19.062 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:19.061+0000 7f9ef1612700 1 Processor -- start 2023-11-24T07:57:19.063 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:19.061+0000 7f9ef1612700 1 -- start start 2023-11-24T07:57:19.066 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:19.061+0000 7f9ef1612700 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f9eec15e460 0x7f9eec0d49d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:57:19.067 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:19.061+0000 7f9ef1612700 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f9eec0d2ba0 0x7f9eec0b40a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:57:19.067 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:19.061+0000 7f9ef1612700 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9eec0b3370 con 0x7f9eec0cf480 2023-11-24T07:57:19.067 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:19.061+0000 7f9ef1612700 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9eec0cfc40 con 0x7f9eec15e460 2023-11-24T07:57:19.067 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:19.061+0000 7f9ef1612700 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9eec0b35c0 con 0x7f9eec0d2ba0 2023-11-24T07:57:19.067 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:19.062+0000 7f9eea7fc700 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7f9eec0cf480 0x7f9eec0d2490 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:54948/0 (socket says 172.21.15.59:54948) 2023-11-24T07:57:19.067 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:19.062+0000 7f9eea7fc700 1 -- 172.21.15.59:0/1005051167 learned_addr learned my addr 172.21.15.59:0/1005051167 (peer_addr_for_me v1:172.21.15.59:0/0) 2023-11-24T07:57:19.067 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:19.062+0000 7f9eeaffd700 1 --2- 172.21.15.59:0/1005051167 >> v2:172.21.15.76:3300/0 conn(0x7f9eec15e460 0x7f9eec0d49d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:57:19.067 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:19.062+0000 7f9ee9ffb700 1 -- 172.21.15.59:0/1005051167 <== mon.0 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1290393850 0 0) 0x7f9eec0b3370 con 0x7f9eec0cf480 2023-11-24T07:57:19.067 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:19.062+0000 7f9eeb7fe700 1 --2- 172.21.15.59:0/1005051167 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f9eec0d2ba0 0x7f9eec0b40a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:57:19.067 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:19.062+0000 7f9ee9ffb700 1 -- 172.21.15.59:0/1005051167 --> v1:172.21.15.59:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9ed40035b0 con 0x7f9eec0cf480 2023-11-24T07:57:19.068 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:19.062+0000 7f9ee9ffb700 1 -- 172.21.15.59:0/1005051167 <== mon.0 v1:172.21.15.59:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2088685951 0 0) 0x7f9ed40035b0 con 0x7f9eec0cf480 2023-11-24T07:57:19.068 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:19.062+0000 7f9ee9ffb700 1 -- 172.21.15.59:0/1005051167 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f9eec0d2ba0 msgr2=0x7f9eec0b40a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T07:57:19.068 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:19.062+0000 7f9ee9ffb700 1 --2- 172.21.15.59:0/1005051167 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f9eec0d2ba0 0x7f9eec0b40a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:19.068 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:19.062+0000 7f9ee9ffb700 1 -- 172.21.15.59:0/1005051167 >> v2:172.21.15.76:3300/0 conn(0x7f9eec15e460 msgr2=0x7f9eec0d49d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T07:57:19.068 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:19.062+0000 7f9ee9ffb700 1 --2- 172.21.15.59:0/1005051167 >> v2:172.21.15.76:3300/0 conn(0x7f9eec15e460 0x7f9eec0d49d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:19.068 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:19.062+0000 7f9ee9ffb700 1 -- 172.21.15.59:0/1005051167 --> v1:172.21.15.59:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9eec0b38a0 con 0x7f9eec0cf480 2023-11-24T07:57:19.068 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:19.062+0000 7f9ee9ffb700 1 -- 172.21.15.59:0/1005051167 <== mon.0 v1:172.21.15.59:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1312333566 0 0) 0x7f9ee0001670 con 0x7f9eec0cf480 2023-11-24T07:57:19.068 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:19.062+0000 7f9ee9ffb700 1 -- 172.21.15.59:0/1005051167 >> v1:172.21.15.59:6789/0 conn(0x7f9eec0cf480 legacy=0x7f9eec0d2490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:57:19.068 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:19.062+0000 7f9ee9ffb700 1 --2- 172.21.15.59:0/1005051167 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9ed4003a00 0x7f9ed4003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:57:19.068 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:19.062+0000 7f9ee9ffb700 1 --2- 172.21.15.59:0/1005051167 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9ed4004850 0x7f9ed4004cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:57:19.068 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:19.062+0000 7f9ee9ffb700 1 --2- 172.21.15.59:0/1005051167 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f9eec0d2ba0 0x7f9ed4005210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:57:19.068 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:19.062+0000 7f9ee9ffb700 1 -- 172.21.15.59:0/1005051167 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9eec0cfc40 con 0x7f9ed4003a00 2023-11-24T07:57:19.068 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:19.062+0000 7f9ee9ffb700 1 -- 172.21.15.59:0/1005051167 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9eec0b35c0 con 0x7f9ed4004850 2023-11-24T07:57:19.069 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:19.062+0000 7f9ee9ffb700 1 -- 172.21.15.59:0/1005051167 --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9ed400ba40 con 0x7f9eec0d2ba0 2023-11-24T07:57:19.069 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:19.062+0000 7f9eeaffd700 1 --2- 172.21.15.59:0/1005051167 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9ed4003a00 0x7f9ed4003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:57:19.069 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:19.062+0000 7f9eea7fc700 1 --2- 172.21.15.59:0/1005051167 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9ed4004850 0x7f9ed4004cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:57:19.069 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:19.062+0000 7f9eeb7fe700 1 --2- 172.21.15.59:0/1005051167 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f9eec0d2ba0 0x7f9ed4005210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:57:19.069 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:19.063+0000 7f9eeaffd700 1 -- 172.21.15.59:0/1005051167 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f9eec0d2ba0 msgr2=0x7f9ed4005210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T07:57:19.069 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:19.063+0000 7f9eeaffd700 1 --2- 172.21.15.59:0/1005051167 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f9eec0d2ba0 0x7f9ed4005210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:19.069 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:19.063+0000 7f9eeaffd700 1 -- 172.21.15.59:0/1005051167 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9ed4004850 msgr2=0x7f9ed4004cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T07:57:19.069 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:19.063+0000 7f9eeaffd700 1 --2- 172.21.15.59:0/1005051167 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9ed4004850 0x7f9ed4004cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:19.069 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:19.063+0000 7f9eeaffd700 1 -- 172.21.15.59:0/1005051167 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9ed40035b0 con 0x7f9ed4003a00 2023-11-24T07:57:19.069 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:19.063+0000 7f9eeb7fe700 1 --2- 172.21.15.59:0/1005051167 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f9eec0d2ba0 0x7f9ed4005210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-24T07:57:19.069 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:19.063+0000 7f9eeaffd700 1 --2- 172.21.15.59:0/1005051167 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9ed4003a00 0x7f9ed4003e20 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f9edc009b60 tx=0x7f9ed4005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1ef92cd475f81459 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:57:19.069 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:19.063+0000 7f9ee9ffb700 1 -- 172.21.15.59:0/1005051167 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9edc00e6c0 con 0x7f9ed4003a00 2023-11-24T07:57:19.070 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:19.063+0000 7f9ee9ffb700 1 -- 172.21.15.59:0/1005051167 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9edc00e820 con 0x7f9ed4003a00 2023-11-24T07:57:19.070 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:19.063+0000 7f9ee9ffb700 1 -- 172.21.15.59:0/1005051167 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9edc00eb90 con 0x7f9ed4003a00 2023-11-24T07:57:19.070 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:19.063+0000 7f9ef1612700 1 -- 172.21.15.59:0/1005051167 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9ed4003a00 msgr2=0x7f9ed4003e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:57:19.070 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:19.063+0000 7f9ef1612700 1 --2- 172.21.15.59:0/1005051167 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9ed4003a00 0x7f9ed4003e20 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f9edc009b60 tx=0x7f9ed4005880 comp rx=0 tx=0).stop 2023-11-24T07:57:19.070 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:19.063+0000 7f9eeb7fe700 1 -- 172.21.15.59:0/1005051167 reap_dead start 2023-11-24T07:57:19.070 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:19.064+0000 7f9ef1612700 1 -- 172.21.15.59:0/1005051167 shutdown_connections 2023-11-24T07:57:19.070 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:19.064+0000 7f9ef1612700 1 -- 172.21.15.59:0/1005051167 >> 172.21.15.59:0/1005051167 conn(0x7f9eec0d3eb0 msgr2=0x7f9eec0d42c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:57:19.070 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:19.064+0000 7f9ef1612700 1 -- 172.21.15.59:0/1005051167 shutdown_connections 2023-11-24T07:57:19.070 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:19.064+0000 7f9ef1612700 1 -- 172.21.15.59:0/1005051167 wait complete. 2023-11-24T07:57:19.070 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:19.065+0000 7f9ef1612700 1 Processor -- start 2023-11-24T07:57:19.070 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:19.065+0000 7f9ef1612700 1 -- start start 2023-11-24T07:57:19.073 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:19.066+0000 7f9ef1612700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9ed4003a00 0x7f9eec1540b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:57:19.073 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:19.066+0000 7f9ef1612700 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f9eec15e840 0x7f9eec0ca2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:57:19.073 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:19.066+0000 7f9ef1612700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9eec15ecc0 0x7f9eec0cff20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:57:19.074 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:19.066+0000 7f9ef1612700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9eec0cb990 con 0x7f9ed4003a00 2023-11-24T07:57:19.074 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:19.066+0000 7f9ef1612700 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9eec0cbbe0 con 0x7f9eec15ecc0 2023-11-24T07:57:19.074 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:19.066+0000 7f9ef1612700 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9eec0cbec0 con 0x7f9eec15e840 2023-11-24T07:57:19.074 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:19.066+0000 7f9eeaffd700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9ed4003a00 0x7f9eec1540b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:57:19.075 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:19.066+0000 7f9eeaffd700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9ed4003a00 0x7f9eec1540b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:51484/0 (socket says 172.21.15.59:51484) 2023-11-24T07:57:19.075 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:19.066+0000 7f9eeaffd700 1 -- 172.21.15.59:0/3229521879 learned_addr learned my addr 172.21.15.59:0/3229521879 (peer_addr_for_me v2:172.21.15.59:0/0) 2023-11-24T07:57:19.075 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:19.066+0000 7f9eeb7fe700 1 --2- 172.21.15.59:0/3229521879 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9eec15ecc0 0x7f9eec0cff20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:57:19.075 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:19.066+0000 7f9eea7fc700 1 --2- 172.21.15.59:0/3229521879 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f9eec15e840 0x7f9eec0ca2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:57:19.076 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:19.066+0000 7f9eeaffd700 1 -- 172.21.15.59:0/3229521879 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f9eec15e840 msgr2=0x7f9eec0ca2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:57:19.076 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:19.066+0000 7f9eeaffd700 1 --2- 172.21.15.59:0/3229521879 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f9eec15e840 0x7f9eec0ca2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:19.076 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:19.066+0000 7f9eeaffd700 1 -- 172.21.15.59:0/3229521879 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9eec15ecc0 msgr2=0x7f9eec0cff20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:57:19.076 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:19.066+0000 7f9eeaffd700 1 --2- 172.21.15.59:0/3229521879 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9eec15ecc0 0x7f9eec0cff20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:19.076 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:19.066+0000 7f9eeaffd700 1 -- 172.21.15.59:0/3229521879 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9eec0cc1a0 con 0x7f9ed4003a00 2023-11-24T07:57:19.077 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:19.066+0000 7f9eeb7fe700 1 --2- 172.21.15.59:0/3229521879 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9eec15ecc0 0x7f9eec0cff20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T07:57:19.077 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:19.066+0000 7f9eea7fc700 1 --2- 172.21.15.59:0/3229521879 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f9eec15e840 0x7f9eec0ca2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T07:57:19.077 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:19.066+0000 7f9eeaffd700 1 --2- 172.21.15.59:0/3229521879 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9ed4003a00 0x7f9eec1540b0 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f9ed4004760 tx=0x7f9edc0047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:57:19.077 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:19.066+0000 7f9ed37fe700 1 -- 172.21.15.59:0/3229521879 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9edc00e6c0 con 0x7f9ed4003a00 2023-11-24T07:57:19.077 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:19.066+0000 7f9ed37fe700 1 -- 172.21.15.59:0/3229521879 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9edc003ab0 con 0x7f9ed4003a00 2023-11-24T07:57:19.077 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:19.066+0000 7f9ef1612700 1 -- 172.21.15.59:0/3229521879 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9eec0cc480 con 0x7f9ed4003a00 2023-11-24T07:57:19.077 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:19.066+0000 7f9ed37fe700 1 -- 172.21.15.59:0/3229521879 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9edc003e60 con 0x7f9ed4003a00 2023-11-24T07:57:19.077 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:19.067+0000 7f9ef1612700 1 -- 172.21.15.59:0/3229521879 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9eec154b80 con 0x7f9ed4003a00 2023-11-24T07:57:19.077 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:19.068+0000 7f9ef1612700 1 -- 172.21.15.59:0/3229521879 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9eec0ccad0 con 0x7f9ed4003a00 2023-11-24T07:57:19.077 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:19.068+0000 7f9ed37fe700 1 -- 172.21.15.59:0/3229521879 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f9edc013470 con 0x7f9ed4003a00 2023-11-24T07:57:19.078 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:19.068+0000 7f9ed37fe700 1 --2- 172.21.15.59:0/3229521879 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f9ed806aed0 0x7f9ed806d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:57:19.078 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:19.069+0000 7f9ed37fe700 1 -- 172.21.15.59:0/3229521879 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f9edc08aab0 con 0x7f9ed4003a00 2023-11-24T07:57:19.078 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:19.069+0000 7f9eea7fc700 1 --2- 172.21.15.59:0/3229521879 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f9ed806aed0 0x7f9ed806d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:57:19.078 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:19.069+0000 7f9eea7fc700 1 --2- 172.21.15.59:0/3229521879 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f9ed806aed0 0x7f9ed806d390 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f9eec0735f0 tx=0x7f9ee0009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:57:19.078 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:19.073+0000 7f9ed37fe700 1 -- 172.21.15.59:0/3229521879 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f9edc056060 con 0x7f9ed4003a00 2023-11-24T07:57:19.357 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:19.355+0000 7f9ef1612700 1 -- 172.21.15.59:0/3229521879 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-mds"} v 0) v1 -- 0x7f9eec0cb990 con 0x7f9ed4003a00 2023-11-24T07:57:19.358 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:19.357+0000 7f9ed37fe700 1 -- 172.21.15.59:0/3229521879 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-mds"}]=0 v7) v1 ==== 90+0+0 (secure 0 0 0) 0x7f9edc059680 con 0x7f9ed4003a00 2023-11-24T07:57:19.362 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:19.360+0000 7f9ef1612700 1 -- 172.21.15.59:0/3229521879 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f9ed806aed0 msgr2=0x7f9ed806d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:57:19.362 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:19.360+0000 7f9ef1612700 1 --2- 172.21.15.59:0/3229521879 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f9ed806aed0 0x7f9ed806d390 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f9eec0735f0 tx=0x7f9ee0009040 comp rx=0 tx=0).stop 2023-11-24T07:57:19.362 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:19.360+0000 7f9ef1612700 1 -- 172.21.15.59:0/3229521879 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9ed4003a00 msgr2=0x7f9eec1540b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:57:19.363 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:19.360+0000 7f9ef1612700 1 --2- 172.21.15.59:0/3229521879 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9ed4003a00 0x7f9eec1540b0 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f9ed4004760 tx=0x7f9edc0047c0 comp rx=0 tx=0).stop 2023-11-24T07:57:19.363 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:19.360+0000 7f9ef1612700 1 -- 172.21.15.59:0/3229521879 shutdown_connections 2023-11-24T07:57:19.363 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:19.360+0000 7f9ef1612700 1 --2- 172.21.15.59:0/3229521879 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f9ed806aed0 0x7f9ed806d390 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:19.363 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:19.360+0000 7f9ef1612700 1 --2- 172.21.15.59:0/3229521879 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9eec15ecc0 0x7f9eec0cff20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:19.364 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:19.360+0000 7f9ef1612700 1 --2- 172.21.15.59:0/3229521879 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9ed4003a00 0x7f9eec1540b0 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:19.364 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:19.360+0000 7f9ef1612700 1 --2- 172.21.15.59:0/3229521879 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f9eec15e840 0x7f9eec0ca2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:19.364 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:19.360+0000 7f9ef1612700 1 -- 172.21.15.59:0/3229521879 >> 172.21.15.59:0/3229521879 conn(0x7f9eec0d3eb0 msgr2=0x7f9eec0ce880 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:57:19.364 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:19.360+0000 7f9ef1612700 1 -- 172.21.15.59:0/3229521879 shutdown_connections 2023-11-24T07:57:19.365 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:19.360+0000 7f9ef1612700 1 -- 172.21.15.59:0/3229521879 wait complete. 2023-11-24T07:57:19.374 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-mgr 2023-11-24T07:57:19.504 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:19.502+0000 7f46769af700 1 Processor -- start 2023-11-24T07:57:19.504 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:19.502+0000 7f46769af700 1 -- start start 2023-11-24T07:57:19.506 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:19.503+0000 7f46769af700 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f467006afd0 0x7f46700b5a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:57:19.506 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:19.503+0000 7f46769af700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f4670071b40 0x7f4670071ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:57:19.507 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:19.503+0000 7f46769af700 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f46700ad860 con 0x7f467006edc0 2023-11-24T07:57:19.507 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:19.503+0000 7f46769af700 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f467006f830 con 0x7f4670071b40 2023-11-24T07:57:19.507 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:19.503+0000 7f46769af700 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f46700d0a20 con 0x7f467006afd0 2023-11-24T07:57:19.507 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:19.503+0000 7f4674f4c700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f4670071b40 0x7f4670071ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:57:19.507 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:19.503+0000 7f466f7fe700 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7f467006edc0 0x7f467006b3b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:54970/0 (socket says 172.21.15.59:54970) 2023-11-24T07:57:19.507 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:19.503+0000 7f4674f4c700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f4670071b40 0x7f4670071ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:51498/0 (socket says 172.21.15.59:51498) 2023-11-24T07:57:19.508 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:19.503+0000 7f466f7fe700 1 -- 172.21.15.59:0/4051144594 learned_addr learned my addr 172.21.15.59:0/4051144594 (peer_addr_for_me v1:172.21.15.59:0/0) 2023-11-24T07:57:19.508 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:19.503+0000 7f466ffff700 1 --2- 172.21.15.59:0/4051144594 >> v2:172.21.15.76:3300/0 conn(0x7f467006afd0 0x7f46700b5a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:57:19.508 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:19.503+0000 7f4674f4c700 1 -- 172.21.15.59:0/4051144594 >> v2:172.21.15.76:3300/0 conn(0x7f467006afd0 msgr2=0x7f46700b5a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T07:57:19.508 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:19.503+0000 7f4674f4c700 1 --2- 172.21.15.59:0/4051144594 >> v2:172.21.15.76:3300/0 conn(0x7f467006afd0 0x7f46700b5a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:19.508 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:19.503+0000 7f4674f4c700 1 -- 172.21.15.59:0/4051144594 >> v1:172.21.15.59:6789/0 conn(0x7f467006edc0 legacy=0x7f467006b3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:57:19.508 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:19.503+0000 7f4674f4c700 1 -- 172.21.15.59:0/4051144594 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4670162a70 con 0x7f4670071b40 2023-11-24T07:57:19.508 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:19.503+0000 7f466effd700 1 -- 172.21.15.59:0/4051144594 <== mon.0 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3465459469 0 0) 0x7f46700ad860 con 0x7f467006edc0 2023-11-24T07:57:19.509 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:19.503+0000 7f4674f4c700 1 --2- 172.21.15.59:0/4051144594 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4670071b40 0x7f4670071ff0 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f4670073820 tx=0x7f466800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e3725983c2dfe5f3 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:57:19.509 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:19.504+0000 7f466effd700 1 -- 172.21.15.59:0/4051144594 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f466800e070 con 0x7f4670071b40 2023-11-24T07:57:19.509 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:19.504+0000 7f466effd700 1 -- 172.21.15.59:0/4051144594 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f466800bbd0 con 0x7f4670071b40 2023-11-24T07:57:19.509 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:19.504+0000 7f466effd700 1 -- 172.21.15.59:0/4051144594 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4668012430 con 0x7f4670071b40 2023-11-24T07:57:19.509 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:19.504+0000 7f46769af700 1 -- 172.21.15.59:0/4051144594 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4670071b40 msgr2=0x7f4670071ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:57:19.509 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:19.504+0000 7f46769af700 1 --2- 172.21.15.59:0/4051144594 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4670071b40 0x7f4670071ff0 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f4670073820 tx=0x7f466800b330 comp rx=0 tx=0).stop 2023-11-24T07:57:19.509 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:19.504+0000 7f46769af700 1 -- 172.21.15.59:0/4051144594 shutdown_connections 2023-11-24T07:57:19.510 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:19.504+0000 7f46769af700 1 --2- 172.21.15.59:0/4051144594 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4670071b40 0x7f4670071ff0 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:19.510 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:19.504+0000 7f46769af700 1 --2- 172.21.15.59:0/4051144594 >> v2:172.21.15.76:3300/0 conn(0x7f467006afd0 0x7f46700b5a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:19.510 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:19.504+0000 7f46769af700 1 -- 172.21.15.59:0/4051144594 >> 172.21.15.59:0/4051144594 conn(0x7f46700b2840 msgr2=0x7f46700b2c20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:57:19.510 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:19.504+0000 7f46769af700 1 -- 172.21.15.59:0/4051144594 shutdown_connections 2023-11-24T07:57:19.510 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:19.504+0000 7f46769af700 1 -- 172.21.15.59:0/4051144594 wait complete. 2023-11-24T07:57:19.514 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:19.505+0000 7f46769af700 1 Processor -- start 2023-11-24T07:57:19.514 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:19.506+0000 7f46769af700 1 -- start start 2023-11-24T07:57:19.514 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:19.506+0000 7f46769af700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f467006afd0 0x7f467015f320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:57:19.514 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:19.506+0000 7f46769af700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f467006edc0 0x7f467015f860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:57:19.514 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:19.506+0000 7f46769af700 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f4670071b40 0x7f467015fda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:57:19.514 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:19.506+0000 7f46769af700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46700cff60 con 0x7f467006edc0 2023-11-24T07:57:19.515 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:19.506+0000 7f46769af700 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4670146ea0 con 0x7f467006afd0 2023-11-24T07:57:19.515 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:19.506+0000 7f46769af700 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f467015d090 con 0x7f4670071b40 2023-11-24T07:57:19.515 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:19.506+0000 7f466f7fe700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f467006edc0 0x7f467015f860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:57:19.515 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:19.507+0000 7f466f7fe700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f467006edc0 0x7f467015f860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:51502/0 (socket says 172.21.15.59:51502) 2023-11-24T07:57:19.515 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:19.507+0000 7f466f7fe700 1 -- 172.21.15.59:0/4168740594 learned_addr learned my addr 172.21.15.59:0/4168740594 (peer_addr_for_me v2:172.21.15.59:0/0) 2023-11-24T07:57:19.516 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:19.507+0000 7f466ffff700 1 --2- 172.21.15.59:0/4168740594 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f467006afd0 0x7f467015f320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:57:19.516 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:19.507+0000 7f4674f4c700 1 --2- 172.21.15.59:0/4168740594 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f4670071b40 0x7f467015fda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:57:19.516 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:19.507+0000 7f466f7fe700 1 -- 172.21.15.59:0/4168740594 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f4670071b40 msgr2=0x7f467015fda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:57:19.516 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:19.507+0000 7f466f7fe700 1 --2- 172.21.15.59:0/4168740594 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f4670071b40 0x7f467015fda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:19.516 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:19.507+0000 7f466f7fe700 1 -- 172.21.15.59:0/4168740594 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f467006afd0 msgr2=0x7f467015f320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:57:19.516 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:19.507+0000 7f466f7fe700 1 --2- 172.21.15.59:0/4168740594 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f467006afd0 0x7f467015f320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:19.516 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:19.507+0000 7f466f7fe700 1 -- 172.21.15.59:0/4168740594 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4668009a70 con 0x7f467006edc0 2023-11-24T07:57:19.517 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:19.507+0000 7f4674f4c700 1 --2- 172.21.15.59:0/4168740594 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f4670071b40 0x7f467015fda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-24T07:57:19.517 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:19.507+0000 7f466f7fe700 1 --2- 172.21.15.59:0/4168740594 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f467006edc0 0x7f467015f860 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f466000c540 tx=0x7f466000c850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:57:19.517 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:19.507+0000 7f466cff9700 1 -- 172.21.15.59:0/4168740594 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4660004c80 con 0x7f467006edc0 2023-11-24T07:57:19.517 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:19.507+0000 7f466cff9700 1 -- 172.21.15.59:0/4168740594 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4660004de0 con 0x7f467006edc0 2023-11-24T07:57:19.517 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:19.507+0000 7f46769af700 1 -- 172.21.15.59:0/4168740594 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4670146850 con 0x7f467006edc0 2023-11-24T07:57:19.517 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:19.507+0000 7f466cff9700 1 -- 172.21.15.59:0/4168740594 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4660007310 con 0x7f467006edc0 2023-11-24T07:57:19.518 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:19.507+0000 7f46769af700 1 -- 172.21.15.59:0/4168740594 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f467015d370 con 0x7f467006edc0 2023-11-24T07:57:19.518 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:19.508+0000 7f46769af700 1 -- 172.21.15.59:0/4168740594 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4670147010 con 0x7f467006edc0 2023-11-24T07:57:19.518 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:19.509+0000 7f466cff9700 1 -- 172.21.15.59:0/4168740594 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f4660007470 con 0x7f467006edc0 2023-11-24T07:57:19.518 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:19.509+0000 7f466cff9700 1 --2- 172.21.15.59:0/4168740594 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f465c06aed0 0x7f465c06d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:57:19.518 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:19.509+0000 7f466cff9700 1 -- 172.21.15.59:0/4168740594 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f4660088f40 con 0x7f467006edc0 2023-11-24T07:57:19.518 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:19.509+0000 7f466ffff700 1 --2- 172.21.15.59:0/4168740594 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f465c06aed0 0x7f465c06d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:57:19.519 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:19.510+0000 7f466ffff700 1 --2- 172.21.15.59:0/4168740594 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f465c06aed0 0x7f465c06d390 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f4664006b20 tx=0x7f4664008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:57:19.519 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:19.514+0000 7f466cff9700 1 -- 172.21.15.59:0/4168740594 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f4660054340 con 0x7f467006edc0 2023-11-24T07:57:19.804 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:19.803+0000 7f46769af700 1 -- 172.21.15.59:0/4168740594 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-mgr"} v 0) v1 -- 0x7f4670068670 con 0x7f467006edc0 2023-11-24T07:57:19.806 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:19.804+0000 7f466cff9700 1 -- 172.21.15.59:0/4168740594 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-mgr"}]=0 v8) v1 ==== 90+0+0 (secure 0 0 0) 0x7f4660057960 con 0x7f467006edc0 2023-11-24T07:57:19.809 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:19.807+0000 7f46769af700 1 -- 172.21.15.59:0/4168740594 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f465c06aed0 msgr2=0x7f465c06d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:57:19.810 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:19.807+0000 7f46769af700 1 --2- 172.21.15.59:0/4168740594 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f465c06aed0 0x7f465c06d390 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f4664006b20 tx=0x7f4664008040 comp rx=0 tx=0).stop 2023-11-24T07:57:19.810 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:19.807+0000 7f46769af700 1 -- 172.21.15.59:0/4168740594 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f467006edc0 msgr2=0x7f467015f860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:57:19.810 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:19.807+0000 7f46769af700 1 --2- 172.21.15.59:0/4168740594 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f467006edc0 0x7f467015f860 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f466000c540 tx=0x7f466000c850 comp rx=0 tx=0).stop 2023-11-24T07:57:19.810 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:19.807+0000 7f46769af700 1 -- 172.21.15.59:0/4168740594 shutdown_connections 2023-11-24T07:57:19.810 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:19.807+0000 7f46769af700 1 --2- 172.21.15.59:0/4168740594 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f465c06aed0 0x7f465c06d390 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:19.811 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:19.807+0000 7f46769af700 1 --2- 172.21.15.59:0/4168740594 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f4670071b40 0x7f467015fda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:19.811 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:19.808+0000 7f46769af700 1 --2- 172.21.15.59:0/4168740594 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f467006afd0 0x7f467015f320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:19.811 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:19.808+0000 7f46769af700 1 --2- 172.21.15.59:0/4168740594 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f467006edc0 0x7f467015f860 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:19.811 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:19.808+0000 7f46769af700 1 -- 172.21.15.59:0/4168740594 >> 172.21.15.59:0/4168740594 conn(0x7f46700b2840 msgr2=0x7f467006b3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:57:19.812 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:19.808+0000 7f46769af700 1 -- 172.21.15.59:0/4168740594 shutdown_connections 2023-11-24T07:57:19.812 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:19.808+0000 7f46769af700 1 -- 172.21.15.59:0/4168740594 wait complete. 2023-11-24T07:57:19.821 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-osd 2023-11-24T07:57:19.947 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:19.945+0000 7fba68ada700 1 Processor -- start 2023-11-24T07:57:19.947 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:19.945+0000 7fba68ada700 1 -- start start 2023-11-24T07:57:19.950 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:19.946+0000 7fba68ada700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fba6406b6e0 0x7fba6406db70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:57:19.950 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:19.946+0000 7fba68ada700 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fba64160720 con 0x7fba64145100 2023-11-24T07:57:19.950 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:19.946+0000 7fba68ada700 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fba6406f710 con 0x7fba64144c10 2023-11-24T07:57:19.950 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:19.946+0000 7fba68ada700 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fba6406f500 con 0x7fba6406b6e0 2023-11-24T07:57:19.950 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:19.946+0000 7fba61d9b700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fba6406b6e0 0x7fba6406db70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:57:19.950 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:19.946+0000 7fba62d9d700 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7fba64145100 0x7fba6406e0b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:54986/0 (socket says 172.21.15.59:54986) 2023-11-24T07:57:19.951 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:19.946+0000 7fba62d9d700 1 -- 172.21.15.59:0/2651805985 learned_addr learned my addr 172.21.15.59:0/2651805985 (peer_addr_for_me v1:172.21.15.59:0/0) 2023-11-24T07:57:19.951 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:19.946+0000 7fba61d9b700 1 -- 172.21.15.59:0/2651805985 >> v1:172.21.15.76:6789/0 conn(0x7fba64144c10 legacy=0x7fba6406afd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T07:57:19.951 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:19.946+0000 7fba61d9b700 1 -- 172.21.15.59:0/2651805985 >> v1:172.21.15.59:6789/0 conn(0x7fba64145100 legacy=0x7fba6406e0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:57:19.951 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:19.946+0000 7fba61d9b700 1 -- 172.21.15.59:0/2651805985 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fba6406f9f0 con 0x7fba6406b6e0 2023-11-24T07:57:19.951 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:19.946+0000 7fba6159a700 1 -- 172.21.15.59:0/2651805985 <== mon.0 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1990623250 0 0) 0x7fba64160720 con 0x7fba64145100 2023-11-24T07:57:19.951 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:19.947+0000 7fba61d9b700 1 --2- 172.21.15.59:0/2651805985 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fba6406b6e0 0x7fba6406db70 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7fba4c009840 tx=0x7fba4c009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=41ff7dc65137f787 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:57:19.951 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:19.947+0000 7fba6159a700 1 -- 172.21.15.59:0/2651805985 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba4c004230 con 0x7fba6406b6e0 2023-11-24T07:57:19.951 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:19.947+0000 7fba6159a700 1 -- 172.21.15.59:0/2651805985 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fba4c004390 con 0x7fba6406b6e0 2023-11-24T07:57:19.951 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:19.947+0000 7fba6159a700 1 -- 172.21.15.59:0/2651805985 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba4c0036a0 con 0x7fba6406b6e0 2023-11-24T07:57:19.951 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:19.947+0000 7fba68ada700 1 -- 172.21.15.59:0/2651805985 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fba6406b6e0 msgr2=0x7fba6406db70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:57:19.951 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:19.947+0000 7fba68ada700 1 --2- 172.21.15.59:0/2651805985 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fba6406b6e0 0x7fba6406db70 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7fba4c009840 tx=0x7fba4c009b50 comp rx=0 tx=0).stop 2023-11-24T07:57:19.951 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:19.947+0000 7fba68ada700 1 -- 172.21.15.59:0/2651805985 shutdown_connections 2023-11-24T07:57:19.951 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:19.947+0000 7fba68ada700 1 --2- 172.21.15.59:0/2651805985 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fba6406b6e0 0x7fba6406db70 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:19.952 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:19.947+0000 7fba68ada700 1 -- 172.21.15.59:0/2651805985 >> 172.21.15.59:0/2651805985 conn(0x7fba640b2740 msgr2=0x7fba640b2b20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:57:19.952 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:19.947+0000 7fba68ada700 1 -- 172.21.15.59:0/2651805985 shutdown_connections 2023-11-24T07:57:19.952 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:19.947+0000 7fba68ada700 1 -- 172.21.15.59:0/2651805985 wait complete. 2023-11-24T07:57:19.957 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:19.949+0000 7fba68ada700 1 Processor -- start 2023-11-24T07:57:19.957 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:19.949+0000 7fba68ada700 1 -- start start 2023-11-24T07:57:19.957 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:19.949+0000 7fba68ada700 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fba6406b6e0 0x7fba640afd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:57:19.957 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:19.949+0000 7fba68ada700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fba64144c10 0x7fba6413d0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:57:19.957 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:19.949+0000 7fba68ada700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fba64145100 0x7fba6413d630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:57:19.958 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:19.949+0000 7fba68ada700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba641f2650 con 0x7fba64144c10 2023-11-24T07:57:19.958 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:19.949+0000 7fba68ada700 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba641f1710 con 0x7fba64145100 2023-11-24T07:57:19.958 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:19.949+0000 7fba68ada700 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fba641f19d0 con 0x7fba6406b6e0 2023-11-24T07:57:19.958 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:19.950+0000 7fba61d9b700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fba64144c10 0x7fba6413d0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:57:19.958 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:19.950+0000 7fba61d9b700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fba64144c10 0x7fba6413d0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:51516/0 (socket says 172.21.15.59:51516) 2023-11-24T07:57:19.958 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:19.950+0000 7fba61d9b700 1 -- 172.21.15.59:0/3542918247 learned_addr learned my addr 172.21.15.59:0/3542918247 (peer_addr_for_me v2:172.21.15.59:0/0) 2023-11-24T07:57:19.959 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:19.950+0000 7fba62d9d700 1 --2- 172.21.15.59:0/3542918247 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fba64145100 0x7fba6413d630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:57:19.959 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:19.950+0000 7fba6259c700 1 --2- 172.21.15.59:0/3542918247 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fba6406b6e0 0x7fba640afd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:57:19.959 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:19.950+0000 7fba61d9b700 1 -- 172.21.15.59:0/3542918247 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fba6406b6e0 msgr2=0x7fba640afd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:57:19.959 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:19.950+0000 7fba61d9b700 1 --2- 172.21.15.59:0/3542918247 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fba6406b6e0 0x7fba640afd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:19.959 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:19.950+0000 7fba61d9b700 1 -- 172.21.15.59:0/3542918247 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fba64145100 msgr2=0x7fba6413d630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:57:19.959 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:19.950+0000 7fba61d9b700 1 --2- 172.21.15.59:0/3542918247 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fba64145100 0x7fba6413d630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:19.960 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:19.950+0000 7fba61d9b700 1 -- 172.21.15.59:0/3542918247 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fba4c0094a0 con 0x7fba64144c10 2023-11-24T07:57:19.960 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:19.950+0000 7fba62d9d700 1 --2- 172.21.15.59:0/3542918247 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fba64145100 0x7fba6413d630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T07:57:19.960 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:19.950+0000 7fba6259c700 1 --2- 172.21.15.59:0/3542918247 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fba6406b6e0 0x7fba640afd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T07:57:19.960 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:19.950+0000 7fba61d9b700 1 --2- 172.21.15.59:0/3542918247 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fba64144c10 0x7fba6413d0f0 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7fba640d1bd0 tx=0x7fba4c004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:57:19.960 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:19.950+0000 7fba5affd700 1 -- 172.21.15.59:0/3542918247 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba4c011630 con 0x7fba64144c10 2023-11-24T07:57:19.960 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:19.950+0000 7fba5affd700 1 -- 172.21.15.59:0/3542918247 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fba4c01e070 con 0x7fba64144c10 2023-11-24T07:57:19.961 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:19.950+0000 7fba68ada700 1 -- 172.21.15.59:0/3542918247 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fba641f1cb0 con 0x7fba64144c10 2023-11-24T07:57:19.961 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:19.950+0000 7fba5affd700 1 -- 172.21.15.59:0/3542918247 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba4c0119b0 con 0x7fba64144c10 2023-11-24T07:57:19.961 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:19.950+0000 7fba68ada700 1 -- 172.21.15.59:0/3542918247 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fba641f2270 con 0x7fba64144c10 2023-11-24T07:57:19.961 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:19.951+0000 7fba68ada700 1 -- 172.21.15.59:0/3542918247 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fba64068670 con 0x7fba64144c10 2023-11-24T07:57:19.961 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:19.952+0000 7fba5affd700 1 -- 172.21.15.59:0/3542918247 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fba4c011b10 con 0x7fba64144c10 2023-11-24T07:57:19.962 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:19.952+0000 7fba5affd700 1 --2- 172.21.15.59:0/3542918247 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7fba4406aed0 0x7fba4406d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:57:19.962 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:19.952+0000 7fba5affd700 1 -- 172.21.15.59:0/3542918247 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 4857+0+0 (secure 0 0 0) 0x7fba4c08a680 con 0x7fba64144c10 2023-11-24T07:57:19.962 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:19.952+0000 7fba6259c700 1 --2- 172.21.15.59:0/3542918247 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7fba4406aed0 0x7fba4406d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:57:19.962 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:19.953+0000 7fba6259c700 1 --2- 172.21.15.59:0/3542918247 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7fba4406aed0 0x7fba4406d390 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fba6413e170 tx=0x7fba54009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:57:19.963 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:19.957+0000 7fba5affd700 1 -- 172.21.15.59:0/3542918247 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fba4c055a90 con 0x7fba64144c10 2023-11-24T07:57:20.238 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:20.236+0000 7fba68ada700 1 -- 172.21.15.59:0/3542918247 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-osd"} v 0) v1 -- 0x7fba640b3460 con 0x7fba64144c10 2023-11-24T07:57:20.240 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:20.238+0000 7fba5affd700 1 -- 172.21.15.59:0/3542918247 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-osd"}]=0 v9) v1 ==== 90+0+0 (secure 0 0 0) 0x7fba4c0590b0 con 0x7fba64144c10 2023-11-24T07:57:20.243 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:20.240+0000 7fba68ada700 1 -- 172.21.15.59:0/3542918247 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7fba4406aed0 msgr2=0x7fba4406d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:57:20.243 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:20.241+0000 7fba68ada700 1 --2- 172.21.15.59:0/3542918247 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7fba4406aed0 0x7fba4406d390 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fba6413e170 tx=0x7fba54009040 comp rx=0 tx=0).stop 2023-11-24T07:57:20.243 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:20.241+0000 7fba68ada700 1 -- 172.21.15.59:0/3542918247 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fba64144c10 msgr2=0x7fba6413d0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:57:20.243 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:20.241+0000 7fba68ada700 1 --2- 172.21.15.59:0/3542918247 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fba64144c10 0x7fba6413d0f0 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7fba640d1bd0 tx=0x7fba4c004750 comp rx=0 tx=0).stop 2023-11-24T07:57:20.244 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:20.241+0000 7fba68ada700 1 -- 172.21.15.59:0/3542918247 shutdown_connections 2023-11-24T07:57:20.244 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:20.241+0000 7fba68ada700 1 --2- 172.21.15.59:0/3542918247 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7fba4406aed0 0x7fba4406d390 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:20.244 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:20.241+0000 7fba68ada700 1 --2- 172.21.15.59:0/3542918247 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fba64145100 0x7fba6413d630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:20.244 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:20.241+0000 7fba68ada700 1 --2- 172.21.15.59:0/3542918247 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fba6406b6e0 0x7fba640afd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:20.245 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:20.241+0000 7fba68ada700 1 --2- 172.21.15.59:0/3542918247 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fba64144c10 0x7fba6413d0f0 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:20.245 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:20.241+0000 7fba68ada700 1 -- 172.21.15.59:0/3542918247 >> 172.21.15.59:0/3542918247 conn(0x7fba640b2740 msgr2=0x7fba640b1ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:57:20.245 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:20.241+0000 7fba68ada700 1 -- 172.21.15.59:0/3542918247 shutdown_connections 2023-11-24T07:57:20.246 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:20.241+0000 7fba68ada700 1 -- 172.21.15.59:0/3542918247 wait complete. 2023-11-24T07:57:20.255 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rbd 2023-11-24T07:57:20.381 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:20.379+0000 7f3b938eb700 1 Processor -- start 2023-11-24T07:57:20.381 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:20.379+0000 7f3b938eb700 1 -- start start 2023-11-24T07:57:20.385 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:20.380+0000 7f3b938eb700 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f3b8c0b3b40 0x7f3b8c0b5020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:57:20.385 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:20.380+0000 7f3b938eb700 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f3b8c06f420 0x7f3b8c0ce1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:57:20.385 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:20.380+0000 7f3b938eb700 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3b8c06ef90 con 0x7f3b8c0b4c40 2023-11-24T07:57:20.385 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:20.380+0000 7f3b938eb700 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3b8c0b5560 con 0x7f3b8c06f420 2023-11-24T07:57:20.385 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:20.380+0000 7f3b938eb700 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3b8c06e380 con 0x7f3b8c0b3b40 2023-11-24T07:57:20.386 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:20.380+0000 7f3b91687700 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7f3b8c0b4c40 0x7f3b8c06dba0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:54996/0 (socket says 172.21.15.59:54996) 2023-11-24T07:57:20.386 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:20.380+0000 7f3b91687700 1 -- 172.21.15.59:0/421636930 learned_addr learned my addr 172.21.15.59:0/421636930 (peer_addr_for_me v1:172.21.15.59:0/0) 2023-11-24T07:57:20.386 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:20.380+0000 7f3b91e88700 1 --2- 172.21.15.59:0/421636930 >> v2:172.21.15.76:3300/0 conn(0x7f3b8c06f420 0x7f3b8c0ce1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:57:20.386 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:20.380+0000 7f3b90e86700 1 --2- 172.21.15.59:0/421636930 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f3b8c0b3b40 0x7f3b8c0b5020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:57:20.386 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:20.381+0000 7f3b83fff700 1 -- 172.21.15.59:0/421636930 <== mon.0 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2068121948 0 0) 0x7f3b8c06ef90 con 0x7f3b8c0b4c40 2023-11-24T07:57:20.386 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:20.381+0000 7f3b83fff700 1 -- 172.21.15.59:0/421636930 --> v1:172.21.15.59:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3b780057c0 con 0x7f3b8c0b4c40 2023-11-24T07:57:20.386 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:20.381+0000 7f3b83fff700 1 -- 172.21.15.59:0/421636930 <== mon.0 v1:172.21.15.59:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1165230840 0 0) 0x7f3b780057c0 con 0x7f3b8c0b4c40 2023-11-24T07:57:20.386 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:20.381+0000 7f3b83fff700 1 -- 172.21.15.59:0/421636930 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f3b8c0b3b40 msgr2=0x7f3b8c0b5020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T07:57:20.386 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:20.381+0000 7f3b83fff700 1 --2- 172.21.15.59:0/421636930 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f3b8c0b3b40 0x7f3b8c0b5020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:20.386 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:20.381+0000 7f3b83fff700 1 -- 172.21.15.59:0/421636930 >> v2:172.21.15.76:3300/0 conn(0x7f3b8c06f420 msgr2=0x7f3b8c0ce1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T07:57:20.386 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:20.381+0000 7f3b83fff700 1 --2- 172.21.15.59:0/421636930 >> v2:172.21.15.76:3300/0 conn(0x7f3b8c06f420 0x7f3b8c0ce1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:20.386 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:20.381+0000 7f3b83fff700 1 -- 172.21.15.59:0/421636930 --> v1:172.21.15.59:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3b8c06e660 con 0x7f3b8c0b4c40 2023-11-24T07:57:20.386 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:20.381+0000 7f3b83fff700 1 -- 172.21.15.59:0/421636930 <== mon.0 v1:172.21.15.59:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1312333566 0 0) 0x7f3b7c001670 con 0x7f3b8c0b4c40 2023-11-24T07:57:20.387 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:20.381+0000 7f3b83fff700 1 -- 172.21.15.59:0/421636930 >> v1:172.21.15.59:6789/0 conn(0x7f3b8c0b4c40 legacy=0x7f3b8c06dba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:57:20.387 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:20.381+0000 7f3b83fff700 1 --2- 172.21.15.59:0/421636930 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3b78005c10 0x7f3b78006030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:57:20.387 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:20.381+0000 7f3b83fff700 1 --2- 172.21.15.59:0/421636930 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3b78006a60 0x7f3b78006ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:57:20.387 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:20.381+0000 7f3b83fff700 1 --2- 172.21.15.59:0/421636930 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f3b78007420 0x7f3b7800dc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:57:20.387 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:20.381+0000 7f3b83fff700 1 -- 172.21.15.59:0/421636930 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b8c0b5560 con 0x7f3b78006a60 2023-11-24T07:57:20.387 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:20.381+0000 7f3b83fff700 1 -- 172.21.15.59:0/421636930 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b8c06e380 con 0x7f3b78005c10 2023-11-24T07:57:20.387 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:20.381+0000 7f3b83fff700 1 -- 172.21.15.59:0/421636930 --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3b7800e160 con 0x7f3b78007420 2023-11-24T07:57:20.387 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:20.381+0000 7f3b91e88700 1 --2- 172.21.15.59:0/421636930 >> v2:172.21.15.76:3300/0 conn(0x7f3b8c06f420 0x7f3b8c0ce1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-24T07:57:20.387 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:20.381+0000 7f3b90e86700 1 --2- 172.21.15.59:0/421636930 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f3b8c0b3b40 0x7f3b8c0b5020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-24T07:57:20.387 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:20.381+0000 7f3b91687700 1 --2- 172.21.15.59:0/421636930 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3b78005c10 0x7f3b78006030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:57:20.387 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:20.381+0000 7f3b90e86700 1 --2- 172.21.15.59:0/421636930 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3b78006a60 0x7f3b78006ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:57:20.387 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:20.381+0000 7f3b91e88700 1 --2- 172.21.15.59:0/421636930 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f3b78007420 0x7f3b7800dc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:57:20.387 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:20.381+0000 7f3b90e86700 1 -- 172.21.15.59:0/421636930 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f3b78007420 msgr2=0x7f3b7800dc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T07:57:20.388 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:20.381+0000 7f3b90e86700 1 --2- 172.21.15.59:0/421636930 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f3b78007420 0x7f3b7800dc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:20.388 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:20.381+0000 7f3b90e86700 1 -- 172.21.15.59:0/421636930 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3b78005c10 msgr2=0x7f3b78006030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T07:57:20.388 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:20.381+0000 7f3b90e86700 1 --2- 172.21.15.59:0/421636930 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3b78005c10 0x7f3b78006030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:20.388 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:20.381+0000 7f3b90e86700 1 -- 172.21.15.59:0/421636930 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3b780057c0 con 0x7f3b78006a60 2023-11-24T07:57:20.388 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:20.381+0000 7f3b91e88700 1 --2- 172.21.15.59:0/421636930 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f3b78007420 0x7f3b7800dc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-24T07:57:20.388 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:20.382+0000 7f3b90e86700 1 --2- 172.21.15.59:0/421636930 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3b78006a60 0x7f3b78006ee0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f3b8c0b3900 tx=0x7f3b78007a90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=db596ebcab882b1f server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:57:20.388 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:20.382+0000 7f3b83fff700 1 -- 172.21.15.59:0/421636930 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b88004030 con 0x7f3b78006a60 2023-11-24T07:57:20.388 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:20.382+0000 7f3b83fff700 1 -- 172.21.15.59:0/421636930 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3b88004190 con 0x7f3b78006a60 2023-11-24T07:57:20.388 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:20.382+0000 7f3b83fff700 1 -- 172.21.15.59:0/421636930 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b88004520 con 0x7f3b78006a60 2023-11-24T07:57:20.388 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:20.382+0000 7f3b938eb700 1 -- 172.21.15.59:0/421636930 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3b78006a60 msgr2=0x7f3b78006ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:57:20.388 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:20.382+0000 7f3b938eb700 1 --2- 172.21.15.59:0/421636930 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3b78006a60 0x7f3b78006ee0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f3b8c0b3900 tx=0x7f3b78007a90 comp rx=0 tx=0).stop 2023-11-24T07:57:20.388 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:20.382+0000 7f3b91e88700 1 -- 172.21.15.59:0/421636930 reap_dead start 2023-11-24T07:57:20.388 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:20.382+0000 7f3b938eb700 1 -- 172.21.15.59:0/421636930 shutdown_connections 2023-11-24T07:57:20.389 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:20.382+0000 7f3b938eb700 1 -- 172.21.15.59:0/421636930 >> 172.21.15.59:0/421636930 conn(0x7f3b8c0b2840 msgr2=0x7f3b8c0b2c20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:57:20.389 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:20.382+0000 7f3b938eb700 1 -- 172.21.15.59:0/421636930 shutdown_connections 2023-11-24T07:57:20.389 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:20.382+0000 7f3b938eb700 1 -- 172.21.15.59:0/421636930 wait complete. 2023-11-24T07:57:20.389 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:20.384+0000 7f3b938eb700 1 Processor -- start 2023-11-24T07:57:20.389 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:20.384+0000 7f3b938eb700 1 -- start start 2023-11-24T07:57:20.391 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:20.384+0000 7f3b938eb700 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f3b78006a60 0x7f3b8c0ad860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:57:20.391 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:20.384+0000 7f3b938eb700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3b8c0b0140 0x7f3b8c140580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:57:20.392 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:20.384+0000 7f3b938eb700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3b8c0b1800 0x7f3b8c0b1c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:57:20.392 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:20.384+0000 7f3b938eb700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b8c0ae9a0 con 0x7f3b8c0b0140 2023-11-24T07:57:20.392 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:20.384+0000 7f3b938eb700 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b8c0b21c0 con 0x7f3b8c0b1800 2023-11-24T07:57:20.392 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:20.384+0000 7f3b938eb700 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3b8c157130 con 0x7f3b78006a60 2023-11-24T07:57:20.392 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:20.384+0000 7f3b90e86700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3b8c0b0140 0x7f3b8c140580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:57:20.392 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:20.385+0000 7f3b90e86700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3b8c0b0140 0x7f3b8c140580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:51536/0 (socket says 172.21.15.59:51536) 2023-11-24T07:57:20.392 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:20.385+0000 7f3b90e86700 1 -- 172.21.15.59:0/1687753463 learned_addr learned my addr 172.21.15.59:0/1687753463 (peer_addr_for_me v2:172.21.15.59:0/0) 2023-11-24T07:57:20.392 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:20.385+0000 7f3b91e88700 1 --2- 172.21.15.59:0/1687753463 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3b8c0b1800 0x7f3b8c0b1c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:57:20.392 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:20.385+0000 7f3b91687700 1 --2- 172.21.15.59:0/1687753463 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f3b78006a60 0x7f3b8c0ad860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:57:20.392 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:20.385+0000 7f3b90e86700 1 -- 172.21.15.59:0/1687753463 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f3b78006a60 msgr2=0x7f3b8c0ad860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:57:20.392 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:20.385+0000 7f3b90e86700 1 --2- 172.21.15.59:0/1687753463 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f3b78006a60 0x7f3b8c0ad860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:20.392 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:20.385+0000 7f3b90e86700 1 -- 172.21.15.59:0/1687753463 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3b8c0b1800 msgr2=0x7f3b8c0b1c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:57:20.393 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:20.385+0000 7f3b90e86700 1 --2- 172.21.15.59:0/1687753463 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3b8c0b1800 0x7f3b8c0b1c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:20.393 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:20.385+0000 7f3b90e86700 1 -- 172.21.15.59:0/1687753463 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3b8c0ae100 con 0x7f3b8c0b0140 2023-11-24T07:57:20.393 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:20.385+0000 7f3b90e86700 1 --2- 172.21.15.59:0/1687753463 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3b8c0b0140 0x7f3b8c140580 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f3b8800ebc0 tx=0x7f3b880047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:57:20.393 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:20.385+0000 7f3b81ffb700 1 -- 172.21.15.59:0/1687753463 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b88006e60 con 0x7f3b8c0b0140 2023-11-24T07:57:20.393 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:20.385+0000 7f3b81ffb700 1 -- 172.21.15.59:0/1687753463 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3b88003ab0 con 0x7f3b8c0b0140 2023-11-24T07:57:20.393 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:20.385+0000 7f3b81ffb700 1 -- 172.21.15.59:0/1687753463 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b88004030 con 0x7f3b8c0b0140 2023-11-24T07:57:20.393 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:20.385+0000 7f3b938eb700 1 -- 172.21.15.59:0/1687753463 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3b8c0ae3e0 con 0x7f3b8c0b0140 2023-11-24T07:57:20.393 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:20.385+0000 7f3b938eb700 1 -- 172.21.15.59:0/1687753463 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3b8c0adda0 con 0x7f3b8c0b0140 2023-11-24T07:57:20.393 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:20.386+0000 7f3b938eb700 1 -- 172.21.15.59:0/1687753463 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3b8c0cdb00 con 0x7f3b8c0b0140 2023-11-24T07:57:20.393 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:20.387+0000 7f3b81ffb700 1 -- 172.21.15.59:0/1687753463 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f3b88013470 con 0x7f3b8c0b0140 2023-11-24T07:57:20.393 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:20.387+0000 7f3b81ffb700 1 --2- 172.21.15.59:0/1687753463 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f3b7006aed0 0x7f3b7006d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:57:20.393 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:20.387+0000 7f3b81ffb700 1 -- 172.21.15.59:0/1687753463 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f3b8808ab80 con 0x7f3b8c0b0140 2023-11-24T07:57:20.393 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:20.387+0000 7f3b91687700 1 --2- 172.21.15.59:0/1687753463 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f3b7006aed0 0x7f3b7006d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:57:20.394 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:20.388+0000 7f3b91687700 1 --2- 172.21.15.59:0/1687753463 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f3b7006aed0 0x7f3b7006d390 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f3b7c001050 tx=0x7f3b7c009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:57:20.394 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:20.392+0000 7f3b81ffb700 1 -- 172.21.15.59:0/1687753463 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f3b88056010 con 0x7f3b8c0b0140 2023-11-24T07:57:20.677 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:20.675+0000 7f3b938eb700 1 -- 172.21.15.59:0/1687753463 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rbd"} v 0) v1 -- 0x7f3b8c157270 con 0x7f3b8c0b0140 2023-11-24T07:57:20.681 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:20.680+0000 7f3b81ffb700 1 -- 172.21.15.59:0/1687753463 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rbd"}]=0 v10) v1 ==== 90+0+0 (secure 0 0 0) 0x7f3b8c157270 con 0x7f3b8c0b0140 2023-11-24T07:57:20.684 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:20.682+0000 7f3b938eb700 1 -- 172.21.15.59:0/1687753463 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f3b7006aed0 msgr2=0x7f3b7006d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:57:20.684 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:20.682+0000 7f3b938eb700 1 --2- 172.21.15.59:0/1687753463 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f3b7006aed0 0x7f3b7006d390 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f3b7c001050 tx=0x7f3b7c009040 comp rx=0 tx=0).stop 2023-11-24T07:57:20.684 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:20.682+0000 7f3b938eb700 1 -- 172.21.15.59:0/1687753463 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3b8c0b0140 msgr2=0x7f3b8c140580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:57:20.684 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:20.682+0000 7f3b938eb700 1 --2- 172.21.15.59:0/1687753463 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3b8c0b0140 0x7f3b8c140580 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f3b8800ebc0 tx=0x7f3b880047c0 comp rx=0 tx=0).stop 2023-11-24T07:57:20.684 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:20.682+0000 7f3b938eb700 1 -- 172.21.15.59:0/1687753463 shutdown_connections 2023-11-24T07:57:20.684 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:20.682+0000 7f3b938eb700 1 --2- 172.21.15.59:0/1687753463 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f3b7006aed0 0x7f3b7006d390 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:20.684 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:20.682+0000 7f3b938eb700 1 --2- 172.21.15.59:0/1687753463 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3b8c0b1800 0x7f3b8c0b1c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:20.684 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:20.682+0000 7f3b938eb700 1 --2- 172.21.15.59:0/1687753463 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f3b78006a60 0x7f3b8c0ad860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:20.684 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:20.682+0000 7f3b938eb700 1 --2- 172.21.15.59:0/1687753463 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3b8c0b0140 0x7f3b8c140580 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:20.684 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:20.682+0000 7f3b938eb700 1 -- 172.21.15.59:0/1687753463 >> 172.21.15.59:0/1687753463 conn(0x7f3b8c0b2840 msgr2=0x7f3b8c15e790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:57:20.684 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:20.682+0000 7f3b938eb700 1 -- 172.21.15.59:0/1687753463 shutdown_connections 2023-11-24T07:57:20.685 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:20.682+0000 7f3b938eb700 1 -- 172.21.15.59:0/1687753463 wait complete. 2023-11-24T07:57:20.696 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rbd-mirror 2023-11-24T07:57:20.821 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:20.820+0000 7fe88cd4e700 1 Processor -- start 2023-11-24T07:57:20.821 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:20.820+0000 7fe88cd4e700 1 -- start start 2023-11-24T07:57:20.824 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:20.821+0000 7fe88cd4e700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fe88815bde0 0x7fe88815c1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:57:20.824 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:20.821+0000 7fe88cd4e700 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fe8880cf2b0 0x7fe8880d01b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:57:20.824 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:20.821+0000 7fe88cd4e700 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe8880d0a90 con 0x7fe8880ce5c0 2023-11-24T07:57:20.824 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:20.821+0000 7fe88cd4e700 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe88806ee10 con 0x7fe88815bde0 2023-11-24T07:57:20.824 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:20.821+0000 7fe88cd4e700 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe88806f0f0 con 0x7fe8880cf2b0 2023-11-24T07:57:20.824 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:20.821+0000 7fe88659c700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fe88815bde0 0x7fe88815c1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:57:20.824 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:20.821+0000 7fe88659c700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fe88815bde0 0x7fe88815c1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:51542/0 (socket says 172.21.15.59:51542) 2023-11-24T07:57:20.824 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:20.821+0000 7fe88659c700 1 -- 172.21.15.59:0/1875008450 learned_addr learned my addr 172.21.15.59:0/1875008450 (peer_addr_for_me v2:172.21.15.59:0/0) 2023-11-24T07:57:20.824 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:20.821+0000 7fe886d9d700 1 --1- 172.21.15.59:0/1875008450 >> v1:172.21.15.59:6789/0 conn(0x7fe8880ce5c0 0x7fe8880cc670 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:55020/0 (socket says 172.21.15.59:55020) 2023-11-24T07:57:20.825 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:20.821+0000 7fe885d9b700 1 --2- 172.21.15.59:0/1875008450 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fe8880cf2b0 0x7fe8880d01b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:57:20.825 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:20.821+0000 7fe88659c700 1 -- 172.21.15.59:0/1875008450 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fe8880cf2b0 msgr2=0x7fe8880d01b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T07:57:20.825 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:20.821+0000 7fe88659c700 1 --2- 172.21.15.59:0/1875008450 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fe8880cf2b0 0x7fe8880d01b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:20.825 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:20.821+0000 7fe88659c700 1 -- 172.21.15.59:0/1875008450 >> v1:172.21.15.59:6789/0 conn(0x7fe8880ce5c0 legacy=0x7fe8880cc670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:57:20.825 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:20.821+0000 7fe88659c700 1 -- 172.21.15.59:0/1875008450 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe88806f3d0 con 0x7fe88815bde0 2023-11-24T07:57:20.825 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:20.821+0000 7fe88559a700 1 -- 172.21.15.59:0/1875008450 <== mon.0 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 201857614 0 0) 0x7fe8880d0a90 con 0x7fe8880ce5c0 2023-11-24T07:57:20.825 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:20.821+0000 7fe88659c700 1 --2- 172.21.15.59:0/1875008450 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe88815bde0 0x7fe88815c1c0 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7fe8700098f0 tx=0x7fe870009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=729092ba98e1972b server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:57:20.825 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:20.821+0000 7fe88559a700 1 -- 172.21.15.59:0/1875008450 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe87000b8a0 con 0x7fe88815bde0 2023-11-24T07:57:20.825 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:20.821+0000 7fe88559a700 1 -- 172.21.15.59:0/1875008450 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe87000ba00 con 0x7fe88815bde0 2023-11-24T07:57:20.825 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:20.821+0000 7fe88559a700 1 -- 172.21.15.59:0/1875008450 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe87000bd90 con 0x7fe88815bde0 2023-11-24T07:57:20.825 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:20.822+0000 7fe88cd4e700 1 -- 172.21.15.59:0/1875008450 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe88815bde0 msgr2=0x7fe88815c1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:57:20.825 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:20.822+0000 7fe88cd4e700 1 --2- 172.21.15.59:0/1875008450 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe88815bde0 0x7fe88815c1c0 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7fe8700098f0 tx=0x7fe870009c00 comp rx=0 tx=0).stop 2023-11-24T07:57:20.826 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:20.822+0000 7fe88cd4e700 1 -- 172.21.15.59:0/1875008450 shutdown_connections 2023-11-24T07:57:20.826 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:20.822+0000 7fe88cd4e700 1 --2- 172.21.15.59:0/1875008450 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe88815bde0 0x7fe88815c1c0 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:20.826 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:20.822+0000 7fe88cd4e700 1 --2- 172.21.15.59:0/1875008450 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fe8880cf2b0 0x7fe8880d01b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:20.826 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:20.822+0000 7fe88cd4e700 1 -- 172.21.15.59:0/1875008450 >> 172.21.15.59:0/1875008450 conn(0x7fe8880b3720 msgr2=0x7fe8880b3b30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:57:20.826 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:20.822+0000 7fe88cd4e700 1 -- 172.21.15.59:0/1875008450 shutdown_connections 2023-11-24T07:57:20.826 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:20.822+0000 7fe88cd4e700 1 -- 172.21.15.59:0/1875008450 wait complete. 2023-11-24T07:57:20.828 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:20.823+0000 7fe88cd4e700 1 Processor -- start 2023-11-24T07:57:20.828 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:20.823+0000 7fe88cd4e700 1 -- start start 2023-11-24T07:57:20.829 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:20.824+0000 7fe88cd4e700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe8880ce5c0 0x7fe8880b0110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:57:20.829 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:20.824+0000 7fe88cd4e700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe8880cf2b0 0x7fe88815cbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:57:20.829 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:20.824+0000 7fe88cd4e700 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fe88815bde0 0x7fe888158260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:57:20.829 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:20.824+0000 7fe88cd4e700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8880b6490 con 0x7fe8880cf2b0 2023-11-24T07:57:20.829 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:20.824+0000 7fe88cd4e700 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe888159740 con 0x7fe8880ce5c0 2023-11-24T07:57:20.829 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:20.824+0000 7fe88cd4e700 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe8880b5570 con 0x7fe88815bde0 2023-11-24T07:57:20.829 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:20.824+0000 7fe885d9b700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe8880cf2b0 0x7fe88815cbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:57:20.829 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:20.824+0000 7fe885d9b700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe8880cf2b0 0x7fe88815cbb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:51550/0 (socket says 172.21.15.59:51550) 2023-11-24T07:57:20.829 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:20.824+0000 7fe885d9b700 1 -- 172.21.15.59:0/1436669368 learned_addr learned my addr 172.21.15.59:0/1436669368 (peer_addr_for_me v2:172.21.15.59:0/0) 2023-11-24T07:57:20.829 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:20.824+0000 7fe88659c700 1 --2- 172.21.15.59:0/1436669368 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe8880ce5c0 0x7fe8880b0110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:57:20.829 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:20.824+0000 7fe886d9d700 1 --2- 172.21.15.59:0/1436669368 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fe88815bde0 0x7fe888158260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:57:20.830 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:20.825+0000 7fe885d9b700 1 -- 172.21.15.59:0/1436669368 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fe88815bde0 msgr2=0x7fe888158260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:57:20.830 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:20.825+0000 7fe885d9b700 1 --2- 172.21.15.59:0/1436669368 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fe88815bde0 0x7fe888158260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:20.830 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:20.825+0000 7fe885d9b700 1 -- 172.21.15.59:0/1436669368 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe8880ce5c0 msgr2=0x7fe8880b0110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:57:20.830 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:20.825+0000 7fe885d9b700 1 --2- 172.21.15.59:0/1436669368 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe8880ce5c0 0x7fe8880b0110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:20.830 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:20.825+0000 7fe885d9b700 1 -- 172.21.15.59:0/1436669368 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe870009550 con 0x7fe8880cf2b0 2023-11-24T07:57:20.830 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:20.825+0000 7fe885d9b700 1 --2- 172.21.15.59:0/1436669368 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe8880cf2b0 0x7fe88815cbb0 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7fe87800e810 tx=0x7fe87800eb20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:57:20.830 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:20.825+0000 7fe87effd700 1 -- 172.21.15.59:0/1436669368 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe878004230 con 0x7fe8880cf2b0 2023-11-24T07:57:20.830 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:20.825+0000 7fe87effd700 1 -- 172.21.15.59:0/1436669368 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe878004390 con 0x7fe8880cf2b0 2023-11-24T07:57:20.830 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:20.825+0000 7fe88cd4e700 1 -- 172.21.15.59:0/1436669368 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe8880b5810 con 0x7fe8880cf2b0 2023-11-24T07:57:20.830 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:20.825+0000 7fe87effd700 1 -- 172.21.15.59:0/1436669368 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8780036a0 con 0x7fe8880cf2b0 2023-11-24T07:57:20.830 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:20.825+0000 7fe88cd4e700 1 -- 172.21.15.59:0/1436669368 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe8880b5dd0 con 0x7fe8880cf2b0 2023-11-24T07:57:20.830 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:20.826+0000 7fe88cd4e700 1 -- 172.21.15.59:0/1436669368 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe888068670 con 0x7fe8880cf2b0 2023-11-24T07:57:20.830 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:20.826+0000 7fe87effd700 1 -- 172.21.15.59:0/1436669368 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fe878003800 con 0x7fe8880cf2b0 2023-11-24T07:57:20.831 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:20.827+0000 7fe87effd700 1 --2- 172.21.15.59:0/1436669368 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7fe86806aed0 0x7fe86806d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:57:20.831 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:20.827+0000 7fe87effd700 1 -- 172.21.15.59:0/1436669368 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 4857+0+0 (secure 0 0 0) 0x7fe878088d70 con 0x7fe8880cf2b0 2023-11-24T07:57:20.831 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:20.827+0000 7fe88659c700 1 --2- 172.21.15.59:0/1436669368 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7fe86806aed0 0x7fe86806d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:57:20.831 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:20.828+0000 7fe88659c700 1 --2- 172.21.15.59:0/1436669368 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7fe86806aed0 0x7fe86806d390 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fe870000c00 tx=0x7fe870008030 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:57:20.833 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:20.831+0000 7fe87effd700 1 -- 172.21.15.59:0/1436669368 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fe8780540d0 con 0x7fe8880cf2b0 2023-11-24T07:57:21.120 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:21.118+0000 7fe88cd4e700 1 -- 172.21.15.59:0/1436669368 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rbd-mirror"} v 0) v1 -- 0x7fe8880c8880 con 0x7fe8880cf2b0 2023-11-24T07:57:21.123 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:21.122+0000 7fe87effd700 1 -- 172.21.15.59:0/1436669368 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rbd-mirror"}]=0 v11) v1 ==== 97+0+0 (secure 0 0 0) 0x7fe8780576f0 con 0x7fe8880cf2b0 2023-11-24T07:57:21.126 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:21.124+0000 7fe88cd4e700 1 -- 172.21.15.59:0/1436669368 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7fe86806aed0 msgr2=0x7fe86806d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:57:21.126 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:21.124+0000 7fe88cd4e700 1 --2- 172.21.15.59:0/1436669368 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7fe86806aed0 0x7fe86806d390 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fe870000c00 tx=0x7fe870008030 comp rx=0 tx=0).stop 2023-11-24T07:57:21.126 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:21.124+0000 7fe88cd4e700 1 -- 172.21.15.59:0/1436669368 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe8880cf2b0 msgr2=0x7fe88815cbb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:57:21.126 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:21.124+0000 7fe88cd4e700 1 --2- 172.21.15.59:0/1436669368 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe8880cf2b0 0x7fe88815cbb0 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7fe87800e810 tx=0x7fe87800eb20 comp rx=0 tx=0).stop 2023-11-24T07:57:21.127 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:21.124+0000 7fe88cd4e700 1 -- 172.21.15.59:0/1436669368 shutdown_connections 2023-11-24T07:57:21.127 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:21.124+0000 7fe88cd4e700 1 --2- 172.21.15.59:0/1436669368 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7fe86806aed0 0x7fe86806d390 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:21.127 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:21.124+0000 7fe88cd4e700 1 --2- 172.21.15.59:0/1436669368 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fe88815bde0 0x7fe888158260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:21.127 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:21.124+0000 7fe88cd4e700 1 --2- 172.21.15.59:0/1436669368 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe8880ce5c0 0x7fe8880b0110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:21.127 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:21.124+0000 7fe88cd4e700 1 --2- 172.21.15.59:0/1436669368 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe8880cf2b0 0x7fe88815cbb0 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:21.127 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:21.124+0000 7fe88cd4e700 1 -- 172.21.15.59:0/1436669368 >> 172.21.15.59:0/1436669368 conn(0x7fe8880b3720 msgr2=0x7fe88815a070 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:57:21.127 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:21.124+0000 7fe88cd4e700 1 -- 172.21.15.59:0/1436669368 shutdown_connections 2023-11-24T07:57:21.127 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:21.124+0000 7fe88cd4e700 1 -- 172.21.15.59:0/1436669368 wait complete. 2023-11-24T07:57:21.139 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rgw 2023-11-24T07:57:21.265 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:21.263+0000 7f2e62e61700 1 Processor -- start 2023-11-24T07:57:21.265 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:21.263+0000 7f2e62e61700 1 -- start start 2023-11-24T07:57:21.267 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:21.264+0000 7f2e62e61700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f2e5c0b5700 0x7f2e5c06d810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:57:21.268 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:21.264+0000 7f2e62e61700 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f2e5c15e8b0 0x7f2e5c15ed20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:57:21.268 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:21.264+0000 7f2e62e61700 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2e5c0ce6a0 con 0x7f2e5c1446a0 2023-11-24T07:57:21.268 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:21.264+0000 7f2e62e61700 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2e5c15f260 con 0x7f2e5c0b5700 2023-11-24T07:57:21.268 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:21.264+0000 7f2e62e61700 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2e5c06dd50 con 0x7f2e5c15e8b0 2023-11-24T07:57:21.268 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:21.264+0000 7f2e5bfff700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f2e5c0b5700 0x7f2e5c06d810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:57:21.268 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:21.264+0000 7f2e5bfff700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f2e5c0b5700 0x7f2e5c06d810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:51558/0 (socket says 172.21.15.59:51558) 2023-11-24T07:57:21.268 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:21.264+0000 7f2e5bfff700 1 -- 172.21.15.59:0/1140260922 learned_addr learned my addr 172.21.15.59:0/1140260922 (peer_addr_for_me v2:172.21.15.59:0/0) 2023-11-24T07:57:21.268 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:21.265+0000 7f2e613fe700 1 --2- 172.21.15.59:0/1140260922 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f2e5c15e8b0 0x7f2e5c15ed20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:57:21.268 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:21.265+0000 7f2e5bfff700 1 -- 172.21.15.59:0/1140260922 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f2e5c15e8b0 msgr2=0x7f2e5c15ed20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T07:57:21.269 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:21.265+0000 7f2e5bfff700 1 --2- 172.21.15.59:0/1140260922 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f2e5c15e8b0 0x7f2e5c15ed20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:21.269 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:21.265+0000 7f2e5bfff700 1 -- 172.21.15.59:0/1140260922 >> v1:172.21.15.76:6789/0 conn(0x7f2e5c1446a0 legacy=0x7f2e5c0cdf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T07:57:21.269 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:21.265+0000 7f2e5bfff700 1 -- 172.21.15.59:0/1140260922 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2e5c144a80 con 0x7f2e5c0b5700 2023-11-24T07:57:21.269 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:21.265+0000 7f2e5bfff700 1 --2- 172.21.15.59:0/1140260922 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2e5c0b5700 0x7f2e5c06d810 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f2e480098f0 tx=0x7f2e48009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bfe5706060d6af6c server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:57:21.269 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:21.265+0000 7f2e5b7fe700 1 -- 172.21.15.59:0/1140260922 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e48004230 con 0x7f2e5c0b5700 2023-11-24T07:57:21.269 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:21.265+0000 7f2e5b7fe700 1 -- 172.21.15.59:0/1140260922 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2e48004390 con 0x7f2e5c0b5700 2023-11-24T07:57:21.269 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:21.265+0000 7f2e5b7fe700 1 -- 172.21.15.59:0/1140260922 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e480036a0 con 0x7f2e5c0b5700 2023-11-24T07:57:21.269 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:21.265+0000 7f2e62e61700 1 -- 172.21.15.59:0/1140260922 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2e5c0b5700 msgr2=0x7f2e5c06d810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:57:21.269 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:21.265+0000 7f2e62e61700 1 --2- 172.21.15.59:0/1140260922 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2e5c0b5700 0x7f2e5c06d810 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f2e480098f0 tx=0x7f2e48009c00 comp rx=0 tx=0).stop 2023-11-24T07:57:21.269 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:21.266+0000 7f2e62e61700 1 -- 172.21.15.59:0/1140260922 shutdown_connections 2023-11-24T07:57:21.269 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:21.266+0000 7f2e62e61700 1 --2- 172.21.15.59:0/1140260922 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f2e5c15e8b0 0x7f2e5c15ed20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:21.270 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:21.266+0000 7f2e62e61700 1 --2- 172.21.15.59:0/1140260922 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2e5c0b5700 0x7f2e5c06d810 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:21.270 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:21.266+0000 7f2e62e61700 1 -- 172.21.15.59:0/1140260922 >> 172.21.15.59:0/1140260922 conn(0x7f2e5c0d0ad0 msgr2=0x7f2e5c0d0ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:57:21.270 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:21.266+0000 7f2e62e61700 1 -- 172.21.15.59:0/1140260922 shutdown_connections 2023-11-24T07:57:21.271 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:21.266+0000 7f2e62e61700 1 -- 172.21.15.59:0/1140260922 wait complete. 2023-11-24T07:57:21.273 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:21.267+0000 7f2e62e61700 1 Processor -- start 2023-11-24T07:57:21.273 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:21.267+0000 7f2e62e61700 1 -- start start 2023-11-24T07:57:21.273 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:21.268+0000 7f2e62e61700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2e5c0b5700 0x7f2e5c1f48b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:57:21.274 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:21.268+0000 7f2e62e61700 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f2e5c1446a0 0x7f2e5c0c9ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:57:21.274 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:21.268+0000 7f2e62e61700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2e5c15e8b0 0x7f2e5c1436b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:57:21.274 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:21.268+0000 7f2e62e61700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e5c1f4df0 con 0x7f2e5c0b5700 2023-11-24T07:57:21.274 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:21.268+0000 7f2e62e61700 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e5c0cc810 con 0x7f2e5c15e8b0 2023-11-24T07:57:21.274 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:21.268+0000 7f2e62e61700 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2e5c0ca530 con 0x7f2e5c1446a0 2023-11-24T07:57:21.274 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:21.268+0000 7f2e60bfd700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2e5c0b5700 0x7f2e5c1f48b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:57:21.274 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:21.268+0000 7f2e60bfd700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2e5c0b5700 0x7f2e5c1f48b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:51562/0 (socket says 172.21.15.59:51562) 2023-11-24T07:57:21.274 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:21.268+0000 7f2e60bfd700 1 -- 172.21.15.59:0/3813375076 learned_addr learned my addr 172.21.15.59:0/3813375076 (peer_addr_for_me v2:172.21.15.59:0/0) 2023-11-24T07:57:21.275 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:21.268+0000 7f2e5bfff700 1 --2- 172.21.15.59:0/3813375076 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f2e5c1446a0 0x7f2e5c0c9ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:57:21.275 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:21.268+0000 7f2e613fe700 1 --2- 172.21.15.59:0/3813375076 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2e5c15e8b0 0x7f2e5c1436b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:57:21.275 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:21.268+0000 7f2e60bfd700 1 -- 172.21.15.59:0/3813375076 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f2e5c1446a0 msgr2=0x7f2e5c0c9ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:57:21.275 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:21.268+0000 7f2e60bfd700 1 --2- 172.21.15.59:0/3813375076 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f2e5c1446a0 0x7f2e5c0c9ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:21.275 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:21.268+0000 7f2e60bfd700 1 -- 172.21.15.59:0/3813375076 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2e5c15e8b0 msgr2=0x7f2e5c1436b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:57:21.275 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:21.268+0000 7f2e60bfd700 1 --2- 172.21.15.59:0/3813375076 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2e5c15e8b0 0x7f2e5c1436b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:21.275 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:21.268+0000 7f2e60bfd700 1 -- 172.21.15.59:0/3813375076 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2e48009550 con 0x7f2e5c0b5700 2023-11-24T07:57:21.275 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:21.268+0000 7f2e613fe700 1 --2- 172.21.15.59:0/3813375076 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2e5c15e8b0 0x7f2e5c1436b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-24T07:57:21.275 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:21.268+0000 7f2e60bfd700 1 --2- 172.21.15.59:0/3813375076 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2e5c0b5700 0x7f2e5c1f48b0 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f2e5000c4d0 tx=0x7f2e5000c7e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:57:21.275 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:21.269+0000 7f2e597fa700 1 -- 172.21.15.59:0/3813375076 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e50011790 con 0x7f2e5c0b5700 2023-11-24T07:57:21.275 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:21.269+0000 7f2e597fa700 1 -- 172.21.15.59:0/3813375076 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2e500118f0 con 0x7f2e5c0b5700 2023-11-24T07:57:21.275 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:21.269+0000 7f2e597fa700 1 -- 172.21.15.59:0/3813375076 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e50011ca0 con 0x7f2e5c0b5700 2023-11-24T07:57:21.276 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:21.269+0000 7f2e62e61700 1 -- 172.21.15.59:0/3813375076 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2e5c0cbac0 con 0x7f2e5c0b5700 2023-11-24T07:57:21.276 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:21.269+0000 7f2e62e61700 1 -- 172.21.15.59:0/3813375076 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2e5c0cc010 con 0x7f2e5c0b5700 2023-11-24T07:57:21.276 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:21.270+0000 7f2e62e61700 1 -- 172.21.15.59:0/3813375076 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2e5c067f60 con 0x7f2e5c0b5700 2023-11-24T07:57:21.276 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:21.270+0000 7f2e597fa700 1 -- 172.21.15.59:0/3813375076 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f2e5001c460 con 0x7f2e5c0b5700 2023-11-24T07:57:21.276 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:21.271+0000 7f2e597fa700 1 --2- 172.21.15.59:0/3813375076 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f2e4006aed0 0x7f2e4006d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:57:21.276 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:21.271+0000 7f2e597fa700 1 -- 172.21.15.59:0/3813375076 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f2e50089060 con 0x7f2e5c0b5700 2023-11-24T07:57:21.276 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:21.271+0000 7f2e5bfff700 1 --2- 172.21.15.59:0/3813375076 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f2e4006aed0 0x7f2e4006d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:57:21.276 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:21.271+0000 7f2e5bfff700 1 --2- 172.21.15.59:0/3813375076 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f2e4006aed0 0x7f2e4006d390 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f2e48009130 tx=0x7f2e48008030 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:57:21.276 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:21.275+0000 7f2e597fa700 1 -- 172.21.15.59:0/3813375076 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f2e50054460 con 0x7f2e5c0b5700 2023-11-24T07:57:21.560 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:21.559+0000 7f2e62e61700 1 -- 172.21.15.59:0/3813375076 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rgw"} v 0) v1 -- 0x7f2e5c0d16f0 con 0x7f2e5c0b5700 2023-11-24T07:57:21.562 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:21.560+0000 7f2e597fa700 1 -- 172.21.15.59:0/3813375076 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rgw"}]=0 v12) v1 ==== 90+0+0 (secure 0 0 0) 0x7f2e50057a80 con 0x7f2e5c0b5700 2023-11-24T07:57:21.565 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:21.563+0000 7f2e62e61700 1 -- 172.21.15.59:0/3813375076 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f2e4006aed0 msgr2=0x7f2e4006d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:57:21.565 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:21.563+0000 7f2e62e61700 1 --2- 172.21.15.59:0/3813375076 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f2e4006aed0 0x7f2e4006d390 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f2e48009130 tx=0x7f2e48008030 comp rx=0 tx=0).stop 2023-11-24T07:57:21.565 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:21.563+0000 7f2e62e61700 1 -- 172.21.15.59:0/3813375076 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2e5c0b5700 msgr2=0x7f2e5c1f48b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:57:21.565 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:21.563+0000 7f2e62e61700 1 --2- 172.21.15.59:0/3813375076 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2e5c0b5700 0x7f2e5c1f48b0 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f2e5000c4d0 tx=0x7f2e5000c7e0 comp rx=0 tx=0).stop 2023-11-24T07:57:21.565 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:21.563+0000 7f2e62e61700 1 -- 172.21.15.59:0/3813375076 shutdown_connections 2023-11-24T07:57:21.565 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:21.563+0000 7f2e62e61700 1 --2- 172.21.15.59:0/3813375076 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f2e4006aed0 0x7f2e4006d390 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:21.565 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:21.563+0000 7f2e62e61700 1 --2- 172.21.15.59:0/3813375076 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2e5c15e8b0 0x7f2e5c1436b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:21.565 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:21.563+0000 7f2e62e61700 1 --2- 172.21.15.59:0/3813375076 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2e5c0b5700 0x7f2e5c1f48b0 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:21.565 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:21.563+0000 7f2e62e61700 1 --2- 172.21.15.59:0/3813375076 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f2e5c1446a0 0x7f2e5c0c9ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:21.565 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:21.563+0000 7f2e62e61700 1 -- 172.21.15.59:0/3813375076 >> 172.21.15.59:0/3813375076 conn(0x7f2e5c0d0ad0 msgr2=0x7f2e5c0b37f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:57:21.566 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:21.563+0000 7f2e62e61700 1 -- 172.21.15.59:0/3813375076 shutdown_connections 2023-11-24T07:57:21.566 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:21.563+0000 7f2e62e61700 1 -- 172.21.15.59:0/3813375076 wait complete. 2023-11-24T07:57:21.577 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-24T07:57:21.703 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:21.701+0000 7f6f1f252700 1 Processor -- start 2023-11-24T07:57:21.703 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:21.701+0000 7f6f1f252700 1 -- start start 2023-11-24T07:57:21.704 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:21.702+0000 7f6f1f252700 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f6f180b4150 0x7f6f1806de50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:57:21.704 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:21.702+0000 7f6f1f252700 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f6f180b5ab0 0x7f6f1806e390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:57:21.705 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:21.702+0000 7f6f1f252700 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6f18071f50 con 0x7f6f1806b620 2023-11-24T07:57:21.705 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:21.702+0000 7f6f1f252700 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6f18072230 con 0x7f6f180b5ab0 2023-11-24T07:57:21.705 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:21.702+0000 7f6f1f252700 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6f18072510 con 0x7f6f180b4150 2023-11-24T07:57:21.705 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:21.702+0000 7f6f1cfee700 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7f6f1806b620 0x7f6f180b3a40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:55044/0 (socket says 172.21.15.59:55044) 2023-11-24T07:57:21.705 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:21.702+0000 7f6f1cfee700 1 -- 172.21.15.59:0/3202848805 learned_addr learned my addr 172.21.15.59:0/3202848805 (peer_addr_for_me v1:172.21.15.59:0/0) 2023-11-24T07:57:21.705 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:21.702+0000 7f6f17fff700 1 --2- 172.21.15.59:0/3202848805 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f6f180b4150 0x7f6f1806de50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:57:21.705 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:21.702+0000 7f6f1d7ef700 1 --2- 172.21.15.59:0/3202848805 >> v2:172.21.15.76:3300/0 conn(0x7f6f180b5ab0 0x7f6f1806e390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:57:21.705 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:21.702+0000 7f6f177fe700 1 -- 172.21.15.59:0/3202848805 <== mon.0 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3670976699 0 0) 0x7f6f18071f50 con 0x7f6f1806b620 2023-11-24T07:57:21.705 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:21.702+0000 7f6f177fe700 1 -- 172.21.15.59:0/3202848805 --> v1:172.21.15.59:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6f080035b0 con 0x7f6f1806b620 2023-11-24T07:57:21.705 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:21.702+0000 7f6f177fe700 1 -- 172.21.15.59:0/3202848805 <== mon.0 v1:172.21.15.59:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1572243942 0 0) 0x7f6f080035b0 con 0x7f6f1806b620 2023-11-24T07:57:21.705 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:21.702+0000 7f6f17fff700 1 -- 172.21.15.59:0/3202848805 >> v2:172.21.15.76:3300/0 conn(0x7f6f180b5ab0 msgr2=0x7f6f1806e390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T07:57:21.705 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:21.702+0000 7f6f17fff700 1 --2- 172.21.15.59:0/3202848805 >> v2:172.21.15.76:3300/0 conn(0x7f6f180b5ab0 0x7f6f1806e390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:21.705 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:21.702+0000 7f6f17fff700 1 -- 172.21.15.59:0/3202848805 >> v1:172.21.15.59:6789/0 conn(0x7f6f1806b620 legacy=0x7f6f180b3a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:57:21.706 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:21.703+0000 7f6f17fff700 1 -- 172.21.15.59:0/3202848805 --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6f180727f0 con 0x7f6f180b4150 2023-11-24T07:57:21.706 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:21.703+0000 7f6f17fff700 1 --2- 172.21.15.59:0/3202848805 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f6f180b4150 0x7f6f1806de50 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f6f04009840 tx=0x7f6f04009b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4f298996a36b20cd server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:57:21.706 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:21.703+0000 7f6f177fe700 1 -- 172.21.15.59:0/3202848805 <== mon.2 v2:172.21.15.76:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f0c001670 con 0x7f6f180b4150 2023-11-24T07:57:21.706 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:21.703+0000 7f6f177fe700 1 -- 172.21.15.59:0/3202848805 <== mon.2 v2:172.21.15.76:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6f04004030 con 0x7f6f180b4150 2023-11-24T07:57:21.706 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:21.703+0000 7f6f177fe700 1 -- 172.21.15.59:0/3202848805 <== mon.2 v2:172.21.15.76:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f040043c0 con 0x7f6f180b4150 2023-11-24T07:57:21.707 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:21.703+0000 7f6f1f252700 1 -- 172.21.15.59:0/3202848805 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f6f180b4150 msgr2=0x7f6f1806de50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:57:21.707 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:21.703+0000 7f6f1f252700 1 --2- 172.21.15.59:0/3202848805 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f6f180b4150 0x7f6f1806de50 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f6f04009840 tx=0x7f6f04009b50 comp rx=0 tx=0).stop 2023-11-24T07:57:21.707 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:21.704+0000 7f6f1f252700 1 -- 172.21.15.59:0/3202848805 shutdown_connections 2023-11-24T07:57:21.707 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:21.704+0000 7f6f1f252700 1 --2- 172.21.15.59:0/3202848805 >> v2:172.21.15.76:3300/0 conn(0x7f6f180b5ab0 0x7f6f1806e390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:21.707 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:21.704+0000 7f6f1f252700 1 --2- 172.21.15.59:0/3202848805 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f6f180b4150 0x7f6f1806de50 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:21.707 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:21.704+0000 7f6f1f252700 1 -- 172.21.15.59:0/3202848805 >> 172.21.15.59:0/3202848805 conn(0x7f6f180b2740 msgr2=0x7f6f180b2b20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:57:21.707 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:21.704+0000 7f6f1f252700 1 -- 172.21.15.59:0/3202848805 shutdown_connections 2023-11-24T07:57:21.707 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:21.704+0000 7f6f1f252700 1 -- 172.21.15.59:0/3202848805 wait complete. 2023-11-24T07:57:21.707 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:21.705+0000 7f6f1f252700 1 Processor -- start 2023-11-24T07:57:21.708 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:21.705+0000 7f6f1f252700 1 -- start start 2023-11-24T07:57:21.709 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:21.706+0000 7f6f1f252700 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f6f1806b620 0x7f6f181d7f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:57:21.709 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:21.706+0000 7f6f1f252700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6f180b4150 0x7f6f181f7940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:57:21.709 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:21.706+0000 7f6f1f252700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6f180b5ab0 0x7f6f1806f1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:57:21.710 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:21.706+0000 7f6f1f252700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f181d8480 con 0x7f6f180b4150 2023-11-24T07:57:21.710 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:21.706+0000 7f6f1f252700 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f1806e110 con 0x7f6f180b5ab0 2023-11-24T07:57:21.710 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:21.706+0000 7f6f1f252700 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6f1806e3f0 con 0x7f6f1806b620 2023-11-24T07:57:21.710 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:21.706+0000 7f6f17fff700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6f180b4150 0x7f6f181f7940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:57:21.710 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:21.706+0000 7f6f17fff700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6f180b4150 0x7f6f181f7940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:51578/0 (socket says 172.21.15.59:51578) 2023-11-24T07:57:21.710 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:21.706+0000 7f6f17fff700 1 -- 172.21.15.59:0/1062871702 learned_addr learned my addr 172.21.15.59:0/1062871702 (peer_addr_for_me v2:172.21.15.59:0/0) 2023-11-24T07:57:21.710 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:21.707+0000 7f6f1d7ef700 1 --2- 172.21.15.59:0/1062871702 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6f180b5ab0 0x7f6f1806f1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:57:21.710 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:21.707+0000 7f6f1cfee700 1 --2- 172.21.15.59:0/1062871702 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f6f1806b620 0x7f6f181d7f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:57:21.710 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:21.707+0000 7f6f17fff700 1 -- 172.21.15.59:0/1062871702 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f6f1806b620 msgr2=0x7f6f181d7f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:57:21.710 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:21.707+0000 7f6f17fff700 1 --2- 172.21.15.59:0/1062871702 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f6f1806b620 0x7f6f181d7f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:21.710 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:21.707+0000 7f6f17fff700 1 -- 172.21.15.59:0/1062871702 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6f180b5ab0 msgr2=0x7f6f1806f1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:57:21.710 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:21.707+0000 7f6f17fff700 1 --2- 172.21.15.59:0/1062871702 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6f180b5ab0 0x7f6f1806f1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:21.710 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:21.707+0000 7f6f17fff700 1 -- 172.21.15.59:0/1062871702 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6f040094a0 con 0x7f6f180b4150 2023-11-24T07:57:21.711 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:21.707+0000 7f6f1cfee700 1 --2- 172.21.15.59:0/1062871702 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f6f1806b620 0x7f6f181d7f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T07:57:21.711 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:21.707+0000 7f6f17fff700 1 --2- 172.21.15.59:0/1062871702 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6f180b4150 0x7f6f181f7940 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f6f04000c00 tx=0x7f6f04004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:57:21.711 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:21.707+0000 7f6f157fa700 1 -- 172.21.15.59:0/1062871702 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f04004570 con 0x7f6f180b4150 2023-11-24T07:57:21.711 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:21.707+0000 7f6f157fa700 1 -- 172.21.15.59:0/1062871702 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6f0401f070 con 0x7f6f180b4150 2023-11-24T07:57:21.711 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:21.707+0000 7f6f1f252700 1 -- 172.21.15.59:0/1062871702 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6f1806e6d0 con 0x7f6f180b4150 2023-11-24T07:57:21.711 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:21.707+0000 7f6f157fa700 1 -- 172.21.15.59:0/1062871702 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f04011850 con 0x7f6f180b4150 2023-11-24T07:57:21.711 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:21.707+0000 7f6f1f252700 1 -- 172.21.15.59:0/1062871702 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6f1806ec90 con 0x7f6f180b4150 2023-11-24T07:57:21.711 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:21.708+0000 7f6f1f252700 1 -- 172.21.15.59:0/1062871702 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6f1806f000 con 0x7f6f180b4150 2023-11-24T07:57:21.712 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:21.708+0000 7f6f157fa700 1 -- 172.21.15.59:0/1062871702 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f6f040119b0 con 0x7f6f180b4150 2023-11-24T07:57:21.712 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:21.709+0000 7f6f157fa700 1 --2- 172.21.15.59:0/1062871702 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f6efc06aed0 0x7f6efc06d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:57:21.712 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:21.709+0000 7f6f157fa700 1 -- 172.21.15.59:0/1062871702 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f6f04021070 con 0x7f6f180b4150 2023-11-24T07:57:21.712 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:21.709+0000 7f6f1cfee700 1 --2- 172.21.15.59:0/1062871702 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f6efc06aed0 0x7f6efc06d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:57:21.712 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:21.709+0000 7f6f1cfee700 1 --2- 172.21.15.59:0/1062871702 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f6efc06aed0 0x7f6efc06d390 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f6f0c0078f0 tx=0x7f6f0c009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:57:21.715 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:21.713+0000 7f6f157fa700 1 -- 172.21.15.59:0/1062871702 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f6f04056010 con 0x7f6f180b4150 2023-11-24T07:57:21.916 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:21.914+0000 7f6f1f252700 1 -- 172.21.15.59:0/1062871702 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f6f180d0430 con 0x7f6f180b4150 2023-11-24T07:57:21.917 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:21.915+0000 7f6f157fa700 1 -- 172.21.15.59:0/1062871702 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v29) v1 ==== 74+0+12785 (secure 0 0 0) 0x7f6f04059630 con 0x7f6f180b4150 2023-11-24T07:57:21.917 INFO:teuthology.orchestra.run.smithi059.stdout: 2023-11-24T07:57:21.917 INFO:teuthology.orchestra.run.smithi059.stdout:{"epoch":29,"fsid":"9f0e8741-b8f9-48d5-a2c3-09f45903303d","created":"2023-11-24T07:56:06.552729+0000","modified":"2023-11-24T07:57:18.485722+0000","last_up_change":"2023-11-24T07:56:17.372352+0000","last_in_change":"2023-11-24T07:56:09.073250+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-24T07:56:18.352823+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"6d8f9e9b-1782-4e41-928b-a39f2a8ebc06","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6800","nonce":3373878772},{"type":"v1","addr":"172.21.15.59:6801","nonce":3373878772}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6802","nonce":3373878772},{"type":"v1","addr":"172.21.15.59:6803","nonce":3373878772}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6806","nonce":3373878772},{"type":"v1","addr":"172.21.15.59:6807","nonce":3373878772}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6804","nonce":3373878772},{"type":"v1","addr":"172.21.15.59:6805","nonce":3373878772}]},"public_addr":"172.21.15.59:6801/3373878772","cluster_addr":"172.21.15.59:6803/3373878772","heartbeat_back_addr":"172.21.15.59:6807/3373878772","heartbeat_front_addr":"172.21.15.59:6805/3373878772","state":["exists","up"]},{"osd":1,"uuid":"e8d882aa-52e1-4c04-9e1e-c44ef9204a5f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6824","nonce":648258563},{"type":"v1","addr":"172.21.15.59:6825","nonce":648258563}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6826","nonce":648258563},{"type":"v1","addr":"172.21.15.59:6827","nonce":648258563}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6830","nonce":648258563},{"type":"v1","addr":"172.21.15.59:6831","nonce":648258563}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6828","nonce":648258563},{"type":"v1","addr":"172.21.15.59:6829","nonce":648258563}]},"public_addr":"172.21.15.59:6825/648258563","cluster_addr":"172.21.15.59:6827/648258563","heartbeat_back_addr":"172.21.15.59:6831/648258563","heartbeat_front_addr":"172.21.15.59:6829/648258563","state":["exists","up"]},{"osd":2,"uuid":"69eebfa0-695b-4b21-90b9-8a3537168a01","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6808","nonce":1840430654},{"type":"v1","addr":"172.21.15.59:6809","nonce":1840430654}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6810","nonce":1840430654},{"type":"v1","addr":"172.21.15.59:6811","nonce":1840430654}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6814","nonce":1840430654},{"type":"v1","addr":"172.21.15.59:6815","nonce":1840430654}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6812","nonce":1840430654},{"type":"v1","addr":"172.21.15.59:6813","nonce":1840430654}]},"public_addr":"172.21.15.59:6809/1840430654","cluster_addr":"172.21.15.59:6811/1840430654","heartbeat_back_addr":"172.21.15.59:6815/1840430654","heartbeat_front_addr":"172.21.15.59:6813/1840430654","state":["exists","up"]},{"osd":3,"uuid":"96fd77fc-f911-46ff-9445-6f0be2daa8f4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6816","nonce":2504025850},{"type":"v1","addr":"172.21.15.59:6817","nonce":2504025850}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6818","nonce":2504025850},{"type":"v1","addr":"172.21.15.59:6819","nonce":2504025850}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6822","nonce":2504025850},{"type":"v1","addr":"172.21.15.59:6823","nonce":2504025850}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6820","nonce":2504025850},{"type":"v1","addr":"172.21.15.59:6821","nonce":2504025850}]},"public_addr":"172.21.15.59:6817/2504025850","cluster_addr":"172.21.15.59:6819/2504025850","heartbeat_back_addr":"172.21.15.59:6823/2504025850","heartbeat_front_addr":"172.21.15.59:6821/2504025850","state":["exists","up"]},{"osd":4,"uuid":"7ac29fca-b5b2-4c95-9d69-2f164cbbc708","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6816","nonce":2382463431},{"type":"v1","addr":"172.21.15.76:6818","nonce":2382463431}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6820","nonce":2382463431},{"type":"v1","addr":"172.21.15.76:6822","nonce":2382463431}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6828","nonce":2382463431},{"type":"v1","addr":"172.21.15.76:6830","nonce":2382463431}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6824","nonce":2382463431},{"type":"v1","addr":"172.21.15.76:6826","nonce":2382463431}]},"public_addr":"172.21.15.76:6818/2382463431","cluster_addr":"172.21.15.76:6822/2382463431","heartbeat_back_addr":"172.21.15.76:6830/2382463431","heartbeat_front_addr":"172.21.15.76:6826/2382463431","state":["exists","up"]},{"osd":5,"uuid":"485bf711-3851-4238-a4f4-51d52e0a9c58","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6800","nonce":329334308},{"type":"v1","addr":"172.21.15.76:6802","nonce":329334308}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6805","nonce":329334308},{"type":"v1","addr":"172.21.15.76:6807","nonce":329334308}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6814","nonce":329334308},{"type":"v1","addr":"172.21.15.76:6815","nonce":329334308}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6810","nonce":329334308},{"type":"v1","addr":"172.21.15.76:6812","nonce":329334308}]},"public_addr":"172.21.15.76:6802/329334308","cluster_addr":"172.21.15.76:6807/329334308","heartbeat_back_addr":"172.21.15.76:6815/329334308","heartbeat_front_addr":"172.21.15.76:6812/329334308","state":["exists","up"]},{"osd":6,"uuid":"72b3f2fc-1e4a-4e32-968d-003647a0286f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6801","nonce":3946579410},{"type":"v1","addr":"172.21.15.76:6803","nonce":3946579410}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6804","nonce":3946579410},{"type":"v1","addr":"172.21.15.76:6806","nonce":3946579410}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6811","nonce":3946579410},{"type":"v1","addr":"172.21.15.76:6813","nonce":3946579410}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6808","nonce":3946579410},{"type":"v1","addr":"172.21.15.76:6809","nonce":3946579410}]},"public_addr":"172.21.15.76:6803/3946579410","cluster_addr":"172.21.15.76:6806/3946579410","heartbeat_back_addr":"172.21.15.76:6813/3946579410","heartbeat_front_addr":"172.21.15.76:6809/3946579410","state":["exists","up"]},{"osd":7,"uuid":"1ddceb90-6c5d-4d1a-ad56-99cb877f8c70","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6817","nonce":841761538},{"type":"v1","addr":"172.21.15.76:6819","nonce":841761538}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6821","nonce":841761538},{"type":"v1","addr":"172.21.15.76:6823","nonce":841761538}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6829","nonce":841761538},{"type":"v1","addr":"172.21.15.76:6831","nonce":841761538}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6825","nonce":841761538},{"type":"v1","addr":"172.21.15.76:6827","nonce":841761538}]},"public_addr":"172.21.15.76:6819/841761538","cluster_addr":"172.21.15.76:6823/841761538","heartbeat_back_addr":"172.21.15.76:6831/841761538","heartbeat_front_addr":"172.21.15.76:6827/841761538","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T07:56:14.294838+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T07:56:14.270779+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T07:56:14.231809+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T07:56:14.260983+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T07:56:14.302997+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T07:56:14.258600+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T07:56:14.251413+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T07:56:14.304727+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-24T07:57:21.919 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:21.917+0000 7f6f1f252700 1 -- 172.21.15.59:0/1062871702 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f6efc06aed0 msgr2=0x7f6efc06d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:57:21.919 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:21.917+0000 7f6f1f252700 1 --2- 172.21.15.59:0/1062871702 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f6efc06aed0 0x7f6efc06d390 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f6f0c0078f0 tx=0x7f6f0c009040 comp rx=0 tx=0).stop 2023-11-24T07:57:21.920 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:21.917+0000 7f6f1f252700 1 -- 172.21.15.59:0/1062871702 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6f180b4150 msgr2=0x7f6f181f7940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:57:21.920 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:21.917+0000 7f6f1f252700 1 --2- 172.21.15.59:0/1062871702 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6f180b4150 0x7f6f181f7940 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f6f04000c00 tx=0x7f6f04004750 comp rx=0 tx=0).stop 2023-11-24T07:57:21.920 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:21.918+0000 7f6f1f252700 1 -- 172.21.15.59:0/1062871702 shutdown_connections 2023-11-24T07:57:21.920 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:21.918+0000 7f6f1f252700 1 --2- 172.21.15.59:0/1062871702 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f6efc06aed0 0x7f6efc06d390 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:21.920 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:21.918+0000 7f6f1f252700 1 --2- 172.21.15.59:0/1062871702 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6f180b5ab0 0x7f6f1806f1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:21.920 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:21.918+0000 7f6f1f252700 1 --2- 172.21.15.59:0/1062871702 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f6f1806b620 0x7f6f181d7f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:21.920 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:21.918+0000 7f6f1f252700 1 --2- 172.21.15.59:0/1062871702 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6f180b4150 0x7f6f181f7940 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:21.920 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:21.918+0000 7f6f1f252700 1 -- 172.21.15.59:0/1062871702 >> 172.21.15.59:0/1062871702 conn(0x7f6f180b2740 msgr2=0x7f6f180aed60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:57:21.920 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:21.918+0000 7f6f1f252700 1 -- 172.21.15.59:0/1062871702 shutdown_connections 2023-11-24T07:57:21.920 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:21.918+0000 7f6f1f252700 1 -- 172.21.15.59:0/1062871702 wait complete. 2023-11-24T07:57:21.931 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-24T07:56:18.352823+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-24T07:57:21.932 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-24T07:57:22.060 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:22.058+0000 7f8d31d50700 1 Processor -- start 2023-11-24T07:57:22.061 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:22.059+0000 7f8d31d50700 1 -- start start 2023-11-24T07:57:22.070 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:22.059+0000 7f8d31d50700 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f8d2c143ae0 0x7f8d2c0b51b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:57:22.071 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:22.059+0000 7f8d31d50700 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f8d2c1440f0 0x7f8d2c071bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:57:22.071 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:22.059+0000 7f8d31d50700 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8d2c143ec0 con 0x7f8d2c0720f0 2023-11-24T07:57:22.071 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:22.059+0000 7f8d31d50700 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8d2c0b40a0 con 0x7f8d2c143ae0 2023-11-24T07:57:22.071 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:22.059+0000 7f8d31d50700 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8d2c072580 con 0x7f8d2c1440f0 2023-11-24T07:57:22.072 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:22.059+0000 7f8d2bfff700 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7f8d2c0720f0 0x7f8d2c15e8b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:55062/0 (socket says 172.21.15.59:55062) 2023-11-24T07:57:22.072 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:22.059+0000 7f8d2bfff700 1 -- 172.21.15.59:0/111333879 learned_addr learned my addr 172.21.15.59:0/111333879 (peer_addr_for_me v1:172.21.15.59:0/0) 2023-11-24T07:57:22.072 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:22.060+0000 7f8d2b7fe700 1 --2- 172.21.15.59:0/111333879 >> v2:172.21.15.76:3300/0 conn(0x7f8d2c143ae0 0x7f8d2c0b51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:57:22.073 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:22.060+0000 7f8d2affd700 1 --2- 172.21.15.59:0/111333879 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f8d2c1440f0 0x7f8d2c071bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:57:22.073 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:22.060+0000 7f8d2a7fc700 1 -- 172.21.15.59:0/111333879 <== mon.0 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3617020966 0 0) 0x7f8d2c143ec0 con 0x7f8d2c0720f0 2023-11-24T07:57:22.073 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:22.060+0000 7f8d2a7fc700 1 -- 172.21.15.59:0/111333879 --> v1:172.21.15.59:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8d180035b0 con 0x7f8d2c0720f0 2023-11-24T07:57:22.074 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:22.060+0000 7f8d2b7fe700 1 -- 172.21.15.59:0/111333879 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f8d2c1440f0 msgr2=0x7f8d2c071bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T07:57:22.074 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:22.060+0000 7f8d2b7fe700 1 --2- 172.21.15.59:0/111333879 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f8d2c1440f0 0x7f8d2c071bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:22.074 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:22.060+0000 7f8d2b7fe700 1 -- 172.21.15.59:0/111333879 >> v1:172.21.15.59:6789/0 conn(0x7f8d2c0720f0 legacy=0x7f8d2c15e8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:57:22.075 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:22.060+0000 7f8d2b7fe700 1 -- 172.21.15.59:0/111333879 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8d2c0b3ae0 con 0x7f8d2c143ae0 2023-11-24T07:57:22.075 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:22.060+0000 7f8d2b7fe700 1 --2- 172.21.15.59:0/111333879 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8d2c143ae0 0x7f8d2c0b51b0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f8d14009840 tx=0x7f8d14009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8e78395b341c82fc server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:57:22.076 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:22.060+0000 7f8d2a7fc700 1 -- 172.21.15.59:0/111333879 <== mon.1 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d14011830 con 0x7f8d2c143ae0 2023-11-24T07:57:22.076 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:22.060+0000 7f8d2a7fc700 1 -- 172.21.15.59:0/111333879 <== mon.1 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8d14011990 con 0x7f8d2c143ae0 2023-11-24T07:57:22.076 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:22.060+0000 7f8d2a7fc700 1 -- 172.21.15.59:0/111333879 <== mon.1 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d14011d20 con 0x7f8d2c143ae0 2023-11-24T07:57:22.077 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:22.061+0000 7f8d31d50700 1 -- 172.21.15.59:0/111333879 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8d2c143ae0 msgr2=0x7f8d2c0b51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:57:22.077 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:22.061+0000 7f8d31d50700 1 --2- 172.21.15.59:0/111333879 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8d2c143ae0 0x7f8d2c0b51b0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f8d14009840 tx=0x7f8d14009b50 comp rx=0 tx=0).stop 2023-11-24T07:57:22.078 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:22.061+0000 7f8d31d50700 1 -- 172.21.15.59:0/111333879 shutdown_connections 2023-11-24T07:57:22.078 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:22.061+0000 7f8d31d50700 1 --2- 172.21.15.59:0/111333879 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8d2c143ae0 0x7f8d2c0b51b0 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:22.078 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:22.061+0000 7f8d31d50700 1 --2- 172.21.15.59:0/111333879 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f8d2c1440f0 0x7f8d2c071bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:22.079 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:22.061+0000 7f8d31d50700 1 -- 172.21.15.59:0/111333879 >> 172.21.15.59:0/111333879 conn(0x7f8d2c0b2740 msgr2=0x7f8d2c0b2b20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:57:22.079 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:22.061+0000 7f8d31d50700 1 -- 172.21.15.59:0/111333879 shutdown_connections 2023-11-24T07:57:22.080 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:22.061+0000 7f8d31d50700 1 -- 172.21.15.59:0/111333879 wait complete. 2023-11-24T07:57:22.080 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:22.063+0000 7f8d31d50700 1 Processor -- start 2023-11-24T07:57:22.080 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:22.063+0000 7f8d31d50700 1 -- start start 2023-11-24T07:57:22.081 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:22.064+0000 7f8d31d50700 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f8d2c0720f0 0x7f8d2c142480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:57:22.081 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:22.064+0000 7f8d31d50700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8d2c143ae0 0x7f8d2c15bb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:57:22.081 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:22.064+0000 7f8d31d50700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8d2c1440f0 0x7f8d2c15c0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:57:22.082 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:22.064+0000 7f8d31d50700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d2c1429c0 con 0x7f8d2c1440f0 2023-11-24T07:57:22.082 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:22.064+0000 7f8d31d50700 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d2c15f290 con 0x7f8d2c143ae0 2023-11-24T07:57:22.083 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:22.064+0000 7f8d31d50700 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8d2c153af0 con 0x7f8d2c0720f0 2023-11-24T07:57:22.083 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:22.064+0000 7f8d2bfff700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8d2c1440f0 0x7f8d2c15c0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:57:22.084 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:22.064+0000 7f8d2bfff700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8d2c1440f0 0x7f8d2c15c0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:51594/0 (socket says 172.21.15.59:51594) 2023-11-24T07:57:22.084 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:22.064+0000 7f8d2bfff700 1 -- 172.21.15.59:0/1578381656 learned_addr learned my addr 172.21.15.59:0/1578381656 (peer_addr_for_me v2:172.21.15.59:0/0) 2023-11-24T07:57:22.084 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:22.064+0000 7f8d2affd700 1 --2- 172.21.15.59:0/1578381656 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8d2c143ae0 0x7f8d2c15bb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:57:22.084 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:22.064+0000 7f8d2b7fe700 1 --2- 172.21.15.59:0/1578381656 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f8d2c0720f0 0x7f8d2c142480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:57:22.085 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:22.064+0000 7f8d2bfff700 1 -- 172.21.15.59:0/1578381656 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f8d2c0720f0 msgr2=0x7f8d2c142480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:57:22.085 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:22.064+0000 7f8d2bfff700 1 --2- 172.21.15.59:0/1578381656 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f8d2c0720f0 0x7f8d2c142480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:22.085 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:22.064+0000 7f8d2bfff700 1 -- 172.21.15.59:0/1578381656 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8d2c143ae0 msgr2=0x7f8d2c15bb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:57:22.085 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:22.064+0000 7f8d2bfff700 1 --2- 172.21.15.59:0/1578381656 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8d2c143ae0 0x7f8d2c15bb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:22.085 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:22.064+0000 7f8d2bfff700 1 -- 172.21.15.59:0/1578381656 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8d140094a0 con 0x7f8d2c1440f0 2023-11-24T07:57:22.086 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:22.064+0000 7f8d2affd700 1 --2- 172.21.15.59:0/1578381656 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8d2c143ae0 0x7f8d2c15bb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-24T07:57:22.086 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:22.064+0000 7f8d2bfff700 1 --2- 172.21.15.59:0/1578381656 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8d2c1440f0 0x7f8d2c15c0b0 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f8d2400caa0 tx=0x7f8d2400ce60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:57:22.086 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:22.065+0000 7f8d23fff700 1 -- 172.21.15.59:0/1578381656 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d2400aa80 con 0x7f8d2c1440f0 2023-11-24T07:57:22.086 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:22.065+0000 7f8d23fff700 1 -- 172.21.15.59:0/1578381656 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8d2400abe0 con 0x7f8d2c1440f0 2023-11-24T07:57:22.086 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:22.065+0000 7f8d31d50700 1 -- 172.21.15.59:0/1578381656 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8d2c153dd0 con 0x7f8d2c1440f0 2023-11-24T07:57:22.086 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:22.065+0000 7f8d23fff700 1 -- 172.21.15.59:0/1578381656 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d24012400 con 0x7f8d2c1440f0 2023-11-24T07:57:22.087 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:22.065+0000 7f8d31d50700 1 -- 172.21.15.59:0/1578381656 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8d2c15e560 con 0x7f8d2c1440f0 2023-11-24T07:57:22.087 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:22.066+0000 7f8d31d50700 1 -- 172.21.15.59:0/1578381656 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8d2c153950 con 0x7f8d2c1440f0 2023-11-24T07:57:22.087 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:22.066+0000 7f8d23fff700 1 -- 172.21.15.59:0/1578381656 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f8d2401b3b0 con 0x7f8d2c1440f0 2023-11-24T07:57:22.087 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:22.066+0000 7f8d23fff700 1 --2- 172.21.15.59:0/1578381656 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f8d0c06aed0 0x7f8d0c06d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:57:22.087 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:22.066+0000 7f8d23fff700 1 -- 172.21.15.59:0/1578381656 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f8d24088f50 con 0x7f8d2c1440f0 2023-11-24T07:57:22.087 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:22.067+0000 7f8d2b7fe700 1 --2- 172.21.15.59:0/1578381656 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f8d0c06aed0 0x7f8d0c06d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:57:22.088 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:22.067+0000 7f8d2b7fe700 1 --2- 172.21.15.59:0/1578381656 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f8d0c06aed0 0x7f8d0c06d390 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f8d14009fd0 tx=0x7f8d14008030 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:57:22.088 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:22.071+0000 7f8d23fff700 1 -- 172.21.15.59:0/1578381656 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f8d2c153950 con 0x7f8d2c1440f0 2023-11-24T07:57:22.270 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:22.269+0000 7f8d31d50700 1 -- 172.21.15.59:0/1578381656 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f8d2c153950 con 0x7f8d2c1440f0 2023-11-24T07:57:22.271 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:22.269+0000 7f8d23fff700 1 -- 172.21.15.59:0/1578381656 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v29) v1 ==== 93+0+10 (secure 0 0 0) 0x7f8d24017020 con 0x7f8d2c1440f0 2023-11-24T07:57:22.271 INFO:teuthology.orchestra.run.smithi059.stdout:pg_num: 1 2023-11-24T07:57:22.273 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:22.271+0000 7f8d31d50700 1 -- 172.21.15.59:0/1578381656 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f8d0c06aed0 msgr2=0x7f8d0c06d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:57:22.273 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:22.271+0000 7f8d31d50700 1 --2- 172.21.15.59:0/1578381656 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f8d0c06aed0 0x7f8d0c06d390 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f8d14009fd0 tx=0x7f8d14008030 comp rx=0 tx=0).stop 2023-11-24T07:57:22.273 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:22.271+0000 7f8d31d50700 1 -- 172.21.15.59:0/1578381656 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8d2c1440f0 msgr2=0x7f8d2c15c0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:57:22.273 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:22.271+0000 7f8d31d50700 1 --2- 172.21.15.59:0/1578381656 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8d2c1440f0 0x7f8d2c15c0b0 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f8d2400caa0 tx=0x7f8d2400ce60 comp rx=0 tx=0).stop 2023-11-24T07:57:22.274 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:22.271+0000 7f8d31d50700 1 -- 172.21.15.59:0/1578381656 shutdown_connections 2023-11-24T07:57:22.274 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:22.271+0000 7f8d31d50700 1 --2- 172.21.15.59:0/1578381656 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f8d0c06aed0 0x7f8d0c06d390 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:22.274 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:22.271+0000 7f8d31d50700 1 --2- 172.21.15.59:0/1578381656 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8d2c1440f0 0x7f8d2c15c0b0 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:22.274 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:22.271+0000 7f8d31d50700 1 --2- 172.21.15.59:0/1578381656 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f8d2c0720f0 0x7f8d2c142480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:22.274 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:22.271+0000 7f8d31d50700 1 --2- 172.21.15.59:0/1578381656 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8d2c143ae0 0x7f8d2c15bb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:22.275 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:22.271+0000 7f8d31d50700 1 -- 172.21.15.59:0/1578381656 >> 172.21.15.59:0/1578381656 conn(0x7f8d2c0b2740 msgr2=0x7f8d2c0b4430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:57:22.275 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:22.271+0000 7f8d31d50700 1 -- 172.21.15.59:0/1578381656 shutdown_connections 2023-11-24T07:57:22.275 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:22.271+0000 7f8d31d50700 1 -- 172.21.15.59:0/1578381656 wait complete. 2023-11-24T07:57:22.286 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2023-11-24T07:57:22.411 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:22.409+0000 7f8f9bf0a700 1 Processor -- start 2023-11-24T07:57:22.411 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:22.410+0000 7f8f9bf0a700 1 -- start start 2023-11-24T07:57:22.414 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:22.411+0000 7f8f9bf0a700 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f8f940b33f0 0x7f8f9406afd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:57:22.414 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:22.411+0000 7f8f9bf0a700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f8f9406b510 0x7f8f9406db30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:57:22.414 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:22.411+0000 7f8f9bf0a700 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8f94142af0 con 0x7f8f94142710 2023-11-24T07:57:22.414 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:22.411+0000 7f8f9bf0a700 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8f9406b9c0 con 0x7f8f9406b510 2023-11-24T07:57:22.414 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:22.411+0000 7f8f9bf0a700 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8f940cf190 con 0x7f8f940b33f0 2023-11-24T07:57:22.414 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:22.411+0000 7f8f9a4a7700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f8f9406b510 0x7f8f9406db30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:57:22.414 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:22.411+0000 7f8f9a4a7700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f8f9406b510 0x7f8f9406db30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:51602/0 (socket says 172.21.15.59:51602) 2023-11-24T07:57:22.414 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:22.411+0000 7f8f9a4a7700 1 -- 172.21.15.59:0/832462862 learned_addr learned my addr 172.21.15.59:0/832462862 (peer_addr_for_me v2:172.21.15.59:0/0) 2023-11-24T07:57:22.414 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:22.411+0000 7f8f994a5700 1 --2- 172.21.15.59:0/832462862 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f8f940b33f0 0x7f8f9406afd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:57:22.415 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:22.411+0000 7f8f9a4a7700 1 -- 172.21.15.59:0/832462862 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f8f940b33f0 msgr2=0x7f8f9406afd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T07:57:22.415 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:22.411+0000 7f8f9a4a7700 1 --2- 172.21.15.59:0/832462862 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f8f940b33f0 0x7f8f9406afd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:22.415 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:22.411+0000 7f8f9a4a7700 1 -- 172.21.15.59:0/832462862 >> v1:172.21.15.76:6789/0 conn(0x7f8f94142710 legacy=0x7f8f940724e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:57:22.415 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:22.411+0000 7f8f9a4a7700 1 -- 172.21.15.59:0/832462862 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8f940cf410 con 0x7f8f9406b510 2023-11-24T07:57:22.415 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:22.411+0000 7f8f9a4a7700 1 --2- 172.21.15.59:0/832462862 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8f9406b510 0x7f8f9406db30 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f8f8c009e50 tx=0x7f8f8c00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=16255e43860c2fa server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:57:22.415 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:22.411+0000 7f8f98ca4700 1 -- 172.21.15.59:0/832462862 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f8c00e070 con 0x7f8f9406b510 2023-11-24T07:57:22.415 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:22.411+0000 7f8f98ca4700 1 -- 172.21.15.59:0/832462862 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8f8c00b910 con 0x7f8f9406b510 2023-11-24T07:57:22.415 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:22.411+0000 7f8f98ca4700 1 -- 172.21.15.59:0/832462862 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f8c00bcc0 con 0x7f8f9406b510 2023-11-24T07:57:22.415 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:22.412+0000 7f8f9bf0a700 1 -- 172.21.15.59:0/832462862 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8f9406b510 msgr2=0x7f8f9406db30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:57:22.415 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:22.412+0000 7f8f9bf0a700 1 --2- 172.21.15.59:0/832462862 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8f9406b510 0x7f8f9406db30 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f8f8c009e50 tx=0x7f8f8c00b330 comp rx=0 tx=0).stop 2023-11-24T07:57:22.415 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:22.412+0000 7f8f9bf0a700 1 -- 172.21.15.59:0/832462862 shutdown_connections 2023-11-24T07:57:22.415 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:22.412+0000 7f8f9bf0a700 1 --2- 172.21.15.59:0/832462862 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8f9406b510 0x7f8f9406db30 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:22.416 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:22.412+0000 7f8f9bf0a700 1 --2- 172.21.15.59:0/832462862 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f8f940b33f0 0x7f8f9406afd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:22.416 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:22.412+0000 7f8f9bf0a700 1 -- 172.21.15.59:0/832462862 >> 172.21.15.59:0/832462862 conn(0x7f8f940b27b0 msgr2=0x7f8f940b2b90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:57:22.416 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:22.412+0000 7f8f9bf0a700 1 -- 172.21.15.59:0/832462862 shutdown_connections 2023-11-24T07:57:22.416 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:22.412+0000 7f8f9bf0a700 1 -- 172.21.15.59:0/832462862 wait complete. 2023-11-24T07:57:22.418 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:22.413+0000 7f8f9bf0a700 1 Processor -- start 2023-11-24T07:57:22.419 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:22.413+0000 7f8f9bf0a700 1 -- start start 2023-11-24T07:57:22.419 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:22.414+0000 7f8f9bf0a700 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f8f9406b510 0x7f8f94157920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:57:22.419 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:22.414+0000 7f8f9bf0a700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8f940b33f0 0x7f8f94141720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:57:22.419 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:22.414+0000 7f8f9bf0a700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8f94142710 0x7f8f94143e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:57:22.419 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:22.414+0000 7f8f9bf0a700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f94157e60 con 0x7f8f94142710 2023-11-24T07:57:22.419 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:22.414+0000 7f8f9bf0a700 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f940aef20 con 0x7f8f940b33f0 2023-11-24T07:57:22.419 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:22.414+0000 7f8f9bf0a700 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8f940b5b90 con 0x7f8f9406b510 2023-11-24T07:57:22.419 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:22.414+0000 7f8f9a4a7700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8f94142710 0x7f8f94143e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:57:22.419 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:22.414+0000 7f8f9a4a7700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8f94142710 0x7f8f94143e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:51608/0 (socket says 172.21.15.59:51608) 2023-11-24T07:57:22.419 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:22.414+0000 7f8f9a4a7700 1 -- 172.21.15.59:0/598896416 learned_addr learned my addr 172.21.15.59:0/598896416 (peer_addr_for_me v2:172.21.15.59:0/0) 2023-11-24T07:57:22.419 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:22.414+0000 7f8f9a4a7700 1 -- 172.21.15.59:0/598896416 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f8f9406b510 msgr2=0x7f8f94157920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:57:22.419 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:22.414+0000 7f8f994a5700 1 --2- 172.21.15.59:0/598896416 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8f940b33f0 0x7f8f94141720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:57:22.419 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:22.414+0000 7f8f9a4a7700 1 --2- 172.21.15.59:0/598896416 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f8f9406b510 0x7f8f94157920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:22.420 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:22.414+0000 7f8f9a4a7700 1 -- 172.21.15.59:0/598896416 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8f940b33f0 msgr2=0x7f8f94141720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:57:22.420 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:22.414+0000 7f8f9a4a7700 1 --2- 172.21.15.59:0/598896416 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8f940b33f0 0x7f8f94141720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:22.420 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:22.414+0000 7f8f9a4a7700 1 -- 172.21.15.59:0/598896416 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8f8c009b30 con 0x7f8f94142710 2023-11-24T07:57:22.420 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:22.415+0000 7f8f9a4a7700 1 --2- 172.21.15.59:0/598896416 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8f94142710 0x7f8f94143e00 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f8f94070680 tx=0x7f8f8c004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:57:22.420 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:22.415+0000 7f8f867fc700 1 -- 172.21.15.59:0/598896416 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f8c004960 con 0x7f8f94142710 2023-11-24T07:57:22.420 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:22.415+0000 7f8f867fc700 1 -- 172.21.15.59:0/598896416 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8f8c0099b0 con 0x7f8f94142710 2023-11-24T07:57:22.420 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:22.415+0000 7f8f9bf0a700 1 -- 172.21.15.59:0/598896416 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8f940af1c0 con 0x7f8f94142710 2023-11-24T07:57:22.420 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:22.415+0000 7f8f867fc700 1 -- 172.21.15.59:0/598896416 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f8c01c680 con 0x7f8f94142710 2023-11-24T07:57:22.420 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:22.415+0000 7f8f9bf0a700 1 -- 172.21.15.59:0/598896416 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8f940af780 con 0x7f8f94142710 2023-11-24T07:57:22.420 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:22.416+0000 7f8f9bf0a700 1 -- 172.21.15.59:0/598896416 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8f94075480 con 0x7f8f94142710 2023-11-24T07:57:22.420 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:22.416+0000 7f8f867fc700 1 -- 172.21.15.59:0/598896416 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f8f8c01c7e0 con 0x7f8f94142710 2023-11-24T07:57:22.420 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:22.416+0000 7f8f867fc700 1 --2- 172.21.15.59:0/598896416 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f8f7806aed0 0x7f8f7806d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:57:22.420 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:22.417+0000 7f8f867fc700 1 -- 172.21.15.59:0/598896416 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f8f8c08afa0 con 0x7f8f94142710 2023-11-24T07:57:22.420 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:22.417+0000 7f8f99ca6700 1 --2- 172.21.15.59:0/598896416 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f8f7806aed0 0x7f8f7806d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:57:22.421 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:22.417+0000 7f8f99ca6700 1 --2- 172.21.15.59:0/598896416 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f8f7806aed0 0x7f8f7806d390 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f8f90000f60 tx=0x7f8f90007a40 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:57:22.422 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:22.421+0000 7f8f867fc700 1 -- 172.21.15.59:0/598896416 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f8f8c0bde90 con 0x7f8f94142710 2023-11-24T07:57:22.628 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:22.626+0000 7f8f9bf0a700 1 -- 172.21.15.59:0/598896416 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f8f94145b70 con 0x7f8f94142710 2023-11-24T07:57:22.628 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:22.627+0000 7f8f867fc700 1 -- 172.21.15.59:0/598896416 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v29) v1 ==== 84+0+65 (secure 0 0 0) 0x7f8f8c056430 con 0x7f8f94142710 2023-11-24T07:57:22.629 INFO:teuthology.orchestra.run.smithi059.stdout: 2023-11-24T07:57:22.629 INFO:teuthology.orchestra.run.smithi059.stdout:[ 2023-11-24T07:57:22.629 INFO:teuthology.orchestra.run.smithi059.stdout: { 2023-11-24T07:57:22.629 INFO:teuthology.orchestra.run.smithi059.stdout: "poolnum": 1, 2023-11-24T07:57:22.629 INFO:teuthology.orchestra.run.smithi059.stdout: "poolname": ".mgr" 2023-11-24T07:57:22.629 INFO:teuthology.orchestra.run.smithi059.stdout: } 2023-11-24T07:57:22.630 INFO:teuthology.orchestra.run.smithi059.stdout:] 2023-11-24T07:57:22.631 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:22.629+0000 7f8f9bf0a700 1 -- 172.21.15.59:0/598896416 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f8f7806aed0 msgr2=0x7f8f7806d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:57:22.631 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:22.629+0000 7f8f9bf0a700 1 --2- 172.21.15.59:0/598896416 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f8f7806aed0 0x7f8f7806d390 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f8f90000f60 tx=0x7f8f90007a40 comp rx=0 tx=0).stop 2023-11-24T07:57:22.631 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:22.629+0000 7f8f9bf0a700 1 -- 172.21.15.59:0/598896416 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8f94142710 msgr2=0x7f8f94143e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:57:22.631 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:22.629+0000 7f8f9bf0a700 1 --2- 172.21.15.59:0/598896416 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8f94142710 0x7f8f94143e00 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f8f94070680 tx=0x7f8f8c004ad0 comp rx=0 tx=0).stop 2023-11-24T07:57:22.632 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:22.629+0000 7f8f9bf0a700 1 -- 172.21.15.59:0/598896416 shutdown_connections 2023-11-24T07:57:22.632 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:22.629+0000 7f8f9bf0a700 1 --2- 172.21.15.59:0/598896416 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f8f7806aed0 0x7f8f7806d390 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:22.632 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:22.629+0000 7f8f9bf0a700 1 --2- 172.21.15.59:0/598896416 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8f94142710 0x7f8f94143e00 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:22.632 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:22.629+0000 7f8f9bf0a700 1 --2- 172.21.15.59:0/598896416 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f8f9406b510 0x7f8f94157920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:22.632 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:22.629+0000 7f8f9bf0a700 1 --2- 172.21.15.59:0/598896416 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8f940b33f0 0x7f8f94141720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:22.632 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:22.629+0000 7f8f9bf0a700 1 -- 172.21.15.59:0/598896416 >> 172.21.15.59:0/598896416 conn(0x7f8f940b27b0 msgr2=0x7f8f9406e6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:57:22.632 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:22.629+0000 7f8f9bf0a700 1 -- 172.21.15.59:0/598896416 shutdown_connections 2023-11-24T07:57:22.632 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:22.629+0000 7f8f9bf0a700 1 -- 172.21.15.59:0/598896416 wait complete. 2023-11-24T07:57:22.643 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2023-11-24T07:57:22.643 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2023-11-24T07:57:22.772 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:22.770+0000 7fc497802700 1 Processor -- start 2023-11-24T07:57:22.772 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:22.770+0000 7fc497802700 1 -- start start 2023-11-24T07:57:22.776 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:22.771+0000 7fc497802700 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fc4900b5ab0 0x7fc49006e560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:57:22.776 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:22.771+0000 7fc497802700 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc490071f50 con 0x7fc4900b4150 2023-11-24T07:57:22.776 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:22.771+0000 7fc497802700 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc490072230 con 0x7fc49006b620 2023-11-24T07:57:22.776 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:22.771+0000 7fc497802700 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc490072510 con 0x7fc4900b5ab0 2023-11-24T07:57:22.776 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:22.771+0000 7fc494d9d700 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7fc4900b4150 0x7fc49006de50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:55088/0 (socket says 172.21.15.59:55088) 2023-11-24T07:57:22.776 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:22.771+0000 7fc494d9d700 1 -- 172.21.15.59:0/2352016901 learned_addr learned my addr 172.21.15.59:0/2352016901 (peer_addr_for_me v1:172.21.15.59:0/0) 2023-11-24T07:57:22.777 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:22.772+0000 7fc495d9f700 1 --2- 172.21.15.59:0/2352016901 >> v2:172.21.15.76:3300/0 conn(0x7fc4900b5ab0 0x7fc49006e560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:57:22.777 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:22.772+0000 7fc487fff700 1 -- 172.21.15.59:0/2352016901 <== mon.0 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3697984382 0 0) 0x7fc490071f50 con 0x7fc4900b4150 2023-11-24T07:57:22.777 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:22.772+0000 7fc487fff700 1 -- 172.21.15.59:0/2352016901 --> v1:172.21.15.59:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc47c0035b0 con 0x7fc4900b4150 2023-11-24T07:57:22.777 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:22.772+0000 7fc487fff700 1 -- 172.21.15.59:0/2352016901 <== mon.1 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 917328202 0 0) 0x7fc490072230 con 0x7fc49006b620 2023-11-24T07:57:22.777 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:22.772+0000 7fc487fff700 1 -- 172.21.15.59:0/2352016901 --> v1:172.21.15.76:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc490071f50 con 0x7fc49006b620 2023-11-24T07:57:22.777 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:22.772+0000 7fc487fff700 1 -- 172.21.15.59:0/2352016901 <== mon.0 v1:172.21.15.59:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1172899618 0 0) 0x7fc47c0035b0 con 0x7fc4900b4150 2023-11-24T07:57:22.777 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:22.772+0000 7fc487fff700 1 -- 172.21.15.59:0/2352016901 >> v2:172.21.15.76:3300/0 conn(0x7fc4900b5ab0 msgr2=0x7fc49006e560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T07:57:22.777 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:22.772+0000 7fc487fff700 1 --2- 172.21.15.59:0/2352016901 >> v2:172.21.15.76:3300/0 conn(0x7fc4900b5ab0 0x7fc49006e560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:22.777 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:22.772+0000 7fc487fff700 1 -- 172.21.15.59:0/2352016901 >> v1:172.21.15.76:6789/0 conn(0x7fc49006b620 legacy=0x7fc4900b3a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:57:22.777 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:22.772+0000 7fc487fff700 1 -- 172.21.15.59:0/2352016901 --> v1:172.21.15.59:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc4900727f0 con 0x7fc4900b4150 2023-11-24T07:57:22.777 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:22.772+0000 7fc487fff700 1 -- 172.21.15.59:0/2352016901 <== mon.0 v1:172.21.15.59:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1312333566 0 0) 0x7fc48c001670 con 0x7fc4900b4150 2023-11-24T07:57:22.777 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:22.772+0000 7fc487fff700 1 -- 172.21.15.59:0/2352016901 >> v1:172.21.15.59:6789/0 conn(0x7fc4900b4150 legacy=0x7fc49006de50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:57:22.777 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:22.772+0000 7fc487fff700 1 --2- 172.21.15.59:0/2352016901 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fc47c003b90 0x7fc47c003f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:57:22.778 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:22.772+0000 7fc487fff700 1 --2- 172.21.15.59:0/2352016901 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc47c004850 0x7fc47c004cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:57:22.778 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:22.772+0000 7fc487fff700 1 --2- 172.21.15.59:0/2352016901 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc47c005210 0x7fc47c00ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:57:22.778 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:22.772+0000 7fc487fff700 1 -- 172.21.15.59:0/2352016901 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc490072510 con 0x7fc47c004850 2023-11-24T07:57:22.778 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:22.772+0000 7fc487fff700 1 -- 172.21.15.59:0/2352016901 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc47c00c090 con 0x7fc47c005210 2023-11-24T07:57:22.778 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:22.772+0000 7fc487fff700 1 -- 172.21.15.59:0/2352016901 --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc47c00c250 con 0x7fc47c003b90 2023-11-24T07:57:22.778 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:22.772+0000 7fc494d9d700 1 --2- 172.21.15.59:0/2352016901 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc47c004850 0x7fc47c004cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:57:22.778 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:22.772+0000 7fc495d9f700 1 --2- 172.21.15.59:0/2352016901 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc47c005210 0x7fc47c00ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:57:22.778 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:22.772+0000 7fc49559e700 1 --2- 172.21.15.59:0/2352016901 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fc47c003b90 0x7fc47c003f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:57:22.778 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:22.772+0000 7fc494d9d700 1 -- 172.21.15.59:0/2352016901 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fc47c003b90 msgr2=0x7fc47c003f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T07:57:22.778 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:22.772+0000 7fc494d9d700 1 --2- 172.21.15.59:0/2352016901 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fc47c003b90 0x7fc47c003f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:22.778 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:22.772+0000 7fc494d9d700 1 -- 172.21.15.59:0/2352016901 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc47c005210 msgr2=0x7fc47c00ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T07:57:22.778 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:22.772+0000 7fc494d9d700 1 --2- 172.21.15.59:0/2352016901 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc47c005210 0x7fc47c00ba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:22.778 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:22.773+0000 7fc494d9d700 1 -- 172.21.15.59:0/2352016901 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc47c0035b0 con 0x7fc47c004850 2023-11-24T07:57:22.779 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:22.773+0000 7fc495d9f700 1 --2- 172.21.15.59:0/2352016901 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc47c005210 0x7fc47c00ba10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-24T07:57:22.779 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:22.773+0000 7fc49559e700 1 --2- 172.21.15.59:0/2352016901 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fc47c003b90 0x7fc47c003f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-24T07:57:22.779 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:22.773+0000 7fc495d9f700 1 -- 172.21.15.59:0/2352016901 reap_dead start 2023-11-24T07:57:22.779 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:22.773+0000 7fc494d9d700 1 --2- 172.21.15.59:0/2352016901 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc47c004850 0x7fc47c004cd0 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7fc48c00c210 tx=0x7fc47c005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=48361e3a111dbb6f server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:57:22.779 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:22.773+0000 7fc487fff700 1 -- 172.21.15.59:0/2352016901 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc48c00cdb0 con 0x7fc47c004850 2023-11-24T07:57:22.779 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:22.773+0000 7fc487fff700 1 -- 172.21.15.59:0/2352016901 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc48c001af0 con 0x7fc47c004850 2023-11-24T07:57:22.779 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:22.773+0000 7fc487fff700 1 -- 172.21.15.59:0/2352016901 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc48c005ba0 con 0x7fc47c004850 2023-11-24T07:57:22.779 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:22.773+0000 7fc497802700 1 -- 172.21.15.59:0/2352016901 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc47c004850 msgr2=0x7fc47c004cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:57:22.779 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:22.773+0000 7fc497802700 1 --2- 172.21.15.59:0/2352016901 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc47c004850 0x7fc47c004cd0 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7fc48c00c210 tx=0x7fc47c005880 comp rx=0 tx=0).stop 2023-11-24T07:57:22.779 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:22.773+0000 7fc497802700 1 -- 172.21.15.59:0/2352016901 shutdown_connections 2023-11-24T07:57:22.779 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:22.773+0000 7fc497802700 1 --2- 172.21.15.59:0/2352016901 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc47c004850 0x7fc47c004cd0 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:22.779 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:22.773+0000 7fc497802700 1 -- 172.21.15.59:0/2352016901 >> 172.21.15.59:0/2352016901 conn(0x7fc4900b2740 msgr2=0x7fc4900b2b20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:57:22.779 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:22.773+0000 7fc497802700 1 -- 172.21.15.59:0/2352016901 shutdown_connections 2023-11-24T07:57:22.780 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:22.773+0000 7fc497802700 1 -- 172.21.15.59:0/2352016901 wait complete. 2023-11-24T07:57:22.780 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:22.775+0000 7fc497802700 1 Processor -- start 2023-11-24T07:57:22.780 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:22.775+0000 7fc497802700 1 -- start start 2023-11-24T07:57:22.782 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:22.775+0000 7fc497802700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc47c004850 0x7fc4901e52f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:57:22.782 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:22.775+0000 7fc497802700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc490134ef0 0x7fc49006ec70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:57:22.782 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:22.775+0000 7fc497802700 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fc490135370 0x7fc49006f1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:57:22.782 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:22.775+0000 7fc497802700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc490135e70 con 0x7fc47c004850 2023-11-24T07:57:22.782 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:22.775+0000 7fc497802700 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4901e5830 con 0x7fc490134ef0 2023-11-24T07:57:22.782 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:22.775+0000 7fc497802700 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc4901358b0 con 0x7fc490135370 2023-11-24T07:57:22.783 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:22.775+0000 7fc49559e700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc47c004850 0x7fc4901e52f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:57:22.783 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:22.776+0000 7fc49559e700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc47c004850 0x7fc4901e52f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:51624/0 (socket says 172.21.15.59:51624) 2023-11-24T07:57:22.783 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:22.776+0000 7fc49559e700 1 -- 172.21.15.59:0/3671754794 learned_addr learned my addr 172.21.15.59:0/3671754794 (peer_addr_for_me v2:172.21.15.59:0/0) 2023-11-24T07:57:22.783 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:22.776+0000 7fc495d9f700 1 --2- 172.21.15.59:0/3671754794 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fc490135370 0x7fc49006f1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:57:22.783 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:22.776+0000 7fc494d9d700 1 --2- 172.21.15.59:0/3671754794 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc490134ef0 0x7fc49006ec70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:57:22.783 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:22.776+0000 7fc49559e700 1 -- 172.21.15.59:0/3671754794 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fc490135370 msgr2=0x7fc49006f1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:57:22.783 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:22.776+0000 7fc49559e700 1 --2- 172.21.15.59:0/3671754794 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fc490135370 0x7fc49006f1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:22.783 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:22.776+0000 7fc49559e700 1 -- 172.21.15.59:0/3671754794 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc490134ef0 msgr2=0x7fc49006ec70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:57:22.783 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:22.776+0000 7fc49559e700 1 --2- 172.21.15.59:0/3671754794 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc490134ef0 0x7fc49006ec70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:22.783 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:22.776+0000 7fc49559e700 1 -- 172.21.15.59:0/3671754794 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc490135b90 con 0x7fc47c004850 2023-11-24T07:57:22.783 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:22.776+0000 7fc49559e700 1 --2- 172.21.15.59:0/3671754794 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc47c004850 0x7fc4901e52f0 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7fc480001eb0 tx=0x7fc48000e8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:57:22.783 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:22.776+0000 7fc485ffb700 1 -- 172.21.15.59:0/3671754794 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4800052e0 con 0x7fc47c004850 2023-11-24T07:57:22.784 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:22.776+0000 7fc485ffb700 1 -- 172.21.15.59:0/3671754794 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc480005440 con 0x7fc47c004850 2023-11-24T07:57:22.784 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:22.776+0000 7fc497802700 1 -- 172.21.15.59:0/3671754794 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc490132f90 con 0x7fc47c004850 2023-11-24T07:57:22.784 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:22.776+0000 7fc485ffb700 1 -- 172.21.15.59:0/3671754794 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4800038c0 con 0x7fc47c004850 2023-11-24T07:57:22.784 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:22.776+0000 7fc497802700 1 -- 172.21.15.59:0/3671754794 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc490144620 con 0x7fc47c004850 2023-11-24T07:57:22.784 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:22.777+0000 7fc497802700 1 -- 172.21.15.59:0/3671754794 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc490144810 con 0x7fc47c004850 2023-11-24T07:57:22.784 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:22.778+0000 7fc485ffb700 1 -- 172.21.15.59:0/3671754794 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fc480003a20 con 0x7fc47c004850 2023-11-24T07:57:22.784 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:22.778+0000 7fc485ffb700 1 --2- 172.21.15.59:0/3671754794 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7fc47406aed0 0x7fc47406d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:57:22.784 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:22.778+0000 7fc485ffb700 1 -- 172.21.15.59:0/3671754794 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 4857+0+0 (secure 0 0 0) 0x7fc4800889c0 con 0x7fc47c004850 2023-11-24T07:57:22.784 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:22.778+0000 7fc494d9d700 1 --2- 172.21.15.59:0/3671754794 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7fc47406aed0 0x7fc47406d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:57:22.784 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:22.779+0000 7fc494d9d700 1 --2- 172.21.15.59:0/3671754794 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7fc47406aed0 0x7fc47406d390 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fc48c0018b0 tx=0x7fc48c00a000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:57:22.785 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:22.782+0000 7fc485ffb700 1 -- 172.21.15.59:0/3671754794 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fc480053f00 con 0x7fc47c004850 2023-11-24T07:57:22.984 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:22.983+0000 7fc497802700 1 -- 172.21.15.59:0/3671754794 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7fc490068670 con 0x7fc47c004850 2023-11-24T07:57:23.572 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:23.571+0000 7fc485ffb700 1 -- 172.21.15.59:0/3671754794 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v30) v1 ==== 138+0+0 (secure 0 0 0) 0x7fc480057520 con 0x7fc47c004850 2023-11-24T07:57:23.576 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:23.575+0000 7fc497802700 1 -- 172.21.15.59:0/3671754794 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7fc47406aed0 msgr2=0x7fc47406d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:57:23.577 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:23.575+0000 7fc497802700 1 --2- 172.21.15.59:0/3671754794 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7fc47406aed0 0x7fc47406d390 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fc48c0018b0 tx=0x7fc48c00a000 comp rx=0 tx=0).stop 2023-11-24T07:57:23.577 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:23.575+0000 7fc497802700 1 -- 172.21.15.59:0/3671754794 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc47c004850 msgr2=0x7fc4901e52f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:57:23.577 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:23.575+0000 7fc497802700 1 --2- 172.21.15.59:0/3671754794 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc47c004850 0x7fc4901e52f0 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7fc480001eb0 tx=0x7fc48000e8e0 comp rx=0 tx=0).stop 2023-11-24T07:57:23.577 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:23.575+0000 7fc497802700 1 -- 172.21.15.59:0/3671754794 shutdown_connections 2023-11-24T07:57:23.577 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:23.575+0000 7fc497802700 1 --2- 172.21.15.59:0/3671754794 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7fc47406aed0 0x7fc47406d390 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:23.577 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:23.575+0000 7fc497802700 1 --2- 172.21.15.59:0/3671754794 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fc490135370 0x7fc49006f1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:23.578 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:23.575+0000 7fc497802700 1 --2- 172.21.15.59:0/3671754794 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc47c004850 0x7fc4901e52f0 secure :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7fc480001eb0 tx=0x7fc48000e8e0 comp rx=0 tx=0).stop 2023-11-24T07:57:23.578 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:23.575+0000 7fc497802700 1 --2- 172.21.15.59:0/3671754794 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc490134ef0 0x7fc49006ec70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:23.578 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:23.575+0000 7fc497802700 1 -- 172.21.15.59:0/3671754794 >> 172.21.15.59:0/3671754794 conn(0x7fc4900b2740 msgr2=0x7fc490146e80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:57:23.578 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:23.576+0000 7fc497802700 1 -- 172.21.15.59:0/3671754794 shutdown_connections 2023-11-24T07:57:23.578 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:23.576+0000 7fc497802700 1 -- 172.21.15.59:0/3671754794 wait complete. 2023-11-24T07:57:23.579 INFO:teuthology.orchestra.run.smithi059.stderr:pool 'cephfs_metadata' created 2023-11-24T07:57:23.593 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2023-11-24T07:57:23.730 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:23.728+0000 7f1478479700 1 Processor -- start 2023-11-24T07:57:23.730 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:23.728+0000 7f1478479700 1 -- start start 2023-11-24T07:57:23.734 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:23.730+0000 7f1478479700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f1470144610 0x7f14700d09f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:57:23.734 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:23.730+0000 7f1478479700 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f147006f700 0x7f14700d0f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:57:23.734 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:23.730+0000 7f1478479700 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f147015e7c0 0x7f147015ec50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:57:23.734 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:23.730+0000 7f1478479700 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f147006e9d0 con 0x7f1470144610 2023-11-24T07:57:23.734 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:23.730+0000 7f1478479700 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f147006f270 con 0x7f147015e7c0 2023-11-24T07:57:23.734 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:23.730+0000 7f1478479700 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f14701449f0 con 0x7f147006f700 2023-11-24T07:57:23.734 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:23.730+0000 7f1476215700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f1470144610 0x7f14700d09f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:57:23.734 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:23.730+0000 7f1476215700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f1470144610 0x7f14700d09f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:51632/0 (socket says 172.21.15.59:51632) 2023-11-24T07:57:23.734 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:23.730+0000 7f1476215700 1 -- 172.21.15.59:0/3355022493 learned_addr learned my addr 172.21.15.59:0/3355022493 (peer_addr_for_me v2:172.21.15.59:0/0) 2023-11-24T07:57:23.735 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:23.730+0000 7f1475a14700 1 --2- 172.21.15.59:0/3355022493 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f147006f700 0x7f14700d0f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:57:23.735 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:23.730+0000 7f1476a16700 1 --2- 172.21.15.59:0/3355022493 >> v2:172.21.15.76:3300/0 conn(0x7f147015e7c0 0x7f147015ec50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:57:23.735 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:23.730+0000 7f1476215700 1 -- 172.21.15.59:0/3355022493 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f147006f700 msgr2=0x7f14700d0f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T07:57:23.735 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:23.730+0000 7f1476215700 1 --2- 172.21.15.59:0/3355022493 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f147006f700 0x7f14700d0f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:23.735 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:23.730+0000 7f1476215700 1 -- 172.21.15.59:0/3355022493 >> v2:172.21.15.76:3300/0 conn(0x7f147015e7c0 msgr2=0x7f147015ec50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T07:57:23.735 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:23.730+0000 7f1476215700 1 --2- 172.21.15.59:0/3355022493 >> v2:172.21.15.76:3300/0 conn(0x7f147015e7c0 0x7f147015ec50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:23.735 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:23.730+0000 7f1476215700 1 -- 172.21.15.59:0/3355022493 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f147015f530 con 0x7f1470144610 2023-11-24T07:57:23.735 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:23.730+0000 7f1475a14700 1 --2- 172.21.15.59:0/3355022493 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f147006f700 0x7f14700d0f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T07:57:23.735 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:23.730+0000 7f1476a16700 1 --2- 172.21.15.59:0/3355022493 >> v2:172.21.15.76:3300/0 conn(0x7f147015e7c0 0x7f147015ec50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T07:57:23.735 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:23.730+0000 7f1476215700 1 --2- 172.21.15.59:0/3355022493 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1470144610 0x7f14700d09f0 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f146c0098f0 tx=0x7f146c009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2f311ff3c78dcc3d server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:57:23.735 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:23.731+0000 7f1475213700 1 -- 172.21.15.59:0/3355022493 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f146c00b860 con 0x7f1470144610 2023-11-24T07:57:23.735 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:23.731+0000 7f1475213700 1 -- 172.21.15.59:0/3355022493 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f146c00b9c0 con 0x7f1470144610 2023-11-24T07:57:23.736 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:23.731+0000 7f1475213700 1 -- 172.21.15.59:0/3355022493 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f146c00bd50 con 0x7f1470144610 2023-11-24T07:57:23.736 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:23.731+0000 7f1478479700 1 -- 172.21.15.59:0/3355022493 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1470144610 msgr2=0x7f14700d09f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:57:23.736 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:23.731+0000 7f1478479700 1 --2- 172.21.15.59:0/3355022493 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1470144610 0x7f14700d09f0 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f146c0098f0 tx=0x7f146c009c00 comp rx=0 tx=0).stop 2023-11-24T07:57:23.736 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:23.731+0000 7f1478479700 1 -- 172.21.15.59:0/3355022493 shutdown_connections 2023-11-24T07:57:23.736 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:23.731+0000 7f1478479700 1 --2- 172.21.15.59:0/3355022493 >> v2:172.21.15.76:3300/0 conn(0x7f147015e7c0 0x7f147015ec50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:23.736 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:23.731+0000 7f1478479700 1 --2- 172.21.15.59:0/3355022493 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1470144610 0x7f14700d09f0 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:23.736 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:23.731+0000 7f1478479700 1 --2- 172.21.15.59:0/3355022493 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f147006f700 0x7f14700d0f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:23.736 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:23.731+0000 7f1478479700 1 -- 172.21.15.59:0/3355022493 >> 172.21.15.59:0/3355022493 conn(0x7f14700b2740 msgr2=0x7f14700b2b50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:57:23.736 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:23.731+0000 7f1478479700 1 -- 172.21.15.59:0/3355022493 shutdown_connections 2023-11-24T07:57:23.736 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:23.731+0000 7f1478479700 1 -- 172.21.15.59:0/3355022493 wait complete. 2023-11-24T07:57:23.736 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:23.732+0000 7f1478479700 1 Processor -- start 2023-11-24T07:57:23.736 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:23.733+0000 7f1478479700 1 -- start start 2023-11-24T07:57:23.739 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:23.733+0000 7f1478479700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f147006f700 0x7f14701f1650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:57:23.739 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:23.733+0000 7f1478479700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1470144610 0x7f147015df60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:57:23.739 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:23.733+0000 7f1478479700 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f147015e7c0 0x7f147006b240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:57:23.739 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:23.733+0000 7f1478479700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14701f1c60 con 0x7f1470144610 2023-11-24T07:57:23.739 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:23.733+0000 7f1478479700 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14701f1f40 con 0x7f147006f700 2023-11-24T07:57:23.739 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:23.733+0000 7f1478479700 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f14701f2220 con 0x7f147015e7c0 2023-11-24T07:57:23.739 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:23.733+0000 7f1475a14700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1470144610 0x7f147015df60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:57:23.739 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:23.733+0000 7f1475a14700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1470144610 0x7f147015df60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:51640/0 (socket says 172.21.15.59:51640) 2023-11-24T07:57:23.739 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:23.733+0000 7f1475a14700 1 -- 172.21.15.59:0/3185126416 learned_addr learned my addr 172.21.15.59:0/3185126416 (peer_addr_for_me v2:172.21.15.59:0/0) 2023-11-24T07:57:23.739 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:23.734+0000 7f1476215700 1 --2- 172.21.15.59:0/3185126416 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f147006f700 0x7f14701f1650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:57:23.740 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:23.734+0000 7f1476a16700 1 --2- 172.21.15.59:0/3185126416 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f147015e7c0 0x7f147006b240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:57:23.740 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:23.734+0000 7f1475a14700 1 -- 172.21.15.59:0/3185126416 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f147015e7c0 msgr2=0x7f147006b240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:57:23.740 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:23.734+0000 7f1475a14700 1 --2- 172.21.15.59:0/3185126416 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f147015e7c0 0x7f147006b240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:23.740 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:23.734+0000 7f1475a14700 1 -- 172.21.15.59:0/3185126416 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f147006f700 msgr2=0x7f14701f1650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:57:23.740 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:23.734+0000 7f1475a14700 1 --2- 172.21.15.59:0/3185126416 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f147006f700 0x7f14701f1650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:23.740 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:23.734+0000 7f1475a14700 1 -- 172.21.15.59:0/3185126416 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f146c009550 con 0x7f1470144610 2023-11-24T07:57:23.740 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:23.734+0000 7f1476215700 1 --2- 172.21.15.59:0/3185126416 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f147006f700 0x7f14701f1650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T07:57:23.740 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:23.734+0000 7f1476a16700 1 --2- 172.21.15.59:0/3185126416 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f147015e7c0 0x7f147006b240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T07:57:23.740 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:23.734+0000 7f1475a14700 1 --2- 172.21.15.59:0/3185126416 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1470144610 0x7f147015df60 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f146000c740 tx=0x7f146000cb00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:57:23.740 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:23.734+0000 7f1466ffd700 1 -- 172.21.15.59:0/3185126416 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1460011800 con 0x7f1470144610 2023-11-24T07:57:23.740 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:23.734+0000 7f1466ffd700 1 -- 172.21.15.59:0/3185126416 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1460011960 con 0x7f1470144610 2023-11-24T07:57:23.740 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:23.734+0000 7f1466ffd700 1 -- 172.21.15.59:0/3185126416 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1460011cd0 con 0x7f1470144610 2023-11-24T07:57:23.741 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:23.734+0000 7f1478479700 1 -- 172.21.15.59:0/3185126416 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f14701f2500 con 0x7f1470144610 2023-11-24T07:57:23.741 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:23.734+0000 7f1478479700 1 -- 172.21.15.59:0/3185126416 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f147015e4a0 con 0x7f1470144610 2023-11-24T07:57:23.741 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:23.735+0000 7f1478479700 1 -- 172.21.15.59:0/3185126416 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1470075480 con 0x7f1470144610 2023-11-24T07:57:23.741 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:23.736+0000 7f1466ffd700 1 -- 172.21.15.59:0/3185126416 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f1460021350 con 0x7f1470144610 2023-11-24T07:57:23.741 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:23.736+0000 7f1466ffd700 1 --2- 172.21.15.59:0/3185126416 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f145406aed0 0x7f145406d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:57:23.741 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:23.736+0000 7f1466ffd700 1 -- 172.21.15.59:0/3185126416 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 5227+0+0 (secure 0 0 0) 0x7f1460089370 con 0x7f1470144610 2023-11-24T07:57:23.741 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:23.736+0000 7f1476215700 1 --2- 172.21.15.59:0/3185126416 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f145406aed0 0x7f145406d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:57:23.741 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:23.737+0000 7f1476215700 1 --2- 172.21.15.59:0/3185126416 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f145406aed0 0x7f145406d390 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f14700cd910 tx=0x7f146c008030 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:57:23.742 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:23.741+0000 7f1466ffd700 1 -- 172.21.15.59:0/3185126416 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f1460054610 con 0x7f1470144610 2023-11-24T07:57:23.942 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:23.940+0000 7f1478479700 1 -- 172.21.15.59:0/3185126416 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7f14701f1da0 con 0x7f1470144610 2023-11-24T07:57:24.578 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:24.576+0000 7f1466ffd700 1 -- 172.21.15.59:0/3185126416 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v31) v1 ==== 170+0+0 (secure 0 0 0) 0x7f14701f1da0 con 0x7f1470144610 2023-11-24T07:57:24.583 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:24.581+0000 7f1478479700 1 -- 172.21.15.59:0/3185126416 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f145406aed0 msgr2=0x7f145406d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:57:24.583 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:24.581+0000 7f1478479700 1 --2- 172.21.15.59:0/3185126416 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f145406aed0 0x7f145406d390 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f14700cd910 tx=0x7f146c008030 comp rx=0 tx=0).stop 2023-11-24T07:57:24.583 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:24.581+0000 7f1478479700 1 -- 172.21.15.59:0/3185126416 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1470144610 msgr2=0x7f147015df60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:57:24.583 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:24.581+0000 7f1478479700 1 --2- 172.21.15.59:0/3185126416 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1470144610 0x7f147015df60 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f146000c740 tx=0x7f146000cb00 comp rx=0 tx=0).stop 2023-11-24T07:57:24.584 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:24.582+0000 7f1478479700 1 -- 172.21.15.59:0/3185126416 shutdown_connections 2023-11-24T07:57:24.584 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:24.582+0000 7f1478479700 1 --2- 172.21.15.59:0/3185126416 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f145406aed0 0x7f145406d390 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:24.584 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:24.582+0000 7f1478479700 1 --2- 172.21.15.59:0/3185126416 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f147015e7c0 0x7f147006b240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:24.584 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:24.582+0000 7f1478479700 1 --2- 172.21.15.59:0/3185126416 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f147006f700 0x7f14701f1650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:24.585 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:24.582+0000 7f1478479700 1 --2- 172.21.15.59:0/3185126416 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1470144610 0x7f147015df60 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:24.585 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:24.582+0000 7f1478479700 1 -- 172.21.15.59:0/3185126416 >> 172.21.15.59:0/3185126416 conn(0x7f14700b2740 msgr2=0x7f14700b4a90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:57:24.586 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:24.584+0000 7f1478479700 1 -- 172.21.15.59:0/3185126416 shutdown_connections 2023-11-24T07:57:24.586 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:24.585+0000 7f1478479700 1 -- 172.21.15.59:0/3185126416 wait complete. 2023-11-24T07:57:24.594 INFO:teuthology.orchestra.run.smithi059.stderr:pool 'cephfs_data' created 2023-11-24T07:57:24.611 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2023-11-24T07:57:24.743 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:24.742+0000 7fa05747b700 1 Processor -- start 2023-11-24T07:57:24.744 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:24.742+0000 7fa05747b700 1 -- start start 2023-11-24T07:57:24.746 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:24.743+0000 7fa05747b700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fa0500722c0 0x7fa0500b51b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:57:24.746 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:24.743+0000 7fa05747b700 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fa0501440f0 0x7fa05006e510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:57:24.746 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:24.743+0000 7fa05747b700 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa050143ec0 con 0x7fa050143ae0 2023-11-24T07:57:24.746 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:24.743+0000 7fa05747b700 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa0500b40a0 con 0x7fa0500722c0 2023-11-24T07:57:24.746 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:24.743+0000 7fa05747b700 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa0500b3ae0 con 0x7fa0501440f0 2023-11-24T07:57:24.746 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:24.743+0000 7fa054a16700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fa0500722c0 0x7fa0500b51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:57:24.746 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:24.743+0000 7fa054a16700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fa0500722c0 0x7fa0500b51b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:51648/0 (socket says 172.21.15.59:51648) 2023-11-24T07:57:24.746 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:24.743+0000 7fa054a16700 1 -- 172.21.15.59:0/3871240358 learned_addr learned my addr 172.21.15.59:0/3871240358 (peer_addr_for_me v2:172.21.15.59:0/0) 2023-11-24T07:57:24.746 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:24.743+0000 7fa055a18700 1 --2- 172.21.15.59:0/3871240358 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fa0501440f0 0x7fa05006e510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:57:24.747 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:24.743+0000 7fa054a16700 1 -- 172.21.15.59:0/3871240358 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fa0501440f0 msgr2=0x7fa05006e510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T07:57:24.747 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:24.743+0000 7fa054a16700 1 --2- 172.21.15.59:0/3871240358 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fa0501440f0 0x7fa05006e510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:24.747 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:24.743+0000 7fa054a16700 1 -- 172.21.15.59:0/3871240358 >> v1:172.21.15.76:6789/0 conn(0x7fa050143ae0 legacy=0x7fa050071bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T07:57:24.747 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:24.743+0000 7fa054a16700 1 -- 172.21.15.59:0/3871240358 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa0500b3dc0 con 0x7fa0500722c0 2023-11-24T07:57:24.747 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:24.743+0000 7fa054a16700 1 --2- 172.21.15.59:0/3871240358 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa0500722c0 0x7fa0500b51b0 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7fa04000bb90 tx=0x7fa04000bea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7d0d92e21818257e server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:57:24.747 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:24.743+0000 7fa047fff700 1 -- 172.21.15.59:0/3871240358 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa04000dd10 con 0x7fa0500722c0 2023-11-24T07:57:24.747 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:24.743+0000 7fa047fff700 1 -- 172.21.15.59:0/3871240358 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa04000de70 con 0x7fa0500722c0 2023-11-24T07:57:24.747 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:24.743+0000 7fa047fff700 1 -- 172.21.15.59:0/3871240358 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa040004230 con 0x7fa0500722c0 2023-11-24T07:57:24.747 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:24.744+0000 7fa05747b700 1 -- 172.21.15.59:0/3871240358 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa0500722c0 msgr2=0x7fa0500b51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:57:24.747 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:24.744+0000 7fa05747b700 1 --2- 172.21.15.59:0/3871240358 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa0500722c0 0x7fa0500b51b0 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7fa04000bb90 tx=0x7fa04000bea0 comp rx=0 tx=0).stop 2023-11-24T07:57:24.747 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:24.744+0000 7fa05747b700 1 -- 172.21.15.59:0/3871240358 shutdown_connections 2023-11-24T07:57:24.747 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:24.744+0000 7fa05747b700 1 --2- 172.21.15.59:0/3871240358 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fa0501440f0 0x7fa05006e510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:24.748 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:24.744+0000 7fa05747b700 1 --2- 172.21.15.59:0/3871240358 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa0500722c0 0x7fa0500b51b0 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:24.748 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:24.744+0000 7fa05747b700 1 -- 172.21.15.59:0/3871240358 >> 172.21.15.59:0/3871240358 conn(0x7fa0500b2740 msgr2=0x7fa0500b2b20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:57:24.748 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:24.744+0000 7fa05747b700 1 -- 172.21.15.59:0/3871240358 shutdown_connections 2023-11-24T07:57:24.748 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:24.744+0000 7fa05747b700 1 -- 172.21.15.59:0/3871240358 wait complete. 2023-11-24T07:57:24.750 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:24.745+0000 7fa05747b700 1 Processor -- start 2023-11-24T07:57:24.750 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:24.745+0000 7fa05747b700 1 -- start start 2023-11-24T07:57:24.750 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:24.746+0000 7fa05747b700 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fa0500722c0 0x7fa05015dd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:57:24.750 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:24.746+0000 7fa05747b700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa050143ae0 0x7fa0501557f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:57:24.750 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:24.746+0000 7fa05747b700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa0501440f0 0x7fa0500cdf90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:57:24.750 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:24.746+0000 7fa05747b700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa050140c50 con 0x7fa050143ae0 2023-11-24T07:57:24.750 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:24.746+0000 7fa05747b700 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa05015e330 con 0x7fa0501440f0 2023-11-24T07:57:24.751 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:24.746+0000 7fa05747b700 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa05012f430 con 0x7fa0500722c0 2023-11-24T07:57:24.751 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:24.746+0000 7fa054a16700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa050143ae0 0x7fa0501557f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:57:24.751 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:24.746+0000 7fa054a16700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa050143ae0 0x7fa0501557f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:51654/0 (socket says 172.21.15.59:51654) 2023-11-24T07:57:24.751 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:24.746+0000 7fa054a16700 1 -- 172.21.15.59:0/856092421 learned_addr learned my addr 172.21.15.59:0/856092421 (peer_addr_for_me v2:172.21.15.59:0/0) 2023-11-24T07:57:24.751 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:24.746+0000 7fa055217700 1 --2- 172.21.15.59:0/856092421 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fa0500722c0 0x7fa05015dd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:57:24.751 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:24.746+0000 7fa055a18700 1 --2- 172.21.15.59:0/856092421 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa0501440f0 0x7fa0500cdf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:57:24.751 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:24.747+0000 7fa054a16700 1 -- 172.21.15.59:0/856092421 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fa0500722c0 msgr2=0x7fa05015dd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:57:24.751 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:24.747+0000 7fa054a16700 1 --2- 172.21.15.59:0/856092421 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fa0500722c0 0x7fa05015dd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:24.751 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:24.747+0000 7fa054a16700 1 -- 172.21.15.59:0/856092421 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa0501440f0 msgr2=0x7fa0500cdf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:57:24.751 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:24.747+0000 7fa054a16700 1 --2- 172.21.15.59:0/856092421 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa0501440f0 0x7fa0500cdf90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:24.751 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:24.747+0000 7fa054a16700 1 -- 172.21.15.59:0/856092421 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa04000b7f0 con 0x7fa050143ae0 2023-11-24T07:57:24.751 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:24.747+0000 7fa055217700 1 --2- 172.21.15.59:0/856092421 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fa0500722c0 0x7fa05015dd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T07:57:24.752 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:24.747+0000 7fa055a18700 1 --2- 172.21.15.59:0/856092421 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa0501440f0 0x7fa0500cdf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T07:57:24.752 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:24.747+0000 7fa054a16700 1 --2- 172.21.15.59:0/856092421 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa050143ae0 0x7fa0501557f0 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7fa04000b340 tx=0x7fa040004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:57:24.752 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:24.747+0000 7fa045ffb700 1 -- 172.21.15.59:0/856092421 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa040020070 con 0x7fa050143ae0 2023-11-24T07:57:24.752 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:24.747+0000 7fa045ffb700 1 -- 172.21.15.59:0/856092421 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa040003bd0 con 0x7fa050143ae0 2023-11-24T07:57:24.752 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:24.747+0000 7fa05747b700 1 -- 172.21.15.59:0/856092421 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa05012f710 con 0x7fa050143ae0 2023-11-24T07:57:24.752 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:24.747+0000 7fa045ffb700 1 -- 172.21.15.59:0/856092421 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa040010470 con 0x7fa050143ae0 2023-11-24T07:57:24.752 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:24.747+0000 7fa05747b700 1 -- 172.21.15.59:0/856092421 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa05012fcd0 con 0x7fa050143ae0 2023-11-24T07:57:24.752 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:24.748+0000 7fa05747b700 1 -- 172.21.15.59:0/856092421 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa05012fe60 con 0x7fa050143ae0 2023-11-24T07:57:24.752 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:24.749+0000 7fa045ffb700 1 -- 172.21.15.59:0/856092421 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fa0400193f0 con 0x7fa050143ae0 2023-11-24T07:57:24.753 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:24.749+0000 7fa045ffb700 1 --2- 172.21.15.59:0/856092421 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7fa03406aed0 0x7fa03406d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:57:24.753 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:24.749+0000 7fa045ffb700 1 -- 172.21.15.59:0/856092421 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 5609+0+0 (secure 0 0 0) 0x7fa04008cb80 con 0x7fa050143ae0 2023-11-24T07:57:24.753 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:24.749+0000 7fa055217700 1 --2- 172.21.15.59:0/856092421 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7fa03406aed0 0x7fa03406d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:57:24.753 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:24.750+0000 7fa055217700 1 --2- 172.21.15.59:0/856092421 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7fa03406aed0 0x7fa03406d390 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fa04c003f40 tx=0x7fa04c009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:57:24.755 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:24.754+0000 7fa045ffb700 1 -- 172.21.15.59:0/856092421 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fa040057d20 con 0x7fa050143ae0 2023-11-24T07:57:25.003 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:25.001+0000 7fa05747b700 1 -- 172.21.15.59:0/856092421 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7fa050068180 con 0x7fa050143ae0 2023-11-24T07:57:25.579 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:25.578+0000 7fa045ffb700 1 -- 172.21.15.59:0/856092421 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 4 and data pool 5 v13) v1 ==== 172+0+0 (secure 0 0 0) 0x7fa04005b340 con 0x7fa050143ae0 2023-11-24T07:57:25.583 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:25.581+0000 7fa05747b700 1 -- 172.21.15.59:0/856092421 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7fa03406aed0 msgr2=0x7fa03406d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:57:25.583 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:25.581+0000 7fa05747b700 1 --2- 172.21.15.59:0/856092421 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7fa03406aed0 0x7fa03406d390 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fa04c003f40 tx=0x7fa04c009040 comp rx=0 tx=0).stop 2023-11-24T07:57:25.583 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:25.581+0000 7fa05747b700 1 -- 172.21.15.59:0/856092421 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa050143ae0 msgr2=0x7fa0501557f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:57:25.583 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:25.581+0000 7fa05747b700 1 --2- 172.21.15.59:0/856092421 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa050143ae0 0x7fa0501557f0 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7fa04000b340 tx=0x7fa040004750 comp rx=0 tx=0).stop 2023-11-24T07:57:25.583 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:25.581+0000 7fa05747b700 1 -- 172.21.15.59:0/856092421 shutdown_connections 2023-11-24T07:57:25.583 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:25.581+0000 7fa05747b700 1 --2- 172.21.15.59:0/856092421 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7fa03406aed0 0x7fa03406d390 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:25.583 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:25.581+0000 7fa05747b700 1 --2- 172.21.15.59:0/856092421 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa0501440f0 0x7fa0500cdf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:25.583 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:25.581+0000 7fa05747b700 1 --2- 172.21.15.59:0/856092421 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fa0500722c0 0x7fa05015dd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:25.583 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:25.581+0000 7fa05747b700 1 --2- 172.21.15.59:0/856092421 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa050143ae0 0x7fa0501557f0 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:25.583 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:25.581+0000 7fa05747b700 1 -- 172.21.15.59:0/856092421 >> 172.21.15.59:0/856092421 conn(0x7fa0500b2740 msgr2=0x7fa0500b41e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:57:25.584 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:25.582+0000 7fa05747b700 1 -- 172.21.15.59:0/856092421 shutdown_connections 2023-11-24T07:57:25.584 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:25.582+0000 7fa05747b700 1 -- 172.21.15.59:0/856092421 wait complete. 2023-11-24T07:57:25.588 INFO:teuthology.orchestra.run.smithi059.stderr:new fs with metadata pool 4 and data pool 5 2023-11-24T07:57:25.615 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-24T07:57:25.744 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:25.742+0000 7f2e3e8c5700 1 Processor -- start 2023-11-24T07:57:25.744 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:25.742+0000 7f2e3e8c5700 1 -- start start 2023-11-24T07:57:25.746 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:25.743+0000 7f2e3e8c5700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f2e381459a0 0x7f2e380cfa80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:57:25.747 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:25.743+0000 7f2e3e8c5700 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f2e3806b6e0 0x7f2e380cffc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:57:25.747 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:25.743+0000 7f2e3e8c5700 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2e380cf5f0 con 0x7f2e38142810 2023-11-24T07:57:25.747 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:25.743+0000 7f2e3e8c5700 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2e3806ea80 con 0x7f2e381459a0 2023-11-24T07:57:25.747 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:25.743+0000 7f2e3e8c5700 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2e3806ed60 con 0x7f2e3806b6e0 2023-11-24T07:57:25.747 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:25.743+0000 7f2e377fe700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f2e381459a0 0x7f2e380cfa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:57:25.747 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:25.743+0000 7f2e377fe700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f2e381459a0 0x7f2e380cfa80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:51674/0 (socket says 172.21.15.59:51674) 2023-11-24T07:57:25.747 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:25.743+0000 7f2e37fff700 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7f2e38142810 0x7f2e3806afd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:55146/0 (socket says 172.21.15.59:55146) 2023-11-24T07:57:25.747 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:25.743+0000 7f2e377fe700 1 -- 172.21.15.59:0/3622162388 learned_addr learned my addr 172.21.15.59:0/3622162388 (peer_addr_for_me v2:172.21.15.59:0/0) 2023-11-24T07:57:25.748 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:25.744+0000 7f2e3ce62700 1 --2- 172.21.15.59:0/3622162388 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f2e3806b6e0 0x7f2e380cffc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:57:25.748 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:25.744+0000 7f2e377fe700 1 -- 172.21.15.59:0/3622162388 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f2e3806b6e0 msgr2=0x7f2e380cffc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T07:57:25.748 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:25.744+0000 7f2e377fe700 1 --2- 172.21.15.59:0/3622162388 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f2e3806b6e0 0x7f2e380cffc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:25.748 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:25.744+0000 7f2e377fe700 1 -- 172.21.15.59:0/3622162388 >> v1:172.21.15.59:6789/0 conn(0x7f2e38142810 legacy=0x7f2e3806afd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:57:25.748 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:25.744+0000 7f2e377fe700 1 -- 172.21.15.59:0/3622162388 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2e3806f040 con 0x7f2e381459a0 2023-11-24T07:57:25.748 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:25.744+0000 7f2e36ffd700 1 -- 172.21.15.59:0/3622162388 <== mon.0 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1206350170 0 0) 0x7f2e380cf5f0 con 0x7f2e38142810 2023-11-24T07:57:25.748 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:25.744+0000 7f2e377fe700 1 --2- 172.21.15.59:0/3622162388 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2e381459a0 0x7f2e380cfa80 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f2e200098f0 tx=0x7f2e20009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2094fc73284076f6 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:57:25.748 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:25.744+0000 7f2e36ffd700 1 -- 172.21.15.59:0/3622162388 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e2000b8a0 con 0x7f2e381459a0 2023-11-24T07:57:25.748 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:25.744+0000 7f2e36ffd700 1 -- 172.21.15.59:0/3622162388 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2e2000ba00 con 0x7f2e381459a0 2023-11-24T07:57:25.748 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:25.744+0000 7f2e36ffd700 1 -- 172.21.15.59:0/3622162388 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e2000bd90 con 0x7f2e381459a0 2023-11-24T07:57:25.748 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:25.745+0000 7f2e3e8c5700 1 -- 172.21.15.59:0/3622162388 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2e381459a0 msgr2=0x7f2e380cfa80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:57:25.748 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:25.745+0000 7f2e3e8c5700 1 --2- 172.21.15.59:0/3622162388 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2e381459a0 0x7f2e380cfa80 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f2e200098f0 tx=0x7f2e20009c00 comp rx=0 tx=0).stop 2023-11-24T07:57:25.748 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:25.745+0000 7f2e3e8c5700 1 -- 172.21.15.59:0/3622162388 shutdown_connections 2023-11-24T07:57:25.749 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:25.745+0000 7f2e3e8c5700 1 --2- 172.21.15.59:0/3622162388 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f2e3806b6e0 0x7f2e380cffc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:25.749 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:25.745+0000 7f2e3e8c5700 1 --2- 172.21.15.59:0/3622162388 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2e381459a0 0x7f2e380cfa80 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:25.749 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:25.745+0000 7f2e3e8c5700 1 -- 172.21.15.59:0/3622162388 >> 172.21.15.59:0/3622162388 conn(0x7f2e380b2740 msgr2=0x7f2e380b2b20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:57:25.749 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:25.745+0000 7f2e3e8c5700 1 -- 172.21.15.59:0/3622162388 shutdown_connections 2023-11-24T07:57:25.749 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:25.745+0000 7f2e3e8c5700 1 -- 172.21.15.59:0/3622162388 wait complete. 2023-11-24T07:57:25.751 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:25.746+0000 7f2e3e8c5700 1 Processor -- start 2023-11-24T07:57:25.751 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:25.746+0000 7f2e3e8c5700 1 -- start start 2023-11-24T07:57:25.751 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:25.747+0000 7f2e3e8c5700 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f2e3806b6e0 0x7f2e381f22f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:57:25.752 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:25.747+0000 7f2e3e8c5700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2e38142810 0x7f2e381f1030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:57:25.752 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:25.747+0000 7f2e3e8c5700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2e381459a0 0x7f2e38153ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:57:25.752 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:25.747+0000 7f2e3e8c5700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e38153240 con 0x7f2e381459a0 2023-11-24T07:57:25.752 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:25.747+0000 7f2e3e8c5700 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e381f2830 con 0x7f2e38142810 2023-11-24T07:57:25.752 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:25.747+0000 7f2e3e8c5700 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2e38152630 con 0x7f2e3806b6e0 2023-11-24T07:57:25.752 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:25.747+0000 7f2e3ce62700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2e381459a0 0x7f2e38153ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:57:25.752 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:25.747+0000 7f2e3ce62700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2e381459a0 0x7f2e38153ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:51680/0 (socket says 172.21.15.59:51680) 2023-11-24T07:57:25.752 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:25.747+0000 7f2e3ce62700 1 -- 172.21.15.59:0/4162325179 learned_addr learned my addr 172.21.15.59:0/4162325179 (peer_addr_for_me v2:172.21.15.59:0/0) 2023-11-24T07:57:25.752 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:25.747+0000 7f2e37fff700 1 --2- 172.21.15.59:0/4162325179 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f2e3806b6e0 0x7f2e381f22f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:57:25.752 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:25.747+0000 7f2e377fe700 1 --2- 172.21.15.59:0/4162325179 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2e38142810 0x7f2e381f1030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:57:25.752 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:25.747+0000 7f2e3ce62700 1 -- 172.21.15.59:0/4162325179 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f2e3806b6e0 msgr2=0x7f2e381f22f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:57:25.753 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:25.747+0000 7f2e3ce62700 1 --2- 172.21.15.59:0/4162325179 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f2e3806b6e0 0x7f2e381f22f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:25.753 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:25.747+0000 7f2e3ce62700 1 -- 172.21.15.59:0/4162325179 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2e38142810 msgr2=0x7f2e381f1030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:57:25.753 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:25.747+0000 7f2e3ce62700 1 --2- 172.21.15.59:0/4162325179 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2e38142810 0x7f2e381f1030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:25.753 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:25.747+0000 7f2e3ce62700 1 -- 172.21.15.59:0/4162325179 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2e20009550 con 0x7f2e381459a0 2023-11-24T07:57:25.753 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:25.747+0000 7f2e37fff700 1 --2- 172.21.15.59:0/4162325179 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f2e3806b6e0 0x7f2e381f22f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-24T07:57:25.753 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:25.747+0000 7f2e3ce62700 1 --2- 172.21.15.59:0/4162325179 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2e381459a0 0x7f2e38153ca0 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f2e30007e30 tx=0x7f2e3000e3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:57:25.753 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:25.748+0000 7f2e34ff9700 1 -- 172.21.15.59:0/4162325179 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e3000ede0 con 0x7f2e381459a0 2023-11-24T07:57:25.753 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:25.748+0000 7f2e34ff9700 1 -- 172.21.15.59:0/4162325179 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2e30014420 con 0x7f2e381459a0 2023-11-24T07:57:25.753 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:25.748+0000 7f2e3e8c5700 1 -- 172.21.15.59:0/4162325179 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2e38152910 con 0x7f2e381459a0 2023-11-24T07:57:25.753 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:25.748+0000 7f2e34ff9700 1 -- 172.21.15.59:0/4162325179 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e300147d0 con 0x7f2e381459a0 2023-11-24T07:57:25.753 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:25.748+0000 7f2e3e8c5700 1 -- 172.21.15.59:0/4162325179 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2e38152ed0 con 0x7f2e381459a0 2023-11-24T07:57:25.754 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:25.749+0000 7f2e3e8c5700 1 -- 172.21.15.59:0/4162325179 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2e38068670 con 0x7f2e381459a0 2023-11-24T07:57:25.754 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:25.749+0000 7f2e34ff9700 1 -- 172.21.15.59:0/4162325179 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f2e30014930 con 0x7f2e381459a0 2023-11-24T07:57:25.754 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:25.750+0000 7f2e34ff9700 1 --2- 172.21.15.59:0/4162325179 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f2e1806aed0 0x7f2e1806d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:57:25.754 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:25.750+0000 7f2e34ff9700 1 -- 172.21.15.59:0/4162325179 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(32..32 src has 1..32) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f2e30089da0 con 0x7f2e381459a0 2023-11-24T07:57:25.754 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:25.750+0000 7f2e37fff700 1 --2- 172.21.15.59:0/4162325179 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f2e1806aed0 0x7f2e1806d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:57:25.754 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:25.750+0000 7f2e37fff700 1 --2- 172.21.15.59:0/4162325179 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f2e1806aed0 0x7f2e1806d390 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f2e28000f60 tx=0x7f2e28007cc0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:57:25.756 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:25.754+0000 7f2e34ff9700 1 -- 172.21.15.59:0/4162325179 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f2e30054e40 con 0x7f2e381459a0 2023-11-24T07:57:25.956 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:25.954+0000 7f2e3e8c5700 1 -- 172.21.15.59:0/4162325179 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f2e380b1e10 con 0x7f2e381459a0 2023-11-24T07:57:25.957 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:25.955+0000 7f2e34ff9700 1 -- 172.21.15.59:0/4162325179 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v32) v1 ==== 74+0+16450 (secure 0 0 0) 0x7f2e30058460 con 0x7f2e381459a0 2023-11-24T07:57:25.957 INFO:teuthology.orchestra.run.smithi059.stdout: 2023-11-24T07:57:25.958 INFO:teuthology.orchestra.run.smithi059.stdout:{"epoch":32,"fsid":"9f0e8741-b8f9-48d5-a2c3-09f45903303d","created":"2023-11-24T07:56:06.552729+0000","modified":"2023-11-24T07:57:25.003306+0000","last_up_change":"2023-11-24T07:56:17.372352+0000","last_in_change":"2023-11-24T07:56:09.073250+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-24T07:56:18.352823+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2023-11-24T07:57:23.014143+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"32","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2023-11-24T07:57:23.971487+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"32","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"6d8f9e9b-1782-4e41-928b-a39f2a8ebc06","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6800","nonce":3373878772},{"type":"v1","addr":"172.21.15.59:6801","nonce":3373878772}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6802","nonce":3373878772},{"type":"v1","addr":"172.21.15.59:6803","nonce":3373878772}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6806","nonce":3373878772},{"type":"v1","addr":"172.21.15.59:6807","nonce":3373878772}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6804","nonce":3373878772},{"type":"v1","addr":"172.21.15.59:6805","nonce":3373878772}]},"public_addr":"172.21.15.59:6801/3373878772","cluster_addr":"172.21.15.59:6803/3373878772","heartbeat_back_addr":"172.21.15.59:6807/3373878772","heartbeat_front_addr":"172.21.15.59:6805/3373878772","state":["exists","up"]},{"osd":1,"uuid":"e8d882aa-52e1-4c04-9e1e-c44ef9204a5f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6824","nonce":648258563},{"type":"v1","addr":"172.21.15.59:6825","nonce":648258563}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6826","nonce":648258563},{"type":"v1","addr":"172.21.15.59:6827","nonce":648258563}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6830","nonce":648258563},{"type":"v1","addr":"172.21.15.59:6831","nonce":648258563}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6828","nonce":648258563},{"type":"v1","addr":"172.21.15.59:6829","nonce":648258563}]},"public_addr":"172.21.15.59:6825/648258563","cluster_addr":"172.21.15.59:6827/648258563","heartbeat_back_addr":"172.21.15.59:6831/648258563","heartbeat_front_addr":"172.21.15.59:6829/648258563","state":["exists","up"]},{"osd":2,"uuid":"69eebfa0-695b-4b21-90b9-8a3537168a01","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6808","nonce":1840430654},{"type":"v1","addr":"172.21.15.59:6809","nonce":1840430654}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6810","nonce":1840430654},{"type":"v1","addr":"172.21.15.59:6811","nonce":1840430654}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6814","nonce":1840430654},{"type":"v1","addr":"172.21.15.59:6815","nonce":1840430654}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6812","nonce":1840430654},{"type":"v1","addr":"172.21.15.59:6813","nonce":1840430654}]},"public_addr":"172.21.15.59:6809/1840430654","cluster_addr":"172.21.15.59:6811/1840430654","heartbeat_back_addr":"172.21.15.59:6815/1840430654","heartbeat_front_addr":"172.21.15.59:6813/1840430654","state":["exists","up"]},{"osd":3,"uuid":"96fd77fc-f911-46ff-9445-6f0be2daa8f4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6816","nonce":2504025850},{"type":"v1","addr":"172.21.15.59:6817","nonce":2504025850}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6818","nonce":2504025850},{"type":"v1","addr":"172.21.15.59:6819","nonce":2504025850}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6822","nonce":2504025850},{"type":"v1","addr":"172.21.15.59:6823","nonce":2504025850}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6820","nonce":2504025850},{"type":"v1","addr":"172.21.15.59:6821","nonce":2504025850}]},"public_addr":"172.21.15.59:6817/2504025850","cluster_addr":"172.21.15.59:6819/2504025850","heartbeat_back_addr":"172.21.15.59:6823/2504025850","heartbeat_front_addr":"172.21.15.59:6821/2504025850","state":["exists","up"]},{"osd":4,"uuid":"7ac29fca-b5b2-4c95-9d69-2f164cbbc708","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6816","nonce":2382463431},{"type":"v1","addr":"172.21.15.76:6818","nonce":2382463431}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6820","nonce":2382463431},{"type":"v1","addr":"172.21.15.76:6822","nonce":2382463431}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6828","nonce":2382463431},{"type":"v1","addr":"172.21.15.76:6830","nonce":2382463431}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6824","nonce":2382463431},{"type":"v1","addr":"172.21.15.76:6826","nonce":2382463431}]},"public_addr":"172.21.15.76:6818/2382463431","cluster_addr":"172.21.15.76:6822/2382463431","heartbeat_back_addr":"172.21.15.76:6830/2382463431","heartbeat_front_addr":"172.21.15.76:6826/2382463431","state":["exists","up"]},{"osd":5,"uuid":"485bf711-3851-4238-a4f4-51d52e0a9c58","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6800","nonce":329334308},{"type":"v1","addr":"172.21.15.76:6802","nonce":329334308}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6805","nonce":329334308},{"type":"v1","addr":"172.21.15.76:6807","nonce":329334308}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6814","nonce":329334308},{"type":"v1","addr":"172.21.15.76:6815","nonce":329334308}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6810","nonce":329334308},{"type":"v1","addr":"172.21.15.76:6812","nonce":329334308}]},"public_addr":"172.21.15.76:6802/329334308","cluster_addr":"172.21.15.76:6807/329334308","heartbeat_back_addr":"172.21.15.76:6815/329334308","heartbeat_front_addr":"172.21.15.76:6812/329334308","state":["exists","up"]},{"osd":6,"uuid":"72b3f2fc-1e4a-4e32-968d-003647a0286f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6801","nonce":3946579410},{"type":"v1","addr":"172.21.15.76:6803","nonce":3946579410}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6804","nonce":3946579410},{"type":"v1","addr":"172.21.15.76:6806","nonce":3946579410}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6811","nonce":3946579410},{"type":"v1","addr":"172.21.15.76:6813","nonce":3946579410}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6808","nonce":3946579410},{"type":"v1","addr":"172.21.15.76:6809","nonce":3946579410}]},"public_addr":"172.21.15.76:6803/3946579410","cluster_addr":"172.21.15.76:6806/3946579410","heartbeat_back_addr":"172.21.15.76:6813/3946579410","heartbeat_front_addr":"172.21.15.76:6809/3946579410","state":["exists","up"]},{"osd":7,"uuid":"1ddceb90-6c5d-4d1a-ad56-99cb877f8c70","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6817","nonce":841761538},{"type":"v1","addr":"172.21.15.76:6819","nonce":841761538}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6821","nonce":841761538},{"type":"v1","addr":"172.21.15.76:6823","nonce":841761538}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6829","nonce":841761538},{"type":"v1","addr":"172.21.15.76:6831","nonce":841761538}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6825","nonce":841761538},{"type":"v1","addr":"172.21.15.76:6827","nonce":841761538}]},"public_addr":"172.21.15.76:6819/841761538","cluster_addr":"172.21.15.76:6823/841761538","heartbeat_back_addr":"172.21.15.76:6831/841761538","heartbeat_front_addr":"172.21.15.76:6827/841761538","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T07:56:14.294838+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T07:56:14.270779+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T07:56:14.231809+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T07:56:14.260983+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T07:56:14.302997+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T07:56:14.258600+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T07:56:14.251413+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T07:56:14.304727+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-24T07:57:25.960 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:25.958+0000 7f2e3e8c5700 1 -- 172.21.15.59:0/4162325179 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f2e1806aed0 msgr2=0x7f2e1806d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:57:25.960 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:25.958+0000 7f2e3e8c5700 1 --2- 172.21.15.59:0/4162325179 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f2e1806aed0 0x7f2e1806d390 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f2e28000f60 tx=0x7f2e28007cc0 comp rx=0 tx=0).stop 2023-11-24T07:57:25.960 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:25.958+0000 7f2e3e8c5700 1 -- 172.21.15.59:0/4162325179 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2e381459a0 msgr2=0x7f2e38153ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:57:25.960 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:25.958+0000 7f2e3e8c5700 1 --2- 172.21.15.59:0/4162325179 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2e381459a0 0x7f2e38153ca0 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f2e30007e30 tx=0x7f2e3000e3c0 comp rx=0 tx=0).stop 2023-11-24T07:57:25.960 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:25.958+0000 7f2e3e8c5700 1 -- 172.21.15.59:0/4162325179 shutdown_connections 2023-11-24T07:57:25.960 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:25.958+0000 7f2e3e8c5700 1 --2- 172.21.15.59:0/4162325179 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f2e1806aed0 0x7f2e1806d390 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:25.960 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:25.958+0000 7f2e3e8c5700 1 --2- 172.21.15.59:0/4162325179 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2e381459a0 0x7f2e38153ca0 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:25.960 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:25.958+0000 7f2e3e8c5700 1 --2- 172.21.15.59:0/4162325179 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f2e3806b6e0 0x7f2e381f22f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:25.960 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:25.958+0000 7f2e3e8c5700 1 --2- 172.21.15.59:0/4162325179 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2e38142810 0x7f2e381f1030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:25.960 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:25.958+0000 7f2e3e8c5700 1 -- 172.21.15.59:0/4162325179 >> 172.21.15.59:0/4162325179 conn(0x7f2e380b2740 msgr2=0x7f2e381416b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:57:25.961 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:25.958+0000 7f2e3e8c5700 1 -- 172.21.15.59:0/4162325179 shutdown_connections 2023-11-24T07:57:25.961 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:25.958+0000 7f2e3e8c5700 1 -- 172.21.15.59:0/4162325179 wait complete. 2023-11-24T07:57:25.972 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-24T07:57:26.097 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:26.095+0000 7f798e0ad700 1 Processor -- start 2023-11-24T07:57:26.097 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:26.095+0000 7f798e0ad700 1 -- start start 2023-11-24T07:57:26.099 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:26.096+0000 7f798e0ad700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f79881440f0 0x7f79880b51b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:57:26.099 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:26.096+0000 7f798e0ad700 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7988143ec0 con 0x7f79880722c0 2023-11-24T07:57:26.099 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:26.096+0000 7f798e0ad700 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f79880b3ae0 con 0x7f7988143ae0 2023-11-24T07:57:26.100 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:26.096+0000 7f798e0ad700 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f79880b40a0 con 0x7f79881440f0 2023-11-24T07:57:26.100 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:26.096+0000 7f7987fff700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f79881440f0 0x7f79880b51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:57:26.100 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:26.096+0000 7f7986ffd700 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7f79880722c0 0x7f798806e510 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:55160/0 (socket says 172.21.15.59:55160) 2023-11-24T07:57:26.100 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:26.096+0000 7f7986ffd700 1 -- 172.21.15.59:0/1985494532 learned_addr learned my addr 172.21.15.59:0/1985494532 (peer_addr_for_me v1:172.21.15.59:0/0) 2023-11-24T07:57:26.100 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:26.097+0000 7f7987fff700 1 -- 172.21.15.59:0/1985494532 >> v1:172.21.15.76:6789/0 conn(0x7f7988143ae0 legacy=0x7f7988071bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:57:26.100 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:26.097+0000 7f7987fff700 1 -- 172.21.15.59:0/1985494532 >> v1:172.21.15.59:6789/0 conn(0x7f79880722c0 legacy=0x7f798806e510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:57:26.100 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:26.097+0000 7f79867fc700 1 -- 172.21.15.59:0/1985494532 <== mon.0 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1810019230 0 0) 0x7f7988143ec0 con 0x7f79880722c0 2023-11-24T07:57:26.100 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:26.097+0000 7f7987fff700 1 -- 172.21.15.59:0/1985494532 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f79880b3dc0 con 0x7f79881440f0 2023-11-24T07:57:26.100 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:26.097+0000 7f7987fff700 1 --2- 172.21.15.59:0/1985494532 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f79881440f0 0x7f79880b51b0 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f7980009d90 tx=0x7f798000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a4469c3f9e62ad7 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:57:26.100 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:26.097+0000 7f79867fc700 1 -- 172.21.15.59:0/1985494532 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f798000e070 con 0x7f79881440f0 2023-11-24T07:57:26.100 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:26.097+0000 7f79867fc700 1 -- 172.21.15.59:0/1985494532 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f798000bbd0 con 0x7f79881440f0 2023-11-24T07:57:26.101 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:26.097+0000 7f79867fc700 1 -- 172.21.15.59:0/1985494532 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7980012430 con 0x7f79881440f0 2023-11-24T07:57:26.101 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:26.098+0000 7f798e0ad700 1 -- 172.21.15.59:0/1985494532 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f79881440f0 msgr2=0x7f79880b51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:57:26.101 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:26.098+0000 7f798e0ad700 1 --2- 172.21.15.59:0/1985494532 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f79881440f0 0x7f79880b51b0 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f7980009d90 tx=0x7f798000b330 comp rx=0 tx=0).stop 2023-11-24T07:57:26.101 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:26.098+0000 7f798e0ad700 1 -- 172.21.15.59:0/1985494532 shutdown_connections 2023-11-24T07:57:26.101 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:26.098+0000 7f798e0ad700 1 --2- 172.21.15.59:0/1985494532 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f79881440f0 0x7f79880b51b0 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:26.101 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:26.098+0000 7f798e0ad700 1 -- 172.21.15.59:0/1985494532 >> 172.21.15.59:0/1985494532 conn(0x7f79880b2740 msgr2=0x7f79880b2b20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:57:26.101 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:26.098+0000 7f798e0ad700 1 -- 172.21.15.59:0/1985494532 shutdown_connections 2023-11-24T07:57:26.101 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:26.098+0000 7f798e0ad700 1 -- 172.21.15.59:0/1985494532 wait complete. 2023-11-24T07:57:26.104 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:26.099+0000 7f798e0ad700 1 Processor -- start 2023-11-24T07:57:26.104 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:26.099+0000 7f798e0ad700 1 -- start start 2023-11-24T07:57:26.104 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:26.100+0000 7f798e0ad700 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f79880722c0 0x7f798813e9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:57:26.104 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:26.100+0000 7f798e0ad700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7988143ae0 0x7f79880cf030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:57:26.104 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:26.100+0000 7f798e0ad700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f79881440f0 0x7f79880cf570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:57:26.104 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:26.100+0000 7f798e0ad700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7988142310 con 0x7f79881440f0 2023-11-24T07:57:26.104 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:26.100+0000 7f798e0ad700 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7988141270 con 0x7f7988143ae0 2023-11-24T07:57:26.104 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:26.100+0000 7f798e0ad700 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7988141550 con 0x7f79880722c0 2023-11-24T07:57:26.104 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:26.100+0000 7f7987fff700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f79881440f0 0x7f79880cf570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:57:26.104 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:26.100+0000 7f7987fff700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f79881440f0 0x7f79880cf570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:51694/0 (socket says 172.21.15.59:51694) 2023-11-24T07:57:26.104 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:26.100+0000 7f7987fff700 1 -- 172.21.15.59:0/503062822 learned_addr learned my addr 172.21.15.59:0/503062822 (peer_addr_for_me v2:172.21.15.59:0/0) 2023-11-24T07:57:26.105 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:26.100+0000 7f7986ffd700 1 --2- 172.21.15.59:0/503062822 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7988143ae0 0x7f79880cf030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:57:26.105 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:26.100+0000 7f79877fe700 1 --2- 172.21.15.59:0/503062822 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f79880722c0 0x7f798813e9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:57:26.105 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:26.100+0000 7f7987fff700 1 -- 172.21.15.59:0/503062822 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f79880722c0 msgr2=0x7f798813e9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:57:26.105 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:26.100+0000 7f7987fff700 1 --2- 172.21.15.59:0/503062822 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f79880722c0 0x7f798813e9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:26.105 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:26.100+0000 7f7987fff700 1 -- 172.21.15.59:0/503062822 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7988143ae0 msgr2=0x7f79880cf030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:57:26.105 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:26.100+0000 7f7987fff700 1 --2- 172.21.15.59:0/503062822 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7988143ae0 0x7f79880cf030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:26.105 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:26.100+0000 7f7987fff700 1 -- 172.21.15.59:0/503062822 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7980009a70 con 0x7f79881440f0 2023-11-24T07:57:26.105 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:26.100+0000 7f7987fff700 1 --2- 172.21.15.59:0/503062822 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f79881440f0 0x7f79880cf570 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f7980014040 tx=0x7f7980004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:57:26.105 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:26.101+0000 7f796ffff700 1 -- 172.21.15.59:0/503062822 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f798001b810 con 0x7f79881440f0 2023-11-24T07:57:26.105 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:26.101+0000 7f796ffff700 1 -- 172.21.15.59:0/503062822 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7980020070 con 0x7f79881440f0 2023-11-24T07:57:26.105 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:26.101+0000 7f796ffff700 1 -- 172.21.15.59:0/503062822 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f798001bbb0 con 0x7f79881440f0 2023-11-24T07:57:26.105 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:26.101+0000 7f798e0ad700 1 -- 172.21.15.59:0/503062822 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7988141830 con 0x7f79881440f0 2023-11-24T07:57:26.106 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:26.101+0000 7f798e0ad700 1 -- 172.21.15.59:0/503062822 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7988141df0 con 0x7f79881440f0 2023-11-24T07:57:26.106 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:26.102+0000 7f798e0ad700 1 -- 172.21.15.59:0/503062822 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7988141fb0 con 0x7f79881440f0 2023-11-24T07:57:26.106 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:26.102+0000 7f796ffff700 1 -- 172.21.15.59:0/503062822 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f798001bd10 con 0x7f79881440f0 2023-11-24T07:57:26.106 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:26.102+0000 7f796ffff700 1 --2- 172.21.15.59:0/503062822 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f797406aed0 0x7f797406d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:57:26.106 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:26.103+0000 7f796ffff700 1 -- 172.21.15.59:0/503062822 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(32..32 src has 1..32) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f7980022070 con 0x7f79881440f0 2023-11-24T07:57:26.107 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:26.103+0000 7f79877fe700 1 --2- 172.21.15.59:0/503062822 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f797406aed0 0x7f797406d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:57:26.107 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:26.103+0000 7f79877fe700 1 --2- 172.21.15.59:0/503062822 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f797406aed0 0x7f797406d390 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f7978007fd0 tx=0x7f7978009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:57:26.108 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:26.107+0000 7f796ffff700 1 -- 172.21.15.59:0/503062822 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f7980056720 con 0x7f79881440f0 2023-11-24T07:57:26.311 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:26.310+0000 7f798e0ad700 1 -- 172.21.15.59:0/503062822 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f798806eba0 con 0x7f79881440f0 2023-11-24T07:57:26.312 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:26.311+0000 7f796ffff700 1 -- 172.21.15.59:0/503062822 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v32) v1 ==== 74+0+16450 (secure 0 0 0) 0x7f7980059d40 con 0x7f79881440f0 2023-11-24T07:57:26.313 INFO:teuthology.orchestra.run.smithi059.stdout: 2023-11-24T07:57:26.313 INFO:teuthology.orchestra.run.smithi059.stdout:{"epoch":32,"fsid":"9f0e8741-b8f9-48d5-a2c3-09f45903303d","created":"2023-11-24T07:56:06.552729+0000","modified":"2023-11-24T07:57:25.003306+0000","last_up_change":"2023-11-24T07:56:17.372352+0000","last_in_change":"2023-11-24T07:56:09.073250+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-24T07:56:18.352823+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2023-11-24T07:57:23.014143+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"32","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2023-11-24T07:57:23.971487+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"32","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"6d8f9e9b-1782-4e41-928b-a39f2a8ebc06","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6800","nonce":3373878772},{"type":"v1","addr":"172.21.15.59:6801","nonce":3373878772}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6802","nonce":3373878772},{"type":"v1","addr":"172.21.15.59:6803","nonce":3373878772}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6806","nonce":3373878772},{"type":"v1","addr":"172.21.15.59:6807","nonce":3373878772}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6804","nonce":3373878772},{"type":"v1","addr":"172.21.15.59:6805","nonce":3373878772}]},"public_addr":"172.21.15.59:6801/3373878772","cluster_addr":"172.21.15.59:6803/3373878772","heartbeat_back_addr":"172.21.15.59:6807/3373878772","heartbeat_front_addr":"172.21.15.59:6805/3373878772","state":["exists","up"]},{"osd":1,"uuid":"e8d882aa-52e1-4c04-9e1e-c44ef9204a5f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6824","nonce":648258563},{"type":"v1","addr":"172.21.15.59:6825","nonce":648258563}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6826","nonce":648258563},{"type":"v1","addr":"172.21.15.59:6827","nonce":648258563}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6830","nonce":648258563},{"type":"v1","addr":"172.21.15.59:6831","nonce":648258563}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6828","nonce":648258563},{"type":"v1","addr":"172.21.15.59:6829","nonce":648258563}]},"public_addr":"172.21.15.59:6825/648258563","cluster_addr":"172.21.15.59:6827/648258563","heartbeat_back_addr":"172.21.15.59:6831/648258563","heartbeat_front_addr":"172.21.15.59:6829/648258563","state":["exists","up"]},{"osd":2,"uuid":"69eebfa0-695b-4b21-90b9-8a3537168a01","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6808","nonce":1840430654},{"type":"v1","addr":"172.21.15.59:6809","nonce":1840430654}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6810","nonce":1840430654},{"type":"v1","addr":"172.21.15.59:6811","nonce":1840430654}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6814","nonce":1840430654},{"type":"v1","addr":"172.21.15.59:6815","nonce":1840430654}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6812","nonce":1840430654},{"type":"v1","addr":"172.21.15.59:6813","nonce":1840430654}]},"public_addr":"172.21.15.59:6809/1840430654","cluster_addr":"172.21.15.59:6811/1840430654","heartbeat_back_addr":"172.21.15.59:6815/1840430654","heartbeat_front_addr":"172.21.15.59:6813/1840430654","state":["exists","up"]},{"osd":3,"uuid":"96fd77fc-f911-46ff-9445-6f0be2daa8f4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6816","nonce":2504025850},{"type":"v1","addr":"172.21.15.59:6817","nonce":2504025850}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6818","nonce":2504025850},{"type":"v1","addr":"172.21.15.59:6819","nonce":2504025850}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6822","nonce":2504025850},{"type":"v1","addr":"172.21.15.59:6823","nonce":2504025850}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6820","nonce":2504025850},{"type":"v1","addr":"172.21.15.59:6821","nonce":2504025850}]},"public_addr":"172.21.15.59:6817/2504025850","cluster_addr":"172.21.15.59:6819/2504025850","heartbeat_back_addr":"172.21.15.59:6823/2504025850","heartbeat_front_addr":"172.21.15.59:6821/2504025850","state":["exists","up"]},{"osd":4,"uuid":"7ac29fca-b5b2-4c95-9d69-2f164cbbc708","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6816","nonce":2382463431},{"type":"v1","addr":"172.21.15.76:6818","nonce":2382463431}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6820","nonce":2382463431},{"type":"v1","addr":"172.21.15.76:6822","nonce":2382463431}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6828","nonce":2382463431},{"type":"v1","addr":"172.21.15.76:6830","nonce":2382463431}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6824","nonce":2382463431},{"type":"v1","addr":"172.21.15.76:6826","nonce":2382463431}]},"public_addr":"172.21.15.76:6818/2382463431","cluster_addr":"172.21.15.76:6822/2382463431","heartbeat_back_addr":"172.21.15.76:6830/2382463431","heartbeat_front_addr":"172.21.15.76:6826/2382463431","state":["exists","up"]},{"osd":5,"uuid":"485bf711-3851-4238-a4f4-51d52e0a9c58","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6800","nonce":329334308},{"type":"v1","addr":"172.21.15.76:6802","nonce":329334308}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6805","nonce":329334308},{"type":"v1","addr":"172.21.15.76:6807","nonce":329334308}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6814","nonce":329334308},{"type":"v1","addr":"172.21.15.76:6815","nonce":329334308}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6810","nonce":329334308},{"type":"v1","addr":"172.21.15.76:6812","nonce":329334308}]},"public_addr":"172.21.15.76:6802/329334308","cluster_addr":"172.21.15.76:6807/329334308","heartbeat_back_addr":"172.21.15.76:6815/329334308","heartbeat_front_addr":"172.21.15.76:6812/329334308","state":["exists","up"]},{"osd":6,"uuid":"72b3f2fc-1e4a-4e32-968d-003647a0286f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6801","nonce":3946579410},{"type":"v1","addr":"172.21.15.76:6803","nonce":3946579410}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6804","nonce":3946579410},{"type":"v1","addr":"172.21.15.76:6806","nonce":3946579410}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6811","nonce":3946579410},{"type":"v1","addr":"172.21.15.76:6813","nonce":3946579410}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6808","nonce":3946579410},{"type":"v1","addr":"172.21.15.76:6809","nonce":3946579410}]},"public_addr":"172.21.15.76:6803/3946579410","cluster_addr":"172.21.15.76:6806/3946579410","heartbeat_back_addr":"172.21.15.76:6813/3946579410","heartbeat_front_addr":"172.21.15.76:6809/3946579410","state":["exists","up"]},{"osd":7,"uuid":"1ddceb90-6c5d-4d1a-ad56-99cb877f8c70","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6817","nonce":841761538},{"type":"v1","addr":"172.21.15.76:6819","nonce":841761538}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6821","nonce":841761538},{"type":"v1","addr":"172.21.15.76:6823","nonce":841761538}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6829","nonce":841761538},{"type":"v1","addr":"172.21.15.76:6831","nonce":841761538}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6825","nonce":841761538},{"type":"v1","addr":"172.21.15.76:6827","nonce":841761538}]},"public_addr":"172.21.15.76:6819/841761538","cluster_addr":"172.21.15.76:6823/841761538","heartbeat_back_addr":"172.21.15.76:6831/841761538","heartbeat_front_addr":"172.21.15.76:6827/841761538","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T07:56:14.294838+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T07:56:14.270779+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T07:56:14.231809+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T07:56:14.260983+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T07:56:14.302997+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T07:56:14.258600+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T07:56:14.251413+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T07:56:14.304727+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-24T07:57:26.315 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:26.313+0000 7f798e0ad700 1 -- 172.21.15.59:0/503062822 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f797406aed0 msgr2=0x7f797406d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:57:26.315 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:26.313+0000 7f798e0ad700 1 --2- 172.21.15.59:0/503062822 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f797406aed0 0x7f797406d390 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f7978007fd0 tx=0x7f7978009040 comp rx=0 tx=0).stop 2023-11-24T07:57:26.315 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:26.313+0000 7f798e0ad700 1 -- 172.21.15.59:0/503062822 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f79881440f0 msgr2=0x7f79880cf570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:57:26.315 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:26.313+0000 7f798e0ad700 1 --2- 172.21.15.59:0/503062822 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f79881440f0 0x7f79880cf570 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f7980014040 tx=0x7f7980004ad0 comp rx=0 tx=0).stop 2023-11-24T07:57:26.315 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:26.313+0000 7f798e0ad700 1 -- 172.21.15.59:0/503062822 shutdown_connections 2023-11-24T07:57:26.315 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:26.313+0000 7f798e0ad700 1 --2- 172.21.15.59:0/503062822 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f797406aed0 0x7f797406d390 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:26.315 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:26.313+0000 7f798e0ad700 1 --2- 172.21.15.59:0/503062822 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f79881440f0 0x7f79880cf570 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:26.316 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:26.313+0000 7f798e0ad700 1 --2- 172.21.15.59:0/503062822 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f79880722c0 0x7f798813e9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:26.316 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:26.313+0000 7f798e0ad700 1 --2- 172.21.15.59:0/503062822 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7988143ae0 0x7f79880cf030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:26.316 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:26.313+0000 7f798e0ad700 1 -- 172.21.15.59:0/503062822 >> 172.21.15.59:0/503062822 conn(0x7f79880b2740 msgr2=0x7f798806e410 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:57:26.316 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:26.313+0000 7f798e0ad700 1 -- 172.21.15.59:0/503062822 shutdown_connections 2023-11-24T07:57:26.316 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:26.313+0000 7f798e0ad700 1 -- 172.21.15.59:0/503062822 wait complete. 2023-11-24T07:57:26.328 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2023-11-24T07:57:26.456 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:26.454+0000 7f190dfd7700 1 Processor -- start 2023-11-24T07:57:26.456 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:26.455+0000 7f190dfd7700 1 -- start start 2023-11-24T07:57:26.459 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:26.455+0000 7f190dfd7700 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f1908143ae0 0x7f19080b51b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:57:26.459 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:26.455+0000 7f190dfd7700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f19080722c0 0x7f190806e510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:57:26.459 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:26.455+0000 7f190dfd7700 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1908143ec0 con 0x7f19081440f0 2023-11-24T07:57:26.459 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:26.455+0000 7f190dfd7700 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f19080b40a0 con 0x7f19080722c0 2023-11-24T07:57:26.459 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:26.455+0000 7f190dfd7700 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f19080b3ae0 con 0x7f1908143ae0 2023-11-24T07:57:26.459 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:26.456+0000 7f1907fff700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f19080722c0 0x7f190806e510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:57:26.460 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:26.456+0000 7f1907fff700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f19080722c0 0x7f190806e510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:51702/0 (socket says 172.21.15.59:51702) 2023-11-24T07:57:26.460 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:26.456+0000 7f1907fff700 1 -- 172.21.15.59:0/1068098565 learned_addr learned my addr 172.21.15.59:0/1068098565 (peer_addr_for_me v2:172.21.15.59:0/0) 2023-11-24T07:57:26.460 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:26.456+0000 7f19077fe700 1 --2- 172.21.15.59:0/1068098565 >> v2:172.21.15.76:3300/0 conn(0x7f1908143ae0 0x7f19080b51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:57:26.460 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:26.456+0000 7f1907fff700 1 -- 172.21.15.59:0/1068098565 >> v2:172.21.15.76:3300/0 conn(0x7f1908143ae0 msgr2=0x7f19080b51b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T07:57:26.460 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:26.456+0000 7f1907fff700 1 --2- 172.21.15.59:0/1068098565 >> v2:172.21.15.76:3300/0 conn(0x7f1908143ae0 0x7f19080b51b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:26.460 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:26.456+0000 7f1907fff700 1 -- 172.21.15.59:0/1068098565 >> v1:172.21.15.76:6789/0 conn(0x7f19081440f0 legacy=0x7f1908071bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:57:26.460 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:26.456+0000 7f1907fff700 1 -- 172.21.15.59:0/1068098565 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f19080b3dc0 con 0x7f19080722c0 2023-11-24T07:57:26.460 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:26.456+0000 7f1907fff700 1 --2- 172.21.15.59:0/1068098565 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f19080722c0 0x7f190806e510 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f190806d2e0 tx=0x7f190000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2839660effd1d9e5 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:57:26.460 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:26.456+0000 7f19067fc700 1 -- 172.21.15.59:0/1068098565 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f190000e070 con 0x7f19080722c0 2023-11-24T07:57:26.460 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:26.456+0000 7f19067fc700 1 -- 172.21.15.59:0/1068098565 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f190000b9c0 con 0x7f19080722c0 2023-11-24T07:57:26.461 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:26.456+0000 7f19067fc700 1 -- 172.21.15.59:0/1068098565 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f190000bd70 con 0x7f19080722c0 2023-11-24T07:57:26.461 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:26.457+0000 7f190dfd7700 1 -- 172.21.15.59:0/1068098565 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f19080722c0 msgr2=0x7f190806e510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:57:26.461 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:26.457+0000 7f190dfd7700 1 --2- 172.21.15.59:0/1068098565 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f19080722c0 0x7f190806e510 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f190806d2e0 tx=0x7f190000b330 comp rx=0 tx=0).stop 2023-11-24T07:57:26.461 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:26.457+0000 7f190dfd7700 1 -- 172.21.15.59:0/1068098565 shutdown_connections 2023-11-24T07:57:26.461 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:26.457+0000 7f190dfd7700 1 --2- 172.21.15.59:0/1068098565 >> v2:172.21.15.76:3300/0 conn(0x7f1908143ae0 0x7f19080b51b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:26.461 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:26.457+0000 7f190dfd7700 1 --2- 172.21.15.59:0/1068098565 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f19080722c0 0x7f190806e510 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:26.461 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:26.457+0000 7f190dfd7700 1 -- 172.21.15.59:0/1068098565 >> 172.21.15.59:0/1068098565 conn(0x7f19080b2740 msgr2=0x7f19080b2b20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:57:26.461 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:26.457+0000 7f190dfd7700 1 -- 172.21.15.59:0/1068098565 shutdown_connections 2023-11-24T07:57:26.461 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:26.457+0000 7f190dfd7700 1 -- 172.21.15.59:0/1068098565 wait complete. 2023-11-24T07:57:26.464 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:26.458+0000 7f190dfd7700 1 Processor -- start 2023-11-24T07:57:26.465 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:26.458+0000 7f190dfd7700 1 -- start start 2023-11-24T07:57:26.465 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:26.459+0000 7f190dfd7700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f19080722c0 0x7f190813f3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:57:26.465 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:26.459+0000 7f190dfd7700 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f1908143ae0 0x7f1908155de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:57:26.465 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:26.459+0000 7f190dfd7700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f19081440f0 0x7f1908156320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:57:26.465 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:26.459+0000 7f190dfd7700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19081f1f80 con 0x7f19081440f0 2023-11-24T07:57:26.465 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:26.459+0000 7f190dfd7700 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19081569a0 con 0x7f19080722c0 2023-11-24T07:57:26.465 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:26.459+0000 7f190dfd7700 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1908156c80 con 0x7f1908143ae0 2023-11-24T07:57:26.465 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:26.459+0000 7f1907fff700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f19081440f0 0x7f1908156320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:57:26.465 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:26.459+0000 7f1907fff700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f19081440f0 0x7f1908156320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:51708/0 (socket says 172.21.15.59:51708) 2023-11-24T07:57:26.466 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:26.459+0000 7f1907fff700 1 -- 172.21.15.59:0/2214526152 learned_addr learned my addr 172.21.15.59:0/2214526152 (peer_addr_for_me v2:172.21.15.59:0/0) 2023-11-24T07:57:26.466 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:26.459+0000 7f19077fe700 1 --2- 172.21.15.59:0/2214526152 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f19080722c0 0x7f190813f3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:57:26.466 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:26.459+0000 7f1906ffd700 1 --2- 172.21.15.59:0/2214526152 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f1908143ae0 0x7f1908155de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:57:26.466 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:26.459+0000 7f1907fff700 1 -- 172.21.15.59:0/2214526152 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f1908143ae0 msgr2=0x7f1908155de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:57:26.466 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:26.459+0000 7f1907fff700 1 --2- 172.21.15.59:0/2214526152 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f1908143ae0 0x7f1908155de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:26.466 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:26.459+0000 7f1907fff700 1 -- 172.21.15.59:0/2214526152 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f19080722c0 msgr2=0x7f190813f3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:57:26.466 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:26.459+0000 7f1907fff700 1 --2- 172.21.15.59:0/2214526152 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f19080722c0 0x7f190813f3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:26.466 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:26.459+0000 7f1907fff700 1 -- 172.21.15.59:0/2214526152 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1900009b30 con 0x7f19081440f0 2023-11-24T07:57:26.466 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:26.459+0000 7f19077fe700 1 --2- 172.21.15.59:0/2214526152 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f19080722c0 0x7f190813f3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-24T07:57:26.466 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:26.459+0000 7f1907fff700 1 --2- 172.21.15.59:0/2214526152 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f19081440f0 0x7f1908156320 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f1900012000 tx=0x7f1900004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:57:26.467 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:26.460+0000 7f18effff700 1 -- 172.21.15.59:0/2214526152 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19000099b0 con 0x7f19081440f0 2023-11-24T07:57:26.467 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:26.460+0000 7f18effff700 1 -- 172.21.15.59:0/2214526152 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f19000092e0 con 0x7f19081440f0 2023-11-24T07:57:26.467 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:26.460+0000 7f190dfd7700 1 -- 172.21.15.59:0/2214526152 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1908156f60 con 0x7f19081440f0 2023-11-24T07:57:26.467 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:26.460+0000 7f18effff700 1 -- 172.21.15.59:0/2214526152 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f190001c640 con 0x7f19081440f0 2023-11-24T07:57:26.467 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:26.460+0000 7f190dfd7700 1 -- 172.21.15.59:0/2214526152 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f19081e4970 con 0x7f19081440f0 2023-11-24T07:57:26.467 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:26.461+0000 7f190dfd7700 1 -- 172.21.15.59:0/2214526152 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f19081f2140 con 0x7f19081440f0 2023-11-24T07:57:26.468 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:26.461+0000 7f18effff700 1 -- 172.21.15.59:0/2214526152 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f190001c7a0 con 0x7f19081440f0 2023-11-24T07:57:26.468 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:26.461+0000 7f18effff700 1 --2- 172.21.15.59:0/2214526152 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f18f406aed0 0x7f18f406d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:57:26.468 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:26.462+0000 7f18effff700 1 -- 172.21.15.59:0/2214526152 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(32..32 src has 1..32) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f190008af50 con 0x7f19081440f0 2023-11-24T07:57:26.468 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:26.462+0000 7f19077fe700 1 --2- 172.21.15.59:0/2214526152 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f18f406aed0 0x7f18f406d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:57:26.468 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:26.462+0000 7f19077fe700 1 --2- 172.21.15.59:0/2214526152 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f18f406aed0 0x7f18f406d390 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f19080d1830 tx=0x7f18f8008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:57:26.469 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:26.466+0000 7f18effff700 1 -- 172.21.15.59:0/2214526152 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f19081f2140 con 0x7f19081440f0 2023-11-24T07:57:26.719 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:26.717+0000 7f190dfd7700 1 -- 172.21.15.59:0/2214526152 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7f1908144830 con 0x7f19081440f0 2023-11-24T07:57:27.606 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:27.605+0000 7f18effff700 1 -- 172.21.15.59:0/2214526152 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v15) v1 ==== 118+0+0 (secure 0 0 0) 0x7f1900056090 con 0x7f19081440f0 2023-11-24T07:57:27.610 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:27.608+0000 7f190dfd7700 1 -- 172.21.15.59:0/2214526152 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f18f406aed0 msgr2=0x7f18f406d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:57:27.610 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:27.608+0000 7f190dfd7700 1 --2- 172.21.15.59:0/2214526152 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f18f406aed0 0x7f18f406d390 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f19080d1830 tx=0x7f18f8008040 comp rx=0 tx=0).stop 2023-11-24T07:57:27.610 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:27.608+0000 7f190dfd7700 1 -- 172.21.15.59:0/2214526152 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f19081440f0 msgr2=0x7f1908156320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:57:27.610 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:27.608+0000 7f190dfd7700 1 --2- 172.21.15.59:0/2214526152 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f19081440f0 0x7f1908156320 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f1900012000 tx=0x7f1900004ad0 comp rx=0 tx=0).stop 2023-11-24T07:57:27.610 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:27.608+0000 7f190dfd7700 1 -- 172.21.15.59:0/2214526152 shutdown_connections 2023-11-24T07:57:27.610 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:27.608+0000 7f190dfd7700 1 --2- 172.21.15.59:0/2214526152 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f18f406aed0 0x7f18f406d390 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:27.610 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:27.608+0000 7f190dfd7700 1 --2- 172.21.15.59:0/2214526152 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f19081440f0 0x7f1908156320 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:27.611 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:27.608+0000 7f190dfd7700 1 --2- 172.21.15.59:0/2214526152 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f19080722c0 0x7f190813f3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:27.611 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:27.608+0000 7f190dfd7700 1 --2- 172.21.15.59:0/2214526152 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f1908143ae0 0x7f1908155de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:27.611 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:27.608+0000 7f190dfd7700 1 -- 172.21.15.59:0/2214526152 >> 172.21.15.59:0/2214526152 conn(0x7f19080b2740 msgr2=0x7f19080ce020 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:57:27.611 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:27.608+0000 7f190dfd7700 1 -- 172.21.15.59:0/2214526152 shutdown_connections 2023-11-24T07:57:27.611 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:27.608+0000 7f190dfd7700 1 -- 172.21.15.59:0/2214526152 wait complete. 2023-11-24T07:57:27.625 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2023-11-24T07:57:27.625 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2023-11-24T07:57:27.762 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:27.760+0000 7f8df5614700 1 Processor -- start 2023-11-24T07:57:27.762 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:27.760+0000 7f8df5614700 1 -- start start 2023-11-24T07:57:27.764 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:27.761+0000 7f8df5614700 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f8df015ad50 0x7f8df00b2c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:57:27.764 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:27.761+0000 7f8df5614700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f8df00cc090 0x7f8df00725d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:57:27.765 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:27.761+0000 7f8df5614700 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f8df00b3790 0x7f8df00b3c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:57:27.765 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:27.761+0000 7f8df5614700 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8df00b4160 con 0x7f8df00cc090 2023-11-24T07:57:27.765 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:27.761+0000 7f8df5614700 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8df00cc540 con 0x7f8df015ad50 2023-11-24T07:57:27.765 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:27.761+0000 7f8df5614700 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8df015b130 con 0x7f8df00b3790 2023-11-24T07:57:27.765 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:27.761+0000 7f8dee7fc700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f8df00cc090 0x7f8df00725d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:57:27.765 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:27.761+0000 7f8dee7fc700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f8df00cc090 0x7f8df00725d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:51712/0 (socket says 172.21.15.59:51712) 2023-11-24T07:57:27.765 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:27.761+0000 7f8dee7fc700 1 -- 172.21.15.59:0/3647119067 learned_addr learned my addr 172.21.15.59:0/3647119067 (peer_addr_for_me v2:172.21.15.59:0/0) 2023-11-24T07:57:27.765 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:27.761+0000 7f8deeffd700 1 --2- 172.21.15.59:0/3647119067 >> v2:172.21.15.76:3300/0 conn(0x7f8df015ad50 0x7f8df00b2c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:57:27.765 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:27.762+0000 7f8def7fe700 1 --2- 172.21.15.59:0/3647119067 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f8df00b3790 0x7f8df00b3c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:57:27.765 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:27.762+0000 7f8dee7fc700 1 -- 172.21.15.59:0/3647119067 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f8df00b3790 msgr2=0x7f8df00b3c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T07:57:27.765 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:27.762+0000 7f8dee7fc700 1 --2- 172.21.15.59:0/3647119067 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f8df00b3790 0x7f8df00b3c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:27.765 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:27.762+0000 7f8dee7fc700 1 -- 172.21.15.59:0/3647119067 >> v2:172.21.15.76:3300/0 conn(0x7f8df015ad50 msgr2=0x7f8df00b2c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T07:57:27.766 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:27.762+0000 7f8dee7fc700 1 --2- 172.21.15.59:0/3647119067 >> v2:172.21.15.76:3300/0 conn(0x7f8df015ad50 0x7f8df00b2c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:27.766 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:27.762+0000 7f8dee7fc700 1 -- 172.21.15.59:0/3647119067 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8df0071530 con 0x7f8df00cc090 2023-11-24T07:57:27.766 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:27.762+0000 7f8dee7fc700 1 --2- 172.21.15.59:0/3647119067 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8df00cc090 0x7f8df00725d0 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f8df0073190 tx=0x7f8de0009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b407f0151a28eaed server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:57:27.766 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:27.762+0000 7f8dedffb700 1 -- 172.21.15.59:0/3647119067 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8de0011830 con 0x7f8df00cc090 2023-11-24T07:57:27.766 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:27.762+0000 7f8dedffb700 1 -- 172.21.15.59:0/3647119067 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8de0011990 con 0x7f8df00cc090 2023-11-24T07:57:27.766 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:27.762+0000 7f8dedffb700 1 -- 172.21.15.59:0/3647119067 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8de0011d20 con 0x7f8df00cc090 2023-11-24T07:57:27.766 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:27.763+0000 7f8df5614700 1 -- 172.21.15.59:0/3647119067 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8df00cc090 msgr2=0x7f8df00725d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:57:27.766 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:27.763+0000 7f8df5614700 1 --2- 172.21.15.59:0/3647119067 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8df00cc090 0x7f8df00725d0 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f8df0073190 tx=0x7f8de0009bd0 comp rx=0 tx=0).stop 2023-11-24T07:57:27.766 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:27.763+0000 7f8df5614700 1 -- 172.21.15.59:0/3647119067 shutdown_connections 2023-11-24T07:57:27.766 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:27.763+0000 7f8df5614700 1 --2- 172.21.15.59:0/3647119067 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f8df00b3790 0x7f8df00b3c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:27.767 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:27.763+0000 7f8df5614700 1 --2- 172.21.15.59:0/3647119067 >> v2:172.21.15.76:3300/0 conn(0x7f8df015ad50 0x7f8df00b2c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:27.767 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:27.763+0000 7f8df5614700 1 --2- 172.21.15.59:0/3647119067 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8df00cc090 0x7f8df00725d0 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:27.767 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:27.763+0000 7f8df5614700 1 -- 172.21.15.59:0/3647119067 >> 172.21.15.59:0/3647119067 conn(0x7f8df00b1650 msgr2=0x7f8df00b1a30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:57:27.767 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:27.763+0000 7f8df5614700 1 -- 172.21.15.59:0/3647119067 shutdown_connections 2023-11-24T07:57:27.767 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:27.763+0000 7f8df5614700 1 -- 172.21.15.59:0/3647119067 wait complete. 2023-11-24T07:57:27.769 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:27.764+0000 7f8df5614700 1 Processor -- start 2023-11-24T07:57:27.769 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:27.764+0000 7f8df5614700 1 -- start start 2023-11-24T07:57:27.770 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:27.765+0000 7f8df5614700 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f8df00b3790 0x7f8df00cddd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:57:27.770 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:27.765+0000 7f8df5614700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8df00cc090 0x7f8df014bdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:57:27.770 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:27.765+0000 7f8df5614700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8df015ad50 0x7f8df00ae870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:57:27.770 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:27.765+0000 7f8df5614700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8df00ae450 con 0x7f8df00cc090 2023-11-24T07:57:27.770 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:27.765+0000 7f8df5614700 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8df00ad860 con 0x7f8df015ad50 2023-11-24T07:57:27.770 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:27.765+0000 7f8df5614700 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8df00adab0 con 0x7f8df00b3790 2023-11-24T07:57:27.770 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:27.765+0000 7f8dee7fc700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8df00cc090 0x7f8df014bdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:57:27.770 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:27.765+0000 7f8dee7fc700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8df00cc090 0x7f8df014bdf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:51718/0 (socket says 172.21.15.59:51718) 2023-11-24T07:57:27.770 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:27.765+0000 7f8dee7fc700 1 -- 172.21.15.59:0/269959029 learned_addr learned my addr 172.21.15.59:0/269959029 (peer_addr_for_me v2:172.21.15.59:0/0) 2023-11-24T07:57:27.770 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:27.765+0000 7f8deeffd700 1 --2- 172.21.15.59:0/269959029 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f8df00b3790 0x7f8df00cddd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:57:27.770 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:27.765+0000 7f8def7fe700 1 --2- 172.21.15.59:0/269959029 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8df015ad50 0x7f8df00ae870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:57:27.771 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:27.765+0000 7f8dee7fc700 1 -- 172.21.15.59:0/269959029 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f8df00b3790 msgr2=0x7f8df00cddd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:57:27.771 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:27.765+0000 7f8dee7fc700 1 --2- 172.21.15.59:0/269959029 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f8df00b3790 0x7f8df00cddd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:27.771 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:27.765+0000 7f8dee7fc700 1 -- 172.21.15.59:0/269959029 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8df015ad50 msgr2=0x7f8df00ae870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:57:27.771 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:27.765+0000 7f8dee7fc700 1 --2- 172.21.15.59:0/269959029 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8df015ad50 0x7f8df00ae870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:27.771 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:27.765+0000 7f8dee7fc700 1 -- 172.21.15.59:0/269959029 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8de0009550 con 0x7f8df00cc090 2023-11-24T07:57:27.771 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:27.765+0000 7f8dee7fc700 1 --2- 172.21.15.59:0/269959029 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8df00cc090 0x7f8df014bdf0 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f8de0000c00 tx=0x7f8de0004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:57:27.771 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:27.765+0000 7f8dd77fe700 1 -- 172.21.15.59:0/269959029 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8de0004240 con 0x7f8df00cc090 2023-11-24T07:57:27.771 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:27.766+0000 7f8dd77fe700 1 -- 172.21.15.59:0/269959029 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8de001f070 con 0x7f8df00cc090 2023-11-24T07:57:27.771 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:27.766+0000 7f8dd77fe700 1 -- 172.21.15.59:0/269959029 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8de00036a0 con 0x7f8df00cc090 2023-11-24T07:57:27.771 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:27.766+0000 7f8df5614700 1 -- 172.21.15.59:0/269959029 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8df00add90 con 0x7f8df00cc090 2023-11-24T07:57:27.771 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:27.766+0000 7f8df5614700 1 -- 172.21.15.59:0/269959029 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8df00ce310 con 0x7f8df00cc090 2023-11-24T07:57:27.771 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:27.767+0000 7f8df5614700 1 -- 172.21.15.59:0/269959029 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8df00ce4a0 con 0x7f8df00cc090 2023-11-24T07:57:27.771 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:27.767+0000 7f8dd77fe700 1 -- 172.21.15.59:0/269959029 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f8de0021030 con 0x7f8df00cc090 2023-11-24T07:57:27.772 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:27.767+0000 7f8dd77fe700 1 --2- 172.21.15.59:0/269959029 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f8ddc06aed0 0x7f8ddc06d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:57:27.772 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:27.767+0000 7f8dd77fe700 1 -- 172.21.15.59:0/269959029 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f8de008a740 con 0x7f8df00cc090 2023-11-24T07:57:27.772 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:27.767+0000 7f8deeffd700 1 --2- 172.21.15.59:0/269959029 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f8ddc06aed0 0x7f8ddc06d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:57:27.772 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:27.768+0000 7f8deeffd700 1 --2- 172.21.15.59:0/269959029 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f8ddc06aed0 0x7f8ddc06d390 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f8df00ce780 tx=0x7f8de4009210 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:57:27.773 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:27.772+0000 7f8dd77fe700 1 -- 172.21.15.59:0/269959029 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f8de0055880 con 0x7f8df00cc090 2023-11-24T07:57:28.022 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:28.020+0000 7f8df5614700 1 -- 172.21.15.59:0/269959029 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7f8df00ae590 con 0x7f8df00cc090 2023-11-24T07:57:28.612 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:28.610+0000 7f8dd77fe700 1 -- 172.21.15.59:0/269959029 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v16) v1 ==== 122+0+0 (secure 0 0 0) 0x7f8df00ae590 con 0x7f8df00cc090 2023-11-24T07:57:28.616 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:28.613+0000 7f8df5614700 1 -- 172.21.15.59:0/269959029 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f8ddc06aed0 msgr2=0x7f8ddc06d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:57:28.617 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:28.613+0000 7f8df5614700 1 --2- 172.21.15.59:0/269959029 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f8ddc06aed0 0x7f8ddc06d390 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f8df00ce780 tx=0x7f8de4009210 comp rx=0 tx=0).stop 2023-11-24T07:57:28.617 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:28.613+0000 7f8df5614700 1 -- 172.21.15.59:0/269959029 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8df00cc090 msgr2=0x7f8df014bdf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:57:28.617 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:28.613+0000 7f8df5614700 1 --2- 172.21.15.59:0/269959029 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8df00cc090 0x7f8df014bdf0 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f8de0000c00 tx=0x7f8de0004750 comp rx=0 tx=0).stop 2023-11-24T07:57:28.618 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:28.613+0000 7f8df5614700 1 -- 172.21.15.59:0/269959029 shutdown_connections 2023-11-24T07:57:28.618 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:28.613+0000 7f8df5614700 1 --2- 172.21.15.59:0/269959029 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f8ddc06aed0 0x7f8ddc06d390 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:28.618 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:28.613+0000 7f8df5614700 1 --2- 172.21.15.59:0/269959029 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8df015ad50 0x7f8df00ae870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:28.618 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:28.613+0000 7f8df5614700 1 --2- 172.21.15.59:0/269959029 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f8df00b3790 0x7f8df00cddd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:28.618 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:28.613+0000 7f8df5614700 1 --2- 172.21.15.59:0/269959029 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8df00cc090 0x7f8df014bdf0 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:28.619 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:28.613+0000 7f8df5614700 1 -- 172.21.15.59:0/269959029 >> 172.21.15.59:0/269959029 conn(0x7f8df00b1650 msgr2=0x7f8df00b04d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:57:28.619 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:28.613+0000 7f8df5614700 1 -- 172.21.15.59:0/269959029 shutdown_connections 2023-11-24T07:57:28.619 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:28.613+0000 7f8df5614700 1 -- 172.21.15.59:0/269959029 wait complete. 2023-11-24T07:57:28.629 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2023-11-24T07:57:28.630 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-24T07:57:28.817 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:28.760+0000 7fadb8375700 1 Processor -- start 2023-11-24T07:57:28.817 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:28.760+0000 7fadb8375700 1 -- start start 2023-11-24T07:57:28.817 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:28.761+0000 7fadb8375700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fadb006cae0 0x7fadb00ce440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:57:28.817 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:28.761+0000 7fadb8375700 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fadb00ce980 0x7fadb00cafa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:57:28.817 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:28.761+0000 7fadb8375700 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fadb00cab10 con 0x7fadb0070a80 2023-11-24T07:57:28.817 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:28.761+0000 7fadb8375700 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fadb00c9fd0 con 0x7fadb006cae0 2023-11-24T07:57:28.817 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:28.761+0000 7fadb8375700 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fadb00cd7e0 con 0x7fadb00ce980 2023-11-24T07:57:28.817 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:28.761+0000 7fadb5910700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fadb006cae0 0x7fadb00ce440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:57:28.818 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:28.761+0000 7fadb5910700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fadb006cae0 0x7fadb00ce440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:51726/0 (socket says 172.21.15.59:51726) 2023-11-24T07:57:28.818 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:28.761+0000 7fadb5910700 1 -- 172.21.15.59:0/1540780887 learned_addr learned my addr 172.21.15.59:0/1540780887 (peer_addr_for_me v2:172.21.15.59:0/0) 2023-11-24T07:57:28.818 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:28.761+0000 7fadb6912700 1 --2- 172.21.15.59:0/1540780887 >> v2:172.21.15.76:3300/0 conn(0x7fadb00ce980 0x7fadb00cafa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:57:28.818 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:28.761+0000 7fadb510f700 1 -- 172.21.15.59:0/1540780887 <== mon.1 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2409195724 0 0) 0x7fadb00cab10 con 0x7fadb0070a80 2023-11-24T07:57:28.818 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:28.814+0000 7fadb510f700 1 -- 172.21.15.59:0/1540780887 --> v1:172.21.15.76:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fad9c003590 con 0x7fadb0070a80 2023-11-24T07:57:28.818 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:28.814+0000 7fadb5910700 1 -- 172.21.15.59:0/1540780887 >> v2:172.21.15.76:3300/0 conn(0x7fadb00ce980 msgr2=0x7fadb00cafa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T07:57:28.818 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:28.814+0000 7fadb5910700 1 --2- 172.21.15.59:0/1540780887 >> v2:172.21.15.76:3300/0 conn(0x7fadb00ce980 0x7fadb00cafa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:28.818 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:28.814+0000 7fadb5910700 1 -- 172.21.15.59:0/1540780887 >> v1:172.21.15.76:6789/0 conn(0x7fadb0070a80 legacy=0x7fadb0070e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:57:28.818 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:28.814+0000 7fadb5910700 1 -- 172.21.15.59:0/1540780887 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fadb00ca1e0 con 0x7fadb006cae0 2023-11-24T07:57:28.818 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:28.814+0000 7fadb5910700 1 --2- 172.21.15.59:0/1540780887 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fadb006cae0 0x7fadb00ce440 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7fadb008b800 tx=0x7fadac00ac80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d2b8deef038826b9 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:57:28.818 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:28.814+0000 7fadb510f700 1 -- 172.21.15.59:0/1540780887 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fadac004230 con 0x7fadb006cae0 2023-11-24T07:57:28.818 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:28.814+0000 7fadb510f700 1 -- 172.21.15.59:0/1540780887 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fadac0043b0 con 0x7fadb006cae0 2023-11-24T07:57:28.819 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:28.814+0000 7fadb510f700 1 -- 172.21.15.59:0/1540780887 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fadac0038e0 con 0x7fadb006cae0 2023-11-24T07:57:28.819 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:28.815+0000 7fadb8375700 1 -- 172.21.15.59:0/1540780887 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fadb006cae0 msgr2=0x7fadb00ce440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:57:28.819 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:28.815+0000 7fadb8375700 1 --2- 172.21.15.59:0/1540780887 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fadb006cae0 0x7fadb00ce440 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7fadb008b800 tx=0x7fadac00ac80 comp rx=0 tx=0).stop 2023-11-24T07:57:28.819 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:28.815+0000 7fadb8375700 1 -- 172.21.15.59:0/1540780887 shutdown_connections 2023-11-24T07:57:28.819 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:28.815+0000 7fadb8375700 1 --2- 172.21.15.59:0/1540780887 >> v2:172.21.15.76:3300/0 conn(0x7fadb00ce980 0x7fadb00cafa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:28.819 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:28.815+0000 7fadb8375700 1 --2- 172.21.15.59:0/1540780887 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fadb006cae0 0x7fadb00ce440 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:28.819 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:28.815+0000 7fadb8375700 1 -- 172.21.15.59:0/1540780887 >> 172.21.15.59:0/1540780887 conn(0x7fadb00cf980 msgr2=0x7fadb00cfd90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:57:28.819 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:28.815+0000 7fadb8375700 1 -- 172.21.15.59:0/1540780887 shutdown_connections 2023-11-24T07:57:28.819 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:28.815+0000 7fadb8375700 1 -- 172.21.15.59:0/1540780887 wait complete. 2023-11-24T07:57:28.822 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:28.816+0000 7fadb8375700 1 Processor -- start 2023-11-24T07:57:28.822 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:28.816+0000 7fadb8375700 1 -- start start 2023-11-24T07:57:28.822 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:28.817+0000 7fadb8375700 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fadb006cae0 0x7fadb016d1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:57:28.822 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:28.817+0000 7fadb8375700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fadb0070a80 0x7fadb0166750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:57:28.822 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:28.817+0000 7fadb8375700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fadb00ce980 0x7fadb01634b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:57:28.822 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:28.817+0000 7fadb8375700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fadb01639f0 con 0x7fadb0070a80 2023-11-24T07:57:28.822 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:28.817+0000 7fadb8375700 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fadb016cd50 con 0x7fadb00ce980 2023-11-24T07:57:28.822 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:28.817+0000 7fadb8375700 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fadb016c420 con 0x7fadb006cae0 2023-11-24T07:57:28.822 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:28.817+0000 7fadb5910700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fadb0070a80 0x7fadb0166750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:57:28.822 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:28.817+0000 7fadb5910700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fadb0070a80 0x7fadb0166750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:51732/0 (socket says 172.21.15.59:51732) 2023-11-24T07:57:28.822 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:28.817+0000 7fadb5910700 1 -- 172.21.15.59:0/2763313490 learned_addr learned my addr 172.21.15.59:0/2763313490 (peer_addr_for_me v2:172.21.15.59:0/0) 2023-11-24T07:57:28.823 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:28.817+0000 7fadb6912700 1 --2- 172.21.15.59:0/2763313490 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fadb00ce980 0x7fadb01634b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:57:28.823 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:28.817+0000 7fadb6111700 1 --2- 172.21.15.59:0/2763313490 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fadb006cae0 0x7fadb016d1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:57:28.823 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:28.817+0000 7fadb5910700 1 -- 172.21.15.59:0/2763313490 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fadb006cae0 msgr2=0x7fadb016d1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:57:28.823 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:28.817+0000 7fadb5910700 1 --2- 172.21.15.59:0/2763313490 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fadb006cae0 0x7fadb016d1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:28.823 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:28.818+0000 7fadb5910700 1 -- 172.21.15.59:0/2763313490 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fadb00ce980 msgr2=0x7fadb01634b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:57:28.823 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:28.818+0000 7fadb5910700 1 --2- 172.21.15.59:0/2763313490 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fadb00ce980 0x7fadb01634b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:28.823 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:28.818+0000 7fadb5910700 1 -- 172.21.15.59:0/2763313490 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fadac00a600 con 0x7fadb0070a80 2023-11-24T07:57:28.823 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:28.818+0000 7fadb5910700 1 --2- 172.21.15.59:0/2763313490 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fadb0070a80 0x7fadb0166750 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7fadac00acc0 tx=0x7fadac004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:57:28.823 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:28.818+0000 7fada6ffd700 1 -- 172.21.15.59:0/2763313490 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fadac01f070 con 0x7fadb0070a80 2023-11-24T07:57:28.823 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:28.818+0000 7fada6ffd700 1 -- 172.21.15.59:0/2763313490 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fadac0038b0 con 0x7fadb0070a80 2023-11-24T07:57:28.823 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:28.818+0000 7fadb8375700 1 -- 172.21.15.59:0/2763313490 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fadb016c700 con 0x7fadb0070a80 2023-11-24T07:57:28.823 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:28.818+0000 7fada6ffd700 1 -- 172.21.15.59:0/2763313490 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fadac00b7a0 con 0x7fadb0070a80 2023-11-24T07:57:28.824 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:28.818+0000 7fadb8375700 1 -- 172.21.15.59:0/2763313490 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fadb0166e50 con 0x7fadb0070a80 2023-11-24T07:57:28.824 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:28.819+0000 7fadb8375700 1 -- 172.21.15.59:0/2763313490 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fadb0067f60 con 0x7fadb0070a80 2023-11-24T07:57:28.824 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:28.819+0000 7fada6ffd700 1 -- 172.21.15.59:0/2763313490 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fadac00b900 con 0x7fadb0070a80 2023-11-24T07:57:28.824 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:28.820+0000 7fada6ffd700 1 --2- 172.21.15.59:0/2763313490 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7fad9406aed0 0x7fad9406d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:57:28.824 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:28.820+0000 7fada6ffd700 1 -- 172.21.15.59:0/2763313490 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fadac08a1e0 con 0x7fadb0070a80 2023-11-24T07:57:28.824 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:28.820+0000 7fadb6111700 1 --2- 172.21.15.59:0/2763313490 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7fad9406aed0 0x7fad9406d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:57:28.824 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:28.821+0000 7fadb6111700 1 --2- 172.21.15.59:0/2763313490 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7fad9406aed0 0x7fad9406d390 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fada00011e0 tx=0x7fada0009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:57:28.826 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:28.824+0000 7fada6ffd700 1 -- 172.21.15.59:0/2763313490 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fadac055280 con 0x7fadb0070a80 2023-11-24T07:57:29.069 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:29.067+0000 7fadb8375700 1 -- 172.21.15.59:0/2763313490 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fadb006d0a0 con 0x7fadb0070a80 2023-11-24T07:57:29.069 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:29.068+0000 7fada6ffd700 1 -- 172.21.15.59:0/2763313490 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 16 v16) v1 ==== 94+0+4913 (secure 0 0 0) 0x7fadac0588a0 con 0x7fadb0070a80 2023-11-24T07:57:29.070 INFO:teuthology.orchestra.run.smithi059.stdout: 2023-11-24T07:57:29.070 INFO:teuthology.orchestra.run.smithi059.stdout:{"epoch":16,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4763,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.76:6836/1669441817","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6834","nonce":1669441817},{"type":"v1","addr":"172.21.15.76:6836","nonce":1669441817}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":4769,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.76:6837/3709928230","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6835","nonce":3709928230},{"type":"v1","addr":"172.21.15.76:6837","nonce":3709928230}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":5106,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6834/2019319496","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6832","nonce":2019319496},{"type":"v1","addr":"172.21.15.59:6834","nonce":2019319496}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":16,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-24T07:57:25.003636+0000","modified":"2023-11-24T07:57:28.021959+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5109},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5109":{"gid":5109,"name":"a","rank":0,"incarnation":13,"state":"up:active","state_seq":3,"addr":"172.21.15.59:6835/2290655475","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6833","nonce":2290655475},{"type":"v1","addr":"172.21.15.59:6835","nonce":2290655475}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2023-11-24T07:57:29.072 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:29.069+0000 7fadb8375700 1 -- 172.21.15.59:0/2763313490 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7fad9406aed0 msgr2=0x7fad9406d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:57:29.072 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:29.069+0000 7fadb8375700 1 --2- 172.21.15.59:0/2763313490 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7fad9406aed0 0x7fad9406d390 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fada00011e0 tx=0x7fada0009040 comp rx=0 tx=0).stop 2023-11-24T07:57:29.073 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:29.070+0000 7fadb8375700 1 -- 172.21.15.59:0/2763313490 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fadb0070a80 msgr2=0x7fadb0166750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:57:29.073 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:29.070+0000 7fadb8375700 1 --2- 172.21.15.59:0/2763313490 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fadb0070a80 0x7fadb0166750 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7fadac00acc0 tx=0x7fadac004750 comp rx=0 tx=0).stop 2023-11-24T07:57:29.073 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:29.070+0000 7fadb8375700 1 -- 172.21.15.59:0/2763313490 shutdown_connections 2023-11-24T07:57:29.073 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:29.070+0000 7fadb8375700 1 --2- 172.21.15.59:0/2763313490 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7fad9406aed0 0x7fad9406d390 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:29.073 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:29.070+0000 7fadb8375700 1 --2- 172.21.15.59:0/2763313490 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fadb00ce980 0x7fadb01634b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:29.073 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:29.070+0000 7fadb8375700 1 --2- 172.21.15.59:0/2763313490 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fadb006cae0 0x7fadb016d1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:29.074 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:29.070+0000 7fadb8375700 1 --2- 172.21.15.59:0/2763313490 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fadb0070a80 0x7fadb0166750 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:29.074 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:29.070+0000 7fadb8375700 1 -- 172.21.15.59:0/2763313490 >> 172.21.15.59:0/2763313490 conn(0x7fadb00cf980 msgr2=0x7fadb016edb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:57:29.074 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:29.070+0000 7fadb8375700 1 -- 172.21.15.59:0/2763313490 shutdown_connections 2023-11-24T07:57:29.074 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:29.070+0000 7fadb8375700 1 -- 172.21.15.59:0/2763313490 wait complete. 2023-11-24T07:57:29.074 INFO:teuthology.orchestra.run.smithi059.stderr:dumped fsmap epoch 16 2023-11-24T07:57:29.084 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-24T07:57:29.209 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:29.207+0000 7fe7e4453700 1 Processor -- start 2023-11-24T07:57:29.209 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:29.207+0000 7fe7e4453700 1 -- start start 2023-11-24T07:57:29.212 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:29.208+0000 7fe7e4453700 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fe7dc15bde0 0x7fe7dc15c1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:57:29.212 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:29.208+0000 7fe7e4453700 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fe7dc0cf2b0 0x7fe7dc0d01b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:57:29.212 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:29.208+0000 7fe7e4453700 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe7dc0d0a90 con 0x7fe7dc0ce5c0 2023-11-24T07:57:29.213 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:29.208+0000 7fe7e4453700 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe7dc06ee10 con 0x7fe7dc0cf2b0 2023-11-24T07:57:29.213 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:29.208+0000 7fe7e4453700 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe7dc06f0f0 con 0x7fe7dc15bde0 2023-11-24T07:57:29.213 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:29.209+0000 7fe7e29f0700 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7fe7dc0ce5c0 0x7fe7dc0cc670 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:55216/0 (socket says 172.21.15.59:55216) 2023-11-24T07:57:29.214 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:29.209+0000 7fe7e29f0700 1 -- 172.21.15.59:0/2082344682 learned_addr learned my addr 172.21.15.59:0/2082344682 (peer_addr_for_me v1:172.21.15.59:0/0) 2023-11-24T07:57:29.214 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:29.209+0000 7fe7e21ef700 1 --2- 172.21.15.59:0/2082344682 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fe7dc15bde0 0x7fe7dc15c1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:57:29.214 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:29.209+0000 7fe7e19ee700 1 --2- 172.21.15.59:0/2082344682 >> v2:172.21.15.76:3300/0 conn(0x7fe7dc0cf2b0 0x7fe7dc0d01b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:57:29.215 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:29.209+0000 7fe7e11ed700 1 -- 172.21.15.59:0/2082344682 <== mon.0 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3213120415 0 0) 0x7fe7dc0d0a90 con 0x7fe7dc0ce5c0 2023-11-24T07:57:29.216 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:29.209+0000 7fe7e11ed700 1 -- 172.21.15.59:0/2082344682 --> v1:172.21.15.59:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe7c80035b0 con 0x7fe7dc0ce5c0 2023-11-24T07:57:29.216 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:29.209+0000 7fe7e11ed700 1 -- 172.21.15.59:0/2082344682 <== mon.0 v1:172.21.15.59:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2727244490 0 0) 0x7fe7c80035b0 con 0x7fe7dc0ce5c0 2023-11-24T07:57:29.216 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:29.209+0000 7fe7e21ef700 1 -- 172.21.15.59:0/2082344682 >> v2:172.21.15.76:3300/0 conn(0x7fe7dc0cf2b0 msgr2=0x7fe7dc0d01b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T07:57:29.217 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:29.209+0000 7fe7e21ef700 1 --2- 172.21.15.59:0/2082344682 >> v2:172.21.15.76:3300/0 conn(0x7fe7dc0cf2b0 0x7fe7dc0d01b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:29.217 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:29.209+0000 7fe7e21ef700 1 -- 172.21.15.59:0/2082344682 >> v1:172.21.15.59:6789/0 conn(0x7fe7dc0ce5c0 legacy=0x7fe7dc0cc670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:57:29.217 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:29.209+0000 7fe7e21ef700 1 -- 172.21.15.59:0/2082344682 --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe7dc06f3d0 con 0x7fe7dc15bde0 2023-11-24T07:57:29.218 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:29.209+0000 7fe7e19ee700 1 --2- 172.21.15.59:0/2082344682 >> v2:172.21.15.76:3300/0 conn(0x7fe7dc0cf2b0 0x7fe7dc0d01b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-24T07:57:29.218 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:29.209+0000 7fe7e21ef700 1 --2- 172.21.15.59:0/2082344682 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fe7dc15bde0 0x7fe7dc15c1c0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fe7cc009840 tx=0x7fe7cc009b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=12128bc836e41400 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:57:29.218 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:29.209+0000 7fe7e11ed700 1 -- 172.21.15.59:0/2082344682 <== mon.2 v2:172.21.15.76:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7d40019f0 con 0x7fe7dc15bde0 2023-11-24T07:57:29.219 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:29.210+0000 7fe7e11ed700 1 -- 172.21.15.59:0/2082344682 <== mon.2 v2:172.21.15.76:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe7cc004030 con 0x7fe7dc15bde0 2023-11-24T07:57:29.219 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:29.210+0000 7fe7e11ed700 1 -- 172.21.15.59:0/2082344682 <== mon.2 v2:172.21.15.76:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7cc0043a0 con 0x7fe7dc15bde0 2023-11-24T07:57:29.219 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:29.210+0000 7fe7e4453700 1 -- 172.21.15.59:0/2082344682 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fe7dc15bde0 msgr2=0x7fe7dc15c1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:57:29.219 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:29.210+0000 7fe7e4453700 1 --2- 172.21.15.59:0/2082344682 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fe7dc15bde0 0x7fe7dc15c1c0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fe7cc009840 tx=0x7fe7cc009b50 comp rx=0 tx=0).stop 2023-11-24T07:57:29.219 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:29.210+0000 7fe7e4453700 1 -- 172.21.15.59:0/2082344682 shutdown_connections 2023-11-24T07:57:29.220 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:29.210+0000 7fe7e4453700 1 --2- 172.21.15.59:0/2082344682 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fe7dc15bde0 0x7fe7dc15c1c0 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:29.220 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:29.210+0000 7fe7e4453700 1 --2- 172.21.15.59:0/2082344682 >> v2:172.21.15.76:3300/0 conn(0x7fe7dc0cf2b0 0x7fe7dc0d01b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:29.220 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:29.210+0000 7fe7e4453700 1 -- 172.21.15.59:0/2082344682 >> 172.21.15.59:0/2082344682 conn(0x7fe7dc0b3720 msgr2=0x7fe7dc0b3b30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:57:29.220 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:29.210+0000 7fe7e4453700 1 -- 172.21.15.59:0/2082344682 shutdown_connections 2023-11-24T07:57:29.221 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:29.210+0000 7fe7e4453700 1 -- 172.21.15.59:0/2082344682 wait complete. 2023-11-24T07:57:29.225 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:29.211+0000 7fe7e4453700 1 Processor -- start 2023-11-24T07:57:29.225 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:29.211+0000 7fe7e4453700 1 -- start start 2023-11-24T07:57:29.225 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:29.212+0000 7fe7e4453700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe7dc0ce5c0 0x7fe7dc0aed50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:57:29.225 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:29.212+0000 7fe7e4453700 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fe7dc0cf2b0 0x7fe7dc14cfb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:57:29.226 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:29.212+0000 7fe7e4453700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe7dc15bde0 0x7fe7dc0b3b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:57:29.226 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:29.212+0000 7fe7e4453700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7dc0b0140 con 0x7fe7dc15bde0 2023-11-24T07:57:29.226 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:29.212+0000 7fe7e4453700 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7dc0af290 con 0x7fe7dc0ce5c0 2023-11-24T07:57:29.227 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:29.212+0000 7fe7e4453700 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe7dc0af4c0 con 0x7fe7dc0cf2b0 2023-11-24T07:57:29.227 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:29.212+0000 7fe7e29f0700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe7dc15bde0 0x7fe7dc0b3b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:57:29.227 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:29.212+0000 7fe7e29f0700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe7dc15bde0 0x7fe7dc0b3b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:51748/0 (socket says 172.21.15.59:51748) 2023-11-24T07:57:29.227 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:29.212+0000 7fe7e29f0700 1 -- 172.21.15.59:0/2259293385 learned_addr learned my addr 172.21.15.59:0/2259293385 (peer_addr_for_me v2:172.21.15.59:0/0) 2023-11-24T07:57:29.228 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:29.212+0000 7fe7e21ef700 1 --2- 172.21.15.59:0/2259293385 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe7dc0ce5c0 0x7fe7dc0aed50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:57:29.228 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:29.212+0000 7fe7e19ee700 1 --2- 172.21.15.59:0/2259293385 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fe7dc0cf2b0 0x7fe7dc14cfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:57:29.229 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:29.213+0000 7fe7e29f0700 1 -- 172.21.15.59:0/2259293385 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fe7dc0cf2b0 msgr2=0x7fe7dc14cfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:57:29.229 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:29.213+0000 7fe7e29f0700 1 --2- 172.21.15.59:0/2259293385 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fe7dc0cf2b0 0x7fe7dc14cfb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:29.229 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:29.213+0000 7fe7e29f0700 1 -- 172.21.15.59:0/2259293385 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe7dc0ce5c0 msgr2=0x7fe7dc0aed50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:57:29.230 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:29.213+0000 7fe7e29f0700 1 --2- 172.21.15.59:0/2259293385 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe7dc0ce5c0 0x7fe7dc0aed50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:29.230 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:29.213+0000 7fe7e29f0700 1 -- 172.21.15.59:0/2259293385 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe7d80094a0 con 0x7fe7dc15bde0 2023-11-24T07:57:29.231 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:29.213+0000 7fe7e29f0700 1 --2- 172.21.15.59:0/2259293385 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe7dc15bde0 0x7fe7dc0b3b00 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7fe7d4001ba0 tx=0x7fe7d400cf50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:57:29.231 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:29.213+0000 7fe7d2ffd700 1 -- 172.21.15.59:0/2259293385 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7d400aac0 con 0x7fe7dc15bde0 2023-11-24T07:57:29.231 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:29.213+0000 7fe7d2ffd700 1 -- 172.21.15.59:0/2259293385 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe7d400ac20 con 0x7fe7dc15bde0 2023-11-24T07:57:29.232 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:29.213+0000 7fe7e4453700 1 -- 172.21.15.59:0/2259293385 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe7cc0094a0 con 0x7fe7dc15bde0 2023-11-24T07:57:29.232 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:29.213+0000 7fe7d2ffd700 1 -- 172.21.15.59:0/2259293385 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7d40070d0 con 0x7fe7dc15bde0 2023-11-24T07:57:29.232 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:29.213+0000 7fe7e4453700 1 -- 172.21.15.59:0/2259293385 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe7dc0afa80 con 0x7fe7dc15bde0 2023-11-24T07:57:29.232 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:29.214+0000 7fe7e4453700 1 -- 172.21.15.59:0/2259293385 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe7dc068670 con 0x7fe7dc15bde0 2023-11-24T07:57:29.233 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:29.215+0000 7fe7d2ffd700 1 -- 172.21.15.59:0/2259293385 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fe7d4007230 con 0x7fe7dc15bde0 2023-11-24T07:57:29.233 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:29.215+0000 7fe7d2ffd700 1 --2- 172.21.15.59:0/2259293385 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7fe7c006aed0 0x7fe7c006d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:57:29.233 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:29.215+0000 7fe7d2ffd700 1 -- 172.21.15.59:0/2259293385 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fe7d40893d0 con 0x7fe7dc15bde0 2023-11-24T07:57:29.233 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:29.215+0000 7fe7e21ef700 1 --2- 172.21.15.59:0/2259293385 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7fe7c006aed0 0x7fe7c006d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:57:29.234 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:29.216+0000 7fe7e21ef700 1 --2- 172.21.15.59:0/2259293385 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7fe7c006aed0 0x7fe7c006d390 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fe7cc0090b0 tx=0x7fe7cc008030 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:57:29.234 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:29.220+0000 7fe7d2ffd700 1 -- 172.21.15.59:0/2259293385 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fe7d4054510 con 0x7fe7dc15bde0 2023-11-24T07:57:29.423 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:29.421+0000 7fe7e4453700 1 -- 172.21.15.59:0/2259293385 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fe7dc0cfac0 con 0x7fe7dc15bde0 2023-11-24T07:57:29.424 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:29.422+0000 7fe7d2ffd700 1 -- 172.21.15.59:0/2259293385 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v33) v1 ==== 74+0+16437 (secure 0 0 0) 0x7fe7d4057b30 con 0x7fe7dc15bde0 2023-11-24T07:57:29.424 INFO:teuthology.orchestra.run.smithi059.stdout: 2023-11-24T07:57:29.425 INFO:teuthology.orchestra.run.smithi059.stdout:{"epoch":33,"fsid":"9f0e8741-b8f9-48d5-a2c3-09f45903303d","created":"2023-11-24T07:56:06.552729+0000","modified":"2023-11-24T07:57:26.579209+0000","last_up_change":"2023-11-24T07:56:17.372352+0000","last_in_change":"2023-11-24T07:56:09.073250+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-24T07:56:18.352823+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2023-11-24T07:57:23.014143+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"32","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2023-11-24T07:57:23.971487+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"33","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"6d8f9e9b-1782-4e41-928b-a39f2a8ebc06","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6800","nonce":3373878772},{"type":"v1","addr":"172.21.15.59:6801","nonce":3373878772}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6802","nonce":3373878772},{"type":"v1","addr":"172.21.15.59:6803","nonce":3373878772}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6806","nonce":3373878772},{"type":"v1","addr":"172.21.15.59:6807","nonce":3373878772}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6804","nonce":3373878772},{"type":"v1","addr":"172.21.15.59:6805","nonce":3373878772}]},"public_addr":"172.21.15.59:6801/3373878772","cluster_addr":"172.21.15.59:6803/3373878772","heartbeat_back_addr":"172.21.15.59:6807/3373878772","heartbeat_front_addr":"172.21.15.59:6805/3373878772","state":["exists","up"]},{"osd":1,"uuid":"e8d882aa-52e1-4c04-9e1e-c44ef9204a5f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6824","nonce":648258563},{"type":"v1","addr":"172.21.15.59:6825","nonce":648258563}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6826","nonce":648258563},{"type":"v1","addr":"172.21.15.59:6827","nonce":648258563}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6830","nonce":648258563},{"type":"v1","addr":"172.21.15.59:6831","nonce":648258563}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6828","nonce":648258563},{"type":"v1","addr":"172.21.15.59:6829","nonce":648258563}]},"public_addr":"172.21.15.59:6825/648258563","cluster_addr":"172.21.15.59:6827/648258563","heartbeat_back_addr":"172.21.15.59:6831/648258563","heartbeat_front_addr":"172.21.15.59:6829/648258563","state":["exists","up"]},{"osd":2,"uuid":"69eebfa0-695b-4b21-90b9-8a3537168a01","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6808","nonce":1840430654},{"type":"v1","addr":"172.21.15.59:6809","nonce":1840430654}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6810","nonce":1840430654},{"type":"v1","addr":"172.21.15.59:6811","nonce":1840430654}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6814","nonce":1840430654},{"type":"v1","addr":"172.21.15.59:6815","nonce":1840430654}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6812","nonce":1840430654},{"type":"v1","addr":"172.21.15.59:6813","nonce":1840430654}]},"public_addr":"172.21.15.59:6809/1840430654","cluster_addr":"172.21.15.59:6811/1840430654","heartbeat_back_addr":"172.21.15.59:6815/1840430654","heartbeat_front_addr":"172.21.15.59:6813/1840430654","state":["exists","up"]},{"osd":3,"uuid":"96fd77fc-f911-46ff-9445-6f0be2daa8f4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6816","nonce":2504025850},{"type":"v1","addr":"172.21.15.59:6817","nonce":2504025850}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6818","nonce":2504025850},{"type":"v1","addr":"172.21.15.59:6819","nonce":2504025850}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6822","nonce":2504025850},{"type":"v1","addr":"172.21.15.59:6823","nonce":2504025850}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6820","nonce":2504025850},{"type":"v1","addr":"172.21.15.59:6821","nonce":2504025850}]},"public_addr":"172.21.15.59:6817/2504025850","cluster_addr":"172.21.15.59:6819/2504025850","heartbeat_back_addr":"172.21.15.59:6823/2504025850","heartbeat_front_addr":"172.21.15.59:6821/2504025850","state":["exists","up"]},{"osd":4,"uuid":"7ac29fca-b5b2-4c95-9d69-2f164cbbc708","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6816","nonce":2382463431},{"type":"v1","addr":"172.21.15.76:6818","nonce":2382463431}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6820","nonce":2382463431},{"type":"v1","addr":"172.21.15.76:6822","nonce":2382463431}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6828","nonce":2382463431},{"type":"v1","addr":"172.21.15.76:6830","nonce":2382463431}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6824","nonce":2382463431},{"type":"v1","addr":"172.21.15.76:6826","nonce":2382463431}]},"public_addr":"172.21.15.76:6818/2382463431","cluster_addr":"172.21.15.76:6822/2382463431","heartbeat_back_addr":"172.21.15.76:6830/2382463431","heartbeat_front_addr":"172.21.15.76:6826/2382463431","state":["exists","up"]},{"osd":5,"uuid":"485bf711-3851-4238-a4f4-51d52e0a9c58","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6800","nonce":329334308},{"type":"v1","addr":"172.21.15.76:6802","nonce":329334308}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6805","nonce":329334308},{"type":"v1","addr":"172.21.15.76:6807","nonce":329334308}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6814","nonce":329334308},{"type":"v1","addr":"172.21.15.76:6815","nonce":329334308}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6810","nonce":329334308},{"type":"v1","addr":"172.21.15.76:6812","nonce":329334308}]},"public_addr":"172.21.15.76:6802/329334308","cluster_addr":"172.21.15.76:6807/329334308","heartbeat_back_addr":"172.21.15.76:6815/329334308","heartbeat_front_addr":"172.21.15.76:6812/329334308","state":["exists","up"]},{"osd":6,"uuid":"72b3f2fc-1e4a-4e32-968d-003647a0286f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6801","nonce":3946579410},{"type":"v1","addr":"172.21.15.76:6803","nonce":3946579410}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6804","nonce":3946579410},{"type":"v1","addr":"172.21.15.76:6806","nonce":3946579410}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6811","nonce":3946579410},{"type":"v1","addr":"172.21.15.76:6813","nonce":3946579410}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6808","nonce":3946579410},{"type":"v1","addr":"172.21.15.76:6809","nonce":3946579410}]},"public_addr":"172.21.15.76:6803/3946579410","cluster_addr":"172.21.15.76:6806/3946579410","heartbeat_back_addr":"172.21.15.76:6813/3946579410","heartbeat_front_addr":"172.21.15.76:6809/3946579410","state":["exists","up"]},{"osd":7,"uuid":"1ddceb90-6c5d-4d1a-ad56-99cb877f8c70","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6817","nonce":841761538},{"type":"v1","addr":"172.21.15.76:6819","nonce":841761538}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6821","nonce":841761538},{"type":"v1","addr":"172.21.15.76:6823","nonce":841761538}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6829","nonce":841761538},{"type":"v1","addr":"172.21.15.76:6831","nonce":841761538}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6825","nonce":841761538},{"type":"v1","addr":"172.21.15.76:6827","nonce":841761538}]},"public_addr":"172.21.15.76:6819/841761538","cluster_addr":"172.21.15.76:6823/841761538","heartbeat_back_addr":"172.21.15.76:6831/841761538","heartbeat_front_addr":"172.21.15.76:6827/841761538","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T07:56:14.294838+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T07:56:14.270779+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T07:56:14.231809+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T07:56:14.260983+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T07:56:14.302997+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T07:56:14.258600+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T07:56:14.251413+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T07:56:14.304727+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-24T07:57:29.427 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:29.425+0000 7fe7e4453700 1 -- 172.21.15.59:0/2259293385 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7fe7c006aed0 msgr2=0x7fe7c006d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:57:29.427 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:29.425+0000 7fe7e4453700 1 --2- 172.21.15.59:0/2259293385 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7fe7c006aed0 0x7fe7c006d390 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fe7cc0090b0 tx=0x7fe7cc008030 comp rx=0 tx=0).stop 2023-11-24T07:57:29.427 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:29.425+0000 7fe7e4453700 1 -- 172.21.15.59:0/2259293385 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe7dc15bde0 msgr2=0x7fe7dc0b3b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:57:29.427 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:29.425+0000 7fe7e4453700 1 --2- 172.21.15.59:0/2259293385 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe7dc15bde0 0x7fe7dc0b3b00 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7fe7d4001ba0 tx=0x7fe7d400cf50 comp rx=0 tx=0).stop 2023-11-24T07:57:29.428 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:29.425+0000 7fe7e4453700 1 -- 172.21.15.59:0/2259293385 shutdown_connections 2023-11-24T07:57:29.428 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:29.425+0000 7fe7e4453700 1 --2- 172.21.15.59:0/2259293385 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7fe7c006aed0 0x7fe7c006d390 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:29.428 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:29.425+0000 7fe7e4453700 1 --2- 172.21.15.59:0/2259293385 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe7dc15bde0 0x7fe7dc0b3b00 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:29.428 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:29.425+0000 7fe7e4453700 1 --2- 172.21.15.59:0/2259293385 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe7dc0ce5c0 0x7fe7dc0aed50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:29.428 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:29.425+0000 7fe7e4453700 1 --2- 172.21.15.59:0/2259293385 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fe7dc0cf2b0 0x7fe7dc14cfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:29.429 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:29.425+0000 7fe7e4453700 1 -- 172.21.15.59:0/2259293385 >> 172.21.15.59:0/2259293385 conn(0x7fe7dc0b3720 msgr2=0x7fe7dc15aff0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:57:29.429 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:29.425+0000 7fe7e4453700 1 -- 172.21.15.59:0/2259293385 shutdown_connections 2023-11-24T07:57:29.429 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:29.425+0000 7fe7e4453700 1 -- 172.21.15.59:0/2259293385 wait complete. 2023-11-24T07:57:29.440 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2023-11-24T07:57:29.442 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-11-24T07:57:29.477 INFO:tasks.ceph.mds.a.smithi059.stderr:2023-11-24T07:57:29.475+0000 7faa67f73700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-11-24T07:57:29.566 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:29.564+0000 7fc6b66a7700 1 Processor -- start 2023-11-24T07:57:29.566 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:29.564+0000 7fc6b66a7700 1 -- start start 2023-11-24T07:57:29.568 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:29.565+0000 7fc6b66a7700 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fc6b00726a0 0x7fc6b0072a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:57:29.568 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:29.565+0000 7fc6b66a7700 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc6b015f3e0 con 0x7fc6b00b3d90 2023-11-24T07:57:29.569 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:29.565+0000 7fc6b66a7700 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc6b006c8e0 con 0x7fc6b0072fc0 2023-11-24T07:57:29.569 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:29.565+0000 7fc6b66a7700 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc6b00affd0 con 0x7fc6b00726a0 2023-11-24T07:57:29.569 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:29.565+0000 7fc6af7fe700 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7fc6b00b3d90 0x7fc6b006bac0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:55228/0 (socket says 172.21.15.59:55228) 2023-11-24T07:57:29.569 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:29.565+0000 7fc6af7fe700 1 -- 172.21.15.59:0/777665260 learned_addr learned my addr 172.21.15.59:0/777665260 (peer_addr_for_me v1:172.21.15.59:0/0) 2023-11-24T07:57:29.569 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:29.566+0000 7fc6affff700 1 --2- 172.21.15.59:0/777665260 >> v2:172.21.15.76:3300/0 conn(0x7fc6b00726a0 0x7fc6b0072a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:57:29.569 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:29.566+0000 7fc6aeffd700 1 -- 172.21.15.59:0/777665260 <== mon.0 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 285388391 0 0) 0x7fc6b015f3e0 con 0x7fc6b00b3d90 2023-11-24T07:57:29.569 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:29.566+0000 7fc6aeffd700 1 -- 172.21.15.59:0/777665260 --> v1:172.21.15.59:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc69c0035b0 con 0x7fc6b00b3d90 2023-11-24T07:57:29.569 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:29.566+0000 7fc6aeffd700 1 -- 172.21.15.59:0/777665260 <== mon.1 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2528188951 0 0) 0x7fc6b006c8e0 con 0x7fc6b0072fc0 2023-11-24T07:57:29.569 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:29.566+0000 7fc6aeffd700 1 -- 172.21.15.59:0/777665260 --> v1:172.21.15.76:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc6b015f3e0 con 0x7fc6b0072fc0 2023-11-24T07:57:29.569 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:29.566+0000 7fc6affff700 1 -- 172.21.15.59:0/777665260 >> v1:172.21.15.76:6789/0 conn(0x7fc6b0072fc0 legacy=0x7fc6b006c1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:57:29.569 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:29.566+0000 7fc6affff700 1 -- 172.21.15.59:0/777665260 >> v1:172.21.15.59:6789/0 conn(0x7fc6b00b3d90 legacy=0x7fc6b006bac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:57:29.569 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:29.566+0000 7fc6affff700 1 -- 172.21.15.59:0/777665260 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc6b015e7d0 con 0x7fc6b00726a0 2023-11-24T07:57:29.570 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:29.566+0000 7fc6affff700 1 --2- 172.21.15.59:0/777665260 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc6b00726a0 0x7fc6b0072a80 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fc6a0009870 tx=0x7fc6a0009b80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2fbe2ecb9eec5d98 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:57:29.570 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:29.566+0000 7fc6aeffd700 1 -- 172.21.15.59:0/777665260 <== mon.1 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6a0011830 con 0x7fc6b00726a0 2023-11-24T07:57:29.570 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:29.566+0000 7fc6aeffd700 1 -- 172.21.15.59:0/777665260 <== mon.1 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc6a0011990 con 0x7fc6b00726a0 2023-11-24T07:57:29.570 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:29.566+0000 7fc6aeffd700 1 -- 172.21.15.59:0/777665260 <== mon.1 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6a0011d20 con 0x7fc6b00726a0 2023-11-24T07:57:29.570 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:29.567+0000 7fc6b66a7700 1 -- 172.21.15.59:0/777665260 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc6b00726a0 msgr2=0x7fc6b0072a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:57:29.570 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:29.567+0000 7fc6b66a7700 1 --2- 172.21.15.59:0/777665260 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc6b00726a0 0x7fc6b0072a80 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fc6a0009870 tx=0x7fc6a0009b80 comp rx=0 tx=0).stop 2023-11-24T07:57:29.570 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:29.567+0000 7fc6b66a7700 1 -- 172.21.15.59:0/777665260 shutdown_connections 2023-11-24T07:57:29.570 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:29.567+0000 7fc6b66a7700 1 --2- 172.21.15.59:0/777665260 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc6b00726a0 0x7fc6b0072a80 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:29.570 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:29.567+0000 7fc6b66a7700 1 -- 172.21.15.59:0/777665260 >> 172.21.15.59:0/777665260 conn(0x7fc6b00717f0 msgr2=0x7fc6b00b5070 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:57:29.570 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:29.567+0000 7fc6b66a7700 1 -- 172.21.15.59:0/777665260 shutdown_connections 2023-11-24T07:57:29.570 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:29.567+0000 7fc6b66a7700 1 -- 172.21.15.59:0/777665260 wait complete. 2023-11-24T07:57:29.572 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:29.568+0000 7fc6b66a7700 1 Processor -- start 2023-11-24T07:57:29.573 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:29.568+0000 7fc6b66a7700 1 -- start start 2023-11-24T07:57:29.573 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:29.569+0000 7fc6b66a7700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc6b00726a0 0x7fc6b0153ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:57:29.573 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:29.569+0000 7fc6b66a7700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc6b0072fc0 0x7fc6b0155310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:57:29.573 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:29.569+0000 7fc6b66a7700 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fc6b00b3d90 0x7fc6b0156b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:57:29.573 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:29.569+0000 7fc6b66a7700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6b0156700 con 0x7fc6b0072fc0 2023-11-24T07:57:29.573 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:29.569+0000 7fc6b66a7700 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6b0155850 con 0x7fc6b00726a0 2023-11-24T07:57:29.573 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:29.569+0000 7fc6b66a7700 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc6b0155af0 con 0x7fc6b00b3d90 2023-11-24T07:57:29.573 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:29.569+0000 7fc6af7fe700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc6b0072fc0 0x7fc6b0155310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:57:29.573 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:29.569+0000 7fc6affff700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc6b00726a0 0x7fc6b0153ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:57:29.573 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:29.569+0000 7fc6af7fe700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc6b0072fc0 0x7fc6b0155310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:51760/0 (socket says 172.21.15.59:51760) 2023-11-24T07:57:29.573 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:29.569+0000 7fc6af7fe700 1 -- 172.21.15.59:0/1688203614 learned_addr learned my addr 172.21.15.59:0/1688203614 (peer_addr_for_me v2:172.21.15.59:0/0) 2023-11-24T07:57:29.574 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:29.569+0000 7fc6b4c44700 1 --2- 172.21.15.59:0/1688203614 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fc6b00b3d90 0x7fc6b0156b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:57:29.574 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:29.569+0000 7fc6af7fe700 1 -- 172.21.15.59:0/1688203614 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fc6b00b3d90 msgr2=0x7fc6b0156b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:57:29.574 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:29.569+0000 7fc6af7fe700 1 --2- 172.21.15.59:0/1688203614 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fc6b00b3d90 0x7fc6b0156b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:29.574 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:29.569+0000 7fc6af7fe700 1 -- 172.21.15.59:0/1688203614 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc6b00726a0 msgr2=0x7fc6b0153ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:57:29.574 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:29.569+0000 7fc6af7fe700 1 --2- 172.21.15.59:0/1688203614 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc6b00726a0 0x7fc6b0153ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:29.574 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:29.569+0000 7fc6af7fe700 1 -- 172.21.15.59:0/1688203614 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc6a00094a0 con 0x7fc6b0072fc0 2023-11-24T07:57:29.574 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:29.569+0000 7fc6affff700 1 --2- 172.21.15.59:0/1688203614 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc6b00726a0 0x7fc6b0153ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-24T07:57:29.574 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:29.570+0000 7fc6af7fe700 1 --2- 172.21.15.59:0/1688203614 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc6b0072fc0 0x7fc6b0155310 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7fc698001eb0 tx=0x7fc69800c7b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:57:29.574 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:29.570+0000 7fc6acff9700 1 -- 172.21.15.59:0/1688203614 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc698005d70 con 0x7fc6b0072fc0 2023-11-24T07:57:29.574 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:29.570+0000 7fc6acff9700 1 -- 172.21.15.59:0/1688203614 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc6980070d0 con 0x7fc6b0072fc0 2023-11-24T07:57:29.574 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:29.570+0000 7fc6b66a7700 1 -- 172.21.15.59:0/1688203614 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc6b0155dd0 con 0x7fc6b0072fc0 2023-11-24T07:57:29.574 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:29.570+0000 7fc6acff9700 1 -- 172.21.15.59:0/1688203614 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc698007460 con 0x7fc6b0072fc0 2023-11-24T07:57:29.574 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:29.570+0000 7fc6b66a7700 1 -- 172.21.15.59:0/1688203614 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc6b0156390 con 0x7fc6b0072fc0 2023-11-24T07:57:29.575 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:29.571+0000 7fc6b66a7700 1 -- 172.21.15.59:0/1688203614 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc6b0156550 con 0x7fc6b0072fc0 2023-11-24T07:57:29.575 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:29.571+0000 7fc6acff9700 1 -- 172.21.15.59:0/1688203614 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fc6980075c0 con 0x7fc6b0072fc0 2023-11-24T07:57:29.576 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:29.572+0000 7fc6acff9700 1 --2- 172.21.15.59:0/1688203614 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7fc69006aed0 0x7fc69006d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:57:29.576 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:29.572+0000 7fc6acff9700 1 -- 172.21.15.59:0/1688203614 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fc698088a80 con 0x7fc6b0072fc0 2023-11-24T07:57:29.576 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:29.572+0000 7fc6affff700 1 --2- 172.21.15.59:0/1688203614 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7fc69006aed0 0x7fc69006d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:57:29.576 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:29.572+0000 7fc6affff700 1 --2- 172.21.15.59:0/1688203614 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7fc69006aed0 0x7fc69006d390 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fc6a00090b0 tx=0x7fc6a001c040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:57:29.578 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:29.576+0000 7fc6acff9700 1 -- 172.21.15.59:0/1688203614 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fc698053a90 con 0x7fc6b0072fc0 2023-11-24T07:57:29.774 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:29.773+0000 7fc6b66a7700 1 -- 172.21.15.59:0/1688203614 --> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fc6b015f070 con 0x7fc69006aed0 2023-11-24T07:57:29.783 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:29.782+0000 7fc6acff9700 1 -- 172.21.15.59:0/1688203614 <== mgr.4100 v2:172.21.15.76:6832/4264046536 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+274252 (secure 0 0 0) 0x7fc6a0011b00 con 0x7fc69006aed0 2023-11-24T07:57:29.787 INFO:teuthology.orchestra.run.smithi059.stdout: 2023-11-24T07:57:29.789 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:29.784+0000 7fc6b66a7700 1 -- 172.21.15.59:0/1688203614 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7fc69006aed0 msgr2=0x7fc69006d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:57:29.789 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:29.784+0000 7fc6b66a7700 1 --2- 172.21.15.59:0/1688203614 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7fc69006aed0 0x7fc69006d390 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fc6a00090b0 tx=0x7fc6a001c040 comp rx=0 tx=0).stop 2023-11-24T07:57:29.789 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:29.784+0000 7fc6b66a7700 1 -- 172.21.15.59:0/1688203614 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc6b0072fc0 msgr2=0x7fc6b0155310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:57:29.789 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:29.784+0000 7fc6b66a7700 1 --2- 172.21.15.59:0/1688203614 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc6b0072fc0 0x7fc6b0155310 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7fc698001eb0 tx=0x7fc69800c7b0 comp rx=0 tx=0).stop 2023-11-24T07:57:29.789 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:29.784+0000 7fc6b66a7700 1 -- 172.21.15.59:0/1688203614 shutdown_connections 2023-11-24T07:57:29.789 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:29.784+0000 7fc6b66a7700 1 --2- 172.21.15.59:0/1688203614 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7fc69006aed0 0x7fc69006d390 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:29.789 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:29.784+0000 7fc6b66a7700 1 --2- 172.21.15.59:0/1688203614 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fc6b00b3d90 0x7fc6b0156b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:29.789 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:29.784+0000 7fc6b66a7700 1 --2- 172.21.15.59:0/1688203614 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc6b00726a0 0x7fc6b0153ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:29.790 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:29.784+0000 7fc6b66a7700 1 --2- 172.21.15.59:0/1688203614 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc6b0072fc0 0x7fc6b0155310 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:29.790 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:29.784+0000 7fc6b66a7700 1 -- 172.21.15.59:0/1688203614 >> 172.21.15.59:0/1688203614 conn(0x7fc6b00717f0 msgr2=0x7fc6b006d350 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:57:29.790 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:29.785+0000 7fc6b66a7700 1 -- 172.21.15.59:0/1688203614 shutdown_connections 2023-11-24T07:57:29.790 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:29.785+0000 7fc6b66a7700 1 -- 172.21.15.59:0/1688203614 wait complete. 2023-11-24T07:57:29.790 INFO:teuthology.orchestra.run.smithi059.stderr:dumped all 2023-11-24T07:57:29.798 INFO:teuthology.orchestra.run.smithi059.stdout:{"pg_ready":true,"pg_map":{"version":61,"stamp":"2023-11-24T07:57:28.313566+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":2,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":2,"kb":754974720,"kb_used":219892,"kb_used_data":3688,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754754828,"statfs":{"total":773094113280,"available":772868943872,"internally_reserved":0,"allocated":3776512,"data_stored":2148184,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12745,"internal_metadata":221236791},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.740627"},"pg_stats":[{"pgid":"5.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T07:57:24.572681+0000","last_change":"2023-11-24T07:57:24.572681+0000","last_active":"2023-11-24T07:57:24.572681+0000","last_peered":"2023-11-24T07:57:24.572681+0000","last_clean":"2023-11-24T07:57:24.572681+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T07:57:24.572681+0000","last_undegraded":"2023-11-24T07:57:24.572681+0000","last_fullsized":"2023-11-24T07:57:24.572681+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:57:24.572681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:57:24.572681+0000","last_clean_scrub_stamp":"2023-11-24T07:57:24.572681+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T07:57:24.572681+0000","last_change":"2023-11-24T07:57:24.572681+0000","last_active":"2023-11-24T07:57:24.572681+0000","last_peered":"2023-11-24T07:57:24.572681+0000","last_clean":"2023-11-24T07:57:24.572681+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T07:57:24.572681+0000","last_undegraded":"2023-11-24T07:57:24.572681+0000","last_fullsized":"2023-11-24T07:57:24.572681+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:57:24.572681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:57:24.572681+0000","last_clean_scrub_stamp":"2023-11-24T07:57:24.572681+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T07:57:24.572681+0000","last_change":"2023-11-24T07:57:24.572681+0000","last_active":"2023-11-24T07:57:24.572681+0000","last_peered":"2023-11-24T07:57:24.572681+0000","last_clean":"2023-11-24T07:57:24.572681+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T07:57:24.572681+0000","last_undegraded":"2023-11-24T07:57:24.572681+0000","last_fullsized":"2023-11-24T07:57:24.572681+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:57:24.572681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:57:24.572681+0000","last_clean_scrub_stamp":"2023-11-24T07:57:24.572681+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T07:57:24.572681+0000","last_change":"2023-11-24T07:57:24.572681+0000","last_active":"2023-11-24T07:57:24.572681+0000","last_peered":"2023-11-24T07:57:24.572681+0000","last_clean":"2023-11-24T07:57:24.572681+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T07:57:24.572681+0000","last_undegraded":"2023-11-24T07:57:24.572681+0000","last_fullsized":"2023-11-24T07:57:24.572681+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:57:24.572681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:57:24.572681+0000","last_clean_scrub_stamp":"2023-11-24T07:57:24.572681+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T07:57:24.572681+0000","last_change":"2023-11-24T07:57:24.572681+0000","last_active":"2023-11-24T07:57:24.572681+0000","last_peered":"2023-11-24T07:57:24.572681+0000","last_clean":"2023-11-24T07:57:24.572681+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T07:57:24.572681+0000","last_undegraded":"2023-11-24T07:57:24.572681+0000","last_fullsized":"2023-11-24T07:57:24.572681+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:57:24.572681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:57:24.572681+0000","last_clean_scrub_stamp":"2023-11-24T07:57:24.572681+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T07:57:24.572681+0000","last_change":"2023-11-24T07:57:24.572681+0000","last_active":"2023-11-24T07:57:24.572681+0000","last_peered":"2023-11-24T07:57:24.572681+0000","last_clean":"2023-11-24T07:57:24.572681+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T07:57:24.572681+0000","last_undegraded":"2023-11-24T07:57:24.572681+0000","last_fullsized":"2023-11-24T07:57:24.572681+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:57:24.572681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:57:24.572681+0000","last_clean_scrub_stamp":"2023-11-24T07:57:24.572681+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T07:57:24.572681+0000","last_change":"2023-11-24T07:57:24.572681+0000","last_active":"2023-11-24T07:57:24.572681+0000","last_peered":"2023-11-24T07:57:24.572681+0000","last_clean":"2023-11-24T07:57:24.572681+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T07:57:24.572681+0000","last_undegraded":"2023-11-24T07:57:24.572681+0000","last_fullsized":"2023-11-24T07:57:24.572681+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:57:24.572681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:57:24.572681+0000","last_clean_scrub_stamp":"2023-11-24T07:57:24.572681+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T07:57:24.572681+0000","last_change":"2023-11-24T07:57:24.572681+0000","last_active":"2023-11-24T07:57:24.572681+0000","last_peered":"2023-11-24T07:57:24.572681+0000","last_clean":"2023-11-24T07:57:24.572681+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T07:57:24.572681+0000","last_undegraded":"2023-11-24T07:57:24.572681+0000","last_fullsized":"2023-11-24T07:57:24.572681+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:57:24.572681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:57:24.572681+0000","last_clean_scrub_stamp":"2023-11-24T07:57:24.572681+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T07:57:24.572681+0000","last_change":"2023-11-24T07:57:24.572681+0000","last_active":"2023-11-24T07:57:24.572681+0000","last_peered":"2023-11-24T07:57:24.572681+0000","last_clean":"2023-11-24T07:57:24.572681+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T07:57:24.572681+0000","last_undegraded":"2023-11-24T07:57:24.572681+0000","last_fullsized":"2023-11-24T07:57:24.572681+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:57:24.572681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:57:24.572681+0000","last_clean_scrub_stamp":"2023-11-24T07:57:24.572681+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T07:57:24.572681+0000","last_change":"2023-11-24T07:57:24.572681+0000","last_active":"2023-11-24T07:57:24.572681+0000","last_peered":"2023-11-24T07:57:24.572681+0000","last_clean":"2023-11-24T07:57:24.572681+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T07:57:24.572681+0000","last_undegraded":"2023-11-24T07:57:24.572681+0000","last_fullsized":"2023-11-24T07:57:24.572681+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:57:24.572681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:57:24.572681+0000","last_clean_scrub_stamp":"2023-11-24T07:57:24.572681+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T07:57:24.572681+0000","last_change":"2023-11-24T07:57:24.572681+0000","last_active":"2023-11-24T07:57:24.572681+0000","last_peered":"2023-11-24T07:57:24.572681+0000","last_clean":"2023-11-24T07:57:24.572681+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T07:57:24.572681+0000","last_undegraded":"2023-11-24T07:57:24.572681+0000","last_fullsized":"2023-11-24T07:57:24.572681+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:57:24.572681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:57:24.572681+0000","last_clean_scrub_stamp":"2023-11-24T07:57:24.572681+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T07:57:24.572681+0000","last_change":"2023-11-24T07:57:24.572681+0000","last_active":"2023-11-24T07:57:24.572681+0000","last_peered":"2023-11-24T07:57:24.572681+0000","last_clean":"2023-11-24T07:57:24.572681+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T07:57:24.572681+0000","last_undegraded":"2023-11-24T07:57:24.572681+0000","last_fullsized":"2023-11-24T07:57:24.572681+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:57:24.572681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:57:24.572681+0000","last_clean_scrub_stamp":"2023-11-24T07:57:24.572681+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T07:57:24.572681+0000","last_change":"2023-11-24T07:57:24.572681+0000","last_active":"2023-11-24T07:57:24.572681+0000","last_peered":"2023-11-24T07:57:24.572681+0000","last_clean":"2023-11-24T07:57:24.572681+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T07:57:24.572681+0000","last_undegraded":"2023-11-24T07:57:24.572681+0000","last_fullsized":"2023-11-24T07:57:24.572681+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:57:24.572681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:57:24.572681+0000","last_clean_scrub_stamp":"2023-11-24T07:57:24.572681+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T07:57:24.572681+0000","last_change":"2023-11-24T07:57:24.572681+0000","last_active":"2023-11-24T07:57:24.572681+0000","last_peered":"2023-11-24T07:57:24.572681+0000","last_clean":"2023-11-24T07:57:24.572681+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T07:57:24.572681+0000","last_undegraded":"2023-11-24T07:57:24.572681+0000","last_fullsized":"2023-11-24T07:57:24.572681+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:57:24.572681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:57:24.572681+0000","last_clean_scrub_stamp":"2023-11-24T07:57:24.572681+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T07:57:24.572681+0000","last_change":"2023-11-24T07:57:24.572681+0000","last_active":"2023-11-24T07:57:24.572681+0000","last_peered":"2023-11-24T07:57:24.572681+0000","last_clean":"2023-11-24T07:57:24.572681+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T07:57:24.572681+0000","last_undegraded":"2023-11-24T07:57:24.572681+0000","last_fullsized":"2023-11-24T07:57:24.572681+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:57:24.572681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:57:24.572681+0000","last_clean_scrub_stamp":"2023-11-24T07:57:24.572681+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T07:57:24.572681+0000","last_change":"2023-11-24T07:57:24.572681+0000","last_active":"2023-11-24T07:57:24.572681+0000","last_peered":"2023-11-24T07:57:24.572681+0000","last_clean":"2023-11-24T07:57:24.572681+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T07:57:24.572681+0000","last_undegraded":"2023-11-24T07:57:24.572681+0000","last_fullsized":"2023-11-24T07:57:24.572681+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:57:24.572681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:57:24.572681+0000","last_clean_scrub_stamp":"2023-11-24T07:57:24.572681+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T07:57:24.572681+0000","last_change":"2023-11-24T07:57:24.572681+0000","last_active":"2023-11-24T07:57:24.572681+0000","last_peered":"2023-11-24T07:57:24.572681+0000","last_clean":"2023-11-24T07:57:24.572681+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T07:57:24.572681+0000","last_undegraded":"2023-11-24T07:57:24.572681+0000","last_fullsized":"2023-11-24T07:57:24.572681+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:57:24.572681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:57:24.572681+0000","last_clean_scrub_stamp":"2023-11-24T07:57:24.572681+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T07:57:24.572681+0000","last_change":"2023-11-24T07:57:24.572681+0000","last_active":"2023-11-24T07:57:24.572681+0000","last_peered":"2023-11-24T07:57:24.572681+0000","last_clean":"2023-11-24T07:57:24.572681+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T07:57:24.572681+0000","last_undegraded":"2023-11-24T07:57:24.572681+0000","last_fullsized":"2023-11-24T07:57:24.572681+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:57:24.572681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:57:24.572681+0000","last_clean_scrub_stamp":"2023-11-24T07:57:24.572681+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T07:57:24.572681+0000","last_change":"2023-11-24T07:57:24.572681+0000","last_active":"2023-11-24T07:57:24.572681+0000","last_peered":"2023-11-24T07:57:24.572681+0000","last_clean":"2023-11-24T07:57:24.572681+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T07:57:24.572681+0000","last_undegraded":"2023-11-24T07:57:24.572681+0000","last_fullsized":"2023-11-24T07:57:24.572681+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:57:24.572681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:57:24.572681+0000","last_clean_scrub_stamp":"2023-11-24T07:57:24.572681+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T07:57:24.572681+0000","last_change":"2023-11-24T07:57:24.572681+0000","last_active":"2023-11-24T07:57:24.572681+0000","last_peered":"2023-11-24T07:57:24.572681+0000","last_clean":"2023-11-24T07:57:24.572681+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T07:57:24.572681+0000","last_undegraded":"2023-11-24T07:57:24.572681+0000","last_fullsized":"2023-11-24T07:57:24.572681+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:57:24.572681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:57:24.572681+0000","last_clean_scrub_stamp":"2023-11-24T07:57:24.572681+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T07:57:24.572681+0000","last_change":"2023-11-24T07:57:24.572681+0000","last_active":"2023-11-24T07:57:24.572681+0000","last_peered":"2023-11-24T07:57:24.572681+0000","last_clean":"2023-11-24T07:57:24.572681+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T07:57:24.572681+0000","last_undegraded":"2023-11-24T07:57:24.572681+0000","last_fullsized":"2023-11-24T07:57:24.572681+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:57:24.572681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:57:24.572681+0000","last_clean_scrub_stamp":"2023-11-24T07:57:24.572681+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T07:57:24.572681+0000","last_change":"2023-11-24T07:57:24.572681+0000","last_active":"2023-11-24T07:57:24.572681+0000","last_peered":"2023-11-24T07:57:24.572681+0000","last_clean":"2023-11-24T07:57:24.572681+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T07:57:24.572681+0000","last_undegraded":"2023-11-24T07:57:24.572681+0000","last_fullsized":"2023-11-24T07:57:24.572681+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:57:24.572681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:57:24.572681+0000","last_clean_scrub_stamp":"2023-11-24T07:57:24.572681+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T07:57:24.572681+0000","last_change":"2023-11-24T07:57:24.572681+0000","last_active":"2023-11-24T07:57:24.572681+0000","last_peered":"2023-11-24T07:57:24.572681+0000","last_clean":"2023-11-24T07:57:24.572681+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T07:57:24.572681+0000","last_undegraded":"2023-11-24T07:57:24.572681+0000","last_fullsized":"2023-11-24T07:57:24.572681+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:57:24.572681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:57:24.572681+0000","last_clean_scrub_stamp":"2023-11-24T07:57:24.572681+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T07:57:24.572681+0000","last_change":"2023-11-24T07:57:24.572681+0000","last_active":"2023-11-24T07:57:24.572681+0000","last_peered":"2023-11-24T07:57:24.572681+0000","last_clean":"2023-11-24T07:57:24.572681+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T07:57:24.572681+0000","last_undegraded":"2023-11-24T07:57:24.572681+0000","last_fullsized":"2023-11-24T07:57:24.572681+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:57:24.572681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:57:24.572681+0000","last_clean_scrub_stamp":"2023-11-24T07:57:24.572681+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T07:57:24.572681+0000","last_change":"2023-11-24T07:57:24.572681+0000","last_active":"2023-11-24T07:57:24.572681+0000","last_peered":"2023-11-24T07:57:24.572681+0000","last_clean":"2023-11-24T07:57:24.572681+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T07:57:24.572681+0000","last_undegraded":"2023-11-24T07:57:24.572681+0000","last_fullsized":"2023-11-24T07:57:24.572681+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:57:24.572681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:57:24.572681+0000","last_clean_scrub_stamp":"2023-11-24T07:57:24.572681+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T07:57:24.572681+0000","last_change":"2023-11-24T07:57:24.572681+0000","last_active":"2023-11-24T07:57:24.572681+0000","last_peered":"2023-11-24T07:57:24.572681+0000","last_clean":"2023-11-24T07:57:24.572681+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T07:57:24.572681+0000","last_undegraded":"2023-11-24T07:57:24.572681+0000","last_fullsized":"2023-11-24T07:57:24.572681+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:57:24.572681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:57:24.572681+0000","last_clean_scrub_stamp":"2023-11-24T07:57:24.572681+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T07:57:24.572681+0000","last_change":"2023-11-24T07:57:24.572681+0000","last_active":"2023-11-24T07:57:24.572681+0000","last_peered":"2023-11-24T07:57:24.572681+0000","last_clean":"2023-11-24T07:57:24.572681+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T07:57:24.572681+0000","last_undegraded":"2023-11-24T07:57:24.572681+0000","last_fullsized":"2023-11-24T07:57:24.572681+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:57:24.572681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:57:24.572681+0000","last_clean_scrub_stamp":"2023-11-24T07:57:24.572681+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T07:57:24.572681+0000","last_change":"2023-11-24T07:57:24.572681+0000","last_active":"2023-11-24T07:57:24.572681+0000","last_peered":"2023-11-24T07:57:24.572681+0000","last_clean":"2023-11-24T07:57:24.572681+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T07:57:24.572681+0000","last_undegraded":"2023-11-24T07:57:24.572681+0000","last_fullsized":"2023-11-24T07:57:24.572681+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:57:24.572681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:57:24.572681+0000","last_clean_scrub_stamp":"2023-11-24T07:57:24.572681+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T07:57:24.572681+0000","last_change":"2023-11-24T07:57:24.572681+0000","last_active":"2023-11-24T07:57:24.572681+0000","last_peered":"2023-11-24T07:57:24.572681+0000","last_clean":"2023-11-24T07:57:24.572681+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T07:57:24.572681+0000","last_undegraded":"2023-11-24T07:57:24.572681+0000","last_fullsized":"2023-11-24T07:57:24.572681+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:57:24.572681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:57:24.572681+0000","last_clean_scrub_stamp":"2023-11-24T07:57:24.572681+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T07:57:24.572681+0000","last_change":"2023-11-24T07:57:24.572681+0000","last_active":"2023-11-24T07:57:24.572681+0000","last_peered":"2023-11-24T07:57:24.572681+0000","last_clean":"2023-11-24T07:57:24.572681+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T07:57:24.572681+0000","last_undegraded":"2023-11-24T07:57:24.572681+0000","last_fullsized":"2023-11-24T07:57:24.572681+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:57:24.572681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:57:24.572681+0000","last_clean_scrub_stamp":"2023-11-24T07:57:24.572681+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T07:57:24.572681+0000","last_change":"2023-11-24T07:57:24.572681+0000","last_active":"2023-11-24T07:57:24.572681+0000","last_peered":"2023-11-24T07:57:24.572681+0000","last_clean":"2023-11-24T07:57:24.572681+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T07:57:24.572681+0000","last_undegraded":"2023-11-24T07:57:24.572681+0000","last_fullsized":"2023-11-24T07:57:24.572681+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:57:24.572681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:57:24.572681+0000","last_clean_scrub_stamp":"2023-11-24T07:57:24.572681+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T07:57:24.572681+0000","last_change":"2023-11-24T07:57:24.572681+0000","last_active":"2023-11-24T07:57:24.572681+0000","last_peered":"2023-11-24T07:57:24.572681+0000","last_clean":"2023-11-24T07:57:24.572681+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T07:57:24.572681+0000","last_undegraded":"2023-11-24T07:57:24.572681+0000","last_fullsized":"2023-11-24T07:57:24.572681+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:57:24.572681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:57:24.572681+0000","last_clean_scrub_stamp":"2023-11-24T07:57:24.572681+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T07:57:24.572681+0000","last_change":"2023-11-24T07:57:24.572681+0000","last_active":"2023-11-24T07:57:24.572681+0000","last_peered":"2023-11-24T07:57:24.572681+0000","last_clean":"2023-11-24T07:57:24.572681+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T07:57:24.572681+0000","last_undegraded":"2023-11-24T07:57:24.572681+0000","last_fullsized":"2023-11-24T07:57:24.572681+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:57:24.572681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:57:24.572681+0000","last_clean_scrub_stamp":"2023-11-24T07:57:24.572681+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T07:57:23.565116+0000","last_change":"2023-11-24T07:57:23.565116+0000","last_active":"2023-11-24T07:57:23.565116+0000","last_peered":"2023-11-24T07:57:23.565116+0000","last_clean":"2023-11-24T07:57:23.565116+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T07:57:23.565116+0000","last_undegraded":"2023-11-24T07:57:23.565116+0000","last_fullsized":"2023-11-24T07:57:23.565116+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:57:23.565116+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:57:23.565116+0000","last_clean_scrub_stamp":"2023-11-24T07:57:23.565116+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T07:57:24.572681+0000","last_change":"2023-11-24T07:57:24.572681+0000","last_active":"2023-11-24T07:57:24.572681+0000","last_peered":"2023-11-24T07:57:24.572681+0000","last_clean":"2023-11-24T07:57:24.572681+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T07:57:24.572681+0000","last_undegraded":"2023-11-24T07:57:24.572681+0000","last_fullsized":"2023-11-24T07:57:24.572681+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:57:24.572681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:57:24.572681+0000","last_clean_scrub_stamp":"2023-11-24T07:57:24.572681+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T07:57:23.565116+0000","last_change":"2023-11-24T07:57:23.565116+0000","last_active":"2023-11-24T07:57:23.565116+0000","last_peered":"2023-11-24T07:57:23.565116+0000","last_clean":"2023-11-24T07:57:23.565116+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T07:57:23.565116+0000","last_undegraded":"2023-11-24T07:57:23.565116+0000","last_fullsized":"2023-11-24T07:57:23.565116+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:57:23.565116+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:57:23.565116+0000","last_clean_scrub_stamp":"2023-11-24T07:57:23.565116+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T07:57:24.572681+0000","last_change":"2023-11-24T07:57:24.572681+0000","last_active":"2023-11-24T07:57:24.572681+0000","last_peered":"2023-11-24T07:57:24.572681+0000","last_clean":"2023-11-24T07:57:24.572681+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T07:57:24.572681+0000","last_undegraded":"2023-11-24T07:57:24.572681+0000","last_fullsized":"2023-11-24T07:57:24.572681+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:57:24.572681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:57:24.572681+0000","last_clean_scrub_stamp":"2023-11-24T07:57:24.572681+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T07:57:23.565116+0000","last_change":"2023-11-24T07:57:23.565116+0000","last_active":"2023-11-24T07:57:23.565116+0000","last_peered":"2023-11-24T07:57:23.565116+0000","last_clean":"2023-11-24T07:57:23.565116+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T07:57:23.565116+0000","last_undegraded":"2023-11-24T07:57:23.565116+0000","last_fullsized":"2023-11-24T07:57:23.565116+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:57:23.565116+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:57:23.565116+0000","last_clean_scrub_stamp":"2023-11-24T07:57:23.565116+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T07:57:24.572681+0000","last_change":"2023-11-24T07:57:24.572681+0000","last_active":"2023-11-24T07:57:24.572681+0000","last_peered":"2023-11-24T07:57:24.572681+0000","last_clean":"2023-11-24T07:57:24.572681+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T07:57:24.572681+0000","last_undegraded":"2023-11-24T07:57:24.572681+0000","last_fullsized":"2023-11-24T07:57:24.572681+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:57:24.572681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:57:24.572681+0000","last_clean_scrub_stamp":"2023-11-24T07:57:24.572681+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T07:57:23.565116+0000","last_change":"2023-11-24T07:57:23.565116+0000","last_active":"2023-11-24T07:57:23.565116+0000","last_peered":"2023-11-24T07:57:23.565116+0000","last_clean":"2023-11-24T07:57:23.565116+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T07:57:23.565116+0000","last_undegraded":"2023-11-24T07:57:23.565116+0000","last_fullsized":"2023-11-24T07:57:23.565116+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:57:23.565116+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:57:23.565116+0000","last_clean_scrub_stamp":"2023-11-24T07:57:23.565116+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T07:57:24.572681+0000","last_change":"2023-11-24T07:57:24.572681+0000","last_active":"2023-11-24T07:57:24.572681+0000","last_peered":"2023-11-24T07:57:24.572681+0000","last_clean":"2023-11-24T07:57:24.572681+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T07:57:24.572681+0000","last_undegraded":"2023-11-24T07:57:24.572681+0000","last_fullsized":"2023-11-24T07:57:24.572681+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:57:24.572681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:57:24.572681+0000","last_clean_scrub_stamp":"2023-11-24T07:57:24.572681+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T07:57:23.565116+0000","last_change":"2023-11-24T07:57:23.565116+0000","last_active":"2023-11-24T07:57:23.565116+0000","last_peered":"2023-11-24T07:57:23.565116+0000","last_clean":"2023-11-24T07:57:23.565116+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T07:57:23.565116+0000","last_undegraded":"2023-11-24T07:57:23.565116+0000","last_fullsized":"2023-11-24T07:57:23.565116+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:57:23.565116+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:57:23.565116+0000","last_clean_scrub_stamp":"2023-11-24T07:57:23.565116+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T07:57:24.572681+0000","last_change":"2023-11-24T07:57:24.572681+0000","last_active":"2023-11-24T07:57:24.572681+0000","last_peered":"2023-11-24T07:57:24.572681+0000","last_clean":"2023-11-24T07:57:24.572681+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T07:57:24.572681+0000","last_undegraded":"2023-11-24T07:57:24.572681+0000","last_fullsized":"2023-11-24T07:57:24.572681+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:57:24.572681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:57:24.572681+0000","last_clean_scrub_stamp":"2023-11-24T07:57:24.572681+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T07:57:23.565116+0000","last_change":"2023-11-24T07:57:23.565116+0000","last_active":"2023-11-24T07:57:23.565116+0000","last_peered":"2023-11-24T07:57:23.565116+0000","last_clean":"2023-11-24T07:57:23.565116+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T07:57:23.565116+0000","last_undegraded":"2023-11-24T07:57:23.565116+0000","last_fullsized":"2023-11-24T07:57:23.565116+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:57:23.565116+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:57:23.565116+0000","last_clean_scrub_stamp":"2023-11-24T07:57:23.565116+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T07:57:24.572681+0000","last_change":"2023-11-24T07:57:24.572681+0000","last_active":"2023-11-24T07:57:24.572681+0000","last_peered":"2023-11-24T07:57:24.572681+0000","last_clean":"2023-11-24T07:57:24.572681+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T07:57:24.572681+0000","last_undegraded":"2023-11-24T07:57:24.572681+0000","last_fullsized":"2023-11-24T07:57:24.572681+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:57:24.572681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:57:24.572681+0000","last_clean_scrub_stamp":"2023-11-24T07:57:24.572681+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T07:57:23.565116+0000","last_change":"2023-11-24T07:57:23.565116+0000","last_active":"2023-11-24T07:57:23.565116+0000","last_peered":"2023-11-24T07:57:23.565116+0000","last_clean":"2023-11-24T07:57:23.565116+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T07:57:23.565116+0000","last_undegraded":"2023-11-24T07:57:23.565116+0000","last_fullsized":"2023-11-24T07:57:23.565116+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:57:23.565116+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:57:23.565116+0000","last_clean_scrub_stamp":"2023-11-24T07:57:23.565116+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T07:57:24.572681+0000","last_change":"2023-11-24T07:57:24.572681+0000","last_active":"2023-11-24T07:57:24.572681+0000","last_peered":"2023-11-24T07:57:24.572681+0000","last_clean":"2023-11-24T07:57:24.572681+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T07:57:24.572681+0000","last_undegraded":"2023-11-24T07:57:24.572681+0000","last_fullsized":"2023-11-24T07:57:24.572681+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:57:24.572681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:57:24.572681+0000","last_clean_scrub_stamp":"2023-11-24T07:57:24.572681+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T07:57:23.565116+0000","last_change":"2023-11-24T07:57:23.565116+0000","last_active":"2023-11-24T07:57:23.565116+0000","last_peered":"2023-11-24T07:57:23.565116+0000","last_clean":"2023-11-24T07:57:23.565116+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T07:57:23.565116+0000","last_undegraded":"2023-11-24T07:57:23.565116+0000","last_fullsized":"2023-11-24T07:57:23.565116+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:57:23.565116+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:57:23.565116+0000","last_clean_scrub_stamp":"2023-11-24T07:57:23.565116+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T07:57:24.572681+0000","last_change":"2023-11-24T07:57:24.572681+0000","last_active":"2023-11-24T07:57:24.572681+0000","last_peered":"2023-11-24T07:57:24.572681+0000","last_clean":"2023-11-24T07:57:24.572681+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T07:57:24.572681+0000","last_undegraded":"2023-11-24T07:57:24.572681+0000","last_fullsized":"2023-11-24T07:57:24.572681+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:57:24.572681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:57:24.572681+0000","last_clean_scrub_stamp":"2023-11-24T07:57:24.572681+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T07:57:23.565116+0000","last_change":"2023-11-24T07:57:23.565116+0000","last_active":"2023-11-24T07:57:23.565116+0000","last_peered":"2023-11-24T07:57:23.565116+0000","last_clean":"2023-11-24T07:57:23.565116+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T07:57:23.565116+0000","last_undegraded":"2023-11-24T07:57:23.565116+0000","last_fullsized":"2023-11-24T07:57:23.565116+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:57:23.565116+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:57:23.565116+0000","last_clean_scrub_stamp":"2023-11-24T07:57:23.565116+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T07:57:24.572681+0000","last_change":"2023-11-24T07:57:24.572681+0000","last_active":"2023-11-24T07:57:24.572681+0000","last_peered":"2023-11-24T07:57:24.572681+0000","last_clean":"2023-11-24T07:57:24.572681+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T07:57:24.572681+0000","last_undegraded":"2023-11-24T07:57:24.572681+0000","last_fullsized":"2023-11-24T07:57:24.572681+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:57:24.572681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:57:24.572681+0000","last_clean_scrub_stamp":"2023-11-24T07:57:24.572681+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T07:57:23.565116+0000","last_change":"2023-11-24T07:57:23.565116+0000","last_active":"2023-11-24T07:57:23.565116+0000","last_peered":"2023-11-24T07:57:23.565116+0000","last_clean":"2023-11-24T07:57:23.565116+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T07:57:23.565116+0000","last_undegraded":"2023-11-24T07:57:23.565116+0000","last_fullsized":"2023-11-24T07:57:23.565116+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:57:23.565116+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:57:23.565116+0000","last_clean_scrub_stamp":"2023-11-24T07:57:23.565116+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T07:57:24.572681+0000","last_change":"2023-11-24T07:57:24.572681+0000","last_active":"2023-11-24T07:57:24.572681+0000","last_peered":"2023-11-24T07:57:24.572681+0000","last_clean":"2023-11-24T07:57:24.572681+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T07:57:24.572681+0000","last_undegraded":"2023-11-24T07:57:24.572681+0000","last_fullsized":"2023-11-24T07:57:24.572681+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:57:24.572681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:57:24.572681+0000","last_clean_scrub_stamp":"2023-11-24T07:57:24.572681+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T07:57:23.565116+0000","last_change":"2023-11-24T07:57:23.565116+0000","last_active":"2023-11-24T07:57:23.565116+0000","last_peered":"2023-11-24T07:57:23.565116+0000","last_clean":"2023-11-24T07:57:23.565116+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T07:57:23.565116+0000","last_undegraded":"2023-11-24T07:57:23.565116+0000","last_fullsized":"2023-11-24T07:57:23.565116+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:57:23.565116+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:57:23.565116+0000","last_clean_scrub_stamp":"2023-11-24T07:57:23.565116+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T07:57:24.572681+0000","last_change":"2023-11-24T07:57:24.572681+0000","last_active":"2023-11-24T07:57:24.572681+0000","last_peered":"2023-11-24T07:57:24.572681+0000","last_clean":"2023-11-24T07:57:24.572681+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T07:57:24.572681+0000","last_undegraded":"2023-11-24T07:57:24.572681+0000","last_fullsized":"2023-11-24T07:57:24.572681+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:57:24.572681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:57:24.572681+0000","last_clean_scrub_stamp":"2023-11-24T07:57:24.572681+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T07:57:23.565116+0000","last_change":"2023-11-24T07:57:23.565116+0000","last_active":"2023-11-24T07:57:23.565116+0000","last_peered":"2023-11-24T07:57:23.565116+0000","last_clean":"2023-11-24T07:57:23.565116+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T07:57:23.565116+0000","last_undegraded":"2023-11-24T07:57:23.565116+0000","last_fullsized":"2023-11-24T07:57:23.565116+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:57:23.565116+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:57:23.565116+0000","last_clean_scrub_stamp":"2023-11-24T07:57:23.565116+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T07:57:24.572681+0000","last_change":"2023-11-24T07:57:24.572681+0000","last_active":"2023-11-24T07:57:24.572681+0000","last_peered":"2023-11-24T07:57:24.572681+0000","last_clean":"2023-11-24T07:57:24.572681+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T07:57:24.572681+0000","last_undegraded":"2023-11-24T07:57:24.572681+0000","last_fullsized":"2023-11-24T07:57:24.572681+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:57:24.572681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:57:24.572681+0000","last_clean_scrub_stamp":"2023-11-24T07:57:24.572681+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T07:57:23.565116+0000","last_change":"2023-11-24T07:57:23.565116+0000","last_active":"2023-11-24T07:57:23.565116+0000","last_peered":"2023-11-24T07:57:23.565116+0000","last_clean":"2023-11-24T07:57:23.565116+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T07:57:23.565116+0000","last_undegraded":"2023-11-24T07:57:23.565116+0000","last_fullsized":"2023-11-24T07:57:23.565116+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:57:23.565116+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:57:23.565116+0000","last_clean_scrub_stamp":"2023-11-24T07:57:23.565116+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T07:57:24.572681+0000","last_change":"2023-11-24T07:57:24.572681+0000","last_active":"2023-11-24T07:57:24.572681+0000","last_peered":"2023-11-24T07:57:24.572681+0000","last_clean":"2023-11-24T07:57:24.572681+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T07:57:24.572681+0000","last_undegraded":"2023-11-24T07:57:24.572681+0000","last_fullsized":"2023-11-24T07:57:24.572681+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:57:24.572681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:57:24.572681+0000","last_clean_scrub_stamp":"2023-11-24T07:57:24.572681+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T07:57:23.565116+0000","last_change":"2023-11-24T07:57:23.565116+0000","last_active":"2023-11-24T07:57:23.565116+0000","last_peered":"2023-11-24T07:57:23.565116+0000","last_clean":"2023-11-24T07:57:23.565116+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T07:57:23.565116+0000","last_undegraded":"2023-11-24T07:57:23.565116+0000","last_fullsized":"2023-11-24T07:57:23.565116+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:57:23.565116+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:57:23.565116+0000","last_clean_scrub_stamp":"2023-11-24T07:57:23.565116+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T07:57:24.572681+0000","last_change":"2023-11-24T07:57:24.572681+0000","last_active":"2023-11-24T07:57:24.572681+0000","last_peered":"2023-11-24T07:57:24.572681+0000","last_clean":"2023-11-24T07:57:24.572681+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T07:57:24.572681+0000","last_undegraded":"2023-11-24T07:57:24.572681+0000","last_fullsized":"2023-11-24T07:57:24.572681+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:57:24.572681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:57:24.572681+0000","last_clean_scrub_stamp":"2023-11-24T07:57:24.572681+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T07:57:23.565116+0000","last_change":"2023-11-24T07:57:23.565116+0000","last_active":"2023-11-24T07:57:23.565116+0000","last_peered":"2023-11-24T07:57:23.565116+0000","last_clean":"2023-11-24T07:57:23.565116+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T07:57:23.565116+0000","last_undegraded":"2023-11-24T07:57:23.565116+0000","last_fullsized":"2023-11-24T07:57:23.565116+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:57:23.565116+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:57:23.565116+0000","last_clean_scrub_stamp":"2023-11-24T07:57:23.565116+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T07:57:24.572681+0000","last_change":"2023-11-24T07:57:24.572681+0000","last_active":"2023-11-24T07:57:24.572681+0000","last_peered":"2023-11-24T07:57:24.572681+0000","last_clean":"2023-11-24T07:57:24.572681+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T07:57:24.572681+0000","last_undegraded":"2023-11-24T07:57:24.572681+0000","last_fullsized":"2023-11-24T07:57:24.572681+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:57:24.572681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:57:24.572681+0000","last_clean_scrub_stamp":"2023-11-24T07:57:24.572681+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T07:57:23.565116+0000","last_change":"2023-11-24T07:57:23.565116+0000","last_active":"2023-11-24T07:57:23.565116+0000","last_peered":"2023-11-24T07:57:23.565116+0000","last_clean":"2023-11-24T07:57:23.565116+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T07:57:23.565116+0000","last_undegraded":"2023-11-24T07:57:23.565116+0000","last_fullsized":"2023-11-24T07:57:23.565116+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:57:23.565116+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:57:23.565116+0000","last_clean_scrub_stamp":"2023-11-24T07:57:23.565116+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T07:57:24.572681+0000","last_change":"2023-11-24T07:57:24.572681+0000","last_active":"2023-11-24T07:57:24.572681+0000","last_peered":"2023-11-24T07:57:24.572681+0000","last_clean":"2023-11-24T07:57:24.572681+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T07:57:24.572681+0000","last_undegraded":"2023-11-24T07:57:24.572681+0000","last_fullsized":"2023-11-24T07:57:24.572681+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:57:24.572681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:57:24.572681+0000","last_clean_scrub_stamp":"2023-11-24T07:57:24.572681+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T07:57:23.565116+0000","last_change":"2023-11-24T07:57:23.565116+0000","last_active":"2023-11-24T07:57:23.565116+0000","last_peered":"2023-11-24T07:57:23.565116+0000","last_clean":"2023-11-24T07:57:23.565116+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T07:57:23.565116+0000","last_undegraded":"2023-11-24T07:57:23.565116+0000","last_fullsized":"2023-11-24T07:57:23.565116+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:57:23.565116+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:57:23.565116+0000","last_clean_scrub_stamp":"2023-11-24T07:57:23.565116+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T07:57:24.572681+0000","last_change":"2023-11-24T07:57:24.572681+0000","last_active":"2023-11-24T07:57:24.572681+0000","last_peered":"2023-11-24T07:57:24.572681+0000","last_clean":"2023-11-24T07:57:24.572681+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T07:57:24.572681+0000","last_undegraded":"2023-11-24T07:57:24.572681+0000","last_fullsized":"2023-11-24T07:57:24.572681+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:57:24.572681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:57:24.572681+0000","last_clean_scrub_stamp":"2023-11-24T07:57:24.572681+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T07:57:23.565116+0000","last_change":"2023-11-24T07:57:23.565116+0000","last_active":"2023-11-24T07:57:23.565116+0000","last_peered":"2023-11-24T07:57:23.565116+0000","last_clean":"2023-11-24T07:57:23.565116+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T07:57:23.565116+0000","last_undegraded":"2023-11-24T07:57:23.565116+0000","last_fullsized":"2023-11-24T07:57:23.565116+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:57:23.565116+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:57:23.565116+0000","last_clean_scrub_stamp":"2023-11-24T07:57:23.565116+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T07:57:24.572681+0000","last_change":"2023-11-24T07:57:24.572681+0000","last_active":"2023-11-24T07:57:24.572681+0000","last_peered":"2023-11-24T07:57:24.572681+0000","last_clean":"2023-11-24T07:57:24.572681+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T07:57:24.572681+0000","last_undegraded":"2023-11-24T07:57:24.572681+0000","last_fullsized":"2023-11-24T07:57:24.572681+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:57:24.572681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:57:24.572681+0000","last_clean_scrub_stamp":"2023-11-24T07:57:24.572681+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T07:57:23.565116+0000","last_change":"2023-11-24T07:57:23.565116+0000","last_active":"2023-11-24T07:57:23.565116+0000","last_peered":"2023-11-24T07:57:23.565116+0000","last_clean":"2023-11-24T07:57:23.565116+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T07:57:23.565116+0000","last_undegraded":"2023-11-24T07:57:23.565116+0000","last_fullsized":"2023-11-24T07:57:23.565116+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:57:23.565116+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:57:23.565116+0000","last_clean_scrub_stamp":"2023-11-24T07:57:23.565116+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T07:57:24.572681+0000","last_change":"2023-11-24T07:57:24.572681+0000","last_active":"2023-11-24T07:57:24.572681+0000","last_peered":"2023-11-24T07:57:24.572681+0000","last_clean":"2023-11-24T07:57:24.572681+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T07:57:24.572681+0000","last_undegraded":"2023-11-24T07:57:24.572681+0000","last_fullsized":"2023-11-24T07:57:24.572681+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:57:24.572681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:57:24.572681+0000","last_clean_scrub_stamp":"2023-11-24T07:57:24.572681+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T07:57:23.565116+0000","last_change":"2023-11-24T07:57:23.565116+0000","last_active":"2023-11-24T07:57:23.565116+0000","last_peered":"2023-11-24T07:57:23.565116+0000","last_clean":"2023-11-24T07:57:23.565116+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T07:57:23.565116+0000","last_undegraded":"2023-11-24T07:57:23.565116+0000","last_fullsized":"2023-11-24T07:57:23.565116+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:57:23.565116+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:57:23.565116+0000","last_clean_scrub_stamp":"2023-11-24T07:57:23.565116+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T07:57:24.572681+0000","last_change":"2023-11-24T07:57:24.572681+0000","last_active":"2023-11-24T07:57:24.572681+0000","last_peered":"2023-11-24T07:57:24.572681+0000","last_clean":"2023-11-24T07:57:24.572681+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T07:57:24.572681+0000","last_undegraded":"2023-11-24T07:57:24.572681+0000","last_fullsized":"2023-11-24T07:57:24.572681+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:57:24.572681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:57:24.572681+0000","last_clean_scrub_stamp":"2023-11-24T07:57:24.572681+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T07:57:23.565116+0000","last_change":"2023-11-24T07:57:23.565116+0000","last_active":"2023-11-24T07:57:23.565116+0000","last_peered":"2023-11-24T07:57:23.565116+0000","last_clean":"2023-11-24T07:57:23.565116+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T07:57:23.565116+0000","last_undegraded":"2023-11-24T07:57:23.565116+0000","last_fullsized":"2023-11-24T07:57:23.565116+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:57:23.565116+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:57:23.565116+0000","last_clean_scrub_stamp":"2023-11-24T07:57:23.565116+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T07:57:24.572681+0000","last_change":"2023-11-24T07:57:24.572681+0000","last_active":"2023-11-24T07:57:24.572681+0000","last_peered":"2023-11-24T07:57:24.572681+0000","last_clean":"2023-11-24T07:57:24.572681+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T07:57:24.572681+0000","last_undegraded":"2023-11-24T07:57:24.572681+0000","last_fullsized":"2023-11-24T07:57:24.572681+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:57:24.572681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:57:24.572681+0000","last_clean_scrub_stamp":"2023-11-24T07:57:24.572681+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T07:57:23.565116+0000","last_change":"2023-11-24T07:57:23.565116+0000","last_active":"2023-11-24T07:57:23.565116+0000","last_peered":"2023-11-24T07:57:23.565116+0000","last_clean":"2023-11-24T07:57:23.565116+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T07:57:23.565116+0000","last_undegraded":"2023-11-24T07:57:23.565116+0000","last_fullsized":"2023-11-24T07:57:23.565116+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:57:23.565116+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:57:23.565116+0000","last_clean_scrub_stamp":"2023-11-24T07:57:23.565116+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T07:57:24.572681+0000","last_change":"2023-11-24T07:57:24.572681+0000","last_active":"2023-11-24T07:57:24.572681+0000","last_peered":"2023-11-24T07:57:24.572681+0000","last_clean":"2023-11-24T07:57:24.572681+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T07:57:24.572681+0000","last_undegraded":"2023-11-24T07:57:24.572681+0000","last_fullsized":"2023-11-24T07:57:24.572681+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:57:24.572681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:57:24.572681+0000","last_clean_scrub_stamp":"2023-11-24T07:57:24.572681+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T07:57:23.565116+0000","last_change":"2023-11-24T07:57:23.565116+0000","last_active":"2023-11-24T07:57:23.565116+0000","last_peered":"2023-11-24T07:57:23.565116+0000","last_clean":"2023-11-24T07:57:23.565116+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T07:57:23.565116+0000","last_undegraded":"2023-11-24T07:57:23.565116+0000","last_fullsized":"2023-11-24T07:57:23.565116+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:57:23.565116+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:57:23.565116+0000","last_clean_scrub_stamp":"2023-11-24T07:57:23.565116+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T07:57:24.572681+0000","last_change":"2023-11-24T07:57:24.572681+0000","last_active":"2023-11-24T07:57:24.572681+0000","last_peered":"2023-11-24T07:57:24.572681+0000","last_clean":"2023-11-24T07:57:24.572681+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T07:57:24.572681+0000","last_undegraded":"2023-11-24T07:57:24.572681+0000","last_fullsized":"2023-11-24T07:57:24.572681+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:57:24.572681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:57:24.572681+0000","last_clean_scrub_stamp":"2023-11-24T07:57:24.572681+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T07:57:23.565116+0000","last_change":"2023-11-24T07:57:23.565116+0000","last_active":"2023-11-24T07:57:23.565116+0000","last_peered":"2023-11-24T07:57:23.565116+0000","last_clean":"2023-11-24T07:57:23.565116+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T07:57:23.565116+0000","last_undegraded":"2023-11-24T07:57:23.565116+0000","last_fullsized":"2023-11-24T07:57:23.565116+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:57:23.565116+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:57:23.565116+0000","last_clean_scrub_stamp":"2023-11-24T07:57:23.565116+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T07:57:24.572681+0000","last_change":"2023-11-24T07:57:24.572681+0000","last_active":"2023-11-24T07:57:24.572681+0000","last_peered":"2023-11-24T07:57:24.572681+0000","last_clean":"2023-11-24T07:57:24.572681+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T07:57:24.572681+0000","last_undegraded":"2023-11-24T07:57:24.572681+0000","last_fullsized":"2023-11-24T07:57:24.572681+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:57:24.572681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:57:24.572681+0000","last_clean_scrub_stamp":"2023-11-24T07:57:24.572681+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T07:57:23.565116+0000","last_change":"2023-11-24T07:57:23.565116+0000","last_active":"2023-11-24T07:57:23.565116+0000","last_peered":"2023-11-24T07:57:23.565116+0000","last_clean":"2023-11-24T07:57:23.565116+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T07:57:23.565116+0000","last_undegraded":"2023-11-24T07:57:23.565116+0000","last_fullsized":"2023-11-24T07:57:23.565116+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:57:23.565116+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:57:23.565116+0000","last_clean_scrub_stamp":"2023-11-24T07:57:23.565116+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T07:57:24.572681+0000","last_change":"2023-11-24T07:57:24.572681+0000","last_active":"2023-11-24T07:57:24.572681+0000","last_peered":"2023-11-24T07:57:24.572681+0000","last_clean":"2023-11-24T07:57:24.572681+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T07:57:24.572681+0000","last_undegraded":"2023-11-24T07:57:24.572681+0000","last_fullsized":"2023-11-24T07:57:24.572681+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:57:24.572681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:57:24.572681+0000","last_clean_scrub_stamp":"2023-11-24T07:57:24.572681+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T07:57:23.565116+0000","last_change":"2023-11-24T07:57:23.565116+0000","last_active":"2023-11-24T07:57:23.565116+0000","last_peered":"2023-11-24T07:57:23.565116+0000","last_clean":"2023-11-24T07:57:23.565116+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T07:57:23.565116+0000","last_undegraded":"2023-11-24T07:57:23.565116+0000","last_fullsized":"2023-11-24T07:57:23.565116+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:57:23.565116+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:57:23.565116+0000","last_clean_scrub_stamp":"2023-11-24T07:57:23.565116+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T07:57:24.572681+0000","last_change":"2023-11-24T07:57:24.572681+0000","last_active":"2023-11-24T07:57:24.572681+0000","last_peered":"2023-11-24T07:57:24.572681+0000","last_clean":"2023-11-24T07:57:24.572681+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T07:57:24.572681+0000","last_undegraded":"2023-11-24T07:57:24.572681+0000","last_fullsized":"2023-11-24T07:57:24.572681+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:57:24.572681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:57:24.572681+0000","last_clean_scrub_stamp":"2023-11-24T07:57:24.572681+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T07:57:23.565116+0000","last_change":"2023-11-24T07:57:23.565116+0000","last_active":"2023-11-24T07:57:23.565116+0000","last_peered":"2023-11-24T07:57:23.565116+0000","last_clean":"2023-11-24T07:57:23.565116+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T07:57:23.565116+0000","last_undegraded":"2023-11-24T07:57:23.565116+0000","last_fullsized":"2023-11-24T07:57:23.565116+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:57:23.565116+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:57:23.565116+0000","last_clean_scrub_stamp":"2023-11-24T07:57:23.565116+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T07:57:24.572681+0000","last_change":"2023-11-24T07:57:24.572681+0000","last_active":"2023-11-24T07:57:24.572681+0000","last_peered":"2023-11-24T07:57:24.572681+0000","last_clean":"2023-11-24T07:57:24.572681+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T07:57:24.572681+0000","last_undegraded":"2023-11-24T07:57:24.572681+0000","last_fullsized":"2023-11-24T07:57:24.572681+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:57:24.572681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:57:24.572681+0000","last_clean_scrub_stamp":"2023-11-24T07:57:24.572681+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T07:57:23.565116+0000","last_change":"2023-11-24T07:57:23.565116+0000","last_active":"2023-11-24T07:57:23.565116+0000","last_peered":"2023-11-24T07:57:23.565116+0000","last_clean":"2023-11-24T07:57:23.565116+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T07:57:23.565116+0000","last_undegraded":"2023-11-24T07:57:23.565116+0000","last_fullsized":"2023-11-24T07:57:23.565116+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:57:23.565116+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:57:23.565116+0000","last_clean_scrub_stamp":"2023-11-24T07:57:23.565116+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T07:57:24.572681+0000","last_change":"2023-11-24T07:57:24.572681+0000","last_active":"2023-11-24T07:57:24.572681+0000","last_peered":"2023-11-24T07:57:24.572681+0000","last_clean":"2023-11-24T07:57:24.572681+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T07:57:24.572681+0000","last_undegraded":"2023-11-24T07:57:24.572681+0000","last_fullsized":"2023-11-24T07:57:24.572681+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:57:24.572681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:57:24.572681+0000","last_clean_scrub_stamp":"2023-11-24T07:57:24.572681+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T07:57:23.565116+0000","last_change":"2023-11-24T07:57:23.565116+0000","last_active":"2023-11-24T07:57:23.565116+0000","last_peered":"2023-11-24T07:57:23.565116+0000","last_clean":"2023-11-24T07:57:23.565116+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T07:57:23.565116+0000","last_undegraded":"2023-11-24T07:57:23.565116+0000","last_fullsized":"2023-11-24T07:57:23.565116+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:57:23.565116+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:57:23.565116+0000","last_clean_scrub_stamp":"2023-11-24T07:57:23.565116+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T07:57:24.572681+0000","last_change":"2023-11-24T07:57:24.572681+0000","last_active":"2023-11-24T07:57:24.572681+0000","last_peered":"2023-11-24T07:57:24.572681+0000","last_clean":"2023-11-24T07:57:24.572681+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T07:57:24.572681+0000","last_undegraded":"2023-11-24T07:57:24.572681+0000","last_fullsized":"2023-11-24T07:57:24.572681+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:57:24.572681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:57:24.572681+0000","last_clean_scrub_stamp":"2023-11-24T07:57:24.572681+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T07:57:23.565116+0000","last_change":"2023-11-24T07:57:23.565116+0000","last_active":"2023-11-24T07:57:23.565116+0000","last_peered":"2023-11-24T07:57:23.565116+0000","last_clean":"2023-11-24T07:57:23.565116+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T07:57:23.565116+0000","last_undegraded":"2023-11-24T07:57:23.565116+0000","last_fullsized":"2023-11-24T07:57:23.565116+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:57:23.565116+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:57:23.565116+0000","last_clean_scrub_stamp":"2023-11-24T07:57:23.565116+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T07:57:24.572681+0000","last_change":"2023-11-24T07:57:24.572681+0000","last_active":"2023-11-24T07:57:24.572681+0000","last_peered":"2023-11-24T07:57:24.572681+0000","last_clean":"2023-11-24T07:57:24.572681+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T07:57:24.572681+0000","last_undegraded":"2023-11-24T07:57:24.572681+0000","last_fullsized":"2023-11-24T07:57:24.572681+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:57:24.572681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:57:24.572681+0000","last_clean_scrub_stamp":"2023-11-24T07:57:24.572681+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T07:57:23.565116+0000","last_change":"2023-11-24T07:57:23.565116+0000","last_active":"2023-11-24T07:57:23.565116+0000","last_peered":"2023-11-24T07:57:23.565116+0000","last_clean":"2023-11-24T07:57:23.565116+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T07:57:23.565116+0000","last_undegraded":"2023-11-24T07:57:23.565116+0000","last_fullsized":"2023-11-24T07:57:23.565116+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:57:23.565116+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:57:23.565116+0000","last_clean_scrub_stamp":"2023-11-24T07:57:23.565116+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T07:57:24.572681+0000","last_change":"2023-11-24T07:57:24.572681+0000","last_active":"2023-11-24T07:57:24.572681+0000","last_peered":"2023-11-24T07:57:24.572681+0000","last_clean":"2023-11-24T07:57:24.572681+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T07:57:24.572681+0000","last_undegraded":"2023-11-24T07:57:24.572681+0000","last_fullsized":"2023-11-24T07:57:24.572681+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:57:24.572681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:57:24.572681+0000","last_clean_scrub_stamp":"2023-11-24T07:57:24.572681+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T07:57:23.565116+0000","last_change":"2023-11-24T07:57:23.565116+0000","last_active":"2023-11-24T07:57:23.565116+0000","last_peered":"2023-11-24T07:57:23.565116+0000","last_clean":"2023-11-24T07:57:23.565116+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T07:57:23.565116+0000","last_undegraded":"2023-11-24T07:57:23.565116+0000","last_fullsized":"2023-11-24T07:57:23.565116+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:57:23.565116+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:57:23.565116+0000","last_clean_scrub_stamp":"2023-11-24T07:57:23.565116+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":172,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-11-24T07:57:18.484315+0000","last_change":"2023-11-24T07:56:19.668905+0000","last_active":"2023-11-24T07:57:18.484315+0000","last_peered":"2023-11-24T07:57:18.484315+0000","last_clean":"2023-11-24T07:57:18.484315+0000","last_became_active":"2023-11-24T07:56:19.668601+0000","last_became_peered":"2023-11-24T07:56:19.668601+0000","last_unstale":"2023-11-24T07:57:18.484315+0000","last_undegraded":"2023-11-24T07:57:18.484315+0000","last_fullsized":"2023-11-24T07:57:18.484315+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:56:18.657844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:56:18.657844+0000","last_clean_scrub_stamp":"2023-11-24T07:56:18.657844+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T11:51:53.828455+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":5,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574864,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27340,"kb_used_data":316,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344500,"statfs":{"total":96636764160,"available":96608768000,"internally_reserved":0,"allocated":323584,"data_stored":120931,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 07:57:19 2023","interfaces":[{"interface":"back","average":{"1min":0.55700000000000005,"5min":0.55700000000000005,"15min":0.55700000000000005},"min":{"1min":0.32800000000000001,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":1.413,"5min":1.413,"15min":1.413},"last":0.71299999999999997},{"interface":"front","average":{"1min":0.55400000000000005,"5min":0.55400000000000005,"15min":0.55400000000000005},"min":{"1min":0.41099999999999998,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":0.81000000000000005,"5min":0.81000000000000005,"15min":0.81000000000000005},"last":0.65300000000000002}]},{"osd":1,"last update":"Fri Nov 24 07:57:19 2023","interfaces":[{"interface":"back","average":{"1min":0.84899999999999998,"5min":0.84899999999999998,"15min":0.84899999999999998},"min":{"1min":0.33600000000000002,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":6.1070000000000002,"5min":6.1070000000000002,"15min":6.1070000000000002},"last":0.60299999999999998},{"interface":"front","average":{"1min":0.57999999999999996,"5min":0.57999999999999996,"15min":0.57999999999999996},"min":{"1min":0.35199999999999998,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":1.5920000000000001,"5min":1.5920000000000001,"15min":1.5920000000000001},"last":0.39700000000000002}]},{"osd":2,"last update":"Fri Nov 24 07:57:19 2023","interfaces":[{"interface":"back","average":{"1min":0.627,"5min":0.627,"15min":0.627},"min":{"1min":0.33800000000000002,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":1.613,"5min":1.613,"15min":1.613},"last":0.56000000000000005},{"interface":"front","average":{"1min":0.79200000000000004,"5min":0.79200000000000004,"15min":0.79200000000000004},"min":{"1min":0.41699999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":4.5220000000000002,"5min":4.5220000000000002,"15min":4.5220000000000002},"last":0.54000000000000004}]},{"osd":3,"last update":"Fri Nov 24 07:57:19 2023","interfaces":[{"interface":"back","average":{"1min":0.80600000000000005,"5min":0.80600000000000005,"15min":0.80600000000000005},"min":{"1min":0.38500000000000001,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":4.6760000000000002,"5min":4.6760000000000002,"15min":4.6760000000000002},"last":0.46300000000000002},{"interface":"front","average":{"1min":0.94299999999999995,"5min":0.94299999999999995,"15min":0.94299999999999995},"min":{"1min":0.505,"5min":0.505,"15min":0.505},"max":{"1min":4.6539999999999999,"5min":4.6539999999999999,"15min":4.6539999999999999},"last":0.63300000000000001}]},{"osd":4,"last update":"Fri Nov 24 07:57:19 2023","interfaces":[{"interface":"back","average":{"1min":0.56799999999999995,"5min":0.56799999999999995,"15min":0.56799999999999995},"min":{"1min":0.378,"5min":0.378,"15min":0.378},"max":{"1min":0.84699999999999998,"5min":0.84699999999999998,"15min":0.84699999999999998},"last":0.67400000000000004},{"interface":"front","average":{"1min":0.66700000000000004,"5min":0.66700000000000004,"15min":0.66700000000000004},"min":{"1min":0.46100000000000002,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":1.629,"5min":1.629,"15min":1.629},"last":0.51200000000000001}]},{"osd":5,"last update":"Fri Nov 24 07:57:19 2023","interfaces":[{"interface":"back","average":{"1min":0.54700000000000004,"5min":0.54700000000000004,"15min":0.54700000000000004},"min":{"1min":0.34499999999999997,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.875,"5min":0.875,"15min":0.875},"last":0.69899999999999995},{"interface":"front","average":{"1min":0.623,"5min":0.623,"15min":0.623},"min":{"1min":0.41499999999999998,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":0.90900000000000003,"5min":0.90900000000000003,"15min":0.90900000000000003},"last":0.79700000000000004}]},{"osd":6,"last update":"Fri Nov 24 07:57:19 2023","interfaces":[{"interface":"back","average":{"1min":0.61099999999999999,"5min":0.61099999999999999,"15min":0.61099999999999999},"min":{"1min":0.438,"5min":0.438,"15min":0.438},"max":{"1min":0.80800000000000005,"5min":0.80800000000000005,"15min":0.80800000000000005},"last":0.76700000000000002},{"interface":"front","average":{"1min":0.67000000000000004,"5min":0.67000000000000004,"15min":0.67000000000000004},"min":{"1min":0.48599999999999999,"5min":0.48599999999999999,"15min":0.48599999999999999},"max":{"1min":0.98399999999999999,"5min":0.98399999999999999,"15min":0.98399999999999999},"last":0.73199999999999998}]}]},{"osd":6,"up_from":13,"seq":55834574865,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27348,"kb_used_data":316,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344492,"statfs":{"total":96636764160,"available":96608759808,"internally_reserved":0,"allocated":323584,"data_stored":120931,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 07:57:18 2023","interfaces":[{"interface":"back","average":{"1min":0.55000000000000004,"5min":0.55000000000000004,"15min":0.55000000000000004},"min":{"1min":0.28399999999999997,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":0.85099999999999998,"5min":0.85099999999999998,"15min":0.85099999999999998},"last":0.67000000000000004},{"interface":"front","average":{"1min":0.51600000000000001,"5min":0.51600000000000001,"15min":0.51600000000000001},"min":{"1min":0.311,"5min":0.311,"15min":0.311},"max":{"1min":0.69999999999999996,"5min":0.69999999999999996,"15min":0.69999999999999996},"last":0.82199999999999995}]},{"osd":1,"last update":"Fri Nov 24 07:57:18 2023","interfaces":[{"interface":"back","average":{"1min":0.51200000000000001,"5min":0.51200000000000001,"15min":0.51200000000000001},"min":{"1min":0.33800000000000002,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.78200000000000003,"5min":0.78200000000000003,"15min":0.78200000000000003},"last":0.54300000000000004},{"interface":"front","average":{"1min":0.61299999999999999,"5min":0.61299999999999999,"15min":0.61299999999999999},"min":{"1min":0.44700000000000001,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":1.0069999999999999,"5min":1.0069999999999999,"15min":1.0069999999999999},"last":0.50800000000000001}]},{"osd":2,"last update":"Fri Nov 24 07:57:18 2023","interfaces":[{"interface":"back","average":{"1min":0.57499999999999996,"5min":0.57499999999999996,"15min":0.57499999999999996},"min":{"1min":0.34300000000000003,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.878,"5min":0.878,"15min":0.878},"last":0.64400000000000002},{"interface":"front","average":{"1min":0.59299999999999997,"5min":0.59299999999999997,"15min":0.59299999999999997},"min":{"1min":0.47099999999999997,"5min":0.47099999999999997,"15min":0.47099999999999997},"max":{"1min":0.82399999999999995,"5min":0.82399999999999995,"15min":0.82399999999999995},"last":0.57099999999999995}]},{"osd":3,"last update":"Fri Nov 24 07:57:18 2023","interfaces":[{"interface":"back","average":{"1min":0.56799999999999995,"5min":0.56799999999999995,"15min":0.56799999999999995},"min":{"1min":0.40000000000000002,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":0.876,"5min":0.876,"15min":0.876},"last":0.65800000000000003},{"interface":"front","average":{"1min":0.55600000000000005,"5min":0.55600000000000005,"15min":0.55600000000000005},"min":{"1min":0.379,"5min":0.379,"15min":0.379},"max":{"1min":0.68799999999999994,"5min":0.68799999999999994,"15min":0.68799999999999994},"last":0.42699999999999999}]},{"osd":4,"last update":"Fri Nov 24 07:57:18 2023","interfaces":[{"interface":"back","average":{"1min":0.51700000000000002,"5min":0.51700000000000002,"15min":0.51700000000000002},"min":{"1min":0.33000000000000002,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.70599999999999996,"5min":0.70599999999999996,"15min":0.70599999999999996},"last":0.46700000000000003},{"interface":"front","average":{"1min":0.60199999999999998,"5min":0.60199999999999998,"15min":0.60199999999999998},"min":{"1min":0.35199999999999998,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":1.087,"5min":1.087,"15min":1.087},"last":0.60199999999999998}]},{"osd":5,"last update":"Fri Nov 24 07:57:18 2023","interfaces":[{"interface":"back","average":{"1min":0.53100000000000003,"5min":0.53100000000000003,"15min":0.53100000000000003},"min":{"1min":0.30199999999999999,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.84999999999999998,"5min":0.84999999999999998,"15min":0.84999999999999998},"last":0.58699999999999997},{"interface":"front","average":{"1min":0.61199999999999999,"5min":0.61199999999999999,"15min":0.61199999999999999},"min":{"1min":0.39200000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.97199999999999998,"5min":0.97199999999999998,"15min":0.97199999999999998},"last":0.627}]},{"osd":7,"last update":"Fri Nov 24 07:57:18 2023","interfaces":[{"interface":"back","average":{"1min":0.61799999999999999,"5min":0.61799999999999999,"15min":0.61799999999999999},"min":{"1min":0.49299999999999999,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":0.81499999999999995,"5min":0.81499999999999995,"15min":0.81499999999999995},"last":0.76100000000000001},{"interface":"front","average":{"1min":0.67000000000000004,"5min":0.67000000000000004,"15min":0.67000000000000004},"min":{"1min":0.497,"5min":0.497,"15min":0.497},"max":{"1min":0.92700000000000005,"5min":0.92700000000000005,"15min":0.92700000000000005},"last":0.72799999999999998}]}]},{"osd":1,"up_from":13,"seq":55834574866,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27924,"kb_used_data":896,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343916,"statfs":{"total":96636764160,"available":96608169984,"internally_reserved":0,"allocated":917504,"data_stored":711299,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 07:57:22 2023","interfaces":[{"interface":"back","average":{"1min":0.36399999999999999,"5min":0.36399999999999999,"15min":0.36399999999999999},"min":{"1min":0.20200000000000001,"5min":0.20200000000000001,"15min":0.20200000000000001},"max":{"1min":0.70899999999999996,"5min":0.70899999999999996,"15min":0.70899999999999996},"last":0.251},{"interface":"front","average":{"1min":0.39800000000000002,"5min":0.39800000000000002,"15min":0.39800000000000002},"min":{"1min":0.214,"5min":0.214,"15min":0.214},"max":{"1min":0.82499999999999996,"5min":0.82499999999999996,"15min":0.82499999999999996},"last":0.214}]},{"osd":2,"last update":"Fri Nov 24 07:57:22 2023","interfaces":[{"interface":"back","average":{"1min":0.373,"5min":0.373,"15min":0.373},"min":{"1min":0.22700000000000001,"5min":0.22700000000000001,"15min":0.22700000000000001},"max":{"1min":0.60999999999999999,"5min":0.60999999999999999,"15min":0.60999999999999999},"last":0.23300000000000001},{"interface":"front","average":{"1min":0.41299999999999998,"5min":0.41299999999999998,"15min":0.41299999999999998},"min":{"1min":0.26800000000000002,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":0.621,"5min":0.621,"15min":0.621},"last":0.28599999999999998}]},{"osd":3,"last update":"Fri Nov 24 07:57:22 2023","interfaces":[{"interface":"back","average":{"1min":0.44400000000000001,"5min":0.44400000000000001,"15min":0.44400000000000001},"min":{"1min":0.32100000000000001,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.66500000000000004,"5min":0.66500000000000004,"15min":0.66500000000000004},"last":0.33100000000000002},{"interface":"front","average":{"1min":0.42799999999999999,"5min":0.42799999999999999,"15min":0.42799999999999999},"min":{"1min":0.28499999999999998,"5min":0.28499999999999998,"15min":0.28499999999999998},"max":{"1min":0.76300000000000001,"5min":0.76300000000000001,"15min":0.76300000000000001},"last":0.31900000000000001}]},{"osd":4,"last update":"Fri Nov 24 07:57:22 2023","interfaces":[{"interface":"back","average":{"1min":0.61099999999999999,"5min":0.61099999999999999,"15min":0.61099999999999999},"min":{"1min":0.43099999999999999,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":0.79800000000000004,"5min":0.79800000000000004,"15min":0.79800000000000004},"last":0.495},{"interface":"front","average":{"1min":0.60599999999999998,"5min":0.60599999999999998,"15min":0.60599999999999998},"min":{"1min":0.41999999999999998,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":0.82199999999999995,"5min":0.82199999999999995,"15min":0.82199999999999995},"last":0.60499999999999998}]},{"osd":5,"last update":"Fri Nov 24 07:57:22 2023","interfaces":[{"interface":"back","average":{"1min":0.72499999999999998,"5min":0.72499999999999998,"15min":0.72499999999999998},"min":{"1min":0.37,"5min":0.37,"15min":0.37},"max":{"1min":2.7269999999999999,"5min":2.7269999999999999,"15min":2.7269999999999999},"last":0.42099999999999999},{"interface":"front","average":{"1min":0.64500000000000002,"5min":0.64500000000000002,"15min":0.64500000000000002},"min":{"1min":0.44700000000000001,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":0.84199999999999997,"5min":0.84199999999999997,"15min":0.84199999999999997},"last":0.53100000000000003}]},{"osd":6,"last update":"Fri Nov 24 07:57:22 2023","interfaces":[{"interface":"back","average":{"1min":0.65800000000000003,"5min":0.65800000000000003,"15min":0.65800000000000003},"min":{"1min":0.47499999999999998,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":0.98199999999999998,"5min":0.98199999999999998,"15min":0.98199999999999998},"last":0.57799999999999996},{"interface":"front","average":{"1min":0.61699999999999999,"5min":0.61699999999999999,"15min":0.61699999999999999},"min":{"1min":0.42899999999999999,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":0.90700000000000003,"5min":0.90700000000000003,"15min":0.90700000000000003},"last":0.48199999999999998}]},{"osd":7,"last update":"Fri Nov 24 07:57:22 2023","interfaces":[{"interface":"back","average":{"1min":0.63700000000000001,"5min":0.63700000000000001,"15min":0.63700000000000001},"min":{"1min":0.40999999999999998,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":1.0649999999999999,"5min":1.0649999999999999,"15min":1.0649999999999999},"last":0.45200000000000001},{"interface":"front","average":{"1min":0.67000000000000004,"5min":0.67000000000000004,"15min":0.67000000000000004},"min":{"1min":0.442,"5min":0.442,"15min":0.442},"max":{"1min":0.88800000000000001,"5min":0.88800000000000001,"15min":0.88800000000000001},"last":0.55700000000000005}]}]},{"osd":0,"up_from":13,"seq":55834574866,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27340,"kb_used_data":316,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344500,"statfs":{"total":96636764160,"available":96608768000,"internally_reserved":0,"allocated":323584,"data_stored":120931,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Nov 24 07:57:18 2023","interfaces":[{"interface":"back","average":{"1min":0.71299999999999997,"5min":0.71299999999999997,"15min":0.71299999999999997},"min":{"1min":0.23499999999999999,"5min":0.23499999999999999,"15min":0.23499999999999999},"max":{"1min":6.3550000000000004,"5min":6.3550000000000004,"15min":6.3550000000000004},"last":0.32400000000000001},{"interface":"front","average":{"1min":1.7290000000000001,"5min":1.7290000000000001,"15min":1.7290000000000001},"min":{"1min":0.30599999999999999,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":22.916,"5min":22.916,"15min":22.916},"last":0.36699999999999999}]},{"osd":2,"last update":"Fri Nov 24 07:57:18 2023","interfaces":[{"interface":"back","average":{"1min":0.69999999999999996,"5min":0.69999999999999996,"15min":0.69999999999999996},"min":{"1min":0.22800000000000001,"5min":0.22800000000000001,"15min":0.22800000000000001},"max":{"1min":6.3899999999999997,"5min":6.3899999999999997,"15min":6.3899999999999997},"last":0.379},{"interface":"front","average":{"1min":1.0629999999999999,"5min":1.0629999999999999,"15min":1.0629999999999999},"min":{"1min":0.26800000000000002,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":11.93,"5min":11.93,"15min":11.93},"last":0.30299999999999999}]},{"osd":3,"last update":"Fri Nov 24 07:57:18 2023","interfaces":[{"interface":"back","average":{"1min":0.41999999999999998,"5min":0.41999999999999998,"15min":0.41999999999999998},"min":{"1min":0.28399999999999997,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":1.155,"5min":1.155,"15min":1.155},"last":0.33600000000000002},{"interface":"front","average":{"1min":1.127,"5min":1.127,"15min":1.127},"min":{"1min":0.32300000000000001,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":12.116,"5min":12.116,"15min":12.116},"last":0.35299999999999998}]},{"osd":4,"last update":"Fri Nov 24 07:57:18 2023","interfaces":[{"interface":"back","average":{"1min":0.64800000000000002,"5min":0.64800000000000002,"15min":0.64800000000000002},"min":{"1min":0.40500000000000003,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":0.97899999999999998,"5min":0.97899999999999998,"15min":0.97899999999999998},"last":0.77400000000000002},{"interface":"front","average":{"1min":1.9339999999999999,"5min":1.9339999999999999,"15min":1.9339999999999999},"min":{"1min":0.44400000000000001,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":22.963999999999999,"5min":22.963999999999999,"15min":22.963999999999999},"last":0.55400000000000005}]},{"osd":5,"last update":"Fri Nov 24 07:57:18 2023","interfaces":[{"interface":"back","average":{"1min":1.2929999999999999,"5min":1.2929999999999999,"15min":1.2929999999999999},"min":{"1min":0.40200000000000002,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":12.084,"5min":12.084,"15min":12.084},"last":0.52100000000000002},{"interface":"front","average":{"1min":0.67800000000000005,"5min":0.67800000000000005,"15min":0.67800000000000005},"min":{"1min":0.45200000000000001,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":1.073,"5min":1.073,"15min":1.073},"last":0.73799999999999999}]},{"osd":6,"last update":"Fri Nov 24 07:57:18 2023","interfaces":[{"interface":"back","average":{"1min":0.69099999999999995,"5min":0.69099999999999995,"15min":0.69099999999999995},"min":{"1min":0.42799999999999999,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":1.0580000000000001,"5min":1.0580000000000001,"15min":1.0580000000000001},"last":0.69499999999999995},{"interface":"front","average":{"1min":1.3140000000000001,"5min":1.3140000000000001,"15min":1.3140000000000001},"min":{"1min":0.41899999999999998,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":12.045,"5min":12.045,"15min":12.045},"last":0.54700000000000004}]},{"osd":7,"last update":"Fri Nov 24 07:57:18 2023","interfaces":[{"interface":"back","average":{"1min":0.65600000000000003,"5min":0.65600000000000003,"15min":0.65600000000000003},"min":{"1min":0.45400000000000001,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":1.038,"5min":1.038,"15min":1.038},"last":0.57499999999999996},{"interface":"front","average":{"1min":1.9630000000000001,"5min":1.9630000000000001,"15min":1.9630000000000001},"min":{"1min":0.49099999999999999,"5min":0.49099999999999999,"15min":0.49099999999999999},"max":{"1min":22.995000000000001,"5min":22.995000000000001,"15min":22.995000000000001},"last":0.752}]}]},{"osd":2,"up_from":13,"seq":55834574866,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27336,"kb_used_data":316,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344504,"statfs":{"total":96636764160,"available":96608772096,"internally_reserved":0,"allocated":323584,"data_stored":120931,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 07:57:17 2023","interfaces":[{"interface":"back","average":{"1min":0.371,"5min":0.371,"15min":0.371},"min":{"1min":0.22500000000000001,"5min":0.22500000000000001,"15min":0.22500000000000001},"max":{"1min":0.58099999999999996,"5min":0.58099999999999996,"15min":0.58099999999999996},"last":0.24199999999999999},{"interface":"front","average":{"1min":0.40200000000000002,"5min":0.40200000000000002,"15min":0.40200000000000002},"min":{"1min":0.27400000000000002,"5min":0.27400000000000002,"15min":0.27400000000000002},"max":{"1min":0.54100000000000004,"5min":0.54100000000000004,"15min":0.54100000000000004},"last":0.38600000000000001}]},{"osd":1,"last update":"Fri Nov 24 07:57:17 2023","interfaces":[{"interface":"back","average":{"1min":0.378,"5min":0.378,"15min":0.378},"min":{"1min":0.23499999999999999,"5min":0.23499999999999999,"15min":0.23499999999999999},"max":{"1min":0.59199999999999997,"5min":0.59199999999999997,"15min":0.59199999999999997},"last":0.27600000000000002},{"interface":"front","average":{"1min":0.41599999999999998,"5min":0.41599999999999998,"15min":0.41599999999999998},"min":{"1min":0.25600000000000001,"5min":0.25600000000000001,"15min":0.25600000000000001},"max":{"1min":0.627,"5min":0.627,"15min":0.627},"last":0.30299999999999999}]},{"osd":3,"last update":"Fri Nov 24 07:57:17 2023","interfaces":[{"interface":"back","average":{"1min":0.41099999999999998,"5min":0.41099999999999998,"15min":0.41099999999999998},"min":{"1min":0.308,"5min":0.308,"15min":0.308},"max":{"1min":0.53300000000000003,"5min":0.53300000000000003,"15min":0.53300000000000003},"last":0.371},{"interface":"front","average":{"1min":0.41399999999999998,"5min":0.41399999999999998,"15min":0.41399999999999998},"min":{"1min":0.27500000000000002,"5min":0.27500000000000002,"15min":0.27500000000000002},"max":{"1min":0.64600000000000002,"5min":0.64600000000000002,"15min":0.64600000000000002},"last":0.35399999999999998}]},{"osd":4,"last update":"Fri Nov 24 07:57:17 2023","interfaces":[{"interface":"back","average":{"1min":0.60799999999999998,"5min":0.60799999999999998,"15min":0.60799999999999998},"min":{"1min":0.42999999999999999,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":0.81599999999999995,"5min":0.81599999999999995,"15min":0.81599999999999995},"last":0.51000000000000001},{"interface":"front","average":{"1min":0.59999999999999998,"5min":0.59999999999999998,"15min":0.59999999999999998},"min":{"1min":0.41599999999999998,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":0.79600000000000004,"5min":0.79600000000000004,"15min":0.79600000000000004},"last":0.66600000000000004}]},{"osd":5,"last update":"Fri Nov 24 07:57:17 2023","interfaces":[{"interface":"back","average":{"1min":0.621,"5min":0.621,"15min":0.621},"min":{"1min":0.44900000000000001,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":0.84899999999999998,"5min":0.84899999999999998,"15min":0.84899999999999998},"last":0.52400000000000002},{"interface":"front","average":{"1min":0.71499999999999997,"5min":0.71499999999999997,"15min":0.71499999999999997},"min":{"1min":0.433,"5min":0.433,"15min":0.433},"max":{"1min":2.1699999999999999,"5min":2.1699999999999999,"15min":2.1699999999999999},"last":0.56399999999999995}]},{"osd":6,"last update":"Fri Nov 24 07:57:17 2023","interfaces":[{"interface":"back","average":{"1min":0.66700000000000004,"5min":0.66700000000000004,"15min":0.66700000000000004},"min":{"1min":0.47399999999999998,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":0.83599999999999997,"5min":0.83599999999999997,"15min":0.83599999999999997},"last":0.65800000000000003},{"interface":"front","average":{"1min":0.67100000000000004,"5min":0.67100000000000004,"15min":0.67100000000000004},"min":{"1min":0.496,"5min":0.496,"15min":0.496},"max":{"1min":0.88500000000000001,"5min":0.88500000000000001,"15min":0.88500000000000001},"last":0.58099999999999996}]},{"osd":7,"last update":"Fri Nov 24 07:57:17 2023","interfaces":[{"interface":"back","average":{"1min":0.66200000000000003,"5min":0.66200000000000003,"15min":0.66200000000000003},"min":{"1min":0.443,"5min":0.443,"15min":0.443},"max":{"1min":0.85999999999999999,"5min":0.85999999999999999,"15min":0.85999999999999999},"last":0.60799999999999998},{"interface":"front","average":{"1min":0.68000000000000005,"5min":0.68000000000000005,"15min":0.68000000000000005},"min":{"1min":0.495,"5min":0.495,"15min":0.495},"max":{"1min":0.86399999999999999,"5min":0.86399999999999999,"15min":0.86399999999999999},"last":0.63100000000000001}]}]},{"osd":3,"up_from":13,"seq":55834574866,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27340,"kb_used_data":316,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344500,"statfs":{"total":96636764160,"available":96608768000,"internally_reserved":0,"allocated":323584,"data_stored":120931,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 07:57:20 2023","interfaces":[{"interface":"back","average":{"1min":0.35599999999999998,"5min":0.35599999999999998,"15min":0.35599999999999998},"min":{"1min":0.222,"5min":0.222,"15min":0.222},"max":{"1min":0.84799999999999998,"5min":0.84799999999999998,"15min":0.84799999999999998},"last":0.439},{"interface":"front","average":{"1min":0.36699999999999999,"5min":0.36699999999999999,"15min":0.36699999999999999},"min":{"1min":0.23699999999999999,"5min":0.23699999999999999,"15min":0.23699999999999999},"max":{"1min":0.873,"5min":0.873,"15min":0.873},"last":0.307}]},{"osd":1,"last update":"Fri Nov 24 07:57:20 2023","interfaces":[{"interface":"back","average":{"1min":0.375,"5min":0.375,"15min":0.375},"min":{"1min":0.25700000000000001,"5min":0.25700000000000001,"15min":0.25700000000000001},"max":{"1min":0.92000000000000004,"5min":0.92000000000000004,"15min":0.92000000000000004},"last":0.36899999999999999},{"interface":"front","average":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"min":{"1min":0.248,"5min":0.248,"15min":0.248},"max":{"1min":0.81699999999999995,"5min":0.81699999999999995,"15min":0.81699999999999995},"last":0.34300000000000003}]},{"osd":2,"last update":"Fri Nov 24 07:57:20 2023","interfaces":[{"interface":"back","average":{"1min":0.36099999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"min":{"1min":0.24399999999999999,"5min":0.24399999999999999,"15min":0.24399999999999999},"max":{"1min":0.88500000000000001,"5min":0.88500000000000001,"15min":0.88500000000000001},"last":0.38900000000000001},{"interface":"front","average":{"1min":0.46800000000000003,"5min":0.46800000000000003,"15min":0.46800000000000003},"min":{"1min":0.31900000000000001,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.98099999999999998,"5min":0.98099999999999998,"15min":0.98099999999999998},"last":0.41199999999999998}]},{"osd":4,"last update":"Fri Nov 24 07:57:20 2023","interfaces":[{"interface":"back","average":{"1min":0.79700000000000004,"5min":0.79700000000000004,"15min":0.79700000000000004},"min":{"1min":0.33500000000000002,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":5.7350000000000003,"5min":5.7350000000000003,"15min":5.7350000000000003},"last":0.57499999999999996},{"interface":"front","average":{"1min":0.58499999999999996,"5min":0.58499999999999996,"15min":0.58499999999999996},"min":{"1min":0.44500000000000001,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":1.27,"5min":1.27,"15min":1.27},"last":0.65800000000000003}]},{"osd":5,"last update":"Fri Nov 24 07:57:20 2023","interfaces":[{"interface":"back","average":{"1min":0.58799999999999997,"5min":0.58799999999999997,"15min":0.58799999999999997},"min":{"1min":0.40699999999999997,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":1.4430000000000001,"5min":1.4430000000000001,"15min":1.4430000000000001},"last":0.50700000000000001},{"interface":"front","average":{"1min":0.627,"5min":0.627,"15min":0.627},"min":{"1min":0.377,"5min":0.377,"15min":0.377},"max":{"1min":1.4239999999999999,"5min":1.4239999999999999,"15min":1.4239999999999999},"last":0.55200000000000005}]},{"osd":6,"last update":"Fri Nov 24 07:57:20 2023","interfaces":[{"interface":"back","average":{"1min":0.62,"5min":0.62,"15min":0.62},"min":{"1min":0.434,"5min":0.434,"15min":0.434},"max":{"1min":1.355,"5min":1.355,"15min":1.355},"last":0.49199999999999999},{"interface":"front","average":{"1min":0.84599999999999997,"5min":0.84599999999999997,"15min":0.84599999999999997},"min":{"1min":0.40999999999999998,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":5.6479999999999997,"5min":5.6479999999999997,"15min":5.6479999999999997},"last":0.53100000000000003}]},{"osd":7,"last update":"Fri Nov 24 07:57:20 2023","interfaces":[{"interface":"back","average":{"1min":0.66600000000000004,"5min":0.66600000000000004,"15min":0.66600000000000004},"min":{"1min":0.42299999999999999,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":1.51,"5min":1.51,"15min":1.51},"last":0.48099999999999998},{"interface":"front","average":{"1min":0.84199999999999997,"5min":0.84199999999999997,"15min":0.84199999999999997},"min":{"1min":0.45600000000000002,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":5.7069999999999999,"5min":5.7069999999999999,"15min":5.7069999999999999},"last":0.54000000000000004}]}]},{"osd":4,"up_from":13,"seq":55834574865,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27336,"kb_used_data":316,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344504,"statfs":{"total":96636764160,"available":96608772096,"internally_reserved":0,"allocated":323584,"data_stored":120931,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 07:57:21 2023","interfaces":[{"interface":"back","average":{"1min":0.58099999999999996,"5min":0.58099999999999996,"15min":0.58099999999999996},"min":{"1min":0.37,"5min":0.37,"15min":0.37},"max":{"1min":0.79300000000000004,"5min":0.79300000000000004,"15min":0.79300000000000004},"last":0.58199999999999996},{"interface":"front","average":{"1min":0.54300000000000004,"5min":0.54300000000000004,"15min":0.54300000000000004},"min":{"1min":0.33600000000000002,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":1.0249999999999999,"5min":1.0249999999999999,"15min":1.0249999999999999},"last":0.73899999999999999}]},{"osd":1,"last update":"Fri Nov 24 07:57:21 2023","interfaces":[{"interface":"back","average":{"1min":0.55200000000000005,"5min":0.55200000000000005,"15min":0.55200000000000005},"min":{"1min":0.40000000000000002,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":0.86399999999999999,"5min":0.86399999999999999,"15min":0.86399999999999999},"last":0.60299999999999998},{"interface":"front","average":{"1min":0.55100000000000005,"5min":0.55100000000000005,"15min":0.55100000000000005},"min":{"1min":0.36099999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.98699999999999999,"5min":0.98699999999999999,"15min":0.98699999999999999},"last":0.42399999999999999}]},{"osd":2,"last update":"Fri Nov 24 07:57:21 2023","interfaces":[{"interface":"back","average":{"1min":0.56100000000000005,"5min":0.56100000000000005,"15min":0.56100000000000005},"min":{"1min":0.35799999999999998,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.81100000000000005,"5min":0.81100000000000005,"15min":0.81100000000000005},"last":0.52100000000000002},{"interface":"front","average":{"1min":0.61299999999999999,"5min":0.61299999999999999,"15min":0.61299999999999999},"min":{"1min":0.38100000000000001,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.89200000000000002,"5min":0.89200000000000002,"15min":0.89200000000000002},"last":0.38100000000000001}]},{"osd":3,"last update":"Fri Nov 24 07:57:21 2023","interfaces":[{"interface":"back","average":{"1min":0.58499999999999996,"5min":0.58499999999999996,"15min":0.58499999999999996},"min":{"1min":0.439,"5min":0.439,"15min":0.439},"max":{"1min":0.89700000000000002,"5min":0.89700000000000002,"15min":0.89700000000000002},"last":0.47499999999999998},{"interface":"front","average":{"1min":0.63100000000000001,"5min":0.63100000000000001,"15min":0.63100000000000001},"min":{"1min":0.40300000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.96199999999999997,"5min":0.96199999999999997,"15min":0.96199999999999997},"last":0.71399999999999997}]},{"osd":5,"last update":"Fri Nov 24 07:57:21 2023","interfaces":[{"interface":"back","average":{"1min":0.45500000000000002,"5min":0.45500000000000002,"15min":0.45500000000000002},"min":{"1min":0.29799999999999999,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":0.70499999999999996,"5min":0.70499999999999996,"15min":0.70499999999999996},"last":0.40200000000000002},{"interface":"front","average":{"1min":0.61899999999999999,"5min":0.61899999999999999,"15min":0.61899999999999999},"min":{"1min":0.439,"5min":0.439,"15min":0.439},"max":{"1min":0.86399999999999999,"5min":0.86399999999999999,"15min":0.86399999999999999},"last":0.68600000000000005}]},{"osd":6,"last update":"Fri Nov 24 07:57:21 2023","interfaces":[{"interface":"back","average":{"1min":0.58599999999999997,"5min":0.58599999999999997,"15min":0.58599999999999997},"min":{"1min":0.35499999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.76800000000000002,"5min":0.76800000000000002,"15min":0.76800000000000002},"last":0.67100000000000004},{"interface":"front","average":{"1min":0.61399999999999999,"5min":0.61399999999999999,"15min":0.61399999999999999},"min":{"1min":0.436,"5min":0.436,"15min":0.436},"max":{"1min":1.095,"5min":1.095,"15min":1.095},"last":0.78600000000000003}]},{"osd":7,"last update":"Fri Nov 24 07:57:21 2023","interfaces":[{"interface":"back","average":{"1min":0.57299999999999995,"5min":0.57299999999999995,"15min":0.57299999999999995},"min":{"1min":0.35099999999999998,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.89300000000000002,"5min":0.89300000000000002,"15min":0.89300000000000002},"last":0.63700000000000001},{"interface":"front","average":{"1min":0.66300000000000003,"5min":0.66300000000000003,"15min":0.66300000000000003},"min":{"1min":0.41099999999999998,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":0.91500000000000004,"5min":0.91500000000000004,"15min":0.91500000000000004},"last":0.83599999999999997}]}]},{"osd":5,"up_from":13,"seq":55834574866,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27928,"kb_used_data":896,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343912,"statfs":{"total":96636764160,"available":96608165888,"internally_reserved":0,"allocated":917504,"data_stored":711299,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 07:57:17 2023","interfaces":[{"interface":"back","average":{"1min":0.56200000000000006,"5min":0.56200000000000006,"15min":0.56200000000000006},"min":{"1min":0.32700000000000001,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":1.0109999999999999,"5min":1.0109999999999999,"15min":1.0109999999999999},"last":0.85999999999999999},{"interface":"front","average":{"1min":0.80000000000000004,"5min":0.80000000000000004,"15min":0.80000000000000004},"min":{"1min":0.33400000000000002,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":5.431,"5min":5.431,"15min":5.431},"last":0.58899999999999997}]},{"osd":1,"last update":"Fri Nov 24 07:57:17 2023","interfaces":[{"interface":"back","average":{"1min":0.55900000000000005,"5min":0.55900000000000005,"15min":0.55900000000000005},"min":{"1min":0.28899999999999998,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":1.016,"5min":1.016,"15min":1.016},"last":0.67300000000000004},{"interface":"front","average":{"1min":0.56999999999999995,"5min":0.56999999999999995,"15min":0.56999999999999995},"min":{"1min":0.311,"5min":0.311,"15min":0.311},"max":{"1min":0.96499999999999997,"5min":0.96499999999999997,"15min":0.96499999999999997},"last":0.54100000000000004}]},{"osd":2,"last update":"Fri Nov 24 07:57:17 2023","interfaces":[{"interface":"back","average":{"1min":0.55100000000000005,"5min":0.55100000000000005,"15min":0.55100000000000005},"min":{"1min":0.377,"5min":0.377,"15min":0.377},"max":{"1min":1.163,"5min":1.163,"15min":1.163},"last":0.443},{"interface":"front","average":{"1min":0.63,"5min":0.63,"15min":0.63},"min":{"1min":0.39200000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":1.119,"5min":1.119,"15min":1.119},"last":0.75700000000000001}]},{"osd":3,"last update":"Fri Nov 24 07:57:17 2023","interfaces":[{"interface":"back","average":{"1min":0.90500000000000003,"5min":0.90500000000000003,"15min":0.90500000000000003},"min":{"1min":0.36499999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":5.609,"5min":5.609,"15min":5.609},"last":0.64000000000000001},{"interface":"front","average":{"1min":0.67600000000000005,"5min":0.67600000000000005,"15min":0.67600000000000005},"min":{"1min":0.41899999999999998,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":1.288,"5min":1.288,"15min":1.288},"last":0.83499999999999996}]},{"osd":4,"last update":"Fri Nov 24 07:57:17 2023","interfaces":[{"interface":"back","average":{"1min":0.51000000000000001,"5min":0.51000000000000001,"15min":0.51000000000000001},"min":{"1min":0.36399999999999999,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.68799999999999994,"5min":0.68799999999999994,"15min":0.68799999999999994},"last":0.623},{"interface":"front","average":{"1min":0.57199999999999995,"5min":0.57199999999999995,"15min":0.57199999999999995},"min":{"1min":0.39800000000000002,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.78300000000000003,"5min":0.78300000000000003,"15min":0.78300000000000003},"last":0.78300000000000003}]},{"osd":6,"last update":"Fri Nov 24 07:57:17 2023","interfaces":[{"interface":"back","average":{"1min":0.57099999999999995,"5min":0.57099999999999995,"15min":0.57099999999999995},"min":{"1min":0.35599999999999998,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.89700000000000002,"5min":0.89700000000000002,"15min":0.89700000000000002},"last":0.89700000000000002},{"interface":"front","average":{"1min":0.625,"5min":0.625,"15min":0.625},"min":{"1min":0.42899999999999999,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":0.88200000000000001,"5min":0.88200000000000001,"15min":0.88200000000000001},"last":0.88200000000000001}]},{"osd":7,"last update":"Fri Nov 24 07:57:17 2023","interfaces":[{"interface":"back","average":{"1min":0.64600000000000002,"5min":0.64600000000000002,"15min":0.64600000000000002},"min":{"1min":0.45000000000000001,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":0.96599999999999997,"5min":0.96599999999999997,"15min":0.96599999999999997},"last":0.91000000000000003},{"interface":"front","average":{"1min":0.65300000000000002,"5min":0.65300000000000002,"15min":0.65300000000000002},"min":{"1min":0.46300000000000002,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":0.94199999999999995,"5min":0.94199999999999995,"15min":0.94199999999999995},"last":0.94199999999999995}]}]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-11-24T07:57:29.803 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-11-24T07:57:29.963 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:29.961+0000 7f502acf1700 1 Processor -- start 2023-11-24T07:57:29.963 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:29.961+0000 7f502acf1700 1 -- start start 2023-11-24T07:57:29.965 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:29.962+0000 7f502acf1700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f5024143ae0 0x7f50240b51b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:57:29.966 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:29.962+0000 7f502acf1700 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5024143ec0 con 0x7f50240722c0 2023-11-24T07:57:29.966 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:29.962+0000 7f502acf1700 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f50240b3ae0 con 0x7f50241440f0 2023-11-24T07:57:29.966 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:29.962+0000 7f502acf1700 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f50240b40a0 con 0x7f5024143ae0 2023-11-24T07:57:29.966 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:29.962+0000 7f5028a8d700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f5024143ae0 0x7f50240b51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:57:29.966 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:29.962+0000 7f5028a8d700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f5024143ae0 0x7f50240b51b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:51766/0 (socket says 172.21.15.59:51766) 2023-11-24T07:57:29.966 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:29.962+0000 7f5028a8d700 1 -- 172.21.15.59:0/107998136 learned_addr learned my addr 172.21.15.59:0/107998136 (peer_addr_for_me v2:172.21.15.59:0/0) 2023-11-24T07:57:29.966 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:29.962+0000 7f502928e700 1 --1- 172.21.15.59:0/107998136 >> v1:172.21.15.59:6789/0 conn(0x7f50240722c0 0x7f502406e510 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:55240/0 (socket says 172.21.15.59:55240) 2023-11-24T07:57:29.966 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:29.963+0000 7f5028a8d700 1 -- 172.21.15.59:0/107998136 >> v1:172.21.15.76:6789/0 conn(0x7f50241440f0 legacy=0x7f5024071bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:57:29.966 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:29.963+0000 7f5028a8d700 1 -- 172.21.15.59:0/107998136 >> v1:172.21.15.59:6789/0 conn(0x7f50240722c0 legacy=0x7f502406e510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:57:29.966 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:29.963+0000 7f50237fe700 1 -- 172.21.15.59:0/107998136 <== mon.0 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3098052346 0 0) 0x7f5024143ec0 con 0x7f50240722c0 2023-11-24T07:57:29.966 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:29.963+0000 7f5028a8d700 1 -- 172.21.15.59:0/107998136 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f50240b3dc0 con 0x7f5024143ae0 2023-11-24T07:57:29.967 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:29.963+0000 7f5028a8d700 1 --2- 172.21.15.59:0/107998136 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5024143ae0 0x7f50240b51b0 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f50180098f0 tx=0x7f5018009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=431e877d06296c94 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:57:29.967 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:29.963+0000 7f50237fe700 1 -- 172.21.15.59:0/107998136 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f501800b8a0 con 0x7f5024143ae0 2023-11-24T07:57:29.967 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:29.963+0000 7f50237fe700 1 -- 172.21.15.59:0/107998136 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f501800ba00 con 0x7f5024143ae0 2023-11-24T07:57:29.967 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:29.963+0000 7f50237fe700 1 -- 172.21.15.59:0/107998136 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f501800bd90 con 0x7f5024143ae0 2023-11-24T07:57:29.967 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:29.964+0000 7f502acf1700 1 -- 172.21.15.59:0/107998136 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5024143ae0 msgr2=0x7f50240b51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:57:29.967 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:29.964+0000 7f502acf1700 1 --2- 172.21.15.59:0/107998136 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5024143ae0 0x7f50240b51b0 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f50180098f0 tx=0x7f5018009c00 comp rx=0 tx=0).stop 2023-11-24T07:57:29.967 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:29.964+0000 7f502acf1700 1 -- 172.21.15.59:0/107998136 shutdown_connections 2023-11-24T07:57:29.967 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:29.964+0000 7f502acf1700 1 --2- 172.21.15.59:0/107998136 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5024143ae0 0x7f50240b51b0 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:29.967 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:29.964+0000 7f502acf1700 1 -- 172.21.15.59:0/107998136 >> 172.21.15.59:0/107998136 conn(0x7f50240b2740 msgr2=0x7f50240b2b20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:57:29.967 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:29.964+0000 7f502acf1700 1 -- 172.21.15.59:0/107998136 shutdown_connections 2023-11-24T07:57:29.967 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:29.964+0000 7f502acf1700 1 -- 172.21.15.59:0/107998136 wait complete. 2023-11-24T07:57:29.969 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:29.965+0000 7f502acf1700 1 Processor -- start 2023-11-24T07:57:29.970 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:29.965+0000 7f502acf1700 1 -- start start 2023-11-24T07:57:29.970 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:29.966+0000 7f502acf1700 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f50240722c0 0x7f5024133c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:57:29.970 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:29.966+0000 7f502acf1700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5024143ae0 0x7f50241598f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:57:29.970 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:29.966+0000 7f502acf1700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f50241440f0 0x7f50241424d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:57:29.970 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:29.966+0000 7f502acf1700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50240c9a20 con 0x7f5024143ae0 2023-11-24T07:57:29.970 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:29.966+0000 7f502acf1700 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f502413fb40 con 0x7f50241440f0 2023-11-24T07:57:29.970 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:29.966+0000 7f502acf1700 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f50240c8da0 con 0x7f50240722c0 2023-11-24T07:57:29.970 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:29.967+0000 7f5023fff700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5024143ae0 0x7f50241598f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:57:29.970 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:29.967+0000 7f5023fff700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5024143ae0 0x7f50241598f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:51774/0 (socket says 172.21.15.59:51774) 2023-11-24T07:57:29.971 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:29.967+0000 7f5023fff700 1 -- 172.21.15.59:0/1177454801 learned_addr learned my addr 172.21.15.59:0/1177454801 (peer_addr_for_me v2:172.21.15.59:0/0) 2023-11-24T07:57:29.971 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:29.967+0000 7f502928e700 1 --2- 172.21.15.59:0/1177454801 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f50241440f0 0x7f50241424d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:57:29.971 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:29.967+0000 7f5023fff700 1 -- 172.21.15.59:0/1177454801 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f50240722c0 msgr2=0x7f5024133c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:57:29.971 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:29.967+0000 7f5023fff700 1 --2- 172.21.15.59:0/1177454801 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f50240722c0 0x7f5024133c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:29.971 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:29.967+0000 7f5023fff700 1 -- 172.21.15.59:0/1177454801 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f50241440f0 msgr2=0x7f50241424d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:57:29.971 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:29.967+0000 7f5023fff700 1 --2- 172.21.15.59:0/1177454801 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f50241440f0 0x7f50241424d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:29.971 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:29.967+0000 7f5023fff700 1 -- 172.21.15.59:0/1177454801 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5018009550 con 0x7f5024143ae0 2023-11-24T07:57:29.971 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:29.967+0000 7f502928e700 1 --2- 172.21.15.59:0/1177454801 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f50241440f0 0x7f50241424d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T07:57:29.971 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:29.967+0000 7f5023fff700 1 --2- 172.21.15.59:0/1177454801 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5024143ae0 0x7f50241598f0 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f501000c450 tx=0x7f501000c810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:57:29.971 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:29.967+0000 7f50217fa700 1 -- 172.21.15.59:0/1177454801 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50100052d0 con 0x7f5024143ae0 2023-11-24T07:57:29.971 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:29.967+0000 7f50217fa700 1 -- 172.21.15.59:0/1177454801 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5010005430 con 0x7f5024143ae0 2023-11-24T07:57:29.972 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:29.967+0000 7f50217fa700 1 -- 172.21.15.59:0/1177454801 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50100038c0 con 0x7f5024143ae0 2023-11-24T07:57:29.972 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:29.967+0000 7f502acf1700 1 -- 172.21.15.59:0/1177454801 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f50240c9080 con 0x7f5024143ae0 2023-11-24T07:57:29.972 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:29.967+0000 7f502acf1700 1 -- 172.21.15.59:0/1177454801 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f50240c9640 con 0x7f5024143ae0 2023-11-24T07:57:29.972 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:29.968+0000 7f502acf1700 1 -- 172.21.15.59:0/1177454801 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f50240c9b60 con 0x7f5024143ae0 2023-11-24T07:57:29.973 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:29.969+0000 7f50217fa700 1 -- 172.21.15.59:0/1177454801 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f5010003a20 con 0x7f5024143ae0 2023-11-24T07:57:29.973 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:29.969+0000 7f50217fa700 1 --2- 172.21.15.59:0/1177454801 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f500806aed0 0x7f500806d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:57:29.973 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:29.969+0000 7f50217fa700 1 -- 172.21.15.59:0/1177454801 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f50100889e0 con 0x7f5024143ae0 2023-11-24T07:57:29.973 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:29.969+0000 7f5028a8d700 1 --2- 172.21.15.59:0/1177454801 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f500806aed0 0x7f500806d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:57:29.973 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:29.970+0000 7f5028a8d700 1 --2- 172.21.15.59:0/1177454801 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f500806aed0 0x7f500806d390 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f5018009130 tx=0x7f5018010000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:57:29.975 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:29.974+0000 7f50217fa700 1 -- 172.21.15.59:0/1177454801 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f50240c9b60 con 0x7f5024143ae0 2023-11-24T07:57:30.170 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:30.168+0000 7f502acf1700 1 -- 172.21.15.59:0/1177454801 --> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f5024144860 con 0x7f500806aed0 2023-11-24T07:57:30.178 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:30.177+0000 7f50217fa700 1 -- 172.21.15.59:0/1177454801 <== mgr.4100 v2:172.21.15.76:6832/4264046536 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+274252 (secure 0 0 0) 0x7f501801f070 con 0x7f500806aed0 2023-11-24T07:57:30.181 INFO:teuthology.orchestra.run.smithi059.stdout: 2023-11-24T07:57:30.183 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-24T07:57:30.184 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:30.179+0000 7f502acf1700 1 -- 172.21.15.59:0/1177454801 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f500806aed0 msgr2=0x7f500806d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:57:30.184 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:30.179+0000 7f502acf1700 1 --2- 172.21.15.59:0/1177454801 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f500806aed0 0x7f500806d390 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f5018009130 tx=0x7f5018010000 comp rx=0 tx=0).stop 2023-11-24T07:57:30.184 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:30.179+0000 7f502acf1700 1 -- 172.21.15.59:0/1177454801 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5024143ae0 msgr2=0x7f50241598f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:57:30.184 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:30.179+0000 7f502acf1700 1 --2- 172.21.15.59:0/1177454801 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5024143ae0 0x7f50241598f0 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f501000c450 tx=0x7f501000c810 comp rx=0 tx=0).stop 2023-11-24T07:57:30.184 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:30.179+0000 7f502acf1700 1 -- 172.21.15.59:0/1177454801 shutdown_connections 2023-11-24T07:57:30.184 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:30.179+0000 7f502acf1700 1 --2- 172.21.15.59:0/1177454801 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f500806aed0 0x7f500806d390 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:30.184 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:30.179+0000 7f502acf1700 1 --2- 172.21.15.59:0/1177454801 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f50241440f0 0x7f50241424d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:30.184 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:30.179+0000 7f502acf1700 1 --2- 172.21.15.59:0/1177454801 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f50240722c0 0x7f5024133c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:30.184 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:30.179+0000 7f502acf1700 1 --2- 172.21.15.59:0/1177454801 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5024143ae0 0x7f50241598f0 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:30.185 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:30.179+0000 7f502acf1700 1 -- 172.21.15.59:0/1177454801 >> 172.21.15.59:0/1177454801 conn(0x7f50240b2740 msgr2=0x7f502415ed60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:57:30.185 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:30.179+0000 7f502acf1700 1 -- 172.21.15.59:0/1177454801 shutdown_connections 2023-11-24T07:57:30.185 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:30.179+0000 7f502acf1700 1 -- 172.21.15.59:0/1177454801 wait complete. 2023-11-24T07:57:30.185 INFO:teuthology.orchestra.run.smithi059.stderr:dumped all 2023-11-24T07:57:30.187 DEBUG:teuthology.orchestra.run.smithi076:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-24T07:57:30.194 INFO:teuthology.orchestra.run.smithi059.stdout:{"pg_ready":true,"pg_map":{"version":61,"stamp":"2023-11-24T07:57:28.313566+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":2,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":2,"kb":754974720,"kb_used":219892,"kb_used_data":3688,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754754828,"statfs":{"total":773094113280,"available":772868943872,"internally_reserved":0,"allocated":3776512,"data_stored":2148184,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12745,"internal_metadata":221236791},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.740627"},"pg_stats":[{"pgid":"5.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T07:57:24.572681+0000","last_change":"2023-11-24T07:57:24.572681+0000","last_active":"2023-11-24T07:57:24.572681+0000","last_peered":"2023-11-24T07:57:24.572681+0000","last_clean":"2023-11-24T07:57:24.572681+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T07:57:24.572681+0000","last_undegraded":"2023-11-24T07:57:24.572681+0000","last_fullsized":"2023-11-24T07:57:24.572681+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:57:24.572681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:57:24.572681+0000","last_clean_scrub_stamp":"2023-11-24T07:57:24.572681+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T07:57:24.572681+0000","last_change":"2023-11-24T07:57:24.572681+0000","last_active":"2023-11-24T07:57:24.572681+0000","last_peered":"2023-11-24T07:57:24.572681+0000","last_clean":"2023-11-24T07:57:24.572681+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T07:57:24.572681+0000","last_undegraded":"2023-11-24T07:57:24.572681+0000","last_fullsized":"2023-11-24T07:57:24.572681+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:57:24.572681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:57:24.572681+0000","last_clean_scrub_stamp":"2023-11-24T07:57:24.572681+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T07:57:24.572681+0000","last_change":"2023-11-24T07:57:24.572681+0000","last_active":"2023-11-24T07:57:24.572681+0000","last_peered":"2023-11-24T07:57:24.572681+0000","last_clean":"2023-11-24T07:57:24.572681+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T07:57:24.572681+0000","last_undegraded":"2023-11-24T07:57:24.572681+0000","last_fullsized":"2023-11-24T07:57:24.572681+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:57:24.572681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:57:24.572681+0000","last_clean_scrub_stamp":"2023-11-24T07:57:24.572681+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T07:57:24.572681+0000","last_change":"2023-11-24T07:57:24.572681+0000","last_active":"2023-11-24T07:57:24.572681+0000","last_peered":"2023-11-24T07:57:24.572681+0000","last_clean":"2023-11-24T07:57:24.572681+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T07:57:24.572681+0000","last_undegraded":"2023-11-24T07:57:24.572681+0000","last_fullsized":"2023-11-24T07:57:24.572681+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:57:24.572681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:57:24.572681+0000","last_clean_scrub_stamp":"2023-11-24T07:57:24.572681+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T07:57:24.572681+0000","last_change":"2023-11-24T07:57:24.572681+0000","last_active":"2023-11-24T07:57:24.572681+0000","last_peered":"2023-11-24T07:57:24.572681+0000","last_clean":"2023-11-24T07:57:24.572681+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T07:57:24.572681+0000","last_undegraded":"2023-11-24T07:57:24.572681+0000","last_fullsized":"2023-11-24T07:57:24.572681+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:57:24.572681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:57:24.572681+0000","last_clean_scrub_stamp":"2023-11-24T07:57:24.572681+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T07:57:24.572681+0000","last_change":"2023-11-24T07:57:24.572681+0000","last_active":"2023-11-24T07:57:24.572681+0000","last_peered":"2023-11-24T07:57:24.572681+0000","last_clean":"2023-11-24T07:57:24.572681+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T07:57:24.572681+0000","last_undegraded":"2023-11-24T07:57:24.572681+0000","last_fullsized":"2023-11-24T07:57:24.572681+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:57:24.572681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:57:24.572681+0000","last_clean_scrub_stamp":"2023-11-24T07:57:24.572681+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T07:57:24.572681+0000","last_change":"2023-11-24T07:57:24.572681+0000","last_active":"2023-11-24T07:57:24.572681+0000","last_peered":"2023-11-24T07:57:24.572681+0000","last_clean":"2023-11-24T07:57:24.572681+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T07:57:24.572681+0000","last_undegraded":"2023-11-24T07:57:24.572681+0000","last_fullsized":"2023-11-24T07:57:24.572681+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:57:24.572681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:57:24.572681+0000","last_clean_scrub_stamp":"2023-11-24T07:57:24.572681+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T07:57:24.572681+0000","last_change":"2023-11-24T07:57:24.572681+0000","last_active":"2023-11-24T07:57:24.572681+0000","last_peered":"2023-11-24T07:57:24.572681+0000","last_clean":"2023-11-24T07:57:24.572681+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T07:57:24.572681+0000","last_undegraded":"2023-11-24T07:57:24.572681+0000","last_fullsized":"2023-11-24T07:57:24.572681+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:57:24.572681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:57:24.572681+0000","last_clean_scrub_stamp":"2023-11-24T07:57:24.572681+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T07:57:24.572681+0000","last_change":"2023-11-24T07:57:24.572681+0000","last_active":"2023-11-24T07:57:24.572681+0000","last_peered":"2023-11-24T07:57:24.572681+0000","last_clean":"2023-11-24T07:57:24.572681+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T07:57:24.572681+0000","last_undegraded":"2023-11-24T07:57:24.572681+0000","last_fullsized":"2023-11-24T07:57:24.572681+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:57:24.572681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:57:24.572681+0000","last_clean_scrub_stamp":"2023-11-24T07:57:24.572681+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T07:57:24.572681+0000","last_change":"2023-11-24T07:57:24.572681+0000","last_active":"2023-11-24T07:57:24.572681+0000","last_peered":"2023-11-24T07:57:24.572681+0000","last_clean":"2023-11-24T07:57:24.572681+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T07:57:24.572681+0000","last_undegraded":"2023-11-24T07:57:24.572681+0000","last_fullsized":"2023-11-24T07:57:24.572681+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:57:24.572681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:57:24.572681+0000","last_clean_scrub_stamp":"2023-11-24T07:57:24.572681+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T07:57:24.572681+0000","last_change":"2023-11-24T07:57:24.572681+0000","last_active":"2023-11-24T07:57:24.572681+0000","last_peered":"2023-11-24T07:57:24.572681+0000","last_clean":"2023-11-24T07:57:24.572681+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T07:57:24.572681+0000","last_undegraded":"2023-11-24T07:57:24.572681+0000","last_fullsized":"2023-11-24T07:57:24.572681+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:57:24.572681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:57:24.572681+0000","last_clean_scrub_stamp":"2023-11-24T07:57:24.572681+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T07:57:24.572681+0000","last_change":"2023-11-24T07:57:24.572681+0000","last_active":"2023-11-24T07:57:24.572681+0000","last_peered":"2023-11-24T07:57:24.572681+0000","last_clean":"2023-11-24T07:57:24.572681+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T07:57:24.572681+0000","last_undegraded":"2023-11-24T07:57:24.572681+0000","last_fullsized":"2023-11-24T07:57:24.572681+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:57:24.572681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:57:24.572681+0000","last_clean_scrub_stamp":"2023-11-24T07:57:24.572681+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T07:57:24.572681+0000","last_change":"2023-11-24T07:57:24.572681+0000","last_active":"2023-11-24T07:57:24.572681+0000","last_peered":"2023-11-24T07:57:24.572681+0000","last_clean":"2023-11-24T07:57:24.572681+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T07:57:24.572681+0000","last_undegraded":"2023-11-24T07:57:24.572681+0000","last_fullsized":"2023-11-24T07:57:24.572681+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:57:24.572681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:57:24.572681+0000","last_clean_scrub_stamp":"2023-11-24T07:57:24.572681+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T07:57:24.572681+0000","last_change":"2023-11-24T07:57:24.572681+0000","last_active":"2023-11-24T07:57:24.572681+0000","last_peered":"2023-11-24T07:57:24.572681+0000","last_clean":"2023-11-24T07:57:24.572681+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T07:57:24.572681+0000","last_undegraded":"2023-11-24T07:57:24.572681+0000","last_fullsized":"2023-11-24T07:57:24.572681+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:57:24.572681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:57:24.572681+0000","last_clean_scrub_stamp":"2023-11-24T07:57:24.572681+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T07:57:24.572681+0000","last_change":"2023-11-24T07:57:24.572681+0000","last_active":"2023-11-24T07:57:24.572681+0000","last_peered":"2023-11-24T07:57:24.572681+0000","last_clean":"2023-11-24T07:57:24.572681+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T07:57:24.572681+0000","last_undegraded":"2023-11-24T07:57:24.572681+0000","last_fullsized":"2023-11-24T07:57:24.572681+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:57:24.572681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:57:24.572681+0000","last_clean_scrub_stamp":"2023-11-24T07:57:24.572681+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T07:57:24.572681+0000","last_change":"2023-11-24T07:57:24.572681+0000","last_active":"2023-11-24T07:57:24.572681+0000","last_peered":"2023-11-24T07:57:24.572681+0000","last_clean":"2023-11-24T07:57:24.572681+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T07:57:24.572681+0000","last_undegraded":"2023-11-24T07:57:24.572681+0000","last_fullsized":"2023-11-24T07:57:24.572681+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:57:24.572681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:57:24.572681+0000","last_clean_scrub_stamp":"2023-11-24T07:57:24.572681+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T07:57:24.572681+0000","last_change":"2023-11-24T07:57:24.572681+0000","last_active":"2023-11-24T07:57:24.572681+0000","last_peered":"2023-11-24T07:57:24.572681+0000","last_clean":"2023-11-24T07:57:24.572681+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T07:57:24.572681+0000","last_undegraded":"2023-11-24T07:57:24.572681+0000","last_fullsized":"2023-11-24T07:57:24.572681+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:57:24.572681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:57:24.572681+0000","last_clean_scrub_stamp":"2023-11-24T07:57:24.572681+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T07:57:24.572681+0000","last_change":"2023-11-24T07:57:24.572681+0000","last_active":"2023-11-24T07:57:24.572681+0000","last_peered":"2023-11-24T07:57:24.572681+0000","last_clean":"2023-11-24T07:57:24.572681+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T07:57:24.572681+0000","last_undegraded":"2023-11-24T07:57:24.572681+0000","last_fullsized":"2023-11-24T07:57:24.572681+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:57:24.572681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:57:24.572681+0000","last_clean_scrub_stamp":"2023-11-24T07:57:24.572681+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T07:57:24.572681+0000","last_change":"2023-11-24T07:57:24.572681+0000","last_active":"2023-11-24T07:57:24.572681+0000","last_peered":"2023-11-24T07:57:24.572681+0000","last_clean":"2023-11-24T07:57:24.572681+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T07:57:24.572681+0000","last_undegraded":"2023-11-24T07:57:24.572681+0000","last_fullsized":"2023-11-24T07:57:24.572681+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:57:24.572681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:57:24.572681+0000","last_clean_scrub_stamp":"2023-11-24T07:57:24.572681+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T07:57:24.572681+0000","last_change":"2023-11-24T07:57:24.572681+0000","last_active":"2023-11-24T07:57:24.572681+0000","last_peered":"2023-11-24T07:57:24.572681+0000","last_clean":"2023-11-24T07:57:24.572681+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T07:57:24.572681+0000","last_undegraded":"2023-11-24T07:57:24.572681+0000","last_fullsized":"2023-11-24T07:57:24.572681+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:57:24.572681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:57:24.572681+0000","last_clean_scrub_stamp":"2023-11-24T07:57:24.572681+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T07:57:24.572681+0000","last_change":"2023-11-24T07:57:24.572681+0000","last_active":"2023-11-24T07:57:24.572681+0000","last_peered":"2023-11-24T07:57:24.572681+0000","last_clean":"2023-11-24T07:57:24.572681+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T07:57:24.572681+0000","last_undegraded":"2023-11-24T07:57:24.572681+0000","last_fullsized":"2023-11-24T07:57:24.572681+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:57:24.572681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:57:24.572681+0000","last_clean_scrub_stamp":"2023-11-24T07:57:24.572681+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T07:57:24.572681+0000","last_change":"2023-11-24T07:57:24.572681+0000","last_active":"2023-11-24T07:57:24.572681+0000","last_peered":"2023-11-24T07:57:24.572681+0000","last_clean":"2023-11-24T07:57:24.572681+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T07:57:24.572681+0000","last_undegraded":"2023-11-24T07:57:24.572681+0000","last_fullsized":"2023-11-24T07:57:24.572681+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:57:24.572681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:57:24.572681+0000","last_clean_scrub_stamp":"2023-11-24T07:57:24.572681+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T07:57:24.572681+0000","last_change":"2023-11-24T07:57:24.572681+0000","last_active":"2023-11-24T07:57:24.572681+0000","last_peered":"2023-11-24T07:57:24.572681+0000","last_clean":"2023-11-24T07:57:24.572681+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T07:57:24.572681+0000","last_undegraded":"2023-11-24T07:57:24.572681+0000","last_fullsized":"2023-11-24T07:57:24.572681+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:57:24.572681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:57:24.572681+0000","last_clean_scrub_stamp":"2023-11-24T07:57:24.572681+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T07:57:24.572681+0000","last_change":"2023-11-24T07:57:24.572681+0000","last_active":"2023-11-24T07:57:24.572681+0000","last_peered":"2023-11-24T07:57:24.572681+0000","last_clean":"2023-11-24T07:57:24.572681+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T07:57:24.572681+0000","last_undegraded":"2023-11-24T07:57:24.572681+0000","last_fullsized":"2023-11-24T07:57:24.572681+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:57:24.572681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:57:24.572681+0000","last_clean_scrub_stamp":"2023-11-24T07:57:24.572681+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T07:57:24.572681+0000","last_change":"2023-11-24T07:57:24.572681+0000","last_active":"2023-11-24T07:57:24.572681+0000","last_peered":"2023-11-24T07:57:24.572681+0000","last_clean":"2023-11-24T07:57:24.572681+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T07:57:24.572681+0000","last_undegraded":"2023-11-24T07:57:24.572681+0000","last_fullsized":"2023-11-24T07:57:24.572681+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:57:24.572681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:57:24.572681+0000","last_clean_scrub_stamp":"2023-11-24T07:57:24.572681+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T07:57:24.572681+0000","last_change":"2023-11-24T07:57:24.572681+0000","last_active":"2023-11-24T07:57:24.572681+0000","last_peered":"2023-11-24T07:57:24.572681+0000","last_clean":"2023-11-24T07:57:24.572681+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T07:57:24.572681+0000","last_undegraded":"2023-11-24T07:57:24.572681+0000","last_fullsized":"2023-11-24T07:57:24.572681+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:57:24.572681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:57:24.572681+0000","last_clean_scrub_stamp":"2023-11-24T07:57:24.572681+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T07:57:24.572681+0000","last_change":"2023-11-24T07:57:24.572681+0000","last_active":"2023-11-24T07:57:24.572681+0000","last_peered":"2023-11-24T07:57:24.572681+0000","last_clean":"2023-11-24T07:57:24.572681+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T07:57:24.572681+0000","last_undegraded":"2023-11-24T07:57:24.572681+0000","last_fullsized":"2023-11-24T07:57:24.572681+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:57:24.572681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:57:24.572681+0000","last_clean_scrub_stamp":"2023-11-24T07:57:24.572681+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T07:57:24.572681+0000","last_change":"2023-11-24T07:57:24.572681+0000","last_active":"2023-11-24T07:57:24.572681+0000","last_peered":"2023-11-24T07:57:24.572681+0000","last_clean":"2023-11-24T07:57:24.572681+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T07:57:24.572681+0000","last_undegraded":"2023-11-24T07:57:24.572681+0000","last_fullsized":"2023-11-24T07:57:24.572681+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:57:24.572681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:57:24.572681+0000","last_clean_scrub_stamp":"2023-11-24T07:57:24.572681+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T07:57:24.572681+0000","last_change":"2023-11-24T07:57:24.572681+0000","last_active":"2023-11-24T07:57:24.572681+0000","last_peered":"2023-11-24T07:57:24.572681+0000","last_clean":"2023-11-24T07:57:24.572681+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T07:57:24.572681+0000","last_undegraded":"2023-11-24T07:57:24.572681+0000","last_fullsized":"2023-11-24T07:57:24.572681+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:57:24.572681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:57:24.572681+0000","last_clean_scrub_stamp":"2023-11-24T07:57:24.572681+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T07:57:24.572681+0000","last_change":"2023-11-24T07:57:24.572681+0000","last_active":"2023-11-24T07:57:24.572681+0000","last_peered":"2023-11-24T07:57:24.572681+0000","last_clean":"2023-11-24T07:57:24.572681+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T07:57:24.572681+0000","last_undegraded":"2023-11-24T07:57:24.572681+0000","last_fullsized":"2023-11-24T07:57:24.572681+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:57:24.572681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:57:24.572681+0000","last_clean_scrub_stamp":"2023-11-24T07:57:24.572681+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T07:57:24.572681+0000","last_change":"2023-11-24T07:57:24.572681+0000","last_active":"2023-11-24T07:57:24.572681+0000","last_peered":"2023-11-24T07:57:24.572681+0000","last_clean":"2023-11-24T07:57:24.572681+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T07:57:24.572681+0000","last_undegraded":"2023-11-24T07:57:24.572681+0000","last_fullsized":"2023-11-24T07:57:24.572681+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:57:24.572681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:57:24.572681+0000","last_clean_scrub_stamp":"2023-11-24T07:57:24.572681+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T07:57:24.572681+0000","last_change":"2023-11-24T07:57:24.572681+0000","last_active":"2023-11-24T07:57:24.572681+0000","last_peered":"2023-11-24T07:57:24.572681+0000","last_clean":"2023-11-24T07:57:24.572681+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T07:57:24.572681+0000","last_undegraded":"2023-11-24T07:57:24.572681+0000","last_fullsized":"2023-11-24T07:57:24.572681+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:57:24.572681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:57:24.572681+0000","last_clean_scrub_stamp":"2023-11-24T07:57:24.572681+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T07:57:24.572681+0000","last_change":"2023-11-24T07:57:24.572681+0000","last_active":"2023-11-24T07:57:24.572681+0000","last_peered":"2023-11-24T07:57:24.572681+0000","last_clean":"2023-11-24T07:57:24.572681+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T07:57:24.572681+0000","last_undegraded":"2023-11-24T07:57:24.572681+0000","last_fullsized":"2023-11-24T07:57:24.572681+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:57:24.572681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:57:24.572681+0000","last_clean_scrub_stamp":"2023-11-24T07:57:24.572681+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T07:57:23.565116+0000","last_change":"2023-11-24T07:57:23.565116+0000","last_active":"2023-11-24T07:57:23.565116+0000","last_peered":"2023-11-24T07:57:23.565116+0000","last_clean":"2023-11-24T07:57:23.565116+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T07:57:23.565116+0000","last_undegraded":"2023-11-24T07:57:23.565116+0000","last_fullsized":"2023-11-24T07:57:23.565116+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:57:23.565116+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:57:23.565116+0000","last_clean_scrub_stamp":"2023-11-24T07:57:23.565116+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T07:57:24.572681+0000","last_change":"2023-11-24T07:57:24.572681+0000","last_active":"2023-11-24T07:57:24.572681+0000","last_peered":"2023-11-24T07:57:24.572681+0000","last_clean":"2023-11-24T07:57:24.572681+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T07:57:24.572681+0000","last_undegraded":"2023-11-24T07:57:24.572681+0000","last_fullsized":"2023-11-24T07:57:24.572681+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:57:24.572681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:57:24.572681+0000","last_clean_scrub_stamp":"2023-11-24T07:57:24.572681+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T07:57:23.565116+0000","last_change":"2023-11-24T07:57:23.565116+0000","last_active":"2023-11-24T07:57:23.565116+0000","last_peered":"2023-11-24T07:57:23.565116+0000","last_clean":"2023-11-24T07:57:23.565116+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T07:57:23.565116+0000","last_undegraded":"2023-11-24T07:57:23.565116+0000","last_fullsized":"2023-11-24T07:57:23.565116+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:57:23.565116+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:57:23.565116+0000","last_clean_scrub_stamp":"2023-11-24T07:57:23.565116+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T07:57:24.572681+0000","last_change":"2023-11-24T07:57:24.572681+0000","last_active":"2023-11-24T07:57:24.572681+0000","last_peered":"2023-11-24T07:57:24.572681+0000","last_clean":"2023-11-24T07:57:24.572681+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T07:57:24.572681+0000","last_undegraded":"2023-11-24T07:57:24.572681+0000","last_fullsized":"2023-11-24T07:57:24.572681+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:57:24.572681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:57:24.572681+0000","last_clean_scrub_stamp":"2023-11-24T07:57:24.572681+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T07:57:23.565116+0000","last_change":"2023-11-24T07:57:23.565116+0000","last_active":"2023-11-24T07:57:23.565116+0000","last_peered":"2023-11-24T07:57:23.565116+0000","last_clean":"2023-11-24T07:57:23.565116+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T07:57:23.565116+0000","last_undegraded":"2023-11-24T07:57:23.565116+0000","last_fullsized":"2023-11-24T07:57:23.565116+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:57:23.565116+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:57:23.565116+0000","last_clean_scrub_stamp":"2023-11-24T07:57:23.565116+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T07:57:24.572681+0000","last_change":"2023-11-24T07:57:24.572681+0000","last_active":"2023-11-24T07:57:24.572681+0000","last_peered":"2023-11-24T07:57:24.572681+0000","last_clean":"2023-11-24T07:57:24.572681+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T07:57:24.572681+0000","last_undegraded":"2023-11-24T07:57:24.572681+0000","last_fullsized":"2023-11-24T07:57:24.572681+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:57:24.572681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:57:24.572681+0000","last_clean_scrub_stamp":"2023-11-24T07:57:24.572681+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T07:57:23.565116+0000","last_change":"2023-11-24T07:57:23.565116+0000","last_active":"2023-11-24T07:57:23.565116+0000","last_peered":"2023-11-24T07:57:23.565116+0000","last_clean":"2023-11-24T07:57:23.565116+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T07:57:23.565116+0000","last_undegraded":"2023-11-24T07:57:23.565116+0000","last_fullsized":"2023-11-24T07:57:23.565116+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:57:23.565116+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:57:23.565116+0000","last_clean_scrub_stamp":"2023-11-24T07:57:23.565116+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T07:57:24.572681+0000","last_change":"2023-11-24T07:57:24.572681+0000","last_active":"2023-11-24T07:57:24.572681+0000","last_peered":"2023-11-24T07:57:24.572681+0000","last_clean":"2023-11-24T07:57:24.572681+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T07:57:24.572681+0000","last_undegraded":"2023-11-24T07:57:24.572681+0000","last_fullsized":"2023-11-24T07:57:24.572681+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:57:24.572681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:57:24.572681+0000","last_clean_scrub_stamp":"2023-11-24T07:57:24.572681+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T07:57:23.565116+0000","last_change":"2023-11-24T07:57:23.565116+0000","last_active":"2023-11-24T07:57:23.565116+0000","last_peered":"2023-11-24T07:57:23.565116+0000","last_clean":"2023-11-24T07:57:23.565116+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T07:57:23.565116+0000","last_undegraded":"2023-11-24T07:57:23.565116+0000","last_fullsized":"2023-11-24T07:57:23.565116+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:57:23.565116+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:57:23.565116+0000","last_clean_scrub_stamp":"2023-11-24T07:57:23.565116+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T07:57:24.572681+0000","last_change":"2023-11-24T07:57:24.572681+0000","last_active":"2023-11-24T07:57:24.572681+0000","last_peered":"2023-11-24T07:57:24.572681+0000","last_clean":"2023-11-24T07:57:24.572681+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T07:57:24.572681+0000","last_undegraded":"2023-11-24T07:57:24.572681+0000","last_fullsized":"2023-11-24T07:57:24.572681+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:57:24.572681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:57:24.572681+0000","last_clean_scrub_stamp":"2023-11-24T07:57:24.572681+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T07:57:23.565116+0000","last_change":"2023-11-24T07:57:23.565116+0000","last_active":"2023-11-24T07:57:23.565116+0000","last_peered":"2023-11-24T07:57:23.565116+0000","last_clean":"2023-11-24T07:57:23.565116+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T07:57:23.565116+0000","last_undegraded":"2023-11-24T07:57:23.565116+0000","last_fullsized":"2023-11-24T07:57:23.565116+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:57:23.565116+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:57:23.565116+0000","last_clean_scrub_stamp":"2023-11-24T07:57:23.565116+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T07:57:24.572681+0000","last_change":"2023-11-24T07:57:24.572681+0000","last_active":"2023-11-24T07:57:24.572681+0000","last_peered":"2023-11-24T07:57:24.572681+0000","last_clean":"2023-11-24T07:57:24.572681+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T07:57:24.572681+0000","last_undegraded":"2023-11-24T07:57:24.572681+0000","last_fullsized":"2023-11-24T07:57:24.572681+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:57:24.572681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:57:24.572681+0000","last_clean_scrub_stamp":"2023-11-24T07:57:24.572681+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T07:57:23.565116+0000","last_change":"2023-11-24T07:57:23.565116+0000","last_active":"2023-11-24T07:57:23.565116+0000","last_peered":"2023-11-24T07:57:23.565116+0000","last_clean":"2023-11-24T07:57:23.565116+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T07:57:23.565116+0000","last_undegraded":"2023-11-24T07:57:23.565116+0000","last_fullsized":"2023-11-24T07:57:23.565116+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:57:23.565116+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:57:23.565116+0000","last_clean_scrub_stamp":"2023-11-24T07:57:23.565116+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T07:57:24.572681+0000","last_change":"2023-11-24T07:57:24.572681+0000","last_active":"2023-11-24T07:57:24.572681+0000","last_peered":"2023-11-24T07:57:24.572681+0000","last_clean":"2023-11-24T07:57:24.572681+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T07:57:24.572681+0000","last_undegraded":"2023-11-24T07:57:24.572681+0000","last_fullsized":"2023-11-24T07:57:24.572681+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:57:24.572681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:57:24.572681+0000","last_clean_scrub_stamp":"2023-11-24T07:57:24.572681+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T07:57:23.565116+0000","last_change":"2023-11-24T07:57:23.565116+0000","last_active":"2023-11-24T07:57:23.565116+0000","last_peered":"2023-11-24T07:57:23.565116+0000","last_clean":"2023-11-24T07:57:23.565116+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T07:57:23.565116+0000","last_undegraded":"2023-11-24T07:57:23.565116+0000","last_fullsized":"2023-11-24T07:57:23.565116+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:57:23.565116+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:57:23.565116+0000","last_clean_scrub_stamp":"2023-11-24T07:57:23.565116+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T07:57:24.572681+0000","last_change":"2023-11-24T07:57:24.572681+0000","last_active":"2023-11-24T07:57:24.572681+0000","last_peered":"2023-11-24T07:57:24.572681+0000","last_clean":"2023-11-24T07:57:24.572681+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T07:57:24.572681+0000","last_undegraded":"2023-11-24T07:57:24.572681+0000","last_fullsized":"2023-11-24T07:57:24.572681+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:57:24.572681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:57:24.572681+0000","last_clean_scrub_stamp":"2023-11-24T07:57:24.572681+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T07:57:23.565116+0000","last_change":"2023-11-24T07:57:23.565116+0000","last_active":"2023-11-24T07:57:23.565116+0000","last_peered":"2023-11-24T07:57:23.565116+0000","last_clean":"2023-11-24T07:57:23.565116+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T07:57:23.565116+0000","last_undegraded":"2023-11-24T07:57:23.565116+0000","last_fullsized":"2023-11-24T07:57:23.565116+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:57:23.565116+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:57:23.565116+0000","last_clean_scrub_stamp":"2023-11-24T07:57:23.565116+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T07:57:24.572681+0000","last_change":"2023-11-24T07:57:24.572681+0000","last_active":"2023-11-24T07:57:24.572681+0000","last_peered":"2023-11-24T07:57:24.572681+0000","last_clean":"2023-11-24T07:57:24.572681+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T07:57:24.572681+0000","last_undegraded":"2023-11-24T07:57:24.572681+0000","last_fullsized":"2023-11-24T07:57:24.572681+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:57:24.572681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:57:24.572681+0000","last_clean_scrub_stamp":"2023-11-24T07:57:24.572681+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T07:57:23.565116+0000","last_change":"2023-11-24T07:57:23.565116+0000","last_active":"2023-11-24T07:57:23.565116+0000","last_peered":"2023-11-24T07:57:23.565116+0000","last_clean":"2023-11-24T07:57:23.565116+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T07:57:23.565116+0000","last_undegraded":"2023-11-24T07:57:23.565116+0000","last_fullsized":"2023-11-24T07:57:23.565116+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:57:23.565116+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:57:23.565116+0000","last_clean_scrub_stamp":"2023-11-24T07:57:23.565116+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T07:57:24.572681+0000","last_change":"2023-11-24T07:57:24.572681+0000","last_active":"2023-11-24T07:57:24.572681+0000","last_peered":"2023-11-24T07:57:24.572681+0000","last_clean":"2023-11-24T07:57:24.572681+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T07:57:24.572681+0000","last_undegraded":"2023-11-24T07:57:24.572681+0000","last_fullsized":"2023-11-24T07:57:24.572681+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:57:24.572681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:57:24.572681+0000","last_clean_scrub_stamp":"2023-11-24T07:57:24.572681+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T07:57:23.565116+0000","last_change":"2023-11-24T07:57:23.565116+0000","last_active":"2023-11-24T07:57:23.565116+0000","last_peered":"2023-11-24T07:57:23.565116+0000","last_clean":"2023-11-24T07:57:23.565116+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T07:57:23.565116+0000","last_undegraded":"2023-11-24T07:57:23.565116+0000","last_fullsized":"2023-11-24T07:57:23.565116+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:57:23.565116+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:57:23.565116+0000","last_clean_scrub_stamp":"2023-11-24T07:57:23.565116+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T07:57:24.572681+0000","last_change":"2023-11-24T07:57:24.572681+0000","last_active":"2023-11-24T07:57:24.572681+0000","last_peered":"2023-11-24T07:57:24.572681+0000","last_clean":"2023-11-24T07:57:24.572681+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T07:57:24.572681+0000","last_undegraded":"2023-11-24T07:57:24.572681+0000","last_fullsized":"2023-11-24T07:57:24.572681+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:57:24.572681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:57:24.572681+0000","last_clean_scrub_stamp":"2023-11-24T07:57:24.572681+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T07:57:23.565116+0000","last_change":"2023-11-24T07:57:23.565116+0000","last_active":"2023-11-24T07:57:23.565116+0000","last_peered":"2023-11-24T07:57:23.565116+0000","last_clean":"2023-11-24T07:57:23.565116+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T07:57:23.565116+0000","last_undegraded":"2023-11-24T07:57:23.565116+0000","last_fullsized":"2023-11-24T07:57:23.565116+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:57:23.565116+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:57:23.565116+0000","last_clean_scrub_stamp":"2023-11-24T07:57:23.565116+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T07:57:24.572681+0000","last_change":"2023-11-24T07:57:24.572681+0000","last_active":"2023-11-24T07:57:24.572681+0000","last_peered":"2023-11-24T07:57:24.572681+0000","last_clean":"2023-11-24T07:57:24.572681+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T07:57:24.572681+0000","last_undegraded":"2023-11-24T07:57:24.572681+0000","last_fullsized":"2023-11-24T07:57:24.572681+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:57:24.572681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:57:24.572681+0000","last_clean_scrub_stamp":"2023-11-24T07:57:24.572681+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T07:57:23.565116+0000","last_change":"2023-11-24T07:57:23.565116+0000","last_active":"2023-11-24T07:57:23.565116+0000","last_peered":"2023-11-24T07:57:23.565116+0000","last_clean":"2023-11-24T07:57:23.565116+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T07:57:23.565116+0000","last_undegraded":"2023-11-24T07:57:23.565116+0000","last_fullsized":"2023-11-24T07:57:23.565116+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:57:23.565116+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:57:23.565116+0000","last_clean_scrub_stamp":"2023-11-24T07:57:23.565116+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T07:57:24.572681+0000","last_change":"2023-11-24T07:57:24.572681+0000","last_active":"2023-11-24T07:57:24.572681+0000","last_peered":"2023-11-24T07:57:24.572681+0000","last_clean":"2023-11-24T07:57:24.572681+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T07:57:24.572681+0000","last_undegraded":"2023-11-24T07:57:24.572681+0000","last_fullsized":"2023-11-24T07:57:24.572681+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:57:24.572681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:57:24.572681+0000","last_clean_scrub_stamp":"2023-11-24T07:57:24.572681+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T07:57:23.565116+0000","last_change":"2023-11-24T07:57:23.565116+0000","last_active":"2023-11-24T07:57:23.565116+0000","last_peered":"2023-11-24T07:57:23.565116+0000","last_clean":"2023-11-24T07:57:23.565116+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T07:57:23.565116+0000","last_undegraded":"2023-11-24T07:57:23.565116+0000","last_fullsized":"2023-11-24T07:57:23.565116+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:57:23.565116+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:57:23.565116+0000","last_clean_scrub_stamp":"2023-11-24T07:57:23.565116+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T07:57:24.572681+0000","last_change":"2023-11-24T07:57:24.572681+0000","last_active":"2023-11-24T07:57:24.572681+0000","last_peered":"2023-11-24T07:57:24.572681+0000","last_clean":"2023-11-24T07:57:24.572681+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T07:57:24.572681+0000","last_undegraded":"2023-11-24T07:57:24.572681+0000","last_fullsized":"2023-11-24T07:57:24.572681+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:57:24.572681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:57:24.572681+0000","last_clean_scrub_stamp":"2023-11-24T07:57:24.572681+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T07:57:23.565116+0000","last_change":"2023-11-24T07:57:23.565116+0000","last_active":"2023-11-24T07:57:23.565116+0000","last_peered":"2023-11-24T07:57:23.565116+0000","last_clean":"2023-11-24T07:57:23.565116+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T07:57:23.565116+0000","last_undegraded":"2023-11-24T07:57:23.565116+0000","last_fullsized":"2023-11-24T07:57:23.565116+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:57:23.565116+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:57:23.565116+0000","last_clean_scrub_stamp":"2023-11-24T07:57:23.565116+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T07:57:24.572681+0000","last_change":"2023-11-24T07:57:24.572681+0000","last_active":"2023-11-24T07:57:24.572681+0000","last_peered":"2023-11-24T07:57:24.572681+0000","last_clean":"2023-11-24T07:57:24.572681+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T07:57:24.572681+0000","last_undegraded":"2023-11-24T07:57:24.572681+0000","last_fullsized":"2023-11-24T07:57:24.572681+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:57:24.572681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:57:24.572681+0000","last_clean_scrub_stamp":"2023-11-24T07:57:24.572681+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T07:57:23.565116+0000","last_change":"2023-11-24T07:57:23.565116+0000","last_active":"2023-11-24T07:57:23.565116+0000","last_peered":"2023-11-24T07:57:23.565116+0000","last_clean":"2023-11-24T07:57:23.565116+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T07:57:23.565116+0000","last_undegraded":"2023-11-24T07:57:23.565116+0000","last_fullsized":"2023-11-24T07:57:23.565116+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:57:23.565116+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:57:23.565116+0000","last_clean_scrub_stamp":"2023-11-24T07:57:23.565116+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T07:57:24.572681+0000","last_change":"2023-11-24T07:57:24.572681+0000","last_active":"2023-11-24T07:57:24.572681+0000","last_peered":"2023-11-24T07:57:24.572681+0000","last_clean":"2023-11-24T07:57:24.572681+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T07:57:24.572681+0000","last_undegraded":"2023-11-24T07:57:24.572681+0000","last_fullsized":"2023-11-24T07:57:24.572681+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:57:24.572681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:57:24.572681+0000","last_clean_scrub_stamp":"2023-11-24T07:57:24.572681+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T07:57:23.565116+0000","last_change":"2023-11-24T07:57:23.565116+0000","last_active":"2023-11-24T07:57:23.565116+0000","last_peered":"2023-11-24T07:57:23.565116+0000","last_clean":"2023-11-24T07:57:23.565116+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T07:57:23.565116+0000","last_undegraded":"2023-11-24T07:57:23.565116+0000","last_fullsized":"2023-11-24T07:57:23.565116+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:57:23.565116+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:57:23.565116+0000","last_clean_scrub_stamp":"2023-11-24T07:57:23.565116+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T07:57:24.572681+0000","last_change":"2023-11-24T07:57:24.572681+0000","last_active":"2023-11-24T07:57:24.572681+0000","last_peered":"2023-11-24T07:57:24.572681+0000","last_clean":"2023-11-24T07:57:24.572681+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T07:57:24.572681+0000","last_undegraded":"2023-11-24T07:57:24.572681+0000","last_fullsized":"2023-11-24T07:57:24.572681+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:57:24.572681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:57:24.572681+0000","last_clean_scrub_stamp":"2023-11-24T07:57:24.572681+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T07:57:23.565116+0000","last_change":"2023-11-24T07:57:23.565116+0000","last_active":"2023-11-24T07:57:23.565116+0000","last_peered":"2023-11-24T07:57:23.565116+0000","last_clean":"2023-11-24T07:57:23.565116+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T07:57:23.565116+0000","last_undegraded":"2023-11-24T07:57:23.565116+0000","last_fullsized":"2023-11-24T07:57:23.565116+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:57:23.565116+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:57:23.565116+0000","last_clean_scrub_stamp":"2023-11-24T07:57:23.565116+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T07:57:24.572681+0000","last_change":"2023-11-24T07:57:24.572681+0000","last_active":"2023-11-24T07:57:24.572681+0000","last_peered":"2023-11-24T07:57:24.572681+0000","last_clean":"2023-11-24T07:57:24.572681+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T07:57:24.572681+0000","last_undegraded":"2023-11-24T07:57:24.572681+0000","last_fullsized":"2023-11-24T07:57:24.572681+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:57:24.572681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:57:24.572681+0000","last_clean_scrub_stamp":"2023-11-24T07:57:24.572681+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T07:57:23.565116+0000","last_change":"2023-11-24T07:57:23.565116+0000","last_active":"2023-11-24T07:57:23.565116+0000","last_peered":"2023-11-24T07:57:23.565116+0000","last_clean":"2023-11-24T07:57:23.565116+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T07:57:23.565116+0000","last_undegraded":"2023-11-24T07:57:23.565116+0000","last_fullsized":"2023-11-24T07:57:23.565116+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:57:23.565116+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:57:23.565116+0000","last_clean_scrub_stamp":"2023-11-24T07:57:23.565116+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T07:57:24.572681+0000","last_change":"2023-11-24T07:57:24.572681+0000","last_active":"2023-11-24T07:57:24.572681+0000","last_peered":"2023-11-24T07:57:24.572681+0000","last_clean":"2023-11-24T07:57:24.572681+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T07:57:24.572681+0000","last_undegraded":"2023-11-24T07:57:24.572681+0000","last_fullsized":"2023-11-24T07:57:24.572681+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:57:24.572681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:57:24.572681+0000","last_clean_scrub_stamp":"2023-11-24T07:57:24.572681+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T07:57:23.565116+0000","last_change":"2023-11-24T07:57:23.565116+0000","last_active":"2023-11-24T07:57:23.565116+0000","last_peered":"2023-11-24T07:57:23.565116+0000","last_clean":"2023-11-24T07:57:23.565116+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T07:57:23.565116+0000","last_undegraded":"2023-11-24T07:57:23.565116+0000","last_fullsized":"2023-11-24T07:57:23.565116+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:57:23.565116+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:57:23.565116+0000","last_clean_scrub_stamp":"2023-11-24T07:57:23.565116+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T07:57:24.572681+0000","last_change":"2023-11-24T07:57:24.572681+0000","last_active":"2023-11-24T07:57:24.572681+0000","last_peered":"2023-11-24T07:57:24.572681+0000","last_clean":"2023-11-24T07:57:24.572681+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T07:57:24.572681+0000","last_undegraded":"2023-11-24T07:57:24.572681+0000","last_fullsized":"2023-11-24T07:57:24.572681+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:57:24.572681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:57:24.572681+0000","last_clean_scrub_stamp":"2023-11-24T07:57:24.572681+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T07:57:23.565116+0000","last_change":"2023-11-24T07:57:23.565116+0000","last_active":"2023-11-24T07:57:23.565116+0000","last_peered":"2023-11-24T07:57:23.565116+0000","last_clean":"2023-11-24T07:57:23.565116+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T07:57:23.565116+0000","last_undegraded":"2023-11-24T07:57:23.565116+0000","last_fullsized":"2023-11-24T07:57:23.565116+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:57:23.565116+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:57:23.565116+0000","last_clean_scrub_stamp":"2023-11-24T07:57:23.565116+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T07:57:24.572681+0000","last_change":"2023-11-24T07:57:24.572681+0000","last_active":"2023-11-24T07:57:24.572681+0000","last_peered":"2023-11-24T07:57:24.572681+0000","last_clean":"2023-11-24T07:57:24.572681+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T07:57:24.572681+0000","last_undegraded":"2023-11-24T07:57:24.572681+0000","last_fullsized":"2023-11-24T07:57:24.572681+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:57:24.572681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:57:24.572681+0000","last_clean_scrub_stamp":"2023-11-24T07:57:24.572681+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T07:57:23.565116+0000","last_change":"2023-11-24T07:57:23.565116+0000","last_active":"2023-11-24T07:57:23.565116+0000","last_peered":"2023-11-24T07:57:23.565116+0000","last_clean":"2023-11-24T07:57:23.565116+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T07:57:23.565116+0000","last_undegraded":"2023-11-24T07:57:23.565116+0000","last_fullsized":"2023-11-24T07:57:23.565116+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:57:23.565116+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:57:23.565116+0000","last_clean_scrub_stamp":"2023-11-24T07:57:23.565116+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T07:57:24.572681+0000","last_change":"2023-11-24T07:57:24.572681+0000","last_active":"2023-11-24T07:57:24.572681+0000","last_peered":"2023-11-24T07:57:24.572681+0000","last_clean":"2023-11-24T07:57:24.572681+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T07:57:24.572681+0000","last_undegraded":"2023-11-24T07:57:24.572681+0000","last_fullsized":"2023-11-24T07:57:24.572681+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:57:24.572681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:57:24.572681+0000","last_clean_scrub_stamp":"2023-11-24T07:57:24.572681+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T07:57:23.565116+0000","last_change":"2023-11-24T07:57:23.565116+0000","last_active":"2023-11-24T07:57:23.565116+0000","last_peered":"2023-11-24T07:57:23.565116+0000","last_clean":"2023-11-24T07:57:23.565116+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T07:57:23.565116+0000","last_undegraded":"2023-11-24T07:57:23.565116+0000","last_fullsized":"2023-11-24T07:57:23.565116+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:57:23.565116+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:57:23.565116+0000","last_clean_scrub_stamp":"2023-11-24T07:57:23.565116+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T07:57:24.572681+0000","last_change":"2023-11-24T07:57:24.572681+0000","last_active":"2023-11-24T07:57:24.572681+0000","last_peered":"2023-11-24T07:57:24.572681+0000","last_clean":"2023-11-24T07:57:24.572681+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T07:57:24.572681+0000","last_undegraded":"2023-11-24T07:57:24.572681+0000","last_fullsized":"2023-11-24T07:57:24.572681+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:57:24.572681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:57:24.572681+0000","last_clean_scrub_stamp":"2023-11-24T07:57:24.572681+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T07:57:23.565116+0000","last_change":"2023-11-24T07:57:23.565116+0000","last_active":"2023-11-24T07:57:23.565116+0000","last_peered":"2023-11-24T07:57:23.565116+0000","last_clean":"2023-11-24T07:57:23.565116+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T07:57:23.565116+0000","last_undegraded":"2023-11-24T07:57:23.565116+0000","last_fullsized":"2023-11-24T07:57:23.565116+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:57:23.565116+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:57:23.565116+0000","last_clean_scrub_stamp":"2023-11-24T07:57:23.565116+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T07:57:24.572681+0000","last_change":"2023-11-24T07:57:24.572681+0000","last_active":"2023-11-24T07:57:24.572681+0000","last_peered":"2023-11-24T07:57:24.572681+0000","last_clean":"2023-11-24T07:57:24.572681+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T07:57:24.572681+0000","last_undegraded":"2023-11-24T07:57:24.572681+0000","last_fullsized":"2023-11-24T07:57:24.572681+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:57:24.572681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:57:24.572681+0000","last_clean_scrub_stamp":"2023-11-24T07:57:24.572681+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T07:57:23.565116+0000","last_change":"2023-11-24T07:57:23.565116+0000","last_active":"2023-11-24T07:57:23.565116+0000","last_peered":"2023-11-24T07:57:23.565116+0000","last_clean":"2023-11-24T07:57:23.565116+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T07:57:23.565116+0000","last_undegraded":"2023-11-24T07:57:23.565116+0000","last_fullsized":"2023-11-24T07:57:23.565116+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:57:23.565116+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:57:23.565116+0000","last_clean_scrub_stamp":"2023-11-24T07:57:23.565116+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T07:57:24.572681+0000","last_change":"2023-11-24T07:57:24.572681+0000","last_active":"2023-11-24T07:57:24.572681+0000","last_peered":"2023-11-24T07:57:24.572681+0000","last_clean":"2023-11-24T07:57:24.572681+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T07:57:24.572681+0000","last_undegraded":"2023-11-24T07:57:24.572681+0000","last_fullsized":"2023-11-24T07:57:24.572681+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:57:24.572681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:57:24.572681+0000","last_clean_scrub_stamp":"2023-11-24T07:57:24.572681+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T07:57:23.565116+0000","last_change":"2023-11-24T07:57:23.565116+0000","last_active":"2023-11-24T07:57:23.565116+0000","last_peered":"2023-11-24T07:57:23.565116+0000","last_clean":"2023-11-24T07:57:23.565116+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T07:57:23.565116+0000","last_undegraded":"2023-11-24T07:57:23.565116+0000","last_fullsized":"2023-11-24T07:57:23.565116+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:57:23.565116+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:57:23.565116+0000","last_clean_scrub_stamp":"2023-11-24T07:57:23.565116+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T07:57:24.572681+0000","last_change":"2023-11-24T07:57:24.572681+0000","last_active":"2023-11-24T07:57:24.572681+0000","last_peered":"2023-11-24T07:57:24.572681+0000","last_clean":"2023-11-24T07:57:24.572681+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T07:57:24.572681+0000","last_undegraded":"2023-11-24T07:57:24.572681+0000","last_fullsized":"2023-11-24T07:57:24.572681+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:57:24.572681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:57:24.572681+0000","last_clean_scrub_stamp":"2023-11-24T07:57:24.572681+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T07:57:23.565116+0000","last_change":"2023-11-24T07:57:23.565116+0000","last_active":"2023-11-24T07:57:23.565116+0000","last_peered":"2023-11-24T07:57:23.565116+0000","last_clean":"2023-11-24T07:57:23.565116+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T07:57:23.565116+0000","last_undegraded":"2023-11-24T07:57:23.565116+0000","last_fullsized":"2023-11-24T07:57:23.565116+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:57:23.565116+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:57:23.565116+0000","last_clean_scrub_stamp":"2023-11-24T07:57:23.565116+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T07:57:24.572681+0000","last_change":"2023-11-24T07:57:24.572681+0000","last_active":"2023-11-24T07:57:24.572681+0000","last_peered":"2023-11-24T07:57:24.572681+0000","last_clean":"2023-11-24T07:57:24.572681+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T07:57:24.572681+0000","last_undegraded":"2023-11-24T07:57:24.572681+0000","last_fullsized":"2023-11-24T07:57:24.572681+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:57:24.572681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:57:24.572681+0000","last_clean_scrub_stamp":"2023-11-24T07:57:24.572681+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T07:57:23.565116+0000","last_change":"2023-11-24T07:57:23.565116+0000","last_active":"2023-11-24T07:57:23.565116+0000","last_peered":"2023-11-24T07:57:23.565116+0000","last_clean":"2023-11-24T07:57:23.565116+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T07:57:23.565116+0000","last_undegraded":"2023-11-24T07:57:23.565116+0000","last_fullsized":"2023-11-24T07:57:23.565116+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:57:23.565116+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:57:23.565116+0000","last_clean_scrub_stamp":"2023-11-24T07:57:23.565116+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T07:57:24.572681+0000","last_change":"2023-11-24T07:57:24.572681+0000","last_active":"2023-11-24T07:57:24.572681+0000","last_peered":"2023-11-24T07:57:24.572681+0000","last_clean":"2023-11-24T07:57:24.572681+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T07:57:24.572681+0000","last_undegraded":"2023-11-24T07:57:24.572681+0000","last_fullsized":"2023-11-24T07:57:24.572681+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:57:24.572681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:57:24.572681+0000","last_clean_scrub_stamp":"2023-11-24T07:57:24.572681+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T07:57:23.565116+0000","last_change":"2023-11-24T07:57:23.565116+0000","last_active":"2023-11-24T07:57:23.565116+0000","last_peered":"2023-11-24T07:57:23.565116+0000","last_clean":"2023-11-24T07:57:23.565116+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T07:57:23.565116+0000","last_undegraded":"2023-11-24T07:57:23.565116+0000","last_fullsized":"2023-11-24T07:57:23.565116+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:57:23.565116+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:57:23.565116+0000","last_clean_scrub_stamp":"2023-11-24T07:57:23.565116+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T07:57:24.572681+0000","last_change":"2023-11-24T07:57:24.572681+0000","last_active":"2023-11-24T07:57:24.572681+0000","last_peered":"2023-11-24T07:57:24.572681+0000","last_clean":"2023-11-24T07:57:24.572681+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T07:57:24.572681+0000","last_undegraded":"2023-11-24T07:57:24.572681+0000","last_fullsized":"2023-11-24T07:57:24.572681+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:57:24.572681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:57:24.572681+0000","last_clean_scrub_stamp":"2023-11-24T07:57:24.572681+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T07:57:23.565116+0000","last_change":"2023-11-24T07:57:23.565116+0000","last_active":"2023-11-24T07:57:23.565116+0000","last_peered":"2023-11-24T07:57:23.565116+0000","last_clean":"2023-11-24T07:57:23.565116+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T07:57:23.565116+0000","last_undegraded":"2023-11-24T07:57:23.565116+0000","last_fullsized":"2023-11-24T07:57:23.565116+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:57:23.565116+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:57:23.565116+0000","last_clean_scrub_stamp":"2023-11-24T07:57:23.565116+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T07:57:24.572681+0000","last_change":"2023-11-24T07:57:24.572681+0000","last_active":"2023-11-24T07:57:24.572681+0000","last_peered":"2023-11-24T07:57:24.572681+0000","last_clean":"2023-11-24T07:57:24.572681+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T07:57:24.572681+0000","last_undegraded":"2023-11-24T07:57:24.572681+0000","last_fullsized":"2023-11-24T07:57:24.572681+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:57:24.572681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:57:24.572681+0000","last_clean_scrub_stamp":"2023-11-24T07:57:24.572681+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T07:57:23.565116+0000","last_change":"2023-11-24T07:57:23.565116+0000","last_active":"2023-11-24T07:57:23.565116+0000","last_peered":"2023-11-24T07:57:23.565116+0000","last_clean":"2023-11-24T07:57:23.565116+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T07:57:23.565116+0000","last_undegraded":"2023-11-24T07:57:23.565116+0000","last_fullsized":"2023-11-24T07:57:23.565116+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:57:23.565116+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:57:23.565116+0000","last_clean_scrub_stamp":"2023-11-24T07:57:23.565116+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T07:57:24.572681+0000","last_change":"2023-11-24T07:57:24.572681+0000","last_active":"2023-11-24T07:57:24.572681+0000","last_peered":"2023-11-24T07:57:24.572681+0000","last_clean":"2023-11-24T07:57:24.572681+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T07:57:24.572681+0000","last_undegraded":"2023-11-24T07:57:24.572681+0000","last_fullsized":"2023-11-24T07:57:24.572681+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:57:24.572681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:57:24.572681+0000","last_clean_scrub_stamp":"2023-11-24T07:57:24.572681+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T07:57:23.565116+0000","last_change":"2023-11-24T07:57:23.565116+0000","last_active":"2023-11-24T07:57:23.565116+0000","last_peered":"2023-11-24T07:57:23.565116+0000","last_clean":"2023-11-24T07:57:23.565116+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T07:57:23.565116+0000","last_undegraded":"2023-11-24T07:57:23.565116+0000","last_fullsized":"2023-11-24T07:57:23.565116+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:57:23.565116+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:57:23.565116+0000","last_clean_scrub_stamp":"2023-11-24T07:57:23.565116+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":172,"reported_epoch":28,"state":"active+clean","last_fresh":"2023-11-24T07:57:18.484315+0000","last_change":"2023-11-24T07:56:19.668905+0000","last_active":"2023-11-24T07:57:18.484315+0000","last_peered":"2023-11-24T07:57:18.484315+0000","last_clean":"2023-11-24T07:57:18.484315+0000","last_became_active":"2023-11-24T07:56:19.668601+0000","last_became_peered":"2023-11-24T07:56:19.668601+0000","last_unstale":"2023-11-24T07:57:18.484315+0000","last_undegraded":"2023-11-24T07:57:18.484315+0000","last_fullsized":"2023-11-24T07:57:18.484315+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:56:18.657844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:56:18.657844+0000","last_clean_scrub_stamp":"2023-11-24T07:56:18.657844+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T11:51:53.828455+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":5,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574864,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27340,"kb_used_data":316,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344500,"statfs":{"total":96636764160,"available":96608768000,"internally_reserved":0,"allocated":323584,"data_stored":120931,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 07:57:19 2023","interfaces":[{"interface":"back","average":{"1min":0.55700000000000005,"5min":0.55700000000000005,"15min":0.55700000000000005},"min":{"1min":0.32800000000000001,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":1.413,"5min":1.413,"15min":1.413},"last":0.71299999999999997},{"interface":"front","average":{"1min":0.55400000000000005,"5min":0.55400000000000005,"15min":0.55400000000000005},"min":{"1min":0.41099999999999998,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":0.81000000000000005,"5min":0.81000000000000005,"15min":0.81000000000000005},"last":0.65300000000000002}]},{"osd":1,"last update":"Fri Nov 24 07:57:19 2023","interfaces":[{"interface":"back","average":{"1min":0.84899999999999998,"5min":0.84899999999999998,"15min":0.84899999999999998},"min":{"1min":0.33600000000000002,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":6.1070000000000002,"5min":6.1070000000000002,"15min":6.1070000000000002},"last":0.60299999999999998},{"interface":"front","average":{"1min":0.57999999999999996,"5min":0.57999999999999996,"15min":0.57999999999999996},"min":{"1min":0.35199999999999998,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":1.5920000000000001,"5min":1.5920000000000001,"15min":1.5920000000000001},"last":0.39700000000000002}]},{"osd":2,"last update":"Fri Nov 24 07:57:19 2023","interfaces":[{"interface":"back","average":{"1min":0.627,"5min":0.627,"15min":0.627},"min":{"1min":0.33800000000000002,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":1.613,"5min":1.613,"15min":1.613},"last":0.56000000000000005},{"interface":"front","average":{"1min":0.79200000000000004,"5min":0.79200000000000004,"15min":0.79200000000000004},"min":{"1min":0.41699999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":4.5220000000000002,"5min":4.5220000000000002,"15min":4.5220000000000002},"last":0.54000000000000004}]},{"osd":3,"last update":"Fri Nov 24 07:57:19 2023","interfaces":[{"interface":"back","average":{"1min":0.80600000000000005,"5min":0.80600000000000005,"15min":0.80600000000000005},"min":{"1min":0.38500000000000001,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":4.6760000000000002,"5min":4.6760000000000002,"15min":4.6760000000000002},"last":0.46300000000000002},{"interface":"front","average":{"1min":0.94299999999999995,"5min":0.94299999999999995,"15min":0.94299999999999995},"min":{"1min":0.505,"5min":0.505,"15min":0.505},"max":{"1min":4.6539999999999999,"5min":4.6539999999999999,"15min":4.6539999999999999},"last":0.63300000000000001}]},{"osd":4,"last update":"Fri Nov 24 07:57:19 2023","interfaces":[{"interface":"back","average":{"1min":0.56799999999999995,"5min":0.56799999999999995,"15min":0.56799999999999995},"min":{"1min":0.378,"5min":0.378,"15min":0.378},"max":{"1min":0.84699999999999998,"5min":0.84699999999999998,"15min":0.84699999999999998},"last":0.67400000000000004},{"interface":"front","average":{"1min":0.66700000000000004,"5min":0.66700000000000004,"15min":0.66700000000000004},"min":{"1min":0.46100000000000002,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":1.629,"5min":1.629,"15min":1.629},"last":0.51200000000000001}]},{"osd":5,"last update":"Fri Nov 24 07:57:19 2023","interfaces":[{"interface":"back","average":{"1min":0.54700000000000004,"5min":0.54700000000000004,"15min":0.54700000000000004},"min":{"1min":0.34499999999999997,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.875,"5min":0.875,"15min":0.875},"last":0.69899999999999995},{"interface":"front","average":{"1min":0.623,"5min":0.623,"15min":0.623},"min":{"1min":0.41499999999999998,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":0.90900000000000003,"5min":0.90900000000000003,"15min":0.90900000000000003},"last":0.79700000000000004}]},{"osd":6,"last update":"Fri Nov 24 07:57:19 2023","interfaces":[{"interface":"back","average":{"1min":0.61099999999999999,"5min":0.61099999999999999,"15min":0.61099999999999999},"min":{"1min":0.438,"5min":0.438,"15min":0.438},"max":{"1min":0.80800000000000005,"5min":0.80800000000000005,"15min":0.80800000000000005},"last":0.76700000000000002},{"interface":"front","average":{"1min":0.67000000000000004,"5min":0.67000000000000004,"15min":0.67000000000000004},"min":{"1min":0.48599999999999999,"5min":0.48599999999999999,"15min":0.48599999999999999},"max":{"1min":0.98399999999999999,"5min":0.98399999999999999,"15min":0.98399999999999999},"last":0.73199999999999998}]}]},{"osd":6,"up_from":13,"seq":55834574865,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27348,"kb_used_data":316,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344492,"statfs":{"total":96636764160,"available":96608759808,"internally_reserved":0,"allocated":323584,"data_stored":120931,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 07:57:18 2023","interfaces":[{"interface":"back","average":{"1min":0.55000000000000004,"5min":0.55000000000000004,"15min":0.55000000000000004},"min":{"1min":0.28399999999999997,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":0.85099999999999998,"5min":0.85099999999999998,"15min":0.85099999999999998},"last":0.67000000000000004},{"interface":"front","average":{"1min":0.51600000000000001,"5min":0.51600000000000001,"15min":0.51600000000000001},"min":{"1min":0.311,"5min":0.311,"15min":0.311},"max":{"1min":0.69999999999999996,"5min":0.69999999999999996,"15min":0.69999999999999996},"last":0.82199999999999995}]},{"osd":1,"last update":"Fri Nov 24 07:57:18 2023","interfaces":[{"interface":"back","average":{"1min":0.51200000000000001,"5min":0.51200000000000001,"15min":0.51200000000000001},"min":{"1min":0.33800000000000002,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.78200000000000003,"5min":0.78200000000000003,"15min":0.78200000000000003},"last":0.54300000000000004},{"interface":"front","average":{"1min":0.61299999999999999,"5min":0.61299999999999999,"15min":0.61299999999999999},"min":{"1min":0.44700000000000001,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":1.0069999999999999,"5min":1.0069999999999999,"15min":1.0069999999999999},"last":0.50800000000000001}]},{"osd":2,"last update":"Fri Nov 24 07:57:18 2023","interfaces":[{"interface":"back","average":{"1min":0.57499999999999996,"5min":0.57499999999999996,"15min":0.57499999999999996},"min":{"1min":0.34300000000000003,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.878,"5min":0.878,"15min":0.878},"last":0.64400000000000002},{"interface":"front","average":{"1min":0.59299999999999997,"5min":0.59299999999999997,"15min":0.59299999999999997},"min":{"1min":0.47099999999999997,"5min":0.47099999999999997,"15min":0.47099999999999997},"max":{"1min":0.82399999999999995,"5min":0.82399999999999995,"15min":0.82399999999999995},"last":0.57099999999999995}]},{"osd":3,"last update":"Fri Nov 24 07:57:18 2023","interfaces":[{"interface":"back","average":{"1min":0.56799999999999995,"5min":0.56799999999999995,"15min":0.56799999999999995},"min":{"1min":0.40000000000000002,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":0.876,"5min":0.876,"15min":0.876},"last":0.65800000000000003},{"interface":"front","average":{"1min":0.55600000000000005,"5min":0.55600000000000005,"15min":0.55600000000000005},"min":{"1min":0.379,"5min":0.379,"15min":0.379},"max":{"1min":0.68799999999999994,"5min":0.68799999999999994,"15min":0.68799999999999994},"last":0.42699999999999999}]},{"osd":4,"last update":"Fri Nov 24 07:57:18 2023","interfaces":[{"interface":"back","average":{"1min":0.51700000000000002,"5min":0.51700000000000002,"15min":0.51700000000000002},"min":{"1min":0.33000000000000002,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.70599999999999996,"5min":0.70599999999999996,"15min":0.70599999999999996},"last":0.46700000000000003},{"interface":"front","average":{"1min":0.60199999999999998,"5min":0.60199999999999998,"15min":0.60199999999999998},"min":{"1min":0.35199999999999998,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":1.087,"5min":1.087,"15min":1.087},"last":0.60199999999999998}]},{"osd":5,"last update":"Fri Nov 24 07:57:18 2023","interfaces":[{"interface":"back","average":{"1min":0.53100000000000003,"5min":0.53100000000000003,"15min":0.53100000000000003},"min":{"1min":0.30199999999999999,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.84999999999999998,"5min":0.84999999999999998,"15min":0.84999999999999998},"last":0.58699999999999997},{"interface":"front","average":{"1min":0.61199999999999999,"5min":0.61199999999999999,"15min":0.61199999999999999},"min":{"1min":0.39200000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.97199999999999998,"5min":0.97199999999999998,"15min":0.97199999999999998},"last":0.627}]},{"osd":7,"last update":"Fri Nov 24 07:57:18 2023","interfaces":[{"interface":"back","average":{"1min":0.61799999999999999,"5min":0.61799999999999999,"15min":0.61799999999999999},"min":{"1min":0.49299999999999999,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":0.81499999999999995,"5min":0.81499999999999995,"15min":0.81499999999999995},"last":0.76100000000000001},{"interface":"front","average":{"1min":0.67000000000000004,"5min":0.67000000000000004,"15min":0.67000000000000004},"min":{"1min":0.497,"5min":0.497,"15min":0.497},"max":{"1min":0.92700000000000005,"5min":0.92700000000000005,"15min":0.92700000000000005},"last":0.72799999999999998}]}]},{"osd":1,"up_from":13,"seq":55834574866,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27924,"kb_used_data":896,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343916,"statfs":{"total":96636764160,"available":96608169984,"internally_reserved":0,"allocated":917504,"data_stored":711299,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 07:57:22 2023","interfaces":[{"interface":"back","average":{"1min":0.36399999999999999,"5min":0.36399999999999999,"15min":0.36399999999999999},"min":{"1min":0.20200000000000001,"5min":0.20200000000000001,"15min":0.20200000000000001},"max":{"1min":0.70899999999999996,"5min":0.70899999999999996,"15min":0.70899999999999996},"last":0.251},{"interface":"front","average":{"1min":0.39800000000000002,"5min":0.39800000000000002,"15min":0.39800000000000002},"min":{"1min":0.214,"5min":0.214,"15min":0.214},"max":{"1min":0.82499999999999996,"5min":0.82499999999999996,"15min":0.82499999999999996},"last":0.214}]},{"osd":2,"last update":"Fri Nov 24 07:57:22 2023","interfaces":[{"interface":"back","average":{"1min":0.373,"5min":0.373,"15min":0.373},"min":{"1min":0.22700000000000001,"5min":0.22700000000000001,"15min":0.22700000000000001},"max":{"1min":0.60999999999999999,"5min":0.60999999999999999,"15min":0.60999999999999999},"last":0.23300000000000001},{"interface":"front","average":{"1min":0.41299999999999998,"5min":0.41299999999999998,"15min":0.41299999999999998},"min":{"1min":0.26800000000000002,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":0.621,"5min":0.621,"15min":0.621},"last":0.28599999999999998}]},{"osd":3,"last update":"Fri Nov 24 07:57:22 2023","interfaces":[{"interface":"back","average":{"1min":0.44400000000000001,"5min":0.44400000000000001,"15min":0.44400000000000001},"min":{"1min":0.32100000000000001,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.66500000000000004,"5min":0.66500000000000004,"15min":0.66500000000000004},"last":0.33100000000000002},{"interface":"front","average":{"1min":0.42799999999999999,"5min":0.42799999999999999,"15min":0.42799999999999999},"min":{"1min":0.28499999999999998,"5min":0.28499999999999998,"15min":0.28499999999999998},"max":{"1min":0.76300000000000001,"5min":0.76300000000000001,"15min":0.76300000000000001},"last":0.31900000000000001}]},{"osd":4,"last update":"Fri Nov 24 07:57:22 2023","interfaces":[{"interface":"back","average":{"1min":0.61099999999999999,"5min":0.61099999999999999,"15min":0.61099999999999999},"min":{"1min":0.43099999999999999,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":0.79800000000000004,"5min":0.79800000000000004,"15min":0.79800000000000004},"last":0.495},{"interface":"front","average":{"1min":0.60599999999999998,"5min":0.60599999999999998,"15min":0.60599999999999998},"min":{"1min":0.41999999999999998,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":0.82199999999999995,"5min":0.82199999999999995,"15min":0.82199999999999995},"last":0.60499999999999998}]},{"osd":5,"last update":"Fri Nov 24 07:57:22 2023","interfaces":[{"interface":"back","average":{"1min":0.72499999999999998,"5min":0.72499999999999998,"15min":0.72499999999999998},"min":{"1min":0.37,"5min":0.37,"15min":0.37},"max":{"1min":2.7269999999999999,"5min":2.7269999999999999,"15min":2.7269999999999999},"last":0.42099999999999999},{"interface":"front","average":{"1min":0.64500000000000002,"5min":0.64500000000000002,"15min":0.64500000000000002},"min":{"1min":0.44700000000000001,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":0.84199999999999997,"5min":0.84199999999999997,"15min":0.84199999999999997},"last":0.53100000000000003}]},{"osd":6,"last update":"Fri Nov 24 07:57:22 2023","interfaces":[{"interface":"back","average":{"1min":0.65800000000000003,"5min":0.65800000000000003,"15min":0.65800000000000003},"min":{"1min":0.47499999999999998,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":0.98199999999999998,"5min":0.98199999999999998,"15min":0.98199999999999998},"last":0.57799999999999996},{"interface":"front","average":{"1min":0.61699999999999999,"5min":0.61699999999999999,"15min":0.61699999999999999},"min":{"1min":0.42899999999999999,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":0.90700000000000003,"5min":0.90700000000000003,"15min":0.90700000000000003},"last":0.48199999999999998}]},{"osd":7,"last update":"Fri Nov 24 07:57:22 2023","interfaces":[{"interface":"back","average":{"1min":0.63700000000000001,"5min":0.63700000000000001,"15min":0.63700000000000001},"min":{"1min":0.40999999999999998,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":1.0649999999999999,"5min":1.0649999999999999,"15min":1.0649999999999999},"last":0.45200000000000001},{"interface":"front","average":{"1min":0.67000000000000004,"5min":0.67000000000000004,"15min":0.67000000000000004},"min":{"1min":0.442,"5min":0.442,"15min":0.442},"max":{"1min":0.88800000000000001,"5min":0.88800000000000001,"15min":0.88800000000000001},"last":0.55700000000000005}]}]},{"osd":0,"up_from":13,"seq":55834574866,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27340,"kb_used_data":316,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344500,"statfs":{"total":96636764160,"available":96608768000,"internally_reserved":0,"allocated":323584,"data_stored":120931,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Nov 24 07:57:18 2023","interfaces":[{"interface":"back","average":{"1min":0.71299999999999997,"5min":0.71299999999999997,"15min":0.71299999999999997},"min":{"1min":0.23499999999999999,"5min":0.23499999999999999,"15min":0.23499999999999999},"max":{"1min":6.3550000000000004,"5min":6.3550000000000004,"15min":6.3550000000000004},"last":0.32400000000000001},{"interface":"front","average":{"1min":1.7290000000000001,"5min":1.7290000000000001,"15min":1.7290000000000001},"min":{"1min":0.30599999999999999,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":22.916,"5min":22.916,"15min":22.916},"last":0.36699999999999999}]},{"osd":2,"last update":"Fri Nov 24 07:57:18 2023","interfaces":[{"interface":"back","average":{"1min":0.69999999999999996,"5min":0.69999999999999996,"15min":0.69999999999999996},"min":{"1min":0.22800000000000001,"5min":0.22800000000000001,"15min":0.22800000000000001},"max":{"1min":6.3899999999999997,"5min":6.3899999999999997,"15min":6.3899999999999997},"last":0.379},{"interface":"front","average":{"1min":1.0629999999999999,"5min":1.0629999999999999,"15min":1.0629999999999999},"min":{"1min":0.26800000000000002,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":11.93,"5min":11.93,"15min":11.93},"last":0.30299999999999999}]},{"osd":3,"last update":"Fri Nov 24 07:57:18 2023","interfaces":[{"interface":"back","average":{"1min":0.41999999999999998,"5min":0.41999999999999998,"15min":0.41999999999999998},"min":{"1min":0.28399999999999997,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":1.155,"5min":1.155,"15min":1.155},"last":0.33600000000000002},{"interface":"front","average":{"1min":1.127,"5min":1.127,"15min":1.127},"min":{"1min":0.32300000000000001,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":12.116,"5min":12.116,"15min":12.116},"last":0.35299999999999998}]},{"osd":4,"last update":"Fri Nov 24 07:57:18 2023","interfaces":[{"interface":"back","average":{"1min":0.64800000000000002,"5min":0.64800000000000002,"15min":0.64800000000000002},"min":{"1min":0.40500000000000003,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":0.97899999999999998,"5min":0.97899999999999998,"15min":0.97899999999999998},"last":0.77400000000000002},{"interface":"front","average":{"1min":1.9339999999999999,"5min":1.9339999999999999,"15min":1.9339999999999999},"min":{"1min":0.44400000000000001,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":22.963999999999999,"5min":22.963999999999999,"15min":22.963999999999999},"last":0.55400000000000005}]},{"osd":5,"last update":"Fri Nov 24 07:57:18 2023","interfaces":[{"interface":"back","average":{"1min":1.2929999999999999,"5min":1.2929999999999999,"15min":1.2929999999999999},"min":{"1min":0.40200000000000002,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":12.084,"5min":12.084,"15min":12.084},"last":0.52100000000000002},{"interface":"front","average":{"1min":0.67800000000000005,"5min":0.67800000000000005,"15min":0.67800000000000005},"min":{"1min":0.45200000000000001,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":1.073,"5min":1.073,"15min":1.073},"last":0.73799999999999999}]},{"osd":6,"last update":"Fri Nov 24 07:57:18 2023","interfaces":[{"interface":"back","average":{"1min":0.69099999999999995,"5min":0.69099999999999995,"15min":0.69099999999999995},"min":{"1min":0.42799999999999999,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":1.0580000000000001,"5min":1.0580000000000001,"15min":1.0580000000000001},"last":0.69499999999999995},{"interface":"front","average":{"1min":1.3140000000000001,"5min":1.3140000000000001,"15min":1.3140000000000001},"min":{"1min":0.41899999999999998,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":12.045,"5min":12.045,"15min":12.045},"last":0.54700000000000004}]},{"osd":7,"last update":"Fri Nov 24 07:57:18 2023","interfaces":[{"interface":"back","average":{"1min":0.65600000000000003,"5min":0.65600000000000003,"15min":0.65600000000000003},"min":{"1min":0.45400000000000001,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":1.038,"5min":1.038,"15min":1.038},"last":0.57499999999999996},{"interface":"front","average":{"1min":1.9630000000000001,"5min":1.9630000000000001,"15min":1.9630000000000001},"min":{"1min":0.49099999999999999,"5min":0.49099999999999999,"15min":0.49099999999999999},"max":{"1min":22.995000000000001,"5min":22.995000000000001,"15min":22.995000000000001},"last":0.752}]}]},{"osd":2,"up_from":13,"seq":55834574866,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27336,"kb_used_data":316,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344504,"statfs":{"total":96636764160,"available":96608772096,"internally_reserved":0,"allocated":323584,"data_stored":120931,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 07:57:17 2023","interfaces":[{"interface":"back","average":{"1min":0.371,"5min":0.371,"15min":0.371},"min":{"1min":0.22500000000000001,"5min":0.22500000000000001,"15min":0.22500000000000001},"max":{"1min":0.58099999999999996,"5min":0.58099999999999996,"15min":0.58099999999999996},"last":0.24199999999999999},{"interface":"front","average":{"1min":0.40200000000000002,"5min":0.40200000000000002,"15min":0.40200000000000002},"min":{"1min":0.27400000000000002,"5min":0.27400000000000002,"15min":0.27400000000000002},"max":{"1min":0.54100000000000004,"5min":0.54100000000000004,"15min":0.54100000000000004},"last":0.38600000000000001}]},{"osd":1,"last update":"Fri Nov 24 07:57:17 2023","interfaces":[{"interface":"back","average":{"1min":0.378,"5min":0.378,"15min":0.378},"min":{"1min":0.23499999999999999,"5min":0.23499999999999999,"15min":0.23499999999999999},"max":{"1min":0.59199999999999997,"5min":0.59199999999999997,"15min":0.59199999999999997},"last":0.27600000000000002},{"interface":"front","average":{"1min":0.41599999999999998,"5min":0.41599999999999998,"15min":0.41599999999999998},"min":{"1min":0.25600000000000001,"5min":0.25600000000000001,"15min":0.25600000000000001},"max":{"1min":0.627,"5min":0.627,"15min":0.627},"last":0.30299999999999999}]},{"osd":3,"last update":"Fri Nov 24 07:57:17 2023","interfaces":[{"interface":"back","average":{"1min":0.41099999999999998,"5min":0.41099999999999998,"15min":0.41099999999999998},"min":{"1min":0.308,"5min":0.308,"15min":0.308},"max":{"1min":0.53300000000000003,"5min":0.53300000000000003,"15min":0.53300000000000003},"last":0.371},{"interface":"front","average":{"1min":0.41399999999999998,"5min":0.41399999999999998,"15min":0.41399999999999998},"min":{"1min":0.27500000000000002,"5min":0.27500000000000002,"15min":0.27500000000000002},"max":{"1min":0.64600000000000002,"5min":0.64600000000000002,"15min":0.64600000000000002},"last":0.35399999999999998}]},{"osd":4,"last update":"Fri Nov 24 07:57:17 2023","interfaces":[{"interface":"back","average":{"1min":0.60799999999999998,"5min":0.60799999999999998,"15min":0.60799999999999998},"min":{"1min":0.42999999999999999,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":0.81599999999999995,"5min":0.81599999999999995,"15min":0.81599999999999995},"last":0.51000000000000001},{"interface":"front","average":{"1min":0.59999999999999998,"5min":0.59999999999999998,"15min":0.59999999999999998},"min":{"1min":0.41599999999999998,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":0.79600000000000004,"5min":0.79600000000000004,"15min":0.79600000000000004},"last":0.66600000000000004}]},{"osd":5,"last update":"Fri Nov 24 07:57:17 2023","interfaces":[{"interface":"back","average":{"1min":0.621,"5min":0.621,"15min":0.621},"min":{"1min":0.44900000000000001,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":0.84899999999999998,"5min":0.84899999999999998,"15min":0.84899999999999998},"last":0.52400000000000002},{"interface":"front","average":{"1min":0.71499999999999997,"5min":0.71499999999999997,"15min":0.71499999999999997},"min":{"1min":0.433,"5min":0.433,"15min":0.433},"max":{"1min":2.1699999999999999,"5min":2.1699999999999999,"15min":2.1699999999999999},"last":0.56399999999999995}]},{"osd":6,"last update":"Fri Nov 24 07:57:17 2023","interfaces":[{"interface":"back","average":{"1min":0.66700000000000004,"5min":0.66700000000000004,"15min":0.66700000000000004},"min":{"1min":0.47399999999999998,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":0.83599999999999997,"5min":0.83599999999999997,"15min":0.83599999999999997},"last":0.65800000000000003},{"interface":"front","average":{"1min":0.67100000000000004,"5min":0.67100000000000004,"15min":0.67100000000000004},"min":{"1min":0.496,"5min":0.496,"15min":0.496},"max":{"1min":0.88500000000000001,"5min":0.88500000000000001,"15min":0.88500000000000001},"last":0.58099999999999996}]},{"osd":7,"last update":"Fri Nov 24 07:57:17 2023","interfaces":[{"interface":"back","average":{"1min":0.66200000000000003,"5min":0.66200000000000003,"15min":0.66200000000000003},"min":{"1min":0.443,"5min":0.443,"15min":0.443},"max":{"1min":0.85999999999999999,"5min":0.85999999999999999,"15min":0.85999999999999999},"last":0.60799999999999998},{"interface":"front","average":{"1min":0.68000000000000005,"5min":0.68000000000000005,"15min":0.68000000000000005},"min":{"1min":0.495,"5min":0.495,"15min":0.495},"max":{"1min":0.86399999999999999,"5min":0.86399999999999999,"15min":0.86399999999999999},"last":0.63100000000000001}]}]},{"osd":3,"up_from":13,"seq":55834574866,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27340,"kb_used_data":316,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344500,"statfs":{"total":96636764160,"available":96608768000,"internally_reserved":0,"allocated":323584,"data_stored":120931,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 07:57:20 2023","interfaces":[{"interface":"back","average":{"1min":0.35599999999999998,"5min":0.35599999999999998,"15min":0.35599999999999998},"min":{"1min":0.222,"5min":0.222,"15min":0.222},"max":{"1min":0.84799999999999998,"5min":0.84799999999999998,"15min":0.84799999999999998},"last":0.439},{"interface":"front","average":{"1min":0.36699999999999999,"5min":0.36699999999999999,"15min":0.36699999999999999},"min":{"1min":0.23699999999999999,"5min":0.23699999999999999,"15min":0.23699999999999999},"max":{"1min":0.873,"5min":0.873,"15min":0.873},"last":0.307}]},{"osd":1,"last update":"Fri Nov 24 07:57:20 2023","interfaces":[{"interface":"back","average":{"1min":0.375,"5min":0.375,"15min":0.375},"min":{"1min":0.25700000000000001,"5min":0.25700000000000001,"15min":0.25700000000000001},"max":{"1min":0.92000000000000004,"5min":0.92000000000000004,"15min":0.92000000000000004},"last":0.36899999999999999},{"interface":"front","average":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"min":{"1min":0.248,"5min":0.248,"15min":0.248},"max":{"1min":0.81699999999999995,"5min":0.81699999999999995,"15min":0.81699999999999995},"last":0.34300000000000003}]},{"osd":2,"last update":"Fri Nov 24 07:57:20 2023","interfaces":[{"interface":"back","average":{"1min":0.36099999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"min":{"1min":0.24399999999999999,"5min":0.24399999999999999,"15min":0.24399999999999999},"max":{"1min":0.88500000000000001,"5min":0.88500000000000001,"15min":0.88500000000000001},"last":0.38900000000000001},{"interface":"front","average":{"1min":0.46800000000000003,"5min":0.46800000000000003,"15min":0.46800000000000003},"min":{"1min":0.31900000000000001,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.98099999999999998,"5min":0.98099999999999998,"15min":0.98099999999999998},"last":0.41199999999999998}]},{"osd":4,"last update":"Fri Nov 24 07:57:20 2023","interfaces":[{"interface":"back","average":{"1min":0.79700000000000004,"5min":0.79700000000000004,"15min":0.79700000000000004},"min":{"1min":0.33500000000000002,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":5.7350000000000003,"5min":5.7350000000000003,"15min":5.7350000000000003},"last":0.57499999999999996},{"interface":"front","average":{"1min":0.58499999999999996,"5min":0.58499999999999996,"15min":0.58499999999999996},"min":{"1min":0.44500000000000001,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":1.27,"5min":1.27,"15min":1.27},"last":0.65800000000000003}]},{"osd":5,"last update":"Fri Nov 24 07:57:20 2023","interfaces":[{"interface":"back","average":{"1min":0.58799999999999997,"5min":0.58799999999999997,"15min":0.58799999999999997},"min":{"1min":0.40699999999999997,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":1.4430000000000001,"5min":1.4430000000000001,"15min":1.4430000000000001},"last":0.50700000000000001},{"interface":"front","average":{"1min":0.627,"5min":0.627,"15min":0.627},"min":{"1min":0.377,"5min":0.377,"15min":0.377},"max":{"1min":1.4239999999999999,"5min":1.4239999999999999,"15min":1.4239999999999999},"last":0.55200000000000005}]},{"osd":6,"last update":"Fri Nov 24 07:57:20 2023","interfaces":[{"interface":"back","average":{"1min":0.62,"5min":0.62,"15min":0.62},"min":{"1min":0.434,"5min":0.434,"15min":0.434},"max":{"1min":1.355,"5min":1.355,"15min":1.355},"last":0.49199999999999999},{"interface":"front","average":{"1min":0.84599999999999997,"5min":0.84599999999999997,"15min":0.84599999999999997},"min":{"1min":0.40999999999999998,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":5.6479999999999997,"5min":5.6479999999999997,"15min":5.6479999999999997},"last":0.53100000000000003}]},{"osd":7,"last update":"Fri Nov 24 07:57:20 2023","interfaces":[{"interface":"back","average":{"1min":0.66600000000000004,"5min":0.66600000000000004,"15min":0.66600000000000004},"min":{"1min":0.42299999999999999,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":1.51,"5min":1.51,"15min":1.51},"last":0.48099999999999998},{"interface":"front","average":{"1min":0.84199999999999997,"5min":0.84199999999999997,"15min":0.84199999999999997},"min":{"1min":0.45600000000000002,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":5.7069999999999999,"5min":5.7069999999999999,"15min":5.7069999999999999},"last":0.54000000000000004}]}]},{"osd":4,"up_from":13,"seq":55834574865,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27336,"kb_used_data":316,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344504,"statfs":{"total":96636764160,"available":96608772096,"internally_reserved":0,"allocated":323584,"data_stored":120931,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 07:57:21 2023","interfaces":[{"interface":"back","average":{"1min":0.58099999999999996,"5min":0.58099999999999996,"15min":0.58099999999999996},"min":{"1min":0.37,"5min":0.37,"15min":0.37},"max":{"1min":0.79300000000000004,"5min":0.79300000000000004,"15min":0.79300000000000004},"last":0.58199999999999996},{"interface":"front","average":{"1min":0.54300000000000004,"5min":0.54300000000000004,"15min":0.54300000000000004},"min":{"1min":0.33600000000000002,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":1.0249999999999999,"5min":1.0249999999999999,"15min":1.0249999999999999},"last":0.73899999999999999}]},{"osd":1,"last update":"Fri Nov 24 07:57:21 2023","interfaces":[{"interface":"back","average":{"1min":0.55200000000000005,"5min":0.55200000000000005,"15min":0.55200000000000005},"min":{"1min":0.40000000000000002,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":0.86399999999999999,"5min":0.86399999999999999,"15min":0.86399999999999999},"last":0.60299999999999998},{"interface":"front","average":{"1min":0.55100000000000005,"5min":0.55100000000000005,"15min":0.55100000000000005},"min":{"1min":0.36099999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.98699999999999999,"5min":0.98699999999999999,"15min":0.98699999999999999},"last":0.42399999999999999}]},{"osd":2,"last update":"Fri Nov 24 07:57:21 2023","interfaces":[{"interface":"back","average":{"1min":0.56100000000000005,"5min":0.56100000000000005,"15min":0.56100000000000005},"min":{"1min":0.35799999999999998,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.81100000000000005,"5min":0.81100000000000005,"15min":0.81100000000000005},"last":0.52100000000000002},{"interface":"front","average":{"1min":0.61299999999999999,"5min":0.61299999999999999,"15min":0.61299999999999999},"min":{"1min":0.38100000000000001,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.89200000000000002,"5min":0.89200000000000002,"15min":0.89200000000000002},"last":0.38100000000000001}]},{"osd":3,"last update":"Fri Nov 24 07:57:21 2023","interfaces":[{"interface":"back","average":{"1min":0.58499999999999996,"5min":0.58499999999999996,"15min":0.58499999999999996},"min":{"1min":0.439,"5min":0.439,"15min":0.439},"max":{"1min":0.89700000000000002,"5min":0.89700000000000002,"15min":0.89700000000000002},"last":0.47499999999999998},{"interface":"front","average":{"1min":0.63100000000000001,"5min":0.63100000000000001,"15min":0.63100000000000001},"min":{"1min":0.40300000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.96199999999999997,"5min":0.96199999999999997,"15min":0.96199999999999997},"last":0.71399999999999997}]},{"osd":5,"last update":"Fri Nov 24 07:57:21 2023","interfaces":[{"interface":"back","average":{"1min":0.45500000000000002,"5min":0.45500000000000002,"15min":0.45500000000000002},"min":{"1min":0.29799999999999999,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":0.70499999999999996,"5min":0.70499999999999996,"15min":0.70499999999999996},"last":0.40200000000000002},{"interface":"front","average":{"1min":0.61899999999999999,"5min":0.61899999999999999,"15min":0.61899999999999999},"min":{"1min":0.439,"5min":0.439,"15min":0.439},"max":{"1min":0.86399999999999999,"5min":0.86399999999999999,"15min":0.86399999999999999},"last":0.68600000000000005}]},{"osd":6,"last update":"Fri Nov 24 07:57:21 2023","interfaces":[{"interface":"back","average":{"1min":0.58599999999999997,"5min":0.58599999999999997,"15min":0.58599999999999997},"min":{"1min":0.35499999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.76800000000000002,"5min":0.76800000000000002,"15min":0.76800000000000002},"last":0.67100000000000004},{"interface":"front","average":{"1min":0.61399999999999999,"5min":0.61399999999999999,"15min":0.61399999999999999},"min":{"1min":0.436,"5min":0.436,"15min":0.436},"max":{"1min":1.095,"5min":1.095,"15min":1.095},"last":0.78600000000000003}]},{"osd":7,"last update":"Fri Nov 24 07:57:21 2023","interfaces":[{"interface":"back","average":{"1min":0.57299999999999995,"5min":0.57299999999999995,"15min":0.57299999999999995},"min":{"1min":0.35099999999999998,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.89300000000000002,"5min":0.89300000000000002,"15min":0.89300000000000002},"last":0.63700000000000001},{"interface":"front","average":{"1min":0.66300000000000003,"5min":0.66300000000000003,"15min":0.66300000000000003},"min":{"1min":0.41099999999999998,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":0.91500000000000004,"5min":0.91500000000000004,"15min":0.91500000000000004},"last":0.83599999999999997}]}]},{"osd":5,"up_from":13,"seq":55834574866,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27928,"kb_used_data":896,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343912,"statfs":{"total":96636764160,"available":96608165888,"internally_reserved":0,"allocated":917504,"data_stored":711299,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 07:57:17 2023","interfaces":[{"interface":"back","average":{"1min":0.56200000000000006,"5min":0.56200000000000006,"15min":0.56200000000000006},"min":{"1min":0.32700000000000001,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":1.0109999999999999,"5min":1.0109999999999999,"15min":1.0109999999999999},"last":0.85999999999999999},{"interface":"front","average":{"1min":0.80000000000000004,"5min":0.80000000000000004,"15min":0.80000000000000004},"min":{"1min":0.33400000000000002,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":5.431,"5min":5.431,"15min":5.431},"last":0.58899999999999997}]},{"osd":1,"last update":"Fri Nov 24 07:57:17 2023","interfaces":[{"interface":"back","average":{"1min":0.55900000000000005,"5min":0.55900000000000005,"15min":0.55900000000000005},"min":{"1min":0.28899999999999998,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":1.016,"5min":1.016,"15min":1.016},"last":0.67300000000000004},{"interface":"front","average":{"1min":0.56999999999999995,"5min":0.56999999999999995,"15min":0.56999999999999995},"min":{"1min":0.311,"5min":0.311,"15min":0.311},"max":{"1min":0.96499999999999997,"5min":0.96499999999999997,"15min":0.96499999999999997},"last":0.54100000000000004}]},{"osd":2,"last update":"Fri Nov 24 07:57:17 2023","interfaces":[{"interface":"back","average":{"1min":0.55100000000000005,"5min":0.55100000000000005,"15min":0.55100000000000005},"min":{"1min":0.377,"5min":0.377,"15min":0.377},"max":{"1min":1.163,"5min":1.163,"15min":1.163},"last":0.443},{"interface":"front","average":{"1min":0.63,"5min":0.63,"15min":0.63},"min":{"1min":0.39200000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":1.119,"5min":1.119,"15min":1.119},"last":0.75700000000000001}]},{"osd":3,"last update":"Fri Nov 24 07:57:17 2023","interfaces":[{"interface":"back","average":{"1min":0.90500000000000003,"5min":0.90500000000000003,"15min":0.90500000000000003},"min":{"1min":0.36499999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":5.609,"5min":5.609,"15min":5.609},"last":0.64000000000000001},{"interface":"front","average":{"1min":0.67600000000000005,"5min":0.67600000000000005,"15min":0.67600000000000005},"min":{"1min":0.41899999999999998,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":1.288,"5min":1.288,"15min":1.288},"last":0.83499999999999996}]},{"osd":4,"last update":"Fri Nov 24 07:57:17 2023","interfaces":[{"interface":"back","average":{"1min":0.51000000000000001,"5min":0.51000000000000001,"15min":0.51000000000000001},"min":{"1min":0.36399999999999999,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.68799999999999994,"5min":0.68799999999999994,"15min":0.68799999999999994},"last":0.623},{"interface":"front","average":{"1min":0.57199999999999995,"5min":0.57199999999999995,"15min":0.57199999999999995},"min":{"1min":0.39800000000000002,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.78300000000000003,"5min":0.78300000000000003,"15min":0.78300000000000003},"last":0.78300000000000003}]},{"osd":6,"last update":"Fri Nov 24 07:57:17 2023","interfaces":[{"interface":"back","average":{"1min":0.57099999999999995,"5min":0.57099999999999995,"15min":0.57099999999999995},"min":{"1min":0.35599999999999998,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.89700000000000002,"5min":0.89700000000000002,"15min":0.89700000000000002},"last":0.89700000000000002},{"interface":"front","average":{"1min":0.625,"5min":0.625,"15min":0.625},"min":{"1min":0.42899999999999999,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":0.88200000000000001,"5min":0.88200000000000001,"15min":0.88200000000000001},"last":0.88200000000000001}]},{"osd":7,"last update":"Fri Nov 24 07:57:17 2023","interfaces":[{"interface":"back","average":{"1min":0.64600000000000002,"5min":0.64600000000000002,"15min":0.64600000000000002},"min":{"1min":0.45000000000000001,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":0.96599999999999997,"5min":0.96599999999999997,"15min":0.96599999999999997},"last":0.91000000000000003},{"interface":"front","average":{"1min":0.65300000000000002,"5min":0.65300000000000002,"15min":0.65300000000000002},"min":{"1min":0.46300000000000002,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":0.94199999999999995,"5min":0.94199999999999995,"15min":0.94199999999999995},"last":0.94199999999999995}]}]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-11-24T07:57:30.198 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-11-24T07:57:30.360 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:30.358+0000 7f40b0e19700 1 Processor -- start 2023-11-24T07:57:30.360 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:30.358+0000 7f40b0e19700 1 -- start start 2023-11-24T07:57:30.363 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:30.359+0000 7f40b0e19700 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f40ac06b620 0x7f40ac0b3a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:57:30.363 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:30.359+0000 7f40b0e19700 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f40ac1629c0 con 0x7f40ac0b5ab0 2023-11-24T07:57:30.363 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:30.359+0000 7f40b0e19700 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f40ac071f50 con 0x7f40ac06ee60 2023-11-24T07:57:30.364 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:30.359+0000 7f40b0e19700 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f40ac072230 con 0x7f40ac06b620 2023-11-24T07:57:30.364 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:30.359+0000 7f40a9d9b700 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7f40ac0b5ab0 0x7f40ac161ba0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:55254/0 (socket says 172.21.15.59:55254) 2023-11-24T07:57:30.364 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:30.359+0000 7f40a9d9b700 1 -- 172.21.15.59:0/2072756444 learned_addr learned my addr 172.21.15.59:0/2072756444 (peer_addr_for_me v1:172.21.15.59:0/0) 2023-11-24T07:57:30.364 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:30.359+0000 7f40aa59c700 1 --2- 172.21.15.59:0/2072756444 >> v2:172.21.15.76:3300/0 conn(0x7f40ac06b620 0x7f40ac0b3a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:57:30.364 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:30.359+0000 7f40a959a700 1 -- 172.21.15.59:0/2072756444 <== mon.0 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3906552686 0 0) 0x7f40ac1629c0 con 0x7f40ac0b5ab0 2023-11-24T07:57:30.364 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:30.360+0000 7f40a959a700 1 -- 172.21.15.59:0/2072756444 --> v1:172.21.15.59:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f40940035b0 con 0x7f40ac0b5ab0 2023-11-24T07:57:30.364 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:30.360+0000 7f40a959a700 1 -- 172.21.15.59:0/2072756444 <== mon.0 v1:172.21.15.59:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3574447321 0 0) 0x7f40940035b0 con 0x7f40ac0b5ab0 2023-11-24T07:57:30.364 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:30.360+0000 7f40a959a700 1 -- 172.21.15.59:0/2072756444 >> v2:172.21.15.76:3300/0 conn(0x7f40ac06b620 msgr2=0x7f40ac0b3a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T07:57:30.364 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:30.360+0000 7f40a959a700 1 --2- 172.21.15.59:0/2072756444 >> v2:172.21.15.76:3300/0 conn(0x7f40ac06b620 0x7f40ac0b3a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:30.364 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:30.360+0000 7f40a959a700 1 -- 172.21.15.59:0/2072756444 >> v1:172.21.15.76:6789/0 conn(0x7f40ac06ee60 legacy=0x7f40ac1622b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:57:30.364 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:30.360+0000 7f40a959a700 1 -- 172.21.15.59:0/2072756444 --> v1:172.21.15.59:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f40ac072510 con 0x7f40ac0b5ab0 2023-11-24T07:57:30.364 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:30.360+0000 7f40a959a700 1 -- 172.21.15.59:0/2072756444 <== mon.0 v1:172.21.15.59:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1312333566 0 0) 0x7f409c001670 con 0x7f40ac0b5ab0 2023-11-24T07:57:30.365 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:30.360+0000 7f40a959a700 1 -- 172.21.15.59:0/2072756444 >> v1:172.21.15.59:6789/0 conn(0x7f40ac0b5ab0 legacy=0x7f40ac161ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:57:30.365 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:30.360+0000 7f40a959a700 1 --2- 172.21.15.59:0/2072756444 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f4094003a00 0x7f4094003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:57:30.365 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:30.360+0000 7f40a959a700 1 --2- 172.21.15.59:0/2072756444 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4094004850 0x7f4094004cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:57:30.365 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:30.360+0000 7f40a959a700 1 --2- 172.21.15.59:0/2072756444 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4094005210 0x7f409400ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:57:30.365 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:30.360+0000 7f40a959a700 1 -- 172.21.15.59:0/2072756444 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40ac072230 con 0x7f4094004850 2023-11-24T07:57:30.365 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:30.360+0000 7f40a959a700 1 -- 172.21.15.59:0/2072756444 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f409400c0b0 con 0x7f4094005210 2023-11-24T07:57:30.365 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:30.360+0000 7f40a959a700 1 -- 172.21.15.59:0/2072756444 --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f409400c270 con 0x7f4094003a00 2023-11-24T07:57:30.365 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:30.360+0000 7f40a9d9b700 1 --2- 172.21.15.59:0/2072756444 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4094004850 0x7f4094004cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:57:30.365 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:30.360+0000 7f40aad9d700 1 --2- 172.21.15.59:0/2072756444 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4094005210 0x7f409400ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:57:30.365 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:30.360+0000 7f40aa59c700 1 --2- 172.21.15.59:0/2072756444 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f4094003a00 0x7f4094003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:57:30.365 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:30.360+0000 7f40a9d9b700 1 -- 172.21.15.59:0/2072756444 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f4094003a00 msgr2=0x7f4094003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T07:57:30.365 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:30.360+0000 7f40a9d9b700 1 --2- 172.21.15.59:0/2072756444 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f4094003a00 0x7f4094003e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:30.366 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:30.360+0000 7f40a9d9b700 1 -- 172.21.15.59:0/2072756444 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4094005210 msgr2=0x7f409400ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T07:57:30.366 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:30.360+0000 7f40a9d9b700 1 --2- 172.21.15.59:0/2072756444 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4094005210 0x7f409400ba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:30.366 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:30.360+0000 7f40a9d9b700 1 -- 172.21.15.59:0/2072756444 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f40940035b0 con 0x7f4094004850 2023-11-24T07:57:30.366 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:30.360+0000 7f40aad9d700 1 -- 172.21.15.59:0/2072756444 reap_dead start 2023-11-24T07:57:30.366 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:30.361+0000 7f40a9d9b700 1 --2- 172.21.15.59:0/2072756444 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4094004850 0x7f4094004cd0 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f409c00c270 tx=0x7f4094005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e59f193449e6a2b3 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:57:30.366 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:30.361+0000 7f40a959a700 1 -- 172.21.15.59:0/2072756444 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f409c007c10 con 0x7f4094004850 2023-11-24T07:57:30.366 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:30.361+0000 7f40a959a700 1 -- 172.21.15.59:0/2072756444 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f409c00cd90 con 0x7f4094004850 2023-11-24T07:57:30.366 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:30.361+0000 7f40a959a700 1 -- 172.21.15.59:0/2072756444 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f409c005ba0 con 0x7f4094004850 2023-11-24T07:57:30.366 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:30.361+0000 7f40b0e19700 1 -- 172.21.15.59:0/2072756444 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4094004850 msgr2=0x7f4094004cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:57:30.366 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:30.361+0000 7f40b0e19700 1 --2- 172.21.15.59:0/2072756444 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4094004850 0x7f4094004cd0 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f409c00c270 tx=0x7f4094005880 comp rx=0 tx=0).stop 2023-11-24T07:57:30.366 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:30.361+0000 7f40b0e19700 1 -- 172.21.15.59:0/2072756444 shutdown_connections 2023-11-24T07:57:30.367 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:30.361+0000 7f40b0e19700 1 --2- 172.21.15.59:0/2072756444 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4094004850 0x7f4094004cd0 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:30.367 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:30.361+0000 7f40b0e19700 1 -- 172.21.15.59:0/2072756444 >> 172.21.15.59:0/2072756444 conn(0x7f40ac0b2740 msgr2=0x7f40ac0b2b20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:57:30.367 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:30.361+0000 7f40b0e19700 1 -- 172.21.15.59:0/2072756444 shutdown_connections 2023-11-24T07:57:30.367 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:30.361+0000 7f40b0e19700 1 -- 172.21.15.59:0/2072756444 wait complete. 2023-11-24T07:57:30.369 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:30.363+0000 7f40b0e19700 1 Processor -- start 2023-11-24T07:57:30.369 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:30.363+0000 7f40b0e19700 1 -- start start 2023-11-24T07:57:30.370 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:30.363+0000 7f40b0e19700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4094004850 0x7f40ac13c440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:57:30.370 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:30.363+0000 7f40b0e19700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f40ac1f6fd0 0x7f40ac1f7450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:57:30.370 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:30.363+0000 7f40b0e19700 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f40ac13e1c0 0x7f40ac13e640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:57:30.370 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:30.363+0000 7f40b0e19700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40ac1f7990 con 0x7f40ac1f6fd0 2023-11-24T07:57:30.370 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:30.363+0000 7f40b0e19700 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40ac13ef40 con 0x7f4094004850 2023-11-24T07:57:30.370 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:30.363+0000 7f40b0e19700 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f40ac151520 con 0x7f40ac13e1c0 2023-11-24T07:57:30.370 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:30.364+0000 7f40a9d9b700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f40ac1f6fd0 0x7f40ac1f7450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:57:30.370 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:30.364+0000 7f40a9d9b700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f40ac1f6fd0 0x7f40ac1f7450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:51794/0 (socket says 172.21.15.59:51794) 2023-11-24T07:57:30.370 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:30.364+0000 7f40a9d9b700 1 -- 172.21.15.59:0/2553874962 learned_addr learned my addr 172.21.15.59:0/2553874962 (peer_addr_for_me v2:172.21.15.59:0/0) 2023-11-24T07:57:30.370 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:30.364+0000 7f40aa59c700 1 --2- 172.21.15.59:0/2553874962 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4094004850 0x7f40ac13c440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:57:30.370 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:30.364+0000 7f40aad9d700 1 --2- 172.21.15.59:0/2553874962 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f40ac13e1c0 0x7f40ac13e640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:57:30.371 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:30.364+0000 7f40a9d9b700 1 -- 172.21.15.59:0/2553874962 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f40ac13e1c0 msgr2=0x7f40ac13e640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:57:30.371 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:30.364+0000 7f40a9d9b700 1 --2- 172.21.15.59:0/2553874962 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f40ac13e1c0 0x7f40ac13e640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:30.371 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:30.364+0000 7f40a9d9b700 1 -- 172.21.15.59:0/2553874962 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4094004850 msgr2=0x7f40ac13c440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:57:30.371 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:30.364+0000 7f40a9d9b700 1 --2- 172.21.15.59:0/2553874962 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4094004850 0x7f40ac13c440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:30.371 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:30.364+0000 7f40a9d9b700 1 -- 172.21.15.59:0/2553874962 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f40ac151800 con 0x7f40ac1f6fd0 2023-11-24T07:57:30.371 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:30.364+0000 7f40aa59c700 1 --2- 172.21.15.59:0/2553874962 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4094004850 0x7f40ac13c440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-24T07:57:30.371 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:30.364+0000 7f40a9d9b700 1 --2- 172.21.15.59:0/2553874962 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f40ac1f6fd0 0x7f40ac1f7450 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f409400c080 tx=0x7f40ac06f2b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:57:30.371 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:30.364+0000 7f409affd700 1 -- 172.21.15.59:0/2553874962 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f409c000ed0 con 0x7f40ac1f6fd0 2023-11-24T07:57:30.371 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:30.364+0000 7f409affd700 1 -- 172.21.15.59:0/2553874962 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f409c0070d0 con 0x7f40ac1f6fd0 2023-11-24T07:57:30.371 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:30.364+0000 7f40b0e19700 1 -- 172.21.15.59:0/2553874962 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f40ac151ae0 con 0x7f40ac1f6fd0 2023-11-24T07:57:30.371 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:30.364+0000 7f409affd700 1 -- 172.21.15.59:0/2553874962 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f409c007440 con 0x7f40ac1f6fd0 2023-11-24T07:57:30.372 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:30.364+0000 7f40b0e19700 1 -- 172.21.15.59:0/2553874962 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f40ac13c980 con 0x7f40ac1f6fd0 2023-11-24T07:57:30.372 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:30.365+0000 7f40b0e19700 1 -- 172.21.15.59:0/2553874962 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f40ac152130 con 0x7f40ac1f6fd0 2023-11-24T07:57:30.372 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:30.366+0000 7f409affd700 1 -- 172.21.15.59:0/2553874962 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f409c0075a0 con 0x7f40ac1f6fd0 2023-11-24T07:57:30.372 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:30.366+0000 7f409affd700 1 --2- 172.21.15.59:0/2553874962 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f408c06aed0 0x7f408c06d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:57:30.372 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:30.366+0000 7f409affd700 1 -- 172.21.15.59:0/2553874962 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f409c08a530 con 0x7f40ac1f6fd0 2023-11-24T07:57:30.372 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:30.366+0000 7f40aa59c700 1 --2- 172.21.15.59:0/2553874962 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f408c06aed0 0x7f408c06d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:57:30.372 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:30.367+0000 7f40aa59c700 1 --2- 172.21.15.59:0/2553874962 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f408c06aed0 0x7f408c06d390 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f40ac1f6f30 tx=0x7f40a0008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:57:30.372 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:30.371+0000 7f409affd700 1 -- 172.21.15.59:0/2553874962 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f409c055670 con 0x7f40ac1f6fd0 2023-11-24T07:57:30.569 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:30.567+0000 7f40b0e19700 1 -- 172.21.15.59:0/2553874962 --> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f40ac072ad0 con 0x7f408c06aed0 2023-11-24T07:57:30.578 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:30.576+0000 7f409affd700 1 -- 172.21.15.59:0/2553874962 <== mgr.4100 v2:172.21.15.76:6832/4264046536 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+288949 (secure 0 0 0) 0x7f40a0009950 con 0x7f408c06aed0 2023-11-24T07:57:30.581 INFO:teuthology.orchestra.run.smithi059.stdout: 2023-11-24T07:57:30.583 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:30.579+0000 7f40b0e19700 1 -- 172.21.15.59:0/2553874962 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f408c06aed0 msgr2=0x7f408c06d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:57:30.584 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:30.579+0000 7f40b0e19700 1 --2- 172.21.15.59:0/2553874962 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f408c06aed0 0x7f408c06d390 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f40ac1f6f30 tx=0x7f40a0008040 comp rx=0 tx=0).stop 2023-11-24T07:57:30.584 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:30.579+0000 7f40b0e19700 1 -- 172.21.15.59:0/2553874962 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f40ac1f6fd0 msgr2=0x7f40ac1f7450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:57:30.584 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:30.579+0000 7f40b0e19700 1 --2- 172.21.15.59:0/2553874962 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f40ac1f6fd0 0x7f40ac1f7450 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f409400c080 tx=0x7f40ac06f2b0 comp rx=0 tx=0).stop 2023-11-24T07:57:30.584 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:30.579+0000 7f40b0e19700 1 -- 172.21.15.59:0/2553874962 shutdown_connections 2023-11-24T07:57:30.584 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:30.579+0000 7f40b0e19700 1 --2- 172.21.15.59:0/2553874962 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f408c06aed0 0x7f408c06d390 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:30.584 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:30.579+0000 7f40b0e19700 1 --2- 172.21.15.59:0/2553874962 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f40ac13e1c0 0x7f40ac13e640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:30.584 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:30.579+0000 7f40b0e19700 1 --2- 172.21.15.59:0/2553874962 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4094004850 0x7f40ac13c440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:30.584 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:30.580+0000 7f40b0e19700 1 --2- 172.21.15.59:0/2553874962 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f40ac1f6fd0 0x7f40ac1f7450 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:30.584 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:30.580+0000 7f40b0e19700 1 -- 172.21.15.59:0/2553874962 >> 172.21.15.59:0/2553874962 conn(0x7f40ac0b2740 msgr2=0x7f40ac0b1ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:57:30.584 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:30.580+0000 7f40b0e19700 1 -- 172.21.15.59:0/2553874962 shutdown_connections 2023-11-24T07:57:30.585 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:30.580+0000 7f40b0e19700 1 -- 172.21.15.59:0/2553874962 wait complete. 2023-11-24T07:57:30.585 INFO:teuthology.orchestra.run.smithi059.stderr:dumped all 2023-11-24T07:57:30.593 INFO:teuthology.orchestra.run.smithi059.stdout:{"pg_ready":true,"pg_map":{"version":62,"stamp":"2023-11-24T07:57:30.314053+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":97,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":220336,"kb_used_data":4136,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754754384,"statfs":{"total":773094113280,"available":772868489216,"internally_reserved":0,"allocated":4235264,"data_stored":2354340,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12745,"internal_metadata":221236791},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":33,"apply_latency_ms":33,"commit_latency_ns":33000000,"apply_latency_ns":33000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001473"},"pg_stats":[{"pgid":"5.3f","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-24T07:57:27.498774+0000","last_change":"2023-11-24T07:57:25.613284+0000","last_active":"2023-11-24T07:57:27.498774+0000","last_peered":"2023-11-24T07:57:27.498774+0000","last_clean":"2023-11-24T07:57:27.498774+0000","last_became_active":"2023-11-24T07:57:25.606187+0000","last_became_peered":"2023-11-24T07:57:25.606187+0000","last_unstale":"2023-11-24T07:57:27.498774+0000","last_undegraded":"2023-11-24T07:57:27.498774+0000","last_fullsized":"2023-11-24T07:57:27.498774+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:57:24.572681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:57:24.572681+0000","last_clean_scrub_stamp":"2023-11-24T07:57:24.572681+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T12:51:13.357718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.3e","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-24T07:57:26.584176+0000","last_change":"2023-11-24T07:57:25.611359+0000","last_active":"2023-11-24T07:57:26.584176+0000","last_peered":"2023-11-24T07:57:26.584176+0000","last_clean":"2023-11-24T07:57:26.584176+0000","last_became_active":"2023-11-24T07:57:25.611053+0000","last_became_peered":"2023-11-24T07:57:25.611053+0000","last_unstale":"2023-11-24T07:57:26.584176+0000","last_undegraded":"2023-11-24T07:57:26.584176+0000","last_fullsized":"2023-11-24T07:57:26.584176+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:57:24.572681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:57:24.572681+0000","last_clean_scrub_stamp":"2023-11-24T07:57:24.572681+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T15:47:12.125658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.3d","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-24T07:57:27.499082+0000","last_change":"2023-11-24T07:57:25.601956+0000","last_active":"2023-11-24T07:57:27.499082+0000","last_peered":"2023-11-24T07:57:27.499082+0000","last_clean":"2023-11-24T07:57:27.499082+0000","last_became_active":"2023-11-24T07:57:25.601517+0000","last_became_peered":"2023-11-24T07:57:25.601517+0000","last_unstale":"2023-11-24T07:57:27.499082+0000","last_undegraded":"2023-11-24T07:57:27.499082+0000","last_fullsized":"2023-11-24T07:57:27.499082+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:57:24.572681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:57:24.572681+0000","last_clean_scrub_stamp":"2023-11-24T07:57:24.572681+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T17:40:49.956916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.3c","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-24T07:57:27.499452+0000","last_change":"2023-11-24T07:57:25.606653+0000","last_active":"2023-11-24T07:57:27.499452+0000","last_peered":"2023-11-24T07:57:27.499452+0000","last_clean":"2023-11-24T07:57:27.499452+0000","last_became_active":"2023-11-24T07:57:25.606315+0000","last_became_peered":"2023-11-24T07:57:25.606315+0000","last_unstale":"2023-11-24T07:57:27.499452+0000","last_undegraded":"2023-11-24T07:57:27.499452+0000","last_fullsized":"2023-11-24T07:57:27.499452+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:57:24.572681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:57:24.572681+0000","last_clean_scrub_stamp":"2023-11-24T07:57:24.572681+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T13:39:16.521602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.3b","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-24T07:57:26.583942+0000","last_change":"2023-11-24T07:57:25.600519+0000","last_active":"2023-11-24T07:57:26.583942+0000","last_peered":"2023-11-24T07:57:26.583942+0000","last_clean":"2023-11-24T07:57:26.583942+0000","last_became_active":"2023-11-24T07:57:25.600235+0000","last_became_peered":"2023-11-24T07:57:25.600235+0000","last_unstale":"2023-11-24T07:57:26.583942+0000","last_undegraded":"2023-11-24T07:57:26.583942+0000","last_fullsized":"2023-11-24T07:57:26.583942+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:57:24.572681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:57:24.572681+0000","last_clean_scrub_stamp":"2023-11-24T07:57:24.572681+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T19:13:40.799042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.3a","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-24T07:57:27.498253+0000","last_change":"2023-11-24T07:57:25.600774+0000","last_active":"2023-11-24T07:57:27.498253+0000","last_peered":"2023-11-24T07:57:27.498253+0000","last_clean":"2023-11-24T07:57:27.498253+0000","last_became_active":"2023-11-24T07:57:25.600390+0000","last_became_peered":"2023-11-24T07:57:25.600390+0000","last_unstale":"2023-11-24T07:57:27.498253+0000","last_undegraded":"2023-11-24T07:57:27.498253+0000","last_fullsized":"2023-11-24T07:57:27.498253+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:57:24.572681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:57:24.572681+0000","last_clean_scrub_stamp":"2023-11-24T07:57:24.572681+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T17:53:33.447022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.39","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-24T07:57:27.499083+0000","last_change":"2023-11-24T07:57:25.596753+0000","last_active":"2023-11-24T07:57:27.499083+0000","last_peered":"2023-11-24T07:57:27.499083+0000","last_clean":"2023-11-24T07:57:27.499083+0000","last_became_active":"2023-11-24T07:57:25.596467+0000","last_became_peered":"2023-11-24T07:57:25.596467+0000","last_unstale":"2023-11-24T07:57:27.499083+0000","last_undegraded":"2023-11-24T07:57:27.499083+0000","last_fullsized":"2023-11-24T07:57:27.499083+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:57:24.572681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:57:24.572681+0000","last_clean_scrub_stamp":"2023-11-24T07:57:24.572681+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T14:04:03.022069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.38","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-24T07:57:27.498354+0000","last_change":"2023-11-24T07:57:25.601277+0000","last_active":"2023-11-24T07:57:27.498354+0000","last_peered":"2023-11-24T07:57:27.498354+0000","last_clean":"2023-11-24T07:57:27.498354+0000","last_became_active":"2023-11-24T07:57:25.600867+0000","last_became_peered":"2023-11-24T07:57:25.600867+0000","last_unstale":"2023-11-24T07:57:27.498354+0000","last_undegraded":"2023-11-24T07:57:27.498354+0000","last_fullsized":"2023-11-24T07:57:27.498354+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:57:24.572681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:57:24.572681+0000","last_clean_scrub_stamp":"2023-11-24T07:57:24.572681+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T16:47:55.590649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.37","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-24T07:57:27.499310+0000","last_change":"2023-11-24T07:57:25.611574+0000","last_active":"2023-11-24T07:57:27.499310+0000","last_peered":"2023-11-24T07:57:27.499310+0000","last_clean":"2023-11-24T07:57:27.499310+0000","last_became_active":"2023-11-24T07:57:25.611258+0000","last_became_peered":"2023-11-24T07:57:25.611258+0000","last_unstale":"2023-11-24T07:57:27.499310+0000","last_undegraded":"2023-11-24T07:57:27.499310+0000","last_fullsized":"2023-11-24T07:57:27.499310+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:57:24.572681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:57:24.572681+0000","last_clean_scrub_stamp":"2023-11-24T07:57:24.572681+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T15:54:26.161043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.36","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-24T07:57:27.499746+0000","last_change":"2023-11-24T07:57:25.606465+0000","last_active":"2023-11-24T07:57:27.499746+0000","last_peered":"2023-11-24T07:57:27.499746+0000","last_clean":"2023-11-24T07:57:27.499746+0000","last_became_active":"2023-11-24T07:57:25.606204+0000","last_became_peered":"2023-11-24T07:57:25.606204+0000","last_unstale":"2023-11-24T07:57:27.499746+0000","last_undegraded":"2023-11-24T07:57:27.499746+0000","last_fullsized":"2023-11-24T07:57:27.499746+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:57:24.572681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:57:24.572681+0000","last_clean_scrub_stamp":"2023-11-24T07:57:24.572681+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T18:02:42.589387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.35","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-24T07:57:26.583720+0000","last_change":"2023-11-24T07:57:25.610107+0000","last_active":"2023-11-24T07:57:26.583720+0000","last_peered":"2023-11-24T07:57:26.583720+0000","last_clean":"2023-11-24T07:57:26.583720+0000","last_became_active":"2023-11-24T07:57:25.609413+0000","last_became_peered":"2023-11-24T07:57:25.609413+0000","last_unstale":"2023-11-24T07:57:26.583720+0000","last_undegraded":"2023-11-24T07:57:26.583720+0000","last_fullsized":"2023-11-24T07:57:26.583720+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:57:24.572681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:57:24.572681+0000","last_clean_scrub_stamp":"2023-11-24T07:57:24.572681+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T17:08:50.195049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.34","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-24T07:57:27.498828+0000","last_change":"2023-11-24T07:57:25.600896+0000","last_active":"2023-11-24T07:57:27.498828+0000","last_peered":"2023-11-24T07:57:27.498828+0000","last_clean":"2023-11-24T07:57:27.498828+0000","last_became_active":"2023-11-24T07:57:25.600611+0000","last_became_peered":"2023-11-24T07:57:25.600611+0000","last_unstale":"2023-11-24T07:57:27.498828+0000","last_undegraded":"2023-11-24T07:57:27.498828+0000","last_fullsized":"2023-11-24T07:57:27.498828+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:57:24.572681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:57:24.572681+0000","last_clean_scrub_stamp":"2023-11-24T07:57:24.572681+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T17:53:33.447022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.33","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-24T07:57:27.499277+0000","last_change":"2023-11-24T07:57:25.609608+0000","last_active":"2023-11-24T07:57:27.499277+0000","last_peered":"2023-11-24T07:57:27.499277+0000","last_clean":"2023-11-24T07:57:27.499277+0000","last_became_active":"2023-11-24T07:57:25.609381+0000","last_became_peered":"2023-11-24T07:57:25.609381+0000","last_unstale":"2023-11-24T07:57:27.499277+0000","last_undegraded":"2023-11-24T07:57:27.499277+0000","last_fullsized":"2023-11-24T07:57:27.499277+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:57:24.572681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:57:24.572681+0000","last_clean_scrub_stamp":"2023-11-24T07:57:24.572681+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T19:30:34.653712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.32","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-24T07:57:27.499306+0000","last_change":"2023-11-24T07:57:25.605234+0000","last_active":"2023-11-24T07:57:27.499306+0000","last_peered":"2023-11-24T07:57:27.499306+0000","last_clean":"2023-11-24T07:57:27.499306+0000","last_became_active":"2023-11-24T07:57:25.604959+0000","last_became_peered":"2023-11-24T07:57:25.604959+0000","last_unstale":"2023-11-24T07:57:27.499306+0000","last_undegraded":"2023-11-24T07:57:27.499306+0000","last_fullsized":"2023-11-24T07:57:27.499306+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:57:24.572681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:57:24.572681+0000","last_clean_scrub_stamp":"2023-11-24T07:57:24.572681+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:18:21.970465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.31","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-24T07:57:26.582759+0000","last_change":"2023-11-24T07:57:25.605974+0000","last_active":"2023-11-24T07:57:26.582759+0000","last_peered":"2023-11-24T07:57:26.582759+0000","last_clean":"2023-11-24T07:57:26.582759+0000","last_became_active":"2023-11-24T07:57:25.604012+0000","last_became_peered":"2023-11-24T07:57:25.604012+0000","last_unstale":"2023-11-24T07:57:26.582759+0000","last_undegraded":"2023-11-24T07:57:26.582759+0000","last_fullsized":"2023-11-24T07:57:26.582759+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:57:24.572681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:57:24.572681+0000","last_clean_scrub_stamp":"2023-11-24T07:57:24.572681+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T12:51:13.357718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.30","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-24T07:57:26.582547+0000","last_change":"2023-11-24T07:57:25.603887+0000","last_active":"2023-11-24T07:57:26.582547+0000","last_peered":"2023-11-24T07:57:26.582547+0000","last_clean":"2023-11-24T07:57:26.582547+0000","last_became_active":"2023-11-24T07:57:25.603484+0000","last_became_peered":"2023-11-24T07:57:25.603484+0000","last_unstale":"2023-11-24T07:57:26.582547+0000","last_undegraded":"2023-11-24T07:57:26.582547+0000","last_fullsized":"2023-11-24T07:57:26.582547+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:57:24.572681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:57:24.572681+0000","last_clean_scrub_stamp":"2023-11-24T07:57:24.572681+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T19:30:34.653712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.2f","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-24T07:57:26.583766+0000","last_change":"2023-11-24T07:57:25.617360+0000","last_active":"2023-11-24T07:57:26.583766+0000","last_peered":"2023-11-24T07:57:26.583766+0000","last_clean":"2023-11-24T07:57:26.583766+0000","last_became_active":"2023-11-24T07:57:25.617094+0000","last_became_peered":"2023-11-24T07:57:25.617094+0000","last_unstale":"2023-11-24T07:57:26.583766+0000","last_undegraded":"2023-11-24T07:57:26.583766+0000","last_fullsized":"2023-11-24T07:57:26.583766+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:57:24.572681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:57:24.572681+0000","last_clean_scrub_stamp":"2023-11-24T07:57:24.572681+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:15:10.655973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2e","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-24T07:57:27.498492+0000","last_change":"2023-11-24T07:57:25.597593+0000","last_active":"2023-11-24T07:57:27.498492+0000","last_peered":"2023-11-24T07:57:27.498492+0000","last_clean":"2023-11-24T07:57:27.498492+0000","last_became_active":"2023-11-24T07:57:25.597301+0000","last_became_peered":"2023-11-24T07:57:25.597301+0000","last_unstale":"2023-11-24T07:57:27.498492+0000","last_undegraded":"2023-11-24T07:57:27.498492+0000","last_fullsized":"2023-11-24T07:57:27.498492+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:57:24.572681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:57:24.572681+0000","last_clean_scrub_stamp":"2023-11-24T07:57:24.572681+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T17:40:49.956916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2d","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-24T07:57:27.498313+0000","last_change":"2023-11-24T07:57:25.598070+0000","last_active":"2023-11-24T07:57:27.498313+0000","last_peered":"2023-11-24T07:57:27.498313+0000","last_clean":"2023-11-24T07:57:27.498313+0000","last_became_active":"2023-11-24T07:57:25.597726+0000","last_became_peered":"2023-11-24T07:57:25.597726+0000","last_unstale":"2023-11-24T07:57:27.498313+0000","last_undegraded":"2023-11-24T07:57:27.498313+0000","last_fullsized":"2023-11-24T07:57:27.498313+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:57:24.572681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:57:24.572681+0000","last_clean_scrub_stamp":"2023-11-24T07:57:24.572681+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T14:49:20.210209+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2c","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-24T07:57:27.498975+0000","last_change":"2023-11-24T07:57:25.607256+0000","last_active":"2023-11-24T07:57:27.498975+0000","last_peered":"2023-11-24T07:57:27.498975+0000","last_clean":"2023-11-24T07:57:27.498975+0000","last_became_active":"2023-11-24T07:57:25.606931+0000","last_became_peered":"2023-11-24T07:57:25.606931+0000","last_unstale":"2023-11-24T07:57:27.498975+0000","last_undegraded":"2023-11-24T07:57:27.498975+0000","last_fullsized":"2023-11-24T07:57:27.498975+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:57:24.572681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:57:24.572681+0000","last_clean_scrub_stamp":"2023-11-24T07:57:24.572681+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T12:58:55.859434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.2b","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-24T07:57:26.583134+0000","last_change":"2023-11-24T07:57:25.603944+0000","last_active":"2023-11-24T07:57:26.583134+0000","last_peered":"2023-11-24T07:57:26.583134+0000","last_clean":"2023-11-24T07:57:26.583134+0000","last_became_active":"2023-11-24T07:57:25.603636+0000","last_became_peered":"2023-11-24T07:57:25.603636+0000","last_unstale":"2023-11-24T07:57:26.583134+0000","last_undegraded":"2023-11-24T07:57:26.583134+0000","last_fullsized":"2023-11-24T07:57:26.583134+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:57:24.572681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:57:24.572681+0000","last_clean_scrub_stamp":"2023-11-24T07:57:24.572681+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T15:47:12.125658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.2a","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-24T07:57:27.498828+0000","last_change":"2023-11-24T07:57:25.615209+0000","last_active":"2023-11-24T07:57:27.498828+0000","last_peered":"2023-11-24T07:57:27.498828+0000","last_clean":"2023-11-24T07:57:27.498828+0000","last_became_active":"2023-11-24T07:57:25.614789+0000","last_became_peered":"2023-11-24T07:57:25.614789+0000","last_unstale":"2023-11-24T07:57:27.498828+0000","last_undegraded":"2023-11-24T07:57:27.498828+0000","last_fullsized":"2023-11-24T07:57:27.498828+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:57:24.572681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:57:24.572681+0000","last_clean_scrub_stamp":"2023-11-24T07:57:24.572681+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T15:32:38.628605+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.29","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-24T07:57:27.498472+0000","last_change":"2023-11-24T07:57:25.596319+0000","last_active":"2023-11-24T07:57:27.498472+0000","last_peered":"2023-11-24T07:57:27.498472+0000","last_clean":"2023-11-24T07:57:27.498472+0000","last_became_active":"2023-11-24T07:57:25.596016+0000","last_became_peered":"2023-11-24T07:57:25.596016+0000","last_unstale":"2023-11-24T07:57:27.498472+0000","last_undegraded":"2023-11-24T07:57:27.498472+0000","last_fullsized":"2023-11-24T07:57:27.498472+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:57:24.572681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:57:24.572681+0000","last_clean_scrub_stamp":"2023-11-24T07:57:24.572681+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T13:25:29.169219+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.28","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-24T07:57:27.498409+0000","last_change":"2023-11-24T07:57:25.613820+0000","last_active":"2023-11-24T07:57:27.498409+0000","last_peered":"2023-11-24T07:57:27.498409+0000","last_clean":"2023-11-24T07:57:27.498409+0000","last_became_active":"2023-11-24T07:57:25.613589+0000","last_became_peered":"2023-11-24T07:57:25.613589+0000","last_unstale":"2023-11-24T07:57:27.498409+0000","last_undegraded":"2023-11-24T07:57:27.498409+0000","last_fullsized":"2023-11-24T07:57:27.498409+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:57:24.572681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:57:24.572681+0000","last_clean_scrub_stamp":"2023-11-24T07:57:24.572681+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T12:58:55.859434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.27","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-24T07:57:27.499878+0000","last_change":"2023-11-24T07:57:25.609636+0000","last_active":"2023-11-24T07:57:27.499878+0000","last_peered":"2023-11-24T07:57:27.499878+0000","last_clean":"2023-11-24T07:57:27.499878+0000","last_became_active":"2023-11-24T07:57:25.609368+0000","last_became_peered":"2023-11-24T07:57:25.609368+0000","last_unstale":"2023-11-24T07:57:27.499878+0000","last_undegraded":"2023-11-24T07:57:27.499878+0000","last_fullsized":"2023-11-24T07:57:27.499878+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:57:24.572681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:57:24.572681+0000","last_clean_scrub_stamp":"2023-11-24T07:57:24.572681+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T19:27:55.072502+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.26","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-24T07:57:27.499815+0000","last_change":"2023-11-24T07:57:25.606180+0000","last_active":"2023-11-24T07:57:27.499815+0000","last_peered":"2023-11-24T07:57:27.499815+0000","last_clean":"2023-11-24T07:57:27.499815+0000","last_became_active":"2023-11-24T07:57:25.605879+0000","last_became_peered":"2023-11-24T07:57:25.605879+0000","last_unstale":"2023-11-24T07:57:27.499815+0000","last_undegraded":"2023-11-24T07:57:27.499815+0000","last_fullsized":"2023-11-24T07:57:27.499815+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:57:24.572681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:57:24.572681+0000","last_clean_scrub_stamp":"2023-11-24T07:57:24.572681+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T15:46:37.491547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.25","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-24T07:57:26.584072+0000","last_change":"2023-11-24T07:57:25.605311+0000","last_active":"2023-11-24T07:57:26.584072+0000","last_peered":"2023-11-24T07:57:26.584072+0000","last_clean":"2023-11-24T07:57:26.584072+0000","last_became_active":"2023-11-24T07:57:25.605045+0000","last_became_peered":"2023-11-24T07:57:25.605045+0000","last_unstale":"2023-11-24T07:57:26.584072+0000","last_undegraded":"2023-11-24T07:57:26.584072+0000","last_fullsized":"2023-11-24T07:57:26.584072+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:57:24.572681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:57:24.572681+0000","last_clean_scrub_stamp":"2023-11-24T07:57:24.572681+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T15:32:38.628605+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.24","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-24T07:57:27.498555+0000","last_change":"2023-11-24T07:57:25.601259+0000","last_active":"2023-11-24T07:57:27.498555+0000","last_peered":"2023-11-24T07:57:27.498555+0000","last_clean":"2023-11-24T07:57:27.498555+0000","last_became_active":"2023-11-24T07:57:25.600941+0000","last_became_peered":"2023-11-24T07:57:25.600941+0000","last_unstale":"2023-11-24T07:57:27.498555+0000","last_undegraded":"2023-11-24T07:57:27.498555+0000","last_fullsized":"2023-11-24T07:57:27.498555+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:57:24.572681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:57:24.572681+0000","last_clean_scrub_stamp":"2023-11-24T07:57:24.572681+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T19:30:34.653712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.23","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-24T07:57:26.583176+0000","last_change":"2023-11-24T07:57:25.609706+0000","last_active":"2023-11-24T07:57:26.583176+0000","last_peered":"2023-11-24T07:57:26.583176+0000","last_clean":"2023-11-24T07:57:26.583176+0000","last_became_active":"2023-11-24T07:57:25.609419+0000","last_became_peered":"2023-11-24T07:57:25.609419+0000","last_unstale":"2023-11-24T07:57:26.583176+0000","last_undegraded":"2023-11-24T07:57:26.583176+0000","last_fullsized":"2023-11-24T07:57:26.583176+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:57:24.572681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:57:24.572681+0000","last_clean_scrub_stamp":"2023-11-24T07:57:24.572681+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T14:04:03.022069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.22","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-24T07:57:26.582928+0000","last_change":"2023-11-24T07:57:25.607107+0000","last_active":"2023-11-24T07:57:26.582928+0000","last_peered":"2023-11-24T07:57:26.582928+0000","last_clean":"2023-11-24T07:57:26.582928+0000","last_became_active":"2023-11-24T07:57:25.606797+0000","last_became_peered":"2023-11-24T07:57:25.606797+0000","last_unstale":"2023-11-24T07:57:26.582928+0000","last_undegraded":"2023-11-24T07:57:26.582928+0000","last_fullsized":"2023-11-24T07:57:26.582928+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:57:24.572681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:57:24.572681+0000","last_clean_scrub_stamp":"2023-11-24T07:57:24.572681+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T15:32:38.628605+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.21","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-24T07:57:27.499178+0000","last_change":"2023-11-24T07:57:25.605343+0000","last_active":"2023-11-24T07:57:27.499178+0000","last_peered":"2023-11-24T07:57:27.499178+0000","last_clean":"2023-11-24T07:57:27.499178+0000","last_became_active":"2023-11-24T07:57:25.605002+0000","last_became_peered":"2023-11-24T07:57:25.605002+0000","last_unstale":"2023-11-24T07:57:27.499178+0000","last_undegraded":"2023-11-24T07:57:27.499178+0000","last_fullsized":"2023-11-24T07:57:27.499178+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:57:24.572681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:57:24.572681+0000","last_clean_scrub_stamp":"2023-11-24T07:57:24.572681+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:18:21.970465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.20","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-24T07:57:26.583562+0000","last_change":"2023-11-24T07:57:25.615397+0000","last_active":"2023-11-24T07:57:26.583562+0000","last_peered":"2023-11-24T07:57:26.583562+0000","last_clean":"2023-11-24T07:57:26.583562+0000","last_became_active":"2023-11-24T07:57:25.615122+0000","last_became_peered":"2023-11-24T07:57:25.615122+0000","last_unstale":"2023-11-24T07:57:26.583562+0000","last_undegraded":"2023-11-24T07:57:26.583562+0000","last_fullsized":"2023-11-24T07:57:26.583562+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:57:24.572681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:57:24.572681+0000","last_clean_scrub_stamp":"2023-11-24T07:57:24.572681+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:32:55.195260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-24T07:57:27.498923+0000","last_change":"2023-11-24T07:57:25.612902+0000","last_active":"2023-11-24T07:57:27.498923+0000","last_peered":"2023-11-24T07:57:27.498923+0000","last_clean":"2023-11-24T07:57:27.498923+0000","last_became_active":"2023-11-24T07:57:25.612479+0000","last_became_peered":"2023-11-24T07:57:25.612479+0000","last_unstale":"2023-11-24T07:57:27.498923+0000","last_undegraded":"2023-11-24T07:57:27.498923+0000","last_fullsized":"2023-11-24T07:57:27.498923+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:57:24.572681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:57:24.572681+0000","last_clean_scrub_stamp":"2023-11-24T07:57:24.572681+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T16:47:55.590649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1f","version":"32'4","reported_seq":16,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-24T07:57:26.584058+0000","last_change":"2023-11-24T07:57:24.600719+0000","last_active":"2023-11-24T07:57:26.584058+0000","last_peered":"2023-11-24T07:57:26.584058+0000","last_clean":"2023-11-24T07:57:26.584058+0000","last_became_active":"2023-11-24T07:57:24.600406+0000","last_became_peered":"2023-11-24T07:57:24.600406+0000","last_unstale":"2023-11-24T07:57:26.584058+0000","last_undegraded":"2023-11-24T07:57:26.584058+0000","last_fullsized":"2023-11-24T07:57:26.584058+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:57:23.565116+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:57:23.565116+0000","last_clean_scrub_stamp":"2023-11-24T07:57:23.565116+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T14:20:43.593019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1f","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-24T07:57:27.498803+0000","last_change":"2023-11-24T07:57:25.600985+0000","last_active":"2023-11-24T07:57:27.498803+0000","last_peered":"2023-11-24T07:57:27.498803+0000","last_clean":"2023-11-24T07:57:27.498803+0000","last_became_active":"2023-11-24T07:57:25.600673+0000","last_became_peered":"2023-11-24T07:57:25.600673+0000","last_unstale":"2023-11-24T07:57:27.498803+0000","last_undegraded":"2023-11-24T07:57:27.498803+0000","last_fullsized":"2023-11-24T07:57:27.498803+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:57:24.572681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:57:24.572681+0000","last_clean_scrub_stamp":"2023-11-24T07:57:24.572681+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T19:13:40.799042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.1e","version":"32'1","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-24T07:57:27.499834+0000","last_change":"2023-11-24T07:57:24.597208+0000","last_active":"2023-11-24T07:57:27.499834+0000","last_peered":"2023-11-24T07:57:27.499834+0000","last_clean":"2023-11-24T07:57:27.499834+0000","last_became_active":"2023-11-24T07:57:24.596957+0000","last_became_peered":"2023-11-24T07:57:24.596957+0000","last_unstale":"2023-11-24T07:57:27.499834+0000","last_undegraded":"2023-11-24T07:57:27.499834+0000","last_fullsized":"2023-11-24T07:57:27.499834+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:57:23.565116+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:57:23.565116+0000","last_clean_scrub_stamp":"2023-11-24T07:57:23.565116+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T12:51:12.350152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-24T07:57:27.498872+0000","last_change":"2023-11-24T07:57:25.615058+0000","last_active":"2023-11-24T07:57:27.498872+0000","last_peered":"2023-11-24T07:57:27.498872+0000","last_clean":"2023-11-24T07:57:27.498872+0000","last_became_active":"2023-11-24T07:57:25.614773+0000","last_became_peered":"2023-11-24T07:57:25.614773+0000","last_unstale":"2023-11-24T07:57:27.498872+0000","last_undegraded":"2023-11-24T07:57:27.498872+0000","last_fullsized":"2023-11-24T07:57:27.498872+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:57:24.572681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:57:24.572681+0000","last_clean_scrub_stamp":"2023-11-24T07:57:24.572681+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T15:47:12.125658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.1d","version":"32'2","reported_seq":14,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-24T07:57:27.498444+0000","last_change":"2023-11-24T07:57:24.602623+0000","last_active":"2023-11-24T07:57:27.498444+0000","last_peered":"2023-11-24T07:57:27.498444+0000","last_clean":"2023-11-24T07:57:27.498444+0000","last_became_active":"2023-11-24T07:57:24.602295+0000","last_became_peered":"2023-11-24T07:57:24.602295+0000","last_unstale":"2023-11-24T07:57:27.498444+0000","last_undegraded":"2023-11-24T07:57:27.498444+0000","last_fullsized":"2023-11-24T07:57:27.498444+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:57:23.565116+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:57:23.565116+0000","last_clean_scrub_stamp":"2023-11-24T07:57:23.565116+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T15:12:48.897533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-24T07:57:27.499822+0000","last_change":"2023-11-24T07:57:25.604332+0000","last_active":"2023-11-24T07:57:27.499822+0000","last_peered":"2023-11-24T07:57:27.499822+0000","last_clean":"2023-11-24T07:57:27.499822+0000","last_became_active":"2023-11-24T07:57:25.603998+0000","last_became_peered":"2023-11-24T07:57:25.603998+0000","last_unstale":"2023-11-24T07:57:27.499822+0000","last_undegraded":"2023-11-24T07:57:27.499822+0000","last_fullsized":"2023-11-24T07:57:27.499822+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:57:24.572681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:57:24.572681+0000","last_clean_scrub_stamp":"2023-11-24T07:57:24.572681+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T11:10:18.640767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1c","version":"32'2","reported_seq":14,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-24T07:57:27.499815+0000","last_change":"2023-11-24T07:57:24.595626+0000","last_active":"2023-11-24T07:57:27.499815+0000","last_peered":"2023-11-24T07:57:27.499815+0000","last_clean":"2023-11-24T07:57:27.499815+0000","last_became_active":"2023-11-24T07:57:24.595343+0000","last_became_peered":"2023-11-24T07:57:24.595343+0000","last_unstale":"2023-11-24T07:57:27.499815+0000","last_undegraded":"2023-11-24T07:57:27.499815+0000","last_fullsized":"2023-11-24T07:57:27.499815+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:57:23.565116+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:57:23.565116+0000","last_clean_scrub_stamp":"2023-11-24T07:57:23.565116+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T19:13:39.791476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-24T07:57:27.498316+0000","last_change":"2023-11-24T07:57:25.611772+0000","last_active":"2023-11-24T07:57:27.498316+0000","last_peered":"2023-11-24T07:57:27.498316+0000","last_clean":"2023-11-24T07:57:27.498316+0000","last_became_active":"2023-11-24T07:57:25.611486+0000","last_became_peered":"2023-11-24T07:57:25.611486+0000","last_unstale":"2023-11-24T07:57:27.498316+0000","last_undegraded":"2023-11-24T07:57:27.498316+0000","last_fullsized":"2023-11-24T07:57:27.498316+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:57:24.572681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:57:24.572681+0000","last_clean_scrub_stamp":"2023-11-24T07:57:24.572681+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T15:12:49.905098+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1b","version":"32'1","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-24T07:57:27.498458+0000","last_change":"2023-11-24T07:57:24.589658+0000","last_active":"2023-11-24T07:57:27.498458+0000","last_peered":"2023-11-24T07:57:27.498458+0000","last_clean":"2023-11-24T07:57:27.498458+0000","last_became_active":"2023-11-24T07:57:24.589385+0000","last_became_peered":"2023-11-24T07:57:24.589385+0000","last_unstale":"2023-11-24T07:57:27.498458+0000","last_undegraded":"2023-11-24T07:57:27.498458+0000","last_fullsized":"2023-11-24T07:57:27.498458+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:57:23.565116+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:57:23.565116+0000","last_clean_scrub_stamp":"2023-11-24T07:57:23.565116+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T16:10:47.555733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-24T07:57:27.498658+0000","last_change":"2023-11-24T07:57:25.605184+0000","last_active":"2023-11-24T07:57:27.498658+0000","last_peered":"2023-11-24T07:57:27.498658+0000","last_clean":"2023-11-24T07:57:27.498658+0000","last_became_active":"2023-11-24T07:57:25.604881+0000","last_became_peered":"2023-11-24T07:57:25.604881+0000","last_unstale":"2023-11-24T07:57:27.498658+0000","last_undegraded":"2023-11-24T07:57:27.498658+0000","last_fullsized":"2023-11-24T07:57:27.498658+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:57:24.572681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:57:24.572681+0000","last_clean_scrub_stamp":"2023-11-24T07:57:24.572681+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T09:32:54.344241+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.1a","version":"32'1","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-24T07:57:27.498824+0000","last_change":"2023-11-24T07:57:24.595865+0000","last_active":"2023-11-24T07:57:27.498824+0000","last_peered":"2023-11-24T07:57:27.498824+0000","last_clean":"2023-11-24T07:57:27.498824+0000","last_became_active":"2023-11-24T07:57:24.595549+0000","last_became_peered":"2023-11-24T07:57:24.595549+0000","last_unstale":"2023-11-24T07:57:27.498824+0000","last_undegraded":"2023-11-24T07:57:27.498824+0000","last_fullsized":"2023-11-24T07:57:27.498824+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:57:23.565116+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:57:23.565116+0000","last_clean_scrub_stamp":"2023-11-24T07:57:23.565116+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T15:03:20.689993+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-24T07:57:27.499424+0000","last_change":"2023-11-24T07:57:25.604514+0000","last_active":"2023-11-24T07:57:27.499424+0000","last_peered":"2023-11-24T07:57:27.499424+0000","last_clean":"2023-11-24T07:57:27.499424+0000","last_became_active":"2023-11-24T07:57:25.604243+0000","last_became_peered":"2023-11-24T07:57:25.604243+0000","last_unstale":"2023-11-24T07:57:27.499424+0000","last_undegraded":"2023-11-24T07:57:27.499424+0000","last_fullsized":"2023-11-24T07:57:27.499424+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:57:24.572681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:57:24.572681+0000","last_clean_scrub_stamp":"2023-11-24T07:57:24.572681+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T15:25:09.940051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-24T07:57:27.498339+0000","last_change":"2023-11-24T07:57:24.600360+0000","last_active":"2023-11-24T07:57:27.498339+0000","last_peered":"2023-11-24T07:57:27.498339+0000","last_clean":"2023-11-24T07:57:27.498339+0000","last_became_active":"2023-11-24T07:57:24.600069+0000","last_became_peered":"2023-11-24T07:57:24.600069+0000","last_unstale":"2023-11-24T07:57:27.498339+0000","last_undegraded":"2023-11-24T07:57:27.498339+0000","last_fullsized":"2023-11-24T07:57:27.498339+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:57:23.565116+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:57:23.565116+0000","last_clean_scrub_stamp":"2023-11-24T07:57:23.565116+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T14:43:06.906996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-24T07:57:27.498321+0000","last_change":"2023-11-24T07:57:25.604299+0000","last_active":"2023-11-24T07:57:27.498321+0000","last_peered":"2023-11-24T07:57:27.498321+0000","last_clean":"2023-11-24T07:57:27.498321+0000","last_became_active":"2023-11-24T07:57:25.604033+0000","last_became_peered":"2023-11-24T07:57:25.604033+0000","last_unstale":"2023-11-24T07:57:27.498321+0000","last_undegraded":"2023-11-24T07:57:27.498321+0000","last_fullsized":"2023-11-24T07:57:27.498321+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:57:24.572681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:57:24.572681+0000","last_clean_scrub_stamp":"2023-11-24T07:57:24.572681+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T15:47:12.125658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-24T07:57:27.498908+0000","last_change":"2023-11-24T07:57:24.592424+0000","last_active":"2023-11-24T07:57:27.498908+0000","last_peered":"2023-11-24T07:57:27.498908+0000","last_clean":"2023-11-24T07:57:27.498908+0000","last_became_active":"2023-11-24T07:57:24.592091+0000","last_became_peered":"2023-11-24T07:57:24.592091+0000","last_unstale":"2023-11-24T07:57:27.498908+0000","last_undegraded":"2023-11-24T07:57:27.498908+0000","last_fullsized":"2023-11-24T07:57:27.498908+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:57:23.565116+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:57:23.565116+0000","last_clean_scrub_stamp":"2023-11-24T07:57:23.565116+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T11:43:02.671428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-24T07:57:26.583989+0000","last_change":"2023-11-24T07:57:25.605278+0000","last_active":"2023-11-24T07:57:26.583989+0000","last_peered":"2023-11-24T07:57:26.583989+0000","last_clean":"2023-11-24T07:57:26.583989+0000","last_became_active":"2023-11-24T07:57:25.604986+0000","last_became_peered":"2023-11-24T07:57:25.604986+0000","last_unstale":"2023-11-24T07:57:26.583989+0000","last_undegraded":"2023-11-24T07:57:26.583989+0000","last_fullsized":"2023-11-24T07:57:26.583989+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:57:24.572681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:57:24.572681+0000","last_clean_scrub_stamp":"2023-11-24T07:57:24.572681+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T14:00:49.762285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-24T07:57:26.584132+0000","last_change":"2023-11-24T07:57:24.587676+0000","last_active":"2023-11-24T07:57:26.584132+0000","last_peered":"2023-11-24T07:57:26.584132+0000","last_clean":"2023-11-24T07:57:26.584132+0000","last_became_active":"2023-11-24T07:57:24.587487+0000","last_became_peered":"2023-11-24T07:57:24.587487+0000","last_unstale":"2023-11-24T07:57:26.584132+0000","last_undegraded":"2023-11-24T07:57:26.584132+0000","last_fullsized":"2023-11-24T07:57:26.584132+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:57:23.565116+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:57:23.565116+0000","last_clean_scrub_stamp":"2023-11-24T07:57:23.565116+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T09:32:53.336675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-24T07:57:27.499752+0000","last_change":"2023-11-24T07:57:25.604618+0000","last_active":"2023-11-24T07:57:27.499752+0000","last_peered":"2023-11-24T07:57:27.499752+0000","last_clean":"2023-11-24T07:57:27.499752+0000","last_became_active":"2023-11-24T07:57:25.603754+0000","last_became_peered":"2023-11-24T07:57:25.603754+0000","last_unstale":"2023-11-24T07:57:27.499752+0000","last_undegraded":"2023-11-24T07:57:27.499752+0000","last_fullsized":"2023-11-24T07:57:27.499752+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:57:24.572681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:57:24.572681+0000","last_clean_scrub_stamp":"2023-11-24T07:57:24.572681+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T09:44:29.688267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-24T07:57:26.584309+0000","last_change":"2023-11-24T07:57:24.597830+0000","last_active":"2023-11-24T07:57:26.584309+0000","last_peered":"2023-11-24T07:57:26.584309+0000","last_clean":"2023-11-24T07:57:26.584309+0000","last_became_active":"2023-11-24T07:57:24.597491+0000","last_became_peered":"2023-11-24T07:57:24.597491+0000","last_unstale":"2023-11-24T07:57:26.584309+0000","last_undegraded":"2023-11-24T07:57:26.584309+0000","last_fullsized":"2023-11-24T07:57:26.584309+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:57:23.565116+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:57:23.565116+0000","last_clean_scrub_stamp":"2023-11-24T07:57:23.565116+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T11:43:02.671428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.14","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-24T07:57:27.499713+0000","last_change":"2023-11-24T07:57:25.606580+0000","last_active":"2023-11-24T07:57:27.499713+0000","last_peered":"2023-11-24T07:57:27.499713+0000","last_clean":"2023-11-24T07:57:27.499713+0000","last_became_active":"2023-11-24T07:57:25.606003+0000","last_became_peered":"2023-11-24T07:57:25.606003+0000","last_unstale":"2023-11-24T07:57:27.499713+0000","last_undegraded":"2023-11-24T07:57:27.499713+0000","last_fullsized":"2023-11-24T07:57:27.499713+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:57:24.572681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:57:24.572681+0000","last_clean_scrub_stamp":"2023-11-24T07:57:24.572681+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T08:54:41.853999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.15","version":"32'3","reported_seq":15,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-24T07:57:26.583811+0000","last_change":"2023-11-24T07:57:24.601458+0000","last_active":"2023-11-24T07:57:26.583811+0000","last_peered":"2023-11-24T07:57:26.583811+0000","last_clean":"2023-11-24T07:57:26.583811+0000","last_became_active":"2023-11-24T07:57:24.601087+0000","last_became_peered":"2023-11-24T07:57:24.601087+0000","last_unstale":"2023-11-24T07:57:26.583811+0000","last_undegraded":"2023-11-24T07:57:26.583811+0000","last_fullsized":"2023-11-24T07:57:26.583811+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:57:23.565116+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:57:23.565116+0000","last_clean_scrub_stamp":"2023-11-24T07:57:23.565116+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T08:54:40.846433+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.15","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-24T07:57:26.583962+0000","last_change":"2023-11-24T07:57:25.610172+0000","last_active":"2023-11-24T07:57:26.583962+0000","last_peered":"2023-11-24T07:57:26.583962+0000","last_clean":"2023-11-24T07:57:26.583962+0000","last_became_active":"2023-11-24T07:57:25.609792+0000","last_became_peered":"2023-11-24T07:57:25.609792+0000","last_unstale":"2023-11-24T07:57:26.583962+0000","last_undegraded":"2023-11-24T07:57:26.583962+0000","last_fullsized":"2023-11-24T07:57:26.583962+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:57:24.572681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:57:24.572681+0000","last_clean_scrub_stamp":"2023-11-24T07:57:24.572681+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:36:54.791587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.14","version":"32'6","reported_seq":18,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-24T07:57:27.499082+0000","last_change":"2023-11-24T07:57:24.595177+0000","last_active":"2023-11-24T07:57:27.499082+0000","last_peered":"2023-11-24T07:57:27.499082+0000","last_clean":"2023-11-24T07:57:27.499082+0000","last_became_active":"2023-11-24T07:57:24.594880+0000","last_became_peered":"2023-11-24T07:57:24.594880+0000","last_unstale":"2023-11-24T07:57:27.499082+0000","last_undegraded":"2023-11-24T07:57:27.499082+0000","last_fullsized":"2023-11-24T07:57:27.499082+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:57:23.565116+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:57:23.565116+0000","last_clean_scrub_stamp":"2023-11-24T07:57:23.565116+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T16:05:14.639876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-24T07:57:27.498346+0000","last_change":"2023-11-24T07:57:25.597947+0000","last_active":"2023-11-24T07:57:27.498346+0000","last_peered":"2023-11-24T07:57:27.498346+0000","last_clean":"2023-11-24T07:57:27.498346+0000","last_became_active":"2023-11-24T07:57:25.597634+0000","last_became_peered":"2023-11-24T07:57:25.597634+0000","last_unstale":"2023-11-24T07:57:27.498346+0000","last_undegraded":"2023-11-24T07:57:27.498346+0000","last_fullsized":"2023-11-24T07:57:27.498346+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:57:24.572681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:57:24.572681+0000","last_clean_scrub_stamp":"2023-11-24T07:57:24.572681+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T14:24:57.175517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.13","version":"32'2","reported_seq":14,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-24T07:57:27.498591+0000","last_change":"2023-11-24T07:57:24.600265+0000","last_active":"2023-11-24T07:57:27.498591+0000","last_peered":"2023-11-24T07:57:27.498591+0000","last_clean":"2023-11-24T07:57:27.498591+0000","last_became_active":"2023-11-24T07:57:24.599858+0000","last_became_peered":"2023-11-24T07:57:24.599858+0000","last_unstale":"2023-11-24T07:57:27.498591+0000","last_undegraded":"2023-11-24T07:57:27.498591+0000","last_fullsized":"2023-11-24T07:57:27.498591+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:57:23.565116+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:57:23.565116+0000","last_clean_scrub_stamp":"2023-11-24T07:57:23.565116+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T14:24:56.167951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-24T07:57:27.499668+0000","last_change":"2023-11-24T07:57:25.605157+0000","last_active":"2023-11-24T07:57:27.499668+0000","last_peered":"2023-11-24T07:57:27.499668+0000","last_clean":"2023-11-24T07:57:27.499668+0000","last_became_active":"2023-11-24T07:57:25.604895+0000","last_became_peered":"2023-11-24T07:57:25.604895+0000","last_unstale":"2023-11-24T07:57:27.499668+0000","last_undegraded":"2023-11-24T07:57:27.499668+0000","last_fullsized":"2023-11-24T07:57:27.499668+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:57:24.572681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:57:24.572681+0000","last_clean_scrub_stamp":"2023-11-24T07:57:24.572681+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:34:57.222018+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-24T07:57:27.499175+0000","last_change":"2023-11-24T07:57:24.597916+0000","last_active":"2023-11-24T07:57:27.499175+0000","last_peered":"2023-11-24T07:57:27.499175+0000","last_clean":"2023-11-24T07:57:27.499175+0000","last_became_active":"2023-11-24T07:57:24.597625+0000","last_became_peered":"2023-11-24T07:57:24.597625+0000","last_unstale":"2023-11-24T07:57:27.499175+0000","last_undegraded":"2023-11-24T07:57:27.499175+0000","last_fullsized":"2023-11-24T07:57:27.499175+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:57:23.565116+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:57:23.565116+0000","last_clean_scrub_stamp":"2023-11-24T07:57:23.565116+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T15:32:37.621039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-24T07:57:27.499392+0000","last_change":"2023-11-24T07:57:25.595758+0000","last_active":"2023-11-24T07:57:27.499392+0000","last_peered":"2023-11-24T07:57:27.499392+0000","last_clean":"2023-11-24T07:57:27.499392+0000","last_became_active":"2023-11-24T07:57:25.595488+0000","last_became_peered":"2023-11-24T07:57:25.595488+0000","last_unstale":"2023-11-24T07:57:27.499392+0000","last_undegraded":"2023-11-24T07:57:27.499392+0000","last_fullsized":"2023-11-24T07:57:27.499392+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:57:24.572681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:57:24.572681+0000","last_clean_scrub_stamp":"2023-11-24T07:57:24.572681+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T13:39:16.521602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-24T07:57:27.499388+0000","last_change":"2023-11-24T07:57:24.595527+0000","last_active":"2023-11-24T07:57:27.499388+0000","last_peered":"2023-11-24T07:57:27.499388+0000","last_clean":"2023-11-24T07:57:27.499388+0000","last_became_active":"2023-11-24T07:57:24.595229+0000","last_became_peered":"2023-11-24T07:57:24.595229+0000","last_unstale":"2023-11-24T07:57:27.499388+0000","last_undegraded":"2023-11-24T07:57:27.499388+0000","last_fullsized":"2023-11-24T07:57:27.499388+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:57:23.565116+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:57:23.565116+0000","last_clean_scrub_stamp":"2023-11-24T07:57:23.565116+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T14:24:56.167951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-24T07:57:26.583629+0000","last_change":"2023-11-24T07:57:25.617331+0000","last_active":"2023-11-24T07:57:26.583629+0000","last_peered":"2023-11-24T07:57:26.583629+0000","last_clean":"2023-11-24T07:57:26.583629+0000","last_became_active":"2023-11-24T07:57:25.617033+0000","last_became_peered":"2023-11-24T07:57:25.617033+0000","last_unstale":"2023-11-24T07:57:26.583629+0000","last_undegraded":"2023-11-24T07:57:26.583629+0000","last_fullsized":"2023-11-24T07:57:26.583629+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:57:24.572681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:57:24.572681+0000","last_clean_scrub_stamp":"2023-11-24T07:57:24.572681+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T14:01:46.665356+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.10","version":"32'8","reported_seq":20,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-24T07:57:27.498079+0000","last_change":"2023-11-24T07:57:24.599748+0000","last_active":"2023-11-24T07:57:27.498079+0000","last_peered":"2023-11-24T07:57:27.498079+0000","last_clean":"2023-11-24T07:57:27.498079+0000","last_became_active":"2023-11-24T07:57:24.599456+0000","last_became_peered":"2023-11-24T07:57:24.599456+0000","last_unstale":"2023-11-24T07:57:27.498079+0000","last_undegraded":"2023-11-24T07:57:27.498079+0000","last_fullsized":"2023-11-24T07:57:27.498079+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:57:23.565116+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:57:23.565116+0000","last_clean_scrub_stamp":"2023-11-24T07:57:23.565116+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T19:40:29.710446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.e","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-24T07:57:26.584123+0000","last_change":"2023-11-24T07:57:25.610621+0000","last_active":"2023-11-24T07:57:26.584123+0000","last_peered":"2023-11-24T07:57:26.584123+0000","last_clean":"2023-11-24T07:57:26.584123+0000","last_became_active":"2023-11-24T07:57:25.610341+0000","last_became_peered":"2023-11-24T07:57:25.610341+0000","last_unstale":"2023-11-24T07:57:26.584123+0000","last_undegraded":"2023-11-24T07:57:26.584123+0000","last_fullsized":"2023-11-24T07:57:26.584123+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:57:24.572681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:57:24.572681+0000","last_clean_scrub_stamp":"2023-11-24T07:57:24.572681+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:32:46.923992+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.f","version":"32'1","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-24T07:57:27.498608+0000","last_change":"2023-11-24T07:57:24.600217+0000","last_active":"2023-11-24T07:57:27.498608+0000","last_peered":"2023-11-24T07:57:27.498608+0000","last_clean":"2023-11-24T07:57:27.498608+0000","last_became_active":"2023-11-24T07:57:24.599831+0000","last_became_peered":"2023-11-24T07:57:24.599831+0000","last_unstale":"2023-11-24T07:57:27.498608+0000","last_undegraded":"2023-11-24T07:57:27.498608+0000","last_fullsized":"2023-11-24T07:57:27.498608+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:57:23.565116+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:57:23.565116+0000","last_clean_scrub_stamp":"2023-11-24T07:57:23.565116+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T14:47:45.788163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-24T07:57:27.498862+0000","last_change":"2023-11-24T07:57:25.604503+0000","last_active":"2023-11-24T07:57:27.498862+0000","last_peered":"2023-11-24T07:57:27.498862+0000","last_clean":"2023-11-24T07:57:27.498862+0000","last_became_active":"2023-11-24T07:57:25.604215+0000","last_became_peered":"2023-11-24T07:57:25.604215+0000","last_unstale":"2023-11-24T07:57:27.498862+0000","last_undegraded":"2023-11-24T07:57:27.498862+0000","last_fullsized":"2023-11-24T07:57:27.498862+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:57:24.572681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:57:24.572681+0000","last_clean_scrub_stamp":"2023-11-24T07:57:24.572681+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T14:49:20.210209+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.e","version":"32'2","reported_seq":14,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-24T07:57:27.498400+0000","last_change":"2023-11-24T07:57:24.600226+0000","last_active":"2023-11-24T07:57:27.498400+0000","last_peered":"2023-11-24T07:57:27.498400+0000","last_clean":"2023-11-24T07:57:27.498400+0000","last_became_active":"2023-11-24T07:57:24.599943+0000","last_became_peered":"2023-11-24T07:57:24.599943+0000","last_unstale":"2023-11-24T07:57:27.498400+0000","last_undegraded":"2023-11-24T07:57:27.498400+0000","last_fullsized":"2023-11-24T07:57:27.498400+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:57:23.565116+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:57:23.565116+0000","last_clean_scrub_stamp":"2023-11-24T07:57:23.565116+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T14:33:18.564905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-24T07:57:26.583937+0000","last_change":"2023-11-24T07:57:25.598876+0000","last_active":"2023-11-24T07:57:26.583937+0000","last_peered":"2023-11-24T07:57:26.583937+0000","last_clean":"2023-11-24T07:57:26.583937+0000","last_became_active":"2023-11-24T07:57:25.598514+0000","last_became_peered":"2023-11-24T07:57:25.598514+0000","last_unstale":"2023-11-24T07:57:26.583937+0000","last_undegraded":"2023-11-24T07:57:26.583937+0000","last_fullsized":"2023-11-24T07:57:26.583937+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:57:24.572681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:57:24.572681+0000","last_clean_scrub_stamp":"2023-11-24T07:57:24.572681+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T12:51:13.357718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.d","version":"32'1","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-24T07:57:26.584012+0000","last_change":"2023-11-24T07:57:24.597442+0000","last_active":"2023-11-24T07:57:26.584012+0000","last_peered":"2023-11-24T07:57:26.584012+0000","last_clean":"2023-11-24T07:57:26.584012+0000","last_became_active":"2023-11-24T07:57:24.596992+0000","last_became_peered":"2023-11-24T07:57:24.596992+0000","last_unstale":"2023-11-24T07:57:26.584012+0000","last_undegraded":"2023-11-24T07:57:26.584012+0000","last_fullsized":"2023-11-24T07:57:26.584012+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:57:23.565116+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:57:23.565116+0000","last_clean_scrub_stamp":"2023-11-24T07:57:23.565116+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T11:43:38.072322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.d","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-24T07:57:27.498699+0000","last_change":"2023-11-24T07:57:25.608066+0000","last_active":"2023-11-24T07:57:27.498699+0000","last_peered":"2023-11-24T07:57:27.498699+0000","last_clean":"2023-11-24T07:57:27.498699+0000","last_became_active":"2023-11-24T07:57:25.607583+0000","last_became_peered":"2023-11-24T07:57:25.607583+0000","last_unstale":"2023-11-24T07:57:27.498699+0000","last_undegraded":"2023-11-24T07:57:27.498699+0000","last_fullsized":"2023-11-24T07:57:27.498699+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:57:24.572681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:57:24.572681+0000","last_clean_scrub_stamp":"2023-11-24T07:57:24.572681+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T13:05:00.624062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-24T07:57:26.582860+0000","last_change":"2023-11-24T07:57:24.588501+0000","last_active":"2023-11-24T07:57:26.582860+0000","last_peered":"2023-11-24T07:57:26.582860+0000","last_clean":"2023-11-24T07:57:26.582860+0000","last_became_active":"2023-11-24T07:57:24.588135+0000","last_became_peered":"2023-11-24T07:57:24.588135+0000","last_unstale":"2023-11-24T07:57:26.582860+0000","last_undegraded":"2023-11-24T07:57:26.582860+0000","last_fullsized":"2023-11-24T07:57:26.582860+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:57:23.565116+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:57:23.565116+0000","last_clean_scrub_stamp":"2023-11-24T07:57:23.565116+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T16:47:54.583083+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-24T07:57:27.498182+0000","last_change":"2023-11-24T07:57:25.606996+0000","last_active":"2023-11-24T07:57:27.498182+0000","last_peered":"2023-11-24T07:57:27.498182+0000","last_clean":"2023-11-24T07:57:27.498182+0000","last_became_active":"2023-11-24T07:57:25.606495+0000","last_became_peered":"2023-11-24T07:57:25.606495+0000","last_unstale":"2023-11-24T07:57:27.498182+0000","last_undegraded":"2023-11-24T07:57:27.498182+0000","last_fullsized":"2023-11-24T07:57:27.498182+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:57:24.572681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:57:24.572681+0000","last_clean_scrub_stamp":"2023-11-24T07:57:24.572681+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T16:10:48.563298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.b","version":"32'2","reported_seq":14,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-24T07:57:26.584060+0000","last_change":"2023-11-24T07:57:24.589230+0000","last_active":"2023-11-24T07:57:26.584060+0000","last_peered":"2023-11-24T07:57:26.584060+0000","last_clean":"2023-11-24T07:57:26.584060+0000","last_became_active":"2023-11-24T07:57:24.588791+0000","last_became_peered":"2023-11-24T07:57:24.588791+0000","last_unstale":"2023-11-24T07:57:26.584060+0000","last_undegraded":"2023-11-24T07:57:26.584060+0000","last_fullsized":"2023-11-24T07:57:26.584060+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:57:23.565116+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:57:23.565116+0000","last_clean_scrub_stamp":"2023-11-24T07:57:23.565116+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T12:58:54.851868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-24T07:57:26.584361+0000","last_change":"2023-11-24T07:57:25.604199+0000","last_active":"2023-11-24T07:57:26.584361+0000","last_peered":"2023-11-24T07:57:26.584361+0000","last_clean":"2023-11-24T07:57:26.584361+0000","last_became_active":"2023-11-24T07:57:25.603979+0000","last_became_peered":"2023-11-24T07:57:25.603979+0000","last_unstale":"2023-11-24T07:57:26.584361+0000","last_undegraded":"2023-11-24T07:57:26.584361+0000","last_fullsized":"2023-11-24T07:57:26.584361+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:57:24.572681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:57:24.572681+0000","last_clean_scrub_stamp":"2023-11-24T07:57:24.572681+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T14:04:03.022069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-24T07:57:27.498922+0000","last_change":"2023-11-24T07:57:24.597707+0000","last_active":"2023-11-24T07:57:27.498922+0000","last_peered":"2023-11-24T07:57:27.498922+0000","last_clean":"2023-11-24T07:57:27.498922+0000","last_became_active":"2023-11-24T07:57:24.597441+0000","last_became_peered":"2023-11-24T07:57:24.597441+0000","last_unstale":"2023-11-24T07:57:27.498922+0000","last_undegraded":"2023-11-24T07:57:27.498922+0000","last_fullsized":"2023-11-24T07:57:27.498922+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:57:23.565116+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:57:23.565116+0000","last_clean_scrub_stamp":"2023-11-24T07:57:23.565116+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T18:23:24.083250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-24T07:57:27.498774+0000","last_change":"2023-11-24T07:57:25.609619+0000","last_active":"2023-11-24T07:57:27.498774+0000","last_peered":"2023-11-24T07:57:27.498774+0000","last_clean":"2023-11-24T07:57:27.498774+0000","last_became_active":"2023-11-24T07:57:25.609261+0000","last_became_peered":"2023-11-24T07:57:25.609261+0000","last_unstale":"2023-11-24T07:57:27.498774+0000","last_undegraded":"2023-11-24T07:57:27.498774+0000","last_fullsized":"2023-11-24T07:57:27.498774+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:57:24.572681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:57:24.572681+0000","last_clean_scrub_stamp":"2023-11-24T07:57:24.572681+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T11:43:03.678993+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-24T07:57:27.498617+0000","last_change":"2023-11-24T07:57:24.589176+0000","last_active":"2023-11-24T07:57:27.498617+0000","last_peered":"2023-11-24T07:57:27.498617+0000","last_clean":"2023-11-24T07:57:27.498617+0000","last_became_active":"2023-11-24T07:57:24.588886+0000","last_became_peered":"2023-11-24T07:57:24.588886+0000","last_unstale":"2023-11-24T07:57:27.498617+0000","last_undegraded":"2023-11-24T07:57:27.498617+0000","last_fullsized":"2023-11-24T07:57:27.498617+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:57:23.565116+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:57:23.565116+0000","last_clean_scrub_stamp":"2023-11-24T07:57:23.565116+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T16:28:17.765064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-24T07:57:27.498976+0000","last_change":"2023-11-24T07:57:25.607219+0000","last_active":"2023-11-24T07:57:27.498976+0000","last_peered":"2023-11-24T07:57:27.498976+0000","last_clean":"2023-11-24T07:57:27.498976+0000","last_became_active":"2023-11-24T07:57:25.606930+0000","last_became_peered":"2023-11-24T07:57:25.606930+0000","last_unstale":"2023-11-24T07:57:27.498976+0000","last_undegraded":"2023-11-24T07:57:27.498976+0000","last_fullsized":"2023-11-24T07:57:27.498976+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:57:24.572681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:57:24.572681+0000","last_clean_scrub_stamp":"2023-11-24T07:57:24.572681+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T13:05:00.624062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-24T07:57:26.583579+0000","last_change":"2023-11-24T07:57:24.597642+0000","last_active":"2023-11-24T07:57:26.583579+0000","last_peered":"2023-11-24T07:57:26.583579+0000","last_clean":"2023-11-24T07:57:26.583579+0000","last_became_active":"2023-11-24T07:57:24.597339+0000","last_became_peered":"2023-11-24T07:57:24.597339+0000","last_unstale":"2023-11-24T07:57:26.583579+0000","last_undegraded":"2023-11-24T07:57:26.583579+0000","last_fullsized":"2023-11-24T07:57:26.583579+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:57:23.565116+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:57:23.565116+0000","last_clean_scrub_stamp":"2023-11-24T07:57:23.565116+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T15:35:45.139676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-24T07:57:27.498618+0000","last_change":"2023-11-24T07:57:25.609017+0000","last_active":"2023-11-24T07:57:27.498618+0000","last_peered":"2023-11-24T07:57:27.498618+0000","last_clean":"2023-11-24T07:57:27.498618+0000","last_became_active":"2023-11-24T07:57:25.608616+0000","last_became_peered":"2023-11-24T07:57:25.608616+0000","last_unstale":"2023-11-24T07:57:27.498618+0000","last_undegraded":"2023-11-24T07:57:27.498618+0000","last_fullsized":"2023-11-24T07:57:27.498618+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:57:24.572681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:57:24.572681+0000","last_clean_scrub_stamp":"2023-11-24T07:57:24.572681+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T15:54:26.161043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.7","version":"32'2","reported_seq":14,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-24T07:57:27.498697+0000","last_change":"2023-11-24T07:57:24.596645+0000","last_active":"2023-11-24T07:57:27.498697+0000","last_peered":"2023-11-24T07:57:27.498697+0000","last_clean":"2023-11-24T07:57:27.498697+0000","last_became_active":"2023-11-24T07:57:24.596327+0000","last_became_peered":"2023-11-24T07:57:24.596327+0000","last_unstale":"2023-11-24T07:57:27.498697+0000","last_undegraded":"2023-11-24T07:57:27.498697+0000","last_fullsized":"2023-11-24T07:57:27.498697+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:57:23.565116+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:57:23.565116+0000","last_clean_scrub_stamp":"2023-11-24T07:57:23.565116+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T14:49:19.202644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-24T07:57:26.583923+0000","last_change":"2023-11-24T07:57:25.612875+0000","last_active":"2023-11-24T07:57:26.583923+0000","last_peered":"2023-11-24T07:57:26.583923+0000","last_clean":"2023-11-24T07:57:26.583923+0000","last_became_active":"2023-11-24T07:57:25.612599+0000","last_became_peered":"2023-11-24T07:57:25.612599+0000","last_unstale":"2023-11-24T07:57:26.583923+0000","last_undegraded":"2023-11-24T07:57:26.583923+0000","last_fullsized":"2023-11-24T07:57:26.583923+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:57:24.572681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:57:24.572681+0000","last_clean_scrub_stamp":"2023-11-24T07:57:24.572681+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T13:42:15.906045+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.6","version":"33'1","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-24T07:57:27.619117+0000","last_change":"2023-11-24T07:57:24.600554+0000","last_active":"2023-11-24T07:57:27.619117+0000","last_peered":"2023-11-24T07:57:27.619117+0000","last_clean":"2023-11-24T07:57:27.619117+0000","last_became_active":"2023-11-24T07:57:24.600257+0000","last_became_peered":"2023-11-24T07:57:24.600257+0000","last_unstale":"2023-11-24T07:57:27.619117+0000","last_undegraded":"2023-11-24T07:57:27.619117+0000","last_fullsized":"2023-11-24T07:57:27.619117+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:57:23.565116+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:57:23.565116+0000","last_clean_scrub_stamp":"2023-11-24T07:57:23.565116+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T16:47:54.583083+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-24T07:57:27.498903+0000","last_change":"2023-11-24T07:57:25.604810+0000","last_active":"2023-11-24T07:57:27.498903+0000","last_peered":"2023-11-24T07:57:27.498903+0000","last_clean":"2023-11-24T07:57:27.498903+0000","last_became_active":"2023-11-24T07:57:25.604557+0000","last_became_peered":"2023-11-24T07:57:25.604557+0000","last_unstale":"2023-11-24T07:57:27.498903+0000","last_undegraded":"2023-11-24T07:57:27.498903+0000","last_fullsized":"2023-11-24T07:57:27.498903+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:57:24.572681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:57:24.572681+0000","last_clean_scrub_stamp":"2023-11-24T07:57:24.572681+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T12:14:04.002318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-24T07:57:27.499172+0000","last_change":"2023-11-24T07:57:24.597887+0000","last_active":"2023-11-24T07:57:27.499172+0000","last_peered":"2023-11-24T07:57:27.499172+0000","last_clean":"2023-11-24T07:57:27.499172+0000","last_became_active":"2023-11-24T07:57:24.597549+0000","last_became_peered":"2023-11-24T07:57:24.597549+0000","last_unstale":"2023-11-24T07:57:27.499172+0000","last_undegraded":"2023-11-24T07:57:27.499172+0000","last_fullsized":"2023-11-24T07:57:27.499172+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:57:23.565116+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:57:23.565116+0000","last_clean_scrub_stamp":"2023-11-24T07:57:23.565116+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:34:29.085812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-24T07:57:26.582793+0000","last_change":"2023-11-24T07:57:25.606098+0000","last_active":"2023-11-24T07:57:26.582793+0000","last_peered":"2023-11-24T07:57:26.582793+0000","last_clean":"2023-11-24T07:57:26.582793+0000","last_became_active":"2023-11-24T07:57:25.603917+0000","last_became_peered":"2023-11-24T07:57:25.603917+0000","last_unstale":"2023-11-24T07:57:26.582793+0000","last_undegraded":"2023-11-24T07:57:26.582793+0000","last_fullsized":"2023-11-24T07:57:26.582793+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:57:24.572681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:57:24.572681+0000","last_clean_scrub_stamp":"2023-11-24T07:57:24.572681+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T19:13:40.799042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-24T07:57:27.498955+0000","last_change":"2023-11-24T07:57:24.596194+0000","last_active":"2023-11-24T07:57:27.498955+0000","last_peered":"2023-11-24T07:57:27.498955+0000","last_clean":"2023-11-24T07:57:27.498955+0000","last_became_active":"2023-11-24T07:57:24.595854+0000","last_became_peered":"2023-11-24T07:57:24.595854+0000","last_unstale":"2023-11-24T07:57:27.498955+0000","last_undegraded":"2023-11-24T07:57:27.498955+0000","last_fullsized":"2023-11-24T07:57:27.498955+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:57:23.565116+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:57:23.565116+0000","last_clean_scrub_stamp":"2023-11-24T07:57:23.565116+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T12:14:02.994752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-24T07:57:27.499699+0000","last_change":"2023-11-24T07:57:25.604191+0000","last_active":"2023-11-24T07:57:27.499699+0000","last_peered":"2023-11-24T07:57:27.499699+0000","last_clean":"2023-11-24T07:57:27.499699+0000","last_became_active":"2023-11-24T07:57:25.603570+0000","last_became_peered":"2023-11-24T07:57:25.603570+0000","last_unstale":"2023-11-24T07:57:27.499699+0000","last_undegraded":"2023-11-24T07:57:27.499699+0000","last_fullsized":"2023-11-24T07:57:27.499699+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:57:24.572681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:57:24.572681+0000","last_clean_scrub_stamp":"2023-11-24T07:57:24.572681+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T11:48:20.813946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-24T07:57:27.498769+0000","last_change":"2023-11-24T07:57:24.599222+0000","last_active":"2023-11-24T07:57:27.498769+0000","last_peered":"2023-11-24T07:57:27.498769+0000","last_clean":"2023-11-24T07:57:27.498769+0000","last_became_active":"2023-11-24T07:57:24.598910+0000","last_became_peered":"2023-11-24T07:57:24.598910+0000","last_unstale":"2023-11-24T07:57:27.498769+0000","last_undegraded":"2023-11-24T07:57:27.498769+0000","last_fullsized":"2023-11-24T07:57:27.498769+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:57:23.565116+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:57:23.565116+0000","last_clean_scrub_stamp":"2023-11-24T07:57:23.565116+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T15:03:20.689993+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-24T07:57:27.498551+0000","last_change":"2023-11-24T07:57:25.603783+0000","last_active":"2023-11-24T07:57:27.498551+0000","last_peered":"2023-11-24T07:57:27.498551+0000","last_clean":"2023-11-24T07:57:27.498551+0000","last_became_active":"2023-11-24T07:57:25.603464+0000","last_became_peered":"2023-11-24T07:57:25.603464+0000","last_unstale":"2023-11-24T07:57:27.498551+0000","last_undegraded":"2023-11-24T07:57:27.498551+0000","last_fullsized":"2023-11-24T07:57:27.498551+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:57:24.572681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:57:24.572681+0000","last_clean_scrub_stamp":"2023-11-24T07:57:24.572681+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T14:04:03.022069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-24T07:57:27.499235+0000","last_change":"2023-11-24T07:57:24.599854+0000","last_active":"2023-11-24T07:57:27.499235+0000","last_peered":"2023-11-24T07:57:27.499235+0000","last_clean":"2023-11-24T07:57:27.499235+0000","last_became_active":"2023-11-24T07:57:24.599310+0000","last_became_peered":"2023-11-24T07:57:24.599310+0000","last_unstale":"2023-11-24T07:57:27.499235+0000","last_undegraded":"2023-11-24T07:57:27.499235+0000","last_fullsized":"2023-11-24T07:57:27.499235+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:57:23.565116+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:57:23.565116+0000","last_clean_scrub_stamp":"2023-11-24T07:57:23.565116+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:34:29.085812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-24T07:57:26.583711+0000","last_change":"2023-11-24T07:57:25.613275+0000","last_active":"2023-11-24T07:57:26.583711+0000","last_peered":"2023-11-24T07:57:26.583711+0000","last_clean":"2023-11-24T07:57:26.583711+0000","last_became_active":"2023-11-24T07:57:25.612934+0000","last_became_peered":"2023-11-24T07:57:25.612934+0000","last_unstale":"2023-11-24T07:57:26.583711+0000","last_undegraded":"2023-11-24T07:57:26.583711+0000","last_fullsized":"2023-11-24T07:57:26.583711+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:57:24.572681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:57:24.572681+0000","last_clean_scrub_stamp":"2023-11-24T07:57:24.572681+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T19:14:43.303209+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-24T07:57:27.498639+0000","last_change":"2023-11-24T07:57:24.597190+0000","last_active":"2023-11-24T07:57:27.498639+0000","last_peered":"2023-11-24T07:57:27.498639+0000","last_clean":"2023-11-24T07:57:27.498639+0000","last_became_active":"2023-11-24T07:57:24.596910+0000","last_became_peered":"2023-11-24T07:57:24.596910+0000","last_unstale":"2023-11-24T07:57:27.498639+0000","last_undegraded":"2023-11-24T07:57:27.498639+0000","last_fullsized":"2023-11-24T07:57:27.498639+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:57:23.565116+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:57:23.565116+0000","last_clean_scrub_stamp":"2023-11-24T07:57:23.565116+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T12:58:54.851868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-24T07:57:26.582741+0000","last_change":"2023-11-24T07:57:25.592562+0000","last_active":"2023-11-24T07:57:26.582741+0000","last_peered":"2023-11-24T07:57:26.582741+0000","last_clean":"2023-11-24T07:57:26.582741+0000","last_became_active":"2023-11-24T07:57:25.592287+0000","last_became_peered":"2023-11-24T07:57:25.592287+0000","last_unstale":"2023-11-24T07:57:26.582741+0000","last_undegraded":"2023-11-24T07:57:26.582741+0000","last_fullsized":"2023-11-24T07:57:26.582741+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:57:24.572681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:57:24.572681+0000","last_clean_scrub_stamp":"2023-11-24T07:57:24.572681+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T09:32:54.344241+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-24T07:57:26.583911+0000","last_change":"2023-11-24T07:57:24.599399+0000","last_active":"2023-11-24T07:57:26.583911+0000","last_peered":"2023-11-24T07:57:26.583911+0000","last_clean":"2023-11-24T07:57:26.583911+0000","last_became_active":"2023-11-24T07:57:24.599096+0000","last_became_peered":"2023-11-24T07:57:24.599096+0000","last_unstale":"2023-11-24T07:57:26.583911+0000","last_undegraded":"2023-11-24T07:57:26.583911+0000","last_fullsized":"2023-11-24T07:57:26.583911+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:57:23.565116+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:57:23.565116+0000","last_clean_scrub_stamp":"2023-11-24T07:57:23.565116+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T18:02:41.581821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":176,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-11-24T07:57:25.591791+0000","last_change":"2023-11-24T07:56:19.668905+0000","last_active":"2023-11-24T07:57:25.591791+0000","last_peered":"2023-11-24T07:57:25.591791+0000","last_clean":"2023-11-24T07:57:25.591791+0000","last_became_active":"2023-11-24T07:56:19.668601+0000","last_became_peered":"2023-11-24T07:56:19.668601+0000","last_unstale":"2023-11-24T07:57:25.591791+0000","last_undegraded":"2023-11-24T07:57:25.591791+0000","last_fullsized":"2023-11-24T07:57:25.591791+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:56:18.657844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:56:18.657844+0000","last_clean_scrub_stamp":"2023-11-24T07:56:18.657844+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T11:51:53.828455+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":5,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574865,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27396,"kb_used_data":368,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344444,"statfs":{"total":96636764160,"available":96608710656,"internally_reserved":0,"allocated":376832,"data_stored":146120,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 07:57:19 2023","interfaces":[{"interface":"back","average":{"1min":0.55700000000000005,"5min":0.55700000000000005,"15min":0.55700000000000005},"min":{"1min":0.32800000000000001,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":1.413,"5min":1.413,"15min":1.413},"last":0.78500000000000003},{"interface":"front","average":{"1min":0.55400000000000005,"5min":0.55400000000000005,"15min":0.55400000000000005},"min":{"1min":0.41099999999999998,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":0.81000000000000005,"5min":0.81000000000000005,"15min":0.81000000000000005},"last":0.77000000000000002}]},{"osd":1,"last update":"Fri Nov 24 07:57:19 2023","interfaces":[{"interface":"back","average":{"1min":0.84899999999999998,"5min":0.84899999999999998,"15min":0.84899999999999998},"min":{"1min":0.33600000000000002,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":6.1070000000000002,"5min":6.1070000000000002,"15min":6.1070000000000002},"last":0.56999999999999995},{"interface":"front","average":{"1min":0.57999999999999996,"5min":0.57999999999999996,"15min":0.57999999999999996},"min":{"1min":0.35199999999999998,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":1.5920000000000001,"5min":1.5920000000000001,"15min":1.5920000000000001},"last":0.73599999999999999}]},{"osd":2,"last update":"Fri Nov 24 07:57:19 2023","interfaces":[{"interface":"back","average":{"1min":0.627,"5min":0.627,"15min":0.627},"min":{"1min":0.33800000000000002,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":1.613,"5min":1.613,"15min":1.613},"last":0.59299999999999997},{"interface":"front","average":{"1min":0.79200000000000004,"5min":0.79200000000000004,"15min":0.79200000000000004},"min":{"1min":0.41699999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":4.5220000000000002,"5min":4.5220000000000002,"15min":4.5220000000000002},"last":0.71299999999999997}]},{"osd":3,"last update":"Fri Nov 24 07:57:19 2023","interfaces":[{"interface":"back","average":{"1min":0.80600000000000005,"5min":0.80600000000000005,"15min":0.80600000000000005},"min":{"1min":0.38500000000000001,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":4.6760000000000002,"5min":4.6760000000000002,"15min":4.6760000000000002},"last":0.754},{"interface":"front","average":{"1min":0.94299999999999995,"5min":0.94299999999999995,"15min":0.94299999999999995},"min":{"1min":0.505,"5min":0.505,"15min":0.505},"max":{"1min":4.6539999999999999,"5min":4.6539999999999999,"15min":4.6539999999999999},"last":0.81599999999999995}]},{"osd":4,"last update":"Fri Nov 24 07:57:19 2023","interfaces":[{"interface":"back","average":{"1min":0.56799999999999995,"5min":0.56799999999999995,"15min":0.56799999999999995},"min":{"1min":0.378,"5min":0.378,"15min":0.378},"max":{"1min":0.84699999999999998,"5min":0.84699999999999998,"15min":0.84699999999999998},"last":0.44900000000000001},{"interface":"front","average":{"1min":0.66700000000000004,"5min":0.66700000000000004,"15min":0.66700000000000004},"min":{"1min":0.46100000000000002,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":1.629,"5min":1.629,"15min":1.629},"last":0.48699999999999999}]},{"osd":5,"last update":"Fri Nov 24 07:57:19 2023","interfaces":[{"interface":"back","average":{"1min":0.54700000000000004,"5min":0.54700000000000004,"15min":0.54700000000000004},"min":{"1min":0.34499999999999997,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.875,"5min":0.875,"15min":0.875},"last":0.46899999999999997},{"interface":"front","average":{"1min":0.623,"5min":0.623,"15min":0.623},"min":{"1min":0.41499999999999998,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":0.90900000000000003,"5min":0.90900000000000003,"15min":0.90900000000000003},"last":0.52500000000000002}]},{"osd":6,"last update":"Fri Nov 24 07:57:19 2023","interfaces":[{"interface":"back","average":{"1min":0.61099999999999999,"5min":0.61099999999999999,"15min":0.61099999999999999},"min":{"1min":0.438,"5min":0.438,"15min":0.438},"max":{"1min":0.80800000000000005,"5min":0.80800000000000005,"15min":0.80800000000000005},"last":0.50800000000000001},{"interface":"front","average":{"1min":0.67000000000000004,"5min":0.67000000000000004,"15min":0.67000000000000004},"min":{"1min":0.48599999999999999,"5min":0.48599999999999999,"15min":0.48599999999999999},"max":{"1min":0.98399999999999999,"5min":0.98399999999999999,"15min":0.98399999999999999},"last":0.67800000000000005}]}]},{"osd":6,"up_from":13,"seq":55834574866,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27396,"kb_used_data":376,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344444,"statfs":{"total":96636764160,"available":96608710656,"internally_reserved":0,"allocated":385024,"data_stored":147710,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 07:57:18 2023","interfaces":[{"interface":"back","average":{"1min":0.55000000000000004,"5min":0.55000000000000004,"15min":0.55000000000000004},"min":{"1min":0.28399999999999997,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":0.85099999999999998,"5min":0.85099999999999998,"15min":0.85099999999999998},"last":0.29799999999999999},{"interface":"front","average":{"1min":0.51600000000000001,"5min":0.51600000000000001,"15min":0.51600000000000001},"min":{"1min":0.311,"5min":0.311,"15min":0.311},"max":{"1min":0.69999999999999996,"5min":0.69999999999999996,"15min":0.69999999999999996},"last":0.32600000000000001}]},{"osd":1,"last update":"Fri Nov 24 07:57:18 2023","interfaces":[{"interface":"back","average":{"1min":0.51200000000000001,"5min":0.51200000000000001,"15min":0.51200000000000001},"min":{"1min":0.33800000000000002,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.78200000000000003,"5min":0.78200000000000003,"15min":0.78200000000000003},"last":0.35599999999999998},{"interface":"front","average":{"1min":0.61299999999999999,"5min":0.61299999999999999,"15min":0.61299999999999999},"min":{"1min":0.44700000000000001,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":1.0069999999999999,"5min":1.0069999999999999,"15min":1.0069999999999999},"last":0.48199999999999998}]},{"osd":2,"last update":"Fri Nov 24 07:57:18 2023","interfaces":[{"interface":"back","average":{"1min":0.57499999999999996,"5min":0.57499999999999996,"15min":0.57499999999999996},"min":{"1min":0.34300000000000003,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.878,"5min":0.878,"15min":0.878},"last":0.46500000000000002},{"interface":"front","average":{"1min":0.59299999999999997,"5min":0.59299999999999997,"15min":0.59299999999999997},"min":{"1min":0.47099999999999997,"5min":0.47099999999999997,"15min":0.47099999999999997},"max":{"1min":0.82399999999999995,"5min":0.82399999999999995,"15min":0.82399999999999995},"last":0.439}]},{"osd":3,"last update":"Fri Nov 24 07:57:18 2023","interfaces":[{"interface":"back","average":{"1min":0.56799999999999995,"5min":0.56799999999999995,"15min":0.56799999999999995},"min":{"1min":0.40000000000000002,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":0.876,"5min":0.876,"15min":0.876},"last":0.41899999999999998},{"interface":"front","average":{"1min":0.55600000000000005,"5min":0.55600000000000005,"15min":0.55600000000000005},"min":{"1min":0.379,"5min":0.379,"15min":0.379},"max":{"1min":0.68799999999999994,"5min":0.68799999999999994,"15min":0.68799999999999994},"last":0.52000000000000002}]},{"osd":4,"last update":"Fri Nov 24 07:57:18 2023","interfaces":[{"interface":"back","average":{"1min":0.51700000000000002,"5min":0.51700000000000002,"15min":0.51700000000000002},"min":{"1min":0.33000000000000002,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.70599999999999996,"5min":0.70599999999999996,"15min":0.70599999999999996},"last":0.505},{"interface":"front","average":{"1min":0.60199999999999998,"5min":0.60199999999999998,"15min":0.60199999999999998},"min":{"1min":0.35199999999999998,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":1.087,"5min":1.087,"15min":1.087},"last":0.53300000000000003}]},{"osd":5,"last update":"Fri Nov 24 07:57:18 2023","interfaces":[{"interface":"back","average":{"1min":0.53100000000000003,"5min":0.53100000000000003,"15min":0.53100000000000003},"min":{"1min":0.30199999999999999,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.84999999999999998,"5min":0.84999999999999998,"15min":0.84999999999999998},"last":0.54800000000000004},{"interface":"front","average":{"1min":0.61199999999999999,"5min":0.61199999999999999,"15min":0.61199999999999999},"min":{"1min":0.39200000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.97199999999999998,"5min":0.97199999999999998,"15min":0.97199999999999998},"last":0.55800000000000005}]},{"osd":7,"last update":"Fri Nov 24 07:57:18 2023","interfaces":[{"interface":"back","average":{"1min":0.61799999999999999,"5min":0.61799999999999999,"15min":0.61799999999999999},"min":{"1min":0.49299999999999999,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":0.81499999999999995,"5min":0.81499999999999995,"15min":0.81499999999999995},"last":0.57599999999999996},{"interface":"front","average":{"1min":0.67000000000000004,"5min":0.67000000000000004,"15min":0.67000000000000004},"min":{"1min":0.497,"5min":0.497,"15min":0.497},"max":{"1min":0.92700000000000005,"5min":0.92700000000000005,"15min":0.92700000000000005},"last":0.621}]}]},{"osd":1,"up_from":13,"seq":55834574867,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27968,"kb_used_data":948,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343872,"statfs":{"total":96636764160,"available":96608124928,"internally_reserved":0,"allocated":970752,"data_stored":736536,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 07:57:22 2023","interfaces":[{"interface":"back","average":{"1min":0.36399999999999999,"5min":0.36399999999999999,"15min":0.36399999999999999},"min":{"1min":0.20200000000000001,"5min":0.20200000000000001,"15min":0.20200000000000001},"max":{"1min":0.70899999999999996,"5min":0.70899999999999996,"15min":0.70899999999999996},"last":0.40799999999999997},{"interface":"front","average":{"1min":0.39800000000000002,"5min":0.39800000000000002,"15min":0.39800000000000002},"min":{"1min":0.214,"5min":0.214,"15min":0.214},"max":{"1min":0.82499999999999996,"5min":0.82499999999999996,"15min":0.82499999999999996},"last":0.39300000000000002}]},{"osd":2,"last update":"Fri Nov 24 07:57:22 2023","interfaces":[{"interface":"back","average":{"1min":0.373,"5min":0.373,"15min":0.373},"min":{"1min":0.22700000000000001,"5min":0.22700000000000001,"15min":0.22700000000000001},"max":{"1min":0.60999999999999999,"5min":0.60999999999999999,"15min":0.60999999999999999},"last":0.375},{"interface":"front","average":{"1min":0.41299999999999998,"5min":0.41299999999999998,"15min":0.41299999999999998},"min":{"1min":0.26800000000000002,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":0.621,"5min":0.621,"15min":0.621},"last":0.33000000000000002}]},{"osd":3,"last update":"Fri Nov 24 07:57:22 2023","interfaces":[{"interface":"back","average":{"1min":0.44400000000000001,"5min":0.44400000000000001,"15min":0.44400000000000001},"min":{"1min":0.32100000000000001,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.66500000000000004,"5min":0.66500000000000004,"15min":0.66500000000000004},"last":0.438},{"interface":"front","average":{"1min":0.42799999999999999,"5min":0.42799999999999999,"15min":0.42799999999999999},"min":{"1min":0.28499999999999998,"5min":0.28499999999999998,"15min":0.28499999999999998},"max":{"1min":0.76300000000000001,"5min":0.76300000000000001,"15min":0.76300000000000001},"last":0.42799999999999999}]},{"osd":4,"last update":"Fri Nov 24 07:57:22 2023","interfaces":[{"interface":"back","average":{"1min":0.61099999999999999,"5min":0.61099999999999999,"15min":0.61099999999999999},"min":{"1min":0.43099999999999999,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":0.79800000000000004,"5min":0.79800000000000004,"15min":0.79800000000000004},"last":0.63400000000000001},{"interface":"front","average":{"1min":0.60599999999999998,"5min":0.60599999999999998,"15min":0.60599999999999998},"min":{"1min":0.41999999999999998,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":0.82199999999999995,"5min":0.82199999999999995,"15min":0.82199999999999995},"last":0.70899999999999996}]},{"osd":5,"last update":"Fri Nov 24 07:57:22 2023","interfaces":[{"interface":"back","average":{"1min":0.72499999999999998,"5min":0.72499999999999998,"15min":0.72499999999999998},"min":{"1min":0.37,"5min":0.37,"15min":0.37},"max":{"1min":2.7269999999999999,"5min":2.7269999999999999,"15min":2.7269999999999999},"last":0.48599999999999999},{"interface":"front","average":{"1min":0.64500000000000002,"5min":0.64500000000000002,"15min":0.64500000000000002},"min":{"1min":0.44700000000000001,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":0.84199999999999997,"5min":0.84199999999999997,"15min":0.84199999999999997},"last":0.59399999999999997}]},{"osd":6,"last update":"Fri Nov 24 07:57:22 2023","interfaces":[{"interface":"back","average":{"1min":0.65800000000000003,"5min":0.65800000000000003,"15min":0.65800000000000003},"min":{"1min":0.47499999999999998,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":0.98199999999999998,"5min":0.98199999999999998,"15min":0.98199999999999998},"last":0.67700000000000005},{"interface":"front","average":{"1min":0.61699999999999999,"5min":0.61699999999999999,"15min":0.61699999999999999},"min":{"1min":0.42899999999999999,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":0.90700000000000003,"5min":0.90700000000000003,"15min":0.90700000000000003},"last":0.51900000000000002}]},{"osd":7,"last update":"Fri Nov 24 07:57:22 2023","interfaces":[{"interface":"back","average":{"1min":0.63700000000000001,"5min":0.63700000000000001,"15min":0.63700000000000001},"min":{"1min":0.40999999999999998,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":1.0649999999999999,"5min":1.0649999999999999,"15min":1.0649999999999999},"last":0.66100000000000003},{"interface":"front","average":{"1min":0.67000000000000004,"5min":0.67000000000000004,"15min":0.67000000000000004},"min":{"1min":0.442,"5min":0.442,"15min":0.442},"max":{"1min":0.88800000000000001,"5min":0.88800000000000001,"15min":0.88800000000000001},"last":0.54300000000000004}]}]},{"osd":0,"up_from":13,"seq":55834574867,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27396,"kb_used_data":368,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344444,"statfs":{"total":96636764160,"available":96608710656,"internally_reserved":0,"allocated":376832,"data_stored":146188,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Nov 24 07:57:18 2023","interfaces":[{"interface":"back","average":{"1min":0.71299999999999997,"5min":0.71299999999999997,"15min":0.71299999999999997},"min":{"1min":0.23499999999999999,"5min":0.23499999999999999,"15min":0.23499999999999999},"max":{"1min":6.3550000000000004,"5min":6.3550000000000004,"15min":6.3550000000000004},"last":0.625},{"interface":"front","average":{"1min":1.7290000000000001,"5min":1.7290000000000001,"15min":1.7290000000000001},"min":{"1min":0.30599999999999999,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":22.916,"5min":22.916,"15min":22.916},"last":0.58699999999999997}]},{"osd":2,"last update":"Fri Nov 24 07:57:18 2023","interfaces":[{"interface":"back","average":{"1min":0.69999999999999996,"5min":0.69999999999999996,"15min":0.69999999999999996},"min":{"1min":0.22800000000000001,"5min":0.22800000000000001,"15min":0.22800000000000001},"max":{"1min":6.3899999999999997,"5min":6.3899999999999997,"15min":6.3899999999999997},"last":0.435},{"interface":"front","average":{"1min":1.0629999999999999,"5min":1.0629999999999999,"15min":1.0629999999999999},"min":{"1min":0.26800000000000002,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":11.93,"5min":11.93,"15min":11.93},"last":0.55300000000000005}]},{"osd":3,"last update":"Fri Nov 24 07:57:18 2023","interfaces":[{"interface":"back","average":{"1min":0.41999999999999998,"5min":0.41999999999999998,"15min":0.41999999999999998},"min":{"1min":0.28399999999999997,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":1.155,"5min":1.155,"15min":1.155},"last":0.79300000000000004},{"interface":"front","average":{"1min":1.127,"5min":1.127,"15min":1.127},"min":{"1min":0.32300000000000001,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":12.116,"5min":12.116,"15min":12.116},"last":0.69699999999999995}]},{"osd":4,"last update":"Fri Nov 24 07:57:18 2023","interfaces":[{"interface":"back","average":{"1min":0.64800000000000002,"5min":0.64800000000000002,"15min":0.64800000000000002},"min":{"1min":0.40500000000000003,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":0.97899999999999998,"5min":0.97899999999999998,"15min":0.97899999999999998},"last":1.044},{"interface":"front","average":{"1min":1.9339999999999999,"5min":1.9339999999999999,"15min":1.9339999999999999},"min":{"1min":0.44400000000000001,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":22.963999999999999,"5min":22.963999999999999,"15min":22.963999999999999},"last":0.90800000000000003}]},{"osd":5,"last update":"Fri Nov 24 07:57:18 2023","interfaces":[{"interface":"back","average":{"1min":1.2929999999999999,"5min":1.2929999999999999,"15min":1.2929999999999999},"min":{"1min":0.40200000000000002,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":12.084,"5min":12.084,"15min":12.084},"last":0.97799999999999998},{"interface":"front","average":{"1min":0.67800000000000005,"5min":0.67800000000000005,"15min":0.67800000000000005},"min":{"1min":0.45200000000000001,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":1.073,"5min":1.073,"15min":1.073},"last":0.95499999999999996}]},{"osd":6,"last update":"Fri Nov 24 07:57:18 2023","interfaces":[{"interface":"back","average":{"1min":0.69099999999999995,"5min":0.69099999999999995,"15min":0.69099999999999995},"min":{"1min":0.42799999999999999,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":1.0580000000000001,"5min":1.0580000000000001,"15min":1.0580000000000001},"last":0.82499999999999996},{"interface":"front","average":{"1min":1.3140000000000001,"5min":1.3140000000000001,"15min":1.3140000000000001},"min":{"1min":0.41899999999999998,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":12.045,"5min":12.045,"15min":12.045},"last":0.88700000000000001}]},{"osd":7,"last update":"Fri Nov 24 07:57:18 2023","interfaces":[{"interface":"back","average":{"1min":0.65600000000000003,"5min":0.65600000000000003,"15min":0.65600000000000003},"min":{"1min":0.45400000000000001,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":1.038,"5min":1.038,"15min":1.038},"last":0.747},{"interface":"front","average":{"1min":1.9630000000000001,"5min":1.9630000000000001,"15min":1.9630000000000001},"min":{"1min":0.49099999999999999,"5min":0.49099999999999999,"15min":0.49099999999999999},"max":{"1min":22.995000000000001,"5min":22.995000000000001,"15min":22.995000000000001},"last":1.0069999999999999}]}]},{"osd":2,"up_from":13,"seq":55834574867,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27396,"kb_used_data":368,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344444,"statfs":{"total":96636764160,"available":96608710656,"internally_reserved":0,"allocated":376832,"data_stored":146132,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 07:57:17 2023","interfaces":[{"interface":"back","average":{"1min":0.371,"5min":0.371,"15min":0.371},"min":{"1min":0.22500000000000001,"5min":0.22500000000000001,"15min":0.22500000000000001},"max":{"1min":0.58099999999999996,"5min":0.58099999999999996,"15min":0.58099999999999996},"last":0.38400000000000001},{"interface":"front","average":{"1min":0.40200000000000002,"5min":0.40200000000000002,"15min":0.40200000000000002},"min":{"1min":0.27400000000000002,"5min":0.27400000000000002,"15min":0.27400000000000002},"max":{"1min":0.54100000000000004,"5min":0.54100000000000004,"15min":0.54100000000000004},"last":0.33500000000000002}]},{"osd":1,"last update":"Fri Nov 24 07:57:17 2023","interfaces":[{"interface":"back","average":{"1min":0.378,"5min":0.378,"15min":0.378},"min":{"1min":0.23499999999999999,"5min":0.23499999999999999,"15min":0.23499999999999999},"max":{"1min":0.59199999999999997,"5min":0.59199999999999997,"15min":0.59199999999999997},"last":0.31},{"interface":"front","average":{"1min":0.41599999999999998,"5min":0.41599999999999998,"15min":0.41599999999999998},"min":{"1min":0.25600000000000001,"5min":0.25600000000000001,"15min":0.25600000000000001},"max":{"1min":0.627,"5min":0.627,"15min":0.627},"last":0.28299999999999997}]},{"osd":3,"last update":"Fri Nov 24 07:57:17 2023","interfaces":[{"interface":"back","average":{"1min":0.41099999999999998,"5min":0.41099999999999998,"15min":0.41099999999999998},"min":{"1min":0.308,"5min":0.308,"15min":0.308},"max":{"1min":0.53300000000000003,"5min":0.53300000000000003,"15min":0.53300000000000003},"last":0.36299999999999999},{"interface":"front","average":{"1min":0.41399999999999998,"5min":0.41399999999999998,"15min":0.41399999999999998},"min":{"1min":0.27500000000000002,"5min":0.27500000000000002,"15min":0.27500000000000002},"max":{"1min":0.64600000000000002,"5min":0.64600000000000002,"15min":0.64600000000000002},"last":0.34999999999999998}]},{"osd":4,"last update":"Fri Nov 24 07:57:17 2023","interfaces":[{"interface":"back","average":{"1min":0.60799999999999998,"5min":0.60799999999999998,"15min":0.60799999999999998},"min":{"1min":0.42999999999999999,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":0.81599999999999995,"5min":0.81599999999999995,"15min":0.81599999999999995},"last":0.46100000000000002},{"interface":"front","average":{"1min":0.59999999999999998,"5min":0.59999999999999998,"15min":0.59999999999999998},"min":{"1min":0.41599999999999998,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":0.79600000000000004,"5min":0.79600000000000004,"15min":0.79600000000000004},"last":0.42699999999999999}]},{"osd":5,"last update":"Fri Nov 24 07:57:17 2023","interfaces":[{"interface":"back","average":{"1min":0.621,"5min":0.621,"15min":0.621},"min":{"1min":0.44900000000000001,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":0.84899999999999998,"5min":0.84899999999999998,"15min":0.84899999999999998},"last":0.46800000000000003},{"interface":"front","average":{"1min":0.71499999999999997,"5min":0.71499999999999997,"15min":0.71499999999999997},"min":{"1min":0.433,"5min":0.433,"15min":0.433},"max":{"1min":2.1699999999999999,"5min":2.1699999999999999,"15min":2.1699999999999999},"last":0.53500000000000003}]},{"osd":6,"last update":"Fri Nov 24 07:57:17 2023","interfaces":[{"interface":"back","average":{"1min":0.66700000000000004,"5min":0.66700000000000004,"15min":0.66700000000000004},"min":{"1min":0.47399999999999998,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":0.83599999999999997,"5min":0.83599999999999997,"15min":0.83599999999999997},"last":0.48399999999999999},{"interface":"front","average":{"1min":0.67100000000000004,"5min":0.67100000000000004,"15min":0.67100000000000004},"min":{"1min":0.496,"5min":0.496,"15min":0.496},"max":{"1min":0.88500000000000001,"5min":0.88500000000000001,"15min":0.88500000000000001},"last":0.41899999999999998}]},{"osd":7,"last update":"Fri Nov 24 07:57:17 2023","interfaces":[{"interface":"back","average":{"1min":0.66200000000000003,"5min":0.66200000000000003,"15min":0.66200000000000003},"min":{"1min":0.443,"5min":0.443,"15min":0.443},"max":{"1min":0.85999999999999999,"5min":0.85999999999999999,"15min":0.85999999999999999},"last":0.51700000000000002},{"interface":"front","average":{"1min":0.68000000000000005,"5min":0.68000000000000005,"15min":0.68000000000000005},"min":{"1min":0.495,"5min":0.495,"15min":0.495},"max":{"1min":0.86399999999999999,"5min":0.86399999999999999,"15min":0.86399999999999999},"last":0.44500000000000001}]}]},{"osd":3,"up_from":13,"seq":55834574867,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27408,"kb_used_data":380,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344432,"statfs":{"total":96636764160,"available":96608698368,"internally_reserved":0,"allocated":389120,"data_stored":146874,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 07:57:20 2023","interfaces":[{"interface":"back","average":{"1min":0.35599999999999998,"5min":0.35599999999999998,"15min":0.35599999999999998},"min":{"1min":0.222,"5min":0.222,"15min":0.222},"max":{"1min":0.84799999999999998,"5min":0.84799999999999998,"15min":0.84799999999999998},"last":0.439},{"interface":"front","average":{"1min":0.36699999999999999,"5min":0.36699999999999999,"15min":0.36699999999999999},"min":{"1min":0.23699999999999999,"5min":0.23699999999999999,"15min":0.23699999999999999},"max":{"1min":0.873,"5min":0.873,"15min":0.873},"last":0.307}]},{"osd":1,"last update":"Fri Nov 24 07:57:20 2023","interfaces":[{"interface":"back","average":{"1min":0.375,"5min":0.375,"15min":0.375},"min":{"1min":0.25700000000000001,"5min":0.25700000000000001,"15min":0.25700000000000001},"max":{"1min":0.92000000000000004,"5min":0.92000000000000004,"15min":0.92000000000000004},"last":0.36899999999999999},{"interface":"front","average":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"min":{"1min":0.248,"5min":0.248,"15min":0.248},"max":{"1min":0.81699999999999995,"5min":0.81699999999999995,"15min":0.81699999999999995},"last":0.34300000000000003}]},{"osd":2,"last update":"Fri Nov 24 07:57:20 2023","interfaces":[{"interface":"back","average":{"1min":0.36099999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"min":{"1min":0.24399999999999999,"5min":0.24399999999999999,"15min":0.24399999999999999},"max":{"1min":0.88500000000000001,"5min":0.88500000000000001,"15min":0.88500000000000001},"last":0.38900000000000001},{"interface":"front","average":{"1min":0.46800000000000003,"5min":0.46800000000000003,"15min":0.46800000000000003},"min":{"1min":0.31900000000000001,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.98099999999999998,"5min":0.98099999999999998,"15min":0.98099999999999998},"last":0.41199999999999998}]},{"osd":4,"last update":"Fri Nov 24 07:57:20 2023","interfaces":[{"interface":"back","average":{"1min":0.79700000000000004,"5min":0.79700000000000004,"15min":0.79700000000000004},"min":{"1min":0.33500000000000002,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":5.7350000000000003,"5min":5.7350000000000003,"15min":5.7350000000000003},"last":0.57499999999999996},{"interface":"front","average":{"1min":0.58499999999999996,"5min":0.58499999999999996,"15min":0.58499999999999996},"min":{"1min":0.44500000000000001,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":1.27,"5min":1.27,"15min":1.27},"last":0.65800000000000003}]},{"osd":5,"last update":"Fri Nov 24 07:57:20 2023","interfaces":[{"interface":"back","average":{"1min":0.58799999999999997,"5min":0.58799999999999997,"15min":0.58799999999999997},"min":{"1min":0.40699999999999997,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":1.4430000000000001,"5min":1.4430000000000001,"15min":1.4430000000000001},"last":0.50700000000000001},{"interface":"front","average":{"1min":0.627,"5min":0.627,"15min":0.627},"min":{"1min":0.377,"5min":0.377,"15min":0.377},"max":{"1min":1.4239999999999999,"5min":1.4239999999999999,"15min":1.4239999999999999},"last":0.55200000000000005}]},{"osd":6,"last update":"Fri Nov 24 07:57:20 2023","interfaces":[{"interface":"back","average":{"1min":0.62,"5min":0.62,"15min":0.62},"min":{"1min":0.434,"5min":0.434,"15min":0.434},"max":{"1min":1.355,"5min":1.355,"15min":1.355},"last":0.49199999999999999},{"interface":"front","average":{"1min":0.84599999999999997,"5min":0.84599999999999997,"15min":0.84599999999999997},"min":{"1min":0.40999999999999998,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":5.6479999999999997,"5min":5.6479999999999997,"15min":5.6479999999999997},"last":0.53100000000000003}]},{"osd":7,"last update":"Fri Nov 24 07:57:20 2023","interfaces":[{"interface":"back","average":{"1min":0.66600000000000004,"5min":0.66600000000000004,"15min":0.66600000000000004},"min":{"1min":0.42299999999999999,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":1.51,"5min":1.51,"15min":1.51},"last":0.48099999999999998},{"interface":"front","average":{"1min":0.84199999999999997,"5min":0.84199999999999997,"15min":0.84199999999999997},"min":{"1min":0.45600000000000002,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":5.7069999999999999,"5min":5.7069999999999999,"15min":5.7069999999999999},"last":0.54000000000000004}]}]},{"osd":4,"up_from":13,"seq":55834574866,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27400,"kb_used_data":380,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344440,"statfs":{"total":96636764160,"available":96608706560,"internally_reserved":0,"allocated":389120,"data_stored":147732,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 07:57:21 2023","interfaces":[{"interface":"back","average":{"1min":0.58099999999999996,"5min":0.58099999999999996,"15min":0.58099999999999996},"min":{"1min":0.37,"5min":0.37,"15min":0.37},"max":{"1min":0.79300000000000004,"5min":0.79300000000000004,"15min":0.79300000000000004},"last":0.34899999999999998},{"interface":"front","average":{"1min":0.54300000000000004,"5min":0.54300000000000004,"15min":0.54300000000000004},"min":{"1min":0.33600000000000002,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":1.0249999999999999,"5min":1.0249999999999999,"15min":1.0249999999999999},"last":0.45000000000000001}]},{"osd":1,"last update":"Fri Nov 24 07:57:21 2023","interfaces":[{"interface":"back","average":{"1min":0.55200000000000005,"5min":0.55200000000000005,"15min":0.55200000000000005},"min":{"1min":0.40000000000000002,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":0.86399999999999999,"5min":0.86399999999999999,"15min":0.86399999999999999},"last":0.55700000000000005},{"interface":"front","average":{"1min":0.55100000000000005,"5min":0.55100000000000005,"15min":0.55100000000000005},"min":{"1min":0.36099999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.98699999999999999,"5min":0.98699999999999999,"15min":0.98699999999999999},"last":0.46999999999999997}]},{"osd":2,"last update":"Fri Nov 24 07:57:21 2023","interfaces":[{"interface":"back","average":{"1min":0.56100000000000005,"5min":0.56100000000000005,"15min":0.56100000000000005},"min":{"1min":0.35799999999999998,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.81100000000000005,"5min":0.81100000000000005,"15min":0.81100000000000005},"last":0.42999999999999999},{"interface":"front","average":{"1min":0.61299999999999999,"5min":0.61299999999999999,"15min":0.61299999999999999},"min":{"1min":0.38100000000000001,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.89200000000000002,"5min":0.89200000000000002,"15min":0.89200000000000002},"last":0.40000000000000002}]},{"osd":3,"last update":"Fri Nov 24 07:57:21 2023","interfaces":[{"interface":"back","average":{"1min":0.58499999999999996,"5min":0.58499999999999996,"15min":0.58499999999999996},"min":{"1min":0.439,"5min":0.439,"15min":0.439},"max":{"1min":0.89700000000000002,"5min":0.89700000000000002,"15min":0.89700000000000002},"last":0.51500000000000001},{"interface":"front","average":{"1min":0.63100000000000001,"5min":0.63100000000000001,"15min":0.63100000000000001},"min":{"1min":0.40300000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.96199999999999997,"5min":0.96199999999999997,"15min":0.96199999999999997},"last":0.56699999999999995}]},{"osd":5,"last update":"Fri Nov 24 07:57:21 2023","interfaces":[{"interface":"back","average":{"1min":0.45500000000000002,"5min":0.45500000000000002,"15min":0.45500000000000002},"min":{"1min":0.29799999999999999,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":0.70499999999999996,"5min":0.70499999999999996,"15min":0.70499999999999996},"last":0.374},{"interface":"front","average":{"1min":0.61899999999999999,"5min":0.61899999999999999,"15min":0.61899999999999999},"min":{"1min":0.439,"5min":0.439,"15min":0.439},"max":{"1min":0.86399999999999999,"5min":0.86399999999999999,"15min":0.86399999999999999},"last":0.58599999999999997}]},{"osd":6,"last update":"Fri Nov 24 07:57:21 2023","interfaces":[{"interface":"back","average":{"1min":0.58599999999999997,"5min":0.58599999999999997,"15min":0.58599999999999997},"min":{"1min":0.35499999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.76800000000000002,"5min":0.76800000000000002,"15min":0.76800000000000002},"last":0.60699999999999998},{"interface":"front","average":{"1min":0.61399999999999999,"5min":0.61399999999999999,"15min":0.61399999999999999},"min":{"1min":0.436,"5min":0.436,"15min":0.436},"max":{"1min":1.095,"5min":1.095,"15min":1.095},"last":0.504}]},{"osd":7,"last update":"Fri Nov 24 07:57:21 2023","interfaces":[{"interface":"back","average":{"1min":0.57299999999999995,"5min":0.57299999999999995,"15min":0.57299999999999995},"min":{"1min":0.35099999999999998,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.89300000000000002,"5min":0.89300000000000002,"15min":0.89300000000000002},"last":0.54400000000000004},{"interface":"front","average":{"1min":0.66300000000000003,"5min":0.66300000000000003,"15min":0.66300000000000003},"min":{"1min":0.41099999999999998,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":0.91500000000000004,"5min":0.91500000000000004,"15min":0.91500000000000004},"last":0.52900000000000003}]}]},{"osd":5,"up_from":13,"seq":55834574867,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27976,"kb_used_data":948,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343864,"statfs":{"total":96636764160,"available":96608116736,"internally_reserved":0,"allocated":970752,"data_stored":737048,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 07:57:17 2023","interfaces":[{"interface":"back","average":{"1min":0.56200000000000006,"5min":0.56200000000000006,"15min":0.56200000000000006},"min":{"1min":0.32700000000000001,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":1.0109999999999999,"5min":1.0109999999999999,"15min":1.0109999999999999},"last":0.41399999999999998},{"interface":"front","average":{"1min":0.80000000000000004,"5min":0.80000000000000004,"15min":0.80000000000000004},"min":{"1min":0.33400000000000002,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":5.431,"5min":5.431,"15min":5.431},"last":0.57299999999999995}]},{"osd":1,"last update":"Fri Nov 24 07:57:17 2023","interfaces":[{"interface":"back","average":{"1min":0.55900000000000005,"5min":0.55900000000000005,"15min":0.55900000000000005},"min":{"1min":0.28899999999999998,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":1.016,"5min":1.016,"15min":1.016},"last":0.39700000000000002},{"interface":"front","average":{"1min":0.56999999999999995,"5min":0.56999999999999995,"15min":0.56999999999999995},"min":{"1min":0.311,"5min":0.311,"15min":0.311},"max":{"1min":0.96499999999999997,"5min":0.96499999999999997,"15min":0.96499999999999997},"last":0.55100000000000005}]},{"osd":2,"last update":"Fri Nov 24 07:57:17 2023","interfaces":[{"interface":"back","average":{"1min":0.55100000000000005,"5min":0.55100000000000005,"15min":0.55100000000000005},"min":{"1min":0.377,"5min":0.377,"15min":0.377},"max":{"1min":1.163,"5min":1.163,"15min":1.163},"last":0.42799999999999999},{"interface":"front","average":{"1min":0.63,"5min":0.63,"15min":0.63},"min":{"1min":0.39200000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":1.119,"5min":1.119,"15min":1.119},"last":0.52100000000000002}]},{"osd":3,"last update":"Fri Nov 24 07:57:17 2023","interfaces":[{"interface":"back","average":{"1min":0.90500000000000003,"5min":0.90500000000000003,"15min":0.90500000000000003},"min":{"1min":0.36499999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":5.609,"5min":5.609,"15min":5.609},"last":0.36899999999999999},{"interface":"front","average":{"1min":0.67600000000000005,"5min":0.67600000000000005,"15min":0.67600000000000005},"min":{"1min":0.41899999999999998,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":1.288,"5min":1.288,"15min":1.288},"last":0.53700000000000003}]},{"osd":4,"last update":"Fri Nov 24 07:57:17 2023","interfaces":[{"interface":"back","average":{"1min":0.51000000000000001,"5min":0.51000000000000001,"15min":0.51000000000000001},"min":{"1min":0.36399999999999999,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.68799999999999994,"5min":0.68799999999999994,"15min":0.68799999999999994},"last":0.32100000000000001},{"interface":"front","average":{"1min":0.57199999999999995,"5min":0.57199999999999995,"15min":0.57199999999999995},"min":{"1min":0.39800000000000002,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.78300000000000003,"5min":0.78300000000000003,"15min":0.78300000000000003},"last":0.29599999999999999}]},{"osd":6,"last update":"Fri Nov 24 07:57:17 2023","interfaces":[{"interface":"back","average":{"1min":0.57099999999999995,"5min":0.57099999999999995,"15min":0.57099999999999995},"min":{"1min":0.35599999999999998,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.89700000000000002,"5min":0.89700000000000002,"15min":0.89700000000000002},"last":0.35199999999999998},{"interface":"front","average":{"1min":0.625,"5min":0.625,"15min":0.625},"min":{"1min":0.42899999999999999,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":0.88200000000000001,"5min":0.88200000000000001,"15min":0.88200000000000001},"last":0.44800000000000001}]},{"osd":7,"last update":"Fri Nov 24 07:57:17 2023","interfaces":[{"interface":"back","average":{"1min":0.64600000000000002,"5min":0.64600000000000002,"15min":0.64600000000000002},"min":{"1min":0.45000000000000001,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":0.96599999999999997,"5min":0.96599999999999997,"15min":0.96599999999999997},"last":0.46700000000000003},{"interface":"front","average":{"1min":0.65300000000000002,"5min":0.65300000000000002,"15min":0.65300000000000002},"min":{"1min":0.46300000000000002,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":0.94199999999999995,"5min":0.94199999999999995,"15min":0.94199999999999995},"last":0.502}]}]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":776,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-11-24T07:57:30.599 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2023-11-24T07:57:30.757 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:30.755+0000 7f04c3300700 1 Processor -- start 2023-11-24T07:57:30.757 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:30.755+0000 7f04c3300700 1 -- start start 2023-11-24T07:57:30.767 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:30.756+0000 7f04c3300700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f04bc15e1c0 0x7f04bc06eef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:57:30.767 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:30.756+0000 7f04c3300700 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f04bc0d1230 con 0x7f04bc06fc50 2023-11-24T07:57:30.767 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:30.756+0000 7f04c3300700 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f04bc0d0c70 con 0x7f04bc0b0e90 2023-11-24T07:57:30.767 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:30.756+0000 7f04c3300700 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f04bc0d0f50 con 0x7f04bc15e1c0 2023-11-24T07:57:30.768 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:30.756+0000 7f04c109c700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f04bc15e1c0 0x7f04bc06eef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:57:30.768 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:30.756+0000 7f04c189d700 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7f04bc06fc50 0x7f04bc06ca10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:55276/0 (socket says 172.21.15.59:55276) 2023-11-24T07:57:30.768 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:30.756+0000 7f04c189d700 1 -- 172.21.15.59:0/2315329199 learned_addr learned my addr 172.21.15.59:0/2315329199 (peer_addr_for_me v1:172.21.15.59:0/0) 2023-11-24T07:57:30.768 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:30.756+0000 7f04c109c700 1 -- 172.21.15.59:0/2315329199 >> v1:172.21.15.76:6789/0 conn(0x7f04bc0b0e90 legacy=0x7f04bc06f540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:57:30.769 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:30.756+0000 7f04c109c700 1 -- 172.21.15.59:0/2315329199 >> v1:172.21.15.59:6789/0 conn(0x7f04bc06fc50 legacy=0x7f04bc06ca10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:57:30.769 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:30.756+0000 7f04b3fff700 1 -- 172.21.15.59:0/2315329199 <== mon.0 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 404195649 0 0) 0x7f04bc0d1230 con 0x7f04bc06fc50 2023-11-24T07:57:30.769 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:30.757+0000 7f04c109c700 1 -- 172.21.15.59:0/2315329199 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f04bc06c210 con 0x7f04bc15e1c0 2023-11-24T07:57:30.769 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:30.757+0000 7f04c109c700 1 --2- 172.21.15.59:0/2315329199 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f04bc15e1c0 0x7f04bc06eef0 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f04ac009840 tx=0x7f04ac009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=25ffb382d4c6557f server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:57:30.770 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:30.757+0000 7f04b3fff700 1 -- 172.21.15.59:0/2315329199 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04ac011830 con 0x7f04bc15e1c0 2023-11-24T07:57:30.770 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:30.757+0000 7f04b3fff700 1 -- 172.21.15.59:0/2315329199 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f04ac011990 con 0x7f04bc15e1c0 2023-11-24T07:57:30.770 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:30.757+0000 7f04b3fff700 1 -- 172.21.15.59:0/2315329199 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04ac011d20 con 0x7f04bc15e1c0 2023-11-24T07:57:30.770 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:30.757+0000 7f04c3300700 1 -- 172.21.15.59:0/2315329199 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f04bc15e1c0 msgr2=0x7f04bc06eef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:57:30.770 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:30.757+0000 7f04c3300700 1 --2- 172.21.15.59:0/2315329199 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f04bc15e1c0 0x7f04bc06eef0 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f04ac009840 tx=0x7f04ac009b50 comp rx=0 tx=0).stop 2023-11-24T07:57:30.770 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:30.757+0000 7f04c3300700 1 -- 172.21.15.59:0/2315329199 shutdown_connections 2023-11-24T07:57:30.771 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:30.757+0000 7f04c3300700 1 --2- 172.21.15.59:0/2315329199 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f04bc15e1c0 0x7f04bc06eef0 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:30.771 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:30.757+0000 7f04c3300700 1 -- 172.21.15.59:0/2315329199 >> 172.21.15.59:0/2315329199 conn(0x7f04bc0b3720 msgr2=0x7f04bc0b3b30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:57:30.771 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:30.757+0000 7f04c3300700 1 -- 172.21.15.59:0/2315329199 shutdown_connections 2023-11-24T07:57:30.771 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:30.757+0000 7f04c3300700 1 -- 172.21.15.59:0/2315329199 wait complete. 2023-11-24T07:57:30.771 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:30.759+0000 7f04c3300700 1 Processor -- start 2023-11-24T07:57:30.771 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:30.759+0000 7f04c3300700 1 -- start start 2023-11-24T07:57:30.771 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:30.759+0000 7f04c3300700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f04bc06fc50 0x7f04bc157e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:57:30.772 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:30.759+0000 7f04c3300700 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f04bc0b0e90 0x7f04bc1f40d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:57:30.772 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:30.759+0000 7f04c3300700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f04bc15e1c0 0x7f04bc0b2fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:57:30.772 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:30.759+0000 7f04c3300700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f04bc0b6050 con 0x7f04bc15e1c0 2023-11-24T07:57:30.772 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:30.759+0000 7f04c3300700 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f04bc1f4610 con 0x7f04bc06fc50 2023-11-24T07:57:30.772 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:30.759+0000 7f04c3300700 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f04bc15fcf0 con 0x7f04bc0b0e90 2023-11-24T07:57:30.772 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:30.759+0000 7f04c189d700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f04bc15e1c0 0x7f04bc0b2fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:57:30.773 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:30.760+0000 7f04c189d700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f04bc15e1c0 0x7f04bc0b2fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:51810/0 (socket says 172.21.15.59:51810) 2023-11-24T07:57:30.773 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:30.760+0000 7f04c189d700 1 -- 172.21.15.59:0/2284672147 learned_addr learned my addr 172.21.15.59:0/2284672147 (peer_addr_for_me v2:172.21.15.59:0/0) 2023-11-24T07:57:30.773 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:30.760+0000 7f04c089b700 1 --2- 172.21.15.59:0/2284672147 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f04bc0b0e90 0x7f04bc1f40d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:57:30.773 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:30.760+0000 7f04c189d700 1 -- 172.21.15.59:0/2284672147 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f04bc0b0e90 msgr2=0x7f04bc1f40d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:57:30.773 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:30.760+0000 7f04c189d700 1 --2- 172.21.15.59:0/2284672147 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f04bc0b0e90 0x7f04bc1f40d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:30.773 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:30.760+0000 7f04c189d700 1 -- 172.21.15.59:0/2284672147 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f04bc06fc50 msgr2=0x7f04bc157e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:57:30.773 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:30.760+0000 7f04c109c700 1 --2- 172.21.15.59:0/2284672147 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f04bc06fc50 0x7f04bc157e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:57:30.774 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:30.760+0000 7f04c189d700 1 --2- 172.21.15.59:0/2284672147 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f04bc06fc50 0x7f04bc157e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:30.774 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:30.760+0000 7f04c189d700 1 -- 172.21.15.59:0/2284672147 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f04ac0094a0 con 0x7f04bc15e1c0 2023-11-24T07:57:30.774 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:30.760+0000 7f04c089b700 1 --2- 172.21.15.59:0/2284672147 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f04bc0b0e90 0x7f04bc1f40d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T07:57:30.774 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:30.760+0000 7f04c189d700 1 --2- 172.21.15.59:0/2284672147 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f04bc15e1c0 0x7f04bc0b2fa0 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f04b400cc70 tx=0x7f04b400a340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:57:30.774 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:30.760+0000 7f04b1ffb700 1 -- 172.21.15.59:0/2284672147 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04b400ad80 con 0x7f04bc15e1c0 2023-11-24T07:57:30.774 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:30.760+0000 7f04b1ffb700 1 -- 172.21.15.59:0/2284672147 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f04b400ce70 con 0x7f04bc15e1c0 2023-11-24T07:57:30.775 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:30.760+0000 7f04c3300700 1 -- 172.21.15.59:0/2284672147 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f04bc0b34e0 con 0x7f04bc15e1c0 2023-11-24T07:57:30.775 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:30.760+0000 7f04b1ffb700 1 -- 172.21.15.59:0/2284672147 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04b40072f0 con 0x7f04bc15e1c0 2023-11-24T07:57:30.775 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:30.760+0000 7f04c3300700 1 -- 172.21.15.59:0/2284672147 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f04bc0b5990 con 0x7f04bc15e1c0 2023-11-24T07:57:30.775 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:30.762+0000 7f04c3300700 1 -- 172.21.15.59:0/2284672147 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f04bc0b6190 con 0x7f04bc15e1c0 2023-11-24T07:57:30.775 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:30.762+0000 7f04b1ffb700 1 -- 172.21.15.59:0/2284672147 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f04b4007450 con 0x7f04bc15e1c0 2023-11-24T07:57:30.775 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:30.762+0000 7f04b1ffb700 1 --2- 172.21.15.59:0/2284672147 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f04a006aed0 0x7f04a006d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:57:30.775 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:30.762+0000 7f04b1ffb700 1 -- 172.21.15.59:0/2284672147 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f04b4089460 con 0x7f04bc15e1c0 2023-11-24T07:57:30.775 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:30.762+0000 7f04c109c700 1 --2- 172.21.15.59:0/2284672147 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f04a006aed0 0x7f04a006d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:57:30.775 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:30.763+0000 7f04c109c700 1 --2- 172.21.15.59:0/2284672147 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f04a006aed0 0x7f04a006d390 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f04bc09fc90 tx=0x7f04ac01c040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:57:30.776 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:30.767+0000 7f04b1ffb700 1 -- 172.21.15.59:0/2284672147 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f04bc0b6190 con 0x7f04bc15e1c0 2023-11-24T07:57:31.056 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:31.054+0000 7f04c3300700 1 -- 172.21.15.59:0/2284672147 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) v1 -- 0x7f04bc0c89a0 con 0x7f04bc15e1c0 2023-11-24T07:57:31.056 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:31.054+0000 7f04b1ffb700 1 -- 172.21.15.59:0/2284672147 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) v1 ==== 72+0+1559 (secure 0 0 0) 0x7f04b40545a0 con 0x7f04bc15e1c0 2023-11-24T07:57:31.056 INFO:teuthology.orchestra.run.smithi059.stdout: 2023-11-24T07:57:31.056 INFO:teuthology.orchestra.run.smithi059.stdout:{"fsid":"9f0e8741-b8f9-48d5-a2c3-09f45903303d","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":84,"monmap":{"epoch":1,"min_mon_release_name":"reef","num_mons":3},"osdmap":{"epoch":33,"num_osds":8,"num_up_osds":8,"osd_up_since":1700812577,"num_in_osds":8,"osd_in_since":1700812569,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":97}],"num_pgs":97,"num_pools":3,"num_objects":24,"data_bytes":592778,"bytes_used":225624064,"bytes_avail":772868489216,"bytes_total":773094113280,"write_bytes_sec":2388,"read_op_per_sec":0,"write_op_per_sec":7},"fsmap":{"epoch":16,"id":2,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":2,"rank":0,"name":"a","status":"up:active","gid":5109}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":4,"modified":"2023-11-24T07:57:20.311884+0000","services":{"mds":{"daemons":{"summary":"","a":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"b":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"c":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"d":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2023-11-24T07:57:31.059 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:31.057+0000 7f04c3300700 1 -- 172.21.15.59:0/2284672147 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f04a006aed0 msgr2=0x7f04a006d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:57:31.059 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:31.057+0000 7f04c3300700 1 --2- 172.21.15.59:0/2284672147 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f04a006aed0 0x7f04a006d390 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f04bc09fc90 tx=0x7f04ac01c040 comp rx=0 tx=0).stop 2023-11-24T07:57:31.059 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:31.057+0000 7f04c3300700 1 -- 172.21.15.59:0/2284672147 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f04bc15e1c0 msgr2=0x7f04bc0b2fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:57:31.059 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:31.057+0000 7f04c3300700 1 --2- 172.21.15.59:0/2284672147 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f04bc15e1c0 0x7f04bc0b2fa0 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f04b400cc70 tx=0x7f04b400a340 comp rx=0 tx=0).stop 2023-11-24T07:57:31.059 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:31.057+0000 7f04c3300700 1 -- 172.21.15.59:0/2284672147 shutdown_connections 2023-11-24T07:57:31.059 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:31.057+0000 7f04c3300700 1 --2- 172.21.15.59:0/2284672147 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f04a006aed0 0x7f04a006d390 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:31.059 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:31.057+0000 7f04c3300700 1 --2- 172.21.15.59:0/2284672147 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f04bc15e1c0 0x7f04bc0b2fa0 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:31.060 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:31.057+0000 7f04c3300700 1 --2- 172.21.15.59:0/2284672147 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f04bc06fc50 0x7f04bc157e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:31.060 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:31.057+0000 7f04c3300700 1 --2- 172.21.15.59:0/2284672147 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f04bc0b0e90 0x7f04bc1f40d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:31.060 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:31.057+0000 7f04c3300700 1 -- 172.21.15.59:0/2284672147 >> 172.21.15.59:0/2284672147 conn(0x7f04bc0b3720 msgr2=0x7f04bc06eef0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:57:31.060 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:31.057+0000 7f04c3300700 1 -- 172.21.15.59:0/2284672147 shutdown_connections 2023-11-24T07:57:31.060 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:31.057+0000 7f04c3300700 1 -- 172.21.15.59:0/2284672147 wait complete. 2023-11-24T07:57:31.070 INFO:tasks.cephfs.filesystem.ceph_manager:no progress seen, keeping timeout for now 2023-11-24T07:57:31.071 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-11-24T07:57:31.195 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:31.193+0000 7f3ed9382700 1 Processor -- start 2023-11-24T07:57:31.195 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:31.193+0000 7f3ed9382700 1 -- start start 2023-11-24T07:57:31.198 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:31.194+0000 7f3ed9382700 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f3ed406b620 0x7f3ed40b3a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:57:31.199 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:31.194+0000 7f3ed9382700 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3ed41629c0 con 0x7f3ed406ee60 2023-11-24T07:57:31.199 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:31.194+0000 7f3ed9382700 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3ed4071f50 con 0x7f3ed40b5ab0 2023-11-24T07:57:31.199 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:31.194+0000 7f3ed9382700 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3ed4072230 con 0x7f3ed406b620 2023-11-24T07:57:31.199 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:31.194+0000 7f3ed37fe700 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7f3ed406ee60 0x7f3ed41622b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:55292/0 (socket says 172.21.15.59:55292) 2023-11-24T07:57:31.199 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:31.194+0000 7f3ed37fe700 1 -- 172.21.15.59:0/310969925 learned_addr learned my addr 172.21.15.59:0/310969925 (peer_addr_for_me v1:172.21.15.59:0/0) 2023-11-24T07:57:31.199 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:31.195+0000 7f3ed2ffd700 1 --2- 172.21.15.59:0/310969925 >> v2:172.21.15.76:3300/0 conn(0x7f3ed406b620 0x7f3ed40b3a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:57:31.199 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:31.195+0000 7f3ed1ffb700 1 -- 172.21.15.59:0/310969925 <== mon.0 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1976044133 0 0) 0x7f3ed41629c0 con 0x7f3ed406ee60 2023-11-24T07:57:31.200 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:31.195+0000 7f3ed1ffb700 1 -- 172.21.15.59:0/310969925 --> v1:172.21.15.59:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3ec00035b0 con 0x7f3ed406ee60 2023-11-24T07:57:31.200 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:31.195+0000 7f3ed1ffb700 1 -- 172.21.15.59:0/310969925 <== mon.1 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3466886735 0 0) 0x7f3ed4071f50 con 0x7f3ed40b5ab0 2023-11-24T07:57:31.200 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:31.195+0000 7f3ed1ffb700 1 -- 172.21.15.59:0/310969925 --> v1:172.21.15.76:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3ed41629c0 con 0x7f3ed40b5ab0 2023-11-24T07:57:31.200 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:31.195+0000 7f3ed2ffd700 1 -- 172.21.15.59:0/310969925 >> v1:172.21.15.76:6789/0 conn(0x7f3ed40b5ab0 legacy=0x7f3ed4161ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:57:31.201 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:31.195+0000 7f3ed2ffd700 1 -- 172.21.15.59:0/310969925 >> v1:172.21.15.59:6789/0 conn(0x7f3ed406ee60 legacy=0x7f3ed41622b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:57:31.201 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:31.195+0000 7f3ed2ffd700 1 -- 172.21.15.59:0/310969925 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3ed4072510 con 0x7f3ed406b620 2023-11-24T07:57:31.201 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:31.195+0000 7f3ed2ffd700 1 --2- 172.21.15.59:0/310969925 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3ed406b620 0x7f3ed40b3a40 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f3ec4009840 tx=0x7f3ec4009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=983c65d5699417ea server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:57:31.201 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:31.195+0000 7f3ed1ffb700 1 -- 172.21.15.59:0/310969925 <== mon.1 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ec4011830 con 0x7f3ed406b620 2023-11-24T07:57:31.202 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:31.195+0000 7f3ed1ffb700 1 -- 172.21.15.59:0/310969925 <== mon.1 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3ec4011990 con 0x7f3ed406b620 2023-11-24T07:57:31.202 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:31.195+0000 7f3ed1ffb700 1 -- 172.21.15.59:0/310969925 <== mon.1 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ec4011d20 con 0x7f3ed406b620 2023-11-24T07:57:31.202 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:31.196+0000 7f3ed9382700 1 -- 172.21.15.59:0/310969925 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3ed406b620 msgr2=0x7f3ed40b3a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:57:31.202 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:31.196+0000 7f3ed9382700 1 --2- 172.21.15.59:0/310969925 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3ed406b620 0x7f3ed40b3a40 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f3ec4009840 tx=0x7f3ec4009b50 comp rx=0 tx=0).stop 2023-11-24T07:57:31.202 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:31.196+0000 7f3ed9382700 1 -- 172.21.15.59:0/310969925 shutdown_connections 2023-11-24T07:57:31.202 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:31.196+0000 7f3ed9382700 1 --2- 172.21.15.59:0/310969925 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3ed406b620 0x7f3ed40b3a40 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:31.203 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:31.196+0000 7f3ed9382700 1 -- 172.21.15.59:0/310969925 >> 172.21.15.59:0/310969925 conn(0x7f3ed40b2740 msgr2=0x7f3ed40b2b20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:57:31.203 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:31.196+0000 7f3ed9382700 1 -- 172.21.15.59:0/310969925 shutdown_connections 2023-11-24T07:57:31.203 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:31.196+0000 7f3ed9382700 1 -- 172.21.15.59:0/310969925 wait complete. 2023-11-24T07:57:31.208 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:31.197+0000 7f3ed9382700 1 Processor -- start 2023-11-24T07:57:31.208 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:31.197+0000 7f3ed9382700 1 -- start start 2023-11-24T07:57:31.208 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:31.198+0000 7f3ed9382700 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f3ed406b620 0x7f3ed413eb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:57:31.208 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:31.198+0000 7f3ed9382700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3ed406ee60 0x7f3ed4157710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:57:31.208 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:31.198+0000 7f3ed9382700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3ed40b5ab0 0x7f3ed4157c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:57:31.208 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:31.198+0000 7f3ed9382700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ed4141420 con 0x7f3ed40b5ab0 2023-11-24T07:57:31.208 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:31.198+0000 7f3ed9382700 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ed41418b0 con 0x7f3ed406ee60 2023-11-24T07:57:31.208 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:31.198+0000 7f3ed9382700 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3ed4140850 con 0x7f3ed406b620 2023-11-24T07:57:31.208 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:31.198+0000 7f3ed37fe700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3ed40b5ab0 0x7f3ed4157c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:57:31.208 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:31.198+0000 7f3ed37fe700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3ed40b5ab0 0x7f3ed4157c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:51824/0 (socket says 172.21.15.59:51824) 2023-11-24T07:57:31.208 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:31.198+0000 7f3ed37fe700 1 -- 172.21.15.59:0/3328000794 learned_addr learned my addr 172.21.15.59:0/3328000794 (peer_addr_for_me v2:172.21.15.59:0/0) 2023-11-24T07:57:31.209 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:31.198+0000 7f3ed27fc700 1 --2- 172.21.15.59:0/3328000794 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3ed406ee60 0x7f3ed4157710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:57:31.209 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:31.198+0000 7f3ed37fe700 1 -- 172.21.15.59:0/3328000794 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f3ed406b620 msgr2=0x7f3ed413eb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:57:31.209 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:31.198+0000 7f3ed2ffd700 1 --2- 172.21.15.59:0/3328000794 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f3ed406b620 0x7f3ed413eb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:57:31.209 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:31.198+0000 7f3ed37fe700 1 --2- 172.21.15.59:0/3328000794 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f3ed406b620 0x7f3ed413eb20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:31.209 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:31.199+0000 7f3ed37fe700 1 -- 172.21.15.59:0/3328000794 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3ed406ee60 msgr2=0x7f3ed4157710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:57:31.209 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:31.199+0000 7f3ed37fe700 1 --2- 172.21.15.59:0/3328000794 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3ed406ee60 0x7f3ed4157710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:31.209 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:31.199+0000 7f3ed37fe700 1 -- 172.21.15.59:0/3328000794 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3ec40094a0 con 0x7f3ed40b5ab0 2023-11-24T07:57:31.209 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:31.199+0000 7f3ed27fc700 1 --2- 172.21.15.59:0/3328000794 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3ed406ee60 0x7f3ed4157710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T07:57:31.209 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:31.199+0000 7f3ed37fe700 1 --2- 172.21.15.59:0/3328000794 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3ed40b5ab0 0x7f3ed4157c50 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f3ecc00cbc0 tx=0x7f3ecc00cf80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:57:31.209 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:31.199+0000 7f3ecb7fe700 1 -- 172.21.15.59:0/3328000794 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ecc00ab50 con 0x7f3ed40b5ab0 2023-11-24T07:57:31.209 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:31.199+0000 7f3ecb7fe700 1 -- 172.21.15.59:0/3328000794 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3ecc00acb0 con 0x7f3ed40b5ab0 2023-11-24T07:57:31.210 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:31.199+0000 7f3ed9382700 1 -- 172.21.15.59:0/3328000794 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3ed4140af0 con 0x7f3ed40b5ab0 2023-11-24T07:57:31.210 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:31.199+0000 7f3ecb7fe700 1 -- 172.21.15.59:0/3328000794 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ecc012600 con 0x7f3ed40b5ab0 2023-11-24T07:57:31.210 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:31.199+0000 7f3ed9382700 1 -- 172.21.15.59:0/3328000794 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3ed41410b0 con 0x7f3ed40b5ab0 2023-11-24T07:57:31.210 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:31.200+0000 7f3ed9382700 1 -- 172.21.15.59:0/3328000794 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3ed406ba00 con 0x7f3ed40b5ab0 2023-11-24T07:57:31.210 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:31.200+0000 7f3ecb7fe700 1 -- 172.21.15.59:0/3328000794 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f3ecc01b3b0 con 0x7f3ed40b5ab0 2023-11-24T07:57:31.210 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:31.201+0000 7f3ecb7fe700 1 --2- 172.21.15.59:0/3328000794 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f3eb406aed0 0x7f3eb406d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:57:31.210 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:31.201+0000 7f3ecb7fe700 1 -- 172.21.15.59:0/3328000794 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f3ecc089210 con 0x7f3ed40b5ab0 2023-11-24T07:57:31.210 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:31.201+0000 7f3ed2ffd700 1 --2- 172.21.15.59:0/3328000794 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f3eb406aed0 0x7f3eb406d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:57:31.211 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:31.202+0000 7f3ed2ffd700 1 --2- 172.21.15.59:0/3328000794 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f3eb406aed0 0x7f3eb406d390 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f3ec4009b90 tx=0x7f3ec401c040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:57:31.211 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:31.205+0000 7f3ecb7fe700 1 -- 172.21.15.59:0/3328000794 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f3ecc054400 con 0x7f3ed40b5ab0 2023-11-24T07:57:31.402 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:31.400+0000 7f3ed9382700 1 -- 172.21.15.59:0/3328000794 --> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f3ed4072ad0 con 0x7f3eb406aed0 2023-11-24T07:57:31.410 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:31.409+0000 7f3ecb7fe700 1 -- 172.21.15.59:0/3328000794 <== mgr.4100 v2:172.21.15.76:6832/4264046536 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+288949 (secure 0 0 0) 0x7f3ec4011b00 con 0x7f3eb406aed0 2023-11-24T07:57:31.416 INFO:teuthology.orchestra.run.smithi059.stdout: 2023-11-24T07:57:31.420 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:31.411+0000 7f3ed9382700 1 -- 172.21.15.59:0/3328000794 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f3eb406aed0 msgr2=0x7f3eb406d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:57:31.420 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:31.411+0000 7f3ed9382700 1 --2- 172.21.15.59:0/3328000794 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f3eb406aed0 0x7f3eb406d390 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f3ec4009b90 tx=0x7f3ec401c040 comp rx=0 tx=0).stop 2023-11-24T07:57:31.420 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:31.411+0000 7f3ed9382700 1 -- 172.21.15.59:0/3328000794 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3ed40b5ab0 msgr2=0x7f3ed4157c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:57:31.420 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:31.411+0000 7f3ed9382700 1 --2- 172.21.15.59:0/3328000794 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3ed40b5ab0 0x7f3ed4157c50 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f3ecc00cbc0 tx=0x7f3ecc00cf80 comp rx=0 tx=0).stop 2023-11-24T07:57:31.420 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:31.412+0000 7f3ed9382700 1 -- 172.21.15.59:0/3328000794 shutdown_connections 2023-11-24T07:57:31.420 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:31.412+0000 7f3ed9382700 1 --2- 172.21.15.59:0/3328000794 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f3eb406aed0 0x7f3eb406d390 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:31.421 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:31.412+0000 7f3ed9382700 1 --2- 172.21.15.59:0/3328000794 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3ed40b5ab0 0x7f3ed4157c50 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:31.421 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:31.412+0000 7f3ed9382700 1 --2- 172.21.15.59:0/3328000794 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f3ed406b620 0x7f3ed413eb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:31.421 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:31.412+0000 7f3ed9382700 1 --2- 172.21.15.59:0/3328000794 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3ed406ee60 0x7f3ed4157710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:31.421 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:31.412+0000 7f3ed9382700 1 -- 172.21.15.59:0/3328000794 >> 172.21.15.59:0/3328000794 conn(0x7f3ed40b2740 msgr2=0x7f3ed40b06b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:57:31.421 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:31.412+0000 7f3ed9382700 1 -- 172.21.15.59:0/3328000794 shutdown_connections 2023-11-24T07:57:31.421 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:31.412+0000 7f3ed9382700 1 -- 172.21.15.59:0/3328000794 wait complete. 2023-11-24T07:57:31.421 INFO:teuthology.orchestra.run.smithi059.stderr:dumped all 2023-11-24T07:57:31.425 INFO:teuthology.orchestra.run.smithi059.stdout:{"pg_ready":true,"pg_map":{"version":62,"stamp":"2023-11-24T07:57:30.314053+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":97,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":220336,"kb_used_data":4136,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754754384,"statfs":{"total":773094113280,"available":772868489216,"internally_reserved":0,"allocated":4235264,"data_stored":2354340,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12745,"internal_metadata":221236791},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":33,"apply_latency_ms":33,"commit_latency_ns":33000000,"apply_latency_ns":33000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001473"},"pg_stats":[{"pgid":"5.3f","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-24T07:57:27.498774+0000","last_change":"2023-11-24T07:57:25.613284+0000","last_active":"2023-11-24T07:57:27.498774+0000","last_peered":"2023-11-24T07:57:27.498774+0000","last_clean":"2023-11-24T07:57:27.498774+0000","last_became_active":"2023-11-24T07:57:25.606187+0000","last_became_peered":"2023-11-24T07:57:25.606187+0000","last_unstale":"2023-11-24T07:57:27.498774+0000","last_undegraded":"2023-11-24T07:57:27.498774+0000","last_fullsized":"2023-11-24T07:57:27.498774+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:57:24.572681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:57:24.572681+0000","last_clean_scrub_stamp":"2023-11-24T07:57:24.572681+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T12:51:13.357718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.3e","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-24T07:57:26.584176+0000","last_change":"2023-11-24T07:57:25.611359+0000","last_active":"2023-11-24T07:57:26.584176+0000","last_peered":"2023-11-24T07:57:26.584176+0000","last_clean":"2023-11-24T07:57:26.584176+0000","last_became_active":"2023-11-24T07:57:25.611053+0000","last_became_peered":"2023-11-24T07:57:25.611053+0000","last_unstale":"2023-11-24T07:57:26.584176+0000","last_undegraded":"2023-11-24T07:57:26.584176+0000","last_fullsized":"2023-11-24T07:57:26.584176+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:57:24.572681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:57:24.572681+0000","last_clean_scrub_stamp":"2023-11-24T07:57:24.572681+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T15:47:12.125658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.3d","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-24T07:57:27.499082+0000","last_change":"2023-11-24T07:57:25.601956+0000","last_active":"2023-11-24T07:57:27.499082+0000","last_peered":"2023-11-24T07:57:27.499082+0000","last_clean":"2023-11-24T07:57:27.499082+0000","last_became_active":"2023-11-24T07:57:25.601517+0000","last_became_peered":"2023-11-24T07:57:25.601517+0000","last_unstale":"2023-11-24T07:57:27.499082+0000","last_undegraded":"2023-11-24T07:57:27.499082+0000","last_fullsized":"2023-11-24T07:57:27.499082+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:57:24.572681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:57:24.572681+0000","last_clean_scrub_stamp":"2023-11-24T07:57:24.572681+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T17:40:49.956916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.3c","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-24T07:57:27.499452+0000","last_change":"2023-11-24T07:57:25.606653+0000","last_active":"2023-11-24T07:57:27.499452+0000","last_peered":"2023-11-24T07:57:27.499452+0000","last_clean":"2023-11-24T07:57:27.499452+0000","last_became_active":"2023-11-24T07:57:25.606315+0000","last_became_peered":"2023-11-24T07:57:25.606315+0000","last_unstale":"2023-11-24T07:57:27.499452+0000","last_undegraded":"2023-11-24T07:57:27.499452+0000","last_fullsized":"2023-11-24T07:57:27.499452+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:57:24.572681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:57:24.572681+0000","last_clean_scrub_stamp":"2023-11-24T07:57:24.572681+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T13:39:16.521602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.3b","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-24T07:57:26.583942+0000","last_change":"2023-11-24T07:57:25.600519+0000","last_active":"2023-11-24T07:57:26.583942+0000","last_peered":"2023-11-24T07:57:26.583942+0000","last_clean":"2023-11-24T07:57:26.583942+0000","last_became_active":"2023-11-24T07:57:25.600235+0000","last_became_peered":"2023-11-24T07:57:25.600235+0000","last_unstale":"2023-11-24T07:57:26.583942+0000","last_undegraded":"2023-11-24T07:57:26.583942+0000","last_fullsized":"2023-11-24T07:57:26.583942+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:57:24.572681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:57:24.572681+0000","last_clean_scrub_stamp":"2023-11-24T07:57:24.572681+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T19:13:40.799042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.3a","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-24T07:57:27.498253+0000","last_change":"2023-11-24T07:57:25.600774+0000","last_active":"2023-11-24T07:57:27.498253+0000","last_peered":"2023-11-24T07:57:27.498253+0000","last_clean":"2023-11-24T07:57:27.498253+0000","last_became_active":"2023-11-24T07:57:25.600390+0000","last_became_peered":"2023-11-24T07:57:25.600390+0000","last_unstale":"2023-11-24T07:57:27.498253+0000","last_undegraded":"2023-11-24T07:57:27.498253+0000","last_fullsized":"2023-11-24T07:57:27.498253+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:57:24.572681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:57:24.572681+0000","last_clean_scrub_stamp":"2023-11-24T07:57:24.572681+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T17:53:33.447022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.39","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-24T07:57:27.499083+0000","last_change":"2023-11-24T07:57:25.596753+0000","last_active":"2023-11-24T07:57:27.499083+0000","last_peered":"2023-11-24T07:57:27.499083+0000","last_clean":"2023-11-24T07:57:27.499083+0000","last_became_active":"2023-11-24T07:57:25.596467+0000","last_became_peered":"2023-11-24T07:57:25.596467+0000","last_unstale":"2023-11-24T07:57:27.499083+0000","last_undegraded":"2023-11-24T07:57:27.499083+0000","last_fullsized":"2023-11-24T07:57:27.499083+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:57:24.572681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:57:24.572681+0000","last_clean_scrub_stamp":"2023-11-24T07:57:24.572681+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T14:04:03.022069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.38","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-24T07:57:27.498354+0000","last_change":"2023-11-24T07:57:25.601277+0000","last_active":"2023-11-24T07:57:27.498354+0000","last_peered":"2023-11-24T07:57:27.498354+0000","last_clean":"2023-11-24T07:57:27.498354+0000","last_became_active":"2023-11-24T07:57:25.600867+0000","last_became_peered":"2023-11-24T07:57:25.600867+0000","last_unstale":"2023-11-24T07:57:27.498354+0000","last_undegraded":"2023-11-24T07:57:27.498354+0000","last_fullsized":"2023-11-24T07:57:27.498354+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:57:24.572681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:57:24.572681+0000","last_clean_scrub_stamp":"2023-11-24T07:57:24.572681+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T16:47:55.590649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.37","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-24T07:57:27.499310+0000","last_change":"2023-11-24T07:57:25.611574+0000","last_active":"2023-11-24T07:57:27.499310+0000","last_peered":"2023-11-24T07:57:27.499310+0000","last_clean":"2023-11-24T07:57:27.499310+0000","last_became_active":"2023-11-24T07:57:25.611258+0000","last_became_peered":"2023-11-24T07:57:25.611258+0000","last_unstale":"2023-11-24T07:57:27.499310+0000","last_undegraded":"2023-11-24T07:57:27.499310+0000","last_fullsized":"2023-11-24T07:57:27.499310+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:57:24.572681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:57:24.572681+0000","last_clean_scrub_stamp":"2023-11-24T07:57:24.572681+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T15:54:26.161043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.36","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-24T07:57:27.499746+0000","last_change":"2023-11-24T07:57:25.606465+0000","last_active":"2023-11-24T07:57:27.499746+0000","last_peered":"2023-11-24T07:57:27.499746+0000","last_clean":"2023-11-24T07:57:27.499746+0000","last_became_active":"2023-11-24T07:57:25.606204+0000","last_became_peered":"2023-11-24T07:57:25.606204+0000","last_unstale":"2023-11-24T07:57:27.499746+0000","last_undegraded":"2023-11-24T07:57:27.499746+0000","last_fullsized":"2023-11-24T07:57:27.499746+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:57:24.572681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:57:24.572681+0000","last_clean_scrub_stamp":"2023-11-24T07:57:24.572681+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T18:02:42.589387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.35","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-24T07:57:26.583720+0000","last_change":"2023-11-24T07:57:25.610107+0000","last_active":"2023-11-24T07:57:26.583720+0000","last_peered":"2023-11-24T07:57:26.583720+0000","last_clean":"2023-11-24T07:57:26.583720+0000","last_became_active":"2023-11-24T07:57:25.609413+0000","last_became_peered":"2023-11-24T07:57:25.609413+0000","last_unstale":"2023-11-24T07:57:26.583720+0000","last_undegraded":"2023-11-24T07:57:26.583720+0000","last_fullsized":"2023-11-24T07:57:26.583720+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:57:24.572681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:57:24.572681+0000","last_clean_scrub_stamp":"2023-11-24T07:57:24.572681+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T17:08:50.195049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.34","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-24T07:57:27.498828+0000","last_change":"2023-11-24T07:57:25.600896+0000","last_active":"2023-11-24T07:57:27.498828+0000","last_peered":"2023-11-24T07:57:27.498828+0000","last_clean":"2023-11-24T07:57:27.498828+0000","last_became_active":"2023-11-24T07:57:25.600611+0000","last_became_peered":"2023-11-24T07:57:25.600611+0000","last_unstale":"2023-11-24T07:57:27.498828+0000","last_undegraded":"2023-11-24T07:57:27.498828+0000","last_fullsized":"2023-11-24T07:57:27.498828+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:57:24.572681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:57:24.572681+0000","last_clean_scrub_stamp":"2023-11-24T07:57:24.572681+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T17:53:33.447022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.33","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-24T07:57:27.499277+0000","last_change":"2023-11-24T07:57:25.609608+0000","last_active":"2023-11-24T07:57:27.499277+0000","last_peered":"2023-11-24T07:57:27.499277+0000","last_clean":"2023-11-24T07:57:27.499277+0000","last_became_active":"2023-11-24T07:57:25.609381+0000","last_became_peered":"2023-11-24T07:57:25.609381+0000","last_unstale":"2023-11-24T07:57:27.499277+0000","last_undegraded":"2023-11-24T07:57:27.499277+0000","last_fullsized":"2023-11-24T07:57:27.499277+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:57:24.572681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:57:24.572681+0000","last_clean_scrub_stamp":"2023-11-24T07:57:24.572681+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T19:30:34.653712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.32","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-24T07:57:27.499306+0000","last_change":"2023-11-24T07:57:25.605234+0000","last_active":"2023-11-24T07:57:27.499306+0000","last_peered":"2023-11-24T07:57:27.499306+0000","last_clean":"2023-11-24T07:57:27.499306+0000","last_became_active":"2023-11-24T07:57:25.604959+0000","last_became_peered":"2023-11-24T07:57:25.604959+0000","last_unstale":"2023-11-24T07:57:27.499306+0000","last_undegraded":"2023-11-24T07:57:27.499306+0000","last_fullsized":"2023-11-24T07:57:27.499306+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:57:24.572681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:57:24.572681+0000","last_clean_scrub_stamp":"2023-11-24T07:57:24.572681+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:18:21.970465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.31","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-24T07:57:26.582759+0000","last_change":"2023-11-24T07:57:25.605974+0000","last_active":"2023-11-24T07:57:26.582759+0000","last_peered":"2023-11-24T07:57:26.582759+0000","last_clean":"2023-11-24T07:57:26.582759+0000","last_became_active":"2023-11-24T07:57:25.604012+0000","last_became_peered":"2023-11-24T07:57:25.604012+0000","last_unstale":"2023-11-24T07:57:26.582759+0000","last_undegraded":"2023-11-24T07:57:26.582759+0000","last_fullsized":"2023-11-24T07:57:26.582759+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:57:24.572681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:57:24.572681+0000","last_clean_scrub_stamp":"2023-11-24T07:57:24.572681+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T12:51:13.357718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.30","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-24T07:57:26.582547+0000","last_change":"2023-11-24T07:57:25.603887+0000","last_active":"2023-11-24T07:57:26.582547+0000","last_peered":"2023-11-24T07:57:26.582547+0000","last_clean":"2023-11-24T07:57:26.582547+0000","last_became_active":"2023-11-24T07:57:25.603484+0000","last_became_peered":"2023-11-24T07:57:25.603484+0000","last_unstale":"2023-11-24T07:57:26.582547+0000","last_undegraded":"2023-11-24T07:57:26.582547+0000","last_fullsized":"2023-11-24T07:57:26.582547+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:57:24.572681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:57:24.572681+0000","last_clean_scrub_stamp":"2023-11-24T07:57:24.572681+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T19:30:34.653712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.2f","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-24T07:57:26.583766+0000","last_change":"2023-11-24T07:57:25.617360+0000","last_active":"2023-11-24T07:57:26.583766+0000","last_peered":"2023-11-24T07:57:26.583766+0000","last_clean":"2023-11-24T07:57:26.583766+0000","last_became_active":"2023-11-24T07:57:25.617094+0000","last_became_peered":"2023-11-24T07:57:25.617094+0000","last_unstale":"2023-11-24T07:57:26.583766+0000","last_undegraded":"2023-11-24T07:57:26.583766+0000","last_fullsized":"2023-11-24T07:57:26.583766+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:57:24.572681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:57:24.572681+0000","last_clean_scrub_stamp":"2023-11-24T07:57:24.572681+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:15:10.655973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2e","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-24T07:57:27.498492+0000","last_change":"2023-11-24T07:57:25.597593+0000","last_active":"2023-11-24T07:57:27.498492+0000","last_peered":"2023-11-24T07:57:27.498492+0000","last_clean":"2023-11-24T07:57:27.498492+0000","last_became_active":"2023-11-24T07:57:25.597301+0000","last_became_peered":"2023-11-24T07:57:25.597301+0000","last_unstale":"2023-11-24T07:57:27.498492+0000","last_undegraded":"2023-11-24T07:57:27.498492+0000","last_fullsized":"2023-11-24T07:57:27.498492+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:57:24.572681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:57:24.572681+0000","last_clean_scrub_stamp":"2023-11-24T07:57:24.572681+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T17:40:49.956916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2d","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-24T07:57:27.498313+0000","last_change":"2023-11-24T07:57:25.598070+0000","last_active":"2023-11-24T07:57:27.498313+0000","last_peered":"2023-11-24T07:57:27.498313+0000","last_clean":"2023-11-24T07:57:27.498313+0000","last_became_active":"2023-11-24T07:57:25.597726+0000","last_became_peered":"2023-11-24T07:57:25.597726+0000","last_unstale":"2023-11-24T07:57:27.498313+0000","last_undegraded":"2023-11-24T07:57:27.498313+0000","last_fullsized":"2023-11-24T07:57:27.498313+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:57:24.572681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:57:24.572681+0000","last_clean_scrub_stamp":"2023-11-24T07:57:24.572681+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T14:49:20.210209+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2c","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-24T07:57:27.498975+0000","last_change":"2023-11-24T07:57:25.607256+0000","last_active":"2023-11-24T07:57:27.498975+0000","last_peered":"2023-11-24T07:57:27.498975+0000","last_clean":"2023-11-24T07:57:27.498975+0000","last_became_active":"2023-11-24T07:57:25.606931+0000","last_became_peered":"2023-11-24T07:57:25.606931+0000","last_unstale":"2023-11-24T07:57:27.498975+0000","last_undegraded":"2023-11-24T07:57:27.498975+0000","last_fullsized":"2023-11-24T07:57:27.498975+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:57:24.572681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:57:24.572681+0000","last_clean_scrub_stamp":"2023-11-24T07:57:24.572681+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T12:58:55.859434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.2b","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-24T07:57:26.583134+0000","last_change":"2023-11-24T07:57:25.603944+0000","last_active":"2023-11-24T07:57:26.583134+0000","last_peered":"2023-11-24T07:57:26.583134+0000","last_clean":"2023-11-24T07:57:26.583134+0000","last_became_active":"2023-11-24T07:57:25.603636+0000","last_became_peered":"2023-11-24T07:57:25.603636+0000","last_unstale":"2023-11-24T07:57:26.583134+0000","last_undegraded":"2023-11-24T07:57:26.583134+0000","last_fullsized":"2023-11-24T07:57:26.583134+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:57:24.572681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:57:24.572681+0000","last_clean_scrub_stamp":"2023-11-24T07:57:24.572681+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T15:47:12.125658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.2a","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-24T07:57:27.498828+0000","last_change":"2023-11-24T07:57:25.615209+0000","last_active":"2023-11-24T07:57:27.498828+0000","last_peered":"2023-11-24T07:57:27.498828+0000","last_clean":"2023-11-24T07:57:27.498828+0000","last_became_active":"2023-11-24T07:57:25.614789+0000","last_became_peered":"2023-11-24T07:57:25.614789+0000","last_unstale":"2023-11-24T07:57:27.498828+0000","last_undegraded":"2023-11-24T07:57:27.498828+0000","last_fullsized":"2023-11-24T07:57:27.498828+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:57:24.572681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:57:24.572681+0000","last_clean_scrub_stamp":"2023-11-24T07:57:24.572681+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T15:32:38.628605+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.29","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-24T07:57:27.498472+0000","last_change":"2023-11-24T07:57:25.596319+0000","last_active":"2023-11-24T07:57:27.498472+0000","last_peered":"2023-11-24T07:57:27.498472+0000","last_clean":"2023-11-24T07:57:27.498472+0000","last_became_active":"2023-11-24T07:57:25.596016+0000","last_became_peered":"2023-11-24T07:57:25.596016+0000","last_unstale":"2023-11-24T07:57:27.498472+0000","last_undegraded":"2023-11-24T07:57:27.498472+0000","last_fullsized":"2023-11-24T07:57:27.498472+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:57:24.572681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:57:24.572681+0000","last_clean_scrub_stamp":"2023-11-24T07:57:24.572681+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T13:25:29.169219+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.28","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-24T07:57:27.498409+0000","last_change":"2023-11-24T07:57:25.613820+0000","last_active":"2023-11-24T07:57:27.498409+0000","last_peered":"2023-11-24T07:57:27.498409+0000","last_clean":"2023-11-24T07:57:27.498409+0000","last_became_active":"2023-11-24T07:57:25.613589+0000","last_became_peered":"2023-11-24T07:57:25.613589+0000","last_unstale":"2023-11-24T07:57:27.498409+0000","last_undegraded":"2023-11-24T07:57:27.498409+0000","last_fullsized":"2023-11-24T07:57:27.498409+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:57:24.572681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:57:24.572681+0000","last_clean_scrub_stamp":"2023-11-24T07:57:24.572681+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T12:58:55.859434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.27","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-24T07:57:27.499878+0000","last_change":"2023-11-24T07:57:25.609636+0000","last_active":"2023-11-24T07:57:27.499878+0000","last_peered":"2023-11-24T07:57:27.499878+0000","last_clean":"2023-11-24T07:57:27.499878+0000","last_became_active":"2023-11-24T07:57:25.609368+0000","last_became_peered":"2023-11-24T07:57:25.609368+0000","last_unstale":"2023-11-24T07:57:27.499878+0000","last_undegraded":"2023-11-24T07:57:27.499878+0000","last_fullsized":"2023-11-24T07:57:27.499878+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:57:24.572681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:57:24.572681+0000","last_clean_scrub_stamp":"2023-11-24T07:57:24.572681+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T19:27:55.072502+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.26","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-24T07:57:27.499815+0000","last_change":"2023-11-24T07:57:25.606180+0000","last_active":"2023-11-24T07:57:27.499815+0000","last_peered":"2023-11-24T07:57:27.499815+0000","last_clean":"2023-11-24T07:57:27.499815+0000","last_became_active":"2023-11-24T07:57:25.605879+0000","last_became_peered":"2023-11-24T07:57:25.605879+0000","last_unstale":"2023-11-24T07:57:27.499815+0000","last_undegraded":"2023-11-24T07:57:27.499815+0000","last_fullsized":"2023-11-24T07:57:27.499815+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:57:24.572681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:57:24.572681+0000","last_clean_scrub_stamp":"2023-11-24T07:57:24.572681+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T15:46:37.491547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.25","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-24T07:57:26.584072+0000","last_change":"2023-11-24T07:57:25.605311+0000","last_active":"2023-11-24T07:57:26.584072+0000","last_peered":"2023-11-24T07:57:26.584072+0000","last_clean":"2023-11-24T07:57:26.584072+0000","last_became_active":"2023-11-24T07:57:25.605045+0000","last_became_peered":"2023-11-24T07:57:25.605045+0000","last_unstale":"2023-11-24T07:57:26.584072+0000","last_undegraded":"2023-11-24T07:57:26.584072+0000","last_fullsized":"2023-11-24T07:57:26.584072+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:57:24.572681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:57:24.572681+0000","last_clean_scrub_stamp":"2023-11-24T07:57:24.572681+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T15:32:38.628605+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.24","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-24T07:57:27.498555+0000","last_change":"2023-11-24T07:57:25.601259+0000","last_active":"2023-11-24T07:57:27.498555+0000","last_peered":"2023-11-24T07:57:27.498555+0000","last_clean":"2023-11-24T07:57:27.498555+0000","last_became_active":"2023-11-24T07:57:25.600941+0000","last_became_peered":"2023-11-24T07:57:25.600941+0000","last_unstale":"2023-11-24T07:57:27.498555+0000","last_undegraded":"2023-11-24T07:57:27.498555+0000","last_fullsized":"2023-11-24T07:57:27.498555+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:57:24.572681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:57:24.572681+0000","last_clean_scrub_stamp":"2023-11-24T07:57:24.572681+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T19:30:34.653712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.23","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-24T07:57:26.583176+0000","last_change":"2023-11-24T07:57:25.609706+0000","last_active":"2023-11-24T07:57:26.583176+0000","last_peered":"2023-11-24T07:57:26.583176+0000","last_clean":"2023-11-24T07:57:26.583176+0000","last_became_active":"2023-11-24T07:57:25.609419+0000","last_became_peered":"2023-11-24T07:57:25.609419+0000","last_unstale":"2023-11-24T07:57:26.583176+0000","last_undegraded":"2023-11-24T07:57:26.583176+0000","last_fullsized":"2023-11-24T07:57:26.583176+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:57:24.572681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:57:24.572681+0000","last_clean_scrub_stamp":"2023-11-24T07:57:24.572681+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T14:04:03.022069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.22","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-24T07:57:26.582928+0000","last_change":"2023-11-24T07:57:25.607107+0000","last_active":"2023-11-24T07:57:26.582928+0000","last_peered":"2023-11-24T07:57:26.582928+0000","last_clean":"2023-11-24T07:57:26.582928+0000","last_became_active":"2023-11-24T07:57:25.606797+0000","last_became_peered":"2023-11-24T07:57:25.606797+0000","last_unstale":"2023-11-24T07:57:26.582928+0000","last_undegraded":"2023-11-24T07:57:26.582928+0000","last_fullsized":"2023-11-24T07:57:26.582928+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:57:24.572681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:57:24.572681+0000","last_clean_scrub_stamp":"2023-11-24T07:57:24.572681+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T15:32:38.628605+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.21","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-24T07:57:27.499178+0000","last_change":"2023-11-24T07:57:25.605343+0000","last_active":"2023-11-24T07:57:27.499178+0000","last_peered":"2023-11-24T07:57:27.499178+0000","last_clean":"2023-11-24T07:57:27.499178+0000","last_became_active":"2023-11-24T07:57:25.605002+0000","last_became_peered":"2023-11-24T07:57:25.605002+0000","last_unstale":"2023-11-24T07:57:27.499178+0000","last_undegraded":"2023-11-24T07:57:27.499178+0000","last_fullsized":"2023-11-24T07:57:27.499178+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:57:24.572681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:57:24.572681+0000","last_clean_scrub_stamp":"2023-11-24T07:57:24.572681+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:18:21.970465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.20","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-24T07:57:26.583562+0000","last_change":"2023-11-24T07:57:25.615397+0000","last_active":"2023-11-24T07:57:26.583562+0000","last_peered":"2023-11-24T07:57:26.583562+0000","last_clean":"2023-11-24T07:57:26.583562+0000","last_became_active":"2023-11-24T07:57:25.615122+0000","last_became_peered":"2023-11-24T07:57:25.615122+0000","last_unstale":"2023-11-24T07:57:26.583562+0000","last_undegraded":"2023-11-24T07:57:26.583562+0000","last_fullsized":"2023-11-24T07:57:26.583562+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:57:24.572681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:57:24.572681+0000","last_clean_scrub_stamp":"2023-11-24T07:57:24.572681+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:32:55.195260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-24T07:57:27.498923+0000","last_change":"2023-11-24T07:57:25.612902+0000","last_active":"2023-11-24T07:57:27.498923+0000","last_peered":"2023-11-24T07:57:27.498923+0000","last_clean":"2023-11-24T07:57:27.498923+0000","last_became_active":"2023-11-24T07:57:25.612479+0000","last_became_peered":"2023-11-24T07:57:25.612479+0000","last_unstale":"2023-11-24T07:57:27.498923+0000","last_undegraded":"2023-11-24T07:57:27.498923+0000","last_fullsized":"2023-11-24T07:57:27.498923+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:57:24.572681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:57:24.572681+0000","last_clean_scrub_stamp":"2023-11-24T07:57:24.572681+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T16:47:55.590649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1f","version":"32'4","reported_seq":16,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-24T07:57:26.584058+0000","last_change":"2023-11-24T07:57:24.600719+0000","last_active":"2023-11-24T07:57:26.584058+0000","last_peered":"2023-11-24T07:57:26.584058+0000","last_clean":"2023-11-24T07:57:26.584058+0000","last_became_active":"2023-11-24T07:57:24.600406+0000","last_became_peered":"2023-11-24T07:57:24.600406+0000","last_unstale":"2023-11-24T07:57:26.584058+0000","last_undegraded":"2023-11-24T07:57:26.584058+0000","last_fullsized":"2023-11-24T07:57:26.584058+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:57:23.565116+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:57:23.565116+0000","last_clean_scrub_stamp":"2023-11-24T07:57:23.565116+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T14:20:43.593019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1f","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-24T07:57:27.498803+0000","last_change":"2023-11-24T07:57:25.600985+0000","last_active":"2023-11-24T07:57:27.498803+0000","last_peered":"2023-11-24T07:57:27.498803+0000","last_clean":"2023-11-24T07:57:27.498803+0000","last_became_active":"2023-11-24T07:57:25.600673+0000","last_became_peered":"2023-11-24T07:57:25.600673+0000","last_unstale":"2023-11-24T07:57:27.498803+0000","last_undegraded":"2023-11-24T07:57:27.498803+0000","last_fullsized":"2023-11-24T07:57:27.498803+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:57:24.572681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:57:24.572681+0000","last_clean_scrub_stamp":"2023-11-24T07:57:24.572681+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T19:13:40.799042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.1e","version":"32'1","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-24T07:57:27.499834+0000","last_change":"2023-11-24T07:57:24.597208+0000","last_active":"2023-11-24T07:57:27.499834+0000","last_peered":"2023-11-24T07:57:27.499834+0000","last_clean":"2023-11-24T07:57:27.499834+0000","last_became_active":"2023-11-24T07:57:24.596957+0000","last_became_peered":"2023-11-24T07:57:24.596957+0000","last_unstale":"2023-11-24T07:57:27.499834+0000","last_undegraded":"2023-11-24T07:57:27.499834+0000","last_fullsized":"2023-11-24T07:57:27.499834+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:57:23.565116+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:57:23.565116+0000","last_clean_scrub_stamp":"2023-11-24T07:57:23.565116+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T12:51:12.350152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-24T07:57:27.498872+0000","last_change":"2023-11-24T07:57:25.615058+0000","last_active":"2023-11-24T07:57:27.498872+0000","last_peered":"2023-11-24T07:57:27.498872+0000","last_clean":"2023-11-24T07:57:27.498872+0000","last_became_active":"2023-11-24T07:57:25.614773+0000","last_became_peered":"2023-11-24T07:57:25.614773+0000","last_unstale":"2023-11-24T07:57:27.498872+0000","last_undegraded":"2023-11-24T07:57:27.498872+0000","last_fullsized":"2023-11-24T07:57:27.498872+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:57:24.572681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:57:24.572681+0000","last_clean_scrub_stamp":"2023-11-24T07:57:24.572681+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T15:47:12.125658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.1d","version":"32'2","reported_seq":14,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-24T07:57:27.498444+0000","last_change":"2023-11-24T07:57:24.602623+0000","last_active":"2023-11-24T07:57:27.498444+0000","last_peered":"2023-11-24T07:57:27.498444+0000","last_clean":"2023-11-24T07:57:27.498444+0000","last_became_active":"2023-11-24T07:57:24.602295+0000","last_became_peered":"2023-11-24T07:57:24.602295+0000","last_unstale":"2023-11-24T07:57:27.498444+0000","last_undegraded":"2023-11-24T07:57:27.498444+0000","last_fullsized":"2023-11-24T07:57:27.498444+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:57:23.565116+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:57:23.565116+0000","last_clean_scrub_stamp":"2023-11-24T07:57:23.565116+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T15:12:48.897533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-24T07:57:27.499822+0000","last_change":"2023-11-24T07:57:25.604332+0000","last_active":"2023-11-24T07:57:27.499822+0000","last_peered":"2023-11-24T07:57:27.499822+0000","last_clean":"2023-11-24T07:57:27.499822+0000","last_became_active":"2023-11-24T07:57:25.603998+0000","last_became_peered":"2023-11-24T07:57:25.603998+0000","last_unstale":"2023-11-24T07:57:27.499822+0000","last_undegraded":"2023-11-24T07:57:27.499822+0000","last_fullsized":"2023-11-24T07:57:27.499822+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:57:24.572681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:57:24.572681+0000","last_clean_scrub_stamp":"2023-11-24T07:57:24.572681+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T11:10:18.640767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1c","version":"32'2","reported_seq":14,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-24T07:57:27.499815+0000","last_change":"2023-11-24T07:57:24.595626+0000","last_active":"2023-11-24T07:57:27.499815+0000","last_peered":"2023-11-24T07:57:27.499815+0000","last_clean":"2023-11-24T07:57:27.499815+0000","last_became_active":"2023-11-24T07:57:24.595343+0000","last_became_peered":"2023-11-24T07:57:24.595343+0000","last_unstale":"2023-11-24T07:57:27.499815+0000","last_undegraded":"2023-11-24T07:57:27.499815+0000","last_fullsized":"2023-11-24T07:57:27.499815+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:57:23.565116+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:57:23.565116+0000","last_clean_scrub_stamp":"2023-11-24T07:57:23.565116+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T19:13:39.791476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-24T07:57:27.498316+0000","last_change":"2023-11-24T07:57:25.611772+0000","last_active":"2023-11-24T07:57:27.498316+0000","last_peered":"2023-11-24T07:57:27.498316+0000","last_clean":"2023-11-24T07:57:27.498316+0000","last_became_active":"2023-11-24T07:57:25.611486+0000","last_became_peered":"2023-11-24T07:57:25.611486+0000","last_unstale":"2023-11-24T07:57:27.498316+0000","last_undegraded":"2023-11-24T07:57:27.498316+0000","last_fullsized":"2023-11-24T07:57:27.498316+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:57:24.572681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:57:24.572681+0000","last_clean_scrub_stamp":"2023-11-24T07:57:24.572681+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T15:12:49.905098+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1b","version":"32'1","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-24T07:57:27.498458+0000","last_change":"2023-11-24T07:57:24.589658+0000","last_active":"2023-11-24T07:57:27.498458+0000","last_peered":"2023-11-24T07:57:27.498458+0000","last_clean":"2023-11-24T07:57:27.498458+0000","last_became_active":"2023-11-24T07:57:24.589385+0000","last_became_peered":"2023-11-24T07:57:24.589385+0000","last_unstale":"2023-11-24T07:57:27.498458+0000","last_undegraded":"2023-11-24T07:57:27.498458+0000","last_fullsized":"2023-11-24T07:57:27.498458+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:57:23.565116+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:57:23.565116+0000","last_clean_scrub_stamp":"2023-11-24T07:57:23.565116+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T16:10:47.555733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-24T07:57:27.498658+0000","last_change":"2023-11-24T07:57:25.605184+0000","last_active":"2023-11-24T07:57:27.498658+0000","last_peered":"2023-11-24T07:57:27.498658+0000","last_clean":"2023-11-24T07:57:27.498658+0000","last_became_active":"2023-11-24T07:57:25.604881+0000","last_became_peered":"2023-11-24T07:57:25.604881+0000","last_unstale":"2023-11-24T07:57:27.498658+0000","last_undegraded":"2023-11-24T07:57:27.498658+0000","last_fullsized":"2023-11-24T07:57:27.498658+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:57:24.572681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:57:24.572681+0000","last_clean_scrub_stamp":"2023-11-24T07:57:24.572681+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T09:32:54.344241+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.1a","version":"32'1","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-24T07:57:27.498824+0000","last_change":"2023-11-24T07:57:24.595865+0000","last_active":"2023-11-24T07:57:27.498824+0000","last_peered":"2023-11-24T07:57:27.498824+0000","last_clean":"2023-11-24T07:57:27.498824+0000","last_became_active":"2023-11-24T07:57:24.595549+0000","last_became_peered":"2023-11-24T07:57:24.595549+0000","last_unstale":"2023-11-24T07:57:27.498824+0000","last_undegraded":"2023-11-24T07:57:27.498824+0000","last_fullsized":"2023-11-24T07:57:27.498824+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:57:23.565116+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:57:23.565116+0000","last_clean_scrub_stamp":"2023-11-24T07:57:23.565116+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T15:03:20.689993+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-24T07:57:27.499424+0000","last_change":"2023-11-24T07:57:25.604514+0000","last_active":"2023-11-24T07:57:27.499424+0000","last_peered":"2023-11-24T07:57:27.499424+0000","last_clean":"2023-11-24T07:57:27.499424+0000","last_became_active":"2023-11-24T07:57:25.604243+0000","last_became_peered":"2023-11-24T07:57:25.604243+0000","last_unstale":"2023-11-24T07:57:27.499424+0000","last_undegraded":"2023-11-24T07:57:27.499424+0000","last_fullsized":"2023-11-24T07:57:27.499424+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:57:24.572681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:57:24.572681+0000","last_clean_scrub_stamp":"2023-11-24T07:57:24.572681+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T15:25:09.940051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-24T07:57:27.498339+0000","last_change":"2023-11-24T07:57:24.600360+0000","last_active":"2023-11-24T07:57:27.498339+0000","last_peered":"2023-11-24T07:57:27.498339+0000","last_clean":"2023-11-24T07:57:27.498339+0000","last_became_active":"2023-11-24T07:57:24.600069+0000","last_became_peered":"2023-11-24T07:57:24.600069+0000","last_unstale":"2023-11-24T07:57:27.498339+0000","last_undegraded":"2023-11-24T07:57:27.498339+0000","last_fullsized":"2023-11-24T07:57:27.498339+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:57:23.565116+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:57:23.565116+0000","last_clean_scrub_stamp":"2023-11-24T07:57:23.565116+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T14:43:06.906996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-24T07:57:27.498321+0000","last_change":"2023-11-24T07:57:25.604299+0000","last_active":"2023-11-24T07:57:27.498321+0000","last_peered":"2023-11-24T07:57:27.498321+0000","last_clean":"2023-11-24T07:57:27.498321+0000","last_became_active":"2023-11-24T07:57:25.604033+0000","last_became_peered":"2023-11-24T07:57:25.604033+0000","last_unstale":"2023-11-24T07:57:27.498321+0000","last_undegraded":"2023-11-24T07:57:27.498321+0000","last_fullsized":"2023-11-24T07:57:27.498321+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:57:24.572681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:57:24.572681+0000","last_clean_scrub_stamp":"2023-11-24T07:57:24.572681+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T15:47:12.125658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-24T07:57:27.498908+0000","last_change":"2023-11-24T07:57:24.592424+0000","last_active":"2023-11-24T07:57:27.498908+0000","last_peered":"2023-11-24T07:57:27.498908+0000","last_clean":"2023-11-24T07:57:27.498908+0000","last_became_active":"2023-11-24T07:57:24.592091+0000","last_became_peered":"2023-11-24T07:57:24.592091+0000","last_unstale":"2023-11-24T07:57:27.498908+0000","last_undegraded":"2023-11-24T07:57:27.498908+0000","last_fullsized":"2023-11-24T07:57:27.498908+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:57:23.565116+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:57:23.565116+0000","last_clean_scrub_stamp":"2023-11-24T07:57:23.565116+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T11:43:02.671428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-24T07:57:26.583989+0000","last_change":"2023-11-24T07:57:25.605278+0000","last_active":"2023-11-24T07:57:26.583989+0000","last_peered":"2023-11-24T07:57:26.583989+0000","last_clean":"2023-11-24T07:57:26.583989+0000","last_became_active":"2023-11-24T07:57:25.604986+0000","last_became_peered":"2023-11-24T07:57:25.604986+0000","last_unstale":"2023-11-24T07:57:26.583989+0000","last_undegraded":"2023-11-24T07:57:26.583989+0000","last_fullsized":"2023-11-24T07:57:26.583989+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:57:24.572681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:57:24.572681+0000","last_clean_scrub_stamp":"2023-11-24T07:57:24.572681+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T14:00:49.762285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-24T07:57:26.584132+0000","last_change":"2023-11-24T07:57:24.587676+0000","last_active":"2023-11-24T07:57:26.584132+0000","last_peered":"2023-11-24T07:57:26.584132+0000","last_clean":"2023-11-24T07:57:26.584132+0000","last_became_active":"2023-11-24T07:57:24.587487+0000","last_became_peered":"2023-11-24T07:57:24.587487+0000","last_unstale":"2023-11-24T07:57:26.584132+0000","last_undegraded":"2023-11-24T07:57:26.584132+0000","last_fullsized":"2023-11-24T07:57:26.584132+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:57:23.565116+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:57:23.565116+0000","last_clean_scrub_stamp":"2023-11-24T07:57:23.565116+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T09:32:53.336675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-24T07:57:27.499752+0000","last_change":"2023-11-24T07:57:25.604618+0000","last_active":"2023-11-24T07:57:27.499752+0000","last_peered":"2023-11-24T07:57:27.499752+0000","last_clean":"2023-11-24T07:57:27.499752+0000","last_became_active":"2023-11-24T07:57:25.603754+0000","last_became_peered":"2023-11-24T07:57:25.603754+0000","last_unstale":"2023-11-24T07:57:27.499752+0000","last_undegraded":"2023-11-24T07:57:27.499752+0000","last_fullsized":"2023-11-24T07:57:27.499752+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:57:24.572681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:57:24.572681+0000","last_clean_scrub_stamp":"2023-11-24T07:57:24.572681+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T09:44:29.688267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-24T07:57:26.584309+0000","last_change":"2023-11-24T07:57:24.597830+0000","last_active":"2023-11-24T07:57:26.584309+0000","last_peered":"2023-11-24T07:57:26.584309+0000","last_clean":"2023-11-24T07:57:26.584309+0000","last_became_active":"2023-11-24T07:57:24.597491+0000","last_became_peered":"2023-11-24T07:57:24.597491+0000","last_unstale":"2023-11-24T07:57:26.584309+0000","last_undegraded":"2023-11-24T07:57:26.584309+0000","last_fullsized":"2023-11-24T07:57:26.584309+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:57:23.565116+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:57:23.565116+0000","last_clean_scrub_stamp":"2023-11-24T07:57:23.565116+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T11:43:02.671428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.14","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-24T07:57:27.499713+0000","last_change":"2023-11-24T07:57:25.606580+0000","last_active":"2023-11-24T07:57:27.499713+0000","last_peered":"2023-11-24T07:57:27.499713+0000","last_clean":"2023-11-24T07:57:27.499713+0000","last_became_active":"2023-11-24T07:57:25.606003+0000","last_became_peered":"2023-11-24T07:57:25.606003+0000","last_unstale":"2023-11-24T07:57:27.499713+0000","last_undegraded":"2023-11-24T07:57:27.499713+0000","last_fullsized":"2023-11-24T07:57:27.499713+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:57:24.572681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:57:24.572681+0000","last_clean_scrub_stamp":"2023-11-24T07:57:24.572681+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T08:54:41.853999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.15","version":"32'3","reported_seq":15,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-24T07:57:26.583811+0000","last_change":"2023-11-24T07:57:24.601458+0000","last_active":"2023-11-24T07:57:26.583811+0000","last_peered":"2023-11-24T07:57:26.583811+0000","last_clean":"2023-11-24T07:57:26.583811+0000","last_became_active":"2023-11-24T07:57:24.601087+0000","last_became_peered":"2023-11-24T07:57:24.601087+0000","last_unstale":"2023-11-24T07:57:26.583811+0000","last_undegraded":"2023-11-24T07:57:26.583811+0000","last_fullsized":"2023-11-24T07:57:26.583811+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:57:23.565116+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:57:23.565116+0000","last_clean_scrub_stamp":"2023-11-24T07:57:23.565116+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T08:54:40.846433+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.15","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-24T07:57:26.583962+0000","last_change":"2023-11-24T07:57:25.610172+0000","last_active":"2023-11-24T07:57:26.583962+0000","last_peered":"2023-11-24T07:57:26.583962+0000","last_clean":"2023-11-24T07:57:26.583962+0000","last_became_active":"2023-11-24T07:57:25.609792+0000","last_became_peered":"2023-11-24T07:57:25.609792+0000","last_unstale":"2023-11-24T07:57:26.583962+0000","last_undegraded":"2023-11-24T07:57:26.583962+0000","last_fullsized":"2023-11-24T07:57:26.583962+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:57:24.572681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:57:24.572681+0000","last_clean_scrub_stamp":"2023-11-24T07:57:24.572681+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:36:54.791587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.14","version":"32'6","reported_seq":18,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-24T07:57:27.499082+0000","last_change":"2023-11-24T07:57:24.595177+0000","last_active":"2023-11-24T07:57:27.499082+0000","last_peered":"2023-11-24T07:57:27.499082+0000","last_clean":"2023-11-24T07:57:27.499082+0000","last_became_active":"2023-11-24T07:57:24.594880+0000","last_became_peered":"2023-11-24T07:57:24.594880+0000","last_unstale":"2023-11-24T07:57:27.499082+0000","last_undegraded":"2023-11-24T07:57:27.499082+0000","last_fullsized":"2023-11-24T07:57:27.499082+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:57:23.565116+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:57:23.565116+0000","last_clean_scrub_stamp":"2023-11-24T07:57:23.565116+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T16:05:14.639876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-24T07:57:27.498346+0000","last_change":"2023-11-24T07:57:25.597947+0000","last_active":"2023-11-24T07:57:27.498346+0000","last_peered":"2023-11-24T07:57:27.498346+0000","last_clean":"2023-11-24T07:57:27.498346+0000","last_became_active":"2023-11-24T07:57:25.597634+0000","last_became_peered":"2023-11-24T07:57:25.597634+0000","last_unstale":"2023-11-24T07:57:27.498346+0000","last_undegraded":"2023-11-24T07:57:27.498346+0000","last_fullsized":"2023-11-24T07:57:27.498346+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:57:24.572681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:57:24.572681+0000","last_clean_scrub_stamp":"2023-11-24T07:57:24.572681+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T14:24:57.175517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.13","version":"32'2","reported_seq":14,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-24T07:57:27.498591+0000","last_change":"2023-11-24T07:57:24.600265+0000","last_active":"2023-11-24T07:57:27.498591+0000","last_peered":"2023-11-24T07:57:27.498591+0000","last_clean":"2023-11-24T07:57:27.498591+0000","last_became_active":"2023-11-24T07:57:24.599858+0000","last_became_peered":"2023-11-24T07:57:24.599858+0000","last_unstale":"2023-11-24T07:57:27.498591+0000","last_undegraded":"2023-11-24T07:57:27.498591+0000","last_fullsized":"2023-11-24T07:57:27.498591+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:57:23.565116+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:57:23.565116+0000","last_clean_scrub_stamp":"2023-11-24T07:57:23.565116+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T14:24:56.167951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-24T07:57:27.499668+0000","last_change":"2023-11-24T07:57:25.605157+0000","last_active":"2023-11-24T07:57:27.499668+0000","last_peered":"2023-11-24T07:57:27.499668+0000","last_clean":"2023-11-24T07:57:27.499668+0000","last_became_active":"2023-11-24T07:57:25.604895+0000","last_became_peered":"2023-11-24T07:57:25.604895+0000","last_unstale":"2023-11-24T07:57:27.499668+0000","last_undegraded":"2023-11-24T07:57:27.499668+0000","last_fullsized":"2023-11-24T07:57:27.499668+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:57:24.572681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:57:24.572681+0000","last_clean_scrub_stamp":"2023-11-24T07:57:24.572681+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:34:57.222018+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-24T07:57:27.499175+0000","last_change":"2023-11-24T07:57:24.597916+0000","last_active":"2023-11-24T07:57:27.499175+0000","last_peered":"2023-11-24T07:57:27.499175+0000","last_clean":"2023-11-24T07:57:27.499175+0000","last_became_active":"2023-11-24T07:57:24.597625+0000","last_became_peered":"2023-11-24T07:57:24.597625+0000","last_unstale":"2023-11-24T07:57:27.499175+0000","last_undegraded":"2023-11-24T07:57:27.499175+0000","last_fullsized":"2023-11-24T07:57:27.499175+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:57:23.565116+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:57:23.565116+0000","last_clean_scrub_stamp":"2023-11-24T07:57:23.565116+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T15:32:37.621039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-24T07:57:27.499392+0000","last_change":"2023-11-24T07:57:25.595758+0000","last_active":"2023-11-24T07:57:27.499392+0000","last_peered":"2023-11-24T07:57:27.499392+0000","last_clean":"2023-11-24T07:57:27.499392+0000","last_became_active":"2023-11-24T07:57:25.595488+0000","last_became_peered":"2023-11-24T07:57:25.595488+0000","last_unstale":"2023-11-24T07:57:27.499392+0000","last_undegraded":"2023-11-24T07:57:27.499392+0000","last_fullsized":"2023-11-24T07:57:27.499392+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:57:24.572681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:57:24.572681+0000","last_clean_scrub_stamp":"2023-11-24T07:57:24.572681+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T13:39:16.521602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-24T07:57:27.499388+0000","last_change":"2023-11-24T07:57:24.595527+0000","last_active":"2023-11-24T07:57:27.499388+0000","last_peered":"2023-11-24T07:57:27.499388+0000","last_clean":"2023-11-24T07:57:27.499388+0000","last_became_active":"2023-11-24T07:57:24.595229+0000","last_became_peered":"2023-11-24T07:57:24.595229+0000","last_unstale":"2023-11-24T07:57:27.499388+0000","last_undegraded":"2023-11-24T07:57:27.499388+0000","last_fullsized":"2023-11-24T07:57:27.499388+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:57:23.565116+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:57:23.565116+0000","last_clean_scrub_stamp":"2023-11-24T07:57:23.565116+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T14:24:56.167951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-24T07:57:26.583629+0000","last_change":"2023-11-24T07:57:25.617331+0000","last_active":"2023-11-24T07:57:26.583629+0000","last_peered":"2023-11-24T07:57:26.583629+0000","last_clean":"2023-11-24T07:57:26.583629+0000","last_became_active":"2023-11-24T07:57:25.617033+0000","last_became_peered":"2023-11-24T07:57:25.617033+0000","last_unstale":"2023-11-24T07:57:26.583629+0000","last_undegraded":"2023-11-24T07:57:26.583629+0000","last_fullsized":"2023-11-24T07:57:26.583629+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:57:24.572681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:57:24.572681+0000","last_clean_scrub_stamp":"2023-11-24T07:57:24.572681+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T14:01:46.665356+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.10","version":"32'8","reported_seq":20,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-24T07:57:27.498079+0000","last_change":"2023-11-24T07:57:24.599748+0000","last_active":"2023-11-24T07:57:27.498079+0000","last_peered":"2023-11-24T07:57:27.498079+0000","last_clean":"2023-11-24T07:57:27.498079+0000","last_became_active":"2023-11-24T07:57:24.599456+0000","last_became_peered":"2023-11-24T07:57:24.599456+0000","last_unstale":"2023-11-24T07:57:27.498079+0000","last_undegraded":"2023-11-24T07:57:27.498079+0000","last_fullsized":"2023-11-24T07:57:27.498079+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:57:23.565116+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:57:23.565116+0000","last_clean_scrub_stamp":"2023-11-24T07:57:23.565116+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T19:40:29.710446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.e","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-24T07:57:26.584123+0000","last_change":"2023-11-24T07:57:25.610621+0000","last_active":"2023-11-24T07:57:26.584123+0000","last_peered":"2023-11-24T07:57:26.584123+0000","last_clean":"2023-11-24T07:57:26.584123+0000","last_became_active":"2023-11-24T07:57:25.610341+0000","last_became_peered":"2023-11-24T07:57:25.610341+0000","last_unstale":"2023-11-24T07:57:26.584123+0000","last_undegraded":"2023-11-24T07:57:26.584123+0000","last_fullsized":"2023-11-24T07:57:26.584123+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:57:24.572681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:57:24.572681+0000","last_clean_scrub_stamp":"2023-11-24T07:57:24.572681+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:32:46.923992+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.f","version":"32'1","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-24T07:57:27.498608+0000","last_change":"2023-11-24T07:57:24.600217+0000","last_active":"2023-11-24T07:57:27.498608+0000","last_peered":"2023-11-24T07:57:27.498608+0000","last_clean":"2023-11-24T07:57:27.498608+0000","last_became_active":"2023-11-24T07:57:24.599831+0000","last_became_peered":"2023-11-24T07:57:24.599831+0000","last_unstale":"2023-11-24T07:57:27.498608+0000","last_undegraded":"2023-11-24T07:57:27.498608+0000","last_fullsized":"2023-11-24T07:57:27.498608+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:57:23.565116+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:57:23.565116+0000","last_clean_scrub_stamp":"2023-11-24T07:57:23.565116+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T14:47:45.788163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-24T07:57:27.498862+0000","last_change":"2023-11-24T07:57:25.604503+0000","last_active":"2023-11-24T07:57:27.498862+0000","last_peered":"2023-11-24T07:57:27.498862+0000","last_clean":"2023-11-24T07:57:27.498862+0000","last_became_active":"2023-11-24T07:57:25.604215+0000","last_became_peered":"2023-11-24T07:57:25.604215+0000","last_unstale":"2023-11-24T07:57:27.498862+0000","last_undegraded":"2023-11-24T07:57:27.498862+0000","last_fullsized":"2023-11-24T07:57:27.498862+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:57:24.572681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:57:24.572681+0000","last_clean_scrub_stamp":"2023-11-24T07:57:24.572681+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T14:49:20.210209+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.e","version":"32'2","reported_seq":14,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-24T07:57:27.498400+0000","last_change":"2023-11-24T07:57:24.600226+0000","last_active":"2023-11-24T07:57:27.498400+0000","last_peered":"2023-11-24T07:57:27.498400+0000","last_clean":"2023-11-24T07:57:27.498400+0000","last_became_active":"2023-11-24T07:57:24.599943+0000","last_became_peered":"2023-11-24T07:57:24.599943+0000","last_unstale":"2023-11-24T07:57:27.498400+0000","last_undegraded":"2023-11-24T07:57:27.498400+0000","last_fullsized":"2023-11-24T07:57:27.498400+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:57:23.565116+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:57:23.565116+0000","last_clean_scrub_stamp":"2023-11-24T07:57:23.565116+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T14:33:18.564905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-24T07:57:26.583937+0000","last_change":"2023-11-24T07:57:25.598876+0000","last_active":"2023-11-24T07:57:26.583937+0000","last_peered":"2023-11-24T07:57:26.583937+0000","last_clean":"2023-11-24T07:57:26.583937+0000","last_became_active":"2023-11-24T07:57:25.598514+0000","last_became_peered":"2023-11-24T07:57:25.598514+0000","last_unstale":"2023-11-24T07:57:26.583937+0000","last_undegraded":"2023-11-24T07:57:26.583937+0000","last_fullsized":"2023-11-24T07:57:26.583937+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:57:24.572681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:57:24.572681+0000","last_clean_scrub_stamp":"2023-11-24T07:57:24.572681+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T12:51:13.357718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.d","version":"32'1","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-24T07:57:26.584012+0000","last_change":"2023-11-24T07:57:24.597442+0000","last_active":"2023-11-24T07:57:26.584012+0000","last_peered":"2023-11-24T07:57:26.584012+0000","last_clean":"2023-11-24T07:57:26.584012+0000","last_became_active":"2023-11-24T07:57:24.596992+0000","last_became_peered":"2023-11-24T07:57:24.596992+0000","last_unstale":"2023-11-24T07:57:26.584012+0000","last_undegraded":"2023-11-24T07:57:26.584012+0000","last_fullsized":"2023-11-24T07:57:26.584012+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:57:23.565116+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:57:23.565116+0000","last_clean_scrub_stamp":"2023-11-24T07:57:23.565116+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T11:43:38.072322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.d","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-24T07:57:27.498699+0000","last_change":"2023-11-24T07:57:25.608066+0000","last_active":"2023-11-24T07:57:27.498699+0000","last_peered":"2023-11-24T07:57:27.498699+0000","last_clean":"2023-11-24T07:57:27.498699+0000","last_became_active":"2023-11-24T07:57:25.607583+0000","last_became_peered":"2023-11-24T07:57:25.607583+0000","last_unstale":"2023-11-24T07:57:27.498699+0000","last_undegraded":"2023-11-24T07:57:27.498699+0000","last_fullsized":"2023-11-24T07:57:27.498699+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:57:24.572681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:57:24.572681+0000","last_clean_scrub_stamp":"2023-11-24T07:57:24.572681+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T13:05:00.624062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-24T07:57:26.582860+0000","last_change":"2023-11-24T07:57:24.588501+0000","last_active":"2023-11-24T07:57:26.582860+0000","last_peered":"2023-11-24T07:57:26.582860+0000","last_clean":"2023-11-24T07:57:26.582860+0000","last_became_active":"2023-11-24T07:57:24.588135+0000","last_became_peered":"2023-11-24T07:57:24.588135+0000","last_unstale":"2023-11-24T07:57:26.582860+0000","last_undegraded":"2023-11-24T07:57:26.582860+0000","last_fullsized":"2023-11-24T07:57:26.582860+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:57:23.565116+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:57:23.565116+0000","last_clean_scrub_stamp":"2023-11-24T07:57:23.565116+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T16:47:54.583083+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-24T07:57:27.498182+0000","last_change":"2023-11-24T07:57:25.606996+0000","last_active":"2023-11-24T07:57:27.498182+0000","last_peered":"2023-11-24T07:57:27.498182+0000","last_clean":"2023-11-24T07:57:27.498182+0000","last_became_active":"2023-11-24T07:57:25.606495+0000","last_became_peered":"2023-11-24T07:57:25.606495+0000","last_unstale":"2023-11-24T07:57:27.498182+0000","last_undegraded":"2023-11-24T07:57:27.498182+0000","last_fullsized":"2023-11-24T07:57:27.498182+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:57:24.572681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:57:24.572681+0000","last_clean_scrub_stamp":"2023-11-24T07:57:24.572681+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T16:10:48.563298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.b","version":"32'2","reported_seq":14,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-24T07:57:26.584060+0000","last_change":"2023-11-24T07:57:24.589230+0000","last_active":"2023-11-24T07:57:26.584060+0000","last_peered":"2023-11-24T07:57:26.584060+0000","last_clean":"2023-11-24T07:57:26.584060+0000","last_became_active":"2023-11-24T07:57:24.588791+0000","last_became_peered":"2023-11-24T07:57:24.588791+0000","last_unstale":"2023-11-24T07:57:26.584060+0000","last_undegraded":"2023-11-24T07:57:26.584060+0000","last_fullsized":"2023-11-24T07:57:26.584060+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:57:23.565116+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:57:23.565116+0000","last_clean_scrub_stamp":"2023-11-24T07:57:23.565116+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T12:58:54.851868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-24T07:57:26.584361+0000","last_change":"2023-11-24T07:57:25.604199+0000","last_active":"2023-11-24T07:57:26.584361+0000","last_peered":"2023-11-24T07:57:26.584361+0000","last_clean":"2023-11-24T07:57:26.584361+0000","last_became_active":"2023-11-24T07:57:25.603979+0000","last_became_peered":"2023-11-24T07:57:25.603979+0000","last_unstale":"2023-11-24T07:57:26.584361+0000","last_undegraded":"2023-11-24T07:57:26.584361+0000","last_fullsized":"2023-11-24T07:57:26.584361+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:57:24.572681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:57:24.572681+0000","last_clean_scrub_stamp":"2023-11-24T07:57:24.572681+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T14:04:03.022069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-24T07:57:27.498922+0000","last_change":"2023-11-24T07:57:24.597707+0000","last_active":"2023-11-24T07:57:27.498922+0000","last_peered":"2023-11-24T07:57:27.498922+0000","last_clean":"2023-11-24T07:57:27.498922+0000","last_became_active":"2023-11-24T07:57:24.597441+0000","last_became_peered":"2023-11-24T07:57:24.597441+0000","last_unstale":"2023-11-24T07:57:27.498922+0000","last_undegraded":"2023-11-24T07:57:27.498922+0000","last_fullsized":"2023-11-24T07:57:27.498922+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:57:23.565116+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:57:23.565116+0000","last_clean_scrub_stamp":"2023-11-24T07:57:23.565116+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T18:23:24.083250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-24T07:57:27.498774+0000","last_change":"2023-11-24T07:57:25.609619+0000","last_active":"2023-11-24T07:57:27.498774+0000","last_peered":"2023-11-24T07:57:27.498774+0000","last_clean":"2023-11-24T07:57:27.498774+0000","last_became_active":"2023-11-24T07:57:25.609261+0000","last_became_peered":"2023-11-24T07:57:25.609261+0000","last_unstale":"2023-11-24T07:57:27.498774+0000","last_undegraded":"2023-11-24T07:57:27.498774+0000","last_fullsized":"2023-11-24T07:57:27.498774+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:57:24.572681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:57:24.572681+0000","last_clean_scrub_stamp":"2023-11-24T07:57:24.572681+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T11:43:03.678993+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-24T07:57:27.498617+0000","last_change":"2023-11-24T07:57:24.589176+0000","last_active":"2023-11-24T07:57:27.498617+0000","last_peered":"2023-11-24T07:57:27.498617+0000","last_clean":"2023-11-24T07:57:27.498617+0000","last_became_active":"2023-11-24T07:57:24.588886+0000","last_became_peered":"2023-11-24T07:57:24.588886+0000","last_unstale":"2023-11-24T07:57:27.498617+0000","last_undegraded":"2023-11-24T07:57:27.498617+0000","last_fullsized":"2023-11-24T07:57:27.498617+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:57:23.565116+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:57:23.565116+0000","last_clean_scrub_stamp":"2023-11-24T07:57:23.565116+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T16:28:17.765064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-24T07:57:27.498976+0000","last_change":"2023-11-24T07:57:25.607219+0000","last_active":"2023-11-24T07:57:27.498976+0000","last_peered":"2023-11-24T07:57:27.498976+0000","last_clean":"2023-11-24T07:57:27.498976+0000","last_became_active":"2023-11-24T07:57:25.606930+0000","last_became_peered":"2023-11-24T07:57:25.606930+0000","last_unstale":"2023-11-24T07:57:27.498976+0000","last_undegraded":"2023-11-24T07:57:27.498976+0000","last_fullsized":"2023-11-24T07:57:27.498976+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:57:24.572681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:57:24.572681+0000","last_clean_scrub_stamp":"2023-11-24T07:57:24.572681+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T13:05:00.624062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-24T07:57:26.583579+0000","last_change":"2023-11-24T07:57:24.597642+0000","last_active":"2023-11-24T07:57:26.583579+0000","last_peered":"2023-11-24T07:57:26.583579+0000","last_clean":"2023-11-24T07:57:26.583579+0000","last_became_active":"2023-11-24T07:57:24.597339+0000","last_became_peered":"2023-11-24T07:57:24.597339+0000","last_unstale":"2023-11-24T07:57:26.583579+0000","last_undegraded":"2023-11-24T07:57:26.583579+0000","last_fullsized":"2023-11-24T07:57:26.583579+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:57:23.565116+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:57:23.565116+0000","last_clean_scrub_stamp":"2023-11-24T07:57:23.565116+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T15:35:45.139676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-24T07:57:27.498618+0000","last_change":"2023-11-24T07:57:25.609017+0000","last_active":"2023-11-24T07:57:27.498618+0000","last_peered":"2023-11-24T07:57:27.498618+0000","last_clean":"2023-11-24T07:57:27.498618+0000","last_became_active":"2023-11-24T07:57:25.608616+0000","last_became_peered":"2023-11-24T07:57:25.608616+0000","last_unstale":"2023-11-24T07:57:27.498618+0000","last_undegraded":"2023-11-24T07:57:27.498618+0000","last_fullsized":"2023-11-24T07:57:27.498618+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:57:24.572681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:57:24.572681+0000","last_clean_scrub_stamp":"2023-11-24T07:57:24.572681+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T15:54:26.161043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.7","version":"32'2","reported_seq":14,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-24T07:57:27.498697+0000","last_change":"2023-11-24T07:57:24.596645+0000","last_active":"2023-11-24T07:57:27.498697+0000","last_peered":"2023-11-24T07:57:27.498697+0000","last_clean":"2023-11-24T07:57:27.498697+0000","last_became_active":"2023-11-24T07:57:24.596327+0000","last_became_peered":"2023-11-24T07:57:24.596327+0000","last_unstale":"2023-11-24T07:57:27.498697+0000","last_undegraded":"2023-11-24T07:57:27.498697+0000","last_fullsized":"2023-11-24T07:57:27.498697+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:57:23.565116+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:57:23.565116+0000","last_clean_scrub_stamp":"2023-11-24T07:57:23.565116+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T14:49:19.202644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-24T07:57:26.583923+0000","last_change":"2023-11-24T07:57:25.612875+0000","last_active":"2023-11-24T07:57:26.583923+0000","last_peered":"2023-11-24T07:57:26.583923+0000","last_clean":"2023-11-24T07:57:26.583923+0000","last_became_active":"2023-11-24T07:57:25.612599+0000","last_became_peered":"2023-11-24T07:57:25.612599+0000","last_unstale":"2023-11-24T07:57:26.583923+0000","last_undegraded":"2023-11-24T07:57:26.583923+0000","last_fullsized":"2023-11-24T07:57:26.583923+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:57:24.572681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:57:24.572681+0000","last_clean_scrub_stamp":"2023-11-24T07:57:24.572681+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T13:42:15.906045+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.6","version":"33'1","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-24T07:57:27.619117+0000","last_change":"2023-11-24T07:57:24.600554+0000","last_active":"2023-11-24T07:57:27.619117+0000","last_peered":"2023-11-24T07:57:27.619117+0000","last_clean":"2023-11-24T07:57:27.619117+0000","last_became_active":"2023-11-24T07:57:24.600257+0000","last_became_peered":"2023-11-24T07:57:24.600257+0000","last_unstale":"2023-11-24T07:57:27.619117+0000","last_undegraded":"2023-11-24T07:57:27.619117+0000","last_fullsized":"2023-11-24T07:57:27.619117+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:57:23.565116+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:57:23.565116+0000","last_clean_scrub_stamp":"2023-11-24T07:57:23.565116+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T16:47:54.583083+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-24T07:57:27.498903+0000","last_change":"2023-11-24T07:57:25.604810+0000","last_active":"2023-11-24T07:57:27.498903+0000","last_peered":"2023-11-24T07:57:27.498903+0000","last_clean":"2023-11-24T07:57:27.498903+0000","last_became_active":"2023-11-24T07:57:25.604557+0000","last_became_peered":"2023-11-24T07:57:25.604557+0000","last_unstale":"2023-11-24T07:57:27.498903+0000","last_undegraded":"2023-11-24T07:57:27.498903+0000","last_fullsized":"2023-11-24T07:57:27.498903+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:57:24.572681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:57:24.572681+0000","last_clean_scrub_stamp":"2023-11-24T07:57:24.572681+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T12:14:04.002318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-24T07:57:27.499172+0000","last_change":"2023-11-24T07:57:24.597887+0000","last_active":"2023-11-24T07:57:27.499172+0000","last_peered":"2023-11-24T07:57:27.499172+0000","last_clean":"2023-11-24T07:57:27.499172+0000","last_became_active":"2023-11-24T07:57:24.597549+0000","last_became_peered":"2023-11-24T07:57:24.597549+0000","last_unstale":"2023-11-24T07:57:27.499172+0000","last_undegraded":"2023-11-24T07:57:27.499172+0000","last_fullsized":"2023-11-24T07:57:27.499172+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:57:23.565116+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:57:23.565116+0000","last_clean_scrub_stamp":"2023-11-24T07:57:23.565116+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:34:29.085812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-24T07:57:26.582793+0000","last_change":"2023-11-24T07:57:25.606098+0000","last_active":"2023-11-24T07:57:26.582793+0000","last_peered":"2023-11-24T07:57:26.582793+0000","last_clean":"2023-11-24T07:57:26.582793+0000","last_became_active":"2023-11-24T07:57:25.603917+0000","last_became_peered":"2023-11-24T07:57:25.603917+0000","last_unstale":"2023-11-24T07:57:26.582793+0000","last_undegraded":"2023-11-24T07:57:26.582793+0000","last_fullsized":"2023-11-24T07:57:26.582793+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:57:24.572681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:57:24.572681+0000","last_clean_scrub_stamp":"2023-11-24T07:57:24.572681+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T19:13:40.799042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-24T07:57:27.498955+0000","last_change":"2023-11-24T07:57:24.596194+0000","last_active":"2023-11-24T07:57:27.498955+0000","last_peered":"2023-11-24T07:57:27.498955+0000","last_clean":"2023-11-24T07:57:27.498955+0000","last_became_active":"2023-11-24T07:57:24.595854+0000","last_became_peered":"2023-11-24T07:57:24.595854+0000","last_unstale":"2023-11-24T07:57:27.498955+0000","last_undegraded":"2023-11-24T07:57:27.498955+0000","last_fullsized":"2023-11-24T07:57:27.498955+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:57:23.565116+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:57:23.565116+0000","last_clean_scrub_stamp":"2023-11-24T07:57:23.565116+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T12:14:02.994752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-24T07:57:27.499699+0000","last_change":"2023-11-24T07:57:25.604191+0000","last_active":"2023-11-24T07:57:27.499699+0000","last_peered":"2023-11-24T07:57:27.499699+0000","last_clean":"2023-11-24T07:57:27.499699+0000","last_became_active":"2023-11-24T07:57:25.603570+0000","last_became_peered":"2023-11-24T07:57:25.603570+0000","last_unstale":"2023-11-24T07:57:27.499699+0000","last_undegraded":"2023-11-24T07:57:27.499699+0000","last_fullsized":"2023-11-24T07:57:27.499699+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:57:24.572681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:57:24.572681+0000","last_clean_scrub_stamp":"2023-11-24T07:57:24.572681+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T11:48:20.813946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-24T07:57:27.498769+0000","last_change":"2023-11-24T07:57:24.599222+0000","last_active":"2023-11-24T07:57:27.498769+0000","last_peered":"2023-11-24T07:57:27.498769+0000","last_clean":"2023-11-24T07:57:27.498769+0000","last_became_active":"2023-11-24T07:57:24.598910+0000","last_became_peered":"2023-11-24T07:57:24.598910+0000","last_unstale":"2023-11-24T07:57:27.498769+0000","last_undegraded":"2023-11-24T07:57:27.498769+0000","last_fullsized":"2023-11-24T07:57:27.498769+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:57:23.565116+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:57:23.565116+0000","last_clean_scrub_stamp":"2023-11-24T07:57:23.565116+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T15:03:20.689993+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-24T07:57:27.498551+0000","last_change":"2023-11-24T07:57:25.603783+0000","last_active":"2023-11-24T07:57:27.498551+0000","last_peered":"2023-11-24T07:57:27.498551+0000","last_clean":"2023-11-24T07:57:27.498551+0000","last_became_active":"2023-11-24T07:57:25.603464+0000","last_became_peered":"2023-11-24T07:57:25.603464+0000","last_unstale":"2023-11-24T07:57:27.498551+0000","last_undegraded":"2023-11-24T07:57:27.498551+0000","last_fullsized":"2023-11-24T07:57:27.498551+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:57:24.572681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:57:24.572681+0000","last_clean_scrub_stamp":"2023-11-24T07:57:24.572681+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T14:04:03.022069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-24T07:57:27.499235+0000","last_change":"2023-11-24T07:57:24.599854+0000","last_active":"2023-11-24T07:57:27.499235+0000","last_peered":"2023-11-24T07:57:27.499235+0000","last_clean":"2023-11-24T07:57:27.499235+0000","last_became_active":"2023-11-24T07:57:24.599310+0000","last_became_peered":"2023-11-24T07:57:24.599310+0000","last_unstale":"2023-11-24T07:57:27.499235+0000","last_undegraded":"2023-11-24T07:57:27.499235+0000","last_fullsized":"2023-11-24T07:57:27.499235+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:57:23.565116+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:57:23.565116+0000","last_clean_scrub_stamp":"2023-11-24T07:57:23.565116+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:34:29.085812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-24T07:57:26.583711+0000","last_change":"2023-11-24T07:57:25.613275+0000","last_active":"2023-11-24T07:57:26.583711+0000","last_peered":"2023-11-24T07:57:26.583711+0000","last_clean":"2023-11-24T07:57:26.583711+0000","last_became_active":"2023-11-24T07:57:25.612934+0000","last_became_peered":"2023-11-24T07:57:25.612934+0000","last_unstale":"2023-11-24T07:57:26.583711+0000","last_undegraded":"2023-11-24T07:57:26.583711+0000","last_fullsized":"2023-11-24T07:57:26.583711+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:57:24.572681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:57:24.572681+0000","last_clean_scrub_stamp":"2023-11-24T07:57:24.572681+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T19:14:43.303209+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-24T07:57:27.498639+0000","last_change":"2023-11-24T07:57:24.597190+0000","last_active":"2023-11-24T07:57:27.498639+0000","last_peered":"2023-11-24T07:57:27.498639+0000","last_clean":"2023-11-24T07:57:27.498639+0000","last_became_active":"2023-11-24T07:57:24.596910+0000","last_became_peered":"2023-11-24T07:57:24.596910+0000","last_unstale":"2023-11-24T07:57:27.498639+0000","last_undegraded":"2023-11-24T07:57:27.498639+0000","last_fullsized":"2023-11-24T07:57:27.498639+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:57:23.565116+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:57:23.565116+0000","last_clean_scrub_stamp":"2023-11-24T07:57:23.565116+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T12:58:54.851868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-24T07:57:26.582741+0000","last_change":"2023-11-24T07:57:25.592562+0000","last_active":"2023-11-24T07:57:26.582741+0000","last_peered":"2023-11-24T07:57:26.582741+0000","last_clean":"2023-11-24T07:57:26.582741+0000","last_became_active":"2023-11-24T07:57:25.592287+0000","last_became_peered":"2023-11-24T07:57:25.592287+0000","last_unstale":"2023-11-24T07:57:26.582741+0000","last_undegraded":"2023-11-24T07:57:26.582741+0000","last_fullsized":"2023-11-24T07:57:26.582741+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:57:24.572681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:57:24.572681+0000","last_clean_scrub_stamp":"2023-11-24T07:57:24.572681+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T09:32:54.344241+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-24T07:57:26.583911+0000","last_change":"2023-11-24T07:57:24.599399+0000","last_active":"2023-11-24T07:57:26.583911+0000","last_peered":"2023-11-24T07:57:26.583911+0000","last_clean":"2023-11-24T07:57:26.583911+0000","last_became_active":"2023-11-24T07:57:24.599096+0000","last_became_peered":"2023-11-24T07:57:24.599096+0000","last_unstale":"2023-11-24T07:57:26.583911+0000","last_undegraded":"2023-11-24T07:57:26.583911+0000","last_fullsized":"2023-11-24T07:57:26.583911+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:57:23.565116+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:57:23.565116+0000","last_clean_scrub_stamp":"2023-11-24T07:57:23.565116+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T18:02:41.581821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":176,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-11-24T07:57:25.591791+0000","last_change":"2023-11-24T07:56:19.668905+0000","last_active":"2023-11-24T07:57:25.591791+0000","last_peered":"2023-11-24T07:57:25.591791+0000","last_clean":"2023-11-24T07:57:25.591791+0000","last_became_active":"2023-11-24T07:56:19.668601+0000","last_became_peered":"2023-11-24T07:56:19.668601+0000","last_unstale":"2023-11-24T07:57:25.591791+0000","last_undegraded":"2023-11-24T07:57:25.591791+0000","last_fullsized":"2023-11-24T07:57:25.591791+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:56:18.657844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:56:18.657844+0000","last_clean_scrub_stamp":"2023-11-24T07:56:18.657844+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T11:51:53.828455+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":5,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574865,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27396,"kb_used_data":368,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344444,"statfs":{"total":96636764160,"available":96608710656,"internally_reserved":0,"allocated":376832,"data_stored":146120,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 07:57:19 2023","interfaces":[{"interface":"back","average":{"1min":0.55700000000000005,"5min":0.55700000000000005,"15min":0.55700000000000005},"min":{"1min":0.32800000000000001,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":1.413,"5min":1.413,"15min":1.413},"last":0.78500000000000003},{"interface":"front","average":{"1min":0.55400000000000005,"5min":0.55400000000000005,"15min":0.55400000000000005},"min":{"1min":0.41099999999999998,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":0.81000000000000005,"5min":0.81000000000000005,"15min":0.81000000000000005},"last":0.77000000000000002}]},{"osd":1,"last update":"Fri Nov 24 07:57:19 2023","interfaces":[{"interface":"back","average":{"1min":0.84899999999999998,"5min":0.84899999999999998,"15min":0.84899999999999998},"min":{"1min":0.33600000000000002,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":6.1070000000000002,"5min":6.1070000000000002,"15min":6.1070000000000002},"last":0.56999999999999995},{"interface":"front","average":{"1min":0.57999999999999996,"5min":0.57999999999999996,"15min":0.57999999999999996},"min":{"1min":0.35199999999999998,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":1.5920000000000001,"5min":1.5920000000000001,"15min":1.5920000000000001},"last":0.73599999999999999}]},{"osd":2,"last update":"Fri Nov 24 07:57:19 2023","interfaces":[{"interface":"back","average":{"1min":0.627,"5min":0.627,"15min":0.627},"min":{"1min":0.33800000000000002,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":1.613,"5min":1.613,"15min":1.613},"last":0.59299999999999997},{"interface":"front","average":{"1min":0.79200000000000004,"5min":0.79200000000000004,"15min":0.79200000000000004},"min":{"1min":0.41699999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":4.5220000000000002,"5min":4.5220000000000002,"15min":4.5220000000000002},"last":0.71299999999999997}]},{"osd":3,"last update":"Fri Nov 24 07:57:19 2023","interfaces":[{"interface":"back","average":{"1min":0.80600000000000005,"5min":0.80600000000000005,"15min":0.80600000000000005},"min":{"1min":0.38500000000000001,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":4.6760000000000002,"5min":4.6760000000000002,"15min":4.6760000000000002},"last":0.754},{"interface":"front","average":{"1min":0.94299999999999995,"5min":0.94299999999999995,"15min":0.94299999999999995},"min":{"1min":0.505,"5min":0.505,"15min":0.505},"max":{"1min":4.6539999999999999,"5min":4.6539999999999999,"15min":4.6539999999999999},"last":0.81599999999999995}]},{"osd":4,"last update":"Fri Nov 24 07:57:19 2023","interfaces":[{"interface":"back","average":{"1min":0.56799999999999995,"5min":0.56799999999999995,"15min":0.56799999999999995},"min":{"1min":0.378,"5min":0.378,"15min":0.378},"max":{"1min":0.84699999999999998,"5min":0.84699999999999998,"15min":0.84699999999999998},"last":0.44900000000000001},{"interface":"front","average":{"1min":0.66700000000000004,"5min":0.66700000000000004,"15min":0.66700000000000004},"min":{"1min":0.46100000000000002,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":1.629,"5min":1.629,"15min":1.629},"last":0.48699999999999999}]},{"osd":5,"last update":"Fri Nov 24 07:57:19 2023","interfaces":[{"interface":"back","average":{"1min":0.54700000000000004,"5min":0.54700000000000004,"15min":0.54700000000000004},"min":{"1min":0.34499999999999997,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.875,"5min":0.875,"15min":0.875},"last":0.46899999999999997},{"interface":"front","average":{"1min":0.623,"5min":0.623,"15min":0.623},"min":{"1min":0.41499999999999998,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":0.90900000000000003,"5min":0.90900000000000003,"15min":0.90900000000000003},"last":0.52500000000000002}]},{"osd":6,"last update":"Fri Nov 24 07:57:19 2023","interfaces":[{"interface":"back","average":{"1min":0.61099999999999999,"5min":0.61099999999999999,"15min":0.61099999999999999},"min":{"1min":0.438,"5min":0.438,"15min":0.438},"max":{"1min":0.80800000000000005,"5min":0.80800000000000005,"15min":0.80800000000000005},"last":0.50800000000000001},{"interface":"front","average":{"1min":0.67000000000000004,"5min":0.67000000000000004,"15min":0.67000000000000004},"min":{"1min":0.48599999999999999,"5min":0.48599999999999999,"15min":0.48599999999999999},"max":{"1min":0.98399999999999999,"5min":0.98399999999999999,"15min":0.98399999999999999},"last":0.67800000000000005}]}]},{"osd":6,"up_from":13,"seq":55834574866,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27396,"kb_used_data":376,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344444,"statfs":{"total":96636764160,"available":96608710656,"internally_reserved":0,"allocated":385024,"data_stored":147710,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 07:57:18 2023","interfaces":[{"interface":"back","average":{"1min":0.55000000000000004,"5min":0.55000000000000004,"15min":0.55000000000000004},"min":{"1min":0.28399999999999997,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":0.85099999999999998,"5min":0.85099999999999998,"15min":0.85099999999999998},"last":0.29799999999999999},{"interface":"front","average":{"1min":0.51600000000000001,"5min":0.51600000000000001,"15min":0.51600000000000001},"min":{"1min":0.311,"5min":0.311,"15min":0.311},"max":{"1min":0.69999999999999996,"5min":0.69999999999999996,"15min":0.69999999999999996},"last":0.32600000000000001}]},{"osd":1,"last update":"Fri Nov 24 07:57:18 2023","interfaces":[{"interface":"back","average":{"1min":0.51200000000000001,"5min":0.51200000000000001,"15min":0.51200000000000001},"min":{"1min":0.33800000000000002,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.78200000000000003,"5min":0.78200000000000003,"15min":0.78200000000000003},"last":0.35599999999999998},{"interface":"front","average":{"1min":0.61299999999999999,"5min":0.61299999999999999,"15min":0.61299999999999999},"min":{"1min":0.44700000000000001,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":1.0069999999999999,"5min":1.0069999999999999,"15min":1.0069999999999999},"last":0.48199999999999998}]},{"osd":2,"last update":"Fri Nov 24 07:57:18 2023","interfaces":[{"interface":"back","average":{"1min":0.57499999999999996,"5min":0.57499999999999996,"15min":0.57499999999999996},"min":{"1min":0.34300000000000003,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.878,"5min":0.878,"15min":0.878},"last":0.46500000000000002},{"interface":"front","average":{"1min":0.59299999999999997,"5min":0.59299999999999997,"15min":0.59299999999999997},"min":{"1min":0.47099999999999997,"5min":0.47099999999999997,"15min":0.47099999999999997},"max":{"1min":0.82399999999999995,"5min":0.82399999999999995,"15min":0.82399999999999995},"last":0.439}]},{"osd":3,"last update":"Fri Nov 24 07:57:18 2023","interfaces":[{"interface":"back","average":{"1min":0.56799999999999995,"5min":0.56799999999999995,"15min":0.56799999999999995},"min":{"1min":0.40000000000000002,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":0.876,"5min":0.876,"15min":0.876},"last":0.41899999999999998},{"interface":"front","average":{"1min":0.55600000000000005,"5min":0.55600000000000005,"15min":0.55600000000000005},"min":{"1min":0.379,"5min":0.379,"15min":0.379},"max":{"1min":0.68799999999999994,"5min":0.68799999999999994,"15min":0.68799999999999994},"last":0.52000000000000002}]},{"osd":4,"last update":"Fri Nov 24 07:57:18 2023","interfaces":[{"interface":"back","average":{"1min":0.51700000000000002,"5min":0.51700000000000002,"15min":0.51700000000000002},"min":{"1min":0.33000000000000002,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.70599999999999996,"5min":0.70599999999999996,"15min":0.70599999999999996},"last":0.505},{"interface":"front","average":{"1min":0.60199999999999998,"5min":0.60199999999999998,"15min":0.60199999999999998},"min":{"1min":0.35199999999999998,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":1.087,"5min":1.087,"15min":1.087},"last":0.53300000000000003}]},{"osd":5,"last update":"Fri Nov 24 07:57:18 2023","interfaces":[{"interface":"back","average":{"1min":0.53100000000000003,"5min":0.53100000000000003,"15min":0.53100000000000003},"min":{"1min":0.30199999999999999,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.84999999999999998,"5min":0.84999999999999998,"15min":0.84999999999999998},"last":0.54800000000000004},{"interface":"front","average":{"1min":0.61199999999999999,"5min":0.61199999999999999,"15min":0.61199999999999999},"min":{"1min":0.39200000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.97199999999999998,"5min":0.97199999999999998,"15min":0.97199999999999998},"last":0.55800000000000005}]},{"osd":7,"last update":"Fri Nov 24 07:57:18 2023","interfaces":[{"interface":"back","average":{"1min":0.61799999999999999,"5min":0.61799999999999999,"15min":0.61799999999999999},"min":{"1min":0.49299999999999999,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":0.81499999999999995,"5min":0.81499999999999995,"15min":0.81499999999999995},"last":0.57599999999999996},{"interface":"front","average":{"1min":0.67000000000000004,"5min":0.67000000000000004,"15min":0.67000000000000004},"min":{"1min":0.497,"5min":0.497,"15min":0.497},"max":{"1min":0.92700000000000005,"5min":0.92700000000000005,"15min":0.92700000000000005},"last":0.621}]}]},{"osd":1,"up_from":13,"seq":55834574867,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27968,"kb_used_data":948,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343872,"statfs":{"total":96636764160,"available":96608124928,"internally_reserved":0,"allocated":970752,"data_stored":736536,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 07:57:22 2023","interfaces":[{"interface":"back","average":{"1min":0.36399999999999999,"5min":0.36399999999999999,"15min":0.36399999999999999},"min":{"1min":0.20200000000000001,"5min":0.20200000000000001,"15min":0.20200000000000001},"max":{"1min":0.70899999999999996,"5min":0.70899999999999996,"15min":0.70899999999999996},"last":0.40799999999999997},{"interface":"front","average":{"1min":0.39800000000000002,"5min":0.39800000000000002,"15min":0.39800000000000002},"min":{"1min":0.214,"5min":0.214,"15min":0.214},"max":{"1min":0.82499999999999996,"5min":0.82499999999999996,"15min":0.82499999999999996},"last":0.39300000000000002}]},{"osd":2,"last update":"Fri Nov 24 07:57:22 2023","interfaces":[{"interface":"back","average":{"1min":0.373,"5min":0.373,"15min":0.373},"min":{"1min":0.22700000000000001,"5min":0.22700000000000001,"15min":0.22700000000000001},"max":{"1min":0.60999999999999999,"5min":0.60999999999999999,"15min":0.60999999999999999},"last":0.375},{"interface":"front","average":{"1min":0.41299999999999998,"5min":0.41299999999999998,"15min":0.41299999999999998},"min":{"1min":0.26800000000000002,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":0.621,"5min":0.621,"15min":0.621},"last":0.33000000000000002}]},{"osd":3,"last update":"Fri Nov 24 07:57:22 2023","interfaces":[{"interface":"back","average":{"1min":0.44400000000000001,"5min":0.44400000000000001,"15min":0.44400000000000001},"min":{"1min":0.32100000000000001,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.66500000000000004,"5min":0.66500000000000004,"15min":0.66500000000000004},"last":0.438},{"interface":"front","average":{"1min":0.42799999999999999,"5min":0.42799999999999999,"15min":0.42799999999999999},"min":{"1min":0.28499999999999998,"5min":0.28499999999999998,"15min":0.28499999999999998},"max":{"1min":0.76300000000000001,"5min":0.76300000000000001,"15min":0.76300000000000001},"last":0.42799999999999999}]},{"osd":4,"last update":"Fri Nov 24 07:57:22 2023","interfaces":[{"interface":"back","average":{"1min":0.61099999999999999,"5min":0.61099999999999999,"15min":0.61099999999999999},"min":{"1min":0.43099999999999999,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":0.79800000000000004,"5min":0.79800000000000004,"15min":0.79800000000000004},"last":0.63400000000000001},{"interface":"front","average":{"1min":0.60599999999999998,"5min":0.60599999999999998,"15min":0.60599999999999998},"min":{"1min":0.41999999999999998,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":0.82199999999999995,"5min":0.82199999999999995,"15min":0.82199999999999995},"last":0.70899999999999996}]},{"osd":5,"last update":"Fri Nov 24 07:57:22 2023","interfaces":[{"interface":"back","average":{"1min":0.72499999999999998,"5min":0.72499999999999998,"15min":0.72499999999999998},"min":{"1min":0.37,"5min":0.37,"15min":0.37},"max":{"1min":2.7269999999999999,"5min":2.7269999999999999,"15min":2.7269999999999999},"last":0.48599999999999999},{"interface":"front","average":{"1min":0.64500000000000002,"5min":0.64500000000000002,"15min":0.64500000000000002},"min":{"1min":0.44700000000000001,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":0.84199999999999997,"5min":0.84199999999999997,"15min":0.84199999999999997},"last":0.59399999999999997}]},{"osd":6,"last update":"Fri Nov 24 07:57:22 2023","interfaces":[{"interface":"back","average":{"1min":0.65800000000000003,"5min":0.65800000000000003,"15min":0.65800000000000003},"min":{"1min":0.47499999999999998,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":0.98199999999999998,"5min":0.98199999999999998,"15min":0.98199999999999998},"last":0.67700000000000005},{"interface":"front","average":{"1min":0.61699999999999999,"5min":0.61699999999999999,"15min":0.61699999999999999},"min":{"1min":0.42899999999999999,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":0.90700000000000003,"5min":0.90700000000000003,"15min":0.90700000000000003},"last":0.51900000000000002}]},{"osd":7,"last update":"Fri Nov 24 07:57:22 2023","interfaces":[{"interface":"back","average":{"1min":0.63700000000000001,"5min":0.63700000000000001,"15min":0.63700000000000001},"min":{"1min":0.40999999999999998,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":1.0649999999999999,"5min":1.0649999999999999,"15min":1.0649999999999999},"last":0.66100000000000003},{"interface":"front","average":{"1min":0.67000000000000004,"5min":0.67000000000000004,"15min":0.67000000000000004},"min":{"1min":0.442,"5min":0.442,"15min":0.442},"max":{"1min":0.88800000000000001,"5min":0.88800000000000001,"15min":0.88800000000000001},"last":0.54300000000000004}]}]},{"osd":0,"up_from":13,"seq":55834574867,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27396,"kb_used_data":368,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344444,"statfs":{"total":96636764160,"available":96608710656,"internally_reserved":0,"allocated":376832,"data_stored":146188,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Nov 24 07:57:18 2023","interfaces":[{"interface":"back","average":{"1min":0.71299999999999997,"5min":0.71299999999999997,"15min":0.71299999999999997},"min":{"1min":0.23499999999999999,"5min":0.23499999999999999,"15min":0.23499999999999999},"max":{"1min":6.3550000000000004,"5min":6.3550000000000004,"15min":6.3550000000000004},"last":0.625},{"interface":"front","average":{"1min":1.7290000000000001,"5min":1.7290000000000001,"15min":1.7290000000000001},"min":{"1min":0.30599999999999999,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":22.916,"5min":22.916,"15min":22.916},"last":0.58699999999999997}]},{"osd":2,"last update":"Fri Nov 24 07:57:18 2023","interfaces":[{"interface":"back","average":{"1min":0.69999999999999996,"5min":0.69999999999999996,"15min":0.69999999999999996},"min":{"1min":0.22800000000000001,"5min":0.22800000000000001,"15min":0.22800000000000001},"max":{"1min":6.3899999999999997,"5min":6.3899999999999997,"15min":6.3899999999999997},"last":0.435},{"interface":"front","average":{"1min":1.0629999999999999,"5min":1.0629999999999999,"15min":1.0629999999999999},"min":{"1min":0.26800000000000002,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":11.93,"5min":11.93,"15min":11.93},"last":0.55300000000000005}]},{"osd":3,"last update":"Fri Nov 24 07:57:18 2023","interfaces":[{"interface":"back","average":{"1min":0.41999999999999998,"5min":0.41999999999999998,"15min":0.41999999999999998},"min":{"1min":0.28399999999999997,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":1.155,"5min":1.155,"15min":1.155},"last":0.79300000000000004},{"interface":"front","average":{"1min":1.127,"5min":1.127,"15min":1.127},"min":{"1min":0.32300000000000001,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":12.116,"5min":12.116,"15min":12.116},"last":0.69699999999999995}]},{"osd":4,"last update":"Fri Nov 24 07:57:18 2023","interfaces":[{"interface":"back","average":{"1min":0.64800000000000002,"5min":0.64800000000000002,"15min":0.64800000000000002},"min":{"1min":0.40500000000000003,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":0.97899999999999998,"5min":0.97899999999999998,"15min":0.97899999999999998},"last":1.044},{"interface":"front","average":{"1min":1.9339999999999999,"5min":1.9339999999999999,"15min":1.9339999999999999},"min":{"1min":0.44400000000000001,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":22.963999999999999,"5min":22.963999999999999,"15min":22.963999999999999},"last":0.90800000000000003}]},{"osd":5,"last update":"Fri Nov 24 07:57:18 2023","interfaces":[{"interface":"back","average":{"1min":1.2929999999999999,"5min":1.2929999999999999,"15min":1.2929999999999999},"min":{"1min":0.40200000000000002,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":12.084,"5min":12.084,"15min":12.084},"last":0.97799999999999998},{"interface":"front","average":{"1min":0.67800000000000005,"5min":0.67800000000000005,"15min":0.67800000000000005},"min":{"1min":0.45200000000000001,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":1.073,"5min":1.073,"15min":1.073},"last":0.95499999999999996}]},{"osd":6,"last update":"Fri Nov 24 07:57:18 2023","interfaces":[{"interface":"back","average":{"1min":0.69099999999999995,"5min":0.69099999999999995,"15min":0.69099999999999995},"min":{"1min":0.42799999999999999,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":1.0580000000000001,"5min":1.0580000000000001,"15min":1.0580000000000001},"last":0.82499999999999996},{"interface":"front","average":{"1min":1.3140000000000001,"5min":1.3140000000000001,"15min":1.3140000000000001},"min":{"1min":0.41899999999999998,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":12.045,"5min":12.045,"15min":12.045},"last":0.88700000000000001}]},{"osd":7,"last update":"Fri Nov 24 07:57:18 2023","interfaces":[{"interface":"back","average":{"1min":0.65600000000000003,"5min":0.65600000000000003,"15min":0.65600000000000003},"min":{"1min":0.45400000000000001,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":1.038,"5min":1.038,"15min":1.038},"last":0.747},{"interface":"front","average":{"1min":1.9630000000000001,"5min":1.9630000000000001,"15min":1.9630000000000001},"min":{"1min":0.49099999999999999,"5min":0.49099999999999999,"15min":0.49099999999999999},"max":{"1min":22.995000000000001,"5min":22.995000000000001,"15min":22.995000000000001},"last":1.0069999999999999}]}]},{"osd":2,"up_from":13,"seq":55834574867,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27396,"kb_used_data":368,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344444,"statfs":{"total":96636764160,"available":96608710656,"internally_reserved":0,"allocated":376832,"data_stored":146132,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 07:57:17 2023","interfaces":[{"interface":"back","average":{"1min":0.371,"5min":0.371,"15min":0.371},"min":{"1min":0.22500000000000001,"5min":0.22500000000000001,"15min":0.22500000000000001},"max":{"1min":0.58099999999999996,"5min":0.58099999999999996,"15min":0.58099999999999996},"last":0.38400000000000001},{"interface":"front","average":{"1min":0.40200000000000002,"5min":0.40200000000000002,"15min":0.40200000000000002},"min":{"1min":0.27400000000000002,"5min":0.27400000000000002,"15min":0.27400000000000002},"max":{"1min":0.54100000000000004,"5min":0.54100000000000004,"15min":0.54100000000000004},"last":0.33500000000000002}]},{"osd":1,"last update":"Fri Nov 24 07:57:17 2023","interfaces":[{"interface":"back","average":{"1min":0.378,"5min":0.378,"15min":0.378},"min":{"1min":0.23499999999999999,"5min":0.23499999999999999,"15min":0.23499999999999999},"max":{"1min":0.59199999999999997,"5min":0.59199999999999997,"15min":0.59199999999999997},"last":0.31},{"interface":"front","average":{"1min":0.41599999999999998,"5min":0.41599999999999998,"15min":0.41599999999999998},"min":{"1min":0.25600000000000001,"5min":0.25600000000000001,"15min":0.25600000000000001},"max":{"1min":0.627,"5min":0.627,"15min":0.627},"last":0.28299999999999997}]},{"osd":3,"last update":"Fri Nov 24 07:57:17 2023","interfaces":[{"interface":"back","average":{"1min":0.41099999999999998,"5min":0.41099999999999998,"15min":0.41099999999999998},"min":{"1min":0.308,"5min":0.308,"15min":0.308},"max":{"1min":0.53300000000000003,"5min":0.53300000000000003,"15min":0.53300000000000003},"last":0.36299999999999999},{"interface":"front","average":{"1min":0.41399999999999998,"5min":0.41399999999999998,"15min":0.41399999999999998},"min":{"1min":0.27500000000000002,"5min":0.27500000000000002,"15min":0.27500000000000002},"max":{"1min":0.64600000000000002,"5min":0.64600000000000002,"15min":0.64600000000000002},"last":0.34999999999999998}]},{"osd":4,"last update":"Fri Nov 24 07:57:17 2023","interfaces":[{"interface":"back","average":{"1min":0.60799999999999998,"5min":0.60799999999999998,"15min":0.60799999999999998},"min":{"1min":0.42999999999999999,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":0.81599999999999995,"5min":0.81599999999999995,"15min":0.81599999999999995},"last":0.46100000000000002},{"interface":"front","average":{"1min":0.59999999999999998,"5min":0.59999999999999998,"15min":0.59999999999999998},"min":{"1min":0.41599999999999998,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":0.79600000000000004,"5min":0.79600000000000004,"15min":0.79600000000000004},"last":0.42699999999999999}]},{"osd":5,"last update":"Fri Nov 24 07:57:17 2023","interfaces":[{"interface":"back","average":{"1min":0.621,"5min":0.621,"15min":0.621},"min":{"1min":0.44900000000000001,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":0.84899999999999998,"5min":0.84899999999999998,"15min":0.84899999999999998},"last":0.46800000000000003},{"interface":"front","average":{"1min":0.71499999999999997,"5min":0.71499999999999997,"15min":0.71499999999999997},"min":{"1min":0.433,"5min":0.433,"15min":0.433},"max":{"1min":2.1699999999999999,"5min":2.1699999999999999,"15min":2.1699999999999999},"last":0.53500000000000003}]},{"osd":6,"last update":"Fri Nov 24 07:57:17 2023","interfaces":[{"interface":"back","average":{"1min":0.66700000000000004,"5min":0.66700000000000004,"15min":0.66700000000000004},"min":{"1min":0.47399999999999998,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":0.83599999999999997,"5min":0.83599999999999997,"15min":0.83599999999999997},"last":0.48399999999999999},{"interface":"front","average":{"1min":0.67100000000000004,"5min":0.67100000000000004,"15min":0.67100000000000004},"min":{"1min":0.496,"5min":0.496,"15min":0.496},"max":{"1min":0.88500000000000001,"5min":0.88500000000000001,"15min":0.88500000000000001},"last":0.41899999999999998}]},{"osd":7,"last update":"Fri Nov 24 07:57:17 2023","interfaces":[{"interface":"back","average":{"1min":0.66200000000000003,"5min":0.66200000000000003,"15min":0.66200000000000003},"min":{"1min":0.443,"5min":0.443,"15min":0.443},"max":{"1min":0.85999999999999999,"5min":0.85999999999999999,"15min":0.85999999999999999},"last":0.51700000000000002},{"interface":"front","average":{"1min":0.68000000000000005,"5min":0.68000000000000005,"15min":0.68000000000000005},"min":{"1min":0.495,"5min":0.495,"15min":0.495},"max":{"1min":0.86399999999999999,"5min":0.86399999999999999,"15min":0.86399999999999999},"last":0.44500000000000001}]}]},{"osd":3,"up_from":13,"seq":55834574867,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27408,"kb_used_data":380,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344432,"statfs":{"total":96636764160,"available":96608698368,"internally_reserved":0,"allocated":389120,"data_stored":146874,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 07:57:20 2023","interfaces":[{"interface":"back","average":{"1min":0.35599999999999998,"5min":0.35599999999999998,"15min":0.35599999999999998},"min":{"1min":0.222,"5min":0.222,"15min":0.222},"max":{"1min":0.84799999999999998,"5min":0.84799999999999998,"15min":0.84799999999999998},"last":0.439},{"interface":"front","average":{"1min":0.36699999999999999,"5min":0.36699999999999999,"15min":0.36699999999999999},"min":{"1min":0.23699999999999999,"5min":0.23699999999999999,"15min":0.23699999999999999},"max":{"1min":0.873,"5min":0.873,"15min":0.873},"last":0.307}]},{"osd":1,"last update":"Fri Nov 24 07:57:20 2023","interfaces":[{"interface":"back","average":{"1min":0.375,"5min":0.375,"15min":0.375},"min":{"1min":0.25700000000000001,"5min":0.25700000000000001,"15min":0.25700000000000001},"max":{"1min":0.92000000000000004,"5min":0.92000000000000004,"15min":0.92000000000000004},"last":0.36899999999999999},{"interface":"front","average":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"min":{"1min":0.248,"5min":0.248,"15min":0.248},"max":{"1min":0.81699999999999995,"5min":0.81699999999999995,"15min":0.81699999999999995},"last":0.34300000000000003}]},{"osd":2,"last update":"Fri Nov 24 07:57:20 2023","interfaces":[{"interface":"back","average":{"1min":0.36099999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"min":{"1min":0.24399999999999999,"5min":0.24399999999999999,"15min":0.24399999999999999},"max":{"1min":0.88500000000000001,"5min":0.88500000000000001,"15min":0.88500000000000001},"last":0.38900000000000001},{"interface":"front","average":{"1min":0.46800000000000003,"5min":0.46800000000000003,"15min":0.46800000000000003},"min":{"1min":0.31900000000000001,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.98099999999999998,"5min":0.98099999999999998,"15min":0.98099999999999998},"last":0.41199999999999998}]},{"osd":4,"last update":"Fri Nov 24 07:57:20 2023","interfaces":[{"interface":"back","average":{"1min":0.79700000000000004,"5min":0.79700000000000004,"15min":0.79700000000000004},"min":{"1min":0.33500000000000002,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":5.7350000000000003,"5min":5.7350000000000003,"15min":5.7350000000000003},"last":0.57499999999999996},{"interface":"front","average":{"1min":0.58499999999999996,"5min":0.58499999999999996,"15min":0.58499999999999996},"min":{"1min":0.44500000000000001,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":1.27,"5min":1.27,"15min":1.27},"last":0.65800000000000003}]},{"osd":5,"last update":"Fri Nov 24 07:57:20 2023","interfaces":[{"interface":"back","average":{"1min":0.58799999999999997,"5min":0.58799999999999997,"15min":0.58799999999999997},"min":{"1min":0.40699999999999997,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":1.4430000000000001,"5min":1.4430000000000001,"15min":1.4430000000000001},"last":0.50700000000000001},{"interface":"front","average":{"1min":0.627,"5min":0.627,"15min":0.627},"min":{"1min":0.377,"5min":0.377,"15min":0.377},"max":{"1min":1.4239999999999999,"5min":1.4239999999999999,"15min":1.4239999999999999},"last":0.55200000000000005}]},{"osd":6,"last update":"Fri Nov 24 07:57:20 2023","interfaces":[{"interface":"back","average":{"1min":0.62,"5min":0.62,"15min":0.62},"min":{"1min":0.434,"5min":0.434,"15min":0.434},"max":{"1min":1.355,"5min":1.355,"15min":1.355},"last":0.49199999999999999},{"interface":"front","average":{"1min":0.84599999999999997,"5min":0.84599999999999997,"15min":0.84599999999999997},"min":{"1min":0.40999999999999998,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":5.6479999999999997,"5min":5.6479999999999997,"15min":5.6479999999999997},"last":0.53100000000000003}]},{"osd":7,"last update":"Fri Nov 24 07:57:20 2023","interfaces":[{"interface":"back","average":{"1min":0.66600000000000004,"5min":0.66600000000000004,"15min":0.66600000000000004},"min":{"1min":0.42299999999999999,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":1.51,"5min":1.51,"15min":1.51},"last":0.48099999999999998},{"interface":"front","average":{"1min":0.84199999999999997,"5min":0.84199999999999997,"15min":0.84199999999999997},"min":{"1min":0.45600000000000002,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":5.7069999999999999,"5min":5.7069999999999999,"15min":5.7069999999999999},"last":0.54000000000000004}]}]},{"osd":4,"up_from":13,"seq":55834574866,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27400,"kb_used_data":380,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344440,"statfs":{"total":96636764160,"available":96608706560,"internally_reserved":0,"allocated":389120,"data_stored":147732,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 07:57:21 2023","interfaces":[{"interface":"back","average":{"1min":0.58099999999999996,"5min":0.58099999999999996,"15min":0.58099999999999996},"min":{"1min":0.37,"5min":0.37,"15min":0.37},"max":{"1min":0.79300000000000004,"5min":0.79300000000000004,"15min":0.79300000000000004},"last":0.34899999999999998},{"interface":"front","average":{"1min":0.54300000000000004,"5min":0.54300000000000004,"15min":0.54300000000000004},"min":{"1min":0.33600000000000002,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":1.0249999999999999,"5min":1.0249999999999999,"15min":1.0249999999999999},"last":0.45000000000000001}]},{"osd":1,"last update":"Fri Nov 24 07:57:21 2023","interfaces":[{"interface":"back","average":{"1min":0.55200000000000005,"5min":0.55200000000000005,"15min":0.55200000000000005},"min":{"1min":0.40000000000000002,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":0.86399999999999999,"5min":0.86399999999999999,"15min":0.86399999999999999},"last":0.55700000000000005},{"interface":"front","average":{"1min":0.55100000000000005,"5min":0.55100000000000005,"15min":0.55100000000000005},"min":{"1min":0.36099999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.98699999999999999,"5min":0.98699999999999999,"15min":0.98699999999999999},"last":0.46999999999999997}]},{"osd":2,"last update":"Fri Nov 24 07:57:21 2023","interfaces":[{"interface":"back","average":{"1min":0.56100000000000005,"5min":0.56100000000000005,"15min":0.56100000000000005},"min":{"1min":0.35799999999999998,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.81100000000000005,"5min":0.81100000000000005,"15min":0.81100000000000005},"last":0.42999999999999999},{"interface":"front","average":{"1min":0.61299999999999999,"5min":0.61299999999999999,"15min":0.61299999999999999},"min":{"1min":0.38100000000000001,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.89200000000000002,"5min":0.89200000000000002,"15min":0.89200000000000002},"last":0.40000000000000002}]},{"osd":3,"last update":"Fri Nov 24 07:57:21 2023","interfaces":[{"interface":"back","average":{"1min":0.58499999999999996,"5min":0.58499999999999996,"15min":0.58499999999999996},"min":{"1min":0.439,"5min":0.439,"15min":0.439},"max":{"1min":0.89700000000000002,"5min":0.89700000000000002,"15min":0.89700000000000002},"last":0.51500000000000001},{"interface":"front","average":{"1min":0.63100000000000001,"5min":0.63100000000000001,"15min":0.63100000000000001},"min":{"1min":0.40300000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.96199999999999997,"5min":0.96199999999999997,"15min":0.96199999999999997},"last":0.56699999999999995}]},{"osd":5,"last update":"Fri Nov 24 07:57:21 2023","interfaces":[{"interface":"back","average":{"1min":0.45500000000000002,"5min":0.45500000000000002,"15min":0.45500000000000002},"min":{"1min":0.29799999999999999,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":0.70499999999999996,"5min":0.70499999999999996,"15min":0.70499999999999996},"last":0.374},{"interface":"front","average":{"1min":0.61899999999999999,"5min":0.61899999999999999,"15min":0.61899999999999999},"min":{"1min":0.439,"5min":0.439,"15min":0.439},"max":{"1min":0.86399999999999999,"5min":0.86399999999999999,"15min":0.86399999999999999},"last":0.58599999999999997}]},{"osd":6,"last update":"Fri Nov 24 07:57:21 2023","interfaces":[{"interface":"back","average":{"1min":0.58599999999999997,"5min":0.58599999999999997,"15min":0.58599999999999997},"min":{"1min":0.35499999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.76800000000000002,"5min":0.76800000000000002,"15min":0.76800000000000002},"last":0.60699999999999998},{"interface":"front","average":{"1min":0.61399999999999999,"5min":0.61399999999999999,"15min":0.61399999999999999},"min":{"1min":0.436,"5min":0.436,"15min":0.436},"max":{"1min":1.095,"5min":1.095,"15min":1.095},"last":0.504}]},{"osd":7,"last update":"Fri Nov 24 07:57:21 2023","interfaces":[{"interface":"back","average":{"1min":0.57299999999999995,"5min":0.57299999999999995,"15min":0.57299999999999995},"min":{"1min":0.35099999999999998,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.89300000000000002,"5min":0.89300000000000002,"15min":0.89300000000000002},"last":0.54400000000000004},{"interface":"front","average":{"1min":0.66300000000000003,"5min":0.66300000000000003,"15min":0.66300000000000003},"min":{"1min":0.41099999999999998,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":0.91500000000000004,"5min":0.91500000000000004,"15min":0.91500000000000004},"last":0.52900000000000003}]}]},{"osd":5,"up_from":13,"seq":55834574867,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27976,"kb_used_data":948,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343864,"statfs":{"total":96636764160,"available":96608116736,"internally_reserved":0,"allocated":970752,"data_stored":737048,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 07:57:17 2023","interfaces":[{"interface":"back","average":{"1min":0.56200000000000006,"5min":0.56200000000000006,"15min":0.56200000000000006},"min":{"1min":0.32700000000000001,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":1.0109999999999999,"5min":1.0109999999999999,"15min":1.0109999999999999},"last":0.41399999999999998},{"interface":"front","average":{"1min":0.80000000000000004,"5min":0.80000000000000004,"15min":0.80000000000000004},"min":{"1min":0.33400000000000002,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":5.431,"5min":5.431,"15min":5.431},"last":0.57299999999999995}]},{"osd":1,"last update":"Fri Nov 24 07:57:17 2023","interfaces":[{"interface":"back","average":{"1min":0.55900000000000005,"5min":0.55900000000000005,"15min":0.55900000000000005},"min":{"1min":0.28899999999999998,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":1.016,"5min":1.016,"15min":1.016},"last":0.39700000000000002},{"interface":"front","average":{"1min":0.56999999999999995,"5min":0.56999999999999995,"15min":0.56999999999999995},"min":{"1min":0.311,"5min":0.311,"15min":0.311},"max":{"1min":0.96499999999999997,"5min":0.96499999999999997,"15min":0.96499999999999997},"last":0.55100000000000005}]},{"osd":2,"last update":"Fri Nov 24 07:57:17 2023","interfaces":[{"interface":"back","average":{"1min":0.55100000000000005,"5min":0.55100000000000005,"15min":0.55100000000000005},"min":{"1min":0.377,"5min":0.377,"15min":0.377},"max":{"1min":1.163,"5min":1.163,"15min":1.163},"last":0.42799999999999999},{"interface":"front","average":{"1min":0.63,"5min":0.63,"15min":0.63},"min":{"1min":0.39200000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":1.119,"5min":1.119,"15min":1.119},"last":0.52100000000000002}]},{"osd":3,"last update":"Fri Nov 24 07:57:17 2023","interfaces":[{"interface":"back","average":{"1min":0.90500000000000003,"5min":0.90500000000000003,"15min":0.90500000000000003},"min":{"1min":0.36499999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":5.609,"5min":5.609,"15min":5.609},"last":0.36899999999999999},{"interface":"front","average":{"1min":0.67600000000000005,"5min":0.67600000000000005,"15min":0.67600000000000005},"min":{"1min":0.41899999999999998,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":1.288,"5min":1.288,"15min":1.288},"last":0.53700000000000003}]},{"osd":4,"last update":"Fri Nov 24 07:57:17 2023","interfaces":[{"interface":"back","average":{"1min":0.51000000000000001,"5min":0.51000000000000001,"15min":0.51000000000000001},"min":{"1min":0.36399999999999999,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.68799999999999994,"5min":0.68799999999999994,"15min":0.68799999999999994},"last":0.32100000000000001},{"interface":"front","average":{"1min":0.57199999999999995,"5min":0.57199999999999995,"15min":0.57199999999999995},"min":{"1min":0.39800000000000002,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.78300000000000003,"5min":0.78300000000000003,"15min":0.78300000000000003},"last":0.29599999999999999}]},{"osd":6,"last update":"Fri Nov 24 07:57:17 2023","interfaces":[{"interface":"back","average":{"1min":0.57099999999999995,"5min":0.57099999999999995,"15min":0.57099999999999995},"min":{"1min":0.35599999999999998,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.89700000000000002,"5min":0.89700000000000002,"15min":0.89700000000000002},"last":0.35199999999999998},{"interface":"front","average":{"1min":0.625,"5min":0.625,"15min":0.625},"min":{"1min":0.42899999999999999,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":0.88200000000000001,"5min":0.88200000000000001,"15min":0.88200000000000001},"last":0.44800000000000001}]},{"osd":7,"last update":"Fri Nov 24 07:57:17 2023","interfaces":[{"interface":"back","average":{"1min":0.64600000000000002,"5min":0.64600000000000002,"15min":0.64600000000000002},"min":{"1min":0.45000000000000001,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":0.96599999999999997,"5min":0.96599999999999997,"15min":0.96599999999999997},"last":0.46700000000000003},{"interface":"front","average":{"1min":0.65300000000000002,"5min":0.65300000000000002,"15min":0.65300000000000002},"min":{"1min":0.46300000000000002,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":0.94199999999999995,"5min":0.94199999999999995,"15min":0.94199999999999995},"last":0.502}]}]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":776,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-11-24T07:57:34.432 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-11-24T07:57:34.571 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:34.569+0000 7f35a4ea8700 1 Processor -- start 2023-11-24T07:57:34.571 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:34.569+0000 7f35a4ea8700 1 -- start start 2023-11-24T07:57:34.573 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:34.570+0000 7f35a4ea8700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f35a006b620 0x7f35a00b3a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:57:34.573 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:34.570+0000 7f35a4ea8700 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f35a01629c0 con 0x7f35a00b5ab0 2023-11-24T07:57:34.573 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:34.570+0000 7f35a4ea8700 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f35a0071f50 con 0x7f35a006ee60 2023-11-24T07:57:34.573 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:34.570+0000 7f35a4ea8700 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f35a0072230 con 0x7f35a006b620 2023-11-24T07:57:34.573 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:34.570+0000 7f359e59c700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f35a006b620 0x7f35a00b3a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:57:34.573 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:34.570+0000 7f359e59c700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f35a006b620 0x7f35a00b3a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:51828/0 (socket says 172.21.15.59:51828) 2023-11-24T07:57:34.573 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:34.570+0000 7f359dd9b700 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7f35a00b5ab0 0x7f35a0161ba0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:55302/0 (socket says 172.21.15.59:55302) 2023-11-24T07:57:34.573 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:34.570+0000 7f359e59c700 1 -- 172.21.15.59:0/4213388764 learned_addr learned my addr 172.21.15.59:0/4213388764 (peer_addr_for_me v2:172.21.15.59:0/0) 2023-11-24T07:57:34.573 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:34.571+0000 7f359e59c700 1 -- 172.21.15.59:0/4213388764 >> v1:172.21.15.76:6789/0 conn(0x7f35a006ee60 legacy=0x7f35a01622b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:57:34.573 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:34.571+0000 7f359e59c700 1 -- 172.21.15.59:0/4213388764 >> v1:172.21.15.59:6789/0 conn(0x7f35a00b5ab0 legacy=0x7f35a0161ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:57:34.574 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:34.571+0000 7f359d59a700 1 -- 172.21.15.59:0/4213388764 <== mon.0 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1216201278 0 0) 0x7f35a01629c0 con 0x7f35a00b5ab0 2023-11-24T07:57:34.574 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:34.571+0000 7f359e59c700 1 -- 172.21.15.59:0/4213388764 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f35a0072510 con 0x7f35a006b620 2023-11-24T07:57:34.574 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:34.571+0000 7f359e59c700 1 --2- 172.21.15.59:0/4213388764 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f35a006b620 0x7f35a00b3a40 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f35900098f0 tx=0x7f3590009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c07960a2bf8c6c64 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:57:34.574 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:34.571+0000 7f359d59a700 1 -- 172.21.15.59:0/4213388764 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f359000b8a0 con 0x7f35a006b620 2023-11-24T07:57:34.574 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:34.571+0000 7f359d59a700 1 -- 172.21.15.59:0/4213388764 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f359000ba00 con 0x7f35a006b620 2023-11-24T07:57:34.574 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:34.571+0000 7f359d59a700 1 -- 172.21.15.59:0/4213388764 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f359000bd90 con 0x7f35a006b620 2023-11-24T07:57:34.575 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:34.572+0000 7f35a4ea8700 1 -- 172.21.15.59:0/4213388764 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f35a006b620 msgr2=0x7f35a00b3a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:57:34.575 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:34.572+0000 7f35a4ea8700 1 --2- 172.21.15.59:0/4213388764 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f35a006b620 0x7f35a00b3a40 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f35900098f0 tx=0x7f3590009c00 comp rx=0 tx=0).stop 2023-11-24T07:57:34.575 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:34.572+0000 7f35a4ea8700 1 -- 172.21.15.59:0/4213388764 shutdown_connections 2023-11-24T07:57:34.575 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:34.572+0000 7f35a4ea8700 1 --2- 172.21.15.59:0/4213388764 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f35a006b620 0x7f35a00b3a40 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:34.575 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:34.572+0000 7f35a4ea8700 1 -- 172.21.15.59:0/4213388764 >> 172.21.15.59:0/4213388764 conn(0x7f35a00b2740 msgr2=0x7f35a00b2b20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:57:34.575 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:34.572+0000 7f35a4ea8700 1 -- 172.21.15.59:0/4213388764 shutdown_connections 2023-11-24T07:57:34.576 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:34.572+0000 7f35a4ea8700 1 -- 172.21.15.59:0/4213388764 wait complete. 2023-11-24T07:57:34.576 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:34.573+0000 7f35a4ea8700 1 Processor -- start 2023-11-24T07:57:34.576 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:34.573+0000 7f35a4ea8700 1 -- start start 2023-11-24T07:57:34.578 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:34.574+0000 7f35a4ea8700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f35a006b620 0x7f35a014fea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:57:34.578 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:34.574+0000 7f35a4ea8700 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f35a006ee60 0x7f35a00c59b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:57:34.578 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:34.574+0000 7f35a4ea8700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f35a00b5ab0 0x7f35a0142b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:57:34.578 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:34.574+0000 7f35a4ea8700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35a0143060 con 0x7f35a00b5ab0 2023-11-24T07:57:34.578 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:34.574+0000 7f35a4ea8700 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35a014eea0 con 0x7f35a006b620 2023-11-24T07:57:34.578 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:34.574+0000 7f35a4ea8700 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f35a014fa10 con 0x7f35a006ee60 2023-11-24T07:57:34.578 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:34.574+0000 7f359ed9d700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f35a00b5ab0 0x7f35a0142b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:57:34.578 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:34.574+0000 7f359ed9d700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f35a00b5ab0 0x7f35a0142b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:51838/0 (socket says 172.21.15.59:51838) 2023-11-24T07:57:34.578 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:34.574+0000 7f359ed9d700 1 -- 172.21.15.59:0/474750619 learned_addr learned my addr 172.21.15.59:0/474750619 (peer_addr_for_me v2:172.21.15.59:0/0) 2023-11-24T07:57:34.578 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:34.574+0000 7f359dd9b700 1 --2- 172.21.15.59:0/474750619 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f35a006ee60 0x7f35a00c59b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:57:34.579 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:34.574+0000 7f359e59c700 1 --2- 172.21.15.59:0/474750619 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f35a006b620 0x7f35a014fea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:57:34.579 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:34.574+0000 7f359ed9d700 1 -- 172.21.15.59:0/474750619 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f35a006ee60 msgr2=0x7f35a00c59b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:57:34.579 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:34.574+0000 7f359ed9d700 1 --2- 172.21.15.59:0/474750619 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f35a006ee60 0x7f35a00c59b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:34.579 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:34.574+0000 7f359ed9d700 1 -- 172.21.15.59:0/474750619 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f35a006b620 msgr2=0x7f35a014fea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:57:34.579 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:34.574+0000 7f359ed9d700 1 --2- 172.21.15.59:0/474750619 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f35a006b620 0x7f35a014fea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:34.579 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:34.574+0000 7f359ed9d700 1 -- 172.21.15.59:0/474750619 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3590009550 con 0x7f35a00b5ab0 2023-11-24T07:57:34.579 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:34.575+0000 7f359ed9d700 1 --2- 172.21.15.59:0/474750619 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f35a00b5ab0 0x7f35a0142b20 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f359800cc80 tx=0x7f359800a340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:57:34.579 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:34.575+0000 7f358effd700 1 -- 172.21.15.59:0/474750619 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f359800ad40 con 0x7f35a00b5ab0 2023-11-24T07:57:34.579 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:34.575+0000 7f358effd700 1 -- 172.21.15.59:0/474750619 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f359800ce20 con 0x7f35a00b5ab0 2023-11-24T07:57:34.579 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:34.575+0000 7f35a4ea8700 1 -- 172.21.15.59:0/474750619 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f35a014f0e0 con 0x7f35a00b5ab0 2023-11-24T07:57:34.579 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:34.575+0000 7f358effd700 1 -- 172.21.15.59:0/474750619 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f359800aea0 con 0x7f35a00b5ab0 2023-11-24T07:57:34.580 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:34.575+0000 7f35a4ea8700 1 -- 172.21.15.59:0/474750619 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f35a014f6a0 con 0x7f35a00b5ab0 2023-11-24T07:57:34.580 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:34.576+0000 7f35a4ea8700 1 -- 172.21.15.59:0/474750619 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f35a0068670 con 0x7f35a00b5ab0 2023-11-24T07:57:34.580 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:34.576+0000 7f358effd700 1 -- 172.21.15.59:0/474750619 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f3598003c40 con 0x7f35a00b5ab0 2023-11-24T07:57:34.580 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:34.577+0000 7f358effd700 1 --2- 172.21.15.59:0/474750619 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f358006aed0 0x7f358006d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:57:34.581 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:34.577+0000 7f358effd700 1 -- 172.21.15.59:0/474750619 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f3598089620 con 0x7f35a00b5ab0 2023-11-24T07:57:34.581 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:34.577+0000 7f359e59c700 1 --2- 172.21.15.59:0/474750619 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f358006aed0 0x7f358006d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:57:34.581 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:34.578+0000 7f359e59c700 1 --2- 172.21.15.59:0/474750619 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f358006aed0 0x7f358006d390 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f3590009c40 tx=0x7f3590008030 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:57:34.583 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:34.581+0000 7f358effd700 1 -- 172.21.15.59:0/474750619 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f3598054760 con 0x7f35a00b5ab0 2023-11-24T07:57:34.781 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:34.779+0000 7f35a4ea8700 1 -- 172.21.15.59:0/474750619 --> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f35a0072ad0 con 0x7f358006aed0 2023-11-24T07:57:34.789 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:34.788+0000 7f358effd700 1 -- 172.21.15.59:0/474750619 <== mgr.4100 v2:172.21.15.76:6832/4264046536 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+288872 (secure 0 0 0) 0x7f359000b880 con 0x7f358006aed0 2023-11-24T07:57:34.793 INFO:teuthology.orchestra.run.smithi059.stdout: 2023-11-24T07:57:34.795 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:34.791+0000 7f35a4ea8700 1 -- 172.21.15.59:0/474750619 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f358006aed0 msgr2=0x7f358006d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:57:34.795 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:34.791+0000 7f35a4ea8700 1 --2- 172.21.15.59:0/474750619 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f358006aed0 0x7f358006d390 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f3590009c40 tx=0x7f3590008030 comp rx=0 tx=0).stop 2023-11-24T07:57:34.796 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:34.791+0000 7f35a4ea8700 1 -- 172.21.15.59:0/474750619 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f35a00b5ab0 msgr2=0x7f35a0142b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:57:34.796 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:34.791+0000 7f35a4ea8700 1 --2- 172.21.15.59:0/474750619 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f35a00b5ab0 0x7f35a0142b20 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f359800cc80 tx=0x7f359800a340 comp rx=0 tx=0).stop 2023-11-24T07:57:34.796 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:34.791+0000 7f35a4ea8700 1 -- 172.21.15.59:0/474750619 shutdown_connections 2023-11-24T07:57:34.796 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:34.791+0000 7f35a4ea8700 1 --2- 172.21.15.59:0/474750619 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f358006aed0 0x7f358006d390 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:34.796 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:34.791+0000 7f35a4ea8700 1 --2- 172.21.15.59:0/474750619 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f35a00b5ab0 0x7f35a0142b20 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:34.796 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:34.791+0000 7f35a4ea8700 1 --2- 172.21.15.59:0/474750619 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f35a006b620 0x7f35a014fea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:34.796 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:34.791+0000 7f35a4ea8700 1 --2- 172.21.15.59:0/474750619 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f35a006ee60 0x7f35a00c59b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:34.796 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:34.791+0000 7f35a4ea8700 1 -- 172.21.15.59:0/474750619 >> 172.21.15.59:0/474750619 conn(0x7f35a00b2740 msgr2=0x7f35a00b3830 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:57:34.796 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:34.791+0000 7f35a4ea8700 1 -- 172.21.15.59:0/474750619 shutdown_connections 2023-11-24T07:57:34.796 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:34.791+0000 7f35a4ea8700 1 -- 172.21.15.59:0/474750619 wait complete. 2023-11-24T07:57:34.796 INFO:teuthology.orchestra.run.smithi059.stderr:dumped all 2023-11-24T07:57:34.805 INFO:teuthology.orchestra.run.smithi059.stdout:{"pg_ready":true,"pg_map":{"version":64,"stamp":"2023-11-24T07:57:34.315208+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":220336,"kb_used_data":4136,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754754384,"statfs":{"total":773094113280,"available":772868489216,"internally_reserved":0,"allocated":4235264,"data_stored":2354340,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12745,"internal_metadata":221236791},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.727424"},"pg_stats":[{"pgid":"5.3f","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-24T07:57:27.498774+0000","last_change":"2023-11-24T07:57:25.613284+0000","last_active":"2023-11-24T07:57:27.498774+0000","last_peered":"2023-11-24T07:57:27.498774+0000","last_clean":"2023-11-24T07:57:27.498774+0000","last_became_active":"2023-11-24T07:57:25.606187+0000","last_became_peered":"2023-11-24T07:57:25.606187+0000","last_unstale":"2023-11-24T07:57:27.498774+0000","last_undegraded":"2023-11-24T07:57:27.498774+0000","last_fullsized":"2023-11-24T07:57:27.498774+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:57:24.572681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:57:24.572681+0000","last_clean_scrub_stamp":"2023-11-24T07:57:24.572681+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T12:51:13.357718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.3e","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-24T07:57:26.584176+0000","last_change":"2023-11-24T07:57:25.611359+0000","last_active":"2023-11-24T07:57:26.584176+0000","last_peered":"2023-11-24T07:57:26.584176+0000","last_clean":"2023-11-24T07:57:26.584176+0000","last_became_active":"2023-11-24T07:57:25.611053+0000","last_became_peered":"2023-11-24T07:57:25.611053+0000","last_unstale":"2023-11-24T07:57:26.584176+0000","last_undegraded":"2023-11-24T07:57:26.584176+0000","last_fullsized":"2023-11-24T07:57:26.584176+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:57:24.572681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:57:24.572681+0000","last_clean_scrub_stamp":"2023-11-24T07:57:24.572681+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T15:47:12.125658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.3d","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-24T07:57:27.499082+0000","last_change":"2023-11-24T07:57:25.601956+0000","last_active":"2023-11-24T07:57:27.499082+0000","last_peered":"2023-11-24T07:57:27.499082+0000","last_clean":"2023-11-24T07:57:27.499082+0000","last_became_active":"2023-11-24T07:57:25.601517+0000","last_became_peered":"2023-11-24T07:57:25.601517+0000","last_unstale":"2023-11-24T07:57:27.499082+0000","last_undegraded":"2023-11-24T07:57:27.499082+0000","last_fullsized":"2023-11-24T07:57:27.499082+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:57:24.572681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:57:24.572681+0000","last_clean_scrub_stamp":"2023-11-24T07:57:24.572681+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T17:40:49.956916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.3c","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-24T07:57:27.499452+0000","last_change":"2023-11-24T07:57:25.606653+0000","last_active":"2023-11-24T07:57:27.499452+0000","last_peered":"2023-11-24T07:57:27.499452+0000","last_clean":"2023-11-24T07:57:27.499452+0000","last_became_active":"2023-11-24T07:57:25.606315+0000","last_became_peered":"2023-11-24T07:57:25.606315+0000","last_unstale":"2023-11-24T07:57:27.499452+0000","last_undegraded":"2023-11-24T07:57:27.499452+0000","last_fullsized":"2023-11-24T07:57:27.499452+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:57:24.572681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:57:24.572681+0000","last_clean_scrub_stamp":"2023-11-24T07:57:24.572681+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T13:39:16.521602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.3b","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-24T07:57:26.583942+0000","last_change":"2023-11-24T07:57:25.600519+0000","last_active":"2023-11-24T07:57:26.583942+0000","last_peered":"2023-11-24T07:57:26.583942+0000","last_clean":"2023-11-24T07:57:26.583942+0000","last_became_active":"2023-11-24T07:57:25.600235+0000","last_became_peered":"2023-11-24T07:57:25.600235+0000","last_unstale":"2023-11-24T07:57:26.583942+0000","last_undegraded":"2023-11-24T07:57:26.583942+0000","last_fullsized":"2023-11-24T07:57:26.583942+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:57:24.572681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:57:24.572681+0000","last_clean_scrub_stamp":"2023-11-24T07:57:24.572681+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T19:13:40.799042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.3a","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-24T07:57:27.498253+0000","last_change":"2023-11-24T07:57:25.600774+0000","last_active":"2023-11-24T07:57:27.498253+0000","last_peered":"2023-11-24T07:57:27.498253+0000","last_clean":"2023-11-24T07:57:27.498253+0000","last_became_active":"2023-11-24T07:57:25.600390+0000","last_became_peered":"2023-11-24T07:57:25.600390+0000","last_unstale":"2023-11-24T07:57:27.498253+0000","last_undegraded":"2023-11-24T07:57:27.498253+0000","last_fullsized":"2023-11-24T07:57:27.498253+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:57:24.572681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:57:24.572681+0000","last_clean_scrub_stamp":"2023-11-24T07:57:24.572681+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T17:53:33.447022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.39","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-24T07:57:27.499083+0000","last_change":"2023-11-24T07:57:25.596753+0000","last_active":"2023-11-24T07:57:27.499083+0000","last_peered":"2023-11-24T07:57:27.499083+0000","last_clean":"2023-11-24T07:57:27.499083+0000","last_became_active":"2023-11-24T07:57:25.596467+0000","last_became_peered":"2023-11-24T07:57:25.596467+0000","last_unstale":"2023-11-24T07:57:27.499083+0000","last_undegraded":"2023-11-24T07:57:27.499083+0000","last_fullsized":"2023-11-24T07:57:27.499083+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:57:24.572681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:57:24.572681+0000","last_clean_scrub_stamp":"2023-11-24T07:57:24.572681+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T14:04:03.022069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.38","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-24T07:57:27.498354+0000","last_change":"2023-11-24T07:57:25.601277+0000","last_active":"2023-11-24T07:57:27.498354+0000","last_peered":"2023-11-24T07:57:27.498354+0000","last_clean":"2023-11-24T07:57:27.498354+0000","last_became_active":"2023-11-24T07:57:25.600867+0000","last_became_peered":"2023-11-24T07:57:25.600867+0000","last_unstale":"2023-11-24T07:57:27.498354+0000","last_undegraded":"2023-11-24T07:57:27.498354+0000","last_fullsized":"2023-11-24T07:57:27.498354+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:57:24.572681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:57:24.572681+0000","last_clean_scrub_stamp":"2023-11-24T07:57:24.572681+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T16:47:55.590649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.37","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-24T07:57:27.499310+0000","last_change":"2023-11-24T07:57:25.611574+0000","last_active":"2023-11-24T07:57:27.499310+0000","last_peered":"2023-11-24T07:57:27.499310+0000","last_clean":"2023-11-24T07:57:27.499310+0000","last_became_active":"2023-11-24T07:57:25.611258+0000","last_became_peered":"2023-11-24T07:57:25.611258+0000","last_unstale":"2023-11-24T07:57:27.499310+0000","last_undegraded":"2023-11-24T07:57:27.499310+0000","last_fullsized":"2023-11-24T07:57:27.499310+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:57:24.572681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:57:24.572681+0000","last_clean_scrub_stamp":"2023-11-24T07:57:24.572681+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T15:54:26.161043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.36","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-24T07:57:27.499746+0000","last_change":"2023-11-24T07:57:25.606465+0000","last_active":"2023-11-24T07:57:27.499746+0000","last_peered":"2023-11-24T07:57:27.499746+0000","last_clean":"2023-11-24T07:57:27.499746+0000","last_became_active":"2023-11-24T07:57:25.606204+0000","last_became_peered":"2023-11-24T07:57:25.606204+0000","last_unstale":"2023-11-24T07:57:27.499746+0000","last_undegraded":"2023-11-24T07:57:27.499746+0000","last_fullsized":"2023-11-24T07:57:27.499746+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:57:24.572681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:57:24.572681+0000","last_clean_scrub_stamp":"2023-11-24T07:57:24.572681+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T18:02:42.589387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.35","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-24T07:57:26.583720+0000","last_change":"2023-11-24T07:57:25.610107+0000","last_active":"2023-11-24T07:57:26.583720+0000","last_peered":"2023-11-24T07:57:26.583720+0000","last_clean":"2023-11-24T07:57:26.583720+0000","last_became_active":"2023-11-24T07:57:25.609413+0000","last_became_peered":"2023-11-24T07:57:25.609413+0000","last_unstale":"2023-11-24T07:57:26.583720+0000","last_undegraded":"2023-11-24T07:57:26.583720+0000","last_fullsized":"2023-11-24T07:57:26.583720+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:57:24.572681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:57:24.572681+0000","last_clean_scrub_stamp":"2023-11-24T07:57:24.572681+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T17:08:50.195049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.34","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-24T07:57:27.498828+0000","last_change":"2023-11-24T07:57:25.600896+0000","last_active":"2023-11-24T07:57:27.498828+0000","last_peered":"2023-11-24T07:57:27.498828+0000","last_clean":"2023-11-24T07:57:27.498828+0000","last_became_active":"2023-11-24T07:57:25.600611+0000","last_became_peered":"2023-11-24T07:57:25.600611+0000","last_unstale":"2023-11-24T07:57:27.498828+0000","last_undegraded":"2023-11-24T07:57:27.498828+0000","last_fullsized":"2023-11-24T07:57:27.498828+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:57:24.572681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:57:24.572681+0000","last_clean_scrub_stamp":"2023-11-24T07:57:24.572681+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T17:53:33.447022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.33","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-24T07:57:27.499277+0000","last_change":"2023-11-24T07:57:25.609608+0000","last_active":"2023-11-24T07:57:27.499277+0000","last_peered":"2023-11-24T07:57:27.499277+0000","last_clean":"2023-11-24T07:57:27.499277+0000","last_became_active":"2023-11-24T07:57:25.609381+0000","last_became_peered":"2023-11-24T07:57:25.609381+0000","last_unstale":"2023-11-24T07:57:27.499277+0000","last_undegraded":"2023-11-24T07:57:27.499277+0000","last_fullsized":"2023-11-24T07:57:27.499277+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:57:24.572681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:57:24.572681+0000","last_clean_scrub_stamp":"2023-11-24T07:57:24.572681+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T19:30:34.653712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.32","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-24T07:57:27.499306+0000","last_change":"2023-11-24T07:57:25.605234+0000","last_active":"2023-11-24T07:57:27.499306+0000","last_peered":"2023-11-24T07:57:27.499306+0000","last_clean":"2023-11-24T07:57:27.499306+0000","last_became_active":"2023-11-24T07:57:25.604959+0000","last_became_peered":"2023-11-24T07:57:25.604959+0000","last_unstale":"2023-11-24T07:57:27.499306+0000","last_undegraded":"2023-11-24T07:57:27.499306+0000","last_fullsized":"2023-11-24T07:57:27.499306+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:57:24.572681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:57:24.572681+0000","last_clean_scrub_stamp":"2023-11-24T07:57:24.572681+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:18:21.970465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.31","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-24T07:57:26.582759+0000","last_change":"2023-11-24T07:57:25.605974+0000","last_active":"2023-11-24T07:57:26.582759+0000","last_peered":"2023-11-24T07:57:26.582759+0000","last_clean":"2023-11-24T07:57:26.582759+0000","last_became_active":"2023-11-24T07:57:25.604012+0000","last_became_peered":"2023-11-24T07:57:25.604012+0000","last_unstale":"2023-11-24T07:57:26.582759+0000","last_undegraded":"2023-11-24T07:57:26.582759+0000","last_fullsized":"2023-11-24T07:57:26.582759+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:57:24.572681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:57:24.572681+0000","last_clean_scrub_stamp":"2023-11-24T07:57:24.572681+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T12:51:13.357718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.30","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-24T07:57:26.582547+0000","last_change":"2023-11-24T07:57:25.603887+0000","last_active":"2023-11-24T07:57:26.582547+0000","last_peered":"2023-11-24T07:57:26.582547+0000","last_clean":"2023-11-24T07:57:26.582547+0000","last_became_active":"2023-11-24T07:57:25.603484+0000","last_became_peered":"2023-11-24T07:57:25.603484+0000","last_unstale":"2023-11-24T07:57:26.582547+0000","last_undegraded":"2023-11-24T07:57:26.582547+0000","last_fullsized":"2023-11-24T07:57:26.582547+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:57:24.572681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:57:24.572681+0000","last_clean_scrub_stamp":"2023-11-24T07:57:24.572681+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T19:30:34.653712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.2f","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-24T07:57:26.583766+0000","last_change":"2023-11-24T07:57:25.617360+0000","last_active":"2023-11-24T07:57:26.583766+0000","last_peered":"2023-11-24T07:57:26.583766+0000","last_clean":"2023-11-24T07:57:26.583766+0000","last_became_active":"2023-11-24T07:57:25.617094+0000","last_became_peered":"2023-11-24T07:57:25.617094+0000","last_unstale":"2023-11-24T07:57:26.583766+0000","last_undegraded":"2023-11-24T07:57:26.583766+0000","last_fullsized":"2023-11-24T07:57:26.583766+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:57:24.572681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:57:24.572681+0000","last_clean_scrub_stamp":"2023-11-24T07:57:24.572681+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:15:10.655973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2e","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-24T07:57:27.498492+0000","last_change":"2023-11-24T07:57:25.597593+0000","last_active":"2023-11-24T07:57:27.498492+0000","last_peered":"2023-11-24T07:57:27.498492+0000","last_clean":"2023-11-24T07:57:27.498492+0000","last_became_active":"2023-11-24T07:57:25.597301+0000","last_became_peered":"2023-11-24T07:57:25.597301+0000","last_unstale":"2023-11-24T07:57:27.498492+0000","last_undegraded":"2023-11-24T07:57:27.498492+0000","last_fullsized":"2023-11-24T07:57:27.498492+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:57:24.572681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:57:24.572681+0000","last_clean_scrub_stamp":"2023-11-24T07:57:24.572681+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T17:40:49.956916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2d","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-24T07:57:27.498313+0000","last_change":"2023-11-24T07:57:25.598070+0000","last_active":"2023-11-24T07:57:27.498313+0000","last_peered":"2023-11-24T07:57:27.498313+0000","last_clean":"2023-11-24T07:57:27.498313+0000","last_became_active":"2023-11-24T07:57:25.597726+0000","last_became_peered":"2023-11-24T07:57:25.597726+0000","last_unstale":"2023-11-24T07:57:27.498313+0000","last_undegraded":"2023-11-24T07:57:27.498313+0000","last_fullsized":"2023-11-24T07:57:27.498313+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:57:24.572681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:57:24.572681+0000","last_clean_scrub_stamp":"2023-11-24T07:57:24.572681+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T14:49:20.210209+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2c","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-24T07:57:27.498975+0000","last_change":"2023-11-24T07:57:25.607256+0000","last_active":"2023-11-24T07:57:27.498975+0000","last_peered":"2023-11-24T07:57:27.498975+0000","last_clean":"2023-11-24T07:57:27.498975+0000","last_became_active":"2023-11-24T07:57:25.606931+0000","last_became_peered":"2023-11-24T07:57:25.606931+0000","last_unstale":"2023-11-24T07:57:27.498975+0000","last_undegraded":"2023-11-24T07:57:27.498975+0000","last_fullsized":"2023-11-24T07:57:27.498975+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:57:24.572681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:57:24.572681+0000","last_clean_scrub_stamp":"2023-11-24T07:57:24.572681+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T12:58:55.859434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.2b","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-24T07:57:26.583134+0000","last_change":"2023-11-24T07:57:25.603944+0000","last_active":"2023-11-24T07:57:26.583134+0000","last_peered":"2023-11-24T07:57:26.583134+0000","last_clean":"2023-11-24T07:57:26.583134+0000","last_became_active":"2023-11-24T07:57:25.603636+0000","last_became_peered":"2023-11-24T07:57:25.603636+0000","last_unstale":"2023-11-24T07:57:26.583134+0000","last_undegraded":"2023-11-24T07:57:26.583134+0000","last_fullsized":"2023-11-24T07:57:26.583134+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:57:24.572681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:57:24.572681+0000","last_clean_scrub_stamp":"2023-11-24T07:57:24.572681+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T15:47:12.125658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.2a","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-24T07:57:27.498828+0000","last_change":"2023-11-24T07:57:25.615209+0000","last_active":"2023-11-24T07:57:27.498828+0000","last_peered":"2023-11-24T07:57:27.498828+0000","last_clean":"2023-11-24T07:57:27.498828+0000","last_became_active":"2023-11-24T07:57:25.614789+0000","last_became_peered":"2023-11-24T07:57:25.614789+0000","last_unstale":"2023-11-24T07:57:27.498828+0000","last_undegraded":"2023-11-24T07:57:27.498828+0000","last_fullsized":"2023-11-24T07:57:27.498828+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:57:24.572681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:57:24.572681+0000","last_clean_scrub_stamp":"2023-11-24T07:57:24.572681+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T15:32:38.628605+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.29","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-24T07:57:27.498472+0000","last_change":"2023-11-24T07:57:25.596319+0000","last_active":"2023-11-24T07:57:27.498472+0000","last_peered":"2023-11-24T07:57:27.498472+0000","last_clean":"2023-11-24T07:57:27.498472+0000","last_became_active":"2023-11-24T07:57:25.596016+0000","last_became_peered":"2023-11-24T07:57:25.596016+0000","last_unstale":"2023-11-24T07:57:27.498472+0000","last_undegraded":"2023-11-24T07:57:27.498472+0000","last_fullsized":"2023-11-24T07:57:27.498472+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:57:24.572681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:57:24.572681+0000","last_clean_scrub_stamp":"2023-11-24T07:57:24.572681+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T13:25:29.169219+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.28","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-24T07:57:27.498409+0000","last_change":"2023-11-24T07:57:25.613820+0000","last_active":"2023-11-24T07:57:27.498409+0000","last_peered":"2023-11-24T07:57:27.498409+0000","last_clean":"2023-11-24T07:57:27.498409+0000","last_became_active":"2023-11-24T07:57:25.613589+0000","last_became_peered":"2023-11-24T07:57:25.613589+0000","last_unstale":"2023-11-24T07:57:27.498409+0000","last_undegraded":"2023-11-24T07:57:27.498409+0000","last_fullsized":"2023-11-24T07:57:27.498409+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:57:24.572681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:57:24.572681+0000","last_clean_scrub_stamp":"2023-11-24T07:57:24.572681+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T12:58:55.859434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.27","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-24T07:57:27.499878+0000","last_change":"2023-11-24T07:57:25.609636+0000","last_active":"2023-11-24T07:57:27.499878+0000","last_peered":"2023-11-24T07:57:27.499878+0000","last_clean":"2023-11-24T07:57:27.499878+0000","last_became_active":"2023-11-24T07:57:25.609368+0000","last_became_peered":"2023-11-24T07:57:25.609368+0000","last_unstale":"2023-11-24T07:57:27.499878+0000","last_undegraded":"2023-11-24T07:57:27.499878+0000","last_fullsized":"2023-11-24T07:57:27.499878+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:57:24.572681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:57:24.572681+0000","last_clean_scrub_stamp":"2023-11-24T07:57:24.572681+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T19:27:55.072502+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.26","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-24T07:57:27.499815+0000","last_change":"2023-11-24T07:57:25.606180+0000","last_active":"2023-11-24T07:57:27.499815+0000","last_peered":"2023-11-24T07:57:27.499815+0000","last_clean":"2023-11-24T07:57:27.499815+0000","last_became_active":"2023-11-24T07:57:25.605879+0000","last_became_peered":"2023-11-24T07:57:25.605879+0000","last_unstale":"2023-11-24T07:57:27.499815+0000","last_undegraded":"2023-11-24T07:57:27.499815+0000","last_fullsized":"2023-11-24T07:57:27.499815+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:57:24.572681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:57:24.572681+0000","last_clean_scrub_stamp":"2023-11-24T07:57:24.572681+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T15:46:37.491547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.25","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-24T07:57:26.584072+0000","last_change":"2023-11-24T07:57:25.605311+0000","last_active":"2023-11-24T07:57:26.584072+0000","last_peered":"2023-11-24T07:57:26.584072+0000","last_clean":"2023-11-24T07:57:26.584072+0000","last_became_active":"2023-11-24T07:57:25.605045+0000","last_became_peered":"2023-11-24T07:57:25.605045+0000","last_unstale":"2023-11-24T07:57:26.584072+0000","last_undegraded":"2023-11-24T07:57:26.584072+0000","last_fullsized":"2023-11-24T07:57:26.584072+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:57:24.572681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:57:24.572681+0000","last_clean_scrub_stamp":"2023-11-24T07:57:24.572681+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T15:32:38.628605+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.24","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-24T07:57:27.498555+0000","last_change":"2023-11-24T07:57:25.601259+0000","last_active":"2023-11-24T07:57:27.498555+0000","last_peered":"2023-11-24T07:57:27.498555+0000","last_clean":"2023-11-24T07:57:27.498555+0000","last_became_active":"2023-11-24T07:57:25.600941+0000","last_became_peered":"2023-11-24T07:57:25.600941+0000","last_unstale":"2023-11-24T07:57:27.498555+0000","last_undegraded":"2023-11-24T07:57:27.498555+0000","last_fullsized":"2023-11-24T07:57:27.498555+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:57:24.572681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:57:24.572681+0000","last_clean_scrub_stamp":"2023-11-24T07:57:24.572681+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T19:30:34.653712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.23","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-24T07:57:26.583176+0000","last_change":"2023-11-24T07:57:25.609706+0000","last_active":"2023-11-24T07:57:26.583176+0000","last_peered":"2023-11-24T07:57:26.583176+0000","last_clean":"2023-11-24T07:57:26.583176+0000","last_became_active":"2023-11-24T07:57:25.609419+0000","last_became_peered":"2023-11-24T07:57:25.609419+0000","last_unstale":"2023-11-24T07:57:26.583176+0000","last_undegraded":"2023-11-24T07:57:26.583176+0000","last_fullsized":"2023-11-24T07:57:26.583176+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:57:24.572681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:57:24.572681+0000","last_clean_scrub_stamp":"2023-11-24T07:57:24.572681+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T14:04:03.022069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.22","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-24T07:57:26.582928+0000","last_change":"2023-11-24T07:57:25.607107+0000","last_active":"2023-11-24T07:57:26.582928+0000","last_peered":"2023-11-24T07:57:26.582928+0000","last_clean":"2023-11-24T07:57:26.582928+0000","last_became_active":"2023-11-24T07:57:25.606797+0000","last_became_peered":"2023-11-24T07:57:25.606797+0000","last_unstale":"2023-11-24T07:57:26.582928+0000","last_undegraded":"2023-11-24T07:57:26.582928+0000","last_fullsized":"2023-11-24T07:57:26.582928+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:57:24.572681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:57:24.572681+0000","last_clean_scrub_stamp":"2023-11-24T07:57:24.572681+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T15:32:38.628605+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.21","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-24T07:57:27.499178+0000","last_change":"2023-11-24T07:57:25.605343+0000","last_active":"2023-11-24T07:57:27.499178+0000","last_peered":"2023-11-24T07:57:27.499178+0000","last_clean":"2023-11-24T07:57:27.499178+0000","last_became_active":"2023-11-24T07:57:25.605002+0000","last_became_peered":"2023-11-24T07:57:25.605002+0000","last_unstale":"2023-11-24T07:57:27.499178+0000","last_undegraded":"2023-11-24T07:57:27.499178+0000","last_fullsized":"2023-11-24T07:57:27.499178+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:57:24.572681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:57:24.572681+0000","last_clean_scrub_stamp":"2023-11-24T07:57:24.572681+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:18:21.970465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.20","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-24T07:57:26.583562+0000","last_change":"2023-11-24T07:57:25.615397+0000","last_active":"2023-11-24T07:57:26.583562+0000","last_peered":"2023-11-24T07:57:26.583562+0000","last_clean":"2023-11-24T07:57:26.583562+0000","last_became_active":"2023-11-24T07:57:25.615122+0000","last_became_peered":"2023-11-24T07:57:25.615122+0000","last_unstale":"2023-11-24T07:57:26.583562+0000","last_undegraded":"2023-11-24T07:57:26.583562+0000","last_fullsized":"2023-11-24T07:57:26.583562+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:57:24.572681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:57:24.572681+0000","last_clean_scrub_stamp":"2023-11-24T07:57:24.572681+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:32:55.195260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-24T07:57:27.498923+0000","last_change":"2023-11-24T07:57:25.612902+0000","last_active":"2023-11-24T07:57:27.498923+0000","last_peered":"2023-11-24T07:57:27.498923+0000","last_clean":"2023-11-24T07:57:27.498923+0000","last_became_active":"2023-11-24T07:57:25.612479+0000","last_became_peered":"2023-11-24T07:57:25.612479+0000","last_unstale":"2023-11-24T07:57:27.498923+0000","last_undegraded":"2023-11-24T07:57:27.498923+0000","last_fullsized":"2023-11-24T07:57:27.498923+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:57:24.572681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:57:24.572681+0000","last_clean_scrub_stamp":"2023-11-24T07:57:24.572681+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T16:47:55.590649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1f","version":"32'4","reported_seq":16,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-24T07:57:26.584058+0000","last_change":"2023-11-24T07:57:24.600719+0000","last_active":"2023-11-24T07:57:26.584058+0000","last_peered":"2023-11-24T07:57:26.584058+0000","last_clean":"2023-11-24T07:57:26.584058+0000","last_became_active":"2023-11-24T07:57:24.600406+0000","last_became_peered":"2023-11-24T07:57:24.600406+0000","last_unstale":"2023-11-24T07:57:26.584058+0000","last_undegraded":"2023-11-24T07:57:26.584058+0000","last_fullsized":"2023-11-24T07:57:26.584058+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:57:23.565116+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:57:23.565116+0000","last_clean_scrub_stamp":"2023-11-24T07:57:23.565116+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T14:20:43.593019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1f","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-24T07:57:27.498803+0000","last_change":"2023-11-24T07:57:25.600985+0000","last_active":"2023-11-24T07:57:27.498803+0000","last_peered":"2023-11-24T07:57:27.498803+0000","last_clean":"2023-11-24T07:57:27.498803+0000","last_became_active":"2023-11-24T07:57:25.600673+0000","last_became_peered":"2023-11-24T07:57:25.600673+0000","last_unstale":"2023-11-24T07:57:27.498803+0000","last_undegraded":"2023-11-24T07:57:27.498803+0000","last_fullsized":"2023-11-24T07:57:27.498803+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:57:24.572681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:57:24.572681+0000","last_clean_scrub_stamp":"2023-11-24T07:57:24.572681+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T19:13:40.799042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.1e","version":"32'1","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-24T07:57:27.499834+0000","last_change":"2023-11-24T07:57:24.597208+0000","last_active":"2023-11-24T07:57:27.499834+0000","last_peered":"2023-11-24T07:57:27.499834+0000","last_clean":"2023-11-24T07:57:27.499834+0000","last_became_active":"2023-11-24T07:57:24.596957+0000","last_became_peered":"2023-11-24T07:57:24.596957+0000","last_unstale":"2023-11-24T07:57:27.499834+0000","last_undegraded":"2023-11-24T07:57:27.499834+0000","last_fullsized":"2023-11-24T07:57:27.499834+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:57:23.565116+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:57:23.565116+0000","last_clean_scrub_stamp":"2023-11-24T07:57:23.565116+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T12:51:12.350152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-24T07:57:27.498872+0000","last_change":"2023-11-24T07:57:25.615058+0000","last_active":"2023-11-24T07:57:27.498872+0000","last_peered":"2023-11-24T07:57:27.498872+0000","last_clean":"2023-11-24T07:57:27.498872+0000","last_became_active":"2023-11-24T07:57:25.614773+0000","last_became_peered":"2023-11-24T07:57:25.614773+0000","last_unstale":"2023-11-24T07:57:27.498872+0000","last_undegraded":"2023-11-24T07:57:27.498872+0000","last_fullsized":"2023-11-24T07:57:27.498872+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:57:24.572681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:57:24.572681+0000","last_clean_scrub_stamp":"2023-11-24T07:57:24.572681+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T15:47:12.125658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.1d","version":"32'2","reported_seq":14,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-24T07:57:27.498444+0000","last_change":"2023-11-24T07:57:24.602623+0000","last_active":"2023-11-24T07:57:27.498444+0000","last_peered":"2023-11-24T07:57:27.498444+0000","last_clean":"2023-11-24T07:57:27.498444+0000","last_became_active":"2023-11-24T07:57:24.602295+0000","last_became_peered":"2023-11-24T07:57:24.602295+0000","last_unstale":"2023-11-24T07:57:27.498444+0000","last_undegraded":"2023-11-24T07:57:27.498444+0000","last_fullsized":"2023-11-24T07:57:27.498444+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:57:23.565116+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:57:23.565116+0000","last_clean_scrub_stamp":"2023-11-24T07:57:23.565116+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T15:12:48.897533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-24T07:57:27.499822+0000","last_change":"2023-11-24T07:57:25.604332+0000","last_active":"2023-11-24T07:57:27.499822+0000","last_peered":"2023-11-24T07:57:27.499822+0000","last_clean":"2023-11-24T07:57:27.499822+0000","last_became_active":"2023-11-24T07:57:25.603998+0000","last_became_peered":"2023-11-24T07:57:25.603998+0000","last_unstale":"2023-11-24T07:57:27.499822+0000","last_undegraded":"2023-11-24T07:57:27.499822+0000","last_fullsized":"2023-11-24T07:57:27.499822+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:57:24.572681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:57:24.572681+0000","last_clean_scrub_stamp":"2023-11-24T07:57:24.572681+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T11:10:18.640767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1c","version":"32'2","reported_seq":14,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-24T07:57:27.499815+0000","last_change":"2023-11-24T07:57:24.595626+0000","last_active":"2023-11-24T07:57:27.499815+0000","last_peered":"2023-11-24T07:57:27.499815+0000","last_clean":"2023-11-24T07:57:27.499815+0000","last_became_active":"2023-11-24T07:57:24.595343+0000","last_became_peered":"2023-11-24T07:57:24.595343+0000","last_unstale":"2023-11-24T07:57:27.499815+0000","last_undegraded":"2023-11-24T07:57:27.499815+0000","last_fullsized":"2023-11-24T07:57:27.499815+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:57:23.565116+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:57:23.565116+0000","last_clean_scrub_stamp":"2023-11-24T07:57:23.565116+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T19:13:39.791476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-24T07:57:27.498316+0000","last_change":"2023-11-24T07:57:25.611772+0000","last_active":"2023-11-24T07:57:27.498316+0000","last_peered":"2023-11-24T07:57:27.498316+0000","last_clean":"2023-11-24T07:57:27.498316+0000","last_became_active":"2023-11-24T07:57:25.611486+0000","last_became_peered":"2023-11-24T07:57:25.611486+0000","last_unstale":"2023-11-24T07:57:27.498316+0000","last_undegraded":"2023-11-24T07:57:27.498316+0000","last_fullsized":"2023-11-24T07:57:27.498316+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:57:24.572681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:57:24.572681+0000","last_clean_scrub_stamp":"2023-11-24T07:57:24.572681+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T15:12:49.905098+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1b","version":"32'1","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-24T07:57:27.498458+0000","last_change":"2023-11-24T07:57:24.589658+0000","last_active":"2023-11-24T07:57:27.498458+0000","last_peered":"2023-11-24T07:57:27.498458+0000","last_clean":"2023-11-24T07:57:27.498458+0000","last_became_active":"2023-11-24T07:57:24.589385+0000","last_became_peered":"2023-11-24T07:57:24.589385+0000","last_unstale":"2023-11-24T07:57:27.498458+0000","last_undegraded":"2023-11-24T07:57:27.498458+0000","last_fullsized":"2023-11-24T07:57:27.498458+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:57:23.565116+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:57:23.565116+0000","last_clean_scrub_stamp":"2023-11-24T07:57:23.565116+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T16:10:47.555733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-24T07:57:27.498658+0000","last_change":"2023-11-24T07:57:25.605184+0000","last_active":"2023-11-24T07:57:27.498658+0000","last_peered":"2023-11-24T07:57:27.498658+0000","last_clean":"2023-11-24T07:57:27.498658+0000","last_became_active":"2023-11-24T07:57:25.604881+0000","last_became_peered":"2023-11-24T07:57:25.604881+0000","last_unstale":"2023-11-24T07:57:27.498658+0000","last_undegraded":"2023-11-24T07:57:27.498658+0000","last_fullsized":"2023-11-24T07:57:27.498658+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:57:24.572681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:57:24.572681+0000","last_clean_scrub_stamp":"2023-11-24T07:57:24.572681+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T09:32:54.344241+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.1a","version":"32'1","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-24T07:57:27.498824+0000","last_change":"2023-11-24T07:57:24.595865+0000","last_active":"2023-11-24T07:57:27.498824+0000","last_peered":"2023-11-24T07:57:27.498824+0000","last_clean":"2023-11-24T07:57:27.498824+0000","last_became_active":"2023-11-24T07:57:24.595549+0000","last_became_peered":"2023-11-24T07:57:24.595549+0000","last_unstale":"2023-11-24T07:57:27.498824+0000","last_undegraded":"2023-11-24T07:57:27.498824+0000","last_fullsized":"2023-11-24T07:57:27.498824+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:57:23.565116+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:57:23.565116+0000","last_clean_scrub_stamp":"2023-11-24T07:57:23.565116+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T15:03:20.689993+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-24T07:57:27.499424+0000","last_change":"2023-11-24T07:57:25.604514+0000","last_active":"2023-11-24T07:57:27.499424+0000","last_peered":"2023-11-24T07:57:27.499424+0000","last_clean":"2023-11-24T07:57:27.499424+0000","last_became_active":"2023-11-24T07:57:25.604243+0000","last_became_peered":"2023-11-24T07:57:25.604243+0000","last_unstale":"2023-11-24T07:57:27.499424+0000","last_undegraded":"2023-11-24T07:57:27.499424+0000","last_fullsized":"2023-11-24T07:57:27.499424+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:57:24.572681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:57:24.572681+0000","last_clean_scrub_stamp":"2023-11-24T07:57:24.572681+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T15:25:09.940051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-24T07:57:27.498339+0000","last_change":"2023-11-24T07:57:24.600360+0000","last_active":"2023-11-24T07:57:27.498339+0000","last_peered":"2023-11-24T07:57:27.498339+0000","last_clean":"2023-11-24T07:57:27.498339+0000","last_became_active":"2023-11-24T07:57:24.600069+0000","last_became_peered":"2023-11-24T07:57:24.600069+0000","last_unstale":"2023-11-24T07:57:27.498339+0000","last_undegraded":"2023-11-24T07:57:27.498339+0000","last_fullsized":"2023-11-24T07:57:27.498339+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:57:23.565116+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:57:23.565116+0000","last_clean_scrub_stamp":"2023-11-24T07:57:23.565116+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T14:43:06.906996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-24T07:57:27.498321+0000","last_change":"2023-11-24T07:57:25.604299+0000","last_active":"2023-11-24T07:57:27.498321+0000","last_peered":"2023-11-24T07:57:27.498321+0000","last_clean":"2023-11-24T07:57:27.498321+0000","last_became_active":"2023-11-24T07:57:25.604033+0000","last_became_peered":"2023-11-24T07:57:25.604033+0000","last_unstale":"2023-11-24T07:57:27.498321+0000","last_undegraded":"2023-11-24T07:57:27.498321+0000","last_fullsized":"2023-11-24T07:57:27.498321+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:57:24.572681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:57:24.572681+0000","last_clean_scrub_stamp":"2023-11-24T07:57:24.572681+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T15:47:12.125658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-24T07:57:27.498908+0000","last_change":"2023-11-24T07:57:24.592424+0000","last_active":"2023-11-24T07:57:27.498908+0000","last_peered":"2023-11-24T07:57:27.498908+0000","last_clean":"2023-11-24T07:57:27.498908+0000","last_became_active":"2023-11-24T07:57:24.592091+0000","last_became_peered":"2023-11-24T07:57:24.592091+0000","last_unstale":"2023-11-24T07:57:27.498908+0000","last_undegraded":"2023-11-24T07:57:27.498908+0000","last_fullsized":"2023-11-24T07:57:27.498908+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:57:23.565116+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:57:23.565116+0000","last_clean_scrub_stamp":"2023-11-24T07:57:23.565116+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T11:43:02.671428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-24T07:57:26.583989+0000","last_change":"2023-11-24T07:57:25.605278+0000","last_active":"2023-11-24T07:57:26.583989+0000","last_peered":"2023-11-24T07:57:26.583989+0000","last_clean":"2023-11-24T07:57:26.583989+0000","last_became_active":"2023-11-24T07:57:25.604986+0000","last_became_peered":"2023-11-24T07:57:25.604986+0000","last_unstale":"2023-11-24T07:57:26.583989+0000","last_undegraded":"2023-11-24T07:57:26.583989+0000","last_fullsized":"2023-11-24T07:57:26.583989+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:57:24.572681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:57:24.572681+0000","last_clean_scrub_stamp":"2023-11-24T07:57:24.572681+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T14:00:49.762285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-24T07:57:26.584132+0000","last_change":"2023-11-24T07:57:24.587676+0000","last_active":"2023-11-24T07:57:26.584132+0000","last_peered":"2023-11-24T07:57:26.584132+0000","last_clean":"2023-11-24T07:57:26.584132+0000","last_became_active":"2023-11-24T07:57:24.587487+0000","last_became_peered":"2023-11-24T07:57:24.587487+0000","last_unstale":"2023-11-24T07:57:26.584132+0000","last_undegraded":"2023-11-24T07:57:26.584132+0000","last_fullsized":"2023-11-24T07:57:26.584132+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:57:23.565116+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:57:23.565116+0000","last_clean_scrub_stamp":"2023-11-24T07:57:23.565116+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T09:32:53.336675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-24T07:57:27.499752+0000","last_change":"2023-11-24T07:57:25.604618+0000","last_active":"2023-11-24T07:57:27.499752+0000","last_peered":"2023-11-24T07:57:27.499752+0000","last_clean":"2023-11-24T07:57:27.499752+0000","last_became_active":"2023-11-24T07:57:25.603754+0000","last_became_peered":"2023-11-24T07:57:25.603754+0000","last_unstale":"2023-11-24T07:57:27.499752+0000","last_undegraded":"2023-11-24T07:57:27.499752+0000","last_fullsized":"2023-11-24T07:57:27.499752+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:57:24.572681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:57:24.572681+0000","last_clean_scrub_stamp":"2023-11-24T07:57:24.572681+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T09:44:29.688267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-24T07:57:26.584309+0000","last_change":"2023-11-24T07:57:24.597830+0000","last_active":"2023-11-24T07:57:26.584309+0000","last_peered":"2023-11-24T07:57:26.584309+0000","last_clean":"2023-11-24T07:57:26.584309+0000","last_became_active":"2023-11-24T07:57:24.597491+0000","last_became_peered":"2023-11-24T07:57:24.597491+0000","last_unstale":"2023-11-24T07:57:26.584309+0000","last_undegraded":"2023-11-24T07:57:26.584309+0000","last_fullsized":"2023-11-24T07:57:26.584309+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:57:23.565116+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:57:23.565116+0000","last_clean_scrub_stamp":"2023-11-24T07:57:23.565116+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T11:43:02.671428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.14","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-24T07:57:27.499713+0000","last_change":"2023-11-24T07:57:25.606580+0000","last_active":"2023-11-24T07:57:27.499713+0000","last_peered":"2023-11-24T07:57:27.499713+0000","last_clean":"2023-11-24T07:57:27.499713+0000","last_became_active":"2023-11-24T07:57:25.606003+0000","last_became_peered":"2023-11-24T07:57:25.606003+0000","last_unstale":"2023-11-24T07:57:27.499713+0000","last_undegraded":"2023-11-24T07:57:27.499713+0000","last_fullsized":"2023-11-24T07:57:27.499713+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:57:24.572681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:57:24.572681+0000","last_clean_scrub_stamp":"2023-11-24T07:57:24.572681+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T08:54:41.853999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.15","version":"32'3","reported_seq":15,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-24T07:57:26.583811+0000","last_change":"2023-11-24T07:57:24.601458+0000","last_active":"2023-11-24T07:57:26.583811+0000","last_peered":"2023-11-24T07:57:26.583811+0000","last_clean":"2023-11-24T07:57:26.583811+0000","last_became_active":"2023-11-24T07:57:24.601087+0000","last_became_peered":"2023-11-24T07:57:24.601087+0000","last_unstale":"2023-11-24T07:57:26.583811+0000","last_undegraded":"2023-11-24T07:57:26.583811+0000","last_fullsized":"2023-11-24T07:57:26.583811+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:57:23.565116+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:57:23.565116+0000","last_clean_scrub_stamp":"2023-11-24T07:57:23.565116+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T08:54:40.846433+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.15","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-24T07:57:26.583962+0000","last_change":"2023-11-24T07:57:25.610172+0000","last_active":"2023-11-24T07:57:26.583962+0000","last_peered":"2023-11-24T07:57:26.583962+0000","last_clean":"2023-11-24T07:57:26.583962+0000","last_became_active":"2023-11-24T07:57:25.609792+0000","last_became_peered":"2023-11-24T07:57:25.609792+0000","last_unstale":"2023-11-24T07:57:26.583962+0000","last_undegraded":"2023-11-24T07:57:26.583962+0000","last_fullsized":"2023-11-24T07:57:26.583962+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:57:24.572681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:57:24.572681+0000","last_clean_scrub_stamp":"2023-11-24T07:57:24.572681+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:36:54.791587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.14","version":"32'6","reported_seq":18,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-24T07:57:27.499082+0000","last_change":"2023-11-24T07:57:24.595177+0000","last_active":"2023-11-24T07:57:27.499082+0000","last_peered":"2023-11-24T07:57:27.499082+0000","last_clean":"2023-11-24T07:57:27.499082+0000","last_became_active":"2023-11-24T07:57:24.594880+0000","last_became_peered":"2023-11-24T07:57:24.594880+0000","last_unstale":"2023-11-24T07:57:27.499082+0000","last_undegraded":"2023-11-24T07:57:27.499082+0000","last_fullsized":"2023-11-24T07:57:27.499082+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:57:23.565116+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:57:23.565116+0000","last_clean_scrub_stamp":"2023-11-24T07:57:23.565116+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T16:05:14.639876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-24T07:57:27.498346+0000","last_change":"2023-11-24T07:57:25.597947+0000","last_active":"2023-11-24T07:57:27.498346+0000","last_peered":"2023-11-24T07:57:27.498346+0000","last_clean":"2023-11-24T07:57:27.498346+0000","last_became_active":"2023-11-24T07:57:25.597634+0000","last_became_peered":"2023-11-24T07:57:25.597634+0000","last_unstale":"2023-11-24T07:57:27.498346+0000","last_undegraded":"2023-11-24T07:57:27.498346+0000","last_fullsized":"2023-11-24T07:57:27.498346+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:57:24.572681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:57:24.572681+0000","last_clean_scrub_stamp":"2023-11-24T07:57:24.572681+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T14:24:57.175517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.13","version":"32'2","reported_seq":14,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-24T07:57:27.498591+0000","last_change":"2023-11-24T07:57:24.600265+0000","last_active":"2023-11-24T07:57:27.498591+0000","last_peered":"2023-11-24T07:57:27.498591+0000","last_clean":"2023-11-24T07:57:27.498591+0000","last_became_active":"2023-11-24T07:57:24.599858+0000","last_became_peered":"2023-11-24T07:57:24.599858+0000","last_unstale":"2023-11-24T07:57:27.498591+0000","last_undegraded":"2023-11-24T07:57:27.498591+0000","last_fullsized":"2023-11-24T07:57:27.498591+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:57:23.565116+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:57:23.565116+0000","last_clean_scrub_stamp":"2023-11-24T07:57:23.565116+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T14:24:56.167951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-24T07:57:27.499668+0000","last_change":"2023-11-24T07:57:25.605157+0000","last_active":"2023-11-24T07:57:27.499668+0000","last_peered":"2023-11-24T07:57:27.499668+0000","last_clean":"2023-11-24T07:57:27.499668+0000","last_became_active":"2023-11-24T07:57:25.604895+0000","last_became_peered":"2023-11-24T07:57:25.604895+0000","last_unstale":"2023-11-24T07:57:27.499668+0000","last_undegraded":"2023-11-24T07:57:27.499668+0000","last_fullsized":"2023-11-24T07:57:27.499668+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:57:24.572681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:57:24.572681+0000","last_clean_scrub_stamp":"2023-11-24T07:57:24.572681+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:34:57.222018+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-24T07:57:27.499175+0000","last_change":"2023-11-24T07:57:24.597916+0000","last_active":"2023-11-24T07:57:27.499175+0000","last_peered":"2023-11-24T07:57:27.499175+0000","last_clean":"2023-11-24T07:57:27.499175+0000","last_became_active":"2023-11-24T07:57:24.597625+0000","last_became_peered":"2023-11-24T07:57:24.597625+0000","last_unstale":"2023-11-24T07:57:27.499175+0000","last_undegraded":"2023-11-24T07:57:27.499175+0000","last_fullsized":"2023-11-24T07:57:27.499175+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:57:23.565116+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:57:23.565116+0000","last_clean_scrub_stamp":"2023-11-24T07:57:23.565116+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T15:32:37.621039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-24T07:57:27.499392+0000","last_change":"2023-11-24T07:57:25.595758+0000","last_active":"2023-11-24T07:57:27.499392+0000","last_peered":"2023-11-24T07:57:27.499392+0000","last_clean":"2023-11-24T07:57:27.499392+0000","last_became_active":"2023-11-24T07:57:25.595488+0000","last_became_peered":"2023-11-24T07:57:25.595488+0000","last_unstale":"2023-11-24T07:57:27.499392+0000","last_undegraded":"2023-11-24T07:57:27.499392+0000","last_fullsized":"2023-11-24T07:57:27.499392+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:57:24.572681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:57:24.572681+0000","last_clean_scrub_stamp":"2023-11-24T07:57:24.572681+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T13:39:16.521602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-24T07:57:27.499388+0000","last_change":"2023-11-24T07:57:24.595527+0000","last_active":"2023-11-24T07:57:27.499388+0000","last_peered":"2023-11-24T07:57:27.499388+0000","last_clean":"2023-11-24T07:57:27.499388+0000","last_became_active":"2023-11-24T07:57:24.595229+0000","last_became_peered":"2023-11-24T07:57:24.595229+0000","last_unstale":"2023-11-24T07:57:27.499388+0000","last_undegraded":"2023-11-24T07:57:27.499388+0000","last_fullsized":"2023-11-24T07:57:27.499388+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:57:23.565116+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:57:23.565116+0000","last_clean_scrub_stamp":"2023-11-24T07:57:23.565116+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T14:24:56.167951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-24T07:57:26.583629+0000","last_change":"2023-11-24T07:57:25.617331+0000","last_active":"2023-11-24T07:57:26.583629+0000","last_peered":"2023-11-24T07:57:26.583629+0000","last_clean":"2023-11-24T07:57:26.583629+0000","last_became_active":"2023-11-24T07:57:25.617033+0000","last_became_peered":"2023-11-24T07:57:25.617033+0000","last_unstale":"2023-11-24T07:57:26.583629+0000","last_undegraded":"2023-11-24T07:57:26.583629+0000","last_fullsized":"2023-11-24T07:57:26.583629+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:57:24.572681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:57:24.572681+0000","last_clean_scrub_stamp":"2023-11-24T07:57:24.572681+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T14:01:46.665356+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.10","version":"32'8","reported_seq":20,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-24T07:57:27.498079+0000","last_change":"2023-11-24T07:57:24.599748+0000","last_active":"2023-11-24T07:57:27.498079+0000","last_peered":"2023-11-24T07:57:27.498079+0000","last_clean":"2023-11-24T07:57:27.498079+0000","last_became_active":"2023-11-24T07:57:24.599456+0000","last_became_peered":"2023-11-24T07:57:24.599456+0000","last_unstale":"2023-11-24T07:57:27.498079+0000","last_undegraded":"2023-11-24T07:57:27.498079+0000","last_fullsized":"2023-11-24T07:57:27.498079+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:57:23.565116+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:57:23.565116+0000","last_clean_scrub_stamp":"2023-11-24T07:57:23.565116+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T19:40:29.710446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.e","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-24T07:57:26.584123+0000","last_change":"2023-11-24T07:57:25.610621+0000","last_active":"2023-11-24T07:57:26.584123+0000","last_peered":"2023-11-24T07:57:26.584123+0000","last_clean":"2023-11-24T07:57:26.584123+0000","last_became_active":"2023-11-24T07:57:25.610341+0000","last_became_peered":"2023-11-24T07:57:25.610341+0000","last_unstale":"2023-11-24T07:57:26.584123+0000","last_undegraded":"2023-11-24T07:57:26.584123+0000","last_fullsized":"2023-11-24T07:57:26.584123+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:57:24.572681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:57:24.572681+0000","last_clean_scrub_stamp":"2023-11-24T07:57:24.572681+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:32:46.923992+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.f","version":"32'1","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-24T07:57:27.498608+0000","last_change":"2023-11-24T07:57:24.600217+0000","last_active":"2023-11-24T07:57:27.498608+0000","last_peered":"2023-11-24T07:57:27.498608+0000","last_clean":"2023-11-24T07:57:27.498608+0000","last_became_active":"2023-11-24T07:57:24.599831+0000","last_became_peered":"2023-11-24T07:57:24.599831+0000","last_unstale":"2023-11-24T07:57:27.498608+0000","last_undegraded":"2023-11-24T07:57:27.498608+0000","last_fullsized":"2023-11-24T07:57:27.498608+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:57:23.565116+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:57:23.565116+0000","last_clean_scrub_stamp":"2023-11-24T07:57:23.565116+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T14:47:45.788163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-24T07:57:27.498862+0000","last_change":"2023-11-24T07:57:25.604503+0000","last_active":"2023-11-24T07:57:27.498862+0000","last_peered":"2023-11-24T07:57:27.498862+0000","last_clean":"2023-11-24T07:57:27.498862+0000","last_became_active":"2023-11-24T07:57:25.604215+0000","last_became_peered":"2023-11-24T07:57:25.604215+0000","last_unstale":"2023-11-24T07:57:27.498862+0000","last_undegraded":"2023-11-24T07:57:27.498862+0000","last_fullsized":"2023-11-24T07:57:27.498862+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:57:24.572681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:57:24.572681+0000","last_clean_scrub_stamp":"2023-11-24T07:57:24.572681+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T14:49:20.210209+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.e","version":"32'2","reported_seq":14,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-24T07:57:27.498400+0000","last_change":"2023-11-24T07:57:24.600226+0000","last_active":"2023-11-24T07:57:27.498400+0000","last_peered":"2023-11-24T07:57:27.498400+0000","last_clean":"2023-11-24T07:57:27.498400+0000","last_became_active":"2023-11-24T07:57:24.599943+0000","last_became_peered":"2023-11-24T07:57:24.599943+0000","last_unstale":"2023-11-24T07:57:27.498400+0000","last_undegraded":"2023-11-24T07:57:27.498400+0000","last_fullsized":"2023-11-24T07:57:27.498400+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:57:23.565116+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:57:23.565116+0000","last_clean_scrub_stamp":"2023-11-24T07:57:23.565116+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T14:33:18.564905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-24T07:57:26.583937+0000","last_change":"2023-11-24T07:57:25.598876+0000","last_active":"2023-11-24T07:57:26.583937+0000","last_peered":"2023-11-24T07:57:26.583937+0000","last_clean":"2023-11-24T07:57:26.583937+0000","last_became_active":"2023-11-24T07:57:25.598514+0000","last_became_peered":"2023-11-24T07:57:25.598514+0000","last_unstale":"2023-11-24T07:57:26.583937+0000","last_undegraded":"2023-11-24T07:57:26.583937+0000","last_fullsized":"2023-11-24T07:57:26.583937+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:57:24.572681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:57:24.572681+0000","last_clean_scrub_stamp":"2023-11-24T07:57:24.572681+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T12:51:13.357718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.d","version":"32'1","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-24T07:57:26.584012+0000","last_change":"2023-11-24T07:57:24.597442+0000","last_active":"2023-11-24T07:57:26.584012+0000","last_peered":"2023-11-24T07:57:26.584012+0000","last_clean":"2023-11-24T07:57:26.584012+0000","last_became_active":"2023-11-24T07:57:24.596992+0000","last_became_peered":"2023-11-24T07:57:24.596992+0000","last_unstale":"2023-11-24T07:57:26.584012+0000","last_undegraded":"2023-11-24T07:57:26.584012+0000","last_fullsized":"2023-11-24T07:57:26.584012+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:57:23.565116+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:57:23.565116+0000","last_clean_scrub_stamp":"2023-11-24T07:57:23.565116+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T11:43:38.072322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.d","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-24T07:57:27.498699+0000","last_change":"2023-11-24T07:57:25.608066+0000","last_active":"2023-11-24T07:57:27.498699+0000","last_peered":"2023-11-24T07:57:27.498699+0000","last_clean":"2023-11-24T07:57:27.498699+0000","last_became_active":"2023-11-24T07:57:25.607583+0000","last_became_peered":"2023-11-24T07:57:25.607583+0000","last_unstale":"2023-11-24T07:57:27.498699+0000","last_undegraded":"2023-11-24T07:57:27.498699+0000","last_fullsized":"2023-11-24T07:57:27.498699+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:57:24.572681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:57:24.572681+0000","last_clean_scrub_stamp":"2023-11-24T07:57:24.572681+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T13:05:00.624062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-24T07:57:26.582860+0000","last_change":"2023-11-24T07:57:24.588501+0000","last_active":"2023-11-24T07:57:26.582860+0000","last_peered":"2023-11-24T07:57:26.582860+0000","last_clean":"2023-11-24T07:57:26.582860+0000","last_became_active":"2023-11-24T07:57:24.588135+0000","last_became_peered":"2023-11-24T07:57:24.588135+0000","last_unstale":"2023-11-24T07:57:26.582860+0000","last_undegraded":"2023-11-24T07:57:26.582860+0000","last_fullsized":"2023-11-24T07:57:26.582860+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:57:23.565116+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:57:23.565116+0000","last_clean_scrub_stamp":"2023-11-24T07:57:23.565116+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T16:47:54.583083+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-24T07:57:27.498182+0000","last_change":"2023-11-24T07:57:25.606996+0000","last_active":"2023-11-24T07:57:27.498182+0000","last_peered":"2023-11-24T07:57:27.498182+0000","last_clean":"2023-11-24T07:57:27.498182+0000","last_became_active":"2023-11-24T07:57:25.606495+0000","last_became_peered":"2023-11-24T07:57:25.606495+0000","last_unstale":"2023-11-24T07:57:27.498182+0000","last_undegraded":"2023-11-24T07:57:27.498182+0000","last_fullsized":"2023-11-24T07:57:27.498182+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:57:24.572681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:57:24.572681+0000","last_clean_scrub_stamp":"2023-11-24T07:57:24.572681+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T16:10:48.563298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.b","version":"32'2","reported_seq":14,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-24T07:57:26.584060+0000","last_change":"2023-11-24T07:57:24.589230+0000","last_active":"2023-11-24T07:57:26.584060+0000","last_peered":"2023-11-24T07:57:26.584060+0000","last_clean":"2023-11-24T07:57:26.584060+0000","last_became_active":"2023-11-24T07:57:24.588791+0000","last_became_peered":"2023-11-24T07:57:24.588791+0000","last_unstale":"2023-11-24T07:57:26.584060+0000","last_undegraded":"2023-11-24T07:57:26.584060+0000","last_fullsized":"2023-11-24T07:57:26.584060+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:57:23.565116+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:57:23.565116+0000","last_clean_scrub_stamp":"2023-11-24T07:57:23.565116+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T12:58:54.851868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-24T07:57:26.584361+0000","last_change":"2023-11-24T07:57:25.604199+0000","last_active":"2023-11-24T07:57:26.584361+0000","last_peered":"2023-11-24T07:57:26.584361+0000","last_clean":"2023-11-24T07:57:26.584361+0000","last_became_active":"2023-11-24T07:57:25.603979+0000","last_became_peered":"2023-11-24T07:57:25.603979+0000","last_unstale":"2023-11-24T07:57:26.584361+0000","last_undegraded":"2023-11-24T07:57:26.584361+0000","last_fullsized":"2023-11-24T07:57:26.584361+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:57:24.572681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:57:24.572681+0000","last_clean_scrub_stamp":"2023-11-24T07:57:24.572681+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T14:04:03.022069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-24T07:57:27.498922+0000","last_change":"2023-11-24T07:57:24.597707+0000","last_active":"2023-11-24T07:57:27.498922+0000","last_peered":"2023-11-24T07:57:27.498922+0000","last_clean":"2023-11-24T07:57:27.498922+0000","last_became_active":"2023-11-24T07:57:24.597441+0000","last_became_peered":"2023-11-24T07:57:24.597441+0000","last_unstale":"2023-11-24T07:57:27.498922+0000","last_undegraded":"2023-11-24T07:57:27.498922+0000","last_fullsized":"2023-11-24T07:57:27.498922+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:57:23.565116+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:57:23.565116+0000","last_clean_scrub_stamp":"2023-11-24T07:57:23.565116+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T18:23:24.083250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-24T07:57:27.498774+0000","last_change":"2023-11-24T07:57:25.609619+0000","last_active":"2023-11-24T07:57:27.498774+0000","last_peered":"2023-11-24T07:57:27.498774+0000","last_clean":"2023-11-24T07:57:27.498774+0000","last_became_active":"2023-11-24T07:57:25.609261+0000","last_became_peered":"2023-11-24T07:57:25.609261+0000","last_unstale":"2023-11-24T07:57:27.498774+0000","last_undegraded":"2023-11-24T07:57:27.498774+0000","last_fullsized":"2023-11-24T07:57:27.498774+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:57:24.572681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:57:24.572681+0000","last_clean_scrub_stamp":"2023-11-24T07:57:24.572681+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T11:43:03.678993+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-24T07:57:27.498617+0000","last_change":"2023-11-24T07:57:24.589176+0000","last_active":"2023-11-24T07:57:27.498617+0000","last_peered":"2023-11-24T07:57:27.498617+0000","last_clean":"2023-11-24T07:57:27.498617+0000","last_became_active":"2023-11-24T07:57:24.588886+0000","last_became_peered":"2023-11-24T07:57:24.588886+0000","last_unstale":"2023-11-24T07:57:27.498617+0000","last_undegraded":"2023-11-24T07:57:27.498617+0000","last_fullsized":"2023-11-24T07:57:27.498617+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:57:23.565116+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:57:23.565116+0000","last_clean_scrub_stamp":"2023-11-24T07:57:23.565116+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T16:28:17.765064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-24T07:57:27.498976+0000","last_change":"2023-11-24T07:57:25.607219+0000","last_active":"2023-11-24T07:57:27.498976+0000","last_peered":"2023-11-24T07:57:27.498976+0000","last_clean":"2023-11-24T07:57:27.498976+0000","last_became_active":"2023-11-24T07:57:25.606930+0000","last_became_peered":"2023-11-24T07:57:25.606930+0000","last_unstale":"2023-11-24T07:57:27.498976+0000","last_undegraded":"2023-11-24T07:57:27.498976+0000","last_fullsized":"2023-11-24T07:57:27.498976+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:57:24.572681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:57:24.572681+0000","last_clean_scrub_stamp":"2023-11-24T07:57:24.572681+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T13:05:00.624062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-24T07:57:26.583579+0000","last_change":"2023-11-24T07:57:24.597642+0000","last_active":"2023-11-24T07:57:26.583579+0000","last_peered":"2023-11-24T07:57:26.583579+0000","last_clean":"2023-11-24T07:57:26.583579+0000","last_became_active":"2023-11-24T07:57:24.597339+0000","last_became_peered":"2023-11-24T07:57:24.597339+0000","last_unstale":"2023-11-24T07:57:26.583579+0000","last_undegraded":"2023-11-24T07:57:26.583579+0000","last_fullsized":"2023-11-24T07:57:26.583579+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:57:23.565116+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:57:23.565116+0000","last_clean_scrub_stamp":"2023-11-24T07:57:23.565116+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T15:35:45.139676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-24T07:57:27.498618+0000","last_change":"2023-11-24T07:57:25.609017+0000","last_active":"2023-11-24T07:57:27.498618+0000","last_peered":"2023-11-24T07:57:27.498618+0000","last_clean":"2023-11-24T07:57:27.498618+0000","last_became_active":"2023-11-24T07:57:25.608616+0000","last_became_peered":"2023-11-24T07:57:25.608616+0000","last_unstale":"2023-11-24T07:57:27.498618+0000","last_undegraded":"2023-11-24T07:57:27.498618+0000","last_fullsized":"2023-11-24T07:57:27.498618+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:57:24.572681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:57:24.572681+0000","last_clean_scrub_stamp":"2023-11-24T07:57:24.572681+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T15:54:26.161043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.7","version":"32'2","reported_seq":14,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-24T07:57:27.498697+0000","last_change":"2023-11-24T07:57:24.596645+0000","last_active":"2023-11-24T07:57:27.498697+0000","last_peered":"2023-11-24T07:57:27.498697+0000","last_clean":"2023-11-24T07:57:27.498697+0000","last_became_active":"2023-11-24T07:57:24.596327+0000","last_became_peered":"2023-11-24T07:57:24.596327+0000","last_unstale":"2023-11-24T07:57:27.498697+0000","last_undegraded":"2023-11-24T07:57:27.498697+0000","last_fullsized":"2023-11-24T07:57:27.498697+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:57:23.565116+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:57:23.565116+0000","last_clean_scrub_stamp":"2023-11-24T07:57:23.565116+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T14:49:19.202644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-24T07:57:26.583923+0000","last_change":"2023-11-24T07:57:25.612875+0000","last_active":"2023-11-24T07:57:26.583923+0000","last_peered":"2023-11-24T07:57:26.583923+0000","last_clean":"2023-11-24T07:57:26.583923+0000","last_became_active":"2023-11-24T07:57:25.612599+0000","last_became_peered":"2023-11-24T07:57:25.612599+0000","last_unstale":"2023-11-24T07:57:26.583923+0000","last_undegraded":"2023-11-24T07:57:26.583923+0000","last_fullsized":"2023-11-24T07:57:26.583923+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:57:24.572681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:57:24.572681+0000","last_clean_scrub_stamp":"2023-11-24T07:57:24.572681+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T13:42:15.906045+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.6","version":"33'1","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-24T07:57:27.619117+0000","last_change":"2023-11-24T07:57:24.600554+0000","last_active":"2023-11-24T07:57:27.619117+0000","last_peered":"2023-11-24T07:57:27.619117+0000","last_clean":"2023-11-24T07:57:27.619117+0000","last_became_active":"2023-11-24T07:57:24.600257+0000","last_became_peered":"2023-11-24T07:57:24.600257+0000","last_unstale":"2023-11-24T07:57:27.619117+0000","last_undegraded":"2023-11-24T07:57:27.619117+0000","last_fullsized":"2023-11-24T07:57:27.619117+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:57:23.565116+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:57:23.565116+0000","last_clean_scrub_stamp":"2023-11-24T07:57:23.565116+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T16:47:54.583083+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-24T07:57:27.498903+0000","last_change":"2023-11-24T07:57:25.604810+0000","last_active":"2023-11-24T07:57:27.498903+0000","last_peered":"2023-11-24T07:57:27.498903+0000","last_clean":"2023-11-24T07:57:27.498903+0000","last_became_active":"2023-11-24T07:57:25.604557+0000","last_became_peered":"2023-11-24T07:57:25.604557+0000","last_unstale":"2023-11-24T07:57:27.498903+0000","last_undegraded":"2023-11-24T07:57:27.498903+0000","last_fullsized":"2023-11-24T07:57:27.498903+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:57:24.572681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:57:24.572681+0000","last_clean_scrub_stamp":"2023-11-24T07:57:24.572681+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T12:14:04.002318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-24T07:57:27.499172+0000","last_change":"2023-11-24T07:57:24.597887+0000","last_active":"2023-11-24T07:57:27.499172+0000","last_peered":"2023-11-24T07:57:27.499172+0000","last_clean":"2023-11-24T07:57:27.499172+0000","last_became_active":"2023-11-24T07:57:24.597549+0000","last_became_peered":"2023-11-24T07:57:24.597549+0000","last_unstale":"2023-11-24T07:57:27.499172+0000","last_undegraded":"2023-11-24T07:57:27.499172+0000","last_fullsized":"2023-11-24T07:57:27.499172+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:57:23.565116+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:57:23.565116+0000","last_clean_scrub_stamp":"2023-11-24T07:57:23.565116+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:34:29.085812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-24T07:57:26.582793+0000","last_change":"2023-11-24T07:57:25.606098+0000","last_active":"2023-11-24T07:57:26.582793+0000","last_peered":"2023-11-24T07:57:26.582793+0000","last_clean":"2023-11-24T07:57:26.582793+0000","last_became_active":"2023-11-24T07:57:25.603917+0000","last_became_peered":"2023-11-24T07:57:25.603917+0000","last_unstale":"2023-11-24T07:57:26.582793+0000","last_undegraded":"2023-11-24T07:57:26.582793+0000","last_fullsized":"2023-11-24T07:57:26.582793+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:57:24.572681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:57:24.572681+0000","last_clean_scrub_stamp":"2023-11-24T07:57:24.572681+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T19:13:40.799042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-24T07:57:27.498955+0000","last_change":"2023-11-24T07:57:24.596194+0000","last_active":"2023-11-24T07:57:27.498955+0000","last_peered":"2023-11-24T07:57:27.498955+0000","last_clean":"2023-11-24T07:57:27.498955+0000","last_became_active":"2023-11-24T07:57:24.595854+0000","last_became_peered":"2023-11-24T07:57:24.595854+0000","last_unstale":"2023-11-24T07:57:27.498955+0000","last_undegraded":"2023-11-24T07:57:27.498955+0000","last_fullsized":"2023-11-24T07:57:27.498955+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:57:23.565116+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:57:23.565116+0000","last_clean_scrub_stamp":"2023-11-24T07:57:23.565116+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T12:14:02.994752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-24T07:57:27.499699+0000","last_change":"2023-11-24T07:57:25.604191+0000","last_active":"2023-11-24T07:57:27.499699+0000","last_peered":"2023-11-24T07:57:27.499699+0000","last_clean":"2023-11-24T07:57:27.499699+0000","last_became_active":"2023-11-24T07:57:25.603570+0000","last_became_peered":"2023-11-24T07:57:25.603570+0000","last_unstale":"2023-11-24T07:57:27.499699+0000","last_undegraded":"2023-11-24T07:57:27.499699+0000","last_fullsized":"2023-11-24T07:57:27.499699+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:57:24.572681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:57:24.572681+0000","last_clean_scrub_stamp":"2023-11-24T07:57:24.572681+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T11:48:20.813946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-24T07:57:27.498769+0000","last_change":"2023-11-24T07:57:24.599222+0000","last_active":"2023-11-24T07:57:27.498769+0000","last_peered":"2023-11-24T07:57:27.498769+0000","last_clean":"2023-11-24T07:57:27.498769+0000","last_became_active":"2023-11-24T07:57:24.598910+0000","last_became_peered":"2023-11-24T07:57:24.598910+0000","last_unstale":"2023-11-24T07:57:27.498769+0000","last_undegraded":"2023-11-24T07:57:27.498769+0000","last_fullsized":"2023-11-24T07:57:27.498769+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:57:23.565116+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:57:23.565116+0000","last_clean_scrub_stamp":"2023-11-24T07:57:23.565116+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T15:03:20.689993+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-24T07:57:27.498551+0000","last_change":"2023-11-24T07:57:25.603783+0000","last_active":"2023-11-24T07:57:27.498551+0000","last_peered":"2023-11-24T07:57:27.498551+0000","last_clean":"2023-11-24T07:57:27.498551+0000","last_became_active":"2023-11-24T07:57:25.603464+0000","last_became_peered":"2023-11-24T07:57:25.603464+0000","last_unstale":"2023-11-24T07:57:27.498551+0000","last_undegraded":"2023-11-24T07:57:27.498551+0000","last_fullsized":"2023-11-24T07:57:27.498551+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:57:24.572681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:57:24.572681+0000","last_clean_scrub_stamp":"2023-11-24T07:57:24.572681+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T14:04:03.022069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-24T07:57:27.499235+0000","last_change":"2023-11-24T07:57:24.599854+0000","last_active":"2023-11-24T07:57:27.499235+0000","last_peered":"2023-11-24T07:57:27.499235+0000","last_clean":"2023-11-24T07:57:27.499235+0000","last_became_active":"2023-11-24T07:57:24.599310+0000","last_became_peered":"2023-11-24T07:57:24.599310+0000","last_unstale":"2023-11-24T07:57:27.499235+0000","last_undegraded":"2023-11-24T07:57:27.499235+0000","last_fullsized":"2023-11-24T07:57:27.499235+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:57:23.565116+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:57:23.565116+0000","last_clean_scrub_stamp":"2023-11-24T07:57:23.565116+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:34:29.085812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-24T07:57:26.583711+0000","last_change":"2023-11-24T07:57:25.613275+0000","last_active":"2023-11-24T07:57:26.583711+0000","last_peered":"2023-11-24T07:57:26.583711+0000","last_clean":"2023-11-24T07:57:26.583711+0000","last_became_active":"2023-11-24T07:57:25.612934+0000","last_became_peered":"2023-11-24T07:57:25.612934+0000","last_unstale":"2023-11-24T07:57:26.583711+0000","last_undegraded":"2023-11-24T07:57:26.583711+0000","last_fullsized":"2023-11-24T07:57:26.583711+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:57:24.572681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:57:24.572681+0000","last_clean_scrub_stamp":"2023-11-24T07:57:24.572681+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T19:14:43.303209+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-24T07:57:27.498639+0000","last_change":"2023-11-24T07:57:24.597190+0000","last_active":"2023-11-24T07:57:27.498639+0000","last_peered":"2023-11-24T07:57:27.498639+0000","last_clean":"2023-11-24T07:57:27.498639+0000","last_became_active":"2023-11-24T07:57:24.596910+0000","last_became_peered":"2023-11-24T07:57:24.596910+0000","last_unstale":"2023-11-24T07:57:27.498639+0000","last_undegraded":"2023-11-24T07:57:27.498639+0000","last_fullsized":"2023-11-24T07:57:27.498639+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:57:23.565116+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:57:23.565116+0000","last_clean_scrub_stamp":"2023-11-24T07:57:23.565116+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T12:58:54.851868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-24T07:57:26.582741+0000","last_change":"2023-11-24T07:57:25.592562+0000","last_active":"2023-11-24T07:57:26.582741+0000","last_peered":"2023-11-24T07:57:26.582741+0000","last_clean":"2023-11-24T07:57:26.582741+0000","last_became_active":"2023-11-24T07:57:25.592287+0000","last_became_peered":"2023-11-24T07:57:25.592287+0000","last_unstale":"2023-11-24T07:57:26.582741+0000","last_undegraded":"2023-11-24T07:57:26.582741+0000","last_fullsized":"2023-11-24T07:57:26.582741+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:57:24.572681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:57:24.572681+0000","last_clean_scrub_stamp":"2023-11-24T07:57:24.572681+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T09:32:54.344241+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-24T07:57:26.583911+0000","last_change":"2023-11-24T07:57:24.599399+0000","last_active":"2023-11-24T07:57:26.583911+0000","last_peered":"2023-11-24T07:57:26.583911+0000","last_clean":"2023-11-24T07:57:26.583911+0000","last_became_active":"2023-11-24T07:57:24.599096+0000","last_became_peered":"2023-11-24T07:57:24.599096+0000","last_unstale":"2023-11-24T07:57:26.583911+0000","last_undegraded":"2023-11-24T07:57:26.583911+0000","last_fullsized":"2023-11-24T07:57:26.583911+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:57:23.565116+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:57:23.565116+0000","last_clean_scrub_stamp":"2023-11-24T07:57:23.565116+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T18:02:41.581821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":176,"reported_epoch":32,"state":"active+clean","last_fresh":"2023-11-24T07:57:25.591791+0000","last_change":"2023-11-24T07:56:19.668905+0000","last_active":"2023-11-24T07:57:25.591791+0000","last_peered":"2023-11-24T07:57:25.591791+0000","last_clean":"2023-11-24T07:57:25.591791+0000","last_became_active":"2023-11-24T07:56:19.668601+0000","last_became_peered":"2023-11-24T07:56:19.668601+0000","last_unstale":"2023-11-24T07:57:25.591791+0000","last_undegraded":"2023-11-24T07:57:25.591791+0000","last_fullsized":"2023-11-24T07:57:25.591791+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:56:18.657844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:56:18.657844+0000","last_clean_scrub_stamp":"2023-11-24T07:56:18.657844+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T11:51:53.828455+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":5,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574866,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27396,"kb_used_data":368,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344444,"statfs":{"total":96636764160,"available":96608710656,"internally_reserved":0,"allocated":376832,"data_stored":146120,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 07:57:19 2023","interfaces":[{"interface":"back","average":{"1min":0.55700000000000005,"5min":0.55700000000000005,"15min":0.55700000000000005},"min":{"1min":0.32800000000000001,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":1.413,"5min":1.413,"15min":1.413},"last":0.54100000000000004},{"interface":"front","average":{"1min":0.55400000000000005,"5min":0.55400000000000005,"15min":0.55400000000000005},"min":{"1min":0.41099999999999998,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":0.81000000000000005,"5min":0.81000000000000005,"15min":0.81000000000000005},"last":0.64300000000000002}]},{"osd":1,"last update":"Fri Nov 24 07:57:19 2023","interfaces":[{"interface":"back","average":{"1min":0.84899999999999998,"5min":0.84899999999999998,"15min":0.84899999999999998},"min":{"1min":0.33600000000000002,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":6.1070000000000002,"5min":6.1070000000000002,"15min":6.1070000000000002},"last":0.42899999999999999},{"interface":"front","average":{"1min":0.57999999999999996,"5min":0.57999999999999996,"15min":0.57999999999999996},"min":{"1min":0.35199999999999998,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":1.5920000000000001,"5min":1.5920000000000001,"15min":1.5920000000000001},"last":0.628}]},{"osd":2,"last update":"Fri Nov 24 07:57:19 2023","interfaces":[{"interface":"back","average":{"1min":0.627,"5min":0.627,"15min":0.627},"min":{"1min":0.33800000000000002,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":1.613,"5min":1.613,"15min":1.613},"last":0.48399999999999999},{"interface":"front","average":{"1min":0.79200000000000004,"5min":0.79200000000000004,"15min":0.79200000000000004},"min":{"1min":0.41699999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":4.5220000000000002,"5min":4.5220000000000002,"15min":4.5220000000000002},"last":0.46300000000000002}]},{"osd":3,"last update":"Fri Nov 24 07:57:19 2023","interfaces":[{"interface":"back","average":{"1min":0.80600000000000005,"5min":0.80600000000000005,"15min":0.80600000000000005},"min":{"1min":0.38500000000000001,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":4.6760000000000002,"5min":4.6760000000000002,"15min":4.6760000000000002},"last":0.61299999999999999},{"interface":"front","average":{"1min":0.94299999999999995,"5min":0.94299999999999995,"15min":0.94299999999999995},"min":{"1min":0.505,"5min":0.505,"15min":0.505},"max":{"1min":4.6539999999999999,"5min":4.6539999999999999,"15min":4.6539999999999999},"last":0.57599999999999996}]},{"osd":4,"last update":"Fri Nov 24 07:57:19 2023","interfaces":[{"interface":"back","average":{"1min":0.56799999999999995,"5min":0.56799999999999995,"15min":0.56799999999999995},"min":{"1min":0.378,"5min":0.378,"15min":0.378},"max":{"1min":0.84699999999999998,"5min":0.84699999999999998,"15min":0.84699999999999998},"last":0.55500000000000005},{"interface":"front","average":{"1min":0.66700000000000004,"5min":0.66700000000000004,"15min":0.66700000000000004},"min":{"1min":0.46100000000000002,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":1.629,"5min":1.629,"15min":1.629},"last":0.40699999999999997}]},{"osd":5,"last update":"Fri Nov 24 07:57:19 2023","interfaces":[{"interface":"back","average":{"1min":0.54700000000000004,"5min":0.54700000000000004,"15min":0.54700000000000004},"min":{"1min":0.34499999999999997,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.875,"5min":0.875,"15min":0.875},"last":0.52100000000000002},{"interface":"front","average":{"1min":0.623,"5min":0.623,"15min":0.623},"min":{"1min":0.41499999999999998,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":0.90900000000000003,"5min":0.90900000000000003,"15min":0.90900000000000003},"last":0.505}]},{"osd":6,"last update":"Fri Nov 24 07:57:19 2023","interfaces":[{"interface":"back","average":{"1min":0.61099999999999999,"5min":0.61099999999999999,"15min":0.61099999999999999},"min":{"1min":0.438,"5min":0.438,"15min":0.438},"max":{"1min":0.80800000000000005,"5min":0.80800000000000005,"15min":0.80800000000000005},"last":0.59899999999999998},{"interface":"front","average":{"1min":0.67000000000000004,"5min":0.67000000000000004,"15min":0.67000000000000004},"min":{"1min":0.48599999999999999,"5min":0.48599999999999999,"15min":0.48599999999999999},"max":{"1min":0.98399999999999999,"5min":0.98399999999999999,"15min":0.98399999999999999},"last":0.65300000000000002}]}]},{"osd":6,"up_from":13,"seq":55834574867,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27396,"kb_used_data":376,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344444,"statfs":{"total":96636764160,"available":96608710656,"internally_reserved":0,"allocated":385024,"data_stored":147710,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 07:57:18 2023","interfaces":[{"interface":"back","average":{"1min":0.55000000000000004,"5min":0.55000000000000004,"15min":0.55000000000000004},"min":{"1min":0.28399999999999997,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":0.85099999999999998,"5min":0.85099999999999998,"15min":0.85099999999999998},"last":0.33300000000000002},{"interface":"front","average":{"1min":0.51600000000000001,"5min":0.51600000000000001,"15min":0.51600000000000001},"min":{"1min":0.311,"5min":0.311,"15min":0.311},"max":{"1min":0.69999999999999996,"5min":0.69999999999999996,"15min":0.69999999999999996},"last":0.36199999999999999}]},{"osd":1,"last update":"Fri Nov 24 07:57:18 2023","interfaces":[{"interface":"back","average":{"1min":0.51200000000000001,"5min":0.51200000000000001,"15min":0.51200000000000001},"min":{"1min":0.33800000000000002,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.78200000000000003,"5min":0.78200000000000003,"15min":0.78200000000000003},"last":0.55700000000000005},{"interface":"front","average":{"1min":0.61299999999999999,"5min":0.61299999999999999,"15min":0.61299999999999999},"min":{"1min":0.44700000000000001,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":1.0069999999999999,"5min":1.0069999999999999,"15min":1.0069999999999999},"last":0.49099999999999999}]},{"osd":2,"last update":"Fri Nov 24 07:57:18 2023","interfaces":[{"interface":"back","average":{"1min":0.57499999999999996,"5min":0.57499999999999996,"15min":0.57499999999999996},"min":{"1min":0.34300000000000003,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.878,"5min":0.878,"15min":0.878},"last":0.63100000000000001},{"interface":"front","average":{"1min":0.59299999999999997,"5min":0.59299999999999997,"15min":0.59299999999999997},"min":{"1min":0.47099999999999997,"5min":0.47099999999999997,"15min":0.47099999999999997},"max":{"1min":0.82399999999999995,"5min":0.82399999999999995,"15min":0.82399999999999995},"last":0.53500000000000003}]},{"osd":3,"last update":"Fri Nov 24 07:57:18 2023","interfaces":[{"interface":"back","average":{"1min":0.56799999999999995,"5min":0.56799999999999995,"15min":0.56799999999999995},"min":{"1min":0.40000000000000002,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":0.876,"5min":0.876,"15min":0.876},"last":0.59999999999999998},{"interface":"front","average":{"1min":0.55600000000000005,"5min":0.55600000000000005,"15min":0.55600000000000005},"min":{"1min":0.379,"5min":0.379,"15min":0.379},"max":{"1min":0.68799999999999994,"5min":0.68799999999999994,"15min":0.68799999999999994},"last":0.73099999999999998}]},{"osd":4,"last update":"Fri Nov 24 07:57:18 2023","interfaces":[{"interface":"back","average":{"1min":0.51700000000000002,"5min":0.51700000000000002,"15min":0.51700000000000002},"min":{"1min":0.33000000000000002,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.70599999999999996,"5min":0.70599999999999996,"15min":0.70599999999999996},"last":0.57899999999999996},{"interface":"front","average":{"1min":0.60199999999999998,"5min":0.60199999999999998,"15min":0.60199999999999998},"min":{"1min":0.35199999999999998,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":1.087,"5min":1.087,"15min":1.087},"last":0.70999999999999996}]},{"osd":5,"last update":"Fri Nov 24 07:57:18 2023","interfaces":[{"interface":"back","average":{"1min":0.53100000000000003,"5min":0.53100000000000003,"15min":0.53100000000000003},"min":{"1min":0.30199999999999999,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.84999999999999998,"5min":0.84999999999999998,"15min":0.84999999999999998},"last":0.69099999999999995},{"interface":"front","average":{"1min":0.61199999999999999,"5min":0.61199999999999999,"15min":0.61199999999999999},"min":{"1min":0.39200000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.97199999999999998,"5min":0.97199999999999998,"15min":0.97199999999999998},"last":0.65500000000000003}]},{"osd":7,"last update":"Fri Nov 24 07:57:18 2023","interfaces":[{"interface":"back","average":{"1min":0.61799999999999999,"5min":0.61799999999999999,"15min":0.61799999999999999},"min":{"1min":0.49299999999999999,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":0.81499999999999995,"5min":0.81499999999999995,"15min":0.81499999999999995},"last":0.67700000000000005},{"interface":"front","average":{"1min":0.67000000000000004,"5min":0.67000000000000004,"15min":0.67000000000000004},"min":{"1min":0.497,"5min":0.497,"15min":0.497},"max":{"1min":0.92700000000000005,"5min":0.92700000000000005,"15min":0.92700000000000005},"last":0.75600000000000001}]}]},{"osd":1,"up_from":13,"seq":55834574868,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27968,"kb_used_data":948,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343872,"statfs":{"total":96636764160,"available":96608124928,"internally_reserved":0,"allocated":970752,"data_stored":736536,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 07:57:22 2023","interfaces":[{"interface":"back","average":{"1min":0.36399999999999999,"5min":0.36399999999999999,"15min":0.36399999999999999},"min":{"1min":0.20200000000000001,"5min":0.20200000000000001,"15min":0.20200000000000001},"max":{"1min":0.70899999999999996,"5min":0.70899999999999996,"15min":0.70899999999999996},"last":0.28299999999999997},{"interface":"front","average":{"1min":0.39800000000000002,"5min":0.39800000000000002,"15min":0.39800000000000002},"min":{"1min":0.214,"5min":0.214,"15min":0.214},"max":{"1min":0.82499999999999996,"5min":0.82499999999999996,"15min":0.82499999999999996},"last":0.39100000000000001}]},{"osd":2,"last update":"Fri Nov 24 07:57:22 2023","interfaces":[{"interface":"back","average":{"1min":0.373,"5min":0.373,"15min":0.373},"min":{"1min":0.22700000000000001,"5min":0.22700000000000001,"15min":0.22700000000000001},"max":{"1min":0.60999999999999999,"5min":0.60999999999999999,"15min":0.60999999999999999},"last":0.30099999999999999},{"interface":"front","average":{"1min":0.41299999999999998,"5min":0.41299999999999998,"15min":0.41299999999999998},"min":{"1min":0.26800000000000002,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":0.621,"5min":0.621,"15min":0.621},"last":0.376}]},{"osd":3,"last update":"Fri Nov 24 07:57:22 2023","interfaces":[{"interface":"back","average":{"1min":0.44400000000000001,"5min":0.44400000000000001,"15min":0.44400000000000001},"min":{"1min":0.32100000000000001,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.66500000000000004,"5min":0.66500000000000004,"15min":0.66500000000000004},"last":0.33300000000000002},{"interface":"front","average":{"1min":0.42799999999999999,"5min":0.42799999999999999,"15min":0.42799999999999999},"min":{"1min":0.28499999999999998,"5min":0.28499999999999998,"15min":0.28499999999999998},"max":{"1min":0.76300000000000001,"5min":0.76300000000000001,"15min":0.76300000000000001},"last":0.56200000000000006}]},{"osd":4,"last update":"Fri Nov 24 07:57:22 2023","interfaces":[{"interface":"back","average":{"1min":0.61099999999999999,"5min":0.61099999999999999,"15min":0.61099999999999999},"min":{"1min":0.43099999999999999,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":0.79800000000000004,"5min":0.79800000000000004,"15min":0.79800000000000004},"last":0.51000000000000001},{"interface":"front","average":{"1min":0.60599999999999998,"5min":0.60599999999999998,"15min":0.60599999999999998},"min":{"1min":0.41999999999999998,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":0.82199999999999995,"5min":0.82199999999999995,"15min":0.82199999999999995},"last":0.52600000000000002}]},{"osd":5,"last update":"Fri Nov 24 07:57:22 2023","interfaces":[{"interface":"back","average":{"1min":0.72499999999999998,"5min":0.72499999999999998,"15min":0.72499999999999998},"min":{"1min":0.37,"5min":0.37,"15min":0.37},"max":{"1min":2.7269999999999999,"5min":2.7269999999999999,"15min":2.7269999999999999},"last":0.625},{"interface":"front","average":{"1min":0.64500000000000002,"5min":0.64500000000000002,"15min":0.64500000000000002},"min":{"1min":0.44700000000000001,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":0.84199999999999997,"5min":0.84199999999999997,"15min":0.84199999999999997},"last":0.45900000000000002}]},{"osd":6,"last update":"Fri Nov 24 07:57:22 2023","interfaces":[{"interface":"back","average":{"1min":0.65800000000000003,"5min":0.65800000000000003,"15min":0.65800000000000003},"min":{"1min":0.47499999999999998,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":0.98199999999999998,"5min":0.98199999999999998,"15min":0.98199999999999998},"last":0.48999999999999999},{"interface":"front","average":{"1min":0.61699999999999999,"5min":0.61699999999999999,"15min":0.61699999999999999},"min":{"1min":0.42899999999999999,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":0.90700000000000003,"5min":0.90700000000000003,"15min":0.90700000000000003},"last":0.69499999999999995}]},{"osd":7,"last update":"Fri Nov 24 07:57:22 2023","interfaces":[{"interface":"back","average":{"1min":0.63700000000000001,"5min":0.63700000000000001,"15min":0.63700000000000001},"min":{"1min":0.40999999999999998,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":1.0649999999999999,"5min":1.0649999999999999,"15min":1.0649999999999999},"last":0.66400000000000003},{"interface":"front","average":{"1min":0.67000000000000004,"5min":0.67000000000000004,"15min":0.67000000000000004},"min":{"1min":0.442,"5min":0.442,"15min":0.442},"max":{"1min":0.88800000000000001,"5min":0.88800000000000001,"15min":0.88800000000000001},"last":0.54500000000000004}]}]},{"osd":0,"up_from":13,"seq":55834574868,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27396,"kb_used_data":368,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344444,"statfs":{"total":96636764160,"available":96608710656,"internally_reserved":0,"allocated":376832,"data_stored":146188,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Nov 24 07:57:18 2023","interfaces":[{"interface":"back","average":{"1min":0.71299999999999997,"5min":0.71299999999999997,"15min":0.71299999999999997},"min":{"1min":0.23499999999999999,"5min":0.23499999999999999,"15min":0.23499999999999999},"max":{"1min":6.3550000000000004,"5min":6.3550000000000004,"15min":6.3550000000000004},"last":0.371},{"interface":"front","average":{"1min":1.7290000000000001,"5min":1.7290000000000001,"15min":1.7290000000000001},"min":{"1min":0.30599999999999999,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":22.916,"5min":22.916,"15min":22.916},"last":0.39200000000000002}]},{"osd":2,"last update":"Fri Nov 24 07:57:18 2023","interfaces":[{"interface":"back","average":{"1min":0.69999999999999996,"5min":0.69999999999999996,"15min":0.69999999999999996},"min":{"1min":0.22800000000000001,"5min":0.22800000000000001,"15min":0.22800000000000001},"max":{"1min":6.3899999999999997,"5min":6.3899999999999997,"15min":6.3899999999999997},"last":0.26200000000000001},{"interface":"front","average":{"1min":1.0629999999999999,"5min":1.0629999999999999,"15min":1.0629999999999999},"min":{"1min":0.26800000000000002,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":11.93,"5min":11.93,"15min":11.93},"last":0.40799999999999997}]},{"osd":3,"last update":"Fri Nov 24 07:57:18 2023","interfaces":[{"interface":"back","average":{"1min":0.41999999999999998,"5min":0.41999999999999998,"15min":0.41999999999999998},"min":{"1min":0.28399999999999997,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":1.155,"5min":1.155,"15min":1.155},"last":0.34399999999999997},{"interface":"front","average":{"1min":1.127,"5min":1.127,"15min":1.127},"min":{"1min":0.32300000000000001,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":12.116,"5min":12.116,"15min":12.116},"last":0.318}]},{"osd":4,"last update":"Fri Nov 24 07:57:18 2023","interfaces":[{"interface":"back","average":{"1min":0.64800000000000002,"5min":0.64800000000000002,"15min":0.64800000000000002},"min":{"1min":0.40500000000000003,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":0.97899999999999998,"5min":0.97899999999999998,"15min":0.97899999999999998},"last":0.56000000000000005},{"interface":"front","average":{"1min":1.9339999999999999,"5min":1.9339999999999999,"15min":1.9339999999999999},"min":{"1min":0.44400000000000001,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":22.963999999999999,"5min":22.963999999999999,"15min":22.963999999999999},"last":0.49299999999999999}]},{"osd":5,"last update":"Fri Nov 24 07:57:18 2023","interfaces":[{"interface":"back","average":{"1min":1.2929999999999999,"5min":1.2929999999999999,"15min":1.2929999999999999},"min":{"1min":0.40200000000000002,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":12.084,"5min":12.084,"15min":12.084},"last":0.51000000000000001},{"interface":"front","average":{"1min":0.67800000000000005,"5min":0.67800000000000005,"15min":0.67800000000000005},"min":{"1min":0.45200000000000001,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":1.073,"5min":1.073,"15min":1.073},"last":0.57999999999999996}]},{"osd":6,"last update":"Fri Nov 24 07:57:18 2023","interfaces":[{"interface":"back","average":{"1min":0.69099999999999995,"5min":0.69099999999999995,"15min":0.69099999999999995},"min":{"1min":0.42799999999999999,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":1.0580000000000001,"5min":1.0580000000000001,"15min":1.0580000000000001},"last":0.48299999999999998},{"interface":"front","average":{"1min":1.3140000000000001,"5min":1.3140000000000001,"15min":1.3140000000000001},"min":{"1min":0.41899999999999998,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":12.045,"5min":12.045,"15min":12.045},"last":0.58799999999999997}]},{"osd":7,"last update":"Fri Nov 24 07:57:18 2023","interfaces":[{"interface":"back","average":{"1min":0.65600000000000003,"5min":0.65600000000000003,"15min":0.65600000000000003},"min":{"1min":0.45400000000000001,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":1.038,"5min":1.038,"15min":1.038},"last":0.53700000000000003},{"interface":"front","average":{"1min":1.9630000000000001,"5min":1.9630000000000001,"15min":1.9630000000000001},"min":{"1min":0.49099999999999999,"5min":0.49099999999999999,"15min":0.49099999999999999},"max":{"1min":22.995000000000001,"5min":22.995000000000001,"15min":22.995000000000001},"last":0.52500000000000002}]}]},{"osd":2,"up_from":13,"seq":55834574868,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27396,"kb_used_data":368,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344444,"statfs":{"total":96636764160,"available":96608710656,"internally_reserved":0,"allocated":376832,"data_stored":146132,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 07:57:17 2023","interfaces":[{"interface":"back","average":{"1min":0.371,"5min":0.371,"15min":0.371},"min":{"1min":0.22500000000000001,"5min":0.22500000000000001,"15min":0.22500000000000001},"max":{"1min":0.58099999999999996,"5min":0.58099999999999996,"15min":0.58099999999999996},"last":0.34499999999999997},{"interface":"front","average":{"1min":0.40200000000000002,"5min":0.40200000000000002,"15min":0.40200000000000002},"min":{"1min":0.27400000000000002,"5min":0.27400000000000002,"15min":0.27400000000000002},"max":{"1min":0.54100000000000004,"5min":0.54100000000000004,"15min":0.54100000000000004},"last":0.46300000000000002}]},{"osd":1,"last update":"Fri Nov 24 07:57:17 2023","interfaces":[{"interface":"back","average":{"1min":0.378,"5min":0.378,"15min":0.378},"min":{"1min":0.23499999999999999,"5min":0.23499999999999999,"15min":0.23499999999999999},"max":{"1min":0.59199999999999997,"5min":0.59199999999999997,"15min":0.59199999999999997},"last":0.28799999999999998},{"interface":"front","average":{"1min":0.41599999999999998,"5min":0.41599999999999998,"15min":0.41599999999999998},"min":{"1min":0.25600000000000001,"5min":0.25600000000000001,"15min":0.25600000000000001},"max":{"1min":0.627,"5min":0.627,"15min":0.627},"last":0.39600000000000002}]},{"osd":3,"last update":"Fri Nov 24 07:57:17 2023","interfaces":[{"interface":"back","average":{"1min":0.41099999999999998,"5min":0.41099999999999998,"15min":0.41099999999999998},"min":{"1min":0.308,"5min":0.308,"15min":0.308},"max":{"1min":0.53300000000000003,"5min":0.53300000000000003,"15min":0.53300000000000003},"last":0.432},{"interface":"front","average":{"1min":0.41399999999999998,"5min":0.41399999999999998,"15min":0.41399999999999998},"min":{"1min":0.27500000000000002,"5min":0.27500000000000002,"15min":0.27500000000000002},"max":{"1min":0.64600000000000002,"5min":0.64600000000000002,"15min":0.64600000000000002},"last":0.47899999999999998}]},{"osd":4,"last update":"Fri Nov 24 07:57:17 2023","interfaces":[{"interface":"back","average":{"1min":0.60799999999999998,"5min":0.60799999999999998,"15min":0.60799999999999998},"min":{"1min":0.42999999999999999,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":0.81599999999999995,"5min":0.81599999999999995,"15min":0.81599999999999995},"last":0.74099999999999999},{"interface":"front","average":{"1min":0.59999999999999998,"5min":0.59999999999999998,"15min":0.59999999999999998},"min":{"1min":0.41599999999999998,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":0.79600000000000004,"5min":0.79600000000000004,"15min":0.79600000000000004},"last":0.63200000000000001}]},{"osd":5,"last update":"Fri Nov 24 07:57:17 2023","interfaces":[{"interface":"back","average":{"1min":0.621,"5min":0.621,"15min":0.621},"min":{"1min":0.44900000000000001,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":0.84899999999999998,"5min":0.84899999999999998,"15min":0.84899999999999998},"last":0.65700000000000003},{"interface":"front","average":{"1min":0.71499999999999997,"5min":0.71499999999999997,"15min":0.71499999999999997},"min":{"1min":0.433,"5min":0.433,"15min":0.433},"max":{"1min":2.1699999999999999,"5min":2.1699999999999999,"15min":2.1699999999999999},"last":0.79600000000000004}]},{"osd":6,"last update":"Fri Nov 24 07:57:17 2023","interfaces":[{"interface":"back","average":{"1min":0.66700000000000004,"5min":0.66700000000000004,"15min":0.66700000000000004},"min":{"1min":0.47399999999999998,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":0.83599999999999997,"5min":0.83599999999999997,"15min":0.83599999999999997},"last":0.755},{"interface":"front","average":{"1min":0.67100000000000004,"5min":0.67100000000000004,"15min":0.67100000000000004},"min":{"1min":0.496,"5min":0.496,"15min":0.496},"max":{"1min":0.88500000000000001,"5min":0.88500000000000001,"15min":0.88500000000000001},"last":0.78100000000000003}]},{"osd":7,"last update":"Fri Nov 24 07:57:17 2023","interfaces":[{"interface":"back","average":{"1min":0.66200000000000003,"5min":0.66200000000000003,"15min":0.66200000000000003},"min":{"1min":0.443,"5min":0.443,"15min":0.443},"max":{"1min":0.85999999999999999,"5min":0.85999999999999999,"15min":0.85999999999999999},"last":0.60599999999999998},{"interface":"front","average":{"1min":0.68000000000000005,"5min":0.68000000000000005,"15min":0.68000000000000005},"min":{"1min":0.495,"5min":0.495,"15min":0.495},"max":{"1min":0.86399999999999999,"5min":0.86399999999999999,"15min":0.86399999999999999},"last":0.70899999999999996}]}]},{"osd":3,"up_from":13,"seq":55834574868,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27408,"kb_used_data":380,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344432,"statfs":{"total":96636764160,"available":96608698368,"internally_reserved":0,"allocated":389120,"data_stored":146874,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 07:57:20 2023","interfaces":[{"interface":"back","average":{"1min":0.35599999999999998,"5min":0.35599999999999998,"15min":0.35599999999999998},"min":{"1min":0.222,"5min":0.222,"15min":0.222},"max":{"1min":0.84799999999999998,"5min":0.84799999999999998,"15min":0.84799999999999998},"last":0.373},{"interface":"front","average":{"1min":0.36699999999999999,"5min":0.36699999999999999,"15min":0.36699999999999999},"min":{"1min":0.23699999999999999,"5min":0.23699999999999999,"15min":0.23699999999999999},"max":{"1min":0.873,"5min":0.873,"15min":0.873},"last":0.40200000000000002}]},{"osd":1,"last update":"Fri Nov 24 07:57:20 2023","interfaces":[{"interface":"back","average":{"1min":0.375,"5min":0.375,"15min":0.375},"min":{"1min":0.25700000000000001,"5min":0.25700000000000001,"15min":0.25700000000000001},"max":{"1min":0.92000000000000004,"5min":0.92000000000000004,"15min":0.92000000000000004},"last":0.41799999999999998},{"interface":"front","average":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"min":{"1min":0.248,"5min":0.248,"15min":0.248},"max":{"1min":0.81699999999999995,"5min":0.81699999999999995,"15min":0.81699999999999995},"last":0.46700000000000003}]},{"osd":2,"last update":"Fri Nov 24 07:57:20 2023","interfaces":[{"interface":"back","average":{"1min":0.36099999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"min":{"1min":0.24399999999999999,"5min":0.24399999999999999,"15min":0.24399999999999999},"max":{"1min":0.88500000000000001,"5min":0.88500000000000001,"15min":0.88500000000000001},"last":0.43099999999999999},{"interface":"front","average":{"1min":0.46800000000000003,"5min":0.46800000000000003,"15min":0.46800000000000003},"min":{"1min":0.31900000000000001,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.98099999999999998,"5min":0.98099999999999998,"15min":0.98099999999999998},"last":0.45100000000000001}]},{"osd":4,"last update":"Fri Nov 24 07:57:20 2023","interfaces":[{"interface":"back","average":{"1min":0.79700000000000004,"5min":0.79700000000000004,"15min":0.79700000000000004},"min":{"1min":0.33500000000000002,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":5.7350000000000003,"5min":5.7350000000000003,"15min":5.7350000000000003},"last":0.59399999999999997},{"interface":"front","average":{"1min":0.58499999999999996,"5min":0.58499999999999996,"15min":0.58499999999999996},"min":{"1min":0.44500000000000001,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":1.27,"5min":1.27,"15min":1.27},"last":0.76300000000000001}]},{"osd":5,"last update":"Fri Nov 24 07:57:20 2023","interfaces":[{"interface":"back","average":{"1min":0.58799999999999997,"5min":0.58799999999999997,"15min":0.58799999999999997},"min":{"1min":0.40699999999999997,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":1.4430000000000001,"5min":1.4430000000000001,"15min":1.4430000000000001},"last":0.66400000000000003},{"interface":"front","average":{"1min":0.627,"5min":0.627,"15min":0.627},"min":{"1min":0.377,"5min":0.377,"15min":0.377},"max":{"1min":1.4239999999999999,"5min":1.4239999999999999,"15min":1.4239999999999999},"last":0.79900000000000004}]},{"osd":6,"last update":"Fri Nov 24 07:57:20 2023","interfaces":[{"interface":"back","average":{"1min":0.62,"5min":0.62,"15min":0.62},"min":{"1min":0.434,"5min":0.434,"15min":0.434},"max":{"1min":1.355,"5min":1.355,"15min":1.355},"last":0.82599999999999996},{"interface":"front","average":{"1min":0.84599999999999997,"5min":0.84599999999999997,"15min":0.84599999999999997},"min":{"1min":0.40999999999999998,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":5.6479999999999997,"5min":5.6479999999999997,"15min":5.6479999999999997},"last":0.71599999999999997}]},{"osd":7,"last update":"Fri Nov 24 07:57:20 2023","interfaces":[{"interface":"back","average":{"1min":0.66600000000000004,"5min":0.66600000000000004,"15min":0.66600000000000004},"min":{"1min":0.42299999999999999,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":1.51,"5min":1.51,"15min":1.51},"last":0.751},{"interface":"front","average":{"1min":0.84199999999999997,"5min":0.84199999999999997,"15min":0.84199999999999997},"min":{"1min":0.45600000000000002,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":5.7069999999999999,"5min":5.7069999999999999,"15min":5.7069999999999999},"last":0.625}]}]},{"osd":4,"up_from":13,"seq":55834574867,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27400,"kb_used_data":380,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344440,"statfs":{"total":96636764160,"available":96608706560,"internally_reserved":0,"allocated":389120,"data_stored":147732,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 07:57:21 2023","interfaces":[{"interface":"back","average":{"1min":0.58099999999999996,"5min":0.58099999999999996,"15min":0.58099999999999996},"min":{"1min":0.37,"5min":0.37,"15min":0.37},"max":{"1min":0.79300000000000004,"5min":0.79300000000000004,"15min":0.79300000000000004},"last":0.53100000000000003},{"interface":"front","average":{"1min":0.54300000000000004,"5min":0.54300000000000004,"15min":0.54300000000000004},"min":{"1min":0.33600000000000002,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":1.0249999999999999,"5min":1.0249999999999999,"15min":1.0249999999999999},"last":0.61899999999999999}]},{"osd":1,"last update":"Fri Nov 24 07:57:21 2023","interfaces":[{"interface":"back","average":{"1min":0.55200000000000005,"5min":0.55200000000000005,"15min":0.55200000000000005},"min":{"1min":0.40000000000000002,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":0.86399999999999999,"5min":0.86399999999999999,"15min":0.86399999999999999},"last":0.67400000000000004},{"interface":"front","average":{"1min":0.55100000000000005,"5min":0.55100000000000005,"15min":0.55100000000000005},"min":{"1min":0.36099999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.98699999999999999,"5min":0.98699999999999999,"15min":0.98699999999999999},"last":0.60299999999999998}]},{"osd":2,"last update":"Fri Nov 24 07:57:21 2023","interfaces":[{"interface":"back","average":{"1min":0.56100000000000005,"5min":0.56100000000000005,"15min":0.56100000000000005},"min":{"1min":0.35799999999999998,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.81100000000000005,"5min":0.81100000000000005,"15min":0.81100000000000005},"last":0.51100000000000001},{"interface":"front","average":{"1min":0.61299999999999999,"5min":0.61299999999999999,"15min":0.61299999999999999},"min":{"1min":0.38100000000000001,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.89200000000000002,"5min":0.89200000000000002,"15min":0.89200000000000002},"last":0.69399999999999995}]},{"osd":3,"last update":"Fri Nov 24 07:57:21 2023","interfaces":[{"interface":"back","average":{"1min":0.58499999999999996,"5min":0.58499999999999996,"15min":0.58499999999999996},"min":{"1min":0.439,"5min":0.439,"15min":0.439},"max":{"1min":0.89700000000000002,"5min":0.89700000000000002,"15min":0.89700000000000002},"last":0.55600000000000005},{"interface":"front","average":{"1min":0.63100000000000001,"5min":0.63100000000000001,"15min":0.63100000000000001},"min":{"1min":0.40300000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.96199999999999997,"5min":0.96199999999999997,"15min":0.96199999999999997},"last":0.65600000000000003}]},{"osd":5,"last update":"Fri Nov 24 07:57:21 2023","interfaces":[{"interface":"back","average":{"1min":0.45500000000000002,"5min":0.45500000000000002,"15min":0.45500000000000002},"min":{"1min":0.29799999999999999,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":0.70499999999999996,"5min":0.70499999999999996,"15min":0.70499999999999996},"last":0.29499999999999998},{"interface":"front","average":{"1min":0.61899999999999999,"5min":0.61899999999999999,"15min":0.61899999999999999},"min":{"1min":0.439,"5min":0.439,"15min":0.439},"max":{"1min":0.86399999999999999,"5min":0.86399999999999999,"15min":0.86399999999999999},"last":0.38300000000000001}]},{"osd":6,"last update":"Fri Nov 24 07:57:21 2023","interfaces":[{"interface":"back","average":{"1min":0.58599999999999997,"5min":0.58599999999999997,"15min":0.58599999999999997},"min":{"1min":0.35499999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.76800000000000002,"5min":0.76800000000000002,"15min":0.76800000000000002},"last":0.41899999999999998},{"interface":"front","average":{"1min":0.61399999999999999,"5min":0.61399999999999999,"15min":0.61399999999999999},"min":{"1min":0.436,"5min":0.436,"15min":0.436},"max":{"1min":1.095,"5min":1.095,"15min":1.095},"last":0.46899999999999997}]},{"osd":7,"last update":"Fri Nov 24 07:57:21 2023","interfaces":[{"interface":"back","average":{"1min":0.57299999999999995,"5min":0.57299999999999995,"15min":0.57299999999999995},"min":{"1min":0.35099999999999998,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.89300000000000002,"5min":0.89300000000000002,"15min":0.89300000000000002},"last":0.434},{"interface":"front","average":{"1min":0.66300000000000003,"5min":0.66300000000000003,"15min":0.66300000000000003},"min":{"1min":0.41099999999999998,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":0.91500000000000004,"5min":0.91500000000000004,"15min":0.91500000000000004},"last":0.34699999999999998}]}]},{"osd":5,"up_from":13,"seq":55834574868,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27976,"kb_used_data":948,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343864,"statfs":{"total":96636764160,"available":96608116736,"internally_reserved":0,"allocated":970752,"data_stored":737048,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 07:57:17 2023","interfaces":[{"interface":"back","average":{"1min":0.56200000000000006,"5min":0.56200000000000006,"15min":0.56200000000000006},"min":{"1min":0.32700000000000001,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":1.0109999999999999,"5min":1.0109999999999999,"15min":1.0109999999999999},"last":0.65100000000000002},{"interface":"front","average":{"1min":0.80000000000000004,"5min":0.80000000000000004,"15min":0.80000000000000004},"min":{"1min":0.33400000000000002,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":5.431,"5min":5.431,"15min":5.431},"last":0.59899999999999998}]},{"osd":1,"last update":"Fri Nov 24 07:57:17 2023","interfaces":[{"interface":"back","average":{"1min":0.55900000000000005,"5min":0.55900000000000005,"15min":0.55900000000000005},"min":{"1min":0.28899999999999998,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":1.016,"5min":1.016,"15min":1.016},"last":0.58399999999999996},{"interface":"front","average":{"1min":0.56999999999999995,"5min":0.56999999999999995,"15min":0.56999999999999995},"min":{"1min":0.311,"5min":0.311,"15min":0.311},"max":{"1min":0.96499999999999997,"5min":0.96499999999999997,"15min":0.96499999999999997},"last":0.60899999999999999}]},{"osd":2,"last update":"Fri Nov 24 07:57:17 2023","interfaces":[{"interface":"back","average":{"1min":0.55100000000000005,"5min":0.55100000000000005,"15min":0.55100000000000005},"min":{"1min":0.377,"5min":0.377,"15min":0.377},"max":{"1min":1.163,"5min":1.163,"15min":1.163},"last":0.628},{"interface":"front","average":{"1min":0.63,"5min":0.63,"15min":0.63},"min":{"1min":0.39200000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":1.119,"5min":1.119,"15min":1.119},"last":0.52600000000000002}]},{"osd":3,"last update":"Fri Nov 24 07:57:17 2023","interfaces":[{"interface":"back","average":{"1min":0.90500000000000003,"5min":0.90500000000000003,"15min":0.90500000000000003},"min":{"1min":0.36499999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":5.609,"5min":5.609,"15min":5.609},"last":0.56499999999999995},{"interface":"front","average":{"1min":0.67600000000000005,"5min":0.67600000000000005,"15min":0.67600000000000005},"min":{"1min":0.41899999999999998,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":1.288,"5min":1.288,"15min":1.288},"last":0.66800000000000004}]},{"osd":4,"last update":"Fri Nov 24 07:57:17 2023","interfaces":[{"interface":"back","average":{"1min":0.51000000000000001,"5min":0.51000000000000001,"15min":0.51000000000000001},"min":{"1min":0.36399999999999999,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.68799999999999994,"5min":0.68799999999999994,"15min":0.68799999999999994},"last":0.378},{"interface":"front","average":{"1min":0.57199999999999995,"5min":0.57199999999999995,"15min":0.57199999999999995},"min":{"1min":0.39800000000000002,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.78300000000000003,"5min":0.78300000000000003,"15min":0.78300000000000003},"last":0.45100000000000001}]},{"osd":6,"last update":"Fri Nov 24 07:57:17 2023","interfaces":[{"interface":"back","average":{"1min":0.57099999999999995,"5min":0.57099999999999995,"15min":0.57099999999999995},"min":{"1min":0.35599999999999998,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.89700000000000002,"5min":0.89700000000000002,"15min":0.89700000000000002},"last":0.39400000000000002},{"interface":"front","average":{"1min":0.625,"5min":0.625,"15min":0.625},"min":{"1min":0.42899999999999999,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":0.88200000000000001,"5min":0.88200000000000001,"15min":0.88200000000000001},"last":0.42299999999999999}]},{"osd":7,"last update":"Fri Nov 24 07:57:17 2023","interfaces":[{"interface":"back","average":{"1min":0.64600000000000002,"5min":0.64600000000000002,"15min":0.64600000000000002},"min":{"1min":0.45000000000000001,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":0.96599999999999997,"5min":0.96599999999999997,"15min":0.96599999999999997},"last":0.40799999999999997},{"interface":"front","average":{"1min":0.65300000000000002,"5min":0.65300000000000002,"15min":0.65300000000000002},"min":{"1min":0.46300000000000002,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":0.94199999999999995,"5min":0.94199999999999995,"15min":0.94199999999999995},"last":0.53900000000000003}]}]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":776,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-11-24T07:57:34.810 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2023-11-24T07:57:34.810 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2023-11-24T07:57:34.978 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:34.976+0000 7f5a0446a700 1 Processor -- start 2023-11-24T07:57:34.978 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:34.976+0000 7f5a0446a700 1 -- start start 2023-11-24T07:57:34.981 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:34.977+0000 7f5a0446a700 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f59fc072a00 0x7f59fc06db60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:57:34.981 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:34.977+0000 7f5a0446a700 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f59fc0d0b80 con 0x7f59fc06e0a0 2023-11-24T07:57:34.981 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:34.977+0000 7f5a0446a700 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f59fc15ed60 con 0x7f59fc0b4090 2023-11-24T07:57:34.981 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:34.977+0000 7f5a0446a700 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f59fc15f040 con 0x7f59fc072a00 2023-11-24T07:57:34.981 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:34.977+0000 7f5a02a07700 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7f59fc06e0a0 0x7f59fc0d1010 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:55320/0 (socket says 172.21.15.59:55320) 2023-11-24T07:57:34.981 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:34.977+0000 7f5a02a07700 1 -- 172.21.15.59:0/1376269257 learned_addr learned my addr 172.21.15.59:0/1376269257 (peer_addr_for_me v1:172.21.15.59:0/0) 2023-11-24T07:57:34.981 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:34.977+0000 7f5a01a05700 1 --2- 172.21.15.59:0/1376269257 >> v2:172.21.15.76:3300/0 conn(0x7f59fc072a00 0x7f59fc06db60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:57:34.981 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:34.977+0000 7f5a01204700 1 -- 172.21.15.59:0/1376269257 <== mon.0 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2280155506 0 0) 0x7f59fc0d0b80 con 0x7f59fc06e0a0 2023-11-24T07:57:34.982 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:34.977+0000 7f5a01204700 1 -- 172.21.15.59:0/1376269257 --> v1:172.21.15.59:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f59e80035b0 con 0x7f59fc06e0a0 2023-11-24T07:57:34.982 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:34.977+0000 7f5a01204700 1 -- 172.21.15.59:0/1376269257 <== mon.1 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3337128116 0 0) 0x7f59fc15ed60 con 0x7f59fc0b4090 2023-11-24T07:57:34.982 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:34.977+0000 7f5a01204700 1 -- 172.21.15.59:0/1376269257 --> v1:172.21.15.76:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f59fc0d0b80 con 0x7f59fc0b4090 2023-11-24T07:57:34.982 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:34.977+0000 7f5a01204700 1 -- 172.21.15.59:0/1376269257 <== mon.0 v1:172.21.15.59:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 836146184 0 0) 0x7f59e80035b0 con 0x7f59fc06e0a0 2023-11-24T07:57:34.982 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:34.977+0000 7f5a01a05700 1 -- 172.21.15.59:0/1376269257 >> v1:172.21.15.76:6789/0 conn(0x7f59fc0b4090 legacy=0x7f59fc15e510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:57:34.982 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:34.977+0000 7f5a01a05700 1 -- 172.21.15.59:0/1376269257 >> v1:172.21.15.59:6789/0 conn(0x7f59fc06e0a0 legacy=0x7f59fc0d1010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:57:34.982 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:34.977+0000 7f5a01a05700 1 -- 172.21.15.59:0/1376269257 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f59fc15f320 con 0x7f59fc072a00 2023-11-24T07:57:34.982 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:34.978+0000 7f5a01a05700 1 --2- 172.21.15.59:0/1376269257 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f59fc072a00 0x7f59fc06db60 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f59ec009840 tx=0x7f59ec009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2108b105ff19e2eb server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:57:34.982 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:34.978+0000 7f5a01204700 1 -- 172.21.15.59:0/1376269257 <== mon.1 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59f40019f0 con 0x7f59fc072a00 2023-11-24T07:57:34.982 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:34.978+0000 7f5a01204700 1 -- 172.21.15.59:0/1376269257 <== mon.1 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f59ec004030 con 0x7f59fc072a00 2023-11-24T07:57:34.982 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:34.978+0000 7f5a01204700 1 -- 172.21.15.59:0/1376269257 <== mon.1 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59ec0043c0 con 0x7f59fc072a00 2023-11-24T07:57:34.983 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:34.978+0000 7f5a0446a700 1 -- 172.21.15.59:0/1376269257 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f59fc072a00 msgr2=0x7f59fc06db60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:57:34.983 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:34.978+0000 7f5a0446a700 1 --2- 172.21.15.59:0/1376269257 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f59fc072a00 0x7f59fc06db60 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f59ec009840 tx=0x7f59ec009b50 comp rx=0 tx=0).stop 2023-11-24T07:57:34.983 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:34.978+0000 7f5a0446a700 1 -- 172.21.15.59:0/1376269257 shutdown_connections 2023-11-24T07:57:34.983 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:34.978+0000 7f5a0446a700 1 --2- 172.21.15.59:0/1376269257 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f59fc072a00 0x7f59fc06db60 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:34.983 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:34.978+0000 7f5a0446a700 1 -- 172.21.15.59:0/1376269257 >> 172.21.15.59:0/1376269257 conn(0x7f59fc0b2740 msgr2=0x7f59fc0b2b50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:57:34.983 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:34.978+0000 7f5a0446a700 1 -- 172.21.15.59:0/1376269257 shutdown_connections 2023-11-24T07:57:34.983 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:34.978+0000 7f5a0446a700 1 -- 172.21.15.59:0/1376269257 wait complete. 2023-11-24T07:57:34.986 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:34.980+0000 7f5a0446a700 1 Processor -- start 2023-11-24T07:57:34.986 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:34.980+0000 7f5a0446a700 1 -- start start 2023-11-24T07:57:34.986 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:34.980+0000 7f5a0446a700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f59fc06e0a0 0x7f59fc14e8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:57:34.986 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:34.980+0000 7f5a0446a700 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f59fc072a00 0x7f59fc159610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:57:34.986 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:34.980+0000 7f5a0446a700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f59fc0b4090 0x7f59fc159b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:57:34.987 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:34.980+0000 7f5a0446a700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59fc0aea20 con 0x7f59fc06e0a0 2023-11-24T07:57:34.987 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:34.980+0000 7f5a0446a700 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59fc1f7c30 con 0x7f59fc0b4090 2023-11-24T07:57:34.987 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:34.980+0000 7f5a0446a700 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f59fc0ae080 con 0x7f59fc072a00 2023-11-24T07:57:34.987 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:34.980+0000 7f5a02206700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f59fc06e0a0 0x7f59fc14e8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:57:34.987 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:34.981+0000 7f5a02206700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f59fc06e0a0 0x7f59fc14e8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:51848/0 (socket says 172.21.15.59:51848) 2023-11-24T07:57:34.987 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:34.981+0000 7f5a02206700 1 -- 172.21.15.59:0/1732054592 learned_addr learned my addr 172.21.15.59:0/1732054592 (peer_addr_for_me v2:172.21.15.59:0/0) 2023-11-24T07:57:34.987 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:34.981+0000 7f5a01a05700 1 --2- 172.21.15.59:0/1732054592 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f59fc072a00 0x7f59fc159610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:57:34.987 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:34.981+0000 7f5a02a07700 1 --2- 172.21.15.59:0/1732054592 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f59fc0b4090 0x7f59fc159b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:57:34.987 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:34.981+0000 7f5a02206700 1 -- 172.21.15.59:0/1732054592 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f59fc072a00 msgr2=0x7f59fc159610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:57:34.987 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:34.981+0000 7f5a02206700 1 --2- 172.21.15.59:0/1732054592 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f59fc072a00 0x7f59fc159610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:34.987 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:34.981+0000 7f5a02206700 1 -- 172.21.15.59:0/1732054592 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f59fc0b4090 msgr2=0x7f59fc159b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:57:34.988 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:34.981+0000 7f5a02206700 1 --2- 172.21.15.59:0/1732054592 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f59fc0b4090 0x7f59fc159b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:34.988 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:34.981+0000 7f5a02206700 1 -- 172.21.15.59:0/1732054592 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f59ec0094a0 con 0x7f59fc06e0a0 2023-11-24T07:57:34.988 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:34.981+0000 7f5a02a07700 1 --2- 172.21.15.59:0/1732054592 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f59fc0b4090 0x7f59fc159b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-24T07:57:34.988 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:34.981+0000 7f5a01a05700 1 --2- 172.21.15.59:0/1732054592 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f59fc072a00 0x7f59fc159610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-24T07:57:34.988 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:34.981+0000 7f5a02206700 1 --2- 172.21.15.59:0/1732054592 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f59fc06e0a0 0x7f59fc14e8d0 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f59f800c500 tx=0x7f59f800c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:57:34.988 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:34.981+0000 7f59f2ffd700 1 -- 172.21.15.59:0/1732054592 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59f80052d0 con 0x7f59fc06e0a0 2023-11-24T07:57:34.988 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:34.981+0000 7f59f2ffd700 1 -- 172.21.15.59:0/1732054592 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f59f8005450 con 0x7f59fc06e0a0 2023-11-24T07:57:34.988 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:34.981+0000 7f5a0446a700 1 -- 172.21.15.59:0/1732054592 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f59fc0ae360 con 0x7f59fc06e0a0 2023-11-24T07:57:34.988 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:34.981+0000 7f59f2ffd700 1 -- 172.21.15.59:0/1732054592 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59f80038c0 con 0x7f59fc06e0a0 2023-11-24T07:57:34.988 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:34.981+0000 7f5a0446a700 1 -- 172.21.15.59:0/1732054592 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f59fc15a1d0 con 0x7f59fc06e0a0 2023-11-24T07:57:34.989 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:34.982+0000 7f5a0446a700 1 -- 172.21.15.59:0/1732054592 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f59fc075480 con 0x7f59fc06e0a0 2023-11-24T07:57:34.989 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:34.983+0000 7f59f2ffd700 1 -- 172.21.15.59:0/1732054592 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f59f8003a20 con 0x7f59fc06e0a0 2023-11-24T07:57:34.989 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:34.983+0000 7f59f2ffd700 1 --2- 172.21.15.59:0/1732054592 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f59e006aed0 0x7f59e006d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:57:34.989 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:34.983+0000 7f59f2ffd700 1 -- 172.21.15.59:0/1732054592 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f59f8088a10 con 0x7f59fc06e0a0 2023-11-24T07:57:34.989 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:34.983+0000 7f5a01a05700 1 --2- 172.21.15.59:0/1732054592 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f59e006aed0 0x7f59e006d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:57:34.989 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:34.984+0000 7f5a01a05700 1 --2- 172.21.15.59:0/1732054592 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f59e006aed0 0x7f59e006d390 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f59ec0090b0 tx=0x7f59ec008030 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:57:34.989 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:34.989+0000 7f59f2ffd700 1 -- 172.21.15.59:0/1732054592 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f59f8053b50 con 0x7f59fc06e0a0 2023-11-24T07:57:35.277 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:35.276+0000 7f5a0446a700 1 -- 172.21.15.59:0/1732054592 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7f59fc0aeb60 con 0x7f59fc06e0a0 2023-11-24T07:57:35.279 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:35.278+0000 7f59f2ffd700 1 -- 172.21.15.59:0/1732054592 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v13) v1 ==== 190+0+147 (secure 0 0 0) 0x7f59fc0aeb60 con 0x7f59fc06e0a0 2023-11-24T07:57:35.280 INFO:teuthology.orchestra.run.smithi059.stdout:[client.0] 2023-11-24T07:57:35.280 INFO:teuthology.orchestra.run.smithi059.stdout: key = AQD1VmBlxPkpBhAAR841hRsF6QtmzFgJIrVZ9w== 2023-11-24T07:57:35.280 INFO:teuthology.orchestra.run.smithi059.stdout: caps mds = "allow" 2023-11-24T07:57:35.280 INFO:teuthology.orchestra.run.smithi059.stdout: caps mon = "allow r" 2023-11-24T07:57:35.280 INFO:teuthology.orchestra.run.smithi059.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2023-11-24T07:57:35.282 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:35.280+0000 7f5a0446a700 1 -- 172.21.15.59:0/1732054592 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f59e006aed0 msgr2=0x7f59e006d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:57:35.282 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:35.280+0000 7f5a0446a700 1 --2- 172.21.15.59:0/1732054592 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f59e006aed0 0x7f59e006d390 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f59ec0090b0 tx=0x7f59ec008030 comp rx=0 tx=0).stop 2023-11-24T07:57:35.282 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:35.280+0000 7f5a0446a700 1 -- 172.21.15.59:0/1732054592 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f59fc06e0a0 msgr2=0x7f59fc14e8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:57:35.283 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:35.280+0000 7f5a0446a700 1 --2- 172.21.15.59:0/1732054592 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f59fc06e0a0 0x7f59fc14e8d0 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f59f800c500 tx=0x7f59f800c8c0 comp rx=0 tx=0).stop 2023-11-24T07:57:35.283 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:35.280+0000 7f5a0446a700 1 -- 172.21.15.59:0/1732054592 shutdown_connections 2023-11-24T07:57:35.283 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:35.280+0000 7f5a0446a700 1 --2- 172.21.15.59:0/1732054592 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f59e006aed0 0x7f59e006d390 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:35.283 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:35.280+0000 7f5a0446a700 1 --2- 172.21.15.59:0/1732054592 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f59fc0b4090 0x7f59fc159b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:35.283 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:35.280+0000 7f5a0446a700 1 --2- 172.21.15.59:0/1732054592 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f59fc06e0a0 0x7f59fc14e8d0 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:35.283 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:35.280+0000 7f5a0446a700 1 --2- 172.21.15.59:0/1732054592 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f59fc072a00 0x7f59fc159610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:35.283 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:35.280+0000 7f5a0446a700 1 -- 172.21.15.59:0/1732054592 >> 172.21.15.59:0/1732054592 conn(0x7f59fc0b2740 msgr2=0x7f59fc0b2b20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:57:35.283 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:35.280+0000 7f5a0446a700 1 -- 172.21.15.59:0/1732054592 shutdown_connections 2023-11-24T07:57:35.283 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:35.280+0000 7f5a0446a700 1 -- 172.21.15.59:0/1732054592 wait complete. 2023-11-24T07:57:35.284 INFO:teuthology.orchestra.run.smithi059.stderr:updated caps for client.0 2023-11-24T07:57:35.294 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-24T07:57:35.419 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:35.419+0000 7f4d02462700 1 Processor -- start 2023-11-24T07:57:35.420 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:35.419+0000 7f4d02462700 1 -- start start 2023-11-24T07:57:35.422 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:35.419+0000 7f4d02462700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f4cfc0d2b50 0x7f4cfc0cfa00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:57:35.422 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:35.419+0000 7f4d02462700 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4cfc0ce9a0 con 0x7f4cfc15e080 2023-11-24T07:57:35.422 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:35.419+0000 7f4d02462700 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4cfc0cec40 con 0x7f4cfc0d0740 2023-11-24T07:57:35.422 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:35.419+0000 7f4d02462700 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4cfc0d14b0 con 0x7f4cfc0d2b50 2023-11-24T07:57:35.422 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:35.419+0000 7f4cfb7fe700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f4cfc0d2b50 0x7f4cfc0cfa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:57:35.422 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:35.419+0000 7f4cfbfff700 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7f4cfc15e080 0x7f4cfc0d2440 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:55330/0 (socket says 172.21.15.59:55330) 2023-11-24T07:57:35.422 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:35.419+0000 7f4cfbfff700 1 -- 172.21.15.59:0/4170603279 learned_addr learned my addr 172.21.15.59:0/4170603279 (peer_addr_for_me v1:172.21.15.59:0/0) 2023-11-24T07:57:35.422 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:35.419+0000 7f4cfb7fe700 1 -- 172.21.15.59:0/4170603279 >> v1:172.21.15.76:6789/0 conn(0x7f4cfc0d0740 legacy=0x7f4cfc0d0b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T07:57:35.422 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:35.419+0000 7f4cfb7fe700 1 -- 172.21.15.59:0/4170603279 >> v1:172.21.15.59:6789/0 conn(0x7f4cfc15e080 legacy=0x7f4cfc0d2440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:57:35.423 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:35.419+0000 7f4cfb7fe700 1 -- 172.21.15.59:0/4170603279 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4cfc0cef20 con 0x7f4cfc0d2b50 2023-11-24T07:57:35.423 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:35.419+0000 7f4cfaffd700 1 -- 172.21.15.59:0/4170603279 <== mon.0 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 417093502 0 0) 0x7f4cfc0ce9a0 con 0x7f4cfc15e080 2023-11-24T07:57:35.423 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:35.419+0000 7f4cfb7fe700 1 --2- 172.21.15.59:0/4170603279 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4cfc0d2b50 0x7f4cfc0cfa00 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f4cf0009840 tx=0x7f4cf0009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1756afe89b132c4c server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:57:35.423 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:35.419+0000 7f4cfaffd700 1 -- 172.21.15.59:0/4170603279 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4cf0004230 con 0x7f4cfc0d2b50 2023-11-24T07:57:35.423 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:35.419+0000 7f4cfaffd700 1 -- 172.21.15.59:0/4170603279 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4cf0004390 con 0x7f4cfc0d2b50 2023-11-24T07:57:35.423 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:35.420+0000 7f4cfaffd700 1 -- 172.21.15.59:0/4170603279 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4cf00036a0 con 0x7f4cfc0d2b50 2023-11-24T07:57:35.423 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:35.420+0000 7f4d02462700 1 -- 172.21.15.59:0/4170603279 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4cfc0d2b50 msgr2=0x7f4cfc0cfa00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:57:35.423 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:35.420+0000 7f4d02462700 1 --2- 172.21.15.59:0/4170603279 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4cfc0d2b50 0x7f4cfc0cfa00 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f4cf0009840 tx=0x7f4cf0009b50 comp rx=0 tx=0).stop 2023-11-24T07:57:35.423 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:35.420+0000 7f4d02462700 1 -- 172.21.15.59:0/4170603279 shutdown_connections 2023-11-24T07:57:35.423 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:35.420+0000 7f4d02462700 1 --2- 172.21.15.59:0/4170603279 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4cfc0d2b50 0x7f4cfc0cfa00 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:35.423 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:35.420+0000 7f4d02462700 1 -- 172.21.15.59:0/4170603279 >> 172.21.15.59:0/4170603279 conn(0x7f4cfc09d180 msgr2=0x7f4cfc0d3e90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:57:35.423 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:35.420+0000 7f4d02462700 1 -- 172.21.15.59:0/4170603279 shutdown_connections 2023-11-24T07:57:35.423 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:35.420+0000 7f4d02462700 1 -- 172.21.15.59:0/4170603279 wait complete. 2023-11-24T07:57:35.426 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:35.422+0000 7f4d02462700 1 Processor -- start 2023-11-24T07:57:35.426 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:35.422+0000 7f4d02462700 1 -- start start 2023-11-24T07:57:35.426 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:35.423+0000 7f4d02462700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4cfc0d0740 0x7f4cfc1f4360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:57:35.426 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:35.423+0000 7f4d02462700 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f4cfc0d2b50 0x7f4cfc1574f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:57:35.426 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:35.423+0000 7f4d02462700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4cfc15e080 0x7f4cfc1f7580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:57:35.426 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:35.423+0000 7f4d02462700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4cfc0c8f90 con 0x7f4cfc0d0740 2023-11-24T07:57:35.426 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:35.423+0000 7f4d02462700 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4cfc1f48a0 con 0x7f4cfc15e080 2023-11-24T07:57:35.426 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:35.423+0000 7f4d02462700 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4cfc0c86f0 con 0x7f4cfc0d2b50 2023-11-24T07:57:35.426 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:35.423+0000 7f4cfbfff700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4cfc0d0740 0x7f4cfc1f4360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:57:35.426 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:35.423+0000 7f4cfbfff700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4cfc0d0740 0x7f4cfc1f4360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:51862/0 (socket says 172.21.15.59:51862) 2023-11-24T07:57:35.427 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:35.423+0000 7f4cfbfff700 1 -- 172.21.15.59:0/235452446 learned_addr learned my addr 172.21.15.59:0/235452446 (peer_addr_for_me v2:172.21.15.59:0/0) 2023-11-24T07:57:35.427 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:35.423+0000 7f4d009ff700 1 --2- 172.21.15.59:0/235452446 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4cfc15e080 0x7f4cfc1f7580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:57:35.427 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:35.423+0000 7f4cfb7fe700 1 --2- 172.21.15.59:0/235452446 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f4cfc0d2b50 0x7f4cfc1574f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:57:35.427 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:35.423+0000 7f4cfbfff700 1 -- 172.21.15.59:0/235452446 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f4cfc0d2b50 msgr2=0x7f4cfc1574f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:57:35.427 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:35.423+0000 7f4cfbfff700 1 --2- 172.21.15.59:0/235452446 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f4cfc0d2b50 0x7f4cfc1574f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:35.427 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:35.423+0000 7f4cfbfff700 1 -- 172.21.15.59:0/235452446 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4cfc15e080 msgr2=0x7f4cfc1f7580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:57:35.427 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:35.423+0000 7f4cfbfff700 1 --2- 172.21.15.59:0/235452446 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4cfc15e080 0x7f4cfc1f7580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:35.427 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:35.423+0000 7f4cfbfff700 1 -- 172.21.15.59:0/235452446 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4cf00094a0 con 0x7f4cfc0d0740 2023-11-24T07:57:35.427 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:35.423+0000 7f4cfbfff700 1 --2- 172.21.15.59:0/235452446 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4cfc0d0740 0x7f4cfc1f4360 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f4cec00c550 tx=0x7f4cec00c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:57:35.427 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:35.424+0000 7f4cf8ff9700 1 -- 172.21.15.59:0/235452446 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4cec004c80 con 0x7f4cfc0d0740 2023-11-24T07:57:35.427 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:35.424+0000 7f4cf8ff9700 1 -- 172.21.15.59:0/235452446 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4cec004e00 con 0x7f4cfc0d0740 2023-11-24T07:57:35.427 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:35.424+0000 7f4d02462700 1 -- 172.21.15.59:0/235452446 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4cfc0c89d0 con 0x7f4cfc0d0740 2023-11-24T07:57:35.428 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:35.424+0000 7f4cf8ff9700 1 -- 172.21.15.59:0/235452446 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4cec007310 con 0x7f4cfc0d0740 2023-11-24T07:57:35.428 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:35.424+0000 7f4d02462700 1 -- 172.21.15.59:0/235452446 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4cfc1f7060 con 0x7f4cfc0d0740 2023-11-24T07:57:35.428 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:35.425+0000 7f4d02462700 1 -- 172.21.15.59:0/235452446 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4cfc1f73d0 con 0x7f4cfc0d0740 2023-11-24T07:57:35.429 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:35.425+0000 7f4cf8ff9700 1 -- 172.21.15.59:0/235452446 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f4cec007470 con 0x7f4cfc0d0740 2023-11-24T07:57:35.429 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:35.426+0000 7f4cf8ff9700 1 --2- 172.21.15.59:0/235452446 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f4ce806aed0 0x7f4ce806d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:57:35.429 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:35.426+0000 7f4cf8ff9700 1 -- 172.21.15.59:0/235452446 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f4cec0892c0 con 0x7f4cfc0d0740 2023-11-24T07:57:35.429 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:35.426+0000 7f4cfb7fe700 1 --2- 172.21.15.59:0/235452446 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f4ce806aed0 0x7f4ce806d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:57:35.429 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:35.426+0000 7f4cfb7fe700 1 --2- 172.21.15.59:0/235452446 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f4ce806aed0 0x7f4ce806d390 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f4cfc0d2f70 tx=0x7f4cf001c040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:57:35.432 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:35.431+0000 7f4cf8ff9700 1 -- 172.21.15.59:0/235452446 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f4cec054360 con 0x7f4cfc0d0740 2023-11-24T07:57:35.678 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:35.676+0000 7f4d02462700 1 -- 172.21.15.59:0/235452446 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f4cfc067f60 con 0x7f4cfc0d0740 2023-11-24T07:57:35.678 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:35.677+0000 7f4cf8ff9700 1 -- 172.21.15.59:0/235452446 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 16 v16) v1 ==== 94+0+4913 (secure 0 0 0) 0x7f4cec057980 con 0x7f4cfc0d0740 2023-11-24T07:57:35.679 INFO:teuthology.orchestra.run.smithi059.stdout: 2023-11-24T07:57:35.679 INFO:teuthology.orchestra.run.smithi059.stdout:{"epoch":16,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4763,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.76:6836/1669441817","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6834","nonce":1669441817},{"type":"v1","addr":"172.21.15.76:6836","nonce":1669441817}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":4769,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.76:6837/3709928230","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6835","nonce":3709928230},{"type":"v1","addr":"172.21.15.76:6837","nonce":3709928230}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":5106,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6834/2019319496","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6832","nonce":2019319496},{"type":"v1","addr":"172.21.15.59:6834","nonce":2019319496}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":16,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-24T07:57:25.003636+0000","modified":"2023-11-24T07:57:28.021959+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5109},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5109":{"gid":5109,"name":"a","rank":0,"incarnation":13,"state":"up:active","state_seq":3,"addr":"172.21.15.59:6835/2290655475","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6833","nonce":2290655475},{"type":"v1","addr":"172.21.15.59:6835","nonce":2290655475}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2023-11-24T07:57:35.681 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:35.679+0000 7f4d02462700 1 -- 172.21.15.59:0/235452446 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f4ce806aed0 msgr2=0x7f4ce806d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:57:35.681 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:35.679+0000 7f4d02462700 1 --2- 172.21.15.59:0/235452446 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f4ce806aed0 0x7f4ce806d390 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f4cfc0d2f70 tx=0x7f4cf001c040 comp rx=0 tx=0).stop 2023-11-24T07:57:35.681 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:35.679+0000 7f4d02462700 1 -- 172.21.15.59:0/235452446 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4cfc0d0740 msgr2=0x7f4cfc1f4360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:57:35.681 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:35.679+0000 7f4d02462700 1 --2- 172.21.15.59:0/235452446 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4cfc0d0740 0x7f4cfc1f4360 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f4cec00c550 tx=0x7f4cec00c910 comp rx=0 tx=0).stop 2023-11-24T07:57:35.681 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:35.679+0000 7f4d02462700 1 -- 172.21.15.59:0/235452446 shutdown_connections 2023-11-24T07:57:35.681 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:35.679+0000 7f4d02462700 1 --2- 172.21.15.59:0/235452446 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f4ce806aed0 0x7f4ce806d390 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:35.681 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:35.679+0000 7f4d02462700 1 --2- 172.21.15.59:0/235452446 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4cfc15e080 0x7f4cfc1f7580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:35.682 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:35.679+0000 7f4d02462700 1 --2- 172.21.15.59:0/235452446 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4cfc0d0740 0x7f4cfc1f4360 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:35.682 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:35.679+0000 7f4d02462700 1 --2- 172.21.15.59:0/235452446 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f4cfc0d2b50 0x7f4cfc1574f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:35.682 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:35.679+0000 7f4d02462700 1 -- 172.21.15.59:0/235452446 >> 172.21.15.59:0/235452446 conn(0x7f4cfc09d180 msgr2=0x7f4cfc06b9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:57:35.682 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:35.679+0000 7f4d02462700 1 -- 172.21.15.59:0/235452446 shutdown_connections 2023-11-24T07:57:35.682 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:35.679+0000 7f4d02462700 1 -- 172.21.15.59:0/235452446 wait complete. 2023-11-24T07:57:35.682 INFO:teuthology.orchestra.run.smithi059.stderr:dumped fsmap epoch 16 2023-11-24T07:57:36.097 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 16, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-11-24T07:57:25.003636+0000', 'modified': '2023-11-24T07:57:28.021959+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 5109}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_5109': {'gid': 5109, 'name': 'a', 'rank': 0, 'incarnation': 13, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.59:6835/2290655475', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.59:6833', 'nonce': 2290655475}, {'type': 'v1', 'addr': '172.21.15.59:6835', 'nonce': 2290655475}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [5], 'metadata_pool': 4, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2023-11-24T07:57:36.097 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2023-11-24T07:57:36.098 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2023-11-24T07:57:36.225 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:36.225+0000 7fe9d4f9b700 1 Processor -- start 2023-11-24T07:57:36.226 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:36.225+0000 7fe9d4f9b700 1 -- start start 2023-11-24T07:57:36.228 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:36.226+0000 7fe9d4f9b700 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fe9d006b620 0x7fe9d00b3a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:57:36.228 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:36.226+0000 7fe9d4f9b700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fe9d006ee60 0x7fe9d00b3f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:57:36.228 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:36.226+0000 7fe9d4f9b700 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe9d0071f50 con 0x7fe9d00b5ab0 2023-11-24T07:57:36.228 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:36.226+0000 7fe9d4f9b700 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe9d0072230 con 0x7fe9d006ee60 2023-11-24T07:57:36.228 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:36.226+0000 7fe9d4f9b700 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe9d0072510 con 0x7fe9d006b620 2023-11-24T07:57:36.228 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:36.226+0000 7fe9ced9d700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fe9d006ee60 0x7fe9d00b3f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:57:36.229 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:36.226+0000 7fe9ced9d700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fe9d006ee60 0x7fe9d00b3f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:51874/0 (socket says 172.21.15.59:51874) 2023-11-24T07:57:36.229 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:36.226+0000 7fe9ced9d700 1 -- 172.21.15.59:0/2658944822 learned_addr learned my addr 172.21.15.59:0/2658944822 (peer_addr_for_me v2:172.21.15.59:0/0) 2023-11-24T07:57:36.229 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:36.226+0000 7fe9ce59c700 1 --2- 172.21.15.59:0/2658944822 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fe9d006b620 0x7fe9d00b3a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:57:36.229 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:36.226+0000 7fe9ced9d700 1 -- 172.21.15.59:0/2658944822 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fe9d006b620 msgr2=0x7fe9d00b3a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T07:57:36.229 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:36.226+0000 7fe9ced9d700 1 --2- 172.21.15.59:0/2658944822 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fe9d006b620 0x7fe9d00b3a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:36.229 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:36.226+0000 7fe9ced9d700 1 -- 172.21.15.59:0/2658944822 >> v1:172.21.15.76:6789/0 conn(0x7fe9d00b5ab0 legacy=0x7fe9d0161ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T07:57:36.229 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:36.226+0000 7fe9ced9d700 1 -- 172.21.15.59:0/2658944822 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe9d00727f0 con 0x7fe9d006ee60 2023-11-24T07:57:36.229 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:36.226+0000 7fe9ce59c700 1 --2- 172.21.15.59:0/2658944822 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fe9d006b620 0x7fe9d00b3a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T07:57:36.229 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:36.226+0000 7fe9ced9d700 1 --2- 172.21.15.59:0/2658944822 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe9d006ee60 0x7fe9d00b3f80 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7fe9c8009e50 tx=0x7fe9c800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=93dfe9e4d36c2e6c server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:57:36.229 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:36.227+0000 7fe9cd59a700 1 -- 172.21.15.59:0/2658944822 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9c800e070 con 0x7fe9d006ee60 2023-11-24T07:57:36.229 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:36.227+0000 7fe9cd59a700 1 -- 172.21.15.59:0/2658944822 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe9c800b9c0 con 0x7fe9d006ee60 2023-11-24T07:57:36.229 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:36.227+0000 7fe9cd59a700 1 -- 172.21.15.59:0/2658944822 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9c800bd70 con 0x7fe9d006ee60 2023-11-24T07:57:36.230 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:36.227+0000 7fe9d4f9b700 1 -- 172.21.15.59:0/2658944822 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe9d006ee60 msgr2=0x7fe9d00b3f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:57:36.230 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:36.227+0000 7fe9d4f9b700 1 --2- 172.21.15.59:0/2658944822 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe9d006ee60 0x7fe9d00b3f80 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7fe9c8009e50 tx=0x7fe9c800b330 comp rx=0 tx=0).stop 2023-11-24T07:57:36.230 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:36.227+0000 7fe9d4f9b700 1 -- 172.21.15.59:0/2658944822 shutdown_connections 2023-11-24T07:57:36.230 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:36.227+0000 7fe9d4f9b700 1 --2- 172.21.15.59:0/2658944822 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fe9d006b620 0x7fe9d00b3a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:36.230 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:36.227+0000 7fe9d4f9b700 1 --2- 172.21.15.59:0/2658944822 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe9d006ee60 0x7fe9d00b3f80 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:36.230 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:36.227+0000 7fe9d4f9b700 1 -- 172.21.15.59:0/2658944822 >> 172.21.15.59:0/2658944822 conn(0x7fe9d00b2740 msgr2=0x7fe9d00b2b20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:57:36.230 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:36.227+0000 7fe9d4f9b700 1 -- 172.21.15.59:0/2658944822 shutdown_connections 2023-11-24T07:57:36.230 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:36.227+0000 7fe9d4f9b700 1 -- 172.21.15.59:0/2658944822 wait complete. 2023-11-24T07:57:36.233 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:36.229+0000 7fe9d4f9b700 1 Processor -- start 2023-11-24T07:57:36.233 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:36.229+0000 7fe9d4f9b700 1 -- start start 2023-11-24T07:57:36.233 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:36.229+0000 7fe9d4f9b700 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fe9d006b620 0x7fe9d01fb700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:57:36.233 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:36.229+0000 7fe9d4f9b700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe9d006ee60 0x7fe9d00cebd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:57:36.233 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:36.229+0000 7fe9d4f9b700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe9d00b5ab0 0x7fe9d01f72f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:57:36.233 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:36.229+0000 7fe9d4f9b700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9d0136f30 con 0x7fe9d006ee60 2023-11-24T07:57:36.234 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:36.229+0000 7fe9d4f9b700 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9d00cf620 con 0x7fe9d00b5ab0 2023-11-24T07:57:36.234 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:36.229+0000 7fe9d4f9b700 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe9d00cf900 con 0x7fe9d006b620 2023-11-24T07:57:36.234 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:36.230+0000 7fe9cdd9b700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe9d006ee60 0x7fe9d00cebd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:57:36.234 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:36.230+0000 7fe9cdd9b700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe9d006ee60 0x7fe9d00cebd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:51878/0 (socket says 172.21.15.59:51878) 2023-11-24T07:57:36.234 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:36.230+0000 7fe9cdd9b700 1 -- 172.21.15.59:0/3418891561 learned_addr learned my addr 172.21.15.59:0/3418891561 (peer_addr_for_me v2:172.21.15.59:0/0) 2023-11-24T07:57:36.234 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:36.230+0000 7fe9ce59c700 1 --2- 172.21.15.59:0/3418891561 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fe9d006b620 0x7fe9d01fb700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:57:36.234 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:36.230+0000 7fe9ced9d700 1 --2- 172.21.15.59:0/3418891561 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe9d00b5ab0 0x7fe9d01f72f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:57:36.234 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:36.230+0000 7fe9cdd9b700 1 -- 172.21.15.59:0/3418891561 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fe9d006b620 msgr2=0x7fe9d01fb700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:57:36.234 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:36.230+0000 7fe9cdd9b700 1 --2- 172.21.15.59:0/3418891561 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fe9d006b620 0x7fe9d01fb700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:36.234 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:36.230+0000 7fe9cdd9b700 1 -- 172.21.15.59:0/3418891561 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe9d00b5ab0 msgr2=0x7fe9d01f72f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:57:36.234 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:36.230+0000 7fe9cdd9b700 1 --2- 172.21.15.59:0/3418891561 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe9d00b5ab0 0x7fe9d01f72f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:36.235 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:36.230+0000 7fe9cdd9b700 1 -- 172.21.15.59:0/3418891561 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe9c8009b30 con 0x7fe9d006ee60 2023-11-24T07:57:36.235 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:36.230+0000 7fe9cdd9b700 1 --2- 172.21.15.59:0/3418891561 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe9d006ee60 0x7fe9d00cebd0 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7fe9c400c420 tx=0x7fe9c400c730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:57:36.235 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:36.230+0000 7fe9beffd700 1 -- 172.21.15.59:0/3418891561 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9c40052d0 con 0x7fe9d006ee60 2023-11-24T07:57:36.235 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:36.230+0000 7fe9beffd700 1 -- 172.21.15.59:0/3418891561 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe9c4005430 con 0x7fe9d006ee60 2023-11-24T07:57:36.235 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:36.230+0000 7fe9d4f9b700 1 -- 172.21.15.59:0/3418891561 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe9d00cfbe0 con 0x7fe9d006ee60 2023-11-24T07:57:36.235 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:36.230+0000 7fe9beffd700 1 -- 172.21.15.59:0/3418891561 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9c40038c0 con 0x7fe9d006ee60 2023-11-24T07:57:36.235 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:36.230+0000 7fe9d4f9b700 1 -- 172.21.15.59:0/3418891561 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe9d013d190 con 0x7fe9d006ee60 2023-11-24T07:57:36.235 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:36.232+0000 7fe9beffd700 1 -- 172.21.15.59:0/3418891561 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fe9c4003a20 con 0x7fe9d006ee60 2023-11-24T07:57:36.235 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:36.232+0000 7fe9beffd700 1 --2- 172.21.15.59:0/3418891561 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7fe9b006aed0 0x7fe9b006d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:57:36.235 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:36.232+0000 7fe9beffd700 1 -- 172.21.15.59:0/3418891561 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fe9c40887b0 con 0x7fe9d006ee60 2023-11-24T07:57:36.235 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:36.233+0000 7fe9ce59c700 1 --2- 172.21.15.59:0/3418891561 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7fe9b006aed0 0x7fe9b006d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:57:36.238 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:36.233+0000 7fe9ce59c700 1 --2- 172.21.15.59:0/3418891561 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7fe9b006aed0 0x7fe9b006d390 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fe9c0006fd0 tx=0x7fe9c0008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:57:36.238 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:36.234+0000 7fe9e1194b80 1 Processor -- start 2023-11-24T07:57:36.238 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:36.234+0000 7fe9e1194b80 1 -- start start 2023-11-24T07:57:36.238 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:36.235+0000 7fe9e1194b80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55fefc777080 0x55fefc7bd840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:57:36.239 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:36.235+0000 7fe9e1194b80 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55fefc78ced0 0x55fefc7bdd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:57:36.239 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:36.235+0000 7fe9e1194b80 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x55fefc774490 0x55fefc7a0e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:57:36.239 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:36.235+0000 7fe9e1194b80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fefc6ea770 con 0x55fefc777080 2023-11-24T07:57:36.239 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:36.235+0000 7fe9e1194b80 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fefc6d1750 con 0x55fefc78ced0 2023-11-24T07:57:36.239 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:36.235+0000 7fe9e1194b80 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 v1 -- 0x55fefc6cf070 con 0x55fefc774490 2023-11-24T07:57:36.239 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:36.235+0000 7fe9ce59c700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55fefc777080 0x55fefc7bd840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:57:36.239 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:36.235+0000 7fe9ce59c700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55fefc777080 0x55fefc7bd840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:51884/0 (socket says 172.21.15.59:51884) 2023-11-24T07:57:36.239 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:36.235+0000 7fe9ce59c700 1 -- 172.21.15.59:0/540514804 learned_addr learned my addr 172.21.15.59:0/540514804 (peer_addr_for_me v2:172.21.15.59:0/0) 2023-11-24T07:57:36.239 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:36.235+0000 7fe9ced9d700 1 --2- 172.21.15.59:0/540514804 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x55fefc774490 0x55fefc7a0e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:57:36.239 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:36.235+0000 7fe9cdd9b700 1 --2- 172.21.15.59:0/540514804 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55fefc78ced0 0x55fefc7bdd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:57:36.239 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:36.235+0000 7fe9ce59c700 1 -- 172.21.15.59:0/540514804 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x55fefc774490 msgr2=0x55fefc7a0e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T07:57:36.240 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:36.235+0000 7fe9ce59c700 1 --2- 172.21.15.59:0/540514804 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x55fefc774490 0x55fefc7a0e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:36.240 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:36.235+0000 7fe9ce59c700 1 -- 172.21.15.59:0/540514804 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55fefc78ced0 msgr2=0x55fefc7bdd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T07:57:36.240 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:36.235+0000 7fe9ce59c700 1 --2- 172.21.15.59:0/540514804 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55fefc78ced0 0x55fefc7bdd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:36.240 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:36.235+0000 7fe9ce59c700 1 -- 172.21.15.59:0/540514804 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55fefc5475e0 con 0x55fefc777080 2023-11-24T07:57:36.240 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:36.236+0000 7fe9ce59c700 1 --2- 172.21.15.59:0/540514804 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55fefc777080 0x55fefc7bd840 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7fe9c000a5e0 tx=0x7fe9c0012000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4e80660d6557bf86 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:57:36.240 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:36.236+0000 7fe9affff700 1 -- 172.21.15.59:0/540514804 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9c0015960 con 0x55fefc777080 2023-11-24T07:57:36.240 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:36.236+0000 7fe9affff700 1 -- 172.21.15.59:0/540514804 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe9c000c040 con 0x55fefc777080 2023-11-24T07:57:36.240 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:36.236+0000 7fe9affff700 1 -- 172.21.15.59:0/540514804 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9c0015cc0 con 0x55fefc777080 2023-11-24T07:57:36.240 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:36.236+0000 7fe9e1194b80 1 -- 172.21.15.59:0/540514804 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55fefc777080 msgr2=0x55fefc7bd840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:57:36.240 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:36.236+0000 7fe9e1194b80 1 --2- 172.21.15.59:0/540514804 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55fefc777080 0x55fefc7bd840 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7fe9c000a5e0 tx=0x7fe9c0012000 comp rx=0 tx=0).stop 2023-11-24T07:57:36.240 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:36.237+0000 7fe9e1194b80 1 -- 172.21.15.59:0/540514804 shutdown_connections 2023-11-24T07:57:36.240 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:36.237+0000 7fe9e1194b80 1 --2- 172.21.15.59:0/540514804 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x55fefc774490 0x55fefc7a0e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:36.241 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:36.237+0000 7fe9e1194b80 1 --2- 172.21.15.59:0/540514804 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55fefc777080 0x55fefc7bd840 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:36.241 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:36.237+0000 7fe9e1194b80 1 --2- 172.21.15.59:0/540514804 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55fefc78ced0 0x55fefc7bdd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:36.241 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:36.237+0000 7fe9e1194b80 1 -- 172.21.15.59:0/540514804 >> 172.21.15.59:0/540514804 conn(0x55fefc6a5310 msgr2=0x55fefc7bbe70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:57:36.241 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:36.237+0000 7fe9e1194b80 1 -- 172.21.15.59:0/540514804 shutdown_connections 2023-11-24T07:57:36.241 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:36.237+0000 7fe9e1194b80 1 -- 172.21.15.59:0/540514804 wait complete. 2023-11-24T07:57:36.241 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:36.237+0000 7fe9e1194b80 1 -- start start 2023-11-24T07:57:36.241 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:36.237+0000 7fe9e1194b80 1 Processor -- start 2023-11-24T07:57:36.245 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:36.238+0000 7fe9e1194b80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55fefc774490 0x55fefc7ce6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:57:36.245 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:36.238+0000 7fe9e1194b80 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x55fefc777080 0x55fefc7cec00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:57:36.245 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:36.238+0000 7fe9e1194b80 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55fefc78ced0 0x55fefc7a3c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:57:36.245 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:36.238+0000 7fe9e1194b80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fefc6ca960 con 0x55fefc774490 2023-11-24T07:57:36.245 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:36.238+0000 7fe9e1194b80 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fefc684510 con 0x55fefc78ced0 2023-11-24T07:57:36.245 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:36.238+0000 7fe9e1194b80 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 v1 -- 0x55fefc6841b0 con 0x55fefc777080 2023-11-24T07:57:36.246 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:36.238+0000 7fe9e1194b80 10 client.0 _ll_register_callbacks cb 0x55fefc5afa30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-24T07:57:36.246 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:36.238+0000 7fe9ce59c700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55fefc774490 0x55fefc7ce6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:57:36.246 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:36.238+0000 7fe9ce59c700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55fefc774490 0x55fefc7ce6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:51890/0 (socket says 172.21.15.59:51890) 2023-11-24T07:57:36.246 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:36.238+0000 7fe9ce59c700 1 -- 172.21.15.59:0/201806142 learned_addr learned my addr 172.21.15.59:0/201806142 (peer_addr_for_me v2:172.21.15.59:0/0) 2023-11-24T07:57:36.246 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:36.239+0000 7fe9ced9d700 1 --2- 172.21.15.59:0/201806142 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55fefc78ced0 0x55fefc7a3c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:57:36.246 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:36.239+0000 7fe9cdd9b700 1 --2- 172.21.15.59:0/201806142 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x55fefc777080 0x55fefc7cec00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:57:36.246 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:36.239+0000 7fe9ce59c700 1 -- 172.21.15.59:0/201806142 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x55fefc777080 msgr2=0x55fefc7cec00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T07:57:36.246 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:36.239+0000 7fe9ce59c700 1 --2- 172.21.15.59:0/201806142 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x55fefc777080 0x55fefc7cec00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:36.246 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:36.239+0000 7fe9ce59c700 1 -- 172.21.15.59:0/201806142 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55fefc78ced0 msgr2=0x55fefc7a3c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T07:57:36.246 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:36.239+0000 7fe9ce59c700 1 --2- 172.21.15.59:0/201806142 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55fefc78ced0 0x55fefc7a3c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:36.246 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:36.239+0000 7fe9ce59c700 1 -- 172.21.15.59:0/201806142 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe9c000f9a0 con 0x55fefc774490 2023-11-24T07:57:36.247 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:36.239+0000 7fe9cdd9b700 1 --2- 172.21.15.59:0/201806142 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x55fefc777080 0x55fefc7cec00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-24T07:57:36.247 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:36.239+0000 7fe9ce59c700 1 --2- 172.21.15.59:0/201806142 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55fefc774490 0x55fefc7ce6c0 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7fe9c0015ce0 tx=0x7fe9c00047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5d240b485e32c298 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:57:36.247 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:36.239+0000 7fe9ad7fa700 10 client.0 ms_handle_connect on v2:172.21.15.59:3300/0 2023-11-24T07:57:36.247 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:36.239+0000 7fe9ad7fa700 1 -- 172.21.15.59:0/201806142 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9c002d070 con 0x55fefc774490 2023-11-24T07:57:36.247 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:36.239+0000 7fe9ad7fa700 1 -- 172.21.15.59:0/201806142 <== mon.0 v2:172.21.15.59:3300/0 2 ==== osd_map(33..33 src has 1..33) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fe9c001a040 con 0x55fefc774490 2023-11-24T07:57:36.247 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:36.239+0000 7fe9e1194b80 1 -- 172.21.15.59:0/201806142 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55fefc769a40 con 0x55fefc774490 2023-11-24T07:57:36.247 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:36.239+0000 7fe9ad7fa700 1 -- 172.21.15.59:0/201806142 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_get_version_reply(handle=1 version=16) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe9c000f840 con 0x55fefc774490 2023-11-24T07:57:36.247 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:36.239+0000 7fe9e1194b80 10 client.5322 fetch_fsmap learned FSMap version 16 2023-11-24T07:57:36.247 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:36.239+0000 7fe9e1194b80 1 -- 172.21.15.59:0/201806142 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,fsmap=16,monmap=2+}) v3 -- 0x55fefc4f3d90 con 0x55fefc774490 2023-11-24T07:57:36.247 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:36.239+0000 7fe9ad7fa700 1 -- 172.21.15.59:0/201806142 <== mon.0 v2:172.21.15.59:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55fefc769a40 con 0x55fefc774490 2023-11-24T07:57:36.247 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:36.240+0000 7fe9ad7fa700 1 -- 172.21.15.59:0/201806142 <== mon.0 v2:172.21.15.59:3300/0 5 ==== fsmap(e 16) v1 ==== 3294+0+0 (secure 0 0 0) 0x7fe9c0028660 con 0x55fefc774490 2023-11-24T07:57:36.248 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:36.240+0000 7fe9e1194b80 10 client.5322 fetch_fsmap finished waiting for FSMap version 16 2023-11-24T07:57:36.248 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:36.240+0000 7fe9e1194b80 10 client.5322 resolve_mds: resolved name 'a' to daemon mds.a 2023-11-24T07:57:36.248 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:36.240+0000 7fe9e1194b80 20 client.5322 populate_metadata read hostname 'smithi059' 2023-11-24T07:57:36.248 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:36.240+0000 7fe9e1194b80 1 --2- 172.21.15.59:0/201806142 >> [v2:172.21.15.59:6833/2290655475,v1:172.21.15.59:6835/2290655475] conn(0x55fefc7a4140 0x55fefc7b8b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:57:36.248 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:36.240+0000 7fe9e1194b80 4 client.5322 mds_command: new command op to 5109 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-11-24T07:57:36.248 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:36.240+0000 7fe9e1194b80 1 -- 172.21.15.59:0/201806142 --> [v2:172.21.15.59:6833/2290655475,v1:172.21.15.59:6835/2290655475] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55fefc745840 con 0x55fefc7a4140 2023-11-24T07:57:36.248 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:36.240+0000 7fe9cdd9b700 1 --2- 172.21.15.59:0/201806142 >> [v2:172.21.15.59:6833/2290655475,v1:172.21.15.59:6835/2290655475] conn(0x55fefc7a4140 0x55fefc7b8b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:57:36.248 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:36.240+0000 7fe9cdd9b700 1 --2- 172.21.15.59:0/201806142 >> [v2:172.21.15.59:6833/2290655475,v1:172.21.15.59:6835/2290655475] conn(0x55fefc7a4140 0x55fefc7b8b10 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2829aaa36e5daa42 server_cookie=f5e3afcc96ca44f8 in_seq=0 out_seq=0 2023-11-24T07:57:36.248 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:36.240+0000 7fe9ad7fa700 10 client.5322 ms_handle_connect on v2:172.21.15.59:6833/2290655475 2023-11-24T07:57:36.248 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:36.241+0000 7fe9ad7fa700 1 -- 172.21.15.59:0/201806142 <== mds.0 v2:172.21.15.59:6833/2290655475 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55fefc745840 con 0x55fefc7a4140 2023-11-24T07:57:36.248 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:36.241+0000 7fe9ad7fa700 10 client.5322 handle_command_reply: tid=0 2023-11-24T07:57:36.248 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:36.241+0000 7fe9e1194b80 1 client.5322 shutdown 2023-11-24T07:57:36.249 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:36.241+0000 7fe9e1194b80 1 -- 172.21.15.59:0/201806142 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55fefc774490 msgr2=0x55fefc7ce6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:57:36.249 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:36.241+0000 7fe9e1194b80 1 --2- 172.21.15.59:0/201806142 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55fefc774490 0x55fefc7ce6c0 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7fe9c0015ce0 tx=0x7fe9c00047c0 comp rx=0 tx=0).stop 2023-11-24T07:57:36.249 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:36.241+0000 7fe9e1194b80 1 -- 172.21.15.59:0/201806142 shutdown_connections 2023-11-24T07:57:36.249 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:36.241+0000 7fe9e1194b80 1 --2- 172.21.15.59:0/201806142 >> [v2:172.21.15.59:6833/2290655475,v1:172.21.15.59:6835/2290655475] conn(0x55fefc7a4140 0x55fefc7b8b10 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:36.249 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:36.241+0000 7fe9e1194b80 1 --2- 172.21.15.59:0/201806142 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55fefc78ced0 0x55fefc7a3c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:36.249 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:36.241+0000 7fe9e1194b80 1 --2- 172.21.15.59:0/201806142 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55fefc774490 0x55fefc7ce6c0 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:36.249 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:36.241+0000 7fe9e1194b80 1 --2- 172.21.15.59:0/201806142 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x55fefc777080 0x55fefc7cec00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:36.249 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:36.241+0000 7fe9ad7fa700 0 client.5322 ms_handle_reset on v2:172.21.15.59:6833/2290655475 2023-11-24T07:57:36.249 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:36.241+0000 7fe9e1194b80 1 -- 172.21.15.59:0/201806142 >> 172.21.15.59:0/201806142 conn(0x55fefc6a5310 msgr2=0x55fefc7bfc80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:57:36.249 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:36.241+0000 7fe9e1194b80 1 -- 172.21.15.59:0/201806142 shutdown_connections 2023-11-24T07:57:36.249 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:36.241+0000 7fe9e1194b80 1 -- 172.21.15.59:0/201806142 wait complete. 2023-11-24T07:57:36.250 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:36.241+0000 7fe9e1194b80 20 client.5322 trim_cache size 0 max 16384 2023-11-24T07:57:36.260 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:36.259+0000 7fe9e1194b80 1 Processor -- start 2023-11-24T07:57:36.260 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:36.260+0000 7fe9e1194b80 1 -- start start 2023-11-24T07:57:36.262 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:36.260+0000 7fe9e1194b80 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55fefc774490 0x55fefc7b5820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:57:36.262 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:36.260+0000 7fe9e1194b80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55fefc777080 0x55fefc7b5d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:57:36.262 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:36.260+0000 7fe9e1194b80 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x55fefc78ced0 0x55fefc770f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:57:36.262 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:36.260+0000 7fe9e1194b80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fefc5498d0 con 0x55fefc777080 2023-11-24T07:57:36.262 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:36.260+0000 7fe9e1194b80 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fefc772570 con 0x55fefc774490 2023-11-24T07:57:36.263 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:36.260+0000 7fe9e1194b80 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 v1 -- 0x55fefc772750 con 0x55fefc78ced0 2023-11-24T07:57:36.263 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:36.260+0000 7fe9cdd9b700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55fefc777080 0x55fefc7b5d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:57:36.263 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:36.260+0000 7fe9cdd9b700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55fefc777080 0x55fefc7b5d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:51898/0 (socket says 172.21.15.59:51898) 2023-11-24T07:57:36.263 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:36.260+0000 7fe9cdd9b700 1 -- 172.21.15.59:0/791073109 learned_addr learned my addr 172.21.15.59:0/791073109 (peer_addr_for_me v2:172.21.15.59:0/0) 2023-11-24T07:57:36.263 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:36.261+0000 7fe9ce59c700 1 --2- 172.21.15.59:0/791073109 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55fefc774490 0x55fefc7b5820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:57:36.263 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:36.261+0000 7fe9cdd9b700 1 -- 172.21.15.59:0/791073109 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x55fefc78ced0 msgr2=0x55fefc770f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T07:57:36.263 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:36.261+0000 7fe9cdd9b700 1 --2- 172.21.15.59:0/791073109 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x55fefc78ced0 0x55fefc770f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:36.263 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:36.261+0000 7fe9cdd9b700 1 -- 172.21.15.59:0/791073109 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55fefc774490 msgr2=0x55fefc7b5820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T07:57:36.263 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:36.261+0000 7fe9cdd9b700 1 --2- 172.21.15.59:0/791073109 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55fefc774490 0x55fefc7b5820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:36.263 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:36.261+0000 7fe9cdd9b700 1 -- 172.21.15.59:0/791073109 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55fefc773330 con 0x55fefc777080 2023-11-24T07:57:36.263 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:36.261+0000 7fe9cdd9b700 1 --2- 172.21.15.59:0/791073109 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55fefc777080 0x55fefc7b5d60 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7fe9c4057540 tx=0x7fe9c4014040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a283a96cd9c432c0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:57:36.263 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:36.261+0000 7fe9bcff9700 1 -- 172.21.15.59:0/791073109 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9c40575b0 con 0x55fefc777080 2023-11-24T07:57:36.264 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:36.261+0000 7fe9bcff9700 1 -- 172.21.15.59:0/791073109 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe9c405b700 con 0x55fefc777080 2023-11-24T07:57:36.264 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:36.261+0000 7fe9bcff9700 1 -- 172.21.15.59:0/791073109 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9c405f8b0 con 0x55fefc777080 2023-11-24T07:57:36.264 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:36.261+0000 7fe9e1194b80 1 -- 172.21.15.59:0/791073109 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55fefc777080 msgr2=0x55fefc7b5d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:57:36.264 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:36.261+0000 7fe9e1194b80 1 --2- 172.21.15.59:0/791073109 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55fefc777080 0x55fefc7b5d60 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7fe9c4057540 tx=0x7fe9c4014040 comp rx=0 tx=0).stop 2023-11-24T07:57:36.264 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:36.261+0000 7fe9e1194b80 1 -- 172.21.15.59:0/791073109 shutdown_connections 2023-11-24T07:57:36.264 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:36.261+0000 7fe9e1194b80 1 --2- 172.21.15.59:0/791073109 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x55fefc78ced0 0x55fefc770f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:36.264 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:36.261+0000 7fe9e1194b80 1 --2- 172.21.15.59:0/791073109 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55fefc774490 0x55fefc7b5820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:36.264 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:36.261+0000 7fe9e1194b80 1 --2- 172.21.15.59:0/791073109 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55fefc777080 0x55fefc7b5d60 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:36.265 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:36.261+0000 7fe9e1194b80 1 -- 172.21.15.59:0/791073109 >> 172.21.15.59:0/791073109 conn(0x55fefc6a5310 msgr2=0x55fefc780990 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:57:36.265 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:36.261+0000 7fe9e1194b80 1 -- 172.21.15.59:0/791073109 shutdown_connections 2023-11-24T07:57:36.265 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:36.262+0000 7fe9e1194b80 1 -- 172.21.15.59:0/791073109 wait complete. 2023-11-24T07:57:36.268 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:36.262+0000 7fe9e1194b80 1 -- start start 2023-11-24T07:57:36.268 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:36.262+0000 7fe9e1194b80 1 Processor -- start 2023-11-24T07:57:36.269 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:36.263+0000 7fe9e1194b80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55fefc774490 0x55fefc7d9b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:57:36.269 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:36.263+0000 7fe9e1194b80 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55fefc777080 0x55fefc7d4d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:57:36.269 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:36.263+0000 7fe9e1194b80 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x55fefc78ced0 0x55fefc7b5c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:57:36.269 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:36.263+0000 7fe9e1194b80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fefc7d56b0 con 0x55fefc774490 2023-11-24T07:57:36.269 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:36.263+0000 7fe9e1194b80 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fefc7da3a0 con 0x55fefc777080 2023-11-24T07:57:36.269 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:36.263+0000 7fe9e1194b80 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 v1 -- 0x55fefc7d5460 con 0x55fefc78ced0 2023-11-24T07:57:36.269 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:36.263+0000 7fe9e1194b80 10 client.0 _ll_register_callbacks cb 0x55fefc5afa30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-24T07:57:36.269 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:36.263+0000 7fe9ce59c700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55fefc774490 0x55fefc7d9b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:57:36.269 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:36.263+0000 7fe9ce59c700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55fefc774490 0x55fefc7d9b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:51902/0 (socket says 172.21.15.59:51902) 2023-11-24T07:57:36.269 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:36.263+0000 7fe9ce59c700 1 -- 172.21.15.59:0/3399257131 learned_addr learned my addr 172.21.15.59:0/3399257131 (peer_addr_for_me v2:172.21.15.59:0/0) 2023-11-24T07:57:36.269 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:36.263+0000 7fe9cdd9b700 1 --2- 172.21.15.59:0/3399257131 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55fefc777080 0x55fefc7d4d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:57:36.270 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:36.263+0000 7fe9ced9d700 1 --2- 172.21.15.59:0/3399257131 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x55fefc78ced0 0x55fefc7b5c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:57:36.270 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:36.263+0000 7fe9ce59c700 1 -- 172.21.15.59:0/3399257131 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x55fefc78ced0 msgr2=0x55fefc7b5c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T07:57:36.270 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:36.263+0000 7fe9ce59c700 1 --2- 172.21.15.59:0/3399257131 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x55fefc78ced0 0x55fefc7b5c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:36.270 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:36.263+0000 7fe9ce59c700 1 -- 172.21.15.59:0/3399257131 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55fefc777080 msgr2=0x55fefc7d4d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T07:57:36.270 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:36.263+0000 7fe9ce59c700 1 --2- 172.21.15.59:0/3399257131 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55fefc777080 0x55fefc7d4d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:36.270 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:36.263+0000 7fe9ce59c700 1 -- 172.21.15.59:0/3399257131 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe9d013d190 con 0x55fefc774490 2023-11-24T07:57:36.270 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:36.264+0000 7fe9ce59c700 1 --2- 172.21.15.59:0/3399257131 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55fefc774490 0x55fefc7d9b10 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7fe9c0013a60 tx=0x7fe9c0019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7d14325d7d9e3012 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:57:36.270 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:36.264+0000 7fe9ad7fa700 10 client.0 ms_handle_connect on v2:172.21.15.59:3300/0 2023-11-24T07:57:36.270 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:36.264+0000 7fe9ad7fa700 1 -- 172.21.15.59:0/3399257131 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9c000a360 con 0x55fefc774490 2023-11-24T07:57:36.270 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:36.264+0000 7fe9ad7fa700 1 -- 172.21.15.59:0/3399257131 <== mon.0 v2:172.21.15.59:3300/0 2 ==== osd_map(33..33 src has 1..33) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fe9c002ae40 con 0x55fefc774490 2023-11-24T07:57:36.270 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:36.264+0000 7fe9e1194b80 1 -- 172.21.15.59:0/3399257131 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fe9c000c040 con 0x55fefc774490 2023-11-24T07:57:36.270 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:36.264+0000 7fe9ad7fa700 1 -- 172.21.15.59:0/3399257131 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_get_version_reply(handle=1 version=16) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe9c000f7e0 con 0x55fefc774490 2023-11-24T07:57:36.271 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:36.264+0000 7fe9e1194b80 10 client.5328 fetch_fsmap learned FSMap version 16 2023-11-24T07:57:36.271 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:36.264+0000 7fe9e1194b80 1 -- 172.21.15.59:0/3399257131 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,fsmap=16,monmap=2+}) v3 -- 0x55fefc4dcc10 con 0x55fefc774490 2023-11-24T07:57:36.271 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:36.264+0000 7fe9ad7fa700 1 -- 172.21.15.59:0/3399257131 <== mon.0 v2:172.21.15.59:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe9c000c040 con 0x55fefc774490 2023-11-24T07:57:36.271 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:36.264+0000 7fe9ad7fa700 1 -- 172.21.15.59:0/3399257131 <== mon.0 v2:172.21.15.59:3300/0 5 ==== fsmap(e 16) v1 ==== 3294+0+0 (secure 0 0 0) 0x7fe9c0028660 con 0x55fefc774490 2023-11-24T07:57:36.271 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:36.264+0000 7fe9e1194b80 10 client.5328 fetch_fsmap finished waiting for FSMap version 16 2023-11-24T07:57:36.271 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:36.264+0000 7fe9e1194b80 10 client.5328 resolve_mds: resolved name 'a' to daemon mds.a 2023-11-24T07:57:36.271 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:36.264+0000 7fe9e1194b80 20 client.5328 populate_metadata read hostname 'smithi059' 2023-11-24T07:57:36.271 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:36.264+0000 7fe9e1194b80 1 --2- 172.21.15.59:0/3399257131 >> [v2:172.21.15.59:6833/2290655475,v1:172.21.15.59:6835/2290655475] conn(0x55fefc8271a0 0x55fefc827580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:57:36.271 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:36.264+0000 7fe9e1194b80 4 client.5328 mds_command: new command op to 5109 tid=0 multi_id=0 [{"prefix": "status"}] 2023-11-24T07:57:36.271 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:36.264+0000 7fe9e1194b80 1 -- 172.21.15.59:0/3399257131 --> [v2:172.21.15.59:6833/2290655475,v1:172.21.15.59:6835/2290655475] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55fefc690f80 con 0x55fefc8271a0 2023-11-24T07:57:36.271 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:36.264+0000 7fe9cdd9b700 1 --2- 172.21.15.59:0/3399257131 >> [v2:172.21.15.59:6833/2290655475,v1:172.21.15.59:6835/2290655475] conn(0x55fefc8271a0 0x55fefc827580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:57:36.271 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:36.265+0000 7fe9ad7fa700 10 client.5328 ms_handle_connect on v2:172.21.15.59:6833/2290655475 2023-11-24T07:57:36.272 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:36.265+0000 7fe9cdd9b700 1 --2- 172.21.15.59:0/3399257131 >> [v2:172.21.15.59:6833/2290655475,v1:172.21.15.59:6835/2290655475] conn(0x55fefc8271a0 0x55fefc827580 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=41a9a9204c978d33 server_cookie=110abc706b885d27 in_seq=0 out_seq=0 2023-11-24T07:57:36.272 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:36.265+0000 7fe9ad7fa700 1 -- 172.21.15.59:0/3399257131 <== mds.0 v2:172.21.15.59:6833/2290655475 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+331 (crc 0 0 0) 0x7fe9c4071160 con 0x55fefc8271a0 2023-11-24T07:57:36.272 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:36.265+0000 7fe9ad7fa700 10 client.5328 handle_command_reply: tid=0 2023-11-24T07:57:36.272 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:36.265+0000 7fe9e1194b80 1 client.5328 shutdown 2023-11-24T07:57:36.272 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:36.265+0000 7fe9e1194b80 1 -- 172.21.15.59:0/3399257131 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55fefc774490 msgr2=0x55fefc7d9b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:57:36.272 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:36.265+0000 7fe9e1194b80 1 --2- 172.21.15.59:0/3399257131 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55fefc774490 0x55fefc7d9b10 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7fe9c0013a60 tx=0x7fe9c0019000 comp rx=0 tx=0).stop 2023-11-24T07:57:36.272 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:36.265+0000 7fe9e1194b80 1 -- 172.21.15.59:0/3399257131 shutdown_connections 2023-11-24T07:57:36.272 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:36.265+0000 7fe9e1194b80 1 --2- 172.21.15.59:0/3399257131 >> [v2:172.21.15.59:6833/2290655475,v1:172.21.15.59:6835/2290655475] conn(0x55fefc8271a0 0x55fefc827580 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:36.272 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:36.265+0000 7fe9e1194b80 1 --2- 172.21.15.59:0/3399257131 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x55fefc78ced0 0x55fefc7b5c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:36.272 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:36.265+0000 7fe9e1194b80 1 --2- 172.21.15.59:0/3399257131 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55fefc774490 0x55fefc7d9b10 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:36.272 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:36.265+0000 7fe9e1194b80 1 --2- 172.21.15.59:0/3399257131 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55fefc777080 0x55fefc7d4d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:36.273 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:36.265+0000 7fe9ad7fa700 0 client.5328 ms_handle_reset on v2:172.21.15.59:6833/2290655475 2023-11-24T07:57:36.273 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:36.265+0000 7fe9e1194b80 1 -- 172.21.15.59:0/3399257131 >> 172.21.15.59:0/3399257131 conn(0x55fefc6a5310 msgr2=0x55fefc7b6e20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:57:36.273 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:36.265+0000 7fe9e1194b80 1 -- 172.21.15.59:0/3399257131 shutdown_connections 2023-11-24T07:57:36.273 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:36.265+0000 7fe9e1194b80 1 -- 172.21.15.59:0/3399257131 wait complete. 2023-11-24T07:57:36.273 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:36.265+0000 7fe9e1194b80 20 client.5328 trim_cache size 0 max 16384 2023-11-24T07:57:36.273 INFO:teuthology.orchestra.run.smithi059.stdout:{ 2023-11-24T07:57:36.273 INFO:teuthology.orchestra.run.smithi059.stdout: "cluster_fsid": "9f0e8741-b8f9-48d5-a2c3-09f45903303d", 2023-11-24T07:57:36.273 INFO:teuthology.orchestra.run.smithi059.stdout: "whoami": 0, 2023-11-24T07:57:36.273 INFO:teuthology.orchestra.run.smithi059.stdout: "id": 5109, 2023-11-24T07:57:36.273 INFO:teuthology.orchestra.run.smithi059.stdout: "want_state": "up:active", 2023-11-24T07:57:36.274 INFO:teuthology.orchestra.run.smithi059.stdout: "state": "up:active", 2023-11-24T07:57:36.274 INFO:teuthology.orchestra.run.smithi059.stdout: "fs_name": "cephfs", 2023-11-24T07:57:36.274 INFO:teuthology.orchestra.run.smithi059.stdout: "rank_uptime": 10.685516334000001, 2023-11-24T07:57:36.274 INFO:teuthology.orchestra.run.smithi059.stdout: "mdsmap_epoch": 16, 2023-11-24T07:57:36.274 INFO:teuthology.orchestra.run.smithi059.stdout: "osdmap_epoch": 33, 2023-11-24T07:57:36.274 INFO:teuthology.orchestra.run.smithi059.stdout: "osdmap_epoch_barrier": 33, 2023-11-24T07:57:36.274 INFO:teuthology.orchestra.run.smithi059.stdout: "uptime": 18.650865958000001 2023-11-24T07:57:36.274 INFO:teuthology.orchestra.run.smithi059.stdout:} 2023-11-24T07:57:36.275 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:36.266+0000 7fe9d4f9b700 1 -- 172.21.15.59:0/3418891561 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7fe9b006aed0 msgr2=0x7fe9b006d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:57:36.275 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:36.266+0000 7fe9d4f9b700 1 --2- 172.21.15.59:0/3418891561 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7fe9b006aed0 0x7fe9b006d390 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fe9c0006fd0 tx=0x7fe9c0008040 comp rx=0 tx=0).stop 2023-11-24T07:57:36.275 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:36.266+0000 7fe9d4f9b700 1 -- 172.21.15.59:0/3418891561 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe9d006ee60 msgr2=0x7fe9d00cebd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:57:36.275 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:36.266+0000 7fe9d4f9b700 1 --2- 172.21.15.59:0/3418891561 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe9d006ee60 0x7fe9d00cebd0 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7fe9c400c420 tx=0x7fe9c400c730 comp rx=0 tx=0).stop 2023-11-24T07:57:36.275 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:36.266+0000 7fe9d4f9b700 1 -- 172.21.15.59:0/3418891561 shutdown_connections 2023-11-24T07:57:36.276 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:36.266+0000 7fe9d4f9b700 1 --2- 172.21.15.59:0/3418891561 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7fe9b006aed0 0x7fe9b006d390 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:36.276 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:36.266+0000 7fe9d4f9b700 1 --2- 172.21.15.59:0/3418891561 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe9d00b5ab0 0x7fe9d01f72f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:36.276 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:36.266+0000 7fe9d4f9b700 1 --2- 172.21.15.59:0/3418891561 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fe9d006b620 0x7fe9d01fb700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:36.276 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:36.266+0000 7fe9d4f9b700 1 --2- 172.21.15.59:0/3418891561 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe9d006ee60 0x7fe9d00cebd0 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:36.276 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:36.266+0000 7fe9d4f9b700 1 -- 172.21.15.59:0/3418891561 >> 172.21.15.59:0/3418891561 conn(0x7fe9d00b2740 msgr2=0x7fe9d00b3830 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:57:36.276 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:36.266+0000 7fe9d4f9b700 1 -- 172.21.15.59:0/3418891561 shutdown_connections 2023-11-24T07:57:36.276 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:36.266+0000 7fe9d4f9b700 1 -- 172.21.15.59:0/3418891561 wait complete. 2023-11-24T07:57:36.279 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-11-24T07:57:36.404 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:36.403+0000 7fbf1f89d700 1 Processor -- start 2023-11-24T07:57:36.404 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:36.403+0000 7fbf1f89d700 1 -- start start 2023-11-24T07:57:36.407 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:36.404+0000 7fbf1f89d700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fbf18143ae0 0x7fbf180b51b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:57:36.407 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:36.404+0000 7fbf1f89d700 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fbf181440f0 0x7fbf18071bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:57:36.407 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:36.404+0000 7fbf1f89d700 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbf18143ec0 con 0x7fbf180720f0 2023-11-24T07:57:36.407 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:36.404+0000 7fbf1f89d700 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbf180b40a0 con 0x7fbf18143ae0 2023-11-24T07:57:36.407 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:36.404+0000 7fbf1f89d700 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbf18072560 con 0x7fbf181440f0 2023-11-24T07:57:36.407 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:36.404+0000 7fbf1d639700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fbf18143ae0 0x7fbf180b51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:57:36.407 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:36.404+0000 7fbf1de3a700 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7fbf180720f0 0x7fbf1815e8b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:55388/0 (socket says 172.21.15.59:55388) 2023-11-24T07:57:36.407 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:36.404+0000 7fbf1d639700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fbf18143ae0 0x7fbf180b51b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:51910/0 (socket says 172.21.15.59:51910) 2023-11-24T07:57:36.407 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:36.404+0000 7fbf1de3a700 1 -- 172.21.15.59:0/1967001074 learned_addr learned my addr 172.21.15.59:0/1967001074 (peer_addr_for_me v1:172.21.15.59:0/0) 2023-11-24T07:57:36.407 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:36.404+0000 7fbf1ce38700 1 --2- 172.21.15.59:0/1967001074 >> v2:172.21.15.76:3300/0 conn(0x7fbf181440f0 0x7fbf18071bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:57:36.407 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:36.404+0000 7fbf1d639700 1 -- 172.21.15.59:0/1967001074 >> v2:172.21.15.76:3300/0 conn(0x7fbf181440f0 msgr2=0x7fbf18071bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T07:57:36.408 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:36.404+0000 7fbf1d639700 1 --2- 172.21.15.59:0/1967001074 >> v2:172.21.15.76:3300/0 conn(0x7fbf181440f0 0x7fbf18071bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:36.408 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:36.404+0000 7fbf1d639700 1 -- 172.21.15.59:0/1967001074 >> v1:172.21.15.59:6789/0 conn(0x7fbf180720f0 legacy=0x7fbf1815e8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:57:36.408 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:36.405+0000 7fbf0bfff700 1 -- 172.21.15.59:0/1967001074 <== mon.0 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 784566227 0 0) 0x7fbf18143ec0 con 0x7fbf180720f0 2023-11-24T07:57:36.408 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:36.405+0000 7fbf1d639700 1 -- 172.21.15.59:0/1967001074 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbf180b3ae0 con 0x7fbf18143ae0 2023-11-24T07:57:36.408 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:36.405+0000 7fbf1ce38700 1 --2- 172.21.15.59:0/1967001074 >> v2:172.21.15.76:3300/0 conn(0x7fbf181440f0 0x7fbf18071bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T07:57:36.408 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:36.405+0000 7fbf1d639700 1 --2- 172.21.15.59:0/1967001074 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbf18143ae0 0x7fbf180b51b0 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7fbf180d1b00 tx=0x7fbf0c009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4e0e2f62525a70a3 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:57:36.408 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:36.405+0000 7fbf0bfff700 1 -- 172.21.15.59:0/1967001074 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf0c011830 con 0x7fbf18143ae0 2023-11-24T07:57:36.408 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:36.405+0000 7fbf0bfff700 1 -- 172.21.15.59:0/1967001074 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbf0c011990 con 0x7fbf18143ae0 2023-11-24T07:57:36.408 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:36.405+0000 7fbf0bfff700 1 -- 172.21.15.59:0/1967001074 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf0c011d20 con 0x7fbf18143ae0 2023-11-24T07:57:36.408 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:36.405+0000 7fbf1f89d700 1 -- 172.21.15.59:0/1967001074 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbf18143ae0 msgr2=0x7fbf180b51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:57:36.408 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:36.405+0000 7fbf1f89d700 1 --2- 172.21.15.59:0/1967001074 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbf18143ae0 0x7fbf180b51b0 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7fbf180d1b00 tx=0x7fbf0c009b50 comp rx=0 tx=0).stop 2023-11-24T07:57:36.408 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:36.405+0000 7fbf1f89d700 1 -- 172.21.15.59:0/1967001074 shutdown_connections 2023-11-24T07:57:36.408 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:36.405+0000 7fbf1f89d700 1 --2- 172.21.15.59:0/1967001074 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbf18143ae0 0x7fbf180b51b0 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:36.409 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:36.405+0000 7fbf1f89d700 1 --2- 172.21.15.59:0/1967001074 >> v2:172.21.15.76:3300/0 conn(0x7fbf181440f0 0x7fbf18071bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:36.409 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:36.406+0000 7fbf1f89d700 1 -- 172.21.15.59:0/1967001074 >> 172.21.15.59:0/1967001074 conn(0x7fbf180b2740 msgr2=0x7fbf180b2b20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:57:36.409 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:36.406+0000 7fbf1f89d700 1 -- 172.21.15.59:0/1967001074 shutdown_connections 2023-11-24T07:57:36.409 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:36.406+0000 7fbf1f89d700 1 -- 172.21.15.59:0/1967001074 wait complete. 2023-11-24T07:57:36.411 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:36.407+0000 7fbf1f89d700 1 Processor -- start 2023-11-24T07:57:36.412 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:36.407+0000 7fbf1f89d700 1 -- start start 2023-11-24T07:57:36.412 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:36.407+0000 7fbf1f89d700 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fbf180720f0 0x7fbf181585f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:57:36.412 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:36.408+0000 7fbf1f89d700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbf18143ae0 0x7fbf1813af80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:57:36.412 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:36.408+0000 7fbf1f89d700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fbf181440f0 0x7fbf18137950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:57:36.412 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:36.408+0000 7fbf1f89d700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf18158bf0 con 0x7fbf18143ae0 2023-11-24T07:57:36.412 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:36.408+0000 7fbf1f89d700 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf18158ed0 con 0x7fbf181440f0 2023-11-24T07:57:36.412 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:36.408+0000 7fbf1f89d700 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbf181f4e30 con 0x7fbf180720f0 2023-11-24T07:57:36.412 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:36.408+0000 7fbf1ce38700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbf18143ae0 0x7fbf1813af80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:57:36.412 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:36.408+0000 7fbf1ce38700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbf18143ae0 0x7fbf1813af80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:51918/0 (socket says 172.21.15.59:51918) 2023-11-24T07:57:36.412 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:36.408+0000 7fbf1ce38700 1 -- 172.21.15.59:0/3697307945 learned_addr learned my addr 172.21.15.59:0/3697307945 (peer_addr_for_me v2:172.21.15.59:0/0) 2023-11-24T07:57:36.413 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:36.408+0000 7fbf1de3a700 1 --2- 172.21.15.59:0/3697307945 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fbf181440f0 0x7fbf18137950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:57:36.413 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:36.408+0000 7fbf1d639700 1 --2- 172.21.15.59:0/3697307945 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fbf180720f0 0x7fbf181585f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:57:36.413 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:36.408+0000 7fbf1ce38700 1 -- 172.21.15.59:0/3697307945 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fbf180720f0 msgr2=0x7fbf181585f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:57:36.413 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:36.408+0000 7fbf1ce38700 1 --2- 172.21.15.59:0/3697307945 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fbf180720f0 0x7fbf181585f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:36.413 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:36.408+0000 7fbf1ce38700 1 -- 172.21.15.59:0/3697307945 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fbf181440f0 msgr2=0x7fbf18137950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:57:36.413 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:36.408+0000 7fbf1ce38700 1 --2- 172.21.15.59:0/3697307945 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fbf181440f0 0x7fbf18137950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:36.413 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:36.408+0000 7fbf1ce38700 1 -- 172.21.15.59:0/3697307945 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbf0c0094a0 con 0x7fbf18143ae0 2023-11-24T07:57:36.413 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:36.408+0000 7fbf1d639700 1 --2- 172.21.15.59:0/3697307945 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fbf180720f0 0x7fbf181585f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-24T07:57:36.413 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:36.408+0000 7fbf1ce38700 1 --2- 172.21.15.59:0/3697307945 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbf18143ae0 0x7fbf1813af80 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7fbf1400c7f0 tx=0x7fbf1400cb00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:57:36.413 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:36.408+0000 7fbf09ffb700 1 -- 172.21.15.59:0/3697307945 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf14011800 con 0x7fbf18143ae0 2023-11-24T07:57:36.413 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:36.408+0000 7fbf09ffb700 1 -- 172.21.15.59:0/3697307945 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbf14011960 con 0x7fbf18143ae0 2023-11-24T07:57:36.413 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:36.408+0000 7fbf1f89d700 1 -- 172.21.15.59:0/3697307945 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbf181f5110 con 0x7fbf18143ae0 2023-11-24T07:57:36.413 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:36.408+0000 7fbf09ffb700 1 -- 172.21.15.59:0/3697307945 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf14011cf0 con 0x7fbf18143ae0 2023-11-24T07:57:36.414 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:36.409+0000 7fbf1f89d700 1 -- 172.21.15.59:0/3697307945 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbf181591b0 con 0x7fbf18143ae0 2023-11-24T07:57:36.414 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:36.410+0000 7fbf1f89d700 1 -- 172.21.15.59:0/3697307945 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbf181f5480 con 0x7fbf18143ae0 2023-11-24T07:57:36.414 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:36.410+0000 7fbf09ffb700 1 -- 172.21.15.59:0/3697307945 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fbf14018360 con 0x7fbf18143ae0 2023-11-24T07:57:36.414 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:36.410+0000 7fbf09ffb700 1 --2- 172.21.15.59:0/3697307945 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7fbefc06aed0 0x7fbefc06d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:57:36.414 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:36.411+0000 7fbf09ffb700 1 -- 172.21.15.59:0/3697307945 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fbf14089600 con 0x7fbf18143ae0 2023-11-24T07:57:36.414 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:36.411+0000 7fbf1d639700 1 --2- 172.21.15.59:0/3697307945 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7fbefc06aed0 0x7fbefc06d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:57:36.414 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:36.411+0000 7fbf1d639700 1 --2- 172.21.15.59:0/3697307945 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7fbefc06aed0 0x7fbefc06d390 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fbf0c000c00 tx=0x7fbf0c01c040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:57:36.415 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:36.415+0000 7fbf09ffb700 1 -- 172.21.15.59:0/3697307945 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fbf14054620 con 0x7fbf18143ae0 2023-11-24T07:57:36.663 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:36.662+0000 7fbf1f89d700 1 -- 172.21.15.59:0/3697307945 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7fbf18144830 con 0x7fbf18143ae0 2023-11-24T07:57:36.663 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:36.662+0000 7fbf09ffb700 1 -- 172.21.15.59:0/3697307945 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v16) v1 ==== 53+0+73 (secure 0 0 0) 0x7fbf1405e070 con 0x7fbf18143ae0 2023-11-24T07:57:36.663 INFO:teuthology.orchestra.run.smithi059.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-11-24T07:57:36.665 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:36.665+0000 7fbf1f89d700 1 -- 172.21.15.59:0/3697307945 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7fbefc06aed0 msgr2=0x7fbefc06d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:57:36.666 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:36.665+0000 7fbf1f89d700 1 --2- 172.21.15.59:0/3697307945 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7fbefc06aed0 0x7fbefc06d390 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fbf0c000c00 tx=0x7fbf0c01c040 comp rx=0 tx=0).stop 2023-11-24T07:57:36.666 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:36.665+0000 7fbf1f89d700 1 -- 172.21.15.59:0/3697307945 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbf18143ae0 msgr2=0x7fbf1813af80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:57:36.666 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:36.665+0000 7fbf1f89d700 1 --2- 172.21.15.59:0/3697307945 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbf18143ae0 0x7fbf1813af80 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7fbf1400c7f0 tx=0x7fbf1400cb00 comp rx=0 tx=0).stop 2023-11-24T07:57:36.666 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:36.665+0000 7fbf1f89d700 1 -- 172.21.15.59:0/3697307945 shutdown_connections 2023-11-24T07:57:36.666 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:36.665+0000 7fbf1f89d700 1 --2- 172.21.15.59:0/3697307945 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7fbefc06aed0 0x7fbefc06d390 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:36.666 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:36.665+0000 7fbf1f89d700 1 --2- 172.21.15.59:0/3697307945 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fbf181440f0 0x7fbf18137950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:36.666 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:36.665+0000 7fbf1f89d700 1 --2- 172.21.15.59:0/3697307945 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fbf180720f0 0x7fbf181585f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:36.666 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:36.665+0000 7fbf1f89d700 1 --2- 172.21.15.59:0/3697307945 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbf18143ae0 0x7fbf1813af80 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:36.666 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:36.665+0000 7fbf1f89d700 1 -- 172.21.15.59:0/3697307945 >> 172.21.15.59:0/3697307945 conn(0x7fbf180b2740 msgr2=0x7fbf1813f240 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:57:36.666 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:36.665+0000 7fbf1f89d700 1 -- 172.21.15.59:0/3697307945 shutdown_connections 2023-11-24T07:57:36.666 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:36.665+0000 7fbf1f89d700 1 -- 172.21.15.59:0/3697307945 wait complete. 2023-11-24T07:57:36.677 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2023-11-24T07:57:36.678 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-11-24T07:57:36.678 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi059.front.sepia.ceph.com 2023-11-24T07:57:36.678 INFO:tasks.cephfs.mount:self.client.name = client.0 2023-11-24T07:57:36.678 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-11-24T07:57:36.678 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2023-11-24T07:57:36.678 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2023-11-24T07:57:36.678 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-11-24T07:57:36.678 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-11-24T07:57:36.678 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T07:57:36.679 DEBUG:teuthology.orchestra.run.smithi059:> ip addr 2023-11-24T07:57:36.696 INFO:teuthology.orchestra.run.smithi059.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-11-24T07:57:36.696 INFO:teuthology.orchestra.run.smithi059.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-11-24T07:57:36.696 INFO:teuthology.orchestra.run.smithi059.stdout: inet 127.0.0.1/8 scope host lo 2023-11-24T07:57:36.696 INFO:teuthology.orchestra.run.smithi059.stdout: valid_lft forever preferred_lft forever 2023-11-24T07:57:36.696 INFO:teuthology.orchestra.run.smithi059.stdout: inet6 ::1/128 scope host 2023-11-24T07:57:36.696 INFO:teuthology.orchestra.run.smithi059.stdout: valid_lft forever preferred_lft forever 2023-11-24T07:57:36.697 INFO:teuthology.orchestra.run.smithi059.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-11-24T07:57:36.697 INFO:teuthology.orchestra.run.smithi059.stdout: link/ether 0c:c4:7a:6b:fe:ca brd ff:ff:ff:ff:ff:ff 2023-11-24T07:57:36.697 INFO:teuthology.orchestra.run.smithi059.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-11-24T07:57:36.697 INFO:teuthology.orchestra.run.smithi059.stdout: link/ether 0c:c4:7a:6b:fe:cb brd ff:ff:ff:ff:ff:ff 2023-11-24T07:57:36.697 INFO:teuthology.orchestra.run.smithi059.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2023-11-24T07:57:36.697 INFO:teuthology.orchestra.run.smithi059.stdout: link/ether 0c:c4:7a:bd:19:2a brd ff:ff:ff:ff:ff:ff 2023-11-24T07:57:36.697 INFO:teuthology.orchestra.run.smithi059.stdout: inet 172.21.15.59/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2023-11-24T07:57:36.697 INFO:teuthology.orchestra.run.smithi059.stdout: valid_lft 42311sec preferred_lft 42311sec 2023-11-24T07:57:36.697 INFO:teuthology.orchestra.run.smithi059.stdout: inet6 fe80::ec4:7aff:febd:192a/64 scope link 2023-11-24T07:57:36.697 INFO:teuthology.orchestra.run.smithi059.stdout: valid_lft forever preferred_lft forever 2023-11-24T07:57:36.697 INFO:teuthology.orchestra.run.smithi059.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-11-24T07:57:36.697 INFO:teuthology.orchestra.run.smithi059.stdout: link/ether 0c:c4:7a:bd:19:2b brd ff:ff:ff:ff:ff:ff 2023-11-24T07:57:36.697 INFO:teuthology.orchestra.run.smithi059.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-11-24T07:57:36.698 INFO:teuthology.orchestra.run.smithi059.stdout: link/ether 72:f2:ce:27:f6:18 brd ff:ff:ff:ff:ff:ff 2023-11-24T07:57:36.698 INFO:teuthology.orchestra.run.smithi059.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-11-24T07:57:36.698 INFO:teuthology.orchestra.run.smithi059.stdout: valid_lft forever preferred_lft forever 2023-11-24T07:57:36.698 INFO:teuthology.orchestra.run.smithi059.stdout: inet6 fe80::64c1:62ff:feb9:5b59/64 scope link 2023-11-24T07:57:36.698 INFO:teuthology.orchestra.run.smithi059.stdout: valid_lft forever preferred_lft forever 2023-11-24T07:57:36.698 INFO:teuthology.orchestra.run.smithi059.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-11-24T07:57:36.698 INFO:teuthology.orchestra.run.smithi059.stdout: link/ether 92:6a:7d:e0:5d:9b brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2023-11-24T07:57:36.698 INFO:teuthology.orchestra.run.smithi059.stdout: inet6 fe80::906a:7dff:fee0:5d9b/64 scope link 2023-11-24T07:57:36.698 INFO:teuthology.orchestra.run.smithi059.stdout: valid_lft forever preferred_lft forever 2023-11-24T07:57:36.698 INFO:teuthology.orchestra.run.smithi059.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-11-24T07:57:36.698 INFO:teuthology.orchestra.run.smithi059.stdout: link/ether 72:f2:ce:27:f6:18 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2023-11-24T07:57:36.699 INFO:teuthology.orchestra.run.smithi059.stdout: inet6 fe80::70f2:ceff:fe27:f618/64 scope link 2023-11-24T07:57:36.699 INFO:teuthology.orchestra.run.smithi059.stdout: valid_lft forever preferred_lft forever 2023-11-24T07:57:36.699 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T07:57:36.699 DEBUG:teuthology.orchestra.run.smithi059:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-11-24T07:57:36.763 INFO:teuthology.orchestra.run.smithi059.stdout:1 2023-11-24T07:57:36.764 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T07:57:36.764 DEBUG:teuthology.orchestra.run.smithi059:> route 2023-11-24T07:57:36.823 INFO:teuthology.orchestra.run.smithi059.stdout:Kernel IP routing table 2023-11-24T07:57:36.824 INFO:teuthology.orchestra.run.smithi059.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-11-24T07:57:36.824 INFO:teuthology.orchestra.run.smithi059.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2023-11-24T07:57:36.824 INFO:teuthology.orchestra.run.smithi059.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2023-11-24T07:57:36.824 INFO:teuthology.orchestra.run.smithi059.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-11-24T07:57:36.824 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T07:57:36.824 DEBUG:teuthology.orchestra.run.smithi059:> (cd / && exec bash -c ' 2023-11-24T07:57:36.825 DEBUG:teuthology.orchestra.run.smithi059:> set -e 2023-11-24T07:57:36.825 DEBUG:teuthology.orchestra.run.smithi059:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2023-11-24T07:57:36.825 DEBUG:teuthology.orchestra.run.smithi059:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2023-11-24T07:57:36.825 DEBUG:teuthology.orchestra.run.smithi059:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2023-11-24T07:57:36.825 DEBUG:teuthology.orchestra.run.smithi059:> ') 2023-11-24T07:57:36.931 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T07:57:36.931 DEBUG:teuthology.orchestra.run.smithi059:> ip netns list 2023-11-24T07:57:36.947 INFO:teuthology.orchestra.run.smithi059.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2023-11-24T07:57:36.948 INFO:teuthology.orchestra.run.smithi059.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-11-24T07:57:36.948 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-11-24T07:57:36.948 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T07:57:36.948 DEBUG:teuthology.orchestra.run.smithi059:> sudo ip link set brx.0 up 2023-11-24T07:57:37.015 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-11-24T07:57:37.015 DEBUG:teuthology.orchestra.run.smithi059:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2023-11-24T07:57:37.072 INFO:teuthology.orchestra.run.smithi059.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2023-11-24T07:57:37.073 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-11-24T07:57:37.073 DEBUG:teuthology.orchestra.run.smithi059:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2023-11-24T07:57:37.129 INFO:tasks.cephfs.kernel_mount:mounting using device: 0@.cephfs=/ 2023-11-24T07:57:37.129 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T07:57:37.130 DEBUG:teuthology.orchestra.run.smithi059:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 0@.cephfs=/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-11-24T07:57:37.216 INFO:teuthology.orchestra.run.smithi059.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-11-24T07:57:37.216 INFO:teuthology.orchestra.run.smithi059.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-11-24T07:57:37.248 INFO:teuthology.orchestra.run.smithi059.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-11-24T07:57:37.248 INFO:teuthology.orchestra.run.smithi059.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-11-24T07:57:37.248 INFO:teuthology.orchestra.run.smithi059.stdout:mount.ceph: resolved to: "172.21.15.59:3300,172.21.15.76:3300,172.21.15.76:3301" 2023-11-24T07:57:37.248 INFO:teuthology.orchestra.run.smithi059.stdout:mount.ceph: trying mount with new device syntax: 0@9f0e8741-b8f9-48d5-a2c3-09f45903303d.cephfs=/ 2023-11-24T07:57:37.248 INFO:teuthology.orchestra.run.smithi059.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,ms_mode=prefer-crc,key=0,mon_addr=172.21.15.59:3300/172.21.15.76:3300/172.21.15.76:3301" will pass to kernel 2023-11-24T07:57:37.248 INFO:teuthology.orchestra.run.smithi059.stdout:mount.ceph: v2 mount support check returned 2 2023-11-24T07:57:37.248 INFO:teuthology.orchestra.run.smithi059.stdout:mount.ceph: kernel does not support v2 syntax 2023-11-24T07:57:37.248 INFO:teuthology.orchestra.run.smithi059.stdout:mount.ceph: switching to using v1 address with old syntax 2023-11-24T07:57:37.288 INFO:teuthology.orchestra.run.smithi059.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-11-24T07:57:37.288 INFO:teuthology.orchestra.run.smithi059.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-11-24T07:57:37.288 INFO:teuthology.orchestra.run.smithi059.stdout:mount.ceph: resolved to: "172.21.15.59:3300,172.21.15.76:3300,172.21.15.76:3301" 2023-11-24T07:57:37.288 INFO:teuthology.orchestra.run.smithi059.stdout:mount.ceph: trying mount with new device syntax: 0@9f0e8741-b8f9-48d5-a2c3-09f45903303d.cephfs=/ 2023-11-24T07:57:37.288 INFO:teuthology.orchestra.run.smithi059.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,ms_mode=prefer-crc,key=0,mon_addr=172.21.15.59:3300/172.21.15.76:3300/172.21.15.76:3301" will pass to kernel 2023-11-24T07:57:37.288 INFO:teuthology.orchestra.run.smithi059.stdout:mount.ceph: v2 mount support check returned 2 2023-11-24T07:57:37.288 INFO:teuthology.orchestra.run.smithi059.stdout:mount.ceph: kernel does not support v2 syntax 2023-11-24T07:57:37.288 INFO:teuthology.orchestra.run.smithi059.stdout:mount.ceph: switching to using v1 address with old syntax 2023-11-24T07:57:37.289 INFO:teuthology.orchestra.run.smithi059.stdout:mount.ceph: resolved to: "172.21.15.59:6789,172.21.15.76:6789,172.21.15.76:6790" 2023-11-24T07:57:37.289 INFO:teuthology.orchestra.run.smithi059.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.59:6789,172.21.15.76:6789,172.21.15.76:6790:/ 2023-11-24T07:57:37.289 INFO:teuthology.orchestra.run.smithi059.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,key=0,mds_namespace=cephfs,fsid=9f0e8741-b8f9-48d5-a2c3-09f45903303d" will pass to kernel 2023-11-24T07:57:37.293 INFO:tasks.cephfs.kernel_mount:mount command passed 2023-11-24T07:57:37.293 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T07:57:37.293 DEBUG:teuthology.orchestra.run.smithi059:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2023-11-24T07:57:37.327 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T07:57:37.327 DEBUG:teuthology.orchestra.run.smithi059:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-11-24T07:57:37.410 INFO:teuthology.orchestra.run.smithi059.stdout:client5340 2023-11-24T07:57:37.410 DEBUG:teuthology.orchestra.run.smithi059:> set -ex 2023-11-24T07:57:37.410 DEBUG:teuthology.orchestra.run.smithi059:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-24T07:57:37.469 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-11-24T07:57:37.633 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:37.632+0000 7f6173d86700 1 Processor -- start 2023-11-24T07:57:37.633 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:37.633+0000 7f6173d86700 1 -- start start 2023-11-24T07:57:37.637 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:37.633+0000 7f6173d86700 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f616c0b03b0 0x7f616c0b5e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:57:37.637 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:37.633+0000 7f6173d86700 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f616c0adda0 0x7f616c06b970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:57:37.637 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:37.633+0000 7f6173d86700 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f616c0b0ec0 con 0x7f616c15ae00 2023-11-24T07:57:37.637 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:37.633+0000 7f6173d86700 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f616c073610 con 0x7f616c0adda0 2023-11-24T07:57:37.638 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:37.633+0000 7f6173d86700 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f616c15b1e0 con 0x7f616c0b03b0 2023-11-24T07:57:37.638 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:37.633+0000 7f6171b22700 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7f616c15ae00 0x7f616c0ad690 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:55402/0 (socket says 172.21.15.59:55402) 2023-11-24T07:57:37.638 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:37.633+0000 7f6171b22700 1 -- 172.21.15.59:0/1299262219 learned_addr learned my addr 172.21.15.59:0/1299262219 (peer_addr_for_me v1:172.21.15.59:0/0) 2023-11-24T07:57:37.638 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:37.634+0000 7f6171321700 1 --2- 172.21.15.59:0/1299262219 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f616c0b03b0 0x7f616c0b5e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:57:37.638 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:37.634+0000 7f6172323700 1 --2- 172.21.15.59:0/1299262219 >> v2:172.21.15.76:3300/0 conn(0x7f616c0adda0 0x7f616c06b970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:57:37.638 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:37.634+0000 7f6170b20700 1 -- 172.21.15.59:0/1299262219 <== mon.0 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3616452908 0 0) 0x7f616c0b0ec0 con 0x7f616c15ae00 2023-11-24T07:57:37.639 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:37.634+0000 7f6170b20700 1 -- 172.21.15.59:0/1299262219 --> v1:172.21.15.59:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f61580035b0 con 0x7f616c15ae00 2023-11-24T07:57:37.639 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:37.634+0000 7f6170b20700 1 -- 172.21.15.59:0/1299262219 <== mon.0 v1:172.21.15.59:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3512194434 0 0) 0x7f61580035b0 con 0x7f616c15ae00 2023-11-24T07:57:37.639 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:37.634+0000 7f6170b20700 1 -- 172.21.15.59:0/1299262219 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f616c0b03b0 msgr2=0x7f616c0b5e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T07:57:37.639 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:37.634+0000 7f6170b20700 1 --2- 172.21.15.59:0/1299262219 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f616c0b03b0 0x7f616c0b5e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:37.639 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:37.634+0000 7f6170b20700 1 -- 172.21.15.59:0/1299262219 >> v2:172.21.15.76:3300/0 conn(0x7f616c0adda0 msgr2=0x7f616c06b970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T07:57:37.639 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:37.634+0000 7f6170b20700 1 --2- 172.21.15.59:0/1299262219 >> v2:172.21.15.76:3300/0 conn(0x7f616c0adda0 0x7f616c06b970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:37.639 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:37.634+0000 7f6170b20700 1 -- 172.21.15.59:0/1299262219 --> v1:172.21.15.59:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f616c0b0900 con 0x7f616c15ae00 2023-11-24T07:57:37.639 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:37.634+0000 7f6170b20700 1 -- 172.21.15.59:0/1299262219 <== mon.0 v1:172.21.15.59:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1312333566 0 0) 0x7f615c001670 con 0x7f616c15ae00 2023-11-24T07:57:37.639 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:37.634+0000 7f6170b20700 1 -- 172.21.15.59:0/1299262219 >> v1:172.21.15.59:6789/0 conn(0x7f616c15ae00 legacy=0x7f616c0ad690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:57:37.639 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:37.634+0000 7f6170b20700 1 --2- 172.21.15.59:0/1299262219 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f616c0b03b0 0x7f6158003a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:57:37.639 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:37.634+0000 7f6170b20700 1 --2- 172.21.15.59:0/1299262219 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6158003f80 0x7f6158004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:57:37.640 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:37.634+0000 7f6170b20700 1 --2- 172.21.15.59:0/1299262219 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6158004d90 0x7f6158005210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:57:37.640 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:37.634+0000 7f6170b20700 1 -- 172.21.15.59:0/1299262219 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f616c073610 con 0x7f6158003f80 2023-11-24T07:57:37.640 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:37.634+0000 7f6170b20700 1 -- 172.21.15.59:0/1299262219 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f616c15b1e0 con 0x7f6158004d90 2023-11-24T07:57:37.640 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:37.634+0000 7f6170b20700 1 -- 172.21.15.59:0/1299262219 --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f615800ba10 con 0x7f616c0b03b0 2023-11-24T07:57:37.640 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:37.634+0000 7f6171321700 1 --2- 172.21.15.59:0/1299262219 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6158003f80 0x7f6158004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:57:37.640 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:37.634+0000 7f6171b22700 1 --2- 172.21.15.59:0/1299262219 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f616c0b03b0 0x7f6158003a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:57:37.640 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:37.634+0000 7f6172323700 1 --2- 172.21.15.59:0/1299262219 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6158004d90 0x7f6158005210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:57:37.640 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:37.634+0000 7f6171321700 1 -- 172.21.15.59:0/1299262219 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f616c0b03b0 msgr2=0x7f6158003a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T07:57:37.640 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:37.634+0000 7f6171321700 1 --2- 172.21.15.59:0/1299262219 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f616c0b03b0 0x7f6158003a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:37.640 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:37.634+0000 7f6171321700 1 -- 172.21.15.59:0/1299262219 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6158004d90 msgr2=0x7f6158005210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T07:57:37.641 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:37.634+0000 7f6171321700 1 --2- 172.21.15.59:0/1299262219 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6158004d90 0x7f6158005210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:37.641 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:37.635+0000 7f6171321700 1 -- 172.21.15.59:0/1299262219 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f61580035b0 con 0x7f6158003f80 2023-11-24T07:57:37.641 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:37.635+0000 7f6171321700 1 --2- 172.21.15.59:0/1299262219 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6158003f80 0x7f6158004850 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f6168009b30 tx=0x7f6158005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6a717388d2ebf462 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:57:37.641 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:37.635+0000 7f6170b20700 1 -- 172.21.15.59:0/1299262219 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f616800e660 con 0x7f6158003f80 2023-11-24T07:57:37.641 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:37.635+0000 7f6170b20700 1 -- 172.21.15.59:0/1299262219 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f616800e7c0 con 0x7f6158003f80 2023-11-24T07:57:37.641 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:37.635+0000 7f6170b20700 1 -- 172.21.15.59:0/1299262219 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f616800eb30 con 0x7f6158003f80 2023-11-24T07:57:37.641 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:37.635+0000 7f6173d86700 1 -- 172.21.15.59:0/1299262219 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6158003f80 msgr2=0x7f6158004850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:57:37.641 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:37.635+0000 7f6173d86700 1 --2- 172.21.15.59:0/1299262219 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6158003f80 0x7f6158004850 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f6168009b30 tx=0x7f6158005880 comp rx=0 tx=0).stop 2023-11-24T07:57:37.641 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:37.635+0000 7f6172323700 1 -- 172.21.15.59:0/1299262219 reap_dead start 2023-11-24T07:57:37.641 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:37.635+0000 7f6173d86700 1 -- 172.21.15.59:0/1299262219 shutdown_connections 2023-11-24T07:57:37.642 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:37.635+0000 7f6173d86700 1 -- 172.21.15.59:0/1299262219 >> 172.21.15.59:0/1299262219 conn(0x7f616c0b5070 msgr2=0x7f616c0b5730 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:57:37.642 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:37.635+0000 7f6173d86700 1 -- 172.21.15.59:0/1299262219 shutdown_connections 2023-11-24T07:57:37.642 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:37.635+0000 7f6173d86700 1 -- 172.21.15.59:0/1299262219 wait complete. 2023-11-24T07:57:37.642 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:37.637+0000 7f6173d86700 1 Processor -- start 2023-11-24T07:57:37.642 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:37.637+0000 7f6173d86700 1 -- start start 2023-11-24T07:57:37.645 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:37.637+0000 7f6173d86700 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f6158003f80 0x7f616c1539d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:57:37.645 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:37.637+0000 7f6173d86700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f616c150d90 0x7f616c151210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:57:37.645 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:37.637+0000 7f6173d86700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f616c158e50 0x7f616c159300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:57:37.645 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:37.637+0000 7f6173d86700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f616c151750 con 0x7f616c150d90 2023-11-24T07:57:37.645 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:37.637+0000 7f6173d86700 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f616c159840 con 0x7f616c158e50 2023-11-24T07:57:37.645 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:37.637+0000 7f6173d86700 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f616c1f42f0 con 0x7f6158003f80 2023-11-24T07:57:37.645 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:37.637+0000 7f6171321700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f616c150d90 0x7f616c151210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:57:37.646 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:37.638+0000 7f6171321700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f616c150d90 0x7f616c151210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:51942/0 (socket says 172.21.15.59:51942) 2023-11-24T07:57:37.646 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:37.638+0000 7f6171321700 1 -- 172.21.15.59:0/3064026699 learned_addr learned my addr 172.21.15.59:0/3064026699 (peer_addr_for_me v2:172.21.15.59:0/0) 2023-11-24T07:57:37.646 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:37.638+0000 7f6171b22700 1 --2- 172.21.15.59:0/3064026699 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f6158003f80 0x7f616c1539d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:57:37.646 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:37.638+0000 7f6172323700 1 --2- 172.21.15.59:0/3064026699 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f616c158e50 0x7f616c159300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:57:37.646 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:37.638+0000 7f6171321700 1 -- 172.21.15.59:0/3064026699 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f6158003f80 msgr2=0x7f616c1539d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:57:37.646 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:37.638+0000 7f6171321700 1 --2- 172.21.15.59:0/3064026699 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f6158003f80 0x7f616c1539d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:37.646 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:37.638+0000 7f6171321700 1 -- 172.21.15.59:0/3064026699 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f616c158e50 msgr2=0x7f616c159300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:57:37.646 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:37.638+0000 7f6171321700 1 --2- 172.21.15.59:0/3064026699 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f616c158e50 0x7f616c159300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:37.646 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:37.638+0000 7f6171321700 1 -- 172.21.15.59:0/3064026699 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f616c159af0 con 0x7f616c150d90 2023-11-24T07:57:37.646 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:37.638+0000 7f6172323700 1 --2- 172.21.15.59:0/3064026699 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f616c158e50 0x7f616c159300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-24T07:57:37.646 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:37.638+0000 7f6171321700 1 --2- 172.21.15.59:0/3064026699 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f616c150d90 0x7f616c151210 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f61580046f0 tx=0x7f61680047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:57:37.646 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:37.638+0000 7f61627fc700 1 -- 172.21.15.59:0/3064026699 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f616800e660 con 0x7f616c150d90 2023-11-24T07:57:37.647 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:37.638+0000 7f61627fc700 1 -- 172.21.15.59:0/3064026699 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6168013d80 con 0x7f616c150d90 2023-11-24T07:57:37.647 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:37.638+0000 7f61627fc700 1 -- 172.21.15.59:0/3064026699 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f616801c690 con 0x7f616c150d90 2023-11-24T07:57:37.647 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:37.638+0000 7f6173d86700 1 -- 172.21.15.59:0/3064026699 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f616c1f45b0 con 0x7f616c150d90 2023-11-24T07:57:37.647 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:37.638+0000 7f6173d86700 1 -- 172.21.15.59:0/3064026699 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f616c1f4b70 con 0x7f616c150d90 2023-11-24T07:57:37.647 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:37.639+0000 7f6173d86700 1 -- 172.21.15.59:0/3064026699 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f616c1f4ee0 con 0x7f616c150d90 2023-11-24T07:57:37.647 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:37.640+0000 7f61627fc700 1 -- 172.21.15.59:0/3064026699 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f616801c7f0 con 0x7f616c150d90 2023-11-24T07:57:37.647 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:37.640+0000 7f61627fc700 1 --2- 172.21.15.59:0/3064026699 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f615006aec0 0x7f615006d380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:57:37.647 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:37.640+0000 7f61627fc700 1 -- 172.21.15.59:0/3064026699 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f616808abc0 con 0x7f616c150d90 2023-11-24T07:57:37.647 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:37.640+0000 7f6171b22700 1 --2- 172.21.15.59:0/3064026699 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f615006aec0 0x7f615006d380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:57:37.647 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:37.641+0000 7f6171b22700 1 --2- 172.21.15.59:0/3064026699 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f615006aec0 0x7f615006d380 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f615c0017a0 tx=0x7f615c0014b0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:57:37.647 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:37.645+0000 7f61627fc700 1 -- 172.21.15.59:0/3064026699 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f6168055e20 con 0x7f616c150d90 2023-11-24T07:57:37.894 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:37.893+0000 7f6173d86700 1 -- 172.21.15.59:0/3064026699 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f616c0b5e40 con 0x7f616c150d90 2023-11-24T07:57:37.894 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:37.893+0000 7f61627fc700 1 -- 172.21.15.59:0/3064026699 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v16) v1 ==== 53+0+73 (secure 0 0 0) 0x7f6168059440 con 0x7f616c150d90 2023-11-24T07:57:37.894 INFO:teuthology.orchestra.run.smithi059.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-11-24T07:57:37.896 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:37.895+0000 7f6173d86700 1 -- 172.21.15.59:0/3064026699 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f615006aec0 msgr2=0x7f615006d380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:57:37.897 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:37.895+0000 7f6173d86700 1 --2- 172.21.15.59:0/3064026699 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f615006aec0 0x7f615006d380 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f615c0017a0 tx=0x7f615c0014b0 comp rx=0 tx=0).stop 2023-11-24T07:57:37.897 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:37.895+0000 7f6173d86700 1 -- 172.21.15.59:0/3064026699 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f616c150d90 msgr2=0x7f616c151210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:57:37.897 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:37.895+0000 7f6173d86700 1 --2- 172.21.15.59:0/3064026699 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f616c150d90 0x7f616c151210 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f61580046f0 tx=0x7f61680047c0 comp rx=0 tx=0).stop 2023-11-24T07:57:37.897 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:37.896+0000 7f6173d86700 1 -- 172.21.15.59:0/3064026699 shutdown_connections 2023-11-24T07:57:37.897 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:37.896+0000 7f6173d86700 1 --2- 172.21.15.59:0/3064026699 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f615006aec0 0x7f615006d380 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:37.897 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:37.896+0000 7f6173d86700 1 --2- 172.21.15.59:0/3064026699 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f616c158e50 0x7f616c159300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:37.897 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:37.896+0000 7f6173d86700 1 --2- 172.21.15.59:0/3064026699 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f6158003f80 0x7f616c1539d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:37.898 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:37.896+0000 7f6173d86700 1 --2- 172.21.15.59:0/3064026699 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f616c150d90 0x7f616c151210 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:37.898 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:37.896+0000 7f6173d86700 1 -- 172.21.15.59:0/3064026699 >> 172.21.15.59:0/3064026699 conn(0x7f616c0b5070 msgr2=0x7f616c0b1c80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:57:37.898 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:37.896+0000 7f6173d86700 1 -- 172.21.15.59:0/3064026699 shutdown_connections 2023-11-24T07:57:37.898 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:37.896+0000 7f6173d86700 1 -- 172.21.15.59:0/3064026699 wait complete. 2023-11-24T07:57:37.908 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2023-11-24T07:57:37.908 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-11-24T07:57:37.909 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi076.front.sepia.ceph.com 2023-11-24T07:57:37.909 INFO:tasks.cephfs.mount:self.client.name = client.1 2023-11-24T07:57:37.909 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2023-11-24T07:57:37.909 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2023-11-24T07:57:37.909 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2023-11-24T07:57:37.909 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-11-24T07:57:37.909 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2023-11-24T07:57:37.910 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T07:57:37.910 DEBUG:teuthology.orchestra.run.smithi076:> ip addr 2023-11-24T07:57:37.931 INFO:teuthology.orchestra.run.smithi076.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-11-24T07:57:37.931 INFO:teuthology.orchestra.run.smithi076.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-11-24T07:57:37.931 INFO:teuthology.orchestra.run.smithi076.stdout: inet 127.0.0.1/8 scope host lo 2023-11-24T07:57:37.931 INFO:teuthology.orchestra.run.smithi076.stdout: valid_lft forever preferred_lft forever 2023-11-24T07:57:37.932 INFO:teuthology.orchestra.run.smithi076.stdout: inet6 ::1/128 scope host 2023-11-24T07:57:37.932 INFO:teuthology.orchestra.run.smithi076.stdout: valid_lft forever preferred_lft forever 2023-11-24T07:57:37.932 INFO:teuthology.orchestra.run.smithi076.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-11-24T07:57:37.932 INFO:teuthology.orchestra.run.smithi076.stdout: link/ether 0c:c4:7a:d8:da:66 brd ff:ff:ff:ff:ff:ff 2023-11-24T07:57:37.932 INFO:teuthology.orchestra.run.smithi076.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-11-24T07:57:37.932 INFO:teuthology.orchestra.run.smithi076.stdout: link/ether 0c:c4:7a:d8:da:67 brd ff:ff:ff:ff:ff:ff 2023-11-24T07:57:37.932 INFO:teuthology.orchestra.run.smithi076.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-11-24T07:57:37.933 INFO:teuthology.orchestra.run.smithi076.stdout: link/ether 0c:c4:7a:8f:47:e0 brd ff:ff:ff:ff:ff:ff 2023-11-24T07:57:37.933 INFO:teuthology.orchestra.run.smithi076.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-11-24T07:57:37.933 INFO:teuthology.orchestra.run.smithi076.stdout: link/ether 0c:c4:7a:8f:47:e1 brd ff:ff:ff:ff:ff:ff 2023-11-24T07:57:37.933 INFO:teuthology.orchestra.run.smithi076.stdout: inet 172.21.15.76/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2023-11-24T07:57:37.933 INFO:teuthology.orchestra.run.smithi076.stdout: valid_lft 42341sec preferred_lft 42341sec 2023-11-24T07:57:37.933 INFO:teuthology.orchestra.run.smithi076.stdout: inet6 fe80::a34d:4ce5:8360:8f9/64 scope link noprefixroute 2023-11-24T07:57:37.933 INFO:teuthology.orchestra.run.smithi076.stdout: valid_lft forever preferred_lft forever 2023-11-24T07:57:37.933 INFO:teuthology.orchestra.run.smithi076.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-11-24T07:57:37.934 INFO:teuthology.orchestra.run.smithi076.stdout: link/ether 2a:59:73:ff:f7:95 brd ff:ff:ff:ff:ff:ff 2023-11-24T07:57:37.934 INFO:teuthology.orchestra.run.smithi076.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-11-24T07:57:37.934 INFO:teuthology.orchestra.run.smithi076.stdout: valid_lft forever preferred_lft forever 2023-11-24T07:57:37.934 INFO:teuthology.orchestra.run.smithi076.stdout: inet6 fe80::4c23:44ff:feb9:d9d4/64 scope link 2023-11-24T07:57:37.934 INFO:teuthology.orchestra.run.smithi076.stdout: valid_lft forever preferred_lft forever 2023-11-24T07:57:37.934 INFO:teuthology.orchestra.run.smithi076.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-11-24T07:57:37.934 INFO:teuthology.orchestra.run.smithi076.stdout: link/ether 46:3e:09:af:65:64 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2023-11-24T07:57:37.934 INFO:teuthology.orchestra.run.smithi076.stdout: inet6 fe80::443e:9ff:feaf:6564/64 scope link 2023-11-24T07:57:37.934 INFO:teuthology.orchestra.run.smithi076.stdout: valid_lft forever preferred_lft forever 2023-11-24T07:57:37.934 INFO:teuthology.orchestra.run.smithi076.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-11-24T07:57:37.934 INFO:teuthology.orchestra.run.smithi076.stdout: link/ether 2a:59:73:ff:f7:95 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.3 2023-11-24T07:57:37.935 INFO:teuthology.orchestra.run.smithi076.stdout: inet6 fe80::2859:73ff:feff:f795/64 scope link 2023-11-24T07:57:37.935 INFO:teuthology.orchestra.run.smithi076.stdout: valid_lft forever preferred_lft forever 2023-11-24T07:57:37.935 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T07:57:37.935 DEBUG:teuthology.orchestra.run.smithi076:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-11-24T07:57:38.002 INFO:teuthology.orchestra.run.smithi076.stdout:1 2023-11-24T07:57:38.004 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T07:57:38.004 DEBUG:teuthology.orchestra.run.smithi076:> route 2023-11-24T07:57:38.066 INFO:teuthology.orchestra.run.smithi076.stdout:Kernel IP routing table 2023-11-24T07:57:38.066 INFO:teuthology.orchestra.run.smithi076.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-11-24T07:57:38.066 INFO:teuthology.orchestra.run.smithi076.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-11-24T07:57:38.066 INFO:teuthology.orchestra.run.smithi076.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-11-24T07:57:38.066 INFO:teuthology.orchestra.run.smithi076.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-11-24T07:57:38.067 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T07:57:38.067 DEBUG:teuthology.orchestra.run.smithi076:> (cd / && exec bash -c ' 2023-11-24T07:57:38.067 DEBUG:teuthology.orchestra.run.smithi076:> set -e 2023-11-24T07:57:38.067 DEBUG:teuthology.orchestra.run.smithi076:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-11-24T07:57:38.067 DEBUG:teuthology.orchestra.run.smithi076:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-11-24T07:57:38.067 DEBUG:teuthology.orchestra.run.smithi076:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-11-24T07:57:38.068 DEBUG:teuthology.orchestra.run.smithi076:> ') 2023-11-24T07:57:38.183 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T07:57:38.183 DEBUG:teuthology.orchestra.run.smithi076:> ip netns list 2023-11-24T07:57:38.201 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-ns--home-ubuntu-cephtest-mnt.3 (id: 1) 2023-11-24T07:57:38.201 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2023-11-24T07:57:38.202 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2023-11-24T07:57:38.202 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T07:57:38.202 DEBUG:teuthology.orchestra.run.smithi076:> sudo ip link set brx.0 up 2023-11-24T07:57:38.270 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-11-24T07:57:38.270 DEBUG:teuthology.orchestra.run.smithi076:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2023-11-24T07:57:38.327 INFO:teuthology.orchestra.run.smithi076.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2023-11-24T07:57:38.327 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-11-24T07:57:38.327 DEBUG:teuthology.orchestra.run.smithi076:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2023-11-24T07:57:38.385 INFO:tasks.cephfs.kernel_mount:mounting using device: 1@.cephfs=/ 2023-11-24T07:57:38.385 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T07:57:38.386 DEBUG:teuthology.orchestra.run.smithi076:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 1@.cephfs=/ /home/ubuntu/cephtest/mnt.1 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-11-24T07:57:38.472 INFO:teuthology.orchestra.run.smithi076.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-11-24T07:57:38.472 INFO:teuthology.orchestra.run.smithi076.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-11-24T07:57:38.495 INFO:teuthology.orchestra.run.smithi076.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-11-24T07:57:38.496 INFO:teuthology.orchestra.run.smithi076.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-11-24T07:57:38.496 INFO:teuthology.orchestra.run.smithi076.stdout:mount.ceph: resolved to: "172.21.15.59:3300,172.21.15.76:3300,172.21.15.76:3301" 2023-11-24T07:57:38.496 INFO:teuthology.orchestra.run.smithi076.stdout:mount.ceph: trying mount with new device syntax: 1@9f0e8741-b8f9-48d5-a2c3-09f45903303d.cephfs=/ 2023-11-24T07:57:38.496 INFO:teuthology.orchestra.run.smithi076.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,ms_mode=prefer-crc,key=1,mon_addr=172.21.15.59:3300/172.21.15.76:3300/172.21.15.76:3301" will pass to kernel 2023-11-24T07:57:38.496 INFO:teuthology.orchestra.run.smithi076.stdout:mount.ceph: v2 mount support check returned 2 2023-11-24T07:57:38.496 INFO:teuthology.orchestra.run.smithi076.stdout:mount.ceph: kernel does not support v2 syntax 2023-11-24T07:57:38.496 INFO:teuthology.orchestra.run.smithi076.stdout:mount.ceph: switching to using v1 address with old syntax 2023-11-24T07:57:38.523 INFO:teuthology.orchestra.run.smithi076.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-11-24T07:57:38.523 INFO:teuthology.orchestra.run.smithi076.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-11-24T07:57:38.523 INFO:teuthology.orchestra.run.smithi076.stdout:mount.ceph: resolved to: "172.21.15.59:3300,172.21.15.76:3300,172.21.15.76:3301" 2023-11-24T07:57:38.523 INFO:teuthology.orchestra.run.smithi076.stdout:mount.ceph: trying mount with new device syntax: 1@9f0e8741-b8f9-48d5-a2c3-09f45903303d.cephfs=/ 2023-11-24T07:57:38.523 INFO:teuthology.orchestra.run.smithi076.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,ms_mode=prefer-crc,key=1,mon_addr=172.21.15.59:3300/172.21.15.76:3300/172.21.15.76:3301" will pass to kernel 2023-11-24T07:57:38.523 INFO:teuthology.orchestra.run.smithi076.stdout:mount.ceph: v2 mount support check returned 2 2023-11-24T07:57:38.523 INFO:teuthology.orchestra.run.smithi076.stdout:mount.ceph: kernel does not support v2 syntax 2023-11-24T07:57:38.523 INFO:teuthology.orchestra.run.smithi076.stdout:mount.ceph: switching to using v1 address with old syntax 2023-11-24T07:57:38.524 INFO:teuthology.orchestra.run.smithi076.stdout:mount.ceph: resolved to: "172.21.15.59:6789,172.21.15.76:6789,172.21.15.76:6790" 2023-11-24T07:57:38.524 INFO:teuthology.orchestra.run.smithi076.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.59:6789,172.21.15.76:6789,172.21.15.76:6790:/ 2023-11-24T07:57:38.524 INFO:teuthology.orchestra.run.smithi076.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,key=1,mds_namespace=cephfs,fsid=9f0e8741-b8f9-48d5-a2c3-09f45903303d" will pass to kernel 2023-11-24T07:57:38.526 INFO:tasks.cephfs.kernel_mount:mount command passed 2023-11-24T07:57:38.526 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T07:57:38.526 DEBUG:teuthology.orchestra.run.smithi076:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2023-11-24T07:57:38.556 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T07:57:38.556 DEBUG:teuthology.orchestra.run.smithi076:> (cd /home/ubuntu/cephtest/mnt.1 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-11-24T07:57:38.629 INFO:teuthology.orchestra.run.smithi076.stdout:client5349 2023-11-24T07:57:38.629 DEBUG:teuthology.orchestra.run.smithi076:> set -ex 2023-11-24T07:57:38.629 DEBUG:teuthology.orchestra.run.smithi076:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-24T07:57:38.687 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs flag set enable_multiple true --yes-i-really-mean-it 2023-11-24T07:57:38.831 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:38.830+0000 7f5ad2d71700 1 Processor -- start 2023-11-24T07:57:38.832 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:38.830+0000 7f5ad2d71700 1 -- start start 2023-11-24T07:57:38.839 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:38.831+0000 7f5ad2d71700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f5acc0b11d0 0x7f5acc0cfd60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:57:38.839 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:38.831+0000 7f5ad2d71700 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f5acc06fcb0 0x7f5acc0ce5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:57:38.839 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:38.831+0000 7f5ad2d71700 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5acc06f110 con 0x7f5acc0ceaf0 2023-11-24T07:57:38.839 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:38.831+0000 7f5ad2d71700 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5acc0cef60 con 0x7f5acc0b11d0 2023-11-24T07:57:38.840 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:38.831+0000 7f5ad2d71700 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5acc161a10 con 0x7f5acc06fcb0 2023-11-24T07:57:38.840 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:38.831+0000 7f5ad0b0d700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f5acc0b11d0 0x7f5acc0cfd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:57:38.840 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:38.831+0000 7f5ad0b0d700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f5acc0b11d0 0x7f5acc0cfd60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:51948/0 (socket says 172.21.15.59:51948) 2023-11-24T07:57:38.840 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:38.831+0000 7f5ad0b0d700 1 -- 172.21.15.59:0/1090882163 learned_addr learned my addr 172.21.15.59:0/1090882163 (peer_addr_for_me v2:172.21.15.59:0/0) 2023-11-24T07:57:38.840 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:38.831+0000 7f5acbfff700 1 --2- 172.21.15.59:0/1090882163 >> v2:172.21.15.76:3300/0 conn(0x7f5acc06fcb0 0x7f5acc0ce5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:57:38.841 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:38.832+0000 7f5ad0b0d700 1 -- 172.21.15.59:0/1090882163 >> v2:172.21.15.76:3300/0 conn(0x7f5acc06fcb0 msgr2=0x7f5acc0ce5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T07:57:38.841 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:38.832+0000 7f5ad0b0d700 1 --2- 172.21.15.59:0/1090882163 >> v2:172.21.15.76:3300/0 conn(0x7f5acc06fcb0 0x7f5acc0ce5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:38.841 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:38.832+0000 7f5ad0b0d700 1 -- 172.21.15.59:0/1090882163 >> v1:172.21.15.76:6789/0 conn(0x7f5acc0ceaf0 legacy=0x7f5acc06c1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:57:38.841 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:38.832+0000 7f5ad0b0d700 1 -- 172.21.15.59:0/1090882163 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5acc0d1010 con 0x7f5acc0b11d0 2023-11-24T07:57:38.841 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:38.832+0000 7f5ad0b0d700 1 --2- 172.21.15.59:0/1090882163 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5acc0b11d0 0x7f5acc0cfd60 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f5acc0c56b0 tx=0x7f5ac0009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e6f2725bd29d9ffa server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:57:38.842 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:38.832+0000 7f5acb7fe700 1 -- 172.21.15.59:0/1090882163 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ac0011830 con 0x7f5acc0b11d0 2023-11-24T07:57:38.842 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:38.832+0000 7f5acb7fe700 1 -- 172.21.15.59:0/1090882163 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5ac0011990 con 0x7f5acc0b11d0 2023-11-24T07:57:38.842 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:38.832+0000 7f5acb7fe700 1 -- 172.21.15.59:0/1090882163 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ac0011d20 con 0x7f5acc0b11d0 2023-11-24T07:57:38.842 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:38.833+0000 7f5ad2d71700 1 -- 172.21.15.59:0/1090882163 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5acc0b11d0 msgr2=0x7f5acc0cfd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:57:38.842 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:38.833+0000 7f5ad2d71700 1 --2- 172.21.15.59:0/1090882163 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5acc0b11d0 0x7f5acc0cfd60 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f5acc0c56b0 tx=0x7f5ac0009bd0 comp rx=0 tx=0).stop 2023-11-24T07:57:38.842 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:38.833+0000 7f5ad2d71700 1 -- 172.21.15.59:0/1090882163 shutdown_connections 2023-11-24T07:57:38.842 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:38.833+0000 7f5ad2d71700 1 --2- 172.21.15.59:0/1090882163 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5acc0b11d0 0x7f5acc0cfd60 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:38.843 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:38.833+0000 7f5ad2d71700 1 --2- 172.21.15.59:0/1090882163 >> v2:172.21.15.76:3300/0 conn(0x7f5acc06fcb0 0x7f5acc0ce5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:38.843 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:38.833+0000 7f5ad2d71700 1 -- 172.21.15.59:0/1090882163 >> 172.21.15.59:0/1090882163 conn(0x7f5acc0b3700 msgr2=0x7f5acc0b3b10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:57:38.843 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:38.833+0000 7f5ad2d71700 1 -- 172.21.15.59:0/1090882163 shutdown_connections 2023-11-24T07:57:38.843 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:38.833+0000 7f5ad2d71700 1 -- 172.21.15.59:0/1090882163 wait complete. 2023-11-24T07:57:38.843 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:38.834+0000 7f5ad2d71700 1 Processor -- start 2023-11-24T07:57:38.843 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:38.834+0000 7f5ad2d71700 1 -- start start 2023-11-24T07:57:38.843 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:38.835+0000 7f5ad2d71700 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f5acc06fcb0 0x7f5acc0c8000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:57:38.843 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:38.835+0000 7f5ad2d71700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5acc0b11d0 0x7f5acc0b6880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:57:38.843 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:38.835+0000 7f5ad2d71700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5acc0ceaf0 0x7f5acc14a040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:57:38.844 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:38.835+0000 7f5ad2d71700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5acc0b60c0 con 0x7f5acc0ceaf0 2023-11-24T07:57:38.844 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:38.835+0000 7f5ad2d71700 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5acc15b600 con 0x7f5acc0b11d0 2023-11-24T07:57:38.844 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:38.835+0000 7f5ad2d71700 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5acc15b8e0 con 0x7f5acc06fcb0 2023-11-24T07:57:38.844 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:38.835+0000 7f5ad130e700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5acc0ceaf0 0x7f5acc14a040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:57:38.844 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:38.835+0000 7f5ad130e700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5acc0ceaf0 0x7f5acc14a040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:51958/0 (socket says 172.21.15.59:51958) 2023-11-24T07:57:38.844 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:38.835+0000 7f5ad130e700 1 -- 172.21.15.59:0/918179325 learned_addr learned my addr 172.21.15.59:0/918179325 (peer_addr_for_me v2:172.21.15.59:0/0) 2023-11-24T07:57:38.844 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:38.835+0000 7f5ad0b0d700 1 --2- 172.21.15.59:0/918179325 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f5acc06fcb0 0x7f5acc0c8000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:57:38.844 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:38.835+0000 7f5acbfff700 1 --2- 172.21.15.59:0/918179325 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5acc0b11d0 0x7f5acc0b6880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:57:38.844 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:38.835+0000 7f5ad130e700 1 -- 172.21.15.59:0/918179325 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f5acc06fcb0 msgr2=0x7f5acc0c8000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:57:38.845 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:38.835+0000 7f5ad130e700 1 --2- 172.21.15.59:0/918179325 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f5acc06fcb0 0x7f5acc0c8000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:38.845 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:38.835+0000 7f5ad130e700 1 -- 172.21.15.59:0/918179325 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5acc0b11d0 msgr2=0x7f5acc0b6880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:57:38.845 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:38.835+0000 7f5ad130e700 1 --2- 172.21.15.59:0/918179325 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5acc0b11d0 0x7f5acc0b6880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:38.845 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:38.835+0000 7f5ad130e700 1 -- 172.21.15.59:0/918179325 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5ac0009550 con 0x7f5acc0ceaf0 2023-11-24T07:57:38.845 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:38.835+0000 7f5acbfff700 1 --2- 172.21.15.59:0/918179325 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5acc0b11d0 0x7f5acc0b6880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-24T07:57:38.845 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:38.836+0000 7f5ad130e700 1 --2- 172.21.15.59:0/918179325 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5acc0ceaf0 0x7f5acc14a040 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f5ac400cb50 tx=0x7f5ac400ce60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:57:38.845 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:38.836+0000 7f5ac97fa700 1 -- 172.21.15.59:0/918179325 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ac4003c00 con 0x7f5acc0ceaf0 2023-11-24T07:57:38.845 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:38.836+0000 7f5ac97fa700 1 -- 172.21.15.59:0/918179325 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5ac4003d60 con 0x7f5acc0ceaf0 2023-11-24T07:57:38.845 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:38.836+0000 7f5ad2d71700 1 -- 172.21.15.59:0/918179325 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5acc0b6360 con 0x7f5acc0ceaf0 2023-11-24T07:57:38.845 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:38.836+0000 7f5ac97fa700 1 -- 172.21.15.59:0/918179325 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ac400aac0 con 0x7f5acc0ceaf0 2023-11-24T07:57:38.845 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:38.836+0000 7f5ad2d71700 1 -- 172.21.15.59:0/918179325 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5acc15bbc0 con 0x7f5acc0ceaf0 2023-11-24T07:57:38.845 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:38.837+0000 7f5ad2d71700 1 -- 172.21.15.59:0/918179325 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5acc06bc40 con 0x7f5acc0ceaf0 2023-11-24T07:57:38.846 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:38.837+0000 7f5ac97fa700 1 -- 172.21.15.59:0/918179325 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f5ac400ac20 con 0x7f5acc0ceaf0 2023-11-24T07:57:38.846 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:38.838+0000 7f5ac97fa700 1 --2- 172.21.15.59:0/918179325 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f5ab006aed0 0x7f5ab006d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:57:38.846 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:38.838+0000 7f5ac97fa700 1 -- 172.21.15.59:0/918179325 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f5ac4089580 con 0x7f5acc0ceaf0 2023-11-24T07:57:38.846 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:38.838+0000 7f5ad0b0d700 1 --2- 172.21.15.59:0/918179325 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f5ab006aed0 0x7f5ab006d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:57:38.846 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:38.838+0000 7f5ad0b0d700 1 --2- 172.21.15.59:0/918179325 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f5ab006aed0 0x7f5ab006d390 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f5ac0009c10 tx=0x7f5ac0008030 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:57:38.846 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:38.842+0000 7f5ac97fa700 1 -- 172.21.15.59:0/918179325 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f5ac4054630 con 0x7f5acc0ceaf0 2023-11-24T07:57:39.090 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:39.090+0000 7f5ad2d71700 1 -- 172.21.15.59:0/918179325 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "fs flag set", "flag_name": "enable_multiple", "val": "true", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f5acc068670 con 0x7f5acc0ceaf0 2023-11-24T07:57:39.379 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:39.379+0000 7f5ac97fa700 1 -- 172.21.15.59:0/918179325 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "fs flag set", "flag_name": "enable_multiple", "val": "true", "yes_i_really_mean_it": true}]=0 v17) v1 ==== 136+0+0 (secure 0 0 0) 0x7f5ac4057c50 con 0x7f5acc0ceaf0 2023-11-24T07:57:39.383 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:39.381+0000 7f5ad2d71700 1 -- 172.21.15.59:0/918179325 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f5ab006aed0 msgr2=0x7f5ab006d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:57:39.383 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:39.381+0000 7f5ad2d71700 1 --2- 172.21.15.59:0/918179325 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f5ab006aed0 0x7f5ab006d390 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f5ac0009c10 tx=0x7f5ac0008030 comp rx=0 tx=0).stop 2023-11-24T07:57:39.383 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:39.381+0000 7f5ad2d71700 1 -- 172.21.15.59:0/918179325 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5acc0ceaf0 msgr2=0x7f5acc14a040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:57:39.383 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:39.381+0000 7f5ad2d71700 1 --2- 172.21.15.59:0/918179325 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5acc0ceaf0 0x7f5acc14a040 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f5ac400cb50 tx=0x7f5ac400ce60 comp rx=0 tx=0).stop 2023-11-24T07:57:39.383 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:39.382+0000 7f5ad2d71700 1 -- 172.21.15.59:0/918179325 shutdown_connections 2023-11-24T07:57:39.383 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:39.382+0000 7f5ad2d71700 1 --2- 172.21.15.59:0/918179325 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f5ab006aed0 0x7f5ab006d390 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:39.383 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:39.382+0000 7f5ad2d71700 1 --2- 172.21.15.59:0/918179325 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5acc0ceaf0 0x7f5acc14a040 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:39.383 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:39.382+0000 7f5ad2d71700 1 --2- 172.21.15.59:0/918179325 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f5acc06fcb0 0x7f5acc0c8000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:39.383 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:39.382+0000 7f5ad2d71700 1 --2- 172.21.15.59:0/918179325 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5acc0b11d0 0x7f5acc0b6880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:39.383 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:39.382+0000 7f5ad2d71700 1 -- 172.21.15.59:0/918179325 >> 172.21.15.59:0/918179325 conn(0x7f5acc0b3700 msgr2=0x7f5acc0d21f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:57:39.384 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:39.382+0000 7f5ad2d71700 1 -- 172.21.15.59:0/918179325 shutdown_connections 2023-11-24T07:57:39.384 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:39.382+0000 7f5ad2d71700 1 -- 172.21.15.59:0/918179325 wait complete. 2023-11-24T07:57:39.396 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume ls 2023-11-24T07:57:39.523 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:39.522+0000 7f805e574700 1 Processor -- start 2023-11-24T07:57:39.523 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:39.522+0000 7f805e574700 1 -- start start 2023-11-24T07:57:39.526 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:39.523+0000 7f805e574700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f80580722c0 0x7f80580b51b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:57:39.526 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:39.523+0000 7f805e574700 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f80581440f0 0x7f805806e510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:57:39.526 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:39.523+0000 7f805e574700 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8058143ec0 con 0x7f8058143ae0 2023-11-24T07:57:39.526 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:39.523+0000 7f805e574700 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f80580b40a0 con 0x7f80580722c0 2023-11-24T07:57:39.526 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:39.523+0000 7f805e574700 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f80580b3ae0 con 0x7f80581440f0 2023-11-24T07:57:39.526 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:39.523+0000 7f80577fe700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f80580722c0 0x7f80580b51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:57:39.527 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:39.523+0000 7f80577fe700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f80580722c0 0x7f80580b51b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:51964/0 (socket says 172.21.15.59:51964) 2023-11-24T07:57:39.527 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:39.523+0000 7f80577fe700 1 -- 172.21.15.59:0/3393990818 learned_addr learned my addr 172.21.15.59:0/3393990818 (peer_addr_for_me v2:172.21.15.59:0/0) 2023-11-24T07:57:39.527 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:39.523+0000 7f805cb11700 1 --2- 172.21.15.59:0/3393990818 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f80581440f0 0x7f805806e510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:57:39.527 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:39.523+0000 7f80577fe700 1 -- 172.21.15.59:0/3393990818 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f80581440f0 msgr2=0x7f805806e510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T07:57:39.527 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:39.523+0000 7f80577fe700 1 --2- 172.21.15.59:0/3393990818 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f80581440f0 0x7f805806e510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:39.527 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:39.523+0000 7f80577fe700 1 -- 172.21.15.59:0/3393990818 >> v1:172.21.15.76:6789/0 conn(0x7f8058143ae0 legacy=0x7f8058071bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:57:39.527 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:39.523+0000 7f80577fe700 1 -- 172.21.15.59:0/3393990818 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f80580b3dc0 con 0x7f80580722c0 2023-11-24T07:57:39.528 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:39.524+0000 7f80577fe700 1 --2- 172.21.15.59:0/3393990818 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f80580722c0 0x7f80580b51b0 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f80480098f0 tx=0x7f8048009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=77d93ed64ff5b0af server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:57:39.528 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:39.524+0000 7f8056ffd700 1 -- 172.21.15.59:0/3393990818 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8048011830 con 0x7f80580722c0 2023-11-24T07:57:39.528 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:39.524+0000 7f8056ffd700 1 -- 172.21.15.59:0/3393990818 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8048011990 con 0x7f80580722c0 2023-11-24T07:57:39.528 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:39.524+0000 7f8056ffd700 1 -- 172.21.15.59:0/3393990818 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8048011d20 con 0x7f80580722c0 2023-11-24T07:57:39.528 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:39.524+0000 7f805e574700 1 -- 172.21.15.59:0/3393990818 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f80580722c0 msgr2=0x7f80580b51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:57:39.529 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:39.524+0000 7f805e574700 1 --2- 172.21.15.59:0/3393990818 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f80580722c0 0x7f80580b51b0 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f80480098f0 tx=0x7f8048009c00 comp rx=0 tx=0).stop 2023-11-24T07:57:39.529 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:39.524+0000 7f805e574700 1 -- 172.21.15.59:0/3393990818 shutdown_connections 2023-11-24T07:57:39.529 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:39.524+0000 7f805e574700 1 --2- 172.21.15.59:0/3393990818 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f80581440f0 0x7f805806e510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:39.529 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:39.524+0000 7f805e574700 1 --2- 172.21.15.59:0/3393990818 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f80580722c0 0x7f80580b51b0 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:39.529 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:39.524+0000 7f805e574700 1 -- 172.21.15.59:0/3393990818 >> 172.21.15.59:0/3393990818 conn(0x7f80580b2740 msgr2=0x7f80580b2b20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:57:39.529 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:39.524+0000 7f805e574700 1 -- 172.21.15.59:0/3393990818 shutdown_connections 2023-11-24T07:57:39.529 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:39.524+0000 7f805e574700 1 -- 172.21.15.59:0/3393990818 wait complete. 2023-11-24T07:57:39.532 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:39.526+0000 7f805e574700 1 Processor -- start 2023-11-24T07:57:39.532 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:39.526+0000 7f805e574700 1 -- start start 2023-11-24T07:57:39.532 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:39.527+0000 7f805e574700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f80580722c0 0x7f80581efb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:57:39.532 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:39.527+0000 7f805e574700 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f8058143ae0 0x7f805813f9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:57:39.533 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:39.527+0000 7f805e574700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f80581440f0 0x7f805815de10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:57:39.533 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:39.527+0000 7f805e574700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8058142290 con 0x7f80580722c0 2023-11-24T07:57:39.533 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:39.527+0000 7f805e574700 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80581f0040 con 0x7f80581440f0 2023-11-24T07:57:39.533 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:39.527+0000 7f805e574700 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8058141680 con 0x7f8058143ae0 2023-11-24T07:57:39.533 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:39.527+0000 7f8057fff700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f80580722c0 0x7f80581efb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:57:39.533 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:39.527+0000 7f8057fff700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f80580722c0 0x7f80581efb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:51968/0 (socket says 172.21.15.59:51968) 2023-11-24T07:57:39.534 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:39.527+0000 7f8057fff700 1 -- 172.21.15.59:0/1158466699 learned_addr learned my addr 172.21.15.59:0/1158466699 (peer_addr_for_me v2:172.21.15.59:0/0) 2023-11-24T07:57:39.534 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:39.527+0000 7f80577fe700 1 --2- 172.21.15.59:0/1158466699 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f8058143ae0 0x7f805813f9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:57:39.534 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:39.527+0000 7f805cb11700 1 --2- 172.21.15.59:0/1158466699 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f80581440f0 0x7f805815de10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:57:39.534 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:39.527+0000 7f8057fff700 1 -- 172.21.15.59:0/1158466699 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f8058143ae0 msgr2=0x7f805813f9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:57:39.534 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:39.527+0000 7f8057fff700 1 --2- 172.21.15.59:0/1158466699 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f8058143ae0 0x7f805813f9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:39.534 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:39.527+0000 7f8057fff700 1 -- 172.21.15.59:0/1158466699 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f80581440f0 msgr2=0x7f805815de10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:57:39.534 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:39.527+0000 7f8057fff700 1 --2- 172.21.15.59:0/1158466699 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f80581440f0 0x7f805815de10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:39.535 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:39.527+0000 7f8057fff700 1 -- 172.21.15.59:0/1158466699 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8048009550 con 0x7f80580722c0 2023-11-24T07:57:39.535 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:39.528+0000 7f8057fff700 1 --2- 172.21.15.59:0/1158466699 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f80580722c0 0x7f80581efb00 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f804000c4e0 tx=0x7f804000c7f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:57:39.535 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:39.528+0000 7f8054ff9700 1 -- 172.21.15.59:0/1158466699 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80400052d0 con 0x7f80580722c0 2023-11-24T07:57:39.535 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:39.528+0000 7f8054ff9700 1 -- 172.21.15.59:0/1158466699 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8040005430 con 0x7f80580722c0 2023-11-24T07:57:39.535 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:39.528+0000 7f805e574700 1 -- 172.21.15.59:0/1158466699 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8058141960 con 0x7f80580722c0 2023-11-24T07:57:39.535 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:39.528+0000 7f8054ff9700 1 -- 172.21.15.59:0/1158466699 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80400038c0 con 0x7f80580722c0 2023-11-24T07:57:39.535 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:39.528+0000 7f805e574700 1 -- 172.21.15.59:0/1158466699 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8058141f20 con 0x7f80580722c0 2023-11-24T07:57:39.536 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:39.529+0000 7f805e574700 1 -- 172.21.15.59:0/1158466699 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f80580b35a0 con 0x7f80580722c0 2023-11-24T07:57:39.536 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:39.529+0000 7f8054ff9700 1 -- 172.21.15.59:0/1158466699 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f8040003a20 con 0x7f80580722c0 2023-11-24T07:57:39.536 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:39.530+0000 7f8054ff9700 1 --2- 172.21.15.59:0/1158466699 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f803806aed0 0x7f803806d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:57:39.536 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:39.530+0000 7f8054ff9700 1 -- 172.21.15.59:0/1158466699 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f8040088960 con 0x7f80580722c0 2023-11-24T07:57:39.536 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:39.530+0000 7f80577fe700 1 --2- 172.21.15.59:0/1158466699 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f803806aed0 0x7f803806d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:57:39.536 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:39.530+0000 7f80577fe700 1 --2- 172.21.15.59:0/1158466699 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f803806aed0 0x7f803806d390 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f8048009c40 tx=0x7f8048008030 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:57:39.537 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:39.534+0000 7f8054ff9700 1 -- 172.21.15.59:0/1158466699 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f8040053a10 con 0x7f80580722c0 2023-11-24T07:57:39.764 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:39.763+0000 7f805e574700 1 -- 172.21.15.59:0/1158466699 --> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] -- mgr_command(tid 0: {"prefix": "fs volume ls", "target": ["mon-mgr", ""]}) v1 -- 0x7f8058144860 con 0x7f803806aed0 2023-11-24T07:57:39.766 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:39.765+0000 7f8054ff9700 1 -- 172.21.15.59:0/1158466699 <== mgr.4100 v2:172.21.15.76:6832/4264046536 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+41 (secure 0 0 0) 0x7f8048011c00 con 0x7f803806aed0 2023-11-24T07:57:39.766 INFO:teuthology.orchestra.run.smithi059.stdout:[ 2023-11-24T07:57:39.766 INFO:teuthology.orchestra.run.smithi059.stdout: { 2023-11-24T07:57:39.766 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "cephfs" 2023-11-24T07:57:39.766 INFO:teuthology.orchestra.run.smithi059.stdout: } 2023-11-24T07:57:39.766 INFO:teuthology.orchestra.run.smithi059.stdout:] 2023-11-24T07:57:39.769 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:39.768+0000 7f805e574700 1 -- 172.21.15.59:0/1158466699 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f803806aed0 msgr2=0x7f803806d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:57:39.769 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:39.768+0000 7f805e574700 1 --2- 172.21.15.59:0/1158466699 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f803806aed0 0x7f803806d390 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f8048009c40 tx=0x7f8048008030 comp rx=0 tx=0).stop 2023-11-24T07:57:39.769 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:39.768+0000 7f805e574700 1 -- 172.21.15.59:0/1158466699 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f80580722c0 msgr2=0x7f80581efb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:57:39.769 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:39.768+0000 7f805e574700 1 --2- 172.21.15.59:0/1158466699 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f80580722c0 0x7f80581efb00 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f804000c4e0 tx=0x7f804000c7f0 comp rx=0 tx=0).stop 2023-11-24T07:57:39.769 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:39.768+0000 7f805e574700 1 -- 172.21.15.59:0/1158466699 shutdown_connections 2023-11-24T07:57:39.769 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:39.768+0000 7f805e574700 1 --2- 172.21.15.59:0/1158466699 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f803806aed0 0x7f803806d390 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:39.769 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:39.768+0000 7f805e574700 1 --2- 172.21.15.59:0/1158466699 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f80581440f0 0x7f805815de10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:39.769 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:39.768+0000 7f805e574700 1 --2- 172.21.15.59:0/1158466699 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f80580722c0 0x7f80581efb00 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:39.769 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:39.768+0000 7f805e574700 1 --2- 172.21.15.59:0/1158466699 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f8058143ae0 0x7f805813f9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:39.769 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:39.768+0000 7f805e574700 1 -- 172.21.15.59:0/1158466699 >> 172.21.15.59:0/1158466699 conn(0x7f80580b2740 msgr2=0x7f805806afd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:57:39.769 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:39.768+0000 7f805e574700 1 -- 172.21.15.59:0/1158466699 shutdown_connections 2023-11-24T07:57:39.769 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:39.768+0000 7f805e574700 1 -- 172.21.15.59:0/1158466699 wait complete. 2023-11-24T07:57:39.781 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2023-11-24T07:57:39.908 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:39.907+0000 7fa55cf76700 1 Processor -- start 2023-11-24T07:57:39.909 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:39.907+0000 7fa55cf76700 1 -- start start 2023-11-24T07:57:39.911 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:39.908+0000 7fa55cf76700 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fa558144c10 0x7fa55806afd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:57:39.911 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:39.908+0000 7fa55cf76700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fa558145100 0x7fa55806b510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:57:39.911 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:39.908+0000 7fa55cf76700 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa55806eb80 con 0x7fa558145650 2023-11-24T07:57:39.911 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:39.908+0000 7fa55cf76700 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa55806f500 con 0x7fa558145100 2023-11-24T07:57:39.911 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:39.908+0000 7fa55cf76700 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa5580cd610 con 0x7fa558144c10 2023-11-24T07:57:39.912 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:39.908+0000 7fa555d9b700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fa558145100 0x7fa55806b510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:57:39.912 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:39.908+0000 7fa556d9d700 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7fa558145650 0x7fa558071bb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:55454/0 (socket says 172.21.15.59:55454) 2023-11-24T07:57:39.912 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:39.908+0000 7fa556d9d700 1 -- 172.21.15.59:0/1598519551 learned_addr learned my addr 172.21.15.59:0/1598519551 (peer_addr_for_me v1:172.21.15.59:0/0) 2023-11-24T07:57:39.912 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:39.908+0000 7fa55659c700 1 --2- 172.21.15.59:0/1598519551 >> v2:172.21.15.76:3300/0 conn(0x7fa558144c10 0x7fa55806afd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:57:39.912 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:39.908+0000 7fa555d9b700 1 -- 172.21.15.59:0/1598519551 >> v2:172.21.15.76:3300/0 conn(0x7fa558144c10 msgr2=0x7fa55806afd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T07:57:39.912 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:39.908+0000 7fa555d9b700 1 --2- 172.21.15.59:0/1598519551 >> v2:172.21.15.76:3300/0 conn(0x7fa558144c10 0x7fa55806afd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:39.912 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:39.908+0000 7fa555d9b700 1 -- 172.21.15.59:0/1598519551 >> v1:172.21.15.59:6789/0 conn(0x7fa558145650 legacy=0x7fa558071bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:57:39.912 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:39.908+0000 7fa555d9b700 1 -- 172.21.15.59:0/1598519551 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa558160720 con 0x7fa558145100 2023-11-24T07:57:39.912 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:39.908+0000 7fa55559a700 1 -- 172.21.15.59:0/1598519551 <== mon.0 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2735614996 0 0) 0x7fa55806eb80 con 0x7fa558145650 2023-11-24T07:57:39.912 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:39.908+0000 7fa555d9b700 1 --2- 172.21.15.59:0/1598519551 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa558145100 0x7fa55806b510 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7fa5580706e0 tx=0x7fa54c009b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e4003743045afe60 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:57:39.912 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:39.909+0000 7fa55559a700 1 -- 172.21.15.59:0/1598519551 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa54c011830 con 0x7fa558145100 2023-11-24T07:57:39.912 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:39.909+0000 7fa55559a700 1 -- 172.21.15.59:0/1598519551 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa54c011990 con 0x7fa558145100 2023-11-24T07:57:39.913 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:39.909+0000 7fa55559a700 1 -- 172.21.15.59:0/1598519551 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa54c011d20 con 0x7fa558145100 2023-11-24T07:57:39.913 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:39.909+0000 7fa55cf76700 1 -- 172.21.15.59:0/1598519551 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa558145100 msgr2=0x7fa55806b510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:57:39.913 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:39.909+0000 7fa55cf76700 1 --2- 172.21.15.59:0/1598519551 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa558145100 0x7fa55806b510 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7fa5580706e0 tx=0x7fa54c009b20 comp rx=0 tx=0).stop 2023-11-24T07:57:39.913 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:39.909+0000 7fa55cf76700 1 -- 172.21.15.59:0/1598519551 shutdown_connections 2023-11-24T07:57:39.913 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:39.909+0000 7fa55cf76700 1 --2- 172.21.15.59:0/1598519551 >> v2:172.21.15.76:3300/0 conn(0x7fa558144c10 0x7fa55806afd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:39.913 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:39.909+0000 7fa55cf76700 1 --2- 172.21.15.59:0/1598519551 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa558145100 0x7fa55806b510 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:39.913 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:39.909+0000 7fa55cf76700 1 -- 172.21.15.59:0/1598519551 >> 172.21.15.59:0/1598519551 conn(0x7fa5580b2740 msgr2=0x7fa5580b2b20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:57:39.913 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:39.909+0000 7fa55cf76700 1 -- 172.21.15.59:0/1598519551 shutdown_connections 2023-11-24T07:57:39.913 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:39.909+0000 7fa55cf76700 1 -- 172.21.15.59:0/1598519551 wait complete. 2023-11-24T07:57:39.916 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:39.911+0000 7fa55cf76700 1 Processor -- start 2023-11-24T07:57:39.916 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:39.911+0000 7fa55cf76700 1 -- start start 2023-11-24T07:57:39.916 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:39.911+0000 7fa55cf76700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa558144c10 0x7fa558141bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:57:39.916 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:39.911+0000 7fa55cf76700 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fa558145100 0x7fa5580afe40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:57:39.916 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:39.911+0000 7fa55cf76700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa558145650 0x7fa558132b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:57:39.916 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:39.911+0000 7fa55cf76700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa558142f50 con 0x7fa558145650 2023-11-24T07:57:39.916 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:39.911+0000 7fa55cf76700 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5580b0380 con 0x7fa558144c10 2023-11-24T07:57:39.916 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:39.911+0000 7fa55cf76700 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa558133070 con 0x7fa558145100 2023-11-24T07:57:39.917 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:39.912+0000 7fa556d9d700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa558145650 0x7fa558132b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:57:39.917 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:39.912+0000 7fa556d9d700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa558145650 0x7fa558132b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:51986/0 (socket says 172.21.15.59:51986) 2023-11-24T07:57:39.917 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:39.912+0000 7fa556d9d700 1 -- 172.21.15.59:0/3988315672 learned_addr learned my addr 172.21.15.59:0/3988315672 (peer_addr_for_me v2:172.21.15.59:0/0) 2023-11-24T07:57:39.917 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:39.912+0000 7fa55659c700 1 --2- 172.21.15.59:0/3988315672 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa558144c10 0x7fa558141bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:57:39.917 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:39.912+0000 7fa555d9b700 1 --2- 172.21.15.59:0/3988315672 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fa558145100 0x7fa5580afe40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:57:39.917 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:39.912+0000 7fa556d9d700 1 -- 172.21.15.59:0/3988315672 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fa558145100 msgr2=0x7fa5580afe40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:57:39.917 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:39.912+0000 7fa556d9d700 1 --2- 172.21.15.59:0/3988315672 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fa558145100 0x7fa5580afe40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:39.917 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:39.912+0000 7fa556d9d700 1 -- 172.21.15.59:0/3988315672 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa558144c10 msgr2=0x7fa558141bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:57:39.917 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:39.912+0000 7fa556d9d700 1 --2- 172.21.15.59:0/3988315672 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa558144c10 0x7fa558141bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:39.918 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:39.912+0000 7fa556d9d700 1 -- 172.21.15.59:0/3988315672 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa54c0094a0 con 0x7fa558145650 2023-11-24T07:57:39.918 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:39.912+0000 7fa556d9d700 1 --2- 172.21.15.59:0/3988315672 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa558145650 0x7fa558132b30 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7fa55000cc10 tx=0x7fa5500021c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:57:39.918 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:39.912+0000 7fa546ffd700 1 -- 172.21.15.59:0/3988315672 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa55000ab80 con 0x7fa558145650 2023-11-24T07:57:39.918 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:39.912+0000 7fa546ffd700 1 -- 172.21.15.59:0/3988315672 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa55000ad00 con 0x7fa558145650 2023-11-24T07:57:39.918 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:39.912+0000 7fa55cf76700 1 -- 172.21.15.59:0/3988315672 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa5581422d0 con 0x7fa558145650 2023-11-24T07:57:39.918 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:39.912+0000 7fa546ffd700 1 -- 172.21.15.59:0/3988315672 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa550007310 con 0x7fa558145650 2023-11-24T07:57:39.918 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:39.912+0000 7fa55cf76700 1 -- 172.21.15.59:0/3988315672 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa558142890 con 0x7fa558145650 2023-11-24T07:57:39.918 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:39.913+0000 7fa55cf76700 1 -- 172.21.15.59:0/3988315672 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa5580cd350 con 0x7fa558145650 2023-11-24T07:57:39.919 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:39.914+0000 7fa546ffd700 1 -- 172.21.15.59:0/3988315672 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fa550007470 con 0x7fa558145650 2023-11-24T07:57:39.919 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:39.914+0000 7fa546ffd700 1 --2- 172.21.15.59:0/3988315672 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7fa53806aed0 0x7fa53806d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:57:39.919 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:39.914+0000 7fa546ffd700 1 -- 172.21.15.59:0/3988315672 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fa550089400 con 0x7fa558145650 2023-11-24T07:57:39.919 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:39.914+0000 7fa55659c700 1 --2- 172.21.15.59:0/3988315672 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7fa53806aed0 0x7fa53806d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:57:39.920 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:39.915+0000 7fa55659c700 1 --2- 172.21.15.59:0/3988315672 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7fa53806aed0 0x7fa53806d390 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fa548009ab0 tx=0x7fa548008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:57:39.920 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:39.919+0000 7fa546ffd700 1 -- 172.21.15.59:0/3988315672 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fa550054540 con 0x7fa558145650 2023-11-24T07:57:40.121 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:40.119+0000 7fa55cf76700 1 -- 172.21.15.59:0/3988315672 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7fa558075480 con 0x7fa558145650 2023-11-24T07:57:40.121 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:40.120+0000 7fa546ffd700 1 -- 172.21.15.59:0/3988315672 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v10)=0 v10) v1 ==== 122+0+0 (secure 0 0 0) 0x7fa550057b60 con 0x7fa558145650 2023-11-24T07:57:40.124 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:40.122+0000 7fa55cf76700 1 -- 172.21.15.59:0/3988315672 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7fa53806aed0 msgr2=0x7fa53806d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:57:40.124 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:40.122+0000 7fa55cf76700 1 --2- 172.21.15.59:0/3988315672 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7fa53806aed0 0x7fa53806d390 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fa548009ab0 tx=0x7fa548008040 comp rx=0 tx=0).stop 2023-11-24T07:57:40.124 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:40.122+0000 7fa55cf76700 1 -- 172.21.15.59:0/3988315672 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa558145650 msgr2=0x7fa558132b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:57:40.124 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:40.122+0000 7fa55cf76700 1 --2- 172.21.15.59:0/3988315672 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa558145650 0x7fa558132b30 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7fa55000cc10 tx=0x7fa5500021c0 comp rx=0 tx=0).stop 2023-11-24T07:57:40.124 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:40.122+0000 7fa55cf76700 1 -- 172.21.15.59:0/3988315672 shutdown_connections 2023-11-24T07:57:40.124 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:40.122+0000 7fa55cf76700 1 --2- 172.21.15.59:0/3988315672 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7fa53806aed0 0x7fa53806d390 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:40.124 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:40.122+0000 7fa55cf76700 1 --2- 172.21.15.59:0/3988315672 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa558145650 0x7fa558132b30 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:40.125 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:40.122+0000 7fa55cf76700 1 --2- 172.21.15.59:0/3988315672 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa558144c10 0x7fa558141bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:40.125 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:40.122+0000 7fa55cf76700 1 --2- 172.21.15.59:0/3988315672 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fa558145100 0x7fa5580afe40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:40.125 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:40.122+0000 7fa55cf76700 1 -- 172.21.15.59:0/3988315672 >> 172.21.15.59:0/3988315672 conn(0x7fa5580b2740 msgr2=0x7fa5580b06b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:57:40.125 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:40.122+0000 7fa55cf76700 1 -- 172.21.15.59:0/3988315672 shutdown_connections 2023-11-24T07:57:40.125 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:40.122+0000 7fa55cf76700 1 -- 172.21.15.59:0/3988315672 wait complete. 2023-11-24T07:57:40.136 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolumegroup create cephfs subvol_grp3824 2023-11-24T07:57:40.264 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:40.262+0000 7f2341318700 1 Processor -- start 2023-11-24T07:57:40.264 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:40.262+0000 7f2341318700 1 -- start start 2023-11-24T07:57:40.266 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:40.263+0000 7f2341318700 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f233c0d26c0 0x7f233c0b7430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:57:40.266 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:40.263+0000 7f2341318700 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f233c0b5370 con 0x7f233c0d0c50 2023-11-24T07:57:40.266 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:40.263+0000 7f2341318700 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f233c0ceae0 con 0x7f233c0d0870 2023-11-24T07:57:40.267 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:40.263+0000 7f2341318700 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f233c0d2f80 con 0x7f233c0d26c0 2023-11-24T07:57:40.267 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:40.263+0000 7f233a7fc700 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7f233c0d0c50 0x7f233c0b6d20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:55466/0 (socket says 172.21.15.59:55466) 2023-11-24T07:57:40.267 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:40.263+0000 7f233a7fc700 1 -- 172.21.15.59:0/304452665 learned_addr learned my addr 172.21.15.59:0/304452665 (peer_addr_for_me v1:172.21.15.59:0/0) 2023-11-24T07:57:40.267 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:40.263+0000 7f233b7fe700 1 --2- 172.21.15.59:0/304452665 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f233c0d26c0 0x7f233c0b7430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:57:40.267 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:40.263+0000 7f2339ffb700 1 -- 172.21.15.59:0/304452665 <== mon.1 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3975348018 0 0) 0x7f233c0ceae0 con 0x7f233c0d0870 2023-11-24T07:57:40.267 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:40.263+0000 7f2339ffb700 1 -- 172.21.15.59:0/304452665 --> v1:172.21.15.76:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f23240035b0 con 0x7f233c0d0870 2023-11-24T07:57:40.267 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:40.263+0000 7f2339ffb700 1 -- 172.21.15.59:0/304452665 <== mon.0 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2983181360 0 0) 0x7f233c0b5370 con 0x7f233c0d0c50 2023-11-24T07:57:40.267 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:40.263+0000 7f2339ffb700 1 -- 172.21.15.59:0/304452665 --> v1:172.21.15.59:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f233c0ceae0 con 0x7f233c0d0c50 2023-11-24T07:57:40.267 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:40.263+0000 7f233b7fe700 1 -- 172.21.15.59:0/304452665 >> v1:172.21.15.76:6789/0 conn(0x7f233c0d0870 legacy=0x7f233c0cf860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:57:40.267 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:40.263+0000 7f233b7fe700 1 -- 172.21.15.59:0/304452665 >> v1:172.21.15.59:6789/0 conn(0x7f233c0d0c50 legacy=0x7f233c0b6d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:57:40.267 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:40.263+0000 7f233b7fe700 1 -- 172.21.15.59:0/304452665 --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f233c0d2ca0 con 0x7f233c0d26c0 2023-11-24T07:57:40.268 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:40.263+0000 7f233b7fe700 1 --2- 172.21.15.59:0/304452665 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f233c0d26c0 0x7f233c0b7430 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f2334009dc0 tx=0x7f233400b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=acab89f53130fe20 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:57:40.268 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:40.264+0000 7f2339ffb700 1 -- 172.21.15.59:0/304452665 <== mon.2 v2:172.21.15.76:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f233400e070 con 0x7f233c0d26c0 2023-11-24T07:57:40.268 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:40.264+0000 7f2339ffb700 1 -- 172.21.15.59:0/304452665 <== mon.2 v2:172.21.15.76:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f233400bb20 con 0x7f233c0d26c0 2023-11-24T07:57:40.268 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:40.264+0000 7f2339ffb700 1 -- 172.21.15.59:0/304452665 <== mon.2 v2:172.21.15.76:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2334012430 con 0x7f233c0d26c0 2023-11-24T07:57:40.268 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:40.264+0000 7f2341318700 1 -- 172.21.15.59:0/304452665 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f233c0d26c0 msgr2=0x7f233c0b7430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:57:40.268 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:40.264+0000 7f2341318700 1 --2- 172.21.15.59:0/304452665 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f233c0d26c0 0x7f233c0b7430 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f2334009dc0 tx=0x7f233400b330 comp rx=0 tx=0).stop 2023-11-24T07:57:40.268 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:40.264+0000 7f2341318700 1 -- 172.21.15.59:0/304452665 shutdown_connections 2023-11-24T07:57:40.268 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:40.264+0000 7f2341318700 1 --2- 172.21.15.59:0/304452665 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f233c0d26c0 0x7f233c0b7430 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:40.268 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:40.264+0000 7f2341318700 1 -- 172.21.15.59:0/304452665 >> 172.21.15.59:0/304452665 conn(0x7f233c0d3ed0 msgr2=0x7f233c0d42e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:57:40.268 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:40.264+0000 7f2341318700 1 -- 172.21.15.59:0/304452665 shutdown_connections 2023-11-24T07:57:40.268 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:40.264+0000 7f2341318700 1 -- 172.21.15.59:0/304452665 wait complete. 2023-11-24T07:57:40.271 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:40.266+0000 7f2341318700 1 Processor -- start 2023-11-24T07:57:40.271 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:40.266+0000 7f2341318700 1 -- start start 2023-11-24T07:57:40.271 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:40.266+0000 7f2341318700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f233c0d0870 0x7f233c0a06c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:57:40.271 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:40.266+0000 7f2341318700 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f233c0d0c50 0x7f233c15a9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:57:40.272 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:40.266+0000 7f2341318700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f233c0d26c0 0x7f233c150200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:57:40.272 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:40.266+0000 7f2341318700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f233c0cff70 con 0x7f233c0d26c0 2023-11-24T07:57:40.272 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:40.266+0000 7f2341318700 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f233c0a1940 con 0x7f233c0d0870 2023-11-24T07:57:40.272 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:40.266+0000 7f2341318700 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f233c150740 con 0x7f233c0d0c50 2023-11-24T07:57:40.272 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:40.266+0000 7f233b7fe700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f233c0d26c0 0x7f233c150200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:57:40.272 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:40.266+0000 7f233b7fe700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f233c0d26c0 0x7f233c150200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:52000/0 (socket says 172.21.15.59:52000) 2023-11-24T07:57:40.272 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:40.266+0000 7f233b7fe700 1 -- 172.21.15.59:0/1495468278 learned_addr learned my addr 172.21.15.59:0/1495468278 (peer_addr_for_me v2:172.21.15.59:0/0) 2023-11-24T07:57:40.272 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:40.267+0000 7f233affd700 1 --2- 172.21.15.59:0/1495468278 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f233c0d0870 0x7f233c0a06c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:57:40.273 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:40.267+0000 7f233a7fc700 1 --2- 172.21.15.59:0/1495468278 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f233c0d0c50 0x7f233c15a9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:57:40.273 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:40.267+0000 7f233b7fe700 1 -- 172.21.15.59:0/1495468278 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f233c0d0c50 msgr2=0x7f233c15a9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:57:40.273 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:40.267+0000 7f233b7fe700 1 --2- 172.21.15.59:0/1495468278 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f233c0d0c50 0x7f233c15a9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:40.273 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:40.267+0000 7f233b7fe700 1 -- 172.21.15.59:0/1495468278 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f233c0d0870 msgr2=0x7f233c0a06c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:57:40.273 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:40.267+0000 7f233b7fe700 1 --2- 172.21.15.59:0/1495468278 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f233c0d0870 0x7f233c0a06c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:40.273 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:40.267+0000 7f233b7fe700 1 -- 172.21.15.59:0/1495468278 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2334009a70 con 0x7f233c0d26c0 2023-11-24T07:57:40.273 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:40.267+0000 7f233a7fc700 1 --2- 172.21.15.59:0/1495468278 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f233c0d0c50 0x7f233c15a9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-24T07:57:40.273 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:40.267+0000 7f233b7fe700 1 --2- 172.21.15.59:0/1495468278 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f233c0d26c0 0x7f233c150200 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f233400b370 tx=0x7f2334004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:57:40.273 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:40.267+0000 7f23237fe700 1 -- 172.21.15.59:0/1495468278 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f233401b810 con 0x7f233c0d26c0 2023-11-24T07:57:40.273 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:40.267+0000 7f23237fe700 1 -- 172.21.15.59:0/1495468278 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2334020070 con 0x7f233c0d26c0 2023-11-24T07:57:40.274 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:40.267+0000 7f2341318700 1 -- 172.21.15.59:0/1495468278 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f233c0a0cc0 con 0x7f233c0d26c0 2023-11-24T07:57:40.274 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:40.267+0000 7f23237fe700 1 -- 172.21.15.59:0/1495468278 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f233401bbf0 con 0x7f233c0d26c0 2023-11-24T07:57:40.274 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:40.267+0000 7f2341318700 1 -- 172.21.15.59:0/1495468278 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f233c0a1280 con 0x7f233c0d26c0 2023-11-24T07:57:40.274 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:40.268+0000 7f2341318700 1 -- 172.21.15.59:0/1495468278 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f233c0b57c0 con 0x7f233c0d26c0 2023-11-24T07:57:40.274 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:40.269+0000 7f23237fe700 1 -- 172.21.15.59:0/1495468278 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f233401bd50 con 0x7f233c0d26c0 2023-11-24T07:57:40.274 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:40.269+0000 7f23237fe700 1 --2- 172.21.15.59:0/1495468278 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f232806af00 0x7f232806d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:57:40.274 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:40.269+0000 7f23237fe700 1 -- 172.21.15.59:0/1495468278 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f2334022070 con 0x7f233c0d26c0 2023-11-24T07:57:40.274 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:40.269+0000 7f233affd700 1 --2- 172.21.15.59:0/1495468278 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f232806af00 0x7f232806d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:57:40.274 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:40.270+0000 7f233affd700 1 --2- 172.21.15.59:0/1495468278 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f232806af00 0x7f232806d3c0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f2330001010 tx=0x7f2330009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:57:40.275 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:40.274+0000 7f23237fe700 1 -- 172.21.15.59:0/1495468278 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f2334056910 con 0x7f233c0d26c0 2023-11-24T07:57:40.507 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:40.505+0000 7f2341318700 1 -- 172.21.15.59:0/1495468278 --> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] -- mgr_command(tid 0: {"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "subvol_grp3824", "target": ["mon-mgr", ""]}) v1 -- 0x7f233c0cfc80 con 0x7f232806af00 2023-11-24T07:57:40.512 INFO:tasks.ceph.mgr.x.smithi076.stderr:2023-11-24T07:57:40.510+0000 7ff196724700 -1 client.0 error registering admin socket command: (17) File exists 2023-11-24T07:57:40.512 INFO:tasks.ceph.mgr.x.smithi076.stderr:2023-11-24T07:57:40.510+0000 7ff196724700 -1 client.0 error registering admin socket command: (17) File exists 2023-11-24T07:57:40.513 INFO:tasks.ceph.mgr.x.smithi076.stderr:2023-11-24T07:57:40.510+0000 7ff196724700 -1 client.0 error registering admin socket command: (17) File exists 2023-11-24T07:57:40.513 INFO:tasks.ceph.mgr.x.smithi076.stderr:2023-11-24T07:57:40.510+0000 7ff196724700 -1 client.0 error registering admin socket command: (17) File exists 2023-11-24T07:57:40.514 INFO:tasks.ceph.mgr.x.smithi076.stderr:2023-11-24T07:57:40.510+0000 7ff196724700 -1 client.0 error registering admin socket command: (17) File exists 2023-11-24T07:57:40.528 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:40.527+0000 7f23237fe700 1 -- 172.21.15.59:0/1495468278 <== mgr.4100 v2:172.21.15.76:6832/4264046536 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7f233c0cfc80 con 0x7f232806af00 2023-11-24T07:57:40.531 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:40.529+0000 7f2341318700 1 -- 172.21.15.59:0/1495468278 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f232806af00 msgr2=0x7f232806d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:57:40.531 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:40.529+0000 7f2341318700 1 --2- 172.21.15.59:0/1495468278 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f232806af00 0x7f232806d3c0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f2330001010 tx=0x7f2330009040 comp rx=0 tx=0).stop 2023-11-24T07:57:40.531 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:40.529+0000 7f2341318700 1 -- 172.21.15.59:0/1495468278 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f233c0d26c0 msgr2=0x7f233c150200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:57:40.531 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:40.529+0000 7f2341318700 1 --2- 172.21.15.59:0/1495468278 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f233c0d26c0 0x7f233c150200 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f233400b370 tx=0x7f2334004ad0 comp rx=0 tx=0).stop 2023-11-24T07:57:40.531 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:40.529+0000 7f2341318700 1 -- 172.21.15.59:0/1495468278 shutdown_connections 2023-11-24T07:57:40.531 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:40.529+0000 7f2341318700 1 --2- 172.21.15.59:0/1495468278 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f232806af00 0x7f232806d3c0 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:40.531 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:40.529+0000 7f2341318700 1 --2- 172.21.15.59:0/1495468278 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f233c0d26c0 0x7f233c150200 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:40.531 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:40.529+0000 7f2341318700 1 --2- 172.21.15.59:0/1495468278 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f233c0d0870 0x7f233c0a06c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:40.531 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:40.529+0000 7f2341318700 1 --2- 172.21.15.59:0/1495468278 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f233c0d0c50 0x7f233c15a9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:40.532 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:40.529+0000 7f2341318700 1 -- 172.21.15.59:0/1495468278 >> 172.21.15.59:0/1495468278 conn(0x7f233c0d3ed0 msgr2=0x7f233c0d42b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:57:40.532 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:40.529+0000 7f2341318700 1 -- 172.21.15.59:0/1495468278 shutdown_connections 2023-11-24T07:57:40.532 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:40.529+0000 7f2341318700 1 -- 172.21.15.59:0/1495468278 wait complete. 2023-11-24T07:57:40.543 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolumegroup snapshot create cephfs subvol_grp3824 subvol_snap5966 2023-11-24T07:57:40.676 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:40.674+0000 7fd78fb46700 1 Processor -- start 2023-11-24T07:57:40.676 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:40.674+0000 7fd78fb46700 1 -- start start 2023-11-24T07:57:40.678 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:40.675+0000 7fd78fb46700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fd78815a340 0x7fd78806e460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:57:40.678 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:40.675+0000 7fd78fb46700 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fd78806d1d0 0x7fd78806d650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:57:40.678 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:40.675+0000 7fd78fb46700 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd7880cf8f0 con 0x7fd78815dfe0 2023-11-24T07:57:40.678 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:40.675+0000 7fd78fb46700 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd78806e9a0 con 0x7fd78815a340 2023-11-24T07:57:40.678 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:40.675+0000 7fd78fb46700 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd78806db90 con 0x7fd78806d1d0 2023-11-24T07:57:40.678 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:40.675+0000 7fd78d8e2700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fd78815a340 0x7fd78806e460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:57:40.678 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:40.675+0000 7fd78e0e3700 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7fd78815dfe0 0x7fd78815e470 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:55482/0 (socket says 172.21.15.59:55482) 2023-11-24T07:57:40.679 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:40.675+0000 7fd78e0e3700 1 -- 172.21.15.59:0/3919000738 learned_addr learned my addr 172.21.15.59:0/3919000738 (peer_addr_for_me v1:172.21.15.59:0/0) 2023-11-24T07:57:40.679 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:40.675+0000 7fd78d8e2700 1 --2- 172.21.15.59:0/3919000738 >> v2:172.21.15.59:3300/0 conn(0x7fd78815a340 0x7fd78806e460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:52004/0 (socket says 172.21.15.59:52004) 2023-11-24T07:57:40.679 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:40.675+0000 7fd78d0e1700 1 --2- 172.21.15.59:0/3919000738 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fd78806d1d0 0x7fd78806d650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:57:40.679 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:40.675+0000 7fd78d8e2700 1 -- 172.21.15.59:0/3919000738 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fd78806d1d0 msgr2=0x7fd78806d650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T07:57:40.679 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:40.675+0000 7fd78d8e2700 1 --2- 172.21.15.59:0/3919000738 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fd78806d1d0 0x7fd78806d650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:40.679 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:40.675+0000 7fd78d8e2700 1 -- 172.21.15.59:0/3919000738 >> v1:172.21.15.59:6789/0 conn(0x7fd78815dfe0 legacy=0x7fd78815e470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:57:40.679 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:40.675+0000 7fd78c8e0700 1 -- 172.21.15.59:0/3919000738 <== mon.0 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2425818027 0 0) 0x7fd7880cf8f0 con 0x7fd78815dfe0 2023-11-24T07:57:40.679 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:40.675+0000 7fd78d8e2700 1 -- 172.21.15.59:0/3919000738 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd7880cf610 con 0x7fd78815a340 2023-11-24T07:57:40.679 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:40.675+0000 7fd78d8e2700 1 --2- 172.21.15.59:0/3919000738 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd78815a340 0x7fd78806e460 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7fd784009840 tx=0x7fd784009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8c3996390086f2db server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:57:40.679 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:40.676+0000 7fd78c8e0700 1 -- 172.21.15.59:0/3919000738 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd784011830 con 0x7fd78815a340 2023-11-24T07:57:40.679 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:40.676+0000 7fd78c8e0700 1 -- 172.21.15.59:0/3919000738 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd784011990 con 0x7fd78815a340 2023-11-24T07:57:40.679 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:40.676+0000 7fd78c8e0700 1 -- 172.21.15.59:0/3919000738 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd784011d20 con 0x7fd78815a340 2023-11-24T07:57:40.679 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:40.676+0000 7fd78fb46700 1 -- 172.21.15.59:0/3919000738 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd78815a340 msgr2=0x7fd78806e460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:57:40.680 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:40.676+0000 7fd78fb46700 1 --2- 172.21.15.59:0/3919000738 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd78815a340 0x7fd78806e460 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7fd784009840 tx=0x7fd784009b50 comp rx=0 tx=0).stop 2023-11-24T07:57:40.680 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:40.676+0000 7fd78fb46700 1 -- 172.21.15.59:0/3919000738 shutdown_connections 2023-11-24T07:57:40.680 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:40.676+0000 7fd78fb46700 1 --2- 172.21.15.59:0/3919000738 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd78815a340 0x7fd78806e460 unknown :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:40.680 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:40.676+0000 7fd78fb46700 1 --2- 172.21.15.59:0/3919000738 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fd78806d1d0 0x7fd78806d650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:40.680 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:40.676+0000 7fd78fb46700 1 -- 172.21.15.59:0/3919000738 >> 172.21.15.59:0/3919000738 conn(0x7fd7880b1830 msgr2=0x7fd7880b1c10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:57:40.680 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:40.676+0000 7fd78fb46700 1 -- 172.21.15.59:0/3919000738 shutdown_connections 2023-11-24T07:57:40.680 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:40.676+0000 7fd78fb46700 1 -- 172.21.15.59:0/3919000738 wait complete. 2023-11-24T07:57:40.683 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:40.677+0000 7fd78fb46700 1 Processor -- start 2023-11-24T07:57:40.683 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:40.677+0000 7fd78fb46700 1 -- start start 2023-11-24T07:57:40.683 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:40.678+0000 7fd78fb46700 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fd78806d1d0 0x7fd7881679c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:57:40.683 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:40.678+0000 7fd78fb46700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd78815a340 0x7fd788149980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:57:40.683 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:40.678+0000 7fd78fb46700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd78815dfe0 0x7fd788158f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:57:40.683 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:40.678+0000 7fd78fb46700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd788167f00 con 0x7fd78815dfe0 2023-11-24T07:57:40.684 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:40.678+0000 7fd78fb46700 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd788157c70 con 0x7fd78815a340 2023-11-24T07:57:40.684 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:40.678+0000 7fd78fb46700 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd788149ec0 con 0x7fd78806d1d0 2023-11-24T07:57:40.684 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:40.678+0000 7fd78e0e3700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd78815dfe0 0x7fd788158f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:57:40.684 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:40.678+0000 7fd78e0e3700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd78815dfe0 0x7fd788158f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:52014/0 (socket says 172.21.15.59:52014) 2023-11-24T07:57:40.684 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:40.678+0000 7fd78e0e3700 1 -- 172.21.15.59:0/540734526 learned_addr learned my addr 172.21.15.59:0/540734526 (peer_addr_for_me v2:172.21.15.59:0/0) 2023-11-24T07:57:40.684 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:40.679+0000 7fd78d0e1700 1 --2- 172.21.15.59:0/540734526 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd78815a340 0x7fd788149980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:57:40.684 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:40.679+0000 7fd78d8e2700 1 --2- 172.21.15.59:0/540734526 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fd78806d1d0 0x7fd7881679c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:57:40.684 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:40.679+0000 7fd78e0e3700 1 -- 172.21.15.59:0/540734526 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fd78806d1d0 msgr2=0x7fd7881679c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:57:40.684 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:40.679+0000 7fd78e0e3700 1 --2- 172.21.15.59:0/540734526 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fd78806d1d0 0x7fd7881679c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:40.684 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:40.679+0000 7fd78e0e3700 1 -- 172.21.15.59:0/540734526 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd78815a340 msgr2=0x7fd788149980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:57:40.684 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:40.679+0000 7fd78e0e3700 1 --2- 172.21.15.59:0/540734526 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd78815a340 0x7fd788149980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:40.685 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:40.679+0000 7fd78e0e3700 1 -- 172.21.15.59:0/540734526 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd7840094a0 con 0x7fd78815dfe0 2023-11-24T07:57:40.685 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:40.679+0000 7fd78d8e2700 1 --2- 172.21.15.59:0/540734526 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fd78806d1d0 0x7fd7881679c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T07:57:40.685 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:40.679+0000 7fd78e0e3700 1 --2- 172.21.15.59:0/540734526 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd78815dfe0 0x7fd788158f60 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7fd78000cba0 tx=0x7fd78000cf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:57:40.685 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:40.679+0000 7fd77e7fc700 1 -- 172.21.15.59:0/540734526 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd78000ab80 con 0x7fd78815dfe0 2023-11-24T07:57:40.685 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:40.679+0000 7fd77e7fc700 1 -- 172.21.15.59:0/540734526 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd78000ad00 con 0x7fd78815dfe0 2023-11-24T07:57:40.685 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:40.679+0000 7fd78fb46700 1 -- 172.21.15.59:0/540734526 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd788157ec0 con 0x7fd78815dfe0 2023-11-24T07:57:40.685 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:40.679+0000 7fd77e7fc700 1 -- 172.21.15.59:0/540734526 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd780007310 con 0x7fd78815dfe0 2023-11-24T07:57:40.685 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:40.679+0000 7fd78fb46700 1 -- 172.21.15.59:0/540734526 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd788158480 con 0x7fd78815dfe0 2023-11-24T07:57:40.685 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:40.680+0000 7fd78fb46700 1 -- 172.21.15.59:0/540734526 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd788067f60 con 0x7fd78815dfe0 2023-11-24T07:57:40.685 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:40.680+0000 7fd77e7fc700 1 -- 172.21.15.59:0/540734526 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fd780007470 con 0x7fd78815dfe0 2023-11-24T07:57:40.685 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:40.681+0000 7fd77e7fc700 1 --2- 172.21.15.59:0/540734526 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7fd76c06aed0 0x7fd76c06d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:57:40.685 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:40.681+0000 7fd77e7fc700 1 -- 172.21.15.59:0/540734526 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fd780089060 con 0x7fd78815dfe0 2023-11-24T07:57:40.685 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:40.681+0000 7fd78d8e2700 1 --2- 172.21.15.59:0/540734526 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7fd76c06aed0 0x7fd76c06d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:57:40.686 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:40.681+0000 7fd78d8e2700 1 --2- 172.21.15.59:0/540734526 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7fd76c06aed0 0x7fd76c06d390 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7fd788166ad0 tx=0x7fd784008030 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:57:40.687 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:40.685+0000 7fd77e7fc700 1 -- 172.21.15.59:0/540734526 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fd780054110 con 0x7fd78815dfe0 2023-11-24T07:57:40.920 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:40.919+0000 7fd78fb46700 1 -- 172.21.15.59:0/540734526 --> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] -- mgr_command(tid 0: {"prefix": "fs subvolumegroup snapshot create", "vol_name": "cephfs", "group_name": "subvol_grp3824", "snap_name": "subvol_snap5966", "target": ["mon-mgr", ""]}) v1 -- 0x7fd788071b80 con 0x7fd76c06aed0 2023-11-24T07:57:40.922 INFO:tasks.ceph.mgr.x.smithi076.stderr:2023-11-24T07:57:40.920+0000 7ff196724700 -1 mgr.server reply reply (38) Function not implemented subvolume group snapshots are not supported 2023-11-24T07:57:40.923 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:40.921+0000 7fd77e7fc700 1 -- 172.21.15.59:0/540734526 <== mgr.4100 v2:172.21.15.76:6832/4264046536 1 ==== mgr_command_reply(tid 0: -38 subvolume group snapshots are not supported) v1 ==== 51+0+0 (secure 0 0 0) 0x7fd784011b00 con 0x7fd76c06aed0 2023-11-24T07:57:40.924 INFO:teuthology.orchestra.run.smithi059.stderr:Error ENOSYS: subvolume group snapshots are not supported 2023-11-24T07:57:40.927 DEBUG:teuthology.orchestra.run:got remote process result: 38 2023-11-24T07:57:40.928 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolumegroup rm cephfs subvol_grp3824 2023-11-24T07:57:41.056 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:41.054+0000 7f893ba4c700 1 Processor -- start 2023-11-24T07:57:41.056 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:41.054+0000 7f893ba4c700 1 -- start start 2023-11-24T07:57:41.059 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:41.055+0000 7f893ba4c700 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f8934143ae0 0x7f89340b51b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:57:41.060 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:41.055+0000 7f893ba4c700 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8934143ec0 con 0x7f89340722c0 2023-11-24T07:57:41.060 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:41.055+0000 7f893ba4c700 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f89340b3ae0 con 0x7f89341440f0 2023-11-24T07:57:41.060 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:41.055+0000 7f893ba4c700 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f89340b40a0 con 0x7f8934143ae0 2023-11-24T07:57:41.060 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:41.055+0000 7f8939fe9700 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7f89340722c0 0x7f893406e510 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:55496/0 (socket says 172.21.15.59:55496) 2023-11-24T07:57:41.060 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:41.055+0000 7f8939fe9700 1 -- 172.21.15.59:0/3899167890 learned_addr learned my addr 172.21.15.59:0/3899167890 (peer_addr_for_me v1:172.21.15.59:0/0) 2023-11-24T07:57:41.060 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:41.055+0000 7f89397e8700 1 --2- 172.21.15.59:0/3899167890 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f8934143ae0 0x7f89340b51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:57:41.060 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:41.055+0000 7f892bfff700 1 -- 172.21.15.59:0/3899167890 <== mon.0 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 654259215 0 0) 0x7f8934143ec0 con 0x7f89340722c0 2023-11-24T07:57:41.061 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:41.055+0000 7f892bfff700 1 -- 172.21.15.59:0/3899167890 --> v1:172.21.15.59:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f89200035b0 con 0x7f89340722c0 2023-11-24T07:57:41.061 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:41.056+0000 7f892bfff700 1 -- 172.21.15.59:0/3899167890 <== mon.1 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2096992120 0 0) 0x7f89340b3ae0 con 0x7f89341440f0 2023-11-24T07:57:41.061 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:41.056+0000 7f892bfff700 1 -- 172.21.15.59:0/3899167890 --> v1:172.21.15.76:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8934143ec0 con 0x7f89341440f0 2023-11-24T07:57:41.061 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:41.056+0000 7f892bfff700 1 -- 172.21.15.59:0/3899167890 <== mon.0 v1:172.21.15.59:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 426674990 0 0) 0x7f89200035b0 con 0x7f89340722c0 2023-11-24T07:57:41.061 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:41.056+0000 7f892bfff700 1 -- 172.21.15.59:0/3899167890 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f8934143ae0 msgr2=0x7f89340b51b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T07:57:41.061 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:41.056+0000 7f892bfff700 1 --2- 172.21.15.59:0/3899167890 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f8934143ae0 0x7f89340b51b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:41.061 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:41.056+0000 7f892bfff700 1 -- 172.21.15.59:0/3899167890 >> v1:172.21.15.76:6789/0 conn(0x7f89341440f0 legacy=0x7f8934071bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:57:41.062 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:41.056+0000 7f892bfff700 1 -- 172.21.15.59:0/3899167890 --> v1:172.21.15.59:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f89340b3dc0 con 0x7f89340722c0 2023-11-24T07:57:41.062 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:41.056+0000 7f892bfff700 1 -- 172.21.15.59:0/3899167890 <== mon.0 v1:172.21.15.59:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1312333566 0 0) 0x7f892c0019f0 con 0x7f89340722c0 2023-11-24T07:57:41.062 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:41.056+0000 7f89397e8700 1 --2- 172.21.15.59:0/3899167890 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f8934143ae0 0x7f89340b51b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-24T07:57:41.062 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:41.056+0000 7f892bfff700 1 -- 172.21.15.59:0/3899167890 >> v1:172.21.15.59:6789/0 conn(0x7f89340722c0 legacy=0x7f893406e510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:57:41.062 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:41.056+0000 7f892bfff700 1 --2- 172.21.15.59:0/3899167890 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8920003bc0 0x7f8920003fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:57:41.062 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:41.056+0000 7f892bfff700 1 --2- 172.21.15.59:0/3899167890 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8920004850 0x7f8920004cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:57:41.063 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:41.056+0000 7f892bfff700 1 --2- 172.21.15.59:0/3899167890 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f8934143ae0 0x7f8920005210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:57:41.063 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:41.056+0000 7f892bfff700 1 -- 172.21.15.59:0/3899167890 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89340b40a0 con 0x7f8920004850 2023-11-24T07:57:41.063 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:41.056+0000 7f892bfff700 1 -- 172.21.15.59:0/3899167890 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f892000dc50 con 0x7f8920003bc0 2023-11-24T07:57:41.063 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:41.056+0000 7f892bfff700 1 -- 172.21.15.59:0/3899167890 --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f892000de10 con 0x7f8934143ae0 2023-11-24T07:57:41.063 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:41.056+0000 7f8938fe7700 1 --2- 172.21.15.59:0/3899167890 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8920004850 0x7f8920004cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:57:41.063 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:41.056+0000 7f89397e8700 1 --2- 172.21.15.59:0/3899167890 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8920003bc0 0x7f8920003fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:57:41.063 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:41.056+0000 7f8939fe9700 1 --2- 172.21.15.59:0/3899167890 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f8934143ae0 0x7f8920005210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:57:41.063 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:41.056+0000 7f8938fe7700 1 -- 172.21.15.59:0/3899167890 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f8934143ae0 msgr2=0x7f8920005210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T07:57:41.064 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:41.056+0000 7f8938fe7700 1 --2- 172.21.15.59:0/3899167890 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f8934143ae0 0x7f8920005210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:41.064 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:41.056+0000 7f8938fe7700 1 -- 172.21.15.59:0/3899167890 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8920003bc0 msgr2=0x7f8920003fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T07:57:41.064 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:41.056+0000 7f8938fe7700 1 --2- 172.21.15.59:0/3899167890 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8920003bc0 0x7f8920003fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:41.064 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:41.056+0000 7f8938fe7700 1 -- 172.21.15.59:0/3899167890 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f89240094a0 con 0x7f8920004850 2023-11-24T07:57:41.064 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:41.056+0000 7f8938fe7700 1 --2- 172.21.15.59:0/3899167890 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8920004850 0x7f8920004cd0 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f8930007fd0 tx=0x7f8920005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c01b0473db19b96d server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:57:41.064 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:41.057+0000 7f892bfff700 1 -- 172.21.15.59:0/3899167890 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8930001430 con 0x7f8920004850 2023-11-24T07:57:41.064 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:41.057+0000 7f892bfff700 1 -- 172.21.15.59:0/3899167890 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f89300052d0 con 0x7f8920004850 2023-11-24T07:57:41.065 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:41.057+0000 7f892bfff700 1 -- 172.21.15.59:0/3899167890 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8930001d40 con 0x7f8920004850 2023-11-24T07:57:41.065 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:41.057+0000 7f893ba4c700 1 -- 172.21.15.59:0/3899167890 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8920004850 msgr2=0x7f8920004cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:57:41.065 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:41.057+0000 7f893ba4c700 1 --2- 172.21.15.59:0/3899167890 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8920004850 0x7f8920004cd0 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f8930007fd0 tx=0x7f8920005880 comp rx=0 tx=0).stop 2023-11-24T07:57:41.065 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:41.057+0000 7f8939fe9700 1 -- 172.21.15.59:0/3899167890 reap_dead start 2023-11-24T07:57:41.065 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:41.057+0000 7f893ba4c700 1 -- 172.21.15.59:0/3899167890 shutdown_connections 2023-11-24T07:57:41.066 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:41.057+0000 7f893ba4c700 1 -- 172.21.15.59:0/3899167890 >> 172.21.15.59:0/3899167890 conn(0x7f89340b2740 msgr2=0x7f89340b2b20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:57:41.066 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:41.057+0000 7f893ba4c700 1 -- 172.21.15.59:0/3899167890 shutdown_connections 2023-11-24T07:57:41.066 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:41.057+0000 7f893ba4c700 1 -- 172.21.15.59:0/3899167890 wait complete. 2023-11-24T07:57:41.069 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:41.059+0000 7f893ba4c700 1 Processor -- start 2023-11-24T07:57:41.069 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:41.059+0000 7f893ba4c700 1 -- start start 2023-11-24T07:57:41.069 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:41.060+0000 7f893ba4c700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8920004850 0x7f89341eed10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:57:41.069 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:41.060+0000 7f893ba4c700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f893415b2f0 0x7f893415b770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:57:41.069 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:41.060+0000 7f893ba4c700 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f893413f1d0 0x7f893413f680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:57:41.070 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:41.060+0000 7f893ba4c700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f893413e780 con 0x7f8920004850 2023-11-24T07:57:41.070 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:41.060+0000 7f893ba4c700 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f893415dfe0 con 0x7f893415b2f0 2023-11-24T07:57:41.070 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:41.060+0000 7f893ba4c700 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f893415e2c0 con 0x7f893413f1d0 2023-11-24T07:57:41.070 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:41.060+0000 7f89397e8700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8920004850 0x7f89341eed10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:57:41.070 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:41.060+0000 7f89397e8700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8920004850 0x7f89341eed10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:52030/0 (socket says 172.21.15.59:52030) 2023-11-24T07:57:41.070 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:41.060+0000 7f89397e8700 1 -- 172.21.15.59:0/3755334752 learned_addr learned my addr 172.21.15.59:0/3755334752 (peer_addr_for_me v2:172.21.15.59:0/0) 2023-11-24T07:57:41.070 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:41.060+0000 7f8938fe7700 1 --2- 172.21.15.59:0/3755334752 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f893415b2f0 0x7f893415b770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:57:41.071 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:41.060+0000 7f8939fe9700 1 --2- 172.21.15.59:0/3755334752 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f893413f1d0 0x7f893413f680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:57:41.071 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:41.060+0000 7f89397e8700 1 -- 172.21.15.59:0/3755334752 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f893413f1d0 msgr2=0x7f893413f680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:57:41.071 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:41.060+0000 7f89397e8700 1 --2- 172.21.15.59:0/3755334752 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f893413f1d0 0x7f893413f680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:41.071 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:41.060+0000 7f89397e8700 1 -- 172.21.15.59:0/3755334752 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f893415b2f0 msgr2=0x7f893415b770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:57:41.071 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:41.060+0000 7f89397e8700 1 --2- 172.21.15.59:0/3755334752 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f893415b2f0 0x7f893415b770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:41.071 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:41.060+0000 7f89397e8700 1 -- 172.21.15.59:0/3755334752 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f893413e9d0 con 0x7f8920004850 2023-11-24T07:57:41.071 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:41.060+0000 7f8939fe9700 1 --2- 172.21.15.59:0/3755334752 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f893413f1d0 0x7f893413f680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T07:57:41.071 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:41.060+0000 7f8938fe7700 1 --2- 172.21.15.59:0/3755334752 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f893415b2f0 0x7f893415b770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T07:57:41.072 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:41.060+0000 7f89397e8700 1 --2- 172.21.15.59:0/3755334752 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8920004850 0x7f89341eed10 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f8924011cd0 tx=0x7f892400f520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:57:41.072 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:41.061+0000 7f8929ffb700 1 -- 172.21.15.59:0/3755334752 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8924011e70 con 0x7f8920004850 2023-11-24T07:57:41.072 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:41.061+0000 7f8929ffb700 1 -- 172.21.15.59:0/3755334752 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f892400fd40 con 0x7f8920004850 2023-11-24T07:57:41.072 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:41.061+0000 7f893ba4c700 1 -- 172.21.15.59:0/3755334752 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f893413ecb0 con 0x7f8920004850 2023-11-24T07:57:41.072 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:41.061+0000 7f8929ffb700 1 -- 172.21.15.59:0/3755334752 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8924004240 con 0x7f8920004850 2023-11-24T07:57:41.072 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:41.061+0000 7f893ba4c700 1 -- 172.21.15.59:0/3755334752 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f89340cf7d0 con 0x7f8920004850 2023-11-24T07:57:41.072 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:41.062+0000 7f893ba4c700 1 -- 172.21.15.59:0/3755334752 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f893413f020 con 0x7f8920004850 2023-11-24T07:57:41.073 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:41.062+0000 7f8929ffb700 1 -- 172.21.15.59:0/3755334752 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f89240043a0 con 0x7f8920004850 2023-11-24T07:57:41.073 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:41.063+0000 7f8929ffb700 1 --2- 172.21.15.59:0/3755334752 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f891806aed0 0x7f891806d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:57:41.073 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:41.063+0000 7f8929ffb700 1 -- 172.21.15.59:0/3755334752 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f8924089b50 con 0x7f8920004850 2023-11-24T07:57:41.073 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:41.063+0000 7f8938fe7700 1 --2- 172.21.15.59:0/3755334752 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f891806aed0 0x7f891806d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:57:41.073 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:41.063+0000 7f8938fe7700 1 --2- 172.21.15.59:0/3755334752 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f891806aed0 0x7f891806d390 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f89340b2590 tx=0x7f893000a000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:57:41.073 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:41.067+0000 7f8929ffb700 1 -- 172.21.15.59:0/3755334752 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f8924054c90 con 0x7f8920004850 2023-11-24T07:57:41.302 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:41.300+0000 7f893ba4c700 1 -- 172.21.15.59:0/3755334752 --> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] -- mgr_command(tid 0: {"prefix": "fs subvolumegroup rm", "vol_name": "cephfs", "group_name": "subvol_grp3824", "target": ["mon-mgr", ""]}) v1 -- 0x7f893406dff0 con 0x7f891806aed0 2023-11-24T07:57:41.306 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:41.304+0000 7f8929ffb700 1 -- 172.21.15.59:0/3755334752 <== mgr.4100 v2:172.21.15.76:6832/4264046536 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7f89300050d0 con 0x7f891806aed0 2023-11-24T07:57:41.309 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:41.307+0000 7f893ba4c700 1 -- 172.21.15.59:0/3755334752 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f891806aed0 msgr2=0x7f891806d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:57:41.309 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:41.307+0000 7f893ba4c700 1 --2- 172.21.15.59:0/3755334752 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f891806aed0 0x7f891806d390 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f89340b2590 tx=0x7f893000a000 comp rx=0 tx=0).stop 2023-11-24T07:57:41.309 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:41.307+0000 7f893ba4c700 1 -- 172.21.15.59:0/3755334752 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8920004850 msgr2=0x7f89341eed10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:57:41.309 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:41.307+0000 7f893ba4c700 1 --2- 172.21.15.59:0/3755334752 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8920004850 0x7f89341eed10 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f8924011cd0 tx=0x7f892400f520 comp rx=0 tx=0).stop 2023-11-24T07:57:41.309 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:41.307+0000 7f893ba4c700 1 -- 172.21.15.59:0/3755334752 shutdown_connections 2023-11-24T07:57:41.309 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:41.307+0000 7f893ba4c700 1 --2- 172.21.15.59:0/3755334752 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f891806aed0 0x7f891806d390 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:41.309 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:41.307+0000 7f893ba4c700 1 --2- 172.21.15.59:0/3755334752 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f893413f1d0 0x7f893413f680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:41.309 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:41.307+0000 7f893ba4c700 1 --2- 172.21.15.59:0/3755334752 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8920004850 0x7f89341eed10 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:41.309 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:41.307+0000 7f893ba4c700 1 --2- 172.21.15.59:0/3755334752 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f893415b2f0 0x7f893415b770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:41.309 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:41.307+0000 7f893ba4c700 1 -- 172.21.15.59:0/3755334752 >> 172.21.15.59:0/3755334752 conn(0x7f89340b2740 msgr2=0x7f8934158420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:57:41.309 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:41.307+0000 7f893ba4c700 1 -- 172.21.15.59:0/3755334752 shutdown_connections 2023-11-24T07:57:41.309 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:41.307+0000 7f893ba4c700 1 -- 172.21.15.59:0/3755334752 wait complete. 2023-11-24T07:57:41.321 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-11-24T07:57:41.321 DEBUG:teuthology.orchestra.run.smithi059:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-11-24T07:57:41.366 DEBUG:teuthology.orchestra.run.smithi076:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-11-24T07:57:41.416 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-11-24T07:57:41.417 DEBUG:teuthology.orchestra.run.smithi059:> set -ex 2023-11-24T07:57:41.417 DEBUG:teuthology.orchestra.run.smithi059:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-24T07:57:41.435 DEBUG:teuthology.orchestra.run.smithi059:> set -ex 2023-11-24T07:57:41.435 DEBUG:teuthology.orchestra.run.smithi059:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-24T07:57:41.493 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-24T07:57:41.661 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:41.659+0000 7f61a81f9700 1 Processor -- start 2023-11-24T07:57:41.662 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:41.659+0000 7f61a81f9700 1 -- start start 2023-11-24T07:57:41.667 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:41.660+0000 7f61a81f9700 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f61a006fc50 0x7f61a006eef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:57:41.667 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:41.660+0000 7f61a81f9700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f61a00b0e90 0x7f61a006ca10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:57:41.667 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:41.660+0000 7f61a81f9700 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f61a00d1230 con 0x7f61a015e1c0 2023-11-24T07:57:41.668 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:41.660+0000 7f61a81f9700 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f61a00d0c70 con 0x7f61a00b0e90 2023-11-24T07:57:41.668 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:41.660+0000 7f61a81f9700 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f61a00d0f50 con 0x7f61a006fc50 2023-11-24T07:57:41.668 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:41.660+0000 7f61a6796700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f61a00b0e90 0x7f61a006ca10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:57:41.668 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:41.660+0000 7f61a6796700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f61a00b0e90 0x7f61a006ca10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:52042/0 (socket says 172.21.15.59:52042) 2023-11-24T07:57:41.668 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:41.660+0000 7f61a6796700 1 -- 172.21.15.59:0/2378380931 learned_addr learned my addr 172.21.15.59:0/2378380931 (peer_addr_for_me v2:172.21.15.59:0/0) 2023-11-24T07:57:41.668 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:41.660+0000 7f61a5794700 1 --2- 172.21.15.59:0/2378380931 >> v2:172.21.15.76:3300/0 conn(0x7f61a006fc50 0x7f61a006eef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:57:41.668 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:41.661+0000 7f61a6796700 1 -- 172.21.15.59:0/2378380931 >> v2:172.21.15.76:3300/0 conn(0x7f61a006fc50 msgr2=0x7f61a006eef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T07:57:41.668 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:41.661+0000 7f61a6796700 1 --2- 172.21.15.59:0/2378380931 >> v2:172.21.15.76:3300/0 conn(0x7f61a006fc50 0x7f61a006eef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:41.668 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:41.661+0000 7f61a6796700 1 -- 172.21.15.59:0/2378380931 >> v1:172.21.15.76:6789/0 conn(0x7f61a015e1c0 legacy=0x7f61a006f540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:57:41.668 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:41.661+0000 7f61a6796700 1 -- 172.21.15.59:0/2378380931 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f61a006c210 con 0x7f61a00b0e90 2023-11-24T07:57:41.668 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:41.661+0000 7f61a6796700 1 --2- 172.21.15.59:0/2378380931 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f61a00b0e90 0x7f61a006ca10 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f61a0071ba0 tx=0x7f619800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b3e7b43d92385801 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:57:41.668 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:41.661+0000 7f61a4f93700 1 -- 172.21.15.59:0/2378380931 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f619800e070 con 0x7f61a00b0e90 2023-11-24T07:57:41.668 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:41.661+0000 7f61a4f93700 1 -- 172.21.15.59:0/2378380931 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f619800b910 con 0x7f61a00b0e90 2023-11-24T07:57:41.669 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:41.661+0000 7f61a4f93700 1 -- 172.21.15.59:0/2378380931 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f619800bcc0 con 0x7f61a00b0e90 2023-11-24T07:57:41.669 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:41.662+0000 7f61a81f9700 1 -- 172.21.15.59:0/2378380931 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f61a00b0e90 msgr2=0x7f61a006ca10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:57:41.669 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:41.662+0000 7f61a81f9700 1 --2- 172.21.15.59:0/2378380931 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f61a00b0e90 0x7f61a006ca10 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f61a0071ba0 tx=0x7f619800b330 comp rx=0 tx=0).stop 2023-11-24T07:57:41.669 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:41.662+0000 7f61a81f9700 1 -- 172.21.15.59:0/2378380931 shutdown_connections 2023-11-24T07:57:41.669 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:41.662+0000 7f61a81f9700 1 --2- 172.21.15.59:0/2378380931 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f61a00b0e90 0x7f61a006ca10 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:41.669 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:41.662+0000 7f61a81f9700 1 --2- 172.21.15.59:0/2378380931 >> v2:172.21.15.76:3300/0 conn(0x7f61a006fc50 0x7f61a006eef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:41.669 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:41.662+0000 7f61a81f9700 1 -- 172.21.15.59:0/2378380931 >> 172.21.15.59:0/2378380931 conn(0x7f61a00b3720 msgr2=0x7f61a00b3b30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:57:41.669 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:41.662+0000 7f61a81f9700 1 -- 172.21.15.59:0/2378380931 shutdown_connections 2023-11-24T07:57:41.669 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:41.662+0000 7f61a81f9700 1 -- 172.21.15.59:0/2378380931 wait complete. 2023-11-24T07:57:41.669 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:41.663+0000 7f61a81f9700 1 Processor -- start 2023-11-24T07:57:41.669 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:41.663+0000 7f61a81f9700 1 -- start start 2023-11-24T07:57:41.669 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:41.664+0000 7f61a81f9700 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f61a006fc50 0x7f61a009f540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:57:41.669 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:41.664+0000 7f61a81f9700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f61a00b0e90 0x7f61a0159070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:57:41.670 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:41.664+0000 7f61a81f9700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f61a015e1c0 0x7f61a01f6430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:57:41.670 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:41.664+0000 7f61a81f9700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61a009fa80 con 0x7f61a00b0e90 2023-11-24T07:57:41.670 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:41.664+0000 7f61a81f9700 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61a01f53f0 con 0x7f61a015e1c0 2023-11-24T07:57:41.670 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:41.664+0000 7f61a81f9700 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f61a01f5670 con 0x7f61a006fc50 2023-11-24T07:57:41.670 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:41.664+0000 7f61a5794700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f61a00b0e90 0x7f61a0159070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:57:41.670 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:41.664+0000 7f61a5794700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f61a00b0e90 0x7f61a0159070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:52046/0 (socket says 172.21.15.59:52046) 2023-11-24T07:57:41.670 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:41.664+0000 7f61a5794700 1 -- 172.21.15.59:0/819352713 learned_addr learned my addr 172.21.15.59:0/819352713 (peer_addr_for_me v2:172.21.15.59:0/0) 2023-11-24T07:57:41.670 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:41.664+0000 7f61a5f95700 1 --2- 172.21.15.59:0/819352713 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f61a006fc50 0x7f61a009f540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:57:41.670 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:41.664+0000 7f61a6796700 1 --2- 172.21.15.59:0/819352713 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f61a015e1c0 0x7f61a01f6430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:57:41.670 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:41.664+0000 7f61a5794700 1 -- 172.21.15.59:0/819352713 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f61a006fc50 msgr2=0x7f61a009f540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:57:41.670 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:41.664+0000 7f61a5794700 1 --2- 172.21.15.59:0/819352713 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f61a006fc50 0x7f61a009f540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:41.670 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:41.664+0000 7f61a5794700 1 -- 172.21.15.59:0/819352713 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f61a015e1c0 msgr2=0x7f61a01f6430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:57:41.670 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:41.664+0000 7f61a5794700 1 --2- 172.21.15.59:0/819352713 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f61a015e1c0 0x7f61a01f6430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:41.671 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:41.664+0000 7f61a5794700 1 -- 172.21.15.59:0/819352713 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6198009b30 con 0x7f61a00b0e90 2023-11-24T07:57:41.671 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:41.664+0000 7f61a5f95700 1 --2- 172.21.15.59:0/819352713 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f61a006fc50 0x7f61a009f540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-24T07:57:41.671 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:41.664+0000 7f61a6796700 1 --2- 172.21.15.59:0/819352713 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f61a015e1c0 0x7f61a01f6430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-24T07:57:41.671 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:41.664+0000 7f61a5794700 1 --2- 172.21.15.59:0/819352713 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f61a00b0e90 0x7f61a0159070 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f619c00e9b0 tx=0x7f619c00ed70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:57:41.671 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:41.664+0000 7f61967fc700 1 -- 172.21.15.59:0/819352713 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f619c009710 con 0x7f61a00b0e90 2023-11-24T07:57:41.671 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:41.664+0000 7f61967fc700 1 -- 172.21.15.59:0/819352713 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f619c00c850 con 0x7f61a00b0e90 2023-11-24T07:57:41.671 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:41.664+0000 7f61a81f9700 1 -- 172.21.15.59:0/819352713 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f61a01f5950 con 0x7f61a00b0e90 2023-11-24T07:57:41.671 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:41.664+0000 7f61967fc700 1 -- 172.21.15.59:0/819352713 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f619c00cbc0 con 0x7f61a00b0e90 2023-11-24T07:57:41.671 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:41.664+0000 7f61a81f9700 1 -- 172.21.15.59:0/819352713 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f61a01f5f10 con 0x7f61a00b0e90 2023-11-24T07:57:41.671 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:41.665+0000 7f61a81f9700 1 -- 172.21.15.59:0/819352713 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f61a0068670 con 0x7f61a00b0e90 2023-11-24T07:57:41.671 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:41.666+0000 7f61967fc700 1 -- 172.21.15.59:0/819352713 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 6) v1 ==== 89282+0+0 (secure 0 0 0) 0x7f619c00cd20 con 0x7f61a00b0e90 2023-11-24T07:57:41.672 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:41.666+0000 7f61967fc700 1 --2- 172.21.15.59:0/819352713 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f618406af60 0x7f618406d420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:57:41.672 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:41.666+0000 7f61967fc700 1 -- 172.21.15.59:0/819352713 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f619c089760 con 0x7f61a00b0e90 2023-11-24T07:57:41.672 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:41.667+0000 7f61a5f95700 1 --2- 172.21.15.59:0/819352713 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f618406af60 0x7f618406d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:57:41.672 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:41.667+0000 7f61a5f95700 1 --2- 172.21.15.59:0/819352713 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f618406af60 0x7f618406d420 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f6190000f60 tx=0x7f6190009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:57:41.672 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:41.671+0000 7f61967fc700 1 -- 172.21.15.59:0/819352713 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f619c054820 con 0x7f61a00b0e90 2023-11-24T07:57:41.872 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:41.871+0000 7f61a81f9700 1 -- 172.21.15.59:0/819352713 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f61a01f6130 con 0x7f61a00b0e90 2023-11-24T07:57:41.874 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:41.872+0000 7f61967fc700 1 -- 172.21.15.59:0/819352713 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v33) v1 ==== 74+0+16437 (secure 0 0 0) 0x7f619c057e40 con 0x7f61a00b0e90 2023-11-24T07:57:41.874 INFO:teuthology.orchestra.run.smithi059.stdout: 2023-11-24T07:57:41.874 INFO:teuthology.orchestra.run.smithi059.stdout:{"epoch":33,"fsid":"9f0e8741-b8f9-48d5-a2c3-09f45903303d","created":"2023-11-24T07:56:06.552729+0000","modified":"2023-11-24T07:57:26.579209+0000","last_up_change":"2023-11-24T07:56:17.372352+0000","last_in_change":"2023-11-24T07:56:09.073250+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-24T07:56:18.352823+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2023-11-24T07:57:23.014143+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"32","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2023-11-24T07:57:23.971487+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"33","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"6d8f9e9b-1782-4e41-928b-a39f2a8ebc06","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6800","nonce":3373878772},{"type":"v1","addr":"172.21.15.59:6801","nonce":3373878772}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6802","nonce":3373878772},{"type":"v1","addr":"172.21.15.59:6803","nonce":3373878772}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6806","nonce":3373878772},{"type":"v1","addr":"172.21.15.59:6807","nonce":3373878772}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6804","nonce":3373878772},{"type":"v1","addr":"172.21.15.59:6805","nonce":3373878772}]},"public_addr":"172.21.15.59:6801/3373878772","cluster_addr":"172.21.15.59:6803/3373878772","heartbeat_back_addr":"172.21.15.59:6807/3373878772","heartbeat_front_addr":"172.21.15.59:6805/3373878772","state":["exists","up"]},{"osd":1,"uuid":"e8d882aa-52e1-4c04-9e1e-c44ef9204a5f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6824","nonce":648258563},{"type":"v1","addr":"172.21.15.59:6825","nonce":648258563}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6826","nonce":648258563},{"type":"v1","addr":"172.21.15.59:6827","nonce":648258563}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6830","nonce":648258563},{"type":"v1","addr":"172.21.15.59:6831","nonce":648258563}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6828","nonce":648258563},{"type":"v1","addr":"172.21.15.59:6829","nonce":648258563}]},"public_addr":"172.21.15.59:6825/648258563","cluster_addr":"172.21.15.59:6827/648258563","heartbeat_back_addr":"172.21.15.59:6831/648258563","heartbeat_front_addr":"172.21.15.59:6829/648258563","state":["exists","up"]},{"osd":2,"uuid":"69eebfa0-695b-4b21-90b9-8a3537168a01","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6808","nonce":1840430654},{"type":"v1","addr":"172.21.15.59:6809","nonce":1840430654}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6810","nonce":1840430654},{"type":"v1","addr":"172.21.15.59:6811","nonce":1840430654}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6814","nonce":1840430654},{"type":"v1","addr":"172.21.15.59:6815","nonce":1840430654}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6812","nonce":1840430654},{"type":"v1","addr":"172.21.15.59:6813","nonce":1840430654}]},"public_addr":"172.21.15.59:6809/1840430654","cluster_addr":"172.21.15.59:6811/1840430654","heartbeat_back_addr":"172.21.15.59:6815/1840430654","heartbeat_front_addr":"172.21.15.59:6813/1840430654","state":["exists","up"]},{"osd":3,"uuid":"96fd77fc-f911-46ff-9445-6f0be2daa8f4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6816","nonce":2504025850},{"type":"v1","addr":"172.21.15.59:6817","nonce":2504025850}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6818","nonce":2504025850},{"type":"v1","addr":"172.21.15.59:6819","nonce":2504025850}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6822","nonce":2504025850},{"type":"v1","addr":"172.21.15.59:6823","nonce":2504025850}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6820","nonce":2504025850},{"type":"v1","addr":"172.21.15.59:6821","nonce":2504025850}]},"public_addr":"172.21.15.59:6817/2504025850","cluster_addr":"172.21.15.59:6819/2504025850","heartbeat_back_addr":"172.21.15.59:6823/2504025850","heartbeat_front_addr":"172.21.15.59:6821/2504025850","state":["exists","up"]},{"osd":4,"uuid":"7ac29fca-b5b2-4c95-9d69-2f164cbbc708","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6816","nonce":2382463431},{"type":"v1","addr":"172.21.15.76:6818","nonce":2382463431}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6820","nonce":2382463431},{"type":"v1","addr":"172.21.15.76:6822","nonce":2382463431}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6828","nonce":2382463431},{"type":"v1","addr":"172.21.15.76:6830","nonce":2382463431}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6824","nonce":2382463431},{"type":"v1","addr":"172.21.15.76:6826","nonce":2382463431}]},"public_addr":"172.21.15.76:6818/2382463431","cluster_addr":"172.21.15.76:6822/2382463431","heartbeat_back_addr":"172.21.15.76:6830/2382463431","heartbeat_front_addr":"172.21.15.76:6826/2382463431","state":["exists","up"]},{"osd":5,"uuid":"485bf711-3851-4238-a4f4-51d52e0a9c58","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6800","nonce":329334308},{"type":"v1","addr":"172.21.15.76:6802","nonce":329334308}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6805","nonce":329334308},{"type":"v1","addr":"172.21.15.76:6807","nonce":329334308}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6814","nonce":329334308},{"type":"v1","addr":"172.21.15.76:6815","nonce":329334308}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6810","nonce":329334308},{"type":"v1","addr":"172.21.15.76:6812","nonce":329334308}]},"public_addr":"172.21.15.76:6802/329334308","cluster_addr":"172.21.15.76:6807/329334308","heartbeat_back_addr":"172.21.15.76:6815/329334308","heartbeat_front_addr":"172.21.15.76:6812/329334308","state":["exists","up"]},{"osd":6,"uuid":"72b3f2fc-1e4a-4e32-968d-003647a0286f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6801","nonce":3946579410},{"type":"v1","addr":"172.21.15.76:6803","nonce":3946579410}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6804","nonce":3946579410},{"type":"v1","addr":"172.21.15.76:6806","nonce":3946579410}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6811","nonce":3946579410},{"type":"v1","addr":"172.21.15.76:6813","nonce":3946579410}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6808","nonce":3946579410},{"type":"v1","addr":"172.21.15.76:6809","nonce":3946579410}]},"public_addr":"172.21.15.76:6803/3946579410","cluster_addr":"172.21.15.76:6806/3946579410","heartbeat_back_addr":"172.21.15.76:6813/3946579410","heartbeat_front_addr":"172.21.15.76:6809/3946579410","state":["exists","up"]},{"osd":7,"uuid":"1ddceb90-6c5d-4d1a-ad56-99cb877f8c70","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6817","nonce":841761538},{"type":"v1","addr":"172.21.15.76:6819","nonce":841761538}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6821","nonce":841761538},{"type":"v1","addr":"172.21.15.76:6823","nonce":841761538}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6829","nonce":841761538},{"type":"v1","addr":"172.21.15.76:6831","nonce":841761538}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6825","nonce":841761538},{"type":"v1","addr":"172.21.15.76:6827","nonce":841761538}]},"public_addr":"172.21.15.76:6819/841761538","cluster_addr":"172.21.15.76:6823/841761538","heartbeat_back_addr":"172.21.15.76:6831/841761538","heartbeat_front_addr":"172.21.15.76:6827/841761538","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T07:56:14.294838+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T07:56:14.270779+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T07:56:14.231809+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T07:56:14.260983+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T07:56:14.302997+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T07:56:14.258600+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T07:56:14.251413+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T07:56:14.304727+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-24T07:57:41.876 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:41.874+0000 7f61a81f9700 1 -- 172.21.15.59:0/819352713 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f618406af60 msgr2=0x7f618406d420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:57:41.876 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:41.874+0000 7f61a81f9700 1 --2- 172.21.15.59:0/819352713 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f618406af60 0x7f618406d420 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f6190000f60 tx=0x7f6190009040 comp rx=0 tx=0).stop 2023-11-24T07:57:41.876 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:41.874+0000 7f61a81f9700 1 -- 172.21.15.59:0/819352713 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f61a00b0e90 msgr2=0x7f61a0159070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:57:41.877 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:41.874+0000 7f61a81f9700 1 --2- 172.21.15.59:0/819352713 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f61a00b0e90 0x7f61a0159070 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f619c00e9b0 tx=0x7f619c00ed70 comp rx=0 tx=0).stop 2023-11-24T07:57:41.877 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:41.874+0000 7f61a81f9700 1 -- 172.21.15.59:0/819352713 shutdown_connections 2023-11-24T07:57:41.877 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:41.874+0000 7f61a81f9700 1 --2- 172.21.15.59:0/819352713 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f618406af60 0x7f618406d420 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:41.877 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:41.874+0000 7f61a81f9700 1 --2- 172.21.15.59:0/819352713 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f61a015e1c0 0x7f61a01f6430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:41.877 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:41.874+0000 7f61a81f9700 1 --2- 172.21.15.59:0/819352713 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f61a006fc50 0x7f61a009f540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:41.877 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:41.874+0000 7f61a81f9700 1 --2- 172.21.15.59:0/819352713 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f61a00b0e90 0x7f61a0159070 unknown :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:41.877 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:41.874+0000 7f61a81f9700 1 -- 172.21.15.59:0/819352713 >> 172.21.15.59:0/819352713 conn(0x7f61a00b3720 msgr2=0x7f61a006d350 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:57:41.877 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:41.874+0000 7f61a81f9700 1 -- 172.21.15.59:0/819352713 shutdown_connections 2023-11-24T07:57:41.877 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:41.874+0000 7f61a81f9700 1 -- 172.21.15.59:0/819352713 wait complete. 2023-11-24T07:57:41.888 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-24T07:56:18.352823+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-24T07:57:23.014143+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '32', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 5, 'pool_name': 'cephfs_data', 'create_time': '2023-11-24T07:57:23.971487+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '33', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-24T07:57:41.889 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-24T07:57:42.016 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:42.015+0000 7f3b13592700 1 Processor -- start 2023-11-24T07:57:42.017 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:42.015+0000 7f3b13592700 1 -- start start 2023-11-24T07:57:42.019 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:42.016+0000 7f3b13592700 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f3b0c15e090 0x7f3b0c06afd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:57:42.019 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:42.016+0000 7f3b13592700 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f3b0c1623d0 0x7f3b0c0cb160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:57:42.019 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:42.016+0000 7f3b13592700 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3b0c15e470 con 0x7f3b0c06b510 2023-11-24T07:57:42.020 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:42.016+0000 7f3b13592700 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3b0c0c6900 con 0x7f3b0c15e090 2023-11-24T07:57:42.020 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:42.016+0000 7f3b13592700 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3b0c06b990 con 0x7f3b0c1623d0 2023-11-24T07:57:42.020 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:42.016+0000 7f3b10b2d700 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7f3b0c06b510 0x7f3b0c161cc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:55528/0 (socket says 172.21.15.59:55528) 2023-11-24T07:57:42.020 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:42.016+0000 7f3b10b2d700 1 -- 172.21.15.59:0/742780091 learned_addr learned my addr 172.21.15.59:0/742780091 (peer_addr_for_me v1:172.21.15.59:0/0) 2023-11-24T07:57:42.020 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:42.016+0000 7f3b11b2f700 1 --2- 172.21.15.59:0/742780091 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f3b0c1623d0 0x7f3b0c0cb160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:57:42.020 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:42.016+0000 7f3b1132e700 1 --2- 172.21.15.59:0/742780091 >> v2:172.21.15.76:3300/0 conn(0x7f3b0c15e090 0x7f3b0c06afd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:57:42.020 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:42.016+0000 7f3b03fff700 1 -- 172.21.15.59:0/742780091 <== mon.0 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3801569237 0 0) 0x7f3b0c15e470 con 0x7f3b0c06b510 2023-11-24T07:57:42.020 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:42.016+0000 7f3b03fff700 1 -- 172.21.15.59:0/742780091 --> v1:172.21.15.59:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3af80035b0 con 0x7f3b0c06b510 2023-11-24T07:57:42.020 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:42.016+0000 7f3b1132e700 1 -- 172.21.15.59:0/742780091 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f3b0c1623d0 msgr2=0x7f3b0c0cb160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T07:57:42.020 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:42.016+0000 7f3b1132e700 1 --2- 172.21.15.59:0/742780091 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f3b0c1623d0 0x7f3b0c0cb160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:42.020 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:42.016+0000 7f3b1132e700 1 -- 172.21.15.59:0/742780091 >> v1:172.21.15.59:6789/0 conn(0x7f3b0c06b510 legacy=0x7f3b0c161cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:57:42.021 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:42.016+0000 7f3b1132e700 1 -- 172.21.15.59:0/742780091 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3b0c1629f0 con 0x7f3b0c15e090 2023-11-24T07:57:42.021 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:42.016+0000 7f3b11b2f700 1 --2- 172.21.15.59:0/742780091 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f3b0c1623d0 0x7f3b0c0cb160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-24T07:57:42.021 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:42.017+0000 7f3b1132e700 1 --2- 172.21.15.59:0/742780091 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3b0c15e090 0x7f3b0c06afd0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f3b08009840 tx=0x7f3b08009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4d3054d88b64a5db server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:57:42.021 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:42.017+0000 7f3b03fff700 1 -- 172.21.15.59:0/742780091 <== mon.1 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b08011830 con 0x7f3b0c15e090 2023-11-24T07:57:42.021 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:42.017+0000 7f3b03fff700 1 -- 172.21.15.59:0/742780091 <== mon.1 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3b08011990 con 0x7f3b0c15e090 2023-11-24T07:57:42.021 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:42.017+0000 7f3b03fff700 1 -- 172.21.15.59:0/742780091 <== mon.1 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b08011d20 con 0x7f3b0c15e090 2023-11-24T07:57:42.021 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:42.017+0000 7f3b13592700 1 -- 172.21.15.59:0/742780091 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3b0c15e090 msgr2=0x7f3b0c06afd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:57:42.021 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:42.017+0000 7f3b13592700 1 --2- 172.21.15.59:0/742780091 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3b0c15e090 0x7f3b0c06afd0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f3b08009840 tx=0x7f3b08009b50 comp rx=0 tx=0).stop 2023-11-24T07:57:42.021 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:42.017+0000 7f3b13592700 1 -- 172.21.15.59:0/742780091 shutdown_connections 2023-11-24T07:57:42.021 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:42.017+0000 7f3b13592700 1 --2- 172.21.15.59:0/742780091 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f3b0c1623d0 0x7f3b0c0cb160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:42.021 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:42.017+0000 7f3b13592700 1 --2- 172.21.15.59:0/742780091 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3b0c15e090 0x7f3b0c06afd0 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:42.021 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:42.017+0000 7f3b13592700 1 -- 172.21.15.59:0/742780091 >> 172.21.15.59:0/742780091 conn(0x7f3b0c0d0b10 msgr2=0x7f3b0c0d0f20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:57:42.022 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:42.017+0000 7f3b13592700 1 -- 172.21.15.59:0/742780091 shutdown_connections 2023-11-24T07:57:42.022 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:42.017+0000 7f3b13592700 1 -- 172.21.15.59:0/742780091 wait complete. 2023-11-24T07:57:42.025 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:42.019+0000 7f3b13592700 1 Processor -- start 2023-11-24T07:57:42.025 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:42.019+0000 7f3b13592700 1 -- start start 2023-11-24T07:57:42.025 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:42.019+0000 7f3b13592700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3b0c06b510 0x7f3b0c06f5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:57:42.025 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:42.019+0000 7f3b13592700 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f3b0c15e090 0x7f3b0c1471c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:57:42.025 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:42.019+0000 7f3b13592700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3b0c1623d0 0x7f3b0c145410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:57:42.025 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:42.019+0000 7f3b13592700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b0c160620 con 0x7f3b0c06b510 2023-11-24T07:57:42.025 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:42.019+0000 7f3b13592700 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b0c06fb00 con 0x7f3b0c1623d0 2023-11-24T07:57:42.025 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:42.019+0000 7f3b13592700 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3b0c15e610 con 0x7f3b0c15e090 2023-11-24T07:57:42.025 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:42.019+0000 7f3b1132e700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3b0c06b510 0x7f3b0c06f5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:57:42.025 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:42.019+0000 7f3b1132e700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3b0c06b510 0x7f3b0c06f5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:52058/0 (socket says 172.21.15.59:52058) 2023-11-24T07:57:42.025 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:42.019+0000 7f3b1132e700 1 -- 172.21.15.59:0/3777761515 learned_addr learned my addr 172.21.15.59:0/3777761515 (peer_addr_for_me v2:172.21.15.59:0/0) 2023-11-24T07:57:42.025 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:42.020+0000 7f3b1132e700 1 -- 172.21.15.59:0/3777761515 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f3b0c15e090 msgr2=0x7f3b0c1471c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:57:42.025 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:42.020+0000 7f3b1132e700 1 --2- 172.21.15.59:0/3777761515 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f3b0c15e090 0x7f3b0c1471c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:42.025 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:42.020+0000 7f3b1132e700 1 -- 172.21.15.59:0/3777761515 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3b0c1623d0 msgr2=0x7f3b0c145410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:57:42.026 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:42.020+0000 7f3b1132e700 1 --2- 172.21.15.59:0/3777761515 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3b0c1623d0 0x7f3b0c145410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:42.026 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:42.020+0000 7f3b1132e700 1 -- 172.21.15.59:0/3777761515 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3b04009a70 con 0x7f3b0c06b510 2023-11-24T07:57:42.026 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:42.020+0000 7f3b1132e700 1 --2- 172.21.15.59:0/3777761515 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3b0c06b510 0x7f3b0c06f5c0 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f3b080090b0 tx=0x7f3b08004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:57:42.026 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:42.020+0000 7f3b01ffb700 1 -- 172.21.15.59:0/3777761515 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b080036a0 con 0x7f3b0c06b510 2023-11-24T07:57:42.026 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:42.020+0000 7f3b01ffb700 1 -- 172.21.15.59:0/3777761515 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3b0801f070 con 0x7f3b0c06b510 2023-11-24T07:57:42.026 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:42.020+0000 7f3b13592700 1 -- 172.21.15.59:0/3777761515 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3b080094a0 con 0x7f3b0c06b510 2023-11-24T07:57:42.026 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:42.020+0000 7f3b01ffb700 1 -- 172.21.15.59:0/3777761515 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b08003a40 con 0x7f3b0c06b510 2023-11-24T07:57:42.026 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:42.020+0000 7f3b13592700 1 -- 172.21.15.59:0/3777761515 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3b0c1602b0 con 0x7f3b0c06b510 2023-11-24T07:57:42.026 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:42.021+0000 7f3b13592700 1 -- 172.21.15.59:0/3777761515 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3b0c15eb30 con 0x7f3b0c06b510 2023-11-24T07:57:42.026 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:42.022+0000 7f3b01ffb700 1 -- 172.21.15.59:0/3777761515 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 6) v1 ==== 89282+0+0 (secure 0 0 0) 0x7f3b08003ba0 con 0x7f3b0c06b510 2023-11-24T07:57:42.026 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:42.022+0000 7f3b01ffb700 1 --2- 172.21.15.59:0/3777761515 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f3af006af60 0x7f3af006d420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:57:42.026 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:42.022+0000 7f3b01ffb700 1 -- 172.21.15.59:0/3777761515 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f3b08021070 con 0x7f3b0c06b510 2023-11-24T07:57:42.027 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:42.022+0000 7f3b10b2d700 1 --2- 172.21.15.59:0/3777761515 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f3af006af60 0x7f3af006d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:57:42.027 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:42.023+0000 7f3b10b2d700 1 --2- 172.21.15.59:0/3777761515 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f3af006af60 0x7f3af006d420 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f3b0c0702e0 tx=0x7f3afc0079c0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:57:42.028 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:42.027+0000 7f3b01ffb700 1 -- 172.21.15.59:0/3777761515 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f3b08055f30 con 0x7f3b0c06b510 2023-11-24T07:57:42.232 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:42.230+0000 7f3b13592700 1 -- 172.21.15.59:0/3777761515 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f3b0c160440 con 0x7f3b0c06b510 2023-11-24T07:57:42.232 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:42.230+0000 7f3b01ffb700 1 -- 172.21.15.59:0/3777761515 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v33) v1 ==== 93+0+10 (secure 0 0 0) 0x7f3b08015020 con 0x7f3b0c06b510 2023-11-24T07:57:42.232 INFO:teuthology.orchestra.run.smithi059.stdout:pg_num: 1 2023-11-24T07:57:42.235 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:42.233+0000 7f3b13592700 1 -- 172.21.15.59:0/3777761515 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f3af006af60 msgr2=0x7f3af006d420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:57:42.235 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:42.233+0000 7f3b13592700 1 --2- 172.21.15.59:0/3777761515 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f3af006af60 0x7f3af006d420 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f3b0c0702e0 tx=0x7f3afc0079c0 comp rx=0 tx=0).stop 2023-11-24T07:57:42.235 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:42.233+0000 7f3b13592700 1 -- 172.21.15.59:0/3777761515 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3b0c06b510 msgr2=0x7f3b0c06f5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:57:42.235 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:42.233+0000 7f3b13592700 1 --2- 172.21.15.59:0/3777761515 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3b0c06b510 0x7f3b0c06f5c0 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f3b080090b0 tx=0x7f3b08004750 comp rx=0 tx=0).stop 2023-11-24T07:57:42.235 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:42.233+0000 7f3b13592700 1 -- 172.21.15.59:0/3777761515 shutdown_connections 2023-11-24T07:57:42.235 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:42.233+0000 7f3b13592700 1 --2- 172.21.15.59:0/3777761515 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f3af006af60 0x7f3af006d420 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:42.235 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:42.233+0000 7f3b13592700 1 --2- 172.21.15.59:0/3777761515 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3b0c1623d0 0x7f3b0c145410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:42.235 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:42.233+0000 7f3b13592700 1 --2- 172.21.15.59:0/3777761515 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3b0c06b510 0x7f3b0c06f5c0 unknown :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:42.235 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:42.233+0000 7f3b13592700 1 --2- 172.21.15.59:0/3777761515 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f3b0c15e090 0x7f3b0c1471c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:42.235 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:42.233+0000 7f3b13592700 1 -- 172.21.15.59:0/3777761515 >> 172.21.15.59:0/3777761515 conn(0x7f3b0c0d0b10 msgr2=0x7f3b0c161940 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:57:42.236 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:42.233+0000 7f3b13592700 1 -- 172.21.15.59:0/3777761515 shutdown_connections 2023-11-24T07:57:42.236 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:42.233+0000 7f3b13592700 1 -- 172.21.15.59:0/3777761515 wait complete. 2023-11-24T07:57:42.247 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-24T07:57:42.374 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:42.373+0000 7fe207aca700 1 Processor -- start 2023-11-24T07:57:42.375 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:42.373+0000 7fe207aca700 1 -- start start 2023-11-24T07:57:42.377 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:42.374+0000 7fe207aca700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fe2000b4090 0x7fe20015e510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:57:42.377 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:42.374+0000 7fe207aca700 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fe2000d1010 0x7fe2000d1420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:57:42.377 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:42.374+0000 7fe207aca700 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe2000d0b80 con 0x7fe200072a00 2023-11-24T07:57:42.377 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:42.374+0000 7fe207aca700 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe2000d1960 con 0x7fe2000b4090 2023-11-24T07:57:42.377 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:42.374+0000 7fe207aca700 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe20015ed60 con 0x7fe2000d1010 2023-11-24T07:57:42.377 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:42.374+0000 7fe205866700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fe2000b4090 0x7fe20015e510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:57:42.377 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:42.374+0000 7fe205866700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fe2000b4090 0x7fe20015e510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:52066/0 (socket says 172.21.15.59:52066) 2023-11-24T07:57:42.377 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:42.374+0000 7fe205866700 1 -- 172.21.15.59:0/4058925022 learned_addr learned my addr 172.21.15.59:0/4058925022 (peer_addr_for_me v2:172.21.15.59:0/0) 2023-11-24T07:57:42.378 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:42.374+0000 7fe206067700 1 --2- 172.21.15.59:0/4058925022 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fe2000d1010 0x7fe2000d1420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:57:42.378 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:42.374+0000 7fe205866700 1 -- 172.21.15.59:0/4058925022 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fe2000d1010 msgr2=0x7fe2000d1420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T07:57:42.378 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:42.374+0000 7fe205866700 1 --2- 172.21.15.59:0/4058925022 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fe2000d1010 0x7fe2000d1420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:42.378 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:42.374+0000 7fe205866700 1 -- 172.21.15.59:0/4058925022 >> v1:172.21.15.76:6789/0 conn(0x7fe200072a00 legacy=0x7fe20006db60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:57:42.378 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:42.374+0000 7fe205866700 1 -- 172.21.15.59:0/4058925022 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe20015f040 con 0x7fe2000b4090 2023-11-24T07:57:42.378 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:42.374+0000 7fe205866700 1 --2- 172.21.15.59:0/4058925022 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe2000b4090 0x7fe20015e510 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7fe1fc0098f0 tx=0x7fe1fc009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=acd833a898f0e6b1 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:57:42.378 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:42.375+0000 7fe204864700 1 -- 172.21.15.59:0/4058925022 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1fc00b8a0 con 0x7fe2000b4090 2023-11-24T07:57:42.378 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:42.375+0000 7fe204864700 1 -- 172.21.15.59:0/4058925022 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe1fc00ba00 con 0x7fe2000b4090 2023-11-24T07:57:42.378 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:42.375+0000 7fe204864700 1 -- 172.21.15.59:0/4058925022 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1fc00bd90 con 0x7fe2000b4090 2023-11-24T07:57:42.378 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:42.375+0000 7fe207aca700 1 -- 172.21.15.59:0/4058925022 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe2000b4090 msgr2=0x7fe20015e510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:57:42.378 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:42.375+0000 7fe207aca700 1 --2- 172.21.15.59:0/4058925022 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe2000b4090 0x7fe20015e510 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7fe1fc0098f0 tx=0x7fe1fc009c00 comp rx=0 tx=0).stop 2023-11-24T07:57:42.378 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:42.375+0000 7fe207aca700 1 -- 172.21.15.59:0/4058925022 shutdown_connections 2023-11-24T07:57:42.378 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:42.375+0000 7fe207aca700 1 --2- 172.21.15.59:0/4058925022 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fe2000d1010 0x7fe2000d1420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:42.379 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:42.375+0000 7fe207aca700 1 --2- 172.21.15.59:0/4058925022 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe2000b4090 0x7fe20015e510 unknown :-1 s=CLOSED pgs=431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:42.379 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:42.375+0000 7fe207aca700 1 -- 172.21.15.59:0/4058925022 >> 172.21.15.59:0/4058925022 conn(0x7fe2000b2740 msgr2=0x7fe2000b2b50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:57:42.379 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:42.375+0000 7fe207aca700 1 -- 172.21.15.59:0/4058925022 shutdown_connections 2023-11-24T07:57:42.379 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:42.375+0000 7fe207aca700 1 -- 172.21.15.59:0/4058925022 wait complete. 2023-11-24T07:57:42.381 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:42.376+0000 7fe207aca700 1 Processor -- start 2023-11-24T07:57:42.381 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:42.376+0000 7fe207aca700 1 -- start start 2023-11-24T07:57:42.382 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:42.377+0000 7fe207aca700 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fe200072a00 0x7fe2000c59b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:57:42.382 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:42.377+0000 7fe207aca700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe2000b4090 0x7fe20014d630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:57:42.382 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:42.377+0000 7fe207aca700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe2000d1010 0x7fe2001f4680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:57:42.382 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:42.377+0000 7fe207aca700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe20014db70 con 0x7fe2000b4090 2023-11-24T07:57:42.382 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:42.377+0000 7fe207aca700 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2001f4bc0 con 0x7fe2000d1010 2023-11-24T07:57:42.382 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:42.377+0000 7fe207aca700 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe2000c6040 con 0x7fe200072a00 2023-11-24T07:57:42.382 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:42.377+0000 7fe205065700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe2000b4090 0x7fe20014d630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:57:42.382 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:42.377+0000 7fe205065700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe2000b4090 0x7fe20014d630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:52074/0 (socket says 172.21.15.59:52074) 2023-11-24T07:57:42.382 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:42.377+0000 7fe205065700 1 -- 172.21.15.59:0/4228287402 learned_addr learned my addr 172.21.15.59:0/4228287402 (peer_addr_for_me v2:172.21.15.59:0/0) 2023-11-24T07:57:42.382 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:42.378+0000 7fe206067700 1 --2- 172.21.15.59:0/4228287402 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe2000d1010 0x7fe2001f4680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:57:42.382 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:42.378+0000 7fe205866700 1 --2- 172.21.15.59:0/4228287402 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fe200072a00 0x7fe2000c59b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:57:42.383 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:42.378+0000 7fe205065700 1 -- 172.21.15.59:0/4228287402 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fe200072a00 msgr2=0x7fe2000c59b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:57:42.383 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:42.378+0000 7fe205065700 1 --2- 172.21.15.59:0/4228287402 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fe200072a00 0x7fe2000c59b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:42.383 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:42.378+0000 7fe205065700 1 -- 172.21.15.59:0/4228287402 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe2000d1010 msgr2=0x7fe2001f4680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:57:42.383 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:42.378+0000 7fe205065700 1 --2- 172.21.15.59:0/4228287402 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe2000d1010 0x7fe2001f4680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:42.383 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:42.378+0000 7fe205065700 1 -- 172.21.15.59:0/4228287402 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe1fc009550 con 0x7fe2000b4090 2023-11-24T07:57:42.383 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:42.378+0000 7fe205866700 1 --2- 172.21.15.59:0/4228287402 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fe200072a00 0x7fe2000c59b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T07:57:42.383 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:42.378+0000 7fe206067700 1 --2- 172.21.15.59:0/4228287402 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe2000d1010 0x7fe2001f4680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T07:57:42.383 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:42.378+0000 7fe205065700 1 --2- 172.21.15.59:0/4228287402 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe2000b4090 0x7fe20014d630 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7fe1f000c4e0 tx=0x7fe1f000c8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:57:42.383 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:42.378+0000 7fe1f67fc700 1 -- 172.21.15.59:0/4228287402 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1f00052d0 con 0x7fe2000b4090 2023-11-24T07:57:42.383 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:42.378+0000 7fe1f67fc700 1 -- 172.21.15.59:0/4228287402 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe1f0005430 con 0x7fe2000b4090 2023-11-24T07:57:42.383 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:42.378+0000 7fe207aca700 1 -- 172.21.15.59:0/4228287402 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe2000ae950 con 0x7fe2000b4090 2023-11-24T07:57:42.384 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:42.378+0000 7fe1f67fc700 1 -- 172.21.15.59:0/4228287402 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1f00038c0 con 0x7fe2000b4090 2023-11-24T07:57:42.384 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:42.378+0000 7fe207aca700 1 -- 172.21.15.59:0/4228287402 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe2000aef10 con 0x7fe2000b4090 2023-11-24T07:57:42.384 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:42.379+0000 7fe207aca700 1 -- 172.21.15.59:0/4228287402 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe200075480 con 0x7fe2000b4090 2023-11-24T07:57:42.384 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:42.379+0000 7fe1f67fc700 1 -- 172.21.15.59:0/4228287402 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 6) v1 ==== 89282+0+0 (secure 0 0 0) 0x7fe1f0003a20 con 0x7fe2000b4090 2023-11-24T07:57:42.384 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:42.380+0000 7fe1f67fc700 1 --2- 172.21.15.59:0/4228287402 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7fe1e406af60 0x7fe1e406d420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:57:42.384 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:42.380+0000 7fe1f67fc700 1 -- 172.21.15.59:0/4228287402 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fe1f0088b50 con 0x7fe2000b4090 2023-11-24T07:57:42.384 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:42.380+0000 7fe205866700 1 --2- 172.21.15.59:0/4228287402 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7fe1e406af60 0x7fe1e406d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:57:42.384 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:42.380+0000 7fe205866700 1 --2- 172.21.15.59:0/4228287402 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7fe1e406af60 0x7fe1e406d420 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fe2000b4bb0 tx=0x7fe1fc008030 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:57:42.385 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:42.384+0000 7fe1f67fc700 1 -- 172.21.15.59:0/4228287402 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fe1f0053ba0 con 0x7fe2000b4090 2023-11-24T07:57:42.585 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:42.583+0000 7fe207aca700 1 -- 172.21.15.59:0/4228287402 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fe200072650 con 0x7fe2000b4090 2023-11-24T07:57:42.585 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:42.583+0000 7fe1f67fc700 1 -- 172.21.15.59:0/4228287402 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v33) v1 ==== 104+0+11 (secure 0 0 0) 0x7fe1f0017020 con 0x7fe2000b4090 2023-11-24T07:57:42.585 INFO:teuthology.orchestra.run.smithi059.stdout:pg_num: 32 2023-11-24T07:57:42.587 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:42.585+0000 7fe207aca700 1 -- 172.21.15.59:0/4228287402 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7fe1e406af60 msgr2=0x7fe1e406d420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:57:42.587 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:42.585+0000 7fe207aca700 1 --2- 172.21.15.59:0/4228287402 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7fe1e406af60 0x7fe1e406d420 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fe2000b4bb0 tx=0x7fe1fc008030 comp rx=0 tx=0).stop 2023-11-24T07:57:42.587 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:42.585+0000 7fe207aca700 1 -- 172.21.15.59:0/4228287402 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe2000b4090 msgr2=0x7fe20014d630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:57:42.588 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:42.585+0000 7fe207aca700 1 --2- 172.21.15.59:0/4228287402 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe2000b4090 0x7fe20014d630 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7fe1f000c4e0 tx=0x7fe1f000c8a0 comp rx=0 tx=0).stop 2023-11-24T07:57:42.588 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:42.585+0000 7fe207aca700 1 -- 172.21.15.59:0/4228287402 shutdown_connections 2023-11-24T07:57:42.588 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:42.585+0000 7fe207aca700 1 --2- 172.21.15.59:0/4228287402 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7fe1e406af60 0x7fe1e406d420 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:42.588 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:42.585+0000 7fe207aca700 1 --2- 172.21.15.59:0/4228287402 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe2000d1010 0x7fe2001f4680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:42.588 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:42.585+0000 7fe207aca700 1 --2- 172.21.15.59:0/4228287402 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fe200072a00 0x7fe2000c59b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:42.588 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:42.585+0000 7fe207aca700 1 --2- 172.21.15.59:0/4228287402 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe2000b4090 0x7fe20014d630 unknown :-1 s=CLOSED pgs=432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:42.588 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:42.585+0000 7fe207aca700 1 -- 172.21.15.59:0/4228287402 >> 172.21.15.59:0/4228287402 conn(0x7fe2000b2740 msgr2=0x7fe200160f10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:57:42.588 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:42.585+0000 7fe207aca700 1 -- 172.21.15.59:0/4228287402 shutdown_connections 2023-11-24T07:57:42.588 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:42.585+0000 7fe207aca700 1 -- 172.21.15.59:0/4228287402 wait complete. 2023-11-24T07:57:42.599 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-24T07:57:42.728 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:42.726+0000 7fdb8c8fe700 1 Processor -- start 2023-11-24T07:57:42.728 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:42.726+0000 7fdb8c8fe700 1 -- start start 2023-11-24T07:57:42.730 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:42.727+0000 7fdb8c8fe700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fdb880b4090 0x7fdb8815e510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:57:42.730 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:42.727+0000 7fdb8c8fe700 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdb880d0b80 con 0x7fdb88072a00 2023-11-24T07:57:42.730 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:42.727+0000 7fdb8c8fe700 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdb8815ed60 con 0x7fdb880d1010 2023-11-24T07:57:42.730 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:42.727+0000 7fdb8c8fe700 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdb8815f040 con 0x7fdb880b4090 2023-11-24T07:57:42.730 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:42.727+0000 7fdb8659c700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fdb880b4090 0x7fdb8815e510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:57:42.730 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:42.727+0000 7fdb8659c700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fdb880b4090 0x7fdb8815e510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:52080/0 (socket says 172.21.15.59:52080) 2023-11-24T07:57:42.731 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:42.727+0000 7fdb8659c700 1 -- 172.21.15.59:0/1829460946 learned_addr learned my addr 172.21.15.59:0/1829460946 (peer_addr_for_me v2:172.21.15.59:0/0) 2023-11-24T07:57:42.731 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:42.727+0000 7fdb85d9b700 1 --1- 172.21.15.59:0/1829460946 >> v1:172.21.15.59:6789/0 conn(0x7fdb88072a00 0x7fdb8806db60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:55556/0 (socket says 172.21.15.59:55556) 2023-11-24T07:57:42.731 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:42.727+0000 7fdb8659c700 1 -- 172.21.15.59:0/1829460946 >> v1:172.21.15.76:6789/0 conn(0x7fdb880d1010 legacy=0x7fdb8806afd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T07:57:42.731 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:42.727+0000 7fdb8659c700 1 -- 172.21.15.59:0/1829460946 >> v1:172.21.15.59:6789/0 conn(0x7fdb88072a00 legacy=0x7fdb8806db60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:57:42.731 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:42.728+0000 7fdb8559a700 1 -- 172.21.15.59:0/1829460946 <== mon.0 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3333665100 0 0) 0x7fdb880d0b80 con 0x7fdb88072a00 2023-11-24T07:57:42.731 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:42.728+0000 7fdb8659c700 1 -- 172.21.15.59:0/1829460946 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdb8815f320 con 0x7fdb880b4090 2023-11-24T07:57:42.731 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:42.728+0000 7fdb8659c700 1 --2- 172.21.15.59:0/1829460946 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fdb880b4090 0x7fdb8815e510 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7fdb7c0098f0 tx=0x7fdb7c009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c1ceb233caf334ac server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:57:42.731 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:42.728+0000 7fdb8559a700 1 -- 172.21.15.59:0/1829460946 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb7c004230 con 0x7fdb880b4090 2023-11-24T07:57:42.731 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:42.728+0000 7fdb8559a700 1 -- 172.21.15.59:0/1829460946 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdb7c004390 con 0x7fdb880b4090 2023-11-24T07:57:42.731 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:42.728+0000 7fdb8559a700 1 -- 172.21.15.59:0/1829460946 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb7c0036a0 con 0x7fdb880b4090 2023-11-24T07:57:42.731 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:42.728+0000 7fdb8c8fe700 1 -- 172.21.15.59:0/1829460946 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fdb880b4090 msgr2=0x7fdb8815e510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:57:42.731 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:42.728+0000 7fdb8c8fe700 1 --2- 172.21.15.59:0/1829460946 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fdb880b4090 0x7fdb8815e510 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7fdb7c0098f0 tx=0x7fdb7c009c00 comp rx=0 tx=0).stop 2023-11-24T07:57:42.732 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:42.728+0000 7fdb8c8fe700 1 -- 172.21.15.59:0/1829460946 shutdown_connections 2023-11-24T07:57:42.732 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:42.728+0000 7fdb8c8fe700 1 --2- 172.21.15.59:0/1829460946 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fdb880b4090 0x7fdb8815e510 unknown :-1 s=CLOSED pgs=434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:42.732 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:42.728+0000 7fdb8c8fe700 1 -- 172.21.15.59:0/1829460946 >> 172.21.15.59:0/1829460946 conn(0x7fdb880b2740 msgr2=0x7fdb880b2b50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:57:42.732 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:42.729+0000 7fdb8c8fe700 1 -- 172.21.15.59:0/1829460946 shutdown_connections 2023-11-24T07:57:42.732 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:42.729+0000 7fdb8c8fe700 1 -- 172.21.15.59:0/1829460946 wait complete. 2023-11-24T07:57:42.734 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:42.730+0000 7fdb8c8fe700 1 Processor -- start 2023-11-24T07:57:42.734 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:42.730+0000 7fdb8c8fe700 1 -- start start 2023-11-24T07:57:42.734 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:42.731+0000 7fdb8c8fe700 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fdb88072a00 0x7fdb880af420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:57:42.734 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:42.731+0000 7fdb8c8fe700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fdb880b4090 0x7fdb88146190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:57:42.734 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:42.731+0000 7fdb8c8fe700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fdb880d1010 0x7fdb881466d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:57:42.734 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:42.731+0000 7fdb8c8fe700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb88146de0 con 0x7fdb880d1010 2023-11-24T07:57:42.735 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:42.731+0000 7fdb8c8fe700 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb88149c80 con 0x7fdb880b4090 2023-11-24T07:57:42.735 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:42.731+0000 7fdb8c8fe700 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdb88147080 con 0x7fdb88072a00 2023-11-24T07:57:42.735 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:42.731+0000 7fdb86d9d700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fdb880d1010 0x7fdb881466d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:57:42.735 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:42.731+0000 7fdb86d9d700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fdb880d1010 0x7fdb881466d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:52090/0 (socket says 172.21.15.59:52090) 2023-11-24T07:57:42.735 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:42.731+0000 7fdb86d9d700 1 -- 172.21.15.59:0/4272433885 learned_addr learned my addr 172.21.15.59:0/4272433885 (peer_addr_for_me v2:172.21.15.59:0/0) 2023-11-24T07:57:42.735 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:42.731+0000 7fdb8659c700 1 --2- 172.21.15.59:0/4272433885 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fdb88072a00 0x7fdb880af420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:57:42.735 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:42.731+0000 7fdb85d9b700 1 --2- 172.21.15.59:0/4272433885 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fdb880b4090 0x7fdb88146190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:57:42.735 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:42.731+0000 7fdb86d9d700 1 -- 172.21.15.59:0/4272433885 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fdb88072a00 msgr2=0x7fdb880af420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:57:42.735 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:42.731+0000 7fdb86d9d700 1 --2- 172.21.15.59:0/4272433885 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fdb88072a00 0x7fdb880af420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:42.735 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:42.731+0000 7fdb86d9d700 1 -- 172.21.15.59:0/4272433885 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fdb880b4090 msgr2=0x7fdb88146190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:57:42.735 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:42.731+0000 7fdb86d9d700 1 --2- 172.21.15.59:0/4272433885 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fdb880b4090 0x7fdb88146190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:42.735 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:42.731+0000 7fdb86d9d700 1 -- 172.21.15.59:0/4272433885 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdb7c009550 con 0x7fdb880d1010 2023-11-24T07:57:42.735 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:42.731+0000 7fdb86d9d700 1 --2- 172.21.15.59:0/4272433885 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fdb880d1010 0x7fdb881466d0 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7fdb8000b970 tx=0x7fdb8000bc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:57:42.736 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:42.732+0000 7fdb76ffd700 1 -- 172.21.15.59:0/4272433885 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb8000d860 con 0x7fdb880d1010 2023-11-24T07:57:42.736 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:42.732+0000 7fdb76ffd700 1 -- 172.21.15.59:0/4272433885 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdb8000d9c0 con 0x7fdb880d1010 2023-11-24T07:57:42.736 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:42.732+0000 7fdb8c8fe700 1 -- 172.21.15.59:0/4272433885 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdb8806e3f0 con 0x7fdb880d1010 2023-11-24T07:57:42.736 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:42.732+0000 7fdb76ffd700 1 -- 172.21.15.59:0/4272433885 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb8000dd70 con 0x7fdb880d1010 2023-11-24T07:57:42.736 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:42.732+0000 7fdb8c8fe700 1 -- 172.21.15.59:0/4272433885 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdb88147640 con 0x7fdb880d1010 2023-11-24T07:57:42.736 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:42.733+0000 7fdb8c8fe700 1 -- 172.21.15.59:0/4272433885 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdb881477d0 con 0x7fdb880d1010 2023-11-24T07:57:42.737 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:42.733+0000 7fdb76ffd700 1 -- 172.21.15.59:0/4272433885 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 6) v1 ==== 89282+0+0 (secure 0 0 0) 0x7fdb8000e430 con 0x7fdb880d1010 2023-11-24T07:57:42.737 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:42.734+0000 7fdb76ffd700 1 --2- 172.21.15.59:0/4272433885 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7fdb6806af60 0x7fdb6806d420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:57:42.737 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:42.734+0000 7fdb76ffd700 1 -- 172.21.15.59:0/4272433885 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fdb80089760 con 0x7fdb880d1010 2023-11-24T07:57:42.737 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:42.734+0000 7fdb8659c700 1 --2- 172.21.15.59:0/4272433885 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7fdb6806af60 0x7fdb6806d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:57:42.737 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:42.734+0000 7fdb8659c700 1 --2- 172.21.15.59:0/4272433885 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7fdb6806af60 0x7fdb6806d420 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7fdb7c009130 tx=0x7fdb7c008030 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:57:42.740 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:42.738+0000 7fdb76ffd700 1 -- 172.21.15.59:0/4272433885 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fdb80054850 con 0x7fdb880d1010 2023-11-24T07:57:42.946 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:42.944+0000 7fdb8c8fe700 1 -- 172.21.15.59:0/4272433885 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fdb88075480 con 0x7fdb880d1010 2023-11-24T07:57:42.946 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:42.944+0000 7fdb76ffd700 1 -- 172.21.15.59:0/4272433885 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v33) v1 ==== 100+0+11 (secure 0 0 0) 0x7fdb8000ded0 con 0x7fdb880d1010 2023-11-24T07:57:42.946 INFO:teuthology.orchestra.run.smithi059.stdout:pg_num: 64 2023-11-24T07:57:42.949 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:42.946+0000 7fdb8c8fe700 1 -- 172.21.15.59:0/4272433885 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7fdb6806af60 msgr2=0x7fdb6806d420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:57:42.949 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:42.946+0000 7fdb8c8fe700 1 --2- 172.21.15.59:0/4272433885 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7fdb6806af60 0x7fdb6806d420 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7fdb7c009130 tx=0x7fdb7c008030 comp rx=0 tx=0).stop 2023-11-24T07:57:42.949 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:42.946+0000 7fdb8c8fe700 1 -- 172.21.15.59:0/4272433885 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fdb880d1010 msgr2=0x7fdb881466d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:57:42.949 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:42.946+0000 7fdb8c8fe700 1 --2- 172.21.15.59:0/4272433885 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fdb880d1010 0x7fdb881466d0 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7fdb8000b970 tx=0x7fdb8000bc80 comp rx=0 tx=0).stop 2023-11-24T07:57:42.949 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:42.946+0000 7fdb8c8fe700 1 -- 172.21.15.59:0/4272433885 shutdown_connections 2023-11-24T07:57:42.949 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:42.946+0000 7fdb8c8fe700 1 --2- 172.21.15.59:0/4272433885 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7fdb6806af60 0x7fdb6806d420 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:42.949 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:42.946+0000 7fdb8c8fe700 1 --2- 172.21.15.59:0/4272433885 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fdb880d1010 0x7fdb881466d0 unknown :-1 s=CLOSED pgs=435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:42.949 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:42.947+0000 7fdb8c8fe700 1 --2- 172.21.15.59:0/4272433885 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fdb88072a00 0x7fdb880af420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:42.949 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:42.947+0000 7fdb8c8fe700 1 --2- 172.21.15.59:0/4272433885 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fdb880b4090 0x7fdb88146190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:42.949 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:42.947+0000 7fdb8c8fe700 1 -- 172.21.15.59:0/4272433885 >> 172.21.15.59:0/4272433885 conn(0x7fdb880b2740 msgr2=0x7fdb881607e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:57:42.949 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:42.947+0000 7fdb8c8fe700 1 -- 172.21.15.59:0/4272433885 shutdown_connections 2023-11-24T07:57:42.949 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:42.947+0000 7fdb8c8fe700 1 -- 172.21.15.59:0/4272433885 wait complete. 2023-11-24T07:57:42.961 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-11-24T07:57:43.086 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:43.084+0000 7f5da2c12700 1 Processor -- start 2023-11-24T07:57:43.086 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:43.085+0000 7f5da2c12700 1 -- start start 2023-11-24T07:57:43.090 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:43.085+0000 7f5da2c12700 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f5d9c15bde0 0x7f5d9c15c1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:57:43.090 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:43.085+0000 7f5da2c12700 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5d9c0d0a90 con 0x7f5d9c0cf2b0 2023-11-24T07:57:43.090 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:43.085+0000 7f5da2c12700 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5d9c06ee10 con 0x7f5d9c0ce5c0 2023-11-24T07:57:43.090 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:43.085+0000 7f5da2c12700 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5d9c06f0f0 con 0x7f5d9c15bde0 2023-11-24T07:57:43.090 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:43.085+0000 7f5d9bfff700 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7f5d9c0cf2b0 0x7f5d9c0d01b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:55570/0 (socket says 172.21.15.59:55570) 2023-11-24T07:57:43.090 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:43.085+0000 7f5d9bfff700 1 -- 172.21.15.59:0/1474176671 learned_addr learned my addr 172.21.15.59:0/1474176671 (peer_addr_for_me v1:172.21.15.59:0/0) 2023-11-24T07:57:43.090 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:43.086+0000 7f5da09ae700 1 --2- 172.21.15.59:0/1474176671 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f5d9c15bde0 0x7f5d9c15c1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:57:43.091 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:43.086+0000 7f5d9b7fe700 1 -- 172.21.15.59:0/1474176671 <== mon.0 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1769965157 0 0) 0x7f5d9c0d0a90 con 0x7f5d9c0cf2b0 2023-11-24T07:57:43.091 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:43.086+0000 7f5d9b7fe700 1 -- 172.21.15.59:0/1474176671 --> v1:172.21.15.59:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5d8c0035b0 con 0x7f5d9c0cf2b0 2023-11-24T07:57:43.091 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:43.086+0000 7f5d9b7fe700 1 -- 172.21.15.59:0/1474176671 <== mon.0 v1:172.21.15.59:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 956228449 0 0) 0x7f5d8c0035b0 con 0x7f5d9c0cf2b0 2023-11-24T07:57:43.091 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:43.086+0000 7f5d9b7fe700 1 -- 172.21.15.59:0/1474176671 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f5d9c15bde0 msgr2=0x7f5d9c15c1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T07:57:43.091 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:43.086+0000 7f5d9b7fe700 1 --2- 172.21.15.59:0/1474176671 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f5d9c15bde0 0x7f5d9c15c1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:43.091 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:43.086+0000 7f5d9b7fe700 1 -- 172.21.15.59:0/1474176671 >> v1:172.21.15.76:6789/0 conn(0x7f5d9c0ce5c0 legacy=0x7f5d9c06fbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:57:43.091 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:43.086+0000 7f5d9b7fe700 1 -- 172.21.15.59:0/1474176671 --> v1:172.21.15.59:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5d9c06f3d0 con 0x7f5d9c0cf2b0 2023-11-24T07:57:43.091 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:43.086+0000 7f5d9b7fe700 1 -- 172.21.15.59:0/1474176671 <== mon.0 v1:172.21.15.59:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1312333566 0 0) 0x7f5d88001670 con 0x7f5d9c0cf2b0 2023-11-24T07:57:43.091 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:43.086+0000 7f5d9b7fe700 1 -- 172.21.15.59:0/1474176671 >> v1:172.21.15.59:6789/0 conn(0x7f5d9c0cf2b0 legacy=0x7f5d9c0d01b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:57:43.091 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:43.086+0000 7f5d9b7fe700 1 --2- 172.21.15.59:0/1474176671 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5d8c003a30 0x7f5d8c003e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:57:43.092 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:43.086+0000 7f5d9b7fe700 1 --2- 172.21.15.59:0/1474176671 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f5d9c15bde0 0x7f5d8c004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:57:43.092 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:43.086+0000 7f5d9b7fe700 1 --2- 172.21.15.59:0/1474176671 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5d8c004d90 0x7f5d8c005210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:57:43.092 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:43.086+0000 7f5d9b7fe700 1 -- 172.21.15.59:0/1474176671 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d9c06f0f0 con 0x7f5d8c003a30 2023-11-24T07:57:43.092 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:43.086+0000 7f5d9b7fe700 1 -- 172.21.15.59:0/1474176671 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d8c00bb40 con 0x7f5d8c004d90 2023-11-24T07:57:43.092 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:43.086+0000 7f5d9b7fe700 1 -- 172.21.15.59:0/1474176671 --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5d8c00bd00 con 0x7f5d9c15bde0 2023-11-24T07:57:43.092 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:43.086+0000 7f5da09ae700 1 --2- 172.21.15.59:0/1474176671 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5d8c003a30 0x7f5d8c003e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:57:43.093 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:43.086+0000 7f5d9bfff700 1 --2- 172.21.15.59:0/1474176671 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f5d9c15bde0 0x7f5d8c004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:57:43.093 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:43.086+0000 7f5da11af700 1 --2- 172.21.15.59:0/1474176671 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5d8c004d90 0x7f5d8c005210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:57:43.093 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:43.087+0000 7f5da09ae700 1 -- 172.21.15.59:0/1474176671 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f5d9c15bde0 msgr2=0x7f5d8c004850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T07:57:43.093 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:43.087+0000 7f5da09ae700 1 --2- 172.21.15.59:0/1474176671 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f5d9c15bde0 0x7f5d8c004850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:43.093 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:43.087+0000 7f5da09ae700 1 -- 172.21.15.59:0/1474176671 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5d8c004d90 msgr2=0x7f5d8c005210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T07:57:43.093 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:43.087+0000 7f5da09ae700 1 --2- 172.21.15.59:0/1474176671 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5d8c004d90 0x7f5d8c005210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:43.093 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:43.087+0000 7f5da09ae700 1 -- 172.21.15.59:0/1474176671 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5d8c0035b0 con 0x7f5d8c003a30 2023-11-24T07:57:43.094 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:43.087+0000 7f5d9bfff700 1 --2- 172.21.15.59:0/1474176671 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f5d9c15bde0 0x7f5d8c004850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-24T07:57:43.094 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:43.087+0000 7f5da09ae700 1 --2- 172.21.15.59:0/1474176671 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5d8c003a30 0x7f5d8c003e50 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f5d90009b60 tx=0x7f5d8c005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8b94907f5ee55e13 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:57:43.094 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:43.087+0000 7f5d9b7fe700 1 -- 172.21.15.59:0/1474176671 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d9000e6c0 con 0x7f5d8c003a30 2023-11-24T07:57:43.094 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:43.087+0000 7f5d9b7fe700 1 -- 172.21.15.59:0/1474176671 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5d9000e820 con 0x7f5d8c003a30 2023-11-24T07:57:43.094 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:43.087+0000 7f5d9b7fe700 1 -- 172.21.15.59:0/1474176671 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d9000eb90 con 0x7f5d8c003a30 2023-11-24T07:57:43.094 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:43.087+0000 7f5da2c12700 1 -- 172.21.15.59:0/1474176671 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5d8c003a30 msgr2=0x7f5d8c003e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:57:43.095 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:43.087+0000 7f5da2c12700 1 --2- 172.21.15.59:0/1474176671 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5d8c003a30 0x7f5d8c003e50 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f5d90009b60 tx=0x7f5d8c005880 comp rx=0 tx=0).stop 2023-11-24T07:57:43.095 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:43.087+0000 7f5da11af700 1 -- 172.21.15.59:0/1474176671 reap_dead start 2023-11-24T07:57:43.095 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:43.087+0000 7f5da2c12700 1 -- 172.21.15.59:0/1474176671 shutdown_connections 2023-11-24T07:57:43.095 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:43.087+0000 7f5da2c12700 1 -- 172.21.15.59:0/1474176671 >> 172.21.15.59:0/1474176671 conn(0x7f5d9c0b3720 msgr2=0x7f5d9c0b3b30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:57:43.095 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:43.087+0000 7f5da2c12700 1 -- 172.21.15.59:0/1474176671 shutdown_connections 2023-11-24T07:57:43.095 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:43.088+0000 7f5da2c12700 1 -- 172.21.15.59:0/1474176671 wait complete. 2023-11-24T07:57:43.098 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:43.089+0000 7f5da2c12700 1 Processor -- start 2023-11-24T07:57:43.098 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:43.089+0000 7f5da2c12700 1 -- start start 2023-11-24T07:57:43.099 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:43.090+0000 7f5da2c12700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5d8c003a30 0x7f5d9c157ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:57:43.099 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:43.090+0000 7f5da2c12700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5d9c0b7510 0x7f5d9c0c7730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:57:43.099 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:43.090+0000 7f5da2c12700 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f5d9c156620 0x7f5d9c0cadd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:57:43.099 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:43.090+0000 7f5da2c12700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d9c158530 con 0x7f5d9c0b7510 2023-11-24T07:57:43.099 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:43.090+0000 7f5da2c12700 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d9c156f50 con 0x7f5d8c003a30 2023-11-24T07:57:43.099 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:43.090+0000 7f5da2c12700 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5d9c157230 con 0x7f5d9c156620 2023-11-24T07:57:43.099 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:43.090+0000 7f5d9bfff700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5d9c0b7510 0x7f5d9c0c7730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:57:43.099 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:43.090+0000 7f5d9bfff700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5d9c0b7510 0x7f5d9c0c7730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:52108/0 (socket says 172.21.15.59:52108) 2023-11-24T07:57:43.100 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:43.090+0000 7f5d9bfff700 1 -- 172.21.15.59:0/2425103177 learned_addr learned my addr 172.21.15.59:0/2425103177 (peer_addr_for_me v2:172.21.15.59:0/0) 2023-11-24T07:57:43.100 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:43.091+0000 7f5da11af700 1 --2- 172.21.15.59:0/2425103177 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f5d9c156620 0x7f5d9c0cadd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:57:43.100 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:43.091+0000 7f5da09ae700 1 --2- 172.21.15.59:0/2425103177 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5d8c003a30 0x7f5d9c157ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:57:43.100 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:43.091+0000 7f5d9bfff700 1 -- 172.21.15.59:0/2425103177 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f5d9c156620 msgr2=0x7f5d9c0cadd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:57:43.100 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:43.091+0000 7f5d9bfff700 1 --2- 172.21.15.59:0/2425103177 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f5d9c156620 0x7f5d9c0cadd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:43.100 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:43.091+0000 7f5d9bfff700 1 -- 172.21.15.59:0/2425103177 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5d8c003a30 msgr2=0x7f5d9c157ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:57:43.100 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:43.091+0000 7f5d9bfff700 1 --2- 172.21.15.59:0/2425103177 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5d8c003a30 0x7f5d9c157ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:43.101 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:43.091+0000 7f5d9bfff700 1 -- 172.21.15.59:0/2425103177 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5d9c157510 con 0x7f5d9c0b7510 2023-11-24T07:57:43.101 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:43.091+0000 7f5da09ae700 1 --2- 172.21.15.59:0/2425103177 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5d8c003a30 0x7f5d9c157ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T07:57:43.101 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:43.091+0000 7f5d9bfff700 1 --2- 172.21.15.59:0/2425103177 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5d9c0b7510 0x7f5d9c0c7730 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f5d8800e8f0 tx=0x7f5d8800ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:57:43.101 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:43.091+0000 7f5d997fa700 1 -- 172.21.15.59:0/2425103177 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d8800c780 con 0x7f5d9c0b7510 2023-11-24T07:57:43.101 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:43.091+0000 7f5d997fa700 1 -- 172.21.15.59:0/2425103177 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5d8800c8e0 con 0x7f5d9c0b7510 2023-11-24T07:57:43.101 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:43.091+0000 7f5da2c12700 1 -- 172.21.15.59:0/2425103177 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5d9c1577f0 con 0x7f5d9c0b7510 2023-11-24T07:57:43.102 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:43.091+0000 7f5d997fa700 1 -- 172.21.15.59:0/2425103177 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d8800cc70 con 0x7f5d9c0b7510 2023-11-24T07:57:43.102 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:43.091+0000 7f5da2c12700 1 -- 172.21.15.59:0/2425103177 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5d9c0c7c70 con 0x7f5d9c0b7510 2023-11-24T07:57:43.102 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:43.092+0000 7f5da2c12700 1 -- 172.21.15.59:0/2425103177 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5d9c068670 con 0x7f5d9c0b7510 2023-11-24T07:57:43.102 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:43.093+0000 7f5d997fa700 1 -- 172.21.15.59:0/2425103177 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 6) v1 ==== 89282+0+0 (secure 0 0 0) 0x7f5d8801d460 con 0x7f5d9c0b7510 2023-11-24T07:57:43.102 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:43.093+0000 7f5d997fa700 1 --2- 172.21.15.59:0/2425103177 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f5d8006af60 0x7f5d8006d420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:57:43.102 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:43.093+0000 7f5d997fa700 1 -- 172.21.15.59:0/2425103177 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f5d88089480 con 0x7f5d9c0b7510 2023-11-24T07:57:43.102 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:43.093+0000 7f5da09ae700 1 --2- 172.21.15.59:0/2425103177 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f5d8006af60 0x7f5d8006d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:57:43.102 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:43.094+0000 7f5da09ae700 1 --2- 172.21.15.59:0/2425103177 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f5d8006af60 0x7f5d8006d420 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f5d900090b0 tx=0x7f5d90008030 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:57:43.103 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:43.098+0000 7f5d997fa700 1 -- 172.21.15.59:0/2425103177 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f5d880545d0 con 0x7f5d9c0b7510 2023-11-24T07:57:43.319 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:43.317+0000 7f5da2c12700 1 -- 172.21.15.59:0/2425103177 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f5d9c15f060 con 0x7f5d9c0b7510 2023-11-24T07:57:43.319 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:43.318+0000 7f5d997fa700 1 -- 172.21.15.59:0/2425103177 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v33) v1 ==== 80+0+0 (secure 0 0 0) 0x7f5d880015c0 con 0x7f5d9c0b7510 2023-11-24T07:57:43.322 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:43.320+0000 7f5da2c12700 1 -- 172.21.15.59:0/2425103177 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f5d8006af60 msgr2=0x7f5d8006d420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:57:43.322 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:43.320+0000 7f5da2c12700 1 --2- 172.21.15.59:0/2425103177 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f5d8006af60 0x7f5d8006d420 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f5d900090b0 tx=0x7f5d90008030 comp rx=0 tx=0).stop 2023-11-24T07:57:43.322 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:43.320+0000 7f5da2c12700 1 -- 172.21.15.59:0/2425103177 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5d9c0b7510 msgr2=0x7f5d9c0c7730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:57:43.322 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:43.320+0000 7f5da2c12700 1 --2- 172.21.15.59:0/2425103177 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5d9c0b7510 0x7f5d9c0c7730 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f5d8800e8f0 tx=0x7f5d8800ecb0 comp rx=0 tx=0).stop 2023-11-24T07:57:43.322 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:43.320+0000 7f5da2c12700 1 -- 172.21.15.59:0/2425103177 shutdown_connections 2023-11-24T07:57:43.323 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:43.320+0000 7f5da2c12700 1 --2- 172.21.15.59:0/2425103177 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f5d8006af60 0x7f5d8006d420 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:43.323 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:43.320+0000 7f5da2c12700 1 --2- 172.21.15.59:0/2425103177 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f5d9c156620 0x7f5d9c0cadd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:43.323 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:43.320+0000 7f5da2c12700 1 --2- 172.21.15.59:0/2425103177 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5d8c003a30 0x7f5d9c157ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:43.323 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:43.320+0000 7f5da2c12700 1 --2- 172.21.15.59:0/2425103177 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5d9c0b7510 0x7f5d9c0c7730 unknown :-1 s=CLOSED pgs=438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:43.323 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:43.320+0000 7f5da2c12700 1 -- 172.21.15.59:0/2425103177 >> 172.21.15.59:0/2425103177 conn(0x7f5d9c0b3720 msgr2=0x7f5d9c0b3b30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:57:43.323 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:43.320+0000 7f5da2c12700 1 -- 172.21.15.59:0/2425103177 shutdown_connections 2023-11-24T07:57:43.323 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:43.320+0000 7f5da2c12700 1 -- 172.21.15.59:0/2425103177 wait complete. 2023-11-24T07:57:43.324 INFO:teuthology.orchestra.run.smithi059.stderr:listed 0 entries 2023-11-24T07:57:43.334 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2023-11-24T07:57:43.334 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T07:57:43.334 DEBUG:teuthology.orchestra.run.smithi059:> sudo umount /home/ubuntu/cephtest/mnt.0 2023-11-24T07:57:43.376 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi059.front.sepia.ceph.com 2023-11-24T07:57:43.376 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T07:57:43.376 DEBUG:teuthology.orchestra.run.smithi059:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-11-24T07:57:43.434 DEBUG:teuthology.orchestra.run.smithi076:> set -ex 2023-11-24T07:57:43.434 DEBUG:teuthology.orchestra.run.smithi076:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-24T07:57:43.456 DEBUG:teuthology.orchestra.run.smithi076:> set -ex 2023-11-24T07:57:43.456 DEBUG:teuthology.orchestra.run.smithi076:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-24T07:57:43.517 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-24T07:57:43.642 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:43.640+0000 7f6501074700 1 Processor -- start 2023-11-24T07:57:43.642 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:43.640+0000 7f6501074700 1 -- start start 2023-11-24T07:57:43.644 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:43.641+0000 7f6501074700 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f64fc0cf2b0 0x7f64fc15c1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:57:43.645 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:43.641+0000 7f6501074700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f64fc0ce5c0 0x7f64fc0cea30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:57:43.645 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:43.641+0000 7f6501074700 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f64fc0d0a90 con 0x7f64fc15bde0 2023-11-24T07:57:43.645 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:43.641+0000 7f6501074700 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f64fc0cef70 con 0x7f64fc0ce5c0 2023-11-24T07:57:43.645 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:43.641+0000 7f6501074700 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f64fc06ee10 con 0x7f64fc0cf2b0 2023-11-24T07:57:43.645 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:43.641+0000 7f64fb59e700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f64fc0ce5c0 0x7f64fc0cea30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:57:43.645 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:43.641+0000 7f64fb59e700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f64fc0ce5c0 0x7f64fc0cea30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:52118/0 (socket says 172.21.15.59:52118) 2023-11-24T07:57:43.645 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:43.641+0000 7f64fb59e700 1 -- 172.21.15.59:0/3937317015 learned_addr learned my addr 172.21.15.59:0/3937317015 (peer_addr_for_me v2:172.21.15.59:0/0) 2023-11-24T07:57:43.645 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:43.641+0000 7f64fa59c700 1 --2- 172.21.15.59:0/3937317015 >> v2:172.21.15.76:3300/0 conn(0x7f64fc0cf2b0 0x7f64fc15c1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:57:43.645 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:43.642+0000 7f64fb59e700 1 -- 172.21.15.59:0/3937317015 >> v2:172.21.15.76:3300/0 conn(0x7f64fc0cf2b0 msgr2=0x7f64fc15c1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T07:57:43.645 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:43.642+0000 7f64fb59e700 1 --2- 172.21.15.59:0/3937317015 >> v2:172.21.15.76:3300/0 conn(0x7f64fc0cf2b0 0x7f64fc15c1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:43.645 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:43.642+0000 7f64fb59e700 1 -- 172.21.15.59:0/3937317015 >> v1:172.21.15.76:6789/0 conn(0x7f64fc15bde0 legacy=0x7f64fc0d01b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:57:43.645 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:43.642+0000 7f64fb59e700 1 -- 172.21.15.59:0/3937317015 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f64fc06f0f0 con 0x7f64fc0ce5c0 2023-11-24T07:57:43.646 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:43.642+0000 7f64fb59e700 1 --2- 172.21.15.59:0/3937317015 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f64fc0ce5c0 0x7f64fc0cea30 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f64fc06e780 tx=0x7f64f400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ad91cd7ac84dd423 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:57:43.646 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:43.642+0000 7f64f9d9b700 1 -- 172.21.15.59:0/3937317015 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64f400e070 con 0x7f64fc0ce5c0 2023-11-24T07:57:43.646 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:43.642+0000 7f64f9d9b700 1 -- 172.21.15.59:0/3937317015 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f64f400b910 con 0x7f64fc0ce5c0 2023-11-24T07:57:43.646 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:43.642+0000 7f64f9d9b700 1 -- 172.21.15.59:0/3937317015 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64f400bcc0 con 0x7f64fc0ce5c0 2023-11-24T07:57:43.646 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:43.642+0000 7f6501074700 1 -- 172.21.15.59:0/3937317015 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f64fc0ce5c0 msgr2=0x7f64fc0cea30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:57:43.646 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:43.642+0000 7f6501074700 1 --2- 172.21.15.59:0/3937317015 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f64fc0ce5c0 0x7f64fc0cea30 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f64fc06e780 tx=0x7f64f400b330 comp rx=0 tx=0).stop 2023-11-24T07:57:43.646 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:43.642+0000 7f6501074700 1 -- 172.21.15.59:0/3937317015 shutdown_connections 2023-11-24T07:57:43.646 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:43.642+0000 7f6501074700 1 --2- 172.21.15.59:0/3937317015 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f64fc0ce5c0 0x7f64fc0cea30 unknown :-1 s=CLOSED pgs=439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:43.646 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:43.642+0000 7f6501074700 1 --2- 172.21.15.59:0/3937317015 >> v2:172.21.15.76:3300/0 conn(0x7f64fc0cf2b0 0x7f64fc15c1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:43.646 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:43.642+0000 7f6501074700 1 -- 172.21.15.59:0/3937317015 >> 172.21.15.59:0/3937317015 conn(0x7f64fc0b3720 msgr2=0x7f64fc0b3b30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:57:43.646 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:43.643+0000 7f6501074700 1 -- 172.21.15.59:0/3937317015 shutdown_connections 2023-11-24T07:57:43.646 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:43.643+0000 7f6501074700 1 -- 172.21.15.59:0/3937317015 wait complete. 2023-11-24T07:57:43.648 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:43.644+0000 7f6501074700 1 Processor -- start 2023-11-24T07:57:43.649 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:43.644+0000 7f6501074700 1 -- start start 2023-11-24T07:57:43.649 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:43.645+0000 7f6501074700 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f64fc0ce5c0 0x7f64fc152790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:57:43.649 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:43.645+0000 7f6501074700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f64fc0cf2b0 0x7f64fc06cd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:57:43.649 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:43.645+0000 7f6501074700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f64fc15bde0 0x7f64fc0c9160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:57:43.649 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:43.645+0000 7f6501074700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64fc0c6550 con 0x7f64fc15bde0 2023-11-24T07:57:43.649 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:43.645+0000 7f6501074700 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64fc0c5f40 con 0x7f64fc0cf2b0 2023-11-24T07:57:43.649 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:43.645+0000 7f6501074700 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f64fc06dcf0 con 0x7f64fc0ce5c0 2023-11-24T07:57:43.649 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:43.645+0000 7f64fb59e700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f64fc15bde0 0x7f64fc0c9160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:57:43.649 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:43.645+0000 7f64fb59e700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f64fc15bde0 0x7f64fc0c9160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:52124/0 (socket says 172.21.15.59:52124) 2023-11-24T07:57:43.649 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:43.645+0000 7f64fb59e700 1 -- 172.21.15.59:0/1093249965 learned_addr learned my addr 172.21.15.59:0/1093249965 (peer_addr_for_me v2:172.21.15.59:0/0) 2023-11-24T07:57:43.649 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:43.645+0000 7f64fad9d700 1 --2- 172.21.15.59:0/1093249965 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f64fc0ce5c0 0x7f64fc152790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:57:43.650 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:43.645+0000 7f64fa59c700 1 --2- 172.21.15.59:0/1093249965 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f64fc0cf2b0 0x7f64fc06cd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:57:43.650 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:43.645+0000 7f64fb59e700 1 -- 172.21.15.59:0/1093249965 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f64fc0ce5c0 msgr2=0x7f64fc152790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:57:43.650 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:43.645+0000 7f64fb59e700 1 --2- 172.21.15.59:0/1093249965 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f64fc0ce5c0 0x7f64fc152790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:43.650 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:43.645+0000 7f64fb59e700 1 -- 172.21.15.59:0/1093249965 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f64fc0cf2b0 msgr2=0x7f64fc06cd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:57:43.650 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:43.645+0000 7f64fb59e700 1 --2- 172.21.15.59:0/1093249965 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f64fc0cf2b0 0x7f64fc06cd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:43.650 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:43.645+0000 7f64fb59e700 1 -- 172.21.15.59:0/1093249965 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f64f4009b30 con 0x7f64fc15bde0 2023-11-24T07:57:43.650 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:43.645+0000 7f64fad9d700 1 --2- 172.21.15.59:0/1093249965 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f64fc0ce5c0 0x7f64fc152790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-24T07:57:43.650 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:43.646+0000 7f64fb59e700 1 --2- 172.21.15.59:0/1093249965 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f64fc15bde0 0x7f64fc0c9160 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f64fc072340 tx=0x7f64f4004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:57:43.650 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:43.646+0000 7f64f37fe700 1 -- 172.21.15.59:0/1093249965 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64f40099b0 con 0x7f64fc15bde0 2023-11-24T07:57:43.650 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:43.646+0000 7f64f37fe700 1 -- 172.21.15.59:0/1093249965 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f64f40092e0 con 0x7f64fc15bde0 2023-11-24T07:57:43.650 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:43.646+0000 7f64f37fe700 1 -- 172.21.15.59:0/1093249965 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64f401c640 con 0x7f64fc15bde0 2023-11-24T07:57:43.651 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:43.646+0000 7f6501074700 1 -- 172.21.15.59:0/1093249965 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f64fc06dfd0 con 0x7f64fc15bde0 2023-11-24T07:57:43.651 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:43.646+0000 7f6501074700 1 -- 172.21.15.59:0/1093249965 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f64fc152cd0 con 0x7f64fc15bde0 2023-11-24T07:57:43.651 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:43.647+0000 7f6501074700 1 -- 172.21.15.59:0/1093249965 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f64fc0c6700 con 0x7f64fc15bde0 2023-11-24T07:57:43.651 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:43.647+0000 7f64f37fe700 1 -- 172.21.15.59:0/1093249965 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 6) v1 ==== 89282+0+0 (secure 0 0 0) 0x7f64f401c7a0 con 0x7f64fc15bde0 2023-11-24T07:57:43.652 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:43.648+0000 7f64f37fe700 1 --2- 172.21.15.59:0/1093249965 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f64dc06af60 0x7f64dc06d420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:57:43.652 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:43.648+0000 7f64f37fe700 1 -- 172.21.15.59:0/1093249965 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f64f408b430 con 0x7f64fc15bde0 2023-11-24T07:57:43.652 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:43.648+0000 7f64fad9d700 1 --2- 172.21.15.59:0/1093249965 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f64dc06af60 0x7f64dc06d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:57:43.652 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:43.648+0000 7f64fad9d700 1 --2- 172.21.15.59:0/1093249965 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f64dc06af60 0x7f64dc06d420 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f64ec000f60 tx=0x7f64ec009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:57:43.653 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:43.652+0000 7f64f37fe700 1 -- 172.21.15.59:0/1093249965 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f64fc0c6700 con 0x7f64fc15bde0 2023-11-24T07:57:43.856 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:43.854+0000 7f6501074700 1 -- 172.21.15.59:0/1093249965 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f64fc068670 con 0x7f64fc15bde0 2023-11-24T07:57:43.857 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:43.855+0000 7f64f37fe700 1 -- 172.21.15.59:0/1093249965 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v33) v1 ==== 74+0+16437 (secure 0 0 0) 0x7f64f4056470 con 0x7f64fc15bde0 2023-11-24T07:57:43.857 INFO:teuthology.orchestra.run.smithi059.stdout: 2023-11-24T07:57:43.858 INFO:teuthology.orchestra.run.smithi059.stdout:{"epoch":33,"fsid":"9f0e8741-b8f9-48d5-a2c3-09f45903303d","created":"2023-11-24T07:56:06.552729+0000","modified":"2023-11-24T07:57:26.579209+0000","last_up_change":"2023-11-24T07:56:17.372352+0000","last_in_change":"2023-11-24T07:56:09.073250+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-24T07:56:18.352823+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2023-11-24T07:57:23.014143+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"32","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2023-11-24T07:57:23.971487+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"33","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"6d8f9e9b-1782-4e41-928b-a39f2a8ebc06","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6800","nonce":3373878772},{"type":"v1","addr":"172.21.15.59:6801","nonce":3373878772}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6802","nonce":3373878772},{"type":"v1","addr":"172.21.15.59:6803","nonce":3373878772}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6806","nonce":3373878772},{"type":"v1","addr":"172.21.15.59:6807","nonce":3373878772}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6804","nonce":3373878772},{"type":"v1","addr":"172.21.15.59:6805","nonce":3373878772}]},"public_addr":"172.21.15.59:6801/3373878772","cluster_addr":"172.21.15.59:6803/3373878772","heartbeat_back_addr":"172.21.15.59:6807/3373878772","heartbeat_front_addr":"172.21.15.59:6805/3373878772","state":["exists","up"]},{"osd":1,"uuid":"e8d882aa-52e1-4c04-9e1e-c44ef9204a5f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6824","nonce":648258563},{"type":"v1","addr":"172.21.15.59:6825","nonce":648258563}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6826","nonce":648258563},{"type":"v1","addr":"172.21.15.59:6827","nonce":648258563}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6830","nonce":648258563},{"type":"v1","addr":"172.21.15.59:6831","nonce":648258563}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6828","nonce":648258563},{"type":"v1","addr":"172.21.15.59:6829","nonce":648258563}]},"public_addr":"172.21.15.59:6825/648258563","cluster_addr":"172.21.15.59:6827/648258563","heartbeat_back_addr":"172.21.15.59:6831/648258563","heartbeat_front_addr":"172.21.15.59:6829/648258563","state":["exists","up"]},{"osd":2,"uuid":"69eebfa0-695b-4b21-90b9-8a3537168a01","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6808","nonce":1840430654},{"type":"v1","addr":"172.21.15.59:6809","nonce":1840430654}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6810","nonce":1840430654},{"type":"v1","addr":"172.21.15.59:6811","nonce":1840430654}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6814","nonce":1840430654},{"type":"v1","addr":"172.21.15.59:6815","nonce":1840430654}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6812","nonce":1840430654},{"type":"v1","addr":"172.21.15.59:6813","nonce":1840430654}]},"public_addr":"172.21.15.59:6809/1840430654","cluster_addr":"172.21.15.59:6811/1840430654","heartbeat_back_addr":"172.21.15.59:6815/1840430654","heartbeat_front_addr":"172.21.15.59:6813/1840430654","state":["exists","up"]},{"osd":3,"uuid":"96fd77fc-f911-46ff-9445-6f0be2daa8f4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6816","nonce":2504025850},{"type":"v1","addr":"172.21.15.59:6817","nonce":2504025850}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6818","nonce":2504025850},{"type":"v1","addr":"172.21.15.59:6819","nonce":2504025850}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6822","nonce":2504025850},{"type":"v1","addr":"172.21.15.59:6823","nonce":2504025850}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6820","nonce":2504025850},{"type":"v1","addr":"172.21.15.59:6821","nonce":2504025850}]},"public_addr":"172.21.15.59:6817/2504025850","cluster_addr":"172.21.15.59:6819/2504025850","heartbeat_back_addr":"172.21.15.59:6823/2504025850","heartbeat_front_addr":"172.21.15.59:6821/2504025850","state":["exists","up"]},{"osd":4,"uuid":"7ac29fca-b5b2-4c95-9d69-2f164cbbc708","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6816","nonce":2382463431},{"type":"v1","addr":"172.21.15.76:6818","nonce":2382463431}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6820","nonce":2382463431},{"type":"v1","addr":"172.21.15.76:6822","nonce":2382463431}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6828","nonce":2382463431},{"type":"v1","addr":"172.21.15.76:6830","nonce":2382463431}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6824","nonce":2382463431},{"type":"v1","addr":"172.21.15.76:6826","nonce":2382463431}]},"public_addr":"172.21.15.76:6818/2382463431","cluster_addr":"172.21.15.76:6822/2382463431","heartbeat_back_addr":"172.21.15.76:6830/2382463431","heartbeat_front_addr":"172.21.15.76:6826/2382463431","state":["exists","up"]},{"osd":5,"uuid":"485bf711-3851-4238-a4f4-51d52e0a9c58","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6800","nonce":329334308},{"type":"v1","addr":"172.21.15.76:6802","nonce":329334308}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6805","nonce":329334308},{"type":"v1","addr":"172.21.15.76:6807","nonce":329334308}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6814","nonce":329334308},{"type":"v1","addr":"172.21.15.76:6815","nonce":329334308}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6810","nonce":329334308},{"type":"v1","addr":"172.21.15.76:6812","nonce":329334308}]},"public_addr":"172.21.15.76:6802/329334308","cluster_addr":"172.21.15.76:6807/329334308","heartbeat_back_addr":"172.21.15.76:6815/329334308","heartbeat_front_addr":"172.21.15.76:6812/329334308","state":["exists","up"]},{"osd":6,"uuid":"72b3f2fc-1e4a-4e32-968d-003647a0286f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6801","nonce":3946579410},{"type":"v1","addr":"172.21.15.76:6803","nonce":3946579410}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6804","nonce":3946579410},{"type":"v1","addr":"172.21.15.76:6806","nonce":3946579410}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6811","nonce":3946579410},{"type":"v1","addr":"172.21.15.76:6813","nonce":3946579410}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6808","nonce":3946579410},{"type":"v1","addr":"172.21.15.76:6809","nonce":3946579410}]},"public_addr":"172.21.15.76:6803/3946579410","cluster_addr":"172.21.15.76:6806/3946579410","heartbeat_back_addr":"172.21.15.76:6813/3946579410","heartbeat_front_addr":"172.21.15.76:6809/3946579410","state":["exists","up"]},{"osd":7,"uuid":"1ddceb90-6c5d-4d1a-ad56-99cb877f8c70","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6817","nonce":841761538},{"type":"v1","addr":"172.21.15.76:6819","nonce":841761538}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6821","nonce":841761538},{"type":"v1","addr":"172.21.15.76:6823","nonce":841761538}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6829","nonce":841761538},{"type":"v1","addr":"172.21.15.76:6831","nonce":841761538}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6825","nonce":841761538},{"type":"v1","addr":"172.21.15.76:6827","nonce":841761538}]},"public_addr":"172.21.15.76:6819/841761538","cluster_addr":"172.21.15.76:6823/841761538","heartbeat_back_addr":"172.21.15.76:6831/841761538","heartbeat_front_addr":"172.21.15.76:6827/841761538","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T07:56:14.294838+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T07:56:14.270779+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T07:56:14.231809+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T07:56:14.260983+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T07:56:14.302997+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T07:56:14.258600+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T07:56:14.251413+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T07:56:14.304727+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-24T07:57:43.860 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:43.858+0000 7f6501074700 1 -- 172.21.15.59:0/1093249965 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f64dc06af60 msgr2=0x7f64dc06d420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:57:43.860 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:43.858+0000 7f6501074700 1 --2- 172.21.15.59:0/1093249965 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f64dc06af60 0x7f64dc06d420 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f64ec000f60 tx=0x7f64ec009040 comp rx=0 tx=0).stop 2023-11-24T07:57:43.860 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:43.858+0000 7f6501074700 1 -- 172.21.15.59:0/1093249965 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f64fc15bde0 msgr2=0x7f64fc0c9160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:57:43.860 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:43.858+0000 7f6501074700 1 --2- 172.21.15.59:0/1093249965 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f64fc15bde0 0x7f64fc0c9160 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f64fc072340 tx=0x7f64f4004ad0 comp rx=0 tx=0).stop 2023-11-24T07:57:43.860 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:43.858+0000 7f6501074700 1 -- 172.21.15.59:0/1093249965 shutdown_connections 2023-11-24T07:57:43.860 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:43.858+0000 7f6501074700 1 --2- 172.21.15.59:0/1093249965 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f64dc06af60 0x7f64dc06d420 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:43.860 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:43.858+0000 7f6501074700 1 --2- 172.21.15.59:0/1093249965 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f64fc15bde0 0x7f64fc0c9160 unknown :-1 s=CLOSED pgs=440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:43.860 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:43.858+0000 7f6501074700 1 --2- 172.21.15.59:0/1093249965 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f64fc0ce5c0 0x7f64fc152790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:43.860 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:43.858+0000 7f6501074700 1 --2- 172.21.15.59:0/1093249965 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f64fc0cf2b0 0x7f64fc06cd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:43.860 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:43.858+0000 7f6501074700 1 -- 172.21.15.59:0/1093249965 >> 172.21.15.59:0/1093249965 conn(0x7f64fc0b3720 msgr2=0x7f64fc06fad0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:57:43.861 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:43.858+0000 7f6501074700 1 -- 172.21.15.59:0/1093249965 shutdown_connections 2023-11-24T07:57:43.861 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:43.858+0000 7f6501074700 1 -- 172.21.15.59:0/1093249965 wait complete. 2023-11-24T07:57:43.872 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-24T07:56:18.352823+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-24T07:57:23.014143+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '32', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 5, 'pool_name': 'cephfs_data', 'create_time': '2023-11-24T07:57:23.971487+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '33', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-24T07:57:43.872 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-24T07:57:44.000 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:43.999+0000 7f5ebd9ea700 1 Processor -- start 2023-11-24T07:57:44.001 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:43.999+0000 7f5ebd9ea700 1 -- start start 2023-11-24T07:57:44.005 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:43.999+0000 7f5ebd9ea700 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f5eb80d0c50 0x7f5eb80d26c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:57:44.005 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:44.000+0000 7f5ebd9ea700 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5eb80d4e20 con 0x7f5eb80d49f0 2023-11-24T07:57:44.005 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:44.000+0000 7f5ebd9ea700 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5eb80b5370 con 0x7f5eb80d0870 2023-11-24T07:57:44.005 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:44.000+0000 7f5ebd9ea700 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5eb80d2f80 con 0x7f5eb80d0c50 2023-11-24T07:57:44.005 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:44.000+0000 7f5eb77fe700 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7f5eb80d49f0 0x7f5eb80b6d20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:55606/0 (socket says 172.21.15.59:55606) 2023-11-24T07:57:44.006 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:44.000+0000 7f5eb77fe700 1 -- 172.21.15.59:0/2857701462 learned_addr learned my addr 172.21.15.59:0/2857701462 (peer_addr_for_me v1:172.21.15.59:0/0) 2023-11-24T07:57:44.006 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:44.000+0000 7f5eb67fc700 1 --2- 172.21.15.59:0/2857701462 >> v2:172.21.15.76:3300/0 conn(0x7f5eb80d0c50 0x7f5eb80d26c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:57:44.006 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:44.000+0000 7f5eb5ffb700 1 -- 172.21.15.59:0/2857701462 <== mon.0 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3563471153 0 0) 0x7f5eb80d4e20 con 0x7f5eb80d49f0 2023-11-24T07:57:44.006 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:44.000+0000 7f5eb5ffb700 1 -- 172.21.15.59:0/2857701462 --> v1:172.21.15.59:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5ea00035b0 con 0x7f5eb80d49f0 2023-11-24T07:57:44.006 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:44.000+0000 7f5eb5ffb700 1 -- 172.21.15.59:0/2857701462 <== mon.0 v1:172.21.15.59:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 277097030 0 0) 0x7f5ea00035b0 con 0x7f5eb80d49f0 2023-11-24T07:57:44.007 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:44.000+0000 7f5eb5ffb700 1 -- 172.21.15.59:0/2857701462 >> v2:172.21.15.76:3300/0 conn(0x7f5eb80d0c50 msgr2=0x7f5eb80d26c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T07:57:44.007 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:44.000+0000 7f5eb5ffb700 1 --2- 172.21.15.59:0/2857701462 >> v2:172.21.15.76:3300/0 conn(0x7f5eb80d0c50 0x7f5eb80d26c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:44.007 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:44.000+0000 7f5eb5ffb700 1 -- 172.21.15.59:0/2857701462 >> v1:172.21.15.76:6789/0 conn(0x7f5eb80d0870 legacy=0x7f5eb80cf860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:57:44.007 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:44.000+0000 7f5eb5ffb700 1 -- 172.21.15.59:0/2857701462 --> v1:172.21.15.59:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5eb80ceae0 con 0x7f5eb80d49f0 2023-11-24T07:57:44.007 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:44.000+0000 7f5eb5ffb700 1 -- 172.21.15.59:0/2857701462 <== mon.0 v1:172.21.15.59:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1312333566 0 0) 0x7f5eb00019f0 con 0x7f5eb80d49f0 2023-11-24T07:57:44.007 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:44.000+0000 7f5eb5ffb700 1 -- 172.21.15.59:0/2857701462 >> v1:172.21.15.59:6789/0 conn(0x7f5eb80d49f0 legacy=0x7f5eb80b6d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:57:44.007 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:44.000+0000 7f5eb5ffb700 1 --2- 172.21.15.59:0/2857701462 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5ea0003a00 0x7f5ea0003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:57:44.007 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:44.000+0000 7f5eb5ffb700 1 --2- 172.21.15.59:0/2857701462 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5ea0004400 0x7f5ea00048d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:57:44.008 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:44.000+0000 7f5eb5ffb700 1 --2- 172.21.15.59:0/2857701462 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f5ea00050d0 0x7f5ea000ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:57:44.008 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:44.000+0000 7f5eb5ffb700 1 -- 172.21.15.59:0/2857701462 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5eb80d2f80 con 0x7f5ea0004400 2023-11-24T07:57:44.008 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:44.000+0000 7f5eb5ffb700 1 -- 172.21.15.59:0/2857701462 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ea000c030 con 0x7f5ea0003a00 2023-11-24T07:57:44.008 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:44.000+0000 7f5eb5ffb700 1 -- 172.21.15.59:0/2857701462 --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5ea000c1f0 con 0x7f5ea00050d0 2023-11-24T07:57:44.008 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:44.000+0000 7f5eb67fc700 1 --2- 172.21.15.59:0/2857701462 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5ea0004400 0x7f5ea00048d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:57:44.008 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:44.001+0000 7f5eb6ffd700 1 --2- 172.21.15.59:0/2857701462 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5ea0003a00 0x7f5ea0003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:57:44.008 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:44.001+0000 7f5eb77fe700 1 --2- 172.21.15.59:0/2857701462 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f5ea00050d0 0x7f5ea000ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:57:44.009 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:44.001+0000 7f5eb67fc700 1 -- 172.21.15.59:0/2857701462 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f5ea00050d0 msgr2=0x7f5ea000ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T07:57:44.009 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:44.001+0000 7f5eb67fc700 1 --2- 172.21.15.59:0/2857701462 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f5ea00050d0 0x7f5ea000ba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:44.009 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:44.001+0000 7f5eb67fc700 1 -- 172.21.15.59:0/2857701462 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5ea0003a00 msgr2=0x7f5ea0003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T07:57:44.009 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:44.001+0000 7f5eb67fc700 1 --2- 172.21.15.59:0/2857701462 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5ea0003a00 0x7f5ea0003de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:44.009 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:44.001+0000 7f5eb67fc700 1 -- 172.21.15.59:0/2857701462 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5ea00035b0 con 0x7f5ea0004400 2023-11-24T07:57:44.009 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:44.001+0000 7f5eb77fe700 1 --2- 172.21.15.59:0/2857701462 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f5ea00050d0 0x7f5ea000ba10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-24T07:57:44.009 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:44.001+0000 7f5eb6ffd700 1 --2- 172.21.15.59:0/2857701462 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5ea0003a00 0x7f5ea0003de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-24T07:57:44.009 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:44.001+0000 7f5eb77fe700 1 -- 172.21.15.59:0/2857701462 reap_dead start 2023-11-24T07:57:44.009 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:44.001+0000 7f5eb67fc700 1 --2- 172.21.15.59:0/2857701462 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5ea0004400 0x7f5ea00048d0 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f5eac009b30 tx=0x7f5ea0005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f54b000648b52320 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:57:44.009 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:44.001+0000 7f5eb5ffb700 1 -- 172.21.15.59:0/2857701462 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5eac00e6c0 con 0x7f5ea0004400 2023-11-24T07:57:44.009 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:44.001+0000 7f5eb5ffb700 1 -- 172.21.15.59:0/2857701462 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5eac00e820 con 0x7f5ea0004400 2023-11-24T07:57:44.010 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:44.001+0000 7f5eb5ffb700 1 -- 172.21.15.59:0/2857701462 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5eac00eb90 con 0x7f5ea0004400 2023-11-24T07:57:44.010 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:44.002+0000 7f5ebd9ea700 1 -- 172.21.15.59:0/2857701462 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5ea0004400 msgr2=0x7f5ea00048d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:57:44.010 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:44.002+0000 7f5ebd9ea700 1 --2- 172.21.15.59:0/2857701462 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5ea0004400 0x7f5ea00048d0 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f5eac009b30 tx=0x7f5ea0005880 comp rx=0 tx=0).stop 2023-11-24T07:57:44.010 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:44.002+0000 7f5ebd9ea700 1 -- 172.21.15.59:0/2857701462 shutdown_connections 2023-11-24T07:57:44.010 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:44.002+0000 7f5ebd9ea700 1 --2- 172.21.15.59:0/2857701462 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5ea0004400 0x7f5ea00048d0 unknown :-1 s=CLOSED pgs=442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:44.010 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:44.002+0000 7f5ebd9ea700 1 -- 172.21.15.59:0/2857701462 >> 172.21.15.59:0/2857701462 conn(0x7f5eb80d3ed0 msgr2=0x7f5eb80d42e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:57:44.010 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:44.002+0000 7f5ebd9ea700 1 -- 172.21.15.59:0/2857701462 shutdown_connections 2023-11-24T07:57:44.010 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:44.002+0000 7f5ebd9ea700 1 -- 172.21.15.59:0/2857701462 wait complete. 2023-11-24T07:57:44.010 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:44.003+0000 7f5ebd9ea700 1 Processor -- start 2023-11-24T07:57:44.010 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:44.003+0000 7f5ebd9ea700 1 -- start start 2023-11-24T07:57:44.013 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:44.004+0000 7f5ebd9ea700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5ea0004400 0x7f5eb815a4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:57:44.013 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:44.004+0000 7f5ebd9ea700 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f5eb80b3a30 0x7f5eb81610f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:57:44.013 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:44.004+0000 7f5ebd9ea700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5eb8161630 0x7f5eb815eba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:57:44.013 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:44.004+0000 7f5ebd9ea700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5eb80cff70 con 0x7f5eb8161630 2023-11-24T07:57:44.013 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:44.004+0000 7f5ebd9ea700 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5eb815e710 con 0x7f5ea0004400 2023-11-24T07:57:44.013 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:44.004+0000 7f5ebd9ea700 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5eb80b23f0 con 0x7f5eb80b3a30 2023-11-24T07:57:44.013 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:44.005+0000 7f5eb77fe700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5eb8161630 0x7f5eb815eba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:57:44.013 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:44.005+0000 7f5eb77fe700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5eb8161630 0x7f5eb815eba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:52144/0 (socket says 172.21.15.59:52144) 2023-11-24T07:57:44.014 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:44.005+0000 7f5eb77fe700 1 -- 172.21.15.59:0/3479071605 learned_addr learned my addr 172.21.15.59:0/3479071605 (peer_addr_for_me v2:172.21.15.59:0/0) 2023-11-24T07:57:44.014 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:44.005+0000 7f5eb67fc700 1 --2- 172.21.15.59:0/3479071605 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f5eb80b3a30 0x7f5eb81610f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:57:44.014 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:44.005+0000 7f5eb6ffd700 1 --2- 172.21.15.59:0/3479071605 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5ea0004400 0x7f5eb815a4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:57:44.014 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:44.005+0000 7f5eb77fe700 1 -- 172.21.15.59:0/3479071605 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f5eb80b3a30 msgr2=0x7f5eb81610f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:57:44.014 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:44.005+0000 7f5eb77fe700 1 --2- 172.21.15.59:0/3479071605 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f5eb80b3a30 0x7f5eb81610f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:44.014 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:44.005+0000 7f5eb77fe700 1 -- 172.21.15.59:0/3479071605 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5ea0004400 msgr2=0x7f5eb815a4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:57:44.014 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:44.005+0000 7f5eb77fe700 1 --2- 172.21.15.59:0/3479071605 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5ea0004400 0x7f5eb815a4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:44.014 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:44.005+0000 7f5eb77fe700 1 -- 172.21.15.59:0/3479071605 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5eb815dde0 con 0x7f5eb8161630 2023-11-24T07:57:44.014 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:44.005+0000 7f5eb67fc700 1 --2- 172.21.15.59:0/3479071605 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f5eb80b3a30 0x7f5eb81610f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-24T07:57:44.014 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:44.005+0000 7f5eb6ffd700 1 --2- 172.21.15.59:0/3479071605 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5ea0004400 0x7f5eb815a4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-24T07:57:44.015 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:44.005+0000 7f5eb77fe700 1 --2- 172.21.15.59:0/3479071605 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5eb8161630 0x7f5eb815eba0 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f5eb80ce5b0 tx=0x7f5eb000e4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:57:44.015 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:44.005+0000 7f5e9ffff700 1 -- 172.21.15.59:0/3479071605 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5eb0017070 con 0x7f5eb8161630 2023-11-24T07:57:44.015 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:44.005+0000 7f5e9ffff700 1 -- 172.21.15.59:0/3479071605 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5eb000ed80 con 0x7f5eb8161630 2023-11-24T07:57:44.015 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:44.005+0000 7f5e9ffff700 1 -- 172.21.15.59:0/3479071605 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5eb0013420 con 0x7f5eb8161630 2023-11-24T07:57:44.015 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:44.005+0000 7f5ebd9ea700 1 -- 172.21.15.59:0/3479071605 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5eb815e0c0 con 0x7f5eb8161630 2023-11-24T07:57:44.015 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:44.005+0000 7f5ebd9ea700 1 -- 172.21.15.59:0/3479071605 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5eb8161ac0 con 0x7f5eb8161630 2023-11-24T07:57:44.015 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:44.006+0000 7f5ebd9ea700 1 -- 172.21.15.59:0/3479071605 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5eb80b57c0 con 0x7f5eb8161630 2023-11-24T07:57:44.015 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:44.007+0000 7f5e9ffff700 1 -- 172.21.15.59:0/3479071605 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 6) v1 ==== 89282+0+0 (secure 0 0 0) 0x7f5eb0013580 con 0x7f5eb8161630 2023-11-24T07:57:44.015 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:44.007+0000 7f5e9ffff700 1 --2- 172.21.15.59:0/3479071605 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f5ea406af60 0x7f5ea406d420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:57:44.015 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:44.007+0000 7f5e9ffff700 1 -- 172.21.15.59:0/3479071605 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f5eb0089370 con 0x7f5eb8161630 2023-11-24T07:57:44.015 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:44.007+0000 7f5eb6ffd700 1 --2- 172.21.15.59:0/3479071605 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f5ea406af60 0x7f5ea406d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:57:44.015 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:44.008+0000 7f5eb6ffd700 1 --2- 172.21.15.59:0/3479071605 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f5ea406af60 0x7f5ea406d420 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f5ea8000c00 tx=0x7f5ea8009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:57:44.016 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:44.012+0000 7f5e9ffff700 1 -- 172.21.15.59:0/3479071605 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f5eb0054430 con 0x7f5eb8161630 2023-11-24T07:57:44.212 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:44.210+0000 7f5ebd9ea700 1 -- 172.21.15.59:0/3479071605 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f5eb8075480 con 0x7f5eb8161630 2023-11-24T07:57:44.212 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:44.211+0000 7f5e9ffff700 1 -- 172.21.15.59:0/3479071605 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v33) v1 ==== 93+0+10 (secure 0 0 0) 0x7f5eb0057a50 con 0x7f5eb8161630 2023-11-24T07:57:44.213 INFO:teuthology.orchestra.run.smithi059.stdout:pg_num: 1 2023-11-24T07:57:44.215 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:44.213+0000 7f5ebd9ea700 1 -- 172.21.15.59:0/3479071605 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f5ea406af60 msgr2=0x7f5ea406d420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:57:44.215 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:44.213+0000 7f5ebd9ea700 1 --2- 172.21.15.59:0/3479071605 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f5ea406af60 0x7f5ea406d420 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f5ea8000c00 tx=0x7f5ea8009040 comp rx=0 tx=0).stop 2023-11-24T07:57:44.215 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:44.213+0000 7f5ebd9ea700 1 -- 172.21.15.59:0/3479071605 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5eb8161630 msgr2=0x7f5eb815eba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:57:44.215 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:44.213+0000 7f5ebd9ea700 1 --2- 172.21.15.59:0/3479071605 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5eb8161630 0x7f5eb815eba0 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f5eb80ce5b0 tx=0x7f5eb000e4f0 comp rx=0 tx=0).stop 2023-11-24T07:57:44.215 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:44.213+0000 7f5ebd9ea700 1 -- 172.21.15.59:0/3479071605 shutdown_connections 2023-11-24T07:57:44.215 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:44.213+0000 7f5ebd9ea700 1 --2- 172.21.15.59:0/3479071605 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f5ea406af60 0x7f5ea406d420 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:44.215 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:44.213+0000 7f5ebd9ea700 1 --2- 172.21.15.59:0/3479071605 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5eb8161630 0x7f5eb815eba0 unknown :-1 s=CLOSED pgs=443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:44.215 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:44.213+0000 7f5ebd9ea700 1 --2- 172.21.15.59:0/3479071605 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5ea0004400 0x7f5eb815a4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:44.215 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:44.213+0000 7f5ebd9ea700 1 --2- 172.21.15.59:0/3479071605 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f5eb80b3a30 0x7f5eb81610f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:44.215 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:44.213+0000 7f5ebd9ea700 1 -- 172.21.15.59:0/3479071605 >> 172.21.15.59:0/3479071605 conn(0x7f5eb80d3ed0 msgr2=0x7f5eb80d42b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:57:44.215 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:44.213+0000 7f5ebd9ea700 1 -- 172.21.15.59:0/3479071605 shutdown_connections 2023-11-24T07:57:44.216 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:44.213+0000 7f5ebd9ea700 1 -- 172.21.15.59:0/3479071605 wait complete. 2023-11-24T07:57:44.227 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-24T07:57:44.354 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:44.352+0000 7ff9664f0700 1 Processor -- start 2023-11-24T07:57:44.354 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:44.352+0000 7ff9664f0700 1 -- start start 2023-11-24T07:57:44.356 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:44.353+0000 7ff9664f0700 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7ff96006fc50 0x7ff96006eef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:57:44.357 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:44.353+0000 7ff9664f0700 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff9600d1230 con 0x7ff96015e1c0 2023-11-24T07:57:44.357 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:44.353+0000 7ff9664f0700 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff9600d0c70 con 0x7ff9600b0e90 2023-11-24T07:57:44.357 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:44.353+0000 7ff9664f0700 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff9600d0f50 con 0x7ff96006fc50 2023-11-24T07:57:44.357 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:44.353+0000 7ff95ffff700 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7ff96015e1c0 0x7ff96006f540 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:55622/0 (socket says 172.21.15.59:55622) 2023-11-24T07:57:44.357 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:44.353+0000 7ff95ffff700 1 -- 172.21.15.59:0/1252508053 learned_addr learned my addr 172.21.15.59:0/1252508053 (peer_addr_for_me v1:172.21.15.59:0/0) 2023-11-24T07:57:44.357 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:44.353+0000 7ff95f7fe700 1 --2- 172.21.15.59:0/1252508053 >> v2:172.21.15.76:3300/0 conn(0x7ff96006fc50 0x7ff96006eef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:57:44.357 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:44.353+0000 7ff95effd700 1 -- 172.21.15.59:0/1252508053 <== mon.0 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4200956553 0 0) 0x7ff9600d1230 con 0x7ff96015e1c0 2023-11-24T07:57:44.358 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:44.353+0000 7ff95effd700 1 -- 172.21.15.59:0/1252508053 --> v1:172.21.15.59:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff9480035b0 con 0x7ff96015e1c0 2023-11-24T07:57:44.358 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:44.354+0000 7ff95effd700 1 -- 172.21.15.59:0/1252508053 <== mon.0 v1:172.21.15.59:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4085431203 0 0) 0x7ff9480035b0 con 0x7ff96015e1c0 2023-11-24T07:57:44.358 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:44.354+0000 7ff95f7fe700 1 -- 172.21.15.59:0/1252508053 >> v1:172.21.15.76:6789/0 conn(0x7ff9600b0e90 legacy=0x7ff96006ca10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:57:44.358 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:44.354+0000 7ff95f7fe700 1 -- 172.21.15.59:0/1252508053 >> v1:172.21.15.59:6789/0 conn(0x7ff96015e1c0 legacy=0x7ff96006f540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:57:44.358 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:44.354+0000 7ff95f7fe700 1 -- 172.21.15.59:0/1252508053 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff96006c210 con 0x7ff96006fc50 2023-11-24T07:57:44.358 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:44.354+0000 7ff95f7fe700 1 --2- 172.21.15.59:0/1252508053 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff96006fc50 0x7ff96006eef0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7ff950009840 tx=0x7ff950009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4c3f76b80419cdff server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:57:44.358 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:44.354+0000 7ff95effd700 1 -- 172.21.15.59:0/1252508053 <== mon.1 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff954001670 con 0x7ff96006fc50 2023-11-24T07:57:44.359 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:44.354+0000 7ff95effd700 1 -- 172.21.15.59:0/1252508053 <== mon.1 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff950004030 con 0x7ff96006fc50 2023-11-24T07:57:44.359 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:44.354+0000 7ff95effd700 1 -- 172.21.15.59:0/1252508053 <== mon.1 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9500043c0 con 0x7ff96006fc50 2023-11-24T07:57:44.359 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:44.355+0000 7ff9664f0700 1 -- 172.21.15.59:0/1252508053 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff96006fc50 msgr2=0x7ff96006eef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:57:44.359 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:44.355+0000 7ff9664f0700 1 --2- 172.21.15.59:0/1252508053 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff96006fc50 0x7ff96006eef0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7ff950009840 tx=0x7ff950009b50 comp rx=0 tx=0).stop 2023-11-24T07:57:44.359 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:44.355+0000 7ff9664f0700 1 -- 172.21.15.59:0/1252508053 shutdown_connections 2023-11-24T07:57:44.359 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:44.355+0000 7ff9664f0700 1 --2- 172.21.15.59:0/1252508053 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff96006fc50 0x7ff96006eef0 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:44.359 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:44.355+0000 7ff9664f0700 1 -- 172.21.15.59:0/1252508053 >> 172.21.15.59:0/1252508053 conn(0x7ff9600b3720 msgr2=0x7ff9600b3b30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:57:44.360 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:44.355+0000 7ff9664f0700 1 -- 172.21.15.59:0/1252508053 shutdown_connections 2023-11-24T07:57:44.360 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:44.355+0000 7ff9664f0700 1 -- 172.21.15.59:0/1252508053 wait complete. 2023-11-24T07:57:44.363 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:44.356+0000 7ff9664f0700 1 Processor -- start 2023-11-24T07:57:44.363 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:44.356+0000 7ff9664f0700 1 -- start start 2023-11-24T07:57:44.363 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:44.357+0000 7ff9664f0700 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7ff96006fc50 0x7ff9600b1f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:57:44.363 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:44.357+0000 7ff9664f0700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff9600b0e90 0x7ff9601f33a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:57:44.363 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:44.357+0000 7ff9664f0700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff96015e1c0 0x7ff9600cd680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:57:44.363 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:44.357+0000 7ff9664f0700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9600c8ca0 con 0x7ff96015e1c0 2023-11-24T07:57:44.364 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:44.357+0000 7ff9664f0700 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9600b7610 con 0x7ff9600b0e90 2023-11-24T07:57:44.364 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:44.357+0000 7ff9664f0700 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff9600b78f0 con 0x7ff96006fc50 2023-11-24T07:57:44.364 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:44.357+0000 7ff964a8d700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff96015e1c0 0x7ff9600cd680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:57:44.364 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:44.357+0000 7ff964a8d700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff96015e1c0 0x7ff9600cd680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:52158/0 (socket says 172.21.15.59:52158) 2023-11-24T07:57:44.364 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:44.357+0000 7ff964a8d700 1 -- 172.21.15.59:0/2969724069 learned_addr learned my addr 172.21.15.59:0/2969724069 (peer_addr_for_me v2:172.21.15.59:0/0) 2023-11-24T07:57:44.364 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:44.357+0000 7ff95f7fe700 1 --2- 172.21.15.59:0/2969724069 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff9600b0e90 0x7ff9601f33a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:57:44.364 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:44.357+0000 7ff95ffff700 1 --2- 172.21.15.59:0/2969724069 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7ff96006fc50 0x7ff9600b1f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:57:44.365 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:44.357+0000 7ff964a8d700 1 -- 172.21.15.59:0/2969724069 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7ff96006fc50 msgr2=0x7ff9600b1f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:57:44.365 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:44.357+0000 7ff964a8d700 1 --2- 172.21.15.59:0/2969724069 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7ff96006fc50 0x7ff9600b1f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:44.365 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:44.357+0000 7ff964a8d700 1 -- 172.21.15.59:0/2969724069 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff9600b0e90 msgr2=0x7ff9601f33a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:57:44.365 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:44.357+0000 7ff964a8d700 1 --2- 172.21.15.59:0/2969724069 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff9600b0e90 0x7ff9601f33a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:44.365 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:44.357+0000 7ff964a8d700 1 -- 172.21.15.59:0/2969724069 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff9500094a0 con 0x7ff96015e1c0 2023-11-24T07:57:44.366 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:44.357+0000 7ff95ffff700 1 --2- 172.21.15.59:0/2969724069 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7ff96006fc50 0x7ff9600b1f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T07:57:44.366 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:44.357+0000 7ff964a8d700 1 --2- 172.21.15.59:0/2969724069 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff96015e1c0 0x7ff9600cd680 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7ff95800c870 tx=0x7ff95800cc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:57:44.366 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:44.357+0000 7ff95cff9700 1 -- 172.21.15.59:0/2969724069 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff958003e30 con 0x7ff96015e1c0 2023-11-24T07:57:44.366 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:44.357+0000 7ff95cff9700 1 -- 172.21.15.59:0/2969724069 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff95800ce70 con 0x7ff96015e1c0 2023-11-24T07:57:44.366 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:44.358+0000 7ff9664f0700 1 -- 172.21.15.59:0/2969724069 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff9600c8ef0 con 0x7ff96015e1c0 2023-11-24T07:57:44.366 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:44.358+0000 7ff95cff9700 1 -- 172.21.15.59:0/2969724069 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff958005290 con 0x7ff96015e1c0 2023-11-24T07:57:44.366 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:44.358+0000 7ff9664f0700 1 -- 172.21.15.59:0/2969724069 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff9600ce360 con 0x7ff96015e1c0 2023-11-24T07:57:44.366 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:44.358+0000 7ff9664f0700 1 -- 172.21.15.59:0/2969724069 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff960075480 con 0x7ff96015e1c0 2023-11-24T07:57:44.366 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:44.359+0000 7ff95cff9700 1 -- 172.21.15.59:0/2969724069 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 6) v1 ==== 89282+0+0 (secure 0 0 0) 0x7ff9580053f0 con 0x7ff96015e1c0 2023-11-24T07:57:44.367 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:44.359+0000 7ff95cff9700 1 --2- 172.21.15.59:0/2969724069 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7ff94c06af60 0x7ff94c06d420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:57:44.367 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:44.359+0000 7ff95cff9700 1 -- 172.21.15.59:0/2969724069 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 5709+0+0 (secure 0 0 0) 0x7ff958089390 con 0x7ff96015e1c0 2023-11-24T07:57:44.367 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:44.360+0000 7ff95ffff700 1 --2- 172.21.15.59:0/2969724069 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7ff94c06af60 0x7ff94c06d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:57:44.367 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:44.360+0000 7ff95ffff700 1 --2- 172.21.15.59:0/2969724069 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7ff94c06af60 0x7ff94c06d420 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7ff954000f60 tx=0x7ff954009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:57:44.367 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:44.364+0000 7ff95cff9700 1 -- 172.21.15.59:0/2969724069 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7ff958054450 con 0x7ff96015e1c0 2023-11-24T07:57:44.566 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:44.564+0000 7ff9664f0700 1 -- 172.21.15.59:0/2969724069 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7ff9600b7ab0 con 0x7ff96015e1c0 2023-11-24T07:57:44.566 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:44.564+0000 7ff95cff9700 1 -- 172.21.15.59:0/2969724069 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v33) v1 ==== 104+0+11 (secure 0 0 0) 0x7ff9600b7ab0 con 0x7ff96015e1c0 2023-11-24T07:57:44.566 INFO:teuthology.orchestra.run.smithi059.stdout:pg_num: 32 2023-11-24T07:57:44.568 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:44.566+0000 7ff9664f0700 1 -- 172.21.15.59:0/2969724069 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7ff94c06af60 msgr2=0x7ff94c06d420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:57:44.568 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:44.566+0000 7ff9664f0700 1 --2- 172.21.15.59:0/2969724069 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7ff94c06af60 0x7ff94c06d420 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7ff954000f60 tx=0x7ff954009040 comp rx=0 tx=0).stop 2023-11-24T07:57:44.568 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:44.566+0000 7ff9664f0700 1 -- 172.21.15.59:0/2969724069 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff96015e1c0 msgr2=0x7ff9600cd680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:57:44.568 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:44.566+0000 7ff9664f0700 1 --2- 172.21.15.59:0/2969724069 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff96015e1c0 0x7ff9600cd680 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7ff95800c870 tx=0x7ff95800cc30 comp rx=0 tx=0).stop 2023-11-24T07:57:44.568 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:44.566+0000 7ff9664f0700 1 -- 172.21.15.59:0/2969724069 shutdown_connections 2023-11-24T07:57:44.568 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:44.566+0000 7ff9664f0700 1 --2- 172.21.15.59:0/2969724069 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7ff94c06af60 0x7ff94c06d420 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:44.568 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:44.566+0000 7ff9664f0700 1 --2- 172.21.15.59:0/2969724069 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff96015e1c0 0x7ff9600cd680 unknown :-1 s=CLOSED pgs=445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:44.569 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:44.566+0000 7ff9664f0700 1 --2- 172.21.15.59:0/2969724069 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7ff96006fc50 0x7ff9600b1f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:44.569 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:44.566+0000 7ff9664f0700 1 --2- 172.21.15.59:0/2969724069 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff9600b0e90 0x7ff9601f33a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:44.569 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:44.566+0000 7ff9664f0700 1 -- 172.21.15.59:0/2969724069 >> 172.21.15.59:0/2969724069 conn(0x7ff9600b3720 msgr2=0x7ff96006d520 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:57:44.569 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:44.566+0000 7ff9664f0700 1 -- 172.21.15.59:0/2969724069 shutdown_connections 2023-11-24T07:57:44.569 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:44.566+0000 7ff9664f0700 1 -- 172.21.15.59:0/2969724069 wait complete. 2023-11-24T07:57:44.580 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-24T07:57:44.710 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:44.708+0000 7f32b2656700 1 Processor -- start 2023-11-24T07:57:44.710 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:44.708+0000 7f32b2656700 1 -- start start 2023-11-24T07:57:44.713 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:44.709+0000 7f32b2656700 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f32ac0cfde0 0x7f32ac0d24d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:57:44.714 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:44.709+0000 7f32b2656700 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f32ac0d2a10 0x7f32ac0b4ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:57:44.714 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:44.709+0000 7f32b2656700 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f32ac0b7810 con 0x7f32ac15e190 2023-11-24T07:57:44.714 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:44.709+0000 7f32b2656700 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f32ac0d1930 con 0x7f32ac0d2a10 2023-11-24T07:57:44.714 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:44.709+0000 7f32b2656700 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f32ac0d2ea0 con 0x7f32ac0cfde0 2023-11-24T07:57:44.714 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:44.709+0000 7f32abfff700 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7f32ac15e190 0x7f32ac0d0f50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:55636/0 (socket says 172.21.15.59:55636) 2023-11-24T07:57:44.714 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:44.709+0000 7f32abfff700 1 -- 172.21.15.59:0/3402886324 learned_addr learned my addr 172.21.15.59:0/3402886324 (peer_addr_for_me v1:172.21.15.59:0/0) 2023-11-24T07:57:44.714 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:44.709+0000 7f32ab7fe700 1 --2- 172.21.15.59:0/3402886324 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f32ac0cfde0 0x7f32ac0d24d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:57:44.714 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:44.709+0000 7f32b0bf3700 1 --2- 172.21.15.59:0/3402886324 >> v2:172.21.15.76:3300/0 conn(0x7f32ac0d2a10 0x7f32ac0b4ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:57:44.714 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:44.709+0000 7f32aaffd700 1 -- 172.21.15.59:0/3402886324 <== mon.0 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4108407708 0 0) 0x7f32ac0b7810 con 0x7f32ac15e190 2023-11-24T07:57:44.715 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:44.709+0000 7f32aaffd700 1 -- 172.21.15.59:0/3402886324 --> v1:172.21.15.59:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f32980035b0 con 0x7f32ac15e190 2023-11-24T07:57:44.715 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:44.710+0000 7f32aaffd700 1 -- 172.21.15.59:0/3402886324 <== mon.0 v1:172.21.15.59:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 436020562 0 0) 0x7f32980035b0 con 0x7f32ac15e190 2023-11-24T07:57:44.715 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:44.710+0000 7f32aaffd700 1 -- 172.21.15.59:0/3402886324 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f32ac0cfde0 msgr2=0x7f32ac0d24d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T07:57:44.715 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:44.710+0000 7f32aaffd700 1 --2- 172.21.15.59:0/3402886324 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f32ac0cfde0 0x7f32ac0d24d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:44.715 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:44.710+0000 7f32aaffd700 1 -- 172.21.15.59:0/3402886324 >> v2:172.21.15.76:3300/0 conn(0x7f32ac0d2a10 msgr2=0x7f32ac0b4ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T07:57:44.715 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:44.710+0000 7f32aaffd700 1 --2- 172.21.15.59:0/3402886324 >> v2:172.21.15.76:3300/0 conn(0x7f32ac0d2a10 0x7f32ac0b4ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:44.715 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:44.710+0000 7f32aaffd700 1 -- 172.21.15.59:0/3402886324 --> v1:172.21.15.59:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f32ac0d1660 con 0x7f32ac15e190 2023-11-24T07:57:44.715 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:44.710+0000 7f32aaffd700 1 -- 172.21.15.59:0/3402886324 <== mon.0 v1:172.21.15.59:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1312333566 0 0) 0x7f3294001670 con 0x7f32ac15e190 2023-11-24T07:57:44.715 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:44.710+0000 7f32aaffd700 1 -- 172.21.15.59:0/3402886324 >> v1:172.21.15.59:6789/0 conn(0x7f32ac15e190 legacy=0x7f32ac0d0f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:57:44.715 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:44.710+0000 7f32aaffd700 1 --2- 172.21.15.59:0/3402886324 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3298003a00 0x7f3298003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:57:44.715 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:44.710+0000 7f32aaffd700 1 --2- 172.21.15.59:0/3402886324 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f32ac0cfde0 0x7f3298004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:57:44.715 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:44.710+0000 7f32aaffd700 1 --2- 172.21.15.59:0/3402886324 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3298004d90 0x7f3298005210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:57:44.715 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:44.710+0000 7f32aaffd700 1 -- 172.21.15.59:0/3402886324 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32ac0d1930 con 0x7f3298004d90 2023-11-24T07:57:44.715 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:44.710+0000 7f32aaffd700 1 -- 172.21.15.59:0/3402886324 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32ac0d2ea0 con 0x7f3298003a00 2023-11-24T07:57:44.716 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:44.710+0000 7f32aaffd700 1 -- 172.21.15.59:0/3402886324 --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f329800ba10 con 0x7f32ac0cfde0 2023-11-24T07:57:44.716 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:44.710+0000 7f32b0bf3700 1 --2- 172.21.15.59:0/3402886324 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3298004d90 0x7f3298005210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:57:44.716 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:44.710+0000 7f32abfff700 1 --2- 172.21.15.59:0/3402886324 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3298003a00 0x7f3298003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:57:44.716 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:44.710+0000 7f32ab7fe700 1 --2- 172.21.15.59:0/3402886324 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f32ac0cfde0 0x7f3298004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:57:44.716 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:44.710+0000 7f32b0bf3700 1 -- 172.21.15.59:0/3402886324 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f32ac0cfde0 msgr2=0x7f3298004850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T07:57:44.717 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:44.710+0000 7f32b0bf3700 1 --2- 172.21.15.59:0/3402886324 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f32ac0cfde0 0x7f3298004850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:44.717 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:44.710+0000 7f32b0bf3700 1 -- 172.21.15.59:0/3402886324 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3298003a00 msgr2=0x7f3298003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T07:57:44.718 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:44.710+0000 7f32b0bf3700 1 --2- 172.21.15.59:0/3402886324 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3298003a00 0x7f3298003e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:44.718 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:44.710+0000 7f32b0bf3700 1 -- 172.21.15.59:0/3402886324 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f32980035b0 con 0x7f3298004d90 2023-11-24T07:57:44.718 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:44.710+0000 7f32b0bf3700 1 --2- 172.21.15.59:0/3402886324 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3298004d90 0x7f3298005210 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f32a4009fc0 tx=0x7f3298005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=337f3672fba34e5d server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:57:44.718 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:44.711+0000 7f32aaffd700 1 -- 172.21.15.59:0/3402886324 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32a400ec30 con 0x7f3298004d90 2023-11-24T07:57:44.718 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:44.711+0000 7f32aaffd700 1 -- 172.21.15.59:0/3402886324 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f32a400f040 con 0x7f3298004d90 2023-11-24T07:57:44.718 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:44.711+0000 7f32aaffd700 1 -- 172.21.15.59:0/3402886324 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32a4014480 con 0x7f3298004d90 2023-11-24T07:57:44.718 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:44.711+0000 7f32b2656700 1 -- 172.21.15.59:0/3402886324 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3298004d90 msgr2=0x7f3298005210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:57:44.718 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:44.711+0000 7f32b2656700 1 --2- 172.21.15.59:0/3402886324 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3298004d90 0x7f3298005210 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f32a4009fc0 tx=0x7f3298005880 comp rx=0 tx=0).stop 2023-11-24T07:57:44.718 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:44.711+0000 7f32b0bf3700 1 -- 172.21.15.59:0/3402886324 reap_dead start 2023-11-24T07:57:44.719 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:44.711+0000 7f32b2656700 1 -- 172.21.15.59:0/3402886324 shutdown_connections 2023-11-24T07:57:44.719 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:44.711+0000 7f32b2656700 1 -- 172.21.15.59:0/3402886324 >> 172.21.15.59:0/3402886324 conn(0x7f32ac09d180 msgr2=0x7f32ac0d3ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:57:44.719 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:44.711+0000 7f32b2656700 1 -- 172.21.15.59:0/3402886324 shutdown_connections 2023-11-24T07:57:44.719 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:44.711+0000 7f32b2656700 1 -- 172.21.15.59:0/3402886324 wait complete. 2023-11-24T07:57:44.721 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:44.712+0000 7f32b2656700 1 Processor -- start 2023-11-24T07:57:44.721 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:44.713+0000 7f32b2656700 1 -- start start 2023-11-24T07:57:44.721 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:44.713+0000 7f32b2656700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f32ac1f3300 0x7f32ac0c9b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:57:44.721 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:44.713+0000 7f32b2656700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f32ac1f4320 0x7f32ac0b86a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:57:44.722 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:44.713+0000 7f32b2656700 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f32ac0b8be0 0x7f32ac0c7970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:57:44.722 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:44.713+0000 7f32b2656700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32ac0b9070 con 0x7f32ac1f3300 2023-11-24T07:57:44.722 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:44.713+0000 7f32b2656700 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32ac1f4800 con 0x7f32ac1f4320 2023-11-24T07:57:44.722 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:44.713+0000 7f32b2656700 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f32ac1f36e0 con 0x7f32ac0b8be0 2023-11-24T07:57:44.722 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:44.713+0000 7f32abfff700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f32ac1f3300 0x7f32ac0c9b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:57:44.722 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:44.714+0000 7f32abfff700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f32ac1f3300 0x7f32ac0c9b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:52174/0 (socket says 172.21.15.59:52174) 2023-11-24T07:57:44.722 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:44.714+0000 7f32abfff700 1 -- 172.21.15.59:0/2305588346 learned_addr learned my addr 172.21.15.59:0/2305588346 (peer_addr_for_me v2:172.21.15.59:0/0) 2023-11-24T07:57:44.722 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:44.714+0000 7f32ab7fe700 1 --2- 172.21.15.59:0/2305588346 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f32ac1f4320 0x7f32ac0b86a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:57:44.722 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:44.714+0000 7f32b0bf3700 1 --2- 172.21.15.59:0/2305588346 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f32ac0b8be0 0x7f32ac0c7970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:57:44.722 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:44.714+0000 7f32abfff700 1 -- 172.21.15.59:0/2305588346 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f32ac0b8be0 msgr2=0x7f32ac0c7970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:57:44.722 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:44.714+0000 7f32abfff700 1 --2- 172.21.15.59:0/2305588346 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f32ac0b8be0 0x7f32ac0c7970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:44.722 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:44.714+0000 7f32abfff700 1 -- 172.21.15.59:0/2305588346 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f32ac1f4320 msgr2=0x7f32ac0b86a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:57:44.722 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:44.714+0000 7f32abfff700 1 --2- 172.21.15.59:0/2305588346 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f32ac1f4320 0x7f32ac0b86a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:44.723 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:44.714+0000 7f32abfff700 1 -- 172.21.15.59:0/2305588346 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f32ac1f6f10 con 0x7f32ac1f3300 2023-11-24T07:57:44.723 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:44.714+0000 7f32abfff700 1 --2- 172.21.15.59:0/2305588346 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f32ac1f3300 0x7f32ac0c9b00 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f3294001e70 tx=0x7f32940076a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:57:44.723 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:44.714+0000 7f32a8ff9700 1 -- 172.21.15.59:0/2305588346 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f329400f8a0 con 0x7f32ac1f3300 2023-11-24T07:57:44.723 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:44.714+0000 7f32a8ff9700 1 -- 172.21.15.59:0/2305588346 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f329400fa20 con 0x7f32ac1f3300 2023-11-24T07:57:44.723 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:44.714+0000 7f32a8ff9700 1 -- 172.21.15.59:0/2305588346 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f329400fdd0 con 0x7f32ac1f3300 2023-11-24T07:57:44.723 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:44.714+0000 7f32b2656700 1 -- 172.21.15.59:0/2305588346 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f32ac1f71f0 con 0x7f32ac1f3300 2023-11-24T07:57:44.723 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:44.714+0000 7f32b2656700 1 -- 172.21.15.59:0/2305588346 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f32ac1f77b0 con 0x7f32ac1f3300 2023-11-24T07:57:44.723 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:44.715+0000 7f32b2656700 1 -- 172.21.15.59:0/2305588346 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f32ac1f7970 con 0x7f32ac1f3300 2023-11-24T07:57:44.723 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:44.716+0000 7f32a8ff9700 1 -- 172.21.15.59:0/2305588346 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 6) v1 ==== 89282+0+0 (secure 0 0 0) 0x7f329401d460 con 0x7f32ac1f3300 2023-11-24T07:57:44.723 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:44.716+0000 7f32a8ff9700 1 --2- 172.21.15.59:0/2305588346 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f328c06af60 0x7f328c06d420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:57:44.723 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:44.716+0000 7f32a8ff9700 1 -- 172.21.15.59:0/2305588346 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f3294089680 con 0x7f32ac1f3300 2023-11-24T07:57:44.723 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:44.717+0000 7f32ab7fe700 1 --2- 172.21.15.59:0/2305588346 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f328c06af60 0x7f328c06d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:57:44.723 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:44.717+0000 7f32ab7fe700 1 --2- 172.21.15.59:0/2305588346 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f328c06af60 0x7f328c06d420 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f32ac0c99c0 tx=0x7f329c008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:57:44.724 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:44.721+0000 7f32a8ff9700 1 -- 172.21.15.59:0/2305588346 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f32940547d0 con 0x7f32ac1f3300 2023-11-24T07:57:44.924 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:44.922+0000 7f32b2656700 1 -- 172.21.15.59:0/2305588346 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f32ac075480 con 0x7f32ac1f3300 2023-11-24T07:57:44.924 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:44.922+0000 7f32a8ff9700 1 -- 172.21.15.59:0/2305588346 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v33) v1 ==== 100+0+11 (secure 0 0 0) 0x7f3294057df0 con 0x7f32ac1f3300 2023-11-24T07:57:44.924 INFO:teuthology.orchestra.run.smithi059.stdout:pg_num: 64 2023-11-24T07:57:44.926 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:44.924+0000 7f32b2656700 1 -- 172.21.15.59:0/2305588346 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f328c06af60 msgr2=0x7f328c06d420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:57:44.926 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:44.924+0000 7f32b2656700 1 --2- 172.21.15.59:0/2305588346 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f328c06af60 0x7f328c06d420 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f32ac0c99c0 tx=0x7f329c008040 comp rx=0 tx=0).stop 2023-11-24T07:57:44.926 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:44.924+0000 7f32b2656700 1 -- 172.21.15.59:0/2305588346 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f32ac1f3300 msgr2=0x7f32ac0c9b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:57:44.927 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:44.924+0000 7f32b2656700 1 --2- 172.21.15.59:0/2305588346 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f32ac1f3300 0x7f32ac0c9b00 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f3294001e70 tx=0x7f32940076a0 comp rx=0 tx=0).stop 2023-11-24T07:57:44.927 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:44.924+0000 7f32b2656700 1 -- 172.21.15.59:0/2305588346 shutdown_connections 2023-11-24T07:57:44.927 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:44.924+0000 7f32b2656700 1 --2- 172.21.15.59:0/2305588346 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f328c06af60 0x7f328c06d420 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:44.927 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:44.924+0000 7f32b2656700 1 --2- 172.21.15.59:0/2305588346 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f32ac0b8be0 0x7f32ac0c7970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:44.927 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:44.924+0000 7f32b2656700 1 --2- 172.21.15.59:0/2305588346 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f32ac1f3300 0x7f32ac0c9b00 unknown :-1 s=CLOSED pgs=448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:44.927 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:44.924+0000 7f32b2656700 1 --2- 172.21.15.59:0/2305588346 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f32ac1f4320 0x7f32ac0b86a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:44.927 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:44.924+0000 7f32b2656700 1 -- 172.21.15.59:0/2305588346 >> 172.21.15.59:0/2305588346 conn(0x7f32ac09d180 msgr2=0x7f32ac15d2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:57:44.927 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:44.924+0000 7f32b2656700 1 -- 172.21.15.59:0/2305588346 shutdown_connections 2023-11-24T07:57:44.927 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:44.924+0000 7f32b2656700 1 -- 172.21.15.59:0/2305588346 wait complete. 2023-11-24T07:57:44.939 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-11-24T07:57:45.065 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:45.064+0000 7fcdf8d3c700 1 Processor -- start 2023-11-24T07:57:45.066 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:45.064+0000 7fcdf8d3c700 1 -- start start 2023-11-24T07:57:45.069 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:45.065+0000 7fcdf8d3c700 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fcdf406fc50 0x7fcdf406eef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:57:45.069 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:45.065+0000 7fcdf8d3c700 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcdf40d1230 con 0x7fcdf40b0e90 2023-11-24T07:57:45.069 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:45.065+0000 7fcdf8d3c700 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcdf40d0c70 con 0x7fcdf415e1c0 2023-11-24T07:57:45.069 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:45.065+0000 7fcdf8d3c700 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcdf40d0f50 con 0x7fcdf406fc50 2023-11-24T07:57:45.069 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:45.065+0000 7fcdf2d9d700 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7fcdf40b0e90 0x7fcdf406ca10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:55660/0 (socket says 172.21.15.59:55660) 2023-11-24T07:57:45.069 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:45.065+0000 7fcdf2d9d700 1 -- 172.21.15.59:0/2371913094 learned_addr learned my addr 172.21.15.59:0/2371913094 (peer_addr_for_me v1:172.21.15.59:0/0) 2023-11-24T07:57:45.070 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:45.065+0000 7fcdf1d9b700 1 --2- 172.21.15.59:0/2371913094 >> v2:172.21.15.76:3300/0 conn(0x7fcdf406fc50 0x7fcdf406eef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:57:45.070 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:45.065+0000 7fcdf159a700 1 -- 172.21.15.59:0/2371913094 <== mon.0 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3591034098 0 0) 0x7fcdf40d1230 con 0x7fcdf40b0e90 2023-11-24T07:57:45.070 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:45.065+0000 7fcdf159a700 1 -- 172.21.15.59:0/2371913094 --> v1:172.21.15.59:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcddc0035b0 con 0x7fcdf40b0e90 2023-11-24T07:57:45.070 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:45.065+0000 7fcdf159a700 1 -- 172.21.15.59:0/2371913094 <== mon.0 v1:172.21.15.59:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1546831043 0 0) 0x7fcddc0035b0 con 0x7fcdf40b0e90 2023-11-24T07:57:45.070 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:45.065+0000 7fcdf159a700 1 -- 172.21.15.59:0/2371913094 >> v2:172.21.15.76:3300/0 conn(0x7fcdf406fc50 msgr2=0x7fcdf406eef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T07:57:45.070 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:45.065+0000 7fcdf159a700 1 --2- 172.21.15.59:0/2371913094 >> v2:172.21.15.76:3300/0 conn(0x7fcdf406fc50 0x7fcdf406eef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:45.070 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:45.065+0000 7fcdf159a700 1 -- 172.21.15.59:0/2371913094 >> v1:172.21.15.76:6789/0 conn(0x7fcdf415e1c0 legacy=0x7fcdf406f540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:57:45.070 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:45.065+0000 7fcdf159a700 1 -- 172.21.15.59:0/2371913094 --> v1:172.21.15.59:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcdf406c210 con 0x7fcdf40b0e90 2023-11-24T07:57:45.070 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:45.065+0000 7fcdf159a700 1 -- 172.21.15.59:0/2371913094 <== mon.0 v1:172.21.15.59:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1312333566 0 0) 0x7fcdec0019f0 con 0x7fcdf40b0e90 2023-11-24T07:57:45.070 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:45.065+0000 7fcdf159a700 1 -- 172.21.15.59:0/2371913094 >> v1:172.21.15.59:6789/0 conn(0x7fcdf40b0e90 legacy=0x7fcdf406ca10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:57:45.070 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:45.065+0000 7fcdf159a700 1 --2- 172.21.15.59:0/2371913094 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fcddc003a00 0x7fcddc003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:57:45.070 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:45.066+0000 7fcdf159a700 1 --2- 172.21.15.59:0/2371913094 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fcddc004850 0x7fcddc004cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:57:45.071 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:45.066+0000 7fcdf159a700 1 --2- 172.21.15.59:0/2371913094 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fcddc005210 0x7fcddc00ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:57:45.071 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:45.066+0000 7fcdf159a700 1 -- 172.21.15.59:0/2371913094 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcdf40d0f50 con 0x7fcddc005210 2023-11-24T07:57:45.071 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:45.066+0000 7fcdf159a700 1 -- 172.21.15.59:0/2371913094 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcddc00c1b0 con 0x7fcddc003a00 2023-11-24T07:57:45.071 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:45.066+0000 7fcdf159a700 1 -- 172.21.15.59:0/2371913094 --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcddc00c370 con 0x7fcddc004850 2023-11-24T07:57:45.071 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:45.066+0000 7fcdf2d9d700 1 --2- 172.21.15.59:0/2371913094 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fcddc005210 0x7fcddc00ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:57:45.071 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:45.066+0000 7fcdf259c700 1 --2- 172.21.15.59:0/2371913094 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fcddc003a00 0x7fcddc003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:57:45.071 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:45.066+0000 7fcdf1d9b700 1 --2- 172.21.15.59:0/2371913094 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fcddc004850 0x7fcddc004cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:57:45.071 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:45.066+0000 7fcdf2d9d700 1 -- 172.21.15.59:0/2371913094 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fcddc004850 msgr2=0x7fcddc004cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T07:57:45.071 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:45.066+0000 7fcdf2d9d700 1 --2- 172.21.15.59:0/2371913094 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fcddc004850 0x7fcddc004cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:45.071 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:45.066+0000 7fcdf2d9d700 1 -- 172.21.15.59:0/2371913094 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fcddc003a00 msgr2=0x7fcddc003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T07:57:45.071 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:45.066+0000 7fcdf2d9d700 1 --2- 172.21.15.59:0/2371913094 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fcddc003a00 0x7fcddc003e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:45.072 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:45.066+0000 7fcdf2d9d700 1 -- 172.21.15.59:0/2371913094 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcddc0035b0 con 0x7fcddc005210 2023-11-24T07:57:45.072 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:45.066+0000 7fcdf2d9d700 1 -- 172.21.15.59:0/2371913094 reap_dead start 2023-11-24T07:57:45.072 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:45.066+0000 7fcdf2d9d700 1 --2- 172.21.15.59:0/2371913094 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fcddc005210 0x7fcddc00ba10 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7fcdec00a800 tx=0x7fcddc005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ff0d2a5cf67fada3 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:57:45.072 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:45.066+0000 7fcdf159a700 1 -- 172.21.15.59:0/2371913094 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcdec00a2f0 con 0x7fcddc005210 2023-11-24T07:57:45.072 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:45.066+0000 7fcdf159a700 1 -- 172.21.15.59:0/2371913094 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcdec012620 con 0x7fcddc005210 2023-11-24T07:57:45.072 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:45.066+0000 7fcdf159a700 1 -- 172.21.15.59:0/2371913094 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcdec0129d0 con 0x7fcddc005210 2023-11-24T07:57:45.072 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:45.067+0000 7fcdf8d3c700 1 -- 172.21.15.59:0/2371913094 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fcddc005210 msgr2=0x7fcddc00ba10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:57:45.072 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:45.067+0000 7fcdf8d3c700 1 --2- 172.21.15.59:0/2371913094 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fcddc005210 0x7fcddc00ba10 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7fcdec00a800 tx=0x7fcddc005880 comp rx=0 tx=0).stop 2023-11-24T07:57:45.072 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:45.067+0000 7fcdf8d3c700 1 -- 172.21.15.59:0/2371913094 shutdown_connections 2023-11-24T07:57:45.072 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:45.067+0000 7fcdf8d3c700 1 --2- 172.21.15.59:0/2371913094 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fcddc005210 0x7fcddc00ba10 unknown :-1 s=CLOSED pgs=450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:45.072 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:45.067+0000 7fcdf8d3c700 1 -- 172.21.15.59:0/2371913094 >> 172.21.15.59:0/2371913094 conn(0x7fcdf40b3720 msgr2=0x7fcdf40b3b30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:57:45.072 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:45.067+0000 7fcdf8d3c700 1 -- 172.21.15.59:0/2371913094 shutdown_connections 2023-11-24T07:57:45.073 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:45.067+0000 7fcdf8d3c700 1 -- 172.21.15.59:0/2371913094 wait complete. 2023-11-24T07:57:45.075 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:45.068+0000 7fcdf8d3c700 1 Processor -- start 2023-11-24T07:57:45.075 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:45.068+0000 7fcdf8d3c700 1 -- start start 2023-11-24T07:57:45.075 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:45.069+0000 7fcdf8d3c700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fcddc005210 0x7fcdf40b7120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:57:45.076 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:45.069+0000 7fcdf8d3c700 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fcdf40c7e90 0x7fcdf40c5850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:57:45.076 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:45.069+0000 7fcdf8d3c700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fcdf4147ef0 0x7fcdf4148370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:57:45.076 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:45.069+0000 7fcdf8d3c700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcdf41488b0 con 0x7fcdf4147ef0 2023-11-24T07:57:45.076 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:45.069+0000 7fcdf8d3c700 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcdf4156ec0 con 0x7fcddc005210 2023-11-24T07:57:45.076 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:45.069+0000 7fcdf8d3c700 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcdf40b7660 con 0x7fcdf40c7e90 2023-11-24T07:57:45.076 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:45.069+0000 7fcdf2d9d700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fcdf4147ef0 0x7fcdf4148370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:57:45.076 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:45.069+0000 7fcdf2d9d700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fcdf4147ef0 0x7fcdf4148370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:52198/0 (socket says 172.21.15.59:52198) 2023-11-24T07:57:45.076 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:45.069+0000 7fcdf2d9d700 1 -- 172.21.15.59:0/641731659 learned_addr learned my addr 172.21.15.59:0/641731659 (peer_addr_for_me v2:172.21.15.59:0/0) 2023-11-24T07:57:45.076 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:45.069+0000 7fcdf259c700 1 --2- 172.21.15.59:0/641731659 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fcddc005210 0x7fcdf40b7120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:57:45.076 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:45.069+0000 7fcdf1d9b700 1 --2- 172.21.15.59:0/641731659 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fcdf40c7e90 0x7fcdf40c5850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:57:45.076 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:45.069+0000 7fcdf2d9d700 1 -- 172.21.15.59:0/641731659 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fcdf40c7e90 msgr2=0x7fcdf40c5850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:57:45.076 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:45.069+0000 7fcdf2d9d700 1 --2- 172.21.15.59:0/641731659 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fcdf40c7e90 0x7fcdf40c5850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:45.077 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:45.069+0000 7fcdf2d9d700 1 -- 172.21.15.59:0/641731659 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fcddc005210 msgr2=0x7fcdf40b7120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:57:45.077 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:45.069+0000 7fcdf2d9d700 1 --2- 172.21.15.59:0/641731659 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fcddc005210 0x7fcdf40b7120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:45.077 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:45.069+0000 7fcdf2d9d700 1 -- 172.21.15.59:0/641731659 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcdf4156b90 con 0x7fcdf4147ef0 2023-11-24T07:57:45.077 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:45.070+0000 7fcdf259c700 1 --2- 172.21.15.59:0/641731659 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fcddc005210 0x7fcdf40b7120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T07:57:45.077 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:45.070+0000 7fcdf1d9b700 1 --2- 172.21.15.59:0/641731659 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fcdf40c7e90 0x7fcdf40c5850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T07:57:45.077 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:45.070+0000 7fcdf2d9d700 1 --2- 172.21.15.59:0/641731659 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fcdf4147ef0 0x7fcdf4148370 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7fcdf4073110 tx=0x7fcdec001d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:57:45.077 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:45.070+0000 7fcde2ffd700 1 -- 172.21.15.59:0/641731659 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcdec024070 con 0x7fcdf4147ef0 2023-11-24T07:57:45.077 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:45.070+0000 7fcde2ffd700 1 -- 172.21.15.59:0/641731659 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcdec013a30 con 0x7fcdf4147ef0 2023-11-24T07:57:45.077 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:45.070+0000 7fcdf8d3c700 1 -- 172.21.15.59:0/641731659 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcdf40c83f0 con 0x7fcdf4147ef0 2023-11-24T07:57:45.077 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:45.070+0000 7fcde2ffd700 1 -- 172.21.15.59:0/641731659 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcdec013de0 con 0x7fcdf4147ef0 2023-11-24T07:57:45.077 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:45.070+0000 7fcdf8d3c700 1 -- 172.21.15.59:0/641731659 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcdf40c89b0 con 0x7fcdf4147ef0 2023-11-24T07:57:45.077 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:45.071+0000 7fcdf8d3c700 1 -- 172.21.15.59:0/641731659 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcdf4068670 con 0x7fcdf4147ef0 2023-11-24T07:57:45.078 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:45.071+0000 7fcde2ffd700 1 -- 172.21.15.59:0/641731659 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 6) v1 ==== 89282+0+0 (secure 0 0 0) 0x7fcdec01d420 con 0x7fcdf4147ef0 2023-11-24T07:57:45.078 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:45.072+0000 7fcde2ffd700 1 --2- 172.21.15.59:0/641731659 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7fcdd406af60 0x7fcdd406d420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:57:45.078 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:45.072+0000 7fcde2ffd700 1 -- 172.21.15.59:0/641731659 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fcdec08aab0 con 0x7fcdf4147ef0 2023-11-24T07:57:45.078 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:45.072+0000 7fcdf259c700 1 --2- 172.21.15.59:0/641731659 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7fcdd406af60 0x7fcdd406d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:57:45.078 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:45.073+0000 7fcdf259c700 1 --2- 172.21.15.59:0/641731659 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7fcdd406af60 0x7fcdd406d420 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7fcde4000c00 tx=0x7fcde4009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:57:45.078 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:45.076+0000 7fcde2ffd700 1 -- 172.21.15.59:0/641731659 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fcdec055b80 con 0x7fcdf4147ef0 2023-11-24T07:57:45.295 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:45.293+0000 7fcdf8d3c700 1 -- 172.21.15.59:0/641731659 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7fcdf40c54b0 con 0x7fcdf4147ef0 2023-11-24T07:57:45.295 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:45.294+0000 7fcde2ffd700 1 -- 172.21.15.59:0/641731659 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v33) v1 ==== 80+0+0 (secure 0 0 0) 0x7fcdec0591a0 con 0x7fcdf4147ef0 2023-11-24T07:57:45.298 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:45.296+0000 7fcdf8d3c700 1 -- 172.21.15.59:0/641731659 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7fcdd406af60 msgr2=0x7fcdd406d420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:57:45.298 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:45.296+0000 7fcdf8d3c700 1 --2- 172.21.15.59:0/641731659 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7fcdd406af60 0x7fcdd406d420 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7fcde4000c00 tx=0x7fcde4009040 comp rx=0 tx=0).stop 2023-11-24T07:57:45.298 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:45.296+0000 7fcdf8d3c700 1 -- 172.21.15.59:0/641731659 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fcdf4147ef0 msgr2=0x7fcdf4148370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:57:45.298 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:45.296+0000 7fcdf8d3c700 1 --2- 172.21.15.59:0/641731659 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fcdf4147ef0 0x7fcdf4148370 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7fcdf4073110 tx=0x7fcdec001d80 comp rx=0 tx=0).stop 2023-11-24T07:57:45.299 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:45.296+0000 7fcdf8d3c700 1 -- 172.21.15.59:0/641731659 shutdown_connections 2023-11-24T07:57:45.299 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:45.296+0000 7fcdf8d3c700 1 --2- 172.21.15.59:0/641731659 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7fcdd406af60 0x7fcdd406d420 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:45.299 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:45.296+0000 7fcdf8d3c700 1 --2- 172.21.15.59:0/641731659 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fcdf4147ef0 0x7fcdf4148370 unknown :-1 s=CLOSED pgs=451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:45.299 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:45.296+0000 7fcdf8d3c700 1 --2- 172.21.15.59:0/641731659 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fcddc005210 0x7fcdf40b7120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:45.299 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:45.296+0000 7fcdf8d3c700 1 --2- 172.21.15.59:0/641731659 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fcdf40c7e90 0x7fcdf40c5850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:45.299 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:45.296+0000 7fcdf8d3c700 1 -- 172.21.15.59:0/641731659 >> 172.21.15.59:0/641731659 conn(0x7fcdf40b3720 msgr2=0x7fcdf4070190 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:57:45.299 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:45.296+0000 7fcdf8d3c700 1 -- 172.21.15.59:0/641731659 shutdown_connections 2023-11-24T07:57:45.299 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:45.296+0000 7fcdf8d3c700 1 -- 172.21.15.59:0/641731659 wait complete. 2023-11-24T07:57:45.300 INFO:teuthology.orchestra.run.smithi059.stderr:listed 0 entries 2023-11-24T07:57:45.309 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.1... 2023-11-24T07:57:45.310 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T07:57:45.310 DEBUG:teuthology.orchestra.run.smithi076:> sudo umount /home/ubuntu/cephtest/mnt.1 2023-11-24T07:57:45.355 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi076.front.sepia.ceph.com 2023-11-24T07:57:45.355 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T07:57:45.355 DEBUG:teuthology.orchestra.run.smithi076:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2023-11-24T07:57:45.374 DEBUG:teuthology.orchestra.run.smithi059:> set -ex 2023-11-24T07:57:45.374 DEBUG:teuthology.orchestra.run.smithi059:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-24T07:57:45.392 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=16287364k,nr_inodes=4071841,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 tmpfs /sys/fs/cgroup tmpfs ro,seclabel,nosuid,nodev,noexec,mode=755 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,seclabel,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,seclabel,nosuid,nodev,noexec,relatime,hugetlb 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,seclabel,nosuid,nodev,noexec,relatime,perf_event 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,seclabel,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/rdma cgroup rw,seclabel,nosuid,nodev,noexec,relatime,rdma 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,seclabel,nosuid,nodev,noexec,relatime,freezer 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,seclabel,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,seclabel,nosuid,nodev,noexec,relatime,devices 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,blkio 0 0 none /sys/kernel/tracing tracefs rw,seclabel,relatime 0 0 configfs /sys/kernel/config configfs rw,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=42,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=16075 0 0 mqueue /dev/mqueue mqueue rw,seclabel,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,relatime 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3261172k,mode=700,uid=1000,gid=1265 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw,seclabel 0 0 2023-11-24T07:57:45.392 DEBUG:teuthology.orchestra.run.smithi076:> set -ex 2023-11-24T07:57:45.393 DEBUG:teuthology.orchestra.run.smithi076:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-24T07:57:45.433 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=16284272k,nr_inodes=4071068,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 tmpfs /sys/fs/cgroup tmpfs ro,seclabel,nosuid,nodev,noexec,mode=755 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,seclabel,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,seclabel,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,seclabel,nosuid,nodev,noexec,relatime,freezer 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,seclabel,nosuid,nodev,noexec,relatime,perf_event 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,seclabel,nosuid,nodev,noexec,relatime,hugetlb 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,seclabel,nosuid,nodev,noexec,relatime,devices 0 0 cgroup /sys/fs/cgroup/rdma cgroup rw,seclabel,nosuid,nodev,noexec,relatime,rdma 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,seclabel,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,blkio 0 0 none /sys/kernel/tracing tracefs rw,seclabel,relatime 0 0 configfs /sys/kernel/config configfs rw,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=35,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=20116 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,relatime 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3260552k,mode=700,uid=1000,gid=1265 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw,seclabel 0 0 2023-11-24T07:57:45.434 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-24T07:57:45.560 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:45.558+0000 7fde856c4700 1 Processor -- start 2023-11-24T07:57:45.560 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:45.558+0000 7fde856c4700 1 -- start start 2023-11-24T07:57:45.563 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:45.559+0000 7fde856c4700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fde8006fc50 0x7fde8006eef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:57:45.563 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:45.559+0000 7fde856c4700 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fde800d1230 con 0x7fde800b0e90 2023-11-24T07:57:45.564 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:45.559+0000 7fde856c4700 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fde800d0c70 con 0x7fde8015e1c0 2023-11-24T07:57:45.564 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:45.559+0000 7fde856c4700 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fde800d0f50 con 0x7fde8006fc50 2023-11-24T07:57:45.564 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:45.559+0000 7fde7e7fc700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fde8006fc50 0x7fde8006eef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:57:45.564 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:45.559+0000 7fde7f7fe700 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7fde800b0e90 0x7fde8006ca10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:55680/0 (socket says 172.21.15.59:55680) 2023-11-24T07:57:45.564 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:45.559+0000 7fde7e7fc700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fde8006fc50 0x7fde8006eef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:52204/0 (socket says 172.21.15.59:52204) 2023-11-24T07:57:45.564 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:45.559+0000 7fde7f7fe700 1 -- 172.21.15.59:0/108443319 learned_addr learned my addr 172.21.15.59:0/108443319 (peer_addr_for_me v1:172.21.15.59:0/0) 2023-11-24T07:57:45.564 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:45.560+0000 7fde7e7fc700 1 -- 172.21.15.59:0/108443319 >> v1:172.21.15.76:6789/0 conn(0x7fde8015e1c0 legacy=0x7fde8006f540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:57:45.565 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:45.560+0000 7fde7e7fc700 1 -- 172.21.15.59:0/108443319 >> v1:172.21.15.59:6789/0 conn(0x7fde800b0e90 legacy=0x7fde8006ca10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:57:45.565 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:45.560+0000 7fde7e7fc700 1 -- 172.21.15.59:0/108443319 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fde8006c210 con 0x7fde8006fc50 2023-11-24T07:57:45.565 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:45.560+0000 7fde7e7fc700 1 --2- 172.21.15.59:0/108443319 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fde8006fc50 0x7fde8006eef0 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7fde70009840 tx=0x7fde70009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=17383b99fcc161fc server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:57:45.565 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:45.560+0000 7fde7dffb700 1 -- 172.21.15.59:0/108443319 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde70011830 con 0x7fde8006fc50 2023-11-24T07:57:45.565 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:45.560+0000 7fde7dffb700 1 -- 172.21.15.59:0/108443319 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fde70011990 con 0x7fde8006fc50 2023-11-24T07:57:45.566 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:45.560+0000 7fde7dffb700 1 -- 172.21.15.59:0/108443319 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde70011d20 con 0x7fde8006fc50 2023-11-24T07:57:45.566 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:45.560+0000 7fde856c4700 1 -- 172.21.15.59:0/108443319 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fde8006fc50 msgr2=0x7fde8006eef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:57:45.566 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:45.560+0000 7fde856c4700 1 --2- 172.21.15.59:0/108443319 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fde8006fc50 0x7fde8006eef0 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7fde70009840 tx=0x7fde70009b50 comp rx=0 tx=0).stop 2023-11-24T07:57:45.566 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:45.560+0000 7fde856c4700 1 -- 172.21.15.59:0/108443319 shutdown_connections 2023-11-24T07:57:45.566 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:45.560+0000 7fde856c4700 1 --2- 172.21.15.59:0/108443319 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fde8006fc50 0x7fde8006eef0 unknown :-1 s=CLOSED pgs=453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:45.566 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:45.560+0000 7fde856c4700 1 -- 172.21.15.59:0/108443319 >> 172.21.15.59:0/108443319 conn(0x7fde800b3720 msgr2=0x7fde800b3b30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:57:45.567 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:45.560+0000 7fde856c4700 1 -- 172.21.15.59:0/108443319 shutdown_connections 2023-11-24T07:57:45.567 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:45.561+0000 7fde856c4700 1 -- 172.21.15.59:0/108443319 wait complete. 2023-11-24T07:57:45.567 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:45.562+0000 7fde856c4700 1 Processor -- start 2023-11-24T07:57:45.567 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:45.562+0000 7fde856c4700 1 -- start start 2023-11-24T07:57:45.570 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:45.563+0000 7fde856c4700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fde8006fc50 0x7fde800b7160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:57:45.570 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:45.563+0000 7fde856c4700 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fde800b0e90 0x7fde8006efe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:57:45.570 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:45.563+0000 7fde856c4700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fde8015e1c0 0x7fde80154d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:57:45.570 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:45.563+0000 7fde856c4700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde80157de0 con 0x7fde8006fc50 2023-11-24T07:57:45.570 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:45.563+0000 7fde856c4700 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde80156f50 con 0x7fde8015e1c0 2023-11-24T07:57:45.571 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:45.563+0000 7fde856c4700 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fde80157160 con 0x7fde800b0e90 2023-11-24T07:57:45.571 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:45.563+0000 7fde7effd700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fde8006fc50 0x7fde800b7160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:57:45.571 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:45.563+0000 7fde7effd700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fde8006fc50 0x7fde800b7160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:52208/0 (socket says 172.21.15.59:52208) 2023-11-24T07:57:45.571 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:45.563+0000 7fde7effd700 1 -- 172.21.15.59:0/2657056707 learned_addr learned my addr 172.21.15.59:0/2657056707 (peer_addr_for_me v2:172.21.15.59:0/0) 2023-11-24T07:57:45.571 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:45.563+0000 7fde7f7fe700 1 --2- 172.21.15.59:0/2657056707 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fde8015e1c0 0x7fde80154d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:57:45.571 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:45.563+0000 7fde7effd700 1 -- 172.21.15.59:0/2657056707 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fde800b0e90 msgr2=0x7fde8006efe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:57:45.572 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:45.563+0000 7fde7effd700 1 --2- 172.21.15.59:0/2657056707 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fde800b0e90 0x7fde8006efe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:45.572 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:45.563+0000 7fde7effd700 1 -- 172.21.15.59:0/2657056707 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fde8015e1c0 msgr2=0x7fde80154d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:57:45.572 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:45.563+0000 7fde7effd700 1 --2- 172.21.15.59:0/2657056707 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fde8015e1c0 0x7fde80154d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:45.573 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:45.563+0000 7fde7effd700 1 -- 172.21.15.59:0/2657056707 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fde700094a0 con 0x7fde8006fc50 2023-11-24T07:57:45.573 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:45.563+0000 7fde7effd700 1 --2- 172.21.15.59:0/2657056707 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fde8006fc50 0x7fde800b7160 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7fde6800c450 tx=0x7fde6800c810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:57:45.573 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:45.563+0000 7fde777fe700 1 -- 172.21.15.59:0/2657056707 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde680052d0 con 0x7fde8006fc50 2023-11-24T07:57:45.573 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:45.563+0000 7fde777fe700 1 -- 172.21.15.59:0/2657056707 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fde68005430 con 0x7fde8006fc50 2023-11-24T07:57:45.573 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:45.563+0000 7fde777fe700 1 -- 172.21.15.59:0/2657056707 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde680038c0 con 0x7fde8006fc50 2023-11-24T07:57:45.573 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:45.563+0000 7fde856c4700 1 -- 172.21.15.59:0/2657056707 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fde80157440 con 0x7fde8006fc50 2023-11-24T07:57:45.573 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:45.563+0000 7fde856c4700 1 -- 172.21.15.59:0/2657056707 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fde80157a00 con 0x7fde8006fc50 2023-11-24T07:57:45.574 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:45.564+0000 7fde856c4700 1 -- 172.21.15.59:0/2657056707 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fde80068670 con 0x7fde8006fc50 2023-11-24T07:57:45.574 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:45.565+0000 7fde777fe700 1 -- 172.21.15.59:0/2657056707 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 6) v1 ==== 89282+0+0 (secure 0 0 0) 0x7fde68003a20 con 0x7fde8006fc50 2023-11-24T07:57:45.574 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:45.565+0000 7fde777fe700 1 --2- 172.21.15.59:0/2657056707 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7fde6006af60 0x7fde6006d420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:57:45.574 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:45.565+0000 7fde777fe700 1 -- 172.21.15.59:0/2657056707 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fde68088970 con 0x7fde8006fc50 2023-11-24T07:57:45.574 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:45.565+0000 7fde7e7fc700 1 --2- 172.21.15.59:0/2657056707 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7fde6006af60 0x7fde6006d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:57:45.574 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:45.566+0000 7fde7e7fc700 1 --2- 172.21.15.59:0/2657056707 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7fde6006af60 0x7fde6006d420 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fde8006d930 tx=0x7fde70010000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:57:45.575 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:45.570+0000 7fde777fe700 1 -- 172.21.15.59:0/2657056707 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fde68053a30 con 0x7fde8006fc50 2023-11-24T07:57:45.818 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:45.816+0000 7fde856c4700 1 -- 172.21.15.59:0/2657056707 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fde800b4240 con 0x7fde8006fc50 2023-11-24T07:57:45.819 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:45.817+0000 7fde777fe700 1 -- 172.21.15.59:0/2657056707 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) v1 ==== 94+0+4913 (secure 0 0 0) 0x7fde68057050 con 0x7fde8006fc50 2023-11-24T07:57:45.819 INFO:teuthology.orchestra.run.smithi059.stdout: 2023-11-24T07:57:45.819 INFO:teuthology.orchestra.run.smithi059.stdout:{"epoch":17,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4763,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.76:6836/1669441817","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6834","nonce":1669441817},{"type":"v1","addr":"172.21.15.76:6836","nonce":1669441817}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":4769,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.76:6837/3709928230","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6835","nonce":3709928230},{"type":"v1","addr":"172.21.15.76:6837","nonce":3709928230}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":5106,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6834/2019319496","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6832","nonce":2019319496},{"type":"v1","addr":"172.21.15.59:6834","nonce":2019319496}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":16,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-24T07:57:25.003636+0000","modified":"2023-11-24T07:57:28.021959+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5109},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5109":{"gid":5109,"name":"a","rank":0,"incarnation":13,"state":"up:active","state_seq":3,"addr":"172.21.15.59:6835/2290655475","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6833","nonce":2290655475},{"type":"v1","addr":"172.21.15.59:6835","nonce":2290655475}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2023-11-24T07:57:45.821 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:45.819+0000 7fde856c4700 1 -- 172.21.15.59:0/2657056707 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7fde6006af60 msgr2=0x7fde6006d420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:57:45.821 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:45.819+0000 7fde856c4700 1 --2- 172.21.15.59:0/2657056707 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7fde6006af60 0x7fde6006d420 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fde8006d930 tx=0x7fde70010000 comp rx=0 tx=0).stop 2023-11-24T07:57:45.821 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:45.819+0000 7fde856c4700 1 -- 172.21.15.59:0/2657056707 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fde8006fc50 msgr2=0x7fde800b7160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:57:45.821 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:45.819+0000 7fde856c4700 1 --2- 172.21.15.59:0/2657056707 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fde8006fc50 0x7fde800b7160 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7fde6800c450 tx=0x7fde6800c810 comp rx=0 tx=0).stop 2023-11-24T07:57:45.822 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:45.819+0000 7fde856c4700 1 -- 172.21.15.59:0/2657056707 shutdown_connections 2023-11-24T07:57:45.822 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:45.819+0000 7fde856c4700 1 --2- 172.21.15.59:0/2657056707 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7fde6006af60 0x7fde6006d420 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:45.822 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:45.819+0000 7fde856c4700 1 --2- 172.21.15.59:0/2657056707 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fde8015e1c0 0x7fde80154d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:45.822 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:45.819+0000 7fde856c4700 1 --2- 172.21.15.59:0/2657056707 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fde8006fc50 0x7fde800b7160 unknown :-1 s=CLOSED pgs=454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:45.822 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:45.819+0000 7fde856c4700 1 --2- 172.21.15.59:0/2657056707 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fde800b0e90 0x7fde8006efe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:45.822 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:45.819+0000 7fde856c4700 1 -- 172.21.15.59:0/2657056707 >> 172.21.15.59:0/2657056707 conn(0x7fde800b3720 msgr2=0x7fde800b1d20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:57:45.822 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:45.819+0000 7fde856c4700 1 -- 172.21.15.59:0/2657056707 shutdown_connections 2023-11-24T07:57:45.823 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:45.819+0000 7fde856c4700 1 -- 172.21.15.59:0/2657056707 wait complete. 2023-11-24T07:57:45.823 INFO:teuthology.orchestra.run.smithi059.stderr:dumped fsmap epoch 17 2023-11-24T07:57:45.833 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-24T07:57:45.958 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:45.956+0000 7f3f85f9a700 1 Processor -- start 2023-11-24T07:57:45.958 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:45.956+0000 7f3f85f9a700 1 -- start start 2023-11-24T07:57:45.961 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:45.957+0000 7f3f85f9a700 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f3f800d49d0 0x7f3f800d2490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:57:45.961 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:45.957+0000 7f3f85f9a700 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f3f800d29d0 0x7f3f800b40a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:57:45.961 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:45.957+0000 7f3f85f9a700 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3f800d4e50 con 0x7f3f8015e460 2023-11-24T07:57:45.961 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:45.957+0000 7f3f85f9a700 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3f800d2e60 con 0x7f3f800d49d0 2023-11-24T07:57:45.962 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:45.957+0000 7f3f85f9a700 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3f800b3370 con 0x7f3f800d29d0 2023-11-24T07:57:45.962 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:45.957+0000 7f3f7f7fe700 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7f3f8015e460 0x7f3f800cf480 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:55690/0 (socket says 172.21.15.59:55690) 2023-11-24T07:57:45.962 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:45.957+0000 7f3f7f7fe700 1 -- 172.21.15.59:0/1121640254 learned_addr learned my addr 172.21.15.59:0/1121640254 (peer_addr_for_me v1:172.21.15.59:0/0) 2023-11-24T07:57:45.962 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:45.957+0000 7f3f7ffff700 1 --2- 172.21.15.59:0/1121640254 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f3f800d29d0 0x7f3f800b40a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:57:45.962 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:45.957+0000 7f3f7effd700 1 --2- 172.21.15.59:0/1121640254 >> v2:172.21.15.76:3300/0 conn(0x7f3f800d49d0 0x7f3f800d2490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:57:45.962 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:45.957+0000 7f3f7e7fc700 1 -- 172.21.15.59:0/1121640254 <== mon.0 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1794723295 0 0) 0x7f3f800d4e50 con 0x7f3f8015e460 2023-11-24T07:57:45.963 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:45.957+0000 7f3f7e7fc700 1 -- 172.21.15.59:0/1121640254 --> v1:172.21.15.59:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3f680035b0 con 0x7f3f8015e460 2023-11-24T07:57:45.963 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:45.958+0000 7f3f7ffff700 1 -- 172.21.15.59:0/1121640254 >> v2:172.21.15.76:3300/0 conn(0x7f3f800d49d0 msgr2=0x7f3f800d2490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T07:57:45.963 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:45.958+0000 7f3f7ffff700 1 --2- 172.21.15.59:0/1121640254 >> v2:172.21.15.76:3300/0 conn(0x7f3f800d49d0 0x7f3f800d2490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:45.963 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:45.958+0000 7f3f7ffff700 1 -- 172.21.15.59:0/1121640254 >> v1:172.21.15.59:6789/0 conn(0x7f3f8015e460 legacy=0x7f3f800cf480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:57:45.963 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:45.958+0000 7f3f7ffff700 1 -- 172.21.15.59:0/1121640254 --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3f800cfc40 con 0x7f3f800d29d0 2023-11-24T07:57:45.963 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:45.958+0000 7f3f7effd700 1 --2- 172.21.15.59:0/1121640254 >> v2:172.21.15.76:3300/0 conn(0x7f3f800d49d0 0x7f3f800d2490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-24T07:57:45.964 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:45.958+0000 7f3f7ffff700 1 --2- 172.21.15.59:0/1121640254 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f3f800d29d0 0x7f3f800b40a0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f3f78009d90 tx=0x7f3f7800b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2feaf71c965fb830 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:57:45.964 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:45.958+0000 7f3f7e7fc700 1 -- 172.21.15.59:0/1121640254 <== mon.2 v2:172.21.15.76:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f7800e070 con 0x7f3f800d29d0 2023-11-24T07:57:45.964 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:45.958+0000 7f3f7e7fc700 1 -- 172.21.15.59:0/1121640254 <== mon.2 v2:172.21.15.76:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3f7800bb20 con 0x7f3f800d29d0 2023-11-24T07:57:45.964 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:45.958+0000 7f3f7e7fc700 1 -- 172.21.15.59:0/1121640254 <== mon.2 v2:172.21.15.76:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f7800be90 con 0x7f3f800d29d0 2023-11-24T07:57:45.964 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:45.959+0000 7f3f85f9a700 1 -- 172.21.15.59:0/1121640254 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f3f800d29d0 msgr2=0x7f3f800b40a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:57:45.964 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:45.959+0000 7f3f85f9a700 1 --2- 172.21.15.59:0/1121640254 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f3f800d29d0 0x7f3f800b40a0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f3f78009d90 tx=0x7f3f7800b330 comp rx=0 tx=0).stop 2023-11-24T07:57:45.964 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:45.959+0000 7f3f85f9a700 1 -- 172.21.15.59:0/1121640254 shutdown_connections 2023-11-24T07:57:45.965 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:45.959+0000 7f3f85f9a700 1 --2- 172.21.15.59:0/1121640254 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f3f800d29d0 0x7f3f800b40a0 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:45.965 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:45.959+0000 7f3f85f9a700 1 --2- 172.21.15.59:0/1121640254 >> v2:172.21.15.76:3300/0 conn(0x7f3f800d49d0 0x7f3f800d2490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:45.965 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:45.959+0000 7f3f85f9a700 1 -- 172.21.15.59:0/1121640254 >> 172.21.15.59:0/1121640254 conn(0x7f3f800d3eb0 msgr2=0x7f3f800d42c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:57:45.965 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:45.959+0000 7f3f85f9a700 1 -- 172.21.15.59:0/1121640254 shutdown_connections 2023-11-24T07:57:45.965 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:45.959+0000 7f3f85f9a700 1 -- 172.21.15.59:0/1121640254 wait complete. 2023-11-24T07:57:45.970 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:45.960+0000 7f3f85f9a700 1 Processor -- start 2023-11-24T07:57:45.970 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:45.960+0000 7f3f85f9a700 1 -- start start 2023-11-24T07:57:45.970 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:45.961+0000 7f3f85f9a700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3f800d29d0 0x7f3f801f31c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:57:45.970 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:45.961+0000 7f3f85f9a700 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f3f800d49d0 0x7f3f801f6e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:57:45.970 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:45.961+0000 7f3f85f9a700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3f8015e460 0x7f3f8015bec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:57:45.970 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:45.961+0000 7f3f85f9a700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f801f7d70 con 0x7f3f800d29d0 2023-11-24T07:57:45.971 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:45.961+0000 7f3f85f9a700 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f801f77b0 con 0x7f3f8015e460 2023-11-24T07:57:45.971 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:45.961+0000 7f3f85f9a700 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3f801f7a90 con 0x7f3f800d49d0 2023-11-24T07:57:45.971 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:45.961+0000 7f3f7f7fe700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3f800d29d0 0x7f3f801f31c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:57:45.971 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:45.961+0000 7f3f7f7fe700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3f800d29d0 0x7f3f801f31c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:52222/0 (socket says 172.21.15.59:52222) 2023-11-24T07:57:45.971 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:45.961+0000 7f3f7f7fe700 1 -- 172.21.15.59:0/2574683349 learned_addr learned my addr 172.21.15.59:0/2574683349 (peer_addr_for_me v2:172.21.15.59:0/0) 2023-11-24T07:57:45.971 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:45.961+0000 7f3f7ffff700 1 --2- 172.21.15.59:0/2574683349 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3f8015e460 0x7f3f8015bec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:57:45.972 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:45.961+0000 7f3f7effd700 1 --2- 172.21.15.59:0/2574683349 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f3f800d49d0 0x7f3f801f6e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:57:45.972 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:45.961+0000 7f3f7f7fe700 1 -- 172.21.15.59:0/2574683349 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f3f800d49d0 msgr2=0x7f3f801f6e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:57:45.972 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:45.961+0000 7f3f7f7fe700 1 --2- 172.21.15.59:0/2574683349 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f3f800d49d0 0x7f3f801f6e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:45.972 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:45.961+0000 7f3f7f7fe700 1 -- 172.21.15.59:0/2574683349 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3f8015e460 msgr2=0x7f3f8015bec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:57:45.972 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:45.961+0000 7f3f7f7fe700 1 --2- 172.21.15.59:0/2574683349 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3f8015e460 0x7f3f8015bec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:45.972 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:45.961+0000 7f3f7f7fe700 1 -- 172.21.15.59:0/2574683349 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3f740094a0 con 0x7f3f800d29d0 2023-11-24T07:57:45.973 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:45.961+0000 7f3f7ffff700 1 --2- 172.21.15.59:0/2574683349 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3f8015e460 0x7f3f8015bec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T07:57:45.973 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:45.961+0000 7f3f7f7fe700 1 --2- 172.21.15.59:0/2574683349 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3f800d29d0 0x7f3f801f31c0 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f3f7000c420 tx=0x7f3f7000c7e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:57:45.973 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:45.961+0000 7f3f67fff700 1 -- 172.21.15.59:0/2574683349 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f70005d70 con 0x7f3f800d29d0 2023-11-24T07:57:45.973 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:45.962+0000 7f3f67fff700 1 -- 172.21.15.59:0/2574683349 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3f700070d0 con 0x7f3f800d29d0 2023-11-24T07:57:45.973 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:45.962+0000 7f3f85f9a700 1 -- 172.21.15.59:0/2574683349 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3f78009a70 con 0x7f3f800d29d0 2023-11-24T07:57:45.973 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:45.962+0000 7f3f67fff700 1 -- 172.21.15.59:0/2574683349 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f70007440 con 0x7f3f800d29d0 2023-11-24T07:57:45.974 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:45.962+0000 7f3f85f9a700 1 -- 172.21.15.59:0/2574683349 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3f800b6e70 con 0x7f3f800d29d0 2023-11-24T07:57:45.974 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:45.963+0000 7f3f85f9a700 1 -- 172.21.15.59:0/2574683349 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3f800b7030 con 0x7f3f800d29d0 2023-11-24T07:57:45.974 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:45.963+0000 7f3f67fff700 1 -- 172.21.15.59:0/2574683349 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 6) v1 ==== 89282+0+0 (secure 0 0 0) 0x7f3f700075a0 con 0x7f3f800d29d0 2023-11-24T07:57:45.974 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:45.963+0000 7f3f67fff700 1 --2- 172.21.15.59:0/2574683349 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f3f6c06af60 0x7f3f6c06d420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:57:45.974 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:45.963+0000 7f3f67fff700 1 -- 172.21.15.59:0/2574683349 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f3f700888c0 con 0x7f3f800d29d0 2023-11-24T07:57:45.975 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:45.964+0000 7f3f7effd700 1 --2- 172.21.15.59:0/2574683349 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f3f6c06af60 0x7f3f6c06d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:57:45.975 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:45.964+0000 7f3f7effd700 1 --2- 172.21.15.59:0/2574683349 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f3f6c06af60 0x7f3f6c06d420 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f3f74009c20 tx=0x7f3f74008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:57:45.975 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:45.968+0000 7f3f67fff700 1 -- 172.21.15.59:0/2574683349 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f3f70053ac0 con 0x7f3f800d29d0 2023-11-24T07:57:46.169 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:46.168+0000 7f3f85f9a700 1 -- 172.21.15.59:0/2574683349 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f3f80067f60 con 0x7f3f800d29d0 2023-11-24T07:57:46.170 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:46.169+0000 7f3f67fff700 1 -- 172.21.15.59:0/2574683349 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v33) v1 ==== 74+0+16437 (secure 0 0 0) 0x7f3f70017090 con 0x7f3f800d29d0 2023-11-24T07:57:46.171 INFO:teuthology.orchestra.run.smithi059.stdout: 2023-11-24T07:57:46.171 INFO:teuthology.orchestra.run.smithi059.stdout:{"epoch":33,"fsid":"9f0e8741-b8f9-48d5-a2c3-09f45903303d","created":"2023-11-24T07:56:06.552729+0000","modified":"2023-11-24T07:57:26.579209+0000","last_up_change":"2023-11-24T07:56:17.372352+0000","last_in_change":"2023-11-24T07:56:09.073250+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-24T07:56:18.352823+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2023-11-24T07:57:23.014143+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"32","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2023-11-24T07:57:23.971487+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"33","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"6d8f9e9b-1782-4e41-928b-a39f2a8ebc06","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6800","nonce":3373878772},{"type":"v1","addr":"172.21.15.59:6801","nonce":3373878772}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6802","nonce":3373878772},{"type":"v1","addr":"172.21.15.59:6803","nonce":3373878772}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6806","nonce":3373878772},{"type":"v1","addr":"172.21.15.59:6807","nonce":3373878772}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6804","nonce":3373878772},{"type":"v1","addr":"172.21.15.59:6805","nonce":3373878772}]},"public_addr":"172.21.15.59:6801/3373878772","cluster_addr":"172.21.15.59:6803/3373878772","heartbeat_back_addr":"172.21.15.59:6807/3373878772","heartbeat_front_addr":"172.21.15.59:6805/3373878772","state":["exists","up"]},{"osd":1,"uuid":"e8d882aa-52e1-4c04-9e1e-c44ef9204a5f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6824","nonce":648258563},{"type":"v1","addr":"172.21.15.59:6825","nonce":648258563}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6826","nonce":648258563},{"type":"v1","addr":"172.21.15.59:6827","nonce":648258563}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6830","nonce":648258563},{"type":"v1","addr":"172.21.15.59:6831","nonce":648258563}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6828","nonce":648258563},{"type":"v1","addr":"172.21.15.59:6829","nonce":648258563}]},"public_addr":"172.21.15.59:6825/648258563","cluster_addr":"172.21.15.59:6827/648258563","heartbeat_back_addr":"172.21.15.59:6831/648258563","heartbeat_front_addr":"172.21.15.59:6829/648258563","state":["exists","up"]},{"osd":2,"uuid":"69eebfa0-695b-4b21-90b9-8a3537168a01","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6808","nonce":1840430654},{"type":"v1","addr":"172.21.15.59:6809","nonce":1840430654}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6810","nonce":1840430654},{"type":"v1","addr":"172.21.15.59:6811","nonce":1840430654}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6814","nonce":1840430654},{"type":"v1","addr":"172.21.15.59:6815","nonce":1840430654}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6812","nonce":1840430654},{"type":"v1","addr":"172.21.15.59:6813","nonce":1840430654}]},"public_addr":"172.21.15.59:6809/1840430654","cluster_addr":"172.21.15.59:6811/1840430654","heartbeat_back_addr":"172.21.15.59:6815/1840430654","heartbeat_front_addr":"172.21.15.59:6813/1840430654","state":["exists","up"]},{"osd":3,"uuid":"96fd77fc-f911-46ff-9445-6f0be2daa8f4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6816","nonce":2504025850},{"type":"v1","addr":"172.21.15.59:6817","nonce":2504025850}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6818","nonce":2504025850},{"type":"v1","addr":"172.21.15.59:6819","nonce":2504025850}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6822","nonce":2504025850},{"type":"v1","addr":"172.21.15.59:6823","nonce":2504025850}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6820","nonce":2504025850},{"type":"v1","addr":"172.21.15.59:6821","nonce":2504025850}]},"public_addr":"172.21.15.59:6817/2504025850","cluster_addr":"172.21.15.59:6819/2504025850","heartbeat_back_addr":"172.21.15.59:6823/2504025850","heartbeat_front_addr":"172.21.15.59:6821/2504025850","state":["exists","up"]},{"osd":4,"uuid":"7ac29fca-b5b2-4c95-9d69-2f164cbbc708","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6816","nonce":2382463431},{"type":"v1","addr":"172.21.15.76:6818","nonce":2382463431}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6820","nonce":2382463431},{"type":"v1","addr":"172.21.15.76:6822","nonce":2382463431}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6828","nonce":2382463431},{"type":"v1","addr":"172.21.15.76:6830","nonce":2382463431}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6824","nonce":2382463431},{"type":"v1","addr":"172.21.15.76:6826","nonce":2382463431}]},"public_addr":"172.21.15.76:6818/2382463431","cluster_addr":"172.21.15.76:6822/2382463431","heartbeat_back_addr":"172.21.15.76:6830/2382463431","heartbeat_front_addr":"172.21.15.76:6826/2382463431","state":["exists","up"]},{"osd":5,"uuid":"485bf711-3851-4238-a4f4-51d52e0a9c58","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6800","nonce":329334308},{"type":"v1","addr":"172.21.15.76:6802","nonce":329334308}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6805","nonce":329334308},{"type":"v1","addr":"172.21.15.76:6807","nonce":329334308}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6814","nonce":329334308},{"type":"v1","addr":"172.21.15.76:6815","nonce":329334308}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6810","nonce":329334308},{"type":"v1","addr":"172.21.15.76:6812","nonce":329334308}]},"public_addr":"172.21.15.76:6802/329334308","cluster_addr":"172.21.15.76:6807/329334308","heartbeat_back_addr":"172.21.15.76:6815/329334308","heartbeat_front_addr":"172.21.15.76:6812/329334308","state":["exists","up"]},{"osd":6,"uuid":"72b3f2fc-1e4a-4e32-968d-003647a0286f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6801","nonce":3946579410},{"type":"v1","addr":"172.21.15.76:6803","nonce":3946579410}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6804","nonce":3946579410},{"type":"v1","addr":"172.21.15.76:6806","nonce":3946579410}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6811","nonce":3946579410},{"type":"v1","addr":"172.21.15.76:6813","nonce":3946579410}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6808","nonce":3946579410},{"type":"v1","addr":"172.21.15.76:6809","nonce":3946579410}]},"public_addr":"172.21.15.76:6803/3946579410","cluster_addr":"172.21.15.76:6806/3946579410","heartbeat_back_addr":"172.21.15.76:6813/3946579410","heartbeat_front_addr":"172.21.15.76:6809/3946579410","state":["exists","up"]},{"osd":7,"uuid":"1ddceb90-6c5d-4d1a-ad56-99cb877f8c70","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6817","nonce":841761538},{"type":"v1","addr":"172.21.15.76:6819","nonce":841761538}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6821","nonce":841761538},{"type":"v1","addr":"172.21.15.76:6823","nonce":841761538}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6829","nonce":841761538},{"type":"v1","addr":"172.21.15.76:6831","nonce":841761538}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6825","nonce":841761538},{"type":"v1","addr":"172.21.15.76:6827","nonce":841761538}]},"public_addr":"172.21.15.76:6819/841761538","cluster_addr":"172.21.15.76:6823/841761538","heartbeat_back_addr":"172.21.15.76:6831/841761538","heartbeat_front_addr":"172.21.15.76:6827/841761538","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T07:56:14.294838+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T07:56:14.270779+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T07:56:14.231809+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T07:56:14.260983+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T07:56:14.302997+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T07:56:14.258600+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T07:56:14.251413+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T07:56:14.304727+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-24T07:57:46.173 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:46.171+0000 7f3f85f9a700 1 -- 172.21.15.59:0/2574683349 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f3f6c06af60 msgr2=0x7f3f6c06d420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:57:46.174 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:46.171+0000 7f3f85f9a700 1 --2- 172.21.15.59:0/2574683349 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f3f6c06af60 0x7f3f6c06d420 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f3f74009c20 tx=0x7f3f74008040 comp rx=0 tx=0).stop 2023-11-24T07:57:46.174 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:46.171+0000 7f3f85f9a700 1 -- 172.21.15.59:0/2574683349 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3f800d29d0 msgr2=0x7f3f801f31c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:57:46.174 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:46.171+0000 7f3f85f9a700 1 --2- 172.21.15.59:0/2574683349 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3f800d29d0 0x7f3f801f31c0 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f3f7000c420 tx=0x7f3f7000c7e0 comp rx=0 tx=0).stop 2023-11-24T07:57:46.175 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:46.171+0000 7f3f85f9a700 1 -- 172.21.15.59:0/2574683349 shutdown_connections 2023-11-24T07:57:46.175 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:46.171+0000 7f3f85f9a700 1 --2- 172.21.15.59:0/2574683349 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f3f6c06af60 0x7f3f6c06d420 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:46.175 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:46.171+0000 7f3f85f9a700 1 --2- 172.21.15.59:0/2574683349 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3f8015e460 0x7f3f8015bec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:46.175 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:46.171+0000 7f3f85f9a700 1 --2- 172.21.15.59:0/2574683349 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3f800d29d0 0x7f3f801f31c0 unknown :-1 s=CLOSED pgs=456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:46.175 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:46.171+0000 7f3f85f9a700 1 --2- 172.21.15.59:0/2574683349 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f3f800d49d0 0x7f3f801f6e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:46.175 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:46.171+0000 7f3f85f9a700 1 -- 172.21.15.59:0/2574683349 >> 172.21.15.59:0/2574683349 conn(0x7f3f800d3eb0 msgr2=0x7f3f800b2480 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:57:46.175 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:46.171+0000 7f3f85f9a700 1 -- 172.21.15.59:0/2574683349 shutdown_connections 2023-11-24T07:57:46.175 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:46.171+0000 7f3f85f9a700 1 -- 172.21.15.59:0/2574683349 wait complete. 2023-11-24T07:57:46.185 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-24T07:56:18.352823+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-24T07:57:23.014143+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '32', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 5, 'pool_name': 'cephfs_data', 'create_time': '2023-11-24T07:57:23.971487+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '33', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-24T07:57:46.186 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-24T07:57:46.317 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:46.312+0000 7f11cc025700 1 Processor -- start 2023-11-24T07:57:46.317 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:46.312+0000 7f11cc025700 1 -- start start 2023-11-24T07:57:46.317 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:46.313+0000 7f11cc025700 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f11c406b620 0x7f11c40b3a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:57:46.317 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:46.313+0000 7f11cc025700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f11c40b5ab0 0x7f11c40b3f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:57:46.317 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:46.313+0000 7f11cc025700 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f11c4071f50 con 0x7f11c41478c0 2023-11-24T07:57:46.317 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:46.313+0000 7f11cc025700 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f11c4072230 con 0x7f11c40b5ab0 2023-11-24T07:57:46.317 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:46.313+0000 7f11cc025700 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f11c4072510 con 0x7f11c406b620 2023-11-24T07:57:46.318 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:46.313+0000 7f11c95c0700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f11c40b5ab0 0x7f11c40b3f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:57:46.318 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:46.313+0000 7f11c95c0700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f11c40b5ab0 0x7f11c40b3f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:52230/0 (socket says 172.21.15.59:52230) 2023-11-24T07:57:46.318 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:46.313+0000 7f11c95c0700 1 -- 172.21.15.59:0/2606045905 learned_addr learned my addr 172.21.15.59:0/2606045905 (peer_addr_for_me v2:172.21.15.59:0/0) 2023-11-24T07:57:46.318 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:46.313+0000 7f11c9dc1700 1 --2- 172.21.15.59:0/2606045905 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f11c406b620 0x7f11c40b3a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:57:46.318 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:46.314+0000 7f11c95c0700 1 -- 172.21.15.59:0/2606045905 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f11c406b620 msgr2=0x7f11c40b3a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T07:57:46.318 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:46.314+0000 7f11c95c0700 1 --2- 172.21.15.59:0/2606045905 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f11c406b620 0x7f11c40b3a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:46.318 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:46.314+0000 7f11c95c0700 1 -- 172.21.15.59:0/2606045905 >> v1:172.21.15.76:6789/0 conn(0x7f11c41478c0 legacy=0x7f11c4161ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:57:46.318 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:46.314+0000 7f11c95c0700 1 -- 172.21.15.59:0/2606045905 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f11c40727f0 con 0x7f11c40b5ab0 2023-11-24T07:57:46.318 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:46.314+0000 7f11c95c0700 1 --2- 172.21.15.59:0/2606045905 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f11c40b5ab0 0x7f11c40b3f80 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f11c00098f0 tx=0x7f11c0009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c9db3f440ac37533 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:57:46.318 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:46.314+0000 7f11c8dbf700 1 -- 172.21.15.59:0/2606045905 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11c000b8a0 con 0x7f11c40b5ab0 2023-11-24T07:57:46.319 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:46.314+0000 7f11c8dbf700 1 -- 172.21.15.59:0/2606045905 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f11c000ba00 con 0x7f11c40b5ab0 2023-11-24T07:57:46.319 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:46.314+0000 7f11c8dbf700 1 -- 172.21.15.59:0/2606045905 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11c000bd90 con 0x7f11c40b5ab0 2023-11-24T07:57:46.319 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:46.314+0000 7f11cc025700 1 -- 172.21.15.59:0/2606045905 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f11c40b5ab0 msgr2=0x7f11c40b3f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:57:46.319 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:46.314+0000 7f11cc025700 1 --2- 172.21.15.59:0/2606045905 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f11c40b5ab0 0x7f11c40b3f80 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f11c00098f0 tx=0x7f11c0009c00 comp rx=0 tx=0).stop 2023-11-24T07:57:46.319 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:46.314+0000 7f11cc025700 1 -- 172.21.15.59:0/2606045905 shutdown_connections 2023-11-24T07:57:46.319 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:46.314+0000 7f11cc025700 1 --2- 172.21.15.59:0/2606045905 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f11c406b620 0x7f11c40b3a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:46.319 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:46.314+0000 7f11cc025700 1 --2- 172.21.15.59:0/2606045905 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f11c40b5ab0 0x7f11c40b3f80 unknown :-1 s=CLOSED pgs=457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:46.319 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:46.314+0000 7f11cc025700 1 -- 172.21.15.59:0/2606045905 >> 172.21.15.59:0/2606045905 conn(0x7f11c40b2740 msgr2=0x7f11c40b2b20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:57:46.319 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:46.315+0000 7f11cc025700 1 -- 172.21.15.59:0/2606045905 shutdown_connections 2023-11-24T07:57:46.319 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:46.315+0000 7f11cc025700 1 -- 172.21.15.59:0/2606045905 wait complete. 2023-11-24T07:57:46.323 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:46.316+0000 7f11cc025700 1 Processor -- start 2023-11-24T07:57:46.324 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:46.316+0000 7f11cc025700 1 -- start start 2023-11-24T07:57:46.324 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:46.317+0000 7f11cc025700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f11c406b620 0x7f11c41def50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:57:46.324 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:46.317+0000 7f11cc025700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f11c40b5ab0 0x7f11c41e85c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:57:46.324 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:46.317+0000 7f11cc025700 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f11c41478c0 0x7f11c41dd220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:57:46.324 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:46.317+0000 7f11cc025700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11c41dcb20 con 0x7f11c40b5ab0 2023-11-24T07:57:46.324 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:46.317+0000 7f11cc025700 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11c41dce00 con 0x7f11c406b620 2023-11-24T07:57:46.324 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:46.317+0000 7f11cc025700 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f11c41dbe30 con 0x7f11c41478c0 2023-11-24T07:57:46.325 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:46.317+0000 7f11c95c0700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f11c40b5ab0 0x7f11c41e85c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:57:46.325 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:46.317+0000 7f11c95c0700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f11c40b5ab0 0x7f11c41e85c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:52240/0 (socket says 172.21.15.59:52240) 2023-11-24T07:57:46.325 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:46.317+0000 7f11c9dc1700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f11c406b620 0x7f11c41def50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:57:46.325 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:46.317+0000 7f11c95c0700 1 -- 172.21.15.59:0/266497242 learned_addr learned my addr 172.21.15.59:0/266497242 (peer_addr_for_me v2:172.21.15.59:0/0) 2023-11-24T07:57:46.325 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:46.317+0000 7f11ca5c2700 1 --2- 172.21.15.59:0/266497242 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f11c41478c0 0x7f11c41dd220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:57:46.325 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:46.317+0000 7f11c95c0700 1 -- 172.21.15.59:0/266497242 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f11c41478c0 msgr2=0x7f11c41dd220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:57:46.325 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:46.317+0000 7f11c95c0700 1 --2- 172.21.15.59:0/266497242 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f11c41478c0 0x7f11c41dd220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:46.326 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:46.317+0000 7f11c95c0700 1 -- 172.21.15.59:0/266497242 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f11c406b620 msgr2=0x7f11c41def50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:57:46.326 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:46.317+0000 7f11c95c0700 1 --2- 172.21.15.59:0/266497242 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f11c406b620 0x7f11c41def50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:46.326 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:46.318+0000 7f11c95c0700 1 -- 172.21.15.59:0/266497242 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f11c0009550 con 0x7f11c40b5ab0 2023-11-24T07:57:46.326 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:46.318+0000 7f11c95c0700 1 --2- 172.21.15.59:0/266497242 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f11c40b5ab0 0x7f11c41e85c0 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f11c0000c00 tx=0x7f11c0004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:57:46.326 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:46.319+0000 7f11ba7fc700 1 -- 172.21.15.59:0/266497242 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11c0004300 con 0x7f11c40b5ab0 2023-11-24T07:57:46.326 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:46.319+0000 7f11ba7fc700 1 -- 172.21.15.59:0/266497242 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f11c001f070 con 0x7f11c40b5ab0 2023-11-24T07:57:46.326 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:46.319+0000 7f11cc025700 1 -- 172.21.15.59:0/266497242 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f11c41e8f10 con 0x7f11c40b5ab0 2023-11-24T07:57:46.326 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:46.319+0000 7f11ba7fc700 1 -- 172.21.15.59:0/266497242 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11c00036a0 con 0x7f11c40b5ab0 2023-11-24T07:57:46.327 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:46.319+0000 7f11cc025700 1 -- 172.21.15.59:0/266497242 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f11c41e94d0 con 0x7f11c40b5ab0 2023-11-24T07:57:46.327 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:46.320+0000 7f11cc025700 1 -- 172.21.15.59:0/266497242 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f11c41dcf40 con 0x7f11c40b5ab0 2023-11-24T07:57:46.327 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:46.321+0000 7f11ba7fc700 1 -- 172.21.15.59:0/266497242 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 6) v1 ==== 89282+0+0 (secure 0 0 0) 0x7f11c0021030 con 0x7f11c40b5ab0 2023-11-24T07:57:46.327 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:46.321+0000 7f11ba7fc700 1 --2- 172.21.15.59:0/266497242 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f11a806af60 0x7f11a806d420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:57:46.327 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:46.321+0000 7f11ba7fc700 1 -- 172.21.15.59:0/266497242 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f11c008a910 con 0x7f11c40b5ab0 2023-11-24T07:57:46.327 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:46.321+0000 7f11c9dc1700 1 --2- 172.21.15.59:0/266497242 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f11a806af60 0x7f11a806d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:57:46.327 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:46.321+0000 7f11c9dc1700 1 --2- 172.21.15.59:0/266497242 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f11a806af60 0x7f11a806d420 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f11b4006b20 tx=0x7f11b4008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:57:46.328 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:46.326+0000 7f11ba7fc700 1 -- 172.21.15.59:0/266497242 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f11c41dcf40 con 0x7f11c40b5ab0 2023-11-24T07:57:46.528 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:46.526+0000 7f11cc025700 1 -- 172.21.15.59:0/266497242 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f11c41dcf40 con 0x7f11c40b5ab0 2023-11-24T07:57:46.528 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:46.526+0000 7f11ba7fc700 1 -- 172.21.15.59:0/266497242 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v33) v1 ==== 93+0+10 (secure 0 0 0) 0x7f11c41dcf40 con 0x7f11c40b5ab0 2023-11-24T07:57:46.529 INFO:teuthology.orchestra.run.smithi059.stdout:pg_num: 1 2023-11-24T07:57:46.531 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:46.528+0000 7f11cc025700 1 -- 172.21.15.59:0/266497242 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f11a806af60 msgr2=0x7f11a806d420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:57:46.531 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:46.528+0000 7f11cc025700 1 --2- 172.21.15.59:0/266497242 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f11a806af60 0x7f11a806d420 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f11b4006b20 tx=0x7f11b4008040 comp rx=0 tx=0).stop 2023-11-24T07:57:46.532 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:46.529+0000 7f11cc025700 1 -- 172.21.15.59:0/266497242 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f11c40b5ab0 msgr2=0x7f11c41e85c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:57:46.532 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:46.529+0000 7f11cc025700 1 --2- 172.21.15.59:0/266497242 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f11c40b5ab0 0x7f11c41e85c0 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f11c0000c00 tx=0x7f11c0004750 comp rx=0 tx=0).stop 2023-11-24T07:57:46.532 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:46.529+0000 7f11cc025700 1 -- 172.21.15.59:0/266497242 shutdown_connections 2023-11-24T07:57:46.532 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:46.529+0000 7f11cc025700 1 --2- 172.21.15.59:0/266497242 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f11a806af60 0x7f11a806d420 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:46.532 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:46.529+0000 7f11cc025700 1 --2- 172.21.15.59:0/266497242 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f11c41478c0 0x7f11c41dd220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:46.532 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:46.529+0000 7f11cc025700 1 --2- 172.21.15.59:0/266497242 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f11c406b620 0x7f11c41def50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:46.532 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:46.529+0000 7f11cc025700 1 --2- 172.21.15.59:0/266497242 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f11c40b5ab0 0x7f11c41e85c0 unknown :-1 s=CLOSED pgs=458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:46.532 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:46.529+0000 7f11cc025700 1 -- 172.21.15.59:0/266497242 >> 172.21.15.59:0/266497242 conn(0x7f11c40b2740 msgr2=0x7f11c406de50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:57:46.532 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:46.529+0000 7f11cc025700 1 -- 172.21.15.59:0/266497242 shutdown_connections 2023-11-24T07:57:46.532 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:46.529+0000 7f11cc025700 1 -- 172.21.15.59:0/266497242 wait complete. 2023-11-24T07:57:46.542 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-24T07:57:46.671 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:46.669+0000 7fe559123700 1 Processor -- start 2023-11-24T07:57:46.671 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:46.669+0000 7fe559123700 1 -- start start 2023-11-24T07:57:46.673 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:46.670+0000 7fe559123700 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fe5540b4150 0x7fe55406de50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:57:46.673 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:46.670+0000 7fe559123700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fe5540b5ab0 0x7fe55406e390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:57:46.673 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:46.670+0000 7fe559123700 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe554071f50 con 0x7fe55406b620 2023-11-24T07:57:46.673 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:46.670+0000 7fe559123700 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe554072230 con 0x7fe5540b5ab0 2023-11-24T07:57:46.673 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:46.670+0000 7fe559123700 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe554072510 con 0x7fe5540b4150 2023-11-24T07:57:46.673 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:46.670+0000 7fe55359e700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fe5540b5ab0 0x7fe55406e390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:57:46.674 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:46.670+0000 7fe55359e700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fe5540b5ab0 0x7fe55406e390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:52248/0 (socket says 172.21.15.59:52248) 2023-11-24T07:57:46.674 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:46.670+0000 7fe55359e700 1 -- 172.21.15.59:0/1817255207 learned_addr learned my addr 172.21.15.59:0/1817255207 (peer_addr_for_me v2:172.21.15.59:0/0) 2023-11-24T07:57:46.674 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:46.670+0000 7fe55259c700 1 --2- 172.21.15.59:0/1817255207 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fe5540b4150 0x7fe55406de50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:57:46.674 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:46.670+0000 7fe55359e700 1 -- 172.21.15.59:0/1817255207 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fe5540b4150 msgr2=0x7fe55406de50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T07:57:46.674 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:46.670+0000 7fe55359e700 1 --2- 172.21.15.59:0/1817255207 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fe5540b4150 0x7fe55406de50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:46.674 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:46.670+0000 7fe55359e700 1 -- 172.21.15.59:0/1817255207 >> v1:172.21.15.76:6789/0 conn(0x7fe55406b620 legacy=0x7fe5540b3a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:57:46.674 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:46.670+0000 7fe55359e700 1 -- 172.21.15.59:0/1817255207 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe5540727f0 con 0x7fe5540b5ab0 2023-11-24T07:57:46.674 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:46.670+0000 7fe55259c700 1 --2- 172.21.15.59:0/1817255207 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fe5540b4150 0x7fe55406de50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-24T07:57:46.674 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:46.670+0000 7fe55359e700 1 --2- 172.21.15.59:0/1817255207 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe5540b5ab0 0x7fe55406e390 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7fe54c009e50 tx=0x7fe54c00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6d5ce3cca56d3709 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:57:46.674 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:46.671+0000 7fe551d9b700 1 -- 172.21.15.59:0/1817255207 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe54c00e070 con 0x7fe5540b5ab0 2023-11-24T07:57:46.674 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:46.671+0000 7fe551d9b700 1 -- 172.21.15.59:0/1817255207 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe54c00b9c0 con 0x7fe5540b5ab0 2023-11-24T07:57:46.674 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:46.671+0000 7fe551d9b700 1 -- 172.21.15.59:0/1817255207 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe54c00bd70 con 0x7fe5540b5ab0 2023-11-24T07:57:46.675 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:46.671+0000 7fe559123700 1 -- 172.21.15.59:0/1817255207 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe5540b5ab0 msgr2=0x7fe55406e390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:57:46.675 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:46.671+0000 7fe559123700 1 --2- 172.21.15.59:0/1817255207 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe5540b5ab0 0x7fe55406e390 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7fe54c009e50 tx=0x7fe54c00b330 comp rx=0 tx=0).stop 2023-11-24T07:57:46.675 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:46.671+0000 7fe559123700 1 -- 172.21.15.59:0/1817255207 shutdown_connections 2023-11-24T07:57:46.675 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:46.671+0000 7fe559123700 1 --2- 172.21.15.59:0/1817255207 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe5540b5ab0 0x7fe55406e390 unknown :-1 s=CLOSED pgs=459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:46.675 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:46.671+0000 7fe559123700 1 --2- 172.21.15.59:0/1817255207 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fe5540b4150 0x7fe55406de50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:46.675 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:46.671+0000 7fe559123700 1 -- 172.21.15.59:0/1817255207 >> 172.21.15.59:0/1817255207 conn(0x7fe5540b2740 msgr2=0x7fe5540b2b20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:57:46.675 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:46.671+0000 7fe559123700 1 -- 172.21.15.59:0/1817255207 shutdown_connections 2023-11-24T07:57:46.675 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:46.671+0000 7fe559123700 1 -- 172.21.15.59:0/1817255207 wait complete. 2023-11-24T07:57:46.678 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:46.673+0000 7fe559123700 1 Processor -- start 2023-11-24T07:57:46.678 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:46.673+0000 7fe559123700 1 -- start start 2023-11-24T07:57:46.678 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:46.673+0000 7fe559123700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe55406b620 0x7fe554146650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:57:46.678 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:46.673+0000 7fe559123700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe5540b4150 0x7fe5541e1d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:57:46.678 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:46.673+0000 7fe559123700 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fe5540b5ab0 0x7fe5541ddb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:57:46.678 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:46.673+0000 7fe559123700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5541e13b0 con 0x7fe55406b620 2023-11-24T07:57:46.678 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:46.673+0000 7fe559123700 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5541e2240 con 0x7fe5540b4150 2023-11-24T07:57:46.679 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:46.673+0000 7fe559123700 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe5541e0550 con 0x7fe5540b5ab0 2023-11-24T07:57:46.679 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:46.673+0000 7fe552d9d700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe55406b620 0x7fe554146650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:57:46.679 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:46.674+0000 7fe552d9d700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe55406b620 0x7fe554146650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:52250/0 (socket says 172.21.15.59:52250) 2023-11-24T07:57:46.679 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:46.674+0000 7fe552d9d700 1 -- 172.21.15.59:0/1898758363 learned_addr learned my addr 172.21.15.59:0/1898758363 (peer_addr_for_me v2:172.21.15.59:0/0) 2023-11-24T07:57:46.679 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:46.674+0000 7fe55259c700 1 --2- 172.21.15.59:0/1898758363 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe5540b4150 0x7fe5541e1d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:57:46.679 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:46.674+0000 7fe55359e700 1 --2- 172.21.15.59:0/1898758363 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fe5540b5ab0 0x7fe5541ddb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:57:46.679 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:46.674+0000 7fe552d9d700 1 -- 172.21.15.59:0/1898758363 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fe5540b5ab0 msgr2=0x7fe5541ddb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:57:46.679 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:46.674+0000 7fe552d9d700 1 --2- 172.21.15.59:0/1898758363 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fe5540b5ab0 0x7fe5541ddb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:46.679 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:46.674+0000 7fe552d9d700 1 -- 172.21.15.59:0/1898758363 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe5540b4150 msgr2=0x7fe5541e1d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:57:46.679 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:46.674+0000 7fe552d9d700 1 --2- 172.21.15.59:0/1898758363 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe5540b4150 0x7fe5541e1d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:46.679 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:46.674+0000 7fe552d9d700 1 -- 172.21.15.59:0/1898758363 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe54c009b30 con 0x7fe55406b620 2023-11-24T07:57:46.680 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:46.674+0000 7fe552d9d700 1 --2- 172.21.15.59:0/1898758363 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe55406b620 0x7fe554146650 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7fe54800c450 tx=0x7fe54800c810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:57:46.680 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:46.674+0000 7fe53b7fe700 1 -- 172.21.15.59:0/1898758363 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5480052d0 con 0x7fe55406b620 2023-11-24T07:57:46.680 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:46.674+0000 7fe53b7fe700 1 -- 172.21.15.59:0/1898758363 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe548005430 con 0x7fe55406b620 2023-11-24T07:57:46.680 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:46.674+0000 7fe559123700 1 -- 172.21.15.59:0/1898758363 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe5541e0830 con 0x7fe55406b620 2023-11-24T07:57:46.680 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:46.674+0000 7fe53b7fe700 1 -- 172.21.15.59:0/1898758363 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5480038c0 con 0x7fe55406b620 2023-11-24T07:57:46.680 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:46.674+0000 7fe559123700 1 -- 172.21.15.59:0/1898758363 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe5541e0df0 con 0x7fe55406b620 2023-11-24T07:57:46.680 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:46.675+0000 7fe559123700 1 -- 172.21.15.59:0/1898758363 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe554075480 con 0x7fe55406b620 2023-11-24T07:57:46.680 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:46.676+0000 7fe53b7fe700 1 -- 172.21.15.59:0/1898758363 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 6) v1 ==== 89282+0+0 (secure 0 0 0) 0x7fe548003a20 con 0x7fe55406b620 2023-11-24T07:57:46.680 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:46.676+0000 7fe53b7fe700 1 --2- 172.21.15.59:0/1898758363 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7fe54006af60 0x7fe54006d420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:57:46.680 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:46.676+0000 7fe53b7fe700 1 -- 172.21.15.59:0/1898758363 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fe548088980 con 0x7fe55406b620 2023-11-24T07:57:46.681 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:46.676+0000 7fe55259c700 1 --2- 172.21.15.59:0/1898758363 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7fe54006af60 0x7fe54006d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:57:46.681 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:46.677+0000 7fe55259c700 1 --2- 172.21.15.59:0/1898758363 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7fe54006af60 0x7fe54006d420 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7fe554146b90 tx=0x7fe544008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:57:46.682 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:46.681+0000 7fe53b7fe700 1 -- 172.21.15.59:0/1898758363 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fe548053ad0 con 0x7fe55406b620 2023-11-24T07:57:46.881 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:46.879+0000 7fe559123700 1 -- 172.21.15.59:0/1898758363 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fe5541e0330 con 0x7fe55406b620 2023-11-24T07:57:46.881 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:46.879+0000 7fe53b7fe700 1 -- 172.21.15.59:0/1898758363 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v33) v1 ==== 104+0+11 (secure 0 0 0) 0x7fe5480570f0 con 0x7fe55406b620 2023-11-24T07:57:46.881 INFO:teuthology.orchestra.run.smithi059.stdout:pg_num: 32 2023-11-24T07:57:46.883 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:46.882+0000 7fe559123700 1 -- 172.21.15.59:0/1898758363 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7fe54006af60 msgr2=0x7fe54006d420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:57:46.884 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:46.882+0000 7fe559123700 1 --2- 172.21.15.59:0/1898758363 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7fe54006af60 0x7fe54006d420 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7fe554146b90 tx=0x7fe544008040 comp rx=0 tx=0).stop 2023-11-24T07:57:46.884 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:46.882+0000 7fe559123700 1 -- 172.21.15.59:0/1898758363 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe55406b620 msgr2=0x7fe554146650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:57:46.884 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:46.882+0000 7fe559123700 1 --2- 172.21.15.59:0/1898758363 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe55406b620 0x7fe554146650 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7fe54800c450 tx=0x7fe54800c810 comp rx=0 tx=0).stop 2023-11-24T07:57:46.884 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:46.882+0000 7fe559123700 1 -- 172.21.15.59:0/1898758363 shutdown_connections 2023-11-24T07:57:46.884 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:46.882+0000 7fe559123700 1 --2- 172.21.15.59:0/1898758363 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7fe54006af60 0x7fe54006d420 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:46.884 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:46.882+0000 7fe559123700 1 --2- 172.21.15.59:0/1898758363 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fe5540b5ab0 0x7fe5541ddb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:46.884 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:46.882+0000 7fe559123700 1 --2- 172.21.15.59:0/1898758363 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe55406b620 0x7fe554146650 unknown :-1 s=CLOSED pgs=460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:46.884 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:46.882+0000 7fe559123700 1 --2- 172.21.15.59:0/1898758363 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe5540b4150 0x7fe5541e1d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:46.884 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:46.882+0000 7fe559123700 1 -- 172.21.15.59:0/1898758363 >> 172.21.15.59:0/1898758363 conn(0x7fe5540b2740 msgr2=0x7fe5540b0110 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:57:46.884 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:46.882+0000 7fe559123700 1 -- 172.21.15.59:0/1898758363 shutdown_connections 2023-11-24T07:57:46.885 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:46.882+0000 7fe559123700 1 -- 172.21.15.59:0/1898758363 wait complete. 2023-11-24T07:57:46.896 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-24T07:57:47.023 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:47.022+0000 7fe4a8a2f700 1 Processor -- start 2023-11-24T07:57:47.024 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:47.022+0000 7fe4a8a2f700 1 -- start start 2023-11-24T07:57:47.027 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:47.023+0000 7fe4a8a2f700 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fe4a4144610 0x7fe4a40d09f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:57:47.027 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:47.023+0000 7fe4a8a2f700 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fe4a406f700 0x7fe4a40d0f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:57:47.027 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:47.023+0000 7fe4a8a2f700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fe4a415e7c0 0x7fe4a415ec50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:57:47.027 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:47.023+0000 7fe4a8a2f700 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe4a406e9d0 con 0x7fe4a415e7c0 2023-11-24T07:57:47.027 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:47.023+0000 7fe4a8a2f700 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe4a406f270 con 0x7fe4a406f700 2023-11-24T07:57:47.027 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:47.023+0000 7fe4a8a2f700 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe4a41449f0 con 0x7fe4a4144610 2023-11-24T07:57:47.027 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:47.023+0000 7fe4a2d9d700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fe4a415e7c0 0x7fe4a415ec50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:57:47.027 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:47.023+0000 7fe4a2d9d700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fe4a415e7c0 0x7fe4a415ec50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:52262/0 (socket says 172.21.15.59:52262) 2023-11-24T07:57:47.028 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:47.023+0000 7fe4a2d9d700 1 -- 172.21.15.59:0/3594604980 learned_addr learned my addr 172.21.15.59:0/3594604980 (peer_addr_for_me v2:172.21.15.59:0/0) 2023-11-24T07:57:47.028 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:47.023+0000 7fe4a259c700 1 --2- 172.21.15.59:0/3594604980 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fe4a4144610 0x7fe4a40d09f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:57:47.028 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:47.023+0000 7fe4a1d9b700 1 --2- 172.21.15.59:0/3594604980 >> v2:172.21.15.76:3300/0 conn(0x7fe4a406f700 0x7fe4a40d0f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:57:47.028 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:47.023+0000 7fe4a2d9d700 1 -- 172.21.15.59:0/3594604980 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fe4a4144610 msgr2=0x7fe4a40d09f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T07:57:47.028 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:47.023+0000 7fe4a2d9d700 1 --2- 172.21.15.59:0/3594604980 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fe4a4144610 0x7fe4a40d09f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:47.028 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:47.023+0000 7fe4a2d9d700 1 -- 172.21.15.59:0/3594604980 >> v2:172.21.15.76:3300/0 conn(0x7fe4a406f700 msgr2=0x7fe4a40d0f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T07:57:47.028 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:47.023+0000 7fe4a2d9d700 1 --2- 172.21.15.59:0/3594604980 >> v2:172.21.15.76:3300/0 conn(0x7fe4a406f700 0x7fe4a40d0f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:47.028 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:47.023+0000 7fe4a2d9d700 1 -- 172.21.15.59:0/3594604980 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe4a415f530 con 0x7fe4a415e7c0 2023-11-24T07:57:47.028 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:47.023+0000 7fe4a2d9d700 1 --2- 172.21.15.59:0/3594604980 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe4a415e7c0 0x7fe4a415ec50 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7fe49c009e50 tx=0x7fe49c00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c6df8b56e0eee6fc server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:57:47.028 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:47.023+0000 7fe4a159a700 1 -- 172.21.15.59:0/3594604980 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe49c00e070 con 0x7fe4a415e7c0 2023-11-24T07:57:47.028 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:47.024+0000 7fe4a159a700 1 -- 172.21.15.59:0/3594604980 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe49c00b8d0 con 0x7fe4a415e7c0 2023-11-24T07:57:47.028 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:47.024+0000 7fe4a159a700 1 -- 172.21.15.59:0/3594604980 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe49c00bc80 con 0x7fe4a415e7c0 2023-11-24T07:57:47.028 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:47.024+0000 7fe4a8a2f700 1 -- 172.21.15.59:0/3594604980 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe4a415e7c0 msgr2=0x7fe4a415ec50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:57:47.029 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:47.024+0000 7fe4a8a2f700 1 --2- 172.21.15.59:0/3594604980 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe4a415e7c0 0x7fe4a415ec50 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7fe49c009e50 tx=0x7fe49c00b330 comp rx=0 tx=0).stop 2023-11-24T07:57:47.029 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:47.024+0000 7fe4a8a2f700 1 -- 172.21.15.59:0/3594604980 shutdown_connections 2023-11-24T07:57:47.029 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:47.024+0000 7fe4a8a2f700 1 --2- 172.21.15.59:0/3594604980 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe4a415e7c0 0x7fe4a415ec50 unknown :-1 s=CLOSED pgs=461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:47.029 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:47.024+0000 7fe4a8a2f700 1 --2- 172.21.15.59:0/3594604980 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fe4a4144610 0x7fe4a40d09f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:47.029 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:47.024+0000 7fe4a8a2f700 1 --2- 172.21.15.59:0/3594604980 >> v2:172.21.15.76:3300/0 conn(0x7fe4a406f700 0x7fe4a40d0f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:47.029 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:47.024+0000 7fe4a8a2f700 1 -- 172.21.15.59:0/3594604980 >> 172.21.15.59:0/3594604980 conn(0x7fe4a40b2740 msgr2=0x7fe4a40b2b50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:57:47.029 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:47.024+0000 7fe4a8a2f700 1 -- 172.21.15.59:0/3594604980 shutdown_connections 2023-11-24T07:57:47.029 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:47.024+0000 7fe4a8a2f700 1 -- 172.21.15.59:0/3594604980 wait complete. 2023-11-24T07:57:47.032 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:47.026+0000 7fe4a8a2f700 1 Processor -- start 2023-11-24T07:57:47.032 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:47.026+0000 7fe4a8a2f700 1 -- start start 2023-11-24T07:57:47.032 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:47.027+0000 7fe4a8a2f700 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fe4a406f700 0x7fe4a41f5690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:57:47.032 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:47.027+0000 7fe4a8a2f700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe4a4144610 0x7fe4a415a500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:57:47.032 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:47.027+0000 7fe4a8a2f700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe4a415e7c0 0x7fe4a40d0a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:57:47.032 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:47.027+0000 7fe4a8a2f700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4a40d05c0 con 0x7fe4a415e7c0 2023-11-24T07:57:47.032 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:47.027+0000 7fe4a8a2f700 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4a40cfc90 con 0x7fe4a4144610 2023-11-24T07:57:47.032 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:47.027+0000 7fe4a8a2f700 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe4a40cff70 con 0x7fe4a406f700 2023-11-24T07:57:47.032 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:47.027+0000 7fe4a2d9d700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe4a415e7c0 0x7fe4a40d0a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:57:47.032 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:47.027+0000 7fe4a2d9d700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe4a415e7c0 0x7fe4a40d0a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:52268/0 (socket says 172.21.15.59:52268) 2023-11-24T07:57:47.033 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:47.027+0000 7fe4a2d9d700 1 -- 172.21.15.59:0/20416351 learned_addr learned my addr 172.21.15.59:0/20416351 (peer_addr_for_me v2:172.21.15.59:0/0) 2023-11-24T07:57:47.033 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:47.027+0000 7fe4a259c700 1 --2- 172.21.15.59:0/20416351 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fe4a406f700 0x7fe4a41f5690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:57:47.033 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:47.027+0000 7fe4a1d9b700 1 --2- 172.21.15.59:0/20416351 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe4a4144610 0x7fe4a415a500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:57:47.033 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:47.027+0000 7fe4a2d9d700 1 -- 172.21.15.59:0/20416351 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fe4a406f700 msgr2=0x7fe4a41f5690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:57:47.033 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:47.027+0000 7fe4a2d9d700 1 --2- 172.21.15.59:0/20416351 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fe4a406f700 0x7fe4a41f5690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:47.033 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:47.027+0000 7fe4a2d9d700 1 -- 172.21.15.59:0/20416351 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe4a4144610 msgr2=0x7fe4a415a500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:57:47.033 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:47.027+0000 7fe4a2d9d700 1 --2- 172.21.15.59:0/20416351 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe4a4144610 0x7fe4a415a500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:47.033 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:47.027+0000 7fe4a2d9d700 1 -- 172.21.15.59:0/20416351 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe49c009b30 con 0x7fe4a415e7c0 2023-11-24T07:57:47.033 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:47.027+0000 7fe4a2d9d700 1 --2- 172.21.15.59:0/20416351 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe4a415e7c0 0x7fe4a40d0a50 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7fe4a406d1a0 tx=0x7fe49c004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:57:47.033 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:47.028+0000 7fe492ffd700 1 -- 172.21.15.59:0/20416351 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe49c004880 con 0x7fe4a415e7c0 2023-11-24T07:57:47.033 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:47.028+0000 7fe492ffd700 1 -- 172.21.15.59:0/20416351 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe49c0092e0 con 0x7fe4a415e7c0 2023-11-24T07:57:47.033 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:47.028+0000 7fe4a8a2f700 1 -- 172.21.15.59:0/20416351 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe4a40d0250 con 0x7fe4a415e7c0 2023-11-24T07:57:47.034 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:47.028+0000 7fe492ffd700 1 -- 172.21.15.59:0/20416351 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe49c01c680 con 0x7fe4a415e7c0 2023-11-24T07:57:47.034 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:47.028+0000 7fe4a8a2f700 1 -- 172.21.15.59:0/20416351 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe4a413f9c0 con 0x7fe4a415e7c0 2023-11-24T07:57:47.034 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:47.029+0000 7fe4a8a2f700 1 -- 172.21.15.59:0/20416351 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe4a40d0770 con 0x7fe4a415e7c0 2023-11-24T07:57:47.034 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:47.029+0000 7fe492ffd700 1 -- 172.21.15.59:0/20416351 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 6) v1 ==== 89282+0+0 (secure 0 0 0) 0x7fe49c01c7e0 con 0x7fe4a415e7c0 2023-11-24T07:57:47.034 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:47.029+0000 7fe492ffd700 1 --2- 172.21.15.59:0/20416351 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7fe48406af60 0x7fe48406d420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:57:47.034 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:47.030+0000 7fe492ffd700 1 -- 172.21.15.59:0/20416351 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fe49c08ae70 con 0x7fe4a415e7c0 2023-11-24T07:57:47.034 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:47.030+0000 7fe4a259c700 1 --2- 172.21.15.59:0/20416351 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7fe48406af60 0x7fe48406d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:57:47.034 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:47.030+0000 7fe4a259c700 1 --2- 172.21.15.59:0/20416351 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7fe48406af60 0x7fe48406d420 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7fe494009be0 tx=0x7fe494009210 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:57:47.035 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:47.034+0000 7fe492ffd700 1 -- 172.21.15.59:0/20416351 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fe4a40d0770 con 0x7fe4a415e7c0 2023-11-24T07:57:47.236 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:47.234+0000 7fe4a8a2f700 1 -- 172.21.15.59:0/20416351 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fe4a4075480 con 0x7fe4a415e7c0 2023-11-24T07:57:47.236 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:47.235+0000 7fe492ffd700 1 -- 172.21.15.59:0/20416351 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v33) v1 ==== 100+0+11 (secure 0 0 0) 0x7fe49c02a020 con 0x7fe4a415e7c0 2023-11-24T07:57:47.237 INFO:teuthology.orchestra.run.smithi059.stdout:pg_num: 64 2023-11-24T07:57:47.239 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:47.237+0000 7fe4a8a2f700 1 -- 172.21.15.59:0/20416351 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7fe48406af60 msgr2=0x7fe48406d420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:57:47.239 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:47.237+0000 7fe4a8a2f700 1 --2- 172.21.15.59:0/20416351 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7fe48406af60 0x7fe48406d420 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7fe494009be0 tx=0x7fe494009210 comp rx=0 tx=0).stop 2023-11-24T07:57:47.239 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:47.237+0000 7fe4a8a2f700 1 -- 172.21.15.59:0/20416351 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe4a415e7c0 msgr2=0x7fe4a40d0a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:57:47.239 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:47.237+0000 7fe4a8a2f700 1 --2- 172.21.15.59:0/20416351 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe4a415e7c0 0x7fe4a40d0a50 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7fe4a406d1a0 tx=0x7fe49c004ad0 comp rx=0 tx=0).stop 2023-11-24T07:57:47.239 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:47.237+0000 7fe4a8a2f700 1 -- 172.21.15.59:0/20416351 shutdown_connections 2023-11-24T07:57:47.240 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:47.237+0000 7fe4a8a2f700 1 --2- 172.21.15.59:0/20416351 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7fe48406af60 0x7fe48406d420 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:47.240 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:47.237+0000 7fe4a8a2f700 1 --2- 172.21.15.59:0/20416351 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe4a415e7c0 0x7fe4a40d0a50 unknown :-1 s=CLOSED pgs=462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:47.240 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:47.237+0000 7fe4a8a2f700 1 --2- 172.21.15.59:0/20416351 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fe4a406f700 0x7fe4a41f5690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:47.240 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:47.237+0000 7fe4a8a2f700 1 --2- 172.21.15.59:0/20416351 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe4a4144610 0x7fe4a415a500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:47.240 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:47.237+0000 7fe4a8a2f700 1 -- 172.21.15.59:0/20416351 >> 172.21.15.59:0/20416351 conn(0x7fe4a40b2740 msgr2=0x7fe4a415d0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:57:47.241 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:47.237+0000 7fe4a8a2f700 1 -- 172.21.15.59:0/20416351 shutdown_connections 2023-11-24T07:57:47.241 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:47.237+0000 7fe4a8a2f700 1 -- 172.21.15.59:0/20416351 wait complete. 2023-11-24T07:57:47.251 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-24T07:57:47.376 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:47.374+0000 7f4eab9b0700 1 Processor -- start 2023-11-24T07:57:47.377 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:47.375+0000 7f4eab9b0700 1 -- start start 2023-11-24T07:57:47.379 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:47.375+0000 7f4eab9b0700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f4ea415df70 0x7f4ea406eef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:57:47.379 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:47.375+0000 7f4eab9b0700 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f4ea415e350 0x7f4ea406f6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:57:47.379 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:47.375+0000 7f4eab9b0700 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4ea4070080 con 0x7f4ea406fc30 2023-11-24T07:57:47.380 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:47.375+0000 7f4eab9b0700 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4ea40d10f0 con 0x7f4ea415df70 2023-11-24T07:57:47.380 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:47.375+0000 7f4eab9b0700 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4ea40d0850 con 0x7f4ea415e350 2023-11-24T07:57:47.380 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:47.376+0000 7f4ea974c700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f4ea415df70 0x7f4ea406eef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:57:47.380 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:47.376+0000 7f4ea9f4d700 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7f4ea406fc30 0x7f4ea40ce430 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:55750/0 (socket says 172.21.15.59:55750) 2023-11-24T07:57:47.380 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:47.376+0000 7f4ea9f4d700 1 -- 172.21.15.59:0/3230214098 learned_addr learned my addr 172.21.15.59:0/3230214098 (peer_addr_for_me v1:172.21.15.59:0/0) 2023-11-24T07:57:47.380 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:47.376+0000 7f4ea974c700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f4ea415df70 0x7f4ea406eef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:52272/0 (socket says 172.21.15.59:52272) 2023-11-24T07:57:47.380 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:47.376+0000 7f4ea8f4b700 1 --2- 172.21.15.59:0/3230214098 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f4ea415e350 0x7f4ea406f6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:57:47.380 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:47.376+0000 7f4ea974c700 1 -- 172.21.15.59:0/3230214098 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f4ea415e350 msgr2=0x7f4ea406f6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T07:57:47.380 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:47.376+0000 7f4ea974c700 1 --2- 172.21.15.59:0/3230214098 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f4ea415e350 0x7f4ea406f6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:47.380 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:47.376+0000 7f4ea974c700 1 -- 172.21.15.59:0/3230214098 >> v1:172.21.15.59:6789/0 conn(0x7f4ea406fc30 legacy=0x7f4ea40ce430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:57:47.380 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:47.376+0000 7f4e9bfff700 1 -- 172.21.15.59:0/3230214098 <== mon.0 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 287266696 0 0) 0x7f4e9c001490 con 0x7f4ea406fc30 2023-11-24T07:57:47.381 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:47.376+0000 7f4ea974c700 1 -- 172.21.15.59:0/3230214098 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4ea40d0b30 con 0x7f4ea415df70 2023-11-24T07:57:47.381 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:47.376+0000 7f4ea974c700 1 --2- 172.21.15.59:0/3230214098 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4ea415df70 0x7f4ea406eef0 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f4ea0009840 tx=0x7f4ea0009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a8179ccf2c13fa72 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:57:47.381 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:47.376+0000 7f4e9bfff700 1 -- 172.21.15.59:0/3230214098 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ea0011830 con 0x7f4ea415df70 2023-11-24T07:57:47.381 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:47.376+0000 7f4e9bfff700 1 -- 172.21.15.59:0/3230214098 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4ea0011990 con 0x7f4ea415df70 2023-11-24T07:57:47.381 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:47.376+0000 7f4e9bfff700 1 -- 172.21.15.59:0/3230214098 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ea0011d20 con 0x7f4ea415df70 2023-11-24T07:57:47.381 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:47.377+0000 7f4eab9b0700 1 -- 172.21.15.59:0/3230214098 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4ea415df70 msgr2=0x7f4ea406eef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:57:47.381 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:47.377+0000 7f4eab9b0700 1 --2- 172.21.15.59:0/3230214098 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4ea415df70 0x7f4ea406eef0 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f4ea0009840 tx=0x7f4ea0009b50 comp rx=0 tx=0).stop 2023-11-24T07:57:47.381 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:47.377+0000 7f4eab9b0700 1 -- 172.21.15.59:0/3230214098 shutdown_connections 2023-11-24T07:57:47.381 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:47.377+0000 7f4eab9b0700 1 --2- 172.21.15.59:0/3230214098 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4ea415df70 0x7f4ea406eef0 unknown :-1 s=CLOSED pgs=464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:47.381 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:47.377+0000 7f4eab9b0700 1 --2- 172.21.15.59:0/3230214098 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f4ea415e350 0x7f4ea406f6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:47.382 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:47.377+0000 7f4eab9b0700 1 -- 172.21.15.59:0/3230214098 >> 172.21.15.59:0/3230214098 conn(0x7f4ea40b3720 msgr2=0x7f4ea40b3b30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:57:47.382 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:47.377+0000 7f4eab9b0700 1 -- 172.21.15.59:0/3230214098 shutdown_connections 2023-11-24T07:57:47.382 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:47.377+0000 7f4eab9b0700 1 -- 172.21.15.59:0/3230214098 wait complete. 2023-11-24T07:57:47.384 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:47.378+0000 7f4eab9b0700 1 Processor -- start 2023-11-24T07:57:47.384 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:47.378+0000 7f4eab9b0700 1 -- start start 2023-11-24T07:57:47.385 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:47.379+0000 7f4eab9b0700 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f4ea406fc30 0x7f4ea40b3b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:57:47.385 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:47.379+0000 7f4eab9b0700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4ea415df70 0x7f4ea414cf70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:57:47.385 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:47.379+0000 7f4eab9b0700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4ea415e350 0x7f4ea40b5320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:57:47.385 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:47.379+0000 7f4eab9b0700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ea40b4350 con 0x7f4ea415e350 2023-11-24T07:57:47.385 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:47.379+0000 7f4eab9b0700 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ea40b4630 con 0x7f4ea415df70 2023-11-24T07:57:47.385 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:47.379+0000 7f4eab9b0700 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4ea40b4910 con 0x7f4ea406fc30 2023-11-24T07:57:47.385 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:47.379+0000 7f4ea9f4d700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4ea415e350 0x7f4ea40b5320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:57:47.385 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:47.379+0000 7f4ea9f4d700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4ea415e350 0x7f4ea40b5320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:52282/0 (socket says 172.21.15.59:52282) 2023-11-24T07:57:47.385 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:47.379+0000 7f4ea9f4d700 1 -- 172.21.15.59:0/2648676406 learned_addr learned my addr 172.21.15.59:0/2648676406 (peer_addr_for_me v2:172.21.15.59:0/0) 2023-11-24T07:57:47.385 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:47.379+0000 7f4ea974c700 1 --2- 172.21.15.59:0/2648676406 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f4ea406fc30 0x7f4ea40b3b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:57:47.385 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:47.379+0000 7f4ea8f4b700 1 --2- 172.21.15.59:0/2648676406 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4ea415df70 0x7f4ea414cf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:57:47.385 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:47.379+0000 7f4ea9f4d700 1 -- 172.21.15.59:0/2648676406 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f4ea406fc30 msgr2=0x7f4ea40b3b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:57:47.386 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:47.379+0000 7f4ea9f4d700 1 --2- 172.21.15.59:0/2648676406 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f4ea406fc30 0x7f4ea40b3b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:47.386 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:47.379+0000 7f4ea9f4d700 1 -- 172.21.15.59:0/2648676406 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4ea415df70 msgr2=0x7f4ea414cf70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:57:47.386 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:47.379+0000 7f4ea9f4d700 1 --2- 172.21.15.59:0/2648676406 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4ea415df70 0x7f4ea414cf70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:47.386 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:47.379+0000 7f4ea9f4d700 1 -- 172.21.15.59:0/2648676406 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4ea00094a0 con 0x7f4ea415e350 2023-11-24T07:57:47.386 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:47.379+0000 7f4ea8f4b700 1 --2- 172.21.15.59:0/2648676406 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4ea415df70 0x7f4ea414cf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-24T07:57:47.386 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:47.380+0000 7f4ea9f4d700 1 --2- 172.21.15.59:0/2648676406 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4ea415e350 0x7f4ea40b5320 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f4e9c00cca0 tx=0x7f4e9c00a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:57:47.386 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:47.380+0000 7f4e99ffb700 1 -- 172.21.15.59:0/2648676406 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e9c00ac20 con 0x7f4ea415e350 2023-11-24T07:57:47.386 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:47.380+0000 7f4e99ffb700 1 -- 172.21.15.59:0/2648676406 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4e9c00ad80 con 0x7f4ea415e350 2023-11-24T07:57:47.386 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:47.380+0000 7f4eab9b0700 1 -- 172.21.15.59:0/2648676406 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4ea40b4bf0 con 0x7f4ea415e350 2023-11-24T07:57:47.386 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:47.380+0000 7f4e99ffb700 1 -- 172.21.15.59:0/2648676406 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e9c007310 con 0x7f4ea415e350 2023-11-24T07:57:47.386 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:47.380+0000 7f4eab9b0700 1 -- 172.21.15.59:0/2648676406 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4ea40b4040 con 0x7f4ea415e350 2023-11-24T07:57:47.386 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:47.381+0000 7f4eab9b0700 1 -- 172.21.15.59:0/2648676406 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4ea4067f60 con 0x7f4ea415e350 2023-11-24T07:57:47.387 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:47.381+0000 7f4e99ffb700 1 -- 172.21.15.59:0/2648676406 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 6) v1 ==== 89282+0+0 (secure 0 0 0) 0x7f4e9c007470 con 0x7f4ea415e350 2023-11-24T07:57:47.387 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:47.382+0000 7f4e99ffb700 1 --2- 172.21.15.59:0/2648676406 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f4e8806af60 0x7f4e8806d420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:57:47.387 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:47.382+0000 7f4e99ffb700 1 -- 172.21.15.59:0/2648676406 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f4e9c088fd0 con 0x7f4ea415e350 2023-11-24T07:57:47.387 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:47.382+0000 7f4ea974c700 1 --2- 172.21.15.59:0/2648676406 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f4e8806af60 0x7f4e8806d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:57:47.387 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:47.382+0000 7f4ea974c700 1 --2- 172.21.15.59:0/2648676406 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f4e8806af60 0x7f4e8806d420 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f4ea00090b0 tx=0x7f4ea0008030 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:57:47.387 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:47.386+0000 7f4e99ffb700 1 -- 172.21.15.59:0/2648676406 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f4e9c054120 con 0x7f4ea415e350 2023-11-24T07:57:47.638 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:47.636+0000 7f4eab9b0700 1 -- 172.21.15.59:0/2648676406 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f4ea4070080 con 0x7f4ea415e350 2023-11-24T07:57:47.638 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:47.636+0000 7f4e99ffb700 1 -- 172.21.15.59:0/2648676406 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) v1 ==== 94+0+4913 (secure 0 0 0) 0x7f4ea4070080 con 0x7f4ea415e350 2023-11-24T07:57:47.639 INFO:teuthology.orchestra.run.smithi059.stdout: 2023-11-24T07:57:47.639 INFO:teuthology.orchestra.run.smithi059.stdout:{"epoch":17,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4763,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.76:6836/1669441817","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6834","nonce":1669441817},{"type":"v1","addr":"172.21.15.76:6836","nonce":1669441817}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":4769,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.76:6837/3709928230","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6835","nonce":3709928230},{"type":"v1","addr":"172.21.15.76:6837","nonce":3709928230}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":5106,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6834/2019319496","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6832","nonce":2019319496},{"type":"v1","addr":"172.21.15.59:6834","nonce":2019319496}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":16,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-24T07:57:25.003636+0000","modified":"2023-11-24T07:57:28.021959+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5109},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5109":{"gid":5109,"name":"a","rank":0,"incarnation":13,"state":"up:active","state_seq":3,"addr":"172.21.15.59:6835/2290655475","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6833","nonce":2290655475},{"type":"v1","addr":"172.21.15.59:6835","nonce":2290655475}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2023-11-24T07:57:47.641 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:47.639+0000 7f4eab9b0700 1 -- 172.21.15.59:0/2648676406 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f4e8806af60 msgr2=0x7f4e8806d420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:57:47.641 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:47.639+0000 7f4eab9b0700 1 --2- 172.21.15.59:0/2648676406 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f4e8806af60 0x7f4e8806d420 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f4ea00090b0 tx=0x7f4ea0008030 comp rx=0 tx=0).stop 2023-11-24T07:57:47.641 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:47.639+0000 7f4eab9b0700 1 -- 172.21.15.59:0/2648676406 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4ea415e350 msgr2=0x7f4ea40b5320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:57:47.641 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:47.639+0000 7f4eab9b0700 1 --2- 172.21.15.59:0/2648676406 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4ea415e350 0x7f4ea40b5320 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f4e9c00cca0 tx=0x7f4e9c00a060 comp rx=0 tx=0).stop 2023-11-24T07:57:47.641 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:47.639+0000 7f4eab9b0700 1 -- 172.21.15.59:0/2648676406 shutdown_connections 2023-11-24T07:57:47.641 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:47.639+0000 7f4eab9b0700 1 --2- 172.21.15.59:0/2648676406 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f4e8806af60 0x7f4e8806d420 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:47.641 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:47.639+0000 7f4eab9b0700 1 --2- 172.21.15.59:0/2648676406 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4ea415e350 0x7f4ea40b5320 unknown :-1 s=CLOSED pgs=465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:47.641 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:47.639+0000 7f4eab9b0700 1 --2- 172.21.15.59:0/2648676406 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f4ea406fc30 0x7f4ea40b3b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:47.641 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:47.639+0000 7f4eab9b0700 1 --2- 172.21.15.59:0/2648676406 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4ea415df70 0x7f4ea414cf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:47.641 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:47.639+0000 7f4eab9b0700 1 -- 172.21.15.59:0/2648676406 >> 172.21.15.59:0/2648676406 conn(0x7f4ea40b3720 msgr2=0x7f4ea40b1690 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:57:47.641 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:47.639+0000 7f4eab9b0700 1 -- 172.21.15.59:0/2648676406 shutdown_connections 2023-11-24T07:57:47.641 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:47.639+0000 7f4eab9b0700 1 -- 172.21.15.59:0/2648676406 wait complete. 2023-11-24T07:57:47.642 INFO:teuthology.orchestra.run.smithi059.stderr:dumped fsmap epoch 17 2023-11-24T07:57:47.653 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-24T07:57:47.777 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:47.776+0000 7f65cdee3700 1 Processor -- start 2023-11-24T07:57:47.778 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:47.776+0000 7f65cdee3700 1 -- start start 2023-11-24T07:57:47.780 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:47.776+0000 7f65cdee3700 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f65c815bde0 0x7f65c815c1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:57:47.780 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:47.776+0000 7f65cdee3700 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f65c80cf2b0 0x7f65c80d01b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:57:47.780 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:47.776+0000 7f65cdee3700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f65c80ce5c0 0x7f65c80cea70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:57:47.780 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:47.777+0000 7f65cdee3700 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f65c80cefb0 con 0x7f65c80ce5c0 2023-11-24T07:57:47.780 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:47.777+0000 7f65cdee3700 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f65c80d0a90 con 0x7f65c815bde0 2023-11-24T07:57:47.780 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:47.777+0000 7f65cdee3700 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f65c806ee10 con 0x7f65c80cf2b0 2023-11-24T07:57:47.780 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:47.777+0000 7f65c7fff700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f65c80ce5c0 0x7f65c80cea70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:57:47.781 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:47.777+0000 7f65c7fff700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f65c80ce5c0 0x7f65c80cea70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:52290/0 (socket says 172.21.15.59:52290) 2023-11-24T07:57:47.781 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:47.777+0000 7f65c7fff700 1 -- 172.21.15.59:0/745833742 learned_addr learned my addr 172.21.15.59:0/745833742 (peer_addr_for_me v2:172.21.15.59:0/0) 2023-11-24T07:57:47.781 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:47.777+0000 7f65c6ffd700 1 --2- 172.21.15.59:0/745833742 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f65c80cf2b0 0x7f65c80d01b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:57:47.781 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:47.777+0000 7f65c77fe700 1 --2- 172.21.15.59:0/745833742 >> v2:172.21.15.76:3300/0 conn(0x7f65c815bde0 0x7f65c815c1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:57:47.781 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:47.777+0000 7f65c7fff700 1 -- 172.21.15.59:0/745833742 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f65c80cf2b0 msgr2=0x7f65c80d01b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T07:57:47.781 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:47.777+0000 7f65c7fff700 1 --2- 172.21.15.59:0/745833742 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f65c80cf2b0 0x7f65c80d01b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:47.781 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:47.777+0000 7f65c7fff700 1 -- 172.21.15.59:0/745833742 >> v2:172.21.15.76:3300/0 conn(0x7f65c815bde0 msgr2=0x7f65c815c1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T07:57:47.781 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:47.777+0000 7f65c7fff700 1 --2- 172.21.15.59:0/745833742 >> v2:172.21.15.76:3300/0 conn(0x7f65c815bde0 0x7f65c815c1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:47.781 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:47.777+0000 7f65c7fff700 1 -- 172.21.15.59:0/745833742 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f65c806f0f0 con 0x7f65c80ce5c0 2023-11-24T07:57:47.781 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:47.777+0000 7f65c6ffd700 1 --2- 172.21.15.59:0/745833742 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f65c80cf2b0 0x7f65c80d01b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-24T07:57:47.781 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:47.777+0000 7f65c7fff700 1 --2- 172.21.15.59:0/745833742 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f65c80ce5c0 0x7f65c80cea70 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f65c8072370 tx=0x7f65c000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5b71c85c286ca4a3 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:57:47.781 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:47.777+0000 7f65c67fc700 1 -- 172.21.15.59:0/745833742 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65c000e070 con 0x7f65c80ce5c0 2023-11-24T07:57:47.782 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:47.777+0000 7f65c67fc700 1 -- 172.21.15.59:0/745833742 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f65c000b9c0 con 0x7f65c80ce5c0 2023-11-24T07:57:47.782 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:47.777+0000 7f65c67fc700 1 -- 172.21.15.59:0/745833742 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65c000bd70 con 0x7f65c80ce5c0 2023-11-24T07:57:47.782 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:47.778+0000 7f65cdee3700 1 -- 172.21.15.59:0/745833742 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f65c80ce5c0 msgr2=0x7f65c80cea70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:57:47.782 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:47.778+0000 7f65cdee3700 1 --2- 172.21.15.59:0/745833742 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f65c80ce5c0 0x7f65c80cea70 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f65c8072370 tx=0x7f65c000b330 comp rx=0 tx=0).stop 2023-11-24T07:57:47.782 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:47.778+0000 7f65cdee3700 1 -- 172.21.15.59:0/745833742 shutdown_connections 2023-11-24T07:57:47.782 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:47.778+0000 7f65cdee3700 1 --2- 172.21.15.59:0/745833742 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f65c80ce5c0 0x7f65c80cea70 unknown :-1 s=CLOSED pgs=466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:47.782 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:47.778+0000 7f65cdee3700 1 --2- 172.21.15.59:0/745833742 >> v2:172.21.15.76:3300/0 conn(0x7f65c815bde0 0x7f65c815c1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:47.782 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:47.778+0000 7f65cdee3700 1 --2- 172.21.15.59:0/745833742 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f65c80cf2b0 0x7f65c80d01b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:47.782 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:47.778+0000 7f65cdee3700 1 -- 172.21.15.59:0/745833742 >> 172.21.15.59:0/745833742 conn(0x7f65c80b3720 msgr2=0x7f65c80b3b30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:57:47.782 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:47.778+0000 7f65cdee3700 1 -- 172.21.15.59:0/745833742 shutdown_connections 2023-11-24T07:57:47.782 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:47.778+0000 7f65cdee3700 1 -- 172.21.15.59:0/745833742 wait complete. 2023-11-24T07:57:47.785 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:47.779+0000 7f65cdee3700 1 Processor -- start 2023-11-24T07:57:47.785 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:47.779+0000 7f65cdee3700 1 -- start start 2023-11-24T07:57:47.785 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:47.780+0000 7f65cdee3700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f65c80ce5c0 0x7f65c81595c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:57:47.785 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:47.780+0000 7f65cdee3700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f65c80cf2b0 0x7f65c80c65a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:57:47.785 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:47.780+0000 7f65cdee3700 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f65c815bde0 0x7f65c80c6ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:57:47.785 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:47.780+0000 7f65cdee3700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65c80b76d0 con 0x7f65c80ce5c0 2023-11-24T07:57:47.785 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:47.780+0000 7f65cdee3700 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65c80b5e00 con 0x7f65c80cf2b0 2023-11-24T07:57:47.785 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:47.780+0000 7f65cdee3700 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f65c80b60e0 con 0x7f65c815bde0 2023-11-24T07:57:47.785 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:47.780+0000 7f65c77fe700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f65c80ce5c0 0x7f65c81595c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:57:47.786 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:47.780+0000 7f65c77fe700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f65c80ce5c0 0x7f65c81595c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:52292/0 (socket says 172.21.15.59:52292) 2023-11-24T07:57:47.786 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:47.780+0000 7f65c77fe700 1 -- 172.21.15.59:0/1275084433 learned_addr learned my addr 172.21.15.59:0/1275084433 (peer_addr_for_me v2:172.21.15.59:0/0) 2023-11-24T07:57:47.786 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:47.780+0000 7f65c7fff700 1 --2- 172.21.15.59:0/1275084433 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f65c815bde0 0x7f65c80c6ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:57:47.786 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:47.780+0000 7f65c6ffd700 1 --2- 172.21.15.59:0/1275084433 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f65c80cf2b0 0x7f65c80c65a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:57:47.786 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:47.781+0000 7f65c77fe700 1 -- 172.21.15.59:0/1275084433 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f65c815bde0 msgr2=0x7f65c80c6ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:57:47.786 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:47.781+0000 7f65c77fe700 1 --2- 172.21.15.59:0/1275084433 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f65c815bde0 0x7f65c80c6ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:47.786 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:47.781+0000 7f65c77fe700 1 -- 172.21.15.59:0/1275084433 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f65c80cf2b0 msgr2=0x7f65c80c65a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:57:47.786 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:47.781+0000 7f65c77fe700 1 --2- 172.21.15.59:0/1275084433 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f65c80cf2b0 0x7f65c80c65a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:47.786 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:47.781+0000 7f65c77fe700 1 -- 172.21.15.59:0/1275084433 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f65c0009b30 con 0x7f65c80ce5c0 2023-11-24T07:57:47.786 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:47.781+0000 7f65c77fe700 1 --2- 172.21.15.59:0/1275084433 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f65c80ce5c0 0x7f65c81595c0 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f65b800d800 tx=0x7f65b800dbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:57:47.786 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:47.781+0000 7f65affff700 1 -- 172.21.15.59:0/1275084433 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65b80125f0 con 0x7f65c80ce5c0 2023-11-24T07:57:47.786 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:47.781+0000 7f65affff700 1 -- 172.21.15.59:0/1275084433 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f65b8012750 con 0x7f65c80ce5c0 2023-11-24T07:57:47.786 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:47.781+0000 7f65cdee3700 1 -- 172.21.15.59:0/1275084433 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f65c80b63c0 con 0x7f65c80ce5c0 2023-11-24T07:57:47.787 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:47.781+0000 7f65affff700 1 -- 172.21.15.59:0/1275084433 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65b8012b00 con 0x7f65c80ce5c0 2023-11-24T07:57:47.787 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:47.781+0000 7f65cdee3700 1 -- 172.21.15.59:0/1275084433 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f65c80b5220 con 0x7f65c80ce5c0 2023-11-24T07:57:47.787 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:47.782+0000 7f65cdee3700 1 -- 172.21.15.59:0/1275084433 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f65c80b68e0 con 0x7f65c80ce5c0 2023-11-24T07:57:47.787 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:47.782+0000 7f65affff700 1 -- 172.21.15.59:0/1275084433 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 6) v1 ==== 89282+0+0 (secure 0 0 0) 0x7f65b8012c60 con 0x7f65c80ce5c0 2023-11-24T07:57:47.787 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:47.783+0000 7f65affff700 1 --2- 172.21.15.59:0/1275084433 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f65b406af60 0x7f65b406d420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:57:47.787 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:47.783+0000 7f65affff700 1 -- 172.21.15.59:0/1275084433 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f65b8089910 con 0x7f65c80ce5c0 2023-11-24T07:57:47.787 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:47.783+0000 7f65c6ffd700 1 --2- 172.21.15.59:0/1275084433 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f65b406af60 0x7f65b406d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:57:47.787 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:47.783+0000 7f65c6ffd700 1 --2- 172.21.15.59:0/1275084433 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f65b406af60 0x7f65b406d420 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f65c815f580 tx=0x7f65bc008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:57:47.789 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:47.787+0000 7f65affff700 1 -- 172.21.15.59:0/1275084433 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f65b80549d0 con 0x7f65c80ce5c0 2023-11-24T07:57:47.991 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:47.989+0000 7f65cdee3700 1 -- 172.21.15.59:0/1275084433 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f65c80c8f90 con 0x7f65c80ce5c0 2023-11-24T07:57:47.992 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:47.990+0000 7f65affff700 1 -- 172.21.15.59:0/1275084433 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v33) v1 ==== 74+0+16437 (secure 0 0 0) 0x7f65b8057ff0 con 0x7f65c80ce5c0 2023-11-24T07:57:47.992 INFO:teuthology.orchestra.run.smithi059.stdout: 2023-11-24T07:57:47.992 INFO:teuthology.orchestra.run.smithi059.stdout:{"epoch":33,"fsid":"9f0e8741-b8f9-48d5-a2c3-09f45903303d","created":"2023-11-24T07:56:06.552729+0000","modified":"2023-11-24T07:57:26.579209+0000","last_up_change":"2023-11-24T07:56:17.372352+0000","last_in_change":"2023-11-24T07:56:09.073250+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-24T07:56:18.352823+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2023-11-24T07:57:23.014143+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"32","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2023-11-24T07:57:23.971487+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"33","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"6d8f9e9b-1782-4e41-928b-a39f2a8ebc06","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6800","nonce":3373878772},{"type":"v1","addr":"172.21.15.59:6801","nonce":3373878772}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6802","nonce":3373878772},{"type":"v1","addr":"172.21.15.59:6803","nonce":3373878772}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6806","nonce":3373878772},{"type":"v1","addr":"172.21.15.59:6807","nonce":3373878772}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6804","nonce":3373878772},{"type":"v1","addr":"172.21.15.59:6805","nonce":3373878772}]},"public_addr":"172.21.15.59:6801/3373878772","cluster_addr":"172.21.15.59:6803/3373878772","heartbeat_back_addr":"172.21.15.59:6807/3373878772","heartbeat_front_addr":"172.21.15.59:6805/3373878772","state":["exists","up"]},{"osd":1,"uuid":"e8d882aa-52e1-4c04-9e1e-c44ef9204a5f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6824","nonce":648258563},{"type":"v1","addr":"172.21.15.59:6825","nonce":648258563}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6826","nonce":648258563},{"type":"v1","addr":"172.21.15.59:6827","nonce":648258563}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6830","nonce":648258563},{"type":"v1","addr":"172.21.15.59:6831","nonce":648258563}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6828","nonce":648258563},{"type":"v1","addr":"172.21.15.59:6829","nonce":648258563}]},"public_addr":"172.21.15.59:6825/648258563","cluster_addr":"172.21.15.59:6827/648258563","heartbeat_back_addr":"172.21.15.59:6831/648258563","heartbeat_front_addr":"172.21.15.59:6829/648258563","state":["exists","up"]},{"osd":2,"uuid":"69eebfa0-695b-4b21-90b9-8a3537168a01","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6808","nonce":1840430654},{"type":"v1","addr":"172.21.15.59:6809","nonce":1840430654}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6810","nonce":1840430654},{"type":"v1","addr":"172.21.15.59:6811","nonce":1840430654}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6814","nonce":1840430654},{"type":"v1","addr":"172.21.15.59:6815","nonce":1840430654}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6812","nonce":1840430654},{"type":"v1","addr":"172.21.15.59:6813","nonce":1840430654}]},"public_addr":"172.21.15.59:6809/1840430654","cluster_addr":"172.21.15.59:6811/1840430654","heartbeat_back_addr":"172.21.15.59:6815/1840430654","heartbeat_front_addr":"172.21.15.59:6813/1840430654","state":["exists","up"]},{"osd":3,"uuid":"96fd77fc-f911-46ff-9445-6f0be2daa8f4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6816","nonce":2504025850},{"type":"v1","addr":"172.21.15.59:6817","nonce":2504025850}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6818","nonce":2504025850},{"type":"v1","addr":"172.21.15.59:6819","nonce":2504025850}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6822","nonce":2504025850},{"type":"v1","addr":"172.21.15.59:6823","nonce":2504025850}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6820","nonce":2504025850},{"type":"v1","addr":"172.21.15.59:6821","nonce":2504025850}]},"public_addr":"172.21.15.59:6817/2504025850","cluster_addr":"172.21.15.59:6819/2504025850","heartbeat_back_addr":"172.21.15.59:6823/2504025850","heartbeat_front_addr":"172.21.15.59:6821/2504025850","state":["exists","up"]},{"osd":4,"uuid":"7ac29fca-b5b2-4c95-9d69-2f164cbbc708","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6816","nonce":2382463431},{"type":"v1","addr":"172.21.15.76:6818","nonce":2382463431}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6820","nonce":2382463431},{"type":"v1","addr":"172.21.15.76:6822","nonce":2382463431}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6828","nonce":2382463431},{"type":"v1","addr":"172.21.15.76:6830","nonce":2382463431}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6824","nonce":2382463431},{"type":"v1","addr":"172.21.15.76:6826","nonce":2382463431}]},"public_addr":"172.21.15.76:6818/2382463431","cluster_addr":"172.21.15.76:6822/2382463431","heartbeat_back_addr":"172.21.15.76:6830/2382463431","heartbeat_front_addr":"172.21.15.76:6826/2382463431","state":["exists","up"]},{"osd":5,"uuid":"485bf711-3851-4238-a4f4-51d52e0a9c58","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6800","nonce":329334308},{"type":"v1","addr":"172.21.15.76:6802","nonce":329334308}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6805","nonce":329334308},{"type":"v1","addr":"172.21.15.76:6807","nonce":329334308}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6814","nonce":329334308},{"type":"v1","addr":"172.21.15.76:6815","nonce":329334308}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6810","nonce":329334308},{"type":"v1","addr":"172.21.15.76:6812","nonce":329334308}]},"public_addr":"172.21.15.76:6802/329334308","cluster_addr":"172.21.15.76:6807/329334308","heartbeat_back_addr":"172.21.15.76:6815/329334308","heartbeat_front_addr":"172.21.15.76:6812/329334308","state":["exists","up"]},{"osd":6,"uuid":"72b3f2fc-1e4a-4e32-968d-003647a0286f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6801","nonce":3946579410},{"type":"v1","addr":"172.21.15.76:6803","nonce":3946579410}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6804","nonce":3946579410},{"type":"v1","addr":"172.21.15.76:6806","nonce":3946579410}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6811","nonce":3946579410},{"type":"v1","addr":"172.21.15.76:6813","nonce":3946579410}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6808","nonce":3946579410},{"type":"v1","addr":"172.21.15.76:6809","nonce":3946579410}]},"public_addr":"172.21.15.76:6803/3946579410","cluster_addr":"172.21.15.76:6806/3946579410","heartbeat_back_addr":"172.21.15.76:6813/3946579410","heartbeat_front_addr":"172.21.15.76:6809/3946579410","state":["exists","up"]},{"osd":7,"uuid":"1ddceb90-6c5d-4d1a-ad56-99cb877f8c70","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6817","nonce":841761538},{"type":"v1","addr":"172.21.15.76:6819","nonce":841761538}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6821","nonce":841761538},{"type":"v1","addr":"172.21.15.76:6823","nonce":841761538}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6829","nonce":841761538},{"type":"v1","addr":"172.21.15.76:6831","nonce":841761538}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6825","nonce":841761538},{"type":"v1","addr":"172.21.15.76:6827","nonce":841761538}]},"public_addr":"172.21.15.76:6819/841761538","cluster_addr":"172.21.15.76:6823/841761538","heartbeat_back_addr":"172.21.15.76:6831/841761538","heartbeat_front_addr":"172.21.15.76:6827/841761538","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T07:56:14.294838+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T07:56:14.270779+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T07:56:14.231809+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T07:56:14.260983+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T07:56:14.302997+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T07:56:14.258600+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T07:56:14.251413+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T07:56:14.304727+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-24T07:57:47.994 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:47.992+0000 7f65cdee3700 1 -- 172.21.15.59:0/1275084433 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f65b406af60 msgr2=0x7f65b406d420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:57:47.994 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:47.992+0000 7f65cdee3700 1 --2- 172.21.15.59:0/1275084433 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f65b406af60 0x7f65b406d420 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f65c815f580 tx=0x7f65bc008040 comp rx=0 tx=0).stop 2023-11-24T07:57:47.995 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:47.992+0000 7f65cdee3700 1 -- 172.21.15.59:0/1275084433 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f65c80ce5c0 msgr2=0x7f65c81595c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:57:47.995 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:47.992+0000 7f65cdee3700 1 --2- 172.21.15.59:0/1275084433 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f65c80ce5c0 0x7f65c81595c0 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f65b800d800 tx=0x7f65b800dbc0 comp rx=0 tx=0).stop 2023-11-24T07:57:47.995 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:47.992+0000 7f65cdee3700 1 -- 172.21.15.59:0/1275084433 shutdown_connections 2023-11-24T07:57:47.995 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:47.992+0000 7f65cdee3700 1 --2- 172.21.15.59:0/1275084433 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f65b406af60 0x7f65b406d420 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:47.995 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:47.992+0000 7f65cdee3700 1 --2- 172.21.15.59:0/1275084433 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f65c815bde0 0x7f65c80c6ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:47.995 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:47.992+0000 7f65cdee3700 1 --2- 172.21.15.59:0/1275084433 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f65c80ce5c0 0x7f65c81595c0 unknown :-1 s=CLOSED pgs=467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:47.995 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:47.992+0000 7f65cdee3700 1 --2- 172.21.15.59:0/1275084433 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f65c80cf2b0 0x7f65c80c65a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:47.995 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:47.992+0000 7f65cdee3700 1 -- 172.21.15.59:0/1275084433 >> 172.21.15.59:0/1275084433 conn(0x7f65c80b3720 msgr2=0x7f65c815a760 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:57:47.995 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:47.993+0000 7f65cdee3700 1 -- 172.21.15.59:0/1275084433 shutdown_connections 2023-11-24T07:57:47.995 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:47.993+0000 7f65cdee3700 1 -- 172.21.15.59:0/1275084433 wait complete. 2023-11-24T07:57:48.007 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2023-11-24T07:57:48.007 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-24T07:57:48.131 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:48.130+0000 7f0eb4d7d700 1 Processor -- start 2023-11-24T07:57:48.132 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:48.130+0000 7f0eb4d7d700 1 -- start start 2023-11-24T07:57:48.135 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:48.131+0000 7f0eb4d7d700 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f0eb0142810 0x7f0eb006afd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:57:48.135 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:48.131+0000 7f0eb4d7d700 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f0eb006e580 0x7f0eb006b510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:57:48.135 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:48.131+0000 7f0eb4d7d700 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0eb00cf5f0 con 0x7f0eb01459a0 2023-11-24T07:57:48.136 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:48.131+0000 7f0eb4d7d700 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0eb006ea80 con 0x7f0eb0142810 2023-11-24T07:57:48.136 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:48.131+0000 7f0eb4d7d700 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0eb006ed60 con 0x7f0eb006e580 2023-11-24T07:57:48.136 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:48.131+0000 7f0eadd9b700 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7f0eb01459a0 0x7f0eb0071bb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:55776/0 (socket says 172.21.15.59:55776) 2023-11-24T07:57:48.136 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:48.131+0000 7f0eadd9b700 1 -- 172.21.15.59:0/3888541155 learned_addr learned my addr 172.21.15.59:0/3888541155 (peer_addr_for_me v1:172.21.15.59:0/0) 2023-11-24T07:57:48.136 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:48.131+0000 7f0eae59c700 1 --2- 172.21.15.59:0/3888541155 >> v2:172.21.15.76:3300/0 conn(0x7f0eb0142810 0x7f0eb006afd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:57:48.136 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:48.131+0000 7f0ead59a700 1 -- 172.21.15.59:0/3888541155 <== mon.0 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3725199889 0 0) 0x7f0eb00cf5f0 con 0x7f0eb01459a0 2023-11-24T07:57:48.136 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:48.131+0000 7f0eaed9d700 1 --2- 172.21.15.59:0/3888541155 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f0eb006e580 0x7f0eb006b510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:57:48.136 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:48.131+0000 7f0ead59a700 1 -- 172.21.15.59:0/3888541155 --> v1:172.21.15.59:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0e980035b0 con 0x7f0eb01459a0 2023-11-24T07:57:48.136 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:48.131+0000 7f0ead59a700 1 -- 172.21.15.59:0/3888541155 <== mon.0 v1:172.21.15.59:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2680103671 0 0) 0x7f0e980035b0 con 0x7f0eb01459a0 2023-11-24T07:57:48.136 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:48.131+0000 7f0ead59a700 1 -- 172.21.15.59:0/3888541155 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f0eb006e580 msgr2=0x7f0eb006b510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T07:57:48.136 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:48.131+0000 7f0ead59a700 1 --2- 172.21.15.59:0/3888541155 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f0eb006e580 0x7f0eb006b510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:48.137 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:48.131+0000 7f0ead59a700 1 -- 172.21.15.59:0/3888541155 >> v2:172.21.15.76:3300/0 conn(0x7f0eb0142810 msgr2=0x7f0eb006afd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T07:57:48.137 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:48.131+0000 7f0ead59a700 1 --2- 172.21.15.59:0/3888541155 >> v2:172.21.15.76:3300/0 conn(0x7f0eb0142810 0x7f0eb006afd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:48.137 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:48.131+0000 7f0ead59a700 1 -- 172.21.15.59:0/3888541155 --> v1:172.21.15.59:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0eb006f040 con 0x7f0eb01459a0 2023-11-24T07:57:48.137 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:48.132+0000 7f0ead59a700 1 -- 172.21.15.59:0/3888541155 <== mon.0 v1:172.21.15.59:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1312333566 0 0) 0x7f0ea4001670 con 0x7f0eb01459a0 2023-11-24T07:57:48.137 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:48.132+0000 7f0ead59a700 1 -- 172.21.15.59:0/3888541155 >> v1:172.21.15.59:6789/0 conn(0x7f0eb01459a0 legacy=0x7f0eb0071bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:57:48.137 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:48.132+0000 7f0ead59a700 1 --2- 172.21.15.59:0/3888541155 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0e98003a00 0x7f0e98003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:57:48.137 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:48.132+0000 7f0ead59a700 1 --2- 172.21.15.59:0/3888541155 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0e98004850 0x7f0e98004cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:57:48.137 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:48.132+0000 7f0ead59a700 1 --2- 172.21.15.59:0/3888541155 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f0eb006e580 0x7f0e98005210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:57:48.137 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:48.132+0000 7f0ead59a700 1 -- 172.21.15.59:0/3888541155 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0eb006ea80 con 0x7f0e98003a00 2023-11-24T07:57:48.137 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:48.132+0000 7f0ead59a700 1 -- 172.21.15.59:0/3888541155 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0eb006ed60 con 0x7f0e98004850 2023-11-24T07:57:48.137 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:48.132+0000 7f0ead59a700 1 -- 172.21.15.59:0/3888541155 --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0e9800ba10 con 0x7f0eb006e580 2023-11-24T07:57:48.137 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:48.132+0000 7f0eae59c700 1 --2- 172.21.15.59:0/3888541155 >> v2:172.21.15.76:3300/0 conn(0x7f0eb0142810 0x7f0eb006afd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-24T07:57:48.138 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:48.132+0000 7f0eae59c700 1 --2- 172.21.15.59:0/3888541155 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0e98003a00 0x7f0e98003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:57:48.138 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:48.132+0000 7f0eaed9d700 1 --2- 172.21.15.59:0/3888541155 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f0eb006e580 0x7f0e98005210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:57:48.138 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:48.132+0000 7f0eadd9b700 1 --2- 172.21.15.59:0/3888541155 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0e98004850 0x7f0e98004cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:57:48.138 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:48.132+0000 7f0eae59c700 1 -- 172.21.15.59:0/3888541155 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f0eb006e580 msgr2=0x7f0e98005210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T07:57:48.138 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:48.132+0000 7f0eae59c700 1 --2- 172.21.15.59:0/3888541155 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f0eb006e580 0x7f0e98005210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:48.138 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:48.132+0000 7f0eae59c700 1 -- 172.21.15.59:0/3888541155 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0e98004850 msgr2=0x7f0e98004cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T07:57:48.138 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:48.132+0000 7f0eae59c700 1 --2- 172.21.15.59:0/3888541155 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0e98004850 0x7f0e98004cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:48.138 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:48.132+0000 7f0eae59c700 1 -- 172.21.15.59:0/3888541155 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0e980035b0 con 0x7f0e98003a00 2023-11-24T07:57:48.138 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:48.132+0000 7f0eaed9d700 1 --2- 172.21.15.59:0/3888541155 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f0eb006e580 0x7f0e98005210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-24T07:57:48.138 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:48.132+0000 7f0eae59c700 1 --2- 172.21.15.59:0/3888541155 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0e98003a00 0x7f0e98003e20 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7f0ea0009d40 tx=0x7f0e98005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=419053feceb4ccea server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:57:48.138 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:48.132+0000 7f0ead59a700 1 -- 172.21.15.59:0/3888541155 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ea0009e90 con 0x7f0e98003a00 2023-11-24T07:57:48.138 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:48.132+0000 7f0ead59a700 1 -- 172.21.15.59:0/3888541155 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0ea000e9a0 con 0x7f0e98003a00 2023-11-24T07:57:48.139 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:48.132+0000 7f0ead59a700 1 -- 172.21.15.59:0/3888541155 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ea000ed10 con 0x7f0e98003a00 2023-11-24T07:57:48.139 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:48.133+0000 7f0eb4d7d700 1 -- 172.21.15.59:0/3888541155 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0e98003a00 msgr2=0x7f0e98003e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:57:48.139 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:48.133+0000 7f0eb4d7d700 1 --2- 172.21.15.59:0/3888541155 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0e98003a00 0x7f0e98003e20 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7f0ea0009d40 tx=0x7f0e98005880 comp rx=0 tx=0).stop 2023-11-24T07:57:48.139 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:48.133+0000 7f0eaed9d700 1 -- 172.21.15.59:0/3888541155 reap_dead start 2023-11-24T07:57:48.139 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:48.133+0000 7f0eb4d7d700 1 -- 172.21.15.59:0/3888541155 shutdown_connections 2023-11-24T07:57:48.139 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:48.133+0000 7f0eb4d7d700 1 -- 172.21.15.59:0/3888541155 >> 172.21.15.59:0/3888541155 conn(0x7f0eb00b2740 msgr2=0x7f0eb00b2b20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:57:48.139 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:48.133+0000 7f0eb4d7d700 1 -- 172.21.15.59:0/3888541155 shutdown_connections 2023-11-24T07:57:48.139 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:48.133+0000 7f0eb4d7d700 1 -- 172.21.15.59:0/3888541155 wait complete. 2023-11-24T07:57:48.141 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:48.135+0000 7f0eb4d7d700 1 Processor -- start 2023-11-24T07:57:48.141 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:48.135+0000 7f0eb4d7d700 1 -- start start 2023-11-24T07:57:48.141 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:48.135+0000 7f0eb4d7d700 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f0e98003a00 0x7f0eb014b1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:57:48.142 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:48.135+0000 7f0eb4d7d700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0eb01f1260 0x7f0eb014ef90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:57:48.142 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:48.135+0000 7f0eb4d7d700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0eb0140a50 0x7f0eb0140ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:57:48.142 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:48.135+0000 7f0eb4d7d700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0eb0141410 con 0x7f0eb0140a50 2023-11-24T07:57:48.142 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:48.135+0000 7f0eb4d7d700 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0eb014f4d0 con 0x7f0eb01f1260 2023-11-24T07:57:48.142 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:48.135+0000 7f0eb4d7d700 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0eb014c0b0 con 0x7f0e98003a00 2023-11-24T07:57:48.142 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:48.135+0000 7f0eaed9d700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0eb0140a50 0x7f0eb0140ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:57:48.142 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:48.135+0000 7f0eaed9d700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0eb0140a50 0x7f0eb0140ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:52316/0 (socket says 172.21.15.59:52316) 2023-11-24T07:57:48.142 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:48.135+0000 7f0eaed9d700 1 -- 172.21.15.59:0/2058023371 learned_addr learned my addr 172.21.15.59:0/2058023371 (peer_addr_for_me v2:172.21.15.59:0/0) 2023-11-24T07:57:48.142 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:48.136+0000 7f0eae59c700 1 --2- 172.21.15.59:0/2058023371 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f0e98003a00 0x7f0eb014b1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:57:48.142 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:48.136+0000 7f0eadd9b700 1 --2- 172.21.15.59:0/2058023371 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0eb01f1260 0x7f0eb014ef90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:57:48.142 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:48.136+0000 7f0eaed9d700 1 -- 172.21.15.59:0/2058023371 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f0e98003a00 msgr2=0x7f0eb014b1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:57:48.142 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:48.136+0000 7f0eaed9d700 1 --2- 172.21.15.59:0/2058023371 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f0e98003a00 0x7f0eb014b1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:48.143 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:48.136+0000 7f0eaed9d700 1 -- 172.21.15.59:0/2058023371 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0eb01f1260 msgr2=0x7f0eb014ef90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:57:48.143 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:48.136+0000 7f0eaed9d700 1 --2- 172.21.15.59:0/2058023371 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0eb01f1260 0x7f0eb014ef90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:48.143 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:48.136+0000 7f0eaed9d700 1 -- 172.21.15.59:0/2058023371 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0eb014b810 con 0x7f0eb0140a50 2023-11-24T07:57:48.143 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:48.136+0000 7f0eadd9b700 1 --2- 172.21.15.59:0/2058023371 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0eb01f1260 0x7f0eb014ef90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-24T07:57:48.143 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:48.136+0000 7f0eae59c700 1 --2- 172.21.15.59:0/2058023371 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f0e98003a00 0x7f0eb014b1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-24T07:57:48.143 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:48.136+0000 7f0eaed9d700 1 --2- 172.21.15.59:0/2058023371 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0eb0140a50 0x7f0eb0140ed0 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7f0eb015fa70 tx=0x7f0ea800f490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:57:48.143 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:48.136+0000 7f0e9effd700 1 -- 172.21.15.59:0/2058023371 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ea800e040 con 0x7f0eb0140a50 2023-11-24T07:57:48.143 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:48.136+0000 7f0e9effd700 1 -- 172.21.15.59:0/2058023371 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0ea800f040 con 0x7f0eb0140a50 2023-11-24T07:57:48.143 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:48.136+0000 7f0eb4d7d700 1 -- 172.21.15.59:0/2058023371 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0eb014baf0 con 0x7f0eb0140a50 2023-11-24T07:57:48.143 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:48.136+0000 7f0e9effd700 1 -- 172.21.15.59:0/2058023371 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ea800fbd0 con 0x7f0eb0140a50 2023-11-24T07:57:48.143 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:48.136+0000 7f0eb4d7d700 1 -- 172.21.15.59:0/2058023371 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0eb013bf20 con 0x7f0eb0140a50 2023-11-24T07:57:48.143 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:48.137+0000 7f0eb4d7d700 1 -- 172.21.15.59:0/2058023371 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0eb013bd80 con 0x7f0eb0140a50 2023-11-24T07:57:48.143 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:48.138+0000 7f0e9effd700 1 -- 172.21.15.59:0/2058023371 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 6) v1 ==== 89282+0+0 (secure 0 0 0) 0x7f0ea800fd30 con 0x7f0eb0140a50 2023-11-24T07:57:48.143 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:48.138+0000 7f0e9effd700 1 --2- 172.21.15.59:0/2058023371 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f0e9006af60 0x7f0e9006d420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:57:48.144 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:48.138+0000 7f0e9effd700 1 -- 172.21.15.59:0/2058023371 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f0ea8089dc0 con 0x7f0eb0140a50 2023-11-24T07:57:48.144 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:48.138+0000 7f0eae59c700 1 --2- 172.21.15.59:0/2058023371 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f0e9006af60 0x7f0e9006d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:57:48.144 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:48.139+0000 7f0eae59c700 1 --2- 172.21.15.59:0/2058023371 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f0e9006af60 0x7f0e9006d420 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f0ea0009ce0 tx=0x7f0ea0008030 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:57:48.144 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:48.142+0000 7f0e9effd700 1 -- 172.21.15.59:0/2058023371 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f0eb013bd80 con 0x7f0eb0140a50 2023-11-24T07:57:48.389 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:48.388+0000 7f0eb4d7d700 1 -- 172.21.15.59:0/2058023371 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f0eb013bd80 con 0x7f0eb0140a50 2023-11-24T07:57:48.390 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:48.388+0000 7f0e9effd700 1 -- 172.21.15.59:0/2058023371 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) v1 ==== 94+0+4913 (secure 0 0 0) 0x7f0eb013bd80 con 0x7f0eb0140a50 2023-11-24T07:57:48.390 INFO:teuthology.orchestra.run.smithi059.stdout: 2023-11-24T07:57:48.390 INFO:teuthology.orchestra.run.smithi059.stdout:{"epoch":17,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4763,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.76:6836/1669441817","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6834","nonce":1669441817},{"type":"v1","addr":"172.21.15.76:6836","nonce":1669441817}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":4769,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.76:6837/3709928230","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6835","nonce":3709928230},{"type":"v1","addr":"172.21.15.76:6837","nonce":3709928230}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":5106,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6834/2019319496","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6832","nonce":2019319496},{"type":"v1","addr":"172.21.15.59:6834","nonce":2019319496}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":16,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-24T07:57:25.003636+0000","modified":"2023-11-24T07:57:28.021959+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5109},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5109":{"gid":5109,"name":"a","rank":0,"incarnation":13,"state":"up:active","state_seq":3,"addr":"172.21.15.59:6835/2290655475","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6833","nonce":2290655475},{"type":"v1","addr":"172.21.15.59:6835","nonce":2290655475}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2023-11-24T07:57:48.392 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:48.390+0000 7f0eb4d7d700 1 -- 172.21.15.59:0/2058023371 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f0e9006af60 msgr2=0x7f0e9006d420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:57:48.392 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:48.390+0000 7f0eb4d7d700 1 --2- 172.21.15.59:0/2058023371 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f0e9006af60 0x7f0e9006d420 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f0ea0009ce0 tx=0x7f0ea0008030 comp rx=0 tx=0).stop 2023-11-24T07:57:48.392 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:48.390+0000 7f0eb4d7d700 1 -- 172.21.15.59:0/2058023371 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0eb0140a50 msgr2=0x7f0eb0140ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:57:48.392 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:48.390+0000 7f0eb4d7d700 1 --2- 172.21.15.59:0/2058023371 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0eb0140a50 0x7f0eb0140ed0 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7f0eb015fa70 tx=0x7f0ea800f490 comp rx=0 tx=0).stop 2023-11-24T07:57:48.392 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:48.390+0000 7f0eb4d7d700 1 -- 172.21.15.59:0/2058023371 shutdown_connections 2023-11-24T07:57:48.393 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:48.390+0000 7f0eb4d7d700 1 --2- 172.21.15.59:0/2058023371 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f0e9006af60 0x7f0e9006d420 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:48.393 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:48.390+0000 7f0eb4d7d700 1 --2- 172.21.15.59:0/2058023371 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0eb0140a50 0x7f0eb0140ed0 unknown :-1 s=CLOSED pgs=470 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:48.393 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:48.390+0000 7f0eb4d7d700 1 --2- 172.21.15.59:0/2058023371 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f0e98003a00 0x7f0eb014b1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:48.393 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:48.390+0000 7f0eb4d7d700 1 --2- 172.21.15.59:0/2058023371 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0eb01f1260 0x7f0eb014ef90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:48.393 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:48.390+0000 7f0eb4d7d700 1 -- 172.21.15.59:0/2058023371 >> 172.21.15.59:0/2058023371 conn(0x7f0eb00b2740 msgr2=0x7f0eb0141c90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:57:48.393 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:48.390+0000 7f0eb4d7d700 1 -- 172.21.15.59:0/2058023371 shutdown_connections 2023-11-24T07:57:48.393 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:48.390+0000 7f0eb4d7d700 1 -- 172.21.15.59:0/2058023371 wait complete. 2023-11-24T07:57:48.393 INFO:teuthology.orchestra.run.smithi059.stderr:dumped fsmap epoch 17 2023-11-24T07:57:48.404 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-24T07:57:48.530 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:48.528+0000 7fae0358f700 1 Processor -- start 2023-11-24T07:57:48.530 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:48.528+0000 7fae0358f700 1 -- start start 2023-11-24T07:57:48.532 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:48.529+0000 7fae0358f700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fadfc143ae0 0x7fadfc0b51b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:57:48.532 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:48.529+0000 7fae0358f700 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fadfc0722c0 0x7fadfc06e510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:57:48.532 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:48.529+0000 7fae0358f700 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fadfc143ec0 con 0x7fadfc1440f0 2023-11-24T07:57:48.532 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:48.529+0000 7fae0358f700 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fadfc0b40a0 con 0x7fadfc143ae0 2023-11-24T07:57:48.533 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:48.529+0000 7fae0358f700 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fadfc0b3ae0 con 0x7fadfc0722c0 2023-11-24T07:57:48.533 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:48.529+0000 7fae0132b700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fadfc143ae0 0x7fadfc0b51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:57:48.533 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:48.529+0000 7fae0132b700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fadfc143ae0 0x7fadfc0b51b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:52320/0 (socket says 172.21.15.59:52320) 2023-11-24T07:57:48.533 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:48.529+0000 7fae0132b700 1 -- 172.21.15.59:0/1551995729 learned_addr learned my addr 172.21.15.59:0/1551995729 (peer_addr_for_me v2:172.21.15.59:0/0) 2023-11-24T07:57:48.533 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:48.529+0000 7fae01b2c700 1 --2- 172.21.15.59:0/1551995729 >> v2:172.21.15.76:3300/0 conn(0x7fadfc0722c0 0x7fadfc06e510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:57:48.533 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:48.530+0000 7fae0132b700 1 -- 172.21.15.59:0/1551995729 >> v2:172.21.15.76:3300/0 conn(0x7fadfc0722c0 msgr2=0x7fadfc06e510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T07:57:48.533 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:48.530+0000 7fae0132b700 1 --2- 172.21.15.59:0/1551995729 >> v2:172.21.15.76:3300/0 conn(0x7fadfc0722c0 0x7fadfc06e510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:48.533 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:48.530+0000 7fae0132b700 1 -- 172.21.15.59:0/1551995729 >> v1:172.21.15.76:6789/0 conn(0x7fadfc1440f0 legacy=0x7fadfc071bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:57:48.533 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:48.530+0000 7fae0132b700 1 -- 172.21.15.59:0/1551995729 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fadfc0b3dc0 con 0x7fadfc143ae0 2023-11-24T07:57:48.533 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:48.530+0000 7fae0132b700 1 --2- 172.21.15.59:0/1551995729 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fadfc143ae0 0x7fadfc0b51b0 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7fadfc0747c0 tx=0x7fadec00be70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f3a392c4a166c659 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:57:48.533 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:48.530+0000 7fadf3fff700 1 -- 172.21.15.59:0/1551995729 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fadec00dd30 con 0x7fadfc143ae0 2023-11-24T07:57:48.533 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:48.530+0000 7fadf3fff700 1 -- 172.21.15.59:0/1551995729 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fadec00de90 con 0x7fadfc143ae0 2023-11-24T07:57:48.533 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:48.530+0000 7fadf3fff700 1 -- 172.21.15.59:0/1551995729 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fadec010690 con 0x7fadfc143ae0 2023-11-24T07:57:48.533 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:48.530+0000 7fae0358f700 1 -- 172.21.15.59:0/1551995729 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fadfc143ae0 msgr2=0x7fadfc0b51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:57:48.534 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:48.530+0000 7fae0358f700 1 --2- 172.21.15.59:0/1551995729 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fadfc143ae0 0x7fadfc0b51b0 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7fadfc0747c0 tx=0x7fadec00be70 comp rx=0 tx=0).stop 2023-11-24T07:57:48.534 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:48.531+0000 7fae0358f700 1 -- 172.21.15.59:0/1551995729 shutdown_connections 2023-11-24T07:57:48.534 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:48.531+0000 7fae0358f700 1 --2- 172.21.15.59:0/1551995729 >> v2:172.21.15.76:3300/0 conn(0x7fadfc0722c0 0x7fadfc06e510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:48.534 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:48.531+0000 7fae0358f700 1 --2- 172.21.15.59:0/1551995729 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fadfc143ae0 0x7fadfc0b51b0 unknown :-1 s=CLOSED pgs=471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:48.534 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:48.531+0000 7fae0358f700 1 -- 172.21.15.59:0/1551995729 >> 172.21.15.59:0/1551995729 conn(0x7fadfc0b2740 msgr2=0x7fadfc0b2b20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:57:48.534 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:48.531+0000 7fae0358f700 1 -- 172.21.15.59:0/1551995729 shutdown_connections 2023-11-24T07:57:48.534 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:48.531+0000 7fae0358f700 1 -- 172.21.15.59:0/1551995729 wait complete. 2023-11-24T07:57:48.537 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:48.532+0000 7fae0358f700 1 Processor -- start 2023-11-24T07:57:48.537 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:48.532+0000 7fae0358f700 1 -- start start 2023-11-24T07:57:48.537 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:48.533+0000 7fae0358f700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fadfc0722c0 0x7fadfc132aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:57:48.537 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:48.533+0000 7fae0358f700 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fadfc143ae0 0x7fadfc1f4da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:57:48.537 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:48.533+0000 7fae0358f700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fadfc1440f0 0x7fadfc134b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:57:48.537 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:48.533+0000 7fae0358f700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fadfc135040 con 0x7fadfc1440f0 2023-11-24T07:57:48.537 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:48.533+0000 7fae0358f700 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fadfc135ad0 con 0x7fadfc0722c0 2023-11-24T07:57:48.537 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:48.533+0000 7fae0358f700 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fadfc141f00 con 0x7fadfc143ae0 2023-11-24T07:57:48.537 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:48.533+0000 7fae01b2c700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fadfc1440f0 0x7fadfc134b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:57:48.537 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:48.533+0000 7fae01b2c700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fadfc1440f0 0x7fadfc134b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:52330/0 (socket says 172.21.15.59:52330) 2023-11-24T07:57:48.537 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:48.533+0000 7fae01b2c700 1 -- 172.21.15.59:0/2744239223 learned_addr learned my addr 172.21.15.59:0/2744239223 (peer_addr_for_me v2:172.21.15.59:0/0) 2023-11-24T07:57:48.538 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:48.533+0000 7fae0132b700 1 --2- 172.21.15.59:0/2744239223 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fadfc0722c0 0x7fadfc132aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:57:48.538 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:48.533+0000 7fae00b2a700 1 --2- 172.21.15.59:0/2744239223 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fadfc143ae0 0x7fadfc1f4da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:57:48.538 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:48.533+0000 7fae01b2c700 1 -- 172.21.15.59:0/2744239223 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fadfc143ae0 msgr2=0x7fadfc1f4da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:57:48.538 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:48.533+0000 7fae01b2c700 1 --2- 172.21.15.59:0/2744239223 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fadfc143ae0 0x7fadfc1f4da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:48.538 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:48.533+0000 7fae01b2c700 1 -- 172.21.15.59:0/2744239223 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fadfc0722c0 msgr2=0x7fadfc132aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:57:48.538 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:48.533+0000 7fae01b2c700 1 --2- 172.21.15.59:0/2744239223 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fadfc0722c0 0x7fadfc132aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:48.538 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:48.533+0000 7fae01b2c700 1 -- 172.21.15.59:0/2744239223 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fadec00b7f0 con 0x7fadfc1440f0 2023-11-24T07:57:48.538 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:48.533+0000 7fae00b2a700 1 --2- 172.21.15.59:0/2744239223 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fadfc143ae0 0x7fadfc1f4da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T07:57:48.538 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:48.533+0000 7fae0132b700 1 --2- 172.21.15.59:0/2744239223 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fadfc0722c0 0x7fadfc132aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-24T07:57:48.538 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:48.533+0000 7fae01b2c700 1 --2- 172.21.15.59:0/2744239223 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fadfc1440f0 0x7fadfc134b00 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7fadf4007de0 tx=0x7fadf400e3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:57:48.538 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:48.533+0000 7fadf1ffb700 1 -- 172.21.15.59:0/2744239223 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fadf400ede0 con 0x7fadfc1440f0 2023-11-24T07:57:48.538 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:48.533+0000 7fadf1ffb700 1 -- 172.21.15.59:0/2744239223 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fadf4014420 con 0x7fadfc1440f0 2023-11-24T07:57:48.538 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:48.534+0000 7fae0358f700 1 -- 172.21.15.59:0/2744239223 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fadfc135230 con 0x7fadfc1440f0 2023-11-24T07:57:48.539 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:48.534+0000 7fadf1ffb700 1 -- 172.21.15.59:0/2744239223 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fadf40147d0 con 0x7fadfc1440f0 2023-11-24T07:57:48.539 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:48.534+0000 7fae0358f700 1 -- 172.21.15.59:0/2744239223 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fadfc1357f0 con 0x7fadfc1440f0 2023-11-24T07:57:48.539 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:48.535+0000 7fae0358f700 1 -- 172.21.15.59:0/2744239223 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fadfc142510 con 0x7fadfc1440f0 2023-11-24T07:57:48.539 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:48.535+0000 7fadf1ffb700 1 -- 172.21.15.59:0/2744239223 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 6) v1 ==== 89282+0+0 (secure 0 0 0) 0x7fadf4014930 con 0x7fadfc1440f0 2023-11-24T07:57:48.539 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:48.535+0000 7fadf1ffb700 1 --2- 172.21.15.59:0/2744239223 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7fade006af60 0x7fade006d420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:57:48.539 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:48.535+0000 7fadf1ffb700 1 -- 172.21.15.59:0/2744239223 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fadf4089d70 con 0x7fadfc1440f0 2023-11-24T07:57:48.539 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:48.536+0000 7fae0132b700 1 --2- 172.21.15.59:0/2744239223 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7fade006af60 0x7fade006d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:57:48.539 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:48.536+0000 7fae0132b700 1 --2- 172.21.15.59:0/2744239223 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7fade006af60 0x7fade006d420 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7fadec00b6f0 tx=0x7fadec012000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:57:48.541 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:48.540+0000 7fadf1ffb700 1 -- 172.21.15.59:0/2744239223 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fadf4054e30 con 0x7fadfc1440f0 2023-11-24T07:57:48.787 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:48.786+0000 7fae0358f700 1 -- 172.21.15.59:0/2744239223 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fadfc144830 con 0x7fadfc1440f0 2023-11-24T07:57:48.788 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:48.786+0000 7fadf1ffb700 1 -- 172.21.15.59:0/2744239223 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) v1 ==== 94+0+4913 (secure 0 0 0) 0x7fadf4058450 con 0x7fadfc1440f0 2023-11-24T07:57:48.788 INFO:teuthology.orchestra.run.smithi059.stdout: 2023-11-24T07:57:48.788 INFO:teuthology.orchestra.run.smithi059.stdout:{"epoch":17,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4763,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.76:6836/1669441817","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6834","nonce":1669441817},{"type":"v1","addr":"172.21.15.76:6836","nonce":1669441817}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":4769,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.76:6837/3709928230","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6835","nonce":3709928230},{"type":"v1","addr":"172.21.15.76:6837","nonce":3709928230}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":5106,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6834/2019319496","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6832","nonce":2019319496},{"type":"v1","addr":"172.21.15.59:6834","nonce":2019319496}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":16,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-24T07:57:25.003636+0000","modified":"2023-11-24T07:57:28.021959+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5109},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5109":{"gid":5109,"name":"a","rank":0,"incarnation":13,"state":"up:active","state_seq":3,"addr":"172.21.15.59:6835/2290655475","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6833","nonce":2290655475},{"type":"v1","addr":"172.21.15.59:6835","nonce":2290655475}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2023-11-24T07:57:48.790 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:48.788+0000 7fae0358f700 1 -- 172.21.15.59:0/2744239223 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7fade006af60 msgr2=0x7fade006d420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:57:48.791 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:48.788+0000 7fae0358f700 1 --2- 172.21.15.59:0/2744239223 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7fade006af60 0x7fade006d420 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7fadec00b6f0 tx=0x7fadec012000 comp rx=0 tx=0).stop 2023-11-24T07:57:48.791 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:48.788+0000 7fae0358f700 1 -- 172.21.15.59:0/2744239223 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fadfc1440f0 msgr2=0x7fadfc134b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:57:48.791 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:48.788+0000 7fae0358f700 1 --2- 172.21.15.59:0/2744239223 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fadfc1440f0 0x7fadfc134b00 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7fadf4007de0 tx=0x7fadf400e3c0 comp rx=0 tx=0).stop 2023-11-24T07:57:48.791 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:48.788+0000 7fae0358f700 1 -- 172.21.15.59:0/2744239223 shutdown_connections 2023-11-24T07:57:48.791 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:48.788+0000 7fae0358f700 1 --2- 172.21.15.59:0/2744239223 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7fade006af60 0x7fade006d420 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:48.791 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:48.788+0000 7fae0358f700 1 --2- 172.21.15.59:0/2744239223 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fadfc1440f0 0x7fadfc134b00 unknown :-1 s=CLOSED pgs=472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:48.791 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:48.788+0000 7fae0358f700 1 --2- 172.21.15.59:0/2744239223 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fadfc0722c0 0x7fadfc132aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:48.791 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:48.788+0000 7fae0358f700 1 --2- 172.21.15.59:0/2744239223 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fadfc143ae0 0x7fadfc1f4da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:48.791 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:48.788+0000 7fae0358f700 1 -- 172.21.15.59:0/2744239223 >> 172.21.15.59:0/2744239223 conn(0x7fadfc0b2740 msgr2=0x7fadfc15a080 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:57:48.791 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:48.789+0000 7fae0358f700 1 -- 172.21.15.59:0/2744239223 shutdown_connections 2023-11-24T07:57:48.791 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:48.789+0000 7fae0358f700 1 -- 172.21.15.59:0/2744239223 wait complete. 2023-11-24T07:57:48.792 INFO:teuthology.orchestra.run.smithi059.stderr:dumped fsmap epoch 17 2023-11-24T07:57:48.803 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-24T07:57:48.927 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:48.926+0000 7f393891e700 1 Processor -- start 2023-11-24T07:57:48.928 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:48.926+0000 7f393891e700 1 -- start start 2023-11-24T07:57:48.930 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:48.927+0000 7f393891e700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f39340cfc60 0x7f39340d0110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:57:48.930 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:48.927+0000 7f393891e700 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f39340d0650 0x7f393406faa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:57:48.930 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:48.927+0000 7f393891e700 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f393406f610 con 0x7f393415c010 2023-11-24T07:57:48.930 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:48.927+0000 7f393891e700 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f39340cdac0 con 0x7f39340cfc60 2023-11-24T07:57:48.930 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:48.927+0000 7f393891e700 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f393406ece0 con 0x7f39340d0650 2023-11-24T07:57:48.930 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:48.927+0000 7f3931d9b700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f39340cfc60 0x7f39340d0110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:57:48.930 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:48.927+0000 7f393259c700 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7f393415c010 0x7f39340cd2b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:55808/0 (socket says 172.21.15.59:55808) 2023-11-24T07:57:48.930 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:48.927+0000 7f3931d9b700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f39340cfc60 0x7f39340d0110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:52336/0 (socket says 172.21.15.59:52336) 2023-11-24T07:57:48.930 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:48.927+0000 7f393259c700 1 -- 172.21.15.59:0/2298646573 learned_addr learned my addr 172.21.15.59:0/2298646573 (peer_addr_for_me v1:172.21.15.59:0/0) 2023-11-24T07:57:48.930 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:48.927+0000 7f393159a700 1 -- 172.21.15.59:0/2298646573 <== mon.0 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 94891818 0 0) 0x7f393406f610 con 0x7f393415c010 2023-11-24T07:57:48.931 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:48.927+0000 7f393159a700 1 -- 172.21.15.59:0/2298646573 --> v1:172.21.15.59:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f391c0035b0 con 0x7f393415c010 2023-11-24T07:57:48.931 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:48.927+0000 7f3932d9d700 1 --2- 172.21.15.59:0/2298646573 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f39340d0650 0x7f393406faa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:57:48.931 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:48.927+0000 7f3931d9b700 1 -- 172.21.15.59:0/2298646573 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f39340d0650 msgr2=0x7f393406faa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T07:57:48.931 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:48.927+0000 7f3931d9b700 1 --2- 172.21.15.59:0/2298646573 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f39340d0650 0x7f393406faa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:48.931 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:48.927+0000 7f3931d9b700 1 -- 172.21.15.59:0/2298646573 >> v1:172.21.15.59:6789/0 conn(0x7f393415c010 legacy=0x7f39340cd2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:57:48.931 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:48.927+0000 7f3931d9b700 1 -- 172.21.15.59:0/2298646573 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f393406efc0 con 0x7f39340cfc60 2023-11-24T07:57:48.931 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:48.927+0000 7f3932d9d700 1 --2- 172.21.15.59:0/2298646573 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f39340d0650 0x7f393406faa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T07:57:48.931 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:48.927+0000 7f3931d9b700 1 --2- 172.21.15.59:0/2298646573 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f39340cfc60 0x7f39340d0110 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f3928009840 tx=0x7f3928009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7edea85b1c854755 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:57:48.931 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:48.927+0000 7f393159a700 1 -- 172.21.15.59:0/2298646573 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3928011830 con 0x7f39340cfc60 2023-11-24T07:57:48.931 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:48.927+0000 7f393159a700 1 -- 172.21.15.59:0/2298646573 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3928011990 con 0x7f39340cfc60 2023-11-24T07:57:48.931 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:48.927+0000 7f393159a700 1 -- 172.21.15.59:0/2298646573 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3928011d20 con 0x7f39340cfc60 2023-11-24T07:57:48.931 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:48.928+0000 7f393891e700 1 -- 172.21.15.59:0/2298646573 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f39340cfc60 msgr2=0x7f39340d0110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:57:48.932 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:48.928+0000 7f393891e700 1 --2- 172.21.15.59:0/2298646573 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f39340cfc60 0x7f39340d0110 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f3928009840 tx=0x7f3928009b50 comp rx=0 tx=0).stop 2023-11-24T07:57:48.932 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:48.928+0000 7f393891e700 1 -- 172.21.15.59:0/2298646573 shutdown_connections 2023-11-24T07:57:48.932 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:48.928+0000 7f393891e700 1 --2- 172.21.15.59:0/2298646573 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f39340d0650 0x7f393406faa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:48.932 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:48.928+0000 7f393891e700 1 --2- 172.21.15.59:0/2298646573 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f39340cfc60 0x7f39340d0110 unknown :-1 s=CLOSED pgs=474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:48.932 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:48.928+0000 7f393891e700 1 -- 172.21.15.59:0/2298646573 >> 172.21.15.59:0/2298646573 conn(0x7f39340b3790 msgr2=0x7f39340b3ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:57:48.932 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:48.928+0000 7f393891e700 1 -- 172.21.15.59:0/2298646573 shutdown_connections 2023-11-24T07:57:48.932 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:48.928+0000 7f393891e700 1 -- 172.21.15.59:0/2298646573 wait complete. 2023-11-24T07:57:48.934 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:48.930+0000 7f393891e700 1 Processor -- start 2023-11-24T07:57:48.935 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:48.930+0000 7f393891e700 1 -- start start 2023-11-24T07:57:48.935 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:48.931+0000 7f393891e700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f39340cfc60 0x7f39340caf50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:57:48.935 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:48.931+0000 7f393891e700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f39340d0650 0x7f393406f9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:57:48.935 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:48.931+0000 7f393891e700 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f393415c010 0x7f393406fee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:57:48.935 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:48.931+0000 7f393891e700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f393415aa60 con 0x7f39340cfc60 2023-11-24T07:57:48.935 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:48.931+0000 7f393891e700 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39340c5730 con 0x7f39340d0650 2023-11-24T07:57:48.935 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:48.931+0000 7f393891e700 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f39340c5a10 con 0x7f393415c010 2023-11-24T07:57:48.935 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:48.931+0000 7f393259c700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f39340cfc60 0x7f39340caf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:57:48.935 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:48.931+0000 7f393259c700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f39340cfc60 0x7f39340caf50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:52340/0 (socket says 172.21.15.59:52340) 2023-11-24T07:57:48.935 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:48.931+0000 7f393259c700 1 -- 172.21.15.59:0/1917624140 learned_addr learned my addr 172.21.15.59:0/1917624140 (peer_addr_for_me v2:172.21.15.59:0/0) 2023-11-24T07:57:48.935 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:48.931+0000 7f3932d9d700 1 --2- 172.21.15.59:0/1917624140 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f393415c010 0x7f393406fee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:57:48.935 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:48.931+0000 7f393259c700 1 -- 172.21.15.59:0/1917624140 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f393415c010 msgr2=0x7f393406fee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:57:48.936 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:48.931+0000 7f3931d9b700 1 --2- 172.21.15.59:0/1917624140 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f39340d0650 0x7f393406f9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:57:48.936 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:48.931+0000 7f393259c700 1 --2- 172.21.15.59:0/1917624140 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f393415c010 0x7f393406fee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:48.936 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:48.931+0000 7f393259c700 1 -- 172.21.15.59:0/1917624140 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f39340d0650 msgr2=0x7f393406f9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:57:48.936 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:48.931+0000 7f393259c700 1 --2- 172.21.15.59:0/1917624140 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f39340d0650 0x7f393406f9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:48.936 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:48.931+0000 7f393259c700 1 -- 172.21.15.59:0/1917624140 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f39280094a0 con 0x7f39340cfc60 2023-11-24T07:57:48.936 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:48.931+0000 7f3932d9d700 1 --2- 172.21.15.59:0/1917624140 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f393415c010 0x7f393406fee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T07:57:48.936 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:48.931+0000 7f3931d9b700 1 --2- 172.21.15.59:0/1917624140 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f39340d0650 0x7f393406f9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T07:57:48.936 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:48.931+0000 7f393259c700 1 --2- 172.21.15.59:0/1917624140 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f39340cfc60 0x7f39340caf50 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f3924001eb0 tx=0x7f392400c860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:57:48.936 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:48.932+0000 7f3922ffd700 1 -- 172.21.15.59:0/1917624140 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3924005d70 con 0x7f39340cfc60 2023-11-24T07:57:48.936 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:48.932+0000 7f3922ffd700 1 -- 172.21.15.59:0/1917624140 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f39240070d0 con 0x7f39340cfc60 2023-11-24T07:57:48.936 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:48.932+0000 7f393891e700 1 -- 172.21.15.59:0/1917624140 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f39340cb850 con 0x7f39340cfc60 2023-11-24T07:57:48.936 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:48.932+0000 7f3922ffd700 1 -- 172.21.15.59:0/1917624140 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3924007460 con 0x7f39340cfc60 2023-11-24T07:57:48.937 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:48.932+0000 7f393891e700 1 -- 172.21.15.59:0/1917624140 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f39340cbe10 con 0x7f39340cfc60 2023-11-24T07:57:48.937 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:48.933+0000 7f393891e700 1 -- 172.21.15.59:0/1917624140 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3934068670 con 0x7f39340cfc60 2023-11-24T07:57:48.937 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:48.933+0000 7f3922ffd700 1 -- 172.21.15.59:0/1917624140 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 6) v1 ==== 89282+0+0 (secure 0 0 0) 0x7f39240075c0 con 0x7f39340cfc60 2023-11-24T07:57:48.937 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:48.933+0000 7f3922ffd700 1 --2- 172.21.15.59:0/1917624140 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f391406af60 0x7f391406d420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:57:48.937 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:48.934+0000 7f3922ffd700 1 -- 172.21.15.59:0/1917624140 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f3924088ac0 con 0x7f39340cfc60 2023-11-24T07:57:48.938 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:48.934+0000 7f3931d9b700 1 --2- 172.21.15.59:0/1917624140 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f391406af60 0x7f391406d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:57:48.938 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:48.934+0000 7f3931d9b700 1 --2- 172.21.15.59:0/1917624140 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f391406af60 0x7f391406d420 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f39340ca230 tx=0x7f3928008030 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:57:48.939 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:48.938+0000 7f3922ffd700 1 -- 172.21.15.59:0/1917624140 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f3924053c10 con 0x7f39340cfc60 2023-11-24T07:57:49.142 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:49.140+0000 7f393891e700 1 -- 172.21.15.59:0/1917624140 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f39340cbfa0 con 0x7f39340cfc60 2023-11-24T07:57:49.143 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:49.141+0000 7f3922ffd700 1 -- 172.21.15.59:0/1917624140 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v33) v1 ==== 74+0+16437 (secure 0 0 0) 0x7f3924017090 con 0x7f39340cfc60 2023-11-24T07:57:49.143 INFO:teuthology.orchestra.run.smithi059.stdout: 2023-11-24T07:57:49.143 INFO:teuthology.orchestra.run.smithi059.stdout:{"epoch":33,"fsid":"9f0e8741-b8f9-48d5-a2c3-09f45903303d","created":"2023-11-24T07:56:06.552729+0000","modified":"2023-11-24T07:57:26.579209+0000","last_up_change":"2023-11-24T07:56:17.372352+0000","last_in_change":"2023-11-24T07:56:09.073250+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-24T07:56:18.352823+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2023-11-24T07:57:23.014143+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"32","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2023-11-24T07:57:23.971487+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"33","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"6d8f9e9b-1782-4e41-928b-a39f2a8ebc06","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6800","nonce":3373878772},{"type":"v1","addr":"172.21.15.59:6801","nonce":3373878772}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6802","nonce":3373878772},{"type":"v1","addr":"172.21.15.59:6803","nonce":3373878772}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6806","nonce":3373878772},{"type":"v1","addr":"172.21.15.59:6807","nonce":3373878772}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6804","nonce":3373878772},{"type":"v1","addr":"172.21.15.59:6805","nonce":3373878772}]},"public_addr":"172.21.15.59:6801/3373878772","cluster_addr":"172.21.15.59:6803/3373878772","heartbeat_back_addr":"172.21.15.59:6807/3373878772","heartbeat_front_addr":"172.21.15.59:6805/3373878772","state":["exists","up"]},{"osd":1,"uuid":"e8d882aa-52e1-4c04-9e1e-c44ef9204a5f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6824","nonce":648258563},{"type":"v1","addr":"172.21.15.59:6825","nonce":648258563}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6826","nonce":648258563},{"type":"v1","addr":"172.21.15.59:6827","nonce":648258563}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6830","nonce":648258563},{"type":"v1","addr":"172.21.15.59:6831","nonce":648258563}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6828","nonce":648258563},{"type":"v1","addr":"172.21.15.59:6829","nonce":648258563}]},"public_addr":"172.21.15.59:6825/648258563","cluster_addr":"172.21.15.59:6827/648258563","heartbeat_back_addr":"172.21.15.59:6831/648258563","heartbeat_front_addr":"172.21.15.59:6829/648258563","state":["exists","up"]},{"osd":2,"uuid":"69eebfa0-695b-4b21-90b9-8a3537168a01","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6808","nonce":1840430654},{"type":"v1","addr":"172.21.15.59:6809","nonce":1840430654}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6810","nonce":1840430654},{"type":"v1","addr":"172.21.15.59:6811","nonce":1840430654}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6814","nonce":1840430654},{"type":"v1","addr":"172.21.15.59:6815","nonce":1840430654}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6812","nonce":1840430654},{"type":"v1","addr":"172.21.15.59:6813","nonce":1840430654}]},"public_addr":"172.21.15.59:6809/1840430654","cluster_addr":"172.21.15.59:6811/1840430654","heartbeat_back_addr":"172.21.15.59:6815/1840430654","heartbeat_front_addr":"172.21.15.59:6813/1840430654","state":["exists","up"]},{"osd":3,"uuid":"96fd77fc-f911-46ff-9445-6f0be2daa8f4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6816","nonce":2504025850},{"type":"v1","addr":"172.21.15.59:6817","nonce":2504025850}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6818","nonce":2504025850},{"type":"v1","addr":"172.21.15.59:6819","nonce":2504025850}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6822","nonce":2504025850},{"type":"v1","addr":"172.21.15.59:6823","nonce":2504025850}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6820","nonce":2504025850},{"type":"v1","addr":"172.21.15.59:6821","nonce":2504025850}]},"public_addr":"172.21.15.59:6817/2504025850","cluster_addr":"172.21.15.59:6819/2504025850","heartbeat_back_addr":"172.21.15.59:6823/2504025850","heartbeat_front_addr":"172.21.15.59:6821/2504025850","state":["exists","up"]},{"osd":4,"uuid":"7ac29fca-b5b2-4c95-9d69-2f164cbbc708","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6816","nonce":2382463431},{"type":"v1","addr":"172.21.15.76:6818","nonce":2382463431}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6820","nonce":2382463431},{"type":"v1","addr":"172.21.15.76:6822","nonce":2382463431}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6828","nonce":2382463431},{"type":"v1","addr":"172.21.15.76:6830","nonce":2382463431}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6824","nonce":2382463431},{"type":"v1","addr":"172.21.15.76:6826","nonce":2382463431}]},"public_addr":"172.21.15.76:6818/2382463431","cluster_addr":"172.21.15.76:6822/2382463431","heartbeat_back_addr":"172.21.15.76:6830/2382463431","heartbeat_front_addr":"172.21.15.76:6826/2382463431","state":["exists","up"]},{"osd":5,"uuid":"485bf711-3851-4238-a4f4-51d52e0a9c58","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6800","nonce":329334308},{"type":"v1","addr":"172.21.15.76:6802","nonce":329334308}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6805","nonce":329334308},{"type":"v1","addr":"172.21.15.76:6807","nonce":329334308}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6814","nonce":329334308},{"type":"v1","addr":"172.21.15.76:6815","nonce":329334308}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6810","nonce":329334308},{"type":"v1","addr":"172.21.15.76:6812","nonce":329334308}]},"public_addr":"172.21.15.76:6802/329334308","cluster_addr":"172.21.15.76:6807/329334308","heartbeat_back_addr":"172.21.15.76:6815/329334308","heartbeat_front_addr":"172.21.15.76:6812/329334308","state":["exists","up"]},{"osd":6,"uuid":"72b3f2fc-1e4a-4e32-968d-003647a0286f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6801","nonce":3946579410},{"type":"v1","addr":"172.21.15.76:6803","nonce":3946579410}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6804","nonce":3946579410},{"type":"v1","addr":"172.21.15.76:6806","nonce":3946579410}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6811","nonce":3946579410},{"type":"v1","addr":"172.21.15.76:6813","nonce":3946579410}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6808","nonce":3946579410},{"type":"v1","addr":"172.21.15.76:6809","nonce":3946579410}]},"public_addr":"172.21.15.76:6803/3946579410","cluster_addr":"172.21.15.76:6806/3946579410","heartbeat_back_addr":"172.21.15.76:6813/3946579410","heartbeat_front_addr":"172.21.15.76:6809/3946579410","state":["exists","up"]},{"osd":7,"uuid":"1ddceb90-6c5d-4d1a-ad56-99cb877f8c70","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6817","nonce":841761538},{"type":"v1","addr":"172.21.15.76:6819","nonce":841761538}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6821","nonce":841761538},{"type":"v1","addr":"172.21.15.76:6823","nonce":841761538}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6829","nonce":841761538},{"type":"v1","addr":"172.21.15.76:6831","nonce":841761538}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6825","nonce":841761538},{"type":"v1","addr":"172.21.15.76:6827","nonce":841761538}]},"public_addr":"172.21.15.76:6819/841761538","cluster_addr":"172.21.15.76:6823/841761538","heartbeat_back_addr":"172.21.15.76:6831/841761538","heartbeat_front_addr":"172.21.15.76:6827/841761538","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T07:56:14.294838+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T07:56:14.270779+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T07:56:14.231809+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T07:56:14.260983+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T07:56:14.302997+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T07:56:14.258600+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T07:56:14.251413+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T07:56:14.304727+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-24T07:57:49.146 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:49.144+0000 7f393891e700 1 -- 172.21.15.59:0/1917624140 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f391406af60 msgr2=0x7f391406d420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:57:49.146 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:49.144+0000 7f393891e700 1 --2- 172.21.15.59:0/1917624140 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f391406af60 0x7f391406d420 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f39340ca230 tx=0x7f3928008030 comp rx=0 tx=0).stop 2023-11-24T07:57:49.146 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:49.144+0000 7f393891e700 1 -- 172.21.15.59:0/1917624140 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f39340cfc60 msgr2=0x7f39340caf50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:57:49.146 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:49.144+0000 7f393891e700 1 --2- 172.21.15.59:0/1917624140 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f39340cfc60 0x7f39340caf50 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f3924001eb0 tx=0x7f392400c860 comp rx=0 tx=0).stop 2023-11-24T07:57:49.146 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:49.144+0000 7f393891e700 1 -- 172.21.15.59:0/1917624140 shutdown_connections 2023-11-24T07:57:49.146 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:49.144+0000 7f393891e700 1 --2- 172.21.15.59:0/1917624140 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f391406af60 0x7f391406d420 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:49.146 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:49.144+0000 7f393891e700 1 --2- 172.21.15.59:0/1917624140 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f393415c010 0x7f393406fee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:49.146 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:49.144+0000 7f393891e700 1 --2- 172.21.15.59:0/1917624140 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f39340cfc60 0x7f39340caf50 unknown :-1 s=CLOSED pgs=475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:49.146 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:49.144+0000 7f393891e700 1 --2- 172.21.15.59:0/1917624140 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f39340d0650 0x7f393406f9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:49.147 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:49.144+0000 7f393891e700 1 -- 172.21.15.59:0/1917624140 >> 172.21.15.59:0/1917624140 conn(0x7f39340b3790 msgr2=0x7f39340b2890 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:57:49.147 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:49.144+0000 7f393891e700 1 -- 172.21.15.59:0/1917624140 shutdown_connections 2023-11-24T07:57:49.147 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:49.144+0000 7f393891e700 1 -- 172.21.15.59:0/1917624140 wait complete. 2023-11-24T07:57:49.158 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2023-11-24T07:57:49.284 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:49.282+0000 7f251bd5f700 1 Processor -- start 2023-11-24T07:57:49.284 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:49.282+0000 7f251bd5f700 1 -- start start 2023-11-24T07:57:49.286 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:49.283+0000 7f251bd5f700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f25140d0510 0x7f251415f2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:57:49.286 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:49.283+0000 7f251bd5f700 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f25140d2b60 0x7f25140ceb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:57:49.286 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:49.283+0000 7f251bd5f700 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f25140d4ce0 con 0x7f251415e3b0 2023-11-24T07:57:49.286 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:49.283+0000 7f251bd5f700 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f25140d1630 con 0x7f25140d0510 2023-11-24T07:57:49.286 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:49.283+0000 7f251bd5f700 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f251415f810 con 0x7f25140d2b60 2023-11-24T07:57:49.286 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:49.283+0000 7f25192fa700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f25140d0510 0x7f251415f2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:57:49.286 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:49.283+0000 7f25192fa700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f25140d0510 0x7f251415f2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:52350/0 (socket says 172.21.15.59:52350) 2023-11-24T07:57:49.286 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:49.284+0000 7f25192fa700 1 -- 172.21.15.59:0/121879626 learned_addr learned my addr 172.21.15.59:0/121879626 (peer_addr_for_me v2:172.21.15.59:0/0) 2023-11-24T07:57:49.287 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:49.284+0000 7f251a2fc700 1 --2- 172.21.15.59:0/121879626 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f25140d2b60 0x7f25140ceb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:57:49.287 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:49.284+0000 7f25192fa700 1 -- 172.21.15.59:0/121879626 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f25140d2b60 msgr2=0x7f25140ceb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T07:57:49.287 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:49.284+0000 7f25192fa700 1 --2- 172.21.15.59:0/121879626 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f25140d2b60 0x7f25140ceb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:49.287 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:49.284+0000 7f25192fa700 1 -- 172.21.15.59:0/121879626 >> v1:172.21.15.76:6789/0 conn(0x7f251415e3b0 legacy=0x7f25140d2450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:57:49.287 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:49.284+0000 7f2518af9700 1 -- 172.21.15.59:0/121879626 <== mon.1 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3925714384 0 0) 0x7f25140d4ce0 con 0x7f251415e3b0 2023-11-24T07:57:49.287 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:49.284+0000 7f25192fa700 1 -- 172.21.15.59:0/121879626 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f25140d09b0 con 0x7f25140d0510 2023-11-24T07:57:49.287 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:49.284+0000 7f25192fa700 1 --2- 172.21.15.59:0/121879626 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f25140d0510 0x7f251415f2d0 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f25100098f0 tx=0x7f2510009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8540d70bb2cb600d server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:57:49.287 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:49.284+0000 7f2518af9700 1 -- 172.21.15.59:0/121879626 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2510011830 con 0x7f25140d0510 2023-11-24T07:57:49.287 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:49.284+0000 7f2518af9700 1 -- 172.21.15.59:0/121879626 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2510011990 con 0x7f25140d0510 2023-11-24T07:57:49.287 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:49.284+0000 7f2518af9700 1 -- 172.21.15.59:0/121879626 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2510011d20 con 0x7f25140d0510 2023-11-24T07:57:49.290 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:49.285+0000 7f251bd5f700 1 -- 172.21.15.59:0/121879626 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f25140d0510 msgr2=0x7f251415f2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:57:49.290 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:49.285+0000 7f251bd5f700 1 --2- 172.21.15.59:0/121879626 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f25140d0510 0x7f251415f2d0 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f25100098f0 tx=0x7f2510009c00 comp rx=0 tx=0).stop 2023-11-24T07:57:49.291 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:49.285+0000 7f251bd5f700 1 -- 172.21.15.59:0/121879626 shutdown_connections 2023-11-24T07:57:49.291 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:49.285+0000 7f251bd5f700 1 --2- 172.21.15.59:0/121879626 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f25140d2b60 0x7f25140ceb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:49.291 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:49.285+0000 7f251bd5f700 1 --2- 172.21.15.59:0/121879626 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f25140d0510 0x7f251415f2d0 unknown :-1 s=CLOSED pgs=476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:49.291 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:49.285+0000 7f251bd5f700 1 -- 172.21.15.59:0/121879626 >> 172.21.15.59:0/121879626 conn(0x7f25140d3e70 msgr2=0x7f25140d4280 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:57:49.291 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:49.285+0000 7f251bd5f700 1 -- 172.21.15.59:0/121879626 shutdown_connections 2023-11-24T07:57:49.291 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:49.285+0000 7f251bd5f700 1 -- 172.21.15.59:0/121879626 wait complete. 2023-11-24T07:57:49.291 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:49.286+0000 7f251bd5f700 1 Processor -- start 2023-11-24T07:57:49.291 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:49.286+0000 7f251bd5f700 1 -- start start 2023-11-24T07:57:49.291 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:49.287+0000 7f251bd5f700 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f25140d0510 0x7f251415b8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:57:49.291 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:49.287+0000 7f251bd5f700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f25140d2b60 0x7f251415a5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:57:49.291 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:49.287+0000 7f251bd5f700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f251415e3b0 0x7f251406c690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:57:49.292 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:49.287+0000 7f251bd5f700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25140b6a00 con 0x7f25140d2b60 2023-11-24T07:57:49.292 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:49.287+0000 7f251bd5f700 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f251415bff0 con 0x7f251415e3b0 2023-11-24T07:57:49.292 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:49.287+0000 7f251bd5f700 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f251415c2d0 con 0x7f25140d0510 2023-11-24T07:57:49.292 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:49.287+0000 7f25192fa700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f25140d2b60 0x7f251415a5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:57:49.292 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:49.287+0000 7f25192fa700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f25140d2b60 0x7f251415a5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:52356/0 (socket says 172.21.15.59:52356) 2023-11-24T07:57:49.292 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:49.287+0000 7f25192fa700 1 -- 172.21.15.59:0/1061427377 learned_addr learned my addr 172.21.15.59:0/1061427377 (peer_addr_for_me v2:172.21.15.59:0/0) 2023-11-24T07:57:49.292 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:49.287+0000 7f251a2fc700 1 --2- 172.21.15.59:0/1061427377 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f251415e3b0 0x7f251406c690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:57:49.292 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:49.287+0000 7f2519afb700 1 --2- 172.21.15.59:0/1061427377 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f25140d0510 0x7f251415b8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:57:49.292 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:49.287+0000 7f25192fa700 1 -- 172.21.15.59:0/1061427377 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f25140d0510 msgr2=0x7f251415b8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:57:49.292 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:49.287+0000 7f25192fa700 1 --2- 172.21.15.59:0/1061427377 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f25140d0510 0x7f251415b8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:49.292 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:49.287+0000 7f25192fa700 1 -- 172.21.15.59:0/1061427377 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f251415e3b0 msgr2=0x7f251406c690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:57:49.292 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:49.287+0000 7f25192fa700 1 --2- 172.21.15.59:0/1061427377 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f251415e3b0 0x7f251406c690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:49.293 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:49.287+0000 7f25192fa700 1 -- 172.21.15.59:0/1061427377 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2510009550 con 0x7f25140d2b60 2023-11-24T07:57:49.293 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:49.287+0000 7f251a2fc700 1 --2- 172.21.15.59:0/1061427377 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f251415e3b0 0x7f251406c690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-24T07:57:49.293 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:49.287+0000 7f25192fa700 1 --2- 172.21.15.59:0/1061427377 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f25140d2b60 0x7f251415a5c0 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7f25140ce470 tx=0x7f2510004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:57:49.293 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:49.288+0000 7f250a7fc700 1 -- 172.21.15.59:0/1061427377 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2510004480 con 0x7f25140d2b60 2023-11-24T07:57:49.293 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:49.288+0000 7f250a7fc700 1 -- 172.21.15.59:0/1061427377 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f251001f070 con 0x7f25140d2b60 2023-11-24T07:57:49.293 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:49.288+0000 7f251bd5f700 1 -- 172.21.15.59:0/1061427377 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f251415c5b0 con 0x7f25140d2b60 2023-11-24T07:57:49.293 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:49.288+0000 7f250a7fc700 1 -- 172.21.15.59:0/1061427377 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25100038a0 con 0x7f25140d2b60 2023-11-24T07:57:49.293 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:49.288+0000 7f251bd5f700 1 -- 172.21.15.59:0/1061427377 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f251406cbd0 con 0x7f25140d2b60 2023-11-24T07:57:49.293 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:49.289+0000 7f251bd5f700 1 -- 172.21.15.59:0/1061427377 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f25140d2980 con 0x7f25140d2b60 2023-11-24T07:57:49.293 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:49.289+0000 7f250a7fc700 1 -- 172.21.15.59:0/1061427377 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 6) v1 ==== 89282+0+0 (secure 0 0 0) 0x7f2510021030 con 0x7f25140d2b60 2023-11-24T07:57:49.294 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:49.289+0000 7f250a7fc700 1 --2- 172.21.15.59:0/1061427377 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f24f806af60 0x7f24f806d420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:57:49.294 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:49.290+0000 7f250a7fc700 1 -- 172.21.15.59:0/1061427377 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f251008aa90 con 0x7f25140d2b60 2023-11-24T07:57:49.294 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:49.290+0000 7f2519afb700 1 --2- 172.21.15.59:0/1061427377 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f24f806af60 0x7f24f806d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:57:49.294 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:49.290+0000 7f2519afb700 1 --2- 172.21.15.59:0/1061427377 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f24f806af60 0x7f24f806d420 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f2504000f60 tx=0x7f2504009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:57:49.295 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:49.294+0000 7f250a7fc700 1 -- 172.21.15.59:0/1061427377 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f2510055d00 con 0x7f25140d2b60 2023-11-24T07:57:49.543 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:49.541+0000 7f251bd5f700 1 -- 172.21.15.59:0/1061427377 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f251406cd60 con 0x7f25140d2b60 2023-11-24T07:57:49.544 INFO:tasks.ceph.mon.a.smithi059.stderr:2023-11-24T07:57:49.542+0000 7f5560821700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-11-24T07:57:49.547 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:49.545+0000 7f250a7fc700 1 -- 172.21.15.59:0/1061427377 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v18) v1 ==== 161+0+0 (secure 0 0 0) 0x7f2510015090 con 0x7f25140d2b60 2023-11-24T07:57:49.549 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:49.548+0000 7f251bd5f700 1 -- 172.21.15.59:0/1061427377 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f24f806af60 msgr2=0x7f24f806d420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:57:49.549 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:49.548+0000 7f251bd5f700 1 --2- 172.21.15.59:0/1061427377 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f24f806af60 0x7f24f806d420 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f2504000f60 tx=0x7f2504009040 comp rx=0 tx=0).stop 2023-11-24T07:57:49.550 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:49.548+0000 7f251bd5f700 1 -- 172.21.15.59:0/1061427377 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f25140d2b60 msgr2=0x7f251415a5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:57:49.550 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:49.548+0000 7f251bd5f700 1 --2- 172.21.15.59:0/1061427377 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f25140d2b60 0x7f251415a5c0 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7f25140ce470 tx=0x7f2510004750 comp rx=0 tx=0).stop 2023-11-24T07:57:49.550 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:49.548+0000 7f251bd5f700 1 -- 172.21.15.59:0/1061427377 shutdown_connections 2023-11-24T07:57:49.550 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:49.548+0000 7f251bd5f700 1 --2- 172.21.15.59:0/1061427377 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f24f806af60 0x7f24f806d420 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:49.550 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:49.548+0000 7f251bd5f700 1 --2- 172.21.15.59:0/1061427377 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f251415e3b0 0x7f251406c690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:49.550 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:49.548+0000 7f251bd5f700 1 --2- 172.21.15.59:0/1061427377 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f25140d0510 0x7f251415b8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:49.550 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:49.548+0000 7f251bd5f700 1 --2- 172.21.15.59:0/1061427377 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f25140d2b60 0x7f251415a5c0 unknown :-1 s=CLOSED pgs=477 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:49.550 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:49.548+0000 7f251bd5f700 1 -- 172.21.15.59:0/1061427377 >> 172.21.15.59:0/1061427377 conn(0x7f25140d3e70 msgr2=0x7f25140b0aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:57:49.551 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:49.550+0000 7f251bd5f700 1 -- 172.21.15.59:0/1061427377 shutdown_connections 2023-11-24T07:57:49.551 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:49.550+0000 7f251bd5f700 1 -- 172.21.15.59:0/1061427377 wait complete. 2023-11-24T07:57:49.552 INFO:teuthology.orchestra.run.smithi059.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2023-11-24T07:57:49.553 INFO:tasks.ceph.mds.a.smithi059.stderr: -5417> 2023-11-24T07:57:29.475+0000 7faa67f73700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-11-24T07:57:49.565 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2023-11-24T07:57:49.586 INFO:tasks.ceph.mds.a.smithi059.stdout:starting mds.a at 2023-11-24T07:57:49.695 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:49.693+0000 7f02a3957700 1 Processor -- start 2023-11-24T07:57:49.695 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:49.694+0000 7f02a3957700 1 -- start start 2023-11-24T07:57:49.698 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:49.694+0000 7f02a3957700 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f029c143ae0 0x7f029c0b51b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:57:49.698 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:49.694+0000 7f02a3957700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f029c0722c0 0x7f029c06e510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:57:49.698 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:49.694+0000 7f02a3957700 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f029c143ec0 con 0x7f029c1440f0 2023-11-24T07:57:49.698 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:49.694+0000 7f02a3957700 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f029c0b40a0 con 0x7f029c0722c0 2023-11-24T07:57:49.698 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:49.694+0000 7f02a3957700 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f029c0b3ae0 con 0x7f029c143ae0 2023-11-24T07:57:49.699 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:49.695+0000 7f02a1ef4700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f029c0722c0 0x7f029c06e510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:57:49.699 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:49.695+0000 7f02a0ef2700 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7f029c1440f0 0x7f029c071bb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:55856/0 (socket says 172.21.15.59:55856) 2023-11-24T07:57:49.699 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:49.695+0000 7f02a0ef2700 1 -- 172.21.15.59:0/1159360743 learned_addr learned my addr 172.21.15.59:0/1159360743 (peer_addr_for_me v1:172.21.15.59:0/0) 2023-11-24T07:57:49.699 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:49.695+0000 7f02a16f3700 1 --2- 172.21.15.59:0/1159360743 >> v2:172.21.15.76:3300/0 conn(0x7f029c143ae0 0x7f029c0b51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:57:49.699 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:49.695+0000 7f02a1ef4700 1 -- 172.21.15.59:0/1159360743 >> v2:172.21.15.76:3300/0 conn(0x7f029c143ae0 msgr2=0x7f029c0b51b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T07:57:49.699 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:49.695+0000 7f02a1ef4700 1 --2- 172.21.15.59:0/1159360743 >> v2:172.21.15.76:3300/0 conn(0x7f029c143ae0 0x7f029c0b51b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:49.700 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:49.695+0000 7f02a1ef4700 1 -- 172.21.15.59:0/1159360743 >> v1:172.21.15.59:6789/0 conn(0x7f029c1440f0 legacy=0x7f029c071bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:57:49.700 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:49.695+0000 7f02a1ef4700 1 -- 172.21.15.59:0/1159360743 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f029c0b3dc0 con 0x7f029c0722c0 2023-11-24T07:57:49.700 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:49.695+0000 7f02a1ef4700 1 --2- 172.21.15.59:0/1159360743 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f029c0722c0 0x7f029c06e510 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f029c06d2e0 tx=0x7f029400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a71893c860bcc866 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:57:49.700 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:49.695+0000 7f0293fff700 1 -- 172.21.15.59:0/1159360743 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f029400e070 con 0x7f029c0722c0 2023-11-24T07:57:49.700 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:49.695+0000 7f0293fff700 1 -- 172.21.15.59:0/1159360743 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f029400bbd0 con 0x7f029c0722c0 2023-11-24T07:57:49.700 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:49.695+0000 7f0293fff700 1 -- 172.21.15.59:0/1159360743 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0294012430 con 0x7f029c0722c0 2023-11-24T07:57:49.700 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:49.696+0000 7f02a3957700 1 -- 172.21.15.59:0/1159360743 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f029c0722c0 msgr2=0x7f029c06e510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:57:49.701 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:49.696+0000 7f02a3957700 1 --2- 172.21.15.59:0/1159360743 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f029c0722c0 0x7f029c06e510 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f029c06d2e0 tx=0x7f029400b330 comp rx=0 tx=0).stop 2023-11-24T07:57:49.701 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:49.696+0000 7f02a3957700 1 -- 172.21.15.59:0/1159360743 shutdown_connections 2023-11-24T07:57:49.701 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:49.696+0000 7f02a3957700 1 --2- 172.21.15.59:0/1159360743 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f029c0722c0 0x7f029c06e510 unknown :-1 s=CLOSED pgs=482 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:49.701 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:49.696+0000 7f02a3957700 1 --2- 172.21.15.59:0/1159360743 >> v2:172.21.15.76:3300/0 conn(0x7f029c143ae0 0x7f029c0b51b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:49.701 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:49.696+0000 7f02a3957700 1 -- 172.21.15.59:0/1159360743 >> 172.21.15.59:0/1159360743 conn(0x7f029c0b2740 msgr2=0x7f029c0b2b20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:57:49.701 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:49.696+0000 7f02a3957700 1 -- 172.21.15.59:0/1159360743 shutdown_connections 2023-11-24T07:57:49.702 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:49.696+0000 7f02a3957700 1 -- 172.21.15.59:0/1159360743 wait complete. 2023-11-24T07:57:49.704 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:49.697+0000 7f02a3957700 1 Processor -- start 2023-11-24T07:57:49.704 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:49.697+0000 7f02a3957700 1 -- start start 2023-11-24T07:57:49.704 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:49.698+0000 7f02a3957700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f029c0722c0 0x7f029c1e5330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:57:49.705 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:49.698+0000 7f02a3957700 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f029c143ae0 0x7f029c1543c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:57:49.705 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:49.698+0000 7f02a3957700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f029c1440f0 0x7f029c0cf1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:57:49.705 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:49.698+0000 7f02a3957700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f029c1e4f10 con 0x7f029c0722c0 2023-11-24T07:57:49.705 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:49.698+0000 7f02a3957700 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f029c0d0040 con 0x7f029c1440f0 2023-11-24T07:57:49.705 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:49.698+0000 7f02a3957700 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f029c1e5870 con 0x7f029c143ae0 2023-11-24T07:57:49.705 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:49.698+0000 7f02a16f3700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f029c0722c0 0x7f029c1e5330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:57:49.705 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:49.698+0000 7f02a16f3700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f029c0722c0 0x7f029c1e5330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:52386/0 (socket says 172.21.15.59:52386) 2023-11-24T07:57:49.706 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:49.698+0000 7f02a16f3700 1 -- 172.21.15.59:0/378630579 learned_addr learned my addr 172.21.15.59:0/378630579 (peer_addr_for_me v2:172.21.15.59:0/0) 2023-11-24T07:57:49.706 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:49.698+0000 7f02a1ef4700 1 --2- 172.21.15.59:0/378630579 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f029c1440f0 0x7f029c0cf1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:57:49.706 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:49.698+0000 7f02a16f3700 1 -- 172.21.15.59:0/378630579 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f029c143ae0 msgr2=0x7f029c1543c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:57:49.706 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:49.698+0000 7f02a0ef2700 1 --2- 172.21.15.59:0/378630579 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f029c143ae0 0x7f029c1543c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:57:49.706 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:49.698+0000 7f02a16f3700 1 --2- 172.21.15.59:0/378630579 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f029c143ae0 0x7f029c1543c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:49.707 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:49.698+0000 7f02a16f3700 1 -- 172.21.15.59:0/378630579 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f029c1440f0 msgr2=0x7f029c0cf1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:57:49.707 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:49.698+0000 7f02a16f3700 1 --2- 172.21.15.59:0/378630579 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f029c1440f0 0x7f029c0cf1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:49.707 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:49.698+0000 7f02a16f3700 1 -- 172.21.15.59:0/378630579 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0294009a70 con 0x7f029c0722c0 2023-11-24T07:57:49.707 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:49.698+0000 7f02a1ef4700 1 --2- 172.21.15.59:0/378630579 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f029c1440f0 0x7f029c0cf1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T07:57:49.707 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:49.699+0000 7f02a16f3700 1 --2- 172.21.15.59:0/378630579 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f029c0722c0 0x7f029c1e5330 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7f028c00e810 tx=0x7f028c00ebd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:57:49.707 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:49.699+0000 7f0291ffb700 1 -- 172.21.15.59:0/378630579 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f028c013610 con 0x7f029c0722c0 2023-11-24T07:57:49.708 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:49.699+0000 7f0291ffb700 1 -- 172.21.15.59:0/378630579 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f028c013770 con 0x7f029c0722c0 2023-11-24T07:57:49.708 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:49.699+0000 7f02a3957700 1 -- 172.21.15.59:0/378630579 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f029c154900 con 0x7f029c0722c0 2023-11-24T07:57:49.708 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:49.699+0000 7f0291ffb700 1 -- 172.21.15.59:0/378630579 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f028c013b20 con 0x7f029c0722c0 2023-11-24T07:57:49.708 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:49.699+0000 7f02a3957700 1 -- 172.21.15.59:0/378630579 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f029c1e4850 con 0x7f029c0722c0 2023-11-24T07:57:49.708 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:49.700+0000 7f02a3957700 1 -- 172.21.15.59:0/378630579 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f029c1e46b0 con 0x7f029c0722c0 2023-11-24T07:57:49.708 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:49.700+0000 7f0291ffb700 1 -- 172.21.15.59:0/378630579 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 6) v1 ==== 89282+0+0 (secure 0 0 0) 0x7f028c013c80 con 0x7f029c0722c0 2023-11-24T07:57:49.709 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:49.701+0000 7f0291ffb700 1 --2- 172.21.15.59:0/378630579 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f028006af60 0x7f028006d420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:57:49.709 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:49.701+0000 7f0291ffb700 1 -- 172.21.15.59:0/378630579 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 5795+0+0 (secure 0 0 0) 0x7f028c089420 con 0x7f029c0722c0 2023-11-24T07:57:49.709 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:49.701+0000 7f02a0ef2700 1 --2- 172.21.15.59:0/378630579 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f028006af60 0x7f028006d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:57:49.709 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:49.701+0000 7f02a0ef2700 1 --2- 172.21.15.59:0/378630579 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f028006af60 0x7f028006d420 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f029c068be0 tx=0x7f0298009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:57:49.710 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:49.705+0000 7f0291ffb700 1 -- 172.21.15.59:0/378630579 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f029c1e46b0 con 0x7f029c0722c0 2023-11-24T07:57:49.957 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:49.956+0000 7f02a3957700 1 -- 172.21.15.59:0/378630579 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f029c1e46b0 con 0x7f029c0722c0 2023-11-24T07:57:50.554 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:50.552+0000 7f0291ffb700 1 -- 172.21.15.59:0/378630579 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v19) v1 ==== 104+0+0 (secure 0 0 0) 0x7f029c1e46b0 con 0x7f029c0722c0 2023-11-24T07:57:50.557 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:50.555+0000 7f02a3957700 1 -- 172.21.15.59:0/378630579 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f028006af60 msgr2=0x7f028006d420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:57:50.557 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:50.555+0000 7f02a3957700 1 --2- 172.21.15.59:0/378630579 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f028006af60 0x7f028006d420 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f029c068be0 tx=0x7f0298009040 comp rx=0 tx=0).stop 2023-11-24T07:57:50.557 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:50.555+0000 7f02a3957700 1 -- 172.21.15.59:0/378630579 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f029c0722c0 msgr2=0x7f029c1e5330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:57:50.557 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:50.555+0000 7f02a3957700 1 --2- 172.21.15.59:0/378630579 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f029c0722c0 0x7f029c1e5330 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7f028c00e810 tx=0x7f028c00ebd0 comp rx=0 tx=0).stop 2023-11-24T07:57:50.557 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:50.555+0000 7f02a3957700 1 -- 172.21.15.59:0/378630579 shutdown_connections 2023-11-24T07:57:50.557 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:50.555+0000 7f02a3957700 1 --2- 172.21.15.59:0/378630579 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f028006af60 0x7f028006d420 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:50.557 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:50.555+0000 7f02a3957700 1 --2- 172.21.15.59:0/378630579 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f029c1440f0 0x7f029c0cf1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:50.557 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:50.555+0000 7f02a3957700 1 --2- 172.21.15.59:0/378630579 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f029c0722c0 0x7f029c1e5330 unknown :-1 s=CLOSED pgs=483 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:50.558 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:50.555+0000 7f02a3957700 1 --2- 172.21.15.59:0/378630579 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f029c143ae0 0x7f029c1543c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:50.558 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:50.555+0000 7f02a3957700 1 -- 172.21.15.59:0/378630579 >> 172.21.15.59:0/378630579 conn(0x7f029c0b2740 msgr2=0x7f029c142980 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:57:50.558 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:50.555+0000 7f02a3957700 1 -- 172.21.15.59:0/378630579 shutdown_connections 2023-11-24T07:57:50.558 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:50.555+0000 7f02a3957700 1 -- 172.21.15.59:0/378630579 wait complete. 2023-11-24T07:57:50.573 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2023-11-24T07:57:50.749 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:50.748+0000 7f944c008700 1 Processor -- start 2023-11-24T07:57:50.749 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:50.748+0000 7f944c008700 1 -- start start 2023-11-24T07:57:50.753 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:50.749+0000 7f944c008700 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f9444145320 0x7f94440b5e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:57:50.753 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:50.749+0000 7f944c008700 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f94440671d0 con 0x7f944415f580 2023-11-24T07:57:50.753 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:50.749+0000 7f944c008700 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9444160820 con 0x7f94440d0790 2023-11-24T07:57:50.753 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:50.749+0000 7f944c008700 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f944406afd0 con 0x7f9444145320 2023-11-24T07:57:50.753 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:50.749+0000 7f9449da4700 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7f944415f580 0x7f94440d0080 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:55870/0 (socket says 172.21.15.59:55870) 2023-11-24T07:57:50.753 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:50.749+0000 7f9449da4700 1 -- 172.21.15.59:0/1918431236 learned_addr learned my addr 172.21.15.59:0/1918431236 (peer_addr_for_me v1:172.21.15.59:0/0) 2023-11-24T07:57:50.753 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:50.749+0000 7f944a5a5700 1 --2- 172.21.15.59:0/1918431236 >> v2:172.21.15.76:3300/0 conn(0x7f9444145320 0x7f94440b5e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:57:50.754 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:50.749+0000 7f9448da2700 1 -- 172.21.15.59:0/1918431236 <== mon.0 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3415836638 0 0) 0x7f94440671d0 con 0x7f944415f580 2023-11-24T07:57:50.754 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:50.749+0000 7f9448da2700 1 -- 172.21.15.59:0/1918431236 --> v1:172.21.15.59:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f94300035b0 con 0x7f944415f580 2023-11-24T07:57:50.754 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:50.749+0000 7f9448da2700 1 -- 172.21.15.59:0/1918431236 <== mon.0 v1:172.21.15.59:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1863521148 0 0) 0x7f94300035b0 con 0x7f944415f580 2023-11-24T07:57:50.754 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:50.749+0000 7f9448da2700 1 -- 172.21.15.59:0/1918431236 >> v2:172.21.15.76:3300/0 conn(0x7f9444145320 msgr2=0x7f94440b5e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T07:57:50.754 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:50.749+0000 7f9448da2700 1 --2- 172.21.15.59:0/1918431236 >> v2:172.21.15.76:3300/0 conn(0x7f9444145320 0x7f94440b5e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:50.754 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:50.749+0000 7f9448da2700 1 -- 172.21.15.59:0/1918431236 >> v1:172.21.15.76:6789/0 conn(0x7f94440d0790 legacy=0x7f94440aeb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:57:50.754 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:50.749+0000 7f9448da2700 1 -- 172.21.15.59:0/1918431236 --> v1:172.21.15.59:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9444066e60 con 0x7f944415f580 2023-11-24T07:57:50.754 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:50.749+0000 7f9448da2700 1 -- 172.21.15.59:0/1918431236 <== mon.0 v1:172.21.15.59:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1312333566 0 0) 0x7f9438001670 con 0x7f944415f580 2023-11-24T07:57:50.754 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:50.749+0000 7f9448da2700 1 -- 172.21.15.59:0/1918431236 >> v1:172.21.15.59:6789/0 conn(0x7f944415f580 legacy=0x7f94440d0080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:57:50.754 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:50.749+0000 7f9448da2700 1 --2- 172.21.15.59:0/1918431236 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f9430003a00 0x7f9430003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:57:50.754 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:50.749+0000 7f9448da2700 1 --2- 172.21.15.59:0/1918431236 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9430004400 0x7f94300048d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:57:50.754 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:50.749+0000 7f9448da2700 1 --2- 172.21.15.59:0/1918431236 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9430005170 0x7f943000ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:57:50.754 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:50.749+0000 7f9448da2700 1 -- 172.21.15.59:0/1918431236 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f943000c080 con 0x7f9430004400 2023-11-24T07:57:50.755 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:50.749+0000 7f9448da2700 1 -- 172.21.15.59:0/1918431236 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f943000c1f0 con 0x7f9430005170 2023-11-24T07:57:50.755 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:50.749+0000 7f9448da2700 1 -- 172.21.15.59:0/1918431236 --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f943000c3b0 con 0x7f9430003a00 2023-11-24T07:57:50.755 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:50.750+0000 7f94495a3700 1 --2- 172.21.15.59:0/1918431236 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9430004400 0x7f94300048d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:57:50.755 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:50.750+0000 7f9449da4700 1 --2- 172.21.15.59:0/1918431236 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f9430003a00 0x7f9430003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:57:50.755 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:50.750+0000 7f944a5a5700 1 --2- 172.21.15.59:0/1918431236 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9430005170 0x7f943000ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:57:50.755 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:50.750+0000 7f94495a3700 1 -- 172.21.15.59:0/1918431236 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f9430003a00 msgr2=0x7f9430003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T07:57:50.755 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:50.750+0000 7f94495a3700 1 --2- 172.21.15.59:0/1918431236 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f9430003a00 0x7f9430003de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:50.755 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:50.750+0000 7f94495a3700 1 -- 172.21.15.59:0/1918431236 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9430005170 msgr2=0x7f943000ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T07:57:50.755 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:50.750+0000 7f94495a3700 1 --2- 172.21.15.59:0/1918431236 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9430005170 0x7f943000ba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:50.755 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:50.750+0000 7f94495a3700 1 -- 172.21.15.59:0/1918431236 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f94300035b0 con 0x7f9430004400 2023-11-24T07:57:50.755 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:50.750+0000 7f944a5a5700 1 -- 172.21.15.59:0/1918431236 reap_dead start 2023-11-24T07:57:50.755 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:50.750+0000 7f94495a3700 1 --2- 172.21.15.59:0/1918431236 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9430004400 0x7f94300048d0 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7f9444073720 tx=0x7f9430005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8d3a80b3c2c76acb server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:57:50.755 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:50.750+0000 7f9448da2700 1 -- 172.21.15.59:0/1918431236 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94400052d0 con 0x7f9430004400 2023-11-24T07:57:50.756 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:50.750+0000 7f9448da2700 1 -- 172.21.15.59:0/1918431236 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9440005430 con 0x7f9430004400 2023-11-24T07:57:50.756 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:50.750+0000 7f9448da2700 1 -- 172.21.15.59:0/1918431236 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94400038a0 con 0x7f9430004400 2023-11-24T07:57:50.756 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:50.751+0000 7f944c008700 1 -- 172.21.15.59:0/1918431236 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9430004400 msgr2=0x7f94300048d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:57:50.756 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:50.751+0000 7f944c008700 1 --2- 172.21.15.59:0/1918431236 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9430004400 0x7f94300048d0 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7f9444073720 tx=0x7f9430005880 comp rx=0 tx=0).stop 2023-11-24T07:57:50.756 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:50.751+0000 7f944c008700 1 -- 172.21.15.59:0/1918431236 shutdown_connections 2023-11-24T07:57:50.756 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:50.751+0000 7f944c008700 1 --2- 172.21.15.59:0/1918431236 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9430004400 0x7f94300048d0 unknown :-1 s=CLOSED pgs=485 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:50.756 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:50.751+0000 7f944c008700 1 -- 172.21.15.59:0/1918431236 >> 172.21.15.59:0/1918431236 conn(0x7f94440b5070 msgr2=0x7f94440b5730 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:57:50.756 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:50.751+0000 7f944c008700 1 -- 172.21.15.59:0/1918431236 shutdown_connections 2023-11-24T07:57:50.756 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:50.751+0000 7f944c008700 1 -- 172.21.15.59:0/1918431236 wait complete. 2023-11-24T07:57:50.759 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:50.752+0000 7f944c008700 1 Processor -- start 2023-11-24T07:57:50.759 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:50.752+0000 7f944c008700 1 -- start start 2023-11-24T07:57:50.759 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:50.753+0000 7f944c008700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9430004400 0x7f94440cc5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:57:50.759 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:50.753+0000 7f944c008700 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f94441f7660 0x7f944415db80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:57:50.759 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:50.753+0000 7f944c008700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f94441f2b10 0x7f94441f2fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:57:50.759 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:50.753+0000 7f944c008700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94441f7b40 con 0x7f94441f2b10 2023-11-24T07:57:50.759 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:50.753+0000 7f944c008700 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94440cd090 con 0x7f9430004400 2023-11-24T07:57:50.759 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:50.753+0000 7f944c008700 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f94440cd370 con 0x7f94441f7660 2023-11-24T07:57:50.759 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:50.753+0000 7f944a5a5700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f94441f2b10 0x7f94441f2fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:57:50.759 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:50.753+0000 7f944a5a5700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f94441f2b10 0x7f94441f2fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:52412/0 (socket says 172.21.15.59:52412) 2023-11-24T07:57:50.760 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:50.753+0000 7f944a5a5700 1 -- 172.21.15.59:0/1975641721 learned_addr learned my addr 172.21.15.59:0/1975641721 (peer_addr_for_me v2:172.21.15.59:0/0) 2023-11-24T07:57:50.760 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:50.753+0000 7f9449da4700 1 --2- 172.21.15.59:0/1975641721 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9430004400 0x7f94440cc5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:57:50.760 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:50.753+0000 7f94495a3700 1 --2- 172.21.15.59:0/1975641721 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f94441f7660 0x7f944415db80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:57:50.760 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:50.753+0000 7f944a5a5700 1 -- 172.21.15.59:0/1975641721 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f94441f7660 msgr2=0x7f944415db80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:57:50.760 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:50.753+0000 7f944a5a5700 1 --2- 172.21.15.59:0/1975641721 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f94441f7660 0x7f944415db80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:50.760 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:50.753+0000 7f944a5a5700 1 -- 172.21.15.59:0/1975641721 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9430004400 msgr2=0x7f94440cc5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:57:50.760 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:50.753+0000 7f944a5a5700 1 --2- 172.21.15.59:0/1975641721 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9430004400 0x7f94440cc5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:50.760 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:50.753+0000 7f944a5a5700 1 -- 172.21.15.59:0/1975641721 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9444145f70 con 0x7f94441f2b10 2023-11-24T07:57:50.760 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:50.753+0000 7f94495a3700 1 --2- 172.21.15.59:0/1975641721 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f94441f7660 0x7f944415db80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-24T07:57:50.760 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:50.753+0000 7f9449da4700 1 --2- 172.21.15.59:0/1975641721 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9430004400 0x7f94440cc5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-24T07:57:50.760 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:50.753+0000 7f944a5a5700 1 --2- 172.21.15.59:0/1975641721 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f94441f2b10 0x7f94441f2fa0 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7f944406fc40 tx=0x7f943c011f80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:57:50.760 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:50.754+0000 7f94367fc700 1 -- 172.21.15.59:0/1975641721 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f943c00e040 con 0x7f94441f2b10 2023-11-24T07:57:50.760 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:50.754+0000 7f94367fc700 1 -- 172.21.15.59:0/1975641721 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f943c00f040 con 0x7f94441f2b10 2023-11-24T07:57:50.761 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:50.754+0000 7f944c008700 1 -- 172.21.15.59:0/1975641721 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f94441f34e0 con 0x7f94441f2b10 2023-11-24T07:57:50.761 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:50.754+0000 7f94367fc700 1 -- 172.21.15.59:0/1975641721 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f943c00fdb0 con 0x7f94441f2b10 2023-11-24T07:57:50.761 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:50.754+0000 7f944c008700 1 -- 172.21.15.59:0/1975641721 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f94440cd650 con 0x7f94441f2b10 2023-11-24T07:57:50.761 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:50.754+0000 7f944c008700 1 -- 172.21.15.59:0/1975641721 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9444068670 con 0x7f94441f2b10 2023-11-24T07:57:50.761 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:50.755+0000 7f94367fc700 1 -- 172.21.15.59:0/1975641721 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 6) v1 ==== 89282+0+0 (secure 0 0 0) 0x7f943c022460 con 0x7f94441f2b10 2023-11-24T07:57:50.761 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:50.755+0000 7f94367fc700 1 --2- 172.21.15.59:0/1975641721 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f942806af60 0x7f942806d420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:57:50.761 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:50.755+0000 7f94367fc700 1 -- 172.21.15.59:0/1975641721 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 5795+0+0 (secure 0 0 0) 0x7f943c089db0 con 0x7f94441f2b10 2023-11-24T07:57:50.761 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:50.756+0000 7f9449da4700 1 --2- 172.21.15.59:0/1975641721 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f942806af60 0x7f942806d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:57:50.761 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:50.756+0000 7f9449da4700 1 --2- 172.21.15.59:0/1975641721 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f942806af60 0x7f942806d420 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f9438000c00 tx=0x7f9438009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:57:50.761 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:50.760+0000 7f94367fc700 1 -- 172.21.15.59:0/1975641721 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f943c054d90 con 0x7f94441f2b10 2023-11-24T07:57:50.962 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:50.960+0000 7f944c008700 1 -- 172.21.15.59:0/1975641721 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f9444146100 con 0x7f94441f2b10 2023-11-24T07:57:51.555 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:51.553+0000 7f94367fc700 1 -- 172.21.15.59:0/1975641721 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v35) v1 ==== 181+0+0 (secure 0 0 0) 0x7f943c0583b0 con 0x7f94441f2b10 2023-11-24T07:57:51.559 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:51.557+0000 7f944c008700 1 -- 172.21.15.59:0/1975641721 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f942806af60 msgr2=0x7f942806d420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:57:51.559 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:51.557+0000 7f944c008700 1 --2- 172.21.15.59:0/1975641721 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f942806af60 0x7f942806d420 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f9438000c00 tx=0x7f9438009040 comp rx=0 tx=0).stop 2023-11-24T07:57:51.559 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:51.557+0000 7f944c008700 1 -- 172.21.15.59:0/1975641721 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f94441f2b10 msgr2=0x7f94441f2fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:57:51.560 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:51.557+0000 7f944c008700 1 --2- 172.21.15.59:0/1975641721 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f94441f2b10 0x7f94441f2fa0 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7f944406fc40 tx=0x7f943c011f80 comp rx=0 tx=0).stop 2023-11-24T07:57:51.560 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:51.558+0000 7f944c008700 1 -- 172.21.15.59:0/1975641721 shutdown_connections 2023-11-24T07:57:51.560 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:51.558+0000 7f944c008700 1 --2- 172.21.15.59:0/1975641721 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f942806af60 0x7f942806d420 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:51.560 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:51.558+0000 7f944c008700 1 --2- 172.21.15.59:0/1975641721 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f94441f2b10 0x7f94441f2fa0 unknown :-1 s=CLOSED pgs=486 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:51.560 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:51.558+0000 7f944c008700 1 --2- 172.21.15.59:0/1975641721 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9430004400 0x7f94440cc5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:51.560 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:51.558+0000 7f944c008700 1 --2- 172.21.15.59:0/1975641721 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f94441f7660 0x7f944415db80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:51.560 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:51.558+0000 7f944c008700 1 -- 172.21.15.59:0/1975641721 >> 172.21.15.59:0/1975641721 conn(0x7f94440b5070 msgr2=0x7f944415d1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:57:51.560 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:51.558+0000 7f944c008700 1 -- 172.21.15.59:0/1975641721 shutdown_connections 2023-11-24T07:57:51.560 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:51.558+0000 7f944c008700 1 -- 172.21.15.59:0/1975641721 wait complete. 2023-11-24T07:57:51.560 INFO:teuthology.orchestra.run.smithi059.stderr:pool 'cephfs_metadata' removed 2023-11-24T07:57:51.576 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2023-11-24T07:57:51.713 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:51.711+0000 7f73cfeaf700 1 Processor -- start 2023-11-24T07:57:51.713 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:51.711+0000 7f73cfeaf700 1 -- start start 2023-11-24T07:57:51.715 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:51.712+0000 7f73cfeaf700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f73c81457b0 0x7f73c80725a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:57:51.715 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:51.712+0000 7f73cfeaf700 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f73c80cf020 0x7f73c80cf4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:57:51.716 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:51.712+0000 7f73cfeaf700 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f73c80cd490 con 0x7f73c806f580 2023-11-24T07:57:51.716 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:51.712+0000 7f73cfeaf700 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f73c806fa00 con 0x7f73c81457b0 2023-11-24T07:57:51.716 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:51.712+0000 7f73cfeaf700 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f73c80cd6e0 con 0x7f73c80cf020 2023-11-24T07:57:51.716 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:51.712+0000 7f73cdc4b700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f73c81457b0 0x7f73c80725a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:57:51.716 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:51.712+0000 7f73cd44a700 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7f73c806f580 0x7f73c806e3d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:55890/0 (socket says 172.21.15.59:55890) 2023-11-24T07:57:51.716 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:51.712+0000 7f73cd44a700 1 -- 172.21.15.59:0/3091125657 learned_addr learned my addr 172.21.15.59:0/3091125657 (peer_addr_for_me v1:172.21.15.59:0/0) 2023-11-24T07:57:51.716 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:51.712+0000 7f73ce44c700 1 --2- 172.21.15.59:0/3091125657 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f73c80cf020 0x7f73c80cf4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:57:51.716 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:51.712+0000 7f73cdc4b700 1 -- 172.21.15.59:0/3091125657 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f73c80cf020 msgr2=0x7f73c80cf4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T07:57:51.716 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:51.712+0000 7f73cdc4b700 1 --2- 172.21.15.59:0/3091125657 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f73c80cf020 0x7f73c80cf4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:51.716 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:51.712+0000 7f73cdc4b700 1 -- 172.21.15.59:0/3091125657 >> v1:172.21.15.59:6789/0 conn(0x7f73c806f580 legacy=0x7f73c806e3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:57:51.717 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:51.712+0000 7f73cdc4b700 1 -- 172.21.15.59:0/3091125657 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f73c80cd9c0 con 0x7f73c81457b0 2023-11-24T07:57:51.717 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:51.712+0000 7f73cdc4b700 1 --2- 172.21.15.59:0/3091125657 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f73c81457b0 0x7f73c80725a0 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7f73c4009840 tx=0x7f73c4009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7844b6aded599be8 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:57:51.717 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:51.713+0000 7f73ccc49700 1 -- 172.21.15.59:0/3091125657 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73c4011830 con 0x7f73c81457b0 2023-11-24T07:57:51.717 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:51.713+0000 7f73ccc49700 1 -- 172.21.15.59:0/3091125657 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f73c4011990 con 0x7f73c81457b0 2023-11-24T07:57:51.717 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:51.713+0000 7f73ccc49700 1 -- 172.21.15.59:0/3091125657 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73c4011d20 con 0x7f73c81457b0 2023-11-24T07:57:51.717 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:51.713+0000 7f73cfeaf700 1 -- 172.21.15.59:0/3091125657 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f73c81457b0 msgr2=0x7f73c80725a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:57:51.717 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:51.713+0000 7f73cfeaf700 1 --2- 172.21.15.59:0/3091125657 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f73c81457b0 0x7f73c80725a0 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7f73c4009840 tx=0x7f73c4009b50 comp rx=0 tx=0).stop 2023-11-24T07:57:51.717 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:51.713+0000 7f73cfeaf700 1 -- 172.21.15.59:0/3091125657 shutdown_connections 2023-11-24T07:57:51.717 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:51.713+0000 7f73cfeaf700 1 --2- 172.21.15.59:0/3091125657 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f73c80cf020 0x7f73c80cf4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:51.717 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:51.713+0000 7f73cfeaf700 1 --2- 172.21.15.59:0/3091125657 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f73c81457b0 0x7f73c80725a0 unknown :-1 s=CLOSED pgs=488 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:51.717 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:51.713+0000 7f73cfeaf700 1 -- 172.21.15.59:0/3091125657 >> 172.21.15.59:0/3091125657 conn(0x7f73c80b2760 msgr2=0x7f73c80b2b70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:57:51.717 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:51.713+0000 7f73cfeaf700 1 -- 172.21.15.59:0/3091125657 shutdown_connections 2023-11-24T07:57:51.717 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:51.713+0000 7f73cfeaf700 1 -- 172.21.15.59:0/3091125657 wait complete. 2023-11-24T07:57:51.720 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:51.714+0000 7f73cfeaf700 1 Processor -- start 2023-11-24T07:57:51.720 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:51.715+0000 7f73cfeaf700 1 -- start start 2023-11-24T07:57:51.720 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:51.715+0000 7f73cfeaf700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f73c806f580 0x7f73c80b4570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:57:51.721 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:51.715+0000 7f73cfeaf700 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f73c80cf020 0x7f73c815e780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:57:51.721 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:51.715+0000 7f73cfeaf700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f73c81457b0 0x7f73c815b880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:57:51.721 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:51.715+0000 7f73cfeaf700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f73c815c410 con 0x7f73c81457b0 2023-11-24T07:57:51.721 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:51.715+0000 7f73cfeaf700 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f73c815f100 con 0x7f73c806f580 2023-11-24T07:57:51.721 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:51.715+0000 7f73cfeaf700 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f73c815bdc0 con 0x7f73c80cf020 2023-11-24T07:57:51.721 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:51.715+0000 7f73ce44c700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f73c81457b0 0x7f73c815b880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:57:51.721 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:51.715+0000 7f73ce44c700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f73c81457b0 0x7f73c815b880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:52426/0 (socket says 172.21.15.59:52426) 2023-11-24T07:57:51.721 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:51.715+0000 7f73ce44c700 1 -- 172.21.15.59:0/3203204054 learned_addr learned my addr 172.21.15.59:0/3203204054 (peer_addr_for_me v2:172.21.15.59:0/0) 2023-11-24T07:57:51.722 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:51.716+0000 7f73cd44a700 1 --2- 172.21.15.59:0/3203204054 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f73c80cf020 0x7f73c815e780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:57:51.722 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:51.716+0000 7f73cdc4b700 1 --2- 172.21.15.59:0/3203204054 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f73c806f580 0x7f73c80b4570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:57:51.722 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:51.716+0000 7f73ce44c700 1 -- 172.21.15.59:0/3203204054 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f73c80cf020 msgr2=0x7f73c815e780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:57:51.722 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:51.716+0000 7f73ce44c700 1 --2- 172.21.15.59:0/3203204054 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f73c80cf020 0x7f73c815e780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:51.722 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:51.716+0000 7f73ce44c700 1 -- 172.21.15.59:0/3203204054 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f73c806f580 msgr2=0x7f73c80b4570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:57:51.722 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:51.716+0000 7f73ce44c700 1 --2- 172.21.15.59:0/3203204054 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f73c806f580 0x7f73c80b4570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:51.722 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:51.716+0000 7f73ce44c700 1 -- 172.21.15.59:0/3203204054 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f73c40094a0 con 0x7f73c81457b0 2023-11-24T07:57:51.722 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:51.716+0000 7f73cdc4b700 1 --2- 172.21.15.59:0/3203204054 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f73c806f580 0x7f73c80b4570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T07:57:51.722 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:51.716+0000 7f73ce44c700 1 --2- 172.21.15.59:0/3203204054 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f73c81457b0 0x7f73c815b880 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7f73c0007e70 tx=0x7f73c000e3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:57:51.723 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:51.716+0000 7f73be7fc700 1 -- 172.21.15.59:0/3203204054 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73c000ee50 con 0x7f73c81457b0 2023-11-24T07:57:51.723 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:51.716+0000 7f73be7fc700 1 -- 172.21.15.59:0/3203204054 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f73c0014420 con 0x7f73c81457b0 2023-11-24T07:57:51.723 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:51.716+0000 7f73be7fc700 1 -- 172.21.15.59:0/3203204054 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73c00147d0 con 0x7f73c81457b0 2023-11-24T07:57:51.723 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:51.716+0000 7f73cfeaf700 1 -- 172.21.15.59:0/3203204054 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f73c80b4b50 con 0x7f73c81457b0 2023-11-24T07:57:51.723 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:51.716+0000 7f73cfeaf700 1 -- 172.21.15.59:0/3203204054 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f73c80b5110 con 0x7f73c81457b0 2023-11-24T07:57:51.723 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:51.717+0000 7f73cfeaf700 1 -- 172.21.15.59:0/3203204054 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f73c8142df0 con 0x7f73c81457b0 2023-11-24T07:57:51.723 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:51.717+0000 7f73be7fc700 1 -- 172.21.15.59:0/3203204054 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 6) v1 ==== 89282+0+0 (secure 0 0 0) 0x7f73c0014930 con 0x7f73c81457b0 2023-11-24T07:57:51.723 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:51.718+0000 7f73be7fc700 1 --2- 172.21.15.59:0/3203204054 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f73ac06af60 0x7f73ac06d420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:57:51.723 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:51.718+0000 7f73be7fc700 1 -- 172.21.15.59:0/3203204054 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 5357+0+0 (secure 0 0 0) 0x7f73c0089c30 con 0x7f73c81457b0 2023-11-24T07:57:51.723 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:51.718+0000 7f73cdc4b700 1 --2- 172.21.15.59:0/3203204054 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f73ac06af60 0x7f73ac06d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:57:51.724 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:51.719+0000 7f73cdc4b700 1 --2- 172.21.15.59:0/3203204054 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f73ac06af60 0x7f73ac06d420 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f73c40090b0 tx=0x7f73c4008030 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:57:51.724 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:51.722+0000 7f73be7fc700 1 -- 172.21.15.59:0/3203204054 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f73c0054ee0 con 0x7f73c81457b0 2023-11-24T07:57:51.923 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:51.921+0000 7f73cfeaf700 1 -- 172.21.15.59:0/3203204054 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f73c8068670 con 0x7f73c81457b0 2023-11-24T07:57:52.560 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:52.558+0000 7f73be7fc700 1 -- 172.21.15.59:0/3203204054 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v36) v1 ==== 169+0+0 (secure 0 0 0) 0x7f73c0058500 con 0x7f73c81457b0 2023-11-24T07:57:52.566 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:52.561+0000 7f73cfeaf700 1 -- 172.21.15.59:0/3203204054 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f73ac06af60 msgr2=0x7f73ac06d420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:57:52.566 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:52.561+0000 7f73cfeaf700 1 --2- 172.21.15.59:0/3203204054 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f73ac06af60 0x7f73ac06d420 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f73c40090b0 tx=0x7f73c4008030 comp rx=0 tx=0).stop 2023-11-24T07:57:52.566 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:52.561+0000 7f73cfeaf700 1 -- 172.21.15.59:0/3203204054 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f73c81457b0 msgr2=0x7f73c815b880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:57:52.566 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:52.561+0000 7f73cfeaf700 1 --2- 172.21.15.59:0/3203204054 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f73c81457b0 0x7f73c815b880 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7f73c0007e70 tx=0x7f73c000e3c0 comp rx=0 tx=0).stop 2023-11-24T07:57:52.567 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:52.562+0000 7f73cfeaf700 1 -- 172.21.15.59:0/3203204054 shutdown_connections 2023-11-24T07:57:52.567 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:52.562+0000 7f73cfeaf700 1 --2- 172.21.15.59:0/3203204054 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f73ac06af60 0x7f73ac06d420 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:52.567 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:52.562+0000 7f73cfeaf700 1 --2- 172.21.15.59:0/3203204054 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f73c81457b0 0x7f73c815b880 unknown :-1 s=CLOSED pgs=489 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:52.567 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:52.562+0000 7f73cfeaf700 1 --2- 172.21.15.59:0/3203204054 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f73c806f580 0x7f73c80b4570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:52.567 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:52.562+0000 7f73cfeaf700 1 --2- 172.21.15.59:0/3203204054 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f73c80cf020 0x7f73c815e780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:52.567 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:52.562+0000 7f73cfeaf700 1 -- 172.21.15.59:0/3203204054 >> 172.21.15.59:0/3203204054 conn(0x7f73c80b2760 msgr2=0x7f73c8141950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:57:52.567 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:52.562+0000 7f73cfeaf700 1 -- 172.21.15.59:0/3203204054 shutdown_connections 2023-11-24T07:57:52.567 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:52.562+0000 7f73cfeaf700 1 -- 172.21.15.59:0/3203204054 wait complete. 2023-11-24T07:57:52.567 INFO:teuthology.orchestra.run.smithi059.stderr:pool 'cephfs_data' removed 2023-11-24T07:57:52.582 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2023-11-24T07:57:52.725 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:52.723+0000 7f67d977c700 1 Processor -- start 2023-11-24T07:57:52.725 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:52.723+0000 7f67d977c700 1 -- start start 2023-11-24T07:57:52.727 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:52.724+0000 7f67d977c700 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f67d415e0e0 0x7f67d40d2420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:57:52.727 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:52.724+0000 7f67d977c700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f67d40d2960 0x7f67d40d0680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:57:52.727 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:52.724+0000 7f67d977c700 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f67d40d1010 con 0x7f67d40d0bc0 2023-11-24T07:57:52.727 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:52.724+0000 7f67d977c700 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f67d40cf800 con 0x7f67d40d2960 2023-11-24T07:57:52.727 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:52.724+0000 7f67d977c700 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f67d40d4ac0 con 0x7f67d415e0e0 2023-11-24T07:57:52.728 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:52.724+0000 7f67d27fc700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f67d40d2960 0x7f67d40d0680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:57:52.728 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:52.724+0000 7f67d37fe700 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7f67d40d0bc0 0x7f67d40b8f40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:55908/0 (socket says 172.21.15.59:55908) 2023-11-24T07:57:52.728 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:52.724+0000 7f67d37fe700 1 -- 172.21.15.59:0/4209636547 learned_addr learned my addr 172.21.15.59:0/4209636547 (peer_addr_for_me v1:172.21.15.59:0/0) 2023-11-24T07:57:52.728 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:52.724+0000 7f67d2ffd700 1 --2- 172.21.15.59:0/4209636547 >> v2:172.21.15.76:3300/0 conn(0x7f67d415e0e0 0x7f67d40d2420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:57:52.728 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:52.724+0000 7f67d27fc700 1 -- 172.21.15.59:0/4209636547 >> v2:172.21.15.76:3300/0 conn(0x7f67d415e0e0 msgr2=0x7f67d40d2420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T07:57:52.728 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:52.724+0000 7f67d27fc700 1 --2- 172.21.15.59:0/4209636547 >> v2:172.21.15.76:3300/0 conn(0x7f67d415e0e0 0x7f67d40d2420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:52.728 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:52.724+0000 7f67d27fc700 1 -- 172.21.15.59:0/4209636547 >> v1:172.21.15.59:6789/0 conn(0x7f67d40d0bc0 legacy=0x7f67d40b8f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:57:52.728 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:52.724+0000 7f67d27fc700 1 -- 172.21.15.59:0/4209636547 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f67d415e4c0 con 0x7f67d40d2960 2023-11-24T07:57:52.728 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:52.724+0000 7f67d1ffb700 1 -- 172.21.15.59:0/4209636547 <== mon.0 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 517340537 0 0) 0x7f67cc001490 con 0x7f67d40d0bc0 2023-11-24T07:57:52.728 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:52.724+0000 7f67d27fc700 1 --2- 172.21.15.59:0/4209636547 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f67d40d2960 0x7f67d40d0680 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7f67d408c520 tx=0x7f67c8009b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cd032be8e2664f22 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:57:52.728 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:52.724+0000 7f67d1ffb700 1 -- 172.21.15.59:0/4209636547 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67c8011830 con 0x7f67d40d2960 2023-11-24T07:57:52.728 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:52.724+0000 7f67d1ffb700 1 -- 172.21.15.59:0/4209636547 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f67c8011990 con 0x7f67d40d2960 2023-11-24T07:57:52.729 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:52.724+0000 7f67d1ffb700 1 -- 172.21.15.59:0/4209636547 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67c8011d20 con 0x7f67d40d2960 2023-11-24T07:57:52.729 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:52.725+0000 7f67d977c700 1 -- 172.21.15.59:0/4209636547 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f67d40d2960 msgr2=0x7f67d40d0680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:57:52.729 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:52.725+0000 7f67d977c700 1 --2- 172.21.15.59:0/4209636547 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f67d40d2960 0x7f67d40d0680 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7f67d408c520 tx=0x7f67c8009b20 comp rx=0 tx=0).stop 2023-11-24T07:57:52.729 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:52.725+0000 7f67d977c700 1 -- 172.21.15.59:0/4209636547 shutdown_connections 2023-11-24T07:57:52.729 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:52.725+0000 7f67d977c700 1 --2- 172.21.15.59:0/4209636547 >> v2:172.21.15.76:3300/0 conn(0x7f67d415e0e0 0x7f67d40d2420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:52.729 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:52.725+0000 7f67d977c700 1 --2- 172.21.15.59:0/4209636547 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f67d40d2960 0x7f67d40d0680 unknown :-1 s=CLOSED pgs=491 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:52.729 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:52.725+0000 7f67d977c700 1 -- 172.21.15.59:0/4209636547 >> 172.21.15.59:0/4209636547 conn(0x7f67d409d110 msgr2=0x7f67d40d3e40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:57:52.729 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:52.725+0000 7f67d977c700 1 -- 172.21.15.59:0/4209636547 shutdown_connections 2023-11-24T07:57:52.729 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:52.725+0000 7f67d977c700 1 -- 172.21.15.59:0/4209636547 wait complete. 2023-11-24T07:57:52.732 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:52.726+0000 7f67d977c700 1 Processor -- start 2023-11-24T07:57:52.732 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:52.726+0000 7f67d977c700 1 -- start start 2023-11-24T07:57:52.732 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:52.727+0000 7f67d977c700 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f67d40d0bc0 0x7f67d40b75d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:57:52.732 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:52.727+0000 7f67d977c700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f67d40d2960 0x7f67d415c9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:57:52.732 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:52.727+0000 7f67d977c700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f67d415e0e0 0x7f67d40a0360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:57:52.732 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:52.727+0000 7f67d977c700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67d40a1860 con 0x7f67d40d2960 2023-11-24T07:57:52.732 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:52.727+0000 7f67d977c700 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67d40a0be0 con 0x7f67d415e0e0 2023-11-24T07:57:52.732 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:52.727+0000 7f67d977c700 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f67d40a0ec0 con 0x7f67d40d0bc0 2023-11-24T07:57:52.732 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:52.727+0000 7f67d27fc700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f67d40d2960 0x7f67d415c9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:57:52.732 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:52.727+0000 7f67d27fc700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f67d40d2960 0x7f67d415c9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:52438/0 (socket says 172.21.15.59:52438) 2023-11-24T07:57:52.733 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:52.727+0000 7f67d27fc700 1 -- 172.21.15.59:0/2476276642 learned_addr learned my addr 172.21.15.59:0/2476276642 (peer_addr_for_me v2:172.21.15.59:0/0) 2023-11-24T07:57:52.733 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:52.727+0000 7f67d27fc700 1 -- 172.21.15.59:0/2476276642 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f67d40d0bc0 msgr2=0x7f67d40b75d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:57:52.733 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:52.727+0000 7f67d37fe700 1 --2- 172.21.15.59:0/2476276642 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f67d415e0e0 0x7f67d40a0360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:57:52.733 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:52.727+0000 7f67d27fc700 1 --2- 172.21.15.59:0/2476276642 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f67d40d0bc0 0x7f67d40b75d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:52.733 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:52.727+0000 7f67d27fc700 1 -- 172.21.15.59:0/2476276642 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f67d415e0e0 msgr2=0x7f67d40a0360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:57:52.733 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:52.727+0000 7f67d27fc700 1 --2- 172.21.15.59:0/2476276642 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f67d415e0e0 0x7f67d40a0360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:52.733 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:52.727+0000 7f67d27fc700 1 -- 172.21.15.59:0/2476276642 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f67c80094a0 con 0x7f67d40d2960 2023-11-24T07:57:52.733 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:52.728+0000 7f67d27fc700 1 --2- 172.21.15.59:0/2476276642 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f67d40d2960 0x7f67d415c9b0 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7f67c80090b0 tx=0x7f67c8004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:57:52.733 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:52.728+0000 7f67bb7fe700 1 -- 172.21.15.59:0/2476276642 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67c80038a0 con 0x7f67d40d2960 2023-11-24T07:57:52.733 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:52.728+0000 7f67bb7fe700 1 -- 172.21.15.59:0/2476276642 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f67c801f070 con 0x7f67d40d2960 2023-11-24T07:57:52.733 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:52.728+0000 7f67d977c700 1 -- 172.21.15.59:0/2476276642 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f67d40a11a0 con 0x7f67d40d2960 2023-11-24T07:57:52.733 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:52.728+0000 7f67bb7fe700 1 -- 172.21.15.59:0/2476276642 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67c8003c20 con 0x7f67d40d2960 2023-11-24T07:57:52.734 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:52.728+0000 7f67d977c700 1 -- 172.21.15.59:0/2476276642 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f67d415cef0 con 0x7f67d40d2960 2023-11-24T07:57:52.734 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:52.729+0000 7f67d977c700 1 -- 172.21.15.59:0/2476276642 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f67d4075480 con 0x7f67d40d2960 2023-11-24T07:57:52.734 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:52.729+0000 7f67bb7fe700 1 -- 172.21.15.59:0/2476276642 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 6) v1 ==== 89282+0+0 (secure 0 0 0) 0x7f67c801a430 con 0x7f67d40d2960 2023-11-24T07:57:52.734 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:52.730+0000 7f67bb7fe700 1 --2- 172.21.15.59:0/2476276642 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f67c006af60 0x7f67c006d420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:57:52.734 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:52.730+0000 7f67bb7fe700 1 -- 172.21.15.59:0/2476276642 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f67c8021070 con 0x7f67d40d2960 2023-11-24T07:57:52.734 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:52.730+0000 7f67d2ffd700 1 --2- 172.21.15.59:0/2476276642 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f67c006af60 0x7f67c006d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:57:52.734 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:52.730+0000 7f67d2ffd700 1 --2- 172.21.15.59:0/2476276642 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f67c006af60 0x7f67c006d420 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f67c40097f0 tx=0x7f67c4008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:57:52.736 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:52.734+0000 7f67bb7fe700 1 -- 172.21.15.59:0/2476276642 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f67c80560c0 con 0x7f67d40d2960 2023-11-24T07:57:52.939 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:52.937+0000 7f67d977c700 1 -- 172.21.15.59:0/2476276642 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7f67d40a19a0 con 0x7f67d40d2960 2023-11-24T07:57:52.942 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:52.940+0000 7f67bb7fe700 1 -- 172.21.15.59:0/2476276642 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v11) v1 ==== 104+0+0 (secure 0 0 0) 0x7f67d40a19a0 con 0x7f67d40d2960 2023-11-24T07:57:52.945 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:52.942+0000 7f67d977c700 1 -- 172.21.15.59:0/2476276642 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f67c006af60 msgr2=0x7f67c006d420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:57:52.945 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:52.942+0000 7f67d977c700 1 --2- 172.21.15.59:0/2476276642 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f67c006af60 0x7f67c006d420 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f67c40097f0 tx=0x7f67c4008040 comp rx=0 tx=0).stop 2023-11-24T07:57:52.945 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:52.942+0000 7f67d977c700 1 -- 172.21.15.59:0/2476276642 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f67d40d2960 msgr2=0x7f67d415c9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:57:52.945 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:52.942+0000 7f67d977c700 1 --2- 172.21.15.59:0/2476276642 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f67d40d2960 0x7f67d415c9b0 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7f67c80090b0 tx=0x7f67c8004750 comp rx=0 tx=0).stop 2023-11-24T07:57:52.945 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:52.942+0000 7f67d977c700 1 -- 172.21.15.59:0/2476276642 shutdown_connections 2023-11-24T07:57:52.945 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:52.942+0000 7f67d977c700 1 --2- 172.21.15.59:0/2476276642 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f67c006af60 0x7f67c006d420 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:52.945 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:52.943+0000 7f67d977c700 1 --2- 172.21.15.59:0/2476276642 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f67d415e0e0 0x7f67d40a0360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:52.945 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:52.943+0000 7f67d977c700 1 --2- 172.21.15.59:0/2476276642 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f67d40d0bc0 0x7f67d40b75d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:52.945 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:52.943+0000 7f67d977c700 1 --2- 172.21.15.59:0/2476276642 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f67d40d2960 0x7f67d415c9b0 unknown :-1 s=CLOSED pgs=492 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:52.945 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:52.943+0000 7f67d977c700 1 -- 172.21.15.59:0/2476276642 >> 172.21.15.59:0/2476276642 conn(0x7f67d409d110 msgr2=0x7f67d40ccf20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:57:52.946 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:52.943+0000 7f67d977c700 1 -- 172.21.15.59:0/2476276642 shutdown_connections 2023-11-24T07:57:52.946 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:52.943+0000 7f67d977c700 1 -- 172.21.15.59:0/2476276642 wait complete. 2023-11-24T07:57:52.956 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_volumes.TestSubvolumeGroupSnapshots.test_subvolume_group_snapshot_unsupported_status' 2023-11-24T07:57:53.091 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:53.080+0000 7f84ba21c700 1 Processor -- start 2023-11-24T07:57:53.091 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:53.080+0000 7f84ba21c700 1 -- start start 2023-11-24T07:57:53.091 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:53.080+0000 7f84ba21c700 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f84b4143ae0 0x7f84b40b51b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:57:53.092 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:53.080+0000 7f84ba21c700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f84b41440f0 0x7f84b4071bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:57:53.092 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:53.080+0000 7f84ba21c700 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f84b4143ec0 con 0x7f84b40720f0 2023-11-24T07:57:53.092 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:53.080+0000 7f84ba21c700 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f84b40b40a0 con 0x7f84b41440f0 2023-11-24T07:57:53.092 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:53.080+0000 7f84ba21c700 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f84b4072580 con 0x7f84b4143ae0 2023-11-24T07:57:53.092 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:53.081+0000 7f84b2ffd700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f84b41440f0 0x7f84b4071bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:57:53.093 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:53.081+0000 7f84b3fff700 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7f84b40720f0 0x7f84b415e8b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:55922/0 (socket says 172.21.15.59:55922) 2023-11-24T07:57:53.093 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:53.081+0000 7f84b3fff700 1 -- 172.21.15.59:0/2662150834 learned_addr learned my addr 172.21.15.59:0/2662150834 (peer_addr_for_me v1:172.21.15.59:0/0) 2023-11-24T07:57:53.094 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:53.081+0000 7f84b37fe700 1 --2- 172.21.15.59:0/2662150834 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f84b4143ae0 0x7f84b40b51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:57:53.094 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:53.081+0000 7f84b2ffd700 1 -- 172.21.15.59:0/2662150834 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f84b4143ae0 msgr2=0x7f84b40b51b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T07:57:53.094 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:53.081+0000 7f84b2ffd700 1 --2- 172.21.15.59:0/2662150834 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f84b4143ae0 0x7f84b40b51b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:53.094 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:53.081+0000 7f84b2ffd700 1 -- 172.21.15.59:0/2662150834 >> v1:172.21.15.59:6789/0 conn(0x7f84b40720f0 legacy=0x7f84b415e8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:57:53.094 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:53.081+0000 7f84b2ffd700 1 -- 172.21.15.59:0/2662150834 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f84b40b3ae0 con 0x7f84b41440f0 2023-11-24T07:57:53.095 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:53.081+0000 7f84b27fc700 1 -- 172.21.15.59:0/2662150834 <== mon.0 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2734975754 0 0) 0x7f84b4143ec0 con 0x7f84b40720f0 2023-11-24T07:57:53.095 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:53.081+0000 7f84b2ffd700 1 --2- 172.21.15.59:0/2662150834 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f84b41440f0 0x7f84b4071bb0 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7f84a8009840 tx=0x7f84a8009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e7d6e971fd9890e6 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:57:53.095 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:53.081+0000 7f84b27fc700 1 -- 172.21.15.59:0/2662150834 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84a8011830 con 0x7f84b41440f0 2023-11-24T07:57:53.095 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:53.081+0000 7f84b27fc700 1 -- 172.21.15.59:0/2662150834 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f84a8011990 con 0x7f84b41440f0 2023-11-24T07:57:53.095 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:53.081+0000 7f84b27fc700 1 -- 172.21.15.59:0/2662150834 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84a8011d20 con 0x7f84b41440f0 2023-11-24T07:57:53.095 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:53.082+0000 7f84ba21c700 1 -- 172.21.15.59:0/2662150834 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f84b41440f0 msgr2=0x7f84b4071bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:57:53.095 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:53.082+0000 7f84ba21c700 1 --2- 172.21.15.59:0/2662150834 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f84b41440f0 0x7f84b4071bb0 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7f84a8009840 tx=0x7f84a8009b50 comp rx=0 tx=0).stop 2023-11-24T07:57:53.095 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:53.082+0000 7f84ba21c700 1 -- 172.21.15.59:0/2662150834 shutdown_connections 2023-11-24T07:57:53.096 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:53.082+0000 7f84ba21c700 1 --2- 172.21.15.59:0/2662150834 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f84b4143ae0 0x7f84b40b51b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:53.096 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:53.082+0000 7f84ba21c700 1 --2- 172.21.15.59:0/2662150834 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f84b41440f0 0x7f84b4071bb0 unknown :-1 s=CLOSED pgs=494 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:53.096 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:53.082+0000 7f84ba21c700 1 -- 172.21.15.59:0/2662150834 >> 172.21.15.59:0/2662150834 conn(0x7f84b40b2740 msgr2=0x7f84b40b2b20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:57:53.096 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:53.082+0000 7f84ba21c700 1 -- 172.21.15.59:0/2662150834 shutdown_connections 2023-11-24T07:57:53.096 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:53.082+0000 7f84ba21c700 1 -- 172.21.15.59:0/2662150834 wait complete. 2023-11-24T07:57:53.096 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:53.083+0000 7f84ba21c700 1 Processor -- start 2023-11-24T07:57:53.096 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:53.083+0000 7f84ba21c700 1 -- start start 2023-11-24T07:57:53.097 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:53.084+0000 7f84ba21c700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f84b40720f0 0x7f84b41368d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:57:53.097 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:53.084+0000 7f84ba21c700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f84b4143ae0 0x7f84b406b630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:57:53.097 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:53.084+0000 7f84ba21c700 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f84b41440f0 0x7f84b4142730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:57:53.097 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:53.084+0000 7f84ba21c700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84b40c5ae0 con 0x7f84b40720f0 2023-11-24T07:57:53.097 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:53.084+0000 7f84ba21c700 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84b4133e70 con 0x7f84b4143ae0 2023-11-24T07:57:53.097 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:53.084+0000 7f84ba21c700 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f84b41f5d70 con 0x7f84b41440f0 2023-11-24T07:57:53.098 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:53.084+0000 7f84b37fe700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f84b40720f0 0x7f84b41368d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:57:53.098 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:53.084+0000 7f84b37fe700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f84b40720f0 0x7f84b41368d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:52450/0 (socket says 172.21.15.59:52450) 2023-11-24T07:57:53.098 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:53.084+0000 7f84b37fe700 1 -- 172.21.15.59:0/2459160346 learned_addr learned my addr 172.21.15.59:0/2459160346 (peer_addr_for_me v2:172.21.15.59:0/0) 2023-11-24T07:57:53.098 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:53.084+0000 7f84b3fff700 1 --2- 172.21.15.59:0/2459160346 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f84b41440f0 0x7f84b4142730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:57:53.098 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:53.084+0000 7f84b2ffd700 1 --2- 172.21.15.59:0/2459160346 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f84b4143ae0 0x7f84b406b630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:57:53.098 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:53.084+0000 7f84b37fe700 1 -- 172.21.15.59:0/2459160346 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f84b41440f0 msgr2=0x7f84b4142730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:57:53.098 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:53.084+0000 7f84b37fe700 1 --2- 172.21.15.59:0/2459160346 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f84b41440f0 0x7f84b4142730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:53.098 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:53.084+0000 7f84b37fe700 1 -- 172.21.15.59:0/2459160346 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f84b4143ae0 msgr2=0x7f84b406b630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:57:53.098 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:53.084+0000 7f84b37fe700 1 --2- 172.21.15.59:0/2459160346 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f84b4143ae0 0x7f84b406b630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:53.098 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:53.084+0000 7f84b37fe700 1 -- 172.21.15.59:0/2459160346 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f84a80094a0 con 0x7f84b40720f0 2023-11-24T07:57:53.098 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:53.085+0000 7f84b37fe700 1 --2- 172.21.15.59:0/2459160346 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f84b40720f0 0x7f84b41368d0 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7f84a400e840 tx=0x7f84a400ec00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:57:53.099 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:53.085+0000 7f849bfff700 1 -- 172.21.15.59:0/2459160346 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84a4013610 con 0x7f84b40720f0 2023-11-24T07:57:53.099 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:53.085+0000 7f849bfff700 1 -- 172.21.15.59:0/2459160346 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f84a4013770 con 0x7f84b40720f0 2023-11-24T07:57:53.099 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:53.085+0000 7f84ba21c700 1 -- 172.21.15.59:0/2459160346 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f84b41f6050 con 0x7f84b40720f0 2023-11-24T07:57:53.099 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:53.085+0000 7f849bfff700 1 -- 172.21.15.59:0/2459160346 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84a4013b20 con 0x7f84b40720f0 2023-11-24T07:57:53.099 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:53.085+0000 7f84ba21c700 1 -- 172.21.15.59:0/2459160346 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f84b41f6610 con 0x7f84b40720f0 2023-11-24T07:57:53.099 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:53.086+0000 7f84ba21c700 1 -- 172.21.15.59:0/2459160346 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f84b4068670 con 0x7f84b40720f0 2023-11-24T07:57:53.099 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:53.086+0000 7f849bfff700 1 -- 172.21.15.59:0/2459160346 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 6) v1 ==== 89282+0+0 (secure 0 0 0) 0x7f84a4013c80 con 0x7f84b40720f0 2023-11-24T07:57:53.099 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:53.087+0000 7f849bfff700 1 --2- 172.21.15.59:0/2459160346 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f84a006af60 0x7f84a006d420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:57:53.099 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:53.087+0000 7f849bfff700 1 -- 172.21.15.59:0/2459160346 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f84a4089120 con 0x7f84b40720f0 2023-11-24T07:57:53.099 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:53.087+0000 7f84b2ffd700 1 --2- 172.21.15.59:0/2459160346 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f84a006af60 0x7f84a006d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:57:53.099 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:53.087+0000 7f84b2ffd700 1 --2- 172.21.15.59:0/2459160346 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f84a006af60 0x7f84a006d420 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f84a8000c00 tx=0x7f84a8008030 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:57:53.100 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:53.091+0000 7f849bfff700 1 -- 172.21.15.59:0/2459160346 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f84a40543d0 con 0x7f84b40720f0 2023-11-24T07:57:53.376 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:53.374+0000 7f84ba21c700 1 -- 172.21.15.59:0/2459160346 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_volumes.TestSubvolumeGroupSnapshots.test_subvolume_group_snapshot_unsupported_status"]} v 0) v1 -- 0x7f84b40c5dc0 con 0x7f84b40720f0 2023-11-24T07:57:53.561 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:53.559+0000 7f849bfff700 1 -- 172.21.15.59:0/2459160346 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_volumes.TestSubvolumeGroupSnapshots.test_subvolume_group_snapshot_unsupported_status"]}]=0 v99) v1 ==== 181+0+0 (secure 0 0 0) 0x7f84a40579f0 con 0x7f84b40720f0 2023-11-24T07:57:53.564 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:53.562+0000 7f84ba21c700 1 -- 172.21.15.59:0/2459160346 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f84a006af60 msgr2=0x7f84a006d420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:57:53.564 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:53.562+0000 7f84ba21c700 1 --2- 172.21.15.59:0/2459160346 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f84a006af60 0x7f84a006d420 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f84a8000c00 tx=0x7f84a8008030 comp rx=0 tx=0).stop 2023-11-24T07:57:53.565 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:53.562+0000 7f84ba21c700 1 -- 172.21.15.59:0/2459160346 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f84b40720f0 msgr2=0x7f84b41368d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:57:53.565 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:53.562+0000 7f84ba21c700 1 --2- 172.21.15.59:0/2459160346 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f84b40720f0 0x7f84b41368d0 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7f84a400e840 tx=0x7f84a400ec00 comp rx=0 tx=0).stop 2023-11-24T07:57:53.565 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:53.562+0000 7f84ba21c700 1 -- 172.21.15.59:0/2459160346 shutdown_connections 2023-11-24T07:57:53.565 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:53.562+0000 7f84ba21c700 1 --2- 172.21.15.59:0/2459160346 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f84a006af60 0x7f84a006d420 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:53.565 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:53.562+0000 7f84ba21c700 1 --2- 172.21.15.59:0/2459160346 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f84b41440f0 0x7f84b4142730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:53.565 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:53.562+0000 7f84ba21c700 1 --2- 172.21.15.59:0/2459160346 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f84b40720f0 0x7f84b41368d0 unknown :-1 s=CLOSED pgs=495 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:53.565 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:53.562+0000 7f84ba21c700 1 --2- 172.21.15.59:0/2459160346 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f84b4143ae0 0x7f84b406b630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:53.565 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:53.562+0000 7f84ba21c700 1 -- 172.21.15.59:0/2459160346 >> 172.21.15.59:0/2459160346 conn(0x7f84b40b2740 msgr2=0x7f84b406e510 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:57:53.565 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:53.562+0000 7f84ba21c700 1 -- 172.21.15.59:0/2459160346 shutdown_connections 2023-11-24T07:57:53.565 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:53.562+0000 7f84ba21c700 1 -- 172.21.15.59:0/2459160346 wait complete. 2023-11-24T07:57:53.577 INFO:tasks.cephfs_test_runner:test_subvolume_group_snapshot_unsupported_status (tasks.cephfs.test_volumes.TestSubvolumeGroupSnapshots) ... ok 2023-11-24T07:57:53.577 INFO:tasks.cephfs_test_runner:Starting test: test_clean_stale_subvolume_snapshot_metadata (tasks.cephfs.test_volumes.TestSubvolumeSnapshots) Validate cleaning of stale subvolume snapshot metadata. 2023-11-24T07:57:53.577 INFO:tasks.cephfs_test_runner:test_clean_stale_subvolume_snapshot_metadata (tasks.cephfs.test_volumes.TestSubvolumeSnapshots) 2023-11-24T07:57:53.577 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-24T07:57:53.702 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:53.701+0000 7f5556112700 1 Processor -- start 2023-11-24T07:57:53.703 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:53.701+0000 7f5556112700 1 -- start start 2023-11-24T07:57:53.706 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:53.701+0000 7f5556112700 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f5550144c10 0x7f555006afd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:57:53.706 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:53.701+0000 7f5556112700 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f5550145650 0x7f555006b510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:57:53.706 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:53.701+0000 7f5556112700 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f555006eb80 con 0x7f5550145100 2023-11-24T07:57:53.706 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:53.701+0000 7f5556112700 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f555006f500 con 0x7f5550145650 2023-11-24T07:57:53.706 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:53.701+0000 7f5556112700 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f55500cd610 con 0x7f5550144c10 2023-11-24T07:57:53.706 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:53.702+0000 7f554effd700 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7f5550145100 0x7f5550071bb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:55934/0 (socket says 172.21.15.59:55934) 2023-11-24T07:57:53.706 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:53.702+0000 7f554effd700 1 -- 172.21.15.59:0/1611202986 learned_addr learned my addr 172.21.15.59:0/1611202986 (peer_addr_for_me v1:172.21.15.59:0/0) 2023-11-24T07:57:53.706 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:53.702+0000 7f554e7fc700 1 -- 172.21.15.59:0/1611202986 <== mon.0 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2891555949 0 0) 0x7f555006eb80 con 0x7f5550145100 2023-11-24T07:57:53.706 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:53.702+0000 7f554ffff700 1 --2- 172.21.15.59:0/1611202986 >> v2:172.21.15.76:3300/0 conn(0x7f5550145650 0x7f555006b510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:57:53.707 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:53.702+0000 7f554e7fc700 1 -- 172.21.15.59:0/1611202986 --> v1:172.21.15.59:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f55380035b0 con 0x7f5550145100 2023-11-24T07:57:53.707 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:53.702+0000 7f554f7fe700 1 --2- 172.21.15.59:0/1611202986 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f5550144c10 0x7f555006afd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:57:53.707 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:53.702+0000 7f554e7fc700 1 -- 172.21.15.59:0/1611202986 <== mon.0 v1:172.21.15.59:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1306739644 0 0) 0x7f55380035b0 con 0x7f5550145100 2023-11-24T07:57:53.707 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:53.702+0000 7f554e7fc700 1 -- 172.21.15.59:0/1611202986 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f5550144c10 msgr2=0x7f555006afd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T07:57:53.707 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:53.702+0000 7f554e7fc700 1 --2- 172.21.15.59:0/1611202986 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f5550144c10 0x7f555006afd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:53.707 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:53.702+0000 7f554e7fc700 1 -- 172.21.15.59:0/1611202986 >> v2:172.21.15.76:3300/0 conn(0x7f5550145650 msgr2=0x7f555006b510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T07:57:53.707 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:53.702+0000 7f554e7fc700 1 --2- 172.21.15.59:0/1611202986 >> v2:172.21.15.76:3300/0 conn(0x7f5550145650 0x7f555006b510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:53.707 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:53.702+0000 7f554e7fc700 1 -- 172.21.15.59:0/1611202986 --> v1:172.21.15.59:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5550160720 con 0x7f5550145100 2023-11-24T07:57:53.707 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:53.702+0000 7f554e7fc700 1 -- 172.21.15.59:0/1611202986 <== mon.0 v1:172.21.15.59:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1312333566 0 0) 0x7f5544001670 con 0x7f5550145100 2023-11-24T07:57:53.708 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:53.702+0000 7f554e7fc700 1 -- 172.21.15.59:0/1611202986 >> v1:172.21.15.59:6789/0 conn(0x7f5550145100 legacy=0x7f5550071bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:57:53.708 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:53.702+0000 7f554e7fc700 1 --2- 172.21.15.59:0/1611202986 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5538003a00 0x7f5538003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:57:53.708 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:53.702+0000 7f554e7fc700 1 --2- 172.21.15.59:0/1611202986 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f5550144c10 0x7f5538004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:57:53.708 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:53.702+0000 7f554e7fc700 1 --2- 172.21.15.59:0/1611202986 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5538004d90 0x7f5538005210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:57:53.708 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:53.702+0000 7f554e7fc700 1 -- 172.21.15.59:0/1611202986 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f555006f500 con 0x7f5538004d90 2023-11-24T07:57:53.708 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:53.702+0000 7f554e7fc700 1 -- 172.21.15.59:0/1611202986 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55500cd610 con 0x7f5538003a00 2023-11-24T07:57:53.708 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:53.702+0000 7f554e7fc700 1 -- 172.21.15.59:0/1611202986 --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f553800ba10 con 0x7f5550144c10 2023-11-24T07:57:53.708 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:53.702+0000 7f554ffff700 1 --2- 172.21.15.59:0/1611202986 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5538004d90 0x7f5538005210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:57:53.709 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:53.702+0000 7f554f7fe700 1 --2- 172.21.15.59:0/1611202986 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5538003a00 0x7f5538003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:57:53.709 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:53.703+0000 7f554effd700 1 --2- 172.21.15.59:0/1611202986 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f5550144c10 0x7f5538004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:57:53.709 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:53.703+0000 7f554ffff700 1 -- 172.21.15.59:0/1611202986 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f5550144c10 msgr2=0x7f5538004850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T07:57:53.709 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:53.703+0000 7f554ffff700 1 --2- 172.21.15.59:0/1611202986 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f5550144c10 0x7f5538004850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:53.710 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:53.703+0000 7f554ffff700 1 -- 172.21.15.59:0/1611202986 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5538003a00 msgr2=0x7f5538003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T07:57:53.710 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:53.703+0000 7f554ffff700 1 --2- 172.21.15.59:0/1611202986 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5538003a00 0x7f5538003e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:53.710 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:53.703+0000 7f554ffff700 1 -- 172.21.15.59:0/1611202986 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f55380035b0 con 0x7f5538004d90 2023-11-24T07:57:53.710 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:53.703+0000 7f554ffff700 1 --2- 172.21.15.59:0/1611202986 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5538004d90 0x7f5538005210 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7f554800e170 tx=0x7f5538005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4ee5a74dd3009e13 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:57:53.710 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:53.703+0000 7f554e7fc700 1 -- 172.21.15.59:0/1611202986 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f554800ec90 con 0x7f5538004d90 2023-11-24T07:57:53.710 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:53.703+0000 7f554e7fc700 1 -- 172.21.15.59:0/1611202986 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f554800f040 con 0x7f5538004d90 2023-11-24T07:57:53.711 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:53.703+0000 7f554e7fc700 1 -- 172.21.15.59:0/1611202986 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5548014480 con 0x7f5538004d90 2023-11-24T07:57:53.711 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:53.704+0000 7f5556112700 1 -- 172.21.15.59:0/1611202986 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5538004d90 msgr2=0x7f5538005210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:57:53.711 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:53.704+0000 7f5556112700 1 --2- 172.21.15.59:0/1611202986 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5538004d90 0x7f5538005210 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7f554800e170 tx=0x7f5538005880 comp rx=0 tx=0).stop 2023-11-24T07:57:53.711 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:53.704+0000 7f554ffff700 1 -- 172.21.15.59:0/1611202986 reap_dead start 2023-11-24T07:57:53.711 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:53.704+0000 7f5556112700 1 -- 172.21.15.59:0/1611202986 shutdown_connections 2023-11-24T07:57:53.711 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:53.704+0000 7f5556112700 1 -- 172.21.15.59:0/1611202986 >> 172.21.15.59:0/1611202986 conn(0x7f55500b2740 msgr2=0x7f55500b2b20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:57:53.712 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:53.704+0000 7f5556112700 1 -- 172.21.15.59:0/1611202986 shutdown_connections 2023-11-24T07:57:53.712 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:53.704+0000 7f5556112700 1 -- 172.21.15.59:0/1611202986 wait complete. 2023-11-24T07:57:53.717 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:53.705+0000 7f5556112700 1 Processor -- start 2023-11-24T07:57:53.717 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:53.705+0000 7f5556112700 1 -- start start 2023-11-24T07:57:53.717 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:53.706+0000 7f5556112700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5538004d90 0x7f55500b55f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:57:53.718 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:53.706+0000 7f5556112700 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f5550153770 0x7f5550143c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:57:53.718 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:53.706+0000 7f5556112700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f55500af040 0x7f55500af4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:57:53.718 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:53.706+0000 7f5556112700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55500adc80 con 0x7f5538004d90 2023-11-24T07:57:53.718 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:53.706+0000 7f5556112700 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f555013f040 con 0x7f55500af040 2023-11-24T07:57:53.718 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:53.706+0000 7f5556112700 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f55501441d0 con 0x7f5550153770 2023-11-24T07:57:53.719 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:53.706+0000 7f554f7fe700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5538004d90 0x7f55500b55f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:57:53.719 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:53.706+0000 7f554f7fe700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5538004d90 0x7f55500b55f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:52470/0 (socket says 172.21.15.59:52470) 2023-11-24T07:57:53.719 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:53.706+0000 7f554f7fe700 1 -- 172.21.15.59:0/209641176 learned_addr learned my addr 172.21.15.59:0/209641176 (peer_addr_for_me v2:172.21.15.59:0/0) 2023-11-24T07:57:53.719 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:53.706+0000 7f554effd700 1 --2- 172.21.15.59:0/209641176 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f5550153770 0x7f5550143c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:57:53.719 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:53.706+0000 7f554ffff700 1 --2- 172.21.15.59:0/209641176 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f55500af040 0x7f55500af4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:57:53.719 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:53.706+0000 7f554f7fe700 1 -- 172.21.15.59:0/209641176 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f5550153770 msgr2=0x7f5550143c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:57:53.720 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:53.706+0000 7f554f7fe700 1 --2- 172.21.15.59:0/209641176 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f5550153770 0x7f5550143c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:53.720 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:53.706+0000 7f554f7fe700 1 -- 172.21.15.59:0/209641176 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f55500af040 msgr2=0x7f55500af4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:57:53.720 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:53.706+0000 7f554f7fe700 1 --2- 172.21.15.59:0/209641176 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f55500af040 0x7f55500af4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:53.720 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:53.706+0000 7f554f7fe700 1 -- 172.21.15.59:0/209641176 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f55500b0390 con 0x7f5538004d90 2023-11-24T07:57:53.720 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:53.706+0000 7f554effd700 1 --2- 172.21.15.59:0/209641176 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f5550153770 0x7f5550143c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T07:57:53.721 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:53.706+0000 7f554f7fe700 1 --2- 172.21.15.59:0/209641176 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5538004d90 0x7f55500b55f0 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7f55400116a0 tx=0x7f5540011a60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:57:53.721 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:53.707+0000 7f5537fff700 1 -- 172.21.15.59:0/209641176 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f554000c070 con 0x7f5538004d90 2023-11-24T07:57:53.721 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:53.707+0000 7f5537fff700 1 -- 172.21.15.59:0/209641176 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5540015870 con 0x7f5538004d90 2023-11-24T07:57:53.721 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:53.707+0000 7f5556112700 1 -- 172.21.15.59:0/209641176 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f55500afdf0 con 0x7f5538004d90 2023-11-24T07:57:53.721 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:53.707+0000 7f5537fff700 1 -- 172.21.15.59:0/209641176 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5540015c20 con 0x7f5538004d90 2023-11-24T07:57:53.721 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:53.707+0000 7f5556112700 1 -- 172.21.15.59:0/209641176 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f555015edd0 con 0x7f5538004d90 2023-11-24T07:57:53.722 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:53.708+0000 7f5556112700 1 -- 172.21.15.59:0/209641176 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f555015ec30 con 0x7f5538004d90 2023-11-24T07:57:53.722 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:53.708+0000 7f5537fff700 1 -- 172.21.15.59:0/209641176 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 6) v1 ==== 89282+0+0 (secure 0 0 0) 0x7f5540013400 con 0x7f5538004d90 2023-11-24T07:57:53.722 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:53.708+0000 7f5537fff700 1 --2- 172.21.15.59:0/209641176 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f553c06af90 0x7f553c06d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:57:53.722 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:53.708+0000 7f5537fff700 1 -- 172.21.15.59:0/209641176 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f5540089670 con 0x7f5538004d90 2023-11-24T07:57:53.722 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:53.709+0000 7f554effd700 1 --2- 172.21.15.59:0/209641176 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f553c06af90 0x7f553c06d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:57:53.722 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:53.709+0000 7f554effd700 1 --2- 172.21.15.59:0/209641176 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f553c06af90 0x7f553c06d450 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f5544001050 tx=0x7f5544009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:57:53.723 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:53.713+0000 7f5537fff700 1 -- 172.21.15.59:0/209641176 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f555015ec30 con 0x7f5538004d90 2023-11-24T07:57:53.915 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:53.914+0000 7f5556112700 1 -- 172.21.15.59:0/209641176 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f555015ec30 con 0x7f5538004d90 2023-11-24T07:57:53.916 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:53.915+0000 7f5537fff700 1 -- 172.21.15.59:0/209641176 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v36) v1 ==== 74+0+12914 (secure 0 0 0) 0x7f555015ec30 con 0x7f5538004d90 2023-11-24T07:57:53.916 INFO:teuthology.orchestra.run.smithi059.stdout: 2023-11-24T07:57:53.917 INFO:teuthology.orchestra.run.smithi059.stdout:{"epoch":36,"fsid":"9f0e8741-b8f9-48d5-a2c3-09f45903303d","created":"2023-11-24T07:56:06.552729+0000","modified":"2023-11-24T07:57:52.555242+0000","last_up_change":"2023-11-24T07:56:17.372352+0000","last_in_change":"2023-11-24T07:56:09.073250+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-24T07:56:18.352823+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"6d8f9e9b-1782-4e41-928b-a39f2a8ebc06","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6800","nonce":3373878772},{"type":"v1","addr":"172.21.15.59:6801","nonce":3373878772}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6802","nonce":3373878772},{"type":"v1","addr":"172.21.15.59:6803","nonce":3373878772}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6806","nonce":3373878772},{"type":"v1","addr":"172.21.15.59:6807","nonce":3373878772}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6804","nonce":3373878772},{"type":"v1","addr":"172.21.15.59:6805","nonce":3373878772}]},"public_addr":"172.21.15.59:6801/3373878772","cluster_addr":"172.21.15.59:6803/3373878772","heartbeat_back_addr":"172.21.15.59:6807/3373878772","heartbeat_front_addr":"172.21.15.59:6805/3373878772","state":["exists","up"]},{"osd":1,"uuid":"e8d882aa-52e1-4c04-9e1e-c44ef9204a5f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6824","nonce":648258563},{"type":"v1","addr":"172.21.15.59:6825","nonce":648258563}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6826","nonce":648258563},{"type":"v1","addr":"172.21.15.59:6827","nonce":648258563}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6830","nonce":648258563},{"type":"v1","addr":"172.21.15.59:6831","nonce":648258563}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6828","nonce":648258563},{"type":"v1","addr":"172.21.15.59:6829","nonce":648258563}]},"public_addr":"172.21.15.59:6825/648258563","cluster_addr":"172.21.15.59:6827/648258563","heartbeat_back_addr":"172.21.15.59:6831/648258563","heartbeat_front_addr":"172.21.15.59:6829/648258563","state":["exists","up"]},{"osd":2,"uuid":"69eebfa0-695b-4b21-90b9-8a3537168a01","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6808","nonce":1840430654},{"type":"v1","addr":"172.21.15.59:6809","nonce":1840430654}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6810","nonce":1840430654},{"type":"v1","addr":"172.21.15.59:6811","nonce":1840430654}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6814","nonce":1840430654},{"type":"v1","addr":"172.21.15.59:6815","nonce":1840430654}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6812","nonce":1840430654},{"type":"v1","addr":"172.21.15.59:6813","nonce":1840430654}]},"public_addr":"172.21.15.59:6809/1840430654","cluster_addr":"172.21.15.59:6811/1840430654","heartbeat_back_addr":"172.21.15.59:6815/1840430654","heartbeat_front_addr":"172.21.15.59:6813/1840430654","state":["exists","up"]},{"osd":3,"uuid":"96fd77fc-f911-46ff-9445-6f0be2daa8f4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6816","nonce":2504025850},{"type":"v1","addr":"172.21.15.59:6817","nonce":2504025850}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6818","nonce":2504025850},{"type":"v1","addr":"172.21.15.59:6819","nonce":2504025850}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6822","nonce":2504025850},{"type":"v1","addr":"172.21.15.59:6823","nonce":2504025850}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6820","nonce":2504025850},{"type":"v1","addr":"172.21.15.59:6821","nonce":2504025850}]},"public_addr":"172.21.15.59:6817/2504025850","cluster_addr":"172.21.15.59:6819/2504025850","heartbeat_back_addr":"172.21.15.59:6823/2504025850","heartbeat_front_addr":"172.21.15.59:6821/2504025850","state":["exists","up"]},{"osd":4,"uuid":"7ac29fca-b5b2-4c95-9d69-2f164cbbc708","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6816","nonce":2382463431},{"type":"v1","addr":"172.21.15.76:6818","nonce":2382463431}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6820","nonce":2382463431},{"type":"v1","addr":"172.21.15.76:6822","nonce":2382463431}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6828","nonce":2382463431},{"type":"v1","addr":"172.21.15.76:6830","nonce":2382463431}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6824","nonce":2382463431},{"type":"v1","addr":"172.21.15.76:6826","nonce":2382463431}]},"public_addr":"172.21.15.76:6818/2382463431","cluster_addr":"172.21.15.76:6822/2382463431","heartbeat_back_addr":"172.21.15.76:6830/2382463431","heartbeat_front_addr":"172.21.15.76:6826/2382463431","state":["exists","up"]},{"osd":5,"uuid":"485bf711-3851-4238-a4f4-51d52e0a9c58","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6800","nonce":329334308},{"type":"v1","addr":"172.21.15.76:6802","nonce":329334308}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6805","nonce":329334308},{"type":"v1","addr":"172.21.15.76:6807","nonce":329334308}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6814","nonce":329334308},{"type":"v1","addr":"172.21.15.76:6815","nonce":329334308}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6810","nonce":329334308},{"type":"v1","addr":"172.21.15.76:6812","nonce":329334308}]},"public_addr":"172.21.15.76:6802/329334308","cluster_addr":"172.21.15.76:6807/329334308","heartbeat_back_addr":"172.21.15.76:6815/329334308","heartbeat_front_addr":"172.21.15.76:6812/329334308","state":["exists","up"]},{"osd":6,"uuid":"72b3f2fc-1e4a-4e32-968d-003647a0286f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6801","nonce":3946579410},{"type":"v1","addr":"172.21.15.76:6803","nonce":3946579410}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6804","nonce":3946579410},{"type":"v1","addr":"172.21.15.76:6806","nonce":3946579410}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6811","nonce":3946579410},{"type":"v1","addr":"172.21.15.76:6813","nonce":3946579410}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6808","nonce":3946579410},{"type":"v1","addr":"172.21.15.76:6809","nonce":3946579410}]},"public_addr":"172.21.15.76:6803/3946579410","cluster_addr":"172.21.15.76:6806/3946579410","heartbeat_back_addr":"172.21.15.76:6813/3946579410","heartbeat_front_addr":"172.21.15.76:6809/3946579410","state":["exists","up"]},{"osd":7,"uuid":"1ddceb90-6c5d-4d1a-ad56-99cb877f8c70","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6817","nonce":841761538},{"type":"v1","addr":"172.21.15.76:6819","nonce":841761538}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6821","nonce":841761538},{"type":"v1","addr":"172.21.15.76:6823","nonce":841761538}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6829","nonce":841761538},{"type":"v1","addr":"172.21.15.76:6831","nonce":841761538}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6825","nonce":841761538},{"type":"v1","addr":"172.21.15.76:6827","nonce":841761538}]},"public_addr":"172.21.15.76:6819/841761538","cluster_addr":"172.21.15.76:6823/841761538","heartbeat_back_addr":"172.21.15.76:6831/841761538","heartbeat_front_addr":"172.21.15.76:6827/841761538","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T07:56:14.294838+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T07:56:14.270779+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T07:56:14.231809+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T07:56:14.260983+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T07:56:14.302997+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T07:56:14.258600+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T07:56:14.251413+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T07:56:14.304727+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.59:6835/2290655475":"2023-11-25T07:57:49.543380+0000","172.21.15.59:6833/2290655475":"2023-11-25T07:57:49.543380+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-24T07:57:53.919 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:53.917+0000 7f5556112700 1 -- 172.21.15.59:0/209641176 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f553c06af90 msgr2=0x7f553c06d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:57:53.919 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:53.917+0000 7f5556112700 1 --2- 172.21.15.59:0/209641176 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f553c06af90 0x7f553c06d450 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f5544001050 tx=0x7f5544009040 comp rx=0 tx=0).stop 2023-11-24T07:57:53.919 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:53.917+0000 7f5556112700 1 -- 172.21.15.59:0/209641176 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5538004d90 msgr2=0x7f55500b55f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:57:53.919 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:53.917+0000 7f5556112700 1 --2- 172.21.15.59:0/209641176 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5538004d90 0x7f55500b55f0 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7f55400116a0 tx=0x7f5540011a60 comp rx=0 tx=0).stop 2023-11-24T07:57:53.919 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:53.917+0000 7f5556112700 1 -- 172.21.15.59:0/209641176 shutdown_connections 2023-11-24T07:57:53.919 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:53.917+0000 7f5556112700 1 --2- 172.21.15.59:0/209641176 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f553c06af90 0x7f553c06d450 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:53.919 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:53.917+0000 7f5556112700 1 --2- 172.21.15.59:0/209641176 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f55500af040 0x7f55500af4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:53.920 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:53.917+0000 7f5556112700 1 --2- 172.21.15.59:0/209641176 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5538004d90 0x7f55500b55f0 unknown :-1 s=CLOSED pgs=498 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:53.920 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:53.917+0000 7f5556112700 1 --2- 172.21.15.59:0/209641176 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f5550153770 0x7f5550143c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:53.920 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:53.917+0000 7f5556112700 1 -- 172.21.15.59:0/209641176 >> 172.21.15.59:0/209641176 conn(0x7f55500b2740 msgr2=0x7f55500d0a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:57:53.920 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:53.917+0000 7f5556112700 1 -- 172.21.15.59:0/209641176 shutdown_connections 2023-11-24T07:57:53.920 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:53.917+0000 7f5556112700 1 -- 172.21.15.59:0/209641176 wait complete. 2023-11-24T07:57:53.931 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-24T07:56:18.352823+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-24T07:57:53.931 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-24T07:57:54.059 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:54.057+0000 7f64f4525700 1 Processor -- start 2023-11-24T07:57:54.059 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:54.057+0000 7f64f4525700 1 -- start start 2023-11-24T07:57:54.061 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:54.058+0000 7f64f4525700 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f64ec15bde0 0x7f64ec15c1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:57:54.062 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:54.058+0000 7f64f4525700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f64ec0cf2b0 0x7f64ec0d01b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:57:54.062 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:54.058+0000 7f64f4525700 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f64ec0ce5c0 0x7f64ec0cea50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:57:54.062 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:54.058+0000 7f64f4525700 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f64ec0cef90 con 0x7f64ec0cf2b0 2023-11-24T07:57:54.062 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:54.058+0000 7f64f4525700 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f64ec0d0a90 con 0x7f64ec0ce5c0 2023-11-24T07:57:54.063 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:54.058+0000 7f64f4525700 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f64ec06ee10 con 0x7f64ec15bde0 2023-11-24T07:57:54.063 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:54.058+0000 7f64f1ac0700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f64ec0cf2b0 0x7f64ec0d01b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:57:54.063 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:54.058+0000 7f64f1ac0700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f64ec0cf2b0 0x7f64ec0d01b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:52480/0 (socket says 172.21.15.59:52480) 2023-11-24T07:57:54.063 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:54.058+0000 7f64f1ac0700 1 -- 172.21.15.59:0/2164613038 learned_addr learned my addr 172.21.15.59:0/2164613038 (peer_addr_for_me v2:172.21.15.59:0/0) 2023-11-24T07:57:54.063 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:54.058+0000 7f64f22c1700 1 --2- 172.21.15.59:0/2164613038 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f64ec15bde0 0x7f64ec15c1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:57:54.063 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:54.058+0000 7f64f2ac2700 1 --2- 172.21.15.59:0/2164613038 >> v2:172.21.15.76:3300/0 conn(0x7f64ec0ce5c0 0x7f64ec0cea50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:57:54.063 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:54.058+0000 7f64f1ac0700 1 -- 172.21.15.59:0/2164613038 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f64ec15bde0 msgr2=0x7f64ec15c1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T07:57:54.063 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:54.058+0000 7f64f1ac0700 1 --2- 172.21.15.59:0/2164613038 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f64ec15bde0 0x7f64ec15c1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:54.064 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:54.058+0000 7f64f1ac0700 1 -- 172.21.15.59:0/2164613038 >> v2:172.21.15.76:3300/0 conn(0x7f64ec0ce5c0 msgr2=0x7f64ec0cea50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T07:57:54.064 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:54.058+0000 7f64f1ac0700 1 --2- 172.21.15.59:0/2164613038 >> v2:172.21.15.76:3300/0 conn(0x7f64ec0ce5c0 0x7f64ec0cea50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:54.064 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:54.058+0000 7f64f1ac0700 1 -- 172.21.15.59:0/2164613038 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f64ec06f0f0 con 0x7f64ec0cf2b0 2023-11-24T07:57:54.064 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:54.058+0000 7f64f22c1700 1 --2- 172.21.15.59:0/2164613038 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f64ec15bde0 0x7f64ec15c1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T07:57:54.064 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:54.058+0000 7f64f2ac2700 1 --2- 172.21.15.59:0/2164613038 >> v2:172.21.15.76:3300/0 conn(0x7f64ec0ce5c0 0x7f64ec0cea50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T07:57:54.064 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:54.059+0000 7f64f1ac0700 1 --2- 172.21.15.59:0/2164613038 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f64ec0cf2b0 0x7f64ec0d01b0 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7f64ec15fb60 tx=0x7f64e8009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=20cedbf056580853 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:57:54.064 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:54.059+0000 7f64f12bf700 1 -- 172.21.15.59:0/2164613038 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64e800b870 con 0x7f64ec0cf2b0 2023-11-24T07:57:54.064 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:54.059+0000 7f64f12bf700 1 -- 172.21.15.59:0/2164613038 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f64e800b9d0 con 0x7f64ec0cf2b0 2023-11-24T07:57:54.064 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:54.059+0000 7f64f12bf700 1 -- 172.21.15.59:0/2164613038 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64e800bd60 con 0x7f64ec0cf2b0 2023-11-24T07:57:54.064 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:54.059+0000 7f64f4525700 1 -- 172.21.15.59:0/2164613038 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f64ec0cf2b0 msgr2=0x7f64ec0d01b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:57:54.064 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:54.059+0000 7f64f4525700 1 --2- 172.21.15.59:0/2164613038 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f64ec0cf2b0 0x7f64ec0d01b0 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7f64ec15fb60 tx=0x7f64e8009bd0 comp rx=0 tx=0).stop 2023-11-24T07:57:54.064 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:54.059+0000 7f64f4525700 1 -- 172.21.15.59:0/2164613038 shutdown_connections 2023-11-24T07:57:54.065 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:54.059+0000 7f64f4525700 1 --2- 172.21.15.59:0/2164613038 >> v2:172.21.15.76:3300/0 conn(0x7f64ec0ce5c0 0x7f64ec0cea50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:54.065 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:54.059+0000 7f64f4525700 1 --2- 172.21.15.59:0/2164613038 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f64ec15bde0 0x7f64ec15c1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:54.065 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:54.059+0000 7f64f4525700 1 --2- 172.21.15.59:0/2164613038 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f64ec0cf2b0 0x7f64ec0d01b0 unknown :-1 s=CLOSED pgs=499 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:54.065 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:54.059+0000 7f64f4525700 1 -- 172.21.15.59:0/2164613038 >> 172.21.15.59:0/2164613038 conn(0x7f64ec0b3720 msgr2=0x7f64ec0b3b30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:57:54.065 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:54.059+0000 7f64f4525700 1 -- 172.21.15.59:0/2164613038 shutdown_connections 2023-11-24T07:57:54.065 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:54.059+0000 7f64f4525700 1 -- 172.21.15.59:0/2164613038 wait complete. 2023-11-24T07:57:54.068 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:54.061+0000 7f64f4525700 1 Processor -- start 2023-11-24T07:57:54.068 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:54.061+0000 7f64f4525700 1 -- start start 2023-11-24T07:57:54.068 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:54.062+0000 7f64f4525700 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f64ec0ce5c0 0x7f64ec0b4ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:57:54.068 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:54.062+0000 7f64f4525700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f64ec0cf2b0 0x7f64ec0cb390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:57:54.068 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:54.062+0000 7f64f4525700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f64ec15bde0 0x7f64ec1f1ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:57:54.068 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:54.062+0000 7f64f4525700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64ec0b5170 con 0x7f64ec0cf2b0 2023-11-24T07:57:54.068 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:54.062+0000 7f64f4525700 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64ec0b5450 con 0x7f64ec15bde0 2023-11-24T07:57:54.068 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:54.062+0000 7f64f4525700 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f64ec0b5730 con 0x7f64ec0ce5c0 2023-11-24T07:57:54.068 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:54.062+0000 7f64f1ac0700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f64ec0cf2b0 0x7f64ec0cb390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:57:54.068 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:54.062+0000 7f64f1ac0700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f64ec0cf2b0 0x7f64ec0cb390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:52486/0 (socket says 172.21.15.59:52486) 2023-11-24T07:57:54.068 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:54.062+0000 7f64f1ac0700 1 -- 172.21.15.59:0/3637625041 learned_addr learned my addr 172.21.15.59:0/3637625041 (peer_addr_for_me v2:172.21.15.59:0/0) 2023-11-24T07:57:54.069 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:54.062+0000 7f64f22c1700 1 --2- 172.21.15.59:0/3637625041 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f64ec0ce5c0 0x7f64ec0b4ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:57:54.069 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:54.062+0000 7f64f2ac2700 1 --2- 172.21.15.59:0/3637625041 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f64ec15bde0 0x7f64ec1f1ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:57:54.069 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:54.062+0000 7f64f1ac0700 1 -- 172.21.15.59:0/3637625041 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f64ec0ce5c0 msgr2=0x7f64ec0b4ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:57:54.069 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:54.062+0000 7f64f1ac0700 1 --2- 172.21.15.59:0/3637625041 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f64ec0ce5c0 0x7f64ec0b4ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:54.069 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:54.062+0000 7f64f1ac0700 1 -- 172.21.15.59:0/3637625041 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f64ec15bde0 msgr2=0x7f64ec1f1ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:57:54.069 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:54.062+0000 7f64f1ac0700 1 --2- 172.21.15.59:0/3637625041 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f64ec15bde0 0x7f64ec1f1ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:54.069 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:54.062+0000 7f64f1ac0700 1 -- 172.21.15.59:0/3637625041 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f64e8009550 con 0x7f64ec0cf2b0 2023-11-24T07:57:54.069 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:54.062+0000 7f64f22c1700 1 --2- 172.21.15.59:0/3637625041 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f64ec0ce5c0 0x7f64ec0b4ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-24T07:57:54.069 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:54.063+0000 7f64f1ac0700 1 --2- 172.21.15.59:0/3637625041 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f64ec0cf2b0 0x7f64ec0cb390 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7f64e8013000 tx=0x7f64e8004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:57:54.069 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:54.063+0000 7f64e2ffd700 1 -- 172.21.15.59:0/3637625041 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64e8004310 con 0x7f64ec0cf2b0 2023-11-24T07:57:54.069 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:54.063+0000 7f64e2ffd700 1 -- 172.21.15.59:0/3637625041 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f64e801f070 con 0x7f64ec0cf2b0 2023-11-24T07:57:54.069 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:54.063+0000 7f64f4525700 1 -- 172.21.15.59:0/3637625041 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f64ec0b5a10 con 0x7f64ec0cf2b0 2023-11-24T07:57:54.070 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:54.063+0000 7f64e2ffd700 1 -- 172.21.15.59:0/3637625041 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64e80036a0 con 0x7f64ec0cf2b0 2023-11-24T07:57:54.070 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:54.063+0000 7f64f4525700 1 -- 172.21.15.59:0/3637625041 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f64ec0b1480 con 0x7f64ec0cf2b0 2023-11-24T07:57:54.070 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:54.064+0000 7f64f4525700 1 -- 172.21.15.59:0/3637625041 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f64ec15d420 con 0x7f64ec0cf2b0 2023-11-24T07:57:54.070 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:54.064+0000 7f64e2ffd700 1 -- 172.21.15.59:0/3637625041 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 6) v1 ==== 89282+0+0 (secure 0 0 0) 0x7f64e8021030 con 0x7f64ec0cf2b0 2023-11-24T07:57:54.070 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:54.065+0000 7f64e2ffd700 1 --2- 172.21.15.59:0/3637625041 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f64d006af60 0x7f64d006d420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:57:54.070 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:54.065+0000 7f64e2ffd700 1 -- 172.21.15.59:0/3637625041 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f64e808a790 con 0x7f64ec0cf2b0 2023-11-24T07:57:54.070 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:54.065+0000 7f64f22c1700 1 --2- 172.21.15.59:0/3637625041 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f64d006af60 0x7f64d006d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:57:54.070 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:54.065+0000 7f64f22c1700 1 --2- 172.21.15.59:0/3637625041 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f64d006af60 0x7f64d006d420 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f64dc006fd0 tx=0x7f64dc008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:57:54.071 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:54.069+0000 7f64e2ffd700 1 -- 172.21.15.59:0/3637625041 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f64e8055d00 con 0x7f64ec0cf2b0 2023-11-24T07:57:54.269 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:54.267+0000 7f64f4525700 1 -- 172.21.15.59:0/3637625041 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f64ec15d210 con 0x7f64ec0cf2b0 2023-11-24T07:57:54.270 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:54.268+0000 7f64e2ffd700 1 -- 172.21.15.59:0/3637625041 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v36) v1 ==== 93+0+10 (secure 0 0 0) 0x7f64e80150a0 con 0x7f64ec0cf2b0 2023-11-24T07:57:54.270 INFO:teuthology.orchestra.run.smithi059.stdout:pg_num: 1 2023-11-24T07:57:54.272 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:54.270+0000 7f64f4525700 1 -- 172.21.15.59:0/3637625041 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f64d006af60 msgr2=0x7f64d006d420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:57:54.272 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:54.270+0000 7f64f4525700 1 --2- 172.21.15.59:0/3637625041 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f64d006af60 0x7f64d006d420 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f64dc006fd0 tx=0x7f64dc008040 comp rx=0 tx=0).stop 2023-11-24T07:57:54.272 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:54.270+0000 7f64f4525700 1 -- 172.21.15.59:0/3637625041 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f64ec0cf2b0 msgr2=0x7f64ec0cb390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:57:54.272 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:54.270+0000 7f64f4525700 1 --2- 172.21.15.59:0/3637625041 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f64ec0cf2b0 0x7f64ec0cb390 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7f64e8013000 tx=0x7f64e8004750 comp rx=0 tx=0).stop 2023-11-24T07:57:54.272 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:54.270+0000 7f64f4525700 1 -- 172.21.15.59:0/3637625041 shutdown_connections 2023-11-24T07:57:54.272 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:54.270+0000 7f64f4525700 1 --2- 172.21.15.59:0/3637625041 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f64d006af60 0x7f64d006d420 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:54.272 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:54.270+0000 7f64f4525700 1 --2- 172.21.15.59:0/3637625041 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f64ec15bde0 0x7f64ec1f1ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:54.272 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:54.270+0000 7f64f4525700 1 --2- 172.21.15.59:0/3637625041 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f64ec0ce5c0 0x7f64ec0b4ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:54.272 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:54.270+0000 7f64f4525700 1 --2- 172.21.15.59:0/3637625041 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f64ec0cf2b0 0x7f64ec0cb390 unknown :-1 s=CLOSED pgs=500 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:54.272 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:54.270+0000 7f64f4525700 1 -- 172.21.15.59:0/3637625041 >> 172.21.15.59:0/3637625041 conn(0x7f64ec0b3720 msgr2=0x7f64ec159be0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:57:54.273 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:54.270+0000 7f64f4525700 1 -- 172.21.15.59:0/3637625041 shutdown_connections 2023-11-24T07:57:54.273 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:54.270+0000 7f64f4525700 1 -- 172.21.15.59:0/3637625041 wait complete. 2023-11-24T07:57:54.284 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_volumes.TestSubvolumeSnapshots.test_clean_stale_subvolume_snapshot_metadata' 2023-11-24T07:57:54.409 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:54.407+0000 7f8a14107700 1 Processor -- start 2023-11-24T07:57:54.409 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:54.408+0000 7f8a14107700 1 -- start start 2023-11-24T07:57:54.412 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:54.408+0000 7f8a14107700 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f8a0c0d0ae0 0x7f8a0c06f6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:57:54.412 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:54.408+0000 7f8a14107700 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f8a0c0d0f60 0x7f8a0c06fc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:57:54.412 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:54.408+0000 7f8a14107700 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8a0c0cfd60 con 0x7f8a0c15e000 2023-11-24T07:57:54.412 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:54.408+0000 7f8a14107700 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8a0c0cf4b0 con 0x7f8a0c0d0ae0 2023-11-24T07:57:54.412 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:54.408+0000 7f8a14107700 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8a0c0cf1d0 con 0x7f8a0c0d0f60 2023-11-24T07:57:54.412 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:54.409+0000 7f8a11ea3700 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7f8a0c15e000 0x7f8a0c06eef0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:55966/0 (socket says 172.21.15.59:55966) 2023-11-24T07:57:54.412 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:54.409+0000 7f8a11ea3700 1 -- 172.21.15.59:0/1114526889 learned_addr learned my addr 172.21.15.59:0/1114526889 (peer_addr_for_me v1:172.21.15.59:0/0) 2023-11-24T07:57:54.412 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:54.409+0000 7f8a126a4700 1 --2- 172.21.15.59:0/1114526889 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f8a0c0d0f60 0x7f8a0c06fc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:57:54.413 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:54.409+0000 7f8a116a2700 1 --2- 172.21.15.59:0/1114526889 >> v2:172.21.15.76:3300/0 conn(0x7f8a0c0d0ae0 0x7f8a0c06f6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:57:54.413 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:54.409+0000 7f8a10ea1700 1 -- 172.21.15.59:0/1114526889 <== mon.0 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2464624530 0 0) 0x7f8a0c0cfd60 con 0x7f8a0c15e000 2023-11-24T07:57:54.413 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:54.409+0000 7f8a10ea1700 1 -- 172.21.15.59:0/1114526889 --> v1:172.21.15.59:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f89f80035b0 con 0x7f8a0c15e000 2023-11-24T07:57:54.413 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:54.409+0000 7f8a10ea1700 1 -- 172.21.15.59:0/1114526889 <== mon.0 v1:172.21.15.59:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1305747958 0 0) 0x7f89f80035b0 con 0x7f8a0c15e000 2023-11-24T07:57:54.413 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:54.409+0000 7f8a126a4700 1 -- 172.21.15.59:0/1114526889 >> v2:172.21.15.76:3300/0 conn(0x7f8a0c0d0ae0 msgr2=0x7f8a0c06f6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T07:57:54.413 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:54.409+0000 7f8a126a4700 1 --2- 172.21.15.59:0/1114526889 >> v2:172.21.15.76:3300/0 conn(0x7f8a0c0d0ae0 0x7f8a0c06f6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:54.413 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:54.409+0000 7f8a126a4700 1 -- 172.21.15.59:0/1114526889 >> v1:172.21.15.59:6789/0 conn(0x7f8a0c15e000 legacy=0x7f8a0c06eef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:57:54.413 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:54.409+0000 7f8a126a4700 1 -- 172.21.15.59:0/1114526889 --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8a0c0d0000 con 0x7f8a0c0d0f60 2023-11-24T07:57:54.413 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:54.409+0000 7f8a126a4700 1 --2- 172.21.15.59:0/1114526889 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f8a0c0d0f60 0x7f8a0c06fc30 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f8a04009d90 tx=0x7f8a0400b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b80fc5b946bfde1d server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:57:54.413 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:54.409+0000 7f8a10ea1700 1 -- 172.21.15.59:0/1114526889 <== mon.2 v2:172.21.15.76:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a0400e070 con 0x7f8a0c0d0f60 2023-11-24T07:57:54.413 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:54.410+0000 7f8a10ea1700 1 -- 172.21.15.59:0/1114526889 <== mon.2 v2:172.21.15.76:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8a04003a20 con 0x7f8a0c0d0f60 2023-11-24T07:57:54.414 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:54.410+0000 7f8a10ea1700 1 -- 172.21.15.59:0/1114526889 <== mon.2 v2:172.21.15.76:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a08001670 con 0x7f8a0c0d0f60 2023-11-24T07:57:54.414 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:54.410+0000 7f8a14107700 1 -- 172.21.15.59:0/1114526889 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f8a0c0d0f60 msgr2=0x7f8a0c06fc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:57:54.414 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:54.410+0000 7f8a14107700 1 --2- 172.21.15.59:0/1114526889 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f8a0c0d0f60 0x7f8a0c06fc30 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f8a04009d90 tx=0x7f8a0400b330 comp rx=0 tx=0).stop 2023-11-24T07:57:54.414 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:54.410+0000 7f8a14107700 1 -- 172.21.15.59:0/1114526889 shutdown_connections 2023-11-24T07:57:54.414 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:54.410+0000 7f8a14107700 1 --2- 172.21.15.59:0/1114526889 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f8a0c0d0f60 0x7f8a0c06fc30 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:54.414 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:54.410+0000 7f8a14107700 1 --2- 172.21.15.59:0/1114526889 >> v2:172.21.15.76:3300/0 conn(0x7f8a0c0d0ae0 0x7f8a0c06f6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:54.414 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:54.410+0000 7f8a14107700 1 -- 172.21.15.59:0/1114526889 >> 172.21.15.59:0/1114526889 conn(0x7f8a0c0b3720 msgr2=0x7f8a0c0b3b30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:57:54.414 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:54.410+0000 7f8a14107700 1 -- 172.21.15.59:0/1114526889 shutdown_connections 2023-11-24T07:57:54.414 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:54.410+0000 7f8a14107700 1 -- 172.21.15.59:0/1114526889 wait complete. 2023-11-24T07:57:54.417 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:54.411+0000 7f8a14107700 1 Processor -- start 2023-11-24T07:57:54.417 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:54.411+0000 7f8a14107700 1 -- start start 2023-11-24T07:57:54.417 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:54.412+0000 7f8a14107700 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f8a0c0d0ae0 0x7f8a0c1f4ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:57:54.417 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:54.412+0000 7f8a14107700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8a0c0d0f60 0x7f8a0c1f65f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:57:54.417 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:54.412+0000 7f8a14107700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8a0c15e000 0x7f8a0c1608c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:57:54.417 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:54.412+0000 7f8a14107700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a0c06c0e0 con 0x7f8a0c0d0f60 2023-11-24T07:57:54.418 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:54.412+0000 7f8a14107700 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a0c06c910 con 0x7f8a0c15e000 2023-11-24T07:57:54.418 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:54.412+0000 7f8a14107700 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8a0c161310 con 0x7f8a0c0d0ae0 2023-11-24T07:57:54.418 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:54.412+0000 7f8a116a2700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8a0c0d0f60 0x7f8a0c1f65f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:57:54.418 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:54.412+0000 7f8a116a2700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8a0c0d0f60 0x7f8a0c1f65f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:52498/0 (socket says 172.21.15.59:52498) 2023-11-24T07:57:54.418 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:54.412+0000 7f8a116a2700 1 -- 172.21.15.59:0/504666921 learned_addr learned my addr 172.21.15.59:0/504666921 (peer_addr_for_me v2:172.21.15.59:0/0) 2023-11-24T07:57:54.418 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:54.412+0000 7f8a11ea3700 1 --2- 172.21.15.59:0/504666921 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f8a0c0d0ae0 0x7f8a0c1f4ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:57:54.418 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:54.412+0000 7f8a126a4700 1 --2- 172.21.15.59:0/504666921 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8a0c15e000 0x7f8a0c1608c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:57:54.418 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:54.413+0000 7f8a116a2700 1 -- 172.21.15.59:0/504666921 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f8a0c0d0ae0 msgr2=0x7f8a0c1f4ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:57:54.418 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:54.413+0000 7f8a116a2700 1 --2- 172.21.15.59:0/504666921 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f8a0c0d0ae0 0x7f8a0c1f4ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:54.418 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:54.413+0000 7f8a116a2700 1 -- 172.21.15.59:0/504666921 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8a0c15e000 msgr2=0x7f8a0c1608c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:57:54.418 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:54.413+0000 7f8a116a2700 1 --2- 172.21.15.59:0/504666921 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8a0c15e000 0x7f8a0c1608c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:54.419 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:54.413+0000 7f8a116a2700 1 -- 172.21.15.59:0/504666921 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8a04009a70 con 0x7f8a0c0d0f60 2023-11-24T07:57:54.419 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:54.413+0000 7f8a116a2700 1 --2- 172.21.15.59:0/504666921 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8a0c0d0f60 0x7f8a0c1f65f0 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7f8a0000e8a0 tx=0x7f8a0000ec60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:57:54.419 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:54.413+0000 7f89fe7fc700 1 -- 172.21.15.59:0/504666921 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a00009590 con 0x7f8a0c0d0f60 2023-11-24T07:57:54.419 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:54.413+0000 7f89fe7fc700 1 -- 172.21.15.59:0/504666921 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8a0000eea0 con 0x7f8a0c0d0f60 2023-11-24T07:57:54.419 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:54.413+0000 7f8a14107700 1 -- 172.21.15.59:0/504666921 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8a0c06c2c0 con 0x7f8a0c0d0f60 2023-11-24T07:57:54.419 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:54.413+0000 7f89fe7fc700 1 -- 172.21.15.59:0/504666921 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a0000ca50 con 0x7f8a0c0d0f60 2023-11-24T07:57:54.419 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:54.413+0000 7f8a14107700 1 -- 172.21.15.59:0/504666921 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8a0c1f7bd0 con 0x7f8a0c0d0f60 2023-11-24T07:57:54.419 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:54.414+0000 7f8a14107700 1 -- 172.21.15.59:0/504666921 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8a0c06caa0 con 0x7f8a0c0d0f60 2023-11-24T07:57:54.420 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:54.414+0000 7f89fe7fc700 1 -- 172.21.15.59:0/504666921 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 6) v1 ==== 89282+0+0 (secure 0 0 0) 0x7f8a0000cbb0 con 0x7f8a0c0d0f60 2023-11-24T07:57:54.420 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:54.415+0000 7f89fe7fc700 1 --2- 172.21.15.59:0/504666921 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f89f006af60 0x7f89f006d420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:57:54.420 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:54.415+0000 7f89fe7fc700 1 -- 172.21.15.59:0/504666921 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f8a00089320 con 0x7f8a0c0d0f60 2023-11-24T07:57:54.420 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:54.415+0000 7f8a11ea3700 1 --2- 172.21.15.59:0/504666921 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f89f006af60 0x7f89f006d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:57:54.420 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:54.416+0000 7f8a11ea3700 1 --2- 172.21.15.59:0/504666921 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f89f006af60 0x7f89f006d420 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f8a08001eb0 tx=0x7f8a08001ce0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:57:54.421 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:54.420+0000 7f89fe7fc700 1 -- 172.21.15.59:0/504666921 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f8a00054670 con 0x7f8a0c0d0f60 2023-11-24T07:57:54.705 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:54.704+0000 7f8a14107700 1 -- 172.21.15.59:0/504666921 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_volumes.TestSubvolumeSnapshots.test_clean_stale_subvolume_snapshot_metadata"]} v 0) v1 -- 0x7f8a0c068670 con 0x7f8a0c0d0f60 2023-11-24T07:57:55.566 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:55.564+0000 7f89fe7fc700 1 -- 172.21.15.59:0/504666921 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_volumes.TestSubvolumeSnapshots.test_clean_stale_subvolume_snapshot_metadata"]}]=0 v101) v1 ==== 175+0+0 (secure 0 0 0) 0x7f8a00057c90 con 0x7f8a0c0d0f60 2023-11-24T07:57:55.569 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:55.567+0000 7f8a14107700 1 -- 172.21.15.59:0/504666921 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f89f006af60 msgr2=0x7f89f006d420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:57:55.569 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:55.567+0000 7f8a14107700 1 --2- 172.21.15.59:0/504666921 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f89f006af60 0x7f89f006d420 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f8a08001eb0 tx=0x7f8a08001ce0 comp rx=0 tx=0).stop 2023-11-24T07:57:55.569 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:55.567+0000 7f8a14107700 1 -- 172.21.15.59:0/504666921 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8a0c0d0f60 msgr2=0x7f8a0c1f65f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:57:55.570 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:55.567+0000 7f8a14107700 1 --2- 172.21.15.59:0/504666921 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8a0c0d0f60 0x7f8a0c1f65f0 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7f8a0000e8a0 tx=0x7f8a0000ec60 comp rx=0 tx=0).stop 2023-11-24T07:57:55.570 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:55.567+0000 7f8a14107700 1 -- 172.21.15.59:0/504666921 shutdown_connections 2023-11-24T07:57:55.570 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:55.567+0000 7f8a14107700 1 --2- 172.21.15.59:0/504666921 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f89f006af60 0x7f89f006d420 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:55.570 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:55.567+0000 7f8a14107700 1 --2- 172.21.15.59:0/504666921 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8a0c15e000 0x7f8a0c1608c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:55.570 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:55.567+0000 7f8a14107700 1 --2- 172.21.15.59:0/504666921 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f8a0c0d0ae0 0x7f8a0c1f4ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:55.570 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:55.567+0000 7f8a14107700 1 --2- 172.21.15.59:0/504666921 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8a0c0d0f60 0x7f8a0c1f65f0 unknown :-1 s=CLOSED pgs=502 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:55.570 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:55.567+0000 7f8a14107700 1 -- 172.21.15.59:0/504666921 >> 172.21.15.59:0/504666921 conn(0x7f8a0c0b3720 msgr2=0x7f8a0c0b3d30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:57:55.570 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:55.567+0000 7f8a14107700 1 -- 172.21.15.59:0/504666921 shutdown_connections 2023-11-24T07:57:55.570 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:55.567+0000 7f8a14107700 1 -- 172.21.15.59:0/504666921 wait complete. 2023-11-24T07:57:55.584 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2023-11-24T07:57:55.719 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:55.717+0000 7f018ae42700 1 Processor -- start 2023-11-24T07:57:55.719 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:55.717+0000 7f018ae42700 1 -- start start 2023-11-24T07:57:55.721 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:55.718+0000 7f018ae42700 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f01840ceb00 0x7f01840cfb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:57:55.721 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:55.718+0000 7f018ae42700 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f01840b0c40 0x7f01840d0050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:57:55.721 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:55.718+0000 7f018ae42700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f01840d0590 0x7f0184161e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:57:55.721 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:55.718+0000 7f018ae42700 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f01840aeb30 con 0x7f01840d0590 2023-11-24T07:57:55.721 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:55.718+0000 7f018ae42700 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f018406d9e0 con 0x7f01840ceb00 2023-11-24T07:57:55.721 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:55.718+0000 7f018ae42700 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f018406d420 con 0x7f01840b0c40 2023-11-24T07:57:55.722 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:55.718+0000 7f01893df700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f01840d0590 0x7f0184161e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:57:55.722 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:55.718+0000 7f01893df700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f01840d0590 0x7f0184161e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:52510/0 (socket says 172.21.15.59:52510) 2023-11-24T07:57:55.722 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:55.718+0000 7f01893df700 1 -- 172.21.15.59:0/501949837 learned_addr learned my addr 172.21.15.59:0/501949837 (peer_addr_for_me v2:172.21.15.59:0/0) 2023-11-24T07:57:55.722 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:55.718+0000 7f0188bde700 1 --2- 172.21.15.59:0/501949837 >> v2:172.21.15.76:3300/0 conn(0x7f01840ceb00 0x7f01840cfb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:57:55.722 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:55.718+0000 7f0183fff700 1 --2- 172.21.15.59:0/501949837 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f01840b0c40 0x7f01840d0050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:57:55.722 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:55.718+0000 7f01893df700 1 -- 172.21.15.59:0/501949837 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f01840b0c40 msgr2=0x7f01840d0050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T07:57:55.722 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:55.718+0000 7f01893df700 1 --2- 172.21.15.59:0/501949837 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f01840b0c40 0x7f01840d0050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:55.722 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:55.718+0000 7f01893df700 1 -- 172.21.15.59:0/501949837 >> v2:172.21.15.76:3300/0 conn(0x7f01840ceb00 msgr2=0x7f01840cfb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T07:57:55.722 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:55.718+0000 7f01893df700 1 --2- 172.21.15.59:0/501949837 >> v2:172.21.15.76:3300/0 conn(0x7f01840ceb00 0x7f01840cfb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:55.722 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:55.718+0000 7f01893df700 1 -- 172.21.15.59:0/501949837 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f018406d700 con 0x7f01840d0590 2023-11-24T07:57:55.722 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:55.718+0000 7f0183fff700 1 --2- 172.21.15.59:0/501949837 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f01840b0c40 0x7f01840d0050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T07:57:55.722 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:55.719+0000 7f01893df700 1 --2- 172.21.15.59:0/501949837 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f01840d0590 0x7f0184161e00 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7f018406bf40 tx=0x7f017c00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=80dfae9fb7066857 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:57:55.723 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:55.719+0000 7f01837fe700 1 -- 172.21.15.59:0/501949837 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f017c00e070 con 0x7f01840d0590 2023-11-24T07:57:55.723 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:55.719+0000 7f01837fe700 1 -- 172.21.15.59:0/501949837 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f017c00b9c0 con 0x7f01840d0590 2023-11-24T07:57:55.723 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:55.719+0000 7f01837fe700 1 -- 172.21.15.59:0/501949837 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f017c00bd70 con 0x7f01840d0590 2023-11-24T07:57:55.723 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:55.719+0000 7f018ae42700 1 -- 172.21.15.59:0/501949837 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f01840d0590 msgr2=0x7f0184161e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:57:55.723 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:55.719+0000 7f018ae42700 1 --2- 172.21.15.59:0/501949837 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f01840d0590 0x7f0184161e00 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7f018406bf40 tx=0x7f017c00b330 comp rx=0 tx=0).stop 2023-11-24T07:57:55.723 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:55.719+0000 7f018ae42700 1 -- 172.21.15.59:0/501949837 shutdown_connections 2023-11-24T07:57:55.723 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:55.719+0000 7f018ae42700 1 --2- 172.21.15.59:0/501949837 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f01840d0590 0x7f0184161e00 unknown :-1 s=CLOSED pgs=503 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:55.724 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:55.719+0000 7f018ae42700 1 --2- 172.21.15.59:0/501949837 >> v2:172.21.15.76:3300/0 conn(0x7f01840ceb00 0x7f01840cfb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:55.724 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:55.719+0000 7f018ae42700 1 --2- 172.21.15.59:0/501949837 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f01840b0c40 0x7f01840d0050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:55.724 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:55.719+0000 7f018ae42700 1 -- 172.21.15.59:0/501949837 >> 172.21.15.59:0/501949837 conn(0x7f01840b3720 msgr2=0x7f01840b3b30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:57:55.724 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:55.719+0000 7f018ae42700 1 -- 172.21.15.59:0/501949837 shutdown_connections 2023-11-24T07:57:55.725 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:55.719+0000 7f018ae42700 1 -- 172.21.15.59:0/501949837 wait complete. 2023-11-24T07:57:55.727 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:55.721+0000 7f018ae42700 1 Processor -- start 2023-11-24T07:57:55.727 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:55.721+0000 7f018ae42700 1 -- start start 2023-11-24T07:57:55.727 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:55.722+0000 7f018ae42700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f01840b0c40 0x7f01840c5720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:57:55.728 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:55.722+0000 7f018ae42700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f01840ceb00 0x7f01840c81c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:57:55.728 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:55.722+0000 7f018ae42700 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f01840d0590 0x7f0184150190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:57:55.728 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:55.722+0000 7f018ae42700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01840c7180 con 0x7f01840b0c40 2023-11-24T07:57:55.728 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:55.722+0000 7f018ae42700 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01840c5d00 con 0x7f01840ceb00 2023-11-24T07:57:55.728 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:55.722+0000 7f018ae42700 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f01840c7400 con 0x7f01840d0590 2023-11-24T07:57:55.729 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:55.722+0000 7f0188bde700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f01840b0c40 0x7f01840c5720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:57:55.729 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:55.722+0000 7f0188bde700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f01840b0c40 0x7f01840c5720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:52512/0 (socket says 172.21.15.59:52512) 2023-11-24T07:57:55.729 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:55.722+0000 7f0188bde700 1 -- 172.21.15.59:0/1195126393 learned_addr learned my addr 172.21.15.59:0/1195126393 (peer_addr_for_me v2:172.21.15.59:0/0) 2023-11-24T07:57:55.729 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:55.722+0000 7f0188bde700 1 -- 172.21.15.59:0/1195126393 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f01840d0590 msgr2=0x7f0184150190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:57:55.729 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:55.722+0000 7f0188bde700 1 --2- 172.21.15.59:0/1195126393 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f01840d0590 0x7f0184150190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:55.729 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:55.722+0000 7f0188bde700 1 -- 172.21.15.59:0/1195126393 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f01840ceb00 msgr2=0x7f01840c81c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:57:55.729 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:55.722+0000 7f0188bde700 1 --2- 172.21.15.59:0/1195126393 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f01840ceb00 0x7f01840c81c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:55.729 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:55.722+0000 7f0188bde700 1 -- 172.21.15.59:0/1195126393 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f017c009b30 con 0x7f01840b0c40 2023-11-24T07:57:55.729 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:55.722+0000 7f0188bde700 1 --2- 172.21.15.59:0/1195126393 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f01840b0c40 0x7f01840c5720 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7f017800e810 tx=0x7f017800ebd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:57:55.729 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:55.722+0000 7f01817fa700 1 -- 172.21.15.59:0/1195126393 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0178013610 con 0x7f01840b0c40 2023-11-24T07:57:55.730 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:55.722+0000 7f01817fa700 1 -- 172.21.15.59:0/1195126393 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0178013770 con 0x7f01840b0c40 2023-11-24T07:57:55.730 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:55.722+0000 7f018ae42700 1 -- 172.21.15.59:0/1195126393 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f01840c76e0 con 0x7f01840b0c40 2023-11-24T07:57:55.730 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:55.722+0000 7f01817fa700 1 -- 172.21.15.59:0/1195126393 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0178013b20 con 0x7f01840b0c40 2023-11-24T07:57:55.730 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:55.722+0000 7f018ae42700 1 -- 172.21.15.59:0/1195126393 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f01840c7ca0 con 0x7f01840b0c40 2023-11-24T07:57:55.730 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:55.723+0000 7f018ae42700 1 -- 172.21.15.59:0/1195126393 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f01840d11f0 con 0x7f01840b0c40 2023-11-24T07:57:55.731 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:55.724+0000 7f01817fa700 1 -- 172.21.15.59:0/1195126393 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 6) v1 ==== 89282+0+0 (secure 0 0 0) 0x7f0178013c80 con 0x7f01840b0c40 2023-11-24T07:57:55.731 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:55.724+0000 7f01817fa700 1 --2- 172.21.15.59:0/1195126393 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f016806af60 0x7f016806d420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:57:55.731 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:55.724+0000 7f01817fa700 1 -- 172.21.15.59:0/1195126393 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f01780890c0 con 0x7f01840b0c40 2023-11-24T07:57:55.731 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:55.724+0000 7f0183fff700 1 --2- 172.21.15.59:0/1195126393 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f016806af60 0x7f016806d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:57:55.731 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:55.725+0000 7f0183fff700 1 --2- 172.21.15.59:0/1195126393 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f016806af60 0x7f016806d420 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f0170006fd0 tx=0x7f0170008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:57:55.732 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:55.728+0000 7f01817fa700 1 -- 172.21.15.59:0/1195126393 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f0178054380 con 0x7f01840b0c40 2023-11-24T07:57:55.931 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:55.930+0000 7f018ae42700 1 -- 172.21.15.59:0/1195126393 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7f0184075480 con 0x7f01840b0c40 2023-11-24T07:57:55.934 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:55.933+0000 7f01817fa700 1 -- 172.21.15.59:0/1195126393 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v12)=0 v12) v1 ==== 122+0+0 (secure 0 0 0) 0x7f01780579a0 con 0x7f01840b0c40 2023-11-24T07:57:55.937 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:55.935+0000 7f018ae42700 1 -- 172.21.15.59:0/1195126393 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f016806af60 msgr2=0x7f016806d420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:57:55.937 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:55.935+0000 7f018ae42700 1 --2- 172.21.15.59:0/1195126393 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f016806af60 0x7f016806d420 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f0170006fd0 tx=0x7f0170008040 comp rx=0 tx=0).stop 2023-11-24T07:57:55.937 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:55.935+0000 7f018ae42700 1 -- 172.21.15.59:0/1195126393 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f01840b0c40 msgr2=0x7f01840c5720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:57:55.937 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:55.935+0000 7f018ae42700 1 --2- 172.21.15.59:0/1195126393 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f01840b0c40 0x7f01840c5720 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7f017800e810 tx=0x7f017800ebd0 comp rx=0 tx=0).stop 2023-11-24T07:57:55.938 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:55.935+0000 7f018ae42700 1 -- 172.21.15.59:0/1195126393 shutdown_connections 2023-11-24T07:57:55.938 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:55.935+0000 7f018ae42700 1 --2- 172.21.15.59:0/1195126393 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f016806af60 0x7f016806d420 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:55.938 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:55.935+0000 7f018ae42700 1 --2- 172.21.15.59:0/1195126393 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f01840d0590 0x7f0184150190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:55.938 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:55.935+0000 7f018ae42700 1 --2- 172.21.15.59:0/1195126393 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f01840b0c40 0x7f01840c5720 unknown :-1 s=CLOSED pgs=504 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:55.939 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:55.935+0000 7f018ae42700 1 --2- 172.21.15.59:0/1195126393 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f01840ceb00 0x7f01840c81c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:55.939 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:55.935+0000 7f018ae42700 1 -- 172.21.15.59:0/1195126393 >> 172.21.15.59:0/1195126393 conn(0x7f01840b3720 msgr2=0x7f0184161670 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:57:55.939 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:55.935+0000 7f018ae42700 1 -- 172.21.15.59:0/1195126393 shutdown_connections 2023-11-24T07:57:55.939 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:55.935+0000 7f018ae42700 1 -- 172.21.15.59:0/1195126393 wait complete. 2023-11-24T07:57:55.949 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-11-24T07:57:55.949 DEBUG:teuthology.orchestra.run.smithi059:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-11-24T07:57:55.987 DEBUG:teuthology.orchestra.run.smithi076:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-11-24T07:57:56.034 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-11-24T07:57:56.034 DEBUG:teuthology.orchestra.run.smithi059:> set -ex 2023-11-24T07:57:56.035 DEBUG:teuthology.orchestra.run.smithi059:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-24T07:57:56.053 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=16287364k,nr_inodes=4071841,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 tmpfs /sys/fs/cgroup tmpfs ro,seclabel,nosuid,nodev,noexec,mode=755 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,seclabel,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,seclabel,nosuid,nodev,noexec,relatime,hugetlb 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,seclabel,nosuid,nodev,noexec,relatime,perf_event 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,seclabel,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/rdma cgroup rw,seclabel,nosuid,nodev,noexec,relatime,rdma 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,seclabel,nosuid,nodev,noexec,relatime,freezer 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,seclabel,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,seclabel,nosuid,nodev,noexec,relatime,devices 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,blkio 0 0 none /sys/kernel/tracing tracefs rw,seclabel,relatime 0 0 configfs /sys/kernel/config configfs rw,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=42,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=16075 0 0 mqueue /dev/mqueue mqueue rw,seclabel,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,relatime 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3261172k,mode=700,uid=1000,gid=1265 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw,seclabel 0 0 2023-11-24T07:57:56.053 DEBUG:teuthology.orchestra.run.smithi076:> set -ex 2023-11-24T07:57:56.053 DEBUG:teuthology.orchestra.run.smithi076:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-24T07:57:56.073 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=16284272k,nr_inodes=4071068,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 tmpfs /sys/fs/cgroup tmpfs ro,seclabel,nosuid,nodev,noexec,mode=755 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,seclabel,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,seclabel,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,seclabel,nosuid,nodev,noexec,relatime,freezer 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,seclabel,nosuid,nodev,noexec,relatime,perf_event 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,seclabel,nosuid,nodev,noexec,relatime,hugetlb 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,seclabel,nosuid,nodev,noexec,relatime,devices 0 0 cgroup /sys/fs/cgroup/rdma cgroup rw,seclabel,nosuid,nodev,noexec,relatime,rdma 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,seclabel,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,blkio 0 0 none /sys/kernel/tracing tracefs rw,seclabel,relatime 0 0 configfs /sys/kernel/config configfs rw,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=35,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=20116 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,relatime 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3260552k,mode=700,uid=1000,gid=1265 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw,seclabel 0 0 2023-11-24T07:57:56.073 DEBUG:teuthology.orchestra.run.smithi059:> set -ex 2023-11-24T07:57:56.073 DEBUG:teuthology.orchestra.run.smithi059:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-24T07:57:56.111 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=16287364k,nr_inodes=4071841,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 tmpfs /sys/fs/cgroup tmpfs ro,seclabel,nosuid,nodev,noexec,mode=755 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,seclabel,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,seclabel,nosuid,nodev,noexec,relatime,hugetlb 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,seclabel,nosuid,nodev,noexec,relatime,perf_event 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,seclabel,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/rdma cgroup rw,seclabel,nosuid,nodev,noexec,relatime,rdma 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,seclabel,nosuid,nodev,noexec,relatime,freezer 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,seclabel,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,seclabel,nosuid,nodev,noexec,relatime,devices 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,blkio 0 0 none /sys/kernel/tracing tracefs rw,seclabel,relatime 0 0 configfs /sys/kernel/config configfs rw,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=42,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=16075 0 0 mqueue /dev/mqueue mqueue rw,seclabel,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,relatime 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3261172k,mode=700,uid=1000,gid=1265 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw,seclabel 0 0 2023-11-24T07:57:56.111 DEBUG:teuthology.orchestra.run.smithi076:> set -ex 2023-11-24T07:57:56.111 DEBUG:teuthology.orchestra.run.smithi076:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-24T07:57:56.131 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=16284272k,nr_inodes=4071068,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 tmpfs /sys/fs/cgroup tmpfs ro,seclabel,nosuid,nodev,noexec,mode=755 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,seclabel,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,seclabel,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,seclabel,nosuid,nodev,noexec,relatime,freezer 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,seclabel,nosuid,nodev,noexec,relatime,perf_event 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,seclabel,nosuid,nodev,noexec,relatime,hugetlb 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,seclabel,nosuid,nodev,noexec,relatime,devices 0 0 cgroup /sys/fs/cgroup/rdma cgroup rw,seclabel,nosuid,nodev,noexec,relatime,rdma 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,seclabel,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,blkio 0 0 none /sys/kernel/tracing tracefs rw,seclabel,relatime 0 0 configfs /sys/kernel/config configfs rw,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=35,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=20116 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,relatime 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3260552k,mode=700,uid=1000,gid=1265 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw,seclabel 0 0 2023-11-24T07:57:56.132 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2023-11-24T07:57:56.132 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-24T07:57:56.276 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:56.274+0000 7f32a5732700 1 Processor -- start 2023-11-24T07:57:56.276 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:56.274+0000 7f32a5732700 1 -- start start 2023-11-24T07:57:56.279 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:56.275+0000 7f32a5732700 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f32a015e460 0x7f32a00d49d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:57:56.279 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:56.275+0000 7f32a5732700 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f32a00b3370 con 0x7f32a00d2ba0 2023-11-24T07:57:56.280 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:56.275+0000 7f32a5732700 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f32a00cfc40 con 0x7f32a00cf480 2023-11-24T07:57:56.280 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:56.275+0000 7f32a5732700 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f32a00b35c0 con 0x7f32a015e460 2023-11-24T07:57:56.280 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:56.275+0000 7f329f7fe700 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7f32a00d2ba0 0x7f32a00b40a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:55998/0 (socket says 172.21.15.59:55998) 2023-11-24T07:57:56.280 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:56.275+0000 7f329f7fe700 1 -- 172.21.15.59:0/320689874 learned_addr learned my addr 172.21.15.59:0/320689874 (peer_addr_for_me v1:172.21.15.59:0/0) 2023-11-24T07:57:56.280 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:56.275+0000 7f329effd700 1 --2- 172.21.15.59:0/320689874 >> v2:172.21.15.76:3300/0 conn(0x7f32a015e460 0x7f32a00d49d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:57:56.280 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:56.275+0000 7f329dffb700 1 -- 172.21.15.59:0/320689874 <== mon.0 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2251107850 0 0) 0x7f32a00b3370 con 0x7f32a00d2ba0 2023-11-24T07:57:56.280 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:56.275+0000 7f329dffb700 1 -- 172.21.15.59:0/320689874 --> v1:172.21.15.59:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f32880035b0 con 0x7f32a00d2ba0 2023-11-24T07:57:56.280 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:56.276+0000 7f329dffb700 1 -- 172.21.15.59:0/320689874 <== mon.0 v1:172.21.15.59:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3993411236 0 0) 0x7f32880035b0 con 0x7f32a00d2ba0 2023-11-24T07:57:56.280 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:56.276+0000 7f329dffb700 1 -- 172.21.15.59:0/320689874 >> v2:172.21.15.76:3300/0 conn(0x7f32a015e460 msgr2=0x7f32a00d49d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T07:57:56.280 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:56.276+0000 7f329dffb700 1 --2- 172.21.15.59:0/320689874 >> v2:172.21.15.76:3300/0 conn(0x7f32a015e460 0x7f32a00d49d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:56.280 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:56.276+0000 7f329dffb700 1 -- 172.21.15.59:0/320689874 >> v1:172.21.15.76:6789/0 conn(0x7f32a00cf480 legacy=0x7f32a00d2490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:57:56.281 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:56.276+0000 7f329dffb700 1 -- 172.21.15.59:0/320689874 --> v1:172.21.15.59:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f32a00b38a0 con 0x7f32a00d2ba0 2023-11-24T07:57:56.281 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:56.276+0000 7f329dffb700 1 -- 172.21.15.59:0/320689874 <== mon.0 v1:172.21.15.59:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1312333566 0 0) 0x7f32980019f0 con 0x7f32a00d2ba0 2023-11-24T07:57:56.281 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:56.276+0000 7f329dffb700 1 -- 172.21.15.59:0/320689874 >> v1:172.21.15.59:6789/0 conn(0x7f32a00d2ba0 legacy=0x7f32a00b40a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:57:56.281 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:56.276+0000 7f329dffb700 1 --2- 172.21.15.59:0/320689874 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3288003a00 0x7f3288003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:57:56.281 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:56.276+0000 7f329dffb700 1 --2- 172.21.15.59:0/320689874 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f3288004850 0x7f3288004cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:57:56.281 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:56.276+0000 7f329dffb700 1 --2- 172.21.15.59:0/320689874 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3288005210 0x7f328800ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:57:56.281 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:56.276+0000 7f329dffb700 1 -- 172.21.15.59:0/320689874 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32a00b35c0 con 0x7f3288003a00 2023-11-24T07:57:56.281 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:56.276+0000 7f329dffb700 1 -- 172.21.15.59:0/320689874 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f328800c0c0 con 0x7f3288005210 2023-11-24T07:57:56.281 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:56.276+0000 7f329dffb700 1 -- 172.21.15.59:0/320689874 --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f328800c280 con 0x7f3288004850 2023-11-24T07:57:56.281 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:56.276+0000 7f329effd700 1 --2- 172.21.15.59:0/320689874 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3288003a00 0x7f3288003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:57:56.281 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:56.276+0000 7f329e7fc700 1 --2- 172.21.15.59:0/320689874 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f3288004850 0x7f3288004cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:57:56.281 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:56.276+0000 7f329f7fe700 1 --2- 172.21.15.59:0/320689874 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3288005210 0x7f328800ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:57:56.282 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:56.276+0000 7f329effd700 1 -- 172.21.15.59:0/320689874 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f3288004850 msgr2=0x7f3288004cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T07:57:56.282 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:56.276+0000 7f329effd700 1 --2- 172.21.15.59:0/320689874 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f3288004850 0x7f3288004cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:56.282 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:56.276+0000 7f329effd700 1 -- 172.21.15.59:0/320689874 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3288005210 msgr2=0x7f328800ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T07:57:56.282 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:56.276+0000 7f329effd700 1 --2- 172.21.15.59:0/320689874 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3288005210 0x7f328800ba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:56.282 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:56.276+0000 7f329effd700 1 -- 172.21.15.59:0/320689874 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f32880035b0 con 0x7f3288003a00 2023-11-24T07:57:56.282 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:56.276+0000 7f329f7fe700 1 -- 172.21.15.59:0/320689874 reap_dead start 2023-11-24T07:57:56.282 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:56.276+0000 7f329effd700 1 --2- 172.21.15.59:0/320689874 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3288003a00 0x7f3288003e20 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7f3294009b60 tx=0x7f3288005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9b4ee55ec4305550 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:57:56.282 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:56.276+0000 7f329dffb700 1 -- 172.21.15.59:0/320689874 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f329400e6c0 con 0x7f3288003a00 2023-11-24T07:57:56.282 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:56.276+0000 7f329dffb700 1 -- 172.21.15.59:0/320689874 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f329400e820 con 0x7f3288003a00 2023-11-24T07:57:56.282 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:56.276+0000 7f329dffb700 1 -- 172.21.15.59:0/320689874 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f329400eb90 con 0x7f3288003a00 2023-11-24T07:57:56.282 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:56.277+0000 7f32a5732700 1 -- 172.21.15.59:0/320689874 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3288003a00 msgr2=0x7f3288003e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:57:56.282 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:56.277+0000 7f32a5732700 1 --2- 172.21.15.59:0/320689874 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3288003a00 0x7f3288003e20 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7f3294009b60 tx=0x7f3288005880 comp rx=0 tx=0).stop 2023-11-24T07:57:56.283 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:56.277+0000 7f32a5732700 1 -- 172.21.15.59:0/320689874 shutdown_connections 2023-11-24T07:57:56.283 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:56.277+0000 7f32a5732700 1 --2- 172.21.15.59:0/320689874 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3288003a00 0x7f3288003e20 unknown :-1 s=CLOSED pgs=506 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:56.283 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:56.277+0000 7f32a5732700 1 -- 172.21.15.59:0/320689874 >> 172.21.15.59:0/320689874 conn(0x7f32a00d3eb0 msgr2=0x7f32a00d42c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:57:56.283 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:56.277+0000 7f32a5732700 1 -- 172.21.15.59:0/320689874 shutdown_connections 2023-11-24T07:57:56.283 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:56.277+0000 7f32a5732700 1 -- 172.21.15.59:0/320689874 wait complete. 2023-11-24T07:57:56.285 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:56.278+0000 7f32a5732700 1 Processor -- start 2023-11-24T07:57:56.286 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:56.278+0000 7f32a5732700 1 -- start start 2023-11-24T07:57:56.286 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:56.279+0000 7f32a5732700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3288003a00 0x7f32a00c79e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:57:56.286 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:56.279+0000 7f32a5732700 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f32a01f3650 0x7f32a00c94b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:57:56.286 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:56.279+0000 7f32a5732700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f32a0153000 0x7f32a00c99f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:57:56.286 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:56.279+0000 7f32a5732700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32a0152e10 con 0x7f32a0153000 2023-11-24T07:57:56.286 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:56.279+0000 7f32a5732700 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32a00c8070 con 0x7f3288003a00 2023-11-24T07:57:56.286 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:56.279+0000 7f32a5732700 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f32a00c8350 con 0x7f32a01f3650 2023-11-24T07:57:56.286 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:56.279+0000 7f329f7fe700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f32a0153000 0x7f32a00c99f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:57:56.286 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:56.279+0000 7f329f7fe700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f32a0153000 0x7f32a00c99f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:52536/0 (socket says 172.21.15.59:52536) 2023-11-24T07:57:56.286 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:56.279+0000 7f329f7fe700 1 -- 172.21.15.59:0/3796765503 learned_addr learned my addr 172.21.15.59:0/3796765503 (peer_addr_for_me v2:172.21.15.59:0/0) 2023-11-24T07:57:56.287 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:56.280+0000 7f329effd700 1 --2- 172.21.15.59:0/3796765503 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3288003a00 0x7f32a00c79e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:57:56.287 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:56.280+0000 7f329e7fc700 1 --2- 172.21.15.59:0/3796765503 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f32a01f3650 0x7f32a00c94b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:57:56.287 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:56.280+0000 7f329f7fe700 1 -- 172.21.15.59:0/3796765503 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f32a01f3650 msgr2=0x7f32a00c94b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:57:56.287 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:56.280+0000 7f329f7fe700 1 --2- 172.21.15.59:0/3796765503 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f32a01f3650 0x7f32a00c94b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:56.287 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:56.280+0000 7f329f7fe700 1 -- 172.21.15.59:0/3796765503 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3288003a00 msgr2=0x7f32a00c79e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:57:56.287 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:56.280+0000 7f329f7fe700 1 --2- 172.21.15.59:0/3796765503 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3288003a00 0x7f32a00c79e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:56.287 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:56.280+0000 7f329f7fe700 1 -- 172.21.15.59:0/3796765503 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f32a00c8630 con 0x7f32a0153000 2023-11-24T07:57:56.287 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:56.280+0000 7f329f7fe700 1 --2- 172.21.15.59:0/3796765503 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f32a0153000 0x7f32a00c99f0 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7f32a0161d60 tx=0x7f329800e440 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:57:56.287 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:56.280+0000 7f32877fe700 1 -- 172.21.15.59:0/3796765503 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f329800ebf0 con 0x7f32a0153000 2023-11-24T07:57:56.287 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:56.280+0000 7f32877fe700 1 -- 172.21.15.59:0/3796765503 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f329800ed50 con 0x7f32a0153000 2023-11-24T07:57:56.287 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:56.280+0000 7f32877fe700 1 -- 172.21.15.59:0/3796765503 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3298013660 con 0x7f32a0153000 2023-11-24T07:57:56.287 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:56.280+0000 7f32a5732700 1 -- 172.21.15.59:0/3796765503 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f32a00c8910 con 0x7f32a0153000 2023-11-24T07:57:56.288 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:56.280+0000 7f32a5732700 1 -- 172.21.15.59:0/3796765503 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f32a01537f0 con 0x7f32a0153000 2023-11-24T07:57:56.288 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:56.281+0000 7f32a5732700 1 -- 172.21.15.59:0/3796765503 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f32a00b6ab0 con 0x7f32a0153000 2023-11-24T07:57:56.288 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:56.282+0000 7f32877fe700 1 -- 172.21.15.59:0/3796765503 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 6) v1 ==== 89282+0+0 (secure 0 0 0) 0x7f32980137c0 con 0x7f32a0153000 2023-11-24T07:57:56.288 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:56.282+0000 7f32877fe700 1 --2- 172.21.15.59:0/3796765503 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f328c06af60 0x7f328c06d420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:57:56.288 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:56.282+0000 7f32877fe700 1 -- 172.21.15.59:0/3796765503 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f3298089920 con 0x7f32a0153000 2023-11-24T07:57:56.288 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:56.282+0000 7f329effd700 1 --2- 172.21.15.59:0/3796765503 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f328c06af60 0x7f328c06d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:57:56.288 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:56.283+0000 7f329effd700 1 --2- 172.21.15.59:0/3796765503 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f328c06af60 0x7f328c06d420 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f32940090b0 tx=0x7f3294008030 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:57:56.288 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:56.286+0000 7f32877fe700 1 -- 172.21.15.59:0/3796765503 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f3298054ce0 con 0x7f32a0153000 2023-11-24T07:57:56.533 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:56.531+0000 7f32a5732700 1 -- 172.21.15.59:0/3796765503 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f32a0067f60 con 0x7f32a0153000 2023-11-24T07:57:56.533 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:56.532+0000 7f32877fe700 1 -- 172.21.15.59:0/3796765503 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 19 v19) v1 ==== 94+0+3696 (secure 0 0 0) 0x7f3298058300 con 0x7f32a0153000 2023-11-24T07:57:56.534 INFO:teuthology.orchestra.run.smithi059.stdout: 2023-11-24T07:57:56.534 INFO:teuthology.orchestra.run.smithi059.stdout:{"epoch":19,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4763,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.76:6836/1669441817","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6834","nonce":1669441817},{"type":"v1","addr":"172.21.15.76:6836","nonce":1669441817}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":4769,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.76:6837/3709928230","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6835","nonce":3709928230},{"type":"v1","addr":"172.21.15.76:6837","nonce":3709928230}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":5106,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6834/2019319496","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6832","nonce":2019319496},{"type":"v1","addr":"172.21.15.59:6834","nonce":2019319496}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":5541,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6835/158861207","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6833","nonce":158861207},{"type":"v1","addr":"172.21.15.59:6835","nonce":158861207}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":19}],"filesystems":[]} 2023-11-24T07:57:56.536 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:56.534+0000 7f32a5732700 1 -- 172.21.15.59:0/3796765503 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f328c06af60 msgr2=0x7f328c06d420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:57:56.536 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:56.534+0000 7f32a5732700 1 --2- 172.21.15.59:0/3796765503 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f328c06af60 0x7f328c06d420 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f32940090b0 tx=0x7f3294008030 comp rx=0 tx=0).stop 2023-11-24T07:57:56.536 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:56.534+0000 7f32a5732700 1 -- 172.21.15.59:0/3796765503 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f32a0153000 msgr2=0x7f32a00c99f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:57:56.536 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:56.534+0000 7f32a5732700 1 --2- 172.21.15.59:0/3796765503 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f32a0153000 0x7f32a00c99f0 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7f32a0161d60 tx=0x7f329800e440 comp rx=0 tx=0).stop 2023-11-24T07:57:56.536 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:56.534+0000 7f32a5732700 1 -- 172.21.15.59:0/3796765503 shutdown_connections 2023-11-24T07:57:56.536 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:56.534+0000 7f32a5732700 1 --2- 172.21.15.59:0/3796765503 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f328c06af60 0x7f328c06d420 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:56.536 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:56.534+0000 7f32a5732700 1 --2- 172.21.15.59:0/3796765503 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f32a0153000 0x7f32a00c99f0 unknown :-1 s=CLOSED pgs=507 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:56.536 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:56.534+0000 7f32a5732700 1 --2- 172.21.15.59:0/3796765503 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3288003a00 0x7f32a00c79e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:56.537 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:56.534+0000 7f32a5732700 1 --2- 172.21.15.59:0/3796765503 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f32a01f3650 0x7f32a00c94b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:56.537 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:56.534+0000 7f32a5732700 1 -- 172.21.15.59:0/3796765503 >> 172.21.15.59:0/3796765503 conn(0x7f32a00d3eb0 msgr2=0x7f32a00b0ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:57:56.537 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:56.534+0000 7f32a5732700 1 -- 172.21.15.59:0/3796765503 shutdown_connections 2023-11-24T07:57:56.537 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:56.534+0000 7f32a5732700 1 -- 172.21.15.59:0/3796765503 wait complete. 2023-11-24T07:57:56.537 INFO:teuthology.orchestra.run.smithi059.stderr:dumped fsmap epoch 19 2023-11-24T07:57:56.548 INFO:tasks.ceph.mds.a:Restarting daemon 2023-11-24T07:57:56.549 INFO:tasks.ceph.mds.a:Stopping old one... 2023-11-24T07:57:56.549 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2023-11-24T07:57:56.549 INFO:teuthology.orchestra.run:waiting for 300 2023-11-24T07:57:56.549 INFO:tasks.ceph.mds.c:Restarting daemon 2023-11-24T07:57:56.549 INFO:tasks.ceph.mds.c:Stopping old one... 2023-11-24T07:57:56.549 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2023-11-24T07:57:56.549 INFO:teuthology.orchestra.run:waiting for 300 2023-11-24T07:57:56.550 INFO:tasks.ceph.mds.b:Restarting daemon 2023-11-24T07:57:56.550 INFO:tasks.ceph.mds.b:Stopping old one... 2023-11-24T07:57:56.550 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2023-11-24T07:57:56.550 INFO:teuthology.orchestra.run:waiting for 300 2023-11-24T07:57:56.550 INFO:tasks.ceph.mds.d:Restarting daemon 2023-11-24T07:57:56.550 INFO:tasks.ceph.mds.d:Stopping old one... 2023-11-24T07:57:56.551 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2023-11-24T07:57:56.551 INFO:teuthology.orchestra.run:waiting for 300 2023-11-24T07:57:56.558 INFO:tasks.ceph.mds.a:Stopped 2023-11-24T07:57:56.558 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2023-11-24T07:57:56.559 INFO:tasks.ceph.mds.c:Stopped 2023-11-24T07:57:56.559 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2023-11-24T07:57:56.562 INFO:tasks.ceph.mds.b:Stopped 2023-11-24T07:57:56.562 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2023-11-24T07:57:56.563 INFO:tasks.ceph.mds.d:Stopped 2023-11-24T07:57:56.563 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2023-11-24T07:57:56.601 INFO:tasks.ceph.mds.a:Started 2023-11-24T07:57:56.601 DEBUG:teuthology.parallel:result is None 2023-11-24T07:57:56.602 INFO:tasks.ceph.mds.c:Started 2023-11-24T07:57:56.602 DEBUG:teuthology.parallel:result is None 2023-11-24T07:57:56.605 INFO:tasks.ceph.mds.b:Started 2023-11-24T07:57:56.605 DEBUG:teuthology.parallel:result is None 2023-11-24T07:57:56.606 INFO:tasks.ceph.mds.d:Started 2023-11-24T07:57:56.606 DEBUG:teuthology.parallel:result is None 2023-11-24T07:57:56.609 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2023-11-24T07:57:56.683 INFO:tasks.ceph.mds.a.smithi059.stdout:starting mds.a at 2023-11-24T07:57:56.684 INFO:tasks.ceph.mds.c.smithi059.stdout:starting mds.c at 2023-11-24T07:57:56.696 INFO:tasks.ceph.mds.b.smithi076.stdout:starting mds.b at 2023-11-24T07:57:56.696 INFO:tasks.ceph.mds.d.smithi076.stdout:starting mds.d at 2023-11-24T07:57:56.740 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:56.739+0000 7f99097ce700 1 Processor -- start 2023-11-24T07:57:56.741 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:56.739+0000 7f99097ce700 1 -- start start 2023-11-24T07:57:56.745 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:56.740+0000 7f99097ce700 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f99040affb0 0x7f99040ae120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:57:56.745 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:56.740+0000 7f99097ce700 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f99040d1b40 con 0x7f99040d16e0 2023-11-24T07:57:56.745 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:56.740+0000 7f99097ce700 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9904070490 con 0x7f990406afd0 2023-11-24T07:57:56.745 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:56.740+0000 7f99097ce700 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f99040b0440 con 0x7f99040affb0 2023-11-24T07:57:56.745 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:56.740+0000 7f99027fc700 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7f99040d16e0 0x7f99040ada10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:56038/0 (socket says 172.21.15.59:56038) 2023-11-24T07:57:56.745 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:56.740+0000 7f99027fc700 1 -- 172.21.15.59:0/445215523 learned_addr learned my addr 172.21.15.59:0/445215523 (peer_addr_for_me v1:172.21.15.59:0/0) 2023-11-24T07:57:56.745 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:56.740+0000 7f99037fe700 1 --2- 172.21.15.59:0/445215523 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f99040affb0 0x7f99040ae120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:57:56.746 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:56.740+0000 7f9901ffb700 1 -- 172.21.15.59:0/445215523 <== mon.0 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2047362104 0 0) 0x7f99040d1b40 con 0x7f99040d16e0 2023-11-24T07:57:56.746 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:56.740+0000 7f9901ffb700 1 -- 172.21.15.59:0/445215523 --> v1:172.21.15.59:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f98f00035b0 con 0x7f99040d16e0 2023-11-24T07:57:56.746 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:56.740+0000 7f9901ffb700 1 -- 172.21.15.59:0/445215523 <== mon.0 v1:172.21.15.59:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2248768132 0 0) 0x7f98f00035b0 con 0x7f99040d16e0 2023-11-24T07:57:56.746 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:56.740+0000 7f9901ffb700 1 -- 172.21.15.59:0/445215523 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f99040affb0 msgr2=0x7f99040ae120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T07:57:56.746 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:56.740+0000 7f9901ffb700 1 --2- 172.21.15.59:0/445215523 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f99040affb0 0x7f99040ae120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:56.746 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:56.740+0000 7f9901ffb700 1 -- 172.21.15.59:0/445215523 >> v1:172.21.15.76:6789/0 conn(0x7f990406afd0 legacy=0x7f990406b3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:57:56.746 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:56.740+0000 7f9901ffb700 1 -- 172.21.15.59:0/445215523 --> v1:172.21.15.59:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f99040af9f0 con 0x7f99040d16e0 2023-11-24T07:57:56.746 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:56.740+0000 7f9901ffb700 1 -- 172.21.15.59:0/445215523 <== mon.0 v1:172.21.15.59:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1312333566 0 0) 0x7f98ec001670 con 0x7f99040d16e0 2023-11-24T07:57:56.746 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:56.741+0000 7f9901ffb700 1 -- 172.21.15.59:0/445215523 >> v1:172.21.15.59:6789/0 conn(0x7f99040d16e0 legacy=0x7f99040ada10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:57:56.746 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:56.741+0000 7f9901ffb700 1 --2- 172.21.15.59:0/445215523 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f99040affb0 0x7f98f0003a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:57:56.747 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:56.741+0000 7f9901ffb700 1 --2- 172.21.15.59:0/445215523 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f98f0003f70 0x7f98f0004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:57:56.747 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:56.741+0000 7f9901ffb700 1 --2- 172.21.15.59:0/445215523 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f98f0004d90 0x7f98f0005210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:57:56.747 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:56.741+0000 7f9901ffb700 1 -- 172.21.15.59:0/445215523 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99040b0440 con 0x7f98f0004d90 2023-11-24T07:57:56.747 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:56.741+0000 7f9901ffb700 1 -- 172.21.15.59:0/445215523 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98f000ba70 con 0x7f98f0003f70 2023-11-24T07:57:56.747 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:56.741+0000 7f9901ffb700 1 -- 172.21.15.59:0/445215523 --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f98f000bc30 con 0x7f99040affb0 2023-11-24T07:57:56.747 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:56.741+0000 7f99037fe700 1 --2- 172.21.15.59:0/445215523 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f98f0004d90 0x7f98f0005210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:57:56.747 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:56.741+0000 7f99027fc700 1 --2- 172.21.15.59:0/445215523 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f98f0003f70 0x7f98f0004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:57:56.747 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:56.741+0000 7f9902ffd700 1 --2- 172.21.15.59:0/445215523 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f99040affb0 0x7f98f0003a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:57:56.747 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:56.741+0000 7f99037fe700 1 -- 172.21.15.59:0/445215523 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f99040affb0 msgr2=0x7f98f0003a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T07:57:56.748 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:56.741+0000 7f99037fe700 1 --2- 172.21.15.59:0/445215523 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f99040affb0 0x7f98f0003a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:56.748 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:56.741+0000 7f99037fe700 1 -- 172.21.15.59:0/445215523 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f98f0003f70 msgr2=0x7f98f0004850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T07:57:56.748 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:56.741+0000 7f99037fe700 1 --2- 172.21.15.59:0/445215523 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f98f0003f70 0x7f98f0004850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:56.748 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:56.741+0000 7f99037fe700 1 -- 172.21.15.59:0/445215523 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f98f00035b0 con 0x7f98f0004d90 2023-11-24T07:57:56.748 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:56.741+0000 7f99027fc700 1 --2- 172.21.15.59:0/445215523 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f98f0003f70 0x7f98f0004850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-24T07:57:56.748 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:56.741+0000 7f9902ffd700 1 --2- 172.21.15.59:0/445215523 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f99040affb0 0x7f98f0003a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-24T07:57:56.748 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:56.741+0000 7f99037fe700 1 --2- 172.21.15.59:0/445215523 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f98f0004d90 0x7f98f0005210 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7f98f4009b60 tx=0x7f98f0005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=51b0807eeec77c1a server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:57:56.748 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:56.741+0000 7f9901ffb700 1 -- 172.21.15.59:0/445215523 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98f400e6c0 con 0x7f98f0004d90 2023-11-24T07:57:56.748 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:56.741+0000 7f9901ffb700 1 -- 172.21.15.59:0/445215523 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f98f400e820 con 0x7f98f0004d90 2023-11-24T07:57:56.748 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:56.741+0000 7f9901ffb700 1 -- 172.21.15.59:0/445215523 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98f400eb90 con 0x7f98f0004d90 2023-11-24T07:57:56.748 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:56.742+0000 7f99097ce700 1 -- 172.21.15.59:0/445215523 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f98f0004d90 msgr2=0x7f98f0005210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:57:56.749 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:56.742+0000 7f99097ce700 1 --2- 172.21.15.59:0/445215523 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f98f0004d90 0x7f98f0005210 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7f98f4009b60 tx=0x7f98f0005880 comp rx=0 tx=0).stop 2023-11-24T07:57:56.749 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:56.742+0000 7f99037fe700 1 -- 172.21.15.59:0/445215523 reap_dead start 2023-11-24T07:57:56.749 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:56.742+0000 7f99097ce700 1 -- 172.21.15.59:0/445215523 shutdown_connections 2023-11-24T07:57:56.749 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:56.742+0000 7f99097ce700 1 -- 172.21.15.59:0/445215523 >> 172.21.15.59:0/445215523 conn(0x7f99040d0bc0 msgr2=0x7f99040d0fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:57:56.749 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:56.742+0000 7f99097ce700 1 -- 172.21.15.59:0/445215523 shutdown_connections 2023-11-24T07:57:56.749 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:56.742+0000 7f99097ce700 1 -- 172.21.15.59:0/445215523 wait complete. 2023-11-24T07:57:56.752 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:56.744+0000 7f99097ce700 1 Processor -- start 2023-11-24T07:57:56.752 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:56.744+0000 7f99097ce700 1 -- start start 2023-11-24T07:57:56.752 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:56.745+0000 7f99097ce700 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f98f0004d90 0x7f9904142a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:57:56.752 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:56.745+0000 7f99097ce700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f99040cdfe0 0x7f9904159a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:57:56.752 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:56.745+0000 7f99097ce700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f99040ce460 0x7f9904159f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:57:56.752 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:56.745+0000 7f99097ce700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9904143640 con 0x7f99040cdfe0 2023-11-24T07:57:56.752 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:56.745+0000 7f99097ce700 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9904143310 con 0x7f99040ce460 2023-11-24T07:57:56.753 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:56.745+0000 7f99097ce700 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9904142f60 con 0x7f98f0004d90 2023-11-24T07:57:56.753 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:56.745+0000 7f99027fc700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f99040cdfe0 0x7f9904159a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:57:56.753 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:56.745+0000 7f99027fc700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f99040cdfe0 0x7f9904159a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:52576/0 (socket says 172.21.15.59:52576) 2023-11-24T07:57:56.753 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:56.745+0000 7f99027fc700 1 -- 172.21.15.59:0/1937454007 learned_addr learned my addr 172.21.15.59:0/1937454007 (peer_addr_for_me v2:172.21.15.59:0/0) 2023-11-24T07:57:56.753 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:56.745+0000 7f9902ffd700 1 --2- 172.21.15.59:0/1937454007 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f98f0004d90 0x7f9904142a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:57:56.753 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:56.745+0000 7f99037fe700 1 --2- 172.21.15.59:0/1937454007 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f99040ce460 0x7f9904159f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:57:56.753 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:56.745+0000 7f99027fc700 1 -- 172.21.15.59:0/1937454007 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f98f0004d90 msgr2=0x7f9904142a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:57:56.753 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:56.745+0000 7f99027fc700 1 --2- 172.21.15.59:0/1937454007 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f98f0004d90 0x7f9904142a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:56.753 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:56.745+0000 7f99027fc700 1 -- 172.21.15.59:0/1937454007 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f99040ce460 msgr2=0x7f9904159f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:57:56.753 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:56.745+0000 7f99027fc700 1 --2- 172.21.15.59:0/1937454007 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f99040ce460 0x7f9904159f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:56.754 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:56.745+0000 7f99027fc700 1 -- 172.21.15.59:0/1937454007 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f99040ce9f0 con 0x7f99040cdfe0 2023-11-24T07:57:56.754 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:56.745+0000 7f99027fc700 1 --2- 172.21.15.59:0/1937454007 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f99040cdfe0 0x7f9904159a50 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7f98ec007bf0 tx=0x7f98ec0078e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:57:56.754 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:56.745+0000 7f98fb7fe700 1 -- 172.21.15.59:0/1937454007 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98ec00f7c0 con 0x7f99040cdfe0 2023-11-24T07:57:56.754 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:56.745+0000 7f98fb7fe700 1 -- 172.21.15.59:0/1937454007 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f98ec00f920 con 0x7f99040cdfe0 2023-11-24T07:57:56.754 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:56.746+0000 7f98fb7fe700 1 -- 172.21.15.59:0/1937454007 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98ec00fc90 con 0x7f99040cdfe0 2023-11-24T07:57:56.754 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:56.746+0000 7f99097ce700 1 -- 172.21.15.59:0/1937454007 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f99040cecd0 con 0x7f99040cdfe0 2023-11-24T07:57:56.754 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:56.746+0000 7f99097ce700 1 -- 172.21.15.59:0/1937454007 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9904156a80 con 0x7f99040cdfe0 2023-11-24T07:57:56.754 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:56.746+0000 7f99097ce700 1 -- 172.21.15.59:0/1937454007 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f990404f3b0 con 0x7f99040cdfe0 2023-11-24T07:57:56.754 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:56.747+0000 7f98fb7fe700 1 -- 172.21.15.59:0/1937454007 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 6) v1 ==== 89282+0+0 (secure 0 0 0) 0x7f98ec01d460 con 0x7f99040cdfe0 2023-11-24T07:57:56.754 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:56.747+0000 7f98fb7fe700 1 --2- 172.21.15.59:0/1937454007 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f98e406af60 0x7f98e406d420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:57:56.755 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:56.747+0000 7f98fb7fe700 1 -- 172.21.15.59:0/1937454007 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f98ec089300 con 0x7f99040cdfe0 2023-11-24T07:57:56.755 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:56.748+0000 7f9902ffd700 1 --2- 172.21.15.59:0/1937454007 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f98e406af60 0x7f98e406d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:57:56.755 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:56.748+0000 7f9902ffd700 1 --2- 172.21.15.59:0/1937454007 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f98e406af60 0x7f98e406d420 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f9904156fe0 tx=0x7f98fc009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:57:56.755 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:56.752+0000 7f98fb7fe700 1 -- 172.21.15.59:0/1937454007 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f98ec054800 con 0x7f99040cdfe0 2023-11-24T07:57:56.969 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:56.967+0000 7f99097ce700 1 -- 172.21.15.59:0/1937454007 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7f9904068670 con 0x7f99040cdfe0 2023-11-24T07:57:58.335 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:58.334+0000 7f98fb7fe700 1 -- 172.21.15.59:0/1937454007 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v41) v1 ==== 97+0+0 (secure 0 0 0) 0x7f98ec00fdf0 con 0x7f99040cdfe0 2023-11-24T07:57:58.339 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:58.338+0000 7f99097ce700 1 -- 172.21.15.59:0/1937454007 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f98e406af60 msgr2=0x7f98e406d420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:57:58.340 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:58.338+0000 7f99097ce700 1 --2- 172.21.15.59:0/1937454007 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f98e406af60 0x7f98e406d420 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f9904156fe0 tx=0x7f98fc009040 comp rx=0 tx=0).stop 2023-11-24T07:57:58.340 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:58.338+0000 7f99097ce700 1 -- 172.21.15.59:0/1937454007 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f99040cdfe0 msgr2=0x7f9904159a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:57:58.340 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:58.338+0000 7f99097ce700 1 --2- 172.21.15.59:0/1937454007 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f99040cdfe0 0x7f9904159a50 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7f98ec007bf0 tx=0x7f98ec0078e0 comp rx=0 tx=0).stop 2023-11-24T07:57:58.340 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:58.338+0000 7f99097ce700 1 -- 172.21.15.59:0/1937454007 shutdown_connections 2023-11-24T07:57:58.340 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:58.338+0000 7f99097ce700 1 --2- 172.21.15.59:0/1937454007 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f98e406af60 0x7f98e406d420 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:58.340 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:58.338+0000 7f99097ce700 1 --2- 172.21.15.59:0/1937454007 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f99040ce460 0x7f9904159f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:58.340 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:58.338+0000 7f99097ce700 1 --2- 172.21.15.59:0/1937454007 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f98f0004d90 0x7f9904142a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:58.340 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:58.338+0000 7f99097ce700 1 --2- 172.21.15.59:0/1937454007 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f99040cdfe0 0x7f9904159a50 unknown :-1 s=CLOSED pgs=516 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:58.340 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:58.338+0000 7f99097ce700 1 -- 172.21.15.59:0/1937454007 >> 172.21.15.59:0/1937454007 conn(0x7f99040d0bc0 msgr2=0x7f990406b3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:57:58.341 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:58.338+0000 7f99097ce700 1 -- 172.21.15.59:0/1937454007 shutdown_connections 2023-11-24T07:57:58.341 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:58.338+0000 7f99097ce700 1 -- 172.21.15.59:0/1937454007 wait complete. 2023-11-24T07:57:58.341 INFO:teuthology.orchestra.run.smithi059.stderr: removed all blocklist entries 2023-11-24T07:57:58.355 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2023-11-24T07:57:58.487 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:58.485+0000 7f0d3df21700 1 Processor -- start 2023-11-24T07:57:58.487 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:58.485+0000 7f0d3df21700 1 -- start start 2023-11-24T07:57:58.489 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:58.486+0000 7f0d3df21700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f0d380d49d0 0x7f0d380d2490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:57:58.489 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:58.486+0000 7f0d3df21700 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f0d380d29d0 0x7f0d380b40a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:57:58.489 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:58.486+0000 7f0d3df21700 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0d380d4e50 con 0x7f0d3815e460 2023-11-24T07:57:58.489 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:58.486+0000 7f0d3df21700 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0d380d2e40 con 0x7f0d380d49d0 2023-11-24T07:57:58.489 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:58.486+0000 7f0d3df21700 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0d380b3370 con 0x7f0d380d29d0 2023-11-24T07:57:58.490 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:58.486+0000 7f0d36ffd700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f0d380d49d0 0x7f0d380d2490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:57:58.490 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:58.486+0000 7f0d377fe700 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7f0d3815e460 0x7f0d380cf480 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:56060/0 (socket says 172.21.15.59:56060) 2023-11-24T07:57:58.490 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:58.486+0000 7f0d377fe700 1 -- 172.21.15.59:0/3925928951 learned_addr learned my addr 172.21.15.59:0/3925928951 (peer_addr_for_me v1:172.21.15.59:0/0) 2023-11-24T07:57:58.490 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:58.486+0000 7f0d37fff700 1 --2- 172.21.15.59:0/3925928951 >> v2:172.21.15.76:3300/0 conn(0x7f0d380d29d0 0x7f0d380b40a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:57:58.490 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:58.486+0000 7f0d36ffd700 1 -- 172.21.15.59:0/3925928951 >> v2:172.21.15.76:3300/0 conn(0x7f0d380d29d0 msgr2=0x7f0d380b40a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T07:57:58.490 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:58.486+0000 7f0d36ffd700 1 --2- 172.21.15.59:0/3925928951 >> v2:172.21.15.76:3300/0 conn(0x7f0d380d29d0 0x7f0d380b40a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:58.490 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:58.486+0000 7f0d36ffd700 1 -- 172.21.15.59:0/3925928951 >> v1:172.21.15.59:6789/0 conn(0x7f0d3815e460 legacy=0x7f0d380cf480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:57:58.490 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:58.486+0000 7f0d367fc700 1 -- 172.21.15.59:0/3925928951 <== mon.0 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3476725229 0 0) 0x7f0d380d4e50 con 0x7f0d3815e460 2023-11-24T07:57:58.490 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:58.486+0000 7f0d36ffd700 1 -- 172.21.15.59:0/3925928951 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0d380cfc40 con 0x7f0d380d49d0 2023-11-24T07:57:58.490 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:58.487+0000 7f0d36ffd700 1 --2- 172.21.15.59:0/3925928951 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0d380d49d0 0x7f0d380d2490 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7f0d3808d240 tx=0x7f0d20009b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3ef621aca66206e4 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:57:58.490 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:58.487+0000 7f0d367fc700 1 -- 172.21.15.59:0/3925928951 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d20011830 con 0x7f0d380d49d0 2023-11-24T07:57:58.490 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:58.487+0000 7f0d367fc700 1 -- 172.21.15.59:0/3925928951 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0d20011990 con 0x7f0d380d49d0 2023-11-24T07:57:58.491 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:58.487+0000 7f0d367fc700 1 -- 172.21.15.59:0/3925928951 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d20011d20 con 0x7f0d380d49d0 2023-11-24T07:57:58.491 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:58.487+0000 7f0d3df21700 1 -- 172.21.15.59:0/3925928951 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0d380d49d0 msgr2=0x7f0d380d2490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:57:58.491 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:58.487+0000 7f0d3df21700 1 --2- 172.21.15.59:0/3925928951 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0d380d49d0 0x7f0d380d2490 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7f0d3808d240 tx=0x7f0d20009b20 comp rx=0 tx=0).stop 2023-11-24T07:57:58.491 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:58.487+0000 7f0d3df21700 1 -- 172.21.15.59:0/3925928951 shutdown_connections 2023-11-24T07:57:58.491 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:58.487+0000 7f0d3df21700 1 --2- 172.21.15.59:0/3925928951 >> v2:172.21.15.76:3300/0 conn(0x7f0d380d29d0 0x7f0d380b40a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:58.491 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:58.487+0000 7f0d3df21700 1 --2- 172.21.15.59:0/3925928951 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0d380d49d0 0x7f0d380d2490 unknown :-1 s=CLOSED pgs=518 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:58.491 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:58.487+0000 7f0d3df21700 1 -- 172.21.15.59:0/3925928951 >> 172.21.15.59:0/3925928951 conn(0x7f0d380d3eb0 msgr2=0x7f0d380d42c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:57:58.491 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:58.487+0000 7f0d3df21700 1 -- 172.21.15.59:0/3925928951 shutdown_connections 2023-11-24T07:57:58.491 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:58.487+0000 7f0d3df21700 1 -- 172.21.15.59:0/3925928951 wait complete. 2023-11-24T07:57:58.494 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:58.489+0000 7f0d3df21700 1 Processor -- start 2023-11-24T07:57:58.494 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:58.489+0000 7f0d3df21700 1 -- start start 2023-11-24T07:57:58.494 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:58.489+0000 7f0d3df21700 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f0d380d29d0 0x7f0d380cb990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:57:58.494 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:58.489+0000 7f0d3df21700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0d380d49d0 0x7f0d381580d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:57:58.494 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:58.490+0000 7f0d3df21700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0d3815e460 0x7f0d380c99c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:57:58.494 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:58.490+0000 7f0d3df21700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d380ccb80 con 0x7f0d3815e460 2023-11-24T07:57:58.494 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:58.490+0000 7f0d3df21700 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d380cbf70 con 0x7f0d380d49d0 2023-11-24T07:57:58.494 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:58.490+0000 7f0d3df21700 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0d380cc250 con 0x7f0d380d29d0 2023-11-24T07:57:58.495 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:58.490+0000 7f0d37fff700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0d3815e460 0x7f0d380c99c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:57:58.495 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:58.490+0000 7f0d37fff700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0d3815e460 0x7f0d380c99c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:52596/0 (socket says 172.21.15.59:52596) 2023-11-24T07:57:58.495 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:58.490+0000 7f0d37fff700 1 -- 172.21.15.59:0/4059433949 learned_addr learned my addr 172.21.15.59:0/4059433949 (peer_addr_for_me v2:172.21.15.59:0/0) 2023-11-24T07:57:58.495 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:58.490+0000 7f0d36ffd700 1 --2- 172.21.15.59:0/4059433949 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0d380d49d0 0x7f0d381580d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:57:58.495 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:58.490+0000 7f0d377fe700 1 --2- 172.21.15.59:0/4059433949 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f0d380d29d0 0x7f0d380cb990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:57:58.495 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:58.490+0000 7f0d37fff700 1 -- 172.21.15.59:0/4059433949 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f0d380d29d0 msgr2=0x7f0d380cb990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:57:58.495 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:58.490+0000 7f0d37fff700 1 --2- 172.21.15.59:0/4059433949 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f0d380d29d0 0x7f0d380cb990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:58.495 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:58.490+0000 7f0d37fff700 1 -- 172.21.15.59:0/4059433949 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0d380d49d0 msgr2=0x7f0d381580d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:57:58.495 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:58.490+0000 7f0d37fff700 1 --2- 172.21.15.59:0/4059433949 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0d380d49d0 0x7f0d381580d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:58.495 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:58.490+0000 7f0d37fff700 1 -- 172.21.15.59:0/4059433949 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0d200094a0 con 0x7f0d3815e460 2023-11-24T07:57:58.495 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:58.490+0000 7f0d36ffd700 1 --2- 172.21.15.59:0/4059433949 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0d380d49d0 0x7f0d381580d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T07:57:58.495 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:58.490+0000 7f0d37fff700 1 --2- 172.21.15.59:0/4059433949 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0d3815e460 0x7f0d380c99c0 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7f0d30007e40 tx=0x7f0d3000e3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:57:58.496 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:58.490+0000 7f0d2ffff700 1 -- 172.21.15.59:0/4059433949 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d3000eea0 con 0x7f0d3815e460 2023-11-24T07:57:58.496 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:58.490+0000 7f0d2ffff700 1 -- 172.21.15.59:0/4059433949 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0d30014420 con 0x7f0d3815e460 2023-11-24T07:57:58.496 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:58.490+0000 7f0d2ffff700 1 -- 172.21.15.59:0/4059433949 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d300147b0 con 0x7f0d3815e460 2023-11-24T07:57:58.496 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:58.490+0000 7f0d3df21700 1 -- 172.21.15.59:0/4059433949 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0d380cc530 con 0x7f0d3815e460 2023-11-24T07:57:58.496 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:58.490+0000 7f0d3df21700 1 -- 172.21.15.59:0/4059433949 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0d380c9f00 con 0x7f0d3815e460 2023-11-24T07:57:58.496 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:58.491+0000 7f0d3df21700 1 -- 172.21.15.59:0/4059433949 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0d380b45c0 con 0x7f0d3815e460 2023-11-24T07:57:58.496 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:58.492+0000 7f0d2ffff700 1 -- 172.21.15.59:0/4059433949 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 6) v1 ==== 89282+0+0 (secure 0 0 0) 0x7f0d30014910 con 0x7f0d3815e460 2023-11-24T07:57:58.496 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:58.492+0000 7f0d2ffff700 1 --2- 172.21.15.59:0/4059433949 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f0d1806af60 0x7f0d1806d420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:57:58.496 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:58.492+0000 7f0d2ffff700 1 -- 172.21.15.59:0/4059433949 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f0d30089aa0 con 0x7f0d3815e460 2023-11-24T07:57:58.496 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:58.492+0000 7f0d377fe700 1 --2- 172.21.15.59:0/4059433949 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f0d1806af60 0x7f0d1806d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:57:58.497 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:58.493+0000 7f0d377fe700 1 --2- 172.21.15.59:0/4059433949 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f0d1806af60 0x7f0d1806d420 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f0d28000f60 tx=0x7f0d28009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:57:58.498 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:58.497+0000 7f0d2ffff700 1 -- 172.21.15.59:0/4059433949 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f0d380b45c0 con 0x7f0d3815e460 2023-11-24T07:57:58.780 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:58.778+0000 7f0d3df21700 1 -- 172.21.15.59:0/4059433949 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7f0d380b45c0 con 0x7f0d3815e460 2023-11-24T07:57:58.780 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:58.778+0000 7f0d2ffff700 1 -- 172.21.15.59:0/4059433949 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v13) v1 ==== 80+0+5304 (secure 0 0 0) 0x7f0d380b45c0 con 0x7f0d3815e460 2023-11-24T07:57:58.780 INFO:teuthology.orchestra.run.smithi059.stdout: 2023-11-24T07:57:58.781 INFO:teuthology.orchestra.run.smithi059.stdout:{ 2023-11-24T07:57:58.781 INFO:teuthology.orchestra.run.smithi059.stdout: "auth_dump": [ 2023-11-24T07:57:58.781 INFO:teuthology.orchestra.run.smithi059.stdout: { 2023-11-24T07:57:58.781 INFO:teuthology.orchestra.run.smithi059.stdout: "entity": "mds.a", 2023-11-24T07:57:58.781 INFO:teuthology.orchestra.run.smithi059.stdout: "key": "AQD0VmBlWGLCKBAASbEQYAkExr8TButtK4KHQQ==", 2023-11-24T07:57:58.781 INFO:teuthology.orchestra.run.smithi059.stdout: "caps": { 2023-11-24T07:57:58.781 INFO:teuthology.orchestra.run.smithi059.stdout: "mds": "allow", 2023-11-24T07:57:58.781 INFO:teuthology.orchestra.run.smithi059.stdout: "mgr": "allow *", 2023-11-24T07:57:58.781 INFO:teuthology.orchestra.run.smithi059.stdout: "mon": "allow *", 2023-11-24T07:57:58.782 INFO:teuthology.orchestra.run.smithi059.stdout: "osd": "allow *" 2023-11-24T07:57:58.782 INFO:teuthology.orchestra.run.smithi059.stdout: } 2023-11-24T07:57:58.782 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-11-24T07:57:58.782 INFO:teuthology.orchestra.run.smithi059.stdout: { 2023-11-24T07:57:58.782 INFO:teuthology.orchestra.run.smithi059.stdout: "entity": "mds.b", 2023-11-24T07:57:58.782 INFO:teuthology.orchestra.run.smithi059.stdout: "key": "AQD0VmBlDIS0NRAAYQ3wTunU7m/Qk1qCM6Mcvg==", 2023-11-24T07:57:58.782 INFO:teuthology.orchestra.run.smithi059.stdout: "caps": { 2023-11-24T07:57:58.782 INFO:teuthology.orchestra.run.smithi059.stdout: "mds": "allow", 2023-11-24T07:57:58.782 INFO:teuthology.orchestra.run.smithi059.stdout: "mgr": "allow *", 2023-11-24T07:57:58.782 INFO:teuthology.orchestra.run.smithi059.stdout: "mon": "allow *", 2023-11-24T07:57:58.783 INFO:teuthology.orchestra.run.smithi059.stdout: "osd": "allow *" 2023-11-24T07:57:58.783 INFO:teuthology.orchestra.run.smithi059.stdout: } 2023-11-24T07:57:58.783 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-11-24T07:57:58.783 INFO:teuthology.orchestra.run.smithi059.stdout: { 2023-11-24T07:57:58.783 INFO:teuthology.orchestra.run.smithi059.stdout: "entity": "mds.c", 2023-11-24T07:57:58.783 INFO:teuthology.orchestra.run.smithi059.stdout: "key": "AQD0VmBlvpsrMBAAr5lh7nb6B1xRfk0Umm7Xig==", 2023-11-24T07:57:58.783 INFO:teuthology.orchestra.run.smithi059.stdout: "caps": { 2023-11-24T07:57:58.783 INFO:teuthology.orchestra.run.smithi059.stdout: "mds": "allow", 2023-11-24T07:57:58.783 INFO:teuthology.orchestra.run.smithi059.stdout: "mgr": "allow *", 2023-11-24T07:57:58.783 INFO:teuthology.orchestra.run.smithi059.stdout: "mon": "allow *", 2023-11-24T07:57:58.783 INFO:teuthology.orchestra.run.smithi059.stdout: "osd": "allow *" 2023-11-24T07:57:58.783 INFO:teuthology.orchestra.run.smithi059.stdout: } 2023-11-24T07:57:58.784 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-11-24T07:57:58.784 INFO:teuthology.orchestra.run.smithi059.stdout: { 2023-11-24T07:57:58.784 INFO:teuthology.orchestra.run.smithi059.stdout: "entity": "mds.d", 2023-11-24T07:57:58.784 INFO:teuthology.orchestra.run.smithi059.stdout: "key": "AQD1VmBlPTG7ARAAvC60Rk2GarnNtRJ6MMRIMw==", 2023-11-24T07:57:58.784 INFO:teuthology.orchestra.run.smithi059.stdout: "caps": { 2023-11-24T07:57:58.784 INFO:teuthology.orchestra.run.smithi059.stdout: "mds": "allow", 2023-11-24T07:57:58.784 INFO:teuthology.orchestra.run.smithi059.stdout: "mgr": "allow *", 2023-11-24T07:57:58.784 INFO:teuthology.orchestra.run.smithi059.stdout: "mon": "allow *", 2023-11-24T07:57:58.784 INFO:teuthology.orchestra.run.smithi059.stdout: "osd": "allow *" 2023-11-24T07:57:58.784 INFO:teuthology.orchestra.run.smithi059.stdout: } 2023-11-24T07:57:58.784 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-11-24T07:57:58.785 INFO:teuthology.orchestra.run.smithi059.stdout: { 2023-11-24T07:57:58.785 INFO:teuthology.orchestra.run.smithi059.stdout: "entity": "osd.0", 2023-11-24T07:57:58.785 INFO:teuthology.orchestra.run.smithi059.stdout: "key": "AQD4VmBlXjdAHRAAjnK2Y/rQhg4a97faNqVuoA==", 2023-11-24T07:57:58.785 INFO:teuthology.orchestra.run.smithi059.stdout: "caps": { 2023-11-24T07:57:58.785 INFO:teuthology.orchestra.run.smithi059.stdout: "mgr": "allow profile osd", 2023-11-24T07:57:58.785 INFO:teuthology.orchestra.run.smithi059.stdout: "mon": "allow profile osd", 2023-11-24T07:57:58.785 INFO:teuthology.orchestra.run.smithi059.stdout: "osd": "allow *" 2023-11-24T07:57:58.786 INFO:teuthology.orchestra.run.smithi059.stdout: } 2023-11-24T07:57:58.786 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-11-24T07:57:58.786 INFO:teuthology.orchestra.run.smithi059.stdout: { 2023-11-24T07:57:58.786 INFO:teuthology.orchestra.run.smithi059.stdout: "entity": "osd.1", 2023-11-24T07:57:58.786 INFO:teuthology.orchestra.run.smithi059.stdout: "key": "AQD7VmBlBwTwHBAAbpMH9rzS95ifrq/gVHnYRA==", 2023-11-24T07:57:58.786 INFO:teuthology.orchestra.run.smithi059.stdout: "caps": { 2023-11-24T07:57:58.786 INFO:teuthology.orchestra.run.smithi059.stdout: "mgr": "allow profile osd", 2023-11-24T07:57:58.786 INFO:teuthology.orchestra.run.smithi059.stdout: "mon": "allow profile osd", 2023-11-24T07:57:58.786 INFO:teuthology.orchestra.run.smithi059.stdout: "osd": "allow *" 2023-11-24T07:57:58.786 INFO:teuthology.orchestra.run.smithi059.stdout: } 2023-11-24T07:57:58.786 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-11-24T07:57:58.787 INFO:teuthology.orchestra.run.smithi059.stdout: { 2023-11-24T07:57:58.787 INFO:teuthology.orchestra.run.smithi059.stdout: "entity": "osd.2", 2023-11-24T07:57:58.787 INFO:teuthology.orchestra.run.smithi059.stdout: "key": "AQD+VmBl0tijHBAAWF2PsV3iH5adOTvHVPFIKw==", 2023-11-24T07:57:58.787 INFO:teuthology.orchestra.run.smithi059.stdout: "caps": { 2023-11-24T07:57:58.787 INFO:teuthology.orchestra.run.smithi059.stdout: "mgr": "allow profile osd", 2023-11-24T07:57:58.787 INFO:teuthology.orchestra.run.smithi059.stdout: "mon": "allow profile osd", 2023-11-24T07:57:58.787 INFO:teuthology.orchestra.run.smithi059.stdout: "osd": "allow *" 2023-11-24T07:57:58.787 INFO:teuthology.orchestra.run.smithi059.stdout: } 2023-11-24T07:57:58.787 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-11-24T07:57:58.787 INFO:teuthology.orchestra.run.smithi059.stdout: { 2023-11-24T07:57:58.787 INFO:teuthology.orchestra.run.smithi059.stdout: "entity": "osd.3", 2023-11-24T07:57:58.788 INFO:teuthology.orchestra.run.smithi059.stdout: "key": "AQABV2Bl3i8/HBAA4waLTD/PSLuNEg1VBmhEmg==", 2023-11-24T07:57:58.788 INFO:teuthology.orchestra.run.smithi059.stdout: "caps": { 2023-11-24T07:57:58.788 INFO:teuthology.orchestra.run.smithi059.stdout: "mgr": "allow profile osd", 2023-11-24T07:57:58.788 INFO:teuthology.orchestra.run.smithi059.stdout: "mon": "allow profile osd", 2023-11-24T07:57:58.788 INFO:teuthology.orchestra.run.smithi059.stdout: "osd": "allow *" 2023-11-24T07:57:58.788 INFO:teuthology.orchestra.run.smithi059.stdout: } 2023-11-24T07:57:58.788 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-11-24T07:57:58.788 INFO:teuthology.orchestra.run.smithi059.stdout: { 2023-11-24T07:57:58.788 INFO:teuthology.orchestra.run.smithi059.stdout: "entity": "osd.4", 2023-11-24T07:57:58.788 INFO:teuthology.orchestra.run.smithi059.stdout: "key": "AQAHV2Blr2zrHxAALOr+kFGdOTr8uQ9+/G2WUg==", 2023-11-24T07:57:58.789 INFO:teuthology.orchestra.run.smithi059.stdout: "caps": { 2023-11-24T07:57:58.789 INFO:teuthology.orchestra.run.smithi059.stdout: "mgr": "allow profile osd", 2023-11-24T07:57:58.789 INFO:teuthology.orchestra.run.smithi059.stdout: "mon": "allow profile osd", 2023-11-24T07:57:58.789 INFO:teuthology.orchestra.run.smithi059.stdout: "osd": "allow *" 2023-11-24T07:57:58.789 INFO:teuthology.orchestra.run.smithi059.stdout: } 2023-11-24T07:57:58.789 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-11-24T07:57:58.789 INFO:teuthology.orchestra.run.smithi059.stdout: { 2023-11-24T07:57:58.789 INFO:teuthology.orchestra.run.smithi059.stdout: "entity": "osd.5", 2023-11-24T07:57:58.789 INFO:teuthology.orchestra.run.smithi059.stdout: "key": "AQAKV2BlDZ3bHxAA7ZPPoxi4zVN0VECYzTsAsg==", 2023-11-24T07:57:58.789 INFO:teuthology.orchestra.run.smithi059.stdout: "caps": { 2023-11-24T07:57:58.789 INFO:teuthology.orchestra.run.smithi059.stdout: "mgr": "allow profile osd", 2023-11-24T07:57:58.790 INFO:teuthology.orchestra.run.smithi059.stdout: "mon": "allow profile osd", 2023-11-24T07:57:58.790 INFO:teuthology.orchestra.run.smithi059.stdout: "osd": "allow *" 2023-11-24T07:57:58.790 INFO:teuthology.orchestra.run.smithi059.stdout: } 2023-11-24T07:57:58.790 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-11-24T07:57:58.790 INFO:teuthology.orchestra.run.smithi059.stdout: { 2023-11-24T07:57:58.790 INFO:teuthology.orchestra.run.smithi059.stdout: "entity": "osd.6", 2023-11-24T07:57:58.790 INFO:teuthology.orchestra.run.smithi059.stdout: "key": "AQANV2BlfDMrHxAAVtm8+kh5FNTd1mJOKg02gg==", 2023-11-24T07:57:58.790 INFO:teuthology.orchestra.run.smithi059.stdout: "caps": { 2023-11-24T07:57:58.790 INFO:teuthology.orchestra.run.smithi059.stdout: "mgr": "allow profile osd", 2023-11-24T07:57:58.790 INFO:teuthology.orchestra.run.smithi059.stdout: "mon": "allow profile osd", 2023-11-24T07:57:58.790 INFO:teuthology.orchestra.run.smithi059.stdout: "osd": "allow *" 2023-11-24T07:57:58.791 INFO:teuthology.orchestra.run.smithi059.stdout: } 2023-11-24T07:57:58.791 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-11-24T07:57:58.791 INFO:teuthology.orchestra.run.smithi059.stdout: { 2023-11-24T07:57:58.791 INFO:teuthology.orchestra.run.smithi059.stdout: "entity": "osd.7", 2023-11-24T07:57:58.791 INFO:teuthology.orchestra.run.smithi059.stdout: "key": "AQAQV2Bli+slHxAAHY4sa84El5NC4KVgPKHs6w==", 2023-11-24T07:57:58.791 INFO:teuthology.orchestra.run.smithi059.stdout: "caps": { 2023-11-24T07:57:58.791 INFO:teuthology.orchestra.run.smithi059.stdout: "mgr": "allow profile osd", 2023-11-24T07:57:58.791 INFO:teuthology.orchestra.run.smithi059.stdout: "mon": "allow profile osd", 2023-11-24T07:57:58.791 INFO:teuthology.orchestra.run.smithi059.stdout: "osd": "allow *" 2023-11-24T07:57:58.791 INFO:teuthology.orchestra.run.smithi059.stdout: } 2023-11-24T07:57:58.791 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-11-24T07:57:58.791 INFO:teuthology.orchestra.run.smithi059.stdout: { 2023-11-24T07:57:58.791 INFO:teuthology.orchestra.run.smithi059.stdout: "entity": "client.0", 2023-11-24T07:57:58.792 INFO:teuthology.orchestra.run.smithi059.stdout: "key": "AQD1VmBlxPkpBhAAR841hRsF6QtmzFgJIrVZ9w==", 2023-11-24T07:57:58.792 INFO:teuthology.orchestra.run.smithi059.stdout: "caps": { 2023-11-24T07:57:58.792 INFO:teuthology.orchestra.run.smithi059.stdout: "mds": "allow", 2023-11-24T07:57:58.792 INFO:teuthology.orchestra.run.smithi059.stdout: "mon": "allow r", 2023-11-24T07:57:58.792 INFO:teuthology.orchestra.run.smithi059.stdout: "osd": "allow rw tag cephfs data=cephfs" 2023-11-24T07:57:58.792 INFO:teuthology.orchestra.run.smithi059.stdout: } 2023-11-24T07:57:58.792 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-11-24T07:57:58.792 INFO:teuthology.orchestra.run.smithi059.stdout: { 2023-11-24T07:57:58.792 INFO:teuthology.orchestra.run.smithi059.stdout: "entity": "client.1", 2023-11-24T07:57:58.792 INFO:teuthology.orchestra.run.smithi059.stdout: "key": "AQD1VmBl46HpDBAAnO9MGWrJczW0epSp0eCkLg==", 2023-11-24T07:57:58.792 INFO:teuthology.orchestra.run.smithi059.stdout: "caps": { 2023-11-24T07:57:58.793 INFO:teuthology.orchestra.run.smithi059.stdout: "mds": "allow", 2023-11-24T07:57:58.793 INFO:teuthology.orchestra.run.smithi059.stdout: "mgr": "allow r", 2023-11-24T07:57:58.793 INFO:teuthology.orchestra.run.smithi059.stdout: "mon": "allow rw", 2023-11-24T07:57:58.793 INFO:teuthology.orchestra.run.smithi059.stdout: "osd": "allow rwx" 2023-11-24T07:57:58.793 INFO:teuthology.orchestra.run.smithi059.stdout: } 2023-11-24T07:57:58.793 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-11-24T07:57:58.793 INFO:teuthology.orchestra.run.smithi059.stdout: { 2023-11-24T07:57:58.793 INFO:teuthology.orchestra.run.smithi059.stdout: "entity": "client.2", 2023-11-24T07:57:58.793 INFO:teuthology.orchestra.run.smithi059.stdout: "key": "AQD1VmBlgQSRCRAAcHS1eQCYMJ3YP21jUEPJ0A==", 2023-11-24T07:57:58.793 INFO:teuthology.orchestra.run.smithi059.stdout: "caps": { 2023-11-24T07:57:58.793 INFO:teuthology.orchestra.run.smithi059.stdout: "mds": "allow", 2023-11-24T07:57:58.794 INFO:teuthology.orchestra.run.smithi059.stdout: "mgr": "allow r", 2023-11-24T07:57:58.794 INFO:teuthology.orchestra.run.smithi059.stdout: "mon": "allow rw", 2023-11-24T07:57:58.794 INFO:teuthology.orchestra.run.smithi059.stdout: "osd": "allow rwx" 2023-11-24T07:57:58.794 INFO:teuthology.orchestra.run.smithi059.stdout: } 2023-11-24T07:57:58.794 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-11-24T07:57:58.794 INFO:teuthology.orchestra.run.smithi059.stdout: { 2023-11-24T07:57:58.794 INFO:teuthology.orchestra.run.smithi059.stdout: "entity": "client.3", 2023-11-24T07:57:58.794 INFO:teuthology.orchestra.run.smithi059.stdout: "key": "AQD1VmBl7gmhEhAA6uWs7XAbOQwJItA6To549w==", 2023-11-24T07:57:58.794 INFO:teuthology.orchestra.run.smithi059.stdout: "caps": { 2023-11-24T07:57:58.794 INFO:teuthology.orchestra.run.smithi059.stdout: "mds": "allow", 2023-11-24T07:57:58.794 INFO:teuthology.orchestra.run.smithi059.stdout: "mgr": "allow r", 2023-11-24T07:57:58.794 INFO:teuthology.orchestra.run.smithi059.stdout: "mon": "allow rw", 2023-11-24T07:57:58.795 INFO:teuthology.orchestra.run.smithi059.stdout: "osd": "allow rwx" 2023-11-24T07:57:58.795 INFO:teuthology.orchestra.run.smithi059.stdout: } 2023-11-24T07:57:58.795 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-11-24T07:57:58.795 INFO:teuthology.orchestra.run.smithi059.stdout: { 2023-11-24T07:57:58.795 INFO:teuthology.orchestra.run.smithi059.stdout: "entity": "client.admin", 2023-11-24T07:57:58.795 INFO:teuthology.orchestra.run.smithi059.stdout: "key": "AQD0VmBlAECpCRAA1CkLlkyjnOovaNUGDSWhaw==", 2023-11-24T07:57:58.795 INFO:teuthology.orchestra.run.smithi059.stdout: "caps": { 2023-11-24T07:57:58.795 INFO:teuthology.orchestra.run.smithi059.stdout: "mds": "allow *", 2023-11-24T07:57:58.795 INFO:teuthology.orchestra.run.smithi059.stdout: "mgr": "allow *", 2023-11-24T07:57:58.795 INFO:teuthology.orchestra.run.smithi059.stdout: "mon": "allow *", 2023-11-24T07:57:58.795 INFO:teuthology.orchestra.run.smithi059.stdout: "osd": "allow *" 2023-11-24T07:57:58.796 INFO:teuthology.orchestra.run.smithi059.stdout: } 2023-11-24T07:57:58.796 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-11-24T07:57:58.796 INFO:teuthology.orchestra.run.smithi059.stdout: { 2023-11-24T07:57:58.796 INFO:teuthology.orchestra.run.smithi059.stdout: "entity": "mgr.x", 2023-11-24T07:57:58.796 INFO:teuthology.orchestra.run.smithi059.stdout: "key": "AQD0VmBlryM+JRAAf72PlDWEHZOwNxhW4k/8Ig==", 2023-11-24T07:57:58.796 INFO:teuthology.orchestra.run.smithi059.stdout: "caps": { 2023-11-24T07:57:58.796 INFO:teuthology.orchestra.run.smithi059.stdout: "mds": "allow *", 2023-11-24T07:57:58.796 INFO:teuthology.orchestra.run.smithi059.stdout: "mon": "allow profile mgr", 2023-11-24T07:57:58.796 INFO:teuthology.orchestra.run.smithi059.stdout: "osd": "allow *" 2023-11-24T07:57:58.796 INFO:teuthology.orchestra.run.smithi059.stdout: } 2023-11-24T07:57:58.796 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-11-24T07:57:58.796 INFO:teuthology.orchestra.run.smithi059.stdout: { 2023-11-24T07:57:58.796 INFO:teuthology.orchestra.run.smithi059.stdout: "entity": "mgr.y", 2023-11-24T07:57:58.797 INFO:teuthology.orchestra.run.smithi059.stdout: "key": "AQD0VmBlpXXaHxAAJMwwht1n+zRbRZuGOD5WTQ==", 2023-11-24T07:57:58.797 INFO:teuthology.orchestra.run.smithi059.stdout: "caps": { 2023-11-24T07:57:58.797 INFO:teuthology.orchestra.run.smithi059.stdout: "mds": "allow *", 2023-11-24T07:57:58.797 INFO:teuthology.orchestra.run.smithi059.stdout: "mon": "allow profile mgr", 2023-11-24T07:57:58.797 INFO:teuthology.orchestra.run.smithi059.stdout: "osd": "allow *" 2023-11-24T07:57:58.797 INFO:teuthology.orchestra.run.smithi059.stdout: } 2023-11-24T07:57:58.797 INFO:teuthology.orchestra.run.smithi059.stdout: } 2023-11-24T07:57:58.797 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2023-11-24T07:57:58.797 INFO:teuthology.orchestra.run.smithi059.stdout:} 2023-11-24T07:57:58.799 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:58.781+0000 7f0d3df21700 1 -- 172.21.15.59:0/4059433949 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f0d1806af60 msgr2=0x7f0d1806d420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:57:58.799 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:58.781+0000 7f0d3df21700 1 --2- 172.21.15.59:0/4059433949 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f0d1806af60 0x7f0d1806d420 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f0d28000f60 tx=0x7f0d28009040 comp rx=0 tx=0).stop 2023-11-24T07:57:58.799 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:58.781+0000 7f0d3df21700 1 -- 172.21.15.59:0/4059433949 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0d3815e460 msgr2=0x7f0d380c99c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:57:58.799 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:58.781+0000 7f0d3df21700 1 --2- 172.21.15.59:0/4059433949 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0d3815e460 0x7f0d380c99c0 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7f0d30007e40 tx=0x7f0d3000e3c0 comp rx=0 tx=0).stop 2023-11-24T07:57:58.799 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:58.781+0000 7f0d3df21700 1 -- 172.21.15.59:0/4059433949 shutdown_connections 2023-11-24T07:57:58.799 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:58.781+0000 7f0d3df21700 1 --2- 172.21.15.59:0/4059433949 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f0d1806af60 0x7f0d1806d420 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:58.799 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:58.781+0000 7f0d3df21700 1 --2- 172.21.15.59:0/4059433949 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0d3815e460 0x7f0d380c99c0 unknown :-1 s=CLOSED pgs=519 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:58.799 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:58.781+0000 7f0d3df21700 1 --2- 172.21.15.59:0/4059433949 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f0d380d29d0 0x7f0d380cb990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:58.799 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:58.781+0000 7f0d3df21700 1 --2- 172.21.15.59:0/4059433949 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0d380d49d0 0x7f0d381580d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:58.799 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:58.781+0000 7f0d3df21700 1 -- 172.21.15.59:0/4059433949 >> 172.21.15.59:0/4059433949 conn(0x7f0d380d3eb0 msgr2=0x7f0d38073f30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:57:58.800 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:58.781+0000 7f0d3df21700 1 -- 172.21.15.59:0/4059433949 shutdown_connections 2023-11-24T07:57:58.800 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:58.781+0000 7f0d3df21700 1 -- 172.21.15.59:0/4059433949 wait complete. 2023-11-24T07:57:58.800 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-24T07:57:58.924 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:58.923+0000 7f2b63612700 1 Processor -- start 2023-11-24T07:57:58.925 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:58.923+0000 7f2b63612700 1 -- start start 2023-11-24T07:57:58.928 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:58.924+0000 7f2b63612700 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f2b5c0b4f20 0x7f2b5c071d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:57:58.928 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:58.924+0000 7f2b63612700 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2b5c072760 con 0x7f2b5c0722d0 2023-11-24T07:57:58.928 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:58.924+0000 7f2b63612700 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2b5c06e550 con 0x7f2b5c15abb0 2023-11-24T07:57:58.928 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:58.924+0000 7f2b63612700 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2b5c06f3b0 con 0x7f2b5c0b4f20 2023-11-24T07:57:58.928 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:58.924+0000 7f2b61baf700 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7f2b5c0722d0 0x7f2b5c0cdee0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:56078/0 (socket says 172.21.15.59:56078) 2023-11-24T07:57:58.928 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:58.924+0000 7f2b61baf700 1 -- 172.21.15.59:0/29906379 learned_addr learned my addr 172.21.15.59:0/29906379 (peer_addr_for_me v1:172.21.15.59:0/0) 2023-11-24T07:57:58.929 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:58.924+0000 7f2b53fff700 1 -- 172.21.15.59:0/29906379 <== mon.0 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2909561630 0 0) 0x7f2b54001490 con 0x7f2b5c0722d0 2023-11-24T07:57:58.929 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:58.924+0000 7f2b60bad700 1 --2- 172.21.15.59:0/29906379 >> v2:172.21.15.76:3300/0 conn(0x7f2b5c0b4f20 0x7f2b5c071d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:57:58.929 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:58.924+0000 7f2b53fff700 1 -- 172.21.15.59:0/29906379 --> v1:172.21.15.59:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2b480035b0 con 0x7f2b5c0722d0 2023-11-24T07:57:58.929 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:58.924+0000 7f2b53fff700 1 -- 172.21.15.59:0/29906379 <== mon.0 v1:172.21.15.59:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 261603220 0 0) 0x7f2b480035b0 con 0x7f2b5c0722d0 2023-11-24T07:57:58.929 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:58.924+0000 7f2b53fff700 1 -- 172.21.15.59:0/29906379 >> v2:172.21.15.76:3300/0 conn(0x7f2b5c0b4f20 msgr2=0x7f2b5c071d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T07:57:58.929 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:58.924+0000 7f2b53fff700 1 --2- 172.21.15.59:0/29906379 >> v2:172.21.15.76:3300/0 conn(0x7f2b5c0b4f20 0x7f2b5c071d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:58.929 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:58.924+0000 7f2b53fff700 1 -- 172.21.15.59:0/29906379 >> v1:172.21.15.76:6789/0 conn(0x7f2b5c15abb0 legacy=0x7f2b5c0b4810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:57:58.929 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:58.924+0000 7f2b53fff700 1 -- 172.21.15.59:0/29906379 --> v1:172.21.15.59:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2b5c06e7a0 con 0x7f2b5c0722d0 2023-11-24T07:57:58.929 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:58.924+0000 7f2b53fff700 1 -- 172.21.15.59:0/29906379 <== mon.0 v1:172.21.15.59:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1312333566 0 0) 0x7f2b54001b80 con 0x7f2b5c0722d0 2023-11-24T07:57:58.929 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:58.924+0000 7f2b53fff700 1 -- 172.21.15.59:0/29906379 >> v1:172.21.15.59:6789/0 conn(0x7f2b5c0722d0 legacy=0x7f2b5c0cdee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:57:58.929 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:58.924+0000 7f2b53fff700 1 --2- 172.21.15.59:0/29906379 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2b48003a00 0x7f2b48003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:57:58.929 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:58.924+0000 7f2b53fff700 1 --2- 172.21.15.59:0/29906379 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2b48004850 0x7f2b48004cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:57:58.930 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:58.924+0000 7f2b53fff700 1 --2- 172.21.15.59:0/29906379 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f2b48005210 0x7f2b4800ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:57:58.930 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:58.924+0000 7f2b53fff700 1 -- 172.21.15.59:0/29906379 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b5c06f3b0 con 0x7f2b48004850 2023-11-24T07:57:58.930 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:58.924+0000 7f2b53fff700 1 -- 172.21.15.59:0/29906379 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b4800c0b0 con 0x7f2b48003a00 2023-11-24T07:57:58.930 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:58.924+0000 7f2b53fff700 1 -- 172.21.15.59:0/29906379 --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2b4800c270 con 0x7f2b48005210 2023-11-24T07:57:58.930 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:58.924+0000 7f2b60bad700 1 --2- 172.21.15.59:0/29906379 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2b48004850 0x7f2b48004cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:57:58.930 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:58.924+0000 7f2b613ae700 1 --2- 172.21.15.59:0/29906379 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2b48003a00 0x7f2b48003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:57:58.930 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:58.925+0000 7f2b61baf700 1 --2- 172.21.15.59:0/29906379 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f2b48005210 0x7f2b4800ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:57:58.930 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:58.925+0000 7f2b613ae700 1 -- 172.21.15.59:0/29906379 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f2b48005210 msgr2=0x7f2b4800ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T07:57:58.930 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:58.925+0000 7f2b613ae700 1 --2- 172.21.15.59:0/29906379 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f2b48005210 0x7f2b4800ba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:58.930 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:58.925+0000 7f2b613ae700 1 -- 172.21.15.59:0/29906379 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2b48004850 msgr2=0x7f2b48004cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T07:57:58.930 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:58.925+0000 7f2b613ae700 1 --2- 172.21.15.59:0/29906379 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2b48004850 0x7f2b48004cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:58.930 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:58.925+0000 7f2b61baf700 1 -- 172.21.15.59:0/29906379 reap_dead start 2023-11-24T07:57:58.931 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:58.925+0000 7f2b613ae700 1 -- 172.21.15.59:0/29906379 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2b480035b0 con 0x7f2b48003a00 2023-11-24T07:57:58.931 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:58.925+0000 7f2b60bad700 1 --2- 172.21.15.59:0/29906379 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2b48004850 0x7f2b48004cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-24T07:57:58.931 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:58.925+0000 7f2b613ae700 1 --2- 172.21.15.59:0/29906379 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2b48003a00 0x7f2b48003e20 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f2b580017c0 tx=0x7f2b48005880 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e0dd3b713561ba74 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:57:58.931 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:58.925+0000 7f2b53fff700 1 -- 172.21.15.59:0/29906379 <== mon.1 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b580052d0 con 0x7f2b48003a00 2023-11-24T07:57:58.931 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:58.925+0000 7f2b53fff700 1 -- 172.21.15.59:0/29906379 <== mon.1 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2b58005430 con 0x7f2b48003a00 2023-11-24T07:57:58.931 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:58.925+0000 7f2b53fff700 1 -- 172.21.15.59:0/29906379 <== mon.1 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b580038c0 con 0x7f2b48003a00 2023-11-24T07:57:58.931 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:58.926+0000 7f2b63612700 1 -- 172.21.15.59:0/29906379 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2b48003a00 msgr2=0x7f2b48003e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:57:58.931 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:58.926+0000 7f2b63612700 1 --2- 172.21.15.59:0/29906379 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2b48003a00 0x7f2b48003e20 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f2b580017c0 tx=0x7f2b48005880 comp rx=0 tx=0).stop 2023-11-24T07:57:58.931 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:58.926+0000 7f2b63612700 1 -- 172.21.15.59:0/29906379 shutdown_connections 2023-11-24T07:57:58.931 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:58.926+0000 7f2b63612700 1 --2- 172.21.15.59:0/29906379 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2b48003a00 0x7f2b48003e20 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:58.931 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:58.926+0000 7f2b63612700 1 -- 172.21.15.59:0/29906379 >> 172.21.15.59:0/29906379 conn(0x7f2b5c0d0aa0 msgr2=0x7f2b5c0d0eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:57:58.931 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:58.926+0000 7f2b63612700 1 -- 172.21.15.59:0/29906379 shutdown_connections 2023-11-24T07:57:58.932 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:58.926+0000 7f2b63612700 1 -- 172.21.15.59:0/29906379 wait complete. 2023-11-24T07:57:58.934 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:58.927+0000 7f2b63612700 1 Processor -- start 2023-11-24T07:57:58.934 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:58.927+0000 7f2b63612700 1 -- start start 2023-11-24T07:57:58.934 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:58.928+0000 7f2b63612700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2b48003a00 0x7f2b5c150af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:57:58.934 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:58.928+0000 7f2b63612700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2b5c1506b0 0x7f2b5c14e9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:57:58.934 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:58.928+0000 7f2b63612700 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f2b5c156af0 0x7f2b5c0b0d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:57:58.935 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:58.928+0000 7f2b63612700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b5c14ef00 con 0x7f2b5c1506b0 2023-11-24T07:57:58.935 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:58.928+0000 7f2b63612700 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b5c159610 con 0x7f2b48003a00 2023-11-24T07:57:58.935 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:58.928+0000 7f2b63612700 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2b5c1598f0 con 0x7f2b5c156af0 2023-11-24T07:57:58.935 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:58.928+0000 7f2b60bad700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2b5c1506b0 0x7f2b5c14e9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:57:58.935 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:58.928+0000 7f2b60bad700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2b5c1506b0 0x7f2b5c14e9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:52616/0 (socket says 172.21.15.59:52616) 2023-11-24T07:57:58.935 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:58.928+0000 7f2b60bad700 1 -- 172.21.15.59:0/2972925809 learned_addr learned my addr 172.21.15.59:0/2972925809 (peer_addr_for_me v2:172.21.15.59:0/0) 2023-11-24T07:57:58.935 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:58.928+0000 7f2b613ae700 1 --2- 172.21.15.59:0/2972925809 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2b48003a00 0x7f2b5c150af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:57:58.935 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:58.928+0000 7f2b61baf700 1 --2- 172.21.15.59:0/2972925809 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f2b5c156af0 0x7f2b5c0b0d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:57:58.935 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:58.928+0000 7f2b60bad700 1 -- 172.21.15.59:0/2972925809 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f2b5c156af0 msgr2=0x7f2b5c0b0d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:57:58.935 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:58.928+0000 7f2b60bad700 1 --2- 172.21.15.59:0/2972925809 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f2b5c156af0 0x7f2b5c0b0d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:58.935 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:58.928+0000 7f2b60bad700 1 -- 172.21.15.59:0/2972925809 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2b48003a00 msgr2=0x7f2b5c150af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:57:58.935 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:58.928+0000 7f2b60bad700 1 --2- 172.21.15.59:0/2972925809 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2b48003a00 0x7f2b5c150af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:58.936 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:58.928+0000 7f2b60bad700 1 -- 172.21.15.59:0/2972925809 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2b5c159bd0 con 0x7f2b5c1506b0 2023-11-24T07:57:58.936 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:58.928+0000 7f2b61baf700 1 --2- 172.21.15.59:0/2972925809 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f2b5c156af0 0x7f2b5c0b0d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-24T07:57:58.936 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:58.928+0000 7f2b60bad700 1 --2- 172.21.15.59:0/2972925809 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2b5c1506b0 0x7f2b5c14e9c0 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7f2b4c011bd0 tx=0x7f2b4c011ee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:57:58.936 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:58.929+0000 7f2b51ffb700 1 -- 172.21.15.59:0/2972925809 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b4c00fa00 con 0x7f2b5c1506b0 2023-11-24T07:57:58.936 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:58.929+0000 7f2b51ffb700 1 -- 172.21.15.59:0/2972925809 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2b4c00fb60 con 0x7f2b5c1506b0 2023-11-24T07:57:58.936 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:58.929+0000 7f2b63612700 1 -- 172.21.15.59:0/2972925809 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2b5c159eb0 con 0x7f2b5c1506b0 2023-11-24T07:57:58.936 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:58.929+0000 7f2b51ffb700 1 -- 172.21.15.59:0/2972925809 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b4c004020 con 0x7f2b5c1506b0 2023-11-24T07:57:58.936 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:58.929+0000 7f2b63612700 1 -- 172.21.15.59:0/2972925809 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2b5c156f60 con 0x7f2b5c1506b0 2023-11-24T07:57:58.936 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:58.930+0000 7f2b63612700 1 -- 172.21.15.59:0/2972925809 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2b5c06f840 con 0x7f2b5c1506b0 2023-11-24T07:57:58.936 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:58.930+0000 7f2b51ffb700 1 -- 172.21.15.59:0/2972925809 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 6) v1 ==== 89282+0+0 (secure 0 0 0) 0x7f2b4c004180 con 0x7f2b5c1506b0 2023-11-24T07:57:58.936 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:58.931+0000 7f2b51ffb700 1 --2- 172.21.15.59:0/2972925809 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f2b4006af60 0x7f2b4006d420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:57:58.936 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:58.931+0000 7f2b51ffb700 1 -- 172.21.15.59:0/2972925809 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f2b4c0894d0 con 0x7f2b5c1506b0 2023-11-24T07:57:58.937 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:58.931+0000 7f2b613ae700 1 --2- 172.21.15.59:0/2972925809 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f2b4006af60 0x7f2b4006d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:57:58.937 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:58.931+0000 7f2b613ae700 1 --2- 172.21.15.59:0/2972925809 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f2b4006af60 0x7f2b4006d420 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f2b58005040 tx=0x7f2b58009030 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:57:58.937 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:58.935+0000 7f2b51ffb700 1 -- 172.21.15.59:0/2972925809 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f2b4c054970 con 0x7f2b5c1506b0 2023-11-24T07:57:59.139 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:59.137+0000 7f2b63612700 1 -- 172.21.15.59:0/2972925809 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f2b5c068670 con 0x7f2b5c1506b0 2023-11-24T07:57:59.139 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:59.138+0000 7f2b51ffb700 1 -- 172.21.15.59:0/2972925809 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v41) v1 ==== 74+0+12785 (secure 0 0 0) 0x7f2b4c022090 con 0x7f2b5c1506b0 2023-11-24T07:57:59.140 INFO:teuthology.orchestra.run.smithi059.stdout: 2023-11-24T07:57:59.140 INFO:teuthology.orchestra.run.smithi059.stdout:{"epoch":41,"fsid":"9f0e8741-b8f9-48d5-a2c3-09f45903303d","created":"2023-11-24T07:56:06.552729+0000","modified":"2023-11-24T07:57:58.333502+0000","last_up_change":"2023-11-24T07:56:17.372352+0000","last_in_change":"2023-11-24T07:56:09.073250+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-24T07:56:18.352823+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"6d8f9e9b-1782-4e41-928b-a39f2a8ebc06","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6800","nonce":3373878772},{"type":"v1","addr":"172.21.15.59:6801","nonce":3373878772}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6802","nonce":3373878772},{"type":"v1","addr":"172.21.15.59:6803","nonce":3373878772}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6806","nonce":3373878772},{"type":"v1","addr":"172.21.15.59:6807","nonce":3373878772}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6804","nonce":3373878772},{"type":"v1","addr":"172.21.15.59:6805","nonce":3373878772}]},"public_addr":"172.21.15.59:6801/3373878772","cluster_addr":"172.21.15.59:6803/3373878772","heartbeat_back_addr":"172.21.15.59:6807/3373878772","heartbeat_front_addr":"172.21.15.59:6805/3373878772","state":["exists","up"]},{"osd":1,"uuid":"e8d882aa-52e1-4c04-9e1e-c44ef9204a5f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6824","nonce":648258563},{"type":"v1","addr":"172.21.15.59:6825","nonce":648258563}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6826","nonce":648258563},{"type":"v1","addr":"172.21.15.59:6827","nonce":648258563}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6830","nonce":648258563},{"type":"v1","addr":"172.21.15.59:6831","nonce":648258563}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6828","nonce":648258563},{"type":"v1","addr":"172.21.15.59:6829","nonce":648258563}]},"public_addr":"172.21.15.59:6825/648258563","cluster_addr":"172.21.15.59:6827/648258563","heartbeat_back_addr":"172.21.15.59:6831/648258563","heartbeat_front_addr":"172.21.15.59:6829/648258563","state":["exists","up"]},{"osd":2,"uuid":"69eebfa0-695b-4b21-90b9-8a3537168a01","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6808","nonce":1840430654},{"type":"v1","addr":"172.21.15.59:6809","nonce":1840430654}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6810","nonce":1840430654},{"type":"v1","addr":"172.21.15.59:6811","nonce":1840430654}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6814","nonce":1840430654},{"type":"v1","addr":"172.21.15.59:6815","nonce":1840430654}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6812","nonce":1840430654},{"type":"v1","addr":"172.21.15.59:6813","nonce":1840430654}]},"public_addr":"172.21.15.59:6809/1840430654","cluster_addr":"172.21.15.59:6811/1840430654","heartbeat_back_addr":"172.21.15.59:6815/1840430654","heartbeat_front_addr":"172.21.15.59:6813/1840430654","state":["exists","up"]},{"osd":3,"uuid":"96fd77fc-f911-46ff-9445-6f0be2daa8f4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6816","nonce":2504025850},{"type":"v1","addr":"172.21.15.59:6817","nonce":2504025850}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6818","nonce":2504025850},{"type":"v1","addr":"172.21.15.59:6819","nonce":2504025850}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6822","nonce":2504025850},{"type":"v1","addr":"172.21.15.59:6823","nonce":2504025850}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6820","nonce":2504025850},{"type":"v1","addr":"172.21.15.59:6821","nonce":2504025850}]},"public_addr":"172.21.15.59:6817/2504025850","cluster_addr":"172.21.15.59:6819/2504025850","heartbeat_back_addr":"172.21.15.59:6823/2504025850","heartbeat_front_addr":"172.21.15.59:6821/2504025850","state":["exists","up"]},{"osd":4,"uuid":"7ac29fca-b5b2-4c95-9d69-2f164cbbc708","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6816","nonce":2382463431},{"type":"v1","addr":"172.21.15.76:6818","nonce":2382463431}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6820","nonce":2382463431},{"type":"v1","addr":"172.21.15.76:6822","nonce":2382463431}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6828","nonce":2382463431},{"type":"v1","addr":"172.21.15.76:6830","nonce":2382463431}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6824","nonce":2382463431},{"type":"v1","addr":"172.21.15.76:6826","nonce":2382463431}]},"public_addr":"172.21.15.76:6818/2382463431","cluster_addr":"172.21.15.76:6822/2382463431","heartbeat_back_addr":"172.21.15.76:6830/2382463431","heartbeat_front_addr":"172.21.15.76:6826/2382463431","state":["exists","up"]},{"osd":5,"uuid":"485bf711-3851-4238-a4f4-51d52e0a9c58","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6800","nonce":329334308},{"type":"v1","addr":"172.21.15.76:6802","nonce":329334308}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6805","nonce":329334308},{"type":"v1","addr":"172.21.15.76:6807","nonce":329334308}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6814","nonce":329334308},{"type":"v1","addr":"172.21.15.76:6815","nonce":329334308}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6810","nonce":329334308},{"type":"v1","addr":"172.21.15.76:6812","nonce":329334308}]},"public_addr":"172.21.15.76:6802/329334308","cluster_addr":"172.21.15.76:6807/329334308","heartbeat_back_addr":"172.21.15.76:6815/329334308","heartbeat_front_addr":"172.21.15.76:6812/329334308","state":["exists","up"]},{"osd":6,"uuid":"72b3f2fc-1e4a-4e32-968d-003647a0286f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6801","nonce":3946579410},{"type":"v1","addr":"172.21.15.76:6803","nonce":3946579410}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6804","nonce":3946579410},{"type":"v1","addr":"172.21.15.76:6806","nonce":3946579410}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6811","nonce":3946579410},{"type":"v1","addr":"172.21.15.76:6813","nonce":3946579410}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6808","nonce":3946579410},{"type":"v1","addr":"172.21.15.76:6809","nonce":3946579410}]},"public_addr":"172.21.15.76:6803/3946579410","cluster_addr":"172.21.15.76:6806/3946579410","heartbeat_back_addr":"172.21.15.76:6813/3946579410","heartbeat_front_addr":"172.21.15.76:6809/3946579410","state":["exists","up"]},{"osd":7,"uuid":"1ddceb90-6c5d-4d1a-ad56-99cb877f8c70","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6817","nonce":841761538},{"type":"v1","addr":"172.21.15.76:6819","nonce":841761538}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6821","nonce":841761538},{"type":"v1","addr":"172.21.15.76:6823","nonce":841761538}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6829","nonce":841761538},{"type":"v1","addr":"172.21.15.76:6831","nonce":841761538}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6825","nonce":841761538},{"type":"v1","addr":"172.21.15.76:6827","nonce":841761538}]},"public_addr":"172.21.15.76:6819/841761538","cluster_addr":"172.21.15.76:6823/841761538","heartbeat_back_addr":"172.21.15.76:6831/841761538","heartbeat_front_addr":"172.21.15.76:6827/841761538","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T07:56:14.294838+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T07:56:14.270779+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T07:56:14.231809+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T07:56:14.260983+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T07:56:14.302997+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T07:56:14.258600+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T07:56:14.251413+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T07:56:14.304727+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-24T07:57:59.142 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:59.140+0000 7f2b63612700 1 -- 172.21.15.59:0/2972925809 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f2b4006af60 msgr2=0x7f2b4006d420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:57:59.142 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:59.140+0000 7f2b63612700 1 --2- 172.21.15.59:0/2972925809 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f2b4006af60 0x7f2b4006d420 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f2b58005040 tx=0x7f2b58009030 comp rx=0 tx=0).stop 2023-11-24T07:57:59.142 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:59.140+0000 7f2b63612700 1 -- 172.21.15.59:0/2972925809 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2b5c1506b0 msgr2=0x7f2b5c14e9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:57:59.142 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:59.140+0000 7f2b63612700 1 --2- 172.21.15.59:0/2972925809 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2b5c1506b0 0x7f2b5c14e9c0 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7f2b4c011bd0 tx=0x7f2b4c011ee0 comp rx=0 tx=0).stop 2023-11-24T07:57:59.142 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:59.140+0000 7f2b63612700 1 -- 172.21.15.59:0/2972925809 shutdown_connections 2023-11-24T07:57:59.143 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:59.140+0000 7f2b63612700 1 --2- 172.21.15.59:0/2972925809 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f2b4006af60 0x7f2b4006d420 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:59.143 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:59.140+0000 7f2b63612700 1 --2- 172.21.15.59:0/2972925809 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f2b5c156af0 0x7f2b5c0b0d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:59.143 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:59.140+0000 7f2b63612700 1 --2- 172.21.15.59:0/2972925809 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2b48003a00 0x7f2b5c150af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:59.143 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:59.140+0000 7f2b63612700 1 --2- 172.21.15.59:0/2972925809 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2b5c1506b0 0x7f2b5c14e9c0 unknown :-1 s=CLOSED pgs=521 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:59.143 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:59.140+0000 7f2b63612700 1 -- 172.21.15.59:0/2972925809 >> 172.21.15.59:0/2972925809 conn(0x7f2b5c0d0aa0 msgr2=0x7f2b5c15e6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:57:59.143 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:59.140+0000 7f2b63612700 1 -- 172.21.15.59:0/2972925809 shutdown_connections 2023-11-24T07:57:59.143 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:59.140+0000 7f2b63612700 1 -- 172.21.15.59:0/2972925809 wait complete. 2023-11-24T07:57:59.154 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-24T07:56:18.352823+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-24T07:57:59.154 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-24T07:57:59.283 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:59.282+0000 7f030322f700 1 Processor -- start 2023-11-24T07:57:59.283 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:59.282+0000 7f030322f700 1 -- start start 2023-11-24T07:57:59.286 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:59.282+0000 7f030322f700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f02fc145490 0x7f02fc06ed80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:57:59.286 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:59.282+0000 7f030322f700 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f02fc1601a0 con 0x7f02fc06f2c0 2023-11-24T07:57:59.286 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:59.282+0000 7f030322f700 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f02fc0d18c0 con 0x7f02fc06b6e0 2023-11-24T07:57:59.286 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:59.282+0000 7f030322f700 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f02fc06f8d0 con 0x7f02fc145490 2023-11-24T07:57:59.286 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:59.283+0000 7f0300fcb700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f02fc145490 0x7f02fc06ed80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:57:59.286 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:59.283+0000 7f02fbfff700 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7f02fc06f2c0 0x7f02fc06afd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:56094/0 (socket says 172.21.15.59:56094) 2023-11-24T07:57:59.286 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:59.283+0000 7f02fbfff700 1 -- 172.21.15.59:0/418866282 learned_addr learned my addr 172.21.15.59:0/418866282 (peer_addr_for_me v1:172.21.15.59:0/0) 2023-11-24T07:57:59.286 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:59.283+0000 7f02fb7fe700 1 -- 172.21.15.59:0/418866282 <== mon.0 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1730602026 0 0) 0x7f02fc1601a0 con 0x7f02fc06f2c0 2023-11-24T07:57:59.286 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:59.283+0000 7f02fb7fe700 1 -- 172.21.15.59:0/418866282 --> v1:172.21.15.59:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f02ec0035b0 con 0x7f02fc06f2c0 2023-11-24T07:57:59.287 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:59.283+0000 7f0300fcb700 1 -- 172.21.15.59:0/418866282 >> v1:172.21.15.76:6789/0 conn(0x7f02fc06b6e0 legacy=0x7f02fc0cef40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:57:59.287 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:59.283+0000 7f0300fcb700 1 -- 172.21.15.59:0/418866282 >> v1:172.21.15.59:6789/0 conn(0x7f02fc06f2c0 legacy=0x7f02fc06afd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:57:59.287 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:59.283+0000 7f0300fcb700 1 -- 172.21.15.59:0/418866282 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f02fc15f870 con 0x7f02fc145490 2023-11-24T07:57:59.287 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:59.283+0000 7f0300fcb700 1 --2- 172.21.15.59:0/418866282 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f02fc145490 0x7f02fc06ed80 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7f02f0009840 tx=0x7f02f0009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=424d1cb71ffd3c81 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:57:59.287 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:59.283+0000 7f02fb7fe700 1 -- 172.21.15.59:0/418866282 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02f0011830 con 0x7f02fc145490 2023-11-24T07:57:59.287 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:59.283+0000 7f02fb7fe700 1 -- 172.21.15.59:0/418866282 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f02f0011990 con 0x7f02fc145490 2023-11-24T07:57:59.287 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:59.283+0000 7f02fb7fe700 1 -- 172.21.15.59:0/418866282 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02f0011d20 con 0x7f02fc145490 2023-11-24T07:57:59.287 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:59.284+0000 7f030322f700 1 -- 172.21.15.59:0/418866282 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f02fc145490 msgr2=0x7f02fc06ed80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:57:59.287 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:59.284+0000 7f030322f700 1 --2- 172.21.15.59:0/418866282 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f02fc145490 0x7f02fc06ed80 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7f02f0009840 tx=0x7f02f0009b50 comp rx=0 tx=0).stop 2023-11-24T07:57:59.287 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:59.284+0000 7f030322f700 1 -- 172.21.15.59:0/418866282 shutdown_connections 2023-11-24T07:57:59.287 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:59.284+0000 7f030322f700 1 --2- 172.21.15.59:0/418866282 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f02fc145490 0x7f02fc06ed80 unknown :-1 s=CLOSED pgs=523 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:59.287 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:59.284+0000 7f030322f700 1 -- 172.21.15.59:0/418866282 >> 172.21.15.59:0/418866282 conn(0x7f02fc0b2740 msgr2=0x7f02fc0b2b50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:57:59.288 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:59.284+0000 7f030322f700 1 -- 172.21.15.59:0/418866282 shutdown_connections 2023-11-24T07:57:59.288 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:59.284+0000 7f030322f700 1 -- 172.21.15.59:0/418866282 wait complete. 2023-11-24T07:57:59.290 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:59.285+0000 7f030322f700 1 Processor -- start 2023-11-24T07:57:59.290 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:59.285+0000 7f030322f700 1 -- start start 2023-11-24T07:57:59.290 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:59.286+0000 7f030322f700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f02fc06b6e0 0x7f02fc142db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:57:59.290 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:59.286+0000 7f030322f700 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f02fc06f2c0 0x7f02fc0b49b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:57:59.290 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:59.286+0000 7f030322f700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f02fc145490 0x7f02fc0b4ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:57:59.290 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:59.286+0000 7f030322f700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02fc134990 con 0x7f02fc06b6e0 2023-11-24T07:57:59.290 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:59.286+0000 7f030322f700 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02fc1432f0 con 0x7f02fc145490 2023-11-24T07:57:59.290 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:59.286+0000 7f030322f700 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f02fc1f4740 con 0x7f02fc06f2c0 2023-11-24T07:57:59.290 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:59.286+0000 7f0300fcb700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f02fc06b6e0 0x7f02fc142db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:57:59.290 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:59.286+0000 7f0300fcb700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f02fc06b6e0 0x7f02fc142db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:52626/0 (socket says 172.21.15.59:52626) 2023-11-24T07:57:59.291 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:59.286+0000 7f0300fcb700 1 -- 172.21.15.59:0/310046093 learned_addr learned my addr 172.21.15.59:0/310046093 (peer_addr_for_me v2:172.21.15.59:0/0) 2023-11-24T07:57:59.291 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:59.286+0000 7f02fbfff700 1 --2- 172.21.15.59:0/310046093 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f02fc06f2c0 0x7f02fc0b49b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:57:59.291 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:59.286+0000 7f03017cc700 1 --2- 172.21.15.59:0/310046093 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f02fc145490 0x7f02fc0b4ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:57:59.291 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:59.286+0000 7f0300fcb700 1 -- 172.21.15.59:0/310046093 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f02fc06f2c0 msgr2=0x7f02fc0b49b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:57:59.291 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:59.286+0000 7f0300fcb700 1 --2- 172.21.15.59:0/310046093 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f02fc06f2c0 0x7f02fc0b49b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:59.291 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:59.286+0000 7f0300fcb700 1 -- 172.21.15.59:0/310046093 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f02fc145490 msgr2=0x7f02fc0b4ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:57:59.291 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:59.286+0000 7f0300fcb700 1 --2- 172.21.15.59:0/310046093 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f02fc145490 0x7f02fc0b4ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:59.291 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:59.286+0000 7f0300fcb700 1 -- 172.21.15.59:0/310046093 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f02f00094a0 con 0x7f02fc06b6e0 2023-11-24T07:57:59.291 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:59.287+0000 7f0300fcb700 1 --2- 172.21.15.59:0/310046093 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f02fc06b6e0 0x7f02fc142db0 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7f02f0009b90 tx=0x7f02f0004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:57:59.291 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:59.287+0000 7f02f97fa700 1 -- 172.21.15.59:0/310046093 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02f00038a0 con 0x7f02fc06b6e0 2023-11-24T07:57:59.291 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:59.287+0000 7f02f97fa700 1 -- 172.21.15.59:0/310046093 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f02f001e070 con 0x7f02fc06b6e0 2023-11-24T07:57:59.293 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:59.287+0000 7f02f97fa700 1 -- 172.21.15.59:0/310046093 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02f0003c40 con 0x7f02fc06b6e0 2023-11-24T07:57:59.293 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:59.287+0000 7f030322f700 1 -- 172.21.15.59:0/310046093 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f02fc1607b0 con 0x7f02fc06b6e0 2023-11-24T07:57:59.293 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:59.287+0000 7f030322f700 1 -- 172.21.15.59:0/310046093 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f02fc1342d0 con 0x7f02fc06b6e0 2023-11-24T07:57:59.293 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:59.288+0000 7f030322f700 1 -- 172.21.15.59:0/310046093 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f02fc134c00 con 0x7f02fc06b6e0 2023-11-24T07:57:59.294 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:59.289+0000 7f02f97fa700 1 -- 172.21.15.59:0/310046093 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 6) v1 ==== 89282+0+0 (secure 0 0 0) 0x7f02f002d450 con 0x7f02fc06b6e0 2023-11-24T07:57:59.294 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:59.289+0000 7f02f97fa700 1 --2- 172.21.15.59:0/310046093 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f02e006af60 0x7f02e006d420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:57:59.294 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:59.289+0000 7f02f97fa700 1 -- 172.21.15.59:0/310046093 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f02f008a600 con 0x7f02fc06b6e0 2023-11-24T07:57:59.294 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:59.289+0000 7f02fbfff700 1 --2- 172.21.15.59:0/310046093 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f02e006af60 0x7f02e006d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:57:59.294 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:59.290+0000 7f02fbfff700 1 --2- 172.21.15.59:0/310046093 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f02e006af60 0x7f02e006d420 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f02fc0b5660 tx=0x7f02e8009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:57:59.295 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:59.293+0000 7f02f97fa700 1 -- 172.21.15.59:0/310046093 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f02f0055b20 con 0x7f02fc06b6e0 2023-11-24T07:57:59.496 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:59.494+0000 7f030322f700 1 -- 172.21.15.59:0/310046093 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f02fc160610 con 0x7f02fc06b6e0 2023-11-24T07:57:59.496 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:59.494+0000 7f02f97fa700 1 -- 172.21.15.59:0/310046093 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v41) v1 ==== 93+0+10 (secure 0 0 0) 0x7f02fc160610 con 0x7f02fc06b6e0 2023-11-24T07:57:59.497 INFO:teuthology.orchestra.run.smithi059.stdout:pg_num: 1 2023-11-24T07:57:59.499 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:59.497+0000 7f030322f700 1 -- 172.21.15.59:0/310046093 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f02e006af60 msgr2=0x7f02e006d420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:57:59.499 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:59.497+0000 7f030322f700 1 --2- 172.21.15.59:0/310046093 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f02e006af60 0x7f02e006d420 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f02fc0b5660 tx=0x7f02e8009040 comp rx=0 tx=0).stop 2023-11-24T07:57:59.499 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:59.497+0000 7f030322f700 1 -- 172.21.15.59:0/310046093 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f02fc06b6e0 msgr2=0x7f02fc142db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:57:59.499 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:59.497+0000 7f030322f700 1 --2- 172.21.15.59:0/310046093 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f02fc06b6e0 0x7f02fc142db0 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7f02f0009b90 tx=0x7f02f0004750 comp rx=0 tx=0).stop 2023-11-24T07:57:59.499 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:59.497+0000 7f030322f700 1 -- 172.21.15.59:0/310046093 shutdown_connections 2023-11-24T07:57:59.499 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:59.497+0000 7f030322f700 1 --2- 172.21.15.59:0/310046093 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f02e006af60 0x7f02e006d420 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:59.499 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:59.497+0000 7f030322f700 1 --2- 172.21.15.59:0/310046093 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f02fc145490 0x7f02fc0b4ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:59.499 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:59.497+0000 7f030322f700 1 --2- 172.21.15.59:0/310046093 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f02fc06b6e0 0x7f02fc142db0 unknown :-1 s=CLOSED pgs=524 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:59.499 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:59.497+0000 7f030322f700 1 --2- 172.21.15.59:0/310046093 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f02fc06f2c0 0x7f02fc0b49b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:59.499 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:59.497+0000 7f030322f700 1 -- 172.21.15.59:0/310046093 >> 172.21.15.59:0/310046093 conn(0x7f02fc0b2740 msgr2=0x7f02fc1446a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:57:59.500 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:59.497+0000 7f030322f700 1 -- 172.21.15.59:0/310046093 shutdown_connections 2023-11-24T07:57:59.500 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:59.497+0000 7f030322f700 1 -- 172.21.15.59:0/310046093 wait complete. 2023-11-24T07:57:59.511 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2023-11-24T07:57:59.637 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:59.635+0000 7f56695cb700 1 Processor -- start 2023-11-24T07:57:59.637 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:59.635+0000 7f56695cb700 1 -- start start 2023-11-24T07:57:59.639 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:59.636+0000 7f56695cb700 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f566406edc0 0x7f56640b5a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:57:59.639 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:59.636+0000 7f56695cb700 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f56640ad860 con 0x7f566406afd0 2023-11-24T07:57:59.639 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:59.636+0000 7f56695cb700 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f56640d0a20 con 0x7f5664071b40 2023-11-24T07:57:59.640 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:59.636+0000 7f56695cb700 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f566406f830 con 0x7f566406edc0 2023-11-24T07:57:59.640 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:59.636+0000 7f5662ffd700 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7f566406afd0 0x7f566406b3b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:56108/0 (socket says 172.21.15.59:56108) 2023-11-24T07:57:59.640 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:59.636+0000 7f5662ffd700 1 -- 172.21.15.59:0/90453691 learned_addr learned my addr 172.21.15.59:0/90453691 (peer_addr_for_me v1:172.21.15.59:0/0) 2023-11-24T07:57:59.640 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:59.636+0000 7f56627fc700 1 --2- 172.21.15.59:0/90453691 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f566406edc0 0x7f56640b5a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:57:59.640 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:59.637+0000 7f5661ffb700 1 -- 172.21.15.59:0/90453691 <== mon.0 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1507616559 0 0) 0x7f56640ad860 con 0x7f566406afd0 2023-11-24T07:57:59.640 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:59.637+0000 7f5661ffb700 1 -- 172.21.15.59:0/90453691 --> v1:172.21.15.59:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f564c0035b0 con 0x7f566406afd0 2023-11-24T07:57:59.640 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:59.637+0000 7f5661ffb700 1 -- 172.21.15.59:0/90453691 <== mon.1 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1745108441 0 0) 0x7f56640d0a20 con 0x7f5664071b40 2023-11-24T07:57:59.640 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:59.637+0000 7f5661ffb700 1 -- 172.21.15.59:0/90453691 --> v1:172.21.15.76:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f56640ad860 con 0x7f5664071b40 2023-11-24T07:57:59.640 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:59.637+0000 7f56627fc700 1 -- 172.21.15.59:0/90453691 >> v1:172.21.15.76:6789/0 conn(0x7f5664071b40 legacy=0x7f5664071ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:57:59.640 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:59.637+0000 7f56627fc700 1 -- 172.21.15.59:0/90453691 >> v1:172.21.15.59:6789/0 conn(0x7f566406afd0 legacy=0x7f566406b3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:57:59.640 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:59.637+0000 7f5661ffb700 1 -- 172.21.15.59:0/90453691 <== mon.0 v1:172.21.15.59:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3641862066 0 0) 0x7f564c0035b0 con 0x7f566406afd0 2023-11-24T07:57:59.641 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:59.637+0000 7f56627fc700 1 -- 172.21.15.59:0/90453691 --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5664162a70 con 0x7f566406edc0 2023-11-24T07:57:59.641 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:59.637+0000 7f56627fc700 1 --2- 172.21.15.59:0/90453691 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f566406edc0 0x7f56640b5a70 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f5654009870 tx=0x7f5654009b80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=91319b6adae10e68 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:57:59.641 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:59.637+0000 7f5661ffb700 1 -- 172.21.15.59:0/90453691 <== mon.2 v2:172.21.15.76:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5658001670 con 0x7f566406edc0 2023-11-24T07:57:59.641 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:59.637+0000 7f5661ffb700 1 -- 172.21.15.59:0/90453691 <== mon.2 v2:172.21.15.76:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5654004030 con 0x7f566406edc0 2023-11-24T07:57:59.641 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:59.637+0000 7f5661ffb700 1 -- 172.21.15.59:0/90453691 <== mon.2 v2:172.21.15.76:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56540043c0 con 0x7f566406edc0 2023-11-24T07:57:59.641 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:59.638+0000 7f56695cb700 1 -- 172.21.15.59:0/90453691 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f566406edc0 msgr2=0x7f56640b5a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:57:59.641 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:59.638+0000 7f56695cb700 1 --2- 172.21.15.59:0/90453691 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f566406edc0 0x7f56640b5a70 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f5654009870 tx=0x7f5654009b80 comp rx=0 tx=0).stop 2023-11-24T07:57:59.641 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:59.638+0000 7f56695cb700 1 -- 172.21.15.59:0/90453691 shutdown_connections 2023-11-24T07:57:59.641 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:59.638+0000 7f56695cb700 1 --2- 172.21.15.59:0/90453691 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f566406edc0 0x7f56640b5a70 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:59.641 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:59.638+0000 7f56695cb700 1 -- 172.21.15.59:0/90453691 >> 172.21.15.59:0/90453691 conn(0x7f56640b2840 msgr2=0x7f56640b2c20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:57:59.641 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:59.638+0000 7f56695cb700 1 -- 172.21.15.59:0/90453691 shutdown_connections 2023-11-24T07:57:59.641 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:59.638+0000 7f56695cb700 1 -- 172.21.15.59:0/90453691 wait complete. 2023-11-24T07:57:59.644 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:59.639+0000 7f56695cb700 1 Processor -- start 2023-11-24T07:57:59.644 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:59.639+0000 7f56695cb700 1 -- start start 2023-11-24T07:57:59.644 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:59.640+0000 7f56695cb700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f566406afd0 0x7f5664156a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:57:59.644 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:59.640+0000 7f56695cb700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f566406edc0 0x7f56641512b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:57:59.644 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:59.640+0000 7f56695cb700 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f5664071b40 0x7f5664160760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:57:59.644 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:59.640+0000 7f56695cb700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56641f37c0 con 0x7f566406edc0 2023-11-24T07:57:59.644 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:59.640+0000 7f56695cb700 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5664156580 con 0x7f566406afd0 2023-11-24T07:57:59.645 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:59.640+0000 7f56695cb700 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5664155f70 con 0x7f5664071b40 2023-11-24T07:57:59.645 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:59.640+0000 7f56627fc700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f566406edc0 0x7f56641512b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:57:59.645 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:59.640+0000 7f56627fc700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f566406edc0 0x7f56641512b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:52642/0 (socket says 172.21.15.59:52642) 2023-11-24T07:57:59.645 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:59.640+0000 7f56627fc700 1 -- 172.21.15.59:0/3813354918 learned_addr learned my addr 172.21.15.59:0/3813354918 (peer_addr_for_me v2:172.21.15.59:0/0) 2023-11-24T07:57:59.645 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:59.640+0000 7f56637fe700 1 --2- 172.21.15.59:0/3813354918 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f5664071b40 0x7f5664160760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:57:59.645 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:59.640+0000 7f5662ffd700 1 --2- 172.21.15.59:0/3813354918 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f566406afd0 0x7f5664156a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:57:59.645 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:59.640+0000 7f56627fc700 1 -- 172.21.15.59:0/3813354918 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f5664071b40 msgr2=0x7f5664160760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:57:59.645 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:59.640+0000 7f56627fc700 1 --2- 172.21.15.59:0/3813354918 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f5664071b40 0x7f5664160760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:59.645 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:59.640+0000 7f56627fc700 1 -- 172.21.15.59:0/3813354918 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f566406afd0 msgr2=0x7f5664156a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:57:59.645 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:59.640+0000 7f56627fc700 1 --2- 172.21.15.59:0/3813354918 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f566406afd0 0x7f5664156a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:59.645 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:59.640+0000 7f56627fc700 1 -- 172.21.15.59:0/3813354918 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f56540094a0 con 0x7f566406edc0 2023-11-24T07:57:59.645 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:59.640+0000 7f56627fc700 1 --2- 172.21.15.59:0/3813354918 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f566406edc0 0x7f56641512b0 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7f5654004000 tx=0x7f5654004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:57:59.646 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:59.641+0000 7f564b7fe700 1 -- 172.21.15.59:0/3813354918 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5654004570 con 0x7f566406edc0 2023-11-24T07:57:59.646 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:59.641+0000 7f564b7fe700 1 -- 172.21.15.59:0/3813354918 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f565401f070 con 0x7f566406edc0 2023-11-24T07:57:59.646 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:59.641+0000 7f564b7fe700 1 -- 172.21.15.59:0/3813354918 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5654011850 con 0x7f566406edc0 2023-11-24T07:57:59.646 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:59.641+0000 7f56695cb700 1 -- 172.21.15.59:0/3813354918 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f566415e690 con 0x7f566406edc0 2023-11-24T07:57:59.646 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:59.641+0000 7f56695cb700 1 -- 172.21.15.59:0/3813354918 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f566415e970 con 0x7f566406edc0 2023-11-24T07:57:59.646 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:59.642+0000 7f56695cb700 1 -- 172.21.15.59:0/3813354918 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5664156710 con 0x7f566406edc0 2023-11-24T07:57:59.646 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:59.642+0000 7f564b7fe700 1 -- 172.21.15.59:0/3813354918 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 6) v1 ==== 89282+0+0 (secure 0 0 0) 0x7f56540119b0 con 0x7f566406edc0 2023-11-24T07:57:59.646 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:59.642+0000 7f564b7fe700 1 --2- 172.21.15.59:0/3813354918 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f565006af60 0x7f565006d420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:57:59.646 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:59.643+0000 7f564b7fe700 1 -- 172.21.15.59:0/3813354918 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f5654021070 con 0x7f566406edc0 2023-11-24T07:57:59.646 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:59.643+0000 7f5662ffd700 1 --2- 172.21.15.59:0/3813354918 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f565006af60 0x7f565006d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:57:59.647 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:59.643+0000 7f5662ffd700 1 --2- 172.21.15.59:0/3813354918 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f565006af60 0x7f565006d420 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f5658000f60 tx=0x7f5658001690 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:57:59.648 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:59.647+0000 7f564b7fe700 1 -- 172.21.15.59:0/3813354918 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f5654055f50 con 0x7f566406edc0 2023-11-24T07:57:59.854 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:59.852+0000 7f56695cb700 1 -- 172.21.15.59:0/3813354918 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f5664068670 con 0x7f566406edc0 2023-11-24T07:57:59.854 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:59.852+0000 7f564b7fe700 1 -- 172.21.15.59:0/3813354918 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v41) v1 ==== 84+0+65 (secure 0 0 0) 0x7f565402b090 con 0x7f566406edc0 2023-11-24T07:57:59.854 INFO:teuthology.orchestra.run.smithi059.stdout: 2023-11-24T07:57:59.854 INFO:teuthology.orchestra.run.smithi059.stdout:[ 2023-11-24T07:57:59.854 INFO:teuthology.orchestra.run.smithi059.stdout: { 2023-11-24T07:57:59.854 INFO:teuthology.orchestra.run.smithi059.stdout: "poolnum": 1, 2023-11-24T07:57:59.854 INFO:teuthology.orchestra.run.smithi059.stdout: "poolname": ".mgr" 2023-11-24T07:57:59.855 INFO:teuthology.orchestra.run.smithi059.stdout: } 2023-11-24T07:57:59.855 INFO:teuthology.orchestra.run.smithi059.stdout:] 2023-11-24T07:57:59.856 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:59.854+0000 7f56695cb700 1 -- 172.21.15.59:0/3813354918 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f565006af60 msgr2=0x7f565006d420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:57:59.857 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:59.854+0000 7f56695cb700 1 --2- 172.21.15.59:0/3813354918 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f565006af60 0x7f565006d420 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f5658000f60 tx=0x7f5658001690 comp rx=0 tx=0).stop 2023-11-24T07:57:59.857 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:59.854+0000 7f56695cb700 1 -- 172.21.15.59:0/3813354918 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f566406edc0 msgr2=0x7f56641512b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:57:59.857 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:59.854+0000 7f56695cb700 1 --2- 172.21.15.59:0/3813354918 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f566406edc0 0x7f56641512b0 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7f5654004000 tx=0x7f5654004750 comp rx=0 tx=0).stop 2023-11-24T07:57:59.857 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:59.855+0000 7f56695cb700 1 -- 172.21.15.59:0/3813354918 shutdown_connections 2023-11-24T07:57:59.857 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:59.855+0000 7f56695cb700 1 --2- 172.21.15.59:0/3813354918 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f565006af60 0x7f565006d420 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:59.857 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:59.855+0000 7f56695cb700 1 --2- 172.21.15.59:0/3813354918 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f5664071b40 0x7f5664160760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:59.857 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:59.855+0000 7f56695cb700 1 --2- 172.21.15.59:0/3813354918 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f566406afd0 0x7f5664156a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:59.857 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:59.855+0000 7f56695cb700 1 --2- 172.21.15.59:0/3813354918 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f566406edc0 0x7f56641512b0 unknown :-1 s=CLOSED pgs=526 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:57:59.858 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:59.855+0000 7f56695cb700 1 -- 172.21.15.59:0/3813354918 >> 172.21.15.59:0/3813354918 conn(0x7f56640b2840 msgr2=0x7f566406b5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:57:59.858 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:59.855+0000 7f56695cb700 1 -- 172.21.15.59:0/3813354918 shutdown_connections 2023-11-24T07:57:59.858 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:59.855+0000 7f56695cb700 1 -- 172.21.15.59:0/3813354918 wait complete. 2023-11-24T07:57:59.868 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2023-11-24T07:57:59.869 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2023-11-24T07:57:59.998 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:59.996+0000 7f8754cd8700 1 Processor -- start 2023-11-24T07:57:59.998 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:59.996+0000 7f8754cd8700 1 -- start start 2023-11-24T07:58:00.001 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:59.997+0000 7f8754cd8700 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f87501457b0 0x7f87500725a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:00.001 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:59.997+0000 7f8754cd8700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f875006f580 0x7f875006e3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:00.001 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:59.997+0000 7f8754cd8700 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f875006e910 0x7f87500cf020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:00.001 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:59.997+0000 7f8754cd8700 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f87500cd490 con 0x7f875006f580 2023-11-24T07:58:00.001 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:59.997+0000 7f8754cd8700 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f875006fa00 con 0x7f875006e910 2023-11-24T07:58:00.001 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:59.997+0000 7f8754cd8700 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f87500cd6e0 con 0x7f87501457b0 2023-11-24T07:58:00.001 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:59.997+0000 7f874dd9b700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f875006f580 0x7f875006e3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:00.001 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:59.998+0000 7f874dd9b700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f875006f580 0x7f875006e3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:52648/0 (socket says 172.21.15.59:52648) 2023-11-24T07:58:00.001 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:59.998+0000 7f874dd9b700 1 -- 172.21.15.59:0/640740762 learned_addr learned my addr 172.21.15.59:0/640740762 (peer_addr_for_me v2:172.21.15.59:0/0) 2023-11-24T07:58:00.002 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:59.998+0000 7f874ed9d700 1 --2- 172.21.15.59:0/640740762 >> v2:172.21.15.76:3300/0 conn(0x7f875006e910 0x7f87500cf020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:00.002 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:59.998+0000 7f874e59c700 1 --2- 172.21.15.59:0/640740762 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f87501457b0 0x7f87500725a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:00.002 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:59.998+0000 7f874dd9b700 1 -- 172.21.15.59:0/640740762 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f87501457b0 msgr2=0x7f87500725a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T07:58:00.002 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:59.998+0000 7f874dd9b700 1 --2- 172.21.15.59:0/640740762 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f87501457b0 0x7f87500725a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:00.002 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:59.998+0000 7f874dd9b700 1 -- 172.21.15.59:0/640740762 >> v2:172.21.15.76:3300/0 conn(0x7f875006e910 msgr2=0x7f87500cf020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T07:58:00.002 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:59.998+0000 7f874dd9b700 1 --2- 172.21.15.59:0/640740762 >> v2:172.21.15.76:3300/0 conn(0x7f875006e910 0x7f87500cf020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:00.002 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:59.998+0000 7f874dd9b700 1 -- 172.21.15.59:0/640740762 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f87500cd9c0 con 0x7f875006f580 2023-11-24T07:58:00.002 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:59.998+0000 7f874dd9b700 1 --2- 172.21.15.59:0/640740762 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f875006f580 0x7f875006e3d0 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7f8750071520 tx=0x7f8740009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d0a8b7c526d25477 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:58:00.002 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:59.998+0000 7f874d59a700 1 -- 172.21.15.59:0/640740762 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8740011830 con 0x7f875006f580 2023-11-24T07:58:00.002 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:59.998+0000 7f874d59a700 1 -- 172.21.15.59:0/640740762 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8740011990 con 0x7f875006f580 2023-11-24T07:58:00.002 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:59.998+0000 7f874d59a700 1 -- 172.21.15.59:0/640740762 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8740011d20 con 0x7f875006f580 2023-11-24T07:58:00.003 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:59.999+0000 7f8754cd8700 1 -- 172.21.15.59:0/640740762 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f875006f580 msgr2=0x7f875006e3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:00.003 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:59.999+0000 7f8754cd8700 1 --2- 172.21.15.59:0/640740762 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f875006f580 0x7f875006e3d0 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7f8750071520 tx=0x7f8740009bd0 comp rx=0 tx=0).stop 2023-11-24T07:58:00.003 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:59.999+0000 7f8754cd8700 1 -- 172.21.15.59:0/640740762 shutdown_connections 2023-11-24T07:58:00.003 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:59.999+0000 7f8754cd8700 1 --2- 172.21.15.59:0/640740762 >> v2:172.21.15.76:3300/0 conn(0x7f875006e910 0x7f87500cf020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:00.003 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:59.999+0000 7f8754cd8700 1 --2- 172.21.15.59:0/640740762 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f87501457b0 0x7f87500725a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:00.003 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:59.999+0000 7f8754cd8700 1 --2- 172.21.15.59:0/640740762 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f875006f580 0x7f875006e3d0 unknown :-1 s=CLOSED pgs=527 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:00.003 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:59.999+0000 7f8754cd8700 1 -- 172.21.15.59:0/640740762 >> 172.21.15.59:0/640740762 conn(0x7f87500b2760 msgr2=0x7f87500b2b70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:58:00.003 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:59.999+0000 7f8754cd8700 1 -- 172.21.15.59:0/640740762 shutdown_connections 2023-11-24T07:58:00.003 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:57:59.999+0000 7f8754cd8700 1 -- 172.21.15.59:0/640740762 wait complete. 2023-11-24T07:58:00.006 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:00.000+0000 7f8754cd8700 1 Processor -- start 2023-11-24T07:58:00.006 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:00.000+0000 7f8754cd8700 1 -- start start 2023-11-24T07:58:00.006 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:00.001+0000 7f8754cd8700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f875006e910 0x7f87501365c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:00.006 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:00.001+0000 7f8754cd8700 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f875006f580 0x7f8750137440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:00.007 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:00.001+0000 7f8754cd8700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f87501457b0 0x7f8750071b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:00.007 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:00.001+0000 7f8754cd8700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f875015d890 con 0x7f87501457b0 2023-11-24T07:58:00.007 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:00.001+0000 7f8754cd8700 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8750136b00 con 0x7f875006e910 2023-11-24T07:58:00.007 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:00.001+0000 7f8754cd8700 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8750137c70 con 0x7f875006f580 2023-11-24T07:58:00.007 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:00.001+0000 7f874ed9d700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f87501457b0 0x7f8750071b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:00.007 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:00.001+0000 7f874ed9d700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f87501457b0 0x7f8750071b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:52654/0 (socket says 172.21.15.59:52654) 2023-11-24T07:58:00.007 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:00.001+0000 7f874ed9d700 1 -- 172.21.15.59:0/2917244612 learned_addr learned my addr 172.21.15.59:0/2917244612 (peer_addr_for_me v2:172.21.15.59:0/0) 2023-11-24T07:58:00.007 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:00.001+0000 7f874e59c700 1 --2- 172.21.15.59:0/2917244612 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f875006e910 0x7f87501365c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:00.007 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:00.001+0000 7f874dd9b700 1 --2- 172.21.15.59:0/2917244612 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f875006f580 0x7f8750137440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:00.007 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:00.001+0000 7f874ed9d700 1 -- 172.21.15.59:0/2917244612 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f875006f580 msgr2=0x7f8750137440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:00.007 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:00.001+0000 7f874ed9d700 1 --2- 172.21.15.59:0/2917244612 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f875006f580 0x7f8750137440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:00.007 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:00.001+0000 7f874ed9d700 1 -- 172.21.15.59:0/2917244612 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f875006e910 msgr2=0x7f87501365c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:00.008 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:00.001+0000 7f874ed9d700 1 --2- 172.21.15.59:0/2917244612 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f875006e910 0x7f87501365c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:00.008 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:00.001+0000 7f874ed9d700 1 -- 172.21.15.59:0/2917244612 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8740009550 con 0x7f87501457b0 2023-11-24T07:58:00.008 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:00.001+0000 7f874dd9b700 1 --2- 172.21.15.59:0/2917244612 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f875006f580 0x7f8750137440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-24T07:58:00.008 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:00.002+0000 7f874ed9d700 1 --2- 172.21.15.59:0/2917244612 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f87501457b0 0x7f8750071b90 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7f874800eed0 tx=0x7f874800b390 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:58:00.008 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:00.002+0000 7f873effd700 1 -- 172.21.15.59:0/2917244612 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8748010040 con 0x7f87501457b0 2023-11-24T07:58:00.008 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:00.002+0000 7f873effd700 1 -- 172.21.15.59:0/2917244612 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f874800bd30 con 0x7f87501457b0 2023-11-24T07:58:00.008 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:00.002+0000 7f873effd700 1 -- 172.21.15.59:0/2917244612 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f874800bea0 con 0x7f87501457b0 2023-11-24T07:58:00.008 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:00.002+0000 7f8754cd8700 1 -- 172.21.15.59:0/2917244612 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f875015cf60 con 0x7f87501457b0 2023-11-24T07:58:00.008 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:00.002+0000 7f8754cd8700 1 -- 172.21.15.59:0/2917244612 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f875015d520 con 0x7f87501457b0 2023-11-24T07:58:00.008 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:00.003+0000 7f8754cd8700 1 -- 172.21.15.59:0/2917244612 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8750068670 con 0x7f87501457b0 2023-11-24T07:58:00.008 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:00.003+0000 7f873effd700 1 -- 172.21.15.59:0/2917244612 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 6) v1 ==== 89282+0+0 (secure 0 0 0) 0x7f8748014620 con 0x7f87501457b0 2023-11-24T07:58:00.008 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:00.004+0000 7f873effd700 1 --2- 172.21.15.59:0/2917244612 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f873006af60 0x7f873006d420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:00.009 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:00.004+0000 7f873effd700 1 -- 172.21.15.59:0/2917244612 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f87480898b0 con 0x7f87501457b0 2023-11-24T07:58:00.009 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:00.004+0000 7f874e59c700 1 --2- 172.21.15.59:0/2917244612 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f873006af60 0x7f873006d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:00.009 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:00.004+0000 7f874e59c700 1 --2- 172.21.15.59:0/2917244612 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f873006af60 0x7f873006d420 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f8744006b20 tx=0x7f8744008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:58:00.010 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:00.008+0000 7f873effd700 1 -- 172.21.15.59:0/2917244612 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f8748054bb0 con 0x7f87501457b0 2023-11-24T07:58:00.210 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:00.209+0000 7f8754cd8700 1 -- 172.21.15.59:0/2917244612 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7f875015d9d0 con 0x7f87501457b0 2023-11-24T07:58:00.246 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-24T07:58:00.249 DEBUG:teuthology.orchestra.run.smithi076:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-24T07:58:00.583 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:00.582+0000 7f873effd700 1 -- 172.21.15.59:0/2917244612 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v42) v1 ==== 138+0+0 (secure 0 0 0) 0x7f87480180b0 con 0x7f87501457b0 2023-11-24T07:58:00.586 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:00.584+0000 7f8754cd8700 1 -- 172.21.15.59:0/2917244612 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f873006af60 msgr2=0x7f873006d420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:00.586 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:00.584+0000 7f8754cd8700 1 --2- 172.21.15.59:0/2917244612 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f873006af60 0x7f873006d420 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f8744006b20 tx=0x7f8744008040 comp rx=0 tx=0).stop 2023-11-24T07:58:00.586 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:00.584+0000 7f8754cd8700 1 -- 172.21.15.59:0/2917244612 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f87501457b0 msgr2=0x7f8750071b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:00.587 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:00.584+0000 7f8754cd8700 1 --2- 172.21.15.59:0/2917244612 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f87501457b0 0x7f8750071b90 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7f874800eed0 tx=0x7f874800b390 comp rx=0 tx=0).stop 2023-11-24T07:58:00.587 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:00.584+0000 7f8754cd8700 1 -- 172.21.15.59:0/2917244612 shutdown_connections 2023-11-24T07:58:00.587 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:00.584+0000 7f8754cd8700 1 --2- 172.21.15.59:0/2917244612 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f873006af60 0x7f873006d420 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:00.587 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:00.584+0000 7f8754cd8700 1 --2- 172.21.15.59:0/2917244612 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f87501457b0 0x7f8750071b90 unknown :-1 s=CLOSED pgs=528 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:00.587 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:00.584+0000 7f8754cd8700 1 --2- 172.21.15.59:0/2917244612 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f875006e910 0x7f87501365c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:00.587 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:00.584+0000 7f8754cd8700 1 --2- 172.21.15.59:0/2917244612 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f875006f580 0x7f8750137440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:00.587 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:00.584+0000 7f8754cd8700 1 -- 172.21.15.59:0/2917244612 >> 172.21.15.59:0/2917244612 conn(0x7f87500b2760 msgr2=0x7f875006afd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:58:00.587 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:00.584+0000 7f8754cd8700 1 -- 172.21.15.59:0/2917244612 shutdown_connections 2023-11-24T07:58:00.587 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:00.584+0000 7f8754cd8700 1 -- 172.21.15.59:0/2917244612 wait complete. 2023-11-24T07:58:00.588 INFO:teuthology.orchestra.run.smithi059.stderr:pool 'cephfs_metadata' created 2023-11-24T07:58:00.602 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2023-11-24T07:58:00.735 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:00.733+0000 7fcfc6e26700 1 Processor -- start 2023-11-24T07:58:00.735 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:00.733+0000 7fcfc6e26700 1 -- start start 2023-11-24T07:58:00.737 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:00.734+0000 7fcfc6e26700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fcfc0143ae0 0x7fcfc00b51b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:00.737 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:00.734+0000 7fcfc6e26700 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fcfc01440f0 0x7fcfc0071bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:00.738 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:00.734+0000 7fcfc6e26700 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fcfc00720f0 0x7fcfc015e8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:00.738 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:00.734+0000 7fcfc6e26700 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcfc00b40a0 con 0x7fcfc0143ae0 2023-11-24T07:58:00.738 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:00.734+0000 7fcfc6e26700 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcfc0072580 con 0x7fcfc00720f0 2023-11-24T07:58:00.738 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:00.734+0000 7fcfc6e26700 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcfc0143ec0 con 0x7fcfc01440f0 2023-11-24T07:58:00.738 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:00.734+0000 7fcfc4bc2700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fcfc0143ae0 0x7fcfc00b51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:00.738 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:00.734+0000 7fcfc4bc2700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fcfc0143ae0 0x7fcfc00b51b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:52658/0 (socket says 172.21.15.59:52658) 2023-11-24T07:58:00.738 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:00.734+0000 7fcfc4bc2700 1 -- 172.21.15.59:0/54927440 learned_addr learned my addr 172.21.15.59:0/54927440 (peer_addr_for_me v2:172.21.15.59:0/0) 2023-11-24T07:58:00.738 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:00.734+0000 7fcfbffff700 1 --2- 172.21.15.59:0/54927440 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fcfc01440f0 0x7fcfc0071bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:00.738 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:00.734+0000 7fcfc53c3700 1 --2- 172.21.15.59:0/54927440 >> v2:172.21.15.76:3300/0 conn(0x7fcfc00720f0 0x7fcfc015e8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:00.739 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:00.734+0000 7fcfc4bc2700 1 -- 172.21.15.59:0/54927440 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fcfc01440f0 msgr2=0x7fcfc0071bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T07:58:00.739 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:00.734+0000 7fcfc4bc2700 1 --2- 172.21.15.59:0/54927440 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fcfc01440f0 0x7fcfc0071bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:00.739 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:00.734+0000 7fcfc4bc2700 1 -- 172.21.15.59:0/54927440 >> v2:172.21.15.76:3300/0 conn(0x7fcfc00720f0 msgr2=0x7fcfc015e8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T07:58:00.739 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:00.734+0000 7fcfc4bc2700 1 --2- 172.21.15.59:0/54927440 >> v2:172.21.15.76:3300/0 conn(0x7fcfc00720f0 0x7fcfc015e8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:00.739 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:00.734+0000 7fcfc4bc2700 1 -- 172.21.15.59:0/54927440 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcfc00b3ae0 con 0x7fcfc0143ae0 2023-11-24T07:58:00.739 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:00.735+0000 7fcfc4bc2700 1 --2- 172.21.15.59:0/54927440 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fcfc0143ae0 0x7fcfc00b51b0 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7fcfc00706c0 tx=0x7fcfb4009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5622a1fb1ee7a281 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:58:00.739 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:00.735+0000 7fcfbf7fe700 1 -- 172.21.15.59:0/54927440 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcfb400b870 con 0x7fcfc0143ae0 2023-11-24T07:58:00.739 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:00.735+0000 7fcfbf7fe700 1 -- 172.21.15.59:0/54927440 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcfb400b9d0 con 0x7fcfc0143ae0 2023-11-24T07:58:00.739 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:00.735+0000 7fcfbf7fe700 1 -- 172.21.15.59:0/54927440 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcfb400bd60 con 0x7fcfc0143ae0 2023-11-24T07:58:00.739 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:00.735+0000 7fcfc6e26700 1 -- 172.21.15.59:0/54927440 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fcfc0143ae0 msgr2=0x7fcfc00b51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:00.740 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:00.735+0000 7fcfc6e26700 1 --2- 172.21.15.59:0/54927440 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fcfc0143ae0 0x7fcfc00b51b0 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7fcfc00706c0 tx=0x7fcfb4009bd0 comp rx=0 tx=0).stop 2023-11-24T07:58:00.740 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:00.735+0000 7fcfc6e26700 1 -- 172.21.15.59:0/54927440 shutdown_connections 2023-11-24T07:58:00.740 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:00.735+0000 7fcfc6e26700 1 --2- 172.21.15.59:0/54927440 >> v2:172.21.15.76:3300/0 conn(0x7fcfc00720f0 0x7fcfc015e8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:00.740 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:00.735+0000 7fcfc6e26700 1 --2- 172.21.15.59:0/54927440 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fcfc0143ae0 0x7fcfc00b51b0 unknown :-1 s=CLOSED pgs=529 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:00.740 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:00.735+0000 7fcfc6e26700 1 --2- 172.21.15.59:0/54927440 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fcfc01440f0 0x7fcfc0071bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:00.740 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:00.735+0000 7fcfc6e26700 1 -- 172.21.15.59:0/54927440 >> 172.21.15.59:0/54927440 conn(0x7fcfc00b2740 msgr2=0x7fcfc00b2b20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:58:00.740 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:00.735+0000 7fcfc6e26700 1 -- 172.21.15.59:0/54927440 shutdown_connections 2023-11-24T07:58:00.740 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:00.735+0000 7fcfc6e26700 1 -- 172.21.15.59:0/54927440 wait complete. 2023-11-24T07:58:00.743 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:00.737+0000 7fcfc6e26700 1 Processor -- start 2023-11-24T07:58:00.743 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:00.737+0000 7fcfc6e26700 1 -- start start 2023-11-24T07:58:00.744 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:00.737+0000 7fcfc6e26700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fcfc00720f0 0x7fcfc0131d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:00.744 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:00.737+0000 7fcfc6e26700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fcfc0143ae0 0x7fcfc01303a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:00.744 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:00.737+0000 7fcfc6e26700 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fcfc01440f0 0x7fcfc01326a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:00.744 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:00.737+0000 7fcfc6e26700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcfc0132240 con 0x7fcfc0143ae0 2023-11-24T07:58:00.744 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:00.737+0000 7fcfc6e26700 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcfc0130c60 con 0x7fcfc00720f0 2023-11-24T07:58:00.744 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:00.737+0000 7fcfc6e26700 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcfc0130f40 con 0x7fcfc01440f0 2023-11-24T07:58:00.744 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:00.738+0000 7fcfbffff700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fcfc0143ae0 0x7fcfc01303a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:00.744 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:00.738+0000 7fcfbffff700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fcfc0143ae0 0x7fcfc01303a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:52664/0 (socket says 172.21.15.59:52664) 2023-11-24T07:58:00.744 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:00.738+0000 7fcfbffff700 1 -- 172.21.15.59:0/3049585362 learned_addr learned my addr 172.21.15.59:0/3049585362 (peer_addr_for_me v2:172.21.15.59:0/0) 2023-11-24T07:58:00.744 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:00.738+0000 7fcfc53c3700 1 --2- 172.21.15.59:0/3049585362 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fcfc01440f0 0x7fcfc01326a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:00.745 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:00.738+0000 7fcfc4bc2700 1 --2- 172.21.15.59:0/3049585362 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fcfc00720f0 0x7fcfc0131d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:00.745 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:00.738+0000 7fcfbffff700 1 -- 172.21.15.59:0/3049585362 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fcfc01440f0 msgr2=0x7fcfc01326a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:00.745 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:00.738+0000 7fcfbffff700 1 --2- 172.21.15.59:0/3049585362 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fcfc01440f0 0x7fcfc01326a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:00.745 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:00.738+0000 7fcfbffff700 1 -- 172.21.15.59:0/3049585362 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fcfc00720f0 msgr2=0x7fcfc0131d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:00.745 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:00.738+0000 7fcfbffff700 1 --2- 172.21.15.59:0/3049585362 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fcfc00720f0 0x7fcfc0131d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:00.745 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:00.738+0000 7fcfbffff700 1 -- 172.21.15.59:0/3049585362 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcfb4009550 con 0x7fcfc0143ae0 2023-11-24T07:58:00.745 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:00.738+0000 7fcfc53c3700 1 --2- 172.21.15.59:0/3049585362 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fcfc01440f0 0x7fcfc01326a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-24T07:58:00.745 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:00.738+0000 7fcfc4bc2700 1 --2- 172.21.15.59:0/3049585362 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fcfc00720f0 0x7fcfc0131d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-24T07:58:00.745 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:00.738+0000 7fcfbffff700 1 --2- 172.21.15.59:0/3049585362 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fcfc0143ae0 0x7fcfc01303a0 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7fcfac00e810 tx=0x7fcfac00ebd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:58:00.745 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:00.738+0000 7fcfbd7fa700 1 -- 172.21.15.59:0/3049585362 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcfac013610 con 0x7fcfc0143ae0 2023-11-24T07:58:00.746 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:00.738+0000 7fcfbd7fa700 1 -- 172.21.15.59:0/3049585362 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcfac013770 con 0x7fcfc0143ae0 2023-11-24T07:58:00.746 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:00.738+0000 7fcfc6e26700 1 -- 172.21.15.59:0/3049585362 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcfc0131220 con 0x7fcfc0143ae0 2023-11-24T07:58:00.746 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:00.738+0000 7fcfbd7fa700 1 -- 172.21.15.59:0/3049585362 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcfac013b20 con 0x7fcfc0143ae0 2023-11-24T07:58:00.746 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:00.738+0000 7fcfc6e26700 1 -- 172.21.15.59:0/3049585362 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcfc01317e0 con 0x7fcfc0143ae0 2023-11-24T07:58:00.746 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:00.739+0000 7fcfc6e26700 1 -- 172.21.15.59:0/3049585362 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcfc0075480 con 0x7fcfc0143ae0 2023-11-24T07:58:00.746 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:00.740+0000 7fcfbd7fa700 1 -- 172.21.15.59:0/3049585362 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 6) v1 ==== 89282+0+0 (secure 0 0 0) 0x7fcfac013c80 con 0x7fcfc0143ae0 2023-11-24T07:58:00.746 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:00.740+0000 7fcfbd7fa700 1 --2- 172.21.15.59:0/3049585362 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7fcfa406af60 0x7fcfa406d420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:00.746 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:00.740+0000 7fcfbd7fa700 1 -- 172.21.15.59:0/3049585362 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 5227+0+0 (secure 0 0 0) 0x7fcfac089130 con 0x7fcfc0143ae0 2023-11-24T07:58:00.746 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:00.741+0000 7fcfc4bc2700 1 --2- 172.21.15.59:0/3049585362 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7fcfa406af60 0x7fcfa406d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:00.746 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:00.741+0000 7fcfc4bc2700 1 --2- 172.21.15.59:0/3049585362 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7fcfa406af60 0x7fcfa406d420 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7fcfb4014040 tx=0x7fcfb4008030 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:58:00.747 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:00.745+0000 7fcfbd7fa700 1 -- 172.21.15.59:0/3049585362 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fcfac0542d0 con 0x7fcfc0143ae0 2023-11-24T07:58:00.948 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:00.946+0000 7fcfc6e26700 1 -- 172.21.15.59:0/3049585362 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7fcfc0144830 con 0x7fcfc0143ae0 2023-11-24T07:58:01.593 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:01.592+0000 7fcfbd7fa700 1 -- 172.21.15.59:0/3049585362 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v43) v1 ==== 170+0+0 (secure 0 0 0) 0x7fcfac0578f0 con 0x7fcfc0143ae0 2023-11-24T07:58:01.597 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:01.595+0000 7fcfc6e26700 1 -- 172.21.15.59:0/3049585362 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7fcfa406af60 msgr2=0x7fcfa406d420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:01.597 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:01.595+0000 7fcfc6e26700 1 --2- 172.21.15.59:0/3049585362 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7fcfa406af60 0x7fcfa406d420 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7fcfb4014040 tx=0x7fcfb4008030 comp rx=0 tx=0).stop 2023-11-24T07:58:01.597 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:01.595+0000 7fcfc6e26700 1 -- 172.21.15.59:0/3049585362 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fcfc0143ae0 msgr2=0x7fcfc01303a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:01.597 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:01.595+0000 7fcfc6e26700 1 --2- 172.21.15.59:0/3049585362 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fcfc0143ae0 0x7fcfc01303a0 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7fcfac00e810 tx=0x7fcfac00ebd0 comp rx=0 tx=0).stop 2023-11-24T07:58:01.598 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:01.596+0000 7fcfc6e26700 1 -- 172.21.15.59:0/3049585362 shutdown_connections 2023-11-24T07:58:01.598 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:01.596+0000 7fcfc6e26700 1 --2- 172.21.15.59:0/3049585362 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7fcfa406af60 0x7fcfa406d420 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:01.598 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:01.596+0000 7fcfc6e26700 1 --2- 172.21.15.59:0/3049585362 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fcfc01440f0 0x7fcfc01326a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:01.598 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:01.596+0000 7fcfc6e26700 1 --2- 172.21.15.59:0/3049585362 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fcfc00720f0 0x7fcfc0131d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:01.598 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:01.596+0000 7fcfc6e26700 1 --2- 172.21.15.59:0/3049585362 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fcfc0143ae0 0x7fcfc01303a0 unknown :-1 s=CLOSED pgs=530 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:01.598 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:01.596+0000 7fcfc6e26700 1 -- 172.21.15.59:0/3049585362 >> 172.21.15.59:0/3049585362 conn(0x7fcfc00b2740 msgr2=0x7fcfc015b3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:58:01.599 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:01.598+0000 7fcfc6e26700 1 -- 172.21.15.59:0/3049585362 shutdown_connections 2023-11-24T07:58:01.600 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:01.598+0000 7fcfc6e26700 1 -- 172.21.15.59:0/3049585362 wait complete. 2023-11-24T07:58:01.611 INFO:teuthology.orchestra.run.smithi059.stderr:pool 'cephfs_data' created 2023-11-24T07:58:01.629 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2023-11-24T07:58:01.762 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:01.760+0000 7f59222bd700 1 Processor -- start 2023-11-24T07:58:01.762 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:01.761+0000 7f59222bd700 1 -- start start 2023-11-24T07:58:01.765 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:01.761+0000 7f59222bd700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f591c15d7f0 0x7f591c06d2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:01.765 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:01.761+0000 7f59222bd700 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f591c06d800 0x7f591c150bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:01.765 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:01.761+0000 7f59222bd700 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f591c151110 0x7f591c151560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:01.765 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:01.761+0000 7f59222bd700 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f591c06dc80 con 0x7f591c15d7f0 2023-11-24T07:58:01.765 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:01.761+0000 7f59222bd700 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f591c0b39b0 con 0x7f591c151110 2023-11-24T07:58:01.765 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:01.761+0000 7f59222bd700 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f591c0cf540 con 0x7f591c06d800 2023-11-24T07:58:01.765 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:01.761+0000 7f591bfff700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f591c15d7f0 0x7f591c06d2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:01.766 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:01.762+0000 7f591bfff700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f591c15d7f0 0x7f591c06d2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:52670/0 (socket says 172.21.15.59:52670) 2023-11-24T07:58:01.766 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:01.762+0000 7f591bfff700 1 -- 172.21.15.59:0/466769527 learned_addr learned my addr 172.21.15.59:0/466769527 (peer_addr_for_me v2:172.21.15.59:0/0) 2023-11-24T07:58:01.766 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:01.762+0000 7f591b7fe700 1 --2- 172.21.15.59:0/466769527 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f591c06d800 0x7f591c150bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:01.766 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:01.762+0000 7f591bfff700 1 -- 172.21.15.59:0/466769527 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f591c06d800 msgr2=0x7f591c150bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T07:58:01.766 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:01.762+0000 7f592085a700 1 --2- 172.21.15.59:0/466769527 >> v2:172.21.15.76:3300/0 conn(0x7f591c151110 0x7f591c151560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:01.766 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:01.762+0000 7f591bfff700 1 --2- 172.21.15.59:0/466769527 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f591c06d800 0x7f591c150bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:01.766 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:01.762+0000 7f591bfff700 1 -- 172.21.15.59:0/466769527 >> v2:172.21.15.76:3300/0 conn(0x7f591c151110 msgr2=0x7f591c151560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T07:58:01.766 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:01.762+0000 7f591bfff700 1 --2- 172.21.15.59:0/466769527 >> v2:172.21.15.76:3300/0 conn(0x7f591c151110 0x7f591c151560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:01.766 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:01.762+0000 7f591bfff700 1 -- 172.21.15.59:0/466769527 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f591c0b2da0 con 0x7f591c15d7f0 2023-11-24T07:58:01.766 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:01.762+0000 7f591b7fe700 1 --2- 172.21.15.59:0/466769527 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f591c06d800 0x7f591c150bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T07:58:01.767 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:01.762+0000 7f592085a700 1 --2- 172.21.15.59:0/466769527 >> v2:172.21.15.76:3300/0 conn(0x7f591c151110 0x7f591c151560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T07:58:01.767 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:01.762+0000 7f591bfff700 1 --2- 172.21.15.59:0/466769527 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f591c15d7f0 0x7f591c06d2c0 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7f591c0cf840 tx=0x7f590400be70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=17518c597864b7f1 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:58:01.767 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:01.762+0000 7f591affd700 1 -- 172.21.15.59:0/466769527 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f590400dc20 con 0x7f591c15d7f0 2023-11-24T07:58:01.767 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:01.762+0000 7f591affd700 1 -- 172.21.15.59:0/466769527 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f590400dd80 con 0x7f591c15d7f0 2023-11-24T07:58:01.767 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:01.762+0000 7f591affd700 1 -- 172.21.15.59:0/466769527 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5904017630 con 0x7f591c15d7f0 2023-11-24T07:58:01.767 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:01.763+0000 7f59222bd700 1 -- 172.21.15.59:0/466769527 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f591c15d7f0 msgr2=0x7f591c06d2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:01.767 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:01.763+0000 7f59222bd700 1 --2- 172.21.15.59:0/466769527 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f591c15d7f0 0x7f591c06d2c0 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7f591c0cf840 tx=0x7f590400be70 comp rx=0 tx=0).stop 2023-11-24T07:58:01.767 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:01.763+0000 7f59222bd700 1 -- 172.21.15.59:0/466769527 shutdown_connections 2023-11-24T07:58:01.767 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:01.763+0000 7f59222bd700 1 --2- 172.21.15.59:0/466769527 >> v2:172.21.15.76:3300/0 conn(0x7f591c151110 0x7f591c151560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:01.767 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:01.763+0000 7f59222bd700 1 --2- 172.21.15.59:0/466769527 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f591c15d7f0 0x7f591c06d2c0 unknown :-1 s=CLOSED pgs=531 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:01.767 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:01.763+0000 7f59222bd700 1 --2- 172.21.15.59:0/466769527 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f591c06d800 0x7f591c150bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:01.768 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:01.763+0000 7f59222bd700 1 -- 172.21.15.59:0/466769527 >> 172.21.15.59:0/466769527 conn(0x7f591c0b1650 msgr2=0x7f591c0b1a30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:58:01.768 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:01.763+0000 7f59222bd700 1 -- 172.21.15.59:0/466769527 shutdown_connections 2023-11-24T07:58:01.768 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:01.763+0000 7f59222bd700 1 -- 172.21.15.59:0/466769527 wait complete. 2023-11-24T07:58:01.770 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:01.764+0000 7f59222bd700 1 Processor -- start 2023-11-24T07:58:01.770 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:01.765+0000 7f59222bd700 1 -- start start 2023-11-24T07:58:01.771 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:01.765+0000 7f59222bd700 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f591c06d800 0x7f591c14d410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:01.771 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:01.765+0000 7f59222bd700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f591c151110 0x7f591c0cea50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:01.771 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:01.765+0000 7f59222bd700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f591c15d7f0 0x7f591c1e3230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:01.771 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:01.765+0000 7f59222bd700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f591c0ce110 con 0x7f591c151110 2023-11-24T07:58:01.771 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:01.765+0000 7f59222bd700 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f591c0aade0 con 0x7f591c15d7f0 2023-11-24T07:58:01.771 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:01.765+0000 7f59222bd700 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f591c14d950 con 0x7f591c06d800 2023-11-24T07:58:01.771 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:01.765+0000 7f591b7fe700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f591c151110 0x7f591c0cea50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:01.771 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:01.765+0000 7f591b7fe700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f591c151110 0x7f591c0cea50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:52676/0 (socket says 172.21.15.59:52676) 2023-11-24T07:58:01.771 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:01.765+0000 7f591b7fe700 1 -- 172.21.15.59:0/2756520845 learned_addr learned my addr 172.21.15.59:0/2756520845 (peer_addr_for_me v2:172.21.15.59:0/0) 2023-11-24T07:58:01.771 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:01.766+0000 7f592085a700 1 --2- 172.21.15.59:0/2756520845 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f591c15d7f0 0x7f591c1e3230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:01.771 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:01.766+0000 7f591bfff700 1 --2- 172.21.15.59:0/2756520845 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f591c06d800 0x7f591c14d410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:01.771 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:01.766+0000 7f591b7fe700 1 -- 172.21.15.59:0/2756520845 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f591c06d800 msgr2=0x7f591c14d410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:01.772 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:01.766+0000 7f591b7fe700 1 --2- 172.21.15.59:0/2756520845 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f591c06d800 0x7f591c14d410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:01.772 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:01.766+0000 7f591b7fe700 1 -- 172.21.15.59:0/2756520845 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f591c15d7f0 msgr2=0x7f591c1e3230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:01.772 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:01.766+0000 7f591b7fe700 1 --2- 172.21.15.59:0/2756520845 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f591c15d7f0 0x7f591c1e3230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:01.772 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:01.766+0000 7f591b7fe700 1 -- 172.21.15.59:0/2756520845 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f590400b7f0 con 0x7f591c151110 2023-11-24T07:58:01.772 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:01.766+0000 7f591b7fe700 1 --2- 172.21.15.59:0/2756520845 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f591c151110 0x7f591c0cea50 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7f590c00c820 tx=0x7f590c00cb30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:58:01.772 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:01.766+0000 7f5918ff9700 1 -- 172.21.15.59:0/2756520845 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f590c011800 con 0x7f591c151110 2023-11-24T07:58:01.772 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:01.766+0000 7f5918ff9700 1 -- 172.21.15.59:0/2756520845 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f590c011960 con 0x7f591c151110 2023-11-24T07:58:01.772 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:01.766+0000 7f59222bd700 1 -- 172.21.15.59:0/2756520845 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f591c0aa1d0 con 0x7f591c151110 2023-11-24T07:58:01.772 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:01.766+0000 7f5918ff9700 1 -- 172.21.15.59:0/2756520845 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f590c011d10 con 0x7f591c151110 2023-11-24T07:58:01.772 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:01.766+0000 7f59222bd700 1 -- 172.21.15.59:0/2756520845 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f591c0aa790 con 0x7f591c151110 2023-11-24T07:58:01.772 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:01.767+0000 7f59222bd700 1 -- 172.21.15.59:0/2756520845 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f591c075480 con 0x7f591c151110 2023-11-24T07:58:01.773 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:01.767+0000 7f5918ff9700 1 -- 172.21.15.59:0/2756520845 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 6) v1 ==== 89282+0+0 (secure 0 0 0) 0x7f590c018360 con 0x7f591c151110 2023-11-24T07:58:01.773 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:01.768+0000 7f5918ff9700 1 --2- 172.21.15.59:0/2756520845 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f58fc06af60 0x7f58fc06d420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:01.773 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:01.768+0000 7f5918ff9700 1 -- 172.21.15.59:0/2756520845 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(43..43 src has 1..43) v4 ==== 5609+0+0 (secure 0 0 0) 0x7f590c089570 con 0x7f591c151110 2023-11-24T07:58:01.773 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:01.768+0000 7f591bfff700 1 --2- 172.21.15.59:0/2756520845 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f58fc06af60 0x7f58fc06d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:01.773 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:01.769+0000 7f591bfff700 1 --2- 172.21.15.59:0/2756520845 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f58fc06af60 0x7f58fc06d420 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f5904019040 tx=0x7f590400bf90 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:58:01.774 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:01.773+0000 7f5918ff9700 1 -- 172.21.15.59:0/2756520845 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f590c0544f0 con 0x7f591c151110 2023-11-24T07:58:02.020 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:02.018+0000 7f59222bd700 1 -- 172.21.15.59:0/2756520845 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7f591c0aaf20 con 0x7f591c151110 2023-11-24T07:58:02.607 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:02.606+0000 7f5918ff9700 1 -- 172.21.15.59:0/2756520845 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 6 and data pool 7 v21) v1 ==== 172+0+0 (secure 0 0 0) 0x7f590c057b10 con 0x7f591c151110 2023-11-24T07:58:02.619 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:02.617+0000 7f59222bd700 1 -- 172.21.15.59:0/2756520845 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f58fc06af60 msgr2=0x7f58fc06d420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:02.619 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:02.617+0000 7f59222bd700 1 --2- 172.21.15.59:0/2756520845 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f58fc06af60 0x7f58fc06d420 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f5904019040 tx=0x7f590400bf90 comp rx=0 tx=0).stop 2023-11-24T07:58:02.619 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:02.617+0000 7f59222bd700 1 -- 172.21.15.59:0/2756520845 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f591c151110 msgr2=0x7f591c0cea50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:02.620 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:02.617+0000 7f59222bd700 1 --2- 172.21.15.59:0/2756520845 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f591c151110 0x7f591c0cea50 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7f590c00c820 tx=0x7f590c00cb30 comp rx=0 tx=0).stop 2023-11-24T07:58:02.620 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:02.617+0000 7f59222bd700 1 -- 172.21.15.59:0/2756520845 shutdown_connections 2023-11-24T07:58:02.620 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:02.617+0000 7f59222bd700 1 --2- 172.21.15.59:0/2756520845 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f58fc06af60 0x7f58fc06d420 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:02.620 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:02.617+0000 7f59222bd700 1 --2- 172.21.15.59:0/2756520845 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f591c15d7f0 0x7f591c1e3230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:02.620 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:02.617+0000 7f59222bd700 1 --2- 172.21.15.59:0/2756520845 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f591c06d800 0x7f591c14d410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:02.620 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:02.617+0000 7f59222bd700 1 --2- 172.21.15.59:0/2756520845 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f591c151110 0x7f591c0cea50 unknown :-1 s=CLOSED pgs=532 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:02.620 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:02.617+0000 7f59222bd700 1 -- 172.21.15.59:0/2756520845 >> 172.21.15.59:0/2756520845 conn(0x7f591c0b1650 msgr2=0x7f591c1508b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:58:02.620 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:02.618+0000 7f59222bd700 1 -- 172.21.15.59:0/2756520845 shutdown_connections 2023-11-24T07:58:02.621 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:02.618+0000 7f59222bd700 1 -- 172.21.15.59:0/2756520845 wait complete. 2023-11-24T07:58:02.626 INFO:teuthology.orchestra.run.smithi059.stderr:new fs with metadata pool 6 and data pool 7 2023-11-24T07:58:02.643 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-24T07:58:02.773 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:02.771+0000 7f95c0470700 1 Processor -- start 2023-11-24T07:58:02.773 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:02.771+0000 7f95c0470700 1 -- start start 2023-11-24T07:58:02.774 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:02.772+0000 7f95c0470700 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f95b806b6e0 0x7f95b806db70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:02.775 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:02.772+0000 7f95c0470700 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f95b8145100 0x7f95b806e0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:02.775 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:02.772+0000 7f95c0470700 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f95b8160720 con 0x7f95b8144c10 2023-11-24T07:58:02.775 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:02.772+0000 7f95c0470700 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f95b806f500 con 0x7f95b806b6e0 2023-11-24T07:58:02.775 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:02.772+0000 7f95c0470700 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f95b80cd610 con 0x7f95b8145100 2023-11-24T07:58:02.775 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:02.772+0000 7f95be20c700 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7f95b8144c10 0x7f95b806afd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:56172/0 (socket says 172.21.15.59:56172) 2023-11-24T07:58:02.775 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:02.772+0000 7f95be20c700 1 -- 172.21.15.59:0/3792385575 learned_addr learned my addr 172.21.15.59:0/3792385575 (peer_addr_for_me v1:172.21.15.59:0/0) 2023-11-24T07:58:02.775 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:02.772+0000 7f95bea0d700 1 --2- 172.21.15.59:0/3792385575 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f95b8145100 0x7f95b806e0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:02.775 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:02.772+0000 7f95bda0b700 1 --2- 172.21.15.59:0/3792385575 >> v2:172.21.15.76:3300/0 conn(0x7f95b806b6e0 0x7f95b806db70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:02.775 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:02.772+0000 7f95bd20a700 1 -- 172.21.15.59:0/3792385575 <== mon.0 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3026462174 0 0) 0x7f95b8160720 con 0x7f95b8144c10 2023-11-24T07:58:02.777 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:02.772+0000 7f95bd20a700 1 -- 172.21.15.59:0/3792385575 --> v1:172.21.15.59:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f95a40035b0 con 0x7f95b8144c10 2023-11-24T07:58:02.777 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:02.773+0000 7f95bd20a700 1 -- 172.21.15.59:0/3792385575 <== mon.0 v1:172.21.15.59:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1896799026 0 0) 0x7f95a40035b0 con 0x7f95b8144c10 2023-11-24T07:58:02.777 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:02.773+0000 7f95bea0d700 1 -- 172.21.15.59:0/3792385575 >> v2:172.21.15.76:3300/0 conn(0x7f95b806b6e0 msgr2=0x7f95b806db70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T07:58:02.777 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:02.773+0000 7f95bea0d700 1 --2- 172.21.15.59:0/3792385575 >> v2:172.21.15.76:3300/0 conn(0x7f95b806b6e0 0x7f95b806db70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:02.777 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:02.773+0000 7f95bea0d700 1 -- 172.21.15.59:0/3792385575 >> v1:172.21.15.59:6789/0 conn(0x7f95b8144c10 legacy=0x7f95b806afd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:02.777 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:02.773+0000 7f95bea0d700 1 -- 172.21.15.59:0/3792385575 --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f95b806f710 con 0x7f95b8145100 2023-11-24T07:58:02.777 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:02.773+0000 7f95bea0d700 1 --2- 172.21.15.59:0/3792385575 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f95b8145100 0x7f95b806e0b0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f95b0009d90 tx=0x7f95b000b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=94520d335c7b8f12 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:58:02.777 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:02.773+0000 7f95bd20a700 1 -- 172.21.15.59:0/3792385575 <== mon.2 v2:172.21.15.76:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95b000e070 con 0x7f95b8145100 2023-11-24T07:58:02.777 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:02.773+0000 7f95bd20a700 1 -- 172.21.15.59:0/3792385575 <== mon.2 v2:172.21.15.76:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f95b0003a20 con 0x7f95b8145100 2023-11-24T07:58:02.777 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:02.773+0000 7f95bd20a700 1 -- 172.21.15.59:0/3792385575 <== mon.2 v2:172.21.15.76:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95a8001670 con 0x7f95b8145100 2023-11-24T07:58:02.781 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:02.773+0000 7f95c0470700 1 -- 172.21.15.59:0/3792385575 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f95b8145100 msgr2=0x7f95b806e0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:02.781 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:02.774+0000 7f95c0470700 1 --2- 172.21.15.59:0/3792385575 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f95b8145100 0x7f95b806e0b0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f95b0009d90 tx=0x7f95b000b330 comp rx=0 tx=0).stop 2023-11-24T07:58:02.781 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:02.774+0000 7f95c0470700 1 -- 172.21.15.59:0/3792385575 shutdown_connections 2023-11-24T07:58:02.781 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:02.774+0000 7f95c0470700 1 --2- 172.21.15.59:0/3792385575 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f95b8145100 0x7f95b806e0b0 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:02.781 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:02.774+0000 7f95c0470700 1 --2- 172.21.15.59:0/3792385575 >> v2:172.21.15.76:3300/0 conn(0x7f95b806b6e0 0x7f95b806db70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:02.781 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:02.774+0000 7f95c0470700 1 -- 172.21.15.59:0/3792385575 >> 172.21.15.59:0/3792385575 conn(0x7f95b80b2740 msgr2=0x7f95b80b2b20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:58:02.781 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:02.774+0000 7f95c0470700 1 -- 172.21.15.59:0/3792385575 shutdown_connections 2023-11-24T07:58:02.781 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:02.774+0000 7f95c0470700 1 -- 172.21.15.59:0/3792385575 wait complete. 2023-11-24T07:58:02.781 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:02.775+0000 7f95c0470700 1 Processor -- start 2023-11-24T07:58:02.782 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:02.775+0000 7f95c0470700 1 -- start start 2023-11-24T07:58:02.782 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:02.776+0000 7f95c0470700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f95b806b6e0 0x7f95b81f6d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:02.782 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:02.776+0000 7f95c0470700 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f95b8144c10 0x7f95b81f7260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:02.782 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:02.776+0000 7f95c0470700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f95b8145100 0x7f95b80c9dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:02.782 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:02.776+0000 7f95c0470700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95b80b3b40 con 0x7f95b806b6e0 2023-11-24T07:58:02.782 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:02.776+0000 7f95c0470700 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95b81f4c20 con 0x7f95b8145100 2023-11-24T07:58:02.782 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:02.776+0000 7f95c0470700 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f95b81f77a0 con 0x7f95b8144c10 2023-11-24T07:58:02.782 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:02.776+0000 7f95be20c700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f95b806b6e0 0x7f95b81f6d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:02.782 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:02.776+0000 7f95be20c700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f95b806b6e0 0x7f95b81f6d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:52704/0 (socket says 172.21.15.59:52704) 2023-11-24T07:58:02.783 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:02.776+0000 7f95be20c700 1 -- 172.21.15.59:0/2187975271 learned_addr learned my addr 172.21.15.59:0/2187975271 (peer_addr_for_me v2:172.21.15.59:0/0) 2023-11-24T07:58:02.783 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:02.776+0000 7f95bda0b700 1 --2- 172.21.15.59:0/2187975271 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f95b8144c10 0x7f95b81f7260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:02.783 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:02.776+0000 7f95bea0d700 1 --2- 172.21.15.59:0/2187975271 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f95b8145100 0x7f95b80c9dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:02.783 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:02.776+0000 7f95be20c700 1 -- 172.21.15.59:0/2187975271 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f95b8144c10 msgr2=0x7f95b81f7260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:02.783 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:02.776+0000 7f95be20c700 1 --2- 172.21.15.59:0/2187975271 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f95b8144c10 0x7f95b81f7260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:02.783 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:02.776+0000 7f95be20c700 1 -- 172.21.15.59:0/2187975271 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f95b8145100 msgr2=0x7f95b80c9dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:02.783 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:02.776+0000 7f95be20c700 1 --2- 172.21.15.59:0/2187975271 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f95b8145100 0x7f95b80c9dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:02.783 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:02.776+0000 7f95be20c700 1 -- 172.21.15.59:0/2187975271 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f95b0009a70 con 0x7f95b806b6e0 2023-11-24T07:58:02.783 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:02.776+0000 7f95be20c700 1 --2- 172.21.15.59:0/2187975271 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f95b806b6e0 0x7f95b81f6d20 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7f95a800c7b0 tx=0x7f95a800cac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:58:02.783 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:02.776+0000 7f95aeffd700 1 -- 172.21.15.59:0/2187975271 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95a8004c80 con 0x7f95b806b6e0 2023-11-24T07:58:02.783 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:02.776+0000 7f95aeffd700 1 -- 172.21.15.59:0/2187975271 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f95a8004de0 con 0x7f95b806b6e0 2023-11-24T07:58:02.783 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:02.776+0000 7f95c0470700 1 -- 172.21.15.59:0/2187975271 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f95b81f79f0 con 0x7f95b806b6e0 2023-11-24T07:58:02.784 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:02.776+0000 7f95aeffd700 1 -- 172.21.15.59:0/2187975271 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95a8007310 con 0x7f95b806b6e0 2023-11-24T07:58:02.784 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:02.776+0000 7f95c0470700 1 -- 172.21.15.59:0/2187975271 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f95b81f5150 con 0x7f95b806b6e0 2023-11-24T07:58:02.784 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:02.777+0000 7f95c0470700 1 -- 172.21.15.59:0/2187975271 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f95b8068670 con 0x7f95b806b6e0 2023-11-24T07:58:02.784 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:02.778+0000 7f95aeffd700 1 -- 172.21.15.59:0/2187975271 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 6) v1 ==== 89282+0+0 (secure 0 0 0) 0x7f95a8007470 con 0x7f95b806b6e0 2023-11-24T07:58:02.784 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:02.778+0000 7f95aeffd700 1 --2- 172.21.15.59:0/2187975271 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f959c06af60 0x7f959c06d420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:02.784 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:02.778+0000 7f95aeffd700 1 -- 172.21.15.59:0/2187975271 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(44..44 src has 1..44) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f95a80892f0 con 0x7f95b806b6e0 2023-11-24T07:58:02.784 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:02.778+0000 7f95bda0b700 1 --2- 172.21.15.59:0/2187975271 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f959c06af60 0x7f959c06d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:02.784 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:02.779+0000 7f95bda0b700 1 --2- 172.21.15.59:0/2187975271 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f959c06af60 0x7f959c06d420 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f95b80b2d80 tx=0x7f95b4009210 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:58:02.784 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:02.783+0000 7f95aeffd700 1 -- 172.21.15.59:0/2187975271 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f95a8054440 con 0x7f95b806b6e0 2023-11-24T07:58:02.983 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:02.982+0000 7f95c0470700 1 -- 172.21.15.59:0/2187975271 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f95b80d0840 con 0x7f95b806b6e0 2023-11-24T07:58:02.984 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:02.983+0000 7f95aeffd700 1 -- 172.21.15.59:0/2187975271 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v44) v1 ==== 74+0+16450 (secure 0 0 0) 0x7f95a8057a60 con 0x7f95b806b6e0 2023-11-24T07:58:02.985 INFO:teuthology.orchestra.run.smithi059.stdout: 2023-11-24T07:58:02.985 INFO:teuthology.orchestra.run.smithi059.stdout:{"epoch":44,"fsid":"9f0e8741-b8f9-48d5-a2c3-09f45903303d","created":"2023-11-24T07:56:06.552729+0000","modified":"2023-11-24T07:58:02.020193+0000","last_up_change":"2023-11-24T07:56:17.372352+0000","last_in_change":"2023-11-24T07:56:09.073250+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-24T07:56:18.352823+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_metadata","create_time":"2023-11-24T07:58:00.240130+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"44","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_data","create_time":"2023-11-24T07:58:00.977486+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"44","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"6d8f9e9b-1782-4e41-928b-a39f2a8ebc06","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6800","nonce":3373878772},{"type":"v1","addr":"172.21.15.59:6801","nonce":3373878772}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6802","nonce":3373878772},{"type":"v1","addr":"172.21.15.59:6803","nonce":3373878772}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6806","nonce":3373878772},{"type":"v1","addr":"172.21.15.59:6807","nonce":3373878772}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6804","nonce":3373878772},{"type":"v1","addr":"172.21.15.59:6805","nonce":3373878772}]},"public_addr":"172.21.15.59:6801/3373878772","cluster_addr":"172.21.15.59:6803/3373878772","heartbeat_back_addr":"172.21.15.59:6807/3373878772","heartbeat_front_addr":"172.21.15.59:6805/3373878772","state":["exists","up"]},{"osd":1,"uuid":"e8d882aa-52e1-4c04-9e1e-c44ef9204a5f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6824","nonce":648258563},{"type":"v1","addr":"172.21.15.59:6825","nonce":648258563}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6826","nonce":648258563},{"type":"v1","addr":"172.21.15.59:6827","nonce":648258563}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6830","nonce":648258563},{"type":"v1","addr":"172.21.15.59:6831","nonce":648258563}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6828","nonce":648258563},{"type":"v1","addr":"172.21.15.59:6829","nonce":648258563}]},"public_addr":"172.21.15.59:6825/648258563","cluster_addr":"172.21.15.59:6827/648258563","heartbeat_back_addr":"172.21.15.59:6831/648258563","heartbeat_front_addr":"172.21.15.59:6829/648258563","state":["exists","up"]},{"osd":2,"uuid":"69eebfa0-695b-4b21-90b9-8a3537168a01","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6808","nonce":1840430654},{"type":"v1","addr":"172.21.15.59:6809","nonce":1840430654}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6810","nonce":1840430654},{"type":"v1","addr":"172.21.15.59:6811","nonce":1840430654}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6814","nonce":1840430654},{"type":"v1","addr":"172.21.15.59:6815","nonce":1840430654}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6812","nonce":1840430654},{"type":"v1","addr":"172.21.15.59:6813","nonce":1840430654}]},"public_addr":"172.21.15.59:6809/1840430654","cluster_addr":"172.21.15.59:6811/1840430654","heartbeat_back_addr":"172.21.15.59:6815/1840430654","heartbeat_front_addr":"172.21.15.59:6813/1840430654","state":["exists","up"]},{"osd":3,"uuid":"96fd77fc-f911-46ff-9445-6f0be2daa8f4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6816","nonce":2504025850},{"type":"v1","addr":"172.21.15.59:6817","nonce":2504025850}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6818","nonce":2504025850},{"type":"v1","addr":"172.21.15.59:6819","nonce":2504025850}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6822","nonce":2504025850},{"type":"v1","addr":"172.21.15.59:6823","nonce":2504025850}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6820","nonce":2504025850},{"type":"v1","addr":"172.21.15.59:6821","nonce":2504025850}]},"public_addr":"172.21.15.59:6817/2504025850","cluster_addr":"172.21.15.59:6819/2504025850","heartbeat_back_addr":"172.21.15.59:6823/2504025850","heartbeat_front_addr":"172.21.15.59:6821/2504025850","state":["exists","up"]},{"osd":4,"uuid":"7ac29fca-b5b2-4c95-9d69-2f164cbbc708","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6816","nonce":2382463431},{"type":"v1","addr":"172.21.15.76:6818","nonce":2382463431}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6820","nonce":2382463431},{"type":"v1","addr":"172.21.15.76:6822","nonce":2382463431}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6828","nonce":2382463431},{"type":"v1","addr":"172.21.15.76:6830","nonce":2382463431}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6824","nonce":2382463431},{"type":"v1","addr":"172.21.15.76:6826","nonce":2382463431}]},"public_addr":"172.21.15.76:6818/2382463431","cluster_addr":"172.21.15.76:6822/2382463431","heartbeat_back_addr":"172.21.15.76:6830/2382463431","heartbeat_front_addr":"172.21.15.76:6826/2382463431","state":["exists","up"]},{"osd":5,"uuid":"485bf711-3851-4238-a4f4-51d52e0a9c58","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6800","nonce":329334308},{"type":"v1","addr":"172.21.15.76:6802","nonce":329334308}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6805","nonce":329334308},{"type":"v1","addr":"172.21.15.76:6807","nonce":329334308}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6814","nonce":329334308},{"type":"v1","addr":"172.21.15.76:6815","nonce":329334308}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6810","nonce":329334308},{"type":"v1","addr":"172.21.15.76:6812","nonce":329334308}]},"public_addr":"172.21.15.76:6802/329334308","cluster_addr":"172.21.15.76:6807/329334308","heartbeat_back_addr":"172.21.15.76:6815/329334308","heartbeat_front_addr":"172.21.15.76:6812/329334308","state":["exists","up"]},{"osd":6,"uuid":"72b3f2fc-1e4a-4e32-968d-003647a0286f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6801","nonce":3946579410},{"type":"v1","addr":"172.21.15.76:6803","nonce":3946579410}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6804","nonce":3946579410},{"type":"v1","addr":"172.21.15.76:6806","nonce":3946579410}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6811","nonce":3946579410},{"type":"v1","addr":"172.21.15.76:6813","nonce":3946579410}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6808","nonce":3946579410},{"type":"v1","addr":"172.21.15.76:6809","nonce":3946579410}]},"public_addr":"172.21.15.76:6803/3946579410","cluster_addr":"172.21.15.76:6806/3946579410","heartbeat_back_addr":"172.21.15.76:6813/3946579410","heartbeat_front_addr":"172.21.15.76:6809/3946579410","state":["exists","up"]},{"osd":7,"uuid":"1ddceb90-6c5d-4d1a-ad56-99cb877f8c70","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6817","nonce":841761538},{"type":"v1","addr":"172.21.15.76:6819","nonce":841761538}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6821","nonce":841761538},{"type":"v1","addr":"172.21.15.76:6823","nonce":841761538}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6829","nonce":841761538},{"type":"v1","addr":"172.21.15.76:6831","nonce":841761538}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6825","nonce":841761538},{"type":"v1","addr":"172.21.15.76:6827","nonce":841761538}]},"public_addr":"172.21.15.76:6819/841761538","cluster_addr":"172.21.15.76:6823/841761538","heartbeat_back_addr":"172.21.15.76:6831/841761538","heartbeat_front_addr":"172.21.15.76:6827/841761538","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T07:56:14.294838+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T07:56:14.270779+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T07:56:14.231809+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T07:56:14.260983+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T07:56:14.302997+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T07:56:14.258600+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T07:56:14.251413+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T07:56:14.304727+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-24T07:58:02.987 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:02.985+0000 7f95c0470700 1 -- 172.21.15.59:0/2187975271 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f959c06af60 msgr2=0x7f959c06d420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:02.987 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:02.985+0000 7f95c0470700 1 --2- 172.21.15.59:0/2187975271 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f959c06af60 0x7f959c06d420 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f95b80b2d80 tx=0x7f95b4009210 comp rx=0 tx=0).stop 2023-11-24T07:58:02.987 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:02.985+0000 7f95c0470700 1 -- 172.21.15.59:0/2187975271 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f95b806b6e0 msgr2=0x7f95b81f6d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:02.987 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:02.985+0000 7f95c0470700 1 --2- 172.21.15.59:0/2187975271 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f95b806b6e0 0x7f95b81f6d20 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7f95a800c7b0 tx=0x7f95a800cac0 comp rx=0 tx=0).stop 2023-11-24T07:58:02.987 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:02.985+0000 7f95c0470700 1 -- 172.21.15.59:0/2187975271 shutdown_connections 2023-11-24T07:58:02.988 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:02.985+0000 7f95c0470700 1 --2- 172.21.15.59:0/2187975271 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f959c06af60 0x7f959c06d420 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:02.988 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:02.985+0000 7f95c0470700 1 --2- 172.21.15.59:0/2187975271 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f95b8145100 0x7f95b80c9dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:02.988 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:02.985+0000 7f95c0470700 1 --2- 172.21.15.59:0/2187975271 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f95b806b6e0 0x7f95b81f6d20 unknown :-1 s=CLOSED pgs=534 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:02.988 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:02.985+0000 7f95c0470700 1 --2- 172.21.15.59:0/2187975271 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f95b8144c10 0x7f95b81f7260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:02.988 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:02.985+0000 7f95c0470700 1 -- 172.21.15.59:0/2187975271 >> 172.21.15.59:0/2187975271 conn(0x7f95b80b2740 msgr2=0x7f95b80cdd90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:58:02.988 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:02.985+0000 7f95c0470700 1 -- 172.21.15.59:0/2187975271 shutdown_connections 2023-11-24T07:58:02.989 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:02.985+0000 7f95c0470700 1 -- 172.21.15.59:0/2187975271 wait complete. 2023-11-24T07:58:02.999 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-24T07:58:03.124 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:03.122+0000 7f1503d3d700 1 Processor -- start 2023-11-24T07:58:03.124 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:03.122+0000 7f1503d3d700 1 -- start start 2023-11-24T07:58:03.126 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:03.123+0000 7f1503d3d700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f14fc15e460 0x7f14fc0d49d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:03.126 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:03.123+0000 7f1503d3d700 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f14fc0d2ba0 0x7f14fc0b40a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:03.126 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:03.123+0000 7f1503d3d700 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f14fc0b3370 con 0x7f14fc0cf480 2023-11-24T07:58:03.127 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:03.123+0000 7f1503d3d700 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f14fc0cfc40 con 0x7f14fc15e460 2023-11-24T07:58:03.127 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:03.123+0000 7f1503d3d700 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f14fc0b35c0 con 0x7f14fc0d2ba0 2023-11-24T07:58:03.127 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:03.123+0000 7f1501ad9700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f14fc15e460 0x7f14fc0d49d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:03.127 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:03.123+0000 7f1501ad9700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f14fc15e460 0x7f14fc0d49d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:52712/0 (socket says 172.21.15.59:52712) 2023-11-24T07:58:03.127 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:03.123+0000 7f1501ad9700 1 -- 172.21.15.59:0/2176458073 learned_addr learned my addr 172.21.15.59:0/2176458073 (peer_addr_for_me v2:172.21.15.59:0/0) 2023-11-24T07:58:03.127 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:03.124+0000 7f15022da700 1 --2- 172.21.15.59:0/2176458073 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f14fc0d2ba0 0x7f14fc0b40a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:03.127 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:03.124+0000 7f1501ad9700 1 -- 172.21.15.59:0/2176458073 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f14fc0d2ba0 msgr2=0x7f14fc0b40a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T07:58:03.127 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:03.124+0000 7f1501ad9700 1 --2- 172.21.15.59:0/2176458073 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f14fc0d2ba0 0x7f14fc0b40a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:03.127 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:03.124+0000 7f1501ad9700 1 -- 172.21.15.59:0/2176458073 >> v1:172.21.15.76:6789/0 conn(0x7f14fc0cf480 legacy=0x7f14fc0d2490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T07:58:03.127 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:03.124+0000 7f1501ad9700 1 -- 172.21.15.59:0/2176458073 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f14fc0b38a0 con 0x7f14fc15e460 2023-11-24T07:58:03.127 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:03.124+0000 7f1501ad9700 1 --2- 172.21.15.59:0/2176458073 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f14fc15e460 0x7f14fc0d49d0 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7f14ec0098f0 tx=0x7f14ec009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=698f4a7a5689a9c9 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:58:03.128 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:03.124+0000 7f1500ad7700 1 -- 172.21.15.59:0/2176458073 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14ec004230 con 0x7f14fc15e460 2023-11-24T07:58:03.128 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:03.124+0000 7f1500ad7700 1 -- 172.21.15.59:0/2176458073 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f14ec004390 con 0x7f14fc15e460 2023-11-24T07:58:03.128 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:03.124+0000 7f1500ad7700 1 -- 172.21.15.59:0/2176458073 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14ec0036a0 con 0x7f14fc15e460 2023-11-24T07:58:03.128 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:03.124+0000 7f1503d3d700 1 -- 172.21.15.59:0/2176458073 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f14fc15e460 msgr2=0x7f14fc0d49d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:03.128 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:03.124+0000 7f1503d3d700 1 --2- 172.21.15.59:0/2176458073 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f14fc15e460 0x7f14fc0d49d0 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7f14ec0098f0 tx=0x7f14ec009c00 comp rx=0 tx=0).stop 2023-11-24T07:58:03.128 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:03.125+0000 7f1503d3d700 1 -- 172.21.15.59:0/2176458073 shutdown_connections 2023-11-24T07:58:03.129 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:03.125+0000 7f1503d3d700 1 --2- 172.21.15.59:0/2176458073 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f14fc0d2ba0 0x7f14fc0b40a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:03.129 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:03.125+0000 7f1503d3d700 1 --2- 172.21.15.59:0/2176458073 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f14fc15e460 0x7f14fc0d49d0 unknown :-1 s=CLOSED pgs=535 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:03.129 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:03.125+0000 7f1503d3d700 1 -- 172.21.15.59:0/2176458073 >> 172.21.15.59:0/2176458073 conn(0x7f14fc0d3eb0 msgr2=0x7f14fc0d42c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:58:03.129 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:03.125+0000 7f1503d3d700 1 -- 172.21.15.59:0/2176458073 shutdown_connections 2023-11-24T07:58:03.129 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:03.125+0000 7f1503d3d700 1 -- 172.21.15.59:0/2176458073 wait complete. 2023-11-24T07:58:03.132 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:03.126+0000 7f1503d3d700 1 Processor -- start 2023-11-24T07:58:03.132 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:03.126+0000 7f1503d3d700 1 -- start start 2023-11-24T07:58:03.132 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:03.127+0000 7f1503d3d700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f14fc0cf480 0x7f14fc160c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:03.132 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:03.127+0000 7f1503d3d700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f14fc0d2ba0 0x7f14fc0b9050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:03.132 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:03.127+0000 7f1503d3d700 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f14fc15e460 0x7f14fc0cb720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:03.132 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:03.127+0000 7f1503d3d700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14fc0b76c0 con 0x7f14fc0d2ba0 2023-11-24T07:58:03.133 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:03.127+0000 7f1503d3d700 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14fc0b6ab0 con 0x7f14fc0cf480 2023-11-24T07:58:03.133 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:03.127+0000 7f1503d3d700 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f14fc0b6d90 con 0x7f14fc15e460 2023-11-24T07:58:03.133 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:03.127+0000 7f15012d8700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f14fc0d2ba0 0x7f14fc0b9050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:03.133 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:03.127+0000 7f15012d8700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f14fc0d2ba0 0x7f14fc0b9050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:52720/0 (socket says 172.21.15.59:52720) 2023-11-24T07:58:03.133 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:03.127+0000 7f15012d8700 1 -- 172.21.15.59:0/437891021 learned_addr learned my addr 172.21.15.59:0/437891021 (peer_addr_for_me v2:172.21.15.59:0/0) 2023-11-24T07:58:03.133 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:03.127+0000 7f1501ad9700 1 --2- 172.21.15.59:0/437891021 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f14fc0cf480 0x7f14fc160c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:03.133 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:03.127+0000 7f15022da700 1 --2- 172.21.15.59:0/437891021 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f14fc15e460 0x7f14fc0cb720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:03.133 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:03.127+0000 7f15012d8700 1 -- 172.21.15.59:0/437891021 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f14fc15e460 msgr2=0x7f14fc0cb720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:03.134 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:03.127+0000 7f15012d8700 1 --2- 172.21.15.59:0/437891021 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f14fc15e460 0x7f14fc0cb720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:03.134 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:03.127+0000 7f15012d8700 1 -- 172.21.15.59:0/437891021 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f14fc0cf480 msgr2=0x7f14fc160c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:03.134 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:03.127+0000 7f15012d8700 1 --2- 172.21.15.59:0/437891021 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f14fc0cf480 0x7f14fc160c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:03.134 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:03.127+0000 7f15012d8700 1 -- 172.21.15.59:0/437891021 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f14ec009550 con 0x7f14fc0d2ba0 2023-11-24T07:58:03.134 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:03.127+0000 7f15012d8700 1 --2- 172.21.15.59:0/437891021 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f14fc0d2ba0 0x7f14fc0b9050 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7f14f800c4d0 tx=0x7f14f800c7e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:58:03.134 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:03.127+0000 7f14f27fc700 1 -- 172.21.15.59:0/437891021 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14f8011790 con 0x7f14fc0d2ba0 2023-11-24T07:58:03.134 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:03.127+0000 7f14f27fc700 1 -- 172.21.15.59:0/437891021 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f14f80118f0 con 0x7f14fc0d2ba0 2023-11-24T07:58:03.134 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:03.127+0000 7f1503d3d700 1 -- 172.21.15.59:0/437891021 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f14fc0b7070 con 0x7f14fc0d2ba0 2023-11-24T07:58:03.134 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:03.127+0000 7f14f27fc700 1 -- 172.21.15.59:0/437891021 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14f8011ca0 con 0x7f14fc0d2ba0 2023-11-24T07:58:03.134 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:03.127+0000 7f1503d3d700 1 -- 172.21.15.59:0/437891021 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f14fc15caf0 con 0x7f14fc0d2ba0 2023-11-24T07:58:03.134 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:03.128+0000 7f1503d3d700 1 -- 172.21.15.59:0/437891021 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f14fc0b7830 con 0x7f14fc0d2ba0 2023-11-24T07:58:03.135 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:03.129+0000 7f14f27fc700 1 -- 172.21.15.59:0/437891021 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 6) v1 ==== 89282+0+0 (secure 0 0 0) 0x7f14f801c460 con 0x7f14fc0d2ba0 2023-11-24T07:58:03.135 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:03.129+0000 7f14f27fc700 1 --2- 172.21.15.59:0/437891021 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f14e006af60 0x7f14e006d420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:03.135 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:03.129+0000 7f14f27fc700 1 -- 172.21.15.59:0/437891021 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(44..44 src has 1..44) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f14f80892f0 con 0x7f14fc0d2ba0 2023-11-24T07:58:03.135 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:03.129+0000 7f1501ad9700 1 --2- 172.21.15.59:0/437891021 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f14e006af60 0x7f14e006d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:03.135 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:03.130+0000 7f1501ad9700 1 --2- 172.21.15.59:0/437891021 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f14e006af60 0x7f14e006d420 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f14ec009130 tx=0x7f14ec008030 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:58:03.135 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:03.134+0000 7f14f27fc700 1 -- 172.21.15.59:0/437891021 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f14f80543b0 con 0x7f14fc0d2ba0 2023-11-24T07:58:03.337 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:03.335+0000 7f1503d3d700 1 -- 172.21.15.59:0/437891021 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f14fc067f60 con 0x7f14fc0d2ba0 2023-11-24T07:58:03.338 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:03.336+0000 7f14f27fc700 1 -- 172.21.15.59:0/437891021 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v44) v1 ==== 74+0+16450 (secure 0 0 0) 0x7f14f80579d0 con 0x7f14fc0d2ba0 2023-11-24T07:58:03.338 INFO:teuthology.orchestra.run.smithi059.stdout: 2023-11-24T07:58:03.338 INFO:teuthology.orchestra.run.smithi059.stdout:{"epoch":44,"fsid":"9f0e8741-b8f9-48d5-a2c3-09f45903303d","created":"2023-11-24T07:56:06.552729+0000","modified":"2023-11-24T07:58:02.020193+0000","last_up_change":"2023-11-24T07:56:17.372352+0000","last_in_change":"2023-11-24T07:56:09.073250+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-24T07:56:18.352823+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_metadata","create_time":"2023-11-24T07:58:00.240130+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"44","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_data","create_time":"2023-11-24T07:58:00.977486+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"44","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"6d8f9e9b-1782-4e41-928b-a39f2a8ebc06","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6800","nonce":3373878772},{"type":"v1","addr":"172.21.15.59:6801","nonce":3373878772}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6802","nonce":3373878772},{"type":"v1","addr":"172.21.15.59:6803","nonce":3373878772}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6806","nonce":3373878772},{"type":"v1","addr":"172.21.15.59:6807","nonce":3373878772}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6804","nonce":3373878772},{"type":"v1","addr":"172.21.15.59:6805","nonce":3373878772}]},"public_addr":"172.21.15.59:6801/3373878772","cluster_addr":"172.21.15.59:6803/3373878772","heartbeat_back_addr":"172.21.15.59:6807/3373878772","heartbeat_front_addr":"172.21.15.59:6805/3373878772","state":["exists","up"]},{"osd":1,"uuid":"e8d882aa-52e1-4c04-9e1e-c44ef9204a5f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6824","nonce":648258563},{"type":"v1","addr":"172.21.15.59:6825","nonce":648258563}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6826","nonce":648258563},{"type":"v1","addr":"172.21.15.59:6827","nonce":648258563}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6830","nonce":648258563},{"type":"v1","addr":"172.21.15.59:6831","nonce":648258563}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6828","nonce":648258563},{"type":"v1","addr":"172.21.15.59:6829","nonce":648258563}]},"public_addr":"172.21.15.59:6825/648258563","cluster_addr":"172.21.15.59:6827/648258563","heartbeat_back_addr":"172.21.15.59:6831/648258563","heartbeat_front_addr":"172.21.15.59:6829/648258563","state":["exists","up"]},{"osd":2,"uuid":"69eebfa0-695b-4b21-90b9-8a3537168a01","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6808","nonce":1840430654},{"type":"v1","addr":"172.21.15.59:6809","nonce":1840430654}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6810","nonce":1840430654},{"type":"v1","addr":"172.21.15.59:6811","nonce":1840430654}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6814","nonce":1840430654},{"type":"v1","addr":"172.21.15.59:6815","nonce":1840430654}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6812","nonce":1840430654},{"type":"v1","addr":"172.21.15.59:6813","nonce":1840430654}]},"public_addr":"172.21.15.59:6809/1840430654","cluster_addr":"172.21.15.59:6811/1840430654","heartbeat_back_addr":"172.21.15.59:6815/1840430654","heartbeat_front_addr":"172.21.15.59:6813/1840430654","state":["exists","up"]},{"osd":3,"uuid":"96fd77fc-f911-46ff-9445-6f0be2daa8f4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6816","nonce":2504025850},{"type":"v1","addr":"172.21.15.59:6817","nonce":2504025850}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6818","nonce":2504025850},{"type":"v1","addr":"172.21.15.59:6819","nonce":2504025850}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6822","nonce":2504025850},{"type":"v1","addr":"172.21.15.59:6823","nonce":2504025850}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6820","nonce":2504025850},{"type":"v1","addr":"172.21.15.59:6821","nonce":2504025850}]},"public_addr":"172.21.15.59:6817/2504025850","cluster_addr":"172.21.15.59:6819/2504025850","heartbeat_back_addr":"172.21.15.59:6823/2504025850","heartbeat_front_addr":"172.21.15.59:6821/2504025850","state":["exists","up"]},{"osd":4,"uuid":"7ac29fca-b5b2-4c95-9d69-2f164cbbc708","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6816","nonce":2382463431},{"type":"v1","addr":"172.21.15.76:6818","nonce":2382463431}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6820","nonce":2382463431},{"type":"v1","addr":"172.21.15.76:6822","nonce":2382463431}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6828","nonce":2382463431},{"type":"v1","addr":"172.21.15.76:6830","nonce":2382463431}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6824","nonce":2382463431},{"type":"v1","addr":"172.21.15.76:6826","nonce":2382463431}]},"public_addr":"172.21.15.76:6818/2382463431","cluster_addr":"172.21.15.76:6822/2382463431","heartbeat_back_addr":"172.21.15.76:6830/2382463431","heartbeat_front_addr":"172.21.15.76:6826/2382463431","state":["exists","up"]},{"osd":5,"uuid":"485bf711-3851-4238-a4f4-51d52e0a9c58","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6800","nonce":329334308},{"type":"v1","addr":"172.21.15.76:6802","nonce":329334308}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6805","nonce":329334308},{"type":"v1","addr":"172.21.15.76:6807","nonce":329334308}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6814","nonce":329334308},{"type":"v1","addr":"172.21.15.76:6815","nonce":329334308}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6810","nonce":329334308},{"type":"v1","addr":"172.21.15.76:6812","nonce":329334308}]},"public_addr":"172.21.15.76:6802/329334308","cluster_addr":"172.21.15.76:6807/329334308","heartbeat_back_addr":"172.21.15.76:6815/329334308","heartbeat_front_addr":"172.21.15.76:6812/329334308","state":["exists","up"]},{"osd":6,"uuid":"72b3f2fc-1e4a-4e32-968d-003647a0286f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6801","nonce":3946579410},{"type":"v1","addr":"172.21.15.76:6803","nonce":3946579410}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6804","nonce":3946579410},{"type":"v1","addr":"172.21.15.76:6806","nonce":3946579410}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6811","nonce":3946579410},{"type":"v1","addr":"172.21.15.76:6813","nonce":3946579410}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6808","nonce":3946579410},{"type":"v1","addr":"172.21.15.76:6809","nonce":3946579410}]},"public_addr":"172.21.15.76:6803/3946579410","cluster_addr":"172.21.15.76:6806/3946579410","heartbeat_back_addr":"172.21.15.76:6813/3946579410","heartbeat_front_addr":"172.21.15.76:6809/3946579410","state":["exists","up"]},{"osd":7,"uuid":"1ddceb90-6c5d-4d1a-ad56-99cb877f8c70","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6817","nonce":841761538},{"type":"v1","addr":"172.21.15.76:6819","nonce":841761538}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6821","nonce":841761538},{"type":"v1","addr":"172.21.15.76:6823","nonce":841761538}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6829","nonce":841761538},{"type":"v1","addr":"172.21.15.76:6831","nonce":841761538}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6825","nonce":841761538},{"type":"v1","addr":"172.21.15.76:6827","nonce":841761538}]},"public_addr":"172.21.15.76:6819/841761538","cluster_addr":"172.21.15.76:6823/841761538","heartbeat_back_addr":"172.21.15.76:6831/841761538","heartbeat_front_addr":"172.21.15.76:6827/841761538","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T07:56:14.294838+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T07:56:14.270779+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T07:56:14.231809+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T07:56:14.260983+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T07:56:14.302997+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T07:56:14.258600+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T07:56:14.251413+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T07:56:14.304727+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-24T07:58:03.340 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:03.338+0000 7f1503d3d700 1 -- 172.21.15.59:0/437891021 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f14e006af60 msgr2=0x7f14e006d420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:03.341 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:03.338+0000 7f1503d3d700 1 --2- 172.21.15.59:0/437891021 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f14e006af60 0x7f14e006d420 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f14ec009130 tx=0x7f14ec008030 comp rx=0 tx=0).stop 2023-11-24T07:58:03.341 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:03.338+0000 7f1503d3d700 1 -- 172.21.15.59:0/437891021 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f14fc0d2ba0 msgr2=0x7f14fc0b9050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:03.341 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:03.338+0000 7f1503d3d700 1 --2- 172.21.15.59:0/437891021 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f14fc0d2ba0 0x7f14fc0b9050 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7f14f800c4d0 tx=0x7f14f800c7e0 comp rx=0 tx=0).stop 2023-11-24T07:58:03.341 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:03.339+0000 7f1503d3d700 1 -- 172.21.15.59:0/437891021 shutdown_connections 2023-11-24T07:58:03.341 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:03.339+0000 7f1503d3d700 1 --2- 172.21.15.59:0/437891021 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f14e006af60 0x7f14e006d420 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:03.341 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:03.339+0000 7f1503d3d700 1 --2- 172.21.15.59:0/437891021 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f14fc15e460 0x7f14fc0cb720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:03.341 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:03.339+0000 7f1503d3d700 1 --2- 172.21.15.59:0/437891021 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f14fc0cf480 0x7f14fc160c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:03.341 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:03.339+0000 7f1503d3d700 1 --2- 172.21.15.59:0/437891021 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f14fc0d2ba0 0x7f14fc0b9050 unknown :-1 s=CLOSED pgs=536 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:03.341 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:03.339+0000 7f1503d3d700 1 -- 172.21.15.59:0/437891021 >> 172.21.15.59:0/437891021 conn(0x7f14fc0d3eb0 msgr2=0x7f14fc0cfd80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:58:03.341 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:03.339+0000 7f1503d3d700 1 -- 172.21.15.59:0/437891021 shutdown_connections 2023-11-24T07:58:03.342 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:03.339+0000 7f1503d3d700 1 -- 172.21.15.59:0/437891021 wait complete. 2023-11-24T07:58:03.353 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2023-11-24T07:58:03.481 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:03.479+0000 7f19a6917700 1 Processor -- start 2023-11-24T07:58:03.481 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:03.479+0000 7f19a6917700 1 -- start start 2023-11-24T07:58:03.485 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:03.480+0000 7f19a6917700 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f19a00af980 0x7f19a0066ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:03.485 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:03.480+0000 7f19a6917700 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f19a00b10c0 con 0x7f19a015bf10 2023-11-24T07:58:03.485 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:03.480+0000 7f19a6917700 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f19a00af730 con 0x7f19a006b7c0 2023-11-24T07:58:03.485 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:03.480+0000 7f19a6917700 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f19a00b1c60 con 0x7f19a00af980 2023-11-24T07:58:03.485 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:03.480+0000 7f199ffff700 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7f19a015bf10 0x7f19a00b1550 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:56200/0 (socket says 172.21.15.59:56200) 2023-11-24T07:58:03.485 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:03.480+0000 7f199ffff700 1 -- 172.21.15.59:0/3534761799 learned_addr learned my addr 172.21.15.59:0/3534761799 (peer_addr_for_me v1:172.21.15.59:0/0) 2023-11-24T07:58:03.486 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:03.481+0000 7f199f7fe700 1 --2- 172.21.15.59:0/3534761799 >> v2:172.21.15.76:3300/0 conn(0x7f19a00af980 0x7f19a0066ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:03.486 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:03.481+0000 7f199effd700 1 -- 172.21.15.59:0/3534761799 <== mon.0 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3123436217 0 0) 0x7f19a00b10c0 con 0x7f19a015bf10 2023-11-24T07:58:03.486 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:03.481+0000 7f199effd700 1 -- 172.21.15.59:0/3534761799 --> v1:172.21.15.59:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f198c0035b0 con 0x7f19a015bf10 2023-11-24T07:58:03.486 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:03.481+0000 7f199effd700 1 -- 172.21.15.59:0/3534761799 <== mon.0 v1:172.21.15.59:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1364450126 0 0) 0x7f198c0035b0 con 0x7f19a015bf10 2023-11-24T07:58:03.486 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:03.481+0000 7f199effd700 1 -- 172.21.15.59:0/3534761799 >> v2:172.21.15.76:3300/0 conn(0x7f19a00af980 msgr2=0x7f19a0066ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T07:58:03.486 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:03.481+0000 7f199effd700 1 --2- 172.21.15.59:0/3534761799 >> v2:172.21.15.76:3300/0 conn(0x7f19a00af980 0x7f19a0066ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:03.486 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:03.481+0000 7f199effd700 1 -- 172.21.15.59:0/3534761799 >> v1:172.21.15.76:6789/0 conn(0x7f19a006b7c0 legacy=0x7f19a00cccf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:03.486 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:03.481+0000 7f199effd700 1 -- 172.21.15.59:0/3534761799 --> v1:172.21.15.59:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f19a00b5ee0 con 0x7f19a015bf10 2023-11-24T07:58:03.486 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:03.481+0000 7f199effd700 1 -- 172.21.15.59:0/3534761799 <== mon.0 v1:172.21.15.59:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1312333566 0 0) 0x7f1988001670 con 0x7f19a015bf10 2023-11-24T07:58:03.486 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:03.481+0000 7f199effd700 1 -- 172.21.15.59:0/3534761799 >> v1:172.21.15.59:6789/0 conn(0x7f19a015bf10 legacy=0x7f19a00b1550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:03.487 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:03.481+0000 7f199effd700 1 --2- 172.21.15.59:0/3534761799 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f198c003a00 0x7f198c003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:03.487 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:03.481+0000 7f199effd700 1 --2- 172.21.15.59:0/3534761799 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f198c004850 0x7f198c004cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:03.487 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:03.481+0000 7f199effd700 1 --2- 172.21.15.59:0/3534761799 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f198c005210 0x7f198c00ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:03.487 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:03.481+0000 7f199effd700 1 -- 172.21.15.59:0/3534761799 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19a00b1c60 con 0x7f198c003a00 2023-11-24T07:58:03.487 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:03.481+0000 7f199effd700 1 -- 172.21.15.59:0/3534761799 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f198c00c0b0 con 0x7f198c005210 2023-11-24T07:58:03.487 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:03.481+0000 7f199effd700 1 -- 172.21.15.59:0/3534761799 --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f198c00c270 con 0x7f198c004850 2023-11-24T07:58:03.487 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:03.481+0000 7f199ffff700 1 --2- 172.21.15.59:0/3534761799 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f198c003a00 0x7f198c003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:03.487 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:03.481+0000 7f19a4eb4700 1 --2- 172.21.15.59:0/3534761799 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f198c005210 0x7f198c00ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:03.487 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:03.481+0000 7f199f7fe700 1 --2- 172.21.15.59:0/3534761799 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f198c004850 0x7f198c004cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:03.487 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:03.481+0000 7f199ffff700 1 -- 172.21.15.59:0/3534761799 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f198c004850 msgr2=0x7f198c004cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T07:58:03.487 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:03.481+0000 7f199ffff700 1 --2- 172.21.15.59:0/3534761799 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f198c004850 0x7f198c004cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:03.488 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:03.481+0000 7f199ffff700 1 -- 172.21.15.59:0/3534761799 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f198c005210 msgr2=0x7f198c00ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T07:58:03.488 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:03.481+0000 7f199ffff700 1 --2- 172.21.15.59:0/3534761799 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f198c005210 0x7f198c00ba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:03.488 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:03.481+0000 7f19a4eb4700 1 -- 172.21.15.59:0/3534761799 reap_dead start 2023-11-24T07:58:03.488 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:03.481+0000 7f199ffff700 1 -- 172.21.15.59:0/3534761799 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f198c0035b0 con 0x7f198c003a00 2023-11-24T07:58:03.488 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:03.482+0000 7f199ffff700 1 --2- 172.21.15.59:0/3534761799 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f198c003a00 0x7f198c003e20 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7f198800c240 tx=0x7f198c005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6ae22c1ba1f48854 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:58:03.488 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:03.482+0000 7f199effd700 1 -- 172.21.15.59:0/3534761799 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1988007c10 con 0x7f198c003a00 2023-11-24T07:58:03.488 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:03.482+0000 7f199effd700 1 -- 172.21.15.59:0/3534761799 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f198800cde0 con 0x7f198c003a00 2023-11-24T07:58:03.488 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:03.482+0000 7f199effd700 1 -- 172.21.15.59:0/3534761799 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1988005ba0 con 0x7f198c003a00 2023-11-24T07:58:03.488 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:03.482+0000 7f19a6917700 1 -- 172.21.15.59:0/3534761799 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f198c003a00 msgr2=0x7f198c003e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:03.488 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:03.482+0000 7f19a6917700 1 --2- 172.21.15.59:0/3534761799 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f198c003a00 0x7f198c003e20 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7f198800c240 tx=0x7f198c005880 comp rx=0 tx=0).stop 2023-11-24T07:58:03.488 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:03.482+0000 7f19a6917700 1 -- 172.21.15.59:0/3534761799 shutdown_connections 2023-11-24T07:58:03.488 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:03.482+0000 7f19a6917700 1 --2- 172.21.15.59:0/3534761799 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f198c003a00 0x7f198c003e20 unknown :-1 s=CLOSED pgs=538 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:03.489 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:03.482+0000 7f19a6917700 1 -- 172.21.15.59:0/3534761799 >> 172.21.15.59:0/3534761799 conn(0x7f19a00b5070 msgr2=0x7f19a00b5730 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:58:03.489 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:03.482+0000 7f19a6917700 1 -- 172.21.15.59:0/3534761799 shutdown_connections 2023-11-24T07:58:03.489 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:03.482+0000 7f19a6917700 1 -- 172.21.15.59:0/3534761799 wait complete. 2023-11-24T07:58:03.489 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:03.483+0000 7f19a6917700 1 Processor -- start 2023-11-24T07:58:03.489 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:03.484+0000 7f19a6917700 1 -- start start 2023-11-24T07:58:03.491 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:03.484+0000 7f19a6917700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f198c003a00 0x7f19a00cccf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:03.491 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:03.484+0000 7f19a6917700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f19a0155260 0x7f19a00cd680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:03.491 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:03.484+0000 7f19a6917700 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f19a01556e0 0x7f19a00cdbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:03.491 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:03.484+0000 7f19a6917700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19a009b0d0 con 0x7f19a0155260 2023-11-24T07:58:03.492 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:03.484+0000 7f19a6917700 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19a009b3b0 con 0x7f198c003a00 2023-11-24T07:58:03.492 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:03.484+0000 7f19a6917700 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f19a009b690 con 0x7f19a01556e0 2023-11-24T07:58:03.492 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:03.484+0000 7f199f7fe700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f19a0155260 0x7f19a00cd680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:03.492 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:03.484+0000 7f199f7fe700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f19a0155260 0x7f19a00cd680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:52740/0 (socket says 172.21.15.59:52740) 2023-11-24T07:58:03.492 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:03.484+0000 7f199f7fe700 1 -- 172.21.15.59:0/2037589207 learned_addr learned my addr 172.21.15.59:0/2037589207 (peer_addr_for_me v2:172.21.15.59:0/0) 2023-11-24T07:58:03.492 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:03.485+0000 7f199f7fe700 1 -- 172.21.15.59:0/2037589207 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f19a01556e0 msgr2=0x7f19a00cdbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:03.492 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:03.485+0000 7f199f7fe700 1 --2- 172.21.15.59:0/2037589207 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f19a01556e0 0x7f19a00cdbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:03.493 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:03.485+0000 7f199f7fe700 1 -- 172.21.15.59:0/2037589207 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f198c003a00 msgr2=0x7f19a00cccf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:03.493 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:03.485+0000 7f199ffff700 1 --2- 172.21.15.59:0/2037589207 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f198c003a00 0x7f19a00cccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:03.493 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:03.485+0000 7f199f7fe700 1 --2- 172.21.15.59:0/2037589207 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f198c003a00 0x7f19a00cccf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:03.493 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:03.485+0000 7f199f7fe700 1 -- 172.21.15.59:0/2037589207 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f19a009b970 con 0x7f19a0155260 2023-11-24T07:58:03.493 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:03.485+0000 7f199f7fe700 1 --2- 172.21.15.59:0/2037589207 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f19a0155260 0x7f19a00cd680 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7f1990011a20 tx=0x7f1990011de0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:58:03.494 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:03.485+0000 7f199cff9700 1 -- 172.21.15.59:0/2037589207 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f199000fa00 con 0x7f19a0155260 2023-11-24T07:58:03.494 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:03.485+0000 7f199cff9700 1 -- 172.21.15.59:0/2037589207 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f199000fb60 con 0x7f19a0155260 2023-11-24T07:58:03.494 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:03.485+0000 7f19a6917700 1 -- 172.21.15.59:0/2037589207 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f19a009bc50 con 0x7f19a0155260 2023-11-24T07:58:03.494 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:03.485+0000 7f199cff9700 1 -- 172.21.15.59:0/2037589207 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1990014440 con 0x7f19a0155260 2023-11-24T07:58:03.494 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:03.485+0000 7f19a6917700 1 -- 172.21.15.59:0/2037589207 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f19a009ad80 con 0x7f19a0155260 2023-11-24T07:58:03.494 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:03.486+0000 7f19a6917700 1 -- 172.21.15.59:0/2037589207 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f19a0075480 con 0x7f19a0155260 2023-11-24T07:58:03.494 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:03.486+0000 7f199cff9700 1 -- 172.21.15.59:0/2037589207 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 6) v1 ==== 89282+0+0 (secure 0 0 0) 0x7f19900145a0 con 0x7f19a0155260 2023-11-24T07:58:03.495 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:03.487+0000 7f199cff9700 1 --2- 172.21.15.59:0/2037589207 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f198006af60 0x7f198006d420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:03.495 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:03.487+0000 7f199cff9700 1 -- 172.21.15.59:0/2037589207 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(44..44 src has 1..44) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f1990089db0 con 0x7f19a0155260 2023-11-24T07:58:03.495 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:03.487+0000 7f199ffff700 1 --2- 172.21.15.59:0/2037589207 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f198006af60 0x7f198006d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:03.495 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:03.487+0000 7f199ffff700 1 --2- 172.21.15.59:0/2037589207 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f198006af60 0x7f198006d420 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f1988015040 tx=0x7f1988007000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:58:03.495 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:03.491+0000 7f199cff9700 1 -- 172.21.15.59:0/2037589207 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f1990054de0 con 0x7f19a0155260 2023-11-24T07:58:03.744 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:03.743+0000 7f19a6917700 1 -- 172.21.15.59:0/2037589207 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7f19a00b1a40 con 0x7f19a0155260 2023-11-24T07:58:04.626 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:04.624+0000 7f199cff9700 1 -- 172.21.15.59:0/2037589207 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v24) v1 ==== 118+0+0 (secure 0 0 0) 0x7f1990058400 con 0x7f19a0155260 2023-11-24T07:58:04.629 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:04.627+0000 7f19a6917700 1 -- 172.21.15.59:0/2037589207 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f198006af60 msgr2=0x7f198006d420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:04.629 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:04.627+0000 7f19a6917700 1 --2- 172.21.15.59:0/2037589207 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f198006af60 0x7f198006d420 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f1988015040 tx=0x7f1988007000 comp rx=0 tx=0).stop 2023-11-24T07:58:04.630 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:04.627+0000 7f19a6917700 1 -- 172.21.15.59:0/2037589207 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f19a0155260 msgr2=0x7f19a00cd680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:04.630 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:04.627+0000 7f19a6917700 1 --2- 172.21.15.59:0/2037589207 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f19a0155260 0x7f19a00cd680 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7f1990011a20 tx=0x7f1990011de0 comp rx=0 tx=0).stop 2023-11-24T07:58:04.630 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:04.627+0000 7f19a6917700 1 -- 172.21.15.59:0/2037589207 shutdown_connections 2023-11-24T07:58:04.630 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:04.627+0000 7f19a6917700 1 --2- 172.21.15.59:0/2037589207 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f198006af60 0x7f198006d420 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:04.630 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:04.627+0000 7f19a6917700 1 --2- 172.21.15.59:0/2037589207 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f19a01556e0 0x7f19a00cdbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:04.630 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:04.627+0000 7f19a6917700 1 --2- 172.21.15.59:0/2037589207 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f198c003a00 0x7f19a00cccf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:04.630 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:04.627+0000 7f19a6917700 1 --2- 172.21.15.59:0/2037589207 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f19a0155260 0x7f19a00cd680 unknown :-1 s=CLOSED pgs=539 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:04.630 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:04.627+0000 7f19a6917700 1 -- 172.21.15.59:0/2037589207 >> 172.21.15.59:0/2037589207 conn(0x7f19a00b5070 msgr2=0x7f19a006d560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:58:04.630 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:04.628+0000 7f19a6917700 1 -- 172.21.15.59:0/2037589207 shutdown_connections 2023-11-24T07:58:04.631 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:04.628+0000 7f19a6917700 1 -- 172.21.15.59:0/2037589207 wait complete. 2023-11-24T07:58:04.644 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2023-11-24T07:58:04.644 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2023-11-24T07:58:04.780 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:04.778+0000 7fb1f364a700 1 Processor -- start 2023-11-24T07:58:04.780 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:04.778+0000 7fb1f364a700 1 -- start start 2023-11-24T07:58:04.782 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:04.779+0000 7fb1f364a700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fb1ec15bde0 0x7fb1ec15c1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:04.783 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:04.779+0000 7fb1f364a700 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fb1ec0cf2b0 0x7fb1ec0d01b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:04.783 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:04.779+0000 7fb1f364a700 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb1ec0d0a90 con 0x7fb1ec0ce5c0 2023-11-24T07:58:04.783 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:04.779+0000 7fb1f364a700 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb1ec06ee10 con 0x7fb1ec15bde0 2023-11-24T07:58:04.783 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:04.779+0000 7fb1f364a700 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb1ec06f0f0 con 0x7fb1ec0cf2b0 2023-11-24T07:58:04.783 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:04.779+0000 7fb1f13e6700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fb1ec15bde0 0x7fb1ec15c1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:04.783 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:04.780+0000 7fb1f13e6700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fb1ec15bde0 0x7fb1ec15c1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:52746/0 (socket says 172.21.15.59:52746) 2023-11-24T07:58:04.783 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:04.780+0000 7fb1f13e6700 1 -- 172.21.15.59:0/912926040 learned_addr learned my addr 172.21.15.59:0/912926040 (peer_addr_for_me v2:172.21.15.59:0/0) 2023-11-24T07:58:04.783 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:04.780+0000 7fb1f0be5700 1 --2- 172.21.15.59:0/912926040 >> v2:172.21.15.76:3300/0 conn(0x7fb1ec0cf2b0 0x7fb1ec0d01b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:04.783 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:04.780+0000 7fb1f13e6700 1 -- 172.21.15.59:0/912926040 >> v2:172.21.15.76:3300/0 conn(0x7fb1ec0cf2b0 msgr2=0x7fb1ec0d01b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T07:58:04.783 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:04.780+0000 7fb1f13e6700 1 --2- 172.21.15.59:0/912926040 >> v2:172.21.15.76:3300/0 conn(0x7fb1ec0cf2b0 0x7fb1ec0d01b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:04.784 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:04.780+0000 7fb1f13e6700 1 -- 172.21.15.59:0/912926040 >> v1:172.21.15.76:6789/0 conn(0x7fb1ec0ce5c0 legacy=0x7fb1ec0cc670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T07:58:04.784 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:04.780+0000 7fb1f13e6700 1 -- 172.21.15.59:0/912926040 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb1ec06f3d0 con 0x7fb1ec15bde0 2023-11-24T07:58:04.784 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:04.780+0000 7fb1f13e6700 1 --2- 172.21.15.59:0/912926040 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb1ec15bde0 0x7fb1ec15c1c0 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7fb1ec06e780 tx=0x7fb1dc009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ffd664962975ef91 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:58:04.784 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:04.780+0000 7fb1e3fff700 1 -- 172.21.15.59:0/912926040 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1dc004230 con 0x7fb1ec15bde0 2023-11-24T07:58:04.784 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:04.780+0000 7fb1e3fff700 1 -- 172.21.15.59:0/912926040 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb1dc004390 con 0x7fb1ec15bde0 2023-11-24T07:58:04.784 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:04.780+0000 7fb1e3fff700 1 -- 172.21.15.59:0/912926040 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1dc0036a0 con 0x7fb1ec15bde0 2023-11-24T07:58:04.784 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:04.781+0000 7fb1f364a700 1 -- 172.21.15.59:0/912926040 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb1ec15bde0 msgr2=0x7fb1ec15c1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:04.784 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:04.781+0000 7fb1f364a700 1 --2- 172.21.15.59:0/912926040 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb1ec15bde0 0x7fb1ec15c1c0 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7fb1ec06e780 tx=0x7fb1dc009bd0 comp rx=0 tx=0).stop 2023-11-24T07:58:04.784 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:04.781+0000 7fb1f364a700 1 -- 172.21.15.59:0/912926040 shutdown_connections 2023-11-24T07:58:04.784 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:04.781+0000 7fb1f364a700 1 --2- 172.21.15.59:0/912926040 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb1ec15bde0 0x7fb1ec15c1c0 unknown :-1 s=CLOSED pgs=540 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:04.784 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:04.781+0000 7fb1f364a700 1 --2- 172.21.15.59:0/912926040 >> v2:172.21.15.76:3300/0 conn(0x7fb1ec0cf2b0 0x7fb1ec0d01b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:04.784 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:04.781+0000 7fb1f364a700 1 -- 172.21.15.59:0/912926040 >> 172.21.15.59:0/912926040 conn(0x7fb1ec0b3720 msgr2=0x7fb1ec0b3b30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:58:04.785 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:04.781+0000 7fb1f364a700 1 -- 172.21.15.59:0/912926040 shutdown_connections 2023-11-24T07:58:04.785 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:04.781+0000 7fb1f364a700 1 -- 172.21.15.59:0/912926040 wait complete. 2023-11-24T07:58:04.787 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:04.782+0000 7fb1f364a700 1 Processor -- start 2023-11-24T07:58:04.787 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:04.782+0000 7fb1f364a700 1 -- start start 2023-11-24T07:58:04.788 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:04.783+0000 7fb1f364a700 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fb1ec0ce5c0 0x7fb1ec0c9660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:04.788 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:04.783+0000 7fb1f364a700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb1ec0cf2b0 0x7fb1ec0b67d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:04.788 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:04.783+0000 7fb1f364a700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb1ec15bde0 0x7fb1ec1592a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:04.788 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:04.783+0000 7fb1f364a700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1ec0c9ba0 con 0x7fb1ec15bde0 2023-11-24T07:58:04.788 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:04.783+0000 7fb1f364a700 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1ec0b6d10 con 0x7fb1ec0cf2b0 2023-11-24T07:58:04.788 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:04.783+0000 7fb1f364a700 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb1ec0c9d90 con 0x7fb1ec0ce5c0 2023-11-24T07:58:04.788 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:04.783+0000 7fb1f1be7700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb1ec15bde0 0x7fb1ec1592a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:04.788 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:04.783+0000 7fb1f1be7700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb1ec15bde0 0x7fb1ec1592a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:52756/0 (socket says 172.21.15.59:52756) 2023-11-24T07:58:04.788 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:04.783+0000 7fb1f1be7700 1 -- 172.21.15.59:0/82425946 learned_addr learned my addr 172.21.15.59:0/82425946 (peer_addr_for_me v2:172.21.15.59:0/0) 2023-11-24T07:58:04.788 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:04.783+0000 7fb1f0be5700 1 --2- 172.21.15.59:0/82425946 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb1ec0cf2b0 0x7fb1ec0b67d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:04.788 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:04.783+0000 7fb1f13e6700 1 --2- 172.21.15.59:0/82425946 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fb1ec0ce5c0 0x7fb1ec0c9660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:04.789 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:04.783+0000 7fb1f1be7700 1 -- 172.21.15.59:0/82425946 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fb1ec0ce5c0 msgr2=0x7fb1ec0c9660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:04.789 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:04.783+0000 7fb1f1be7700 1 --2- 172.21.15.59:0/82425946 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fb1ec0ce5c0 0x7fb1ec0c9660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:04.789 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:04.783+0000 7fb1f1be7700 1 -- 172.21.15.59:0/82425946 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb1ec0cf2b0 msgr2=0x7fb1ec0b67d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:04.789 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:04.783+0000 7fb1f1be7700 1 --2- 172.21.15.59:0/82425946 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb1ec0cf2b0 0x7fb1ec0b67d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:04.789 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:04.783+0000 7fb1f1be7700 1 -- 172.21.15.59:0/82425946 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb1dc009550 con 0x7fb1ec15bde0 2023-11-24T07:58:04.789 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:04.783+0000 7fb1f0be5700 1 --2- 172.21.15.59:0/82425946 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb1ec0cf2b0 0x7fb1ec0b67d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-24T07:58:04.789 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:04.783+0000 7fb1f1be7700 1 --2- 172.21.15.59:0/82425946 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb1ec15bde0 0x7fb1ec1592a0 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7fb1e400cc10 tx=0x7fb1e400a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:58:04.789 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:04.784+0000 7fb1e1ffb700 1 -- 172.21.15.59:0/82425946 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1e400abb0 con 0x7fb1ec15bde0 2023-11-24T07:58:04.789 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:04.784+0000 7fb1e1ffb700 1 -- 172.21.15.59:0/82425946 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb1e400ad10 con 0x7fb1ec15bde0 2023-11-24T07:58:04.789 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:04.784+0000 7fb1f364a700 1 -- 172.21.15.59:0/82425946 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb1ec0ca070 con 0x7fb1ec15bde0 2023-11-24T07:58:04.789 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:04.784+0000 7fb1e1ffb700 1 -- 172.21.15.59:0/82425946 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1e4012660 con 0x7fb1ec15bde0 2023-11-24T07:58:04.789 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:04.784+0000 7fb1f364a700 1 -- 172.21.15.59:0/82425946 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb1ec0ca630 con 0x7fb1ec15bde0 2023-11-24T07:58:04.790 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:04.785+0000 7fb1f364a700 1 -- 172.21.15.59:0/82425946 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb1ec075480 con 0x7fb1ec15bde0 2023-11-24T07:58:04.790 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:04.785+0000 7fb1e1ffb700 1 -- 172.21.15.59:0/82425946 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 6) v1 ==== 89282+0+0 (secure 0 0 0) 0x7fb1e40127c0 con 0x7fb1ec15bde0 2023-11-24T07:58:04.790 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:04.786+0000 7fb1e1ffb700 1 --2- 172.21.15.59:0/82425946 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7fb1d006af60 0x7fb1d006d420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:04.790 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:04.786+0000 7fb1e1ffb700 1 -- 172.21.15.59:0/82425946 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fb1e4089ce0 con 0x7fb1ec15bde0 2023-11-24T07:58:04.790 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:04.786+0000 7fb1f13e6700 1 --2- 172.21.15.59:0/82425946 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7fb1d006af60 0x7fb1d006d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:04.790 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:04.786+0000 7fb1f13e6700 1 --2- 172.21.15.59:0/82425946 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7fb1d006af60 0x7fb1d006d420 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7fb1dc009130 tx=0x7fb1dc008030 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:58:04.791 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:04.790+0000 7fb1e1ffb700 1 -- 172.21.15.59:0/82425946 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fb1e4054e30 con 0x7fb1ec15bde0 2023-11-24T07:58:05.038 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:05.036+0000 7fb1f364a700 1 -- 172.21.15.59:0/82425946 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7fb1ec06fd20 con 0x7fb1ec15bde0 2023-11-24T07:58:05.627 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:05.626+0000 7fb1e1ffb700 1 -- 172.21.15.59:0/82425946 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v25) v1 ==== 122+0+0 (secure 0 0 0) 0x7fb1e4016070 con 0x7fb1ec15bde0 2023-11-24T07:58:05.634 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:05.629+0000 7fb1f364a700 1 -- 172.21.15.59:0/82425946 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7fb1d006af60 msgr2=0x7fb1d006d420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:05.634 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:05.629+0000 7fb1f364a700 1 --2- 172.21.15.59:0/82425946 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7fb1d006af60 0x7fb1d006d420 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7fb1dc009130 tx=0x7fb1dc008030 comp rx=0 tx=0).stop 2023-11-24T07:58:05.634 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:05.629+0000 7fb1f364a700 1 -- 172.21.15.59:0/82425946 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb1ec15bde0 msgr2=0x7fb1ec1592a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:05.634 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:05.629+0000 7fb1f364a700 1 --2- 172.21.15.59:0/82425946 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb1ec15bde0 0x7fb1ec1592a0 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7fb1e400cc10 tx=0x7fb1e400a060 comp rx=0 tx=0).stop 2023-11-24T07:58:05.634 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:05.629+0000 7fb1f364a700 1 -- 172.21.15.59:0/82425946 shutdown_connections 2023-11-24T07:58:05.634 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:05.629+0000 7fb1f364a700 1 --2- 172.21.15.59:0/82425946 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7fb1d006af60 0x7fb1d006d420 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:05.635 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:05.629+0000 7fb1f364a700 1 --2- 172.21.15.59:0/82425946 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb1ec15bde0 0x7fb1ec1592a0 unknown :-1 s=CLOSED pgs=541 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:05.635 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:05.629+0000 7fb1f364a700 1 --2- 172.21.15.59:0/82425946 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fb1ec0ce5c0 0x7fb1ec0c9660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:05.635 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:05.629+0000 7fb1f364a700 1 --2- 172.21.15.59:0/82425946 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb1ec0cf2b0 0x7fb1ec0b67d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:05.635 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:05.629+0000 7fb1f364a700 1 -- 172.21.15.59:0/82425946 >> 172.21.15.59:0/82425946 conn(0x7fb1ec0b3720 msgr2=0x7fb1ec15b0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:58:05.635 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:05.629+0000 7fb1f364a700 1 -- 172.21.15.59:0/82425946 shutdown_connections 2023-11-24T07:58:05.635 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:05.629+0000 7fb1f364a700 1 -- 172.21.15.59:0/82425946 wait complete. 2023-11-24T07:58:05.645 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2023-11-24T07:58:05.646 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-24T07:58:05.777 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:05.775+0000 7eff14cdb700 1 Processor -- start 2023-11-24T07:58:05.777 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:05.775+0000 7eff14cdb700 1 -- start start 2023-11-24T07:58:05.781 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:05.776+0000 7eff14cdb700 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7eff100b3cb0 0x7eff10072e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:05.781 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:05.776+0000 7eff14cdb700 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7eff1006b470 con 0x7eff1015f820 2023-11-24T07:58:05.781 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:05.776+0000 7eff14cdb700 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7eff10073360 con 0x7eff1015f440 2023-11-24T07:58:05.781 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:05.776+0000 7eff14cdb700 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 v1 -- 0x7eff100735e0 con 0x7eff100b3cb0 2023-11-24T07:58:05.781 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:05.776+0000 7eff0ed9d700 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7eff1015f820 0x7eff100b0930 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:56238/0 (socket says 172.21.15.59:56238) 2023-11-24T07:58:05.781 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:05.776+0000 7eff0ed9d700 1 -- 172.21.15.59:0/980146738 learned_addr learned my addr 172.21.15.59:0/980146738 (peer_addr_for_me v1:172.21.15.59:0/0) 2023-11-24T07:58:05.781 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:05.777+0000 7eff0dd9b700 1 --2- 172.21.15.59:0/980146738 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7eff100b3cb0 0x7eff10072e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:05.781 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:05.777+0000 7eff0d59a700 1 -- 172.21.15.59:0/980146738 <== mon.0 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 750169639 0 0) 0x7eff1006b470 con 0x7eff1015f820 2023-11-24T07:58:05.781 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:05.777+0000 7eff0d59a700 1 -- 172.21.15.59:0/980146738 --> v1:172.21.15.59:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7efef80035b0 con 0x7eff1015f820 2023-11-24T07:58:05.782 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:05.777+0000 7eff0d59a700 1 -- 172.21.15.59:0/980146738 <== mon.1 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3241151400 0 0) 0x7eff10073360 con 0x7eff1015f440 2023-11-24T07:58:05.782 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:05.777+0000 7eff0d59a700 1 -- 172.21.15.59:0/980146738 --> v1:172.21.15.76:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7eff1006b470 con 0x7eff1015f440 2023-11-24T07:58:05.782 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:05.777+0000 7eff0d59a700 1 -- 172.21.15.59:0/980146738 <== mon.0 v1:172.21.15.59:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1001149608 0 0) 0x7efef80035b0 con 0x7eff1015f820 2023-11-24T07:58:05.782 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:05.777+0000 7eff0d59a700 1 -- 172.21.15.59:0/980146738 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7eff100b3cb0 msgr2=0x7eff10072e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T07:58:05.782 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:05.777+0000 7eff0d59a700 1 --2- 172.21.15.59:0/980146738 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7eff100b3cb0 0x7eff10072e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:05.782 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:05.777+0000 7eff0d59a700 1 -- 172.21.15.59:0/980146738 >> v1:172.21.15.76:6789/0 conn(0x7eff1015f440 legacy=0x7eff10072710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:05.782 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:05.777+0000 7eff0d59a700 1 -- 172.21.15.59:0/980146738 --> v1:172.21.15.59:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7eff10067370 con 0x7eff1015f820 2023-11-24T07:58:05.782 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:05.777+0000 7eff0d59a700 1 -- 172.21.15.59:0/980146738 <== mon.0 v1:172.21.15.59:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1312333566 0 0) 0x7eff080019f0 con 0x7eff1015f820 2023-11-24T07:58:05.782 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:05.777+0000 7eff0d59a700 1 -- 172.21.15.59:0/980146738 >> v1:172.21.15.59:6789/0 conn(0x7eff1015f820 legacy=0x7eff100b0930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:05.782 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:05.777+0000 7eff0d59a700 1 --2- 172.21.15.59:0/980146738 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7efef8003bc0 0x7efef8003fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:05.782 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:05.777+0000 7eff0d59a700 1 --2- 172.21.15.59:0/980146738 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7efef8004850 0x7efef8004cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:05.783 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:05.777+0000 7eff0d59a700 1 --2- 172.21.15.59:0/980146738 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7eff100b3cb0 0x7efef80052c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:05.783 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:05.777+0000 7eff0d59a700 1 -- 172.21.15.59:0/980146738 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff100735e0 con 0x7efef8004850 2023-11-24T07:58:05.783 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:05.777+0000 7eff0d59a700 1 -- 172.21.15.59:0/980146738 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efef800bbd0 con 0x7efef8003bc0 2023-11-24T07:58:05.783 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:05.777+0000 7eff0d59a700 1 -- 172.21.15.59:0/980146738 --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efef800bd90 con 0x7eff100b3cb0 2023-11-24T07:58:05.783 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:05.777+0000 7eff0dd9b700 1 --2- 172.21.15.59:0/980146738 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7efef8004850 0x7efef8004cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:05.783 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:05.777+0000 7eff0e59c700 1 --2- 172.21.15.59:0/980146738 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7efef8003bc0 0x7efef8003fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:05.783 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:05.777+0000 7eff0ed9d700 1 --2- 172.21.15.59:0/980146738 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7eff100b3cb0 0x7efef80052c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:05.783 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:05.778+0000 7eff0dd9b700 1 -- 172.21.15.59:0/980146738 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7eff100b3cb0 msgr2=0x7efef80052c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T07:58:05.783 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:05.778+0000 7eff0dd9b700 1 --2- 172.21.15.59:0/980146738 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7eff100b3cb0 0x7efef80052c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:05.783 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:05.778+0000 7eff0dd9b700 1 -- 172.21.15.59:0/980146738 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7efef8003bc0 msgr2=0x7efef8003fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T07:58:05.783 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:05.778+0000 7eff0dd9b700 1 --2- 172.21.15.59:0/980146738 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7efef8003bc0 0x7efef8003fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:05.783 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:05.778+0000 7eff0dd9b700 1 -- 172.21.15.59:0/980146738 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efef80035b0 con 0x7efef8004850 2023-11-24T07:58:05.784 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:05.778+0000 7eff0dd9b700 1 --2- 172.21.15.59:0/980146738 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7efef8004850 0x7efef8004cd0 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7eff00009b30 tx=0x7efef8005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1a8b37a9e9ba7205 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:58:05.784 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:05.778+0000 7eff0d59a700 1 -- 172.21.15.59:0/980146738 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff0000e6c0 con 0x7efef8004850 2023-11-24T07:58:05.784 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:05.778+0000 7eff0d59a700 1 -- 172.21.15.59:0/980146738 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7eff0000e820 con 0x7efef8004850 2023-11-24T07:58:05.784 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:05.778+0000 7eff0d59a700 1 -- 172.21.15.59:0/980146738 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff0000eb90 con 0x7efef8004850 2023-11-24T07:58:05.784 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:05.778+0000 7eff14cdb700 1 -- 172.21.15.59:0/980146738 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7efef8004850 msgr2=0x7efef8004cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:05.784 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:05.778+0000 7eff14cdb700 1 --2- 172.21.15.59:0/980146738 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7efef8004850 0x7efef8004cd0 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7eff00009b30 tx=0x7efef8005880 comp rx=0 tx=0).stop 2023-11-24T07:58:05.784 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:05.778+0000 7eff0ed9d700 1 -- 172.21.15.59:0/980146738 reap_dead start 2023-11-24T07:58:05.784 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:05.779+0000 7eff14cdb700 1 -- 172.21.15.59:0/980146738 shutdown_connections 2023-11-24T07:58:05.784 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:05.779+0000 7eff14cdb700 1 -- 172.21.15.59:0/980146738 >> 172.21.15.59:0/980146738 conn(0x7eff100b3190 msgr2=0x7eff100b35a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:58:05.784 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:05.779+0000 7eff14cdb700 1 -- 172.21.15.59:0/980146738 shutdown_connections 2023-11-24T07:58:05.784 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:05.779+0000 7eff14cdb700 1 -- 172.21.15.59:0/980146738 wait complete. 2023-11-24T07:58:05.787 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:05.780+0000 7eff14cdb700 1 Processor -- start 2023-11-24T07:58:05.787 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:05.780+0000 7eff14cdb700 1 -- start start 2023-11-24T07:58:05.787 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:05.781+0000 7eff14cdb700 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7efef8004850 0x7eff100c3260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:05.787 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:05.781+0000 7eff14cdb700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7eff100c9eb0 0x7eff100c55e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:05.787 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:05.781+0000 7eff14cdb700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7eff100c98f0 0x7eff100c5b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:05.787 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:05.781+0000 7eff14cdb700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff100c96b0 con 0x7eff100c98f0 2023-11-24T07:58:05.787 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:05.781+0000 7eff14cdb700 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff100c8e10 con 0x7eff100c9eb0 2023-11-24T07:58:05.787 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:05.781+0000 7eff14cdb700 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 v1 -- 0x7eff100c90f0 con 0x7efef8004850 2023-11-24T07:58:05.787 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:05.781+0000 7eff0ed9d700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7eff100c98f0 0x7eff100c5b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:05.788 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:05.781+0000 7eff0ed9d700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7eff100c98f0 0x7eff100c5b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:52776/0 (socket says 172.21.15.59:52776) 2023-11-24T07:58:05.788 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:05.781+0000 7eff0ed9d700 1 -- 172.21.15.59:0/708759776 learned_addr learned my addr 172.21.15.59:0/708759776 (peer_addr_for_me v2:172.21.15.59:0/0) 2023-11-24T07:58:05.788 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:05.781+0000 7eff0e59c700 1 --2- 172.21.15.59:0/708759776 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7efef8004850 0x7eff100c3260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:05.788 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:05.781+0000 7eff0dd9b700 1 --2- 172.21.15.59:0/708759776 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7eff100c9eb0 0x7eff100c55e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:05.788 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:05.781+0000 7eff0ed9d700 1 -- 172.21.15.59:0/708759776 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7efef8004850 msgr2=0x7eff100c3260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:05.788 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:05.781+0000 7eff0ed9d700 1 --2- 172.21.15.59:0/708759776 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7efef8004850 0x7eff100c3260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:05.788 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:05.781+0000 7eff0ed9d700 1 -- 172.21.15.59:0/708759776 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7eff100c9eb0 msgr2=0x7eff100c55e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:05.788 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:05.781+0000 7eff0ed9d700 1 --2- 172.21.15.59:0/708759776 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7eff100c9eb0 0x7eff100c55e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:05.788 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:05.781+0000 7eff0ed9d700 1 -- 172.21.15.59:0/708759776 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7eff100c93d0 con 0x7eff100c98f0 2023-11-24T07:58:05.788 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:05.781+0000 7eff0ed9d700 1 --2- 172.21.15.59:0/708759776 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7eff100c98f0 0x7eff100c5b20 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7eff10070770 tx=0x7eff0800a5b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:58:05.788 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:05.781+0000 7efefeffd700 1 -- 172.21.15.59:0/708759776 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff080013a0 con 0x7eff100c98f0 2023-11-24T07:58:05.788 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:05.781+0000 7efefeffd700 1 -- 172.21.15.59:0/708759776 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7eff0800ad20 con 0x7eff100c98f0 2023-11-24T07:58:05.789 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:05.781+0000 7eff14cdb700 1 -- 172.21.15.59:0/708759776 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7eff100ca930 con 0x7eff100c98f0 2023-11-24T07:58:05.789 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:05.781+0000 7efefeffd700 1 -- 172.21.15.59:0/708759776 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff08013630 con 0x7eff100c98f0 2023-11-24T07:58:05.789 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:05.782+0000 7eff14cdb700 1 -- 172.21.15.59:0/708759776 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7eff100c3970 con 0x7eff100c98f0 2023-11-24T07:58:05.789 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:05.782+0000 7eff14cdb700 1 -- 172.21.15.59:0/708759776 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7eff10068670 con 0x7eff100c98f0 2023-11-24T07:58:05.789 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:05.783+0000 7efefeffd700 1 -- 172.21.15.59:0/708759776 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 6) v1 ==== 89282+0+0 (secure 0 0 0) 0x7eff08013790 con 0x7eff100c98f0 2023-11-24T07:58:05.789 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:05.783+0000 7efefeffd700 1 --2- 172.21.15.59:0/708759776 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7efef006af90 0x7efef006d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:05.789 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:05.783+0000 7efefeffd700 1 -- 172.21.15.59:0/708759776 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 5709+0+0 (secure 0 0 0) 0x7eff08089ca0 con 0x7eff100c98f0 2023-11-24T07:58:05.789 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:05.783+0000 7eff0e59c700 1 --2- 172.21.15.59:0/708759776 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7efef006af90 0x7efef006d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:05.789 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:05.784+0000 7eff0e59c700 1 --2- 172.21.15.59:0/708759776 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7efef006af90 0x7efef006d450 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7eff04001120 tx=0x7eff04009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:58:05.790 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:05.788+0000 7efefeffd700 1 -- 172.21.15.59:0/708759776 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7eff0808e070 con 0x7eff100c98f0 2023-11-24T07:58:06.034 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:06.033+0000 7eff14cdb700 1 -- 172.21.15.59:0/708759776 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7eff100cfe80 con 0x7eff100c98f0 2023-11-24T07:58:06.035 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:06.033+0000 7efefeffd700 1 -- 172.21.15.59:0/708759776 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) v1 ==== 94+0+4904 (secure 0 0 0) 0x7eff08054d60 con 0x7eff100c98f0 2023-11-24T07:58:06.035 INFO:teuthology.orchestra.run.smithi059.stdout: 2023-11-24T07:58:06.035 INFO:teuthology.orchestra.run.smithi059.stdout:{"epoch":25,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5290,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.76:6836/508095662","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6834","nonce":508095662},{"type":"v1","addr":"172.21.15.76:6836","nonce":508095662}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":20},{"gid":5293,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.76:6837/4158118811","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6835","nonce":4158118811},{"type":"v1","addr":"172.21.15.76:6837","nonce":4158118811}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":20},{"gid":5622,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6834/59936286","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6832","nonce":59936286},{"type":"v1","addr":"172.21.15.59:6834","nonce":59936286}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":20}],"filesystems":[{"mdsmap":{"epoch":25,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-24T07:58:02.020496+0000","modified":"2023-11-24T07:58:05.037742+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5625},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5625":{"gid":5625,"name":"c","rank":0,"incarnation":21,"state":"up:active","state_seq":3,"addr":"172.21.15.59:6835/1468266626","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6833","nonce":1468266626},{"type":"v1","addr":"172.21.15.59:6835","nonce":1468266626}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2023-11-24T07:58:06.037 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:06.035+0000 7eff14cdb700 1 -- 172.21.15.59:0/708759776 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7efef006af90 msgr2=0x7efef006d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:06.037 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:06.035+0000 7eff14cdb700 1 --2- 172.21.15.59:0/708759776 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7efef006af90 0x7efef006d450 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7eff04001120 tx=0x7eff04009040 comp rx=0 tx=0).stop 2023-11-24T07:58:06.037 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:06.035+0000 7eff14cdb700 1 -- 172.21.15.59:0/708759776 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7eff100c98f0 msgr2=0x7eff100c5b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:06.037 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:06.035+0000 7eff14cdb700 1 --2- 172.21.15.59:0/708759776 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7eff100c98f0 0x7eff100c5b20 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7eff10070770 tx=0x7eff0800a5b0 comp rx=0 tx=0).stop 2023-11-24T07:58:06.038 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:06.035+0000 7eff14cdb700 1 -- 172.21.15.59:0/708759776 shutdown_connections 2023-11-24T07:58:06.038 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:06.035+0000 7eff14cdb700 1 --2- 172.21.15.59:0/708759776 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7efef006af90 0x7efef006d450 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:06.038 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:06.035+0000 7eff14cdb700 1 --2- 172.21.15.59:0/708759776 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7eff100c98f0 0x7eff100c5b20 unknown :-1 s=CLOSED pgs=544 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:06.038 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:06.035+0000 7eff14cdb700 1 --2- 172.21.15.59:0/708759776 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7efef8004850 0x7eff100c3260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:06.038 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:06.035+0000 7eff14cdb700 1 --2- 172.21.15.59:0/708759776 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7eff100c9eb0 0x7eff100c55e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:06.038 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:06.035+0000 7eff14cdb700 1 -- 172.21.15.59:0/708759776 >> 172.21.15.59:0/708759776 conn(0x7eff100b3190 msgr2=0x7eff1006d5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:58:06.038 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:06.035+0000 7eff14cdb700 1 -- 172.21.15.59:0/708759776 shutdown_connections 2023-11-24T07:58:06.038 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:06.035+0000 7eff14cdb700 1 -- 172.21.15.59:0/708759776 wait complete. 2023-11-24T07:58:06.038 INFO:teuthology.orchestra.run.smithi059.stderr:dumped fsmap epoch 25 2023-11-24T07:58:06.049 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-24T07:58:06.173 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:06.172+0000 7fcae3a36700 1 Processor -- start 2023-11-24T07:58:06.174 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:06.172+0000 7fcae3a36700 1 -- start start 2023-11-24T07:58:06.176 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:06.173+0000 7fcae3a36700 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fcadc145100 0x7fcadc06e280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:06.176 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:06.173+0000 7fcae3a36700 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcadc160720 con 0x7fcadc06b6e0 2023-11-24T07:58:06.176 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:06.173+0000 7fcae3a36700 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcadc06f710 con 0x7fcadc144c10 2023-11-24T07:58:06.176 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:06.173+0000 7fcae3a36700 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcadc06f500 con 0x7fcadc145100 2023-11-24T07:58:06.176 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:06.173+0000 7fcae0fd1700 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7fcadc06b6e0 0x7fcadc06db70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:56256/0 (socket says 172.21.15.59:56256) 2023-11-24T07:58:06.176 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:06.173+0000 7fcae0fd1700 1 -- 172.21.15.59:0/2553556462 learned_addr learned my addr 172.21.15.59:0/2553556462 (peer_addr_for_me v1:172.21.15.59:0/0) 2023-11-24T07:58:06.176 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:06.173+0000 7fcae1fd3700 1 --2- 172.21.15.59:0/2553556462 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fcadc145100 0x7fcadc06e280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:06.177 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:06.173+0000 7fcad3fff700 1 -- 172.21.15.59:0/2553556462 <== mon.0 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1023728711 0 0) 0x7fcadc160720 con 0x7fcadc06b6e0 2023-11-24T07:58:06.177 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:06.173+0000 7fcad3fff700 1 -- 172.21.15.59:0/2553556462 --> v1:172.21.15.59:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcac80035b0 con 0x7fcadc06b6e0 2023-11-24T07:58:06.177 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:06.173+0000 7fcad3fff700 1 -- 172.21.15.59:0/2553556462 <== mon.1 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 44561217 0 0) 0x7fcadc06f710 con 0x7fcadc144c10 2023-11-24T07:58:06.177 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:06.173+0000 7fcad3fff700 1 -- 172.21.15.59:0/2553556462 --> v1:172.21.15.76:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcadc160720 con 0x7fcadc144c10 2023-11-24T07:58:06.177 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:06.173+0000 7fcae1fd3700 1 -- 172.21.15.59:0/2553556462 >> v1:172.21.15.76:6789/0 conn(0x7fcadc144c10 legacy=0x7fcadc06afd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:06.177 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:06.173+0000 7fcae1fd3700 1 -- 172.21.15.59:0/2553556462 >> v1:172.21.15.59:6789/0 conn(0x7fcadc06b6e0 legacy=0x7fcadc06db70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:06.177 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:06.173+0000 7fcad3fff700 1 -- 172.21.15.59:0/2553556462 <== mon.0 v1:172.21.15.59:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3421199423 0 0) 0x7fcac80035b0 con 0x7fcadc06b6e0 2023-11-24T07:58:06.177 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:06.173+0000 7fcae1fd3700 1 -- 172.21.15.59:0/2553556462 --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcadc06f9f0 con 0x7fcadc145100 2023-11-24T07:58:06.177 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:06.174+0000 7fcae1fd3700 1 --2- 172.21.15.59:0/2553556462 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fcadc145100 0x7fcadc06e280 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fcad4009dc0 tx=0x7fcad400b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=67daeda18ce407f5 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:58:06.177 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:06.174+0000 7fcad3fff700 1 -- 172.21.15.59:0/2553556462 <== mon.2 v2:172.21.15.76:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcad400e070 con 0x7fcadc145100 2023-11-24T07:58:06.177 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:06.174+0000 7fcad3fff700 1 -- 172.21.15.59:0/2553556462 <== mon.2 v2:172.21.15.76:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcad4003a20 con 0x7fcadc145100 2023-11-24T07:58:06.177 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:06.174+0000 7fcad3fff700 1 -- 172.21.15.59:0/2553556462 <== mon.2 v2:172.21.15.76:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcacc001670 con 0x7fcadc145100 2023-11-24T07:58:06.178 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:06.174+0000 7fcae3a36700 1 -- 172.21.15.59:0/2553556462 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fcadc145100 msgr2=0x7fcadc06e280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:06.178 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:06.174+0000 7fcae3a36700 1 --2- 172.21.15.59:0/2553556462 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fcadc145100 0x7fcadc06e280 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fcad4009dc0 tx=0x7fcad400b330 comp rx=0 tx=0).stop 2023-11-24T07:58:06.178 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:06.174+0000 7fcae3a36700 1 -- 172.21.15.59:0/2553556462 shutdown_connections 2023-11-24T07:58:06.178 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:06.174+0000 7fcae3a36700 1 --2- 172.21.15.59:0/2553556462 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fcadc145100 0x7fcadc06e280 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:06.178 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:06.174+0000 7fcae3a36700 1 -- 172.21.15.59:0/2553556462 >> 172.21.15.59:0/2553556462 conn(0x7fcadc0b2740 msgr2=0x7fcadc0b2b20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:58:06.178 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:06.174+0000 7fcae3a36700 1 -- 172.21.15.59:0/2553556462 shutdown_connections 2023-11-24T07:58:06.178 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:06.174+0000 7fcae3a36700 1 -- 172.21.15.59:0/2553556462 wait complete. 2023-11-24T07:58:06.180 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:06.176+0000 7fcae3a36700 1 Processor -- start 2023-11-24T07:58:06.180 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:06.176+0000 7fcae3a36700 1 -- start start 2023-11-24T07:58:06.181 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:06.177+0000 7fcae3a36700 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fcadc06b6e0 0x7fcadc15efc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:06.181 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:06.177+0000 7fcae3a36700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fcadc144c10 0x7fcadc140ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:06.181 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:06.177+0000 7fcae3a36700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fcadc145100 0x7fcadc1e7a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:06.181 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:06.177+0000 7fcae3a36700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcadc1411e0 con 0x7fcadc145100 2023-11-24T07:58:06.181 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:06.177+0000 7fcae3a36700 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcadc15eb30 con 0x7fcadc144c10 2023-11-24T07:58:06.181 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:06.177+0000 7fcae3a36700 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcadc15df20 con 0x7fcadc06b6e0 2023-11-24T07:58:06.181 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:06.177+0000 7fcae1fd3700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fcadc145100 0x7fcadc1e7a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:06.181 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:06.177+0000 7fcae1fd3700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fcadc145100 0x7fcadc1e7a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:52790/0 (socket says 172.21.15.59:52790) 2023-11-24T07:58:06.181 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:06.177+0000 7fcae1fd3700 1 -- 172.21.15.59:0/1886863510 learned_addr learned my addr 172.21.15.59:0/1886863510 (peer_addr_for_me v2:172.21.15.59:0/0) 2023-11-24T07:58:06.181 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:06.177+0000 7fcae17d2700 1 --2- 172.21.15.59:0/1886863510 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fcadc06b6e0 0x7fcadc15efc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:06.181 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:06.177+0000 7fcae0fd1700 1 --2- 172.21.15.59:0/1886863510 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fcadc144c10 0x7fcadc140ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:06.182 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:06.177+0000 7fcae1fd3700 1 -- 172.21.15.59:0/1886863510 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fcadc06b6e0 msgr2=0x7fcadc15efc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:06.182 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:06.177+0000 7fcae1fd3700 1 --2- 172.21.15.59:0/1886863510 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fcadc06b6e0 0x7fcadc15efc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:06.182 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:06.177+0000 7fcae1fd3700 1 -- 172.21.15.59:0/1886863510 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fcadc144c10 msgr2=0x7fcadc140ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:06.182 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:06.177+0000 7fcae1fd3700 1 --2- 172.21.15.59:0/1886863510 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fcadc144c10 0x7fcadc140ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:06.182 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:06.177+0000 7fcae1fd3700 1 -- 172.21.15.59:0/1886863510 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcad4009a70 con 0x7fcadc145100 2023-11-24T07:58:06.182 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:06.177+0000 7fcae0fd1700 1 --2- 172.21.15.59:0/1886863510 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fcadc144c10 0x7fcadc140ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T07:58:06.182 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:06.178+0000 7fcae1fd3700 1 --2- 172.21.15.59:0/1886863510 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fcadc145100 0x7fcadc1e7a00 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7fcad4015040 tx=0x7fcad4004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:58:06.182 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:06.178+0000 7fcad1ffb700 1 -- 172.21.15.59:0/1886863510 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcad40043c0 con 0x7fcadc145100 2023-11-24T07:58:06.182 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:06.178+0000 7fcad1ffb700 1 -- 172.21.15.59:0/1886863510 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcad4004520 con 0x7fcadc145100 2023-11-24T07:58:06.182 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:06.178+0000 7fcae3a36700 1 -- 172.21.15.59:0/1886863510 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcadc15e200 con 0x7fcadc145100 2023-11-24T07:58:06.183 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:06.178+0000 7fcad1ffb700 1 -- 172.21.15.59:0/1886863510 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcad40048b0 con 0x7fcadc145100 2023-11-24T07:58:06.183 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:06.178+0000 7fcae3a36700 1 -- 172.21.15.59:0/1886863510 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcadc15e7c0 con 0x7fcadc145100 2023-11-24T07:58:06.183 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:06.179+0000 7fcae3a36700 1 -- 172.21.15.59:0/1886863510 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcadc15ecc0 con 0x7fcadc145100 2023-11-24T07:58:06.183 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:06.179+0000 7fcad1ffb700 1 -- 172.21.15.59:0/1886863510 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 6) v1 ==== 89282+0+0 (secure 0 0 0) 0x7fcad401c460 con 0x7fcadc145100 2023-11-24T07:58:06.184 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:06.180+0000 7fcad1ffb700 1 --2- 172.21.15.59:0/1886863510 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7fcac006af60 0x7fcac006d420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:06.184 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:06.180+0000 7fcad1ffb700 1 -- 172.21.15.59:0/1886863510 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fcad408ad50 con 0x7fcadc145100 2023-11-24T07:58:06.184 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:06.180+0000 7fcae17d2700 1 --2- 172.21.15.59:0/1886863510 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7fcac006af60 0x7fcac006d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:06.184 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:06.181+0000 7fcae17d2700 1 --2- 172.21.15.59:0/1886863510 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7fcac006af60 0x7fcac006d420 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7fcad8001220 tx=0x7fcad8009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:58:06.186 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:06.185+0000 7fcad1ffb700 1 -- 172.21.15.59:0/1886863510 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fcad4055d90 con 0x7fcadc145100 2023-11-24T07:58:06.385 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:06.383+0000 7fcae3a36700 1 -- 172.21.15.59:0/1886863510 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fcadc068670 con 0x7fcadc145100 2023-11-24T07:58:06.386 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:06.385+0000 7fcad1ffb700 1 -- 172.21.15.59:0/1886863510 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v45) v1 ==== 74+0+16437 (secure 0 0 0) 0x7fcad4016090 con 0x7fcadc145100 2023-11-24T07:58:06.387 INFO:teuthology.orchestra.run.smithi059.stdout: 2023-11-24T07:58:06.387 INFO:teuthology.orchestra.run.smithi059.stdout:{"epoch":45,"fsid":"9f0e8741-b8f9-48d5-a2c3-09f45903303d","created":"2023-11-24T07:56:06.552729+0000","modified":"2023-11-24T07:58:03.605203+0000","last_up_change":"2023-11-24T07:56:17.372352+0000","last_in_change":"2023-11-24T07:56:09.073250+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-24T07:56:18.352823+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_metadata","create_time":"2023-11-24T07:58:00.240130+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"44","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_data","create_time":"2023-11-24T07:58:00.977486+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"45","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"6d8f9e9b-1782-4e41-928b-a39f2a8ebc06","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6800","nonce":3373878772},{"type":"v1","addr":"172.21.15.59:6801","nonce":3373878772}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6802","nonce":3373878772},{"type":"v1","addr":"172.21.15.59:6803","nonce":3373878772}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6806","nonce":3373878772},{"type":"v1","addr":"172.21.15.59:6807","nonce":3373878772}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6804","nonce":3373878772},{"type":"v1","addr":"172.21.15.59:6805","nonce":3373878772}]},"public_addr":"172.21.15.59:6801/3373878772","cluster_addr":"172.21.15.59:6803/3373878772","heartbeat_back_addr":"172.21.15.59:6807/3373878772","heartbeat_front_addr":"172.21.15.59:6805/3373878772","state":["exists","up"]},{"osd":1,"uuid":"e8d882aa-52e1-4c04-9e1e-c44ef9204a5f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6824","nonce":648258563},{"type":"v1","addr":"172.21.15.59:6825","nonce":648258563}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6826","nonce":648258563},{"type":"v1","addr":"172.21.15.59:6827","nonce":648258563}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6830","nonce":648258563},{"type":"v1","addr":"172.21.15.59:6831","nonce":648258563}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6828","nonce":648258563},{"type":"v1","addr":"172.21.15.59:6829","nonce":648258563}]},"public_addr":"172.21.15.59:6825/648258563","cluster_addr":"172.21.15.59:6827/648258563","heartbeat_back_addr":"172.21.15.59:6831/648258563","heartbeat_front_addr":"172.21.15.59:6829/648258563","state":["exists","up"]},{"osd":2,"uuid":"69eebfa0-695b-4b21-90b9-8a3537168a01","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6808","nonce":1840430654},{"type":"v1","addr":"172.21.15.59:6809","nonce":1840430654}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6810","nonce":1840430654},{"type":"v1","addr":"172.21.15.59:6811","nonce":1840430654}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6814","nonce":1840430654},{"type":"v1","addr":"172.21.15.59:6815","nonce":1840430654}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6812","nonce":1840430654},{"type":"v1","addr":"172.21.15.59:6813","nonce":1840430654}]},"public_addr":"172.21.15.59:6809/1840430654","cluster_addr":"172.21.15.59:6811/1840430654","heartbeat_back_addr":"172.21.15.59:6815/1840430654","heartbeat_front_addr":"172.21.15.59:6813/1840430654","state":["exists","up"]},{"osd":3,"uuid":"96fd77fc-f911-46ff-9445-6f0be2daa8f4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6816","nonce":2504025850},{"type":"v1","addr":"172.21.15.59:6817","nonce":2504025850}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6818","nonce":2504025850},{"type":"v1","addr":"172.21.15.59:6819","nonce":2504025850}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6822","nonce":2504025850},{"type":"v1","addr":"172.21.15.59:6823","nonce":2504025850}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6820","nonce":2504025850},{"type":"v1","addr":"172.21.15.59:6821","nonce":2504025850}]},"public_addr":"172.21.15.59:6817/2504025850","cluster_addr":"172.21.15.59:6819/2504025850","heartbeat_back_addr":"172.21.15.59:6823/2504025850","heartbeat_front_addr":"172.21.15.59:6821/2504025850","state":["exists","up"]},{"osd":4,"uuid":"7ac29fca-b5b2-4c95-9d69-2f164cbbc708","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6816","nonce":2382463431},{"type":"v1","addr":"172.21.15.76:6818","nonce":2382463431}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6820","nonce":2382463431},{"type":"v1","addr":"172.21.15.76:6822","nonce":2382463431}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6828","nonce":2382463431},{"type":"v1","addr":"172.21.15.76:6830","nonce":2382463431}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6824","nonce":2382463431},{"type":"v1","addr":"172.21.15.76:6826","nonce":2382463431}]},"public_addr":"172.21.15.76:6818/2382463431","cluster_addr":"172.21.15.76:6822/2382463431","heartbeat_back_addr":"172.21.15.76:6830/2382463431","heartbeat_front_addr":"172.21.15.76:6826/2382463431","state":["exists","up"]},{"osd":5,"uuid":"485bf711-3851-4238-a4f4-51d52e0a9c58","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6800","nonce":329334308},{"type":"v1","addr":"172.21.15.76:6802","nonce":329334308}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6805","nonce":329334308},{"type":"v1","addr":"172.21.15.76:6807","nonce":329334308}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6814","nonce":329334308},{"type":"v1","addr":"172.21.15.76:6815","nonce":329334308}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6810","nonce":329334308},{"type":"v1","addr":"172.21.15.76:6812","nonce":329334308}]},"public_addr":"172.21.15.76:6802/329334308","cluster_addr":"172.21.15.76:6807/329334308","heartbeat_back_addr":"172.21.15.76:6815/329334308","heartbeat_front_addr":"172.21.15.76:6812/329334308","state":["exists","up"]},{"osd":6,"uuid":"72b3f2fc-1e4a-4e32-968d-003647a0286f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6801","nonce":3946579410},{"type":"v1","addr":"172.21.15.76:6803","nonce":3946579410}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6804","nonce":3946579410},{"type":"v1","addr":"172.21.15.76:6806","nonce":3946579410}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6811","nonce":3946579410},{"type":"v1","addr":"172.21.15.76:6813","nonce":3946579410}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6808","nonce":3946579410},{"type":"v1","addr":"172.21.15.76:6809","nonce":3946579410}]},"public_addr":"172.21.15.76:6803/3946579410","cluster_addr":"172.21.15.76:6806/3946579410","heartbeat_back_addr":"172.21.15.76:6813/3946579410","heartbeat_front_addr":"172.21.15.76:6809/3946579410","state":["exists","up"]},{"osd":7,"uuid":"1ddceb90-6c5d-4d1a-ad56-99cb877f8c70","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6817","nonce":841761538},{"type":"v1","addr":"172.21.15.76:6819","nonce":841761538}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6821","nonce":841761538},{"type":"v1","addr":"172.21.15.76:6823","nonce":841761538}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6829","nonce":841761538},{"type":"v1","addr":"172.21.15.76:6831","nonce":841761538}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6825","nonce":841761538},{"type":"v1","addr":"172.21.15.76:6827","nonce":841761538}]},"public_addr":"172.21.15.76:6819/841761538","cluster_addr":"172.21.15.76:6823/841761538","heartbeat_back_addr":"172.21.15.76:6831/841761538","heartbeat_front_addr":"172.21.15.76:6827/841761538","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T07:56:14.294838+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T07:56:14.270779+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T07:56:14.231809+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T07:56:14.260983+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T07:56:14.302997+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T07:56:14.258600+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T07:56:14.251413+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T07:56:14.304727+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-24T07:58:06.389 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:06.387+0000 7fcae3a36700 1 -- 172.21.15.59:0/1886863510 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7fcac006af60 msgr2=0x7fcac006d420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:06.389 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:06.387+0000 7fcae3a36700 1 --2- 172.21.15.59:0/1886863510 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7fcac006af60 0x7fcac006d420 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7fcad8001220 tx=0x7fcad8009040 comp rx=0 tx=0).stop 2023-11-24T07:58:06.389 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:06.387+0000 7fcae3a36700 1 -- 172.21.15.59:0/1886863510 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fcadc145100 msgr2=0x7fcadc1e7a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:06.389 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:06.387+0000 7fcae3a36700 1 --2- 172.21.15.59:0/1886863510 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fcadc145100 0x7fcadc1e7a00 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7fcad4015040 tx=0x7fcad4004ad0 comp rx=0 tx=0).stop 2023-11-24T07:58:06.389 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:06.387+0000 7fcae3a36700 1 -- 172.21.15.59:0/1886863510 shutdown_connections 2023-11-24T07:58:06.389 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:06.387+0000 7fcae3a36700 1 --2- 172.21.15.59:0/1886863510 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7fcac006af60 0x7fcac006d420 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:06.390 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:06.387+0000 7fcae3a36700 1 --2- 172.21.15.59:0/1886863510 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fcadc145100 0x7fcadc1e7a00 unknown :-1 s=CLOSED pgs=546 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:06.390 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:06.387+0000 7fcae3a36700 1 --2- 172.21.15.59:0/1886863510 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fcadc06b6e0 0x7fcadc15efc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:06.390 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:06.387+0000 7fcae3a36700 1 --2- 172.21.15.59:0/1886863510 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fcadc144c10 0x7fcadc140ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:06.390 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:06.387+0000 7fcae3a36700 1 -- 172.21.15.59:0/1886863510 >> 172.21.15.59:0/1886863510 conn(0x7fcadc0b2740 msgr2=0x7fcadc072390 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:58:06.390 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:06.387+0000 7fcae3a36700 1 -- 172.21.15.59:0/1886863510 shutdown_connections 2023-11-24T07:58:06.390 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:06.387+0000 7fcae3a36700 1 -- 172.21.15.59:0/1886863510 wait complete. 2023-11-24T07:58:06.401 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2023-11-24T07:58:06.401 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-11-24T07:58:06.527 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:06.525+0000 7fcd7e887700 1 Processor -- start 2023-11-24T07:58:06.527 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:06.525+0000 7fcd7e887700 1 -- start start 2023-11-24T07:58:06.529 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:06.526+0000 7fcd7e887700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fcd780722c0 0x7fcd780b51b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:06.529 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:06.526+0000 7fcd7e887700 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcd78143ec0 con 0x7fcd78143ae0 2023-11-24T07:58:06.529 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:06.526+0000 7fcd7e887700 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcd780b3ae0 con 0x7fcd781440f0 2023-11-24T07:58:06.529 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:06.526+0000 7fcd7e887700 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcd780b40a0 con 0x7fcd780722c0 2023-11-24T07:58:06.529 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:06.526+0000 7fcd777fe700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fcd780722c0 0x7fcd780b51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:06.530 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:06.526+0000 7fcd77fff700 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7fcd78143ae0 0x7fcd78071bb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:56268/0 (socket says 172.21.15.59:56268) 2023-11-24T07:58:06.530 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:06.526+0000 7fcd77fff700 1 -- 172.21.15.59:0/2269328053 learned_addr learned my addr 172.21.15.59:0/2269328053 (peer_addr_for_me v1:172.21.15.59:0/0) 2023-11-24T07:58:06.530 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:06.526+0000 7fcd777fe700 1 -- 172.21.15.59:0/2269328053 >> v1:172.21.15.76:6789/0 conn(0x7fcd781440f0 legacy=0x7fcd7806e510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:06.530 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:06.526+0000 7fcd777fe700 1 -- 172.21.15.59:0/2269328053 >> v1:172.21.15.59:6789/0 conn(0x7fcd78143ae0 legacy=0x7fcd78071bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:06.530 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:06.526+0000 7fcd777fe700 1 -- 172.21.15.59:0/2269328053 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcd780b3dc0 con 0x7fcd780722c0 2023-11-24T07:58:06.530 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:06.526+0000 7fcd777fe700 1 --2- 172.21.15.59:0/2269328053 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fcd780722c0 0x7fcd780b51b0 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7fcd6c009840 tx=0x7fcd6c009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=456f2174703f96ff server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:58:06.530 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:06.527+0000 7fcd76ffd700 1 -- 172.21.15.59:0/2269328053 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd6c004230 con 0x7fcd780722c0 2023-11-24T07:58:06.530 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:06.527+0000 7fcd76ffd700 1 -- 172.21.15.59:0/2269328053 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcd6c004390 con 0x7fcd780722c0 2023-11-24T07:58:06.530 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:06.527+0000 7fcd76ffd700 1 -- 172.21.15.59:0/2269328053 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd6c0036a0 con 0x7fcd780722c0 2023-11-24T07:58:06.531 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:06.527+0000 7fcd7e887700 1 -- 172.21.15.59:0/2269328053 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fcd780722c0 msgr2=0x7fcd780b51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:06.531 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:06.527+0000 7fcd7e887700 1 --2- 172.21.15.59:0/2269328053 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fcd780722c0 0x7fcd780b51b0 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7fcd6c009840 tx=0x7fcd6c009b50 comp rx=0 tx=0).stop 2023-11-24T07:58:06.531 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:06.527+0000 7fcd7e887700 1 -- 172.21.15.59:0/2269328053 shutdown_connections 2023-11-24T07:58:06.531 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:06.527+0000 7fcd7e887700 1 --2- 172.21.15.59:0/2269328053 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fcd780722c0 0x7fcd780b51b0 unknown :-1 s=CLOSED pgs=548 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:06.531 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:06.527+0000 7fcd7e887700 1 -- 172.21.15.59:0/2269328053 >> 172.21.15.59:0/2269328053 conn(0x7fcd780b2740 msgr2=0x7fcd780b2b20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:58:06.531 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:06.527+0000 7fcd7e887700 1 -- 172.21.15.59:0/2269328053 shutdown_connections 2023-11-24T07:58:06.531 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:06.527+0000 7fcd7e887700 1 -- 172.21.15.59:0/2269328053 wait complete. 2023-11-24T07:58:06.534 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:06.529+0000 7fcd7e887700 1 Processor -- start 2023-11-24T07:58:06.534 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:06.529+0000 7fcd7e887700 1 -- start start 2023-11-24T07:58:06.534 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:06.530+0000 7fcd7e887700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fcd780722c0 0x7fcd780cfbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:06.535 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:06.530+0000 7fcd7e887700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fcd78143ae0 0x7fcd7815cd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:06.535 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:06.530+0000 7fcd7e887700 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fcd781440f0 0x7fcd7813f0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:06.535 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:06.530+0000 7fcd7e887700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd7815d6c0 con 0x7fcd78143ae0 2023-11-24T07:58:06.535 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:06.530+0000 7fcd7e887700 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd7815d9a0 con 0x7fcd780722c0 2023-11-24T07:58:06.535 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:06.530+0000 7fcd7e887700 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcd7815dc80 con 0x7fcd781440f0 2023-11-24T07:58:06.535 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:06.530+0000 7fcd777fe700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fcd78143ae0 0x7fcd7815cd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:06.535 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:06.530+0000 7fcd777fe700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fcd78143ae0 0x7fcd7815cd10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:52800/0 (socket says 172.21.15.59:52800) 2023-11-24T07:58:06.535 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:06.530+0000 7fcd777fe700 1 -- 172.21.15.59:0/1354785494 learned_addr learned my addr 172.21.15.59:0/1354785494 (peer_addr_for_me v2:172.21.15.59:0/0) 2023-11-24T07:58:06.535 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:06.530+0000 7fcd77fff700 1 --2- 172.21.15.59:0/1354785494 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fcd780722c0 0x7fcd780cfbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:06.535 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:06.530+0000 7fcd7ce24700 1 --2- 172.21.15.59:0/1354785494 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fcd781440f0 0x7fcd7813f0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:06.535 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:06.530+0000 7fcd777fe700 1 -- 172.21.15.59:0/1354785494 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fcd781440f0 msgr2=0x7fcd7813f0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:06.535 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:06.530+0000 7fcd777fe700 1 --2- 172.21.15.59:0/1354785494 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fcd781440f0 0x7fcd7813f0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:06.536 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:06.530+0000 7fcd777fe700 1 -- 172.21.15.59:0/1354785494 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fcd780722c0 msgr2=0x7fcd780cfbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:06.536 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:06.530+0000 7fcd777fe700 1 --2- 172.21.15.59:0/1354785494 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fcd780722c0 0x7fcd780cfbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:06.536 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:06.530+0000 7fcd777fe700 1 -- 172.21.15.59:0/1354785494 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcd6c0094a0 con 0x7fcd78143ae0 2023-11-24T07:58:06.536 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:06.530+0000 7fcd77fff700 1 --2- 172.21.15.59:0/1354785494 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fcd780722c0 0x7fcd780cfbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-24T07:58:06.536 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:06.530+0000 7fcd777fe700 1 --2- 172.21.15.59:0/1354785494 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fcd78143ae0 0x7fcd7815cd10 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7fcd6c0090b0 tx=0x7fcd6c004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:58:06.536 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:06.531+0000 7fcd74ff9700 1 -- 172.21.15.59:0/1354785494 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd6c011830 con 0x7fcd78143ae0 2023-11-24T07:58:06.536 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:06.531+0000 7fcd74ff9700 1 -- 172.21.15.59:0/1354785494 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcd6c01f070 con 0x7fcd78143ae0 2023-11-24T07:58:06.536 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:06.531+0000 7fcd7e887700 1 -- 172.21.15.59:0/1354785494 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcd7815df60 con 0x7fcd78143ae0 2023-11-24T07:58:06.536 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:06.531+0000 7fcd74ff9700 1 -- 172.21.15.59:0/1354785494 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd6c011bd0 con 0x7fcd78143ae0 2023-11-24T07:58:06.536 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:06.531+0000 7fcd7e887700 1 -- 172.21.15.59:0/1354785494 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcd780d0100 con 0x7fcd78143ae0 2023-11-24T07:58:06.536 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:06.532+0000 7fcd7e887700 1 -- 172.21.15.59:0/1354785494 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcd780cd8d0 con 0x7fcd78143ae0 2023-11-24T07:58:06.537 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:06.532+0000 7fcd74ff9700 1 -- 172.21.15.59:0/1354785494 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 6) v1 ==== 89282+0+0 (secure 0 0 0) 0x7fcd6c011d30 con 0x7fcd78143ae0 2023-11-24T07:58:06.537 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:06.532+0000 7fcd74ff9700 1 --2- 172.21.15.59:0/1354785494 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7fcd6406af60 0x7fcd6406d420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:06.537 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:06.533+0000 7fcd74ff9700 1 -- 172.21.15.59:0/1354785494 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fcd6c021070 con 0x7fcd78143ae0 2023-11-24T07:58:06.537 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:06.533+0000 7fcd77fff700 1 --2- 172.21.15.59:0/1354785494 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7fcd6406af60 0x7fcd6406d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:06.537 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:06.533+0000 7fcd77fff700 1 --2- 172.21.15.59:0/1354785494 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7fcd6406af60 0x7fcd6406d420 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7fcd68000f60 tx=0x7fcd68007e30 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:58:06.538 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:06.537+0000 7fcd74ff9700 1 -- 172.21.15.59:0/1354785494 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fcd6c0562a0 con 0x7fcd78143ae0 2023-11-24T07:58:06.737 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:06.736+0000 7fcd7e887700 1 -- 172.21.15.59:0/1354785494 --> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fcd7815db10 con 0x7fcd6406af60 2023-11-24T07:58:06.746 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:06.744+0000 7fcd74ff9700 1 -- 172.21.15.59:0/1354785494 <== mgr.4100 v2:172.21.15.76:6832/4264046536 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+288939 (secure 0 0 0) 0x7fcd7815db10 con 0x7fcd6406af60 2023-11-24T07:58:06.749 INFO:teuthology.orchestra.run.smithi059.stdout: 2023-11-24T07:58:06.752 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:06.748+0000 7fcd7e887700 1 -- 172.21.15.59:0/1354785494 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7fcd6406af60 msgr2=0x7fcd6406d420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:06.752 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:06.748+0000 7fcd7e887700 1 --2- 172.21.15.59:0/1354785494 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7fcd6406af60 0x7fcd6406d420 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7fcd68000f60 tx=0x7fcd68007e30 comp rx=0 tx=0).stop 2023-11-24T07:58:06.752 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:06.748+0000 7fcd7e887700 1 -- 172.21.15.59:0/1354785494 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fcd78143ae0 msgr2=0x7fcd7815cd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:06.752 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:06.748+0000 7fcd7e887700 1 --2- 172.21.15.59:0/1354785494 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fcd78143ae0 0x7fcd7815cd10 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7fcd6c0090b0 tx=0x7fcd6c004750 comp rx=0 tx=0).stop 2023-11-24T07:58:06.752 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:06.748+0000 7fcd7e887700 1 -- 172.21.15.59:0/1354785494 shutdown_connections 2023-11-24T07:58:06.753 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:06.748+0000 7fcd7e887700 1 --2- 172.21.15.59:0/1354785494 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7fcd6406af60 0x7fcd6406d420 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:06.753 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:06.748+0000 7fcd7e887700 1 --2- 172.21.15.59:0/1354785494 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fcd781440f0 0x7fcd7813f0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:06.753 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:06.748+0000 7fcd7e887700 1 --2- 172.21.15.59:0/1354785494 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fcd780722c0 0x7fcd780cfbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:06.753 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:06.748+0000 7fcd7e887700 1 --2- 172.21.15.59:0/1354785494 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fcd78143ae0 0x7fcd7815cd10 unknown :-1 s=CLOSED pgs=549 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:06.753 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:06.748+0000 7fcd7e887700 1 -- 172.21.15.59:0/1354785494 >> 172.21.15.59:0/1354785494 conn(0x7fcd780b2740 msgr2=0x7fcd7806e410 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:58:06.753 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:06.748+0000 7fcd7e887700 1 -- 172.21.15.59:0/1354785494 shutdown_connections 2023-11-24T07:58:06.753 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:06.748+0000 7fcd7e887700 1 -- 172.21.15.59:0/1354785494 wait complete. 2023-11-24T07:58:06.754 INFO:teuthology.orchestra.run.smithi059.stderr:dumped all 2023-11-24T07:58:06.762 INFO:teuthology.orchestra.run.smithi059.stdout:{"pg_ready":true,"pg_map":{"version":92,"stamp":"2023-11-24T07:58:06.322486+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":42,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":221408,"kb_used_data":5204,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754753312,"statfs":{"total":773094113280,"available":772867391488,"internally_reserved":0,"allocated":5328896,"data_stored":2830971,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12745,"internal_metadata":221236791},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":31,"apply_latency_ms":31,"commit_latency_ns":31000000,"apply_latency_ns":31000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.738457"},"pg_stats":[{"pgid":"7.3f","version":"0'0","reported_seq":9,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-11-24T07:58:02.625818+0000","last_change":"2023-11-24T07:58:02.619364+0000","last_active":"2023-11-24T07:58:02.625818+0000","last_peered":"2023-11-24T07:58:02.625818+0000","last_clean":"2023-11-24T07:58:02.625818+0000","last_became_active":"2023-11-24T07:58:02.618670+0000","last_became_peered":"2023-11-24T07:58:02.618670+0000","last_unstale":"2023-11-24T07:58:02.625818+0000","last_undegraded":"2023-11-24T07:58:02.625818+0000","last_fullsized":"2023-11-24T07:58:02.625818+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:58:01.583599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:58:01.583599+0000","last_clean_scrub_stamp":"2023-11-24T07:58:01.583599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T14:01:26.773202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.3e","version":"0'0","reported_seq":9,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-11-24T07:58:02.624594+0000","last_change":"2023-11-24T07:58:02.621152+0000","last_active":"2023-11-24T07:58:02.624594+0000","last_peered":"2023-11-24T07:58:02.624594+0000","last_clean":"2023-11-24T07:58:02.624594+0000","last_became_active":"2023-11-24T07:58:02.620855+0000","last_became_peered":"2023-11-24T07:58:02.620855+0000","last_unstale":"2023-11-24T07:58:02.624594+0000","last_undegraded":"2023-11-24T07:58:02.624594+0000","last_fullsized":"2023-11-24T07:58:02.624594+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:58:01.583599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:58:01.583599+0000","last_clean_scrub_stamp":"2023-11-24T07:58:01.583599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:33:23.934909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.3d","version":"0'0","reported_seq":9,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-11-24T07:58:02.627667+0000","last_change":"2023-11-24T07:58:02.622555+0000","last_active":"2023-11-24T07:58:02.627667+0000","last_peered":"2023-11-24T07:58:02.627667+0000","last_clean":"2023-11-24T07:58:02.627667+0000","last_became_active":"2023-11-24T07:58:02.622262+0000","last_became_peered":"2023-11-24T07:58:02.622262+0000","last_unstale":"2023-11-24T07:58:02.627667+0000","last_undegraded":"2023-11-24T07:58:02.627667+0000","last_fullsized":"2023-11-24T07:58:02.627667+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:58:01.583599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:58:01.583599+0000","last_clean_scrub_stamp":"2023-11-24T07:58:01.583599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:18:58.981383+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.3c","version":"0'0","reported_seq":9,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-11-24T07:58:02.617142+0000","last_change":"2023-11-24T07:58:02.612749+0000","last_active":"2023-11-24T07:58:02.617142+0000","last_peered":"2023-11-24T07:58:02.617142+0000","last_clean":"2023-11-24T07:58:02.617142+0000","last_became_active":"2023-11-24T07:58:02.612311+0000","last_became_peered":"2023-11-24T07:58:02.612311+0000","last_unstale":"2023-11-24T07:58:02.617142+0000","last_undegraded":"2023-11-24T07:58:02.617142+0000","last_fullsized":"2023-11-24T07:58:02.617142+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:58:01.583599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:58:01.583599+0000","last_clean_scrub_stamp":"2023-11-24T07:58:01.583599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T14:01:26.773202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.3b","version":"0'0","reported_seq":9,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-11-24T07:58:02.619558+0000","last_change":"2023-11-24T07:58:02.612584+0000","last_active":"2023-11-24T07:58:02.619558+0000","last_peered":"2023-11-24T07:58:02.619558+0000","last_clean":"2023-11-24T07:58:02.619558+0000","last_became_active":"2023-11-24T07:58:02.612277+0000","last_became_peered":"2023-11-24T07:58:02.612277+0000","last_unstale":"2023-11-24T07:58:02.619558+0000","last_undegraded":"2023-11-24T07:58:02.619558+0000","last_fullsized":"2023-11-24T07:58:02.619558+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:58:01.583599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:58:01.583599+0000","last_clean_scrub_stamp":"2023-11-24T07:58:01.583599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T13:42:52.916962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.3a","version":"0'0","reported_seq":9,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-11-24T07:58:02.616647+0000","last_change":"2023-11-24T07:58:02.609202+0000","last_active":"2023-11-24T07:58:02.616647+0000","last_peered":"2023-11-24T07:58:02.616647+0000","last_clean":"2023-11-24T07:58:02.616647+0000","last_became_active":"2023-11-24T07:58:02.608893+0000","last_became_peered":"2023-11-24T07:58:02.608893+0000","last_unstale":"2023-11-24T07:58:02.616647+0000","last_undegraded":"2023-11-24T07:58:02.616647+0000","last_fullsized":"2023-11-24T07:58:02.616647+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:58:01.583599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:58:01.583599+0000","last_clean_scrub_stamp":"2023-11-24T07:58:01.583599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T14:02:23.676273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.39","version":"0'0","reported_seq":9,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-11-24T07:58:02.628269+0000","last_change":"2023-11-24T07:58:02.613433+0000","last_active":"2023-11-24T07:58:02.628269+0000","last_peered":"2023-11-24T07:58:02.628269+0000","last_clean":"2023-11-24T07:58:02.628269+0000","last_became_active":"2023-11-24T07:58:02.613091+0000","last_became_peered":"2023-11-24T07:58:02.613091+0000","last_unstale":"2023-11-24T07:58:02.628269+0000","last_undegraded":"2023-11-24T07:58:02.628269+0000","last_fullsized":"2023-11-24T07:58:02.628269+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:58:01.583599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:58:01.583599+0000","last_clean_scrub_stamp":"2023-11-24T07:58:01.583599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T19:14:06.767372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.38","version":"0'0","reported_seq":9,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-11-24T07:58:02.624756+0000","last_change":"2023-11-24T07:58:02.622700+0000","last_active":"2023-11-24T07:58:02.624756+0000","last_peered":"2023-11-24T07:58:02.624756+0000","last_clean":"2023-11-24T07:58:02.624756+0000","last_became_active":"2023-11-24T07:58:02.622338+0000","last_became_peered":"2023-11-24T07:58:02.622338+0000","last_unstale":"2023-11-24T07:58:02.624756+0000","last_undegraded":"2023-11-24T07:58:02.624756+0000","last_fullsized":"2023-11-24T07:58:02.624756+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:58:01.583599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:58:01.583599+0000","last_clean_scrub_stamp":"2023-11-24T07:58:01.583599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T12:50:53.080554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.37","version":"0'0","reported_seq":9,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-11-24T07:58:02.624523+0000","last_change":"2023-11-24T07:58:02.619368+0000","last_active":"2023-11-24T07:58:02.624523+0000","last_peered":"2023-11-24T07:58:02.624523+0000","last_clean":"2023-11-24T07:58:02.624523+0000","last_became_active":"2023-11-24T07:58:02.619037+0000","last_became_peered":"2023-11-24T07:58:02.619037+0000","last_unstale":"2023-11-24T07:58:02.624523+0000","last_undegraded":"2023-11-24T07:58:02.624523+0000","last_fullsized":"2023-11-24T07:58:02.624523+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:58:01.583599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:58:01.583599+0000","last_clean_scrub_stamp":"2023-11-24T07:58:01.583599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T19:15:20.314127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.36","version":"0'0","reported_seq":9,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-11-24T07:58:02.628556+0000","last_change":"2023-11-24T07:58:02.625865+0000","last_active":"2023-11-24T07:58:02.628556+0000","last_peered":"2023-11-24T07:58:02.628556+0000","last_clean":"2023-11-24T07:58:02.628556+0000","last_became_active":"2023-11-24T07:58:02.625623+0000","last_became_peered":"2023-11-24T07:58:02.625623+0000","last_unstale":"2023-11-24T07:58:02.628556+0000","last_undegraded":"2023-11-24T07:58:02.628556+0000","last_fullsized":"2023-11-24T07:58:02.628556+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:58:01.583599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:58:01.583599+0000","last_clean_scrub_stamp":"2023-11-24T07:58:01.583599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T09:35:04.179278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.35","version":"0'0","reported_seq":9,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-11-24T07:58:02.613474+0000","last_change":"2023-11-24T07:58:02.608246+0000","last_active":"2023-11-24T07:58:02.613474+0000","last_peered":"2023-11-24T07:58:02.613474+0000","last_clean":"2023-11-24T07:58:02.613474+0000","last_became_active":"2023-11-24T07:58:02.607979+0000","last_became_peered":"2023-11-24T07:58:02.607979+0000","last_unstale":"2023-11-24T07:58:02.613474+0000","last_undegraded":"2023-11-24T07:58:02.613474+0000","last_fullsized":"2023-11-24T07:58:02.613474+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:58:01.583599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:58:01.583599+0000","last_clean_scrub_stamp":"2023-11-24T07:58:01.583599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T15:36:23.158160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.34","version":"0'0","reported_seq":9,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-11-24T07:58:02.629459+0000","last_change":"2023-11-24T07:58:02.625020+0000","last_active":"2023-11-24T07:58:02.629459+0000","last_peered":"2023-11-24T07:58:02.629459+0000","last_clean":"2023-11-24T07:58:02.629459+0000","last_became_active":"2023-11-24T07:58:02.615508+0000","last_became_peered":"2023-11-24T07:58:02.615508+0000","last_unstale":"2023-11-24T07:58:02.629459+0000","last_undegraded":"2023-11-24T07:58:02.629459+0000","last_fullsized":"2023-11-24T07:58:02.629459+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:58:01.583599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:58:01.583599+0000","last_clean_scrub_stamp":"2023-11-24T07:58:01.583599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T14:02:23.676273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.33","version":"0'0","reported_seq":9,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-11-24T07:58:02.616949+0000","last_change":"2023-11-24T07:58:02.609164+0000","last_active":"2023-11-24T07:58:02.616949+0000","last_peered":"2023-11-24T07:58:02.616949+0000","last_clean":"2023-11-24T07:58:02.616949+0000","last_became_active":"2023-11-24T07:58:02.608870+0000","last_became_peered":"2023-11-24T07:58:02.608870+0000","last_unstale":"2023-11-24T07:58:02.616949+0000","last_undegraded":"2023-11-24T07:58:02.616949+0000","last_fullsized":"2023-11-24T07:58:02.616949+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:58:01.583599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:58:01.583599+0000","last_clean_scrub_stamp":"2023-11-24T07:58:01.583599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:37:31.802505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.32","version":"0'0","reported_seq":9,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-11-24T07:58:02.624190+0000","last_change":"2023-11-24T07:58:02.621645+0000","last_active":"2023-11-24T07:58:02.624190+0000","last_peered":"2023-11-24T07:58:02.624190+0000","last_clean":"2023-11-24T07:58:02.624190+0000","last_became_active":"2023-11-24T07:58:02.621380+0000","last_became_peered":"2023-11-24T07:58:02.621380+0000","last_unstale":"2023-11-24T07:58:02.624190+0000","last_undegraded":"2023-11-24T07:58:02.624190+0000","last_fullsized":"2023-11-24T07:58:02.624190+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:58:01.583599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:58:01.583599+0000","last_clean_scrub_stamp":"2023-11-24T07:58:01.583599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T14:21:21.611502+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.31","version":"0'0","reported_seq":9,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-11-24T07:58:02.624303+0000","last_change":"2023-11-24T07:58:02.621867+0000","last_active":"2023-11-24T07:58:02.624303+0000","last_peered":"2023-11-24T07:58:02.624303+0000","last_clean":"2023-11-24T07:58:02.624303+0000","last_became_active":"2023-11-24T07:58:02.621168+0000","last_became_peered":"2023-11-24T07:58:02.621168+0000","last_unstale":"2023-11-24T07:58:02.624303+0000","last_undegraded":"2023-11-24T07:58:02.624303+0000","last_fullsized":"2023-11-24T07:58:02.624303+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:58:01.583599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:58:01.583599+0000","last_clean_scrub_stamp":"2023-11-24T07:58:01.583599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T12:08:53.706469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.30","version":"0'0","reported_seq":9,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-11-24T07:58:02.629690+0000","last_change":"2023-11-24T07:58:02.622017+0000","last_active":"2023-11-24T07:58:02.629690+0000","last_peered":"2023-11-24T07:58:02.629690+0000","last_clean":"2023-11-24T07:58:02.629690+0000","last_became_active":"2023-11-24T07:58:02.621703+0000","last_became_peered":"2023-11-24T07:58:02.621703+0000","last_unstale":"2023-11-24T07:58:02.629690+0000","last_undegraded":"2023-11-24T07:58:02.629690+0000","last_fullsized":"2023-11-24T07:58:02.629690+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:58:01.583599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:58:01.583599+0000","last_clean_scrub_stamp":"2023-11-24T07:58:01.583599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T19:15:20.314127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.2f","version":"0'0","reported_seq":9,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-11-24T07:58:02.627097+0000","last_change":"2023-11-24T07:58:02.623672+0000","last_active":"2023-11-24T07:58:02.627097+0000","last_peered":"2023-11-24T07:58:02.627097+0000","last_clean":"2023-11-24T07:58:02.627097+0000","last_became_active":"2023-11-24T07:58:02.623260+0000","last_became_peered":"2023-11-24T07:58:02.623260+0000","last_unstale":"2023-11-24T07:58:02.627097+0000","last_undegraded":"2023-11-24T07:58:02.627097+0000","last_fullsized":"2023-11-24T07:58:02.627097+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:58:01.583599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:58:01.583599+0000","last_clean_scrub_stamp":"2023-11-24T07:58:01.583599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T14:01:26.773202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.2e","version":"0'0","reported_seq":9,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-11-24T07:58:02.627834+0000","last_change":"2023-11-24T07:58:02.623591+0000","last_active":"2023-11-24T07:58:02.627834+0000","last_peered":"2023-11-24T07:58:02.627834+0000","last_clean":"2023-11-24T07:58:02.627834+0000","last_became_active":"2023-11-24T07:58:02.620591+0000","last_became_peered":"2023-11-24T07:58:02.620591+0000","last_unstale":"2023-11-24T07:58:02.627834+0000","last_undegraded":"2023-11-24T07:58:02.627834+0000","last_fullsized":"2023-11-24T07:58:02.627834+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:58:01.583599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:58:01.583599+0000","last_clean_scrub_stamp":"2023-11-24T07:58:01.583599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T09:33:31.355159+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.2d","version":"0'0","reported_seq":9,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-11-24T07:58:02.630725+0000","last_change":"2023-11-24T07:58:02.626896+0000","last_active":"2023-11-24T07:58:02.630725+0000","last_peered":"2023-11-24T07:58:02.630725+0000","last_clean":"2023-11-24T07:58:02.630725+0000","last_became_active":"2023-11-24T07:58:02.626316+0000","last_became_peered":"2023-11-24T07:58:02.626316+0000","last_unstale":"2023-11-24T07:58:02.630725+0000","last_undegraded":"2023-11-24T07:58:02.630725+0000","last_fullsized":"2023-11-24T07:58:02.630725+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:58:01.583599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:58:01.583599+0000","last_clean_scrub_stamp":"2023-11-24T07:58:01.583599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:33:23.934909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.2c","version":"0'0","reported_seq":9,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-11-24T07:58:02.623652+0000","last_change":"2023-11-24T07:58:02.621073+0000","last_active":"2023-11-24T07:58:02.623652+0000","last_peered":"2023-11-24T07:58:02.623652+0000","last_clean":"2023-11-24T07:58:02.623652+0000","last_became_active":"2023-11-24T07:58:02.620377+0000","last_became_peered":"2023-11-24T07:58:02.620377+0000","last_unstale":"2023-11-24T07:58:02.623652+0000","last_undegraded":"2023-11-24T07:58:02.623652+0000","last_fullsized":"2023-11-24T07:58:02.623652+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:58:01.583599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:58:01.583599+0000","last_clean_scrub_stamp":"2023-11-24T07:58:01.583599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T13:56:01.401441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.2b","version":"0'0","reported_seq":9,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-11-24T07:58:02.623552+0000","last_change":"2023-11-24T07:58:02.616504+0000","last_active":"2023-11-24T07:58:02.623552+0000","last_peered":"2023-11-24T07:58:02.623552+0000","last_clean":"2023-11-24T07:58:02.623552+0000","last_became_active":"2023-11-24T07:58:02.616198+0000","last_became_peered":"2023-11-24T07:58:02.616198+0000","last_unstale":"2023-11-24T07:58:02.623552+0000","last_undegraded":"2023-11-24T07:58:02.623552+0000","last_fullsized":"2023-11-24T07:58:02.623552+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:58:01.583599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:58:01.583599+0000","last_clean_scrub_stamp":"2023-11-24T07:58:01.583599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:15:47.666890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.2a","version":"0'0","reported_seq":9,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-11-24T07:58:02.626853+0000","last_change":"2023-11-24T07:58:02.623172+0000","last_active":"2023-11-24T07:58:02.626853+0000","last_peered":"2023-11-24T07:58:02.626853+0000","last_clean":"2023-11-24T07:58:02.626853+0000","last_became_active":"2023-11-24T07:58:02.622843+0000","last_became_peered":"2023-11-24T07:58:02.622843+0000","last_unstale":"2023-11-24T07:58:02.626853+0000","last_undegraded":"2023-11-24T07:58:02.626853+0000","last_fullsized":"2023-11-24T07:58:02.626853+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:58:01.583599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:58:01.583599+0000","last_clean_scrub_stamp":"2023-11-24T07:58:01.583599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T09:33:31.355159+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.29","version":"0'0","reported_seq":9,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-11-24T07:58:02.629086+0000","last_change":"2023-11-24T07:58:02.619184+0000","last_active":"2023-11-24T07:58:02.629086+0000","last_peered":"2023-11-24T07:58:02.629086+0000","last_clean":"2023-11-24T07:58:02.629086+0000","last_became_active":"2023-11-24T07:58:02.616552+0000","last_became_peered":"2023-11-24T07:58:02.616552+0000","last_unstale":"2023-11-24T07:58:02.629086+0000","last_undegraded":"2023-11-24T07:58:02.629086+0000","last_fullsized":"2023-11-24T07:58:02.629086+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:58:01.583599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:58:01.583599+0000","last_clean_scrub_stamp":"2023-11-24T07:58:01.583599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T18:22:49.761771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.28","version":"0'0","reported_seq":9,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-11-24T07:58:02.622139+0000","last_change":"2023-11-24T07:58:02.616088+0000","last_active":"2023-11-24T07:58:02.622139+0000","last_peered":"2023-11-24T07:58:02.622139+0000","last_clean":"2023-11-24T07:58:02.622139+0000","last_became_active":"2023-11-24T07:58:02.615784+0000","last_became_peered":"2023-11-24T07:58:02.615784+0000","last_unstale":"2023-11-24T07:58:02.622139+0000","last_undegraded":"2023-11-24T07:58:02.622139+0000","last_fullsized":"2023-11-24T07:58:02.622139+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:58:01.583599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:58:01.583599+0000","last_clean_scrub_stamp":"2023-11-24T07:58:01.583599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:37:31.802505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.27","version":"0'0","reported_seq":9,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-11-24T07:58:02.625843+0000","last_change":"2023-11-24T07:58:02.619687+0000","last_active":"2023-11-24T07:58:02.625843+0000","last_peered":"2023-11-24T07:58:02.625843+0000","last_clean":"2023-11-24T07:58:02.625843+0000","last_became_active":"2023-11-24T07:58:02.618980+0000","last_became_peered":"2023-11-24T07:58:02.618980+0000","last_unstale":"2023-11-24T07:58:02.625843+0000","last_undegraded":"2023-11-24T07:58:02.625843+0000","last_fullsized":"2023-11-24T07:58:02.625843+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:58:01.583599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:58:01.583599+0000","last_clean_scrub_stamp":"2023-11-24T07:58:01.583599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:15:47.666890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.26","version":"0'0","reported_seq":9,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-11-24T07:58:02.613528+0000","last_change":"2023-11-24T07:58:02.608890+0000","last_active":"2023-11-24T07:58:02.613528+0000","last_peered":"2023-11-24T07:58:02.613528+0000","last_clean":"2023-11-24T07:58:02.613528+0000","last_became_active":"2023-11-24T07:58:02.608591+0000","last_became_peered":"2023-11-24T07:58:02.608591+0000","last_unstale":"2023-11-24T07:58:02.613528+0000","last_undegraded":"2023-11-24T07:58:02.613528+0000","last_fullsized":"2023-11-24T07:58:02.613528+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:58:01.583599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:58:01.583599+0000","last_clean_scrub_stamp":"2023-11-24T07:58:01.583599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T17:09:27.205967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.25","version":"0'0","reported_seq":9,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-11-24T07:58:02.631166+0000","last_change":"2023-11-24T07:58:02.629031+0000","last_active":"2023-11-24T07:58:02.631166+0000","last_peered":"2023-11-24T07:58:02.631166+0000","last_clean":"2023-11-24T07:58:02.631166+0000","last_became_active":"2023-11-24T07:58:02.628329+0000","last_became_peered":"2023-11-24T07:58:02.628329+0000","last_unstale":"2023-11-24T07:58:02.631166+0000","last_undegraded":"2023-11-24T07:58:02.631166+0000","last_fullsized":"2023-11-24T07:58:02.631166+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:58:01.583599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:58:01.583599+0000","last_clean_scrub_stamp":"2023-11-24T07:58:01.583599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T18:09:46.006718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.24","version":"0'0","reported_seq":9,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-11-24T07:58:02.629006+0000","last_change":"2023-11-24T07:58:02.617827+0000","last_active":"2023-11-24T07:58:02.629006+0000","last_peered":"2023-11-24T07:58:02.629006+0000","last_clean":"2023-11-24T07:58:02.629006+0000","last_became_active":"2023-11-24T07:58:02.617508+0000","last_became_peered":"2023-11-24T07:58:02.617508+0000","last_unstale":"2023-11-24T07:58:02.629006+0000","last_undegraded":"2023-11-24T07:58:02.629006+0000","last_fullsized":"2023-11-24T07:58:02.629006+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:58:01.583599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:58:01.583599+0000","last_clean_scrub_stamp":"2023-11-24T07:58:01.583599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T16:17:54.841970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.23","version":"0'0","reported_seq":9,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-11-24T07:58:02.625719+0000","last_change":"2023-11-24T07:58:02.621025+0000","last_active":"2023-11-24T07:58:02.625719+0000","last_peered":"2023-11-24T07:58:02.625719+0000","last_clean":"2023-11-24T07:58:02.625719+0000","last_became_active":"2023-11-24T07:58:02.620751+0000","last_became_peered":"2023-11-24T07:58:02.620751+0000","last_unstale":"2023-11-24T07:58:02.625719+0000","last_undegraded":"2023-11-24T07:58:02.625719+0000","last_fullsized":"2023-11-24T07:58:02.625719+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:58:01.583599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:58:01.583599+0000","last_clean_scrub_stamp":"2023-11-24T07:58:01.583599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T17:09:27.205967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.22","version":"0'0","reported_seq":9,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-11-24T07:58:02.621295+0000","last_change":"2023-11-24T07:58:02.611961+0000","last_active":"2023-11-24T07:58:02.621295+0000","last_peered":"2023-11-24T07:58:02.621295+0000","last_clean":"2023-11-24T07:58:02.621295+0000","last_became_active":"2023-11-24T07:58:02.611684+0000","last_became_peered":"2023-11-24T07:58:02.611684+0000","last_unstale":"2023-11-24T07:58:02.621295+0000","last_undegraded":"2023-11-24T07:58:02.621295+0000","last_fullsized":"2023-11-24T07:58:02.621295+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:58:01.583599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:58:01.583599+0000","last_clean_scrub_stamp":"2023-11-24T07:58:01.583599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T14:21:21.611502+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.21","version":"0'0","reported_seq":9,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-11-24T07:58:02.620762+0000","last_change":"2023-11-24T07:58:02.615770+0000","last_active":"2023-11-24T07:58:02.620762+0000","last_peered":"2023-11-24T07:58:02.620762+0000","last_clean":"2023-11-24T07:58:02.620762+0000","last_became_active":"2023-11-24T07:58:02.615464+0000","last_became_peered":"2023-11-24T07:58:02.615464+0000","last_unstale":"2023-11-24T07:58:02.620762+0000","last_undegraded":"2023-11-24T07:58:02.620762+0000","last_fullsized":"2023-11-24T07:58:02.620762+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:58:01.583599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:58:01.583599+0000","last_clean_scrub_stamp":"2023-11-24T07:58:01.583599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T14:21:21.611502+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.20","version":"0'0","reported_seq":9,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-11-24T07:58:02.613719+0000","last_change":"2023-11-24T07:58:02.611123+0000","last_active":"2023-11-24T07:58:02.613719+0000","last_peered":"2023-11-24T07:58:02.613719+0000","last_clean":"2023-11-24T07:58:02.613719+0000","last_became_active":"2023-11-24T07:58:02.610844+0000","last_became_peered":"2023-11-24T07:58:02.610844+0000","last_unstale":"2023-11-24T07:58:02.613719+0000","last_undegraded":"2023-11-24T07:58:02.613719+0000","last_fullsized":"2023-11-24T07:58:02.613719+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:58:01.583599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:58:01.583599+0000","last_clean_scrub_stamp":"2023-11-24T07:58:01.583599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:37:31.802505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.1e","version":"0'0","reported_seq":9,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-11-24T07:58:02.626648+0000","last_change":"2023-11-24T07:58:02.620653+0000","last_active":"2023-11-24T07:58:02.626648+0000","last_peered":"2023-11-24T07:58:02.626648+0000","last_clean":"2023-11-24T07:58:02.626648+0000","last_became_active":"2023-11-24T07:58:02.620141+0000","last_became_peered":"2023-11-24T07:58:02.620141+0000","last_unstale":"2023-11-24T07:58:02.626648+0000","last_undegraded":"2023-11-24T07:58:02.626648+0000","last_fullsized":"2023-11-24T07:58:02.626648+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:58:01.583599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:58:01.583599+0000","last_clean_scrub_stamp":"2023-11-24T07:58:01.583599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T14:04:40.032987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.1f","version":"44'4","reported_seq":15,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-11-24T07:58:02.632057+0000","last_change":"2023-11-24T07:58:01.611939+0000","last_active":"2023-11-24T07:58:02.632057+0000","last_peered":"2023-11-24T07:58:02.632057+0000","last_clean":"2023-11-24T07:58:02.632057+0000","last_became_active":"2023-11-24T07:58:01.611667+0000","last_became_peered":"2023-11-24T07:58:01.611667+0000","last_unstale":"2023-11-24T07:58:02.632057+0000","last_undegraded":"2023-11-24T07:58:02.632057+0000","last_fullsized":"2023-11-24T07:58:02.632057+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:58:00.578147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:58:00.578147+0000","last_clean_scrub_stamp":"2023-11-24T07:58:00.578147+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T15:36:22.152707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.1f","version":"0'0","reported_seq":9,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-11-24T07:58:02.619780+0000","last_change":"2023-11-24T07:58:02.613596+0000","last_active":"2023-11-24T07:58:02.619780+0000","last_peered":"2023-11-24T07:58:02.619780+0000","last_clean":"2023-11-24T07:58:02.619780+0000","last_became_active":"2023-11-24T07:58:02.613309+0000","last_became_peered":"2023-11-24T07:58:02.613309+0000","last_unstale":"2023-11-24T07:58:02.619780+0000","last_undegraded":"2023-11-24T07:58:02.619780+0000","last_fullsized":"2023-11-24T07:58:02.619780+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:58:01.583599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:58:01.583599+0000","last_clean_scrub_stamp":"2023-11-24T07:58:01.583599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:37:31.802505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1e","version":"44'1","reported_seq":12,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-11-24T07:58:02.628853+0000","last_change":"2023-11-24T07:58:01.610596+0000","last_active":"2023-11-24T07:58:02.628853+0000","last_peered":"2023-11-24T07:58:02.628853+0000","last_clean":"2023-11-24T07:58:02.628853+0000","last_became_active":"2023-11-24T07:58:01.610311+0000","last_became_peered":"2023-11-24T07:58:01.610311+0000","last_unstale":"2023-11-24T07:58:02.628853+0000","last_undegraded":"2023-11-24T07:58:02.628853+0000","last_fullsized":"2023-11-24T07:58:02.628853+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:58:00.578147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:58:00.578147+0000","last_clean_scrub_stamp":"2023-11-24T07:58:00.578147+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T08:58:19.378878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1c","version":"0'0","reported_seq":9,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-11-24T07:58:02.629438+0000","last_change":"2023-11-24T07:58:02.624739+0000","last_active":"2023-11-24T07:58:02.629438+0000","last_peered":"2023-11-24T07:58:02.629438+0000","last_clean":"2023-11-24T07:58:02.629438+0000","last_became_active":"2023-11-24T07:58:02.624458+0000","last_became_peered":"2023-11-24T07:58:02.624458+0000","last_unstale":"2023-11-24T07:58:02.629438+0000","last_undegraded":"2023-11-24T07:58:02.629438+0000","last_fullsized":"2023-11-24T07:58:02.629438+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:58:01.583599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:58:01.583599+0000","last_clean_scrub_stamp":"2023-11-24T07:58:01.583599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T13:42:52.916962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.1d","version":"44'2","reported_seq":13,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-11-24T07:58:02.631053+0000","last_change":"2023-11-24T07:58:01.611179+0000","last_active":"2023-11-24T07:58:02.631053+0000","last_peered":"2023-11-24T07:58:02.631053+0000","last_clean":"2023-11-24T07:58:02.631053+0000","last_became_active":"2023-11-24T07:58:01.610856+0000","last_became_peered":"2023-11-24T07:58:01.610856+0000","last_unstale":"2023-11-24T07:58:02.631053+0000","last_undegraded":"2023-11-24T07:58:02.631053+0000","last_fullsized":"2023-11-24T07:58:02.631053+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:58:00.578147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:58:00.578147+0000","last_clean_scrub_stamp":"2023-11-24T07:58:00.578147+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:15:46.661438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.1d","version":"0'0","reported_seq":9,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-11-24T07:58:02.617365+0000","last_change":"2023-11-24T07:58:02.612602+0000","last_active":"2023-11-24T07:58:02.617365+0000","last_peered":"2023-11-24T07:58:02.617365+0000","last_clean":"2023-11-24T07:58:02.617365+0000","last_became_active":"2023-11-24T07:58:02.612296+0000","last_became_peered":"2023-11-24T07:58:02.612296+0000","last_unstale":"2023-11-24T07:58:02.617365+0000","last_undegraded":"2023-11-24T07:58:02.617365+0000","last_fullsized":"2023-11-24T07:58:02.617365+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:58:01.583599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:58:01.583599+0000","last_clean_scrub_stamp":"2023-11-24T07:58:01.583599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T19:15:20.314127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1c","version":"44'2","reported_seq":13,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-11-24T07:58:02.629970+0000","last_change":"2023-11-24T07:58:01.611406+0000","last_active":"2023-11-24T07:58:02.629970+0000","last_peered":"2023-11-24T07:58:02.629970+0000","last_clean":"2023-11-24T07:58:02.629970+0000","last_became_active":"2023-11-24T07:58:01.611065+0000","last_became_peered":"2023-11-24T07:58:01.611065+0000","last_unstale":"2023-11-24T07:58:02.629970+0000","last_undegraded":"2023-11-24T07:58:02.629970+0000","last_fullsized":"2023-11-24T07:58:02.629970+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:58:00.578147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:58:00.578147+0000","last_clean_scrub_stamp":"2023-11-24T07:58:00.578147+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T12:38:07.521171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1a","version":"0'0","reported_seq":9,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-11-24T07:58:02.631154+0000","last_change":"2023-11-24T07:58:02.628830+0000","last_active":"2023-11-24T07:58:02.631154+0000","last_peered":"2023-11-24T07:58:02.631154+0000","last_clean":"2023-11-24T07:58:02.631154+0000","last_became_active":"2023-11-24T07:58:02.628529+0000","last_became_peered":"2023-11-24T07:58:02.628529+0000","last_unstale":"2023-11-24T07:58:02.631154+0000","last_undegraded":"2023-11-24T07:58:02.631154+0000","last_fullsized":"2023-11-24T07:58:02.631154+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:58:01.583599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:58:01.583599+0000","last_clean_scrub_stamp":"2023-11-24T07:58:01.583599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T08:49:16.225604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.1b","version":"44'1","reported_seq":12,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-11-24T07:58:02.629635+0000","last_change":"2023-11-24T07:58:01.611305+0000","last_active":"2023-11-24T07:58:02.629635+0000","last_peered":"2023-11-24T07:58:02.629635+0000","last_clean":"2023-11-24T07:58:02.629635+0000","last_became_active":"2023-11-24T07:58:01.610994+0000","last_became_peered":"2023-11-24T07:58:01.610994+0000","last_unstale":"2023-11-24T07:58:02.629635+0000","last_undegraded":"2023-11-24T07:58:02.629635+0000","last_fullsized":"2023-11-24T07:58:02.629635+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:58:00.578147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:58:00.578147+0000","last_clean_scrub_stamp":"2023-11-24T07:58:00.578147+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T13:42:51.911510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.1b","version":"0'0","reported_seq":9,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-11-24T07:58:02.623818+0000","last_change":"2023-11-24T07:58:02.620015+0000","last_active":"2023-11-24T07:58:02.623818+0000","last_peered":"2023-11-24T07:58:02.623818+0000","last_clean":"2023-11-24T07:58:02.623818+0000","last_became_active":"2023-11-24T07:58:02.619716+0000","last_became_peered":"2023-11-24T07:58:02.619716+0000","last_unstale":"2023-11-24T07:58:02.623818+0000","last_undegraded":"2023-11-24T07:58:02.623818+0000","last_fullsized":"2023-11-24T07:58:02.623818+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:58:01.583599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:58:01.583599+0000","last_clean_scrub_stamp":"2023-11-24T07:58:01.583599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T11:01:48.657488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1a","version":"44'1","reported_seq":12,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-11-24T07:58:02.629139+0000","last_change":"2023-11-24T07:58:01.609468+0000","last_active":"2023-11-24T07:58:02.629139+0000","last_peered":"2023-11-24T07:58:02.629139+0000","last_clean":"2023-11-24T07:58:02.629139+0000","last_became_active":"2023-11-24T07:58:01.609163+0000","last_became_peered":"2023-11-24T07:58:01.609163+0000","last_unstale":"2023-11-24T07:58:02.629139+0000","last_undegraded":"2023-11-24T07:58:02.629139+0000","last_fullsized":"2023-11-24T07:58:02.629139+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:58:00.578147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:58:00.578147+0000","last_clean_scrub_stamp":"2023-11-24T07:58:00.578147+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T16:30:16.725810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.18","version":"0'0","reported_seq":9,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-11-24T07:58:02.623221+0000","last_change":"2023-11-24T07:58:02.617452+0000","last_active":"2023-11-24T07:58:02.623221+0000","last_peered":"2023-11-24T07:58:02.623221+0000","last_clean":"2023-11-24T07:58:02.623221+0000","last_became_active":"2023-11-24T07:58:02.617158+0000","last_became_peered":"2023-11-24T07:58:02.617158+0000","last_unstale":"2023-11-24T07:58:02.623221+0000","last_undegraded":"2023-11-24T07:58:02.623221+0000","last_fullsized":"2023-11-24T07:58:02.623221+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:58:01.583599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:58:01.583599+0000","last_clean_scrub_stamp":"2023-11-24T07:58:01.583599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T12:50:53.080554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":11,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-11-24T07:58:02.607146+0000","last_change":"2023-11-24T07:58:01.612644+0000","last_active":"2023-11-24T07:58:02.607146+0000","last_peered":"2023-11-24T07:58:02.607146+0000","last_clean":"2023-11-24T07:58:02.607146+0000","last_became_active":"2023-11-24T07:58:01.612301+0000","last_became_peered":"2023-11-24T07:58:01.612301+0000","last_unstale":"2023-11-24T07:58:02.607146+0000","last_undegraded":"2023-11-24T07:58:02.607146+0000","last_fullsized":"2023-11-24T07:58:02.607146+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:58:00.578147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:58:00.578147+0000","last_clean_scrub_stamp":"2023-11-24T07:58:00.578147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T12:08:52.701016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.19","version":"0'0","reported_seq":9,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-11-24T07:58:02.611313+0000","last_change":"2023-11-24T07:58:02.608850+0000","last_active":"2023-11-24T07:58:02.611313+0000","last_peered":"2023-11-24T07:58:02.611313+0000","last_clean":"2023-11-24T07:58:02.611313+0000","last_became_active":"2023-11-24T07:58:02.608563+0000","last_became_peered":"2023-11-24T07:58:02.608563+0000","last_unstale":"2023-11-24T07:58:02.611313+0000","last_undegraded":"2023-11-24T07:58:02.611313+0000","last_fullsized":"2023-11-24T07:58:02.611313+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:58:01.583599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:58:01.583599+0000","last_clean_scrub_stamp":"2023-11-24T07:58:01.583599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T16:49:00.649513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":11,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-11-24T07:58:02.605880+0000","last_change":"2023-11-24T07:58:01.616438+0000","last_active":"2023-11-24T07:58:02.605880+0000","last_peered":"2023-11-24T07:58:02.605880+0000","last_clean":"2023-11-24T07:58:02.605880+0000","last_became_active":"2023-11-24T07:58:01.616159+0000","last_became_peered":"2023-11-24T07:58:01.616159+0000","last_unstale":"2023-11-24T07:58:02.605880+0000","last_undegraded":"2023-11-24T07:58:02.605880+0000","last_fullsized":"2023-11-24T07:58:02.605880+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:58:00.578147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:58:00.578147+0000","last_clean_scrub_stamp":"2023-11-24T07:58:00.578147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T19:44:06.756960+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.16","version":"0'0","reported_seq":9,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-11-24T07:58:02.623685+0000","last_change":"2023-11-24T07:58:02.613182+0000","last_active":"2023-11-24T07:58:02.623685+0000","last_peered":"2023-11-24T07:58:02.623685+0000","last_clean":"2023-11-24T07:58:02.623685+0000","last_became_active":"2023-11-24T07:58:02.612863+0000","last_became_peered":"2023-11-24T07:58:02.612863+0000","last_unstale":"2023-11-24T07:58:02.623685+0000","last_undegraded":"2023-11-24T07:58:02.623685+0000","last_fullsized":"2023-11-24T07:58:02.623685+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:58:01.583599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:58:01.583599+0000","last_clean_scrub_stamp":"2023-11-24T07:58:01.583599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T12:59:32.870352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":11,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-11-24T07:58:02.607285+0000","last_change":"2023-11-24T07:58:01.614068+0000","last_active":"2023-11-24T07:58:02.607285+0000","last_peered":"2023-11-24T07:58:02.607285+0000","last_clean":"2023-11-24T07:58:02.607285+0000","last_became_active":"2023-11-24T07:58:01.613346+0000","last_became_peered":"2023-11-24T07:58:01.613346+0000","last_unstale":"2023-11-24T07:58:02.607285+0000","last_undegraded":"2023-11-24T07:58:02.607285+0000","last_fullsized":"2023-11-24T07:58:02.607285+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:58:00.578147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:58:00.578147+0000","last_clean_scrub_stamp":"2023-11-24T07:58:00.578147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T15:33:14.634070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.17","version":"0'0","reported_seq":9,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-11-24T07:58:02.627730+0000","last_change":"2023-11-24T07:58:02.623316+0000","last_active":"2023-11-24T07:58:02.627730+0000","last_peered":"2023-11-24T07:58:02.627730+0000","last_clean":"2023-11-24T07:58:02.627730+0000","last_became_active":"2023-11-24T07:58:02.623035+0000","last_became_peered":"2023-11-24T07:58:02.623035+0000","last_unstale":"2023-11-24T07:58:02.627730+0000","last_undegraded":"2023-11-24T07:58:02.627730+0000","last_fullsized":"2023-11-24T07:58:02.627730+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:58:01.583599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:58:01.583599+0000","last_clean_scrub_stamp":"2023-11-24T07:58:01.583599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T12:51:50.368636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":11,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-11-24T07:58:02.612463+0000","last_change":"2023-11-24T07:58:01.616836+0000","last_active":"2023-11-24T07:58:02.612463+0000","last_peered":"2023-11-24T07:58:02.612463+0000","last_clean":"2023-11-24T07:58:02.612463+0000","last_became_active":"2023-11-24T07:58:01.616552+0000","last_became_peered":"2023-11-24T07:58:01.616552+0000","last_unstale":"2023-11-24T07:58:02.612463+0000","last_undegraded":"2023-11-24T07:58:02.612463+0000","last_fullsized":"2023-11-24T07:58:02.612463+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:58:00.578147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:58:00.578147+0000","last_clean_scrub_stamp":"2023-11-24T07:58:00.578147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T15:43:05.511574+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.14","version":"0'0","reported_seq":9,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-11-24T07:58:02.630915+0000","last_change":"2023-11-24T07:58:02.624262+0000","last_active":"2023-11-24T07:58:02.630915+0000","last_peered":"2023-11-24T07:58:02.630915+0000","last_clean":"2023-11-24T07:58:02.630915+0000","last_became_active":"2023-11-24T07:58:02.623994+0000","last_became_peered":"2023-11-24T07:58:02.623994+0000","last_unstale":"2023-11-24T07:58:02.630915+0000","last_undegraded":"2023-11-24T07:58:02.630915+0000","last_fullsized":"2023-11-24T07:58:02.630915+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:58:01.583599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:58:01.583599+0000","last_clean_scrub_stamp":"2023-11-24T07:58:01.583599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T13:34:36.854213+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.15","version":"44'3","reported_seq":14,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-11-24T07:58:02.630987+0000","last_change":"2023-11-24T07:58:01.608162+0000","last_active":"2023-11-24T07:58:02.630987+0000","last_peered":"2023-11-24T07:58:02.630987+0000","last_clean":"2023-11-24T07:58:02.630987+0000","last_became_active":"2023-11-24T07:58:01.607860+0000","last_became_peered":"2023-11-24T07:58:01.607860+0000","last_unstale":"2023-11-24T07:58:02.630987+0000","last_undegraded":"2023-11-24T07:58:02.630987+0000","last_fullsized":"2023-11-24T07:58:02.630987+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:58:00.578147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:58:00.578147+0000","last_clean_scrub_stamp":"2023-11-24T07:58:00.578147+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T19:15:19.308675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.15","version":"0'0","reported_seq":9,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-11-24T07:58:02.620370+0000","last_change":"2023-11-24T07:58:02.614576+0000","last_active":"2023-11-24T07:58:02.620370+0000","last_peered":"2023-11-24T07:58:02.620370+0000","last_clean":"2023-11-24T07:58:02.620370+0000","last_became_active":"2023-11-24T07:58:02.614305+0000","last_became_peered":"2023-11-24T07:58:02.614305+0000","last_unstale":"2023-11-24T07:58:02.620370+0000","last_undegraded":"2023-11-24T07:58:02.620370+0000","last_fullsized":"2023-11-24T07:58:02.620370+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:58:01.583599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:58:01.583599+0000","last_clean_scrub_stamp":"2023-11-24T07:58:01.583599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T13:17:31.956955+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.14","version":"44'6","reported_seq":17,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-11-24T07:58:02.633381+0000","last_change":"2023-11-24T07:58:01.612668+0000","last_active":"2023-11-24T07:58:02.633381+0000","last_peered":"2023-11-24T07:58:02.633381+0000","last_clean":"2023-11-24T07:58:02.633381+0000","last_became_active":"2023-11-24T07:58:01.612221+0000","last_became_peered":"2023-11-24T07:58:01.612221+0000","last_unstale":"2023-11-24T07:58:02.633381+0000","last_undegraded":"2023-11-24T07:58:02.633381+0000","last_fullsized":"2023-11-24T07:58:02.633381+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:58:00.578147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:58:00.578147+0000","last_clean_scrub_stamp":"2023-11-24T07:58:00.578147+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T19:40:21.190531+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.12","version":"0'0","reported_seq":9,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-11-24T07:58:02.624400+0000","last_change":"2023-11-24T07:58:02.620890+0000","last_active":"2023-11-24T07:58:02.624400+0000","last_peered":"2023-11-24T07:58:02.624400+0000","last_clean":"2023-11-24T07:58:02.624400+0000","last_became_active":"2023-11-24T07:58:02.620596+0000","last_became_peered":"2023-11-24T07:58:02.620596+0000","last_unstale":"2023-11-24T07:58:02.624400+0000","last_undegraded":"2023-11-24T07:58:02.624400+0000","last_fullsized":"2023-11-24T07:58:02.624400+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:58:01.583599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:58:01.583599+0000","last_clean_scrub_stamp":"2023-11-24T07:58:01.583599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T19:31:11.664630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.13","version":"44'2","reported_seq":13,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-11-24T07:58:02.624947+0000","last_change":"2023-11-24T07:58:01.614050+0000","last_active":"2023-11-24T07:58:02.624947+0000","last_peered":"2023-11-24T07:58:02.624947+0000","last_clean":"2023-11-24T07:58:02.624947+0000","last_became_active":"2023-11-24T07:58:01.613746+0000","last_became_peered":"2023-11-24T07:58:01.613746+0000","last_unstale":"2023-11-24T07:58:02.624947+0000","last_undegraded":"2023-11-24T07:58:02.624947+0000","last_fullsized":"2023-11-24T07:58:02.624947+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:58:00.578147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:58:00.578147+0000","last_clean_scrub_stamp":"2023-11-24T07:58:00.578147+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T19:14:16.804507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.13","version":"0'0","reported_seq":9,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-11-24T07:58:02.626736+0000","last_change":"2023-11-24T07:58:02.622179+0000","last_active":"2023-11-24T07:58:02.626736+0000","last_peered":"2023-11-24T07:58:02.626736+0000","last_clean":"2023-11-24T07:58:02.626736+0000","last_became_active":"2023-11-24T07:58:02.621869+0000","last_became_peered":"2023-11-24T07:58:02.621869+0000","last_unstale":"2023-11-24T07:58:02.626736+0000","last_undegraded":"2023-11-24T07:58:02.626736+0000","last_fullsized":"2023-11-24T07:58:02.626736+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:58:01.583599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:58:01.583599+0000","last_clean_scrub_stamp":"2023-11-24T07:58:01.583599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T15:33:15.639523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":11,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-11-24T07:58:02.606211+0000","last_change":"2023-11-24T07:58:01.612148+0000","last_active":"2023-11-24T07:58:02.606211+0000","last_peered":"2023-11-24T07:58:02.606211+0000","last_clean":"2023-11-24T07:58:02.606211+0000","last_became_active":"2023-11-24T07:58:01.611797+0000","last_became_peered":"2023-11-24T07:58:01.611797+0000","last_unstale":"2023-11-24T07:58:02.606211+0000","last_undegraded":"2023-11-24T07:58:02.606211+0000","last_fullsized":"2023-11-24T07:58:02.606211+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:58:00.578147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:58:00.578147+0000","last_clean_scrub_stamp":"2023-11-24T07:58:00.578147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T15:39:02.083805+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.10","version":"0'0","reported_seq":9,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-11-24T07:58:02.626842+0000","last_change":"2023-11-24T07:58:02.622907+0000","last_active":"2023-11-24T07:58:02.626842+0000","last_peered":"2023-11-24T07:58:02.626842+0000","last_clean":"2023-11-24T07:58:02.626842+0000","last_became_active":"2023-11-24T07:58:02.622583+0000","last_became_peered":"2023-11-24T07:58:02.622583+0000","last_unstale":"2023-11-24T07:58:02.626842+0000","last_undegraded":"2023-11-24T07:58:02.626842+0000","last_fullsized":"2023-11-24T07:58:02.626842+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:58:01.583599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:58:01.583599+0000","last_clean_scrub_stamp":"2023-11-24T07:58:01.583599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:18:58.981383+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":11,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-11-24T07:58:02.599996+0000","last_change":"2023-11-24T07:58:01.609614+0000","last_active":"2023-11-24T07:58:02.599996+0000","last_peered":"2023-11-24T07:58:02.599996+0000","last_clean":"2023-11-24T07:58:02.599996+0000","last_became_active":"2023-11-24T07:58:01.609377+0000","last_became_peered":"2023-11-24T07:58:01.609377+0000","last_unstale":"2023-11-24T07:58:02.599996+0000","last_undegraded":"2023-11-24T07:58:02.599996+0000","last_fullsized":"2023-11-24T07:58:02.599996+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:58:00.578147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:58:00.578147+0000","last_clean_scrub_stamp":"2023-11-24T07:58:00.578147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:33:31.200726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.11","version":"0'0","reported_seq":9,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-11-24T07:58:02.637977+0000","last_change":"2023-11-24T07:58:02.633597+0000","last_active":"2023-11-24T07:58:02.637977+0000","last_peered":"2023-11-24T07:58:02.637977+0000","last_clean":"2023-11-24T07:58:02.637977+0000","last_became_active":"2023-11-24T07:58:02.633290+0000","last_became_peered":"2023-11-24T07:58:02.633290+0000","last_unstale":"2023-11-24T07:58:02.637977+0000","last_undegraded":"2023-11-24T07:58:02.637977+0000","last_fullsized":"2023-11-24T07:58:02.637977+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:58:01.583599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:58:01.583599+0000","last_clean_scrub_stamp":"2023-11-24T07:58:01.583599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:15:47.666890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.10","version":"44'8","reported_seq":19,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-11-24T07:58:02.627271+0000","last_change":"2023-11-24T07:58:01.610705+0000","last_active":"2023-11-24T07:58:02.627271+0000","last_peered":"2023-11-24T07:58:02.627271+0000","last_clean":"2023-11-24T07:58:02.627271+0000","last_became_active":"2023-11-24T07:58:01.610408+0000","last_became_peered":"2023-11-24T07:58:01.610408+0000","last_unstale":"2023-11-24T07:58:02.627271+0000","last_undegraded":"2023-11-24T07:58:02.627271+0000","last_fullsized":"2023-11-24T07:58:02.627271+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:58:00.578147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:58:00.578147+0000","last_clean_scrub_stamp":"2023-11-24T07:58:00.578147+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T16:22:38.421798+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.e","version":"0'0","reported_seq":9,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-11-24T07:58:02.620754+0000","last_change":"2023-11-24T07:58:02.614815+0000","last_active":"2023-11-24T07:58:02.620754+0000","last_peered":"2023-11-24T07:58:02.620754+0000","last_clean":"2023-11-24T07:58:02.620754+0000","last_became_active":"2023-11-24T07:58:02.614519+0000","last_became_peered":"2023-11-24T07:58:02.614519+0000","last_unstale":"2023-11-24T07:58:02.620754+0000","last_undegraded":"2023-11-24T07:58:02.620754+0000","last_fullsized":"2023-11-24T07:58:02.620754+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:58:01.583599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:58:01.583599+0000","last_clean_scrub_stamp":"2023-11-24T07:58:01.583599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:35:34.232935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.f","version":"44'1","reported_seq":12,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-11-24T07:58:02.623851+0000","last_change":"2023-11-24T07:58:01.613289+0000","last_active":"2023-11-24T07:58:02.623851+0000","last_peered":"2023-11-24T07:58:02.623851+0000","last_clean":"2023-11-24T07:58:02.623851+0000","last_became_active":"2023-11-24T07:58:01.612961+0000","last_became_peered":"2023-11-24T07:58:01.612961+0000","last_unstale":"2023-11-24T07:58:02.623851+0000","last_undegraded":"2023-11-24T07:58:02.623851+0000","last_fullsized":"2023-11-24T07:58:02.623851+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:58:00.578147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:58:00.578147+0000","last_clean_scrub_stamp":"2023-11-24T07:58:00.578147+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T13:42:51.911510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.f","version":"0'0","reported_seq":9,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-11-24T07:58:02.626620+0000","last_change":"2023-11-24T07:58:02.620464+0000","last_active":"2023-11-24T07:58:02.626620+0000","last_peered":"2023-11-24T07:58:02.626620+0000","last_clean":"2023-11-24T07:58:02.626620+0000","last_became_active":"2023-11-24T07:58:02.620187+0000","last_became_peered":"2023-11-24T07:58:02.620187+0000","last_unstale":"2023-11-24T07:58:02.626620+0000","last_undegraded":"2023-11-24T07:58:02.626620+0000","last_fullsized":"2023-11-24T07:58:02.626620+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:58:01.583599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:58:01.583599+0000","last_clean_scrub_stamp":"2023-11-24T07:58:01.583599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T15:47:49.136575+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.e","version":"44'2","reported_seq":13,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-11-24T07:58:02.630782+0000","last_change":"2023-11-24T07:58:01.615918+0000","last_active":"2023-11-24T07:58:02.630782+0000","last_peered":"2023-11-24T07:58:02.630782+0000","last_clean":"2023-11-24T07:58:02.630782+0000","last_became_active":"2023-11-24T07:58:01.615196+0000","last_became_peered":"2023-11-24T07:58:01.615196+0000","last_unstale":"2023-11-24T07:58:02.630782+0000","last_undegraded":"2023-11-24T07:58:02.630782+0000","last_fullsized":"2023-11-24T07:58:02.630782+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:58:00.578147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:58:00.578147+0000","last_clean_scrub_stamp":"2023-11-24T07:58:00.578147+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T15:47:48.131123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.c","version":"0'0","reported_seq":9,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-11-24T07:58:02.624213+0000","last_change":"2023-11-24T07:58:02.616927+0000","last_active":"2023-11-24T07:58:02.624213+0000","last_peered":"2023-11-24T07:58:02.624213+0000","last_clean":"2023-11-24T07:58:02.624213+0000","last_became_active":"2023-11-24T07:58:02.615155+0000","last_became_peered":"2023-11-24T07:58:02.615155+0000","last_unstale":"2023-11-24T07:58:02.624213+0000","last_undegraded":"2023-11-24T07:58:02.624213+0000","last_fullsized":"2023-11-24T07:58:02.624213+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:58:01.583599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:58:01.583599+0000","last_clean_scrub_stamp":"2023-11-24T07:58:01.583599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:33:32.206178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.d","version":"44'1","reported_seq":12,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-11-24T07:58:02.631456+0000","last_change":"2023-11-24T07:58:01.612501+0000","last_active":"2023-11-24T07:58:02.631456+0000","last_peered":"2023-11-24T07:58:02.631456+0000","last_clean":"2023-11-24T07:58:02.631456+0000","last_became_active":"2023-11-24T07:58:01.612212+0000","last_became_peered":"2023-11-24T07:58:01.612212+0000","last_unstale":"2023-11-24T07:58:02.631456+0000","last_undegraded":"2023-11-24T07:58:02.631456+0000","last_fullsized":"2023-11-24T07:58:02.631456+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:58:00.578147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:58:00.578147+0000","last_clean_scrub_stamp":"2023-11-24T07:58:00.578147+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T13:39:52.527067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.d","version":"0'0","reported_seq":9,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-11-24T07:58:02.627645+0000","last_change":"2023-11-24T07:58:02.622388+0000","last_active":"2023-11-24T07:58:02.627645+0000","last_peered":"2023-11-24T07:58:02.627645+0000","last_clean":"2023-11-24T07:58:02.627645+0000","last_became_active":"2023-11-24T07:58:02.622040+0000","last_became_peered":"2023-11-24T07:58:02.622040+0000","last_unstale":"2023-11-24T07:58:02.627645+0000","last_undegraded":"2023-11-24T07:58:02.627645+0000","last_fullsized":"2023-11-24T07:58:02.627645+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:58:01.583599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:58:01.583599+0000","last_clean_scrub_stamp":"2023-11-24T07:58:01.583599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T14:04:40.032987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":11,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-11-24T07:58:02.599718+0000","last_change":"2023-11-24T07:58:01.614016+0000","last_active":"2023-11-24T07:58:02.599718+0000","last_peered":"2023-11-24T07:58:02.599718+0000","last_clean":"2023-11-24T07:58:02.599718+0000","last_became_active":"2023-11-24T07:58:01.613757+0000","last_became_peered":"2023-11-24T07:58:01.613757+0000","last_unstale":"2023-11-24T07:58:02.599718+0000","last_undegraded":"2023-11-24T07:58:02.599718+0000","last_fullsized":"2023-11-24T07:58:02.599718+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:58:00.578147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:58:00.578147+0000","last_clean_scrub_stamp":"2023-11-24T07:58:00.578147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T11:44:15.085353+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.a","version":"0'0","reported_seq":9,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-11-24T07:58:02.623272+0000","last_change":"2023-11-24T07:58:02.617926+0000","last_active":"2023-11-24T07:58:02.623272+0000","last_peered":"2023-11-24T07:58:02.623272+0000","last_clean":"2023-11-24T07:58:02.623272+0000","last_became_active":"2023-11-24T07:58:02.616846+0000","last_became_peered":"2023-11-24T07:58:02.616846+0000","last_unstale":"2023-11-24T07:58:02.623272+0000","last_undegraded":"2023-11-24T07:58:02.623272+0000","last_fullsized":"2023-11-24T07:58:02.623272+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:58:01.583599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:58:01.583599+0000","last_clean_scrub_stamp":"2023-11-24T07:58:01.583599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T18:57:31.529962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.b","version":"44'2","reported_seq":13,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-11-24T07:58:02.623730+0000","last_change":"2023-11-24T07:58:01.613024+0000","last_active":"2023-11-24T07:58:02.623730+0000","last_peered":"2023-11-24T07:58:02.623730+0000","last_clean":"2023-11-24T07:58:02.623730+0000","last_became_active":"2023-11-24T07:58:01.612716+0000","last_became_peered":"2023-11-24T07:58:01.612716+0000","last_unstale":"2023-11-24T07:58:02.623730+0000","last_undegraded":"2023-11-24T07:58:02.623730+0000","last_fullsized":"2023-11-24T07:58:02.623730+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:58:00.578147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:58:00.578147+0000","last_clean_scrub_stamp":"2023-11-24T07:58:00.578147+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T18:51:00.570901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.b","version":"0'0","reported_seq":9,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-11-24T07:58:02.624565+0000","last_change":"2023-11-24T07:58:02.621881+0000","last_active":"2023-11-24T07:58:02.624565+0000","last_peered":"2023-11-24T07:58:02.624565+0000","last_clean":"2023-11-24T07:58:02.624565+0000","last_became_active":"2023-11-24T07:58:02.621608+0000","last_became_peered":"2023-11-24T07:58:02.621608+0000","last_unstale":"2023-11-24T07:58:02.624565+0000","last_undegraded":"2023-11-24T07:58:02.624565+0000","last_fullsized":"2023-11-24T07:58:02.624565+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:58:01.583599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:58:01.583599+0000","last_clean_scrub_stamp":"2023-11-24T07:58:01.583599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T17:09:27.205967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":11,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-11-24T07:58:02.606570+0000","last_change":"2023-11-24T07:58:01.612723+0000","last_active":"2023-11-24T07:58:02.606570+0000","last_peered":"2023-11-24T07:58:02.606570+0000","last_clean":"2023-11-24T07:58:02.606570+0000","last_became_active":"2023-11-24T07:58:01.612426+0000","last_became_peered":"2023-11-24T07:58:01.612426+0000","last_unstale":"2023-11-24T07:58:02.606570+0000","last_undegraded":"2023-11-24T07:58:02.606570+0000","last_fullsized":"2023-11-24T07:58:02.606570+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:58:00.578147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:58:00.578147+0000","last_clean_scrub_stamp":"2023-11-24T07:58:00.578147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T13:34:35.848761+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.8","version":"0'0","reported_seq":9,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-11-24T07:58:02.624739+0000","last_change":"2023-11-24T07:58:02.622212+0000","last_active":"2023-11-24T07:58:02.624739+0000","last_peered":"2023-11-24T07:58:02.624739+0000","last_clean":"2023-11-24T07:58:02.624739+0000","last_became_active":"2023-11-24T07:58:02.620808+0000","last_became_peered":"2023-11-24T07:58:02.620808+0000","last_unstale":"2023-11-24T07:58:02.624739+0000","last_undegraded":"2023-11-24T07:58:02.624739+0000","last_fullsized":"2023-11-24T07:58:02.624739+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:58:01.583599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:58:01.583599+0000","last_clean_scrub_stamp":"2023-11-24T07:58:01.583599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T08:32:51.874274+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":11,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-11-24T07:58:02.606256+0000","last_change":"2023-11-24T07:58:01.613942+0000","last_active":"2023-11-24T07:58:02.606256+0000","last_peered":"2023-11-24T07:58:02.606256+0000","last_clean":"2023-11-24T07:58:02.606256+0000","last_became_active":"2023-11-24T07:58:01.613658+0000","last_became_peered":"2023-11-24T07:58:01.613658+0000","last_unstale":"2023-11-24T07:58:02.606256+0000","last_undegraded":"2023-11-24T07:58:02.606256+0000","last_fullsized":"2023-11-24T07:58:02.606256+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:58:00.578147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:58:00.578147+0000","last_clean_scrub_stamp":"2023-11-24T07:58:00.578147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:05:04.154724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.9","version":"0'0","reported_seq":9,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-11-24T07:58:02.623717+0000","last_change":"2023-11-24T07:58:02.618190+0000","last_active":"2023-11-24T07:58:02.623717+0000","last_peered":"2023-11-24T07:58:02.623717+0000","last_clean":"2023-11-24T07:58:02.623717+0000","last_became_active":"2023-11-24T07:58:02.617899+0000","last_became_peered":"2023-11-24T07:58:02.617899+0000","last_unstale":"2023-11-24T07:58:02.623717+0000","last_undegraded":"2023-11-24T07:58:02.623717+0000","last_fullsized":"2023-11-24T07:58:02.623717+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:58:01.583599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:58:01.583599+0000","last_clean_scrub_stamp":"2023-11-24T07:58:01.583599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:33:23.934909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":11,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-11-24T07:58:02.599487+0000","last_change":"2023-11-24T07:58:01.615893+0000","last_active":"2023-11-24T07:58:02.599487+0000","last_peered":"2023-11-24T07:58:02.599487+0000","last_clean":"2023-11-24T07:58:02.599487+0000","last_became_active":"2023-11-24T07:58:01.615604+0000","last_became_peered":"2023-11-24T07:58:01.615604+0000","last_unstale":"2023-11-24T07:58:02.599487+0000","last_undegraded":"2023-11-24T07:58:02.599487+0000","last_fullsized":"2023-11-24T07:58:02.599487+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:58:00.578147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:58:00.578147+0000","last_clean_scrub_stamp":"2023-11-24T07:58:00.578147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:51:57.585934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.7","version":"0'0","reported_seq":9,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-11-24T07:58:02.610851+0000","last_change":"2023-11-24T07:58:02.607936+0000","last_active":"2023-11-24T07:58:02.610851+0000","last_peered":"2023-11-24T07:58:02.610851+0000","last_clean":"2023-11-24T07:58:02.610851+0000","last_became_active":"2023-11-24T07:58:02.607648+0000","last_became_peered":"2023-11-24T07:58:02.607648+0000","last_unstale":"2023-11-24T07:58:02.610851+0000","last_undegraded":"2023-11-24T07:58:02.610851+0000","last_fullsized":"2023-11-24T07:58:02.610851+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:58:01.583599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:58:01.583599+0000","last_clean_scrub_stamp":"2023-11-24T07:58:01.583599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T16:56:20.684081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":11,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-11-24T07:58:02.601145+0000","last_change":"2023-11-24T07:58:01.615430+0000","last_active":"2023-11-24T07:58:02.601145+0000","last_peered":"2023-11-24T07:58:02.601145+0000","last_clean":"2023-11-24T07:58:02.601145+0000","last_became_active":"2023-11-24T07:58:01.615077+0000","last_became_peered":"2023-11-24T07:58:01.615077+0000","last_unstale":"2023-11-24T07:58:02.601145+0000","last_undegraded":"2023-11-24T07:58:02.601145+0000","last_fullsized":"2023-11-24T07:58:02.601145+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:58:00.578147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:58:00.578147+0000","last_clean_scrub_stamp":"2023-11-24T07:58:00.578147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T15:39:02.083805+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.4","version":"0'0","reported_seq":9,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-11-24T07:58:02.631919+0000","last_change":"2023-11-24T07:58:02.628793+0000","last_active":"2023-11-24T07:58:02.631919+0000","last_peered":"2023-11-24T07:58:02.631919+0000","last_clean":"2023-11-24T07:58:02.631919+0000","last_became_active":"2023-11-24T07:58:02.628501+0000","last_became_peered":"2023-11-24T07:58:02.628501+0000","last_unstale":"2023-11-24T07:58:02.631919+0000","last_undegraded":"2023-11-24T07:58:02.631919+0000","last_fullsized":"2023-11-24T07:58:02.631919+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:58:01.583599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:58:01.583599+0000","last_clean_scrub_stamp":"2023-11-24T07:58:01.583599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:04:31.376151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":11,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-11-24T07:58:02.606575+0000","last_change":"2023-11-24T07:58:01.614161+0000","last_active":"2023-11-24T07:58:02.606575+0000","last_peered":"2023-11-24T07:58:02.606575+0000","last_clean":"2023-11-24T07:58:02.606575+0000","last_became_active":"2023-11-24T07:58:01.613866+0000","last_became_peered":"2023-11-24T07:58:01.613866+0000","last_unstale":"2023-11-24T07:58:02.606575+0000","last_undegraded":"2023-11-24T07:58:02.606575+0000","last_fullsized":"2023-11-24T07:58:02.606575+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:58:00.578147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:58:00.578147+0000","last_clean_scrub_stamp":"2023-11-24T07:58:00.578147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T08:32:50.868822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.5","version":"0'0","reported_seq":9,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-11-24T07:58:02.619899+0000","last_change":"2023-11-24T07:58:02.612665+0000","last_active":"2023-11-24T07:58:02.619899+0000","last_peered":"2023-11-24T07:58:02.619899+0000","last_clean":"2023-11-24T07:58:02.619899+0000","last_became_active":"2023-11-24T07:58:02.612194+0000","last_became_peered":"2023-11-24T07:58:02.612194+0000","last_unstale":"2023-11-24T07:58:02.619899+0000","last_undegraded":"2023-11-24T07:58:02.619899+0000","last_fullsized":"2023-11-24T07:58:02.619899+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:58:01.583599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:58:01.583599+0000","last_clean_scrub_stamp":"2023-11-24T07:58:01.583599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T17:00:36.111442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":11,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-11-24T07:58:02.606398+0000","last_change":"2023-11-24T07:58:01.613918+0000","last_active":"2023-11-24T07:58:02.606398+0000","last_peered":"2023-11-24T07:58:02.606398+0000","last_clean":"2023-11-24T07:58:02.606398+0000","last_became_active":"2023-11-24T07:58:01.613574+0000","last_became_peered":"2023-11-24T07:58:01.613574+0000","last_unstale":"2023-11-24T07:58:02.606398+0000","last_undegraded":"2023-11-24T07:58:02.606398+0000","last_fullsized":"2023-11-24T07:58:02.606398+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:58:00.578147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:58:00.578147+0000","last_clean_scrub_stamp":"2023-11-24T07:58:00.578147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T12:51:49.363183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.2","version":"0'0","reported_seq":9,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-11-24T07:58:02.622649+0000","last_change":"2023-11-24T07:58:02.617105+0000","last_active":"2023-11-24T07:58:02.622649+0000","last_peered":"2023-11-24T07:58:02.622649+0000","last_clean":"2023-11-24T07:58:02.622649+0000","last_became_active":"2023-11-24T07:58:02.616811+0000","last_became_peered":"2023-11-24T07:58:02.616811+0000","last_unstale":"2023-11-24T07:58:02.622649+0000","last_undegraded":"2023-11-24T07:58:02.622649+0000","last_fullsized":"2023-11-24T07:58:02.622649+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:58:01.583599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:58:01.583599+0000","last_clean_scrub_stamp":"2023-11-24T07:58:01.583599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T08:32:51.874274+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":11,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-11-24T07:58:02.607514+0000","last_change":"2023-11-24T07:58:01.615697+0000","last_active":"2023-11-24T07:58:02.607514+0000","last_peered":"2023-11-24T07:58:02.607514+0000","last_clean":"2023-11-24T07:58:02.607514+0000","last_became_active":"2023-11-24T07:58:01.615335+0000","last_became_peered":"2023-11-24T07:58:01.615335+0000","last_unstale":"2023-11-24T07:58:02.607514+0000","last_undegraded":"2023-11-24T07:58:02.607514+0000","last_fullsized":"2023-11-24T07:58:02.607514+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:58:00.578147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:58:00.578147+0000","last_clean_scrub_stamp":"2023-11-24T07:58:00.578147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T12:50:52.075101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.3","version":"0'0","reported_seq":9,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-11-24T07:58:02.623529+0000","last_change":"2023-11-24T07:58:02.616483+0000","last_active":"2023-11-24T07:58:02.623529+0000","last_peered":"2023-11-24T07:58:02.623529+0000","last_clean":"2023-11-24T07:58:02.623529+0000","last_became_active":"2023-11-24T07:58:02.616088+0000","last_became_peered":"2023-11-24T07:58:02.616088+0000","last_unstale":"2023-11-24T07:58:02.623529+0000","last_undegraded":"2023-11-24T07:58:02.623529+0000","last_fullsized":"2023-11-24T07:58:02.623529+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:58:01.583599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:58:01.583599+0000","last_clean_scrub_stamp":"2023-11-24T07:58:01.583599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T14:02:23.676273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":11,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-11-24T07:58:02.600594+0000","last_change":"2023-11-24T07:58:01.612303+0000","last_active":"2023-11-24T07:58:02.600594+0000","last_peered":"2023-11-24T07:58:02.600594+0000","last_clean":"2023-11-24T07:58:02.600594+0000","last_became_active":"2023-11-24T07:58:01.612020+0000","last_became_peered":"2023-11-24T07:58:01.612020+0000","last_unstale":"2023-11-24T07:58:02.600594+0000","last_undegraded":"2023-11-24T07:58:02.600594+0000","last_fullsized":"2023-11-24T07:58:02.600594+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:58:00.578147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:58:00.578147+0000","last_clean_scrub_stamp":"2023-11-24T07:58:00.578147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T18:54:06.222117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.0","version":"0'0","reported_seq":9,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-11-24T07:58:02.624371+0000","last_change":"2023-11-24T07:58:02.621445+0000","last_active":"2023-11-24T07:58:02.624371+0000","last_peered":"2023-11-24T07:58:02.624371+0000","last_clean":"2023-11-24T07:58:02.624371+0000","last_became_active":"2023-11-24T07:58:02.621111+0000","last_became_peered":"2023-11-24T07:58:02.621111+0000","last_unstale":"2023-11-24T07:58:02.624371+0000","last_undegraded":"2023-11-24T07:58:02.624371+0000","last_fullsized":"2023-11-24T07:58:02.624371+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:58:01.583599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:58:01.583599+0000","last_clean_scrub_stamp":"2023-11-24T07:58:01.583599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T19:14:17.809960+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":11,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-11-24T07:58:02.599844+0000","last_change":"2023-11-24T07:58:01.612062+0000","last_active":"2023-11-24T07:58:02.599844+0000","last_peered":"2023-11-24T07:58:02.599844+0000","last_clean":"2023-11-24T07:58:02.599844+0000","last_became_active":"2023-11-24T07:58:01.611754+0000","last_became_peered":"2023-11-24T07:58:01.611754+0000","last_unstale":"2023-11-24T07:58:02.599844+0000","last_undegraded":"2023-11-24T07:58:02.599844+0000","last_fullsized":"2023-11-24T07:58:02.599844+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:58:00.578147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:58:00.578147+0000","last_clean_scrub_stamp":"2023-11-24T07:58:00.578147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T15:27:34.075869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.1","version":"0'0","reported_seq":9,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-11-24T07:58:02.622921+0000","last_change":"2023-11-24T07:58:02.615550+0000","last_active":"2023-11-24T07:58:02.622921+0000","last_peered":"2023-11-24T07:58:02.622921+0000","last_clean":"2023-11-24T07:58:02.622921+0000","last_became_active":"2023-11-24T07:58:02.615098+0000","last_became_peered":"2023-11-24T07:58:02.615098+0000","last_unstale":"2023-11-24T07:58:02.622921+0000","last_undegraded":"2023-11-24T07:58:02.622921+0000","last_fullsized":"2023-11-24T07:58:02.622921+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:58:01.583599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:58:01.583599+0000","last_clean_scrub_stamp":"2023-11-24T07:58:01.583599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T17:09:27.205967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":11,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-11-24T07:58:02.607581+0000","last_change":"2023-11-24T07:58:01.609433+0000","last_active":"2023-11-24T07:58:02.607581+0000","last_peered":"2023-11-24T07:58:02.607581+0000","last_clean":"2023-11-24T07:58:02.607581+0000","last_became_active":"2023-11-24T07:58:01.609189+0000","last_became_peered":"2023-11-24T07:58:01.609189+0000","last_unstale":"2023-11-24T07:58:02.607581+0000","last_undegraded":"2023-11-24T07:58:02.607581+0000","last_fullsized":"2023-11-24T07:58:02.607581+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:58:00.578147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:58:00.578147+0000","last_clean_scrub_stamp":"2023-11-24T07:58:00.578147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T13:39:52.527067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.6","version":"0'0","reported_seq":9,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-11-24T07:58:02.624714+0000","last_change":"2023-11-24T07:58:02.622461+0000","last_active":"2023-11-24T07:58:02.624714+0000","last_peered":"2023-11-24T07:58:02.624714+0000","last_clean":"2023-11-24T07:58:02.624714+0000","last_became_active":"2023-11-24T07:58:02.621899+0000","last_became_peered":"2023-11-24T07:58:02.621899+0000","last_unstale":"2023-11-24T07:58:02.624714+0000","last_undegraded":"2023-11-24T07:58:02.624714+0000","last_fullsized":"2023-11-24T07:58:02.624714+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:58:01.583599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:58:01.583599+0000","last_clean_scrub_stamp":"2023-11-24T07:58:01.583599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:33:32.206178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.7","version":"44'2","reported_seq":13,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-11-24T07:58:02.630044+0000","last_change":"2023-11-24T07:58:01.616492+0000","last_active":"2023-11-24T07:58:02.630044+0000","last_peered":"2023-11-24T07:58:02.630044+0000","last_clean":"2023-11-24T07:58:02.630044+0000","last_became_active":"2023-11-24T07:58:01.616200+0000","last_became_peered":"2023-11-24T07:58:01.616200+0000","last_unstale":"2023-11-24T07:58:02.630044+0000","last_undegraded":"2023-11-24T07:58:02.630044+0000","last_fullsized":"2023-11-24T07:58:02.630044+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:58:00.578147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:58:00.578147+0000","last_clean_scrub_stamp":"2023-11-24T07:58:00.578147+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:02:45.169861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":184,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-11-24T07:58:01.607889+0000","last_change":"2023-11-24T07:56:19.668905+0000","last_active":"2023-11-24T07:58:01.607889+0000","last_peered":"2023-11-24T07:58:01.607889+0000","last_clean":"2023-11-24T07:58:01.607889+0000","last_became_active":"2023-11-24T07:56:19.668601+0000","last_became_peered":"2023-11-24T07:56:19.668601+0000","last_unstale":"2023-11-24T07:58:01.607889+0000","last_undegraded":"2023-11-24T07:58:01.607889+0000","last_fullsized":"2023-11-24T07:58:01.607889+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:56:18.657844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:56:18.657844+0000","last_clean_scrub_stamp":"2023-11-24T07:56:18.657844+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T11:51:53.828455+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":7,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574872,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27536,"kb_used_data":508,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344304,"statfs":{"total":96636764160,"available":96608567296,"internally_reserved":0,"allocated":520192,"data_stored":208347,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 07:57:19 2023","interfaces":[{"interface":"back","average":{"1min":0.55700000000000005,"5min":0.55700000000000005,"15min":0.55700000000000005},"min":{"1min":0.32800000000000001,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":1.413,"5min":1.413,"15min":1.413},"last":0.65500000000000003},{"interface":"front","average":{"1min":0.55400000000000005,"5min":0.55400000000000005,"15min":0.55400000000000005},"min":{"1min":0.41099999999999998,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":0.81000000000000005,"5min":0.81000000000000005,"15min":0.81000000000000005},"last":0.626}]},{"osd":1,"last update":"Fri Nov 24 07:57:19 2023","interfaces":[{"interface":"back","average":{"1min":0.84899999999999998,"5min":0.84899999999999998,"15min":0.84899999999999998},"min":{"1min":0.33600000000000002,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":6.1070000000000002,"5min":6.1070000000000002,"15min":6.1070000000000002},"last":0.47599999999999998},{"interface":"front","average":{"1min":0.57999999999999996,"5min":0.57999999999999996,"15min":0.57999999999999996},"min":{"1min":0.35199999999999998,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":1.5920000000000001,"5min":1.5920000000000001,"15min":1.5920000000000001},"last":0.43099999999999999}]},{"osd":2,"last update":"Fri Nov 24 07:57:19 2023","interfaces":[{"interface":"back","average":{"1min":0.627,"5min":0.627,"15min":0.627},"min":{"1min":0.33800000000000002,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":1.613,"5min":1.613,"15min":1.613},"last":0.60499999999999998},{"interface":"front","average":{"1min":0.79200000000000004,"5min":0.79200000000000004,"15min":0.79200000000000004},"min":{"1min":0.41699999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":4.5220000000000002,"5min":4.5220000000000002,"15min":4.5220000000000002},"last":0.45800000000000002}]},{"osd":3,"last update":"Fri Nov 24 07:57:19 2023","interfaces":[{"interface":"back","average":{"1min":0.80600000000000005,"5min":0.80600000000000005,"15min":0.80600000000000005},"min":{"1min":0.38500000000000001,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":4.6760000000000002,"5min":4.6760000000000002,"15min":4.6760000000000002},"last":0.53800000000000003},{"interface":"front","average":{"1min":0.94299999999999995,"5min":0.94299999999999995,"15min":0.94299999999999995},"min":{"1min":0.505,"5min":0.505,"15min":0.505},"max":{"1min":4.6539999999999999,"5min":4.6539999999999999,"15min":4.6539999999999999},"last":0.68899999999999995}]},{"osd":4,"last update":"Fri Nov 24 07:57:19 2023","interfaces":[{"interface":"back","average":{"1min":0.56799999999999995,"5min":0.56799999999999995,"15min":0.56799999999999995},"min":{"1min":0.378,"5min":0.378,"15min":0.378},"max":{"1min":0.84699999999999998,"5min":0.84699999999999998,"15min":0.84699999999999998},"last":0.58399999999999996},{"interface":"front","average":{"1min":0.66700000000000004,"5min":0.66700000000000004,"15min":0.66700000000000004},"min":{"1min":0.46100000000000002,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":1.629,"5min":1.629,"15min":1.629},"last":0.52200000000000002}]},{"osd":5,"last update":"Fri Nov 24 07:57:19 2023","interfaces":[{"interface":"back","average":{"1min":0.54700000000000004,"5min":0.54700000000000004,"15min":0.54700000000000004},"min":{"1min":0.34499999999999997,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.875,"5min":0.875,"15min":0.875},"last":0.50600000000000001},{"interface":"front","average":{"1min":0.623,"5min":0.623,"15min":0.623},"min":{"1min":0.41499999999999998,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":0.90900000000000003,"5min":0.90900000000000003,"15min":0.90900000000000003},"last":0.67400000000000004}]},{"osd":6,"last update":"Fri Nov 24 07:57:19 2023","interfaces":[{"interface":"back","average":{"1min":0.61099999999999999,"5min":0.61099999999999999,"15min":0.61099999999999999},"min":{"1min":0.438,"5min":0.438,"15min":0.438},"max":{"1min":0.80800000000000005,"5min":0.80800000000000005,"15min":0.80800000000000005},"last":0.55200000000000005},{"interface":"front","average":{"1min":0.67000000000000004,"5min":0.67000000000000004,"15min":0.67000000000000004},"min":{"1min":0.48599999999999999,"5min":0.48599999999999999,"15min":0.48599999999999999},"max":{"1min":0.98399999999999999,"5min":0.98399999999999999,"15min":0.98399999999999999},"last":0.63700000000000001}]}]},{"osd":6,"up_from":13,"seq":55834574873,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27540,"kb_used_data":512,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344300,"statfs":{"total":96636764160,"available":96608563200,"internally_reserved":0,"allocated":524288,"data_stored":208929,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 07:57:18 2023","interfaces":[{"interface":"back","average":{"1min":0.55000000000000004,"5min":0.55000000000000004,"15min":0.55000000000000004},"min":{"1min":0.28399999999999997,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":0.85099999999999998,"5min":0.85099999999999998,"15min":0.85099999999999998},"last":0.73899999999999999},{"interface":"front","average":{"1min":0.51600000000000001,"5min":0.51600000000000001,"15min":0.51600000000000001},"min":{"1min":0.311,"5min":0.311,"15min":0.311},"max":{"1min":0.69999999999999996,"5min":0.69999999999999996,"15min":0.69999999999999996},"last":0.67200000000000004}]},{"osd":1,"last update":"Fri Nov 24 07:57:18 2023","interfaces":[{"interface":"back","average":{"1min":0.51200000000000001,"5min":0.51200000000000001,"15min":0.51200000000000001},"min":{"1min":0.33800000000000002,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.78200000000000003,"5min":0.78200000000000003,"15min":0.78200000000000003},"last":0.51100000000000001},{"interface":"front","average":{"1min":0.61299999999999999,"5min":0.61299999999999999,"15min":0.61299999999999999},"min":{"1min":0.44700000000000001,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":1.0069999999999999,"5min":1.0069999999999999,"15min":1.0069999999999999},"last":0.76700000000000002}]},{"osd":2,"last update":"Fri Nov 24 07:57:18 2023","interfaces":[{"interface":"back","average":{"1min":0.57499999999999996,"5min":0.57499999999999996,"15min":0.57499999999999996},"min":{"1min":0.34300000000000003,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.878,"5min":0.878,"15min":0.878},"last":0.71499999999999997},{"interface":"front","average":{"1min":0.59299999999999997,"5min":0.59299999999999997,"15min":0.59299999999999997},"min":{"1min":0.47099999999999997,"5min":0.47099999999999997,"15min":0.47099999999999997},"max":{"1min":0.82399999999999995,"5min":0.82399999999999995,"15min":0.82399999999999995},"last":0.56299999999999994}]},{"osd":3,"last update":"Fri Nov 24 07:57:18 2023","interfaces":[{"interface":"back","average":{"1min":0.56799999999999995,"5min":0.56799999999999995,"15min":0.56799999999999995},"min":{"1min":0.40000000000000002,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":0.876,"5min":0.876,"15min":0.876},"last":0.54500000000000004},{"interface":"front","average":{"1min":0.55600000000000005,"5min":0.55600000000000005,"15min":0.55600000000000005},"min":{"1min":0.379,"5min":0.379,"15min":0.379},"max":{"1min":0.68799999999999994,"5min":0.68799999999999994,"15min":0.68799999999999994},"last":0.60299999999999998}]},{"osd":4,"last update":"Fri Nov 24 07:57:18 2023","interfaces":[{"interface":"back","average":{"1min":0.51700000000000002,"5min":0.51700000000000002,"15min":0.51700000000000002},"min":{"1min":0.33000000000000002,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.70599999999999996,"5min":0.70599999999999996,"15min":0.70599999999999996},"last":0.36699999999999999},{"interface":"front","average":{"1min":0.60199999999999998,"5min":0.60199999999999998,"15min":0.60199999999999998},"min":{"1min":0.35199999999999998,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":1.087,"5min":1.087,"15min":1.087},"last":0.48599999999999999}]},{"osd":5,"last update":"Fri Nov 24 07:57:18 2023","interfaces":[{"interface":"back","average":{"1min":0.53100000000000003,"5min":0.53100000000000003,"15min":0.53100000000000003},"min":{"1min":0.30199999999999999,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.84999999999999998,"5min":0.84999999999999998,"15min":0.84999999999999998},"last":0.78200000000000003},{"interface":"front","average":{"1min":0.61199999999999999,"5min":0.61199999999999999,"15min":0.61199999999999999},"min":{"1min":0.39200000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.97199999999999998,"5min":0.97199999999999998,"15min":0.97199999999999998},"last":0.65100000000000002}]},{"osd":7,"last update":"Fri Nov 24 07:57:18 2023","interfaces":[{"interface":"back","average":{"1min":0.61799999999999999,"5min":0.61799999999999999,"15min":0.61799999999999999},"min":{"1min":0.49299999999999999,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":0.81499999999999995,"5min":0.81499999999999995,"15min":0.81499999999999995},"last":0.63600000000000001},{"interface":"front","average":{"1min":0.67000000000000004,"5min":0.67000000000000004,"15min":0.67000000000000004},"min":{"1min":0.497,"5min":0.497,"15min":0.497},"max":{"1min":0.92700000000000005,"5min":0.92700000000000005,"15min":0.92700000000000005},"last":0.81999999999999995}]}]},{"osd":1,"up_from":13,"seq":55834574874,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28096,"kb_used_data":1072,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343744,"statfs":{"total":96636764160,"available":96607993856,"internally_reserved":0,"allocated":1097728,"data_stored":791906,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 07:57:22 2023","interfaces":[{"interface":"back","average":{"1min":0.36399999999999999,"5min":0.36399999999999999,"15min":0.36399999999999999},"min":{"1min":0.20200000000000001,"5min":0.20200000000000001,"15min":0.20200000000000001},"max":{"1min":0.70899999999999996,"5min":0.70899999999999996,"15min":0.70899999999999996},"last":0.29099999999999998},{"interface":"front","average":{"1min":0.39800000000000002,"5min":0.39800000000000002,"15min":0.39800000000000002},"min":{"1min":0.214,"5min":0.214,"15min":0.214},"max":{"1min":0.82499999999999996,"5min":0.82499999999999996,"15min":0.82499999999999996},"last":0.30299999999999999}]},{"osd":2,"last update":"Fri Nov 24 07:57:22 2023","interfaces":[{"interface":"back","average":{"1min":0.373,"5min":0.373,"15min":0.373},"min":{"1min":0.22700000000000001,"5min":0.22700000000000001,"15min":0.22700000000000001},"max":{"1min":0.60999999999999999,"5min":0.60999999999999999,"15min":0.60999999999999999},"last":0.26700000000000002},{"interface":"front","average":{"1min":0.41299999999999998,"5min":0.41299999999999998,"15min":0.41299999999999998},"min":{"1min":0.26800000000000002,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":0.621,"5min":0.621,"15min":0.621},"last":0.35799999999999998}]},{"osd":3,"last update":"Fri Nov 24 07:57:22 2023","interfaces":[{"interface":"back","average":{"1min":0.44400000000000001,"5min":0.44400000000000001,"15min":0.44400000000000001},"min":{"1min":0.32100000000000001,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.66500000000000004,"5min":0.66500000000000004,"15min":0.66500000000000004},"last":0.379},{"interface":"front","average":{"1min":0.42799999999999999,"5min":0.42799999999999999,"15min":0.42799999999999999},"min":{"1min":0.28499999999999998,"5min":0.28499999999999998,"15min":0.28499999999999998},"max":{"1min":0.76300000000000001,"5min":0.76300000000000001,"15min":0.76300000000000001},"last":0.34599999999999997}]},{"osd":4,"last update":"Fri Nov 24 07:57:22 2023","interfaces":[{"interface":"back","average":{"1min":0.61099999999999999,"5min":0.61099999999999999,"15min":0.61099999999999999},"min":{"1min":0.43099999999999999,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":0.79800000000000004,"5min":0.79800000000000004,"15min":0.79800000000000004},"last":0.53700000000000003},{"interface":"front","average":{"1min":0.60599999999999998,"5min":0.60599999999999998,"15min":0.60599999999999998},"min":{"1min":0.41999999999999998,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":0.82199999999999995,"5min":0.82199999999999995,"15min":0.82199999999999995},"last":0.58999999999999997}]},{"osd":5,"last update":"Fri Nov 24 07:57:22 2023","interfaces":[{"interface":"back","average":{"1min":0.72499999999999998,"5min":0.72499999999999998,"15min":0.72499999999999998},"min":{"1min":0.37,"5min":0.37,"15min":0.37},"max":{"1min":2.7269999999999999,"5min":2.7269999999999999,"15min":2.7269999999999999},"last":0.439},{"interface":"front","average":{"1min":0.64500000000000002,"5min":0.64500000000000002,"15min":0.64500000000000002},"min":{"1min":0.44700000000000001,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":0.84199999999999997,"5min":0.84199999999999997,"15min":0.84199999999999997},"last":0.40899999999999997}]},{"osd":6,"last update":"Fri Nov 24 07:57:22 2023","interfaces":[{"interface":"back","average":{"1min":0.65800000000000003,"5min":0.65800000000000003,"15min":0.65800000000000003},"min":{"1min":0.47499999999999998,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":0.98199999999999998,"5min":0.98199999999999998,"15min":0.98199999999999998},"last":0.52200000000000002},{"interface":"front","average":{"1min":0.61699999999999999,"5min":0.61699999999999999,"15min":0.61699999999999999},"min":{"1min":0.42899999999999999,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":0.90700000000000003,"5min":0.90700000000000003,"15min":0.90700000000000003},"last":0.55100000000000005}]},{"osd":7,"last update":"Fri Nov 24 07:57:22 2023","interfaces":[{"interface":"back","average":{"1min":0.63700000000000001,"5min":0.63700000000000001,"15min":0.63700000000000001},"min":{"1min":0.40999999999999998,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":1.0649999999999999,"5min":1.0649999999999999,"15min":1.0649999999999999},"last":0.60099999999999998},{"interface":"front","average":{"1min":0.67000000000000004,"5min":0.67000000000000004,"15min":0.67000000000000004},"min":{"1min":0.442,"5min":0.442,"15min":0.442},"max":{"1min":0.88800000000000001,"5min":0.88800000000000001,"15min":0.88800000000000001},"last":0.504}]}]},{"osd":0,"up_from":13,"seq":55834574874,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27512,"kb_used_data":492,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344328,"statfs":{"total":96636764160,"available":96608591872,"internally_reserved":0,"allocated":503808,"data_stored":201538,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Nov 24 07:57:18 2023","interfaces":[{"interface":"back","average":{"1min":0.71299999999999997,"5min":0.71299999999999997,"15min":0.71299999999999997},"min":{"1min":0.23499999999999999,"5min":0.23499999999999999,"15min":0.23499999999999999},"max":{"1min":6.3550000000000004,"5min":6.3550000000000004,"15min":6.3550000000000004},"last":0.54300000000000004},{"interface":"front","average":{"1min":1.7290000000000001,"5min":1.7290000000000001,"15min":1.7290000000000001},"min":{"1min":0.30599999999999999,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":22.916,"5min":22.916,"15min":22.916},"last":0.60999999999999999}]},{"osd":2,"last update":"Fri Nov 24 07:57:18 2023","interfaces":[{"interface":"back","average":{"1min":0.69999999999999996,"5min":0.69999999999999996,"15min":0.69999999999999996},"min":{"1min":0.22800000000000001,"5min":0.22800000000000001,"15min":0.22800000000000001},"max":{"1min":6.3899999999999997,"5min":6.3899999999999997,"15min":6.3899999999999997},"last":0.47499999999999998},{"interface":"front","average":{"1min":1.0629999999999999,"5min":1.0629999999999999,"15min":1.0629999999999999},"min":{"1min":0.26800000000000002,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":11.93,"5min":11.93,"15min":11.93},"last":0.51700000000000002}]},{"osd":3,"last update":"Fri Nov 24 07:57:18 2023","interfaces":[{"interface":"back","average":{"1min":0.41999999999999998,"5min":0.41999999999999998,"15min":0.41999999999999998},"min":{"1min":0.28399999999999997,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":1.155,"5min":1.155,"15min":1.155},"last":0.56000000000000005},{"interface":"front","average":{"1min":1.127,"5min":1.127,"15min":1.127},"min":{"1min":0.32300000000000001,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":12.116,"5min":12.116,"15min":12.116},"last":0.58599999999999997}]},{"osd":4,"last update":"Fri Nov 24 07:57:18 2023","interfaces":[{"interface":"back","average":{"1min":0.64800000000000002,"5min":0.64800000000000002,"15min":0.64800000000000002},"min":{"1min":0.40500000000000003,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":0.97899999999999998,"5min":0.97899999999999998,"15min":0.97899999999999998},"last":0.82999999999999996},{"interface":"front","average":{"1min":1.9339999999999999,"5min":1.9339999999999999,"15min":1.9339999999999999},"min":{"1min":0.44400000000000001,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":22.963999999999999,"5min":22.963999999999999,"15min":22.963999999999999},"last":0.84599999999999997}]},{"osd":5,"last update":"Fri Nov 24 07:57:18 2023","interfaces":[{"interface":"back","average":{"1min":1.2929999999999999,"5min":1.2929999999999999,"15min":1.2929999999999999},"min":{"1min":0.40200000000000002,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":12.084,"5min":12.084,"15min":12.084},"last":0.86399999999999999},{"interface":"front","average":{"1min":0.67800000000000005,"5min":0.67800000000000005,"15min":0.67800000000000005},"min":{"1min":0.45200000000000001,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":1.073,"5min":1.073,"15min":1.073},"last":0.78800000000000003}]},{"osd":6,"last update":"Fri Nov 24 07:57:18 2023","interfaces":[{"interface":"back","average":{"1min":0.69099999999999995,"5min":0.69099999999999995,"15min":0.69099999999999995},"min":{"1min":0.42799999999999999,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":1.0580000000000001,"5min":1.0580000000000001,"15min":1.0580000000000001},"last":0.93000000000000005},{"interface":"front","average":{"1min":1.3140000000000001,"5min":1.3140000000000001,"15min":1.3140000000000001},"min":{"1min":0.41899999999999998,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":12.045,"5min":12.045,"15min":12.045},"last":0.74199999999999999}]},{"osd":7,"last update":"Fri Nov 24 07:57:18 2023","interfaces":[{"interface":"back","average":{"1min":0.65600000000000003,"5min":0.65600000000000003,"15min":0.65600000000000003},"min":{"1min":0.45400000000000001,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":1.038,"5min":1.038,"15min":1.038},"last":0.88300000000000001},{"interface":"front","average":{"1min":1.9630000000000001,"5min":1.9630000000000001,"15min":1.9630000000000001},"min":{"1min":0.49099999999999999,"5min":0.49099999999999999,"15min":0.49099999999999999},"max":{"1min":22.995000000000001,"5min":22.995000000000001,"15min":22.995000000000001},"last":0.77000000000000002}]}]},{"osd":2,"up_from":13,"seq":55834574874,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27536,"kb_used_data":508,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344304,"statfs":{"total":96636764160,"available":96608567296,"internally_reserved":0,"allocated":520192,"data_stored":208403,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 07:57:17 2023","interfaces":[{"interface":"back","average":{"1min":0.371,"5min":0.371,"15min":0.371},"min":{"1min":0.22500000000000001,"5min":0.22500000000000001,"15min":0.22500000000000001},"max":{"1min":0.58099999999999996,"5min":0.58099999999999996,"15min":0.58099999999999996},"last":0.34699999999999998},{"interface":"front","average":{"1min":0.40200000000000002,"5min":0.40200000000000002,"15min":0.40200000000000002},"min":{"1min":0.27400000000000002,"5min":0.27400000000000002,"15min":0.27400000000000002},"max":{"1min":0.54100000000000004,"5min":0.54100000000000004,"15min":0.54100000000000004},"last":0.33200000000000002}]},{"osd":1,"last update":"Fri Nov 24 07:57:17 2023","interfaces":[{"interface":"back","average":{"1min":0.378,"5min":0.378,"15min":0.378},"min":{"1min":0.23499999999999999,"5min":0.23499999999999999,"15min":0.23499999999999999},"max":{"1min":0.59199999999999997,"5min":0.59199999999999997,"15min":0.59199999999999997},"last":0.29599999999999999},{"interface":"front","average":{"1min":0.41599999999999998,"5min":0.41599999999999998,"15min":0.41599999999999998},"min":{"1min":0.25600000000000001,"5min":0.25600000000000001,"15min":0.25600000000000001},"max":{"1min":0.627,"5min":0.627,"15min":0.627},"last":0.38300000000000001}]},{"osd":3,"last update":"Fri Nov 24 07:57:17 2023","interfaces":[{"interface":"back","average":{"1min":0.41099999999999998,"5min":0.41099999999999998,"15min":0.41099999999999998},"min":{"1min":0.308,"5min":0.308,"15min":0.308},"max":{"1min":0.53300000000000003,"5min":0.53300000000000003,"15min":0.53300000000000003},"last":0.36699999999999999},{"interface":"front","average":{"1min":0.41399999999999998,"5min":0.41399999999999998,"15min":0.41399999999999998},"min":{"1min":0.27500000000000002,"5min":0.27500000000000002,"15min":0.27500000000000002},"max":{"1min":0.64600000000000002,"5min":0.64600000000000002,"15min":0.64600000000000002},"last":0.42199999999999999}]},{"osd":4,"last update":"Fri Nov 24 07:57:17 2023","interfaces":[{"interface":"back","average":{"1min":0.60799999999999998,"5min":0.60799999999999998,"15min":0.60799999999999998},"min":{"1min":0.42999999999999999,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":0.81599999999999995,"5min":0.81599999999999995,"15min":0.81599999999999995},"last":0.54200000000000004},{"interface":"front","average":{"1min":0.59999999999999998,"5min":0.59999999999999998,"15min":0.59999999999999998},"min":{"1min":0.41599999999999998,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":0.79600000000000004,"5min":0.79600000000000004,"15min":0.79600000000000004},"last":0.59499999999999997}]},{"osd":5,"last update":"Fri Nov 24 07:57:17 2023","interfaces":[{"interface":"back","average":{"1min":0.621,"5min":0.621,"15min":0.621},"min":{"1min":0.44900000000000001,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":0.84899999999999998,"5min":0.84899999999999998,"15min":0.84899999999999998},"last":0.51600000000000001},{"interface":"front","average":{"1min":0.71499999999999997,"5min":0.71499999999999997,"15min":0.71499999999999997},"min":{"1min":0.433,"5min":0.433,"15min":0.433},"max":{"1min":2.1699999999999999,"5min":2.1699999999999999,"15min":2.1699999999999999},"last":0.57799999999999996}]},{"osd":6,"last update":"Fri Nov 24 07:57:17 2023","interfaces":[{"interface":"back","average":{"1min":0.66700000000000004,"5min":0.66700000000000004,"15min":0.66700000000000004},"min":{"1min":0.47399999999999998,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":0.83599999999999997,"5min":0.83599999999999997,"15min":0.83599999999999997},"last":0.60599999999999998},{"interface":"front","average":{"1min":0.67100000000000004,"5min":0.67100000000000004,"15min":0.67100000000000004},"min":{"1min":0.496,"5min":0.496,"15min":0.496},"max":{"1min":0.88500000000000001,"5min":0.88500000000000001,"15min":0.88500000000000001},"last":0.5}]},{"osd":7,"last update":"Fri Nov 24 07:57:17 2023","interfaces":[{"interface":"back","average":{"1min":0.66200000000000003,"5min":0.66200000000000003,"15min":0.66200000000000003},"min":{"1min":0.443,"5min":0.443,"15min":0.443},"max":{"1min":0.85999999999999999,"5min":0.85999999999999999,"15min":0.85999999999999999},"last":0.56399999999999995},{"interface":"front","average":{"1min":0.68000000000000005,"5min":0.68000000000000005,"15min":0.68000000000000005},"min":{"1min":0.495,"5min":0.495,"15min":0.495},"max":{"1min":0.86399999999999999,"5min":0.86399999999999999,"15min":0.86399999999999999},"last":0.52900000000000003}]}]},{"osd":3,"up_from":13,"seq":55834574874,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27540,"kb_used_data":516,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344300,"statfs":{"total":96636764160,"available":96608563200,"internally_reserved":0,"allocated":528384,"data_stored":209925,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 07:57:20 2023","interfaces":[{"interface":"back","average":{"1min":0.35599999999999998,"5min":0.35599999999999998,"15min":0.35599999999999998},"min":{"1min":0.222,"5min":0.222,"15min":0.222},"max":{"1min":0.84799999999999998,"5min":0.84799999999999998,"15min":0.84799999999999998},"last":0.371},{"interface":"front","average":{"1min":0.36699999999999999,"5min":0.36699999999999999,"15min":0.36699999999999999},"min":{"1min":0.23699999999999999,"5min":0.23699999999999999,"15min":0.23699999999999999},"max":{"1min":0.873,"5min":0.873,"15min":0.873},"last":0.29099999999999998}]},{"osd":1,"last update":"Fri Nov 24 07:57:20 2023","interfaces":[{"interface":"back","average":{"1min":0.375,"5min":0.375,"15min":0.375},"min":{"1min":0.25700000000000001,"5min":0.25700000000000001,"15min":0.25700000000000001},"max":{"1min":0.92000000000000004,"5min":0.92000000000000004,"15min":0.92000000000000004},"last":0.33100000000000002},{"interface":"front","average":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"min":{"1min":0.248,"5min":0.248,"15min":0.248},"max":{"1min":0.81699999999999995,"5min":0.81699999999999995,"15min":0.81699999999999995},"last":0.35299999999999998}]},{"osd":2,"last update":"Fri Nov 24 07:57:20 2023","interfaces":[{"interface":"back","average":{"1min":0.36099999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"min":{"1min":0.24399999999999999,"5min":0.24399999999999999,"15min":0.24399999999999999},"max":{"1min":0.88500000000000001,"5min":0.88500000000000001,"15min":0.88500000000000001},"last":0.31},{"interface":"front","average":{"1min":0.46800000000000003,"5min":0.46800000000000003,"15min":0.46800000000000003},"min":{"1min":0.31900000000000001,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.98099999999999998,"5min":0.98099999999999998,"15min":0.98099999999999998},"last":0.39300000000000002}]},{"osd":4,"last update":"Fri Nov 24 07:57:20 2023","interfaces":[{"interface":"back","average":{"1min":0.79700000000000004,"5min":0.79700000000000004,"15min":0.79700000000000004},"min":{"1min":0.33500000000000002,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":5.7350000000000003,"5min":5.7350000000000003,"15min":5.7350000000000003},"last":0.65700000000000003},{"interface":"front","average":{"1min":0.58499999999999996,"5min":0.58499999999999996,"15min":0.58499999999999996},"min":{"1min":0.44500000000000001,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":1.27,"5min":1.27,"15min":1.27},"last":0.67900000000000005}]},{"osd":5,"last update":"Fri Nov 24 07:57:20 2023","interfaces":[{"interface":"back","average":{"1min":0.58799999999999997,"5min":0.58799999999999997,"15min":0.58799999999999997},"min":{"1min":0.40699999999999997,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":1.4430000000000001,"5min":1.4430000000000001,"15min":1.4430000000000001},"last":0.52900000000000003},{"interface":"front","average":{"1min":0.627,"5min":0.627,"15min":0.627},"min":{"1min":0.377,"5min":0.377,"15min":0.377},"max":{"1min":1.4239999999999999,"5min":1.4239999999999999,"15min":1.4239999999999999},"last":0.58799999999999997}]},{"osd":6,"last update":"Fri Nov 24 07:57:20 2023","interfaces":[{"interface":"back","average":{"1min":0.62,"5min":0.62,"15min":0.62},"min":{"1min":0.434,"5min":0.434,"15min":0.434},"max":{"1min":1.355,"5min":1.355,"15min":1.355},"last":0.51200000000000001},{"interface":"front","average":{"1min":0.84599999999999997,"5min":0.84599999999999997,"15min":0.84599999999999997},"min":{"1min":0.40999999999999998,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":5.6479999999999997,"5min":5.6479999999999997,"15min":5.6479999999999997},"last":0.59799999999999998}]},{"osd":7,"last update":"Fri Nov 24 07:57:20 2023","interfaces":[{"interface":"back","average":{"1min":0.66600000000000004,"5min":0.66600000000000004,"15min":0.66600000000000004},"min":{"1min":0.42299999999999999,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":1.51,"5min":1.51,"15min":1.51},"last":0.54500000000000004},{"interface":"front","average":{"1min":0.84199999999999997,"5min":0.84199999999999997,"15min":0.84199999999999997},"min":{"1min":0.45600000000000002,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":5.7069999999999999,"5min":5.7069999999999999,"15min":5.7069999999999999},"last":0.63}]}]},{"osd":4,"up_from":13,"seq":55834574873,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27552,"kb_used_data":524,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344288,"statfs":{"total":96636764160,"available":96608550912,"internally_reserved":0,"allocated":536576,"data_stored":210017,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 07:57:21 2023","interfaces":[{"interface":"back","average":{"1min":0.58099999999999996,"5min":0.58099999999999996,"15min":0.58099999999999996},"min":{"1min":0.37,"5min":0.37,"15min":0.37},"max":{"1min":0.79300000000000004,"5min":0.79300000000000004,"15min":0.79300000000000004},"last":0.54700000000000004},{"interface":"front","average":{"1min":0.54300000000000004,"5min":0.54300000000000004,"15min":0.54300000000000004},"min":{"1min":0.33600000000000002,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":1.0249999999999999,"5min":1.0249999999999999,"15min":1.0249999999999999},"last":0.45000000000000001}]},{"osd":1,"last update":"Fri Nov 24 07:57:21 2023","interfaces":[{"interface":"back","average":{"1min":0.55200000000000005,"5min":0.55200000000000005,"15min":0.55200000000000005},"min":{"1min":0.40000000000000002,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":0.86399999999999999,"5min":0.86399999999999999,"15min":0.86399999999999999},"last":0.51800000000000002},{"interface":"front","average":{"1min":0.55100000000000005,"5min":0.55100000000000005,"15min":0.55100000000000005},"min":{"1min":0.36099999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.98699999999999999,"5min":0.98699999999999999,"15min":0.98699999999999999},"last":0.42799999999999999}]},{"osd":2,"last update":"Fri Nov 24 07:57:21 2023","interfaces":[{"interface":"back","average":{"1min":0.56100000000000005,"5min":0.56100000000000005,"15min":0.56100000000000005},"min":{"1min":0.35799999999999998,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.81100000000000005,"5min":0.81100000000000005,"15min":0.81100000000000005},"last":0.629},{"interface":"front","average":{"1min":0.61299999999999999,"5min":0.61299999999999999,"15min":0.61299999999999999},"min":{"1min":0.38100000000000001,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.89200000000000002,"5min":0.89200000000000002,"15min":0.89200000000000002},"last":0.502}]},{"osd":3,"last update":"Fri Nov 24 07:57:21 2023","interfaces":[{"interface":"back","average":{"1min":0.58499999999999996,"5min":0.58499999999999996,"15min":0.58499999999999996},"min":{"1min":0.439,"5min":0.439,"15min":0.439},"max":{"1min":0.89700000000000002,"5min":0.89700000000000002,"15min":0.89700000000000002},"last":0.58799999999999997},{"interface":"front","average":{"1min":0.63100000000000001,"5min":0.63100000000000001,"15min":0.63100000000000001},"min":{"1min":0.40300000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.96199999999999997,"5min":0.96199999999999997,"15min":0.96199999999999997},"last":0.60499999999999998}]},{"osd":5,"last update":"Fri Nov 24 07:57:21 2023","interfaces":[{"interface":"back","average":{"1min":0.45500000000000002,"5min":0.45500000000000002,"15min":0.45500000000000002},"min":{"1min":0.29799999999999999,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":0.70499999999999996,"5min":0.70499999999999996,"15min":0.70499999999999996},"last":0.56999999999999995},{"interface":"front","average":{"1min":0.61899999999999999,"5min":0.61899999999999999,"15min":0.61899999999999999},"min":{"1min":0.439,"5min":0.439,"15min":0.439},"max":{"1min":0.86399999999999999,"5min":0.86399999999999999,"15min":0.86399999999999999},"last":0.69499999999999995}]},{"osd":6,"last update":"Fri Nov 24 07:57:21 2023","interfaces":[{"interface":"back","average":{"1min":0.58599999999999997,"5min":0.58599999999999997,"15min":0.58599999999999997},"min":{"1min":0.35499999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.76800000000000002,"5min":0.76800000000000002,"15min":0.76800000000000002},"last":0.65600000000000003},{"interface":"front","average":{"1min":0.61399999999999999,"5min":0.61399999999999999,"15min":0.61399999999999999},"min":{"1min":0.436,"5min":0.436,"15min":0.436},"max":{"1min":1.095,"5min":1.095,"15min":1.095},"last":0.64200000000000002}]},{"osd":7,"last update":"Fri Nov 24 07:57:21 2023","interfaces":[{"interface":"back","average":{"1min":0.57299999999999995,"5min":0.57299999999999995,"15min":0.57299999999999995},"min":{"1min":0.35099999999999998,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.89300000000000002,"5min":0.89300000000000002,"15min":0.89300000000000002},"last":0.66600000000000004},{"interface":"front","average":{"1min":0.66300000000000003,"5min":0.66300000000000003,"15min":0.66300000000000003},"min":{"1min":0.41099999999999998,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":0.91500000000000004,"5min":0.91500000000000004,"15min":0.91500000000000004},"last":0.68200000000000005}]}]},{"osd":5,"up_from":13,"seq":55834574874,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28096,"kb_used_data":1072,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343744,"statfs":{"total":96636764160,"available":96607993856,"internally_reserved":0,"allocated":1097728,"data_stored":791906,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 07:57:17 2023","interfaces":[{"interface":"back","average":{"1min":0.56200000000000006,"5min":0.56200000000000006,"15min":0.56200000000000006},"min":{"1min":0.32700000000000001,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":1.0109999999999999,"5min":1.0109999999999999,"15min":1.0109999999999999},"last":0.51600000000000001},{"interface":"front","average":{"1min":0.80000000000000004,"5min":0.80000000000000004,"15min":0.80000000000000004},"min":{"1min":0.33400000000000002,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":5.431,"5min":5.431,"15min":5.431},"last":0.437}]},{"osd":1,"last update":"Fri Nov 24 07:57:17 2023","interfaces":[{"interface":"back","average":{"1min":0.55900000000000005,"5min":0.55900000000000005,"15min":0.55900000000000005},"min":{"1min":0.28899999999999998,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":1.016,"5min":1.016,"15min":1.016},"last":0.49299999999999999},{"interface":"front","average":{"1min":0.56999999999999995,"5min":0.56999999999999995,"15min":0.56999999999999995},"min":{"1min":0.311,"5min":0.311,"15min":0.311},"max":{"1min":0.96499999999999997,"5min":0.96499999999999997,"15min":0.96499999999999997},"last":0.58399999999999996}]},{"osd":2,"last update":"Fri Nov 24 07:57:17 2023","interfaces":[{"interface":"back","average":{"1min":0.55100000000000005,"5min":0.55100000000000005,"15min":0.55100000000000005},"min":{"1min":0.377,"5min":0.377,"15min":0.377},"max":{"1min":1.163,"5min":1.163,"15min":1.163},"last":0.53400000000000003},{"interface":"front","average":{"1min":0.63,"5min":0.63,"15min":0.63},"min":{"1min":0.39200000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":1.119,"5min":1.119,"15min":1.119},"last":0.41599999999999998}]},{"osd":3,"last update":"Fri Nov 24 07:57:17 2023","interfaces":[{"interface":"back","average":{"1min":0.90500000000000003,"5min":0.90500000000000003,"15min":0.90500000000000003},"min":{"1min":0.36499999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":5.609,"5min":5.609,"15min":5.609},"last":0.46800000000000003},{"interface":"front","average":{"1min":0.67600000000000005,"5min":0.67600000000000005,"15min":0.67600000000000005},"min":{"1min":0.41899999999999998,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":1.288,"5min":1.288,"15min":1.288},"last":0.64300000000000002}]},{"osd":4,"last update":"Fri Nov 24 07:57:17 2023","interfaces":[{"interface":"back","average":{"1min":0.51000000000000001,"5min":0.51000000000000001,"15min":0.51000000000000001},"min":{"1min":0.36399999999999999,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.68799999999999994,"5min":0.68799999999999994,"15min":0.68799999999999994},"last":0.55000000000000004},{"interface":"front","average":{"1min":0.57199999999999995,"5min":0.57199999999999995,"15min":0.57199999999999995},"min":{"1min":0.39800000000000002,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.78300000000000003,"5min":0.78300000000000003,"15min":0.78300000000000003},"last":0.626}]},{"osd":6,"last update":"Fri Nov 24 07:57:17 2023","interfaces":[{"interface":"back","average":{"1min":0.57099999999999995,"5min":0.57099999999999995,"15min":0.57099999999999995},"min":{"1min":0.35599999999999998,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.89700000000000002,"5min":0.89700000000000002,"15min":0.89700000000000002},"last":0.57299999999999995},{"interface":"front","average":{"1min":0.625,"5min":0.625,"15min":0.625},"min":{"1min":0.42899999999999999,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":0.88200000000000001,"5min":0.88200000000000001,"15min":0.88200000000000001},"last":0.60499999999999998}]},{"osd":7,"last update":"Fri Nov 24 07:57:17 2023","interfaces":[{"interface":"back","average":{"1min":0.64600000000000002,"5min":0.64600000000000002,"15min":0.64600000000000002},"min":{"1min":0.45000000000000001,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":0.96599999999999997,"5min":0.96599999999999997,"15min":0.96599999999999997},"last":0.70299999999999996},{"interface":"front","average":{"1min":0.65300000000000002,"5min":0.65300000000000002,"15min":0.65300000000000002},"min":{"1min":0.46300000000000002,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":0.94199999999999995,"5min":0.94199999999999995,"15min":0.94199999999999995},"last":0.68799999999999994}]}]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1704,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-11-24T07:58:06.768 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-11-24T07:58:06.925 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:06.924+0000 7f6550044700 1 Processor -- start 2023-11-24T07:58:06.926 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:06.924+0000 7f6550044700 1 -- start start 2023-11-24T07:58:06.928 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:06.925+0000 7f6550044700 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f654806b620 0x7f65480b3a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:06.928 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:06.925+0000 7f6550044700 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f65480b5ab0 0x7f65480b3f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:06.928 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:06.925+0000 7f6550044700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f65481478c0 0x7f6548161ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:06.928 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:06.925+0000 7f6550044700 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6548071f50 con 0x7f65481478c0 2023-11-24T07:58:06.928 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:06.925+0000 7f6550044700 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6548072230 con 0x7f65480b5ab0 2023-11-24T07:58:06.928 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:06.925+0000 7f6550044700 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6548072510 con 0x7f654806b620 2023-11-24T07:58:06.929 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:06.925+0000 7f654e5e1700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f65481478c0 0x7f6548161ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:06.929 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:06.925+0000 7f654e5e1700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f65481478c0 0x7f6548161ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:52812/0 (socket says 172.21.15.59:52812) 2023-11-24T07:58:06.929 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:06.925+0000 7f654e5e1700 1 -- 172.21.15.59:0/76164232 learned_addr learned my addr 172.21.15.59:0/76164232 (peer_addr_for_me v2:172.21.15.59:0/0) 2023-11-24T07:58:06.929 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:06.925+0000 7f654d5df700 1 --2- 172.21.15.59:0/76164232 >> v2:172.21.15.76:3300/0 conn(0x7f65480b5ab0 0x7f65480b3f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:06.929 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:06.925+0000 7f654dde0700 1 --2- 172.21.15.59:0/76164232 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f654806b620 0x7f65480b3a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:06.929 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:06.925+0000 7f654e5e1700 1 -- 172.21.15.59:0/76164232 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f654806b620 msgr2=0x7f65480b3a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T07:58:06.929 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:06.925+0000 7f654e5e1700 1 --2- 172.21.15.59:0/76164232 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f654806b620 0x7f65480b3a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:06.930 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:06.925+0000 7f654e5e1700 1 -- 172.21.15.59:0/76164232 >> v2:172.21.15.76:3300/0 conn(0x7f65480b5ab0 msgr2=0x7f65480b3f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T07:58:06.930 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:06.925+0000 7f654e5e1700 1 --2- 172.21.15.59:0/76164232 >> v2:172.21.15.76:3300/0 conn(0x7f65480b5ab0 0x7f65480b3f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:06.930 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:06.925+0000 7f654e5e1700 1 -- 172.21.15.59:0/76164232 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f65480727f0 con 0x7f65481478c0 2023-11-24T07:58:06.930 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:06.926+0000 7f654e5e1700 1 --2- 172.21.15.59:0/76164232 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f65481478c0 0x7f6548161ba0 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7f6548069550 tx=0x7f654000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=84e0e4707de9eddd server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:58:06.930 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:06.926+0000 7f654cdde700 1 -- 172.21.15.59:0/76164232 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f654000bbc0 con 0x7f65481478c0 2023-11-24T07:58:06.930 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:06.926+0000 7f654cdde700 1 -- 172.21.15.59:0/76164232 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f654000bd20 con 0x7f65481478c0 2023-11-24T07:58:06.930 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:06.926+0000 7f654cdde700 1 -- 172.21.15.59:0/76164232 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6540014b10 con 0x7f65481478c0 2023-11-24T07:58:06.930 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:06.926+0000 7f6550044700 1 -- 172.21.15.59:0/76164232 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f65481478c0 msgr2=0x7f6548161ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:06.930 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:06.926+0000 7f6550044700 1 --2- 172.21.15.59:0/76164232 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f65481478c0 0x7f6548161ba0 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7f6548069550 tx=0x7f654000b330 comp rx=0 tx=0).stop 2023-11-24T07:58:06.930 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:06.926+0000 7f6550044700 1 -- 172.21.15.59:0/76164232 shutdown_connections 2023-11-24T07:58:06.930 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:06.926+0000 7f6550044700 1 --2- 172.21.15.59:0/76164232 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f65481478c0 0x7f6548161ba0 unknown :-1 s=CLOSED pgs=550 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:06.931 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:06.926+0000 7f6550044700 1 --2- 172.21.15.59:0/76164232 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f654806b620 0x7f65480b3a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:06.931 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:06.926+0000 7f6550044700 1 --2- 172.21.15.59:0/76164232 >> v2:172.21.15.76:3300/0 conn(0x7f65480b5ab0 0x7f65480b3f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:06.931 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:06.926+0000 7f6550044700 1 -- 172.21.15.59:0/76164232 >> 172.21.15.59:0/76164232 conn(0x7f65480b2740 msgr2=0x7f65480b2b20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:58:06.931 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:06.926+0000 7f6550044700 1 -- 172.21.15.59:0/76164232 shutdown_connections 2023-11-24T07:58:06.931 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:06.926+0000 7f6550044700 1 -- 172.21.15.59:0/76164232 wait complete. 2023-11-24T07:58:06.934 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:06.928+0000 7f6550044700 1 Processor -- start 2023-11-24T07:58:06.934 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:06.928+0000 7f6550044700 1 -- start start 2023-11-24T07:58:06.934 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:06.928+0000 7f6550044700 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f654806b620 0x7f6548137080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:06.934 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:06.928+0000 7f6550044700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f65480b5ab0 0x7f65480aae60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:06.934 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:06.928+0000 7f6550044700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f65481478c0 0x7f65480b13c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:06.935 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:06.928+0000 7f6550044700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65481375c0 con 0x7f65480b5ab0 2023-11-24T07:58:06.935 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:06.928+0000 7f6550044700 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65480cfa30 con 0x7f65481478c0 2023-11-24T07:58:06.935 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:06.928+0000 7f6550044700 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f65480ab3a0 con 0x7f654806b620 2023-11-24T07:58:06.935 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:06.929+0000 7f654d5df700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f65480b5ab0 0x7f65480aae60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:06.935 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:06.929+0000 7f654d5df700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f65480b5ab0 0x7f65480aae60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:52816/0 (socket says 172.21.15.59:52816) 2023-11-24T07:58:06.935 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:06.929+0000 7f654d5df700 1 -- 172.21.15.59:0/1130575686 learned_addr learned my addr 172.21.15.59:0/1130575686 (peer_addr_for_me v2:172.21.15.59:0/0) 2023-11-24T07:58:06.935 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:06.929+0000 7f654dde0700 1 --2- 172.21.15.59:0/1130575686 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f654806b620 0x7f6548137080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:06.935 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:06.929+0000 7f654e5e1700 1 --2- 172.21.15.59:0/1130575686 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f65481478c0 0x7f65480b13c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:06.935 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:06.929+0000 7f654d5df700 1 -- 172.21.15.59:0/1130575686 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f654806b620 msgr2=0x7f6548137080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:06.936 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:06.929+0000 7f654d5df700 1 --2- 172.21.15.59:0/1130575686 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f654806b620 0x7f6548137080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:06.936 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:06.929+0000 7f654d5df700 1 -- 172.21.15.59:0/1130575686 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f65481478c0 msgr2=0x7f65480b13c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:06.936 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:06.929+0000 7f654d5df700 1 --2- 172.21.15.59:0/1130575686 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f65481478c0 0x7f65480b13c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:06.936 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:06.929+0000 7f654d5df700 1 -- 172.21.15.59:0/1130575686 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6540009b30 con 0x7f65480b5ab0 2023-11-24T07:58:06.936 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:06.929+0000 7f654dde0700 1 --2- 172.21.15.59:0/1130575686 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f654806b620 0x7f6548137080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-24T07:58:06.936 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:06.929+0000 7f654e5e1700 1 --2- 172.21.15.59:0/1130575686 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f65481478c0 0x7f65480b13c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-24T07:58:06.936 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:06.929+0000 7f654d5df700 1 --2- 172.21.15.59:0/1130575686 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f65480b5ab0 0x7f65480aae60 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7f653800e810 tx=0x7f653800ebd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:58:06.936 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:06.929+0000 7f653e7fc700 1 -- 172.21.15.59:0/1130575686 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6538013610 con 0x7f65480b5ab0 2023-11-24T07:58:06.936 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:06.929+0000 7f653e7fc700 1 -- 172.21.15.59:0/1130575686 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6538013770 con 0x7f65480b5ab0 2023-11-24T07:58:06.936 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:06.929+0000 7f653e7fc700 1 -- 172.21.15.59:0/1130575686 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6538013b00 con 0x7f65480b5ab0 2023-11-24T07:58:06.936 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:06.929+0000 7f6550044700 1 -- 172.21.15.59:0/1130575686 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f65480a9cf0 con 0x7f65480b5ab0 2023-11-24T07:58:06.937 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:06.929+0000 7f6550044700 1 -- 172.21.15.59:0/1130575686 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f65480aa2b0 con 0x7f65480b5ab0 2023-11-24T07:58:06.937 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:06.930+0000 7f6550044700 1 -- 172.21.15.59:0/1130575686 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6548068670 con 0x7f65480b5ab0 2023-11-24T07:58:06.937 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:06.931+0000 7f653e7fc700 1 -- 172.21.15.59:0/1130575686 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 6) v1 ==== 89282+0+0 (secure 0 0 0) 0x7f6538013c60 con 0x7f65480b5ab0 2023-11-24T07:58:06.937 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:06.931+0000 7f653e7fc700 1 --2- 172.21.15.59:0/1130575686 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f652c06af60 0x7f652c06d420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:06.937 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:06.931+0000 7f653e7fc700 1 -- 172.21.15.59:0/1130575686 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f6538089310 con 0x7f65480b5ab0 2023-11-24T07:58:06.937 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:06.931+0000 7f654dde0700 1 --2- 172.21.15.59:0/1130575686 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f652c06af60 0x7f652c06d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:06.937 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:06.932+0000 7f654dde0700 1 --2- 172.21.15.59:0/1130575686 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f652c06af60 0x7f652c06d420 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f6544009970 tx=0x7f6544008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:58:06.937 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:06.936+0000 7f653e7fc700 1 -- 172.21.15.59:0/1130575686 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f6538054350 con 0x7f65480b5ab0 2023-11-24T07:58:07.133 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:07.131+0000 7f6550044700 1 -- 172.21.15.59:0/1130575686 --> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f654806ba00 con 0x7f652c06af60 2023-11-24T07:58:07.141 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:07.140+0000 7f653e7fc700 1 -- 172.21.15.59:0/1130575686 <== mgr.4100 v2:172.21.15.76:6832/4264046536 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+288939 (secure 0 0 0) 0x7f6544009e40 con 0x7f652c06af60 2023-11-24T07:58:07.144 INFO:teuthology.orchestra.run.smithi059.stdout: 2023-11-24T07:58:07.147 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:07.143+0000 7f6550044700 1 -- 172.21.15.59:0/1130575686 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f652c06af60 msgr2=0x7f652c06d420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:07.147 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:07.143+0000 7f6550044700 1 --2- 172.21.15.59:0/1130575686 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f652c06af60 0x7f652c06d420 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f6544009970 tx=0x7f6544008040 comp rx=0 tx=0).stop 2023-11-24T07:58:07.147 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:07.143+0000 7f6550044700 1 -- 172.21.15.59:0/1130575686 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f65480b5ab0 msgr2=0x7f65480aae60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:07.147 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:07.143+0000 7f6550044700 1 --2- 172.21.15.59:0/1130575686 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f65480b5ab0 0x7f65480aae60 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7f653800e810 tx=0x7f653800ebd0 comp rx=0 tx=0).stop 2023-11-24T07:58:07.147 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:07.143+0000 7f6550044700 1 -- 172.21.15.59:0/1130575686 shutdown_connections 2023-11-24T07:58:07.147 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:07.143+0000 7f6550044700 1 --2- 172.21.15.59:0/1130575686 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f652c06af60 0x7f652c06d420 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:07.147 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:07.143+0000 7f6550044700 1 --2- 172.21.15.59:0/1130575686 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f65481478c0 0x7f65480b13c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:07.147 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:07.143+0000 7f6550044700 1 --2- 172.21.15.59:0/1130575686 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f654806b620 0x7f6548137080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:07.148 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:07.143+0000 7f6550044700 1 --2- 172.21.15.59:0/1130575686 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f65480b5ab0 0x7f65480aae60 unknown :-1 s=CLOSED pgs=551 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:07.148 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:07.143+0000 7f6550044700 1 -- 172.21.15.59:0/1130575686 >> 172.21.15.59:0/1130575686 conn(0x7f65480b2740 msgr2=0x7f65480cfe10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:58:07.148 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:07.143+0000 7f6550044700 1 -- 172.21.15.59:0/1130575686 shutdown_connections 2023-11-24T07:58:07.148 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:07.143+0000 7f6550044700 1 -- 172.21.15.59:0/1130575686 wait complete. 2023-11-24T07:58:07.148 INFO:teuthology.orchestra.run.smithi059.stderr:dumped all 2023-11-24T07:58:07.156 INFO:teuthology.orchestra.run.smithi059.stdout:{"pg_ready":true,"pg_map":{"version":92,"stamp":"2023-11-24T07:58:06.322486+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":42,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":221408,"kb_used_data":5204,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754753312,"statfs":{"total":773094113280,"available":772867391488,"internally_reserved":0,"allocated":5328896,"data_stored":2830971,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12745,"internal_metadata":221236791},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":31,"apply_latency_ms":31,"commit_latency_ns":31000000,"apply_latency_ns":31000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.738457"},"pg_stats":[{"pgid":"7.3f","version":"0'0","reported_seq":9,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-11-24T07:58:02.625818+0000","last_change":"2023-11-24T07:58:02.619364+0000","last_active":"2023-11-24T07:58:02.625818+0000","last_peered":"2023-11-24T07:58:02.625818+0000","last_clean":"2023-11-24T07:58:02.625818+0000","last_became_active":"2023-11-24T07:58:02.618670+0000","last_became_peered":"2023-11-24T07:58:02.618670+0000","last_unstale":"2023-11-24T07:58:02.625818+0000","last_undegraded":"2023-11-24T07:58:02.625818+0000","last_fullsized":"2023-11-24T07:58:02.625818+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:58:01.583599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:58:01.583599+0000","last_clean_scrub_stamp":"2023-11-24T07:58:01.583599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T14:01:26.773202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.3e","version":"0'0","reported_seq":9,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-11-24T07:58:02.624594+0000","last_change":"2023-11-24T07:58:02.621152+0000","last_active":"2023-11-24T07:58:02.624594+0000","last_peered":"2023-11-24T07:58:02.624594+0000","last_clean":"2023-11-24T07:58:02.624594+0000","last_became_active":"2023-11-24T07:58:02.620855+0000","last_became_peered":"2023-11-24T07:58:02.620855+0000","last_unstale":"2023-11-24T07:58:02.624594+0000","last_undegraded":"2023-11-24T07:58:02.624594+0000","last_fullsized":"2023-11-24T07:58:02.624594+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:58:01.583599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:58:01.583599+0000","last_clean_scrub_stamp":"2023-11-24T07:58:01.583599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:33:23.934909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.3d","version":"0'0","reported_seq":9,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-11-24T07:58:02.627667+0000","last_change":"2023-11-24T07:58:02.622555+0000","last_active":"2023-11-24T07:58:02.627667+0000","last_peered":"2023-11-24T07:58:02.627667+0000","last_clean":"2023-11-24T07:58:02.627667+0000","last_became_active":"2023-11-24T07:58:02.622262+0000","last_became_peered":"2023-11-24T07:58:02.622262+0000","last_unstale":"2023-11-24T07:58:02.627667+0000","last_undegraded":"2023-11-24T07:58:02.627667+0000","last_fullsized":"2023-11-24T07:58:02.627667+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:58:01.583599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:58:01.583599+0000","last_clean_scrub_stamp":"2023-11-24T07:58:01.583599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:18:58.981383+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.3c","version":"0'0","reported_seq":9,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-11-24T07:58:02.617142+0000","last_change":"2023-11-24T07:58:02.612749+0000","last_active":"2023-11-24T07:58:02.617142+0000","last_peered":"2023-11-24T07:58:02.617142+0000","last_clean":"2023-11-24T07:58:02.617142+0000","last_became_active":"2023-11-24T07:58:02.612311+0000","last_became_peered":"2023-11-24T07:58:02.612311+0000","last_unstale":"2023-11-24T07:58:02.617142+0000","last_undegraded":"2023-11-24T07:58:02.617142+0000","last_fullsized":"2023-11-24T07:58:02.617142+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:58:01.583599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:58:01.583599+0000","last_clean_scrub_stamp":"2023-11-24T07:58:01.583599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T14:01:26.773202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.3b","version":"0'0","reported_seq":9,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-11-24T07:58:02.619558+0000","last_change":"2023-11-24T07:58:02.612584+0000","last_active":"2023-11-24T07:58:02.619558+0000","last_peered":"2023-11-24T07:58:02.619558+0000","last_clean":"2023-11-24T07:58:02.619558+0000","last_became_active":"2023-11-24T07:58:02.612277+0000","last_became_peered":"2023-11-24T07:58:02.612277+0000","last_unstale":"2023-11-24T07:58:02.619558+0000","last_undegraded":"2023-11-24T07:58:02.619558+0000","last_fullsized":"2023-11-24T07:58:02.619558+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:58:01.583599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:58:01.583599+0000","last_clean_scrub_stamp":"2023-11-24T07:58:01.583599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T13:42:52.916962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.3a","version":"0'0","reported_seq":9,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-11-24T07:58:02.616647+0000","last_change":"2023-11-24T07:58:02.609202+0000","last_active":"2023-11-24T07:58:02.616647+0000","last_peered":"2023-11-24T07:58:02.616647+0000","last_clean":"2023-11-24T07:58:02.616647+0000","last_became_active":"2023-11-24T07:58:02.608893+0000","last_became_peered":"2023-11-24T07:58:02.608893+0000","last_unstale":"2023-11-24T07:58:02.616647+0000","last_undegraded":"2023-11-24T07:58:02.616647+0000","last_fullsized":"2023-11-24T07:58:02.616647+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:58:01.583599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:58:01.583599+0000","last_clean_scrub_stamp":"2023-11-24T07:58:01.583599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T14:02:23.676273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.39","version":"0'0","reported_seq":9,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-11-24T07:58:02.628269+0000","last_change":"2023-11-24T07:58:02.613433+0000","last_active":"2023-11-24T07:58:02.628269+0000","last_peered":"2023-11-24T07:58:02.628269+0000","last_clean":"2023-11-24T07:58:02.628269+0000","last_became_active":"2023-11-24T07:58:02.613091+0000","last_became_peered":"2023-11-24T07:58:02.613091+0000","last_unstale":"2023-11-24T07:58:02.628269+0000","last_undegraded":"2023-11-24T07:58:02.628269+0000","last_fullsized":"2023-11-24T07:58:02.628269+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:58:01.583599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:58:01.583599+0000","last_clean_scrub_stamp":"2023-11-24T07:58:01.583599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T19:14:06.767372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.38","version":"0'0","reported_seq":9,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-11-24T07:58:02.624756+0000","last_change":"2023-11-24T07:58:02.622700+0000","last_active":"2023-11-24T07:58:02.624756+0000","last_peered":"2023-11-24T07:58:02.624756+0000","last_clean":"2023-11-24T07:58:02.624756+0000","last_became_active":"2023-11-24T07:58:02.622338+0000","last_became_peered":"2023-11-24T07:58:02.622338+0000","last_unstale":"2023-11-24T07:58:02.624756+0000","last_undegraded":"2023-11-24T07:58:02.624756+0000","last_fullsized":"2023-11-24T07:58:02.624756+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:58:01.583599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:58:01.583599+0000","last_clean_scrub_stamp":"2023-11-24T07:58:01.583599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T12:50:53.080554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.37","version":"0'0","reported_seq":9,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-11-24T07:58:02.624523+0000","last_change":"2023-11-24T07:58:02.619368+0000","last_active":"2023-11-24T07:58:02.624523+0000","last_peered":"2023-11-24T07:58:02.624523+0000","last_clean":"2023-11-24T07:58:02.624523+0000","last_became_active":"2023-11-24T07:58:02.619037+0000","last_became_peered":"2023-11-24T07:58:02.619037+0000","last_unstale":"2023-11-24T07:58:02.624523+0000","last_undegraded":"2023-11-24T07:58:02.624523+0000","last_fullsized":"2023-11-24T07:58:02.624523+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:58:01.583599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:58:01.583599+0000","last_clean_scrub_stamp":"2023-11-24T07:58:01.583599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T19:15:20.314127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.36","version":"0'0","reported_seq":9,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-11-24T07:58:02.628556+0000","last_change":"2023-11-24T07:58:02.625865+0000","last_active":"2023-11-24T07:58:02.628556+0000","last_peered":"2023-11-24T07:58:02.628556+0000","last_clean":"2023-11-24T07:58:02.628556+0000","last_became_active":"2023-11-24T07:58:02.625623+0000","last_became_peered":"2023-11-24T07:58:02.625623+0000","last_unstale":"2023-11-24T07:58:02.628556+0000","last_undegraded":"2023-11-24T07:58:02.628556+0000","last_fullsized":"2023-11-24T07:58:02.628556+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:58:01.583599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:58:01.583599+0000","last_clean_scrub_stamp":"2023-11-24T07:58:01.583599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T09:35:04.179278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.35","version":"0'0","reported_seq":9,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-11-24T07:58:02.613474+0000","last_change":"2023-11-24T07:58:02.608246+0000","last_active":"2023-11-24T07:58:02.613474+0000","last_peered":"2023-11-24T07:58:02.613474+0000","last_clean":"2023-11-24T07:58:02.613474+0000","last_became_active":"2023-11-24T07:58:02.607979+0000","last_became_peered":"2023-11-24T07:58:02.607979+0000","last_unstale":"2023-11-24T07:58:02.613474+0000","last_undegraded":"2023-11-24T07:58:02.613474+0000","last_fullsized":"2023-11-24T07:58:02.613474+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:58:01.583599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:58:01.583599+0000","last_clean_scrub_stamp":"2023-11-24T07:58:01.583599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T15:36:23.158160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.34","version":"0'0","reported_seq":9,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-11-24T07:58:02.629459+0000","last_change":"2023-11-24T07:58:02.625020+0000","last_active":"2023-11-24T07:58:02.629459+0000","last_peered":"2023-11-24T07:58:02.629459+0000","last_clean":"2023-11-24T07:58:02.629459+0000","last_became_active":"2023-11-24T07:58:02.615508+0000","last_became_peered":"2023-11-24T07:58:02.615508+0000","last_unstale":"2023-11-24T07:58:02.629459+0000","last_undegraded":"2023-11-24T07:58:02.629459+0000","last_fullsized":"2023-11-24T07:58:02.629459+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:58:01.583599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:58:01.583599+0000","last_clean_scrub_stamp":"2023-11-24T07:58:01.583599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T14:02:23.676273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.33","version":"0'0","reported_seq":9,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-11-24T07:58:02.616949+0000","last_change":"2023-11-24T07:58:02.609164+0000","last_active":"2023-11-24T07:58:02.616949+0000","last_peered":"2023-11-24T07:58:02.616949+0000","last_clean":"2023-11-24T07:58:02.616949+0000","last_became_active":"2023-11-24T07:58:02.608870+0000","last_became_peered":"2023-11-24T07:58:02.608870+0000","last_unstale":"2023-11-24T07:58:02.616949+0000","last_undegraded":"2023-11-24T07:58:02.616949+0000","last_fullsized":"2023-11-24T07:58:02.616949+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:58:01.583599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:58:01.583599+0000","last_clean_scrub_stamp":"2023-11-24T07:58:01.583599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:37:31.802505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.32","version":"0'0","reported_seq":9,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-11-24T07:58:02.624190+0000","last_change":"2023-11-24T07:58:02.621645+0000","last_active":"2023-11-24T07:58:02.624190+0000","last_peered":"2023-11-24T07:58:02.624190+0000","last_clean":"2023-11-24T07:58:02.624190+0000","last_became_active":"2023-11-24T07:58:02.621380+0000","last_became_peered":"2023-11-24T07:58:02.621380+0000","last_unstale":"2023-11-24T07:58:02.624190+0000","last_undegraded":"2023-11-24T07:58:02.624190+0000","last_fullsized":"2023-11-24T07:58:02.624190+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:58:01.583599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:58:01.583599+0000","last_clean_scrub_stamp":"2023-11-24T07:58:01.583599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T14:21:21.611502+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.31","version":"0'0","reported_seq":9,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-11-24T07:58:02.624303+0000","last_change":"2023-11-24T07:58:02.621867+0000","last_active":"2023-11-24T07:58:02.624303+0000","last_peered":"2023-11-24T07:58:02.624303+0000","last_clean":"2023-11-24T07:58:02.624303+0000","last_became_active":"2023-11-24T07:58:02.621168+0000","last_became_peered":"2023-11-24T07:58:02.621168+0000","last_unstale":"2023-11-24T07:58:02.624303+0000","last_undegraded":"2023-11-24T07:58:02.624303+0000","last_fullsized":"2023-11-24T07:58:02.624303+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:58:01.583599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:58:01.583599+0000","last_clean_scrub_stamp":"2023-11-24T07:58:01.583599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T12:08:53.706469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.30","version":"0'0","reported_seq":9,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-11-24T07:58:02.629690+0000","last_change":"2023-11-24T07:58:02.622017+0000","last_active":"2023-11-24T07:58:02.629690+0000","last_peered":"2023-11-24T07:58:02.629690+0000","last_clean":"2023-11-24T07:58:02.629690+0000","last_became_active":"2023-11-24T07:58:02.621703+0000","last_became_peered":"2023-11-24T07:58:02.621703+0000","last_unstale":"2023-11-24T07:58:02.629690+0000","last_undegraded":"2023-11-24T07:58:02.629690+0000","last_fullsized":"2023-11-24T07:58:02.629690+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:58:01.583599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:58:01.583599+0000","last_clean_scrub_stamp":"2023-11-24T07:58:01.583599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T19:15:20.314127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.2f","version":"0'0","reported_seq":9,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-11-24T07:58:02.627097+0000","last_change":"2023-11-24T07:58:02.623672+0000","last_active":"2023-11-24T07:58:02.627097+0000","last_peered":"2023-11-24T07:58:02.627097+0000","last_clean":"2023-11-24T07:58:02.627097+0000","last_became_active":"2023-11-24T07:58:02.623260+0000","last_became_peered":"2023-11-24T07:58:02.623260+0000","last_unstale":"2023-11-24T07:58:02.627097+0000","last_undegraded":"2023-11-24T07:58:02.627097+0000","last_fullsized":"2023-11-24T07:58:02.627097+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:58:01.583599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:58:01.583599+0000","last_clean_scrub_stamp":"2023-11-24T07:58:01.583599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T14:01:26.773202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.2e","version":"0'0","reported_seq":9,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-11-24T07:58:02.627834+0000","last_change":"2023-11-24T07:58:02.623591+0000","last_active":"2023-11-24T07:58:02.627834+0000","last_peered":"2023-11-24T07:58:02.627834+0000","last_clean":"2023-11-24T07:58:02.627834+0000","last_became_active":"2023-11-24T07:58:02.620591+0000","last_became_peered":"2023-11-24T07:58:02.620591+0000","last_unstale":"2023-11-24T07:58:02.627834+0000","last_undegraded":"2023-11-24T07:58:02.627834+0000","last_fullsized":"2023-11-24T07:58:02.627834+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:58:01.583599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:58:01.583599+0000","last_clean_scrub_stamp":"2023-11-24T07:58:01.583599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T09:33:31.355159+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.2d","version":"0'0","reported_seq":9,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-11-24T07:58:02.630725+0000","last_change":"2023-11-24T07:58:02.626896+0000","last_active":"2023-11-24T07:58:02.630725+0000","last_peered":"2023-11-24T07:58:02.630725+0000","last_clean":"2023-11-24T07:58:02.630725+0000","last_became_active":"2023-11-24T07:58:02.626316+0000","last_became_peered":"2023-11-24T07:58:02.626316+0000","last_unstale":"2023-11-24T07:58:02.630725+0000","last_undegraded":"2023-11-24T07:58:02.630725+0000","last_fullsized":"2023-11-24T07:58:02.630725+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:58:01.583599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:58:01.583599+0000","last_clean_scrub_stamp":"2023-11-24T07:58:01.583599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:33:23.934909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.2c","version":"0'0","reported_seq":9,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-11-24T07:58:02.623652+0000","last_change":"2023-11-24T07:58:02.621073+0000","last_active":"2023-11-24T07:58:02.623652+0000","last_peered":"2023-11-24T07:58:02.623652+0000","last_clean":"2023-11-24T07:58:02.623652+0000","last_became_active":"2023-11-24T07:58:02.620377+0000","last_became_peered":"2023-11-24T07:58:02.620377+0000","last_unstale":"2023-11-24T07:58:02.623652+0000","last_undegraded":"2023-11-24T07:58:02.623652+0000","last_fullsized":"2023-11-24T07:58:02.623652+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:58:01.583599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:58:01.583599+0000","last_clean_scrub_stamp":"2023-11-24T07:58:01.583599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T13:56:01.401441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.2b","version":"0'0","reported_seq":9,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-11-24T07:58:02.623552+0000","last_change":"2023-11-24T07:58:02.616504+0000","last_active":"2023-11-24T07:58:02.623552+0000","last_peered":"2023-11-24T07:58:02.623552+0000","last_clean":"2023-11-24T07:58:02.623552+0000","last_became_active":"2023-11-24T07:58:02.616198+0000","last_became_peered":"2023-11-24T07:58:02.616198+0000","last_unstale":"2023-11-24T07:58:02.623552+0000","last_undegraded":"2023-11-24T07:58:02.623552+0000","last_fullsized":"2023-11-24T07:58:02.623552+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:58:01.583599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:58:01.583599+0000","last_clean_scrub_stamp":"2023-11-24T07:58:01.583599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:15:47.666890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.2a","version":"0'0","reported_seq":9,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-11-24T07:58:02.626853+0000","last_change":"2023-11-24T07:58:02.623172+0000","last_active":"2023-11-24T07:58:02.626853+0000","last_peered":"2023-11-24T07:58:02.626853+0000","last_clean":"2023-11-24T07:58:02.626853+0000","last_became_active":"2023-11-24T07:58:02.622843+0000","last_became_peered":"2023-11-24T07:58:02.622843+0000","last_unstale":"2023-11-24T07:58:02.626853+0000","last_undegraded":"2023-11-24T07:58:02.626853+0000","last_fullsized":"2023-11-24T07:58:02.626853+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:58:01.583599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:58:01.583599+0000","last_clean_scrub_stamp":"2023-11-24T07:58:01.583599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T09:33:31.355159+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.29","version":"0'0","reported_seq":9,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-11-24T07:58:02.629086+0000","last_change":"2023-11-24T07:58:02.619184+0000","last_active":"2023-11-24T07:58:02.629086+0000","last_peered":"2023-11-24T07:58:02.629086+0000","last_clean":"2023-11-24T07:58:02.629086+0000","last_became_active":"2023-11-24T07:58:02.616552+0000","last_became_peered":"2023-11-24T07:58:02.616552+0000","last_unstale":"2023-11-24T07:58:02.629086+0000","last_undegraded":"2023-11-24T07:58:02.629086+0000","last_fullsized":"2023-11-24T07:58:02.629086+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:58:01.583599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:58:01.583599+0000","last_clean_scrub_stamp":"2023-11-24T07:58:01.583599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T18:22:49.761771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.28","version":"0'0","reported_seq":9,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-11-24T07:58:02.622139+0000","last_change":"2023-11-24T07:58:02.616088+0000","last_active":"2023-11-24T07:58:02.622139+0000","last_peered":"2023-11-24T07:58:02.622139+0000","last_clean":"2023-11-24T07:58:02.622139+0000","last_became_active":"2023-11-24T07:58:02.615784+0000","last_became_peered":"2023-11-24T07:58:02.615784+0000","last_unstale":"2023-11-24T07:58:02.622139+0000","last_undegraded":"2023-11-24T07:58:02.622139+0000","last_fullsized":"2023-11-24T07:58:02.622139+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:58:01.583599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:58:01.583599+0000","last_clean_scrub_stamp":"2023-11-24T07:58:01.583599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:37:31.802505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.27","version":"0'0","reported_seq":9,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-11-24T07:58:02.625843+0000","last_change":"2023-11-24T07:58:02.619687+0000","last_active":"2023-11-24T07:58:02.625843+0000","last_peered":"2023-11-24T07:58:02.625843+0000","last_clean":"2023-11-24T07:58:02.625843+0000","last_became_active":"2023-11-24T07:58:02.618980+0000","last_became_peered":"2023-11-24T07:58:02.618980+0000","last_unstale":"2023-11-24T07:58:02.625843+0000","last_undegraded":"2023-11-24T07:58:02.625843+0000","last_fullsized":"2023-11-24T07:58:02.625843+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:58:01.583599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:58:01.583599+0000","last_clean_scrub_stamp":"2023-11-24T07:58:01.583599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:15:47.666890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.26","version":"0'0","reported_seq":9,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-11-24T07:58:02.613528+0000","last_change":"2023-11-24T07:58:02.608890+0000","last_active":"2023-11-24T07:58:02.613528+0000","last_peered":"2023-11-24T07:58:02.613528+0000","last_clean":"2023-11-24T07:58:02.613528+0000","last_became_active":"2023-11-24T07:58:02.608591+0000","last_became_peered":"2023-11-24T07:58:02.608591+0000","last_unstale":"2023-11-24T07:58:02.613528+0000","last_undegraded":"2023-11-24T07:58:02.613528+0000","last_fullsized":"2023-11-24T07:58:02.613528+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:58:01.583599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:58:01.583599+0000","last_clean_scrub_stamp":"2023-11-24T07:58:01.583599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T17:09:27.205967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.25","version":"0'0","reported_seq":9,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-11-24T07:58:02.631166+0000","last_change":"2023-11-24T07:58:02.629031+0000","last_active":"2023-11-24T07:58:02.631166+0000","last_peered":"2023-11-24T07:58:02.631166+0000","last_clean":"2023-11-24T07:58:02.631166+0000","last_became_active":"2023-11-24T07:58:02.628329+0000","last_became_peered":"2023-11-24T07:58:02.628329+0000","last_unstale":"2023-11-24T07:58:02.631166+0000","last_undegraded":"2023-11-24T07:58:02.631166+0000","last_fullsized":"2023-11-24T07:58:02.631166+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:58:01.583599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:58:01.583599+0000","last_clean_scrub_stamp":"2023-11-24T07:58:01.583599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T18:09:46.006718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.24","version":"0'0","reported_seq":9,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-11-24T07:58:02.629006+0000","last_change":"2023-11-24T07:58:02.617827+0000","last_active":"2023-11-24T07:58:02.629006+0000","last_peered":"2023-11-24T07:58:02.629006+0000","last_clean":"2023-11-24T07:58:02.629006+0000","last_became_active":"2023-11-24T07:58:02.617508+0000","last_became_peered":"2023-11-24T07:58:02.617508+0000","last_unstale":"2023-11-24T07:58:02.629006+0000","last_undegraded":"2023-11-24T07:58:02.629006+0000","last_fullsized":"2023-11-24T07:58:02.629006+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:58:01.583599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:58:01.583599+0000","last_clean_scrub_stamp":"2023-11-24T07:58:01.583599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T16:17:54.841970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.23","version":"0'0","reported_seq":9,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-11-24T07:58:02.625719+0000","last_change":"2023-11-24T07:58:02.621025+0000","last_active":"2023-11-24T07:58:02.625719+0000","last_peered":"2023-11-24T07:58:02.625719+0000","last_clean":"2023-11-24T07:58:02.625719+0000","last_became_active":"2023-11-24T07:58:02.620751+0000","last_became_peered":"2023-11-24T07:58:02.620751+0000","last_unstale":"2023-11-24T07:58:02.625719+0000","last_undegraded":"2023-11-24T07:58:02.625719+0000","last_fullsized":"2023-11-24T07:58:02.625719+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:58:01.583599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:58:01.583599+0000","last_clean_scrub_stamp":"2023-11-24T07:58:01.583599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T17:09:27.205967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.22","version":"0'0","reported_seq":9,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-11-24T07:58:02.621295+0000","last_change":"2023-11-24T07:58:02.611961+0000","last_active":"2023-11-24T07:58:02.621295+0000","last_peered":"2023-11-24T07:58:02.621295+0000","last_clean":"2023-11-24T07:58:02.621295+0000","last_became_active":"2023-11-24T07:58:02.611684+0000","last_became_peered":"2023-11-24T07:58:02.611684+0000","last_unstale":"2023-11-24T07:58:02.621295+0000","last_undegraded":"2023-11-24T07:58:02.621295+0000","last_fullsized":"2023-11-24T07:58:02.621295+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:58:01.583599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:58:01.583599+0000","last_clean_scrub_stamp":"2023-11-24T07:58:01.583599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T14:21:21.611502+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.21","version":"0'0","reported_seq":9,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-11-24T07:58:02.620762+0000","last_change":"2023-11-24T07:58:02.615770+0000","last_active":"2023-11-24T07:58:02.620762+0000","last_peered":"2023-11-24T07:58:02.620762+0000","last_clean":"2023-11-24T07:58:02.620762+0000","last_became_active":"2023-11-24T07:58:02.615464+0000","last_became_peered":"2023-11-24T07:58:02.615464+0000","last_unstale":"2023-11-24T07:58:02.620762+0000","last_undegraded":"2023-11-24T07:58:02.620762+0000","last_fullsized":"2023-11-24T07:58:02.620762+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:58:01.583599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:58:01.583599+0000","last_clean_scrub_stamp":"2023-11-24T07:58:01.583599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T14:21:21.611502+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.20","version":"0'0","reported_seq":9,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-11-24T07:58:02.613719+0000","last_change":"2023-11-24T07:58:02.611123+0000","last_active":"2023-11-24T07:58:02.613719+0000","last_peered":"2023-11-24T07:58:02.613719+0000","last_clean":"2023-11-24T07:58:02.613719+0000","last_became_active":"2023-11-24T07:58:02.610844+0000","last_became_peered":"2023-11-24T07:58:02.610844+0000","last_unstale":"2023-11-24T07:58:02.613719+0000","last_undegraded":"2023-11-24T07:58:02.613719+0000","last_fullsized":"2023-11-24T07:58:02.613719+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:58:01.583599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:58:01.583599+0000","last_clean_scrub_stamp":"2023-11-24T07:58:01.583599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:37:31.802505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.1e","version":"0'0","reported_seq":9,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-11-24T07:58:02.626648+0000","last_change":"2023-11-24T07:58:02.620653+0000","last_active":"2023-11-24T07:58:02.626648+0000","last_peered":"2023-11-24T07:58:02.626648+0000","last_clean":"2023-11-24T07:58:02.626648+0000","last_became_active":"2023-11-24T07:58:02.620141+0000","last_became_peered":"2023-11-24T07:58:02.620141+0000","last_unstale":"2023-11-24T07:58:02.626648+0000","last_undegraded":"2023-11-24T07:58:02.626648+0000","last_fullsized":"2023-11-24T07:58:02.626648+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:58:01.583599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:58:01.583599+0000","last_clean_scrub_stamp":"2023-11-24T07:58:01.583599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T14:04:40.032987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.1f","version":"44'4","reported_seq":15,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-11-24T07:58:02.632057+0000","last_change":"2023-11-24T07:58:01.611939+0000","last_active":"2023-11-24T07:58:02.632057+0000","last_peered":"2023-11-24T07:58:02.632057+0000","last_clean":"2023-11-24T07:58:02.632057+0000","last_became_active":"2023-11-24T07:58:01.611667+0000","last_became_peered":"2023-11-24T07:58:01.611667+0000","last_unstale":"2023-11-24T07:58:02.632057+0000","last_undegraded":"2023-11-24T07:58:02.632057+0000","last_fullsized":"2023-11-24T07:58:02.632057+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:58:00.578147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:58:00.578147+0000","last_clean_scrub_stamp":"2023-11-24T07:58:00.578147+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T15:36:22.152707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.1f","version":"0'0","reported_seq":9,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-11-24T07:58:02.619780+0000","last_change":"2023-11-24T07:58:02.613596+0000","last_active":"2023-11-24T07:58:02.619780+0000","last_peered":"2023-11-24T07:58:02.619780+0000","last_clean":"2023-11-24T07:58:02.619780+0000","last_became_active":"2023-11-24T07:58:02.613309+0000","last_became_peered":"2023-11-24T07:58:02.613309+0000","last_unstale":"2023-11-24T07:58:02.619780+0000","last_undegraded":"2023-11-24T07:58:02.619780+0000","last_fullsized":"2023-11-24T07:58:02.619780+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:58:01.583599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:58:01.583599+0000","last_clean_scrub_stamp":"2023-11-24T07:58:01.583599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:37:31.802505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1e","version":"44'1","reported_seq":12,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-11-24T07:58:02.628853+0000","last_change":"2023-11-24T07:58:01.610596+0000","last_active":"2023-11-24T07:58:02.628853+0000","last_peered":"2023-11-24T07:58:02.628853+0000","last_clean":"2023-11-24T07:58:02.628853+0000","last_became_active":"2023-11-24T07:58:01.610311+0000","last_became_peered":"2023-11-24T07:58:01.610311+0000","last_unstale":"2023-11-24T07:58:02.628853+0000","last_undegraded":"2023-11-24T07:58:02.628853+0000","last_fullsized":"2023-11-24T07:58:02.628853+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:58:00.578147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:58:00.578147+0000","last_clean_scrub_stamp":"2023-11-24T07:58:00.578147+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T08:58:19.378878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1c","version":"0'0","reported_seq":9,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-11-24T07:58:02.629438+0000","last_change":"2023-11-24T07:58:02.624739+0000","last_active":"2023-11-24T07:58:02.629438+0000","last_peered":"2023-11-24T07:58:02.629438+0000","last_clean":"2023-11-24T07:58:02.629438+0000","last_became_active":"2023-11-24T07:58:02.624458+0000","last_became_peered":"2023-11-24T07:58:02.624458+0000","last_unstale":"2023-11-24T07:58:02.629438+0000","last_undegraded":"2023-11-24T07:58:02.629438+0000","last_fullsized":"2023-11-24T07:58:02.629438+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:58:01.583599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:58:01.583599+0000","last_clean_scrub_stamp":"2023-11-24T07:58:01.583599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T13:42:52.916962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.1d","version":"44'2","reported_seq":13,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-11-24T07:58:02.631053+0000","last_change":"2023-11-24T07:58:01.611179+0000","last_active":"2023-11-24T07:58:02.631053+0000","last_peered":"2023-11-24T07:58:02.631053+0000","last_clean":"2023-11-24T07:58:02.631053+0000","last_became_active":"2023-11-24T07:58:01.610856+0000","last_became_peered":"2023-11-24T07:58:01.610856+0000","last_unstale":"2023-11-24T07:58:02.631053+0000","last_undegraded":"2023-11-24T07:58:02.631053+0000","last_fullsized":"2023-11-24T07:58:02.631053+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:58:00.578147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:58:00.578147+0000","last_clean_scrub_stamp":"2023-11-24T07:58:00.578147+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:15:46.661438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.1d","version":"0'0","reported_seq":9,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-11-24T07:58:02.617365+0000","last_change":"2023-11-24T07:58:02.612602+0000","last_active":"2023-11-24T07:58:02.617365+0000","last_peered":"2023-11-24T07:58:02.617365+0000","last_clean":"2023-11-24T07:58:02.617365+0000","last_became_active":"2023-11-24T07:58:02.612296+0000","last_became_peered":"2023-11-24T07:58:02.612296+0000","last_unstale":"2023-11-24T07:58:02.617365+0000","last_undegraded":"2023-11-24T07:58:02.617365+0000","last_fullsized":"2023-11-24T07:58:02.617365+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:58:01.583599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:58:01.583599+0000","last_clean_scrub_stamp":"2023-11-24T07:58:01.583599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T19:15:20.314127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1c","version":"44'2","reported_seq":13,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-11-24T07:58:02.629970+0000","last_change":"2023-11-24T07:58:01.611406+0000","last_active":"2023-11-24T07:58:02.629970+0000","last_peered":"2023-11-24T07:58:02.629970+0000","last_clean":"2023-11-24T07:58:02.629970+0000","last_became_active":"2023-11-24T07:58:01.611065+0000","last_became_peered":"2023-11-24T07:58:01.611065+0000","last_unstale":"2023-11-24T07:58:02.629970+0000","last_undegraded":"2023-11-24T07:58:02.629970+0000","last_fullsized":"2023-11-24T07:58:02.629970+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:58:00.578147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:58:00.578147+0000","last_clean_scrub_stamp":"2023-11-24T07:58:00.578147+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T12:38:07.521171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1a","version":"0'0","reported_seq":9,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-11-24T07:58:02.631154+0000","last_change":"2023-11-24T07:58:02.628830+0000","last_active":"2023-11-24T07:58:02.631154+0000","last_peered":"2023-11-24T07:58:02.631154+0000","last_clean":"2023-11-24T07:58:02.631154+0000","last_became_active":"2023-11-24T07:58:02.628529+0000","last_became_peered":"2023-11-24T07:58:02.628529+0000","last_unstale":"2023-11-24T07:58:02.631154+0000","last_undegraded":"2023-11-24T07:58:02.631154+0000","last_fullsized":"2023-11-24T07:58:02.631154+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:58:01.583599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:58:01.583599+0000","last_clean_scrub_stamp":"2023-11-24T07:58:01.583599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T08:49:16.225604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.1b","version":"44'1","reported_seq":12,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-11-24T07:58:02.629635+0000","last_change":"2023-11-24T07:58:01.611305+0000","last_active":"2023-11-24T07:58:02.629635+0000","last_peered":"2023-11-24T07:58:02.629635+0000","last_clean":"2023-11-24T07:58:02.629635+0000","last_became_active":"2023-11-24T07:58:01.610994+0000","last_became_peered":"2023-11-24T07:58:01.610994+0000","last_unstale":"2023-11-24T07:58:02.629635+0000","last_undegraded":"2023-11-24T07:58:02.629635+0000","last_fullsized":"2023-11-24T07:58:02.629635+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:58:00.578147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:58:00.578147+0000","last_clean_scrub_stamp":"2023-11-24T07:58:00.578147+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T13:42:51.911510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.1b","version":"0'0","reported_seq":9,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-11-24T07:58:02.623818+0000","last_change":"2023-11-24T07:58:02.620015+0000","last_active":"2023-11-24T07:58:02.623818+0000","last_peered":"2023-11-24T07:58:02.623818+0000","last_clean":"2023-11-24T07:58:02.623818+0000","last_became_active":"2023-11-24T07:58:02.619716+0000","last_became_peered":"2023-11-24T07:58:02.619716+0000","last_unstale":"2023-11-24T07:58:02.623818+0000","last_undegraded":"2023-11-24T07:58:02.623818+0000","last_fullsized":"2023-11-24T07:58:02.623818+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:58:01.583599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:58:01.583599+0000","last_clean_scrub_stamp":"2023-11-24T07:58:01.583599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T11:01:48.657488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1a","version":"44'1","reported_seq":12,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-11-24T07:58:02.629139+0000","last_change":"2023-11-24T07:58:01.609468+0000","last_active":"2023-11-24T07:58:02.629139+0000","last_peered":"2023-11-24T07:58:02.629139+0000","last_clean":"2023-11-24T07:58:02.629139+0000","last_became_active":"2023-11-24T07:58:01.609163+0000","last_became_peered":"2023-11-24T07:58:01.609163+0000","last_unstale":"2023-11-24T07:58:02.629139+0000","last_undegraded":"2023-11-24T07:58:02.629139+0000","last_fullsized":"2023-11-24T07:58:02.629139+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:58:00.578147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:58:00.578147+0000","last_clean_scrub_stamp":"2023-11-24T07:58:00.578147+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T16:30:16.725810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.18","version":"0'0","reported_seq":9,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-11-24T07:58:02.623221+0000","last_change":"2023-11-24T07:58:02.617452+0000","last_active":"2023-11-24T07:58:02.623221+0000","last_peered":"2023-11-24T07:58:02.623221+0000","last_clean":"2023-11-24T07:58:02.623221+0000","last_became_active":"2023-11-24T07:58:02.617158+0000","last_became_peered":"2023-11-24T07:58:02.617158+0000","last_unstale":"2023-11-24T07:58:02.623221+0000","last_undegraded":"2023-11-24T07:58:02.623221+0000","last_fullsized":"2023-11-24T07:58:02.623221+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:58:01.583599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:58:01.583599+0000","last_clean_scrub_stamp":"2023-11-24T07:58:01.583599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T12:50:53.080554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":11,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-11-24T07:58:02.607146+0000","last_change":"2023-11-24T07:58:01.612644+0000","last_active":"2023-11-24T07:58:02.607146+0000","last_peered":"2023-11-24T07:58:02.607146+0000","last_clean":"2023-11-24T07:58:02.607146+0000","last_became_active":"2023-11-24T07:58:01.612301+0000","last_became_peered":"2023-11-24T07:58:01.612301+0000","last_unstale":"2023-11-24T07:58:02.607146+0000","last_undegraded":"2023-11-24T07:58:02.607146+0000","last_fullsized":"2023-11-24T07:58:02.607146+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:58:00.578147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:58:00.578147+0000","last_clean_scrub_stamp":"2023-11-24T07:58:00.578147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T12:08:52.701016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.19","version":"0'0","reported_seq":9,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-11-24T07:58:02.611313+0000","last_change":"2023-11-24T07:58:02.608850+0000","last_active":"2023-11-24T07:58:02.611313+0000","last_peered":"2023-11-24T07:58:02.611313+0000","last_clean":"2023-11-24T07:58:02.611313+0000","last_became_active":"2023-11-24T07:58:02.608563+0000","last_became_peered":"2023-11-24T07:58:02.608563+0000","last_unstale":"2023-11-24T07:58:02.611313+0000","last_undegraded":"2023-11-24T07:58:02.611313+0000","last_fullsized":"2023-11-24T07:58:02.611313+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:58:01.583599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:58:01.583599+0000","last_clean_scrub_stamp":"2023-11-24T07:58:01.583599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T16:49:00.649513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":11,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-11-24T07:58:02.605880+0000","last_change":"2023-11-24T07:58:01.616438+0000","last_active":"2023-11-24T07:58:02.605880+0000","last_peered":"2023-11-24T07:58:02.605880+0000","last_clean":"2023-11-24T07:58:02.605880+0000","last_became_active":"2023-11-24T07:58:01.616159+0000","last_became_peered":"2023-11-24T07:58:01.616159+0000","last_unstale":"2023-11-24T07:58:02.605880+0000","last_undegraded":"2023-11-24T07:58:02.605880+0000","last_fullsized":"2023-11-24T07:58:02.605880+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:58:00.578147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:58:00.578147+0000","last_clean_scrub_stamp":"2023-11-24T07:58:00.578147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T19:44:06.756960+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.16","version":"0'0","reported_seq":9,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-11-24T07:58:02.623685+0000","last_change":"2023-11-24T07:58:02.613182+0000","last_active":"2023-11-24T07:58:02.623685+0000","last_peered":"2023-11-24T07:58:02.623685+0000","last_clean":"2023-11-24T07:58:02.623685+0000","last_became_active":"2023-11-24T07:58:02.612863+0000","last_became_peered":"2023-11-24T07:58:02.612863+0000","last_unstale":"2023-11-24T07:58:02.623685+0000","last_undegraded":"2023-11-24T07:58:02.623685+0000","last_fullsized":"2023-11-24T07:58:02.623685+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:58:01.583599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:58:01.583599+0000","last_clean_scrub_stamp":"2023-11-24T07:58:01.583599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T12:59:32.870352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":11,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-11-24T07:58:02.607285+0000","last_change":"2023-11-24T07:58:01.614068+0000","last_active":"2023-11-24T07:58:02.607285+0000","last_peered":"2023-11-24T07:58:02.607285+0000","last_clean":"2023-11-24T07:58:02.607285+0000","last_became_active":"2023-11-24T07:58:01.613346+0000","last_became_peered":"2023-11-24T07:58:01.613346+0000","last_unstale":"2023-11-24T07:58:02.607285+0000","last_undegraded":"2023-11-24T07:58:02.607285+0000","last_fullsized":"2023-11-24T07:58:02.607285+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:58:00.578147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:58:00.578147+0000","last_clean_scrub_stamp":"2023-11-24T07:58:00.578147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T15:33:14.634070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.17","version":"0'0","reported_seq":9,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-11-24T07:58:02.627730+0000","last_change":"2023-11-24T07:58:02.623316+0000","last_active":"2023-11-24T07:58:02.627730+0000","last_peered":"2023-11-24T07:58:02.627730+0000","last_clean":"2023-11-24T07:58:02.627730+0000","last_became_active":"2023-11-24T07:58:02.623035+0000","last_became_peered":"2023-11-24T07:58:02.623035+0000","last_unstale":"2023-11-24T07:58:02.627730+0000","last_undegraded":"2023-11-24T07:58:02.627730+0000","last_fullsized":"2023-11-24T07:58:02.627730+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:58:01.583599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:58:01.583599+0000","last_clean_scrub_stamp":"2023-11-24T07:58:01.583599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T12:51:50.368636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":11,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-11-24T07:58:02.612463+0000","last_change":"2023-11-24T07:58:01.616836+0000","last_active":"2023-11-24T07:58:02.612463+0000","last_peered":"2023-11-24T07:58:02.612463+0000","last_clean":"2023-11-24T07:58:02.612463+0000","last_became_active":"2023-11-24T07:58:01.616552+0000","last_became_peered":"2023-11-24T07:58:01.616552+0000","last_unstale":"2023-11-24T07:58:02.612463+0000","last_undegraded":"2023-11-24T07:58:02.612463+0000","last_fullsized":"2023-11-24T07:58:02.612463+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:58:00.578147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:58:00.578147+0000","last_clean_scrub_stamp":"2023-11-24T07:58:00.578147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T15:43:05.511574+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.14","version":"0'0","reported_seq":9,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-11-24T07:58:02.630915+0000","last_change":"2023-11-24T07:58:02.624262+0000","last_active":"2023-11-24T07:58:02.630915+0000","last_peered":"2023-11-24T07:58:02.630915+0000","last_clean":"2023-11-24T07:58:02.630915+0000","last_became_active":"2023-11-24T07:58:02.623994+0000","last_became_peered":"2023-11-24T07:58:02.623994+0000","last_unstale":"2023-11-24T07:58:02.630915+0000","last_undegraded":"2023-11-24T07:58:02.630915+0000","last_fullsized":"2023-11-24T07:58:02.630915+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:58:01.583599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:58:01.583599+0000","last_clean_scrub_stamp":"2023-11-24T07:58:01.583599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T13:34:36.854213+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.15","version":"44'3","reported_seq":14,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-11-24T07:58:02.630987+0000","last_change":"2023-11-24T07:58:01.608162+0000","last_active":"2023-11-24T07:58:02.630987+0000","last_peered":"2023-11-24T07:58:02.630987+0000","last_clean":"2023-11-24T07:58:02.630987+0000","last_became_active":"2023-11-24T07:58:01.607860+0000","last_became_peered":"2023-11-24T07:58:01.607860+0000","last_unstale":"2023-11-24T07:58:02.630987+0000","last_undegraded":"2023-11-24T07:58:02.630987+0000","last_fullsized":"2023-11-24T07:58:02.630987+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:58:00.578147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:58:00.578147+0000","last_clean_scrub_stamp":"2023-11-24T07:58:00.578147+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T19:15:19.308675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.15","version":"0'0","reported_seq":9,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-11-24T07:58:02.620370+0000","last_change":"2023-11-24T07:58:02.614576+0000","last_active":"2023-11-24T07:58:02.620370+0000","last_peered":"2023-11-24T07:58:02.620370+0000","last_clean":"2023-11-24T07:58:02.620370+0000","last_became_active":"2023-11-24T07:58:02.614305+0000","last_became_peered":"2023-11-24T07:58:02.614305+0000","last_unstale":"2023-11-24T07:58:02.620370+0000","last_undegraded":"2023-11-24T07:58:02.620370+0000","last_fullsized":"2023-11-24T07:58:02.620370+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:58:01.583599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:58:01.583599+0000","last_clean_scrub_stamp":"2023-11-24T07:58:01.583599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T13:17:31.956955+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.14","version":"44'6","reported_seq":17,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-11-24T07:58:02.633381+0000","last_change":"2023-11-24T07:58:01.612668+0000","last_active":"2023-11-24T07:58:02.633381+0000","last_peered":"2023-11-24T07:58:02.633381+0000","last_clean":"2023-11-24T07:58:02.633381+0000","last_became_active":"2023-11-24T07:58:01.612221+0000","last_became_peered":"2023-11-24T07:58:01.612221+0000","last_unstale":"2023-11-24T07:58:02.633381+0000","last_undegraded":"2023-11-24T07:58:02.633381+0000","last_fullsized":"2023-11-24T07:58:02.633381+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:58:00.578147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:58:00.578147+0000","last_clean_scrub_stamp":"2023-11-24T07:58:00.578147+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T19:40:21.190531+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.12","version":"0'0","reported_seq":9,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-11-24T07:58:02.624400+0000","last_change":"2023-11-24T07:58:02.620890+0000","last_active":"2023-11-24T07:58:02.624400+0000","last_peered":"2023-11-24T07:58:02.624400+0000","last_clean":"2023-11-24T07:58:02.624400+0000","last_became_active":"2023-11-24T07:58:02.620596+0000","last_became_peered":"2023-11-24T07:58:02.620596+0000","last_unstale":"2023-11-24T07:58:02.624400+0000","last_undegraded":"2023-11-24T07:58:02.624400+0000","last_fullsized":"2023-11-24T07:58:02.624400+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:58:01.583599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:58:01.583599+0000","last_clean_scrub_stamp":"2023-11-24T07:58:01.583599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T19:31:11.664630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.13","version":"44'2","reported_seq":13,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-11-24T07:58:02.624947+0000","last_change":"2023-11-24T07:58:01.614050+0000","last_active":"2023-11-24T07:58:02.624947+0000","last_peered":"2023-11-24T07:58:02.624947+0000","last_clean":"2023-11-24T07:58:02.624947+0000","last_became_active":"2023-11-24T07:58:01.613746+0000","last_became_peered":"2023-11-24T07:58:01.613746+0000","last_unstale":"2023-11-24T07:58:02.624947+0000","last_undegraded":"2023-11-24T07:58:02.624947+0000","last_fullsized":"2023-11-24T07:58:02.624947+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:58:00.578147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:58:00.578147+0000","last_clean_scrub_stamp":"2023-11-24T07:58:00.578147+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T19:14:16.804507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.13","version":"0'0","reported_seq":9,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-11-24T07:58:02.626736+0000","last_change":"2023-11-24T07:58:02.622179+0000","last_active":"2023-11-24T07:58:02.626736+0000","last_peered":"2023-11-24T07:58:02.626736+0000","last_clean":"2023-11-24T07:58:02.626736+0000","last_became_active":"2023-11-24T07:58:02.621869+0000","last_became_peered":"2023-11-24T07:58:02.621869+0000","last_unstale":"2023-11-24T07:58:02.626736+0000","last_undegraded":"2023-11-24T07:58:02.626736+0000","last_fullsized":"2023-11-24T07:58:02.626736+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:58:01.583599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:58:01.583599+0000","last_clean_scrub_stamp":"2023-11-24T07:58:01.583599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T15:33:15.639523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":11,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-11-24T07:58:02.606211+0000","last_change":"2023-11-24T07:58:01.612148+0000","last_active":"2023-11-24T07:58:02.606211+0000","last_peered":"2023-11-24T07:58:02.606211+0000","last_clean":"2023-11-24T07:58:02.606211+0000","last_became_active":"2023-11-24T07:58:01.611797+0000","last_became_peered":"2023-11-24T07:58:01.611797+0000","last_unstale":"2023-11-24T07:58:02.606211+0000","last_undegraded":"2023-11-24T07:58:02.606211+0000","last_fullsized":"2023-11-24T07:58:02.606211+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:58:00.578147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:58:00.578147+0000","last_clean_scrub_stamp":"2023-11-24T07:58:00.578147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T15:39:02.083805+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.10","version":"0'0","reported_seq":9,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-11-24T07:58:02.626842+0000","last_change":"2023-11-24T07:58:02.622907+0000","last_active":"2023-11-24T07:58:02.626842+0000","last_peered":"2023-11-24T07:58:02.626842+0000","last_clean":"2023-11-24T07:58:02.626842+0000","last_became_active":"2023-11-24T07:58:02.622583+0000","last_became_peered":"2023-11-24T07:58:02.622583+0000","last_unstale":"2023-11-24T07:58:02.626842+0000","last_undegraded":"2023-11-24T07:58:02.626842+0000","last_fullsized":"2023-11-24T07:58:02.626842+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:58:01.583599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:58:01.583599+0000","last_clean_scrub_stamp":"2023-11-24T07:58:01.583599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:18:58.981383+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":11,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-11-24T07:58:02.599996+0000","last_change":"2023-11-24T07:58:01.609614+0000","last_active":"2023-11-24T07:58:02.599996+0000","last_peered":"2023-11-24T07:58:02.599996+0000","last_clean":"2023-11-24T07:58:02.599996+0000","last_became_active":"2023-11-24T07:58:01.609377+0000","last_became_peered":"2023-11-24T07:58:01.609377+0000","last_unstale":"2023-11-24T07:58:02.599996+0000","last_undegraded":"2023-11-24T07:58:02.599996+0000","last_fullsized":"2023-11-24T07:58:02.599996+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:58:00.578147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:58:00.578147+0000","last_clean_scrub_stamp":"2023-11-24T07:58:00.578147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:33:31.200726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.11","version":"0'0","reported_seq":9,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-11-24T07:58:02.637977+0000","last_change":"2023-11-24T07:58:02.633597+0000","last_active":"2023-11-24T07:58:02.637977+0000","last_peered":"2023-11-24T07:58:02.637977+0000","last_clean":"2023-11-24T07:58:02.637977+0000","last_became_active":"2023-11-24T07:58:02.633290+0000","last_became_peered":"2023-11-24T07:58:02.633290+0000","last_unstale":"2023-11-24T07:58:02.637977+0000","last_undegraded":"2023-11-24T07:58:02.637977+0000","last_fullsized":"2023-11-24T07:58:02.637977+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:58:01.583599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:58:01.583599+0000","last_clean_scrub_stamp":"2023-11-24T07:58:01.583599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:15:47.666890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.10","version":"44'8","reported_seq":19,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-11-24T07:58:02.627271+0000","last_change":"2023-11-24T07:58:01.610705+0000","last_active":"2023-11-24T07:58:02.627271+0000","last_peered":"2023-11-24T07:58:02.627271+0000","last_clean":"2023-11-24T07:58:02.627271+0000","last_became_active":"2023-11-24T07:58:01.610408+0000","last_became_peered":"2023-11-24T07:58:01.610408+0000","last_unstale":"2023-11-24T07:58:02.627271+0000","last_undegraded":"2023-11-24T07:58:02.627271+0000","last_fullsized":"2023-11-24T07:58:02.627271+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:58:00.578147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:58:00.578147+0000","last_clean_scrub_stamp":"2023-11-24T07:58:00.578147+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T16:22:38.421798+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.e","version":"0'0","reported_seq":9,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-11-24T07:58:02.620754+0000","last_change":"2023-11-24T07:58:02.614815+0000","last_active":"2023-11-24T07:58:02.620754+0000","last_peered":"2023-11-24T07:58:02.620754+0000","last_clean":"2023-11-24T07:58:02.620754+0000","last_became_active":"2023-11-24T07:58:02.614519+0000","last_became_peered":"2023-11-24T07:58:02.614519+0000","last_unstale":"2023-11-24T07:58:02.620754+0000","last_undegraded":"2023-11-24T07:58:02.620754+0000","last_fullsized":"2023-11-24T07:58:02.620754+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:58:01.583599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:58:01.583599+0000","last_clean_scrub_stamp":"2023-11-24T07:58:01.583599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:35:34.232935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.f","version":"44'1","reported_seq":12,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-11-24T07:58:02.623851+0000","last_change":"2023-11-24T07:58:01.613289+0000","last_active":"2023-11-24T07:58:02.623851+0000","last_peered":"2023-11-24T07:58:02.623851+0000","last_clean":"2023-11-24T07:58:02.623851+0000","last_became_active":"2023-11-24T07:58:01.612961+0000","last_became_peered":"2023-11-24T07:58:01.612961+0000","last_unstale":"2023-11-24T07:58:02.623851+0000","last_undegraded":"2023-11-24T07:58:02.623851+0000","last_fullsized":"2023-11-24T07:58:02.623851+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:58:00.578147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:58:00.578147+0000","last_clean_scrub_stamp":"2023-11-24T07:58:00.578147+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T13:42:51.911510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.f","version":"0'0","reported_seq":9,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-11-24T07:58:02.626620+0000","last_change":"2023-11-24T07:58:02.620464+0000","last_active":"2023-11-24T07:58:02.626620+0000","last_peered":"2023-11-24T07:58:02.626620+0000","last_clean":"2023-11-24T07:58:02.626620+0000","last_became_active":"2023-11-24T07:58:02.620187+0000","last_became_peered":"2023-11-24T07:58:02.620187+0000","last_unstale":"2023-11-24T07:58:02.626620+0000","last_undegraded":"2023-11-24T07:58:02.626620+0000","last_fullsized":"2023-11-24T07:58:02.626620+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:58:01.583599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:58:01.583599+0000","last_clean_scrub_stamp":"2023-11-24T07:58:01.583599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T15:47:49.136575+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.e","version":"44'2","reported_seq":13,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-11-24T07:58:02.630782+0000","last_change":"2023-11-24T07:58:01.615918+0000","last_active":"2023-11-24T07:58:02.630782+0000","last_peered":"2023-11-24T07:58:02.630782+0000","last_clean":"2023-11-24T07:58:02.630782+0000","last_became_active":"2023-11-24T07:58:01.615196+0000","last_became_peered":"2023-11-24T07:58:01.615196+0000","last_unstale":"2023-11-24T07:58:02.630782+0000","last_undegraded":"2023-11-24T07:58:02.630782+0000","last_fullsized":"2023-11-24T07:58:02.630782+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:58:00.578147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:58:00.578147+0000","last_clean_scrub_stamp":"2023-11-24T07:58:00.578147+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T15:47:48.131123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.c","version":"0'0","reported_seq":9,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-11-24T07:58:02.624213+0000","last_change":"2023-11-24T07:58:02.616927+0000","last_active":"2023-11-24T07:58:02.624213+0000","last_peered":"2023-11-24T07:58:02.624213+0000","last_clean":"2023-11-24T07:58:02.624213+0000","last_became_active":"2023-11-24T07:58:02.615155+0000","last_became_peered":"2023-11-24T07:58:02.615155+0000","last_unstale":"2023-11-24T07:58:02.624213+0000","last_undegraded":"2023-11-24T07:58:02.624213+0000","last_fullsized":"2023-11-24T07:58:02.624213+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:58:01.583599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:58:01.583599+0000","last_clean_scrub_stamp":"2023-11-24T07:58:01.583599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:33:32.206178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.d","version":"44'1","reported_seq":12,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-11-24T07:58:02.631456+0000","last_change":"2023-11-24T07:58:01.612501+0000","last_active":"2023-11-24T07:58:02.631456+0000","last_peered":"2023-11-24T07:58:02.631456+0000","last_clean":"2023-11-24T07:58:02.631456+0000","last_became_active":"2023-11-24T07:58:01.612212+0000","last_became_peered":"2023-11-24T07:58:01.612212+0000","last_unstale":"2023-11-24T07:58:02.631456+0000","last_undegraded":"2023-11-24T07:58:02.631456+0000","last_fullsized":"2023-11-24T07:58:02.631456+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:58:00.578147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:58:00.578147+0000","last_clean_scrub_stamp":"2023-11-24T07:58:00.578147+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T13:39:52.527067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.d","version":"0'0","reported_seq":9,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-11-24T07:58:02.627645+0000","last_change":"2023-11-24T07:58:02.622388+0000","last_active":"2023-11-24T07:58:02.627645+0000","last_peered":"2023-11-24T07:58:02.627645+0000","last_clean":"2023-11-24T07:58:02.627645+0000","last_became_active":"2023-11-24T07:58:02.622040+0000","last_became_peered":"2023-11-24T07:58:02.622040+0000","last_unstale":"2023-11-24T07:58:02.627645+0000","last_undegraded":"2023-11-24T07:58:02.627645+0000","last_fullsized":"2023-11-24T07:58:02.627645+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:58:01.583599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:58:01.583599+0000","last_clean_scrub_stamp":"2023-11-24T07:58:01.583599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T14:04:40.032987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":11,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-11-24T07:58:02.599718+0000","last_change":"2023-11-24T07:58:01.614016+0000","last_active":"2023-11-24T07:58:02.599718+0000","last_peered":"2023-11-24T07:58:02.599718+0000","last_clean":"2023-11-24T07:58:02.599718+0000","last_became_active":"2023-11-24T07:58:01.613757+0000","last_became_peered":"2023-11-24T07:58:01.613757+0000","last_unstale":"2023-11-24T07:58:02.599718+0000","last_undegraded":"2023-11-24T07:58:02.599718+0000","last_fullsized":"2023-11-24T07:58:02.599718+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:58:00.578147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:58:00.578147+0000","last_clean_scrub_stamp":"2023-11-24T07:58:00.578147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T11:44:15.085353+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.a","version":"0'0","reported_seq":9,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-11-24T07:58:02.623272+0000","last_change":"2023-11-24T07:58:02.617926+0000","last_active":"2023-11-24T07:58:02.623272+0000","last_peered":"2023-11-24T07:58:02.623272+0000","last_clean":"2023-11-24T07:58:02.623272+0000","last_became_active":"2023-11-24T07:58:02.616846+0000","last_became_peered":"2023-11-24T07:58:02.616846+0000","last_unstale":"2023-11-24T07:58:02.623272+0000","last_undegraded":"2023-11-24T07:58:02.623272+0000","last_fullsized":"2023-11-24T07:58:02.623272+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:58:01.583599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:58:01.583599+0000","last_clean_scrub_stamp":"2023-11-24T07:58:01.583599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T18:57:31.529962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.b","version":"44'2","reported_seq":13,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-11-24T07:58:02.623730+0000","last_change":"2023-11-24T07:58:01.613024+0000","last_active":"2023-11-24T07:58:02.623730+0000","last_peered":"2023-11-24T07:58:02.623730+0000","last_clean":"2023-11-24T07:58:02.623730+0000","last_became_active":"2023-11-24T07:58:01.612716+0000","last_became_peered":"2023-11-24T07:58:01.612716+0000","last_unstale":"2023-11-24T07:58:02.623730+0000","last_undegraded":"2023-11-24T07:58:02.623730+0000","last_fullsized":"2023-11-24T07:58:02.623730+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:58:00.578147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:58:00.578147+0000","last_clean_scrub_stamp":"2023-11-24T07:58:00.578147+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T18:51:00.570901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.b","version":"0'0","reported_seq":9,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-11-24T07:58:02.624565+0000","last_change":"2023-11-24T07:58:02.621881+0000","last_active":"2023-11-24T07:58:02.624565+0000","last_peered":"2023-11-24T07:58:02.624565+0000","last_clean":"2023-11-24T07:58:02.624565+0000","last_became_active":"2023-11-24T07:58:02.621608+0000","last_became_peered":"2023-11-24T07:58:02.621608+0000","last_unstale":"2023-11-24T07:58:02.624565+0000","last_undegraded":"2023-11-24T07:58:02.624565+0000","last_fullsized":"2023-11-24T07:58:02.624565+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:58:01.583599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:58:01.583599+0000","last_clean_scrub_stamp":"2023-11-24T07:58:01.583599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T17:09:27.205967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":11,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-11-24T07:58:02.606570+0000","last_change":"2023-11-24T07:58:01.612723+0000","last_active":"2023-11-24T07:58:02.606570+0000","last_peered":"2023-11-24T07:58:02.606570+0000","last_clean":"2023-11-24T07:58:02.606570+0000","last_became_active":"2023-11-24T07:58:01.612426+0000","last_became_peered":"2023-11-24T07:58:01.612426+0000","last_unstale":"2023-11-24T07:58:02.606570+0000","last_undegraded":"2023-11-24T07:58:02.606570+0000","last_fullsized":"2023-11-24T07:58:02.606570+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:58:00.578147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:58:00.578147+0000","last_clean_scrub_stamp":"2023-11-24T07:58:00.578147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T13:34:35.848761+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.8","version":"0'0","reported_seq":9,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-11-24T07:58:02.624739+0000","last_change":"2023-11-24T07:58:02.622212+0000","last_active":"2023-11-24T07:58:02.624739+0000","last_peered":"2023-11-24T07:58:02.624739+0000","last_clean":"2023-11-24T07:58:02.624739+0000","last_became_active":"2023-11-24T07:58:02.620808+0000","last_became_peered":"2023-11-24T07:58:02.620808+0000","last_unstale":"2023-11-24T07:58:02.624739+0000","last_undegraded":"2023-11-24T07:58:02.624739+0000","last_fullsized":"2023-11-24T07:58:02.624739+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:58:01.583599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:58:01.583599+0000","last_clean_scrub_stamp":"2023-11-24T07:58:01.583599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T08:32:51.874274+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":11,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-11-24T07:58:02.606256+0000","last_change":"2023-11-24T07:58:01.613942+0000","last_active":"2023-11-24T07:58:02.606256+0000","last_peered":"2023-11-24T07:58:02.606256+0000","last_clean":"2023-11-24T07:58:02.606256+0000","last_became_active":"2023-11-24T07:58:01.613658+0000","last_became_peered":"2023-11-24T07:58:01.613658+0000","last_unstale":"2023-11-24T07:58:02.606256+0000","last_undegraded":"2023-11-24T07:58:02.606256+0000","last_fullsized":"2023-11-24T07:58:02.606256+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:58:00.578147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:58:00.578147+0000","last_clean_scrub_stamp":"2023-11-24T07:58:00.578147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:05:04.154724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.9","version":"0'0","reported_seq":9,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-11-24T07:58:02.623717+0000","last_change":"2023-11-24T07:58:02.618190+0000","last_active":"2023-11-24T07:58:02.623717+0000","last_peered":"2023-11-24T07:58:02.623717+0000","last_clean":"2023-11-24T07:58:02.623717+0000","last_became_active":"2023-11-24T07:58:02.617899+0000","last_became_peered":"2023-11-24T07:58:02.617899+0000","last_unstale":"2023-11-24T07:58:02.623717+0000","last_undegraded":"2023-11-24T07:58:02.623717+0000","last_fullsized":"2023-11-24T07:58:02.623717+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:58:01.583599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:58:01.583599+0000","last_clean_scrub_stamp":"2023-11-24T07:58:01.583599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:33:23.934909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":11,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-11-24T07:58:02.599487+0000","last_change":"2023-11-24T07:58:01.615893+0000","last_active":"2023-11-24T07:58:02.599487+0000","last_peered":"2023-11-24T07:58:02.599487+0000","last_clean":"2023-11-24T07:58:02.599487+0000","last_became_active":"2023-11-24T07:58:01.615604+0000","last_became_peered":"2023-11-24T07:58:01.615604+0000","last_unstale":"2023-11-24T07:58:02.599487+0000","last_undegraded":"2023-11-24T07:58:02.599487+0000","last_fullsized":"2023-11-24T07:58:02.599487+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:58:00.578147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:58:00.578147+0000","last_clean_scrub_stamp":"2023-11-24T07:58:00.578147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:51:57.585934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.7","version":"0'0","reported_seq":9,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-11-24T07:58:02.610851+0000","last_change":"2023-11-24T07:58:02.607936+0000","last_active":"2023-11-24T07:58:02.610851+0000","last_peered":"2023-11-24T07:58:02.610851+0000","last_clean":"2023-11-24T07:58:02.610851+0000","last_became_active":"2023-11-24T07:58:02.607648+0000","last_became_peered":"2023-11-24T07:58:02.607648+0000","last_unstale":"2023-11-24T07:58:02.610851+0000","last_undegraded":"2023-11-24T07:58:02.610851+0000","last_fullsized":"2023-11-24T07:58:02.610851+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:58:01.583599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:58:01.583599+0000","last_clean_scrub_stamp":"2023-11-24T07:58:01.583599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T16:56:20.684081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":11,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-11-24T07:58:02.601145+0000","last_change":"2023-11-24T07:58:01.615430+0000","last_active":"2023-11-24T07:58:02.601145+0000","last_peered":"2023-11-24T07:58:02.601145+0000","last_clean":"2023-11-24T07:58:02.601145+0000","last_became_active":"2023-11-24T07:58:01.615077+0000","last_became_peered":"2023-11-24T07:58:01.615077+0000","last_unstale":"2023-11-24T07:58:02.601145+0000","last_undegraded":"2023-11-24T07:58:02.601145+0000","last_fullsized":"2023-11-24T07:58:02.601145+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:58:00.578147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:58:00.578147+0000","last_clean_scrub_stamp":"2023-11-24T07:58:00.578147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T15:39:02.083805+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.4","version":"0'0","reported_seq":9,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-11-24T07:58:02.631919+0000","last_change":"2023-11-24T07:58:02.628793+0000","last_active":"2023-11-24T07:58:02.631919+0000","last_peered":"2023-11-24T07:58:02.631919+0000","last_clean":"2023-11-24T07:58:02.631919+0000","last_became_active":"2023-11-24T07:58:02.628501+0000","last_became_peered":"2023-11-24T07:58:02.628501+0000","last_unstale":"2023-11-24T07:58:02.631919+0000","last_undegraded":"2023-11-24T07:58:02.631919+0000","last_fullsized":"2023-11-24T07:58:02.631919+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:58:01.583599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:58:01.583599+0000","last_clean_scrub_stamp":"2023-11-24T07:58:01.583599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:04:31.376151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":11,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-11-24T07:58:02.606575+0000","last_change":"2023-11-24T07:58:01.614161+0000","last_active":"2023-11-24T07:58:02.606575+0000","last_peered":"2023-11-24T07:58:02.606575+0000","last_clean":"2023-11-24T07:58:02.606575+0000","last_became_active":"2023-11-24T07:58:01.613866+0000","last_became_peered":"2023-11-24T07:58:01.613866+0000","last_unstale":"2023-11-24T07:58:02.606575+0000","last_undegraded":"2023-11-24T07:58:02.606575+0000","last_fullsized":"2023-11-24T07:58:02.606575+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:58:00.578147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:58:00.578147+0000","last_clean_scrub_stamp":"2023-11-24T07:58:00.578147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T08:32:50.868822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.5","version":"0'0","reported_seq":9,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-11-24T07:58:02.619899+0000","last_change":"2023-11-24T07:58:02.612665+0000","last_active":"2023-11-24T07:58:02.619899+0000","last_peered":"2023-11-24T07:58:02.619899+0000","last_clean":"2023-11-24T07:58:02.619899+0000","last_became_active":"2023-11-24T07:58:02.612194+0000","last_became_peered":"2023-11-24T07:58:02.612194+0000","last_unstale":"2023-11-24T07:58:02.619899+0000","last_undegraded":"2023-11-24T07:58:02.619899+0000","last_fullsized":"2023-11-24T07:58:02.619899+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:58:01.583599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:58:01.583599+0000","last_clean_scrub_stamp":"2023-11-24T07:58:01.583599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T17:00:36.111442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":11,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-11-24T07:58:02.606398+0000","last_change":"2023-11-24T07:58:01.613918+0000","last_active":"2023-11-24T07:58:02.606398+0000","last_peered":"2023-11-24T07:58:02.606398+0000","last_clean":"2023-11-24T07:58:02.606398+0000","last_became_active":"2023-11-24T07:58:01.613574+0000","last_became_peered":"2023-11-24T07:58:01.613574+0000","last_unstale":"2023-11-24T07:58:02.606398+0000","last_undegraded":"2023-11-24T07:58:02.606398+0000","last_fullsized":"2023-11-24T07:58:02.606398+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:58:00.578147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:58:00.578147+0000","last_clean_scrub_stamp":"2023-11-24T07:58:00.578147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T12:51:49.363183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.2","version":"0'0","reported_seq":9,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-11-24T07:58:02.622649+0000","last_change":"2023-11-24T07:58:02.617105+0000","last_active":"2023-11-24T07:58:02.622649+0000","last_peered":"2023-11-24T07:58:02.622649+0000","last_clean":"2023-11-24T07:58:02.622649+0000","last_became_active":"2023-11-24T07:58:02.616811+0000","last_became_peered":"2023-11-24T07:58:02.616811+0000","last_unstale":"2023-11-24T07:58:02.622649+0000","last_undegraded":"2023-11-24T07:58:02.622649+0000","last_fullsized":"2023-11-24T07:58:02.622649+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:58:01.583599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:58:01.583599+0000","last_clean_scrub_stamp":"2023-11-24T07:58:01.583599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T08:32:51.874274+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":11,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-11-24T07:58:02.607514+0000","last_change":"2023-11-24T07:58:01.615697+0000","last_active":"2023-11-24T07:58:02.607514+0000","last_peered":"2023-11-24T07:58:02.607514+0000","last_clean":"2023-11-24T07:58:02.607514+0000","last_became_active":"2023-11-24T07:58:01.615335+0000","last_became_peered":"2023-11-24T07:58:01.615335+0000","last_unstale":"2023-11-24T07:58:02.607514+0000","last_undegraded":"2023-11-24T07:58:02.607514+0000","last_fullsized":"2023-11-24T07:58:02.607514+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:58:00.578147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:58:00.578147+0000","last_clean_scrub_stamp":"2023-11-24T07:58:00.578147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T12:50:52.075101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.3","version":"0'0","reported_seq":9,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-11-24T07:58:02.623529+0000","last_change":"2023-11-24T07:58:02.616483+0000","last_active":"2023-11-24T07:58:02.623529+0000","last_peered":"2023-11-24T07:58:02.623529+0000","last_clean":"2023-11-24T07:58:02.623529+0000","last_became_active":"2023-11-24T07:58:02.616088+0000","last_became_peered":"2023-11-24T07:58:02.616088+0000","last_unstale":"2023-11-24T07:58:02.623529+0000","last_undegraded":"2023-11-24T07:58:02.623529+0000","last_fullsized":"2023-11-24T07:58:02.623529+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:58:01.583599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:58:01.583599+0000","last_clean_scrub_stamp":"2023-11-24T07:58:01.583599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T14:02:23.676273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":11,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-11-24T07:58:02.600594+0000","last_change":"2023-11-24T07:58:01.612303+0000","last_active":"2023-11-24T07:58:02.600594+0000","last_peered":"2023-11-24T07:58:02.600594+0000","last_clean":"2023-11-24T07:58:02.600594+0000","last_became_active":"2023-11-24T07:58:01.612020+0000","last_became_peered":"2023-11-24T07:58:01.612020+0000","last_unstale":"2023-11-24T07:58:02.600594+0000","last_undegraded":"2023-11-24T07:58:02.600594+0000","last_fullsized":"2023-11-24T07:58:02.600594+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:58:00.578147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:58:00.578147+0000","last_clean_scrub_stamp":"2023-11-24T07:58:00.578147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T18:54:06.222117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.0","version":"0'0","reported_seq":9,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-11-24T07:58:02.624371+0000","last_change":"2023-11-24T07:58:02.621445+0000","last_active":"2023-11-24T07:58:02.624371+0000","last_peered":"2023-11-24T07:58:02.624371+0000","last_clean":"2023-11-24T07:58:02.624371+0000","last_became_active":"2023-11-24T07:58:02.621111+0000","last_became_peered":"2023-11-24T07:58:02.621111+0000","last_unstale":"2023-11-24T07:58:02.624371+0000","last_undegraded":"2023-11-24T07:58:02.624371+0000","last_fullsized":"2023-11-24T07:58:02.624371+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:58:01.583599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:58:01.583599+0000","last_clean_scrub_stamp":"2023-11-24T07:58:01.583599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T19:14:17.809960+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":11,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-11-24T07:58:02.599844+0000","last_change":"2023-11-24T07:58:01.612062+0000","last_active":"2023-11-24T07:58:02.599844+0000","last_peered":"2023-11-24T07:58:02.599844+0000","last_clean":"2023-11-24T07:58:02.599844+0000","last_became_active":"2023-11-24T07:58:01.611754+0000","last_became_peered":"2023-11-24T07:58:01.611754+0000","last_unstale":"2023-11-24T07:58:02.599844+0000","last_undegraded":"2023-11-24T07:58:02.599844+0000","last_fullsized":"2023-11-24T07:58:02.599844+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:58:00.578147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:58:00.578147+0000","last_clean_scrub_stamp":"2023-11-24T07:58:00.578147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T15:27:34.075869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.1","version":"0'0","reported_seq":9,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-11-24T07:58:02.622921+0000","last_change":"2023-11-24T07:58:02.615550+0000","last_active":"2023-11-24T07:58:02.622921+0000","last_peered":"2023-11-24T07:58:02.622921+0000","last_clean":"2023-11-24T07:58:02.622921+0000","last_became_active":"2023-11-24T07:58:02.615098+0000","last_became_peered":"2023-11-24T07:58:02.615098+0000","last_unstale":"2023-11-24T07:58:02.622921+0000","last_undegraded":"2023-11-24T07:58:02.622921+0000","last_fullsized":"2023-11-24T07:58:02.622921+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:58:01.583599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:58:01.583599+0000","last_clean_scrub_stamp":"2023-11-24T07:58:01.583599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T17:09:27.205967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":11,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-11-24T07:58:02.607581+0000","last_change":"2023-11-24T07:58:01.609433+0000","last_active":"2023-11-24T07:58:02.607581+0000","last_peered":"2023-11-24T07:58:02.607581+0000","last_clean":"2023-11-24T07:58:02.607581+0000","last_became_active":"2023-11-24T07:58:01.609189+0000","last_became_peered":"2023-11-24T07:58:01.609189+0000","last_unstale":"2023-11-24T07:58:02.607581+0000","last_undegraded":"2023-11-24T07:58:02.607581+0000","last_fullsized":"2023-11-24T07:58:02.607581+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:58:00.578147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:58:00.578147+0000","last_clean_scrub_stamp":"2023-11-24T07:58:00.578147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T13:39:52.527067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.6","version":"0'0","reported_seq":9,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-11-24T07:58:02.624714+0000","last_change":"2023-11-24T07:58:02.622461+0000","last_active":"2023-11-24T07:58:02.624714+0000","last_peered":"2023-11-24T07:58:02.624714+0000","last_clean":"2023-11-24T07:58:02.624714+0000","last_became_active":"2023-11-24T07:58:02.621899+0000","last_became_peered":"2023-11-24T07:58:02.621899+0000","last_unstale":"2023-11-24T07:58:02.624714+0000","last_undegraded":"2023-11-24T07:58:02.624714+0000","last_fullsized":"2023-11-24T07:58:02.624714+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:58:01.583599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:58:01.583599+0000","last_clean_scrub_stamp":"2023-11-24T07:58:01.583599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:33:32.206178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.7","version":"44'2","reported_seq":13,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-11-24T07:58:02.630044+0000","last_change":"2023-11-24T07:58:01.616492+0000","last_active":"2023-11-24T07:58:02.630044+0000","last_peered":"2023-11-24T07:58:02.630044+0000","last_clean":"2023-11-24T07:58:02.630044+0000","last_became_active":"2023-11-24T07:58:01.616200+0000","last_became_peered":"2023-11-24T07:58:01.616200+0000","last_unstale":"2023-11-24T07:58:02.630044+0000","last_undegraded":"2023-11-24T07:58:02.630044+0000","last_fullsized":"2023-11-24T07:58:02.630044+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:58:00.578147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:58:00.578147+0000","last_clean_scrub_stamp":"2023-11-24T07:58:00.578147+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:02:45.169861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":184,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-11-24T07:58:01.607889+0000","last_change":"2023-11-24T07:56:19.668905+0000","last_active":"2023-11-24T07:58:01.607889+0000","last_peered":"2023-11-24T07:58:01.607889+0000","last_clean":"2023-11-24T07:58:01.607889+0000","last_became_active":"2023-11-24T07:56:19.668601+0000","last_became_peered":"2023-11-24T07:56:19.668601+0000","last_unstale":"2023-11-24T07:58:01.607889+0000","last_undegraded":"2023-11-24T07:58:01.607889+0000","last_fullsized":"2023-11-24T07:58:01.607889+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:56:18.657844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:56:18.657844+0000","last_clean_scrub_stamp":"2023-11-24T07:56:18.657844+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T11:51:53.828455+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":7,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574872,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27536,"kb_used_data":508,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344304,"statfs":{"total":96636764160,"available":96608567296,"internally_reserved":0,"allocated":520192,"data_stored":208347,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 07:57:19 2023","interfaces":[{"interface":"back","average":{"1min":0.55700000000000005,"5min":0.55700000000000005,"15min":0.55700000000000005},"min":{"1min":0.32800000000000001,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":1.413,"5min":1.413,"15min":1.413},"last":0.65500000000000003},{"interface":"front","average":{"1min":0.55400000000000005,"5min":0.55400000000000005,"15min":0.55400000000000005},"min":{"1min":0.41099999999999998,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":0.81000000000000005,"5min":0.81000000000000005,"15min":0.81000000000000005},"last":0.626}]},{"osd":1,"last update":"Fri Nov 24 07:57:19 2023","interfaces":[{"interface":"back","average":{"1min":0.84899999999999998,"5min":0.84899999999999998,"15min":0.84899999999999998},"min":{"1min":0.33600000000000002,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":6.1070000000000002,"5min":6.1070000000000002,"15min":6.1070000000000002},"last":0.47599999999999998},{"interface":"front","average":{"1min":0.57999999999999996,"5min":0.57999999999999996,"15min":0.57999999999999996},"min":{"1min":0.35199999999999998,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":1.5920000000000001,"5min":1.5920000000000001,"15min":1.5920000000000001},"last":0.43099999999999999}]},{"osd":2,"last update":"Fri Nov 24 07:57:19 2023","interfaces":[{"interface":"back","average":{"1min":0.627,"5min":0.627,"15min":0.627},"min":{"1min":0.33800000000000002,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":1.613,"5min":1.613,"15min":1.613},"last":0.60499999999999998},{"interface":"front","average":{"1min":0.79200000000000004,"5min":0.79200000000000004,"15min":0.79200000000000004},"min":{"1min":0.41699999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":4.5220000000000002,"5min":4.5220000000000002,"15min":4.5220000000000002},"last":0.45800000000000002}]},{"osd":3,"last update":"Fri Nov 24 07:57:19 2023","interfaces":[{"interface":"back","average":{"1min":0.80600000000000005,"5min":0.80600000000000005,"15min":0.80600000000000005},"min":{"1min":0.38500000000000001,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":4.6760000000000002,"5min":4.6760000000000002,"15min":4.6760000000000002},"last":0.53800000000000003},{"interface":"front","average":{"1min":0.94299999999999995,"5min":0.94299999999999995,"15min":0.94299999999999995},"min":{"1min":0.505,"5min":0.505,"15min":0.505},"max":{"1min":4.6539999999999999,"5min":4.6539999999999999,"15min":4.6539999999999999},"last":0.68899999999999995}]},{"osd":4,"last update":"Fri Nov 24 07:57:19 2023","interfaces":[{"interface":"back","average":{"1min":0.56799999999999995,"5min":0.56799999999999995,"15min":0.56799999999999995},"min":{"1min":0.378,"5min":0.378,"15min":0.378},"max":{"1min":0.84699999999999998,"5min":0.84699999999999998,"15min":0.84699999999999998},"last":0.58399999999999996},{"interface":"front","average":{"1min":0.66700000000000004,"5min":0.66700000000000004,"15min":0.66700000000000004},"min":{"1min":0.46100000000000002,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":1.629,"5min":1.629,"15min":1.629},"last":0.52200000000000002}]},{"osd":5,"last update":"Fri Nov 24 07:57:19 2023","interfaces":[{"interface":"back","average":{"1min":0.54700000000000004,"5min":0.54700000000000004,"15min":0.54700000000000004},"min":{"1min":0.34499999999999997,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.875,"5min":0.875,"15min":0.875},"last":0.50600000000000001},{"interface":"front","average":{"1min":0.623,"5min":0.623,"15min":0.623},"min":{"1min":0.41499999999999998,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":0.90900000000000003,"5min":0.90900000000000003,"15min":0.90900000000000003},"last":0.67400000000000004}]},{"osd":6,"last update":"Fri Nov 24 07:57:19 2023","interfaces":[{"interface":"back","average":{"1min":0.61099999999999999,"5min":0.61099999999999999,"15min":0.61099999999999999},"min":{"1min":0.438,"5min":0.438,"15min":0.438},"max":{"1min":0.80800000000000005,"5min":0.80800000000000005,"15min":0.80800000000000005},"last":0.55200000000000005},{"interface":"front","average":{"1min":0.67000000000000004,"5min":0.67000000000000004,"15min":0.67000000000000004},"min":{"1min":0.48599999999999999,"5min":0.48599999999999999,"15min":0.48599999999999999},"max":{"1min":0.98399999999999999,"5min":0.98399999999999999,"15min":0.98399999999999999},"last":0.63700000000000001}]}]},{"osd":6,"up_from":13,"seq":55834574873,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27540,"kb_used_data":512,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344300,"statfs":{"total":96636764160,"available":96608563200,"internally_reserved":0,"allocated":524288,"data_stored":208929,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 07:57:18 2023","interfaces":[{"interface":"back","average":{"1min":0.55000000000000004,"5min":0.55000000000000004,"15min":0.55000000000000004},"min":{"1min":0.28399999999999997,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":0.85099999999999998,"5min":0.85099999999999998,"15min":0.85099999999999998},"last":0.73899999999999999},{"interface":"front","average":{"1min":0.51600000000000001,"5min":0.51600000000000001,"15min":0.51600000000000001},"min":{"1min":0.311,"5min":0.311,"15min":0.311},"max":{"1min":0.69999999999999996,"5min":0.69999999999999996,"15min":0.69999999999999996},"last":0.67200000000000004}]},{"osd":1,"last update":"Fri Nov 24 07:57:18 2023","interfaces":[{"interface":"back","average":{"1min":0.51200000000000001,"5min":0.51200000000000001,"15min":0.51200000000000001},"min":{"1min":0.33800000000000002,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.78200000000000003,"5min":0.78200000000000003,"15min":0.78200000000000003},"last":0.51100000000000001},{"interface":"front","average":{"1min":0.61299999999999999,"5min":0.61299999999999999,"15min":0.61299999999999999},"min":{"1min":0.44700000000000001,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":1.0069999999999999,"5min":1.0069999999999999,"15min":1.0069999999999999},"last":0.76700000000000002}]},{"osd":2,"last update":"Fri Nov 24 07:57:18 2023","interfaces":[{"interface":"back","average":{"1min":0.57499999999999996,"5min":0.57499999999999996,"15min":0.57499999999999996},"min":{"1min":0.34300000000000003,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.878,"5min":0.878,"15min":0.878},"last":0.71499999999999997},{"interface":"front","average":{"1min":0.59299999999999997,"5min":0.59299999999999997,"15min":0.59299999999999997},"min":{"1min":0.47099999999999997,"5min":0.47099999999999997,"15min":0.47099999999999997},"max":{"1min":0.82399999999999995,"5min":0.82399999999999995,"15min":0.82399999999999995},"last":0.56299999999999994}]},{"osd":3,"last update":"Fri Nov 24 07:57:18 2023","interfaces":[{"interface":"back","average":{"1min":0.56799999999999995,"5min":0.56799999999999995,"15min":0.56799999999999995},"min":{"1min":0.40000000000000002,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":0.876,"5min":0.876,"15min":0.876},"last":0.54500000000000004},{"interface":"front","average":{"1min":0.55600000000000005,"5min":0.55600000000000005,"15min":0.55600000000000005},"min":{"1min":0.379,"5min":0.379,"15min":0.379},"max":{"1min":0.68799999999999994,"5min":0.68799999999999994,"15min":0.68799999999999994},"last":0.60299999999999998}]},{"osd":4,"last update":"Fri Nov 24 07:57:18 2023","interfaces":[{"interface":"back","average":{"1min":0.51700000000000002,"5min":0.51700000000000002,"15min":0.51700000000000002},"min":{"1min":0.33000000000000002,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.70599999999999996,"5min":0.70599999999999996,"15min":0.70599999999999996},"last":0.36699999999999999},{"interface":"front","average":{"1min":0.60199999999999998,"5min":0.60199999999999998,"15min":0.60199999999999998},"min":{"1min":0.35199999999999998,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":1.087,"5min":1.087,"15min":1.087},"last":0.48599999999999999}]},{"osd":5,"last update":"Fri Nov 24 07:57:18 2023","interfaces":[{"interface":"back","average":{"1min":0.53100000000000003,"5min":0.53100000000000003,"15min":0.53100000000000003},"min":{"1min":0.30199999999999999,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.84999999999999998,"5min":0.84999999999999998,"15min":0.84999999999999998},"last":0.78200000000000003},{"interface":"front","average":{"1min":0.61199999999999999,"5min":0.61199999999999999,"15min":0.61199999999999999},"min":{"1min":0.39200000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.97199999999999998,"5min":0.97199999999999998,"15min":0.97199999999999998},"last":0.65100000000000002}]},{"osd":7,"last update":"Fri Nov 24 07:57:18 2023","interfaces":[{"interface":"back","average":{"1min":0.61799999999999999,"5min":0.61799999999999999,"15min":0.61799999999999999},"min":{"1min":0.49299999999999999,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":0.81499999999999995,"5min":0.81499999999999995,"15min":0.81499999999999995},"last":0.63600000000000001},{"interface":"front","average":{"1min":0.67000000000000004,"5min":0.67000000000000004,"15min":0.67000000000000004},"min":{"1min":0.497,"5min":0.497,"15min":0.497},"max":{"1min":0.92700000000000005,"5min":0.92700000000000005,"15min":0.92700000000000005},"last":0.81999999999999995}]}]},{"osd":1,"up_from":13,"seq":55834574874,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28096,"kb_used_data":1072,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343744,"statfs":{"total":96636764160,"available":96607993856,"internally_reserved":0,"allocated":1097728,"data_stored":791906,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 07:57:22 2023","interfaces":[{"interface":"back","average":{"1min":0.36399999999999999,"5min":0.36399999999999999,"15min":0.36399999999999999},"min":{"1min":0.20200000000000001,"5min":0.20200000000000001,"15min":0.20200000000000001},"max":{"1min":0.70899999999999996,"5min":0.70899999999999996,"15min":0.70899999999999996},"last":0.29099999999999998},{"interface":"front","average":{"1min":0.39800000000000002,"5min":0.39800000000000002,"15min":0.39800000000000002},"min":{"1min":0.214,"5min":0.214,"15min":0.214},"max":{"1min":0.82499999999999996,"5min":0.82499999999999996,"15min":0.82499999999999996},"last":0.30299999999999999}]},{"osd":2,"last update":"Fri Nov 24 07:57:22 2023","interfaces":[{"interface":"back","average":{"1min":0.373,"5min":0.373,"15min":0.373},"min":{"1min":0.22700000000000001,"5min":0.22700000000000001,"15min":0.22700000000000001},"max":{"1min":0.60999999999999999,"5min":0.60999999999999999,"15min":0.60999999999999999},"last":0.26700000000000002},{"interface":"front","average":{"1min":0.41299999999999998,"5min":0.41299999999999998,"15min":0.41299999999999998},"min":{"1min":0.26800000000000002,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":0.621,"5min":0.621,"15min":0.621},"last":0.35799999999999998}]},{"osd":3,"last update":"Fri Nov 24 07:57:22 2023","interfaces":[{"interface":"back","average":{"1min":0.44400000000000001,"5min":0.44400000000000001,"15min":0.44400000000000001},"min":{"1min":0.32100000000000001,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.66500000000000004,"5min":0.66500000000000004,"15min":0.66500000000000004},"last":0.379},{"interface":"front","average":{"1min":0.42799999999999999,"5min":0.42799999999999999,"15min":0.42799999999999999},"min":{"1min":0.28499999999999998,"5min":0.28499999999999998,"15min":0.28499999999999998},"max":{"1min":0.76300000000000001,"5min":0.76300000000000001,"15min":0.76300000000000001},"last":0.34599999999999997}]},{"osd":4,"last update":"Fri Nov 24 07:57:22 2023","interfaces":[{"interface":"back","average":{"1min":0.61099999999999999,"5min":0.61099999999999999,"15min":0.61099999999999999},"min":{"1min":0.43099999999999999,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":0.79800000000000004,"5min":0.79800000000000004,"15min":0.79800000000000004},"last":0.53700000000000003},{"interface":"front","average":{"1min":0.60599999999999998,"5min":0.60599999999999998,"15min":0.60599999999999998},"min":{"1min":0.41999999999999998,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":0.82199999999999995,"5min":0.82199999999999995,"15min":0.82199999999999995},"last":0.58999999999999997}]},{"osd":5,"last update":"Fri Nov 24 07:57:22 2023","interfaces":[{"interface":"back","average":{"1min":0.72499999999999998,"5min":0.72499999999999998,"15min":0.72499999999999998},"min":{"1min":0.37,"5min":0.37,"15min":0.37},"max":{"1min":2.7269999999999999,"5min":2.7269999999999999,"15min":2.7269999999999999},"last":0.439},{"interface":"front","average":{"1min":0.64500000000000002,"5min":0.64500000000000002,"15min":0.64500000000000002},"min":{"1min":0.44700000000000001,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":0.84199999999999997,"5min":0.84199999999999997,"15min":0.84199999999999997},"last":0.40899999999999997}]},{"osd":6,"last update":"Fri Nov 24 07:57:22 2023","interfaces":[{"interface":"back","average":{"1min":0.65800000000000003,"5min":0.65800000000000003,"15min":0.65800000000000003},"min":{"1min":0.47499999999999998,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":0.98199999999999998,"5min":0.98199999999999998,"15min":0.98199999999999998},"last":0.52200000000000002},{"interface":"front","average":{"1min":0.61699999999999999,"5min":0.61699999999999999,"15min":0.61699999999999999},"min":{"1min":0.42899999999999999,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":0.90700000000000003,"5min":0.90700000000000003,"15min":0.90700000000000003},"last":0.55100000000000005}]},{"osd":7,"last update":"Fri Nov 24 07:57:22 2023","interfaces":[{"interface":"back","average":{"1min":0.63700000000000001,"5min":0.63700000000000001,"15min":0.63700000000000001},"min":{"1min":0.40999999999999998,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":1.0649999999999999,"5min":1.0649999999999999,"15min":1.0649999999999999},"last":0.60099999999999998},{"interface":"front","average":{"1min":0.67000000000000004,"5min":0.67000000000000004,"15min":0.67000000000000004},"min":{"1min":0.442,"5min":0.442,"15min":0.442},"max":{"1min":0.88800000000000001,"5min":0.88800000000000001,"15min":0.88800000000000001},"last":0.504}]}]},{"osd":0,"up_from":13,"seq":55834574874,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27512,"kb_used_data":492,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344328,"statfs":{"total":96636764160,"available":96608591872,"internally_reserved":0,"allocated":503808,"data_stored":201538,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Nov 24 07:57:18 2023","interfaces":[{"interface":"back","average":{"1min":0.71299999999999997,"5min":0.71299999999999997,"15min":0.71299999999999997},"min":{"1min":0.23499999999999999,"5min":0.23499999999999999,"15min":0.23499999999999999},"max":{"1min":6.3550000000000004,"5min":6.3550000000000004,"15min":6.3550000000000004},"last":0.54300000000000004},{"interface":"front","average":{"1min":1.7290000000000001,"5min":1.7290000000000001,"15min":1.7290000000000001},"min":{"1min":0.30599999999999999,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":22.916,"5min":22.916,"15min":22.916},"last":0.60999999999999999}]},{"osd":2,"last update":"Fri Nov 24 07:57:18 2023","interfaces":[{"interface":"back","average":{"1min":0.69999999999999996,"5min":0.69999999999999996,"15min":0.69999999999999996},"min":{"1min":0.22800000000000001,"5min":0.22800000000000001,"15min":0.22800000000000001},"max":{"1min":6.3899999999999997,"5min":6.3899999999999997,"15min":6.3899999999999997},"last":0.47499999999999998},{"interface":"front","average":{"1min":1.0629999999999999,"5min":1.0629999999999999,"15min":1.0629999999999999},"min":{"1min":0.26800000000000002,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":11.93,"5min":11.93,"15min":11.93},"last":0.51700000000000002}]},{"osd":3,"last update":"Fri Nov 24 07:57:18 2023","interfaces":[{"interface":"back","average":{"1min":0.41999999999999998,"5min":0.41999999999999998,"15min":0.41999999999999998},"min":{"1min":0.28399999999999997,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":1.155,"5min":1.155,"15min":1.155},"last":0.56000000000000005},{"interface":"front","average":{"1min":1.127,"5min":1.127,"15min":1.127},"min":{"1min":0.32300000000000001,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":12.116,"5min":12.116,"15min":12.116},"last":0.58599999999999997}]},{"osd":4,"last update":"Fri Nov 24 07:57:18 2023","interfaces":[{"interface":"back","average":{"1min":0.64800000000000002,"5min":0.64800000000000002,"15min":0.64800000000000002},"min":{"1min":0.40500000000000003,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":0.97899999999999998,"5min":0.97899999999999998,"15min":0.97899999999999998},"last":0.82999999999999996},{"interface":"front","average":{"1min":1.9339999999999999,"5min":1.9339999999999999,"15min":1.9339999999999999},"min":{"1min":0.44400000000000001,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":22.963999999999999,"5min":22.963999999999999,"15min":22.963999999999999},"last":0.84599999999999997}]},{"osd":5,"last update":"Fri Nov 24 07:57:18 2023","interfaces":[{"interface":"back","average":{"1min":1.2929999999999999,"5min":1.2929999999999999,"15min":1.2929999999999999},"min":{"1min":0.40200000000000002,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":12.084,"5min":12.084,"15min":12.084},"last":0.86399999999999999},{"interface":"front","average":{"1min":0.67800000000000005,"5min":0.67800000000000005,"15min":0.67800000000000005},"min":{"1min":0.45200000000000001,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":1.073,"5min":1.073,"15min":1.073},"last":0.78800000000000003}]},{"osd":6,"last update":"Fri Nov 24 07:57:18 2023","interfaces":[{"interface":"back","average":{"1min":0.69099999999999995,"5min":0.69099999999999995,"15min":0.69099999999999995},"min":{"1min":0.42799999999999999,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":1.0580000000000001,"5min":1.0580000000000001,"15min":1.0580000000000001},"last":0.93000000000000005},{"interface":"front","average":{"1min":1.3140000000000001,"5min":1.3140000000000001,"15min":1.3140000000000001},"min":{"1min":0.41899999999999998,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":12.045,"5min":12.045,"15min":12.045},"last":0.74199999999999999}]},{"osd":7,"last update":"Fri Nov 24 07:57:18 2023","interfaces":[{"interface":"back","average":{"1min":0.65600000000000003,"5min":0.65600000000000003,"15min":0.65600000000000003},"min":{"1min":0.45400000000000001,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":1.038,"5min":1.038,"15min":1.038},"last":0.88300000000000001},{"interface":"front","average":{"1min":1.9630000000000001,"5min":1.9630000000000001,"15min":1.9630000000000001},"min":{"1min":0.49099999999999999,"5min":0.49099999999999999,"15min":0.49099999999999999},"max":{"1min":22.995000000000001,"5min":22.995000000000001,"15min":22.995000000000001},"last":0.77000000000000002}]}]},{"osd":2,"up_from":13,"seq":55834574874,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27536,"kb_used_data":508,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344304,"statfs":{"total":96636764160,"available":96608567296,"internally_reserved":0,"allocated":520192,"data_stored":208403,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 07:57:17 2023","interfaces":[{"interface":"back","average":{"1min":0.371,"5min":0.371,"15min":0.371},"min":{"1min":0.22500000000000001,"5min":0.22500000000000001,"15min":0.22500000000000001},"max":{"1min":0.58099999999999996,"5min":0.58099999999999996,"15min":0.58099999999999996},"last":0.34699999999999998},{"interface":"front","average":{"1min":0.40200000000000002,"5min":0.40200000000000002,"15min":0.40200000000000002},"min":{"1min":0.27400000000000002,"5min":0.27400000000000002,"15min":0.27400000000000002},"max":{"1min":0.54100000000000004,"5min":0.54100000000000004,"15min":0.54100000000000004},"last":0.33200000000000002}]},{"osd":1,"last update":"Fri Nov 24 07:57:17 2023","interfaces":[{"interface":"back","average":{"1min":0.378,"5min":0.378,"15min":0.378},"min":{"1min":0.23499999999999999,"5min":0.23499999999999999,"15min":0.23499999999999999},"max":{"1min":0.59199999999999997,"5min":0.59199999999999997,"15min":0.59199999999999997},"last":0.29599999999999999},{"interface":"front","average":{"1min":0.41599999999999998,"5min":0.41599999999999998,"15min":0.41599999999999998},"min":{"1min":0.25600000000000001,"5min":0.25600000000000001,"15min":0.25600000000000001},"max":{"1min":0.627,"5min":0.627,"15min":0.627},"last":0.38300000000000001}]},{"osd":3,"last update":"Fri Nov 24 07:57:17 2023","interfaces":[{"interface":"back","average":{"1min":0.41099999999999998,"5min":0.41099999999999998,"15min":0.41099999999999998},"min":{"1min":0.308,"5min":0.308,"15min":0.308},"max":{"1min":0.53300000000000003,"5min":0.53300000000000003,"15min":0.53300000000000003},"last":0.36699999999999999},{"interface":"front","average":{"1min":0.41399999999999998,"5min":0.41399999999999998,"15min":0.41399999999999998},"min":{"1min":0.27500000000000002,"5min":0.27500000000000002,"15min":0.27500000000000002},"max":{"1min":0.64600000000000002,"5min":0.64600000000000002,"15min":0.64600000000000002},"last":0.42199999999999999}]},{"osd":4,"last update":"Fri Nov 24 07:57:17 2023","interfaces":[{"interface":"back","average":{"1min":0.60799999999999998,"5min":0.60799999999999998,"15min":0.60799999999999998},"min":{"1min":0.42999999999999999,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":0.81599999999999995,"5min":0.81599999999999995,"15min":0.81599999999999995},"last":0.54200000000000004},{"interface":"front","average":{"1min":0.59999999999999998,"5min":0.59999999999999998,"15min":0.59999999999999998},"min":{"1min":0.41599999999999998,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":0.79600000000000004,"5min":0.79600000000000004,"15min":0.79600000000000004},"last":0.59499999999999997}]},{"osd":5,"last update":"Fri Nov 24 07:57:17 2023","interfaces":[{"interface":"back","average":{"1min":0.621,"5min":0.621,"15min":0.621},"min":{"1min":0.44900000000000001,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":0.84899999999999998,"5min":0.84899999999999998,"15min":0.84899999999999998},"last":0.51600000000000001},{"interface":"front","average":{"1min":0.71499999999999997,"5min":0.71499999999999997,"15min":0.71499999999999997},"min":{"1min":0.433,"5min":0.433,"15min":0.433},"max":{"1min":2.1699999999999999,"5min":2.1699999999999999,"15min":2.1699999999999999},"last":0.57799999999999996}]},{"osd":6,"last update":"Fri Nov 24 07:57:17 2023","interfaces":[{"interface":"back","average":{"1min":0.66700000000000004,"5min":0.66700000000000004,"15min":0.66700000000000004},"min":{"1min":0.47399999999999998,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":0.83599999999999997,"5min":0.83599999999999997,"15min":0.83599999999999997},"last":0.60599999999999998},{"interface":"front","average":{"1min":0.67100000000000004,"5min":0.67100000000000004,"15min":0.67100000000000004},"min":{"1min":0.496,"5min":0.496,"15min":0.496},"max":{"1min":0.88500000000000001,"5min":0.88500000000000001,"15min":0.88500000000000001},"last":0.5}]},{"osd":7,"last update":"Fri Nov 24 07:57:17 2023","interfaces":[{"interface":"back","average":{"1min":0.66200000000000003,"5min":0.66200000000000003,"15min":0.66200000000000003},"min":{"1min":0.443,"5min":0.443,"15min":0.443},"max":{"1min":0.85999999999999999,"5min":0.85999999999999999,"15min":0.85999999999999999},"last":0.56399999999999995},{"interface":"front","average":{"1min":0.68000000000000005,"5min":0.68000000000000005,"15min":0.68000000000000005},"min":{"1min":0.495,"5min":0.495,"15min":0.495},"max":{"1min":0.86399999999999999,"5min":0.86399999999999999,"15min":0.86399999999999999},"last":0.52900000000000003}]}]},{"osd":3,"up_from":13,"seq":55834574874,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27540,"kb_used_data":516,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344300,"statfs":{"total":96636764160,"available":96608563200,"internally_reserved":0,"allocated":528384,"data_stored":209925,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 07:57:20 2023","interfaces":[{"interface":"back","average":{"1min":0.35599999999999998,"5min":0.35599999999999998,"15min":0.35599999999999998},"min":{"1min":0.222,"5min":0.222,"15min":0.222},"max":{"1min":0.84799999999999998,"5min":0.84799999999999998,"15min":0.84799999999999998},"last":0.371},{"interface":"front","average":{"1min":0.36699999999999999,"5min":0.36699999999999999,"15min":0.36699999999999999},"min":{"1min":0.23699999999999999,"5min":0.23699999999999999,"15min":0.23699999999999999},"max":{"1min":0.873,"5min":0.873,"15min":0.873},"last":0.29099999999999998}]},{"osd":1,"last update":"Fri Nov 24 07:57:20 2023","interfaces":[{"interface":"back","average":{"1min":0.375,"5min":0.375,"15min":0.375},"min":{"1min":0.25700000000000001,"5min":0.25700000000000001,"15min":0.25700000000000001},"max":{"1min":0.92000000000000004,"5min":0.92000000000000004,"15min":0.92000000000000004},"last":0.33100000000000002},{"interface":"front","average":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"min":{"1min":0.248,"5min":0.248,"15min":0.248},"max":{"1min":0.81699999999999995,"5min":0.81699999999999995,"15min":0.81699999999999995},"last":0.35299999999999998}]},{"osd":2,"last update":"Fri Nov 24 07:57:20 2023","interfaces":[{"interface":"back","average":{"1min":0.36099999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"min":{"1min":0.24399999999999999,"5min":0.24399999999999999,"15min":0.24399999999999999},"max":{"1min":0.88500000000000001,"5min":0.88500000000000001,"15min":0.88500000000000001},"last":0.31},{"interface":"front","average":{"1min":0.46800000000000003,"5min":0.46800000000000003,"15min":0.46800000000000003},"min":{"1min":0.31900000000000001,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.98099999999999998,"5min":0.98099999999999998,"15min":0.98099999999999998},"last":0.39300000000000002}]},{"osd":4,"last update":"Fri Nov 24 07:57:20 2023","interfaces":[{"interface":"back","average":{"1min":0.79700000000000004,"5min":0.79700000000000004,"15min":0.79700000000000004},"min":{"1min":0.33500000000000002,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":5.7350000000000003,"5min":5.7350000000000003,"15min":5.7350000000000003},"last":0.65700000000000003},{"interface":"front","average":{"1min":0.58499999999999996,"5min":0.58499999999999996,"15min":0.58499999999999996},"min":{"1min":0.44500000000000001,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":1.27,"5min":1.27,"15min":1.27},"last":0.67900000000000005}]},{"osd":5,"last update":"Fri Nov 24 07:57:20 2023","interfaces":[{"interface":"back","average":{"1min":0.58799999999999997,"5min":0.58799999999999997,"15min":0.58799999999999997},"min":{"1min":0.40699999999999997,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":1.4430000000000001,"5min":1.4430000000000001,"15min":1.4430000000000001},"last":0.52900000000000003},{"interface":"front","average":{"1min":0.627,"5min":0.627,"15min":0.627},"min":{"1min":0.377,"5min":0.377,"15min":0.377},"max":{"1min":1.4239999999999999,"5min":1.4239999999999999,"15min":1.4239999999999999},"last":0.58799999999999997}]},{"osd":6,"last update":"Fri Nov 24 07:57:20 2023","interfaces":[{"interface":"back","average":{"1min":0.62,"5min":0.62,"15min":0.62},"min":{"1min":0.434,"5min":0.434,"15min":0.434},"max":{"1min":1.355,"5min":1.355,"15min":1.355},"last":0.51200000000000001},{"interface":"front","average":{"1min":0.84599999999999997,"5min":0.84599999999999997,"15min":0.84599999999999997},"min":{"1min":0.40999999999999998,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":5.6479999999999997,"5min":5.6479999999999997,"15min":5.6479999999999997},"last":0.59799999999999998}]},{"osd":7,"last update":"Fri Nov 24 07:57:20 2023","interfaces":[{"interface":"back","average":{"1min":0.66600000000000004,"5min":0.66600000000000004,"15min":0.66600000000000004},"min":{"1min":0.42299999999999999,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":1.51,"5min":1.51,"15min":1.51},"last":0.54500000000000004},{"interface":"front","average":{"1min":0.84199999999999997,"5min":0.84199999999999997,"15min":0.84199999999999997},"min":{"1min":0.45600000000000002,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":5.7069999999999999,"5min":5.7069999999999999,"15min":5.7069999999999999},"last":0.63}]}]},{"osd":4,"up_from":13,"seq":55834574873,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27552,"kb_used_data":524,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344288,"statfs":{"total":96636764160,"available":96608550912,"internally_reserved":0,"allocated":536576,"data_stored":210017,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 07:57:21 2023","interfaces":[{"interface":"back","average":{"1min":0.58099999999999996,"5min":0.58099999999999996,"15min":0.58099999999999996},"min":{"1min":0.37,"5min":0.37,"15min":0.37},"max":{"1min":0.79300000000000004,"5min":0.79300000000000004,"15min":0.79300000000000004},"last":0.54700000000000004},{"interface":"front","average":{"1min":0.54300000000000004,"5min":0.54300000000000004,"15min":0.54300000000000004},"min":{"1min":0.33600000000000002,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":1.0249999999999999,"5min":1.0249999999999999,"15min":1.0249999999999999},"last":0.45000000000000001}]},{"osd":1,"last update":"Fri Nov 24 07:57:21 2023","interfaces":[{"interface":"back","average":{"1min":0.55200000000000005,"5min":0.55200000000000005,"15min":0.55200000000000005},"min":{"1min":0.40000000000000002,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":0.86399999999999999,"5min":0.86399999999999999,"15min":0.86399999999999999},"last":0.51800000000000002},{"interface":"front","average":{"1min":0.55100000000000005,"5min":0.55100000000000005,"15min":0.55100000000000005},"min":{"1min":0.36099999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.98699999999999999,"5min":0.98699999999999999,"15min":0.98699999999999999},"last":0.42799999999999999}]},{"osd":2,"last update":"Fri Nov 24 07:57:21 2023","interfaces":[{"interface":"back","average":{"1min":0.56100000000000005,"5min":0.56100000000000005,"15min":0.56100000000000005},"min":{"1min":0.35799999999999998,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.81100000000000005,"5min":0.81100000000000005,"15min":0.81100000000000005},"last":0.629},{"interface":"front","average":{"1min":0.61299999999999999,"5min":0.61299999999999999,"15min":0.61299999999999999},"min":{"1min":0.38100000000000001,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.89200000000000002,"5min":0.89200000000000002,"15min":0.89200000000000002},"last":0.502}]},{"osd":3,"last update":"Fri Nov 24 07:57:21 2023","interfaces":[{"interface":"back","average":{"1min":0.58499999999999996,"5min":0.58499999999999996,"15min":0.58499999999999996},"min":{"1min":0.439,"5min":0.439,"15min":0.439},"max":{"1min":0.89700000000000002,"5min":0.89700000000000002,"15min":0.89700000000000002},"last":0.58799999999999997},{"interface":"front","average":{"1min":0.63100000000000001,"5min":0.63100000000000001,"15min":0.63100000000000001},"min":{"1min":0.40300000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.96199999999999997,"5min":0.96199999999999997,"15min":0.96199999999999997},"last":0.60499999999999998}]},{"osd":5,"last update":"Fri Nov 24 07:57:21 2023","interfaces":[{"interface":"back","average":{"1min":0.45500000000000002,"5min":0.45500000000000002,"15min":0.45500000000000002},"min":{"1min":0.29799999999999999,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":0.70499999999999996,"5min":0.70499999999999996,"15min":0.70499999999999996},"last":0.56999999999999995},{"interface":"front","average":{"1min":0.61899999999999999,"5min":0.61899999999999999,"15min":0.61899999999999999},"min":{"1min":0.439,"5min":0.439,"15min":0.439},"max":{"1min":0.86399999999999999,"5min":0.86399999999999999,"15min":0.86399999999999999},"last":0.69499999999999995}]},{"osd":6,"last update":"Fri Nov 24 07:57:21 2023","interfaces":[{"interface":"back","average":{"1min":0.58599999999999997,"5min":0.58599999999999997,"15min":0.58599999999999997},"min":{"1min":0.35499999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.76800000000000002,"5min":0.76800000000000002,"15min":0.76800000000000002},"last":0.65600000000000003},{"interface":"front","average":{"1min":0.61399999999999999,"5min":0.61399999999999999,"15min":0.61399999999999999},"min":{"1min":0.436,"5min":0.436,"15min":0.436},"max":{"1min":1.095,"5min":1.095,"15min":1.095},"last":0.64200000000000002}]},{"osd":7,"last update":"Fri Nov 24 07:57:21 2023","interfaces":[{"interface":"back","average":{"1min":0.57299999999999995,"5min":0.57299999999999995,"15min":0.57299999999999995},"min":{"1min":0.35099999999999998,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.89300000000000002,"5min":0.89300000000000002,"15min":0.89300000000000002},"last":0.66600000000000004},{"interface":"front","average":{"1min":0.66300000000000003,"5min":0.66300000000000003,"15min":0.66300000000000003},"min":{"1min":0.41099999999999998,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":0.91500000000000004,"5min":0.91500000000000004,"15min":0.91500000000000004},"last":0.68200000000000005}]}]},{"osd":5,"up_from":13,"seq":55834574874,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28096,"kb_used_data":1072,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343744,"statfs":{"total":96636764160,"available":96607993856,"internally_reserved":0,"allocated":1097728,"data_stored":791906,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 07:57:17 2023","interfaces":[{"interface":"back","average":{"1min":0.56200000000000006,"5min":0.56200000000000006,"15min":0.56200000000000006},"min":{"1min":0.32700000000000001,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":1.0109999999999999,"5min":1.0109999999999999,"15min":1.0109999999999999},"last":0.51600000000000001},{"interface":"front","average":{"1min":0.80000000000000004,"5min":0.80000000000000004,"15min":0.80000000000000004},"min":{"1min":0.33400000000000002,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":5.431,"5min":5.431,"15min":5.431},"last":0.437}]},{"osd":1,"last update":"Fri Nov 24 07:57:17 2023","interfaces":[{"interface":"back","average":{"1min":0.55900000000000005,"5min":0.55900000000000005,"15min":0.55900000000000005},"min":{"1min":0.28899999999999998,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":1.016,"5min":1.016,"15min":1.016},"last":0.49299999999999999},{"interface":"front","average":{"1min":0.56999999999999995,"5min":0.56999999999999995,"15min":0.56999999999999995},"min":{"1min":0.311,"5min":0.311,"15min":0.311},"max":{"1min":0.96499999999999997,"5min":0.96499999999999997,"15min":0.96499999999999997},"last":0.58399999999999996}]},{"osd":2,"last update":"Fri Nov 24 07:57:17 2023","interfaces":[{"interface":"back","average":{"1min":0.55100000000000005,"5min":0.55100000000000005,"15min":0.55100000000000005},"min":{"1min":0.377,"5min":0.377,"15min":0.377},"max":{"1min":1.163,"5min":1.163,"15min":1.163},"last":0.53400000000000003},{"interface":"front","average":{"1min":0.63,"5min":0.63,"15min":0.63},"min":{"1min":0.39200000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":1.119,"5min":1.119,"15min":1.119},"last":0.41599999999999998}]},{"osd":3,"last update":"Fri Nov 24 07:57:17 2023","interfaces":[{"interface":"back","average":{"1min":0.90500000000000003,"5min":0.90500000000000003,"15min":0.90500000000000003},"min":{"1min":0.36499999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":5.609,"5min":5.609,"15min":5.609},"last":0.46800000000000003},{"interface":"front","average":{"1min":0.67600000000000005,"5min":0.67600000000000005,"15min":0.67600000000000005},"min":{"1min":0.41899999999999998,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":1.288,"5min":1.288,"15min":1.288},"last":0.64300000000000002}]},{"osd":4,"last update":"Fri Nov 24 07:57:17 2023","interfaces":[{"interface":"back","average":{"1min":0.51000000000000001,"5min":0.51000000000000001,"15min":0.51000000000000001},"min":{"1min":0.36399999999999999,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.68799999999999994,"5min":0.68799999999999994,"15min":0.68799999999999994},"last":0.55000000000000004},{"interface":"front","average":{"1min":0.57199999999999995,"5min":0.57199999999999995,"15min":0.57199999999999995},"min":{"1min":0.39800000000000002,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.78300000000000003,"5min":0.78300000000000003,"15min":0.78300000000000003},"last":0.626}]},{"osd":6,"last update":"Fri Nov 24 07:57:17 2023","interfaces":[{"interface":"back","average":{"1min":0.57099999999999995,"5min":0.57099999999999995,"15min":0.57099999999999995},"min":{"1min":0.35599999999999998,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.89700000000000002,"5min":0.89700000000000002,"15min":0.89700000000000002},"last":0.57299999999999995},{"interface":"front","average":{"1min":0.625,"5min":0.625,"15min":0.625},"min":{"1min":0.42899999999999999,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":0.88200000000000001,"5min":0.88200000000000001,"15min":0.88200000000000001},"last":0.60499999999999998}]},{"osd":7,"last update":"Fri Nov 24 07:57:17 2023","interfaces":[{"interface":"back","average":{"1min":0.64600000000000002,"5min":0.64600000000000002,"15min":0.64600000000000002},"min":{"1min":0.45000000000000001,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":0.96599999999999997,"5min":0.96599999999999997,"15min":0.96599999999999997},"last":0.70299999999999996},{"interface":"front","average":{"1min":0.65300000000000002,"5min":0.65300000000000002,"15min":0.65300000000000002},"min":{"1min":0.46300000000000002,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":0.94199999999999995,"5min":0.94199999999999995,"15min":0.94199999999999995},"last":0.68799999999999994}]}]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1704,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-11-24T07:58:07.161 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2023-11-24T07:58:07.161 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2023-11-24T07:58:07.329 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:07.328+0000 7faf5e4dd700 1 Processor -- start 2023-11-24T07:58:07.330 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:07.328+0000 7faf5e4dd700 1 -- start start 2023-11-24T07:58:07.333 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:07.328+0000 7faf5e4dd700 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7faf581440f0 0x7faf580b51b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:07.333 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:07.328+0000 7faf5e4dd700 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faf58143ec0 con 0x7faf580722c0 2023-11-24T07:58:07.333 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:07.328+0000 7faf5e4dd700 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faf580b3ae0 con 0x7faf58143ae0 2023-11-24T07:58:07.333 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:07.328+0000 7faf5e4dd700 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faf580b40a0 con 0x7faf581440f0 2023-11-24T07:58:07.333 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:07.329+0000 7faf577fe700 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7faf580722c0 0x7faf5806e510 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:56298/0 (socket says 172.21.15.59:56298) 2023-11-24T07:58:07.334 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:07.329+0000 7faf577fe700 1 -- 172.21.15.59:0/668687141 learned_addr learned my addr 172.21.15.59:0/668687141 (peer_addr_for_me v1:172.21.15.59:0/0) 2023-11-24T07:58:07.334 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:07.329+0000 7faf56ffd700 1 -- 172.21.15.59:0/668687141 <== mon.0 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2150782183 0 0) 0x7faf58143ec0 con 0x7faf580722c0 2023-11-24T07:58:07.334 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:07.329+0000 7faf56ffd700 1 -- 172.21.15.59:0/668687141 --> v1:172.21.15.59:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7faf400057c0 con 0x7faf580722c0 2023-11-24T07:58:07.334 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:07.329+0000 7faf5ca7a700 1 --2- 172.21.15.59:0/668687141 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7faf581440f0 0x7faf580b51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:07.334 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:07.329+0000 7faf56ffd700 1 -- 172.21.15.59:0/668687141 <== mon.0 v1:172.21.15.59:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2651653431 0 0) 0x7faf400057c0 con 0x7faf580722c0 2023-11-24T07:58:07.334 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:07.329+0000 7faf56ffd700 1 -- 172.21.15.59:0/668687141 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7faf581440f0 msgr2=0x7faf580b51b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T07:58:07.334 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:07.329+0000 7faf56ffd700 1 --2- 172.21.15.59:0/668687141 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7faf581440f0 0x7faf580b51b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:07.334 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:07.329+0000 7faf56ffd700 1 -- 172.21.15.59:0/668687141 >> v1:172.21.15.76:6789/0 conn(0x7faf58143ae0 legacy=0x7faf58071bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:07.334 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:07.329+0000 7faf56ffd700 1 -- 172.21.15.59:0/668687141 --> v1:172.21.15.59:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faf580b3dc0 con 0x7faf580722c0 2023-11-24T07:58:07.335 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:07.329+0000 7faf56ffd700 1 -- 172.21.15.59:0/668687141 <== mon.0 v1:172.21.15.59:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1312333566 0 0) 0x7faf4c001670 con 0x7faf580722c0 2023-11-24T07:58:07.335 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:07.329+0000 7faf56ffd700 1 -- 172.21.15.59:0/668687141 >> v1:172.21.15.59:6789/0 conn(0x7faf580722c0 legacy=0x7faf5806e510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:07.335 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:07.329+0000 7faf56ffd700 1 --2- 172.21.15.59:0/668687141 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7faf581440f0 0x7faf40005c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:07.335 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:07.329+0000 7faf56ffd700 1 --2- 172.21.15.59:0/668687141 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7faf40006180 0x7faf40006a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:07.335 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:07.329+0000 7faf56ffd700 1 --2- 172.21.15.59:0/668687141 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7faf40006fa0 0x7faf40007420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:07.335 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:07.329+0000 7faf56ffd700 1 -- 172.21.15.59:0/668687141 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf580b40a0 con 0x7faf40006fa0 2023-11-24T07:58:07.335 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:07.329+0000 7faf56ffd700 1 -- 172.21.15.59:0/668687141 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf4000dc80 con 0x7faf40006180 2023-11-24T07:58:07.335 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:07.329+0000 7faf56ffd700 1 -- 172.21.15.59:0/668687141 --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faf4000de40 con 0x7faf581440f0 2023-11-24T07:58:07.335 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:07.329+0000 7faf5ca7a700 1 --2- 172.21.15.59:0/668687141 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7faf40006fa0 0x7faf40007420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:07.335 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:07.329+0000 7faf57fff700 1 --2- 172.21.15.59:0/668687141 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7faf581440f0 0x7faf40005c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:07.335 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:07.329+0000 7faf577fe700 1 --2- 172.21.15.59:0/668687141 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7faf40006180 0x7faf40006a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:07.335 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:07.330+0000 7faf5ca7a700 1 -- 172.21.15.59:0/668687141 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7faf581440f0 msgr2=0x7faf40005c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T07:58:07.336 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:07.330+0000 7faf5ca7a700 1 --2- 172.21.15.59:0/668687141 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7faf581440f0 0x7faf40005c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:07.336 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:07.330+0000 7faf5ca7a700 1 -- 172.21.15.59:0/668687141 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7faf40006180 msgr2=0x7faf40006a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T07:58:07.336 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:07.330+0000 7faf5ca7a700 1 --2- 172.21.15.59:0/668687141 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7faf40006180 0x7faf40006a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:07.336 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:07.330+0000 7faf5ca7a700 1 -- 172.21.15.59:0/668687141 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faf400057c0 con 0x7faf40006fa0 2023-11-24T07:58:07.336 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:07.330+0000 7faf577fe700 1 --2- 172.21.15.59:0/668687141 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7faf40006180 0x7faf40006a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-24T07:58:07.336 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:07.330+0000 7faf5ca7a700 1 --2- 172.21.15.59:0/668687141 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7faf40006fa0 0x7faf40007420 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7faf50009fc0 tx=0x7faf40007a90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=476082aa84c45499 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:58:07.336 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:07.330+0000 7faf56ffd700 1 -- 172.21.15.59:0/668687141 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf5000ec30 con 0x7faf40006fa0 2023-11-24T07:58:07.336 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:07.330+0000 7faf56ffd700 1 -- 172.21.15.59:0/668687141 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faf5000f040 con 0x7faf40006fa0 2023-11-24T07:58:07.336 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:07.330+0000 7faf56ffd700 1 -- 172.21.15.59:0/668687141 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf50014480 con 0x7faf40006fa0 2023-11-24T07:58:07.336 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:07.330+0000 7faf5e4dd700 1 -- 172.21.15.59:0/668687141 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7faf40006fa0 msgr2=0x7faf40007420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:07.336 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:07.330+0000 7faf5e4dd700 1 --2- 172.21.15.59:0/668687141 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7faf40006fa0 0x7faf40007420 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7faf50009fc0 tx=0x7faf40007a90 comp rx=0 tx=0).stop 2023-11-24T07:58:07.336 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:07.330+0000 7faf5ca7a700 1 -- 172.21.15.59:0/668687141 reap_dead start 2023-11-24T07:58:07.337 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:07.331+0000 7faf5e4dd700 1 -- 172.21.15.59:0/668687141 shutdown_connections 2023-11-24T07:58:07.337 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:07.331+0000 7faf5e4dd700 1 -- 172.21.15.59:0/668687141 >> 172.21.15.59:0/668687141 conn(0x7faf580b2740 msgr2=0x7faf580b2b20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:58:07.337 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:07.331+0000 7faf5e4dd700 1 -- 172.21.15.59:0/668687141 shutdown_connections 2023-11-24T07:58:07.337 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:07.331+0000 7faf5e4dd700 1 -- 172.21.15.59:0/668687141 wait complete. 2023-11-24T07:58:07.339 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:07.332+0000 7faf5e4dd700 1 Processor -- start 2023-11-24T07:58:07.340 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:07.332+0000 7faf5e4dd700 1 -- start start 2023-11-24T07:58:07.340 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:07.333+0000 7faf5e4dd700 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7faf40006fa0 0x7faf58150bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:07.340 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:07.333+0000 7faf5e4dd700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7faf581efbe0 0x7faf5815c8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:07.340 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:07.333+0000 7faf5e4dd700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7faf5813ebf0 0x7faf58152da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:07.340 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:07.333+0000 7faf5e4dd700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf5815d450 con 0x7faf581efbe0 2023-11-24T07:58:07.340 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:07.333+0000 7faf5e4dd700 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf581f0060 con 0x7faf5813ebf0 2023-11-24T07:58:07.340 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:07.333+0000 7faf5e4dd700 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faf5815cdf0 con 0x7faf40006fa0 2023-11-24T07:58:07.340 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:07.333+0000 7faf577fe700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7faf581efbe0 0x7faf5815c8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:07.341 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:07.333+0000 7faf577fe700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7faf581efbe0 0x7faf5815c8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:52836/0 (socket says 172.21.15.59:52836) 2023-11-24T07:58:07.341 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:07.333+0000 7faf577fe700 1 -- 172.21.15.59:0/552509639 learned_addr learned my addr 172.21.15.59:0/552509639 (peer_addr_for_me v2:172.21.15.59:0/0) 2023-11-24T07:58:07.341 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:07.334+0000 7faf5ca7a700 1 --2- 172.21.15.59:0/552509639 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7faf5813ebf0 0x7faf58152da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:07.341 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:07.334+0000 7faf57fff700 1 --2- 172.21.15.59:0/552509639 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7faf40006fa0 0x7faf58150bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:07.341 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:07.334+0000 7faf577fe700 1 -- 172.21.15.59:0/552509639 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7faf40006fa0 msgr2=0x7faf58150bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:07.341 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:07.334+0000 7faf577fe700 1 --2- 172.21.15.59:0/552509639 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7faf40006fa0 0x7faf58150bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:07.342 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:07.334+0000 7faf577fe700 1 -- 172.21.15.59:0/552509639 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7faf5813ebf0 msgr2=0x7faf58152da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:07.342 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:07.334+0000 7faf577fe700 1 --2- 172.21.15.59:0/552509639 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7faf5813ebf0 0x7faf58152da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:07.342 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:07.334+0000 7faf577fe700 1 -- 172.21.15.59:0/552509639 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faf58151200 con 0x7faf581efbe0 2023-11-24T07:58:07.342 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:07.334+0000 7faf577fe700 1 --2- 172.21.15.59:0/552509639 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7faf581efbe0 0x7faf5815c8b0 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7faf4c00e870 tx=0x7faf4c00ec30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:58:07.342 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:07.334+0000 7faf54ff9700 1 -- 172.21.15.59:0/552509639 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf4c00ee70 con 0x7faf581efbe0 2023-11-24T07:58:07.342 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:07.334+0000 7faf54ff9700 1 -- 172.21.15.59:0/552509639 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faf4c00c780 con 0x7faf581efbe0 2023-11-24T07:58:07.342 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:07.334+0000 7faf54ff9700 1 -- 172.21.15.59:0/552509639 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf4c00caf0 con 0x7faf581efbe0 2023-11-24T07:58:07.342 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:07.334+0000 7faf5e4dd700 1 -- 172.21.15.59:0/552509639 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faf581514e0 con 0x7faf581efbe0 2023-11-24T07:58:07.342 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:07.334+0000 7faf5e4dd700 1 -- 172.21.15.59:0/552509639 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faf5815d070 con 0x7faf581efbe0 2023-11-24T07:58:07.342 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:07.335+0000 7faf5e4dd700 1 -- 172.21.15.59:0/552509639 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faf58067a00 con 0x7faf581efbe0 2023-11-24T07:58:07.343 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:07.336+0000 7faf54ff9700 1 -- 172.21.15.59:0/552509639 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 6) v1 ==== 89282+0+0 (secure 0 0 0) 0x7faf4c00cc50 con 0x7faf581efbe0 2023-11-24T07:58:07.343 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:07.336+0000 7faf54ff9700 1 --2- 172.21.15.59:0/552509639 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7faf4406af60 0x7faf4406d420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:07.343 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:07.336+0000 7faf54ff9700 1 -- 172.21.15.59:0/552509639 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 5709+0+0 (secure 0 0 0) 0x7faf4c0893d0 con 0x7faf581efbe0 2023-11-24T07:58:07.343 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:07.336+0000 7faf57fff700 1 --2- 172.21.15.59:0/552509639 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7faf4406af60 0x7faf4406d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:07.343 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:07.337+0000 7faf57fff700 1 --2- 172.21.15.59:0/552509639 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7faf4406af60 0x7faf4406d420 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7faf480011a0 tx=0x7faf48009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:58:07.343 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:07.341+0000 7faf54ff9700 1 -- 172.21.15.59:0/552509639 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7faf4c054520 con 0x7faf581efbe0 2023-11-24T07:58:07.575 INFO:tasks.ceph.mds.c.smithi059.stderr:2023-11-24T07:58:07.573+0000 7f137eea8700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-11-24T07:58:07.626 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:07.624+0000 7faf5e4dd700 1 -- 172.21.15.59:0/552509639 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7faf580cd8d0 con 0x7faf581efbe0 2023-11-24T07:58:07.627 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:07.626+0000 7faf54ff9700 1 -- 172.21.15.59:0/552509639 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v14) v1 ==== 190+0+147 (secure 0 0 0) 0x7faf4c057b40 con 0x7faf581efbe0 2023-11-24T07:58:07.628 INFO:teuthology.orchestra.run.smithi059.stdout:[client.0] 2023-11-24T07:58:07.628 INFO:teuthology.orchestra.run.smithi059.stdout: key = AQD1VmBlxPkpBhAAR841hRsF6QtmzFgJIrVZ9w== 2023-11-24T07:58:07.628 INFO:teuthology.orchestra.run.smithi059.stdout: caps mds = "allow" 2023-11-24T07:58:07.628 INFO:teuthology.orchestra.run.smithi059.stdout: caps mon = "allow r" 2023-11-24T07:58:07.628 INFO:teuthology.orchestra.run.smithi059.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2023-11-24T07:58:07.630 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:07.628+0000 7faf5e4dd700 1 -- 172.21.15.59:0/552509639 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7faf4406af60 msgr2=0x7faf4406d420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:07.631 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:07.628+0000 7faf5e4dd700 1 --2- 172.21.15.59:0/552509639 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7faf4406af60 0x7faf4406d420 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7faf480011a0 tx=0x7faf48009040 comp rx=0 tx=0).stop 2023-11-24T07:58:07.631 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:07.628+0000 7faf5e4dd700 1 -- 172.21.15.59:0/552509639 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7faf581efbe0 msgr2=0x7faf5815c8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:07.631 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:07.628+0000 7faf5e4dd700 1 --2- 172.21.15.59:0/552509639 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7faf581efbe0 0x7faf5815c8b0 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7faf4c00e870 tx=0x7faf4c00ec30 comp rx=0 tx=0).stop 2023-11-24T07:58:07.631 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:07.628+0000 7faf5e4dd700 1 -- 172.21.15.59:0/552509639 shutdown_connections 2023-11-24T07:58:07.631 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:07.628+0000 7faf5e4dd700 1 --2- 172.21.15.59:0/552509639 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7faf4406af60 0x7faf4406d420 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:07.631 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:07.628+0000 7faf5e4dd700 1 --2- 172.21.15.59:0/552509639 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7faf5813ebf0 0x7faf58152da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:07.631 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:07.628+0000 7faf5e4dd700 1 --2- 172.21.15.59:0/552509639 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7faf40006fa0 0x7faf58150bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:07.631 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:07.628+0000 7faf5e4dd700 1 --2- 172.21.15.59:0/552509639 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7faf581efbe0 0x7faf5815c8b0 unknown :-1 s=CLOSED pgs=554 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:07.631 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:07.628+0000 7faf5e4dd700 1 -- 172.21.15.59:0/552509639 >> 172.21.15.59:0/552509639 conn(0x7faf580b2740 msgr2=0x7faf58141250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:58:07.631 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:07.629+0000 7faf5e4dd700 1 -- 172.21.15.59:0/552509639 shutdown_connections 2023-11-24T07:58:07.631 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:07.629+0000 7faf5e4dd700 1 -- 172.21.15.59:0/552509639 wait complete. 2023-11-24T07:58:07.632 INFO:teuthology.orchestra.run.smithi059.stderr:updated caps for client.0 2023-11-24T07:58:07.642 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-24T07:58:07.767 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:07.765+0000 7f1c4fb35700 1 Processor -- start 2023-11-24T07:58:07.767 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:07.766+0000 7f1c4fb35700 1 -- start start 2023-11-24T07:58:07.770 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:07.766+0000 7f1c4fb35700 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f1c4815bde0 0x7f1c4806ece0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:07.770 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:07.766+0000 7f1c4fb35700 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f1c4815c1c0 0x7f1c480cd0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:07.770 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:07.767+0000 7f1c4fb35700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f1c480b4240 0x7f1c480cd5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:07.770 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:07.767+0000 7f1c4fb35700 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1c480cffb0 con 0x7f1c480b4240 2023-11-24T07:58:07.771 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:07.767+0000 7f1c4fb35700 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1c480cc670 con 0x7f1c4815bde0 2023-11-24T07:58:07.771 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:07.767+0000 7f1c4fb35700 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1c480cf7a0 con 0x7f1c4815c1c0 2023-11-24T07:58:07.771 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:07.767+0000 7f1c4e0d2700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f1c480b4240 0x7f1c480cd5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:07.771 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:07.767+0000 7f1c4e0d2700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f1c480b4240 0x7f1c480cd5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:52846/0 (socket says 172.21.15.59:52846) 2023-11-24T07:58:07.771 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:07.767+0000 7f1c4e0d2700 1 -- 172.21.15.59:0/44962101 learned_addr learned my addr 172.21.15.59:0/44962101 (peer_addr_for_me v2:172.21.15.59:0/0) 2023-11-24T07:58:07.771 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:07.767+0000 7f1c4d0d0700 1 --2- 172.21.15.59:0/44962101 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f1c4815c1c0 0x7f1c480cd0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:07.772 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:07.767+0000 7f1c4d8d1700 1 --2- 172.21.15.59:0/44962101 >> v2:172.21.15.76:3300/0 conn(0x7f1c4815bde0 0x7f1c4806ece0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:07.772 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:07.767+0000 7f1c4e0d2700 1 -- 172.21.15.59:0/44962101 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f1c4815c1c0 msgr2=0x7f1c480cd0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T07:58:07.772 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:07.767+0000 7f1c4e0d2700 1 --2- 172.21.15.59:0/44962101 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f1c4815c1c0 0x7f1c480cd0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:07.772 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:07.767+0000 7f1c4e0d2700 1 -- 172.21.15.59:0/44962101 >> v2:172.21.15.76:3300/0 conn(0x7f1c4815bde0 msgr2=0x7f1c4806ece0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T07:58:07.772 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:07.767+0000 7f1c4e0d2700 1 --2- 172.21.15.59:0/44962101 >> v2:172.21.15.76:3300/0 conn(0x7f1c4815bde0 0x7f1c4806ece0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:07.772 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:07.767+0000 7f1c4e0d2700 1 -- 172.21.15.59:0/44962101 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1c480cdb20 con 0x7f1c480b4240 2023-11-24T07:58:07.772 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:07.767+0000 7f1c4e0d2700 1 --2- 172.21.15.59:0/44962101 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1c480b4240 0x7f1c480cd5e0 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7f1c4806bf70 tx=0x7f1c4000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=235ea01f254acd8a server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:58:07.772 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:07.767+0000 7f1c4c8cf700 1 -- 172.21.15.59:0/44962101 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c4000e070 con 0x7f1c480b4240 2023-11-24T07:58:07.772 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:07.767+0000 7f1c4c8cf700 1 -- 172.21.15.59:0/44962101 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1c4000b9c0 con 0x7f1c480b4240 2023-11-24T07:58:07.772 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:07.767+0000 7f1c4c8cf700 1 -- 172.21.15.59:0/44962101 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c4000bd70 con 0x7f1c480b4240 2023-11-24T07:58:07.772 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:07.768+0000 7f1c4fb35700 1 -- 172.21.15.59:0/44962101 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1c480b4240 msgr2=0x7f1c480cd5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:07.773 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:07.768+0000 7f1c4fb35700 1 --2- 172.21.15.59:0/44962101 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1c480b4240 0x7f1c480cd5e0 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7f1c4806bf70 tx=0x7f1c4000b330 comp rx=0 tx=0).stop 2023-11-24T07:58:07.773 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:07.768+0000 7f1c4fb35700 1 -- 172.21.15.59:0/44962101 shutdown_connections 2023-11-24T07:58:07.773 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:07.768+0000 7f1c4fb35700 1 --2- 172.21.15.59:0/44962101 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1c480b4240 0x7f1c480cd5e0 unknown :-1 s=CLOSED pgs=555 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:07.773 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:07.768+0000 7f1c4fb35700 1 --2- 172.21.15.59:0/44962101 >> v2:172.21.15.76:3300/0 conn(0x7f1c4815bde0 0x7f1c4806ece0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:07.773 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:07.768+0000 7f1c4fb35700 1 --2- 172.21.15.59:0/44962101 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f1c4815c1c0 0x7f1c480cd0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:07.773 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:07.768+0000 7f1c4fb35700 1 -- 172.21.15.59:0/44962101 >> 172.21.15.59:0/44962101 conn(0x7f1c480b3720 msgr2=0x7f1c480b3b30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:58:07.773 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:07.768+0000 7f1c4fb35700 1 -- 172.21.15.59:0/44962101 shutdown_connections 2023-11-24T07:58:07.773 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:07.768+0000 7f1c4fb35700 1 -- 172.21.15.59:0/44962101 wait complete. 2023-11-24T07:58:07.776 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:07.769+0000 7f1c4fb35700 1 Processor -- start 2023-11-24T07:58:07.776 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:07.769+0000 7f1c4fb35700 1 -- start start 2023-11-24T07:58:07.776 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:07.770+0000 7f1c4fb35700 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f1c480b4240 0x7f1c481f4d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:07.776 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:07.770+0000 7f1c4fb35700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1c4815bde0 0x7f1c480b5ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:07.776 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:07.770+0000 7f1c4fb35700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1c4815d600 0x7f1c480b23c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:07.776 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:07.770+0000 7f1c4fb35700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c480b7310 con 0x7f1c4815bde0 2023-11-24T07:58:07.776 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:07.770+0000 7f1c4fb35700 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c480b6700 con 0x7f1c4815d600 2023-11-24T07:58:07.776 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:07.770+0000 7f1c4fb35700 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1c480b69e0 con 0x7f1c480b4240 2023-11-24T07:58:07.777 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:07.770+0000 7f1c4d0d0700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1c4815bde0 0x7f1c480b5ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:07.777 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:07.770+0000 7f1c4d0d0700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1c4815bde0 0x7f1c480b5ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:52850/0 (socket says 172.21.15.59:52850) 2023-11-24T07:58:07.777 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:07.770+0000 7f1c4d0d0700 1 -- 172.21.15.59:0/647380386 learned_addr learned my addr 172.21.15.59:0/647380386 (peer_addr_for_me v2:172.21.15.59:0/0) 2023-11-24T07:58:07.777 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:07.770+0000 7f1c4e0d2700 1 --2- 172.21.15.59:0/647380386 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1c4815d600 0x7f1c480b23c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:07.777 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:07.770+0000 7f1c4d8d1700 1 --2- 172.21.15.59:0/647380386 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f1c480b4240 0x7f1c481f4d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:07.777 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:07.770+0000 7f1c4d0d0700 1 -- 172.21.15.59:0/647380386 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f1c480b4240 msgr2=0x7f1c481f4d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:07.778 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:07.770+0000 7f1c4d0d0700 1 --2- 172.21.15.59:0/647380386 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f1c480b4240 0x7f1c481f4d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:07.778 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:07.770+0000 7f1c4d0d0700 1 -- 172.21.15.59:0/647380386 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1c4815d600 msgr2=0x7f1c480b23c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:07.778 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:07.770+0000 7f1c4d0d0700 1 --2- 172.21.15.59:0/647380386 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1c4815d600 0x7f1c480b23c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:07.778 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:07.770+0000 7f1c4d0d0700 1 -- 172.21.15.59:0/647380386 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1c40009b30 con 0x7f1c4815bde0 2023-11-24T07:58:07.778 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:07.770+0000 7f1c4e0d2700 1 --2- 172.21.15.59:0/647380386 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1c4815d600 0x7f1c480b23c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-24T07:58:07.778 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:07.770+0000 7f1c4d0d0700 1 --2- 172.21.15.59:0/647380386 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1c4815bde0 0x7f1c480b5ed0 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7f1c3c00e810 tx=0x7f1c3c00ebd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:58:07.778 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:07.771+0000 7f1c3a7fc700 1 -- 172.21.15.59:0/647380386 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c3c013610 con 0x7f1c4815bde0 2023-11-24T07:58:07.778 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:07.771+0000 7f1c3a7fc700 1 -- 172.21.15.59:0/647380386 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1c3c013770 con 0x7f1c4815bde0 2023-11-24T07:58:07.778 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:07.771+0000 7f1c3a7fc700 1 -- 172.21.15.59:0/647380386 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c3c013b20 con 0x7f1c4815bde0 2023-11-24T07:58:07.779 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:07.771+0000 7f1c4fb35700 1 -- 172.21.15.59:0/647380386 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1c480b6cc0 con 0x7f1c4815bde0 2023-11-24T07:58:07.779 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:07.771+0000 7f1c4fb35700 1 -- 172.21.15.59:0/647380386 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1c480b6410 con 0x7f1c4815bde0 2023-11-24T07:58:07.779 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:07.772+0000 7f1c4fb35700 1 -- 172.21.15.59:0/647380386 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1c480b7480 con 0x7f1c4815bde0 2023-11-24T07:58:07.779 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:07.772+0000 7f1c3a7fc700 1 -- 172.21.15.59:0/647380386 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 6) v1 ==== 89282+0+0 (secure 0 0 0) 0x7f1c3c013c80 con 0x7f1c4815bde0 2023-11-24T07:58:07.779 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:07.773+0000 7f1c3a7fc700 1 --2- 172.21.15.59:0/647380386 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f1c2c06af60 0x7f1c2c06d420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:07.779 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:07.773+0000 7f1c3a7fc700 1 -- 172.21.15.59:0/647380386 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f1c3c0893c0 con 0x7f1c4815bde0 2023-11-24T07:58:07.779 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:07.773+0000 7f1c4d8d1700 1 --2- 172.21.15.59:0/647380386 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f1c2c06af60 0x7f1c2c06d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:07.779 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:07.774+0000 7f1c4d8d1700 1 --2- 172.21.15.59:0/647380386 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f1c2c06af60 0x7f1c2c06d420 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f1c4809fd50 tx=0x7f1c44008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:58:07.780 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:07.777+0000 7f1c3a7fc700 1 -- 172.21.15.59:0/647380386 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f1c3c054380 con 0x7f1c4815bde0 2023-11-24T07:58:08.024 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:08.023+0000 7f1c4fb35700 1 -- 172.21.15.59:0/647380386 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f1c480c5b00 con 0x7f1c4815bde0 2023-11-24T07:58:08.025 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:08.023+0000 7f1c3a7fc700 1 -- 172.21.15.59:0/647380386 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) v1 ==== 94+0+4904 (secure 0 0 0) 0x7f1c3c0579a0 con 0x7f1c4815bde0 2023-11-24T07:58:08.025 INFO:teuthology.orchestra.run.smithi059.stdout: 2023-11-24T07:58:08.025 INFO:teuthology.orchestra.run.smithi059.stdout:{"epoch":25,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5290,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.76:6836/508095662","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6834","nonce":508095662},{"type":"v1","addr":"172.21.15.76:6836","nonce":508095662}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":20},{"gid":5293,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.76:6837/4158118811","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6835","nonce":4158118811},{"type":"v1","addr":"172.21.15.76:6837","nonce":4158118811}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":20},{"gid":5622,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6834/59936286","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6832","nonce":59936286},{"type":"v1","addr":"172.21.15.59:6834","nonce":59936286}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":20}],"filesystems":[{"mdsmap":{"epoch":25,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-24T07:58:02.020496+0000","modified":"2023-11-24T07:58:05.037742+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5625},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5625":{"gid":5625,"name":"c","rank":0,"incarnation":21,"state":"up:active","state_seq":3,"addr":"172.21.15.59:6835/1468266626","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6833","nonce":1468266626},{"type":"v1","addr":"172.21.15.59:6835","nonce":1468266626}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2023-11-24T07:58:08.027 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:08.026+0000 7f1c4fb35700 1 -- 172.21.15.59:0/647380386 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f1c2c06af60 msgr2=0x7f1c2c06d420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:08.028 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:08.026+0000 7f1c4fb35700 1 --2- 172.21.15.59:0/647380386 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f1c2c06af60 0x7f1c2c06d420 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f1c4809fd50 tx=0x7f1c44008040 comp rx=0 tx=0).stop 2023-11-24T07:58:08.028 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:08.026+0000 7f1c4fb35700 1 -- 172.21.15.59:0/647380386 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1c4815bde0 msgr2=0x7f1c480b5ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:08.028 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:08.026+0000 7f1c4fb35700 1 --2- 172.21.15.59:0/647380386 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1c4815bde0 0x7f1c480b5ed0 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7f1c3c00e810 tx=0x7f1c3c00ebd0 comp rx=0 tx=0).stop 2023-11-24T07:58:08.028 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:08.026+0000 7f1c4fb35700 1 -- 172.21.15.59:0/647380386 shutdown_connections 2023-11-24T07:58:08.028 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:08.026+0000 7f1c4fb35700 1 --2- 172.21.15.59:0/647380386 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f1c2c06af60 0x7f1c2c06d420 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:08.028 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:08.026+0000 7f1c4fb35700 1 --2- 172.21.15.59:0/647380386 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1c4815d600 0x7f1c480b23c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:08.028 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:08.026+0000 7f1c4fb35700 1 --2- 172.21.15.59:0/647380386 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f1c480b4240 0x7f1c481f4d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:08.029 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:08.026+0000 7f1c4fb35700 1 --2- 172.21.15.59:0/647380386 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1c4815bde0 0x7f1c480b5ed0 unknown :-1 s=CLOSED pgs=556 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:08.029 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:08.026+0000 7f1c4fb35700 1 -- 172.21.15.59:0/647380386 >> 172.21.15.59:0/647380386 conn(0x7f1c480b3720 msgr2=0x7f1c480ccfa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:58:08.029 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:08.026+0000 7f1c4fb35700 1 -- 172.21.15.59:0/647380386 shutdown_connections 2023-11-24T07:58:08.029 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:08.026+0000 7f1c4fb35700 1 -- 172.21.15.59:0/647380386 wait complete. 2023-11-24T07:58:08.029 INFO:teuthology.orchestra.run.smithi059.stderr:dumped fsmap epoch 25 2023-11-24T07:58:08.443 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 25, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-11-24T07:58:02.020496+0000', 'modified': '2023-11-24T07:58:05.037742+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 5625}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_5625': {'gid': 5625, 'name': 'c', 'rank': 0, 'incarnation': 21, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.59:6835/1468266626', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.59:6833', 'nonce': 1468266626}, {'type': 'v1', 'addr': '172.21.15.59:6835', 'nonce': 1468266626}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [7], 'metadata_pool': 6, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2023-11-24T07:58:08.443 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2023-11-24T07:58:08.443 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2023-11-24T07:58:08.573 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:08.572+0000 7fd4979de700 1 Processor -- start 2023-11-24T07:58:08.573 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:08.572+0000 7fd4979de700 1 -- start start 2023-11-24T07:58:08.576 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:08.573+0000 7fd4979de700 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fd4900b4c40 0x7fd49006dba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:08.576 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:08.573+0000 7fd4979de700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fd49006f420 0x7fd4900ce1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:08.576 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:08.573+0000 7fd4979de700 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd49006ef90 con 0x7fd4900b3b40 2023-11-24T07:58:08.576 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:08.573+0000 7fd4979de700 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd49006e0e0 con 0x7fd49006f420 2023-11-24T07:58:08.576 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:08.573+0000 7fd4979de700 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd49006e380 con 0x7fd4900b4c40 2023-11-24T07:58:08.576 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:08.573+0000 7fd495f7b700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fd49006f420 0x7fd4900ce1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:08.576 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:08.573+0000 7fd495f7b700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fd49006f420 0x7fd4900ce1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:52860/0 (socket says 172.21.15.59:52860) 2023-11-24T07:58:08.576 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:08.573+0000 7fd495f7b700 1 -- 172.21.15.59:0/4181337248 learned_addr learned my addr 172.21.15.59:0/4181337248 (peer_addr_for_me v2:172.21.15.59:0/0) 2023-11-24T07:58:08.576 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:08.573+0000 7fd49577a700 1 --2- 172.21.15.59:0/4181337248 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fd4900b4c40 0x7fd49006dba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:08.576 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:08.573+0000 7fd495f7b700 1 -- 172.21.15.59:0/4181337248 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fd4900b4c40 msgr2=0x7fd49006dba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T07:58:08.577 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:08.573+0000 7fd495f7b700 1 --2- 172.21.15.59:0/4181337248 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fd4900b4c40 0x7fd49006dba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:08.577 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:08.573+0000 7fd495f7b700 1 -- 172.21.15.59:0/4181337248 >> v1:172.21.15.76:6789/0 conn(0x7fd4900b3b40 legacy=0x7fd4900b5020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:08.577 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:08.573+0000 7fd495f7b700 1 -- 172.21.15.59:0/4181337248 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd49006e660 con 0x7fd49006f420 2023-11-24T07:58:08.577 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:08.573+0000 7fd495f7b700 1 --2- 172.21.15.59:0/4181337248 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd49006f420 0x7fd4900ce1d0 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7fd48800d150 tx=0x7fd48800d460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a25576d7288712da server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:58:08.577 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:08.573+0000 7fd487fff700 1 -- 172.21.15.59:0/4181337248 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd488010070 con 0x7fd49006f420 2023-11-24T07:58:08.577 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:08.573+0000 7fd487fff700 1 -- 172.21.15.59:0/4181337248 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd48800ddc0 con 0x7fd49006f420 2023-11-24T07:58:08.577 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:08.573+0000 7fd487fff700 1 -- 172.21.15.59:0/4181337248 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd48800b310 con 0x7fd49006f420 2023-11-24T07:58:08.577 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:08.574+0000 7fd4979de700 1 -- 172.21.15.59:0/4181337248 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd49006f420 msgr2=0x7fd4900ce1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:08.577 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:08.574+0000 7fd4979de700 1 --2- 172.21.15.59:0/4181337248 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd49006f420 0x7fd4900ce1d0 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7fd48800d150 tx=0x7fd48800d460 comp rx=0 tx=0).stop 2023-11-24T07:58:08.577 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:08.574+0000 7fd4979de700 1 -- 172.21.15.59:0/4181337248 shutdown_connections 2023-11-24T07:58:08.577 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:08.574+0000 7fd4979de700 1 --2- 172.21.15.59:0/4181337248 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fd4900b4c40 0x7fd49006dba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:08.577 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:08.574+0000 7fd4979de700 1 --2- 172.21.15.59:0/4181337248 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd49006f420 0x7fd4900ce1d0 unknown :-1 s=CLOSED pgs=557 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:08.577 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:08.574+0000 7fd4979de700 1 -- 172.21.15.59:0/4181337248 >> 172.21.15.59:0/4181337248 conn(0x7fd4900b2840 msgr2=0x7fd4900b2c20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:58:08.578 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:08.574+0000 7fd4979de700 1 -- 172.21.15.59:0/4181337248 shutdown_connections 2023-11-24T07:58:08.578 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:08.574+0000 7fd4979de700 1 -- 172.21.15.59:0/4181337248 wait complete. 2023-11-24T07:58:08.579 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:08.575+0000 7fd4979de700 1 Processor -- start 2023-11-24T07:58:08.579 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:08.575+0000 7fd4979de700 1 -- start start 2023-11-24T07:58:08.579 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:08.576+0000 7fd4979de700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd49006f420 0x7fd49015c350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:08.580 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:08.576+0000 7fd4979de700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd4900b3b40 0x7fd49015a0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:08.580 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:08.576+0000 7fd4979de700 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fd4900b4c40 0x7fd4900d00d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:08.580 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:08.576+0000 7fd4979de700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd49015c890 con 0x7fd49006f420 2023-11-24T07:58:08.580 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:08.576+0000 7fd4979de700 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4901f76a0 con 0x7fd4900b3b40 2023-11-24T07:58:08.580 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:08.576+0000 7fd4979de700 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd4901f7980 con 0x7fd4900b4c40 2023-11-24T07:58:08.580 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:08.576+0000 7fd49577a700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd49006f420 0x7fd49015c350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:08.580 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:08.576+0000 7fd49577a700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd49006f420 0x7fd49015c350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:52862/0 (socket says 172.21.15.59:52862) 2023-11-24T07:58:08.580 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:08.576+0000 7fd49577a700 1 -- 172.21.15.59:0/3384845750 learned_addr learned my addr 172.21.15.59:0/3384845750 (peer_addr_for_me v2:172.21.15.59:0/0) 2023-11-24T07:58:08.580 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:08.576+0000 7fd494f79700 1 --2- 172.21.15.59:0/3384845750 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd4900b3b40 0x7fd49015a0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:08.580 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:08.576+0000 7fd495f7b700 1 --2- 172.21.15.59:0/3384845750 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fd4900b4c40 0x7fd4900d00d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:08.580 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:08.576+0000 7fd49577a700 1 -- 172.21.15.59:0/3384845750 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fd4900b4c40 msgr2=0x7fd4900d00d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:08.580 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:08.576+0000 7fd49577a700 1 --2- 172.21.15.59:0/3384845750 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fd4900b4c40 0x7fd4900d00d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:08.580 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:08.576+0000 7fd49577a700 1 -- 172.21.15.59:0/3384845750 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd4900b3b40 msgr2=0x7fd49015a0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:08.581 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:08.576+0000 7fd49577a700 1 --2- 172.21.15.59:0/3384845750 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd4900b3b40 0x7fd49015a0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:08.581 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:08.576+0000 7fd49577a700 1 -- 172.21.15.59:0/3384845750 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd48800bd40 con 0x7fd49006f420 2023-11-24T07:58:08.581 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:08.576+0000 7fd495f7b700 1 --2- 172.21.15.59:0/3384845750 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fd4900b4c40 0x7fd4900d00d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T07:58:08.581 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:08.577+0000 7fd49577a700 1 --2- 172.21.15.59:0/3384845750 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd49006f420 0x7fd49015c350 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7fd48c00d800 tx=0x7fd48c00dbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:58:08.581 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:08.577+0000 7fd485ffb700 1 -- 172.21.15.59:0/3384845750 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd48c0125f0 con 0x7fd49006f420 2023-11-24T07:58:08.581 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:08.577+0000 7fd485ffb700 1 -- 172.21.15.59:0/3384845750 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd48c012750 con 0x7fd49006f420 2023-11-24T07:58:08.581 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:08.577+0000 7fd4979de700 1 -- 172.21.15.59:0/3384845750 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd4901f7c60 con 0x7fd49006f420 2023-11-24T07:58:08.581 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:08.577+0000 7fd485ffb700 1 -- 172.21.15.59:0/3384845750 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd48c012b00 con 0x7fd49006f420 2023-11-24T07:58:08.581 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:08.577+0000 7fd4979de700 1 -- 172.21.15.59:0/3384845750 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd4901f8220 con 0x7fd49006f420 2023-11-24T07:58:08.582 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:08.579+0000 7fd485ffb700 1 -- 172.21.15.59:0/3384845750 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 6) v1 ==== 89282+0+0 (secure 0 0 0) 0x7fd48c012c60 con 0x7fd49006f420 2023-11-24T07:58:08.582 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:08.579+0000 7fd485ffb700 1 --2- 172.21.15.59:0/3384845750 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7fd47406af60 0x7fd47406d420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:08.582 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:08.579+0000 7fd485ffb700 1 -- 172.21.15.59:0/3384845750 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fd48c0899b0 con 0x7fd49006f420 2023-11-24T07:58:08.582 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:08.579+0000 7fd494f79700 1 --2- 172.21.15.59:0/3384845750 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7fd47406af60 0x7fd47406d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:08.582 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:08.580+0000 7fd494f79700 1 --2- 172.21.15.59:0/3384845750 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7fd47406af60 0x7fd47406d420 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7fd490072820 tx=0x7fd480009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:58:08.583 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:08.581+0000 7fd4a3bd7b80 1 Processor -- start 2023-11-24T07:58:08.583 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:08.581+0000 7fd4a3bd7b80 1 -- start start 2023-11-24T07:58:08.584 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:08.582+0000 7fd4a3bd7b80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561d4e7fc080 0x561d4e842840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:08.584 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:08.582+0000 7fd4a3bd7b80 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x561d4e811ed0 0x561d4e842d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:08.585 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:08.582+0000 7fd4a3bd7b80 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x561d4e7f9490 0x561d4e825e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:08.585 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:08.582+0000 7fd4a3bd7b80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x561d4e76f770 con 0x561d4e7fc080 2023-11-24T07:58:08.585 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:08.582+0000 7fd4a3bd7b80 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x561d4e756750 con 0x561d4e7f9490 2023-11-24T07:58:08.585 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:08.582+0000 7fd4a3bd7b80 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 v1 -- 0x561d4e754070 con 0x561d4e811ed0 2023-11-24T07:58:08.585 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:08.582+0000 7fd49577a700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561d4e7fc080 0x561d4e842840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:08.585 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:08.582+0000 7fd49577a700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561d4e7fc080 0x561d4e842840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:52870/0 (socket says 172.21.15.59:52870) 2023-11-24T07:58:08.585 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:08.582+0000 7fd49577a700 1 -- 172.21.15.59:0/4225948903 learned_addr learned my addr 172.21.15.59:0/4225948903 (peer_addr_for_me v2:172.21.15.59:0/0) 2023-11-24T07:58:08.585 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:08.582+0000 7fd495f7b700 1 --2- 172.21.15.59:0/4225948903 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x561d4e7f9490 0x561d4e825e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:08.585 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:08.582+0000 7fd494f79700 1 --2- 172.21.15.59:0/4225948903 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x561d4e811ed0 0x561d4e842d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:08.585 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:08.582+0000 7fd49577a700 1 -- 172.21.15.59:0/4225948903 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x561d4e811ed0 msgr2=0x561d4e842d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T07:58:08.585 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:08.582+0000 7fd49577a700 1 --2- 172.21.15.59:0/4225948903 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x561d4e811ed0 0x561d4e842d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:08.585 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:08.582+0000 7fd49577a700 1 -- 172.21.15.59:0/4225948903 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x561d4e7f9490 msgr2=0x561d4e825e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T07:58:08.585 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:08.582+0000 7fd49577a700 1 --2- 172.21.15.59:0/4225948903 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x561d4e7f9490 0x561d4e825e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:08.586 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:08.582+0000 7fd49577a700 1 -- 172.21.15.59:0/4225948903 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561d4e5cc5e0 con 0x561d4e7fc080 2023-11-24T07:58:08.586 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:08.583+0000 7fd49577a700 1 --2- 172.21.15.59:0/4225948903 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561d4e7fc080 0x561d4e842840 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7fd48c0586c0 tx=0x7fd48c0583a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ea3302324ce1fad1 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:58:08.586 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:08.583+0000 7fd47affd700 1 -- 172.21.15.59:0/4225948903 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd48c05c880 con 0x561d4e7fc080 2023-11-24T07:58:08.586 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:08.583+0000 7fd47affd700 1 -- 172.21.15.59:0/4225948903 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd48c060a30 con 0x561d4e7fc080 2023-11-24T07:58:08.586 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:08.583+0000 7fd47affd700 1 -- 172.21.15.59:0/4225948903 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd48c058bd0 con 0x561d4e7fc080 2023-11-24T07:58:08.590 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:08.583+0000 7fd4a3bd7b80 1 -- 172.21.15.59:0/4225948903 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561d4e7fc080 msgr2=0x561d4e842840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:08.590 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:08.583+0000 7fd4a3bd7b80 1 --2- 172.21.15.59:0/4225948903 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561d4e7fc080 0x561d4e842840 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7fd48c0586c0 tx=0x7fd48c0583a0 comp rx=0 tx=0).stop 2023-11-24T07:58:08.590 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:08.583+0000 7fd4a3bd7b80 1 -- 172.21.15.59:0/4225948903 shutdown_connections 2023-11-24T07:58:08.590 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:08.583+0000 7fd4a3bd7b80 1 --2- 172.21.15.59:0/4225948903 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x561d4e7f9490 0x561d4e825e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:08.591 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:08.583+0000 7fd4a3bd7b80 1 --2- 172.21.15.59:0/4225948903 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561d4e7fc080 0x561d4e842840 unknown :-1 s=CLOSED pgs=559 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:08.591 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:08.583+0000 7fd4a3bd7b80 1 --2- 172.21.15.59:0/4225948903 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x561d4e811ed0 0x561d4e842d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:08.591 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:08.583+0000 7fd4a3bd7b80 1 -- 172.21.15.59:0/4225948903 >> 172.21.15.59:0/4225948903 conn(0x561d4e72a310 msgr2=0x561d4e840e70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:58:08.591 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:08.583+0000 7fd4a3bd7b80 1 -- 172.21.15.59:0/4225948903 shutdown_connections 2023-11-24T07:58:08.591 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:08.583+0000 7fd4a3bd7b80 1 -- 172.21.15.59:0/4225948903 wait complete. 2023-11-24T07:58:08.591 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:08.584+0000 7fd4a3bd7b80 1 -- start start 2023-11-24T07:58:08.591 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:08.584+0000 7fd4a3bd7b80 1 Processor -- start 2023-11-24T07:58:08.591 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:08.585+0000 7fd4a3bd7b80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561d4e7f9490 0x561d4e825fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:08.591 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:08.585+0000 7fd4a3bd7b80 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x561d4e7fc080 0x561d4e83e030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:08.591 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:08.585+0000 7fd4a3bd7b80 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x561d4e811ed0 0x561d4e7f0330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:08.591 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:08.585+0000 7fd4a3bd7b80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x561d4e74f960 con 0x561d4e7f9490 2023-11-24T07:58:08.591 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:08.585+0000 7fd4a3bd7b80 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x561d4e709510 con 0x561d4e811ed0 2023-11-24T07:58:08.592 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:08.585+0000 7fd4a3bd7b80 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 v1 -- 0x561d4e7091b0 con 0x561d4e7fc080 2023-11-24T07:58:08.592 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:08.585+0000 7fd4a3bd7b80 10 client.0 _ll_register_callbacks cb 0x561d4e634a30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-24T07:58:08.592 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:08.585+0000 7fd49577a700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561d4e7f9490 0x561d4e825fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:08.592 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:08.585+0000 7fd49577a700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561d4e7f9490 0x561d4e825fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:52876/0 (socket says 172.21.15.59:52876) 2023-11-24T07:58:08.592 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:08.585+0000 7fd49577a700 1 -- 172.21.15.59:0/2452050451 learned_addr learned my addr 172.21.15.59:0/2452050451 (peer_addr_for_me v2:172.21.15.59:0/0) 2023-11-24T07:58:08.592 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:08.585+0000 7fd494f79700 1 --2- 172.21.15.59:0/2452050451 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x561d4e7fc080 0x561d4e83e030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:08.592 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:08.585+0000 7fd495f7b700 1 --2- 172.21.15.59:0/2452050451 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x561d4e811ed0 0x561d4e7f0330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:08.592 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:08.585+0000 7fd49577a700 1 -- 172.21.15.59:0/2452050451 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x561d4e7fc080 msgr2=0x561d4e83e030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T07:58:08.592 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:08.585+0000 7fd49577a700 1 --2- 172.21.15.59:0/2452050451 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x561d4e7fc080 0x561d4e83e030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:08.592 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:08.585+0000 7fd49577a700 1 -- 172.21.15.59:0/2452050451 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x561d4e811ed0 msgr2=0x561d4e7f0330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T07:58:08.592 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:08.585+0000 7fd49577a700 1 --2- 172.21.15.59:0/2452050451 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x561d4e811ed0 0x561d4e7f0330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:08.592 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:08.585+0000 7fd49577a700 1 -- 172.21.15.59:0/2452050451 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd4901f8220 con 0x561d4e7f9490 2023-11-24T07:58:08.592 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:08.585+0000 7fd494f79700 1 --2- 172.21.15.59:0/2452050451 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x561d4e7fc080 0x561d4e83e030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-24T07:58:08.593 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:08.586+0000 7fd49577a700 1 --2- 172.21.15.59:0/2452050451 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561d4e7f9490 0x561d4e825fe0 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7fd48c087f80 tx=0x7fd48c020070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2952fd9869c30545 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:58:08.593 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:08.586+0000 7fd46ffff700 10 client.0 ms_handle_connect on v2:172.21.15.59:3300/0 2023-11-24T07:58:08.593 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:08.586+0000 7fd46ffff700 1 -- 172.21.15.59:0/2452050451 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd48c057ef0 con 0x561d4e7f9490 2023-11-24T07:58:08.593 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:08.586+0000 7fd46ffff700 1 -- 172.21.15.59:0/2452050451 <== mon.0 v2:172.21.15.59:3300/0 2 ==== osd_map(45..45 src has 1..45) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fd48c058b90 con 0x561d4e7f9490 2023-11-24T07:58:08.593 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:08.586+0000 7fd4a3bd7b80 1 -- 172.21.15.59:0/2452050451 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x561d4e7eea40 con 0x561d4e7f9490 2023-11-24T07:58:08.593 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:08.586+0000 7fd46ffff700 1 -- 172.21.15.59:0/2452050451 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd48c077080 con 0x561d4e7f9490 2023-11-24T07:58:08.593 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:08.586+0000 7fd4a3bd7b80 10 client.5772 fetch_fsmap learned FSMap version 25 2023-11-24T07:58:08.593 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:08.586+0000 7fd4a3bd7b80 1 -- 172.21.15.59:0/2452050451 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=2+}) v3 -- 0x561d4e578d90 con 0x561d4e7f9490 2023-11-24T07:58:08.593 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:08.586+0000 7fd46ffff700 1 -- 172.21.15.59:0/2452050451 <== mon.0 v2:172.21.15.59:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd48c07c020 con 0x561d4e7f9490 2023-11-24T07:58:08.593 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:08.586+0000 7fd46ffff700 1 -- 172.21.15.59:0/2452050451 <== mon.0 v2:172.21.15.59:3300/0 5 ==== fsmap(e 25) v1 ==== 3294+0+0 (secure 0 0 0) 0x7fd48c0605a0 con 0x561d4e7f9490 2023-11-24T07:58:08.593 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:08.586+0000 7fd4a3bd7b80 10 client.5772 fetch_fsmap finished waiting for FSMap version 25 2023-11-24T07:58:08.593 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:08.586+0000 7fd4a3bd7b80 10 client.5772 resolve_mds: resolved name 'c' to daemon mds.c 2023-11-24T07:58:08.594 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:08.586+0000 7fd4a3bd7b80 20 client.5772 populate_metadata read hostname 'smithi059' 2023-11-24T07:58:08.594 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:08.586+0000 7fd4a3bd7b80 1 --2- 172.21.15.59:0/2452050451 >> [v2:172.21.15.59:6833/1468266626,v1:172.21.15.59:6835/1468266626] conn(0x561d4e829610 0x561d4e8299f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:08.594 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:08.586+0000 7fd4a3bd7b80 4 client.5772 mds_command: new command op to 5625 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-11-24T07:58:08.594 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:08.586+0000 7fd4a3bd7b80 1 -- 172.21.15.59:0/2452050451 --> [v2:172.21.15.59:6833/1468266626,v1:172.21.15.59:6835/1468266626] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x561d4e7ca840 con 0x561d4e829610 2023-11-24T07:58:08.594 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:08.586+0000 7fd494f79700 1 --2- 172.21.15.59:0/2452050451 >> [v2:172.21.15.59:6833/1468266626,v1:172.21.15.59:6835/1468266626] conn(0x561d4e829610 0x561d4e8299f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:08.594 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:08.587+0000 7fd46ffff700 10 client.5772 ms_handle_connect on v2:172.21.15.59:6833/1468266626 2023-11-24T07:58:08.594 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:08.587+0000 7fd494f79700 1 --2- 172.21.15.59:0/2452050451 >> [v2:172.21.15.59:6833/1468266626,v1:172.21.15.59:6835/1468266626] conn(0x561d4e829610 0x561d4e8299f0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9a328cabf09b176e server_cookie=2e773fe3742a9539 in_seq=0 out_seq=0 2023-11-24T07:58:08.594 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:08.587+0000 7fd46ffff700 1 -- 172.21.15.59:0/2452050451 <== mds.0 v2:172.21.15.59:6833/1468266626 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x561d4e7ca840 con 0x561d4e829610 2023-11-24T07:58:08.594 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:08.587+0000 7fd46ffff700 10 client.5772 handle_command_reply: tid=0 2023-11-24T07:58:08.594 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:08.587+0000 7fd4a3bd7b80 1 client.5772 shutdown 2023-11-24T07:58:08.594 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:08.587+0000 7fd4a3bd7b80 1 -- 172.21.15.59:0/2452050451 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561d4e7f9490 msgr2=0x561d4e825fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:08.594 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:08.587+0000 7fd4a3bd7b80 1 --2- 172.21.15.59:0/2452050451 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561d4e7f9490 0x561d4e825fe0 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7fd48c087f80 tx=0x7fd48c020070 comp rx=0 tx=0).stop 2023-11-24T07:58:08.595 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:08.587+0000 7fd4a3bd7b80 1 -- 172.21.15.59:0/2452050451 shutdown_connections 2023-11-24T07:58:08.595 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:08.587+0000 7fd4a3bd7b80 1 --2- 172.21.15.59:0/2452050451 >> [v2:172.21.15.59:6833/1468266626,v1:172.21.15.59:6835/1468266626] conn(0x561d4e829610 0x561d4e8299f0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:08.595 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:08.587+0000 7fd4a3bd7b80 1 --2- 172.21.15.59:0/2452050451 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x561d4e811ed0 0x561d4e7f0330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:08.595 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:08.587+0000 7fd4a3bd7b80 1 --2- 172.21.15.59:0/2452050451 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561d4e7f9490 0x561d4e825fe0 unknown :-1 s=CLOSED pgs=560 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:08.595 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:08.587+0000 7fd46ffff700 0 client.5772 ms_handle_reset on v2:172.21.15.59:6833/1468266626 2023-11-24T07:58:08.595 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:08.587+0000 7fd4a3bd7b80 1 --2- 172.21.15.59:0/2452050451 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x561d4e7fc080 0x561d4e83e030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:08.595 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:08.587+0000 7fd4a3bd7b80 1 -- 172.21.15.59:0/2452050451 >> 172.21.15.59:0/2452050451 conn(0x561d4e72a310 msgr2=0x561d4e844c80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:58:08.595 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:08.588+0000 7fd4a3bd7b80 1 -- 172.21.15.59:0/2452050451 shutdown_connections 2023-11-24T07:58:08.595 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:08.588+0000 7fd4a3bd7b80 1 -- 172.21.15.59:0/2452050451 wait complete. 2023-11-24T07:58:08.595 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:08.588+0000 7fd4a3bd7b80 20 client.5772 trim_cache size 0 max 16384 2023-11-24T07:58:08.607 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:08.606+0000 7fd4a3bd7b80 1 Processor -- start 2023-11-24T07:58:08.607 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:08.606+0000 7fd4a3bd7b80 1 -- start start 2023-11-24T07:58:08.610 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:08.607+0000 7fd4a3bd7b80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561d4e7f9490 0x561d4e7f7fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:08.610 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:08.607+0000 7fd4a3bd7b80 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x561d4e7fc080 0x561d4e829be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:08.610 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:08.607+0000 7fd4a3bd7b80 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x561d4e811ed0 0x561d4e841cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:08.610 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:08.607+0000 7fd4a3bd7b80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x561d4e5ce8d0 con 0x561d4e7f9490 2023-11-24T07:58:08.610 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:08.607+0000 7fd4a3bd7b80 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x561d4e83c670 con 0x561d4e811ed0 2023-11-24T07:58:08.610 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:08.607+0000 7fd4a3bd7b80 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 v1 -- 0x561d4e7f2a60 con 0x561d4e7fc080 2023-11-24T07:58:08.610 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:08.607+0000 7fd49577a700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561d4e7f9490 0x561d4e7f7fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:08.610 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:08.607+0000 7fd49577a700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561d4e7f9490 0x561d4e7f7fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:52884/0 (socket says 172.21.15.59:52884) 2023-11-24T07:58:08.610 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:08.607+0000 7fd49577a700 1 -- 172.21.15.59:0/3719982209 learned_addr learned my addr 172.21.15.59:0/3719982209 (peer_addr_for_me v2:172.21.15.59:0/0) 2023-11-24T07:58:08.611 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:08.607+0000 7fd495f7b700 1 --2- 172.21.15.59:0/3719982209 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x561d4e811ed0 0x561d4e841cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:08.611 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:08.607+0000 7fd494f79700 1 --2- 172.21.15.59:0/3719982209 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x561d4e7fc080 0x561d4e829be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:08.611 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:08.607+0000 7fd49577a700 1 -- 172.21.15.59:0/3719982209 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x561d4e7fc080 msgr2=0x561d4e829be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T07:58:08.611 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:08.607+0000 7fd49577a700 1 --2- 172.21.15.59:0/3719982209 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x561d4e7fc080 0x561d4e829be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:08.611 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:08.607+0000 7fd49577a700 1 -- 172.21.15.59:0/3719982209 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x561d4e811ed0 msgr2=0x561d4e841cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T07:58:08.611 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:08.607+0000 7fd49577a700 1 --2- 172.21.15.59:0/3719982209 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x561d4e811ed0 0x561d4e841cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:08.611 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:08.607+0000 7fd49577a700 1 -- 172.21.15.59:0/3719982209 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561d4e561c10 con 0x561d4e7f9490 2023-11-24T07:58:08.611 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:08.607+0000 7fd49577a700 1 --2- 172.21.15.59:0/3719982209 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561d4e7f9490 0x561d4e7f7fd0 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7fd48c007fd0 tx=0x7fd48c083070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f162c8a1533c6cec server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:58:08.611 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:08.607+0000 7fd47b7fe700 1 -- 172.21.15.59:0/3719982209 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd48c077080 con 0x561d4e7f9490 2023-11-24T07:58:08.611 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:08.607+0000 7fd47b7fe700 1 -- 172.21.15.59:0/3719982209 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd48c07c070 con 0x561d4e7f9490 2023-11-24T07:58:08.611 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:08.607+0000 7fd47b7fe700 1 -- 172.21.15.59:0/3719982209 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd48c057ef0 con 0x561d4e7f9490 2023-11-24T07:58:08.611 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:08.608+0000 7fd4a3bd7b80 1 -- 172.21.15.59:0/3719982209 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561d4e7f9490 msgr2=0x561d4e7f7fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:08.612 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:08.608+0000 7fd4a3bd7b80 1 --2- 172.21.15.59:0/3719982209 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561d4e7f9490 0x561d4e7f7fd0 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7fd48c007fd0 tx=0x7fd48c083070 comp rx=0 tx=0).stop 2023-11-24T07:58:08.612 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:08.608+0000 7fd4a3bd7b80 1 -- 172.21.15.59:0/3719982209 shutdown_connections 2023-11-24T07:58:08.612 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:08.608+0000 7fd4a3bd7b80 1 --2- 172.21.15.59:0/3719982209 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x561d4e811ed0 0x561d4e841cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:08.612 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:08.608+0000 7fd4a3bd7b80 1 --2- 172.21.15.59:0/3719982209 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561d4e7f9490 0x561d4e7f7fd0 unknown :-1 s=CLOSED pgs=561 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:08.612 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:08.608+0000 7fd4a3bd7b80 1 --2- 172.21.15.59:0/3719982209 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x561d4e7fc080 0x561d4e829be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:08.612 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:08.608+0000 7fd4a3bd7b80 1 -- 172.21.15.59:0/3719982209 >> 172.21.15.59:0/3719982209 conn(0x561d4e72a310 msgr2=0x561d4e841300 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:58:08.612 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:08.608+0000 7fd4a3bd7b80 1 -- 172.21.15.59:0/3719982209 shutdown_connections 2023-11-24T07:58:08.612 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:08.608+0000 7fd4a3bd7b80 1 -- 172.21.15.59:0/3719982209 wait complete. 2023-11-24T07:58:08.612 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:08.608+0000 7fd4a3bd7b80 1 -- start start 2023-11-24T07:58:08.612 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:08.609+0000 7fd4a3bd7b80 1 Processor -- start 2023-11-24T07:58:08.616 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:08.610+0000 7fd4a3bd7b80 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x561d4e7f9490 0x561d4e8a95d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:08.616 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:08.610+0000 7fd4a3bd7b80 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x561d4e7fc080 0x561d4e7efdf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:08.616 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:08.610+0000 7fd4a3bd7b80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561d4e811ed0 0x561d4e836a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:08.616 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:08.610+0000 7fd4a3bd7b80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x561d4e8ab0c0 con 0x561d4e811ed0 2023-11-24T07:58:08.616 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:08.610+0000 7fd4a3bd7b80 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x561d4e837210 con 0x561d4e7f9490 2023-11-24T07:58:08.616 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:08.610+0000 7fd4a3bd7b80 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 v1 -- 0x561d4e8377d0 con 0x561d4e7fc080 2023-11-24T07:58:08.616 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:08.610+0000 7fd4a3bd7b80 10 client.0 _ll_register_callbacks cb 0x561d4e634a30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-24T07:58:08.616 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:08.610+0000 7fd495f7b700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561d4e811ed0 0x561d4e836a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:08.617 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:08.610+0000 7fd495f7b700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561d4e811ed0 0x561d4e836a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:52894/0 (socket says 172.21.15.59:52894) 2023-11-24T07:58:08.617 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:08.610+0000 7fd495f7b700 1 -- 172.21.15.59:0/648538307 learned_addr learned my addr 172.21.15.59:0/648538307 (peer_addr_for_me v2:172.21.15.59:0/0) 2023-11-24T07:58:08.617 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:08.610+0000 7fd49577a700 1 --2- 172.21.15.59:0/648538307 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x561d4e7f9490 0x561d4e8a95d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:08.617 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:08.610+0000 7fd494f79700 1 --2- 172.21.15.59:0/648538307 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x561d4e7fc080 0x561d4e7efdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:08.617 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:08.610+0000 7fd495f7b700 1 -- 172.21.15.59:0/648538307 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x561d4e7fc080 msgr2=0x561d4e7efdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T07:58:08.617 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:08.610+0000 7fd495f7b700 1 --2- 172.21.15.59:0/648538307 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x561d4e7fc080 0x561d4e7efdf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:08.617 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:08.610+0000 7fd495f7b700 1 -- 172.21.15.59:0/648538307 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x561d4e7f9490 msgr2=0x561d4e8a95d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T07:58:08.617 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:08.610+0000 7fd495f7b700 1 --2- 172.21.15.59:0/648538307 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x561d4e7f9490 0x561d4e8a95d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:08.617 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:08.610+0000 7fd495f7b700 1 -- 172.21.15.59:0/648538307 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd48c085070 con 0x561d4e811ed0 2023-11-24T07:58:08.617 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:08.611+0000 7fd495f7b700 1 --2- 172.21.15.59:0/648538307 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561d4e811ed0 0x561d4e836a40 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7fd488014000 tx=0x7fd488004b40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3aa8bc8f4d44af73 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:58:08.617 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:08.611+0000 7fd46ffff700 10 client.0 ms_handle_connect on v2:172.21.15.59:3300/0 2023-11-24T07:58:08.617 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:08.611+0000 7fd46ffff700 1 -- 172.21.15.59:0/648538307 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd488004240 con 0x561d4e811ed0 2023-11-24T07:58:08.618 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:08.611+0000 7fd4a3bd7b80 1 -- 172.21.15.59:0/648538307 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fd48c060a30 con 0x561d4e811ed0 2023-11-24T07:58:08.618 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:08.611+0000 7fd46ffff700 1 -- 172.21.15.59:0/648538307 <== mon.0 v2:172.21.15.59:3300/0 2 ==== osd_map(45..45 src has 1..45) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fd4880043a0 con 0x561d4e811ed0 2023-11-24T07:58:08.618 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:08.611+0000 7fd46ffff700 1 -- 172.21.15.59:0/648538307 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd488004560 con 0x561d4e811ed0 2023-11-24T07:58:08.618 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:08.611+0000 7fd4a3bd7b80 10 client.5778 fetch_fsmap learned FSMap version 25 2023-11-24T07:58:08.618 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:08.611+0000 7fd4a3bd7b80 1 -- 172.21.15.59:0/648538307 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=2+}) v3 -- 0x561d4e59f200 con 0x561d4e811ed0 2023-11-24T07:58:08.618 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:08.611+0000 7fd46ffff700 1 -- 172.21.15.59:0/648538307 <== mon.0 v2:172.21.15.59:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd48c060a30 con 0x561d4e811ed0 2023-11-24T07:58:08.618 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:08.611+0000 7fd46ffff700 1 -- 172.21.15.59:0/648538307 <== mon.0 v2:172.21.15.59:3300/0 5 ==== fsmap(e 25) v1 ==== 3294+0+0 (secure 0 0 0) 0x7fd48800d180 con 0x561d4e811ed0 2023-11-24T07:58:08.618 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:08.611+0000 7fd4a3bd7b80 10 client.5778 fetch_fsmap finished waiting for FSMap version 25 2023-11-24T07:58:08.618 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:08.611+0000 7fd4a3bd7b80 10 client.5778 resolve_mds: resolved name 'c' to daemon mds.c 2023-11-24T07:58:08.618 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:08.611+0000 7fd4a3bd7b80 20 client.5778 populate_metadata read hostname 'smithi059' 2023-11-24T07:58:08.618 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:08.611+0000 7fd4a3bd7b80 1 --2- 172.21.15.59:0/648538307 >> [v2:172.21.15.59:6833/1468266626,v1:172.21.15.59:6835/1468266626] conn(0x561d4e841e20 0x561d4e842200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:08.618 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:08.611+0000 7fd4a3bd7b80 4 client.5778 mds_command: new command op to 5625 tid=0 multi_id=0 [{"prefix": "status"}] 2023-11-24T07:58:08.618 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:08.611+0000 7fd4a3bd7b80 1 -- 172.21.15.59:0/648538307 --> [v2:172.21.15.59:6833/1468266626,v1:172.21.15.59:6835/1468266626] -- command(tid 0: {"prefix": "status"}) v1 -- 0x561d4e715f80 con 0x561d4e841e20 2023-11-24T07:58:08.619 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:08.611+0000 7fd49577a700 1 --2- 172.21.15.59:0/648538307 >> [v2:172.21.15.59:6833/1468266626,v1:172.21.15.59:6835/1468266626] conn(0x561d4e841e20 0x561d4e842200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:08.619 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:08.612+0000 7fd46ffff700 10 client.5778 ms_handle_connect on v2:172.21.15.59:6833/1468266626 2023-11-24T07:58:08.619 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:08.612+0000 7fd49577a700 1 --2- 172.21.15.59:0/648538307 >> [v2:172.21.15.59:6833/1468266626,v1:172.21.15.59:6835/1468266626] conn(0x561d4e841e20 0x561d4e842200 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=74524112b715480e server_cookie=b6b4c3485031550f in_seq=0 out_seq=0 2023-11-24T07:58:08.619 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:08.612+0000 7fd46ffff700 1 -- 172.21.15.59:0/648538307 <== mds.0 v2:172.21.15.59:6833/1468266626 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+325 (crc 0 0 0) 0x561d4e715f80 con 0x561d4e841e20 2023-11-24T07:58:08.619 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:08.612+0000 7fd46ffff700 10 client.5778 handle_command_reply: tid=0 2023-11-24T07:58:08.619 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:08.612+0000 7fd4a3bd7b80 1 client.5778 shutdown 2023-11-24T07:58:08.619 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:08.612+0000 7fd4a3bd7b80 1 -- 172.21.15.59:0/648538307 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561d4e811ed0 msgr2=0x561d4e836a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:08.619 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:08.612+0000 7fd4a3bd7b80 1 --2- 172.21.15.59:0/648538307 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561d4e811ed0 0x561d4e836a40 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7fd488014000 tx=0x7fd488004b40 comp rx=0 tx=0).stop 2023-11-24T07:58:08.619 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:08.612+0000 7fd4a3bd7b80 1 -- 172.21.15.59:0/648538307 shutdown_connections 2023-11-24T07:58:08.619 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:08.612+0000 7fd4a3bd7b80 1 --2- 172.21.15.59:0/648538307 >> [v2:172.21.15.59:6833/1468266626,v1:172.21.15.59:6835/1468266626] conn(0x561d4e841e20 0x561d4e842200 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:08.619 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:08.612+0000 7fd4a3bd7b80 1 --2- 172.21.15.59:0/648538307 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561d4e811ed0 0x561d4e836a40 unknown :-1 s=CLOSED pgs=562 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:08.619 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:08.612+0000 7fd4a3bd7b80 1 --2- 172.21.15.59:0/648538307 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x561d4e7f9490 0x561d4e8a95d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:08.619 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:08.612+0000 7fd4a3bd7b80 1 --2- 172.21.15.59:0/648538307 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x561d4e7fc080 0x561d4e7efdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:08.620 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:08.612+0000 7fd4a3bd7b80 1 -- 172.21.15.59:0/648538307 >> 172.21.15.59:0/648538307 conn(0x561d4e72a310 msgr2=0x561d4e7f7dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:58:08.620 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:08.612+0000 7fd46ffff700 0 client.5778 ms_handle_reset on v2:172.21.15.59:6833/1468266626 2023-11-24T07:58:08.620 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:08.612+0000 7fd4a3bd7b80 1 -- 172.21.15.59:0/648538307 shutdown_connections 2023-11-24T07:58:08.620 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:08.612+0000 7fd4a3bd7b80 1 -- 172.21.15.59:0/648538307 wait complete. 2023-11-24T07:58:08.620 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:08.612+0000 7fd4a3bd7b80 20 client.5778 trim_cache size 0 max 16384 2023-11-24T07:58:08.620 INFO:teuthology.orchestra.run.smithi059.stdout:{ 2023-11-24T07:58:08.620 INFO:teuthology.orchestra.run.smithi059.stdout: "cluster_fsid": "9f0e8741-b8f9-48d5-a2c3-09f45903303d", 2023-11-24T07:58:08.620 INFO:teuthology.orchestra.run.smithi059.stdout: "whoami": 0, 2023-11-24T07:58:08.620 INFO:teuthology.orchestra.run.smithi059.stdout: "id": 5625, 2023-11-24T07:58:08.621 INFO:teuthology.orchestra.run.smithi059.stdout: "want_state": "up:active", 2023-11-24T07:58:08.621 INFO:teuthology.orchestra.run.smithi059.stdout: "state": "up:active", 2023-11-24T07:58:08.621 INFO:teuthology.orchestra.run.smithi059.stdout: "fs_name": "cephfs", 2023-11-24T07:58:08.621 INFO:teuthology.orchestra.run.smithi059.stdout: "rank_uptime": 6.0074592620000002, 2023-11-24T07:58:08.621 INFO:teuthology.orchestra.run.smithi059.stdout: "mdsmap_epoch": 25, 2023-11-24T07:58:08.621 INFO:teuthology.orchestra.run.smithi059.stdout: "osdmap_epoch": 45, 2023-11-24T07:58:08.621 INFO:teuthology.orchestra.run.smithi059.stdout: "osdmap_epoch_barrier": 45, 2023-11-24T07:58:08.621 INFO:teuthology.orchestra.run.smithi059.stdout: "uptime": 11.928271012 2023-11-24T07:58:08.621 INFO:teuthology.orchestra.run.smithi059.stdout:} 2023-11-24T07:58:08.622 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:08.613+0000 7fd4979de700 1 -- 172.21.15.59:0/3384845750 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7fd47406af60 msgr2=0x7fd47406d420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:08.622 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:08.613+0000 7fd4979de700 1 --2- 172.21.15.59:0/3384845750 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7fd47406af60 0x7fd47406d420 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7fd490072820 tx=0x7fd480009040 comp rx=0 tx=0).stop 2023-11-24T07:58:08.622 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:08.613+0000 7fd4979de700 1 -- 172.21.15.59:0/3384845750 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd49006f420 msgr2=0x7fd49015c350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:08.622 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:08.613+0000 7fd4979de700 1 --2- 172.21.15.59:0/3384845750 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd49006f420 0x7fd49015c350 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7fd48c00d800 tx=0x7fd48c00dbc0 comp rx=0 tx=0).stop 2023-11-24T07:58:08.622 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:08.613+0000 7fd4979de700 1 -- 172.21.15.59:0/3384845750 shutdown_connections 2023-11-24T07:58:08.622 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:08.613+0000 7fd4979de700 1 --2- 172.21.15.59:0/3384845750 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7fd47406af60 0x7fd47406d420 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:08.622 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:08.613+0000 7fd4979de700 1 --2- 172.21.15.59:0/3384845750 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fd4900b4c40 0x7fd4900d00d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:08.622 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:08.613+0000 7fd4979de700 1 --2- 172.21.15.59:0/3384845750 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd49006f420 0x7fd49015c350 unknown :-1 s=CLOSED pgs=558 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:08.623 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:08.613+0000 7fd4979de700 1 --2- 172.21.15.59:0/3384845750 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd4900b3b40 0x7fd49015a0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:08.623 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:08.613+0000 7fd4979de700 1 -- 172.21.15.59:0/3384845750 >> 172.21.15.59:0/3384845750 conn(0x7fd4900b2840 msgr2=0x7fd49015e4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:58:08.623 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:08.613+0000 7fd4979de700 1 -- 172.21.15.59:0/3384845750 shutdown_connections 2023-11-24T07:58:08.623 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:08.613+0000 7fd4979de700 1 -- 172.21.15.59:0/3384845750 wait complete. 2023-11-24T07:58:08.627 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-11-24T07:58:08.752 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:08.750+0000 7fe198f81700 1 Processor -- start 2023-11-24T07:58:08.752 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:08.750+0000 7fe198f81700 1 -- start start 2023-11-24T07:58:08.754 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:08.751+0000 7fe198f81700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fe194143ae0 0x7fe1940b51b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:08.754 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:08.751+0000 7fe198f81700 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fe1941440f0 0x7fe194071bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:08.754 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:08.751+0000 7fe198f81700 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fe1940720f0 0x7fe19415e8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:08.754 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:08.751+0000 7fe198f81700 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe1940b40a0 con 0x7fe194143ae0 2023-11-24T07:58:08.755 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:08.751+0000 7fe198f81700 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe194072580 con 0x7fe1941440f0 2023-11-24T07:58:08.755 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:08.751+0000 7fe198f81700 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe194143ec0 con 0x7fe1940720f0 2023-11-24T07:58:08.755 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:08.751+0000 7fe19259c700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fe194143ae0 0x7fe1940b51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:08.755 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:08.751+0000 7fe19259c700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fe194143ae0 0x7fe1940b51b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:52898/0 (socket says 172.21.15.59:52898) 2023-11-24T07:58:08.755 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:08.751+0000 7fe19259c700 1 -- 172.21.15.59:0/1593797659 learned_addr learned my addr 172.21.15.59:0/1593797659 (peer_addr_for_me v2:172.21.15.59:0/0) 2023-11-24T07:58:08.755 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:08.751+0000 7fe191d9b700 1 --2- 172.21.15.59:0/1593797659 >> v2:172.21.15.76:3300/0 conn(0x7fe1941440f0 0x7fe194071bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:08.755 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:08.751+0000 7fe192d9d700 1 --2- 172.21.15.59:0/1593797659 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fe1940720f0 0x7fe19415e8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:08.755 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:08.751+0000 7fe19259c700 1 -- 172.21.15.59:0/1593797659 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fe1940720f0 msgr2=0x7fe19415e8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T07:58:08.755 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:08.751+0000 7fe19259c700 1 --2- 172.21.15.59:0/1593797659 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fe1940720f0 0x7fe19415e8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:08.755 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:08.751+0000 7fe19259c700 1 -- 172.21.15.59:0/1593797659 >> v2:172.21.15.76:3300/0 conn(0x7fe1941440f0 msgr2=0x7fe194071bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T07:58:08.755 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:08.751+0000 7fe19259c700 1 --2- 172.21.15.59:0/1593797659 >> v2:172.21.15.76:3300/0 conn(0x7fe1941440f0 0x7fe194071bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:08.755 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:08.751+0000 7fe19259c700 1 -- 172.21.15.59:0/1593797659 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe1940b3ae0 con 0x7fe194143ae0 2023-11-24T07:58:08.755 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:08.751+0000 7fe192d9d700 1 --2- 172.21.15.59:0/1593797659 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fe1940720f0 0x7fe19415e8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-24T07:58:08.756 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:08.751+0000 7fe191d9b700 1 --2- 172.21.15.59:0/1593797659 >> v2:172.21.15.76:3300/0 conn(0x7fe1941440f0 0x7fe194071bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-24T07:58:08.756 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:08.752+0000 7fe19259c700 1 --2- 172.21.15.59:0/1593797659 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe194143ae0 0x7fe1940b51b0 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7fe1940d1b00 tx=0x7fe188009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7d54de0da82b22da server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:58:08.756 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:08.752+0000 7fe19159a700 1 -- 172.21.15.59:0/1593797659 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe18800b870 con 0x7fe194143ae0 2023-11-24T07:58:08.756 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:08.752+0000 7fe19159a700 1 -- 172.21.15.59:0/1593797659 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe18800b9d0 con 0x7fe194143ae0 2023-11-24T07:58:08.756 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:08.752+0000 7fe19159a700 1 -- 172.21.15.59:0/1593797659 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe18800bd60 con 0x7fe194143ae0 2023-11-24T07:58:08.756 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:08.752+0000 7fe198f81700 1 -- 172.21.15.59:0/1593797659 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe194143ae0 msgr2=0x7fe1940b51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:08.756 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:08.752+0000 7fe198f81700 1 --2- 172.21.15.59:0/1593797659 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe194143ae0 0x7fe1940b51b0 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7fe1940d1b00 tx=0x7fe188009bd0 comp rx=0 tx=0).stop 2023-11-24T07:58:08.756 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:08.752+0000 7fe198f81700 1 -- 172.21.15.59:0/1593797659 shutdown_connections 2023-11-24T07:58:08.756 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:08.752+0000 7fe198f81700 1 --2- 172.21.15.59:0/1593797659 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fe1940720f0 0x7fe19415e8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:08.756 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:08.752+0000 7fe198f81700 1 --2- 172.21.15.59:0/1593797659 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe194143ae0 0x7fe1940b51b0 unknown :-1 s=CLOSED pgs=563 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:08.756 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:08.752+0000 7fe198f81700 1 --2- 172.21.15.59:0/1593797659 >> v2:172.21.15.76:3300/0 conn(0x7fe1941440f0 0x7fe194071bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:08.756 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:08.752+0000 7fe198f81700 1 -- 172.21.15.59:0/1593797659 >> 172.21.15.59:0/1593797659 conn(0x7fe1940b2740 msgr2=0x7fe1940b2b20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:58:08.756 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:08.752+0000 7fe198f81700 1 -- 172.21.15.59:0/1593797659 shutdown_connections 2023-11-24T07:58:08.756 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:08.752+0000 7fe198f81700 1 -- 172.21.15.59:0/1593797659 wait complete. 2023-11-24T07:58:08.759 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:08.754+0000 7fe198f81700 1 Processor -- start 2023-11-24T07:58:08.759 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:08.754+0000 7fe198f81700 1 -- start start 2023-11-24T07:58:08.759 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:08.755+0000 7fe198f81700 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fe1940720f0 0x7fe19413f550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:08.759 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:08.755+0000 7fe198f81700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe194143ae0 0x7fe1940c9dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:08.759 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:08.755+0000 7fe198f81700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe1941440f0 0x7fe1940cee00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:08.759 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:08.755+0000 7fe198f81700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe194156360 con 0x7fe194143ae0 2023-11-24T07:58:08.759 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:08.755+0000 7fe198f81700 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe19413fa90 con 0x7fe1941440f0 2023-11-24T07:58:08.759 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:08.755+0000 7fe198f81700 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe1940ca7a0 con 0x7fe1940720f0 2023-11-24T07:58:08.759 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:08.755+0000 7fe191d9b700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe194143ae0 0x7fe1940c9dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:08.759 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:08.755+0000 7fe191d9b700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe194143ae0 0x7fe1940c9dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:52906/0 (socket says 172.21.15.59:52906) 2023-11-24T07:58:08.760 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:08.755+0000 7fe191d9b700 1 -- 172.21.15.59:0/1692010877 learned_addr learned my addr 172.21.15.59:0/1692010877 (peer_addr_for_me v2:172.21.15.59:0/0) 2023-11-24T07:58:08.760 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:08.755+0000 7fe19259c700 1 --2- 172.21.15.59:0/1692010877 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fe1940720f0 0x7fe19413f550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:08.760 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:08.755+0000 7fe192d9d700 1 --2- 172.21.15.59:0/1692010877 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe1941440f0 0x7fe1940cee00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:08.760 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:08.755+0000 7fe191d9b700 1 -- 172.21.15.59:0/1692010877 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fe1940720f0 msgr2=0x7fe19413f550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:08.760 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:08.755+0000 7fe191d9b700 1 --2- 172.21.15.59:0/1692010877 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fe1940720f0 0x7fe19413f550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:08.760 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:08.755+0000 7fe191d9b700 1 -- 172.21.15.59:0/1692010877 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe1941440f0 msgr2=0x7fe1940cee00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:08.760 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:08.755+0000 7fe191d9b700 1 --2- 172.21.15.59:0/1692010877 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe1941440f0 0x7fe1940cee00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:08.760 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:08.755+0000 7fe191d9b700 1 -- 172.21.15.59:0/1692010877 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe188009550 con 0x7fe194143ae0 2023-11-24T07:58:08.760 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:08.755+0000 7fe191d9b700 1 --2- 172.21.15.59:0/1692010877 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe194143ae0 0x7fe1940c9dc0 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7fe18400e840 tx=0x7fe18400ec00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:58:08.760 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:08.755+0000 7fe182ffd700 1 -- 172.21.15.59:0/1692010877 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe184013610 con 0x7fe194143ae0 2023-11-24T07:58:08.760 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:08.755+0000 7fe182ffd700 1 -- 172.21.15.59:0/1692010877 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe184013770 con 0x7fe194143ae0 2023-11-24T07:58:08.760 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:08.755+0000 7fe198f81700 1 -- 172.21.15.59:0/1692010877 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe1940caa80 con 0x7fe194143ae0 2023-11-24T07:58:08.760 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:08.755+0000 7fe182ffd700 1 -- 172.21.15.59:0/1692010877 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe184013ae0 con 0x7fe194143ae0 2023-11-24T07:58:08.761 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:08.755+0000 7fe198f81700 1 -- 172.21.15.59:0/1692010877 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe1940cb040 con 0x7fe194143ae0 2023-11-24T07:58:08.761 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:08.756+0000 7fe198f81700 1 -- 172.21.15.59:0/1692010877 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe194068670 con 0x7fe194143ae0 2023-11-24T07:58:08.761 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:08.757+0000 7fe182ffd700 1 -- 172.21.15.59:0/1692010877 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 6) v1 ==== 89282+0+0 (secure 0 0 0) 0x7fe184013c40 con 0x7fe194143ae0 2023-11-24T07:58:08.761 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:08.757+0000 7fe182ffd700 1 --2- 172.21.15.59:0/1692010877 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7fe17406af90 0x7fe17406d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:08.761 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:08.757+0000 7fe182ffd700 1 -- 172.21.15.59:0/1692010877 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fe184089310 con 0x7fe194143ae0 2023-11-24T07:58:08.761 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:08.758+0000 7fe19259c700 1 --2- 172.21.15.59:0/1692010877 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7fe17406af90 0x7fe17406d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:08.761 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:08.758+0000 7fe19259c700 1 --2- 172.21.15.59:0/1692010877 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7fe17406af90 0x7fe17406d450 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7fe194158100 tx=0x7fe188008030 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:58:08.763 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:08.762+0000 7fe182ffd700 1 -- 172.21.15.59:0/1692010877 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fe184054350 con 0x7fe194143ae0 2023-11-24T07:58:09.010 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:09.008+0000 7fe198f81700 1 -- 172.21.15.59:0/1692010877 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7fe194156510 con 0x7fe194143ae0 2023-11-24T07:58:09.010 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:09.009+0000 7fe182ffd700 1 -- 172.21.15.59:0/1692010877 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v25) v1 ==== 53+0+73 (secure 0 0 0) 0x7fe194156510 con 0x7fe194143ae0 2023-11-24T07:58:09.011 INFO:teuthology.orchestra.run.smithi059.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-11-24T07:58:09.013 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:09.011+0000 7fe198f81700 1 -- 172.21.15.59:0/1692010877 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7fe17406af90 msgr2=0x7fe17406d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:09.013 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:09.011+0000 7fe198f81700 1 --2- 172.21.15.59:0/1692010877 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7fe17406af90 0x7fe17406d450 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7fe194158100 tx=0x7fe188008030 comp rx=0 tx=0).stop 2023-11-24T07:58:09.013 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:09.011+0000 7fe198f81700 1 -- 172.21.15.59:0/1692010877 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe194143ae0 msgr2=0x7fe1940c9dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:09.013 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:09.011+0000 7fe198f81700 1 --2- 172.21.15.59:0/1692010877 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe194143ae0 0x7fe1940c9dc0 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7fe18400e840 tx=0x7fe18400ec00 comp rx=0 tx=0).stop 2023-11-24T07:58:09.013 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:09.011+0000 7fe198f81700 1 -- 172.21.15.59:0/1692010877 shutdown_connections 2023-11-24T07:58:09.013 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:09.011+0000 7fe198f81700 1 --2- 172.21.15.59:0/1692010877 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7fe17406af90 0x7fe17406d450 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:09.013 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:09.011+0000 7fe198f81700 1 --2- 172.21.15.59:0/1692010877 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe1941440f0 0x7fe1940cee00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:09.013 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:09.011+0000 7fe198f81700 1 --2- 172.21.15.59:0/1692010877 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fe1940720f0 0x7fe19413f550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:09.013 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:09.011+0000 7fe198f81700 1 --2- 172.21.15.59:0/1692010877 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe194143ae0 0x7fe1940c9dc0 unknown :-1 s=CLOSED pgs=564 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:09.014 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:09.011+0000 7fe198f81700 1 -- 172.21.15.59:0/1692010877 >> 172.21.15.59:0/1692010877 conn(0x7fe1940b2740 msgr2=0x7fe19413fe80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:58:09.014 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:09.011+0000 7fe198f81700 1 -- 172.21.15.59:0/1692010877 shutdown_connections 2023-11-24T07:58:09.014 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:09.011+0000 7fe198f81700 1 -- 172.21.15.59:0/1692010877 wait complete. 2023-11-24T07:58:09.024 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2023-11-24T07:58:09.025 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-11-24T07:58:09.025 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi059.front.sepia.ceph.com 2023-11-24T07:58:09.025 INFO:tasks.cephfs.mount:self.client.name = client.0 2023-11-24T07:58:09.025 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-11-24T07:58:09.025 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2023-11-24T07:58:09.025 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2023-11-24T07:58:09.025 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-11-24T07:58:09.025 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-11-24T07:58:09.025 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T07:58:09.025 DEBUG:teuthology.orchestra.run.smithi059:> ip addr 2023-11-24T07:58:09.043 INFO:teuthology.orchestra.run.smithi059.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-11-24T07:58:09.044 INFO:teuthology.orchestra.run.smithi059.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-11-24T07:58:09.044 INFO:teuthology.orchestra.run.smithi059.stdout: inet 127.0.0.1/8 scope host lo 2023-11-24T07:58:09.044 INFO:teuthology.orchestra.run.smithi059.stdout: valid_lft forever preferred_lft forever 2023-11-24T07:58:09.044 INFO:teuthology.orchestra.run.smithi059.stdout: inet6 ::1/128 scope host 2023-11-24T07:58:09.044 INFO:teuthology.orchestra.run.smithi059.stdout: valid_lft forever preferred_lft forever 2023-11-24T07:58:09.044 INFO:teuthology.orchestra.run.smithi059.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-11-24T07:58:09.045 INFO:teuthology.orchestra.run.smithi059.stdout: link/ether 0c:c4:7a:6b:fe:ca brd ff:ff:ff:ff:ff:ff 2023-11-24T07:58:09.045 INFO:teuthology.orchestra.run.smithi059.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-11-24T07:58:09.045 INFO:teuthology.orchestra.run.smithi059.stdout: link/ether 0c:c4:7a:6b:fe:cb brd ff:ff:ff:ff:ff:ff 2023-11-24T07:58:09.045 INFO:teuthology.orchestra.run.smithi059.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2023-11-24T07:58:09.045 INFO:teuthology.orchestra.run.smithi059.stdout: link/ether 0c:c4:7a:bd:19:2a brd ff:ff:ff:ff:ff:ff 2023-11-24T07:58:09.045 INFO:teuthology.orchestra.run.smithi059.stdout: inet 172.21.15.59/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2023-11-24T07:58:09.045 INFO:teuthology.orchestra.run.smithi059.stdout: valid_lft 42278sec preferred_lft 42278sec 2023-11-24T07:58:09.045 INFO:teuthology.orchestra.run.smithi059.stdout: inet6 fe80::ec4:7aff:febd:192a/64 scope link 2023-11-24T07:58:09.045 INFO:teuthology.orchestra.run.smithi059.stdout: valid_lft forever preferred_lft forever 2023-11-24T07:58:09.045 INFO:teuthology.orchestra.run.smithi059.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-11-24T07:58:09.045 INFO:teuthology.orchestra.run.smithi059.stdout: link/ether 0c:c4:7a:bd:19:2b brd ff:ff:ff:ff:ff:ff 2023-11-24T07:58:09.046 INFO:teuthology.orchestra.run.smithi059.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-11-24T07:58:09.046 INFO:teuthology.orchestra.run.smithi059.stdout: link/ether 72:f2:ce:27:f6:18 brd ff:ff:ff:ff:ff:ff 2023-11-24T07:58:09.046 INFO:teuthology.orchestra.run.smithi059.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-11-24T07:58:09.046 INFO:teuthology.orchestra.run.smithi059.stdout: valid_lft forever preferred_lft forever 2023-11-24T07:58:09.046 INFO:teuthology.orchestra.run.smithi059.stdout: inet6 fe80::64c1:62ff:feb9:5b59/64 scope link 2023-11-24T07:58:09.046 INFO:teuthology.orchestra.run.smithi059.stdout: valid_lft forever preferred_lft forever 2023-11-24T07:58:09.046 INFO:teuthology.orchestra.run.smithi059.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-11-24T07:58:09.046 INFO:teuthology.orchestra.run.smithi059.stdout: link/ether 92:6a:7d:e0:5d:9b brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2023-11-24T07:58:09.046 INFO:teuthology.orchestra.run.smithi059.stdout: inet6 fe80::906a:7dff:fee0:5d9b/64 scope link 2023-11-24T07:58:09.046 INFO:teuthology.orchestra.run.smithi059.stdout: valid_lft forever preferred_lft forever 2023-11-24T07:58:09.047 INFO:teuthology.orchestra.run.smithi059.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-11-24T07:58:09.047 INFO:teuthology.orchestra.run.smithi059.stdout: link/ether 72:f2:ce:27:f6:18 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2023-11-24T07:58:09.047 INFO:teuthology.orchestra.run.smithi059.stdout: inet6 fe80::70f2:ceff:fe27:f618/64 scope link 2023-11-24T07:58:09.047 INFO:teuthology.orchestra.run.smithi059.stdout: valid_lft forever preferred_lft forever 2023-11-24T07:58:09.047 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T07:58:09.047 DEBUG:teuthology.orchestra.run.smithi059:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-11-24T07:58:09.110 INFO:teuthology.orchestra.run.smithi059.stdout:1 2023-11-24T07:58:09.111 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T07:58:09.112 DEBUG:teuthology.orchestra.run.smithi059:> route 2023-11-24T07:58:09.171 INFO:teuthology.orchestra.run.smithi059.stdout:Kernel IP routing table 2023-11-24T07:58:09.171 INFO:teuthology.orchestra.run.smithi059.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-11-24T07:58:09.172 INFO:teuthology.orchestra.run.smithi059.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2023-11-24T07:58:09.172 INFO:teuthology.orchestra.run.smithi059.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2023-11-24T07:58:09.172 INFO:teuthology.orchestra.run.smithi059.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-11-24T07:58:09.172 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T07:58:09.173 DEBUG:teuthology.orchestra.run.smithi059:> (cd / && exec bash -c ' 2023-11-24T07:58:09.173 DEBUG:teuthology.orchestra.run.smithi059:> set -e 2023-11-24T07:58:09.173 DEBUG:teuthology.orchestra.run.smithi059:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2023-11-24T07:58:09.173 DEBUG:teuthology.orchestra.run.smithi059:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2023-11-24T07:58:09.173 DEBUG:teuthology.orchestra.run.smithi059:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2023-11-24T07:58:09.173 DEBUG:teuthology.orchestra.run.smithi059:> ') 2023-11-24T07:58:09.277 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T07:58:09.278 DEBUG:teuthology.orchestra.run.smithi059:> ip netns list 2023-11-24T07:58:09.295 INFO:teuthology.orchestra.run.smithi059.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2023-11-24T07:58:09.295 INFO:teuthology.orchestra.run.smithi059.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-11-24T07:58:09.295 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-11-24T07:58:09.295 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T07:58:09.295 DEBUG:teuthology.orchestra.run.smithi059:> sudo ip link set brx.0 up 2023-11-24T07:58:09.363 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-11-24T07:58:09.363 DEBUG:teuthology.orchestra.run.smithi059:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2023-11-24T07:58:09.420 INFO:teuthology.orchestra.run.smithi059.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2023-11-24T07:58:09.421 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-11-24T07:58:09.421 DEBUG:teuthology.orchestra.run.smithi059:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2023-11-24T07:58:09.477 INFO:tasks.cephfs.kernel_mount:mounting using device: 0@.cephfs=/ 2023-11-24T07:58:09.477 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T07:58:09.477 DEBUG:teuthology.orchestra.run.smithi059:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 0@.cephfs=/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-11-24T07:58:09.564 INFO:teuthology.orchestra.run.smithi059.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-11-24T07:58:09.565 INFO:teuthology.orchestra.run.smithi059.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-11-24T07:58:09.594 INFO:teuthology.orchestra.run.smithi059.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-11-24T07:58:09.594 INFO:teuthology.orchestra.run.smithi059.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-11-24T07:58:09.594 INFO:teuthology.orchestra.run.smithi059.stdout:mount.ceph: resolved to: "172.21.15.59:3300,172.21.15.76:3300,172.21.15.76:3301" 2023-11-24T07:58:09.594 INFO:teuthology.orchestra.run.smithi059.stdout:mount.ceph: trying mount with new device syntax: 0@9f0e8741-b8f9-48d5-a2c3-09f45903303d.cephfs=/ 2023-11-24T07:58:09.594 INFO:teuthology.orchestra.run.smithi059.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,ms_mode=prefer-crc,key=0,mon_addr=172.21.15.59:3300/172.21.15.76:3300/172.21.15.76:3301" will pass to kernel 2023-11-24T07:58:09.594 INFO:teuthology.orchestra.run.smithi059.stdout:mount.ceph: v2 mount support check returned 2 2023-11-24T07:58:09.594 INFO:teuthology.orchestra.run.smithi059.stdout:mount.ceph: kernel does not support v2 syntax 2023-11-24T07:58:09.594 INFO:teuthology.orchestra.run.smithi059.stdout:mount.ceph: switching to using v1 address with old syntax 2023-11-24T07:58:09.668 INFO:teuthology.orchestra.run.smithi059.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-11-24T07:58:09.668 INFO:teuthology.orchestra.run.smithi059.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-11-24T07:58:09.668 INFO:teuthology.orchestra.run.smithi059.stdout:mount.ceph: resolved to: "172.21.15.59:3300,172.21.15.76:3300,172.21.15.76:3301" 2023-11-24T07:58:09.668 INFO:teuthology.orchestra.run.smithi059.stdout:mount.ceph: trying mount with new device syntax: 0@9f0e8741-b8f9-48d5-a2c3-09f45903303d.cephfs=/ 2023-11-24T07:58:09.668 INFO:teuthology.orchestra.run.smithi059.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,ms_mode=prefer-crc,key=0,mon_addr=172.21.15.59:3300/172.21.15.76:3300/172.21.15.76:3301" will pass to kernel 2023-11-24T07:58:09.668 INFO:teuthology.orchestra.run.smithi059.stdout:mount.ceph: v2 mount support check returned 2 2023-11-24T07:58:09.669 INFO:teuthology.orchestra.run.smithi059.stdout:mount.ceph: kernel does not support v2 syntax 2023-11-24T07:58:09.669 INFO:teuthology.orchestra.run.smithi059.stdout:mount.ceph: switching to using v1 address with old syntax 2023-11-24T07:58:09.669 INFO:teuthology.orchestra.run.smithi059.stdout:mount.ceph: resolved to: "172.21.15.59:6789,172.21.15.76:6789,172.21.15.76:6790" 2023-11-24T07:58:09.669 INFO:teuthology.orchestra.run.smithi059.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.59:6789,172.21.15.76:6789,172.21.15.76:6790:/ 2023-11-24T07:58:09.669 INFO:teuthology.orchestra.run.smithi059.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,key=0,mds_namespace=cephfs,fsid=9f0e8741-b8f9-48d5-a2c3-09f45903303d" will pass to kernel 2023-11-24T07:58:09.671 INFO:tasks.cephfs.kernel_mount:mount command passed 2023-11-24T07:58:09.671 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T07:58:09.671 DEBUG:teuthology.orchestra.run.smithi059:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2023-11-24T07:58:09.698 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T07:58:09.699 DEBUG:teuthology.orchestra.run.smithi059:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-11-24T07:58:09.781 INFO:teuthology.orchestra.run.smithi059.stdout:client5416 2023-11-24T07:58:09.782 DEBUG:teuthology.orchestra.run.smithi059:> set -ex 2023-11-24T07:58:09.782 DEBUG:teuthology.orchestra.run.smithi059:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-24T07:58:09.840 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-11-24T07:58:10.005 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:10.003+0000 7f5987dd3700 1 Processor -- start 2023-11-24T07:58:10.005 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:10.004+0000 7f5987dd3700 1 -- start start 2023-11-24T07:58:10.009 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:10.004+0000 7f5987dd3700 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f59800b0ac0 0x7f5980165dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:10.009 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:10.004+0000 7f5987dd3700 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f598015f6b0 con 0x7f598006c5d0 2023-11-24T07:58:10.009 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:10.004+0000 7f5987dd3700 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f598015eb40 con 0x7f598015afc0 2023-11-24T07:58:10.009 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:10.004+0000 7f5987dd3700 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f598015eda0 con 0x7f59800b0ac0 2023-11-24T07:58:10.009 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:10.005+0000 7f598536e700 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7f598006c5d0 0x7f59800b03b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:56392/0 (socket says 172.21.15.59:56392) 2023-11-24T07:58:10.009 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:10.005+0000 7f598536e700 1 -- 172.21.15.59:0/3926906033 learned_addr learned my addr 172.21.15.59:0/3926906033 (peer_addr_for_me v1:172.21.15.59:0/0) 2023-11-24T07:58:10.009 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:10.005+0000 7f5984b6d700 1 -- 172.21.15.59:0/3926906033 <== mon.0 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 777439775 0 0) 0x7f598015f6b0 con 0x7f598006c5d0 2023-11-24T07:58:10.009 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:10.005+0000 7f5984b6d700 1 -- 172.21.15.59:0/3926906033 --> v1:172.21.15.59:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f596c0035b0 con 0x7f598006c5d0 2023-11-24T07:58:10.009 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:10.005+0000 7f5986370700 1 --2- 172.21.15.59:0/3926906033 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f59800b0ac0 0x7f5980165dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:10.010 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:10.005+0000 7f5984b6d700 1 -- 172.21.15.59:0/3926906033 <== mon.0 v1:172.21.15.59:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 40777396 0 0) 0x7f596c0035b0 con 0x7f598006c5d0 2023-11-24T07:58:10.010 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:10.005+0000 7f5984b6d700 1 -- 172.21.15.59:0/3926906033 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f59800b0ac0 msgr2=0x7f5980165dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T07:58:10.010 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:10.005+0000 7f5984b6d700 1 --2- 172.21.15.59:0/3926906033 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f59800b0ac0 0x7f5980165dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:10.010 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:10.005+0000 7f5984b6d700 1 -- 172.21.15.59:0/3926906033 >> v1:172.21.15.76:6789/0 conn(0x7f598015afc0 legacy=0x7f59800730b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:10.010 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:10.005+0000 7f5984b6d700 1 -- 172.21.15.59:0/3926906033 --> v1:172.21.15.59:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f598015f060 con 0x7f598006c5d0 2023-11-24T07:58:10.010 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:10.005+0000 7f5984b6d700 1 -- 172.21.15.59:0/3926906033 <== mon.0 v1:172.21.15.59:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1312333566 0 0) 0x7f5974001670 con 0x7f598006c5d0 2023-11-24T07:58:10.010 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:10.005+0000 7f5984b6d700 1 -- 172.21.15.59:0/3926906033 >> v1:172.21.15.59:6789/0 conn(0x7f598006c5d0 legacy=0x7f59800b03b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:10.010 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:10.005+0000 7f5984b6d700 1 --2- 172.21.15.59:0/3926906033 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f596c003a30 0x7f596c003e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:10.010 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:10.005+0000 7f5984b6d700 1 --2- 172.21.15.59:0/3926906033 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f59800b0ac0 0x7f596c004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:10.010 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:10.005+0000 7f5984b6d700 1 --2- 172.21.15.59:0/3926906033 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f596c004d90 0x7f596c005210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:10.010 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:10.005+0000 7f5984b6d700 1 -- 172.21.15.59:0/3926906033 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f598015eda0 con 0x7f596c003a30 2023-11-24T07:58:10.011 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:10.005+0000 7f5984b6d700 1 -- 172.21.15.59:0/3926906033 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f596c00dd00 con 0x7f596c004d90 2023-11-24T07:58:10.011 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:10.005+0000 7f5984b6d700 1 -- 172.21.15.59:0/3926906033 --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f596c00dec0 con 0x7f59800b0ac0 2023-11-24T07:58:10.011 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:10.005+0000 7f5985b6f700 1 --2- 172.21.15.59:0/3926906033 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f596c003a30 0x7f596c003e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:10.011 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:10.005+0000 7f5986370700 1 --2- 172.21.15.59:0/3926906033 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f596c004d90 0x7f596c005210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:10.011 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:10.005+0000 7f598536e700 1 --2- 172.21.15.59:0/3926906033 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f59800b0ac0 0x7f596c004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:10.011 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:10.006+0000 7f5985b6f700 1 -- 172.21.15.59:0/3926906033 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f59800b0ac0 msgr2=0x7f596c004850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T07:58:10.011 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:10.006+0000 7f5985b6f700 1 --2- 172.21.15.59:0/3926906033 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f59800b0ac0 0x7f596c004850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:10.011 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:10.006+0000 7f5985b6f700 1 -- 172.21.15.59:0/3926906033 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f596c004d90 msgr2=0x7f596c005210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T07:58:10.012 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:10.006+0000 7f5985b6f700 1 --2- 172.21.15.59:0/3926906033 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f596c004d90 0x7f596c005210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:10.012 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:10.006+0000 7f5985b6f700 1 -- 172.21.15.59:0/3926906033 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f596c0035b0 con 0x7f596c003a30 2023-11-24T07:58:10.012 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:10.006+0000 7f5985b6f700 1 --2- 172.21.15.59:0/3926906033 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f596c003a30 0x7f596c003e50 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7f597c007fd0 tx=0x7f596c005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7e6bb42ca856e5ee server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:58:10.012 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:10.006+0000 7f5984b6d700 1 -- 172.21.15.59:0/3926906033 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f597c0052d0 con 0x7f596c003a30 2023-11-24T07:58:10.012 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:10.006+0000 7f5984b6d700 1 -- 172.21.15.59:0/3926906033 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f597c005470 con 0x7f596c003a30 2023-11-24T07:58:10.012 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:10.006+0000 7f5984b6d700 1 -- 172.21.15.59:0/3926906033 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f597c0038c0 con 0x7f596c003a30 2023-11-24T07:58:10.012 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:10.006+0000 7f5987dd3700 1 -- 172.21.15.59:0/3926906033 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f596c003a30 msgr2=0x7f596c003e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:10.012 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:10.006+0000 7f5987dd3700 1 --2- 172.21.15.59:0/3926906033 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f596c003a30 0x7f596c003e50 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7f597c007fd0 tx=0x7f596c005880 comp rx=0 tx=0).stop 2023-11-24T07:58:10.012 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:10.006+0000 7f5986370700 1 -- 172.21.15.59:0/3926906033 reap_dead start 2023-11-24T07:58:10.012 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:10.006+0000 7f5987dd3700 1 -- 172.21.15.59:0/3926906033 shutdown_connections 2023-11-24T07:58:10.013 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:10.006+0000 7f5987dd3700 1 -- 172.21.15.59:0/3926906033 >> 172.21.15.59:0/3926906033 conn(0x7f59800717f0 msgr2=0x7f59800b5070 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:58:10.013 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:10.007+0000 7f5987dd3700 1 -- 172.21.15.59:0/3926906033 shutdown_connections 2023-11-24T07:58:10.013 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:10.007+0000 7f5987dd3700 1 -- 172.21.15.59:0/3926906033 wait complete. 2023-11-24T07:58:10.016 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:10.008+0000 7f5987dd3700 1 Processor -- start 2023-11-24T07:58:10.016 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:10.008+0000 7f5987dd3700 1 -- start start 2023-11-24T07:58:10.016 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:10.009+0000 7f5987dd3700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5980150c70 0x7f59800af770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:10.016 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:10.009+0000 7f5987dd3700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5980151050 0x7f5980153360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:10.016 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:10.009+0000 7f5987dd3700 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f59801514a0 0x7f59801538a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:10.016 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:10.009+0000 7f5987dd3700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59800ae9b0 con 0x7f5980150c70 2023-11-24T07:58:10.016 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:10.009+0000 7f5987dd3700 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59800aec90 con 0x7f5980151050 2023-11-24T07:58:10.016 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:10.009+0000 7f5987dd3700 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f59800aef70 con 0x7f59801514a0 2023-11-24T07:58:10.016 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:10.009+0000 7f5985b6f700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5980150c70 0x7f59800af770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:10.017 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:10.009+0000 7f5985b6f700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5980150c70 0x7f59800af770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:52928/0 (socket says 172.21.15.59:52928) 2023-11-24T07:58:10.017 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:10.009+0000 7f5985b6f700 1 -- 172.21.15.59:0/2243383245 learned_addr learned my addr 172.21.15.59:0/2243383245 (peer_addr_for_me v2:172.21.15.59:0/0) 2023-11-24T07:58:10.017 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:10.009+0000 7f598536e700 1 --2- 172.21.15.59:0/2243383245 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5980151050 0x7f5980153360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:10.017 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:10.009+0000 7f5985b6f700 1 -- 172.21.15.59:0/2243383245 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f59801514a0 msgr2=0x7f59801538a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:10.017 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:10.009+0000 7f5986370700 1 --2- 172.21.15.59:0/2243383245 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f59801514a0 0x7f59801538a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:10.017 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:10.009+0000 7f5985b6f700 1 --2- 172.21.15.59:0/2243383245 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f59801514a0 0x7f59801538a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:10.017 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:10.009+0000 7f5985b6f700 1 -- 172.21.15.59:0/2243383245 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5980151050 msgr2=0x7f5980153360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:10.017 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:10.009+0000 7f5985b6f700 1 --2- 172.21.15.59:0/2243383245 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5980151050 0x7f5980153360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:10.017 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:10.009+0000 7f5985b6f700 1 -- 172.21.15.59:0/2243383245 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f59800af250 con 0x7f5980150c70 2023-11-24T07:58:10.017 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:10.009+0000 7f598536e700 1 --2- 172.21.15.59:0/2243383245 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5980151050 0x7f5980153360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T07:58:10.017 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:10.009+0000 7f5985b6f700 1 --2- 172.21.15.59:0/2243383245 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5980150c70 0x7f59800af770 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7f597c004000 tx=0x7f597c001120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:58:10.017 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:10.009+0000 7f59727fc700 1 -- 172.21.15.59:0/2243383245 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f597c007240 con 0x7f5980150c70 2023-11-24T07:58:10.018 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:10.009+0000 7f59727fc700 1 -- 172.21.15.59:0/2243383245 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f597c0073e0 con 0x7f5980150c70 2023-11-24T07:58:10.018 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:10.009+0000 7f5987dd3700 1 -- 172.21.15.59:0/2243383245 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f59800afcb0 con 0x7f5980150c70 2023-11-24T07:58:10.018 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:10.009+0000 7f59727fc700 1 -- 172.21.15.59:0/2243383245 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f597c001290 con 0x7f5980150c70 2023-11-24T07:58:10.018 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:10.009+0000 7f5987dd3700 1 -- 172.21.15.59:0/2243383245 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5980159f70 con 0x7f5980150c70 2023-11-24T07:58:10.018 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:10.010+0000 7f5987dd3700 1 -- 172.21.15.59:0/2243383245 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f59800af3e0 con 0x7f5980150c70 2023-11-24T07:58:10.018 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:10.011+0000 7f59727fc700 1 -- 172.21.15.59:0/2243383245 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 6) v1 ==== 89282+0+0 (secure 0 0 0) 0x7f597c019070 con 0x7f5980150c70 2023-11-24T07:58:10.018 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:10.011+0000 7f59727fc700 1 --2- 172.21.15.59:0/2243383245 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f596406af90 0x7f596406d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:10.018 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:10.011+0000 7f59727fc700 1 -- 172.21.15.59:0/2243383245 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f597c089c60 con 0x7f5980150c70 2023-11-24T07:58:10.018 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:10.011+0000 7f598536e700 1 --2- 172.21.15.59:0/2243383245 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f596406af90 0x7f596406d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:10.018 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:10.012+0000 7f598536e700 1 --2- 172.21.15.59:0/2243383245 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f596406af90 0x7f596406d450 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f5974001c90 tx=0x7f5974009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:58:10.019 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:10.016+0000 7f59727fc700 1 -- 172.21.15.59:0/2243383245 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f597c054cb0 con 0x7f5980150c70 2023-11-24T07:58:10.263 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:10.262+0000 7f5987dd3700 1 -- 172.21.15.59:0/2243383245 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f5980068670 con 0x7f5980150c70 2023-11-24T07:58:10.264 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:10.262+0000 7f59727fc700 1 -- 172.21.15.59:0/2243383245 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v25) v1 ==== 53+0+73 (secure 0 0 0) 0x7f597c02a060 con 0x7f5980150c70 2023-11-24T07:58:10.264 INFO:teuthology.orchestra.run.smithi059.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-11-24T07:58:10.266 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:10.264+0000 7f5987dd3700 1 -- 172.21.15.59:0/2243383245 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f596406af90 msgr2=0x7f596406d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:10.266 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:10.264+0000 7f5987dd3700 1 --2- 172.21.15.59:0/2243383245 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f596406af90 0x7f596406d450 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f5974001c90 tx=0x7f5974009040 comp rx=0 tx=0).stop 2023-11-24T07:58:10.266 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:10.264+0000 7f5987dd3700 1 -- 172.21.15.59:0/2243383245 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5980150c70 msgr2=0x7f59800af770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:10.267 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:10.264+0000 7f5987dd3700 1 --2- 172.21.15.59:0/2243383245 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5980150c70 0x7f59800af770 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7f597c004000 tx=0x7f597c001120 comp rx=0 tx=0).stop 2023-11-24T07:58:10.267 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:10.264+0000 7f5987dd3700 1 -- 172.21.15.59:0/2243383245 shutdown_connections 2023-11-24T07:58:10.267 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:10.264+0000 7f5987dd3700 1 --2- 172.21.15.59:0/2243383245 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f596406af90 0x7f596406d450 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:10.267 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:10.264+0000 7f5987dd3700 1 --2- 172.21.15.59:0/2243383245 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f59801514a0 0x7f59801538a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:10.267 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:10.264+0000 7f5987dd3700 1 --2- 172.21.15.59:0/2243383245 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5980150c70 0x7f59800af770 unknown :-1 s=CLOSED pgs=567 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:10.267 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:10.264+0000 7f5987dd3700 1 --2- 172.21.15.59:0/2243383245 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5980151050 0x7f5980153360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:10.267 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:10.264+0000 7f5987dd3700 1 -- 172.21.15.59:0/2243383245 >> 172.21.15.59:0/2243383245 conn(0x7f59800717f0 msgr2=0x7f598015cb70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:58:10.267 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:10.264+0000 7f5987dd3700 1 -- 172.21.15.59:0/2243383245 shutdown_connections 2023-11-24T07:58:10.267 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:10.264+0000 7f5987dd3700 1 -- 172.21.15.59:0/2243383245 wait complete. 2023-11-24T07:58:10.278 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2023-11-24T07:58:10.278 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-11-24T07:58:10.278 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi076.front.sepia.ceph.com 2023-11-24T07:58:10.278 INFO:tasks.cephfs.mount:self.client.name = client.1 2023-11-24T07:58:10.278 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2023-11-24T07:58:10.278 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2023-11-24T07:58:10.278 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2023-11-24T07:58:10.279 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-11-24T07:58:10.279 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2023-11-24T07:58:10.279 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T07:58:10.279 DEBUG:teuthology.orchestra.run.smithi076:> ip addr 2023-11-24T07:58:10.300 INFO:teuthology.orchestra.run.smithi076.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-11-24T07:58:10.301 INFO:teuthology.orchestra.run.smithi076.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-11-24T07:58:10.301 INFO:teuthology.orchestra.run.smithi076.stdout: inet 127.0.0.1/8 scope host lo 2023-11-24T07:58:10.301 INFO:teuthology.orchestra.run.smithi076.stdout: valid_lft forever preferred_lft forever 2023-11-24T07:58:10.301 INFO:teuthology.orchestra.run.smithi076.stdout: inet6 ::1/128 scope host 2023-11-24T07:58:10.301 INFO:teuthology.orchestra.run.smithi076.stdout: valid_lft forever preferred_lft forever 2023-11-24T07:58:10.301 INFO:teuthology.orchestra.run.smithi076.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-11-24T07:58:10.301 INFO:teuthology.orchestra.run.smithi076.stdout: link/ether 0c:c4:7a:d8:da:66 brd ff:ff:ff:ff:ff:ff 2023-11-24T07:58:10.301 INFO:teuthology.orchestra.run.smithi076.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-11-24T07:58:10.301 INFO:teuthology.orchestra.run.smithi076.stdout: link/ether 0c:c4:7a:d8:da:67 brd ff:ff:ff:ff:ff:ff 2023-11-24T07:58:10.301 INFO:teuthology.orchestra.run.smithi076.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-11-24T07:58:10.301 INFO:teuthology.orchestra.run.smithi076.stdout: link/ether 0c:c4:7a:8f:47:e0 brd ff:ff:ff:ff:ff:ff 2023-11-24T07:58:10.302 INFO:teuthology.orchestra.run.smithi076.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-11-24T07:58:10.302 INFO:teuthology.orchestra.run.smithi076.stdout: link/ether 0c:c4:7a:8f:47:e1 brd ff:ff:ff:ff:ff:ff 2023-11-24T07:58:10.302 INFO:teuthology.orchestra.run.smithi076.stdout: inet 172.21.15.76/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2023-11-24T07:58:10.302 INFO:teuthology.orchestra.run.smithi076.stdout: valid_lft 42309sec preferred_lft 42309sec 2023-11-24T07:58:10.302 INFO:teuthology.orchestra.run.smithi076.stdout: inet6 fe80::a34d:4ce5:8360:8f9/64 scope link noprefixroute 2023-11-24T07:58:10.302 INFO:teuthology.orchestra.run.smithi076.stdout: valid_lft forever preferred_lft forever 2023-11-24T07:58:10.302 INFO:teuthology.orchestra.run.smithi076.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-11-24T07:58:10.302 INFO:teuthology.orchestra.run.smithi076.stdout: link/ether 2a:59:73:ff:f7:95 brd ff:ff:ff:ff:ff:ff 2023-11-24T07:58:10.302 INFO:teuthology.orchestra.run.smithi076.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-11-24T07:58:10.302 INFO:teuthology.orchestra.run.smithi076.stdout: valid_lft forever preferred_lft forever 2023-11-24T07:58:10.302 INFO:teuthology.orchestra.run.smithi076.stdout: inet6 fe80::4c23:44ff:feb9:d9d4/64 scope link 2023-11-24T07:58:10.303 INFO:teuthology.orchestra.run.smithi076.stdout: valid_lft forever preferred_lft forever 2023-11-24T07:58:10.303 INFO:teuthology.orchestra.run.smithi076.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-11-24T07:58:10.303 INFO:teuthology.orchestra.run.smithi076.stdout: link/ether 46:3e:09:af:65:64 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2023-11-24T07:58:10.303 INFO:teuthology.orchestra.run.smithi076.stdout: inet6 fe80::443e:9ff:feaf:6564/64 scope link 2023-11-24T07:58:10.303 INFO:teuthology.orchestra.run.smithi076.stdout: valid_lft forever preferred_lft forever 2023-11-24T07:58:10.303 INFO:teuthology.orchestra.run.smithi076.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-11-24T07:58:10.303 INFO:teuthology.orchestra.run.smithi076.stdout: link/ether 2a:59:73:ff:f7:95 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.3 2023-11-24T07:58:10.303 INFO:teuthology.orchestra.run.smithi076.stdout: inet6 fe80::2859:73ff:feff:f795/64 scope link 2023-11-24T07:58:10.303 INFO:teuthology.orchestra.run.smithi076.stdout: valid_lft forever preferred_lft forever 2023-11-24T07:58:10.304 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T07:58:10.304 DEBUG:teuthology.orchestra.run.smithi076:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-11-24T07:58:10.372 INFO:teuthology.orchestra.run.smithi076.stdout:1 2023-11-24T07:58:10.374 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T07:58:10.374 DEBUG:teuthology.orchestra.run.smithi076:> route 2023-11-24T07:58:10.436 INFO:teuthology.orchestra.run.smithi076.stdout:Kernel IP routing table 2023-11-24T07:58:10.436 INFO:teuthology.orchestra.run.smithi076.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-11-24T07:58:10.437 INFO:teuthology.orchestra.run.smithi076.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-11-24T07:58:10.437 INFO:teuthology.orchestra.run.smithi076.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-11-24T07:58:10.437 INFO:teuthology.orchestra.run.smithi076.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-11-24T07:58:10.437 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T07:58:10.437 DEBUG:teuthology.orchestra.run.smithi076:> (cd / && exec bash -c ' 2023-11-24T07:58:10.437 DEBUG:teuthology.orchestra.run.smithi076:> set -e 2023-11-24T07:58:10.437 DEBUG:teuthology.orchestra.run.smithi076:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-11-24T07:58:10.438 DEBUG:teuthology.orchestra.run.smithi076:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-11-24T07:58:10.438 DEBUG:teuthology.orchestra.run.smithi076:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-11-24T07:58:10.438 DEBUG:teuthology.orchestra.run.smithi076:> ') 2023-11-24T07:58:10.556 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T07:58:10.556 DEBUG:teuthology.orchestra.run.smithi076:> ip netns list 2023-11-24T07:58:10.576 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-ns--home-ubuntu-cephtest-mnt.3 (id: 1) 2023-11-24T07:58:10.577 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2023-11-24T07:58:10.577 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2023-11-24T07:58:10.577 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T07:58:10.577 DEBUG:teuthology.orchestra.run.smithi076:> sudo ip link set brx.0 up 2023-11-24T07:58:10.650 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-11-24T07:58:10.650 DEBUG:teuthology.orchestra.run.smithi076:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2023-11-24T07:58:10.708 INFO:teuthology.orchestra.run.smithi076.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2023-11-24T07:58:10.709 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-11-24T07:58:10.709 DEBUG:teuthology.orchestra.run.smithi076:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2023-11-24T07:58:10.767 INFO:tasks.cephfs.kernel_mount:mounting using device: 1@.cephfs=/ 2023-11-24T07:58:10.767 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T07:58:10.768 DEBUG:teuthology.orchestra.run.smithi076:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 1@.cephfs=/ /home/ubuntu/cephtest/mnt.1 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-11-24T07:58:10.858 INFO:teuthology.orchestra.run.smithi076.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-11-24T07:58:10.858 INFO:teuthology.orchestra.run.smithi076.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-11-24T07:58:10.883 INFO:teuthology.orchestra.run.smithi076.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-11-24T07:58:10.883 INFO:teuthology.orchestra.run.smithi076.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-11-24T07:58:10.883 INFO:teuthology.orchestra.run.smithi076.stdout:mount.ceph: resolved to: "172.21.15.59:3300,172.21.15.76:3300,172.21.15.76:3301" 2023-11-24T07:58:10.883 INFO:teuthology.orchestra.run.smithi076.stdout:mount.ceph: trying mount with new device syntax: 1@9f0e8741-b8f9-48d5-a2c3-09f45903303d.cephfs=/ 2023-11-24T07:58:10.883 INFO:teuthology.orchestra.run.smithi076.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,ms_mode=prefer-crc,key=1,mon_addr=172.21.15.59:3300/172.21.15.76:3300/172.21.15.76:3301" will pass to kernel 2023-11-24T07:58:10.883 INFO:teuthology.orchestra.run.smithi076.stdout:mount.ceph: v2 mount support check returned 2 2023-11-24T07:58:10.884 INFO:teuthology.orchestra.run.smithi076.stdout:mount.ceph: kernel does not support v2 syntax 2023-11-24T07:58:10.884 INFO:teuthology.orchestra.run.smithi076.stdout:mount.ceph: switching to using v1 address with old syntax 2023-11-24T07:58:10.958 INFO:teuthology.orchestra.run.smithi076.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-11-24T07:58:10.959 INFO:teuthology.orchestra.run.smithi076.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-11-24T07:58:10.959 INFO:teuthology.orchestra.run.smithi076.stdout:mount.ceph: resolved to: "172.21.15.59:3300,172.21.15.76:3300,172.21.15.76:3301" 2023-11-24T07:58:10.959 INFO:teuthology.orchestra.run.smithi076.stdout:mount.ceph: trying mount with new device syntax: 1@9f0e8741-b8f9-48d5-a2c3-09f45903303d.cephfs=/ 2023-11-24T07:58:10.959 INFO:teuthology.orchestra.run.smithi076.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,ms_mode=prefer-crc,key=1,mon_addr=172.21.15.59:3300/172.21.15.76:3300/172.21.15.76:3301" will pass to kernel 2023-11-24T07:58:10.959 INFO:teuthology.orchestra.run.smithi076.stdout:mount.ceph: v2 mount support check returned 2 2023-11-24T07:58:10.959 INFO:teuthology.orchestra.run.smithi076.stdout:mount.ceph: kernel does not support v2 syntax 2023-11-24T07:58:10.959 INFO:teuthology.orchestra.run.smithi076.stdout:mount.ceph: switching to using v1 address with old syntax 2023-11-24T07:58:10.959 INFO:teuthology.orchestra.run.smithi076.stdout:mount.ceph: resolved to: "172.21.15.59:6789,172.21.15.76:6789,172.21.15.76:6790" 2023-11-24T07:58:10.959 INFO:teuthology.orchestra.run.smithi076.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.59:6789,172.21.15.76:6789,172.21.15.76:6790:/ 2023-11-24T07:58:10.959 INFO:teuthology.orchestra.run.smithi076.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,key=1,mds_namespace=cephfs,fsid=9f0e8741-b8f9-48d5-a2c3-09f45903303d" will pass to kernel 2023-11-24T07:58:10.961 INFO:tasks.cephfs.kernel_mount:mount command passed 2023-11-24T07:58:10.961 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T07:58:10.961 DEBUG:teuthology.orchestra.run.smithi076:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2023-11-24T07:58:10.987 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T07:58:10.987 DEBUG:teuthology.orchestra.run.smithi076:> (cd /home/ubuntu/cephtest/mnt.1 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-11-24T07:58:11.060 INFO:teuthology.orchestra.run.smithi076.stdout:client5422 2023-11-24T07:58:11.060 DEBUG:teuthology.orchestra.run.smithi076:> set -ex 2023-11-24T07:58:11.060 DEBUG:teuthology.orchestra.run.smithi076:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-24T07:58:11.117 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs flag set enable_multiple true --yes-i-really-mean-it 2023-11-24T07:58:11.260 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:11.258+0000 7f1af23f9700 1 Processor -- start 2023-11-24T07:58:11.260 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:11.258+0000 7f1af23f9700 1 -- start start 2023-11-24T07:58:11.265 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:11.259+0000 7f1af23f9700 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f1aec166e20 0x7f1aec169210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:11.265 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:11.259+0000 7f1af23f9700 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1aec15e5c0 con 0x7f1aec0add30 2023-11-24T07:58:11.265 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:11.259+0000 7f1af23f9700 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1aec15e8a0 con 0x7f1aec06afd0 2023-11-24T07:58:11.265 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:11.259+0000 7f1af23f9700 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1aec072c20 con 0x7f1aec166e20 2023-11-24T07:58:11.265 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:11.259+0000 7f1aeb7fe700 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7f1aec0add30 0x7f1aec0ae1b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:56410/0 (socket says 172.21.15.59:56410) 2023-11-24T07:58:11.265 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:11.259+0000 7f1aeb7fe700 1 -- 172.21.15.59:0/2329902016 learned_addr learned my addr 172.21.15.59:0/2329902016 (peer_addr_for_me v1:172.21.15.59:0/0) 2023-11-24T07:58:11.265 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:11.260+0000 7f1af0996700 1 --2- 172.21.15.59:0/2329902016 >> v2:172.21.15.76:3300/0 conn(0x7f1aec166e20 0x7f1aec169210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:11.266 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:11.260+0000 7f1aeaffd700 1 -- 172.21.15.59:0/2329902016 <== mon.0 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3197009501 0 0) 0x7f1aec15e5c0 con 0x7f1aec0add30 2023-11-24T07:58:11.266 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:11.260+0000 7f1aeaffd700 1 -- 172.21.15.59:0/2329902016 --> v1:172.21.15.59:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1ad80035b0 con 0x7f1aec0add30 2023-11-24T07:58:11.266 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:11.260+0000 7f1aeaffd700 1 -- 172.21.15.59:0/2329902016 <== mon.0 v1:172.21.15.59:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2517374564 0 0) 0x7f1ad80035b0 con 0x7f1aec0add30 2023-11-24T07:58:11.266 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:11.260+0000 7f1aeaffd700 1 -- 172.21.15.59:0/2329902016 >> v2:172.21.15.76:3300/0 conn(0x7f1aec166e20 msgr2=0x7f1aec169210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T07:58:11.266 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:11.260+0000 7f1aeaffd700 1 --2- 172.21.15.59:0/2329902016 >> v2:172.21.15.76:3300/0 conn(0x7f1aec166e20 0x7f1aec169210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:11.266 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:11.260+0000 7f1aeaffd700 1 -- 172.21.15.59:0/2329902016 >> v1:172.21.15.76:6789/0 conn(0x7f1aec06afd0 legacy=0x7f1aec06b3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:11.266 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:11.260+0000 7f1aeaffd700 1 -- 172.21.15.59:0/2329902016 --> v1:172.21.15.59:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1aec15eb80 con 0x7f1aec0add30 2023-11-24T07:58:11.266 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:11.260+0000 7f1aeaffd700 1 -- 172.21.15.59:0/2329902016 <== mon.0 v1:172.21.15.59:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1312333566 0 0) 0x7f1adc001670 con 0x7f1aec0add30 2023-11-24T07:58:11.266 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:11.260+0000 7f1aeaffd700 1 -- 172.21.15.59:0/2329902016 >> v1:172.21.15.59:6789/0 conn(0x7f1aec0add30 legacy=0x7f1aec0ae1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:11.266 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:11.260+0000 7f1aeaffd700 1 --2- 172.21.15.59:0/2329902016 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f1ad8003a00 0x7f1ad8003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:11.267 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:11.260+0000 7f1aeaffd700 1 --2- 172.21.15.59:0/2329902016 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1ad8004850 0x7f1ad8004cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:11.267 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:11.260+0000 7f1aeaffd700 1 --2- 172.21.15.59:0/2329902016 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1ad8005210 0x7f1ad800ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:11.267 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:11.260+0000 7f1aeaffd700 1 -- 172.21.15.59:0/2329902016 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1aec072c20 con 0x7f1ad8004850 2023-11-24T07:58:11.267 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:11.260+0000 7f1aeaffd700 1 -- 172.21.15.59:0/2329902016 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ad800c1b0 con 0x7f1ad8005210 2023-11-24T07:58:11.267 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:11.260+0000 7f1aeaffd700 1 -- 172.21.15.59:0/2329902016 --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1ad800c370 con 0x7f1ad8003a00 2023-11-24T07:58:11.267 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:11.260+0000 7f1aeb7fe700 1 --2- 172.21.15.59:0/2329902016 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1ad8004850 0x7f1ad8004cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:11.267 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:11.260+0000 7f1aebfff700 1 --2- 172.21.15.59:0/2329902016 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f1ad8003a00 0x7f1ad8003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:11.267 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:11.260+0000 7f1af0996700 1 --2- 172.21.15.59:0/2329902016 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1ad8005210 0x7f1ad800ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:11.267 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:11.260+0000 7f1aeb7fe700 1 -- 172.21.15.59:0/2329902016 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f1ad8003a00 msgr2=0x7f1ad8003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T07:58:11.267 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:11.260+0000 7f1aeb7fe700 1 --2- 172.21.15.59:0/2329902016 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f1ad8003a00 0x7f1ad8003e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:11.268 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:11.261+0000 7f1aeb7fe700 1 -- 172.21.15.59:0/2329902016 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1ad8005210 msgr2=0x7f1ad800ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T07:58:11.268 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:11.261+0000 7f1aeb7fe700 1 --2- 172.21.15.59:0/2329902016 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1ad8005210 0x7f1ad800ba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:11.268 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:11.261+0000 7f1af0996700 1 -- 172.21.15.59:0/2329902016 reap_dead start 2023-11-24T07:58:11.268 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:11.261+0000 7f1aeb7fe700 1 -- 172.21.15.59:0/2329902016 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1ad80035b0 con 0x7f1ad8004850 2023-11-24T07:58:11.268 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:11.261+0000 7f1aeb7fe700 1 --2- 172.21.15.59:0/2329902016 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1ad8004850 0x7f1ad8004cd0 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7f1adc00c240 tx=0x7f1ad8005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b1c31545b6497a1e server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:58:11.268 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:11.261+0000 7f1aeaffd700 1 -- 172.21.15.59:0/2329902016 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1adc007c10 con 0x7f1ad8004850 2023-11-24T07:58:11.268 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:11.261+0000 7f1aeaffd700 1 -- 172.21.15.59:0/2329902016 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1adc00cd80 con 0x7f1ad8004850 2023-11-24T07:58:11.268 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:11.261+0000 7f1aeaffd700 1 -- 172.21.15.59:0/2329902016 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1adc005ba0 con 0x7f1ad8004850 2023-11-24T07:58:11.269 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:11.261+0000 7f1af23f9700 1 -- 172.21.15.59:0/2329902016 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1ad8004850 msgr2=0x7f1ad8004cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:11.269 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:11.261+0000 7f1af23f9700 1 --2- 172.21.15.59:0/2329902016 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1ad8004850 0x7f1ad8004cd0 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7f1adc00c240 tx=0x7f1ad8005880 comp rx=0 tx=0).stop 2023-11-24T07:58:11.269 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:11.261+0000 7f1af23f9700 1 -- 172.21.15.59:0/2329902016 shutdown_connections 2023-11-24T07:58:11.269 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:11.261+0000 7f1af23f9700 1 --2- 172.21.15.59:0/2329902016 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1ad8004850 0x7f1ad8004cd0 unknown :-1 s=CLOSED pgs=569 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:11.269 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:11.261+0000 7f1af23f9700 1 -- 172.21.15.59:0/2329902016 >> 172.21.15.59:0/2329902016 conn(0x7f1aec0b2870 msgr2=0x7f1aec0b2c80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:58:11.269 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:11.261+0000 7f1af23f9700 1 -- 172.21.15.59:0/2329902016 shutdown_connections 2023-11-24T07:58:11.269 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:11.262+0000 7f1af23f9700 1 -- 172.21.15.59:0/2329902016 wait complete. 2023-11-24T07:58:11.270 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:11.263+0000 7f1af23f9700 1 Processor -- start 2023-11-24T07:58:11.270 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:11.263+0000 7f1af23f9700 1 -- start start 2023-11-24T07:58:11.273 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:11.264+0000 7f1af23f9700 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f1ad8004850 0x7f1aec150e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:11.273 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:11.264+0000 7f1af23f9700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1aec152af0 0x7f1aec0b43b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:11.273 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:11.264+0000 7f1af23f9700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1aec13f410 0x7f1aec13f8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:11.273 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:11.264+0000 7f1af23f9700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1aec13fde0 con 0x7f1aec13f410 2023-11-24T07:58:11.273 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:11.264+0000 7f1af23f9700 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1aec155340 con 0x7f1aec152af0 2023-11-24T07:58:11.273 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:11.264+0000 7f1af23f9700 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1aec154770 con 0x7f1ad8004850 2023-11-24T07:58:11.273 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:11.264+0000 7f1af0996700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1aec13f410 0x7f1aec13f8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:11.273 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:11.264+0000 7f1af0996700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1aec13f410 0x7f1aec13f8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:52952/0 (socket says 172.21.15.59:52952) 2023-11-24T07:58:11.273 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:11.264+0000 7f1af0996700 1 -- 172.21.15.59:0/2388338632 learned_addr learned my addr 172.21.15.59:0/2388338632 (peer_addr_for_me v2:172.21.15.59:0/0) 2023-11-24T07:58:11.273 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:11.264+0000 7f1aeb7fe700 1 --2- 172.21.15.59:0/2388338632 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1aec152af0 0x7f1aec0b43b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:11.274 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:11.264+0000 7f1aebfff700 1 --2- 172.21.15.59:0/2388338632 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f1ad8004850 0x7f1aec150e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:11.274 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:11.264+0000 7f1af0996700 1 -- 172.21.15.59:0/2388338632 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f1ad8004850 msgr2=0x7f1aec150e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:11.274 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:11.264+0000 7f1af0996700 1 --2- 172.21.15.59:0/2388338632 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f1ad8004850 0x7f1aec150e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:11.274 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:11.264+0000 7f1af0996700 1 -- 172.21.15.59:0/2388338632 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1aec152af0 msgr2=0x7f1aec0b43b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:11.274 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:11.264+0000 7f1af0996700 1 --2- 172.21.15.59:0/2388338632 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1aec152af0 0x7f1aec0b43b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:11.274 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:11.264+0000 7f1af0996700 1 -- 172.21.15.59:0/2388338632 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1aec154a10 con 0x7f1aec13f410 2023-11-24T07:58:11.274 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:11.264+0000 7f1aebfff700 1 --2- 172.21.15.59:0/2388338632 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f1ad8004850 0x7f1aec150e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T07:58:11.274 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:11.264+0000 7f1af0996700 1 --2- 172.21.15.59:0/2388338632 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1aec13f410 0x7f1aec13f8a0 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7f1aec069590 tx=0x7f1ae4011f80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:58:11.274 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:11.264+0000 7f1ae8ff9700 1 -- 172.21.15.59:0/2388338632 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ae400e040 con 0x7f1aec13f410 2023-11-24T07:58:11.274 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:11.264+0000 7f1ae8ff9700 1 -- 172.21.15.59:0/2388338632 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1ae400f040 con 0x7f1aec13f410 2023-11-24T07:58:11.274 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:11.264+0000 7f1af23f9700 1 -- 172.21.15.59:0/2388338632 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1aec154cf0 con 0x7f1aec13f410 2023-11-24T07:58:11.274 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:11.264+0000 7f1ae8ff9700 1 -- 172.21.15.59:0/2388338632 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ae400f940 con 0x7f1aec13f410 2023-11-24T07:58:11.275 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:11.265+0000 7f1af23f9700 1 -- 172.21.15.59:0/2388338632 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1aec152f70 con 0x7f1aec13f410 2023-11-24T07:58:11.275 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:11.266+0000 7f1af23f9700 1 -- 172.21.15.59:0/2388338632 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1aec068670 con 0x7f1aec13f410 2023-11-24T07:58:11.275 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:11.266+0000 7f1ae8ff9700 1 -- 172.21.15.59:0/2388338632 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 6) v1 ==== 89282+0+0 (secure 0 0 0) 0x7f1ae400faa0 con 0x7f1aec13f410 2023-11-24T07:58:11.275 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:11.266+0000 7f1ae8ff9700 1 --2- 172.21.15.59:0/2388338632 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f1acc06af60 0x7f1acc06d420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:11.275 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:11.266+0000 7f1ae8ff9700 1 -- 172.21.15.59:0/2388338632 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f1ae4089f60 con 0x7f1aec13f410 2023-11-24T07:58:11.275 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:11.266+0000 7f1aebfff700 1 --2- 172.21.15.59:0/2388338632 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f1acc06af60 0x7f1acc06d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:11.275 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:11.267+0000 7f1aebfff700 1 --2- 172.21.15.59:0/2388338632 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f1acc06af60 0x7f1acc06d420 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f1ad4000c00 tx=0x7f1ad4009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:58:11.275 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:11.271+0000 7f1ae8ff9700 1 -- 172.21.15.59:0/2388338632 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f1ae4054fa0 con 0x7f1aec13f410 2023-11-24T07:58:11.522 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:11.520+0000 7f1af23f9700 1 -- 172.21.15.59:0/2388338632 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "fs flag set", "flag_name": "enable_multiple", "val": "true", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f1aec155480 con 0x7f1aec13f410 2023-11-24T07:58:12.345 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:12.343+0000 7f1ae8ff9700 1 -- 172.21.15.59:0/2388338632 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "fs flag set", "flag_name": "enable_multiple", "val": "true", "yes_i_really_mean_it": true}]=0 v26) v1 ==== 136+0+0 (secure 0 0 0) 0x7f1aec155480 con 0x7f1aec13f410 2023-11-24T07:58:12.348 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:12.346+0000 7f1af23f9700 1 -- 172.21.15.59:0/2388338632 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f1acc06af60 msgr2=0x7f1acc06d420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:12.348 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:12.346+0000 7f1af23f9700 1 --2- 172.21.15.59:0/2388338632 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f1acc06af60 0x7f1acc06d420 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f1ad4000c00 tx=0x7f1ad4009040 comp rx=0 tx=0).stop 2023-11-24T07:58:12.348 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:12.346+0000 7f1af23f9700 1 -- 172.21.15.59:0/2388338632 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1aec13f410 msgr2=0x7f1aec13f8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:12.348 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:12.346+0000 7f1af23f9700 1 --2- 172.21.15.59:0/2388338632 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1aec13f410 0x7f1aec13f8a0 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7f1aec069590 tx=0x7f1ae4011f80 comp rx=0 tx=0).stop 2023-11-24T07:58:12.348 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:12.346+0000 7f1af23f9700 1 -- 172.21.15.59:0/2388338632 shutdown_connections 2023-11-24T07:58:12.348 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:12.346+0000 7f1af23f9700 1 --2- 172.21.15.59:0/2388338632 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f1acc06af60 0x7f1acc06d420 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:12.348 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:12.346+0000 7f1af23f9700 1 --2- 172.21.15.59:0/2388338632 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1aec13f410 0x7f1aec13f8a0 unknown :-1 s=CLOSED pgs=570 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:12.349 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:12.346+0000 7f1af23f9700 1 --2- 172.21.15.59:0/2388338632 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f1ad8004850 0x7f1aec150e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:12.349 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:12.346+0000 7f1af23f9700 1 --2- 172.21.15.59:0/2388338632 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1aec152af0 0x7f1aec0b43b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:12.349 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:12.346+0000 7f1af23f9700 1 -- 172.21.15.59:0/2388338632 >> 172.21.15.59:0/2388338632 conn(0x7f1aec0b2870 msgr2=0x7f1aec0d0b10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:58:12.349 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:12.346+0000 7f1af23f9700 1 -- 172.21.15.59:0/2388338632 shutdown_connections 2023-11-24T07:58:12.349 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:12.346+0000 7f1af23f9700 1 -- 172.21.15.59:0/2388338632 wait complete. 2023-11-24T07:58:12.363 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume ls 2023-11-24T07:58:12.495 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:12.493+0000 7fe37b8df700 1 Processor -- start 2023-11-24T07:58:12.495 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:12.493+0000 7fe37b8df700 1 -- start start 2023-11-24T07:58:12.499 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:12.494+0000 7fe37b8df700 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fe37415afc0 0x7fe3740730b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:12.499 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:12.494+0000 7fe37b8df700 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe37415f6b0 con 0x7fe37406c5d0 2023-11-24T07:58:12.499 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:12.494+0000 7fe37b8df700 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe37415eb40 con 0x7fe3740b0ac0 2023-11-24T07:58:12.499 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:12.494+0000 7fe37b8df700 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe3740735f0 con 0x7fe37415afc0 2023-11-24T07:58:12.499 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:12.494+0000 7fe378e7a700 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7fe37406c5d0 0x7fe3740b03b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:56432/0 (socket says 172.21.15.59:56432) 2023-11-24T07:58:12.499 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:12.494+0000 7fe378e7a700 1 -- 172.21.15.59:0/2710959864 learned_addr learned my addr 172.21.15.59:0/2710959864 (peer_addr_for_me v1:172.21.15.59:0/0) 2023-11-24T07:58:12.499 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:12.495+0000 7fe37967b700 1 --2- 172.21.15.59:0/2710959864 >> v2:172.21.15.76:3300/0 conn(0x7fe37415afc0 0x7fe3740730b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:12.499 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:12.495+0000 7fe36bfff700 1 -- 172.21.15.59:0/2710959864 <== mon.0 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 440632693 0 0) 0x7fe37415f6b0 con 0x7fe37406c5d0 2023-11-24T07:58:12.499 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:12.495+0000 7fe36bfff700 1 -- 172.21.15.59:0/2710959864 --> v1:172.21.15.59:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe3600035b0 con 0x7fe37406c5d0 2023-11-24T07:58:12.499 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:12.495+0000 7fe36bfff700 1 -- 172.21.15.59:0/2710959864 <== mon.1 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 548379931 0 0) 0x7fe37415eb40 con 0x7fe3740b0ac0 2023-11-24T07:58:12.500 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:12.495+0000 7fe36bfff700 1 -- 172.21.15.59:0/2710959864 --> v1:172.21.15.76:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe37415f6b0 con 0x7fe3740b0ac0 2023-11-24T07:58:12.500 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:12.495+0000 7fe36bfff700 1 -- 172.21.15.59:0/2710959864 <== mon.0 v1:172.21.15.59:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3794116854 0 0) 0x7fe3600035b0 con 0x7fe37406c5d0 2023-11-24T07:58:12.500 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:12.495+0000 7fe36bfff700 1 -- 172.21.15.59:0/2710959864 >> v2:172.21.15.76:3300/0 conn(0x7fe37415afc0 msgr2=0x7fe3740730b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T07:58:12.500 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:12.495+0000 7fe36bfff700 1 --2- 172.21.15.59:0/2710959864 >> v2:172.21.15.76:3300/0 conn(0x7fe37415afc0 0x7fe3740730b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:12.500 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:12.495+0000 7fe36bfff700 1 -- 172.21.15.59:0/2710959864 >> v1:172.21.15.76:6789/0 conn(0x7fe3740b0ac0 legacy=0x7fe37415b3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:12.500 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:12.495+0000 7fe36bfff700 1 -- 172.21.15.59:0/2710959864 --> v1:172.21.15.59:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe37415eda0 con 0x7fe37406c5d0 2023-11-24T07:58:12.500 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:12.495+0000 7fe36bfff700 1 -- 172.21.15.59:0/2710959864 <== mon.0 v1:172.21.15.59:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1312333566 0 0) 0x7fe370001670 con 0x7fe37406c5d0 2023-11-24T07:58:12.500 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:12.495+0000 7fe36bfff700 1 -- 172.21.15.59:0/2710959864 >> v1:172.21.15.59:6789/0 conn(0x7fe37406c5d0 legacy=0x7fe3740b03b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:12.500 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:12.495+0000 7fe36bfff700 1 --2- 172.21.15.59:0/2710959864 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fe360003b90 0x7fe360003fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:12.500 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:12.495+0000 7fe36bfff700 1 --2- 172.21.15.59:0/2710959864 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe360004850 0x7fe360004cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:12.500 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:12.495+0000 7fe36bfff700 1 --2- 172.21.15.59:0/2710959864 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe360005210 0x7fe36000ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:12.500 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:12.495+0000 7fe36bfff700 1 -- 172.21.15.59:0/2710959864 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3740735f0 con 0x7fe360004850 2023-11-24T07:58:12.500 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:12.495+0000 7fe36bfff700 1 -- 172.21.15.59:0/2710959864 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe36000c0b0 con 0x7fe360005210 2023-11-24T07:58:12.501 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:12.495+0000 7fe36bfff700 1 -- 172.21.15.59:0/2710959864 --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe36000c270 con 0x7fe360003b90 2023-11-24T07:58:12.501 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:12.495+0000 7fe378e7a700 1 --2- 172.21.15.59:0/2710959864 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe360004850 0x7fe360004cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:12.501 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:12.495+0000 7fe379e7c700 1 --2- 172.21.15.59:0/2710959864 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe360005210 0x7fe36000ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:12.501 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:12.495+0000 7fe37967b700 1 --2- 172.21.15.59:0/2710959864 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fe360003b90 0x7fe360003fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:12.501 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:12.495+0000 7fe378e7a700 1 -- 172.21.15.59:0/2710959864 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fe360003b90 msgr2=0x7fe360003fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T07:58:12.501 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:12.495+0000 7fe378e7a700 1 --2- 172.21.15.59:0/2710959864 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fe360003b90 0x7fe360003fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:12.501 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:12.495+0000 7fe378e7a700 1 -- 172.21.15.59:0/2710959864 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe360005210 msgr2=0x7fe36000ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T07:58:12.501 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:12.495+0000 7fe378e7a700 1 --2- 172.21.15.59:0/2710959864 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe360005210 0x7fe36000ba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:12.501 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:12.495+0000 7fe379e7c700 1 -- 172.21.15.59:0/2710959864 reap_dead start 2023-11-24T07:58:12.501 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:12.495+0000 7fe378e7a700 1 -- 172.21.15.59:0/2710959864 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe3600035b0 con 0x7fe360004850 2023-11-24T07:58:12.501 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:12.496+0000 7fe378e7a700 1 --2- 172.21.15.59:0/2710959864 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe360004850 0x7fe360004cd0 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7fe37000c210 tx=0x7fe360005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8bd4ec656086a0d1 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:58:12.501 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:12.496+0000 7fe36bfff700 1 -- 172.21.15.59:0/2710959864 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe37000cd50 con 0x7fe360004850 2023-11-24T07:58:12.501 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:12.496+0000 7fe36bfff700 1 -- 172.21.15.59:0/2710959864 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe370005960 con 0x7fe360004850 2023-11-24T07:58:12.502 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:12.496+0000 7fe36bfff700 1 -- 172.21.15.59:0/2710959864 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe370005d10 con 0x7fe360004850 2023-11-24T07:58:12.502 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:12.496+0000 7fe37b8df700 1 -- 172.21.15.59:0/2710959864 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe360004850 msgr2=0x7fe360004cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:12.502 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:12.496+0000 7fe37b8df700 1 --2- 172.21.15.59:0/2710959864 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe360004850 0x7fe360004cd0 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7fe37000c210 tx=0x7fe360005880 comp rx=0 tx=0).stop 2023-11-24T07:58:12.502 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:12.496+0000 7fe37b8df700 1 -- 172.21.15.59:0/2710959864 shutdown_connections 2023-11-24T07:58:12.502 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:12.496+0000 7fe37b8df700 1 --2- 172.21.15.59:0/2710959864 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe360004850 0x7fe360004cd0 unknown :-1 s=CLOSED pgs=572 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:12.502 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:12.496+0000 7fe37b8df700 1 -- 172.21.15.59:0/2710959864 >> 172.21.15.59:0/2710959864 conn(0x7fe3740717f0 msgr2=0x7fe3740b5070 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:58:12.502 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:12.496+0000 7fe37b8df700 1 -- 172.21.15.59:0/2710959864 shutdown_connections 2023-11-24T07:58:12.502 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:12.496+0000 7fe37b8df700 1 -- 172.21.15.59:0/2710959864 wait complete. 2023-11-24T07:58:12.505 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:12.498+0000 7fe37b8df700 1 Processor -- start 2023-11-24T07:58:12.505 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:12.498+0000 7fe37b8df700 1 -- start start 2023-11-24T07:58:12.505 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:12.498+0000 7fe37b8df700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe360004850 0x7fe37409d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:12.505 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:12.498+0000 7fe37b8df700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe37409ad80 0x7fe37414e740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:12.505 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:12.498+0000 7fe37b8df700 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fe37406e700 0x7fe37406eb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:12.505 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:12.498+0000 7fe37b8df700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe37406f0d0 con 0x7fe37409ad80 2023-11-24T07:58:12.505 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:12.498+0000 7fe37b8df700 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe37409b340 con 0x7fe360004850 2023-11-24T07:58:12.505 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:12.498+0000 7fe37b8df700 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe37409b620 con 0x7fe37406e700 2023-11-24T07:58:12.505 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:12.498+0000 7fe378e7a700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe37409ad80 0x7fe37414e740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:12.505 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:12.498+0000 7fe378e7a700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe37409ad80 0x7fe37414e740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:52970/0 (socket says 172.21.15.59:52970) 2023-11-24T07:58:12.505 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:12.498+0000 7fe378e7a700 1 -- 172.21.15.59:0/1969347635 learned_addr learned my addr 172.21.15.59:0/1969347635 (peer_addr_for_me v2:172.21.15.59:0/0) 2023-11-24T07:58:12.506 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:12.499+0000 7fe37967b700 1 --2- 172.21.15.59:0/1969347635 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe360004850 0x7fe37409d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:12.506 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:12.499+0000 7fe379e7c700 1 --2- 172.21.15.59:0/1969347635 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fe37406e700 0x7fe37406eb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:12.506 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:12.499+0000 7fe378e7a700 1 -- 172.21.15.59:0/1969347635 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fe37406e700 msgr2=0x7fe37406eb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:12.506 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:12.499+0000 7fe378e7a700 1 --2- 172.21.15.59:0/1969347635 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fe37406e700 0x7fe37406eb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:12.506 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:12.499+0000 7fe378e7a700 1 -- 172.21.15.59:0/1969347635 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe360004850 msgr2=0x7fe37409d0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:12.506 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:12.499+0000 7fe378e7a700 1 --2- 172.21.15.59:0/1969347635 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe360004850 0x7fe37409d0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:12.506 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:12.499+0000 7fe378e7a700 1 -- 172.21.15.59:0/1969347635 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe37409b900 con 0x7fe37409ad80 2023-11-24T07:58:12.506 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:12.499+0000 7fe378e7a700 1 --2- 172.21.15.59:0/1969347635 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe37409ad80 0x7fe37414e740 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7fe36000c080 tx=0x7fe370001980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:58:12.506 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:12.499+0000 7fe369ffb700 1 -- 172.21.15.59:0/1969347635 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe370004c80 con 0x7fe37409ad80 2023-11-24T07:58:12.506 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:12.499+0000 7fe369ffb700 1 -- 172.21.15.59:0/1969347635 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe370004e00 con 0x7fe37409ad80 2023-11-24T07:58:12.506 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:12.499+0000 7fe37b8df700 1 -- 172.21.15.59:0/1969347635 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe37409bbe0 con 0x7fe37409ad80 2023-11-24T07:58:12.506 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:12.499+0000 7fe369ffb700 1 -- 172.21.15.59:0/1969347635 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe370007310 con 0x7fe37409ad80 2023-11-24T07:58:12.507 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:12.499+0000 7fe37b8df700 1 -- 172.21.15.59:0/1969347635 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe37409c3e0 con 0x7fe37409ad80 2023-11-24T07:58:12.507 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:12.500+0000 7fe37b8df700 1 -- 172.21.15.59:0/1969347635 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe374068670 con 0x7fe37409ad80 2023-11-24T07:58:12.507 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:12.501+0000 7fe369ffb700 1 -- 172.21.15.59:0/1969347635 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 6) v1 ==== 89282+0+0 (secure 0 0 0) 0x7fe370007470 con 0x7fe37409ad80 2023-11-24T07:58:12.507 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:12.501+0000 7fe369ffb700 1 --2- 172.21.15.59:0/1969347635 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7fe35806af60 0x7fe35806d420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:12.507 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:12.501+0000 7fe369ffb700 1 -- 172.21.15.59:0/1969347635 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fe37008a510 con 0x7fe37409ad80 2023-11-24T07:58:12.507 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:12.501+0000 7fe37967b700 1 --2- 172.21.15.59:0/1969347635 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7fe35806af60 0x7fe35806d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:12.507 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:12.502+0000 7fe37967b700 1 --2- 172.21.15.59:0/1969347635 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7fe35806af60 0x7fe35806d420 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7fe374068610 tx=0x7fe364008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:58:12.507 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:12.506+0000 7fe369ffb700 1 -- 172.21.15.59:0/1969347635 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fe370055660 con 0x7fe37409ad80 2023-11-24T07:58:12.739 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:12.737+0000 7fe37b8df700 1 -- 172.21.15.59:0/1969347635 --> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] -- mgr_command(tid 0: {"prefix": "fs volume ls", "target": ["mon-mgr", ""]}) v1 -- 0x7fe37415f340 con 0x7fe35806af60 2023-11-24T07:58:12.741 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:12.740+0000 7fe369ffb700 1 -- 172.21.15.59:0/1969347635 <== mgr.4100 v2:172.21.15.76:6832/4264046536 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+41 (secure 0 0 0) 0x7fe364009790 con 0x7fe35806af60 2023-11-24T07:58:12.741 INFO:teuthology.orchestra.run.smithi059.stdout:[ 2023-11-24T07:58:12.741 INFO:teuthology.orchestra.run.smithi059.stdout: { 2023-11-24T07:58:12.741 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "cephfs" 2023-11-24T07:58:12.741 INFO:teuthology.orchestra.run.smithi059.stdout: } 2023-11-24T07:58:12.741 INFO:teuthology.orchestra.run.smithi059.stdout:] 2023-11-24T07:58:12.743 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:12.741+0000 7fe37b8df700 1 -- 172.21.15.59:0/1969347635 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7fe35806af60 msgr2=0x7fe35806d420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:12.743 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:12.741+0000 7fe37b8df700 1 --2- 172.21.15.59:0/1969347635 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7fe35806af60 0x7fe35806d420 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7fe374068610 tx=0x7fe364008040 comp rx=0 tx=0).stop 2023-11-24T07:58:12.743 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:12.741+0000 7fe37b8df700 1 -- 172.21.15.59:0/1969347635 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe37409ad80 msgr2=0x7fe37414e740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:12.743 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:12.741+0000 7fe37b8df700 1 --2- 172.21.15.59:0/1969347635 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe37409ad80 0x7fe37414e740 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7fe36000c080 tx=0x7fe370001980 comp rx=0 tx=0).stop 2023-11-24T07:58:12.743 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:12.741+0000 7fe37b8df700 1 -- 172.21.15.59:0/1969347635 shutdown_connections 2023-11-24T07:58:12.744 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:12.741+0000 7fe37b8df700 1 --2- 172.21.15.59:0/1969347635 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7fe35806af60 0x7fe35806d420 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:12.744 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:12.741+0000 7fe37b8df700 1 --2- 172.21.15.59:0/1969347635 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fe37406e700 0x7fe37406eb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:12.744 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:12.741+0000 7fe37b8df700 1 --2- 172.21.15.59:0/1969347635 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe360004850 0x7fe37409d0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:12.744 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:12.741+0000 7fe37b8df700 1 --2- 172.21.15.59:0/1969347635 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe37409ad80 0x7fe37414e740 unknown :-1 s=CLOSED pgs=573 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:12.744 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:12.741+0000 7fe37b8df700 1 -- 172.21.15.59:0/1969347635 >> 172.21.15.59:0/1969347635 conn(0x7fe3740717f0 msgr2=0x7fe37415bc30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:58:12.744 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:12.741+0000 7fe37b8df700 1 -- 172.21.15.59:0/1969347635 shutdown_connections 2023-11-24T07:58:12.744 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:12.741+0000 7fe37b8df700 1 -- 172.21.15.59:0/1969347635 wait complete. 2023-11-24T07:58:12.755 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2023-11-24T07:58:12.883 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:12.881+0000 7f8edd870700 1 Processor -- start 2023-11-24T07:58:12.883 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:12.881+0000 7f8edd870700 1 -- start start 2023-11-24T07:58:12.885 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:12.882+0000 7f8edd870700 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f8ed815f410 0x7f8ed80b4240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:12.885 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:12.882+0000 7f8edd870700 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f8ed815fc20 0x7f8ed81600a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:12.885 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:12.882+0000 7f8edd870700 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8ed80b14c0 con 0x7f8ed81605e0 2023-11-24T07:58:12.885 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:12.882+0000 7f8edd870700 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8ed80b3070 con 0x7f8ed815fc20 2023-11-24T07:58:12.885 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:12.882+0000 7f8edd870700 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8ed80b3270 con 0x7f8ed815f410 2023-11-24T07:58:12.885 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:12.882+0000 7f8ed67fc700 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f8ed815fc20 0x7f8ed81600a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:12.885 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:12.882+0000 7f8ed6ffd700 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f8ed815f410 0x7f8ed80b4240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:12.885 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:12.883+0000 7f8ed77fe700 1 --1- >> v1:172.21.15.76:6789/0 conn(0x7f8ed81605e0 0x7f8ed80cfb10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.59:43732/0 (socket says 172.21.15.59:43732) 2023-11-24T07:58:12.885 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:12.883+0000 7f8ed77fe700 1 -- 172.21.15.59:0/2957626773 learned_addr learned my addr 172.21.15.59:0/2957626773 (peer_addr_for_me v1:172.21.15.59:0/0) 2023-11-24T07:58:12.885 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:12.883+0000 7f8ed67fc700 1 -- 172.21.15.59:0/2957626773 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f8ed815f410 msgr2=0x7f8ed80b4240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T07:58:12.886 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:12.883+0000 7f8ed67fc700 1 --2- 172.21.15.59:0/2957626773 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f8ed815f410 0x7f8ed80b4240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:12.886 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:12.883+0000 7f8ed67fc700 1 -- 172.21.15.59:0/2957626773 >> v1:172.21.15.76:6789/0 conn(0x7f8ed81605e0 legacy=0x7f8ed80cfb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:12.886 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:12.883+0000 7f8ed67fc700 1 -- 172.21.15.59:0/2957626773 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8ed80b3510 con 0x7f8ed815fc20 2023-11-24T07:58:12.886 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:12.883+0000 7f8ed5ffb700 1 -- 172.21.15.59:0/2957626773 <== mon.1 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3406805686 0 0) 0x7f8ed80b14c0 con 0x7f8ed81605e0 2023-11-24T07:58:12.886 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:12.883+0000 7f8ed6ffd700 1 --2- 172.21.15.59:0/2957626773 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f8ed815f410 0x7f8ed80b4240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-24T07:58:12.886 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:12.883+0000 7f8ed67fc700 1 --2- 172.21.15.59:0/2957626773 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8ed815fc20 0x7f8ed81600a0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f8ec8009840 tx=0x7f8ec8009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7fbd9adaf30fe7e server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:58:12.886 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:12.883+0000 7f8ed5ffb700 1 -- 172.21.15.59:0/2957626773 <== mon.1 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ec8011830 con 0x7f8ed815fc20 2023-11-24T07:58:12.886 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:12.883+0000 7f8ed5ffb700 1 -- 172.21.15.59:0/2957626773 <== mon.1 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8ec8011990 con 0x7f8ed815fc20 2023-11-24T07:58:12.886 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:12.883+0000 7f8ed5ffb700 1 -- 172.21.15.59:0/2957626773 <== mon.1 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ec8011d20 con 0x7f8ed815fc20 2023-11-24T07:58:12.889 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:12.884+0000 7f8edd870700 1 -- 172.21.15.59:0/2957626773 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8ed815fc20 msgr2=0x7f8ed81600a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:12.889 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:12.884+0000 7f8edd870700 1 --2- 172.21.15.59:0/2957626773 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8ed815fc20 0x7f8ed81600a0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f8ec8009840 tx=0x7f8ec8009b50 comp rx=0 tx=0).stop 2023-11-24T07:58:12.889 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:12.884+0000 7f8edd870700 1 -- 172.21.15.59:0/2957626773 shutdown_connections 2023-11-24T07:58:12.889 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:12.884+0000 7f8edd870700 1 --2- 172.21.15.59:0/2957626773 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f8ed815f410 0x7f8ed80b4240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:12.889 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:12.884+0000 7f8edd870700 1 --2- 172.21.15.59:0/2957626773 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8ed815fc20 0x7f8ed81600a0 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:12.889 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:12.884+0000 7f8edd870700 1 -- 172.21.15.59:0/2957626773 >> 172.21.15.59:0/2957626773 conn(0x7f8ed80b3720 msgr2=0x7f8ed80b3b30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:58:12.889 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:12.884+0000 7f8edd870700 1 -- 172.21.15.59:0/2957626773 shutdown_connections 2023-11-24T07:58:12.889 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:12.884+0000 7f8edd870700 1 -- 172.21.15.59:0/2957626773 wait complete. 2023-11-24T07:58:12.890 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:12.885+0000 7f8edd870700 1 Processor -- start 2023-11-24T07:58:12.890 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:12.885+0000 7f8edd870700 1 -- start start 2023-11-24T07:58:12.890 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:12.886+0000 7f8edd870700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8ed815f410 0x7f8ed80c8ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:12.890 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:12.886+0000 7f8edd870700 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f8ed815fc20 0x7f8ed8159290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:12.890 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:12.886+0000 7f8edd870700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8ed81605e0 0x7f8ed80cb180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:12.890 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:12.886+0000 7f8edd870700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ed81597d0 con 0x7f8ed81605e0 2023-11-24T07:58:12.890 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:12.886+0000 7f8edd870700 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ed80b0a40 con 0x7f8ed815f410 2023-11-24T07:58:12.890 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:12.886+0000 7f8edd870700 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8ed80cb6c0 con 0x7f8ed815fc20 2023-11-24T07:58:12.890 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:12.886+0000 7f8ed77fe700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8ed81605e0 0x7f8ed80cb180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:12.890 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:12.886+0000 7f8ed77fe700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8ed81605e0 0x7f8ed80cb180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:52986/0 (socket says 172.21.15.59:52986) 2023-11-24T07:58:12.891 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:12.886+0000 7f8ed77fe700 1 -- 172.21.15.59:0/2763895889 learned_addr learned my addr 172.21.15.59:0/2763895889 (peer_addr_for_me v2:172.21.15.59:0/0) 2023-11-24T07:58:12.891 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:12.886+0000 7f8ed67fc700 1 --2- 172.21.15.59:0/2763895889 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f8ed815fc20 0x7f8ed8159290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:12.891 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:12.886+0000 7f8ed6ffd700 1 --2- 172.21.15.59:0/2763895889 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8ed815f410 0x7f8ed80c8ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:12.891 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:12.886+0000 7f8ed77fe700 1 -- 172.21.15.59:0/2763895889 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f8ed815fc20 msgr2=0x7f8ed8159290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:12.891 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:12.886+0000 7f8ed77fe700 1 --2- 172.21.15.59:0/2763895889 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f8ed815fc20 0x7f8ed8159290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:12.891 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:12.887+0000 7f8ed77fe700 1 -- 172.21.15.59:0/2763895889 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8ed815f410 msgr2=0x7f8ed80c8ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:12.891 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:12.887+0000 7f8ed77fe700 1 --2- 172.21.15.59:0/2763895889 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8ed815f410 0x7f8ed80c8ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:12.891 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:12.887+0000 7f8ed77fe700 1 -- 172.21.15.59:0/2763895889 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8ecc0094a0 con 0x7f8ed81605e0 2023-11-24T07:58:12.891 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:12.887+0000 7f8ed6ffd700 1 --2- 172.21.15.59:0/2763895889 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8ed815f410 0x7f8ed80c8ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-24T07:58:12.891 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:12.887+0000 7f8ed77fe700 1 --2- 172.21.15.59:0/2763895889 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8ed81605e0 0x7f8ed80cb180 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7f8ed000cc70 tx=0x7f8ed000cf80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:58:12.891 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:12.887+0000 7f8ebffff700 1 -- 172.21.15.59:0/2763895889 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ed000ab70 con 0x7f8ed81605e0 2023-11-24T07:58:12.891 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:12.887+0000 7f8ebffff700 1 -- 172.21.15.59:0/2763895889 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8ed000acf0 con 0x7f8ed81605e0 2023-11-24T07:58:12.892 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:12.887+0000 7f8ebffff700 1 -- 172.21.15.59:0/2763895889 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ed0007310 con 0x7f8ed81605e0 2023-11-24T07:58:12.892 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:12.887+0000 7f8edd870700 1 -- 172.21.15.59:0/2763895889 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8ec80094a0 con 0x7f8ed81605e0 2023-11-24T07:58:12.892 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:12.887+0000 7f8edd870700 1 -- 172.21.15.59:0/2763895889 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8ed80c95a0 con 0x7f8ed81605e0 2023-11-24T07:58:12.893 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:12.888+0000 7f8edd870700 1 -- 172.21.15.59:0/2763895889 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8ed8075480 con 0x7f8ed81605e0 2023-11-24T07:58:12.893 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:12.888+0000 7f8ebffff700 1 -- 172.21.15.59:0/2763895889 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 6) v1 ==== 89282+0+0 (secure 0 0 0) 0x7f8ed0007470 con 0x7f8ed81605e0 2023-11-24T07:58:12.893 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:12.889+0000 7f8ebffff700 1 --2- 172.21.15.59:0/2763895889 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f8ec406af60 0x7f8ec406d420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:12.893 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:12.889+0000 7f8ebffff700 1 -- 172.21.15.59:0/2763895889 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f8ed0089070 con 0x7f8ed81605e0 2023-11-24T07:58:12.893 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:12.889+0000 7f8ed6ffd700 1 --2- 172.21.15.59:0/2763895889 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f8ec406af60 0x7f8ec406d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:12.893 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:12.889+0000 7f8ed6ffd700 1 --2- 172.21.15.59:0/2763895889 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f8ec406af60 0x7f8ec406d420 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f8ed80d2760 tx=0x7f8ecc008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:58:12.894 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:12.893+0000 7f8ebffff700 1 -- 172.21.15.59:0/2763895889 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f8ed00541c0 con 0x7f8ed81605e0 2023-11-24T07:58:13.095 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:13.094+0000 7f8edd870700 1 -- 172.21.15.59:0/2763895889 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7f8ed80b7b90 con 0x7f8ed81605e0 2023-11-24T07:58:13.096 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:13.094+0000 7f8ebffff700 1 -- 172.21.15.59:0/2763895889 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v12)=0 v12) v1 ==== 122+0+0 (secure 0 0 0) 0x7f8ed000ae60 con 0x7f8ed81605e0 2023-11-24T07:58:13.098 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:13.096+0000 7f8edd870700 1 -- 172.21.15.59:0/2763895889 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f8ec406af60 msgr2=0x7f8ec406d420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:13.098 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:13.096+0000 7f8edd870700 1 --2- 172.21.15.59:0/2763895889 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f8ec406af60 0x7f8ec406d420 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f8ed80d2760 tx=0x7f8ecc008040 comp rx=0 tx=0).stop 2023-11-24T07:58:13.098 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:13.096+0000 7f8edd870700 1 -- 172.21.15.59:0/2763895889 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8ed81605e0 msgr2=0x7f8ed80cb180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:13.099 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:13.096+0000 7f8edd870700 1 --2- 172.21.15.59:0/2763895889 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8ed81605e0 0x7f8ed80cb180 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7f8ed000cc70 tx=0x7f8ed000cf80 comp rx=0 tx=0).stop 2023-11-24T07:58:13.099 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:13.096+0000 7f8edd870700 1 -- 172.21.15.59:0/2763895889 shutdown_connections 2023-11-24T07:58:13.099 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:13.096+0000 7f8edd870700 1 --2- 172.21.15.59:0/2763895889 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f8ec406af60 0x7f8ec406d420 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:13.099 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:13.096+0000 7f8edd870700 1 --2- 172.21.15.59:0/2763895889 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8ed81605e0 0x7f8ed80cb180 unknown :-1 s=CLOSED pgs=574 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:13.099 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:13.096+0000 7f8edd870700 1 --2- 172.21.15.59:0/2763895889 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8ed815f410 0x7f8ed80c8ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:13.099 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:13.096+0000 7f8edd870700 1 --2- 172.21.15.59:0/2763895889 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f8ed815fc20 0x7f8ed8159290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:13.099 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:13.096+0000 7f8edd870700 1 -- 172.21.15.59:0/2763895889 >> 172.21.15.59:0/2763895889 conn(0x7f8ed80b3720 msgr2=0x7f8ed80b1f50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:58:13.099 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:13.096+0000 7f8edd870700 1 -- 172.21.15.59:0/2763895889 shutdown_connections 2023-11-24T07:58:13.099 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:13.096+0000 7f8edd870700 1 -- 172.21.15.59:0/2763895889 wait complete. 2023-11-24T07:58:13.111 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolumegroup create cephfs subvol_grp1023 2023-11-24T07:58:13.238 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:13.236+0000 7f2b847c1700 1 Processor -- start 2023-11-24T07:58:13.238 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:13.237+0000 7f2b847c1700 1 -- start start 2023-11-24T07:58:13.241 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:13.238+0000 7f2b847c1700 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f2b7c0b5ab0 0x7f2b7c06e560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:13.241 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:13.238+0000 7f2b847c1700 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2b7c071f50 con 0x7f2b7c0b4150 2023-11-24T07:58:13.241 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:13.238+0000 7f2b847c1700 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2b7c072230 con 0x7f2b7c06b620 2023-11-24T07:58:13.241 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:13.238+0000 7f2b847c1700 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2b7c072510 con 0x7f2b7c0b5ab0 2023-11-24T07:58:13.241 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:13.238+0000 7f2b81d5c700 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7f2b7c0b4150 0x7f2b7c06de50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:56464/0 (socket says 172.21.15.59:56464) 2023-11-24T07:58:13.241 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:13.238+0000 7f2b81d5c700 1 -- 172.21.15.59:0/2500696221 learned_addr learned my addr 172.21.15.59:0/2500696221 (peer_addr_for_me v1:172.21.15.59:0/0) 2023-11-24T07:58:13.241 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:13.238+0000 7f2b82d5e700 1 --2- 172.21.15.59:0/2500696221 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f2b7c0b5ab0 0x7f2b7c06e560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:13.241 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:13.238+0000 7f2b8155b700 1 -- 172.21.15.59:0/2500696221 <== mon.0 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1661160030 0 0) 0x7f2b7c071f50 con 0x7f2b7c0b4150 2023-11-24T07:58:13.241 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:13.238+0000 7f2b8155b700 1 -- 172.21.15.59:0/2500696221 --> v1:172.21.15.59:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2b680035b0 con 0x7f2b7c0b4150 2023-11-24T07:58:13.241 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:13.238+0000 7f2b8155b700 1 -- 172.21.15.59:0/2500696221 <== mon.1 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3039836440 0 0) 0x7f2b7c072230 con 0x7f2b7c06b620 2023-11-24T07:58:13.242 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:13.238+0000 7f2b8155b700 1 -- 172.21.15.59:0/2500696221 --> v1:172.21.15.76:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2b7c071f50 con 0x7f2b7c06b620 2023-11-24T07:58:13.242 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:13.238+0000 7f2b8155b700 1 -- 172.21.15.59:0/2500696221 <== mon.0 v1:172.21.15.59:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 528149307 0 0) 0x7f2b680035b0 con 0x7f2b7c0b4150 2023-11-24T07:58:13.242 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:13.238+0000 7f2b82d5e700 1 -- 172.21.15.59:0/2500696221 >> v1:172.21.15.76:6789/0 conn(0x7f2b7c06b620 legacy=0x7f2b7c0b3a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:13.242 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:13.238+0000 7f2b82d5e700 1 -- 172.21.15.59:0/2500696221 >> v1:172.21.15.59:6789/0 conn(0x7f2b7c0b4150 legacy=0x7f2b7c06de50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:13.242 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:13.238+0000 7f2b82d5e700 1 -- 172.21.15.59:0/2500696221 --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2b7c0727f0 con 0x7f2b7c0b5ab0 2023-11-24T07:58:13.242 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:13.238+0000 7f2b82d5e700 1 --2- 172.21.15.59:0/2500696221 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f2b7c0b5ab0 0x7f2b7c06e560 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f2b74009dc0 tx=0x7f2b7400b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ebc1911933378030 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:58:13.242 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:13.239+0000 7f2b8155b700 1 -- 172.21.15.59:0/2500696221 <== mon.2 v2:172.21.15.76:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b7400e070 con 0x7f2b7c0b5ab0 2023-11-24T07:58:13.242 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:13.239+0000 7f2b8155b700 1 -- 172.21.15.59:0/2500696221 <== mon.2 v2:172.21.15.76:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2b74003a20 con 0x7f2b7c0b5ab0 2023-11-24T07:58:13.242 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:13.239+0000 7f2b8155b700 1 -- 172.21.15.59:0/2500696221 <== mon.2 v2:172.21.15.76:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b70001670 con 0x7f2b7c0b5ab0 2023-11-24T07:58:13.243 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:13.239+0000 7f2b847c1700 1 -- 172.21.15.59:0/2500696221 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f2b7c0b5ab0 msgr2=0x7f2b7c06e560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:13.243 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:13.239+0000 7f2b847c1700 1 --2- 172.21.15.59:0/2500696221 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f2b7c0b5ab0 0x7f2b7c06e560 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f2b74009dc0 tx=0x7f2b7400b330 comp rx=0 tx=0).stop 2023-11-24T07:58:13.243 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:13.239+0000 7f2b847c1700 1 -- 172.21.15.59:0/2500696221 shutdown_connections 2023-11-24T07:58:13.243 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:13.239+0000 7f2b847c1700 1 --2- 172.21.15.59:0/2500696221 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f2b7c0b5ab0 0x7f2b7c06e560 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:13.243 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:13.239+0000 7f2b847c1700 1 -- 172.21.15.59:0/2500696221 >> 172.21.15.59:0/2500696221 conn(0x7f2b7c0b2740 msgr2=0x7f2b7c0b2b20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:58:13.243 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:13.239+0000 7f2b847c1700 1 -- 172.21.15.59:0/2500696221 shutdown_connections 2023-11-24T07:58:13.243 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:13.239+0000 7f2b847c1700 1 -- 172.21.15.59:0/2500696221 wait complete. 2023-11-24T07:58:13.246 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:13.240+0000 7f2b847c1700 1 Processor -- start 2023-11-24T07:58:13.246 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:13.241+0000 7f2b847c1700 1 -- start start 2023-11-24T07:58:13.246 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:13.241+0000 7f2b847c1700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2b7c06b620 0x7f2b7c13e170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:13.246 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:13.241+0000 7f2b847c1700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2b7c0b4150 0x7f2b7c13c780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:13.246 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:13.241+0000 7f2b847c1700 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f2b7c0b5ab0 0x7f2b7c1dc650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:13.246 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:13.241+0000 7f2b847c1700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b7c13e6b0 con 0x7f2b7c0b4150 2023-11-24T07:58:13.246 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:13.241+0000 7f2b847c1700 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b7c13ccc0 con 0x7f2b7c06b620 2023-11-24T07:58:13.246 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:13.241+0000 7f2b847c1700 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2b7c13f2f0 con 0x7f2b7c0b5ab0 2023-11-24T07:58:13.246 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:13.241+0000 7f2b81d5c700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2b7c0b4150 0x7f2b7c13c780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:13.246 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:13.241+0000 7f2b81d5c700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2b7c0b4150 0x7f2b7c13c780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:52998/0 (socket says 172.21.15.59:52998) 2023-11-24T07:58:13.247 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:13.241+0000 7f2b81d5c700 1 -- 172.21.15.59:0/2629709251 learned_addr learned my addr 172.21.15.59:0/2629709251 (peer_addr_for_me v2:172.21.15.59:0/0) 2023-11-24T07:58:13.247 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:13.242+0000 7f2b82d5e700 1 --2- 172.21.15.59:0/2629709251 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f2b7c0b5ab0 0x7f2b7c1dc650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:13.247 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:13.242+0000 7f2b8255d700 1 --2- 172.21.15.59:0/2629709251 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2b7c06b620 0x7f2b7c13e170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:13.247 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:13.242+0000 7f2b81d5c700 1 -- 172.21.15.59:0/2629709251 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f2b7c0b5ab0 msgr2=0x7f2b7c1dc650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:13.247 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:13.242+0000 7f2b81d5c700 1 --2- 172.21.15.59:0/2629709251 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f2b7c0b5ab0 0x7f2b7c1dc650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:13.247 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:13.242+0000 7f2b81d5c700 1 -- 172.21.15.59:0/2629709251 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2b7c06b620 msgr2=0x7f2b7c13e170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:13.247 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:13.242+0000 7f2b81d5c700 1 --2- 172.21.15.59:0/2629709251 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2b7c06b620 0x7f2b7c13e170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:13.247 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:13.242+0000 7f2b81d5c700 1 -- 172.21.15.59:0/2629709251 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2b74009a70 con 0x7f2b7c0b4150 2023-11-24T07:58:13.247 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:13.242+0000 7f2b81d5c700 1 --2- 172.21.15.59:0/2629709251 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2b7c0b4150 0x7f2b7c13c780 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7f2b7000c750 tx=0x7f2b7000cb10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:58:13.247 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:13.242+0000 7f2b6effd700 1 -- 172.21.15.59:0/2629709251 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b70004c80 con 0x7f2b7c0b4150 2023-11-24T07:58:13.247 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:13.242+0000 7f2b6effd700 1 -- 172.21.15.59:0/2629709251 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2b70004e00 con 0x7f2b7c0b4150 2023-11-24T07:58:13.247 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:13.242+0000 7f2b847c1700 1 -- 172.21.15.59:0/2629709251 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2b7c13f8b0 con 0x7f2b7c0b4150 2023-11-24T07:58:13.247 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:13.242+0000 7f2b6effd700 1 -- 172.21.15.59:0/2629709251 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b70007310 con 0x7f2b7c0b4150 2023-11-24T07:58:13.248 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:13.242+0000 7f2b847c1700 1 -- 172.21.15.59:0/2629709251 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2b7c1dcb90 con 0x7f2b7c0b4150 2023-11-24T07:58:13.248 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:13.243+0000 7f2b847c1700 1 -- 172.21.15.59:0/2629709251 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2b7c075480 con 0x7f2b7c0b4150 2023-11-24T07:58:13.248 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:13.244+0000 7f2b6effd700 1 -- 172.21.15.59:0/2629709251 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 6) v1 ==== 89282+0+0 (secure 0 0 0) 0x7f2b70007470 con 0x7f2b7c0b4150 2023-11-24T07:58:13.248 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:13.244+0000 7f2b6effd700 1 --2- 172.21.15.59:0/2629709251 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f2b6006af60 0x7f2b6006d420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:13.248 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:13.244+0000 7f2b6effd700 1 -- 172.21.15.59:0/2629709251 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f2b70089980 con 0x7f2b7c0b4150 2023-11-24T07:58:13.248 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:13.244+0000 7f2b8255d700 1 --2- 172.21.15.59:0/2629709251 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f2b6006af60 0x7f2b6006d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:13.248 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:13.245+0000 7f2b8255d700 1 --2- 172.21.15.59:0/2629709251 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f2b6006af60 0x7f2b6006d420 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f2b780011e0 tx=0x7f2b78009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:58:13.249 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:13.248+0000 7f2b6effd700 1 -- 172.21.15.59:0/2629709251 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f2b70054b50 con 0x7f2b7c0b4150 2023-11-24T07:58:13.486 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:13.484+0000 7f2b847c1700 1 -- 172.21.15.59:0/2629709251 --> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] -- mgr_command(tid 0: {"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "subvol_grp1023", "target": ["mon-mgr", ""]}) v1 -- 0x7f2b7c06ba00 con 0x7f2b6006af60 2023-11-24T07:58:13.492 INFO:tasks.ceph.mgr.x.smithi076.stderr:2023-11-24T07:58:13.490+0000 7ff196724700 -1 client.0 error registering admin socket command: (17) File exists 2023-11-24T07:58:13.492 INFO:tasks.ceph.mgr.x.smithi076.stderr:2023-11-24T07:58:13.490+0000 7ff196724700 -1 client.0 error registering admin socket command: (17) File exists 2023-11-24T07:58:13.492 INFO:tasks.ceph.mgr.x.smithi076.stderr:2023-11-24T07:58:13.490+0000 7ff196724700 -1 client.0 error registering admin socket command: (17) File exists 2023-11-24T07:58:13.492 INFO:tasks.ceph.mgr.x.smithi076.stderr:2023-11-24T07:58:13.490+0000 7ff196724700 -1 client.0 error registering admin socket command: (17) File exists 2023-11-24T07:58:13.492 INFO:tasks.ceph.mgr.x.smithi076.stderr:2023-11-24T07:58:13.490+0000 7ff196724700 -1 client.0 error registering admin socket command: (17) File exists 2023-11-24T07:58:13.507 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:13.505+0000 7f2b6effd700 1 -- 172.21.15.59:0/2629709251 <== mgr.4100 v2:172.21.15.76:6832/4264046536 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7f2b780013f0 con 0x7f2b6006af60 2023-11-24T07:58:13.509 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:13.507+0000 7f2b847c1700 1 -- 172.21.15.59:0/2629709251 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f2b6006af60 msgr2=0x7f2b6006d420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:13.509 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:13.507+0000 7f2b847c1700 1 --2- 172.21.15.59:0/2629709251 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f2b6006af60 0x7f2b6006d420 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f2b780011e0 tx=0x7f2b78009040 comp rx=0 tx=0).stop 2023-11-24T07:58:13.510 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:13.507+0000 7f2b847c1700 1 -- 172.21.15.59:0/2629709251 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2b7c0b4150 msgr2=0x7f2b7c13c780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:13.510 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:13.507+0000 7f2b847c1700 1 --2- 172.21.15.59:0/2629709251 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2b7c0b4150 0x7f2b7c13c780 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7f2b7000c750 tx=0x7f2b7000cb10 comp rx=0 tx=0).stop 2023-11-24T07:58:13.510 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:13.507+0000 7f2b847c1700 1 -- 172.21.15.59:0/2629709251 shutdown_connections 2023-11-24T07:58:13.510 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:13.507+0000 7f2b847c1700 1 --2- 172.21.15.59:0/2629709251 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f2b6006af60 0x7f2b6006d420 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:13.510 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:13.507+0000 7f2b847c1700 1 --2- 172.21.15.59:0/2629709251 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f2b7c0b5ab0 0x7f2b7c1dc650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:13.510 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:13.507+0000 7f2b847c1700 1 --2- 172.21.15.59:0/2629709251 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2b7c06b620 0x7f2b7c13e170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:13.510 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:13.507+0000 7f2b847c1700 1 --2- 172.21.15.59:0/2629709251 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2b7c0b4150 0x7f2b7c13c780 unknown :-1 s=CLOSED pgs=576 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:13.510 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:13.507+0000 7f2b847c1700 1 -- 172.21.15.59:0/2629709251 >> 172.21.15.59:0/2629709251 conn(0x7f2b7c0b2740 msgr2=0x7f2b7c06df60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:58:13.510 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:13.508+0000 7f2b847c1700 1 -- 172.21.15.59:0/2629709251 shutdown_connections 2023-11-24T07:58:13.510 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:13.508+0000 7f2b847c1700 1 -- 172.21.15.59:0/2629709251 wait complete. 2023-11-24T07:58:13.521 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume create cephfs subvol4495 subvol_grp1023 2023-11-24T07:58:13.650 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:13.648+0000 7f2367f26700 1 Processor -- start 2023-11-24T07:58:13.650 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:13.648+0000 7f2367f26700 1 -- start start 2023-11-24T07:58:13.653 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:13.649+0000 7f2367f26700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f23601446e0 0x7f23600cdf60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:13.653 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:13.649+0000 7f2367f26700 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f236006f850 0x7f236006e780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:13.653 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:13.649+0000 7f2367f26700 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f23600b44a0 con 0x7f236006e2d0 2023-11-24T07:58:13.653 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:13.649+0000 7f2367f26700 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2360144ac0 con 0x7f23601446e0 2023-11-24T07:58:13.653 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:13.649+0000 7f2367f26700 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f23600ce4a0 con 0x7f236006f850 2023-11-24T07:58:13.653 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:13.649+0000 7f2365cc2700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f23601446e0 0x7f23600cdf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:13.653 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:13.649+0000 7f2365cc2700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f23601446e0 0x7f23600cdf60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:53004/0 (socket says 172.21.15.59:53004) 2023-11-24T07:58:13.653 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:13.649+0000 7f2365cc2700 1 -- 172.21.15.59:0/689157499 learned_addr learned my addr 172.21.15.59:0/689157499 (peer_addr_for_me v2:172.21.15.59:0/0) 2023-11-24T07:58:13.653 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:13.650+0000 7f23664c3700 1 --2- 172.21.15.59:0/689157499 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f236006f850 0x7f236006e780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:13.654 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:13.650+0000 7f2365cc2700 1 -- 172.21.15.59:0/689157499 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f236006f850 msgr2=0x7f236006e780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T07:58:13.654 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:13.650+0000 7f2365cc2700 1 --2- 172.21.15.59:0/689157499 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f236006f850 0x7f236006e780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:13.654 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:13.650+0000 7f2365cc2700 1 -- 172.21.15.59:0/689157499 >> v1:172.21.15.76:6789/0 conn(0x7f236006e2d0 legacy=0x7f236006f140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:13.654 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:13.650+0000 7f2365cc2700 1 -- 172.21.15.59:0/689157499 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f236006ed90 con 0x7f23601446e0 2023-11-24T07:58:13.654 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:13.650+0000 7f23664c3700 1 --2- 172.21.15.59:0/689157499 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f236006f850 0x7f236006e780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-24T07:58:13.654 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:13.650+0000 7f2365cc2700 1 --2- 172.21.15.59:0/689157499 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f23601446e0 0x7f23600cdf60 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7f235000bb90 tx=0x7f235000bea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d0c02d1b2ea5416b server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:58:13.654 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:13.650+0000 7f2364cc0700 1 -- 172.21.15.59:0/689157499 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f235000dd50 con 0x7f23601446e0 2023-11-24T07:58:13.654 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:13.650+0000 7f2364cc0700 1 -- 172.21.15.59:0/689157499 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2350010470 con 0x7f23601446e0 2023-11-24T07:58:13.654 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:13.650+0000 7f2364cc0700 1 -- 172.21.15.59:0/689157499 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2350010800 con 0x7f23601446e0 2023-11-24T07:58:13.654 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:13.651+0000 7f2367f26700 1 -- 172.21.15.59:0/689157499 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f23601446e0 msgr2=0x7f23600cdf60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:13.654 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:13.651+0000 7f2367f26700 1 --2- 172.21.15.59:0/689157499 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f23601446e0 0x7f23600cdf60 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7f235000bb90 tx=0x7f235000bea0 comp rx=0 tx=0).stop 2023-11-24T07:58:13.654 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:13.651+0000 7f2367f26700 1 -- 172.21.15.59:0/689157499 shutdown_connections 2023-11-24T07:58:13.655 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:13.651+0000 7f2367f26700 1 --2- 172.21.15.59:0/689157499 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f236006f850 0x7f236006e780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:13.655 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:13.651+0000 7f2367f26700 1 --2- 172.21.15.59:0/689157499 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f23601446e0 0x7f23600cdf60 unknown :-1 s=CLOSED pgs=577 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:13.655 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:13.651+0000 7f2367f26700 1 -- 172.21.15.59:0/689157499 >> 172.21.15.59:0/689157499 conn(0x7f23600d0aa0 msgr2=0x7f23600d0eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:58:13.655 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:13.651+0000 7f2367f26700 1 -- 172.21.15.59:0/689157499 shutdown_connections 2023-11-24T07:58:13.655 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:13.651+0000 7f2367f26700 1 -- 172.21.15.59:0/689157499 wait complete. 2023-11-24T07:58:13.658 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:13.652+0000 7f2367f26700 1 Processor -- start 2023-11-24T07:58:13.658 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:13.652+0000 7f2367f26700 1 -- start start 2023-11-24T07:58:13.658 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:13.653+0000 7f2367f26700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f236006e2d0 0x7f236013f3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:13.658 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:13.653+0000 7f2367f26700 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f236006f850 0x7f236013f910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:13.658 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:13.653+0000 7f2367f26700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f23601446e0 0x7f236014b2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:13.658 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:13.653+0000 7f2367f26700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f236014c230 con 0x7f236006e2d0 2023-11-24T07:58:13.658 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:13.653+0000 7f2367f26700 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f236014bf50 con 0x7f23601446e0 2023-11-24T07:58:13.658 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:13.653+0000 7f2367f26700 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2360149350 con 0x7f236006f850 2023-11-24T07:58:13.658 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:13.653+0000 7f2365cc2700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f236006e2d0 0x7f236013f3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:13.658 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:13.653+0000 7f2365cc2700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f236006e2d0 0x7f236013f3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:53010/0 (socket says 172.21.15.59:53010) 2023-11-24T07:58:13.659 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:13.653+0000 7f2365cc2700 1 -- 172.21.15.59:0/4205064376 learned_addr learned my addr 172.21.15.59:0/4205064376 (peer_addr_for_me v2:172.21.15.59:0/0) 2023-11-24T07:58:13.659 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:13.653+0000 7f23654c1700 1 --2- 172.21.15.59:0/4205064376 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f236006f850 0x7f236013f910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:13.659 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:13.653+0000 7f23664c3700 1 --2- 172.21.15.59:0/4205064376 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f23601446e0 0x7f236014b2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:13.659 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:13.653+0000 7f2365cc2700 1 -- 172.21.15.59:0/4205064376 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f236006f850 msgr2=0x7f236013f910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:13.659 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:13.653+0000 7f2365cc2700 1 --2- 172.21.15.59:0/4205064376 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f236006f850 0x7f236013f910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:13.659 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:13.653+0000 7f2365cc2700 1 -- 172.21.15.59:0/4205064376 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f23601446e0 msgr2=0x7f236014b2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:13.659 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:13.653+0000 7f2365cc2700 1 --2- 172.21.15.59:0/4205064376 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f23601446e0 0x7f236014b2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:13.659 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:13.653+0000 7f2365cc2700 1 -- 172.21.15.59:0/4205064376 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f235000b7f0 con 0x7f236006e2d0 2023-11-24T07:58:13.659 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:13.653+0000 7f2365cc2700 1 --2- 172.21.15.59:0/4205064376 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f236006e2d0 0x7f236013f3d0 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7f235000bee0 tx=0x7f2350004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:58:13.659 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:13.653+0000 7f23567fc700 1 -- 172.21.15.59:0/4205064376 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f235001f070 con 0x7f236006e2d0 2023-11-24T07:58:13.659 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:13.654+0000 7f23567fc700 1 -- 172.21.15.59:0/4205064376 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f23500038b0 con 0x7f236006e2d0 2023-11-24T07:58:13.659 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:13.654+0000 7f2367f26700 1 -- 172.21.15.59:0/4205064376 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2360149630 con 0x7f236006e2d0 2023-11-24T07:58:13.659 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:13.654+0000 7f23567fc700 1 -- 172.21.15.59:0/4205064376 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2350003c40 con 0x7f236006e2d0 2023-11-24T07:58:13.660 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:13.654+0000 7f2367f26700 1 -- 172.21.15.59:0/4205064376 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f23600b0060 con 0x7f236006e2d0 2023-11-24T07:58:13.660 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:13.654+0000 7f2367f26700 1 -- 172.21.15.59:0/4205064376 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2360149c80 con 0x7f236006e2d0 2023-11-24T07:58:13.660 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:13.655+0000 7f23567fc700 1 -- 172.21.15.59:0/4205064376 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 6) v1 ==== 89282+0+0 (secure 0 0 0) 0x7f2350019400 con 0x7f236006e2d0 2023-11-24T07:58:13.660 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:13.655+0000 7f23567fc700 1 --2- 172.21.15.59:0/4205064376 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f234406af60 0x7f234406d420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:13.660 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:13.655+0000 7f23567fc700 1 -- 172.21.15.59:0/4205064376 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5854+0+0 (secure 0 0 0) 0x7f235008d2e0 con 0x7f236006e2d0 2023-11-24T07:58:13.660 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:13.656+0000 7f23654c1700 1 --2- 172.21.15.59:0/4205064376 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f234406af60 0x7f234406d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:13.660 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:13.656+0000 7f23654c1700 1 --2- 172.21.15.59:0/4205064376 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f234406af60 0x7f234406d420 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f236015a800 tx=0x7f235c009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:58:13.661 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:13.660+0000 7f23567fc700 1 -- 172.21.15.59:0/4205064376 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f2350058190 con 0x7f236006e2d0 2023-11-24T07:58:13.895 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:13.894+0000 7f2367f26700 1 -- 172.21.15.59:0/4205064376 --> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "subvol4495", "group_name": "subvol_grp1023", "target": ["mon-mgr", ""]}) v1 -- 0x7f23600b4130 con 0x7f234406af60 2023-11-24T07:58:13.920 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:13.919+0000 7f23567fc700 1 -- 172.21.15.59:0/4205064376 <== mgr.4100 v2:172.21.15.76:6832/4264046536 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7f235c001d60 con 0x7f234406af60 2023-11-24T07:58:13.923 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:13.921+0000 7f2367f26700 1 -- 172.21.15.59:0/4205064376 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f234406af60 msgr2=0x7f234406d420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:13.923 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:13.921+0000 7f2367f26700 1 --2- 172.21.15.59:0/4205064376 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f234406af60 0x7f234406d420 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f236015a800 tx=0x7f235c009040 comp rx=0 tx=0).stop 2023-11-24T07:58:13.924 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:13.921+0000 7f2367f26700 1 -- 172.21.15.59:0/4205064376 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f236006e2d0 msgr2=0x7f236013f3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:13.924 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:13.921+0000 7f2367f26700 1 --2- 172.21.15.59:0/4205064376 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f236006e2d0 0x7f236013f3d0 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7f235000bee0 tx=0x7f2350004750 comp rx=0 tx=0).stop 2023-11-24T07:58:13.924 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:13.921+0000 7f2367f26700 1 -- 172.21.15.59:0/4205064376 shutdown_connections 2023-11-24T07:58:13.924 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:13.921+0000 7f2367f26700 1 --2- 172.21.15.59:0/4205064376 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f234406af60 0x7f234406d420 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:13.924 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:13.921+0000 7f2367f26700 1 --2- 172.21.15.59:0/4205064376 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f23601446e0 0x7f236014b2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:13.924 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:13.921+0000 7f2367f26700 1 --2- 172.21.15.59:0/4205064376 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f236006e2d0 0x7f236013f3d0 unknown :-1 s=CLOSED pgs=578 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:13.924 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:13.921+0000 7f2367f26700 1 --2- 172.21.15.59:0/4205064376 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f236006f850 0x7f236013f910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:13.924 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:13.921+0000 7f2367f26700 1 -- 172.21.15.59:0/4205064376 >> 172.21.15.59:0/4205064376 conn(0x7f23600d0aa0 msgr2=0x7f236006db60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:58:13.924 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:13.921+0000 7f2367f26700 1 -- 172.21.15.59:0/4205064376 shutdown_connections 2023-11-24T07:58:13.924 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:13.921+0000 7f2367f26700 1 -- 172.21.15.59:0/4205064376 wait complete. 2023-11-24T07:58:13.935 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot create cephfs subvol4495 subvol_snap1172 subvol_grp1023 2023-11-24T07:58:14.066 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:14.064+0000 7f905dd95700 1 Processor -- start 2023-11-24T07:58:14.066 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:14.064+0000 7f905dd95700 1 -- start start 2023-11-24T07:58:14.070 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:14.065+0000 7f905dd95700 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f9058143ae0 0x7f90580b51b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:14.071 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:14.065+0000 7f905dd95700 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9058143ec0 con 0x7f90580722c0 2023-11-24T07:58:14.071 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:14.065+0000 7f905dd95700 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f90580b3ae0 con 0x7f90581440f0 2023-11-24T07:58:14.071 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:14.065+0000 7f905dd95700 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f90580b40a0 con 0x7f9058143ae0 2023-11-24T07:58:14.071 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:14.065+0000 7f9057fff700 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7f90580722c0 0x7f905806e510 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:56496/0 (socket says 172.21.15.59:56496) 2023-11-24T07:58:14.071 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:14.065+0000 7f9057fff700 1 -- 172.21.15.59:0/3446198906 learned_addr learned my addr 172.21.15.59:0/3446198906 (peer_addr_for_me v1:172.21.15.59:0/0) 2023-11-24T07:58:14.071 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:14.065+0000 7f90577fe700 1 --2- 172.21.15.59:0/3446198906 >> v2:172.21.15.76:3300/0 conn(0x7f9058143ae0 0x7f90580b51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:14.071 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:14.066+0000 7f90567fc700 1 -- 172.21.15.59:0/3446198906 <== mon.0 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 905371027 0 0) 0x7f9058143ec0 con 0x7f90580722c0 2023-11-24T07:58:14.072 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:14.066+0000 7f90567fc700 1 -- 172.21.15.59:0/3446198906 --> v1:172.21.15.59:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f90440035b0 con 0x7f90580722c0 2023-11-24T07:58:14.072 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:14.066+0000 7f90567fc700 1 -- 172.21.15.59:0/3446198906 <== mon.1 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2621320590 0 0) 0x7f90580b3ae0 con 0x7f90581440f0 2023-11-24T07:58:14.072 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:14.066+0000 7f90567fc700 1 -- 172.21.15.59:0/3446198906 --> v1:172.21.15.76:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9058143ec0 con 0x7f90581440f0 2023-11-24T07:58:14.072 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:14.066+0000 7f90567fc700 1 -- 172.21.15.59:0/3446198906 <== mon.0 v1:172.21.15.59:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1213471377 0 0) 0x7f90440035b0 con 0x7f90580722c0 2023-11-24T07:58:14.072 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:14.066+0000 7f90567fc700 1 -- 172.21.15.59:0/3446198906 >> v2:172.21.15.76:3300/0 conn(0x7f9058143ae0 msgr2=0x7f90580b51b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T07:58:14.072 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:14.066+0000 7f90567fc700 1 --2- 172.21.15.59:0/3446198906 >> v2:172.21.15.76:3300/0 conn(0x7f9058143ae0 0x7f90580b51b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:14.072 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:14.066+0000 7f90567fc700 1 -- 172.21.15.59:0/3446198906 >> v1:172.21.15.76:6789/0 conn(0x7f90581440f0 legacy=0x7f9058071bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:14.073 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:14.066+0000 7f90567fc700 1 -- 172.21.15.59:0/3446198906 --> v1:172.21.15.59:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f90580b3dc0 con 0x7f90580722c0 2023-11-24T07:58:14.073 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:14.066+0000 7f90567fc700 1 -- 172.21.15.59:0/3446198906 <== mon.0 v1:172.21.15.59:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1312333566 0 0) 0x7f9050001970 con 0x7f90580722c0 2023-11-24T07:58:14.073 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:14.066+0000 7f90567fc700 1 -- 172.21.15.59:0/3446198906 >> v1:172.21.15.59:6789/0 conn(0x7f90580722c0 legacy=0x7f905806e510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:14.073 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:14.066+0000 7f90567fc700 1 --2- 172.21.15.59:0/3446198906 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9044003be0 0x7f9044003fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:14.073 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:14.066+0000 7f90567fc700 1 --2- 172.21.15.59:0/3446198906 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9044004850 0x7f9044004cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:14.073 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:14.066+0000 7f90567fc700 1 --2- 172.21.15.59:0/3446198906 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f9044005210 0x7f904400ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:14.074 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:14.066+0000 7f90567fc700 1 -- 172.21.15.59:0/3446198906 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f90580b40a0 con 0x7f9044003be0 2023-11-24T07:58:14.074 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:14.066+0000 7f90567fc700 1 -- 172.21.15.59:0/3446198906 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f904400c0e0 con 0x7f9044004850 2023-11-24T07:58:14.074 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:14.066+0000 7f90567fc700 1 -- 172.21.15.59:0/3446198906 --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f904400c2a0 con 0x7f9044005210 2023-11-24T07:58:14.074 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:14.066+0000 7f90577fe700 1 --2- 172.21.15.59:0/3446198906 >> v2:172.21.15.76:3300/0 conn(0x7f9058143ae0 0x7f90580b51b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-24T07:58:14.074 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:14.066+0000 7f90577fe700 1 --2- 172.21.15.59:0/3446198906 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9044003be0 0x7f9044003fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:14.074 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:14.066+0000 7f9056ffd700 1 --2- 172.21.15.59:0/3446198906 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9044004850 0x7f9044004cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:14.075 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:14.066+0000 7f9057fff700 1 --2- 172.21.15.59:0/3446198906 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f9044005210 0x7f904400ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:14.075 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:14.066+0000 7f90577fe700 1 -- 172.21.15.59:0/3446198906 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f9044005210 msgr2=0x7f904400ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T07:58:14.075 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:14.066+0000 7f90577fe700 1 --2- 172.21.15.59:0/3446198906 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f9044005210 0x7f904400ba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:14.075 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:14.066+0000 7f90577fe700 1 -- 172.21.15.59:0/3446198906 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9044004850 msgr2=0x7f9044004cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T07:58:14.075 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:14.066+0000 7f90577fe700 1 --2- 172.21.15.59:0/3446198906 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9044004850 0x7f9044004cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:14.075 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:14.066+0000 7f90577fe700 1 -- 172.21.15.59:0/3446198906 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f90440035b0 con 0x7f9044003be0 2023-11-24T07:58:14.075 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:14.066+0000 7f9057fff700 1 -- 172.21.15.59:0/3446198906 reap_dead start 2023-11-24T07:58:14.076 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:14.067+0000 7f90577fe700 1 --2- 172.21.15.59:0/3446198906 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9044003be0 0x7f9044003fe0 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7f9048009d10 tx=0x7f9044005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ea3a2a84f8961bd7 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:58:14.076 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:14.067+0000 7f90567fc700 1 -- 172.21.15.59:0/3446198906 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9048009e60 con 0x7f9044003be0 2023-11-24T07:58:14.076 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:14.067+0000 7f90567fc700 1 -- 172.21.15.59:0/3446198906 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f904800e9a0 con 0x7f9044003be0 2023-11-24T07:58:14.076 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:14.067+0000 7f90567fc700 1 -- 172.21.15.59:0/3446198906 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f904800ed30 con 0x7f9044003be0 2023-11-24T07:58:14.076 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:14.067+0000 7f905dd95700 1 -- 172.21.15.59:0/3446198906 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9044003be0 msgr2=0x7f9044003fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:14.076 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:14.067+0000 7f905dd95700 1 --2- 172.21.15.59:0/3446198906 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9044003be0 0x7f9044003fe0 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7f9048009d10 tx=0x7f9044005880 comp rx=0 tx=0).stop 2023-11-24T07:58:14.077 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:14.067+0000 7f905dd95700 1 -- 172.21.15.59:0/3446198906 shutdown_connections 2023-11-24T07:58:14.077 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:14.067+0000 7f905dd95700 1 --2- 172.21.15.59:0/3446198906 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9044003be0 0x7f9044003fe0 unknown :-1 s=CLOSED pgs=580 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:14.077 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:14.067+0000 7f905dd95700 1 -- 172.21.15.59:0/3446198906 >> 172.21.15.59:0/3446198906 conn(0x7f90580b2740 msgr2=0x7f90580b2b20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:58:14.077 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:14.067+0000 7f905dd95700 1 -- 172.21.15.59:0/3446198906 shutdown_connections 2023-11-24T07:58:14.077 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:14.067+0000 7f905dd95700 1 -- 172.21.15.59:0/3446198906 wait complete. 2023-11-24T07:58:14.077 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:14.069+0000 7f905dd95700 1 Processor -- start 2023-11-24T07:58:14.077 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:14.069+0000 7f905dd95700 1 -- start start 2023-11-24T07:58:14.080 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:14.069+0000 7f905dd95700 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f9044003be0 0x7f90581397a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:14.081 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:14.070+0000 7f905dd95700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f905806afd0 0x7f905806b450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:14.081 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:14.070+0000 7f905dd95700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9058154bb0 0x7f9058155030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:14.081 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:14.070+0000 7f905dd95700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9058139ce0 con 0x7f9058154bb0 2023-11-24T07:58:14.081 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:14.070+0000 7f905dd95700 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f90581e5c70 con 0x7f905806afd0 2023-11-24T07:58:14.081 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:14.070+0000 7f905dd95700 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f905806b990 con 0x7f9044003be0 2023-11-24T07:58:14.081 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:14.070+0000 7f9057fff700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9058154bb0 0x7f9058155030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:14.081 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:14.070+0000 7f9057fff700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9058154bb0 0x7f9058155030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:53034/0 (socket says 172.21.15.59:53034) 2023-11-24T07:58:14.082 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:14.070+0000 7f9057fff700 1 -- 172.21.15.59:0/3112745477 learned_addr learned my addr 172.21.15.59:0/3112745477 (peer_addr_for_me v2:172.21.15.59:0/0) 2023-11-24T07:58:14.082 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:14.070+0000 7f90577fe700 1 --2- 172.21.15.59:0/3112745477 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f9044003be0 0x7f90581397a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:14.082 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:14.070+0000 7f9056ffd700 1 --2- 172.21.15.59:0/3112745477 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f905806afd0 0x7f905806b450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:14.082 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:14.070+0000 7f9057fff700 1 -- 172.21.15.59:0/3112745477 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f9044003be0 msgr2=0x7f90581397a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:14.082 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:14.070+0000 7f9057fff700 1 --2- 172.21.15.59:0/3112745477 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f9044003be0 0x7f90581397a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:14.082 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:14.070+0000 7f9057fff700 1 -- 172.21.15.59:0/3112745477 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f905806afd0 msgr2=0x7f905806b450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:14.083 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:14.070+0000 7f9057fff700 1 --2- 172.21.15.59:0/3112745477 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f905806afd0 0x7f905806b450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:14.083 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:14.070+0000 7f9057fff700 1 -- 172.21.15.59:0/3112745477 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f90580ca660 con 0x7f9058154bb0 2023-11-24T07:58:14.083 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:14.070+0000 7f9056ffd700 1 --2- 172.21.15.59:0/3112745477 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f905806afd0 0x7f905806b450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-24T07:58:14.083 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:14.070+0000 7f9057fff700 1 --2- 172.21.15.59:0/3112745477 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9058154bb0 0x7f9058155030 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7f9050001330 tx=0x7f905000f4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:58:14.083 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:14.070+0000 7f904ffff700 1 -- 172.21.15.59:0/3112745477 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f905000fc00 con 0x7f9058154bb0 2023-11-24T07:58:14.083 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:14.070+0000 7f904ffff700 1 -- 172.21.15.59:0/3112745477 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f905000fd60 con 0x7f9058154bb0 2023-11-24T07:58:14.083 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:14.070+0000 7f905dd95700 1 -- 172.21.15.59:0/3112745477 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9058155570 con 0x7f9058154bb0 2023-11-24T07:58:14.084 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:14.070+0000 7f904ffff700 1 -- 172.21.15.59:0/3112745477 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9050013630 con 0x7f9058154bb0 2023-11-24T07:58:14.084 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:14.070+0000 7f905dd95700 1 -- 172.21.15.59:0/3112745477 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9058155ae0 con 0x7f9058154bb0 2023-11-24T07:58:14.084 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:14.071+0000 7f905dd95700 1 -- 172.21.15.59:0/3112745477 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9058067a40 con 0x7f9058154bb0 2023-11-24T07:58:14.084 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:14.072+0000 7f904ffff700 1 -- 172.21.15.59:0/3112745477 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 6) v1 ==== 89282+0+0 (secure 0 0 0) 0x7f9050013790 con 0x7f9058154bb0 2023-11-24T07:58:14.084 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:14.072+0000 7f904ffff700 1 --2- 172.21.15.59:0/3112745477 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f903806af60 0x7f903806d420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:14.084 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:14.072+0000 7f904ffff700 1 -- 172.21.15.59:0/3112745477 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5854+0+0 (secure 0 0 0) 0x7f9050089b80 con 0x7f9058154bb0 2023-11-24T07:58:14.084 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:14.072+0000 7f90577fe700 1 --2- 172.21.15.59:0/3112745477 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f903806af60 0x7f903806d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:14.085 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:14.073+0000 7f90577fe700 1 --2- 172.21.15.59:0/3112745477 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f903806af60 0x7f903806d420 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f90480090b0 tx=0x7f904801e040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:58:14.085 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:14.077+0000 7f904ffff700 1 -- 172.21.15.59:0/3112745477 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f9058067a40 con 0x7f9058154bb0 2023-11-24T07:58:14.318 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:14.317+0000 7f905dd95700 1 -- 172.21.15.59:0/3112745477 --> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot create", "vol_name": "cephfs", "sub_name": "subvol4495", "snap_name": "subvol_snap1172", "group_name": "subvol_grp1023", "target": ["mon-mgr", ""]}) v1 -- 0x7f9058144830 con 0x7f903806af60 2023-11-24T07:58:14.333 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:14.331+0000 7f904ffff700 1 -- 172.21.15.59:0/3112745477 <== mgr.4100 v2:172.21.15.76:6832/4264046536 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7f9058144830 con 0x7f903806af60 2023-11-24T07:58:14.335 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:14.333+0000 7f905dd95700 1 -- 172.21.15.59:0/3112745477 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f903806af60 msgr2=0x7f903806d420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:14.335 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:14.333+0000 7f905dd95700 1 --2- 172.21.15.59:0/3112745477 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f903806af60 0x7f903806d420 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f90480090b0 tx=0x7f904801e040 comp rx=0 tx=0).stop 2023-11-24T07:58:14.336 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:14.333+0000 7f905dd95700 1 -- 172.21.15.59:0/3112745477 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9058154bb0 msgr2=0x7f9058155030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:14.336 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:14.333+0000 7f905dd95700 1 --2- 172.21.15.59:0/3112745477 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9058154bb0 0x7f9058155030 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7f9050001330 tx=0x7f905000f4b0 comp rx=0 tx=0).stop 2023-11-24T07:58:14.336 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:14.334+0000 7f905dd95700 1 -- 172.21.15.59:0/3112745477 shutdown_connections 2023-11-24T07:58:14.336 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:14.334+0000 7f905dd95700 1 --2- 172.21.15.59:0/3112745477 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f903806af60 0x7f903806d420 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:14.336 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:14.334+0000 7f905dd95700 1 --2- 172.21.15.59:0/3112745477 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9058154bb0 0x7f9058155030 unknown :-1 s=CLOSED pgs=581 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:14.336 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:14.334+0000 7f905dd95700 1 --2- 172.21.15.59:0/3112745477 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f9044003be0 0x7f90581397a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:14.336 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:14.334+0000 7f905dd95700 1 --2- 172.21.15.59:0/3112745477 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f905806afd0 0x7f905806b450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:14.336 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:14.334+0000 7f905dd95700 1 -- 172.21.15.59:0/3112745477 >> 172.21.15.59:0/3112745477 conn(0x7f90580b2740 msgr2=0x7f905806e410 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:58:14.336 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:14.334+0000 7f905dd95700 1 -- 172.21.15.59:0/3112745477 shutdown_connections 2023-11-24T07:58:14.336 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:14.334+0000 7f905dd95700 1 -- 172.21.15.59:0/3112745477 wait complete. 2023-11-24T07:58:14.347 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot metadata set cephfs subvol4495 subvol_snap1172 key value subvol_grp1023 2023-11-24T07:58:14.484 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:14.482+0000 7fc071239700 1 Processor -- start 2023-11-24T07:58:14.484 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:14.482+0000 7fc071239700 1 -- start start 2023-11-24T07:58:14.486 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:14.483+0000 7fc071239700 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fc06c0b21d0 0x7fc06c0b2dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:14.486 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:14.483+0000 7fc071239700 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc06c06caf0 con 0x7fc06c0b28a0 2023-11-24T07:58:14.486 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:14.483+0000 7fc071239700 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc06c0cf250 con 0x7fc06c0d0ff0 2023-11-24T07:58:14.486 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:14.483+0000 7fc071239700 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc06c06cdb0 con 0x7fc06c0b21d0 2023-11-24T07:58:14.487 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:14.483+0000 7fc06a59c700 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7fc06c0b28a0 0x7fc06c0ce4d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:56514/0 (socket says 172.21.15.59:56514) 2023-11-24T07:58:14.487 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:14.483+0000 7fc06a59c700 1 -- 172.21.15.59:0/3539961448 learned_addr learned my addr 172.21.15.59:0/3539961448 (peer_addr_for_me v1:172.21.15.59:0/0) 2023-11-24T07:58:14.487 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:14.483+0000 7fc06ad9d700 1 --2- 172.21.15.59:0/3539961448 >> v2:172.21.15.76:3300/0 conn(0x7fc06c0b21d0 0x7fc06c0b2dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:14.487 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:14.483+0000 7fc069d9b700 1 -- 172.21.15.59:0/3539961448 <== mon.0 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 637413699 0 0) 0x7fc06c06caf0 con 0x7fc06c0b28a0 2023-11-24T07:58:14.487 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:14.483+0000 7fc069d9b700 1 -- 172.21.15.59:0/3539961448 --> v1:172.21.15.59:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc0580035b0 con 0x7fc06c0b28a0 2023-11-24T07:58:14.487 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:14.484+0000 7fc06ad9d700 1 -- 172.21.15.59:0/3539961448 >> v1:172.21.15.76:6789/0 conn(0x7fc06c0d0ff0 legacy=0x7fc06c0cc2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:14.487 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:14.484+0000 7fc06ad9d700 1 -- 172.21.15.59:0/3539961448 >> v1:172.21.15.59:6789/0 conn(0x7fc06c0b28a0 legacy=0x7fc06c0ce4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:14.487 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:14.484+0000 7fc069d9b700 1 -- 172.21.15.59:0/3539961448 <== mon.0 v1:172.21.15.59:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1030058471 0 0) 0x7fc0580035b0 con 0x7fc06c0b28a0 2023-11-24T07:58:14.487 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:14.484+0000 7fc06ad9d700 1 -- 172.21.15.59:0/3539961448 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc06c06d090 con 0x7fc06c0b21d0 2023-11-24T07:58:14.487 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:14.484+0000 7fc06ad9d700 1 --2- 172.21.15.59:0/3539961448 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc06c0b21d0 0x7fc06c0b2dd0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fc064009840 tx=0x7fc064009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e79caff8a15ac70b server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:58:14.487 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:14.484+0000 7fc069d9b700 1 -- 172.21.15.59:0/3539961448 <== mon.1 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc064011830 con 0x7fc06c0b21d0 2023-11-24T07:58:14.487 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:14.484+0000 7fc069d9b700 1 -- 172.21.15.59:0/3539961448 <== mon.1 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc064011990 con 0x7fc06c0b21d0 2023-11-24T07:58:14.487 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:14.484+0000 7fc069d9b700 1 -- 172.21.15.59:0/3539961448 <== mon.1 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc064011d20 con 0x7fc06c0b21d0 2023-11-24T07:58:14.488 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:14.485+0000 7fc071239700 1 -- 172.21.15.59:0/3539961448 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc06c0b21d0 msgr2=0x7fc06c0b2dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:14.488 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:14.485+0000 7fc071239700 1 --2- 172.21.15.59:0/3539961448 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc06c0b21d0 0x7fc06c0b2dd0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fc064009840 tx=0x7fc064009b50 comp rx=0 tx=0).stop 2023-11-24T07:58:14.488 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:14.485+0000 7fc071239700 1 -- 172.21.15.59:0/3539961448 shutdown_connections 2023-11-24T07:58:14.488 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:14.485+0000 7fc071239700 1 --2- 172.21.15.59:0/3539961448 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc06c0b21d0 0x7fc06c0b2dd0 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:14.488 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:14.485+0000 7fc071239700 1 -- 172.21.15.59:0/3539961448 >> 172.21.15.59:0/3539961448 conn(0x7fc06c0b3800 msgr2=0x7fc06c0b3c10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:58:14.488 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:14.485+0000 7fc071239700 1 -- 172.21.15.59:0/3539961448 shutdown_connections 2023-11-24T07:58:14.488 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:14.485+0000 7fc071239700 1 -- 172.21.15.59:0/3539961448 wait complete. 2023-11-24T07:58:14.490 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:14.486+0000 7fc071239700 1 Processor -- start 2023-11-24T07:58:14.490 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:14.486+0000 7fc071239700 1 -- start start 2023-11-24T07:58:14.490 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:14.487+0000 7fc071239700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc06c0b21d0 0x7fc06c154110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:14.491 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:14.487+0000 7fc071239700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc06c0d0ff0 0x7fc06c0b6ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:14.491 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:14.487+0000 7fc071239700 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fc06c0b7000 0x7fc06c09e400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:14.491 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:14.487+0000 7fc071239700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc06c09fc60 con 0x7fc06c0d0ff0 2023-11-24T07:58:14.491 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:14.487+0000 7fc071239700 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc06c0b7490 con 0x7fc06c0b21d0 2023-11-24T07:58:14.491 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:14.487+0000 7fc071239700 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc06c1607a0 con 0x7fc06c0b7000 2023-11-24T07:58:14.491 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:14.487+0000 7fc06a59c700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc06c0d0ff0 0x7fc06c0b6ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:14.491 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:14.487+0000 7fc06a59c700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc06c0d0ff0 0x7fc06c0b6ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:53046/0 (socket says 172.21.15.59:53046) 2023-11-24T07:58:14.491 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:14.487+0000 7fc06a59c700 1 -- 172.21.15.59:0/3120226933 learned_addr learned my addr 172.21.15.59:0/3120226933 (peer_addr_for_me v2:172.21.15.59:0/0) 2023-11-24T07:58:14.491 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:14.487+0000 7fc06ad9d700 1 --2- 172.21.15.59:0/3120226933 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc06c0b21d0 0x7fc06c154110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:14.491 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:14.487+0000 7fc06b59e700 1 --2- 172.21.15.59:0/3120226933 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fc06c0b7000 0x7fc06c09e400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:14.491 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:14.487+0000 7fc06a59c700 1 -- 172.21.15.59:0/3120226933 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fc06c0b7000 msgr2=0x7fc06c09e400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:14.491 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:14.487+0000 7fc06a59c700 1 --2- 172.21.15.59:0/3120226933 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fc06c0b7000 0x7fc06c09e400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:14.492 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:14.487+0000 7fc06a59c700 1 -- 172.21.15.59:0/3120226933 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc06c0b21d0 msgr2=0x7fc06c154110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:14.492 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:14.487+0000 7fc06a59c700 1 --2- 172.21.15.59:0/3120226933 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc06c0b21d0 0x7fc06c154110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:14.492 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:14.487+0000 7fc06a59c700 1 -- 172.21.15.59:0/3120226933 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc0640094a0 con 0x7fc06c0d0ff0 2023-11-24T07:58:14.492 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:14.488+0000 7fc06a59c700 1 --2- 172.21.15.59:0/3120226933 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc06c0d0ff0 0x7fc06c0b6ac0 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7fc05400c780 tx=0x7fc05400ca90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:58:14.492 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:14.488+0000 7fc0637fe700 1 -- 172.21.15.59:0/3120226933 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc054005d70 con 0x7fc06c0d0ff0 2023-11-24T07:58:14.492 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:14.488+0000 7fc0637fe700 1 -- 172.21.15.59:0/3120226933 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc0540070d0 con 0x7fc06c0d0ff0 2023-11-24T07:58:14.492 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:14.488+0000 7fc071239700 1 -- 172.21.15.59:0/3120226933 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc06c154650 con 0x7fc06c0d0ff0 2023-11-24T07:58:14.492 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:14.488+0000 7fc0637fe700 1 -- 172.21.15.59:0/3120226933 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc054007480 con 0x7fc06c0d0ff0 2023-11-24T07:58:14.492 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:14.488+0000 7fc071239700 1 -- 172.21.15.59:0/3120226933 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc06c09f3c0 con 0x7fc06c0d0ff0 2023-11-24T07:58:14.492 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:14.489+0000 7fc071239700 1 -- 172.21.15.59:0/3120226933 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc06c075480 con 0x7fc06c0d0ff0 2023-11-24T07:58:14.493 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:14.489+0000 7fc0637fe700 1 -- 172.21.15.59:0/3120226933 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 6) v1 ==== 89282+0+0 (secure 0 0 0) 0x7fc0540075e0 con 0x7fc06c0d0ff0 2023-11-24T07:58:14.493 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:14.490+0000 7fc0637fe700 1 --2- 172.21.15.59:0/3120226933 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7fc04c06af60 0x7fc04c06d420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:14.493 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:14.490+0000 7fc0637fe700 1 -- 172.21.15.59:0/3120226933 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5854+0+0 (secure 0 0 0) 0x7fc054088b20 con 0x7fc06c0d0ff0 2023-11-24T07:58:14.493 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:14.490+0000 7fc06ad9d700 1 --2- 172.21.15.59:0/3120226933 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7fc04c06af60 0x7fc04c06d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:14.493 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:14.490+0000 7fc06ad9d700 1 --2- 172.21.15.59:0/3120226933 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7fc04c06af60 0x7fc04c06d420 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7fc064009fd0 tx=0x7fc06401c040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:58:14.495 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:14.494+0000 7fc0637fe700 1 -- 172.21.15.59:0/3120226933 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fc054053ac0 con 0x7fc06c0d0ff0 2023-11-24T07:58:14.737 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:14.735+0000 7fc071239700 1 -- 172.21.15.59:0/3120226933 --> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot metadata set", "vol_name": "cephfs", "sub_name": "subvol4495", "snap_name": "subvol_snap1172", "key_name": "key", "value": "value", "group_name": "subvol_grp1023", "target": ["mon-mgr", ""]}) v1 -- 0x7fc06c0b1b30 con 0x7fc04c06af60 2023-11-24T07:58:14.753 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:14.752+0000 7fc0637fe700 1 -- 172.21.15.59:0/3120226933 <== mgr.4100 v2:172.21.15.76:6832/4264046536 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7fc06c0b1b30 con 0x7fc04c06af60 2023-11-24T07:58:14.756 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:14.754+0000 7fc071239700 1 -- 172.21.15.59:0/3120226933 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7fc04c06af60 msgr2=0x7fc04c06d420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:14.756 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:14.754+0000 7fc071239700 1 --2- 172.21.15.59:0/3120226933 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7fc04c06af60 0x7fc04c06d420 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7fc064009fd0 tx=0x7fc06401c040 comp rx=0 tx=0).stop 2023-11-24T07:58:14.756 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:14.754+0000 7fc071239700 1 -- 172.21.15.59:0/3120226933 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc06c0d0ff0 msgr2=0x7fc06c0b6ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:14.756 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:14.754+0000 7fc071239700 1 --2- 172.21.15.59:0/3120226933 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc06c0d0ff0 0x7fc06c0b6ac0 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7fc05400c780 tx=0x7fc05400ca90 comp rx=0 tx=0).stop 2023-11-24T07:58:14.756 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:14.754+0000 7fc071239700 1 -- 172.21.15.59:0/3120226933 shutdown_connections 2023-11-24T07:58:14.757 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:14.754+0000 7fc071239700 1 --2- 172.21.15.59:0/3120226933 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7fc04c06af60 0x7fc04c06d420 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:14.757 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:14.754+0000 7fc071239700 1 --2- 172.21.15.59:0/3120226933 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fc06c0b7000 0x7fc06c09e400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:14.757 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:14.754+0000 7fc071239700 1 --2- 172.21.15.59:0/3120226933 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc06c0b21d0 0x7fc06c154110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:14.757 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:14.754+0000 7fc071239700 1 --2- 172.21.15.59:0/3120226933 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc06c0d0ff0 0x7fc06c0b6ac0 unknown :-1 s=CLOSED pgs=583 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:14.757 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:14.754+0000 7fc071239700 1 -- 172.21.15.59:0/3120226933 >> 172.21.15.59:0/3120226933 conn(0x7fc06c0b3800 msgr2=0x7fc06c0b1420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:58:14.757 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:14.754+0000 7fc071239700 1 -- 172.21.15.59:0/3120226933 shutdown_connections 2023-11-24T07:58:14.757 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:14.754+0000 7fc071239700 1 -- 172.21.15.59:0/3120226933 wait complete. 2023-11-24T07:58:14.768 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T07:58:14.768 DEBUG:teuthology.orchestra.run.smithi059:> (cd /home/ubuntu/cephtest/mnt.0 && exec sudo cp -p ./volumes/subvol_grp1023/subvol4495/.meta ./volumes/subvol_grp1023/subvol4495/.meta.stale_snap_section) 2023-11-24T07:58:16.700 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot rm cephfs subvol4495 subvol_snap1172 subvol_grp1023 2023-11-24T07:58:16.843 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:16.841+0000 7fa5898fa700 1 Processor -- start 2023-11-24T07:58:16.843 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:16.842+0000 7fa5898fa700 1 -- start start 2023-11-24T07:58:16.846 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:16.842+0000 7fa5898fa700 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fa5840d0870 0x7fa5840d26c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:16.846 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:16.842+0000 7fa5898fa700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fa5840d49f0 0x7fa5840b6d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:16.846 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:16.842+0000 7fa5898fa700 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa5840b5370 con 0x7fa5840d0c50 2023-11-24T07:58:16.846 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:16.842+0000 7fa5898fa700 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa5840d2f80 con 0x7fa5840d49f0 2023-11-24T07:58:16.846 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:16.842+0000 7fa5898fa700 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa5840cff70 con 0x7fa5840d0870 2023-11-24T07:58:16.846 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:16.842+0000 7fa5837fe700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fa5840d49f0 0x7fa5840b6d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:16.846 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:16.842+0000 7fa5837fe700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fa5840d49f0 0x7fa5840b6d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:53058/0 (socket says 172.21.15.59:53058) 2023-11-24T07:58:16.846 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:16.843+0000 7fa5837fe700 1 -- 172.21.15.59:0/1450057525 learned_addr learned my addr 172.21.15.59:0/1450057525 (peer_addr_for_me v2:172.21.15.59:0/0) 2023-11-24T07:58:16.846 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:16.843+0000 7fa582ffd700 1 --2- 172.21.15.59:0/1450057525 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fa5840d0870 0x7fa5840d26c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:16.846 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:16.843+0000 7fa5837fe700 1 -- 172.21.15.59:0/1450057525 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fa5840d0870 msgr2=0x7fa5840d26c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T07:58:16.847 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:16.843+0000 7fa5837fe700 1 --2- 172.21.15.59:0/1450057525 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fa5840d0870 0x7fa5840d26c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:16.847 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:16.843+0000 7fa5837fe700 1 -- 172.21.15.59:0/1450057525 >> v1:172.21.15.76:6789/0 conn(0x7fa5840d0c50 legacy=0x7fa5840cf860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T07:58:16.847 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:16.843+0000 7fa5837fe700 1 -- 172.21.15.59:0/1450057525 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa5840ceae0 con 0x7fa5840d49f0 2023-11-24T07:58:16.847 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:16.843+0000 7fa5837fe700 1 --2- 172.21.15.59:0/1450057525 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa5840d49f0 0x7fa5840b6d20 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7fa57c009e50 tx=0x7fa57c00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fc675d63daaefff2 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:58:16.847 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:16.843+0000 7fa581ffb700 1 -- 172.21.15.59:0/1450057525 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa57c00e070 con 0x7fa5840d49f0 2023-11-24T07:58:16.847 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:16.843+0000 7fa581ffb700 1 -- 172.21.15.59:0/1450057525 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa57c003a20 con 0x7fa5840d49f0 2023-11-24T07:58:16.847 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:16.843+0000 7fa581ffb700 1 -- 172.21.15.59:0/1450057525 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa57c003dd0 con 0x7fa5840d49f0 2023-11-24T07:58:16.847 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:16.844+0000 7fa5898fa700 1 -- 172.21.15.59:0/1450057525 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa5840d49f0 msgr2=0x7fa5840b6d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:16.847 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:16.844+0000 7fa5898fa700 1 --2- 172.21.15.59:0/1450057525 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa5840d49f0 0x7fa5840b6d20 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7fa57c009e50 tx=0x7fa57c00b330 comp rx=0 tx=0).stop 2023-11-24T07:58:16.847 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:16.844+0000 7fa5898fa700 1 -- 172.21.15.59:0/1450057525 shutdown_connections 2023-11-24T07:58:16.847 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:16.844+0000 7fa5898fa700 1 --2- 172.21.15.59:0/1450057525 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fa5840d0870 0x7fa5840d26c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:16.847 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:16.844+0000 7fa5898fa700 1 --2- 172.21.15.59:0/1450057525 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa5840d49f0 0x7fa5840b6d20 unknown :-1 s=CLOSED pgs=584 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:16.847 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:16.844+0000 7fa5898fa700 1 -- 172.21.15.59:0/1450057525 >> 172.21.15.59:0/1450057525 conn(0x7fa5840d3ed0 msgr2=0x7fa5840d42e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:58:16.848 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:16.844+0000 7fa5898fa700 1 -- 172.21.15.59:0/1450057525 shutdown_connections 2023-11-24T07:58:16.848 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:16.844+0000 7fa5898fa700 1 -- 172.21.15.59:0/1450057525 wait complete. 2023-11-24T07:58:16.850 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:16.845+0000 7fa5898fa700 1 Processor -- start 2023-11-24T07:58:16.850 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:16.845+0000 7fa5898fa700 1 -- start start 2023-11-24T07:58:16.850 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:16.846+0000 7fa5898fa700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa5840d0870 0x7fa5840b32a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:16.850 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:16.846+0000 7fa5898fa700 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fa5840d0c50 0x7fa58414cd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:16.850 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:16.846+0000 7fa5898fa700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa5840d49f0 0x7fa58414b150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:16.850 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:16.846+0000 7fa5898fa700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa58414c8d0 con 0x7fa5840d0870 2023-11-24T07:58:16.850 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:16.846+0000 7fa5898fa700 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa58414dfe0 con 0x7fa5840d49f0 2023-11-24T07:58:16.850 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:16.846+0000 7fa5898fa700 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa58414bcc0 con 0x7fa5840d0c50 2023-11-24T07:58:16.850 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:16.846+0000 7fa582ffd700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa5840d0870 0x7fa5840b32a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:16.850 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:16.846+0000 7fa582ffd700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa5840d0870 0x7fa5840b32a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:53064/0 (socket says 172.21.15.59:53064) 2023-11-24T07:58:16.850 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:16.846+0000 7fa582ffd700 1 -- 172.21.15.59:0/192672653 learned_addr learned my addr 172.21.15.59:0/192672653 (peer_addr_for_me v2:172.21.15.59:0/0) 2023-11-24T07:58:16.851 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:16.846+0000 7fa5827fc700 1 --2- 172.21.15.59:0/192672653 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fa5840d0c50 0x7fa58414cd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:16.851 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:16.846+0000 7fa5837fe700 1 --2- 172.21.15.59:0/192672653 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa5840d49f0 0x7fa58414b150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:16.851 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:16.846+0000 7fa582ffd700 1 -- 172.21.15.59:0/192672653 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fa5840d0c50 msgr2=0x7fa58414cd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:16.851 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:16.846+0000 7fa582ffd700 1 --2- 172.21.15.59:0/192672653 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fa5840d0c50 0x7fa58414cd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:16.851 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:16.846+0000 7fa582ffd700 1 -- 172.21.15.59:0/192672653 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa5840d49f0 msgr2=0x7fa58414b150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:16.851 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:16.846+0000 7fa582ffd700 1 --2- 172.21.15.59:0/192672653 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa5840d49f0 0x7fa58414b150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:16.851 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:16.846+0000 7fa582ffd700 1 -- 172.21.15.59:0/192672653 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa57c009b30 con 0x7fa5840d0870 2023-11-24T07:58:16.851 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:16.846+0000 7fa582ffd700 1 --2- 172.21.15.59:0/192672653 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa5840d0870 0x7fa5840b32a0 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7fa57800c7f0 tx=0x7fa57800cb00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:58:16.851 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:16.847+0000 7fa56bfff700 1 -- 172.21.15.59:0/192672653 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa578004220 con 0x7fa5840d0870 2023-11-24T07:58:16.851 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:16.847+0000 7fa56bfff700 1 -- 172.21.15.59:0/192672653 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa578004380 con 0x7fa5840d0870 2023-11-24T07:58:16.851 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:16.847+0000 7fa5898fa700 1 -- 172.21.15.59:0/192672653 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa58414bfa0 con 0x7fa5840d0870 2023-11-24T07:58:16.851 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:16.847+0000 7fa56bfff700 1 -- 172.21.15.59:0/192672653 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5780036a0 con 0x7fa5840d0870 2023-11-24T07:58:16.852 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:16.847+0000 7fa5898fa700 1 -- 172.21.15.59:0/192672653 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa58414c560 con 0x7fa5840d0870 2023-11-24T07:58:16.852 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:16.848+0000 7fa5898fa700 1 -- 172.21.15.59:0/192672653 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa58414cbb0 con 0x7fa5840d0870 2023-11-24T07:58:16.852 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:16.848+0000 7fa56bfff700 1 -- 172.21.15.59:0/192672653 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 7) v1 ==== 89333+0+0 (secure 0 0 0) 0x7fa578003800 con 0x7fa5840d0870 2023-11-24T07:58:16.852 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:16.849+0000 7fa56bfff700 1 --2- 172.21.15.59:0/192672653 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7fa57006aff0 0x7fa57006d4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:16.852 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:16.849+0000 7fa56bfff700 1 -- 172.21.15.59:0/192672653 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5854+0+0 (secure 0 0 0) 0x7fa5780893d0 con 0x7fa5840d0870 2023-11-24T07:58:16.852 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:16.849+0000 7fa5827fc700 1 --2- 172.21.15.59:0/192672653 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7fa57006aff0 0x7fa57006d4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:16.853 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:16.849+0000 7fa5827fc700 1 --2- 172.21.15.59:0/192672653 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7fa57006aff0 0x7fa57006d4b0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7fa584147bc0 tx=0x7fa574009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:58:16.855 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:16.854+0000 7fa56bfff700 1 -- 172.21.15.59:0/192672653 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fa5780541d0 con 0x7fa5840d0870 2023-11-24T07:58:17.095 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:17.094+0000 7fa5898fa700 1 -- 172.21.15.59:0/192672653 --> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot rm", "vol_name": "cephfs", "sub_name": "subvol4495", "snap_name": "subvol_snap1172", "group_name": "subvol_grp1023", "target": ["mon-mgr", ""]}) v1 -- 0x7fa5840cf060 con 0x7fa57006aff0 2023-11-24T07:58:17.128 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:17.127+0000 7fa56bfff700 1 -- 172.21.15.59:0/192672653 <== mgr.4100 v2:172.21.15.76:6832/4264046536 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7fa574001d50 con 0x7fa57006aff0 2023-11-24T07:58:17.131 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:17.129+0000 7fa5898fa700 1 -- 172.21.15.59:0/192672653 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7fa57006aff0 msgr2=0x7fa57006d4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:17.131 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:17.129+0000 7fa5898fa700 1 --2- 172.21.15.59:0/192672653 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7fa57006aff0 0x7fa57006d4b0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7fa584147bc0 tx=0x7fa574009040 comp rx=0 tx=0).stop 2023-11-24T07:58:17.131 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:17.129+0000 7fa5898fa700 1 -- 172.21.15.59:0/192672653 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa5840d0870 msgr2=0x7fa5840b32a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:17.131 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:17.129+0000 7fa5898fa700 1 --2- 172.21.15.59:0/192672653 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa5840d0870 0x7fa5840b32a0 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7fa57800c7f0 tx=0x7fa57800cb00 comp rx=0 tx=0).stop 2023-11-24T07:58:17.131 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:17.129+0000 7fa5898fa700 1 -- 172.21.15.59:0/192672653 shutdown_connections 2023-11-24T07:58:17.132 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:17.129+0000 7fa5898fa700 1 --2- 172.21.15.59:0/192672653 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7fa57006aff0 0x7fa57006d4b0 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:17.132 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:17.129+0000 7fa5898fa700 1 --2- 172.21.15.59:0/192672653 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa5840d49f0 0x7fa58414b150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:17.132 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:17.129+0000 7fa5898fa700 1 --2- 172.21.15.59:0/192672653 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa5840d0870 0x7fa5840b32a0 unknown :-1 s=CLOSED pgs=585 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:17.132 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:17.129+0000 7fa5898fa700 1 --2- 172.21.15.59:0/192672653 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fa5840d0c50 0x7fa58414cd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:17.132 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:17.129+0000 7fa5898fa700 1 -- 172.21.15.59:0/192672653 >> 172.21.15.59:0/192672653 conn(0x7fa5840d3ed0 msgr2=0x7fa5840c75e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:58:17.132 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:17.129+0000 7fa5898fa700 1 -- 172.21.15.59:0/192672653 shutdown_connections 2023-11-24T07:58:17.132 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:17.129+0000 7fa5898fa700 1 -- 172.21.15.59:0/192672653 wait complete. 2023-11-24T07:58:17.143 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T07:58:17.143 DEBUG:teuthology.orchestra.run.smithi059:> (cd /home/ubuntu/cephtest/mnt.0 && exec sudo cp -p ./volumes/subvol_grp1023/subvol4495/.meta.stale_snap_section ./volumes/subvol_grp1023/subvol4495/.meta) 2023-11-24T07:58:17.181 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T07:58:17.181 DEBUG:teuthology.orchestra.run.smithi059:> (cd /home/ubuntu/cephtest/mnt.0 && exec sudo grep SNAP_METADATA_subvol_snap1172 ./volumes/subvol_grp1023/subvol4495/.meta) 2023-11-24T07:58:17.247 INFO:teuthology.orchestra.run.smithi059.stdout:[SNAP_METADATA_subvol_snap1172] 2023-11-24T07:58:17.249 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume info cephfs subvol4495 subvol_grp1023 2023-11-24T07:58:17.418 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:17.416+0000 7f429b7b7700 1 Processor -- start 2023-11-24T07:58:17.418 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:17.416+0000 7f429b7b7700 1 -- start start 2023-11-24T07:58:17.420 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:17.417+0000 7f429b7b7700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f429406afd0 0x7f42940b59d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:17.420 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:17.417+0000 7f429b7b7700 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f429406b3b0 0x7f429415e470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:17.420 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:17.417+0000 7f429b7b7700 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f4294072350 0x7f42940727a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:17.420 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:17.417+0000 7f429b7b7700 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4294071ec0 con 0x7f429406afd0 2023-11-24T07:58:17.420 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:17.417+0000 7f429b7b7700 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4294071b90 con 0x7f4294072350 2023-11-24T07:58:17.421 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:17.417+0000 7f429b7b7700 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f429415ed30 con 0x7f429406b3b0 2023-11-24T07:58:17.421 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:17.417+0000 7f4299553700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f429406afd0 0x7f42940b59d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:17.421 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:17.417+0000 7f4299553700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f429406afd0 0x7f42940b59d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:53068/0 (socket says 172.21.15.59:53068) 2023-11-24T07:58:17.421 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:17.417+0000 7f4299553700 1 -- 172.21.15.59:0/2910986765 learned_addr learned my addr 172.21.15.59:0/2910986765 (peer_addr_for_me v2:172.21.15.59:0/0) 2023-11-24T07:58:17.421 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:17.417+0000 7f4299d54700 1 --2- 172.21.15.59:0/2910986765 >> v2:172.21.15.76:3300/0 conn(0x7f4294072350 0x7f42940727a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:17.421 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:17.417+0000 7f4298d52700 1 --2- 172.21.15.59:0/2910986765 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f429406b3b0 0x7f429415e470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:17.421 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:17.417+0000 7f4299553700 1 -- 172.21.15.59:0/2910986765 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f429406b3b0 msgr2=0x7f429415e470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T07:58:17.421 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:17.417+0000 7f4299553700 1 --2- 172.21.15.59:0/2910986765 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f429406b3b0 0x7f429415e470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:17.421 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:17.417+0000 7f4299553700 1 -- 172.21.15.59:0/2910986765 >> v2:172.21.15.76:3300/0 conn(0x7f4294072350 msgr2=0x7f42940727a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T07:58:17.422 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:17.417+0000 7f4299553700 1 --2- 172.21.15.59:0/2910986765 >> v2:172.21.15.76:3300/0 conn(0x7f4294072350 0x7f42940727a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:17.422 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:17.417+0000 7f4299553700 1 -- 172.21.15.59:0/2910986765 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f429415f010 con 0x7f429406afd0 2023-11-24T07:58:17.422 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:17.417+0000 7f4299d54700 1 --2- 172.21.15.59:0/2910986765 >> v2:172.21.15.76:3300/0 conn(0x7f4294072350 0x7f42940727a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-24T07:58:17.422 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:17.417+0000 7f4299553700 1 --2- 172.21.15.59:0/2910986765 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f429406afd0 0x7f42940b59d0 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7f4294162b00 tx=0x7f4284009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aeca951730dc9e21 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:58:17.422 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:17.418+0000 7f428bfff700 1 -- 172.21.15.59:0/2910986765 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f428400b870 con 0x7f429406afd0 2023-11-24T07:58:17.422 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:17.418+0000 7f428bfff700 1 -- 172.21.15.59:0/2910986765 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f428400b9d0 con 0x7f429406afd0 2023-11-24T07:58:17.422 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:17.418+0000 7f428bfff700 1 -- 172.21.15.59:0/2910986765 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f428400bd60 con 0x7f429406afd0 2023-11-24T07:58:17.423 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:17.418+0000 7f429b7b7700 1 -- 172.21.15.59:0/2910986765 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f429406afd0 msgr2=0x7f42940b59d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:17.423 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:17.418+0000 7f429b7b7700 1 --2- 172.21.15.59:0/2910986765 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f429406afd0 0x7f42940b59d0 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7f4294162b00 tx=0x7f4284009bd0 comp rx=0 tx=0).stop 2023-11-24T07:58:17.423 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:17.418+0000 7f429b7b7700 1 -- 172.21.15.59:0/2910986765 shutdown_connections 2023-11-24T07:58:17.423 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:17.418+0000 7f429b7b7700 1 --2- 172.21.15.59:0/2910986765 >> v2:172.21.15.76:3300/0 conn(0x7f4294072350 0x7f42940727a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:17.423 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:17.418+0000 7f429b7b7700 1 --2- 172.21.15.59:0/2910986765 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f429406afd0 0x7f42940b59d0 unknown :-1 s=CLOSED pgs=586 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:17.423 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:17.418+0000 7f429b7b7700 1 --2- 172.21.15.59:0/2910986765 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f429406b3b0 0x7f429415e470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:17.424 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:17.418+0000 7f429b7b7700 1 -- 172.21.15.59:0/2910986765 >> 172.21.15.59:0/2910986765 conn(0x7f42940b2760 msgr2=0x7f42940b2b70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:58:17.424 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:17.418+0000 7f429b7b7700 1 -- 172.21.15.59:0/2910986765 shutdown_connections 2023-11-24T07:58:17.424 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:17.418+0000 7f429b7b7700 1 -- 172.21.15.59:0/2910986765 wait complete. 2023-11-24T07:58:17.427 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:17.420+0000 7f429b7b7700 1 Processor -- start 2023-11-24T07:58:17.427 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:17.420+0000 7f429b7b7700 1 -- start start 2023-11-24T07:58:17.427 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:17.421+0000 7f429b7b7700 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f429406afd0 0x7f42940c86a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:17.427 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:17.421+0000 7f429b7b7700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f429406b3b0 0x7f42940cf930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:17.427 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:17.421+0000 7f429b7b7700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4294072350 0x7f42940cfe70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:17.427 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:17.421+0000 7f429b7b7700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4294133b80 con 0x7f4294072350 2023-11-24T07:58:17.427 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:17.421+0000 7f429b7b7700 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4294136170 con 0x7f429406b3b0 2023-11-24T07:58:17.427 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:17.421+0000 7f429b7b7700 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4294133530 con 0x7f429406afd0 2023-11-24T07:58:17.427 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:17.421+0000 7f4299d54700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4294072350 0x7f42940cfe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:17.427 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:17.421+0000 7f4299d54700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4294072350 0x7f42940cfe70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:53078/0 (socket says 172.21.15.59:53078) 2023-11-24T07:58:17.427 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:17.421+0000 7f4299d54700 1 -- 172.21.15.59:0/298978349 learned_addr learned my addr 172.21.15.59:0/298978349 (peer_addr_for_me v2:172.21.15.59:0/0) 2023-11-24T07:58:17.428 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:17.421+0000 7f4298d52700 1 --2- 172.21.15.59:0/298978349 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f429406b3b0 0x7f42940cf930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:17.428 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:17.421+0000 7f4299553700 1 --2- 172.21.15.59:0/298978349 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f429406afd0 0x7f42940c86a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:17.428 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:17.421+0000 7f4299d54700 1 -- 172.21.15.59:0/298978349 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f429406afd0 msgr2=0x7f42940c86a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:17.428 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:17.421+0000 7f4299d54700 1 --2- 172.21.15.59:0/298978349 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f429406afd0 0x7f42940c86a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:17.428 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:17.421+0000 7f4299d54700 1 -- 172.21.15.59:0/298978349 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f429406b3b0 msgr2=0x7f42940cf930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:17.428 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:17.421+0000 7f4299d54700 1 --2- 172.21.15.59:0/298978349 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f429406b3b0 0x7f42940cf930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:17.428 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:17.421+0000 7f4299d54700 1 -- 172.21.15.59:0/298978349 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4284009550 con 0x7f4294072350 2023-11-24T07:58:17.428 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:17.421+0000 7f4298d52700 1 --2- 172.21.15.59:0/298978349 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f429406b3b0 0x7f42940cf930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-24T07:58:17.428 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:17.421+0000 7f4299d54700 1 --2- 172.21.15.59:0/298978349 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4294072350 0x7f42940cfe70 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7f428c007e70 tx=0x7f428c00d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:58:17.428 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:17.422+0000 7f4289ffb700 1 -- 172.21.15.59:0/298978349 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f428c00de40 con 0x7f4294072350 2023-11-24T07:58:17.428 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:17.422+0000 7f4289ffb700 1 -- 172.21.15.59:0/298978349 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f428c00f040 con 0x7f4294072350 2023-11-24T07:58:17.428 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:17.422+0000 7f429b7b7700 1 -- 172.21.15.59:0/298978349 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4294133810 con 0x7f4294072350 2023-11-24T07:58:17.429 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:17.422+0000 7f4289ffb700 1 -- 172.21.15.59:0/298978349 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f428c003c60 con 0x7f4294072350 2023-11-24T07:58:17.429 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:17.422+0000 7f429b7b7700 1 -- 172.21.15.59:0/298978349 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f42940c8dc0 con 0x7f4294072350 2023-11-24T07:58:17.429 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:17.423+0000 7f429b7b7700 1 -- 172.21.15.59:0/298978349 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4294068670 con 0x7f4294072350 2023-11-24T07:58:17.429 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:17.423+0000 7f4289ffb700 1 -- 172.21.15.59:0/298978349 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 7) v1 ==== 89333+0+0 (secure 0 0 0) 0x7f428c004030 con 0x7f4294072350 2023-11-24T07:58:17.429 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:17.424+0000 7f4289ffb700 1 --2- 172.21.15.59:0/298978349 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f427806aff0 0x7f427806d4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:17.429 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:17.424+0000 7f4289ffb700 1 -- 172.21.15.59:0/298978349 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5854+0+0 (secure 0 0 0) 0x7f428c089730 con 0x7f4294072350 2023-11-24T07:58:17.429 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:17.424+0000 7f4299553700 1 --2- 172.21.15.59:0/298978349 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f427806aff0 0x7f427806d4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:17.429 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:17.424+0000 7f4299553700 1 --2- 172.21.15.59:0/298978349 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f427806aff0 0x7f427806d4b0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f4284000c00 tx=0x7f4284008030 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:58:17.429 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:17.428+0000 7f4289ffb700 1 -- 172.21.15.59:0/298978349 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f428c0545c0 con 0x7f4294072350 2023-11-24T07:58:17.666 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:17.665+0000 7f429b7b7700 1 -- 172.21.15.59:0/298978349 --> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] -- mgr_command(tid 0: {"prefix": "fs subvolume info", "vol_name": "cephfs", "sub_name": "subvol4495", "group_name": "subvol_grp1023", "target": ["mon-mgr", ""]}) v1 -- 0x7f42941333b0 con 0x7f427806aff0 2023-11-24T07:58:17.705 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:17.703+0000 7f4289ffb700 1 -- 172.21.15.59:0/298978349 <== mgr.4100 v2:172.21.15.76:6832/4264046536 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+702 (secure 0 0 0) 0x7f428400bb40 con 0x7f427806aff0 2023-11-24T07:58:17.705 INFO:teuthology.orchestra.run.smithi059.stdout:{ 2023-11-24T07:58:17.705 INFO:teuthology.orchestra.run.smithi059.stdout: "atime": "2023-11-24 07:58:13", 2023-11-24T07:58:17.705 INFO:teuthology.orchestra.run.smithi059.stdout: "bytes_pcent": "undefined", 2023-11-24T07:58:17.705 INFO:teuthology.orchestra.run.smithi059.stdout: "bytes_quota": "infinite", 2023-11-24T07:58:17.705 INFO:teuthology.orchestra.run.smithi059.stdout: "bytes_used": 0, 2023-11-24T07:58:17.706 INFO:teuthology.orchestra.run.smithi059.stdout: "created_at": "2023-11-24 07:58:13", 2023-11-24T07:58:17.706 INFO:teuthology.orchestra.run.smithi059.stdout: "ctime": "2023-11-24 07:58:13", 2023-11-24T07:58:17.706 INFO:teuthology.orchestra.run.smithi059.stdout: "data_pool": "cephfs_data", 2023-11-24T07:58:17.706 INFO:teuthology.orchestra.run.smithi059.stdout: "features": [ 2023-11-24T07:58:17.706 INFO:teuthology.orchestra.run.smithi059.stdout: "snapshot-clone", 2023-11-24T07:58:17.706 INFO:teuthology.orchestra.run.smithi059.stdout: "snapshot-autoprotect", 2023-11-24T07:58:17.706 INFO:teuthology.orchestra.run.smithi059.stdout: "snapshot-retention" 2023-11-24T07:58:17.706 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2023-11-24T07:58:17.706 INFO:teuthology.orchestra.run.smithi059.stdout: "gid": 0, 2023-11-24T07:58:17.706 INFO:teuthology.orchestra.run.smithi059.stdout: "mode": 16877, 2023-11-24T07:58:17.706 INFO:teuthology.orchestra.run.smithi059.stdout: "mon_addrs": [ 2023-11-24T07:58:17.706 INFO:teuthology.orchestra.run.smithi059.stdout: "172.21.15.59:6789", 2023-11-24T07:58:17.706 INFO:teuthology.orchestra.run.smithi059.stdout: "172.21.15.76:6789", 2023-11-24T07:58:17.707 INFO:teuthology.orchestra.run.smithi059.stdout: "172.21.15.76:6790" 2023-11-24T07:58:17.707 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2023-11-24T07:58:17.707 INFO:teuthology.orchestra.run.smithi059.stdout: "mtime": "2023-11-24 07:58:13", 2023-11-24T07:58:17.707 INFO:teuthology.orchestra.run.smithi059.stdout: "path": "/volumes/subvol_grp1023/subvol4495/18a3fe9f-79ea-46c7-8ca6-25952841b8af", 2023-11-24T07:58:17.707 INFO:teuthology.orchestra.run.smithi059.stdout: "pool_namespace": "", 2023-11-24T07:58:17.707 INFO:teuthology.orchestra.run.smithi059.stdout: "state": "complete", 2023-11-24T07:58:17.707 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "subvolume", 2023-11-24T07:58:17.707 INFO:teuthology.orchestra.run.smithi059.stdout: "uid": 0 2023-11-24T07:58:17.707 INFO:teuthology.orchestra.run.smithi059.stdout:} 2023-11-24T07:58:17.708 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:17.705+0000 7f429b7b7700 1 -- 172.21.15.59:0/298978349 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f427806aff0 msgr2=0x7f427806d4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:17.708 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:17.705+0000 7f429b7b7700 1 --2- 172.21.15.59:0/298978349 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f427806aff0 0x7f427806d4b0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f4284000c00 tx=0x7f4284008030 comp rx=0 tx=0).stop 2023-11-24T07:58:17.708 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:17.705+0000 7f429b7b7700 1 -- 172.21.15.59:0/298978349 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4294072350 msgr2=0x7f42940cfe70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:17.708 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:17.706+0000 7f429b7b7700 1 --2- 172.21.15.59:0/298978349 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4294072350 0x7f42940cfe70 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7f428c007e70 tx=0x7f428c00d3b0 comp rx=0 tx=0).stop 2023-11-24T07:58:17.709 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:17.706+0000 7f429b7b7700 1 -- 172.21.15.59:0/298978349 shutdown_connections 2023-11-24T07:58:17.709 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:17.706+0000 7f429b7b7700 1 --2- 172.21.15.59:0/298978349 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f427806aff0 0x7f427806d4b0 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:17.709 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:17.706+0000 7f429b7b7700 1 --2- 172.21.15.59:0/298978349 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4294072350 0x7f42940cfe70 unknown :-1 s=CLOSED pgs=587 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:17.709 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:17.706+0000 7f429b7b7700 1 --2- 172.21.15.59:0/298978349 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f429406afd0 0x7f42940c86a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:17.709 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:17.706+0000 7f429b7b7700 1 --2- 172.21.15.59:0/298978349 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f429406b3b0 0x7f42940cf930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:17.709 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:17.706+0000 7f429b7b7700 1 -- 172.21.15.59:0/298978349 >> 172.21.15.59:0/298978349 conn(0x7f42940b2760 msgr2=0x7f42940b06b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:58:17.709 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:17.706+0000 7f429b7b7700 1 -- 172.21.15.59:0/298978349 shutdown_connections 2023-11-24T07:58:17.709 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:17.706+0000 7f429b7b7700 1 -- 172.21.15.59:0/298978349 wait complete. 2023-11-24T07:58:17.722 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T07:58:17.722 DEBUG:teuthology.orchestra.run.smithi059:> (cd /home/ubuntu/cephtest/mnt.0 && exec sudo grep SNAP_METADATA_subvol_snap1172 ./volumes/subvol_grp1023/subvol4495/.meta) 2023-11-24T07:58:17.753 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-11-24T07:58:17.754 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume rm cephfs subvol4495 subvol_grp1023 2023-11-24T07:58:17.932 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:17.921+0000 7f3237164700 1 Processor -- start 2023-11-24T07:58:17.932 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:17.921+0000 7f3237164700 1 -- start start 2023-11-24T07:58:17.933 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:17.922+0000 7f3237164700 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f32301447a0 0x7f3230071b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:17.933 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:17.922+0000 7f3237164700 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f3230072ab0 0x7f32300720d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:17.933 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:17.922+0000 7f3237164700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f323015b130 0x7f323015e750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:17.933 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:17.922+0000 7f3237164700 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f323006dbd0 con 0x7f323015b130 2023-11-24T07:58:17.933 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:17.922+0000 7f3237164700 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f323015b700 con 0x7f32301447a0 2023-11-24T07:58:17.933 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:17.922+0000 7f3237164700 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f323015b9e0 con 0x7f3230072ab0 2023-11-24T07:58:17.934 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:17.922+0000 7f3235701700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f323015b130 0x7f323015e750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:17.934 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:17.922+0000 7f3235701700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f323015b130 0x7f323015e750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:53090/0 (socket says 172.21.15.59:53090) 2023-11-24T07:58:17.934 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:17.922+0000 7f3235701700 1 -- 172.21.15.59:0/1848437644 learned_addr learned my addr 172.21.15.59:0/1848437644 (peer_addr_for_me v2:172.21.15.59:0/0) 2023-11-24T07:58:17.934 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:17.922+0000 7f322ffff700 1 --2- 172.21.15.59:0/1848437644 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f3230072ab0 0x7f32300720d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:17.934 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:17.922+0000 7f3234f00700 1 --2- 172.21.15.59:0/1848437644 >> v2:172.21.15.76:3300/0 conn(0x7f32301447a0 0x7f3230071b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:17.934 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:17.923+0000 7f3235701700 1 -- 172.21.15.59:0/1848437644 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f3230072ab0 msgr2=0x7f32300720d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T07:58:17.935 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:17.923+0000 7f3235701700 1 --2- 172.21.15.59:0/1848437644 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f3230072ab0 0x7f32300720d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:17.935 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:17.923+0000 7f3235701700 1 -- 172.21.15.59:0/1848437644 >> v2:172.21.15.76:3300/0 conn(0x7f32301447a0 msgr2=0x7f3230071b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T07:58:17.935 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:17.923+0000 7f3235701700 1 --2- 172.21.15.59:0/1848437644 >> v2:172.21.15.76:3300/0 conn(0x7f32301447a0 0x7f3230071b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:17.935 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:17.923+0000 7f3235701700 1 -- 172.21.15.59:0/1848437644 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f323015bcc0 con 0x7f323015b130 2023-11-24T07:58:17.935 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:17.923+0000 7f3235701700 1 --2- 172.21.15.59:0/1848437644 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f323015b130 0x7f323015e750 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7f32300b05a0 tx=0x7f322800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bc6b5acee682fb4c server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:58:17.935 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:17.923+0000 7f322f7fe700 1 -- 172.21.15.59:0/1848437644 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f322800e070 con 0x7f323015b130 2023-11-24T07:58:17.935 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:17.923+0000 7f322f7fe700 1 -- 172.21.15.59:0/1848437644 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f322800b910 con 0x7f323015b130 2023-11-24T07:58:17.936 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:17.923+0000 7f322f7fe700 1 -- 172.21.15.59:0/1848437644 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f322800bcc0 con 0x7f323015b130 2023-11-24T07:58:17.936 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:17.924+0000 7f3237164700 1 -- 172.21.15.59:0/1848437644 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f323015b130 msgr2=0x7f323015e750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:17.936 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:17.924+0000 7f3237164700 1 --2- 172.21.15.59:0/1848437644 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f323015b130 0x7f323015e750 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7f32300b05a0 tx=0x7f322800b330 comp rx=0 tx=0).stop 2023-11-24T07:58:17.936 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:17.924+0000 7f3237164700 1 -- 172.21.15.59:0/1848437644 shutdown_connections 2023-11-24T07:58:17.936 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:17.924+0000 7f3237164700 1 --2- 172.21.15.59:0/1848437644 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f323015b130 0x7f323015e750 unknown :-1 s=CLOSED pgs=588 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:17.936 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:17.924+0000 7f3237164700 1 --2- 172.21.15.59:0/1848437644 >> v2:172.21.15.76:3300/0 conn(0x7f32301447a0 0x7f3230071b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:17.936 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:17.924+0000 7f3237164700 1 --2- 172.21.15.59:0/1848437644 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f3230072ab0 0x7f32300720d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:17.936 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:17.924+0000 7f3237164700 1 -- 172.21.15.59:0/1848437644 >> 172.21.15.59:0/1848437644 conn(0x7f32300b2760 msgr2=0x7f32300b2b70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:58:17.937 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:17.924+0000 7f3237164700 1 -- 172.21.15.59:0/1848437644 shutdown_connections 2023-11-24T07:58:17.937 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:17.924+0000 7f3237164700 1 -- 172.21.15.59:0/1848437644 wait complete. 2023-11-24T07:58:17.937 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:17.925+0000 7f3237164700 1 Processor -- start 2023-11-24T07:58:17.937 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:17.925+0000 7f3237164700 1 -- start start 2023-11-24T07:58:17.937 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:17.926+0000 7f3237164700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3230072ab0 0x7f3230142de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:17.937 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:17.926+0000 7f3237164700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f32301447a0 0x7f32300cfb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:17.937 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:17.926+0000 7f3237164700 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f323015b130 0x7f32300af850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:17.938 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:17.926+0000 7f3237164700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32300ae480 con 0x7f3230072ab0 2023-11-24T07:58:17.938 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:17.926+0000 7f3237164700 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32300ae230 con 0x7f32301447a0 2023-11-24T07:58:17.938 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:17.926+0000 7f3237164700 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f32300ad900 con 0x7f323015b130 2023-11-24T07:58:17.938 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:17.926+0000 7f3234f00700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3230072ab0 0x7f3230142de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:17.938 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:17.926+0000 7f3234f00700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3230072ab0 0x7f3230142de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:53092/0 (socket says 172.21.15.59:53092) 2023-11-24T07:58:17.938 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:17.926+0000 7f3234f00700 1 -- 172.21.15.59:0/281385156 learned_addr learned my addr 172.21.15.59:0/281385156 (peer_addr_for_me v2:172.21.15.59:0/0) 2023-11-24T07:58:17.938 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:17.926+0000 7f322ffff700 1 --2- 172.21.15.59:0/281385156 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f32301447a0 0x7f32300cfb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:17.939 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:17.926+0000 7f3235701700 1 --2- 172.21.15.59:0/281385156 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f323015b130 0x7f32300af850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:17.939 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:17.926+0000 7f3234f00700 1 -- 172.21.15.59:0/281385156 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f323015b130 msgr2=0x7f32300af850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:17.939 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:17.926+0000 7f3234f00700 1 --2- 172.21.15.59:0/281385156 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f323015b130 0x7f32300af850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:17.939 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:17.926+0000 7f3234f00700 1 -- 172.21.15.59:0/281385156 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f32301447a0 msgr2=0x7f32300cfb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:17.939 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:17.926+0000 7f3234f00700 1 --2- 172.21.15.59:0/281385156 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f32301447a0 0x7f32300cfb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:17.939 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:17.926+0000 7f3234f00700 1 -- 172.21.15.59:0/281385156 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3228009b30 con 0x7f3230072ab0 2023-11-24T07:58:17.939 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:17.926+0000 7f3234f00700 1 --2- 172.21.15.59:0/281385156 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3230072ab0 0x7f3230142de0 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7f322400e9b0 tx=0x7f322400ecc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:58:17.939 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:17.926+0000 7f322d7fa700 1 -- 172.21.15.59:0/281385156 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3224009710 con 0x7f3230072ab0 2023-11-24T07:58:17.940 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:17.926+0000 7f322d7fa700 1 -- 172.21.15.59:0/281385156 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f322400c850 con 0x7f3230072ab0 2023-11-24T07:58:17.940 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:17.926+0000 7f3237164700 1 -- 172.21.15.59:0/281385156 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f32300adbe0 con 0x7f3230072ab0 2023-11-24T07:58:17.940 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:17.927+0000 7f322d7fa700 1 -- 172.21.15.59:0/281385156 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f322400cbc0 con 0x7f3230072ab0 2023-11-24T07:58:17.940 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:17.927+0000 7f3237164700 1 -- 172.21.15.59:0/281385156 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3230143320 con 0x7f3230072ab0 2023-11-24T07:58:17.940 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:17.927+0000 7f3237164700 1 -- 172.21.15.59:0/281385156 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3230138180 con 0x7f3230072ab0 2023-11-24T07:58:17.940 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:17.928+0000 7f322d7fa700 1 -- 172.21.15.59:0/281385156 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 7) v1 ==== 89333+0+0 (secure 0 0 0) 0x7f322400cd20 con 0x7f3230072ab0 2023-11-24T07:58:17.940 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:17.928+0000 7f322d7fa700 1 --2- 172.21.15.59:0/281385156 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f321406aff0 0x7f321406d4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:17.941 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:17.928+0000 7f322d7fa700 1 -- 172.21.15.59:0/281385156 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5854+0+0 (secure 0 0 0) 0x7f3224089860 con 0x7f3230072ab0 2023-11-24T07:58:17.941 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:17.929+0000 7f322ffff700 1 --2- 172.21.15.59:0/281385156 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f321406aff0 0x7f321406d4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:17.941 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:17.929+0000 7f322ffff700 1 --2- 172.21.15.59:0/281385156 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f321406aff0 0x7f321406d4b0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f32301f85d0 tx=0x7f321c009210 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:58:17.942 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:17.933+0000 7f322d7fa700 1 -- 172.21.15.59:0/281385156 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f3224054770 con 0x7f3230072ab0 2023-11-24T07:58:18.174 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:18.172+0000 7f3237164700 1 -- 172.21.15.59:0/281385156 --> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] -- mgr_command(tid 0: {"prefix": "fs subvolume rm", "vol_name": "cephfs", "sub_name": "subvol4495", "group_name": "subvol_grp1023", "target": ["mon-mgr", ""]}) v1 -- 0x7f323015ec90 con 0x7f321406aff0 2023-11-24T07:58:18.192 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:18.191+0000 7f322d7fa700 1 -- 172.21.15.59:0/281385156 <== mgr.4100 v2:172.21.15.76:6832/4264046536 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7f321c004030 con 0x7f321406aff0 2023-11-24T07:58:18.195 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:18.193+0000 7f3237164700 1 -- 172.21.15.59:0/281385156 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f321406aff0 msgr2=0x7f321406d4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:18.195 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:18.193+0000 7f3237164700 1 --2- 172.21.15.59:0/281385156 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f321406aff0 0x7f321406d4b0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f32301f85d0 tx=0x7f321c009210 comp rx=0 tx=0).stop 2023-11-24T07:58:18.195 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:18.193+0000 7f3237164700 1 -- 172.21.15.59:0/281385156 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3230072ab0 msgr2=0x7f3230142de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:18.195 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:18.193+0000 7f3237164700 1 --2- 172.21.15.59:0/281385156 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3230072ab0 0x7f3230142de0 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7f322400e9b0 tx=0x7f322400ecc0 comp rx=0 tx=0).stop 2023-11-24T07:58:18.195 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:18.193+0000 7f3237164700 1 -- 172.21.15.59:0/281385156 shutdown_connections 2023-11-24T07:58:18.195 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:18.193+0000 7f3237164700 1 --2- 172.21.15.59:0/281385156 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f321406aff0 0x7f321406d4b0 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:18.196 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:18.193+0000 7f3237164700 1 --2- 172.21.15.59:0/281385156 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f323015b130 0x7f32300af850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:18.196 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:18.193+0000 7f3237164700 1 --2- 172.21.15.59:0/281385156 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3230072ab0 0x7f3230142de0 unknown :-1 s=CLOSED pgs=589 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:18.196 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:18.193+0000 7f3237164700 1 --2- 172.21.15.59:0/281385156 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f32301447a0 0x7f32300cfb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:18.196 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:18.193+0000 7f3237164700 1 -- 172.21.15.59:0/281385156 >> 172.21.15.59:0/281385156 conn(0x7f32300b2760 msgr2=0x7f3230071b90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:58:18.196 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:18.193+0000 7f3237164700 1 -- 172.21.15.59:0/281385156 shutdown_connections 2023-11-24T07:58:18.196 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:18.193+0000 7f3237164700 1 -- 172.21.15.59:0/281385156 wait complete. 2023-11-24T07:58:18.196 INFO:tasks.ceph.mgr.x.smithi076.stderr:2023-11-24T07:58:18.194+0000 7ff19972a700 -1 client.0 error registering admin socket command: (17) File exists 2023-11-24T07:58:18.196 INFO:tasks.ceph.mgr.x.smithi076.stderr:2023-11-24T07:58:18.194+0000 7ff19972a700 -1 client.0 error registering admin socket command: (17) File exists 2023-11-24T07:58:18.196 INFO:tasks.ceph.mgr.x.smithi076.stderr:2023-11-24T07:58:18.194+0000 7ff19972a700 -1 client.0 error registering admin socket command: (17) File exists 2023-11-24T07:58:18.196 INFO:tasks.ceph.mgr.x.smithi076.stderr:2023-11-24T07:58:18.194+0000 7ff19972a700 -1 client.0 error registering admin socket command: (17) File exists 2023-11-24T07:58:18.197 INFO:tasks.ceph.mgr.x.smithi076.stderr:2023-11-24T07:58:18.194+0000 7ff19972a700 -1 client.0 error registering admin socket command: (17) File exists 2023-11-24T07:58:18.209 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolumegroup rm cephfs subvol_grp1023 2023-11-24T07:58:18.212 INFO:tasks.ceph.mgr.x.smithi076.stderr:2023-11-24T07:58:18.210+0000 7ff197f27700 -1 client.0 error registering admin socket command: (17) File exists 2023-11-24T07:58:18.212 INFO:tasks.ceph.mgr.x.smithi076.stderr:2023-11-24T07:58:18.210+0000 7ff197f27700 -1 client.0 error registering admin socket command: (17) File exists 2023-11-24T07:58:18.212 INFO:tasks.ceph.mgr.x.smithi076.stderr:2023-11-24T07:58:18.210+0000 7ff197f27700 -1 client.0 error registering admin socket command: (17) File exists 2023-11-24T07:58:18.212 INFO:tasks.ceph.mgr.x.smithi076.stderr:2023-11-24T07:58:18.210+0000 7ff197f27700 -1 client.0 error registering admin socket command: (17) File exists 2023-11-24T07:58:18.212 INFO:tasks.ceph.mgr.x.smithi076.stderr:2023-11-24T07:58:18.210+0000 7ff197f27700 -1 client.0 error registering admin socket command: (17) File exists 2023-11-24T07:58:18.337 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:18.335+0000 7fbf0865a700 1 Processor -- start 2023-11-24T07:58:18.337 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:18.335+0000 7fbf0865a700 1 -- start start 2023-11-24T07:58:18.339 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:18.336+0000 7fbf0865a700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fbf000cfde0 0x7fbf000d24d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:18.339 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:18.336+0000 7fbf0865a700 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fbf000d2a10 0x7fbf000b4ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:18.339 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:18.336+0000 7fbf0865a700 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbf000b7810 con 0x7fbf0015e190 2023-11-24T07:58:18.339 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:18.336+0000 7fbf0865a700 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbf000d1930 con 0x7fbf000cfde0 2023-11-24T07:58:18.339 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:18.336+0000 7fbf0865a700 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbf000d2ea0 con 0x7fbf000d2a10 2023-11-24T07:58:18.339 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:18.337+0000 7fbf05bf5700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fbf000cfde0 0x7fbf000d24d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:18.339 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:18.337+0000 7fbf063f6700 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7fbf0015e190 0x7fbf000d0f50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:56574/0 (socket says 172.21.15.59:56574) 2023-11-24T07:58:18.339 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:18.337+0000 7fbf063f6700 1 -- 172.21.15.59:0/4251933675 learned_addr learned my addr 172.21.15.59:0/4251933675 (peer_addr_for_me v1:172.21.15.59:0/0) 2023-11-24T07:58:18.339 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:18.337+0000 7fbf06bf7700 1 --2- 172.21.15.59:0/4251933675 >> v2:172.21.15.76:3300/0 conn(0x7fbf000d2a10 0x7fbf000b4ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:18.340 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:18.337+0000 7fbf05bf5700 1 -- 172.21.15.59:0/4251933675 >> v2:172.21.15.76:3300/0 conn(0x7fbf000d2a10 msgr2=0x7fbf000b4ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T07:58:18.340 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:18.337+0000 7fbf05bf5700 1 --2- 172.21.15.59:0/4251933675 >> v2:172.21.15.76:3300/0 conn(0x7fbf000d2a10 0x7fbf000b4ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:18.340 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:18.337+0000 7fbf05bf5700 1 -- 172.21.15.59:0/4251933675 >> v1:172.21.15.59:6789/0 conn(0x7fbf0015e190 legacy=0x7fbf000d0f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:18.340 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:18.337+0000 7fbf05bf5700 1 -- 172.21.15.59:0/4251933675 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbf000d1660 con 0x7fbf000cfde0 2023-11-24T07:58:18.340 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:18.337+0000 7fbf053f4700 1 -- 172.21.15.59:0/4251933675 <== mon.0 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 696643980 0 0) 0x7fbf000b7810 con 0x7fbf0015e190 2023-11-24T07:58:18.340 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:18.337+0000 7fbf05bf5700 1 --2- 172.21.15.59:0/4251933675 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbf000cfde0 0x7fbf000d24d0 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7fbf000b7780 tx=0x7fbefc009b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6f374f7f73acf3ae server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:58:18.340 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:18.337+0000 7fbf053f4700 1 -- 172.21.15.59:0/4251933675 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbefc011830 con 0x7fbf000cfde0 2023-11-24T07:58:18.340 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:18.337+0000 7fbf053f4700 1 -- 172.21.15.59:0/4251933675 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbefc011990 con 0x7fbf000cfde0 2023-11-24T07:58:18.340 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:18.337+0000 7fbf053f4700 1 -- 172.21.15.59:0/4251933675 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbefc011d20 con 0x7fbf000cfde0 2023-11-24T07:58:18.343 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:18.338+0000 7fbf0865a700 1 -- 172.21.15.59:0/4251933675 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbf000cfde0 msgr2=0x7fbf000d24d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:18.343 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:18.338+0000 7fbf0865a700 1 --2- 172.21.15.59:0/4251933675 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbf000cfde0 0x7fbf000d24d0 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7fbf000b7780 tx=0x7fbefc009b20 comp rx=0 tx=0).stop 2023-11-24T07:58:18.343 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:18.338+0000 7fbf0865a700 1 -- 172.21.15.59:0/4251933675 shutdown_connections 2023-11-24T07:58:18.343 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:18.338+0000 7fbf0865a700 1 --2- 172.21.15.59:0/4251933675 >> v2:172.21.15.76:3300/0 conn(0x7fbf000d2a10 0x7fbf000b4ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:18.343 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:18.338+0000 7fbf0865a700 1 --2- 172.21.15.59:0/4251933675 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbf000cfde0 0x7fbf000d24d0 unknown :-1 s=CLOSED pgs=591 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:18.343 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:18.338+0000 7fbf0865a700 1 -- 172.21.15.59:0/4251933675 >> 172.21.15.59:0/4251933675 conn(0x7fbf0009d180 msgr2=0x7fbf000d3ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:58:18.343 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:18.338+0000 7fbf0865a700 1 -- 172.21.15.59:0/4251933675 shutdown_connections 2023-11-24T07:58:18.343 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:18.338+0000 7fbf0865a700 1 -- 172.21.15.59:0/4251933675 wait complete. 2023-11-24T07:58:18.343 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:18.339+0000 7fbf0865a700 1 Processor -- start 2023-11-24T07:58:18.344 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:18.339+0000 7fbf0865a700 1 -- start start 2023-11-24T07:58:18.344 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:18.340+0000 7fbf0865a700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fbf000cfde0 0x7fbf0015a8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:18.344 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:18.340+0000 7fbf0865a700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbf000d2a10 0x7fbf000cd7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:18.344 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:18.340+0000 7fbf0865a700 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fbf0015e190 0x7fbf0015be70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:18.344 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:18.340+0000 7fbf0865a700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf000cc740 con 0x7fbf000d2a10 2023-11-24T07:58:18.344 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:18.340+0000 7fbf0865a700 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf000cca20 con 0x7fbf000cfde0 2023-11-24T07:58:18.344 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:18.340+0000 7fbf0865a700 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbf000ccd00 con 0x7fbf0015e190 2023-11-24T07:58:18.344 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:18.340+0000 7fbf05bf5700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbf000d2a10 0x7fbf000cd7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:18.344 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:18.340+0000 7fbf05bf5700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbf000d2a10 0x7fbf000cd7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:53108/0 (socket says 172.21.15.59:53108) 2023-11-24T07:58:18.344 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:18.340+0000 7fbf05bf5700 1 -- 172.21.15.59:0/3018168269 learned_addr learned my addr 172.21.15.59:0/3018168269 (peer_addr_for_me v2:172.21.15.59:0/0) 2023-11-24T07:58:18.344 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:18.340+0000 7fbf063f6700 1 --2- 172.21.15.59:0/3018168269 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fbf000cfde0 0x7fbf0015a8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:18.344 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:18.340+0000 7fbf06bf7700 1 --2- 172.21.15.59:0/3018168269 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fbf0015e190 0x7fbf0015be70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:18.344 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:18.340+0000 7fbf05bf5700 1 -- 172.21.15.59:0/3018168269 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fbf0015e190 msgr2=0x7fbf0015be70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:18.344 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:18.340+0000 7fbf05bf5700 1 --2- 172.21.15.59:0/3018168269 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fbf0015e190 0x7fbf0015be70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:18.345 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:18.340+0000 7fbf05bf5700 1 -- 172.21.15.59:0/3018168269 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fbf000cfde0 msgr2=0x7fbf0015a8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:18.345 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:18.340+0000 7fbf05bf5700 1 --2- 172.21.15.59:0/3018168269 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fbf000cfde0 0x7fbf0015a8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:18.345 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:18.340+0000 7fbf05bf5700 1 -- 172.21.15.59:0/3018168269 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbefc0094a0 con 0x7fbf000d2a10 2023-11-24T07:58:18.345 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:18.340+0000 7fbf06bf7700 1 --2- 172.21.15.59:0/3018168269 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fbf0015e190 0x7fbf0015be70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-24T07:58:18.345 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:18.340+0000 7fbf063f6700 1 --2- 172.21.15.59:0/3018168269 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fbf000cfde0 0x7fbf0015a8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-24T07:58:18.345 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:18.341+0000 7fbf05bf5700 1 --2- 172.21.15.59:0/3018168269 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbf000d2a10 0x7fbf000cd7e0 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7fbefc0090b0 tx=0x7fbefc004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:58:18.345 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:18.341+0000 7fbef6ffd700 1 -- 172.21.15.59:0/3018168269 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbefc0038a0 con 0x7fbf000d2a10 2023-11-24T07:58:18.345 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:18.341+0000 7fbef6ffd700 1 -- 172.21.15.59:0/3018168269 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbefc01f070 con 0x7fbf000d2a10 2023-11-24T07:58:18.345 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:18.341+0000 7fbef6ffd700 1 -- 172.21.15.59:0/3018168269 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbefc003c20 con 0x7fbf000d2a10 2023-11-24T07:58:18.345 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:18.341+0000 7fbf0865a700 1 -- 172.21.15.59:0/3018168269 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbf000ccfe0 con 0x7fbf000d2a10 2023-11-24T07:58:18.345 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:18.341+0000 7fbf0865a700 1 -- 172.21.15.59:0/3018168269 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbf000cdd20 con 0x7fbf000d2a10 2023-11-24T07:58:18.346 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:18.342+0000 7fbf0865a700 1 -- 172.21.15.59:0/3018168269 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbf000cd630 con 0x7fbf000d2a10 2023-11-24T07:58:18.346 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:18.342+0000 7fbef6ffd700 1 -- 172.21.15.59:0/3018168269 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 7) v1 ==== 89333+0+0 (secure 0 0 0) 0x7fbefc01a430 con 0x7fbf000d2a10 2023-11-24T07:58:18.346 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:18.343+0000 7fbef6ffd700 1 --2- 172.21.15.59:0/3018168269 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7fbee406aff0 0x7fbee406d4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:18.346 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:18.343+0000 7fbef6ffd700 1 -- 172.21.15.59:0/3018168269 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5854+0+0 (secure 0 0 0) 0x7fbefc021070 con 0x7fbf000d2a10 2023-11-24T07:58:18.347 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:18.343+0000 7fbf063f6700 1 --2- 172.21.15.59:0/3018168269 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7fbee406aff0 0x7fbee406d4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:18.347 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:18.343+0000 7fbf063f6700 1 --2- 172.21.15.59:0/3018168269 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7fbee406aff0 0x7fbee406d4b0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7fbef0000f60 tx=0x7fbef0007d30 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:58:18.349 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:18.347+0000 7fbef6ffd700 1 -- 172.21.15.59:0/3018168269 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fbefc0bde90 con 0x7fbf000d2a10 2023-11-24T07:58:18.583 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:18.582+0000 7fbf0865a700 1 -- 172.21.15.59:0/3018168269 --> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] -- mgr_command(tid 0: {"prefix": "fs subvolumegroup rm", "vol_name": "cephfs", "group_name": "subvol_grp1023", "target": ["mon-mgr", ""]}) v1 -- 0x7fbf000ccb90 con 0x7fbee406aff0 2023-11-24T07:58:18.588 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:18.586+0000 7fbef6ffd700 1 -- 172.21.15.59:0/3018168269 <== mgr.4100 v2:172.21.15.76:6832/4264046536 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7fbf000ccb90 con 0x7fbee406aff0 2023-11-24T07:58:18.590 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:18.588+0000 7fbf0865a700 1 -- 172.21.15.59:0/3018168269 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7fbee406aff0 msgr2=0x7fbee406d4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:18.590 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:18.588+0000 7fbf0865a700 1 --2- 172.21.15.59:0/3018168269 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7fbee406aff0 0x7fbee406d4b0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7fbef0000f60 tx=0x7fbef0007d30 comp rx=0 tx=0).stop 2023-11-24T07:58:18.590 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:18.588+0000 7fbf0865a700 1 -- 172.21.15.59:0/3018168269 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbf000d2a10 msgr2=0x7fbf000cd7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:18.591 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:18.588+0000 7fbf0865a700 1 --2- 172.21.15.59:0/3018168269 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbf000d2a10 0x7fbf000cd7e0 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7fbefc0090b0 tx=0x7fbefc004750 comp rx=0 tx=0).stop 2023-11-24T07:58:18.591 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:18.588+0000 7fbf0865a700 1 -- 172.21.15.59:0/3018168269 shutdown_connections 2023-11-24T07:58:18.591 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:18.588+0000 7fbf0865a700 1 --2- 172.21.15.59:0/3018168269 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7fbee406aff0 0x7fbee406d4b0 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:18.591 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:18.588+0000 7fbf0865a700 1 --2- 172.21.15.59:0/3018168269 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fbf0015e190 0x7fbf0015be70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:18.591 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:18.588+0000 7fbf0865a700 1 --2- 172.21.15.59:0/3018168269 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fbf000cfde0 0x7fbf0015a8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:18.591 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:18.588+0000 7fbf0865a700 1 --2- 172.21.15.59:0/3018168269 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbf000d2a10 0x7fbf000cd7e0 unknown :-1 s=CLOSED pgs=592 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:18.591 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:18.588+0000 7fbf0865a700 1 -- 172.21.15.59:0/3018168269 >> 172.21.15.59:0/3018168269 conn(0x7fbf0009d180 msgr2=0x7fbf0006c2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:58:18.591 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:18.588+0000 7fbf0865a700 1 -- 172.21.15.59:0/3018168269 shutdown_connections 2023-11-24T07:58:18.591 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:18.588+0000 7fbf0865a700 1 -- 172.21.15.59:0/3018168269 wait complete. 2023-11-24T07:58:18.602 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T07:58:18.602 DEBUG:teuthology.orchestra.run.smithi059:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'stat -c %h /home/ubuntu/cephtest/mnt.0/./volumes/_deleting') 2023-11-24T07:58:18.640 INFO:teuthology.orchestra.run.smithi059.stdout:2 2023-11-24T07:58:18.641 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T07:58:18.641 DEBUG:teuthology.orchestra.run.smithi059:> (cd /home/ubuntu/cephtest/mnt.0 && exec sudo rm -f ./volumes/subvol_grp1023/subvol4495/.meta.stale_snap_section) 2023-11-24T07:58:18.710 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-11-24T07:58:18.710 DEBUG:teuthology.orchestra.run.smithi059:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-11-24T07:58:18.791 DEBUG:teuthology.orchestra.run.smithi076:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-11-24T07:58:18.841 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-11-24T07:58:18.841 DEBUG:teuthology.orchestra.run.smithi059:> set -ex 2023-11-24T07:58:18.842 DEBUG:teuthology.orchestra.run.smithi059:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-24T07:58:18.859 DEBUG:teuthology.orchestra.run.smithi059:> set -ex 2023-11-24T07:58:18.859 DEBUG:teuthology.orchestra.run.smithi059:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-24T07:58:18.917 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-24T07:58:19.082 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:19.080+0000 7f9501e69700 1 Processor -- start 2023-11-24T07:58:19.082 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:19.080+0000 7f9501e69700 1 -- start start 2023-11-24T07:58:19.084 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:19.081+0000 7f9501e69700 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f94fc0d0080 0x7f94fc0b5890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:19.084 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:19.081+0000 7f9501e69700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f94fc071bb0 0x7f94fc06afd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:19.085 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:19.081+0000 7f9501e69700 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f94fc147bb0 con 0x7f94fc06b510 2023-11-24T07:58:19.085 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:19.081+0000 7f9501e69700 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f94fc072cd0 con 0x7f94fc071bb0 2023-11-24T07:58:19.085 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:19.081+0000 7f9501e69700 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f94fc06b9a0 con 0x7f94fc0d0080 2023-11-24T07:58:19.085 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:19.081+0000 7f94faffd700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f94fc071bb0 0x7f94fc06afd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:19.085 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:19.081+0000 7f94faffd700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f94fc071bb0 0x7f94fc06afd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:53116/0 (socket says 172.21.15.59:53116) 2023-11-24T07:58:19.085 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:19.081+0000 7f94faffd700 1 -- 172.21.15.59:0/987963600 learned_addr learned my addr 172.21.15.59:0/987963600 (peer_addr_for_me v2:172.21.15.59:0/0) 2023-11-24T07:58:19.085 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:19.081+0000 7f94fb7fe700 1 --2- 172.21.15.59:0/987963600 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f94fc0d0080 0x7f94fc0b5890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:19.085 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:19.081+0000 7f94faffd700 1 -- 172.21.15.59:0/987963600 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f94fc0d0080 msgr2=0x7f94fc0b5890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T07:58:19.085 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:19.081+0000 7f94faffd700 1 --2- 172.21.15.59:0/987963600 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f94fc0d0080 0x7f94fc0b5890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:19.085 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:19.081+0000 7f94faffd700 1 -- 172.21.15.59:0/987963600 >> v1:172.21.15.76:6789/0 conn(0x7f94fc06b510 legacy=0x7f94fc161b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:19.085 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:19.081+0000 7f94faffd700 1 -- 172.21.15.59:0/987963600 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f94fc0b5dd0 con 0x7f94fc071bb0 2023-11-24T07:58:19.085 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:19.082+0000 7f94faffd700 1 --2- 172.21.15.59:0/987963600 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f94fc071bb0 0x7f94fc06afd0 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7f94e40098f0 tx=0x7f94e4009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=acb64bfb3596a94 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:58:19.085 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:19.082+0000 7f94fa7fc700 1 -- 172.21.15.59:0/987963600 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94e400b8a0 con 0x7f94fc071bb0 2023-11-24T07:58:19.086 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:19.082+0000 7f94fa7fc700 1 -- 172.21.15.59:0/987963600 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f94e400ba00 con 0x7f94fc071bb0 2023-11-24T07:58:19.086 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:19.082+0000 7f94fa7fc700 1 -- 172.21.15.59:0/987963600 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94e400bd90 con 0x7f94fc071bb0 2023-11-24T07:58:19.086 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:19.082+0000 7f9501e69700 1 -- 172.21.15.59:0/987963600 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f94fc071bb0 msgr2=0x7f94fc06afd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:19.086 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:19.082+0000 7f9501e69700 1 --2- 172.21.15.59:0/987963600 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f94fc071bb0 0x7f94fc06afd0 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7f94e40098f0 tx=0x7f94e4009c00 comp rx=0 tx=0).stop 2023-11-24T07:58:19.086 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:19.082+0000 7f9501e69700 1 -- 172.21.15.59:0/987963600 shutdown_connections 2023-11-24T07:58:19.086 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:19.082+0000 7f9501e69700 1 --2- 172.21.15.59:0/987963600 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f94fc0d0080 0x7f94fc0b5890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:19.086 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:19.082+0000 7f9501e69700 1 --2- 172.21.15.59:0/987963600 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f94fc071bb0 0x7f94fc06afd0 unknown :-1 s=CLOSED pgs=593 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:19.086 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:19.082+0000 7f9501e69700 1 -- 172.21.15.59:0/987963600 >> 172.21.15.59:0/987963600 conn(0x7f94fc0b27d0 msgr2=0x7f94fc0b2be0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:58:19.086 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:19.082+0000 7f9501e69700 1 -- 172.21.15.59:0/987963600 shutdown_connections 2023-11-24T07:58:19.086 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:19.082+0000 7f9501e69700 1 -- 172.21.15.59:0/987963600 wait complete. 2023-11-24T07:58:19.089 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:19.084+0000 7f9501e69700 1 Processor -- start 2023-11-24T07:58:19.089 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:19.084+0000 7f9501e69700 1 -- start start 2023-11-24T07:58:19.089 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:19.085+0000 7f9501e69700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f94fc06b510 0x7f94fc0b1520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:19.089 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:19.085+0000 7f9501e69700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f94fc071bb0 0x7f94fc15d550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:19.089 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:19.085+0000 7f9501e69700 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f94fc0d0080 0x7f94fc144e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:19.089 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:19.085+0000 7f9501e69700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94fc0ae2a0 con 0x7f94fc071bb0 2023-11-24T07:58:19.089 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:19.085+0000 7f9501e69700 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94fc143b80 con 0x7f94fc06b510 2023-11-24T07:58:19.090 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:19.085+0000 7f9501e69700 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f94fc0ad900 con 0x7f94fc0d0080 2023-11-24T07:58:19.090 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:19.085+0000 7f94faffd700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f94fc071bb0 0x7f94fc15d550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:19.090 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:19.085+0000 7f94faffd700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f94fc071bb0 0x7f94fc15d550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:53124/0 (socket says 172.21.15.59:53124) 2023-11-24T07:58:19.090 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:19.085+0000 7f94faffd700 1 -- 172.21.15.59:0/3830583168 learned_addr learned my addr 172.21.15.59:0/3830583168 (peer_addr_for_me v2:172.21.15.59:0/0) 2023-11-24T07:58:19.090 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:19.085+0000 7f94fb7fe700 1 --2- 172.21.15.59:0/3830583168 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f94fc06b510 0x7f94fc0b1520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:19.090 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:19.085+0000 7f94fbfff700 1 --2- 172.21.15.59:0/3830583168 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f94fc0d0080 0x7f94fc144e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:19.090 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:19.085+0000 7f94faffd700 1 -- 172.21.15.59:0/3830583168 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f94fc0d0080 msgr2=0x7f94fc144e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:19.090 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:19.085+0000 7f94faffd700 1 --2- 172.21.15.59:0/3830583168 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f94fc0d0080 0x7f94fc144e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:19.090 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:19.085+0000 7f94faffd700 1 -- 172.21.15.59:0/3830583168 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f94fc06b510 msgr2=0x7f94fc0b1520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:19.090 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:19.085+0000 7f94faffd700 1 --2- 172.21.15.59:0/3830583168 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f94fc06b510 0x7f94fc0b1520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:19.090 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:19.085+0000 7f94faffd700 1 -- 172.21.15.59:0/3830583168 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f94e4009550 con 0x7f94fc071bb0 2023-11-24T07:58:19.090 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:19.085+0000 7f94faffd700 1 --2- 172.21.15.59:0/3830583168 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f94fc071bb0 0x7f94fc15d550 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7f94e4000c00 tx=0x7f94e4004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:58:19.091 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:19.085+0000 7f94f3fff700 1 -- 172.21.15.59:0/3830583168 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94e4004300 con 0x7f94fc071bb0 2023-11-24T07:58:19.091 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:19.085+0000 7f94f3fff700 1 -- 172.21.15.59:0/3830583168 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f94e401f070 con 0x7f94fc071bb0 2023-11-24T07:58:19.091 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:19.085+0000 7f9501e69700 1 -- 172.21.15.59:0/3830583168 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f94fc0adbe0 con 0x7f94fc071bb0 2023-11-24T07:58:19.091 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:19.085+0000 7f94f3fff700 1 -- 172.21.15.59:0/3830583168 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94e40036a0 con 0x7f94fc071bb0 2023-11-24T07:58:19.091 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:19.085+0000 7f9501e69700 1 -- 172.21.15.59:0/3830583168 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f94fc15da90 con 0x7f94fc071bb0 2023-11-24T07:58:19.091 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:19.086+0000 7f9501e69700 1 -- 172.21.15.59:0/3830583168 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f94fc068670 con 0x7f94fc071bb0 2023-11-24T07:58:19.091 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:19.087+0000 7f94f3fff700 1 -- 172.21.15.59:0/3830583168 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 7) v1 ==== 89333+0+0 (secure 0 0 0) 0x7f94e4021030 con 0x7f94fc071bb0 2023-11-24T07:58:19.091 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:19.087+0000 7f94f3fff700 1 --2- 172.21.15.59:0/3830583168 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f94dc06aff0 0x7f94dc06d4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:19.091 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:19.087+0000 7f94f3fff700 1 -- 172.21.15.59:0/3830583168 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5854+0+0 (secure 0 0 0) 0x7f94e408aa70 con 0x7f94fc071bb0 2023-11-24T07:58:19.091 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:19.087+0000 7f94fb7fe700 1 --2- 172.21.15.59:0/3830583168 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f94dc06aff0 0x7f94dc06d4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:19.092 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:19.088+0000 7f94fb7fe700 1 --2- 172.21.15.59:0/3830583168 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f94dc06aff0 0x7f94dc06d4b0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f94ec007a30 tx=0x7f94ec008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:58:19.093 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:19.092+0000 7f94f3fff700 1 -- 172.21.15.59:0/3830583168 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f94e4055ab0 con 0x7f94fc071bb0 2023-11-24T07:58:19.295 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:19.294+0000 7f9501e69700 1 -- 172.21.15.59:0/3830583168 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f94fc0ae3e0 con 0x7f94fc071bb0 2023-11-24T07:58:19.296 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:19.295+0000 7f94f3fff700 1 -- 172.21.15.59:0/3830583168 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v47) v1 ==== 74+0+16668 (secure 0 0 0) 0x7f94fc0ae3e0 con 0x7f94fc071bb0 2023-11-24T07:58:19.297 INFO:teuthology.orchestra.run.smithi059.stdout: 2023-11-24T07:58:19.297 INFO:teuthology.orchestra.run.smithi059.stdout:{"epoch":47,"fsid":"9f0e8741-b8f9-48d5-a2c3-09f45903303d","created":"2023-11-24T07:56:06.552729+0000","modified":"2023-11-24T07:58:14.353149+0000","last_up_change":"2023-11-24T07:56:17.372352+0000","last_in_change":"2023-11-24T07:56:09.073250+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-24T07:56:18.352823+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_metadata","create_time":"2023-11-24T07:58:00.240130+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"44","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_data","create_time":"2023-11-24T07:58:00.977486+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"45","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"6d8f9e9b-1782-4e41-928b-a39f2a8ebc06","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6800","nonce":3373878772},{"type":"v1","addr":"172.21.15.59:6801","nonce":3373878772}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6802","nonce":3373878772},{"type":"v1","addr":"172.21.15.59:6803","nonce":3373878772}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6806","nonce":3373878772},{"type":"v1","addr":"172.21.15.59:6807","nonce":3373878772}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6804","nonce":3373878772},{"type":"v1","addr":"172.21.15.59:6805","nonce":3373878772}]},"public_addr":"172.21.15.59:6801/3373878772","cluster_addr":"172.21.15.59:6803/3373878772","heartbeat_back_addr":"172.21.15.59:6807/3373878772","heartbeat_front_addr":"172.21.15.59:6805/3373878772","state":["exists","up"]},{"osd":1,"uuid":"e8d882aa-52e1-4c04-9e1e-c44ef9204a5f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6824","nonce":648258563},{"type":"v1","addr":"172.21.15.59:6825","nonce":648258563}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6826","nonce":648258563},{"type":"v1","addr":"172.21.15.59:6827","nonce":648258563}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6830","nonce":648258563},{"type":"v1","addr":"172.21.15.59:6831","nonce":648258563}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6828","nonce":648258563},{"type":"v1","addr":"172.21.15.59:6829","nonce":648258563}]},"public_addr":"172.21.15.59:6825/648258563","cluster_addr":"172.21.15.59:6827/648258563","heartbeat_back_addr":"172.21.15.59:6831/648258563","heartbeat_front_addr":"172.21.15.59:6829/648258563","state":["exists","up"]},{"osd":2,"uuid":"69eebfa0-695b-4b21-90b9-8a3537168a01","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6808","nonce":1840430654},{"type":"v1","addr":"172.21.15.59:6809","nonce":1840430654}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6810","nonce":1840430654},{"type":"v1","addr":"172.21.15.59:6811","nonce":1840430654}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6814","nonce":1840430654},{"type":"v1","addr":"172.21.15.59:6815","nonce":1840430654}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6812","nonce":1840430654},{"type":"v1","addr":"172.21.15.59:6813","nonce":1840430654}]},"public_addr":"172.21.15.59:6809/1840430654","cluster_addr":"172.21.15.59:6811/1840430654","heartbeat_back_addr":"172.21.15.59:6815/1840430654","heartbeat_front_addr":"172.21.15.59:6813/1840430654","state":["exists","up"]},{"osd":3,"uuid":"96fd77fc-f911-46ff-9445-6f0be2daa8f4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6816","nonce":2504025850},{"type":"v1","addr":"172.21.15.59:6817","nonce":2504025850}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6818","nonce":2504025850},{"type":"v1","addr":"172.21.15.59:6819","nonce":2504025850}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6822","nonce":2504025850},{"type":"v1","addr":"172.21.15.59:6823","nonce":2504025850}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6820","nonce":2504025850},{"type":"v1","addr":"172.21.15.59:6821","nonce":2504025850}]},"public_addr":"172.21.15.59:6817/2504025850","cluster_addr":"172.21.15.59:6819/2504025850","heartbeat_back_addr":"172.21.15.59:6823/2504025850","heartbeat_front_addr":"172.21.15.59:6821/2504025850","state":["exists","up"]},{"osd":4,"uuid":"7ac29fca-b5b2-4c95-9d69-2f164cbbc708","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6816","nonce":2382463431},{"type":"v1","addr":"172.21.15.76:6818","nonce":2382463431}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6820","nonce":2382463431},{"type":"v1","addr":"172.21.15.76:6822","nonce":2382463431}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6828","nonce":2382463431},{"type":"v1","addr":"172.21.15.76:6830","nonce":2382463431}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6824","nonce":2382463431},{"type":"v1","addr":"172.21.15.76:6826","nonce":2382463431}]},"public_addr":"172.21.15.76:6818/2382463431","cluster_addr":"172.21.15.76:6822/2382463431","heartbeat_back_addr":"172.21.15.76:6830/2382463431","heartbeat_front_addr":"172.21.15.76:6826/2382463431","state":["exists","up"]},{"osd":5,"uuid":"485bf711-3851-4238-a4f4-51d52e0a9c58","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6800","nonce":329334308},{"type":"v1","addr":"172.21.15.76:6802","nonce":329334308}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6805","nonce":329334308},{"type":"v1","addr":"172.21.15.76:6807","nonce":329334308}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6814","nonce":329334308},{"type":"v1","addr":"172.21.15.76:6815","nonce":329334308}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6810","nonce":329334308},{"type":"v1","addr":"172.21.15.76:6812","nonce":329334308}]},"public_addr":"172.21.15.76:6802/329334308","cluster_addr":"172.21.15.76:6807/329334308","heartbeat_back_addr":"172.21.15.76:6815/329334308","heartbeat_front_addr":"172.21.15.76:6812/329334308","state":["exists","up"]},{"osd":6,"uuid":"72b3f2fc-1e4a-4e32-968d-003647a0286f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6801","nonce":3946579410},{"type":"v1","addr":"172.21.15.76:6803","nonce":3946579410}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6804","nonce":3946579410},{"type":"v1","addr":"172.21.15.76:6806","nonce":3946579410}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6811","nonce":3946579410},{"type":"v1","addr":"172.21.15.76:6813","nonce":3946579410}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6808","nonce":3946579410},{"type":"v1","addr":"172.21.15.76:6809","nonce":3946579410}]},"public_addr":"172.21.15.76:6803/3946579410","cluster_addr":"172.21.15.76:6806/3946579410","heartbeat_back_addr":"172.21.15.76:6813/3946579410","heartbeat_front_addr":"172.21.15.76:6809/3946579410","state":["exists","up"]},{"osd":7,"uuid":"1ddceb90-6c5d-4d1a-ad56-99cb877f8c70","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6817","nonce":841761538},{"type":"v1","addr":"172.21.15.76:6819","nonce":841761538}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6821","nonce":841761538},{"type":"v1","addr":"172.21.15.76:6823","nonce":841761538}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6829","nonce":841761538},{"type":"v1","addr":"172.21.15.76:6831","nonce":841761538}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6825","nonce":841761538},{"type":"v1","addr":"172.21.15.76:6827","nonce":841761538}]},"public_addr":"172.21.15.76:6819/841761538","cluster_addr":"172.21.15.76:6823/841761538","heartbeat_back_addr":"172.21.15.76:6831/841761538","heartbeat_front_addr":"172.21.15.76:6827/841761538","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T07:56:14.294838+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T07:56:14.270779+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T07:56:14.231809+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T07:56:14.260983+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T07:56:14.302997+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T07:56:14.258600+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T07:56:14.251413+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T07:56:14.304727+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"7.0","mappings":[{"from":5,"to":7}]},{"pgid":"7.c","mappings":[{"from":5,"to":7}]},{"pgid":"7.12","mappings":[{"from":5,"to":7}]},{"pgid":"7.14","mappings":[{"from":5,"to":7}]},{"pgid":"7.38","mappings":[{"from":5,"to":1}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-24T07:58:19.299 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:19.297+0000 7f9501e69700 1 -- 172.21.15.59:0/3830583168 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f94dc06aff0 msgr2=0x7f94dc06d4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:19.299 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:19.297+0000 7f9501e69700 1 --2- 172.21.15.59:0/3830583168 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f94dc06aff0 0x7f94dc06d4b0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f94ec007a30 tx=0x7f94ec008040 comp rx=0 tx=0).stop 2023-11-24T07:58:19.299 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:19.297+0000 7f9501e69700 1 -- 172.21.15.59:0/3830583168 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f94fc071bb0 msgr2=0x7f94fc15d550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:19.299 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:19.297+0000 7f9501e69700 1 --2- 172.21.15.59:0/3830583168 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f94fc071bb0 0x7f94fc15d550 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7f94e4000c00 tx=0x7f94e4004750 comp rx=0 tx=0).stop 2023-11-24T07:58:19.299 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:19.297+0000 7f9501e69700 1 -- 172.21.15.59:0/3830583168 shutdown_connections 2023-11-24T07:58:19.299 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:19.297+0000 7f9501e69700 1 --2- 172.21.15.59:0/3830583168 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f94dc06aff0 0x7f94dc06d4b0 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:19.299 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:19.297+0000 7f9501e69700 1 --2- 172.21.15.59:0/3830583168 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f94fc0d0080 0x7f94fc144e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:19.299 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:19.297+0000 7f9501e69700 1 --2- 172.21.15.59:0/3830583168 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f94fc06b510 0x7f94fc0b1520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:19.299 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:19.297+0000 7f9501e69700 1 --2- 172.21.15.59:0/3830583168 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f94fc071bb0 0x7f94fc15d550 unknown :-1 s=CLOSED pgs=594 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:19.300 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:19.297+0000 7f9501e69700 1 -- 172.21.15.59:0/3830583168 >> 172.21.15.59:0/3830583168 conn(0x7f94fc0b27d0 msgr2=0x7f94fc15f990 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:58:19.300 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:19.297+0000 7f9501e69700 1 -- 172.21.15.59:0/3830583168 shutdown_connections 2023-11-24T07:58:19.300 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:19.297+0000 7f9501e69700 1 -- 172.21.15.59:0/3830583168 wait complete. 2023-11-24T07:58:19.311 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-24T07:56:18.352823+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 6, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-24T07:58:00.240130+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '44', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 7, 'pool_name': 'cephfs_data', 'create_time': '2023-11-24T07:58:00.977486+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '45', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-24T07:58:19.311 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-24T07:58:19.441 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:19.439+0000 7f9025080700 1 Processor -- start 2023-11-24T07:58:19.441 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:19.439+0000 7f9025080700 1 -- start start 2023-11-24T07:58:19.443 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:19.440+0000 7f9025080700 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f90201440f0 0x7f90200b51b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:19.444 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:19.440+0000 7f9025080700 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9020143ec0 con 0x7f90200722c0 2023-11-24T07:58:19.444 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:19.440+0000 7f9025080700 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f90200b3ae0 con 0x7f9020143ae0 2023-11-24T07:58:19.444 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:19.440+0000 7f9025080700 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f90200b40a0 con 0x7f90201440f0 2023-11-24T07:58:19.444 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:19.440+0000 7f901e59c700 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7f90200722c0 0x7f902006e510 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:56606/0 (socket says 172.21.15.59:56606) 2023-11-24T07:58:19.444 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:19.440+0000 7f901e59c700 1 -- 172.21.15.59:0/874733333 learned_addr learned my addr 172.21.15.59:0/874733333 (peer_addr_for_me v1:172.21.15.59:0/0) 2023-11-24T07:58:19.444 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:19.441+0000 7f901f59e700 1 --2- 172.21.15.59:0/874733333 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f90201440f0 0x7f90200b51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:19.444 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:19.441+0000 7f901dd9b700 1 -- 172.21.15.59:0/874733333 <== mon.0 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3318248605 0 0) 0x7f9020143ec0 con 0x7f90200722c0 2023-11-24T07:58:19.444 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:19.441+0000 7f901dd9b700 1 -- 172.21.15.59:0/874733333 --> v1:172.21.15.59:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f900c0035b0 con 0x7f90200722c0 2023-11-24T07:58:19.444 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:19.441+0000 7f901dd9b700 1 -- 172.21.15.59:0/874733333 <== mon.1 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1871228300 0 0) 0x7f90200b3ae0 con 0x7f9020143ae0 2023-11-24T07:58:19.444 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:19.441+0000 7f901dd9b700 1 -- 172.21.15.59:0/874733333 --> v1:172.21.15.76:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9020143ec0 con 0x7f9020143ae0 2023-11-24T07:58:19.444 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:19.441+0000 7f901f59e700 1 -- 172.21.15.59:0/874733333 >> v1:172.21.15.76:6789/0 conn(0x7f9020143ae0 legacy=0x7f9020071bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:19.444 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:19.441+0000 7f901f59e700 1 -- 172.21.15.59:0/874733333 >> v1:172.21.15.59:6789/0 conn(0x7f90200722c0 legacy=0x7f902006e510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:19.445 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:19.441+0000 7f901dd9b700 1 -- 172.21.15.59:0/874733333 <== mon.0 v1:172.21.15.59:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3595160059 0 0) 0x7f900c0035b0 con 0x7f90200722c0 2023-11-24T07:58:19.445 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:19.441+0000 7f901f59e700 1 -- 172.21.15.59:0/874733333 --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f90200b3dc0 con 0x7f90201440f0 2023-11-24T07:58:19.445 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:19.441+0000 7f901f59e700 1 --2- 172.21.15.59:0/874733333 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f90201440f0 0x7f90200b51b0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f9018009dc0 tx=0x7f901800b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=409a07a17e6f220c server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:58:19.445 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:19.441+0000 7f901dd9b700 1 -- 172.21.15.59:0/874733333 <== mon.2 v2:172.21.15.76:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f901800e070 con 0x7f90201440f0 2023-11-24T07:58:19.445 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:19.441+0000 7f901dd9b700 1 -- 172.21.15.59:0/874733333 <== mon.2 v2:172.21.15.76:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9018003a20 con 0x7f90201440f0 2023-11-24T07:58:19.445 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:19.441+0000 7f901dd9b700 1 -- 172.21.15.59:0/874733333 <== mon.2 v2:172.21.15.76:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9008001670 con 0x7f90201440f0 2023-11-24T07:58:19.445 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:19.442+0000 7f9025080700 1 -- 172.21.15.59:0/874733333 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f90201440f0 msgr2=0x7f90200b51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:19.445 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:19.442+0000 7f9025080700 1 --2- 172.21.15.59:0/874733333 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f90201440f0 0x7f90200b51b0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f9018009dc0 tx=0x7f901800b330 comp rx=0 tx=0).stop 2023-11-24T07:58:19.445 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:19.442+0000 7f9025080700 1 -- 172.21.15.59:0/874733333 shutdown_connections 2023-11-24T07:58:19.445 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:19.442+0000 7f9025080700 1 --2- 172.21.15.59:0/874733333 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f90201440f0 0x7f90200b51b0 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:19.445 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:19.442+0000 7f9025080700 1 -- 172.21.15.59:0/874733333 >> 172.21.15.59:0/874733333 conn(0x7f90200b2740 msgr2=0x7f90200b2b20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:58:19.445 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:19.442+0000 7f9025080700 1 -- 172.21.15.59:0/874733333 shutdown_connections 2023-11-24T07:58:19.445 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:19.442+0000 7f9025080700 1 -- 172.21.15.59:0/874733333 wait complete. 2023-11-24T07:58:19.448 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:19.443+0000 7f9025080700 1 Processor -- start 2023-11-24T07:58:19.448 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:19.444+0000 7f9025080700 1 -- start start 2023-11-24T07:58:19.448 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:19.444+0000 7f9025080700 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f90200722c0 0x7f9020140270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:19.448 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:19.444+0000 7f9025080700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9020143ae0 0x7f902015cde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:19.448 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:19.444+0000 7f9025080700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f90201440f0 0x7f902015b3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:19.448 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:19.444+0000 7f9025080700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9020140900 con 0x7f9020143ae0 2023-11-24T07:58:19.448 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:19.444+0000 7f9025080700 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9020169d80 con 0x7f90201440f0 2023-11-24T07:58:19.449 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:19.444+0000 7f9025080700 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9020156dc0 con 0x7f90200722c0 2023-11-24T07:58:19.449 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:19.444+0000 7f901e59c700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9020143ae0 0x7f902015cde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:19.449 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:19.444+0000 7f901e59c700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9020143ae0 0x7f902015cde0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:53136/0 (socket says 172.21.15.59:53136) 2023-11-24T07:58:19.449 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:19.444+0000 7f901e59c700 1 -- 172.21.15.59:0/606403575 learned_addr learned my addr 172.21.15.59:0/606403575 (peer_addr_for_me v2:172.21.15.59:0/0) 2023-11-24T07:58:19.449 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:19.445+0000 7f901f59e700 1 --2- 172.21.15.59:0/606403575 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f90201440f0 0x7f902015b3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:19.449 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:19.445+0000 7f901ed9d700 1 --2- 172.21.15.59:0/606403575 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f90200722c0 0x7f9020140270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:19.449 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:19.445+0000 7f901e59c700 1 -- 172.21.15.59:0/606403575 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f90200722c0 msgr2=0x7f9020140270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:19.449 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:19.445+0000 7f901e59c700 1 --2- 172.21.15.59:0/606403575 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f90200722c0 0x7f9020140270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:19.449 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:19.445+0000 7f901e59c700 1 -- 172.21.15.59:0/606403575 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f90201440f0 msgr2=0x7f902015b3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:19.449 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:19.445+0000 7f901e59c700 1 --2- 172.21.15.59:0/606403575 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f90201440f0 0x7f902015b3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:19.449 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:19.445+0000 7f901e59c700 1 -- 172.21.15.59:0/606403575 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9018009a70 con 0x7f9020143ae0 2023-11-24T07:58:19.449 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:19.445+0000 7f901e59c700 1 --2- 172.21.15.59:0/606403575 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9020143ae0 0x7f902015cde0 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7f900800c770 tx=0x7f900800cb30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:58:19.450 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:19.445+0000 7f90177fe700 1 -- 172.21.15.59:0/606403575 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9008007560 con 0x7f9020143ae0 2023-11-24T07:58:19.450 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:19.445+0000 7f90177fe700 1 -- 172.21.15.59:0/606403575 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f90080076c0 con 0x7f9020143ae0 2023-11-24T07:58:19.450 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:19.445+0000 7f9025080700 1 -- 172.21.15.59:0/606403575 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f902015d320 con 0x7f9020143ae0 2023-11-24T07:58:19.450 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:19.445+0000 7f90177fe700 1 -- 172.21.15.59:0/606403575 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9008004a70 con 0x7f9020143ae0 2023-11-24T07:58:19.450 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:19.445+0000 7f9025080700 1 -- 172.21.15.59:0/606403575 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9020169aa0 con 0x7f9020143ae0 2023-11-24T07:58:19.450 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:19.446+0000 7f9025080700 1 -- 172.21.15.59:0/606403575 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f90201570f0 con 0x7f9020143ae0 2023-11-24T07:58:19.450 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:19.447+0000 7f90177fe700 1 -- 172.21.15.59:0/606403575 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 8) v1 ==== 89435+0+0 (secure 0 0 0) 0x7f9008004bd0 con 0x7f9020143ae0 2023-11-24T07:58:19.451 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:19.447+0000 7f90177fe700 1 --2- 172.21.15.59:0/606403575 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f900006b110 0x7f900006d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:19.451 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:19.447+0000 7f90177fe700 1 -- 172.21.15.59:0/606403575 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5854+0+0 (secure 0 0 0) 0x7f9008088dd0 con 0x7f9020143ae0 2023-11-24T07:58:19.451 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:19.447+0000 7f901ed9d700 1 --2- 172.21.15.59:0/606403575 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f900006b110 0x7f900006d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:19.451 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:19.448+0000 7f901ed9d700 1 --2- 172.21.15.59:0/606403575 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f900006b110 0x7f900006d5d0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f9010001120 tx=0x7f9010009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:58:19.453 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:19.451+0000 7f90177fe700 1 -- 172.21.15.59:0/606403575 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f900808d070 con 0x7f9020143ae0 2023-11-24T07:58:19.654 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:19.652+0000 7f9025080700 1 -- 172.21.15.59:0/606403575 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f9020156f00 con 0x7f9020143ae0 2023-11-24T07:58:19.654 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:19.653+0000 7f90177fe700 1 -- 172.21.15.59:0/606403575 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v47) v1 ==== 93+0+10 (secure 0 0 0) 0x7f90080180a0 con 0x7f9020143ae0 2023-11-24T07:58:19.655 INFO:teuthology.orchestra.run.smithi059.stdout:pg_num: 1 2023-11-24T07:58:19.657 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:19.655+0000 7f9025080700 1 -- 172.21.15.59:0/606403575 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f900006b110 msgr2=0x7f900006d5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:19.657 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:19.655+0000 7f9025080700 1 --2- 172.21.15.59:0/606403575 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f900006b110 0x7f900006d5d0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f9010001120 tx=0x7f9010009040 comp rx=0 tx=0).stop 2023-11-24T07:58:19.657 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:19.655+0000 7f9025080700 1 -- 172.21.15.59:0/606403575 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9020143ae0 msgr2=0x7f902015cde0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:19.657 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:19.655+0000 7f9025080700 1 --2- 172.21.15.59:0/606403575 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9020143ae0 0x7f902015cde0 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7f900800c770 tx=0x7f900800cb30 comp rx=0 tx=0).stop 2023-11-24T07:58:19.657 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:19.655+0000 7f9025080700 1 -- 172.21.15.59:0/606403575 shutdown_connections 2023-11-24T07:58:19.657 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:19.655+0000 7f9025080700 1 --2- 172.21.15.59:0/606403575 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f900006b110 0x7f900006d5d0 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:19.657 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:19.655+0000 7f9025080700 1 --2- 172.21.15.59:0/606403575 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f90201440f0 0x7f902015b3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:19.657 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:19.655+0000 7f9025080700 1 --2- 172.21.15.59:0/606403575 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f90200722c0 0x7f9020140270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:19.657 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:19.655+0000 7f9025080700 1 --2- 172.21.15.59:0/606403575 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9020143ae0 0x7f902015cde0 unknown :-1 s=CLOSED pgs=596 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:19.657 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:19.655+0000 7f9025080700 1 -- 172.21.15.59:0/606403575 >> 172.21.15.59:0/606403575 conn(0x7f90200b2740 msgr2=0x7f90200b41e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:58:19.658 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:19.655+0000 7f9025080700 1 -- 172.21.15.59:0/606403575 shutdown_connections 2023-11-24T07:58:19.658 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:19.655+0000 7f9025080700 1 -- 172.21.15.59:0/606403575 wait complete. 2023-11-24T07:58:19.669 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-24T07:58:19.796 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:19.794+0000 7f1c63293700 1 Processor -- start 2023-11-24T07:58:19.796 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:19.795+0000 7f1c63293700 1 -- start start 2023-11-24T07:58:19.798 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:19.795+0000 7f1c63293700 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f1c5c0722c0 0x7f1c5c0b51b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:19.798 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:19.795+0000 7f1c63293700 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f1c5c1440f0 0x7f1c5c06e510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:19.798 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:19.795+0000 7f1c63293700 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1c5c143ec0 con 0x7f1c5c143ae0 2023-11-24T07:58:19.799 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:19.795+0000 7f1c63293700 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1c5c0b40a0 con 0x7f1c5c0722c0 2023-11-24T07:58:19.799 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:19.795+0000 7f1c63293700 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1c5c0b3ae0 con 0x7f1c5c1440f0 2023-11-24T07:58:19.799 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:19.796+0000 7f1c6082e700 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f1c5c0722c0 0x7f1c5c0b51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:19.799 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:19.796+0000 7f1c61830700 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f1c5c1440f0 0x7f1c5c06e510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:19.799 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:19.796+0000 7f1c6102f700 1 --1- >> v1:172.21.15.76:6789/0 conn(0x7f1c5c143ae0 0x7f1c5c071bb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.59:43896/0 (socket says 172.21.15.59:43896) 2023-11-24T07:58:19.799 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:19.796+0000 7f1c6102f700 1 -- 172.21.15.59:0/3383446991 learned_addr learned my addr 172.21.15.59:0/3383446991 (peer_addr_for_me v1:172.21.15.59:0/0) 2023-11-24T07:58:19.799 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:19.796+0000 7f1c6082e700 1 -- 172.21.15.59:0/3383446991 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f1c5c1440f0 msgr2=0x7f1c5c06e510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T07:58:19.799 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:19.796+0000 7f1c6082e700 1 --2- 172.21.15.59:0/3383446991 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f1c5c1440f0 0x7f1c5c06e510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:19.799 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:19.796+0000 7f1c6082e700 1 -- 172.21.15.59:0/3383446991 >> v1:172.21.15.76:6789/0 conn(0x7f1c5c143ae0 legacy=0x7f1c5c071bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:19.799 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:19.796+0000 7f1c6082e700 1 -- 172.21.15.59:0/3383446991 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1c5c0b3dc0 con 0x7f1c5c0722c0 2023-11-24T07:58:19.800 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:19.796+0000 7f1c53fff700 1 -- 172.21.15.59:0/3383446991 <== mon.1 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 456804049 0 0) 0x7f1c5c143ec0 con 0x7f1c5c143ae0 2023-11-24T07:58:19.800 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:19.796+0000 7f1c61830700 1 --2- 172.21.15.59:0/3383446991 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f1c5c1440f0 0x7f1c5c06e510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-24T07:58:19.800 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:19.796+0000 7f1c6082e700 1 --2- 172.21.15.59:0/3383446991 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1c5c0722c0 0x7f1c5c0b51b0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f1c58009840 tx=0x7f1c58009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f4d0ff8e373397f3 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:58:19.800 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:19.797+0000 7f1c53fff700 1 -- 172.21.15.59:0/3383446991 <== mon.1 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c58011830 con 0x7f1c5c0722c0 2023-11-24T07:58:19.800 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:19.797+0000 7f1c53fff700 1 -- 172.21.15.59:0/3383446991 <== mon.1 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1c58011990 con 0x7f1c5c0722c0 2023-11-24T07:58:19.800 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:19.797+0000 7f1c53fff700 1 -- 172.21.15.59:0/3383446991 <== mon.1 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c58011d20 con 0x7f1c5c0722c0 2023-11-24T07:58:19.803 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:19.797+0000 7f1c63293700 1 -- 172.21.15.59:0/3383446991 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1c5c0722c0 msgr2=0x7f1c5c0b51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:19.803 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:19.797+0000 7f1c63293700 1 --2- 172.21.15.59:0/3383446991 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1c5c0722c0 0x7f1c5c0b51b0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f1c58009840 tx=0x7f1c58009b50 comp rx=0 tx=0).stop 2023-11-24T07:58:19.803 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:19.797+0000 7f1c63293700 1 -- 172.21.15.59:0/3383446991 shutdown_connections 2023-11-24T07:58:19.803 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:19.797+0000 7f1c63293700 1 --2- 172.21.15.59:0/3383446991 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f1c5c1440f0 0x7f1c5c06e510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:19.803 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:19.797+0000 7f1c63293700 1 --2- 172.21.15.59:0/3383446991 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1c5c0722c0 0x7f1c5c0b51b0 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:19.803 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:19.797+0000 7f1c63293700 1 -- 172.21.15.59:0/3383446991 >> 172.21.15.59:0/3383446991 conn(0x7f1c5c0b2740 msgr2=0x7f1c5c0b2b20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:58:19.803 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:19.797+0000 7f1c63293700 1 -- 172.21.15.59:0/3383446991 shutdown_connections 2023-11-24T07:58:19.804 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:19.797+0000 7f1c63293700 1 -- 172.21.15.59:0/3383446991 wait complete. 2023-11-24T07:58:19.804 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:19.798+0000 7f1c63293700 1 Processor -- start 2023-11-24T07:58:19.804 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:19.799+0000 7f1c63293700 1 -- start start 2023-11-24T07:58:19.804 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:19.799+0000 7f1c63293700 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f1c5c0722c0 0x7f1c5c13e6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:19.804 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:19.799+0000 7f1c63293700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1c5c143ae0 0x7f1c5c1576c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:19.804 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:19.799+0000 7f1c63293700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1c5c1440f0 0x7f1c5c157c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:19.804 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:19.799+0000 7f1c63293700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c5c156410 con 0x7f1c5c143ae0 2023-11-24T07:58:19.804 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:19.799+0000 7f1c63293700 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c5c15e4b0 con 0x7f1c5c1440f0 2023-11-24T07:58:19.804 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:19.799+0000 7f1c63293700 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1c5c156660 con 0x7f1c5c0722c0 2023-11-24T07:58:19.804 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:19.799+0000 7f1c6082e700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1c5c143ae0 0x7f1c5c1576c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:19.804 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:19.799+0000 7f1c6082e700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1c5c143ae0 0x7f1c5c1576c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:53152/0 (socket says 172.21.15.59:53152) 2023-11-24T07:58:19.805 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:19.799+0000 7f1c6082e700 1 -- 172.21.15.59:0/3146379149 learned_addr learned my addr 172.21.15.59:0/3146379149 (peer_addr_for_me v2:172.21.15.59:0/0) 2023-11-24T07:58:19.805 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:19.800+0000 7f1c61830700 1 --2- 172.21.15.59:0/3146379149 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1c5c1440f0 0x7f1c5c157c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:19.805 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:19.800+0000 7f1c6102f700 1 --2- 172.21.15.59:0/3146379149 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f1c5c0722c0 0x7f1c5c13e6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:19.805 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:19.800+0000 7f1c6082e700 1 -- 172.21.15.59:0/3146379149 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f1c5c0722c0 msgr2=0x7f1c5c13e6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:19.805 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:19.800+0000 7f1c6082e700 1 --2- 172.21.15.59:0/3146379149 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f1c5c0722c0 0x7f1c5c13e6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:19.805 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:19.800+0000 7f1c6082e700 1 -- 172.21.15.59:0/3146379149 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1c5c1440f0 msgr2=0x7f1c5c157c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:19.805 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:19.800+0000 7f1c6082e700 1 --2- 172.21.15.59:0/3146379149 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1c5c1440f0 0x7f1c5c157c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:19.805 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:19.800+0000 7f1c6082e700 1 -- 172.21.15.59:0/3146379149 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1c54009a70 con 0x7f1c5c143ae0 2023-11-24T07:58:19.805 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:19.800+0000 7f1c61830700 1 --2- 172.21.15.59:0/3146379149 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1c5c1440f0 0x7f1c5c157c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-24T07:58:19.805 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:19.800+0000 7f1c6102f700 1 --2- 172.21.15.59:0/3146379149 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f1c5c0722c0 0x7f1c5c13e6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-24T07:58:19.805 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:19.800+0000 7f1c6082e700 1 --2- 172.21.15.59:0/3146379149 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1c5c143ae0 0x7f1c5c1576c0 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7f1c58009fd0 tx=0x7f1c58004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:58:19.805 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:19.800+0000 7f1c51ffb700 1 -- 172.21.15.59:0/3146379149 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c580038a0 con 0x7f1c5c143ae0 2023-11-24T07:58:19.806 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:19.800+0000 7f1c63293700 1 -- 172.21.15.59:0/3146379149 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1c580094a0 con 0x7f1c5c143ae0 2023-11-24T07:58:19.806 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:19.800+0000 7f1c51ffb700 1 -- 172.21.15.59:0/3146379149 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1c5801f070 con 0x7f1c5c143ae0 2023-11-24T07:58:19.806 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:19.800+0000 7f1c51ffb700 1 -- 172.21.15.59:0/3146379149 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c58003c40 con 0x7f1c5c143ae0 2023-11-24T07:58:19.806 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:19.800+0000 7f1c63293700 1 -- 172.21.15.59:0/3146379149 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1c5c156c20 con 0x7f1c5c143ae0 2023-11-24T07:58:19.806 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:19.801+0000 7f1c63293700 1 -- 172.21.15.59:0/3146379149 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1c5c075480 con 0x7f1c5c143ae0 2023-11-24T07:58:19.807 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:19.802+0000 7f1c51ffb700 1 -- 172.21.15.59:0/3146379149 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 8) v1 ==== 89435+0+0 (secure 0 0 0) 0x7f1c58026480 con 0x7f1c5c143ae0 2023-11-24T07:58:19.807 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:19.802+0000 7f1c51ffb700 1 --2- 172.21.15.59:0/3146379149 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f1c4006b110 0x7f1c4006d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:19.807 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:19.802+0000 7f1c51ffb700 1 -- 172.21.15.59:0/3146379149 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5854+0+0 (secure 0 0 0) 0x7f1c58021070 con 0x7f1c5c143ae0 2023-11-24T07:58:19.807 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:19.802+0000 7f1c6102f700 1 --2- 172.21.15.59:0/3146379149 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f1c4006b110 0x7f1c4006d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:19.807 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:19.803+0000 7f1c6102f700 1 --2- 172.21.15.59:0/3146379149 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f1c4006b110 0x7f1c4006d5d0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f1c4c000f60 tx=0x7f1c4c007ca0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:58:19.808 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:19.807+0000 7f1c51ffb700 1 -- 172.21.15.59:0/3146379149 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f1c580560b0 con 0x7f1c5c143ae0 2023-11-24T07:58:20.008 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:20.006+0000 7f1c63293700 1 -- 172.21.15.59:0/3146379149 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f1c5c156e10 con 0x7f1c5c143ae0 2023-11-24T07:58:20.009 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:20.006+0000 7f1c51ffb700 1 -- 172.21.15.59:0/3146379149 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v47) v1 ==== 104+0+11 (secure 0 0 0) 0x7f1c580596d0 con 0x7f1c5c143ae0 2023-11-24T07:58:20.009 INFO:teuthology.orchestra.run.smithi059.stdout:pg_num: 32 2023-11-24T07:58:20.010 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:20.008+0000 7f1c63293700 1 -- 172.21.15.59:0/3146379149 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f1c4006b110 msgr2=0x7f1c4006d5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:20.010 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:20.008+0000 7f1c63293700 1 --2- 172.21.15.59:0/3146379149 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f1c4006b110 0x7f1c4006d5d0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f1c4c000f60 tx=0x7f1c4c007ca0 comp rx=0 tx=0).stop 2023-11-24T07:58:20.011 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:20.008+0000 7f1c63293700 1 -- 172.21.15.59:0/3146379149 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1c5c143ae0 msgr2=0x7f1c5c1576c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:20.011 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:20.008+0000 7f1c63293700 1 --2- 172.21.15.59:0/3146379149 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1c5c143ae0 0x7f1c5c1576c0 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7f1c58009fd0 tx=0x7f1c58004750 comp rx=0 tx=0).stop 2023-11-24T07:58:20.011 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:20.008+0000 7f1c63293700 1 -- 172.21.15.59:0/3146379149 shutdown_connections 2023-11-24T07:58:20.011 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:20.008+0000 7f1c63293700 1 --2- 172.21.15.59:0/3146379149 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f1c4006b110 0x7f1c4006d5d0 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:20.011 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:20.008+0000 7f1c63293700 1 --2- 172.21.15.59:0/3146379149 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1c5c1440f0 0x7f1c5c157c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:20.011 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:20.008+0000 7f1c63293700 1 --2- 172.21.15.59:0/3146379149 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f1c5c0722c0 0x7f1c5c13e6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:20.011 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:20.008+0000 7f1c63293700 1 --2- 172.21.15.59:0/3146379149 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1c5c143ae0 0x7f1c5c1576c0 unknown :-1 s=CLOSED pgs=597 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:20.011 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:20.008+0000 7f1c63293700 1 -- 172.21.15.59:0/3146379149 >> 172.21.15.59:0/3146379149 conn(0x7f1c5c0b2740 msgr2=0x7f1c5c06afd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:58:20.011 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:20.008+0000 7f1c63293700 1 -- 172.21.15.59:0/3146379149 shutdown_connections 2023-11-24T07:58:20.011 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:20.009+0000 7f1c63293700 1 -- 172.21.15.59:0/3146379149 wait complete. 2023-11-24T07:58:20.022 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-24T07:58:20.150 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:20.148+0000 7f8771144700 1 Processor -- start 2023-11-24T07:58:20.150 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:20.148+0000 7f8771144700 1 -- start start 2023-11-24T07:58:20.153 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:20.149+0000 7f8771144700 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f876c1440f0 0x7f876c0b51b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:20.153 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:20.149+0000 7f8771144700 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f876c143ec0 con 0x7f876c143ae0 2023-11-24T07:58:20.153 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:20.149+0000 7f8771144700 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f876c0b3ae0 con 0x7f876c0722c0 2023-11-24T07:58:20.153 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:20.149+0000 7f8771144700 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f876c0b40a0 con 0x7f876c1440f0 2023-11-24T07:58:20.153 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:20.149+0000 7f876ad9d700 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7f876c143ae0 0x7f876c071bb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:56632/0 (socket says 172.21.15.59:56632) 2023-11-24T07:58:20.153 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:20.149+0000 7f876ad9d700 1 -- 172.21.15.59:0/3315343929 learned_addr learned my addr 172.21.15.59:0/3315343929 (peer_addr_for_me v1:172.21.15.59:0/0) 2023-11-24T07:58:20.153 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:20.150+0000 7f876b59e700 1 --2- 172.21.15.59:0/3315343929 >> v2:172.21.15.76:3300/0 conn(0x7f876c1440f0 0x7f876c0b51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:20.153 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:20.150+0000 7f8769d9b700 1 -- 172.21.15.59:0/3315343929 <== mon.0 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1554873273 0 0) 0x7f876c143ec0 con 0x7f876c143ae0 2023-11-24T07:58:20.153 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:20.150+0000 7f8769d9b700 1 -- 172.21.15.59:0/3315343929 --> v1:172.21.15.59:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f87540035b0 con 0x7f876c143ae0 2023-11-24T07:58:20.153 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:20.150+0000 7f8769d9b700 1 -- 172.21.15.59:0/3315343929 <== mon.0 v1:172.21.15.59:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2306106131 0 0) 0x7f87540035b0 con 0x7f876c143ae0 2023-11-24T07:58:20.154 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:20.150+0000 7f876b59e700 1 -- 172.21.15.59:0/3315343929 >> v1:172.21.15.76:6789/0 conn(0x7f876c0722c0 legacy=0x7f876c06e510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:20.154 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:20.150+0000 7f876b59e700 1 -- 172.21.15.59:0/3315343929 >> v1:172.21.15.59:6789/0 conn(0x7f876c143ae0 legacy=0x7f876c071bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:20.154 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:20.150+0000 7f876b59e700 1 -- 172.21.15.59:0/3315343929 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f876c0b3dc0 con 0x7f876c1440f0 2023-11-24T07:58:20.154 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:20.150+0000 7f876b59e700 1 --2- 172.21.15.59:0/3315343929 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f876c1440f0 0x7f876c0b51b0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f8764009d90 tx=0x7f876400b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=631cda8b6a2dfb16 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:58:20.154 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:20.150+0000 7f8769d9b700 1 -- 172.21.15.59:0/3315343929 <== mon.1 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f876400e070 con 0x7f876c1440f0 2023-11-24T07:58:20.154 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:20.150+0000 7f8769d9b700 1 -- 172.21.15.59:0/3315343929 <== mon.1 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8764003a20 con 0x7f876c1440f0 2023-11-24T07:58:20.154 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:20.150+0000 7f8769d9b700 1 -- 172.21.15.59:0/3315343929 <== mon.1 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8760001670 con 0x7f876c1440f0 2023-11-24T07:58:20.154 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:20.151+0000 7f8771144700 1 -- 172.21.15.59:0/3315343929 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f876c1440f0 msgr2=0x7f876c0b51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:20.154 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:20.151+0000 7f8771144700 1 --2- 172.21.15.59:0/3315343929 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f876c1440f0 0x7f876c0b51b0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f8764009d90 tx=0x7f876400b330 comp rx=0 tx=0).stop 2023-11-24T07:58:20.154 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:20.151+0000 7f8771144700 1 -- 172.21.15.59:0/3315343929 shutdown_connections 2023-11-24T07:58:20.154 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:20.151+0000 7f8771144700 1 --2- 172.21.15.59:0/3315343929 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f876c1440f0 0x7f876c0b51b0 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:20.154 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:20.151+0000 7f8771144700 1 -- 172.21.15.59:0/3315343929 >> 172.21.15.59:0/3315343929 conn(0x7f876c0b2740 msgr2=0x7f876c0b2b20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:58:20.155 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:20.151+0000 7f8771144700 1 -- 172.21.15.59:0/3315343929 shutdown_connections 2023-11-24T07:58:20.155 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:20.151+0000 7f8771144700 1 -- 172.21.15.59:0/3315343929 wait complete. 2023-11-24T07:58:20.157 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:20.152+0000 7f8771144700 1 Processor -- start 2023-11-24T07:58:20.157 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:20.152+0000 7f8771144700 1 -- start start 2023-11-24T07:58:20.157 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:20.153+0000 7f8771144700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f876c0722c0 0x7f876c15d830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:20.157 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:20.153+0000 7f8771144700 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f876c143ae0 0x7f876c1f4580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:20.157 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:20.153+0000 7f8771144700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f876c1440f0 0x7f876c13ff10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:20.158 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:20.153+0000 7f8771144700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f876c0a9750 con 0x7f876c1440f0 2023-11-24T07:58:20.158 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:20.153+0000 7f8771144700 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f876c1f4ac0 con 0x7f876c0722c0 2023-11-24T07:58:20.158 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:20.153+0000 7f8771144700 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f876c15c790 con 0x7f876c143ae0 2023-11-24T07:58:20.158 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:20.153+0000 7f876b59e700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f876c1440f0 0x7f876c13ff10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:20.158 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:20.153+0000 7f876b59e700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f876c1440f0 0x7f876c13ff10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:53168/0 (socket says 172.21.15.59:53168) 2023-11-24T07:58:20.158 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:20.153+0000 7f876b59e700 1 -- 172.21.15.59:0/285676188 learned_addr learned my addr 172.21.15.59:0/285676188 (peer_addr_for_me v2:172.21.15.59:0/0) 2023-11-24T07:58:20.158 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:20.153+0000 7f876ad9d700 1 --2- 172.21.15.59:0/285676188 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f876c0722c0 0x7f876c15d830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:20.158 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:20.153+0000 7f876a59c700 1 --2- 172.21.15.59:0/285676188 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f876c143ae0 0x7f876c1f4580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:20.158 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:20.154+0000 7f876b59e700 1 -- 172.21.15.59:0/285676188 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f876c143ae0 msgr2=0x7f876c1f4580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:20.158 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:20.154+0000 7f876b59e700 1 --2- 172.21.15.59:0/285676188 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f876c143ae0 0x7f876c1f4580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:20.158 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:20.154+0000 7f876b59e700 1 -- 172.21.15.59:0/285676188 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f876c0722c0 msgr2=0x7f876c15d830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:20.158 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:20.154+0000 7f876b59e700 1 --2- 172.21.15.59:0/285676188 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f876c0722c0 0x7f876c15d830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:20.159 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:20.154+0000 7f876b59e700 1 -- 172.21.15.59:0/285676188 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8764009a70 con 0x7f876c1440f0 2023-11-24T07:58:20.159 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:20.154+0000 7f876ad9d700 1 --2- 172.21.15.59:0/285676188 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f876c0722c0 0x7f876c15d830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T07:58:20.159 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:20.154+0000 7f876a59c700 1 --2- 172.21.15.59:0/285676188 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f876c143ae0 0x7f876c1f4580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T07:58:20.159 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:20.154+0000 7f876b59e700 1 --2- 172.21.15.59:0/285676188 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f876c1440f0 0x7f876c13ff10 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7f876400b370 tx=0x7f8764004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:58:20.159 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:20.154+0000 7f87537fe700 1 -- 172.21.15.59:0/285676188 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8764009450 con 0x7f876c1440f0 2023-11-24T07:58:20.159 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:20.154+0000 7f87537fe700 1 -- 172.21.15.59:0/285676188 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8764004650 con 0x7f876c1440f0 2023-11-24T07:58:20.159 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:20.154+0000 7f8771144700 1 -- 172.21.15.59:0/285676188 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f876c15ca70 con 0x7f876c1440f0 2023-11-24T07:58:20.159 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:20.154+0000 7f87537fe700 1 -- 172.21.15.59:0/285676188 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8764012450 con 0x7f876c1440f0 2023-11-24T07:58:20.159 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:20.154+0000 7f8771144700 1 -- 172.21.15.59:0/285676188 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f876c15d030 con 0x7f876c1440f0 2023-11-24T07:58:20.159 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:20.155+0000 7f8771144700 1 -- 172.21.15.59:0/285676188 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f876c075480 con 0x7f876c1440f0 2023-11-24T07:58:20.159 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:20.155+0000 7f87537fe700 1 -- 172.21.15.59:0/285676188 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 8) v1 ==== 89435+0+0 (secure 0 0 0) 0x7f87640125b0 con 0x7f876c1440f0 2023-11-24T07:58:20.159 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:20.156+0000 7f87537fe700 1 --2- 172.21.15.59:0/285676188 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f875806b110 0x7f875806d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:20.159 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:20.156+0000 7f87537fe700 1 -- 172.21.15.59:0/285676188 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5854+0+0 (secure 0 0 0) 0x7f876408b4b0 con 0x7f876c1440f0 2023-11-24T07:58:20.160 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:20.156+0000 7f876ad9d700 1 --2- 172.21.15.59:0/285676188 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f875806b110 0x7f875806d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:20.160 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:20.156+0000 7f876ad9d700 1 --2- 172.21.15.59:0/285676188 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f875806b110 0x7f875806d5d0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f8760000f60 tx=0x7f8760009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:58:20.162 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:20.160+0000 7f87537fe700 1 -- 172.21.15.59:0/285676188 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f8764059c30 con 0x7f876c1440f0 2023-11-24T07:58:20.367 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:20.365+0000 7f8771144700 1 -- 172.21.15.59:0/285676188 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f876c0b5d20 con 0x7f876c1440f0 2023-11-24T07:58:20.367 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:20.365+0000 7f87537fe700 1 -- 172.21.15.59:0/285676188 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v47) v1 ==== 100+0+11 (secure 0 0 0) 0x7f87640597c0 con 0x7f876c1440f0 2023-11-24T07:58:20.367 INFO:teuthology.orchestra.run.smithi059.stdout:pg_num: 64 2023-11-24T07:58:20.369 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:20.367+0000 7f8771144700 1 -- 172.21.15.59:0/285676188 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f875806b110 msgr2=0x7f875806d5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:20.369 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:20.367+0000 7f8771144700 1 --2- 172.21.15.59:0/285676188 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f875806b110 0x7f875806d5d0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f8760000f60 tx=0x7f8760009040 comp rx=0 tx=0).stop 2023-11-24T07:58:20.369 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:20.367+0000 7f8771144700 1 -- 172.21.15.59:0/285676188 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f876c1440f0 msgr2=0x7f876c13ff10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:20.369 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:20.367+0000 7f8771144700 1 --2- 172.21.15.59:0/285676188 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f876c1440f0 0x7f876c13ff10 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7f876400b370 tx=0x7f8764004ad0 comp rx=0 tx=0).stop 2023-11-24T07:58:20.369 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:20.367+0000 7f8771144700 1 -- 172.21.15.59:0/285676188 shutdown_connections 2023-11-24T07:58:20.369 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:20.367+0000 7f8771144700 1 --2- 172.21.15.59:0/285676188 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f875806b110 0x7f875806d5d0 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:20.370 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:20.367+0000 7f8771144700 1 --2- 172.21.15.59:0/285676188 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f876c1440f0 0x7f876c13ff10 unknown :-1 s=CLOSED pgs=599 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:20.370 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:20.367+0000 7f8771144700 1 --2- 172.21.15.59:0/285676188 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f876c0722c0 0x7f876c15d830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:20.370 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:20.367+0000 7f8771144700 1 --2- 172.21.15.59:0/285676188 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f876c143ae0 0x7f876c1f4580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:20.370 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:20.367+0000 7f8771144700 1 -- 172.21.15.59:0/285676188 >> 172.21.15.59:0/285676188 conn(0x7f876c0b2740 msgr2=0x7f876c141160 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:58:20.370 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:20.367+0000 7f8771144700 1 -- 172.21.15.59:0/285676188 shutdown_connections 2023-11-24T07:58:20.370 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:20.367+0000 7f8771144700 1 -- 172.21.15.59:0/285676188 wait complete. 2023-11-24T07:58:20.382 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-11-24T07:58:20.508 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:20.506+0000 7f120536d700 1 Processor -- start 2023-11-24T07:58:20.508 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:20.506+0000 7f120536d700 1 -- start start 2023-11-24T07:58:20.514 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:20.506+0000 7f120536d700 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f12000d16e0 0x7f12000affb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:20.515 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:20.506+0000 7f120536d700 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f12000d1b40 con 0x7f12000ada10 2023-11-24T07:58:20.515 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:20.506+0000 7f120536d700 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1200070490 con 0x7f120006afd0 2023-11-24T07:58:20.515 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:20.506+0000 7f120536d700 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f12000b04f0 con 0x7f12000d16e0 2023-11-24T07:58:20.515 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:20.507+0000 7f11ff7fe700 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7f12000ada10 0x7f12000adea0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:56650/0 (socket says 172.21.15.59:56650) 2023-11-24T07:58:20.515 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:20.507+0000 7f11ff7fe700 1 -- 172.21.15.59:0/427651972 learned_addr learned my addr 172.21.15.59:0/427651972 (peer_addr_for_me v1:172.21.15.59:0/0) 2023-11-24T07:58:20.515 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:20.507+0000 7f11fdffb700 1 -- 172.21.15.59:0/427651972 <== mon.0 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 94082889 0 0) 0x7f12000d1b40 con 0x7f12000ada10 2023-11-24T07:58:20.515 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:20.507+0000 7f11fdffb700 1 -- 172.21.15.59:0/427651972 --> v1:172.21.15.59:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f11ec0035b0 con 0x7f12000ada10 2023-11-24T07:58:20.515 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:20.507+0000 7f11fe7fc700 1 --2- 172.21.15.59:0/427651972 >> v2:172.21.15.76:3300/0 conn(0x7f12000d16e0 0x7f12000affb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:20.515 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:20.507+0000 7f11fdffb700 1 -- 172.21.15.59:0/427651972 <== mon.0 v1:172.21.15.59:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 556473904 0 0) 0x7f11ec0035b0 con 0x7f12000ada10 2023-11-24T07:58:20.515 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:20.507+0000 7f11fdffb700 1 -- 172.21.15.59:0/427651972 >> v2:172.21.15.76:3300/0 conn(0x7f12000d16e0 msgr2=0x7f12000affb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T07:58:20.515 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:20.507+0000 7f11fdffb700 1 --2- 172.21.15.59:0/427651972 >> v2:172.21.15.76:3300/0 conn(0x7f12000d16e0 0x7f12000affb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:20.516 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:20.507+0000 7f11fdffb700 1 -- 172.21.15.59:0/427651972 >> v1:172.21.15.76:6789/0 conn(0x7f120006afd0 legacy=0x7f120006b3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:20.516 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:20.507+0000 7f11fdffb700 1 -- 172.21.15.59:0/427651972 --> v1:172.21.15.59:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f12000af9f0 con 0x7f12000ada10 2023-11-24T07:58:20.516 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:20.507+0000 7f11fdffb700 1 -- 172.21.15.59:0/427651972 <== mon.0 v1:172.21.15.59:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1312333566 0 0) 0x7f11f80019f0 con 0x7f12000ada10 2023-11-24T07:58:20.516 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:20.507+0000 7f11fdffb700 1 -- 172.21.15.59:0/427651972 >> v1:172.21.15.59:6789/0 conn(0x7f12000ada10 legacy=0x7f12000adea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:20.516 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:20.507+0000 7f11fdffb700 1 --2- 172.21.15.59:0/427651972 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f11ec003a00 0x7f11ec003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:20.516 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:20.507+0000 7f11fdffb700 1 --2- 172.21.15.59:0/427651972 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f11ec004850 0x7f11ec004cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:20.516 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:20.507+0000 7f11fdffb700 1 --2- 172.21.15.59:0/427651972 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f11ec005210 0x7f11ec00ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:20.516 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:20.507+0000 7f11fdffb700 1 -- 172.21.15.59:0/427651972 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f12000b04f0 con 0x7f11ec005210 2023-11-24T07:58:20.516 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:20.507+0000 7f11fdffb700 1 -- 172.21.15.59:0/427651972 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11ec00c0b0 con 0x7f11ec004850 2023-11-24T07:58:20.516 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:20.507+0000 7f11fdffb700 1 -- 172.21.15.59:0/427651972 --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f11ec00c270 con 0x7f11ec003a00 2023-11-24T07:58:20.516 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:20.507+0000 7f11ff7fe700 1 --2- 172.21.15.59:0/427651972 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f11ec005210 0x7f11ec00ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:20.516 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:20.507+0000 7f11fe7fc700 1 --2- 172.21.15.59:0/427651972 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f11ec004850 0x7f11ec004cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:20.517 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:20.507+0000 7f11feffd700 1 --2- 172.21.15.59:0/427651972 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f11ec003a00 0x7f11ec003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:20.517 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:20.508+0000 7f11ff7fe700 1 -- 172.21.15.59:0/427651972 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f11ec003a00 msgr2=0x7f11ec003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T07:58:20.517 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:20.508+0000 7f11ff7fe700 1 --2- 172.21.15.59:0/427651972 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f11ec003a00 0x7f11ec003e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:20.517 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:20.508+0000 7f11ff7fe700 1 -- 172.21.15.59:0/427651972 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f11ec004850 msgr2=0x7f11ec004cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T07:58:20.517 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:20.508+0000 7f11ff7fe700 1 --2- 172.21.15.59:0/427651972 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f11ec004850 0x7f11ec004cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:20.517 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:20.508+0000 7f11ff7fe700 1 -- 172.21.15.59:0/427651972 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f11ec0035b0 con 0x7f11ec005210 2023-11-24T07:58:20.518 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:20.508+0000 7f11feffd700 1 --2- 172.21.15.59:0/427651972 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f11ec003a00 0x7f11ec003e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-24T07:58:20.518 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:20.508+0000 7f11ff7fe700 1 -- 172.21.15.59:0/427651972 reap_dead start 2023-11-24T07:58:20.518 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:20.508+0000 7f11ff7fe700 1 --2- 172.21.15.59:0/427651972 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f11ec005210 0x7f11ec00ba10 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7f11f800a830 tx=0x7f11ec005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9bdf56acd7ec0070 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:58:20.518 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:20.508+0000 7f11fdffb700 1 -- 172.21.15.59:0/427651972 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11f800a2f0 con 0x7f11ec005210 2023-11-24T07:58:20.518 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:20.508+0000 7f11fdffb700 1 -- 172.21.15.59:0/427651972 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f11f8012620 con 0x7f11ec005210 2023-11-24T07:58:20.518 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:20.508+0000 7f11fdffb700 1 -- 172.21.15.59:0/427651972 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11f80129d0 con 0x7f11ec005210 2023-11-24T07:58:20.518 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:20.508+0000 7f120536d700 1 -- 172.21.15.59:0/427651972 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f11ec005210 msgr2=0x7f11ec00ba10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:20.518 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:20.508+0000 7f120536d700 1 --2- 172.21.15.59:0/427651972 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f11ec005210 0x7f11ec00ba10 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7f11f800a830 tx=0x7f11ec005880 comp rx=0 tx=0).stop 2023-11-24T07:58:20.518 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:20.509+0000 7f120536d700 1 -- 172.21.15.59:0/427651972 shutdown_connections 2023-11-24T07:58:20.518 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:20.509+0000 7f120536d700 1 --2- 172.21.15.59:0/427651972 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f11ec005210 0x7f11ec00ba10 unknown :-1 s=CLOSED pgs=601 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:20.518 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:20.509+0000 7f120536d700 1 -- 172.21.15.59:0/427651972 >> 172.21.15.59:0/427651972 conn(0x7f12000d0bc0 msgr2=0x7f12000d0fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:58:20.518 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:20.509+0000 7f120536d700 1 -- 172.21.15.59:0/427651972 shutdown_connections 2023-11-24T07:58:20.519 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:20.509+0000 7f120536d700 1 -- 172.21.15.59:0/427651972 wait complete. 2023-11-24T07:58:20.519 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:20.510+0000 7f120536d700 1 Processor -- start 2023-11-24T07:58:20.519 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:20.510+0000 7f120536d700 1 -- start start 2023-11-24T07:58:20.519 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:20.511+0000 7f120536d700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f11ec005210 0x7f12001431c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:20.519 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:20.511+0000 7f120536d700 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f1200149af0 0x7f12001581b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:20.519 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:20.511+0000 7f120536d700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f12001586f0 0x7f1200158b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:20.519 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:20.511+0000 7f120536d700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1200143700 con 0x7f11ec005210 2023-11-24T07:58:20.519 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:20.511+0000 7f120536d700 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f12000d09a0 con 0x7f12001586f0 2023-11-24T07:58:20.519 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:20.511+0000 7f120536d700 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f12000d06c0 con 0x7f1200149af0 2023-11-24T07:58:20.519 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:20.511+0000 7f11feffd700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f11ec005210 0x7f12001431c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:20.519 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:20.511+0000 7f11feffd700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f11ec005210 0x7f12001431c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:53184/0 (socket says 172.21.15.59:53184) 2023-11-24T07:58:20.520 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:20.511+0000 7f11feffd700 1 -- 172.21.15.59:0/3294987470 learned_addr learned my addr 172.21.15.59:0/3294987470 (peer_addr_for_me v2:172.21.15.59:0/0) 2023-11-24T07:58:20.520 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:20.511+0000 7f11fe7fc700 1 --2- 172.21.15.59:0/3294987470 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f1200149af0 0x7f12001581b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:20.520 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:20.511+0000 7f11ff7fe700 1 --2- 172.21.15.59:0/3294987470 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f12001586f0 0x7f1200158b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:20.520 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:20.511+0000 7f11feffd700 1 -- 172.21.15.59:0/3294987470 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f1200149af0 msgr2=0x7f12001581b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:20.520 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:20.511+0000 7f11feffd700 1 --2- 172.21.15.59:0/3294987470 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f1200149af0 0x7f12001581b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:20.520 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:20.511+0000 7f11feffd700 1 -- 172.21.15.59:0/3294987470 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f12001586f0 msgr2=0x7f1200158b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:20.520 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:20.511+0000 7f11feffd700 1 --2- 172.21.15.59:0/3294987470 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f12001586f0 0x7f1200158b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:20.520 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:20.511+0000 7f11feffd700 1 -- 172.21.15.59:0/3294987470 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1200143a80 con 0x7f11ec005210 2023-11-24T07:58:20.520 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:20.511+0000 7f11feffd700 1 --2- 172.21.15.59:0/3294987470 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f11ec005210 0x7f12001431c0 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7f11f000e550 tx=0x7f11f000e860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:58:20.520 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:20.511+0000 7f11f77fe700 1 -- 172.21.15.59:0/3294987470 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11f000ee70 con 0x7f11ec005210 2023-11-24T07:58:20.520 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:20.512+0000 7f11f77fe700 1 -- 172.21.15.59:0/3294987470 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f11f00052e0 con 0x7f11ec005210 2023-11-24T07:58:20.520 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:20.512+0000 7f120536d700 1 -- 172.21.15.59:0/3294987470 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1200143d60 con 0x7f11ec005210 2023-11-24T07:58:20.520 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:20.512+0000 7f11f77fe700 1 -- 172.21.15.59:0/3294987470 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11f0003680 con 0x7f11ec005210 2023-11-24T07:58:20.521 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:20.512+0000 7f120536d700 1 -- 172.21.15.59:0/3294987470 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f12000d0360 con 0x7f11ec005210 2023-11-24T07:58:20.521 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:20.513+0000 7f120536d700 1 -- 172.21.15.59:0/3294987470 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f12001443b0 con 0x7f11ec005210 2023-11-24T07:58:20.521 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:20.513+0000 7f11f77fe700 1 -- 172.21.15.59:0/3294987470 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 8) v1 ==== 89435+0+0 (secure 0 0 0) 0x7f11f0003800 con 0x7f11ec005210 2023-11-24T07:58:20.522 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:20.513+0000 7f11f77fe700 1 --2- 172.21.15.59:0/3294987470 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f11e006b140 0x7f11e006d600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:20.522 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:20.514+0000 7f11f77fe700 1 -- 172.21.15.59:0/3294987470 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5854+0+0 (secure 0 0 0) 0x7f11f0088fb0 con 0x7f11ec005210 2023-11-24T07:58:20.522 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:20.514+0000 7f11fe7fc700 1 --2- 172.21.15.59:0/3294987470 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f11e006b140 0x7f11e006d600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:20.522 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:20.514+0000 7f11fe7fc700 1 --2- 172.21.15.59:0/3294987470 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f11e006b140 0x7f11e006d600 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f12000b3fd0 tx=0x7f11e8009210 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:58:20.522 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:20.518+0000 7f11f77fe700 1 -- 172.21.15.59:0/3294987470 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f11f00bbe90 con 0x7f11ec005210 2023-11-24T07:58:20.740 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:20.738+0000 7f120536d700 1 -- 172.21.15.59:0/3294987470 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f1200068670 con 0x7f11ec005210 2023-11-24T07:58:20.740 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:20.739+0000 7f11f77fe700 1 -- 172.21.15.59:0/3294987470 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v47) v1 ==== 80+0+0 (secure 0 0 0) 0x7f11f0019070 con 0x7f11ec005210 2023-11-24T07:58:20.743 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:20.741+0000 7f120536d700 1 -- 172.21.15.59:0/3294987470 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f11e006b140 msgr2=0x7f11e006d600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:20.743 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:20.741+0000 7f120536d700 1 --2- 172.21.15.59:0/3294987470 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f11e006b140 0x7f11e006d600 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f12000b3fd0 tx=0x7f11e8009210 comp rx=0 tx=0).stop 2023-11-24T07:58:20.743 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:20.741+0000 7f120536d700 1 -- 172.21.15.59:0/3294987470 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f11ec005210 msgr2=0x7f12001431c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:20.743 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:20.741+0000 7f120536d700 1 --2- 172.21.15.59:0/3294987470 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f11ec005210 0x7f12001431c0 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7f11f000e550 tx=0x7f11f000e860 comp rx=0 tx=0).stop 2023-11-24T07:58:20.743 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:20.741+0000 7f120536d700 1 -- 172.21.15.59:0/3294987470 shutdown_connections 2023-11-24T07:58:20.743 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:20.741+0000 7f120536d700 1 --2- 172.21.15.59:0/3294987470 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f11e006b140 0x7f11e006d600 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:20.743 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:20.741+0000 7f120536d700 1 --2- 172.21.15.59:0/3294987470 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f12001586f0 0x7f1200158b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:20.744 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:20.741+0000 7f120536d700 1 --2- 172.21.15.59:0/3294987470 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f11ec005210 0x7f12001431c0 unknown :-1 s=CLOSED pgs=602 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:20.744 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:20.741+0000 7f120536d700 1 --2- 172.21.15.59:0/3294987470 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f1200149af0 0x7f12001581b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:20.744 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:20.741+0000 7f120536d700 1 -- 172.21.15.59:0/3294987470 >> 172.21.15.59:0/3294987470 conn(0x7f12000d0bc0 msgr2=0x7f120015d190 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:58:20.744 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:20.741+0000 7f120536d700 1 -- 172.21.15.59:0/3294987470 shutdown_connections 2023-11-24T07:58:20.744 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:20.741+0000 7f120536d700 1 -- 172.21.15.59:0/3294987470 wait complete. 2023-11-24T07:58:20.744 INFO:teuthology.orchestra.run.smithi059.stderr:listed 0 entries 2023-11-24T07:58:20.755 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2023-11-24T07:58:20.755 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T07:58:20.755 DEBUG:teuthology.orchestra.run.smithi059:> sudo umount /home/ubuntu/cephtest/mnt.0 2023-11-24T07:58:20.801 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi059.front.sepia.ceph.com 2023-11-24T07:58:20.801 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T07:58:20.801 DEBUG:teuthology.orchestra.run.smithi059:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-11-24T07:58:20.857 DEBUG:teuthology.orchestra.run.smithi076:> set -ex 2023-11-24T07:58:20.857 DEBUG:teuthology.orchestra.run.smithi076:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-24T07:58:20.879 DEBUG:teuthology.orchestra.run.smithi076:> set -ex 2023-11-24T07:58:20.879 DEBUG:teuthology.orchestra.run.smithi076:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-24T07:58:20.939 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-24T07:58:21.065 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:21.063+0000 7fb1bab5d700 1 Processor -- start 2023-11-24T07:58:21.065 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:21.063+0000 7fb1bab5d700 1 -- start start 2023-11-24T07:58:21.069 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:21.064+0000 7fb1bab5d700 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fb1b40b3b40 0x7fb1b40b5020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:21.069 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:21.064+0000 7fb1bab5d700 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb1b406ef90 con 0x7fb1b40b4c40 2023-11-24T07:58:21.069 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:21.064+0000 7fb1bab5d700 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb1b40b5560 con 0x7fb1b406f420 2023-11-24T07:58:21.069 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:21.064+0000 7fb1bab5d700 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb1b406e380 con 0x7fb1b40b3b40 2023-11-24T07:58:21.069 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:21.064+0000 7fb1b88f9700 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7fb1b40b4c40 0x7fb1b406dba0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:56668/0 (socket says 172.21.15.59:56668) 2023-11-24T07:58:21.069 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:21.064+0000 7fb1b88f9700 1 -- 172.21.15.59:0/755233756 learned_addr learned my addr 172.21.15.59:0/755233756 (peer_addr_for_me v1:172.21.15.59:0/0) 2023-11-24T07:58:21.069 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:21.064+0000 7fb1b3fff700 1 --2- 172.21.15.59:0/755233756 >> v2:172.21.15.76:3300/0 conn(0x7fb1b40b3b40 0x7fb1b40b5020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:21.069 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:21.064+0000 7fb1b37fe700 1 -- 172.21.15.59:0/755233756 <== mon.0 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 269377877 0 0) 0x7fb1b406ef90 con 0x7fb1b40b4c40 2023-11-24T07:58:21.069 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:21.064+0000 7fb1b37fe700 1 -- 172.21.15.59:0/755233756 --> v1:172.21.15.59:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb1a40035b0 con 0x7fb1b40b4c40 2023-11-24T07:58:21.070 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:21.064+0000 7fb1b37fe700 1 -- 172.21.15.59:0/755233756 <== mon.1 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2917133654 0 0) 0x7fb1b40b5560 con 0x7fb1b406f420 2023-11-24T07:58:21.070 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:21.064+0000 7fb1b37fe700 1 -- 172.21.15.59:0/755233756 --> v1:172.21.15.76:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb1b406ef90 con 0x7fb1b406f420 2023-11-24T07:58:21.070 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:21.064+0000 7fb1b37fe700 1 -- 172.21.15.59:0/755233756 <== mon.0 v1:172.21.15.59:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2285155741 0 0) 0x7fb1a40035b0 con 0x7fb1b40b4c40 2023-11-24T07:58:21.070 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:21.064+0000 7fb1b37fe700 1 -- 172.21.15.59:0/755233756 >> v2:172.21.15.76:3300/0 conn(0x7fb1b40b3b40 msgr2=0x7fb1b40b5020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T07:58:21.070 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:21.064+0000 7fb1b37fe700 1 --2- 172.21.15.59:0/755233756 >> v2:172.21.15.76:3300/0 conn(0x7fb1b40b3b40 0x7fb1b40b5020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:21.070 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:21.064+0000 7fb1b37fe700 1 -- 172.21.15.59:0/755233756 >> v1:172.21.15.76:6789/0 conn(0x7fb1b406f420 legacy=0x7fb1b40ce1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:21.070 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:21.064+0000 7fb1b37fe700 1 -- 172.21.15.59:0/755233756 --> v1:172.21.15.59:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb1b406e660 con 0x7fb1b40b4c40 2023-11-24T07:58:21.070 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:21.064+0000 7fb1b37fe700 1 -- 172.21.15.59:0/755233756 <== mon.0 v1:172.21.15.59:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1312333566 0 0) 0x7fb1a8001670 con 0x7fb1b40b4c40 2023-11-24T07:58:21.070 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:21.064+0000 7fb1b37fe700 1 -- 172.21.15.59:0/755233756 >> v1:172.21.15.59:6789/0 conn(0x7fb1b40b4c40 legacy=0x7fb1b406dba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:21.070 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:21.064+0000 7fb1b37fe700 1 --2- 172.21.15.59:0/755233756 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb1a4003be0 0x7fb1a4003fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:21.070 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:21.064+0000 7fb1b37fe700 1 --2- 172.21.15.59:0/755233756 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fb1a4004850 0x7fb1a4004cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:21.070 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:21.064+0000 7fb1b37fe700 1 --2- 172.21.15.59:0/755233756 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb1a4005210 0x7fb1a400ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:21.071 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:21.064+0000 7fb1b37fe700 1 -- 172.21.15.59:0/755233756 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1b406e380 con 0x7fb1a4005210 2023-11-24T07:58:21.071 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:21.064+0000 7fb1b37fe700 1 -- 172.21.15.59:0/755233756 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1a400c0e0 con 0x7fb1a4003be0 2023-11-24T07:58:21.071 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:21.064+0000 7fb1b37fe700 1 -- 172.21.15.59:0/755233756 --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb1a400c2a0 con 0x7fb1a4004850 2023-11-24T07:58:21.071 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:21.065+0000 7fb1b90fa700 1 --2- 172.21.15.59:0/755233756 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb1a4005210 0x7fb1a400ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:21.071 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:21.065+0000 7fb1b88f9700 1 --2- 172.21.15.59:0/755233756 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb1a4003be0 0x7fb1a4003fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:21.071 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:21.065+0000 7fb1b3fff700 1 --2- 172.21.15.59:0/755233756 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fb1a4004850 0x7fb1a4004cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:21.071 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:21.065+0000 7fb1b90fa700 1 -- 172.21.15.59:0/755233756 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fb1a4004850 msgr2=0x7fb1a4004cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T07:58:21.071 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:21.065+0000 7fb1b90fa700 1 --2- 172.21.15.59:0/755233756 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fb1a4004850 0x7fb1a4004cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:21.071 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:21.065+0000 7fb1b90fa700 1 -- 172.21.15.59:0/755233756 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb1a4003be0 msgr2=0x7fb1a4003fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T07:58:21.072 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:21.065+0000 7fb1b90fa700 1 --2- 172.21.15.59:0/755233756 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb1a4003be0 0x7fb1a4003fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:21.072 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:21.065+0000 7fb1b90fa700 1 -- 172.21.15.59:0/755233756 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb1a40035b0 con 0x7fb1a4005210 2023-11-24T07:58:21.072 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:21.065+0000 7fb1b88f9700 1 --2- 172.21.15.59:0/755233756 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb1a4003be0 0x7fb1a4003fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-24T07:58:21.072 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:21.065+0000 7fb1b90fa700 1 -- 172.21.15.59:0/755233756 reap_dead start 2023-11-24T07:58:21.072 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:21.065+0000 7fb1b90fa700 1 --2- 172.21.15.59:0/755233756 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb1a4005210 0x7fb1a400ba10 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7fb1ac00c2f0 tx=0x7fb1a4005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=24b66bb275d8220e server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:58:21.072 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:21.065+0000 7fb1b37fe700 1 -- 172.21.15.59:0/755233756 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1ac001f30 con 0x7fb1a4005210 2023-11-24T07:58:21.072 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:21.065+0000 7fb1b37fe700 1 -- 172.21.15.59:0/755233756 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb1ac003a00 con 0x7fb1a4005210 2023-11-24T07:58:21.072 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:21.065+0000 7fb1b37fe700 1 -- 172.21.15.59:0/755233756 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1ac003db0 con 0x7fb1a4005210 2023-11-24T07:58:21.072 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:21.066+0000 7fb1bab5d700 1 -- 172.21.15.59:0/755233756 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb1a4005210 msgr2=0x7fb1a400ba10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:21.072 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:21.066+0000 7fb1bab5d700 1 --2- 172.21.15.59:0/755233756 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb1a4005210 0x7fb1a400ba10 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7fb1ac00c2f0 tx=0x7fb1a4005880 comp rx=0 tx=0).stop 2023-11-24T07:58:21.072 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:21.066+0000 7fb1bab5d700 1 -- 172.21.15.59:0/755233756 shutdown_connections 2023-11-24T07:58:21.072 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:21.066+0000 7fb1bab5d700 1 --2- 172.21.15.59:0/755233756 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb1a4005210 0x7fb1a400ba10 unknown :-1 s=CLOSED pgs=604 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:21.073 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:21.066+0000 7fb1bab5d700 1 -- 172.21.15.59:0/755233756 >> 172.21.15.59:0/755233756 conn(0x7fb1b40b2840 msgr2=0x7fb1b40b2c20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:58:21.073 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:21.066+0000 7fb1bab5d700 1 -- 172.21.15.59:0/755233756 shutdown_connections 2023-11-24T07:58:21.073 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:21.066+0000 7fb1bab5d700 1 -- 172.21.15.59:0/755233756 wait complete. 2023-11-24T07:58:21.073 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:21.067+0000 7fb1bab5d700 1 Processor -- start 2023-11-24T07:58:21.073 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:21.067+0000 7fb1bab5d700 1 -- start start 2023-11-24T07:58:21.076 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:21.068+0000 7fb1bab5d700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb1a4005210 0x7fb1b412bcc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:21.076 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:21.069+0000 7fb1bab5d700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb1b413fdd0 0x7fb1b41eb180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:21.076 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:21.069+0000 7fb1bab5d700 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fb1b41eb6c0 0x7fb1b412adc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:21.076 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:21.069+0000 7fb1bab5d700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1b412c200 con 0x7fb1b413fdd0 2023-11-24T07:58:21.076 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:21.069+0000 7fb1bab5d700 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1b4129d20 con 0x7fb1a4005210 2023-11-24T07:58:21.076 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:21.069+0000 7fb1bab5d700 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb1b412a000 con 0x7fb1b41eb6c0 2023-11-24T07:58:21.076 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:21.069+0000 7fb1b3fff700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb1b413fdd0 0x7fb1b41eb180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:21.076 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:21.069+0000 7fb1b3fff700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb1b413fdd0 0x7fb1b41eb180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:53208/0 (socket says 172.21.15.59:53208) 2023-11-24T07:58:21.076 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:21.069+0000 7fb1b3fff700 1 -- 172.21.15.59:0/2198453024 learned_addr learned my addr 172.21.15.59:0/2198453024 (peer_addr_for_me v2:172.21.15.59:0/0) 2023-11-24T07:58:21.077 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:21.069+0000 7fb1b88f9700 1 --2- 172.21.15.59:0/2198453024 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb1a4005210 0x7fb1b412bcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:21.077 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:21.069+0000 7fb1b90fa700 1 --2- 172.21.15.59:0/2198453024 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fb1b41eb6c0 0x7fb1b412adc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:21.077 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:21.069+0000 7fb1b3fff700 1 -- 172.21.15.59:0/2198453024 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fb1b41eb6c0 msgr2=0x7fb1b412adc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:21.077 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:21.069+0000 7fb1b3fff700 1 --2- 172.21.15.59:0/2198453024 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fb1b41eb6c0 0x7fb1b412adc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:21.077 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:21.069+0000 7fb1b3fff700 1 -- 172.21.15.59:0/2198453024 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb1a4005210 msgr2=0x7fb1b412bcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:21.077 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:21.069+0000 7fb1b3fff700 1 --2- 172.21.15.59:0/2198453024 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb1a4005210 0x7fb1b412bcc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:21.077 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:21.069+0000 7fb1b3fff700 1 -- 172.21.15.59:0/2198453024 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb1b412a2e0 con 0x7fb1b413fdd0 2023-11-24T07:58:21.077 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:21.069+0000 7fb1b88f9700 1 --2- 172.21.15.59:0/2198453024 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb1a4005210 0x7fb1b412bcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-24T07:58:21.077 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:21.069+0000 7fb1b3fff700 1 --2- 172.21.15.59:0/2198453024 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb1b413fdd0 0x7fb1b41eb180 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7fb1a0011930 tx=0x7fb1a0011cf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:58:21.077 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:21.069+0000 7fb1b17fa700 1 -- 172.21.15.59:0/2198453024 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1a000f940 con 0x7fb1b413fdd0 2023-11-24T07:58:21.077 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:21.069+0000 7fb1b17fa700 1 -- 172.21.15.59:0/2198453024 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb1a000faa0 con 0x7fb1b413fdd0 2023-11-24T07:58:21.077 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:21.069+0000 7fb1bab5d700 1 -- 172.21.15.59:0/2198453024 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb1b412a5c0 con 0x7fb1b413fdd0 2023-11-24T07:58:21.077 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:21.069+0000 7fb1b17fa700 1 -- 172.21.15.59:0/2198453024 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1a000fe30 con 0x7fb1b413fdd0 2023-11-24T07:58:21.078 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:21.069+0000 7fb1bab5d700 1 -- 172.21.15.59:0/2198453024 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb1b412b300 con 0x7fb1b413fdd0 2023-11-24T07:58:21.078 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:21.070+0000 7fb1bab5d700 1 -- 172.21.15.59:0/2198453024 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb1b412ac10 con 0x7fb1b413fdd0 2023-11-24T07:58:21.078 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:21.071+0000 7fb1b17fa700 1 -- 172.21.15.59:0/2198453024 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 8) v1 ==== 89435+0+0 (secure 0 0 0) 0x7fb1a0014440 con 0x7fb1b413fdd0 2023-11-24T07:58:21.078 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:21.071+0000 7fb1b17fa700 1 --2- 172.21.15.59:0/2198453024 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7fb19806b110 0x7fb19806d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:21.078 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:21.071+0000 7fb1b17fa700 1 -- 172.21.15.59:0/2198453024 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5854+0+0 (secure 0 0 0) 0x7fb1a008a230 con 0x7fb1b413fdd0 2023-11-24T07:58:21.078 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:21.072+0000 7fb1b88f9700 1 --2- 172.21.15.59:0/2198453024 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7fb19806b110 0x7fb19806d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:21.078 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:21.072+0000 7fb1b88f9700 1 --2- 172.21.15.59:0/2198453024 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7fb19806b110 0x7fb19806d5d0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7fb1a8001030 tx=0x7fb1a8009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:58:21.079 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:21.076+0000 7fb1b17fa700 1 -- 172.21.15.59:0/2198453024 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fb1a0054f00 con 0x7fb1b413fdd0 2023-11-24T07:58:21.279 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:21.278+0000 7fb1bab5d700 1 -- 172.21.15.59:0/2198453024 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fb1b40cdb00 con 0x7fb1b413fdd0 2023-11-24T07:58:21.280 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:21.279+0000 7fb1b17fa700 1 -- 172.21.15.59:0/2198453024 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v47) v1 ==== 74+0+16668 (secure 0 0 0) 0x7fb1a0058520 con 0x7fb1b413fdd0 2023-11-24T07:58:21.281 INFO:teuthology.orchestra.run.smithi059.stdout: 2023-11-24T07:58:21.282 INFO:teuthology.orchestra.run.smithi059.stdout:{"epoch":47,"fsid":"9f0e8741-b8f9-48d5-a2c3-09f45903303d","created":"2023-11-24T07:56:06.552729+0000","modified":"2023-11-24T07:58:14.353149+0000","last_up_change":"2023-11-24T07:56:17.372352+0000","last_in_change":"2023-11-24T07:56:09.073250+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-24T07:56:18.352823+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_metadata","create_time":"2023-11-24T07:58:00.240130+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"44","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_data","create_time":"2023-11-24T07:58:00.977486+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"45","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"6d8f9e9b-1782-4e41-928b-a39f2a8ebc06","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6800","nonce":3373878772},{"type":"v1","addr":"172.21.15.59:6801","nonce":3373878772}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6802","nonce":3373878772},{"type":"v1","addr":"172.21.15.59:6803","nonce":3373878772}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6806","nonce":3373878772},{"type":"v1","addr":"172.21.15.59:6807","nonce":3373878772}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6804","nonce":3373878772},{"type":"v1","addr":"172.21.15.59:6805","nonce":3373878772}]},"public_addr":"172.21.15.59:6801/3373878772","cluster_addr":"172.21.15.59:6803/3373878772","heartbeat_back_addr":"172.21.15.59:6807/3373878772","heartbeat_front_addr":"172.21.15.59:6805/3373878772","state":["exists","up"]},{"osd":1,"uuid":"e8d882aa-52e1-4c04-9e1e-c44ef9204a5f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6824","nonce":648258563},{"type":"v1","addr":"172.21.15.59:6825","nonce":648258563}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6826","nonce":648258563},{"type":"v1","addr":"172.21.15.59:6827","nonce":648258563}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6830","nonce":648258563},{"type":"v1","addr":"172.21.15.59:6831","nonce":648258563}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6828","nonce":648258563},{"type":"v1","addr":"172.21.15.59:6829","nonce":648258563}]},"public_addr":"172.21.15.59:6825/648258563","cluster_addr":"172.21.15.59:6827/648258563","heartbeat_back_addr":"172.21.15.59:6831/648258563","heartbeat_front_addr":"172.21.15.59:6829/648258563","state":["exists","up"]},{"osd":2,"uuid":"69eebfa0-695b-4b21-90b9-8a3537168a01","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6808","nonce":1840430654},{"type":"v1","addr":"172.21.15.59:6809","nonce":1840430654}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6810","nonce":1840430654},{"type":"v1","addr":"172.21.15.59:6811","nonce":1840430654}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6814","nonce":1840430654},{"type":"v1","addr":"172.21.15.59:6815","nonce":1840430654}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6812","nonce":1840430654},{"type":"v1","addr":"172.21.15.59:6813","nonce":1840430654}]},"public_addr":"172.21.15.59:6809/1840430654","cluster_addr":"172.21.15.59:6811/1840430654","heartbeat_back_addr":"172.21.15.59:6815/1840430654","heartbeat_front_addr":"172.21.15.59:6813/1840430654","state":["exists","up"]},{"osd":3,"uuid":"96fd77fc-f911-46ff-9445-6f0be2daa8f4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6816","nonce":2504025850},{"type":"v1","addr":"172.21.15.59:6817","nonce":2504025850}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6818","nonce":2504025850},{"type":"v1","addr":"172.21.15.59:6819","nonce":2504025850}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6822","nonce":2504025850},{"type":"v1","addr":"172.21.15.59:6823","nonce":2504025850}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6820","nonce":2504025850},{"type":"v1","addr":"172.21.15.59:6821","nonce":2504025850}]},"public_addr":"172.21.15.59:6817/2504025850","cluster_addr":"172.21.15.59:6819/2504025850","heartbeat_back_addr":"172.21.15.59:6823/2504025850","heartbeat_front_addr":"172.21.15.59:6821/2504025850","state":["exists","up"]},{"osd":4,"uuid":"7ac29fca-b5b2-4c95-9d69-2f164cbbc708","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6816","nonce":2382463431},{"type":"v1","addr":"172.21.15.76:6818","nonce":2382463431}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6820","nonce":2382463431},{"type":"v1","addr":"172.21.15.76:6822","nonce":2382463431}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6828","nonce":2382463431},{"type":"v1","addr":"172.21.15.76:6830","nonce":2382463431}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6824","nonce":2382463431},{"type":"v1","addr":"172.21.15.76:6826","nonce":2382463431}]},"public_addr":"172.21.15.76:6818/2382463431","cluster_addr":"172.21.15.76:6822/2382463431","heartbeat_back_addr":"172.21.15.76:6830/2382463431","heartbeat_front_addr":"172.21.15.76:6826/2382463431","state":["exists","up"]},{"osd":5,"uuid":"485bf711-3851-4238-a4f4-51d52e0a9c58","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6800","nonce":329334308},{"type":"v1","addr":"172.21.15.76:6802","nonce":329334308}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6805","nonce":329334308},{"type":"v1","addr":"172.21.15.76:6807","nonce":329334308}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6814","nonce":329334308},{"type":"v1","addr":"172.21.15.76:6815","nonce":329334308}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6810","nonce":329334308},{"type":"v1","addr":"172.21.15.76:6812","nonce":329334308}]},"public_addr":"172.21.15.76:6802/329334308","cluster_addr":"172.21.15.76:6807/329334308","heartbeat_back_addr":"172.21.15.76:6815/329334308","heartbeat_front_addr":"172.21.15.76:6812/329334308","state":["exists","up"]},{"osd":6,"uuid":"72b3f2fc-1e4a-4e32-968d-003647a0286f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6801","nonce":3946579410},{"type":"v1","addr":"172.21.15.76:6803","nonce":3946579410}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6804","nonce":3946579410},{"type":"v1","addr":"172.21.15.76:6806","nonce":3946579410}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6811","nonce":3946579410},{"type":"v1","addr":"172.21.15.76:6813","nonce":3946579410}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6808","nonce":3946579410},{"type":"v1","addr":"172.21.15.76:6809","nonce":3946579410}]},"public_addr":"172.21.15.76:6803/3946579410","cluster_addr":"172.21.15.76:6806/3946579410","heartbeat_back_addr":"172.21.15.76:6813/3946579410","heartbeat_front_addr":"172.21.15.76:6809/3946579410","state":["exists","up"]},{"osd":7,"uuid":"1ddceb90-6c5d-4d1a-ad56-99cb877f8c70","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6817","nonce":841761538},{"type":"v1","addr":"172.21.15.76:6819","nonce":841761538}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6821","nonce":841761538},{"type":"v1","addr":"172.21.15.76:6823","nonce":841761538}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6829","nonce":841761538},{"type":"v1","addr":"172.21.15.76:6831","nonce":841761538}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6825","nonce":841761538},{"type":"v1","addr":"172.21.15.76:6827","nonce":841761538}]},"public_addr":"172.21.15.76:6819/841761538","cluster_addr":"172.21.15.76:6823/841761538","heartbeat_back_addr":"172.21.15.76:6831/841761538","heartbeat_front_addr":"172.21.15.76:6827/841761538","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T07:56:14.294838+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T07:56:14.270779+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T07:56:14.231809+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T07:56:14.260983+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T07:56:14.302997+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T07:56:14.258600+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T07:56:14.251413+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T07:56:14.304727+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"7.0","mappings":[{"from":5,"to":7}]},{"pgid":"7.c","mappings":[{"from":5,"to":7}]},{"pgid":"7.12","mappings":[{"from":5,"to":7}]},{"pgid":"7.14","mappings":[{"from":5,"to":7}]},{"pgid":"7.38","mappings":[{"from":5,"to":1}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-24T07:58:21.283 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:21.281+0000 7fb1bab5d700 1 -- 172.21.15.59:0/2198453024 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7fb19806b110 msgr2=0x7fb19806d5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:21.283 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:21.281+0000 7fb1bab5d700 1 --2- 172.21.15.59:0/2198453024 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7fb19806b110 0x7fb19806d5d0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7fb1a8001030 tx=0x7fb1a8009040 comp rx=0 tx=0).stop 2023-11-24T07:58:21.284 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:21.281+0000 7fb1bab5d700 1 -- 172.21.15.59:0/2198453024 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb1b413fdd0 msgr2=0x7fb1b41eb180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:21.284 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:21.281+0000 7fb1bab5d700 1 --2- 172.21.15.59:0/2198453024 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb1b413fdd0 0x7fb1b41eb180 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7fb1a0011930 tx=0x7fb1a0011cf0 comp rx=0 tx=0).stop 2023-11-24T07:58:21.284 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:21.281+0000 7fb1bab5d700 1 -- 172.21.15.59:0/2198453024 shutdown_connections 2023-11-24T07:58:21.284 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:21.281+0000 7fb1bab5d700 1 --2- 172.21.15.59:0/2198453024 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7fb19806b110 0x7fb19806d5d0 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:21.285 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:21.281+0000 7fb1bab5d700 1 --2- 172.21.15.59:0/2198453024 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fb1b41eb6c0 0x7fb1b412adc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:21.285 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:21.281+0000 7fb1bab5d700 1 --2- 172.21.15.59:0/2198453024 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb1a4005210 0x7fb1b412bcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:21.285 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:21.281+0000 7fb1bab5d700 1 --2- 172.21.15.59:0/2198453024 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb1b413fdd0 0x7fb1b41eb180 unknown :-1 s=CLOSED pgs=605 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:21.285 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:21.281+0000 7fb1bab5d700 1 -- 172.21.15.59:0/2198453024 >> 172.21.15.59:0/2198453024 conn(0x7fb1b40b2840 msgr2=0x7fb1b40af910 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:58:21.286 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:21.281+0000 7fb1bab5d700 1 -- 172.21.15.59:0/2198453024 shutdown_connections 2023-11-24T07:58:21.286 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:21.281+0000 7fb1bab5d700 1 -- 172.21.15.59:0/2198453024 wait complete. 2023-11-24T07:58:21.295 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-24T07:56:18.352823+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 6, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-24T07:58:00.240130+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '44', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 7, 'pool_name': 'cephfs_data', 'create_time': '2023-11-24T07:58:00.977486+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '45', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-24T07:58:21.296 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-24T07:58:21.424 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:21.422+0000 7f05e39a1700 1 Processor -- start 2023-11-24T07:58:21.424 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:21.422+0000 7f05e39a1700 1 -- start start 2023-11-24T07:58:21.426 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:21.423+0000 7f05e39a1700 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f05dc0d0870 0x7f05dc0d26c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:21.426 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:21.423+0000 7f05e39a1700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f05dc0d49f0 0x7f05dc0b6d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:21.426 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:21.423+0000 7f05e39a1700 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f05dc0d4e20 con 0x7f05dc0d0c50 2023-11-24T07:58:21.426 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:21.423+0000 7f05e39a1700 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f05dc0d2f80 con 0x7f05dc0d49f0 2023-11-24T07:58:21.426 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:21.423+0000 7f05e39a1700 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f05dc0cff70 con 0x7f05dc0d0870 2023-11-24T07:58:21.427 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:21.423+0000 7f05e1f3e700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f05dc0d49f0 0x7f05dc0b6d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:21.427 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:21.423+0000 7f05e0f3c700 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7f05dc0d0c50 0x7f05dc0cf860 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:56690/0 (socket says 172.21.15.59:56690) 2023-11-24T07:58:21.427 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:21.423+0000 7f05e0f3c700 1 -- 172.21.15.59:0/4018378184 learned_addr learned my addr 172.21.15.59:0/4018378184 (peer_addr_for_me v1:172.21.15.59:0/0) 2023-11-24T07:58:21.427 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:21.423+0000 7f05e173d700 1 --2- 172.21.15.59:0/4018378184 >> v2:172.21.15.76:3300/0 conn(0x7f05dc0d0870 0x7f05dc0d26c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:21.427 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:21.423+0000 7f05e1f3e700 1 -- 172.21.15.59:0/4018378184 >> v2:172.21.15.76:3300/0 conn(0x7f05dc0d0870 msgr2=0x7f05dc0d26c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T07:58:21.427 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:21.423+0000 7f05e1f3e700 1 --2- 172.21.15.59:0/4018378184 >> v2:172.21.15.76:3300/0 conn(0x7f05dc0d0870 0x7f05dc0d26c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:21.427 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:21.423+0000 7f05e1f3e700 1 -- 172.21.15.59:0/4018378184 >> v1:172.21.15.59:6789/0 conn(0x7f05dc0d0c50 legacy=0x7f05dc0cf860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:21.427 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:21.423+0000 7f05dbfff700 1 -- 172.21.15.59:0/4018378184 <== mon.0 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1254401348 0 0) 0x7f05dc0d4e20 con 0x7f05dc0d0c50 2023-11-24T07:58:21.430 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:21.423+0000 7f05e1f3e700 1 -- 172.21.15.59:0/4018378184 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f05dc0b5370 con 0x7f05dc0d49f0 2023-11-24T07:58:21.430 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:21.424+0000 7f05e173d700 1 --2- 172.21.15.59:0/4018378184 >> v2:172.21.15.76:3300/0 conn(0x7f05dc0d0870 0x7f05dc0d26c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T07:58:21.430 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:21.424+0000 7f05e1f3e700 1 --2- 172.21.15.59:0/4018378184 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f05dc0d49f0 0x7f05dc0b6d20 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7f05dc08c560 tx=0x7f05cc00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=30286d109b7facf7 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:58:21.430 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:21.424+0000 7f05dbfff700 1 -- 172.21.15.59:0/4018378184 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05cc00e070 con 0x7f05dc0d49f0 2023-11-24T07:58:21.431 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:21.424+0000 7f05dbfff700 1 -- 172.21.15.59:0/4018378184 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f05cc00bbd0 con 0x7f05dc0d49f0 2023-11-24T07:58:21.431 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:21.424+0000 7f05dbfff700 1 -- 172.21.15.59:0/4018378184 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05cc012430 con 0x7f05dc0d49f0 2023-11-24T07:58:21.431 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:21.424+0000 7f05e39a1700 1 -- 172.21.15.59:0/4018378184 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f05dc0d49f0 msgr2=0x7f05dc0b6d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:21.432 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:21.424+0000 7f05e39a1700 1 --2- 172.21.15.59:0/4018378184 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f05dc0d49f0 0x7f05dc0b6d20 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7f05dc08c560 tx=0x7f05cc00b330 comp rx=0 tx=0).stop 2023-11-24T07:58:21.432 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:21.424+0000 7f05e39a1700 1 -- 172.21.15.59:0/4018378184 shutdown_connections 2023-11-24T07:58:21.433 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:21.424+0000 7f05e39a1700 1 --2- 172.21.15.59:0/4018378184 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f05dc0d49f0 0x7f05dc0b6d20 unknown :-1 s=CLOSED pgs=607 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:21.433 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:21.424+0000 7f05e39a1700 1 --2- 172.21.15.59:0/4018378184 >> v2:172.21.15.76:3300/0 conn(0x7f05dc0d0870 0x7f05dc0d26c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:21.433 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:21.424+0000 7f05e39a1700 1 -- 172.21.15.59:0/4018378184 >> 172.21.15.59:0/4018378184 conn(0x7f05dc0d3ed0 msgr2=0x7f05dc0d42e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:58:21.433 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:21.424+0000 7f05e39a1700 1 -- 172.21.15.59:0/4018378184 shutdown_connections 2023-11-24T07:58:21.433 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:21.425+0000 7f05e39a1700 1 -- 172.21.15.59:0/4018378184 wait complete. 2023-11-24T07:58:21.436 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:21.426+0000 7f05e39a1700 1 Processor -- start 2023-11-24T07:58:21.436 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:21.426+0000 7f05e39a1700 1 -- start start 2023-11-24T07:58:21.436 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:21.426+0000 7f05e39a1700 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f05dc0d0870 0x7f05dc157950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:21.436 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:21.426+0000 7f05e39a1700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f05dc0d0c50 0x7f05dc152650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:21.436 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:21.427+0000 7f05e39a1700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f05dc0d49f0 0x7f05dc155af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:21.437 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:21.427+0000 7f05e39a1700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05dc154300 con 0x7f05dc0d0c50 2023-11-24T07:58:21.437 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:21.427+0000 7f05e39a1700 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05dc1551d0 con 0x7f05dc0d49f0 2023-11-24T07:58:21.437 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:21.427+0000 7f05e39a1700 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f05dc152b90 con 0x7f05dc0d0870 2023-11-24T07:58:21.437 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:21.427+0000 7f05e0f3c700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f05dc0d0c50 0x7f05dc152650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:21.437 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:21.427+0000 7f05e0f3c700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f05dc0d0c50 0x7f05dc152650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:53222/0 (socket says 172.21.15.59:53222) 2023-11-24T07:58:21.437 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:21.427+0000 7f05e0f3c700 1 -- 172.21.15.59:0/859833613 learned_addr learned my addr 172.21.15.59:0/859833613 (peer_addr_for_me v2:172.21.15.59:0/0) 2023-11-24T07:58:21.437 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:21.427+0000 7f05e173d700 1 --2- 172.21.15.59:0/859833613 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f05dc0d0870 0x7f05dc157950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:21.437 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:21.427+0000 7f05e1f3e700 1 --2- 172.21.15.59:0/859833613 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f05dc0d49f0 0x7f05dc155af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:21.437 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:21.427+0000 7f05e0f3c700 1 -- 172.21.15.59:0/859833613 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f05dc0d0870 msgr2=0x7f05dc157950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:21.437 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:21.427+0000 7f05e0f3c700 1 --2- 172.21.15.59:0/859833613 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f05dc0d0870 0x7f05dc157950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:21.437 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:21.427+0000 7f05e0f3c700 1 -- 172.21.15.59:0/859833613 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f05dc0d49f0 msgr2=0x7f05dc155af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:21.437 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:21.427+0000 7f05e0f3c700 1 --2- 172.21.15.59:0/859833613 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f05dc0d49f0 0x7f05dc155af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:21.437 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:21.427+0000 7f05e0f3c700 1 -- 172.21.15.59:0/859833613 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f05cc009a70 con 0x7f05dc0d0c50 2023-11-24T07:58:21.437 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:21.427+0000 7f05e1f3e700 1 --2- 172.21.15.59:0/859833613 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f05dc0d49f0 0x7f05dc155af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-24T07:58:21.438 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:21.427+0000 7f05e173d700 1 --2- 172.21.15.59:0/859833613 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f05dc0d0870 0x7f05dc157950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-24T07:58:21.438 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:21.427+0000 7f05e0f3c700 1 --2- 172.21.15.59:0/859833613 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f05dc0d0c50 0x7f05dc152650 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7f05d000c540 tx=0x7f05d000c900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:58:21.438 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:21.427+0000 7f05d9ffb700 1 -- 172.21.15.59:0/859833613 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05d0004c80 con 0x7f05dc0d0c50 2023-11-24T07:58:21.438 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:21.427+0000 7f05d9ffb700 1 -- 172.21.15.59:0/859833613 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f05d0004de0 con 0x7f05dc0d0c50 2023-11-24T07:58:21.438 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:21.427+0000 7f05e39a1700 1 -- 172.21.15.59:0/859833613 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f05dc156030 con 0x7f05dc0d0c50 2023-11-24T07:58:21.438 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:21.427+0000 7f05d9ffb700 1 -- 172.21.15.59:0/859833613 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05d0007310 con 0x7f05dc0d0c50 2023-11-24T07:58:21.438 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:21.427+0000 7f05e39a1700 1 -- 172.21.15.59:0/859833613 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f05dc158020 con 0x7f05dc0d0c50 2023-11-24T07:58:21.438 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:21.428+0000 7f05e39a1700 1 -- 172.21.15.59:0/859833613 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f05dc158e60 con 0x7f05dc0d0c50 2023-11-24T07:58:21.438 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:21.429+0000 7f05d9ffb700 1 -- 172.21.15.59:0/859833613 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 8) v1 ==== 89435+0+0 (secure 0 0 0) 0x7f05d0007470 con 0x7f05dc0d0c50 2023-11-24T07:58:21.438 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:21.429+0000 7f05d9ffb700 1 --2- 172.21.15.59:0/859833613 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f05c806b110 0x7f05c806d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:21.438 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:21.429+0000 7f05d9ffb700 1 -- 172.21.15.59:0/859833613 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5854+0+0 (secure 0 0 0) 0x7f05d0089760 con 0x7f05dc0d0c50 2023-11-24T07:58:21.438 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:21.430+0000 7f05e173d700 1 --2- 172.21.15.59:0/859833613 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f05c806b110 0x7f05c806d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:21.438 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:21.430+0000 7f05e173d700 1 --2- 172.21.15.59:0/859833613 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f05c806b110 0x7f05c806d5d0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f05d4006fd0 tx=0x7f05d4008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:58:21.439 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:21.434+0000 7f05d9ffb700 1 -- 172.21.15.59:0/859833613 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f05d00544b0 con 0x7f05dc0d0c50 2023-11-24T07:58:21.637 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:21.635+0000 7f05e39a1700 1 -- 172.21.15.59:0/859833613 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f05dc067a20 con 0x7f05dc0d0c50 2023-11-24T07:58:21.637 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:21.635+0000 7f05d9ffb700 1 -- 172.21.15.59:0/859833613 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v47) v1 ==== 93+0+10 (secure 0 0 0) 0x7f05d0057ad0 con 0x7f05dc0d0c50 2023-11-24T07:58:21.637 INFO:teuthology.orchestra.run.smithi059.stdout:pg_num: 1 2023-11-24T07:58:21.640 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:21.637+0000 7f05e39a1700 1 -- 172.21.15.59:0/859833613 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f05c806b110 msgr2=0x7f05c806d5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:21.640 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:21.637+0000 7f05e39a1700 1 --2- 172.21.15.59:0/859833613 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f05c806b110 0x7f05c806d5d0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f05d4006fd0 tx=0x7f05d4008040 comp rx=0 tx=0).stop 2023-11-24T07:58:21.640 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:21.637+0000 7f05e39a1700 1 -- 172.21.15.59:0/859833613 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f05dc0d0c50 msgr2=0x7f05dc152650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:21.640 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:21.637+0000 7f05e39a1700 1 --2- 172.21.15.59:0/859833613 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f05dc0d0c50 0x7f05dc152650 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7f05d000c540 tx=0x7f05d000c900 comp rx=0 tx=0).stop 2023-11-24T07:58:21.641 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:21.637+0000 7f05e39a1700 1 -- 172.21.15.59:0/859833613 shutdown_connections 2023-11-24T07:58:21.641 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:21.637+0000 7f05e39a1700 1 --2- 172.21.15.59:0/859833613 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f05c806b110 0x7f05c806d5d0 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:21.641 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:21.637+0000 7f05e39a1700 1 --2- 172.21.15.59:0/859833613 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f05dc0d49f0 0x7f05dc155af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:21.641 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:21.637+0000 7f05e39a1700 1 --2- 172.21.15.59:0/859833613 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f05dc0d0870 0x7f05dc157950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:21.641 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:21.637+0000 7f05e39a1700 1 --2- 172.21.15.59:0/859833613 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f05dc0d0c50 0x7f05dc152650 unknown :-1 s=CLOSED pgs=608 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:21.642 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:21.637+0000 7f05e39a1700 1 -- 172.21.15.59:0/859833613 >> 172.21.15.59:0/859833613 conn(0x7f05dc0d3ed0 msgr2=0x7f05dc0d2390 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:58:21.642 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:21.637+0000 7f05e39a1700 1 -- 172.21.15.59:0/859833613 shutdown_connections 2023-11-24T07:58:21.642 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:21.638+0000 7f05e39a1700 1 -- 172.21.15.59:0/859833613 wait complete. 2023-11-24T07:58:21.652 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-24T07:58:21.781 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:21.780+0000 7f5084237700 1 Processor -- start 2023-11-24T07:58:21.782 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:21.780+0000 7f5084237700 1 -- start start 2023-11-24T07:58:21.785 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:21.781+0000 7f5084237700 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f507c1440f0 0x7f507c0b51b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:21.785 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:21.781+0000 7f5084237700 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f507c143ec0 con 0x7f507c0722c0 2023-11-24T07:58:21.785 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:21.781+0000 7f5084237700 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f507c0b3ae0 con 0x7f507c143ae0 2023-11-24T07:58:21.785 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:21.781+0000 7f5084237700 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f507c0b40a0 con 0x7f507c1440f0 2023-11-24T07:58:21.785 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:21.781+0000 7f50817d2700 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7f507c0722c0 0x7f507c06e510 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:56704/0 (socket says 172.21.15.59:56704) 2023-11-24T07:58:21.786 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:21.781+0000 7f50817d2700 1 -- 172.21.15.59:0/3178504374 learned_addr learned my addr 172.21.15.59:0/3178504374 (peer_addr_for_me v1:172.21.15.59:0/0) 2023-11-24T07:58:21.786 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:21.781+0000 7f50827d4700 1 --2- 172.21.15.59:0/3178504374 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f507c1440f0 0x7f507c0b51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:21.786 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:21.781+0000 7f5080fd1700 1 -- 172.21.15.59:0/3178504374 <== mon.0 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3684380659 0 0) 0x7f507c143ec0 con 0x7f507c0722c0 2023-11-24T07:58:21.786 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:21.781+0000 7f5080fd1700 1 -- 172.21.15.59:0/3178504374 --> v1:172.21.15.59:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f50680035b0 con 0x7f507c0722c0 2023-11-24T07:58:21.786 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:21.781+0000 7f5080fd1700 1 -- 172.21.15.59:0/3178504374 <== mon.1 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3847151739 0 0) 0x7f507c0b3ae0 con 0x7f507c143ae0 2023-11-24T07:58:21.786 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:21.781+0000 7f5080fd1700 1 -- 172.21.15.59:0/3178504374 --> v1:172.21.15.76:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f507c143ec0 con 0x7f507c143ae0 2023-11-24T07:58:21.787 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:21.781+0000 7f5080fd1700 1 -- 172.21.15.59:0/3178504374 <== mon.0 v1:172.21.15.59:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3859920490 0 0) 0x7f50680035b0 con 0x7f507c0722c0 2023-11-24T07:58:21.787 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:21.781+0000 7f5080fd1700 1 -- 172.21.15.59:0/3178504374 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f507c1440f0 msgr2=0x7f507c0b51b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T07:58:21.787 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:21.781+0000 7f5080fd1700 1 --2- 172.21.15.59:0/3178504374 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f507c1440f0 0x7f507c0b51b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:21.787 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:21.781+0000 7f5080fd1700 1 -- 172.21.15.59:0/3178504374 >> v1:172.21.15.76:6789/0 conn(0x7f507c143ae0 legacy=0x7f507c071bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:21.787 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:21.781+0000 7f5080fd1700 1 -- 172.21.15.59:0/3178504374 --> v1:172.21.15.59:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f507c0b3dc0 con 0x7f507c0722c0 2023-11-24T07:58:21.787 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:21.781+0000 7f5080fd1700 1 -- 172.21.15.59:0/3178504374 <== mon.0 v1:172.21.15.59:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1312333566 0 0) 0x7f5070001670 con 0x7f507c0722c0 2023-11-24T07:58:21.787 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:21.781+0000 7f5080fd1700 1 -- 172.21.15.59:0/3178504374 >> v1:172.21.15.59:6789/0 conn(0x7f507c0722c0 legacy=0x7f507c06e510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:21.787 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:21.781+0000 7f5080fd1700 1 --2- 172.21.15.59:0/3178504374 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5068003bc0 0x7f5068003fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:21.788 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:21.781+0000 7f5080fd1700 1 --2- 172.21.15.59:0/3178504374 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5068004850 0x7f5068004cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:21.788 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:21.781+0000 7f5080fd1700 1 --2- 172.21.15.59:0/3178504374 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f5068005210 0x7f506800ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:21.788 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:21.781+0000 7f5080fd1700 1 -- 172.21.15.59:0/3178504374 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f507c0b40a0 con 0x7f5068003bc0 2023-11-24T07:58:21.788 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:21.782+0000 7f5080fd1700 1 -- 172.21.15.59:0/3178504374 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f506800c030 con 0x7f5068004850 2023-11-24T07:58:21.788 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:21.782+0000 7f5080fd1700 1 -- 172.21.15.59:0/3178504374 --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f506800c1f0 con 0x7f5068005210 2023-11-24T07:58:21.788 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:21.782+0000 7f50827d4700 1 --2- 172.21.15.59:0/3178504374 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f507c1440f0 0x7f507c0b51b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-24T07:58:21.788 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:21.782+0000 7f5081fd3700 1 --2- 172.21.15.59:0/3178504374 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5068003bc0 0x7f5068003fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:21.789 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:21.782+0000 7f50817d2700 1 --2- 172.21.15.59:0/3178504374 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5068004850 0x7f5068004cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:21.789 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:21.782+0000 7f50827d4700 1 --2- 172.21.15.59:0/3178504374 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f5068005210 0x7f506800ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:21.789 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:21.782+0000 7f5081fd3700 1 -- 172.21.15.59:0/3178504374 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f5068005210 msgr2=0x7f506800ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T07:58:21.789 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:21.782+0000 7f5081fd3700 1 --2- 172.21.15.59:0/3178504374 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f5068005210 0x7f506800ba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:21.789 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:21.782+0000 7f5081fd3700 1 -- 172.21.15.59:0/3178504374 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5068004850 msgr2=0x7f5068004cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T07:58:21.789 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:21.782+0000 7f5081fd3700 1 --2- 172.21.15.59:0/3178504374 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5068004850 0x7f5068004cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:21.790 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:21.782+0000 7f5081fd3700 1 -- 172.21.15.59:0/3178504374 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f50680035b0 con 0x7f5068003bc0 2023-11-24T07:58:21.790 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:21.782+0000 7f5081fd3700 1 --2- 172.21.15.59:0/3178504374 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5068003bc0 0x7f5068003fc0 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7f5078001970 tx=0x7f5068005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b1af9058b39fc688 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:58:21.790 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:21.782+0000 7f5080fd1700 1 -- 172.21.15.59:0/3178504374 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50780052d0 con 0x7f5068003bc0 2023-11-24T07:58:21.790 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:21.782+0000 7f5080fd1700 1 -- 172.21.15.59:0/3178504374 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5078005430 con 0x7f5068003bc0 2023-11-24T07:58:21.790 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:21.782+0000 7f5080fd1700 1 -- 172.21.15.59:0/3178504374 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50780038c0 con 0x7f5068003bc0 2023-11-24T07:58:21.790 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:21.783+0000 7f5084237700 1 -- 172.21.15.59:0/3178504374 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5068003bc0 msgr2=0x7f5068003fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:21.790 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:21.783+0000 7f5084237700 1 --2- 172.21.15.59:0/3178504374 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5068003bc0 0x7f5068003fc0 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7f5078001970 tx=0x7f5068005880 comp rx=0 tx=0).stop 2023-11-24T07:58:21.790 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:21.783+0000 7f50827d4700 1 -- 172.21.15.59:0/3178504374 reap_dead start 2023-11-24T07:58:21.791 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:21.783+0000 7f5084237700 1 -- 172.21.15.59:0/3178504374 shutdown_connections 2023-11-24T07:58:21.791 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:21.783+0000 7f5084237700 1 -- 172.21.15.59:0/3178504374 >> 172.21.15.59:0/3178504374 conn(0x7f507c0b2740 msgr2=0x7f507c0b2b20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:58:21.791 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:21.783+0000 7f5084237700 1 -- 172.21.15.59:0/3178504374 shutdown_connections 2023-11-24T07:58:21.791 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:21.783+0000 7f5084237700 1 -- 172.21.15.59:0/3178504374 wait complete. 2023-11-24T07:58:21.793 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:21.784+0000 7f5084237700 1 Processor -- start 2023-11-24T07:58:21.793 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:21.784+0000 7f5084237700 1 -- start start 2023-11-24T07:58:21.793 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:21.785+0000 7f5084237700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5068003bc0 0x7f507c153540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:21.793 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:21.785+0000 7f5084237700 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f507c154f30 0x7f507c141340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:21.794 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:21.785+0000 7f5084237700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f507c1553b0 0x7f507c141880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:21.794 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:21.785+0000 7f5084237700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f507c15bf00 con 0x7f507c1553b0 2023-11-24T07:58:21.794 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:21.785+0000 7f5084237700 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f507c15b2f0 con 0x7f5068003bc0 2023-11-24T07:58:21.794 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:21.785+0000 7f5084237700 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f507c15b5d0 con 0x7f507c154f30 2023-11-24T07:58:21.794 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:21.785+0000 7f50827d4700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f507c1553b0 0x7f507c141880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:21.794 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:21.785+0000 7f50827d4700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f507c1553b0 0x7f507c141880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:53244/0 (socket says 172.21.15.59:53244) 2023-11-24T07:58:21.794 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:21.785+0000 7f50827d4700 1 -- 172.21.15.59:0/3107339578 learned_addr learned my addr 172.21.15.59:0/3107339578 (peer_addr_for_me v2:172.21.15.59:0/0) 2023-11-24T07:58:21.794 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:21.785+0000 7f5081fd3700 1 --2- 172.21.15.59:0/3107339578 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5068003bc0 0x7f507c153540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:21.794 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:21.785+0000 7f50817d2700 1 --2- 172.21.15.59:0/3107339578 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f507c154f30 0x7f507c141340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:21.794 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:21.785+0000 7f50827d4700 1 -- 172.21.15.59:0/3107339578 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f507c154f30 msgr2=0x7f507c141340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:21.794 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:21.785+0000 7f50827d4700 1 --2- 172.21.15.59:0/3107339578 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f507c154f30 0x7f507c141340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:21.794 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:21.785+0000 7f50827d4700 1 -- 172.21.15.59:0/3107339578 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5068003bc0 msgr2=0x7f507c153540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:21.795 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:21.785+0000 7f50827d4700 1 --2- 172.21.15.59:0/3107339578 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5068003bc0 0x7f507c153540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:21.795 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:21.785+0000 7f50827d4700 1 -- 172.21.15.59:0/3107339578 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f507c15b8b0 con 0x7f507c1553b0 2023-11-24T07:58:21.795 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:21.786+0000 7f50827d4700 1 --2- 172.21.15.59:0/3107339578 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f507c1553b0 0x7f507c141880 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7f507c0747c0 tx=0x7f50740116e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:58:21.795 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:21.786+0000 7f506e7fc700 1 -- 172.21.15.59:0/3107339578 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f507400e070 con 0x7f507c1553b0 2023-11-24T07:58:21.795 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:21.786+0000 7f506e7fc700 1 -- 172.21.15.59:0/3107339578 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5074011da0 con 0x7f507c1553b0 2023-11-24T07:58:21.795 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:21.786+0000 7f5084237700 1 -- 172.21.15.59:0/3107339578 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f507c15bb90 con 0x7f507c1553b0 2023-11-24T07:58:21.795 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:21.786+0000 7f506e7fc700 1 -- 172.21.15.59:0/3107339578 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5074009ea0 con 0x7f507c1553b0 2023-11-24T07:58:21.795 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:21.786+0000 7f5084237700 1 -- 172.21.15.59:0/3107339578 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f507c13f9f0 con 0x7f507c1553b0 2023-11-24T07:58:21.795 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:21.787+0000 7f5084237700 1 -- 172.21.15.59:0/3107339578 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f507c067a40 con 0x7f507c1553b0 2023-11-24T07:58:21.795 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:21.787+0000 7f506e7fc700 1 -- 172.21.15.59:0/3107339578 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 8) v1 ==== 89435+0+0 (secure 0 0 0) 0x7f5074003a20 con 0x7f507c1553b0 2023-11-24T07:58:21.795 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:21.787+0000 7f506e7fc700 1 --2- 172.21.15.59:0/3107339578 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f506006b110 0x7f506006d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:21.795 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:21.788+0000 7f506e7fc700 1 -- 172.21.15.59:0/3107339578 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5854+0+0 (secure 0 0 0) 0x7f507408a240 con 0x7f507c1553b0 2023-11-24T07:58:21.796 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:21.788+0000 7f5081fd3700 1 --2- 172.21.15.59:0/3107339578 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f506006b110 0x7f506006d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:21.796 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:21.788+0000 7f5081fd3700 1 --2- 172.21.15.59:0/3107339578 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f506006b110 0x7f506006d5d0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f5078005040 tx=0x7f5078009030 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:58:21.796 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:21.792+0000 7f506e7fc700 1 -- 172.21.15.59:0/3107339578 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f507c067a40 con 0x7f507c1553b0 2023-11-24T07:58:21.993 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:21.991+0000 7f5084237700 1 -- 172.21.15.59:0/3107339578 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f507c067a40 con 0x7f507c1553b0 2023-11-24T07:58:21.993 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:21.991+0000 7f506e7fc700 1 -- 172.21.15.59:0/3107339578 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v47) v1 ==== 104+0+11 (secure 0 0 0) 0x7f507c067a40 con 0x7f507c1553b0 2023-11-24T07:58:21.993 INFO:teuthology.orchestra.run.smithi059.stdout:pg_num: 32 2023-11-24T07:58:21.995 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:21.993+0000 7f5084237700 1 -- 172.21.15.59:0/3107339578 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f506006b110 msgr2=0x7f506006d5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:21.995 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:21.993+0000 7f5084237700 1 --2- 172.21.15.59:0/3107339578 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f506006b110 0x7f506006d5d0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f5078005040 tx=0x7f5078009030 comp rx=0 tx=0).stop 2023-11-24T07:58:21.996 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:21.993+0000 7f5084237700 1 -- 172.21.15.59:0/3107339578 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f507c1553b0 msgr2=0x7f507c141880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:21.996 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:21.993+0000 7f5084237700 1 --2- 172.21.15.59:0/3107339578 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f507c1553b0 0x7f507c141880 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7f507c0747c0 tx=0x7f50740116e0 comp rx=0 tx=0).stop 2023-11-24T07:58:21.996 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:21.993+0000 7f5084237700 1 -- 172.21.15.59:0/3107339578 shutdown_connections 2023-11-24T07:58:21.996 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:21.993+0000 7f5084237700 1 --2- 172.21.15.59:0/3107339578 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f506006b110 0x7f506006d5d0 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:21.996 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:21.993+0000 7f5084237700 1 --2- 172.21.15.59:0/3107339578 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f507c1553b0 0x7f507c141880 unknown :-1 s=CLOSED pgs=611 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:21.996 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:21.993+0000 7f5084237700 1 --2- 172.21.15.59:0/3107339578 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5068003bc0 0x7f507c153540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:21.996 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:21.993+0000 7f5084237700 1 --2- 172.21.15.59:0/3107339578 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f507c154f30 0x7f507c141340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:21.996 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:21.993+0000 7f5084237700 1 -- 172.21.15.59:0/3107339578 >> 172.21.15.59:0/3107339578 conn(0x7f507c0b2740 msgr2=0x7f507c0cfd90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:58:21.996 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:21.993+0000 7f5084237700 1 -- 172.21.15.59:0/3107339578 shutdown_connections 2023-11-24T07:58:21.996 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:21.993+0000 7f5084237700 1 -- 172.21.15.59:0/3107339578 wait complete. 2023-11-24T07:58:22.007 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-24T07:58:22.135 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:22.133+0000 7f2a71796700 1 Processor -- start 2023-11-24T07:58:22.135 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:22.133+0000 7f2a71796700 1 -- start start 2023-11-24T07:58:22.137 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:22.134+0000 7f2a71796700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f2a6c0d0870 0x7f2a6c0d26c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:22.138 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:22.134+0000 7f2a71796700 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f2a6c0d49f0 0x7f2a6c0b6d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:22.138 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:22.134+0000 7f2a71796700 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2a6c0d4e20 con 0x7f2a6c0d0c50 2023-11-24T07:58:22.138 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:22.134+0000 7f2a71796700 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2a6c0d2f80 con 0x7f2a6c0d0870 2023-11-24T07:58:22.138 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:22.134+0000 7f2a71796700 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2a6c0cff70 con 0x7f2a6c0d49f0 2023-11-24T07:58:22.138 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:22.134+0000 7f2a6affd700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f2a6c0d0870 0x7f2a6c0d26c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:22.138 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:22.134+0000 7f2a6affd700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f2a6c0d0870 0x7f2a6c0d26c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:53248/0 (socket says 172.21.15.59:53248) 2023-11-24T07:58:22.138 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:22.134+0000 7f2a6affd700 1 -- 172.21.15.59:0/1781456237 learned_addr learned my addr 172.21.15.59:0/1781456237 (peer_addr_for_me v2:172.21.15.59:0/0) 2023-11-24T07:58:22.138 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:22.134+0000 7f2a6b7fe700 1 --2- 172.21.15.59:0/1781456237 >> v2:172.21.15.76:3300/0 conn(0x7f2a6c0d49f0 0x7f2a6c0b6d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:22.138 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:22.134+0000 7f2a6affd700 1 -- 172.21.15.59:0/1781456237 >> v2:172.21.15.76:3300/0 conn(0x7f2a6c0d49f0 msgr2=0x7f2a6c0b6d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T07:58:22.139 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:22.134+0000 7f2a6affd700 1 --2- 172.21.15.59:0/1781456237 >> v2:172.21.15.76:3300/0 conn(0x7f2a6c0d49f0 0x7f2a6c0b6d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:22.139 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:22.134+0000 7f2a6affd700 1 -- 172.21.15.59:0/1781456237 >> v1:172.21.15.76:6789/0 conn(0x7f2a6c0d0c50 legacy=0x7f2a6c0cf860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T07:58:22.139 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:22.134+0000 7f2a6affd700 1 -- 172.21.15.59:0/1781456237 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2a6c0b5370 con 0x7f2a6c0d0870 2023-11-24T07:58:22.139 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:22.135+0000 7f2a6affd700 1 --2- 172.21.15.59:0/1781456237 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2a6c0d0870 0x7f2a6c0d26c0 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7f2a6c0736c0 tx=0x7f2a60009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a4b780cf03f7defc server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:58:22.139 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:22.135+0000 7f2a69ffb700 1 -- 172.21.15.59:0/1781456237 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a60004230 con 0x7f2a6c0d0870 2023-11-24T07:58:22.139 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:22.135+0000 7f2a69ffb700 1 -- 172.21.15.59:0/1781456237 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2a60004390 con 0x7f2a6c0d0870 2023-11-24T07:58:22.139 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:22.135+0000 7f2a69ffb700 1 -- 172.21.15.59:0/1781456237 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a600036a0 con 0x7f2a6c0d0870 2023-11-24T07:58:22.139 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:22.135+0000 7f2a71796700 1 -- 172.21.15.59:0/1781456237 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2a6c0d0870 msgr2=0x7f2a6c0d26c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:22.140 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:22.135+0000 7f2a71796700 1 --2- 172.21.15.59:0/1781456237 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2a6c0d0870 0x7f2a6c0d26c0 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7f2a6c0736c0 tx=0x7f2a60009bd0 comp rx=0 tx=0).stop 2023-11-24T07:58:22.140 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:22.135+0000 7f2a71796700 1 -- 172.21.15.59:0/1781456237 shutdown_connections 2023-11-24T07:58:22.140 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:22.135+0000 7f2a71796700 1 --2- 172.21.15.59:0/1781456237 >> v2:172.21.15.76:3300/0 conn(0x7f2a6c0d49f0 0x7f2a6c0b6d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:22.140 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:22.135+0000 7f2a71796700 1 --2- 172.21.15.59:0/1781456237 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2a6c0d0870 0x7f2a6c0d26c0 unknown :-1 s=CLOSED pgs=612 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:22.140 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:22.135+0000 7f2a71796700 1 -- 172.21.15.59:0/1781456237 >> 172.21.15.59:0/1781456237 conn(0x7f2a6c0d3ed0 msgr2=0x7f2a6c0d42e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:58:22.140 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:22.135+0000 7f2a71796700 1 -- 172.21.15.59:0/1781456237 shutdown_connections 2023-11-24T07:58:22.140 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:22.135+0000 7f2a71796700 1 -- 172.21.15.59:0/1781456237 wait complete. 2023-11-24T07:58:22.143 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:22.137+0000 7f2a71796700 1 Processor -- start 2023-11-24T07:58:22.143 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:22.137+0000 7f2a71796700 1 -- start start 2023-11-24T07:58:22.143 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:22.137+0000 7f2a71796700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2a6c0d0870 0x7f2a6c15a660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:22.143 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:22.137+0000 7f2a71796700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2a6c0d0c50 0x7f2a6c0b5950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:22.143 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:22.138+0000 7f2a71796700 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f2a6c0d49f0 0x7f2a6c1f3bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:22.143 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:22.138+0000 7f2a71796700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a6c15aba0 con 0x7f2a6c0d0c50 2023-11-24T07:58:22.143 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:22.138+0000 7f2a71796700 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a6c0b5e90 con 0x7f2a6c0d0870 2023-11-24T07:58:22.144 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:22.138+0000 7f2a71796700 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2a6c09dc10 con 0x7f2a6c0d49f0 2023-11-24T07:58:22.144 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:22.138+0000 7f2a6a7fc700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2a6c0d0c50 0x7f2a6c0b5950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:22.144 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:22.138+0000 7f2a6a7fc700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2a6c0d0c50 0x7f2a6c0b5950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:53256/0 (socket says 172.21.15.59:53256) 2023-11-24T07:58:22.144 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:22.138+0000 7f2a6a7fc700 1 -- 172.21.15.59:0/4221222800 learned_addr learned my addr 172.21.15.59:0/4221222800 (peer_addr_for_me v2:172.21.15.59:0/0) 2023-11-24T07:58:22.144 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:22.138+0000 7f2a6affd700 1 --2- 172.21.15.59:0/4221222800 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2a6c0d0870 0x7f2a6c15a660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:22.144 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:22.138+0000 7f2a6b7fe700 1 --2- 172.21.15.59:0/4221222800 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f2a6c0d49f0 0x7f2a6c1f3bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:22.144 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:22.138+0000 7f2a6a7fc700 1 -- 172.21.15.59:0/4221222800 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f2a6c0d49f0 msgr2=0x7f2a6c1f3bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:22.144 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:22.138+0000 7f2a6a7fc700 1 --2- 172.21.15.59:0/4221222800 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f2a6c0d49f0 0x7f2a6c1f3bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:22.144 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:22.138+0000 7f2a6a7fc700 1 -- 172.21.15.59:0/4221222800 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2a6c0d0870 msgr2=0x7f2a6c15a660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:22.144 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:22.138+0000 7f2a6a7fc700 1 --2- 172.21.15.59:0/4221222800 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2a6c0d0870 0x7f2a6c15a660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:22.144 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:22.138+0000 7f2a6a7fc700 1 -- 172.21.15.59:0/4221222800 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2a60009550 con 0x7f2a6c0d0c50 2023-11-24T07:58:22.144 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:22.138+0000 7f2a6a7fc700 1 --2- 172.21.15.59:0/4221222800 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2a6c0d0c50 0x7f2a6c0b5950 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7f2a5c00c4a0 tx=0x7f2a5c00c860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:58:22.144 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:22.138+0000 7f2a537fe700 1 -- 172.21.15.59:0/4221222800 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a5c011790 con 0x7f2a6c0d0c50 2023-11-24T07:58:22.145 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:22.138+0000 7f2a537fe700 1 -- 172.21.15.59:0/4221222800 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2a5c0118f0 con 0x7f2a6c0d0c50 2023-11-24T07:58:22.145 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:22.138+0000 7f2a71796700 1 -- 172.21.15.59:0/4221222800 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2a6c1f44e0 con 0x7f2a6c0d0c50 2023-11-24T07:58:22.145 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:22.138+0000 7f2a537fe700 1 -- 172.21.15.59:0/4221222800 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a5c011ca0 con 0x7f2a6c0d0c50 2023-11-24T07:58:22.145 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:22.138+0000 7f2a71796700 1 -- 172.21.15.59:0/4221222800 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2a6c1f4aa0 con 0x7f2a6c0d0c50 2023-11-24T07:58:22.145 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:22.139+0000 7f2a71796700 1 -- 172.21.15.59:0/4221222800 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2a6c0b57c0 con 0x7f2a6c0d0c50 2023-11-24T07:58:22.145 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:22.140+0000 7f2a537fe700 1 -- 172.21.15.59:0/4221222800 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 8) v1 ==== 89435+0+0 (secure 0 0 0) 0x7f2a5c01c460 con 0x7f2a6c0d0c50 2023-11-24T07:58:22.145 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:22.140+0000 7f2a537fe700 1 --2- 172.21.15.59:0/4221222800 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f2a5806b110 0x7f2a5806d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:22.145 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:22.140+0000 7f2a537fe700 1 -- 172.21.15.59:0/4221222800 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5854+0+0 (secure 0 0 0) 0x7f2a5c089820 con 0x7f2a6c0d0c50 2023-11-24T07:58:22.145 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:22.140+0000 7f2a6affd700 1 --2- 172.21.15.59:0/4221222800 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f2a5806b110 0x7f2a5806d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:22.145 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:22.141+0000 7f2a6affd700 1 --2- 172.21.15.59:0/4221222800 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f2a5806b110 0x7f2a5806d5d0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f2a60009130 tx=0x7f2a60008030 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:58:22.146 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:22.145+0000 7f2a537fe700 1 -- 172.21.15.59:0/4221222800 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f2a5c0545b0 con 0x7f2a6c0d0c50 2023-11-24T07:58:22.345 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:22.343+0000 7f2a71796700 1 -- 172.21.15.59:0/4221222800 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f2a6c067a20 con 0x7f2a6c0d0c50 2023-11-24T07:58:22.345 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:22.344+0000 7f2a537fe700 1 -- 172.21.15.59:0/4221222800 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v47) v1 ==== 100+0+11 (secure 0 0 0) 0x7f2a5c057bd0 con 0x7f2a6c0d0c50 2023-11-24T07:58:22.345 INFO:teuthology.orchestra.run.smithi059.stdout:pg_num: 64 2023-11-24T07:58:22.348 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:22.346+0000 7f2a71796700 1 -- 172.21.15.59:0/4221222800 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f2a5806b110 msgr2=0x7f2a5806d5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:22.348 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:22.346+0000 7f2a71796700 1 --2- 172.21.15.59:0/4221222800 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f2a5806b110 0x7f2a5806d5d0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f2a60009130 tx=0x7f2a60008030 comp rx=0 tx=0).stop 2023-11-24T07:58:22.348 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:22.346+0000 7f2a71796700 1 -- 172.21.15.59:0/4221222800 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2a6c0d0c50 msgr2=0x7f2a6c0b5950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:22.348 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:22.346+0000 7f2a71796700 1 --2- 172.21.15.59:0/4221222800 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2a6c0d0c50 0x7f2a6c0b5950 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7f2a5c00c4a0 tx=0x7f2a5c00c860 comp rx=0 tx=0).stop 2023-11-24T07:58:22.348 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:22.346+0000 7f2a71796700 1 -- 172.21.15.59:0/4221222800 shutdown_connections 2023-11-24T07:58:22.348 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:22.346+0000 7f2a71796700 1 --2- 172.21.15.59:0/4221222800 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f2a5806b110 0x7f2a5806d5d0 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:22.348 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:22.346+0000 7f2a71796700 1 --2- 172.21.15.59:0/4221222800 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f2a6c0d49f0 0x7f2a6c1f3bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:22.348 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:22.346+0000 7f2a71796700 1 --2- 172.21.15.59:0/4221222800 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2a6c0d0870 0x7f2a6c15a660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:22.348 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:22.346+0000 7f2a71796700 1 --2- 172.21.15.59:0/4221222800 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2a6c0d0c50 0x7f2a6c0b5950 unknown :-1 s=CLOSED pgs=613 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:22.349 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:22.346+0000 7f2a71796700 1 -- 172.21.15.59:0/4221222800 >> 172.21.15.59:0/4221222800 conn(0x7f2a6c0d3ed0 msgr2=0x7f2a6c0c75e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:58:22.349 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:22.346+0000 7f2a71796700 1 -- 172.21.15.59:0/4221222800 shutdown_connections 2023-11-24T07:58:22.349 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:22.346+0000 7f2a71796700 1 -- 172.21.15.59:0/4221222800 wait complete. 2023-11-24T07:58:22.360 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-11-24T07:58:22.491 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:22.490+0000 7fe76c630700 1 Processor -- start 2023-11-24T07:58:22.492 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:22.490+0000 7fe76c630700 1 -- start start 2023-11-24T07:58:22.495 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:22.491+0000 7fe76c630700 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fe76415e000 0x7fe76406eef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:22.495 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:22.491+0000 7fe76c630700 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fe7640d0ae0 0x7fe76406f6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:22.495 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:22.491+0000 7fe76c630700 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe7640cfd60 con 0x7fe7640d0f60 2023-11-24T07:58:22.495 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:22.491+0000 7fe76c630700 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe7640cf4b0 con 0x7fe7640d0ae0 2023-11-24T07:58:22.495 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:22.491+0000 7fe76c630700 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe76406f430 con 0x7fe76415e000 2023-11-24T07:58:22.495 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:22.491+0000 7fe76abcd700 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7fe7640d0f60 0x7fe7640cc220 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:56740/0 (socket says 172.21.15.59:56740) 2023-11-24T07:58:22.495 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:22.491+0000 7fe76abcd700 1 -- 172.21.15.59:0/2070755378 learned_addr learned my addr 172.21.15.59:0/2070755378 (peer_addr_for_me v1:172.21.15.59:0/0) 2023-11-24T07:58:22.495 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:22.491+0000 7fe769bcb700 1 --2- 172.21.15.59:0/2070755378 >> v2:172.21.15.76:3300/0 conn(0x7fe7640d0ae0 0x7fe76406f6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:22.495 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:22.491+0000 7fe76a3cc700 1 --2- 172.21.15.59:0/2070755378 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fe76415e000 0x7fe76406eef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:22.495 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:22.491+0000 7fe7693ca700 1 -- 172.21.15.59:0/2070755378 <== mon.0 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 61868470 0 0) 0x7fe7640cfd60 con 0x7fe7640d0f60 2023-11-24T07:58:22.496 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:22.491+0000 7fe7693ca700 1 -- 172.21.15.59:0/2070755378 --> v1:172.21.15.59:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe7500035b0 con 0x7fe7640d0f60 2023-11-24T07:58:22.496 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:22.491+0000 7fe7693ca700 1 -- 172.21.15.59:0/2070755378 <== mon.0 v1:172.21.15.59:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2281363442 0 0) 0x7fe7500035b0 con 0x7fe7640d0f60 2023-11-24T07:58:22.496 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:22.491+0000 7fe7693ca700 1 -- 172.21.15.59:0/2070755378 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fe76415e000 msgr2=0x7fe76406eef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T07:58:22.496 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:22.491+0000 7fe7693ca700 1 --2- 172.21.15.59:0/2070755378 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fe76415e000 0x7fe76406eef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:22.496 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:22.491+0000 7fe7693ca700 1 -- 172.21.15.59:0/2070755378 >> v2:172.21.15.76:3300/0 conn(0x7fe7640d0ae0 msgr2=0x7fe76406f6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T07:58:22.496 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:22.491+0000 7fe7693ca700 1 --2- 172.21.15.59:0/2070755378 >> v2:172.21.15.76:3300/0 conn(0x7fe7640d0ae0 0x7fe76406f6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:22.496 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:22.491+0000 7fe7693ca700 1 -- 172.21.15.59:0/2070755378 --> v1:172.21.15.59:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe7640cf1d0 con 0x7fe7640d0f60 2023-11-24T07:58:22.496 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:22.491+0000 7fe7693ca700 1 -- 172.21.15.59:0/2070755378 <== mon.0 v1:172.21.15.59:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1312333566 0 0) 0x7fe75c0019f0 con 0x7fe7640d0f60 2023-11-24T07:58:22.496 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:22.491+0000 7fe7693ca700 1 -- 172.21.15.59:0/2070755378 >> v1:172.21.15.59:6789/0 conn(0x7fe7640d0f60 legacy=0x7fe7640cc220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:22.496 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:22.491+0000 7fe7693ca700 1 --2- 172.21.15.59:0/2070755378 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe750003a00 0x7fe750003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:22.496 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:22.491+0000 7fe7693ca700 1 --2- 172.21.15.59:0/2070755378 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fe76415e000 0x7fe750004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:22.496 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:22.491+0000 7fe7693ca700 1 --2- 172.21.15.59:0/2070755378 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe750004d90 0x7fe750005210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:22.496 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:22.491+0000 7fe7693ca700 1 -- 172.21.15.59:0/2070755378 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7640cf4b0 con 0x7fe750003a00 2023-11-24T07:58:22.497 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:22.491+0000 7fe7693ca700 1 -- 172.21.15.59:0/2070755378 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe76406f430 con 0x7fe750004d90 2023-11-24T07:58:22.497 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:22.491+0000 7fe7693ca700 1 -- 172.21.15.59:0/2070755378 --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe75000ba10 con 0x7fe76415e000 2023-11-24T07:58:22.497 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:22.491+0000 7fe76a3cc700 1 --2- 172.21.15.59:0/2070755378 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe750003a00 0x7fe750003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:22.497 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:22.492+0000 7fe769bcb700 1 --2- 172.21.15.59:0/2070755378 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fe76415e000 0x7fe750004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:22.497 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:22.492+0000 7fe76abcd700 1 --2- 172.21.15.59:0/2070755378 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe750004d90 0x7fe750005210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:22.497 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:22.492+0000 7fe76a3cc700 1 -- 172.21.15.59:0/2070755378 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fe76415e000 msgr2=0x7fe750004850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T07:58:22.497 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:22.492+0000 7fe76a3cc700 1 --2- 172.21.15.59:0/2070755378 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fe76415e000 0x7fe750004850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:22.497 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:22.492+0000 7fe76a3cc700 1 -- 172.21.15.59:0/2070755378 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe750004d90 msgr2=0x7fe750005210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T07:58:22.497 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:22.492+0000 7fe76a3cc700 1 --2- 172.21.15.59:0/2070755378 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe750004d90 0x7fe750005210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:22.497 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:22.492+0000 7fe76a3cc700 1 -- 172.21.15.59:0/2070755378 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe7500035b0 con 0x7fe750003a00 2023-11-24T07:58:22.497 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:22.492+0000 7fe76abcd700 1 --2- 172.21.15.59:0/2070755378 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe750004d90 0x7fe750005210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-24T07:58:22.497 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:22.492+0000 7fe76a3cc700 1 --2- 172.21.15.59:0/2070755378 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe750003a00 0x7fe750003e20 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7fe754009b60 tx=0x7fe750005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=93c11e6e8867ba8b server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:58:22.497 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:22.492+0000 7fe7693ca700 1 -- 172.21.15.59:0/2070755378 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe75400e6c0 con 0x7fe750003a00 2023-11-24T07:58:22.498 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:22.492+0000 7fe7693ca700 1 -- 172.21.15.59:0/2070755378 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe75400e820 con 0x7fe750003a00 2023-11-24T07:58:22.498 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:22.492+0000 7fe7693ca700 1 -- 172.21.15.59:0/2070755378 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe75400eb90 con 0x7fe750003a00 2023-11-24T07:58:22.498 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:22.493+0000 7fe76c630700 1 -- 172.21.15.59:0/2070755378 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe750003a00 msgr2=0x7fe750003e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:22.498 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:22.493+0000 7fe76c630700 1 --2- 172.21.15.59:0/2070755378 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe750003a00 0x7fe750003e20 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7fe754009b60 tx=0x7fe750005880 comp rx=0 tx=0).stop 2023-11-24T07:58:22.498 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:22.493+0000 7fe76abcd700 1 -- 172.21.15.59:0/2070755378 reap_dead start 2023-11-24T07:58:22.498 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:22.493+0000 7fe76c630700 1 -- 172.21.15.59:0/2070755378 shutdown_connections 2023-11-24T07:58:22.498 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:22.493+0000 7fe76c630700 1 -- 172.21.15.59:0/2070755378 >> 172.21.15.59:0/2070755378 conn(0x7fe7640b3720 msgr2=0x7fe7640b3b30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:58:22.498 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:22.493+0000 7fe76c630700 1 -- 172.21.15.59:0/2070755378 shutdown_connections 2023-11-24T07:58:22.498 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:22.493+0000 7fe76c630700 1 -- 172.21.15.59:0/2070755378 wait complete. 2023-11-24T07:58:22.501 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:22.494+0000 7fe76c630700 1 Processor -- start 2023-11-24T07:58:22.501 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:22.494+0000 7fe76c630700 1 -- start start 2023-11-24T07:58:22.501 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:22.495+0000 7fe76c630700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe750003a00 0x7fe7640b4c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:22.501 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:22.495+0000 7fe76c630700 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fe7641f7060 0x7fe76414f0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:22.501 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:22.495+0000 7fe76c630700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe7641f4180 0x7fe7641f4630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:22.501 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:22.495+0000 7fe76c630700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7640b7920 con 0x7fe7641f4180 2023-11-24T07:58:22.501 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:22.495+0000 7fe76c630700 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7641f8030 con 0x7fe750003a00 2023-11-24T07:58:22.501 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:22.495+0000 7fe76c630700 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe7641f3a30 con 0x7fe7641f7060 2023-11-24T07:58:22.501 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:22.495+0000 7fe76abcd700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe7641f4180 0x7fe7641f4630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:22.502 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:22.495+0000 7fe76abcd700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe7641f4180 0x7fe7641f4630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:53278/0 (socket says 172.21.15.59:53278) 2023-11-24T07:58:22.502 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:22.495+0000 7fe76abcd700 1 -- 172.21.15.59:0/101266506 learned_addr learned my addr 172.21.15.59:0/101266506 (peer_addr_for_me v2:172.21.15.59:0/0) 2023-11-24T07:58:22.502 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:22.495+0000 7fe76a3cc700 1 --2- 172.21.15.59:0/101266506 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe750003a00 0x7fe7640b4c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:22.502 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:22.495+0000 7fe769bcb700 1 --2- 172.21.15.59:0/101266506 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fe7641f7060 0x7fe76414f0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:22.502 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:22.495+0000 7fe76abcd700 1 -- 172.21.15.59:0/101266506 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fe7641f7060 msgr2=0x7fe76414f0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:22.502 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:22.495+0000 7fe76abcd700 1 --2- 172.21.15.59:0/101266506 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fe7641f7060 0x7fe76414f0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:22.502 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:22.495+0000 7fe76abcd700 1 -- 172.21.15.59:0/101266506 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe750003a00 msgr2=0x7fe7640b4c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:22.502 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:22.495+0000 7fe76abcd700 1 --2- 172.21.15.59:0/101266506 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe750003a00 0x7fe7640b4c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:22.502 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:22.495+0000 7fe76abcd700 1 -- 172.21.15.59:0/101266506 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe76414f5f0 con 0x7fe7641f4180 2023-11-24T07:58:22.503 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:22.495+0000 7fe76abcd700 1 --2- 172.21.15.59:0/101266506 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe7641f4180 0x7fe7641f4630 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7fe764071990 tx=0x7fe75c00e440 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:58:22.503 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:22.496+0000 7fe75affd700 1 -- 172.21.15.59:0/101266506 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe75c00ebb0 con 0x7fe7641f4180 2023-11-24T07:58:22.503 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:22.496+0000 7fe75affd700 1 -- 172.21.15.59:0/101266506 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe75c00ed10 con 0x7fe7641f4180 2023-11-24T07:58:22.503 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:22.496+0000 7fe76c630700 1 -- 172.21.15.59:0/101266506 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe7641f74e0 con 0x7fe7641f4180 2023-11-24T07:58:22.503 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:22.496+0000 7fe75affd700 1 -- 172.21.15.59:0/101266506 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe75c013660 con 0x7fe7641f4180 2023-11-24T07:58:22.503 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:22.496+0000 7fe76c630700 1 -- 172.21.15.59:0/101266506 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe7641f7a70 con 0x7fe7641f4180 2023-11-24T07:58:22.503 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:22.497+0000 7fe76c630700 1 -- 172.21.15.59:0/101266506 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe764068670 con 0x7fe7641f4180 2023-11-24T07:58:22.503 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:22.497+0000 7fe75affd700 1 -- 172.21.15.59:0/101266506 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 8) v1 ==== 89435+0+0 (secure 0 0 0) 0x7fe75c0137c0 con 0x7fe7641f4180 2023-11-24T07:58:22.503 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:22.497+0000 7fe75affd700 1 --2- 172.21.15.59:0/101266506 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7fe74806b110 0x7fe74806d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:22.503 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:22.498+0000 7fe75affd700 1 -- 172.21.15.59:0/101266506 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 5910+0+0 (secure 0 0 0) 0x7fe75c089f00 con 0x7fe7641f4180 2023-11-24T07:58:22.503 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:22.498+0000 7fe76a3cc700 1 --2- 172.21.15.59:0/101266506 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7fe74806b110 0x7fe74806d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:22.503 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:22.498+0000 7fe76a3cc700 1 --2- 172.21.15.59:0/101266506 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7fe74806b110 0x7fe74806d5d0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7fe7540090b0 tx=0x7fe75401f040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:58:22.504 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:22.502+0000 7fe75affd700 1 -- 172.21.15.59:0/101266506 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fe75c054c50 con 0x7fe7641f4180 2023-11-24T07:58:22.724 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:22.722+0000 7fe76c630700 1 -- 172.21.15.59:0/101266506 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7fe76415d240 con 0x7fe7641f4180 2023-11-24T07:58:22.724 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:22.723+0000 7fe75affd700 1 -- 172.21.15.59:0/101266506 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v48) v1 ==== 80+0+0 (secure 0 0 0) 0x7fe75c058270 con 0x7fe7641f4180 2023-11-24T07:58:22.726 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:22.725+0000 7fe76c630700 1 -- 172.21.15.59:0/101266506 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7fe74806b110 msgr2=0x7fe74806d5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:22.727 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:22.725+0000 7fe76c630700 1 --2- 172.21.15.59:0/101266506 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7fe74806b110 0x7fe74806d5d0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7fe7540090b0 tx=0x7fe75401f040 comp rx=0 tx=0).stop 2023-11-24T07:58:22.727 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:22.725+0000 7fe76c630700 1 -- 172.21.15.59:0/101266506 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe7641f4180 msgr2=0x7fe7641f4630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:22.727 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:22.725+0000 7fe76c630700 1 --2- 172.21.15.59:0/101266506 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe7641f4180 0x7fe7641f4630 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7fe764071990 tx=0x7fe75c00e440 comp rx=0 tx=0).stop 2023-11-24T07:58:22.727 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:22.725+0000 7fe76c630700 1 -- 172.21.15.59:0/101266506 shutdown_connections 2023-11-24T07:58:22.727 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:22.725+0000 7fe76c630700 1 --2- 172.21.15.59:0/101266506 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7fe74806b110 0x7fe74806d5d0 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:22.727 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:22.725+0000 7fe76c630700 1 --2- 172.21.15.59:0/101266506 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe7641f4180 0x7fe7641f4630 unknown :-1 s=CLOSED pgs=616 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:22.727 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:22.725+0000 7fe76c630700 1 --2- 172.21.15.59:0/101266506 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe750003a00 0x7fe7640b4c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:22.727 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:22.725+0000 7fe76c630700 1 --2- 172.21.15.59:0/101266506 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fe7641f7060 0x7fe76414f0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:22.727 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:22.725+0000 7fe76c630700 1 -- 172.21.15.59:0/101266506 >> 172.21.15.59:0/101266506 conn(0x7fe7640b3720 msgr2=0x7fe7640b0070 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:58:22.727 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:22.725+0000 7fe76c630700 1 -- 172.21.15.59:0/101266506 shutdown_connections 2023-11-24T07:58:22.727 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:22.725+0000 7fe76c630700 1 -- 172.21.15.59:0/101266506 wait complete. 2023-11-24T07:58:22.728 INFO:teuthology.orchestra.run.smithi059.stderr:listed 0 entries 2023-11-24T07:58:22.738 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.1... 2023-11-24T07:58:22.739 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T07:58:22.739 DEBUG:teuthology.orchestra.run.smithi076:> sudo umount /home/ubuntu/cephtest/mnt.1 2023-11-24T07:58:22.791 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi076.front.sepia.ceph.com 2023-11-24T07:58:22.792 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T07:58:22.792 DEBUG:teuthology.orchestra.run.smithi076:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2023-11-24T07:58:22.812 DEBUG:teuthology.orchestra.run.smithi059:> set -ex 2023-11-24T07:58:22.812 DEBUG:teuthology.orchestra.run.smithi059:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-24T07:58:22.830 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=16287364k,nr_inodes=4071841,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 tmpfs /sys/fs/cgroup tmpfs ro,seclabel,nosuid,nodev,noexec,mode=755 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,seclabel,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,seclabel,nosuid,nodev,noexec,relatime,hugetlb 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,seclabel,nosuid,nodev,noexec,relatime,perf_event 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,seclabel,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/rdma cgroup rw,seclabel,nosuid,nodev,noexec,relatime,rdma 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,seclabel,nosuid,nodev,noexec,relatime,freezer 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,seclabel,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,seclabel,nosuid,nodev,noexec,relatime,devices 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,blkio 0 0 none /sys/kernel/tracing tracefs rw,seclabel,relatime 0 0 configfs /sys/kernel/config configfs rw,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=42,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=16075 0 0 mqueue /dev/mqueue mqueue rw,seclabel,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,relatime 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3261172k,mode=700,uid=1000,gid=1265 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw,seclabel 0 0 2023-11-24T07:58:22.830 DEBUG:teuthology.orchestra.run.smithi076:> set -ex 2023-11-24T07:58:22.830 DEBUG:teuthology.orchestra.run.smithi076:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-24T07:58:22.872 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=16284272k,nr_inodes=4071068,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 tmpfs /sys/fs/cgroup tmpfs ro,seclabel,nosuid,nodev,noexec,mode=755 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,seclabel,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,seclabel,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,seclabel,nosuid,nodev,noexec,relatime,freezer 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,seclabel,nosuid,nodev,noexec,relatime,perf_event 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,seclabel,nosuid,nodev,noexec,relatime,hugetlb 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,seclabel,nosuid,nodev,noexec,relatime,devices 0 0 cgroup /sys/fs/cgroup/rdma cgroup rw,seclabel,nosuid,nodev,noexec,relatime,rdma 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,seclabel,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,blkio 0 0 none /sys/kernel/tracing tracefs rw,seclabel,relatime 0 0 configfs /sys/kernel/config configfs rw,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=35,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=20116 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,relatime 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3260552k,mode=700,uid=1000,gid=1265 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw,seclabel 0 0 2023-11-24T07:58:22.873 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-24T07:58:22.997 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:22.995+0000 7fd5c4f4d700 1 Processor -- start 2023-11-24T07:58:22.997 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:22.995+0000 7fd5c4f4d700 1 -- start start 2023-11-24T07:58:23.001 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:22.996+0000 7fd5c4f4d700 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fd5c00d0ec0 0x7fd5c00af400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:23.001 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:22.996+0000 7fd5c4f4d700 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd5c00af940 con 0x7fd5c015be40 2023-11-24T07:58:23.001 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:22.996+0000 7fd5c4f4d700 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd5c00b0ab0 con 0x7fd5c006b7c0 2023-11-24T07:58:23.001 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:22.996+0000 7fd5c4f4d700 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd5c00b1130 con 0x7fd5c00d0ec0 2023-11-24T07:58:23.002 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:22.996+0000 7fd5be59c700 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7fd5c015be40 0x7fd5c00b1550 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:56756/0 (socket says 172.21.15.59:56756) 2023-11-24T07:58:23.002 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:22.996+0000 7fd5be59c700 1 -- 172.21.15.59:0/916876911 learned_addr learned my addr 172.21.15.59:0/916876911 (peer_addr_for_me v1:172.21.15.59:0/0) 2023-11-24T07:58:23.002 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:22.997+0000 7fd5bdd9b700 1 --2- 172.21.15.59:0/916876911 >> v2:172.21.15.76:3300/0 conn(0x7fd5c00d0ec0 0x7fd5c00af400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:23.002 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:22.997+0000 7fd5bd59a700 1 -- 172.21.15.59:0/916876911 <== mon.0 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 880620684 0 0) 0x7fd5b4001110 con 0x7fd5c015be40 2023-11-24T07:58:23.002 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:22.997+0000 7fd5bd59a700 1 -- 172.21.15.59:0/916876911 --> v1:172.21.15.59:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd5a80035b0 con 0x7fd5c015be40 2023-11-24T07:58:23.002 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:22.997+0000 7fd5bd59a700 1 -- 172.21.15.59:0/916876911 <== mon.0 v1:172.21.15.59:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 963895540 0 0) 0x7fd5a80035b0 con 0x7fd5c015be40 2023-11-24T07:58:23.002 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:22.997+0000 7fd5bd59a700 1 -- 172.21.15.59:0/916876911 >> v2:172.21.15.76:3300/0 conn(0x7fd5c00d0ec0 msgr2=0x7fd5c00af400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T07:58:23.002 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:22.997+0000 7fd5bd59a700 1 --2- 172.21.15.59:0/916876911 >> v2:172.21.15.76:3300/0 conn(0x7fd5c00d0ec0 0x7fd5c00af400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:23.002 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:22.997+0000 7fd5bd59a700 1 -- 172.21.15.59:0/916876911 >> v1:172.21.15.76:6789/0 conn(0x7fd5c006b7c0 legacy=0x7fd5c015c220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:23.002 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:22.997+0000 7fd5bd59a700 1 -- 172.21.15.59:0/916876911 --> v1:172.21.15.59:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd5c00b1c60 con 0x7fd5c015be40 2023-11-24T07:58:23.002 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:22.997+0000 7fd5bd59a700 1 -- 172.21.15.59:0/916876911 <== mon.0 v1:172.21.15.59:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1312333566 0 0) 0x7fd5b4001800 con 0x7fd5c015be40 2023-11-24T07:58:23.003 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:22.997+0000 7fd5bd59a700 1 -- 172.21.15.59:0/916876911 >> v1:172.21.15.59:6789/0 conn(0x7fd5c015be40 legacy=0x7fd5c00b1550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:23.003 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:22.997+0000 7fd5bd59a700 1 --2- 172.21.15.59:0/916876911 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fd5a8003a00 0x7fd5a8003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:23.003 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:22.997+0000 7fd5bd59a700 1 --2- 172.21.15.59:0/916876911 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd5a8004850 0x7fd5a8004cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:23.003 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:22.997+0000 7fd5bd59a700 1 --2- 172.21.15.59:0/916876911 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd5a8005210 0x7fd5a800ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:23.003 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:22.997+0000 7fd5bd59a700 1 -- 172.21.15.59:0/916876911 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5c00b1130 con 0x7fd5a8005210 2023-11-24T07:58:23.003 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:22.997+0000 7fd5bd59a700 1 -- 172.21.15.59:0/916876911 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5a800c0b0 con 0x7fd5a8004850 2023-11-24T07:58:23.003 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:22.997+0000 7fd5bd59a700 1 -- 172.21.15.59:0/916876911 --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd5a800c270 con 0x7fd5a8003a00 2023-11-24T07:58:23.003 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:22.997+0000 7fd5bed9d700 1 --2- 172.21.15.59:0/916876911 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd5a8005210 0x7fd5a800ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:23.003 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:22.997+0000 7fd5bdd9b700 1 --2- 172.21.15.59:0/916876911 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd5a8004850 0x7fd5a8004cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:23.003 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:22.997+0000 7fd5be59c700 1 --2- 172.21.15.59:0/916876911 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fd5a8003a00 0x7fd5a8003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:23.003 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:22.997+0000 7fd5bed9d700 1 -- 172.21.15.59:0/916876911 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fd5a8003a00 msgr2=0x7fd5a8003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T07:58:23.003 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:22.997+0000 7fd5bed9d700 1 --2- 172.21.15.59:0/916876911 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fd5a8003a00 0x7fd5a8003e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:23.004 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:22.997+0000 7fd5bed9d700 1 -- 172.21.15.59:0/916876911 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd5a8004850 msgr2=0x7fd5a8004cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T07:58:23.004 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:22.997+0000 7fd5bed9d700 1 --2- 172.21.15.59:0/916876911 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd5a8004850 0x7fd5a8004cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:23.004 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:22.998+0000 7fd5bed9d700 1 -- 172.21.15.59:0/916876911 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd5a80035b0 con 0x7fd5a8005210 2023-11-24T07:58:23.004 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:22.998+0000 7fd5be59c700 1 --2- 172.21.15.59:0/916876911 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fd5a8003a00 0x7fd5a8003e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-24T07:58:23.004 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:22.998+0000 7fd5bdd9b700 1 --2- 172.21.15.59:0/916876911 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd5a8004850 0x7fd5a8004cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-24T07:58:23.004 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:22.998+0000 7fd5bed9d700 1 -- 172.21.15.59:0/916876911 reap_dead start 2023-11-24T07:58:23.004 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:22.998+0000 7fd5bed9d700 1 --2- 172.21.15.59:0/916876911 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd5a8005210 0x7fd5a800ba10 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7fd5b800c190 tx=0x7fd5a8005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=14702cb8cf4fef18 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:58:23.004 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:22.998+0000 7fd5bd59a700 1 -- 172.21.15.59:0/916876911 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5b8001db0 con 0x7fd5a8005210 2023-11-24T07:58:23.004 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:22.998+0000 7fd5bd59a700 1 -- 172.21.15.59:0/916876911 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd5b800ccf0 con 0x7fd5a8005210 2023-11-24T07:58:23.004 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:22.998+0000 7fd5bd59a700 1 -- 172.21.15.59:0/916876911 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5b8003c40 con 0x7fd5a8005210 2023-11-24T07:58:23.004 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:22.998+0000 7fd5c4f4d700 1 -- 172.21.15.59:0/916876911 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd5a8005210 msgr2=0x7fd5a800ba10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:23.004 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:22.998+0000 7fd5c4f4d700 1 --2- 172.21.15.59:0/916876911 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd5a8005210 0x7fd5a800ba10 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7fd5b800c190 tx=0x7fd5a8005880 comp rx=0 tx=0).stop 2023-11-24T07:58:23.005 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:22.998+0000 7fd5c4f4d700 1 -- 172.21.15.59:0/916876911 shutdown_connections 2023-11-24T07:58:23.005 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:22.998+0000 7fd5c4f4d700 1 --2- 172.21.15.59:0/916876911 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd5a8005210 0x7fd5a800ba10 unknown :-1 s=CLOSED pgs=618 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:23.005 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:22.998+0000 7fd5c4f4d700 1 -- 172.21.15.59:0/916876911 >> 172.21.15.59:0/916876911 conn(0x7fd5c00717f0 msgr2=0x7fd5c00b5070 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:58:23.005 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:22.998+0000 7fd5c4f4d700 1 -- 172.21.15.59:0/916876911 shutdown_connections 2023-11-24T07:58:23.005 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:22.998+0000 7fd5c4f4d700 1 -- 172.21.15.59:0/916876911 wait complete. 2023-11-24T07:58:23.005 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:23.000+0000 7fd5c4f4d700 1 Processor -- start 2023-11-24T07:58:23.005 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:23.000+0000 7fd5c4f4d700 1 -- start start 2023-11-24T07:58:23.007 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:23.000+0000 7fd5c4f4d700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd5a8005210 0x7fd5c0157e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:23.007 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:23.000+0000 7fd5c4f4d700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd5c0154750 0x7fd5c01f12a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:23.007 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:23.000+0000 7fd5c4f4d700 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fd5c01435e0 0x7fd5c0143a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:23.008 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:23.001+0000 7fd5c4f4d700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5c0144560 con 0x7fd5a8005210 2023-11-24T07:58:23.008 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:23.001+0000 7fd5c4f4d700 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5c0156d10 con 0x7fd5c0154750 2023-11-24T07:58:23.008 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:23.001+0000 7fd5c4f4d700 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd5c0141680 con 0x7fd5c01435e0 2023-11-24T07:58:23.008 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:23.001+0000 7fd5be59c700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd5a8005210 0x7fd5c0157e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:23.008 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:23.001+0000 7fd5be59c700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd5a8005210 0x7fd5c0157e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:53294/0 (socket says 172.21.15.59:53294) 2023-11-24T07:58:23.008 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:23.001+0000 7fd5be59c700 1 -- 172.21.15.59:0/3726515334 learned_addr learned my addr 172.21.15.59:0/3726515334 (peer_addr_for_me v2:172.21.15.59:0/0) 2023-11-24T07:58:23.008 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:23.001+0000 7fd5bdd9b700 1 --2- 172.21.15.59:0/3726515334 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd5c0154750 0x7fd5c01f12a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:23.008 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:23.001+0000 7fd5bed9d700 1 --2- 172.21.15.59:0/3726515334 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fd5c01435e0 0x7fd5c0143a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:23.008 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:23.001+0000 7fd5be59c700 1 -- 172.21.15.59:0/3726515334 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fd5c01435e0 msgr2=0x7fd5c0143a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:23.008 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:23.001+0000 7fd5be59c700 1 --2- 172.21.15.59:0/3726515334 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fd5c01435e0 0x7fd5c0143a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:23.008 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:23.001+0000 7fd5be59c700 1 -- 172.21.15.59:0/3726515334 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd5c0154750 msgr2=0x7fd5c01f12a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:23.009 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:23.001+0000 7fd5be59c700 1 --2- 172.21.15.59:0/3726515334 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd5c0154750 0x7fd5c01f12a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:23.009 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:23.001+0000 7fd5be59c700 1 -- 172.21.15.59:0/3726515334 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd5c0141960 con 0x7fd5a8005210 2023-11-24T07:58:23.009 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:23.001+0000 7fd5be59c700 1 --2- 172.21.15.59:0/3726515334 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd5a8005210 0x7fd5c0157e00 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7fd5b400eb20 tx=0x7fd5b400ee30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:58:23.009 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:23.001+0000 7fd5aeffd700 1 -- 172.21.15.59:0/3726515334 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5b4005e00 con 0x7fd5a8005210 2023-11-24T07:58:23.009 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:23.001+0000 7fd5aeffd700 1 -- 172.21.15.59:0/3726515334 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd5b400c5d0 con 0x7fd5a8005210 2023-11-24T07:58:23.009 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:23.001+0000 7fd5c4f4d700 1 -- 172.21.15.59:0/3726515334 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd5c0141c40 con 0x7fd5a8005210 2023-11-24T07:58:23.009 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:23.001+0000 7fd5aeffd700 1 -- 172.21.15.59:0/3726515334 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5b400c940 con 0x7fd5a8005210 2023-11-24T07:58:23.009 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:23.001+0000 7fd5c4f4d700 1 -- 172.21.15.59:0/3726515334 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd5c009a880 con 0x7fd5a8005210 2023-11-24T07:58:23.009 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:23.002+0000 7fd5c4f4d700 1 -- 172.21.15.59:0/3726515334 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd5c009aa40 con 0x7fd5a8005210 2023-11-24T07:58:23.009 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:23.003+0000 7fd5aeffd700 1 -- 172.21.15.59:0/3726515334 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 8) v1 ==== 89435+0+0 (secure 0 0 0) 0x7fd5b400caa0 con 0x7fd5a8005210 2023-11-24T07:58:23.009 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:23.003+0000 7fd5aeffd700 1 --2- 172.21.15.59:0/3726515334 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7fd5a006b110 0x7fd5a006d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:23.009 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:23.003+0000 7fd5aeffd700 1 -- 172.21.15.59:0/3726515334 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 5910+0+0 (secure 0 0 0) 0x7fd5b4089490 con 0x7fd5a8005210 2023-11-24T07:58:23.009 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:23.003+0000 7fd5bdd9b700 1 --2- 172.21.15.59:0/3726515334 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7fd5a006b110 0x7fd5a006d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:23.010 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:23.004+0000 7fd5bdd9b700 1 --2- 172.21.15.59:0/3726515334 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7fd5a006b110 0x7fd5a006d5d0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7fd5c00b4810 tx=0x7fd5b0008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:58:23.010 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:23.008+0000 7fd5aeffd700 1 -- 172.21.15.59:0/3726515334 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fd5b4057c70 con 0x7fd5a8005210 2023-11-24T07:58:23.257 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:23.255+0000 7fd5c4f4d700 1 -- 172.21.15.59:0/3726515334 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fd5c00af940 con 0x7fd5a8005210 2023-11-24T07:58:23.258 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:23.256+0000 7fd5aeffd700 1 -- 172.21.15.59:0/3726515334 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) v1 ==== 94+0+4904 (secure 0 0 0) 0x7fd5c00af940 con 0x7fd5a8005210 2023-11-24T07:58:23.258 INFO:teuthology.orchestra.run.smithi059.stdout: 2023-11-24T07:58:23.258 INFO:teuthology.orchestra.run.smithi059.stdout:{"epoch":26,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5290,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.76:6836/508095662","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6834","nonce":508095662},{"type":"v1","addr":"172.21.15.76:6836","nonce":508095662}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":20},{"gid":5293,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.76:6837/4158118811","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6835","nonce":4158118811},{"type":"v1","addr":"172.21.15.76:6837","nonce":4158118811}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":20},{"gid":5622,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6834/59936286","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6832","nonce":59936286},{"type":"v1","addr":"172.21.15.59:6834","nonce":59936286}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":20}],"filesystems":[{"mdsmap":{"epoch":25,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-24T07:58:02.020496+0000","modified":"2023-11-24T07:58:05.037742+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5625},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5625":{"gid":5625,"name":"c","rank":0,"incarnation":21,"state":"up:active","state_seq":3,"addr":"172.21.15.59:6835/1468266626","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6833","nonce":1468266626},{"type":"v1","addr":"172.21.15.59:6835","nonce":1468266626}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2023-11-24T07:58:23.260 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:23.258+0000 7fd5c4f4d700 1 -- 172.21.15.59:0/3726515334 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7fd5a006b110 msgr2=0x7fd5a006d5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:23.260 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:23.258+0000 7fd5c4f4d700 1 --2- 172.21.15.59:0/3726515334 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7fd5a006b110 0x7fd5a006d5d0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7fd5c00b4810 tx=0x7fd5b0008040 comp rx=0 tx=0).stop 2023-11-24T07:58:23.260 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:23.258+0000 7fd5c4f4d700 1 -- 172.21.15.59:0/3726515334 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd5a8005210 msgr2=0x7fd5c0157e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:23.260 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:23.258+0000 7fd5c4f4d700 1 --2- 172.21.15.59:0/3726515334 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd5a8005210 0x7fd5c0157e00 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7fd5b400eb20 tx=0x7fd5b400ee30 comp rx=0 tx=0).stop 2023-11-24T07:58:23.261 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:23.258+0000 7fd5c4f4d700 1 -- 172.21.15.59:0/3726515334 shutdown_connections 2023-11-24T07:58:23.261 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:23.258+0000 7fd5c4f4d700 1 --2- 172.21.15.59:0/3726515334 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7fd5a006b110 0x7fd5a006d5d0 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:23.261 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:23.258+0000 7fd5c4f4d700 1 --2- 172.21.15.59:0/3726515334 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fd5c01435e0 0x7fd5c0143a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:23.261 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:23.258+0000 7fd5c4f4d700 1 --2- 172.21.15.59:0/3726515334 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd5a8005210 0x7fd5c0157e00 unknown :-1 s=CLOSED pgs=619 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:23.261 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:23.258+0000 7fd5c4f4d700 1 --2- 172.21.15.59:0/3726515334 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd5c0154750 0x7fd5c01f12a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:23.261 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:23.258+0000 7fd5c4f4d700 1 -- 172.21.15.59:0/3726515334 >> 172.21.15.59:0/3726515334 conn(0x7fd5c00717f0 msgr2=0x7fd5c00b1e50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:58:23.261 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:23.258+0000 7fd5c4f4d700 1 -- 172.21.15.59:0/3726515334 shutdown_connections 2023-11-24T07:58:23.261 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:23.258+0000 7fd5c4f4d700 1 -- 172.21.15.59:0/3726515334 wait complete. 2023-11-24T07:58:23.262 INFO:teuthology.orchestra.run.smithi059.stderr:dumped fsmap epoch 26 2023-11-24T07:58:23.273 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-24T07:58:23.397 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:23.396+0000 7f5e13654700 1 Processor -- start 2023-11-24T07:58:23.397 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:23.396+0000 7f5e13654700 1 -- start start 2023-11-24T07:58:23.399 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:23.397+0000 7f5e13654700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f5e0c143ae0 0x7f5e0c0b51b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:23.399 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:23.397+0000 7f5e13654700 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f5e0c1440f0 0x7f5e0c071bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:23.399 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:23.397+0000 7f5e13654700 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5e0c143ec0 con 0x7f5e0c0720f0 2023-11-24T07:58:23.400 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:23.397+0000 7f5e13654700 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5e0c0b40a0 con 0x7f5e0c143ae0 2023-11-24T07:58:23.400 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:23.397+0000 7f5e13654700 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5e0c072560 con 0x7f5e0c1440f0 2023-11-24T07:58:23.400 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:23.397+0000 7f5e113f0700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f5e0c143ae0 0x7f5e0c0b51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:23.400 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:23.397+0000 7f5e11bf1700 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7f5e0c0720f0 0x7f5e0c15e8b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:56780/0 (socket says 172.21.15.59:56780) 2023-11-24T07:58:23.400 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:23.397+0000 7f5e11bf1700 1 -- 172.21.15.59:0/1762301323 learned_addr learned my addr 172.21.15.59:0/1762301323 (peer_addr_for_me v1:172.21.15.59:0/0) 2023-11-24T07:58:23.400 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:23.397+0000 7f5e113f0700 1 -- 172.21.15.59:0/1762301323 >> v2:172.21.15.76:3300/0 conn(0x7f5e0c1440f0 msgr2=0x7f5e0c071bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T07:58:23.400 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:23.397+0000 7f5e113f0700 1 --2- 172.21.15.59:0/1762301323 >> v2:172.21.15.76:3300/0 conn(0x7f5e0c1440f0 0x7f5e0c071bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:23.400 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:23.397+0000 7f5e113f0700 1 -- 172.21.15.59:0/1762301323 >> v1:172.21.15.59:6789/0 conn(0x7f5e0c0720f0 legacy=0x7f5e0c15e8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:23.400 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:23.397+0000 7f5e03fff700 1 -- 172.21.15.59:0/1762301323 <== mon.0 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 883795940 0 0) 0x7f5e0c143ec0 con 0x7f5e0c0720f0 2023-11-24T07:58:23.401 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:23.397+0000 7f5e113f0700 1 -- 172.21.15.59:0/1762301323 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5e0c0b3ae0 con 0x7f5e0c143ae0 2023-11-24T07:58:23.401 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:23.398+0000 7f5e113f0700 1 --2- 172.21.15.59:0/1762301323 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5e0c143ae0 0x7f5e0c0b51b0 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7f5e0c0d1b00 tx=0x7f5e08009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4e393ba0e60ff660 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:58:23.401 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:23.398+0000 7f5e03fff700 1 -- 172.21.15.59:0/1762301323 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e08011830 con 0x7f5e0c143ae0 2023-11-24T07:58:23.401 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:23.398+0000 7f5e03fff700 1 -- 172.21.15.59:0/1762301323 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5e08011990 con 0x7f5e0c143ae0 2023-11-24T07:58:23.401 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:23.398+0000 7f5e03fff700 1 -- 172.21.15.59:0/1762301323 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e08011d20 con 0x7f5e0c143ae0 2023-11-24T07:58:23.406 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:23.398+0000 7f5e13654700 1 -- 172.21.15.59:0/1762301323 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5e0c143ae0 msgr2=0x7f5e0c0b51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:23.406 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:23.398+0000 7f5e13654700 1 --2- 172.21.15.59:0/1762301323 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5e0c143ae0 0x7f5e0c0b51b0 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7f5e0c0d1b00 tx=0x7f5e08009b50 comp rx=0 tx=0).stop 2023-11-24T07:58:23.407 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:23.398+0000 7f5e13654700 1 -- 172.21.15.59:0/1762301323 shutdown_connections 2023-11-24T07:58:23.407 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:23.398+0000 7f5e13654700 1 --2- 172.21.15.59:0/1762301323 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5e0c143ae0 0x7f5e0c0b51b0 unknown :-1 s=CLOSED pgs=621 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:23.407 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:23.398+0000 7f5e13654700 1 --2- 172.21.15.59:0/1762301323 >> v2:172.21.15.76:3300/0 conn(0x7f5e0c1440f0 0x7f5e0c071bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:23.407 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:23.398+0000 7f5e13654700 1 -- 172.21.15.59:0/1762301323 >> 172.21.15.59:0/1762301323 conn(0x7f5e0c0b2740 msgr2=0x7f5e0c0b2b20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:58:23.407 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:23.399+0000 7f5e13654700 1 -- 172.21.15.59:0/1762301323 shutdown_connections 2023-11-24T07:58:23.407 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:23.399+0000 7f5e13654700 1 -- 172.21.15.59:0/1762301323 wait complete. 2023-11-24T07:58:23.407 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:23.400+0000 7f5e13654700 1 Processor -- start 2023-11-24T07:58:23.407 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:23.400+0000 7f5e13654700 1 -- start start 2023-11-24T07:58:23.407 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:23.400+0000 7f5e13654700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5e0c0720f0 0x7f5e0c13f330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:23.407 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:23.400+0000 7f5e13654700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5e0c143ae0 0x7f5e0c1e46b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:23.407 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:23.400+0000 7f5e13654700 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f5e0c1440f0 0x7f5e0c152870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:23.407 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:23.400+0000 7f5e13654700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e0c1e5cf0 con 0x7f5e0c0720f0 2023-11-24T07:58:23.408 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:23.401+0000 7f5e13654700 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e0c1e50e0 con 0x7f5e0c143ae0 2023-11-24T07:58:23.408 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:23.401+0000 7f5e13654700 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5e0c1e53c0 con 0x7f5e0c1440f0 2023-11-24T07:58:23.408 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:23.401+0000 7f5e113f0700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5e0c0720f0 0x7f5e0c13f330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:23.408 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:23.401+0000 7f5e113f0700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5e0c0720f0 0x7f5e0c13f330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:53308/0 (socket says 172.21.15.59:53308) 2023-11-24T07:58:23.408 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:23.401+0000 7f5e113f0700 1 -- 172.21.15.59:0/3133044815 learned_addr learned my addr 172.21.15.59:0/3133044815 (peer_addr_for_me v2:172.21.15.59:0/0) 2023-11-24T07:58:23.408 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:23.401+0000 7f5e113f0700 1 -- 172.21.15.59:0/3133044815 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f5e0c1440f0 msgr2=0x7f5e0c152870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:23.408 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:23.401+0000 7f5e113f0700 1 --2- 172.21.15.59:0/3133044815 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f5e0c1440f0 0x7f5e0c152870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:23.408 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:23.401+0000 7f5e113f0700 1 -- 172.21.15.59:0/3133044815 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5e0c143ae0 msgr2=0x7f5e0c1e46b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:23.408 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:23.401+0000 7f5e113f0700 1 --2- 172.21.15.59:0/3133044815 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5e0c143ae0 0x7f5e0c1e46b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:23.408 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:23.401+0000 7f5e113f0700 1 -- 172.21.15.59:0/3133044815 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5e080094a0 con 0x7f5e0c0720f0 2023-11-24T07:58:23.409 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:23.401+0000 7f5e113f0700 1 --2- 172.21.15.59:0/3133044815 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5e0c0720f0 0x7f5e0c13f330 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7f5e08009b90 tx=0x7f5e08004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:58:23.409 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:23.401+0000 7f5e01ffb700 1 -- 172.21.15.59:0/3133044815 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e080045f0 con 0x7f5e0c0720f0 2023-11-24T07:58:23.409 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:23.401+0000 7f5e01ffb700 1 -- 172.21.15.59:0/3133044815 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5e0801e070 con 0x7f5e0c0720f0 2023-11-24T07:58:23.409 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:23.401+0000 7f5e01ffb700 1 -- 172.21.15.59:0/3133044815 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e08003aa0 con 0x7f5e0c0720f0 2023-11-24T07:58:23.409 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:23.401+0000 7f5e13654700 1 -- 172.21.15.59:0/3133044815 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5e0c1e56a0 con 0x7f5e0c0720f0 2023-11-24T07:58:23.409 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:23.401+0000 7f5e13654700 1 -- 172.21.15.59:0/3133044815 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5e0c142d50 con 0x7f5e0c0720f0 2023-11-24T07:58:23.409 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:23.402+0000 7f5e13654700 1 -- 172.21.15.59:0/3133044815 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5e0c068670 con 0x7f5e0c0720f0 2023-11-24T07:58:23.409 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:23.403+0000 7f5e01ffb700 1 -- 172.21.15.59:0/3133044815 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 8) v1 ==== 89435+0+0 (secure 0 0 0) 0x7f5e08003c00 con 0x7f5e0c0720f0 2023-11-24T07:58:23.409 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:23.403+0000 7f5e01ffb700 1 --2- 172.21.15.59:0/3133044815 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f5df006b110 0x7f5df006d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:23.409 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:23.403+0000 7f5e01ffb700 1 -- 172.21.15.59:0/3133044815 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 5910+0+0 (secure 0 0 0) 0x7f5e0808abe0 con 0x7f5e0c0720f0 2023-11-24T07:58:23.409 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:23.404+0000 7f5e10bef700 1 --2- 172.21.15.59:0/3133044815 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f5df006b110 0x7f5df006d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:23.409 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:23.404+0000 7f5e10bef700 1 --2- 172.21.15.59:0/3133044815 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f5df006b110 0x7f5df006d5d0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f5e0c0ccd20 tx=0x7f5dfc006c80 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:58:23.410 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:23.408+0000 7f5e01ffb700 1 -- 172.21.15.59:0/3133044815 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f5e080558f0 con 0x7f5e0c0720f0 2023-11-24T07:58:23.610 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:23.608+0000 7f5e13654700 1 -- 172.21.15.59:0/3133044815 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f5e0c154900 con 0x7f5e0c0720f0 2023-11-24T07:58:23.611 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:23.609+0000 7f5e01ffb700 1 -- 172.21.15.59:0/3133044815 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v48) v1 ==== 74+0+16776 (secure 0 0 0) 0x7f5e08015090 con 0x7f5e0c0720f0 2023-11-24T07:58:23.611 INFO:teuthology.orchestra.run.smithi059.stdout: 2023-11-24T07:58:23.611 INFO:teuthology.orchestra.run.smithi059.stdout:{"epoch":48,"fsid":"9f0e8741-b8f9-48d5-a2c3-09f45903303d","created":"2023-11-24T07:56:06.552729+0000","modified":"2023-11-24T07:58:22.384748+0000","last_up_change":"2023-11-24T07:56:17.372352+0000","last_in_change":"2023-11-24T07:56:09.073250+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-24T07:56:18.352823+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_metadata","create_time":"2023-11-24T07:58:00.240130+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"44","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_data","create_time":"2023-11-24T07:58:00.977486+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"48","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":3,"snap_epoch":48,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"6d8f9e9b-1782-4e41-928b-a39f2a8ebc06","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6800","nonce":3373878772},{"type":"v1","addr":"172.21.15.59:6801","nonce":3373878772}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6802","nonce":3373878772},{"type":"v1","addr":"172.21.15.59:6803","nonce":3373878772}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6806","nonce":3373878772},{"type":"v1","addr":"172.21.15.59:6807","nonce":3373878772}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6804","nonce":3373878772},{"type":"v1","addr":"172.21.15.59:6805","nonce":3373878772}]},"public_addr":"172.21.15.59:6801/3373878772","cluster_addr":"172.21.15.59:6803/3373878772","heartbeat_back_addr":"172.21.15.59:6807/3373878772","heartbeat_front_addr":"172.21.15.59:6805/3373878772","state":["exists","up"]},{"osd":1,"uuid":"e8d882aa-52e1-4c04-9e1e-c44ef9204a5f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6824","nonce":648258563},{"type":"v1","addr":"172.21.15.59:6825","nonce":648258563}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6826","nonce":648258563},{"type":"v1","addr":"172.21.15.59:6827","nonce":648258563}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6830","nonce":648258563},{"type":"v1","addr":"172.21.15.59:6831","nonce":648258563}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6828","nonce":648258563},{"type":"v1","addr":"172.21.15.59:6829","nonce":648258563}]},"public_addr":"172.21.15.59:6825/648258563","cluster_addr":"172.21.15.59:6827/648258563","heartbeat_back_addr":"172.21.15.59:6831/648258563","heartbeat_front_addr":"172.21.15.59:6829/648258563","state":["exists","up"]},{"osd":2,"uuid":"69eebfa0-695b-4b21-90b9-8a3537168a01","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6808","nonce":1840430654},{"type":"v1","addr":"172.21.15.59:6809","nonce":1840430654}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6810","nonce":1840430654},{"type":"v1","addr":"172.21.15.59:6811","nonce":1840430654}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6814","nonce":1840430654},{"type":"v1","addr":"172.21.15.59:6815","nonce":1840430654}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6812","nonce":1840430654},{"type":"v1","addr":"172.21.15.59:6813","nonce":1840430654}]},"public_addr":"172.21.15.59:6809/1840430654","cluster_addr":"172.21.15.59:6811/1840430654","heartbeat_back_addr":"172.21.15.59:6815/1840430654","heartbeat_front_addr":"172.21.15.59:6813/1840430654","state":["exists","up"]},{"osd":3,"uuid":"96fd77fc-f911-46ff-9445-6f0be2daa8f4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6816","nonce":2504025850},{"type":"v1","addr":"172.21.15.59:6817","nonce":2504025850}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6818","nonce":2504025850},{"type":"v1","addr":"172.21.15.59:6819","nonce":2504025850}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6822","nonce":2504025850},{"type":"v1","addr":"172.21.15.59:6823","nonce":2504025850}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6820","nonce":2504025850},{"type":"v1","addr":"172.21.15.59:6821","nonce":2504025850}]},"public_addr":"172.21.15.59:6817/2504025850","cluster_addr":"172.21.15.59:6819/2504025850","heartbeat_back_addr":"172.21.15.59:6823/2504025850","heartbeat_front_addr":"172.21.15.59:6821/2504025850","state":["exists","up"]},{"osd":4,"uuid":"7ac29fca-b5b2-4c95-9d69-2f164cbbc708","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6816","nonce":2382463431},{"type":"v1","addr":"172.21.15.76:6818","nonce":2382463431}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6820","nonce":2382463431},{"type":"v1","addr":"172.21.15.76:6822","nonce":2382463431}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6828","nonce":2382463431},{"type":"v1","addr":"172.21.15.76:6830","nonce":2382463431}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6824","nonce":2382463431},{"type":"v1","addr":"172.21.15.76:6826","nonce":2382463431}]},"public_addr":"172.21.15.76:6818/2382463431","cluster_addr":"172.21.15.76:6822/2382463431","heartbeat_back_addr":"172.21.15.76:6830/2382463431","heartbeat_front_addr":"172.21.15.76:6826/2382463431","state":["exists","up"]},{"osd":5,"uuid":"485bf711-3851-4238-a4f4-51d52e0a9c58","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6800","nonce":329334308},{"type":"v1","addr":"172.21.15.76:6802","nonce":329334308}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6805","nonce":329334308},{"type":"v1","addr":"172.21.15.76:6807","nonce":329334308}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6814","nonce":329334308},{"type":"v1","addr":"172.21.15.76:6815","nonce":329334308}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6810","nonce":329334308},{"type":"v1","addr":"172.21.15.76:6812","nonce":329334308}]},"public_addr":"172.21.15.76:6802/329334308","cluster_addr":"172.21.15.76:6807/329334308","heartbeat_back_addr":"172.21.15.76:6815/329334308","heartbeat_front_addr":"172.21.15.76:6812/329334308","state":["exists","up"]},{"osd":6,"uuid":"72b3f2fc-1e4a-4e32-968d-003647a0286f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6801","nonce":3946579410},{"type":"v1","addr":"172.21.15.76:6803","nonce":3946579410}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6804","nonce":3946579410},{"type":"v1","addr":"172.21.15.76:6806","nonce":3946579410}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6811","nonce":3946579410},{"type":"v1","addr":"172.21.15.76:6813","nonce":3946579410}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6808","nonce":3946579410},{"type":"v1","addr":"172.21.15.76:6809","nonce":3946579410}]},"public_addr":"172.21.15.76:6803/3946579410","cluster_addr":"172.21.15.76:6806/3946579410","heartbeat_back_addr":"172.21.15.76:6813/3946579410","heartbeat_front_addr":"172.21.15.76:6809/3946579410","state":["exists","up"]},{"osd":7,"uuid":"1ddceb90-6c5d-4d1a-ad56-99cb877f8c70","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6817","nonce":841761538},{"type":"v1","addr":"172.21.15.76:6819","nonce":841761538}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6821","nonce":841761538},{"type":"v1","addr":"172.21.15.76:6823","nonce":841761538}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6829","nonce":841761538},{"type":"v1","addr":"172.21.15.76:6831","nonce":841761538}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6825","nonce":841761538},{"type":"v1","addr":"172.21.15.76:6827","nonce":841761538}]},"public_addr":"172.21.15.76:6819/841761538","cluster_addr":"172.21.15.76:6823/841761538","heartbeat_back_addr":"172.21.15.76:6831/841761538","heartbeat_front_addr":"172.21.15.76:6827/841761538","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T07:56:14.294838+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T07:56:14.270779+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T07:56:14.231809+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T07:56:14.260983+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T07:56:14.302997+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T07:56:14.258600+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T07:56:14.251413+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T07:56:14.304727+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"7.0","mappings":[{"from":5,"to":7}]},{"pgid":"7.c","mappings":[{"from":5,"to":7}]},{"pgid":"7.12","mappings":[{"from":5,"to":7}]},{"pgid":"7.14","mappings":[{"from":5,"to":7}]},{"pgid":"7.38","mappings":[{"from":5,"to":1}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":7,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[{"pool":7,"snaps":[{"begin":2,"length":2}]}],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-24T07:58:23.614 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:23.612+0000 7f5e13654700 1 -- 172.21.15.59:0/3133044815 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f5df006b110 msgr2=0x7f5df006d5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:23.614 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:23.612+0000 7f5e13654700 1 --2- 172.21.15.59:0/3133044815 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f5df006b110 0x7f5df006d5d0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f5e0c0ccd20 tx=0x7f5dfc006c80 comp rx=0 tx=0).stop 2023-11-24T07:58:23.614 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:23.612+0000 7f5e13654700 1 -- 172.21.15.59:0/3133044815 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5e0c0720f0 msgr2=0x7f5e0c13f330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:23.614 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:23.612+0000 7f5e13654700 1 --2- 172.21.15.59:0/3133044815 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5e0c0720f0 0x7f5e0c13f330 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7f5e08009b90 tx=0x7f5e08004750 comp rx=0 tx=0).stop 2023-11-24T07:58:23.614 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:23.612+0000 7f5e13654700 1 -- 172.21.15.59:0/3133044815 shutdown_connections 2023-11-24T07:58:23.614 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:23.612+0000 7f5e13654700 1 --2- 172.21.15.59:0/3133044815 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f5df006b110 0x7f5df006d5d0 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:23.614 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:23.612+0000 7f5e13654700 1 --2- 172.21.15.59:0/3133044815 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f5e0c1440f0 0x7f5e0c152870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:23.614 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:23.612+0000 7f5e13654700 1 --2- 172.21.15.59:0/3133044815 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5e0c0720f0 0x7f5e0c13f330 unknown :-1 s=CLOSED pgs=622 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:23.614 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:23.612+0000 7f5e13654700 1 --2- 172.21.15.59:0/3133044815 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5e0c143ae0 0x7f5e0c1e46b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:23.614 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:23.612+0000 7f5e13654700 1 -- 172.21.15.59:0/3133044815 >> 172.21.15.59:0/3133044815 conn(0x7f5e0c0b2740 msgr2=0x7f5e0c13ff30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:58:23.614 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:23.612+0000 7f5e13654700 1 -- 172.21.15.59:0/3133044815 shutdown_connections 2023-11-24T07:58:23.615 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:23.612+0000 7f5e13654700 1 -- 172.21.15.59:0/3133044815 wait complete. 2023-11-24T07:58:23.627 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-24T07:56:18.352823+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 6, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-24T07:58:00.240130+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '44', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 7, 'pool_name': 'cephfs_data', 'create_time': '2023-11-24T07:58:00.977486+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '48', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 3, 'snap_epoch': 48, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-24T07:58:23.627 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-24T07:58:23.757 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:23.753+0000 7f61d25c2700 1 Processor -- start 2023-11-24T07:58:23.757 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:23.753+0000 7f61d25c2700 1 -- start start 2023-11-24T07:58:23.757 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:23.754+0000 7f61d25c2700 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f61cc0722c0 0x7f61cc0b51b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:23.757 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:23.754+0000 7f61d25c2700 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f61cc143ec0 con 0x7f61cc1440f0 2023-11-24T07:58:23.757 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:23.754+0000 7f61d25c2700 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f61cc0b3ae0 con 0x7f61cc143ae0 2023-11-24T07:58:23.757 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:23.754+0000 7f61d25c2700 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f61cc0b40a0 con 0x7f61cc0722c0 2023-11-24T07:58:23.757 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:23.754+0000 7f61d0b5f700 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7f61cc1440f0 0x7f61cc06e510 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:56794/0 (socket says 172.21.15.59:56794) 2023-11-24T07:58:23.757 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:23.754+0000 7f61d0b5f700 1 -- 172.21.15.59:0/3448041474 learned_addr learned my addr 172.21.15.59:0/3448041474 (peer_addr_for_me v1:172.21.15.59:0/0) 2023-11-24T07:58:23.758 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:23.754+0000 7f61cb7fe700 1 --2- 172.21.15.59:0/3448041474 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f61cc0722c0 0x7f61cc0b51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:23.758 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:23.754+0000 7f61caffd700 1 -- 172.21.15.59:0/3448041474 <== mon.0 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2024466975 0 0) 0x7f61cc143ec0 con 0x7f61cc1440f0 2023-11-24T07:58:23.758 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:23.754+0000 7f61caffd700 1 -- 172.21.15.59:0/3448041474 --> v1:172.21.15.59:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f61b80035b0 con 0x7f61cc1440f0 2023-11-24T07:58:23.758 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:23.755+0000 7f61cb7fe700 1 -- 172.21.15.59:0/3448041474 >> v1:172.21.15.76:6789/0 conn(0x7f61cc143ae0 legacy=0x7f61cc071bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:23.758 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:23.755+0000 7f61cb7fe700 1 -- 172.21.15.59:0/3448041474 >> v1:172.21.15.59:6789/0 conn(0x7f61cc1440f0 legacy=0x7f61cc06e510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:23.758 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:23.755+0000 7f61cb7fe700 1 -- 172.21.15.59:0/3448041474 --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f61cc0b3dc0 con 0x7f61cc0722c0 2023-11-24T07:58:23.758 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:23.755+0000 7f61cb7fe700 1 --2- 172.21.15.59:0/3448041474 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f61cc0722c0 0x7f61cc0b51b0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f61b4009870 tx=0x7f61b4009b80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=812bdd46c8eb59eb server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:58:23.758 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:23.755+0000 7f61caffd700 1 -- 172.21.15.59:0/3448041474 <== mon.2 v2:172.21.15.76:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61b4011830 con 0x7f61cc0722c0 2023-11-24T07:58:23.758 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:23.755+0000 7f61caffd700 1 -- 172.21.15.59:0/3448041474 <== mon.2 v2:172.21.15.76:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f61b4011990 con 0x7f61cc0722c0 2023-11-24T07:58:23.758 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:23.755+0000 7f61caffd700 1 -- 172.21.15.59:0/3448041474 <== mon.2 v2:172.21.15.76:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61b4011d20 con 0x7f61cc0722c0 2023-11-24T07:58:23.762 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:23.756+0000 7f61d25c2700 1 -- 172.21.15.59:0/3448041474 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f61cc0722c0 msgr2=0x7f61cc0b51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:23.762 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:23.756+0000 7f61d25c2700 1 --2- 172.21.15.59:0/3448041474 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f61cc0722c0 0x7f61cc0b51b0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f61b4009870 tx=0x7f61b4009b80 comp rx=0 tx=0).stop 2023-11-24T07:58:23.762 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:23.756+0000 7f61d25c2700 1 -- 172.21.15.59:0/3448041474 shutdown_connections 2023-11-24T07:58:23.762 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:23.756+0000 7f61d25c2700 1 --2- 172.21.15.59:0/3448041474 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f61cc0722c0 0x7f61cc0b51b0 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:23.762 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:23.756+0000 7f61d25c2700 1 -- 172.21.15.59:0/3448041474 >> 172.21.15.59:0/3448041474 conn(0x7f61cc0b2740 msgr2=0x7f61cc0b2b20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:58:23.763 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:23.756+0000 7f61d25c2700 1 -- 172.21.15.59:0/3448041474 shutdown_connections 2023-11-24T07:58:23.763 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:23.756+0000 7f61d25c2700 1 -- 172.21.15.59:0/3448041474 wait complete. 2023-11-24T07:58:23.763 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:23.757+0000 7f61d25c2700 1 Processor -- start 2023-11-24T07:58:23.763 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:23.757+0000 7f61d25c2700 1 -- start start 2023-11-24T07:58:23.763 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:23.758+0000 7f61d25c2700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f61cc0722c0 0x7f61cc1587b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:23.763 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:23.758+0000 7f61d25c2700 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f61cc143ae0 0x7f61cc142140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:23.763 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:23.758+0000 7f61d25c2700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f61cc1440f0 0x7f61cc142680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:23.763 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:23.758+0000 7f61d25c2700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61cc0cb250 con 0x7f61cc0722c0 2023-11-24T07:58:23.763 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:23.758+0000 7f61d25c2700 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61cc135b10 con 0x7f61cc1440f0 2023-11-24T07:58:23.763 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:23.758+0000 7f61d25c2700 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f61cc1f5770 con 0x7f61cc143ae0 2023-11-24T07:58:23.763 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:23.758+0000 7f61cbfff700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f61cc0722c0 0x7f61cc1587b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:23.763 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:23.758+0000 7f61cbfff700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f61cc0722c0 0x7f61cc1587b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:53322/0 (socket says 172.21.15.59:53322) 2023-11-24T07:58:23.764 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:23.758+0000 7f61cbfff700 1 -- 172.21.15.59:0/1319349601 learned_addr learned my addr 172.21.15.59:0/1319349601 (peer_addr_for_me v2:172.21.15.59:0/0) 2023-11-24T07:58:23.764 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:23.758+0000 7f61cb7fe700 1 --2- 172.21.15.59:0/1319349601 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f61cc143ae0 0x7f61cc142140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:23.764 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:23.758+0000 7f61d0b5f700 1 --2- 172.21.15.59:0/1319349601 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f61cc1440f0 0x7f61cc142680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:23.764 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:23.758+0000 7f61cbfff700 1 -- 172.21.15.59:0/1319349601 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f61cc143ae0 msgr2=0x7f61cc142140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:23.764 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:23.758+0000 7f61cbfff700 1 --2- 172.21.15.59:0/1319349601 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f61cc143ae0 0x7f61cc142140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:23.764 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:23.758+0000 7f61cbfff700 1 -- 172.21.15.59:0/1319349601 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f61cc1440f0 msgr2=0x7f61cc142680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:23.764 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:23.758+0000 7f61cbfff700 1 --2- 172.21.15.59:0/1319349601 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f61cc1440f0 0x7f61cc142680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:23.764 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:23.758+0000 7f61cbfff700 1 -- 172.21.15.59:0/1319349601 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f61b40094a0 con 0x7f61cc0722c0 2023-11-24T07:58:23.764 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:23.758+0000 7f61cbfff700 1 --2- 172.21.15.59:0/1319349601 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f61cc0722c0 0x7f61cc1587b0 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7f61bc00c4e0 tx=0x7f61bc00c7f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:58:23.765 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:23.759+0000 7f61c8ff9700 1 -- 172.21.15.59:0/1319349601 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61bc0052d0 con 0x7f61cc0722c0 2023-11-24T07:58:23.765 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:23.759+0000 7f61c8ff9700 1 -- 172.21.15.59:0/1319349601 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f61bc005430 con 0x7f61cc0722c0 2023-11-24T07:58:23.765 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:23.759+0000 7f61d25c2700 1 -- 172.21.15.59:0/1319349601 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f61cc1f5a50 con 0x7f61cc0722c0 2023-11-24T07:58:23.765 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:23.759+0000 7f61c8ff9700 1 -- 172.21.15.59:0/1319349601 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61bc0038c0 con 0x7f61cc0722c0 2023-11-24T07:58:23.765 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:23.759+0000 7f61d25c2700 1 -- 172.21.15.59:0/1319349601 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f61cc0cae70 con 0x7f61cc0722c0 2023-11-24T07:58:23.765 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:23.759+0000 7f61d25c2700 1 -- 172.21.15.59:0/1319349601 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f61cc067a00 con 0x7f61cc0722c0 2023-11-24T07:58:23.765 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:23.760+0000 7f61c8ff9700 1 -- 172.21.15.59:0/1319349601 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 8) v1 ==== 89435+0+0 (secure 0 0 0) 0x7f61bc003a20 con 0x7f61cc0722c0 2023-11-24T07:58:23.765 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:23.760+0000 7f61c8ff9700 1 --2- 172.21.15.59:0/1319349601 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f61ac06b110 0x7f61ac06d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:23.765 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:23.760+0000 7f61c8ff9700 1 -- 172.21.15.59:0/1319349601 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 5910+0+0 (secure 0 0 0) 0x7f61bc088d00 con 0x7f61cc0722c0 2023-11-24T07:58:23.765 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:23.761+0000 7f61cb7fe700 1 --2- 172.21.15.59:0/1319349601 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f61ac06b110 0x7f61ac06d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:23.766 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:23.761+0000 7f61cb7fe700 1 --2- 172.21.15.59:0/1319349601 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f61ac06b110 0x7f61ac06d5d0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f61b40090b0 tx=0x7f61b4008030 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:58:23.766 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:23.765+0000 7f61c8ff9700 1 -- 172.21.15.59:0/1319349601 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f61bc053ae0 con 0x7f61cc0722c0 2023-11-24T07:58:23.963 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:23.961+0000 7f61d25c2700 1 -- 172.21.15.59:0/1319349601 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f61cc0ca9f0 con 0x7f61cc0722c0 2023-11-24T07:58:23.964 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:23.962+0000 7f61c8ff9700 1 -- 172.21.15.59:0/1319349601 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v48) v1 ==== 93+0+10 (secure 0 0 0) 0x7f61cc0ca9f0 con 0x7f61cc0722c0 2023-11-24T07:58:23.964 INFO:teuthology.orchestra.run.smithi059.stdout:pg_num: 1 2023-11-24T07:58:23.966 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:23.964+0000 7f61d25c2700 1 -- 172.21.15.59:0/1319349601 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f61ac06b110 msgr2=0x7f61ac06d5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:23.966 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:23.964+0000 7f61d25c2700 1 --2- 172.21.15.59:0/1319349601 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f61ac06b110 0x7f61ac06d5d0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f61b40090b0 tx=0x7f61b4008030 comp rx=0 tx=0).stop 2023-11-24T07:58:23.966 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:23.964+0000 7f61d25c2700 1 -- 172.21.15.59:0/1319349601 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f61cc0722c0 msgr2=0x7f61cc1587b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:23.967 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:23.964+0000 7f61d25c2700 1 --2- 172.21.15.59:0/1319349601 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f61cc0722c0 0x7f61cc1587b0 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7f61bc00c4e0 tx=0x7f61bc00c7f0 comp rx=0 tx=0).stop 2023-11-24T07:58:23.967 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:23.964+0000 7f61d25c2700 1 -- 172.21.15.59:0/1319349601 shutdown_connections 2023-11-24T07:58:23.967 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:23.964+0000 7f61d25c2700 1 --2- 172.21.15.59:0/1319349601 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f61ac06b110 0x7f61ac06d5d0 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:23.967 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:23.964+0000 7f61d25c2700 1 --2- 172.21.15.59:0/1319349601 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f61cc1440f0 0x7f61cc142680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:23.967 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:23.964+0000 7f61d25c2700 1 --2- 172.21.15.59:0/1319349601 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f61cc0722c0 0x7f61cc1587b0 unknown :-1 s=CLOSED pgs=624 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:23.967 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:23.964+0000 7f61d25c2700 1 --2- 172.21.15.59:0/1319349601 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f61cc143ae0 0x7f61cc142140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:23.967 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:23.964+0000 7f61d25c2700 1 -- 172.21.15.59:0/1319349601 >> 172.21.15.59:0/1319349601 conn(0x7f61cc0b2740 msgr2=0x7f61cc0cdf90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:58:23.967 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:23.964+0000 7f61d25c2700 1 -- 172.21.15.59:0/1319349601 shutdown_connections 2023-11-24T07:58:23.967 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:23.964+0000 7f61d25c2700 1 -- 172.21.15.59:0/1319349601 wait complete. 2023-11-24T07:58:23.978 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-24T07:58:24.106 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:24.104+0000 7fcae6d3f700 1 Processor -- start 2023-11-24T07:58:24.106 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:24.104+0000 7fcae6d3f700 1 -- start start 2023-11-24T07:58:24.109 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:24.105+0000 7fcae6d3f700 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fcae01440f0 0x7fcae00b51b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:24.109 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:24.105+0000 7fcae6d3f700 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcae0143ec0 con 0x7fcae00722c0 2023-11-24T07:58:24.109 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:24.105+0000 7fcae6d3f700 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcae00b3ae0 con 0x7fcae0143ae0 2023-11-24T07:58:24.109 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:24.105+0000 7fcae6d3f700 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcae00b40a0 con 0x7fcae01440f0 2023-11-24T07:58:24.109 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:24.105+0000 7fcadffff700 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7fcae00722c0 0x7fcae006e510 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:56806/0 (socket says 172.21.15.59:56806) 2023-11-24T07:58:24.109 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:24.105+0000 7fcadffff700 1 -- 172.21.15.59:0/3304359727 learned_addr learned my addr 172.21.15.59:0/3304359727 (peer_addr_for_me v1:172.21.15.59:0/0) 2023-11-24T07:58:24.109 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:24.105+0000 7fcae52dc700 1 --2- 172.21.15.59:0/3304359727 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fcae01440f0 0x7fcae00b51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:24.109 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:24.106+0000 7fcadf7fe700 1 -- 172.21.15.59:0/3304359727 <== mon.0 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 917668964 0 0) 0x7fcae0143ec0 con 0x7fcae00722c0 2023-11-24T07:58:24.110 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:24.106+0000 7fcadf7fe700 1 -- 172.21.15.59:0/3304359727 --> v1:172.21.15.59:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcad00035b0 con 0x7fcae00722c0 2023-11-24T07:58:24.110 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:24.106+0000 7fcadf7fe700 1 -- 172.21.15.59:0/3304359727 <== mon.1 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2024253127 0 0) 0x7fcae00b3ae0 con 0x7fcae0143ae0 2023-11-24T07:58:24.110 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:24.106+0000 7fcadf7fe700 1 -- 172.21.15.59:0/3304359727 --> v1:172.21.15.76:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcae0143ec0 con 0x7fcae0143ae0 2023-11-24T07:58:24.110 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:24.106+0000 7fcae52dc700 1 -- 172.21.15.59:0/3304359727 >> v1:172.21.15.76:6789/0 conn(0x7fcae0143ae0 legacy=0x7fcae0071bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:24.110 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:24.106+0000 7fcae52dc700 1 -- 172.21.15.59:0/3304359727 >> v1:172.21.15.59:6789/0 conn(0x7fcae00722c0 legacy=0x7fcae006e510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:24.110 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:24.106+0000 7fcae52dc700 1 -- 172.21.15.59:0/3304359727 --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcae00b3dc0 con 0x7fcae01440f0 2023-11-24T07:58:24.110 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:24.106+0000 7fcae52dc700 1 --2- 172.21.15.59:0/3304359727 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fcae01440f0 0x7fcae00b51b0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fcad8009dc0 tx=0x7fcad800b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=451bbe572c6cd7a2 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:58:24.110 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:24.106+0000 7fcadf7fe700 1 -- 172.21.15.59:0/3304359727 <== mon.2 v2:172.21.15.76:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcad800e070 con 0x7fcae01440f0 2023-11-24T07:58:24.110 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:24.106+0000 7fcadf7fe700 1 -- 172.21.15.59:0/3304359727 <== mon.2 v2:172.21.15.76:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcad800bb20 con 0x7fcae01440f0 2023-11-24T07:58:24.110 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:24.106+0000 7fcadf7fe700 1 -- 172.21.15.59:0/3304359727 <== mon.2 v2:172.21.15.76:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcad8012430 con 0x7fcae01440f0 2023-11-24T07:58:24.111 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:24.107+0000 7fcae6d3f700 1 -- 172.21.15.59:0/3304359727 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fcae01440f0 msgr2=0x7fcae00b51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:24.111 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:24.107+0000 7fcae6d3f700 1 --2- 172.21.15.59:0/3304359727 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fcae01440f0 0x7fcae00b51b0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fcad8009dc0 tx=0x7fcad800b330 comp rx=0 tx=0).stop 2023-11-24T07:58:24.111 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:24.107+0000 7fcae6d3f700 1 -- 172.21.15.59:0/3304359727 shutdown_connections 2023-11-24T07:58:24.111 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:24.107+0000 7fcae6d3f700 1 --2- 172.21.15.59:0/3304359727 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fcae01440f0 0x7fcae00b51b0 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:24.111 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:24.107+0000 7fcae6d3f700 1 -- 172.21.15.59:0/3304359727 >> 172.21.15.59:0/3304359727 conn(0x7fcae00b2740 msgr2=0x7fcae00b2b20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:58:24.111 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:24.107+0000 7fcae6d3f700 1 -- 172.21.15.59:0/3304359727 shutdown_connections 2023-11-24T07:58:24.111 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:24.107+0000 7fcae6d3f700 1 -- 172.21.15.59:0/3304359727 wait complete. 2023-11-24T07:58:24.114 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:24.108+0000 7fcae6d3f700 1 Processor -- start 2023-11-24T07:58:24.114 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:24.108+0000 7fcae6d3f700 1 -- start start 2023-11-24T07:58:24.114 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:24.109+0000 7fcae6d3f700 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fcae00722c0 0x7fcae01eee10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:24.114 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:24.109+0000 7fcae6d3f700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fcae0143ae0 0x7fcae00c63c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:24.114 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:24.109+0000 7fcae6d3f700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fcae01440f0 0x7fcae006afd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:24.115 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:24.109+0000 7fcae6d3f700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcae0130ed0 con 0x7fcae0143ae0 2023-11-24T07:58:24.115 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:24.109+0000 7fcae6d3f700 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcae00c7740 con 0x7fcae01440f0 2023-11-24T07:58:24.115 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:24.109+0000 7fcae6d3f700 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcae01ef350 con 0x7fcae00722c0 2023-11-24T07:58:24.115 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:24.109+0000 7fcadffff700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fcae0143ae0 0x7fcae00c63c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:24.115 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:24.109+0000 7fcadffff700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fcae0143ae0 0x7fcae00c63c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:53338/0 (socket says 172.21.15.59:53338) 2023-11-24T07:58:24.115 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:24.109+0000 7fcadffff700 1 -- 172.21.15.59:0/1791102738 learned_addr learned my addr 172.21.15.59:0/1791102738 (peer_addr_for_me v2:172.21.15.59:0/0) 2023-11-24T07:58:24.115 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:24.109+0000 7fcae4adb700 1 --2- 172.21.15.59:0/1791102738 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fcae00722c0 0x7fcae01eee10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:24.115 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:24.109+0000 7fcae52dc700 1 --2- 172.21.15.59:0/1791102738 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fcae01440f0 0x7fcae006afd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:24.115 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:24.109+0000 7fcadffff700 1 -- 172.21.15.59:0/1791102738 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fcae00722c0 msgr2=0x7fcae01eee10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:24.115 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:24.109+0000 7fcadffff700 1 --2- 172.21.15.59:0/1791102738 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fcae00722c0 0x7fcae01eee10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:24.116 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:24.109+0000 7fcadffff700 1 -- 172.21.15.59:0/1791102738 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fcae01440f0 msgr2=0x7fcae006afd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:24.116 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:24.109+0000 7fcadffff700 1 --2- 172.21.15.59:0/1791102738 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fcae01440f0 0x7fcae006afd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:24.116 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:24.109+0000 7fcadffff700 1 -- 172.21.15.59:0/1791102738 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcad8009a70 con 0x7fcae0143ae0 2023-11-24T07:58:24.116 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:24.109+0000 7fcae4adb700 1 --2- 172.21.15.59:0/1791102738 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fcae00722c0 0x7fcae01eee10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T07:58:24.116 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:24.109+0000 7fcae52dc700 1 --2- 172.21.15.59:0/1791102738 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fcae01440f0 0x7fcae006afd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T07:58:24.116 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:24.110+0000 7fcadffff700 1 --2- 172.21.15.59:0/1791102738 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fcae0143ae0 0x7fcae00c63c0 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7fcacc00c700 tx=0x7fcacc00cac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:58:24.116 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:24.110+0000 7fcadd7fa700 1 -- 172.21.15.59:0/1791102738 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcacc005d70 con 0x7fcae0143ae0 2023-11-24T07:58:24.116 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:24.110+0000 7fcadd7fa700 1 -- 172.21.15.59:0/1791102738 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcacc0070d0 con 0x7fcae0143ae0 2023-11-24T07:58:24.116 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:24.110+0000 7fcadd7fa700 1 -- 172.21.15.59:0/1791102738 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcacc007480 con 0x7fcae0143ae0 2023-11-24T07:58:24.116 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:24.110+0000 7fcae6d3f700 1 -- 172.21.15.59:0/1791102738 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcae00c6ea0 con 0x7fcae0143ae0 2023-11-24T07:58:24.116 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:24.110+0000 7fcae6d3f700 1 -- 172.21.15.59:0/1791102738 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcae00c7460 con 0x7fcae0143ae0 2023-11-24T07:58:24.117 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:24.111+0000 7fcae6d3f700 1 -- 172.21.15.59:0/1791102738 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcae0075480 con 0x7fcae0143ae0 2023-11-24T07:58:24.117 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:24.111+0000 7fcadd7fa700 1 -- 172.21.15.59:0/1791102738 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 8) v1 ==== 89435+0+0 (secure 0 0 0) 0x7fcacc0075e0 con 0x7fcae0143ae0 2023-11-24T07:58:24.117 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:24.111+0000 7fcadd7fa700 1 --2- 172.21.15.59:0/1791102738 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7fcac406b110 0x7fcac406d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:24.117 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:24.112+0000 7fcadd7fa700 1 -- 172.21.15.59:0/1791102738 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 5910+0+0 (secure 0 0 0) 0x7fcacc089120 con 0x7fcae0143ae0 2023-11-24T07:58:24.117 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:24.112+0000 7fcae4adb700 1 --2- 172.21.15.59:0/1791102738 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7fcac406b110 0x7fcac406d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:24.117 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:24.112+0000 7fcae4adb700 1 --2- 172.21.15.59:0/1791102738 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7fcac406b110 0x7fcac406d5d0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7fcad4007fd0 tx=0x7fcad4009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:58:24.118 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:24.116+0000 7fcadd7fa700 1 -- 172.21.15.59:0/1791102738 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fcacc053db0 con 0x7fcae0143ae0 2023-11-24T07:58:24.319 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:24.317+0000 7fcae6d3f700 1 -- 172.21.15.59:0/1791102738 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fcae00b5d20 con 0x7fcae0143ae0 2023-11-24T07:58:24.320 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:24.318+0000 7fcadd7fa700 1 -- 172.21.15.59:0/1791102738 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v48) v1 ==== 104+0+11 (secure 0 0 0) 0x7fcacc0573d0 con 0x7fcae0143ae0 2023-11-24T07:58:24.320 INFO:teuthology.orchestra.run.smithi059.stdout:pg_num: 32 2023-11-24T07:58:24.322 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:24.320+0000 7fcae6d3f700 1 -- 172.21.15.59:0/1791102738 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7fcac406b110 msgr2=0x7fcac406d5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:24.322 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:24.320+0000 7fcae6d3f700 1 --2- 172.21.15.59:0/1791102738 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7fcac406b110 0x7fcac406d5d0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7fcad4007fd0 tx=0x7fcad4009040 comp rx=0 tx=0).stop 2023-11-24T07:58:24.322 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:24.320+0000 7fcae6d3f700 1 -- 172.21.15.59:0/1791102738 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fcae0143ae0 msgr2=0x7fcae00c63c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:24.322 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:24.320+0000 7fcae6d3f700 1 --2- 172.21.15.59:0/1791102738 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fcae0143ae0 0x7fcae00c63c0 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7fcacc00c700 tx=0x7fcacc00cac0 comp rx=0 tx=0).stop 2023-11-24T07:58:24.322 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:24.320+0000 7fcae6d3f700 1 -- 172.21.15.59:0/1791102738 shutdown_connections 2023-11-24T07:58:24.323 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:24.320+0000 7fcae6d3f700 1 --2- 172.21.15.59:0/1791102738 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7fcac406b110 0x7fcac406d5d0 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:24.323 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:24.320+0000 7fcae6d3f700 1 --2- 172.21.15.59:0/1791102738 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fcae01440f0 0x7fcae006afd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:24.323 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:24.320+0000 7fcae6d3f700 1 --2- 172.21.15.59:0/1791102738 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fcae00722c0 0x7fcae01eee10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:24.323 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:24.320+0000 7fcae6d3f700 1 --2- 172.21.15.59:0/1791102738 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fcae0143ae0 0x7fcae00c63c0 unknown :-1 s=CLOSED pgs=626 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:24.323 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:24.320+0000 7fcae6d3f700 1 -- 172.21.15.59:0/1791102738 >> 172.21.15.59:0/1791102738 conn(0x7fcae00b2740 msgr2=0x7fcae006e510 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:58:24.323 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:24.320+0000 7fcae6d3f700 1 -- 172.21.15.59:0/1791102738 shutdown_connections 2023-11-24T07:58:24.323 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:24.320+0000 7fcae6d3f700 1 -- 172.21.15.59:0/1791102738 wait complete. 2023-11-24T07:58:24.334 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-24T07:58:24.463 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:24.461+0000 7f71da97e700 1 Processor -- start 2023-11-24T07:58:24.463 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:24.461+0000 7f71da97e700 1 -- start start 2023-11-24T07:58:24.466 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:24.462+0000 7f71da97e700 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f71d40722c0 0x7f71d40b51b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:24.466 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:24.462+0000 7f71da97e700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f71d41440f0 0x7f71d406e510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:24.466 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:24.462+0000 7f71da97e700 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f71d4143ec0 con 0x7f71d4143ae0 2023-11-24T07:58:24.466 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:24.462+0000 7f71da97e700 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f71d40b40a0 con 0x7f71d41440f0 2023-11-24T07:58:24.466 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:24.462+0000 7f71da97e700 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f71d40b3ae0 con 0x7f71d40722c0 2023-11-24T07:58:24.466 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:24.462+0000 7f71d8f1b700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f71d41440f0 0x7f71d406e510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:24.466 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:24.462+0000 7f71d8f1b700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f71d41440f0 0x7f71d406e510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:53348/0 (socket says 172.21.15.59:53348) 2023-11-24T07:58:24.466 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:24.462+0000 7f71d8f1b700 1 -- 172.21.15.59:0/988930715 learned_addr learned my addr 172.21.15.59:0/988930715 (peer_addr_for_me v2:172.21.15.59:0/0) 2023-11-24T07:58:24.466 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:24.462+0000 7f71d37fe700 1 --2- 172.21.15.59:0/988930715 >> v2:172.21.15.76:3300/0 conn(0x7f71d40722c0 0x7f71d40b51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:24.467 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:24.463+0000 7f71d8f1b700 1 -- 172.21.15.59:0/988930715 >> v2:172.21.15.76:3300/0 conn(0x7f71d40722c0 msgr2=0x7f71d40b51b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T07:58:24.467 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:24.463+0000 7f71d8f1b700 1 --2- 172.21.15.59:0/988930715 >> v2:172.21.15.76:3300/0 conn(0x7f71d40722c0 0x7f71d40b51b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:24.467 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:24.463+0000 7f71d8f1b700 1 -- 172.21.15.59:0/988930715 >> v1:172.21.15.76:6789/0 conn(0x7f71d4143ae0 legacy=0x7f71d4071bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T07:58:24.467 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:24.463+0000 7f71d8f1b700 1 -- 172.21.15.59:0/988930715 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f71d40b3dc0 con 0x7f71d41440f0 2023-11-24T07:58:24.467 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:24.463+0000 7f71d37fe700 1 --2- 172.21.15.59:0/988930715 >> v2:172.21.15.76:3300/0 conn(0x7f71d40722c0 0x7f71d40b51b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T07:58:24.467 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:24.463+0000 7f71d8f1b700 1 --2- 172.21.15.59:0/988930715 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f71d41440f0 0x7f71d406e510 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7f71d40d1b00 tx=0x7f71cc00d430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6ffa81db0ede5569 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:58:24.467 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:24.463+0000 7f71d2ffd700 1 -- 172.21.15.59:0/988930715 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71cc010070 con 0x7f71d41440f0 2023-11-24T07:58:24.467 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:24.463+0000 7f71d2ffd700 1 -- 172.21.15.59:0/988930715 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f71cc00dd50 con 0x7f71d41440f0 2023-11-24T07:58:24.467 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:24.463+0000 7f71d2ffd700 1 -- 172.21.15.59:0/988930715 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71cc00b310 con 0x7f71d41440f0 2023-11-24T07:58:24.467 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:24.463+0000 7f71da97e700 1 -- 172.21.15.59:0/988930715 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f71d41440f0 msgr2=0x7f71d406e510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:24.467 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:24.463+0000 7f71da97e700 1 --2- 172.21.15.59:0/988930715 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f71d41440f0 0x7f71d406e510 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7f71d40d1b00 tx=0x7f71cc00d430 comp rx=0 tx=0).stop 2023-11-24T07:58:24.467 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:24.463+0000 7f71da97e700 1 -- 172.21.15.59:0/988930715 shutdown_connections 2023-11-24T07:58:24.468 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:24.463+0000 7f71da97e700 1 --2- 172.21.15.59:0/988930715 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f71d41440f0 0x7f71d406e510 unknown :-1 s=CLOSED pgs=627 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:24.468 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:24.463+0000 7f71da97e700 1 --2- 172.21.15.59:0/988930715 >> v2:172.21.15.76:3300/0 conn(0x7f71d40722c0 0x7f71d40b51b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:24.468 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:24.463+0000 7f71da97e700 1 -- 172.21.15.59:0/988930715 >> 172.21.15.59:0/988930715 conn(0x7f71d40b2740 msgr2=0x7f71d40b2b20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:58:24.468 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:24.464+0000 7f71da97e700 1 -- 172.21.15.59:0/988930715 shutdown_connections 2023-11-24T07:58:24.468 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:24.464+0000 7f71da97e700 1 -- 172.21.15.59:0/988930715 wait complete. 2023-11-24T07:58:24.470 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:24.465+0000 7f71da97e700 1 Processor -- start 2023-11-24T07:58:24.470 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:24.465+0000 7f71da97e700 1 -- start start 2023-11-24T07:58:24.471 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:24.466+0000 7f71da97e700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f71d40722c0 0x7f71d415b530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:24.471 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:24.466+0000 7f71da97e700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f71d4143ae0 0x7f71d4159d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:24.471 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:24.466+0000 7f71da97e700 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f71d41440f0 0x7f71d415a280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:24.471 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:24.466+0000 7f71da97e700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71d4135490 con 0x7f71d40722c0 2023-11-24T07:58:24.471 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:24.466+0000 7f71da97e700 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71d41f3c30 con 0x7f71d4143ae0 2023-11-24T07:58:24.471 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:24.466+0000 7f71da97e700 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f71d4135670 con 0x7f71d41440f0 2023-11-24T07:58:24.471 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:24.466+0000 7f71d3fff700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f71d40722c0 0x7f71d415b530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:24.471 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:24.466+0000 7f71d3fff700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f71d40722c0 0x7f71d415b530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:53350/0 (socket says 172.21.15.59:53350) 2023-11-24T07:58:24.471 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:24.466+0000 7f71d3fff700 1 -- 172.21.15.59:0/2772976342 learned_addr learned my addr 172.21.15.59:0/2772976342 (peer_addr_for_me v2:172.21.15.59:0/0) 2023-11-24T07:58:24.471 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:24.466+0000 7f71d37fe700 1 --2- 172.21.15.59:0/2772976342 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f71d4143ae0 0x7f71d4159d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:24.471 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:24.466+0000 7f71d8f1b700 1 --2- 172.21.15.59:0/2772976342 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f71d41440f0 0x7f71d415a280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:24.472 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:24.466+0000 7f71d3fff700 1 -- 172.21.15.59:0/2772976342 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f71d41440f0 msgr2=0x7f71d415a280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:24.472 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:24.466+0000 7f71d3fff700 1 --2- 172.21.15.59:0/2772976342 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f71d41440f0 0x7f71d415a280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:24.472 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:24.466+0000 7f71d3fff700 1 -- 172.21.15.59:0/2772976342 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f71d4143ae0 msgr2=0x7f71d4159d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:24.472 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:24.466+0000 7f71d3fff700 1 --2- 172.21.15.59:0/2772976342 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f71d4143ae0 0x7f71d4159d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:24.472 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:24.466+0000 7f71d3fff700 1 -- 172.21.15.59:0/2772976342 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f71cc00bd40 con 0x7f71d40722c0 2023-11-24T07:58:24.472 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:24.466+0000 7f71d8f1b700 1 --2- 172.21.15.59:0/2772976342 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f71d41440f0 0x7f71d415a280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-24T07:58:24.472 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:24.466+0000 7f71d3fff700 1 --2- 172.21.15.59:0/2772976342 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f71d40722c0 0x7f71d415b530 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7f71c400c4d0 tx=0x7f71c400c7e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:58:24.472 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:24.466+0000 7f71d0ff9700 1 -- 172.21.15.59:0/2772976342 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71c4011790 con 0x7f71d40722c0 2023-11-24T07:58:24.472 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:24.467+0000 7f71d0ff9700 1 -- 172.21.15.59:0/2772976342 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f71c40118f0 con 0x7f71d40722c0 2023-11-24T07:58:24.472 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:24.467+0000 7f71da97e700 1 -- 172.21.15.59:0/2772976342 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f71d4135950 con 0x7f71d40722c0 2023-11-24T07:58:24.472 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:24.467+0000 7f71d0ff9700 1 -- 172.21.15.59:0/2772976342 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71c4011ca0 con 0x7f71d40722c0 2023-11-24T07:58:24.472 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:24.467+0000 7f71da97e700 1 -- 172.21.15.59:0/2772976342 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f71d4135f10 con 0x7f71d40722c0 2023-11-24T07:58:24.473 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:24.468+0000 7f71da97e700 1 -- 172.21.15.59:0/2772976342 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f71d4075480 con 0x7f71d40722c0 2023-11-24T07:58:24.473 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:24.468+0000 7f71d0ff9700 1 -- 172.21.15.59:0/2772976342 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 8) v1 ==== 89435+0+0 (secure 0 0 0) 0x7f71c401c460 con 0x7f71d40722c0 2023-11-24T07:58:24.473 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:24.468+0000 7f71d0ff9700 1 --2- 172.21.15.59:0/2772976342 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f71b406b110 0x7f71b406d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:24.473 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:24.469+0000 7f71d0ff9700 1 -- 172.21.15.59:0/2772976342 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 5910+0+0 (secure 0 0 0) 0x7f71c40897b0 con 0x7f71d40722c0 2023-11-24T07:58:24.473 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:24.469+0000 7f71d37fe700 1 --2- 172.21.15.59:0/2772976342 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f71b406b110 0x7f71b406d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:24.473 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:24.469+0000 7f71d37fe700 1 --2- 172.21.15.59:0/2772976342 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f71b406b110 0x7f71b406d5d0 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f71d4143470 tx=0x7f71bc008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:58:24.474 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:24.473+0000 7f71d0ff9700 1 -- 172.21.15.59:0/2772976342 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f71c4054500 con 0x7f71d40722c0 2023-11-24T07:58:24.676 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:24.674+0000 7f71da97e700 1 -- 172.21.15.59:0/2772976342 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f71d415e4b0 con 0x7f71d40722c0 2023-11-24T07:58:24.676 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:24.675+0000 7f71d0ff9700 1 -- 172.21.15.59:0/2772976342 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v48) v1 ==== 100+0+11 (secure 0 0 0) 0x7f71c4057b20 con 0x7f71d40722c0 2023-11-24T07:58:24.676 INFO:teuthology.orchestra.run.smithi059.stdout:pg_num: 64 2023-11-24T07:58:24.679 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:24.677+0000 7f71da97e700 1 -- 172.21.15.59:0/2772976342 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f71b406b110 msgr2=0x7f71b406d5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:24.679 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:24.677+0000 7f71da97e700 1 --2- 172.21.15.59:0/2772976342 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f71b406b110 0x7f71b406d5d0 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f71d4143470 tx=0x7f71bc008040 comp rx=0 tx=0).stop 2023-11-24T07:58:24.679 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:24.677+0000 7f71da97e700 1 -- 172.21.15.59:0/2772976342 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f71d40722c0 msgr2=0x7f71d415b530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:24.679 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:24.677+0000 7f71da97e700 1 --2- 172.21.15.59:0/2772976342 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f71d40722c0 0x7f71d415b530 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7f71c400c4d0 tx=0x7f71c400c7e0 comp rx=0 tx=0).stop 2023-11-24T07:58:24.679 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:24.677+0000 7f71da97e700 1 -- 172.21.15.59:0/2772976342 shutdown_connections 2023-11-24T07:58:24.679 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:24.677+0000 7f71da97e700 1 --2- 172.21.15.59:0/2772976342 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f71b406b110 0x7f71b406d5d0 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:24.679 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:24.677+0000 7f71da97e700 1 --2- 172.21.15.59:0/2772976342 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f71d41440f0 0x7f71d415a280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:24.680 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:24.677+0000 7f71da97e700 1 --2- 172.21.15.59:0/2772976342 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f71d40722c0 0x7f71d415b530 unknown :-1 s=CLOSED pgs=628 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:24.680 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:24.677+0000 7f71da97e700 1 --2- 172.21.15.59:0/2772976342 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f71d4143ae0 0x7f71d4159d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:24.680 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:24.677+0000 7f71da97e700 1 -- 172.21.15.59:0/2772976342 >> 172.21.15.59:0/2772976342 conn(0x7f71d40b2740 msgr2=0x7f71d40b42d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:58:24.680 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:24.677+0000 7f71da97e700 1 -- 172.21.15.59:0/2772976342 shutdown_connections 2023-11-24T07:58:24.680 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:24.677+0000 7f71da97e700 1 -- 172.21.15.59:0/2772976342 wait complete. 2023-11-24T07:58:24.690 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-24T07:58:24.815 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:24.813+0000 7f8b22a10700 1 Processor -- start 2023-11-24T07:58:24.815 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:24.813+0000 7f8b22a10700 1 -- start start 2023-11-24T07:58:24.817 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:24.814+0000 7f8b22a10700 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f8b1c15e000 0x7f8b1c06eef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:24.818 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:24.814+0000 7f8b22a10700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f8b1c0d0ae0 0x7f8b1c06f6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:24.818 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:24.814+0000 7f8b22a10700 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f8b1c0d0f60 0x7f8b1c06fc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:24.818 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:24.814+0000 7f8b22a10700 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8b1c0cf4b0 con 0x7f8b1c0d0ae0 2023-11-24T07:58:24.818 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:24.814+0000 7f8b22a10700 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8b1c0cfd60 con 0x7f8b1c0d0f60 2023-11-24T07:58:24.818 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:24.814+0000 7f8b22a10700 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8b1c06f430 con 0x7f8b1c15e000 2023-11-24T07:58:24.818 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:24.814+0000 7f8b1b7fe700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f8b1c0d0ae0 0x7f8b1c06f6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:24.818 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:24.814+0000 7f8b1b7fe700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f8b1c0d0ae0 0x7f8b1c06f6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:53358/0 (socket says 172.21.15.59:53358) 2023-11-24T07:58:24.818 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:24.814+0000 7f8b1b7fe700 1 -- 172.21.15.59:0/3672306198 learned_addr learned my addr 172.21.15.59:0/3672306198 (peer_addr_for_me v2:172.21.15.59:0/0) 2023-11-24T07:58:24.818 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:24.814+0000 7f8b20fad700 1 --2- 172.21.15.59:0/3672306198 >> v2:172.21.15.76:3300/0 conn(0x7f8b1c0d0f60 0x7f8b1c06fc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:24.819 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:24.814+0000 7f8b1bfff700 1 --2- 172.21.15.59:0/3672306198 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f8b1c15e000 0x7f8b1c06eef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:24.819 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:24.815+0000 7f8b1b7fe700 1 -- 172.21.15.59:0/3672306198 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f8b1c15e000 msgr2=0x7f8b1c06eef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T07:58:24.819 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:24.815+0000 7f8b1b7fe700 1 --2- 172.21.15.59:0/3672306198 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f8b1c15e000 0x7f8b1c06eef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:24.819 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:24.815+0000 7f8b1b7fe700 1 -- 172.21.15.59:0/3672306198 >> v2:172.21.15.76:3300/0 conn(0x7f8b1c0d0f60 msgr2=0x7f8b1c06fc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T07:58:24.819 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:24.815+0000 7f8b1b7fe700 1 --2- 172.21.15.59:0/3672306198 >> v2:172.21.15.76:3300/0 conn(0x7f8b1c0d0f60 0x7f8b1c06fc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:24.819 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:24.815+0000 7f8b1b7fe700 1 -- 172.21.15.59:0/3672306198 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8b1c0cf1d0 con 0x7f8b1c0d0ae0 2023-11-24T07:58:24.819 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:24.815+0000 7f8b20fad700 1 --2- 172.21.15.59:0/3672306198 >> v2:172.21.15.76:3300/0 conn(0x7f8b1c0d0f60 0x7f8b1c06fc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-24T07:58:24.819 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:24.815+0000 7f8b1bfff700 1 --2- 172.21.15.59:0/3672306198 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f8b1c15e000 0x7f8b1c06eef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-24T07:58:24.819 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:24.815+0000 7f8b1b7fe700 1 --2- 172.21.15.59:0/3672306198 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8b1c0d0ae0 0x7f8b1c06f6f0 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7f8b1c073110 tx=0x7f8b0c009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=42cad14c97bbf0b9 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:58:24.819 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:24.815+0000 7f8b1affd700 1 -- 172.21.15.59:0/3672306198 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b0c00b870 con 0x7f8b1c0d0ae0 2023-11-24T07:58:24.819 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:24.815+0000 7f8b1affd700 1 -- 172.21.15.59:0/3672306198 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8b0c00b9d0 con 0x7f8b1c0d0ae0 2023-11-24T07:58:24.820 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:24.815+0000 7f8b1affd700 1 -- 172.21.15.59:0/3672306198 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b0c00bd60 con 0x7f8b1c0d0ae0 2023-11-24T07:58:24.820 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:24.815+0000 7f8b22a10700 1 -- 172.21.15.59:0/3672306198 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8b1c0d0ae0 msgr2=0x7f8b1c06f6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:24.820 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:24.815+0000 7f8b22a10700 1 --2- 172.21.15.59:0/3672306198 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8b1c0d0ae0 0x7f8b1c06f6f0 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7f8b1c073110 tx=0x7f8b0c009bd0 comp rx=0 tx=0).stop 2023-11-24T07:58:24.820 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:24.816+0000 7f8b22a10700 1 -- 172.21.15.59:0/3672306198 shutdown_connections 2023-11-24T07:58:24.820 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:24.816+0000 7f8b22a10700 1 --2- 172.21.15.59:0/3672306198 >> v2:172.21.15.76:3300/0 conn(0x7f8b1c0d0f60 0x7f8b1c06fc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:24.820 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:24.816+0000 7f8b22a10700 1 --2- 172.21.15.59:0/3672306198 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f8b1c15e000 0x7f8b1c06eef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:24.820 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:24.816+0000 7f8b22a10700 1 --2- 172.21.15.59:0/3672306198 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8b1c0d0ae0 0x7f8b1c06f6f0 unknown :-1 s=CLOSED pgs=629 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:24.820 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:24.816+0000 7f8b22a10700 1 -- 172.21.15.59:0/3672306198 >> 172.21.15.59:0/3672306198 conn(0x7f8b1c0b3720 msgr2=0x7f8b1c0b3b30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:58:24.820 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:24.816+0000 7f8b22a10700 1 -- 172.21.15.59:0/3672306198 shutdown_connections 2023-11-24T07:58:24.820 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:24.816+0000 7f8b22a10700 1 -- 172.21.15.59:0/3672306198 wait complete. 2023-11-24T07:58:24.823 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:24.817+0000 7f8b22a10700 1 Processor -- start 2023-11-24T07:58:24.823 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:24.817+0000 7f8b22a10700 1 -- start start 2023-11-24T07:58:24.823 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:24.818+0000 7f8b22a10700 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f8b1c0d0ae0 0x7f8b1c1f3a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:24.823 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:24.818+0000 7f8b22a10700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8b1c0d0f60 0x7f8b1c15e3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:24.823 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:24.818+0000 7f8b22a10700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8b1c15e000 0x7f8b1c160d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:24.823 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:24.818+0000 7f8b22a10700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b1c147b10 con 0x7f8b1c0d0f60 2023-11-24T07:58:24.823 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:24.818+0000 7f8b22a10700 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b1c1f3fb0 con 0x7f8b1c15e000 2023-11-24T07:58:24.824 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:24.818+0000 7f8b22a10700 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8b1c1468f0 con 0x7f8b1c0d0ae0 2023-11-24T07:58:24.824 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:24.818+0000 7f8b1b7fe700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8b1c0d0f60 0x7f8b1c15e3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:24.824 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:24.818+0000 7f8b1b7fe700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8b1c0d0f60 0x7f8b1c15e3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:53364/0 (socket says 172.21.15.59:53364) 2023-11-24T07:58:24.824 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:24.818+0000 7f8b1b7fe700 1 -- 172.21.15.59:0/1298213253 learned_addr learned my addr 172.21.15.59:0/1298213253 (peer_addr_for_me v2:172.21.15.59:0/0) 2023-11-24T07:58:24.824 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:24.818+0000 7f8b1bfff700 1 --2- 172.21.15.59:0/1298213253 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f8b1c0d0ae0 0x7f8b1c1f3a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:24.824 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:24.818+0000 7f8b20fad700 1 --2- 172.21.15.59:0/1298213253 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8b1c15e000 0x7f8b1c160d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:24.824 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:24.818+0000 7f8b1b7fe700 1 -- 172.21.15.59:0/1298213253 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f8b1c0d0ae0 msgr2=0x7f8b1c1f3a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:24.824 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:24.818+0000 7f8b1b7fe700 1 --2- 172.21.15.59:0/1298213253 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f8b1c0d0ae0 0x7f8b1c1f3a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:24.824 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:24.818+0000 7f8b1b7fe700 1 -- 172.21.15.59:0/1298213253 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8b1c15e000 msgr2=0x7f8b1c160d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:24.824 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:24.818+0000 7f8b1b7fe700 1 --2- 172.21.15.59:0/1298213253 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8b1c15e000 0x7f8b1c160d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:24.824 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:24.818+0000 7f8b1b7fe700 1 -- 172.21.15.59:0/1298213253 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8b0c009550 con 0x7f8b1c0d0f60 2023-11-24T07:58:24.824 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:24.818+0000 7f8b1b7fe700 1 --2- 172.21.15.59:0/1298213253 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8b1c0d0f60 0x7f8b1c15e3e0 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7f8b0c014040 tx=0x7f8b0c004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:58:24.825 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:24.818+0000 7f8b18ff9700 1 -- 172.21.15.59:0/1298213253 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b0c004520 con 0x7f8b1c0d0f60 2023-11-24T07:58:24.825 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:24.819+0000 7f8b18ff9700 1 -- 172.21.15.59:0/1298213253 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8b0c01e070 con 0x7f8b1c0d0f60 2023-11-24T07:58:24.825 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:24.819+0000 7f8b22a10700 1 -- 172.21.15.59:0/1298213253 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8b1c149aa0 con 0x7f8b1c0d0f60 2023-11-24T07:58:24.825 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:24.819+0000 7f8b18ff9700 1 -- 172.21.15.59:0/1298213253 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b0c0038c0 con 0x7f8b1c0d0f60 2023-11-24T07:58:24.825 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:24.819+0000 7f8b22a10700 1 -- 172.21.15.59:0/1298213253 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8b1c146e90 con 0x7f8b1c0d0f60 2023-11-24T07:58:24.825 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:24.820+0000 7f8b22a10700 1 -- 172.21.15.59:0/1298213253 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8b1c147c50 con 0x7f8b1c0d0f60 2023-11-24T07:58:24.825 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:24.820+0000 7f8b18ff9700 1 -- 172.21.15.59:0/1298213253 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 8) v1 ==== 89435+0+0 (secure 0 0 0) 0x7f8b0c003a20 con 0x7f8b1c0d0f60 2023-11-24T07:58:24.825 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:24.820+0000 7f8b18ff9700 1 --2- 172.21.15.59:0/1298213253 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f8afc06b110 0x7f8afc06d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:24.825 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:24.821+0000 7f8b18ff9700 1 -- 172.21.15.59:0/1298213253 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 5910+0+0 (secure 0 0 0) 0x7f8b0c020070 con 0x7f8b1c0d0f60 2023-11-24T07:58:24.825 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:24.821+0000 7f8b1bfff700 1 --2- 172.21.15.59:0/1298213253 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f8afc06b110 0x7f8afc06d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:24.825 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:24.821+0000 7f8b1bfff700 1 --2- 172.21.15.59:0/1298213253 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f8afc06b110 0x7f8afc06d5d0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f8b04009870 tx=0x7f8b04008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:58:24.827 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:24.825+0000 7f8b18ff9700 1 -- 172.21.15.59:0/1298213253 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f8b1c147c50 con 0x7f8b1c0d0f60 2023-11-24T07:58:25.069 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:25.068+0000 7f8b22a10700 1 -- 172.21.15.59:0/1298213253 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f8b1c147c50 con 0x7f8b1c0d0f60 2023-11-24T07:58:25.070 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:25.068+0000 7f8b18ff9700 1 -- 172.21.15.59:0/1298213253 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) v1 ==== 94+0+4904 (secure 0 0 0) 0x7f8b1c147c50 con 0x7f8b1c0d0f60 2023-11-24T07:58:25.070 INFO:teuthology.orchestra.run.smithi059.stdout: 2023-11-24T07:58:25.070 INFO:teuthology.orchestra.run.smithi059.stdout:{"epoch":26,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5290,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.76:6836/508095662","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6834","nonce":508095662},{"type":"v1","addr":"172.21.15.76:6836","nonce":508095662}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":20},{"gid":5293,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.76:6837/4158118811","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6835","nonce":4158118811},{"type":"v1","addr":"172.21.15.76:6837","nonce":4158118811}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":20},{"gid":5622,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6834/59936286","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6832","nonce":59936286},{"type":"v1","addr":"172.21.15.59:6834","nonce":59936286}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":20}],"filesystems":[{"mdsmap":{"epoch":25,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-24T07:58:02.020496+0000","modified":"2023-11-24T07:58:05.037742+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5625},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5625":{"gid":5625,"name":"c","rank":0,"incarnation":21,"state":"up:active","state_seq":3,"addr":"172.21.15.59:6835/1468266626","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6833","nonce":1468266626},{"type":"v1","addr":"172.21.15.59:6835","nonce":1468266626}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2023-11-24T07:58:25.072 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:25.070+0000 7f8b22a10700 1 -- 172.21.15.59:0/1298213253 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f8afc06b110 msgr2=0x7f8afc06d5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:25.073 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:25.070+0000 7f8b22a10700 1 --2- 172.21.15.59:0/1298213253 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f8afc06b110 0x7f8afc06d5d0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f8b04009870 tx=0x7f8b04008040 comp rx=0 tx=0).stop 2023-11-24T07:58:25.073 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:25.070+0000 7f8b22a10700 1 -- 172.21.15.59:0/1298213253 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8b1c0d0f60 msgr2=0x7f8b1c15e3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:25.073 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:25.070+0000 7f8b22a10700 1 --2- 172.21.15.59:0/1298213253 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8b1c0d0f60 0x7f8b1c15e3e0 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7f8b0c014040 tx=0x7f8b0c004750 comp rx=0 tx=0).stop 2023-11-24T07:58:25.073 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:25.071+0000 7f8b22a10700 1 -- 172.21.15.59:0/1298213253 shutdown_connections 2023-11-24T07:58:25.073 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:25.071+0000 7f8b22a10700 1 --2- 172.21.15.59:0/1298213253 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f8afc06b110 0x7f8afc06d5d0 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:25.073 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:25.071+0000 7f8b22a10700 1 --2- 172.21.15.59:0/1298213253 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8b1c15e000 0x7f8b1c160d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:25.073 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:25.071+0000 7f8b22a10700 1 --2- 172.21.15.59:0/1298213253 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f8b1c0d0ae0 0x7f8b1c1f3a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:25.073 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:25.071+0000 7f8b22a10700 1 --2- 172.21.15.59:0/1298213253 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8b1c0d0f60 0x7f8b1c15e3e0 unknown :-1 s=CLOSED pgs=630 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:25.073 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:25.071+0000 7f8b22a10700 1 -- 172.21.15.59:0/1298213253 >> 172.21.15.59:0/1298213253 conn(0x7f8b1c0b3720 msgr2=0x7f8b1c0b3b00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:58:25.073 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:25.071+0000 7f8b22a10700 1 -- 172.21.15.59:0/1298213253 shutdown_connections 2023-11-24T07:58:25.073 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:25.071+0000 7f8b22a10700 1 -- 172.21.15.59:0/1298213253 wait complete. 2023-11-24T07:58:25.074 INFO:teuthology.orchestra.run.smithi059.stderr:dumped fsmap epoch 26 2023-11-24T07:58:25.085 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-24T07:58:25.209 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:25.207+0000 7ff74b91f700 1 Processor -- start 2023-11-24T07:58:25.209 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:25.207+0000 7ff74b91f700 1 -- start start 2023-11-24T07:58:25.211 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:25.208+0000 7ff74b91f700 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7ff7440b4150 0x7ff74406de50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:25.211 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:25.208+0000 7ff74b91f700 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7ff7440b5ab0 0x7ff74406e390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:25.212 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:25.208+0000 7ff74b91f700 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff744071f50 con 0x7ff74406b620 2023-11-24T07:58:25.212 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:25.208+0000 7ff74b91f700 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff744072230 con 0x7ff7440b4150 2023-11-24T07:58:25.212 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:25.208+0000 7ff74b91f700 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff744072510 con 0x7ff7440b5ab0 2023-11-24T07:58:25.212 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:25.208+0000 7ff7496bb700 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7ff74406b620 0x7ff7440b3a40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:56846/0 (socket says 172.21.15.59:56846) 2023-11-24T07:58:25.212 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:25.208+0000 7ff7496bb700 1 -- 172.21.15.59:0/3827092715 learned_addr learned my addr 172.21.15.59:0/3827092715 (peer_addr_for_me v1:172.21.15.59:0/0) 2023-11-24T07:58:25.212 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:25.208+0000 7ff749ebc700 1 --2- 172.21.15.59:0/3827092715 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7ff7440b5ab0 0x7ff74406e390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:25.212 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:25.208+0000 7ff748eba700 1 --2- 172.21.15.59:0/3827092715 >> v2:172.21.15.76:3300/0 conn(0x7ff7440b4150 0x7ff74406de50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:25.212 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:25.208+0000 7ff73bfff700 1 -- 172.21.15.59:0/3827092715 <== mon.0 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1404504937 0 0) 0x7ff744071f50 con 0x7ff74406b620 2023-11-24T07:58:25.212 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:25.209+0000 7ff73bfff700 1 -- 172.21.15.59:0/3827092715 --> v1:172.21.15.59:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff7300035b0 con 0x7ff74406b620 2023-11-24T07:58:25.212 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:25.209+0000 7ff73bfff700 1 -- 172.21.15.59:0/3827092715 <== mon.0 v1:172.21.15.59:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2694670187 0 0) 0x7ff7300035b0 con 0x7ff74406b620 2023-11-24T07:58:25.212 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:25.209+0000 7ff749ebc700 1 -- 172.21.15.59:0/3827092715 >> v2:172.21.15.76:3300/0 conn(0x7ff7440b4150 msgr2=0x7ff74406de50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T07:58:25.213 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:25.209+0000 7ff749ebc700 1 --2- 172.21.15.59:0/3827092715 >> v2:172.21.15.76:3300/0 conn(0x7ff7440b4150 0x7ff74406de50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:25.213 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:25.209+0000 7ff749ebc700 1 -- 172.21.15.59:0/3827092715 >> v1:172.21.15.59:6789/0 conn(0x7ff74406b620 legacy=0x7ff7440b3a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:25.213 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:25.209+0000 7ff749ebc700 1 -- 172.21.15.59:0/3827092715 --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff7440727f0 con 0x7ff7440b5ab0 2023-11-24T07:58:25.213 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:25.209+0000 7ff749ebc700 1 --2- 172.21.15.59:0/3827092715 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7ff7440b5ab0 0x7ff74406e390 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7ff73c009d90 tx=0x7ff73c00b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=194d7347726144c6 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:58:25.213 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:25.209+0000 7ff73bfff700 1 -- 172.21.15.59:0/3827092715 <== mon.2 v2:172.21.15.76:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff73c00e070 con 0x7ff7440b5ab0 2023-11-24T07:58:25.213 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:25.209+0000 7ff73bfff700 1 -- 172.21.15.59:0/3827092715 <== mon.2 v2:172.21.15.76:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff73c003a20 con 0x7ff7440b5ab0 2023-11-24T07:58:25.213 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:25.209+0000 7ff73bfff700 1 -- 172.21.15.59:0/3827092715 <== mon.2 v2:172.21.15.76:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff734001670 con 0x7ff7440b5ab0 2023-11-24T07:58:25.213 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:25.210+0000 7ff74b91f700 1 -- 172.21.15.59:0/3827092715 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7ff7440b5ab0 msgr2=0x7ff74406e390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:25.213 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:25.210+0000 7ff74b91f700 1 --2- 172.21.15.59:0/3827092715 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7ff7440b5ab0 0x7ff74406e390 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7ff73c009d90 tx=0x7ff73c00b330 comp rx=0 tx=0).stop 2023-11-24T07:58:25.213 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:25.210+0000 7ff74b91f700 1 -- 172.21.15.59:0/3827092715 shutdown_connections 2023-11-24T07:58:25.213 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:25.210+0000 7ff74b91f700 1 --2- 172.21.15.59:0/3827092715 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7ff7440b5ab0 0x7ff74406e390 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:25.213 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:25.210+0000 7ff74b91f700 1 --2- 172.21.15.59:0/3827092715 >> v2:172.21.15.76:3300/0 conn(0x7ff7440b4150 0x7ff74406de50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:25.214 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:25.210+0000 7ff74b91f700 1 -- 172.21.15.59:0/3827092715 >> 172.21.15.59:0/3827092715 conn(0x7ff7440b2740 msgr2=0x7ff7440b2b20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:58:25.214 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:25.210+0000 7ff74b91f700 1 -- 172.21.15.59:0/3827092715 shutdown_connections 2023-11-24T07:58:25.214 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:25.210+0000 7ff74b91f700 1 -- 172.21.15.59:0/3827092715 wait complete. 2023-11-24T07:58:25.216 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:25.211+0000 7ff74b91f700 1 Processor -- start 2023-11-24T07:58:25.216 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:25.211+0000 7ff74b91f700 1 -- start start 2023-11-24T07:58:25.216 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:25.212+0000 7ff74b91f700 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7ff74406b620 0x7ff744145db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:25.216 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:25.212+0000 7ff74b91f700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff7440b4150 0x7ff7440caad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:25.216 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:25.212+0000 7ff74b91f700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff7440b5ab0 0x7ff7440cb010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:25.217 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:25.212+0000 7ff74b91f700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7440c6930 con 0x7ff7440b5ab0 2023-11-24T07:58:25.217 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:25.212+0000 7ff74b91f700 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7440c6c10 con 0x7ff7440b4150 2023-11-24T07:58:25.217 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:25.212+0000 7ff74b91f700 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff7440c6ef0 con 0x7ff74406b620 2023-11-24T07:58:25.217 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:25.212+0000 7ff749ebc700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff7440b5ab0 0x7ff7440cb010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:25.217 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:25.212+0000 7ff749ebc700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff7440b5ab0 0x7ff7440cb010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:53382/0 (socket says 172.21.15.59:53382) 2023-11-24T07:58:25.217 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:25.212+0000 7ff749ebc700 1 -- 172.21.15.59:0/2923622070 learned_addr learned my addr 172.21.15.59:0/2923622070 (peer_addr_for_me v2:172.21.15.59:0/0) 2023-11-24T07:58:25.217 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:25.212+0000 7ff7496bb700 1 --2- 172.21.15.59:0/2923622070 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7ff74406b620 0x7ff744145db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:25.217 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:25.212+0000 7ff748eba700 1 --2- 172.21.15.59:0/2923622070 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff7440b4150 0x7ff7440caad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:25.217 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:25.212+0000 7ff749ebc700 1 -- 172.21.15.59:0/2923622070 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7ff74406b620 msgr2=0x7ff744145db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:25.217 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:25.212+0000 7ff749ebc700 1 --2- 172.21.15.59:0/2923622070 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7ff74406b620 0x7ff744145db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:25.217 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:25.212+0000 7ff749ebc700 1 -- 172.21.15.59:0/2923622070 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff7440b4150 msgr2=0x7ff7440caad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:25.217 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:25.212+0000 7ff749ebc700 1 --2- 172.21.15.59:0/2923622070 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff7440b4150 0x7ff7440caad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:25.218 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:25.212+0000 7ff749ebc700 1 -- 172.21.15.59:0/2923622070 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff73c009a70 con 0x7ff7440b5ab0 2023-11-24T07:58:25.218 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:25.212+0000 7ff7496bb700 1 --2- 172.21.15.59:0/2923622070 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7ff74406b620 0x7ff744145db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-24T07:58:25.218 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:25.213+0000 7ff749ebc700 1 --2- 172.21.15.59:0/2923622070 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff7440b5ab0 0x7ff7440cb010 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7ff744072650 tx=0x7ff73c004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:58:25.218 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:25.213+0000 7ff739ffb700 1 -- 172.21.15.59:0/2923622070 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff73c0045f0 con 0x7ff7440b5ab0 2023-11-24T07:58:25.218 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:25.213+0000 7ff739ffb700 1 -- 172.21.15.59:0/2923622070 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff73c004750 con 0x7ff7440b5ab0 2023-11-24T07:58:25.218 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:25.213+0000 7ff74b91f700 1 -- 172.21.15.59:0/2923622070 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff7440c71d0 con 0x7ff7440b5ab0 2023-11-24T07:58:25.218 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:25.213+0000 7ff739ffb700 1 -- 172.21.15.59:0/2923622070 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff73c009450 con 0x7ff7440b5ab0 2023-11-24T07:58:25.218 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:25.213+0000 7ff74b91f700 1 -- 172.21.15.59:0/2923622070 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff7441462f0 con 0x7ff7440b5ab0 2023-11-24T07:58:25.218 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:25.214+0000 7ff74b91f700 1 -- 172.21.15.59:0/2923622070 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff7440c8b30 con 0x7ff7440b5ab0 2023-11-24T07:58:25.218 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:25.214+0000 7ff739ffb700 1 -- 172.21.15.59:0/2923622070 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 8) v1 ==== 89435+0+0 (secure 0 0 0) 0x7ff73c012490 con 0x7ff7440b5ab0 2023-11-24T07:58:25.219 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:25.215+0000 7ff739ffb700 1 --2- 172.21.15.59:0/2923622070 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7ff72806b110 0x7ff72806d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:25.219 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:25.215+0000 7ff739ffb700 1 -- 172.21.15.59:0/2923622070 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 5910+0+0 (secure 0 0 0) 0x7ff73c08bd20 con 0x7ff7440b5ab0 2023-11-24T07:58:25.219 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:25.215+0000 7ff7496bb700 1 --2- 172.21.15.59:0/2923622070 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7ff72806b110 0x7ff72806d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:25.219 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:25.215+0000 7ff7496bb700 1 --2- 172.21.15.59:0/2923622070 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7ff72806b110 0x7ff72806d5d0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7ff7340011e0 tx=0x7ff734007b00 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:58:25.221 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:25.219+0000 7ff739ffb700 1 -- 172.21.15.59:0/2923622070 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7ff73c056880 con 0x7ff7440b5ab0 2023-11-24T07:58:25.423 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:25.421+0000 7ff74b91f700 1 -- 172.21.15.59:0/2923622070 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7ff744068180 con 0x7ff7440b5ab0 2023-11-24T07:58:25.424 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:25.422+0000 7ff739ffb700 1 -- 172.21.15.59:0/2923622070 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v48) v1 ==== 74+0+16776 (secure 0 0 0) 0x7ff73c059ea0 con 0x7ff7440b5ab0 2023-11-24T07:58:25.424 INFO:teuthology.orchestra.run.smithi059.stdout: 2023-11-24T07:58:25.424 INFO:teuthology.orchestra.run.smithi059.stdout:{"epoch":48,"fsid":"9f0e8741-b8f9-48d5-a2c3-09f45903303d","created":"2023-11-24T07:56:06.552729+0000","modified":"2023-11-24T07:58:22.384748+0000","last_up_change":"2023-11-24T07:56:17.372352+0000","last_in_change":"2023-11-24T07:56:09.073250+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-24T07:56:18.352823+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_metadata","create_time":"2023-11-24T07:58:00.240130+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"44","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_data","create_time":"2023-11-24T07:58:00.977486+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"48","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":3,"snap_epoch":48,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"6d8f9e9b-1782-4e41-928b-a39f2a8ebc06","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6800","nonce":3373878772},{"type":"v1","addr":"172.21.15.59:6801","nonce":3373878772}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6802","nonce":3373878772},{"type":"v1","addr":"172.21.15.59:6803","nonce":3373878772}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6806","nonce":3373878772},{"type":"v1","addr":"172.21.15.59:6807","nonce":3373878772}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6804","nonce":3373878772},{"type":"v1","addr":"172.21.15.59:6805","nonce":3373878772}]},"public_addr":"172.21.15.59:6801/3373878772","cluster_addr":"172.21.15.59:6803/3373878772","heartbeat_back_addr":"172.21.15.59:6807/3373878772","heartbeat_front_addr":"172.21.15.59:6805/3373878772","state":["exists","up"]},{"osd":1,"uuid":"e8d882aa-52e1-4c04-9e1e-c44ef9204a5f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6824","nonce":648258563},{"type":"v1","addr":"172.21.15.59:6825","nonce":648258563}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6826","nonce":648258563},{"type":"v1","addr":"172.21.15.59:6827","nonce":648258563}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6830","nonce":648258563},{"type":"v1","addr":"172.21.15.59:6831","nonce":648258563}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6828","nonce":648258563},{"type":"v1","addr":"172.21.15.59:6829","nonce":648258563}]},"public_addr":"172.21.15.59:6825/648258563","cluster_addr":"172.21.15.59:6827/648258563","heartbeat_back_addr":"172.21.15.59:6831/648258563","heartbeat_front_addr":"172.21.15.59:6829/648258563","state":["exists","up"]},{"osd":2,"uuid":"69eebfa0-695b-4b21-90b9-8a3537168a01","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6808","nonce":1840430654},{"type":"v1","addr":"172.21.15.59:6809","nonce":1840430654}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6810","nonce":1840430654},{"type":"v1","addr":"172.21.15.59:6811","nonce":1840430654}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6814","nonce":1840430654},{"type":"v1","addr":"172.21.15.59:6815","nonce":1840430654}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6812","nonce":1840430654},{"type":"v1","addr":"172.21.15.59:6813","nonce":1840430654}]},"public_addr":"172.21.15.59:6809/1840430654","cluster_addr":"172.21.15.59:6811/1840430654","heartbeat_back_addr":"172.21.15.59:6815/1840430654","heartbeat_front_addr":"172.21.15.59:6813/1840430654","state":["exists","up"]},{"osd":3,"uuid":"96fd77fc-f911-46ff-9445-6f0be2daa8f4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6816","nonce":2504025850},{"type":"v1","addr":"172.21.15.59:6817","nonce":2504025850}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6818","nonce":2504025850},{"type":"v1","addr":"172.21.15.59:6819","nonce":2504025850}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6822","nonce":2504025850},{"type":"v1","addr":"172.21.15.59:6823","nonce":2504025850}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6820","nonce":2504025850},{"type":"v1","addr":"172.21.15.59:6821","nonce":2504025850}]},"public_addr":"172.21.15.59:6817/2504025850","cluster_addr":"172.21.15.59:6819/2504025850","heartbeat_back_addr":"172.21.15.59:6823/2504025850","heartbeat_front_addr":"172.21.15.59:6821/2504025850","state":["exists","up"]},{"osd":4,"uuid":"7ac29fca-b5b2-4c95-9d69-2f164cbbc708","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6816","nonce":2382463431},{"type":"v1","addr":"172.21.15.76:6818","nonce":2382463431}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6820","nonce":2382463431},{"type":"v1","addr":"172.21.15.76:6822","nonce":2382463431}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6828","nonce":2382463431},{"type":"v1","addr":"172.21.15.76:6830","nonce":2382463431}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6824","nonce":2382463431},{"type":"v1","addr":"172.21.15.76:6826","nonce":2382463431}]},"public_addr":"172.21.15.76:6818/2382463431","cluster_addr":"172.21.15.76:6822/2382463431","heartbeat_back_addr":"172.21.15.76:6830/2382463431","heartbeat_front_addr":"172.21.15.76:6826/2382463431","state":["exists","up"]},{"osd":5,"uuid":"485bf711-3851-4238-a4f4-51d52e0a9c58","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6800","nonce":329334308},{"type":"v1","addr":"172.21.15.76:6802","nonce":329334308}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6805","nonce":329334308},{"type":"v1","addr":"172.21.15.76:6807","nonce":329334308}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6814","nonce":329334308},{"type":"v1","addr":"172.21.15.76:6815","nonce":329334308}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6810","nonce":329334308},{"type":"v1","addr":"172.21.15.76:6812","nonce":329334308}]},"public_addr":"172.21.15.76:6802/329334308","cluster_addr":"172.21.15.76:6807/329334308","heartbeat_back_addr":"172.21.15.76:6815/329334308","heartbeat_front_addr":"172.21.15.76:6812/329334308","state":["exists","up"]},{"osd":6,"uuid":"72b3f2fc-1e4a-4e32-968d-003647a0286f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6801","nonce":3946579410},{"type":"v1","addr":"172.21.15.76:6803","nonce":3946579410}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6804","nonce":3946579410},{"type":"v1","addr":"172.21.15.76:6806","nonce":3946579410}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6811","nonce":3946579410},{"type":"v1","addr":"172.21.15.76:6813","nonce":3946579410}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6808","nonce":3946579410},{"type":"v1","addr":"172.21.15.76:6809","nonce":3946579410}]},"public_addr":"172.21.15.76:6803/3946579410","cluster_addr":"172.21.15.76:6806/3946579410","heartbeat_back_addr":"172.21.15.76:6813/3946579410","heartbeat_front_addr":"172.21.15.76:6809/3946579410","state":["exists","up"]},{"osd":7,"uuid":"1ddceb90-6c5d-4d1a-ad56-99cb877f8c70","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6817","nonce":841761538},{"type":"v1","addr":"172.21.15.76:6819","nonce":841761538}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6821","nonce":841761538},{"type":"v1","addr":"172.21.15.76:6823","nonce":841761538}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6829","nonce":841761538},{"type":"v1","addr":"172.21.15.76:6831","nonce":841761538}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6825","nonce":841761538},{"type":"v1","addr":"172.21.15.76:6827","nonce":841761538}]},"public_addr":"172.21.15.76:6819/841761538","cluster_addr":"172.21.15.76:6823/841761538","heartbeat_back_addr":"172.21.15.76:6831/841761538","heartbeat_front_addr":"172.21.15.76:6827/841761538","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T07:56:14.294838+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T07:56:14.270779+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T07:56:14.231809+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T07:56:14.260983+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T07:56:14.302997+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T07:56:14.258600+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T07:56:14.251413+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T07:56:14.304727+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"7.0","mappings":[{"from":5,"to":7}]},{"pgid":"7.c","mappings":[{"from":5,"to":7}]},{"pgid":"7.12","mappings":[{"from":5,"to":7}]},{"pgid":"7.14","mappings":[{"from":5,"to":7}]},{"pgid":"7.38","mappings":[{"from":5,"to":1}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":7,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[{"pool":7,"snaps":[{"begin":2,"length":2}]}],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-24T07:58:25.427 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:25.425+0000 7ff74b91f700 1 -- 172.21.15.59:0/2923622070 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7ff72806b110 msgr2=0x7ff72806d5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:25.427 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:25.425+0000 7ff74b91f700 1 --2- 172.21.15.59:0/2923622070 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7ff72806b110 0x7ff72806d5d0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7ff7340011e0 tx=0x7ff734007b00 comp rx=0 tx=0).stop 2023-11-24T07:58:25.427 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:25.425+0000 7ff74b91f700 1 -- 172.21.15.59:0/2923622070 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff7440b5ab0 msgr2=0x7ff7440cb010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:25.427 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:25.425+0000 7ff74b91f700 1 --2- 172.21.15.59:0/2923622070 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff7440b5ab0 0x7ff7440cb010 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7ff744072650 tx=0x7ff73c004ad0 comp rx=0 tx=0).stop 2023-11-24T07:58:25.427 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:25.425+0000 7ff74b91f700 1 -- 172.21.15.59:0/2923622070 shutdown_connections 2023-11-24T07:58:25.427 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:25.425+0000 7ff74b91f700 1 --2- 172.21.15.59:0/2923622070 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7ff72806b110 0x7ff72806d5d0 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:25.427 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:25.425+0000 7ff74b91f700 1 --2- 172.21.15.59:0/2923622070 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff7440b5ab0 0x7ff7440cb010 unknown :-1 s=CLOSED pgs=632 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:25.427 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:25.425+0000 7ff74b91f700 1 --2- 172.21.15.59:0/2923622070 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7ff74406b620 0x7ff744145db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:25.427 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:25.425+0000 7ff74b91f700 1 --2- 172.21.15.59:0/2923622070 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff7440b4150 0x7ff7440caad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:25.428 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:25.425+0000 7ff74b91f700 1 -- 172.21.15.59:0/2923622070 >> 172.21.15.59:0/2923622070 conn(0x7ff7440b2740 msgr2=0x7ff7440b1090 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:58:25.428 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:25.425+0000 7ff74b91f700 1 -- 172.21.15.59:0/2923622070 shutdown_connections 2023-11-24T07:58:25.428 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:25.425+0000 7ff74b91f700 1 -- 172.21.15.59:0/2923622070 wait complete. 2023-11-24T07:58:25.439 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2023-11-24T07:58:25.439 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-24T07:58:25.563 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:25.562+0000 7f0824833700 1 Processor -- start 2023-11-24T07:58:25.564 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:25.562+0000 7f0824833700 1 -- start start 2023-11-24T07:58:25.566 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:25.563+0000 7f0824833700 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f0820143ae0 0x7f08200b51b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:25.566 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:25.563+0000 7f0824833700 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f08201440f0 0x7f0820071bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:25.566 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:25.563+0000 7f0824833700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f08200720f0 0x7f082015e8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:25.566 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:25.563+0000 7f0824833700 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f08200b40a0 con 0x7f08200720f0 2023-11-24T07:58:25.566 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:25.563+0000 7f0824833700 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0820072580 con 0x7f08201440f0 2023-11-24T07:58:25.567 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:25.563+0000 7f0824833700 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0820143ec0 con 0x7f0820143ae0 2023-11-24T07:58:25.567 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:25.563+0000 7f081ed9d700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f08200720f0 0x7f082015e8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:25.567 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:25.563+0000 7f081ed9d700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f08200720f0 0x7f082015e8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:53390/0 (socket says 172.21.15.59:53390) 2023-11-24T07:58:25.567 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:25.563+0000 7f081ed9d700 1 -- 172.21.15.59:0/3760090960 learned_addr learned my addr 172.21.15.59:0/3760090960 (peer_addr_for_me v2:172.21.15.59:0/0) 2023-11-24T07:58:25.567 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:25.563+0000 7f081e59c700 1 --2- 172.21.15.59:0/3760090960 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f0820143ae0 0x7f08200b51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:25.567 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:25.563+0000 7f081dd9b700 1 --2- 172.21.15.59:0/3760090960 >> v2:172.21.15.76:3300/0 conn(0x7f08201440f0 0x7f0820071bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:25.567 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:25.563+0000 7f081ed9d700 1 -- 172.21.15.59:0/3760090960 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f0820143ae0 msgr2=0x7f08200b51b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T07:58:25.567 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:25.563+0000 7f081ed9d700 1 --2- 172.21.15.59:0/3760090960 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f0820143ae0 0x7f08200b51b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:25.567 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:25.563+0000 7f081ed9d700 1 -- 172.21.15.59:0/3760090960 >> v2:172.21.15.76:3300/0 conn(0x7f08201440f0 msgr2=0x7f0820071bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T07:58:25.567 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:25.563+0000 7f081ed9d700 1 --2- 172.21.15.59:0/3760090960 >> v2:172.21.15.76:3300/0 conn(0x7f08201440f0 0x7f0820071bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:25.567 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:25.563+0000 7f081ed9d700 1 -- 172.21.15.59:0/3760090960 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f08200b3ae0 con 0x7f08200720f0 2023-11-24T07:58:25.568 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:25.563+0000 7f081e59c700 1 --2- 172.21.15.59:0/3760090960 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f0820143ae0 0x7f08200b51b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-24T07:58:25.568 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:25.563+0000 7f081ed9d700 1 --2- 172.21.15.59:0/3760090960 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f08200720f0 0x7f082015e8b0 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7f082006d070 tx=0x7f081800d430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3736b50c7476770c server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:58:25.568 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:25.564+0000 7f081d59a700 1 -- 172.21.15.59:0/3760090960 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0818010070 con 0x7f08200720f0 2023-11-24T07:58:25.568 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:25.564+0000 7f081d59a700 1 -- 172.21.15.59:0/3760090960 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f081800dce0 con 0x7f08200720f0 2023-11-24T07:58:25.568 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:25.564+0000 7f081d59a700 1 -- 172.21.15.59:0/3760090960 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f081800b310 con 0x7f08200720f0 2023-11-24T07:58:25.568 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:25.564+0000 7f0824833700 1 -- 172.21.15.59:0/3760090960 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f08200720f0 msgr2=0x7f082015e8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:25.568 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:25.564+0000 7f0824833700 1 --2- 172.21.15.59:0/3760090960 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f08200720f0 0x7f082015e8b0 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7f082006d070 tx=0x7f081800d430 comp rx=0 tx=0).stop 2023-11-24T07:58:25.568 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:25.564+0000 7f0824833700 1 -- 172.21.15.59:0/3760090960 shutdown_connections 2023-11-24T07:58:25.568 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:25.564+0000 7f0824833700 1 --2- 172.21.15.59:0/3760090960 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f08200720f0 0x7f082015e8b0 unknown :-1 s=CLOSED pgs=633 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:25.568 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:25.564+0000 7f0824833700 1 --2- 172.21.15.59:0/3760090960 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f0820143ae0 0x7f08200b51b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:25.569 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:25.564+0000 7f0824833700 1 --2- 172.21.15.59:0/3760090960 >> v2:172.21.15.76:3300/0 conn(0x7f08201440f0 0x7f0820071bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:25.569 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:25.564+0000 7f0824833700 1 -- 172.21.15.59:0/3760090960 >> 172.21.15.59:0/3760090960 conn(0x7f08200b2740 msgr2=0x7f08200b2b20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:58:25.569 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:25.564+0000 7f0824833700 1 -- 172.21.15.59:0/3760090960 shutdown_connections 2023-11-24T07:58:25.569 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:25.564+0000 7f0824833700 1 -- 172.21.15.59:0/3760090960 wait complete. 2023-11-24T07:58:25.572 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:25.566+0000 7f0824833700 1 Processor -- start 2023-11-24T07:58:25.572 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:25.566+0000 7f0824833700 1 -- start start 2023-11-24T07:58:25.572 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:25.566+0000 7f0824833700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f08200720f0 0x7f082015df70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:25.572 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:25.566+0000 7f0824833700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0820143ae0 0x7f08201555d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:25.572 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:25.566+0000 7f0824833700 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f08201440f0 0x7f08200cf320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:25.572 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:25.566+0000 7f0824833700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08200cee90 con 0x7f08200720f0 2023-11-24T07:58:25.573 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:25.566+0000 7f0824833700 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0820155b10 con 0x7f0820143ae0 2023-11-24T07:58:25.573 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:25.566+0000 7f0824833700 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0820156640 con 0x7f08201440f0 2023-11-24T07:58:25.573 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:25.566+0000 7f081e59c700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f08200720f0 0x7f082015df70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:25.573 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:25.566+0000 7f081e59c700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f08200720f0 0x7f082015df70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:53392/0 (socket says 172.21.15.59:53392) 2023-11-24T07:58:25.574 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:25.566+0000 7f081e59c700 1 -- 172.21.15.59:0/7784321 learned_addr learned my addr 172.21.15.59:0/7784321 (peer_addr_for_me v2:172.21.15.59:0/0) 2023-11-24T07:58:25.574 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:25.567+0000 7f081e59c700 1 -- 172.21.15.59:0/7784321 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f08201440f0 msgr2=0x7f08200cf320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:25.574 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:25.567+0000 7f081ed9d700 1 --2- 172.21.15.59:0/7784321 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f08201440f0 0x7f08200cf320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:25.574 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:25.567+0000 7f081dd9b700 1 --2- 172.21.15.59:0/7784321 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0820143ae0 0x7f08201555d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:25.574 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:25.567+0000 7f081e59c700 1 --2- 172.21.15.59:0/7784321 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f08201440f0 0x7f08200cf320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:25.574 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:25.567+0000 7f081e59c700 1 -- 172.21.15.59:0/7784321 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0820143ae0 msgr2=0x7f08201555d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:25.574 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:25.567+0000 7f081e59c700 1 --2- 172.21.15.59:0/7784321 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0820143ae0 0x7f08201555d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:25.574 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:25.567+0000 7f081e59c700 1 -- 172.21.15.59:0/7784321 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f081800bd40 con 0x7f08200720f0 2023-11-24T07:58:25.574 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:25.567+0000 7f081e59c700 1 --2- 172.21.15.59:0/7784321 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f08200720f0 0x7f082015df70 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7f081000e840 tx=0x7f081000ec00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:58:25.575 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:25.567+0000 7f080effd700 1 -- 172.21.15.59:0/7784321 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f081000c020 con 0x7f08200720f0 2023-11-24T07:58:25.575 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:25.567+0000 7f080effd700 1 -- 172.21.15.59:0/7784321 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0810004250 con 0x7f08200720f0 2023-11-24T07:58:25.575 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:25.567+0000 7f0824833700 1 -- 172.21.15.59:0/7784321 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0820156920 con 0x7f08200720f0 2023-11-24T07:58:25.575 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:25.567+0000 7f080effd700 1 -- 172.21.15.59:0/7784321 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08100036a0 con 0x7f08200720f0 2023-11-24T07:58:25.575 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:25.567+0000 7f0824833700 1 -- 172.21.15.59:0/7784321 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f08200cf860 con 0x7f08200720f0 2023-11-24T07:58:25.575 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:25.568+0000 7f0824833700 1 -- 172.21.15.59:0/7784321 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f08200cfa80 con 0x7f08200720f0 2023-11-24T07:58:25.575 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:25.568+0000 7f080effd700 1 -- 172.21.15.59:0/7784321 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 8) v1 ==== 89435+0+0 (secure 0 0 0) 0x7f0810003800 con 0x7f08200720f0 2023-11-24T07:58:25.575 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:25.569+0000 7f080effd700 1 --2- 172.21.15.59:0/7784321 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f080006b110 0x7f080006d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:25.575 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:25.569+0000 7f080effd700 1 -- 172.21.15.59:0/7784321 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 5910+0+0 (secure 0 0 0) 0x7f0810089930 con 0x7f08200720f0 2023-11-24T07:58:25.575 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:25.569+0000 7f081dd9b700 1 --2- 172.21.15.59:0/7784321 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f080006b110 0x7f080006d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:25.575 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:25.570+0000 7f081dd9b700 1 --2- 172.21.15.59:0/7784321 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f080006b110 0x7f080006d5d0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f082015e4b0 tx=0x7f0814008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:58:25.576 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:25.574+0000 7f080effd700 1 -- 172.21.15.59:0/7784321 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f0810054530 con 0x7f08200720f0 2023-11-24T07:58:25.821 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:25.819+0000 7f0824833700 1 -- 172.21.15.59:0/7784321 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f0820068670 con 0x7f08200720f0 2023-11-24T07:58:25.821 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:25.820+0000 7f080effd700 1 -- 172.21.15.59:0/7784321 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) v1 ==== 94+0+4904 (secure 0 0 0) 0x7f0810057b50 con 0x7f08200720f0 2023-11-24T07:58:25.822 INFO:teuthology.orchestra.run.smithi059.stdout: 2023-11-24T07:58:25.822 INFO:teuthology.orchestra.run.smithi059.stdout:{"epoch":26,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5290,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.76:6836/508095662","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6834","nonce":508095662},{"type":"v1","addr":"172.21.15.76:6836","nonce":508095662}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":20},{"gid":5293,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.76:6837/4158118811","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6835","nonce":4158118811},{"type":"v1","addr":"172.21.15.76:6837","nonce":4158118811}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":20},{"gid":5622,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6834/59936286","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6832","nonce":59936286},{"type":"v1","addr":"172.21.15.59:6834","nonce":59936286}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":20}],"filesystems":[{"mdsmap":{"epoch":25,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-24T07:58:02.020496+0000","modified":"2023-11-24T07:58:05.037742+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5625},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5625":{"gid":5625,"name":"c","rank":0,"incarnation":21,"state":"up:active","state_seq":3,"addr":"172.21.15.59:6835/1468266626","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6833","nonce":1468266626},{"type":"v1","addr":"172.21.15.59:6835","nonce":1468266626}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2023-11-24T07:58:25.823 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:25.821+0000 7f0824833700 1 -- 172.21.15.59:0/7784321 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f080006b110 msgr2=0x7f080006d5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:25.824 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:25.821+0000 7f0824833700 1 --2- 172.21.15.59:0/7784321 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f080006b110 0x7f080006d5d0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f082015e4b0 tx=0x7f0814008040 comp rx=0 tx=0).stop 2023-11-24T07:58:25.824 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:25.821+0000 7f0824833700 1 -- 172.21.15.59:0/7784321 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f08200720f0 msgr2=0x7f082015df70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:25.824 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:25.821+0000 7f0824833700 1 --2- 172.21.15.59:0/7784321 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f08200720f0 0x7f082015df70 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7f081000e840 tx=0x7f081000ec00 comp rx=0 tx=0).stop 2023-11-24T07:58:25.824 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:25.821+0000 7f0824833700 1 -- 172.21.15.59:0/7784321 shutdown_connections 2023-11-24T07:58:25.825 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:25.821+0000 7f0824833700 1 --2- 172.21.15.59:0/7784321 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f080006b110 0x7f080006d5d0 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:25.825 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:25.821+0000 7f0824833700 1 --2- 172.21.15.59:0/7784321 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f08201440f0 0x7f08200cf320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:25.825 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:25.822+0000 7f0824833700 1 --2- 172.21.15.59:0/7784321 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f08200720f0 0x7f082015df70 unknown :-1 s=CLOSED pgs=634 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:25.825 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:25.822+0000 7f0824833700 1 --2- 172.21.15.59:0/7784321 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0820143ae0 0x7f08201555d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:25.825 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:25.822+0000 7f0824833700 1 -- 172.21.15.59:0/7784321 >> 172.21.15.59:0/7784321 conn(0x7f08200b2740 msgr2=0x7f082015e840 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:58:25.826 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:25.822+0000 7f0824833700 1 -- 172.21.15.59:0/7784321 shutdown_connections 2023-11-24T07:58:25.826 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:25.822+0000 7f0824833700 1 -- 172.21.15.59:0/7784321 wait complete. 2023-11-24T07:58:25.826 INFO:teuthology.orchestra.run.smithi059.stderr:dumped fsmap epoch 26 2023-11-24T07:58:25.835 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-24T07:58:25.959 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:25.958+0000 7f03787a1700 1 Processor -- start 2023-11-24T07:58:25.960 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:25.958+0000 7f03787a1700 1 -- start start 2023-11-24T07:58:25.962 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:25.959+0000 7f03787a1700 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f037015cf10 0x7f037006f490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:25.962 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:25.959+0000 7f03787a1700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f037006afd0 0x7f037006b400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:25.962 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:25.959+0000 7f03787a1700 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f03700cd2a0 con 0x7f0370145740 2023-11-24T07:58:25.962 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:25.959+0000 7f03787a1700 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f03700cdab0 con 0x7f037006afd0 2023-11-24T07:58:25.963 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:25.959+0000 7f03787a1700 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f037006f9d0 con 0x7f037015cf10 2023-11-24T07:58:25.963 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:25.959+0000 7f0376d3e700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f037006afd0 0x7f037006b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:25.963 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:25.959+0000 7f0376d3e700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f037006afd0 0x7f037006b400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:53404/0 (socket says 172.21.15.59:53404) 2023-11-24T07:58:25.963 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:25.959+0000 7f0376d3e700 1 -- 172.21.15.59:0/2967632563 learned_addr learned my addr 172.21.15.59:0/2967632563 (peer_addr_for_me v2:172.21.15.59:0/0) 2023-11-24T07:58:25.963 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:25.959+0000 7f0375d3c700 1 --2- 172.21.15.59:0/2967632563 >> v2:172.21.15.76:3300/0 conn(0x7f037015cf10 0x7f037006f490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:25.963 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:25.959+0000 7f0376d3e700 1 -- 172.21.15.59:0/2967632563 >> v2:172.21.15.76:3300/0 conn(0x7f037015cf10 msgr2=0x7f037006f490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T07:58:25.963 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:25.959+0000 7f0376d3e700 1 --2- 172.21.15.59:0/2967632563 >> v2:172.21.15.76:3300/0 conn(0x7f037015cf10 0x7f037006f490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:25.963 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:25.959+0000 7f0376d3e700 1 -- 172.21.15.59:0/2967632563 >> v1:172.21.15.76:6789/0 conn(0x7f0370145740 legacy=0x7f03700d10c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:25.963 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:25.959+0000 7f0376d3e700 1 -- 172.21.15.59:0/2967632563 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f037006b940 con 0x7f037006afd0 2023-11-24T07:58:25.963 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:25.959+0000 7f0376d3e700 1 --2- 172.21.15.59:0/2967632563 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f037006afd0 0x7f037006b400 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7f03700ced10 tx=0x7f036800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bfdd98f7db57f1af server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:58:25.964 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:25.960+0000 7f037553b700 1 -- 172.21.15.59:0/2967632563 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f036800e070 con 0x7f037006afd0 2023-11-24T07:58:25.964 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:25.960+0000 7f037553b700 1 -- 172.21.15.59:0/2967632563 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f036800b910 con 0x7f037006afd0 2023-11-24T07:58:25.964 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:25.960+0000 7f037553b700 1 -- 172.21.15.59:0/2967632563 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f036800bcc0 con 0x7f037006afd0 2023-11-24T07:58:25.964 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:25.960+0000 7f03787a1700 1 -- 172.21.15.59:0/2967632563 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f037006afd0 msgr2=0x7f037006b400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:25.964 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:25.960+0000 7f03787a1700 1 --2- 172.21.15.59:0/2967632563 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f037006afd0 0x7f037006b400 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7f03700ced10 tx=0x7f036800b330 comp rx=0 tx=0).stop 2023-11-24T07:58:25.964 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:25.960+0000 7f03787a1700 1 -- 172.21.15.59:0/2967632563 shutdown_connections 2023-11-24T07:58:25.964 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:25.960+0000 7f03787a1700 1 --2- 172.21.15.59:0/2967632563 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f037006afd0 0x7f037006b400 unknown :-1 s=CLOSED pgs=635 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:25.964 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:25.960+0000 7f03787a1700 1 --2- 172.21.15.59:0/2967632563 >> v2:172.21.15.76:3300/0 conn(0x7f037015cf10 0x7f037006f490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:25.964 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:25.960+0000 7f03787a1700 1 -- 172.21.15.59:0/2967632563 >> 172.21.15.59:0/2967632563 conn(0x7f037006c0e0 msgr2=0x7f037006c4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:58:25.964 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:25.960+0000 7f03787a1700 1 -- 172.21.15.59:0/2967632563 shutdown_connections 2023-11-24T07:58:25.964 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:25.960+0000 7f03787a1700 1 -- 172.21.15.59:0/2967632563 wait complete. 2023-11-24T07:58:25.967 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:25.961+0000 7f03787a1700 1 Processor -- start 2023-11-24T07:58:25.967 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:25.961+0000 7f03787a1700 1 -- start start 2023-11-24T07:58:25.967 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:25.962+0000 7f03787a1700 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f037006afd0 0x7f03700af840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:25.967 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:25.962+0000 7f03787a1700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0370145740 0x7f0370143d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:25.967 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:25.962+0000 7f03787a1700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f037015cf10 0x7f03701442c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:25.967 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:25.962+0000 7f03787a1700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03700b3b40 con 0x7f0370145740 2023-11-24T07:58:25.968 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:25.962+0000 7f03787a1700 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03700b01b0 con 0x7f037015cf10 2023-11-24T07:58:25.968 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:25.962+0000 7f03787a1700 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f03700b3d90 con 0x7f037006afd0 2023-11-24T07:58:25.968 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:25.962+0000 7f0375d3c700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0370145740 0x7f0370143d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:25.968 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:25.962+0000 7f0375d3c700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0370145740 0x7f0370143d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:53408/0 (socket says 172.21.15.59:53408) 2023-11-24T07:58:25.968 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:25.962+0000 7f0375d3c700 1 -- 172.21.15.59:0/1310885649 learned_addr learned my addr 172.21.15.59:0/1310885649 (peer_addr_for_me v2:172.21.15.59:0/0) 2023-11-24T07:58:25.968 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:25.962+0000 7f037653d700 1 --2- 172.21.15.59:0/1310885649 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f037006afd0 0x7f03700af840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:25.968 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:25.962+0000 7f0376d3e700 1 --2- 172.21.15.59:0/1310885649 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f037015cf10 0x7f03701442c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:25.968 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:25.963+0000 7f0375d3c700 1 -- 172.21.15.59:0/1310885649 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f037006afd0 msgr2=0x7f03700af840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:25.968 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:25.963+0000 7f0375d3c700 1 --2- 172.21.15.59:0/1310885649 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f037006afd0 0x7f03700af840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:25.968 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:25.963+0000 7f0375d3c700 1 -- 172.21.15.59:0/1310885649 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f037015cf10 msgr2=0x7f03701442c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:25.968 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:25.963+0000 7f0375d3c700 1 --2- 172.21.15.59:0/1310885649 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f037015cf10 0x7f03701442c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:25.969 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:25.963+0000 7f0375d3c700 1 -- 172.21.15.59:0/1310885649 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0368009b30 con 0x7f0370145740 2023-11-24T07:58:25.969 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:25.963+0000 7f0376d3e700 1 --2- 172.21.15.59:0/1310885649 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f037015cf10 0x7f03701442c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-24T07:58:25.969 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:25.963+0000 7f0375d3c700 1 --2- 172.21.15.59:0/1310885649 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0370145740 0x7f0370143d80 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7f036000e870 tx=0x7f036000eb80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:58:25.969 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:25.963+0000 7f0366ffd700 1 -- 172.21.15.59:0/1310885649 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0360009590 con 0x7f0370145740 2023-11-24T07:58:25.969 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:25.963+0000 7f0366ffd700 1 -- 172.21.15.59:0/1310885649 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0360013680 con 0x7f0370145740 2023-11-24T07:58:25.969 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:25.963+0000 7f03787a1700 1 -- 172.21.15.59:0/1310885649 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f03700b4070 con 0x7f0370145740 2023-11-24T07:58:25.969 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:25.963+0000 7f0366ffd700 1 -- 172.21.15.59:0/1310885649 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03600139f0 con 0x7f0370145740 2023-11-24T07:58:25.969 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:25.963+0000 7f03787a1700 1 -- 172.21.15.59:0/1310885649 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f03700b4870 con 0x7f0370145740 2023-11-24T07:58:25.969 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:25.964+0000 7f03787a1700 1 -- 172.21.15.59:0/1310885649 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f03700cdcc0 con 0x7f0370145740 2023-11-24T07:58:25.969 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:25.964+0000 7f0366ffd700 1 -- 172.21.15.59:0/1310885649 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 8) v1 ==== 89435+0+0 (secure 0 0 0) 0x7f0360013b50 con 0x7f0370145740 2023-11-24T07:58:25.969 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:25.965+0000 7f0366ffd700 1 --2- 172.21.15.59:0/1310885649 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f035406b110 0x7f035406d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:25.969 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:25.965+0000 7f0366ffd700 1 -- 172.21.15.59:0/1310885649 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 5910+0+0 (secure 0 0 0) 0x7f0360089980 con 0x7f0370145740 2023-11-24T07:58:25.969 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:25.965+0000 7f037653d700 1 --2- 172.21.15.59:0/1310885649 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f035406b110 0x7f035406d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:25.970 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:25.966+0000 7f037653d700 1 --2- 172.21.15.59:0/1310885649 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f035406b110 0x7f035406d5d0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f036c007fd0 tx=0x7f036c009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:58:25.971 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:25.970+0000 7f0366ffd700 1 -- 172.21.15.59:0/1310885649 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f0360054610 con 0x7f0370145740 2023-11-24T07:58:26.218 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:26.216+0000 7f03787a1700 1 -- 172.21.15.59:0/1310885649 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f03700b3750 con 0x7f0370145740 2023-11-24T07:58:26.219 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:26.217+0000 7f0366ffd700 1 -- 172.21.15.59:0/1310885649 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) v1 ==== 94+0+4904 (secure 0 0 0) 0x7f0360057c30 con 0x7f0370145740 2023-11-24T07:58:26.219 INFO:teuthology.orchestra.run.smithi059.stdout: 2023-11-24T07:58:26.219 INFO:teuthology.orchestra.run.smithi059.stdout:{"epoch":26,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5290,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.76:6836/508095662","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6834","nonce":508095662},{"type":"v1","addr":"172.21.15.76:6836","nonce":508095662}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":20},{"gid":5293,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.76:6837/4158118811","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6835","nonce":4158118811},{"type":"v1","addr":"172.21.15.76:6837","nonce":4158118811}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":20},{"gid":5622,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6834/59936286","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6832","nonce":59936286},{"type":"v1","addr":"172.21.15.59:6834","nonce":59936286}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":20}],"filesystems":[{"mdsmap":{"epoch":25,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-24T07:58:02.020496+0000","modified":"2023-11-24T07:58:05.037742+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5625},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5625":{"gid":5625,"name":"c","rank":0,"incarnation":21,"state":"up:active","state_seq":3,"addr":"172.21.15.59:6835/1468266626","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6833","nonce":1468266626},{"type":"v1","addr":"172.21.15.59:6835","nonce":1468266626}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2023-11-24T07:58:26.221 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:26.219+0000 7f03787a1700 1 -- 172.21.15.59:0/1310885649 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f035406b110 msgr2=0x7f035406d5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:26.221 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:26.219+0000 7f03787a1700 1 --2- 172.21.15.59:0/1310885649 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f035406b110 0x7f035406d5d0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f036c007fd0 tx=0x7f036c009040 comp rx=0 tx=0).stop 2023-11-24T07:58:26.221 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:26.219+0000 7f03787a1700 1 -- 172.21.15.59:0/1310885649 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0370145740 msgr2=0x7f0370143d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:26.221 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:26.219+0000 7f03787a1700 1 --2- 172.21.15.59:0/1310885649 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0370145740 0x7f0370143d80 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7f036000e870 tx=0x7f036000eb80 comp rx=0 tx=0).stop 2023-11-24T07:58:26.222 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:26.219+0000 7f03787a1700 1 -- 172.21.15.59:0/1310885649 shutdown_connections 2023-11-24T07:58:26.222 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:26.219+0000 7f03787a1700 1 --2- 172.21.15.59:0/1310885649 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f035406b110 0x7f035406d5d0 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:26.222 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:26.219+0000 7f03787a1700 1 --2- 172.21.15.59:0/1310885649 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f037015cf10 0x7f03701442c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:26.222 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:26.219+0000 7f03787a1700 1 --2- 172.21.15.59:0/1310885649 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f037006afd0 0x7f03700af840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:26.222 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:26.219+0000 7f03787a1700 1 --2- 172.21.15.59:0/1310885649 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0370145740 0x7f0370143d80 unknown :-1 s=CLOSED pgs=636 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:26.222 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:26.219+0000 7f03787a1700 1 -- 172.21.15.59:0/1310885649 >> 172.21.15.59:0/1310885649 conn(0x7f037006c0e0 msgr2=0x7f03700cef00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:58:26.222 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:26.219+0000 7f03787a1700 1 -- 172.21.15.59:0/1310885649 shutdown_connections 2023-11-24T07:58:26.222 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:26.219+0000 7f03787a1700 1 -- 172.21.15.59:0/1310885649 wait complete. 2023-11-24T07:58:26.222 INFO:teuthology.orchestra.run.smithi059.stderr:dumped fsmap epoch 26 2023-11-24T07:58:26.234 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-24T07:58:26.361 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:26.359+0000 7f2664c73700 1 Processor -- start 2023-11-24T07:58:26.361 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:26.359+0000 7f2664c73700 1 -- start start 2023-11-24T07:58:26.364 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:26.360+0000 7f2664c73700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f266015ae00 0x7f26600b5e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:26.364 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:26.360+0000 7f2664c73700 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f26600adda0 0x7f266006b970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:26.364 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:26.360+0000 7f2664c73700 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f26600b0ec0 con 0x7f26600b03b0 2023-11-24T07:58:26.364 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:26.360+0000 7f2664c73700 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2660073610 con 0x7f266015ae00 2023-11-24T07:58:26.364 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:26.360+0000 7f2664c73700 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f266015b1e0 con 0x7f26600adda0 2023-11-24T07:58:26.364 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:26.361+0000 7f265dd9b700 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7f26600b03b0 0x7f26600ad690 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:56888/0 (socket says 172.21.15.59:56888) 2023-11-24T07:58:26.365 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:26.361+0000 7f265e59c700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f266015ae00 0x7f26600b5e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:26.365 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:26.361+0000 7f265dd9b700 1 -- 172.21.15.59:0/1989467571 learned_addr learned my addr 172.21.15.59:0/1989467571 (peer_addr_for_me v1:172.21.15.59:0/0) 2023-11-24T07:58:26.365 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:26.361+0000 7f265ed9d700 1 --2- 172.21.15.59:0/1989467571 >> v2:172.21.15.76:3300/0 conn(0x7f26600adda0 0x7f266006b970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:26.365 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:26.361+0000 7f265e59c700 1 -- 172.21.15.59:0/1989467571 >> v2:172.21.15.76:3300/0 conn(0x7f26600adda0 msgr2=0x7f266006b970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T07:58:26.365 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:26.361+0000 7f265e59c700 1 --2- 172.21.15.59:0/1989467571 >> v2:172.21.15.76:3300/0 conn(0x7f26600adda0 0x7f266006b970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:26.365 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:26.361+0000 7f265e59c700 1 -- 172.21.15.59:0/1989467571 >> v1:172.21.15.59:6789/0 conn(0x7f26600b03b0 legacy=0x7f26600ad690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:26.365 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:26.361+0000 7f265e59c700 1 -- 172.21.15.59:0/1989467571 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f26600b0900 con 0x7f266015ae00 2023-11-24T07:58:26.366 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:26.361+0000 7f265d59a700 1 -- 172.21.15.59:0/1989467571 <== mon.0 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 918439614 0 0) 0x7f26600b0ec0 con 0x7f26600b03b0 2023-11-24T07:58:26.366 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:26.361+0000 7f265ed9d700 1 --2- 172.21.15.59:0/1989467571 >> v2:172.21.15.76:3300/0 conn(0x7f26600adda0 0x7f266006b970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-24T07:58:26.366 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:26.361+0000 7f265e59c700 1 --2- 172.21.15.59:0/1989467571 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f266015ae00 0x7f26600b5e40 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7f26600706d0 tx=0x7f2650009b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=55cf603dec32ea8a server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:58:26.366 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:26.361+0000 7f265d59a700 1 -- 172.21.15.59:0/1989467571 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2650011830 con 0x7f266015ae00 2023-11-24T07:58:26.366 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:26.361+0000 7f265d59a700 1 -- 172.21.15.59:0/1989467571 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2650011990 con 0x7f266015ae00 2023-11-24T07:58:26.366 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:26.361+0000 7f265d59a700 1 -- 172.21.15.59:0/1989467571 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2650011d20 con 0x7f266015ae00 2023-11-24T07:58:26.366 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:26.362+0000 7f2664c73700 1 -- 172.21.15.59:0/1989467571 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f266015ae00 msgr2=0x7f26600b5e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:26.367 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:26.362+0000 7f2664c73700 1 --2- 172.21.15.59:0/1989467571 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f266015ae00 0x7f26600b5e40 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7f26600706d0 tx=0x7f2650009b20 comp rx=0 tx=0).stop 2023-11-24T07:58:26.367 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:26.362+0000 7f2664c73700 1 -- 172.21.15.59:0/1989467571 shutdown_connections 2023-11-24T07:58:26.367 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:26.362+0000 7f2664c73700 1 --2- 172.21.15.59:0/1989467571 >> v2:172.21.15.76:3300/0 conn(0x7f26600adda0 0x7f266006b970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:26.367 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:26.362+0000 7f2664c73700 1 --2- 172.21.15.59:0/1989467571 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f266015ae00 0x7f26600b5e40 unknown :-1 s=CLOSED pgs=638 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:26.367 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:26.362+0000 7f2664c73700 1 -- 172.21.15.59:0/1989467571 >> 172.21.15.59:0/1989467571 conn(0x7f26600b5070 msgr2=0x7f26600b5730 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:58:26.367 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:26.362+0000 7f2664c73700 1 -- 172.21.15.59:0/1989467571 shutdown_connections 2023-11-24T07:58:26.367 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:26.362+0000 7f2664c73700 1 -- 172.21.15.59:0/1989467571 wait complete. 2023-11-24T07:58:26.370 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:26.363+0000 7f2664c73700 1 Processor -- start 2023-11-24T07:58:26.370 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:26.363+0000 7f2664c73700 1 -- start start 2023-11-24T07:58:26.370 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:26.364+0000 7f2664c73700 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f26600adda0 0x7f2660151dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:26.370 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:26.364+0000 7f2664c73700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f26600b03b0 0x7f26600cecb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:26.370 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:26.364+0000 7f2664c73700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f266015ae00 0x7f26601575a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:26.370 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:26.364+0000 7f2664c73700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2660154ab0 con 0x7f266015ae00 2023-11-24T07:58:26.370 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:26.364+0000 7f2664c73700 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f266015c8a0 con 0x7f26600b03b0 2023-11-24T07:58:26.371 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:26.364+0000 7f2664c73700 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f266015cb80 con 0x7f26600adda0 2023-11-24T07:58:26.371 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:26.364+0000 7f265ed9d700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f266015ae00 0x7f26601575a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:26.371 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:26.364+0000 7f265ed9d700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f266015ae00 0x7f26601575a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:53422/0 (socket says 172.21.15.59:53422) 2023-11-24T07:58:26.371 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:26.364+0000 7f265ed9d700 1 -- 172.21.15.59:0/1391451960 learned_addr learned my addr 172.21.15.59:0/1391451960 (peer_addr_for_me v2:172.21.15.59:0/0) 2023-11-24T07:58:26.371 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:26.365+0000 7f265dd9b700 1 --2- 172.21.15.59:0/1391451960 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f26600b03b0 0x7f26600cecb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:26.371 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:26.365+0000 7f265ed9d700 1 -- 172.21.15.59:0/1391451960 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f26600adda0 msgr2=0x7f2660151dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:26.371 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:26.365+0000 7f265e59c700 1 --2- 172.21.15.59:0/1391451960 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f26600adda0 0x7f2660151dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:26.371 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:26.365+0000 7f265ed9d700 1 --2- 172.21.15.59:0/1391451960 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f26600adda0 0x7f2660151dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:26.371 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:26.365+0000 7f265ed9d700 1 -- 172.21.15.59:0/1391451960 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f26600b03b0 msgr2=0x7f26600cecb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:26.371 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:26.365+0000 7f265ed9d700 1 --2- 172.21.15.59:0/1391451960 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f26600b03b0 0x7f26600cecb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:26.371 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:26.365+0000 7f265ed9d700 1 -- 172.21.15.59:0/1391451960 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f26500094a0 con 0x7f266015ae00 2023-11-24T07:58:26.371 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:26.365+0000 7f265dd9b700 1 --2- 172.21.15.59:0/1391451960 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f26600b03b0 0x7f26600cecb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T07:58:26.372 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:26.365+0000 7f265e59c700 1 --2- 172.21.15.59:0/1391451960 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f26600adda0 0x7f2660151dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T07:58:26.372 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:26.365+0000 7f265ed9d700 1 --2- 172.21.15.59:0/1391451960 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f266015ae00 0x7f26601575a0 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7f2658007e70 tx=0x7f265800d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:58:26.372 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:26.365+0000 7f2656ffd700 1 -- 172.21.15.59:0/1391451960 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f265800de40 con 0x7f266015ae00 2023-11-24T07:58:26.372 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:26.365+0000 7f2656ffd700 1 -- 172.21.15.59:0/1391451960 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f265800f040 con 0x7f266015ae00 2023-11-24T07:58:26.372 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:26.365+0000 7f2664c73700 1 -- 172.21.15.59:0/1391451960 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2660154740 con 0x7f266015ae00 2023-11-24T07:58:26.372 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:26.365+0000 7f2656ffd700 1 -- 172.21.15.59:0/1391451960 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2658003c60 con 0x7f266015ae00 2023-11-24T07:58:26.372 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:26.365+0000 7f2664c73700 1 -- 172.21.15.59:0/1391451960 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2660157ae0 con 0x7f266015ae00 2023-11-24T07:58:26.372 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:26.366+0000 7f2664c73700 1 -- 172.21.15.59:0/1391451960 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f26601545a0 con 0x7f266015ae00 2023-11-24T07:58:26.372 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:26.366+0000 7f2656ffd700 1 -- 172.21.15.59:0/1391451960 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 8) v1 ==== 89435+0+0 (secure 0 0 0) 0x7f2658004030 con 0x7f266015ae00 2023-11-24T07:58:26.372 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:26.367+0000 7f2656ffd700 1 --2- 172.21.15.59:0/1391451960 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f264006b110 0x7f264006d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:26.373 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:26.367+0000 7f2656ffd700 1 -- 172.21.15.59:0/1391451960 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 5882+0+0 (secure 0 0 0) 0x7f2658089cf0 con 0x7f266015ae00 2023-11-24T07:58:26.373 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:26.367+0000 7f265e59c700 1 --2- 172.21.15.59:0/1391451960 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f264006b110 0x7f264006d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:26.373 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:26.367+0000 7f265e59c700 1 --2- 172.21.15.59:0/1391451960 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f264006b110 0x7f264006d5d0 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f26500090b0 tx=0x7f2650008030 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:58:26.373 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:26.371+0000 7f2656ffd700 1 -- 172.21.15.59:0/1391451960 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f26601545a0 con 0x7f266015ae00 2023-11-24T07:58:26.578 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:26.577+0000 7f2664c73700 1 -- 172.21.15.59:0/1391451960 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f26601545a0 con 0x7f266015ae00 2023-11-24T07:58:26.579 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:26.578+0000 7f2656ffd700 1 -- 172.21.15.59:0/1391451960 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v49) v1 ==== 74+0+16733 (secure 0 0 0) 0x7f26601545a0 con 0x7f266015ae00 2023-11-24T07:58:26.580 INFO:teuthology.orchestra.run.smithi059.stdout: 2023-11-24T07:58:26.581 INFO:teuthology.orchestra.run.smithi059.stdout:{"epoch":49,"fsid":"9f0e8741-b8f9-48d5-a2c3-09f45903303d","created":"2023-11-24T07:56:06.552729+0000","modified":"2023-11-24T07:58:26.324367+0000","last_up_change":"2023-11-24T07:56:17.372352+0000","last_in_change":"2023-11-24T07:56:09.073250+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-24T07:56:18.352823+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_metadata","create_time":"2023-11-24T07:58:00.240130+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"44","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_data","create_time":"2023-11-24T07:58:00.977486+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"48","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":3,"snap_epoch":48,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"6d8f9e9b-1782-4e41-928b-a39f2a8ebc06","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6800","nonce":3373878772},{"type":"v1","addr":"172.21.15.59:6801","nonce":3373878772}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6802","nonce":3373878772},{"type":"v1","addr":"172.21.15.59:6803","nonce":3373878772}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6806","nonce":3373878772},{"type":"v1","addr":"172.21.15.59:6807","nonce":3373878772}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6804","nonce":3373878772},{"type":"v1","addr":"172.21.15.59:6805","nonce":3373878772}]},"public_addr":"172.21.15.59:6801/3373878772","cluster_addr":"172.21.15.59:6803/3373878772","heartbeat_back_addr":"172.21.15.59:6807/3373878772","heartbeat_front_addr":"172.21.15.59:6805/3373878772","state":["exists","up"]},{"osd":1,"uuid":"e8d882aa-52e1-4c04-9e1e-c44ef9204a5f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6824","nonce":648258563},{"type":"v1","addr":"172.21.15.59:6825","nonce":648258563}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6826","nonce":648258563},{"type":"v1","addr":"172.21.15.59:6827","nonce":648258563}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6830","nonce":648258563},{"type":"v1","addr":"172.21.15.59:6831","nonce":648258563}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6828","nonce":648258563},{"type":"v1","addr":"172.21.15.59:6829","nonce":648258563}]},"public_addr":"172.21.15.59:6825/648258563","cluster_addr":"172.21.15.59:6827/648258563","heartbeat_back_addr":"172.21.15.59:6831/648258563","heartbeat_front_addr":"172.21.15.59:6829/648258563","state":["exists","up"]},{"osd":2,"uuid":"69eebfa0-695b-4b21-90b9-8a3537168a01","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6808","nonce":1840430654},{"type":"v1","addr":"172.21.15.59:6809","nonce":1840430654}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6810","nonce":1840430654},{"type":"v1","addr":"172.21.15.59:6811","nonce":1840430654}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6814","nonce":1840430654},{"type":"v1","addr":"172.21.15.59:6815","nonce":1840430654}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6812","nonce":1840430654},{"type":"v1","addr":"172.21.15.59:6813","nonce":1840430654}]},"public_addr":"172.21.15.59:6809/1840430654","cluster_addr":"172.21.15.59:6811/1840430654","heartbeat_back_addr":"172.21.15.59:6815/1840430654","heartbeat_front_addr":"172.21.15.59:6813/1840430654","state":["exists","up"]},{"osd":3,"uuid":"96fd77fc-f911-46ff-9445-6f0be2daa8f4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6816","nonce":2504025850},{"type":"v1","addr":"172.21.15.59:6817","nonce":2504025850}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6818","nonce":2504025850},{"type":"v1","addr":"172.21.15.59:6819","nonce":2504025850}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6822","nonce":2504025850},{"type":"v1","addr":"172.21.15.59:6823","nonce":2504025850}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6820","nonce":2504025850},{"type":"v1","addr":"172.21.15.59:6821","nonce":2504025850}]},"public_addr":"172.21.15.59:6817/2504025850","cluster_addr":"172.21.15.59:6819/2504025850","heartbeat_back_addr":"172.21.15.59:6823/2504025850","heartbeat_front_addr":"172.21.15.59:6821/2504025850","state":["exists","up"]},{"osd":4,"uuid":"7ac29fca-b5b2-4c95-9d69-2f164cbbc708","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6816","nonce":2382463431},{"type":"v1","addr":"172.21.15.76:6818","nonce":2382463431}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6820","nonce":2382463431},{"type":"v1","addr":"172.21.15.76:6822","nonce":2382463431}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6828","nonce":2382463431},{"type":"v1","addr":"172.21.15.76:6830","nonce":2382463431}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6824","nonce":2382463431},{"type":"v1","addr":"172.21.15.76:6826","nonce":2382463431}]},"public_addr":"172.21.15.76:6818/2382463431","cluster_addr":"172.21.15.76:6822/2382463431","heartbeat_back_addr":"172.21.15.76:6830/2382463431","heartbeat_front_addr":"172.21.15.76:6826/2382463431","state":["exists","up"]},{"osd":5,"uuid":"485bf711-3851-4238-a4f4-51d52e0a9c58","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6800","nonce":329334308},{"type":"v1","addr":"172.21.15.76:6802","nonce":329334308}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6805","nonce":329334308},{"type":"v1","addr":"172.21.15.76:6807","nonce":329334308}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6814","nonce":329334308},{"type":"v1","addr":"172.21.15.76:6815","nonce":329334308}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6810","nonce":329334308},{"type":"v1","addr":"172.21.15.76:6812","nonce":329334308}]},"public_addr":"172.21.15.76:6802/329334308","cluster_addr":"172.21.15.76:6807/329334308","heartbeat_back_addr":"172.21.15.76:6815/329334308","heartbeat_front_addr":"172.21.15.76:6812/329334308","state":["exists","up"]},{"osd":6,"uuid":"72b3f2fc-1e4a-4e32-968d-003647a0286f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6801","nonce":3946579410},{"type":"v1","addr":"172.21.15.76:6803","nonce":3946579410}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6804","nonce":3946579410},{"type":"v1","addr":"172.21.15.76:6806","nonce":3946579410}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6811","nonce":3946579410},{"type":"v1","addr":"172.21.15.76:6813","nonce":3946579410}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6808","nonce":3946579410},{"type":"v1","addr":"172.21.15.76:6809","nonce":3946579410}]},"public_addr":"172.21.15.76:6803/3946579410","cluster_addr":"172.21.15.76:6806/3946579410","heartbeat_back_addr":"172.21.15.76:6813/3946579410","heartbeat_front_addr":"172.21.15.76:6809/3946579410","state":["exists","up"]},{"osd":7,"uuid":"1ddceb90-6c5d-4d1a-ad56-99cb877f8c70","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6817","nonce":841761538},{"type":"v1","addr":"172.21.15.76:6819","nonce":841761538}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6821","nonce":841761538},{"type":"v1","addr":"172.21.15.76:6823","nonce":841761538}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6829","nonce":841761538},{"type":"v1","addr":"172.21.15.76:6831","nonce":841761538}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6825","nonce":841761538},{"type":"v1","addr":"172.21.15.76:6827","nonce":841761538}]},"public_addr":"172.21.15.76:6819/841761538","cluster_addr":"172.21.15.76:6823/841761538","heartbeat_back_addr":"172.21.15.76:6831/841761538","heartbeat_front_addr":"172.21.15.76:6827/841761538","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T07:56:14.294838+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T07:56:14.270779+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T07:56:14.231809+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T07:56:14.260983+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T07:56:14.302997+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T07:56:14.258600+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T07:56:14.251413+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T07:56:14.304727+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"7.0","mappings":[{"from":5,"to":7}]},{"pgid":"7.c","mappings":[{"from":5,"to":7}]},{"pgid":"7.12","mappings":[{"from":5,"to":7}]},{"pgid":"7.14","mappings":[{"from":5,"to":7}]},{"pgid":"7.38","mappings":[{"from":5,"to":1}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[{"pool":7,"snaps":[{"begin":2,"length":2}]}],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-24T07:58:26.583 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:26.580+0000 7f2664c73700 1 -- 172.21.15.59:0/1391451960 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f264006b110 msgr2=0x7f264006d5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:26.583 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:26.580+0000 7f2664c73700 1 --2- 172.21.15.59:0/1391451960 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f264006b110 0x7f264006d5d0 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f26500090b0 tx=0x7f2650008030 comp rx=0 tx=0).stop 2023-11-24T07:58:26.583 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:26.580+0000 7f2664c73700 1 -- 172.21.15.59:0/1391451960 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f266015ae00 msgr2=0x7f26601575a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:26.584 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:26.580+0000 7f2664c73700 1 --2- 172.21.15.59:0/1391451960 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f266015ae00 0x7f26601575a0 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7f2658007e70 tx=0x7f265800d3b0 comp rx=0 tx=0).stop 2023-11-24T07:58:26.584 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:26.580+0000 7f2664c73700 1 -- 172.21.15.59:0/1391451960 shutdown_connections 2023-11-24T07:58:26.584 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:26.580+0000 7f2664c73700 1 --2- 172.21.15.59:0/1391451960 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f264006b110 0x7f264006d5d0 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:26.584 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:26.580+0000 7f2664c73700 1 --2- 172.21.15.59:0/1391451960 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f266015ae00 0x7f26601575a0 unknown :-1 s=CLOSED pgs=639 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:26.584 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:26.580+0000 7f2664c73700 1 --2- 172.21.15.59:0/1391451960 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f26600adda0 0x7f2660151dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:26.585 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:26.580+0000 7f2664c73700 1 --2- 172.21.15.59:0/1391451960 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f26600b03b0 0x7f26600cecb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:26.585 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:26.580+0000 7f2664c73700 1 -- 172.21.15.59:0/1391451960 >> 172.21.15.59:0/1391451960 conn(0x7f26600b5070 msgr2=0x7f26600ad690 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:58:26.585 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:26.580+0000 7f2664c73700 1 -- 172.21.15.59:0/1391451960 shutdown_connections 2023-11-24T07:58:26.585 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:26.580+0000 7f2664c73700 1 -- 172.21.15.59:0/1391451960 wait complete. 2023-11-24T07:58:26.595 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2023-11-24T07:58:26.724 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:26.722+0000 7f45522c1700 1 Processor -- start 2023-11-24T07:58:26.724 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:26.722+0000 7f45522c1700 1 -- start start 2023-11-24T07:58:26.726 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:26.723+0000 7f45522c1700 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f454c06b620 0x7f454c0b3a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:26.726 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:26.723+0000 7f45522c1700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f454c0b5ab0 0x7f454c0b3f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:26.726 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:26.723+0000 7f45522c1700 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f454c1478c0 0x7f454c161ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:26.726 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:26.723+0000 7f45522c1700 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f454c071f50 con 0x7f454c0b5ab0 2023-11-24T07:58:26.726 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:26.723+0000 7f45522c1700 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f454c072230 con 0x7f454c1478c0 2023-11-24T07:58:26.726 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:26.723+0000 7f45522c1700 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f454c072510 con 0x7f454c06b620 2023-11-24T07:58:26.727 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:26.723+0000 7f454b7fe700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f454c0b5ab0 0x7f454c0b3f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:26.727 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:26.723+0000 7f454b7fe700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f454c0b5ab0 0x7f454c0b3f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:53426/0 (socket says 172.21.15.59:53426) 2023-11-24T07:58:26.727 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:26.723+0000 7f454b7fe700 1 -- 172.21.15.59:0/3363095485 learned_addr learned my addr 172.21.15.59:0/3363095485 (peer_addr_for_me v2:172.21.15.59:0/0) 2023-11-24T07:58:26.727 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:26.723+0000 7f454b7fe700 1 -- 172.21.15.59:0/3363095485 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f454c06b620 msgr2=0x7f454c0b3a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T07:58:26.727 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:26.723+0000 7f454b7fe700 1 --2- 172.21.15.59:0/3363095485 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f454c06b620 0x7f454c0b3a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:26.727 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:26.723+0000 7f454b7fe700 1 -- 172.21.15.59:0/3363095485 >> v2:172.21.15.76:3300/0 conn(0x7f454c1478c0 msgr2=0x7f454c161ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T07:58:26.727 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:26.723+0000 7f455085e700 1 --2- 172.21.15.59:0/3363095485 >> v2:172.21.15.76:3300/0 conn(0x7f454c1478c0 0x7f454c161ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:26.727 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:26.723+0000 7f454b7fe700 1 --2- 172.21.15.59:0/3363095485 >> v2:172.21.15.76:3300/0 conn(0x7f454c1478c0 0x7f454c161ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:26.727 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:26.723+0000 7f454b7fe700 1 -- 172.21.15.59:0/3363095485 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f454c0727f0 con 0x7f454c0b5ab0 2023-11-24T07:58:26.727 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:26.723+0000 7f454b7fe700 1 --2- 172.21.15.59:0/3363095485 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f454c0b5ab0 0x7f454c0b3f80 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7f454c0747c0 tx=0x7f4540009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6f41d410d88dba48 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:58:26.727 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:26.723+0000 7f454affd700 1 -- 172.21.15.59:0/3363095485 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f454000b870 con 0x7f454c0b5ab0 2023-11-24T07:58:26.727 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:26.724+0000 7f454affd700 1 -- 172.21.15.59:0/3363095485 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f454000b9d0 con 0x7f454c0b5ab0 2023-11-24T07:58:26.728 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:26.724+0000 7f454affd700 1 -- 172.21.15.59:0/3363095485 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f454000bd60 con 0x7f454c0b5ab0 2023-11-24T07:58:26.728 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:26.724+0000 7f45522c1700 1 -- 172.21.15.59:0/3363095485 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f454c0b5ab0 msgr2=0x7f454c0b3f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:26.728 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:26.724+0000 7f45522c1700 1 --2- 172.21.15.59:0/3363095485 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f454c0b5ab0 0x7f454c0b3f80 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7f454c0747c0 tx=0x7f4540009bd0 comp rx=0 tx=0).stop 2023-11-24T07:58:26.728 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:26.724+0000 7f45522c1700 1 -- 172.21.15.59:0/3363095485 shutdown_connections 2023-11-24T07:58:26.728 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:26.724+0000 7f45522c1700 1 --2- 172.21.15.59:0/3363095485 >> v2:172.21.15.76:3300/0 conn(0x7f454c1478c0 0x7f454c161ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:26.728 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:26.724+0000 7f45522c1700 1 --2- 172.21.15.59:0/3363095485 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f454c06b620 0x7f454c0b3a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:26.728 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:26.724+0000 7f45522c1700 1 --2- 172.21.15.59:0/3363095485 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f454c0b5ab0 0x7f454c0b3f80 unknown :-1 s=CLOSED pgs=640 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:26.728 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:26.724+0000 7f45522c1700 1 -- 172.21.15.59:0/3363095485 >> 172.21.15.59:0/3363095485 conn(0x7f454c0b2740 msgr2=0x7f454c0b2b20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:58:26.728 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:26.724+0000 7f45522c1700 1 -- 172.21.15.59:0/3363095485 shutdown_connections 2023-11-24T07:58:26.728 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:26.724+0000 7f45522c1700 1 -- 172.21.15.59:0/3363095485 wait complete. 2023-11-24T07:58:26.731 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:26.725+0000 7f45522c1700 1 Processor -- start 2023-11-24T07:58:26.731 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:26.726+0000 7f45522c1700 1 -- start start 2023-11-24T07:58:26.731 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:26.726+0000 7f45522c1700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f454c06b620 0x7f454c1e85c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:26.732 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:26.726+0000 7f45522c1700 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f454c0b5ab0 0x7f454c1f7c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:26.732 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:26.726+0000 7f45522c1700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f454c1478c0 0x7f454c0cf2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:26.732 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:26.726+0000 7f45522c1700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f454c1f81d0 con 0x7f454c1478c0 2023-11-24T07:58:26.732 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:26.726+0000 7f45522c1700 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f454c1f4fd0 con 0x7f454c06b620 2023-11-24T07:58:26.732 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:26.726+0000 7f45522c1700 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f454c1e8ba0 con 0x7f454c0b5ab0 2023-11-24T07:58:26.732 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:26.726+0000 7f455085e700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f454c1478c0 0x7f454c0cf2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:26.732 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:26.726+0000 7f455085e700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f454c1478c0 0x7f454c0cf2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:53436/0 (socket says 172.21.15.59:53436) 2023-11-24T07:58:26.733 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:26.726+0000 7f455085e700 1 -- 172.21.15.59:0/1796047483 learned_addr learned my addr 172.21.15.59:0/1796047483 (peer_addr_for_me v2:172.21.15.59:0/0) 2023-11-24T07:58:26.733 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:26.727+0000 7f454b7fe700 1 --2- 172.21.15.59:0/1796047483 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f454c0b5ab0 0x7f454c1f7c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:26.733 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:26.727+0000 7f454bfff700 1 --2- 172.21.15.59:0/1796047483 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f454c06b620 0x7f454c1e85c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:26.733 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:26.727+0000 7f455085e700 1 -- 172.21.15.59:0/1796047483 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f454c0b5ab0 msgr2=0x7f454c1f7c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:26.733 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:26.727+0000 7f455085e700 1 --2- 172.21.15.59:0/1796047483 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f454c0b5ab0 0x7f454c1f7c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:26.733 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:26.727+0000 7f455085e700 1 -- 172.21.15.59:0/1796047483 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f454c06b620 msgr2=0x7f454c1e85c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:26.734 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:26.727+0000 7f455085e700 1 --2- 172.21.15.59:0/1796047483 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f454c06b620 0x7f454c1e85c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:26.734 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:26.727+0000 7f455085e700 1 -- 172.21.15.59:0/1796047483 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4540009550 con 0x7f454c1478c0 2023-11-24T07:58:26.734 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:26.727+0000 7f455085e700 1 --2- 172.21.15.59:0/1796047483 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f454c1478c0 0x7f454c0cf2e0 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7f454400cc30 tx=0x7f454400cf40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:58:26.734 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:26.727+0000 7f4548ff9700 1 -- 172.21.15.59:0/1796047483 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f454400aa50 con 0x7f454c1478c0 2023-11-24T07:58:26.734 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:26.727+0000 7f4548ff9700 1 -- 172.21.15.59:0/1796047483 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f454400abb0 con 0x7f454c1478c0 2023-11-24T07:58:26.734 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:26.727+0000 7f45522c1700 1 -- 172.21.15.59:0/1796047483 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f454c1f5290 con 0x7f454c1478c0 2023-11-24T07:58:26.734 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:26.727+0000 7f4548ff9700 1 -- 172.21.15.59:0/1796047483 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4544003a20 con 0x7f454c1478c0 2023-11-24T07:58:26.735 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:26.727+0000 7f45522c1700 1 -- 172.21.15.59:0/1796047483 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f454c0cf820 con 0x7f454c1478c0 2023-11-24T07:58:26.735 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:26.728+0000 7f45522c1700 1 -- 172.21.15.59:0/1796047483 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f454c068670 con 0x7f454c1478c0 2023-11-24T07:58:26.735 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:26.729+0000 7f4548ff9700 1 -- 172.21.15.59:0/1796047483 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 8) v1 ==== 89435+0+0 (secure 0 0 0) 0x7f454400f3f0 con 0x7f454c1478c0 2023-11-24T07:58:26.735 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:26.729+0000 7f4548ff9700 1 --2- 172.21.15.59:0/1796047483 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f453806b110 0x7f453806d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:26.735 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:26.729+0000 7f4548ff9700 1 -- 172.21.15.59:0/1796047483 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 5882+0+0 (secure 0 0 0) 0x7f4544089c10 con 0x7f454c1478c0 2023-11-24T07:58:26.735 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:26.729+0000 7f454bfff700 1 --2- 172.21.15.59:0/1796047483 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f453806b110 0x7f453806d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:26.736 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:26.730+0000 7f454bfff700 1 --2- 172.21.15.59:0/1796047483 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f453806b110 0x7f453806d5d0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f453c009a10 tx=0x7f453c009190 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:58:26.736 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:26.733+0000 7f4548ff9700 1 -- 172.21.15.59:0/1796047483 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f45440547a0 con 0x7f454c1478c0 2023-11-24T07:58:26.983 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:26.981+0000 7f45522c1700 1 -- 172.21.15.59:0/1796047483 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f454c06bd40 con 0x7f454c1478c0 2023-11-24T07:58:26.984 INFO:tasks.ceph.mon.a.smithi059.stderr:2023-11-24T07:58:26.982+0000 7f5560821700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-11-24T07:58:26.985 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:26.983+0000 7f4548ff9700 1 -- 172.21.15.59:0/1796047483 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v27) v1 ==== 161+0+0 (secure 0 0 0) 0x7f454401d070 con 0x7f454c1478c0 2023-11-24T07:58:26.988 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:26.986+0000 7f45522c1700 1 -- 172.21.15.59:0/1796047483 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f453806b110 msgr2=0x7f453806d5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:26.988 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:26.986+0000 7f45522c1700 1 --2- 172.21.15.59:0/1796047483 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f453806b110 0x7f453806d5d0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f453c009a10 tx=0x7f453c009190 comp rx=0 tx=0).stop 2023-11-24T07:58:26.988 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:26.986+0000 7f45522c1700 1 -- 172.21.15.59:0/1796047483 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f454c1478c0 msgr2=0x7f454c0cf2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:26.988 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:26.986+0000 7f45522c1700 1 --2- 172.21.15.59:0/1796047483 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f454c1478c0 0x7f454c0cf2e0 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7f454400cc30 tx=0x7f454400cf40 comp rx=0 tx=0).stop 2023-11-24T07:58:26.988 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:26.986+0000 7f45522c1700 1 -- 172.21.15.59:0/1796047483 shutdown_connections 2023-11-24T07:58:26.988 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:26.986+0000 7f45522c1700 1 --2- 172.21.15.59:0/1796047483 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f453806b110 0x7f453806d5d0 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:26.988 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:26.986+0000 7f45522c1700 1 --2- 172.21.15.59:0/1796047483 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f454c1478c0 0x7f454c0cf2e0 unknown :-1 s=CLOSED pgs=641 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:26.988 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:26.986+0000 7f45522c1700 1 --2- 172.21.15.59:0/1796047483 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f454c06b620 0x7f454c1e85c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:26.989 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:26.986+0000 7f45522c1700 1 --2- 172.21.15.59:0/1796047483 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f454c0b5ab0 0x7f454c1f7c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:26.989 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:26.986+0000 7f45522c1700 1 -- 172.21.15.59:0/1796047483 >> 172.21.15.59:0/1796047483 conn(0x7f454c0b2740 msgr2=0x7f454c073f30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:58:26.989 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:26.986+0000 7f45522c1700 1 -- 172.21.15.59:0/1796047483 shutdown_connections 2023-11-24T07:58:26.989 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:26.986+0000 7f45522c1700 1 -- 172.21.15.59:0/1796047483 wait complete. 2023-11-24T07:58:26.989 INFO:teuthology.orchestra.run.smithi059.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2023-11-24T07:58:27.001 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2023-11-24T07:58:27.027 INFO:tasks.ceph.mds.c.smithi059.stdout:starting mds.c at 2023-11-24T07:58:27.130 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:27.128+0000 7f06ea8a2700 1 Processor -- start 2023-11-24T07:58:27.130 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:27.128+0000 7f06ea8a2700 1 -- start start 2023-11-24T07:58:27.132 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:27.129+0000 7f06ea8a2700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f06e4142810 0x7f06e406afd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:27.132 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:27.129+0000 7f06ea8a2700 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f06e406e580 0x7f06e406b510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:27.132 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:27.129+0000 7f06ea8a2700 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f06e40cf5f0 con 0x7f06e41459a0 2023-11-24T07:58:27.132 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:27.129+0000 7f06ea8a2700 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f06e406ea80 con 0x7f06e4142810 2023-11-24T07:58:27.132 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:27.129+0000 7f06ea8a2700 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f06e406ed60 con 0x7f06e406e580 2023-11-24T07:58:27.133 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:27.129+0000 7f06e3fff700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f06e4142810 0x7f06e406afd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:27.133 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:27.129+0000 7f06e37fe700 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7f06e41459a0 0x7f06e4071bb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:56932/0 (socket says 172.21.15.59:56932) 2023-11-24T07:58:27.133 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:27.129+0000 7f06e37fe700 1 -- 172.21.15.59:0/816185629 learned_addr learned my addr 172.21.15.59:0/816185629 (peer_addr_for_me v1:172.21.15.59:0/0) 2023-11-24T07:58:27.133 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:27.129+0000 7f06e8e3f700 1 --2- 172.21.15.59:0/816185629 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f06e406e580 0x7f06e406b510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:27.133 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:27.129+0000 7f06e3fff700 1 -- 172.21.15.59:0/816185629 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f06e406e580 msgr2=0x7f06e406b510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T07:58:27.133 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:27.129+0000 7f06e3fff700 1 --2- 172.21.15.59:0/816185629 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f06e406e580 0x7f06e406b510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:27.133 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:27.129+0000 7f06e3fff700 1 -- 172.21.15.59:0/816185629 >> v1:172.21.15.59:6789/0 conn(0x7f06e41459a0 legacy=0x7f06e4071bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:27.133 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:27.129+0000 7f06e3fff700 1 -- 172.21.15.59:0/816185629 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f06e406f040 con 0x7f06e4142810 2023-11-24T07:58:27.134 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:27.129+0000 7f06e2ffd700 1 -- 172.21.15.59:0/816185629 <== mon.0 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 413551156 0 0) 0x7f06e40cf5f0 con 0x7f06e41459a0 2023-11-24T07:58:27.134 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:27.130+0000 7f06e3fff700 1 --2- 172.21.15.59:0/816185629 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f06e4142810 0x7f06e406afd0 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7f06d4009840 tx=0x7f06d4009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f1cd1809d899db0e server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:58:27.134 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:27.130+0000 7f06e2ffd700 1 -- 172.21.15.59:0/816185629 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06d4011830 con 0x7f06e4142810 2023-11-24T07:58:27.134 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:27.130+0000 7f06e2ffd700 1 -- 172.21.15.59:0/816185629 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f06d4011990 con 0x7f06e4142810 2023-11-24T07:58:27.134 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:27.130+0000 7f06e2ffd700 1 -- 172.21.15.59:0/816185629 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06d4011d20 con 0x7f06e4142810 2023-11-24T07:58:27.134 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:27.130+0000 7f06ea8a2700 1 -- 172.21.15.59:0/816185629 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f06e4142810 msgr2=0x7f06e406afd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:27.134 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:27.130+0000 7f06ea8a2700 1 --2- 172.21.15.59:0/816185629 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f06e4142810 0x7f06e406afd0 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7f06d4009840 tx=0x7f06d4009b50 comp rx=0 tx=0).stop 2023-11-24T07:58:27.134 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:27.130+0000 7f06ea8a2700 1 -- 172.21.15.59:0/816185629 shutdown_connections 2023-11-24T07:58:27.134 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:27.130+0000 7f06ea8a2700 1 --2- 172.21.15.59:0/816185629 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f06e406e580 0x7f06e406b510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:27.134 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:27.130+0000 7f06ea8a2700 1 --2- 172.21.15.59:0/816185629 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f06e4142810 0x7f06e406afd0 unknown :-1 s=CLOSED pgs=646 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:27.135 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:27.130+0000 7f06ea8a2700 1 -- 172.21.15.59:0/816185629 >> 172.21.15.59:0/816185629 conn(0x7f06e40b2740 msgr2=0x7f06e40b2b20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:58:27.135 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:27.130+0000 7f06ea8a2700 1 -- 172.21.15.59:0/816185629 shutdown_connections 2023-11-24T07:58:27.135 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:27.130+0000 7f06ea8a2700 1 -- 172.21.15.59:0/816185629 wait complete. 2023-11-24T07:58:27.137 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:27.132+0000 7f06ea8a2700 1 Processor -- start 2023-11-24T07:58:27.137 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:27.132+0000 7f06ea8a2700 1 -- start start 2023-11-24T07:58:27.137 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:27.132+0000 7f06ea8a2700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f06e406e580 0x7f06e4158240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:27.138 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:27.132+0000 7f06ea8a2700 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f06e4142810 0x7f06e40b5320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:27.138 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:27.132+0000 7f06ea8a2700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f06e41459a0 0x7f06e413eb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:27.138 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:27.132+0000 7f06ea8a2700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06e41441e0 con 0x7f06e406e580 2023-11-24T07:58:27.138 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:27.132+0000 7f06ea8a2700 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06e41435d0 con 0x7f06e41459a0 2023-11-24T07:58:27.138 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:27.132+0000 7f06ea8a2700 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f06e41438b0 con 0x7f06e4142810 2023-11-24T07:58:27.138 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:27.133+0000 7f06e3fff700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f06e406e580 0x7f06e4158240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:27.138 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:27.133+0000 7f06e3fff700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f06e406e580 0x7f06e4158240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:53464/0 (socket says 172.21.15.59:53464) 2023-11-24T07:58:27.138 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:27.133+0000 7f06e3fff700 1 -- 172.21.15.59:0/3114219150 learned_addr learned my addr 172.21.15.59:0/3114219150 (peer_addr_for_me v2:172.21.15.59:0/0) 2023-11-24T07:58:27.138 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:27.133+0000 7f06e3fff700 1 -- 172.21.15.59:0/3114219150 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f06e4142810 msgr2=0x7f06e40b5320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:27.138 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:27.133+0000 7f06e3fff700 1 --2- 172.21.15.59:0/3114219150 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f06e4142810 0x7f06e40b5320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:27.138 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:27.133+0000 7f06e3fff700 1 -- 172.21.15.59:0/3114219150 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f06e41459a0 msgr2=0x7f06e413eb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:27.139 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:27.133+0000 7f06e3fff700 1 --2- 172.21.15.59:0/3114219150 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f06e41459a0 0x7f06e413eb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:27.139 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:27.133+0000 7f06e3fff700 1 -- 172.21.15.59:0/3114219150 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f06d40094a0 con 0x7f06e406e580 2023-11-24T07:58:27.139 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:27.133+0000 7f06e3fff700 1 --2- 172.21.15.59:0/3114219150 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f06e406e580 0x7f06e4158240 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7f06d4009fd0 tx=0x7f06d4004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:58:27.139 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:27.133+0000 7f06e0ff9700 1 -- 172.21.15.59:0/3114219150 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06d40036a0 con 0x7f06e406e580 2023-11-24T07:58:27.139 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:27.133+0000 7f06e0ff9700 1 -- 172.21.15.59:0/3114219150 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f06d401f070 con 0x7f06e406e580 2023-11-24T07:58:27.139 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:27.133+0000 7f06ea8a2700 1 -- 172.21.15.59:0/3114219150 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f06e4143b90 con 0x7f06e406e580 2023-11-24T07:58:27.139 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:27.133+0000 7f06e0ff9700 1 -- 172.21.15.59:0/3114219150 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06d4003a20 con 0x7f06e406e580 2023-11-24T07:58:27.139 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:27.133+0000 7f06ea8a2700 1 -- 172.21.15.59:0/3114219150 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f06e4143210 con 0x7f06e406e580 2023-11-24T07:58:27.139 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:27.134+0000 7f06ea8a2700 1 -- 172.21.15.59:0/3114219150 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f06e41433a0 con 0x7f06e406e580 2023-11-24T07:58:27.139 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:27.135+0000 7f06e0ff9700 1 -- 172.21.15.59:0/3114219150 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 8) v1 ==== 89435+0+0 (secure 0 0 0) 0x7f06d4003b80 con 0x7f06e406e580 2023-11-24T07:58:27.139 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:27.135+0000 7f06e0ff9700 1 --2- 172.21.15.59:0/3114219150 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f06c406b110 0x7f06c406d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:27.139 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:27.135+0000 7f06e0ff9700 1 -- 172.21.15.59:0/3114219150 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 5940+0+0 (secure 0 0 0) 0x7f06d4021070 con 0x7f06e406e580 2023-11-24T07:58:27.140 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:27.135+0000 7f06e37fe700 1 --2- 172.21.15.59:0/3114219150 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f06c406b110 0x7f06c406d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:27.140 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:27.136+0000 7f06e37fe700 1 --2- 172.21.15.59:0/3114219150 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f06c406b110 0x7f06c406d5d0 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f06e415f8d0 tx=0x7f06cc0079f0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:58:27.141 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:27.139+0000 7f06e0ff9700 1 -- 172.21.15.59:0/3114219150 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f06d40560d0 con 0x7f06e406e580 2023-11-24T07:58:27.388 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:27.386+0000 7f06ea8a2700 1 -- 172.21.15.59:0/3114219150 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f06e40679c0 con 0x7f06e406e580 2023-11-24T07:58:27.986 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:27.985+0000 7f06e0ff9700 1 -- 172.21.15.59:0/3114219150 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v28) v1 ==== 104+0+0 (secure 0 0 0) 0x7f06d40596f0 con 0x7f06e406e580 2023-11-24T07:58:27.989 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:27.988+0000 7f06ea8a2700 1 -- 172.21.15.59:0/3114219150 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f06c406b110 msgr2=0x7f06c406d5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:27.990 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:27.988+0000 7f06ea8a2700 1 --2- 172.21.15.59:0/3114219150 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f06c406b110 0x7f06c406d5d0 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f06e415f8d0 tx=0x7f06cc0079f0 comp rx=0 tx=0).stop 2023-11-24T07:58:27.990 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:27.988+0000 7f06ea8a2700 1 -- 172.21.15.59:0/3114219150 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f06e406e580 msgr2=0x7f06e4158240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:27.990 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:27.988+0000 7f06ea8a2700 1 --2- 172.21.15.59:0/3114219150 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f06e406e580 0x7f06e4158240 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7f06d4009fd0 tx=0x7f06d4004750 comp rx=0 tx=0).stop 2023-11-24T07:58:27.990 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:27.988+0000 7f06ea8a2700 1 -- 172.21.15.59:0/3114219150 shutdown_connections 2023-11-24T07:58:27.990 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:27.988+0000 7f06ea8a2700 1 --2- 172.21.15.59:0/3114219150 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f06c406b110 0x7f06c406d5d0 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:27.990 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:27.988+0000 7f06ea8a2700 1 --2- 172.21.15.59:0/3114219150 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f06e41459a0 0x7f06e413eb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:27.990 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:27.988+0000 7f06ea8a2700 1 --2- 172.21.15.59:0/3114219150 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f06e406e580 0x7f06e4158240 unknown :-1 s=CLOSED pgs=647 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:27.990 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:27.988+0000 7f06ea8a2700 1 --2- 172.21.15.59:0/3114219150 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f06e4142810 0x7f06e40b5320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:27.990 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:27.988+0000 7f06ea8a2700 1 -- 172.21.15.59:0/3114219150 >> 172.21.15.59:0/3114219150 conn(0x7f06e40b2740 msgr2=0x7f06e4141460 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:58:27.991 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:27.988+0000 7f06ea8a2700 1 -- 172.21.15.59:0/3114219150 shutdown_connections 2023-11-24T07:58:27.991 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:27.988+0000 7f06ea8a2700 1 -- 172.21.15.59:0/3114219150 wait complete. 2023-11-24T07:58:28.005 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2023-11-24T07:58:28.184 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:28.182+0000 7fda00854700 1 Processor -- start 2023-11-24T07:58:28.184 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:28.182+0000 7fda00854700 1 -- start start 2023-11-24T07:58:28.188 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:28.183+0000 7fda00854700 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fd9fc0722a0 0x7fd9fc06e060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:28.188 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:28.183+0000 7fda00854700 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd9fc06dbd0 con 0x7fd9fc1447a0 2023-11-24T07:58:28.188 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:28.183+0000 7fda00854700 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd9fc15b700 con 0x7fd9fc072ab0 2023-11-24T07:58:28.188 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:28.183+0000 7fda00854700 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd9fc15b9e0 con 0x7fd9fc0722a0 2023-11-24T07:58:28.188 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:28.183+0000 7fd9fa59c700 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7fd9fc1447a0 0x7fd9fc071b90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:56946/0 (socket says 172.21.15.59:56946) 2023-11-24T07:58:28.188 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:28.183+0000 7fd9fa59c700 1 -- 172.21.15.59:0/1349459983 learned_addr learned my addr 172.21.15.59:0/1349459983 (peer_addr_for_me v1:172.21.15.59:0/0) 2023-11-24T07:58:28.188 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:28.183+0000 7fd9f9d9b700 1 --2- 172.21.15.59:0/1349459983 >> v2:172.21.15.76:3300/0 conn(0x7fd9fc0722a0 0x7fd9fc06e060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:28.189 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:28.183+0000 7fd9f959a700 1 -- 172.21.15.59:0/1349459983 <== mon.0 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2454270509 0 0) 0x7fd9fc06dbd0 con 0x7fd9fc1447a0 2023-11-24T07:58:28.189 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:28.183+0000 7fd9f959a700 1 -- 172.21.15.59:0/1349459983 --> v1:172.21.15.59:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd9e40035b0 con 0x7fd9fc1447a0 2023-11-24T07:58:28.189 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:28.183+0000 7fd9f959a700 1 -- 172.21.15.59:0/1349459983 <== mon.0 v1:172.21.15.59:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1490636797 0 0) 0x7fd9e40035b0 con 0x7fd9fc1447a0 2023-11-24T07:58:28.189 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:28.183+0000 7fd9f959a700 1 -- 172.21.15.59:0/1349459983 >> v2:172.21.15.76:3300/0 conn(0x7fd9fc0722a0 msgr2=0x7fd9fc06e060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T07:58:28.189 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:28.183+0000 7fd9f959a700 1 --2- 172.21.15.59:0/1349459983 >> v2:172.21.15.76:3300/0 conn(0x7fd9fc0722a0 0x7fd9fc06e060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:28.189 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:28.183+0000 7fd9f959a700 1 -- 172.21.15.59:0/1349459983 >> v1:172.21.15.76:6789/0 conn(0x7fd9fc072ab0 legacy=0x7fd9fc06e5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:28.189 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:28.183+0000 7fd9f959a700 1 -- 172.21.15.59:0/1349459983 --> v1:172.21.15.59:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd9fc15bcc0 con 0x7fd9fc1447a0 2023-11-24T07:58:28.189 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:28.183+0000 7fd9f959a700 1 -- 172.21.15.59:0/1349459983 <== mon.0 v1:172.21.15.59:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1312333566 0 0) 0x7fd9ec001670 con 0x7fd9fc1447a0 2023-11-24T07:58:28.189 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:28.183+0000 7fd9f959a700 1 -- 172.21.15.59:0/1349459983 >> v1:172.21.15.59:6789/0 conn(0x7fd9fc1447a0 legacy=0x7fd9fc071b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:28.189 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:28.183+0000 7fd9f959a700 1 --2- 172.21.15.59:0/1349459983 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fd9e4003a00 0x7fd9e4003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:28.189 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:28.183+0000 7fd9f959a700 1 --2- 172.21.15.59:0/1349459983 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd9e4004400 0x7fd9e40048d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:28.189 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:28.184+0000 7fd9f959a700 1 --2- 172.21.15.59:0/1349459983 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd9e40050d0 0x7fd9e400ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:28.190 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:28.184+0000 7fd9f959a700 1 -- 172.21.15.59:0/1349459983 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9fc15b9e0 con 0x7fd9e40050d0 2023-11-24T07:58:28.190 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:28.184+0000 7fd9f959a700 1 -- 172.21.15.59:0/1349459983 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9e400c030 con 0x7fd9e4004400 2023-11-24T07:58:28.190 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:28.184+0000 7fd9f959a700 1 -- 172.21.15.59:0/1349459983 --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd9e400c1f0 con 0x7fd9e4003a00 2023-11-24T07:58:28.190 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:28.184+0000 7fd9fad9d700 1 --2- 172.21.15.59:0/1349459983 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd9e40050d0 0x7fd9e400ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:28.190 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:28.184+0000 7fd9f9d9b700 1 --2- 172.21.15.59:0/1349459983 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd9e4004400 0x7fd9e40048d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:28.190 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:28.184+0000 7fd9fa59c700 1 --2- 172.21.15.59:0/1349459983 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fd9e4003a00 0x7fd9e4003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:28.190 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:28.184+0000 7fd9fad9d700 1 -- 172.21.15.59:0/1349459983 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fd9e4003a00 msgr2=0x7fd9e4003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T07:58:28.190 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:28.184+0000 7fd9fad9d700 1 --2- 172.21.15.59:0/1349459983 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fd9e4003a00 0x7fd9e4003de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:28.190 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:28.184+0000 7fd9fad9d700 1 -- 172.21.15.59:0/1349459983 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd9e4004400 msgr2=0x7fd9e40048d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T07:58:28.190 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:28.184+0000 7fd9fad9d700 1 --2- 172.21.15.59:0/1349459983 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd9e4004400 0x7fd9e40048d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:28.190 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:28.184+0000 7fd9fad9d700 1 -- 172.21.15.59:0/1349459983 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd9e40035b0 con 0x7fd9e40050d0 2023-11-24T07:58:28.191 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:28.184+0000 7fd9fa59c700 1 --2- 172.21.15.59:0/1349459983 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fd9e4003a00 0x7fd9e4003de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-24T07:58:28.191 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:28.184+0000 7fd9fad9d700 1 -- 172.21.15.59:0/1349459983 reap_dead start 2023-11-24T07:58:28.191 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:28.184+0000 7fd9fad9d700 1 --2- 172.21.15.59:0/1349459983 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd9e40050d0 0x7fd9e400ba10 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7fd9f400c290 tx=0x7fd9e4005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6fc2652dc7f9842e server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:58:28.191 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:28.184+0000 7fd9f959a700 1 -- 172.21.15.59:0/1349459983 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9f4001e70 con 0x7fd9e40050d0 2023-11-24T07:58:28.191 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:28.184+0000 7fd9f959a700 1 -- 172.21.15.59:0/1349459983 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd9f400ce70 con 0x7fd9e40050d0 2023-11-24T07:58:28.191 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:28.184+0000 7fd9f959a700 1 -- 172.21.15.59:0/1349459983 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9f4003c20 con 0x7fd9e40050d0 2023-11-24T07:58:28.191 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:28.185+0000 7fda00854700 1 -- 172.21.15.59:0/1349459983 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd9e40050d0 msgr2=0x7fd9e400ba10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:28.191 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:28.185+0000 7fda00854700 1 --2- 172.21.15.59:0/1349459983 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd9e40050d0 0x7fd9e400ba10 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7fd9f400c290 tx=0x7fd9e4005880 comp rx=0 tx=0).stop 2023-11-24T07:58:28.191 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:28.185+0000 7fda00854700 1 -- 172.21.15.59:0/1349459983 shutdown_connections 2023-11-24T07:58:28.191 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:28.185+0000 7fda00854700 1 --2- 172.21.15.59:0/1349459983 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd9e40050d0 0x7fd9e400ba10 unknown :-1 s=CLOSED pgs=649 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:28.191 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:28.185+0000 7fda00854700 1 -- 172.21.15.59:0/1349459983 >> 172.21.15.59:0/1349459983 conn(0x7fd9fc0b2760 msgr2=0x7fd9fc0b2b70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:58:28.192 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:28.185+0000 7fda00854700 1 -- 172.21.15.59:0/1349459983 shutdown_connections 2023-11-24T07:58:28.192 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:28.185+0000 7fda00854700 1 -- 172.21.15.59:0/1349459983 wait complete. 2023-11-24T07:58:28.192 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:28.186+0000 7fda00854700 1 Processor -- start 2023-11-24T07:58:28.192 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:28.186+0000 7fda00854700 1 -- start start 2023-11-24T07:58:28.194 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:28.187+0000 7fda00854700 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fd9e40050d0 0x7fd9fc13cae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:28.194 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:28.187+0000 7fda00854700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd9fc0ccfe0 0x7fd9fc13d110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:28.194 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:28.187+0000 7fda00854700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd9fc0cd460 0x7fd9fc13bdd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:28.195 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:28.187+0000 7fda00854700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9fc13b8f0 con 0x7fd9fc0ccfe0 2023-11-24T07:58:28.195 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:28.187+0000 7fda00854700 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9fc141f10 con 0x7fd9fc0cd460 2023-11-24T07:58:28.195 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:28.187+0000 7fda00854700 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd9fc141670 con 0x7fd9e40050d0 2023-11-24T07:58:28.195 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:28.187+0000 7fd9f9d9b700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd9fc0ccfe0 0x7fd9fc13d110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:28.195 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:28.187+0000 7fd9f9d9b700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd9fc0ccfe0 0x7fd9fc13d110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:53488/0 (socket says 172.21.15.59:53488) 2023-11-24T07:58:28.195 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:28.187+0000 7fd9f9d9b700 1 -- 172.21.15.59:0/3051896128 learned_addr learned my addr 172.21.15.59:0/3051896128 (peer_addr_for_me v2:172.21.15.59:0/0) 2023-11-24T07:58:28.195 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:28.187+0000 7fd9fa59c700 1 --2- 172.21.15.59:0/3051896128 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fd9e40050d0 0x7fd9fc13cae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:28.195 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:28.187+0000 7fd9fad9d700 1 --2- 172.21.15.59:0/3051896128 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd9fc0cd460 0x7fd9fc13bdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:28.195 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:28.188+0000 7fd9f9d9b700 1 -- 172.21.15.59:0/3051896128 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fd9e40050d0 msgr2=0x7fd9fc13cae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:28.195 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:28.188+0000 7fd9f9d9b700 1 --2- 172.21.15.59:0/3051896128 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fd9e40050d0 0x7fd9fc13cae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:28.195 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:28.188+0000 7fd9f9d9b700 1 -- 172.21.15.59:0/3051896128 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd9fc0cd460 msgr2=0x7fd9fc13bdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:28.196 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:28.188+0000 7fd9f9d9b700 1 --2- 172.21.15.59:0/3051896128 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd9fc0cd460 0x7fd9fc13bdd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:28.196 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:28.188+0000 7fd9f9d9b700 1 -- 172.21.15.59:0/3051896128 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd9fc141950 con 0x7fd9fc0ccfe0 2023-11-24T07:58:28.196 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:28.188+0000 7fd9f9d9b700 1 --2- 172.21.15.59:0/3051896128 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd9fc0ccfe0 0x7fd9fc13d110 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7fd9f00119a0 tx=0x7fd9f0011cb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:58:28.196 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:28.188+0000 7fd9eaffd700 1 -- 172.21.15.59:0/3051896128 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9f000f780 con 0x7fd9fc0ccfe0 2023-11-24T07:58:28.196 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:28.188+0000 7fd9eaffd700 1 -- 172.21.15.59:0/3051896128 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd9f000f8e0 con 0x7fd9fc0ccfe0 2023-11-24T07:58:28.196 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:28.188+0000 7fda00854700 1 -- 172.21.15.59:0/3051896128 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd9fc141c30 con 0x7fd9fc0ccfe0 2023-11-24T07:58:28.196 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:28.188+0000 7fd9eaffd700 1 -- 172.21.15.59:0/3051896128 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9f000fc70 con 0x7fd9fc0ccfe0 2023-11-24T07:58:28.196 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:28.188+0000 7fda00854700 1 -- 172.21.15.59:0/3051896128 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd9fc140940 con 0x7fd9fc0ccfe0 2023-11-24T07:58:28.196 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:28.189+0000 7fda00854700 1 -- 172.21.15.59:0/3051896128 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd9fc13b750 con 0x7fd9fc0ccfe0 2023-11-24T07:58:28.196 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:28.189+0000 7fd9eaffd700 1 -- 172.21.15.59:0/3051896128 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 8) v1 ==== 89435+0+0 (secure 0 0 0) 0x7fd9f0014440 con 0x7fd9fc0ccfe0 2023-11-24T07:58:28.196 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:28.190+0000 7fd9eaffd700 1 --2- 172.21.15.59:0/3051896128 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7fd9dc06b110 0x7fd9dc06d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:28.197 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:28.190+0000 7fd9eaffd700 1 -- 172.21.15.59:0/3051896128 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 5940+0+0 (secure 0 0 0) 0x7fd9f008a080 con 0x7fd9fc0ccfe0 2023-11-24T07:58:28.197 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:28.190+0000 7fd9fa59c700 1 --2- 172.21.15.59:0/3051896128 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7fd9dc06b110 0x7fd9dc06d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:28.197 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:28.191+0000 7fd9fa59c700 1 --2- 172.21.15.59:0/3051896128 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7fd9dc06b110 0x7fd9dc06d5d0 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7fd9ec000f60 tx=0x7fd9ec007040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:58:28.197 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:28.195+0000 7fd9eaffd700 1 -- 172.21.15.59:0/3051896128 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fd9fc13b750 con 0x7fd9fc0ccfe0 2023-11-24T07:58:28.399 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:28.398+0000 7fda00854700 1 -- 172.21.15.59:0/3051896128 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fd9fc13b750 con 0x7fd9fc0ccfe0 2023-11-24T07:58:28.994 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:28.993+0000 7fd9eaffd700 1 -- 172.21.15.59:0/3051896128 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v51) v1 ==== 181+0+0 (secure 0 0 0) 0x7fd9fc13b750 con 0x7fd9fc0ccfe0 2023-11-24T07:58:28.997 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:28.995+0000 7fda00854700 1 -- 172.21.15.59:0/3051896128 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7fd9dc06b110 msgr2=0x7fd9dc06d5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:28.997 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:28.995+0000 7fda00854700 1 --2- 172.21.15.59:0/3051896128 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7fd9dc06b110 0x7fd9dc06d5d0 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7fd9ec000f60 tx=0x7fd9ec007040 comp rx=0 tx=0).stop 2023-11-24T07:58:28.997 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:28.995+0000 7fda00854700 1 -- 172.21.15.59:0/3051896128 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd9fc0ccfe0 msgr2=0x7fd9fc13d110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:28.997 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:28.995+0000 7fda00854700 1 --2- 172.21.15.59:0/3051896128 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd9fc0ccfe0 0x7fd9fc13d110 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7fd9f00119a0 tx=0x7fd9f0011cb0 comp rx=0 tx=0).stop 2023-11-24T07:58:28.997 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:28.995+0000 7fda00854700 1 -- 172.21.15.59:0/3051896128 shutdown_connections 2023-11-24T07:58:28.997 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:28.995+0000 7fda00854700 1 --2- 172.21.15.59:0/3051896128 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7fd9dc06b110 0x7fd9dc06d5d0 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:28.997 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:28.995+0000 7fda00854700 1 --2- 172.21.15.59:0/3051896128 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd9fc0cd460 0x7fd9fc13bdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:28.998 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:28.995+0000 7fda00854700 1 --2- 172.21.15.59:0/3051896128 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fd9e40050d0 0x7fd9fc13cae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:28.998 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:28.995+0000 7fda00854700 1 --2- 172.21.15.59:0/3051896128 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd9fc0ccfe0 0x7fd9fc13d110 unknown :-1 s=CLOSED pgs=650 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:28.998 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:28.995+0000 7fda00854700 1 -- 172.21.15.59:0/3051896128 >> 172.21.15.59:0/3051896128 conn(0x7fd9fc0b2760 msgr2=0x7fd9fc06e5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:58:28.998 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:28.995+0000 7fda00854700 1 -- 172.21.15.59:0/3051896128 shutdown_connections 2023-11-24T07:58:28.998 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:28.995+0000 7fda00854700 1 -- 172.21.15.59:0/3051896128 wait complete. 2023-11-24T07:58:29.004 INFO:teuthology.orchestra.run.smithi059.stderr:pool 'cephfs_metadata' removed 2023-11-24T07:58:29.014 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2023-11-24T07:58:29.147 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:29.145+0000 7f7b35c03700 1 Processor -- start 2023-11-24T07:58:29.147 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:29.145+0000 7f7b35c03700 1 -- start start 2023-11-24T07:58:29.150 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:29.146+0000 7f7b35c03700 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f7b3015e1c0 0x7f7b3006eef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:29.150 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:29.146+0000 7f7b35c03700 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7b300d1230 con 0x7f7b300b0e90 2023-11-24T07:58:29.150 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:29.146+0000 7f7b35c03700 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7b300d0c70 con 0x7f7b3006fc50 2023-11-24T07:58:29.151 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:29.146+0000 7f7b35c03700 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7b300d0f50 con 0x7f7b3015e1c0 2023-11-24T07:58:29.151 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:29.146+0000 7f7b2effd700 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7f7b300b0e90 0x7f7b3006f540 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:56968/0 (socket says 172.21.15.59:56968) 2023-11-24T07:58:29.151 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:29.146+0000 7f7b2effd700 1 -- 172.21.15.59:0/1487005087 learned_addr learned my addr 172.21.15.59:0/1487005087 (peer_addr_for_me v1:172.21.15.59:0/0) 2023-11-24T07:58:29.151 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:29.146+0000 7f7b2f7fe700 1 --2- 172.21.15.59:0/1487005087 >> v2:172.21.15.76:3300/0 conn(0x7f7b3015e1c0 0x7f7b3006eef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:29.151 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:29.146+0000 7f7b2e7fc700 1 -- 172.21.15.59:0/1487005087 <== mon.0 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 667705827 0 0) 0x7f7b300d1230 con 0x7f7b300b0e90 2023-11-24T07:58:29.151 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:29.146+0000 7f7b2e7fc700 1 -- 172.21.15.59:0/1487005087 --> v1:172.21.15.59:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7b1c0035b0 con 0x7f7b300b0e90 2023-11-24T07:58:29.151 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:29.147+0000 7f7b2e7fc700 1 -- 172.21.15.59:0/1487005087 <== mon.1 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 319864661 0 0) 0x7f7b300d0c70 con 0x7f7b3006fc50 2023-11-24T07:58:29.151 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:29.147+0000 7f7b2e7fc700 1 -- 172.21.15.59:0/1487005087 --> v1:172.21.15.76:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7b300d1230 con 0x7f7b3006fc50 2023-11-24T07:58:29.151 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:29.147+0000 7f7b2e7fc700 1 -- 172.21.15.59:0/1487005087 <== mon.0 v1:172.21.15.59:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2884484889 0 0) 0x7f7b1c0035b0 con 0x7f7b300b0e90 2023-11-24T07:58:29.151 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:29.147+0000 7f7b2e7fc700 1 -- 172.21.15.59:0/1487005087 >> v2:172.21.15.76:3300/0 conn(0x7f7b3015e1c0 msgr2=0x7f7b3006eef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T07:58:29.151 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:29.147+0000 7f7b2e7fc700 1 --2- 172.21.15.59:0/1487005087 >> v2:172.21.15.76:3300/0 conn(0x7f7b3015e1c0 0x7f7b3006eef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:29.151 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:29.147+0000 7f7b2e7fc700 1 -- 172.21.15.59:0/1487005087 >> v1:172.21.15.76:6789/0 conn(0x7f7b3006fc50 legacy=0x7f7b3006ca10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:29.151 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:29.147+0000 7f7b2e7fc700 1 -- 172.21.15.59:0/1487005087 --> v1:172.21.15.59:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7b3006c210 con 0x7f7b300b0e90 2023-11-24T07:58:29.152 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:29.147+0000 7f7b2e7fc700 1 -- 172.21.15.59:0/1487005087 <== mon.0 v1:172.21.15.59:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1312333566 0 0) 0x7f7b18001670 con 0x7f7b300b0e90 2023-11-24T07:58:29.152 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:29.147+0000 7f7b2e7fc700 1 -- 172.21.15.59:0/1487005087 >> v1:172.21.15.59:6789/0 conn(0x7f7b300b0e90 legacy=0x7f7b3006f540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:29.152 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:29.147+0000 7f7b2e7fc700 1 --2- 172.21.15.59:0/1487005087 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f7b1c003be0 0x7f7b1c003fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:29.152 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:29.147+0000 7f7b2e7fc700 1 --2- 172.21.15.59:0/1487005087 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7b1c004850 0x7f7b1c004cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:29.152 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:29.147+0000 7f7b2e7fc700 1 --2- 172.21.15.59:0/1487005087 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7b1c005260 0x7f7b1c00ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:29.152 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:29.147+0000 7f7b2e7fc700 1 -- 172.21.15.59:0/1487005087 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b300d0f50 con 0x7f7b1c004850 2023-11-24T07:58:29.152 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:29.147+0000 7f7b2e7fc700 1 -- 172.21.15.59:0/1487005087 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b1c00c190 con 0x7f7b1c005260 2023-11-24T07:58:29.152 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:29.147+0000 7f7b2e7fc700 1 -- 172.21.15.59:0/1487005087 --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7b1c00c350 con 0x7f7b1c003be0 2023-11-24T07:58:29.152 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:29.147+0000 7f7b2effd700 1 --2- 172.21.15.59:0/1487005087 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7b1c004850 0x7f7b1c004cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:29.152 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:29.147+0000 7f7b2f7fe700 1 --2- 172.21.15.59:0/1487005087 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f7b1c003be0 0x7f7b1c003fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:29.152 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:29.147+0000 7f7b2ffff700 1 --2- 172.21.15.59:0/1487005087 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7b1c005260 0x7f7b1c00ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:29.152 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:29.147+0000 7f7b2effd700 1 -- 172.21.15.59:0/1487005087 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f7b1c003be0 msgr2=0x7f7b1c003fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T07:58:29.152 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:29.147+0000 7f7b2effd700 1 --2- 172.21.15.59:0/1487005087 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f7b1c003be0 0x7f7b1c003fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:29.153 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:29.147+0000 7f7b2effd700 1 -- 172.21.15.59:0/1487005087 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7b1c005260 msgr2=0x7f7b1c00ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T07:58:29.153 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:29.147+0000 7f7b2effd700 1 --2- 172.21.15.59:0/1487005087 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7b1c005260 0x7f7b1c00ba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:29.153 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:29.147+0000 7f7b2effd700 1 -- 172.21.15.59:0/1487005087 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7b1c0035b0 con 0x7f7b1c004850 2023-11-24T07:58:29.153 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:29.147+0000 7f7b2ffff700 1 -- 172.21.15.59:0/1487005087 reap_dead start 2023-11-24T07:58:29.153 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:29.147+0000 7f7b2effd700 1 --2- 172.21.15.59:0/1487005087 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7b1c004850 0x7f7b1c004cd0 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7f7b1800c160 tx=0x7f7b1c005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b7ad1a11ac6388cd server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:58:29.153 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:29.147+0000 7f7b2e7fc700 1 -- 172.21.15.59:0/1487005087 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b1800cc40 con 0x7f7b1c004850 2023-11-24T07:58:29.153 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:29.148+0000 7f7b2e7fc700 1 -- 172.21.15.59:0/1487005087 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7b1800cda0 con 0x7f7b1c004850 2023-11-24T07:58:29.153 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:29.148+0000 7f7b2e7fc700 1 -- 172.21.15.59:0/1487005087 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b18005ba0 con 0x7f7b1c004850 2023-11-24T07:58:29.153 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:29.148+0000 7f7b35c03700 1 -- 172.21.15.59:0/1487005087 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7b1c004850 msgr2=0x7f7b1c004cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:29.153 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:29.148+0000 7f7b35c03700 1 --2- 172.21.15.59:0/1487005087 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7b1c004850 0x7f7b1c004cd0 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7f7b1800c160 tx=0x7f7b1c005880 comp rx=0 tx=0).stop 2023-11-24T07:58:29.153 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:29.148+0000 7f7b35c03700 1 -- 172.21.15.59:0/1487005087 shutdown_connections 2023-11-24T07:58:29.153 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:29.148+0000 7f7b35c03700 1 --2- 172.21.15.59:0/1487005087 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7b1c004850 0x7f7b1c004cd0 unknown :-1 s=CLOSED pgs=652 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:29.154 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:29.148+0000 7f7b35c03700 1 -- 172.21.15.59:0/1487005087 >> 172.21.15.59:0/1487005087 conn(0x7f7b300b3720 msgr2=0x7f7b300b3b30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:58:29.154 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:29.148+0000 7f7b35c03700 1 -- 172.21.15.59:0/1487005087 shutdown_connections 2023-11-24T07:58:29.154 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:29.148+0000 7f7b35c03700 1 -- 172.21.15.59:0/1487005087 wait complete. 2023-11-24T07:58:29.156 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:29.150+0000 7f7b35c03700 1 Processor -- start 2023-11-24T07:58:29.156 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:29.150+0000 7f7b35c03700 1 -- start start 2023-11-24T07:58:29.157 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:29.151+0000 7f7b35c03700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7b1c004850 0x7f7b300b5d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:29.157 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:29.151+0000 7f7b35c03700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7b301f7180 0x7f7b3009ba80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:29.157 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:29.151+0000 7f7b35c03700 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f7b300b2e20 0x7f7b30159cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:29.157 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:29.151+0000 7f7b35c03700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b301603d0 con 0x7f7b1c004850 2023-11-24T07:58:29.157 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:29.151+0000 7f7b35c03700 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b301f7660 con 0x7f7b301f7180 2023-11-24T07:58:29.157 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:29.151+0000 7f7b35c03700 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7b301f7940 con 0x7f7b300b2e20 2023-11-24T07:58:29.157 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:29.151+0000 7f7b2f7fe700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7b1c004850 0x7f7b300b5d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:29.157 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:29.151+0000 7f7b2f7fe700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7b1c004850 0x7f7b300b5d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:53506/0 (socket says 172.21.15.59:53506) 2023-11-24T07:58:29.157 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:29.151+0000 7f7b2f7fe700 1 -- 172.21.15.59:0/3198594626 learned_addr learned my addr 172.21.15.59:0/3198594626 (peer_addr_for_me v2:172.21.15.59:0/0) 2023-11-24T07:58:29.157 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:29.151+0000 7f7b2ffff700 1 --2- 172.21.15.59:0/3198594626 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f7b300b2e20 0x7f7b30159cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:29.157 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:29.151+0000 7f7b2effd700 1 --2- 172.21.15.59:0/3198594626 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7b301f7180 0x7f7b3009ba80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:29.157 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:29.151+0000 7f7b2f7fe700 1 -- 172.21.15.59:0/3198594626 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f7b300b2e20 msgr2=0x7f7b30159cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:29.158 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:29.151+0000 7f7b2f7fe700 1 --2- 172.21.15.59:0/3198594626 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f7b300b2e20 0x7f7b30159cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:29.158 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:29.151+0000 7f7b2f7fe700 1 -- 172.21.15.59:0/3198594626 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7b301f7180 msgr2=0x7f7b3009ba80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:29.158 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:29.151+0000 7f7b2f7fe700 1 --2- 172.21.15.59:0/3198594626 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7b301f7180 0x7f7b3009ba80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:29.158 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:29.151+0000 7f7b2f7fe700 1 -- 172.21.15.59:0/3198594626 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7b301f7c20 con 0x7f7b1c004850 2023-11-24T07:58:29.158 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:29.151+0000 7f7b2ffff700 1 --2- 172.21.15.59:0/3198594626 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f7b300b2e20 0x7f7b30159cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-24T07:58:29.158 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:29.151+0000 7f7b2f7fe700 1 --2- 172.21.15.59:0/3198594626 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7b1c004850 0x7f7b300b5d20 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7f7b20011aa0 tx=0x7f7b20011e60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:58:29.158 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:29.152+0000 7f7b27fff700 1 -- 172.21.15.59:0/3198594626 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b2000f980 con 0x7f7b1c004850 2023-11-24T07:58:29.158 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:29.152+0000 7f7b27fff700 1 -- 172.21.15.59:0/3198594626 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7b2000fb00 con 0x7f7b1c004850 2023-11-24T07:58:29.158 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:29.152+0000 7f7b35c03700 1 -- 172.21.15.59:0/3198594626 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7b301f7f00 con 0x7f7b1c004850 2023-11-24T07:58:29.158 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:29.152+0000 7f7b27fff700 1 -- 172.21.15.59:0/3198594626 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b2000feb0 con 0x7f7b1c004850 2023-11-24T07:58:29.158 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:29.152+0000 7f7b35c03700 1 -- 172.21.15.59:0/3198594626 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7b300b32d0 con 0x7f7b1c004850 2023-11-24T07:58:29.158 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:29.153+0000 7f7b35c03700 1 -- 172.21.15.59:0/3198594626 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7b300b3550 con 0x7f7b1c004850 2023-11-24T07:58:29.159 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:29.153+0000 7f7b27fff700 1 -- 172.21.15.59:0/3198594626 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 8) v1 ==== 89435+0+0 (secure 0 0 0) 0x7f7b20014440 con 0x7f7b1c004850 2023-11-24T07:58:29.159 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:29.154+0000 7f7b27fff700 1 --2- 172.21.15.59:0/3198594626 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f7b1006b110 0x7f7b1006d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:29.159 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:29.154+0000 7f7b27fff700 1 -- 172.21.15.59:0/3198594626 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 5502+0+0 (secure 0 0 0) 0x7f7b20089f70 con 0x7f7b1c004850 2023-11-24T07:58:29.159 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:29.154+0000 7f7b2effd700 1 --2- 172.21.15.59:0/3198594626 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f7b1006b110 0x7f7b1006d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:29.159 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:29.154+0000 7f7b2effd700 1 --2- 172.21.15.59:0/3198594626 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f7b1006b110 0x7f7b1006d5d0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f7b18000c00 tx=0x7f7b18007000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:58:29.159 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:29.158+0000 7f7b27fff700 1 -- 172.21.15.59:0/3198594626 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f7b200bce90 con 0x7f7b1c004850 2023-11-24T07:58:29.362 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:29.361+0000 7f7b35c03700 1 -- 172.21.15.59:0/3198594626 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f7b30068670 con 0x7f7b1c004850 2023-11-24T07:58:29.996 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:29.995+0000 7f7b27fff700 1 -- 172.21.15.59:0/3198594626 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v52) v1 ==== 169+0+0 (secure 0 0 0) 0x7f7b20022020 con 0x7f7b1c004850 2023-11-24T07:58:31.883 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-24T07:58:31.886 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:29.998+0000 7f7b35c03700 1 -- 172.21.15.59:0/3198594626 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f7b1006b110 msgr2=0x7f7b1006d5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:31.886 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:29.998+0000 7f7b35c03700 1 --2- 172.21.15.59:0/3198594626 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f7b1006b110 0x7f7b1006d5d0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f7b18000c00 tx=0x7f7b18007000 comp rx=0 tx=0).stop 2023-11-24T07:58:31.887 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:29.998+0000 7f7b35c03700 1 -- 172.21.15.59:0/3198594626 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7b1c004850 msgr2=0x7f7b300b5d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:31.887 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:29.998+0000 7f7b35c03700 1 --2- 172.21.15.59:0/3198594626 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7b1c004850 0x7f7b300b5d20 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7f7b20011aa0 tx=0x7f7b20011e60 comp rx=0 tx=0).stop 2023-11-24T07:58:31.887 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:29.998+0000 7f7b35c03700 1 -- 172.21.15.59:0/3198594626 shutdown_connections 2023-11-24T07:58:31.887 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:29.998+0000 7f7b35c03700 1 --2- 172.21.15.59:0/3198594626 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f7b1006b110 0x7f7b1006d5d0 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:31.887 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:29.998+0000 7f7b35c03700 1 --2- 172.21.15.59:0/3198594626 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f7b300b2e20 0x7f7b30159cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:31.887 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:29.998+0000 7f7b35c03700 1 --2- 172.21.15.59:0/3198594626 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7b1c004850 0x7f7b300b5d20 unknown :-1 s=CLOSED pgs=653 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:31.887 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:29.998+0000 7f7b35c03700 1 --2- 172.21.15.59:0/3198594626 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7b301f7180 0x7f7b3009ba80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:31.888 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:29.998+0000 7f7b35c03700 1 -- 172.21.15.59:0/3198594626 >> 172.21.15.59:0/3198594626 conn(0x7f7b300b3720 msgr2=0x7f7b300b1f50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:58:31.888 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:29.998+0000 7f7b35c03700 1 -- 172.21.15.59:0/3198594626 shutdown_connections 2023-11-24T07:58:31.888 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:29.998+0000 7f7b35c03700 1 -- 172.21.15.59:0/3198594626 wait complete. 2023-11-24T07:58:31.888 INFO:teuthology.orchestra.run.smithi059.stderr:pool 'cephfs_data' removed 2023-11-24T07:58:31.889 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2023-11-24T07:58:31.929 DEBUG:teuthology.orchestra.run.smithi076:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-24T07:58:32.069 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:32.067+0000 7f0bf8666700 1 Processor -- start 2023-11-24T07:58:32.070 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:32.068+0000 7f0bf8666700 1 -- start start 2023-11-24T07:58:32.070 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:32.069+0000 7f0bf8666700 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f0bf00b12a0 0x7f0bf00b1700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:32.071 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:32.069+0000 7f0bf8666700 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0bf006b9a0 con 0x7f0bf0162b50 2023-11-24T07:58:32.071 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:32.069+0000 7f0bf8666700 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0bf006b070 con 0x7f0bf0162770 2023-11-24T07:58:32.071 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:32.069+0000 7f0bf8666700 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0bf006b350 con 0x7f0bf00b12a0 2023-11-24T07:58:32.071 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:32.069+0000 7f0bf6c03700 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7f0bf0162b50 0x7f0bf0162fe0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:56990/0 (socket says 172.21.15.59:56990) 2023-11-24T07:58:32.071 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:32.069+0000 7f0bf6c03700 1 -- 172.21.15.59:0/559656856 learned_addr learned my addr 172.21.15.59:0/559656856 (peer_addr_for_me v1:172.21.15.59:0/0) 2023-11-24T07:58:32.075 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:32.069+0000 7f0bf5c01700 1 --2- 172.21.15.59:0/559656856 >> v2:172.21.15.76:3300/0 conn(0x7f0bf00b12a0 0x7f0bf00b1700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:32.075 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:32.070+0000 7f0bf5400700 1 -- 172.21.15.59:0/559656856 <== mon.0 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2183240382 0 0) 0x7f0bf006b9a0 con 0x7f0bf0162b50 2023-11-24T07:58:32.075 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:32.070+0000 7f0bf5400700 1 -- 172.21.15.59:0/559656856 --> v1:172.21.15.59:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0bdc0035b0 con 0x7f0bf0162b50 2023-11-24T07:58:32.075 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:32.070+0000 7f0bf5400700 1 -- 172.21.15.59:0/559656856 <== mon.1 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 127656055 0 0) 0x7f0bf006b070 con 0x7f0bf0162770 2023-11-24T07:58:32.075 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:32.070+0000 7f0bf5400700 1 -- 172.21.15.59:0/559656856 --> v1:172.21.15.76:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0bf006b9a0 con 0x7f0bf0162770 2023-11-24T07:58:32.076 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:32.070+0000 7f0bf5400700 1 -- 172.21.15.59:0/559656856 <== mon.0 v1:172.21.15.59:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2261983740 0 0) 0x7f0bdc0035b0 con 0x7f0bf0162b50 2023-11-24T07:58:32.076 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:32.070+0000 7f0bf5400700 1 -- 172.21.15.59:0/559656856 >> v2:172.21.15.76:3300/0 conn(0x7f0bf00b12a0 msgr2=0x7f0bf00b1700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T07:58:32.076 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:32.070+0000 7f0bf5400700 1 --2- 172.21.15.59:0/559656856 >> v2:172.21.15.76:3300/0 conn(0x7f0bf00b12a0 0x7f0bf00b1700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:32.076 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:32.070+0000 7f0bf5400700 1 -- 172.21.15.59:0/559656856 >> v1:172.21.15.76:6789/0 conn(0x7f0bf0162770 legacy=0x7f0bf006bf60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:32.076 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:32.070+0000 7f0bf5400700 1 -- 172.21.15.59:0/559656856 --> v1:172.21.15.59:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0bf006b630 con 0x7f0bf0162b50 2023-11-24T07:58:32.076 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:32.070+0000 7f0bf5400700 1 -- 172.21.15.59:0/559656856 <== mon.0 v1:172.21.15.59:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1312333566 0 0) 0x7f0be80019f0 con 0x7f0bf0162b50 2023-11-24T07:58:32.076 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:32.070+0000 7f0bf5c01700 1 --2- 172.21.15.59:0/559656856 >> v2:172.21.15.76:3300/0 conn(0x7f0bf00b12a0 0x7f0bf00b1700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-24T07:58:32.077 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:32.070+0000 7f0bf5400700 1 -- 172.21.15.59:0/559656856 >> v1:172.21.15.59:6789/0 conn(0x7f0bf0162b50 legacy=0x7f0bf0162fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:32.077 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:32.070+0000 7f0bf5400700 1 --2- 172.21.15.59:0/559656856 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f0bdc003bc0 0x7f0bdc003fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:32.077 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:32.070+0000 7f0bf5400700 1 --2- 172.21.15.59:0/559656856 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0bdc004850 0x7f0bdc004cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:32.077 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:32.070+0000 7f0bf5400700 1 --2- 172.21.15.59:0/559656856 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0bdc005210 0x7f0bdc00ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:32.077 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:32.070+0000 7f0bf5400700 1 -- 172.21.15.59:0/559656856 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0be0001230 con 0x7f0bdc004850 2023-11-24T07:58:32.077 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:32.070+0000 7f0bf5400700 1 -- 172.21.15.59:0/559656856 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0bf006b350 con 0x7f0bdc005210 2023-11-24T07:58:32.077 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:32.070+0000 7f0bf5400700 1 -- 172.21.15.59:0/559656856 --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0bdc00c0e0 con 0x7f0bdc003bc0 2023-11-24T07:58:32.078 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:32.070+0000 7f0bf5c01700 1 --2- 172.21.15.59:0/559656856 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0bdc004850 0x7f0bdc004cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:32.078 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:32.070+0000 7f0bf6c03700 1 --2- 172.21.15.59:0/559656856 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0bdc005210 0x7f0bdc00ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:32.078 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:32.070+0000 7f0bf6402700 1 --2- 172.21.15.59:0/559656856 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f0bdc003bc0 0x7f0bdc003fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:32.078 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:32.071+0000 7f0bf5c01700 1 -- 172.21.15.59:0/559656856 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f0bdc003bc0 msgr2=0x7f0bdc003fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T07:58:32.078 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:32.071+0000 7f0bf5c01700 1 --2- 172.21.15.59:0/559656856 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f0bdc003bc0 0x7f0bdc003fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:32.078 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:32.071+0000 7f0bf5c01700 1 -- 172.21.15.59:0/559656856 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0bdc005210 msgr2=0x7f0bdc00ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T07:58:32.078 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:32.071+0000 7f0bf5c01700 1 --2- 172.21.15.59:0/559656856 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0bdc005210 0x7f0bdc00ba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:32.079 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:32.071+0000 7f0bf5c01700 1 -- 172.21.15.59:0/559656856 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0bdc0035b0 con 0x7f0bdc004850 2023-11-24T07:58:32.079 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:32.071+0000 7f0bf6c03700 1 -- 172.21.15.59:0/559656856 reap_dead start 2023-11-24T07:58:32.079 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:32.071+0000 7f0bf5c01700 1 --2- 172.21.15.59:0/559656856 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0bdc004850 0x7f0bdc004cd0 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7f0bec009d30 tx=0x7f0bdc005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fb1e622323045c7c server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:58:32.079 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:32.071+0000 7f0bf5400700 1 -- 172.21.15.59:0/559656856 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0bec009e80 con 0x7f0bdc004850 2023-11-24T07:58:32.079 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:32.071+0000 7f0bf5400700 1 -- 172.21.15.59:0/559656856 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0bec00e9a0 con 0x7f0bdc004850 2023-11-24T07:58:32.079 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:32.071+0000 7f0bf5400700 1 -- 172.21.15.59:0/559656856 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0bec00ed10 con 0x7f0bdc004850 2023-11-24T07:58:32.079 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:32.072+0000 7f0bf8666700 1 -- 172.21.15.59:0/559656856 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0bdc004850 msgr2=0x7f0bdc004cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:32.080 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:32.072+0000 7f0bf8666700 1 --2- 172.21.15.59:0/559656856 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0bdc004850 0x7f0bdc004cd0 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7f0bec009d30 tx=0x7f0bdc005880 comp rx=0 tx=0).stop 2023-11-24T07:58:32.080 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:32.072+0000 7f0bf8666700 1 -- 172.21.15.59:0/559656856 shutdown_connections 2023-11-24T07:58:32.080 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:32.072+0000 7f0bf8666700 1 --2- 172.21.15.59:0/559656856 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0bdc004850 0x7f0bdc004cd0 unknown :-1 s=CLOSED pgs=655 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:32.080 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:32.072+0000 7f0bf8666700 1 -- 172.21.15.59:0/559656856 >> 172.21.15.59:0/559656856 conn(0x7f0bf00b5070 msgr2=0x7f0bf00b5730 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:58:32.080 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:32.072+0000 7f0bf8666700 1 -- 172.21.15.59:0/559656856 shutdown_connections 2023-11-24T07:58:32.080 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:32.072+0000 7f0bf8666700 1 -- 172.21.15.59:0/559656856 wait complete. 2023-11-24T07:58:32.080 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:32.073+0000 7f0bf8666700 1 Processor -- start 2023-11-24T07:58:32.081 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:32.073+0000 7f0bf8666700 1 -- start start 2023-11-24T07:58:32.086 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:32.074+0000 7f0bf8666700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0bdc004850 0x7f0bf006e750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:32.086 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:32.074+0000 7f0bf8666700 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f0bf006ec90 0x7f0bf01fae60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:32.086 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:32.074+0000 7f0bf8666700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0bf01fb3a0 0x7f0bf01fb830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:32.086 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:32.074+0000 7f0bf8666700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0bf009a650 con 0x7f0bdc004850 2023-11-24T07:58:32.086 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:32.074+0000 7f0bf8666700 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0bf015fe40 con 0x7f0bf01fb3a0 2023-11-24T07:58:32.087 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:32.074+0000 7f0bf8666700 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0bf0160120 con 0x7f0bf006ec90 2023-11-24T07:58:32.087 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:32.074+0000 7f0bf6402700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0bdc004850 0x7f0bf006e750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:32.087 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:32.074+0000 7f0bf6402700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0bdc004850 0x7f0bf006e750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:53524/0 (socket says 172.21.15.59:53524) 2023-11-24T07:58:32.087 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:32.074+0000 7f0bf6402700 1 -- 172.21.15.59:0/3141569694 learned_addr learned my addr 172.21.15.59:0/3141569694 (peer_addr_for_me v2:172.21.15.59:0/0) 2023-11-24T07:58:32.087 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:32.074+0000 7f0bf6c03700 1 --2- 172.21.15.59:0/3141569694 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0bf01fb3a0 0x7f0bf01fb830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:32.087 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:32.074+0000 7f0bf5c01700 1 --2- 172.21.15.59:0/3141569694 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f0bf006ec90 0x7f0bf01fae60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:32.087 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:32.074+0000 7f0bf6402700 1 -- 172.21.15.59:0/3141569694 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f0bf006ec90 msgr2=0x7f0bf01fae60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:32.087 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:32.074+0000 7f0bf6402700 1 --2- 172.21.15.59:0/3141569694 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f0bf006ec90 0x7f0bf01fae60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:32.088 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:32.074+0000 7f0bf6402700 1 -- 172.21.15.59:0/3141569694 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0bf01fb3a0 msgr2=0x7f0bf01fb830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:32.088 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:32.074+0000 7f0bf6402700 1 --2- 172.21.15.59:0/3141569694 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0bf01fb3a0 0x7f0bf01fb830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:32.088 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:32.074+0000 7f0bf6402700 1 -- 172.21.15.59:0/3141569694 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0bf0160400 con 0x7f0bdc004850 2023-11-24T07:58:32.088 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:32.075+0000 7f0bf6402700 1 --2- 172.21.15.59:0/3141569694 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0bdc004850 0x7f0bf006e750 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7f0be000fb60 tx=0x7f0be000ff20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:58:32.088 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:32.075+0000 7f0be6ffd700 1 -- 172.21.15.59:0/3141569694 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0be00052e0 con 0x7f0bdc004850 2023-11-24T07:58:32.088 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:32.075+0000 7f0be6ffd700 1 -- 172.21.15.59:0/3141569694 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0be0005460 con 0x7f0bdc004850 2023-11-24T07:58:32.089 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:32.075+0000 7f0bf8666700 1 -- 172.21.15.59:0/3141569694 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0bf01606e0 con 0x7f0bdc004850 2023-11-24T07:58:32.089 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:32.075+0000 7f0be6ffd700 1 -- 172.21.15.59:0/3141569694 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0be00038c0 con 0x7f0bdc004850 2023-11-24T07:58:32.089 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:32.075+0000 7f0bf8666700 1 -- 172.21.15.59:0/3141569694 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0bf015fa10 con 0x7f0bdc004850 2023-11-24T07:58:32.089 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:32.076+0000 7f0bf8666700 1 -- 172.21.15.59:0/3141569694 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0bf009bf30 con 0x7f0bdc004850 2023-11-24T07:58:32.089 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:32.076+0000 7f0be6ffd700 1 -- 172.21.15.59:0/3141569694 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 8) v1 ==== 89435+0+0 (secure 0 0 0) 0x7f0be0003a20 con 0x7f0bdc004850 2023-11-24T07:58:32.089 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:32.077+0000 7f0be6ffd700 1 --2- 172.21.15.59:0/3141569694 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f0bd406b110 0x7f0bd406d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:32.089 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:32.077+0000 7f0be6ffd700 1 -- 172.21.15.59:0/3141569694 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(52..52 src has 1..52) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f0be0089780 con 0x7f0bdc004850 2023-11-24T07:58:32.090 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:32.077+0000 7f0bf5c01700 1 --2- 172.21.15.59:0/3141569694 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f0bd406b110 0x7f0bd406d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:32.090 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:32.077+0000 7f0bf5c01700 1 --2- 172.21.15.59:0/3141569694 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f0bd406b110 0x7f0bd406d5d0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f0bf009e510 tx=0x7f0bec008030 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:58:32.090 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:32.081+0000 7f0be6ffd700 1 -- 172.21.15.59:0/3141569694 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f0be00547d0 con 0x7f0bdc004850 2023-11-24T07:58:32.285 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:32.284+0000 7f0bf8666700 1 -- 172.21.15.59:0/3141569694 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7f0bf015fba0 con 0x7f0bdc004850 2023-11-24T07:58:32.288 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:32.287+0000 7f0be6ffd700 1 -- 172.21.15.59:0/3141569694 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v13) v1 ==== 104+0+0 (secure 0 0 0) 0x7f0be0057df0 con 0x7f0bdc004850 2023-11-24T07:58:32.291 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:32.289+0000 7f0bf8666700 1 -- 172.21.15.59:0/3141569694 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f0bd406b110 msgr2=0x7f0bd406d5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:32.292 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:32.289+0000 7f0bf8666700 1 --2- 172.21.15.59:0/3141569694 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f0bd406b110 0x7f0bd406d5d0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f0bf009e510 tx=0x7f0bec008030 comp rx=0 tx=0).stop 2023-11-24T07:58:32.292 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:32.289+0000 7f0bf8666700 1 -- 172.21.15.59:0/3141569694 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0bdc004850 msgr2=0x7f0bf006e750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:32.292 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:32.289+0000 7f0bf8666700 1 --2- 172.21.15.59:0/3141569694 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0bdc004850 0x7f0bf006e750 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7f0be000fb60 tx=0x7f0be000ff20 comp rx=0 tx=0).stop 2023-11-24T07:58:32.292 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:32.289+0000 7f0bf8666700 1 -- 172.21.15.59:0/3141569694 shutdown_connections 2023-11-24T07:58:32.394 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:32.289+0000 7f0bf8666700 1 --2- 172.21.15.59:0/3141569694 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f0bd406b110 0x7f0bd406d5d0 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:32.394 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:32.289+0000 7f0bf8666700 1 --2- 172.21.15.59:0/3141569694 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0bf01fb3a0 0x7f0bf01fb830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:32.395 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:32.289+0000 7f0bf8666700 1 --2- 172.21.15.59:0/3141569694 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0bdc004850 0x7f0bf006e750 unknown :-1 s=CLOSED pgs=656 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:32.395 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:32.289+0000 7f0bf8666700 1 --2- 172.21.15.59:0/3141569694 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f0bf006ec90 0x7f0bf01fae60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:32.395 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:32.289+0000 7f0bf8666700 1 -- 172.21.15.59:0/3141569694 >> 172.21.15.59:0/3141569694 conn(0x7f0bf00b5070 msgr2=0x7f0bf006d430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:58:32.395 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:32.289+0000 7f0bf8666700 1 -- 172.21.15.59:0/3141569694 shutdown_connections 2023-11-24T07:58:32.395 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:32.289+0000 7f0bf8666700 1 -- 172.21.15.59:0/3141569694 wait complete. 2023-11-24T07:58:32.396 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_volumes.TestSubvolumeSnapshots.test_clean_stale_subvolume_snapshot_metadata' 2023-11-24T07:58:32.522 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:32.520+0000 7f01e5ca4700 1 Processor -- start 2023-11-24T07:58:32.522 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:32.520+0000 7f01e5ca4700 1 -- start start 2023-11-24T07:58:32.526 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:32.521+0000 7f01e5ca4700 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f01e0143ae0 0x7f01e00b51b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:32.526 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:32.521+0000 7f01e5ca4700 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f01e0143ec0 con 0x7f01e00722c0 2023-11-24T07:58:32.527 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:32.521+0000 7f01e5ca4700 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f01e00b3ae0 con 0x7f01e01440f0 2023-11-24T07:58:32.527 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:32.521+0000 7f01e5ca4700 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f01e00b40a0 con 0x7f01e0143ae0 2023-11-24T07:58:32.527 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:32.521+0000 7f01dffff700 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7f01e00722c0 0x7f01e006e510 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:57010/0 (socket says 172.21.15.59:57010) 2023-11-24T07:58:32.527 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:32.521+0000 7f01dffff700 1 -- 172.21.15.59:0/3143719741 learned_addr learned my addr 172.21.15.59:0/3143719741 (peer_addr_for_me v1:172.21.15.59:0/0) 2023-11-24T07:58:32.527 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:32.521+0000 7f01df7fe700 1 --2- 172.21.15.59:0/3143719741 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f01e0143ae0 0x7f01e00b51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:32.527 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:32.521+0000 7f01de7fc700 1 -- 172.21.15.59:0/3143719741 <== mon.0 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2968040973 0 0) 0x7f01e0143ec0 con 0x7f01e00722c0 2023-11-24T07:58:32.527 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:32.521+0000 7f01de7fc700 1 -- 172.21.15.59:0/3143719741 --> v1:172.21.15.59:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f01cc0035b0 con 0x7f01e00722c0 2023-11-24T07:58:32.527 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:32.522+0000 7f01de7fc700 1 -- 172.21.15.59:0/3143719741 <== mon.1 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2959404877 0 0) 0x7f01e00b3ae0 con 0x7f01e01440f0 2023-11-24T07:58:32.527 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:32.522+0000 7f01de7fc700 1 -- 172.21.15.59:0/3143719741 --> v1:172.21.15.76:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f01e0143ec0 con 0x7f01e01440f0 2023-11-24T07:58:32.528 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:32.522+0000 7f01de7fc700 1 -- 172.21.15.59:0/3143719741 <== mon.0 v1:172.21.15.59:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 918721119 0 0) 0x7f01cc0035b0 con 0x7f01e00722c0 2023-11-24T07:58:32.528 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:32.522+0000 7f01de7fc700 1 -- 172.21.15.59:0/3143719741 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f01e0143ae0 msgr2=0x7f01e00b51b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T07:58:32.528 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:32.522+0000 7f01de7fc700 1 --2- 172.21.15.59:0/3143719741 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f01e0143ae0 0x7f01e00b51b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:32.528 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:32.522+0000 7f01de7fc700 1 -- 172.21.15.59:0/3143719741 >> v1:172.21.15.76:6789/0 conn(0x7f01e01440f0 legacy=0x7f01e0071bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:32.528 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:32.522+0000 7f01de7fc700 1 -- 172.21.15.59:0/3143719741 --> v1:172.21.15.59:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f01e00b3dc0 con 0x7f01e00722c0 2023-11-24T07:58:32.528 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:32.522+0000 7f01de7fc700 1 -- 172.21.15.59:0/3143719741 <== mon.0 v1:172.21.15.59:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1312333566 0 0) 0x7f01d80019f0 con 0x7f01e00722c0 2023-11-24T07:58:32.529 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:32.522+0000 7f01de7fc700 1 -- 172.21.15.59:0/3143719741 >> v1:172.21.15.59:6789/0 conn(0x7f01e00722c0 legacy=0x7f01e006e510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:32.529 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:32.522+0000 7f01de7fc700 1 --2- 172.21.15.59:0/3143719741 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f01cc003bc0 0x7f01cc003fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:32.529 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:32.522+0000 7f01de7fc700 1 --2- 172.21.15.59:0/3143719741 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f01e0143ae0 0x7f01cc004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:32.529 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:32.522+0000 7f01de7fc700 1 --2- 172.21.15.59:0/3143719741 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f01cc004d90 0x7f01cc005210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:32.529 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:32.522+0000 7f01de7fc700 1 -- 172.21.15.59:0/3143719741 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01e00b40a0 con 0x7f01cc003bc0 2023-11-24T07:58:32.529 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:32.522+0000 7f01de7fc700 1 -- 172.21.15.59:0/3143719741 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01cc00bac0 con 0x7f01cc004d90 2023-11-24T07:58:32.529 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:32.522+0000 7f01de7fc700 1 -- 172.21.15.59:0/3143719741 --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f01cc00de40 con 0x7f01e0143ae0 2023-11-24T07:58:32.529 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:32.522+0000 7f01df7fe700 1 --2- 172.21.15.59:0/3143719741 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f01cc003bc0 0x7f01cc003fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:32.529 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:32.522+0000 7f01deffd700 1 --2- 172.21.15.59:0/3143719741 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f01e0143ae0 0x7f01cc004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:32.529 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:32.522+0000 7f01dffff700 1 --2- 172.21.15.59:0/3143719741 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f01cc004d90 0x7f01cc005210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:32.529 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:32.522+0000 7f01df7fe700 1 -- 172.21.15.59:0/3143719741 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f01e0143ae0 msgr2=0x7f01cc004850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T07:58:32.530 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:32.522+0000 7f01df7fe700 1 --2- 172.21.15.59:0/3143719741 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f01e0143ae0 0x7f01cc004850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:32.530 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:32.522+0000 7f01df7fe700 1 -- 172.21.15.59:0/3143719741 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f01cc004d90 msgr2=0x7f01cc005210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T07:58:32.530 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:32.522+0000 7f01df7fe700 1 --2- 172.21.15.59:0/3143719741 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f01cc004d90 0x7f01cc005210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:32.530 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:32.522+0000 7f01df7fe700 1 -- 172.21.15.59:0/3143719741 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f01cc0035b0 con 0x7f01cc003bc0 2023-11-24T07:58:32.530 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:32.522+0000 7f01df7fe700 1 --2- 172.21.15.59:0/3143719741 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f01cc003bc0 0x7f01cc003fc0 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7f01d0009b60 tx=0x7f01cc005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=791c0848da598493 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:58:32.530 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:32.522+0000 7f01de7fc700 1 -- 172.21.15.59:0/3143719741 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01d000e6c0 con 0x7f01cc003bc0 2023-11-24T07:58:32.530 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:32.523+0000 7f01de7fc700 1 -- 172.21.15.59:0/3143719741 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f01d000e820 con 0x7f01cc003bc0 2023-11-24T07:58:32.531 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:32.523+0000 7f01de7fc700 1 -- 172.21.15.59:0/3143719741 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01d000eb90 con 0x7f01cc003bc0 2023-11-24T07:58:32.531 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:32.523+0000 7f01e5ca4700 1 -- 172.21.15.59:0/3143719741 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f01cc003bc0 msgr2=0x7f01cc003fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:32.531 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:32.523+0000 7f01e5ca4700 1 --2- 172.21.15.59:0/3143719741 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f01cc003bc0 0x7f01cc003fc0 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7f01d0009b60 tx=0x7f01cc005880 comp rx=0 tx=0).stop 2023-11-24T07:58:32.531 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:32.523+0000 7f01dffff700 1 -- 172.21.15.59:0/3143719741 reap_dead start 2023-11-24T07:58:32.531 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:32.523+0000 7f01e5ca4700 1 -- 172.21.15.59:0/3143719741 shutdown_connections 2023-11-24T07:58:32.531 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:32.523+0000 7f01e5ca4700 1 -- 172.21.15.59:0/3143719741 >> 172.21.15.59:0/3143719741 conn(0x7f01e00b2740 msgr2=0x7f01e00b2b20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:58:32.531 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:32.523+0000 7f01e5ca4700 1 -- 172.21.15.59:0/3143719741 shutdown_connections 2023-11-24T07:58:32.531 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:32.523+0000 7f01e5ca4700 1 -- 172.21.15.59:0/3143719741 wait complete. 2023-11-24T07:58:32.531 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:32.524+0000 7f01e5ca4700 1 Processor -- start 2023-11-24T07:58:32.531 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:32.525+0000 7f01e5ca4700 1 -- start start 2023-11-24T07:58:32.534 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:32.525+0000 7f01e5ca4700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f01cc003bc0 0x7f01e013f3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:32.535 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:32.525+0000 7f01e5ca4700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f01e0141fe0 0x7f01e0142460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:32.535 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:32.525+0000 7f01e5ca4700 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f01e015b2f0 0x7f01e015b770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:32.535 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:32.525+0000 7f01e5ca4700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01e01429a0 con 0x7f01e0141fe0 2023-11-24T07:58:32.535 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:32.525+0000 7f01e5ca4700 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01e015d0c0 con 0x7f01cc003bc0 2023-11-24T07:58:32.535 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:32.525+0000 7f01e5ca4700 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f01e015c020 con 0x7f01e015b2f0 2023-11-24T07:58:32.535 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:32.525+0000 7f01deffd700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f01e0141fe0 0x7f01e0142460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:32.535 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:32.525+0000 7f01deffd700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f01e0141fe0 0x7f01e0142460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:53546/0 (socket says 172.21.15.59:53546) 2023-11-24T07:58:32.535 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:32.525+0000 7f01deffd700 1 -- 172.21.15.59:0/3269276373 learned_addr learned my addr 172.21.15.59:0/3269276373 (peer_addr_for_me v2:172.21.15.59:0/0) 2023-11-24T07:58:32.535 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:32.525+0000 7f01dffff700 1 --2- 172.21.15.59:0/3269276373 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f01e015b2f0 0x7f01e015b770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:32.536 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:32.525+0000 7f01df7fe700 1 --2- 172.21.15.59:0/3269276373 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f01cc003bc0 0x7f01e013f3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:32.536 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:32.526+0000 7f01deffd700 1 -- 172.21.15.59:0/3269276373 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f01e015b2f0 msgr2=0x7f01e015b770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:32.537 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:32.526+0000 7f01deffd700 1 --2- 172.21.15.59:0/3269276373 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f01e015b2f0 0x7f01e015b770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:32.537 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:32.526+0000 7f01deffd700 1 -- 172.21.15.59:0/3269276373 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f01cc003bc0 msgr2=0x7f01e013f3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:32.537 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:32.526+0000 7f01deffd700 1 --2- 172.21.15.59:0/3269276373 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f01cc003bc0 0x7f01e013f3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:32.538 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:32.526+0000 7f01deffd700 1 -- 172.21.15.59:0/3269276373 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f01e015c300 con 0x7f01e0141fe0 2023-11-24T07:58:32.538 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:32.526+0000 7f01deffd700 1 --2- 172.21.15.59:0/3269276373 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f01e0141fe0 0x7f01e0142460 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7f01c800e7b0 tx=0x7f01c800eac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:58:32.538 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:32.526+0000 7f01d7fff700 1 -- 172.21.15.59:0/3269276373 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01c80052e0 con 0x7f01e0141fe0 2023-11-24T07:58:32.538 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:32.526+0000 7f01d7fff700 1 -- 172.21.15.59:0/3269276373 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f01c8005460 con 0x7f01e0141fe0 2023-11-24T07:58:32.538 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:32.526+0000 7f01d7fff700 1 -- 172.21.15.59:0/3269276373 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01c8006aa0 con 0x7f01e0141fe0 2023-11-24T07:58:32.538 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:32.526+0000 7f01e5ca4700 1 -- 172.21.15.59:0/3269276373 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f01e015c5e0 con 0x7f01e0141fe0 2023-11-24T07:58:32.539 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:32.526+0000 7f01e5ca4700 1 -- 172.21.15.59:0/3269276373 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f01e015cba0 con 0x7f01e0141fe0 2023-11-24T07:58:32.539 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:32.527+0000 7f01e5ca4700 1 -- 172.21.15.59:0/3269276373 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f01e00754d0 con 0x7f01e0141fe0 2023-11-24T07:58:32.539 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:32.527+0000 7f01d7fff700 1 -- 172.21.15.59:0/3269276373 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 8) v1 ==== 89435+0+0 (secure 0 0 0) 0x7f01c8006c00 con 0x7f01e0141fe0 2023-11-24T07:58:32.539 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:32.528+0000 7f01d7fff700 1 --2- 172.21.15.59:0/3269276373 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f01c006b140 0x7f01c006d600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:32.539 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:32.528+0000 7f01d7fff700 1 -- 172.21.15.59:0/3269276373 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(52..52 src has 1..52) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f01c8088680 con 0x7f01e0141fe0 2023-11-24T07:58:32.539 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:32.528+0000 7f01df7fe700 1 --2- 172.21.15.59:0/3269276373 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f01c006b140 0x7f01c006d600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:32.540 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:32.529+0000 7f01df7fe700 1 --2- 172.21.15.59:0/3269276373 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f01c006b140 0x7f01c006d600 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f01d00090b0 tx=0x7f01d001f040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:58:32.540 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:32.532+0000 7f01d7fff700 1 -- 172.21.15.59:0/3269276373 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f01c8053640 con 0x7f01e0141fe0 2023-11-24T07:58:32.821 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:32.820+0000 7f01e5ca4700 1 -- 172.21.15.59:0/3269276373 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_volumes.TestSubvolumeSnapshots.test_clean_stale_subvolume_snapshot_metadata"]} v 0) v1 -- 0x7f01e0068670 con 0x7f01e0141fe0 2023-11-24T07:58:33.333 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:33.332+0000 7f01d7fff700 1 -- 172.21.15.59:0/3269276373 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_volumes.TestSubvolumeSnapshots.test_clean_stale_subvolume_snapshot_metadata"]}]=0 v137) v1 ==== 172+0+0 (secure 0 0 0) 0x7f01c80190f0 con 0x7f01e0141fe0 2023-11-24T07:58:33.337 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:33.334+0000 7f01e5ca4700 1 -- 172.21.15.59:0/3269276373 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f01c006b140 msgr2=0x7f01c006d600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:33.337 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:33.334+0000 7f01e5ca4700 1 --2- 172.21.15.59:0/3269276373 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f01c006b140 0x7f01c006d600 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f01d00090b0 tx=0x7f01d001f040 comp rx=0 tx=0).stop 2023-11-24T07:58:33.337 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:33.334+0000 7f01e5ca4700 1 -- 172.21.15.59:0/3269276373 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f01e0141fe0 msgr2=0x7f01e0142460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:33.337 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:33.334+0000 7f01e5ca4700 1 --2- 172.21.15.59:0/3269276373 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f01e0141fe0 0x7f01e0142460 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7f01c800e7b0 tx=0x7f01c800eac0 comp rx=0 tx=0).stop 2023-11-24T07:58:33.337 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:33.334+0000 7f01e5ca4700 1 -- 172.21.15.59:0/3269276373 shutdown_connections 2023-11-24T07:58:33.337 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:33.334+0000 7f01e5ca4700 1 --2- 172.21.15.59:0/3269276373 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f01c006b140 0x7f01c006d600 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:33.337 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:33.334+0000 7f01e5ca4700 1 --2- 172.21.15.59:0/3269276373 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f01e015b2f0 0x7f01e015b770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:33.337 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:33.334+0000 7f01e5ca4700 1 --2- 172.21.15.59:0/3269276373 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f01cc003bc0 0x7f01e013f3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:33.338 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:33.334+0000 7f01e5ca4700 1 --2- 172.21.15.59:0/3269276373 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f01e0141fe0 0x7f01e0142460 unknown :-1 s=CLOSED pgs=659 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:33.338 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:33.334+0000 7f01e5ca4700 1 -- 172.21.15.59:0/3269276373 >> 172.21.15.59:0/3269276373 conn(0x7f01e00b2740 msgr2=0x7f01e006f4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:58:33.338 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:33.335+0000 7f01e5ca4700 1 -- 172.21.15.59:0/3269276373 shutdown_connections 2023-11-24T07:58:33.338 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:33.335+0000 7f01e5ca4700 1 -- 172.21.15.59:0/3269276373 wait complete. 2023-11-24T07:58:33.350 INFO:tasks.cephfs_test_runner:Validate cleaning of stale subvolume snapshot metadata. ... ok 2023-11-24T07:58:33.351 INFO:tasks.cephfs_test_runner:Starting test: test_nonexistent_subvolume_snapshot_rm (tasks.cephfs.test_volumes.TestSubvolumeSnapshots) 2023-11-24T07:58:33.351 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-24T07:58:33.480 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:33.478+0000 7f17fb6df700 1 Processor -- start 2023-11-24T07:58:33.481 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:33.479+0000 7f17fb6df700 1 -- start start 2023-11-24T07:58:33.483 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:33.479+0000 7f17fb6df700 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f17f4143ae0 0x7f17f40b51b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:33.483 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:33.479+0000 7f17fb6df700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f17f41440f0 0x7f17f4071bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:33.483 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:33.479+0000 7f17fb6df700 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f17f4143ec0 con 0x7f17f40720f0 2023-11-24T07:58:33.483 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:33.479+0000 7f17fb6df700 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f17f40b40a0 con 0x7f17f41440f0 2023-11-24T07:58:33.483 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:33.479+0000 7f17fb6df700 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f17f4072560 con 0x7f17f4143ae0 2023-11-24T07:58:33.484 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:33.480+0000 7f17f8c7a700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f17f41440f0 0x7f17f4071bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:33.484 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:33.480+0000 7f17f8c7a700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f17f41440f0 0x7f17f4071bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:53554/0 (socket says 172.21.15.59:53554) 2023-11-24T07:58:33.484 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:33.480+0000 7f17f8c7a700 1 -- 172.21.15.59:0/1421047792 learned_addr learned my addr 172.21.15.59:0/1421047792 (peer_addr_for_me v2:172.21.15.59:0/0) 2023-11-24T07:58:33.485 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:33.480+0000 7f17f8c7a700 1 -- 172.21.15.59:0/1421047792 >> v2:172.21.15.76:3300/0 conn(0x7f17f4143ae0 msgr2=0x7f17f40b51b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T07:58:33.485 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:33.480+0000 7f17f947b700 1 --2- 172.21.15.59:0/1421047792 >> v2:172.21.15.76:3300/0 conn(0x7f17f4143ae0 0x7f17f40b51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:33.486 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:33.480+0000 7f17f8c7a700 1 --2- 172.21.15.59:0/1421047792 >> v2:172.21.15.76:3300/0 conn(0x7f17f4143ae0 0x7f17f40b51b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:33.486 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:33.480+0000 7f17f8c7a700 1 -- 172.21.15.59:0/1421047792 >> v1:172.21.15.76:6789/0 conn(0x7f17f40720f0 legacy=0x7f17f415e8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T07:58:33.486 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:33.480+0000 7f17f8c7a700 1 -- 172.21.15.59:0/1421047792 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f17f40b3ae0 con 0x7f17f41440f0 2023-11-24T07:58:33.487 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:33.480+0000 7f17f8c7a700 1 --2- 172.21.15.59:0/1421047792 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f17f41440f0 0x7f17f4071bb0 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7f17f406d2e0 tx=0x7f17e8009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ab977bdd1e2ea215 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:58:33.487 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:33.480+0000 7f17e7fff700 1 -- 172.21.15.59:0/1421047792 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17e8004230 con 0x7f17f41440f0 2023-11-24T07:58:33.487 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:33.480+0000 7f17e7fff700 1 -- 172.21.15.59:0/1421047792 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f17e8004390 con 0x7f17f41440f0 2023-11-24T07:58:33.487 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:33.480+0000 7f17e7fff700 1 -- 172.21.15.59:0/1421047792 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17e80036a0 con 0x7f17f41440f0 2023-11-24T07:58:33.487 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:33.481+0000 7f17fb6df700 1 -- 172.21.15.59:0/1421047792 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f17f41440f0 msgr2=0x7f17f4071bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:33.488 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:33.481+0000 7f17fb6df700 1 --2- 172.21.15.59:0/1421047792 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f17f41440f0 0x7f17f4071bb0 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7f17f406d2e0 tx=0x7f17e8009c00 comp rx=0 tx=0).stop 2023-11-24T07:58:33.488 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:33.481+0000 7f17fb6df700 1 -- 172.21.15.59:0/1421047792 shutdown_connections 2023-11-24T07:58:33.488 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:33.481+0000 7f17fb6df700 1 --2- 172.21.15.59:0/1421047792 >> v2:172.21.15.76:3300/0 conn(0x7f17f4143ae0 0x7f17f40b51b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:33.488 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:33.481+0000 7f17fb6df700 1 --2- 172.21.15.59:0/1421047792 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f17f41440f0 0x7f17f4071bb0 unknown :-1 s=CLOSED pgs=660 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:33.488 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:33.481+0000 7f17fb6df700 1 -- 172.21.15.59:0/1421047792 >> 172.21.15.59:0/1421047792 conn(0x7f17f40b2740 msgr2=0x7f17f40b2b20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:58:33.488 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:33.481+0000 7f17fb6df700 1 -- 172.21.15.59:0/1421047792 shutdown_connections 2023-11-24T07:58:33.488 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:33.481+0000 7f17fb6df700 1 -- 172.21.15.59:0/1421047792 wait complete. 2023-11-24T07:58:33.491 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:33.482+0000 7f17fb6df700 1 Processor -- start 2023-11-24T07:58:33.491 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:33.482+0000 7f17fb6df700 1 -- start start 2023-11-24T07:58:33.491 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:33.483+0000 7f17fb6df700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f17f40720f0 0x7f17f4157210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:33.491 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:33.483+0000 7f17fb6df700 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f17f4143ae0 0x7f17f4156700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:33.491 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:33.483+0000 7f17fb6df700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f17f41440f0 0x7f17f41ef300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:33.492 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:33.483+0000 7f17fb6df700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17f4156c40 con 0x7f17f40720f0 2023-11-24T07:58:33.492 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:33.483+0000 7f17fb6df700 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17f4158520 con 0x7f17f41440f0 2023-11-24T07:58:33.492 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:33.483+0000 7f17fb6df700 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f17f41ef840 con 0x7f17f4143ae0 2023-11-24T07:58:33.492 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:33.483+0000 7f17f947b700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f17f40720f0 0x7f17f4157210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:33.492 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:33.483+0000 7f17f947b700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f17f40720f0 0x7f17f4157210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:53558/0 (socket says 172.21.15.59:53558) 2023-11-24T07:58:33.492 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:33.483+0000 7f17f947b700 1 -- 172.21.15.59:0/3177399914 learned_addr learned my addr 172.21.15.59:0/3177399914 (peer_addr_for_me v2:172.21.15.59:0/0) 2023-11-24T07:58:33.492 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:33.483+0000 7f17f9c7c700 1 --2- 172.21.15.59:0/3177399914 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f17f41440f0 0x7f17f41ef300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:33.492 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:33.483+0000 7f17f8c7a700 1 --2- 172.21.15.59:0/3177399914 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f17f4143ae0 0x7f17f4156700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:33.492 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:33.483+0000 7f17f947b700 1 -- 172.21.15.59:0/3177399914 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f17f4143ae0 msgr2=0x7f17f4156700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:33.492 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:33.483+0000 7f17f947b700 1 --2- 172.21.15.59:0/3177399914 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f17f4143ae0 0x7f17f4156700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:33.492 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:33.483+0000 7f17f947b700 1 -- 172.21.15.59:0/3177399914 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f17f41440f0 msgr2=0x7f17f41ef300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:33.492 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:33.483+0000 7f17f947b700 1 --2- 172.21.15.59:0/3177399914 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f17f41440f0 0x7f17f41ef300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:33.493 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:33.483+0000 7f17f947b700 1 -- 172.21.15.59:0/3177399914 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f17e8009550 con 0x7f17f40720f0 2023-11-24T07:58:33.493 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:33.483+0000 7f17f9c7c700 1 --2- 172.21.15.59:0/3177399914 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f17f41440f0 0x7f17f41ef300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T07:58:33.493 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:33.483+0000 7f17f8c7a700 1 --2- 172.21.15.59:0/3177399914 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f17f4143ae0 0x7f17f4156700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T07:58:33.493 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:33.483+0000 7f17f947b700 1 --2- 172.21.15.59:0/3177399914 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f17f40720f0 0x7f17f4157210 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7f17f000c820 tx=0x7f17f000cbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:58:33.493 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:33.484+0000 7f17e5ffb700 1 -- 172.21.15.59:0/3177399914 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17f0011800 con 0x7f17f40720f0 2023-11-24T07:58:33.493 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:33.484+0000 7f17e5ffb700 1 -- 172.21.15.59:0/3177399914 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f17f0011960 con 0x7f17f40720f0 2023-11-24T07:58:33.493 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:33.484+0000 7f17fb6df700 1 -- 172.21.15.59:0/3177399914 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f17f4157910 con 0x7f17f40720f0 2023-11-24T07:58:33.493 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:33.484+0000 7f17e5ffb700 1 -- 172.21.15.59:0/3177399914 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17f0011cf0 con 0x7f17f40720f0 2023-11-24T07:58:33.493 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:33.484+0000 7f17fb6df700 1 -- 172.21.15.59:0/3177399914 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f17f4157ed0 con 0x7f17f40720f0 2023-11-24T07:58:33.493 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:33.485+0000 7f17fb6df700 1 -- 172.21.15.59:0/3177399914 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f17f4158800 con 0x7f17f40720f0 2023-11-24T07:58:33.493 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:33.485+0000 7f17e5ffb700 1 -- 172.21.15.59:0/3177399914 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 8) v1 ==== 89435+0+0 (secure 0 0 0) 0x7f17f0018360 con 0x7f17f40720f0 2023-11-24T07:58:33.494 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:33.486+0000 7f17e5ffb700 1 --2- 172.21.15.59:0/3177399914 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f17d806b110 0x7f17d806d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:33.494 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:33.486+0000 7f17e5ffb700 1 -- 172.21.15.59:0/3177399914 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(52..52 src has 1..52) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f17f0089750 con 0x7f17f40720f0 2023-11-24T07:58:33.494 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:33.486+0000 7f17f8c7a700 1 --2- 172.21.15.59:0/3177399914 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f17d806b110 0x7f17d806d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:33.494 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:33.487+0000 7f17f8c7a700 1 --2- 172.21.15.59:0/3177399914 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f17d806b110 0x7f17d806d5d0 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f17f41371b0 tx=0x7f17e8008030 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:58:33.494 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:33.490+0000 7f17e5ffb700 1 -- 172.21.15.59:0/3177399914 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f17f0054600 con 0x7f17f40720f0 2023-11-24T07:58:33.694 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:33.692+0000 7f17fb6df700 1 -- 172.21.15.59:0/3177399914 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f17f4144830 con 0x7f17f40720f0 2023-11-24T07:58:33.694 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:33.693+0000 7f17e5ffb700 1 -- 172.21.15.59:0/3177399914 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v52) v1 ==== 74+0+12914 (secure 0 0 0) 0x7f17f001f090 con 0x7f17f40720f0 2023-11-24T07:58:33.694 INFO:teuthology.orchestra.run.smithi059.stdout: 2023-11-24T07:58:33.695 INFO:teuthology.orchestra.run.smithi059.stdout:{"epoch":52,"fsid":"9f0e8741-b8f9-48d5-a2c3-09f45903303d","created":"2023-11-24T07:56:06.552729+0000","modified":"2023-11-24T07:58:29.994096+0000","last_up_change":"2023-11-24T07:56:17.372352+0000","last_in_change":"2023-11-24T07:56:09.073250+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-24T07:56:18.352823+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"6d8f9e9b-1782-4e41-928b-a39f2a8ebc06","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6800","nonce":3373878772},{"type":"v1","addr":"172.21.15.59:6801","nonce":3373878772}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6802","nonce":3373878772},{"type":"v1","addr":"172.21.15.59:6803","nonce":3373878772}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6806","nonce":3373878772},{"type":"v1","addr":"172.21.15.59:6807","nonce":3373878772}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6804","nonce":3373878772},{"type":"v1","addr":"172.21.15.59:6805","nonce":3373878772}]},"public_addr":"172.21.15.59:6801/3373878772","cluster_addr":"172.21.15.59:6803/3373878772","heartbeat_back_addr":"172.21.15.59:6807/3373878772","heartbeat_front_addr":"172.21.15.59:6805/3373878772","state":["exists","up"]},{"osd":1,"uuid":"e8d882aa-52e1-4c04-9e1e-c44ef9204a5f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6824","nonce":648258563},{"type":"v1","addr":"172.21.15.59:6825","nonce":648258563}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6826","nonce":648258563},{"type":"v1","addr":"172.21.15.59:6827","nonce":648258563}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6830","nonce":648258563},{"type":"v1","addr":"172.21.15.59:6831","nonce":648258563}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6828","nonce":648258563},{"type":"v1","addr":"172.21.15.59:6829","nonce":648258563}]},"public_addr":"172.21.15.59:6825/648258563","cluster_addr":"172.21.15.59:6827/648258563","heartbeat_back_addr":"172.21.15.59:6831/648258563","heartbeat_front_addr":"172.21.15.59:6829/648258563","state":["exists","up"]},{"osd":2,"uuid":"69eebfa0-695b-4b21-90b9-8a3537168a01","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6808","nonce":1840430654},{"type":"v1","addr":"172.21.15.59:6809","nonce":1840430654}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6810","nonce":1840430654},{"type":"v1","addr":"172.21.15.59:6811","nonce":1840430654}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6814","nonce":1840430654},{"type":"v1","addr":"172.21.15.59:6815","nonce":1840430654}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6812","nonce":1840430654},{"type":"v1","addr":"172.21.15.59:6813","nonce":1840430654}]},"public_addr":"172.21.15.59:6809/1840430654","cluster_addr":"172.21.15.59:6811/1840430654","heartbeat_back_addr":"172.21.15.59:6815/1840430654","heartbeat_front_addr":"172.21.15.59:6813/1840430654","state":["exists","up"]},{"osd":3,"uuid":"96fd77fc-f911-46ff-9445-6f0be2daa8f4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6816","nonce":2504025850},{"type":"v1","addr":"172.21.15.59:6817","nonce":2504025850}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6818","nonce":2504025850},{"type":"v1","addr":"172.21.15.59:6819","nonce":2504025850}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6822","nonce":2504025850},{"type":"v1","addr":"172.21.15.59:6823","nonce":2504025850}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6820","nonce":2504025850},{"type":"v1","addr":"172.21.15.59:6821","nonce":2504025850}]},"public_addr":"172.21.15.59:6817/2504025850","cluster_addr":"172.21.15.59:6819/2504025850","heartbeat_back_addr":"172.21.15.59:6823/2504025850","heartbeat_front_addr":"172.21.15.59:6821/2504025850","state":["exists","up"]},{"osd":4,"uuid":"7ac29fca-b5b2-4c95-9d69-2f164cbbc708","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6816","nonce":2382463431},{"type":"v1","addr":"172.21.15.76:6818","nonce":2382463431}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6820","nonce":2382463431},{"type":"v1","addr":"172.21.15.76:6822","nonce":2382463431}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6828","nonce":2382463431},{"type":"v1","addr":"172.21.15.76:6830","nonce":2382463431}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6824","nonce":2382463431},{"type":"v1","addr":"172.21.15.76:6826","nonce":2382463431}]},"public_addr":"172.21.15.76:6818/2382463431","cluster_addr":"172.21.15.76:6822/2382463431","heartbeat_back_addr":"172.21.15.76:6830/2382463431","heartbeat_front_addr":"172.21.15.76:6826/2382463431","state":["exists","up"]},{"osd":5,"uuid":"485bf711-3851-4238-a4f4-51d52e0a9c58","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6800","nonce":329334308},{"type":"v1","addr":"172.21.15.76:6802","nonce":329334308}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6805","nonce":329334308},{"type":"v1","addr":"172.21.15.76:6807","nonce":329334308}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6814","nonce":329334308},{"type":"v1","addr":"172.21.15.76:6815","nonce":329334308}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6810","nonce":329334308},{"type":"v1","addr":"172.21.15.76:6812","nonce":329334308}]},"public_addr":"172.21.15.76:6802/329334308","cluster_addr":"172.21.15.76:6807/329334308","heartbeat_back_addr":"172.21.15.76:6815/329334308","heartbeat_front_addr":"172.21.15.76:6812/329334308","state":["exists","up"]},{"osd":6,"uuid":"72b3f2fc-1e4a-4e32-968d-003647a0286f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6801","nonce":3946579410},{"type":"v1","addr":"172.21.15.76:6803","nonce":3946579410}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6804","nonce":3946579410},{"type":"v1","addr":"172.21.15.76:6806","nonce":3946579410}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6811","nonce":3946579410},{"type":"v1","addr":"172.21.15.76:6813","nonce":3946579410}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6808","nonce":3946579410},{"type":"v1","addr":"172.21.15.76:6809","nonce":3946579410}]},"public_addr":"172.21.15.76:6803/3946579410","cluster_addr":"172.21.15.76:6806/3946579410","heartbeat_back_addr":"172.21.15.76:6813/3946579410","heartbeat_front_addr":"172.21.15.76:6809/3946579410","state":["exists","up"]},{"osd":7,"uuid":"1ddceb90-6c5d-4d1a-ad56-99cb877f8c70","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6817","nonce":841761538},{"type":"v1","addr":"172.21.15.76:6819","nonce":841761538}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6821","nonce":841761538},{"type":"v1","addr":"172.21.15.76:6823","nonce":841761538}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6829","nonce":841761538},{"type":"v1","addr":"172.21.15.76:6831","nonce":841761538}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6825","nonce":841761538},{"type":"v1","addr":"172.21.15.76:6827","nonce":841761538}]},"public_addr":"172.21.15.76:6819/841761538","cluster_addr":"172.21.15.76:6823/841761538","heartbeat_back_addr":"172.21.15.76:6831/841761538","heartbeat_front_addr":"172.21.15.76:6827/841761538","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T07:56:14.294838+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T07:56:14.270779+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T07:56:14.231809+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T07:56:14.260983+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T07:56:14.302997+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T07:56:14.258600+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T07:56:14.251413+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T07:56:14.304727+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.59:6835/1468266626":"2023-11-25T07:58:26.983333+0000","172.21.15.59:6833/1468266626":"2023-11-25T07:58:26.983333+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-24T07:58:33.697 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:33.695+0000 7f17fb6df700 1 -- 172.21.15.59:0/3177399914 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f17d806b110 msgr2=0x7f17d806d5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:33.697 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:33.695+0000 7f17fb6df700 1 --2- 172.21.15.59:0/3177399914 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f17d806b110 0x7f17d806d5d0 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f17f41371b0 tx=0x7f17e8008030 comp rx=0 tx=0).stop 2023-11-24T07:58:33.697 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:33.695+0000 7f17fb6df700 1 -- 172.21.15.59:0/3177399914 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f17f40720f0 msgr2=0x7f17f4157210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:33.698 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:33.695+0000 7f17fb6df700 1 --2- 172.21.15.59:0/3177399914 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f17f40720f0 0x7f17f4157210 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7f17f000c820 tx=0x7f17f000cbe0 comp rx=0 tx=0).stop 2023-11-24T07:58:33.698 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:33.695+0000 7f17fb6df700 1 -- 172.21.15.59:0/3177399914 shutdown_connections 2023-11-24T07:58:33.698 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:33.695+0000 7f17fb6df700 1 --2- 172.21.15.59:0/3177399914 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f17d806b110 0x7f17d806d5d0 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:33.698 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:33.695+0000 7f17fb6df700 1 --2- 172.21.15.59:0/3177399914 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f17f41440f0 0x7f17f41ef300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:33.698 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:33.695+0000 7f17fb6df700 1 --2- 172.21.15.59:0/3177399914 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f17f40720f0 0x7f17f4157210 unknown :-1 s=CLOSED pgs=661 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:33.698 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:33.695+0000 7f17fb6df700 1 --2- 172.21.15.59:0/3177399914 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f17f4143ae0 0x7f17f4156700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:33.698 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:33.695+0000 7f17fb6df700 1 -- 172.21.15.59:0/3177399914 >> 172.21.15.59:0/3177399914 conn(0x7f17f40b2740 msgr2=0x7f17f413f790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:58:33.698 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:33.695+0000 7f17fb6df700 1 -- 172.21.15.59:0/3177399914 shutdown_connections 2023-11-24T07:58:33.698 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:33.695+0000 7f17fb6df700 1 -- 172.21.15.59:0/3177399914 wait complete. 2023-11-24T07:58:33.709 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-24T07:56:18.352823+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-24T07:58:33.710 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-24T07:58:33.837 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:33.836+0000 7f1f7dc0e700 1 Processor -- start 2023-11-24T07:58:33.838 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:33.836+0000 7f1f7dc0e700 1 -- start start 2023-11-24T07:58:33.841 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:33.837+0000 7f1f7dc0e700 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f1f780d1010 0x7f1f780d1420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:33.842 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:33.837+0000 7f1f7dc0e700 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1f780d0b80 con 0x7f1f780b4090 2023-11-24T07:58:33.842 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:33.837+0000 7f1f7dc0e700 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1f7815ed60 con 0x7f1f78072a00 2023-11-24T07:58:33.842 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:33.837+0000 7f1f7dc0e700 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1f780d1960 con 0x7f1f780d1010 2023-11-24T07:58:33.842 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:33.837+0000 7f1f777fe700 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7f1f780b4090 0x7f1f7815e510 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:57040/0 (socket says 172.21.15.59:57040) 2023-11-24T07:58:33.842 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:33.837+0000 7f1f777fe700 1 -- 172.21.15.59:0/2181115035 learned_addr learned my addr 172.21.15.59:0/2181115035 (peer_addr_for_me v1:172.21.15.59:0/0) 2023-11-24T07:58:33.842 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:33.837+0000 7f1f77fff700 1 --2- 172.21.15.59:0/2181115035 >> v2:172.21.15.76:3300/0 conn(0x7f1f780d1010 0x7f1f780d1420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:33.842 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:33.837+0000 7f1f767fc700 1 -- 172.21.15.59:0/2181115035 <== mon.0 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2859576636 0 0) 0x7f1f780d0b80 con 0x7f1f780b4090 2023-11-24T07:58:33.842 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:33.837+0000 7f1f767fc700 1 -- 172.21.15.59:0/2181115035 --> v1:172.21.15.59:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1f600035b0 con 0x7f1f780b4090 2023-11-24T07:58:33.842 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:33.837+0000 7f1f767fc700 1 -- 172.21.15.59:0/2181115035 <== mon.0 v1:172.21.15.59:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2306928424 0 0) 0x7f1f600035b0 con 0x7f1f780b4090 2023-11-24T07:58:33.842 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:33.837+0000 7f1f767fc700 1 -- 172.21.15.59:0/2181115035 >> v2:172.21.15.76:3300/0 conn(0x7f1f780d1010 msgr2=0x7f1f780d1420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T07:58:33.843 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:33.837+0000 7f1f767fc700 1 --2- 172.21.15.59:0/2181115035 >> v2:172.21.15.76:3300/0 conn(0x7f1f780d1010 0x7f1f780d1420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:33.843 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:33.837+0000 7f1f767fc700 1 -- 172.21.15.59:0/2181115035 >> v1:172.21.15.76:6789/0 conn(0x7f1f78072a00 legacy=0x7f1f7806db60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:33.843 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:33.837+0000 7f1f767fc700 1 -- 172.21.15.59:0/2181115035 --> v1:172.21.15.59:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1f7815f040 con 0x7f1f780b4090 2023-11-24T07:58:33.843 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:33.837+0000 7f1f767fc700 1 -- 172.21.15.59:0/2181115035 <== mon.0 v1:172.21.15.59:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1312333566 0 0) 0x7f1f68001670 con 0x7f1f780b4090 2023-11-24T07:58:33.843 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:33.837+0000 7f1f767fc700 1 -- 172.21.15.59:0/2181115035 >> v1:172.21.15.59:6789/0 conn(0x7f1f780b4090 legacy=0x7f1f7815e510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:33.843 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:33.837+0000 7f1f767fc700 1 --2- 172.21.15.59:0/2181115035 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1f60003a00 0x7f1f60003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:33.843 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:33.837+0000 7f1f767fc700 1 --2- 172.21.15.59:0/2181115035 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1f60004850 0x7f1f60004cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:33.843 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:33.837+0000 7f1f767fc700 1 --2- 172.21.15.59:0/2181115035 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f1f60005210 0x7f1f6000ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:33.843 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:33.837+0000 7f1f767fc700 1 -- 172.21.15.59:0/2181115035 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f780d1960 con 0x7f1f60003a00 2023-11-24T07:58:33.843 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:33.837+0000 7f1f767fc700 1 -- 172.21.15.59:0/2181115035 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f6000c1b0 con 0x7f1f60004850 2023-11-24T07:58:33.843 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:33.837+0000 7f1f767fc700 1 -- 172.21.15.59:0/2181115035 --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1f6000c370 con 0x7f1f60005210 2023-11-24T07:58:33.843 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:33.837+0000 7f1f777fe700 1 --2- 172.21.15.59:0/2181115035 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1f60003a00 0x7f1f60003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:33.844 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:33.838+0000 7f1f76ffd700 1 --2- 172.21.15.59:0/2181115035 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1f60004850 0x7f1f60004cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:33.844 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:33.838+0000 7f1f77fff700 1 --2- 172.21.15.59:0/2181115035 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f1f60005210 0x7f1f6000ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:33.844 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:33.838+0000 7f1f777fe700 1 -- 172.21.15.59:0/2181115035 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f1f60005210 msgr2=0x7f1f6000ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T07:58:33.844 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:33.838+0000 7f1f777fe700 1 --2- 172.21.15.59:0/2181115035 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f1f60005210 0x7f1f6000ba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:33.844 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:33.838+0000 7f1f777fe700 1 -- 172.21.15.59:0/2181115035 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1f60004850 msgr2=0x7f1f60004cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T07:58:33.844 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:33.838+0000 7f1f777fe700 1 --2- 172.21.15.59:0/2181115035 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1f60004850 0x7f1f60004cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:33.844 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:33.838+0000 7f1f777fe700 1 -- 172.21.15.59:0/2181115035 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1f600035b0 con 0x7f1f60003a00 2023-11-24T07:58:33.844 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:33.838+0000 7f1f77fff700 1 --2- 172.21.15.59:0/2181115035 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f1f60005210 0x7f1f6000ba10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T07:58:33.844 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:33.838+0000 7f1f76ffd700 1 --2- 172.21.15.59:0/2181115035 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1f60004850 0x7f1f60004cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-24T07:58:33.844 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:33.838+0000 7f1f77fff700 1 -- 172.21.15.59:0/2181115035 reap_dead start 2023-11-24T07:58:33.844 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:33.838+0000 7f1f777fe700 1 --2- 172.21.15.59:0/2181115035 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1f60003a00 0x7f1f60003e20 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7f1f6800c240 tx=0x7f1f60005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8742b4e8bbdd9afe server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:58:33.845 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:33.838+0000 7f1f767fc700 1 -- 172.21.15.59:0/2181115035 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f68007c10 con 0x7f1f60003a00 2023-11-24T07:58:33.845 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:33.838+0000 7f1f767fc700 1 -- 172.21.15.59:0/2181115035 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1f6800ce00 con 0x7f1f60003a00 2023-11-24T07:58:33.845 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:33.838+0000 7f1f767fc700 1 -- 172.21.15.59:0/2181115035 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f68005ba0 con 0x7f1f60003a00 2023-11-24T07:58:33.845 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:33.838+0000 7f1f7dc0e700 1 -- 172.21.15.59:0/2181115035 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1f60003a00 msgr2=0x7f1f60003e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:33.845 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:33.838+0000 7f1f7dc0e700 1 --2- 172.21.15.59:0/2181115035 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1f60003a00 0x7f1f60003e20 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7f1f6800c240 tx=0x7f1f60005880 comp rx=0 tx=0).stop 2023-11-24T07:58:33.845 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:33.839+0000 7f1f7dc0e700 1 -- 172.21.15.59:0/2181115035 shutdown_connections 2023-11-24T07:58:33.845 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:33.839+0000 7f1f7dc0e700 1 --2- 172.21.15.59:0/2181115035 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1f60003a00 0x7f1f60003e20 unknown :-1 s=CLOSED pgs=663 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:33.845 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:33.839+0000 7f1f7dc0e700 1 -- 172.21.15.59:0/2181115035 >> 172.21.15.59:0/2181115035 conn(0x7f1f780b2740 msgr2=0x7f1f780b2b50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:58:33.845 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:33.839+0000 7f1f7dc0e700 1 -- 172.21.15.59:0/2181115035 shutdown_connections 2023-11-24T07:58:33.845 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:33.839+0000 7f1f7dc0e700 1 -- 172.21.15.59:0/2181115035 wait complete. 2023-11-24T07:58:33.846 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:33.840+0000 7f1f7dc0e700 1 Processor -- start 2023-11-24T07:58:33.846 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:33.840+0000 7f1f7dc0e700 1 -- start start 2023-11-24T07:58:33.849 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:33.841+0000 7f1f7dc0e700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1f60003a00 0x7f1f78160910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:33.849 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:33.841+0000 7f1f7dc0e700 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f1f78160e50 0x7f1f78145080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:33.849 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:33.841+0000 7f1f7dc0e700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1f781455c0 0x7f1f780ae8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:33.849 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:33.841+0000 7f1f7dc0e700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f7815b190 con 0x7f1f60003a00 2023-11-24T07:58:33.849 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:33.841+0000 7f1f7dc0e700 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f78158b40 con 0x7f1f781455c0 2023-11-24T07:58:33.849 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:33.841+0000 7f1f7dc0e700 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1f781612d0 con 0x7f1f78160e50 2023-11-24T07:58:33.849 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:33.841+0000 7f1f777fe700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1f60003a00 0x7f1f78160910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:33.849 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:33.841+0000 7f1f777fe700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1f60003a00 0x7f1f78160910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:53578/0 (socket says 172.21.15.59:53578) 2023-11-24T07:58:33.849 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:33.841+0000 7f1f777fe700 1 -- 172.21.15.59:0/401862196 learned_addr learned my addr 172.21.15.59:0/401862196 (peer_addr_for_me v2:172.21.15.59:0/0) 2023-11-24T07:58:33.850 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:33.841+0000 7f1f77fff700 1 --2- 172.21.15.59:0/401862196 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1f781455c0 0x7f1f780ae8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:33.850 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:33.841+0000 7f1f76ffd700 1 --2- 172.21.15.59:0/401862196 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f1f78160e50 0x7f1f78145080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:33.850 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:33.841+0000 7f1f777fe700 1 -- 172.21.15.59:0/401862196 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f1f78160e50 msgr2=0x7f1f78145080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:33.850 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:33.841+0000 7f1f777fe700 1 --2- 172.21.15.59:0/401862196 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f1f78160e50 0x7f1f78145080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:33.850 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:33.841+0000 7f1f777fe700 1 -- 172.21.15.59:0/401862196 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1f781455c0 msgr2=0x7f1f780ae8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:33.850 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:33.841+0000 7f1f777fe700 1 --2- 172.21.15.59:0/401862196 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1f781455c0 0x7f1f780ae8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:33.850 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:33.841+0000 7f1f777fe700 1 -- 172.21.15.59:0/401862196 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1f78158de0 con 0x7f1f60003a00 2023-11-24T07:58:33.850 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:33.841+0000 7f1f76ffd700 1 --2- 172.21.15.59:0/401862196 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f1f78160e50 0x7f1f78145080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-24T07:58:33.850 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:33.841+0000 7f1f777fe700 1 --2- 172.21.15.59:0/401862196 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1f60003a00 0x7f1f78160910 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7f1f6000c180 tx=0x7f1f68001a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:58:33.850 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:33.841+0000 7f1f5ffff700 1 -- 172.21.15.59:0/401862196 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f68001670 con 0x7f1f60003a00 2023-11-24T07:58:33.851 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:33.842+0000 7f1f5ffff700 1 -- 172.21.15.59:0/401862196 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1f680070d0 con 0x7f1f60003a00 2023-11-24T07:58:33.851 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:33.842+0000 7f1f5ffff700 1 -- 172.21.15.59:0/401862196 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f68007480 con 0x7f1f60003a00 2023-11-24T07:58:33.851 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:33.842+0000 7f1f7dc0e700 1 -- 172.21.15.59:0/401862196 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1f78145e30 con 0x7f1f60003a00 2023-11-24T07:58:33.851 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:33.842+0000 7f1f7dc0e700 1 -- 172.21.15.59:0/401862196 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1f781463f0 con 0x7f1f60003a00 2023-11-24T07:58:33.851 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:33.842+0000 7f1f7dc0e700 1 -- 172.21.15.59:0/401862196 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1f781465b0 con 0x7f1f60003a00 2023-11-24T07:58:33.851 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:33.843+0000 7f1f5ffff700 1 -- 172.21.15.59:0/401862196 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 8) v1 ==== 89435+0+0 (secure 0 0 0) 0x7f1f680075e0 con 0x7f1f60003a00 2023-11-24T07:58:33.851 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:33.844+0000 7f1f5ffff700 1 --2- 172.21.15.59:0/401862196 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f1f6406b110 0x7f1f6406d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:33.851 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:33.844+0000 7f1f5ffff700 1 -- 172.21.15.59:0/401862196 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(52..52 src has 1..52) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f1f6808a840 con 0x7f1f60003a00 2023-11-24T07:58:33.851 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:33.844+0000 7f1f76ffd700 1 --2- 172.21.15.59:0/401862196 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f1f6406b110 0x7f1f6406d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:33.851 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:33.844+0000 7f1f76ffd700 1 --2- 172.21.15.59:0/401862196 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f1f6406b110 0x7f1f6406d5d0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f1f6c000c00 tx=0x7f1f6c009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:58:33.852 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:33.848+0000 7f1f5ffff700 1 -- 172.21.15.59:0/401862196 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f1f680558a0 con 0x7f1f60003a00 2023-11-24T07:58:34.049 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:34.047+0000 7f1f7dc0e700 1 -- 172.21.15.59:0/401862196 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f1f78075480 con 0x7f1f60003a00 2023-11-24T07:58:34.049 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:34.047+0000 7f1f5ffff700 1 -- 172.21.15.59:0/401862196 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v52) v1 ==== 93+0+10 (secure 0 0 0) 0x7f1f68058ec0 con 0x7f1f60003a00 2023-11-24T07:58:34.049 INFO:teuthology.orchestra.run.smithi059.stdout:pg_num: 1 2023-11-24T07:58:34.051 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:34.049+0000 7f1f7dc0e700 1 -- 172.21.15.59:0/401862196 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f1f6406b110 msgr2=0x7f1f6406d5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:34.051 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:34.049+0000 7f1f7dc0e700 1 --2- 172.21.15.59:0/401862196 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f1f6406b110 0x7f1f6406d5d0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f1f6c000c00 tx=0x7f1f6c009040 comp rx=0 tx=0).stop 2023-11-24T07:58:34.052 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:34.049+0000 7f1f7dc0e700 1 -- 172.21.15.59:0/401862196 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1f60003a00 msgr2=0x7f1f78160910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:34.052 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:34.049+0000 7f1f7dc0e700 1 --2- 172.21.15.59:0/401862196 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1f60003a00 0x7f1f78160910 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7f1f6000c180 tx=0x7f1f68001a00 comp rx=0 tx=0).stop 2023-11-24T07:58:34.052 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:34.049+0000 7f1f7dc0e700 1 -- 172.21.15.59:0/401862196 shutdown_connections 2023-11-24T07:58:34.052 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:34.049+0000 7f1f7dc0e700 1 --2- 172.21.15.59:0/401862196 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f1f6406b110 0x7f1f6406d5d0 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:34.052 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:34.049+0000 7f1f7dc0e700 1 --2- 172.21.15.59:0/401862196 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1f781455c0 0x7f1f780ae8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:34.052 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:34.049+0000 7f1f7dc0e700 1 --2- 172.21.15.59:0/401862196 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1f60003a00 0x7f1f78160910 unknown :-1 s=CLOSED pgs=664 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:34.053 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:34.049+0000 7f1f7dc0e700 1 --2- 172.21.15.59:0/401862196 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f1f78160e50 0x7f1f78145080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:34.053 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:34.049+0000 7f1f7dc0e700 1 -- 172.21.15.59:0/401862196 >> 172.21.15.59:0/401862196 conn(0x7f1f780b2740 msgr2=0x7f1f7806ece0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:58:34.053 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:34.049+0000 7f1f7dc0e700 1 -- 172.21.15.59:0/401862196 shutdown_connections 2023-11-24T07:58:34.053 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:34.050+0000 7f1f7dc0e700 1 -- 172.21.15.59:0/401862196 wait complete. 2023-11-24T07:58:34.064 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_volumes.TestSubvolumeSnapshots.test_nonexistent_subvolume_snapshot_rm' 2023-11-24T07:58:34.189 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:34.187+0000 7f83eae16700 1 Processor -- start 2023-11-24T07:58:34.189 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:34.187+0000 7f83eae16700 1 -- start start 2023-11-24T07:58:34.192 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:34.188+0000 7f83eae16700 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f83e4144c10 0x7f83e406afd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:34.192 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:34.188+0000 7f83eae16700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f83e4145650 0x7f83e406b510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:34.192 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:34.188+0000 7f83eae16700 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f83e406eb80 con 0x7f83e4145100 2023-11-24T07:58:34.192 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:34.188+0000 7f83eae16700 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f83e406f500 con 0x7f83e4145650 2023-11-24T07:58:34.192 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:34.188+0000 7f83eae16700 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f83e40cd610 con 0x7f83e4144c10 2023-11-24T07:58:34.192 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:34.188+0000 7f83e3fff700 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7f83e4145100 0x7f83e4071bb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:57060/0 (socket says 172.21.15.59:57060) 2023-11-24T07:58:34.192 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:34.188+0000 7f83e93b3700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f83e4145650 0x7f83e406b510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:34.193 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:34.188+0000 7f83e3fff700 1 -- 172.21.15.59:0/3074222188 learned_addr learned my addr 172.21.15.59:0/3074222188 (peer_addr_for_me v1:172.21.15.59:0/0) 2023-11-24T07:58:34.193 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:34.188+0000 7f83e8bb2700 1 --2- 172.21.15.59:0/3074222188 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f83e4144c10 0x7f83e406afd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:34.193 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:34.188+0000 7f83e37fe700 1 -- 172.21.15.59:0/3074222188 <== mon.0 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 790830586 0 0) 0x7f83e406eb80 con 0x7f83e4145100 2023-11-24T07:58:34.193 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:34.188+0000 7f83e37fe700 1 -- 172.21.15.59:0/3074222188 --> v1:172.21.15.59:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f83d40035b0 con 0x7f83e4145100 2023-11-24T07:58:34.193 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:34.189+0000 7f83e93b3700 1 -- 172.21.15.59:0/3074222188 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f83e4144c10 msgr2=0x7f83e406afd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T07:58:34.194 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:34.189+0000 7f83e93b3700 1 --2- 172.21.15.59:0/3074222188 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f83e4144c10 0x7f83e406afd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:34.194 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:34.189+0000 7f83e93b3700 1 -- 172.21.15.59:0/3074222188 >> v1:172.21.15.59:6789/0 conn(0x7f83e4145100 legacy=0x7f83e4071bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:34.194 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:34.189+0000 7f83e93b3700 1 -- 172.21.15.59:0/3074222188 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f83e4160720 con 0x7f83e4145650 2023-11-24T07:58:34.194 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:34.189+0000 7f83e93b3700 1 --2- 172.21.15.59:0/3074222188 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f83e4145650 0x7f83e406b510 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7f83dc009d90 tx=0x7f83dc00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bdf3f88540b4f720 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:58:34.194 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:34.189+0000 7f83e37fe700 1 -- 172.21.15.59:0/3074222188 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83dc00e070 con 0x7f83e4145650 2023-11-24T07:58:34.195 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:34.189+0000 7f83e37fe700 1 -- 172.21.15.59:0/3074222188 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f83dc00bb20 con 0x7f83e4145650 2023-11-24T07:58:34.195 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:34.189+0000 7f83e37fe700 1 -- 172.21.15.59:0/3074222188 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83dc012430 con 0x7f83e4145650 2023-11-24T07:58:34.195 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:34.189+0000 7f83eae16700 1 -- 172.21.15.59:0/3074222188 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f83e4145650 msgr2=0x7f83e406b510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:34.195 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:34.189+0000 7f83eae16700 1 --2- 172.21.15.59:0/3074222188 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f83e4145650 0x7f83e406b510 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7f83dc009d90 tx=0x7f83dc00b330 comp rx=0 tx=0).stop 2023-11-24T07:58:34.195 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:34.189+0000 7f83eae16700 1 -- 172.21.15.59:0/3074222188 shutdown_connections 2023-11-24T07:58:34.195 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:34.189+0000 7f83eae16700 1 --2- 172.21.15.59:0/3074222188 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f83e4145650 0x7f83e406b510 unknown :-1 s=CLOSED pgs=666 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:34.195 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:34.189+0000 7f83eae16700 1 --2- 172.21.15.59:0/3074222188 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f83e4144c10 0x7f83e406afd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:34.195 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:34.189+0000 7f83eae16700 1 -- 172.21.15.59:0/3074222188 >> 172.21.15.59:0/3074222188 conn(0x7f83e40b2740 msgr2=0x7f83e40b2b20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:58:34.196 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:34.190+0000 7f83eae16700 1 -- 172.21.15.59:0/3074222188 shutdown_connections 2023-11-24T07:58:34.196 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:34.190+0000 7f83eae16700 1 -- 172.21.15.59:0/3074222188 wait complete. 2023-11-24T07:58:34.198 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:34.191+0000 7f83eae16700 1 Processor -- start 2023-11-24T07:58:34.199 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:34.191+0000 7f83eae16700 1 -- start start 2023-11-24T07:58:34.199 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:34.192+0000 7f83eae16700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f83e4144c10 0x7f83e415d560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:34.199 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:34.192+0000 7f83eae16700 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f83e4145100 0x7f83e41f0db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:34.199 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:34.192+0000 7f83eae16700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f83e4145650 0x7f83e40af7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:34.199 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:34.192+0000 7f83eae16700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83e415daa0 con 0x7f83e4144c10 2023-11-24T07:58:34.199 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:34.192+0000 7f83eae16700 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83e41f2630 con 0x7f83e4145650 2023-11-24T07:58:34.199 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:34.192+0000 7f83eae16700 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f83e4142280 con 0x7f83e4145100 2023-11-24T07:58:34.199 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:34.192+0000 7f83e8bb2700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f83e4144c10 0x7f83e415d560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:34.200 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:34.192+0000 7f83e8bb2700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f83e4144c10 0x7f83e415d560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:53592/0 (socket says 172.21.15.59:53592) 2023-11-24T07:58:34.200 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:34.192+0000 7f83e8bb2700 1 -- 172.21.15.59:0/3331179793 learned_addr learned my addr 172.21.15.59:0/3331179793 (peer_addr_for_me v2:172.21.15.59:0/0) 2023-11-24T07:58:34.200 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:34.192+0000 7f83e3fff700 1 --2- 172.21.15.59:0/3331179793 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f83e4145100 0x7f83e41f0db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:34.200 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:34.192+0000 7f83e93b3700 1 --2- 172.21.15.59:0/3331179793 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f83e4145650 0x7f83e40af7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:34.200 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:34.192+0000 7f83e8bb2700 1 -- 172.21.15.59:0/3331179793 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f83e4145100 msgr2=0x7f83e41f0db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:34.200 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:34.192+0000 7f83e8bb2700 1 --2- 172.21.15.59:0/3331179793 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f83e4145100 0x7f83e41f0db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:34.200 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:34.192+0000 7f83e8bb2700 1 -- 172.21.15.59:0/3331179793 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f83e4145650 msgr2=0x7f83e40af7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:34.200 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:34.192+0000 7f83e8bb2700 1 --2- 172.21.15.59:0/3331179793 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f83e4145650 0x7f83e40af7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:34.201 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:34.192+0000 7f83e8bb2700 1 -- 172.21.15.59:0/3331179793 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f83dc009a70 con 0x7f83e4144c10 2023-11-24T07:58:34.201 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:34.192+0000 7f83e8bb2700 1 --2- 172.21.15.59:0/3331179793 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f83e4144c10 0x7f83e415d560 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7f83d800e9b0 tx=0x7f83d800ed70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:58:34.201 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:34.192+0000 7f83e17fa700 1 -- 172.21.15.59:0/3331179793 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83d8009710 con 0x7f83e4144c10 2023-11-24T07:58:34.201 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:34.192+0000 7f83e17fa700 1 -- 172.21.15.59:0/3331179793 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f83d800c850 con 0x7f83e4144c10 2023-11-24T07:58:34.201 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:34.192+0000 7f83eae16700 1 -- 172.21.15.59:0/3331179793 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f83e4142560 con 0x7f83e4144c10 2023-11-24T07:58:34.201 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:34.192+0000 7f83e17fa700 1 -- 172.21.15.59:0/3331179793 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83d800cbc0 con 0x7f83e4144c10 2023-11-24T07:58:34.201 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:34.192+0000 7f83eae16700 1 -- 172.21.15.59:0/3331179793 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f83e4142b20 con 0x7f83e4144c10 2023-11-24T07:58:34.201 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:34.193+0000 7f83eae16700 1 -- 172.21.15.59:0/3331179793 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f83e4068670 con 0x7f83e4144c10 2023-11-24T07:58:34.202 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:34.194+0000 7f83e17fa700 1 -- 172.21.15.59:0/3331179793 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 8) v1 ==== 89435+0+0 (secure 0 0 0) 0x7f83d800cd20 con 0x7f83e4144c10 2023-11-24T07:58:34.202 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:34.194+0000 7f83e17fa700 1 --2- 172.21.15.59:0/3331179793 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f83c806b110 0x7f83c806d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:34.202 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:34.194+0000 7f83e17fa700 1 -- 172.21.15.59:0/3331179793 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(52..52 src has 1..52) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f83d8089770 con 0x7f83e4144c10 2023-11-24T07:58:34.202 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:34.195+0000 7f83e3fff700 1 --2- 172.21.15.59:0/3331179793 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f83c806b110 0x7f83c806d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:34.202 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:34.195+0000 7f83e3fff700 1 --2- 172.21.15.59:0/3331179793 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f83c806b110 0x7f83c806d5d0 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f83e40b04c0 tx=0x7f83d0009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:58:34.203 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:34.199+0000 7f83e17fa700 1 -- 172.21.15.59:0/3331179793 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f83d8054850 con 0x7f83e4144c10 2023-11-24T07:58:34.491 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:34.489+0000 7f83eae16700 1 -- 172.21.15.59:0/3331179793 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_volumes.TestSubvolumeSnapshots.test_nonexistent_subvolume_snapshot_rm"]} v 0) v1 -- 0x7f83e40d0840 con 0x7f83e4144c10 2023-11-24T07:58:35.340 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:35.338+0000 7f83e17fa700 1 -- 172.21.15.59:0/3331179793 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_volumes.TestSubvolumeSnapshots.test_nonexistent_subvolume_snapshot_rm"]}]=0 v139) v1 ==== 169+0+0 (secure 0 0 0) 0x7f83d8057e70 con 0x7f83e4144c10 2023-11-24T07:58:35.343 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:35.341+0000 7f83eae16700 1 -- 172.21.15.59:0/3331179793 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f83c806b110 msgr2=0x7f83c806d5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:35.343 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:35.341+0000 7f83eae16700 1 --2- 172.21.15.59:0/3331179793 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f83c806b110 0x7f83c806d5d0 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f83e40b04c0 tx=0x7f83d0009040 comp rx=0 tx=0).stop 2023-11-24T07:58:35.343 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:35.341+0000 7f83eae16700 1 -- 172.21.15.59:0/3331179793 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f83e4144c10 msgr2=0x7f83e415d560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:35.343 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:35.341+0000 7f83eae16700 1 --2- 172.21.15.59:0/3331179793 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f83e4144c10 0x7f83e415d560 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7f83d800e9b0 tx=0x7f83d800ed70 comp rx=0 tx=0).stop 2023-11-24T07:58:35.343 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:35.341+0000 7f83eae16700 1 -- 172.21.15.59:0/3331179793 shutdown_connections 2023-11-24T07:58:35.344 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:35.341+0000 7f83eae16700 1 --2- 172.21.15.59:0/3331179793 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f83c806b110 0x7f83c806d5d0 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:35.344 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:35.341+0000 7f83eae16700 1 --2- 172.21.15.59:0/3331179793 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f83e4145650 0x7f83e40af7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:35.344 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:35.341+0000 7f83eae16700 1 --2- 172.21.15.59:0/3331179793 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f83e4144c10 0x7f83e415d560 unknown :-1 s=CLOSED pgs=667 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:35.344 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:35.341+0000 7f83eae16700 1 --2- 172.21.15.59:0/3331179793 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f83e4145100 0x7f83e41f0db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:35.344 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:35.341+0000 7f83eae16700 1 -- 172.21.15.59:0/3331179793 >> 172.21.15.59:0/3331179793 conn(0x7f83e40b2740 msgr2=0x7f83e40ce3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:58:35.344 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:35.341+0000 7f83eae16700 1 -- 172.21.15.59:0/3331179793 shutdown_connections 2023-11-24T07:58:35.344 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:35.341+0000 7f83eae16700 1 -- 172.21.15.59:0/3331179793 wait complete. 2023-11-24T07:58:35.361 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2023-11-24T07:58:35.496 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:35.494+0000 7f95e9bc2700 1 Processor -- start 2023-11-24T07:58:35.496 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:35.494+0000 7f95e9bc2700 1 -- start start 2023-11-24T07:58:35.498 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:35.495+0000 7f95e9bc2700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f95e415f380 0x7f95e4066ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:35.498 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:35.495+0000 7f95e9bc2700 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f95e415f760 con 0x7f95e40b5eb0 2023-11-24T07:58:35.498 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:35.495+0000 7f95e9bc2700 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f95e406b2f0 con 0x7f95e406c670 2023-11-24T07:58:35.498 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:35.495+0000 7f95e9bc2700 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f95e406b5b0 con 0x7f95e415f380 2023-11-24T07:58:35.499 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:35.495+0000 7f95e37fe700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f95e415f380 0x7f95e4066ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:35.499 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:35.495+0000 7f95e2ffd700 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7f95e40b5eb0 0x7f95e406bf60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:57074/0 (socket says 172.21.15.59:57074) 2023-11-24T07:58:35.499 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:35.495+0000 7f95e2ffd700 1 -- 172.21.15.59:0/3510505077 learned_addr learned my addr 172.21.15.59:0/3510505077 (peer_addr_for_me v1:172.21.15.59:0/0) 2023-11-24T07:58:35.499 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:35.495+0000 7f95e37fe700 1 -- 172.21.15.59:0/3510505077 >> v1:172.21.15.76:6789/0 conn(0x7f95e406c670 legacy=0x7f95e40aed60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T07:58:35.499 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:35.495+0000 7f95e37fe700 1 -- 172.21.15.59:0/3510505077 >> v1:172.21.15.59:6789/0 conn(0x7f95e40b5eb0 legacy=0x7f95e406bf60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:35.499 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:35.495+0000 7f95e37fe700 1 -- 172.21.15.59:0/3510505077 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f95e406b890 con 0x7f95e415f380 2023-11-24T07:58:35.499 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:35.495+0000 7f95e27fc700 1 -- 172.21.15.59:0/3510505077 <== mon.0 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3646708790 0 0) 0x7f95e415f760 con 0x7f95e40b5eb0 2023-11-24T07:58:35.499 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:35.496+0000 7f95e37fe700 1 --2- 172.21.15.59:0/3510505077 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f95e415f380 0x7f95e4066ce0 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x7f95d4009840 tx=0x7f95d4009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=139e7aa8c49445a1 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:58:35.499 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:35.496+0000 7f95e27fc700 1 -- 172.21.15.59:0/3510505077 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95d4004230 con 0x7f95e415f380 2023-11-24T07:58:35.499 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:35.496+0000 7f95e27fc700 1 -- 172.21.15.59:0/3510505077 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f95d4004390 con 0x7f95e415f380 2023-11-24T07:58:35.499 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:35.496+0000 7f95e27fc700 1 -- 172.21.15.59:0/3510505077 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95d40036a0 con 0x7f95e415f380 2023-11-24T07:58:35.500 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:35.496+0000 7f95e9bc2700 1 -- 172.21.15.59:0/3510505077 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f95e415f380 msgr2=0x7f95e4066ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:35.500 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:35.496+0000 7f95e9bc2700 1 --2- 172.21.15.59:0/3510505077 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f95e415f380 0x7f95e4066ce0 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x7f95d4009840 tx=0x7f95d4009b50 comp rx=0 tx=0).stop 2023-11-24T07:58:35.500 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:35.496+0000 7f95e9bc2700 1 -- 172.21.15.59:0/3510505077 shutdown_connections 2023-11-24T07:58:35.500 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:35.496+0000 7f95e9bc2700 1 --2- 172.21.15.59:0/3510505077 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f95e415f380 0x7f95e4066ce0 unknown :-1 s=CLOSED pgs=669 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:35.500 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:35.496+0000 7f95e9bc2700 1 -- 172.21.15.59:0/3510505077 >> 172.21.15.59:0/3510505077 conn(0x7f95e40b5070 msgr2=0x7f95e40b5730 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:58:35.500 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:35.496+0000 7f95e9bc2700 1 -- 172.21.15.59:0/3510505077 shutdown_connections 2023-11-24T07:58:35.500 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:35.496+0000 7f95e9bc2700 1 -- 172.21.15.59:0/3510505077 wait complete. 2023-11-24T07:58:35.502 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:35.498+0000 7f95e9bc2700 1 Processor -- start 2023-11-24T07:58:35.502 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:35.498+0000 7f95e9bc2700 1 -- start start 2023-11-24T07:58:35.502 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:35.498+0000 7f95e9bc2700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f95e406c670 0x7f95e4157080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:35.503 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:35.498+0000 7f95e9bc2700 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f95e40b5eb0 0x7f95e41f3dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:35.503 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:35.499+0000 7f95e9bc2700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f95e415f380 0x7f95e41589f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:35.503 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:35.499+0000 7f95e9bc2700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95e406d7d0 con 0x7f95e415f380 2023-11-24T07:58:35.503 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:35.499+0000 7f95e9bc2700 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95e41575c0 con 0x7f95e406c670 2023-11-24T07:58:35.503 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:35.499+0000 7f95e9bc2700 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f95e40ca0d0 con 0x7f95e40b5eb0 2023-11-24T07:58:35.503 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:35.499+0000 7f95e3fff700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f95e415f380 0x7f95e41589f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:35.503 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:35.499+0000 7f95e3fff700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f95e415f380 0x7f95e41589f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:53608/0 (socket says 172.21.15.59:53608) 2023-11-24T07:58:35.503 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:35.499+0000 7f95e3fff700 1 -- 172.21.15.59:0/3091957907 learned_addr learned my addr 172.21.15.59:0/3091957907 (peer_addr_for_me v2:172.21.15.59:0/0) 2023-11-24T07:58:35.503 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:35.499+0000 7f95e2ffd700 1 --2- 172.21.15.59:0/3091957907 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f95e40b5eb0 0x7f95e41f3dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:35.503 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:35.499+0000 7f95e37fe700 1 --2- 172.21.15.59:0/3091957907 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f95e406c670 0x7f95e4157080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:35.503 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:35.499+0000 7f95e3fff700 1 -- 172.21.15.59:0/3091957907 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f95e40b5eb0 msgr2=0x7f95e41f3dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:35.503 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:35.499+0000 7f95e3fff700 1 --2- 172.21.15.59:0/3091957907 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f95e40b5eb0 0x7f95e41f3dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:35.504 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:35.499+0000 7f95e3fff700 1 -- 172.21.15.59:0/3091957907 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f95e406c670 msgr2=0x7f95e4157080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:35.504 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:35.499+0000 7f95e3fff700 1 --2- 172.21.15.59:0/3091957907 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f95e406c670 0x7f95e4157080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:35.504 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:35.499+0000 7f95e3fff700 1 -- 172.21.15.59:0/3091957907 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f95d40094a0 con 0x7f95e415f380 2023-11-24T07:58:35.504 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:35.499+0000 7f95e37fe700 1 --2- 172.21.15.59:0/3091957907 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f95e406c670 0x7f95e4157080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-24T07:58:35.504 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:35.499+0000 7f95e2ffd700 1 --2- 172.21.15.59:0/3091957907 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f95e40b5eb0 0x7f95e41f3dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-24T07:58:35.504 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:35.499+0000 7f95e3fff700 1 --2- 172.21.15.59:0/3091957907 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f95e415f380 0x7f95e41589f0 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x7f95dc00cba0 tx=0x7f95dc00cf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:58:35.504 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:35.499+0000 7f95dbfff700 1 -- 172.21.15.59:0/3091957907 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95dc00ab80 con 0x7f95e415f380 2023-11-24T07:58:35.504 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:35.499+0000 7f95dbfff700 1 -- 172.21.15.59:0/3091957907 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f95dc00ace0 con 0x7f95e415f380 2023-11-24T07:58:35.504 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:35.499+0000 7f95dbfff700 1 -- 172.21.15.59:0/3091957907 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95dc012660 con 0x7f95e415f380 2023-11-24T07:58:35.504 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:35.499+0000 7f95e9bc2700 1 -- 172.21.15.59:0/3091957907 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f95e41f4cc0 con 0x7f95e415f380 2023-11-24T07:58:35.504 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:35.499+0000 7f95e9bc2700 1 -- 172.21.15.59:0/3091957907 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f95e41f49e0 con 0x7f95e415f380 2023-11-24T07:58:35.505 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:35.500+0000 7f95e9bc2700 1 -- 172.21.15.59:0/3091957907 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f95e406d940 con 0x7f95e415f380 2023-11-24T07:58:35.505 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:35.501+0000 7f95dbfff700 1 -- 172.21.15.59:0/3091957907 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 8) v1 ==== 89435+0+0 (secure 0 0 0) 0x7f95dc0127c0 con 0x7f95e415f380 2023-11-24T07:58:35.505 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:35.501+0000 7f95dbfff700 1 --2- 172.21.15.59:0/3091957907 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f95c406b110 0x7f95c406d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:35.505 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:35.501+0000 7f95dbfff700 1 -- 172.21.15.59:0/3091957907 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(52..52 src has 1..52) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f95dc089f70 con 0x7f95e415f380 2023-11-24T07:58:35.505 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:35.501+0000 7f95e37fe700 1 --2- 172.21.15.59:0/3091957907 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f95c406b110 0x7f95c406d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:35.506 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:35.502+0000 7f95e37fe700 1 --2- 172.21.15.59:0/3091957907 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f95c406b110 0x7f95c406d5d0 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f95d40090b0 tx=0x7f95d4008030 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:58:35.507 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:35.506+0000 7f95dbfff700 1 -- 172.21.15.59:0/3091957907 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f95dc0bce90 con 0x7f95e415f380 2023-11-24T07:58:35.711 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:35.710+0000 7f95e9bc2700 1 -- 172.21.15.59:0/3091957907 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7f95e4075480 con 0x7f95e415f380 2023-11-24T07:58:35.714 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:35.713+0000 7f95dbfff700 1 -- 172.21.15.59:0/3091957907 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v14)=0 v14) v1 ==== 122+0+0 (secure 0 0 0) 0x7f95dc0160c0 con 0x7f95e415f380 2023-11-24T07:58:35.717 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:35.715+0000 7f95e9bc2700 1 -- 172.21.15.59:0/3091957907 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f95c406b110 msgr2=0x7f95c406d5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:35.717 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:35.715+0000 7f95e9bc2700 1 --2- 172.21.15.59:0/3091957907 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f95c406b110 0x7f95c406d5d0 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f95d40090b0 tx=0x7f95d4008030 comp rx=0 tx=0).stop 2023-11-24T07:58:35.717 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:35.715+0000 7f95e9bc2700 1 -- 172.21.15.59:0/3091957907 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f95e415f380 msgr2=0x7f95e41589f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:35.717 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:35.715+0000 7f95e9bc2700 1 --2- 172.21.15.59:0/3091957907 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f95e415f380 0x7f95e41589f0 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x7f95dc00cba0 tx=0x7f95dc00cf60 comp rx=0 tx=0).stop 2023-11-24T07:58:35.717 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:35.715+0000 7f95e9bc2700 1 -- 172.21.15.59:0/3091957907 shutdown_connections 2023-11-24T07:58:35.717 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:35.715+0000 7f95e9bc2700 1 --2- 172.21.15.59:0/3091957907 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f95c406b110 0x7f95c406d5d0 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:35.717 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:35.715+0000 7f95e9bc2700 1 --2- 172.21.15.59:0/3091957907 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f95e415f380 0x7f95e41589f0 unknown :-1 s=CLOSED pgs=670 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:35.718 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:35.715+0000 7f95e9bc2700 1 --2- 172.21.15.59:0/3091957907 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f95e406c670 0x7f95e4157080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:35.718 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:35.715+0000 7f95e9bc2700 1 --2- 172.21.15.59:0/3091957907 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f95e40b5eb0 0x7f95e41f3dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:35.718 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:35.715+0000 7f95e9bc2700 1 -- 172.21.15.59:0/3091957907 >> 172.21.15.59:0/3091957907 conn(0x7f95e40b5070 msgr2=0x7f95e415e1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:58:35.718 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:35.715+0000 7f95e9bc2700 1 -- 172.21.15.59:0/3091957907 shutdown_connections 2023-11-24T07:58:35.718 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:35.715+0000 7f95e9bc2700 1 -- 172.21.15.59:0/3091957907 wait complete. 2023-11-24T07:58:35.729 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-11-24T07:58:35.729 DEBUG:teuthology.orchestra.run.smithi059:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-11-24T07:58:35.768 DEBUG:teuthology.orchestra.run.smithi076:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-11-24T07:58:35.812 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-11-24T07:58:35.813 DEBUG:teuthology.orchestra.run.smithi059:> set -ex 2023-11-24T07:58:35.813 DEBUG:teuthology.orchestra.run.smithi059:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-24T07:58:35.831 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=16287364k,nr_inodes=4071841,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 tmpfs /sys/fs/cgroup tmpfs ro,seclabel,nosuid,nodev,noexec,mode=755 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,seclabel,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,seclabel,nosuid,nodev,noexec,relatime,hugetlb 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,seclabel,nosuid,nodev,noexec,relatime,perf_event 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,seclabel,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/rdma cgroup rw,seclabel,nosuid,nodev,noexec,relatime,rdma 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,seclabel,nosuid,nodev,noexec,relatime,freezer 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,seclabel,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,seclabel,nosuid,nodev,noexec,relatime,devices 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,blkio 0 0 none /sys/kernel/tracing tracefs rw,seclabel,relatime 0 0 configfs /sys/kernel/config configfs rw,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=42,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=16075 0 0 mqueue /dev/mqueue mqueue rw,seclabel,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,relatime 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3261172k,mode=700,uid=1000,gid=1265 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw,seclabel 0 0 2023-11-24T07:58:35.831 DEBUG:teuthology.orchestra.run.smithi076:> set -ex 2023-11-24T07:58:35.831 DEBUG:teuthology.orchestra.run.smithi076:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-24T07:58:35.850 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=16284272k,nr_inodes=4071068,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 tmpfs /sys/fs/cgroup tmpfs ro,seclabel,nosuid,nodev,noexec,mode=755 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,seclabel,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,seclabel,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,seclabel,nosuid,nodev,noexec,relatime,freezer 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,seclabel,nosuid,nodev,noexec,relatime,perf_event 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,seclabel,nosuid,nodev,noexec,relatime,hugetlb 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,seclabel,nosuid,nodev,noexec,relatime,devices 0 0 cgroup /sys/fs/cgroup/rdma cgroup rw,seclabel,nosuid,nodev,noexec,relatime,rdma 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,seclabel,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,blkio 0 0 none /sys/kernel/tracing tracefs rw,seclabel,relatime 0 0 configfs /sys/kernel/config configfs rw,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=35,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=20116 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,relatime 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3260552k,mode=700,uid=1000,gid=1265 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw,seclabel 0 0 2023-11-24T07:58:35.851 DEBUG:teuthology.orchestra.run.smithi059:> set -ex 2023-11-24T07:58:35.851 DEBUG:teuthology.orchestra.run.smithi059:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-24T07:58:35.888 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=16287364k,nr_inodes=4071841,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 tmpfs /sys/fs/cgroup tmpfs ro,seclabel,nosuid,nodev,noexec,mode=755 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,seclabel,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,seclabel,nosuid,nodev,noexec,relatime,hugetlb 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,seclabel,nosuid,nodev,noexec,relatime,perf_event 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,seclabel,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/rdma cgroup rw,seclabel,nosuid,nodev,noexec,relatime,rdma 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,seclabel,nosuid,nodev,noexec,relatime,freezer 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,seclabel,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,seclabel,nosuid,nodev,noexec,relatime,devices 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,blkio 0 0 none /sys/kernel/tracing tracefs rw,seclabel,relatime 0 0 configfs /sys/kernel/config configfs rw,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=42,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=16075 0 0 mqueue /dev/mqueue mqueue rw,seclabel,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,relatime 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3261172k,mode=700,uid=1000,gid=1265 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw,seclabel 0 0 2023-11-24T07:58:35.888 DEBUG:teuthology.orchestra.run.smithi076:> set -ex 2023-11-24T07:58:35.889 DEBUG:teuthology.orchestra.run.smithi076:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-24T07:58:35.909 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=16284272k,nr_inodes=4071068,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 tmpfs /sys/fs/cgroup tmpfs ro,seclabel,nosuid,nodev,noexec,mode=755 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,seclabel,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,seclabel,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,seclabel,nosuid,nodev,noexec,relatime,freezer 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,seclabel,nosuid,nodev,noexec,relatime,perf_event 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,seclabel,nosuid,nodev,noexec,relatime,hugetlb 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,seclabel,nosuid,nodev,noexec,relatime,devices 0 0 cgroup /sys/fs/cgroup/rdma cgroup rw,seclabel,nosuid,nodev,noexec,relatime,rdma 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,seclabel,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,blkio 0 0 none /sys/kernel/tracing tracefs rw,seclabel,relatime 0 0 configfs /sys/kernel/config configfs rw,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=35,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=20116 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,relatime 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3260552k,mode=700,uid=1000,gid=1265 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw,seclabel 0 0 2023-11-24T07:58:35.910 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2023-11-24T07:58:35.910 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-24T07:58:36.053 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:36.051+0000 7f6817c91700 1 Processor -- start 2023-11-24T07:58:36.053 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:36.051+0000 7f6817c91700 1 -- start start 2023-11-24T07:58:36.056 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:36.052+0000 7f6817c91700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f6810143ae0 0x7f68100b51b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:36.056 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:36.052+0000 7f6817c91700 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f68101440f0 0x7f6810071bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:36.056 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:36.052+0000 7f6817c91700 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6810143ec0 con 0x7f68100720f0 2023-11-24T07:58:36.056 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:36.052+0000 7f6817c91700 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f68100b40a0 con 0x7f6810143ae0 2023-11-24T07:58:36.056 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:36.052+0000 7f6817c91700 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6810072580 con 0x7f68101440f0 2023-11-24T07:58:36.057 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:36.052+0000 7f6815a2d700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f6810143ae0 0x7f68100b51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:36.057 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:36.052+0000 7f6815a2d700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f6810143ae0 0x7f68100b51b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:53612/0 (socket says 172.21.15.59:53612) 2023-11-24T07:58:36.057 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:36.052+0000 7f6815a2d700 1 -- 172.21.15.59:0/2400521226 learned_addr learned my addr 172.21.15.59:0/2400521226 (peer_addr_for_me v2:172.21.15.59:0/0) 2023-11-24T07:58:36.057 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:36.053+0000 7f681522c700 1 --2- 172.21.15.59:0/2400521226 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f68101440f0 0x7f6810071bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:36.057 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:36.053+0000 7f6815a2d700 1 -- 172.21.15.59:0/2400521226 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f68101440f0 msgr2=0x7f6810071bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T07:58:36.058 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:36.053+0000 7f6815a2d700 1 --2- 172.21.15.59:0/2400521226 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f68101440f0 0x7f6810071bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:36.058 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:36.053+0000 7f6815a2d700 1 -- 172.21.15.59:0/2400521226 >> v1:172.21.15.76:6789/0 conn(0x7f68100720f0 legacy=0x7f681015e8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T07:58:36.058 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:36.053+0000 7f6815a2d700 1 -- 172.21.15.59:0/2400521226 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f68100b3ae0 con 0x7f6810143ae0 2023-11-24T07:58:36.058 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:36.053+0000 7f6815a2d700 1 --2- 172.21.15.59:0/2400521226 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6810143ae0 0x7f68100b51b0 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7f680c0098f0 tx=0x7f680c009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f2871c47be8acfb9 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:58:36.058 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:36.053+0000 7f6814a2b700 1 -- 172.21.15.59:0/2400521226 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f680c004230 con 0x7f6810143ae0 2023-11-24T07:58:36.058 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:36.053+0000 7f6814a2b700 1 -- 172.21.15.59:0/2400521226 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f680c004390 con 0x7f6810143ae0 2023-11-24T07:58:36.059 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:36.053+0000 7f6814a2b700 1 -- 172.21.15.59:0/2400521226 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f680c0036a0 con 0x7f6810143ae0 2023-11-24T07:58:36.059 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:36.054+0000 7f6817c91700 1 -- 172.21.15.59:0/2400521226 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6810143ae0 msgr2=0x7f68100b51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:36.059 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:36.054+0000 7f6817c91700 1 --2- 172.21.15.59:0/2400521226 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6810143ae0 0x7f68100b51b0 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7f680c0098f0 tx=0x7f680c009c00 comp rx=0 tx=0).stop 2023-11-24T07:58:36.059 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:36.054+0000 7f6817c91700 1 -- 172.21.15.59:0/2400521226 shutdown_connections 2023-11-24T07:58:36.059 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:36.054+0000 7f6817c91700 1 --2- 172.21.15.59:0/2400521226 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6810143ae0 0x7f68100b51b0 unknown :-1 s=CLOSED pgs=671 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:36.059 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:36.054+0000 7f6817c91700 1 --2- 172.21.15.59:0/2400521226 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f68101440f0 0x7f6810071bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:36.060 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:36.054+0000 7f6817c91700 1 -- 172.21.15.59:0/2400521226 >> 172.21.15.59:0/2400521226 conn(0x7f68100b2740 msgr2=0x7f68100b2b20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:58:36.060 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:36.054+0000 7f6817c91700 1 -- 172.21.15.59:0/2400521226 shutdown_connections 2023-11-24T07:58:36.060 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:36.054+0000 7f6817c91700 1 -- 172.21.15.59:0/2400521226 wait complete. 2023-11-24T07:58:36.063 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:36.055+0000 7f6817c91700 1 Processor -- start 2023-11-24T07:58:36.063 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:36.055+0000 7f6817c91700 1 -- start start 2023-11-24T07:58:36.063 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:36.056+0000 7f6817c91700 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f68100720f0 0x7f6810142b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:36.063 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:36.056+0000 7f6817c91700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6810143ae0 0x7f681015e750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:36.064 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:36.056+0000 7f6817c91700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f68101440f0 0x7f681015ec90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:36.064 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:36.056+0000 7f6817c91700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6810141a70 con 0x7f68101440f0 2023-11-24T07:58:36.064 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:36.056+0000 7f6817c91700 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6810141d50 con 0x7f6810143ae0 2023-11-24T07:58:36.064 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:36.056+0000 7f6817c91700 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6810142030 con 0x7f68100720f0 2023-11-24T07:58:36.064 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:36.056+0000 7f681622e700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f68101440f0 0x7f681015ec90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:36.064 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:36.056+0000 7f681622e700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f68101440f0 0x7f681015ec90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:53622/0 (socket says 172.21.15.59:53622) 2023-11-24T07:58:36.065 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:36.056+0000 7f681622e700 1 -- 172.21.15.59:0/158197276 learned_addr learned my addr 172.21.15.59:0/158197276 (peer_addr_for_me v2:172.21.15.59:0/0) 2023-11-24T07:58:36.065 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:36.056+0000 7f6815a2d700 1 --2- 172.21.15.59:0/158197276 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f68100720f0 0x7f6810142b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:36.065 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:36.056+0000 7f681522c700 1 --2- 172.21.15.59:0/158197276 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6810143ae0 0x7f681015e750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:36.065 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:36.056+0000 7f681622e700 1 -- 172.21.15.59:0/158197276 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f68100720f0 msgr2=0x7f6810142b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:36.065 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:36.056+0000 7f681622e700 1 --2- 172.21.15.59:0/158197276 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f68100720f0 0x7f6810142b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:36.066 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:36.056+0000 7f681622e700 1 -- 172.21.15.59:0/158197276 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6810143ae0 msgr2=0x7f681015e750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:36.066 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:36.056+0000 7f681622e700 1 --2- 172.21.15.59:0/158197276 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6810143ae0 0x7f681015e750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:36.066 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:36.056+0000 7f681622e700 1 -- 172.21.15.59:0/158197276 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f680c009550 con 0x7f68101440f0 2023-11-24T07:58:36.066 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:36.056+0000 7f681622e700 1 --2- 172.21.15.59:0/158197276 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f68101440f0 0x7f681015ec90 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7f680800b970 tx=0x7f680800bc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:58:36.066 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:36.057+0000 7f68027fc700 1 -- 172.21.15.59:0/158197276 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f680800d860 con 0x7f68101440f0 2023-11-24T07:58:36.067 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:36.057+0000 7f68027fc700 1 -- 172.21.15.59:0/158197276 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f680800d9c0 con 0x7f68101440f0 2023-11-24T07:58:36.067 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:36.057+0000 7f6817c91700 1 -- 172.21.15.59:0/158197276 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6810142310 con 0x7f68101440f0 2023-11-24T07:58:36.067 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:36.057+0000 7f68027fc700 1 -- 172.21.15.59:0/158197276 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f680800dd70 con 0x7f68101440f0 2023-11-24T07:58:36.067 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:36.057+0000 7f6817c91700 1 -- 172.21.15.59:0/158197276 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f681015deb0 con 0x7f68101440f0 2023-11-24T07:58:36.067 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:36.058+0000 7f6817c91700 1 -- 172.21.15.59:0/158197276 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6810068670 con 0x7f68101440f0 2023-11-24T07:58:36.067 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:36.058+0000 7f68027fc700 1 -- 172.21.15.59:0/158197276 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 8) v1 ==== 89435+0+0 (secure 0 0 0) 0x7f680800e430 con 0x7f68101440f0 2023-11-24T07:58:36.068 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:36.059+0000 7f68027fc700 1 --2- 172.21.15.59:0/158197276 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f67f406b110 0x7f67f406d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:36.068 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:36.059+0000 7f68027fc700 1 -- 172.21.15.59:0/158197276 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(52..52 src has 1..52) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f6808089980 con 0x7f68101440f0 2023-11-24T07:58:36.068 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:36.059+0000 7f6815a2d700 1 --2- 172.21.15.59:0/158197276 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f67f406b110 0x7f67f406d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:36.068 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:36.060+0000 7f6815a2d700 1 --2- 172.21.15.59:0/158197276 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f67f406b110 0x7f67f406d5d0 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f680c009130 tx=0x7f680c008030 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:58:36.069 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:36.063+0000 7f68027fc700 1 -- 172.21.15.59:0/158197276 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f68080548b0 con 0x7f68101440f0 2023-11-24T07:58:36.311 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:36.309+0000 7f6817c91700 1 -- 172.21.15.59:0/158197276 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f6810067a00 con 0x7f68101440f0 2023-11-24T07:58:36.311 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:36.309+0000 7f68027fc700 1 -- 172.21.15.59:0/158197276 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) v1 ==== 94+0+3687 (secure 0 0 0) 0x7f680800ded0 con 0x7f68101440f0 2023-11-24T07:58:36.312 INFO:teuthology.orchestra.run.smithi059.stdout: 2023-11-24T07:58:36.312 INFO:teuthology.orchestra.run.smithi059.stdout:{"epoch":28,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5290,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.76:6836/508095662","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6834","nonce":508095662},{"type":"v1","addr":"172.21.15.76:6836","nonce":508095662}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":20},{"gid":5293,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.76:6837/4158118811","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6835","nonce":4158118811},{"type":"v1","addr":"172.21.15.76:6837","nonce":4158118811}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":20},{"gid":5622,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6834/59936286","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6832","nonce":59936286},{"type":"v1","addr":"172.21.15.59:6834","nonce":59936286}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":20},{"gid":6012,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6835/343131217","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6833","nonce":343131217},{"type":"v1","addr":"172.21.15.59:6835","nonce":343131217}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":28}],"filesystems":[]} 2023-11-24T07:58:36.314 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:36.312+0000 7f6817c91700 1 -- 172.21.15.59:0/158197276 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f67f406b110 msgr2=0x7f67f406d5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:36.314 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:36.312+0000 7f6817c91700 1 --2- 172.21.15.59:0/158197276 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f67f406b110 0x7f67f406d5d0 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f680c009130 tx=0x7f680c008030 comp rx=0 tx=0).stop 2023-11-24T07:58:36.314 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:36.312+0000 7f6817c91700 1 -- 172.21.15.59:0/158197276 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f68101440f0 msgr2=0x7f681015ec90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:36.314 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:36.312+0000 7f6817c91700 1 --2- 172.21.15.59:0/158197276 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f68101440f0 0x7f681015ec90 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7f680800b970 tx=0x7f680800bc80 comp rx=0 tx=0).stop 2023-11-24T07:58:36.314 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:36.312+0000 7f6817c91700 1 -- 172.21.15.59:0/158197276 shutdown_connections 2023-11-24T07:58:36.314 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:36.312+0000 7f6817c91700 1 --2- 172.21.15.59:0/158197276 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f67f406b110 0x7f67f406d5d0 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:36.314 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:36.312+0000 7f6817c91700 1 --2- 172.21.15.59:0/158197276 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f68101440f0 0x7f681015ec90 unknown :-1 s=CLOSED pgs=672 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:36.314 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:36.312+0000 7f6817c91700 1 --2- 172.21.15.59:0/158197276 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f68100720f0 0x7f6810142b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:36.314 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:36.312+0000 7f6817c91700 1 --2- 172.21.15.59:0/158197276 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6810143ae0 0x7f681015e750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:36.315 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:36.312+0000 7f6817c91700 1 -- 172.21.15.59:0/158197276 >> 172.21.15.59:0/158197276 conn(0x7f68100b2740 msgr2=0x7f6810154a50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:58:36.315 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:36.312+0000 7f6817c91700 1 -- 172.21.15.59:0/158197276 shutdown_connections 2023-11-24T07:58:36.315 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:36.312+0000 7f6817c91700 1 -- 172.21.15.59:0/158197276 wait complete. 2023-11-24T07:58:36.315 INFO:teuthology.orchestra.run.smithi059.stderr:dumped fsmap epoch 28 2023-11-24T07:58:36.326 INFO:tasks.ceph.mds.a:Restarting daemon 2023-11-24T07:58:36.326 INFO:tasks.ceph.mds.a:Stopping old one... 2023-11-24T07:58:36.326 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2023-11-24T07:58:36.326 INFO:teuthology.orchestra.run:waiting for 300 2023-11-24T07:58:36.326 INFO:tasks.ceph.mds.c:Restarting daemon 2023-11-24T07:58:36.327 INFO:tasks.ceph.mds.c:Stopping old one... 2023-11-24T07:58:36.327 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2023-11-24T07:58:36.327 INFO:teuthology.orchestra.run:waiting for 300 2023-11-24T07:58:36.327 INFO:tasks.ceph.mds.b:Restarting daemon 2023-11-24T07:58:36.327 INFO:tasks.ceph.mds.b:Stopping old one... 2023-11-24T07:58:36.327 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2023-11-24T07:58:36.327 INFO:teuthology.orchestra.run:waiting for 300 2023-11-24T07:58:36.328 INFO:tasks.ceph.mds.d:Restarting daemon 2023-11-24T07:58:36.328 INFO:tasks.ceph.mds.d:Stopping old one... 2023-11-24T07:58:36.328 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2023-11-24T07:58:36.328 INFO:teuthology.orchestra.run:waiting for 300 2023-11-24T07:58:36.336 INFO:tasks.ceph.mds.a:Stopped 2023-11-24T07:58:36.336 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2023-11-24T07:58:36.338 INFO:tasks.ceph.mds.c:Stopped 2023-11-24T07:58:36.338 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2023-11-24T07:58:36.339 INFO:tasks.ceph.mds.b:Stopped 2023-11-24T07:58:36.339 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2023-11-24T07:58:36.340 INFO:tasks.ceph.mds.d:Stopped 2023-11-24T07:58:36.340 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2023-11-24T07:58:36.379 INFO:tasks.ceph.mds.a:Started 2023-11-24T07:58:36.379 DEBUG:teuthology.parallel:result is None 2023-11-24T07:58:36.380 INFO:tasks.ceph.mds.c:Started 2023-11-24T07:58:36.380 DEBUG:teuthology.parallel:result is None 2023-11-24T07:58:36.382 INFO:tasks.ceph.mds.b:Started 2023-11-24T07:58:36.383 DEBUG:teuthology.parallel:result is None 2023-11-24T07:58:36.383 INFO:tasks.ceph.mds.d:Started 2023-11-24T07:58:36.383 DEBUG:teuthology.parallel:result is None 2023-11-24T07:58:36.384 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2023-11-24T07:58:36.461 INFO:tasks.ceph.mds.a.smithi059.stdout:starting mds.a at 2023-11-24T07:58:36.462 INFO:tasks.ceph.mds.c.smithi059.stdout:starting mds.c at 2023-11-24T07:58:36.469 INFO:tasks.ceph.mds.b.smithi076.stdout:starting mds.b at 2023-11-24T07:58:36.469 INFO:tasks.ceph.mds.d.smithi076.stdout:starting mds.d at 2023-11-24T07:58:36.469 INFO:tasks.ceph.mds.d.smithi076.stderr:2023-11-24T07:58:36.467+0000 7f293cc11700 -1 PosixStack listen unable to listen on v2:0.0.0.0:6834/0: (98) Address already in use 2023-11-24T07:58:36.515 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:36.513+0000 7f8bbe490700 1 Processor -- start 2023-11-24T07:58:36.515 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:36.513+0000 7f8bbe490700 1 -- start start 2023-11-24T07:58:36.517 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:36.514+0000 7f8bbe490700 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f8bb80b5ab0 0x7f8bb806e560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:36.517 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:36.514+0000 7f8bbe490700 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8bb8071f50 con 0x7f8bb806b620 2023-11-24T07:58:36.518 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:36.514+0000 7f8bbe490700 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8bb8072230 con 0x7f8bb80b4150 2023-11-24T07:58:36.518 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:36.514+0000 7f8bbe490700 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8bb8072510 con 0x7f8bb80b5ab0 2023-11-24T07:58:36.518 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:36.514+0000 7f8bb7fff700 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7f8bb806b620 0x7f8bb80b3a40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:57124/0 (socket says 172.21.15.59:57124) 2023-11-24T07:58:36.518 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:36.514+0000 7f8bb7fff700 1 -- 172.21.15.59:0/1965578156 learned_addr learned my addr 172.21.15.59:0/1965578156 (peer_addr_for_me v1:172.21.15.59:0/0) 2023-11-24T07:58:36.518 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:36.514+0000 7f8bbca2d700 1 --2- 172.21.15.59:0/1965578156 >> v2:172.21.15.76:3300/0 conn(0x7f8bb80b5ab0 0x7f8bb806e560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:36.518 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:36.515+0000 7f8bb6ffd700 1 -- 172.21.15.59:0/1965578156 <== mon.0 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3930230228 0 0) 0x7f8bb8071f50 con 0x7f8bb806b620 2023-11-24T07:58:36.518 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:36.515+0000 7f8bb6ffd700 1 -- 172.21.15.59:0/1965578156 --> v1:172.21.15.59:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8ba00035b0 con 0x7f8bb806b620 2023-11-24T07:58:36.518 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:36.515+0000 7f8bbca2d700 1 -- 172.21.15.59:0/1965578156 >> v1:172.21.15.76:6789/0 conn(0x7f8bb80b4150 legacy=0x7f8bb806de50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:36.518 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:36.515+0000 7f8bb6ffd700 1 -- 172.21.15.59:0/1965578156 <== mon.1 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 346016445 0 0) 0x7f8bb8072230 con 0x7f8bb80b4150 2023-11-24T07:58:36.518 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:36.515+0000 7f8bbca2d700 1 -- 172.21.15.59:0/1965578156 >> v1:172.21.15.59:6789/0 conn(0x7f8bb806b620 legacy=0x7f8bb80b3a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:36.519 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:36.515+0000 7f8bbca2d700 1 -- 172.21.15.59:0/1965578156 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8bb80727f0 con 0x7f8bb80b5ab0 2023-11-24T07:58:36.519 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:36.515+0000 7f8bbca2d700 1 --2- 172.21.15.59:0/1965578156 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8bb80b5ab0 0x7f8bb806e560 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f8bb0009d90 tx=0x7f8bb000b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=63fa2d43eb7c4ece server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:58:36.519 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:36.515+0000 7f8bb6ffd700 1 -- 172.21.15.59:0/1965578156 <== mon.1 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8bb000e070 con 0x7f8bb80b5ab0 2023-11-24T07:58:36.519 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:36.515+0000 7f8bb6ffd700 1 -- 172.21.15.59:0/1965578156 <== mon.1 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8bb000bb20 con 0x7f8bb80b5ab0 2023-11-24T07:58:36.519 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:36.515+0000 7f8bb6ffd700 1 -- 172.21.15.59:0/1965578156 <== mon.1 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8bb000bed0 con 0x7f8bb80b5ab0 2023-11-24T07:58:36.519 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:36.516+0000 7f8bbe490700 1 -- 172.21.15.59:0/1965578156 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8bb80b5ab0 msgr2=0x7f8bb806e560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:36.519 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:36.516+0000 7f8bbe490700 1 --2- 172.21.15.59:0/1965578156 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8bb80b5ab0 0x7f8bb806e560 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f8bb0009d90 tx=0x7f8bb000b330 comp rx=0 tx=0).stop 2023-11-24T07:58:36.519 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:36.516+0000 7f8bbe490700 1 -- 172.21.15.59:0/1965578156 shutdown_connections 2023-11-24T07:58:36.519 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:36.516+0000 7f8bbe490700 1 --2- 172.21.15.59:0/1965578156 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8bb80b5ab0 0x7f8bb806e560 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:36.519 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:36.516+0000 7f8bbe490700 1 -- 172.21.15.59:0/1965578156 >> 172.21.15.59:0/1965578156 conn(0x7f8bb80b2740 msgr2=0x7f8bb80b2b20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:58:36.519 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:36.516+0000 7f8bbe490700 1 -- 172.21.15.59:0/1965578156 shutdown_connections 2023-11-24T07:58:36.520 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:36.516+0000 7f8bbe490700 1 -- 172.21.15.59:0/1965578156 wait complete. 2023-11-24T07:58:36.522 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:36.517+0000 7f8bbe490700 1 Processor -- start 2023-11-24T07:58:36.522 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:36.517+0000 7f8bbe490700 1 -- start start 2023-11-24T07:58:36.522 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:36.518+0000 7f8bbe490700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8bb806b620 0x7f8bb813d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:36.522 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:36.518+0000 7f8bbe490700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8bb80b4150 0x7f8bb81f3280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:36.523 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:36.518+0000 7f8bbe490700 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f8bb80b5ab0 0x7f8bb81f43a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:36.523 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:36.518+0000 7f8bbe490700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8bb813cd70 con 0x7f8bb80b4150 2023-11-24T07:58:36.523 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:36.518+0000 7f8bbe490700 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8bb813d6d0 con 0x7f8bb806b620 2023-11-24T07:58:36.523 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:36.518+0000 7f8bbe490700 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8bb81f37c0 con 0x7f8bb80b5ab0 2023-11-24T07:58:36.523 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:36.518+0000 7f8bb77fe700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8bb80b4150 0x7f8bb81f3280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:36.523 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:36.518+0000 7f8bb77fe700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8bb80b4150 0x7f8bb81f3280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:53658/0 (socket says 172.21.15.59:53658) 2023-11-24T07:58:36.523 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:36.518+0000 7f8bb77fe700 1 -- 172.21.15.59:0/211134648 learned_addr learned my addr 172.21.15.59:0/211134648 (peer_addr_for_me v2:172.21.15.59:0/0) 2023-11-24T07:58:36.523 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:36.518+0000 7f8bb7fff700 1 --2- 172.21.15.59:0/211134648 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8bb806b620 0x7f8bb813d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:36.523 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:36.518+0000 7f8bbca2d700 1 --2- 172.21.15.59:0/211134648 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f8bb80b5ab0 0x7f8bb81f43a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:36.523 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:36.518+0000 7f8bb77fe700 1 -- 172.21.15.59:0/211134648 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f8bb80b5ab0 msgr2=0x7f8bb81f43a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:36.524 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:36.518+0000 7f8bb77fe700 1 --2- 172.21.15.59:0/211134648 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f8bb80b5ab0 0x7f8bb81f43a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:36.524 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:36.518+0000 7f8bb77fe700 1 -- 172.21.15.59:0/211134648 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8bb806b620 msgr2=0x7f8bb813d190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:36.524 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:36.518+0000 7f8bb77fe700 1 --2- 172.21.15.59:0/211134648 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8bb806b620 0x7f8bb813d190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:36.524 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:36.518+0000 7f8bb77fe700 1 -- 172.21.15.59:0/211134648 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8bb0009a70 con 0x7f8bb80b4150 2023-11-24T07:58:36.524 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:36.518+0000 7f8bb77fe700 1 --2- 172.21.15.59:0/211134648 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8bb80b4150 0x7f8bb81f3280 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x7f8bac00c750 tx=0x7f8bac00ca60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:58:36.524 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:36.519+0000 7f8bb4ff9700 1 -- 172.21.15.59:0/211134648 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8bac0054e0 con 0x7f8bb80b4150 2023-11-24T07:58:36.524 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:36.519+0000 7f8bb4ff9700 1 -- 172.21.15.59:0/211134648 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8bac003680 con 0x7f8bb80b4150 2023-11-24T07:58:36.524 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:36.519+0000 7f8bbe490700 1 -- 172.21.15.59:0/211134648 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8bb81f48e0 con 0x7f8bb80b4150 2023-11-24T07:58:36.524 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:36.519+0000 7f8bb4ff9700 1 -- 172.21.15.59:0/211134648 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8bac003a30 con 0x7f8bb80b4150 2023-11-24T07:58:36.524 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:36.519+0000 7f8bbe490700 1 -- 172.21.15.59:0/211134648 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8bb813c3d0 con 0x7f8bb80b4150 2023-11-24T07:58:36.524 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:36.520+0000 7f8bbe490700 1 -- 172.21.15.59:0/211134648 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8bb8068670 con 0x7f8bb80b4150 2023-11-24T07:58:36.524 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:36.520+0000 7f8bb4ff9700 1 -- 172.21.15.59:0/211134648 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 8) v1 ==== 89435+0+0 (secure 0 0 0) 0x7f8bac003b90 con 0x7f8bb80b4150 2023-11-24T07:58:36.525 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:36.521+0000 7f8bb4ff9700 1 --2- 172.21.15.59:0/211134648 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f8ba406b110 0x7f8ba406d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:36.525 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:36.521+0000 7f8bb4ff9700 1 -- 172.21.15.59:0/211134648 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(52..52 src has 1..52) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f8bac088ad0 con 0x7f8bb80b4150 2023-11-24T07:58:36.525 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:36.521+0000 7f8bb7fff700 1 --2- 172.21.15.59:0/211134648 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f8ba406b110 0x7f8ba406d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:36.525 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:36.521+0000 7f8bb7fff700 1 --2- 172.21.15.59:0/211134648 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f8ba406b110 0x7f8ba406d5d0 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f8ba8007fd0 tx=0x7f8ba8009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:58:36.527 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:36.525+0000 7f8bb4ff9700 1 -- 172.21.15.59:0/211134648 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f8bac053b10 con 0x7f8bb80b4150 2023-11-24T07:58:36.743 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:36.741+0000 7f8bbe490700 1 -- 172.21.15.59:0/211134648 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7f8bb813ceb0 con 0x7f8bb80b4150 2023-11-24T07:58:37.363 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:37.361+0000 7f8bb4ff9700 1 -- 172.21.15.59:0/211134648 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v56) v1 ==== 97+0+0 (secure 0 0 0) 0x7f8bb813ceb0 con 0x7f8bb80b4150 2023-11-24T07:58:37.366 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:37.364+0000 7f8bbe490700 1 -- 172.21.15.59:0/211134648 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f8ba406b110 msgr2=0x7f8ba406d5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:37.366 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:37.364+0000 7f8bbe490700 1 --2- 172.21.15.59:0/211134648 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f8ba406b110 0x7f8ba406d5d0 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f8ba8007fd0 tx=0x7f8ba8009040 comp rx=0 tx=0).stop 2023-11-24T07:58:37.366 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:37.364+0000 7f8bbe490700 1 -- 172.21.15.59:0/211134648 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8bb80b4150 msgr2=0x7f8bb81f3280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:37.366 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:37.364+0000 7f8bbe490700 1 --2- 172.21.15.59:0/211134648 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8bb80b4150 0x7f8bb81f3280 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x7f8bac00c750 tx=0x7f8bac00ca60 comp rx=0 tx=0).stop 2023-11-24T07:58:37.366 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:37.364+0000 7f8bbe490700 1 -- 172.21.15.59:0/211134648 shutdown_connections 2023-11-24T07:58:37.366 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:37.364+0000 7f8bbe490700 1 --2- 172.21.15.59:0/211134648 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f8ba406b110 0x7f8ba406d5d0 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:37.366 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:37.364+0000 7f8bbe490700 1 --2- 172.21.15.59:0/211134648 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f8bb80b5ab0 0x7f8bb81f43a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:37.366 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:37.364+0000 7f8bbe490700 1 --2- 172.21.15.59:0/211134648 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8bb806b620 0x7f8bb813d190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:37.366 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:37.364+0000 7f8bbe490700 1 --2- 172.21.15.59:0/211134648 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8bb80b4150 0x7f8bb81f3280 unknown :-1 s=CLOSED pgs=681 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:37.367 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:37.364+0000 7f8bbe490700 1 -- 172.21.15.59:0/211134648 >> 172.21.15.59:0/211134648 conn(0x7f8bb80b2740 msgr2=0x7f8bb8146320 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:58:37.367 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:37.364+0000 7f8bbe490700 1 -- 172.21.15.59:0/211134648 shutdown_connections 2023-11-24T07:58:37.367 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:37.364+0000 7f8bbe490700 1 -- 172.21.15.59:0/211134648 wait complete. 2023-11-24T07:58:37.367 INFO:teuthology.orchestra.run.smithi059.stderr: removed all blocklist entries 2023-11-24T07:58:37.381 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2023-11-24T07:58:37.512 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:37.510+0000 7fd91d55d700 1 Processor -- start 2023-11-24T07:58:37.512 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:37.510+0000 7fd91d55d700 1 -- start start 2023-11-24T07:58:37.514 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:37.511+0000 7fd91d55d700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fd91806cae0 0x7fd9180cb2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:37.514 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:37.511+0000 7fd91d55d700 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd9181449d0 con 0x7fd9181445f0 2023-11-24T07:58:37.514 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:37.511+0000 7fd91d55d700 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd918071650 con 0x7fd91806e480 2023-11-24T07:58:37.514 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:37.511+0000 7fd91d55d700 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd918072e00 con 0x7fd91806cae0 2023-11-24T07:58:37.515 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:37.511+0000 7fd9167fc700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fd91806cae0 0x7fd9180cb2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:37.515 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:37.511+0000 7fd916ffd700 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7fd9181445f0 0x7fd9180726f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:57142/0 (socket says 172.21.15.59:57142) 2023-11-24T07:58:37.515 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:37.511+0000 7fd916ffd700 1 -- 172.21.15.59:0/1856897573 learned_addr learned my addr 172.21.15.59:0/1856897573 (peer_addr_for_me v1:172.21.15.59:0/0) 2023-11-24T07:58:37.515 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:37.511+0000 7fd9167fc700 1 -- 172.21.15.59:0/1856897573 >> v1:172.21.15.76:6789/0 conn(0x7fd91806e480 legacy=0x7fd9180cd550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:37.515 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:37.511+0000 7fd9167fc700 1 -- 172.21.15.59:0/1856897573 >> v1:172.21.15.59:6789/0 conn(0x7fd9181445f0 legacy=0x7fd9180726f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:37.515 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:37.511+0000 7fd9167fc700 1 -- 172.21.15.59:0/1856897573 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd918071930 con 0x7fd91806cae0 2023-11-24T07:58:37.515 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:37.511+0000 7fd915ffb700 1 -- 172.21.15.59:0/1856897573 <== mon.0 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2169007084 0 0) 0x7fd9181449d0 con 0x7fd9181445f0 2023-11-24T07:58:37.515 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:37.512+0000 7fd9167fc700 1 --2- 172.21.15.59:0/1856897573 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd91806cae0 0x7fd9180cb2f0 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 crypto rx=0x7fd908009840 tx=0x7fd908009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3b3d84da1ac4f6ba server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:58:37.515 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:37.512+0000 7fd915ffb700 1 -- 172.21.15.59:0/1856897573 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd908011830 con 0x7fd91806cae0 2023-11-24T07:58:37.515 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:37.512+0000 7fd915ffb700 1 -- 172.21.15.59:0/1856897573 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd908011990 con 0x7fd91806cae0 2023-11-24T07:58:37.515 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:37.512+0000 7fd915ffb700 1 -- 172.21.15.59:0/1856897573 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd908011d20 con 0x7fd91806cae0 2023-11-24T07:58:37.516 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:37.512+0000 7fd91d55d700 1 -- 172.21.15.59:0/1856897573 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd91806cae0 msgr2=0x7fd9180cb2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:37.516 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:37.512+0000 7fd91d55d700 1 --2- 172.21.15.59:0/1856897573 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd91806cae0 0x7fd9180cb2f0 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 crypto rx=0x7fd908009840 tx=0x7fd908009b50 comp rx=0 tx=0).stop 2023-11-24T07:58:37.516 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:37.512+0000 7fd91d55d700 1 -- 172.21.15.59:0/1856897573 shutdown_connections 2023-11-24T07:58:37.516 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:37.512+0000 7fd91d55d700 1 --2- 172.21.15.59:0/1856897573 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd91806cae0 0x7fd9180cb2f0 unknown :-1 s=CLOSED pgs=683 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:37.516 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:37.512+0000 7fd91d55d700 1 -- 172.21.15.59:0/1856897573 >> 172.21.15.59:0/1856897573 conn(0x7fd9180cf980 msgr2=0x7fd9180cfd60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:58:37.516 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:37.512+0000 7fd91d55d700 1 -- 172.21.15.59:0/1856897573 shutdown_connections 2023-11-24T07:58:37.516 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:37.512+0000 7fd91d55d700 1 -- 172.21.15.59:0/1856897573 wait complete. 2023-11-24T07:58:37.519 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:37.514+0000 7fd91d55d700 1 Processor -- start 2023-11-24T07:58:37.519 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:37.514+0000 7fd91d55d700 1 -- start start 2023-11-24T07:58:37.519 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:37.514+0000 7fd91d55d700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd91806cae0 0x7fd9181549a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:37.519 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:37.514+0000 7fd91d55d700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd91806e480 0x7fd9180c8f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:37.519 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:37.514+0000 7fd91d55d700 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fd9181445f0 0x7fd9181f37d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:37.519 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:37.514+0000 7fd91d55d700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd918137960 con 0x7fd91806e480 2023-11-24T07:58:37.519 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:37.514+0000 7fd91d55d700 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd918136170 con 0x7fd91806cae0 2023-11-24T07:58:37.519 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:37.514+0000 7fd91d55d700 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd91813bb10 con 0x7fd9181445f0 2023-11-24T07:58:37.520 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:37.514+0000 7fd9167fc700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd91806e480 0x7fd9180c8f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:37.520 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:37.515+0000 7fd9167fc700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd91806e480 0x7fd9180c8f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:53676/0 (socket says 172.21.15.59:53676) 2023-11-24T07:58:37.520 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:37.515+0000 7fd9167fc700 1 -- 172.21.15.59:0/1429860118 learned_addr learned my addr 172.21.15.59:0/1429860118 (peer_addr_for_me v2:172.21.15.59:0/0) 2023-11-24T07:58:37.520 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:37.515+0000 7fd916ffd700 1 --2- 172.21.15.59:0/1429860118 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd91806cae0 0x7fd9181549a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:37.520 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:37.515+0000 7fd9177fe700 1 --2- 172.21.15.59:0/1429860118 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fd9181445f0 0x7fd9181f37d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:37.520 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:37.515+0000 7fd9167fc700 1 -- 172.21.15.59:0/1429860118 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fd9181445f0 msgr2=0x7fd9181f37d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:37.520 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:37.515+0000 7fd9167fc700 1 --2- 172.21.15.59:0/1429860118 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fd9181445f0 0x7fd9181f37d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:37.520 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:37.515+0000 7fd9167fc700 1 -- 172.21.15.59:0/1429860118 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd91806cae0 msgr2=0x7fd9181549a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:37.520 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:37.515+0000 7fd9167fc700 1 --2- 172.21.15.59:0/1429860118 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd91806cae0 0x7fd9181549a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:37.520 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:37.515+0000 7fd9167fc700 1 -- 172.21.15.59:0/1429860118 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd9080094a0 con 0x7fd91806e480 2023-11-24T07:58:37.520 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:37.515+0000 7fd9167fc700 1 --2- 172.21.15.59:0/1429860118 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd91806e480 0x7fd9180c8f40 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 crypto rx=0x7fd9080090b0 tx=0x7fd908004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:58:37.520 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:37.515+0000 7fd8ff7fe700 1 -- 172.21.15.59:0/1429860118 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd90801a900 con 0x7fd91806e480 2023-11-24T07:58:37.521 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:37.515+0000 7fd8ff7fe700 1 -- 172.21.15.59:0/1429860118 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd90801aa60 con 0x7fd91806e480 2023-11-24T07:58:37.521 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:37.515+0000 7fd91d55d700 1 -- 172.21.15.59:0/1429860118 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd918135b40 con 0x7fd91806e480 2023-11-24T07:58:37.521 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:37.515+0000 7fd8ff7fe700 1 -- 172.21.15.59:0/1429860118 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd90801add0 con 0x7fd91806e480 2023-11-24T07:58:37.521 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:37.515+0000 7fd91d55d700 1 -- 172.21.15.59:0/1429860118 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd91813df10 con 0x7fd91806e480 2023-11-24T07:58:37.521 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:37.516+0000 7fd91d55d700 1 -- 172.21.15.59:0/1429860118 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd918067f60 con 0x7fd91806e480 2023-11-24T07:58:37.521 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:37.517+0000 7fd8ff7fe700 1 -- 172.21.15.59:0/1429860118 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 8) v1 ==== 89435+0+0 (secure 0 0 0) 0x7fd908022070 con 0x7fd91806e480 2023-11-24T07:58:37.521 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:37.517+0000 7fd8ff7fe700 1 --2- 172.21.15.59:0/1429860118 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7fd90406b110 0x7fd90406d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:37.521 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:37.517+0000 7fd8ff7fe700 1 -- 172.21.15.59:0/1429860118 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(56..56 src has 1..56) v4 ==== 4857+0+0 (secure 0 0 0) 0x7fd90808a8e0 con 0x7fd91806e480 2023-11-24T07:58:37.521 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:37.517+0000 7fd916ffd700 1 --2- 172.21.15.59:0/1429860118 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7fd90406b110 0x7fd90406d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:37.522 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:37.518+0000 7fd916ffd700 1 --2- 172.21.15.59:0/1429860118 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7fd90406b110 0x7fd90406d5d0 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7fd90c000f60 tx=0x7fd90c007d90 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:58:37.523 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:37.522+0000 7fd8ff7fe700 1 -- 172.21.15.59:0/1429860118 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fd908055ac0 con 0x7fd91806e480 2023-11-24T07:58:37.805 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:37.804+0000 7fd91d55d700 1 -- 172.21.15.59:0/1429860118 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7fd9181362b0 con 0x7fd91806e480 2023-11-24T07:58:37.806 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:37.804+0000 7fd8ff7fe700 1 -- 172.21.15.59:0/1429860118 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v14) v1 ==== 80+0+5304 (secure 0 0 0) 0x7fd908029090 con 0x7fd91806e480 2023-11-24T07:58:37.806 INFO:teuthology.orchestra.run.smithi059.stdout: 2023-11-24T07:58:37.806 INFO:teuthology.orchestra.run.smithi059.stdout:{ 2023-11-24T07:58:37.806 INFO:teuthology.orchestra.run.smithi059.stdout: "auth_dump": [ 2023-11-24T07:58:37.807 INFO:teuthology.orchestra.run.smithi059.stdout: { 2023-11-24T07:58:37.807 INFO:teuthology.orchestra.run.smithi059.stdout: "entity": "mds.a", 2023-11-24T07:58:37.807 INFO:teuthology.orchestra.run.smithi059.stdout: "key": "AQD0VmBlWGLCKBAASbEQYAkExr8TButtK4KHQQ==", 2023-11-24T07:58:37.807 INFO:teuthology.orchestra.run.smithi059.stdout: "caps": { 2023-11-24T07:58:37.807 INFO:teuthology.orchestra.run.smithi059.stdout: "mds": "allow", 2023-11-24T07:58:37.807 INFO:teuthology.orchestra.run.smithi059.stdout: "mgr": "allow *", 2023-11-24T07:58:37.807 INFO:teuthology.orchestra.run.smithi059.stdout: "mon": "allow *", 2023-11-24T07:58:37.807 INFO:teuthology.orchestra.run.smithi059.stdout: "osd": "allow *" 2023-11-24T07:58:37.807 INFO:teuthology.orchestra.run.smithi059.stdout: } 2023-11-24T07:58:37.807 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-11-24T07:58:37.807 INFO:teuthology.orchestra.run.smithi059.stdout: { 2023-11-24T07:58:37.807 INFO:teuthology.orchestra.run.smithi059.stdout: "entity": "mds.b", 2023-11-24T07:58:37.808 INFO:teuthology.orchestra.run.smithi059.stdout: "key": "AQD0VmBlDIS0NRAAYQ3wTunU7m/Qk1qCM6Mcvg==", 2023-11-24T07:58:37.808 INFO:teuthology.orchestra.run.smithi059.stdout: "caps": { 2023-11-24T07:58:37.808 INFO:teuthology.orchestra.run.smithi059.stdout: "mds": "allow", 2023-11-24T07:58:37.808 INFO:teuthology.orchestra.run.smithi059.stdout: "mgr": "allow *", 2023-11-24T07:58:37.808 INFO:teuthology.orchestra.run.smithi059.stdout: "mon": "allow *", 2023-11-24T07:58:37.808 INFO:teuthology.orchestra.run.smithi059.stdout: "osd": "allow *" 2023-11-24T07:58:37.808 INFO:teuthology.orchestra.run.smithi059.stdout: } 2023-11-24T07:58:37.808 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-11-24T07:58:37.808 INFO:teuthology.orchestra.run.smithi059.stdout: { 2023-11-24T07:58:37.808 INFO:teuthology.orchestra.run.smithi059.stdout: "entity": "mds.c", 2023-11-24T07:58:37.808 INFO:teuthology.orchestra.run.smithi059.stdout: "key": "AQD0VmBlvpsrMBAAr5lh7nb6B1xRfk0Umm7Xig==", 2023-11-24T07:58:37.809 INFO:teuthology.orchestra.run.smithi059.stdout: "caps": { 2023-11-24T07:58:37.809 INFO:teuthology.orchestra.run.smithi059.stdout: "mds": "allow", 2023-11-24T07:58:37.809 INFO:teuthology.orchestra.run.smithi059.stdout: "mgr": "allow *", 2023-11-24T07:58:37.809 INFO:teuthology.orchestra.run.smithi059.stdout: "mon": "allow *", 2023-11-24T07:58:37.809 INFO:teuthology.orchestra.run.smithi059.stdout: "osd": "allow *" 2023-11-24T07:58:37.809 INFO:teuthology.orchestra.run.smithi059.stdout: } 2023-11-24T07:58:37.809 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-11-24T07:58:37.809 INFO:teuthology.orchestra.run.smithi059.stdout: { 2023-11-24T07:58:37.809 INFO:teuthology.orchestra.run.smithi059.stdout: "entity": "mds.d", 2023-11-24T07:58:37.809 INFO:teuthology.orchestra.run.smithi059.stdout: "key": "AQD1VmBlPTG7ARAAvC60Rk2GarnNtRJ6MMRIMw==", 2023-11-24T07:58:37.809 INFO:teuthology.orchestra.run.smithi059.stdout: "caps": { 2023-11-24T07:58:37.809 INFO:teuthology.orchestra.run.smithi059.stdout: "mds": "allow", 2023-11-24T07:58:37.809 INFO:teuthology.orchestra.run.smithi059.stdout: "mgr": "allow *", 2023-11-24T07:58:37.810 INFO:teuthology.orchestra.run.smithi059.stdout: "mon": "allow *", 2023-11-24T07:58:37.810 INFO:teuthology.orchestra.run.smithi059.stdout: "osd": "allow *" 2023-11-24T07:58:37.810 INFO:teuthology.orchestra.run.smithi059.stdout: } 2023-11-24T07:58:37.810 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-11-24T07:58:37.810 INFO:teuthology.orchestra.run.smithi059.stdout: { 2023-11-24T07:58:37.810 INFO:teuthology.orchestra.run.smithi059.stdout: "entity": "osd.0", 2023-11-24T07:58:37.810 INFO:teuthology.orchestra.run.smithi059.stdout: "key": "AQD4VmBlXjdAHRAAjnK2Y/rQhg4a97faNqVuoA==", 2023-11-24T07:58:37.810 INFO:teuthology.orchestra.run.smithi059.stdout: "caps": { 2023-11-24T07:58:37.810 INFO:teuthology.orchestra.run.smithi059.stdout: "mgr": "allow profile osd", 2023-11-24T07:58:37.810 INFO:teuthology.orchestra.run.smithi059.stdout: "mon": "allow profile osd", 2023-11-24T07:58:37.810 INFO:teuthology.orchestra.run.smithi059.stdout: "osd": "allow *" 2023-11-24T07:58:37.811 INFO:teuthology.orchestra.run.smithi059.stdout: } 2023-11-24T07:58:37.811 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-11-24T07:58:37.811 INFO:teuthology.orchestra.run.smithi059.stdout: { 2023-11-24T07:58:37.811 INFO:teuthology.orchestra.run.smithi059.stdout: "entity": "osd.1", 2023-11-24T07:58:37.811 INFO:teuthology.orchestra.run.smithi059.stdout: "key": "AQD7VmBlBwTwHBAAbpMH9rzS95ifrq/gVHnYRA==", 2023-11-24T07:58:37.811 INFO:teuthology.orchestra.run.smithi059.stdout: "caps": { 2023-11-24T07:58:37.811 INFO:teuthology.orchestra.run.smithi059.stdout: "mgr": "allow profile osd", 2023-11-24T07:58:37.811 INFO:teuthology.orchestra.run.smithi059.stdout: "mon": "allow profile osd", 2023-11-24T07:58:37.811 INFO:teuthology.orchestra.run.smithi059.stdout: "osd": "allow *" 2023-11-24T07:58:37.811 INFO:teuthology.orchestra.run.smithi059.stdout: } 2023-11-24T07:58:37.811 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-11-24T07:58:37.811 INFO:teuthology.orchestra.run.smithi059.stdout: { 2023-11-24T07:58:37.812 INFO:teuthology.orchestra.run.smithi059.stdout: "entity": "osd.2", 2023-11-24T07:58:37.812 INFO:teuthology.orchestra.run.smithi059.stdout: "key": "AQD+VmBl0tijHBAAWF2PsV3iH5adOTvHVPFIKw==", 2023-11-24T07:58:37.812 INFO:teuthology.orchestra.run.smithi059.stdout: "caps": { 2023-11-24T07:58:37.812 INFO:teuthology.orchestra.run.smithi059.stdout: "mgr": "allow profile osd", 2023-11-24T07:58:37.812 INFO:teuthology.orchestra.run.smithi059.stdout: "mon": "allow profile osd", 2023-11-24T07:58:37.812 INFO:teuthology.orchestra.run.smithi059.stdout: "osd": "allow *" 2023-11-24T07:58:37.812 INFO:teuthology.orchestra.run.smithi059.stdout: } 2023-11-24T07:58:37.812 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-11-24T07:58:37.812 INFO:teuthology.orchestra.run.smithi059.stdout: { 2023-11-24T07:58:37.812 INFO:teuthology.orchestra.run.smithi059.stdout: "entity": "osd.3", 2023-11-24T07:58:37.812 INFO:teuthology.orchestra.run.smithi059.stdout: "key": "AQABV2Bl3i8/HBAA4waLTD/PSLuNEg1VBmhEmg==", 2023-11-24T07:58:37.812 INFO:teuthology.orchestra.run.smithi059.stdout: "caps": { 2023-11-24T07:58:37.813 INFO:teuthology.orchestra.run.smithi059.stdout: "mgr": "allow profile osd", 2023-11-24T07:58:37.813 INFO:teuthology.orchestra.run.smithi059.stdout: "mon": "allow profile osd", 2023-11-24T07:58:37.813 INFO:teuthology.orchestra.run.smithi059.stdout: "osd": "allow *" 2023-11-24T07:58:37.813 INFO:teuthology.orchestra.run.smithi059.stdout: } 2023-11-24T07:58:37.813 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-11-24T07:58:37.813 INFO:teuthology.orchestra.run.smithi059.stdout: { 2023-11-24T07:58:37.813 INFO:teuthology.orchestra.run.smithi059.stdout: "entity": "osd.4", 2023-11-24T07:58:37.813 INFO:teuthology.orchestra.run.smithi059.stdout: "key": "AQAHV2Blr2zrHxAALOr+kFGdOTr8uQ9+/G2WUg==", 2023-11-24T07:58:37.813 INFO:teuthology.orchestra.run.smithi059.stdout: "caps": { 2023-11-24T07:58:37.813 INFO:teuthology.orchestra.run.smithi059.stdout: "mgr": "allow profile osd", 2023-11-24T07:58:37.813 INFO:teuthology.orchestra.run.smithi059.stdout: "mon": "allow profile osd", 2023-11-24T07:58:37.814 INFO:teuthology.orchestra.run.smithi059.stdout: "osd": "allow *" 2023-11-24T07:58:37.814 INFO:teuthology.orchestra.run.smithi059.stdout: } 2023-11-24T07:58:37.814 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-11-24T07:58:37.814 INFO:teuthology.orchestra.run.smithi059.stdout: { 2023-11-24T07:58:37.814 INFO:teuthology.orchestra.run.smithi059.stdout: "entity": "osd.5", 2023-11-24T07:58:37.814 INFO:teuthology.orchestra.run.smithi059.stdout: "key": "AQAKV2BlDZ3bHxAA7ZPPoxi4zVN0VECYzTsAsg==", 2023-11-24T07:58:37.814 INFO:teuthology.orchestra.run.smithi059.stdout: "caps": { 2023-11-24T07:58:37.814 INFO:teuthology.orchestra.run.smithi059.stdout: "mgr": "allow profile osd", 2023-11-24T07:58:37.814 INFO:teuthology.orchestra.run.smithi059.stdout: "mon": "allow profile osd", 2023-11-24T07:58:37.814 INFO:teuthology.orchestra.run.smithi059.stdout: "osd": "allow *" 2023-11-24T07:58:37.814 INFO:teuthology.orchestra.run.smithi059.stdout: } 2023-11-24T07:58:37.814 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-11-24T07:58:37.815 INFO:teuthology.orchestra.run.smithi059.stdout: { 2023-11-24T07:58:37.815 INFO:teuthology.orchestra.run.smithi059.stdout: "entity": "osd.6", 2023-11-24T07:58:37.815 INFO:teuthology.orchestra.run.smithi059.stdout: "key": "AQANV2BlfDMrHxAAVtm8+kh5FNTd1mJOKg02gg==", 2023-11-24T07:58:37.815 INFO:teuthology.orchestra.run.smithi059.stdout: "caps": { 2023-11-24T07:58:37.815 INFO:teuthology.orchestra.run.smithi059.stdout: "mgr": "allow profile osd", 2023-11-24T07:58:37.815 INFO:teuthology.orchestra.run.smithi059.stdout: "mon": "allow profile osd", 2023-11-24T07:58:37.815 INFO:teuthology.orchestra.run.smithi059.stdout: "osd": "allow *" 2023-11-24T07:58:37.815 INFO:teuthology.orchestra.run.smithi059.stdout: } 2023-11-24T07:58:37.815 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-11-24T07:58:37.815 INFO:teuthology.orchestra.run.smithi059.stdout: { 2023-11-24T07:58:37.815 INFO:teuthology.orchestra.run.smithi059.stdout: "entity": "osd.7", 2023-11-24T07:58:37.815 INFO:teuthology.orchestra.run.smithi059.stdout: "key": "AQAQV2Bli+slHxAAHY4sa84El5NC4KVgPKHs6w==", 2023-11-24T07:58:37.816 INFO:teuthology.orchestra.run.smithi059.stdout: "caps": { 2023-11-24T07:58:37.816 INFO:teuthology.orchestra.run.smithi059.stdout: "mgr": "allow profile osd", 2023-11-24T07:58:37.816 INFO:teuthology.orchestra.run.smithi059.stdout: "mon": "allow profile osd", 2023-11-24T07:58:37.816 INFO:teuthology.orchestra.run.smithi059.stdout: "osd": "allow *" 2023-11-24T07:58:37.816 INFO:teuthology.orchestra.run.smithi059.stdout: } 2023-11-24T07:58:37.816 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-11-24T07:58:37.816 INFO:teuthology.orchestra.run.smithi059.stdout: { 2023-11-24T07:58:37.816 INFO:teuthology.orchestra.run.smithi059.stdout: "entity": "client.0", 2023-11-24T07:58:37.816 INFO:teuthology.orchestra.run.smithi059.stdout: "key": "AQD1VmBlxPkpBhAAR841hRsF6QtmzFgJIrVZ9w==", 2023-11-24T07:58:37.816 INFO:teuthology.orchestra.run.smithi059.stdout: "caps": { 2023-11-24T07:58:37.816 INFO:teuthology.orchestra.run.smithi059.stdout: "mds": "allow", 2023-11-24T07:58:37.816 INFO:teuthology.orchestra.run.smithi059.stdout: "mon": "allow r", 2023-11-24T07:58:37.817 INFO:teuthology.orchestra.run.smithi059.stdout: "osd": "allow rw tag cephfs data=cephfs" 2023-11-24T07:58:37.817 INFO:teuthology.orchestra.run.smithi059.stdout: } 2023-11-24T07:58:37.817 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-11-24T07:58:37.817 INFO:teuthology.orchestra.run.smithi059.stdout: { 2023-11-24T07:58:37.817 INFO:teuthology.orchestra.run.smithi059.stdout: "entity": "client.1", 2023-11-24T07:58:37.817 INFO:teuthology.orchestra.run.smithi059.stdout: "key": "AQD1VmBl46HpDBAAnO9MGWrJczW0epSp0eCkLg==", 2023-11-24T07:58:37.817 INFO:teuthology.orchestra.run.smithi059.stdout: "caps": { 2023-11-24T07:58:37.817 INFO:teuthology.orchestra.run.smithi059.stdout: "mds": "allow", 2023-11-24T07:58:37.817 INFO:teuthology.orchestra.run.smithi059.stdout: "mgr": "allow r", 2023-11-24T07:58:37.817 INFO:teuthology.orchestra.run.smithi059.stdout: "mon": "allow rw", 2023-11-24T07:58:37.817 INFO:teuthology.orchestra.run.smithi059.stdout: "osd": "allow rwx" 2023-11-24T07:58:37.817 INFO:teuthology.orchestra.run.smithi059.stdout: } 2023-11-24T07:58:37.817 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-11-24T07:58:37.818 INFO:teuthology.orchestra.run.smithi059.stdout: { 2023-11-24T07:58:37.818 INFO:teuthology.orchestra.run.smithi059.stdout: "entity": "client.2", 2023-11-24T07:58:37.818 INFO:teuthology.orchestra.run.smithi059.stdout: "key": "AQD1VmBlgQSRCRAAcHS1eQCYMJ3YP21jUEPJ0A==", 2023-11-24T07:58:37.818 INFO:teuthology.orchestra.run.smithi059.stdout: "caps": { 2023-11-24T07:58:37.818 INFO:teuthology.orchestra.run.smithi059.stdout: "mds": "allow", 2023-11-24T07:58:37.818 INFO:teuthology.orchestra.run.smithi059.stdout: "mgr": "allow r", 2023-11-24T07:58:37.818 INFO:teuthology.orchestra.run.smithi059.stdout: "mon": "allow rw", 2023-11-24T07:58:37.818 INFO:teuthology.orchestra.run.smithi059.stdout: "osd": "allow rwx" 2023-11-24T07:58:37.818 INFO:teuthology.orchestra.run.smithi059.stdout: } 2023-11-24T07:58:37.818 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-11-24T07:58:37.818 INFO:teuthology.orchestra.run.smithi059.stdout: { 2023-11-24T07:58:37.818 INFO:teuthology.orchestra.run.smithi059.stdout: "entity": "client.3", 2023-11-24T07:58:37.819 INFO:teuthology.orchestra.run.smithi059.stdout: "key": "AQD1VmBl7gmhEhAA6uWs7XAbOQwJItA6To549w==", 2023-11-24T07:58:37.819 INFO:teuthology.orchestra.run.smithi059.stdout: "caps": { 2023-11-24T07:58:37.819 INFO:teuthology.orchestra.run.smithi059.stdout: "mds": "allow", 2023-11-24T07:58:37.819 INFO:teuthology.orchestra.run.smithi059.stdout: "mgr": "allow r", 2023-11-24T07:58:37.819 INFO:teuthology.orchestra.run.smithi059.stdout: "mon": "allow rw", 2023-11-24T07:58:37.819 INFO:teuthology.orchestra.run.smithi059.stdout: "osd": "allow rwx" 2023-11-24T07:58:37.819 INFO:teuthology.orchestra.run.smithi059.stdout: } 2023-11-24T07:58:37.819 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-11-24T07:58:37.819 INFO:teuthology.orchestra.run.smithi059.stdout: { 2023-11-24T07:58:37.819 INFO:teuthology.orchestra.run.smithi059.stdout: "entity": "client.admin", 2023-11-24T07:58:37.819 INFO:teuthology.orchestra.run.smithi059.stdout: "key": "AQD0VmBlAECpCRAA1CkLlkyjnOovaNUGDSWhaw==", 2023-11-24T07:58:37.819 INFO:teuthology.orchestra.run.smithi059.stdout: "caps": { 2023-11-24T07:58:37.820 INFO:teuthology.orchestra.run.smithi059.stdout: "mds": "allow *", 2023-11-24T07:58:37.820 INFO:teuthology.orchestra.run.smithi059.stdout: "mgr": "allow *", 2023-11-24T07:58:37.820 INFO:teuthology.orchestra.run.smithi059.stdout: "mon": "allow *", 2023-11-24T07:58:37.820 INFO:teuthology.orchestra.run.smithi059.stdout: "osd": "allow *" 2023-11-24T07:58:37.820 INFO:teuthology.orchestra.run.smithi059.stdout: } 2023-11-24T07:58:37.820 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-11-24T07:58:37.820 INFO:teuthology.orchestra.run.smithi059.stdout: { 2023-11-24T07:58:37.820 INFO:teuthology.orchestra.run.smithi059.stdout: "entity": "mgr.x", 2023-11-24T07:58:37.820 INFO:teuthology.orchestra.run.smithi059.stdout: "key": "AQD0VmBlryM+JRAAf72PlDWEHZOwNxhW4k/8Ig==", 2023-11-24T07:58:37.820 INFO:teuthology.orchestra.run.smithi059.stdout: "caps": { 2023-11-24T07:58:37.820 INFO:teuthology.orchestra.run.smithi059.stdout: "mds": "allow *", 2023-11-24T07:58:37.820 INFO:teuthology.orchestra.run.smithi059.stdout: "mon": "allow profile mgr", 2023-11-24T07:58:37.821 INFO:teuthology.orchestra.run.smithi059.stdout: "osd": "allow *" 2023-11-24T07:58:37.821 INFO:teuthology.orchestra.run.smithi059.stdout: } 2023-11-24T07:58:37.821 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-11-24T07:58:37.821 INFO:teuthology.orchestra.run.smithi059.stdout: { 2023-11-24T07:58:37.821 INFO:teuthology.orchestra.run.smithi059.stdout: "entity": "mgr.y", 2023-11-24T07:58:37.821 INFO:teuthology.orchestra.run.smithi059.stdout: "key": "AQD0VmBlpXXaHxAAJMwwht1n+zRbRZuGOD5WTQ==", 2023-11-24T07:58:37.821 INFO:teuthology.orchestra.run.smithi059.stdout: "caps": { 2023-11-24T07:58:37.821 INFO:teuthology.orchestra.run.smithi059.stdout: "mds": "allow *", 2023-11-24T07:58:37.821 INFO:teuthology.orchestra.run.smithi059.stdout: "mon": "allow profile mgr", 2023-11-24T07:58:37.821 INFO:teuthology.orchestra.run.smithi059.stdout: "osd": "allow *" 2023-11-24T07:58:37.821 INFO:teuthology.orchestra.run.smithi059.stdout: } 2023-11-24T07:58:37.821 INFO:teuthology.orchestra.run.smithi059.stdout: } 2023-11-24T07:58:37.822 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2023-11-24T07:58:37.822 INFO:teuthology.orchestra.run.smithi059.stdout:} 2023-11-24T07:58:37.823 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:37.806+0000 7fd91d55d700 1 -- 172.21.15.59:0/1429860118 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7fd90406b110 msgr2=0x7fd90406d5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:37.823 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:37.806+0000 7fd91d55d700 1 --2- 172.21.15.59:0/1429860118 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7fd90406b110 0x7fd90406d5d0 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7fd90c000f60 tx=0x7fd90c007d90 comp rx=0 tx=0).stop 2023-11-24T07:58:37.823 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:37.806+0000 7fd91d55d700 1 -- 172.21.15.59:0/1429860118 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd91806e480 msgr2=0x7fd9180c8f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:37.823 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:37.806+0000 7fd91d55d700 1 --2- 172.21.15.59:0/1429860118 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd91806e480 0x7fd9180c8f40 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 crypto rx=0x7fd9080090b0 tx=0x7fd908004750 comp rx=0 tx=0).stop 2023-11-24T07:58:37.823 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:37.806+0000 7fd91d55d700 1 -- 172.21.15.59:0/1429860118 shutdown_connections 2023-11-24T07:58:37.824 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:37.806+0000 7fd91d55d700 1 --2- 172.21.15.59:0/1429860118 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7fd90406b110 0x7fd90406d5d0 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:37.824 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:37.806+0000 7fd91d55d700 1 --2- 172.21.15.59:0/1429860118 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fd9181445f0 0x7fd9181f37d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:37.824 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:37.806+0000 7fd91d55d700 1 --2- 172.21.15.59:0/1429860118 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd91806cae0 0x7fd9181549a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:37.824 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:37.806+0000 7fd91d55d700 1 --2- 172.21.15.59:0/1429860118 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd91806e480 0x7fd9180c8f40 unknown :-1 s=CLOSED pgs=684 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:37.824 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:37.806+0000 7fd91d55d700 1 -- 172.21.15.59:0/1429860118 >> 172.21.15.59:0/1429860118 conn(0x7fd9180cf980 msgr2=0x7fd9180cb2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:58:37.824 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:37.806+0000 7fd91d55d700 1 -- 172.21.15.59:0/1429860118 shutdown_connections 2023-11-24T07:58:37.824 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:37.806+0000 7fd91d55d700 1 -- 172.21.15.59:0/1429860118 wait complete. 2023-11-24T07:58:37.824 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-24T07:58:37.950 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:37.948+0000 7f75d3b89700 1 Processor -- start 2023-11-24T07:58:37.950 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:37.948+0000 7f75d3b89700 1 -- start start 2023-11-24T07:58:37.950 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:37.949+0000 7f75d3b89700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f75cc06fc50 0x7f75cc06eef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:37.951 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:37.949+0000 7f75d3b89700 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f75cc0d1230 con 0x7f75cc15e1c0 2023-11-24T07:58:37.951 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:37.949+0000 7f75d3b89700 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f75cc0d0c70 con 0x7f75cc0b0e90 2023-11-24T07:58:37.951 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:37.949+0000 7f75d3b89700 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f75cc0d0f50 con 0x7f75cc06fc50 2023-11-24T07:58:37.951 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:37.949+0000 7f75d1124700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f75cc06fc50 0x7f75cc06eef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:37.951 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:37.949+0000 7f75d1925700 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7f75cc15e1c0 0x7f75cc06f540 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:57156/0 (socket says 172.21.15.59:57156) 2023-11-24T07:58:37.951 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:37.949+0000 7f75d1925700 1 -- 172.21.15.59:0/2158762063 learned_addr learned my addr 172.21.15.59:0/2158762063 (peer_addr_for_me v1:172.21.15.59:0/0) 2023-11-24T07:58:37.952 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:37.949+0000 7f75d1124700 1 -- 172.21.15.59:0/2158762063 >> v1:172.21.15.76:6789/0 conn(0x7f75cc0b0e90 legacy=0x7f75cc06ca10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:37.953 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:37.949+0000 7f75d1124700 1 -- 172.21.15.59:0/2158762063 >> v1:172.21.15.59:6789/0 conn(0x7f75cc15e1c0 legacy=0x7f75cc06f540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:37.953 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:37.949+0000 7f75d1124700 1 -- 172.21.15.59:0/2158762063 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f75cc06c210 con 0x7f75cc06fc50 2023-11-24T07:58:37.953 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:37.949+0000 7f75d0923700 1 -- 172.21.15.59:0/2158762063 <== mon.0 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2986367804 0 0) 0x7f75cc0d1230 con 0x7f75cc15e1c0 2023-11-24T07:58:37.953 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:37.950+0000 7f75d1124700 1 --2- 172.21.15.59:0/2158762063 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f75cc06fc50 0x7f75cc06eef0 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 crypto rx=0x7f75c8009840 tx=0x7f75c8009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9aa5890ce8fdaeee server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:58:37.953 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:37.950+0000 7f75d0923700 1 -- 172.21.15.59:0/2158762063 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75c8011830 con 0x7f75cc06fc50 2023-11-24T07:58:37.953 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:37.950+0000 7f75d0923700 1 -- 172.21.15.59:0/2158762063 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f75c8011990 con 0x7f75cc06fc50 2023-11-24T07:58:37.953 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:37.950+0000 7f75d0923700 1 -- 172.21.15.59:0/2158762063 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75c8011d20 con 0x7f75cc06fc50 2023-11-24T07:58:37.953 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:37.950+0000 7f75d3b89700 1 -- 172.21.15.59:0/2158762063 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f75cc06fc50 msgr2=0x7f75cc06eef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:37.953 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:37.950+0000 7f75d3b89700 1 --2- 172.21.15.59:0/2158762063 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f75cc06fc50 0x7f75cc06eef0 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 crypto rx=0x7f75c8009840 tx=0x7f75c8009b50 comp rx=0 tx=0).stop 2023-11-24T07:58:37.953 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:37.950+0000 7f75d3b89700 1 -- 172.21.15.59:0/2158762063 shutdown_connections 2023-11-24T07:58:37.954 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:37.950+0000 7f75d3b89700 1 --2- 172.21.15.59:0/2158762063 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f75cc06fc50 0x7f75cc06eef0 unknown :-1 s=CLOSED pgs=686 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:37.954 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:37.950+0000 7f75d3b89700 1 -- 172.21.15.59:0/2158762063 >> 172.21.15.59:0/2158762063 conn(0x7f75cc0b3720 msgr2=0x7f75cc0b3b30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:58:37.954 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:37.950+0000 7f75d3b89700 1 -- 172.21.15.59:0/2158762063 shutdown_connections 2023-11-24T07:58:37.954 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:37.950+0000 7f75d3b89700 1 -- 172.21.15.59:0/2158762063 wait complete. 2023-11-24T07:58:37.956 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:37.952+0000 7f75d3b89700 1 Processor -- start 2023-11-24T07:58:37.956 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:37.952+0000 7f75d3b89700 1 -- start start 2023-11-24T07:58:37.956 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:37.952+0000 7f75d3b89700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f75cc06fc50 0x7f75cc1f3370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:37.956 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:37.952+0000 7f75d3b89700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f75cc0b0e90 0x7f75cc14bce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:37.956 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:37.952+0000 7f75d3b89700 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f75cc15e1c0 0x7f75cc0b6ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:37.956 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:37.952+0000 7f75d3b89700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75cc1f38b0 con 0x7f75cc06fc50 2023-11-24T07:58:37.957 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:37.952+0000 7f75d3b89700 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75cc148b60 con 0x7f75cc0b0e90 2023-11-24T07:58:37.957 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:37.952+0000 7f75d3b89700 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f75cc148e40 con 0x7f75cc15e1c0 2023-11-24T07:58:37.957 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:37.953+0000 7f75d1925700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f75cc06fc50 0x7f75cc1f3370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:37.957 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:37.953+0000 7f75d1925700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f75cc06fc50 0x7f75cc1f3370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:53688/0 (socket says 172.21.15.59:53688) 2023-11-24T07:58:37.957 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:37.953+0000 7f75d1925700 1 -- 172.21.15.59:0/1383664462 learned_addr learned my addr 172.21.15.59:0/1383664462 (peer_addr_for_me v2:172.21.15.59:0/0) 2023-11-24T07:58:37.957 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:37.953+0000 7f75d1124700 1 --2- 172.21.15.59:0/1383664462 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f75cc0b0e90 0x7f75cc14bce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:37.957 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:37.953+0000 7f75d2126700 1 --2- 172.21.15.59:0/1383664462 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f75cc15e1c0 0x7f75cc0b6ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:37.957 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:37.953+0000 7f75d1925700 1 -- 172.21.15.59:0/1383664462 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f75cc15e1c0 msgr2=0x7f75cc0b6ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:37.957 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:37.953+0000 7f75d1925700 1 --2- 172.21.15.59:0/1383664462 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f75cc15e1c0 0x7f75cc0b6ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:37.957 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:37.953+0000 7f75d1925700 1 -- 172.21.15.59:0/1383664462 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f75cc0b0e90 msgr2=0x7f75cc14bce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:37.957 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:37.953+0000 7f75d1925700 1 --2- 172.21.15.59:0/1383664462 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f75cc0b0e90 0x7f75cc14bce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:37.957 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:37.953+0000 7f75d1925700 1 -- 172.21.15.59:0/1383664462 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f75c80094a0 con 0x7f75cc06fc50 2023-11-24T07:58:37.958 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:37.953+0000 7f75d1925700 1 --2- 172.21.15.59:0/1383664462 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f75cc06fc50 0x7f75cc1f3370 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x7f75bc00c520 tx=0x7f75bc00c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:58:37.958 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:37.953+0000 7f75c27fc700 1 -- 172.21.15.59:0/1383664462 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75bc004c80 con 0x7f75cc06fc50 2023-11-24T07:58:37.958 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:37.953+0000 7f75c27fc700 1 -- 172.21.15.59:0/1383664462 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f75bc004e20 con 0x7f75cc06fc50 2023-11-24T07:58:37.958 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:37.953+0000 7f75d3b89700 1 -- 172.21.15.59:0/1383664462 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f75cc149120 con 0x7f75cc06fc50 2023-11-24T07:58:37.958 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:37.953+0000 7f75c27fc700 1 -- 172.21.15.59:0/1383664462 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75bc007310 con 0x7f75cc06fc50 2023-11-24T07:58:37.958 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:37.953+0000 7f75d3b89700 1 -- 172.21.15.59:0/1383664462 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f75cc1496e0 con 0x7f75cc06fc50 2023-11-24T07:58:37.958 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:37.954+0000 7f75d3b89700 1 -- 172.21.15.59:0/1383664462 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f75cc149a50 con 0x7f75cc06fc50 2023-11-24T07:58:37.958 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:37.955+0000 7f75c27fc700 1 -- 172.21.15.59:0/1383664462 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 8) v1 ==== 89435+0+0 (secure 0 0 0) 0x7f75bc007470 con 0x7f75cc06fc50 2023-11-24T07:58:37.959 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:37.955+0000 7f75c27fc700 1 --2- 172.21.15.59:0/1383664462 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f75b006b110 0x7f75b006d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:37.959 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:37.955+0000 7f75c27fc700 1 -- 172.21.15.59:0/1383664462 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(56..56 src has 1..56) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f75bc0893b0 con 0x7f75cc06fc50 2023-11-24T07:58:37.959 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:37.955+0000 7f75d1124700 1 --2- 172.21.15.59:0/1383664462 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f75b006b110 0x7f75b006d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:37.959 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:37.956+0000 7f75d1124700 1 --2- 172.21.15.59:0/1383664462 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f75b006b110 0x7f75b006d5d0 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f75cc1f6f10 tx=0x7f75c801c040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:58:37.961 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:37.960+0000 7f75c27fc700 1 -- 172.21.15.59:0/1383664462 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f75bc054450 con 0x7f75cc06fc50 2023-11-24T07:58:38.161 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:38.160+0000 7f75d3b89700 1 -- 172.21.15.59:0/1383664462 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f75cc068670 con 0x7f75cc06fc50 2023-11-24T07:58:38.162 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:38.160+0000 7f75c27fc700 1 -- 172.21.15.59:0/1383664462 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v56) v1 ==== 74+0+12785 (secure 0 0 0) 0x7f75bc057a70 con 0x7f75cc06fc50 2023-11-24T07:58:38.162 INFO:teuthology.orchestra.run.smithi059.stdout: 2023-11-24T07:58:38.162 INFO:teuthology.orchestra.run.smithi059.stdout:{"epoch":56,"fsid":"9f0e8741-b8f9-48d5-a2c3-09f45903303d","created":"2023-11-24T07:56:06.552729+0000","modified":"2023-11-24T07:58:37.359143+0000","last_up_change":"2023-11-24T07:56:17.372352+0000","last_in_change":"2023-11-24T07:56:09.073250+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-24T07:56:18.352823+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"6d8f9e9b-1782-4e41-928b-a39f2a8ebc06","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6800","nonce":3373878772},{"type":"v1","addr":"172.21.15.59:6801","nonce":3373878772}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6802","nonce":3373878772},{"type":"v1","addr":"172.21.15.59:6803","nonce":3373878772}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6806","nonce":3373878772},{"type":"v1","addr":"172.21.15.59:6807","nonce":3373878772}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6804","nonce":3373878772},{"type":"v1","addr":"172.21.15.59:6805","nonce":3373878772}]},"public_addr":"172.21.15.59:6801/3373878772","cluster_addr":"172.21.15.59:6803/3373878772","heartbeat_back_addr":"172.21.15.59:6807/3373878772","heartbeat_front_addr":"172.21.15.59:6805/3373878772","state":["exists","up"]},{"osd":1,"uuid":"e8d882aa-52e1-4c04-9e1e-c44ef9204a5f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6824","nonce":648258563},{"type":"v1","addr":"172.21.15.59:6825","nonce":648258563}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6826","nonce":648258563},{"type":"v1","addr":"172.21.15.59:6827","nonce":648258563}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6830","nonce":648258563},{"type":"v1","addr":"172.21.15.59:6831","nonce":648258563}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6828","nonce":648258563},{"type":"v1","addr":"172.21.15.59:6829","nonce":648258563}]},"public_addr":"172.21.15.59:6825/648258563","cluster_addr":"172.21.15.59:6827/648258563","heartbeat_back_addr":"172.21.15.59:6831/648258563","heartbeat_front_addr":"172.21.15.59:6829/648258563","state":["exists","up"]},{"osd":2,"uuid":"69eebfa0-695b-4b21-90b9-8a3537168a01","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6808","nonce":1840430654},{"type":"v1","addr":"172.21.15.59:6809","nonce":1840430654}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6810","nonce":1840430654},{"type":"v1","addr":"172.21.15.59:6811","nonce":1840430654}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6814","nonce":1840430654},{"type":"v1","addr":"172.21.15.59:6815","nonce":1840430654}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6812","nonce":1840430654},{"type":"v1","addr":"172.21.15.59:6813","nonce":1840430654}]},"public_addr":"172.21.15.59:6809/1840430654","cluster_addr":"172.21.15.59:6811/1840430654","heartbeat_back_addr":"172.21.15.59:6815/1840430654","heartbeat_front_addr":"172.21.15.59:6813/1840430654","state":["exists","up"]},{"osd":3,"uuid":"96fd77fc-f911-46ff-9445-6f0be2daa8f4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6816","nonce":2504025850},{"type":"v1","addr":"172.21.15.59:6817","nonce":2504025850}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6818","nonce":2504025850},{"type":"v1","addr":"172.21.15.59:6819","nonce":2504025850}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6822","nonce":2504025850},{"type":"v1","addr":"172.21.15.59:6823","nonce":2504025850}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6820","nonce":2504025850},{"type":"v1","addr":"172.21.15.59:6821","nonce":2504025850}]},"public_addr":"172.21.15.59:6817/2504025850","cluster_addr":"172.21.15.59:6819/2504025850","heartbeat_back_addr":"172.21.15.59:6823/2504025850","heartbeat_front_addr":"172.21.15.59:6821/2504025850","state":["exists","up"]},{"osd":4,"uuid":"7ac29fca-b5b2-4c95-9d69-2f164cbbc708","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6816","nonce":2382463431},{"type":"v1","addr":"172.21.15.76:6818","nonce":2382463431}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6820","nonce":2382463431},{"type":"v1","addr":"172.21.15.76:6822","nonce":2382463431}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6828","nonce":2382463431},{"type":"v1","addr":"172.21.15.76:6830","nonce":2382463431}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6824","nonce":2382463431},{"type":"v1","addr":"172.21.15.76:6826","nonce":2382463431}]},"public_addr":"172.21.15.76:6818/2382463431","cluster_addr":"172.21.15.76:6822/2382463431","heartbeat_back_addr":"172.21.15.76:6830/2382463431","heartbeat_front_addr":"172.21.15.76:6826/2382463431","state":["exists","up"]},{"osd":5,"uuid":"485bf711-3851-4238-a4f4-51d52e0a9c58","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6800","nonce":329334308},{"type":"v1","addr":"172.21.15.76:6802","nonce":329334308}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6805","nonce":329334308},{"type":"v1","addr":"172.21.15.76:6807","nonce":329334308}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6814","nonce":329334308},{"type":"v1","addr":"172.21.15.76:6815","nonce":329334308}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6810","nonce":329334308},{"type":"v1","addr":"172.21.15.76:6812","nonce":329334308}]},"public_addr":"172.21.15.76:6802/329334308","cluster_addr":"172.21.15.76:6807/329334308","heartbeat_back_addr":"172.21.15.76:6815/329334308","heartbeat_front_addr":"172.21.15.76:6812/329334308","state":["exists","up"]},{"osd":6,"uuid":"72b3f2fc-1e4a-4e32-968d-003647a0286f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6801","nonce":3946579410},{"type":"v1","addr":"172.21.15.76:6803","nonce":3946579410}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6804","nonce":3946579410},{"type":"v1","addr":"172.21.15.76:6806","nonce":3946579410}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6811","nonce":3946579410},{"type":"v1","addr":"172.21.15.76:6813","nonce":3946579410}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6808","nonce":3946579410},{"type":"v1","addr":"172.21.15.76:6809","nonce":3946579410}]},"public_addr":"172.21.15.76:6803/3946579410","cluster_addr":"172.21.15.76:6806/3946579410","heartbeat_back_addr":"172.21.15.76:6813/3946579410","heartbeat_front_addr":"172.21.15.76:6809/3946579410","state":["exists","up"]},{"osd":7,"uuid":"1ddceb90-6c5d-4d1a-ad56-99cb877f8c70","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6817","nonce":841761538},{"type":"v1","addr":"172.21.15.76:6819","nonce":841761538}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6821","nonce":841761538},{"type":"v1","addr":"172.21.15.76:6823","nonce":841761538}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6829","nonce":841761538},{"type":"v1","addr":"172.21.15.76:6831","nonce":841761538}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6825","nonce":841761538},{"type":"v1","addr":"172.21.15.76:6827","nonce":841761538}]},"public_addr":"172.21.15.76:6819/841761538","cluster_addr":"172.21.15.76:6823/841761538","heartbeat_back_addr":"172.21.15.76:6831/841761538","heartbeat_front_addr":"172.21.15.76:6827/841761538","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T07:56:14.294838+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T07:56:14.270779+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T07:56:14.231809+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T07:56:14.260983+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T07:56:14.302997+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T07:56:14.258600+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T07:56:14.251413+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T07:56:14.304727+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-24T07:58:38.164 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:38.162+0000 7f75d3b89700 1 -- 172.21.15.59:0/1383664462 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f75b006b110 msgr2=0x7f75b006d5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:38.164 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:38.162+0000 7f75d3b89700 1 --2- 172.21.15.59:0/1383664462 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f75b006b110 0x7f75b006d5d0 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f75cc1f6f10 tx=0x7f75c801c040 comp rx=0 tx=0).stop 2023-11-24T07:58:38.165 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:38.162+0000 7f75d3b89700 1 -- 172.21.15.59:0/1383664462 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f75cc06fc50 msgr2=0x7f75cc1f3370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:38.165 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:38.162+0000 7f75d3b89700 1 --2- 172.21.15.59:0/1383664462 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f75cc06fc50 0x7f75cc1f3370 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x7f75bc00c520 tx=0x7f75bc00c8e0 comp rx=0 tx=0).stop 2023-11-24T07:58:38.165 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:38.163+0000 7f75d3b89700 1 -- 172.21.15.59:0/1383664462 shutdown_connections 2023-11-24T07:58:38.165 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:38.163+0000 7f75d3b89700 1 --2- 172.21.15.59:0/1383664462 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f75b006b110 0x7f75b006d5d0 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:38.165 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:38.163+0000 7f75d3b89700 1 --2- 172.21.15.59:0/1383664462 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f75cc15e1c0 0x7f75cc0b6ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:38.165 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:38.163+0000 7f75d3b89700 1 --2- 172.21.15.59:0/1383664462 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f75cc06fc50 0x7f75cc1f3370 unknown :-1 s=CLOSED pgs=687 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:38.165 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:38.163+0000 7f75d3b89700 1 --2- 172.21.15.59:0/1383664462 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f75cc0b0e90 0x7f75cc14bce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:38.165 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:38.163+0000 7f75d3b89700 1 -- 172.21.15.59:0/1383664462 >> 172.21.15.59:0/1383664462 conn(0x7f75cc0b3720 msgr2=0x7f75cc0b1820 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:58:38.165 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:38.163+0000 7f75d3b89700 1 -- 172.21.15.59:0/1383664462 shutdown_connections 2023-11-24T07:58:38.165 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:38.163+0000 7f75d3b89700 1 -- 172.21.15.59:0/1383664462 wait complete. 2023-11-24T07:58:38.176 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-24T07:56:18.352823+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-24T07:58:38.177 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-24T07:58:38.304 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:38.302+0000 7f1327fdf700 1 Processor -- start 2023-11-24T07:58:38.304 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:38.302+0000 7f1327fdf700 1 -- start start 2023-11-24T07:58:38.306 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:38.303+0000 7f1327fdf700 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f13200b4240 0x7f1320160330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:38.306 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:38.303+0000 7f1327fdf700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f13200ce140 0x7f13200ce590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:38.307 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:38.303+0000 7f1327fdf700 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f13200b14c0 con 0x7f132015f410 2023-11-24T07:58:38.307 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:38.303+0000 7f1327fdf700 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f13200b46f0 con 0x7f13200ce140 2023-11-24T07:58:38.307 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:38.303+0000 7f1327fdf700 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f13200b3070 con 0x7f13200b4240 2023-11-24T07:58:38.307 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:38.303+0000 7f132657c700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f13200ce140 0x7f13200ce590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:38.307 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:38.303+0000 7f132657c700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f13200ce140 0x7f13200ce590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:53700/0 (socket says 172.21.15.59:53700) 2023-11-24T07:58:38.307 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:38.303+0000 7f132657c700 1 -- 172.21.15.59:0/2319555195 learned_addr learned my addr 172.21.15.59:0/2319555195 (peer_addr_for_me v2:172.21.15.59:0/0) 2023-11-24T07:58:38.307 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:38.304+0000 7f132557a700 1 --2- 172.21.15.59:0/2319555195 >> v2:172.21.15.76:3300/0 conn(0x7f13200b4240 0x7f1320160330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:38.307 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:38.304+0000 7f132657c700 1 -- 172.21.15.59:0/2319555195 >> v2:172.21.15.76:3300/0 conn(0x7f13200b4240 msgr2=0x7f1320160330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T07:58:38.307 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:38.304+0000 7f132657c700 1 --2- 172.21.15.59:0/2319555195 >> v2:172.21.15.76:3300/0 conn(0x7f13200b4240 0x7f1320160330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:38.308 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:38.304+0000 7f132657c700 1 -- 172.21.15.59:0/2319555195 >> v1:172.21.15.76:6789/0 conn(0x7f132015f410 legacy=0x7f132015fc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T07:58:38.308 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:38.304+0000 7f132657c700 1 -- 172.21.15.59:0/2319555195 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f13200b3270 con 0x7f13200ce140 2023-11-24T07:58:38.308 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:38.304+0000 7f132657c700 1 --2- 172.21.15.59:0/2319555195 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f13200ce140 0x7f13200ce590 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 crypto rx=0x7f13200cdd50 tx=0x7f131800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=921b4735b1555760 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:58:38.308 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:38.304+0000 7f1324d79700 1 -- 172.21.15.59:0/2319555195 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f131800e070 con 0x7f13200ce140 2023-11-24T07:58:38.308 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:38.304+0000 7f1324d79700 1 -- 172.21.15.59:0/2319555195 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1318003a20 con 0x7f13200ce140 2023-11-24T07:58:38.308 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:38.304+0000 7f1324d79700 1 -- 172.21.15.59:0/2319555195 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1318003dd0 con 0x7f13200ce140 2023-11-24T07:58:38.308 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:38.305+0000 7f1327fdf700 1 -- 172.21.15.59:0/2319555195 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f13200ce140 msgr2=0x7f13200ce590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:38.308 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:38.305+0000 7f1327fdf700 1 --2- 172.21.15.59:0/2319555195 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f13200ce140 0x7f13200ce590 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 crypto rx=0x7f13200cdd50 tx=0x7f131800b330 comp rx=0 tx=0).stop 2023-11-24T07:58:38.308 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:38.305+0000 7f1327fdf700 1 -- 172.21.15.59:0/2319555195 shutdown_connections 2023-11-24T07:58:38.308 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:38.305+0000 7f1327fdf700 1 --2- 172.21.15.59:0/2319555195 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f13200ce140 0x7f13200ce590 unknown :-1 s=CLOSED pgs=688 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:38.308 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:38.305+0000 7f1327fdf700 1 --2- 172.21.15.59:0/2319555195 >> v2:172.21.15.76:3300/0 conn(0x7f13200b4240 0x7f1320160330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:38.308 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:38.305+0000 7f1327fdf700 1 -- 172.21.15.59:0/2319555195 >> 172.21.15.59:0/2319555195 conn(0x7f13200b3720 msgr2=0x7f13200b3b30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:58:38.309 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:38.305+0000 7f1327fdf700 1 -- 172.21.15.59:0/2319555195 shutdown_connections 2023-11-24T07:58:38.309 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:38.305+0000 7f1327fdf700 1 -- 172.21.15.59:0/2319555195 wait complete. 2023-11-24T07:58:38.311 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:38.306+0000 7f1327fdf700 1 Processor -- start 2023-11-24T07:58:38.311 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:38.306+0000 7f1327fdf700 1 -- start start 2023-11-24T07:58:38.311 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:38.307+0000 7f1327fdf700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f13200b4240 0x7f1320151030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:38.311 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:38.307+0000 7f1327fdf700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f13200ce140 0x7f13200c7530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:38.311 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:38.307+0000 7f1327fdf700 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f132015f410 0x7f13200c7a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:38.311 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:38.307+0000 7f1327fdf700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13200c6370 con 0x7f13200b4240 2023-11-24T07:58:38.312 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:38.307+0000 7f1327fdf700 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13200c6650 con 0x7f13200ce140 2023-11-24T07:58:38.312 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:38.307+0000 7f1327fdf700 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f13200c6930 con 0x7f132015f410 2023-11-24T07:58:38.312 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:38.307+0000 7f1325d7b700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f13200b4240 0x7f1320151030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:38.312 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:38.307+0000 7f1325d7b700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f13200b4240 0x7f1320151030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:53702/0 (socket says 172.21.15.59:53702) 2023-11-24T07:58:38.312 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:38.307+0000 7f1325d7b700 1 -- 172.21.15.59:0/4104023581 learned_addr learned my addr 172.21.15.59:0/4104023581 (peer_addr_for_me v2:172.21.15.59:0/0) 2023-11-24T07:58:38.312 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:38.307+0000 7f132557a700 1 --2- 172.21.15.59:0/4104023581 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f13200ce140 0x7f13200c7530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:38.312 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:38.307+0000 7f132657c700 1 --2- 172.21.15.59:0/4104023581 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f132015f410 0x7f13200c7a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:38.312 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:38.307+0000 7f1325d7b700 1 -- 172.21.15.59:0/4104023581 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f132015f410 msgr2=0x7f13200c7a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:38.312 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:38.307+0000 7f1325d7b700 1 --2- 172.21.15.59:0/4104023581 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f132015f410 0x7f13200c7a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:38.312 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:38.307+0000 7f1325d7b700 1 -- 172.21.15.59:0/4104023581 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f13200ce140 msgr2=0x7f13200c7530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:38.312 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:38.307+0000 7f1325d7b700 1 --2- 172.21.15.59:0/4104023581 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f13200ce140 0x7f13200c7530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:38.312 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:38.307+0000 7f1325d7b700 1 -- 172.21.15.59:0/4104023581 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1318009b30 con 0x7f13200b4240 2023-11-24T07:58:38.313 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:38.307+0000 7f132657c700 1 --2- 172.21.15.59:0/4104023581 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f132015f410 0x7f13200c7a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-24T07:58:38.313 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:38.307+0000 7f132557a700 1 --2- 172.21.15.59:0/4104023581 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f13200ce140 0x7f13200c7530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-24T07:58:38.313 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:38.307+0000 7f1325d7b700 1 --2- 172.21.15.59:0/4104023581 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f13200b4240 0x7f1320151030 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 crypto rx=0x7f131000c4d0 tx=0x7f131000c890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:58:38.313 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:38.307+0000 7f13167fc700 1 -- 172.21.15.59:0/4104023581 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1310011790 con 0x7f13200b4240 2023-11-24T07:58:38.313 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:38.308+0000 7f13167fc700 1 -- 172.21.15.59:0/4104023581 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f13100118f0 con 0x7f13200b4240 2023-11-24T07:58:38.313 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:38.308+0000 7f13167fc700 1 -- 172.21.15.59:0/4104023581 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1310011ca0 con 0x7f13200b4240 2023-11-24T07:58:38.313 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:38.308+0000 7f1327fdf700 1 -- 172.21.15.59:0/4104023581 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f13200c6c10 con 0x7f13200b4240 2023-11-24T07:58:38.313 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:38.308+0000 7f1327fdf700 1 -- 172.21.15.59:0/4104023581 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1320151570 con 0x7f13200b4240 2023-11-24T07:58:38.313 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:38.308+0000 7f1327fdf700 1 -- 172.21.15.59:0/4104023581 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f13200c9480 con 0x7f13200b4240 2023-11-24T07:58:38.313 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:38.309+0000 7f13167fc700 1 -- 172.21.15.59:0/4104023581 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 8) v1 ==== 89435+0+0 (secure 0 0 0) 0x7f131001c460 con 0x7f13200b4240 2023-11-24T07:58:38.313 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:38.309+0000 7f13167fc700 1 --2- 172.21.15.59:0/4104023581 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f130406b110 0x7f130406d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:38.313 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:38.309+0000 7f13167fc700 1 -- 172.21.15.59:0/4104023581 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(56..56 src has 1..56) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f1310089450 con 0x7f13200b4240 2023-11-24T07:58:38.314 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:38.310+0000 7f132557a700 1 --2- 172.21.15.59:0/4104023581 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f130406b110 0x7f130406d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:38.314 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:38.310+0000 7f132557a700 1 --2- 172.21.15.59:0/4104023581 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f130406b110 0x7f130406d5d0 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f132015ee20 tx=0x7f131c008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:58:38.315 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:38.314+0000 7f13167fc700 1 -- 172.21.15.59:0/4104023581 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f13100545c0 con 0x7f13200b4240 2023-11-24T07:58:38.516 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:38.515+0000 7f1327fdf700 1 -- 172.21.15.59:0/4104023581 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f1320075480 con 0x7f13200b4240 2023-11-24T07:58:38.517 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:38.515+0000 7f13167fc700 1 -- 172.21.15.59:0/4104023581 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v56) v1 ==== 93+0+10 (secure 0 0 0) 0x7f1310057be0 con 0x7f13200b4240 2023-11-24T07:58:38.517 INFO:teuthology.orchestra.run.smithi059.stdout:pg_num: 1 2023-11-24T07:58:38.519 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:38.517+0000 7f1327fdf700 1 -- 172.21.15.59:0/4104023581 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f130406b110 msgr2=0x7f130406d5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:38.519 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:38.517+0000 7f1327fdf700 1 --2- 172.21.15.59:0/4104023581 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f130406b110 0x7f130406d5d0 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f132015ee20 tx=0x7f131c008040 comp rx=0 tx=0).stop 2023-11-24T07:58:38.519 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:38.517+0000 7f1327fdf700 1 -- 172.21.15.59:0/4104023581 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f13200b4240 msgr2=0x7f1320151030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:38.519 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:38.517+0000 7f1327fdf700 1 --2- 172.21.15.59:0/4104023581 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f13200b4240 0x7f1320151030 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 crypto rx=0x7f131000c4d0 tx=0x7f131000c890 comp rx=0 tx=0).stop 2023-11-24T07:58:38.519 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:38.517+0000 7f1327fdf700 1 -- 172.21.15.59:0/4104023581 shutdown_connections 2023-11-24T07:58:38.520 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:38.517+0000 7f1327fdf700 1 --2- 172.21.15.59:0/4104023581 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f130406b110 0x7f130406d5d0 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:38.520 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:38.517+0000 7f1327fdf700 1 --2- 172.21.15.59:0/4104023581 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f132015f410 0x7f13200c7a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:38.520 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:38.517+0000 7f1327fdf700 1 --2- 172.21.15.59:0/4104023581 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f13200b4240 0x7f1320151030 unknown :-1 s=CLOSED pgs=689 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:38.520 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:38.517+0000 7f1327fdf700 1 --2- 172.21.15.59:0/4104023581 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f13200ce140 0x7f13200c7530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:38.520 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:38.517+0000 7f1327fdf700 1 -- 172.21.15.59:0/4104023581 >> 172.21.15.59:0/4104023581 conn(0x7f13200b3720 msgr2=0x7f13200b3b00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:58:38.520 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:38.518+0000 7f1327fdf700 1 -- 172.21.15.59:0/4104023581 shutdown_connections 2023-11-24T07:58:38.520 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:38.518+0000 7f1327fdf700 1 -- 172.21.15.59:0/4104023581 wait complete. 2023-11-24T07:58:38.531 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2023-11-24T07:58:38.657 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:38.655+0000 7fd78acf1700 1 Processor -- start 2023-11-24T07:58:38.657 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:38.655+0000 7fd78acf1700 1 -- start start 2023-11-24T07:58:38.660 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:38.656+0000 7fd78acf1700 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fd7840b5ab0 0x7fd78406e560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:38.660 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:38.656+0000 7fd78acf1700 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd784071f50 con 0x7fd78406b620 2023-11-24T07:58:38.660 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:38.656+0000 7fd78acf1700 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd784072230 con 0x7fd7840b4150 2023-11-24T07:58:38.661 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:38.656+0000 7fd78acf1700 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd784072510 con 0x7fd7840b5ab0 2023-11-24T07:58:38.661 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:38.656+0000 7fd788a8d700 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7fd78406b620 0x7fd7840b3a40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:57184/0 (socket says 172.21.15.59:57184) 2023-11-24T07:58:38.661 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:38.656+0000 7fd788a8d700 1 -- 172.21.15.59:0/3300805613 learned_addr learned my addr 172.21.15.59:0/3300805613 (peer_addr_for_me v1:172.21.15.59:0/0) 2023-11-24T07:58:38.661 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:38.656+0000 7fd78928e700 1 --2- 172.21.15.59:0/3300805613 >> v2:172.21.15.76:3300/0 conn(0x7fd7840b5ab0 0x7fd78406e560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:38.661 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:38.656+0000 7fd7837fe700 1 -- 172.21.15.59:0/3300805613 <== mon.0 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2771891399 0 0) 0x7fd784071f50 con 0x7fd78406b620 2023-11-24T07:58:38.661 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:38.656+0000 7fd7837fe700 1 -- 172.21.15.59:0/3300805613 --> v1:172.21.15.59:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd7740035b0 con 0x7fd78406b620 2023-11-24T07:58:38.661 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:38.657+0000 7fd7837fe700 1 -- 172.21.15.59:0/3300805613 <== mon.0 v1:172.21.15.59:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2059884634 0 0) 0x7fd7740035b0 con 0x7fd78406b620 2023-11-24T07:58:38.661 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:38.657+0000 7fd7837fe700 1 -- 172.21.15.59:0/3300805613 >> v2:172.21.15.76:3300/0 conn(0x7fd7840b5ab0 msgr2=0x7fd78406e560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T07:58:38.661 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:38.657+0000 7fd7837fe700 1 --2- 172.21.15.59:0/3300805613 >> v2:172.21.15.76:3300/0 conn(0x7fd7840b5ab0 0x7fd78406e560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:38.661 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:38.657+0000 7fd7837fe700 1 -- 172.21.15.59:0/3300805613 >> v1:172.21.15.76:6789/0 conn(0x7fd7840b4150 legacy=0x7fd78406de50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:38.661 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:38.657+0000 7fd7837fe700 1 -- 172.21.15.59:0/3300805613 --> v1:172.21.15.59:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd7840727f0 con 0x7fd78406b620 2023-11-24T07:58:38.661 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:38.657+0000 7fd7837fe700 1 -- 172.21.15.59:0/3300805613 <== mon.0 v1:172.21.15.59:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1312333566 0 0) 0x7fd778001670 con 0x7fd78406b620 2023-11-24T07:58:38.662 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:38.657+0000 7fd7837fe700 1 -- 172.21.15.59:0/3300805613 >> v1:172.21.15.59:6789/0 conn(0x7fd78406b620 legacy=0x7fd7840b3a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:38.662 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:38.657+0000 7fd7837fe700 1 --2- 172.21.15.59:0/3300805613 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd774003a00 0x7fd774003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:38.662 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:38.657+0000 7fd7837fe700 1 --2- 172.21.15.59:0/3300805613 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd774004400 0x7fd7740048d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:38.662 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:38.657+0000 7fd7837fe700 1 --2- 172.21.15.59:0/3300805613 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fd7740050d0 0x7fd77400ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:38.662 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:38.657+0000 7fd7837fe700 1 -- 172.21.15.59:0/3300805613 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd784072510 con 0x7fd774003a00 2023-11-24T07:58:38.662 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:38.657+0000 7fd7837fe700 1 -- 172.21.15.59:0/3300805613 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd77400c030 con 0x7fd774004400 2023-11-24T07:58:38.662 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:38.657+0000 7fd7837fe700 1 -- 172.21.15.59:0/3300805613 --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd77400c1f0 con 0x7fd7740050d0 2023-11-24T07:58:38.662 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:38.657+0000 7fd788a8d700 1 --2- 172.21.15.59:0/3300805613 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd774003a00 0x7fd774003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:38.662 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:38.657+0000 7fd783fff700 1 --2- 172.21.15.59:0/3300805613 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd774004400 0x7fd7740048d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:38.662 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:38.657+0000 7fd78928e700 1 --2- 172.21.15.59:0/3300805613 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fd7740050d0 0x7fd77400ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:38.662 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:38.657+0000 7fd788a8d700 1 -- 172.21.15.59:0/3300805613 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fd7740050d0 msgr2=0x7fd77400ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T07:58:38.663 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:38.657+0000 7fd788a8d700 1 --2- 172.21.15.59:0/3300805613 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fd7740050d0 0x7fd77400ba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:38.663 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:38.657+0000 7fd788a8d700 1 -- 172.21.15.59:0/3300805613 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd774004400 msgr2=0x7fd7740048d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T07:58:38.663 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:38.657+0000 7fd788a8d700 1 --2- 172.21.15.59:0/3300805613 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd774004400 0x7fd7740048d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:38.663 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:38.657+0000 7fd78928e700 1 -- 172.21.15.59:0/3300805613 reap_dead start 2023-11-24T07:58:38.663 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:38.657+0000 7fd788a8d700 1 -- 172.21.15.59:0/3300805613 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd7740035b0 con 0x7fd774003a00 2023-11-24T07:58:38.663 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:38.657+0000 7fd783fff700 1 --2- 172.21.15.59:0/3300805613 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd774004400 0x7fd7740048d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-24T07:58:38.663 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:38.657+0000 7fd788a8d700 1 --2- 172.21.15.59:0/3300805613 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd774003a00 0x7fd774003de0 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 crypto rx=0x7fd77800c2f0 tx=0x7fd774005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1179b8db56c81587 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:58:38.663 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:38.658+0000 7fd7837fe700 1 -- 172.21.15.59:0/3300805613 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd77800cdb0 con 0x7fd774003a00 2023-11-24T07:58:38.663 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:38.658+0000 7fd7837fe700 1 -- 172.21.15.59:0/3300805613 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd778005960 con 0x7fd774003a00 2023-11-24T07:58:38.663 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:38.658+0000 7fd7837fe700 1 -- 172.21.15.59:0/3300805613 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd778005cf0 con 0x7fd774003a00 2023-11-24T07:58:38.663 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:38.658+0000 7fd78acf1700 1 -- 172.21.15.59:0/3300805613 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd774003a00 msgr2=0x7fd774003de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:38.663 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:38.658+0000 7fd78acf1700 1 --2- 172.21.15.59:0/3300805613 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd774003a00 0x7fd774003de0 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 crypto rx=0x7fd77800c2f0 tx=0x7fd774005880 comp rx=0 tx=0).stop 2023-11-24T07:58:38.664 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:38.658+0000 7fd78acf1700 1 -- 172.21.15.59:0/3300805613 shutdown_connections 2023-11-24T07:58:38.664 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:38.658+0000 7fd78acf1700 1 --2- 172.21.15.59:0/3300805613 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd774003a00 0x7fd774003de0 unknown :-1 s=CLOSED pgs=691 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:38.664 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:38.658+0000 7fd78acf1700 1 -- 172.21.15.59:0/3300805613 >> 172.21.15.59:0/3300805613 conn(0x7fd7840b2740 msgr2=0x7fd7840b2b20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:58:38.664 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:38.658+0000 7fd78acf1700 1 -- 172.21.15.59:0/3300805613 shutdown_connections 2023-11-24T07:58:38.664 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:38.658+0000 7fd78acf1700 1 -- 172.21.15.59:0/3300805613 wait complete. 2023-11-24T07:58:38.667 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:38.660+0000 7fd78acf1700 1 Processor -- start 2023-11-24T07:58:38.667 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:38.660+0000 7fd78acf1700 1 -- start start 2023-11-24T07:58:38.667 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:38.661+0000 7fd78acf1700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd774003a00 0x7fd784131c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:38.667 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:38.661+0000 7fd78acf1700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd7841f37a0 0x7fd7841f7c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:38.667 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:38.661+0000 7fd78acf1700 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fd7840c9140 0x7fd7840c95a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:38.668 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:38.661+0000 7fd78acf1700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7841e5060 con 0x7fd7841f37a0 2023-11-24T07:58:38.668 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:38.661+0000 7fd78acf1700 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd78406df90 con 0x7fd774003a00 2023-11-24T07:58:38.668 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:38.661+0000 7fd78acf1700 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd7840c9ae0 con 0x7fd7840c9140 2023-11-24T07:58:38.668 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:38.661+0000 7fd783fff700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd7841f37a0 0x7fd7841f7c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:38.668 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:38.661+0000 7fd783fff700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd7841f37a0 0x7fd7841f7c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:53724/0 (socket says 172.21.15.59:53724) 2023-11-24T07:58:38.668 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:38.661+0000 7fd783fff700 1 -- 172.21.15.59:0/815956445 learned_addr learned my addr 172.21.15.59:0/815956445 (peer_addr_for_me v2:172.21.15.59:0/0) 2023-11-24T07:58:38.668 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:38.661+0000 7fd788a8d700 1 --2- 172.21.15.59:0/815956445 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd774003a00 0x7fd784131c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:38.668 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:38.661+0000 7fd78928e700 1 --2- 172.21.15.59:0/815956445 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fd7840c9140 0x7fd7840c95a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:38.668 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:38.661+0000 7fd783fff700 1 -- 172.21.15.59:0/815956445 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fd7840c9140 msgr2=0x7fd7840c95a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:38.669 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:38.661+0000 7fd783fff700 1 --2- 172.21.15.59:0/815956445 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fd7840c9140 0x7fd7840c95a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:38.669 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:38.661+0000 7fd783fff700 1 -- 172.21.15.59:0/815956445 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd774003a00 msgr2=0x7fd784131c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:38.669 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:38.661+0000 7fd783fff700 1 --2- 172.21.15.59:0/815956445 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd774003a00 0x7fd784131c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:38.669 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:38.661+0000 7fd783fff700 1 -- 172.21.15.59:0/815956445 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd784132160 con 0x7fd7841f37a0 2023-11-24T07:58:38.669 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:38.662+0000 7fd783fff700 1 --2- 172.21.15.59:0/815956445 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd7841f37a0 0x7fd7841f7c10 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 crypto rx=0x7fd77000e780 tx=0x7fd77000ea90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:58:38.669 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:38.662+0000 7fd7817fa700 1 -- 172.21.15.59:0/815956445 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7700052e0 con 0x7fd7841f37a0 2023-11-24T07:58:38.669 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:38.662+0000 7fd7817fa700 1 -- 172.21.15.59:0/815956445 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd770005440 con 0x7fd7841f37a0 2023-11-24T07:58:38.669 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:38.662+0000 7fd78acf1700 1 -- 172.21.15.59:0/815956445 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd7841f8150 con 0x7fd7841f37a0 2023-11-24T07:58:38.669 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:38.662+0000 7fd7817fa700 1 -- 172.21.15.59:0/815956445 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7700038a0 con 0x7fd7841f37a0 2023-11-24T07:58:38.669 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:38.662+0000 7fd78acf1700 1 -- 172.21.15.59:0/815956445 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd78406e1e0 con 0x7fd7841f37a0 2023-11-24T07:58:38.670 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:38.663+0000 7fd78acf1700 1 -- 172.21.15.59:0/815956445 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd784068670 con 0x7fd7841f37a0 2023-11-24T07:58:38.670 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:38.663+0000 7fd7817fa700 1 -- 172.21.15.59:0/815956445 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 8) v1 ==== 89435+0+0 (secure 0 0 0) 0x7fd770003a00 con 0x7fd7841f37a0 2023-11-24T07:58:38.670 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:38.663+0000 7fd7817fa700 1 --2- 172.21.15.59:0/815956445 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7fd76806b110 0x7fd76806d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:38.670 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:38.664+0000 7fd7817fa700 1 -- 172.21.15.59:0/815956445 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(56..56 src has 1..56) v4 ==== 4857+0+0 (secure 0 0 0) 0x7fd770088c20 con 0x7fd7841f37a0 2023-11-24T07:58:38.670 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:38.664+0000 7fd788a8d700 1 --2- 172.21.15.59:0/815956445 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7fd76806b110 0x7fd76806d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:38.670 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:38.664+0000 7fd788a8d700 1 --2- 172.21.15.59:0/815956445 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7fd76806b110 0x7fd76806d5d0 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7fd778000c00 tx=0x7fd778007000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:58:38.670 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:38.668+0000 7fd7817fa700 1 -- 172.21.15.59:0/815956445 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fd770053e30 con 0x7fd7841f37a0 2023-11-24T07:58:38.875 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:38.873+0000 7fd78acf1700 1 -- 172.21.15.59:0/815956445 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7fd78406bd40 con 0x7fd7841f37a0 2023-11-24T07:58:38.876 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:38.874+0000 7fd7817fa700 1 -- 172.21.15.59:0/815956445 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v56) v1 ==== 84+0+65 (secure 0 0 0) 0x7fd770019020 con 0x7fd7841f37a0 2023-11-24T07:58:38.876 INFO:teuthology.orchestra.run.smithi059.stdout: 2023-11-24T07:58:38.876 INFO:teuthology.orchestra.run.smithi059.stdout:[ 2023-11-24T07:58:38.876 INFO:teuthology.orchestra.run.smithi059.stdout: { 2023-11-24T07:58:38.876 INFO:teuthology.orchestra.run.smithi059.stdout: "poolnum": 1, 2023-11-24T07:58:38.876 INFO:teuthology.orchestra.run.smithi059.stdout: "poolname": ".mgr" 2023-11-24T07:58:38.876 INFO:teuthology.orchestra.run.smithi059.stdout: } 2023-11-24T07:58:38.876 INFO:teuthology.orchestra.run.smithi059.stdout:] 2023-11-24T07:58:38.877 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:38.876+0000 7fd78acf1700 1 -- 172.21.15.59:0/815956445 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7fd76806b110 msgr2=0x7fd76806d5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:38.878 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:38.876+0000 7fd78acf1700 1 --2- 172.21.15.59:0/815956445 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7fd76806b110 0x7fd76806d5d0 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7fd778000c00 tx=0x7fd778007000 comp rx=0 tx=0).stop 2023-11-24T07:58:38.878 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:38.876+0000 7fd78acf1700 1 -- 172.21.15.59:0/815956445 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd7841f37a0 msgr2=0x7fd7841f7c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:38.878 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:38.876+0000 7fd78acf1700 1 --2- 172.21.15.59:0/815956445 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd7841f37a0 0x7fd7841f7c10 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 crypto rx=0x7fd77000e780 tx=0x7fd77000ea90 comp rx=0 tx=0).stop 2023-11-24T07:58:38.878 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:38.876+0000 7fd78acf1700 1 -- 172.21.15.59:0/815956445 shutdown_connections 2023-11-24T07:58:38.878 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:38.876+0000 7fd78acf1700 1 --2- 172.21.15.59:0/815956445 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7fd76806b110 0x7fd76806d5d0 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:38.878 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:38.876+0000 7fd78acf1700 1 --2- 172.21.15.59:0/815956445 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fd7840c9140 0x7fd7840c95a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:38.878 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:38.876+0000 7fd78acf1700 1 --2- 172.21.15.59:0/815956445 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd774003a00 0x7fd784131c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:38.878 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:38.876+0000 7fd78acf1700 1 --2- 172.21.15.59:0/815956445 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd7841f37a0 0x7fd7841f7c10 unknown :-1 s=CLOSED pgs=692 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:38.878 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:38.876+0000 7fd78acf1700 1 -- 172.21.15.59:0/815956445 >> 172.21.15.59:0/815956445 conn(0x7fd7840b2740 msgr2=0x7fd784146240 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:58:38.879 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:38.876+0000 7fd78acf1700 1 -- 172.21.15.59:0/815956445 shutdown_connections 2023-11-24T07:58:38.879 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:38.876+0000 7fd78acf1700 1 -- 172.21.15.59:0/815956445 wait complete. 2023-11-24T07:58:38.890 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2023-11-24T07:58:38.890 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2023-11-24T07:58:39.019 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:39.018+0000 7f3869896700 1 Processor -- start 2023-11-24T07:58:39.020 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:39.018+0000 7f3869896700 1 -- start start 2023-11-24T07:58:39.029 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:39.019+0000 7f3869896700 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f386415dd30 0x7f386415e110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:39.029 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:39.019+0000 7f3869896700 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f38640d2490 0x7f38640d2900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:39.029 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:39.019+0000 7f3869896700 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f38640d22b0 con 0x7f38640cea30 2023-11-24T07:58:39.029 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:39.019+0000 7f3869896700 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f38640d2e40 con 0x7f38640d2490 2023-11-24T07:58:39.030 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:39.019+0000 7f3869896700 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f386415e650 con 0x7f386415dd30 2023-11-24T07:58:39.030 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:39.019+0000 7f38627fc700 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7f38640cea30 0x7f38640d1ba0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:57206/0 (socket says 172.21.15.59:57206) 2023-11-24T07:58:39.030 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:39.019+0000 7f38627fc700 1 -- 172.21.15.59:0/242035247 learned_addr learned my addr 172.21.15.59:0/242035247 (peer_addr_for_me v1:172.21.15.59:0/0) 2023-11-24T07:58:39.030 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:39.019+0000 7f38637fe700 1 --2- 172.21.15.59:0/242035247 >> v2:172.21.15.76:3300/0 conn(0x7f38640d2490 0x7f38640d2900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:39.030 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:39.019+0000 7f3862ffd700 1 --2- 172.21.15.59:0/242035247 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f386415dd30 0x7f386415e110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:39.031 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:39.019+0000 7f3861ffb700 1 -- 172.21.15.59:0/242035247 <== mon.0 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1371865076 0 0) 0x7f38640d22b0 con 0x7f38640cea30 2023-11-24T07:58:39.031 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:39.019+0000 7f3861ffb700 1 -- 172.21.15.59:0/242035247 --> v1:172.21.15.59:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f384c0035b0 con 0x7f38640cea30 2023-11-24T07:58:39.031 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:39.019+0000 7f3861ffb700 1 -- 172.21.15.59:0/242035247 <== mon.0 v1:172.21.15.59:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1166817786 0 0) 0x7f384c0035b0 con 0x7f38640cea30 2023-11-24T07:58:39.031 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:39.019+0000 7f3861ffb700 1 -- 172.21.15.59:0/242035247 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f386415dd30 msgr2=0x7f386415e110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T07:58:39.031 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:39.019+0000 7f3861ffb700 1 --2- 172.21.15.59:0/242035247 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f386415dd30 0x7f386415e110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:39.031 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:39.019+0000 7f3861ffb700 1 -- 172.21.15.59:0/242035247 >> v2:172.21.15.76:3300/0 conn(0x7f38640d2490 msgr2=0x7f38640d2900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T07:58:39.032 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:39.019+0000 7f3861ffb700 1 --2- 172.21.15.59:0/242035247 >> v2:172.21.15.76:3300/0 conn(0x7f38640d2490 0x7f38640d2900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:39.032 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:39.019+0000 7f3861ffb700 1 -- 172.21.15.59:0/242035247 --> v1:172.21.15.59:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f38640cef10 con 0x7f38640cea30 2023-11-24T07:58:39.032 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:39.019+0000 7f3861ffb700 1 -- 172.21.15.59:0/242035247 <== mon.0 v1:172.21.15.59:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1312333566 0 0) 0x7f3858001670 con 0x7f38640cea30 2023-11-24T07:58:39.032 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:39.019+0000 7f3861ffb700 1 -- 172.21.15.59:0/242035247 >> v1:172.21.15.59:6789/0 conn(0x7f38640cea30 legacy=0x7f38640d1ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:39.032 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:39.019+0000 7f3861ffb700 1 --2- 172.21.15.59:0/242035247 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f384c003a00 0x7f384c003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:39.032 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:39.019+0000 7f3861ffb700 1 --2- 172.21.15.59:0/242035247 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f384c004850 0x7f384c004cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:39.033 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:39.019+0000 7f3861ffb700 1 --2- 172.21.15.59:0/242035247 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f386415dd30 0x7f384c005210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:39.033 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:39.019+0000 7f3861ffb700 1 -- 172.21.15.59:0/242035247 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38640d2e40 con 0x7f384c003a00 2023-11-24T07:58:39.033 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:39.019+0000 7f3861ffb700 1 -- 172.21.15.59:0/242035247 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f386415e650 con 0x7f384c004850 2023-11-24T07:58:39.033 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:39.019+0000 7f3861ffb700 1 -- 172.21.15.59:0/242035247 --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f384c00ba10 con 0x7f386415dd30 2023-11-24T07:58:39.033 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:39.020+0000 7f3862ffd700 1 --2- 172.21.15.59:0/242035247 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f384c003a00 0x7f384c003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:39.034 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:39.020+0000 7f38627fc700 1 --2- 172.21.15.59:0/242035247 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f384c004850 0x7f384c004cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:39.034 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:39.020+0000 7f38637fe700 1 --2- 172.21.15.59:0/242035247 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f386415dd30 0x7f384c005210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:39.034 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:39.020+0000 7f3862ffd700 1 -- 172.21.15.59:0/242035247 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f386415dd30 msgr2=0x7f384c005210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T07:58:39.034 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:39.020+0000 7f3862ffd700 1 --2- 172.21.15.59:0/242035247 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f386415dd30 0x7f384c005210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:39.034 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:39.020+0000 7f3862ffd700 1 -- 172.21.15.59:0/242035247 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f384c004850 msgr2=0x7f384c004cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T07:58:39.034 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:39.020+0000 7f3862ffd700 1 --2- 172.21.15.59:0/242035247 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f384c004850 0x7f384c004cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:39.035 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:39.020+0000 7f3862ffd700 1 -- 172.21.15.59:0/242035247 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f384c0035b0 con 0x7f384c003a00 2023-11-24T07:58:39.035 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:39.020+0000 7f38627fc700 1 --2- 172.21.15.59:0/242035247 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f384c004850 0x7f384c004cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-24T07:58:39.035 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:39.020+0000 7f3862ffd700 1 --2- 172.21.15.59:0/242035247 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f384c003a00 0x7f384c003e20 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 crypto rx=0x7f3854009b60 tx=0x7f384c005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2d2136ba8a9bf300 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:58:39.035 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:39.020+0000 7f3861ffb700 1 -- 172.21.15.59:0/242035247 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f385400e6c0 con 0x7f384c003a00 2023-11-24T07:58:39.035 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:39.020+0000 7f3861ffb700 1 -- 172.21.15.59:0/242035247 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f385400e820 con 0x7f384c003a00 2023-11-24T07:58:39.036 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:39.020+0000 7f3861ffb700 1 -- 172.21.15.59:0/242035247 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f385400eb90 con 0x7f384c003a00 2023-11-24T07:58:39.036 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:39.021+0000 7f3869896700 1 -- 172.21.15.59:0/242035247 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f384c003a00 msgr2=0x7f384c003e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:39.036 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:39.021+0000 7f3869896700 1 --2- 172.21.15.59:0/242035247 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f384c003a00 0x7f384c003e20 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 crypto rx=0x7f3854009b60 tx=0x7f384c005880 comp rx=0 tx=0).stop 2023-11-24T07:58:39.036 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:39.021+0000 7f38637fe700 1 -- 172.21.15.59:0/242035247 reap_dead start 2023-11-24T07:58:39.036 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:39.021+0000 7f3869896700 1 -- 172.21.15.59:0/242035247 shutdown_connections 2023-11-24T07:58:39.036 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:39.021+0000 7f3869896700 1 -- 172.21.15.59:0/242035247 >> 172.21.15.59:0/242035247 conn(0x7f386409d110 msgr2=0x7f38640d3eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:58:39.037 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:39.021+0000 7f3869896700 1 -- 172.21.15.59:0/242035247 shutdown_connections 2023-11-24T07:58:39.037 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:39.021+0000 7f3869896700 1 -- 172.21.15.59:0/242035247 wait complete. 2023-11-24T07:58:39.037 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:39.022+0000 7f3869896700 1 Processor -- start 2023-11-24T07:58:39.037 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:39.022+0000 7f3869896700 1 -- start start 2023-11-24T07:58:39.037 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:39.023+0000 7f3869896700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f384c003a00 0x7f38640c79c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:39.037 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:39.023+0000 7f3869896700 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f38640cbcb0 0x7f3864157630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:39.038 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:39.023+0000 7f3869896700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f38640a1710 0x7f38640c8710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:39.038 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:39.023+0000 7f3869896700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38640caf80 con 0x7f38640a1710 2023-11-24T07:58:39.038 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:39.023+0000 7f3869896700 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3864157b70 con 0x7f384c003a00 2023-11-24T07:58:39.038 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:39.023+0000 7f3869896700 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f38640c8030 con 0x7f38640cbcb0 2023-11-24T07:58:39.038 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:39.023+0000 7f38637fe700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f38640a1710 0x7f38640c8710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:39.039 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:39.023+0000 7f38637fe700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f38640a1710 0x7f38640c8710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:53746/0 (socket says 172.21.15.59:53746) 2023-11-24T07:58:39.039 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:39.023+0000 7f38637fe700 1 -- 172.21.15.59:0/2702302191 learned_addr learned my addr 172.21.15.59:0/2702302191 (peer_addr_for_me v2:172.21.15.59:0/0) 2023-11-24T07:58:39.039 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:39.023+0000 7f38627fc700 1 --2- 172.21.15.59:0/2702302191 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f38640cbcb0 0x7f3864157630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:39.039 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:39.023+0000 7f3862ffd700 1 --2- 172.21.15.59:0/2702302191 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f384c003a00 0x7f38640c79c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:39.039 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:39.023+0000 7f38637fe700 1 -- 172.21.15.59:0/2702302191 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f38640cbcb0 msgr2=0x7f3864157630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:39.039 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:39.023+0000 7f38637fe700 1 --2- 172.21.15.59:0/2702302191 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f38640cbcb0 0x7f3864157630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:39.040 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:39.023+0000 7f38637fe700 1 -- 172.21.15.59:0/2702302191 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f384c003a00 msgr2=0x7f38640c79c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:39.040 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:39.023+0000 7f38637fe700 1 --2- 172.21.15.59:0/2702302191 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f384c003a00 0x7f38640c79c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:39.040 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:39.023+0000 7f38637fe700 1 -- 172.21.15.59:0/2702302191 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f38640cb1d0 con 0x7f38640a1710 2023-11-24T07:58:39.040 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:39.023+0000 7f38637fe700 1 --2- 172.21.15.59:0/2702302191 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f38640a1710 0x7f38640c8710 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 crypto rx=0x7f386406b170 tx=0x7f385c011f60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:58:39.040 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:39.024+0000 7f384bfff700 1 -- 172.21.15.59:0/2702302191 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f385c00e040 con 0x7f38640a1710 2023-11-24T07:58:39.041 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:39.024+0000 7f384bfff700 1 -- 172.21.15.59:0/2702302191 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f385c00f040 con 0x7f38640a1710 2023-11-24T07:58:39.041 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:39.024+0000 7f3869896700 1 -- 172.21.15.59:0/2702302191 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f38640cb4b0 con 0x7f38640a1710 2023-11-24T07:58:39.041 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:39.024+0000 7f384bfff700 1 -- 172.21.15.59:0/2702302191 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f385c00fe40 con 0x7f38640a1710 2023-11-24T07:58:39.041 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:39.024+0000 7f3869896700 1 -- 172.21.15.59:0/2702302191 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f38640a1b90 con 0x7f38640a1710 2023-11-24T07:58:39.041 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:39.025+0000 7f3869896700 1 -- 172.21.15.59:0/2702302191 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3864067f60 con 0x7f38640a1710 2023-11-24T07:58:39.041 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:39.025+0000 7f384bfff700 1 -- 172.21.15.59:0/2702302191 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 8) v1 ==== 89435+0+0 (secure 0 0 0) 0x7f385c015430 con 0x7f38640a1710 2023-11-24T07:58:39.042 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:39.025+0000 7f384bfff700 1 --2- 172.21.15.59:0/2702302191 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f385006b110 0x7f385006d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:39.042 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:39.026+0000 7f384bfff700 1 -- 172.21.15.59:0/2702302191 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(56..56 src has 1..56) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f385c089e40 con 0x7f38640a1710 2023-11-24T07:58:39.042 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:39.026+0000 7f3862ffd700 1 --2- 172.21.15.59:0/2702302191 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f385006b110 0x7f385006d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:39.042 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:39.026+0000 7f3862ffd700 1 --2- 172.21.15.59:0/2702302191 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f385006b110 0x7f385006d5d0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f38540090b0 tx=0x7f3854008030 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:58:39.046 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:39.030+0000 7f384bfff700 1 -- 172.21.15.59:0/2702302191 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f385c054e60 con 0x7f38640a1710 2023-11-24T07:58:39.233 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:39.231+0000 7f3869896700 1 -- 172.21.15.59:0/2702302191 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7f38640cb9d0 con 0x7f38640a1710 2023-11-24T07:58:39.368 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:39.367+0000 7f384bfff700 1 -- 172.21.15.59:0/2702302191 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v57) v1 ==== 138+0+0 (secure 0 0 0) 0x7f38640cb9d0 con 0x7f38640a1710 2023-11-24T07:58:39.372 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:39.370+0000 7f3869896700 1 -- 172.21.15.59:0/2702302191 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f385006b110 msgr2=0x7f385006d5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:39.372 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:39.370+0000 7f3869896700 1 --2- 172.21.15.59:0/2702302191 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f385006b110 0x7f385006d5d0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f38540090b0 tx=0x7f3854008030 comp rx=0 tx=0).stop 2023-11-24T07:58:39.372 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:39.370+0000 7f3869896700 1 -- 172.21.15.59:0/2702302191 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f38640a1710 msgr2=0x7f38640c8710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:39.372 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:39.370+0000 7f3869896700 1 --2- 172.21.15.59:0/2702302191 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f38640a1710 0x7f38640c8710 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 crypto rx=0x7f386406b170 tx=0x7f385c011f60 comp rx=0 tx=0).stop 2023-11-24T07:58:39.372 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:39.370+0000 7f3869896700 1 -- 172.21.15.59:0/2702302191 shutdown_connections 2023-11-24T07:58:39.372 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:39.370+0000 7f3869896700 1 --2- 172.21.15.59:0/2702302191 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f385006b110 0x7f385006d5d0 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:39.373 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:39.370+0000 7f3869896700 1 --2- 172.21.15.59:0/2702302191 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f38640a1710 0x7f38640c8710 unknown :-1 s=CLOSED pgs=695 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:39.373 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:39.370+0000 7f3869896700 1 --2- 172.21.15.59:0/2702302191 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f384c003a00 0x7f38640c79c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:39.373 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:39.370+0000 7f3869896700 1 --2- 172.21.15.59:0/2702302191 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f38640cbcb0 0x7f3864157630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:39.373 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:39.370+0000 7f3869896700 1 -- 172.21.15.59:0/2702302191 >> 172.21.15.59:0/2702302191 conn(0x7f386409d110 msgr2=0x7f38640b91a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:58:39.373 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:39.370+0000 7f3869896700 1 -- 172.21.15.59:0/2702302191 shutdown_connections 2023-11-24T07:58:39.373 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:39.370+0000 7f3869896700 1 -- 172.21.15.59:0/2702302191 wait complete. 2023-11-24T07:58:39.373 INFO:teuthology.orchestra.run.smithi059.stderr:pool 'cephfs_metadata' created 2023-11-24T07:58:39.387 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2023-11-24T07:58:39.520 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:39.518+0000 7fa8318d2700 1 Processor -- start 2023-11-24T07:58:39.520 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:39.518+0000 7fa8318d2700 1 -- start start 2023-11-24T07:58:39.522 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:39.519+0000 7fa8318d2700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fa82c0d0870 0x7fa82c0d26c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:39.522 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:39.519+0000 7fa8318d2700 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fa82c0d49f0 0x7fa82c0b6d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:39.522 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:39.519+0000 7fa8318d2700 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa82c0d4e20 con 0x7fa82c0d0c50 2023-11-24T07:58:39.523 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:39.519+0000 7fa8318d2700 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa82c0d2f80 con 0x7fa82c0d0870 2023-11-24T07:58:39.523 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:39.519+0000 7fa8318d2700 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa82c0cff70 con 0x7fa82c0d49f0 2023-11-24T07:58:39.523 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:39.519+0000 7fa82affd700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fa82c0d0870 0x7fa82c0d26c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:39.523 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:39.519+0000 7fa82affd700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fa82c0d0870 0x7fa82c0d26c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:53750/0 (socket says 172.21.15.59:53750) 2023-11-24T07:58:39.523 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:39.519+0000 7fa82affd700 1 -- 172.21.15.59:0/4021329775 learned_addr learned my addr 172.21.15.59:0/4021329775 (peer_addr_for_me v2:172.21.15.59:0/0) 2023-11-24T07:58:39.523 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:39.519+0000 7fa82b7fe700 1 --2- 172.21.15.59:0/4021329775 >> v2:172.21.15.76:3300/0 conn(0x7fa82c0d49f0 0x7fa82c0b6d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:39.524 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:39.519+0000 7fa82affd700 1 -- 172.21.15.59:0/4021329775 >> v2:172.21.15.76:3300/0 conn(0x7fa82c0d49f0 msgr2=0x7fa82c0b6d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T07:58:39.524 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:39.519+0000 7fa82affd700 1 --2- 172.21.15.59:0/4021329775 >> v2:172.21.15.76:3300/0 conn(0x7fa82c0d49f0 0x7fa82c0b6d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:39.524 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:39.519+0000 7fa82affd700 1 -- 172.21.15.59:0/4021329775 >> v1:172.21.15.76:6789/0 conn(0x7fa82c0d0c50 legacy=0x7fa82c0cf860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:39.524 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:39.519+0000 7fa82affd700 1 -- 172.21.15.59:0/4021329775 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa82c0b5370 con 0x7fa82c0d0870 2023-11-24T07:58:39.524 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:39.520+0000 7fa82affd700 1 --2- 172.21.15.59:0/4021329775 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa82c0d0870 0x7fa82c0d26c0 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 crypto rx=0x7fa82c0736c0 tx=0x7fa820009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ff5a258a71489199 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:58:39.524 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:39.520+0000 7fa829ffb700 1 -- 172.21.15.59:0/4021329775 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa820011830 con 0x7fa82c0d0870 2023-11-24T07:58:39.525 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:39.520+0000 7fa829ffb700 1 -- 172.21.15.59:0/4021329775 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa820011990 con 0x7fa82c0d0870 2023-11-24T07:58:39.525 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:39.520+0000 7fa829ffb700 1 -- 172.21.15.59:0/4021329775 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa820011d20 con 0x7fa82c0d0870 2023-11-24T07:58:39.525 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:39.520+0000 7fa8318d2700 1 -- 172.21.15.59:0/4021329775 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa82c0d0870 msgr2=0x7fa82c0d26c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:39.525 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:39.520+0000 7fa8318d2700 1 --2- 172.21.15.59:0/4021329775 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa82c0d0870 0x7fa82c0d26c0 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 crypto rx=0x7fa82c0736c0 tx=0x7fa820009bd0 comp rx=0 tx=0).stop 2023-11-24T07:58:39.525 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:39.520+0000 7fa8318d2700 1 -- 172.21.15.59:0/4021329775 shutdown_connections 2023-11-24T07:58:39.525 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:39.520+0000 7fa8318d2700 1 --2- 172.21.15.59:0/4021329775 >> v2:172.21.15.76:3300/0 conn(0x7fa82c0d49f0 0x7fa82c0b6d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:39.525 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:39.520+0000 7fa8318d2700 1 --2- 172.21.15.59:0/4021329775 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa82c0d0870 0x7fa82c0d26c0 unknown :-1 s=CLOSED pgs=696 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:39.525 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:39.520+0000 7fa8318d2700 1 -- 172.21.15.59:0/4021329775 >> 172.21.15.59:0/4021329775 conn(0x7fa82c0d3ed0 msgr2=0x7fa82c0d42e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:58:39.525 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:39.520+0000 7fa8318d2700 1 -- 172.21.15.59:0/4021329775 shutdown_connections 2023-11-24T07:58:39.525 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:39.520+0000 7fa8318d2700 1 -- 172.21.15.59:0/4021329775 wait complete. 2023-11-24T07:58:39.528 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:39.522+0000 7fa8318d2700 1 Processor -- start 2023-11-24T07:58:39.528 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:39.522+0000 7fa8318d2700 1 -- start start 2023-11-24T07:58:39.528 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:39.522+0000 7fa8318d2700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa82c0d0870 0x7fa82c1583b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:39.528 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:39.522+0000 7fa8318d2700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa82c0d0c50 0x7fa82c15c960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:39.528 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:39.522+0000 7fa8318d2700 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fa82c0d49f0 0x7fa82c158bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:39.528 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:39.522+0000 7fa8318d2700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa82c15c4d0 con 0x7fa82c0d0870 2023-11-24T07:58:39.528 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:39.522+0000 7fa8318d2700 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa82c15b8c0 con 0x7fa82c0d0c50 2023-11-24T07:58:39.528 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:39.522+0000 7fa8318d2700 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa82c15bba0 con 0x7fa82c0d49f0 2023-11-24T07:58:39.528 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:39.523+0000 7fa82affd700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa82c0d0870 0x7fa82c1583b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:39.529 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:39.523+0000 7fa82affd700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa82c0d0870 0x7fa82c1583b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:53756/0 (socket says 172.21.15.59:53756) 2023-11-24T07:58:39.529 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:39.523+0000 7fa82affd700 1 -- 172.21.15.59:0/1708595974 learned_addr learned my addr 172.21.15.59:0/1708595974 (peer_addr_for_me v2:172.21.15.59:0/0) 2023-11-24T07:58:39.529 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:39.523+0000 7fa82b7fe700 1 --2- 172.21.15.59:0/1708595974 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fa82c0d49f0 0x7fa82c158bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:39.529 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:39.523+0000 7fa82a7fc700 1 --2- 172.21.15.59:0/1708595974 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa82c0d0c50 0x7fa82c15c960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:39.529 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:39.523+0000 7fa82affd700 1 -- 172.21.15.59:0/1708595974 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fa82c0d49f0 msgr2=0x7fa82c158bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:39.529 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:39.523+0000 7fa82affd700 1 --2- 172.21.15.59:0/1708595974 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fa82c0d49f0 0x7fa82c158bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:39.529 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:39.523+0000 7fa82affd700 1 -- 172.21.15.59:0/1708595974 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa82c0d0c50 msgr2=0x7fa82c15c960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:39.529 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:39.523+0000 7fa82affd700 1 --2- 172.21.15.59:0/1708595974 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa82c0d0c50 0x7fa82c15c960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:39.529 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:39.523+0000 7fa82affd700 1 -- 172.21.15.59:0/1708595974 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa820009550 con 0x7fa82c0d0870 2023-11-24T07:58:39.529 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:39.523+0000 7fa82a7fc700 1 --2- 172.21.15.59:0/1708595974 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa82c0d0c50 0x7fa82c15c960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T07:58:39.529 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:39.523+0000 7fa82affd700 1 --2- 172.21.15.59:0/1708595974 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa82c0d0870 0x7fa82c1583b0 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 crypto rx=0x7fa820009fd0 tx=0x7fa820004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:58:39.529 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:39.523+0000 7fa813fff700 1 -- 172.21.15.59:0/1708595974 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8200038a0 con 0x7fa82c0d0870 2023-11-24T07:58:39.530 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:39.523+0000 7fa813fff700 1 -- 172.21.15.59:0/1708595974 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa82001f070 con 0x7fa82c0d0870 2023-11-24T07:58:39.530 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:39.523+0000 7fa8318d2700 1 -- 172.21.15.59:0/1708595974 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa82c15be80 con 0x7fa82c0d0870 2023-11-24T07:58:39.530 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:39.523+0000 7fa813fff700 1 -- 172.21.15.59:0/1708595974 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa820003c40 con 0x7fa82c0d0870 2023-11-24T07:58:39.530 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:39.523+0000 7fa8318d2700 1 -- 172.21.15.59:0/1708595974 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa82c1588f0 con 0x7fa82c0d0870 2023-11-24T07:58:39.530 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:39.524+0000 7fa8318d2700 1 -- 172.21.15.59:0/1708595974 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa82c0b57c0 con 0x7fa82c0d0870 2023-11-24T07:58:39.530 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:39.525+0000 7fa813fff700 1 -- 172.21.15.59:0/1708595974 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 8) v1 ==== 89435+0+0 (secure 0 0 0) 0x7fa820026480 con 0x7fa82c0d0870 2023-11-24T07:58:39.530 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:39.526+0000 7fa813fff700 1 --2- 172.21.15.59:0/1708595974 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7fa81806b110 0x7fa81806d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:39.530 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:39.526+0000 7fa813fff700 1 -- 172.21.15.59:0/1708595974 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(57..57 src has 1..57) v4 ==== 5227+0+0 (secure 0 0 0) 0x7fa820021070 con 0x7fa82c0d0870 2023-11-24T07:58:39.530 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:39.526+0000 7fa82a7fc700 1 --2- 172.21.15.59:0/1708595974 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7fa81806b110 0x7fa81806d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:39.530 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:39.526+0000 7fa82a7fc700 1 --2- 172.21.15.59:0/1708595974 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7fa81806b110 0x7fa81806d5d0 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7fa82c0c6b50 tx=0x7fa81c009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:58:39.531 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:39.530+0000 7fa813fff700 1 -- 172.21.15.59:0/1708595974 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fa8200566d0 con 0x7fa82c0d0870 2023-11-24T07:58:39.733 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:39.732+0000 7fa8318d2700 1 -- 172.21.15.59:0/1708595974 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7fa82c067a20 con 0x7fa82c0d0870 2023-11-24T07:58:40.374 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:40.373+0000 7fa813fff700 1 -- 172.21.15.59:0/1708595974 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v58) v1 ==== 170+0+0 (secure 0 0 0) 0x7fa8200150d0 con 0x7fa82c0d0870 2023-11-24T07:58:40.377 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:40.376+0000 7fa8318d2700 1 -- 172.21.15.59:0/1708595974 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7fa81806b110 msgr2=0x7fa81806d5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:40.377 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:40.376+0000 7fa8318d2700 1 --2- 172.21.15.59:0/1708595974 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7fa81806b110 0x7fa81806d5d0 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7fa82c0c6b50 tx=0x7fa81c009040 comp rx=0 tx=0).stop 2023-11-24T07:58:40.377 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:40.376+0000 7fa8318d2700 1 -- 172.21.15.59:0/1708595974 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa82c0d0870 msgr2=0x7fa82c1583b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:40.377 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:40.376+0000 7fa8318d2700 1 --2- 172.21.15.59:0/1708595974 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa82c0d0870 0x7fa82c1583b0 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 crypto rx=0x7fa820009fd0 tx=0x7fa820004750 comp rx=0 tx=0).stop 2023-11-24T07:58:40.379 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:40.377+0000 7fa8318d2700 1 -- 172.21.15.59:0/1708595974 shutdown_connections 2023-11-24T07:58:40.379 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:40.377+0000 7fa8318d2700 1 --2- 172.21.15.59:0/1708595974 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7fa81806b110 0x7fa81806d5d0 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:40.379 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:40.377+0000 7fa8318d2700 1 --2- 172.21.15.59:0/1708595974 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fa82c0d49f0 0x7fa82c158bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:40.379 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:40.377+0000 7fa8318d2700 1 --2- 172.21.15.59:0/1708595974 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa82c0d0870 0x7fa82c1583b0 unknown :-1 s=CLOSED pgs=697 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:40.379 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:40.377+0000 7fa8318d2700 1 --2- 172.21.15.59:0/1708595974 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa82c0d0c50 0x7fa82c15c960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:40.379 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:40.377+0000 7fa8318d2700 1 -- 172.21.15.59:0/1708595974 >> 172.21.15.59:0/1708595974 conn(0x7fa82c0d3ed0 msgr2=0x7fa82c0d2270 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:58:40.380 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:40.378+0000 7fa8318d2700 1 -- 172.21.15.59:0/1708595974 shutdown_connections 2023-11-24T07:58:40.380 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:40.378+0000 7fa8318d2700 1 -- 172.21.15.59:0/1708595974 wait complete. 2023-11-24T07:58:40.387 INFO:teuthology.orchestra.run.smithi059.stderr:pool 'cephfs_data' created 2023-11-24T07:58:40.404 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2023-11-24T07:58:40.535 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:40.533+0000 7f44293cb700 1 Processor -- start 2023-11-24T07:58:40.535 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:40.533+0000 7f44293cb700 1 -- start start 2023-11-24T07:58:40.538 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:40.534+0000 7f44293cb700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f44240d0c50 0x7f44240d26c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:40.538 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:40.534+0000 7f44293cb700 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f44240d49f0 0x7f44240b6d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:40.538 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:40.534+0000 7f44293cb700 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f44240b5370 con 0x7f44240d0870 2023-11-24T07:58:40.538 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:40.534+0000 7f44293cb700 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f44240d2f80 con 0x7f44240d0c50 2023-11-24T07:58:40.538 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:40.534+0000 7f44293cb700 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f44240cff70 con 0x7f44240d49f0 2023-11-24T07:58:40.538 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:40.535+0000 7f44227fc700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f44240d0c50 0x7f44240d26c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:40.538 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:40.535+0000 7f44227fc700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f44240d0c50 0x7f44240d26c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:53764/0 (socket says 172.21.15.59:53764) 2023-11-24T07:58:40.538 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:40.535+0000 7f44227fc700 1 -- 172.21.15.59:0/1963365090 learned_addr learned my addr 172.21.15.59:0/1963365090 (peer_addr_for_me v2:172.21.15.59:0/0) 2023-11-24T07:58:40.538 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:40.535+0000 7f44237fe700 1 --2- 172.21.15.59:0/1963365090 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f44240d49f0 0x7f44240b6d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:40.539 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:40.535+0000 7f44227fc700 1 -- 172.21.15.59:0/1963365090 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f44240d49f0 msgr2=0x7f44240b6d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T07:58:40.539 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:40.535+0000 7f44227fc700 1 --2- 172.21.15.59:0/1963365090 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f44240d49f0 0x7f44240b6d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:40.539 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:40.535+0000 7f44227fc700 1 -- 172.21.15.59:0/1963365090 >> v1:172.21.15.76:6789/0 conn(0x7f44240d0870 legacy=0x7f44240cf860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:40.539 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:40.535+0000 7f44227fc700 1 -- 172.21.15.59:0/1963365090 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f44240ceae0 con 0x7f44240d0c50 2023-11-24T07:58:40.539 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:40.535+0000 7f44227fc700 1 --2- 172.21.15.59:0/1963365090 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f44240d0c50 0x7f44240d26c0 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 crypto rx=0x7f44180098f0 tx=0x7f4418009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4eb632befca3bf8c server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:58:40.539 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:40.535+0000 7f4421ffb700 1 -- 172.21.15.59:0/1963365090 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4418011830 con 0x7f44240d0c50 2023-11-24T07:58:40.539 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:40.535+0000 7f4421ffb700 1 -- 172.21.15.59:0/1963365090 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4418011990 con 0x7f44240d0c50 2023-11-24T07:58:40.539 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:40.535+0000 7f4421ffb700 1 -- 172.21.15.59:0/1963365090 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4418011d20 con 0x7f44240d0c50 2023-11-24T07:58:40.539 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:40.536+0000 7f44293cb700 1 -- 172.21.15.59:0/1963365090 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f44240d0c50 msgr2=0x7f44240d26c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:40.539 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:40.536+0000 7f44293cb700 1 --2- 172.21.15.59:0/1963365090 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f44240d0c50 0x7f44240d26c0 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 crypto rx=0x7f44180098f0 tx=0x7f4418009c00 comp rx=0 tx=0).stop 2023-11-24T07:58:40.539 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:40.536+0000 7f44293cb700 1 -- 172.21.15.59:0/1963365090 shutdown_connections 2023-11-24T07:58:40.540 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:40.536+0000 7f44293cb700 1 --2- 172.21.15.59:0/1963365090 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f44240d49f0 0x7f44240b6d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:40.540 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:40.536+0000 7f44293cb700 1 --2- 172.21.15.59:0/1963365090 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f44240d0c50 0x7f44240d26c0 unknown :-1 s=CLOSED pgs=698 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:40.540 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:40.536+0000 7f44293cb700 1 -- 172.21.15.59:0/1963365090 >> 172.21.15.59:0/1963365090 conn(0x7f44240d3ed0 msgr2=0x7f44240d42e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:58:40.540 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:40.536+0000 7f44293cb700 1 -- 172.21.15.59:0/1963365090 shutdown_connections 2023-11-24T07:58:40.540 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:40.536+0000 7f44293cb700 1 -- 172.21.15.59:0/1963365090 wait complete. 2023-11-24T07:58:40.543 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:40.537+0000 7f44293cb700 1 Processor -- start 2023-11-24T07:58:40.543 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:40.537+0000 7f44293cb700 1 -- start start 2023-11-24T07:58:40.543 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:40.538+0000 7f44293cb700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f44240d0870 0x7f442415ad20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:40.543 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:40.538+0000 7f44293cb700 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f44240d0c50 0x7f442415e730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:40.543 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:40.538+0000 7f44293cb700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f44240d49f0 0x7f4424161560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:40.543 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:40.538+0000 7f44293cb700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4424161140 con 0x7f44240d0870 2023-11-24T07:58:40.543 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:40.538+0000 7f44293cb700 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f442415b260 con 0x7f44240d49f0 2023-11-24T07:58:40.543 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:40.538+0000 7f44293cb700 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4424160240 con 0x7f44240d0c50 2023-11-24T07:58:40.543 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:40.538+0000 7f4422ffd700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f44240d0870 0x7f442415ad20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:40.543 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:40.538+0000 7f4422ffd700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f44240d0870 0x7f442415ad20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:53768/0 (socket says 172.21.15.59:53768) 2023-11-24T07:58:40.544 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:40.538+0000 7f4422ffd700 1 -- 172.21.15.59:0/223922131 learned_addr learned my addr 172.21.15.59:0/223922131 (peer_addr_for_me v2:172.21.15.59:0/0) 2023-11-24T07:58:40.544 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:40.538+0000 7f44227fc700 1 --2- 172.21.15.59:0/223922131 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f44240d0c50 0x7f442415e730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:40.544 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:40.539+0000 7f44237fe700 1 --2- 172.21.15.59:0/223922131 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f44240d49f0 0x7f4424161560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:40.544 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:40.539+0000 7f4422ffd700 1 -- 172.21.15.59:0/223922131 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f44240d0c50 msgr2=0x7f442415e730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:40.544 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:40.539+0000 7f4422ffd700 1 --2- 172.21.15.59:0/223922131 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f44240d0c50 0x7f442415e730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:40.544 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:40.539+0000 7f4422ffd700 1 -- 172.21.15.59:0/223922131 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f44240d49f0 msgr2=0x7f4424161560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:40.544 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:40.539+0000 7f4422ffd700 1 --2- 172.21.15.59:0/223922131 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f44240d49f0 0x7f4424161560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:40.544 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:40.539+0000 7f4422ffd700 1 -- 172.21.15.59:0/223922131 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4418009550 con 0x7f44240d0870 2023-11-24T07:58:40.544 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:40.539+0000 7f44227fc700 1 --2- 172.21.15.59:0/223922131 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f44240d0c50 0x7f442415e730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T07:58:40.544 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:40.539+0000 7f44237fe700 1 --2- 172.21.15.59:0/223922131 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f44240d49f0 0x7f4424161560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T07:58:40.544 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:40.539+0000 7f4422ffd700 1 --2- 172.21.15.59:0/223922131 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f44240d0870 0x7f442415ad20 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 crypto rx=0x7f441400c510 tx=0x7f441400c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:58:40.544 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:40.539+0000 7f440b7fe700 1 -- 172.21.15.59:0/223922131 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44140052d0 con 0x7f44240d0870 2023-11-24T07:58:40.545 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:40.539+0000 7f440b7fe700 1 -- 172.21.15.59:0/223922131 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4414005430 con 0x7f44240d0870 2023-11-24T07:58:40.545 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:40.539+0000 7f44293cb700 1 -- 172.21.15.59:0/223922131 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f44241604c0 con 0x7f44240d0870 2023-11-24T07:58:40.545 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:40.539+0000 7f440b7fe700 1 -- 172.21.15.59:0/223922131 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44140038c0 con 0x7f44240d0870 2023-11-24T07:58:40.545 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:40.539+0000 7f44293cb700 1 -- 172.21.15.59:0/223922131 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4424160a80 con 0x7f44240d0870 2023-11-24T07:58:40.545 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:40.540+0000 7f44293cb700 1 -- 172.21.15.59:0/223922131 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f44241613b0 con 0x7f44240d0870 2023-11-24T07:58:40.545 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:40.540+0000 7f440b7fe700 1 -- 172.21.15.59:0/223922131 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 8) v1 ==== 89435+0+0 (secure 0 0 0) 0x7f4414003a20 con 0x7f44240d0870 2023-11-24T07:58:40.545 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:40.541+0000 7f440b7fe700 1 --2- 172.21.15.59:0/223922131 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f441006b110 0x7f441006d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:40.545 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:40.541+0000 7f440b7fe700 1 -- 172.21.15.59:0/223922131 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(58..58 src has 1..58) v4 ==== 5609+0+0 (secure 0 0 0) 0x7f4414088d10 con 0x7f44240d0870 2023-11-24T07:58:40.545 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:40.541+0000 7f44227fc700 1 --2- 172.21.15.59:0/223922131 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f441006b110 0x7f441006d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:40.545 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:40.541+0000 7f44227fc700 1 --2- 172.21.15.59:0/223922131 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f441006b110 0x7f441006d5d0 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f44180098f0 tx=0x7f4418008030 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:58:40.547 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:40.546+0000 7f440b7fe700 1 -- 172.21.15.59:0/223922131 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f4414053c20 con 0x7f44240d0870 2023-11-24T07:58:40.795 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:40.793+0000 7f44293cb700 1 -- 172.21.15.59:0/223922131 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7f4424067a20 con 0x7f44240d0870 2023-11-24T07:58:41.376 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:41.374+0000 7f440b7fe700 1 -- 172.21.15.59:0/223922131 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 8 and data pool 9 v31) v1 ==== 172+0+0 (secure 0 0 0) 0x7f44140170c0 con 0x7f44240d0870 2023-11-24T07:58:41.380 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:41.377+0000 7f44293cb700 1 -- 172.21.15.59:0/223922131 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f441006b110 msgr2=0x7f441006d5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:41.380 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:41.377+0000 7f44293cb700 1 --2- 172.21.15.59:0/223922131 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f441006b110 0x7f441006d5d0 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f44180098f0 tx=0x7f4418008030 comp rx=0 tx=0).stop 2023-11-24T07:58:41.380 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:41.377+0000 7f44293cb700 1 -- 172.21.15.59:0/223922131 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f44240d0870 msgr2=0x7f442415ad20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:41.380 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:41.378+0000 7f44293cb700 1 --2- 172.21.15.59:0/223922131 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f44240d0870 0x7f442415ad20 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 crypto rx=0x7f441400c510 tx=0x7f441400c8d0 comp rx=0 tx=0).stop 2023-11-24T07:58:41.380 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:41.378+0000 7f44293cb700 1 -- 172.21.15.59:0/223922131 shutdown_connections 2023-11-24T07:58:41.381 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:41.378+0000 7f44293cb700 1 --2- 172.21.15.59:0/223922131 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f441006b110 0x7f441006d5d0 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:41.381 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:41.378+0000 7f44293cb700 1 --2- 172.21.15.59:0/223922131 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f44240d49f0 0x7f4424161560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:41.381 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:41.378+0000 7f44293cb700 1 --2- 172.21.15.59:0/223922131 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f44240d0870 0x7f442415ad20 unknown :-1 s=CLOSED pgs=699 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:41.381 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:41.378+0000 7f44293cb700 1 --2- 172.21.15.59:0/223922131 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f44240d0c50 0x7f442415e730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:41.381 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:41.378+0000 7f44293cb700 1 -- 172.21.15.59:0/223922131 >> 172.21.15.59:0/223922131 conn(0x7f44240d3ed0 msgr2=0x7f442415b550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:58:41.381 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:41.378+0000 7f44293cb700 1 -- 172.21.15.59:0/223922131 shutdown_connections 2023-11-24T07:58:41.381 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:41.378+0000 7f44293cb700 1 -- 172.21.15.59:0/223922131 wait complete. 2023-11-24T07:58:41.385 INFO:teuthology.orchestra.run.smithi059.stderr:new fs with metadata pool 8 and data pool 9 2023-11-24T07:58:41.400 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-24T07:58:41.531 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:41.529+0000 7fc325f65700 1 Processor -- start 2023-11-24T07:58:41.531 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:41.529+0000 7fc325f65700 1 -- start start 2023-11-24T07:58:41.533 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:41.530+0000 7fc325f65700 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fc3200b4150 0x7fc32006de50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:41.534 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:41.530+0000 7fc325f65700 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fc3200b5ab0 0x7fc32006e390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:41.534 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:41.530+0000 7fc325f65700 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc320071f50 con 0x7fc32006b620 2023-11-24T07:58:41.534 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:41.530+0000 7fc325f65700 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc320072230 con 0x7fc3200b4150 2023-11-24T07:58:41.534 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:41.530+0000 7fc325f65700 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc320072510 con 0x7fc3200b5ab0 2023-11-24T07:58:41.534 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:41.530+0000 7fc31effd700 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fc3200b4150 0x7fc32006de50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:41.534 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:41.531+0000 7fc31f7fe700 1 --1- >> v1:172.21.15.76:6789/0 conn(0x7fc32006b620 0x7fc3200b3a40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.59:44542/0 (socket says 172.21.15.59:44542) 2023-11-24T07:58:41.534 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:41.531+0000 7fc31f7fe700 1 -- 172.21.15.59:0/3558131562 learned_addr learned my addr 172.21.15.59:0/3558131562 (peer_addr_for_me v1:172.21.15.59:0/0) 2023-11-24T07:58:41.534 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:41.531+0000 7fc31ffff700 1 --2- 172.21.15.59:0/3558131562 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fc3200b5ab0 0x7fc32006e390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:41.534 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:41.531+0000 7fc31ffff700 1 -- 172.21.15.59:0/3558131562 >> v2:172.21.15.76:3300/0 conn(0x7fc3200b4150 msgr2=0x7fc32006de50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T07:58:41.534 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:41.531+0000 7fc31ffff700 1 --2- 172.21.15.59:0/3558131562 >> v2:172.21.15.76:3300/0 conn(0x7fc3200b4150 0x7fc32006de50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:41.534 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:41.531+0000 7fc31ffff700 1 -- 172.21.15.59:0/3558131562 >> v1:172.21.15.76:6789/0 conn(0x7fc32006b620 legacy=0x7fc3200b3a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:41.535 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:41.531+0000 7fc31e7fc700 1 -- 172.21.15.59:0/3558131562 <== mon.1 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1145031289 0 0) 0x7fc320071f50 con 0x7fc32006b620 2023-11-24T07:58:41.535 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:41.531+0000 7fc31ffff700 1 -- 172.21.15.59:0/3558131562 --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc3200727f0 con 0x7fc3200b5ab0 2023-11-24T07:58:41.535 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:41.531+0000 7fc31effd700 1 --2- 172.21.15.59:0/3558131562 >> v2:172.21.15.76:3300/0 conn(0x7fc3200b4150 0x7fc32006de50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-24T07:58:41.535 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:41.531+0000 7fc31ffff700 1 --2- 172.21.15.59:0/3558131562 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fc3200b5ab0 0x7fc32006e390 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fc318009d90 tx=0x7fc31800b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2896bdc3deaed397 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:58:41.535 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:41.531+0000 7fc31e7fc700 1 -- 172.21.15.59:0/3558131562 <== mon.2 v2:172.21.15.76:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc31800e070 con 0x7fc3200b5ab0 2023-11-24T07:58:41.535 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:41.531+0000 7fc31e7fc700 1 -- 172.21.15.59:0/3558131562 <== mon.2 v2:172.21.15.76:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc31800bb20 con 0x7fc3200b5ab0 2023-11-24T07:58:41.535 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:41.531+0000 7fc31e7fc700 1 -- 172.21.15.59:0/3558131562 <== mon.2 v2:172.21.15.76:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc318012430 con 0x7fc3200b5ab0 2023-11-24T07:58:41.535 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:41.532+0000 7fc325f65700 1 -- 172.21.15.59:0/3558131562 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fc3200b5ab0 msgr2=0x7fc32006e390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:41.535 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:41.532+0000 7fc325f65700 1 --2- 172.21.15.59:0/3558131562 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fc3200b5ab0 0x7fc32006e390 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fc318009d90 tx=0x7fc31800b330 comp rx=0 tx=0).stop 2023-11-24T07:58:41.535 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:41.532+0000 7fc325f65700 1 -- 172.21.15.59:0/3558131562 shutdown_connections 2023-11-24T07:58:41.535 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:41.532+0000 7fc325f65700 1 --2- 172.21.15.59:0/3558131562 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fc3200b5ab0 0x7fc32006e390 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:41.535 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:41.532+0000 7fc325f65700 1 --2- 172.21.15.59:0/3558131562 >> v2:172.21.15.76:3300/0 conn(0x7fc3200b4150 0x7fc32006de50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:41.535 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:41.532+0000 7fc325f65700 1 -- 172.21.15.59:0/3558131562 >> 172.21.15.59:0/3558131562 conn(0x7fc3200b2740 msgr2=0x7fc3200b2b20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:58:41.536 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:41.532+0000 7fc325f65700 1 -- 172.21.15.59:0/3558131562 shutdown_connections 2023-11-24T07:58:41.536 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:41.532+0000 7fc325f65700 1 -- 172.21.15.59:0/3558131562 wait complete. 2023-11-24T07:58:41.538 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:41.533+0000 7fc325f65700 1 Processor -- start 2023-11-24T07:58:41.538 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:41.534+0000 7fc325f65700 1 -- start start 2023-11-24T07:58:41.538 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:41.535+0000 7fc325f65700 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fc32006b620 0x7fc32006dd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:41.538 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:41.535+0000 7fc325f65700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc3200b4150 0x7fc32006e290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:41.538 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:41.535+0000 7fc325f65700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc3200b5ab0 0x7fc3201f6e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:41.538 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:41.535+0000 7fc325f65700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc320135930 con 0x7fc3200b4150 2023-11-24T07:58:41.538 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:41.535+0000 7fc325f65700 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3201f7b70 con 0x7fc3200b5ab0 2023-11-24T07:58:41.539 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:41.535+0000 7fc325f65700 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc320135c00 con 0x7fc32006b620 2023-11-24T07:58:41.539 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:41.535+0000 7fc31effd700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc3200b4150 0x7fc32006e290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:41.539 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:41.535+0000 7fc31effd700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc3200b4150 0x7fc32006e290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:53798/0 (socket says 172.21.15.59:53798) 2023-11-24T07:58:41.539 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:41.535+0000 7fc31effd700 1 -- 172.21.15.59:0/2207805585 learned_addr learned my addr 172.21.15.59:0/2207805585 (peer_addr_for_me v2:172.21.15.59:0/0) 2023-11-24T07:58:41.539 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:41.535+0000 7fc31f7fe700 1 --2- 172.21.15.59:0/2207805585 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fc32006b620 0x7fc32006dd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:41.539 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:41.535+0000 7fc31ffff700 1 --2- 172.21.15.59:0/2207805585 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc3200b5ab0 0x7fc3201f6e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:41.539 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:41.535+0000 7fc31effd700 1 -- 172.21.15.59:0/2207805585 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fc32006b620 msgr2=0x7fc32006dd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:41.539 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:41.535+0000 7fc31effd700 1 --2- 172.21.15.59:0/2207805585 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fc32006b620 0x7fc32006dd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:41.539 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:41.535+0000 7fc31effd700 1 -- 172.21.15.59:0/2207805585 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc3200b5ab0 msgr2=0x7fc3201f6e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:41.539 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:41.535+0000 7fc31effd700 1 --2- 172.21.15.59:0/2207805585 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc3200b5ab0 0x7fc3201f6e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:41.539 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:41.535+0000 7fc31effd700 1 -- 172.21.15.59:0/2207805585 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc3140094a0 con 0x7fc3200b4150 2023-11-24T07:58:41.539 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:41.535+0000 7fc31effd700 1 --2- 172.21.15.59:0/2207805585 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc3200b4150 0x7fc32006e290 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 crypto rx=0x7fc31400ebe0 tx=0x7fc31400eef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:58:41.539 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:41.536+0000 7fc307fff700 1 -- 172.21.15.59:0/2207805585 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc31400c850 con 0x7fc3200b4150 2023-11-24T07:58:41.540 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:41.536+0000 7fc307fff700 1 -- 172.21.15.59:0/2207805585 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc31400c9b0 con 0x7fc3200b4150 2023-11-24T07:58:41.540 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:41.536+0000 7fc325f65700 1 -- 172.21.15.59:0/2207805585 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc318009a70 con 0x7fc3200b4150 2023-11-24T07:58:41.540 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:41.536+0000 7fc307fff700 1 -- 172.21.15.59:0/2207805585 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc31400cd40 con 0x7fc3200b4150 2023-11-24T07:58:41.540 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:41.536+0000 7fc325f65700 1 -- 172.21.15.59:0/2207805585 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc3201361c0 con 0x7fc3200b4150 2023-11-24T07:58:41.540 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:41.537+0000 7fc325f65700 1 -- 172.21.15.59:0/2207805585 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc320068670 con 0x7fc3200b4150 2023-11-24T07:58:41.541 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:41.537+0000 7fc307fff700 1 -- 172.21.15.59:0/2207805585 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 8) v1 ==== 89435+0+0 (secure 0 0 0) 0x7fc314013470 con 0x7fc3200b4150 2023-11-24T07:58:41.541 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:41.537+0000 7fc307fff700 1 --2- 172.21.15.59:0/2207805585 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7fc30c06b110 0x7fc30c06d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:41.541 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:41.538+0000 7fc307fff700 1 -- 172.21.15.59:0/2207805585 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(59..59 src has 1..59) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fc314089ac0 con 0x7fc3200b4150 2023-11-24T07:58:41.541 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:41.538+0000 7fc31f7fe700 1 --2- 172.21.15.59:0/2207805585 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7fc30c06b110 0x7fc30c06d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:41.541 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:41.538+0000 7fc31f7fe700 1 --2- 172.21.15.59:0/2207805585 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7fc30c06b110 0x7fc30c06d5d0 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7fc310000f60 tx=0x7fc310007e70 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:58:41.543 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:41.542+0000 7fc307fff700 1 -- 172.21.15.59:0/2207805585 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fc314054810 con 0x7fc3200b4150 2023-11-24T07:58:41.746 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:41.745+0000 7fc325f65700 1 -- 172.21.15.59:0/2207805585 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fc32006bd40 con 0x7fc3200b4150 2023-11-24T07:58:41.748 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:41.746+0000 7fc307fff700 1 -- 172.21.15.59:0/2207805585 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v59) v1 ==== 74+0+16506 (secure 0 0 0) 0x7fc31400cea0 con 0x7fc3200b4150 2023-11-24T07:58:41.748 INFO:teuthology.orchestra.run.smithi059.stdout: 2023-11-24T07:58:41.748 INFO:teuthology.orchestra.run.smithi059.stdout:{"epoch":59,"fsid":"9f0e8741-b8f9-48d5-a2c3-09f45903303d","created":"2023-11-24T07:56:06.552729+0000","modified":"2023-11-24T07:58:40.795035+0000","last_up_change":"2023-11-24T07:56:17.372352+0000","last_in_change":"2023-11-24T07:56:09.073250+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-24T07:56:18.352823+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_metadata","create_time":"2023-11-24T07:58:39.262463+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"59","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_data","create_time":"2023-11-24T07:58:39.763399+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"59","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"6d8f9e9b-1782-4e41-928b-a39f2a8ebc06","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6800","nonce":3373878772},{"type":"v1","addr":"172.21.15.59:6801","nonce":3373878772}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6802","nonce":3373878772},{"type":"v1","addr":"172.21.15.59:6803","nonce":3373878772}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6806","nonce":3373878772},{"type":"v1","addr":"172.21.15.59:6807","nonce":3373878772}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6804","nonce":3373878772},{"type":"v1","addr":"172.21.15.59:6805","nonce":3373878772}]},"public_addr":"172.21.15.59:6801/3373878772","cluster_addr":"172.21.15.59:6803/3373878772","heartbeat_back_addr":"172.21.15.59:6807/3373878772","heartbeat_front_addr":"172.21.15.59:6805/3373878772","state":["exists","up"]},{"osd":1,"uuid":"e8d882aa-52e1-4c04-9e1e-c44ef9204a5f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6824","nonce":648258563},{"type":"v1","addr":"172.21.15.59:6825","nonce":648258563}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6826","nonce":648258563},{"type":"v1","addr":"172.21.15.59:6827","nonce":648258563}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6830","nonce":648258563},{"type":"v1","addr":"172.21.15.59:6831","nonce":648258563}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6828","nonce":648258563},{"type":"v1","addr":"172.21.15.59:6829","nonce":648258563}]},"public_addr":"172.21.15.59:6825/648258563","cluster_addr":"172.21.15.59:6827/648258563","heartbeat_back_addr":"172.21.15.59:6831/648258563","heartbeat_front_addr":"172.21.15.59:6829/648258563","state":["exists","up"]},{"osd":2,"uuid":"69eebfa0-695b-4b21-90b9-8a3537168a01","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6808","nonce":1840430654},{"type":"v1","addr":"172.21.15.59:6809","nonce":1840430654}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6810","nonce":1840430654},{"type":"v1","addr":"172.21.15.59:6811","nonce":1840430654}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6814","nonce":1840430654},{"type":"v1","addr":"172.21.15.59:6815","nonce":1840430654}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6812","nonce":1840430654},{"type":"v1","addr":"172.21.15.59:6813","nonce":1840430654}]},"public_addr":"172.21.15.59:6809/1840430654","cluster_addr":"172.21.15.59:6811/1840430654","heartbeat_back_addr":"172.21.15.59:6815/1840430654","heartbeat_front_addr":"172.21.15.59:6813/1840430654","state":["exists","up"]},{"osd":3,"uuid":"96fd77fc-f911-46ff-9445-6f0be2daa8f4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6816","nonce":2504025850},{"type":"v1","addr":"172.21.15.59:6817","nonce":2504025850}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6818","nonce":2504025850},{"type":"v1","addr":"172.21.15.59:6819","nonce":2504025850}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6822","nonce":2504025850},{"type":"v1","addr":"172.21.15.59:6823","nonce":2504025850}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6820","nonce":2504025850},{"type":"v1","addr":"172.21.15.59:6821","nonce":2504025850}]},"public_addr":"172.21.15.59:6817/2504025850","cluster_addr":"172.21.15.59:6819/2504025850","heartbeat_back_addr":"172.21.15.59:6823/2504025850","heartbeat_front_addr":"172.21.15.59:6821/2504025850","state":["exists","up"]},{"osd":4,"uuid":"7ac29fca-b5b2-4c95-9d69-2f164cbbc708","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6816","nonce":2382463431},{"type":"v1","addr":"172.21.15.76:6818","nonce":2382463431}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6820","nonce":2382463431},{"type":"v1","addr":"172.21.15.76:6822","nonce":2382463431}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6828","nonce":2382463431},{"type":"v1","addr":"172.21.15.76:6830","nonce":2382463431}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6824","nonce":2382463431},{"type":"v1","addr":"172.21.15.76:6826","nonce":2382463431}]},"public_addr":"172.21.15.76:6818/2382463431","cluster_addr":"172.21.15.76:6822/2382463431","heartbeat_back_addr":"172.21.15.76:6830/2382463431","heartbeat_front_addr":"172.21.15.76:6826/2382463431","state":["exists","up"]},{"osd":5,"uuid":"485bf711-3851-4238-a4f4-51d52e0a9c58","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6800","nonce":329334308},{"type":"v1","addr":"172.21.15.76:6802","nonce":329334308}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6805","nonce":329334308},{"type":"v1","addr":"172.21.15.76:6807","nonce":329334308}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6814","nonce":329334308},{"type":"v1","addr":"172.21.15.76:6815","nonce":329334308}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6810","nonce":329334308},{"type":"v1","addr":"172.21.15.76:6812","nonce":329334308}]},"public_addr":"172.21.15.76:6802/329334308","cluster_addr":"172.21.15.76:6807/329334308","heartbeat_back_addr":"172.21.15.76:6815/329334308","heartbeat_front_addr":"172.21.15.76:6812/329334308","state":["exists","up"]},{"osd":6,"uuid":"72b3f2fc-1e4a-4e32-968d-003647a0286f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6801","nonce":3946579410},{"type":"v1","addr":"172.21.15.76:6803","nonce":3946579410}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6804","nonce":3946579410},{"type":"v1","addr":"172.21.15.76:6806","nonce":3946579410}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6811","nonce":3946579410},{"type":"v1","addr":"172.21.15.76:6813","nonce":3946579410}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6808","nonce":3946579410},{"type":"v1","addr":"172.21.15.76:6809","nonce":3946579410}]},"public_addr":"172.21.15.76:6803/3946579410","cluster_addr":"172.21.15.76:6806/3946579410","heartbeat_back_addr":"172.21.15.76:6813/3946579410","heartbeat_front_addr":"172.21.15.76:6809/3946579410","state":["exists","up"]},{"osd":7,"uuid":"1ddceb90-6c5d-4d1a-ad56-99cb877f8c70","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6817","nonce":841761538},{"type":"v1","addr":"172.21.15.76:6819","nonce":841761538}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6821","nonce":841761538},{"type":"v1","addr":"172.21.15.76:6823","nonce":841761538}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6829","nonce":841761538},{"type":"v1","addr":"172.21.15.76:6831","nonce":841761538}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6825","nonce":841761538},{"type":"v1","addr":"172.21.15.76:6827","nonce":841761538}]},"public_addr":"172.21.15.76:6819/841761538","cluster_addr":"172.21.15.76:6823/841761538","heartbeat_back_addr":"172.21.15.76:6831/841761538","heartbeat_front_addr":"172.21.15.76:6827/841761538","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T07:56:14.294838+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T07:56:14.270779+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T07:56:14.231809+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T07:56:14.260983+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T07:56:14.302997+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T07:56:14.258600+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T07:56:14.251413+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T07:56:14.304727+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-24T07:58:41.750 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:41.748+0000 7fc325f65700 1 -- 172.21.15.59:0/2207805585 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7fc30c06b110 msgr2=0x7fc30c06d5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:41.751 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:41.748+0000 7fc325f65700 1 --2- 172.21.15.59:0/2207805585 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7fc30c06b110 0x7fc30c06d5d0 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7fc310000f60 tx=0x7fc310007e70 comp rx=0 tx=0).stop 2023-11-24T07:58:41.751 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:41.748+0000 7fc325f65700 1 -- 172.21.15.59:0/2207805585 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc3200b4150 msgr2=0x7fc32006e290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:41.751 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:41.748+0000 7fc325f65700 1 --2- 172.21.15.59:0/2207805585 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc3200b4150 0x7fc32006e290 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 crypto rx=0x7fc31400ebe0 tx=0x7fc31400eef0 comp rx=0 tx=0).stop 2023-11-24T07:58:41.751 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:41.749+0000 7fc325f65700 1 -- 172.21.15.59:0/2207805585 shutdown_connections 2023-11-24T07:58:41.751 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:41.749+0000 7fc325f65700 1 --2- 172.21.15.59:0/2207805585 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7fc30c06b110 0x7fc30c06d5d0 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:41.751 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:41.749+0000 7fc325f65700 1 --2- 172.21.15.59:0/2207805585 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc3200b5ab0 0x7fc3201f6e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:41.751 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:41.749+0000 7fc325f65700 1 --2- 172.21.15.59:0/2207805585 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fc32006b620 0x7fc32006dd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:41.751 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:41.749+0000 7fc325f65700 1 --2- 172.21.15.59:0/2207805585 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc3200b4150 0x7fc32006e290 unknown :-1 s=CLOSED pgs=700 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:41.751 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:41.749+0000 7fc325f65700 1 -- 172.21.15.59:0/2207805585 >> 172.21.15.59:0/2207805585 conn(0x7fc3200b2740 msgr2=0x7fc320146240 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:58:41.751 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:41.749+0000 7fc325f65700 1 -- 172.21.15.59:0/2207805585 shutdown_connections 2023-11-24T07:58:41.751 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:41.749+0000 7fc325f65700 1 -- 172.21.15.59:0/2207805585 wait complete. 2023-11-24T07:58:41.763 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-24T07:58:41.888 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:41.887+0000 7f69a1e0b700 1 Processor -- start 2023-11-24T07:58:41.888 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:41.887+0000 7f69a1e0b700 1 -- start start 2023-11-24T07:58:41.890 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:41.887+0000 7f69a1e0b700 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f699c15e460 0x7f699c0d49d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:41.890 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:41.887+0000 7f69a1e0b700 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f699c0cf480 0x7f699c0d2490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:41.890 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:41.887+0000 7f69a1e0b700 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f699c0b3370 con 0x7f699c0d29d0 2023-11-24T07:58:41.890 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:41.887+0000 7f69a1e0b700 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f699c0d2e60 con 0x7f699c0cf480 2023-11-24T07:58:41.890 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:41.887+0000 7f69a1e0b700 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f699c0cfc40 con 0x7f699c15e460 2023-11-24T07:58:41.890 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:41.888+0000 7f699b7fe700 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f699c15e460 0x7f699c0d49d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:41.890 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:41.888+0000 7f699affd700 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f699c0cf480 0x7f699c0d2490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:41.891 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:41.888+0000 7f699bfff700 1 --1- >> v1:172.21.15.76:6789/0 conn(0x7f699c0d29d0 0x7f699c0d0550 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.59:44560/0 (socket says 172.21.15.59:44560) 2023-11-24T07:58:41.891 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:41.888+0000 7f699affd700 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f699c0cf480 0x7f699c0d2490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.59:41004/0 (socket says 172.21.15.59:41004) 2023-11-24T07:58:41.891 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:41.888+0000 7f699b7fe700 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f699c15e460 0x7f699c0d49d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3301/0 says I am v2:172.21.15.59:53244/0 (socket says 172.21.15.59:53244) 2023-11-24T07:58:41.891 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:41.888+0000 7f699bfff700 1 -- 172.21.15.59:0/3871880727 learned_addr learned my addr 172.21.15.59:0/3871880727 (peer_addr_for_me v1:172.21.15.59:0/0) 2023-11-24T07:58:41.891 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:41.888+0000 7f699affd700 1 -- 172.21.15.59:0/3871880727 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f699c15e460 msgr2=0x7f699c0d49d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T07:58:41.891 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:41.888+0000 7f699affd700 1 --2- 172.21.15.59:0/3871880727 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f699c15e460 0x7f699c0d49d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:41.891 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:41.888+0000 7f699affd700 1 -- 172.21.15.59:0/3871880727 >> v1:172.21.15.76:6789/0 conn(0x7f699c0d29d0 legacy=0x7f699c0d0550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:41.891 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:41.888+0000 7f699affd700 1 -- 172.21.15.59:0/3871880727 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f699c0b35c0 con 0x7f699c0cf480 2023-11-24T07:58:41.891 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:41.888+0000 7f699a7fc700 1 -- 172.21.15.59:0/3871880727 <== mon.1 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1700807638 0 0) 0x7f699c0b3370 con 0x7f699c0d29d0 2023-11-24T07:58:41.891 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:41.888+0000 7f699b7fe700 1 --2- 172.21.15.59:0/3871880727 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f699c15e460 0x7f699c0d49d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-24T07:58:41.891 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:41.888+0000 7f699affd700 1 --2- 172.21.15.59:0/3871880727 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f699c0cf480 0x7f699c0d2490 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f698c009840 tx=0x7f698c009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=961ce0bc2b5bd19 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:58:41.891 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:41.888+0000 7f699a7fc700 1 -- 172.21.15.59:0/3871880727 <== mon.1 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f698c011830 con 0x7f699c0cf480 2023-11-24T07:58:41.892 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:41.888+0000 7f699a7fc700 1 -- 172.21.15.59:0/3871880727 <== mon.1 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f698c011990 con 0x7f699c0cf480 2023-11-24T07:58:41.892 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:41.888+0000 7f699a7fc700 1 -- 172.21.15.59:0/3871880727 <== mon.1 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f698c011d20 con 0x7f699c0cf480 2023-11-24T07:58:41.894 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:41.889+0000 7f69a1e0b700 1 -- 172.21.15.59:0/3871880727 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f699c0cf480 msgr2=0x7f699c0d2490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:41.894 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:41.889+0000 7f69a1e0b700 1 --2- 172.21.15.59:0/3871880727 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f699c0cf480 0x7f699c0d2490 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f698c009840 tx=0x7f698c009b50 comp rx=0 tx=0).stop 2023-11-24T07:58:41.894 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:41.889+0000 7f69a1e0b700 1 -- 172.21.15.59:0/3871880727 shutdown_connections 2023-11-24T07:58:41.894 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:41.889+0000 7f69a1e0b700 1 --2- 172.21.15.59:0/3871880727 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f699c15e460 0x7f699c0d49d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:41.894 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:41.889+0000 7f69a1e0b700 1 --2- 172.21.15.59:0/3871880727 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f699c0cf480 0x7f699c0d2490 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:41.895 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:41.889+0000 7f69a1e0b700 1 -- 172.21.15.59:0/3871880727 >> 172.21.15.59:0/3871880727 conn(0x7f699c0d3eb0 msgr2=0x7f699c0d42c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:58:41.895 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:41.889+0000 7f69a1e0b700 1 -- 172.21.15.59:0/3871880727 shutdown_connections 2023-11-24T07:58:41.895 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:41.889+0000 7f69a1e0b700 1 -- 172.21.15.59:0/3871880727 wait complete. 2023-11-24T07:58:41.895 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:41.890+0000 7f69a1e0b700 1 Processor -- start 2023-11-24T07:58:41.895 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:41.890+0000 7f69a1e0b700 1 -- start start 2023-11-24T07:58:41.895 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:41.891+0000 7f69a1e0b700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f699c0cf480 0x7f699c1617f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:41.895 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:41.891+0000 7f69a1e0b700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f699c0d29d0 0x7f699c09e910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:41.895 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:41.891+0000 7f69a1e0b700 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f699c15e460 0x7f699c0ccdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:41.895 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:41.891+0000 7f69a1e0b700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f699c09ee50 con 0x7f699c0cf480 2023-11-24T07:58:41.895 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:41.891+0000 7f69a1e0b700 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f699c0a0d60 con 0x7f699c0d29d0 2023-11-24T07:58:41.895 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:41.891+0000 7f69a1e0b700 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f699c0cd600 con 0x7f699c15e460 2023-11-24T07:58:41.895 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:41.891+0000 7f699b7fe700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f699c0cf480 0x7f699c1617f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:41.896 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:41.891+0000 7f699b7fe700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f699c0cf480 0x7f699c1617f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:53810/0 (socket says 172.21.15.59:53810) 2023-11-24T07:58:41.896 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:41.891+0000 7f699b7fe700 1 -- 172.21.15.59:0/339969479 learned_addr learned my addr 172.21.15.59:0/339969479 (peer_addr_for_me v2:172.21.15.59:0/0) 2023-11-24T07:58:41.896 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:41.892+0000 7f699bfff700 1 --2- 172.21.15.59:0/339969479 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f699c15e460 0x7f699c0ccdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:41.896 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:41.892+0000 7f699affd700 1 --2- 172.21.15.59:0/339969479 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f699c0d29d0 0x7f699c09e910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:41.896 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:41.892+0000 7f699b7fe700 1 -- 172.21.15.59:0/339969479 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f699c15e460 msgr2=0x7f699c0ccdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:41.896 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:41.892+0000 7f699b7fe700 1 --2- 172.21.15.59:0/339969479 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f699c15e460 0x7f699c0ccdb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:41.896 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:41.892+0000 7f699b7fe700 1 -- 172.21.15.59:0/339969479 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f699c0d29d0 msgr2=0x7f699c09e910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:41.896 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:41.892+0000 7f699b7fe700 1 --2- 172.21.15.59:0/339969479 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f699c0d29d0 0x7f699c09e910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:41.896 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:41.892+0000 7f699b7fe700 1 -- 172.21.15.59:0/339969479 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f69840094a0 con 0x7f699c0cf480 2023-11-24T07:58:41.896 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:41.892+0000 7f699b7fe700 1 --2- 172.21.15.59:0/339969479 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f699c0cf480 0x7f699c1617f0 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 crypto rx=0x7f698400ebe0 tx=0x7f6984009e20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:58:41.896 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:41.892+0000 7f6993fff700 1 -- 172.21.15.59:0/339969479 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f698400c8b0 con 0x7f699c0cf480 2023-11-24T07:58:41.896 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:41.892+0000 7f6993fff700 1 -- 172.21.15.59:0/339969479 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f698400ca10 con 0x7f699c0cf480 2023-11-24T07:58:41.896 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:41.892+0000 7f69a1e0b700 1 -- 172.21.15.59:0/339969479 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f698c0094a0 con 0x7f699c0cf480 2023-11-24T07:58:41.897 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:41.892+0000 7f6993fff700 1 -- 172.21.15.59:0/339969479 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f698400cda0 con 0x7f699c0cf480 2023-11-24T07:58:41.897 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:41.892+0000 7f69a1e0b700 1 -- 172.21.15.59:0/339969479 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f699c0cd990 con 0x7f699c0cf480 2023-11-24T07:58:41.897 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:41.893+0000 7f69a1e0b700 1 -- 172.21.15.59:0/339969479 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f699c0cb670 con 0x7f699c0cf480 2023-11-24T07:58:41.898 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:41.894+0000 7f6993fff700 1 -- 172.21.15.59:0/339969479 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 8) v1 ==== 89435+0+0 (secure 0 0 0) 0x7f6984013470 con 0x7f699c0cf480 2023-11-24T07:58:41.898 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:41.894+0000 7f6993fff700 1 --2- 172.21.15.59:0/339969479 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f697c06b140 0x7f697c06d600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:41.898 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:41.894+0000 7f6993fff700 1 -- 172.21.15.59:0/339969479 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(59..59 src has 1..59) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f6984089ba0 con 0x7f699c0cf480 2023-11-24T07:58:41.898 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:41.894+0000 7f699affd700 1 --2- 172.21.15.59:0/339969479 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f697c06b140 0x7f697c06d600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:41.898 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:41.895+0000 7f699affd700 1 --2- 172.21.15.59:0/339969479 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f697c06b140 0x7f697c06d600 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f699c0b20c0 tx=0x7f698c008030 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:58:41.900 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:41.898+0000 7f6993fff700 1 -- 172.21.15.59:0/339969479 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f6984054860 con 0x7f699c0cf480 2023-11-24T07:58:42.106 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:42.104+0000 7f69a1e0b700 1 -- 172.21.15.59:0/339969479 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f699c067f60 con 0x7f699c0cf480 2023-11-24T07:58:42.107 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:42.106+0000 7f6993fff700 1 -- 172.21.15.59:0/339969479 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v59) v1 ==== 74+0+16506 (secure 0 0 0) 0x7f6984017090 con 0x7f699c0cf480 2023-11-24T07:58:42.108 INFO:teuthology.orchestra.run.smithi059.stdout: 2023-11-24T07:58:42.108 INFO:teuthology.orchestra.run.smithi059.stdout:{"epoch":59,"fsid":"9f0e8741-b8f9-48d5-a2c3-09f45903303d","created":"2023-11-24T07:56:06.552729+0000","modified":"2023-11-24T07:58:40.795035+0000","last_up_change":"2023-11-24T07:56:17.372352+0000","last_in_change":"2023-11-24T07:56:09.073250+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-24T07:56:18.352823+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_metadata","create_time":"2023-11-24T07:58:39.262463+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"59","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_data","create_time":"2023-11-24T07:58:39.763399+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"59","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"6d8f9e9b-1782-4e41-928b-a39f2a8ebc06","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6800","nonce":3373878772},{"type":"v1","addr":"172.21.15.59:6801","nonce":3373878772}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6802","nonce":3373878772},{"type":"v1","addr":"172.21.15.59:6803","nonce":3373878772}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6806","nonce":3373878772},{"type":"v1","addr":"172.21.15.59:6807","nonce":3373878772}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6804","nonce":3373878772},{"type":"v1","addr":"172.21.15.59:6805","nonce":3373878772}]},"public_addr":"172.21.15.59:6801/3373878772","cluster_addr":"172.21.15.59:6803/3373878772","heartbeat_back_addr":"172.21.15.59:6807/3373878772","heartbeat_front_addr":"172.21.15.59:6805/3373878772","state":["exists","up"]},{"osd":1,"uuid":"e8d882aa-52e1-4c04-9e1e-c44ef9204a5f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6824","nonce":648258563},{"type":"v1","addr":"172.21.15.59:6825","nonce":648258563}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6826","nonce":648258563},{"type":"v1","addr":"172.21.15.59:6827","nonce":648258563}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6830","nonce":648258563},{"type":"v1","addr":"172.21.15.59:6831","nonce":648258563}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6828","nonce":648258563},{"type":"v1","addr":"172.21.15.59:6829","nonce":648258563}]},"public_addr":"172.21.15.59:6825/648258563","cluster_addr":"172.21.15.59:6827/648258563","heartbeat_back_addr":"172.21.15.59:6831/648258563","heartbeat_front_addr":"172.21.15.59:6829/648258563","state":["exists","up"]},{"osd":2,"uuid":"69eebfa0-695b-4b21-90b9-8a3537168a01","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6808","nonce":1840430654},{"type":"v1","addr":"172.21.15.59:6809","nonce":1840430654}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6810","nonce":1840430654},{"type":"v1","addr":"172.21.15.59:6811","nonce":1840430654}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6814","nonce":1840430654},{"type":"v1","addr":"172.21.15.59:6815","nonce":1840430654}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6812","nonce":1840430654},{"type":"v1","addr":"172.21.15.59:6813","nonce":1840430654}]},"public_addr":"172.21.15.59:6809/1840430654","cluster_addr":"172.21.15.59:6811/1840430654","heartbeat_back_addr":"172.21.15.59:6815/1840430654","heartbeat_front_addr":"172.21.15.59:6813/1840430654","state":["exists","up"]},{"osd":3,"uuid":"96fd77fc-f911-46ff-9445-6f0be2daa8f4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6816","nonce":2504025850},{"type":"v1","addr":"172.21.15.59:6817","nonce":2504025850}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6818","nonce":2504025850},{"type":"v1","addr":"172.21.15.59:6819","nonce":2504025850}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6822","nonce":2504025850},{"type":"v1","addr":"172.21.15.59:6823","nonce":2504025850}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6820","nonce":2504025850},{"type":"v1","addr":"172.21.15.59:6821","nonce":2504025850}]},"public_addr":"172.21.15.59:6817/2504025850","cluster_addr":"172.21.15.59:6819/2504025850","heartbeat_back_addr":"172.21.15.59:6823/2504025850","heartbeat_front_addr":"172.21.15.59:6821/2504025850","state":["exists","up"]},{"osd":4,"uuid":"7ac29fca-b5b2-4c95-9d69-2f164cbbc708","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6816","nonce":2382463431},{"type":"v1","addr":"172.21.15.76:6818","nonce":2382463431}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6820","nonce":2382463431},{"type":"v1","addr":"172.21.15.76:6822","nonce":2382463431}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6828","nonce":2382463431},{"type":"v1","addr":"172.21.15.76:6830","nonce":2382463431}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6824","nonce":2382463431},{"type":"v1","addr":"172.21.15.76:6826","nonce":2382463431}]},"public_addr":"172.21.15.76:6818/2382463431","cluster_addr":"172.21.15.76:6822/2382463431","heartbeat_back_addr":"172.21.15.76:6830/2382463431","heartbeat_front_addr":"172.21.15.76:6826/2382463431","state":["exists","up"]},{"osd":5,"uuid":"485bf711-3851-4238-a4f4-51d52e0a9c58","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6800","nonce":329334308},{"type":"v1","addr":"172.21.15.76:6802","nonce":329334308}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6805","nonce":329334308},{"type":"v1","addr":"172.21.15.76:6807","nonce":329334308}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6814","nonce":329334308},{"type":"v1","addr":"172.21.15.76:6815","nonce":329334308}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6810","nonce":329334308},{"type":"v1","addr":"172.21.15.76:6812","nonce":329334308}]},"public_addr":"172.21.15.76:6802/329334308","cluster_addr":"172.21.15.76:6807/329334308","heartbeat_back_addr":"172.21.15.76:6815/329334308","heartbeat_front_addr":"172.21.15.76:6812/329334308","state":["exists","up"]},{"osd":6,"uuid":"72b3f2fc-1e4a-4e32-968d-003647a0286f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6801","nonce":3946579410},{"type":"v1","addr":"172.21.15.76:6803","nonce":3946579410}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6804","nonce":3946579410},{"type":"v1","addr":"172.21.15.76:6806","nonce":3946579410}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6811","nonce":3946579410},{"type":"v1","addr":"172.21.15.76:6813","nonce":3946579410}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6808","nonce":3946579410},{"type":"v1","addr":"172.21.15.76:6809","nonce":3946579410}]},"public_addr":"172.21.15.76:6803/3946579410","cluster_addr":"172.21.15.76:6806/3946579410","heartbeat_back_addr":"172.21.15.76:6813/3946579410","heartbeat_front_addr":"172.21.15.76:6809/3946579410","state":["exists","up"]},{"osd":7,"uuid":"1ddceb90-6c5d-4d1a-ad56-99cb877f8c70","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6817","nonce":841761538},{"type":"v1","addr":"172.21.15.76:6819","nonce":841761538}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6821","nonce":841761538},{"type":"v1","addr":"172.21.15.76:6823","nonce":841761538}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6829","nonce":841761538},{"type":"v1","addr":"172.21.15.76:6831","nonce":841761538}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6825","nonce":841761538},{"type":"v1","addr":"172.21.15.76:6827","nonce":841761538}]},"public_addr":"172.21.15.76:6819/841761538","cluster_addr":"172.21.15.76:6823/841761538","heartbeat_back_addr":"172.21.15.76:6831/841761538","heartbeat_front_addr":"172.21.15.76:6827/841761538","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T07:56:14.294838+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T07:56:14.270779+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T07:56:14.231809+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T07:56:14.260983+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T07:56:14.302997+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T07:56:14.258600+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T07:56:14.251413+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T07:56:14.304727+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-24T07:58:42.110 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:42.108+0000 7f69a1e0b700 1 -- 172.21.15.59:0/339969479 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f697c06b140 msgr2=0x7f697c06d600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:42.110 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:42.108+0000 7f69a1e0b700 1 --2- 172.21.15.59:0/339969479 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f697c06b140 0x7f697c06d600 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f699c0b20c0 tx=0x7f698c008030 comp rx=0 tx=0).stop 2023-11-24T07:58:42.110 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:42.108+0000 7f69a1e0b700 1 -- 172.21.15.59:0/339969479 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f699c0cf480 msgr2=0x7f699c1617f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:42.110 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:42.108+0000 7f69a1e0b700 1 --2- 172.21.15.59:0/339969479 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f699c0cf480 0x7f699c1617f0 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 crypto rx=0x7f698400ebe0 tx=0x7f6984009e20 comp rx=0 tx=0).stop 2023-11-24T07:58:42.110 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:42.108+0000 7f69a1e0b700 1 -- 172.21.15.59:0/339969479 shutdown_connections 2023-11-24T07:58:42.110 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:42.108+0000 7f69a1e0b700 1 --2- 172.21.15.59:0/339969479 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f697c06b140 0x7f697c06d600 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:42.110 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:42.108+0000 7f69a1e0b700 1 --2- 172.21.15.59:0/339969479 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f699c15e460 0x7f699c0ccdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:42.110 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:42.108+0000 7f69a1e0b700 1 --2- 172.21.15.59:0/339969479 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f699c0cf480 0x7f699c1617f0 unknown :-1 s=CLOSED pgs=701 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:42.111 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:42.108+0000 7f69a1e0b700 1 --2- 172.21.15.59:0/339969479 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f699c0d29d0 0x7f699c09e910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:42.111 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:42.108+0000 7f69a1e0b700 1 -- 172.21.15.59:0/339969479 >> 172.21.15.59:0/339969479 conn(0x7f699c0d3eb0 msgr2=0x7f699c0cebb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:58:42.111 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:42.108+0000 7f69a1e0b700 1 -- 172.21.15.59:0/339969479 shutdown_connections 2023-11-24T07:58:42.111 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:42.108+0000 7f69a1e0b700 1 -- 172.21.15.59:0/339969479 wait complete. 2023-11-24T07:58:42.122 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2023-11-24T07:58:42.250 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:42.248+0000 7f5cc8907700 1 Processor -- start 2023-11-24T07:58:42.250 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:42.249+0000 7f5cc8907700 1 -- start start 2023-11-24T07:58:42.252 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:42.249+0000 7f5cc8907700 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f5cc40b4150 0x7f5cc406de50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:42.252 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:42.249+0000 7f5cc8907700 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f5cc40b5ab0 0x7f5cc406e390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:42.252 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:42.250+0000 7f5cc8907700 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5cc4071f50 con 0x7f5cc406b620 2023-11-24T07:58:42.252 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:42.250+0000 7f5cc8907700 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5cc4072230 con 0x7f5cc40b5ab0 2023-11-24T07:58:42.252 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:42.250+0000 7f5cc8907700 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5cc4072510 con 0x7f5cc40b4150 2023-11-24T07:58:42.252 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:42.250+0000 7f5cc259c700 1 --1- >> v1:172.21.15.76:6789/0 conn(0x7f5cc406b620 0x7f5cc40b3a40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.59:44570/0 (socket says 172.21.15.59:44570) 2023-11-24T07:58:42.252 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:42.250+0000 7f5cc2d9d700 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f5cc40b5ab0 0x7f5cc406e390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:42.253 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:42.250+0000 7f5cc259c700 1 -- 172.21.15.59:0/3082310458 learned_addr learned my addr 172.21.15.59:0/3082310458 (peer_addr_for_me v1:172.21.15.59:0/0) 2023-11-24T07:58:42.253 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:42.250+0000 7f5cc1d9b700 1 --2- 172.21.15.59:0/3082310458 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f5cc40b4150 0x7f5cc406de50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:42.253 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:42.250+0000 7f5cc1d9b700 1 -- 172.21.15.59:0/3082310458 >> v2:172.21.15.76:3300/0 conn(0x7f5cc40b5ab0 msgr2=0x7f5cc406e390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T07:58:42.253 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:42.250+0000 7f5cc1d9b700 1 --2- 172.21.15.59:0/3082310458 >> v2:172.21.15.76:3300/0 conn(0x7f5cc40b5ab0 0x7f5cc406e390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:42.253 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:42.250+0000 7f5cc1d9b700 1 -- 172.21.15.59:0/3082310458 >> v1:172.21.15.76:6789/0 conn(0x7f5cc406b620 legacy=0x7f5cc40b3a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:42.253 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:42.250+0000 7f5cc1d9b700 1 -- 172.21.15.59:0/3082310458 --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5cc40727f0 con 0x7f5cc40b4150 2023-11-24T07:58:42.253 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:42.250+0000 7f5cc1d9b700 1 --2- 172.21.15.59:0/3082310458 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f5cc40b4150 0x7f5cc406de50 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f5cac009840 tx=0x7f5cac009b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ae4db2f1a50a0e4a server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:58:42.253 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:42.250+0000 7f5cc159a700 1 -- 172.21.15.59:0/3082310458 <== mon.2 v2:172.21.15.76:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5cac011830 con 0x7f5cc40b4150 2023-11-24T07:58:42.253 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:42.251+0000 7f5cc159a700 1 -- 172.21.15.59:0/3082310458 <== mon.2 v2:172.21.15.76:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5cac011990 con 0x7f5cc40b4150 2023-11-24T07:58:42.253 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:42.251+0000 7f5cc159a700 1 -- 172.21.15.59:0/3082310458 <== mon.2 v2:172.21.15.76:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5cac011d20 con 0x7f5cc40b4150 2023-11-24T07:58:42.254 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:42.251+0000 7f5cc8907700 1 -- 172.21.15.59:0/3082310458 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f5cc40b4150 msgr2=0x7f5cc406de50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:42.254 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:42.251+0000 7f5cc8907700 1 --2- 172.21.15.59:0/3082310458 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f5cc40b4150 0x7f5cc406de50 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f5cac009840 tx=0x7f5cac009b50 comp rx=0 tx=0).stop 2023-11-24T07:58:42.255 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:42.251+0000 7f5cc8907700 1 -- 172.21.15.59:0/3082310458 shutdown_connections 2023-11-24T07:58:42.255 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:42.251+0000 7f5cc8907700 1 --2- 172.21.15.59:0/3082310458 >> v2:172.21.15.76:3300/0 conn(0x7f5cc40b5ab0 0x7f5cc406e390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:42.255 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:42.251+0000 7f5cc8907700 1 --2- 172.21.15.59:0/3082310458 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f5cc40b4150 0x7f5cc406de50 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:42.255 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:42.251+0000 7f5cc8907700 1 -- 172.21.15.59:0/3082310458 >> 172.21.15.59:0/3082310458 conn(0x7f5cc40b2740 msgr2=0x7f5cc40b2b20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:58:42.255 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:42.251+0000 7f5cc8907700 1 -- 172.21.15.59:0/3082310458 shutdown_connections 2023-11-24T07:58:42.255 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:42.251+0000 7f5cc8907700 1 -- 172.21.15.59:0/3082310458 wait complete. 2023-11-24T07:58:42.255 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:42.253+0000 7f5cc8907700 1 Processor -- start 2023-11-24T07:58:42.255 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:42.253+0000 7f5cc8907700 1 -- start start 2023-11-24T07:58:42.257 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:42.253+0000 7f5cc8907700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5cc406b620 0x7f5cc41386e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:42.257 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:42.253+0000 7f5cc8907700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5cc40b4150 0x7f5cc40c74f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:42.257 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:42.253+0000 7f5cc8907700 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f5cc40b5ab0 0x7f5cc41e8bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:42.257 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:42.253+0000 7f5cc8907700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5cc41e4d30 con 0x7f5cc40b4150 2023-11-24T07:58:42.257 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:42.253+0000 7f5cc8907700 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5cc40c7e50 con 0x7f5cc406b620 2023-11-24T07:58:42.258 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:42.253+0000 7f5cc8907700 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5cc40c8130 con 0x7f5cc40b5ab0 2023-11-24T07:58:42.258 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:42.254+0000 7f5cc1d9b700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5cc40b4150 0x7f5cc40c74f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:42.258 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:42.254+0000 7f5cc1d9b700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5cc40b4150 0x7f5cc40c74f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:53826/0 (socket says 172.21.15.59:53826) 2023-11-24T07:58:42.258 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:42.254+0000 7f5cc1d9b700 1 -- 172.21.15.59:0/3358212347 learned_addr learned my addr 172.21.15.59:0/3358212347 (peer_addr_for_me v2:172.21.15.59:0/0) 2023-11-24T07:58:42.258 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:42.254+0000 7f5cc2d9d700 1 --2- 172.21.15.59:0/3358212347 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f5cc40b5ab0 0x7f5cc41e8bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:42.258 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:42.254+0000 7f5cc259c700 1 --2- 172.21.15.59:0/3358212347 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5cc406b620 0x7f5cc41386e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:42.258 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:42.254+0000 7f5cc1d9b700 1 -- 172.21.15.59:0/3358212347 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f5cc40b5ab0 msgr2=0x7f5cc41e8bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:42.258 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:42.254+0000 7f5cc1d9b700 1 --2- 172.21.15.59:0/3358212347 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f5cc40b5ab0 0x7f5cc41e8bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:42.258 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:42.254+0000 7f5cc1d9b700 1 -- 172.21.15.59:0/3358212347 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5cc406b620 msgr2=0x7f5cc41386e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:42.258 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:42.254+0000 7f5cc1d9b700 1 --2- 172.21.15.59:0/3358212347 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5cc406b620 0x7f5cc41386e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:42.258 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:42.254+0000 7f5cc1d9b700 1 -- 172.21.15.59:0/3358212347 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5cac0094a0 con 0x7f5cc40b4150 2023-11-24T07:58:42.259 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:42.254+0000 7f5cc259c700 1 --2- 172.21.15.59:0/3358212347 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5cc406b620 0x7f5cc41386e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-24T07:58:42.259 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:42.254+0000 7f5cc1d9b700 1 --2- 172.21.15.59:0/3358212347 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5cc40b4150 0x7f5cc40c74f0 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 crypto rx=0x7f5cac009fd0 tx=0x7f5cac004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:58:42.259 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:42.254+0000 7f5cbaffd700 1 -- 172.21.15.59:0/3358212347 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5cac0036a0 con 0x7f5cc40b4150 2023-11-24T07:58:42.259 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:42.254+0000 7f5cbaffd700 1 -- 172.21.15.59:0/3358212347 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5cac01f070 con 0x7f5cc40b4150 2023-11-24T07:58:42.259 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:42.254+0000 7f5cc8907700 1 -- 172.21.15.59:0/3358212347 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5cc40c8410 con 0x7f5cc40b4150 2023-11-24T07:58:42.259 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:42.254+0000 7f5cbaffd700 1 -- 172.21.15.59:0/3358212347 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5cac003a60 con 0x7f5cc40b4150 2023-11-24T07:58:42.259 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:42.254+0000 7f5cc8907700 1 -- 172.21.15.59:0/3358212347 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5cc40c89d0 con 0x7f5cc40b4150 2023-11-24T07:58:42.259 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:42.255+0000 7f5cc8907700 1 -- 172.21.15.59:0/3358212347 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5cc40c8b60 con 0x7f5cc40b4150 2023-11-24T07:58:42.259 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:42.256+0000 7f5cbaffd700 1 -- 172.21.15.59:0/3358212347 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 8) v1 ==== 89435+0+0 (secure 0 0 0) 0x7f5cac003bc0 con 0x7f5cc40b4150 2023-11-24T07:58:42.260 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:42.256+0000 7f5cbaffd700 1 --2- 172.21.15.59:0/3358212347 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f5ca406b110 0x7f5ca406d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:42.260 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:42.256+0000 7f5cbaffd700 1 -- 172.21.15.59:0/3358212347 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(59..59 src has 1..59) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f5cac021070 con 0x7f5cc40b4150 2023-11-24T07:58:42.260 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:42.256+0000 7f5cc259c700 1 --2- 172.21.15.59:0/3358212347 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f5ca406b110 0x7f5ca406d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:42.260 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:42.257+0000 7f5cc259c700 1 --2- 172.21.15.59:0/3358212347 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f5ca406b110 0x7f5ca406d5d0 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f5cb4007fd0 tx=0x7f5cb4009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:58:42.262 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:42.260+0000 7f5cbaffd700 1 -- 172.21.15.59:0/3358212347 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f5cac056030 con 0x7f5cc40b4150 2023-11-24T07:58:42.516 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:42.514+0000 7f5cc8907700 1 -- 172.21.15.59:0/3358212347 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7f5cc4075480 con 0x7f5cc40b4150 2023-11-24T07:58:43.397 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:43.395+0000 7f5cbaffd700 1 -- 172.21.15.59:0/3358212347 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v34) v1 ==== 118+0+0 (secure 0 0 0) 0x7f5cac059650 con 0x7f5cc40b4150 2023-11-24T07:58:43.400 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:43.398+0000 7f5cc8907700 1 -- 172.21.15.59:0/3358212347 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f5ca406b110 msgr2=0x7f5ca406d5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:43.400 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:43.398+0000 7f5cc8907700 1 --2- 172.21.15.59:0/3358212347 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f5ca406b110 0x7f5ca406d5d0 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f5cb4007fd0 tx=0x7f5cb4009040 comp rx=0 tx=0).stop 2023-11-24T07:58:43.400 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:43.398+0000 7f5cc8907700 1 -- 172.21.15.59:0/3358212347 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5cc40b4150 msgr2=0x7f5cc40c74f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:43.400 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:43.398+0000 7f5cc8907700 1 --2- 172.21.15.59:0/3358212347 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5cc40b4150 0x7f5cc40c74f0 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 crypto rx=0x7f5cac009fd0 tx=0x7f5cac004750 comp rx=0 tx=0).stop 2023-11-24T07:58:43.401 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:43.398+0000 7f5cc8907700 1 -- 172.21.15.59:0/3358212347 shutdown_connections 2023-11-24T07:58:43.401 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:43.398+0000 7f5cc8907700 1 --2- 172.21.15.59:0/3358212347 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f5ca406b110 0x7f5ca406d5d0 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:43.401 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:43.398+0000 7f5cc8907700 1 --2- 172.21.15.59:0/3358212347 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f5cc40b5ab0 0x7f5cc41e8bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:43.401 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:43.398+0000 7f5cc8907700 1 --2- 172.21.15.59:0/3358212347 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5cc406b620 0x7f5cc41386e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:43.401 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:43.398+0000 7f5cc8907700 1 --2- 172.21.15.59:0/3358212347 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5cc40b4150 0x7f5cc40c74f0 unknown :-1 s=CLOSED pgs=702 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:43.401 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:43.398+0000 7f5cc8907700 1 -- 172.21.15.59:0/3358212347 >> 172.21.15.59:0/3358212347 conn(0x7f5cc40b2740 msgr2=0x7f5cc41476e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:58:43.401 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:43.398+0000 7f5cc8907700 1 -- 172.21.15.59:0/3358212347 shutdown_connections 2023-11-24T07:58:43.401 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:43.398+0000 7f5cc8907700 1 -- 172.21.15.59:0/3358212347 wait complete. 2023-11-24T07:58:43.415 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2023-11-24T07:58:43.415 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2023-11-24T07:58:43.551 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:43.550+0000 7fd3fd1fe700 1 Processor -- start 2023-11-24T07:58:43.552 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:43.550+0000 7fd3fd1fe700 1 -- start start 2023-11-24T07:58:43.554 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:43.551+0000 7fd3fd1fe700 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fd3f80d01e0 0x7fd3f80d05c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:43.554 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:43.551+0000 7fd3fd1fe700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fd3f80cf020 0x7fd3f806ed70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:43.554 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:43.551+0000 7fd3fd1fe700 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd3f80cf7d0 con 0x7fd3f806f2b0 2023-11-24T07:58:43.554 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:43.551+0000 7fd3fd1fe700 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd3f80d0b00 con 0x7fd3f80cf020 2023-11-24T07:58:43.554 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:43.551+0000 7fd3fd1fe700 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd3f80cfa20 con 0x7fd3f80d01e0 2023-11-24T07:58:43.554 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:43.551+0000 7fd3f659c700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fd3f80cf020 0x7fd3f806ed70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:43.554 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:43.551+0000 7fd3f759e700 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7fd3f806f2b0 0x7fd3f80cd260 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:57308/0 (socket says 172.21.15.59:57308) 2023-11-24T07:58:43.555 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:43.551+0000 7fd3f659c700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fd3f80cf020 0x7fd3f806ed70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:53834/0 (socket says 172.21.15.59:53834) 2023-11-24T07:58:43.555 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:43.551+0000 7fd3f759e700 1 -- 172.21.15.59:0/1068423096 learned_addr learned my addr 172.21.15.59:0/1068423096 (peer_addr_for_me v1:172.21.15.59:0/0) 2023-11-24T07:58:43.555 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:43.551+0000 7fd3f6d9d700 1 --2- 172.21.15.59:0/1068423096 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fd3f80d01e0 0x7fd3f80d05c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:43.555 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:43.551+0000 7fd3f659c700 1 -- 172.21.15.59:0/1068423096 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fd3f80d01e0 msgr2=0x7fd3f80d05c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T07:58:43.555 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:43.551+0000 7fd3f659c700 1 --2- 172.21.15.59:0/1068423096 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fd3f80d01e0 0x7fd3f80d05c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:43.555 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:43.551+0000 7fd3f659c700 1 -- 172.21.15.59:0/1068423096 >> v1:172.21.15.59:6789/0 conn(0x7fd3f806f2b0 legacy=0x7fd3f80cd260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:43.555 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:43.551+0000 7fd3f659c700 1 -- 172.21.15.59:0/1068423096 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd3f80cfd00 con 0x7fd3f80cf020 2023-11-24T07:58:43.555 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:43.551+0000 7fd3f5d9b700 1 -- 172.21.15.59:0/1068423096 <== mon.0 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1882843917 0 0) 0x7fd3f80cf7d0 con 0x7fd3f806f2b0 2023-11-24T07:58:43.555 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:43.551+0000 7fd3f659c700 1 --2- 172.21.15.59:0/1068423096 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd3f80cf020 0x7fd3f806ed70 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 crypto rx=0x7fd3ec009840 tx=0x7fd3ec009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=be14313bc70abf82 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:58:43.555 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:43.551+0000 7fd3f5d9b700 1 -- 172.21.15.59:0/1068423096 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3ec011830 con 0x7fd3f80cf020 2023-11-24T07:58:43.555 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:43.551+0000 7fd3f5d9b700 1 -- 172.21.15.59:0/1068423096 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd3ec011990 con 0x7fd3f80cf020 2023-11-24T07:58:43.556 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:43.552+0000 7fd3f5d9b700 1 -- 172.21.15.59:0/1068423096 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3ec011d20 con 0x7fd3f80cf020 2023-11-24T07:58:43.556 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:43.552+0000 7fd3fd1fe700 1 -- 172.21.15.59:0/1068423096 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd3f80cf020 msgr2=0x7fd3f806ed70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:43.556 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:43.552+0000 7fd3fd1fe700 1 --2- 172.21.15.59:0/1068423096 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd3f80cf020 0x7fd3f806ed70 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 crypto rx=0x7fd3ec009840 tx=0x7fd3ec009b50 comp rx=0 tx=0).stop 2023-11-24T07:58:43.556 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:43.552+0000 7fd3fd1fe700 1 -- 172.21.15.59:0/1068423096 shutdown_connections 2023-11-24T07:58:43.556 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:43.552+0000 7fd3fd1fe700 1 --2- 172.21.15.59:0/1068423096 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fd3f80d01e0 0x7fd3f80d05c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:43.556 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:43.552+0000 7fd3fd1fe700 1 --2- 172.21.15.59:0/1068423096 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd3f80cf020 0x7fd3f806ed70 unknown :-1 s=CLOSED pgs=704 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:43.556 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:43.552+0000 7fd3fd1fe700 1 -- 172.21.15.59:0/1068423096 >> 172.21.15.59:0/1068423096 conn(0x7fd3f80b3790 msgr2=0x7fd3f80b3ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:58:43.556 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:43.552+0000 7fd3fd1fe700 1 -- 172.21.15.59:0/1068423096 shutdown_connections 2023-11-24T07:58:43.556 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:43.552+0000 7fd3fd1fe700 1 -- 172.21.15.59:0/1068423096 wait complete. 2023-11-24T07:58:43.559 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:43.554+0000 7fd3fd1fe700 1 Processor -- start 2023-11-24T07:58:43.559 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:43.554+0000 7fd3fd1fe700 1 -- start start 2023-11-24T07:58:43.559 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:43.554+0000 7fd3fd1fe700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd3f806f2b0 0x7fd3f8152810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:43.559 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:43.554+0000 7fd3fd1fe700 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fd3f80cf020 0x7fd3f80cb1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:43.559 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:43.554+0000 7fd3fd1fe700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd3f80d01e0 0x7fd3f80afac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:43.559 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:43.554+0000 7fd3fd1fe700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3f80c6200 con 0x7fd3f806f2b0 2023-11-24T07:58:43.559 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:43.554+0000 7fd3fd1fe700 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3f80c58d0 con 0x7fd3f80d01e0 2023-11-24T07:58:43.559 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:43.554+0000 7fd3fd1fe700 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd3f80c5bb0 con 0x7fd3f80cf020 2023-11-24T07:58:43.560 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:43.554+0000 7fd3f6d9d700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd3f806f2b0 0x7fd3f8152810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:43.560 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:43.555+0000 7fd3f6d9d700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd3f806f2b0 0x7fd3f8152810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:53838/0 (socket says 172.21.15.59:53838) 2023-11-24T07:58:43.560 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:43.555+0000 7fd3f6d9d700 1 -- 172.21.15.59:0/1966245685 learned_addr learned my addr 172.21.15.59:0/1966245685 (peer_addr_for_me v2:172.21.15.59:0/0) 2023-11-24T07:58:43.560 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:43.555+0000 7fd3f6d9d700 1 -- 172.21.15.59:0/1966245685 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fd3f80cf020 msgr2=0x7fd3f80cb1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:43.560 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:43.555+0000 7fd3f6d9d700 1 --2- 172.21.15.59:0/1966245685 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fd3f80cf020 0x7fd3f80cb1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:43.560 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:43.555+0000 7fd3f6d9d700 1 -- 172.21.15.59:0/1966245685 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd3f80d01e0 msgr2=0x7fd3f80afac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:43.560 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:43.555+0000 7fd3f759e700 1 --2- 172.21.15.59:0/1966245685 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd3f80d01e0 0x7fd3f80afac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:43.560 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:43.555+0000 7fd3f6d9d700 1 --2- 172.21.15.59:0/1966245685 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd3f80d01e0 0x7fd3f80afac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:43.560 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:43.555+0000 7fd3f6d9d700 1 -- 172.21.15.59:0/1966245685 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd3ec0094a0 con 0x7fd3f806f2b0 2023-11-24T07:58:43.561 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:43.555+0000 7fd3f759e700 1 --2- 172.21.15.59:0/1966245685 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd3f80d01e0 0x7fd3f80afac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T07:58:43.561 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:43.555+0000 7fd3f6d9d700 1 --2- 172.21.15.59:0/1966245685 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd3f806f2b0 0x7fd3f8152810 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 crypto rx=0x7fd3e800e870 tx=0x7fd3e800ec30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:58:43.561 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:43.555+0000 7fd3df7fe700 1 -- 172.21.15.59:0/1966245685 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3e8009590 con 0x7fd3f806f2b0 2023-11-24T07:58:43.561 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:43.555+0000 7fd3df7fe700 1 -- 172.21.15.59:0/1966245685 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd3e800ee70 con 0x7fd3f806f2b0 2023-11-24T07:58:43.561 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:43.555+0000 7fd3fd1fe700 1 -- 172.21.15.59:0/1966245685 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd3f80c5e90 con 0x7fd3f806f2b0 2023-11-24T07:58:43.561 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:43.555+0000 7fd3df7fe700 1 -- 172.21.15.59:0/1966245685 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3e800ca50 con 0x7fd3f806f2b0 2023-11-24T07:58:43.561 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:43.555+0000 7fd3fd1fe700 1 -- 172.21.15.59:0/1966245685 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd3f815c800 con 0x7fd3f806f2b0 2023-11-24T07:58:43.561 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:43.556+0000 7fd3fd1fe700 1 -- 172.21.15.59:0/1966245685 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd3f8075480 con 0x7fd3f806f2b0 2023-11-24T07:58:43.561 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:43.557+0000 7fd3df7fe700 1 -- 172.21.15.59:0/1966245685 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 8) v1 ==== 89435+0+0 (secure 0 0 0) 0x7fd3e800cbb0 con 0x7fd3f806f2b0 2023-11-24T07:58:43.561 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:43.557+0000 7fd3df7fe700 1 --2- 172.21.15.59:0/1966245685 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7fd3e406b110 0x7fd3e406d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:43.561 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:43.557+0000 7fd3df7fe700 1 -- 172.21.15.59:0/1966245685 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(60..60 src has 1..60) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fd3e80898f0 con 0x7fd3f806f2b0 2023-11-24T07:58:43.562 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:43.557+0000 7fd3f659c700 1 --2- 172.21.15.59:0/1966245685 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7fd3e406b110 0x7fd3e406d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:43.562 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:43.558+0000 7fd3f659c700 1 --2- 172.21.15.59:0/1966245685 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7fd3e406b110 0x7fd3e406d5d0 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7fd3f80cdc40 tx=0x7fd3ec010000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:58:43.563 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:43.561+0000 7fd3df7fe700 1 -- 172.21.15.59:0/1966245685 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fd3e80546d0 con 0x7fd3f806f2b0 2023-11-24T07:58:43.812 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:43.811+0000 7fd3fd1fe700 1 -- 172.21.15.59:0/1966245685 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7fd3f80743b0 con 0x7fd3f806f2b0 2023-11-24T07:58:44.402 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:44.401+0000 7fd3df7fe700 1 -- 172.21.15.59:0/1966245685 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v35) v1 ==== 122+0+0 (secure 0 0 0) 0x7fd3e80210c0 con 0x7fd3f806f2b0 2023-11-24T07:58:44.406 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:44.404+0000 7fd3fd1fe700 1 -- 172.21.15.59:0/1966245685 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7fd3e406b110 msgr2=0x7fd3e406d5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:44.406 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:44.404+0000 7fd3fd1fe700 1 --2- 172.21.15.59:0/1966245685 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7fd3e406b110 0x7fd3e406d5d0 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7fd3f80cdc40 tx=0x7fd3ec010000 comp rx=0 tx=0).stop 2023-11-24T07:58:44.406 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:44.404+0000 7fd3fd1fe700 1 -- 172.21.15.59:0/1966245685 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd3f806f2b0 msgr2=0x7fd3f8152810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:44.406 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:44.404+0000 7fd3fd1fe700 1 --2- 172.21.15.59:0/1966245685 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd3f806f2b0 0x7fd3f8152810 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 crypto rx=0x7fd3e800e870 tx=0x7fd3e800ec30 comp rx=0 tx=0).stop 2023-11-24T07:58:44.406 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:44.404+0000 7fd3fd1fe700 1 -- 172.21.15.59:0/1966245685 shutdown_connections 2023-11-24T07:58:44.406 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:44.404+0000 7fd3fd1fe700 1 --2- 172.21.15.59:0/1966245685 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7fd3e406b110 0x7fd3e406d5d0 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:44.406 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:44.404+0000 7fd3fd1fe700 1 --2- 172.21.15.59:0/1966245685 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd3f80d01e0 0x7fd3f80afac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:44.406 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:44.404+0000 7fd3fd1fe700 1 --2- 172.21.15.59:0/1966245685 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd3f806f2b0 0x7fd3f8152810 unknown :-1 s=CLOSED pgs=705 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:44.406 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:44.404+0000 7fd3fd1fe700 1 --2- 172.21.15.59:0/1966245685 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fd3f80cf020 0x7fd3f80cb1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:44.407 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:44.404+0000 7fd3fd1fe700 1 -- 172.21.15.59:0/1966245685 >> 172.21.15.59:0/1966245685 conn(0x7fd3f80b3790 msgr2=0x7fd3f806f8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:58:44.407 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:44.404+0000 7fd3fd1fe700 1 -- 172.21.15.59:0/1966245685 shutdown_connections 2023-11-24T07:58:44.407 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:44.404+0000 7fd3fd1fe700 1 -- 172.21.15.59:0/1966245685 wait complete. 2023-11-24T07:58:44.421 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2023-11-24T07:58:44.421 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-24T07:58:44.552 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:44.551+0000 7fd3d412e700 1 Processor -- start 2023-11-24T07:58:44.553 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:44.551+0000 7fd3d412e700 1 -- start start 2023-11-24T07:58:44.555 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:44.552+0000 7fd3d412e700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fd3cc0af980 0x7fd3cc066ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:44.555 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:44.552+0000 7fd3d412e700 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fd3cc06b7c0 0x7fd3cc06bc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:44.555 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:44.552+0000 7fd3d412e700 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd3cc0b10c0 con 0x7fd3cc15bf10 2023-11-24T07:58:44.555 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:44.552+0000 7fd3d412e700 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd3cc06c170 con 0x7fd3cc0af980 2023-11-24T07:58:44.555 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:44.552+0000 7fd3d412e700 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd3cc0af730 con 0x7fd3cc06b7c0 2023-11-24T07:58:44.555 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:44.552+0000 7fd3d16c9700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fd3cc0af980 0x7fd3cc066ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:44.555 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:44.552+0000 7fd3d16c9700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fd3cc0af980 0x7fd3cc066ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:53848/0 (socket says 172.21.15.59:53848) 2023-11-24T07:58:44.556 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:44.552+0000 7fd3d1eca700 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7fd3cc15bf10 0x7fd3cc0b1550 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:57320/0 (socket says 172.21.15.59:57320) 2023-11-24T07:58:44.556 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:44.552+0000 7fd3d16c9700 1 -- 172.21.15.59:0/3964031956 learned_addr learned my addr 172.21.15.59:0/3964031956 (peer_addr_for_me v2:172.21.15.59:0/0) 2023-11-24T07:58:44.556 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:44.552+0000 7fd3d26cb700 1 --2- 172.21.15.59:0/3964031956 >> v2:172.21.15.76:3300/0 conn(0x7fd3cc06b7c0 0x7fd3cc06bc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:44.556 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:44.552+0000 7fd3d16c9700 1 -- 172.21.15.59:0/3964031956 >> v2:172.21.15.76:3300/0 conn(0x7fd3cc06b7c0 msgr2=0x7fd3cc06bc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T07:58:44.556 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:44.552+0000 7fd3d16c9700 1 --2- 172.21.15.59:0/3964031956 >> v2:172.21.15.76:3300/0 conn(0x7fd3cc06b7c0 0x7fd3cc06bc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:44.556 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:44.552+0000 7fd3d16c9700 1 -- 172.21.15.59:0/3964031956 >> v1:172.21.15.59:6789/0 conn(0x7fd3cc15bf10 legacy=0x7fd3cc0b1550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:44.556 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:44.552+0000 7fd3d16c9700 1 -- 172.21.15.59:0/3964031956 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd3cc0b1c60 con 0x7fd3cc0af980 2023-11-24T07:58:44.556 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:44.552+0000 7fd3d0ec8700 1 -- 172.21.15.59:0/3964031956 <== mon.0 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2919930685 0 0) 0x7fd3cc0b10c0 con 0x7fd3cc15bf10 2023-11-24T07:58:44.556 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:44.552+0000 7fd3d26cb700 1 --2- 172.21.15.59:0/3964031956 >> v2:172.21.15.76:3300/0 conn(0x7fd3cc06b7c0 0x7fd3cc06bc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-24T07:58:44.556 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:44.553+0000 7fd3d16c9700 1 --2- 172.21.15.59:0/3964031956 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd3cc0af980 0x7fd3cc066ce0 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 crypto rx=0x7fd3cc071720 tx=0x7fd3bc009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=460a78851ab433e3 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:58:44.556 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:44.553+0000 7fd3d0ec8700 1 -- 172.21.15.59:0/3964031956 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3bc00b870 con 0x7fd3cc0af980 2023-11-24T07:58:44.556 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:44.553+0000 7fd3d0ec8700 1 -- 172.21.15.59:0/3964031956 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd3bc00b9d0 con 0x7fd3cc0af980 2023-11-24T07:58:44.557 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:44.553+0000 7fd3d0ec8700 1 -- 172.21.15.59:0/3964031956 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3bc00bd60 con 0x7fd3cc0af980 2023-11-24T07:58:44.557 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:44.553+0000 7fd3d412e700 1 -- 172.21.15.59:0/3964031956 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd3cc0af980 msgr2=0x7fd3cc066ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:44.557 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:44.553+0000 7fd3d412e700 1 --2- 172.21.15.59:0/3964031956 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd3cc0af980 0x7fd3cc066ce0 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 crypto rx=0x7fd3cc071720 tx=0x7fd3bc009bd0 comp rx=0 tx=0).stop 2023-11-24T07:58:44.557 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:44.553+0000 7fd3d412e700 1 -- 172.21.15.59:0/3964031956 shutdown_connections 2023-11-24T07:58:44.557 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:44.553+0000 7fd3d412e700 1 --2- 172.21.15.59:0/3964031956 >> v2:172.21.15.76:3300/0 conn(0x7fd3cc06b7c0 0x7fd3cc06bc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:44.557 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:44.553+0000 7fd3d412e700 1 --2- 172.21.15.59:0/3964031956 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd3cc0af980 0x7fd3cc066ce0 unknown :-1 s=CLOSED pgs=707 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:44.557 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:44.553+0000 7fd3d412e700 1 -- 172.21.15.59:0/3964031956 >> 172.21.15.59:0/3964031956 conn(0x7fd3cc0b5070 msgr2=0x7fd3cc0b5730 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:58:44.557 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:44.553+0000 7fd3d412e700 1 -- 172.21.15.59:0/3964031956 shutdown_connections 2023-11-24T07:58:44.557 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:44.553+0000 7fd3d412e700 1 -- 172.21.15.59:0/3964031956 wait complete. 2023-11-24T07:58:44.560 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:44.555+0000 7fd3d412e700 1 Processor -- start 2023-11-24T07:58:44.560 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:44.555+0000 7fd3d412e700 1 -- start start 2023-11-24T07:58:44.560 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:44.555+0000 7fd3d412e700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd3cc06b7c0 0x7fd3cc14d750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:44.560 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:44.555+0000 7fd3d412e700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd3cc0af980 0x7fd3cc1f3760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:44.560 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:44.555+0000 7fd3d412e700 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fd3cc15bf10 0x7fd3cc15e400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:44.560 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:44.555+0000 7fd3d412e700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3cc14dc90 con 0x7fd3cc0af980 2023-11-24T07:58:44.560 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:44.555+0000 7fd3d412e700 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3cc1f7240 con 0x7fd3cc06b7c0 2023-11-24T07:58:44.560 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:44.555+0000 7fd3d412e700 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd3cc1f7520 con 0x7fd3cc15bf10 2023-11-24T07:58:44.560 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:44.555+0000 7fd3d16c9700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd3cc0af980 0x7fd3cc1f3760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:44.560 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:44.555+0000 7fd3d16c9700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd3cc0af980 0x7fd3cc1f3760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:53854/0 (socket says 172.21.15.59:53854) 2023-11-24T07:58:44.560 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:44.555+0000 7fd3d16c9700 1 -- 172.21.15.59:0/510078723 learned_addr learned my addr 172.21.15.59:0/510078723 (peer_addr_for_me v2:172.21.15.59:0/0) 2023-11-24T07:58:44.560 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:44.556+0000 7fd3d16c9700 1 -- 172.21.15.59:0/510078723 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fd3cc15bf10 msgr2=0x7fd3cc15e400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:44.561 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:44.556+0000 7fd3d16c9700 1 --2- 172.21.15.59:0/510078723 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fd3cc15bf10 0x7fd3cc15e400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:44.561 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:44.556+0000 7fd3d16c9700 1 -- 172.21.15.59:0/510078723 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd3cc06b7c0 msgr2=0x7fd3cc14d750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:44.561 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:44.556+0000 7fd3d16c9700 1 --2- 172.21.15.59:0/510078723 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd3cc06b7c0 0x7fd3cc14d750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:44.561 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:44.556+0000 7fd3d16c9700 1 -- 172.21.15.59:0/510078723 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd3bc009550 con 0x7fd3cc0af980 2023-11-24T07:58:44.561 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:44.556+0000 7fd3d16c9700 1 --2- 172.21.15.59:0/510078723 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd3cc0af980 0x7fd3cc1f3760 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x7fd3bc009c10 tx=0x7fd3bc004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:58:44.561 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:44.556+0000 7fd3c27fc700 1 -- 172.21.15.59:0/510078723 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3bc004250 con 0x7fd3cc0af980 2023-11-24T07:58:44.561 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:44.556+0000 7fd3c27fc700 1 -- 172.21.15.59:0/510078723 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd3bc01f070 con 0x7fd3cc0af980 2023-11-24T07:58:44.561 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:44.556+0000 7fd3d412e700 1 -- 172.21.15.59:0/510078723 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd3cc1f7800 con 0x7fd3cc0af980 2023-11-24T07:58:44.561 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:44.556+0000 7fd3c27fc700 1 -- 172.21.15.59:0/510078723 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3bc0045d0 con 0x7fd3cc0af980 2023-11-24T07:58:44.561 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:44.556+0000 7fd3d412e700 1 -- 172.21.15.59:0/510078723 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd3cc0c9590 con 0x7fd3cc0af980 2023-11-24T07:58:44.561 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:44.557+0000 7fd3d412e700 1 -- 172.21.15.59:0/510078723 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd3cc0c9720 con 0x7fd3cc0af980 2023-11-24T07:58:44.561 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:44.558+0000 7fd3c27fc700 1 -- 172.21.15.59:0/510078723 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 8) v1 ==== 89435+0+0 (secure 0 0 0) 0x7fd3bc021030 con 0x7fd3cc0af980 2023-11-24T07:58:44.562 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:44.558+0000 7fd3c27fc700 1 --2- 172.21.15.59:0/510078723 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7fd3b006b110 0x7fd3b006d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:44.562 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:44.558+0000 7fd3c27fc700 1 -- 172.21.15.59:0/510078723 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(60..60 src has 1..60) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fd3bc08ae40 con 0x7fd3cc0af980 2023-11-24T07:58:44.562 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:44.558+0000 7fd3d1eca700 1 --2- 172.21.15.59:0/510078723 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7fd3b006b110 0x7fd3b006d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:44.562 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:44.559+0000 7fd3d1eca700 1 --2- 172.21.15.59:0/510078723 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7fd3b006b110 0x7fd3b006d5d0 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7fd3cc1578a0 tx=0x7fd3c80079f0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:58:44.564 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:44.563+0000 7fd3c27fc700 1 -- 172.21.15.59:0/510078723 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fd3bc055d40 con 0x7fd3cc0af980 2023-11-24T07:58:44.809 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:44.808+0000 7fd3d412e700 1 -- 172.21.15.59:0/510078723 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fd3cc068670 con 0x7fd3cc0af980 2023-11-24T07:58:44.810 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:44.808+0000 7fd3c27fc700 1 -- 172.21.15.59:0/510078723 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 35 v35) v1 ==== 94+0+4907 (secure 0 0 0) 0x7fd3bc015090 con 0x7fd3cc0af980 2023-11-24T07:58:44.810 INFO:teuthology.orchestra.run.smithi059.stdout: 2023-11-24T07:58:44.810 INFO:teuthology.orchestra.run.smithi059.stdout:{"epoch":35,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5300,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6834/172682993","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6833","nonce":172682993},{"type":"v1","addr":"172.21.15.59:6834","nonce":172682993}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":29},{"gid":5701,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.76:6836/923821877","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6834","nonce":923821877},{"type":"v1","addr":"172.21.15.76:6836","nonce":923821877}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":30},{"gid":5704,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.76:6837/4166482716","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6835","nonce":4166482716},{"type":"v1","addr":"172.21.15.76:6837","nonce":4166482716}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":29}],"filesystems":[{"mdsmap":{"epoch":35,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-24T07:58:40.795340+0000","modified":"2023-11-24T07:58:43.812382+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":6093},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6093":{"gid":6093,"name":"a","rank":0,"incarnation":31,"state":"up:active","state_seq":3,"addr":"172.21.15.59:6835/1599845006","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6832","nonce":1599845006},{"type":"v1","addr":"172.21.15.59:6835","nonce":1599845006}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[9],"metadata_pool":8,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":4}]} 2023-11-24T07:58:44.812 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:44.810+0000 7fd3d412e700 1 -- 172.21.15.59:0/510078723 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7fd3b006b110 msgr2=0x7fd3b006d5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:44.812 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:44.810+0000 7fd3d412e700 1 --2- 172.21.15.59:0/510078723 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7fd3b006b110 0x7fd3b006d5d0 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7fd3cc1578a0 tx=0x7fd3c80079f0 comp rx=0 tx=0).stop 2023-11-24T07:58:44.812 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:44.810+0000 7fd3d412e700 1 -- 172.21.15.59:0/510078723 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd3cc0af980 msgr2=0x7fd3cc1f3760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:44.812 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:44.810+0000 7fd3d412e700 1 --2- 172.21.15.59:0/510078723 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd3cc0af980 0x7fd3cc1f3760 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x7fd3bc009c10 tx=0x7fd3bc004750 comp rx=0 tx=0).stop 2023-11-24T07:58:44.813 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:44.810+0000 7fd3d412e700 1 -- 172.21.15.59:0/510078723 shutdown_connections 2023-11-24T07:58:44.813 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:44.810+0000 7fd3d412e700 1 --2- 172.21.15.59:0/510078723 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7fd3b006b110 0x7fd3b006d5d0 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:44.813 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:44.810+0000 7fd3d412e700 1 --2- 172.21.15.59:0/510078723 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fd3cc15bf10 0x7fd3cc15e400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:44.813 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:44.810+0000 7fd3d412e700 1 --2- 172.21.15.59:0/510078723 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd3cc06b7c0 0x7fd3cc14d750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:44.813 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:44.810+0000 7fd3d412e700 1 --2- 172.21.15.59:0/510078723 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd3cc0af980 0x7fd3cc1f3760 unknown :-1 s=CLOSED pgs=708 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:44.813 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:44.810+0000 7fd3d412e700 1 -- 172.21.15.59:0/510078723 >> 172.21.15.59:0/510078723 conn(0x7fd3cc0b5070 msgr2=0x7fd3cc0aed20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:58:44.813 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:44.810+0000 7fd3d412e700 1 -- 172.21.15.59:0/510078723 shutdown_connections 2023-11-24T07:58:44.813 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:44.810+0000 7fd3d412e700 1 -- 172.21.15.59:0/510078723 wait complete. 2023-11-24T07:58:44.813 INFO:teuthology.orchestra.run.smithi059.stderr:dumped fsmap epoch 35 2023-11-24T07:58:44.825 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-24T07:58:44.949 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:44.947+0000 7f92ee12a700 1 Processor -- start 2023-11-24T07:58:44.949 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:44.947+0000 7f92ee12a700 1 -- start start 2023-11-24T07:58:44.953 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:44.948+0000 7f92ee12a700 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f92e806b620 0x7f92e80b3a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:44.953 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:44.948+0000 7f92ee12a700 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f92e81629c0 con 0x7f92e80b5ab0 2023-11-24T07:58:44.953 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:44.948+0000 7f92ee12a700 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f92e8071f50 con 0x7f92e806ee60 2023-11-24T07:58:44.953 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:44.948+0000 7f92ee12a700 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f92e8072230 con 0x7f92e806b620 2023-11-24T07:58:44.953 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:44.948+0000 7f92e6ffd700 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7f92e80b5ab0 0x7f92e8161ba0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:57336/0 (socket says 172.21.15.59:57336) 2023-11-24T07:58:44.954 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:44.948+0000 7f92e6ffd700 1 -- 172.21.15.59:0/4256247702 learned_addr learned my addr 172.21.15.59:0/4256247702 (peer_addr_for_me v1:172.21.15.59:0/0) 2023-11-24T07:58:44.954 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:44.948+0000 7f92e77fe700 1 --2- 172.21.15.59:0/4256247702 >> v2:172.21.15.76:3300/0 conn(0x7f92e806b620 0x7f92e80b3a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:44.954 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:44.948+0000 7f92e67fc700 1 -- 172.21.15.59:0/4256247702 <== mon.0 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1704428445 0 0) 0x7f92e81629c0 con 0x7f92e80b5ab0 2023-11-24T07:58:44.954 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:44.948+0000 7f92e67fc700 1 -- 172.21.15.59:0/4256247702 --> v1:172.21.15.59:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f92d40035b0 con 0x7f92e80b5ab0 2023-11-24T07:58:44.954 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:44.948+0000 7f92e67fc700 1 -- 172.21.15.59:0/4256247702 <== mon.0 v1:172.21.15.59:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3190054462 0 0) 0x7f92d40035b0 con 0x7f92e80b5ab0 2023-11-24T07:58:44.954 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:44.948+0000 7f92e67fc700 1 -- 172.21.15.59:0/4256247702 >> v2:172.21.15.76:3300/0 conn(0x7f92e806b620 msgr2=0x7f92e80b3a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T07:58:44.954 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:44.948+0000 7f92e67fc700 1 --2- 172.21.15.59:0/4256247702 >> v2:172.21.15.76:3300/0 conn(0x7f92e806b620 0x7f92e80b3a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:44.954 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:44.948+0000 7f92e67fc700 1 -- 172.21.15.59:0/4256247702 >> v1:172.21.15.76:6789/0 conn(0x7f92e806ee60 legacy=0x7f92e81622b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:44.954 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:44.949+0000 7f92e67fc700 1 -- 172.21.15.59:0/4256247702 --> v1:172.21.15.59:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f92e8072510 con 0x7f92e80b5ab0 2023-11-24T07:58:44.954 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:44.949+0000 7f92e67fc700 1 -- 172.21.15.59:0/4256247702 <== mon.0 v1:172.21.15.59:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1312333566 0 0) 0x7f92d0001670 con 0x7f92e80b5ab0 2023-11-24T07:58:44.954 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:44.949+0000 7f92e67fc700 1 -- 172.21.15.59:0/4256247702 >> v1:172.21.15.59:6789/0 conn(0x7f92e80b5ab0 legacy=0x7f92e8161ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:44.955 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:44.949+0000 7f92e67fc700 1 --2- 172.21.15.59:0/4256247702 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f92d4003a00 0x7f92d4003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:44.955 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:44.949+0000 7f92e67fc700 1 --2- 172.21.15.59:0/4256247702 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f92d4004400 0x7f92d40048d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:44.955 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:44.949+0000 7f92e67fc700 1 --2- 172.21.15.59:0/4256247702 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f92d40050d0 0x7f92d400ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:44.955 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:44.949+0000 7f92e67fc700 1 -- 172.21.15.59:0/4256247702 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92e8072230 con 0x7f92d4004400 2023-11-24T07:58:44.955 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:44.949+0000 7f92e67fc700 1 -- 172.21.15.59:0/4256247702 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92d400c030 con 0x7f92d4003a00 2023-11-24T07:58:44.955 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:44.949+0000 7f92e67fc700 1 -- 172.21.15.59:0/4256247702 --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f92d400c1f0 con 0x7f92d40050d0 2023-11-24T07:58:44.955 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:44.949+0000 7f92e6ffd700 1 --2- 172.21.15.59:0/4256247702 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f92d4004400 0x7f92d40048d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:44.955 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:44.949+0000 7f92e77fe700 1 --2- 172.21.15.59:0/4256247702 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f92d4003a00 0x7f92d4003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:44.955 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:44.949+0000 7f92e7fff700 1 --2- 172.21.15.59:0/4256247702 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f92d40050d0 0x7f92d400ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:44.955 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:44.949+0000 7f92e6ffd700 1 -- 172.21.15.59:0/4256247702 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f92d40050d0 msgr2=0x7f92d400ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T07:58:44.955 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:44.949+0000 7f92e6ffd700 1 --2- 172.21.15.59:0/4256247702 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f92d40050d0 0x7f92d400ba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:44.956 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:44.949+0000 7f92e6ffd700 1 -- 172.21.15.59:0/4256247702 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f92d4003a00 msgr2=0x7f92d4003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T07:58:44.956 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:44.949+0000 7f92e6ffd700 1 --2- 172.21.15.59:0/4256247702 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f92d4003a00 0x7f92d4003de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:44.956 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:44.949+0000 7f92e6ffd700 1 -- 172.21.15.59:0/4256247702 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f92d40035b0 con 0x7f92d4004400 2023-11-24T07:58:44.956 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:44.949+0000 7f92e7fff700 1 --2- 172.21.15.59:0/4256247702 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f92d40050d0 0x7f92d400ba10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T07:58:44.956 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:44.949+0000 7f92e7fff700 1 -- 172.21.15.59:0/4256247702 reap_dead start 2023-11-24T07:58:44.956 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:44.949+0000 7f92e6ffd700 1 --2- 172.21.15.59:0/4256247702 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f92d4004400 0x7f92d40048d0 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 crypto rx=0x7f92d000c210 tx=0x7f92d4005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=338d342d2884d42f server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:58:44.956 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:44.949+0000 7f92e67fc700 1 -- 172.21.15.59:0/4256247702 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92d000cdb0 con 0x7f92d4004400 2023-11-24T07:58:44.956 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:44.949+0000 7f92e67fc700 1 -- 172.21.15.59:0/4256247702 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f92d0007ba0 con 0x7f92d4004400 2023-11-24T07:58:44.956 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:44.949+0000 7f92e67fc700 1 -- 172.21.15.59:0/4256247702 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92d0005ba0 con 0x7f92d4004400 2023-11-24T07:58:44.956 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:44.950+0000 7f92ee12a700 1 -- 172.21.15.59:0/4256247702 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f92d4004400 msgr2=0x7f92d40048d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:44.956 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:44.950+0000 7f92ee12a700 1 --2- 172.21.15.59:0/4256247702 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f92d4004400 0x7f92d40048d0 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 crypto rx=0x7f92d000c210 tx=0x7f92d4005880 comp rx=0 tx=0).stop 2023-11-24T07:58:44.957 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:44.950+0000 7f92ee12a700 1 -- 172.21.15.59:0/4256247702 shutdown_connections 2023-11-24T07:58:44.957 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:44.950+0000 7f92ee12a700 1 --2- 172.21.15.59:0/4256247702 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f92d4004400 0x7f92d40048d0 unknown :-1 s=CLOSED pgs=710 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:44.957 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:44.950+0000 7f92ee12a700 1 -- 172.21.15.59:0/4256247702 >> 172.21.15.59:0/4256247702 conn(0x7f92e80b2740 msgr2=0x7f92e80b2b20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:58:44.957 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:44.950+0000 7f92ee12a700 1 -- 172.21.15.59:0/4256247702 shutdown_connections 2023-11-24T07:58:44.957 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:44.950+0000 7f92ee12a700 1 -- 172.21.15.59:0/4256247702 wait complete. 2023-11-24T07:58:44.957 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:44.951+0000 7f92ee12a700 1 Processor -- start 2023-11-24T07:58:44.957 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:44.951+0000 7f92ee12a700 1 -- start start 2023-11-24T07:58:44.960 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:44.952+0000 7f92ee12a700 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f92d4004400 0x7f92e80cfd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:44.960 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:44.952+0000 7f92ee12a700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f92e81410a0 0x7f92e813f180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:44.960 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:44.952+0000 7f92ee12a700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f92e8141500 0x7f92e813f6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:44.961 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:44.952+0000 7f92ee12a700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92e80d02a0 con 0x7f92e8141500 2023-11-24T07:58:44.961 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:44.952+0000 7f92ee12a700 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92e80c6330 con 0x7f92e81410a0 2023-11-24T07:58:44.961 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:44.952+0000 7f92ee12a700 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f92e8151960 con 0x7f92d4004400 2023-11-24T07:58:44.961 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:44.952+0000 7f92e7fff700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f92e8141500 0x7f92e813f6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:44.961 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:44.952+0000 7f92e7fff700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f92e8141500 0x7f92e813f6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:53876/0 (socket says 172.21.15.59:53876) 2023-11-24T07:58:44.961 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:44.952+0000 7f92e7fff700 1 -- 172.21.15.59:0/406214749 learned_addr learned my addr 172.21.15.59:0/406214749 (peer_addr_for_me v2:172.21.15.59:0/0) 2023-11-24T07:58:44.961 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:44.952+0000 7f92e77fe700 1 --2- 172.21.15.59:0/406214749 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f92d4004400 0x7f92e80cfd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:44.961 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:44.952+0000 7f92e6ffd700 1 --2- 172.21.15.59:0/406214749 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f92e81410a0 0x7f92e813f180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:44.961 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:44.952+0000 7f92e7fff700 1 -- 172.21.15.59:0/406214749 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f92d4004400 msgr2=0x7f92e80cfd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:44.961 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:44.952+0000 7f92e7fff700 1 --2- 172.21.15.59:0/406214749 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f92d4004400 0x7f92e80cfd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:44.961 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:44.952+0000 7f92e7fff700 1 -- 172.21.15.59:0/406214749 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f92e81410a0 msgr2=0x7f92e813f180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:44.961 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:44.952+0000 7f92e7fff700 1 --2- 172.21.15.59:0/406214749 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f92e81410a0 0x7f92e813f180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:44.962 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:44.953+0000 7f92e7fff700 1 -- 172.21.15.59:0/406214749 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f92e8151c40 con 0x7f92e8141500 2023-11-24T07:58:44.962 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:44.953+0000 7f92e7fff700 1 --2- 172.21.15.59:0/406214749 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f92e8141500 0x7f92e813f6c0 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 crypto rx=0x7f92e8073020 tx=0x7f92e000de60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:58:44.962 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:44.953+0000 7f92dffff700 1 -- 172.21.15.59:0/406214749 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92e0003e10 con 0x7f92e8141500 2023-11-24T07:58:44.962 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:44.953+0000 7f92dffff700 1 -- 172.21.15.59:0/406214749 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f92e0005070 con 0x7f92e8141500 2023-11-24T07:58:44.962 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:44.953+0000 7f92ee12a700 1 -- 172.21.15.59:0/406214749 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f92e8151f20 con 0x7f92e8141500 2023-11-24T07:58:44.962 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:44.953+0000 7f92dffff700 1 -- 172.21.15.59:0/406214749 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92e0005420 con 0x7f92e8141500 2023-11-24T07:58:44.962 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:44.953+0000 7f92ee12a700 1 -- 172.21.15.59:0/406214749 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f92e81524e0 con 0x7f92e8141500 2023-11-24T07:58:44.962 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:44.954+0000 7f92ee12a700 1 -- 172.21.15.59:0/406214749 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f92e80b54d0 con 0x7f92e8141500 2023-11-24T07:58:44.962 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:44.954+0000 7f92dffff700 1 -- 172.21.15.59:0/406214749 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 8) v1 ==== 89435+0+0 (secure 0 0 0) 0x7f92e0005580 con 0x7f92e8141500 2023-11-24T07:58:44.962 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:44.955+0000 7f92dffff700 1 --2- 172.21.15.59:0/406214749 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f92c806b110 0x7f92c806d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:44.962 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:44.955+0000 7f92dffff700 1 -- 172.21.15.59:0/406214749 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(60..60 src has 1..60) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f92e00895f0 con 0x7f92e8141500 2023-11-24T07:58:44.963 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:44.955+0000 7f92e77fe700 1 --2- 172.21.15.59:0/406214749 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f92c806b110 0x7f92c806d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:44.963 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:44.955+0000 7f92e77fe700 1 --2- 172.21.15.59:0/406214749 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f92c806b110 0x7f92c806d5d0 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f92e81f76a0 tx=0x7f92d8009210 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:58:44.963 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:44.959+0000 7f92dffff700 1 -- 172.21.15.59:0/406214749 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f92e0054230 con 0x7f92e8141500 2023-11-24T07:58:45.162 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:45.161+0000 7f92ee12a700 1 -- 172.21.15.59:0/406214749 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f92e8068670 con 0x7f92e8141500 2023-11-24T07:58:45.163 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:45.162+0000 7f92dffff700 1 -- 172.21.15.59:0/406214749 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v60) v1 ==== 74+0+16493 (secure 0 0 0) 0x7f92e0057850 con 0x7f92e8141500 2023-11-24T07:58:45.164 INFO:teuthology.orchestra.run.smithi059.stdout: 2023-11-24T07:58:45.164 INFO:teuthology.orchestra.run.smithi059.stdout:{"epoch":60,"fsid":"9f0e8741-b8f9-48d5-a2c3-09f45903303d","created":"2023-11-24T07:56:06.552729+0000","modified":"2023-11-24T07:58:42.328743+0000","last_up_change":"2023-11-24T07:56:17.372352+0000","last_in_change":"2023-11-24T07:56:09.073250+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-24T07:56:18.352823+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_metadata","create_time":"2023-11-24T07:58:39.262463+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"59","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_data","create_time":"2023-11-24T07:58:39.763399+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"60","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"6d8f9e9b-1782-4e41-928b-a39f2a8ebc06","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6800","nonce":3373878772},{"type":"v1","addr":"172.21.15.59:6801","nonce":3373878772}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6802","nonce":3373878772},{"type":"v1","addr":"172.21.15.59:6803","nonce":3373878772}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6806","nonce":3373878772},{"type":"v1","addr":"172.21.15.59:6807","nonce":3373878772}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6804","nonce":3373878772},{"type":"v1","addr":"172.21.15.59:6805","nonce":3373878772}]},"public_addr":"172.21.15.59:6801/3373878772","cluster_addr":"172.21.15.59:6803/3373878772","heartbeat_back_addr":"172.21.15.59:6807/3373878772","heartbeat_front_addr":"172.21.15.59:6805/3373878772","state":["exists","up"]},{"osd":1,"uuid":"e8d882aa-52e1-4c04-9e1e-c44ef9204a5f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6824","nonce":648258563},{"type":"v1","addr":"172.21.15.59:6825","nonce":648258563}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6826","nonce":648258563},{"type":"v1","addr":"172.21.15.59:6827","nonce":648258563}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6830","nonce":648258563},{"type":"v1","addr":"172.21.15.59:6831","nonce":648258563}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6828","nonce":648258563},{"type":"v1","addr":"172.21.15.59:6829","nonce":648258563}]},"public_addr":"172.21.15.59:6825/648258563","cluster_addr":"172.21.15.59:6827/648258563","heartbeat_back_addr":"172.21.15.59:6831/648258563","heartbeat_front_addr":"172.21.15.59:6829/648258563","state":["exists","up"]},{"osd":2,"uuid":"69eebfa0-695b-4b21-90b9-8a3537168a01","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6808","nonce":1840430654},{"type":"v1","addr":"172.21.15.59:6809","nonce":1840430654}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6810","nonce":1840430654},{"type":"v1","addr":"172.21.15.59:6811","nonce":1840430654}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6814","nonce":1840430654},{"type":"v1","addr":"172.21.15.59:6815","nonce":1840430654}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6812","nonce":1840430654},{"type":"v1","addr":"172.21.15.59:6813","nonce":1840430654}]},"public_addr":"172.21.15.59:6809/1840430654","cluster_addr":"172.21.15.59:6811/1840430654","heartbeat_back_addr":"172.21.15.59:6815/1840430654","heartbeat_front_addr":"172.21.15.59:6813/1840430654","state":["exists","up"]},{"osd":3,"uuid":"96fd77fc-f911-46ff-9445-6f0be2daa8f4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6816","nonce":2504025850},{"type":"v1","addr":"172.21.15.59:6817","nonce":2504025850}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6818","nonce":2504025850},{"type":"v1","addr":"172.21.15.59:6819","nonce":2504025850}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6822","nonce":2504025850},{"type":"v1","addr":"172.21.15.59:6823","nonce":2504025850}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6820","nonce":2504025850},{"type":"v1","addr":"172.21.15.59:6821","nonce":2504025850}]},"public_addr":"172.21.15.59:6817/2504025850","cluster_addr":"172.21.15.59:6819/2504025850","heartbeat_back_addr":"172.21.15.59:6823/2504025850","heartbeat_front_addr":"172.21.15.59:6821/2504025850","state":["exists","up"]},{"osd":4,"uuid":"7ac29fca-b5b2-4c95-9d69-2f164cbbc708","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6816","nonce":2382463431},{"type":"v1","addr":"172.21.15.76:6818","nonce":2382463431}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6820","nonce":2382463431},{"type":"v1","addr":"172.21.15.76:6822","nonce":2382463431}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6828","nonce":2382463431},{"type":"v1","addr":"172.21.15.76:6830","nonce":2382463431}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6824","nonce":2382463431},{"type":"v1","addr":"172.21.15.76:6826","nonce":2382463431}]},"public_addr":"172.21.15.76:6818/2382463431","cluster_addr":"172.21.15.76:6822/2382463431","heartbeat_back_addr":"172.21.15.76:6830/2382463431","heartbeat_front_addr":"172.21.15.76:6826/2382463431","state":["exists","up"]},{"osd":5,"uuid":"485bf711-3851-4238-a4f4-51d52e0a9c58","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6800","nonce":329334308},{"type":"v1","addr":"172.21.15.76:6802","nonce":329334308}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6805","nonce":329334308},{"type":"v1","addr":"172.21.15.76:6807","nonce":329334308}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6814","nonce":329334308},{"type":"v1","addr":"172.21.15.76:6815","nonce":329334308}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6810","nonce":329334308},{"type":"v1","addr":"172.21.15.76:6812","nonce":329334308}]},"public_addr":"172.21.15.76:6802/329334308","cluster_addr":"172.21.15.76:6807/329334308","heartbeat_back_addr":"172.21.15.76:6815/329334308","heartbeat_front_addr":"172.21.15.76:6812/329334308","state":["exists","up"]},{"osd":6,"uuid":"72b3f2fc-1e4a-4e32-968d-003647a0286f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6801","nonce":3946579410},{"type":"v1","addr":"172.21.15.76:6803","nonce":3946579410}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6804","nonce":3946579410},{"type":"v1","addr":"172.21.15.76:6806","nonce":3946579410}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6811","nonce":3946579410},{"type":"v1","addr":"172.21.15.76:6813","nonce":3946579410}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6808","nonce":3946579410},{"type":"v1","addr":"172.21.15.76:6809","nonce":3946579410}]},"public_addr":"172.21.15.76:6803/3946579410","cluster_addr":"172.21.15.76:6806/3946579410","heartbeat_back_addr":"172.21.15.76:6813/3946579410","heartbeat_front_addr":"172.21.15.76:6809/3946579410","state":["exists","up"]},{"osd":7,"uuid":"1ddceb90-6c5d-4d1a-ad56-99cb877f8c70","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6817","nonce":841761538},{"type":"v1","addr":"172.21.15.76:6819","nonce":841761538}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6821","nonce":841761538},{"type":"v1","addr":"172.21.15.76:6823","nonce":841761538}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6829","nonce":841761538},{"type":"v1","addr":"172.21.15.76:6831","nonce":841761538}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6825","nonce":841761538},{"type":"v1","addr":"172.21.15.76:6827","nonce":841761538}]},"public_addr":"172.21.15.76:6819/841761538","cluster_addr":"172.21.15.76:6823/841761538","heartbeat_back_addr":"172.21.15.76:6831/841761538","heartbeat_front_addr":"172.21.15.76:6827/841761538","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T07:56:14.294838+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T07:56:14.270779+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T07:56:14.231809+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T07:56:14.260983+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T07:56:14.302997+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T07:56:14.258600+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T07:56:14.251413+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T07:56:14.304727+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-24T07:58:45.166 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:45.164+0000 7f92ee12a700 1 -- 172.21.15.59:0/406214749 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f92c806b110 msgr2=0x7f92c806d5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:45.166 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:45.164+0000 7f92ee12a700 1 --2- 172.21.15.59:0/406214749 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f92c806b110 0x7f92c806d5d0 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f92e81f76a0 tx=0x7f92d8009210 comp rx=0 tx=0).stop 2023-11-24T07:58:45.166 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:45.164+0000 7f92ee12a700 1 -- 172.21.15.59:0/406214749 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f92e8141500 msgr2=0x7f92e813f6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:45.166 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:45.164+0000 7f92ee12a700 1 --2- 172.21.15.59:0/406214749 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f92e8141500 0x7f92e813f6c0 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 crypto rx=0x7f92e8073020 tx=0x7f92e000de60 comp rx=0 tx=0).stop 2023-11-24T07:58:45.166 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:45.164+0000 7f92ee12a700 1 -- 172.21.15.59:0/406214749 shutdown_connections 2023-11-24T07:58:45.167 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:45.164+0000 7f92ee12a700 1 --2- 172.21.15.59:0/406214749 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f92c806b110 0x7f92c806d5d0 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:45.167 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:45.164+0000 7f92ee12a700 1 --2- 172.21.15.59:0/406214749 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f92e8141500 0x7f92e813f6c0 unknown :-1 s=CLOSED pgs=711 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:45.167 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:45.164+0000 7f92ee12a700 1 --2- 172.21.15.59:0/406214749 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f92d4004400 0x7f92e80cfd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:45.167 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:45.164+0000 7f92ee12a700 1 --2- 172.21.15.59:0/406214749 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f92e81410a0 0x7f92e813f180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:45.167 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:45.164+0000 7f92ee12a700 1 -- 172.21.15.59:0/406214749 >> 172.21.15.59:0/406214749 conn(0x7f92e80b2740 msgr2=0x7f92e80b2b20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:58:45.167 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:45.164+0000 7f92ee12a700 1 -- 172.21.15.59:0/406214749 shutdown_connections 2023-11-24T07:58:45.167 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:45.164+0000 7f92ee12a700 1 -- 172.21.15.59:0/406214749 wait complete. 2023-11-24T07:58:45.178 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2023-11-24T07:58:45.179 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-11-24T07:58:45.302 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:45.301+0000 7f9368092700 1 Processor -- start 2023-11-24T07:58:45.303 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:45.301+0000 7f9368092700 1 -- start start 2023-11-24T07:58:45.305 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:45.302+0000 7f9368092700 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f936006afd0 0x7f93600d16e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:45.305 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:45.302+0000 7f9368092700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f93600affb0 0x7f936006b3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:45.305 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:45.302+0000 7f9368092700 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f936015e2e0 0x7f936015e770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:45.305 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:45.302+0000 7f9368092700 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9360070490 con 0x7f93600affb0 2023-11-24T07:58:45.305 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:45.302+0000 7f9368092700 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f93600b0430 con 0x7f936015e2e0 2023-11-24T07:58:45.305 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:45.302+0000 7f9368092700 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f936006b8f0 con 0x7f936006afd0 2023-11-24T07:58:45.306 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:45.302+0000 7f936562d700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f93600affb0 0x7f936006b3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:45.306 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:45.302+0000 7f936562d700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f93600affb0 0x7f936006b3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:53882/0 (socket says 172.21.15.59:53882) 2023-11-24T07:58:45.306 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:45.302+0000 7f936562d700 1 -- 172.21.15.59:0/1195455408 learned_addr learned my addr 172.21.15.59:0/1195455408 (peer_addr_for_me v2:172.21.15.59:0/0) 2023-11-24T07:58:45.306 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:45.302+0000 7f9365e2e700 1 --2- 172.21.15.59:0/1195455408 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f936006afd0 0x7f93600d16e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:45.306 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:45.302+0000 7f936662f700 1 --2- 172.21.15.59:0/1195455408 >> v2:172.21.15.76:3300/0 conn(0x7f936015e2e0 0x7f936015e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:45.306 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:45.302+0000 7f936562d700 1 -- 172.21.15.59:0/1195455408 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f936006afd0 msgr2=0x7f93600d16e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T07:58:45.306 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:45.302+0000 7f936562d700 1 --2- 172.21.15.59:0/1195455408 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f936006afd0 0x7f93600d16e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:45.306 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:45.302+0000 7f936562d700 1 -- 172.21.15.59:0/1195455408 >> v2:172.21.15.76:3300/0 conn(0x7f936015e2e0 msgr2=0x7f936015e770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T07:58:45.306 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:45.302+0000 7f936562d700 1 --2- 172.21.15.59:0/1195455408 >> v2:172.21.15.76:3300/0 conn(0x7f936015e2e0 0x7f936015e770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:45.306 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:45.302+0000 7f936562d700 1 -- 172.21.15.59:0/1195455408 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f93600af9f0 con 0x7f93600affb0 2023-11-24T07:58:45.307 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:45.303+0000 7f936562d700 1 --2- 172.21.15.59:0/1195455408 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f93600affb0 0x7f936006b3b0 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 crypto rx=0x7f9360072f30 tx=0x7f9350009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e9740e895f0051d9 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:58:45.307 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:45.303+0000 7f9364e2c700 1 -- 172.21.15.59:0/1195455408 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9350011830 con 0x7f93600affb0 2023-11-24T07:58:45.307 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:45.303+0000 7f9364e2c700 1 -- 172.21.15.59:0/1195455408 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9350011990 con 0x7f93600affb0 2023-11-24T07:58:45.307 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:45.303+0000 7f9364e2c700 1 -- 172.21.15.59:0/1195455408 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9350011d20 con 0x7f93600affb0 2023-11-24T07:58:45.307 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:45.303+0000 7f9368092700 1 -- 172.21.15.59:0/1195455408 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f93600affb0 msgr2=0x7f936006b3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:45.307 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:45.303+0000 7f9368092700 1 --2- 172.21.15.59:0/1195455408 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f93600affb0 0x7f936006b3b0 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 crypto rx=0x7f9360072f30 tx=0x7f9350009bd0 comp rx=0 tx=0).stop 2023-11-24T07:58:45.307 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:45.303+0000 7f9368092700 1 -- 172.21.15.59:0/1195455408 shutdown_connections 2023-11-24T07:58:45.307 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:45.303+0000 7f9368092700 1 --2- 172.21.15.59:0/1195455408 >> v2:172.21.15.76:3300/0 conn(0x7f936015e2e0 0x7f936015e770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:45.307 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:45.303+0000 7f9368092700 1 --2- 172.21.15.59:0/1195455408 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f936006afd0 0x7f93600d16e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:45.307 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:45.303+0000 7f9368092700 1 --2- 172.21.15.59:0/1195455408 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f93600affb0 0x7f936006b3b0 unknown :-1 s=CLOSED pgs=712 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:45.307 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:45.303+0000 7f9368092700 1 -- 172.21.15.59:0/1195455408 >> 172.21.15.59:0/1195455408 conn(0x7f93600d0bc0 msgr2=0x7f93600d0fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:58:45.307 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:45.303+0000 7f9368092700 1 -- 172.21.15.59:0/1195455408 shutdown_connections 2023-11-24T07:58:45.307 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:45.304+0000 7f9368092700 1 -- 172.21.15.59:0/1195455408 wait complete. 2023-11-24T07:58:45.310 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:45.305+0000 7f9368092700 1 Processor -- start 2023-11-24T07:58:45.310 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:45.305+0000 7f9368092700 1 -- start start 2023-11-24T07:58:45.310 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:45.306+0000 7f9368092700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f936006afd0 0x7f93601f36c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:45.310 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:45.306+0000 7f9368092700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f93600affb0 0x7f936014c020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:45.310 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:45.306+0000 7f9368092700 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f936015e2e0 0x7f9360155fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:45.310 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:45.306+0000 7f9368092700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9360155b50 con 0x7f93600affb0 2023-11-24T07:58:45.310 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:45.306+0000 7f9368092700 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93600d0470 con 0x7f936006afd0 2023-11-24T07:58:45.310 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:45.306+0000 7f9368092700 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f93601f3c00 con 0x7f936015e2e0 2023-11-24T07:58:45.310 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:45.306+0000 7f936562d700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f93600affb0 0x7f936014c020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:45.311 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:45.306+0000 7f936562d700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f93600affb0 0x7f936014c020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:53888/0 (socket says 172.21.15.59:53888) 2023-11-24T07:58:45.311 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:45.306+0000 7f936562d700 1 -- 172.21.15.59:0/4147192515 learned_addr learned my addr 172.21.15.59:0/4147192515 (peer_addr_for_me v2:172.21.15.59:0/0) 2023-11-24T07:58:45.311 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:45.306+0000 7f9365e2e700 1 --2- 172.21.15.59:0/4147192515 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f936006afd0 0x7f93601f36c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:45.311 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:45.306+0000 7f936662f700 1 --2- 172.21.15.59:0/4147192515 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f936015e2e0 0x7f9360155fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:45.311 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:45.306+0000 7f936562d700 1 -- 172.21.15.59:0/4147192515 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f936015e2e0 msgr2=0x7f9360155fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:45.311 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:45.306+0000 7f936562d700 1 --2- 172.21.15.59:0/4147192515 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f936015e2e0 0x7f9360155fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:45.311 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:45.306+0000 7f936562d700 1 -- 172.21.15.59:0/4147192515 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f936006afd0 msgr2=0x7f93601f36c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:45.311 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:45.306+0000 7f936562d700 1 --2- 172.21.15.59:0/4147192515 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f936006afd0 0x7f93601f36c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:45.311 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:45.306+0000 7f936562d700 1 -- 172.21.15.59:0/4147192515 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9350009550 con 0x7f93600affb0 2023-11-24T07:58:45.311 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:45.306+0000 7f936662f700 1 --2- 172.21.15.59:0/4147192515 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f936015e2e0 0x7f9360155fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-24T07:58:45.311 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:45.306+0000 7f936562d700 1 --2- 172.21.15.59:0/4147192515 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f93600affb0 0x7f936014c020 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 crypto rx=0x7f9350000c00 tx=0x7f9350004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:58:45.311 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:45.306+0000 7f93567fc700 1 -- 172.21.15.59:0/4147192515 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9350004240 con 0x7f93600affb0 2023-11-24T07:58:45.312 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:45.306+0000 7f93567fc700 1 -- 172.21.15.59:0/4147192515 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f935001f070 con 0x7f93600affb0 2023-11-24T07:58:45.312 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:45.306+0000 7f9368092700 1 -- 172.21.15.59:0/4147192515 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f93600d0190 con 0x7f93600affb0 2023-11-24T07:58:45.312 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:45.306+0000 7f93567fc700 1 -- 172.21.15.59:0/4147192515 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93500045e0 con 0x7f93600affb0 2023-11-24T07:58:45.312 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:45.306+0000 7f9368092700 1 -- 172.21.15.59:0/4147192515 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f93601557e0 con 0x7f93600affb0 2023-11-24T07:58:45.312 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:45.307+0000 7f9368092700 1 -- 172.21.15.59:0/4147192515 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9360155970 con 0x7f93600affb0 2023-11-24T07:58:45.312 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:45.308+0000 7f93567fc700 1 -- 172.21.15.59:0/4147192515 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 8) v1 ==== 89435+0+0 (secure 0 0 0) 0x7f9350021030 con 0x7f93600affb0 2023-11-24T07:58:45.312 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:45.308+0000 7f93567fc700 1 --2- 172.21.15.59:0/4147192515 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f934406b110 0x7f934406d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:45.312 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:45.308+0000 7f93567fc700 1 -- 172.21.15.59:0/4147192515 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(60..60 src has 1..60) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f935008abc0 con 0x7f93600affb0 2023-11-24T07:58:45.312 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:45.308+0000 7f9365e2e700 1 --2- 172.21.15.59:0/4147192515 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f934406b110 0x7f934406d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:45.313 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:45.309+0000 7f9365e2e700 1 --2- 172.21.15.59:0/4147192515 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f934406b110 0x7f934406d5d0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f935c009b60 tx=0x7f935c009210 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:58:45.314 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:45.313+0000 7f93567fc700 1 -- 172.21.15.59:0/4147192515 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f9350055900 con 0x7f93600affb0 2023-11-24T07:58:45.513 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:45.511+0000 7f9368092700 1 -- 172.21.15.59:0/4147192515 --> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f936006e980 con 0x7f934406b110 2023-11-24T07:58:45.522 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:45.520+0000 7f93567fc700 1 -- 172.21.15.59:0/4147192515 <== mgr.4100 v2:172.21.15.76:6832/4264046536 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+289224 (secure 0 0 0) 0x7f935c0117e0 con 0x7f934406b110 2023-11-24T07:58:45.526 INFO:teuthology.orchestra.run.smithi059.stdout: 2023-11-24T07:58:45.529 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:45.523+0000 7f9368092700 1 -- 172.21.15.59:0/4147192515 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f934406b110 msgr2=0x7f934406d5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:45.529 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:45.523+0000 7f9368092700 1 --2- 172.21.15.59:0/4147192515 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f934406b110 0x7f934406d5d0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f935c009b60 tx=0x7f935c009210 comp rx=0 tx=0).stop 2023-11-24T07:58:45.529 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:45.524+0000 7f9368092700 1 -- 172.21.15.59:0/4147192515 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f93600affb0 msgr2=0x7f936014c020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:45.529 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:45.524+0000 7f9368092700 1 --2- 172.21.15.59:0/4147192515 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f93600affb0 0x7f936014c020 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 crypto rx=0x7f9350000c00 tx=0x7f9350004750 comp rx=0 tx=0).stop 2023-11-24T07:58:45.529 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:45.524+0000 7f9368092700 1 -- 172.21.15.59:0/4147192515 shutdown_connections 2023-11-24T07:58:45.530 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:45.524+0000 7f9368092700 1 --2- 172.21.15.59:0/4147192515 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f934406b110 0x7f934406d5d0 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:45.530 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:45.524+0000 7f9368092700 1 --2- 172.21.15.59:0/4147192515 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f936015e2e0 0x7f9360155fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:45.530 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:45.524+0000 7f9368092700 1 --2- 172.21.15.59:0/4147192515 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f936006afd0 0x7f93601f36c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:45.530 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:45.524+0000 7f9368092700 1 --2- 172.21.15.59:0/4147192515 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f93600affb0 0x7f936014c020 unknown :-1 s=CLOSED pgs=713 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:45.530 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:45.524+0000 7f9368092700 1 -- 172.21.15.59:0/4147192515 >> 172.21.15.59:0/4147192515 conn(0x7f93600d0bc0 msgr2=0x7f936006df60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:58:45.530 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:45.524+0000 7f9368092700 1 -- 172.21.15.59:0/4147192515 shutdown_connections 2023-11-24T07:58:45.530 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:45.524+0000 7f9368092700 1 -- 172.21.15.59:0/4147192515 wait complete. 2023-11-24T07:58:45.531 INFO:teuthology.orchestra.run.smithi059.stderr:dumped all 2023-11-24T07:58:45.538 INFO:teuthology.orchestra.run.smithi059.stdout:{"pg_ready":true,"pg_map":{"version":126,"stamp":"2023-11-24T07:58:44.331762+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":104,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":223008,"kb_used_data":6808,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754751712,"statfs":{"total":773094113280,"available":772865753088,"internally_reserved":0,"allocated":6971392,"data_stored":3586564,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12745,"internal_metadata":221236791},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":30,"apply_latency_ms":30,"commit_latency_ns":30000000,"apply_latency_ns":30000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.961770"},"pg_stats":[{"pgid":"9.3f","version":"0'0","reported_seq":11,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-11-24T07:58:42.333841+0000","last_change":"2023-11-24T07:58:41.343589+0000","last_active":"2023-11-24T07:58:42.333841+0000","last_peered":"2023-11-24T07:58:42.333841+0000","last_clean":"2023-11-24T07:58:42.333841+0000","last_became_active":"2023-11-24T07:58:41.341662+0000","last_became_peered":"2023-11-24T07:58:41.341662+0000","last_unstale":"2023-11-24T07:58:42.333841+0000","last_undegraded":"2023-11-24T07:58:42.333841+0000","last_fullsized":"2023-11-24T07:58:42.333841+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:58:40.368784+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:58:40.368784+0000","last_clean_scrub_stamp":"2023-11-24T07:58:40.368784+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T17:33:41.509928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.3e","version":"0'0","reported_seq":9,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-11-24T07:58:41.357731+0000","last_change":"2023-11-24T07:58:41.352453+0000","last_active":"2023-11-24T07:58:41.357731+0000","last_peered":"2023-11-24T07:58:41.357731+0000","last_clean":"2023-11-24T07:58:41.357731+0000","last_became_active":"2023-11-24T07:58:41.351988+0000","last_became_peered":"2023-11-24T07:58:41.351988+0000","last_unstale":"2023-11-24T07:58:41.357731+0000","last_undegraded":"2023-11-24T07:58:41.357731+0000","last_fullsized":"2023-11-24T07:58:41.357731+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:58:40.368784+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:58:40.368784+0000","last_clean_scrub_stamp":"2023-11-24T07:58:40.368784+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:52:37.376571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.3d","version":"0'0","reported_seq":9,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-11-24T07:58:41.347763+0000","last_change":"2023-11-24T07:58:41.341529+0000","last_active":"2023-11-24T07:58:41.347763+0000","last_peered":"2023-11-24T07:58:41.347763+0000","last_clean":"2023-11-24T07:58:41.347763+0000","last_became_active":"2023-11-24T07:58:41.341183+0000","last_became_peered":"2023-11-24T07:58:41.341183+0000","last_unstale":"2023-11-24T07:58:41.347763+0000","last_undegraded":"2023-11-24T07:58:41.347763+0000","last_fullsized":"2023-11-24T07:58:41.347763+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:58:40.368784+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:58:40.368784+0000","last_clean_scrub_stamp":"2023-11-24T07:58:40.368784+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T18:42:49.190068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.3c","version":"0'0","reported_seq":9,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-11-24T07:58:41.355209+0000","last_change":"2023-11-24T07:58:41.353611+0000","last_active":"2023-11-24T07:58:41.355209+0000","last_peered":"2023-11-24T07:58:41.355209+0000","last_clean":"2023-11-24T07:58:41.355209+0000","last_became_active":"2023-11-24T07:58:41.353250+0000","last_became_peered":"2023-11-24T07:58:41.353250+0000","last_unstale":"2023-11-24T07:58:41.355209+0000","last_undegraded":"2023-11-24T07:58:41.355209+0000","last_fullsized":"2023-11-24T07:58:41.355209+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:58:40.368784+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:58:40.368784+0000","last_clean_scrub_stamp":"2023-11-24T07:58:40.368784+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T19:01:13.479431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.3b","version":"0'0","reported_seq":11,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-11-24T07:58:42.334492+0000","last_change":"2023-11-24T07:58:41.348515+0000","last_active":"2023-11-24T07:58:42.334492+0000","last_peered":"2023-11-24T07:58:42.334492+0000","last_clean":"2023-11-24T07:58:42.334492+0000","last_became_active":"2023-11-24T07:58:41.348217+0000","last_became_peered":"2023-11-24T07:58:41.348217+0000","last_unstale":"2023-11-24T07:58:42.334492+0000","last_undegraded":"2023-11-24T07:58:42.334492+0000","last_fullsized":"2023-11-24T07:58:42.334492+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:58:40.368784+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:58:40.368784+0000","last_clean_scrub_stamp":"2023-11-24T07:58:40.368784+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T15:33:45.482832+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.3a","version":"0'0","reported_seq":11,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-11-24T07:58:42.333397+0000","last_change":"2023-11-24T07:58:41.344696+0000","last_active":"2023-11-24T07:58:42.333397+0000","last_peered":"2023-11-24T07:58:42.333397+0000","last_clean":"2023-11-24T07:58:42.333397+0000","last_became_active":"2023-11-24T07:58:41.344413+0000","last_became_peered":"2023-11-24T07:58:41.344413+0000","last_unstale":"2023-11-24T07:58:42.333397+0000","last_undegraded":"2023-11-24T07:58:42.333397+0000","last_fullsized":"2023-11-24T07:58:42.333397+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:58:40.368784+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:58:40.368784+0000","last_clean_scrub_stamp":"2023-11-24T07:58:40.368784+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T12:06:30.664094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.39","version":"0'0","reported_seq":9,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-11-24T07:58:41.355139+0000","last_change":"2023-11-24T07:58:41.350288+0000","last_active":"2023-11-24T07:58:41.355139+0000","last_peered":"2023-11-24T07:58:41.355139+0000","last_clean":"2023-11-24T07:58:41.355139+0000","last_became_active":"2023-11-24T07:58:41.349842+0000","last_became_peered":"2023-11-24T07:58:41.349842+0000","last_unstale":"2023-11-24T07:58:41.355139+0000","last_undegraded":"2023-11-24T07:58:41.355139+0000","last_fullsized":"2023-11-24T07:58:41.355139+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:58:40.368784+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:58:40.368784+0000","last_clean_scrub_stamp":"2023-11-24T07:58:40.368784+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T13:24:39.680103+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.38","version":"0'0","reported_seq":11,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-11-24T07:58:42.332886+0000","last_change":"2023-11-24T07:58:41.341208+0000","last_active":"2023-11-24T07:58:42.332886+0000","last_peered":"2023-11-24T07:58:42.332886+0000","last_clean":"2023-11-24T07:58:42.332886+0000","last_became_active":"2023-11-24T07:58:41.340907+0000","last_became_peered":"2023-11-24T07:58:41.340907+0000","last_unstale":"2023-11-24T07:58:42.332886+0000","last_undegraded":"2023-11-24T07:58:42.332886+0000","last_fullsized":"2023-11-24T07:58:42.332886+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:58:40.368784+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:58:40.368784+0000","last_clean_scrub_stamp":"2023-11-24T07:58:40.368784+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T12:40:58.256501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.37","version":"0'0","reported_seq":11,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-11-24T07:58:42.332999+0000","last_change":"2023-11-24T07:58:41.342031+0000","last_active":"2023-11-24T07:58:42.332999+0000","last_peered":"2023-11-24T07:58:42.332999+0000","last_clean":"2023-11-24T07:58:42.332999+0000","last_became_active":"2023-11-24T07:58:41.341735+0000","last_became_peered":"2023-11-24T07:58:41.341735+0000","last_unstale":"2023-11-24T07:58:42.332999+0000","last_undegraded":"2023-11-24T07:58:42.332999+0000","last_fullsized":"2023-11-24T07:58:42.332999+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:58:40.368784+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:58:40.368784+0000","last_clean_scrub_stamp":"2023-11-24T07:58:40.368784+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T16:30:56.516447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.36","version":"0'0","reported_seq":11,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-11-24T07:58:42.334176+0000","last_change":"2023-11-24T07:58:41.345198+0000","last_active":"2023-11-24T07:58:42.334176+0000","last_peered":"2023-11-24T07:58:42.334176+0000","last_clean":"2023-11-24T07:58:42.334176+0000","last_became_active":"2023-11-24T07:58:41.344929+0000","last_became_peered":"2023-11-24T07:58:41.344929+0000","last_unstale":"2023-11-24T07:58:42.334176+0000","last_undegraded":"2023-11-24T07:58:42.334176+0000","last_fullsized":"2023-11-24T07:58:42.334176+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:58:40.368784+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:58:40.368784+0000","last_clean_scrub_stamp":"2023-11-24T07:58:40.368784+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T18:35:26.382371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.35","version":"0'0","reported_seq":9,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-11-24T07:58:41.350866+0000","last_change":"2023-11-24T07:58:41.345011+0000","last_active":"2023-11-24T07:58:41.350866+0000","last_peered":"2023-11-24T07:58:41.350866+0000","last_clean":"2023-11-24T07:58:41.350866+0000","last_became_active":"2023-11-24T07:58:41.344754+0000","last_became_peered":"2023-11-24T07:58:41.344754+0000","last_unstale":"2023-11-24T07:58:41.350866+0000","last_undegraded":"2023-11-24T07:58:41.350866+0000","last_fullsized":"2023-11-24T07:58:41.350866+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:58:40.368784+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:58:40.368784+0000","last_clean_scrub_stamp":"2023-11-24T07:58:40.368784+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T17:33:41.509928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.34","version":"0'0","reported_seq":11,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-11-24T07:58:42.332816+0000","last_change":"2023-11-24T07:58:41.345098+0000","last_active":"2023-11-24T07:58:42.332816+0000","last_peered":"2023-11-24T07:58:42.332816+0000","last_clean":"2023-11-24T07:58:42.332816+0000","last_became_active":"2023-11-24T07:58:41.344675+0000","last_became_peered":"2023-11-24T07:58:41.344675+0000","last_unstale":"2023-11-24T07:58:42.332816+0000","last_undegraded":"2023-11-24T07:58:42.332816+0000","last_fullsized":"2023-11-24T07:58:42.332816+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:58:40.368784+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:58:40.368784+0000","last_clean_scrub_stamp":"2023-11-24T07:58:40.368784+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:03:24.960498+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.33","version":"0'0","reported_seq":11,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-11-24T07:58:43.107677+0000","last_change":"2023-11-24T07:58:41.350526+0000","last_active":"2023-11-24T07:58:43.107677+0000","last_peered":"2023-11-24T07:58:43.107677+0000","last_clean":"2023-11-24T07:58:43.107677+0000","last_became_active":"2023-11-24T07:58:41.350256+0000","last_became_peered":"2023-11-24T07:58:41.350256+0000","last_unstale":"2023-11-24T07:58:43.107677+0000","last_undegraded":"2023-11-24T07:58:43.107677+0000","last_fullsized":"2023-11-24T07:58:43.107677+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:58:40.368784+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:58:40.368784+0000","last_clean_scrub_stamp":"2023-11-24T07:58:40.368784+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T17:33:41.509928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.32","version":"0'0","reported_seq":9,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-11-24T07:58:41.349276+0000","last_change":"2023-11-24T07:58:41.344005+0000","last_active":"2023-11-24T07:58:41.349276+0000","last_peered":"2023-11-24T07:58:41.349276+0000","last_clean":"2023-11-24T07:58:41.349276+0000","last_became_active":"2023-11-24T07:58:41.343552+0000","last_became_peered":"2023-11-24T07:58:41.343552+0000","last_unstale":"2023-11-24T07:58:41.349276+0000","last_undegraded":"2023-11-24T07:58:41.349276+0000","last_fullsized":"2023-11-24T07:58:41.349276+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:58:40.368784+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:58:40.368784+0000","last_clean_scrub_stamp":"2023-11-24T07:58:40.368784+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T13:30:24.650752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.31","version":"0'0","reported_seq":11,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-11-24T07:58:43.107638+0000","last_change":"2023-11-24T07:58:41.351580+0000","last_active":"2023-11-24T07:58:43.107638+0000","last_peered":"2023-11-24T07:58:43.107638+0000","last_clean":"2023-11-24T07:58:43.107638+0000","last_became_active":"2023-11-24T07:58:41.351264+0000","last_became_peered":"2023-11-24T07:58:41.351264+0000","last_unstale":"2023-11-24T07:58:43.107638+0000","last_undegraded":"2023-11-24T07:58:43.107638+0000","last_fullsized":"2023-11-24T07:58:43.107638+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:58:40.368784+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:58:40.368784+0000","last_clean_scrub_stamp":"2023-11-24T07:58:40.368784+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T08:25:53.941316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.30","version":"0'0","reported_seq":11,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-11-24T07:58:42.334434+0000","last_change":"2023-11-24T07:58:41.346408+0000","last_active":"2023-11-24T07:58:42.334434+0000","last_peered":"2023-11-24T07:58:42.334434+0000","last_clean":"2023-11-24T07:58:42.334434+0000","last_became_active":"2023-11-24T07:58:41.346120+0000","last_became_peered":"2023-11-24T07:58:41.346120+0000","last_unstale":"2023-11-24T07:58:42.334434+0000","last_undegraded":"2023-11-24T07:58:42.334434+0000","last_fullsized":"2023-11-24T07:58:42.334434+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:58:40.368784+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:58:40.368784+0000","last_clean_scrub_stamp":"2023-11-24T07:58:40.368784+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T14:26:52.323467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.2f","version":"0'0","reported_seq":11,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-11-24T07:58:42.333945+0000","last_change":"2023-11-24T07:58:41.348220+0000","last_active":"2023-11-24T07:58:42.333945+0000","last_peered":"2023-11-24T07:58:42.333945+0000","last_clean":"2023-11-24T07:58:42.333945+0000","last_became_active":"2023-11-24T07:58:41.347912+0000","last_became_peered":"2023-11-24T07:58:41.347912+0000","last_unstale":"2023-11-24T07:58:42.333945+0000","last_undegraded":"2023-11-24T07:58:42.333945+0000","last_fullsized":"2023-11-24T07:58:42.333945+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:58:40.368784+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:58:40.368784+0000","last_clean_scrub_stamp":"2023-11-24T07:58:40.368784+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T15:41:29.374563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.2e","version":"0'0","reported_seq":9,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-11-24T07:58:41.354998+0000","last_change":"2023-11-24T07:58:41.351780+0000","last_active":"2023-11-24T07:58:41.354998+0000","last_peered":"2023-11-24T07:58:41.354998+0000","last_clean":"2023-11-24T07:58:41.354998+0000","last_became_active":"2023-11-24T07:58:41.348144+0000","last_became_peered":"2023-11-24T07:58:41.348144+0000","last_unstale":"2023-11-24T07:58:41.354998+0000","last_undegraded":"2023-11-24T07:58:41.354998+0000","last_fullsized":"2023-11-24T07:58:41.354998+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:58:40.368784+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:58:40.368784+0000","last_clean_scrub_stamp":"2023-11-24T07:58:40.368784+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T18:40:41.517193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.2d","version":"0'0","reported_seq":9,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-11-24T07:58:41.351097+0000","last_change":"2023-11-24T07:58:41.347720+0000","last_active":"2023-11-24T07:58:41.351097+0000","last_peered":"2023-11-24T07:58:41.351097+0000","last_clean":"2023-11-24T07:58:41.351097+0000","last_became_active":"2023-11-24T07:58:41.347163+0000","last_became_peered":"2023-11-24T07:58:41.347163+0000","last_unstale":"2023-11-24T07:58:41.351097+0000","last_undegraded":"2023-11-24T07:58:41.351097+0000","last_fullsized":"2023-11-24T07:58:41.351097+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:58:40.368784+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:58:40.368784+0000","last_clean_scrub_stamp":"2023-11-24T07:58:40.368784+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T15:14:15.802031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.2c","version":"0'0","reported_seq":9,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-11-24T07:58:41.350914+0000","last_change":"2023-11-24T07:58:41.345267+0000","last_active":"2023-11-24T07:58:41.350914+0000","last_peered":"2023-11-24T07:58:41.350914+0000","last_clean":"2023-11-24T07:58:41.350914+0000","last_became_active":"2023-11-24T07:58:41.344947+0000","last_became_peered":"2023-11-24T07:58:41.344947+0000","last_unstale":"2023-11-24T07:58:41.350914+0000","last_undegraded":"2023-11-24T07:58:41.350914+0000","last_fullsized":"2023-11-24T07:58:41.350914+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:58:40.368784+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:58:40.368784+0000","last_clean_scrub_stamp":"2023-11-24T07:58:40.368784+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T15:41:29.374563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.2b","version":"0'0","reported_seq":11,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-11-24T07:58:43.107675+0000","last_change":"2023-11-24T07:58:41.348998+0000","last_active":"2023-11-24T07:58:43.107675+0000","last_peered":"2023-11-24T07:58:43.107675+0000","last_clean":"2023-11-24T07:58:43.107675+0000","last_became_active":"2023-11-24T07:58:41.348413+0000","last_became_peered":"2023-11-24T07:58:41.348413+0000","last_unstale":"2023-11-24T07:58:43.107675+0000","last_undegraded":"2023-11-24T07:58:43.107675+0000","last_fullsized":"2023-11-24T07:58:43.107675+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:58:40.368784+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:58:40.368784+0000","last_clean_scrub_stamp":"2023-11-24T07:58:40.368784+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T15:01:45.344564+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.2a","version":"0'0","reported_seq":9,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-11-24T07:58:41.351691+0000","last_change":"2023-11-24T07:58:41.349683+0000","last_active":"2023-11-24T07:58:41.351691+0000","last_peered":"2023-11-24T07:58:41.351691+0000","last_clean":"2023-11-24T07:58:41.351691+0000","last_became_active":"2023-11-24T07:58:41.346943+0000","last_became_peered":"2023-11-24T07:58:41.346943+0000","last_unstale":"2023-11-24T07:58:41.351691+0000","last_undegraded":"2023-11-24T07:58:41.351691+0000","last_fullsized":"2023-11-24T07:58:41.351691+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:58:40.368784+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:58:40.368784+0000","last_clean_scrub_stamp":"2023-11-24T07:58:40.368784+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T12:06:30.664094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.29","version":"0'0","reported_seq":9,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-11-24T07:58:41.355164+0000","last_change":"2023-11-24T07:58:41.352644+0000","last_active":"2023-11-24T07:58:41.355164+0000","last_peered":"2023-11-24T07:58:41.355164+0000","last_clean":"2023-11-24T07:58:41.355164+0000","last_became_active":"2023-11-24T07:58:41.352206+0000","last_became_peered":"2023-11-24T07:58:41.352206+0000","last_unstale":"2023-11-24T07:58:41.355164+0000","last_undegraded":"2023-11-24T07:58:41.355164+0000","last_fullsized":"2023-11-24T07:58:41.355164+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:58:40.368784+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:58:40.368784+0000","last_clean_scrub_stamp":"2023-11-24T07:58:40.368784+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T17:41:50.674323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.28","version":"0'0","reported_seq":11,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-11-24T07:58:42.332673+0000","last_change":"2023-11-24T07:58:41.346228+0000","last_active":"2023-11-24T07:58:42.332673+0000","last_peered":"2023-11-24T07:58:42.332673+0000","last_clean":"2023-11-24T07:58:42.332673+0000","last_became_active":"2023-11-24T07:58:41.345944+0000","last_became_peered":"2023-11-24T07:58:41.345944+0000","last_unstale":"2023-11-24T07:58:42.332673+0000","last_undegraded":"2023-11-24T07:58:42.332673+0000","last_fullsized":"2023-11-24T07:58:42.332673+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:58:40.368784+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:58:40.368784+0000","last_clean_scrub_stamp":"2023-11-24T07:58:40.368784+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T19:14:45.552557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.27","version":"0'0","reported_seq":9,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-11-24T07:58:41.357294+0000","last_change":"2023-11-24T07:58:41.352347+0000","last_active":"2023-11-24T07:58:41.357294+0000","last_peered":"2023-11-24T07:58:41.357294+0000","last_clean":"2023-11-24T07:58:41.357294+0000","last_became_active":"2023-11-24T07:58:41.352004+0000","last_became_peered":"2023-11-24T07:58:41.352004+0000","last_unstale":"2023-11-24T07:58:41.357294+0000","last_undegraded":"2023-11-24T07:58:41.357294+0000","last_fullsized":"2023-11-24T07:58:41.357294+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:58:40.368784+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:58:40.368784+0000","last_clean_scrub_stamp":"2023-11-24T07:58:40.368784+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T17:01:14.896627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.26","version":"0'0","reported_seq":11,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-11-24T07:58:43.107906+0000","last_change":"2023-11-24T07:58:41.354622+0000","last_active":"2023-11-24T07:58:43.107906+0000","last_peered":"2023-11-24T07:58:43.107906+0000","last_clean":"2023-11-24T07:58:43.107906+0000","last_became_active":"2023-11-24T07:58:41.354430+0000","last_became_peered":"2023-11-24T07:58:41.354430+0000","last_unstale":"2023-11-24T07:58:43.107906+0000","last_undegraded":"2023-11-24T07:58:43.107906+0000","last_fullsized":"2023-11-24T07:58:43.107906+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:58:40.368784+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:58:40.368784+0000","last_clean_scrub_stamp":"2023-11-24T07:58:40.368784+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T08:12:56.009848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.25","version":"0'0","reported_seq":9,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-11-24T07:58:41.351232+0000","last_change":"2023-11-24T07:58:41.348878+0000","last_active":"2023-11-24T07:58:41.351232+0000","last_peered":"2023-11-24T07:58:41.351232+0000","last_clean":"2023-11-24T07:58:41.351232+0000","last_became_active":"2023-11-24T07:58:41.348571+0000","last_became_peered":"2023-11-24T07:58:41.348571+0000","last_unstale":"2023-11-24T07:58:41.351232+0000","last_undegraded":"2023-11-24T07:58:41.351232+0000","last_fullsized":"2023-11-24T07:58:41.351232+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:58:40.368784+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:58:40.368784+0000","last_clean_scrub_stamp":"2023-11-24T07:58:40.368784+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T18:23:28.546956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.24","version":"0'0","reported_seq":11,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-11-24T07:58:42.333994+0000","last_change":"2023-11-24T07:58:41.345942+0000","last_active":"2023-11-24T07:58:42.333994+0000","last_peered":"2023-11-24T07:58:42.333994+0000","last_clean":"2023-11-24T07:58:42.333994+0000","last_became_active":"2023-11-24T07:58:41.345653+0000","last_became_peered":"2023-11-24T07:58:41.345653+0000","last_unstale":"2023-11-24T07:58:42.333994+0000","last_undegraded":"2023-11-24T07:58:42.333994+0000","last_fullsized":"2023-11-24T07:58:42.333994+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:58:40.368784+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:58:40.368784+0000","last_clean_scrub_stamp":"2023-11-24T07:58:40.368784+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T17:41:50.674323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.23","version":"0'0","reported_seq":11,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-11-24T07:58:42.333184+0000","last_change":"2023-11-24T07:58:41.341276+0000","last_active":"2023-11-24T07:58:42.333184+0000","last_peered":"2023-11-24T07:58:42.333184+0000","last_clean":"2023-11-24T07:58:42.333184+0000","last_became_active":"2023-11-24T07:58:41.340920+0000","last_became_peered":"2023-11-24T07:58:41.340920+0000","last_unstale":"2023-11-24T07:58:42.333184+0000","last_undegraded":"2023-11-24T07:58:42.333184+0000","last_fullsized":"2023-11-24T07:58:42.333184+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:58:40.368784+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:58:40.368784+0000","last_clean_scrub_stamp":"2023-11-24T07:58:40.368784+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T12:32:28.462666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.22","version":"0'0","reported_seq":9,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-11-24T07:58:41.355135+0000","last_change":"2023-11-24T07:58:41.350651+0000","last_active":"2023-11-24T07:58:41.355135+0000","last_peered":"2023-11-24T07:58:41.355135+0000","last_clean":"2023-11-24T07:58:41.355135+0000","last_became_active":"2023-11-24T07:58:41.350193+0000","last_became_peered":"2023-11-24T07:58:41.350193+0000","last_unstale":"2023-11-24T07:58:41.355135+0000","last_undegraded":"2023-11-24T07:58:41.355135+0000","last_fullsized":"2023-11-24T07:58:41.355135+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:58:40.368784+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:58:40.368784+0000","last_clean_scrub_stamp":"2023-11-24T07:58:40.368784+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T17:14:37.101791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.21","version":"0'0","reported_seq":9,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-11-24T07:58:41.351166+0000","last_change":"2023-11-24T07:58:41.344155+0000","last_active":"2023-11-24T07:58:41.351166+0000","last_peered":"2023-11-24T07:58:41.351166+0000","last_clean":"2023-11-24T07:58:41.351166+0000","last_became_active":"2023-11-24T07:58:41.342927+0000","last_became_peered":"2023-11-24T07:58:41.342927+0000","last_unstale":"2023-11-24T07:58:41.351166+0000","last_undegraded":"2023-11-24T07:58:41.351166+0000","last_fullsized":"2023-11-24T07:58:41.351166+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:58:40.368784+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:58:40.368784+0000","last_clean_scrub_stamp":"2023-11-24T07:58:40.368784+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T15:43:45.302210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.20","version":"0'0","reported_seq":9,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-11-24T07:58:41.350923+0000","last_change":"2023-11-24T07:58:41.345769+0000","last_active":"2023-11-24T07:58:41.350923+0000","last_peered":"2023-11-24T07:58:41.350923+0000","last_clean":"2023-11-24T07:58:41.350923+0000","last_became_active":"2023-11-24T07:58:41.345449+0000","last_became_peered":"2023-11-24T07:58:41.345449+0000","last_unstale":"2023-11-24T07:58:41.350923+0000","last_undegraded":"2023-11-24T07:58:41.350923+0000","last_fullsized":"2023-11-24T07:58:41.350923+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:58:40.368784+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:58:40.368784+0000","last_clean_scrub_stamp":"2023-11-24T07:58:40.368784+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T08:25:53.941316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.1e","version":"0'0","reported_seq":11,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-11-24T07:58:42.333779+0000","last_change":"2023-11-24T07:58:41.341578+0000","last_active":"2023-11-24T07:58:42.333779+0000","last_peered":"2023-11-24T07:58:42.333779+0000","last_clean":"2023-11-24T07:58:42.333779+0000","last_became_active":"2023-11-24T07:58:41.341307+0000","last_became_peered":"2023-11-24T07:58:41.341307+0000","last_unstale":"2023-11-24T07:58:42.333779+0000","last_undegraded":"2023-11-24T07:58:42.333779+0000","last_fullsized":"2023-11-24T07:58:42.333779+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:58:40.368784+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:58:40.368784+0000","last_clean_scrub_stamp":"2023-11-24T07:58:40.368784+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T08:25:53.941316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.1f","version":"59'4","reported_seq":15,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-11-24T07:58:41.387993+0000","last_change":"2023-11-24T07:58:40.395356+0000","last_active":"2023-11-24T07:58:41.387993+0000","last_peered":"2023-11-24T07:58:41.387993+0000","last_clean":"2023-11-24T07:58:41.387993+0000","last_became_active":"2023-11-24T07:58:40.395026+0000","last_became_peered":"2023-11-24T07:58:40.395026+0000","last_unstale":"2023-11-24T07:58:41.387993+0000","last_undegraded":"2023-11-24T07:58:41.387993+0000","last_fullsized":"2023-11-24T07:58:41.387993+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:58:39.367089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:58:39.367089+0000","last_clean_scrub_stamp":"2023-11-24T07:58:39.367089+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T13:26:32.521022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.1f","version":"0'0","reported_seq":11,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-11-24T07:58:42.334003+0000","last_change":"2023-11-24T07:58:41.344680+0000","last_active":"2023-11-24T07:58:42.334003+0000","last_peered":"2023-11-24T07:58:42.334003+0000","last_clean":"2023-11-24T07:58:42.334003+0000","last_became_active":"2023-11-24T07:58:41.344366+0000","last_became_peered":"2023-11-24T07:58:41.344366+0000","last_unstale":"2023-11-24T07:58:42.334003+0000","last_undegraded":"2023-11-24T07:58:42.334003+0000","last_fullsized":"2023-11-24T07:58:42.334003+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:58:40.368784+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:58:40.368784+0000","last_clean_scrub_stamp":"2023-11-24T07:58:40.368784+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T08:12:56.009848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.1e","version":"59'1","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-11-24T07:58:41.388732+0000","last_change":"2023-11-24T07:58:40.392247+0000","last_active":"2023-11-24T07:58:41.388732+0000","last_peered":"2023-11-24T07:58:41.388732+0000","last_clean":"2023-11-24T07:58:41.388732+0000","last_became_active":"2023-11-24T07:58:40.391972+0000","last_became_peered":"2023-11-24T07:58:40.391972+0000","last_unstale":"2023-11-24T07:58:41.388732+0000","last_undegraded":"2023-11-24T07:58:41.388732+0000","last_fullsized":"2023-11-24T07:58:41.388732+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:58:39.367089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:58:39.367089+0000","last_clean_scrub_stamp":"2023-11-24T07:58:39.367089+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T08:58:58.167821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.1c","version":"0'0","reported_seq":11,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-11-24T07:58:42.333873+0000","last_change":"2023-11-24T07:58:41.342131+0000","last_active":"2023-11-24T07:58:42.333873+0000","last_peered":"2023-11-24T07:58:42.333873+0000","last_clean":"2023-11-24T07:58:42.333873+0000","last_became_active":"2023-11-24T07:58:41.341885+0000","last_became_peered":"2023-11-24T07:58:41.341885+0000","last_unstale":"2023-11-24T07:58:42.333873+0000","last_undegraded":"2023-11-24T07:58:42.333873+0000","last_fullsized":"2023-11-24T07:58:42.333873+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:58:40.368784+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:58:40.368784+0000","last_clean_scrub_stamp":"2023-11-24T07:58:40.368784+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:16:10.890321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.1d","version":"59'2","reported_seq":14,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-11-24T07:58:42.334194+0000","last_change":"2023-11-24T07:58:40.392392+0000","last_active":"2023-11-24T07:58:42.334194+0000","last_peered":"2023-11-24T07:58:42.334194+0000","last_clean":"2023-11-24T07:58:42.334194+0000","last_became_active":"2023-11-24T07:58:40.389605+0000","last_became_peered":"2023-11-24T07:58:40.389605+0000","last_unstale":"2023-11-24T07:58:42.334194+0000","last_undegraded":"2023-11-24T07:58:42.334194+0000","last_fullsized":"2023-11-24T07:58:42.334194+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:58:39.367089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:58:39.367089+0000","last_clean_scrub_stamp":"2023-11-24T07:58:39.367089+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T18:54:45.011060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.1d","version":"0'0","reported_seq":11,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-11-24T07:58:42.334109+0000","last_change":"2023-11-24T07:58:41.344966+0000","last_active":"2023-11-24T07:58:42.334109+0000","last_peered":"2023-11-24T07:58:42.334109+0000","last_clean":"2023-11-24T07:58:42.334109+0000","last_became_active":"2023-11-24T07:58:41.344669+0000","last_became_peered":"2023-11-24T07:58:41.344669+0000","last_unstale":"2023-11-24T07:58:42.334109+0000","last_undegraded":"2023-11-24T07:58:42.334109+0000","last_fullsized":"2023-11-24T07:58:42.334109+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:58:40.368784+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:58:40.368784+0000","last_clean_scrub_stamp":"2023-11-24T07:58:40.368784+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T13:26:33.522716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.1c","version":"59'2","reported_seq":13,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-11-24T07:58:41.386778+0000","last_change":"2023-11-24T07:58:40.390089+0000","last_active":"2023-11-24T07:58:41.386778+0000","last_peered":"2023-11-24T07:58:41.386778+0000","last_clean":"2023-11-24T07:58:41.386778+0000","last_became_active":"2023-11-24T07:58:40.389684+0000","last_became_peered":"2023-11-24T07:58:40.389684+0000","last_unstale":"2023-11-24T07:58:41.386778+0000","last_undegraded":"2023-11-24T07:58:41.386778+0000","last_fullsized":"2023-11-24T07:58:41.386778+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:58:39.367089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:58:39.367089+0000","last_clean_scrub_stamp":"2023-11-24T07:58:39.367089+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T12:01:24.691976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.1a","version":"0'0","reported_seq":11,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-11-24T07:58:42.333085+0000","last_change":"2023-11-24T07:58:41.342876+0000","last_active":"2023-11-24T07:58:42.333085+0000","last_peered":"2023-11-24T07:58:42.333085+0000","last_clean":"2023-11-24T07:58:42.333085+0000","last_became_active":"2023-11-24T07:58:41.342600+0000","last_became_peered":"2023-11-24T07:58:41.342600+0000","last_unstale":"2023-11-24T07:58:42.333085+0000","last_undegraded":"2023-11-24T07:58:42.333085+0000","last_fullsized":"2023-11-24T07:58:42.333085+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:58:40.368784+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:58:40.368784+0000","last_clean_scrub_stamp":"2023-11-24T07:58:40.368784+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T13:42:50.912064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.1b","version":"59'1","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-11-24T07:58:41.388068+0000","last_change":"2023-11-24T07:58:40.392976+0000","last_active":"2023-11-24T07:58:41.388068+0000","last_peered":"2023-11-24T07:58:41.388068+0000","last_clean":"2023-11-24T07:58:41.388068+0000","last_became_active":"2023-11-24T07:58:40.392662+0000","last_became_peered":"2023-11-24T07:58:40.392662+0000","last_unstale":"2023-11-24T07:58:41.388068+0000","last_undegraded":"2023-11-24T07:58:41.388068+0000","last_fullsized":"2023-11-24T07:58:41.388068+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:58:39.367089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:58:39.367089+0000","last_clean_scrub_stamp":"2023-11-24T07:58:39.367089+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T15:01:44.342870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.1b","version":"0'0","reported_seq":9,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-11-24T07:58:41.351639+0000","last_change":"2023-11-24T07:58:41.349639+0000","last_active":"2023-11-24T07:58:41.351639+0000","last_peered":"2023-11-24T07:58:41.351639+0000","last_clean":"2023-11-24T07:58:41.351639+0000","last_became_active":"2023-11-24T07:58:41.349356+0000","last_became_peered":"2023-11-24T07:58:41.349356+0000","last_unstale":"2023-11-24T07:58:41.351639+0000","last_undegraded":"2023-11-24T07:58:41.351639+0000","last_fullsized":"2023-11-24T07:58:41.351639+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:58:40.368784+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:58:40.368784+0000","last_clean_scrub_stamp":"2023-11-24T07:58:40.368784+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T15:01:45.344564+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.1a","version":"59'1","reported_seq":13,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-11-24T07:58:42.334360+0000","last_change":"2023-11-24T07:58:40.389653+0000","last_active":"2023-11-24T07:58:42.334360+0000","last_peered":"2023-11-24T07:58:42.334360+0000","last_clean":"2023-11-24T07:58:42.334360+0000","last_became_active":"2023-11-24T07:58:40.389104+0000","last_became_peered":"2023-11-24T07:58:40.389104+0000","last_unstale":"2023-11-24T07:58:42.334360+0000","last_undegraded":"2023-11-24T07:58:42.334360+0000","last_fullsized":"2023-11-24T07:58:42.334360+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:58:39.367089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:58:39.367089+0000","last_clean_scrub_stamp":"2023-11-24T07:58:39.367089+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:47:06.454357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.18","version":"0'0","reported_seq":9,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-11-24T07:58:41.350794+0000","last_change":"2023-11-24T07:58:41.347456+0000","last_active":"2023-11-24T07:58:41.350794+0000","last_peered":"2023-11-24T07:58:41.350794+0000","last_clean":"2023-11-24T07:58:41.350794+0000","last_became_active":"2023-11-24T07:58:41.347121+0000","last_became_peered":"2023-11-24T07:58:41.347121+0000","last_unstale":"2023-11-24T07:58:41.350794+0000","last_undegraded":"2023-11-24T07:58:41.350794+0000","last_fullsized":"2023-11-24T07:58:41.350794+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:58:40.368784+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:58:40.368784+0000","last_clean_scrub_stamp":"2023-11-24T07:58:40.368784+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T16:35:45.613491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.19","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-11-24T07:58:42.332763+0000","last_change":"2023-11-24T07:58:40.394195+0000","last_active":"2023-11-24T07:58:42.332763+0000","last_peered":"2023-11-24T07:58:42.332763+0000","last_clean":"2023-11-24T07:58:42.332763+0000","last_became_active":"2023-11-24T07:58:40.393911+0000","last_became_peered":"2023-11-24T07:58:40.393911+0000","last_unstale":"2023-11-24T07:58:42.332763+0000","last_undegraded":"2023-11-24T07:58:42.332763+0000","last_fullsized":"2023-11-24T07:58:42.332763+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:58:39.367089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:58:39.367089+0000","last_clean_scrub_stamp":"2023-11-24T07:58:39.367089+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T16:18:32.625460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.19","version":"0'0","reported_seq":11,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-11-24T07:58:42.333894+0000","last_change":"2023-11-24T07:58:41.350584+0000","last_active":"2023-11-24T07:58:42.333894+0000","last_peered":"2023-11-24T07:58:42.333894+0000","last_clean":"2023-11-24T07:58:42.333894+0000","last_became_active":"2023-11-24T07:58:41.350266+0000","last_became_peered":"2023-11-24T07:58:41.350266+0000","last_unstale":"2023-11-24T07:58:42.333894+0000","last_undegraded":"2023-11-24T07:58:42.333894+0000","last_fullsized":"2023-11-24T07:58:42.333894+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:58:40.368784+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:58:40.368784+0000","last_clean_scrub_stamp":"2023-11-24T07:58:40.368784+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T15:01:45.344564+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.18","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-11-24T07:58:43.107392+0000","last_change":"2023-11-24T07:58:40.392080+0000","last_active":"2023-11-24T07:58:43.107392+0000","last_peered":"2023-11-24T07:58:43.107392+0000","last_clean":"2023-11-24T07:58:43.107392+0000","last_became_active":"2023-11-24T07:58:40.391768+0000","last_became_peered":"2023-11-24T07:58:40.391768+0000","last_unstale":"2023-11-24T07:58:43.107392+0000","last_undegraded":"2023-11-24T07:58:43.107392+0000","last_fullsized":"2023-11-24T07:58:43.107392+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:58:39.367089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:58:39.367089+0000","last_clean_scrub_stamp":"2023-11-24T07:58:39.367089+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T11:47:36.508014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.16","version":"0'0","reported_seq":11,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-11-24T07:58:43.107965+0000","last_change":"2023-11-24T07:58:41.344394+0000","last_active":"2023-11-24T07:58:43.107965+0000","last_peered":"2023-11-24T07:58:43.107965+0000","last_clean":"2023-11-24T07:58:43.107965+0000","last_became_active":"2023-11-24T07:58:41.343961+0000","last_became_peered":"2023-11-24T07:58:41.343961+0000","last_unstale":"2023-11-24T07:58:43.107965+0000","last_undegraded":"2023-11-24T07:58:43.107965+0000","last_fullsized":"2023-11-24T07:58:43.107965+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:58:40.368784+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:58:40.368784+0000","last_clean_scrub_stamp":"2023-11-24T07:58:40.368784+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T13:26:33.522716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.17","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-11-24T07:58:42.333107+0000","last_change":"2023-11-24T07:58:40.395150+0000","last_active":"2023-11-24T07:58:42.333107+0000","last_peered":"2023-11-24T07:58:42.333107+0000","last_clean":"2023-11-24T07:58:42.333107+0000","last_became_active":"2023-11-24T07:58:40.394774+0000","last_became_peered":"2023-11-24T07:58:40.394774+0000","last_unstale":"2023-11-24T07:58:42.333107+0000","last_undegraded":"2023-11-24T07:58:42.333107+0000","last_fullsized":"2023-11-24T07:58:42.333107+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:58:39.367089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:58:39.367089+0000","last_clean_scrub_stamp":"2023-11-24T07:58:39.367089+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T18:10:23.790209+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.17","version":"0'0","reported_seq":9,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-11-24T07:58:41.351132+0000","last_change":"2023-11-24T07:58:41.348333+0000","last_active":"2023-11-24T07:58:41.351132+0000","last_peered":"2023-11-24T07:58:41.351132+0000","last_clean":"2023-11-24T07:58:41.351132+0000","last_became_active":"2023-11-24T07:58:41.348007+0000","last_became_peered":"2023-11-24T07:58:41.348007+0000","last_unstale":"2023-11-24T07:58:41.351132+0000","last_undegraded":"2023-11-24T07:58:41.351132+0000","last_fullsized":"2023-11-24T07:58:41.351132+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:58:40.368784+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:58:40.368784+0000","last_clean_scrub_stamp":"2023-11-24T07:58:40.368784+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T08:25:53.941316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.16","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-11-24T07:58:42.333172+0000","last_change":"2023-11-24T07:58:40.390657+0000","last_active":"2023-11-24T07:58:42.333172+0000","last_peered":"2023-11-24T07:58:42.333172+0000","last_clean":"2023-11-24T07:58:42.333172+0000","last_became_active":"2023-11-24T07:58:40.390017+0000","last_became_peered":"2023-11-24T07:58:40.390017+0000","last_unstale":"2023-11-24T07:58:42.333172+0000","last_undegraded":"2023-11-24T07:58:42.333172+0000","last_fullsized":"2023-11-24T07:58:42.333172+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:58:39.367089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:58:39.367089+0000","last_clean_scrub_stamp":"2023-11-24T07:58:39.367089+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T08:20:14.329086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.14","version":"0'0","reported_seq":11,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-11-24T07:58:42.334015+0000","last_change":"2023-11-24T07:58:41.342321+0000","last_active":"2023-11-24T07:58:42.334015+0000","last_peered":"2023-11-24T07:58:42.334015+0000","last_clean":"2023-11-24T07:58:42.334015+0000","last_became_active":"2023-11-24T07:58:41.341630+0000","last_became_peered":"2023-11-24T07:58:41.341630+0000","last_unstale":"2023-11-24T07:58:42.334015+0000","last_undegraded":"2023-11-24T07:58:42.334015+0000","last_fullsized":"2023-11-24T07:58:42.334015+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:58:40.368784+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:58:40.368784+0000","last_clean_scrub_stamp":"2023-11-24T07:58:40.368784+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T16:36:14.620555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.15","version":"59'3","reported_seq":14,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-11-24T07:58:41.387205+0000","last_change":"2023-11-24T07:58:40.389211+0000","last_active":"2023-11-24T07:58:41.387205+0000","last_peered":"2023-11-24T07:58:41.387205+0000","last_clean":"2023-11-24T07:58:41.387205+0000","last_became_active":"2023-11-24T07:58:40.388804+0000","last_became_peered":"2023-11-24T07:58:40.388804+0000","last_unstale":"2023-11-24T07:58:41.387205+0000","last_undegraded":"2023-11-24T07:58:41.387205+0000","last_fullsized":"2023-11-24T07:58:41.387205+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:58:39.367089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:58:39.367089+0000","last_clean_scrub_stamp":"2023-11-24T07:58:39.367089+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T17:15:19.973003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.15","version":"0'0","reported_seq":9,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-11-24T07:58:41.355212+0000","last_change":"2023-11-24T07:58:41.353703+0000","last_active":"2023-11-24T07:58:41.355212+0000","last_peered":"2023-11-24T07:58:41.355212+0000","last_clean":"2023-11-24T07:58:41.355212+0000","last_became_active":"2023-11-24T07:58:41.353373+0000","last_became_peered":"2023-11-24T07:58:41.353373+0000","last_unstale":"2023-11-24T07:58:41.355212+0000","last_undegraded":"2023-11-24T07:58:41.355212+0000","last_fullsized":"2023-11-24T07:58:41.355212+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:58:40.368784+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:58:40.368784+0000","last_clean_scrub_stamp":"2023-11-24T07:58:40.368784+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:16:10.890321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.14","version":"59'6","reported_seq":18,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-11-24T07:58:43.107924+0000","last_change":"2023-11-24T07:58:40.389832+0000","last_active":"2023-11-24T07:58:43.107924+0000","last_peered":"2023-11-24T07:58:43.107924+0000","last_clean":"2023-11-24T07:58:43.107924+0000","last_became_active":"2023-11-24T07:58:40.389552+0000","last_became_peered":"2023-11-24T07:58:40.389552+0000","last_unstale":"2023-11-24T07:58:43.107924+0000","last_undegraded":"2023-11-24T07:58:43.107924+0000","last_fullsized":"2023-11-24T07:58:43.107924+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:58:39.367089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:58:39.367089+0000","last_clean_scrub_stamp":"2023-11-24T07:58:39.367089+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T13:30:23.649058+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.12","version":"0'0","reported_seq":11,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-11-24T07:58:42.332841+0000","last_change":"2023-11-24T07:58:41.347540+0000","last_active":"2023-11-24T07:58:42.332841+0000","last_peered":"2023-11-24T07:58:42.332841+0000","last_clean":"2023-11-24T07:58:42.332841+0000","last_became_active":"2023-11-24T07:58:41.347259+0000","last_became_peered":"2023-11-24T07:58:41.347259+0000","last_unstale":"2023-11-24T07:58:42.332841+0000","last_undegraded":"2023-11-24T07:58:42.332841+0000","last_fullsized":"2023-11-24T07:58:42.332841+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:58:40.368784+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:58:40.368784+0000","last_clean_scrub_stamp":"2023-11-24T07:58:40.368784+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T08:49:55.010788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.13","version":"59'2","reported_seq":14,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-11-24T07:58:42.333293+0000","last_change":"2023-11-24T07:58:40.390370+0000","last_active":"2023-11-24T07:58:42.333293+0000","last_peered":"2023-11-24T07:58:42.333293+0000","last_clean":"2023-11-24T07:58:42.333293+0000","last_became_active":"2023-11-24T07:58:40.390079+0000","last_became_peered":"2023-11-24T07:58:40.390079+0000","last_unstale":"2023-11-24T07:58:42.333293+0000","last_undegraded":"2023-11-24T07:58:42.333293+0000","last_fullsized":"2023-11-24T07:58:42.333293+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:58:39.367089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:58:39.367089+0000","last_clean_scrub_stamp":"2023-11-24T07:58:39.367089+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:54:22.823103+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.13","version":"0'0","reported_seq":9,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-11-24T07:58:41.347679+0000","last_change":"2023-11-24T07:58:41.342023+0000","last_active":"2023-11-24T07:58:41.347679+0000","last_peered":"2023-11-24T07:58:41.347679+0000","last_clean":"2023-11-24T07:58:41.347679+0000","last_became_active":"2023-11-24T07:58:41.341485+0000","last_became_peered":"2023-11-24T07:58:41.341485+0000","last_unstale":"2023-11-24T07:58:41.347679+0000","last_undegraded":"2023-11-24T07:58:41.347679+0000","last_fullsized":"2023-11-24T07:58:41.347679+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:58:40.368784+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:58:40.368784+0000","last_clean_scrub_stamp":"2023-11-24T07:58:40.368784+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:47:07.456051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.12","version":"0'0","reported_seq":11,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-11-24T07:58:41.346961+0000","last_change":"2023-11-24T07:58:40.391105+0000","last_active":"2023-11-24T07:58:41.346961+0000","last_peered":"2023-11-24T07:58:41.346961+0000","last_clean":"2023-11-24T07:58:41.346961+0000","last_became_active":"2023-11-24T07:58:40.390811+0000","last_became_peered":"2023-11-24T07:58:40.390811+0000","last_unstale":"2023-11-24T07:58:41.346961+0000","last_undegraded":"2023-11-24T07:58:41.346961+0000","last_fullsized":"2023-11-24T07:58:41.346961+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:58:39.367089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:58:39.367089+0000","last_clean_scrub_stamp":"2023-11-24T07:58:39.367089+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T14:07:19.428853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.10","version":"0'0","reported_seq":11,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-11-24T07:58:42.332858+0000","last_change":"2023-11-24T07:58:41.344911+0000","last_active":"2023-11-24T07:58:42.332858+0000","last_peered":"2023-11-24T07:58:42.332858+0000","last_clean":"2023-11-24T07:58:42.332858+0000","last_became_active":"2023-11-24T07:58:41.344579+0000","last_became_peered":"2023-11-24T07:58:41.344579+0000","last_unstale":"2023-11-24T07:58:42.332858+0000","last_undegraded":"2023-11-24T07:58:42.332858+0000","last_fullsized":"2023-11-24T07:58:42.332858+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:58:40.368784+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:58:40.368784+0000","last_clean_scrub_stamp":"2023-11-24T07:58:40.368784+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T13:18:10.742140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.11","version":"0'0","reported_seq":11,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-11-24T07:58:41.345898+0000","last_change":"2023-11-24T07:58:40.396157+0000","last_active":"2023-11-24T07:58:41.345898+0000","last_peered":"2023-11-24T07:58:41.345898+0000","last_clean":"2023-11-24T07:58:41.345898+0000","last_became_active":"2023-11-24T07:58:40.395857+0000","last_became_peered":"2023-11-24T07:58:40.395857+0000","last_unstale":"2023-11-24T07:58:41.345898+0000","last_undegraded":"2023-11-24T07:58:41.345898+0000","last_fullsized":"2023-11-24T07:58:41.345898+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:58:39.367089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:58:39.367089+0000","last_clean_scrub_stamp":"2023-11-24T07:58:39.367089+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T16:50:29.259946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.11","version":"0'0","reported_seq":9,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-11-24T07:58:41.355194+0000","last_change":"2023-11-24T07:58:41.352778+0000","last_active":"2023-11-24T07:58:41.355194+0000","last_peered":"2023-11-24T07:58:41.355194+0000","last_clean":"2023-11-24T07:58:41.355194+0000","last_became_active":"2023-11-24T07:58:41.352465+0000","last_became_peered":"2023-11-24T07:58:41.352465+0000","last_unstale":"2023-11-24T07:58:41.355194+0000","last_undegraded":"2023-11-24T07:58:41.355194+0000","last_fullsized":"2023-11-24T07:58:41.355194+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:58:40.368784+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:58:40.368784+0000","last_clean_scrub_stamp":"2023-11-24T07:58:40.368784+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T16:36:14.620555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.10","version":"59'8","reported_seq":19,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-11-24T07:58:41.390083+0000","last_change":"2023-11-24T07:58:40.390948+0000","last_active":"2023-11-24T07:58:41.390083+0000","last_peered":"2023-11-24T07:58:41.390083+0000","last_clean":"2023-11-24T07:58:41.390083+0000","last_became_active":"2023-11-24T07:58:40.390768+0000","last_became_peered":"2023-11-24T07:58:40.390768+0000","last_unstale":"2023-11-24T07:58:41.390083+0000","last_undegraded":"2023-11-24T07:58:41.390083+0000","last_fullsized":"2023-11-24T07:58:41.390083+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:58:39.367089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:58:39.367089+0000","last_clean_scrub_stamp":"2023-11-24T07:58:39.367089+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:47:08.018319+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.e","version":"0'0","reported_seq":9,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-11-24T07:58:41.351873+0000","last_change":"2023-11-24T07:58:41.348636+0000","last_active":"2023-11-24T07:58:41.351873+0000","last_peered":"2023-11-24T07:58:41.351873+0000","last_clean":"2023-11-24T07:58:41.351873+0000","last_became_active":"2023-11-24T07:58:41.347768+0000","last_became_peered":"2023-11-24T07:58:41.347768+0000","last_unstale":"2023-11-24T07:58:41.351873+0000","last_undegraded":"2023-11-24T07:58:41.351873+0000","last_fullsized":"2023-11-24T07:58:41.351873+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:58:40.368784+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:58:40.368784+0000","last_clean_scrub_stamp":"2023-11-24T07:58:40.368784+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T15:36:54.443304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.f","version":"59'1","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-11-24T07:58:41.387708+0000","last_change":"2023-11-24T07:58:40.394617+0000","last_active":"2023-11-24T07:58:41.387708+0000","last_peered":"2023-11-24T07:58:41.387708+0000","last_clean":"2023-11-24T07:58:41.387708+0000","last_became_active":"2023-11-24T07:58:40.394336+0000","last_became_peered":"2023-11-24T07:58:40.394336+0000","last_unstale":"2023-11-24T07:58:41.387708+0000","last_undegraded":"2023-11-24T07:58:41.387708+0000","last_fullsized":"2023-11-24T07:58:41.387708+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:58:39.367089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:58:39.367089+0000","last_clean_scrub_stamp":"2023-11-24T07:58:39.367089+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T16:18:32.625460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.f","version":"0'0","reported_seq":9,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-11-24T07:58:41.351114+0000","last_change":"2023-11-24T07:58:41.347577+0000","last_active":"2023-11-24T07:58:41.351114+0000","last_peered":"2023-11-24T07:58:41.351114+0000","last_clean":"2023-11-24T07:58:41.351114+0000","last_became_active":"2023-11-24T07:58:41.347105+0000","last_became_peered":"2023-11-24T07:58:41.347105+0000","last_unstale":"2023-11-24T07:58:41.351114+0000","last_undegraded":"2023-11-24T07:58:41.351114+0000","last_fullsized":"2023-11-24T07:58:41.351114+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:58:40.368784+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:58:40.368784+0000","last_clean_scrub_stamp":"2023-11-24T07:58:40.368784+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T12:06:30.664094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.e","version":"59'2","reported_seq":14,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-11-24T07:58:42.333277+0000","last_change":"2023-11-24T07:58:40.390825+0000","last_active":"2023-11-24T07:58:42.333277+0000","last_peered":"2023-11-24T07:58:42.333277+0000","last_clean":"2023-11-24T07:58:42.333277+0000","last_became_active":"2023-11-24T07:58:40.390392+0000","last_became_peered":"2023-11-24T07:58:40.390392+0000","last_unstale":"2023-11-24T07:58:42.333277+0000","last_undegraded":"2023-11-24T07:58:42.333277+0000","last_fullsized":"2023-11-24T07:58:42.333277+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:58:39.367089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:58:39.367089+0000","last_clean_scrub_stamp":"2023-11-24T07:58:39.367089+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T17:11:13.395000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.c","version":"0'0","reported_seq":9,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-11-24T07:58:41.351301+0000","last_change":"2023-11-24T07:58:41.346777+0000","last_active":"2023-11-24T07:58:41.351301+0000","last_peered":"2023-11-24T07:58:41.351301+0000","last_clean":"2023-11-24T07:58:41.351301+0000","last_became_active":"2023-11-24T07:58:41.346435+0000","last_became_peered":"2023-11-24T07:58:41.346435+0000","last_unstale":"2023-11-24T07:58:41.351301+0000","last_undegraded":"2023-11-24T07:58:41.351301+0000","last_fullsized":"2023-11-24T07:58:41.351301+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:58:40.368784+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:58:40.368784+0000","last_clean_scrub_stamp":"2023-11-24T07:58:40.368784+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:53:55.049234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.d","version":"59'1","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-11-24T07:58:41.387979+0000","last_change":"2023-11-24T07:58:40.395492+0000","last_active":"2023-11-24T07:58:41.387979+0000","last_peered":"2023-11-24T07:58:41.387979+0000","last_clean":"2023-11-24T07:58:41.387979+0000","last_became_active":"2023-11-24T07:58:40.395197+0000","last_became_peered":"2023-11-24T07:58:40.395197+0000","last_unstale":"2023-11-24T07:58:41.387979+0000","last_undegraded":"2023-11-24T07:58:41.387979+0000","last_fullsized":"2023-11-24T07:58:41.387979+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:58:39.367089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:58:39.367089+0000","last_clean_scrub_stamp":"2023-11-24T07:58:39.367089+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T15:41:28.372869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.d","version":"0'0","reported_seq":9,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-11-24T07:58:41.355191+0000","last_change":"2023-11-24T07:58:41.353502+0000","last_active":"2023-11-24T07:58:41.355191+0000","last_peered":"2023-11-24T07:58:41.355191+0000","last_clean":"2023-11-24T07:58:41.355191+0000","last_became_active":"2023-11-24T07:58:41.353177+0000","last_became_peered":"2023-11-24T07:58:41.353177+0000","last_unstale":"2023-11-24T07:58:41.355191+0000","last_undegraded":"2023-11-24T07:58:41.355191+0000","last_fullsized":"2023-11-24T07:58:41.355191+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:58:40.368784+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:58:40.368784+0000","last_clean_scrub_stamp":"2023-11-24T07:58:40.368784+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T17:19:15.866101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.c","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-11-24T07:58:42.334121+0000","last_change":"2023-11-24T07:58:40.385207+0000","last_active":"2023-11-24T07:58:42.334121+0000","last_peered":"2023-11-24T07:58:42.334121+0000","last_clean":"2023-11-24T07:58:42.334121+0000","last_became_active":"2023-11-24T07:58:40.384936+0000","last_became_peered":"2023-11-24T07:58:40.384936+0000","last_unstale":"2023-11-24T07:58:42.334121+0000","last_undegraded":"2023-11-24T07:58:42.334121+0000","last_fullsized":"2023-11-24T07:58:42.334121+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:58:39.367089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:58:39.367089+0000","last_clean_scrub_stamp":"2023-11-24T07:58:39.367089+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T13:48:18.639688+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.a","version":"0'0","reported_seq":11,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-11-24T07:58:42.332873+0000","last_change":"2023-11-24T07:58:41.352815+0000","last_active":"2023-11-24T07:58:42.332873+0000","last_peered":"2023-11-24T07:58:42.332873+0000","last_clean":"2023-11-24T07:58:42.332873+0000","last_became_active":"2023-11-24T07:58:41.352531+0000","last_became_peered":"2023-11-24T07:58:41.352531+0000","last_unstale":"2023-11-24T07:58:42.332873+0000","last_undegraded":"2023-11-24T07:58:42.332873+0000","last_fullsized":"2023-11-24T07:58:42.332873+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:58:40.368784+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:58:40.368784+0000","last_clean_scrub_stamp":"2023-11-24T07:58:40.368784+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T12:38:47.311808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.b","version":"59'2","reported_seq":13,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-11-24T07:58:41.388853+0000","last_change":"2023-11-24T07:58:40.392826+0000","last_active":"2023-11-24T07:58:41.388853+0000","last_peered":"2023-11-24T07:58:41.388853+0000","last_clean":"2023-11-24T07:58:41.388853+0000","last_became_active":"2023-11-24T07:58:40.392575+0000","last_became_peered":"2023-11-24T07:58:40.392575+0000","last_unstale":"2023-11-24T07:58:41.388853+0000","last_undegraded":"2023-11-24T07:58:41.388853+0000","last_fullsized":"2023-11-24T07:58:41.388853+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:58:39.367089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:58:39.367089+0000","last_clean_scrub_stamp":"2023-11-24T07:58:39.367089+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T08:12:55.008154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.b","version":"0'0","reported_seq":11,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-11-24T07:58:42.333231+0000","last_change":"2023-11-24T07:58:41.347920+0000","last_active":"2023-11-24T07:58:42.333231+0000","last_peered":"2023-11-24T07:58:42.333231+0000","last_clean":"2023-11-24T07:58:42.333231+0000","last_became_active":"2023-11-24T07:58:41.346222+0000","last_became_peered":"2023-11-24T07:58:41.346222+0000","last_unstale":"2023-11-24T07:58:42.333231+0000","last_undegraded":"2023-11-24T07:58:42.333231+0000","last_fullsized":"2023-11-24T07:58:42.333231+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:58:40.368784+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:58:40.368784+0000","last_clean_scrub_stamp":"2023-11-24T07:58:40.368784+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T08:19:12.331002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.a","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-11-24T07:58:42.333574+0000","last_change":"2023-11-24T07:58:40.390723+0000","last_active":"2023-11-24T07:58:42.333574+0000","last_peered":"2023-11-24T07:58:42.333574+0000","last_clean":"2023-11-24T07:58:42.333574+0000","last_became_active":"2023-11-24T07:58:40.390439+0000","last_became_peered":"2023-11-24T07:58:40.390439+0000","last_unstale":"2023-11-24T07:58:42.333574+0000","last_undegraded":"2023-11-24T07:58:42.333574+0000","last_fullsized":"2023-11-24T07:58:42.333574+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:58:39.367089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:58:39.367089+0000","last_clean_scrub_stamp":"2023-11-24T07:58:39.367089+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:02:06.534520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.9","version":"0'0","reported_seq":11,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-11-24T07:58:42.333267+0000","last_change":"2023-11-24T07:58:41.344399+0000","last_active":"2023-11-24T07:58:42.333267+0000","last_peered":"2023-11-24T07:58:42.333267+0000","last_clean":"2023-11-24T07:58:42.333267+0000","last_became_active":"2023-11-24T07:58:41.344105+0000","last_became_peered":"2023-11-24T07:58:41.344105+0000","last_unstale":"2023-11-24T07:58:42.333267+0000","last_undegraded":"2023-11-24T07:58:42.333267+0000","last_fullsized":"2023-11-24T07:58:42.333267+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:58:40.368784+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:58:40.368784+0000","last_clean_scrub_stamp":"2023-11-24T07:58:40.368784+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T18:08:20.595010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.8","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-11-24T07:58:42.332842+0000","last_change":"2023-11-24T07:58:40.398784+0000","last_active":"2023-11-24T07:58:42.332842+0000","last_peered":"2023-11-24T07:58:42.332842+0000","last_clean":"2023-11-24T07:58:42.332842+0000","last_became_active":"2023-11-24T07:58:40.398558+0000","last_became_peered":"2023-11-24T07:58:40.398558+0000","last_unstale":"2023-11-24T07:58:42.332842+0000","last_undegraded":"2023-11-24T07:58:42.332842+0000","last_fullsized":"2023-11-24T07:58:42.332842+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:58:39.367089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:58:39.367089+0000","last_clean_scrub_stamp":"2023-11-24T07:58:39.367089+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T08:58:58.167821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.6","version":"0'0","reported_seq":9,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-11-24T07:58:41.351863+0000","last_change":"2023-11-24T07:58:41.348458+0000","last_active":"2023-11-24T07:58:41.351863+0000","last_peered":"2023-11-24T07:58:41.351863+0000","last_clean":"2023-11-24T07:58:41.351863+0000","last_became_active":"2023-11-24T07:58:41.347771+0000","last_became_peered":"2023-11-24T07:58:41.347771+0000","last_unstale":"2023-11-24T07:58:41.351863+0000","last_undegraded":"2023-11-24T07:58:41.351863+0000","last_fullsized":"2023-11-24T07:58:41.351863+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:58:40.368784+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:58:40.368784+0000","last_clean_scrub_stamp":"2023-11-24T07:58:40.368784+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T12:24:10.051730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.7","version":"59'2","reported_seq":13,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-11-24T07:58:41.387625+0000","last_change":"2023-11-24T07:58:40.391619+0000","last_active":"2023-11-24T07:58:41.387625+0000","last_peered":"2023-11-24T07:58:41.387625+0000","last_clean":"2023-11-24T07:58:41.387625+0000","last_became_active":"2023-11-24T07:58:40.391378+0000","last_became_peered":"2023-11-24T07:58:40.391378+0000","last_unstale":"2023-11-24T07:58:41.387625+0000","last_undegraded":"2023-11-24T07:58:41.387625+0000","last_fullsized":"2023-11-24T07:58:41.387625+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:58:39.367089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:58:39.367089+0000","last_clean_scrub_stamp":"2023-11-24T07:58:39.367089+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T16:56:58.467571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.7","version":"0'0","reported_seq":9,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-11-24T07:58:41.351148+0000","last_change":"2023-11-24T07:58:41.348843+0000","last_active":"2023-11-24T07:58:41.351148+0000","last_peered":"2023-11-24T07:58:41.351148+0000","last_clean":"2023-11-24T07:58:41.351148+0000","last_became_active":"2023-11-24T07:58:41.348229+0000","last_became_peered":"2023-11-24T07:58:41.348229+0000","last_unstale":"2023-11-24T07:58:41.351148+0000","last_undegraded":"2023-11-24T07:58:41.351148+0000","last_fullsized":"2023-11-24T07:58:41.351148+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:58:40.368784+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:58:40.368784+0000","last_clean_scrub_stamp":"2023-11-24T07:58:40.368784+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T12:40:49.007909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.6","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-11-24T07:58:42.334272+0000","last_change":"2023-11-24T07:58:40.389732+0000","last_active":"2023-11-24T07:58:42.334272+0000","last_peered":"2023-11-24T07:58:42.334272+0000","last_clean":"2023-11-24T07:58:42.334272+0000","last_became_active":"2023-11-24T07:58:40.389441+0000","last_became_peered":"2023-11-24T07:58:40.389441+0000","last_unstale":"2023-11-24T07:58:42.334272+0000","last_undegraded":"2023-11-24T07:58:42.334272+0000","last_fullsized":"2023-11-24T07:58:42.334272+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:58:39.367089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:58:39.367089+0000","last_clean_scrub_stamp":"2023-11-24T07:58:39.367089+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T12:24:09.050036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.4","version":"0'0","reported_seq":9,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-11-24T07:58:41.351321+0000","last_change":"2023-11-24T07:58:41.347399+0000","last_active":"2023-11-24T07:58:41.351321+0000","last_peered":"2023-11-24T07:58:41.351321+0000","last_clean":"2023-11-24T07:58:41.351321+0000","last_became_active":"2023-11-24T07:58:41.347054+0000","last_became_peered":"2023-11-24T07:58:41.347054+0000","last_unstale":"2023-11-24T07:58:41.351321+0000","last_undegraded":"2023-11-24T07:58:41.351321+0000","last_fullsized":"2023-11-24T07:58:41.351321+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:58:40.368784+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:58:40.368784+0000","last_clean_scrub_stamp":"2023-11-24T07:58:40.368784+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:37:54.868724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.5","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-11-24T07:58:42.333160+0000","last_change":"2023-11-24T07:58:40.398278+0000","last_active":"2023-11-24T07:58:42.333160+0000","last_peered":"2023-11-24T07:58:42.333160+0000","last_clean":"2023-11-24T07:58:42.333160+0000","last_became_active":"2023-11-24T07:58:40.397929+0000","last_became_peered":"2023-11-24T07:58:40.397929+0000","last_unstale":"2023-11-24T07:58:42.333160+0000","last_undegraded":"2023-11-24T07:58:42.333160+0000","last_fullsized":"2023-11-24T07:58:42.333160+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:58:39.367089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:58:39.367089+0000","last_clean_scrub_stamp":"2023-11-24T07:58:39.367089+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T16:56:58.467571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.5","version":"0'0","reported_seq":11,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-11-24T07:58:42.333485+0000","last_change":"2023-11-24T07:58:41.347031+0000","last_active":"2023-11-24T07:58:42.333485+0000","last_peered":"2023-11-24T07:58:42.333485+0000","last_clean":"2023-11-24T07:58:42.333485+0000","last_became_active":"2023-11-24T07:58:41.345570+0000","last_became_peered":"2023-11-24T07:58:41.345570+0000","last_unstale":"2023-11-24T07:58:42.333485+0000","last_undegraded":"2023-11-24T07:58:42.333485+0000","last_fullsized":"2023-11-24T07:58:42.333485+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:58:40.368784+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:58:40.368784+0000","last_clean_scrub_stamp":"2023-11-24T07:58:40.368784+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T19:01:13.479431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.4","version":"0'0","reported_seq":11,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-11-24T07:58:41.337625+0000","last_change":"2023-11-24T07:58:40.393928+0000","last_active":"2023-11-24T07:58:41.337625+0000","last_peered":"2023-11-24T07:58:41.337625+0000","last_clean":"2023-11-24T07:58:41.337625+0000","last_became_active":"2023-11-24T07:58:40.393493+0000","last_became_peered":"2023-11-24T07:58:40.393493+0000","last_unstale":"2023-11-24T07:58:41.337625+0000","last_undegraded":"2023-11-24T07:58:41.337625+0000","last_fullsized":"2023-11-24T07:58:41.337625+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:58:39.367089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:58:39.367089+0000","last_clean_scrub_stamp":"2023-11-24T07:58:39.367089+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T08:12:55.008154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.2","version":"0'0","reported_seq":11,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-11-24T07:58:42.334240+0000","last_change":"2023-11-24T07:58:41.350597+0000","last_active":"2023-11-24T07:58:42.334240+0000","last_peered":"2023-11-24T07:58:42.334240+0000","last_clean":"2023-11-24T07:58:42.334240+0000","last_became_active":"2023-11-24T07:58:41.350298+0000","last_became_peered":"2023-11-24T07:58:41.350298+0000","last_unstale":"2023-11-24T07:58:42.334240+0000","last_undegraded":"2023-11-24T07:58:42.334240+0000","last_fullsized":"2023-11-24T07:58:42.334240+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:58:40.368784+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:58:40.368784+0000","last_clean_scrub_stamp":"2023-11-24T07:58:40.368784+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T17:15:30.537711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.3","version":"0'0","reported_seq":11,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-11-24T07:58:41.337028+0000","last_change":"2023-11-24T07:58:40.390913+0000","last_active":"2023-11-24T07:58:41.337028+0000","last_peered":"2023-11-24T07:58:41.337028+0000","last_clean":"2023-11-24T07:58:41.337028+0000","last_became_active":"2023-11-24T07:58:40.390543+0000","last_became_peered":"2023-11-24T07:58:40.390543+0000","last_unstale":"2023-11-24T07:58:41.337028+0000","last_undegraded":"2023-11-24T07:58:41.337028+0000","last_fullsized":"2023-11-24T07:58:41.337028+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:58:39.367089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:58:39.367089+0000","last_clean_scrub_stamp":"2023-11-24T07:58:39.367089+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T19:40:59.979474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.3","version":"0'0","reported_seq":11,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-11-24T07:58:43.107891+0000","last_change":"2023-11-24T07:58:41.344241+0000","last_active":"2023-11-24T07:58:43.107891+0000","last_peered":"2023-11-24T07:58:43.107891+0000","last_clean":"2023-11-24T07:58:43.107891+0000","last_became_active":"2023-11-24T07:58:41.343906+0000","last_became_peered":"2023-11-24T07:58:41.343906+0000","last_unstale":"2023-11-24T07:58:43.107891+0000","last_undegraded":"2023-11-24T07:58:43.107891+0000","last_fullsized":"2023-11-24T07:58:43.107891+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:58:40.368784+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:58:40.368784+0000","last_clean_scrub_stamp":"2023-11-24T07:58:40.368784+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T15:41:29.374563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.2","version":"0'0","reported_seq":11,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-11-24T07:58:41.336653+0000","last_change":"2023-11-24T07:58:40.396698+0000","last_active":"2023-11-24T07:58:41.336653+0000","last_peered":"2023-11-24T07:58:41.336653+0000","last_clean":"2023-11-24T07:58:41.336653+0000","last_became_active":"2023-11-24T07:58:40.396381+0000","last_became_peered":"2023-11-24T07:58:40.396381+0000","last_unstale":"2023-11-24T07:58:41.336653+0000","last_undegraded":"2023-11-24T07:58:41.336653+0000","last_fullsized":"2023-11-24T07:58:41.336653+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:58:39.367089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:58:39.367089+0000","last_clean_scrub_stamp":"2023-11-24T07:58:39.367089+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T17:33:40.508233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.0","version":"0'0","reported_seq":11,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-11-24T07:58:42.333376+0000","last_change":"2023-11-24T07:58:41.341803+0000","last_active":"2023-11-24T07:58:42.333376+0000","last_peered":"2023-11-24T07:58:42.333376+0000","last_clean":"2023-11-24T07:58:42.333376+0000","last_became_active":"2023-11-24T07:58:41.341525+0000","last_became_peered":"2023-11-24T07:58:41.341525+0000","last_unstale":"2023-11-24T07:58:42.333376+0000","last_undegraded":"2023-11-24T07:58:42.333376+0000","last_fullsized":"2023-11-24T07:58:42.333376+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:58:40.368784+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:58:40.368784+0000","last_clean_scrub_stamp":"2023-11-24T07:58:40.368784+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T17:14:37.101791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.1","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-11-24T07:58:42.333983+0000","last_change":"2023-11-24T07:58:40.393735+0000","last_active":"2023-11-24T07:58:42.333983+0000","last_peered":"2023-11-24T07:58:42.333983+0000","last_clean":"2023-11-24T07:58:42.333983+0000","last_became_active":"2023-11-24T07:58:40.393406+0000","last_became_peered":"2023-11-24T07:58:40.393406+0000","last_unstale":"2023-11-24T07:58:42.333983+0000","last_undegraded":"2023-11-24T07:58:42.333983+0000","last_fullsized":"2023-11-24T07:58:42.333983+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:58:39.367089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:58:39.367089+0000","last_clean_scrub_stamp":"2023-11-24T07:58:39.367089+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T12:38:46.310114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.1","version":"0'0","reported_seq":9,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-11-24T07:58:41.351195+0000","last_change":"2023-11-24T07:58:41.344358+0000","last_active":"2023-11-24T07:58:41.351195+0000","last_peered":"2023-11-24T07:58:41.351195+0000","last_clean":"2023-11-24T07:58:41.351195+0000","last_became_active":"2023-11-24T07:58:41.343875+0000","last_became_peered":"2023-11-24T07:58:41.343875+0000","last_unstale":"2023-11-24T07:58:41.351195+0000","last_undegraded":"2023-11-24T07:58:41.351195+0000","last_fullsized":"2023-11-24T07:58:41.351195+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:58:40.368784+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:58:40.368784+0000","last_clean_scrub_stamp":"2023-11-24T07:58:40.368784+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T18:51:40.361538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.0","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-11-24T07:58:42.333038+0000","last_change":"2023-11-24T07:58:40.391991+0000","last_active":"2023-11-24T07:58:42.333038+0000","last_peered":"2023-11-24T07:58:42.333038+0000","last_clean":"2023-11-24T07:58:42.333038+0000","last_became_active":"2023-11-24T07:58:40.391729+0000","last_became_peered":"2023-11-24T07:58:40.391729+0000","last_unstale":"2023-11-24T07:58:42.333038+0000","last_undegraded":"2023-11-24T07:58:42.333038+0000","last_fullsized":"2023-11-24T07:58:42.333038+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:58:39.367089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:58:39.367089+0000","last_clean_scrub_stamp":"2023-11-24T07:58:39.367089+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T15:40:07.386082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.8","version":"0'0","reported_seq":11,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-11-24T07:58:42.334508+0000","last_change":"2023-11-24T07:58:41.347848+0000","last_active":"2023-11-24T07:58:42.334508+0000","last_peered":"2023-11-24T07:58:42.334508+0000","last_clean":"2023-11-24T07:58:42.334508+0000","last_became_active":"2023-11-24T07:58:41.347579+0000","last_became_peered":"2023-11-24T07:58:41.347579+0000","last_unstale":"2023-11-24T07:58:42.334508+0000","last_undegraded":"2023-11-24T07:58:42.334508+0000","last_fullsized":"2023-11-24T07:58:42.334508+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:58:40.368784+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:58:40.368784+0000","last_clean_scrub_stamp":"2023-11-24T07:58:40.368784+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T17:19:15.866101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.9","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-11-24T07:58:43.107592+0000","last_change":"2023-11-24T07:58:40.389566+0000","last_active":"2023-11-24T07:58:43.107592+0000","last_peered":"2023-11-24T07:58:43.107592+0000","last_clean":"2023-11-24T07:58:43.107592+0000","last_became_active":"2023-11-24T07:58:40.389289+0000","last_became_peered":"2023-11-24T07:58:40.389289+0000","last_unstale":"2023-11-24T07:58:43.107592+0000","last_undegraded":"2023-11-24T07:58:43.107592+0000","last_fullsized":"2023-11-24T07:58:43.107592+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:58:39.367089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:58:39.367089+0000","last_clean_scrub_stamp":"2023-11-24T07:58:39.367089+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T18:42:48.188373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":199,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-11-24T07:58:41.338592+0000","last_change":"2023-11-24T07:56:19.668905+0000","last_active":"2023-11-24T07:58:41.338592+0000","last_peered":"2023-11-24T07:58:41.338592+0000","last_clean":"2023-11-24T07:58:41.338592+0000","last_became_active":"2023-11-24T07:56:19.668601+0000","last_became_peered":"2023-11-24T07:56:19.668601+0000","last_unstale":"2023-11-24T07:58:41.338592+0000","last_undegraded":"2023-11-24T07:58:41.338592+0000","last_fullsized":"2023-11-24T07:58:41.338592+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:56:18.657844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:56:18.657844+0000","last_clean_scrub_stamp":"2023-11-24T07:56:18.657844+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T11:51:53.828455+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":9,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":8,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574880,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27724,"kb_used_data":700,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344116,"statfs":{"total":96636764160,"available":96608374784,"internally_reserved":0,"allocated":716800,"data_stored":297068,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 07:58:20 2023","interfaces":[{"interface":"back","average":{"1min":0.51400000000000001,"5min":0.54800000000000004,"15min":0.55400000000000005},"min":{"1min":0.309,"5min":0.309,"15min":0.309},"max":{"1min":0.78500000000000003,"5min":1.413,"15min":1.413},"last":0.45200000000000001},{"interface":"front","average":{"1min":0.56299999999999994,"5min":0.55500000000000005,"15min":0.55400000000000005},"min":{"1min":0.436,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":0.77000000000000002,"5min":0.81000000000000005,"15min":0.81000000000000005},"last":0.48099999999999998}]},{"osd":1,"last update":"Fri Nov 24 07:58:20 2023","interfaces":[{"interface":"back","average":{"1min":0.501,"5min":0.77900000000000003,"15min":0.82499999999999996},"min":{"1min":0.32000000000000001,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.69899999999999995,"5min":6.1070000000000002,"15min":6.1070000000000002},"last":0.53900000000000003},{"interface":"front","average":{"1min":0.51300000000000001,"5min":0.56599999999999995,"15min":0.57499999999999996},"min":{"1min":0.40699999999999997,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.73599999999999999,"5min":1.5920000000000001,"15min":1.5920000000000001},"last":0.35199999999999998}]},{"osd":2,"last update":"Fri Nov 24 07:58:20 2023","interfaces":[{"interface":"back","average":{"1min":0.498,"5min":0.60099999999999998,"15min":0.61799999999999999},"min":{"1min":0.35099999999999998,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.71599999999999997,"5min":1.613,"15min":1.613},"last":0.40200000000000002},{"interface":"front","average":{"1min":0.502,"5min":0.73399999999999999,"15min":0.77200000000000002},"min":{"1min":0.34899999999999998,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.71299999999999997,"5min":4.5220000000000002,"15min":4.5220000000000002},"last":0.42299999999999999}]},{"osd":3,"last update":"Fri Nov 24 07:58:20 2023","interfaces":[{"interface":"back","average":{"1min":0.55100000000000005,"5min":0.755,"15min":0.78900000000000003},"min":{"1min":0.41699999999999998,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.754,"5min":4.6760000000000002,"15min":4.6760000000000002},"last":0.46800000000000003},{"interface":"front","average":{"1min":0.60699999999999998,"5min":0.875,"15min":0.92000000000000004},"min":{"1min":0.38500000000000001,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.81599999999999995,"5min":4.6539999999999999,"15min":4.6539999999999999},"last":0.51500000000000001}]},{"osd":4,"last update":"Fri Nov 24 07:58:20 2023","interfaces":[{"interface":"back","average":{"1min":0.54900000000000004,"5min":0.56399999999999995,"15min":0.56599999999999995},"min":{"1min":0.41799999999999998,"5min":0.378,"15min":0.378},"max":{"1min":0.70999999999999996,"5min":0.84699999999999998,"15min":0.84699999999999998},"last":0.374},{"interface":"front","average":{"1min":0.54700000000000004,"5min":0.64300000000000002,"15min":0.65900000000000003},"min":{"1min":0.35199999999999998,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.79900000000000004,"5min":1.629,"15min":1.629},"last":0.57799999999999996}]},{"osd":5,"last update":"Fri Nov 24 07:58:20 2023","interfaces":[{"interface":"back","average":{"1min":0.54200000000000004,"5min":0.54600000000000004,"15min":0.54600000000000004},"min":{"1min":0.41599999999999998,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.67900000000000005,"5min":0.875,"15min":0.875},"last":0.55500000000000005},{"interface":"front","average":{"1min":0.57799999999999996,"5min":0.61399999999999999,"15min":0.62},"min":{"1min":0.36799999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.77700000000000002,"5min":0.90900000000000003,"15min":0.90900000000000003},"last":0.60599999999999998}]},{"osd":6,"last update":"Fri Nov 24 07:58:20 2023","interfaces":[{"interface":"back","average":{"1min":0.59999999999999998,"5min":0.60799999999999998,"15min":0.60999999999999999},"min":{"1min":0.49099999999999999,"5min":0.438,"15min":0.438},"max":{"1min":0.73399999999999999,"5min":0.80800000000000005,"15min":0.80800000000000005},"last":0.59299999999999997},{"interface":"front","average":{"1min":0.63900000000000001,"5min":0.66300000000000003,"15min":0.66700000000000004},"min":{"1min":0.45600000000000002,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":0.79700000000000004,"5min":0.98399999999999999,"15min":0.98399999999999999},"last":0.499}]}]},{"osd":6,"up_from":13,"seq":55834574881,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27736,"kb_used_data":708,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344104,"statfs":{"total":96636764160,"available":96608362496,"internally_reserved":0,"allocated":724992,"data_stored":303298,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 07:58:19 2023","interfaces":[{"interface":"back","average":{"1min":0.53300000000000003,"5min":0.54600000000000004,"15min":0.54800000000000004},"min":{"1min":0.29799999999999999,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":0.93100000000000005,"5min":0.93100000000000005,"15min":0.93100000000000005},"last":0.69099999999999995},{"interface":"front","average":{"1min":0.51600000000000001,"5min":0.51600000000000001,"15min":0.51600000000000001},"min":{"1min":0.311,"5min":0.311,"15min":0.311},"max":{"1min":0.82199999999999995,"5min":0.82199999999999995,"15min":0.82199999999999995},"last":0.497}]},{"osd":1,"last update":"Fri Nov 24 07:58:19 2023","interfaces":[{"interface":"back","average":{"1min":0.51900000000000002,"5min":0.51300000000000001,"15min":0.51200000000000001},"min":{"1min":0.35599999999999998,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.81299999999999994,"5min":0.81299999999999994,"15min":0.81299999999999994},"last":0.443},{"interface":"front","average":{"1min":0.54000000000000004,"5min":0.59799999999999998,"15min":0.60799999999999998},"min":{"1min":0.35999999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.82799999999999996,"5min":1.0069999999999999,"15min":1.0069999999999999},"last":0.64800000000000002}]},{"osd":2,"last update":"Fri Nov 24 07:58:19 2023","interfaces":[{"interface":"back","average":{"1min":0.58399999999999996,"5min":0.57599999999999996,"15min":0.57499999999999996},"min":{"1min":0.34899999999999998,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.90600000000000003,"5min":0.90600000000000003,"15min":0.90600000000000003},"last":0.55200000000000005},{"interface":"front","average":{"1min":0.56499999999999995,"5min":0.58699999999999997,"15min":0.59099999999999997},"min":{"1min":0.34300000000000003,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.84899999999999998,"5min":0.84899999999999998,"15min":0.84899999999999998},"last":0.52500000000000002}]},{"osd":3,"last update":"Fri Nov 24 07:58:19 2023","interfaces":[{"interface":"back","average":{"1min":0.56000000000000005,"5min":0.56599999999999995,"15min":0.56699999999999995},"min":{"1min":0.38400000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.86499999999999999,"5min":0.876,"15min":0.876},"last":0.75700000000000001},{"interface":"front","average":{"1min":0.57699999999999996,"5min":0.56000000000000005,"15min":0.55700000000000005},"min":{"1min":0.39800000000000002,"5min":0.379,"15min":0.379},"max":{"1min":0.876,"5min":0.876,"15min":0.876},"last":0.62}]},{"osd":4,"last update":"Fri Nov 24 07:58:19 2023","interfaces":[{"interface":"back","average":{"1min":0.53100000000000003,"5min":0.51900000000000002,"15min":0.51700000000000002},"min":{"1min":0.25,"5min":0.25,"15min":0.25},"max":{"1min":0.89400000000000002,"5min":0.89400000000000002,"15min":0.89400000000000002},"last":0.73999999999999999},{"interface":"front","average":{"1min":0.58899999999999997,"5min":0.59899999999999998,"15min":0.60099999999999998},"min":{"1min":0.29099999999999998,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":1.044,"5min":1.087,"15min":1.087},"last":0.58299999999999996}]},{"osd":5,"last update":"Fri Nov 24 07:58:19 2023","interfaces":[{"interface":"back","average":{"1min":0.59499999999999997,"5min":0.54300000000000004,"15min":0.53500000000000003},"min":{"1min":0.27800000000000002,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":0.97999999999999998,"5min":0.97999999999999998,"15min":0.97999999999999998},"last":0.67200000000000004},{"interface":"front","average":{"1min":0.61399999999999999,"5min":0.61199999999999999,"15min":0.61199999999999999},"min":{"1min":0.31900000000000001,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":1.0009999999999999,"5min":1.0009999999999999,"15min":1.0009999999999999},"last":0.60299999999999998}]},{"osd":7,"last update":"Fri Nov 24 07:58:19 2023","interfaces":[{"interface":"back","average":{"1min":0.61799999999999999,"5min":0.61799999999999999,"15min":0.61799999999999999},"min":{"1min":0.26500000000000001,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":1.0880000000000001,"5min":1.0880000000000001,"15min":1.0880000000000001},"last":0.72399999999999998},{"interface":"front","average":{"1min":0.65200000000000002,"5min":0.66600000000000004,"15min":0.66800000000000004},"min":{"1min":0.34300000000000003,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.95599999999999996,"5min":0.95599999999999996,"15min":0.95599999999999996},"last":0.71099999999999997}]}]},{"osd":1,"up_from":13,"seq":55834574882,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28316,"kb_used_data":1288,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343524,"statfs":{"total":96636764160,"available":96607768576,"internally_reserved":0,"allocated":1318912,"data_stored":894226,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 07:58:24 2023","interfaces":[{"interface":"back","average":{"1min":0.33000000000000002,"5min":0.35699999999999998,"15min":0.36099999999999999},"min":{"1min":0.249,"5min":0.20200000000000001,"15min":0.20200000000000001},"max":{"1min":0.48799999999999999,"5min":0.70899999999999996,"15min":0.70899999999999996},"last":0.46100000000000002},{"interface":"front","average":{"1min":0.36399999999999999,"5min":0.39100000000000001,"15min":0.39500000000000002},"min":{"1min":0.23599999999999999,"5min":0.214,"15min":0.214},"max":{"1min":0.54700000000000004,"5min":0.82499999999999996,"15min":0.82499999999999996},"last":0.65600000000000003}]},{"osd":2,"last update":"Fri Nov 24 07:58:24 2023","interfaces":[{"interface":"back","average":{"1min":0.312,"5min":0.35999999999999999,"15min":0.36799999999999999},"min":{"1min":0.253,"5min":0.22700000000000001,"15min":0.22700000000000001},"max":{"1min":0.44800000000000001,"5min":0.60999999999999999,"15min":0.60999999999999999},"last":0.53000000000000003},{"interface":"front","average":{"1min":0.36499999999999999,"5min":0.40300000000000002,"15min":0.40899999999999997},"min":{"1min":0.28100000000000003,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":0.47599999999999998,"5min":0.621,"15min":0.621},"last":0.55200000000000005}]},{"osd":3,"last update":"Fri Nov 24 07:58:24 2023","interfaces":[{"interface":"back","average":{"1min":0.40300000000000002,"5min":0.435,"15min":0.441},"min":{"1min":0.31900000000000001,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.59199999999999997,"5min":0.66500000000000004,"15min":0.66500000000000004},"last":0.62},{"interface":"front","average":{"1min":0.38700000000000001,"5min":0.41899999999999998,"15min":0.42499999999999999},"min":{"1min":0.29199999999999998,"5min":0.28499999999999998,"15min":0.28499999999999998},"max":{"1min":0.56200000000000006,"5min":0.76300000000000001,"15min":0.76300000000000001},"last":0.49299999999999999}]},{"osd":4,"last update":"Fri Nov 24 07:58:24 2023","interfaces":[{"interface":"back","average":{"1min":0.56200000000000006,"5min":0.60099999999999998,"15min":0.60699999999999998},"min":{"1min":0.442,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":0.84099999999999997,"5min":0.84099999999999997,"15min":0.84099999999999997},"last":0.83799999999999997},{"interface":"front","average":{"1min":0.61099999999999999,"5min":0.60699999999999998,"15min":0.60599999999999998},"min":{"1min":0.499,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":0.79500000000000004,"5min":0.82199999999999995,"15min":0.82199999999999995},"last":0.68700000000000006}]},{"osd":5,"last update":"Fri Nov 24 07:58:24 2023","interfaces":[{"interface":"back","average":{"1min":0.59199999999999997,"5min":0.69799999999999995,"15min":0.71599999999999997},"min":{"1min":0.439,"5min":0.37,"15min":0.37},"max":{"1min":0.97599999999999998,"5min":2.7269999999999999,"15min":2.7269999999999999},"last":0.71499999999999997},{"interface":"front","average":{"1min":0.57699999999999996,"5min":0.63100000000000001,"15min":0.64000000000000001},"min":{"1min":0.39100000000000001,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.94499999999999995,"5min":0.94499999999999995,"15min":0.94499999999999995},"last":0.78600000000000003}]},{"osd":6,"last update":"Fri Nov 24 07:58:24 2023","interfaces":[{"interface":"back","average":{"1min":0.61199999999999999,"5min":0.64800000000000002,"15min":0.65400000000000003},"min":{"1min":0.47699999999999998,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":0.77500000000000002,"5min":0.98199999999999998,"15min":0.98199999999999998},"last":0.77800000000000002},{"interface":"front","average":{"1min":0.57599999999999996,"5min":0.60799999999999998,"15min":0.61399999999999999},"min":{"1min":0.40600000000000003,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":0.89600000000000002,"5min":0.90700000000000003,"15min":0.90700000000000003},"last":0.91600000000000004}]},{"osd":7,"last update":"Fri Nov 24 07:58:24 2023","interfaces":[{"interface":"back","average":{"1min":0.63100000000000001,"5min":0.63500000000000001,"15min":0.63600000000000001},"min":{"1min":0.43099999999999999,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.79800000000000004,"5min":1.0649999999999999,"15min":1.0649999999999999},"last":0.95599999999999996},{"interface":"front","average":{"1min":0.57099999999999995,"5min":0.65000000000000002,"15min":0.66300000000000003},"min":{"1min":0.45700000000000002,"5min":0.442,"15min":0.442},"max":{"1min":0.71299999999999997,"5min":0.88800000000000001,"15min":0.88800000000000001},"last":0.93300000000000005}]}]},{"osd":0,"up_from":13,"seq":55834574882,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27736,"kb_used_data":716,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344104,"statfs":{"total":96636764160,"available":96608362496,"internally_reserved":0,"allocated":733184,"data_stored":304888,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Nov 24 07:58:20 2023","interfaces":[{"interface":"back","average":{"1min":0.38500000000000001,"5min":0.64700000000000002,"15min":0.69099999999999995},"min":{"1min":0.21299999999999999,"5min":0.21299999999999999,"15min":0.21299999999999999},"max":{"1min":0.625,"5min":6.3550000000000004,"15min":6.3550000000000004},"last":0.56100000000000005},{"interface":"front","average":{"1min":0.40799999999999997,"5min":1.464,"15min":1.6399999999999999},"min":{"1min":0.26000000000000001,"5min":0.26000000000000001,"15min":0.26000000000000001},"max":{"1min":0.60999999999999999,"5min":22.916,"15min":22.916},"last":0.59099999999999997}]},{"osd":2,"last update":"Fri Nov 24 07:58:20 2023","interfaces":[{"interface":"back","average":{"1min":0.35299999999999998,"5min":0.63,"15min":0.67600000000000005},"min":{"1min":0.22900000000000001,"5min":0.22800000000000001,"15min":0.22800000000000001},"max":{"1min":0.499,"5min":6.3899999999999997,"15min":6.3899999999999997},"last":0.42299999999999999},{"interface":"front","average":{"1min":0.39500000000000002,"5min":0.92900000000000005,"15min":1.018},"min":{"1min":0.27900000000000003,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":0.60199999999999998,"5min":11.93,"15min":11.93},"last":0.495}]},{"osd":3,"last update":"Fri Nov 24 07:58:20 2023","interfaces":[{"interface":"back","average":{"1min":0.373,"5min":0.40999999999999998,"15min":0.41599999999999998},"min":{"1min":0.23799999999999999,"5min":0.23799999999999999,"15min":0.23799999999999999},"max":{"1min":0.79300000000000004,"5min":1.155,"15min":1.155},"last":0.45500000000000002},{"interface":"front","average":{"1min":0.438,"5min":0.98899999999999999,"15min":1.081},"min":{"1min":0.30199999999999999,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.69699999999999995,"5min":12.116,"15min":12.116},"last":0.51700000000000002}]},{"osd":4,"last update":"Fri Nov 24 07:58:20 2023","interfaces":[{"interface":"back","average":{"1min":0.626,"5min":0.64300000000000002,"15min":0.64600000000000002},"min":{"1min":0.40100000000000002,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":1.044,"5min":1.044,"15min":1.044},"last":0.748},{"interface":"front","average":{"1min":0.64300000000000002,"5min":1.675,"15min":1.847},"min":{"1min":0.35699999999999998,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.93000000000000005,"5min":22.963999999999999,"15min":22.963999999999999},"last":0.84999999999999998}]},{"osd":5,"last update":"Fri Nov 24 07:58:20 2023","interfaces":[{"interface":"back","average":{"1min":0.65200000000000002,"5min":1.1639999999999999,"15min":1.25},"min":{"1min":0.41999999999999998,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.97799999999999998,"5min":12.084,"15min":12.084},"last":0.81999999999999995},{"interface":"front","average":{"1min":0.65800000000000003,"5min":0.67400000000000004,"15min":0.67600000000000005},"min":{"1min":0.39700000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.95499999999999996,"5min":1.073,"15min":1.073},"last":0.875}]},{"osd":6,"last update":"Fri Nov 24 07:58:20 2023","interfaces":[{"interface":"back","average":{"1min":0.65200000000000002,"5min":0.68300000000000005,"15min":0.68799999999999994},"min":{"1min":0.45000000000000001,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":0.94699999999999995,"5min":1.0580000000000001,"15min":1.0580000000000001},"last":0.92400000000000004},{"interface":"front","average":{"1min":0.67000000000000004,"5min":1.1850000000000001,"15min":1.2709999999999999},"min":{"1min":0.46200000000000002,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":0.95899999999999996,"5min":12.045,"15min":12.045},"last":0.78900000000000003}]},{"osd":7,"last update":"Fri Nov 24 07:58:20 2023","interfaces":[{"interface":"back","average":{"1min":0.67000000000000004,"5min":0.65800000000000003,"15min":0.65600000000000003},"min":{"1min":0.45200000000000001,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":0.98299999999999998,"5min":1.038,"15min":1.038},"last":0.82999999999999996},{"interface":"front","average":{"1min":0.67500000000000004,"5min":1.7050000000000001,"15min":1.877},"min":{"1min":0.41299999999999998,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":1.0069999999999999,"5min":22.995000000000001,"15min":22.995000000000001},"last":0.80300000000000005}]}]},{"osd":2,"up_from":13,"seq":55834574882,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27736,"kb_used_data":712,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344104,"statfs":{"total":96636764160,"available":96608362496,"internally_reserved":0,"allocated":729088,"data_stored":298748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 07:58:19 2023","interfaces":[{"interface":"back","average":{"1min":0.36399999999999999,"5min":0.36899999999999999,"15min":0.37},"min":{"1min":0.219,"5min":0.219,"15min":0.219},"max":{"1min":0.55900000000000005,"5min":0.58099999999999996,"15min":0.58099999999999996},"last":0.27100000000000002},{"interface":"front","average":{"1min":0.38900000000000001,"5min":0.39900000000000002,"15min":0.40100000000000002},"min":{"1min":0.29299999999999998,"5min":0.27400000000000002,"15min":0.27400000000000002},"max":{"1min":0.54100000000000004,"5min":0.54100000000000004,"15min":0.54100000000000004},"last":0.40200000000000002}]},{"osd":1,"last update":"Fri Nov 24 07:58:19 2023","interfaces":[{"interface":"back","average":{"1min":0.34699999999999998,"5min":0.371,"15min":0.375},"min":{"1min":0.22,"5min":0.22,"15min":0.22},"max":{"1min":0.54000000000000004,"5min":0.59199999999999997,"15min":0.59199999999999997},"last":0.29899999999999999},{"interface":"front","average":{"1min":0.36599999999999999,"5min":0.40600000000000003,"15min":0.41199999999999998},"min":{"1min":0.28299999999999997,"5min":0.25600000000000001,"15min":0.25600000000000001},"max":{"1min":0.497,"5min":0.627,"15min":0.627},"last":0.36199999999999999}]},{"osd":3,"last update":"Fri Nov 24 07:58:19 2023","interfaces":[{"interface":"back","average":{"1min":0.40500000000000003,"5min":0.40899999999999997,"15min":0.40999999999999998},"min":{"1min":0.30399999999999999,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":0.64200000000000002,"5min":0.64200000000000002,"15min":0.64200000000000002},"last":0.33900000000000002},{"interface":"front","average":{"1min":0.41699999999999998,"5min":0.41399999999999998,"15min":0.41399999999999998},"min":{"1min":0.28599999999999998,"5min":0.27500000000000002,"15min":0.27500000000000002},"max":{"1min":0.59299999999999997,"5min":0.64600000000000002,"15min":0.64600000000000002},"last":0.51300000000000001}]},{"osd":4,"last update":"Fri Nov 24 07:58:19 2023","interfaces":[{"interface":"back","average":{"1min":0.54800000000000004,"5min":0.59599999999999997,"15min":0.60399999999999998},"min":{"1min":0.38700000000000001,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.75,"5min":0.81599999999999995,"15min":0.81599999999999995},"last":0.57799999999999996},{"interface":"front","average":{"1min":0.55500000000000005,"5min":0.59099999999999997,"15min":0.59699999999999998},"min":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":0.83799999999999997,"5min":0.83799999999999997,"15min":0.83799999999999997},"last":0.59199999999999997}]},{"osd":5,"last update":"Fri Nov 24 07:58:19 2023","interfaces":[{"interface":"back","average":{"1min":0.56000000000000005,"5min":0.60799999999999998,"15min":0.61599999999999999},"min":{"1min":0.434,"5min":0.434,"15min":0.434},"max":{"1min":0.73499999999999999,"5min":0.84899999999999998,"15min":0.84899999999999998},"last":0.67200000000000004},{"interface":"front","average":{"1min":0.624,"5min":0.69599999999999995,"15min":0.70799999999999996},"min":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":0.82499999999999996,"5min":2.1699999999999999,"15min":2.1699999999999999},"last":0.46200000000000002}]},{"osd":6,"last update":"Fri Nov 24 07:58:19 2023","interfaces":[{"interface":"back","average":{"1min":0.60499999999999998,"5min":0.65400000000000003,"15min":0.66200000000000003},"min":{"1min":0.42499999999999999,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":0.878,"5min":0.878,"15min":0.878},"last":0.48199999999999998},{"interface":"front","average":{"1min":0.60599999999999998,"5min":0.65800000000000003,"15min":0.66600000000000004},"min":{"1min":0.41899999999999998,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":0.80300000000000005,"5min":0.88500000000000001,"15min":0.88500000000000001},"last":0.61899999999999999}]},{"osd":7,"last update":"Fri Nov 24 07:58:19 2023","interfaces":[{"interface":"back","average":{"1min":0.60399999999999998,"5min":0.65000000000000002,"15min":0.65800000000000003},"min":{"1min":0.441,"5min":0.441,"15min":0.441},"max":{"1min":0.77600000000000002,"5min":0.85999999999999999,"15min":0.85999999999999999},"last":0.64500000000000002},{"interface":"front","average":{"1min":0.623,"5min":0.66800000000000004,"15min":0.67600000000000005},"min":{"1min":0.40000000000000002,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":0.90000000000000002,"5min":0.90000000000000002,"15min":0.90000000000000002},"last":0.502}]}]},{"osd":3,"up_from":13,"seq":55834574882,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27720,"kb_used_data":696,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344120,"statfs":{"total":96636764160,"available":96608378880,"internally_reserved":0,"allocated":712704,"data_stored":297066,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 07:58:22 2023","interfaces":[{"interface":"back","average":{"1min":0.34200000000000003,"5min":0.35299999999999998,"15min":0.35499999999999998},"min":{"1min":0.20899999999999999,"5min":0.20899999999999999,"15min":0.20899999999999999},"max":{"1min":0.439,"5min":0.84799999999999998,"15min":0.84799999999999998},"last":0.22900000000000001},{"interface":"front","average":{"1min":0.32700000000000001,"5min":0.35899999999999999,"15min":0.36399999999999999},"min":{"1min":0.253,"5min":0.23699999999999999,"15min":0.23699999999999999},"max":{"1min":0.42499999999999999,"5min":0.873,"15min":0.873},"last":0.32900000000000001}]},{"osd":1,"last update":"Fri Nov 24 07:58:22 2023","interfaces":[{"interface":"back","average":{"1min":0.35899999999999999,"5min":0.371,"15min":0.373},"min":{"1min":0.29499999999999998,"5min":0.25700000000000001,"15min":0.25700000000000001},"max":{"1min":0.443,"5min":0.92000000000000004,"15min":0.92000000000000004},"last":0.28799999999999998},{"interface":"front","average":{"1min":0.35099999999999998,"5min":0.379,"15min":0.38300000000000001},"min":{"1min":0.28299999999999997,"5min":0.248,"15min":0.248},"max":{"1min":0.46700000000000003,"5min":0.81699999999999995,"15min":0.81699999999999995},"last":0.39900000000000002}]},{"osd":2,"last update":"Fri Nov 24 07:58:22 2023","interfaces":[{"interface":"back","average":{"1min":0.33400000000000002,"5min":0.35499999999999998,"15min":0.35899999999999999},"min":{"1min":0.255,"5min":0.24399999999999999,"15min":0.24399999999999999},"max":{"1min":0.43099999999999999,"5min":0.88500000000000001,"15min":0.88500000000000001},"last":0.35399999999999998},{"interface":"front","average":{"1min":0.39500000000000002,"5min":0.45300000000000001,"15min":0.46300000000000002},"min":{"1min":0.311,"5min":0.311,"15min":0.311},"max":{"1min":0.45800000000000002,"5min":0.98099999999999998,"15min":0.98099999999999998},"last":0.38300000000000001}]},{"osd":4,"last update":"Fri Nov 24 07:58:22 2023","interfaces":[{"interface":"back","average":{"1min":0.57099999999999995,"5min":0.751,"15min":0.78100000000000003},"min":{"1min":0.42099999999999999,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.71999999999999997,"5min":5.7350000000000003,"15min":5.7350000000000003},"last":0.54000000000000004},{"interface":"front","average":{"1min":0.59299999999999997,"5min":0.58599999999999997,"15min":0.58499999999999996},"min":{"1min":0.37,"5min":0.37,"15min":0.37},"max":{"1min":0.77000000000000002,"5min":1.27,"15min":1.27},"last":0.45800000000000002}]},{"osd":5,"last update":"Fri Nov 24 07:58:22 2023","interfaces":[{"interface":"back","average":{"1min":0.56899999999999995,"5min":0.58399999999999996,"15min":0.58599999999999997},"min":{"1min":0.40799999999999997,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":0.82799999999999996,"5min":1.4430000000000001,"15min":1.4430000000000001},"last":0.42899999999999999},{"interface":"front","average":{"1min":0.60699999999999998,"5min":0.623,"15min":0.625},"min":{"1min":0.443,"5min":0.377,"15min":0.377},"max":{"1min":0.79900000000000004,"5min":1.4239999999999999,"15min":1.4239999999999999},"last":0.48699999999999999}]},{"osd":6,"last update":"Fri Nov 24 07:58:22 2023","interfaces":[{"interface":"back","average":{"1min":0.58999999999999997,"5min":0.61399999999999999,"15min":0.61799999999999999},"min":{"1min":0.46200000000000002,"5min":0.434,"15min":0.434},"max":{"1min":0.82599999999999996,"5min":1.355,"15min":1.355},"last":0.52600000000000002},{"interface":"front","average":{"1min":0.61199999999999999,"5min":0.79900000000000004,"15min":0.82999999999999996},"min":{"1min":0.47499999999999998,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.76900000000000002,"5min":5.6479999999999997,"15min":5.6479999999999997},"last":0.44800000000000001}]},{"osd":7,"last update":"Fri Nov 24 07:58:22 2023","interfaces":[{"interface":"back","average":{"1min":0.59199999999999997,"5min":0.65100000000000002,"15min":0.66100000000000003},"min":{"1min":0.48099999999999998,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":0.79600000000000004,"5min":1.51,"15min":1.51},"last":0.502},{"interface":"front","average":{"1min":0.61099999999999999,"5min":0.79500000000000004,"15min":0.82599999999999996},"min":{"1min":0.434,"5min":0.434,"15min":0.434},"max":{"1min":0.75900000000000001,"5min":5.7069999999999999,"15min":5.7069999999999999},"last":0.47999999999999998}]}]},{"osd":4,"up_from":13,"seq":55834574881,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27724,"kb_used_data":700,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344116,"statfs":{"total":96636764160,"available":96608374784,"internally_reserved":0,"allocated":716800,"data_stored":297592,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 07:58:25 2023","interfaces":[{"interface":"back","average":{"1min":0.504,"5min":0.56499999999999995,"15min":0.57499999999999996},"min":{"1min":0.34899999999999998,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.70799999999999996,"5min":0.79300000000000004,"15min":0.79300000000000004},"last":0.48899999999999999},{"interface":"front","average":{"1min":0.502,"5min":0.53400000000000003,"15min":0.54000000000000004},"min":{"1min":0.33600000000000002,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.70199999999999996,"5min":1.0249999999999999,"15min":1.0249999999999999},"last":0.58699999999999997}]},{"osd":1,"last update":"Fri Nov 24 07:58:25 2023","interfaces":[{"interface":"back","average":{"1min":0.52800000000000002,"5min":0.54700000000000004,"15min":0.55000000000000004},"min":{"1min":0.39500000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.73999999999999999,"5min":0.86399999999999999,"15min":0.86399999999999999},"last":0.64300000000000002},{"interface":"front","average":{"1min":0.51900000000000002,"5min":0.54400000000000004,"15min":0.54800000000000004},"min":{"1min":0.374,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.72199999999999998,"5min":0.98699999999999999,"15min":0.98699999999999999},"last":0.60899999999999999}]},{"osd":2,"last update":"Fri Nov 24 07:58:25 2023","interfaces":[{"interface":"back","average":{"1min":0.52000000000000002,"5min":0.55200000000000005,"15min":0.55800000000000005},"min":{"1min":0.38800000000000001,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.66400000000000003,"5min":0.81100000000000005,"15min":0.81100000000000005},"last":0.45900000000000002},{"interface":"front","average":{"1min":0.56899999999999995,"5min":0.60399999999999998,"15min":0.60999999999999999},"min":{"1min":0.40000000000000002,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.75700000000000001,"5min":0.89200000000000002,"15min":0.89200000000000002},"last":0.66000000000000003}]},{"osd":3,"last update":"Fri Nov 24 07:58:25 2023","interfaces":[{"interface":"back","average":{"1min":0.54900000000000004,"5min":0.57699999999999996,"15min":0.58199999999999996},"min":{"1min":0.42899999999999999,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":0.65800000000000003,"5min":0.89700000000000002,"15min":0.89700000000000002},"last":0.51800000000000002},{"interface":"front","average":{"1min":0.57499999999999996,"5min":0.61899999999999999,"15min":0.627},"min":{"1min":0.442,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.72899999999999998,"5min":0.96199999999999997,"15min":0.96199999999999997},"last":0.69099999999999995}]},{"osd":5,"last update":"Fri Nov 24 07:58:25 2023","interfaces":[{"interface":"back","average":{"1min":0.48599999999999999,"5min":0.46100000000000002,"15min":0.45700000000000002},"min":{"1min":0.29499999999999998,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":0.67600000000000005,"5min":0.70499999999999996,"15min":0.70499999999999996},"last":0.54000000000000004},{"interface":"front","average":{"1min":0.59899999999999998,"5min":0.61499999999999999,"15min":0.61699999999999999},"min":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.746,"5min":0.86399999999999999,"15min":0.86399999999999999},"last":0.56399999999999995}]},{"osd":6,"last update":"Fri Nov 24 07:58:25 2023","interfaces":[{"interface":"back","average":{"1min":0.56799999999999995,"5min":0.58199999999999996,"15min":0.58399999999999996},"min":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.69799999999999995,"5min":0.76800000000000002,"15min":0.76800000000000002},"last":0.71699999999999997},{"interface":"front","average":{"1min":0.58999999999999997,"5min":0.60899999999999999,"15min":0.61199999999999999},"min":{"1min":0.46899999999999997,"5min":0.436,"15min":0.436},"max":{"1min":0.77700000000000002,"5min":1.095,"15min":1.095},"last":0.68100000000000005}]},{"osd":7,"last update":"Fri Nov 24 07:58:25 2023","interfaces":[{"interface":"back","average":{"1min":0.56799999999999995,"5min":0.57199999999999995,"15min":0.57199999999999995},"min":{"1min":0.39300000000000002,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.74199999999999999,"5min":0.89300000000000002,"15min":0.89300000000000002},"last":0.626},{"interface":"front","average":{"1min":0.61699999999999999,"5min":0.65300000000000002,"15min":0.65900000000000003},"min":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.79100000000000004,"5min":0.91500000000000004,"15min":0.91500000000000004},"last":0.74299999999999999}]}]},{"osd":5,"up_from":13,"seq":55834574882,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28316,"kb_used_data":1288,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343524,"statfs":{"total":96636764160,"available":96607768576,"internally_reserved":0,"allocated":1318912,"data_stored":893678,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 07:58:20 2023","interfaces":[{"interface":"back","average":{"1min":0.52300000000000002,"5min":0.55400000000000005,"15min":0.55900000000000005},"min":{"1min":0.35299999999999998,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.65800000000000003,"5min":1.0109999999999999,"15min":1.0109999999999999},"last":0.442},{"interface":"front","average":{"1min":0.53200000000000003,"5min":0.746,"15min":0.78200000000000003},"min":{"1min":0.34399999999999997,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.72099999999999997,"5min":5.431,"15min":5.431},"last":0.48299999999999998}]},{"osd":1,"last update":"Fri Nov 24 07:58:20 2023","interfaces":[{"interface":"back","average":{"1min":0.51100000000000001,"5min":0.54900000000000004,"15min":0.55500000000000005},"min":{"1min":0.36399999999999999,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":0.67100000000000004,"5min":1.016,"15min":1.016},"last":0.46400000000000002},{"interface":"front","average":{"1min":0.54600000000000004,"5min":0.56499999999999995,"15min":0.56799999999999995},"min":{"1min":0.28599999999999998,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":0.78300000000000003,"5min":0.96499999999999997,"15min":0.96499999999999997},"last":0.54800000000000004}]},{"osd":2,"last update":"Fri Nov 24 07:58:20 2023","interfaces":[{"interface":"back","average":{"1min":0.54500000000000004,"5min":0.54900000000000004,"15min":0.55000000000000004},"min":{"1min":0.33700000000000002,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.67100000000000004,"5min":1.163,"15min":1.163},"last":0.60499999999999998},{"interface":"front","average":{"1min":0.54500000000000004,"5min":0.61299999999999999,"15min":0.624},"min":{"1min":0.41099999999999998,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.78900000000000003,"5min":1.119,"15min":1.119},"last":0.57699999999999996}]},{"osd":3,"last update":"Fri Nov 24 07:58:20 2023","interfaces":[{"interface":"back","average":{"1min":0.57199999999999995,"5min":0.83799999999999997,"15min":0.88200000000000001},"min":{"1min":0.36899999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.85899999999999999,"5min":5.609,"15min":5.609},"last":0.59299999999999997},{"interface":"front","average":{"1min":0.629,"5min":0.66600000000000004,"15min":0.67200000000000004},"min":{"1min":0.44600000000000001,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":0.76400000000000001,"5min":1.288,"15min":1.288},"last":0.65100000000000002}]},{"osd":4,"last update":"Fri Nov 24 07:58:20 2023","interfaces":[{"interface":"back","average":{"1min":0.51500000000000001,"5min":0.51100000000000001,"15min":0.51000000000000001},"min":{"1min":0.32100000000000001,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.76400000000000001,"5min":0.76400000000000001,"15min":0.76400000000000001},"last":0.79100000000000004},{"interface":"front","average":{"1min":0.58799999999999997,"5min":0.57499999999999996,"15min":0.57299999999999995},"min":{"1min":0.29599999999999999,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":0.92900000000000005,"5min":0.92900000000000005,"15min":0.92900000000000005},"last":0.63700000000000001}]},{"osd":6,"last update":"Fri Nov 24 07:58:20 2023","interfaces":[{"interface":"back","average":{"1min":0.59599999999999997,"5min":0.57599999999999996,"15min":0.57199999999999995},"min":{"1min":0.35199999999999998,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.81799999999999995,"5min":0.89700000000000002,"15min":0.89700000000000002},"last":0.69699999999999995},{"interface":"front","average":{"1min":0.63800000000000001,"5min":0.627,"15min":0.625},"min":{"1min":0.40300000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":1.083,"5min":1.083,"15min":1.083},"last":0.74299999999999999}]},{"osd":7,"last update":"Fri Nov 24 07:58:20 2023","interfaces":[{"interface":"back","average":{"1min":0.66200000000000003,"5min":0.64900000000000002,"15min":0.64700000000000002},"min":{"1min":0.36799999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.90100000000000002,"5min":0.96599999999999997,"15min":0.96599999999999997},"last":0.67500000000000004},{"interface":"front","average":{"1min":0.66800000000000004,"5min":0.65600000000000003,"15min":0.65400000000000003},"min":{"1min":0.42699999999999999,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":0.85599999999999998,"5min":0.94199999999999995,"15min":0.94199999999999995},"last":0.72899999999999998}]}]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1772,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-11-24T07:58:45.547 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-11-24T07:58:45.703 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:45.701+0000 7f3a87604700 1 Processor -- start 2023-11-24T07:58:45.703 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:45.701+0000 7f3a87604700 1 -- start start 2023-11-24T07:58:45.705 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:45.702+0000 7f3a87604700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f3a8006b620 0x7f3a800b3a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:45.706 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:45.702+0000 7f3a87604700 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f3a800b5ab0 0x7f3a800b3f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:45.706 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:45.702+0000 7f3a87604700 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3a80071f50 con 0x7f3a801478c0 2023-11-24T07:58:45.706 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:45.702+0000 7f3a87604700 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3a80072230 con 0x7f3a8006b620 2023-11-24T07:58:45.706 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:45.702+0000 7f3a87604700 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3a80072510 con 0x7f3a800b5ab0 2023-11-24T07:58:45.706 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:45.702+0000 7f3a853a0700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f3a8006b620 0x7f3a800b3a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:45.706 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:45.702+0000 7f3a853a0700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f3a8006b620 0x7f3a800b3a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:53894/0 (socket says 172.21.15.59:53894) 2023-11-24T07:58:45.707 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:45.702+0000 7f3a853a0700 1 -- 172.21.15.59:0/3506027724 learned_addr learned my addr 172.21.15.59:0/3506027724 (peer_addr_for_me v2:172.21.15.59:0/0) 2023-11-24T07:58:45.707 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:45.703+0000 7f3a84b9f700 1 --2- 172.21.15.59:0/3506027724 >> v2:172.21.15.76:3300/0 conn(0x7f3a800b5ab0 0x7f3a800b3f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:45.707 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:45.703+0000 7f3a853a0700 1 -- 172.21.15.59:0/3506027724 >> v2:172.21.15.76:3300/0 conn(0x7f3a800b5ab0 msgr2=0x7f3a800b3f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T07:58:45.707 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:45.703+0000 7f3a853a0700 1 --2- 172.21.15.59:0/3506027724 >> v2:172.21.15.76:3300/0 conn(0x7f3a800b5ab0 0x7f3a800b3f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:45.707 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:45.703+0000 7f3a853a0700 1 -- 172.21.15.59:0/3506027724 >> v1:172.21.15.76:6789/0 conn(0x7f3a801478c0 legacy=0x7f3a80161ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T07:58:45.708 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:45.703+0000 7f3a853a0700 1 -- 172.21.15.59:0/3506027724 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3a800727f0 con 0x7f3a8006b620 2023-11-24T07:58:45.708 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:45.703+0000 7f3a853a0700 1 --2- 172.21.15.59:0/3506027724 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3a8006b620 0x7f3a800b3a40 secure :-1 s=READY pgs=714 cs=0 l=1 rev1=1 crypto rx=0x7f3a8006d6f0 tx=0x7f3a7c009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5efbc4ddca85ea33 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:58:45.708 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:45.703+0000 7f3a73fff700 1 -- 172.21.15.59:0/3506027724 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a7c004230 con 0x7f3a8006b620 2023-11-24T07:58:45.708 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:45.703+0000 7f3a73fff700 1 -- 172.21.15.59:0/3506027724 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3a7c004390 con 0x7f3a8006b620 2023-11-24T07:58:45.708 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:45.703+0000 7f3a73fff700 1 -- 172.21.15.59:0/3506027724 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a7c0036a0 con 0x7f3a8006b620 2023-11-24T07:58:45.708 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:45.704+0000 7f3a87604700 1 -- 172.21.15.59:0/3506027724 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3a8006b620 msgr2=0x7f3a800b3a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:45.708 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:45.704+0000 7f3a87604700 1 --2- 172.21.15.59:0/3506027724 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3a8006b620 0x7f3a800b3a40 secure :-1 s=READY pgs=714 cs=0 l=1 rev1=1 crypto rx=0x7f3a8006d6f0 tx=0x7f3a7c009bd0 comp rx=0 tx=0).stop 2023-11-24T07:58:45.709 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:45.704+0000 7f3a87604700 1 -- 172.21.15.59:0/3506027724 shutdown_connections 2023-11-24T07:58:45.709 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:45.704+0000 7f3a87604700 1 --2- 172.21.15.59:0/3506027724 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3a8006b620 0x7f3a800b3a40 unknown :-1 s=CLOSED pgs=714 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:45.709 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:45.704+0000 7f3a87604700 1 --2- 172.21.15.59:0/3506027724 >> v2:172.21.15.76:3300/0 conn(0x7f3a800b5ab0 0x7f3a800b3f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:45.709 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:45.704+0000 7f3a87604700 1 -- 172.21.15.59:0/3506027724 >> 172.21.15.59:0/3506027724 conn(0x7f3a800b2740 msgr2=0x7f3a800b2b20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:58:45.710 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:45.704+0000 7f3a87604700 1 -- 172.21.15.59:0/3506027724 shutdown_connections 2023-11-24T07:58:45.710 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:45.704+0000 7f3a87604700 1 -- 172.21.15.59:0/3506027724 wait complete. 2023-11-24T07:58:45.714 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:45.705+0000 7f3a87604700 1 Processor -- start 2023-11-24T07:58:45.715 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:45.705+0000 7f3a87604700 1 -- start start 2023-11-24T07:58:45.715 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:45.706+0000 7f3a87604700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3a8006b620 0x7f3a800c9120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:45.715 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:45.706+0000 7f3a87604700 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f3a800b5ab0 0x7f3a800c9660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:45.715 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:45.706+0000 7f3a87604700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3a801478c0 0x7f3a801f72d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:45.715 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:45.706+0000 7f3a87604700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a800c7170 con 0x7f3a8006b620 2023-11-24T07:58:45.715 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:45.706+0000 7f3a87604700 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a800af4a0 con 0x7f3a801478c0 2023-11-24T07:58:45.716 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:45.706+0000 7f3a87604700 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3a800c67d0 con 0x7f3a800b5ab0 2023-11-24T07:58:45.716 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:45.706+0000 7f3a853a0700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3a8006b620 0x7f3a800c9120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:45.716 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:45.706+0000 7f3a853a0700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3a8006b620 0x7f3a800c9120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:53900/0 (socket says 172.21.15.59:53900) 2023-11-24T07:58:45.716 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:45.706+0000 7f3a853a0700 1 -- 172.21.15.59:0/943313377 learned_addr learned my addr 172.21.15.59:0/943313377 (peer_addr_for_me v2:172.21.15.59:0/0) 2023-11-24T07:58:45.716 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:45.706+0000 7f3a85ba1700 1 --2- 172.21.15.59:0/943313377 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3a801478c0 0x7f3a801f72d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:45.716 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:45.706+0000 7f3a84b9f700 1 --2- 172.21.15.59:0/943313377 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f3a800b5ab0 0x7f3a800c9660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:45.716 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:45.706+0000 7f3a853a0700 1 -- 172.21.15.59:0/943313377 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f3a800b5ab0 msgr2=0x7f3a800c9660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:45.717 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:45.706+0000 7f3a853a0700 1 --2- 172.21.15.59:0/943313377 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f3a800b5ab0 0x7f3a800c9660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:45.717 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:45.706+0000 7f3a853a0700 1 -- 172.21.15.59:0/943313377 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3a801478c0 msgr2=0x7f3a801f72d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:45.717 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:45.706+0000 7f3a853a0700 1 --2- 172.21.15.59:0/943313377 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3a801478c0 0x7f3a801f72d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:45.717 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:45.706+0000 7f3a853a0700 1 -- 172.21.15.59:0/943313377 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3a7c009550 con 0x7f3a8006b620 2023-11-24T07:58:45.717 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:45.706+0000 7f3a85ba1700 1 --2- 172.21.15.59:0/943313377 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3a801478c0 0x7f3a801f72d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T07:58:45.717 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:45.706+0000 7f3a84b9f700 1 --2- 172.21.15.59:0/943313377 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f3a800b5ab0 0x7f3a800c9660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T07:58:45.717 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:45.706+0000 7f3a853a0700 1 --2- 172.21.15.59:0/943313377 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3a8006b620 0x7f3a800c9120 secure :-1 s=READY pgs=715 cs=0 l=1 rev1=1 crypto rx=0x7f3a7c009130 tx=0x7f3a7c004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:58:45.718 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:45.706+0000 7f3a71ffb700 1 -- 172.21.15.59:0/943313377 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a7c003dc0 con 0x7f3a8006b620 2023-11-24T07:58:45.718 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:45.706+0000 7f3a71ffb700 1 -- 172.21.15.59:0/943313377 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3a7c01f070 con 0x7f3a8006b620 2023-11-24T07:58:45.718 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:45.706+0000 7f3a87604700 1 -- 172.21.15.59:0/943313377 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3a800c6ab0 con 0x7f3a8006b620 2023-11-24T07:58:45.718 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:45.706+0000 7f3a71ffb700 1 -- 172.21.15.59:0/943313377 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a7c011870 con 0x7f3a8006b620 2023-11-24T07:58:45.718 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:45.707+0000 7f3a87604700 1 -- 172.21.15.59:0/943313377 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3a801f7810 con 0x7f3a8006b620 2023-11-24T07:58:45.718 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:45.708+0000 7f3a87604700 1 -- 172.21.15.59:0/943313377 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3a80068670 con 0x7f3a8006b620 2023-11-24T07:58:45.719 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:45.708+0000 7f3a71ffb700 1 -- 172.21.15.59:0/943313377 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 8) v1 ==== 89435+0+0 (secure 0 0 0) 0x7f3a7c021030 con 0x7f3a8006b620 2023-11-24T07:58:45.719 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:45.708+0000 7f3a71ffb700 1 --2- 172.21.15.59:0/943313377 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f3a6406b110 0x7f3a6406d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:45.719 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:45.709+0000 7f3a71ffb700 1 -- 172.21.15.59:0/943313377 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(60..60 src has 1..60) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f3a7c08b0b0 con 0x7f3a8006b620 2023-11-24T07:58:45.719 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:45.709+0000 7f3a84b9f700 1 --2- 172.21.15.59:0/943313377 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f3a6406b110 0x7f3a6406d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:45.719 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:45.709+0000 7f3a84b9f700 1 --2- 172.21.15.59:0/943313377 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f3a6406b110 0x7f3a6406d5d0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f3a74006fd0 tx=0x7f3a74008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:58:45.719 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:45.713+0000 7f3a71ffb700 1 -- 172.21.15.59:0/943313377 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f3a7c056080 con 0x7f3a8006b620 2023-11-24T07:58:45.911 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:45.909+0000 7f3a87604700 1 -- 172.21.15.59:0/943313377 --> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f3a8006ba00 con 0x7f3a6406b110 2023-11-24T07:58:45.920 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:45.918+0000 7f3a71ffb700 1 -- 172.21.15.59:0/943313377 <== mgr.4100 v2:172.21.15.76:6832/4264046536 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+289224 (secure 0 0 0) 0x7f3a7400c040 con 0x7f3a6406b110 2023-11-24T07:58:45.923 INFO:teuthology.orchestra.run.smithi059.stdout: 2023-11-24T07:58:45.926 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:45.922+0000 7f3a87604700 1 -- 172.21.15.59:0/943313377 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f3a6406b110 msgr2=0x7f3a6406d5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:45.926 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:45.922+0000 7f3a87604700 1 --2- 172.21.15.59:0/943313377 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f3a6406b110 0x7f3a6406d5d0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f3a74006fd0 tx=0x7f3a74008040 comp rx=0 tx=0).stop 2023-11-24T07:58:45.926 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:45.922+0000 7f3a87604700 1 -- 172.21.15.59:0/943313377 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3a8006b620 msgr2=0x7f3a800c9120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:45.926 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:45.922+0000 7f3a87604700 1 --2- 172.21.15.59:0/943313377 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3a8006b620 0x7f3a800c9120 secure :-1 s=READY pgs=715 cs=0 l=1 rev1=1 crypto rx=0x7f3a7c009130 tx=0x7f3a7c004750 comp rx=0 tx=0).stop 2023-11-24T07:58:45.926 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:45.922+0000 7f3a87604700 1 -- 172.21.15.59:0/943313377 shutdown_connections 2023-11-24T07:58:45.926 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:45.922+0000 7f3a87604700 1 --2- 172.21.15.59:0/943313377 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f3a6406b110 0x7f3a6406d5d0 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:45.926 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:45.922+0000 7f3a87604700 1 --2- 172.21.15.59:0/943313377 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3a801478c0 0x7f3a801f72d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:45.926 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:45.922+0000 7f3a87604700 1 --2- 172.21.15.59:0/943313377 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3a8006b620 0x7f3a800c9120 unknown :-1 s=CLOSED pgs=715 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:45.926 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:45.922+0000 7f3a87604700 1 --2- 172.21.15.59:0/943313377 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f3a800b5ab0 0x7f3a800c9660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:45.926 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:45.922+0000 7f3a87604700 1 -- 172.21.15.59:0/943313377 >> 172.21.15.59:0/943313377 conn(0x7f3a800b2740 msgr2=0x7f3a801462d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:58:45.927 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:45.922+0000 7f3a87604700 1 -- 172.21.15.59:0/943313377 shutdown_connections 2023-11-24T07:58:45.927 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:45.922+0000 7f3a87604700 1 -- 172.21.15.59:0/943313377 wait complete. 2023-11-24T07:58:45.927 INFO:teuthology.orchestra.run.smithi059.stderr:dumped all 2023-11-24T07:58:45.935 INFO:teuthology.orchestra.run.smithi059.stdout:{"pg_ready":true,"pg_map":{"version":126,"stamp":"2023-11-24T07:58:44.331762+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":104,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":223008,"kb_used_data":6808,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754751712,"statfs":{"total":773094113280,"available":772865753088,"internally_reserved":0,"allocated":6971392,"data_stored":3586564,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12745,"internal_metadata":221236791},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":30,"apply_latency_ms":30,"commit_latency_ns":30000000,"apply_latency_ns":30000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.961770"},"pg_stats":[{"pgid":"9.3f","version":"0'0","reported_seq":11,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-11-24T07:58:42.333841+0000","last_change":"2023-11-24T07:58:41.343589+0000","last_active":"2023-11-24T07:58:42.333841+0000","last_peered":"2023-11-24T07:58:42.333841+0000","last_clean":"2023-11-24T07:58:42.333841+0000","last_became_active":"2023-11-24T07:58:41.341662+0000","last_became_peered":"2023-11-24T07:58:41.341662+0000","last_unstale":"2023-11-24T07:58:42.333841+0000","last_undegraded":"2023-11-24T07:58:42.333841+0000","last_fullsized":"2023-11-24T07:58:42.333841+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:58:40.368784+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:58:40.368784+0000","last_clean_scrub_stamp":"2023-11-24T07:58:40.368784+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T17:33:41.509928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.3e","version":"0'0","reported_seq":9,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-11-24T07:58:41.357731+0000","last_change":"2023-11-24T07:58:41.352453+0000","last_active":"2023-11-24T07:58:41.357731+0000","last_peered":"2023-11-24T07:58:41.357731+0000","last_clean":"2023-11-24T07:58:41.357731+0000","last_became_active":"2023-11-24T07:58:41.351988+0000","last_became_peered":"2023-11-24T07:58:41.351988+0000","last_unstale":"2023-11-24T07:58:41.357731+0000","last_undegraded":"2023-11-24T07:58:41.357731+0000","last_fullsized":"2023-11-24T07:58:41.357731+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:58:40.368784+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:58:40.368784+0000","last_clean_scrub_stamp":"2023-11-24T07:58:40.368784+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:52:37.376571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.3d","version":"0'0","reported_seq":9,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-11-24T07:58:41.347763+0000","last_change":"2023-11-24T07:58:41.341529+0000","last_active":"2023-11-24T07:58:41.347763+0000","last_peered":"2023-11-24T07:58:41.347763+0000","last_clean":"2023-11-24T07:58:41.347763+0000","last_became_active":"2023-11-24T07:58:41.341183+0000","last_became_peered":"2023-11-24T07:58:41.341183+0000","last_unstale":"2023-11-24T07:58:41.347763+0000","last_undegraded":"2023-11-24T07:58:41.347763+0000","last_fullsized":"2023-11-24T07:58:41.347763+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:58:40.368784+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:58:40.368784+0000","last_clean_scrub_stamp":"2023-11-24T07:58:40.368784+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T18:42:49.190068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.3c","version":"0'0","reported_seq":9,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-11-24T07:58:41.355209+0000","last_change":"2023-11-24T07:58:41.353611+0000","last_active":"2023-11-24T07:58:41.355209+0000","last_peered":"2023-11-24T07:58:41.355209+0000","last_clean":"2023-11-24T07:58:41.355209+0000","last_became_active":"2023-11-24T07:58:41.353250+0000","last_became_peered":"2023-11-24T07:58:41.353250+0000","last_unstale":"2023-11-24T07:58:41.355209+0000","last_undegraded":"2023-11-24T07:58:41.355209+0000","last_fullsized":"2023-11-24T07:58:41.355209+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:58:40.368784+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:58:40.368784+0000","last_clean_scrub_stamp":"2023-11-24T07:58:40.368784+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T19:01:13.479431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.3b","version":"0'0","reported_seq":11,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-11-24T07:58:42.334492+0000","last_change":"2023-11-24T07:58:41.348515+0000","last_active":"2023-11-24T07:58:42.334492+0000","last_peered":"2023-11-24T07:58:42.334492+0000","last_clean":"2023-11-24T07:58:42.334492+0000","last_became_active":"2023-11-24T07:58:41.348217+0000","last_became_peered":"2023-11-24T07:58:41.348217+0000","last_unstale":"2023-11-24T07:58:42.334492+0000","last_undegraded":"2023-11-24T07:58:42.334492+0000","last_fullsized":"2023-11-24T07:58:42.334492+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:58:40.368784+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:58:40.368784+0000","last_clean_scrub_stamp":"2023-11-24T07:58:40.368784+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T15:33:45.482832+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.3a","version":"0'0","reported_seq":11,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-11-24T07:58:42.333397+0000","last_change":"2023-11-24T07:58:41.344696+0000","last_active":"2023-11-24T07:58:42.333397+0000","last_peered":"2023-11-24T07:58:42.333397+0000","last_clean":"2023-11-24T07:58:42.333397+0000","last_became_active":"2023-11-24T07:58:41.344413+0000","last_became_peered":"2023-11-24T07:58:41.344413+0000","last_unstale":"2023-11-24T07:58:42.333397+0000","last_undegraded":"2023-11-24T07:58:42.333397+0000","last_fullsized":"2023-11-24T07:58:42.333397+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:58:40.368784+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:58:40.368784+0000","last_clean_scrub_stamp":"2023-11-24T07:58:40.368784+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T12:06:30.664094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.39","version":"0'0","reported_seq":9,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-11-24T07:58:41.355139+0000","last_change":"2023-11-24T07:58:41.350288+0000","last_active":"2023-11-24T07:58:41.355139+0000","last_peered":"2023-11-24T07:58:41.355139+0000","last_clean":"2023-11-24T07:58:41.355139+0000","last_became_active":"2023-11-24T07:58:41.349842+0000","last_became_peered":"2023-11-24T07:58:41.349842+0000","last_unstale":"2023-11-24T07:58:41.355139+0000","last_undegraded":"2023-11-24T07:58:41.355139+0000","last_fullsized":"2023-11-24T07:58:41.355139+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:58:40.368784+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:58:40.368784+0000","last_clean_scrub_stamp":"2023-11-24T07:58:40.368784+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T13:24:39.680103+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.38","version":"0'0","reported_seq":11,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-11-24T07:58:42.332886+0000","last_change":"2023-11-24T07:58:41.341208+0000","last_active":"2023-11-24T07:58:42.332886+0000","last_peered":"2023-11-24T07:58:42.332886+0000","last_clean":"2023-11-24T07:58:42.332886+0000","last_became_active":"2023-11-24T07:58:41.340907+0000","last_became_peered":"2023-11-24T07:58:41.340907+0000","last_unstale":"2023-11-24T07:58:42.332886+0000","last_undegraded":"2023-11-24T07:58:42.332886+0000","last_fullsized":"2023-11-24T07:58:42.332886+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:58:40.368784+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:58:40.368784+0000","last_clean_scrub_stamp":"2023-11-24T07:58:40.368784+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T12:40:58.256501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.37","version":"0'0","reported_seq":11,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-11-24T07:58:42.332999+0000","last_change":"2023-11-24T07:58:41.342031+0000","last_active":"2023-11-24T07:58:42.332999+0000","last_peered":"2023-11-24T07:58:42.332999+0000","last_clean":"2023-11-24T07:58:42.332999+0000","last_became_active":"2023-11-24T07:58:41.341735+0000","last_became_peered":"2023-11-24T07:58:41.341735+0000","last_unstale":"2023-11-24T07:58:42.332999+0000","last_undegraded":"2023-11-24T07:58:42.332999+0000","last_fullsized":"2023-11-24T07:58:42.332999+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:58:40.368784+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:58:40.368784+0000","last_clean_scrub_stamp":"2023-11-24T07:58:40.368784+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T16:30:56.516447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.36","version":"0'0","reported_seq":11,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-11-24T07:58:42.334176+0000","last_change":"2023-11-24T07:58:41.345198+0000","last_active":"2023-11-24T07:58:42.334176+0000","last_peered":"2023-11-24T07:58:42.334176+0000","last_clean":"2023-11-24T07:58:42.334176+0000","last_became_active":"2023-11-24T07:58:41.344929+0000","last_became_peered":"2023-11-24T07:58:41.344929+0000","last_unstale":"2023-11-24T07:58:42.334176+0000","last_undegraded":"2023-11-24T07:58:42.334176+0000","last_fullsized":"2023-11-24T07:58:42.334176+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:58:40.368784+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:58:40.368784+0000","last_clean_scrub_stamp":"2023-11-24T07:58:40.368784+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T18:35:26.382371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.35","version":"0'0","reported_seq":9,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-11-24T07:58:41.350866+0000","last_change":"2023-11-24T07:58:41.345011+0000","last_active":"2023-11-24T07:58:41.350866+0000","last_peered":"2023-11-24T07:58:41.350866+0000","last_clean":"2023-11-24T07:58:41.350866+0000","last_became_active":"2023-11-24T07:58:41.344754+0000","last_became_peered":"2023-11-24T07:58:41.344754+0000","last_unstale":"2023-11-24T07:58:41.350866+0000","last_undegraded":"2023-11-24T07:58:41.350866+0000","last_fullsized":"2023-11-24T07:58:41.350866+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:58:40.368784+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:58:40.368784+0000","last_clean_scrub_stamp":"2023-11-24T07:58:40.368784+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T17:33:41.509928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.34","version":"0'0","reported_seq":11,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-11-24T07:58:42.332816+0000","last_change":"2023-11-24T07:58:41.345098+0000","last_active":"2023-11-24T07:58:42.332816+0000","last_peered":"2023-11-24T07:58:42.332816+0000","last_clean":"2023-11-24T07:58:42.332816+0000","last_became_active":"2023-11-24T07:58:41.344675+0000","last_became_peered":"2023-11-24T07:58:41.344675+0000","last_unstale":"2023-11-24T07:58:42.332816+0000","last_undegraded":"2023-11-24T07:58:42.332816+0000","last_fullsized":"2023-11-24T07:58:42.332816+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:58:40.368784+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:58:40.368784+0000","last_clean_scrub_stamp":"2023-11-24T07:58:40.368784+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:03:24.960498+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.33","version":"0'0","reported_seq":11,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-11-24T07:58:43.107677+0000","last_change":"2023-11-24T07:58:41.350526+0000","last_active":"2023-11-24T07:58:43.107677+0000","last_peered":"2023-11-24T07:58:43.107677+0000","last_clean":"2023-11-24T07:58:43.107677+0000","last_became_active":"2023-11-24T07:58:41.350256+0000","last_became_peered":"2023-11-24T07:58:41.350256+0000","last_unstale":"2023-11-24T07:58:43.107677+0000","last_undegraded":"2023-11-24T07:58:43.107677+0000","last_fullsized":"2023-11-24T07:58:43.107677+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:58:40.368784+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:58:40.368784+0000","last_clean_scrub_stamp":"2023-11-24T07:58:40.368784+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T17:33:41.509928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.32","version":"0'0","reported_seq":9,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-11-24T07:58:41.349276+0000","last_change":"2023-11-24T07:58:41.344005+0000","last_active":"2023-11-24T07:58:41.349276+0000","last_peered":"2023-11-24T07:58:41.349276+0000","last_clean":"2023-11-24T07:58:41.349276+0000","last_became_active":"2023-11-24T07:58:41.343552+0000","last_became_peered":"2023-11-24T07:58:41.343552+0000","last_unstale":"2023-11-24T07:58:41.349276+0000","last_undegraded":"2023-11-24T07:58:41.349276+0000","last_fullsized":"2023-11-24T07:58:41.349276+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:58:40.368784+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:58:40.368784+0000","last_clean_scrub_stamp":"2023-11-24T07:58:40.368784+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T13:30:24.650752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.31","version":"0'0","reported_seq":11,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-11-24T07:58:43.107638+0000","last_change":"2023-11-24T07:58:41.351580+0000","last_active":"2023-11-24T07:58:43.107638+0000","last_peered":"2023-11-24T07:58:43.107638+0000","last_clean":"2023-11-24T07:58:43.107638+0000","last_became_active":"2023-11-24T07:58:41.351264+0000","last_became_peered":"2023-11-24T07:58:41.351264+0000","last_unstale":"2023-11-24T07:58:43.107638+0000","last_undegraded":"2023-11-24T07:58:43.107638+0000","last_fullsized":"2023-11-24T07:58:43.107638+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:58:40.368784+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:58:40.368784+0000","last_clean_scrub_stamp":"2023-11-24T07:58:40.368784+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T08:25:53.941316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.30","version":"0'0","reported_seq":11,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-11-24T07:58:42.334434+0000","last_change":"2023-11-24T07:58:41.346408+0000","last_active":"2023-11-24T07:58:42.334434+0000","last_peered":"2023-11-24T07:58:42.334434+0000","last_clean":"2023-11-24T07:58:42.334434+0000","last_became_active":"2023-11-24T07:58:41.346120+0000","last_became_peered":"2023-11-24T07:58:41.346120+0000","last_unstale":"2023-11-24T07:58:42.334434+0000","last_undegraded":"2023-11-24T07:58:42.334434+0000","last_fullsized":"2023-11-24T07:58:42.334434+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:58:40.368784+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:58:40.368784+0000","last_clean_scrub_stamp":"2023-11-24T07:58:40.368784+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T14:26:52.323467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.2f","version":"0'0","reported_seq":11,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-11-24T07:58:42.333945+0000","last_change":"2023-11-24T07:58:41.348220+0000","last_active":"2023-11-24T07:58:42.333945+0000","last_peered":"2023-11-24T07:58:42.333945+0000","last_clean":"2023-11-24T07:58:42.333945+0000","last_became_active":"2023-11-24T07:58:41.347912+0000","last_became_peered":"2023-11-24T07:58:41.347912+0000","last_unstale":"2023-11-24T07:58:42.333945+0000","last_undegraded":"2023-11-24T07:58:42.333945+0000","last_fullsized":"2023-11-24T07:58:42.333945+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:58:40.368784+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:58:40.368784+0000","last_clean_scrub_stamp":"2023-11-24T07:58:40.368784+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T15:41:29.374563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.2e","version":"0'0","reported_seq":9,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-11-24T07:58:41.354998+0000","last_change":"2023-11-24T07:58:41.351780+0000","last_active":"2023-11-24T07:58:41.354998+0000","last_peered":"2023-11-24T07:58:41.354998+0000","last_clean":"2023-11-24T07:58:41.354998+0000","last_became_active":"2023-11-24T07:58:41.348144+0000","last_became_peered":"2023-11-24T07:58:41.348144+0000","last_unstale":"2023-11-24T07:58:41.354998+0000","last_undegraded":"2023-11-24T07:58:41.354998+0000","last_fullsized":"2023-11-24T07:58:41.354998+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:58:40.368784+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:58:40.368784+0000","last_clean_scrub_stamp":"2023-11-24T07:58:40.368784+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T18:40:41.517193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.2d","version":"0'0","reported_seq":9,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-11-24T07:58:41.351097+0000","last_change":"2023-11-24T07:58:41.347720+0000","last_active":"2023-11-24T07:58:41.351097+0000","last_peered":"2023-11-24T07:58:41.351097+0000","last_clean":"2023-11-24T07:58:41.351097+0000","last_became_active":"2023-11-24T07:58:41.347163+0000","last_became_peered":"2023-11-24T07:58:41.347163+0000","last_unstale":"2023-11-24T07:58:41.351097+0000","last_undegraded":"2023-11-24T07:58:41.351097+0000","last_fullsized":"2023-11-24T07:58:41.351097+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:58:40.368784+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:58:40.368784+0000","last_clean_scrub_stamp":"2023-11-24T07:58:40.368784+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T15:14:15.802031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.2c","version":"0'0","reported_seq":9,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-11-24T07:58:41.350914+0000","last_change":"2023-11-24T07:58:41.345267+0000","last_active":"2023-11-24T07:58:41.350914+0000","last_peered":"2023-11-24T07:58:41.350914+0000","last_clean":"2023-11-24T07:58:41.350914+0000","last_became_active":"2023-11-24T07:58:41.344947+0000","last_became_peered":"2023-11-24T07:58:41.344947+0000","last_unstale":"2023-11-24T07:58:41.350914+0000","last_undegraded":"2023-11-24T07:58:41.350914+0000","last_fullsized":"2023-11-24T07:58:41.350914+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:58:40.368784+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:58:40.368784+0000","last_clean_scrub_stamp":"2023-11-24T07:58:40.368784+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T15:41:29.374563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.2b","version":"0'0","reported_seq":11,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-11-24T07:58:43.107675+0000","last_change":"2023-11-24T07:58:41.348998+0000","last_active":"2023-11-24T07:58:43.107675+0000","last_peered":"2023-11-24T07:58:43.107675+0000","last_clean":"2023-11-24T07:58:43.107675+0000","last_became_active":"2023-11-24T07:58:41.348413+0000","last_became_peered":"2023-11-24T07:58:41.348413+0000","last_unstale":"2023-11-24T07:58:43.107675+0000","last_undegraded":"2023-11-24T07:58:43.107675+0000","last_fullsized":"2023-11-24T07:58:43.107675+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:58:40.368784+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:58:40.368784+0000","last_clean_scrub_stamp":"2023-11-24T07:58:40.368784+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T15:01:45.344564+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.2a","version":"0'0","reported_seq":9,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-11-24T07:58:41.351691+0000","last_change":"2023-11-24T07:58:41.349683+0000","last_active":"2023-11-24T07:58:41.351691+0000","last_peered":"2023-11-24T07:58:41.351691+0000","last_clean":"2023-11-24T07:58:41.351691+0000","last_became_active":"2023-11-24T07:58:41.346943+0000","last_became_peered":"2023-11-24T07:58:41.346943+0000","last_unstale":"2023-11-24T07:58:41.351691+0000","last_undegraded":"2023-11-24T07:58:41.351691+0000","last_fullsized":"2023-11-24T07:58:41.351691+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:58:40.368784+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:58:40.368784+0000","last_clean_scrub_stamp":"2023-11-24T07:58:40.368784+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T12:06:30.664094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.29","version":"0'0","reported_seq":9,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-11-24T07:58:41.355164+0000","last_change":"2023-11-24T07:58:41.352644+0000","last_active":"2023-11-24T07:58:41.355164+0000","last_peered":"2023-11-24T07:58:41.355164+0000","last_clean":"2023-11-24T07:58:41.355164+0000","last_became_active":"2023-11-24T07:58:41.352206+0000","last_became_peered":"2023-11-24T07:58:41.352206+0000","last_unstale":"2023-11-24T07:58:41.355164+0000","last_undegraded":"2023-11-24T07:58:41.355164+0000","last_fullsized":"2023-11-24T07:58:41.355164+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:58:40.368784+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:58:40.368784+0000","last_clean_scrub_stamp":"2023-11-24T07:58:40.368784+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T17:41:50.674323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.28","version":"0'0","reported_seq":11,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-11-24T07:58:42.332673+0000","last_change":"2023-11-24T07:58:41.346228+0000","last_active":"2023-11-24T07:58:42.332673+0000","last_peered":"2023-11-24T07:58:42.332673+0000","last_clean":"2023-11-24T07:58:42.332673+0000","last_became_active":"2023-11-24T07:58:41.345944+0000","last_became_peered":"2023-11-24T07:58:41.345944+0000","last_unstale":"2023-11-24T07:58:42.332673+0000","last_undegraded":"2023-11-24T07:58:42.332673+0000","last_fullsized":"2023-11-24T07:58:42.332673+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:58:40.368784+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:58:40.368784+0000","last_clean_scrub_stamp":"2023-11-24T07:58:40.368784+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T19:14:45.552557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.27","version":"0'0","reported_seq":9,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-11-24T07:58:41.357294+0000","last_change":"2023-11-24T07:58:41.352347+0000","last_active":"2023-11-24T07:58:41.357294+0000","last_peered":"2023-11-24T07:58:41.357294+0000","last_clean":"2023-11-24T07:58:41.357294+0000","last_became_active":"2023-11-24T07:58:41.352004+0000","last_became_peered":"2023-11-24T07:58:41.352004+0000","last_unstale":"2023-11-24T07:58:41.357294+0000","last_undegraded":"2023-11-24T07:58:41.357294+0000","last_fullsized":"2023-11-24T07:58:41.357294+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:58:40.368784+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:58:40.368784+0000","last_clean_scrub_stamp":"2023-11-24T07:58:40.368784+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T17:01:14.896627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.26","version":"0'0","reported_seq":11,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-11-24T07:58:43.107906+0000","last_change":"2023-11-24T07:58:41.354622+0000","last_active":"2023-11-24T07:58:43.107906+0000","last_peered":"2023-11-24T07:58:43.107906+0000","last_clean":"2023-11-24T07:58:43.107906+0000","last_became_active":"2023-11-24T07:58:41.354430+0000","last_became_peered":"2023-11-24T07:58:41.354430+0000","last_unstale":"2023-11-24T07:58:43.107906+0000","last_undegraded":"2023-11-24T07:58:43.107906+0000","last_fullsized":"2023-11-24T07:58:43.107906+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:58:40.368784+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:58:40.368784+0000","last_clean_scrub_stamp":"2023-11-24T07:58:40.368784+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T08:12:56.009848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.25","version":"0'0","reported_seq":9,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-11-24T07:58:41.351232+0000","last_change":"2023-11-24T07:58:41.348878+0000","last_active":"2023-11-24T07:58:41.351232+0000","last_peered":"2023-11-24T07:58:41.351232+0000","last_clean":"2023-11-24T07:58:41.351232+0000","last_became_active":"2023-11-24T07:58:41.348571+0000","last_became_peered":"2023-11-24T07:58:41.348571+0000","last_unstale":"2023-11-24T07:58:41.351232+0000","last_undegraded":"2023-11-24T07:58:41.351232+0000","last_fullsized":"2023-11-24T07:58:41.351232+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:58:40.368784+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:58:40.368784+0000","last_clean_scrub_stamp":"2023-11-24T07:58:40.368784+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T18:23:28.546956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.24","version":"0'0","reported_seq":11,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-11-24T07:58:42.333994+0000","last_change":"2023-11-24T07:58:41.345942+0000","last_active":"2023-11-24T07:58:42.333994+0000","last_peered":"2023-11-24T07:58:42.333994+0000","last_clean":"2023-11-24T07:58:42.333994+0000","last_became_active":"2023-11-24T07:58:41.345653+0000","last_became_peered":"2023-11-24T07:58:41.345653+0000","last_unstale":"2023-11-24T07:58:42.333994+0000","last_undegraded":"2023-11-24T07:58:42.333994+0000","last_fullsized":"2023-11-24T07:58:42.333994+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:58:40.368784+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:58:40.368784+0000","last_clean_scrub_stamp":"2023-11-24T07:58:40.368784+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T17:41:50.674323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.23","version":"0'0","reported_seq":11,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-11-24T07:58:42.333184+0000","last_change":"2023-11-24T07:58:41.341276+0000","last_active":"2023-11-24T07:58:42.333184+0000","last_peered":"2023-11-24T07:58:42.333184+0000","last_clean":"2023-11-24T07:58:42.333184+0000","last_became_active":"2023-11-24T07:58:41.340920+0000","last_became_peered":"2023-11-24T07:58:41.340920+0000","last_unstale":"2023-11-24T07:58:42.333184+0000","last_undegraded":"2023-11-24T07:58:42.333184+0000","last_fullsized":"2023-11-24T07:58:42.333184+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:58:40.368784+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:58:40.368784+0000","last_clean_scrub_stamp":"2023-11-24T07:58:40.368784+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T12:32:28.462666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.22","version":"0'0","reported_seq":9,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-11-24T07:58:41.355135+0000","last_change":"2023-11-24T07:58:41.350651+0000","last_active":"2023-11-24T07:58:41.355135+0000","last_peered":"2023-11-24T07:58:41.355135+0000","last_clean":"2023-11-24T07:58:41.355135+0000","last_became_active":"2023-11-24T07:58:41.350193+0000","last_became_peered":"2023-11-24T07:58:41.350193+0000","last_unstale":"2023-11-24T07:58:41.355135+0000","last_undegraded":"2023-11-24T07:58:41.355135+0000","last_fullsized":"2023-11-24T07:58:41.355135+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:58:40.368784+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:58:40.368784+0000","last_clean_scrub_stamp":"2023-11-24T07:58:40.368784+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T17:14:37.101791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.21","version":"0'0","reported_seq":9,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-11-24T07:58:41.351166+0000","last_change":"2023-11-24T07:58:41.344155+0000","last_active":"2023-11-24T07:58:41.351166+0000","last_peered":"2023-11-24T07:58:41.351166+0000","last_clean":"2023-11-24T07:58:41.351166+0000","last_became_active":"2023-11-24T07:58:41.342927+0000","last_became_peered":"2023-11-24T07:58:41.342927+0000","last_unstale":"2023-11-24T07:58:41.351166+0000","last_undegraded":"2023-11-24T07:58:41.351166+0000","last_fullsized":"2023-11-24T07:58:41.351166+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:58:40.368784+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:58:40.368784+0000","last_clean_scrub_stamp":"2023-11-24T07:58:40.368784+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T15:43:45.302210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.20","version":"0'0","reported_seq":9,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-11-24T07:58:41.350923+0000","last_change":"2023-11-24T07:58:41.345769+0000","last_active":"2023-11-24T07:58:41.350923+0000","last_peered":"2023-11-24T07:58:41.350923+0000","last_clean":"2023-11-24T07:58:41.350923+0000","last_became_active":"2023-11-24T07:58:41.345449+0000","last_became_peered":"2023-11-24T07:58:41.345449+0000","last_unstale":"2023-11-24T07:58:41.350923+0000","last_undegraded":"2023-11-24T07:58:41.350923+0000","last_fullsized":"2023-11-24T07:58:41.350923+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:58:40.368784+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:58:40.368784+0000","last_clean_scrub_stamp":"2023-11-24T07:58:40.368784+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T08:25:53.941316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.1e","version":"0'0","reported_seq":11,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-11-24T07:58:42.333779+0000","last_change":"2023-11-24T07:58:41.341578+0000","last_active":"2023-11-24T07:58:42.333779+0000","last_peered":"2023-11-24T07:58:42.333779+0000","last_clean":"2023-11-24T07:58:42.333779+0000","last_became_active":"2023-11-24T07:58:41.341307+0000","last_became_peered":"2023-11-24T07:58:41.341307+0000","last_unstale":"2023-11-24T07:58:42.333779+0000","last_undegraded":"2023-11-24T07:58:42.333779+0000","last_fullsized":"2023-11-24T07:58:42.333779+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:58:40.368784+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:58:40.368784+0000","last_clean_scrub_stamp":"2023-11-24T07:58:40.368784+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T08:25:53.941316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.1f","version":"59'4","reported_seq":15,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-11-24T07:58:41.387993+0000","last_change":"2023-11-24T07:58:40.395356+0000","last_active":"2023-11-24T07:58:41.387993+0000","last_peered":"2023-11-24T07:58:41.387993+0000","last_clean":"2023-11-24T07:58:41.387993+0000","last_became_active":"2023-11-24T07:58:40.395026+0000","last_became_peered":"2023-11-24T07:58:40.395026+0000","last_unstale":"2023-11-24T07:58:41.387993+0000","last_undegraded":"2023-11-24T07:58:41.387993+0000","last_fullsized":"2023-11-24T07:58:41.387993+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:58:39.367089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:58:39.367089+0000","last_clean_scrub_stamp":"2023-11-24T07:58:39.367089+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T13:26:32.521022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.1f","version":"0'0","reported_seq":11,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-11-24T07:58:42.334003+0000","last_change":"2023-11-24T07:58:41.344680+0000","last_active":"2023-11-24T07:58:42.334003+0000","last_peered":"2023-11-24T07:58:42.334003+0000","last_clean":"2023-11-24T07:58:42.334003+0000","last_became_active":"2023-11-24T07:58:41.344366+0000","last_became_peered":"2023-11-24T07:58:41.344366+0000","last_unstale":"2023-11-24T07:58:42.334003+0000","last_undegraded":"2023-11-24T07:58:42.334003+0000","last_fullsized":"2023-11-24T07:58:42.334003+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:58:40.368784+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:58:40.368784+0000","last_clean_scrub_stamp":"2023-11-24T07:58:40.368784+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T08:12:56.009848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.1e","version":"59'1","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-11-24T07:58:41.388732+0000","last_change":"2023-11-24T07:58:40.392247+0000","last_active":"2023-11-24T07:58:41.388732+0000","last_peered":"2023-11-24T07:58:41.388732+0000","last_clean":"2023-11-24T07:58:41.388732+0000","last_became_active":"2023-11-24T07:58:40.391972+0000","last_became_peered":"2023-11-24T07:58:40.391972+0000","last_unstale":"2023-11-24T07:58:41.388732+0000","last_undegraded":"2023-11-24T07:58:41.388732+0000","last_fullsized":"2023-11-24T07:58:41.388732+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:58:39.367089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:58:39.367089+0000","last_clean_scrub_stamp":"2023-11-24T07:58:39.367089+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T08:58:58.167821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.1c","version":"0'0","reported_seq":11,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-11-24T07:58:42.333873+0000","last_change":"2023-11-24T07:58:41.342131+0000","last_active":"2023-11-24T07:58:42.333873+0000","last_peered":"2023-11-24T07:58:42.333873+0000","last_clean":"2023-11-24T07:58:42.333873+0000","last_became_active":"2023-11-24T07:58:41.341885+0000","last_became_peered":"2023-11-24T07:58:41.341885+0000","last_unstale":"2023-11-24T07:58:42.333873+0000","last_undegraded":"2023-11-24T07:58:42.333873+0000","last_fullsized":"2023-11-24T07:58:42.333873+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:58:40.368784+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:58:40.368784+0000","last_clean_scrub_stamp":"2023-11-24T07:58:40.368784+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:16:10.890321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.1d","version":"59'2","reported_seq":14,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-11-24T07:58:42.334194+0000","last_change":"2023-11-24T07:58:40.392392+0000","last_active":"2023-11-24T07:58:42.334194+0000","last_peered":"2023-11-24T07:58:42.334194+0000","last_clean":"2023-11-24T07:58:42.334194+0000","last_became_active":"2023-11-24T07:58:40.389605+0000","last_became_peered":"2023-11-24T07:58:40.389605+0000","last_unstale":"2023-11-24T07:58:42.334194+0000","last_undegraded":"2023-11-24T07:58:42.334194+0000","last_fullsized":"2023-11-24T07:58:42.334194+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:58:39.367089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:58:39.367089+0000","last_clean_scrub_stamp":"2023-11-24T07:58:39.367089+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T18:54:45.011060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.1d","version":"0'0","reported_seq":11,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-11-24T07:58:42.334109+0000","last_change":"2023-11-24T07:58:41.344966+0000","last_active":"2023-11-24T07:58:42.334109+0000","last_peered":"2023-11-24T07:58:42.334109+0000","last_clean":"2023-11-24T07:58:42.334109+0000","last_became_active":"2023-11-24T07:58:41.344669+0000","last_became_peered":"2023-11-24T07:58:41.344669+0000","last_unstale":"2023-11-24T07:58:42.334109+0000","last_undegraded":"2023-11-24T07:58:42.334109+0000","last_fullsized":"2023-11-24T07:58:42.334109+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:58:40.368784+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:58:40.368784+0000","last_clean_scrub_stamp":"2023-11-24T07:58:40.368784+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T13:26:33.522716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.1c","version":"59'2","reported_seq":13,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-11-24T07:58:41.386778+0000","last_change":"2023-11-24T07:58:40.390089+0000","last_active":"2023-11-24T07:58:41.386778+0000","last_peered":"2023-11-24T07:58:41.386778+0000","last_clean":"2023-11-24T07:58:41.386778+0000","last_became_active":"2023-11-24T07:58:40.389684+0000","last_became_peered":"2023-11-24T07:58:40.389684+0000","last_unstale":"2023-11-24T07:58:41.386778+0000","last_undegraded":"2023-11-24T07:58:41.386778+0000","last_fullsized":"2023-11-24T07:58:41.386778+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:58:39.367089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:58:39.367089+0000","last_clean_scrub_stamp":"2023-11-24T07:58:39.367089+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T12:01:24.691976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.1a","version":"0'0","reported_seq":11,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-11-24T07:58:42.333085+0000","last_change":"2023-11-24T07:58:41.342876+0000","last_active":"2023-11-24T07:58:42.333085+0000","last_peered":"2023-11-24T07:58:42.333085+0000","last_clean":"2023-11-24T07:58:42.333085+0000","last_became_active":"2023-11-24T07:58:41.342600+0000","last_became_peered":"2023-11-24T07:58:41.342600+0000","last_unstale":"2023-11-24T07:58:42.333085+0000","last_undegraded":"2023-11-24T07:58:42.333085+0000","last_fullsized":"2023-11-24T07:58:42.333085+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:58:40.368784+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:58:40.368784+0000","last_clean_scrub_stamp":"2023-11-24T07:58:40.368784+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T13:42:50.912064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.1b","version":"59'1","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-11-24T07:58:41.388068+0000","last_change":"2023-11-24T07:58:40.392976+0000","last_active":"2023-11-24T07:58:41.388068+0000","last_peered":"2023-11-24T07:58:41.388068+0000","last_clean":"2023-11-24T07:58:41.388068+0000","last_became_active":"2023-11-24T07:58:40.392662+0000","last_became_peered":"2023-11-24T07:58:40.392662+0000","last_unstale":"2023-11-24T07:58:41.388068+0000","last_undegraded":"2023-11-24T07:58:41.388068+0000","last_fullsized":"2023-11-24T07:58:41.388068+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:58:39.367089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:58:39.367089+0000","last_clean_scrub_stamp":"2023-11-24T07:58:39.367089+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T15:01:44.342870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.1b","version":"0'0","reported_seq":9,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-11-24T07:58:41.351639+0000","last_change":"2023-11-24T07:58:41.349639+0000","last_active":"2023-11-24T07:58:41.351639+0000","last_peered":"2023-11-24T07:58:41.351639+0000","last_clean":"2023-11-24T07:58:41.351639+0000","last_became_active":"2023-11-24T07:58:41.349356+0000","last_became_peered":"2023-11-24T07:58:41.349356+0000","last_unstale":"2023-11-24T07:58:41.351639+0000","last_undegraded":"2023-11-24T07:58:41.351639+0000","last_fullsized":"2023-11-24T07:58:41.351639+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:58:40.368784+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:58:40.368784+0000","last_clean_scrub_stamp":"2023-11-24T07:58:40.368784+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T15:01:45.344564+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.1a","version":"59'1","reported_seq":13,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-11-24T07:58:42.334360+0000","last_change":"2023-11-24T07:58:40.389653+0000","last_active":"2023-11-24T07:58:42.334360+0000","last_peered":"2023-11-24T07:58:42.334360+0000","last_clean":"2023-11-24T07:58:42.334360+0000","last_became_active":"2023-11-24T07:58:40.389104+0000","last_became_peered":"2023-11-24T07:58:40.389104+0000","last_unstale":"2023-11-24T07:58:42.334360+0000","last_undegraded":"2023-11-24T07:58:42.334360+0000","last_fullsized":"2023-11-24T07:58:42.334360+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:58:39.367089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:58:39.367089+0000","last_clean_scrub_stamp":"2023-11-24T07:58:39.367089+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:47:06.454357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.18","version":"0'0","reported_seq":9,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-11-24T07:58:41.350794+0000","last_change":"2023-11-24T07:58:41.347456+0000","last_active":"2023-11-24T07:58:41.350794+0000","last_peered":"2023-11-24T07:58:41.350794+0000","last_clean":"2023-11-24T07:58:41.350794+0000","last_became_active":"2023-11-24T07:58:41.347121+0000","last_became_peered":"2023-11-24T07:58:41.347121+0000","last_unstale":"2023-11-24T07:58:41.350794+0000","last_undegraded":"2023-11-24T07:58:41.350794+0000","last_fullsized":"2023-11-24T07:58:41.350794+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:58:40.368784+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:58:40.368784+0000","last_clean_scrub_stamp":"2023-11-24T07:58:40.368784+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T16:35:45.613491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.19","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-11-24T07:58:42.332763+0000","last_change":"2023-11-24T07:58:40.394195+0000","last_active":"2023-11-24T07:58:42.332763+0000","last_peered":"2023-11-24T07:58:42.332763+0000","last_clean":"2023-11-24T07:58:42.332763+0000","last_became_active":"2023-11-24T07:58:40.393911+0000","last_became_peered":"2023-11-24T07:58:40.393911+0000","last_unstale":"2023-11-24T07:58:42.332763+0000","last_undegraded":"2023-11-24T07:58:42.332763+0000","last_fullsized":"2023-11-24T07:58:42.332763+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:58:39.367089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:58:39.367089+0000","last_clean_scrub_stamp":"2023-11-24T07:58:39.367089+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T16:18:32.625460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.19","version":"0'0","reported_seq":11,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-11-24T07:58:42.333894+0000","last_change":"2023-11-24T07:58:41.350584+0000","last_active":"2023-11-24T07:58:42.333894+0000","last_peered":"2023-11-24T07:58:42.333894+0000","last_clean":"2023-11-24T07:58:42.333894+0000","last_became_active":"2023-11-24T07:58:41.350266+0000","last_became_peered":"2023-11-24T07:58:41.350266+0000","last_unstale":"2023-11-24T07:58:42.333894+0000","last_undegraded":"2023-11-24T07:58:42.333894+0000","last_fullsized":"2023-11-24T07:58:42.333894+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:58:40.368784+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:58:40.368784+0000","last_clean_scrub_stamp":"2023-11-24T07:58:40.368784+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T15:01:45.344564+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.18","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-11-24T07:58:43.107392+0000","last_change":"2023-11-24T07:58:40.392080+0000","last_active":"2023-11-24T07:58:43.107392+0000","last_peered":"2023-11-24T07:58:43.107392+0000","last_clean":"2023-11-24T07:58:43.107392+0000","last_became_active":"2023-11-24T07:58:40.391768+0000","last_became_peered":"2023-11-24T07:58:40.391768+0000","last_unstale":"2023-11-24T07:58:43.107392+0000","last_undegraded":"2023-11-24T07:58:43.107392+0000","last_fullsized":"2023-11-24T07:58:43.107392+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:58:39.367089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:58:39.367089+0000","last_clean_scrub_stamp":"2023-11-24T07:58:39.367089+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T11:47:36.508014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.16","version":"0'0","reported_seq":11,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-11-24T07:58:43.107965+0000","last_change":"2023-11-24T07:58:41.344394+0000","last_active":"2023-11-24T07:58:43.107965+0000","last_peered":"2023-11-24T07:58:43.107965+0000","last_clean":"2023-11-24T07:58:43.107965+0000","last_became_active":"2023-11-24T07:58:41.343961+0000","last_became_peered":"2023-11-24T07:58:41.343961+0000","last_unstale":"2023-11-24T07:58:43.107965+0000","last_undegraded":"2023-11-24T07:58:43.107965+0000","last_fullsized":"2023-11-24T07:58:43.107965+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:58:40.368784+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:58:40.368784+0000","last_clean_scrub_stamp":"2023-11-24T07:58:40.368784+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T13:26:33.522716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.17","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-11-24T07:58:42.333107+0000","last_change":"2023-11-24T07:58:40.395150+0000","last_active":"2023-11-24T07:58:42.333107+0000","last_peered":"2023-11-24T07:58:42.333107+0000","last_clean":"2023-11-24T07:58:42.333107+0000","last_became_active":"2023-11-24T07:58:40.394774+0000","last_became_peered":"2023-11-24T07:58:40.394774+0000","last_unstale":"2023-11-24T07:58:42.333107+0000","last_undegraded":"2023-11-24T07:58:42.333107+0000","last_fullsized":"2023-11-24T07:58:42.333107+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:58:39.367089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:58:39.367089+0000","last_clean_scrub_stamp":"2023-11-24T07:58:39.367089+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T18:10:23.790209+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.17","version":"0'0","reported_seq":9,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-11-24T07:58:41.351132+0000","last_change":"2023-11-24T07:58:41.348333+0000","last_active":"2023-11-24T07:58:41.351132+0000","last_peered":"2023-11-24T07:58:41.351132+0000","last_clean":"2023-11-24T07:58:41.351132+0000","last_became_active":"2023-11-24T07:58:41.348007+0000","last_became_peered":"2023-11-24T07:58:41.348007+0000","last_unstale":"2023-11-24T07:58:41.351132+0000","last_undegraded":"2023-11-24T07:58:41.351132+0000","last_fullsized":"2023-11-24T07:58:41.351132+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:58:40.368784+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:58:40.368784+0000","last_clean_scrub_stamp":"2023-11-24T07:58:40.368784+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T08:25:53.941316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.16","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-11-24T07:58:42.333172+0000","last_change":"2023-11-24T07:58:40.390657+0000","last_active":"2023-11-24T07:58:42.333172+0000","last_peered":"2023-11-24T07:58:42.333172+0000","last_clean":"2023-11-24T07:58:42.333172+0000","last_became_active":"2023-11-24T07:58:40.390017+0000","last_became_peered":"2023-11-24T07:58:40.390017+0000","last_unstale":"2023-11-24T07:58:42.333172+0000","last_undegraded":"2023-11-24T07:58:42.333172+0000","last_fullsized":"2023-11-24T07:58:42.333172+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:58:39.367089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:58:39.367089+0000","last_clean_scrub_stamp":"2023-11-24T07:58:39.367089+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T08:20:14.329086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.14","version":"0'0","reported_seq":11,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-11-24T07:58:42.334015+0000","last_change":"2023-11-24T07:58:41.342321+0000","last_active":"2023-11-24T07:58:42.334015+0000","last_peered":"2023-11-24T07:58:42.334015+0000","last_clean":"2023-11-24T07:58:42.334015+0000","last_became_active":"2023-11-24T07:58:41.341630+0000","last_became_peered":"2023-11-24T07:58:41.341630+0000","last_unstale":"2023-11-24T07:58:42.334015+0000","last_undegraded":"2023-11-24T07:58:42.334015+0000","last_fullsized":"2023-11-24T07:58:42.334015+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:58:40.368784+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:58:40.368784+0000","last_clean_scrub_stamp":"2023-11-24T07:58:40.368784+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T16:36:14.620555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.15","version":"59'3","reported_seq":14,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-11-24T07:58:41.387205+0000","last_change":"2023-11-24T07:58:40.389211+0000","last_active":"2023-11-24T07:58:41.387205+0000","last_peered":"2023-11-24T07:58:41.387205+0000","last_clean":"2023-11-24T07:58:41.387205+0000","last_became_active":"2023-11-24T07:58:40.388804+0000","last_became_peered":"2023-11-24T07:58:40.388804+0000","last_unstale":"2023-11-24T07:58:41.387205+0000","last_undegraded":"2023-11-24T07:58:41.387205+0000","last_fullsized":"2023-11-24T07:58:41.387205+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:58:39.367089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:58:39.367089+0000","last_clean_scrub_stamp":"2023-11-24T07:58:39.367089+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T17:15:19.973003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.15","version":"0'0","reported_seq":9,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-11-24T07:58:41.355212+0000","last_change":"2023-11-24T07:58:41.353703+0000","last_active":"2023-11-24T07:58:41.355212+0000","last_peered":"2023-11-24T07:58:41.355212+0000","last_clean":"2023-11-24T07:58:41.355212+0000","last_became_active":"2023-11-24T07:58:41.353373+0000","last_became_peered":"2023-11-24T07:58:41.353373+0000","last_unstale":"2023-11-24T07:58:41.355212+0000","last_undegraded":"2023-11-24T07:58:41.355212+0000","last_fullsized":"2023-11-24T07:58:41.355212+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:58:40.368784+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:58:40.368784+0000","last_clean_scrub_stamp":"2023-11-24T07:58:40.368784+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:16:10.890321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.14","version":"59'6","reported_seq":18,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-11-24T07:58:43.107924+0000","last_change":"2023-11-24T07:58:40.389832+0000","last_active":"2023-11-24T07:58:43.107924+0000","last_peered":"2023-11-24T07:58:43.107924+0000","last_clean":"2023-11-24T07:58:43.107924+0000","last_became_active":"2023-11-24T07:58:40.389552+0000","last_became_peered":"2023-11-24T07:58:40.389552+0000","last_unstale":"2023-11-24T07:58:43.107924+0000","last_undegraded":"2023-11-24T07:58:43.107924+0000","last_fullsized":"2023-11-24T07:58:43.107924+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:58:39.367089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:58:39.367089+0000","last_clean_scrub_stamp":"2023-11-24T07:58:39.367089+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T13:30:23.649058+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.12","version":"0'0","reported_seq":11,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-11-24T07:58:42.332841+0000","last_change":"2023-11-24T07:58:41.347540+0000","last_active":"2023-11-24T07:58:42.332841+0000","last_peered":"2023-11-24T07:58:42.332841+0000","last_clean":"2023-11-24T07:58:42.332841+0000","last_became_active":"2023-11-24T07:58:41.347259+0000","last_became_peered":"2023-11-24T07:58:41.347259+0000","last_unstale":"2023-11-24T07:58:42.332841+0000","last_undegraded":"2023-11-24T07:58:42.332841+0000","last_fullsized":"2023-11-24T07:58:42.332841+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:58:40.368784+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:58:40.368784+0000","last_clean_scrub_stamp":"2023-11-24T07:58:40.368784+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T08:49:55.010788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.13","version":"59'2","reported_seq":14,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-11-24T07:58:42.333293+0000","last_change":"2023-11-24T07:58:40.390370+0000","last_active":"2023-11-24T07:58:42.333293+0000","last_peered":"2023-11-24T07:58:42.333293+0000","last_clean":"2023-11-24T07:58:42.333293+0000","last_became_active":"2023-11-24T07:58:40.390079+0000","last_became_peered":"2023-11-24T07:58:40.390079+0000","last_unstale":"2023-11-24T07:58:42.333293+0000","last_undegraded":"2023-11-24T07:58:42.333293+0000","last_fullsized":"2023-11-24T07:58:42.333293+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:58:39.367089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:58:39.367089+0000","last_clean_scrub_stamp":"2023-11-24T07:58:39.367089+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:54:22.823103+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.13","version":"0'0","reported_seq":9,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-11-24T07:58:41.347679+0000","last_change":"2023-11-24T07:58:41.342023+0000","last_active":"2023-11-24T07:58:41.347679+0000","last_peered":"2023-11-24T07:58:41.347679+0000","last_clean":"2023-11-24T07:58:41.347679+0000","last_became_active":"2023-11-24T07:58:41.341485+0000","last_became_peered":"2023-11-24T07:58:41.341485+0000","last_unstale":"2023-11-24T07:58:41.347679+0000","last_undegraded":"2023-11-24T07:58:41.347679+0000","last_fullsized":"2023-11-24T07:58:41.347679+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:58:40.368784+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:58:40.368784+0000","last_clean_scrub_stamp":"2023-11-24T07:58:40.368784+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:47:07.456051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.12","version":"0'0","reported_seq":11,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-11-24T07:58:41.346961+0000","last_change":"2023-11-24T07:58:40.391105+0000","last_active":"2023-11-24T07:58:41.346961+0000","last_peered":"2023-11-24T07:58:41.346961+0000","last_clean":"2023-11-24T07:58:41.346961+0000","last_became_active":"2023-11-24T07:58:40.390811+0000","last_became_peered":"2023-11-24T07:58:40.390811+0000","last_unstale":"2023-11-24T07:58:41.346961+0000","last_undegraded":"2023-11-24T07:58:41.346961+0000","last_fullsized":"2023-11-24T07:58:41.346961+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:58:39.367089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:58:39.367089+0000","last_clean_scrub_stamp":"2023-11-24T07:58:39.367089+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T14:07:19.428853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.10","version":"0'0","reported_seq":11,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-11-24T07:58:42.332858+0000","last_change":"2023-11-24T07:58:41.344911+0000","last_active":"2023-11-24T07:58:42.332858+0000","last_peered":"2023-11-24T07:58:42.332858+0000","last_clean":"2023-11-24T07:58:42.332858+0000","last_became_active":"2023-11-24T07:58:41.344579+0000","last_became_peered":"2023-11-24T07:58:41.344579+0000","last_unstale":"2023-11-24T07:58:42.332858+0000","last_undegraded":"2023-11-24T07:58:42.332858+0000","last_fullsized":"2023-11-24T07:58:42.332858+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:58:40.368784+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:58:40.368784+0000","last_clean_scrub_stamp":"2023-11-24T07:58:40.368784+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T13:18:10.742140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.11","version":"0'0","reported_seq":11,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-11-24T07:58:41.345898+0000","last_change":"2023-11-24T07:58:40.396157+0000","last_active":"2023-11-24T07:58:41.345898+0000","last_peered":"2023-11-24T07:58:41.345898+0000","last_clean":"2023-11-24T07:58:41.345898+0000","last_became_active":"2023-11-24T07:58:40.395857+0000","last_became_peered":"2023-11-24T07:58:40.395857+0000","last_unstale":"2023-11-24T07:58:41.345898+0000","last_undegraded":"2023-11-24T07:58:41.345898+0000","last_fullsized":"2023-11-24T07:58:41.345898+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:58:39.367089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:58:39.367089+0000","last_clean_scrub_stamp":"2023-11-24T07:58:39.367089+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T16:50:29.259946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.11","version":"0'0","reported_seq":9,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-11-24T07:58:41.355194+0000","last_change":"2023-11-24T07:58:41.352778+0000","last_active":"2023-11-24T07:58:41.355194+0000","last_peered":"2023-11-24T07:58:41.355194+0000","last_clean":"2023-11-24T07:58:41.355194+0000","last_became_active":"2023-11-24T07:58:41.352465+0000","last_became_peered":"2023-11-24T07:58:41.352465+0000","last_unstale":"2023-11-24T07:58:41.355194+0000","last_undegraded":"2023-11-24T07:58:41.355194+0000","last_fullsized":"2023-11-24T07:58:41.355194+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:58:40.368784+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:58:40.368784+0000","last_clean_scrub_stamp":"2023-11-24T07:58:40.368784+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T16:36:14.620555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.10","version":"59'8","reported_seq":19,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-11-24T07:58:41.390083+0000","last_change":"2023-11-24T07:58:40.390948+0000","last_active":"2023-11-24T07:58:41.390083+0000","last_peered":"2023-11-24T07:58:41.390083+0000","last_clean":"2023-11-24T07:58:41.390083+0000","last_became_active":"2023-11-24T07:58:40.390768+0000","last_became_peered":"2023-11-24T07:58:40.390768+0000","last_unstale":"2023-11-24T07:58:41.390083+0000","last_undegraded":"2023-11-24T07:58:41.390083+0000","last_fullsized":"2023-11-24T07:58:41.390083+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:58:39.367089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:58:39.367089+0000","last_clean_scrub_stamp":"2023-11-24T07:58:39.367089+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:47:08.018319+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.e","version":"0'0","reported_seq":9,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-11-24T07:58:41.351873+0000","last_change":"2023-11-24T07:58:41.348636+0000","last_active":"2023-11-24T07:58:41.351873+0000","last_peered":"2023-11-24T07:58:41.351873+0000","last_clean":"2023-11-24T07:58:41.351873+0000","last_became_active":"2023-11-24T07:58:41.347768+0000","last_became_peered":"2023-11-24T07:58:41.347768+0000","last_unstale":"2023-11-24T07:58:41.351873+0000","last_undegraded":"2023-11-24T07:58:41.351873+0000","last_fullsized":"2023-11-24T07:58:41.351873+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:58:40.368784+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:58:40.368784+0000","last_clean_scrub_stamp":"2023-11-24T07:58:40.368784+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T15:36:54.443304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.f","version":"59'1","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-11-24T07:58:41.387708+0000","last_change":"2023-11-24T07:58:40.394617+0000","last_active":"2023-11-24T07:58:41.387708+0000","last_peered":"2023-11-24T07:58:41.387708+0000","last_clean":"2023-11-24T07:58:41.387708+0000","last_became_active":"2023-11-24T07:58:40.394336+0000","last_became_peered":"2023-11-24T07:58:40.394336+0000","last_unstale":"2023-11-24T07:58:41.387708+0000","last_undegraded":"2023-11-24T07:58:41.387708+0000","last_fullsized":"2023-11-24T07:58:41.387708+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:58:39.367089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:58:39.367089+0000","last_clean_scrub_stamp":"2023-11-24T07:58:39.367089+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T16:18:32.625460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.f","version":"0'0","reported_seq":9,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-11-24T07:58:41.351114+0000","last_change":"2023-11-24T07:58:41.347577+0000","last_active":"2023-11-24T07:58:41.351114+0000","last_peered":"2023-11-24T07:58:41.351114+0000","last_clean":"2023-11-24T07:58:41.351114+0000","last_became_active":"2023-11-24T07:58:41.347105+0000","last_became_peered":"2023-11-24T07:58:41.347105+0000","last_unstale":"2023-11-24T07:58:41.351114+0000","last_undegraded":"2023-11-24T07:58:41.351114+0000","last_fullsized":"2023-11-24T07:58:41.351114+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:58:40.368784+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:58:40.368784+0000","last_clean_scrub_stamp":"2023-11-24T07:58:40.368784+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T12:06:30.664094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.e","version":"59'2","reported_seq":14,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-11-24T07:58:42.333277+0000","last_change":"2023-11-24T07:58:40.390825+0000","last_active":"2023-11-24T07:58:42.333277+0000","last_peered":"2023-11-24T07:58:42.333277+0000","last_clean":"2023-11-24T07:58:42.333277+0000","last_became_active":"2023-11-24T07:58:40.390392+0000","last_became_peered":"2023-11-24T07:58:40.390392+0000","last_unstale":"2023-11-24T07:58:42.333277+0000","last_undegraded":"2023-11-24T07:58:42.333277+0000","last_fullsized":"2023-11-24T07:58:42.333277+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:58:39.367089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:58:39.367089+0000","last_clean_scrub_stamp":"2023-11-24T07:58:39.367089+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T17:11:13.395000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.c","version":"0'0","reported_seq":9,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-11-24T07:58:41.351301+0000","last_change":"2023-11-24T07:58:41.346777+0000","last_active":"2023-11-24T07:58:41.351301+0000","last_peered":"2023-11-24T07:58:41.351301+0000","last_clean":"2023-11-24T07:58:41.351301+0000","last_became_active":"2023-11-24T07:58:41.346435+0000","last_became_peered":"2023-11-24T07:58:41.346435+0000","last_unstale":"2023-11-24T07:58:41.351301+0000","last_undegraded":"2023-11-24T07:58:41.351301+0000","last_fullsized":"2023-11-24T07:58:41.351301+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:58:40.368784+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:58:40.368784+0000","last_clean_scrub_stamp":"2023-11-24T07:58:40.368784+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:53:55.049234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.d","version":"59'1","reported_seq":12,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-11-24T07:58:41.387979+0000","last_change":"2023-11-24T07:58:40.395492+0000","last_active":"2023-11-24T07:58:41.387979+0000","last_peered":"2023-11-24T07:58:41.387979+0000","last_clean":"2023-11-24T07:58:41.387979+0000","last_became_active":"2023-11-24T07:58:40.395197+0000","last_became_peered":"2023-11-24T07:58:40.395197+0000","last_unstale":"2023-11-24T07:58:41.387979+0000","last_undegraded":"2023-11-24T07:58:41.387979+0000","last_fullsized":"2023-11-24T07:58:41.387979+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:58:39.367089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:58:39.367089+0000","last_clean_scrub_stamp":"2023-11-24T07:58:39.367089+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T15:41:28.372869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.d","version":"0'0","reported_seq":9,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-11-24T07:58:41.355191+0000","last_change":"2023-11-24T07:58:41.353502+0000","last_active":"2023-11-24T07:58:41.355191+0000","last_peered":"2023-11-24T07:58:41.355191+0000","last_clean":"2023-11-24T07:58:41.355191+0000","last_became_active":"2023-11-24T07:58:41.353177+0000","last_became_peered":"2023-11-24T07:58:41.353177+0000","last_unstale":"2023-11-24T07:58:41.355191+0000","last_undegraded":"2023-11-24T07:58:41.355191+0000","last_fullsized":"2023-11-24T07:58:41.355191+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:58:40.368784+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:58:40.368784+0000","last_clean_scrub_stamp":"2023-11-24T07:58:40.368784+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T17:19:15.866101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.c","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-11-24T07:58:42.334121+0000","last_change":"2023-11-24T07:58:40.385207+0000","last_active":"2023-11-24T07:58:42.334121+0000","last_peered":"2023-11-24T07:58:42.334121+0000","last_clean":"2023-11-24T07:58:42.334121+0000","last_became_active":"2023-11-24T07:58:40.384936+0000","last_became_peered":"2023-11-24T07:58:40.384936+0000","last_unstale":"2023-11-24T07:58:42.334121+0000","last_undegraded":"2023-11-24T07:58:42.334121+0000","last_fullsized":"2023-11-24T07:58:42.334121+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:58:39.367089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:58:39.367089+0000","last_clean_scrub_stamp":"2023-11-24T07:58:39.367089+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T13:48:18.639688+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.a","version":"0'0","reported_seq":11,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-11-24T07:58:42.332873+0000","last_change":"2023-11-24T07:58:41.352815+0000","last_active":"2023-11-24T07:58:42.332873+0000","last_peered":"2023-11-24T07:58:42.332873+0000","last_clean":"2023-11-24T07:58:42.332873+0000","last_became_active":"2023-11-24T07:58:41.352531+0000","last_became_peered":"2023-11-24T07:58:41.352531+0000","last_unstale":"2023-11-24T07:58:42.332873+0000","last_undegraded":"2023-11-24T07:58:42.332873+0000","last_fullsized":"2023-11-24T07:58:42.332873+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:58:40.368784+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:58:40.368784+0000","last_clean_scrub_stamp":"2023-11-24T07:58:40.368784+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T12:38:47.311808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.b","version":"59'2","reported_seq":13,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-11-24T07:58:41.388853+0000","last_change":"2023-11-24T07:58:40.392826+0000","last_active":"2023-11-24T07:58:41.388853+0000","last_peered":"2023-11-24T07:58:41.388853+0000","last_clean":"2023-11-24T07:58:41.388853+0000","last_became_active":"2023-11-24T07:58:40.392575+0000","last_became_peered":"2023-11-24T07:58:40.392575+0000","last_unstale":"2023-11-24T07:58:41.388853+0000","last_undegraded":"2023-11-24T07:58:41.388853+0000","last_fullsized":"2023-11-24T07:58:41.388853+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:58:39.367089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:58:39.367089+0000","last_clean_scrub_stamp":"2023-11-24T07:58:39.367089+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T08:12:55.008154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.b","version":"0'0","reported_seq":11,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-11-24T07:58:42.333231+0000","last_change":"2023-11-24T07:58:41.347920+0000","last_active":"2023-11-24T07:58:42.333231+0000","last_peered":"2023-11-24T07:58:42.333231+0000","last_clean":"2023-11-24T07:58:42.333231+0000","last_became_active":"2023-11-24T07:58:41.346222+0000","last_became_peered":"2023-11-24T07:58:41.346222+0000","last_unstale":"2023-11-24T07:58:42.333231+0000","last_undegraded":"2023-11-24T07:58:42.333231+0000","last_fullsized":"2023-11-24T07:58:42.333231+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:58:40.368784+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:58:40.368784+0000","last_clean_scrub_stamp":"2023-11-24T07:58:40.368784+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T08:19:12.331002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.a","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-11-24T07:58:42.333574+0000","last_change":"2023-11-24T07:58:40.390723+0000","last_active":"2023-11-24T07:58:42.333574+0000","last_peered":"2023-11-24T07:58:42.333574+0000","last_clean":"2023-11-24T07:58:42.333574+0000","last_became_active":"2023-11-24T07:58:40.390439+0000","last_became_peered":"2023-11-24T07:58:40.390439+0000","last_unstale":"2023-11-24T07:58:42.333574+0000","last_undegraded":"2023-11-24T07:58:42.333574+0000","last_fullsized":"2023-11-24T07:58:42.333574+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:58:39.367089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:58:39.367089+0000","last_clean_scrub_stamp":"2023-11-24T07:58:39.367089+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:02:06.534520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.9","version":"0'0","reported_seq":11,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-11-24T07:58:42.333267+0000","last_change":"2023-11-24T07:58:41.344399+0000","last_active":"2023-11-24T07:58:42.333267+0000","last_peered":"2023-11-24T07:58:42.333267+0000","last_clean":"2023-11-24T07:58:42.333267+0000","last_became_active":"2023-11-24T07:58:41.344105+0000","last_became_peered":"2023-11-24T07:58:41.344105+0000","last_unstale":"2023-11-24T07:58:42.333267+0000","last_undegraded":"2023-11-24T07:58:42.333267+0000","last_fullsized":"2023-11-24T07:58:42.333267+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:58:40.368784+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:58:40.368784+0000","last_clean_scrub_stamp":"2023-11-24T07:58:40.368784+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T18:08:20.595010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.8","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-11-24T07:58:42.332842+0000","last_change":"2023-11-24T07:58:40.398784+0000","last_active":"2023-11-24T07:58:42.332842+0000","last_peered":"2023-11-24T07:58:42.332842+0000","last_clean":"2023-11-24T07:58:42.332842+0000","last_became_active":"2023-11-24T07:58:40.398558+0000","last_became_peered":"2023-11-24T07:58:40.398558+0000","last_unstale":"2023-11-24T07:58:42.332842+0000","last_undegraded":"2023-11-24T07:58:42.332842+0000","last_fullsized":"2023-11-24T07:58:42.332842+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:58:39.367089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:58:39.367089+0000","last_clean_scrub_stamp":"2023-11-24T07:58:39.367089+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T08:58:58.167821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.6","version":"0'0","reported_seq":9,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-11-24T07:58:41.351863+0000","last_change":"2023-11-24T07:58:41.348458+0000","last_active":"2023-11-24T07:58:41.351863+0000","last_peered":"2023-11-24T07:58:41.351863+0000","last_clean":"2023-11-24T07:58:41.351863+0000","last_became_active":"2023-11-24T07:58:41.347771+0000","last_became_peered":"2023-11-24T07:58:41.347771+0000","last_unstale":"2023-11-24T07:58:41.351863+0000","last_undegraded":"2023-11-24T07:58:41.351863+0000","last_fullsized":"2023-11-24T07:58:41.351863+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:58:40.368784+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:58:40.368784+0000","last_clean_scrub_stamp":"2023-11-24T07:58:40.368784+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T12:24:10.051730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.7","version":"59'2","reported_seq":13,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-11-24T07:58:41.387625+0000","last_change":"2023-11-24T07:58:40.391619+0000","last_active":"2023-11-24T07:58:41.387625+0000","last_peered":"2023-11-24T07:58:41.387625+0000","last_clean":"2023-11-24T07:58:41.387625+0000","last_became_active":"2023-11-24T07:58:40.391378+0000","last_became_peered":"2023-11-24T07:58:40.391378+0000","last_unstale":"2023-11-24T07:58:41.387625+0000","last_undegraded":"2023-11-24T07:58:41.387625+0000","last_fullsized":"2023-11-24T07:58:41.387625+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:58:39.367089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:58:39.367089+0000","last_clean_scrub_stamp":"2023-11-24T07:58:39.367089+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T16:56:58.467571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.7","version":"0'0","reported_seq":9,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-11-24T07:58:41.351148+0000","last_change":"2023-11-24T07:58:41.348843+0000","last_active":"2023-11-24T07:58:41.351148+0000","last_peered":"2023-11-24T07:58:41.351148+0000","last_clean":"2023-11-24T07:58:41.351148+0000","last_became_active":"2023-11-24T07:58:41.348229+0000","last_became_peered":"2023-11-24T07:58:41.348229+0000","last_unstale":"2023-11-24T07:58:41.351148+0000","last_undegraded":"2023-11-24T07:58:41.351148+0000","last_fullsized":"2023-11-24T07:58:41.351148+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:58:40.368784+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:58:40.368784+0000","last_clean_scrub_stamp":"2023-11-24T07:58:40.368784+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T12:40:49.007909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.6","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-11-24T07:58:42.334272+0000","last_change":"2023-11-24T07:58:40.389732+0000","last_active":"2023-11-24T07:58:42.334272+0000","last_peered":"2023-11-24T07:58:42.334272+0000","last_clean":"2023-11-24T07:58:42.334272+0000","last_became_active":"2023-11-24T07:58:40.389441+0000","last_became_peered":"2023-11-24T07:58:40.389441+0000","last_unstale":"2023-11-24T07:58:42.334272+0000","last_undegraded":"2023-11-24T07:58:42.334272+0000","last_fullsized":"2023-11-24T07:58:42.334272+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:58:39.367089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:58:39.367089+0000","last_clean_scrub_stamp":"2023-11-24T07:58:39.367089+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T12:24:09.050036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.4","version":"0'0","reported_seq":9,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-11-24T07:58:41.351321+0000","last_change":"2023-11-24T07:58:41.347399+0000","last_active":"2023-11-24T07:58:41.351321+0000","last_peered":"2023-11-24T07:58:41.351321+0000","last_clean":"2023-11-24T07:58:41.351321+0000","last_became_active":"2023-11-24T07:58:41.347054+0000","last_became_peered":"2023-11-24T07:58:41.347054+0000","last_unstale":"2023-11-24T07:58:41.351321+0000","last_undegraded":"2023-11-24T07:58:41.351321+0000","last_fullsized":"2023-11-24T07:58:41.351321+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:58:40.368784+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:58:40.368784+0000","last_clean_scrub_stamp":"2023-11-24T07:58:40.368784+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:37:54.868724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.5","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-11-24T07:58:42.333160+0000","last_change":"2023-11-24T07:58:40.398278+0000","last_active":"2023-11-24T07:58:42.333160+0000","last_peered":"2023-11-24T07:58:42.333160+0000","last_clean":"2023-11-24T07:58:42.333160+0000","last_became_active":"2023-11-24T07:58:40.397929+0000","last_became_peered":"2023-11-24T07:58:40.397929+0000","last_unstale":"2023-11-24T07:58:42.333160+0000","last_undegraded":"2023-11-24T07:58:42.333160+0000","last_fullsized":"2023-11-24T07:58:42.333160+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:58:39.367089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:58:39.367089+0000","last_clean_scrub_stamp":"2023-11-24T07:58:39.367089+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T16:56:58.467571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.5","version":"0'0","reported_seq":11,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-11-24T07:58:42.333485+0000","last_change":"2023-11-24T07:58:41.347031+0000","last_active":"2023-11-24T07:58:42.333485+0000","last_peered":"2023-11-24T07:58:42.333485+0000","last_clean":"2023-11-24T07:58:42.333485+0000","last_became_active":"2023-11-24T07:58:41.345570+0000","last_became_peered":"2023-11-24T07:58:41.345570+0000","last_unstale":"2023-11-24T07:58:42.333485+0000","last_undegraded":"2023-11-24T07:58:42.333485+0000","last_fullsized":"2023-11-24T07:58:42.333485+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:58:40.368784+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:58:40.368784+0000","last_clean_scrub_stamp":"2023-11-24T07:58:40.368784+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T19:01:13.479431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.4","version":"0'0","reported_seq":11,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-11-24T07:58:41.337625+0000","last_change":"2023-11-24T07:58:40.393928+0000","last_active":"2023-11-24T07:58:41.337625+0000","last_peered":"2023-11-24T07:58:41.337625+0000","last_clean":"2023-11-24T07:58:41.337625+0000","last_became_active":"2023-11-24T07:58:40.393493+0000","last_became_peered":"2023-11-24T07:58:40.393493+0000","last_unstale":"2023-11-24T07:58:41.337625+0000","last_undegraded":"2023-11-24T07:58:41.337625+0000","last_fullsized":"2023-11-24T07:58:41.337625+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:58:39.367089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:58:39.367089+0000","last_clean_scrub_stamp":"2023-11-24T07:58:39.367089+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T08:12:55.008154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.2","version":"0'0","reported_seq":11,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-11-24T07:58:42.334240+0000","last_change":"2023-11-24T07:58:41.350597+0000","last_active":"2023-11-24T07:58:42.334240+0000","last_peered":"2023-11-24T07:58:42.334240+0000","last_clean":"2023-11-24T07:58:42.334240+0000","last_became_active":"2023-11-24T07:58:41.350298+0000","last_became_peered":"2023-11-24T07:58:41.350298+0000","last_unstale":"2023-11-24T07:58:42.334240+0000","last_undegraded":"2023-11-24T07:58:42.334240+0000","last_fullsized":"2023-11-24T07:58:42.334240+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:58:40.368784+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:58:40.368784+0000","last_clean_scrub_stamp":"2023-11-24T07:58:40.368784+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T17:15:30.537711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.3","version":"0'0","reported_seq":11,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-11-24T07:58:41.337028+0000","last_change":"2023-11-24T07:58:40.390913+0000","last_active":"2023-11-24T07:58:41.337028+0000","last_peered":"2023-11-24T07:58:41.337028+0000","last_clean":"2023-11-24T07:58:41.337028+0000","last_became_active":"2023-11-24T07:58:40.390543+0000","last_became_peered":"2023-11-24T07:58:40.390543+0000","last_unstale":"2023-11-24T07:58:41.337028+0000","last_undegraded":"2023-11-24T07:58:41.337028+0000","last_fullsized":"2023-11-24T07:58:41.337028+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:58:39.367089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:58:39.367089+0000","last_clean_scrub_stamp":"2023-11-24T07:58:39.367089+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T19:40:59.979474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.3","version":"0'0","reported_seq":11,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-11-24T07:58:43.107891+0000","last_change":"2023-11-24T07:58:41.344241+0000","last_active":"2023-11-24T07:58:43.107891+0000","last_peered":"2023-11-24T07:58:43.107891+0000","last_clean":"2023-11-24T07:58:43.107891+0000","last_became_active":"2023-11-24T07:58:41.343906+0000","last_became_peered":"2023-11-24T07:58:41.343906+0000","last_unstale":"2023-11-24T07:58:43.107891+0000","last_undegraded":"2023-11-24T07:58:43.107891+0000","last_fullsized":"2023-11-24T07:58:43.107891+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:58:40.368784+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:58:40.368784+0000","last_clean_scrub_stamp":"2023-11-24T07:58:40.368784+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T15:41:29.374563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.2","version":"0'0","reported_seq":11,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-11-24T07:58:41.336653+0000","last_change":"2023-11-24T07:58:40.396698+0000","last_active":"2023-11-24T07:58:41.336653+0000","last_peered":"2023-11-24T07:58:41.336653+0000","last_clean":"2023-11-24T07:58:41.336653+0000","last_became_active":"2023-11-24T07:58:40.396381+0000","last_became_peered":"2023-11-24T07:58:40.396381+0000","last_unstale":"2023-11-24T07:58:41.336653+0000","last_undegraded":"2023-11-24T07:58:41.336653+0000","last_fullsized":"2023-11-24T07:58:41.336653+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:58:39.367089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:58:39.367089+0000","last_clean_scrub_stamp":"2023-11-24T07:58:39.367089+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T17:33:40.508233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.0","version":"0'0","reported_seq":11,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-11-24T07:58:42.333376+0000","last_change":"2023-11-24T07:58:41.341803+0000","last_active":"2023-11-24T07:58:42.333376+0000","last_peered":"2023-11-24T07:58:42.333376+0000","last_clean":"2023-11-24T07:58:42.333376+0000","last_became_active":"2023-11-24T07:58:41.341525+0000","last_became_peered":"2023-11-24T07:58:41.341525+0000","last_unstale":"2023-11-24T07:58:42.333376+0000","last_undegraded":"2023-11-24T07:58:42.333376+0000","last_fullsized":"2023-11-24T07:58:42.333376+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:58:40.368784+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:58:40.368784+0000","last_clean_scrub_stamp":"2023-11-24T07:58:40.368784+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T17:14:37.101791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.1","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-11-24T07:58:42.333983+0000","last_change":"2023-11-24T07:58:40.393735+0000","last_active":"2023-11-24T07:58:42.333983+0000","last_peered":"2023-11-24T07:58:42.333983+0000","last_clean":"2023-11-24T07:58:42.333983+0000","last_became_active":"2023-11-24T07:58:40.393406+0000","last_became_peered":"2023-11-24T07:58:40.393406+0000","last_unstale":"2023-11-24T07:58:42.333983+0000","last_undegraded":"2023-11-24T07:58:42.333983+0000","last_fullsized":"2023-11-24T07:58:42.333983+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:58:39.367089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:58:39.367089+0000","last_clean_scrub_stamp":"2023-11-24T07:58:39.367089+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T12:38:46.310114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.1","version":"0'0","reported_seq":9,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-11-24T07:58:41.351195+0000","last_change":"2023-11-24T07:58:41.344358+0000","last_active":"2023-11-24T07:58:41.351195+0000","last_peered":"2023-11-24T07:58:41.351195+0000","last_clean":"2023-11-24T07:58:41.351195+0000","last_became_active":"2023-11-24T07:58:41.343875+0000","last_became_peered":"2023-11-24T07:58:41.343875+0000","last_unstale":"2023-11-24T07:58:41.351195+0000","last_undegraded":"2023-11-24T07:58:41.351195+0000","last_fullsized":"2023-11-24T07:58:41.351195+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:58:40.368784+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:58:40.368784+0000","last_clean_scrub_stamp":"2023-11-24T07:58:40.368784+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T18:51:40.361538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.0","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-11-24T07:58:42.333038+0000","last_change":"2023-11-24T07:58:40.391991+0000","last_active":"2023-11-24T07:58:42.333038+0000","last_peered":"2023-11-24T07:58:42.333038+0000","last_clean":"2023-11-24T07:58:42.333038+0000","last_became_active":"2023-11-24T07:58:40.391729+0000","last_became_peered":"2023-11-24T07:58:40.391729+0000","last_unstale":"2023-11-24T07:58:42.333038+0000","last_undegraded":"2023-11-24T07:58:42.333038+0000","last_fullsized":"2023-11-24T07:58:42.333038+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:58:39.367089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:58:39.367089+0000","last_clean_scrub_stamp":"2023-11-24T07:58:39.367089+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T15:40:07.386082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.8","version":"0'0","reported_seq":11,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-11-24T07:58:42.334508+0000","last_change":"2023-11-24T07:58:41.347848+0000","last_active":"2023-11-24T07:58:42.334508+0000","last_peered":"2023-11-24T07:58:42.334508+0000","last_clean":"2023-11-24T07:58:42.334508+0000","last_became_active":"2023-11-24T07:58:41.347579+0000","last_became_peered":"2023-11-24T07:58:41.347579+0000","last_unstale":"2023-11-24T07:58:42.334508+0000","last_undegraded":"2023-11-24T07:58:42.334508+0000","last_fullsized":"2023-11-24T07:58:42.334508+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:58:40.368784+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:58:40.368784+0000","last_clean_scrub_stamp":"2023-11-24T07:58:40.368784+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T17:19:15.866101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.9","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-11-24T07:58:43.107592+0000","last_change":"2023-11-24T07:58:40.389566+0000","last_active":"2023-11-24T07:58:43.107592+0000","last_peered":"2023-11-24T07:58:43.107592+0000","last_clean":"2023-11-24T07:58:43.107592+0000","last_became_active":"2023-11-24T07:58:40.389289+0000","last_became_peered":"2023-11-24T07:58:40.389289+0000","last_unstale":"2023-11-24T07:58:43.107592+0000","last_undegraded":"2023-11-24T07:58:43.107592+0000","last_fullsized":"2023-11-24T07:58:43.107592+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:58:39.367089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:58:39.367089+0000","last_clean_scrub_stamp":"2023-11-24T07:58:39.367089+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T18:42:48.188373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":199,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-11-24T07:58:41.338592+0000","last_change":"2023-11-24T07:56:19.668905+0000","last_active":"2023-11-24T07:58:41.338592+0000","last_peered":"2023-11-24T07:58:41.338592+0000","last_clean":"2023-11-24T07:58:41.338592+0000","last_became_active":"2023-11-24T07:56:19.668601+0000","last_became_peered":"2023-11-24T07:56:19.668601+0000","last_unstale":"2023-11-24T07:58:41.338592+0000","last_undegraded":"2023-11-24T07:58:41.338592+0000","last_fullsized":"2023-11-24T07:58:41.338592+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:56:18.657844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:56:18.657844+0000","last_clean_scrub_stamp":"2023-11-24T07:56:18.657844+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T11:51:53.828455+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":9,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":8,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574880,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27724,"kb_used_data":700,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344116,"statfs":{"total":96636764160,"available":96608374784,"internally_reserved":0,"allocated":716800,"data_stored":297068,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 07:58:20 2023","interfaces":[{"interface":"back","average":{"1min":0.51400000000000001,"5min":0.54800000000000004,"15min":0.55400000000000005},"min":{"1min":0.309,"5min":0.309,"15min":0.309},"max":{"1min":0.78500000000000003,"5min":1.413,"15min":1.413},"last":0.45200000000000001},{"interface":"front","average":{"1min":0.56299999999999994,"5min":0.55500000000000005,"15min":0.55400000000000005},"min":{"1min":0.436,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":0.77000000000000002,"5min":0.81000000000000005,"15min":0.81000000000000005},"last":0.48099999999999998}]},{"osd":1,"last update":"Fri Nov 24 07:58:20 2023","interfaces":[{"interface":"back","average":{"1min":0.501,"5min":0.77900000000000003,"15min":0.82499999999999996},"min":{"1min":0.32000000000000001,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.69899999999999995,"5min":6.1070000000000002,"15min":6.1070000000000002},"last":0.53900000000000003},{"interface":"front","average":{"1min":0.51300000000000001,"5min":0.56599999999999995,"15min":0.57499999999999996},"min":{"1min":0.40699999999999997,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.73599999999999999,"5min":1.5920000000000001,"15min":1.5920000000000001},"last":0.35199999999999998}]},{"osd":2,"last update":"Fri Nov 24 07:58:20 2023","interfaces":[{"interface":"back","average":{"1min":0.498,"5min":0.60099999999999998,"15min":0.61799999999999999},"min":{"1min":0.35099999999999998,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.71599999999999997,"5min":1.613,"15min":1.613},"last":0.40200000000000002},{"interface":"front","average":{"1min":0.502,"5min":0.73399999999999999,"15min":0.77200000000000002},"min":{"1min":0.34899999999999998,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.71299999999999997,"5min":4.5220000000000002,"15min":4.5220000000000002},"last":0.42299999999999999}]},{"osd":3,"last update":"Fri Nov 24 07:58:20 2023","interfaces":[{"interface":"back","average":{"1min":0.55100000000000005,"5min":0.755,"15min":0.78900000000000003},"min":{"1min":0.41699999999999998,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.754,"5min":4.6760000000000002,"15min":4.6760000000000002},"last":0.46800000000000003},{"interface":"front","average":{"1min":0.60699999999999998,"5min":0.875,"15min":0.92000000000000004},"min":{"1min":0.38500000000000001,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.81599999999999995,"5min":4.6539999999999999,"15min":4.6539999999999999},"last":0.51500000000000001}]},{"osd":4,"last update":"Fri Nov 24 07:58:20 2023","interfaces":[{"interface":"back","average":{"1min":0.54900000000000004,"5min":0.56399999999999995,"15min":0.56599999999999995},"min":{"1min":0.41799999999999998,"5min":0.378,"15min":0.378},"max":{"1min":0.70999999999999996,"5min":0.84699999999999998,"15min":0.84699999999999998},"last":0.374},{"interface":"front","average":{"1min":0.54700000000000004,"5min":0.64300000000000002,"15min":0.65900000000000003},"min":{"1min":0.35199999999999998,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.79900000000000004,"5min":1.629,"15min":1.629},"last":0.57799999999999996}]},{"osd":5,"last update":"Fri Nov 24 07:58:20 2023","interfaces":[{"interface":"back","average":{"1min":0.54200000000000004,"5min":0.54600000000000004,"15min":0.54600000000000004},"min":{"1min":0.41599999999999998,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.67900000000000005,"5min":0.875,"15min":0.875},"last":0.55500000000000005},{"interface":"front","average":{"1min":0.57799999999999996,"5min":0.61399999999999999,"15min":0.62},"min":{"1min":0.36799999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.77700000000000002,"5min":0.90900000000000003,"15min":0.90900000000000003},"last":0.60599999999999998}]},{"osd":6,"last update":"Fri Nov 24 07:58:20 2023","interfaces":[{"interface":"back","average":{"1min":0.59999999999999998,"5min":0.60799999999999998,"15min":0.60999999999999999},"min":{"1min":0.49099999999999999,"5min":0.438,"15min":0.438},"max":{"1min":0.73399999999999999,"5min":0.80800000000000005,"15min":0.80800000000000005},"last":0.59299999999999997},{"interface":"front","average":{"1min":0.63900000000000001,"5min":0.66300000000000003,"15min":0.66700000000000004},"min":{"1min":0.45600000000000002,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":0.79700000000000004,"5min":0.98399999999999999,"15min":0.98399999999999999},"last":0.499}]}]},{"osd":6,"up_from":13,"seq":55834574881,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27736,"kb_used_data":708,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344104,"statfs":{"total":96636764160,"available":96608362496,"internally_reserved":0,"allocated":724992,"data_stored":303298,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 07:58:19 2023","interfaces":[{"interface":"back","average":{"1min":0.53300000000000003,"5min":0.54600000000000004,"15min":0.54800000000000004},"min":{"1min":0.29799999999999999,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":0.93100000000000005,"5min":0.93100000000000005,"15min":0.93100000000000005},"last":0.69099999999999995},{"interface":"front","average":{"1min":0.51600000000000001,"5min":0.51600000000000001,"15min":0.51600000000000001},"min":{"1min":0.311,"5min":0.311,"15min":0.311},"max":{"1min":0.82199999999999995,"5min":0.82199999999999995,"15min":0.82199999999999995},"last":0.497}]},{"osd":1,"last update":"Fri Nov 24 07:58:19 2023","interfaces":[{"interface":"back","average":{"1min":0.51900000000000002,"5min":0.51300000000000001,"15min":0.51200000000000001},"min":{"1min":0.35599999999999998,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.81299999999999994,"5min":0.81299999999999994,"15min":0.81299999999999994},"last":0.443},{"interface":"front","average":{"1min":0.54000000000000004,"5min":0.59799999999999998,"15min":0.60799999999999998},"min":{"1min":0.35999999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.82799999999999996,"5min":1.0069999999999999,"15min":1.0069999999999999},"last":0.64800000000000002}]},{"osd":2,"last update":"Fri Nov 24 07:58:19 2023","interfaces":[{"interface":"back","average":{"1min":0.58399999999999996,"5min":0.57599999999999996,"15min":0.57499999999999996},"min":{"1min":0.34899999999999998,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.90600000000000003,"5min":0.90600000000000003,"15min":0.90600000000000003},"last":0.55200000000000005},{"interface":"front","average":{"1min":0.56499999999999995,"5min":0.58699999999999997,"15min":0.59099999999999997},"min":{"1min":0.34300000000000003,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.84899999999999998,"5min":0.84899999999999998,"15min":0.84899999999999998},"last":0.52500000000000002}]},{"osd":3,"last update":"Fri Nov 24 07:58:19 2023","interfaces":[{"interface":"back","average":{"1min":0.56000000000000005,"5min":0.56599999999999995,"15min":0.56699999999999995},"min":{"1min":0.38400000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.86499999999999999,"5min":0.876,"15min":0.876},"last":0.75700000000000001},{"interface":"front","average":{"1min":0.57699999999999996,"5min":0.56000000000000005,"15min":0.55700000000000005},"min":{"1min":0.39800000000000002,"5min":0.379,"15min":0.379},"max":{"1min":0.876,"5min":0.876,"15min":0.876},"last":0.62}]},{"osd":4,"last update":"Fri Nov 24 07:58:19 2023","interfaces":[{"interface":"back","average":{"1min":0.53100000000000003,"5min":0.51900000000000002,"15min":0.51700000000000002},"min":{"1min":0.25,"5min":0.25,"15min":0.25},"max":{"1min":0.89400000000000002,"5min":0.89400000000000002,"15min":0.89400000000000002},"last":0.73999999999999999},{"interface":"front","average":{"1min":0.58899999999999997,"5min":0.59899999999999998,"15min":0.60099999999999998},"min":{"1min":0.29099999999999998,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":1.044,"5min":1.087,"15min":1.087},"last":0.58299999999999996}]},{"osd":5,"last update":"Fri Nov 24 07:58:19 2023","interfaces":[{"interface":"back","average":{"1min":0.59499999999999997,"5min":0.54300000000000004,"15min":0.53500000000000003},"min":{"1min":0.27800000000000002,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":0.97999999999999998,"5min":0.97999999999999998,"15min":0.97999999999999998},"last":0.67200000000000004},{"interface":"front","average":{"1min":0.61399999999999999,"5min":0.61199999999999999,"15min":0.61199999999999999},"min":{"1min":0.31900000000000001,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":1.0009999999999999,"5min":1.0009999999999999,"15min":1.0009999999999999},"last":0.60299999999999998}]},{"osd":7,"last update":"Fri Nov 24 07:58:19 2023","interfaces":[{"interface":"back","average":{"1min":0.61799999999999999,"5min":0.61799999999999999,"15min":0.61799999999999999},"min":{"1min":0.26500000000000001,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":1.0880000000000001,"5min":1.0880000000000001,"15min":1.0880000000000001},"last":0.72399999999999998},{"interface":"front","average":{"1min":0.65200000000000002,"5min":0.66600000000000004,"15min":0.66800000000000004},"min":{"1min":0.34300000000000003,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.95599999999999996,"5min":0.95599999999999996,"15min":0.95599999999999996},"last":0.71099999999999997}]}]},{"osd":1,"up_from":13,"seq":55834574882,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28316,"kb_used_data":1288,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343524,"statfs":{"total":96636764160,"available":96607768576,"internally_reserved":0,"allocated":1318912,"data_stored":894226,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 07:58:24 2023","interfaces":[{"interface":"back","average":{"1min":0.33000000000000002,"5min":0.35699999999999998,"15min":0.36099999999999999},"min":{"1min":0.249,"5min":0.20200000000000001,"15min":0.20200000000000001},"max":{"1min":0.48799999999999999,"5min":0.70899999999999996,"15min":0.70899999999999996},"last":0.46100000000000002},{"interface":"front","average":{"1min":0.36399999999999999,"5min":0.39100000000000001,"15min":0.39500000000000002},"min":{"1min":0.23599999999999999,"5min":0.214,"15min":0.214},"max":{"1min":0.54700000000000004,"5min":0.82499999999999996,"15min":0.82499999999999996},"last":0.65600000000000003}]},{"osd":2,"last update":"Fri Nov 24 07:58:24 2023","interfaces":[{"interface":"back","average":{"1min":0.312,"5min":0.35999999999999999,"15min":0.36799999999999999},"min":{"1min":0.253,"5min":0.22700000000000001,"15min":0.22700000000000001},"max":{"1min":0.44800000000000001,"5min":0.60999999999999999,"15min":0.60999999999999999},"last":0.53000000000000003},{"interface":"front","average":{"1min":0.36499999999999999,"5min":0.40300000000000002,"15min":0.40899999999999997},"min":{"1min":0.28100000000000003,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":0.47599999999999998,"5min":0.621,"15min":0.621},"last":0.55200000000000005}]},{"osd":3,"last update":"Fri Nov 24 07:58:24 2023","interfaces":[{"interface":"back","average":{"1min":0.40300000000000002,"5min":0.435,"15min":0.441},"min":{"1min":0.31900000000000001,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.59199999999999997,"5min":0.66500000000000004,"15min":0.66500000000000004},"last":0.62},{"interface":"front","average":{"1min":0.38700000000000001,"5min":0.41899999999999998,"15min":0.42499999999999999},"min":{"1min":0.29199999999999998,"5min":0.28499999999999998,"15min":0.28499999999999998},"max":{"1min":0.56200000000000006,"5min":0.76300000000000001,"15min":0.76300000000000001},"last":0.49299999999999999}]},{"osd":4,"last update":"Fri Nov 24 07:58:24 2023","interfaces":[{"interface":"back","average":{"1min":0.56200000000000006,"5min":0.60099999999999998,"15min":0.60699999999999998},"min":{"1min":0.442,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":0.84099999999999997,"5min":0.84099999999999997,"15min":0.84099999999999997},"last":0.83799999999999997},{"interface":"front","average":{"1min":0.61099999999999999,"5min":0.60699999999999998,"15min":0.60599999999999998},"min":{"1min":0.499,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":0.79500000000000004,"5min":0.82199999999999995,"15min":0.82199999999999995},"last":0.68700000000000006}]},{"osd":5,"last update":"Fri Nov 24 07:58:24 2023","interfaces":[{"interface":"back","average":{"1min":0.59199999999999997,"5min":0.69799999999999995,"15min":0.71599999999999997},"min":{"1min":0.439,"5min":0.37,"15min":0.37},"max":{"1min":0.97599999999999998,"5min":2.7269999999999999,"15min":2.7269999999999999},"last":0.71499999999999997},{"interface":"front","average":{"1min":0.57699999999999996,"5min":0.63100000000000001,"15min":0.64000000000000001},"min":{"1min":0.39100000000000001,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.94499999999999995,"5min":0.94499999999999995,"15min":0.94499999999999995},"last":0.78600000000000003}]},{"osd":6,"last update":"Fri Nov 24 07:58:24 2023","interfaces":[{"interface":"back","average":{"1min":0.61199999999999999,"5min":0.64800000000000002,"15min":0.65400000000000003},"min":{"1min":0.47699999999999998,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":0.77500000000000002,"5min":0.98199999999999998,"15min":0.98199999999999998},"last":0.77800000000000002},{"interface":"front","average":{"1min":0.57599999999999996,"5min":0.60799999999999998,"15min":0.61399999999999999},"min":{"1min":0.40600000000000003,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":0.89600000000000002,"5min":0.90700000000000003,"15min":0.90700000000000003},"last":0.91600000000000004}]},{"osd":7,"last update":"Fri Nov 24 07:58:24 2023","interfaces":[{"interface":"back","average":{"1min":0.63100000000000001,"5min":0.63500000000000001,"15min":0.63600000000000001},"min":{"1min":0.43099999999999999,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.79800000000000004,"5min":1.0649999999999999,"15min":1.0649999999999999},"last":0.95599999999999996},{"interface":"front","average":{"1min":0.57099999999999995,"5min":0.65000000000000002,"15min":0.66300000000000003},"min":{"1min":0.45700000000000002,"5min":0.442,"15min":0.442},"max":{"1min":0.71299999999999997,"5min":0.88800000000000001,"15min":0.88800000000000001},"last":0.93300000000000005}]}]},{"osd":0,"up_from":13,"seq":55834574882,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27736,"kb_used_data":716,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344104,"statfs":{"total":96636764160,"available":96608362496,"internally_reserved":0,"allocated":733184,"data_stored":304888,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Nov 24 07:58:20 2023","interfaces":[{"interface":"back","average":{"1min":0.38500000000000001,"5min":0.64700000000000002,"15min":0.69099999999999995},"min":{"1min":0.21299999999999999,"5min":0.21299999999999999,"15min":0.21299999999999999},"max":{"1min":0.625,"5min":6.3550000000000004,"15min":6.3550000000000004},"last":0.56100000000000005},{"interface":"front","average":{"1min":0.40799999999999997,"5min":1.464,"15min":1.6399999999999999},"min":{"1min":0.26000000000000001,"5min":0.26000000000000001,"15min":0.26000000000000001},"max":{"1min":0.60999999999999999,"5min":22.916,"15min":22.916},"last":0.59099999999999997}]},{"osd":2,"last update":"Fri Nov 24 07:58:20 2023","interfaces":[{"interface":"back","average":{"1min":0.35299999999999998,"5min":0.63,"15min":0.67600000000000005},"min":{"1min":0.22900000000000001,"5min":0.22800000000000001,"15min":0.22800000000000001},"max":{"1min":0.499,"5min":6.3899999999999997,"15min":6.3899999999999997},"last":0.42299999999999999},{"interface":"front","average":{"1min":0.39500000000000002,"5min":0.92900000000000005,"15min":1.018},"min":{"1min":0.27900000000000003,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":0.60199999999999998,"5min":11.93,"15min":11.93},"last":0.495}]},{"osd":3,"last update":"Fri Nov 24 07:58:20 2023","interfaces":[{"interface":"back","average":{"1min":0.373,"5min":0.40999999999999998,"15min":0.41599999999999998},"min":{"1min":0.23799999999999999,"5min":0.23799999999999999,"15min":0.23799999999999999},"max":{"1min":0.79300000000000004,"5min":1.155,"15min":1.155},"last":0.45500000000000002},{"interface":"front","average":{"1min":0.438,"5min":0.98899999999999999,"15min":1.081},"min":{"1min":0.30199999999999999,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.69699999999999995,"5min":12.116,"15min":12.116},"last":0.51700000000000002}]},{"osd":4,"last update":"Fri Nov 24 07:58:20 2023","interfaces":[{"interface":"back","average":{"1min":0.626,"5min":0.64300000000000002,"15min":0.64600000000000002},"min":{"1min":0.40100000000000002,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":1.044,"5min":1.044,"15min":1.044},"last":0.748},{"interface":"front","average":{"1min":0.64300000000000002,"5min":1.675,"15min":1.847},"min":{"1min":0.35699999999999998,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.93000000000000005,"5min":22.963999999999999,"15min":22.963999999999999},"last":0.84999999999999998}]},{"osd":5,"last update":"Fri Nov 24 07:58:20 2023","interfaces":[{"interface":"back","average":{"1min":0.65200000000000002,"5min":1.1639999999999999,"15min":1.25},"min":{"1min":0.41999999999999998,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.97799999999999998,"5min":12.084,"15min":12.084},"last":0.81999999999999995},{"interface":"front","average":{"1min":0.65800000000000003,"5min":0.67400000000000004,"15min":0.67600000000000005},"min":{"1min":0.39700000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.95499999999999996,"5min":1.073,"15min":1.073},"last":0.875}]},{"osd":6,"last update":"Fri Nov 24 07:58:20 2023","interfaces":[{"interface":"back","average":{"1min":0.65200000000000002,"5min":0.68300000000000005,"15min":0.68799999999999994},"min":{"1min":0.45000000000000001,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":0.94699999999999995,"5min":1.0580000000000001,"15min":1.0580000000000001},"last":0.92400000000000004},{"interface":"front","average":{"1min":0.67000000000000004,"5min":1.1850000000000001,"15min":1.2709999999999999},"min":{"1min":0.46200000000000002,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":0.95899999999999996,"5min":12.045,"15min":12.045},"last":0.78900000000000003}]},{"osd":7,"last update":"Fri Nov 24 07:58:20 2023","interfaces":[{"interface":"back","average":{"1min":0.67000000000000004,"5min":0.65800000000000003,"15min":0.65600000000000003},"min":{"1min":0.45200000000000001,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":0.98299999999999998,"5min":1.038,"15min":1.038},"last":0.82999999999999996},{"interface":"front","average":{"1min":0.67500000000000004,"5min":1.7050000000000001,"15min":1.877},"min":{"1min":0.41299999999999998,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":1.0069999999999999,"5min":22.995000000000001,"15min":22.995000000000001},"last":0.80300000000000005}]}]},{"osd":2,"up_from":13,"seq":55834574882,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27736,"kb_used_data":712,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344104,"statfs":{"total":96636764160,"available":96608362496,"internally_reserved":0,"allocated":729088,"data_stored":298748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 07:58:19 2023","interfaces":[{"interface":"back","average":{"1min":0.36399999999999999,"5min":0.36899999999999999,"15min":0.37},"min":{"1min":0.219,"5min":0.219,"15min":0.219},"max":{"1min":0.55900000000000005,"5min":0.58099999999999996,"15min":0.58099999999999996},"last":0.27100000000000002},{"interface":"front","average":{"1min":0.38900000000000001,"5min":0.39900000000000002,"15min":0.40100000000000002},"min":{"1min":0.29299999999999998,"5min":0.27400000000000002,"15min":0.27400000000000002},"max":{"1min":0.54100000000000004,"5min":0.54100000000000004,"15min":0.54100000000000004},"last":0.40200000000000002}]},{"osd":1,"last update":"Fri Nov 24 07:58:19 2023","interfaces":[{"interface":"back","average":{"1min":0.34699999999999998,"5min":0.371,"15min":0.375},"min":{"1min":0.22,"5min":0.22,"15min":0.22},"max":{"1min":0.54000000000000004,"5min":0.59199999999999997,"15min":0.59199999999999997},"last":0.29899999999999999},{"interface":"front","average":{"1min":0.36599999999999999,"5min":0.40600000000000003,"15min":0.41199999999999998},"min":{"1min":0.28299999999999997,"5min":0.25600000000000001,"15min":0.25600000000000001},"max":{"1min":0.497,"5min":0.627,"15min":0.627},"last":0.36199999999999999}]},{"osd":3,"last update":"Fri Nov 24 07:58:19 2023","interfaces":[{"interface":"back","average":{"1min":0.40500000000000003,"5min":0.40899999999999997,"15min":0.40999999999999998},"min":{"1min":0.30399999999999999,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":0.64200000000000002,"5min":0.64200000000000002,"15min":0.64200000000000002},"last":0.33900000000000002},{"interface":"front","average":{"1min":0.41699999999999998,"5min":0.41399999999999998,"15min":0.41399999999999998},"min":{"1min":0.28599999999999998,"5min":0.27500000000000002,"15min":0.27500000000000002},"max":{"1min":0.59299999999999997,"5min":0.64600000000000002,"15min":0.64600000000000002},"last":0.51300000000000001}]},{"osd":4,"last update":"Fri Nov 24 07:58:19 2023","interfaces":[{"interface":"back","average":{"1min":0.54800000000000004,"5min":0.59599999999999997,"15min":0.60399999999999998},"min":{"1min":0.38700000000000001,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.75,"5min":0.81599999999999995,"15min":0.81599999999999995},"last":0.57799999999999996},{"interface":"front","average":{"1min":0.55500000000000005,"5min":0.59099999999999997,"15min":0.59699999999999998},"min":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":0.83799999999999997,"5min":0.83799999999999997,"15min":0.83799999999999997},"last":0.59199999999999997}]},{"osd":5,"last update":"Fri Nov 24 07:58:19 2023","interfaces":[{"interface":"back","average":{"1min":0.56000000000000005,"5min":0.60799999999999998,"15min":0.61599999999999999},"min":{"1min":0.434,"5min":0.434,"15min":0.434},"max":{"1min":0.73499999999999999,"5min":0.84899999999999998,"15min":0.84899999999999998},"last":0.67200000000000004},{"interface":"front","average":{"1min":0.624,"5min":0.69599999999999995,"15min":0.70799999999999996},"min":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":0.82499999999999996,"5min":2.1699999999999999,"15min":2.1699999999999999},"last":0.46200000000000002}]},{"osd":6,"last update":"Fri Nov 24 07:58:19 2023","interfaces":[{"interface":"back","average":{"1min":0.60499999999999998,"5min":0.65400000000000003,"15min":0.66200000000000003},"min":{"1min":0.42499999999999999,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":0.878,"5min":0.878,"15min":0.878},"last":0.48199999999999998},{"interface":"front","average":{"1min":0.60599999999999998,"5min":0.65800000000000003,"15min":0.66600000000000004},"min":{"1min":0.41899999999999998,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":0.80300000000000005,"5min":0.88500000000000001,"15min":0.88500000000000001},"last":0.61899999999999999}]},{"osd":7,"last update":"Fri Nov 24 07:58:19 2023","interfaces":[{"interface":"back","average":{"1min":0.60399999999999998,"5min":0.65000000000000002,"15min":0.65800000000000003},"min":{"1min":0.441,"5min":0.441,"15min":0.441},"max":{"1min":0.77600000000000002,"5min":0.85999999999999999,"15min":0.85999999999999999},"last":0.64500000000000002},{"interface":"front","average":{"1min":0.623,"5min":0.66800000000000004,"15min":0.67600000000000005},"min":{"1min":0.40000000000000002,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":0.90000000000000002,"5min":0.90000000000000002,"15min":0.90000000000000002},"last":0.502}]}]},{"osd":3,"up_from":13,"seq":55834574882,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27720,"kb_used_data":696,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344120,"statfs":{"total":96636764160,"available":96608378880,"internally_reserved":0,"allocated":712704,"data_stored":297066,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 07:58:22 2023","interfaces":[{"interface":"back","average":{"1min":0.34200000000000003,"5min":0.35299999999999998,"15min":0.35499999999999998},"min":{"1min":0.20899999999999999,"5min":0.20899999999999999,"15min":0.20899999999999999},"max":{"1min":0.439,"5min":0.84799999999999998,"15min":0.84799999999999998},"last":0.22900000000000001},{"interface":"front","average":{"1min":0.32700000000000001,"5min":0.35899999999999999,"15min":0.36399999999999999},"min":{"1min":0.253,"5min":0.23699999999999999,"15min":0.23699999999999999},"max":{"1min":0.42499999999999999,"5min":0.873,"15min":0.873},"last":0.32900000000000001}]},{"osd":1,"last update":"Fri Nov 24 07:58:22 2023","interfaces":[{"interface":"back","average":{"1min":0.35899999999999999,"5min":0.371,"15min":0.373},"min":{"1min":0.29499999999999998,"5min":0.25700000000000001,"15min":0.25700000000000001},"max":{"1min":0.443,"5min":0.92000000000000004,"15min":0.92000000000000004},"last":0.28799999999999998},{"interface":"front","average":{"1min":0.35099999999999998,"5min":0.379,"15min":0.38300000000000001},"min":{"1min":0.28299999999999997,"5min":0.248,"15min":0.248},"max":{"1min":0.46700000000000003,"5min":0.81699999999999995,"15min":0.81699999999999995},"last":0.39900000000000002}]},{"osd":2,"last update":"Fri Nov 24 07:58:22 2023","interfaces":[{"interface":"back","average":{"1min":0.33400000000000002,"5min":0.35499999999999998,"15min":0.35899999999999999},"min":{"1min":0.255,"5min":0.24399999999999999,"15min":0.24399999999999999},"max":{"1min":0.43099999999999999,"5min":0.88500000000000001,"15min":0.88500000000000001},"last":0.35399999999999998},{"interface":"front","average":{"1min":0.39500000000000002,"5min":0.45300000000000001,"15min":0.46300000000000002},"min":{"1min":0.311,"5min":0.311,"15min":0.311},"max":{"1min":0.45800000000000002,"5min":0.98099999999999998,"15min":0.98099999999999998},"last":0.38300000000000001}]},{"osd":4,"last update":"Fri Nov 24 07:58:22 2023","interfaces":[{"interface":"back","average":{"1min":0.57099999999999995,"5min":0.751,"15min":0.78100000000000003},"min":{"1min":0.42099999999999999,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.71999999999999997,"5min":5.7350000000000003,"15min":5.7350000000000003},"last":0.54000000000000004},{"interface":"front","average":{"1min":0.59299999999999997,"5min":0.58599999999999997,"15min":0.58499999999999996},"min":{"1min":0.37,"5min":0.37,"15min":0.37},"max":{"1min":0.77000000000000002,"5min":1.27,"15min":1.27},"last":0.45800000000000002}]},{"osd":5,"last update":"Fri Nov 24 07:58:22 2023","interfaces":[{"interface":"back","average":{"1min":0.56899999999999995,"5min":0.58399999999999996,"15min":0.58599999999999997},"min":{"1min":0.40799999999999997,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":0.82799999999999996,"5min":1.4430000000000001,"15min":1.4430000000000001},"last":0.42899999999999999},{"interface":"front","average":{"1min":0.60699999999999998,"5min":0.623,"15min":0.625},"min":{"1min":0.443,"5min":0.377,"15min":0.377},"max":{"1min":0.79900000000000004,"5min":1.4239999999999999,"15min":1.4239999999999999},"last":0.48699999999999999}]},{"osd":6,"last update":"Fri Nov 24 07:58:22 2023","interfaces":[{"interface":"back","average":{"1min":0.58999999999999997,"5min":0.61399999999999999,"15min":0.61799999999999999},"min":{"1min":0.46200000000000002,"5min":0.434,"15min":0.434},"max":{"1min":0.82599999999999996,"5min":1.355,"15min":1.355},"last":0.52600000000000002},{"interface":"front","average":{"1min":0.61199999999999999,"5min":0.79900000000000004,"15min":0.82999999999999996},"min":{"1min":0.47499999999999998,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.76900000000000002,"5min":5.6479999999999997,"15min":5.6479999999999997},"last":0.44800000000000001}]},{"osd":7,"last update":"Fri Nov 24 07:58:22 2023","interfaces":[{"interface":"back","average":{"1min":0.59199999999999997,"5min":0.65100000000000002,"15min":0.66100000000000003},"min":{"1min":0.48099999999999998,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":0.79600000000000004,"5min":1.51,"15min":1.51},"last":0.502},{"interface":"front","average":{"1min":0.61099999999999999,"5min":0.79500000000000004,"15min":0.82599999999999996},"min":{"1min":0.434,"5min":0.434,"15min":0.434},"max":{"1min":0.75900000000000001,"5min":5.7069999999999999,"15min":5.7069999999999999},"last":0.47999999999999998}]}]},{"osd":4,"up_from":13,"seq":55834574881,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27724,"kb_used_data":700,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344116,"statfs":{"total":96636764160,"available":96608374784,"internally_reserved":0,"allocated":716800,"data_stored":297592,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 07:58:25 2023","interfaces":[{"interface":"back","average":{"1min":0.504,"5min":0.56499999999999995,"15min":0.57499999999999996},"min":{"1min":0.34899999999999998,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.70799999999999996,"5min":0.79300000000000004,"15min":0.79300000000000004},"last":0.48899999999999999},{"interface":"front","average":{"1min":0.502,"5min":0.53400000000000003,"15min":0.54000000000000004},"min":{"1min":0.33600000000000002,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.70199999999999996,"5min":1.0249999999999999,"15min":1.0249999999999999},"last":0.58699999999999997}]},{"osd":1,"last update":"Fri Nov 24 07:58:25 2023","interfaces":[{"interface":"back","average":{"1min":0.52800000000000002,"5min":0.54700000000000004,"15min":0.55000000000000004},"min":{"1min":0.39500000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.73999999999999999,"5min":0.86399999999999999,"15min":0.86399999999999999},"last":0.64300000000000002},{"interface":"front","average":{"1min":0.51900000000000002,"5min":0.54400000000000004,"15min":0.54800000000000004},"min":{"1min":0.374,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.72199999999999998,"5min":0.98699999999999999,"15min":0.98699999999999999},"last":0.60899999999999999}]},{"osd":2,"last update":"Fri Nov 24 07:58:25 2023","interfaces":[{"interface":"back","average":{"1min":0.52000000000000002,"5min":0.55200000000000005,"15min":0.55800000000000005},"min":{"1min":0.38800000000000001,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.66400000000000003,"5min":0.81100000000000005,"15min":0.81100000000000005},"last":0.45900000000000002},{"interface":"front","average":{"1min":0.56899999999999995,"5min":0.60399999999999998,"15min":0.60999999999999999},"min":{"1min":0.40000000000000002,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.75700000000000001,"5min":0.89200000000000002,"15min":0.89200000000000002},"last":0.66000000000000003}]},{"osd":3,"last update":"Fri Nov 24 07:58:25 2023","interfaces":[{"interface":"back","average":{"1min":0.54900000000000004,"5min":0.57699999999999996,"15min":0.58199999999999996},"min":{"1min":0.42899999999999999,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":0.65800000000000003,"5min":0.89700000000000002,"15min":0.89700000000000002},"last":0.51800000000000002},{"interface":"front","average":{"1min":0.57499999999999996,"5min":0.61899999999999999,"15min":0.627},"min":{"1min":0.442,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.72899999999999998,"5min":0.96199999999999997,"15min":0.96199999999999997},"last":0.69099999999999995}]},{"osd":5,"last update":"Fri Nov 24 07:58:25 2023","interfaces":[{"interface":"back","average":{"1min":0.48599999999999999,"5min":0.46100000000000002,"15min":0.45700000000000002},"min":{"1min":0.29499999999999998,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":0.67600000000000005,"5min":0.70499999999999996,"15min":0.70499999999999996},"last":0.54000000000000004},{"interface":"front","average":{"1min":0.59899999999999998,"5min":0.61499999999999999,"15min":0.61699999999999999},"min":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.746,"5min":0.86399999999999999,"15min":0.86399999999999999},"last":0.56399999999999995}]},{"osd":6,"last update":"Fri Nov 24 07:58:25 2023","interfaces":[{"interface":"back","average":{"1min":0.56799999999999995,"5min":0.58199999999999996,"15min":0.58399999999999996},"min":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.69799999999999995,"5min":0.76800000000000002,"15min":0.76800000000000002},"last":0.71699999999999997},{"interface":"front","average":{"1min":0.58999999999999997,"5min":0.60899999999999999,"15min":0.61199999999999999},"min":{"1min":0.46899999999999997,"5min":0.436,"15min":0.436},"max":{"1min":0.77700000000000002,"5min":1.095,"15min":1.095},"last":0.68100000000000005}]},{"osd":7,"last update":"Fri Nov 24 07:58:25 2023","interfaces":[{"interface":"back","average":{"1min":0.56799999999999995,"5min":0.57199999999999995,"15min":0.57199999999999995},"min":{"1min":0.39300000000000002,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.74199999999999999,"5min":0.89300000000000002,"15min":0.89300000000000002},"last":0.626},{"interface":"front","average":{"1min":0.61699999999999999,"5min":0.65300000000000002,"15min":0.65900000000000003},"min":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.79100000000000004,"5min":0.91500000000000004,"15min":0.91500000000000004},"last":0.74299999999999999}]}]},{"osd":5,"up_from":13,"seq":55834574882,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28316,"kb_used_data":1288,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343524,"statfs":{"total":96636764160,"available":96607768576,"internally_reserved":0,"allocated":1318912,"data_stored":893678,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 07:58:20 2023","interfaces":[{"interface":"back","average":{"1min":0.52300000000000002,"5min":0.55400000000000005,"15min":0.55900000000000005},"min":{"1min":0.35299999999999998,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.65800000000000003,"5min":1.0109999999999999,"15min":1.0109999999999999},"last":0.442},{"interface":"front","average":{"1min":0.53200000000000003,"5min":0.746,"15min":0.78200000000000003},"min":{"1min":0.34399999999999997,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.72099999999999997,"5min":5.431,"15min":5.431},"last":0.48299999999999998}]},{"osd":1,"last update":"Fri Nov 24 07:58:20 2023","interfaces":[{"interface":"back","average":{"1min":0.51100000000000001,"5min":0.54900000000000004,"15min":0.55500000000000005},"min":{"1min":0.36399999999999999,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":0.67100000000000004,"5min":1.016,"15min":1.016},"last":0.46400000000000002},{"interface":"front","average":{"1min":0.54600000000000004,"5min":0.56499999999999995,"15min":0.56799999999999995},"min":{"1min":0.28599999999999998,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":0.78300000000000003,"5min":0.96499999999999997,"15min":0.96499999999999997},"last":0.54800000000000004}]},{"osd":2,"last update":"Fri Nov 24 07:58:20 2023","interfaces":[{"interface":"back","average":{"1min":0.54500000000000004,"5min":0.54900000000000004,"15min":0.55000000000000004},"min":{"1min":0.33700000000000002,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.67100000000000004,"5min":1.163,"15min":1.163},"last":0.60499999999999998},{"interface":"front","average":{"1min":0.54500000000000004,"5min":0.61299999999999999,"15min":0.624},"min":{"1min":0.41099999999999998,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.78900000000000003,"5min":1.119,"15min":1.119},"last":0.57699999999999996}]},{"osd":3,"last update":"Fri Nov 24 07:58:20 2023","interfaces":[{"interface":"back","average":{"1min":0.57199999999999995,"5min":0.83799999999999997,"15min":0.88200000000000001},"min":{"1min":0.36899999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.85899999999999999,"5min":5.609,"15min":5.609},"last":0.59299999999999997},{"interface":"front","average":{"1min":0.629,"5min":0.66600000000000004,"15min":0.67200000000000004},"min":{"1min":0.44600000000000001,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":0.76400000000000001,"5min":1.288,"15min":1.288},"last":0.65100000000000002}]},{"osd":4,"last update":"Fri Nov 24 07:58:20 2023","interfaces":[{"interface":"back","average":{"1min":0.51500000000000001,"5min":0.51100000000000001,"15min":0.51000000000000001},"min":{"1min":0.32100000000000001,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.76400000000000001,"5min":0.76400000000000001,"15min":0.76400000000000001},"last":0.79100000000000004},{"interface":"front","average":{"1min":0.58799999999999997,"5min":0.57499999999999996,"15min":0.57299999999999995},"min":{"1min":0.29599999999999999,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":0.92900000000000005,"5min":0.92900000000000005,"15min":0.92900000000000005},"last":0.63700000000000001}]},{"osd":6,"last update":"Fri Nov 24 07:58:20 2023","interfaces":[{"interface":"back","average":{"1min":0.59599999999999997,"5min":0.57599999999999996,"15min":0.57199999999999995},"min":{"1min":0.35199999999999998,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.81799999999999995,"5min":0.89700000000000002,"15min":0.89700000000000002},"last":0.69699999999999995},{"interface":"front","average":{"1min":0.63800000000000001,"5min":0.627,"15min":0.625},"min":{"1min":0.40300000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":1.083,"5min":1.083,"15min":1.083},"last":0.74299999999999999}]},{"osd":7,"last update":"Fri Nov 24 07:58:20 2023","interfaces":[{"interface":"back","average":{"1min":0.66200000000000003,"5min":0.64900000000000002,"15min":0.64700000000000002},"min":{"1min":0.36799999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.90100000000000002,"5min":0.96599999999999997,"15min":0.96599999999999997},"last":0.67500000000000004},{"interface":"front","average":{"1min":0.66800000000000004,"5min":0.65600000000000003,"15min":0.65400000000000003},"min":{"1min":0.42699999999999999,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":0.85599999999999998,"5min":0.94199999999999995,"15min":0.94199999999999995},"last":0.72899999999999998}]}]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1772,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-11-24T07:58:45.940 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2023-11-24T07:58:45.941 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2023-11-24T07:58:46.108 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:46.106+0000 7f6f230e7700 1 Processor -- start 2023-11-24T07:58:46.108 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:46.107+0000 7f6f230e7700 1 -- start start 2023-11-24T07:58:46.112 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:46.107+0000 7f6f230e7700 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f6f1c15e1c0 0x7f6f1c06eef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:46.112 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:46.107+0000 7f6f230e7700 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f6f1c0b0e90 0x7f6f1c06f540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:46.112 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:46.107+0000 7f6f230e7700 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6f1c0d1230 con 0x7f6f1c06fa80 2023-11-24T07:58:46.112 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:46.107+0000 7f6f230e7700 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6f1c06ff10 con 0x7f6f1c0b0e90 2023-11-24T07:58:46.112 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:46.107+0000 7f6f230e7700 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6f1c0d0c70 con 0x7f6f1c15e1c0 2023-11-24T07:58:46.112 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:46.108+0000 7f6f21684700 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7f6f1c06fa80 0x7f6f1c161d90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:57382/0 (socket says 172.21.15.59:57382) 2023-11-24T07:58:46.112 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:46.108+0000 7f6f21684700 1 -- 172.21.15.59:0/2983655316 learned_addr learned my addr 172.21.15.59:0/2983655316 (peer_addr_for_me v1:172.21.15.59:0/0) 2023-11-24T07:58:46.112 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:46.108+0000 7f6f1b7fe700 1 -- 172.21.15.59:0/2983655316 <== mon.0 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3661427981 0 0) 0x7f6f1c0d1230 con 0x7f6f1c06fa80 2023-11-24T07:58:46.113 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:46.108+0000 7f6f20e83700 1 --2- 172.21.15.59:0/2983655316 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f6f1c15e1c0 0x7f6f1c06eef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:46.113 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:46.108+0000 7f6f1bfff700 1 --2- 172.21.15.59:0/2983655316 >> v2:172.21.15.76:3300/0 conn(0x7f6f1c0b0e90 0x7f6f1c06f540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:46.113 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:46.108+0000 7f6f1b7fe700 1 -- 172.21.15.59:0/2983655316 --> v1:172.21.15.59:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6f080035b0 con 0x7f6f1c06fa80 2023-11-24T07:58:46.113 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:46.108+0000 7f6f1b7fe700 1 -- 172.21.15.59:0/2983655316 <== mon.0 v1:172.21.15.59:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4187651907 0 0) 0x7f6f080035b0 con 0x7f6f1c06fa80 2023-11-24T07:58:46.113 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:46.108+0000 7f6f1b7fe700 1 -- 172.21.15.59:0/2983655316 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f6f1c15e1c0 msgr2=0x7f6f1c06eef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T07:58:46.113 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:46.108+0000 7f6f1b7fe700 1 --2- 172.21.15.59:0/2983655316 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f6f1c15e1c0 0x7f6f1c06eef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:46.113 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:46.108+0000 7f6f1b7fe700 1 -- 172.21.15.59:0/2983655316 >> v2:172.21.15.76:3300/0 conn(0x7f6f1c0b0e90 msgr2=0x7f6f1c06f540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T07:58:46.113 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:46.108+0000 7f6f1b7fe700 1 --2- 172.21.15.59:0/2983655316 >> v2:172.21.15.76:3300/0 conn(0x7f6f1c0b0e90 0x7f6f1c06f540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:46.113 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:46.108+0000 7f6f1b7fe700 1 -- 172.21.15.59:0/2983655316 --> v1:172.21.15.59:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6f1c0d0f50 con 0x7f6f1c06fa80 2023-11-24T07:58:46.113 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:46.108+0000 7f6f1b7fe700 1 -- 172.21.15.59:0/2983655316 <== mon.0 v1:172.21.15.59:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1312333566 0 0) 0x7f6f0c001670 con 0x7f6f1c06fa80 2023-11-24T07:58:46.113 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:46.108+0000 7f6f1b7fe700 1 -- 172.21.15.59:0/2983655316 >> v1:172.21.15.59:6789/0 conn(0x7f6f1c06fa80 legacy=0x7f6f1c161d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:46.113 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:46.108+0000 7f6f1b7fe700 1 --2- 172.21.15.59:0/2983655316 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6f08003a00 0x7f6f08003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:46.114 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:46.108+0000 7f6f1b7fe700 1 --2- 172.21.15.59:0/2983655316 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f6f1c15e1c0 0x7f6f08004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:46.114 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:46.108+0000 7f6f1b7fe700 1 --2- 172.21.15.59:0/2983655316 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6f08004d90 0x7f6f08005210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:46.114 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:46.108+0000 7f6f1b7fe700 1 -- 172.21.15.59:0/2983655316 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f1c06ff10 con 0x7f6f08004d90 2023-11-24T07:58:46.114 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:46.108+0000 7f6f1b7fe700 1 -- 172.21.15.59:0/2983655316 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f1c0d0c70 con 0x7f6f08003a00 2023-11-24T07:58:46.114 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:46.108+0000 7f6f1b7fe700 1 -- 172.21.15.59:0/2983655316 --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6f0800ba10 con 0x7f6f1c15e1c0 2023-11-24T07:58:46.114 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:46.108+0000 7f6f21684700 1 --2- 172.21.15.59:0/2983655316 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6f08004d90 0x7f6f08005210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:46.114 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:46.108+0000 7f6f20e83700 1 --2- 172.21.15.59:0/2983655316 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6f08003a00 0x7f6f08003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:46.114 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:46.108+0000 7f6f1bfff700 1 --2- 172.21.15.59:0/2983655316 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f6f1c15e1c0 0x7f6f08004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:46.114 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:46.109+0000 7f6f21684700 1 -- 172.21.15.59:0/2983655316 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f6f1c15e1c0 msgr2=0x7f6f08004850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T07:58:46.115 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:46.109+0000 7f6f21684700 1 --2- 172.21.15.59:0/2983655316 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f6f1c15e1c0 0x7f6f08004850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:46.115 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:46.109+0000 7f6f21684700 1 -- 172.21.15.59:0/2983655316 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6f08003a00 msgr2=0x7f6f08003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T07:58:46.115 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:46.109+0000 7f6f21684700 1 --2- 172.21.15.59:0/2983655316 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6f08003a00 0x7f6f08003e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:46.115 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:46.109+0000 7f6f21684700 1 -- 172.21.15.59:0/2983655316 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6f080035b0 con 0x7f6f08004d90 2023-11-24T07:58:46.115 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:46.109+0000 7f6f21684700 1 --2- 172.21.15.59:0/2983655316 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6f08004d90 0x7f6f08005210 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 crypto rx=0x7f6f0c00c320 tx=0x7f6f08005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c32f5472a3e8594c server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:58:46.115 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:46.109+0000 7f6f1b7fe700 1 -- 172.21.15.59:0/2983655316 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f0c00ce60 con 0x7f6f08004d90 2023-11-24T07:58:46.115 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:46.109+0000 7f6f1b7fe700 1 -- 172.21.15.59:0/2983655316 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6f0c005960 con 0x7f6f08004d90 2023-11-24T07:58:46.115 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:46.109+0000 7f6f1b7fe700 1 -- 172.21.15.59:0/2983655316 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f0c005cf0 con 0x7f6f08004d90 2023-11-24T07:58:46.115 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:46.109+0000 7f6f230e7700 1 -- 172.21.15.59:0/2983655316 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6f08004d90 msgr2=0x7f6f08005210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:46.115 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:46.109+0000 7f6f230e7700 1 --2- 172.21.15.59:0/2983655316 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6f08004d90 0x7f6f08005210 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 crypto rx=0x7f6f0c00c320 tx=0x7f6f08005880 comp rx=0 tx=0).stop 2023-11-24T07:58:46.115 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:46.109+0000 7f6f21684700 1 -- 172.21.15.59:0/2983655316 reap_dead start 2023-11-24T07:58:46.115 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:46.109+0000 7f6f230e7700 1 -- 172.21.15.59:0/2983655316 shutdown_connections 2023-11-24T07:58:46.116 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:46.109+0000 7f6f230e7700 1 -- 172.21.15.59:0/2983655316 >> 172.21.15.59:0/2983655316 conn(0x7f6f1c0b3720 msgr2=0x7f6f1c0b3b30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:58:46.116 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:46.110+0000 7f6f230e7700 1 -- 172.21.15.59:0/2983655316 shutdown_connections 2023-11-24T07:58:46.116 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:46.110+0000 7f6f230e7700 1 -- 172.21.15.59:0/2983655316 wait complete. 2023-11-24T07:58:46.119 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:46.111+0000 7f6f230e7700 1 Processor -- start 2023-11-24T07:58:46.119 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:46.111+0000 7f6f230e7700 1 -- start start 2023-11-24T07:58:46.119 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:46.112+0000 7f6f230e7700 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f6f1c14d990 0x7f6f1c0b5aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:46.119 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:46.112+0000 7f6f230e7700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6f1c0b5fe0 0x7f6f1c0afff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:46.119 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:46.112+0000 7f6f230e7700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6f1c0b0530 0x7f6f1c0b4810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:46.119 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:46.112+0000 7f6f230e7700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f1c0b09b0 con 0x7f6f1c0b0530 2023-11-24T07:58:46.120 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:46.112+0000 7f6f230e7700 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f1c0b64f0 con 0x7f6f1c0b5fe0 2023-11-24T07:58:46.120 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:46.112+0000 7f6f230e7700 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6f1c0b67d0 con 0x7f6f1c14d990 2023-11-24T07:58:46.120 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:46.112+0000 7f6f21684700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6f1c0b0530 0x7f6f1c0b4810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:46.120 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:46.112+0000 7f6f21684700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6f1c0b0530 0x7f6f1c0b4810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:53922/0 (socket says 172.21.15.59:53922) 2023-11-24T07:58:46.120 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:46.112+0000 7f6f21684700 1 -- 172.21.15.59:0/2358554914 learned_addr learned my addr 172.21.15.59:0/2358554914 (peer_addr_for_me v2:172.21.15.59:0/0) 2023-11-24T07:58:46.120 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:46.112+0000 7f6f20e83700 1 --2- 172.21.15.59:0/2358554914 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f6f1c14d990 0x7f6f1c0b5aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:46.120 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:46.112+0000 7f6f1bfff700 1 --2- 172.21.15.59:0/2358554914 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6f1c0b5fe0 0x7f6f1c0afff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:46.120 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:46.112+0000 7f6f21684700 1 -- 172.21.15.59:0/2358554914 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f6f1c14d990 msgr2=0x7f6f1c0b5aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:46.120 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:46.112+0000 7f6f21684700 1 --2- 172.21.15.59:0/2358554914 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f6f1c14d990 0x7f6f1c0b5aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:46.120 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:46.112+0000 7f6f21684700 1 -- 172.21.15.59:0/2358554914 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6f1c0b5fe0 msgr2=0x7f6f1c0afff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:46.120 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:46.112+0000 7f6f21684700 1 --2- 172.21.15.59:0/2358554914 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6f1c0b5fe0 0x7f6f1c0afff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:46.120 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:46.112+0000 7f6f21684700 1 -- 172.21.15.59:0/2358554914 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6f1c0b6ab0 con 0x7f6f1c0b0530 2023-11-24T07:58:46.121 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:46.112+0000 7f6f21684700 1 --2- 172.21.15.59:0/2358554914 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6f1c0b0530 0x7f6f1c0b4810 secure :-1 s=READY pgs=718 cs=0 l=1 rev1=1 crypto rx=0x7f6f0c011000 tx=0x7f6f0c001a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:58:46.121 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:46.112+0000 7f6f197fa700 1 -- 172.21.15.59:0/2358554914 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f0c006e10 con 0x7f6f1c0b0530 2023-11-24T07:58:46.121 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:46.113+0000 7f6f197fa700 1 -- 172.21.15.59:0/2358554914 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6f0c0072e0 con 0x7f6f1c0b0530 2023-11-24T07:58:46.121 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:46.113+0000 7f6f230e7700 1 -- 172.21.15.59:0/2358554914 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6f1c0b6d90 con 0x7f6f1c0b0530 2023-11-24T07:58:46.121 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:46.113+0000 7f6f197fa700 1 -- 172.21.15.59:0/2358554914 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f0c007670 con 0x7f6f1c0b0530 2023-11-24T07:58:46.121 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:46.113+0000 7f6f230e7700 1 -- 172.21.15.59:0/2358554914 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6f1c1616b0 con 0x7f6f1c0b0530 2023-11-24T07:58:46.121 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:46.114+0000 7f6f230e7700 1 -- 172.21.15.59:0/2358554914 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6f1c0b73e0 con 0x7f6f1c0b0530 2023-11-24T07:58:46.121 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:46.114+0000 7f6f197fa700 1 -- 172.21.15.59:0/2358554914 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 8) v1 ==== 89435+0+0 (secure 0 0 0) 0x7f6f0c019050 con 0x7f6f1c0b0530 2023-11-24T07:58:46.121 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:46.114+0000 7f6f197fa700 1 --2- 172.21.15.59:0/2358554914 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f6efc06b110 0x7f6efc06d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:46.121 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:46.114+0000 7f6f197fa700 1 -- 172.21.15.59:0/2358554914 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(60..60 src has 1..60) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f6f0c0895f0 con 0x7f6f1c0b0530 2023-11-24T07:58:46.122 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:46.115+0000 7f6f20e83700 1 --2- 172.21.15.59:0/2358554914 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f6efc06b110 0x7f6efc06d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:46.122 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:46.115+0000 7f6f20e83700 1 --2- 172.21.15.59:0/2358554914 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f6efc06b110 0x7f6efc06d5d0 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f6f14014450 tx=0x7f6f1400b040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:58:46.122 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:46.119+0000 7f6f197fa700 1 -- 172.21.15.59:0/2358554914 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f6f0c054490 con 0x7f6f1c0b0530 2023-11-24T07:58:46.407 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:46.405+0000 7f6f230e7700 1 -- 172.21.15.59:0/2358554914 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7f6f1c161840 con 0x7f6f1c0b0530 2023-11-24T07:58:46.408 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:46.406+0000 7f6f197fa700 1 -- 172.21.15.59:0/2358554914 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v15) v1 ==== 190+0+147 (secure 0 0 0) 0x7f6f0c057ab0 con 0x7f6f1c0b0530 2023-11-24T07:58:46.408 INFO:teuthology.orchestra.run.smithi059.stdout:[client.0] 2023-11-24T07:58:46.409 INFO:teuthology.orchestra.run.smithi059.stdout: key = AQD1VmBlxPkpBhAAR841hRsF6QtmzFgJIrVZ9w== 2023-11-24T07:58:46.409 INFO:teuthology.orchestra.run.smithi059.stdout: caps mds = "allow" 2023-11-24T07:58:46.409 INFO:teuthology.orchestra.run.smithi059.stdout: caps mon = "allow r" 2023-11-24T07:58:46.409 INFO:teuthology.orchestra.run.smithi059.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2023-11-24T07:58:46.411 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:46.409+0000 7f6f230e7700 1 -- 172.21.15.59:0/2358554914 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f6efc06b110 msgr2=0x7f6efc06d5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:46.412 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:46.409+0000 7f6f230e7700 1 --2- 172.21.15.59:0/2358554914 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f6efc06b110 0x7f6efc06d5d0 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f6f14014450 tx=0x7f6f1400b040 comp rx=0 tx=0).stop 2023-11-24T07:58:46.412 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:46.409+0000 7f6f230e7700 1 -- 172.21.15.59:0/2358554914 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6f1c0b0530 msgr2=0x7f6f1c0b4810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:46.412 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:46.409+0000 7f6f230e7700 1 --2- 172.21.15.59:0/2358554914 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6f1c0b0530 0x7f6f1c0b4810 secure :-1 s=READY pgs=718 cs=0 l=1 rev1=1 crypto rx=0x7f6f0c011000 tx=0x7f6f0c001a00 comp rx=0 tx=0).stop 2023-11-24T07:58:46.412 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:46.409+0000 7f6f230e7700 1 -- 172.21.15.59:0/2358554914 shutdown_connections 2023-11-24T07:58:46.412 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:46.409+0000 7f6f230e7700 1 --2- 172.21.15.59:0/2358554914 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f6efc06b110 0x7f6efc06d5d0 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:46.412 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:46.409+0000 7f6f230e7700 1 --2- 172.21.15.59:0/2358554914 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6f1c0b0530 0x7f6f1c0b4810 unknown :-1 s=CLOSED pgs=718 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:46.413 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:46.409+0000 7f6f230e7700 1 --2- 172.21.15.59:0/2358554914 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f6f1c14d990 0x7f6f1c0b5aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:46.413 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:46.409+0000 7f6f230e7700 1 --2- 172.21.15.59:0/2358554914 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6f1c0b5fe0 0x7f6f1c0afff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:46.413 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:46.409+0000 7f6f230e7700 1 -- 172.21.15.59:0/2358554914 >> 172.21.15.59:0/2358554914 conn(0x7f6f1c0b3720 msgr2=0x7f6f1c0cf0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:58:46.413 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:46.409+0000 7f6f230e7700 1 -- 172.21.15.59:0/2358554914 shutdown_connections 2023-11-24T07:58:46.413 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:46.409+0000 7f6f230e7700 1 -- 172.21.15.59:0/2358554914 wait complete. 2023-11-24T07:58:46.414 INFO:teuthology.orchestra.run.smithi059.stderr:updated caps for client.0 2023-11-24T07:58:46.423 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-24T07:58:46.548 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:46.547+0000 7f9398a9b700 1 Processor -- start 2023-11-24T07:58:46.549 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:46.547+0000 7f9398a9b700 1 -- start start 2023-11-24T07:58:46.551 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:46.548+0000 7f9398a9b700 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f9394143ae0 0x7f93940b51b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:46.551 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:46.548+0000 7f9398a9b700 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f93941440f0 0x7f9394071bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:46.551 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:46.548+0000 7f9398a9b700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f93940720f0 0x7f939415e8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:46.551 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:46.548+0000 7f9398a9b700 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f93940b40a0 con 0x7f93940720f0 2023-11-24T07:58:46.551 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:46.548+0000 7f9398a9b700 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9394072580 con 0x7f93941440f0 2023-11-24T07:58:46.551 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:46.548+0000 7f9398a9b700 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9394143ec0 con 0x7f9394143ae0 2023-11-24T07:58:46.551 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:46.548+0000 7f9392d9d700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f93940720f0 0x7f939415e8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:46.552 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:46.548+0000 7f9392d9d700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f93940720f0 0x7f939415e8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:53930/0 (socket says 172.21.15.59:53930) 2023-11-24T07:58:46.552 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:46.548+0000 7f9392d9d700 1 -- 172.21.15.59:0/2460576260 learned_addr learned my addr 172.21.15.59:0/2460576260 (peer_addr_for_me v2:172.21.15.59:0/0) 2023-11-24T07:58:46.552 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:46.548+0000 7f9392d9d700 1 -- 172.21.15.59:0/2460576260 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f9394143ae0 msgr2=0x7f93940b51b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T07:58:46.552 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:46.548+0000 7f9392d9d700 1 --2- 172.21.15.59:0/2460576260 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f9394143ae0 0x7f93940b51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:46.552 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:46.548+0000 7f9392d9d700 1 -- 172.21.15.59:0/2460576260 >> v2:172.21.15.76:3300/0 conn(0x7f93941440f0 msgr2=0x7f9394071bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T07:58:46.552 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:46.548+0000 7f9391d9b700 1 --2- 172.21.15.59:0/2460576260 >> v2:172.21.15.76:3300/0 conn(0x7f93941440f0 0x7f9394071bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:46.552 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:46.548+0000 7f9392d9d700 1 --2- 172.21.15.59:0/2460576260 >> v2:172.21.15.76:3300/0 conn(0x7f93941440f0 0x7f9394071bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:46.552 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:46.548+0000 7f9392d9d700 1 -- 172.21.15.59:0/2460576260 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f93940b3ae0 con 0x7f93940720f0 2023-11-24T07:58:46.552 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:46.548+0000 7f9391d9b700 1 --2- 172.21.15.59:0/2460576260 >> v2:172.21.15.76:3300/0 conn(0x7f93941440f0 0x7f9394071bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T07:58:46.552 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:46.548+0000 7f9392d9d700 1 --2- 172.21.15.59:0/2460576260 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f93940720f0 0x7f939415e8b0 secure :-1 s=READY pgs=719 cs=0 l=1 rev1=1 crypto rx=0x7f939406d070 tx=0x7f938c00d430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=84a74547f0d2815 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:58:46.552 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:46.549+0000 7f939159a700 1 -- 172.21.15.59:0/2460576260 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f938c010070 con 0x7f93940720f0 2023-11-24T07:58:46.552 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:46.549+0000 7f939159a700 1 -- 172.21.15.59:0/2460576260 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f938c00dd90 con 0x7f93940720f0 2023-11-24T07:58:46.552 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:46.549+0000 7f939159a700 1 -- 172.21.15.59:0/2460576260 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f938c00b310 con 0x7f93940720f0 2023-11-24T07:58:46.552 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:46.549+0000 7f9398a9b700 1 -- 172.21.15.59:0/2460576260 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f93940720f0 msgr2=0x7f939415e8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:46.553 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:46.549+0000 7f9398a9b700 1 --2- 172.21.15.59:0/2460576260 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f93940720f0 0x7f939415e8b0 secure :-1 s=READY pgs=719 cs=0 l=1 rev1=1 crypto rx=0x7f939406d070 tx=0x7f938c00d430 comp rx=0 tx=0).stop 2023-11-24T07:58:46.553 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:46.549+0000 7f9398a9b700 1 -- 172.21.15.59:0/2460576260 shutdown_connections 2023-11-24T07:58:46.553 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:46.549+0000 7f9398a9b700 1 --2- 172.21.15.59:0/2460576260 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f93940720f0 0x7f939415e8b0 unknown :-1 s=CLOSED pgs=719 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:46.553 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:46.549+0000 7f9398a9b700 1 --2- 172.21.15.59:0/2460576260 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f9394143ae0 0x7f93940b51b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:46.553 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:46.549+0000 7f9398a9b700 1 --2- 172.21.15.59:0/2460576260 >> v2:172.21.15.76:3300/0 conn(0x7f93941440f0 0x7f9394071bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:46.553 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:46.549+0000 7f9398a9b700 1 -- 172.21.15.59:0/2460576260 >> 172.21.15.59:0/2460576260 conn(0x7f93940b2740 msgr2=0x7f93940b2b20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:58:46.553 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:46.549+0000 7f9398a9b700 1 -- 172.21.15.59:0/2460576260 shutdown_connections 2023-11-24T07:58:46.553 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:46.549+0000 7f9398a9b700 1 -- 172.21.15.59:0/2460576260 wait complete. 2023-11-24T07:58:46.555 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:46.551+0000 7f9398a9b700 1 Processor -- start 2023-11-24T07:58:46.556 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:46.551+0000 7f9398a9b700 1 -- start start 2023-11-24T07:58:46.556 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:46.551+0000 7f9398a9b700 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f93940720f0 0x7f9394156a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:46.556 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:46.551+0000 7f9398a9b700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9394143ae0 0x7f9394157e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:46.556 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:46.551+0000 7f9398a9b700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f93941440f0 0x7f939415d910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:46.556 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:46.551+0000 7f9398a9b700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f939415ca80 con 0x7f93941440f0 2023-11-24T07:58:46.556 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:46.551+0000 7f9398a9b700 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93941efa50 con 0x7f9394143ae0 2023-11-24T07:58:46.556 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:46.551+0000 7f9398a9b700 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9394157010 con 0x7f93940720f0 2023-11-24T07:58:46.556 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:46.551+0000 7f9392d9d700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f93941440f0 0x7f939415d910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:46.556 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:46.551+0000 7f9392d9d700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f93941440f0 0x7f939415d910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:53936/0 (socket says 172.21.15.59:53936) 2023-11-24T07:58:46.556 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:46.551+0000 7f9392d9d700 1 -- 172.21.15.59:0/2982146165 learned_addr learned my addr 172.21.15.59:0/2982146165 (peer_addr_for_me v2:172.21.15.59:0/0) 2023-11-24T07:58:46.556 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:46.552+0000 7f9391d9b700 1 --2- 172.21.15.59:0/2982146165 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9394143ae0 0x7f9394157e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:46.556 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:46.552+0000 7f939259c700 1 --2- 172.21.15.59:0/2982146165 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f93940720f0 0x7f9394156a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:46.556 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:46.552+0000 7f9392d9d700 1 -- 172.21.15.59:0/2982146165 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f93940720f0 msgr2=0x7f9394156a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:46.557 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:46.552+0000 7f9392d9d700 1 --2- 172.21.15.59:0/2982146165 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f93940720f0 0x7f9394156a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:46.557 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:46.552+0000 7f9392d9d700 1 -- 172.21.15.59:0/2982146165 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9394143ae0 msgr2=0x7f9394157e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:46.557 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:46.552+0000 7f9392d9d700 1 --2- 172.21.15.59:0/2982146165 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9394143ae0 0x7f9394157e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:46.557 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:46.552+0000 7f9392d9d700 1 -- 172.21.15.59:0/2982146165 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f938c00bd40 con 0x7f93941440f0 2023-11-24T07:58:46.557 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:46.552+0000 7f9392d9d700 1 --2- 172.21.15.59:0/2982146165 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f93941440f0 0x7f939415d910 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 crypto rx=0x7f93940d1bd0 tx=0x7f938c004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:58:46.557 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:46.552+0000 7f9382ffd700 1 -- 172.21.15.59:0/2982146165 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f938c00b4c0 con 0x7f93941440f0 2023-11-24T07:58:46.557 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:46.552+0000 7f9382ffd700 1 -- 172.21.15.59:0/2982146165 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f938c00ae30 con 0x7f93941440f0 2023-11-24T07:58:46.557 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:46.552+0000 7f9398a9b700 1 -- 172.21.15.59:0/2982146165 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f93941572f0 con 0x7f93941440f0 2023-11-24T07:58:46.557 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:46.552+0000 7f9382ffd700 1 -- 172.21.15.59:0/2982146165 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f938c004280 con 0x7f93941440f0 2023-11-24T07:58:46.557 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:46.552+0000 7f9398a9b700 1 -- 172.21.15.59:0/2982146165 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f93941578b0 con 0x7f93941440f0 2023-11-24T07:58:46.557 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:46.553+0000 7f9398a9b700 1 -- 172.21.15.59:0/2982146165 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f93941efc70 con 0x7f93941440f0 2023-11-24T07:58:46.557 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:46.553+0000 7f9382ffd700 1 -- 172.21.15.59:0/2982146165 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 8) v1 ==== 89435+0+0 (secure 0 0 0) 0x7f938c0043e0 con 0x7f93941440f0 2023-11-24T07:58:46.557 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:46.554+0000 7f9382ffd700 1 --2- 172.21.15.59:0/2982146165 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f937406b110 0x7f937406d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:46.557 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:46.554+0000 7f9382ffd700 1 -- 172.21.15.59:0/2982146165 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(60..60 src has 1..60) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f938c08d460 con 0x7f93941440f0 2023-11-24T07:58:46.558 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:46.554+0000 7f939259c700 1 --2- 172.21.15.59:0/2982146165 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f937406b110 0x7f937406d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:46.558 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:46.554+0000 7f939259c700 1 --2- 172.21.15.59:0/2982146165 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f937406b110 0x7f937406d5d0 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f9384006fd0 tx=0x7f9384008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:58:46.560 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:46.558+0000 7f9382ffd700 1 -- 172.21.15.59:0/2982146165 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f93941efc70 con 0x7f93941440f0 2023-11-24T07:58:46.803 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:46.801+0000 7f9398a9b700 1 -- 172.21.15.59:0/2982146165 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f93941efc70 con 0x7f93941440f0 2023-11-24T07:58:46.803 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:46.802+0000 7f9382ffd700 1 -- 172.21.15.59:0/2982146165 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 35 v35) v1 ==== 94+0+4907 (secure 0 0 0) 0x7f938c058130 con 0x7f93941440f0 2023-11-24T07:58:46.804 INFO:teuthology.orchestra.run.smithi059.stdout: 2023-11-24T07:58:46.804 INFO:teuthology.orchestra.run.smithi059.stdout:{"epoch":35,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5300,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6834/172682993","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6833","nonce":172682993},{"type":"v1","addr":"172.21.15.59:6834","nonce":172682993}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":29},{"gid":5701,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.76:6836/923821877","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6834","nonce":923821877},{"type":"v1","addr":"172.21.15.76:6836","nonce":923821877}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":30},{"gid":5704,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.76:6837/4166482716","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6835","nonce":4166482716},{"type":"v1","addr":"172.21.15.76:6837","nonce":4166482716}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":29}],"filesystems":[{"mdsmap":{"epoch":35,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-24T07:58:40.795340+0000","modified":"2023-11-24T07:58:43.812382+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":6093},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6093":{"gid":6093,"name":"a","rank":0,"incarnation":31,"state":"up:active","state_seq":3,"addr":"172.21.15.59:6835/1599845006","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6832","nonce":1599845006},{"type":"v1","addr":"172.21.15.59:6835","nonce":1599845006}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[9],"metadata_pool":8,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":4}]} 2023-11-24T07:58:46.806 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:46.804+0000 7f9398a9b700 1 -- 172.21.15.59:0/2982146165 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f937406b110 msgr2=0x7f937406d5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:46.806 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:46.804+0000 7f9398a9b700 1 --2- 172.21.15.59:0/2982146165 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f937406b110 0x7f937406d5d0 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f9384006fd0 tx=0x7f9384008040 comp rx=0 tx=0).stop 2023-11-24T07:58:46.806 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:46.804+0000 7f9398a9b700 1 -- 172.21.15.59:0/2982146165 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f93941440f0 msgr2=0x7f939415d910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:46.806 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:46.804+0000 7f9398a9b700 1 --2- 172.21.15.59:0/2982146165 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f93941440f0 0x7f939415d910 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 crypto rx=0x7f93940d1bd0 tx=0x7f938c004ad0 comp rx=0 tx=0).stop 2023-11-24T07:58:46.807 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:46.804+0000 7f9398a9b700 1 -- 172.21.15.59:0/2982146165 shutdown_connections 2023-11-24T07:58:46.807 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:46.804+0000 7f9398a9b700 1 --2- 172.21.15.59:0/2982146165 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f937406b110 0x7f937406d5d0 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:46.807 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:46.804+0000 7f9398a9b700 1 --2- 172.21.15.59:0/2982146165 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f93941440f0 0x7f939415d910 unknown :-1 s=CLOSED pgs=720 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:46.807 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:46.804+0000 7f9398a9b700 1 --2- 172.21.15.59:0/2982146165 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f93940720f0 0x7f9394156a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:46.807 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:46.804+0000 7f9398a9b700 1 --2- 172.21.15.59:0/2982146165 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9394143ae0 0x7f9394157e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:46.807 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:46.804+0000 7f9398a9b700 1 -- 172.21.15.59:0/2982146165 >> 172.21.15.59:0/2982146165 conn(0x7f93940b2740 msgr2=0x7f939406e410 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:58:46.807 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:46.804+0000 7f9398a9b700 1 -- 172.21.15.59:0/2982146165 shutdown_connections 2023-11-24T07:58:46.807 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:46.804+0000 7f9398a9b700 1 -- 172.21.15.59:0/2982146165 wait complete. 2023-11-24T07:58:46.807 INFO:teuthology.orchestra.run.smithi059.stderr:dumped fsmap epoch 35 2023-11-24T07:58:47.221 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 35, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-11-24T07:58:40.795340+0000', 'modified': '2023-11-24T07:58:43.812382+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 6093}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_6093': {'gid': 6093, 'name': 'a', 'rank': 0, 'incarnation': 31, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.59:6835/1599845006', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.59:6832', 'nonce': 1599845006}, {'type': 'v1', 'addr': '172.21.15.59:6835', 'nonce': 1599845006}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [9], 'metadata_pool': 8, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2023-11-24T07:58:47.221 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2023-11-24T07:58:47.221 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2023-11-24T07:58:47.354 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:47.352+0000 7f14b7f06700 1 Processor -- start 2023-11-24T07:58:47.354 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:47.352+0000 7f14b7f06700 1 -- start start 2023-11-24T07:58:47.361 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:47.353+0000 7f14b7f06700 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f14b00b4c40 0x7f14b006dba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:47.361 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:47.353+0000 7f14b7f06700 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f14b006ef90 con 0x7f14b00b3b40 2023-11-24T07:58:47.362 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:47.353+0000 7f14b7f06700 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f14b006e0e0 con 0x7f14b006f420 2023-11-24T07:58:47.362 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:47.353+0000 7f14b7f06700 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f14b006e380 con 0x7f14b00b4c40 2023-11-24T07:58:47.362 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:47.353+0000 7f14b54a1700 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7f14b00b3b40 0x7f14b00b5020 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:57416/0 (socket says 172.21.15.59:57416) 2023-11-24T07:58:47.362 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:47.353+0000 7f14b54a1700 1 -- 172.21.15.59:0/1069117978 learned_addr learned my addr 172.21.15.59:0/1069117978 (peer_addr_for_me v1:172.21.15.59:0/0) 2023-11-24T07:58:47.362 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:47.353+0000 7f14b5ca2700 1 --2- 172.21.15.59:0/1069117978 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f14b00b4c40 0x7f14b006dba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:47.362 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:47.353+0000 7f14b4ca0700 1 -- 172.21.15.59:0/1069117978 <== mon.0 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3634670244 0 0) 0x7f14b006ef90 con 0x7f14b00b3b40 2023-11-24T07:58:47.362 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:47.354+0000 7f14b4ca0700 1 -- 172.21.15.59:0/1069117978 --> v1:172.21.15.59:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f149c0035b0 con 0x7f14b00b3b40 2023-11-24T07:58:47.362 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:47.354+0000 7f14b5ca2700 1 -- 172.21.15.59:0/1069117978 >> v1:172.21.15.76:6789/0 conn(0x7f14b006f420 legacy=0x7f14b00ce1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:47.363 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:47.354+0000 7f14b5ca2700 1 -- 172.21.15.59:0/1069117978 >> v1:172.21.15.59:6789/0 conn(0x7f14b00b3b40 legacy=0x7f14b00b5020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:47.363 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:47.354+0000 7f14b4ca0700 1 -- 172.21.15.59:0/1069117978 <== mon.1 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3524002576 0 0) 0x7f14b006e0e0 con 0x7f14b006f420 2023-11-24T07:58:47.363 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:47.354+0000 7f14b5ca2700 1 -- 172.21.15.59:0/1069117978 --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f14b006e660 con 0x7f14b00b4c40 2023-11-24T07:58:47.363 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:47.354+0000 7f14b5ca2700 1 --2- 172.21.15.59:0/1069117978 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f14b00b4c40 0x7f14b006dba0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f14a4009870 tx=0x7f14a4009b80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=905e3561708eb250 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:58:47.363 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:47.354+0000 7f14b4ca0700 1 -- 172.21.15.59:0/1069117978 <== mon.2 v2:172.21.15.76:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14a4011830 con 0x7f14b00b4c40 2023-11-24T07:58:47.363 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:47.354+0000 7f14b4ca0700 1 -- 172.21.15.59:0/1069117978 <== mon.2 v2:172.21.15.76:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f14a4011990 con 0x7f14b00b4c40 2023-11-24T07:58:47.364 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:47.354+0000 7f14b4ca0700 1 -- 172.21.15.59:0/1069117978 <== mon.2 v2:172.21.15.76:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14a4011d20 con 0x7f14b00b4c40 2023-11-24T07:58:47.364 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:47.355+0000 7f14b7f06700 1 -- 172.21.15.59:0/1069117978 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f14b00b4c40 msgr2=0x7f14b006dba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:47.364 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:47.355+0000 7f14b7f06700 1 --2- 172.21.15.59:0/1069117978 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f14b00b4c40 0x7f14b006dba0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f14a4009870 tx=0x7f14a4009b80 comp rx=0 tx=0).stop 2023-11-24T07:58:47.364 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:47.355+0000 7f14b7f06700 1 -- 172.21.15.59:0/1069117978 shutdown_connections 2023-11-24T07:58:47.364 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:47.355+0000 7f14b7f06700 1 --2- 172.21.15.59:0/1069117978 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f14b00b4c40 0x7f14b006dba0 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:47.364 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:47.355+0000 7f14b7f06700 1 -- 172.21.15.59:0/1069117978 >> 172.21.15.59:0/1069117978 conn(0x7f14b00b2840 msgr2=0x7f14b00b2c20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:58:47.364 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:47.355+0000 7f14b7f06700 1 -- 172.21.15.59:0/1069117978 shutdown_connections 2023-11-24T07:58:47.364 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:47.355+0000 7f14b7f06700 1 -- 172.21.15.59:0/1069117978 wait complete. 2023-11-24T07:58:47.365 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:47.356+0000 7f14b7f06700 1 Processor -- start 2023-11-24T07:58:47.365 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:47.356+0000 7f14b7f06700 1 -- start start 2023-11-24T07:58:47.365 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:47.357+0000 7f14b7f06700 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f14b006f420 0x7f14b00b0010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:47.365 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:47.357+0000 7f14b7f06700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f14b00b3b40 0x7f14b015ed40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:47.365 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:47.357+0000 7f14b7f06700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f14b00b4c40 0x7f14b015f280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:47.365 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:47.357+0000 7f14b7f06700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14b01f7980 con 0x7f14b00b4c40 2023-11-24T07:58:47.365 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:47.357+0000 7f14b7f06700 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14b015cf80 con 0x7f14b00b3b40 2023-11-24T07:58:47.365 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:47.357+0000 7f14b7f06700 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f14b01f7070 con 0x7f14b006f420 2023-11-24T07:58:47.366 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:47.357+0000 7f14b64a3700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f14b00b4c40 0x7f14b015f280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:47.366 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:47.357+0000 7f14b64a3700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f14b00b4c40 0x7f14b015f280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:53950/0 (socket says 172.21.15.59:53950) 2023-11-24T07:58:47.366 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:47.357+0000 7f14b64a3700 1 -- 172.21.15.59:0/1401701100 learned_addr learned my addr 172.21.15.59:0/1401701100 (peer_addr_for_me v2:172.21.15.59:0/0) 2023-11-24T07:58:47.366 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:47.357+0000 7f14b5ca2700 1 --2- 172.21.15.59:0/1401701100 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f14b006f420 0x7f14b00b0010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:47.366 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:47.357+0000 7f14b54a1700 1 --2- 172.21.15.59:0/1401701100 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f14b00b3b40 0x7f14b015ed40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:47.366 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:47.357+0000 7f14b64a3700 1 -- 172.21.15.59:0/1401701100 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f14b006f420 msgr2=0x7f14b00b0010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:47.366 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:47.357+0000 7f14b64a3700 1 --2- 172.21.15.59:0/1401701100 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f14b006f420 0x7f14b00b0010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:47.366 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:47.357+0000 7f14b64a3700 1 -- 172.21.15.59:0/1401701100 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f14b00b3b40 msgr2=0x7f14b015ed40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:47.367 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:47.357+0000 7f14b64a3700 1 --2- 172.21.15.59:0/1401701100 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f14b00b3b40 0x7f14b015ed40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:47.367 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:47.357+0000 7f14b64a3700 1 -- 172.21.15.59:0/1401701100 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f14a40094a0 con 0x7f14b00b4c40 2023-11-24T07:58:47.367 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:47.357+0000 7f14b64a3700 1 --2- 172.21.15.59:0/1401701100 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f14b00b4c40 0x7f14b015f280 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 crypto rx=0x7f14a800c7a0 tx=0x7f14a800cab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:58:47.367 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:47.357+0000 7f14a27fc700 1 -- 172.21.15.59:0/1401701100 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14a8003e30 con 0x7f14b00b4c40 2023-11-24T07:58:47.367 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:47.357+0000 7f14a27fc700 1 -- 172.21.15.59:0/1401701100 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f14a8005070 con 0x7f14b00b4c40 2023-11-24T07:58:47.367 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:47.357+0000 7f14b7f06700 1 -- 172.21.15.59:0/1401701100 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f14b01f4820 con 0x7f14b00b4c40 2023-11-24T07:58:47.367 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:47.357+0000 7f14a27fc700 1 -- 172.21.15.59:0/1401701100 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14a8005420 con 0x7f14b00b4c40 2023-11-24T07:58:47.368 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:47.357+0000 7f14b7f06700 1 -- 172.21.15.59:0/1401701100 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f14b01f75a0 con 0x7f14b00b4c40 2023-11-24T07:58:47.368 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:47.359+0000 7f14a27fc700 1 -- 172.21.15.59:0/1401701100 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 8) v1 ==== 89435+0+0 (secure 0 0 0) 0x7f14a8005580 con 0x7f14b00b4c40 2023-11-24T07:58:47.368 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:47.359+0000 7f14a27fc700 1 --2- 172.21.15.59:0/1401701100 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f149406b110 0x7f149406d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:47.368 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:47.360+0000 7f14a27fc700 1 -- 172.21.15.59:0/1401701100 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(60..60 src has 1..60) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f14a8088f80 con 0x7f14b00b4c40 2023-11-24T07:58:47.368 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:47.360+0000 7f14b5ca2700 1 --2- 172.21.15.59:0/1401701100 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f149406b110 0x7f149406d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:47.370 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:47.360+0000 7f14b5ca2700 1 --2- 172.21.15.59:0/1401701100 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f149406b110 0x7f149406d5d0 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f14a40097c0 tx=0x7f14a401c040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:58:47.371 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:47.362+0000 7f14c40ffb80 1 Processor -- start 2023-11-24T07:58:47.371 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:47.362+0000 7f14c40ffb80 1 -- start start 2023-11-24T07:58:47.371 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:47.362+0000 7f14c40ffb80 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55952637c080 0x5595263c2840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:47.371 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:47.362+0000 7f14c40ffb80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559526391ed0 0x5595263c2d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:47.371 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:47.362+0000 7f14c40ffb80 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x559526379490 0x5595263a5e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:47.371 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:47.362+0000 7f14c40ffb80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x5595262ef770 con 0x559526391ed0 2023-11-24T07:58:47.371 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:47.362+0000 7f14c40ffb80 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x5595262d6750 con 0x55952637c080 2023-11-24T07:58:47.371 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:47.362+0000 7f14c40ffb80 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 v1 -- 0x5595262d4070 con 0x559526379490 2023-11-24T07:58:47.371 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:47.362+0000 7f14b54a1700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559526391ed0 0x5595263c2d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:47.371 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:47.363+0000 7f14b54a1700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559526391ed0 0x5595263c2d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:53956/0 (socket says 172.21.15.59:53956) 2023-11-24T07:58:47.371 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:47.363+0000 7f14b54a1700 1 -- 172.21.15.59:0/2600700660 learned_addr learned my addr 172.21.15.59:0/2600700660 (peer_addr_for_me v2:172.21.15.59:0/0) 2023-11-24T07:58:47.371 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:47.363+0000 7f14b5ca2700 1 --2- 172.21.15.59:0/2600700660 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55952637c080 0x5595263c2840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:47.371 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:47.363+0000 7f14b64a3700 1 --2- 172.21.15.59:0/2600700660 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x559526379490 0x5595263a5e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:47.372 INFO:tasks.ceph.mds.a.smithi059.stderr:2023-11-24T07:58:47.363+0000 7f9f05ff1700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-11-24T07:58:47.377 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:47.363+0000 7f14b54a1700 1 -- 172.21.15.59:0/2600700660 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x559526379490 msgr2=0x5595263a5e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T07:58:47.377 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:47.363+0000 7f14b54a1700 1 --2- 172.21.15.59:0/2600700660 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x559526379490 0x5595263a5e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:47.377 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:47.363+0000 7f14b54a1700 1 -- 172.21.15.59:0/2600700660 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55952637c080 msgr2=0x5595263c2840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T07:58:47.377 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:47.363+0000 7f14b54a1700 1 --2- 172.21.15.59:0/2600700660 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55952637c080 0x5595263c2840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:47.377 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:47.363+0000 7f14b54a1700 1 -- 172.21.15.59:0/2600700660 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55952614c5e0 con 0x559526391ed0 2023-11-24T07:58:47.377 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:47.363+0000 7f14b64a3700 1 --2- 172.21.15.59:0/2600700660 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x559526379490 0x5595263a5e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-24T07:58:47.377 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:47.363+0000 7f14b54a1700 1 --2- 172.21.15.59:0/2600700660 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559526391ed0 0x5595263c2d80 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 crypto rx=0x7f14ac000f60 tx=0x7f14ac000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=744de18df8f09c97 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:58:47.377 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:47.363+0000 7f14937fe700 1 -- 172.21.15.59:0/2600700660 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14ac005b50 con 0x559526391ed0 2023-11-24T07:58:47.377 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:47.363+0000 7f14937fe700 1 -- 172.21.15.59:0/2600700660 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f14ac005cb0 con 0x559526391ed0 2023-11-24T07:58:47.377 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:47.363+0000 7f14937fe700 1 -- 172.21.15.59:0/2600700660 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14ac007310 con 0x559526391ed0 2023-11-24T07:58:47.377 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:47.364+0000 7f14c40ffb80 1 -- 172.21.15.59:0/2600700660 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559526391ed0 msgr2=0x5595263c2d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:47.377 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:47.364+0000 7f14c40ffb80 1 --2- 172.21.15.59:0/2600700660 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559526391ed0 0x5595263c2d80 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 crypto rx=0x7f14ac000f60 tx=0x7f14ac000bc0 comp rx=0 tx=0).stop 2023-11-24T07:58:47.378 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:47.364+0000 7f14c40ffb80 1 -- 172.21.15.59:0/2600700660 shutdown_connections 2023-11-24T07:58:47.378 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:47.364+0000 7f14c40ffb80 1 --2- 172.21.15.59:0/2600700660 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x559526379490 0x5595263a5e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:47.378 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:47.364+0000 7f14c40ffb80 1 --2- 172.21.15.59:0/2600700660 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55952637c080 0x5595263c2840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:47.378 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:47.364+0000 7f14c40ffb80 1 --2- 172.21.15.59:0/2600700660 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559526391ed0 0x5595263c2d80 unknown :-1 s=CLOSED pgs=723 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:47.378 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:47.364+0000 7f14c40ffb80 1 -- 172.21.15.59:0/2600700660 >> 172.21.15.59:0/2600700660 conn(0x5595262aa310 msgr2=0x5595263c0e70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:58:47.378 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:47.364+0000 7f14c40ffb80 1 -- 172.21.15.59:0/2600700660 shutdown_connections 2023-11-24T07:58:47.378 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:47.364+0000 7f14c40ffb80 1 -- 172.21.15.59:0/2600700660 wait complete. 2023-11-24T07:58:47.378 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:47.365+0000 7f14c40ffb80 1 -- start start 2023-11-24T07:58:47.378 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:47.365+0000 7f14c40ffb80 1 Processor -- start 2023-11-24T07:58:47.378 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:47.366+0000 7f14c40ffb80 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x559526379490 0x5595263cfc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:47.378 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:47.366+0000 7f14c40ffb80 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x55952637c080 0x5595263a3820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:47.378 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:47.366+0000 7f14c40ffb80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559526391ed0 0x5595263a79f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:47.379 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:47.366+0000 7f14c40ffb80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x5595262cf960 con 0x559526391ed0 2023-11-24T07:58:47.379 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:47.366+0000 7f14c40ffb80 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x559526289510 con 0x559526379490 2023-11-24T07:58:47.379 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:47.366+0000 7f14c40ffb80 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 v1 -- 0x5595262891b0 con 0x55952637c080 2023-11-24T07:58:47.380 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:47.366+0000 7f14c40ffb80 10 client.0 _ll_register_callbacks cb 0x5595261b4a30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-24T07:58:47.380 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:47.366+0000 7f14b64a3700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559526391ed0 0x5595263a79f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:47.380 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:47.366+0000 7f14b64a3700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559526391ed0 0x5595263a79f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:53964/0 (socket says 172.21.15.59:53964) 2023-11-24T07:58:47.380 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:47.366+0000 7f14b64a3700 1 -- 172.21.15.59:0/1761960989 learned_addr learned my addr 172.21.15.59:0/1761960989 (peer_addr_for_me v2:172.21.15.59:0/0) 2023-11-24T07:58:47.380 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:47.366+0000 7f14b5ca2700 1 --2- 172.21.15.59:0/1761960989 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x559526379490 0x5595263cfc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:47.380 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:47.366+0000 7f14b54a1700 1 --2- 172.21.15.59:0/1761960989 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x55952637c080 0x5595263a3820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:47.380 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:47.367+0000 7f14b64a3700 1 -- 172.21.15.59:0/1761960989 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x55952637c080 msgr2=0x5595263a3820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T07:58:47.380 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:47.367+0000 7f14b64a3700 1 --2- 172.21.15.59:0/1761960989 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x55952637c080 0x5595263a3820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:47.380 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:47.367+0000 7f14b64a3700 1 -- 172.21.15.59:0/1761960989 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x559526379490 msgr2=0x5595263cfc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T07:58:47.380 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:47.367+0000 7f14b64a3700 1 --2- 172.21.15.59:0/1761960989 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x559526379490 0x5595263cfc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:47.380 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:47.367+0000 7f14b64a3700 1 -- 172.21.15.59:0/1761960989 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f149c0035b0 con 0x559526391ed0 2023-11-24T07:58:47.380 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:47.367+0000 7f14b54a1700 1 --2- 172.21.15.59:0/1761960989 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x55952637c080 0x5595263a3820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-24T07:58:47.380 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:47.367+0000 7f14b64a3700 1 --2- 172.21.15.59:0/1761960989 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559526391ed0 0x5595263a79f0 secure :-1 s=READY pgs=724 cs=0 l=1 rev1=1 crypto rx=0x7f14a8057900 tx=0x7f14a8057eb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=caae37b53ce119b5 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:58:47.381 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:47.367+0000 7f1490ff9700 10 client.0 ms_handle_connect on v2:172.21.15.59:3300/0 2023-11-24T07:58:47.381 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:47.367+0000 7f1490ff9700 1 -- 172.21.15.59:0/1761960989 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14a805fc70 con 0x559526391ed0 2023-11-24T07:58:47.381 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:47.367+0000 7f1490ff9700 1 -- 172.21.15.59:0/1761960989 <== mon.0 v2:172.21.15.59:3300/0 2 ==== osd_map(60..60 src has 1..60) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f14a805f7e0 con 0x559526391ed0 2023-11-24T07:58:47.381 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:47.367+0000 7f14c40ffb80 1 -- 172.21.15.59:0/1761960989 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55952636ea40 con 0x559526391ed0 2023-11-24T07:58:47.381 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:47.367+0000 7f1490ff9700 1 -- 172.21.15.59:0/1761960989 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) v2 ==== 24+0+0 (secure 0 0 0) 0x7f14a805f9a0 con 0x559526391ed0 2023-11-24T07:58:47.381 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:47.367+0000 7f14c40ffb80 10 client.6234 fetch_fsmap learned FSMap version 35 2023-11-24T07:58:47.381 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:47.367+0000 7f14c40ffb80 1 -- 172.21.15.59:0/1761960989 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=2+}) v3 -- 0x5595260f8d90 con 0x559526391ed0 2023-11-24T07:58:47.381 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:47.367+0000 7f1490ff9700 1 -- 172.21.15.59:0/1761960989 <== mon.0 v2:172.21.15.59:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55952636ea40 con 0x559526391ed0 2023-11-24T07:58:47.381 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:47.367+0000 7f1490ff9700 1 -- 172.21.15.59:0/1761960989 <== mon.0 v2:172.21.15.59:3300/0 5 ==== fsmap(e 35) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f14a8087360 con 0x559526391ed0 2023-11-24T07:58:47.381 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:47.367+0000 7f14c40ffb80 10 client.6234 fetch_fsmap finished waiting for FSMap version 35 2023-11-24T07:58:47.381 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:47.367+0000 7f14c40ffb80 10 client.6234 resolve_mds: resolved name 'a' to daemon mds.a 2023-11-24T07:58:47.381 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:47.367+0000 7f14c40ffb80 20 client.6234 populate_metadata read hostname 'smithi059' 2023-11-24T07:58:47.381 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:47.367+0000 7f14c40ffb80 1 --2- 172.21.15.59:0/1761960989 >> [v2:172.21.15.59:6832/1599845006,v1:172.21.15.59:6835/1599845006] conn(0x5595263a3d60 0x5595263a2460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:47.381 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:47.367+0000 7f14c40ffb80 4 client.6234 mds_command: new command op to 6093 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-11-24T07:58:47.382 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:47.367+0000 7f14c40ffb80 1 -- 172.21.15.59:0/1761960989 --> [v2:172.21.15.59:6832/1599845006,v1:172.21.15.59:6835/1599845006] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55952634a840 con 0x5595263a3d60 2023-11-24T07:58:47.382 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:47.367+0000 7f14b5ca2700 1 --2- 172.21.15.59:0/1761960989 >> [v2:172.21.15.59:6832/1599845006,v1:172.21.15.59:6835/1599845006] conn(0x5595263a3d60 0x5595263a2460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:47.382 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:47.368+0000 7f1490ff9700 10 client.6234 ms_handle_connect on v2:172.21.15.59:6832/1599845006 2023-11-24T07:58:47.382 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:47.368+0000 7f14b5ca2700 1 --2- 172.21.15.59:0/1761960989 >> [v2:172.21.15.59:6832/1599845006,v1:172.21.15.59:6835/1599845006] conn(0x5595263a3d60 0x5595263a2460 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=35b4195faf0c3023 server_cookie=accd1819e6a103f5 in_seq=0 out_seq=0 2023-11-24T07:58:47.382 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:47.368+0000 7f1490ff9700 1 -- 172.21.15.59:0/1761960989 <== mds.0 v2:172.21.15.59:6832/1599845006 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55952634a840 con 0x5595263a3d60 2023-11-24T07:58:47.382 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:47.368+0000 7f1490ff9700 10 client.6234 handle_command_reply: tid=0 2023-11-24T07:58:47.382 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:47.368+0000 7f14c40ffb80 1 client.6234 shutdown 2023-11-24T07:58:47.382 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:47.368+0000 7f14c40ffb80 1 -- 172.21.15.59:0/1761960989 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559526391ed0 msgr2=0x5595263a79f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:47.382 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:47.368+0000 7f14c40ffb80 1 --2- 172.21.15.59:0/1761960989 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559526391ed0 0x5595263a79f0 secure :-1 s=READY pgs=724 cs=0 l=1 rev1=1 crypto rx=0x7f14a8057900 tx=0x7f14a8057eb0 comp rx=0 tx=0).stop 2023-11-24T07:58:47.382 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:47.369+0000 7f14c40ffb80 1 -- 172.21.15.59:0/1761960989 shutdown_connections 2023-11-24T07:58:47.382 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:47.369+0000 7f14c40ffb80 1 --2- 172.21.15.59:0/1761960989 >> [v2:172.21.15.59:6832/1599845006,v1:172.21.15.59:6835/1599845006] conn(0x5595263a3d60 0x5595263a2460 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:47.382 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:47.369+0000 7f14c40ffb80 1 --2- 172.21.15.59:0/1761960989 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559526391ed0 0x5595263a79f0 unknown :-1 s=CLOSED pgs=724 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:47.382 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:47.369+0000 7f14c40ffb80 1 --2- 172.21.15.59:0/1761960989 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x559526379490 0x5595263cfc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:47.383 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:47.369+0000 7f14c40ffb80 1 --2- 172.21.15.59:0/1761960989 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x55952637c080 0x5595263a3820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:47.383 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:47.369+0000 7f1490ff9700 0 client.6234 ms_handle_reset on v2:172.21.15.59:6832/1599845006 2023-11-24T07:58:47.383 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:47.369+0000 7f14c40ffb80 1 -- 172.21.15.59:0/1761960989 >> 172.21.15.59:0/1761960989 conn(0x5595262aa310 msgr2=0x5595263c4c80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:58:47.383 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:47.369+0000 7f14c40ffb80 1 -- 172.21.15.59:0/1761960989 shutdown_connections 2023-11-24T07:58:47.383 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:47.369+0000 7f14c40ffb80 1 -- 172.21.15.59:0/1761960989 wait complete. 2023-11-24T07:58:47.383 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:47.369+0000 7f14c40ffb80 20 client.6234 trim_cache size 0 max 16384 2023-11-24T07:58:47.388 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:47.387+0000 7f14c40ffb80 1 Processor -- start 2023-11-24T07:58:47.389 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:47.387+0000 7f14c40ffb80 1 -- start start 2023-11-24T07:58:47.391 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:47.388+0000 7f14c40ffb80 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x559526379490 0x559526423c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:47.391 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:47.388+0000 7f14c40ffb80 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55952637c080 0x559526376230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:47.391 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:47.388+0000 7f14c40ffb80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559526391ed0 0x559526376770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:47.391 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:47.388+0000 7f14c40ffb80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x5595263712f0 con 0x559526391ed0 2023-11-24T07:58:47.391 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:47.388+0000 7f14c40ffb80 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x55952614e8d0 con 0x55952637c080 2023-11-24T07:58:47.391 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:47.388+0000 7f14c40ffb80 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 v1 -- 0x5595263c0af0 con 0x559526379490 2023-11-24T07:58:47.392 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:47.388+0000 7f14b64a3700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559526391ed0 0x559526376770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:47.392 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:47.388+0000 7f14b64a3700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559526391ed0 0x559526376770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:53972/0 (socket says 172.21.15.59:53972) 2023-11-24T07:58:47.392 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:47.388+0000 7f14b64a3700 1 -- 172.21.15.59:0/2121663384 learned_addr learned my addr 172.21.15.59:0/2121663384 (peer_addr_for_me v2:172.21.15.59:0/0) 2023-11-24T07:58:47.392 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:47.388+0000 7f14b54a1700 1 --2- 172.21.15.59:0/2121663384 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55952637c080 0x559526376230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:47.392 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:47.388+0000 7f14b5ca2700 1 --2- 172.21.15.59:0/2121663384 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x559526379490 0x559526423c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:47.392 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:47.388+0000 7f14b64a3700 1 -- 172.21.15.59:0/2121663384 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x559526379490 msgr2=0x559526423c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T07:58:47.392 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:47.388+0000 7f14b64a3700 1 --2- 172.21.15.59:0/2121663384 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x559526379490 0x559526423c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:47.392 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:47.388+0000 7f14b64a3700 1 -- 172.21.15.59:0/2121663384 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55952637c080 msgr2=0x559526376230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T07:58:47.392 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:47.388+0000 7f14b64a3700 1 --2- 172.21.15.59:0/2121663384 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55952637c080 0x559526376230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:47.392 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:47.388+0000 7f14b64a3700 1 -- 172.21.15.59:0/2121663384 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5595260e1c10 con 0x559526391ed0 2023-11-24T07:58:47.392 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:47.388+0000 7f14b64a3700 1 --2- 172.21.15.59:0/2121663384 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559526391ed0 0x559526376770 secure :-1 s=READY pgs=725 cs=0 l=1 rev1=1 crypto rx=0x7f14a8007fd0 tx=0x7f14a801e050 comp rx=0 tx=0).ready entity=mon.0 client_cookie=97ca8d7b86f57f71 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:58:47.392 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:47.388+0000 7f1493fff700 1 -- 172.21.15.59:0/2121663384 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14a800e040 con 0x559526391ed0 2023-11-24T07:58:47.393 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:47.388+0000 7f1493fff700 1 -- 172.21.15.59:0/2121663384 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f14a805fc70 con 0x559526391ed0 2023-11-24T07:58:47.393 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:47.389+0000 7f1493fff700 1 -- 172.21.15.59:0/2121663384 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14a8058ca0 con 0x559526391ed0 2023-11-24T07:58:47.393 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:47.389+0000 7f14c40ffb80 1 -- 172.21.15.59:0/2121663384 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559526391ed0 msgr2=0x559526376770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:47.393 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:47.389+0000 7f14c40ffb80 1 --2- 172.21.15.59:0/2121663384 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559526391ed0 0x559526376770 secure :-1 s=READY pgs=725 cs=0 l=1 rev1=1 crypto rx=0x7f14a8007fd0 tx=0x7f14a801e050 comp rx=0 tx=0).stop 2023-11-24T07:58:47.393 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:47.389+0000 7f14c40ffb80 1 -- 172.21.15.59:0/2121663384 shutdown_connections 2023-11-24T07:58:47.393 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:47.389+0000 7f14c40ffb80 1 --2- 172.21.15.59:0/2121663384 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559526391ed0 0x559526376770 unknown :-1 s=CLOSED pgs=725 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:47.393 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:47.389+0000 7f14c40ffb80 1 --2- 172.21.15.59:0/2121663384 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x559526379490 0x559526423c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:47.393 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:47.389+0000 7f14c40ffb80 1 --2- 172.21.15.59:0/2121663384 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55952637c080 0x559526376230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:47.393 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:47.389+0000 7f14c40ffb80 1 -- 172.21.15.59:0/2121663384 >> 172.21.15.59:0/2121663384 conn(0x5595262aa310 msgr2=0x559526377600 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:58:47.393 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:47.389+0000 7f14c40ffb80 1 -- 172.21.15.59:0/2121663384 shutdown_connections 2023-11-24T07:58:47.393 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:47.389+0000 7f14c40ffb80 1 -- 172.21.15.59:0/2121663384 wait complete. 2023-11-24T07:58:47.393 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:47.390+0000 7f14c40ffb80 1 -- start start 2023-11-24T07:58:47.394 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:47.390+0000 7f14c40ffb80 1 Processor -- start 2023-11-24T07:58:47.398 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:47.391+0000 7f14c40ffb80 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x559526379490 0x55952640fd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:47.399 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:47.391+0000 7f14c40ffb80 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55952637c080 0x5595263a5230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:47.399 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:47.391+0000 7f14c40ffb80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559526391ed0 0x5595263a9210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:47.399 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:47.391+0000 7f14c40ffb80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x559526410260 con 0x559526391ed0 2023-11-24T07:58:47.399 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:47.391+0000 7f14c40ffb80 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x559526422890 con 0x55952637c080 2023-11-24T07:58:47.399 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:47.391+0000 7f14c40ffb80 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 v1 -- 0x5595263a3240 con 0x559526379490 2023-11-24T07:58:47.400 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:47.391+0000 7f14c40ffb80 10 client.0 _ll_register_callbacks cb 0x5595261b4a30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-24T07:58:47.400 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:47.391+0000 7f14b64a3700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559526391ed0 0x5595263a9210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:47.400 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:47.391+0000 7f14b64a3700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559526391ed0 0x5595263a9210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:53978/0 (socket says 172.21.15.59:53978) 2023-11-24T07:58:47.400 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:47.391+0000 7f14b64a3700 1 -- 172.21.15.59:0/1122370892 learned_addr learned my addr 172.21.15.59:0/1122370892 (peer_addr_for_me v2:172.21.15.59:0/0) 2023-11-24T07:58:47.400 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:47.391+0000 7f14b5ca2700 1 --2- 172.21.15.59:0/1122370892 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x559526379490 0x55952640fd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:47.400 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:47.391+0000 7f14b54a1700 1 --2- 172.21.15.59:0/1122370892 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55952637c080 0x5595263a5230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:47.401 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:47.391+0000 7f14b64a3700 1 -- 172.21.15.59:0/1122370892 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x559526379490 msgr2=0x55952640fd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T07:58:47.401 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:47.391+0000 7f14b64a3700 1 --2- 172.21.15.59:0/1122370892 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x559526379490 0x55952640fd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:47.401 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:47.391+0000 7f14b64a3700 1 -- 172.21.15.59:0/1122370892 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55952637c080 msgr2=0x5595263a5230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T07:58:47.401 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:47.391+0000 7f14b64a3700 1 --2- 172.21.15.59:0/1122370892 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55952637c080 0x5595263a5230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:47.401 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:47.391+0000 7f14b64a3700 1 -- 172.21.15.59:0/1122370892 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5595260f8d90 con 0x559526391ed0 2023-11-24T07:58:47.401 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:47.391+0000 7f14b64a3700 1 --2- 172.21.15.59:0/1122370892 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559526391ed0 0x5595263a9210 secure :-1 s=READY pgs=726 cs=0 l=1 rev1=1 crypto rx=0x7f14a8001440 tx=0x7f14a8003f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=28f3211bc03951c0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:58:47.402 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:47.391+0000 7f1490ff9700 10 client.0 ms_handle_connect on v2:172.21.15.59:3300/0 2023-11-24T07:58:47.402 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:47.391+0000 7f1490ff9700 1 -- 172.21.15.59:0/1122370892 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14a800e040 con 0x559526391ed0 2023-11-24T07:58:47.402 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:47.391+0000 7f1490ff9700 1 -- 172.21.15.59:0/1122370892 <== mon.0 v2:172.21.15.59:3300/0 2 ==== osd_map(60..60 src has 1..60) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f14a805f7e0 con 0x559526391ed0 2023-11-24T07:58:47.402 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:47.392+0000 7f14c40ffb80 1 -- 172.21.15.59:0/1122370892 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f14ac005cb0 con 0x559526391ed0 2023-11-24T07:58:47.402 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:47.392+0000 7f1490ff9700 1 -- 172.21.15.59:0/1122370892 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) v2 ==== 24+0+0 (secure 0 0 0) 0x7f14a805f9a0 con 0x559526391ed0 2023-11-24T07:58:47.403 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:47.392+0000 7f14c40ffb80 10 client.6240 fetch_fsmap learned FSMap version 35 2023-11-24T07:58:47.403 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:47.392+0000 7f14c40ffb80 1 -- 172.21.15.59:0/1122370892 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=2+}) v3 -- 0x55952611f200 con 0x559526391ed0 2023-11-24T07:58:47.403 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:47.392+0000 7f1490ff9700 1 -- 172.21.15.59:0/1122370892 <== mon.0 v2:172.21.15.59:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f14a8064020 con 0x559526391ed0 2023-11-24T07:58:47.403 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:47.392+0000 7f1490ff9700 1 -- 172.21.15.59:0/1122370892 <== mon.0 v2:172.21.15.59:3300/0 5 ==== fsmap(e 35) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f14a807c070 con 0x559526391ed0 2023-11-24T07:58:47.403 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:47.392+0000 7f14c40ffb80 10 client.6240 fetch_fsmap finished waiting for FSMap version 35 2023-11-24T07:58:47.403 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:47.392+0000 7f14c40ffb80 10 client.6240 resolve_mds: resolved name 'a' to daemon mds.a 2023-11-24T07:58:47.404 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:47.392+0000 7f14c40ffb80 20 client.6240 populate_metadata read hostname 'smithi059' 2023-11-24T07:58:47.404 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:47.392+0000 7f14c40ffb80 1 --2- 172.21.15.59:0/1122370892 >> [v2:172.21.15.59:6832/1599845006,v1:172.21.15.59:6835/1599845006] conn(0x5595264217f0 0x559526421bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:47.404 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:47.392+0000 7f14c40ffb80 4 client.6240 mds_command: new command op to 6093 tid=0 multi_id=0 [{"prefix": "status"}] 2023-11-24T07:58:47.404 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:47.392+0000 7f14c40ffb80 1 -- 172.21.15.59:0/1122370892 --> [v2:172.21.15.59:6832/1599845006,v1:172.21.15.59:6835/1599845006] -- command(tid 0: {"prefix": "status"}) v1 -- 0x559526295f80 con 0x5595264217f0 2023-11-24T07:58:47.404 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:47.392+0000 7f14b5ca2700 1 --2- 172.21.15.59:0/1122370892 >> [v2:172.21.15.59:6832/1599845006,v1:172.21.15.59:6835/1599845006] conn(0x5595264217f0 0x559526421bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:47.405 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:47.392+0000 7f1490ff9700 10 client.6240 ms_handle_connect on v2:172.21.15.59:6832/1599845006 2023-11-24T07:58:47.405 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:47.392+0000 7f14b5ca2700 1 --2- 172.21.15.59:0/1122370892 >> [v2:172.21.15.59:6832/1599845006,v1:172.21.15.59:6835/1599845006] conn(0x5595264217f0 0x559526421bd0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c5c7e7ecf77aacdf server_cookie=61162162637e43c in_seq=0 out_seq=0 2023-11-24T07:58:47.405 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:47.393+0000 7f1490ff9700 1 -- 172.21.15.59:0/1122370892 <== mds.0 v2:172.21.15.59:6832/1599845006 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+324 (crc 0 0 0) 0x559526295f80 con 0x5595264217f0 2023-11-24T07:58:47.405 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:47.393+0000 7f1490ff9700 10 client.6240 handle_command_reply: tid=0 2023-11-24T07:58:47.405 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:47.393+0000 7f14c40ffb80 1 client.6240 shutdown 2023-11-24T07:58:47.405 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:47.393+0000 7f14c40ffb80 1 -- 172.21.15.59:0/1122370892 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559526391ed0 msgr2=0x5595263a9210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:47.406 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:47.393+0000 7f14c40ffb80 1 --2- 172.21.15.59:0/1122370892 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559526391ed0 0x5595263a9210 secure :-1 s=READY pgs=726 cs=0 l=1 rev1=1 crypto rx=0x7f14a8001440 tx=0x7f14a8003f90 comp rx=0 tx=0).stop 2023-11-24T07:58:47.406 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:47.393+0000 7f14c40ffb80 1 -- 172.21.15.59:0/1122370892 shutdown_connections 2023-11-24T07:58:47.406 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:47.393+0000 7f14c40ffb80 1 --2- 172.21.15.59:0/1122370892 >> [v2:172.21.15.59:6832/1599845006,v1:172.21.15.59:6835/1599845006] conn(0x5595264217f0 0x559526421bd0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:47.406 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:47.393+0000 7f14c40ffb80 1 --2- 172.21.15.59:0/1122370892 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559526391ed0 0x5595263a9210 unknown :-1 s=CLOSED pgs=726 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:47.406 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:47.393+0000 7f14c40ffb80 1 --2- 172.21.15.59:0/1122370892 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x559526379490 0x55952640fd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:47.406 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:47.393+0000 7f1490ff9700 0 client.6240 ms_handle_reset on v2:172.21.15.59:6832/1599845006 2023-11-24T07:58:47.406 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:47.393+0000 7f14c40ffb80 1 --2- 172.21.15.59:0/1122370892 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55952637c080 0x5595263a5230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:47.406 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:47.393+0000 7f14c40ffb80 1 -- 172.21.15.59:0/1122370892 >> 172.21.15.59:0/1122370892 conn(0x5595262aa310 msgr2=0x5595263c0130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:58:47.406 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:47.393+0000 7f14c40ffb80 1 -- 172.21.15.59:0/1122370892 shutdown_connections 2023-11-24T07:58:47.406 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:47.393+0000 7f14c40ffb80 1 -- 172.21.15.59:0/1122370892 wait complete. 2023-11-24T07:58:47.406 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:47.393+0000 7f14c40ffb80 20 client.6240 trim_cache size 0 max 16384 2023-11-24T07:58:47.407 INFO:teuthology.orchestra.run.smithi059.stdout:{ 2023-11-24T07:58:47.407 INFO:teuthology.orchestra.run.smithi059.stdout: "cluster_fsid": "9f0e8741-b8f9-48d5-a2c3-09f45903303d", 2023-11-24T07:58:47.407 INFO:teuthology.orchestra.run.smithi059.stdout: "whoami": 0, 2023-11-24T07:58:47.407 INFO:teuthology.orchestra.run.smithi059.stdout: "id": 6093, 2023-11-24T07:58:47.407 INFO:teuthology.orchestra.run.smithi059.stdout: "want_state": "up:active", 2023-11-24T07:58:47.407 INFO:teuthology.orchestra.run.smithi059.stdout: "state": "up:active", 2023-11-24T07:58:47.407 INFO:teuthology.orchestra.run.smithi059.stdout: "fs_name": "cephfs", 2023-11-24T07:58:47.407 INFO:teuthology.orchestra.run.smithi059.stdout: "rank_uptime": 6.0179759150000001, 2023-11-24T07:58:47.407 INFO:teuthology.orchestra.run.smithi059.stdout: "mdsmap_epoch": 35, 2023-11-24T07:58:47.408 INFO:teuthology.orchestra.run.smithi059.stdout: "osdmap_epoch": 60, 2023-11-24T07:58:47.408 INFO:teuthology.orchestra.run.smithi059.stdout: "osdmap_epoch_barrier": 60, 2023-11-24T07:58:47.408 INFO:teuthology.orchestra.run.smithi059.stdout: "uptime": 10.93167614 2023-11-24T07:58:47.408 INFO:teuthology.orchestra.run.smithi059.stdout:} 2023-11-24T07:58:47.409 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:47.393+0000 7f14b7f06700 1 -- 172.21.15.59:0/1401701100 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f149406b110 msgr2=0x7f149406d5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:47.409 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:47.393+0000 7f14b7f06700 1 --2- 172.21.15.59:0/1401701100 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f149406b110 0x7f149406d5d0 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f14a40097c0 tx=0x7f14a401c040 comp rx=0 tx=0).stop 2023-11-24T07:58:47.409 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:47.394+0000 7f14b7f06700 1 -- 172.21.15.59:0/1401701100 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f14b00b4c40 msgr2=0x7f14b015f280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:47.409 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:47.394+0000 7f14b7f06700 1 --2- 172.21.15.59:0/1401701100 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f14b00b4c40 0x7f14b015f280 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 crypto rx=0x7f14a800c7a0 tx=0x7f14a800cab0 comp rx=0 tx=0).stop 2023-11-24T07:58:47.409 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:47.394+0000 7f14b7f06700 1 -- 172.21.15.59:0/1401701100 shutdown_connections 2023-11-24T07:58:47.409 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:47.394+0000 7f14b7f06700 1 --2- 172.21.15.59:0/1401701100 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f149406b110 0x7f149406d5d0 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:47.409 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:47.394+0000 7f14b7f06700 1 --2- 172.21.15.59:0/1401701100 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f14b00b4c40 0x7f14b015f280 unknown :-1 s=CLOSED pgs=722 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:47.410 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:47.394+0000 7f14b7f06700 1 --2- 172.21.15.59:0/1401701100 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f14b006f420 0x7f14b00b0010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:47.410 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:47.394+0000 7f14b7f06700 1 --2- 172.21.15.59:0/1401701100 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f14b00b3b40 0x7f14b015ed40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:47.410 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:47.394+0000 7f14b7f06700 1 -- 172.21.15.59:0/1401701100 >> 172.21.15.59:0/1401701100 conn(0x7f14b00b2840 msgr2=0x7f14b013f340 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:58:47.410 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:47.394+0000 7f14b7f06700 1 -- 172.21.15.59:0/1401701100 shutdown_connections 2023-11-24T07:58:47.410 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:47.394+0000 7f14b7f06700 1 -- 172.21.15.59:0/1401701100 wait complete. 2023-11-24T07:58:47.411 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-11-24T07:58:47.535 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:47.534+0000 7ff692ef3700 1 Processor -- start 2023-11-24T07:58:47.536 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:47.534+0000 7ff692ef3700 1 -- start start 2023-11-24T07:58:47.539 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:47.535+0000 7ff692ef3700 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7ff68c143ae0 0x7ff68c0b51b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:47.539 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:47.535+0000 7ff692ef3700 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7ff68c0722c0 0x7ff68c06e510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:47.539 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:47.535+0000 7ff692ef3700 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff68c143ec0 con 0x7ff68c1440f0 2023-11-24T07:58:47.539 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:47.535+0000 7ff692ef3700 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff68c0b40a0 con 0x7ff68c0722c0 2023-11-24T07:58:47.539 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:47.535+0000 7ff692ef3700 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff68c0b3ae0 con 0x7ff68c143ae0 2023-11-24T07:58:47.539 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:47.535+0000 7ff690c8f700 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7ff68c143ae0 0x7ff68c0b51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:47.539 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:47.535+0000 7ff691490700 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7ff68c0722c0 0x7ff68c06e510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:47.539 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:47.535+0000 7ff68bfff700 1 --1- >> v1:172.21.15.76:6789/0 conn(0x7ff68c1440f0 0x7ff68c071bb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.59:44734/0 (socket says 172.21.15.59:44734) 2023-11-24T07:58:47.539 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:47.535+0000 7ff691490700 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7ff68c0722c0 0x7ff68c06e510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.59:41184/0 (socket says 172.21.15.59:41184) 2023-11-24T07:58:47.539 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:47.535+0000 7ff68bfff700 1 -- 172.21.15.59:0/4238313732 learned_addr learned my addr 172.21.15.59:0/4238313732 (peer_addr_for_me v1:172.21.15.59:0/0) 2023-11-24T07:58:47.539 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:47.535+0000 7ff691490700 1 -- 172.21.15.59:0/4238313732 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7ff68c143ae0 msgr2=0x7ff68c0b51b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T07:58:47.540 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:47.535+0000 7ff691490700 1 --2- 172.21.15.59:0/4238313732 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7ff68c143ae0 0x7ff68c0b51b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:47.540 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:47.535+0000 7ff691490700 1 -- 172.21.15.59:0/4238313732 >> v1:172.21.15.76:6789/0 conn(0x7ff68c1440f0 legacy=0x7ff68c071bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:47.540 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:47.536+0000 7ff691490700 1 -- 172.21.15.59:0/4238313732 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff68c0b3dc0 con 0x7ff68c0722c0 2023-11-24T07:58:47.540 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:47.536+0000 7ff68b7fe700 1 -- 172.21.15.59:0/4238313732 <== mon.1 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1212481195 0 0) 0x7ff68c143ec0 con 0x7ff68c1440f0 2023-11-24T07:58:47.540 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:47.536+0000 7ff690c8f700 1 --2- 172.21.15.59:0/4238313732 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7ff68c143ae0 0x7ff68c0b51b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-24T07:58:47.540 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:47.536+0000 7ff691490700 1 --2- 172.21.15.59:0/4238313732 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff68c0722c0 0x7ff68c06e510 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7ff684009d90 tx=0x7ff68400b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9e54c299a4177c78 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:58:47.540 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:47.536+0000 7ff68b7fe700 1 -- 172.21.15.59:0/4238313732 <== mon.1 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff68400e070 con 0x7ff68c0722c0 2023-11-24T07:58:47.540 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:47.536+0000 7ff68b7fe700 1 -- 172.21.15.59:0/4238313732 <== mon.1 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff68400bb20 con 0x7ff68c0722c0 2023-11-24T07:58:47.540 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:47.536+0000 7ff68b7fe700 1 -- 172.21.15.59:0/4238313732 <== mon.1 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff684012430 con 0x7ff68c0722c0 2023-11-24T07:58:47.540 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:47.536+0000 7ff692ef3700 1 -- 172.21.15.59:0/4238313732 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff68c0722c0 msgr2=0x7ff68c06e510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:47.540 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:47.536+0000 7ff692ef3700 1 --2- 172.21.15.59:0/4238313732 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff68c0722c0 0x7ff68c06e510 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7ff684009d90 tx=0x7ff68400b330 comp rx=0 tx=0).stop 2023-11-24T07:58:47.540 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:47.536+0000 7ff692ef3700 1 -- 172.21.15.59:0/4238313732 shutdown_connections 2023-11-24T07:58:47.541 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:47.536+0000 7ff692ef3700 1 --2- 172.21.15.59:0/4238313732 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff68c0722c0 0x7ff68c06e510 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:47.541 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:47.536+0000 7ff692ef3700 1 --2- 172.21.15.59:0/4238313732 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7ff68c143ae0 0x7ff68c0b51b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:47.541 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:47.536+0000 7ff692ef3700 1 -- 172.21.15.59:0/4238313732 >> 172.21.15.59:0/4238313732 conn(0x7ff68c0b2740 msgr2=0x7ff68c0b2b20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:58:47.541 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:47.537+0000 7ff692ef3700 1 -- 172.21.15.59:0/4238313732 shutdown_connections 2023-11-24T07:58:47.541 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:47.537+0000 7ff692ef3700 1 -- 172.21.15.59:0/4238313732 wait complete. 2023-11-24T07:58:47.543 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:47.538+0000 7ff692ef3700 1 Processor -- start 2023-11-24T07:58:47.543 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:47.538+0000 7ff692ef3700 1 -- start start 2023-11-24T07:58:47.544 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:47.539+0000 7ff692ef3700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff68c0722c0 0x7ff68c13ee60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:47.544 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:47.539+0000 7ff692ef3700 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7ff68c143ae0 0x7ff68c166bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:47.544 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:47.539+0000 7ff692ef3700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff68c1440f0 0x7ff68c1684e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:47.544 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:47.539+0000 7ff692ef3700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff68c167440 con 0x7ff68c1440f0 2023-11-24T07:58:47.544 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:47.539+0000 7ff692ef3700 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff68c167720 con 0x7ff68c0722c0 2023-11-24T07:58:47.544 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:47.539+0000 7ff692ef3700 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff68c167a00 con 0x7ff68c143ae0 2023-11-24T07:58:47.544 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:47.539+0000 7ff691490700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff68c1440f0 0x7ff68c1684e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:47.544 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:47.539+0000 7ff691490700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff68c1440f0 0x7ff68c1684e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:53992/0 (socket says 172.21.15.59:53992) 2023-11-24T07:58:47.544 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:47.539+0000 7ff691490700 1 -- 172.21.15.59:0/3741467245 learned_addr learned my addr 172.21.15.59:0/3741467245 (peer_addr_for_me v2:172.21.15.59:0/0) 2023-11-24T07:58:47.544 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:47.539+0000 7ff690c8f700 1 --2- 172.21.15.59:0/3741467245 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff68c0722c0 0x7ff68c13ee60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:47.544 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:47.539+0000 7ff68bfff700 1 --2- 172.21.15.59:0/3741467245 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7ff68c143ae0 0x7ff68c166bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:47.544 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:47.539+0000 7ff691490700 1 -- 172.21.15.59:0/3741467245 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7ff68c143ae0 msgr2=0x7ff68c166bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:47.545 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:47.539+0000 7ff691490700 1 --2- 172.21.15.59:0/3741467245 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7ff68c143ae0 0x7ff68c166bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:47.545 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:47.539+0000 7ff691490700 1 -- 172.21.15.59:0/3741467245 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff68c0722c0 msgr2=0x7ff68c13ee60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:47.545 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:47.539+0000 7ff691490700 1 --2- 172.21.15.59:0/3741467245 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff68c0722c0 0x7ff68c13ee60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:47.545 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:47.539+0000 7ff691490700 1 -- 172.21.15.59:0/3741467245 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff6800094a0 con 0x7ff68c1440f0 2023-11-24T07:58:47.545 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:47.539+0000 7ff691490700 1 --2- 172.21.15.59:0/3741467245 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff68c1440f0 0x7ff68c1684e0 secure :-1 s=READY pgs=727 cs=0 l=1 rev1=1 crypto rx=0x7ff684013000 tx=0x7ff684004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:58:47.545 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:47.539+0000 7ff6897fa700 1 -- 172.21.15.59:0/3741467245 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff684012430 con 0x7ff68c1440f0 2023-11-24T07:58:47.545 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:47.540+0000 7ff6897fa700 1 -- 172.21.15.59:0/3741467245 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff684020070 con 0x7ff68c1440f0 2023-11-24T07:58:47.545 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:47.540+0000 7ff692ef3700 1 -- 172.21.15.59:0/3741467245 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff684009a70 con 0x7ff68c1440f0 2023-11-24T07:58:47.545 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:47.540+0000 7ff6897fa700 1 -- 172.21.15.59:0/3741467245 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff684004280 con 0x7ff68c1440f0 2023-11-24T07:58:47.545 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:47.540+0000 7ff692ef3700 1 -- 172.21.15.59:0/3741467245 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff68c167fc0 con 0x7ff68c1440f0 2023-11-24T07:58:47.545 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:47.541+0000 7ff692ef3700 1 -- 172.21.15.59:0/3741467245 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff68c168180 con 0x7ff68c1440f0 2023-11-24T07:58:47.545 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:47.541+0000 7ff6897fa700 1 -- 172.21.15.59:0/3741467245 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 8) v1 ==== 89435+0+0 (secure 0 0 0) 0x7ff6840043e0 con 0x7ff68c1440f0 2023-11-24T07:58:47.546 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:47.541+0000 7ff6897fa700 1 --2- 172.21.15.59:0/3741467245 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7ff67006b110 0x7ff67006d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:47.546 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:47.541+0000 7ff6897fa700 1 -- 172.21.15.59:0/3741467245 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(60..60 src has 1..60) v4 ==== 5709+0+0 (secure 0 0 0) 0x7ff684022070 con 0x7ff68c1440f0 2023-11-24T07:58:47.546 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:47.542+0000 7ff690c8f700 1 --2- 172.21.15.59:0/3741467245 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7ff67006b110 0x7ff67006d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:47.546 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:47.542+0000 7ff690c8f700 1 --2- 172.21.15.59:0/3741467245 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7ff67006b110 0x7ff67006d5d0 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7ff680009d70 tx=0x7ff680009210 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:58:47.547 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:47.546+0000 7ff6897fa700 1 -- 172.21.15.59:0/3741467245 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7ff684056430 con 0x7ff68c1440f0 2023-11-24T07:58:47.795 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:47.793+0000 7ff692ef3700 1 -- 172.21.15.59:0/3741467245 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7ff68c166a10 con 0x7ff68c1440f0 2023-11-24T07:58:47.796 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:47.793+0000 7ff6897fa700 1 -- 172.21.15.59:0/3741467245 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v35) v1 ==== 53+0+73 (secure 0 0 0) 0x7ff684059a50 con 0x7ff68c1440f0 2023-11-24T07:58:47.796 INFO:teuthology.orchestra.run.smithi059.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-11-24T07:58:47.797 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:47.795+0000 7ff692ef3700 1 -- 172.21.15.59:0/3741467245 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7ff67006b110 msgr2=0x7ff67006d5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:47.798 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:47.795+0000 7ff692ef3700 1 --2- 172.21.15.59:0/3741467245 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7ff67006b110 0x7ff67006d5d0 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7ff680009d70 tx=0x7ff680009210 comp rx=0 tx=0).stop 2023-11-24T07:58:47.798 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:47.795+0000 7ff692ef3700 1 -- 172.21.15.59:0/3741467245 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff68c1440f0 msgr2=0x7ff68c1684e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:47.798 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:47.795+0000 7ff692ef3700 1 --2- 172.21.15.59:0/3741467245 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff68c1440f0 0x7ff68c1684e0 secure :-1 s=READY pgs=727 cs=0 l=1 rev1=1 crypto rx=0x7ff684013000 tx=0x7ff684004ad0 comp rx=0 tx=0).stop 2023-11-24T07:58:47.798 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:47.796+0000 7ff692ef3700 1 -- 172.21.15.59:0/3741467245 shutdown_connections 2023-11-24T07:58:47.798 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:47.796+0000 7ff692ef3700 1 --2- 172.21.15.59:0/3741467245 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7ff67006b110 0x7ff67006d5d0 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:47.798 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:47.796+0000 7ff692ef3700 1 --2- 172.21.15.59:0/3741467245 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff68c1440f0 0x7ff68c1684e0 unknown :-1 s=CLOSED pgs=727 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:47.798 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:47.796+0000 7ff692ef3700 1 --2- 172.21.15.59:0/3741467245 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff68c0722c0 0x7ff68c13ee60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:47.798 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:47.796+0000 7ff692ef3700 1 --2- 172.21.15.59:0/3741467245 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7ff68c143ae0 0x7ff68c166bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:47.798 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:47.796+0000 7ff692ef3700 1 -- 172.21.15.59:0/3741467245 >> 172.21.15.59:0/3741467245 conn(0x7ff68c0b2740 msgr2=0x7ff68c154e80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:58:47.798 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:47.796+0000 7ff692ef3700 1 -- 172.21.15.59:0/3741467245 shutdown_connections 2023-11-24T07:58:47.798 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:47.796+0000 7ff692ef3700 1 -- 172.21.15.59:0/3741467245 wait complete. 2023-11-24T07:58:47.810 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2023-11-24T07:58:47.810 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-11-24T07:58:47.810 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi059.front.sepia.ceph.com 2023-11-24T07:58:47.810 INFO:tasks.cephfs.mount:self.client.name = client.0 2023-11-24T07:58:47.810 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-11-24T07:58:47.810 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2023-11-24T07:58:47.810 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2023-11-24T07:58:47.810 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-11-24T07:58:47.810 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-11-24T07:58:47.810 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T07:58:47.811 DEBUG:teuthology.orchestra.run.smithi059:> ip addr 2023-11-24T07:58:47.828 INFO:teuthology.orchestra.run.smithi059.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-11-24T07:58:47.829 INFO:teuthology.orchestra.run.smithi059.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-11-24T07:58:47.829 INFO:teuthology.orchestra.run.smithi059.stdout: inet 127.0.0.1/8 scope host lo 2023-11-24T07:58:47.829 INFO:teuthology.orchestra.run.smithi059.stdout: valid_lft forever preferred_lft forever 2023-11-24T07:58:47.829 INFO:teuthology.orchestra.run.smithi059.stdout: inet6 ::1/128 scope host 2023-11-24T07:58:47.829 INFO:teuthology.orchestra.run.smithi059.stdout: valid_lft forever preferred_lft forever 2023-11-24T07:58:47.829 INFO:teuthology.orchestra.run.smithi059.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-11-24T07:58:47.829 INFO:teuthology.orchestra.run.smithi059.stdout: link/ether 0c:c4:7a:6b:fe:ca brd ff:ff:ff:ff:ff:ff 2023-11-24T07:58:47.829 INFO:teuthology.orchestra.run.smithi059.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-11-24T07:58:47.829 INFO:teuthology.orchestra.run.smithi059.stdout: link/ether 0c:c4:7a:6b:fe:cb brd ff:ff:ff:ff:ff:ff 2023-11-24T07:58:47.830 INFO:teuthology.orchestra.run.smithi059.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2023-11-24T07:58:47.830 INFO:teuthology.orchestra.run.smithi059.stdout: link/ether 0c:c4:7a:bd:19:2a brd ff:ff:ff:ff:ff:ff 2023-11-24T07:58:47.830 INFO:teuthology.orchestra.run.smithi059.stdout: inet 172.21.15.59/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2023-11-24T07:58:47.830 INFO:teuthology.orchestra.run.smithi059.stdout: valid_lft 42240sec preferred_lft 42240sec 2023-11-24T07:58:47.830 INFO:teuthology.orchestra.run.smithi059.stdout: inet6 fe80::ec4:7aff:febd:192a/64 scope link 2023-11-24T07:58:47.830 INFO:teuthology.orchestra.run.smithi059.stdout: valid_lft forever preferred_lft forever 2023-11-24T07:58:47.830 INFO:teuthology.orchestra.run.smithi059.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-11-24T07:58:47.830 INFO:teuthology.orchestra.run.smithi059.stdout: link/ether 0c:c4:7a:bd:19:2b brd ff:ff:ff:ff:ff:ff 2023-11-24T07:58:47.830 INFO:teuthology.orchestra.run.smithi059.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-11-24T07:58:47.830 INFO:teuthology.orchestra.run.smithi059.stdout: link/ether 72:f2:ce:27:f6:18 brd ff:ff:ff:ff:ff:ff 2023-11-24T07:58:47.830 INFO:teuthology.orchestra.run.smithi059.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-11-24T07:58:47.830 INFO:teuthology.orchestra.run.smithi059.stdout: valid_lft forever preferred_lft forever 2023-11-24T07:58:47.830 INFO:teuthology.orchestra.run.smithi059.stdout: inet6 fe80::64c1:62ff:feb9:5b59/64 scope link 2023-11-24T07:58:47.831 INFO:teuthology.orchestra.run.smithi059.stdout: valid_lft forever preferred_lft forever 2023-11-24T07:58:47.831 INFO:teuthology.orchestra.run.smithi059.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-11-24T07:58:47.831 INFO:teuthology.orchestra.run.smithi059.stdout: link/ether 92:6a:7d:e0:5d:9b brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2023-11-24T07:58:47.831 INFO:teuthology.orchestra.run.smithi059.stdout: inet6 fe80::906a:7dff:fee0:5d9b/64 scope link 2023-11-24T07:58:47.831 INFO:teuthology.orchestra.run.smithi059.stdout: valid_lft forever preferred_lft forever 2023-11-24T07:58:47.831 INFO:teuthology.orchestra.run.smithi059.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-11-24T07:58:47.831 INFO:teuthology.orchestra.run.smithi059.stdout: link/ether 72:f2:ce:27:f6:18 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2023-11-24T07:58:47.831 INFO:teuthology.orchestra.run.smithi059.stdout: inet6 fe80::70f2:ceff:fe27:f618/64 scope link 2023-11-24T07:58:47.831 INFO:teuthology.orchestra.run.smithi059.stdout: valid_lft forever preferred_lft forever 2023-11-24T07:58:47.831 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T07:58:47.832 DEBUG:teuthology.orchestra.run.smithi059:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-11-24T07:58:47.895 INFO:teuthology.orchestra.run.smithi059.stdout:1 2023-11-24T07:58:47.896 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T07:58:47.897 DEBUG:teuthology.orchestra.run.smithi059:> route 2023-11-24T07:58:47.956 INFO:teuthology.orchestra.run.smithi059.stdout:Kernel IP routing table 2023-11-24T07:58:47.956 INFO:teuthology.orchestra.run.smithi059.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-11-24T07:58:47.956 INFO:teuthology.orchestra.run.smithi059.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2023-11-24T07:58:47.957 INFO:teuthology.orchestra.run.smithi059.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2023-11-24T07:58:47.957 INFO:teuthology.orchestra.run.smithi059.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-11-24T07:58:47.957 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T07:58:47.957 DEBUG:teuthology.orchestra.run.smithi059:> (cd / && exec bash -c ' 2023-11-24T07:58:47.957 DEBUG:teuthology.orchestra.run.smithi059:> set -e 2023-11-24T07:58:47.957 DEBUG:teuthology.orchestra.run.smithi059:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2023-11-24T07:58:47.957 DEBUG:teuthology.orchestra.run.smithi059:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2023-11-24T07:58:47.957 DEBUG:teuthology.orchestra.run.smithi059:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2023-11-24T07:58:47.957 DEBUG:teuthology.orchestra.run.smithi059:> ') 2023-11-24T07:58:48.062 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T07:58:48.063 DEBUG:teuthology.orchestra.run.smithi059:> ip netns list 2023-11-24T07:58:48.079 INFO:teuthology.orchestra.run.smithi059.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2023-11-24T07:58:48.079 INFO:teuthology.orchestra.run.smithi059.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-11-24T07:58:48.080 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-11-24T07:58:48.080 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T07:58:48.080 DEBUG:teuthology.orchestra.run.smithi059:> sudo ip link set brx.0 up 2023-11-24T07:58:48.146 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-11-24T07:58:48.147 DEBUG:teuthology.orchestra.run.smithi059:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2023-11-24T07:58:48.204 INFO:teuthology.orchestra.run.smithi059.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2023-11-24T07:58:48.204 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-11-24T07:58:48.204 DEBUG:teuthology.orchestra.run.smithi059:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2023-11-24T07:58:48.262 INFO:tasks.cephfs.kernel_mount:mounting using device: 0@.cephfs=/ 2023-11-24T07:58:48.262 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T07:58:48.262 DEBUG:teuthology.orchestra.run.smithi059:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 0@.cephfs=/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-11-24T07:58:48.349 INFO:teuthology.orchestra.run.smithi059.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-11-24T07:58:48.349 INFO:teuthology.orchestra.run.smithi059.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-11-24T07:58:48.379 INFO:teuthology.orchestra.run.smithi059.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-11-24T07:58:48.379 INFO:teuthology.orchestra.run.smithi059.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-11-24T07:58:48.379 INFO:teuthology.orchestra.run.smithi059.stdout:mount.ceph: resolved to: "172.21.15.59:3300,172.21.15.76:3300,172.21.15.76:3301" 2023-11-24T07:58:48.379 INFO:teuthology.orchestra.run.smithi059.stdout:mount.ceph: trying mount with new device syntax: 0@9f0e8741-b8f9-48d5-a2c3-09f45903303d.cephfs=/ 2023-11-24T07:58:48.379 INFO:teuthology.orchestra.run.smithi059.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,ms_mode=prefer-crc,key=0,mon_addr=172.21.15.59:3300/172.21.15.76:3300/172.21.15.76:3301" will pass to kernel 2023-11-24T07:58:48.379 INFO:teuthology.orchestra.run.smithi059.stdout:mount.ceph: v2 mount support check returned 2 2023-11-24T07:58:48.379 INFO:teuthology.orchestra.run.smithi059.stdout:mount.ceph: kernel does not support v2 syntax 2023-11-24T07:58:48.379 INFO:teuthology.orchestra.run.smithi059.stdout:mount.ceph: switching to using v1 address with old syntax 2023-11-24T07:58:48.429 INFO:teuthology.orchestra.run.smithi059.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-11-24T07:58:48.429 INFO:teuthology.orchestra.run.smithi059.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-11-24T07:58:48.429 INFO:teuthology.orchestra.run.smithi059.stdout:mount.ceph: resolved to: "172.21.15.59:3300,172.21.15.76:3300,172.21.15.76:3301" 2023-11-24T07:58:48.429 INFO:teuthology.orchestra.run.smithi059.stdout:mount.ceph: trying mount with new device syntax: 0@9f0e8741-b8f9-48d5-a2c3-09f45903303d.cephfs=/ 2023-11-24T07:58:48.429 INFO:teuthology.orchestra.run.smithi059.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,ms_mode=prefer-crc,key=0,mon_addr=172.21.15.59:3300/172.21.15.76:3300/172.21.15.76:3301" will pass to kernel 2023-11-24T07:58:48.429 INFO:teuthology.orchestra.run.smithi059.stdout:mount.ceph: v2 mount support check returned 2 2023-11-24T07:58:48.429 INFO:teuthology.orchestra.run.smithi059.stdout:mount.ceph: kernel does not support v2 syntax 2023-11-24T07:58:48.429 INFO:teuthology.orchestra.run.smithi059.stdout:mount.ceph: switching to using v1 address with old syntax 2023-11-24T07:58:48.429 INFO:teuthology.orchestra.run.smithi059.stdout:mount.ceph: resolved to: "172.21.15.59:6789,172.21.15.76:6789,172.21.15.76:6790" 2023-11-24T07:58:48.429 INFO:teuthology.orchestra.run.smithi059.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.59:6789,172.21.15.76:6789,172.21.15.76:6790:/ 2023-11-24T07:58:48.430 INFO:teuthology.orchestra.run.smithi059.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,key=0,mds_namespace=cephfs,fsid=9f0e8741-b8f9-48d5-a2c3-09f45903303d" will pass to kernel 2023-11-24T07:58:48.435 INFO:tasks.cephfs.kernel_mount:mount command passed 2023-11-24T07:58:48.435 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T07:58:48.436 DEBUG:teuthology.orchestra.run.smithi059:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2023-11-24T07:58:48.463 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T07:58:48.463 DEBUG:teuthology.orchestra.run.smithi059:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-11-24T07:58:48.545 INFO:teuthology.orchestra.run.smithi059.stdout:client6246 2023-11-24T07:58:48.546 DEBUG:teuthology.orchestra.run.smithi059:> set -ex 2023-11-24T07:58:48.546 DEBUG:teuthology.orchestra.run.smithi059:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-24T07:58:48.603 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-11-24T07:58:48.768 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:48.766+0000 7f509c5ac700 1 Processor -- start 2023-11-24T07:58:48.768 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:48.766+0000 7f509c5ac700 1 -- start start 2023-11-24T07:58:48.771 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:48.767+0000 7f509c5ac700 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f5094144c10 0x7f509406afd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:48.771 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:48.767+0000 7f509c5ac700 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f5094145100 0x7f509406b510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:48.771 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:48.767+0000 7f509c5ac700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f5094145650 0x7f5094071bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:48.771 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:48.767+0000 7f509c5ac700 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f509406f500 con 0x7f5094145650 2023-11-24T07:58:48.771 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:48.767+0000 7f509c5ac700 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f50940cd610 con 0x7f5094145100 2023-11-24T07:58:48.771 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:48.767+0000 7f509c5ac700 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5094160720 con 0x7f5094144c10 2023-11-24T07:58:48.772 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:48.768+0000 7f509ab49700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f5094145650 0x7f5094071bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:48.772 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:48.768+0000 7f509ab49700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f5094145650 0x7f5094071bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:54004/0 (socket says 172.21.15.59:54004) 2023-11-24T07:58:48.772 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:48.768+0000 7f509ab49700 1 -- 172.21.15.59:0/2936311345 learned_addr learned my addr 172.21.15.59:0/2936311345 (peer_addr_for_me v2:172.21.15.59:0/0) 2023-11-24T07:58:48.772 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:48.768+0000 7f509a348700 1 --2- 172.21.15.59:0/2936311345 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f5094144c10 0x7f509406afd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:48.772 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:48.768+0000 7f5099b47700 1 --2- 172.21.15.59:0/2936311345 >> v2:172.21.15.76:3300/0 conn(0x7f5094145100 0x7f509406b510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:48.772 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:48.768+0000 7f509ab49700 1 -- 172.21.15.59:0/2936311345 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f5094144c10 msgr2=0x7f509406afd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T07:58:48.772 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:48.768+0000 7f509ab49700 1 --2- 172.21.15.59:0/2936311345 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f5094144c10 0x7f509406afd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:48.772 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:48.768+0000 7f509ab49700 1 -- 172.21.15.59:0/2936311345 >> v2:172.21.15.76:3300/0 conn(0x7f5094145100 msgr2=0x7f509406b510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T07:58:48.772 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:48.768+0000 7f509ab49700 1 --2- 172.21.15.59:0/2936311345 >> v2:172.21.15.76:3300/0 conn(0x7f5094145100 0x7f509406b510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:48.773 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:48.768+0000 7f509ab49700 1 -- 172.21.15.59:0/2936311345 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f509406f710 con 0x7f5094145650 2023-11-24T07:58:48.773 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:48.768+0000 7f5099b47700 1 --2- 172.21.15.59:0/2936311345 >> v2:172.21.15.76:3300/0 conn(0x7f5094145100 0x7f509406b510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-24T07:58:48.773 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:48.768+0000 7f509ab49700 1 --2- 172.21.15.59:0/2936311345 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5094145650 0x7f5094071bb0 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 crypto rx=0x7f5094067cb0 tx=0x7f508c00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2afd34d07f38a52a server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:58:48.773 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:48.768+0000 7f5099346700 1 -- 172.21.15.59:0/2936311345 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f508c00e070 con 0x7f5094145650 2023-11-24T07:58:48.773 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:48.768+0000 7f5099346700 1 -- 172.21.15.59:0/2936311345 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f508c00b9c0 con 0x7f5094145650 2023-11-24T07:58:48.773 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:48.768+0000 7f5099346700 1 -- 172.21.15.59:0/2936311345 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f508c00bd70 con 0x7f5094145650 2023-11-24T07:58:48.773 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:48.769+0000 7f509c5ac700 1 -- 172.21.15.59:0/2936311345 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5094145650 msgr2=0x7f5094071bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:48.773 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:48.769+0000 7f509c5ac700 1 --2- 172.21.15.59:0/2936311345 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5094145650 0x7f5094071bb0 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 crypto rx=0x7f5094067cb0 tx=0x7f508c00b330 comp rx=0 tx=0).stop 2023-11-24T07:58:48.773 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:48.769+0000 7f509c5ac700 1 -- 172.21.15.59:0/2936311345 shutdown_connections 2023-11-24T07:58:48.773 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:48.769+0000 7f509c5ac700 1 --2- 172.21.15.59:0/2936311345 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5094145650 0x7f5094071bb0 unknown :-1 s=CLOSED pgs=729 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:48.773 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:48.769+0000 7f509c5ac700 1 --2- 172.21.15.59:0/2936311345 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f5094144c10 0x7f509406afd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:48.773 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:48.769+0000 7f509c5ac700 1 --2- 172.21.15.59:0/2936311345 >> v2:172.21.15.76:3300/0 conn(0x7f5094145100 0x7f509406b510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:48.774 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:48.769+0000 7f509c5ac700 1 -- 172.21.15.59:0/2936311345 >> 172.21.15.59:0/2936311345 conn(0x7f50940b2740 msgr2=0x7f50940b2b20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:58:48.774 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:48.769+0000 7f509c5ac700 1 -- 172.21.15.59:0/2936311345 shutdown_connections 2023-11-24T07:58:48.774 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:48.769+0000 7f509c5ac700 1 -- 172.21.15.59:0/2936311345 wait complete. 2023-11-24T07:58:48.777 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:48.770+0000 7f509c5ac700 1 Processor -- start 2023-11-24T07:58:48.777 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:48.770+0000 7f509c5ac700 1 -- start start 2023-11-24T07:58:48.777 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:48.771+0000 7f509c5ac700 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f5094144c10 0x7f509415b8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:48.777 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:48.771+0000 7f509c5ac700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5094145100 0x7f509413f260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:48.777 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:48.771+0000 7f509c5ac700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5094145650 0x7f509413ddb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:48.777 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:48.771+0000 7f509c5ac700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f509413f7a0 con 0x7f5094145650 2023-11-24T07:58:48.777 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:48.771+0000 7f509c5ac700 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50940a9880 con 0x7f5094145100 2023-11-24T07:58:48.777 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:48.771+0000 7f509c5ac700 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f509414b290 con 0x7f5094144c10 2023-11-24T07:58:48.778 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:48.771+0000 7f509ab49700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5094145650 0x7f509413ddb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:48.778 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:48.771+0000 7f509ab49700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5094145650 0x7f509413ddb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:54010/0 (socket says 172.21.15.59:54010) 2023-11-24T07:58:48.778 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:48.771+0000 7f509ab49700 1 -- 172.21.15.59:0/746472078 learned_addr learned my addr 172.21.15.59:0/746472078 (peer_addr_for_me v2:172.21.15.59:0/0) 2023-11-24T07:58:48.778 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:48.771+0000 7f509ab49700 1 -- 172.21.15.59:0/746472078 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f5094144c10 msgr2=0x7f509415b8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:48.778 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:48.771+0000 7f509ab49700 1 --2- 172.21.15.59:0/746472078 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f5094144c10 0x7f509415b8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:48.778 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:48.771+0000 7f509ab49700 1 -- 172.21.15.59:0/746472078 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5094145100 msgr2=0x7f509413f260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:48.778 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:48.771+0000 7f5099b47700 1 --2- 172.21.15.59:0/746472078 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5094145100 0x7f509413f260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:48.778 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:48.771+0000 7f509ab49700 1 --2- 172.21.15.59:0/746472078 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5094145100 0x7f509413f260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:48.778 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:48.771+0000 7f509ab49700 1 -- 172.21.15.59:0/746472078 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f508c009b30 con 0x7f5094145650 2023-11-24T07:58:48.779 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:48.772+0000 7f509ab49700 1 --2- 172.21.15.59:0/746472078 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5094145650 0x7f509413ddb0 secure :-1 s=READY pgs=730 cs=0 l=1 rev1=1 crypto rx=0x7f508c00b370 tx=0x7f508c004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:58:48.779 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:48.772+0000 7f5086ffd700 1 -- 172.21.15.59:0/746472078 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f508c004250 con 0x7f5094145650 2023-11-24T07:58:48.779 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:48.772+0000 7f5086ffd700 1 -- 172.21.15.59:0/746472078 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f508c0043b0 con 0x7f5094145650 2023-11-24T07:58:48.779 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:48.772+0000 7f509c5ac700 1 -- 172.21.15.59:0/746472078 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f509415f290 con 0x7f5094145650 2023-11-24T07:58:48.779 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:48.772+0000 7f5086ffd700 1 -- 172.21.15.59:0/746472078 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f508c004760 con 0x7f5094145650 2023-11-24T07:58:48.779 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:48.772+0000 7f509c5ac700 1 -- 172.21.15.59:0/746472078 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f509414b4e0 con 0x7f5094145650 2023-11-24T07:58:48.779 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:48.773+0000 7f509c5ac700 1 -- 172.21.15.59:0/746472078 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5094068670 con 0x7f5094145650 2023-11-24T07:58:48.779 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:48.773+0000 7f5086ffd700 1 -- 172.21.15.59:0/746472078 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 8) v1 ==== 89435+0+0 (secure 0 0 0) 0x7f508c01b430 con 0x7f5094145650 2023-11-24T07:58:48.779 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:48.774+0000 7f5086ffd700 1 --2- 172.21.15.59:0/746472078 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f507806b110 0x7f507806d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:48.779 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:48.774+0000 7f5086ffd700 1 -- 172.21.15.59:0/746472078 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(60..60 src has 1..60) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f508c08ac90 con 0x7f5094145650 2023-11-24T07:58:48.779 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:48.774+0000 7f509a348700 1 --2- 172.21.15.59:0/746472078 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f507806b110 0x7f507806d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:48.779 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:48.774+0000 7f509a348700 1 --2- 172.21.15.59:0/746472078 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f507806b110 0x7f507806d5d0 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f5090009770 tx=0x7f5090008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:58:48.780 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:48.778+0000 7f5086ffd700 1 -- 172.21.15.59:0/746472078 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f508c055960 con 0x7f5094145650 2023-11-24T07:58:49.026 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:49.024+0000 7f509c5ac700 1 -- 172.21.15.59:0/746472078 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f50940d0840 con 0x7f5094145650 2023-11-24T07:58:49.026 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:49.024+0000 7f5086ffd700 1 -- 172.21.15.59:0/746472078 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v35) v1 ==== 53+0+73 (secure 0 0 0) 0x7f508c01f070 con 0x7f5094145650 2023-11-24T07:58:49.026 INFO:teuthology.orchestra.run.smithi059.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-11-24T07:58:49.028 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:49.026+0000 7f509c5ac700 1 -- 172.21.15.59:0/746472078 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f507806b110 msgr2=0x7f507806d5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:49.028 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:49.026+0000 7f509c5ac700 1 --2- 172.21.15.59:0/746472078 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f507806b110 0x7f507806d5d0 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f5090009770 tx=0x7f5090008040 comp rx=0 tx=0).stop 2023-11-24T07:58:49.028 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:49.026+0000 7f509c5ac700 1 -- 172.21.15.59:0/746472078 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5094145650 msgr2=0x7f509413ddb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:49.028 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:49.026+0000 7f509c5ac700 1 --2- 172.21.15.59:0/746472078 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5094145650 0x7f509413ddb0 secure :-1 s=READY pgs=730 cs=0 l=1 rev1=1 crypto rx=0x7f508c00b370 tx=0x7f508c004ad0 comp rx=0 tx=0).stop 2023-11-24T07:58:49.029 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:49.026+0000 7f509c5ac700 1 -- 172.21.15.59:0/746472078 shutdown_connections 2023-11-24T07:58:49.029 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:49.026+0000 7f509c5ac700 1 --2- 172.21.15.59:0/746472078 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f507806b110 0x7f507806d5d0 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:49.029 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:49.026+0000 7f509c5ac700 1 --2- 172.21.15.59:0/746472078 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5094145650 0x7f509413ddb0 unknown :-1 s=CLOSED pgs=730 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:49.029 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:49.026+0000 7f509c5ac700 1 --2- 172.21.15.59:0/746472078 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f5094144c10 0x7f509415b8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:49.029 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:49.026+0000 7f509c5ac700 1 --2- 172.21.15.59:0/746472078 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5094145100 0x7f509413f260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:49.029 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:49.026+0000 7f509c5ac700 1 -- 172.21.15.59:0/746472078 >> 172.21.15.59:0/746472078 conn(0x7f50940b2740 msgr2=0x7f50940cfce0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:58:49.029 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:49.026+0000 7f509c5ac700 1 -- 172.21.15.59:0/746472078 shutdown_connections 2023-11-24T07:58:49.029 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:49.026+0000 7f509c5ac700 1 -- 172.21.15.59:0/746472078 wait complete. 2023-11-24T07:58:49.040 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2023-11-24T07:58:49.040 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-11-24T07:58:49.040 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi076.front.sepia.ceph.com 2023-11-24T07:58:49.040 INFO:tasks.cephfs.mount:self.client.name = client.1 2023-11-24T07:58:49.040 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2023-11-24T07:58:49.040 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2023-11-24T07:58:49.040 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2023-11-24T07:58:49.041 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-11-24T07:58:49.041 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2023-11-24T07:58:49.041 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T07:58:49.041 DEBUG:teuthology.orchestra.run.smithi076:> ip addr 2023-11-24T07:58:49.062 INFO:teuthology.orchestra.run.smithi076.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-11-24T07:58:49.063 INFO:teuthology.orchestra.run.smithi076.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-11-24T07:58:49.063 INFO:teuthology.orchestra.run.smithi076.stdout: inet 127.0.0.1/8 scope host lo 2023-11-24T07:58:49.063 INFO:teuthology.orchestra.run.smithi076.stdout: valid_lft forever preferred_lft forever 2023-11-24T07:58:49.063 INFO:teuthology.orchestra.run.smithi076.stdout: inet6 ::1/128 scope host 2023-11-24T07:58:49.063 INFO:teuthology.orchestra.run.smithi076.stdout: valid_lft forever preferred_lft forever 2023-11-24T07:58:49.063 INFO:teuthology.orchestra.run.smithi076.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-11-24T07:58:49.063 INFO:teuthology.orchestra.run.smithi076.stdout: link/ether 0c:c4:7a:d8:da:66 brd ff:ff:ff:ff:ff:ff 2023-11-24T07:58:49.063 INFO:teuthology.orchestra.run.smithi076.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-11-24T07:58:49.063 INFO:teuthology.orchestra.run.smithi076.stdout: link/ether 0c:c4:7a:d8:da:67 brd ff:ff:ff:ff:ff:ff 2023-11-24T07:58:49.063 INFO:teuthology.orchestra.run.smithi076.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-11-24T07:58:49.063 INFO:teuthology.orchestra.run.smithi076.stdout: link/ether 0c:c4:7a:8f:47:e0 brd ff:ff:ff:ff:ff:ff 2023-11-24T07:58:49.063 INFO:teuthology.orchestra.run.smithi076.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-11-24T07:58:49.064 INFO:teuthology.orchestra.run.smithi076.stdout: link/ether 0c:c4:7a:8f:47:e1 brd ff:ff:ff:ff:ff:ff 2023-11-24T07:58:49.064 INFO:teuthology.orchestra.run.smithi076.stdout: inet 172.21.15.76/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2023-11-24T07:58:49.064 INFO:teuthology.orchestra.run.smithi076.stdout: valid_lft 42270sec preferred_lft 42270sec 2023-11-24T07:58:49.064 INFO:teuthology.orchestra.run.smithi076.stdout: inet6 fe80::a34d:4ce5:8360:8f9/64 scope link noprefixroute 2023-11-24T07:58:49.064 INFO:teuthology.orchestra.run.smithi076.stdout: valid_lft forever preferred_lft forever 2023-11-24T07:58:49.064 INFO:teuthology.orchestra.run.smithi076.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-11-24T07:58:49.064 INFO:teuthology.orchestra.run.smithi076.stdout: link/ether 2a:59:73:ff:f7:95 brd ff:ff:ff:ff:ff:ff 2023-11-24T07:58:49.064 INFO:teuthology.orchestra.run.smithi076.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-11-24T07:58:49.064 INFO:teuthology.orchestra.run.smithi076.stdout: valid_lft forever preferred_lft forever 2023-11-24T07:58:49.064 INFO:teuthology.orchestra.run.smithi076.stdout: inet6 fe80::4c23:44ff:feb9:d9d4/64 scope link 2023-11-24T07:58:49.064 INFO:teuthology.orchestra.run.smithi076.stdout: valid_lft forever preferred_lft forever 2023-11-24T07:58:49.064 INFO:teuthology.orchestra.run.smithi076.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-11-24T07:58:49.064 INFO:teuthology.orchestra.run.smithi076.stdout: link/ether 46:3e:09:af:65:64 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2023-11-24T07:58:49.064 INFO:teuthology.orchestra.run.smithi076.stdout: inet6 fe80::443e:9ff:feaf:6564/64 scope link 2023-11-24T07:58:49.065 INFO:teuthology.orchestra.run.smithi076.stdout: valid_lft forever preferred_lft forever 2023-11-24T07:58:49.065 INFO:teuthology.orchestra.run.smithi076.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-11-24T07:58:49.065 INFO:teuthology.orchestra.run.smithi076.stdout: link/ether 2a:59:73:ff:f7:95 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.3 2023-11-24T07:58:49.065 INFO:teuthology.orchestra.run.smithi076.stdout: inet6 fe80::2859:73ff:feff:f795/64 scope link 2023-11-24T07:58:49.065 INFO:teuthology.orchestra.run.smithi076.stdout: valid_lft forever preferred_lft forever 2023-11-24T07:58:49.065 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T07:58:49.065 DEBUG:teuthology.orchestra.run.smithi076:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-11-24T07:58:49.133 INFO:teuthology.orchestra.run.smithi076.stdout:1 2023-11-24T07:58:49.136 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T07:58:49.136 DEBUG:teuthology.orchestra.run.smithi076:> route 2023-11-24T07:58:49.197 INFO:teuthology.orchestra.run.smithi076.stdout:Kernel IP routing table 2023-11-24T07:58:49.197 INFO:teuthology.orchestra.run.smithi076.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-11-24T07:58:49.197 INFO:teuthology.orchestra.run.smithi076.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-11-24T07:58:49.197 INFO:teuthology.orchestra.run.smithi076.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-11-24T07:58:49.197 INFO:teuthology.orchestra.run.smithi076.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-11-24T07:58:49.198 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T07:58:49.198 DEBUG:teuthology.orchestra.run.smithi076:> (cd / && exec bash -c ' 2023-11-24T07:58:49.198 DEBUG:teuthology.orchestra.run.smithi076:> set -e 2023-11-24T07:58:49.198 DEBUG:teuthology.orchestra.run.smithi076:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-11-24T07:58:49.198 DEBUG:teuthology.orchestra.run.smithi076:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-11-24T07:58:49.198 DEBUG:teuthology.orchestra.run.smithi076:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-11-24T07:58:49.198 DEBUG:teuthology.orchestra.run.smithi076:> ') 2023-11-24T07:58:49.317 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T07:58:49.317 DEBUG:teuthology.orchestra.run.smithi076:> ip netns list 2023-11-24T07:58:49.337 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-ns--home-ubuntu-cephtest-mnt.3 (id: 1) 2023-11-24T07:58:49.337 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2023-11-24T07:58:49.338 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2023-11-24T07:58:49.338 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T07:58:49.338 DEBUG:teuthology.orchestra.run.smithi076:> sudo ip link set brx.0 up 2023-11-24T07:58:49.409 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-11-24T07:58:49.409 DEBUG:teuthology.orchestra.run.smithi076:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2023-11-24T07:58:49.468 INFO:teuthology.orchestra.run.smithi076.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2023-11-24T07:58:49.468 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-11-24T07:58:49.468 DEBUG:teuthology.orchestra.run.smithi076:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2023-11-24T07:58:49.526 INFO:tasks.cephfs.kernel_mount:mounting using device: 1@.cephfs=/ 2023-11-24T07:58:49.527 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T07:58:49.527 DEBUG:teuthology.orchestra.run.smithi076:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 1@.cephfs=/ /home/ubuntu/cephtest/mnt.1 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-11-24T07:58:49.614 INFO:teuthology.orchestra.run.smithi076.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-11-24T07:58:49.615 INFO:teuthology.orchestra.run.smithi076.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-11-24T07:58:49.641 INFO:teuthology.orchestra.run.smithi076.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-11-24T07:58:49.641 INFO:teuthology.orchestra.run.smithi076.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-11-24T07:58:49.642 INFO:teuthology.orchestra.run.smithi076.stdout:mount.ceph: resolved to: "172.21.15.59:3300,172.21.15.76:3300,172.21.15.76:3301" 2023-11-24T07:58:49.642 INFO:teuthology.orchestra.run.smithi076.stdout:mount.ceph: trying mount with new device syntax: 1@9f0e8741-b8f9-48d5-a2c3-09f45903303d.cephfs=/ 2023-11-24T07:58:49.642 INFO:teuthology.orchestra.run.smithi076.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,ms_mode=prefer-crc,key=1,mon_addr=172.21.15.59:3300/172.21.15.76:3300/172.21.15.76:3301" will pass to kernel 2023-11-24T07:58:49.642 INFO:teuthology.orchestra.run.smithi076.stdout:mount.ceph: v2 mount support check returned 2 2023-11-24T07:58:49.642 INFO:teuthology.orchestra.run.smithi076.stdout:mount.ceph: kernel does not support v2 syntax 2023-11-24T07:58:49.642 INFO:teuthology.orchestra.run.smithi076.stdout:mount.ceph: switching to using v1 address with old syntax 2023-11-24T07:58:49.682 INFO:teuthology.orchestra.run.smithi076.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-11-24T07:58:49.682 INFO:teuthology.orchestra.run.smithi076.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-11-24T07:58:49.682 INFO:teuthology.orchestra.run.smithi076.stdout:mount.ceph: resolved to: "172.21.15.59:3300,172.21.15.76:3300,172.21.15.76:3301" 2023-11-24T07:58:49.682 INFO:teuthology.orchestra.run.smithi076.stdout:mount.ceph: trying mount with new device syntax: 1@9f0e8741-b8f9-48d5-a2c3-09f45903303d.cephfs=/ 2023-11-24T07:58:49.682 INFO:teuthology.orchestra.run.smithi076.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,ms_mode=prefer-crc,key=1,mon_addr=172.21.15.59:3300/172.21.15.76:3300/172.21.15.76:3301" will pass to kernel 2023-11-24T07:58:49.682 INFO:teuthology.orchestra.run.smithi076.stdout:mount.ceph: v2 mount support check returned 2 2023-11-24T07:58:49.682 INFO:teuthology.orchestra.run.smithi076.stdout:mount.ceph: kernel does not support v2 syntax 2023-11-24T07:58:49.683 INFO:teuthology.orchestra.run.smithi076.stdout:mount.ceph: switching to using v1 address with old syntax 2023-11-24T07:58:49.683 INFO:teuthology.orchestra.run.smithi076.stdout:mount.ceph: resolved to: "172.21.15.59:6789,172.21.15.76:6789,172.21.15.76:6790" 2023-11-24T07:58:49.683 INFO:teuthology.orchestra.run.smithi076.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.59:6789,172.21.15.76:6789,172.21.15.76:6790:/ 2023-11-24T07:58:49.683 INFO:teuthology.orchestra.run.smithi076.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,key=1,mds_namespace=cephfs,fsid=9f0e8741-b8f9-48d5-a2c3-09f45903303d" will pass to kernel 2023-11-24T07:58:49.685 INFO:tasks.cephfs.kernel_mount:mount command passed 2023-11-24T07:58:49.685 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T07:58:49.685 DEBUG:teuthology.orchestra.run.smithi076:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2023-11-24T07:58:49.717 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T07:58:49.717 DEBUG:teuthology.orchestra.run.smithi076:> (cd /home/ubuntu/cephtest/mnt.1 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-11-24T07:58:49.790 INFO:teuthology.orchestra.run.smithi076.stdout:client5836 2023-11-24T07:58:49.790 DEBUG:teuthology.orchestra.run.smithi076:> set -ex 2023-11-24T07:58:49.790 DEBUG:teuthology.orchestra.run.smithi076:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-24T07:58:49.848 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs flag set enable_multiple true --yes-i-really-mean-it 2023-11-24T07:58:49.991 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:49.990+0000 7efcdc116700 1 Processor -- start 2023-11-24T07:58:49.992 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:49.990+0000 7efcdc116700 1 -- start start 2023-11-24T07:58:49.992 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:49.991+0000 7efcdc116700 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7efcd406afd0 0x7efcd40b59d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:49.992 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:49.991+0000 7efcdc116700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7efcd406b3b0 0x7efcd415e470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:49.992 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:49.991+0000 7efcdc116700 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efcd4071ec0 con 0x7efcd4072350 2023-11-24T07:58:49.993 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:49.991+0000 7efcdc116700 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efcd4071b90 con 0x7efcd406b3b0 2023-11-24T07:58:49.993 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:49.991+0000 7efcdc116700 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efcd415ed30 con 0x7efcd406afd0 2023-11-24T07:58:49.993 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:49.991+0000 7efcd96b1700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7efcd406b3b0 0x7efcd415e470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:49.993 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:49.991+0000 7efcda6b3700 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7efcd4072350 0x7efcd406dbd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:57490/0 (socket says 172.21.15.59:57490) 2023-11-24T07:58:49.993 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:49.991+0000 7efcda6b3700 1 -- 172.21.15.59:0/1997759674 learned_addr learned my addr 172.21.15.59:0/1997759674 (peer_addr_for_me v1:172.21.15.59:0/0) 2023-11-24T07:58:49.995 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:49.991+0000 7efcd9eb2700 1 --2- 172.21.15.59:0/1997759674 >> v2:172.21.15.76:3300/0 conn(0x7efcd406afd0 0x7efcd40b59d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:49.995 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:49.991+0000 7efcd96b1700 1 -- 172.21.15.59:0/1997759674 >> v2:172.21.15.76:3300/0 conn(0x7efcd406afd0 msgr2=0x7efcd40b59d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T07:58:49.995 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:49.991+0000 7efcd96b1700 1 --2- 172.21.15.59:0/1997759674 >> v2:172.21.15.76:3300/0 conn(0x7efcd406afd0 0x7efcd40b59d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:49.995 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:49.991+0000 7efcd96b1700 1 -- 172.21.15.59:0/1997759674 >> v1:172.21.15.59:6789/0 conn(0x7efcd4072350 legacy=0x7efcd406dbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:49.996 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:49.991+0000 7efcd8eb0700 1 -- 172.21.15.59:0/1997759674 <== mon.0 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2152246511 0 0) 0x7efcd4071ec0 con 0x7efcd4072350 2023-11-24T07:58:49.996 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:49.991+0000 7efcd96b1700 1 -- 172.21.15.59:0/1997759674 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efcd415f010 con 0x7efcd406b3b0 2023-11-24T07:58:49.996 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:49.991+0000 7efcd9eb2700 1 --2- 172.21.15.59:0/1997759674 >> v2:172.21.15.76:3300/0 conn(0x7efcd406afd0 0x7efcd40b59d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T07:58:49.996 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:49.991+0000 7efcd96b1700 1 --2- 172.21.15.59:0/1997759674 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7efcd406b3b0 0x7efcd415e470 secure :-1 s=READY pgs=732 cs=0 l=1 rev1=1 crypto rx=0x7efcd40b33c0 tx=0x7efcc4009b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7793afa42393b5f4 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:58:49.996 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:49.992+0000 7efcd8eb0700 1 -- 172.21.15.59:0/1997759674 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efcc4011830 con 0x7efcd406b3b0 2023-11-24T07:58:49.996 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:49.992+0000 7efcd8eb0700 1 -- 172.21.15.59:0/1997759674 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efcc4011990 con 0x7efcd406b3b0 2023-11-24T07:58:49.996 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:49.992+0000 7efcd8eb0700 1 -- 172.21.15.59:0/1997759674 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efcc4011d20 con 0x7efcd406b3b0 2023-11-24T07:58:49.996 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:49.992+0000 7efcdc116700 1 -- 172.21.15.59:0/1997759674 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7efcd406b3b0 msgr2=0x7efcd415e470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:49.996 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:49.992+0000 7efcdc116700 1 --2- 172.21.15.59:0/1997759674 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7efcd406b3b0 0x7efcd415e470 secure :-1 s=READY pgs=732 cs=0 l=1 rev1=1 crypto rx=0x7efcd40b33c0 tx=0x7efcc4009b20 comp rx=0 tx=0).stop 2023-11-24T07:58:49.996 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:49.992+0000 7efcdc116700 1 -- 172.21.15.59:0/1997759674 shutdown_connections 2023-11-24T07:58:49.996 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:49.992+0000 7efcdc116700 1 --2- 172.21.15.59:0/1997759674 >> v2:172.21.15.76:3300/0 conn(0x7efcd406afd0 0x7efcd40b59d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:49.996 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:49.992+0000 7efcdc116700 1 --2- 172.21.15.59:0/1997759674 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7efcd406b3b0 0x7efcd415e470 unknown :-1 s=CLOSED pgs=732 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:49.997 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:49.992+0000 7efcdc116700 1 -- 172.21.15.59:0/1997759674 >> 172.21.15.59:0/1997759674 conn(0x7efcd40b2760 msgr2=0x7efcd40b2b70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:58:49.997 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:49.992+0000 7efcdc116700 1 -- 172.21.15.59:0/1997759674 shutdown_connections 2023-11-24T07:58:49.997 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:49.992+0000 7efcdc116700 1 -- 172.21.15.59:0/1997759674 wait complete. 2023-11-24T07:58:49.997 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:49.993+0000 7efcdc116700 1 Processor -- start 2023-11-24T07:58:49.997 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:49.994+0000 7efcdc116700 1 -- start start 2023-11-24T07:58:49.999 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:49.994+0000 7efcdc116700 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7efcd406afd0 0x7efcd415d900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:49.999 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:49.994+0000 7efcdc116700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7efcd406b3b0 0x7efcd4160020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:49.999 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:49.994+0000 7efcdc116700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7efcd4072350 0x7efcd4160560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:49.999 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:49.994+0000 7efcdc116700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efcd4160b90 con 0x7efcd4072350 2023-11-24T07:58:50.000 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:49.994+0000 7efcdc116700 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efcd4160e70 con 0x7efcd406b3b0 2023-11-24T07:58:50.000 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:49.994+0000 7efcdc116700 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efcd4133110 con 0x7efcd406afd0 2023-11-24T07:58:50.000 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:49.994+0000 7efcda6b3700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7efcd4072350 0x7efcd4160560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:50.000 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:49.994+0000 7efcda6b3700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7efcd4072350 0x7efcd4160560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:54022/0 (socket says 172.21.15.59:54022) 2023-11-24T07:58:50.000 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:49.994+0000 7efcda6b3700 1 -- 172.21.15.59:0/3499977227 learned_addr learned my addr 172.21.15.59:0/3499977227 (peer_addr_for_me v2:172.21.15.59:0/0) 2023-11-24T07:58:50.000 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:49.994+0000 7efcd96b1700 1 --2- 172.21.15.59:0/3499977227 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7efcd406b3b0 0x7efcd4160020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:50.000 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:49.994+0000 7efcd9eb2700 1 --2- 172.21.15.59:0/3499977227 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7efcd406afd0 0x7efcd415d900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:50.000 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:49.995+0000 7efcda6b3700 1 -- 172.21.15.59:0/3499977227 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7efcd406afd0 msgr2=0x7efcd415d900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:50.000 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:49.995+0000 7efcda6b3700 1 --2- 172.21.15.59:0/3499977227 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7efcd406afd0 0x7efcd415d900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:50.000 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:49.995+0000 7efcda6b3700 1 -- 172.21.15.59:0/3499977227 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7efcd406b3b0 msgr2=0x7efcd4160020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:50.001 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:49.995+0000 7efcda6b3700 1 --2- 172.21.15.59:0/3499977227 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7efcd406b3b0 0x7efcd4160020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:50.001 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:49.995+0000 7efcda6b3700 1 -- 172.21.15.59:0/3499977227 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efcc40094a0 con 0x7efcd4072350 2023-11-24T07:58:50.001 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:49.995+0000 7efcda6b3700 1 --2- 172.21.15.59:0/3499977227 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7efcd4072350 0x7efcd4160560 secure :-1 s=READY pgs=733 cs=0 l=1 rev1=1 crypto rx=0x7efccc00cb60 tx=0x7efccc00ce70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:58:50.001 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:49.995+0000 7efcca7fc700 1 -- 172.21.15.59:0/3499977227 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efccc00a9e0 con 0x7efcd4072350 2023-11-24T07:58:50.001 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:49.995+0000 7efcca7fc700 1 -- 172.21.15.59:0/3499977227 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efccc00ab40 con 0x7efcd4072350 2023-11-24T07:58:50.001 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:49.995+0000 7efcdc116700 1 -- 172.21.15.59:0/3499977227 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efcd41333f0 con 0x7efcd4072350 2023-11-24T07:58:50.001 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:49.995+0000 7efcca7fc700 1 -- 172.21.15.59:0/3499977227 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efccc0070d0 con 0x7efcd4072350 2023-11-24T07:58:50.001 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:49.995+0000 7efcdc116700 1 -- 172.21.15.59:0/3499977227 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efcd41434a0 con 0x7efcd4072350 2023-11-24T07:58:50.001 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:49.996+0000 7efcdc116700 1 -- 172.21.15.59:0/3499977227 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efcd4068670 con 0x7efcd4072350 2023-11-24T07:58:50.001 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:49.997+0000 7efcca7fc700 1 -- 172.21.15.59:0/3499977227 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 8) v1 ==== 89435+0+0 (secure 0 0 0) 0x7efccc007230 con 0x7efcd4072350 2023-11-24T07:58:50.001 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:49.997+0000 7efcca7fc700 1 --2- 172.21.15.59:0/3499977227 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7efcb806b110 0x7efcb806d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:50.002 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:49.997+0000 7efcca7fc700 1 -- 172.21.15.59:0/3499977227 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(60..60 src has 1..60) v4 ==== 5709+0+0 (secure 0 0 0) 0x7efccc0896a0 con 0x7efcd4072350 2023-11-24T07:58:50.002 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:49.997+0000 7efcd9eb2700 1 --2- 172.21.15.59:0/3499977227 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7efcb806b110 0x7efcb806d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:50.002 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:49.998+0000 7efcd9eb2700 1 --2- 172.21.15.59:0/3499977227 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7efcb806b110 0x7efcb806d5d0 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7efcd0006fd0 tx=0x7efcd0008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:58:50.003 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:50.002+0000 7efcca7fc700 1 -- 172.21.15.59:0/3499977227 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7efccc0543f0 con 0x7efcd4072350 2023-11-24T07:58:50.250 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:50.249+0000 7efcdc116700 1 -- 172.21.15.59:0/3499977227 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "fs flag set", "flag_name": "enable_multiple", "val": "true", "yes_i_really_mean_it": true} v 0) v1 -- 0x7efcd40b33f0 con 0x7efcd4072350 2023-11-24T07:58:50.428 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:50.427+0000 7efcca7fc700 1 -- 172.21.15.59:0/3499977227 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "fs flag set", "flag_name": "enable_multiple", "val": "true", "yes_i_really_mean_it": true}]=0 v36) v1 ==== 136+0+0 (secure 0 0 0) 0x7efccc057a10 con 0x7efcd4072350 2023-11-24T07:58:50.431 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:50.429+0000 7efcdc116700 1 -- 172.21.15.59:0/3499977227 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7efcb806b110 msgr2=0x7efcb806d5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:50.431 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:50.429+0000 7efcdc116700 1 --2- 172.21.15.59:0/3499977227 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7efcb806b110 0x7efcb806d5d0 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7efcd0006fd0 tx=0x7efcd0008040 comp rx=0 tx=0).stop 2023-11-24T07:58:50.432 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:50.429+0000 7efcdc116700 1 -- 172.21.15.59:0/3499977227 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7efcd4072350 msgr2=0x7efcd4160560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:50.432 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:50.429+0000 7efcdc116700 1 --2- 172.21.15.59:0/3499977227 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7efcd4072350 0x7efcd4160560 secure :-1 s=READY pgs=733 cs=0 l=1 rev1=1 crypto rx=0x7efccc00cb60 tx=0x7efccc00ce70 comp rx=0 tx=0).stop 2023-11-24T07:58:50.432 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:50.429+0000 7efcdc116700 1 -- 172.21.15.59:0/3499977227 shutdown_connections 2023-11-24T07:58:50.432 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:50.429+0000 7efcdc116700 1 --2- 172.21.15.59:0/3499977227 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7efcb806b110 0x7efcb806d5d0 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:50.432 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:50.429+0000 7efcdc116700 1 --2- 172.21.15.59:0/3499977227 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7efcd4072350 0x7efcd4160560 unknown :-1 s=CLOSED pgs=733 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:50.432 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:50.429+0000 7efcdc116700 1 --2- 172.21.15.59:0/3499977227 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7efcd406afd0 0x7efcd415d900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:50.432 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:50.429+0000 7efcdc116700 1 --2- 172.21.15.59:0/3499977227 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7efcd406b3b0 0x7efcd4160020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:50.432 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:50.429+0000 7efcdc116700 1 -- 172.21.15.59:0/3499977227 >> 172.21.15.59:0/3499977227 conn(0x7efcd40b2760 msgr2=0x7efcd406e460 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:58:50.432 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:50.429+0000 7efcdc116700 1 -- 172.21.15.59:0/3499977227 shutdown_connections 2023-11-24T07:58:50.433 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:50.429+0000 7efcdc116700 1 -- 172.21.15.59:0/3499977227 wait complete. 2023-11-24T07:58:50.443 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume ls 2023-11-24T07:58:50.569 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:50.567+0000 7f3590cb9700 1 Processor -- start 2023-11-24T07:58:50.569 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:50.567+0000 7f3590cb9700 1 -- start start 2023-11-24T07:58:50.571 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:50.568+0000 7f3590cb9700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f358c0cfc60 0x7f358c0d0110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:50.572 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:50.568+0000 7f3590cb9700 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f358c06f610 con 0x7f358c0d0650 2023-11-24T07:58:50.572 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:50.568+0000 7f3590cb9700 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f358c0cdac0 con 0x7f358c15c010 2023-11-24T07:58:50.572 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:50.568+0000 7f3590cb9700 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f358c06ece0 con 0x7f358c0cfc60 2023-11-24T07:58:50.572 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:50.568+0000 7f3589d9b700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f358c0cfc60 0x7f358c0d0110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:50.572 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:50.568+0000 7f358ad9d700 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7f358c0d0650 0x7f358c06faa0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:57504/0 (socket says 172.21.15.59:57504) 2023-11-24T07:58:50.572 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:50.568+0000 7f358ad9d700 1 -- 172.21.15.59:0/3925604100 learned_addr learned my addr 172.21.15.59:0/3925604100 (peer_addr_for_me v1:172.21.15.59:0/0) 2023-11-24T07:58:50.572 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:50.569+0000 7f3589d9b700 1 -- 172.21.15.59:0/3925604100 >> v1:172.21.15.76:6789/0 conn(0x7f358c15c010 legacy=0x7f358c0cd2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:50.572 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:50.569+0000 7f3589d9b700 1 -- 172.21.15.59:0/3925604100 >> v1:172.21.15.59:6789/0 conn(0x7f358c0d0650 legacy=0x7f358c06faa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:50.572 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:50.569+0000 7f3589d9b700 1 -- 172.21.15.59:0/3925604100 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f358c06efc0 con 0x7f358c0cfc60 2023-11-24T07:58:50.572 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:50.569+0000 7f3589d9b700 1 --2- 172.21.15.59:0/3925604100 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f358c0cfc60 0x7f358c0d0110 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 crypto rx=0x7f3580009840 tx=0x7f3580009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=27a1fe8c5a15e46b server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:58:50.572 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:50.569+0000 7f358959a700 1 -- 172.21.15.59:0/3925604100 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3580011830 con 0x7f358c0cfc60 2023-11-24T07:58:50.573 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:50.569+0000 7f358959a700 1 -- 172.21.15.59:0/3925604100 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3580011990 con 0x7f358c0cfc60 2023-11-24T07:58:50.573 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:50.569+0000 7f358959a700 1 -- 172.21.15.59:0/3925604100 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3580011d20 con 0x7f358c0cfc60 2023-11-24T07:58:50.573 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:50.570+0000 7f3590cb9700 1 -- 172.21.15.59:0/3925604100 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f358c0cfc60 msgr2=0x7f358c0d0110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:50.573 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:50.570+0000 7f3590cb9700 1 --2- 172.21.15.59:0/3925604100 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f358c0cfc60 0x7f358c0d0110 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 crypto rx=0x7f3580009840 tx=0x7f3580009b50 comp rx=0 tx=0).stop 2023-11-24T07:58:50.573 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:50.570+0000 7f3590cb9700 1 -- 172.21.15.59:0/3925604100 shutdown_connections 2023-11-24T07:58:50.573 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:50.570+0000 7f3590cb9700 1 --2- 172.21.15.59:0/3925604100 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f358c0cfc60 0x7f358c0d0110 unknown :-1 s=CLOSED pgs=735 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:50.573 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:50.570+0000 7f3590cb9700 1 -- 172.21.15.59:0/3925604100 >> 172.21.15.59:0/3925604100 conn(0x7f358c0b3790 msgr2=0x7f358c0b3ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:58:50.573 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:50.570+0000 7f3590cb9700 1 -- 172.21.15.59:0/3925604100 shutdown_connections 2023-11-24T07:58:50.573 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:50.570+0000 7f3590cb9700 1 -- 172.21.15.59:0/3925604100 wait complete. 2023-11-24T07:58:50.575 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:50.571+0000 7f3590cb9700 1 Processor -- start 2023-11-24T07:58:50.575 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:50.571+0000 7f3590cb9700 1 -- start start 2023-11-24T07:58:50.575 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:50.572+0000 7f3590cb9700 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f358c0cfc60 0x7f358c155d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:50.576 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:50.572+0000 7f3590cb9700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f358c0d0650 0x7f358c06d1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:50.576 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:50.572+0000 7f3590cb9700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f358c15c010 0x7f358c06d6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:50.576 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:50.572+0000 7f3590cb9700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f358c06cd90 con 0x7f358c0d0650 2023-11-24T07:58:50.576 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:50.572+0000 7f3590cb9700 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f358c156260 con 0x7f358c15c010 2023-11-24T07:58:50.576 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:50.572+0000 7f3590cb9700 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f358c06c180 con 0x7f358c0cfc60 2023-11-24T07:58:50.576 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:50.572+0000 7f3589d9b700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f358c0d0650 0x7f358c06d1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:50.576 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:50.572+0000 7f3589d9b700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f358c0d0650 0x7f358c06d1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:54032/0 (socket says 172.21.15.59:54032) 2023-11-24T07:58:50.576 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:50.572+0000 7f3589d9b700 1 -- 172.21.15.59:0/1158301832 learned_addr learned my addr 172.21.15.59:0/1158301832 (peer_addr_for_me v2:172.21.15.59:0/0) 2023-11-24T07:58:50.576 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:50.572+0000 7f358ad9d700 1 --2- 172.21.15.59:0/1158301832 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f358c15c010 0x7f358c06d6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:50.576 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:50.572+0000 7f358a59c700 1 --2- 172.21.15.59:0/1158301832 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f358c0cfc60 0x7f358c155d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:50.577 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:50.572+0000 7f3589d9b700 1 -- 172.21.15.59:0/1158301832 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f358c0cfc60 msgr2=0x7f358c155d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:50.577 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:50.572+0000 7f3589d9b700 1 --2- 172.21.15.59:0/1158301832 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f358c0cfc60 0x7f358c155d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:50.577 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:50.572+0000 7f3589d9b700 1 -- 172.21.15.59:0/1158301832 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f358c15c010 msgr2=0x7f358c06d6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:50.577 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:50.572+0000 7f3589d9b700 1 --2- 172.21.15.59:0/1158301832 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f358c15c010 0x7f358c06d6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:50.577 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:50.572+0000 7f3589d9b700 1 -- 172.21.15.59:0/1158301832 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f35800094a0 con 0x7f358c0d0650 2023-11-24T07:58:50.577 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:50.572+0000 7f358ad9d700 1 --2- 172.21.15.59:0/1158301832 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f358c15c010 0x7f358c06d6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-24T07:58:50.577 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:50.573+0000 7f3589d9b700 1 --2- 172.21.15.59:0/1158301832 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f358c0d0650 0x7f358c06d1a0 secure :-1 s=READY pgs=736 cs=0 l=1 rev1=1 crypto rx=0x7f35800090b0 tx=0x7f3580004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:58:50.577 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:50.573+0000 7f357affd700 1 -- 172.21.15.59:0/1158301832 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35800038a0 con 0x7f358c0d0650 2023-11-24T07:58:50.577 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:50.573+0000 7f357affd700 1 -- 172.21.15.59:0/1158301832 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f358001f070 con 0x7f358c0d0650 2023-11-24T07:58:50.577 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:50.573+0000 7f3590cb9700 1 -- 172.21.15.59:0/1158301832 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f358c06c460 con 0x7f358c0d0650 2023-11-24T07:58:50.577 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:50.573+0000 7f357affd700 1 -- 172.21.15.59:0/1158301832 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3580003c60 con 0x7f358c0d0650 2023-11-24T07:58:50.578 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:50.573+0000 7f3590cb9700 1 -- 172.21.15.59:0/1158301832 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f358c06ca20 con 0x7f358c0d0650 2023-11-24T07:58:50.578 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:50.574+0000 7f3590cb9700 1 -- 172.21.15.59:0/1158301832 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f358c068670 con 0x7f358c0d0650 2023-11-24T07:58:50.579 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:50.574+0000 7f357affd700 1 -- 172.21.15.59:0/1158301832 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 8) v1 ==== 89435+0+0 (secure 0 0 0) 0x7f358001a430 con 0x7f358c0d0650 2023-11-24T07:58:50.579 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:50.575+0000 7f357affd700 1 --2- 172.21.15.59:0/1158301832 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f356c06b110 0x7f356c06d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:50.579 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:50.575+0000 7f357affd700 1 -- 172.21.15.59:0/1158301832 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(60..60 src has 1..60) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f3580021070 con 0x7f358c0d0650 2023-11-24T07:58:50.579 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:50.575+0000 7f358a59c700 1 --2- 172.21.15.59:0/1158301832 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f356c06b110 0x7f356c06d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:50.580 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:50.575+0000 7f358a59c700 1 --2- 172.21.15.59:0/1158301832 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f356c06b110 0x7f356c06d5d0 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f357c000f60 tx=0x7f357c009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:58:50.581 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:50.579+0000 7f357affd700 1 -- 172.21.15.59:0/1158301832 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f3580056210 con 0x7f358c0d0650 2023-11-24T07:58:50.810 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:50.808+0000 7f3590cb9700 1 -- 172.21.15.59:0/1158301832 --> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] -- mgr_command(tid 0: {"prefix": "fs volume ls", "target": ["mon-mgr", ""]}) v1 -- 0x7f358c0aea70 con 0x7f356c06b110 2023-11-24T07:58:50.812 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:50.811+0000 7f357affd700 1 -- 172.21.15.59:0/1158301832 <== mgr.4100 v2:172.21.15.76:6832/4264046536 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+41 (secure 0 0 0) 0x7f357c0013e0 con 0x7f356c06b110 2023-11-24T07:58:50.812 INFO:teuthology.orchestra.run.smithi059.stdout:[ 2023-11-24T07:58:50.812 INFO:teuthology.orchestra.run.smithi059.stdout: { 2023-11-24T07:58:50.812 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "cephfs" 2023-11-24T07:58:50.812 INFO:teuthology.orchestra.run.smithi059.stdout: } 2023-11-24T07:58:50.812 INFO:teuthology.orchestra.run.smithi059.stdout:] 2023-11-24T07:58:50.814 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:50.812+0000 7f3590cb9700 1 -- 172.21.15.59:0/1158301832 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f356c06b110 msgr2=0x7f356c06d5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:50.814 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:50.812+0000 7f3590cb9700 1 --2- 172.21.15.59:0/1158301832 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f356c06b110 0x7f356c06d5d0 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f357c000f60 tx=0x7f357c009040 comp rx=0 tx=0).stop 2023-11-24T07:58:50.814 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:50.812+0000 7f3590cb9700 1 -- 172.21.15.59:0/1158301832 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f358c0d0650 msgr2=0x7f358c06d1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:50.814 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:50.812+0000 7f3590cb9700 1 --2- 172.21.15.59:0/1158301832 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f358c0d0650 0x7f358c06d1a0 secure :-1 s=READY pgs=736 cs=0 l=1 rev1=1 crypto rx=0x7f35800090b0 tx=0x7f3580004750 comp rx=0 tx=0).stop 2023-11-24T07:58:50.814 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:50.812+0000 7f3590cb9700 1 -- 172.21.15.59:0/1158301832 shutdown_connections 2023-11-24T07:58:50.814 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:50.812+0000 7f3590cb9700 1 --2- 172.21.15.59:0/1158301832 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f356c06b110 0x7f356c06d5d0 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:50.814 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:50.812+0000 7f3590cb9700 1 --2- 172.21.15.59:0/1158301832 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f358c15c010 0x7f358c06d6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:50.814 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:50.812+0000 7f3590cb9700 1 --2- 172.21.15.59:0/1158301832 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f358c0cfc60 0x7f358c155d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:50.815 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:50.812+0000 7f3590cb9700 1 --2- 172.21.15.59:0/1158301832 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f358c0d0650 0x7f358c06d1a0 unknown :-1 s=CLOSED pgs=736 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:50.815 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:50.812+0000 7f3590cb9700 1 -- 172.21.15.59:0/1158301832 >> 172.21.15.59:0/1158301832 conn(0x7f358c0b3790 msgr2=0x7f358c0b1700 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:58:50.815 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:50.812+0000 7f3590cb9700 1 -- 172.21.15.59:0/1158301832 shutdown_connections 2023-11-24T07:58:50.815 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:50.812+0000 7f3590cb9700 1 -- 172.21.15.59:0/1158301832 wait complete. 2023-11-24T07:58:50.826 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2023-11-24T07:58:50.954 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:50.952+0000 7f6f7eb41700 1 Processor -- start 2023-11-24T07:58:50.954 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:50.952+0000 7f6f7eb41700 1 -- start start 2023-11-24T07:58:50.956 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:50.953+0000 7f6f7eb41700 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f6f780b4090 0x7f6f7815e510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:50.956 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:50.953+0000 7f6f7eb41700 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f6f780d1010 0x7f6f780d1440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:50.956 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:50.953+0000 7f6f7eb41700 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6f780d0b80 con 0x7f6f78072a00 2023-11-24T07:58:50.957 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:50.953+0000 7f6f7eb41700 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6f780d1980 con 0x7f6f780d1010 2023-11-24T07:58:50.957 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:50.953+0000 7f6f7eb41700 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6f7815ed60 con 0x7f6f780b4090 2023-11-24T07:58:50.957 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:50.953+0000 7f6f7c8dd700 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f6f780b4090 0x7f6f7815e510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:50.957 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:50.953+0000 7f6f7d0de700 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f6f780d1010 0x7f6f780d1440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:50.957 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:50.953+0000 7f6f7c8dd700 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f6f780b4090 0x7f6f7815e510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3301/0 says I am v2:172.21.15.59:53478/0 (socket says 172.21.15.59:53478) 2023-11-24T07:58:50.957 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:50.953+0000 7f6f7d0de700 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f6f780d1010 0x7f6f780d1440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.59:41240/0 (socket says 172.21.15.59:41240) 2023-11-24T07:58:50.958 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:50.953+0000 7f6f77fff700 1 --1- >> v1:172.21.15.76:6789/0 conn(0x7f6f78072a00 0x7f6f7806db60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.59:44792/0 (socket says 172.21.15.59:44792) 2023-11-24T07:58:50.958 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:50.953+0000 7f6f7c8dd700 1 -- 172.21.15.59:0/3509698237 learned_addr learned my addr 172.21.15.59:0/3509698237 (peer_addr_for_me v2:172.21.15.59:0/0) 2023-11-24T07:58:50.958 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:50.953+0000 7f6f7d0de700 1 -- 172.21.15.59:0/3509698237 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f6f780b4090 msgr2=0x7f6f7815e510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T07:58:50.958 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:50.953+0000 7f6f7d0de700 1 --2- 172.21.15.59:0/3509698237 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f6f780b4090 0x7f6f7815e510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:50.958 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:50.953+0000 7f6f7d0de700 1 -- 172.21.15.59:0/3509698237 >> v1:172.21.15.76:6789/0 conn(0x7f6f78072a00 legacy=0x7f6f7806db60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:50.959 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:50.953+0000 7f6f7d0de700 1 -- 172.21.15.59:0/3509698237 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6f7815f040 con 0x7f6f780d1010 2023-11-24T07:58:50.959 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:50.953+0000 7f6f7c8dd700 1 --2- 172.21.15.59:0/3509698237 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f6f780b4090 0x7f6f7815e510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-24T07:58:50.959 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:50.953+0000 7f6f777fe700 1 -- 172.21.15.59:0/3509698237 <== mon.1 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 973938071 0 0) 0x7f6f780d0b80 con 0x7f6f78072a00 2023-11-24T07:58:50.959 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:50.954+0000 7f6f7d0de700 1 --2- 172.21.15.59:0/3509698237 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6f780d1010 0x7f6f780d1440 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f6f70009d90 tx=0x7f6f7000b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f1c0c7fa8812c806 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:58:50.959 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:50.954+0000 7f6f777fe700 1 -- 172.21.15.59:0/3509698237 <== mon.1 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f7000e070 con 0x7f6f780d1010 2023-11-24T07:58:50.959 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:50.954+0000 7f6f777fe700 1 -- 172.21.15.59:0/3509698237 <== mon.1 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6f7000bb20 con 0x7f6f780d1010 2023-11-24T07:58:50.959 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:50.954+0000 7f6f777fe700 1 -- 172.21.15.59:0/3509698237 <== mon.1 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f70012430 con 0x7f6f780d1010 2023-11-24T07:58:50.959 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:50.954+0000 7f6f7eb41700 1 -- 172.21.15.59:0/3509698237 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6f780d1010 msgr2=0x7f6f780d1440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:50.959 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:50.954+0000 7f6f7eb41700 1 --2- 172.21.15.59:0/3509698237 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6f780d1010 0x7f6f780d1440 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f6f70009d90 tx=0x7f6f7000b330 comp rx=0 tx=0).stop 2023-11-24T07:58:50.959 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:50.954+0000 7f6f7eb41700 1 -- 172.21.15.59:0/3509698237 shutdown_connections 2023-11-24T07:58:50.960 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:50.954+0000 7f6f7eb41700 1 --2- 172.21.15.59:0/3509698237 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6f780d1010 0x7f6f780d1440 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:50.960 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:50.954+0000 7f6f7eb41700 1 --2- 172.21.15.59:0/3509698237 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f6f780b4090 0x7f6f7815e510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:50.960 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:50.954+0000 7f6f7eb41700 1 -- 172.21.15.59:0/3509698237 >> 172.21.15.59:0/3509698237 conn(0x7f6f780b2740 msgr2=0x7f6f780b2b50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:58:50.960 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:50.954+0000 7f6f7eb41700 1 -- 172.21.15.59:0/3509698237 shutdown_connections 2023-11-24T07:58:50.960 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:50.955+0000 7f6f7eb41700 1 -- 172.21.15.59:0/3509698237 wait complete. 2023-11-24T07:58:50.963 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:50.956+0000 7f6f7eb41700 1 Processor -- start 2023-11-24T07:58:50.963 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:50.956+0000 7f6f7eb41700 1 -- start start 2023-11-24T07:58:50.963 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:50.957+0000 7f6f7eb41700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6f78072a00 0x7f6f78154350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:50.963 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:50.957+0000 7f6f7eb41700 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f6f780b4090 0x7f6f78158d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:50.963 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:50.957+0000 7f6f7eb41700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6f780d1010 0x7f6f78161a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:50.963 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:50.957+0000 7f6f7eb41700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f780ad9f0 con 0x7f6f78072a00 2023-11-24T07:58:50.963 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:50.957+0000 7f6f7eb41700 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f78154890 con 0x7f6f780d1010 2023-11-24T07:58:50.964 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:50.957+0000 7f6f7eb41700 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6f7815b850 con 0x7f6f780b4090 2023-11-24T07:58:50.964 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:50.957+0000 7f6f7c8dd700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6f78072a00 0x7f6f78154350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:50.964 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:50.957+0000 7f6f7c8dd700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6f78072a00 0x7f6f78154350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:54044/0 (socket says 172.21.15.59:54044) 2023-11-24T07:58:50.964 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:50.957+0000 7f6f7c8dd700 1 -- 172.21.15.59:0/1855568778 learned_addr learned my addr 172.21.15.59:0/1855568778 (peer_addr_for_me v2:172.21.15.59:0/0) 2023-11-24T07:58:50.964 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:50.957+0000 7f6f7d0de700 1 --2- 172.21.15.59:0/1855568778 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6f780d1010 0x7f6f78161a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:50.964 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:50.957+0000 7f6f77fff700 1 --2- 172.21.15.59:0/1855568778 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f6f780b4090 0x7f6f78158d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:50.964 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:50.957+0000 7f6f7c8dd700 1 -- 172.21.15.59:0/1855568778 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f6f780b4090 msgr2=0x7f6f78158d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:50.964 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:50.957+0000 7f6f7c8dd700 1 --2- 172.21.15.59:0/1855568778 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f6f780b4090 0x7f6f78158d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:50.964 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:50.957+0000 7f6f7c8dd700 1 -- 172.21.15.59:0/1855568778 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6f780d1010 msgr2=0x7f6f78161a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:50.964 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:50.957+0000 7f6f7c8dd700 1 --2- 172.21.15.59:0/1855568778 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6f780d1010 0x7f6f78161a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:50.964 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:50.957+0000 7f6f7c8dd700 1 -- 172.21.15.59:0/1855568778 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6f6c009550 con 0x7f6f78072a00 2023-11-24T07:58:50.964 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:50.957+0000 7f6f7c8dd700 1 --2- 172.21.15.59:0/1855568778 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6f78072a00 0x7f6f78154350 secure :-1 s=READY pgs=737 cs=0 l=1 rev1=1 crypto rx=0x7f6f6c00ebe0 tx=0x7f6f6c00eef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:58:50.965 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:50.957+0000 7f6f757fa700 1 -- 172.21.15.59:0/1855568778 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f6c00c850 con 0x7f6f78072a00 2023-11-24T07:58:50.965 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:50.957+0000 7f6f757fa700 1 -- 172.21.15.59:0/1855568778 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6f6c00c9b0 con 0x7f6f78072a00 2023-11-24T07:58:50.965 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:50.957+0000 7f6f7eb41700 1 -- 172.21.15.59:0/1855568778 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6f70009a70 con 0x7f6f78072a00 2023-11-24T07:58:50.965 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:50.957+0000 7f6f757fa700 1 -- 172.21.15.59:0/1855568778 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f6c00cd40 con 0x7f6f78072a00 2023-11-24T07:58:50.965 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:50.958+0000 7f6f7eb41700 1 -- 172.21.15.59:0/1855568778 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6f78158690 con 0x7f6f78072a00 2023-11-24T07:58:50.965 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:50.958+0000 7f6f7eb41700 1 -- 172.21.15.59:0/1855568778 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6f78075480 con 0x7f6f78072a00 2023-11-24T07:58:50.965 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:50.959+0000 7f6f757fa700 1 -- 172.21.15.59:0/1855568778 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 8) v1 ==== 89435+0+0 (secure 0 0 0) 0x7f6f6c013470 con 0x7f6f78072a00 2023-11-24T07:58:50.965 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:50.959+0000 7f6f757fa700 1 --2- 172.21.15.59:0/1855568778 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f6f5c06b110 0x7f6f5c06d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:50.965 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:50.959+0000 7f6f757fa700 1 -- 172.21.15.59:0/1855568778 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(60..60 src has 1..60) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f6f6c089e30 con 0x7f6f78072a00 2023-11-24T07:58:50.965 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:50.960+0000 7f6f77fff700 1 --2- 172.21.15.59:0/1855568778 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f6f5c06b110 0x7f6f5c06d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:50.965 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:50.960+0000 7f6f77fff700 1 --2- 172.21.15.59:0/1855568778 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f6f5c06b110 0x7f6f5c06d5d0 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f6f7806f590 tx=0x7f6f64007de0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:58:50.966 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:50.964+0000 7f6f757fa700 1 -- 172.21.15.59:0/1855568778 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f6f6c054c50 con 0x7f6f78072a00 2023-11-24T07:58:51.167 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:51.166+0000 7f6f7eb41700 1 -- 172.21.15.59:0/1855568778 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7f6f78072650 con 0x7f6f78072a00 2023-11-24T07:58:51.168 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:51.166+0000 7f6f757fa700 1 -- 172.21.15.59:0/1855568778 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v14)=0 v14) v1 ==== 122+0+0 (secure 0 0 0) 0x7f6f6c058270 con 0x7f6f78072a00 2023-11-24T07:58:51.170 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:51.168+0000 7f6f7eb41700 1 -- 172.21.15.59:0/1855568778 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f6f5c06b110 msgr2=0x7f6f5c06d5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:51.170 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:51.168+0000 7f6f7eb41700 1 --2- 172.21.15.59:0/1855568778 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f6f5c06b110 0x7f6f5c06d5d0 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f6f7806f590 tx=0x7f6f64007de0 comp rx=0 tx=0).stop 2023-11-24T07:58:51.170 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:51.168+0000 7f6f7eb41700 1 -- 172.21.15.59:0/1855568778 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6f78072a00 msgr2=0x7f6f78154350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:51.171 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:51.168+0000 7f6f7eb41700 1 --2- 172.21.15.59:0/1855568778 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6f78072a00 0x7f6f78154350 secure :-1 s=READY pgs=737 cs=0 l=1 rev1=1 crypto rx=0x7f6f6c00ebe0 tx=0x7f6f6c00eef0 comp rx=0 tx=0).stop 2023-11-24T07:58:51.171 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:51.168+0000 7f6f7eb41700 1 -- 172.21.15.59:0/1855568778 shutdown_connections 2023-11-24T07:58:51.171 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:51.168+0000 7f6f7eb41700 1 --2- 172.21.15.59:0/1855568778 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f6f5c06b110 0x7f6f5c06d5d0 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:51.171 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:51.168+0000 7f6f7eb41700 1 --2- 172.21.15.59:0/1855568778 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6f780d1010 0x7f6f78161a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:51.171 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:51.168+0000 7f6f7eb41700 1 --2- 172.21.15.59:0/1855568778 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6f78072a00 0x7f6f78154350 unknown :-1 s=CLOSED pgs=737 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:51.171 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:51.168+0000 7f6f7eb41700 1 --2- 172.21.15.59:0/1855568778 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f6f780b4090 0x7f6f78158d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:51.171 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:51.168+0000 7f6f7eb41700 1 -- 172.21.15.59:0/1855568778 >> 172.21.15.59:0/1855568778 conn(0x7f6f780b2740 msgr2=0x7f6f7806db60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:58:51.171 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:51.168+0000 7f6f7eb41700 1 -- 172.21.15.59:0/1855568778 shutdown_connections 2023-11-24T07:58:51.171 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:51.168+0000 7f6f7eb41700 1 -- 172.21.15.59:0/1855568778 wait complete. 2023-11-24T07:58:51.182 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume create cephfs subvol1532 2023-11-24T07:58:51.310 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:51.308+0000 7f563db81700 1 Processor -- start 2023-11-24T07:58:51.310 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:51.308+0000 7f563db81700 1 -- start start 2023-11-24T07:58:51.313 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:51.309+0000 7f563db81700 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f56380cfde0 0x7f56380d24d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:51.313 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:51.309+0000 7f563db81700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f56380d2a10 0x7f56380b4ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:51.313 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:51.309+0000 7f563db81700 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f56380b7810 con 0x7f563815e190 2023-11-24T07:58:51.313 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:51.309+0000 7f563db81700 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f56380d1930 con 0x7f56380d2a10 2023-11-24T07:58:51.313 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:51.309+0000 7f563db81700 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f56380d2ea0 con 0x7f56380cfde0 2023-11-24T07:58:51.313 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:51.309+0000 7f56377fe700 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7f563815e190 0x7f56380d0f50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:57526/0 (socket says 172.21.15.59:57526) 2023-11-24T07:58:51.313 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:51.309+0000 7f56377fe700 1 -- 172.21.15.59:0/148000831 learned_addr learned my addr 172.21.15.59:0/148000831 (peer_addr_for_me v1:172.21.15.59:0/0) 2023-11-24T07:58:51.314 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:51.309+0000 7f5637fff700 1 --2- 172.21.15.59:0/148000831 >> v2:172.21.15.59:3300/0 conn(0x7f56380d2a10 0x7f56380b4ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:51.314 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:51.310+0000 7f5636ffd700 1 --2- 172.21.15.59:0/148000831 >> v2:172.21.15.76:3300/0 conn(0x7f56380cfde0 0x7f56380d24d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:51.314 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:51.310+0000 7f5637fff700 1 -- 172.21.15.59:0/148000831 >> v2:172.21.15.76:3300/0 conn(0x7f56380cfde0 msgr2=0x7f56380d24d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T07:58:51.314 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:51.310+0000 7f5637fff700 1 --2- 172.21.15.59:0/148000831 >> v2:172.21.15.76:3300/0 conn(0x7f56380cfde0 0x7f56380d24d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:51.314 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:51.310+0000 7f5637fff700 1 -- 172.21.15.59:0/148000831 >> v1:172.21.15.59:6789/0 conn(0x7f563815e190 legacy=0x7f56380d0f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:51.314 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:51.310+0000 7f5637fff700 1 -- 172.21.15.59:0/148000831 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f56380d1660 con 0x7f56380d2a10 2023-11-24T07:58:51.314 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:51.310+0000 7f56367fc700 1 -- 172.21.15.59:0/148000831 <== mon.0 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3142508247 0 0) 0x7f56380b7810 con 0x7f563815e190 2023-11-24T07:58:51.314 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:51.310+0000 7f5636ffd700 1 --2- 172.21.15.59:0/148000831 >> v2:172.21.15.76:3300/0 conn(0x7f56380cfde0 0x7f56380d24d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-24T07:58:51.314 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:51.310+0000 7f5637fff700 1 --2- 172.21.15.59:0/148000831 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f56380d2a10 0x7f56380b4ab0 secure :-1 s=READY pgs=739 cs=0 l=1 rev1=1 crypto rx=0x7f56380caf70 tx=0x7f563000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=399e9ebbd314d4bf server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:58:51.314 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:51.310+0000 7f56367fc700 1 -- 172.21.15.59:0/148000831 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f563000e070 con 0x7f56380d2a10 2023-11-24T07:58:51.315 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:51.310+0000 7f56367fc700 1 -- 172.21.15.59:0/148000831 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f563000bbd0 con 0x7f56380d2a10 2023-11-24T07:58:51.315 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:51.310+0000 7f56367fc700 1 -- 172.21.15.59:0/148000831 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5630012430 con 0x7f56380d2a10 2023-11-24T07:58:51.315 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:51.311+0000 7f563db81700 1 -- 172.21.15.59:0/148000831 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f56380d2a10 msgr2=0x7f56380b4ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:51.315 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:51.311+0000 7f563db81700 1 --2- 172.21.15.59:0/148000831 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f56380d2a10 0x7f56380b4ab0 secure :-1 s=READY pgs=739 cs=0 l=1 rev1=1 crypto rx=0x7f56380caf70 tx=0x7f563000b330 comp rx=0 tx=0).stop 2023-11-24T07:58:51.315 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:51.311+0000 7f563db81700 1 -- 172.21.15.59:0/148000831 shutdown_connections 2023-11-24T07:58:51.315 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:51.311+0000 7f563db81700 1 --2- 172.21.15.59:0/148000831 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f56380d2a10 0x7f56380b4ab0 unknown :-1 s=CLOSED pgs=739 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:51.315 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:51.311+0000 7f563db81700 1 --2- 172.21.15.59:0/148000831 >> v2:172.21.15.76:3300/0 conn(0x7f56380cfde0 0x7f56380d24d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:51.315 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:51.311+0000 7f563db81700 1 -- 172.21.15.59:0/148000831 >> 172.21.15.59:0/148000831 conn(0x7f563809d180 msgr2=0x7f56380d3ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:58:51.315 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:51.311+0000 7f563db81700 1 -- 172.21.15.59:0/148000831 shutdown_connections 2023-11-24T07:58:51.315 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:51.311+0000 7f563db81700 1 -- 172.21.15.59:0/148000831 wait complete. 2023-11-24T07:58:51.318 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:51.312+0000 7f563db81700 1 Processor -- start 2023-11-24T07:58:51.318 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:51.312+0000 7f563db81700 1 -- start start 2023-11-24T07:58:51.318 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:51.313+0000 7f563db81700 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f56380cfde0 0x7f56381f7aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:51.318 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:51.313+0000 7f563db81700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f56380d2a10 0x7f56380cd250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:51.318 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:51.313+0000 7f563db81700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f563815e190 0x7f563815e810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:51.318 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:51.313+0000 7f563db81700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56381f7fe0 con 0x7f56380d2a10 2023-11-24T07:58:51.318 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:51.313+0000 7f563db81700 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56380cc1b0 con 0x7f563815e190 2023-11-24T07:58:51.319 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:51.313+0000 7f563db81700 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f56380cc490 con 0x7f56380cfde0 2023-11-24T07:58:51.319 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:51.313+0000 7f5636ffd700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f56380d2a10 0x7f56380cd250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:51.319 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:51.313+0000 7f5636ffd700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f56380d2a10 0x7f56380cd250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:54060/0 (socket says 172.21.15.59:54060) 2023-11-24T07:58:51.319 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:51.313+0000 7f5636ffd700 1 -- 172.21.15.59:0/1749900784 learned_addr learned my addr 172.21.15.59:0/1749900784 (peer_addr_for_me v2:172.21.15.59:0/0) 2023-11-24T07:58:51.319 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:51.313+0000 7f5637fff700 1 --2- 172.21.15.59:0/1749900784 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f563815e190 0x7f563815e810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:51.319 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:51.313+0000 7f56377fe700 1 --2- 172.21.15.59:0/1749900784 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f56380cfde0 0x7f56381f7aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:51.319 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:51.313+0000 7f5636ffd700 1 -- 172.21.15.59:0/1749900784 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f56380cfde0 msgr2=0x7f56381f7aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:51.319 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:51.313+0000 7f5636ffd700 1 --2- 172.21.15.59:0/1749900784 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f56380cfde0 0x7f56381f7aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:51.319 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:51.313+0000 7f5636ffd700 1 -- 172.21.15.59:0/1749900784 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f563815e190 msgr2=0x7f563815e810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:51.319 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:51.313+0000 7f5636ffd700 1 --2- 172.21.15.59:0/1749900784 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f563815e190 0x7f563815e810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:51.319 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:51.313+0000 7f5636ffd700 1 -- 172.21.15.59:0/1749900784 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5630009a70 con 0x7f56380d2a10 2023-11-24T07:58:51.319 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:51.314+0000 7f5636ffd700 1 --2- 172.21.15.59:0/1749900784 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f56380d2a10 0x7f56380cd250 secure :-1 s=READY pgs=740 cs=0 l=1 rev1=1 crypto rx=0x7f562000e7e0 tx=0x7f562000eaf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:58:51.320 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:51.314+0000 7f562ffff700 1 -- 172.21.15.59:0/1749900784 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f562000c070 con 0x7f56380d2a10 2023-11-24T07:58:51.320 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:51.314+0000 7f562ffff700 1 -- 172.21.15.59:0/1749900784 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5620004230 con 0x7f56380d2a10 2023-11-24T07:58:51.320 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:51.314+0000 7f562ffff700 1 -- 172.21.15.59:0/1749900784 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56200045a0 con 0x7f56380d2a10 2023-11-24T07:58:51.320 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:51.314+0000 7f563db81700 1 -- 172.21.15.59:0/1749900784 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f56380cc770 con 0x7f56380d2a10 2023-11-24T07:58:51.320 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:51.314+0000 7f563db81700 1 -- 172.21.15.59:0/1749900784 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f56380ccd30 con 0x7f56380d2a10 2023-11-24T07:58:51.320 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:51.315+0000 7f563db81700 1 -- 172.21.15.59:0/1749900784 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5638075480 con 0x7f56380d2a10 2023-11-24T07:58:51.320 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:51.315+0000 7f562ffff700 1 -- 172.21.15.59:0/1749900784 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 8) v1 ==== 89435+0+0 (secure 0 0 0) 0x7f56200036a0 con 0x7f56380d2a10 2023-11-24T07:58:51.320 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:51.316+0000 7f562ffff700 1 --2- 172.21.15.59:0/1749900784 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f561806b110 0x7f561806d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:51.320 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:51.316+0000 7f562ffff700 1 -- 172.21.15.59:0/1749900784 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(60..60 src has 1..60) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f56200897c0 con 0x7f56380d2a10 2023-11-24T07:58:51.320 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:51.316+0000 7f56377fe700 1 --2- 172.21.15.59:0/1749900784 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f561806b110 0x7f561806d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:51.321 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:51.316+0000 7f56377fe700 1 --2- 172.21.15.59:0/1749900784 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f561806b110 0x7f561806d5d0 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f5628000f60 tx=0x7f5628007e70 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:58:51.321 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:51.321+0000 7f562ffff700 1 -- 172.21.15.59:0/1749900784 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f5620054410 con 0x7f56380d2a10 2023-11-24T07:58:51.556 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:51.554+0000 7f563db81700 1 -- 172.21.15.59:0/1749900784 --> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "subvol1532", "target": ["mon-mgr", ""]}) v1 -- 0x7f56380cacf0 con 0x7f561806b110 2023-11-24T07:58:51.562 INFO:tasks.ceph.mgr.x.smithi076.stderr:2023-11-24T07:58:51.560+0000 7ff196724700 -1 client.0 error registering admin socket command: (17) File exists 2023-11-24T07:58:51.562 INFO:tasks.ceph.mgr.x.smithi076.stderr:2023-11-24T07:58:51.560+0000 7ff196724700 -1 client.0 error registering admin socket command: (17) File exists 2023-11-24T07:58:51.562 INFO:tasks.ceph.mgr.x.smithi076.stderr:2023-11-24T07:58:51.560+0000 7ff196724700 -1 client.0 error registering admin socket command: (17) File exists 2023-11-24T07:58:51.562 INFO:tasks.ceph.mgr.x.smithi076.stderr:2023-11-24T07:58:51.560+0000 7ff196724700 -1 client.0 error registering admin socket command: (17) File exists 2023-11-24T07:58:51.562 INFO:tasks.ceph.mgr.x.smithi076.stderr:2023-11-24T07:58:51.560+0000 7ff196724700 -1 client.0 error registering admin socket command: (17) File exists 2023-11-24T07:58:51.601 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:51.599+0000 7f562ffff700 1 -- 172.21.15.59:0/1749900784 <== mgr.4100 v2:172.21.15.76:6832/4264046536 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7f56380cacf0 con 0x7f561806b110 2023-11-24T07:58:51.604 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:51.602+0000 7f563db81700 1 -- 172.21.15.59:0/1749900784 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f561806b110 msgr2=0x7f561806d5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:51.604 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:51.602+0000 7f563db81700 1 --2- 172.21.15.59:0/1749900784 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f561806b110 0x7f561806d5d0 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f5628000f60 tx=0x7f5628007e70 comp rx=0 tx=0).stop 2023-11-24T07:58:51.604 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:51.602+0000 7f563db81700 1 -- 172.21.15.59:0/1749900784 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f56380d2a10 msgr2=0x7f56380cd250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:51.604 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:51.602+0000 7f563db81700 1 --2- 172.21.15.59:0/1749900784 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f56380d2a10 0x7f56380cd250 secure :-1 s=READY pgs=740 cs=0 l=1 rev1=1 crypto rx=0x7f562000e7e0 tx=0x7f562000eaf0 comp rx=0 tx=0).stop 2023-11-24T07:58:51.604 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:51.602+0000 7f563db81700 1 -- 172.21.15.59:0/1749900784 shutdown_connections 2023-11-24T07:58:51.604 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:51.602+0000 7f563db81700 1 --2- 172.21.15.59:0/1749900784 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f561806b110 0x7f561806d5d0 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:51.604 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:51.602+0000 7f563db81700 1 --2- 172.21.15.59:0/1749900784 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f563815e190 0x7f563815e810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:51.605 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:51.602+0000 7f563db81700 1 --2- 172.21.15.59:0/1749900784 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f56380cfde0 0x7f56381f7aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:51.605 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:51.602+0000 7f563db81700 1 --2- 172.21.15.59:0/1749900784 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f56380d2a10 0x7f56380cd250 unknown :-1 s=CLOSED pgs=740 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:51.605 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:51.602+0000 7f563db81700 1 -- 172.21.15.59:0/1749900784 >> 172.21.15.59:0/1749900784 conn(0x7f563809d180 msgr2=0x7f563806bd90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:58:51.605 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:51.602+0000 7f563db81700 1 -- 172.21.15.59:0/1749900784 shutdown_connections 2023-11-24T07:58:51.605 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:51.602+0000 7f563db81700 1 -- 172.21.15.59:0/1749900784 wait complete. 2023-11-24T07:58:51.616 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot create cephfs subvol1532 subvol_snap2913 2023-11-24T07:58:51.746 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:51.744+0000 7f607f152700 1 Processor -- start 2023-11-24T07:58:51.746 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:51.744+0000 7f607f152700 1 -- start start 2023-11-24T07:58:51.753 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:51.745+0000 7f607f152700 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f60780cd150 0x7f60780ceb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:51.753 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:51.745+0000 7f607f152700 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f607806f940 0x7f607806fd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:51.753 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:51.745+0000 7f607f152700 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f607806f4b0 con 0x7f607815bfe0 2023-11-24T07:58:51.754 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:51.745+0000 7f607f152700 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f607815c3c0 con 0x7f607806f940 2023-11-24T07:58:51.754 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:51.745+0000 7f607f152700 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f60780cd760 con 0x7f60780cd150 2023-11-24T07:58:51.754 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:51.745+0000 7f607ceee700 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7f607815bfe0 0x7f60780ce430 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:57540/0 (socket says 172.21.15.59:57540) 2023-11-24T07:58:51.754 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:51.745+0000 7f607ceee700 1 -- 172.21.15.59:0/213771594 learned_addr learned my addr 172.21.15.59:0/213771594 (peer_addr_for_me v1:172.21.15.59:0/0) 2023-11-24T07:58:51.754 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:51.745+0000 7f6077fff700 1 --2- 172.21.15.59:0/213771594 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f60780cd150 0x7f60780ceb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:51.754 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:51.745+0000 7f607d6ef700 1 --2- 172.21.15.59:0/213771594 >> v2:172.21.15.76:3300/0 conn(0x7f607806f940 0x7f607806fd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:51.754 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:51.745+0000 7f60777fe700 1 -- 172.21.15.59:0/213771594 <== mon.0 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1136326795 0 0) 0x7f607806f4b0 con 0x7f607815bfe0 2023-11-24T07:58:51.754 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:51.745+0000 7f60777fe700 1 -- 172.21.15.59:0/213771594 --> v1:172.21.15.59:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f60680035b0 con 0x7f607815bfe0 2023-11-24T07:58:51.755 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:51.746+0000 7f60777fe700 1 -- 172.21.15.59:0/213771594 <== mon.0 v1:172.21.15.59:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 630228112 0 0) 0x7f60680035b0 con 0x7f607815bfe0 2023-11-24T07:58:51.755 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:51.746+0000 7f60777fe700 1 -- 172.21.15.59:0/213771594 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f60780cd150 msgr2=0x7f60780ceb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T07:58:51.755 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:51.746+0000 7f60777fe700 1 --2- 172.21.15.59:0/213771594 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f60780cd150 0x7f60780ceb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:51.755 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:51.746+0000 7f60777fe700 1 -- 172.21.15.59:0/213771594 >> v2:172.21.15.76:3300/0 conn(0x7f607806f940 msgr2=0x7f607806fd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T07:58:51.755 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:51.746+0000 7f60777fe700 1 --2- 172.21.15.59:0/213771594 >> v2:172.21.15.76:3300/0 conn(0x7f607806f940 0x7f607806fd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:51.755 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:51.746+0000 7f60777fe700 1 -- 172.21.15.59:0/213771594 --> v1:172.21.15.59:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f607806ee60 con 0x7f607815bfe0 2023-11-24T07:58:51.755 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:51.746+0000 7f60777fe700 1 -- 172.21.15.59:0/213771594 <== mon.0 v1:172.21.15.59:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1312333566 0 0) 0x7f606c001670 con 0x7f607815bfe0 2023-11-24T07:58:51.755 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:51.746+0000 7f60777fe700 1 -- 172.21.15.59:0/213771594 >> v1:172.21.15.59:6789/0 conn(0x7f607815bfe0 legacy=0x7f60780ce430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:51.755 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:51.746+0000 7f60777fe700 1 --2- 172.21.15.59:0/213771594 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6068003a00 0x7f6068003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:51.755 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:51.746+0000 7f60777fe700 1 --2- 172.21.15.59:0/213771594 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f60780cd150 0x7f6068004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:51.756 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:51.746+0000 7f60777fe700 1 --2- 172.21.15.59:0/213771594 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6068004d90 0x7f6068005210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:51.756 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:51.746+0000 7f60777fe700 1 -- 172.21.15.59:0/213771594 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f607815c3c0 con 0x7f6068004d90 2023-11-24T07:58:51.756 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:51.746+0000 7f60777fe700 1 -- 172.21.15.59:0/213771594 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60780cd760 con 0x7f6068003a00 2023-11-24T07:58:51.756 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:51.746+0000 7f60777fe700 1 -- 172.21.15.59:0/213771594 --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f606800ba10 con 0x7f60780cd150 2023-11-24T07:58:51.756 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:51.746+0000 7f607d6ef700 1 --2- 172.21.15.59:0/213771594 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6068004d90 0x7f6068005210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:51.756 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:51.746+0000 7f607ceee700 1 --2- 172.21.15.59:0/213771594 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6068003a00 0x7f6068003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:51.756 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:51.746+0000 7f6077fff700 1 --2- 172.21.15.59:0/213771594 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f60780cd150 0x7f6068004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:51.756 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:51.746+0000 7f607d6ef700 1 -- 172.21.15.59:0/213771594 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f60780cd150 msgr2=0x7f6068004850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T07:58:51.756 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:51.746+0000 7f607d6ef700 1 --2- 172.21.15.59:0/213771594 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f60780cd150 0x7f6068004850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:51.757 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:51.746+0000 7f607d6ef700 1 -- 172.21.15.59:0/213771594 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6068003a00 msgr2=0x7f6068003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T07:58:51.757 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:51.746+0000 7f607d6ef700 1 --2- 172.21.15.59:0/213771594 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6068003a00 0x7f6068003e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:51.757 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:51.746+0000 7f607d6ef700 1 -- 172.21.15.59:0/213771594 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f60680035b0 con 0x7f6068004d90 2023-11-24T07:58:51.757 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:51.746+0000 7f607d6ef700 1 --2- 172.21.15.59:0/213771594 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6068004d90 0x7f6068005210 secure :-1 s=READY pgs=742 cs=0 l=1 rev1=1 crypto rx=0x7f6070009fc0 tx=0x7f6068005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=276f4a2c5920a077 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:58:51.757 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:51.746+0000 7f60777fe700 1 -- 172.21.15.59:0/213771594 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f607000ec30 con 0x7f6068004d90 2023-11-24T07:58:51.757 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:51.746+0000 7f60777fe700 1 -- 172.21.15.59:0/213771594 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f607000f040 con 0x7f6068004d90 2023-11-24T07:58:51.757 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:51.746+0000 7f60777fe700 1 -- 172.21.15.59:0/213771594 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6070014480 con 0x7f6068004d90 2023-11-24T07:58:51.757 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:51.747+0000 7f607f152700 1 -- 172.21.15.59:0/213771594 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6068004d90 msgr2=0x7f6068005210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:51.757 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:51.747+0000 7f607f152700 1 --2- 172.21.15.59:0/213771594 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6068004d90 0x7f6068005210 secure :-1 s=READY pgs=742 cs=0 l=1 rev1=1 crypto rx=0x7f6070009fc0 tx=0x7f6068005880 comp rx=0 tx=0).stop 2023-11-24T07:58:51.757 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:51.747+0000 7f607d6ef700 1 -- 172.21.15.59:0/213771594 reap_dead start 2023-11-24T07:58:51.758 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:51.747+0000 7f607f152700 1 -- 172.21.15.59:0/213771594 shutdown_connections 2023-11-24T07:58:51.758 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:51.747+0000 7f607f152700 1 -- 172.21.15.59:0/213771594 >> 172.21.15.59:0/213771594 conn(0x7f60780b3700 msgr2=0x7f60780b3b10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:58:51.758 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:51.747+0000 7f607f152700 1 -- 172.21.15.59:0/213771594 shutdown_connections 2023-11-24T07:58:51.758 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:51.747+0000 7f607f152700 1 -- 172.21.15.59:0/213771594 wait complete. 2023-11-24T07:58:51.758 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:51.748+0000 7f607f152700 1 Processor -- start 2023-11-24T07:58:51.758 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:51.748+0000 7f607f152700 1 -- start start 2023-11-24T07:58:51.758 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:51.749+0000 7f607f152700 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f6068004d90 0x7f60780c5b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:51.758 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:51.749+0000 7f607f152700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f60781f3c50 0x7f6078156e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:51.758 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:51.749+0000 7f607f152700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f607806c0e0 0x7f6078157fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:51.759 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:51.749+0000 7f607f152700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60780c60b0 con 0x7f60781f3c50 2023-11-24T07:58:51.759 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:51.749+0000 7f607f152700 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60780c7490 con 0x7f607806c0e0 2023-11-24T07:58:51.759 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:51.749+0000 7f607f152700 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f60781f4430 con 0x7f6068004d90 2023-11-24T07:58:51.759 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:51.749+0000 7f6077fff700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f60781f3c50 0x7f6078156e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:51.759 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:51.749+0000 7f6077fff700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f60781f3c50 0x7f6078156e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:54080/0 (socket says 172.21.15.59:54080) 2023-11-24T07:58:51.759 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:51.749+0000 7f6077fff700 1 -- 172.21.15.59:0/1346222934 learned_addr learned my addr 172.21.15.59:0/1346222934 (peer_addr_for_me v2:172.21.15.59:0/0) 2023-11-24T07:58:51.759 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:51.749+0000 7f607ceee700 1 --2- 172.21.15.59:0/1346222934 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f6068004d90 0x7f60780c5b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:51.759 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:51.749+0000 7f607d6ef700 1 --2- 172.21.15.59:0/1346222934 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f607806c0e0 0x7f6078157fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:51.759 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:51.750+0000 7f6077fff700 1 -- 172.21.15.59:0/1346222934 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f6068004d90 msgr2=0x7f60780c5b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:51.759 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:51.750+0000 7f6077fff700 1 --2- 172.21.15.59:0/1346222934 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f6068004d90 0x7f60780c5b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:51.760 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:51.750+0000 7f6077fff700 1 -- 172.21.15.59:0/1346222934 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f607806c0e0 msgr2=0x7f6078157fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:51.760 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:51.750+0000 7f6077fff700 1 --2- 172.21.15.59:0/1346222934 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f607806c0e0 0x7f6078157fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:51.760 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:51.750+0000 7f6077fff700 1 -- 172.21.15.59:0/1346222934 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f60781f4710 con 0x7f60781f3c50 2023-11-24T07:58:51.760 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:51.750+0000 7f607d6ef700 1 --2- 172.21.15.59:0/1346222934 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f607806c0e0 0x7f6078157fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T07:58:51.760 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:51.750+0000 7f6077fff700 1 --2- 172.21.15.59:0/1346222934 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f60781f3c50 0x7f6078156e80 secure :-1 s=READY pgs=743 cs=0 l=1 rev1=1 crypto rx=0x7f6064011aa0 tx=0x7f6064011e60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:58:51.760 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:51.750+0000 7f60757fa700 1 -- 172.21.15.59:0/1346222934 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f606400f980 con 0x7f60781f3c50 2023-11-24T07:58:51.760 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:51.750+0000 7f60757fa700 1 -- 172.21.15.59:0/1346222934 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f606400fae0 con 0x7f60781f3c50 2023-11-24T07:58:51.760 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:51.750+0000 7f607f152700 1 -- 172.21.15.59:0/1346222934 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f60781f49f0 con 0x7f60781f3c50 2023-11-24T07:58:51.760 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:51.750+0000 7f60757fa700 1 -- 172.21.15.59:0/1346222934 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f606400fe90 con 0x7f60781f3c50 2023-11-24T07:58:51.760 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:51.750+0000 7f607f152700 1 -- 172.21.15.59:0/1346222934 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f60781573c0 con 0x7f60781f3c50 2023-11-24T07:58:51.761 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:51.751+0000 7f607f152700 1 -- 172.21.15.59:0/1346222934 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f607806bc40 con 0x7f60781f3c50 2023-11-24T07:58:51.762 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:51.752+0000 7f60757fa700 1 -- 172.21.15.59:0/1346222934 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 8) v1 ==== 89435+0+0 (secure 0 0 0) 0x7f60640263f0 con 0x7f60781f3c50 2023-11-24T07:58:51.762 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:51.752+0000 7f60757fa700 1 --2- 172.21.15.59:0/1346222934 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f605c06b110 0x7f605c06d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:51.762 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:51.752+0000 7f60757fa700 1 -- 172.21.15.59:0/1346222934 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(60..60 src has 1..60) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f606408a590 con 0x7f60781f3c50 2023-11-24T07:58:51.762 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:51.753+0000 7f607ceee700 1 --2- 172.21.15.59:0/1346222934 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f605c06b110 0x7f605c06d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:51.762 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:51.753+0000 7f607ceee700 1 --2- 172.21.15.59:0/1346222934 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f605c06b110 0x7f605c06d5d0 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f606c000c00 tx=0x7f606c009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:58:51.762 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:51.757+0000 7f60757fa700 1 -- 172.21.15.59:0/1346222934 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f6064055410 con 0x7f60781f3c50 2023-11-24T07:58:51.995 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:51.993+0000 7f607f152700 1 -- 172.21.15.59:0/1346222934 --> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot create", "vol_name": "cephfs", "sub_name": "subvol1532", "snap_name": "subvol_snap2913", "target": ["mon-mgr", ""]}) v1 -- 0x7f60780d09d0 con 0x7f605c06b110 2023-11-24T07:58:52.011 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:52.010+0000 7f60757fa700 1 -- 172.21.15.59:0/1346222934 <== mgr.4100 v2:172.21.15.76:6832/4264046536 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7f606c007e80 con 0x7f605c06b110 2023-11-24T07:58:52.014 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:52.013+0000 7f607f152700 1 -- 172.21.15.59:0/1346222934 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f605c06b110 msgr2=0x7f605c06d5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:52.014 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:52.013+0000 7f607f152700 1 --2- 172.21.15.59:0/1346222934 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f605c06b110 0x7f605c06d5d0 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f606c000c00 tx=0x7f606c009040 comp rx=0 tx=0).stop 2023-11-24T07:58:52.014 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:52.013+0000 7f607f152700 1 -- 172.21.15.59:0/1346222934 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f60781f3c50 msgr2=0x7f6078156e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:52.014 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:52.013+0000 7f607f152700 1 --2- 172.21.15.59:0/1346222934 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f60781f3c50 0x7f6078156e80 secure :-1 s=READY pgs=743 cs=0 l=1 rev1=1 crypto rx=0x7f6064011aa0 tx=0x7f6064011e60 comp rx=0 tx=0).stop 2023-11-24T07:58:52.015 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:52.013+0000 7f607f152700 1 -- 172.21.15.59:0/1346222934 shutdown_connections 2023-11-24T07:58:52.015 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:52.013+0000 7f607f152700 1 --2- 172.21.15.59:0/1346222934 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f605c06b110 0x7f605c06d5d0 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:52.015 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:52.013+0000 7f607f152700 1 --2- 172.21.15.59:0/1346222934 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f607806c0e0 0x7f6078157fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:52.015 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:52.013+0000 7f607f152700 1 --2- 172.21.15.59:0/1346222934 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f6068004d90 0x7f60780c5b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:52.015 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:52.013+0000 7f607f152700 1 --2- 172.21.15.59:0/1346222934 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f60781f3c50 0x7f6078156e80 unknown :-1 s=CLOSED pgs=743 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:52.015 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:52.013+0000 7f607f152700 1 -- 172.21.15.59:0/1346222934 >> 172.21.15.59:0/1346222934 conn(0x7f60780b3700 msgr2=0x7f60780b1670 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:58:52.015 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:52.013+0000 7f607f152700 1 -- 172.21.15.59:0/1346222934 shutdown_connections 2023-11-24T07:58:52.015 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:52.013+0000 7f607f152700 1 -- 172.21.15.59:0/1346222934 wait complete. 2023-11-24T07:58:52.026 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot rm cephfs subvol1532 subvol_snap2913 2023-11-24T07:58:52.156 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:52.154+0000 7f864c4bb700 1 Processor -- start 2023-11-24T07:58:52.156 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:52.154+0000 7f864c4bb700 1 -- start start 2023-11-24T07:58:52.158 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:52.155+0000 7f864c4bb700 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f86440af1e0 0x7f86440b08c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:52.159 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:52.155+0000 7f864c4bb700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f8644070650 0x7f8644165970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:52.159 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:52.155+0000 7f864c4bb700 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f864406ee80 con 0x7f86440b54a0 2023-11-24T07:58:52.159 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:52.155+0000 7f864c4bb700 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f864406f220 con 0x7f8644070650 2023-11-24T07:58:52.159 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:52.155+0000 7f864c4bb700 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f86440b5920 con 0x7f86440af1e0 2023-11-24T07:58:52.159 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:52.155+0000 7f864aa58700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f8644070650 0x7f8644165970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:52.159 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:52.155+0000 7f864aa58700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f8644070650 0x7f8644165970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:54090/0 (socket says 172.21.15.59:54090) 2023-11-24T07:58:52.159 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:52.155+0000 7f864aa58700 1 -- 172.21.15.59:0/1183507942 learned_addr learned my addr 172.21.15.59:0/1183507942 (peer_addr_for_me v2:172.21.15.59:0/0) 2023-11-24T07:58:52.159 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:52.156+0000 7f864a257700 1 --2- 172.21.15.59:0/1183507942 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f86440af1e0 0x7f86440b08c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:52.159 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:52.156+0000 7f864aa58700 1 -- 172.21.15.59:0/1183507942 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f86440af1e0 msgr2=0x7f86440b08c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T07:58:52.159 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:52.156+0000 7f864aa58700 1 --2- 172.21.15.59:0/1183507942 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f86440af1e0 0x7f86440b08c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:52.159 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:52.156+0000 7f864aa58700 1 -- 172.21.15.59:0/1183507942 >> v1:172.21.15.76:6789/0 conn(0x7f86440b54a0 legacy=0x7f86440af5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:52.159 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:52.156+0000 7f864aa58700 1 -- 172.21.15.59:0/1183507942 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f86440b5b30 con 0x7f8644070650 2023-11-24T07:58:52.160 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:52.156+0000 7f864a257700 1 --2- 172.21.15.59:0/1183507942 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f86440af1e0 0x7f86440b08c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-24T07:58:52.160 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:52.156+0000 7f864aa58700 1 --2- 172.21.15.59:0/1183507942 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8644070650 0x7f8644165970 secure :-1 s=READY pgs=744 cs=0 l=1 rev1=1 crypto rx=0x7f8634009e50 tx=0x7f863400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c3f806c3912b5884 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:58:52.160 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:52.156+0000 7f8649255700 1 -- 172.21.15.59:0/1183507942 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f863400e070 con 0x7f8644070650 2023-11-24T07:58:52.160 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:52.156+0000 7f8649255700 1 -- 172.21.15.59:0/1183507942 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f863400b9c0 con 0x7f8644070650 2023-11-24T07:58:52.160 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:52.156+0000 7f8649255700 1 -- 172.21.15.59:0/1183507942 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f863400bd70 con 0x7f8644070650 2023-11-24T07:58:52.160 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:52.157+0000 7f864c4bb700 1 -- 172.21.15.59:0/1183507942 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8644070650 msgr2=0x7f8644165970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:52.160 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:52.157+0000 7f864c4bb700 1 --2- 172.21.15.59:0/1183507942 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8644070650 0x7f8644165970 secure :-1 s=READY pgs=744 cs=0 l=1 rev1=1 crypto rx=0x7f8634009e50 tx=0x7f863400b330 comp rx=0 tx=0).stop 2023-11-24T07:58:52.160 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:52.157+0000 7f864c4bb700 1 -- 172.21.15.59:0/1183507942 shutdown_connections 2023-11-24T07:58:52.160 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:52.157+0000 7f864c4bb700 1 --2- 172.21.15.59:0/1183507942 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f86440af1e0 0x7f86440b08c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:52.160 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:52.157+0000 7f864c4bb700 1 --2- 172.21.15.59:0/1183507942 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8644070650 0x7f8644165970 unknown :-1 s=CLOSED pgs=744 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:52.160 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:52.157+0000 7f864c4bb700 1 -- 172.21.15.59:0/1183507942 >> 172.21.15.59:0/1183507942 conn(0x7f864409a9f0 msgr2=0x7f864409ae00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:58:52.160 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:52.157+0000 7f864c4bb700 1 -- 172.21.15.59:0/1183507942 shutdown_connections 2023-11-24T07:58:52.160 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:52.157+0000 7f864c4bb700 1 -- 172.21.15.59:0/1183507942 wait complete. 2023-11-24T07:58:52.165 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:52.158+0000 7f864c4bb700 1 Processor -- start 2023-11-24T07:58:52.165 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:52.158+0000 7f864c4bb700 1 -- start start 2023-11-24T07:58:52.165 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:52.159+0000 7f864c4bb700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8644070650 0x7f86441e19c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:52.165 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:52.159+0000 7f864c4bb700 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f86440af1e0 0x7f86441581f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:52.165 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:52.159+0000 7f864c4bb700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f86440b54a0 0x7f8644146d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:52.165 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:52.159+0000 7f864c4bb700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8644147870 con 0x7f8644070650 2023-11-24T07:58:52.165 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:52.159+0000 7f864c4bb700 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86441e2560 con 0x7f86440b54a0 2023-11-24T07:58:52.165 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:52.159+0000 7f864c4bb700 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f86441e1fa0 con 0x7f86440af1e0 2023-11-24T07:58:52.165 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:52.159+0000 7f864a257700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8644070650 0x7f86441e19c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:52.166 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:52.159+0000 7f864a257700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8644070650 0x7f86441e19c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:54092/0 (socket says 172.21.15.59:54092) 2023-11-24T07:58:52.166 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:52.159+0000 7f864a257700 1 -- 172.21.15.59:0/695956619 learned_addr learned my addr 172.21.15.59:0/695956619 (peer_addr_for_me v2:172.21.15.59:0/0) 2023-11-24T07:58:52.166 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:52.159+0000 7f864aa58700 1 --2- 172.21.15.59:0/695956619 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f86440b54a0 0x7f8644146d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:52.166 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:52.159+0000 7f8649a56700 1 --2- 172.21.15.59:0/695956619 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f86440af1e0 0x7f86441581f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:52.166 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:52.159+0000 7f864a257700 1 -- 172.21.15.59:0/695956619 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f86440af1e0 msgr2=0x7f86441581f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:52.166 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:52.159+0000 7f864a257700 1 --2- 172.21.15.59:0/695956619 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f86440af1e0 0x7f86441581f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:52.166 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:52.159+0000 7f864a257700 1 -- 172.21.15.59:0/695956619 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f86440b54a0 msgr2=0x7f8644146d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:52.166 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:52.159+0000 7f864a257700 1 --2- 172.21.15.59:0/695956619 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f86440b54a0 0x7f8644146d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:52.166 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:52.159+0000 7f864a257700 1 -- 172.21.15.59:0/695956619 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8634009b30 con 0x7f8644070650 2023-11-24T07:58:52.166 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:52.159+0000 7f864a257700 1 --2- 172.21.15.59:0/695956619 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8644070650 0x7f86441e19c0 secure :-1 s=READY pgs=745 cs=0 l=1 rev1=1 crypto rx=0x7f863c00e810 tx=0x7f863c00eb20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:58:52.166 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:52.160+0000 7f8642ffd700 1 -- 172.21.15.59:0/695956619 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f863c00c020 con 0x7f8644070650 2023-11-24T07:58:52.167 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:52.160+0000 7f8642ffd700 1 -- 172.21.15.59:0/695956619 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f863c004250 con 0x7f8644070650 2023-11-24T07:58:52.167 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:52.160+0000 7f864c4bb700 1 -- 172.21.15.59:0/695956619 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f86441e2280 con 0x7f8644070650 2023-11-24T07:58:52.167 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:52.160+0000 7f8642ffd700 1 -- 172.21.15.59:0/695956619 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f863c0036a0 con 0x7f8644070650 2023-11-24T07:58:52.167 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:52.160+0000 7f864c4bb700 1 -- 172.21.15.59:0/695956619 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8644155bc0 con 0x7f8644070650 2023-11-24T07:58:52.167 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:52.160+0000 7f864c4bb700 1 -- 172.21.15.59:0/695956619 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8644155f30 con 0x7f8644070650 2023-11-24T07:58:52.167 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:52.161+0000 7f8642ffd700 1 -- 172.21.15.59:0/695956619 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 8) v1 ==== 89435+0+0 (secure 0 0 0) 0x7f863c003800 con 0x7f8644070650 2023-11-24T07:58:52.167 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:52.161+0000 7f8642ffd700 1 --2- 172.21.15.59:0/695956619 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f863006b110 0x7f863006d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:52.167 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:52.161+0000 7f8642ffd700 1 -- 172.21.15.59:0/695956619 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(60..60 src has 1..60) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f863c089770 con 0x7f8644070650 2023-11-24T07:58:52.167 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:52.162+0000 7f8649a56700 1 --2- 172.21.15.59:0/695956619 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f863006b110 0x7f863006d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:52.168 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:52.162+0000 7f8649a56700 1 --2- 172.21.15.59:0/695956619 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f863006b110 0x7f863006d5d0 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f86440b5310 tx=0x7f8638009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:58:52.168 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:52.167+0000 7f8642ffd700 1 -- 172.21.15.59:0/695956619 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f863c054440 con 0x7f8644070650 2023-11-24T07:58:52.407 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:52.407+0000 7f864c4bb700 1 -- 172.21.15.59:0/695956619 --> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot rm", "vol_name": "cephfs", "sub_name": "subvol1532", "snap_name": "subvol_snap2913", "target": ["mon-mgr", ""]}) v1 -- 0x7f86440b60f0 con 0x7f863006b110 2023-11-24T07:58:52.433 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:52.432+0000 7f8642ffd700 1 -- 172.21.15.59:0/695956619 <== mgr.4100 v2:172.21.15.76:6832/4264046536 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7f86380013e0 con 0x7f863006b110 2023-11-24T07:58:52.435 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:52.434+0000 7f864c4bb700 1 -- 172.21.15.59:0/695956619 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f863006b110 msgr2=0x7f863006d5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:52.435 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:52.434+0000 7f864c4bb700 1 --2- 172.21.15.59:0/695956619 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f863006b110 0x7f863006d5d0 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f86440b5310 tx=0x7f8638009040 comp rx=0 tx=0).stop 2023-11-24T07:58:52.436 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:52.434+0000 7f864c4bb700 1 -- 172.21.15.59:0/695956619 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8644070650 msgr2=0x7f86441e19c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:52.436 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:52.434+0000 7f864c4bb700 1 --2- 172.21.15.59:0/695956619 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8644070650 0x7f86441e19c0 secure :-1 s=READY pgs=745 cs=0 l=1 rev1=1 crypto rx=0x7f863c00e810 tx=0x7f863c00eb20 comp rx=0 tx=0).stop 2023-11-24T07:58:52.436 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:52.434+0000 7f864c4bb700 1 -- 172.21.15.59:0/695956619 shutdown_connections 2023-11-24T07:58:52.436 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:52.434+0000 7f864c4bb700 1 --2- 172.21.15.59:0/695956619 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f863006b110 0x7f863006d5d0 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:52.436 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:52.434+0000 7f864c4bb700 1 --2- 172.21.15.59:0/695956619 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f86440b54a0 0x7f8644146d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:52.436 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:52.434+0000 7f864c4bb700 1 --2- 172.21.15.59:0/695956619 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8644070650 0x7f86441e19c0 unknown :-1 s=CLOSED pgs=745 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:52.436 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:52.434+0000 7f864c4bb700 1 --2- 172.21.15.59:0/695956619 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f86440af1e0 0x7f86441581f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:52.436 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:52.434+0000 7f864c4bb700 1 -- 172.21.15.59:0/695956619 >> 172.21.15.59:0/695956619 conn(0x7f864409a9f0 msgr2=0x7f86440b40a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:58:52.436 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:52.434+0000 7f864c4bb700 1 -- 172.21.15.59:0/695956619 shutdown_connections 2023-11-24T07:58:52.436 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:52.434+0000 7f864c4bb700 1 -- 172.21.15.59:0/695956619 wait complete. 2023-11-24T07:58:52.447 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot rm cephfs subvol1532 subvol_snap2913 2023-11-24T07:58:52.578 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:52.577+0000 7f4cdebd6700 1 Processor -- start 2023-11-24T07:58:52.578 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:52.577+0000 7f4cdebd6700 1 -- start start 2023-11-24T07:58:52.584 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:52.578+0000 7f4cdebd6700 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f4cd80cd150 0x7f4cd80ceb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:52.584 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:52.578+0000 7f4cdebd6700 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4cd806f4b0 con 0x7f4cd815bfe0 2023-11-24T07:58:52.584 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:52.578+0000 7f4cdebd6700 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4cd815c3c0 con 0x7f4cd806f940 2023-11-24T07:58:52.585 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:52.578+0000 7f4cdebd6700 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4cd80cd760 con 0x7f4cd80cd150 2023-11-24T07:58:52.585 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:52.578+0000 7f4cdc972700 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7f4cd815bfe0 0x7f4cd80ce430 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:57574/0 (socket says 172.21.15.59:57574) 2023-11-24T07:58:52.585 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:52.578+0000 7f4cdc972700 1 -- 172.21.15.59:0/4142644458 learned_addr learned my addr 172.21.15.59:0/4142644458 (peer_addr_for_me v1:172.21.15.59:0/0) 2023-11-24T07:58:52.585 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:52.578+0000 7f4cd77fe700 1 -- 172.21.15.59:0/4142644458 <== mon.0 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2377326661 0 0) 0x7f4cd806f4b0 con 0x7f4cd815bfe0 2023-11-24T07:58:52.585 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:52.578+0000 7f4cd7fff700 1 --2- 172.21.15.59:0/4142644458 >> v2:172.21.15.76:3300/0 conn(0x7f4cd80cd150 0x7f4cd80ceb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:52.585 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:52.578+0000 7f4cd77fe700 1 -- 172.21.15.59:0/4142644458 --> v1:172.21.15.59:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4cc80035b0 con 0x7f4cd815bfe0 2023-11-24T07:58:52.585 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:52.579+0000 7f4cd77fe700 1 -- 172.21.15.59:0/4142644458 <== mon.0 v1:172.21.15.59:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4266793835 0 0) 0x7f4cc80035b0 con 0x7f4cd815bfe0 2023-11-24T07:58:52.585 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:52.579+0000 7f4cd77fe700 1 -- 172.21.15.59:0/4142644458 >> v2:172.21.15.76:3300/0 conn(0x7f4cd80cd150 msgr2=0x7f4cd80ceb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T07:58:52.585 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:52.579+0000 7f4cd77fe700 1 --2- 172.21.15.59:0/4142644458 >> v2:172.21.15.76:3300/0 conn(0x7f4cd80cd150 0x7f4cd80ceb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:52.585 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:52.579+0000 7f4cd77fe700 1 -- 172.21.15.59:0/4142644458 >> v1:172.21.15.76:6789/0 conn(0x7f4cd806f940 legacy=0x7f4cd806fd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:52.585 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:52.579+0000 7f4cd77fe700 1 -- 172.21.15.59:0/4142644458 --> v1:172.21.15.59:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4cd806ee60 con 0x7f4cd815bfe0 2023-11-24T07:58:52.586 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:52.579+0000 7f4cd77fe700 1 -- 172.21.15.59:0/4142644458 <== mon.0 v1:172.21.15.59:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1312333566 0 0) 0x7f4ccc001670 con 0x7f4cd815bfe0 2023-11-24T07:58:52.586 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:52.579+0000 7f4cd77fe700 1 -- 172.21.15.59:0/4142644458 >> v1:172.21.15.59:6789/0 conn(0x7f4cd815bfe0 legacy=0x7f4cd80ce430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:52.586 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:52.579+0000 7f4cd77fe700 1 --2- 172.21.15.59:0/4142644458 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4cc8003a00 0x7f4cc8003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:52.586 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:52.579+0000 7f4cd77fe700 1 --2- 172.21.15.59:0/4142644458 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4cc8004850 0x7f4cc8004cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:52.586 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:52.579+0000 7f4cd77fe700 1 --2- 172.21.15.59:0/4142644458 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f4cc8005210 0x7f4cc800ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:52.586 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:52.579+0000 7f4cd77fe700 1 -- 172.21.15.59:0/4142644458 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4cd80cd760 con 0x7f4cc8004850 2023-11-24T07:58:52.586 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:52.579+0000 7f4cd77fe700 1 -- 172.21.15.59:0/4142644458 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4cc800c0b0 con 0x7f4cc8003a00 2023-11-24T07:58:52.586 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:52.579+0000 7f4cd77fe700 1 -- 172.21.15.59:0/4142644458 --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4cc800c270 con 0x7f4cc8005210 2023-11-24T07:58:52.586 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:52.579+0000 7f4cd7fff700 1 --2- 172.21.15.59:0/4142644458 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4cc8004850 0x7f4cc8004cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:52.586 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:52.579+0000 7f4cdc972700 1 --2- 172.21.15.59:0/4142644458 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4cc8003a00 0x7f4cc8003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:52.586 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:52.579+0000 7f4cdd173700 1 --2- 172.21.15.59:0/4142644458 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f4cc8005210 0x7f4cc800ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:52.587 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:52.579+0000 7f4cd7fff700 1 -- 172.21.15.59:0/4142644458 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f4cc8005210 msgr2=0x7f4cc800ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T07:58:52.587 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:52.579+0000 7f4cd7fff700 1 --2- 172.21.15.59:0/4142644458 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f4cc8005210 0x7f4cc800ba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:52.587 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:52.579+0000 7f4cd7fff700 1 -- 172.21.15.59:0/4142644458 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4cc8003a00 msgr2=0x7f4cc8003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T07:58:52.587 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:52.579+0000 7f4cd7fff700 1 --2- 172.21.15.59:0/4142644458 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4cc8003a00 0x7f4cc8003e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:52.587 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:52.579+0000 7f4cd7fff700 1 -- 172.21.15.59:0/4142644458 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4cc80035b0 con 0x7f4cc8004850 2023-11-24T07:58:52.587 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:52.579+0000 7f4cdc972700 1 --2- 172.21.15.59:0/4142644458 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4cc8003a00 0x7f4cc8003e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-24T07:58:52.587 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:52.579+0000 7f4cdd173700 1 -- 172.21.15.59:0/4142644458 reap_dead start 2023-11-24T07:58:52.587 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:52.579+0000 7f4cd7fff700 1 --2- 172.21.15.59:0/4142644458 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4cc8004850 0x7f4cc8004cd0 secure :-1 s=READY pgs=747 cs=0 l=1 rev1=1 crypto rx=0x7f4cc4009b60 tx=0x7f4cc8005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=764dedbd933072df server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:58:52.587 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:52.579+0000 7f4cd77fe700 1 -- 172.21.15.59:0/4142644458 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4cc400e6c0 con 0x7f4cc8004850 2023-11-24T07:58:52.587 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:52.580+0000 7f4cd77fe700 1 -- 172.21.15.59:0/4142644458 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4cc400e820 con 0x7f4cc8004850 2023-11-24T07:58:52.587 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:52.580+0000 7f4cd77fe700 1 -- 172.21.15.59:0/4142644458 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4cc400eb90 con 0x7f4cc8004850 2023-11-24T07:58:52.588 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:52.580+0000 7f4cdebd6700 1 -- 172.21.15.59:0/4142644458 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4cc8004850 msgr2=0x7f4cc8004cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:52.588 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:52.580+0000 7f4cdebd6700 1 --2- 172.21.15.59:0/4142644458 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4cc8004850 0x7f4cc8004cd0 secure :-1 s=READY pgs=747 cs=0 l=1 rev1=1 crypto rx=0x7f4cc4009b60 tx=0x7f4cc8005880 comp rx=0 tx=0).stop 2023-11-24T07:58:52.588 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:52.580+0000 7f4cdebd6700 1 -- 172.21.15.59:0/4142644458 shutdown_connections 2023-11-24T07:58:52.588 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:52.580+0000 7f4cdebd6700 1 --2- 172.21.15.59:0/4142644458 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4cc8004850 0x7f4cc8004cd0 unknown :-1 s=CLOSED pgs=747 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:52.588 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:52.580+0000 7f4cdebd6700 1 -- 172.21.15.59:0/4142644458 >> 172.21.15.59:0/4142644458 conn(0x7f4cd80b3700 msgr2=0x7f4cd80b3b10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:58:52.588 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:52.580+0000 7f4cdebd6700 1 -- 172.21.15.59:0/4142644458 shutdown_connections 2023-11-24T07:58:52.588 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:52.580+0000 7f4cdebd6700 1 -- 172.21.15.59:0/4142644458 wait complete. 2023-11-24T07:58:52.588 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:52.581+0000 7f4cdebd6700 1 Processor -- start 2023-11-24T07:58:52.588 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:52.581+0000 7f4cdebd6700 1 -- start start 2023-11-24T07:58:52.588 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:52.582+0000 7f4cdebd6700 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f4cc8004850 0x7f4cd815d350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:52.588 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:52.582+0000 7f4cdebd6700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4cd81581d0 0x7f4cd815d910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:52.588 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:52.582+0000 7f4cdebd6700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4cd80cacb0 0x7f4cd80cb130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:52.589 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:52.582+0000 7f4cdebd6700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4cd806c800 con 0x7f4cd81581d0 2023-11-24T07:58:52.589 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:52.582+0000 7f4cdebd6700 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4cd80cb670 con 0x7f4cd80cacb0 2023-11-24T07:58:52.589 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:52.582+0000 7f4cdebd6700 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4cd806c180 con 0x7f4cc8004850 2023-11-24T07:58:52.589 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:52.582+0000 7f4cd7fff700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4cd81581d0 0x7f4cd815d910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:52.589 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:52.582+0000 7f4cd7fff700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4cd81581d0 0x7f4cd815d910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:54114/0 (socket says 172.21.15.59:54114) 2023-11-24T07:58:52.589 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:52.582+0000 7f4cd7fff700 1 -- 172.21.15.59:0/2490528762 learned_addr learned my addr 172.21.15.59:0/2490528762 (peer_addr_for_me v2:172.21.15.59:0/0) 2023-11-24T07:58:52.589 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:52.583+0000 7f4cdd173700 1 --2- 172.21.15.59:0/2490528762 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4cd80cacb0 0x7f4cd80cb130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:52.589 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:52.583+0000 7f4cd7fff700 1 -- 172.21.15.59:0/2490528762 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f4cc8004850 msgr2=0x7f4cd815d350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:52.589 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:52.583+0000 7f4cd7fff700 1 --2- 172.21.15.59:0/2490528762 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f4cc8004850 0x7f4cd815d350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:52.589 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:52.583+0000 7f4cd7fff700 1 -- 172.21.15.59:0/2490528762 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4cd80cacb0 msgr2=0x7f4cd80cb130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:52.589 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:52.583+0000 7f4cd7fff700 1 --2- 172.21.15.59:0/2490528762 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4cd80cacb0 0x7f4cd80cb130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:52.590 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:52.583+0000 7f4cd7fff700 1 -- 172.21.15.59:0/2490528762 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4cd815de50 con 0x7f4cd81581d0 2023-11-24T07:58:52.590 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:52.583+0000 7f4cd7fff700 1 --2- 172.21.15.59:0/2490528762 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4cd81581d0 0x7f4cd815d910 secure :-1 s=READY pgs=748 cs=0 l=1 rev1=1 crypto rx=0x7f4cc800c080 tx=0x7f4cc40047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:58:52.590 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:52.583+0000 7f4cd57fa700 1 -- 172.21.15.59:0/2490528762 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4cc400e6c0 con 0x7f4cd81581d0 2023-11-24T07:58:52.590 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:52.583+0000 7f4cd57fa700 1 -- 172.21.15.59:0/2490528762 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4cc4003ad0 con 0x7f4cd81581d0 2023-11-24T07:58:52.590 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:52.583+0000 7f4cd57fa700 1 -- 172.21.15.59:0/2490528762 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4cc4003e80 con 0x7f4cd81581d0 2023-11-24T07:58:52.590 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:52.583+0000 7f4cdebd6700 1 -- 172.21.15.59:0/2490528762 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4cd8158650 con 0x7f4cd81581d0 2023-11-24T07:58:52.590 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:52.583+0000 7f4cdebd6700 1 -- 172.21.15.59:0/2490528762 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4cd806c420 con 0x7f4cd81581d0 2023-11-24T07:58:52.591 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:52.584+0000 7f4cdebd6700 1 -- 172.21.15.59:0/2490528762 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4cd806ca70 con 0x7f4cd81581d0 2023-11-24T07:58:52.591 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:52.585+0000 7f4cd57fa700 1 -- 172.21.15.59:0/2490528762 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 8) v1 ==== 89435+0+0 (secure 0 0 0) 0x7f4cc4013470 con 0x7f4cd81581d0 2023-11-24T07:58:52.591 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:52.585+0000 7f4cd57fa700 1 --2- 172.21.15.59:0/2490528762 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f4cbc06b110 0x7f4cbc06d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:52.591 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:52.585+0000 7f4cd57fa700 1 -- 172.21.15.59:0/2490528762 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(60..60 src has 1..60) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f4cc408b270 con 0x7f4cd81581d0 2023-11-24T07:58:52.591 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:52.585+0000 7f4cdc972700 1 --2- 172.21.15.59:0/2490528762 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f4cbc06b110 0x7f4cbc06d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:52.591 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:52.586+0000 7f4cdc972700 1 --2- 172.21.15.59:0/2490528762 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f4cbc06b110 0x7f4cbc06d5d0 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f4ccc001c90 tx=0x7f4ccc001190 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:58:52.592 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:52.589+0000 7f4cd57fa700 1 -- 172.21.15.59:0/2490528762 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f4cc4056170 con 0x7f4cd81581d0 2023-11-24T07:58:52.830 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:52.829+0000 7f4cdebd6700 1 -- 172.21.15.59:0/2490528762 --> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot rm", "vol_name": "cephfs", "sub_name": "subvol1532", "snap_name": "subvol_snap2913", "target": ["mon-mgr", ""]}) v1 -- 0x7f4cd80cda40 con 0x7f4cbc06b110 2023-11-24T07:58:52.846 INFO:tasks.ceph.mgr.x.smithi076.stderr:2023-11-24T07:58:52.845+0000 7ff196724700 -1 mgr.server reply reply (2) No such file or directory snapshot 'b'subvol_snap2913'' does not exist 2023-11-24T07:58:52.847 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:52.846+0000 7f4cd57fa700 1 -- 172.21.15.59:0/2490528762 <== mgr.4100 v2:172.21.15.76:6832/4264046536 1 ==== mgr_command_reply(tid 0: -2 snapshot 'b'subvol_snap2913'' does not exist) v1 ==== 52+0+0 (secure 0 0 0) 0x7f4ccc004870 con 0x7f4cbc06b110 2023-11-24T07:58:52.848 INFO:teuthology.orchestra.run.smithi059.stderr:Error ENOENT: snapshot 'b'subvol_snap2913'' does not exist 2023-11-24T07:58:52.852 DEBUG:teuthology.orchestra.run:got remote process result: 2 2023-11-24T07:58:52.852 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume rm cephfs subvol1532 2023-11-24T07:58:52.980 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:52.980+0000 7f0fd946f700 1 Processor -- start 2023-11-24T07:58:52.981 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:52.980+0000 7f0fd946f700 1 -- start start 2023-11-24T07:58:52.983 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:52.980+0000 7f0fd946f700 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f0fd40ceb00 0x7f0fd40cfb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:52.983 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:52.980+0000 7f0fd946f700 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f0fd40b0c40 0x7f0fd40d0050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:52.983 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:52.980+0000 7f0fd946f700 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0fd40aeb30 con 0x7f0fd40d0590 2023-11-24T07:58:52.984 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:52.980+0000 7f0fd946f700 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0fd406d9e0 con 0x7f0fd40b0c40 2023-11-24T07:58:52.984 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:52.980+0000 7f0fd946f700 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0fd406d420 con 0x7f0fd40ceb00 2023-11-24T07:58:52.984 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:52.981+0000 7f0fd2ffd700 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f0fd40ceb00 0x7f0fd40cfb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:52.984 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:52.981+0000 7f0fd27fc700 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f0fd40b0c40 0x7f0fd40d0050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:52.984 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:52.981+0000 7f0fd2ffd700 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f0fd40ceb00 0x7f0fd40cfb10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3301/0 says I am v2:172.21.15.59:53560/0 (socket says 172.21.15.59:53560) 2023-11-24T07:58:52.985 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:52.981+0000 7f0fd2ffd700 1 -- 172.21.15.59:0/1068716715 learned_addr learned my addr 172.21.15.59:0/1068716715 (peer_addr_for_me v2:172.21.15.59:0/0) 2023-11-24T07:58:52.985 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:52.981+0000 7f0fd2ffd700 1 -- 172.21.15.59:0/1068716715 >> v2:172.21.15.76:3300/0 conn(0x7f0fd40b0c40 msgr2=0x7f0fd40d0050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T07:58:52.985 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:52.981+0000 7f0fd2ffd700 1 --2- 172.21.15.59:0/1068716715 >> v2:172.21.15.76:3300/0 conn(0x7f0fd40b0c40 0x7f0fd40d0050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:52.985 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:52.981+0000 7f0fd2ffd700 1 -- 172.21.15.59:0/1068716715 >> v1:172.21.15.76:6789/0 conn(0x7f0fd40d0590 legacy=0x7f0fd4161e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:52.985 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:52.981+0000 7f0fd2ffd700 1 -- 172.21.15.59:0/1068716715 --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0fd406d700 con 0x7f0fd40ceb00 2023-11-24T07:58:52.986 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:52.981+0000 7f0fd2ffd700 1 --2- 172.21.15.59:0/1068716715 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f0fd40ceb00 0x7f0fd40cfb10 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f0fc40098f0 tx=0x7f0fc4009c00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=70878a42e2effc0a server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:58:52.986 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:52.982+0000 7f0fd1ffb700 1 -- 172.21.15.59:0/1068716715 <== mon.2 v2:172.21.15.76:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0fc4011830 con 0x7f0fd40ceb00 2023-11-24T07:58:52.986 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:52.982+0000 7f0fd1ffb700 1 -- 172.21.15.59:0/1068716715 <== mon.2 v2:172.21.15.76:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0fc4011990 con 0x7f0fd40ceb00 2023-11-24T07:58:52.986 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:52.982+0000 7f0fd1ffb700 1 -- 172.21.15.59:0/1068716715 <== mon.2 v2:172.21.15.76:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0fc4011d20 con 0x7f0fd40ceb00 2023-11-24T07:58:52.986 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:52.982+0000 7f0fd946f700 1 -- 172.21.15.59:0/1068716715 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f0fd40ceb00 msgr2=0x7f0fd40cfb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:52.987 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:52.982+0000 7f0fd946f700 1 --2- 172.21.15.59:0/1068716715 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f0fd40ceb00 0x7f0fd40cfb10 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f0fc40098f0 tx=0x7f0fc4009c00 comp rx=0 tx=0).stop 2023-11-24T07:58:52.987 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:52.982+0000 7f0fd946f700 1 -- 172.21.15.59:0/1068716715 shutdown_connections 2023-11-24T07:58:52.987 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:52.982+0000 7f0fd946f700 1 --2- 172.21.15.59:0/1068716715 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f0fd40ceb00 0x7f0fd40cfb10 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:52.987 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:52.982+0000 7f0fd946f700 1 --2- 172.21.15.59:0/1068716715 >> v2:172.21.15.76:3300/0 conn(0x7f0fd40b0c40 0x7f0fd40d0050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:52.987 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:52.982+0000 7f0fd946f700 1 -- 172.21.15.59:0/1068716715 >> 172.21.15.59:0/1068716715 conn(0x7f0fd40b3720 msgr2=0x7f0fd40b3b30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:58:52.988 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:52.982+0000 7f0fd946f700 1 -- 172.21.15.59:0/1068716715 shutdown_connections 2023-11-24T07:58:52.988 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:52.982+0000 7f0fd946f700 1 -- 172.21.15.59:0/1068716715 wait complete. 2023-11-24T07:58:52.991 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:52.984+0000 7f0fd946f700 1 Processor -- start 2023-11-24T07:58:52.993 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:52.984+0000 7f0fd946f700 1 -- start start 2023-11-24T07:58:52.993 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:52.984+0000 7f0fd946f700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0fd40b0c40 0x7f0fd40c64b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:52.993 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:52.984+0000 7f0fd946f700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0fd40ceb00 0x7f0fd406f9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:52.993 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:52.984+0000 7f0fd946f700 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f0fd40d0590 0x7f0fd406ff30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:52.994 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:52.984+0000 7f0fd946f700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0fd40c86e0 con 0x7f0fd40b0c40 2023-11-24T07:58:52.994 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:52.984+0000 7f0fd946f700 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0fd40c69f0 con 0x7f0fd40ceb00 2023-11-24T07:58:52.994 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:52.984+0000 7f0fd946f700 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0fd40c7b60 con 0x7f0fd40d0590 2023-11-24T07:58:52.994 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:52.985+0000 7f0fd2ffd700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0fd40b0c40 0x7f0fd40c64b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:52.994 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:52.985+0000 7f0fd2ffd700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0fd40b0c40 0x7f0fd40c64b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:54126/0 (socket says 172.21.15.59:54126) 2023-11-24T07:58:52.995 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:52.985+0000 7f0fd2ffd700 1 -- 172.21.15.59:0/2136243481 learned_addr learned my addr 172.21.15.59:0/2136243481 (peer_addr_for_me v2:172.21.15.59:0/0) 2023-11-24T07:58:52.995 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:52.985+0000 7f0fd27fc700 1 --2- 172.21.15.59:0/2136243481 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0fd40ceb00 0x7f0fd406f9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:52.995 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:52.985+0000 7f0fd37fe700 1 --2- 172.21.15.59:0/2136243481 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f0fd40d0590 0x7f0fd406ff30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:52.995 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:52.985+0000 7f0fd2ffd700 1 -- 172.21.15.59:0/2136243481 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f0fd40d0590 msgr2=0x7f0fd406ff30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:52.995 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:52.985+0000 7f0fd2ffd700 1 --2- 172.21.15.59:0/2136243481 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f0fd40d0590 0x7f0fd406ff30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:52.996 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:52.985+0000 7f0fd2ffd700 1 -- 172.21.15.59:0/2136243481 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0fd40ceb00 msgr2=0x7f0fd406f9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:52.996 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:52.985+0000 7f0fd2ffd700 1 --2- 172.21.15.59:0/2136243481 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0fd40ceb00 0x7f0fd406f9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:52.996 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:52.985+0000 7f0fd2ffd700 1 -- 172.21.15.59:0/2136243481 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0fc4009550 con 0x7f0fd40b0c40 2023-11-24T07:58:52.996 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:52.985+0000 7f0fd27fc700 1 --2- 172.21.15.59:0/2136243481 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0fd40ceb00 0x7f0fd406f9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T07:58:52.996 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:52.985+0000 7f0fd37fe700 1 --2- 172.21.15.59:0/2136243481 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f0fd40d0590 0x7f0fd406ff30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T07:58:52.996 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:52.985+0000 7f0fd2ffd700 1 --2- 172.21.15.59:0/2136243481 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0fd40b0c40 0x7f0fd40c64b0 secure :-1 s=READY pgs=749 cs=0 l=1 rev1=1 crypto rx=0x7f0fc4000c00 tx=0x7f0fc4004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:58:52.997 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:52.985+0000 7f0fbb7fe700 1 -- 172.21.15.59:0/2136243481 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0fc4004570 con 0x7f0fd40b0c40 2023-11-24T07:58:52.997 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:52.985+0000 7f0fbb7fe700 1 -- 172.21.15.59:0/2136243481 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0fc401f070 con 0x7f0fd40b0c40 2023-11-24T07:58:52.997 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:52.985+0000 7f0fd946f700 1 -- 172.21.15.59:0/2136243481 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0fd40c7e40 con 0x7f0fd40b0c40 2023-11-24T07:58:52.997 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:52.985+0000 7f0fbb7fe700 1 -- 172.21.15.59:0/2136243481 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0fc40038e0 con 0x7f0fd40b0c40 2023-11-24T07:58:52.997 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:52.985+0000 7f0fd946f700 1 -- 172.21.15.59:0/2136243481 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0fd40c8400 con 0x7f0fd40b0c40 2023-11-24T07:58:52.998 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:52.986+0000 7f0fd946f700 1 -- 172.21.15.59:0/2136243481 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0fd40d11f0 con 0x7f0fd40b0c40 2023-11-24T07:58:52.998 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:52.987+0000 7f0fbb7fe700 1 -- 172.21.15.59:0/2136243481 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 8) v1 ==== 89435+0+0 (secure 0 0 0) 0x7f0fc4021030 con 0x7f0fd40b0c40 2023-11-24T07:58:52.998 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:52.987+0000 7f0fbb7fe700 1 --2- 172.21.15.59:0/2136243481 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f0fc006b110 0x7f0fc006d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:52.998 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:52.987+0000 7f0fbb7fe700 1 -- 172.21.15.59:0/2136243481 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(60..60 src has 1..60) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f0fc408ae70 con 0x7f0fd40b0c40 2023-11-24T07:58:52.998 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:52.987+0000 7f0fd27fc700 1 --2- 172.21.15.59:0/2136243481 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f0fc006b110 0x7f0fc006d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:52.999 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:52.988+0000 7f0fd27fc700 1 --2- 172.21.15.59:0/2136243481 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f0fc006b110 0x7f0fc006d5d0 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f0fd415cc00 tx=0x7f0fc8008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:58:52.999 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:52.992+0000 7f0fbb7fe700 1 -- 172.21.15.59:0/2136243481 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f0fc4055c50 con 0x7f0fd40b0c40 2023-11-24T07:58:53.227 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:53.226+0000 7f0fd946f700 1 -- 172.21.15.59:0/2136243481 --> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] -- mgr_command(tid 0: {"prefix": "fs subvolume rm", "vol_name": "cephfs", "sub_name": "subvol1532", "target": ["mon-mgr", ""]}) v1 -- 0x7f0fd40ceee0 con 0x7f0fc006b110 2023-11-24T07:58:53.245 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:53.244+0000 7f0fbb7fe700 1 -- 172.21.15.59:0/2136243481 <== mgr.4100 v2:172.21.15.76:6832/4264046536 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7f0fc8004020 con 0x7f0fc006b110 2023-11-24T07:58:53.248 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:53.247+0000 7f0fd946f700 1 -- 172.21.15.59:0/2136243481 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f0fc006b110 msgr2=0x7f0fc006d5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:53.248 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:53.247+0000 7f0fd946f700 1 --2- 172.21.15.59:0/2136243481 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f0fc006b110 0x7f0fc006d5d0 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f0fd415cc00 tx=0x7f0fc8008040 comp rx=0 tx=0).stop 2023-11-24T07:58:53.248 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:53.247+0000 7f0fd946f700 1 -- 172.21.15.59:0/2136243481 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0fd40b0c40 msgr2=0x7f0fd40c64b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:53.248 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:53.247+0000 7f0fd946f700 1 --2- 172.21.15.59:0/2136243481 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0fd40b0c40 0x7f0fd40c64b0 secure :-1 s=READY pgs=749 cs=0 l=1 rev1=1 crypto rx=0x7f0fc4000c00 tx=0x7f0fc4004750 comp rx=0 tx=0).stop 2023-11-24T07:58:53.248 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:53.247+0000 7f0fd946f700 1 -- 172.21.15.59:0/2136243481 shutdown_connections 2023-11-24T07:58:53.248 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:53.247+0000 7f0fd946f700 1 --2- 172.21.15.59:0/2136243481 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f0fc006b110 0x7f0fc006d5d0 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:53.248 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:53.247+0000 7f0fd946f700 1 --2- 172.21.15.59:0/2136243481 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f0fd40d0590 0x7f0fd406ff30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:53.248 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:53.247+0000 7f0fd946f700 1 --2- 172.21.15.59:0/2136243481 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0fd40b0c40 0x7f0fd40c64b0 unknown :-1 s=CLOSED pgs=749 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:53.248 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:53.247+0000 7f0fd946f700 1 --2- 172.21.15.59:0/2136243481 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0fd40ceb00 0x7f0fd406f9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:53.248 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:53.247+0000 7f0fd946f700 1 -- 172.21.15.59:0/2136243481 >> 172.21.15.59:0/2136243481 conn(0x7f0fd40b3720 msgr2=0x7f0fd4073f30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:58:53.249 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:53.247+0000 7f0fd946f700 1 -- 172.21.15.59:0/2136243481 shutdown_connections 2023-11-24T07:58:53.249 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:53.247+0000 7f0fd946f700 1 -- 172.21.15.59:0/2136243481 wait complete. 2023-11-24T07:58:53.250 INFO:tasks.ceph.mgr.x.smithi076.stderr:2023-11-24T07:58:53.250+0000 7ff197f27700 -1 client.0 error registering admin socket command: (17) File exists 2023-11-24T07:58:53.251 INFO:tasks.ceph.mgr.x.smithi076.stderr:2023-11-24T07:58:53.250+0000 7ff197f27700 -1 client.0 error registering admin socket command: (17) File exists 2023-11-24T07:58:53.251 INFO:tasks.ceph.mgr.x.smithi076.stderr:2023-11-24T07:58:53.250+0000 7ff197f27700 -1 client.0 error registering admin socket command: (17) File exists 2023-11-24T07:58:53.251 INFO:tasks.ceph.mgr.x.smithi076.stderr:2023-11-24T07:58:53.250+0000 7ff197f27700 -1 client.0 error registering admin socket command: (17) File exists 2023-11-24T07:58:53.251 INFO:tasks.ceph.mgr.x.smithi076.stderr:2023-11-24T07:58:53.250+0000 7ff197f27700 -1 client.0 error registering admin socket command: (17) File exists 2023-11-24T07:58:53.260 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T07:58:53.261 DEBUG:teuthology.orchestra.run.smithi059:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'stat -c %h /home/ubuntu/cephtest/mnt.0/./volumes/_deleting') 2023-11-24T07:58:53.266 INFO:tasks.ceph.mgr.x.smithi076.stderr:2023-11-24T07:58:53.265+0000 7ff19972a700 -1 client.0 error registering admin socket command: (17) File exists 2023-11-24T07:58:53.266 INFO:tasks.ceph.mgr.x.smithi076.stderr:2023-11-24T07:58:53.265+0000 7ff19972a700 -1 client.0 error registering admin socket command: (17) File exists 2023-11-24T07:58:53.266 INFO:tasks.ceph.mgr.x.smithi076.stderr:2023-11-24T07:58:53.265+0000 7ff19972a700 -1 client.0 error registering admin socket command: (17) File exists 2023-11-24T07:58:53.266 INFO:tasks.ceph.mgr.x.smithi076.stderr:2023-11-24T07:58:53.265+0000 7ff19972a700 -1 client.0 error registering admin socket command: (17) File exists 2023-11-24T07:58:53.266 INFO:tasks.ceph.mgr.x.smithi076.stderr:2023-11-24T07:58:53.265+0000 7ff19972a700 -1 client.0 error registering admin socket command: (17) File exists 2023-11-24T07:58:53.300 INFO:teuthology.orchestra.run.smithi059.stdout:2 2023-11-24T07:58:53.300 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-11-24T07:58:53.300 DEBUG:teuthology.orchestra.run.smithi059:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-11-24T07:58:53.382 DEBUG:teuthology.orchestra.run.smithi076:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-11-24T07:58:53.434 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-11-24T07:58:53.434 DEBUG:teuthology.orchestra.run.smithi059:> set -ex 2023-11-24T07:58:53.434 DEBUG:teuthology.orchestra.run.smithi059:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-24T07:58:53.454 DEBUG:teuthology.orchestra.run.smithi059:> set -ex 2023-11-24T07:58:53.454 DEBUG:teuthology.orchestra.run.smithi059:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-24T07:58:53.512 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-24T07:58:53.678 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:53.677+0000 7f4436377700 1 Processor -- start 2023-11-24T07:58:53.678 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:53.678+0000 7f4436377700 1 -- start start 2023-11-24T07:58:53.681 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:53.678+0000 7f4436377700 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f4430142810 0x7f443006afd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:53.681 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:53.678+0000 7f4436377700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f44301459a0 0x7f443006b510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:53.681 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:53.678+0000 7f4436377700 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f443006e580 0x7f4430071bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:53.681 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:53.678+0000 7f4436377700 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f44300cf5f0 con 0x7f44301459a0 2023-11-24T07:58:53.681 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:53.678+0000 7f4436377700 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f443006ea80 con 0x7f4430142810 2023-11-24T07:58:53.681 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:53.678+0000 7f4436377700 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f443006ed60 con 0x7f443006e580 2023-11-24T07:58:53.681 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:53.678+0000 7f442f7fe700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f44301459a0 0x7f443006b510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:53.681 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:53.679+0000 7f442f7fe700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f44301459a0 0x7f443006b510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:54136/0 (socket says 172.21.15.59:54136) 2023-11-24T07:58:53.681 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:53.679+0000 7f442f7fe700 1 -- 172.21.15.59:0/2004323835 learned_addr learned my addr 172.21.15.59:0/2004323835 (peer_addr_for_me v2:172.21.15.59:0/0) 2023-11-24T07:58:53.681 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:53.679+0000 7f442ffff700 1 --2- 172.21.15.59:0/2004323835 >> v2:172.21.15.76:3300/0 conn(0x7f4430142810 0x7f443006afd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:53.682 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:53.679+0000 7f4434914700 1 --2- 172.21.15.59:0/2004323835 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f443006e580 0x7f4430071bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:53.682 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:53.679+0000 7f442f7fe700 1 -- 172.21.15.59:0/2004323835 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f443006e580 msgr2=0x7f4430071bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T07:58:53.682 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:53.679+0000 7f442f7fe700 1 --2- 172.21.15.59:0/2004323835 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f443006e580 0x7f4430071bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:53.682 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:53.679+0000 7f442f7fe700 1 -- 172.21.15.59:0/2004323835 >> v2:172.21.15.76:3300/0 conn(0x7f4430142810 msgr2=0x7f443006afd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T07:58:53.682 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:53.679+0000 7f442f7fe700 1 --2- 172.21.15.59:0/2004323835 >> v2:172.21.15.76:3300/0 conn(0x7f4430142810 0x7f443006afd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:53.682 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:53.679+0000 7f442f7fe700 1 -- 172.21.15.59:0/2004323835 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f443006f040 con 0x7f44301459a0 2023-11-24T07:58:53.682 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:53.679+0000 7f442f7fe700 1 --2- 172.21.15.59:0/2004323835 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f44301459a0 0x7f443006b510 secure :-1 s=READY pgs=751 cs=0 l=1 rev1=1 crypto rx=0x7f4430069550 tx=0x7f4424009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cad9912e78322381 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:58:53.682 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:53.679+0000 7f442effd700 1 -- 172.21.15.59:0/2004323835 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4424011830 con 0x7f44301459a0 2023-11-24T07:58:53.682 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:53.679+0000 7f442effd700 1 -- 172.21.15.59:0/2004323835 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4424011990 con 0x7f44301459a0 2023-11-24T07:58:53.682 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:53.679+0000 7f442effd700 1 -- 172.21.15.59:0/2004323835 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4424011d20 con 0x7f44301459a0 2023-11-24T07:58:53.683 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:53.680+0000 7f4436377700 1 -- 172.21.15.59:0/2004323835 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f44301459a0 msgr2=0x7f443006b510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:53.683 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:53.680+0000 7f4436377700 1 --2- 172.21.15.59:0/2004323835 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f44301459a0 0x7f443006b510 secure :-1 s=READY pgs=751 cs=0 l=1 rev1=1 crypto rx=0x7f4430069550 tx=0x7f4424009bd0 comp rx=0 tx=0).stop 2023-11-24T07:58:53.683 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:53.680+0000 7f4436377700 1 -- 172.21.15.59:0/2004323835 shutdown_connections 2023-11-24T07:58:53.683 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:53.680+0000 7f4436377700 1 --2- 172.21.15.59:0/2004323835 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f443006e580 0x7f4430071bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:53.683 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:53.680+0000 7f4436377700 1 --2- 172.21.15.59:0/2004323835 >> v2:172.21.15.76:3300/0 conn(0x7f4430142810 0x7f443006afd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:53.683 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:53.680+0000 7f4436377700 1 --2- 172.21.15.59:0/2004323835 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f44301459a0 0x7f443006b510 unknown :-1 s=CLOSED pgs=751 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:53.683 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:53.680+0000 7f4436377700 1 -- 172.21.15.59:0/2004323835 >> 172.21.15.59:0/2004323835 conn(0x7f44300b2740 msgr2=0x7f44300b2b20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:58:53.683 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:53.680+0000 7f4436377700 1 -- 172.21.15.59:0/2004323835 shutdown_connections 2023-11-24T07:58:53.683 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:53.680+0000 7f4436377700 1 -- 172.21.15.59:0/2004323835 wait complete. 2023-11-24T07:58:53.686 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:53.681+0000 7f4436377700 1 Processor -- start 2023-11-24T07:58:53.686 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:53.681+0000 7f4436377700 1 -- start start 2023-11-24T07:58:53.686 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:53.682+0000 7f4436377700 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f443006e580 0x7f44301339c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:53.687 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:53.682+0000 7f4436377700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4430142810 0x7f443015e470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:53.687 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:53.682+0000 7f4436377700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f44301459a0 0x7f443015e9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:53.687 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:53.682+0000 7f4436377700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4430143650 con 0x7f4430142810 2023-11-24T07:58:53.687 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:53.682+0000 7f4436377700 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f44300a9700 con 0x7f44301459a0 2023-11-24T07:58:53.687 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:53.682+0000 7f4436377700 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f443015efe0 con 0x7f443006e580 2023-11-24T07:58:53.687 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:53.682+0000 7f442f7fe700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4430142810 0x7f443015e470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:53.687 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:53.682+0000 7f442f7fe700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4430142810 0x7f443015e470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:54142/0 (socket says 172.21.15.59:54142) 2023-11-24T07:58:53.687 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:53.682+0000 7f442f7fe700 1 -- 172.21.15.59:0/1850150904 learned_addr learned my addr 172.21.15.59:0/1850150904 (peer_addr_for_me v2:172.21.15.59:0/0) 2023-11-24T07:58:53.687 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:53.682+0000 7f442ffff700 1 --2- 172.21.15.59:0/1850150904 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f443006e580 0x7f44301339c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:53.687 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:53.682+0000 7f4434914700 1 --2- 172.21.15.59:0/1850150904 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f44301459a0 0x7f443015e9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:53.687 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:53.682+0000 7f442f7fe700 1 -- 172.21.15.59:0/1850150904 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f443006e580 msgr2=0x7f44301339c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:53.688 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:53.682+0000 7f442f7fe700 1 --2- 172.21.15.59:0/1850150904 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f443006e580 0x7f44301339c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:53.688 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:53.682+0000 7f442f7fe700 1 -- 172.21.15.59:0/1850150904 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f44301459a0 msgr2=0x7f443015e9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:53.688 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:53.683+0000 7f442f7fe700 1 --2- 172.21.15.59:0/1850150904 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f44301459a0 0x7f443015e9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:53.688 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:53.683+0000 7f442f7fe700 1 -- 172.21.15.59:0/1850150904 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4424009550 con 0x7f4430142810 2023-11-24T07:58:53.688 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:53.683+0000 7f442ffff700 1 --2- 172.21.15.59:0/1850150904 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f443006e580 0x7f44301339c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T07:58:53.688 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:53.683+0000 7f4434914700 1 --2- 172.21.15.59:0/1850150904 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f44301459a0 0x7f443015e9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T07:58:53.688 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:53.683+0000 7f442f7fe700 1 --2- 172.21.15.59:0/1850150904 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4430142810 0x7f443015e470 secure :-1 s=READY pgs=752 cs=0 l=1 rev1=1 crypto rx=0x7f4424009c10 tx=0x7f4424004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:58:53.688 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:53.683+0000 7f442cff9700 1 -- 172.21.15.59:0/1850150904 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4424004510 con 0x7f4430142810 2023-11-24T07:58:53.688 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:53.683+0000 7f442cff9700 1 -- 172.21.15.59:0/1850150904 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f442401e070 con 0x7f4430142810 2023-11-24T07:58:53.688 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:53.683+0000 7f4436377700 1 -- 172.21.15.59:0/1850150904 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f443015f2c0 con 0x7f4430142810 2023-11-24T07:58:53.688 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:53.683+0000 7f442cff9700 1 -- 172.21.15.59:0/1850150904 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44240038e0 con 0x7f4430142810 2023-11-24T07:58:53.688 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:53.683+0000 7f4436377700 1 -- 172.21.15.59:0/1850150904 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4430143b80 con 0x7f4430142810 2023-11-24T07:58:53.689 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:53.684+0000 7f4436377700 1 -- 172.21.15.59:0/1850150904 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4430068670 con 0x7f4430142810 2023-11-24T07:58:53.689 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:53.684+0000 7f442cff9700 1 -- 172.21.15.59:0/1850150904 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 9) v1 ==== 89486+0+0 (secure 0 0 0) 0x7f4424003a40 con 0x7f4430142810 2023-11-24T07:58:53.689 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:53.685+0000 7f442cff9700 1 --2- 172.21.15.59:0/1850150904 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f441c06b1a0 0x7f441c06d660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:53.689 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:53.685+0000 7f442cff9700 1 -- 172.21.15.59:0/1850150904 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(60..60 src has 1..60) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f442408a800 con 0x7f4430142810 2023-11-24T07:58:53.689 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:53.685+0000 7f442ffff700 1 --2- 172.21.15.59:0/1850150904 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f441c06b1a0 0x7f441c06d660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:53.689 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:53.686+0000 7f442ffff700 1 --2- 172.21.15.59:0/1850150904 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f441c06b1a0 0x7f441c06d660 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f4420009b60 tx=0x7f4420008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:58:53.690 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:53.689+0000 7f442cff9700 1 -- 172.21.15.59:0/1850150904 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f44240555f0 con 0x7f4430142810 2023-11-24T07:58:53.894 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:53.892+0000 7f4436377700 1 -- 172.21.15.59:0/1850150904 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f44300679e0 con 0x7f4430142810 2023-11-24T07:58:53.894 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:53.893+0000 7f442cff9700 1 -- 172.21.15.59:0/1850150904 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v60) v1 ==== 74+0+16493 (secure 0 0 0) 0x7f4424020090 con 0x7f4430142810 2023-11-24T07:58:53.895 INFO:teuthology.orchestra.run.smithi059.stdout: 2023-11-24T07:58:53.895 INFO:teuthology.orchestra.run.smithi059.stdout:{"epoch":60,"fsid":"9f0e8741-b8f9-48d5-a2c3-09f45903303d","created":"2023-11-24T07:56:06.552729+0000","modified":"2023-11-24T07:58:42.328743+0000","last_up_change":"2023-11-24T07:56:17.372352+0000","last_in_change":"2023-11-24T07:56:09.073250+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-24T07:56:18.352823+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_metadata","create_time":"2023-11-24T07:58:39.262463+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"59","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_data","create_time":"2023-11-24T07:58:39.763399+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"60","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"6d8f9e9b-1782-4e41-928b-a39f2a8ebc06","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6800","nonce":3373878772},{"type":"v1","addr":"172.21.15.59:6801","nonce":3373878772}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6802","nonce":3373878772},{"type":"v1","addr":"172.21.15.59:6803","nonce":3373878772}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6806","nonce":3373878772},{"type":"v1","addr":"172.21.15.59:6807","nonce":3373878772}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6804","nonce":3373878772},{"type":"v1","addr":"172.21.15.59:6805","nonce":3373878772}]},"public_addr":"172.21.15.59:6801/3373878772","cluster_addr":"172.21.15.59:6803/3373878772","heartbeat_back_addr":"172.21.15.59:6807/3373878772","heartbeat_front_addr":"172.21.15.59:6805/3373878772","state":["exists","up"]},{"osd":1,"uuid":"e8d882aa-52e1-4c04-9e1e-c44ef9204a5f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6824","nonce":648258563},{"type":"v1","addr":"172.21.15.59:6825","nonce":648258563}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6826","nonce":648258563},{"type":"v1","addr":"172.21.15.59:6827","nonce":648258563}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6830","nonce":648258563},{"type":"v1","addr":"172.21.15.59:6831","nonce":648258563}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6828","nonce":648258563},{"type":"v1","addr":"172.21.15.59:6829","nonce":648258563}]},"public_addr":"172.21.15.59:6825/648258563","cluster_addr":"172.21.15.59:6827/648258563","heartbeat_back_addr":"172.21.15.59:6831/648258563","heartbeat_front_addr":"172.21.15.59:6829/648258563","state":["exists","up"]},{"osd":2,"uuid":"69eebfa0-695b-4b21-90b9-8a3537168a01","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6808","nonce":1840430654},{"type":"v1","addr":"172.21.15.59:6809","nonce":1840430654}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6810","nonce":1840430654},{"type":"v1","addr":"172.21.15.59:6811","nonce":1840430654}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6814","nonce":1840430654},{"type":"v1","addr":"172.21.15.59:6815","nonce":1840430654}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6812","nonce":1840430654},{"type":"v1","addr":"172.21.15.59:6813","nonce":1840430654}]},"public_addr":"172.21.15.59:6809/1840430654","cluster_addr":"172.21.15.59:6811/1840430654","heartbeat_back_addr":"172.21.15.59:6815/1840430654","heartbeat_front_addr":"172.21.15.59:6813/1840430654","state":["exists","up"]},{"osd":3,"uuid":"96fd77fc-f911-46ff-9445-6f0be2daa8f4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6816","nonce":2504025850},{"type":"v1","addr":"172.21.15.59:6817","nonce":2504025850}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6818","nonce":2504025850},{"type":"v1","addr":"172.21.15.59:6819","nonce":2504025850}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6822","nonce":2504025850},{"type":"v1","addr":"172.21.15.59:6823","nonce":2504025850}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6820","nonce":2504025850},{"type":"v1","addr":"172.21.15.59:6821","nonce":2504025850}]},"public_addr":"172.21.15.59:6817/2504025850","cluster_addr":"172.21.15.59:6819/2504025850","heartbeat_back_addr":"172.21.15.59:6823/2504025850","heartbeat_front_addr":"172.21.15.59:6821/2504025850","state":["exists","up"]},{"osd":4,"uuid":"7ac29fca-b5b2-4c95-9d69-2f164cbbc708","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6816","nonce":2382463431},{"type":"v1","addr":"172.21.15.76:6818","nonce":2382463431}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6820","nonce":2382463431},{"type":"v1","addr":"172.21.15.76:6822","nonce":2382463431}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6828","nonce":2382463431},{"type":"v1","addr":"172.21.15.76:6830","nonce":2382463431}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6824","nonce":2382463431},{"type":"v1","addr":"172.21.15.76:6826","nonce":2382463431}]},"public_addr":"172.21.15.76:6818/2382463431","cluster_addr":"172.21.15.76:6822/2382463431","heartbeat_back_addr":"172.21.15.76:6830/2382463431","heartbeat_front_addr":"172.21.15.76:6826/2382463431","state":["exists","up"]},{"osd":5,"uuid":"485bf711-3851-4238-a4f4-51d52e0a9c58","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6800","nonce":329334308},{"type":"v1","addr":"172.21.15.76:6802","nonce":329334308}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6805","nonce":329334308},{"type":"v1","addr":"172.21.15.76:6807","nonce":329334308}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6814","nonce":329334308},{"type":"v1","addr":"172.21.15.76:6815","nonce":329334308}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6810","nonce":329334308},{"type":"v1","addr":"172.21.15.76:6812","nonce":329334308}]},"public_addr":"172.21.15.76:6802/329334308","cluster_addr":"172.21.15.76:6807/329334308","heartbeat_back_addr":"172.21.15.76:6815/329334308","heartbeat_front_addr":"172.21.15.76:6812/329334308","state":["exists","up"]},{"osd":6,"uuid":"72b3f2fc-1e4a-4e32-968d-003647a0286f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6801","nonce":3946579410},{"type":"v1","addr":"172.21.15.76:6803","nonce":3946579410}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6804","nonce":3946579410},{"type":"v1","addr":"172.21.15.76:6806","nonce":3946579410}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6811","nonce":3946579410},{"type":"v1","addr":"172.21.15.76:6813","nonce":3946579410}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6808","nonce":3946579410},{"type":"v1","addr":"172.21.15.76:6809","nonce":3946579410}]},"public_addr":"172.21.15.76:6803/3946579410","cluster_addr":"172.21.15.76:6806/3946579410","heartbeat_back_addr":"172.21.15.76:6813/3946579410","heartbeat_front_addr":"172.21.15.76:6809/3946579410","state":["exists","up"]},{"osd":7,"uuid":"1ddceb90-6c5d-4d1a-ad56-99cb877f8c70","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6817","nonce":841761538},{"type":"v1","addr":"172.21.15.76:6819","nonce":841761538}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6821","nonce":841761538},{"type":"v1","addr":"172.21.15.76:6823","nonce":841761538}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6829","nonce":841761538},{"type":"v1","addr":"172.21.15.76:6831","nonce":841761538}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6825","nonce":841761538},{"type":"v1","addr":"172.21.15.76:6827","nonce":841761538}]},"public_addr":"172.21.15.76:6819/841761538","cluster_addr":"172.21.15.76:6823/841761538","heartbeat_back_addr":"172.21.15.76:6831/841761538","heartbeat_front_addr":"172.21.15.76:6827/841761538","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T07:56:14.294838+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T07:56:14.270779+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T07:56:14.231809+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T07:56:14.260983+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T07:56:14.302997+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T07:56:14.258600+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T07:56:14.251413+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T07:56:14.304727+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-24T07:58:53.897 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:53.896+0000 7f4436377700 1 -- 172.21.15.59:0/1850150904 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f441c06b1a0 msgr2=0x7f441c06d660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:53.897 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:53.896+0000 7f4436377700 1 --2- 172.21.15.59:0/1850150904 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f441c06b1a0 0x7f441c06d660 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f4420009b60 tx=0x7f4420008040 comp rx=0 tx=0).stop 2023-11-24T07:58:53.897 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:53.896+0000 7f4436377700 1 -- 172.21.15.59:0/1850150904 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4430142810 msgr2=0x7f443015e470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:53.897 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:53.896+0000 7f4436377700 1 --2- 172.21.15.59:0/1850150904 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4430142810 0x7f443015e470 secure :-1 s=READY pgs=752 cs=0 l=1 rev1=1 crypto rx=0x7f4424009c10 tx=0x7f4424004750 comp rx=0 tx=0).stop 2023-11-24T07:58:53.897 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:53.896+0000 7f4436377700 1 -- 172.21.15.59:0/1850150904 shutdown_connections 2023-11-24T07:58:53.897 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:53.896+0000 7f4436377700 1 --2- 172.21.15.59:0/1850150904 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f441c06b1a0 0x7f441c06d660 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:53.897 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:53.896+0000 7f4436377700 1 --2- 172.21.15.59:0/1850150904 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f44301459a0 0x7f443015e9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:53.897 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:53.896+0000 7f4436377700 1 --2- 172.21.15.59:0/1850150904 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f443006e580 0x7f44301339c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:53.897 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:53.896+0000 7f4436377700 1 --2- 172.21.15.59:0/1850150904 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4430142810 0x7f443015e470 unknown :-1 s=CLOSED pgs=752 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:53.897 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:53.896+0000 7f4436377700 1 -- 172.21.15.59:0/1850150904 >> 172.21.15.59:0/1850150904 conn(0x7f44300b2740 msgr2=0x7f44300b1920 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:58:53.898 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:53.896+0000 7f4436377700 1 -- 172.21.15.59:0/1850150904 shutdown_connections 2023-11-24T07:58:53.898 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:53.896+0000 7f4436377700 1 -- 172.21.15.59:0/1850150904 wait complete. 2023-11-24T07:58:53.909 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-24T07:56:18.352823+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 8, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-24T07:58:39.262463+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '59', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 9, 'pool_name': 'cephfs_data', 'create_time': '2023-11-24T07:58:39.763399+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '60', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-24T07:58:53.910 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-24T07:58:54.037 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:54.036+0000 7f1134b88700 1 Processor -- start 2023-11-24T07:58:54.037 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:54.036+0000 7f1134b88700 1 -- start start 2023-11-24T07:58:54.040 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:54.037+0000 7f1134b88700 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f113015bde0 0x7f113015c1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:54.040 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:54.037+0000 7f1134b88700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f11300ce5c0 0x7f11300cea30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:54.040 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:54.037+0000 7f1134b88700 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f11300d0a90 con 0x7f11300cf2b0 2023-11-24T07:58:54.040 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:54.037+0000 7f1134b88700 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f11300cef70 con 0x7f11300ce5c0 2023-11-24T07:58:54.040 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:54.037+0000 7f1134b88700 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f113006ee10 con 0x7f113015bde0 2023-11-24T07:58:54.040 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:54.038+0000 7f112ed9d700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f11300ce5c0 0x7f11300cea30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:54.040 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:54.038+0000 7f112ed9d700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f11300ce5c0 0x7f11300cea30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:54152/0 (socket says 172.21.15.59:54152) 2023-11-24T07:58:54.040 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:54.038+0000 7f112dd9b700 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7f11300cf2b0 0x7f11300d01b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:57622/0 (socket says 172.21.15.59:57622) 2023-11-24T07:58:54.040 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:54.038+0000 7f112ed9d700 1 -- 172.21.15.59:0/850351622 learned_addr learned my addr 172.21.15.59:0/850351622 (peer_addr_for_me v2:172.21.15.59:0/0) 2023-11-24T07:58:54.040 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:54.038+0000 7f112e59c700 1 --2- 172.21.15.59:0/850351622 >> v2:172.21.15.76:3300/0 conn(0x7f113015bde0 0x7f113015c1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:54.040 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:54.038+0000 7f112ed9d700 1 -- 172.21.15.59:0/850351622 >> v2:172.21.15.76:3300/0 conn(0x7f113015bde0 msgr2=0x7f113015c1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T07:58:54.041 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:54.038+0000 7f112ed9d700 1 --2- 172.21.15.59:0/850351622 >> v2:172.21.15.76:3300/0 conn(0x7f113015bde0 0x7f113015c1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:54.041 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:54.038+0000 7f112ed9d700 1 -- 172.21.15.59:0/850351622 >> v1:172.21.15.59:6789/0 conn(0x7f11300cf2b0 legacy=0x7f11300d01b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:54.041 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:54.038+0000 7f112d59a700 1 -- 172.21.15.59:0/850351622 <== mon.0 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 438187854 0 0) 0x7f11300d0a90 con 0x7f11300cf2b0 2023-11-24T07:58:54.041 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:54.038+0000 7f112ed9d700 1 -- 172.21.15.59:0/850351622 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f113006f0f0 con 0x7f11300ce5c0 2023-11-24T07:58:54.041 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:54.038+0000 7f112e59c700 1 --2- 172.21.15.59:0/850351622 >> v2:172.21.15.76:3300/0 conn(0x7f113015bde0 0x7f113015c1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T07:58:54.041 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:54.038+0000 7f112ed9d700 1 --2- 172.21.15.59:0/850351622 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f11300ce5c0 0x7f11300cea30 secure :-1 s=READY pgs=754 cs=0 l=1 rev1=1 crypto rx=0x7f1130072370 tx=0x7f112800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2411defa12450a90 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:58:54.041 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:54.038+0000 7f112d59a700 1 -- 172.21.15.59:0/850351622 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f112800e070 con 0x7f11300ce5c0 2023-11-24T07:58:54.041 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:54.038+0000 7f112d59a700 1 -- 172.21.15.59:0/850351622 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f112800b9c0 con 0x7f11300ce5c0 2023-11-24T07:58:54.041 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:54.038+0000 7f112d59a700 1 -- 172.21.15.59:0/850351622 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f112800bd70 con 0x7f11300ce5c0 2023-11-24T07:58:54.041 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:54.039+0000 7f1134b88700 1 -- 172.21.15.59:0/850351622 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f11300ce5c0 msgr2=0x7f11300cea30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:54.041 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:54.039+0000 7f1134b88700 1 --2- 172.21.15.59:0/850351622 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f11300ce5c0 0x7f11300cea30 secure :-1 s=READY pgs=754 cs=0 l=1 rev1=1 crypto rx=0x7f1130072370 tx=0x7f112800b330 comp rx=0 tx=0).stop 2023-11-24T07:58:54.041 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:54.039+0000 7f1134b88700 1 -- 172.21.15.59:0/850351622 shutdown_connections 2023-11-24T07:58:54.041 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:54.039+0000 7f1134b88700 1 --2- 172.21.15.59:0/850351622 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f11300ce5c0 0x7f11300cea30 unknown :-1 s=CLOSED pgs=754 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:54.042 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:54.039+0000 7f1134b88700 1 --2- 172.21.15.59:0/850351622 >> v2:172.21.15.76:3300/0 conn(0x7f113015bde0 0x7f113015c1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:54.042 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:54.039+0000 7f1134b88700 1 -- 172.21.15.59:0/850351622 >> 172.21.15.59:0/850351622 conn(0x7f11300b3720 msgr2=0x7f11300b3b30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:58:54.042 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:54.039+0000 7f1134b88700 1 -- 172.21.15.59:0/850351622 shutdown_connections 2023-11-24T07:58:54.042 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:54.039+0000 7f1134b88700 1 -- 172.21.15.59:0/850351622 wait complete. 2023-11-24T07:58:54.044 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:54.040+0000 7f1134b88700 1 Processor -- start 2023-11-24T07:58:54.044 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:54.040+0000 7f1134b88700 1 -- start start 2023-11-24T07:58:54.044 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:54.041+0000 7f1134b88700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f11300ce5c0 0x7f11300cb5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:54.044 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:54.041+0000 7f1134b88700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f11300cf2b0 0x7f11300b65a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:54.044 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:54.041+0000 7f1134b88700 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f113015bde0 0x7f113014a880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:54.045 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:54.041+0000 7f1134b88700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11300b7a70 con 0x7f11300cf2b0 2023-11-24T07:58:54.045 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:54.041+0000 7f1134b88700 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11300b6ef0 con 0x7f11300ce5c0 2023-11-24T07:58:54.045 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:54.041+0000 7f1134b88700 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f11300b71d0 con 0x7f113015bde0 2023-11-24T07:58:54.045 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:54.041+0000 7f112dd9b700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f11300cf2b0 0x7f11300b65a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:54.045 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:54.041+0000 7f112dd9b700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f11300cf2b0 0x7f11300b65a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:54156/0 (socket says 172.21.15.59:54156) 2023-11-24T07:58:54.045 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:54.041+0000 7f112dd9b700 1 -- 172.21.15.59:0/1475304187 learned_addr learned my addr 172.21.15.59:0/1475304187 (peer_addr_for_me v2:172.21.15.59:0/0) 2023-11-24T07:58:54.045 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:54.041+0000 7f112ed9d700 1 --2- 172.21.15.59:0/1475304187 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f113015bde0 0x7f113014a880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:54.045 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:54.041+0000 7f112e59c700 1 --2- 172.21.15.59:0/1475304187 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f11300ce5c0 0x7f11300cb5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:54.045 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:54.041+0000 7f112dd9b700 1 -- 172.21.15.59:0/1475304187 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f113015bde0 msgr2=0x7f113014a880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:54.045 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:54.041+0000 7f112dd9b700 1 --2- 172.21.15.59:0/1475304187 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f113015bde0 0x7f113014a880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:54.045 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:54.041+0000 7f112dd9b700 1 -- 172.21.15.59:0/1475304187 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f11300ce5c0 msgr2=0x7f11300cb5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:54.045 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:54.041+0000 7f112dd9b700 1 --2- 172.21.15.59:0/1475304187 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f11300ce5c0 0x7f11300cb5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:54.046 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:54.041+0000 7f112dd9b700 1 -- 172.21.15.59:0/1475304187 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1128009b30 con 0x7f11300cf2b0 2023-11-24T07:58:54.046 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:54.041+0000 7f112ed9d700 1 --2- 172.21.15.59:0/1475304187 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f113015bde0 0x7f113014a880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-24T07:58:54.046 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:54.041+0000 7f112e59c700 1 --2- 172.21.15.59:0/1475304187 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f11300ce5c0 0x7f11300cb5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-24T07:58:54.046 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:54.042+0000 7f112dd9b700 1 --2- 172.21.15.59:0/1475304187 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f11300cf2b0 0x7f11300b65a0 secure :-1 s=READY pgs=755 cs=0 l=1 rev1=1 crypto rx=0x7f112000c540 tx=0x7f112000c900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:58:54.046 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:54.042+0000 7f111effd700 1 -- 172.21.15.59:0/1475304187 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1120004c80 con 0x7f11300cf2b0 2023-11-24T07:58:54.046 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:54.042+0000 7f111effd700 1 -- 172.21.15.59:0/1475304187 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1120004de0 con 0x7f11300cf2b0 2023-11-24T07:58:54.046 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:54.042+0000 7f1134b88700 1 -- 172.21.15.59:0/1475304187 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f11300b74b0 con 0x7f11300cf2b0 2023-11-24T07:58:54.046 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:54.042+0000 7f111effd700 1 -- 172.21.15.59:0/1475304187 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1120007310 con 0x7f11300cf2b0 2023-11-24T07:58:54.046 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:54.042+0000 7f1134b88700 1 -- 172.21.15.59:0/1475304187 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f11300cbb20 con 0x7f11300cf2b0 2023-11-24T07:58:54.046 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:54.043+0000 7f1134b88700 1 -- 172.21.15.59:0/1475304187 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f113015d3d0 con 0x7f11300cf2b0 2023-11-24T07:58:54.046 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:54.043+0000 7f111effd700 1 -- 172.21.15.59:0/1475304187 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 9) v1 ==== 89486+0+0 (secure 0 0 0) 0x7f1120007470 con 0x7f11300cf2b0 2023-11-24T07:58:54.046 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:54.044+0000 7f111effd700 1 --2- 172.21.15.59:0/1475304187 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f111006b1a0 0x7f111006d660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:54.047 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:54.044+0000 7f111effd700 1 -- 172.21.15.59:0/1475304187 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(60..60 src has 1..60) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f11200897a0 con 0x7f11300cf2b0 2023-11-24T07:58:54.047 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:54.044+0000 7f112e59c700 1 --2- 172.21.15.59:0/1475304187 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f111006b1a0 0x7f111006d660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:54.047 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:54.044+0000 7f112e59c700 1 --2- 172.21.15.59:0/1475304187 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f111006b1a0 0x7f111006d660 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f112400aa20 tx=0x7f1124008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:58:54.049 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:54.048+0000 7f111effd700 1 -- 172.21.15.59:0/1475304187 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f1120054550 con 0x7f11300cf2b0 2023-11-24T07:58:54.248 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:54.247+0000 7f1134b88700 1 -- 172.21.15.59:0/1475304187 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f11300d0cf0 con 0x7f11300cf2b0 2023-11-24T07:58:54.249 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:54.248+0000 7f111effd700 1 -- 172.21.15.59:0/1475304187 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v60) v1 ==== 93+0+10 (secure 0 0 0) 0x7f1120057b70 con 0x7f11300cf2b0 2023-11-24T07:58:54.249 INFO:teuthology.orchestra.run.smithi059.stdout:pg_num: 1 2023-11-24T07:58:54.251 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:54.250+0000 7f1134b88700 1 -- 172.21.15.59:0/1475304187 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f111006b1a0 msgr2=0x7f111006d660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:54.251 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:54.250+0000 7f1134b88700 1 --2- 172.21.15.59:0/1475304187 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f111006b1a0 0x7f111006d660 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f112400aa20 tx=0x7f1124008040 comp rx=0 tx=0).stop 2023-11-24T07:58:54.251 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:54.250+0000 7f1134b88700 1 -- 172.21.15.59:0/1475304187 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f11300cf2b0 msgr2=0x7f11300b65a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:54.251 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:54.250+0000 7f1134b88700 1 --2- 172.21.15.59:0/1475304187 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f11300cf2b0 0x7f11300b65a0 secure :-1 s=READY pgs=755 cs=0 l=1 rev1=1 crypto rx=0x7f112000c540 tx=0x7f112000c900 comp rx=0 tx=0).stop 2023-11-24T07:58:54.251 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:54.250+0000 7f1134b88700 1 -- 172.21.15.59:0/1475304187 shutdown_connections 2023-11-24T07:58:54.252 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:54.250+0000 7f1134b88700 1 --2- 172.21.15.59:0/1475304187 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f111006b1a0 0x7f111006d660 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:54.252 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:54.250+0000 7f1134b88700 1 --2- 172.21.15.59:0/1475304187 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f113015bde0 0x7f113014a880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:54.252 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:54.250+0000 7f1134b88700 1 --2- 172.21.15.59:0/1475304187 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f11300ce5c0 0x7f11300cb5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:54.252 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:54.250+0000 7f1134b88700 1 --2- 172.21.15.59:0/1475304187 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f11300cf2b0 0x7f11300b65a0 unknown :-1 s=CLOSED pgs=755 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:54.252 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:54.250+0000 7f1134b88700 1 -- 172.21.15.59:0/1475304187 >> 172.21.15.59:0/1475304187 conn(0x7f11300b3720 msgr2=0x7f113015adf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:58:54.253 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:54.250+0000 7f1134b88700 1 -- 172.21.15.59:0/1475304187 shutdown_connections 2023-11-24T07:58:54.253 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:54.250+0000 7f1134b88700 1 -- 172.21.15.59:0/1475304187 wait complete. 2023-11-24T07:58:54.263 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-24T07:58:54.392 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:54.391+0000 7fdeae259700 1 Processor -- start 2023-11-24T07:58:54.392 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:54.391+0000 7fdeae259700 1 -- start start 2023-11-24T07:58:54.394 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:54.392+0000 7fdeae259700 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fdea815e190 0x7fdea80d0f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:54.394 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:54.392+0000 7fdeae259700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fdea80d1490 0x7fdea80d24d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:54.394 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:54.392+0000 7fdeae259700 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdea80b7810 con 0x7fdea80cfde0 2023-11-24T07:58:54.394 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:54.392+0000 7fdeae259700 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdea80d1930 con 0x7fdea80d1490 2023-11-24T07:58:54.394 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:54.392+0000 7fdeae259700 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdea80b6ee0 con 0x7fdea815e190 2023-11-24T07:58:54.394 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:54.392+0000 7fdea7fff700 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7fdea80cfde0 0x7fdea80b4ab0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:57640/0 (socket says 172.21.15.59:57640) 2023-11-24T07:58:54.394 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:54.392+0000 7fdea7fff700 1 -- 172.21.15.59:0/3584491483 learned_addr learned my addr 172.21.15.59:0/3584491483 (peer_addr_for_me v1:172.21.15.59:0/0) 2023-11-24T07:58:54.394 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:54.392+0000 7fdea6ffd700 1 --2- 172.21.15.59:0/3584491483 >> v2:172.21.15.59:3300/0 conn(0x7fdea80d1490 0x7fdea80d24d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:54.395 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:54.393+0000 7fdea77fe700 1 --2- 172.21.15.59:0/3584491483 >> v2:172.21.15.76:3300/0 conn(0x7fdea815e190 0x7fdea80d0f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:54.395 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:54.393+0000 7fdea67fc700 1 -- 172.21.15.59:0/3584491483 <== mon.0 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 290268185 0 0) 0x7fdea80b7810 con 0x7fdea80cfde0 2023-11-24T07:58:54.395 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:54.393+0000 7fdea67fc700 1 -- 172.21.15.59:0/3584491483 --> v1:172.21.15.59:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fde900035b0 con 0x7fdea80cfde0 2023-11-24T07:58:54.395 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:54.393+0000 7fdea6ffd700 1 -- 172.21.15.59:0/3584491483 >> v2:172.21.15.76:3300/0 conn(0x7fdea815e190 msgr2=0x7fdea80d0f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T07:58:54.396 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:54.393+0000 7fdea6ffd700 1 --2- 172.21.15.59:0/3584491483 >> v2:172.21.15.76:3300/0 conn(0x7fdea815e190 0x7fdea80d0f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:54.396 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:54.393+0000 7fdea6ffd700 1 -- 172.21.15.59:0/3584491483 >> v1:172.21.15.59:6789/0 conn(0x7fdea80cfde0 legacy=0x7fdea80b4ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:54.396 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:54.393+0000 7fdea6ffd700 1 -- 172.21.15.59:0/3584491483 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdea80b71c0 con 0x7fdea80d1490 2023-11-24T07:58:54.396 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:54.393+0000 7fdea6ffd700 1 --2- 172.21.15.59:0/3584491483 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fdea80d1490 0x7fdea80d24d0 secure :-1 s=READY pgs=757 cs=0 l=1 rev1=1 crypto rx=0x7fdea808c730 tx=0x7fde98009b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c355967f42c3d690 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:58:54.396 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:54.393+0000 7fdea67fc700 1 -- 172.21.15.59:0/3584491483 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde98011830 con 0x7fdea80d1490 2023-11-24T07:58:54.396 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:54.393+0000 7fdea67fc700 1 -- 172.21.15.59:0/3584491483 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fde98011990 con 0x7fdea80d1490 2023-11-24T07:58:54.397 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:54.393+0000 7fdea67fc700 1 -- 172.21.15.59:0/3584491483 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde98011d20 con 0x7fdea80d1490 2023-11-24T07:58:54.400 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:54.394+0000 7fdeae259700 1 -- 172.21.15.59:0/3584491483 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fdea80d1490 msgr2=0x7fdea80d24d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:54.400 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:54.394+0000 7fdeae259700 1 --2- 172.21.15.59:0/3584491483 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fdea80d1490 0x7fdea80d24d0 secure :-1 s=READY pgs=757 cs=0 l=1 rev1=1 crypto rx=0x7fdea808c730 tx=0x7fde98009b20 comp rx=0 tx=0).stop 2023-11-24T07:58:54.401 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:54.394+0000 7fdeae259700 1 -- 172.21.15.59:0/3584491483 shutdown_connections 2023-11-24T07:58:54.401 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:54.394+0000 7fdeae259700 1 --2- 172.21.15.59:0/3584491483 >> v2:172.21.15.76:3300/0 conn(0x7fdea815e190 0x7fdea80d0f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:54.401 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:54.394+0000 7fdeae259700 1 --2- 172.21.15.59:0/3584491483 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fdea80d1490 0x7fdea80d24d0 unknown :-1 s=CLOSED pgs=757 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:54.401 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:54.394+0000 7fdeae259700 1 -- 172.21.15.59:0/3584491483 >> 172.21.15.59:0/3584491483 conn(0x7fdea809d180 msgr2=0x7fdea80d3ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:58:54.401 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:54.394+0000 7fdeae259700 1 -- 172.21.15.59:0/3584491483 shutdown_connections 2023-11-24T07:58:54.402 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:54.394+0000 7fdeae259700 1 -- 172.21.15.59:0/3584491483 wait complete. 2023-11-24T07:58:54.402 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:54.395+0000 7fdeae259700 1 Processor -- start 2023-11-24T07:58:54.402 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:54.395+0000 7fdeae259700 1 -- start start 2023-11-24T07:58:54.402 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:54.396+0000 7fdeae259700 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fdea80cfde0 0x7fdea8161160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:54.403 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:54.396+0000 7fdeae259700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fdea80d1490 0x7fdea80ca3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:54.403 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:54.396+0000 7fdeae259700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fdea815e190 0x7fdea80c8b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:54.403 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:54.396+0000 7fdeae259700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdea80ca920 con 0x7fdea815e190 2023-11-24T07:58:54.403 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:54.396+0000 7fdeae259700 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdea80c9f50 con 0x7fdea80d1490 2023-11-24T07:58:54.403 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:54.396+0000 7fdeae259700 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdea80c9340 con 0x7fdea80cfde0 2023-11-24T07:58:54.403 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:54.396+0000 7fdea7fff700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fdea815e190 0x7fdea80c8b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:54.403 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:54.396+0000 7fdea7fff700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fdea815e190 0x7fdea80c8b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:54174/0 (socket says 172.21.15.59:54174) 2023-11-24T07:58:54.404 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:54.396+0000 7fdea7fff700 1 -- 172.21.15.59:0/2875680238 learned_addr learned my addr 172.21.15.59:0/2875680238 (peer_addr_for_me v2:172.21.15.59:0/0) 2023-11-24T07:58:54.405 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:54.396+0000 7fdea6ffd700 1 --2- 172.21.15.59:0/2875680238 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fdea80d1490 0x7fdea80ca3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:54.405 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:54.396+0000 7fdea77fe700 1 --2- 172.21.15.59:0/2875680238 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fdea80cfde0 0x7fdea8161160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:54.405 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:54.397+0000 7fdea7fff700 1 -- 172.21.15.59:0/2875680238 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fdea80cfde0 msgr2=0x7fdea8161160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:54.405 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:54.397+0000 7fdea7fff700 1 --2- 172.21.15.59:0/2875680238 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fdea80cfde0 0x7fdea8161160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:54.406 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:54.397+0000 7fdea7fff700 1 -- 172.21.15.59:0/2875680238 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fdea80d1490 msgr2=0x7fdea80ca3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:54.406 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:54.397+0000 7fdea7fff700 1 --2- 172.21.15.59:0/2875680238 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fdea80d1490 0x7fdea80ca3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:54.406 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:54.397+0000 7fdea7fff700 1 -- 172.21.15.59:0/2875680238 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fde980094a0 con 0x7fdea815e190 2023-11-24T07:58:54.406 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:54.397+0000 7fdea77fe700 1 --2- 172.21.15.59:0/2875680238 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fdea80cfde0 0x7fdea8161160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-24T07:58:54.406 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:54.397+0000 7fdea7fff700 1 --2- 172.21.15.59:0/2875680238 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fdea815e190 0x7fdea80c8b10 secure :-1 s=READY pgs=758 cs=0 l=1 rev1=1 crypto rx=0x7fde9c001eb0 tx=0x7fde9c00c7b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:58:54.406 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:54.397+0000 7fde8ffff700 1 -- 172.21.15.59:0/2875680238 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde9c005d70 con 0x7fdea815e190 2023-11-24T07:58:54.406 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:54.397+0000 7fde8ffff700 1 -- 172.21.15.59:0/2875680238 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fde9c0070d0 con 0x7fdea815e190 2023-11-24T07:58:54.407 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:54.397+0000 7fde8ffff700 1 -- 172.21.15.59:0/2875680238 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde9c007440 con 0x7fdea815e190 2023-11-24T07:58:54.407 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:54.397+0000 7fdeae259700 1 -- 172.21.15.59:0/2875680238 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdea80c9620 con 0x7fdea815e190 2023-11-24T07:58:54.407 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:54.397+0000 7fdeae259700 1 -- 172.21.15.59:0/2875680238 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdea80c9be0 con 0x7fdea815e190 2023-11-24T07:58:54.407 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:54.398+0000 7fdeae259700 1 -- 172.21.15.59:0/2875680238 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdea8075480 con 0x7fdea815e190 2023-11-24T07:58:54.407 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:54.398+0000 7fde8ffff700 1 -- 172.21.15.59:0/2875680238 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 9) v1 ==== 89486+0+0 (secure 0 0 0) 0x7fde9c0075a0 con 0x7fdea815e190 2023-11-24T07:58:54.407 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:54.399+0000 7fde8ffff700 1 --2- 172.21.15.59:0/2875680238 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7fde9406b1a0 0x7fde9406d660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:54.407 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:54.399+0000 7fde8ffff700 1 -- 172.21.15.59:0/2875680238 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(60..60 src has 1..60) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fde9c088970 con 0x7fdea815e190 2023-11-24T07:58:54.408 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:54.399+0000 7fdea77fe700 1 --2- 172.21.15.59:0/2875680238 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7fde9406b1a0 0x7fde9406d660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:54.408 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:54.399+0000 7fdea77fe700 1 --2- 172.21.15.59:0/2875680238 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7fde9406b1a0 0x7fde9406d660 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7fdea80b65a0 tx=0x7fdea000b040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:58:54.408 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:54.404+0000 7fde8ffff700 1 -- 172.21.15.59:0/2875680238 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fde9c0538d0 con 0x7fdea815e190 2023-11-24T07:58:54.607 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:54.606+0000 7fdeae259700 1 -- 172.21.15.59:0/2875680238 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fdea80d31c0 con 0x7fdea815e190 2023-11-24T07:58:54.607 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:54.606+0000 7fde8ffff700 1 -- 172.21.15.59:0/2875680238 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v60) v1 ==== 104+0+11 (secure 0 0 0) 0x7fde9c017020 con 0x7fdea815e190 2023-11-24T07:58:54.607 INFO:teuthology.orchestra.run.smithi059.stdout:pg_num: 32 2023-11-24T07:58:54.610 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:54.608+0000 7fdeae259700 1 -- 172.21.15.59:0/2875680238 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7fde9406b1a0 msgr2=0x7fde9406d660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:54.610 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:54.608+0000 7fdeae259700 1 --2- 172.21.15.59:0/2875680238 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7fde9406b1a0 0x7fde9406d660 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7fdea80b65a0 tx=0x7fdea000b040 comp rx=0 tx=0).stop 2023-11-24T07:58:54.610 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:54.608+0000 7fdeae259700 1 -- 172.21.15.59:0/2875680238 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fdea815e190 msgr2=0x7fdea80c8b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:54.610 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:54.608+0000 7fdeae259700 1 --2- 172.21.15.59:0/2875680238 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fdea815e190 0x7fdea80c8b10 secure :-1 s=READY pgs=758 cs=0 l=1 rev1=1 crypto rx=0x7fde9c001eb0 tx=0x7fde9c00c7b0 comp rx=0 tx=0).stop 2023-11-24T07:58:54.610 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:54.608+0000 7fdeae259700 1 -- 172.21.15.59:0/2875680238 shutdown_connections 2023-11-24T07:58:54.611 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:54.608+0000 7fdeae259700 1 --2- 172.21.15.59:0/2875680238 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7fde9406b1a0 0x7fde9406d660 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:54.611 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:54.608+0000 7fdeae259700 1 --2- 172.21.15.59:0/2875680238 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fdea815e190 0x7fdea80c8b10 unknown :-1 s=CLOSED pgs=758 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:54.611 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:54.608+0000 7fdeae259700 1 --2- 172.21.15.59:0/2875680238 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fdea80cfde0 0x7fdea8161160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:54.611 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:54.608+0000 7fdeae259700 1 --2- 172.21.15.59:0/2875680238 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fdea80d1490 0x7fdea80ca3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:54.611 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:54.608+0000 7fdeae259700 1 -- 172.21.15.59:0/2875680238 >> 172.21.15.59:0/2875680238 conn(0x7fdea809d180 msgr2=0x7fdea815c410 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:58:54.611 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:54.608+0000 7fdeae259700 1 -- 172.21.15.59:0/2875680238 shutdown_connections 2023-11-24T07:58:54.612 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:54.608+0000 7fdeae259700 1 -- 172.21.15.59:0/2875680238 wait complete. 2023-11-24T07:58:54.621 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-24T07:58:54.749 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:54.748+0000 7fe309dec700 1 Processor -- start 2023-11-24T07:58:54.750 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:54.748+0000 7fe309dec700 1 -- start start 2023-11-24T07:58:54.759 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:54.749+0000 7fe309dec700 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fe30415e1c0 0x7fe30406eef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:54.760 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:54.749+0000 7fe309dec700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fe3040b0e90 0x7fe30406f540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:54.760 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:54.749+0000 7fe309dec700 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe3040d1230 con 0x7fe30406fa80 2023-11-24T07:58:54.760 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:54.749+0000 7fe309dec700 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe30406ff10 con 0x7fe3040b0e90 2023-11-24T07:58:54.760 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:54.749+0000 7fe309dec700 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe3040d0c70 con 0x7fe30415e1c0 2023-11-24T07:58:54.761 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:54.749+0000 7fe302ffd700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fe3040b0e90 0x7fe30406f540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:54.761 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:54.749+0000 7fe302ffd700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fe3040b0e90 0x7fe30406f540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:54180/0 (socket says 172.21.15.59:54180) 2023-11-24T07:58:54.761 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:54.749+0000 7fe302ffd700 1 -- 172.21.15.59:0/2348268560 learned_addr learned my addr 172.21.15.59:0/2348268560 (peer_addr_for_me v2:172.21.15.59:0/0) 2023-11-24T07:58:54.762 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:54.750+0000 7fe3037fe700 1 --2- 172.21.15.59:0/2348268560 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fe30415e1c0 0x7fe30406eef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:54.762 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:54.750+0000 7fe302ffd700 1 -- 172.21.15.59:0/2348268560 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fe30415e1c0 msgr2=0x7fe30406eef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T07:58:54.762 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:54.750+0000 7fe302ffd700 1 --2- 172.21.15.59:0/2348268560 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fe30415e1c0 0x7fe30406eef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:54.762 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:54.750+0000 7fe302ffd700 1 -- 172.21.15.59:0/2348268560 >> v1:172.21.15.59:6789/0 conn(0x7fe30406fa80 legacy=0x7fe304161d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:54.763 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:54.750+0000 7fe302ffd700 1 -- 172.21.15.59:0/2348268560 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe3040d0f50 con 0x7fe3040b0e90 2023-11-24T07:58:54.763 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:54.750+0000 7fe3027fc700 1 -- 172.21.15.59:0/2348268560 <== mon.0 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2312490161 0 0) 0x7fe3040d1230 con 0x7fe30406fa80 2023-11-24T07:58:54.763 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:54.750+0000 7fe3037fe700 1 --2- 172.21.15.59:0/2348268560 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fe30415e1c0 0x7fe30406eef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T07:58:54.764 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:54.750+0000 7fe302ffd700 1 --2- 172.21.15.59:0/2348268560 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe3040b0e90 0x7fe30406f540 secure :-1 s=READY pgs=760 cs=0 l=1 rev1=1 crypto rx=0x7fe2ec0098f0 tx=0x7fe2ec009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3d4efe6925091b6 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:58:54.764 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:54.750+0000 7fe3027fc700 1 -- 172.21.15.59:0/2348268560 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2ec00b8a0 con 0x7fe3040b0e90 2023-11-24T07:58:54.764 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:54.750+0000 7fe3027fc700 1 -- 172.21.15.59:0/2348268560 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe2ec00ba00 con 0x7fe3040b0e90 2023-11-24T07:58:54.764 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:54.750+0000 7fe3027fc700 1 -- 172.21.15.59:0/2348268560 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2ec00bd90 con 0x7fe3040b0e90 2023-11-24T07:58:54.765 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:54.750+0000 7fe309dec700 1 -- 172.21.15.59:0/2348268560 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe3040b0e90 msgr2=0x7fe30406f540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:54.765 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:54.750+0000 7fe309dec700 1 --2- 172.21.15.59:0/2348268560 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe3040b0e90 0x7fe30406f540 secure :-1 s=READY pgs=760 cs=0 l=1 rev1=1 crypto rx=0x7fe2ec0098f0 tx=0x7fe2ec009c00 comp rx=0 tx=0).stop 2023-11-24T07:58:54.765 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:54.751+0000 7fe309dec700 1 -- 172.21.15.59:0/2348268560 shutdown_connections 2023-11-24T07:58:54.765 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:54.751+0000 7fe309dec700 1 --2- 172.21.15.59:0/2348268560 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fe30415e1c0 0x7fe30406eef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:54.766 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:54.751+0000 7fe309dec700 1 --2- 172.21.15.59:0/2348268560 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe3040b0e90 0x7fe30406f540 unknown :-1 s=CLOSED pgs=760 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:54.766 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:54.751+0000 7fe309dec700 1 -- 172.21.15.59:0/2348268560 >> 172.21.15.59:0/2348268560 conn(0x7fe3040b3720 msgr2=0x7fe3040b3b30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:58:54.766 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:54.751+0000 7fe309dec700 1 -- 172.21.15.59:0/2348268560 shutdown_connections 2023-11-24T07:58:54.767 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:54.751+0000 7fe309dec700 1 -- 172.21.15.59:0/2348268560 wait complete. 2023-11-24T07:58:54.767 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:54.752+0000 7fe309dec700 1 Processor -- start 2023-11-24T07:58:54.767 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:54.752+0000 7fe309dec700 1 -- start start 2023-11-24T07:58:54.768 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:54.753+0000 7fe309dec700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe30406fa80 0x7fe3040b1840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:54.768 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:54.753+0000 7fe309dec700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe3040b0e90 0x7fe3040b1d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:54.768 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:54.753+0000 7fe309dec700 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fe30415e1c0 0x7fe3041f3b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:54.768 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:54.753+0000 7fe309dec700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe304158a10 con 0x7fe30406fa80 2023-11-24T07:58:54.769 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:54.753+0000 7fe309dec700 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe304160390 con 0x7fe3040b0e90 2023-11-24T07:58:54.769 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:54.753+0000 7fe309dec700 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe3041f4950 con 0x7fe30415e1c0 2023-11-24T07:58:54.769 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:54.753+0000 7fe3037fe700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe30406fa80 0x7fe3040b1840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:54.769 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:54.753+0000 7fe3037fe700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe30406fa80 0x7fe3040b1840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:54184/0 (socket says 172.21.15.59:54184) 2023-11-24T07:58:54.770 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:54.753+0000 7fe3037fe700 1 -- 172.21.15.59:0/1743762851 learned_addr learned my addr 172.21.15.59:0/1743762851 (peer_addr_for_me v2:172.21.15.59:0/0) 2023-11-24T07:58:54.770 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:54.753+0000 7fe303fff700 1 --2- 172.21.15.59:0/1743762851 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fe30415e1c0 0x7fe3041f3b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:54.770 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:54.753+0000 7fe302ffd700 1 --2- 172.21.15.59:0/1743762851 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe3040b0e90 0x7fe3040b1d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:54.771 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:54.753+0000 7fe3037fe700 1 -- 172.21.15.59:0/1743762851 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fe30415e1c0 msgr2=0x7fe3041f3b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:54.771 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:54.753+0000 7fe3037fe700 1 --2- 172.21.15.59:0/1743762851 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fe30415e1c0 0x7fe3041f3b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:54.771 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:54.753+0000 7fe3037fe700 1 -- 172.21.15.59:0/1743762851 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe3040b0e90 msgr2=0x7fe3040b1d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:54.771 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:54.753+0000 7fe3037fe700 1 --2- 172.21.15.59:0/1743762851 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe3040b0e90 0x7fe3040b1d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:54.772 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:54.753+0000 7fe3037fe700 1 -- 172.21.15.59:0/1743762851 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe2ec009550 con 0x7fe30406fa80 2023-11-24T07:58:54.772 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:54.753+0000 7fe3037fe700 1 --2- 172.21.15.59:0/1743762851 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe30406fa80 0x7fe3040b1840 secure :-1 s=READY pgs=761 cs=0 l=1 rev1=1 crypto rx=0x7fe2f400c7f0 tx=0x7fe2f400cbb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:58:54.772 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:54.753+0000 7fe2fbfff700 1 -- 172.21.15.59:0/1743762851 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2f4011800 con 0x7fe30406fa80 2023-11-24T07:58:54.773 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:54.753+0000 7fe2fbfff700 1 -- 172.21.15.59:0/1743762851 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe2f4011960 con 0x7fe30406fa80 2023-11-24T07:58:54.773 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:54.753+0000 7fe2fbfff700 1 -- 172.21.15.59:0/1743762851 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2f4011d10 con 0x7fe30406fa80 2023-11-24T07:58:54.773 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:54.753+0000 7fe309dec700 1 -- 172.21.15.59:0/1743762851 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe30415fa60 con 0x7fe30406fa80 2023-11-24T07:58:54.773 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:54.754+0000 7fe309dec700 1 -- 172.21.15.59:0/1743762851 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe304160020 con 0x7fe30406fa80 2023-11-24T07:58:54.774 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:54.754+0000 7fe309dec700 1 -- 172.21.15.59:0/1743762851 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe304075480 con 0x7fe30406fa80 2023-11-24T07:58:54.774 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:54.755+0000 7fe2fbfff700 1 -- 172.21.15.59:0/1743762851 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 9) v1 ==== 89486+0+0 (secure 0 0 0) 0x7fe2f4018360 con 0x7fe30406fa80 2023-11-24T07:58:54.774 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:54.755+0000 7fe2fbfff700 1 --2- 172.21.15.59:0/1743762851 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7fe2e406b1a0 0x7fe2e406d660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:54.775 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:54.755+0000 7fe2fbfff700 1 -- 172.21.15.59:0/1743762851 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(60..60 src has 1..60) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fe2f4089b50 con 0x7fe30406fa80 2023-11-24T07:58:54.775 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:54.756+0000 7fe302ffd700 1 --2- 172.21.15.59:0/1743762851 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7fe2e406b1a0 0x7fe2e406d660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:54.775 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:54.756+0000 7fe302ffd700 1 --2- 172.21.15.59:0/1743762851 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7fe2e406b1a0 0x7fe2e406d660 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7fe2ec000c00 tx=0x7fe2ec008030 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:58:54.776 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:54.760+0000 7fe2fbfff700 1 -- 172.21.15.59:0/1743762851 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fe2f40547a0 con 0x7fe30406fa80 2023-11-24T07:58:54.959 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:54.958+0000 7fe309dec700 1 -- 172.21.15.59:0/1743762851 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fe3040cf730 con 0x7fe30406fa80 2023-11-24T07:58:54.959 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:54.959+0000 7fe2fbfff700 1 -- 172.21.15.59:0/1743762851 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v60) v1 ==== 100+0+11 (secure 0 0 0) 0x7fe2f401f0a0 con 0x7fe30406fa80 2023-11-24T07:58:54.960 INFO:teuthology.orchestra.run.smithi059.stdout:pg_num: 64 2023-11-24T07:58:54.961 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:54.960+0000 7fe309dec700 1 -- 172.21.15.59:0/1743762851 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7fe2e406b1a0 msgr2=0x7fe2e406d660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:54.962 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:54.960+0000 7fe309dec700 1 --2- 172.21.15.59:0/1743762851 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7fe2e406b1a0 0x7fe2e406d660 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7fe2ec000c00 tx=0x7fe2ec008030 comp rx=0 tx=0).stop 2023-11-24T07:58:54.962 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:54.960+0000 7fe309dec700 1 -- 172.21.15.59:0/1743762851 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe30406fa80 msgr2=0x7fe3040b1840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:54.962 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:54.960+0000 7fe309dec700 1 --2- 172.21.15.59:0/1743762851 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe30406fa80 0x7fe3040b1840 secure :-1 s=READY pgs=761 cs=0 l=1 rev1=1 crypto rx=0x7fe2f400c7f0 tx=0x7fe2f400cbb0 comp rx=0 tx=0).stop 2023-11-24T07:58:54.962 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:54.961+0000 7fe309dec700 1 -- 172.21.15.59:0/1743762851 shutdown_connections 2023-11-24T07:58:54.962 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:54.961+0000 7fe309dec700 1 --2- 172.21.15.59:0/1743762851 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7fe2e406b1a0 0x7fe2e406d660 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:54.962 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:54.961+0000 7fe309dec700 1 --2- 172.21.15.59:0/1743762851 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fe30415e1c0 0x7fe3041f3b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:54.963 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:54.961+0000 7fe309dec700 1 --2- 172.21.15.59:0/1743762851 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe30406fa80 0x7fe3040b1840 unknown :-1 s=CLOSED pgs=761 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:54.963 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:54.961+0000 7fe309dec700 1 --2- 172.21.15.59:0/1743762851 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe3040b0e90 0x7fe3040b1d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:54.963 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:54.961+0000 7fe309dec700 1 -- 172.21.15.59:0/1743762851 >> 172.21.15.59:0/1743762851 conn(0x7fe3040b3720 msgr2=0x7fe3040b2ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:58:54.963 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:54.961+0000 7fe309dec700 1 -- 172.21.15.59:0/1743762851 shutdown_connections 2023-11-24T07:58:54.963 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:54.961+0000 7fe309dec700 1 -- 172.21.15.59:0/1743762851 wait complete. 2023-11-24T07:58:54.974 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-11-24T07:58:55.098 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:55.098+0000 7f0afc4c2700 1 Processor -- start 2023-11-24T07:58:55.099 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:55.098+0000 7f0afc4c2700 1 -- start start 2023-11-24T07:58:55.101 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:55.099+0000 7f0afc4c2700 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f0af406afd0 0x7f0af40b5a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:55.101 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:55.099+0000 7f0afc4c2700 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f0af4071b40 0x7f0af4071ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:55.101 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:55.099+0000 7f0afc4c2700 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0af40ad860 con 0x7f0af406edc0 2023-11-24T07:58:55.101 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:55.099+0000 7f0afc4c2700 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0af406f830 con 0x7f0af4071b40 2023-11-24T07:58:55.102 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:55.099+0000 7f0afc4c2700 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0af40d0a20 con 0x7f0af406afd0 2023-11-24T07:58:55.102 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:55.099+0000 7f0af9a5d700 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7f0af406edc0 0x7f0af406b3b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:57666/0 (socket says 172.21.15.59:57666) 2023-11-24T07:58:55.102 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:55.099+0000 7f0af9a5d700 1 -- 172.21.15.59:0/883832868 learned_addr learned my addr 172.21.15.59:0/883832868 (peer_addr_for_me v1:172.21.15.59:0/0) 2023-11-24T07:58:55.102 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:55.099+0000 7f0afa25e700 1 --2- 172.21.15.59:0/883832868 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f0af406afd0 0x7f0af40b5a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:55.102 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:55.099+0000 7f0afaa5f700 1 --2- 172.21.15.59:0/883832868 >> v2:172.21.15.76:3300/0 conn(0x7f0af4071b40 0x7f0af4071ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:55.102 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:55.099+0000 7f0af925c700 1 -- 172.21.15.59:0/883832868 <== mon.0 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1057157308 0 0) 0x7f0af40ad860 con 0x7f0af406edc0 2023-11-24T07:58:55.102 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:55.099+0000 7f0af925c700 1 -- 172.21.15.59:0/883832868 --> v1:172.21.15.59:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0ae00035b0 con 0x7f0af406edc0 2023-11-24T07:58:55.102 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:55.099+0000 7f0afa25e700 1 -- 172.21.15.59:0/883832868 >> v2:172.21.15.76:3300/0 conn(0x7f0af4071b40 msgr2=0x7f0af4071ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T07:58:55.102 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:55.099+0000 7f0afa25e700 1 --2- 172.21.15.59:0/883832868 >> v2:172.21.15.76:3300/0 conn(0x7f0af4071b40 0x7f0af4071ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:55.103 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:55.100+0000 7f0afa25e700 1 -- 172.21.15.59:0/883832868 >> v1:172.21.15.59:6789/0 conn(0x7f0af406edc0 legacy=0x7f0af406b3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:55.103 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:55.100+0000 7f0afa25e700 1 -- 172.21.15.59:0/883832868 --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0af4162a70 con 0x7f0af406afd0 2023-11-24T07:58:55.103 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:55.100+0000 7f0afaa5f700 1 --2- 172.21.15.59:0/883832868 >> v2:172.21.15.76:3300/0 conn(0x7f0af4071b40 0x7f0af4071ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-24T07:58:55.103 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:55.100+0000 7f0afa25e700 1 --2- 172.21.15.59:0/883832868 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f0af406afd0 0x7f0af40b5a70 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f0ae4009840 tx=0x7f0ae4009b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6cc382257c1e8726 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:58:55.103 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:55.100+0000 7f0af925c700 1 -- 172.21.15.59:0/883832868 <== mon.2 v2:172.21.15.76:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ae4011830 con 0x7f0af406afd0 2023-11-24T07:58:55.103 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:55.100+0000 7f0af925c700 1 -- 172.21.15.59:0/883832868 <== mon.2 v2:172.21.15.76:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0ae4011990 con 0x7f0af406afd0 2023-11-24T07:58:55.103 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:55.100+0000 7f0af925c700 1 -- 172.21.15.59:0/883832868 <== mon.2 v2:172.21.15.76:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ae4011d00 con 0x7f0af406afd0 2023-11-24T07:58:55.103 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:55.100+0000 7f0afc4c2700 1 -- 172.21.15.59:0/883832868 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f0af406afd0 msgr2=0x7f0af40b5a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:55.103 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:55.100+0000 7f0afc4c2700 1 --2- 172.21.15.59:0/883832868 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f0af406afd0 0x7f0af40b5a70 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f0ae4009840 tx=0x7f0ae4009b50 comp rx=0 tx=0).stop 2023-11-24T07:58:55.104 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:55.101+0000 7f0afc4c2700 1 -- 172.21.15.59:0/883832868 shutdown_connections 2023-11-24T07:58:55.104 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:55.101+0000 7f0afc4c2700 1 --2- 172.21.15.59:0/883832868 >> v2:172.21.15.76:3300/0 conn(0x7f0af4071b40 0x7f0af4071ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:55.104 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:55.101+0000 7f0afc4c2700 1 --2- 172.21.15.59:0/883832868 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f0af406afd0 0x7f0af40b5a70 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:55.104 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:55.101+0000 7f0afc4c2700 1 -- 172.21.15.59:0/883832868 >> 172.21.15.59:0/883832868 conn(0x7f0af40b2840 msgr2=0x7f0af40b2c20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:58:55.104 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:55.101+0000 7f0afc4c2700 1 -- 172.21.15.59:0/883832868 shutdown_connections 2023-11-24T07:58:55.104 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:55.101+0000 7f0afc4c2700 1 -- 172.21.15.59:0/883832868 wait complete. 2023-11-24T07:58:55.106 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:55.102+0000 7f0afc4c2700 1 Processor -- start 2023-11-24T07:58:55.107 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:55.102+0000 7f0afc4c2700 1 -- start start 2023-11-24T07:58:55.107 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:55.103+0000 7f0afc4c2700 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f0af406afd0 0x7f0af4154160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:55.107 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:55.103+0000 7f0afc4c2700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0af406edc0 0x7f0af4156d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:55.107 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:55.103+0000 7f0afc4c2700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0af4071b40 0x7f0af415f8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:55.107 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:55.103+0000 7f0afc4c2700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0af415a620 con 0x7f0af4071b40 2023-11-24T07:58:55.107 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:55.103+0000 7f0afc4c2700 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0af415bf20 con 0x7f0af406edc0 2023-11-24T07:58:55.107 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:55.103+0000 7f0afc4c2700 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0af415c200 con 0x7f0af406afd0 2023-11-24T07:58:55.107 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:55.103+0000 7f0afaa5f700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0af4071b40 0x7f0af415f8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:55.107 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:55.103+0000 7f0afaa5f700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0af4071b40 0x7f0af415f8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:54202/0 (socket says 172.21.15.59:54202) 2023-11-24T07:58:55.108 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:55.103+0000 7f0afaa5f700 1 -- 172.21.15.59:0/2606344352 learned_addr learned my addr 172.21.15.59:0/2606344352 (peer_addr_for_me v2:172.21.15.59:0/0) 2023-11-24T07:58:55.108 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:55.103+0000 7f0afa25e700 1 --2- 172.21.15.59:0/2606344352 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f0af406afd0 0x7f0af4154160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:55.108 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:55.104+0000 7f0af9a5d700 1 --2- 172.21.15.59:0/2606344352 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0af406edc0 0x7f0af4156d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:55.108 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:55.104+0000 7f0afaa5f700 1 -- 172.21.15.59:0/2606344352 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f0af406afd0 msgr2=0x7f0af4154160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:55.108 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:55.104+0000 7f0afaa5f700 1 --2- 172.21.15.59:0/2606344352 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f0af406afd0 0x7f0af4154160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:55.108 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:55.104+0000 7f0afaa5f700 1 -- 172.21.15.59:0/2606344352 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0af406edc0 msgr2=0x7f0af4156d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:55.108 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:55.104+0000 7f0afaa5f700 1 --2- 172.21.15.59:0/2606344352 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0af406edc0 0x7f0af4156d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:55.108 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:55.104+0000 7f0afaa5f700 1 -- 172.21.15.59:0/2606344352 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0aec009a70 con 0x7f0af4071b40 2023-11-24T07:58:55.108 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:55.104+0000 7f0afaa5f700 1 --2- 172.21.15.59:0/2606344352 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0af4071b40 0x7f0af415f8c0 secure :-1 s=READY pgs=763 cs=0 l=1 rev1=1 crypto rx=0x7f0aec00efc0 tx=0x7f0aec00c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:58:55.109 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:55.104+0000 7f0aeaffd700 1 -- 172.21.15.59:0/2606344352 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0aec009cd0 con 0x7f0af4071b40 2023-11-24T07:58:55.109 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:55.104+0000 7f0aeaffd700 1 -- 172.21.15.59:0/2606344352 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0aec00f040 con 0x7f0af4071b40 2023-11-24T07:58:55.109 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:55.104+0000 7f0aeaffd700 1 -- 172.21.15.59:0/2606344352 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0aec00cd60 con 0x7f0af4071b40 2023-11-24T07:58:55.109 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:55.104+0000 7f0afc4c2700 1 -- 172.21.15.59:0/2606344352 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0ae40094a0 con 0x7f0af4071b40 2023-11-24T07:58:55.109 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:55.104+0000 7f0afc4c2700 1 -- 172.21.15.59:0/2606344352 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0af415c7c0 con 0x7f0af4071b40 2023-11-24T07:58:55.109 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:55.105+0000 7f0afc4c2700 1 -- 172.21.15.59:0/2606344352 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0af4068670 con 0x7f0af4071b40 2023-11-24T07:58:55.109 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:55.106+0000 7f0aeaffd700 1 -- 172.21.15.59:0/2606344352 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 9) v1 ==== 89486+0+0 (secure 0 0 0) 0x7f0aec014480 con 0x7f0af4071b40 2023-11-24T07:58:55.109 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:55.106+0000 7f0aeaffd700 1 --2- 172.21.15.59:0/2606344352 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f0ad806b1a0 0x7f0ad806d660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:55.109 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:55.106+0000 7f0aeaffd700 1 -- 172.21.15.59:0/2606344352 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(60..60 src has 1..60) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f0aec08a1f0 con 0x7f0af4071b40 2023-11-24T07:58:55.110 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:55.106+0000 7f0afa25e700 1 --2- 172.21.15.59:0/2606344352 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f0ad806b1a0 0x7f0ad806d660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:55.110 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:55.107+0000 7f0afa25e700 1 --2- 172.21.15.59:0/2606344352 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f0ad806b1a0 0x7f0ad806d660 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f0ae4000c00 tx=0x7f0ae4008030 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:58:55.111 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:55.111+0000 7f0aeaffd700 1 -- 172.21.15.59:0/2606344352 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f0aec054e40 con 0x7f0af4071b40 2023-11-24T07:58:55.326 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:55.325+0000 7f0afc4c2700 1 -- 172.21.15.59:0/2606344352 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f0af4072d00 con 0x7f0af4071b40 2023-11-24T07:58:55.326 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:55.326+0000 7f0aeaffd700 1 -- 172.21.15.59:0/2606344352 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v60) v1 ==== 80+0+0 (secure 0 0 0) 0x7f0aec058460 con 0x7f0af4071b40 2023-11-24T07:58:55.328 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:55.327+0000 7f0afc4c2700 1 -- 172.21.15.59:0/2606344352 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f0ad806b1a0 msgr2=0x7f0ad806d660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:55.329 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:55.327+0000 7f0afc4c2700 1 --2- 172.21.15.59:0/2606344352 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f0ad806b1a0 0x7f0ad806d660 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f0ae4000c00 tx=0x7f0ae4008030 comp rx=0 tx=0).stop 2023-11-24T07:58:55.329 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:55.327+0000 7f0afc4c2700 1 -- 172.21.15.59:0/2606344352 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0af4071b40 msgr2=0x7f0af415f8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:55.329 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:55.327+0000 7f0afc4c2700 1 --2- 172.21.15.59:0/2606344352 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0af4071b40 0x7f0af415f8c0 secure :-1 s=READY pgs=763 cs=0 l=1 rev1=1 crypto rx=0x7f0aec00efc0 tx=0x7f0aec00c3a0 comp rx=0 tx=0).stop 2023-11-24T07:58:55.329 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:55.328+0000 7f0afc4c2700 1 -- 172.21.15.59:0/2606344352 shutdown_connections 2023-11-24T07:58:55.329 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:55.328+0000 7f0afc4c2700 1 --2- 172.21.15.59:0/2606344352 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f0ad806b1a0 0x7f0ad806d660 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:55.329 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:55.328+0000 7f0afc4c2700 1 --2- 172.21.15.59:0/2606344352 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0af4071b40 0x7f0af415f8c0 unknown :-1 s=CLOSED pgs=763 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:55.330 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:55.328+0000 7f0afc4c2700 1 --2- 172.21.15.59:0/2606344352 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f0af406afd0 0x7f0af4154160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:55.330 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:55.328+0000 7f0afc4c2700 1 --2- 172.21.15.59:0/2606344352 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0af406edc0 0x7f0af4156d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:55.330 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:55.328+0000 7f0afc4c2700 1 -- 172.21.15.59:0/2606344352 >> 172.21.15.59:0/2606344352 conn(0x7f0af40b2840 msgr2=0x7f0af40b07b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:58:55.330 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:55.328+0000 7f0afc4c2700 1 -- 172.21.15.59:0/2606344352 shutdown_connections 2023-11-24T07:58:55.330 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:55.328+0000 7f0afc4c2700 1 -- 172.21.15.59:0/2606344352 wait complete. 2023-11-24T07:58:55.331 INFO:teuthology.orchestra.run.smithi059.stderr:listed 0 entries 2023-11-24T07:58:55.340 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2023-11-24T07:58:55.341 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T07:58:55.341 DEBUG:teuthology.orchestra.run.smithi059:> sudo umount /home/ubuntu/cephtest/mnt.0 2023-11-24T07:58:55.382 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi059.front.sepia.ceph.com 2023-11-24T07:58:55.383 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T07:58:55.383 DEBUG:teuthology.orchestra.run.smithi059:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-11-24T07:58:55.440 DEBUG:teuthology.orchestra.run.smithi076:> set -ex 2023-11-24T07:58:55.440 DEBUG:teuthology.orchestra.run.smithi076:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-24T07:58:55.461 DEBUG:teuthology.orchestra.run.smithi076:> set -ex 2023-11-24T07:58:55.462 DEBUG:teuthology.orchestra.run.smithi076:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-24T07:58:55.522 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-24T07:58:55.648 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:55.648+0000 7f269566e700 1 Processor -- start 2023-11-24T07:58:55.649 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:55.648+0000 7f269566e700 1 -- start start 2023-11-24T07:58:55.651 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:55.649+0000 7f269566e700 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f26900caae0 0x7f26900ceb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:55.651 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:55.649+0000 7f269566e700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f26900d0b80 0x7f26900b40d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:55.651 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:55.649+0000 7f269566e700 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f26900b3c40 con 0x7f2690161900 2023-11-24T07:58:55.651 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:55.649+0000 7f269566e700 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f26900b36e0 con 0x7f26900d0b80 2023-11-24T07:58:55.651 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:55.649+0000 7f269566e700 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f26900cfbc0 con 0x7f26900caae0 2023-11-24T07:58:55.651 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:55.649+0000 7f268f7fe700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f26900d0b80 0x7f26900b40d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:55.651 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:55.649+0000 7f268f7fe700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f26900d0b80 0x7f26900b40d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:54210/0 (socket says 172.21.15.59:54210) 2023-11-24T07:58:55.652 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:55.649+0000 7f268f7fe700 1 -- 172.21.15.59:0/3861806625 learned_addr learned my addr 172.21.15.59:0/3861806625 (peer_addr_for_me v2:172.21.15.59:0/0) 2023-11-24T07:58:55.652 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:55.649+0000 7f268e7fc700 1 --2- 172.21.15.59:0/3861806625 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f26900caae0 0x7f26900ceb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:55.652 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:55.649+0000 7f268f7fe700 1 -- 172.21.15.59:0/3861806625 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f26900caae0 msgr2=0x7f26900ceb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T07:58:55.652 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:55.649+0000 7f268f7fe700 1 --2- 172.21.15.59:0/3861806625 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f26900caae0 0x7f26900ceb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:55.652 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:55.649+0000 7f268f7fe700 1 -- 172.21.15.59:0/3861806625 >> v1:172.21.15.76:6789/0 conn(0x7f2690161900 legacy=0x7f26900d0470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T07:58:55.652 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:55.649+0000 7f268f7fe700 1 -- 172.21.15.59:0/3861806625 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f26900cfe60 con 0x7f26900d0b80 2023-11-24T07:58:55.652 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:55.649+0000 7f268f7fe700 1 --2- 172.21.15.59:0/3861806625 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f26900d0b80 0x7f26900b40d0 secure :-1 s=READY pgs=764 cs=0 l=1 rev1=1 crypto rx=0x7f2688009e50 tx=0x7f268800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=57298c9074f1f346 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:58:55.652 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:55.649+0000 7f268dffb700 1 -- 172.21.15.59:0/3861806625 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f268800e070 con 0x7f26900d0b80 2023-11-24T07:58:55.652 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:55.649+0000 7f268dffb700 1 -- 172.21.15.59:0/3861806625 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2688003a20 con 0x7f26900d0b80 2023-11-24T07:58:55.652 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:55.649+0000 7f268dffb700 1 -- 172.21.15.59:0/3861806625 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2688003dd0 con 0x7f26900d0b80 2023-11-24T07:58:55.652 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:55.650+0000 7f269566e700 1 -- 172.21.15.59:0/3861806625 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f26900d0b80 msgr2=0x7f26900b40d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:55.652 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:55.650+0000 7f269566e700 1 --2- 172.21.15.59:0/3861806625 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f26900d0b80 0x7f26900b40d0 secure :-1 s=READY pgs=764 cs=0 l=1 rev1=1 crypto rx=0x7f2688009e50 tx=0x7f268800b330 comp rx=0 tx=0).stop 2023-11-24T07:58:55.653 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:55.650+0000 7f269566e700 1 -- 172.21.15.59:0/3861806625 shutdown_connections 2023-11-24T07:58:55.653 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:55.650+0000 7f269566e700 1 --2- 172.21.15.59:0/3861806625 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f26900d0b80 0x7f26900b40d0 unknown :-1 s=CLOSED pgs=764 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:55.653 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:55.650+0000 7f269566e700 1 --2- 172.21.15.59:0/3861806625 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f26900caae0 0x7f26900ceb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:55.653 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:55.650+0000 7f269566e700 1 -- 172.21.15.59:0/3861806625 >> 172.21.15.59:0/3861806625 conn(0x7f26900d3e40 msgr2=0x7f26900d4250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:58:55.653 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:55.650+0000 7f269566e700 1 -- 172.21.15.59:0/3861806625 shutdown_connections 2023-11-24T07:58:55.653 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:55.650+0000 7f269566e700 1 -- 172.21.15.59:0/3861806625 wait complete. 2023-11-24T07:58:55.656 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:55.652+0000 7f269566e700 1 Processor -- start 2023-11-24T07:58:55.656 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:55.652+0000 7f269566e700 1 -- start start 2023-11-24T07:58:55.656 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:55.653+0000 7f269566e700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f26900caae0 0x7f26900b7b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:55.656 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:55.653+0000 7f269566e700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f26900d0b80 0x7f269015d500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:55.657 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:55.653+0000 7f269566e700 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f2690161900 0x7f26900c89c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:55.657 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:55.653+0000 7f269566e700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26900b8080 con 0x7f26900d0b80 2023-11-24T07:58:55.657 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:55.653+0000 7f269566e700 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f269015da40 con 0x7f26900caae0 2023-11-24T07:58:55.657 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:55.653+0000 7f269566e700 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f26900c9330 con 0x7f2690161900 2023-11-24T07:58:55.657 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:55.653+0000 7f268e7fc700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f26900d0b80 0x7f269015d500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:55.657 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:55.653+0000 7f268e7fc700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f26900d0b80 0x7f269015d500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:54214/0 (socket says 172.21.15.59:54214) 2023-11-24T07:58:55.657 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:55.653+0000 7f268e7fc700 1 -- 172.21.15.59:0/4018420577 learned_addr learned my addr 172.21.15.59:0/4018420577 (peer_addr_for_me v2:172.21.15.59:0/0) 2023-11-24T07:58:55.657 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:55.653+0000 7f268effd700 1 --2- 172.21.15.59:0/4018420577 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f26900caae0 0x7f26900b7b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:55.657 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:55.653+0000 7f268f7fe700 1 --2- 172.21.15.59:0/4018420577 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f2690161900 0x7f26900c89c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:55.657 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:55.653+0000 7f268e7fc700 1 -- 172.21.15.59:0/4018420577 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f2690161900 msgr2=0x7f26900c89c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:55.658 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:55.653+0000 7f268e7fc700 1 --2- 172.21.15.59:0/4018420577 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f2690161900 0x7f26900c89c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:55.658 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:55.653+0000 7f268e7fc700 1 -- 172.21.15.59:0/4018420577 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f26900caae0 msgr2=0x7f26900b7b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:55.658 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:55.653+0000 7f268e7fc700 1 --2- 172.21.15.59:0/4018420577 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f26900caae0 0x7f26900b7b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:55.658 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:55.653+0000 7f268e7fc700 1 -- 172.21.15.59:0/4018420577 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2688009b30 con 0x7f26900d0b80 2023-11-24T07:58:55.658 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:55.653+0000 7f268e7fc700 1 --2- 172.21.15.59:0/4018420577 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f26900d0b80 0x7f269015d500 secure :-1 s=READY pgs=765 cs=0 l=1 rev1=1 crypto rx=0x7f268400c7f0 tx=0x7f268400cb00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:58:55.658 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:55.653+0000 7f26777fe700 1 -- 172.21.15.59:0/4018420577 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2684004220 con 0x7f26900d0b80 2023-11-24T07:58:55.658 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:55.653+0000 7f26777fe700 1 -- 172.21.15.59:0/4018420577 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2684004380 con 0x7f26900d0b80 2023-11-24T07:58:55.658 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:55.653+0000 7f269566e700 1 -- 172.21.15.59:0/4018420577 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f26900c9610 con 0x7f26900d0b80 2023-11-24T07:58:55.658 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:55.653+0000 7f26777fe700 1 -- 172.21.15.59:0/4018420577 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26840036a0 con 0x7f26900d0b80 2023-11-24T07:58:55.658 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:55.654+0000 7f269566e700 1 -- 172.21.15.59:0/4018420577 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f26900caf10 con 0x7f26900d0b80 2023-11-24T07:58:55.658 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:55.655+0000 7f269566e700 1 -- 172.21.15.59:0/4018420577 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f26900cb560 con 0x7f26900d0b80 2023-11-24T07:58:55.659 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:55.655+0000 7f26777fe700 1 -- 172.21.15.59:0/4018420577 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 10) v1 ==== 89588+0+0 (secure 0 0 0) 0x7f2684003800 con 0x7f26900d0b80 2023-11-24T07:58:55.659 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:55.655+0000 7f26777fe700 1 --2- 172.21.15.59:0/4018420577 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f267c06b2c0 0x7f267c06d780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:55.659 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:55.655+0000 7f26777fe700 1 -- 172.21.15.59:0/4018420577 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(60..60 src has 1..60) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f2684089950 con 0x7f26900d0b80 2023-11-24T07:58:55.659 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:55.656+0000 7f268effd700 1 --2- 172.21.15.59:0/4018420577 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f267c06b2c0 0x7f267c06d780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:55.659 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:55.656+0000 7f268effd700 1 --2- 172.21.15.59:0/4018420577 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f267c06b2c0 0x7f267c06d780 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f2680009960 tx=0x7f26800090e0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:58:55.662 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:55.660+0000 7f26777fe700 1 -- 172.21.15.59:0/4018420577 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f26840542d0 con 0x7f26900d0b80 2023-11-24T07:58:55.865 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:55.864+0000 7f269566e700 1 -- 172.21.15.59:0/4018420577 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f2690067a20 con 0x7f26900d0b80 2023-11-24T07:58:55.866 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:55.866+0000 7f26777fe700 1 -- 172.21.15.59:0/4018420577 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v60) v1 ==== 74+0+16493 (secure 0 0 0) 0x7f268401e090 con 0x7f26900d0b80 2023-11-24T07:58:55.867 INFO:teuthology.orchestra.run.smithi059.stdout: 2023-11-24T07:58:55.867 INFO:teuthology.orchestra.run.smithi059.stdout:{"epoch":60,"fsid":"9f0e8741-b8f9-48d5-a2c3-09f45903303d","created":"2023-11-24T07:56:06.552729+0000","modified":"2023-11-24T07:58:42.328743+0000","last_up_change":"2023-11-24T07:56:17.372352+0000","last_in_change":"2023-11-24T07:56:09.073250+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-24T07:56:18.352823+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_metadata","create_time":"2023-11-24T07:58:39.262463+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"59","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_data","create_time":"2023-11-24T07:58:39.763399+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"60","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"6d8f9e9b-1782-4e41-928b-a39f2a8ebc06","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6800","nonce":3373878772},{"type":"v1","addr":"172.21.15.59:6801","nonce":3373878772}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6802","nonce":3373878772},{"type":"v1","addr":"172.21.15.59:6803","nonce":3373878772}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6806","nonce":3373878772},{"type":"v1","addr":"172.21.15.59:6807","nonce":3373878772}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6804","nonce":3373878772},{"type":"v1","addr":"172.21.15.59:6805","nonce":3373878772}]},"public_addr":"172.21.15.59:6801/3373878772","cluster_addr":"172.21.15.59:6803/3373878772","heartbeat_back_addr":"172.21.15.59:6807/3373878772","heartbeat_front_addr":"172.21.15.59:6805/3373878772","state":["exists","up"]},{"osd":1,"uuid":"e8d882aa-52e1-4c04-9e1e-c44ef9204a5f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6824","nonce":648258563},{"type":"v1","addr":"172.21.15.59:6825","nonce":648258563}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6826","nonce":648258563},{"type":"v1","addr":"172.21.15.59:6827","nonce":648258563}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6830","nonce":648258563},{"type":"v1","addr":"172.21.15.59:6831","nonce":648258563}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6828","nonce":648258563},{"type":"v1","addr":"172.21.15.59:6829","nonce":648258563}]},"public_addr":"172.21.15.59:6825/648258563","cluster_addr":"172.21.15.59:6827/648258563","heartbeat_back_addr":"172.21.15.59:6831/648258563","heartbeat_front_addr":"172.21.15.59:6829/648258563","state":["exists","up"]},{"osd":2,"uuid":"69eebfa0-695b-4b21-90b9-8a3537168a01","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6808","nonce":1840430654},{"type":"v1","addr":"172.21.15.59:6809","nonce":1840430654}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6810","nonce":1840430654},{"type":"v1","addr":"172.21.15.59:6811","nonce":1840430654}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6814","nonce":1840430654},{"type":"v1","addr":"172.21.15.59:6815","nonce":1840430654}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6812","nonce":1840430654},{"type":"v1","addr":"172.21.15.59:6813","nonce":1840430654}]},"public_addr":"172.21.15.59:6809/1840430654","cluster_addr":"172.21.15.59:6811/1840430654","heartbeat_back_addr":"172.21.15.59:6815/1840430654","heartbeat_front_addr":"172.21.15.59:6813/1840430654","state":["exists","up"]},{"osd":3,"uuid":"96fd77fc-f911-46ff-9445-6f0be2daa8f4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6816","nonce":2504025850},{"type":"v1","addr":"172.21.15.59:6817","nonce":2504025850}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6818","nonce":2504025850},{"type":"v1","addr":"172.21.15.59:6819","nonce":2504025850}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6822","nonce":2504025850},{"type":"v1","addr":"172.21.15.59:6823","nonce":2504025850}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6820","nonce":2504025850},{"type":"v1","addr":"172.21.15.59:6821","nonce":2504025850}]},"public_addr":"172.21.15.59:6817/2504025850","cluster_addr":"172.21.15.59:6819/2504025850","heartbeat_back_addr":"172.21.15.59:6823/2504025850","heartbeat_front_addr":"172.21.15.59:6821/2504025850","state":["exists","up"]},{"osd":4,"uuid":"7ac29fca-b5b2-4c95-9d69-2f164cbbc708","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6816","nonce":2382463431},{"type":"v1","addr":"172.21.15.76:6818","nonce":2382463431}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6820","nonce":2382463431},{"type":"v1","addr":"172.21.15.76:6822","nonce":2382463431}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6828","nonce":2382463431},{"type":"v1","addr":"172.21.15.76:6830","nonce":2382463431}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6824","nonce":2382463431},{"type":"v1","addr":"172.21.15.76:6826","nonce":2382463431}]},"public_addr":"172.21.15.76:6818/2382463431","cluster_addr":"172.21.15.76:6822/2382463431","heartbeat_back_addr":"172.21.15.76:6830/2382463431","heartbeat_front_addr":"172.21.15.76:6826/2382463431","state":["exists","up"]},{"osd":5,"uuid":"485bf711-3851-4238-a4f4-51d52e0a9c58","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6800","nonce":329334308},{"type":"v1","addr":"172.21.15.76:6802","nonce":329334308}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6805","nonce":329334308},{"type":"v1","addr":"172.21.15.76:6807","nonce":329334308}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6814","nonce":329334308},{"type":"v1","addr":"172.21.15.76:6815","nonce":329334308}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6810","nonce":329334308},{"type":"v1","addr":"172.21.15.76:6812","nonce":329334308}]},"public_addr":"172.21.15.76:6802/329334308","cluster_addr":"172.21.15.76:6807/329334308","heartbeat_back_addr":"172.21.15.76:6815/329334308","heartbeat_front_addr":"172.21.15.76:6812/329334308","state":["exists","up"]},{"osd":6,"uuid":"72b3f2fc-1e4a-4e32-968d-003647a0286f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6801","nonce":3946579410},{"type":"v1","addr":"172.21.15.76:6803","nonce":3946579410}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6804","nonce":3946579410},{"type":"v1","addr":"172.21.15.76:6806","nonce":3946579410}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6811","nonce":3946579410},{"type":"v1","addr":"172.21.15.76:6813","nonce":3946579410}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6808","nonce":3946579410},{"type":"v1","addr":"172.21.15.76:6809","nonce":3946579410}]},"public_addr":"172.21.15.76:6803/3946579410","cluster_addr":"172.21.15.76:6806/3946579410","heartbeat_back_addr":"172.21.15.76:6813/3946579410","heartbeat_front_addr":"172.21.15.76:6809/3946579410","state":["exists","up"]},{"osd":7,"uuid":"1ddceb90-6c5d-4d1a-ad56-99cb877f8c70","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6817","nonce":841761538},{"type":"v1","addr":"172.21.15.76:6819","nonce":841761538}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6821","nonce":841761538},{"type":"v1","addr":"172.21.15.76:6823","nonce":841761538}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6829","nonce":841761538},{"type":"v1","addr":"172.21.15.76:6831","nonce":841761538}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6825","nonce":841761538},{"type":"v1","addr":"172.21.15.76:6827","nonce":841761538}]},"public_addr":"172.21.15.76:6819/841761538","cluster_addr":"172.21.15.76:6823/841761538","heartbeat_back_addr":"172.21.15.76:6831/841761538","heartbeat_front_addr":"172.21.15.76:6827/841761538","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T07:56:14.294838+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T07:56:14.270779+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T07:56:14.231809+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T07:56:14.260983+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T07:56:14.302997+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T07:56:14.258600+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T07:56:14.251413+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T07:56:14.304727+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-24T07:58:55.868 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:55.867+0000 7f269566e700 1 -- 172.21.15.59:0/4018420577 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f267c06b2c0 msgr2=0x7f267c06d780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:55.869 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:55.868+0000 7f269566e700 1 --2- 172.21.15.59:0/4018420577 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f267c06b2c0 0x7f267c06d780 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f2680009960 tx=0x7f26800090e0 comp rx=0 tx=0).stop 2023-11-24T07:58:55.869 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:55.868+0000 7f269566e700 1 -- 172.21.15.59:0/4018420577 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f26900d0b80 msgr2=0x7f269015d500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:55.869 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:55.868+0000 7f269566e700 1 --2- 172.21.15.59:0/4018420577 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f26900d0b80 0x7f269015d500 secure :-1 s=READY pgs=765 cs=0 l=1 rev1=1 crypto rx=0x7f268400c7f0 tx=0x7f268400cb00 comp rx=0 tx=0).stop 2023-11-24T07:58:55.869 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:55.868+0000 7f269566e700 1 -- 172.21.15.59:0/4018420577 shutdown_connections 2023-11-24T07:58:55.869 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:55.868+0000 7f269566e700 1 --2- 172.21.15.59:0/4018420577 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f267c06b2c0 0x7f267c06d780 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:55.869 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:55.868+0000 7f269566e700 1 --2- 172.21.15.59:0/4018420577 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f2690161900 0x7f26900c89c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:55.869 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:55.868+0000 7f269566e700 1 --2- 172.21.15.59:0/4018420577 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f26900caae0 0x7f26900b7b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:55.869 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:55.868+0000 7f269566e700 1 --2- 172.21.15.59:0/4018420577 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f26900d0b80 0x7f269015d500 unknown :-1 s=CLOSED pgs=765 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:55.869 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:55.868+0000 7f269566e700 1 -- 172.21.15.59:0/4018420577 >> 172.21.15.59:0/4018420577 conn(0x7f26900d3e40 msgr2=0x7f26900b47f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:58:55.869 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:55.868+0000 7f269566e700 1 -- 172.21.15.59:0/4018420577 shutdown_connections 2023-11-24T07:58:55.869 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:55.868+0000 7f269566e700 1 -- 172.21.15.59:0/4018420577 wait complete. 2023-11-24T07:58:55.881 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-24T07:56:18.352823+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 8, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-24T07:58:39.262463+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '59', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 9, 'pool_name': 'cephfs_data', 'create_time': '2023-11-24T07:58:39.763399+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '60', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-24T07:58:55.881 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-24T07:58:56.009 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:56.007+0000 7f83d74e3700 1 Processor -- start 2023-11-24T07:58:56.009 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:56.007+0000 7f83d74e3700 1 -- start start 2023-11-24T07:58:56.011 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:56.008+0000 7f83d74e3700 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f83d00b4150 0x7f83d006de50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:56.012 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:56.008+0000 7f83d74e3700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f83d00b5ab0 0x7f83d006e390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:56.012 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:56.008+0000 7f83d74e3700 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f83d0071f50 con 0x7f83d006b620 2023-11-24T07:58:56.012 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:56.008+0000 7f83d74e3700 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f83d0072230 con 0x7f83d00b5ab0 2023-11-24T07:58:56.012 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:56.008+0000 7f83d74e3700 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f83d0072510 con 0x7f83d00b4150 2023-11-24T07:58:56.012 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:56.008+0000 7f83d5a80700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f83d00b5ab0 0x7f83d006e390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:56.012 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:56.008+0000 7f83d527f700 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7f83d006b620 0x7f83d00b3a40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:57694/0 (socket says 172.21.15.59:57694) 2023-11-24T07:58:56.012 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:56.008+0000 7f83d5a80700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f83d00b5ab0 0x7f83d006e390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:54224/0 (socket says 172.21.15.59:54224) 2023-11-24T07:58:56.012 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:56.008+0000 7f83d527f700 1 -- 172.21.15.59:0/2011143222 learned_addr learned my addr 172.21.15.59:0/2011143222 (peer_addr_for_me v1:172.21.15.59:0/0) 2023-11-24T07:58:56.012 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:56.009+0000 7f83d4a7e700 1 --2- 172.21.15.59:0/2011143222 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f83d00b4150 0x7f83d006de50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:56.012 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:56.009+0000 7f83d5a80700 1 -- 172.21.15.59:0/2011143222 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f83d00b4150 msgr2=0x7f83d006de50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T07:58:56.013 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:56.009+0000 7f83d5a80700 1 --2- 172.21.15.59:0/2011143222 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f83d00b4150 0x7f83d006de50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:56.013 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:56.009+0000 7f83d5a80700 1 -- 172.21.15.59:0/2011143222 >> v1:172.21.15.59:6789/0 conn(0x7f83d006b620 legacy=0x7f83d00b3a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:56.013 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:56.009+0000 7f83d5a80700 1 -- 172.21.15.59:0/2011143222 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f83d00727f0 con 0x7f83d00b5ab0 2023-11-24T07:58:56.013 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:56.009+0000 7f83c7fff700 1 -- 172.21.15.59:0/2011143222 <== mon.0 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2991333854 0 0) 0x7f83d0071f50 con 0x7f83d006b620 2023-11-24T07:58:56.013 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:56.009+0000 7f83d5a80700 1 --2- 172.21.15.59:0/2011143222 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f83d00b5ab0 0x7f83d006e390 secure :-1 s=READY pgs=767 cs=0 l=1 rev1=1 crypto rx=0x7f83c8009d90 tx=0x7f83c800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=29101c169d084f6c server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:58:56.013 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:56.009+0000 7f83c7fff700 1 -- 172.21.15.59:0/2011143222 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83c800e070 con 0x7f83d00b5ab0 2023-11-24T07:58:56.013 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:56.009+0000 7f83c7fff700 1 -- 172.21.15.59:0/2011143222 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f83c800bbd0 con 0x7f83d00b5ab0 2023-11-24T07:58:56.013 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:56.009+0000 7f83c7fff700 1 -- 172.21.15.59:0/2011143222 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83c8012430 con 0x7f83d00b5ab0 2023-11-24T07:58:56.013 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:56.010+0000 7f83d74e3700 1 -- 172.21.15.59:0/2011143222 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f83d00b5ab0 msgr2=0x7f83d006e390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:56.014 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:56.010+0000 7f83d74e3700 1 --2- 172.21.15.59:0/2011143222 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f83d00b5ab0 0x7f83d006e390 secure :-1 s=READY pgs=767 cs=0 l=1 rev1=1 crypto rx=0x7f83c8009d90 tx=0x7f83c800b330 comp rx=0 tx=0).stop 2023-11-24T07:58:56.014 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:56.010+0000 7f83d74e3700 1 -- 172.21.15.59:0/2011143222 shutdown_connections 2023-11-24T07:58:56.014 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:56.010+0000 7f83d74e3700 1 --2- 172.21.15.59:0/2011143222 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f83d00b5ab0 0x7f83d006e390 unknown :-1 s=CLOSED pgs=767 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:56.014 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:56.010+0000 7f83d74e3700 1 --2- 172.21.15.59:0/2011143222 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f83d00b4150 0x7f83d006de50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:56.014 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:56.010+0000 7f83d74e3700 1 -- 172.21.15.59:0/2011143222 >> 172.21.15.59:0/2011143222 conn(0x7f83d00b2740 msgr2=0x7f83d00b2b20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:58:56.014 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:56.010+0000 7f83d74e3700 1 -- 172.21.15.59:0/2011143222 shutdown_connections 2023-11-24T07:58:56.014 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:56.010+0000 7f83d74e3700 1 -- 172.21.15.59:0/2011143222 wait complete. 2023-11-24T07:58:56.016 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:56.011+0000 7f83d74e3700 1 Processor -- start 2023-11-24T07:58:56.016 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:56.011+0000 7f83d74e3700 1 -- start start 2023-11-24T07:58:56.017 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:56.012+0000 7f83d74e3700 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f83d006b620 0x7f83d013fc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:56.017 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:56.012+0000 7f83d74e3700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f83d00b4150 0x7f83d0140df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:56.017 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:56.012+0000 7f83d74e3700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f83d00b5ab0 0x7f83d01e3590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:56.017 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:56.012+0000 7f83d74e3700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83d01408f0 con 0x7f83d00b4150 2023-11-24T07:58:56.017 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:56.012+0000 7f83d74e3700 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83d01e4990 con 0x7f83d00b5ab0 2023-11-24T07:58:56.017 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:56.012+0000 7f83d74e3700 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f83d0140330 con 0x7f83d006b620 2023-11-24T07:58:56.017 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:56.013+0000 7f83d4a7e700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f83d00b4150 0x7f83d0140df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:56.017 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:56.013+0000 7f83d4a7e700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f83d00b4150 0x7f83d0140df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:54228/0 (socket says 172.21.15.59:54228) 2023-11-24T07:58:56.017 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:56.013+0000 7f83d4a7e700 1 -- 172.21.15.59:0/4140367143 learned_addr learned my addr 172.21.15.59:0/4140367143 (peer_addr_for_me v2:172.21.15.59:0/0) 2023-11-24T07:58:56.018 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:56.013+0000 7f83d527f700 1 --2- 172.21.15.59:0/4140367143 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f83d006b620 0x7f83d013fc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:56.018 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:56.013+0000 7f83d5a80700 1 --2- 172.21.15.59:0/4140367143 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f83d00b5ab0 0x7f83d01e3590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:56.018 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:56.013+0000 7f83d4a7e700 1 -- 172.21.15.59:0/4140367143 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f83d006b620 msgr2=0x7f83d013fc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:56.018 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:56.013+0000 7f83d4a7e700 1 --2- 172.21.15.59:0/4140367143 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f83d006b620 0x7f83d013fc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:56.018 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:56.013+0000 7f83d4a7e700 1 -- 172.21.15.59:0/4140367143 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f83d00b5ab0 msgr2=0x7f83d01e3590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:56.018 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:56.013+0000 7f83d4a7e700 1 --2- 172.21.15.59:0/4140367143 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f83d00b5ab0 0x7f83d01e3590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:56.018 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:56.013+0000 7f83d4a7e700 1 -- 172.21.15.59:0/4140367143 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f83c8009a70 con 0x7f83d00b4150 2023-11-24T07:58:56.018 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:56.013+0000 7f83d527f700 1 --2- 172.21.15.59:0/4140367143 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f83d006b620 0x7f83d013fc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-24T07:58:56.018 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:56.013+0000 7f83d4a7e700 1 --2- 172.21.15.59:0/4140367143 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f83d00b4150 0x7f83d0140df0 secure :-1 s=READY pgs=768 cs=0 l=1 rev1=1 crypto rx=0x7f83cc00e810 tx=0x7f83cc00ebd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:58:56.018 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:56.013+0000 7f83c5ffb700 1 -- 172.21.15.59:0/4140367143 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83cc013610 con 0x7f83d00b4150 2023-11-24T07:58:56.018 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:56.013+0000 7f83c5ffb700 1 -- 172.21.15.59:0/4140367143 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f83cc013770 con 0x7f83d00b4150 2023-11-24T07:58:56.019 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:56.013+0000 7f83d74e3700 1 -- 172.21.15.59:0/4140367143 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f83d0140610 con 0x7f83d00b4150 2023-11-24T07:58:56.019 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:56.013+0000 7f83c5ffb700 1 -- 172.21.15.59:0/4140367143 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83cc013b20 con 0x7f83d00b4150 2023-11-24T07:58:56.019 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:56.014+0000 7f83d74e3700 1 -- 172.21.15.59:0/4140367143 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f83d01e5e40 con 0x7f83d00b4150 2023-11-24T07:58:56.019 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:56.014+0000 7f83d74e3700 1 -- 172.21.15.59:0/4140367143 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f83d01e59c0 con 0x7f83d00b4150 2023-11-24T07:58:56.019 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:56.015+0000 7f83c5ffb700 1 -- 172.21.15.59:0/4140367143 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 10) v1 ==== 89588+0+0 (secure 0 0 0) 0x7f83cc013c80 con 0x7f83d00b4150 2023-11-24T07:58:56.020 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:56.015+0000 7f83c5ffb700 1 --2- 172.21.15.59:0/4140367143 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f83b406b2c0 0x7f83b406d780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:56.020 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:56.015+0000 7f83c5ffb700 1 -- 172.21.15.59:0/4140367143 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(60..60 src has 1..60) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f83cc089b60 con 0x7f83d00b4150 2023-11-24T07:58:56.020 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:56.015+0000 7f83d527f700 1 --2- 172.21.15.59:0/4140367143 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f83b406b2c0 0x7f83b406d780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:56.020 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:56.016+0000 7f83d527f700 1 --2- 172.21.15.59:0/4140367143 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f83b406b2c0 0x7f83b406d780 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f83c0000f60 tx=0x7f83c0007d30 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:58:56.021 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:56.020+0000 7f83c5ffb700 1 -- 172.21.15.59:0/4140367143 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f83d01e59c0 con 0x7f83d00b4150 2023-11-24T07:58:56.221 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:56.219+0000 7f83d74e3700 1 -- 172.21.15.59:0/4140367143 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f83d01e59c0 con 0x7f83d00b4150 2023-11-24T07:58:56.221 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:56.219+0000 7f83c5ffb700 1 -- 172.21.15.59:0/4140367143 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v60) v1 ==== 93+0+10 (secure 0 0 0) 0x7f83d01e59c0 con 0x7f83d00b4150 2023-11-24T07:58:56.221 INFO:teuthology.orchestra.run.smithi059.stdout:pg_num: 1 2023-11-24T07:58:56.224 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:56.221+0000 7f83d74e3700 1 -- 172.21.15.59:0/4140367143 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f83b406b2c0 msgr2=0x7f83b406d780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:56.224 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:56.221+0000 7f83d74e3700 1 --2- 172.21.15.59:0/4140367143 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f83b406b2c0 0x7f83b406d780 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f83c0000f60 tx=0x7f83c0007d30 comp rx=0 tx=0).stop 2023-11-24T07:58:56.224 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:56.221+0000 7f83d74e3700 1 -- 172.21.15.59:0/4140367143 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f83d00b4150 msgr2=0x7f83d0140df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:56.224 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:56.221+0000 7f83d74e3700 1 --2- 172.21.15.59:0/4140367143 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f83d00b4150 0x7f83d0140df0 secure :-1 s=READY pgs=768 cs=0 l=1 rev1=1 crypto rx=0x7f83cc00e810 tx=0x7f83cc00ebd0 comp rx=0 tx=0).stop 2023-11-24T07:58:56.224 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:56.222+0000 7f83d74e3700 1 -- 172.21.15.59:0/4140367143 shutdown_connections 2023-11-24T07:58:56.224 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:56.222+0000 7f83d74e3700 1 --2- 172.21.15.59:0/4140367143 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f83b406b2c0 0x7f83b406d780 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:56.224 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:56.222+0000 7f83d74e3700 1 --2- 172.21.15.59:0/4140367143 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f83d00b5ab0 0x7f83d01e3590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:56.224 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:56.222+0000 7f83d74e3700 1 --2- 172.21.15.59:0/4140367143 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f83d006b620 0x7f83d013fc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:56.224 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:56.222+0000 7f83d74e3700 1 --2- 172.21.15.59:0/4140367143 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f83d00b4150 0x7f83d0140df0 unknown :-1 s=CLOSED pgs=768 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:56.224 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:56.222+0000 7f83d74e3700 1 -- 172.21.15.59:0/4140367143 >> 172.21.15.59:0/4140367143 conn(0x7f83d00b2740 msgr2=0x7f83d01e5060 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:58:56.224 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:56.222+0000 7f83d74e3700 1 -- 172.21.15.59:0/4140367143 shutdown_connections 2023-11-24T07:58:56.225 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:56.222+0000 7f83d74e3700 1 -- 172.21.15.59:0/4140367143 wait complete. 2023-11-24T07:58:56.235 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-24T07:58:56.365 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:56.363+0000 7f24a6a06700 1 Processor -- start 2023-11-24T07:58:56.365 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:56.363+0000 7f24a6a06700 1 -- start start 2023-11-24T07:58:56.367 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:56.364+0000 7f24a6a06700 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f24a015f610 0x7f24a00b2880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:56.367 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:56.364+0000 7f24a6a06700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f24a00cf0e0 0x7f24a00cfae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:56.367 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:56.364+0000 7f24a6a06700 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f24a00d0020 0x7f24a00b4a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:56.368 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:56.364+0000 7f24a6a06700 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f24a00d04b0 con 0x7f24a00cf0e0 2023-11-24T07:58:56.368 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:56.364+0000 7f24a6a06700 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f24a00d0c70 con 0x7f24a00d0020 2023-11-24T07:58:56.368 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:56.364+0000 7f24a6a06700 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f24a00b3570 con 0x7f24a015f610 2023-11-24T07:58:56.368 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:56.364+0000 7f249f7fe700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f24a00cf0e0 0x7f24a00cfae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:56.368 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:56.364+0000 7f249f7fe700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f24a00cf0e0 0x7f24a00cfae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:54234/0 (socket says 172.21.15.59:54234) 2023-11-24T07:58:56.368 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:56.364+0000 7f249f7fe700 1 -- 172.21.15.59:0/3471515979 learned_addr learned my addr 172.21.15.59:0/3471515979 (peer_addr_for_me v2:172.21.15.59:0/0) 2023-11-24T07:58:56.368 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:56.364+0000 7f249ffff700 1 --2- 172.21.15.59:0/3471515979 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f24a015f610 0x7f24a00b2880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:56.368 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:56.364+0000 7f24a4fa3700 1 --2- 172.21.15.59:0/3471515979 >> v2:172.21.15.76:3300/0 conn(0x7f24a00d0020 0x7f24a00b4a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:56.368 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:56.364+0000 7f249f7fe700 1 -- 172.21.15.59:0/3471515979 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f24a015f610 msgr2=0x7f24a00b2880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T07:58:56.368 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:56.364+0000 7f249f7fe700 1 --2- 172.21.15.59:0/3471515979 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f24a015f610 0x7f24a00b2880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:56.368 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:56.364+0000 7f249f7fe700 1 -- 172.21.15.59:0/3471515979 >> v2:172.21.15.76:3300/0 conn(0x7f24a00d0020 msgr2=0x7f24a00b4a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T07:58:56.369 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:56.364+0000 7f249f7fe700 1 --2- 172.21.15.59:0/3471515979 >> v2:172.21.15.76:3300/0 conn(0x7f24a00d0020 0x7f24a00b4a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:56.369 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:56.364+0000 7f249f7fe700 1 -- 172.21.15.59:0/3471515979 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f24a00d06b0 con 0x7f24a00cf0e0 2023-11-24T07:58:56.369 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:56.365+0000 7f249f7fe700 1 --2- 172.21.15.59:0/3471515979 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f24a00cf0e0 0x7f24a00cfae0 secure :-1 s=READY pgs=769 cs=0 l=1 rev1=1 crypto rx=0x7f24a00d1940 tx=0x7f2490009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8a0835d099eb98e6 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:58:56.369 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:56.365+0000 7f249effd700 1 -- 172.21.15.59:0/3471515979 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2490011830 con 0x7f24a00cf0e0 2023-11-24T07:58:56.369 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:56.365+0000 7f249effd700 1 -- 172.21.15.59:0/3471515979 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2490011990 con 0x7f24a00cf0e0 2023-11-24T07:58:56.369 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:56.365+0000 7f249effd700 1 -- 172.21.15.59:0/3471515979 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2490011d20 con 0x7f24a00cf0e0 2023-11-24T07:58:56.369 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:56.365+0000 7f24a6a06700 1 -- 172.21.15.59:0/3471515979 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f24a00cf0e0 msgr2=0x7f24a00cfae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:56.369 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:56.365+0000 7f24a6a06700 1 --2- 172.21.15.59:0/3471515979 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f24a00cf0e0 0x7f24a00cfae0 secure :-1 s=READY pgs=769 cs=0 l=1 rev1=1 crypto rx=0x7f24a00d1940 tx=0x7f2490009bd0 comp rx=0 tx=0).stop 2023-11-24T07:58:56.369 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:56.365+0000 7f24a6a06700 1 -- 172.21.15.59:0/3471515979 shutdown_connections 2023-11-24T07:58:56.369 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:56.365+0000 7f24a6a06700 1 --2- 172.21.15.59:0/3471515979 >> v2:172.21.15.76:3300/0 conn(0x7f24a00d0020 0x7f24a00b4a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:56.369 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:56.365+0000 7f24a6a06700 1 --2- 172.21.15.59:0/3471515979 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f24a015f610 0x7f24a00b2880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:56.369 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:56.365+0000 7f24a6a06700 1 --2- 172.21.15.59:0/3471515979 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f24a00cf0e0 0x7f24a00cfae0 unknown :-1 s=CLOSED pgs=769 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:56.370 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:56.365+0000 7f24a6a06700 1 -- 172.21.15.59:0/3471515979 >> 172.21.15.59:0/3471515979 conn(0x7f24a00b3790 msgr2=0x7f24a00b3ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:58:56.370 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:56.365+0000 7f24a6a06700 1 -- 172.21.15.59:0/3471515979 shutdown_connections 2023-11-24T07:58:56.370 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:56.365+0000 7f24a6a06700 1 -- 172.21.15.59:0/3471515979 wait complete. 2023-11-24T07:58:56.372 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:56.367+0000 7f24a6a06700 1 Processor -- start 2023-11-24T07:58:56.373 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:56.367+0000 7f24a6a06700 1 -- start start 2023-11-24T07:58:56.373 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:56.368+0000 7f24a6a06700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f24a00cf0e0 0x7f24a015aa90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:56.373 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:56.368+0000 7f24a6a06700 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f24a00d0020 0x7f24a00c7ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:56.373 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:56.368+0000 7f24a6a06700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f24a015f610 0x7f24a01f70a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:56.373 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:56.368+0000 7f24a6a06700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24a00c4690 con 0x7f24a00cf0e0 2023-11-24T07:58:56.373 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:56.368+0000 7f24a6a06700 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24a00c4ea0 con 0x7f24a015f610 2023-11-24T07:58:56.373 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:56.368+0000 7f24a6a06700 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f24a015afd0 con 0x7f24a00d0020 2023-11-24T07:58:56.373 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:56.368+0000 7f249ffff700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f24a00cf0e0 0x7f24a015aa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:56.373 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:56.368+0000 7f249ffff700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f24a00cf0e0 0x7f24a015aa90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:54240/0 (socket says 172.21.15.59:54240) 2023-11-24T07:58:56.373 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:56.368+0000 7f249ffff700 1 -- 172.21.15.59:0/1611122425 learned_addr learned my addr 172.21.15.59:0/1611122425 (peer_addr_for_me v2:172.21.15.59:0/0) 2023-11-24T07:58:56.373 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:56.368+0000 7f249f7fe700 1 --2- 172.21.15.59:0/1611122425 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f24a00d0020 0x7f24a00c7ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:56.373 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:56.368+0000 7f24a4fa3700 1 --2- 172.21.15.59:0/1611122425 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f24a015f610 0x7f24a01f70a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:56.373 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:56.368+0000 7f249ffff700 1 -- 172.21.15.59:0/1611122425 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f24a00d0020 msgr2=0x7f24a00c7ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:56.374 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:56.368+0000 7f249ffff700 1 --2- 172.21.15.59:0/1611122425 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f24a00d0020 0x7f24a00c7ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:56.374 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:56.368+0000 7f249ffff700 1 -- 172.21.15.59:0/1611122425 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f24a015f610 msgr2=0x7f24a01f70a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:56.374 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:56.368+0000 7f249ffff700 1 --2- 172.21.15.59:0/1611122425 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f24a015f610 0x7f24a01f70a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:56.374 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:56.368+0000 7f249ffff700 1 -- 172.21.15.59:0/1611122425 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2490009550 con 0x7f24a00cf0e0 2023-11-24T07:58:56.374 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:56.368+0000 7f24a4fa3700 1 --2- 172.21.15.59:0/1611122425 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f24a015f610 0x7f24a01f70a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T07:58:56.374 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:56.369+0000 7f249ffff700 1 --2- 172.21.15.59:0/1611122425 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f24a00cf0e0 0x7f24a015aa90 secure :-1 s=READY pgs=770 cs=0 l=1 rev1=1 crypto rx=0x7f248800e870 tx=0x7f248800ec30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:58:56.374 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:56.369+0000 7f249cff9700 1 -- 172.21.15.59:0/1611122425 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2488009590 con 0x7f24a00cf0e0 2023-11-24T07:58:56.374 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:56.369+0000 7f249cff9700 1 -- 172.21.15.59:0/1611122425 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f248800ee70 con 0x7f24a00cf0e0 2023-11-24T07:58:56.374 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:56.369+0000 7f24a6a06700 1 -- 172.21.15.59:0/1611122425 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f24a00c8cb0 con 0x7f24a00cf0e0 2023-11-24T07:58:56.374 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:56.369+0000 7f249cff9700 1 -- 172.21.15.59:0/1611122425 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f248800ca50 con 0x7f24a00cf0e0 2023-11-24T07:58:56.374 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:56.369+0000 7f24a6a06700 1 -- 172.21.15.59:0/1611122425 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f24a00c4bc0 con 0x7f24a00cf0e0 2023-11-24T07:58:56.374 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:56.370+0000 7f24a6a06700 1 -- 172.21.15.59:0/1611122425 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f24a006c890 con 0x7f24a00cf0e0 2023-11-24T07:58:56.375 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:56.370+0000 7f249cff9700 1 -- 172.21.15.59:0/1611122425 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 10) v1 ==== 89588+0+0 (secure 0 0 0) 0x7f248800cbb0 con 0x7f24a00cf0e0 2023-11-24T07:58:56.375 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:56.371+0000 7f249cff9700 1 --2- 172.21.15.59:0/1611122425 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f248006b2c0 0x7f248006d780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:56.375 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:56.371+0000 7f249cff9700 1 -- 172.21.15.59:0/1611122425 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(60..60 src has 1..60) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f2488089c10 con 0x7f24a00cf0e0 2023-11-24T07:58:56.375 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:56.371+0000 7f249f7fe700 1 --2- 172.21.15.59:0/1611122425 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f248006b2c0 0x7f248006d780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:56.375 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:56.371+0000 7f249f7fe700 1 --2- 172.21.15.59:0/1611122425 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f248006b2c0 0x7f248006d780 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f24a00c5620 tx=0x7f2490008030 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:58:56.377 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:56.375+0000 7f249cff9700 1 -- 172.21.15.59:0/1611122425 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f2488054770 con 0x7f24a00cf0e0 2023-11-24T07:58:56.582 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:56.580+0000 7f24a6a06700 1 -- 172.21.15.59:0/1611122425 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f24a006dbc0 con 0x7f24a00cf0e0 2023-11-24T07:58:56.582 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:56.580+0000 7f249cff9700 1 -- 172.21.15.59:0/1611122425 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v60) v1 ==== 104+0+11 (secure 0 0 0) 0x7f2488021020 con 0x7f24a00cf0e0 2023-11-24T07:58:56.582 INFO:teuthology.orchestra.run.smithi059.stdout:pg_num: 32 2023-11-24T07:58:56.584 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:56.582+0000 7f24a6a06700 1 -- 172.21.15.59:0/1611122425 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f248006b2c0 msgr2=0x7f248006d780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:56.584 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:56.582+0000 7f24a6a06700 1 --2- 172.21.15.59:0/1611122425 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f248006b2c0 0x7f248006d780 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f24a00c5620 tx=0x7f2490008030 comp rx=0 tx=0).stop 2023-11-24T07:58:56.585 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:56.582+0000 7f24a6a06700 1 -- 172.21.15.59:0/1611122425 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f24a00cf0e0 msgr2=0x7f24a015aa90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:56.585 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:56.582+0000 7f24a6a06700 1 --2- 172.21.15.59:0/1611122425 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f24a00cf0e0 0x7f24a015aa90 secure :-1 s=READY pgs=770 cs=0 l=1 rev1=1 crypto rx=0x7f248800e870 tx=0x7f248800ec30 comp rx=0 tx=0).stop 2023-11-24T07:58:56.585 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:56.583+0000 7f24a6a06700 1 -- 172.21.15.59:0/1611122425 shutdown_connections 2023-11-24T07:58:56.585 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:56.583+0000 7f24a6a06700 1 --2- 172.21.15.59:0/1611122425 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f248006b2c0 0x7f248006d780 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:56.585 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:56.583+0000 7f24a6a06700 1 --2- 172.21.15.59:0/1611122425 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f24a015f610 0x7f24a01f70a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:56.585 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:56.583+0000 7f24a6a06700 1 --2- 172.21.15.59:0/1611122425 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f24a00cf0e0 0x7f24a015aa90 unknown :-1 s=CLOSED pgs=770 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:56.585 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:56.583+0000 7f24a6a06700 1 --2- 172.21.15.59:0/1611122425 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f24a00d0020 0x7f24a00c7ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:56.585 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:56.583+0000 7f24a6a06700 1 -- 172.21.15.59:0/1611122425 >> 172.21.15.59:0/1611122425 conn(0x7f24a00b3790 msgr2=0x7f24a006fb50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:58:56.585 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:56.583+0000 7f24a6a06700 1 -- 172.21.15.59:0/1611122425 shutdown_connections 2023-11-24T07:58:56.585 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:56.583+0000 7f24a6a06700 1 -- 172.21.15.59:0/1611122425 wait complete. 2023-11-24T07:58:56.596 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-24T07:58:56.724 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:56.723+0000 7f69bb233700 1 Processor -- start 2023-11-24T07:58:56.725 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:56.723+0000 7f69bb233700 1 -- start start 2023-11-24T07:58:56.727 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:56.724+0000 7f69bb233700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f69b406b620 0x7f69b40b3a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:56.727 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:56.724+0000 7f69bb233700 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f69b406ee60 0x7f69b40b3f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:56.727 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:56.724+0000 7f69bb233700 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f69b4071f50 con 0x7f69b40b5ab0 2023-11-24T07:58:56.727 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:56.724+0000 7f69bb233700 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f69b4072230 con 0x7f69b406b620 2023-11-24T07:58:56.727 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:56.724+0000 7f69bb233700 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f69b4072510 con 0x7f69b406ee60 2023-11-24T07:58:56.727 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:56.724+0000 7f69b8fcf700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f69b406b620 0x7f69b40b3a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:56.727 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:56.724+0000 7f69b8fcf700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f69b406b620 0x7f69b40b3a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:54248/0 (socket says 172.21.15.59:54248) 2023-11-24T07:58:56.727 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:56.724+0000 7f69b8fcf700 1 -- 172.21.15.59:0/712402426 learned_addr learned my addr 172.21.15.59:0/712402426 (peer_addr_for_me v2:172.21.15.59:0/0) 2023-11-24T07:58:56.728 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:56.724+0000 7f69b97d0700 1 --2- 172.21.15.59:0/712402426 >> v2:172.21.15.76:3300/0 conn(0x7f69b406ee60 0x7f69b40b3f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:56.728 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:56.724+0000 7f69b8fcf700 1 -- 172.21.15.59:0/712402426 >> v2:172.21.15.76:3300/0 conn(0x7f69b406ee60 msgr2=0x7f69b40b3f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T07:58:56.728 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:56.724+0000 7f69b8fcf700 1 --2- 172.21.15.59:0/712402426 >> v2:172.21.15.76:3300/0 conn(0x7f69b406ee60 0x7f69b40b3f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:56.728 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:56.724+0000 7f69b8fcf700 1 -- 172.21.15.59:0/712402426 >> v1:172.21.15.76:6789/0 conn(0x7f69b40b5ab0 legacy=0x7f69b4161ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T07:58:56.728 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:56.724+0000 7f69b8fcf700 1 -- 172.21.15.59:0/712402426 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f69b40727f0 con 0x7f69b406b620 2023-11-24T07:58:56.728 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:56.724+0000 7f69b8fcf700 1 --2- 172.21.15.59:0/712402426 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f69b406b620 0x7f69b40b3a40 secure :-1 s=READY pgs=771 cs=0 l=1 rev1=1 crypto rx=0x7f69b40b0660 tx=0x7f69a8009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8f12bd8b726b8b20 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:58:56.728 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:56.724+0000 7f69b37fe700 1 -- 172.21.15.59:0/712402426 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69a8004230 con 0x7f69b406b620 2023-11-24T07:58:56.728 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:56.725+0000 7f69b37fe700 1 -- 172.21.15.59:0/712402426 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f69a8004390 con 0x7f69b406b620 2023-11-24T07:58:56.728 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:56.725+0000 7f69b37fe700 1 -- 172.21.15.59:0/712402426 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69a80036a0 con 0x7f69b406b620 2023-11-24T07:58:56.729 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:56.725+0000 7f69bb233700 1 -- 172.21.15.59:0/712402426 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f69b406b620 msgr2=0x7f69b40b3a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:56.729 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:56.725+0000 7f69bb233700 1 --2- 172.21.15.59:0/712402426 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f69b406b620 0x7f69b40b3a40 secure :-1 s=READY pgs=771 cs=0 l=1 rev1=1 crypto rx=0x7f69b40b0660 tx=0x7f69a8009bd0 comp rx=0 tx=0).stop 2023-11-24T07:58:56.729 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:56.725+0000 7f69bb233700 1 -- 172.21.15.59:0/712402426 shutdown_connections 2023-11-24T07:58:56.729 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:56.725+0000 7f69bb233700 1 --2- 172.21.15.59:0/712402426 >> v2:172.21.15.76:3300/0 conn(0x7f69b406ee60 0x7f69b40b3f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:56.729 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:56.725+0000 7f69bb233700 1 --2- 172.21.15.59:0/712402426 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f69b406b620 0x7f69b40b3a40 unknown :-1 s=CLOSED pgs=771 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:56.729 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:56.725+0000 7f69bb233700 1 -- 172.21.15.59:0/712402426 >> 172.21.15.59:0/712402426 conn(0x7f69b40b2740 msgr2=0x7f69b40b2b20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:58:56.729 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:56.725+0000 7f69bb233700 1 -- 172.21.15.59:0/712402426 shutdown_connections 2023-11-24T07:58:56.729 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:56.725+0000 7f69bb233700 1 -- 172.21.15.59:0/712402426 wait complete. 2023-11-24T07:58:56.732 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:56.726+0000 7f69bb233700 1 Processor -- start 2023-11-24T07:58:56.732 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:56.726+0000 7f69bb233700 1 -- start start 2023-11-24T07:58:56.732 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:56.727+0000 7f69bb233700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f69b406b620 0x7f69b40cf260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:56.732 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:56.727+0000 7f69bb233700 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f69b406ee60 0x7f69b40c7030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:56.732 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:56.727+0000 7f69bb233700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f69b40b5ab0 0x7f69b40b1ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:56.732 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:56.727+0000 7f69bb233700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f69b40c7a10 con 0x7f69b40b5ab0 2023-11-24T07:58:56.732 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:56.727+0000 7f69bb233700 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f69b40c7cf0 con 0x7f69b406b620 2023-11-24T07:58:56.732 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:56.727+0000 7f69bb233700 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f69b40c7fd0 con 0x7f69b406ee60 2023-11-24T07:58:56.732 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:56.727+0000 7f69b97d0700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f69b40b5ab0 0x7f69b40b1ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:56.733 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:56.727+0000 7f69b97d0700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f69b40b5ab0 0x7f69b40b1ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:54258/0 (socket says 172.21.15.59:54258) 2023-11-24T07:58:56.733 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:56.727+0000 7f69b97d0700 1 -- 172.21.15.59:0/2927959414 learned_addr learned my addr 172.21.15.59:0/2927959414 (peer_addr_for_me v2:172.21.15.59:0/0) 2023-11-24T07:58:56.733 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:56.727+0000 7f69b8fcf700 1 --2- 172.21.15.59:0/2927959414 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f69b406b620 0x7f69b40cf260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:56.733 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:56.727+0000 7f69b3fff700 1 --2- 172.21.15.59:0/2927959414 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f69b406ee60 0x7f69b40c7030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:56.733 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:56.728+0000 7f69b97d0700 1 -- 172.21.15.59:0/2927959414 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f69b406ee60 msgr2=0x7f69b40c7030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:56.733 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:56.728+0000 7f69b97d0700 1 --2- 172.21.15.59:0/2927959414 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f69b406ee60 0x7f69b40c7030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:56.733 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:56.728+0000 7f69b97d0700 1 -- 172.21.15.59:0/2927959414 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f69b406b620 msgr2=0x7f69b40cf260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:56.733 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:56.728+0000 7f69b97d0700 1 --2- 172.21.15.59:0/2927959414 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f69b406b620 0x7f69b40cf260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:56.733 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:56.728+0000 7f69b97d0700 1 -- 172.21.15.59:0/2927959414 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f69a8009550 con 0x7f69b40b5ab0 2023-11-24T07:58:56.733 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:56.728+0000 7f69b8fcf700 1 --2- 172.21.15.59:0/2927959414 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f69b406b620 0x7f69b40cf260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-24T07:58:56.734 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:56.728+0000 7f69b97d0700 1 --2- 172.21.15.59:0/2927959414 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f69b40b5ab0 0x7f69b40b1ad0 secure :-1 s=READY pgs=772 cs=0 l=1 rev1=1 crypto rx=0x7f69ac007e40 tx=0x7f69ac00e3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:58:56.734 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:56.728+0000 7f69b17fa700 1 -- 172.21.15.59:0/2927959414 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69ac00eea0 con 0x7f69b40b5ab0 2023-11-24T07:58:56.734 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:56.728+0000 7f69b17fa700 1 -- 172.21.15.59:0/2927959414 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f69ac014420 con 0x7f69b40b5ab0 2023-11-24T07:58:56.734 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:56.728+0000 7f69b17fa700 1 -- 172.21.15.59:0/2927959414 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69ac0147b0 con 0x7f69b40b5ab0 2023-11-24T07:58:56.734 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:56.728+0000 7f69bb233700 1 -- 172.21.15.59:0/2927959414 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f69b40c82b0 con 0x7f69b40b5ab0 2023-11-24T07:58:56.734 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:56.728+0000 7f69bb233700 1 -- 172.21.15.59:0/2927959414 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f69b40cf7a0 con 0x7f69b40b5ab0 2023-11-24T07:58:56.734 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:56.729+0000 7f69bb233700 1 -- 172.21.15.59:0/2927959414 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f69b40c8900 con 0x7f69b40b5ab0 2023-11-24T07:58:56.734 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:56.730+0000 7f69b17fa700 1 -- 172.21.15.59:0/2927959414 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 10) v1 ==== 89588+0+0 (secure 0 0 0) 0x7f69ac014910 con 0x7f69b40b5ab0 2023-11-24T07:58:56.734 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:56.730+0000 7f69b17fa700 1 --2- 172.21.15.59:0/2927959414 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f699806b2c0 0x7f699806d780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:56.735 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:56.730+0000 7f69b17fa700 1 -- 172.21.15.59:0/2927959414 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(60..60 src has 1..60) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f69ac08a510 con 0x7f69b40b5ab0 2023-11-24T07:58:56.735 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:56.730+0000 7f69b8fcf700 1 --2- 172.21.15.59:0/2927959414 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f699806b2c0 0x7f699806d780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:56.735 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:56.731+0000 7f69b8fcf700 1 --2- 172.21.15.59:0/2927959414 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f699806b2c0 0x7f699806d780 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f69a8009130 tx=0x7f69a8008030 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:58:56.736 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:56.734+0000 7f69b17fa700 1 -- 172.21.15.59:0/2927959414 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f69ac054fe0 con 0x7f69b40b5ab0 2023-11-24T07:58:56.937 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:56.935+0000 7f69bb233700 1 -- 172.21.15.59:0/2927959414 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f69b40cf930 con 0x7f69b40b5ab0 2023-11-24T07:58:56.937 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:56.936+0000 7f69b17fa700 1 -- 172.21.15.59:0/2927959414 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v60) v1 ==== 100+0+11 (secure 0 0 0) 0x7f69ac088e60 con 0x7f69b40b5ab0 2023-11-24T07:58:56.938 INFO:teuthology.orchestra.run.smithi059.stdout:pg_num: 64 2023-11-24T07:58:56.941 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:56.938+0000 7f69bb233700 1 -- 172.21.15.59:0/2927959414 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f699806b2c0 msgr2=0x7f699806d780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:56.941 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:56.938+0000 7f69bb233700 1 --2- 172.21.15.59:0/2927959414 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f699806b2c0 0x7f699806d780 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f69a8009130 tx=0x7f69a8008030 comp rx=0 tx=0).stop 2023-11-24T07:58:56.941 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:56.938+0000 7f69bb233700 1 -- 172.21.15.59:0/2927959414 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f69b40b5ab0 msgr2=0x7f69b40b1ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:56.941 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:56.938+0000 7f69bb233700 1 --2- 172.21.15.59:0/2927959414 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f69b40b5ab0 0x7f69b40b1ad0 secure :-1 s=READY pgs=772 cs=0 l=1 rev1=1 crypto rx=0x7f69ac007e40 tx=0x7f69ac00e3c0 comp rx=0 tx=0).stop 2023-11-24T07:58:56.941 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:56.938+0000 7f69bb233700 1 -- 172.21.15.59:0/2927959414 shutdown_connections 2023-11-24T07:58:56.942 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:56.938+0000 7f69bb233700 1 --2- 172.21.15.59:0/2927959414 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f699806b2c0 0x7f699806d780 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:56.942 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:56.938+0000 7f69bb233700 1 --2- 172.21.15.59:0/2927959414 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f69b40b5ab0 0x7f69b40b1ad0 unknown :-1 s=CLOSED pgs=772 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:56.942 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:56.938+0000 7f69bb233700 1 --2- 172.21.15.59:0/2927959414 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f69b406b620 0x7f69b40cf260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:56.942 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:56.938+0000 7f69bb233700 1 --2- 172.21.15.59:0/2927959414 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f69b406ee60 0x7f69b40c7030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:56.942 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:56.938+0000 7f69bb233700 1 -- 172.21.15.59:0/2927959414 >> 172.21.15.59:0/2927959414 conn(0x7f69b40b2740 msgr2=0x7f69b41e7b90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:58:56.942 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:56.938+0000 7f69bb233700 1 -- 172.21.15.59:0/2927959414 shutdown_connections 2023-11-24T07:58:56.942 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:56.938+0000 7f69bb233700 1 -- 172.21.15.59:0/2927959414 wait complete. 2023-11-24T07:58:56.952 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-11-24T07:58:57.078 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:57.076+0000 7f6cd3fc7700 1 Processor -- start 2023-11-24T07:58:57.078 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:57.076+0000 7f6cd3fc7700 1 -- start start 2023-11-24T07:58:57.081 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:57.077+0000 7f6cd3fc7700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f6ccc0cecd0 0x7f6ccc15c1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:57.081 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:57.077+0000 7f6cd3fc7700 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6ccc0d0a90 con 0x7f6ccc0cf2b0 2023-11-24T07:58:57.081 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:57.077+0000 7f6cd3fc7700 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6ccc06ee10 con 0x7f6ccc15bde0 2023-11-24T07:58:57.081 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:57.077+0000 7f6cd3fc7700 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6ccc06f0f0 con 0x7f6ccc0cecd0 2023-11-24T07:58:57.082 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:57.077+0000 7f6cd2564700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f6ccc0cecd0 0x7f6ccc15c1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:57.082 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:57.077+0000 7f6cd1562700 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7f6ccc0cf2b0 0x7f6ccc0ce5c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:57736/0 (socket says 172.21.15.59:57736) 2023-11-24T07:58:57.082 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:57.078+0000 7f6cd1562700 1 -- 172.21.15.59:0/4275433208 learned_addr learned my addr 172.21.15.59:0/4275433208 (peer_addr_for_me v1:172.21.15.59:0/0) 2023-11-24T07:58:57.083 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:57.078+0000 7f6cd2564700 1 -- 172.21.15.59:0/4275433208 >> v1:172.21.15.76:6789/0 conn(0x7f6ccc15bde0 legacy=0x7f6ccc0d01b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:57.083 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:57.078+0000 7f6cd2564700 1 -- 172.21.15.59:0/4275433208 >> v1:172.21.15.59:6789/0 conn(0x7f6ccc0cf2b0 legacy=0x7f6ccc0ce5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:57.084 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:57.078+0000 7f6cd2564700 1 -- 172.21.15.59:0/4275433208 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6ccc06f3d0 con 0x7f6ccc0cecd0 2023-11-24T07:58:57.084 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:57.078+0000 7f6cd2564700 1 --2- 172.21.15.59:0/4275433208 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6ccc0cecd0 0x7f6ccc15c1c0 secure :-1 s=READY pgs=774 cs=0 l=1 rev1=1 crypto rx=0x7f6cc4009d90 tx=0x7f6cc400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=96ced3b16e1f6c0b server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:58:57.084 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:57.078+0000 7f6cd0d61700 1 -- 172.21.15.59:0/4275433208 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6cc400e070 con 0x7f6ccc0cecd0 2023-11-24T07:58:57.085 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:57.078+0000 7f6cd0d61700 1 -- 172.21.15.59:0/4275433208 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6cc400bbd0 con 0x7f6ccc0cecd0 2023-11-24T07:58:57.085 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:57.078+0000 7f6cd0d61700 1 -- 172.21.15.59:0/4275433208 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6cc4012430 con 0x7f6ccc0cecd0 2023-11-24T07:58:57.085 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:57.079+0000 7f6cd3fc7700 1 -- 172.21.15.59:0/4275433208 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6ccc0cecd0 msgr2=0x7f6ccc15c1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:57.085 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:57.079+0000 7f6cd3fc7700 1 --2- 172.21.15.59:0/4275433208 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6ccc0cecd0 0x7f6ccc15c1c0 secure :-1 s=READY pgs=774 cs=0 l=1 rev1=1 crypto rx=0x7f6cc4009d90 tx=0x7f6cc400b330 comp rx=0 tx=0).stop 2023-11-24T07:58:57.085 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:57.079+0000 7f6cd3fc7700 1 -- 172.21.15.59:0/4275433208 shutdown_connections 2023-11-24T07:58:57.085 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:57.079+0000 7f6cd3fc7700 1 --2- 172.21.15.59:0/4275433208 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6ccc0cecd0 0x7f6ccc15c1c0 unknown :-1 s=CLOSED pgs=774 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:57.086 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:57.079+0000 7f6cd3fc7700 1 -- 172.21.15.59:0/4275433208 >> 172.21.15.59:0/4275433208 conn(0x7f6ccc0b3720 msgr2=0x7f6ccc0b3b30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:58:57.086 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:57.079+0000 7f6cd3fc7700 1 -- 172.21.15.59:0/4275433208 shutdown_connections 2023-11-24T07:58:57.086 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:57.079+0000 7f6cd3fc7700 1 -- 172.21.15.59:0/4275433208 wait complete. 2023-11-24T07:58:57.094 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:57.080+0000 7f6cd3fc7700 1 Processor -- start 2023-11-24T07:58:57.094 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:57.080+0000 7f6cd3fc7700 1 -- start start 2023-11-24T07:58:57.094 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:57.081+0000 7f6cd3fc7700 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f6ccc0cecd0 0x7f6ccc0c9c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:57.094 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:57.081+0000 7f6cd3fc7700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6ccc0cf2b0 0x7f6ccc152750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:57.095 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:57.081+0000 7f6cd3fc7700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6ccc15bde0 0x7f6ccc0cb490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:57.095 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:57.081+0000 7f6cd3fc7700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ccc0c7260 con 0x7f6ccc0cf2b0 2023-11-24T07:58:57.095 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:57.081+0000 7f6cd3fc7700 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ccc152c90 con 0x7f6ccc15bde0 2023-11-24T07:58:57.096 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:57.081+0000 7f6cd3fc7700 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6ccc0c65e0 con 0x7f6ccc0cecd0 2023-11-24T07:58:57.096 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:57.082+0000 7f6cd1562700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6ccc0cf2b0 0x7f6ccc152750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:57.096 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:57.082+0000 7f6cd1562700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6ccc0cf2b0 0x7f6ccc152750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:54270/0 (socket says 172.21.15.59:54270) 2023-11-24T07:58:57.096 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:57.082+0000 7f6cd1562700 1 -- 172.21.15.59:0/783718276 learned_addr learned my addr 172.21.15.59:0/783718276 (peer_addr_for_me v2:172.21.15.59:0/0) 2023-11-24T07:58:57.096 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:57.082+0000 7f6cd1d63700 1 --2- 172.21.15.59:0/783718276 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f6ccc0cecd0 0x7f6ccc0c9c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:57.096 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:57.082+0000 7f6cd2564700 1 --2- 172.21.15.59:0/783718276 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6ccc15bde0 0x7f6ccc0cb490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:57.097 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:57.082+0000 7f6cd1562700 1 -- 172.21.15.59:0/783718276 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f6ccc0cecd0 msgr2=0x7f6ccc0c9c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:57.097 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:57.082+0000 7f6cd1562700 1 --2- 172.21.15.59:0/783718276 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f6ccc0cecd0 0x7f6ccc0c9c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:57.097 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:57.082+0000 7f6cd1562700 1 -- 172.21.15.59:0/783718276 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6ccc15bde0 msgr2=0x7f6ccc0cb490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:57.097 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:57.082+0000 7f6cd1562700 1 --2- 172.21.15.59:0/783718276 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6ccc15bde0 0x7f6ccc0cb490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:57.097 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:57.082+0000 7f6cd1562700 1 -- 172.21.15.59:0/783718276 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6cc4009a70 con 0x7f6ccc0cf2b0 2023-11-24T07:58:57.098 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:57.082+0000 7f6cd1562700 1 --2- 172.21.15.59:0/783718276 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6ccc0cf2b0 0x7f6ccc152750 secure :-1 s=READY pgs=775 cs=0 l=1 rev1=1 crypto rx=0x7f6cc800c530 tx=0x7f6cc800c840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:58:57.098 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:57.082+0000 7f6cc27fc700 1 -- 172.21.15.59:0/783718276 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6cc8005d70 con 0x7f6ccc0cf2b0 2023-11-24T07:58:57.098 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:57.082+0000 7f6cc27fc700 1 -- 172.21.15.59:0/783718276 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6cc80070d0 con 0x7f6ccc0cf2b0 2023-11-24T07:58:57.098 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:57.082+0000 7f6cd3fc7700 1 -- 172.21.15.59:0/783718276 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6ccc0c68c0 con 0x7f6ccc0cf2b0 2023-11-24T07:58:57.099 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:57.082+0000 7f6cc27fc700 1 -- 172.21.15.59:0/783718276 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6cc8007440 con 0x7f6ccc0cf2b0 2023-11-24T07:58:57.099 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:57.082+0000 7f6cd3fc7700 1 -- 172.21.15.59:0/783718276 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6ccc0c6e80 con 0x7f6ccc0cf2b0 2023-11-24T07:58:57.099 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:57.083+0000 7f6cd3fc7700 1 -- 172.21.15.59:0/783718276 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6ccc0c73a0 con 0x7f6ccc0cf2b0 2023-11-24T07:58:57.099 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:57.084+0000 7f6cc27fc700 1 -- 172.21.15.59:0/783718276 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 10) v1 ==== 89588+0+0 (secure 0 0 0) 0x7f6cc80075a0 con 0x7f6ccc0cf2b0 2023-11-24T07:58:57.099 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:57.084+0000 7f6cc27fc700 1 --2- 172.21.15.59:0/783718276 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f6cb006b2c0 0x7f6cb006d780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:57.099 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:57.084+0000 7f6cc27fc700 1 -- 172.21.15.59:0/783718276 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(60..60 src has 1..60) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f6cc8088f80 con 0x7f6ccc0cf2b0 2023-11-24T07:58:57.100 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:57.084+0000 7f6cd1d63700 1 --2- 172.21.15.59:0/783718276 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f6cb006b2c0 0x7f6cb006d780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:57.100 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:57.085+0000 7f6cd1d63700 1 --2- 172.21.15.59:0/783718276 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f6cb006b2c0 0x7f6cb006d780 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f6ccc153770 tx=0x7f6cbc009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:58:57.100 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:57.089+0000 7f6cc27fc700 1 -- 172.21.15.59:0/783718276 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f6ccc0c73a0 con 0x7f6ccc0cf2b0 2023-11-24T07:58:57.309 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:57.307+0000 7f6cd3fc7700 1 -- 172.21.15.59:0/783718276 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f6ccc0c73a0 con 0x7f6ccc0cf2b0 2023-11-24T07:58:57.309 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:57.308+0000 7f6cc27fc700 1 -- 172.21.15.59:0/783718276 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v60) v1 ==== 80+0+0 (secure 0 0 0) 0x7f6ccc0c73a0 con 0x7f6ccc0cf2b0 2023-11-24T07:58:57.312 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:57.310+0000 7f6cd3fc7700 1 -- 172.21.15.59:0/783718276 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f6cb006b2c0 msgr2=0x7f6cb006d780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:57.312 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:57.310+0000 7f6cd3fc7700 1 --2- 172.21.15.59:0/783718276 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f6cb006b2c0 0x7f6cb006d780 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f6ccc153770 tx=0x7f6cbc009040 comp rx=0 tx=0).stop 2023-11-24T07:58:57.312 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:57.310+0000 7f6cd3fc7700 1 -- 172.21.15.59:0/783718276 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6ccc0cf2b0 msgr2=0x7f6ccc152750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:57.312 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:57.310+0000 7f6cd3fc7700 1 --2- 172.21.15.59:0/783718276 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6ccc0cf2b0 0x7f6ccc152750 secure :-1 s=READY pgs=775 cs=0 l=1 rev1=1 crypto rx=0x7f6cc800c530 tx=0x7f6cc800c840 comp rx=0 tx=0).stop 2023-11-24T07:58:57.313 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:57.310+0000 7f6cd3fc7700 1 -- 172.21.15.59:0/783718276 shutdown_connections 2023-11-24T07:58:57.313 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:57.310+0000 7f6cd3fc7700 1 --2- 172.21.15.59:0/783718276 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f6cb006b2c0 0x7f6cb006d780 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:57.313 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:57.310+0000 7f6cd3fc7700 1 --2- 172.21.15.59:0/783718276 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6ccc15bde0 0x7f6ccc0cb490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:57.313 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:57.310+0000 7f6cd3fc7700 1 --2- 172.21.15.59:0/783718276 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f6ccc0cecd0 0x7f6ccc0c9c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:57.313 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:57.310+0000 7f6cd3fc7700 1 --2- 172.21.15.59:0/783718276 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6ccc0cf2b0 0x7f6ccc152750 unknown :-1 s=CLOSED pgs=775 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:57.313 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:57.310+0000 7f6cd3fc7700 1 -- 172.21.15.59:0/783718276 >> 172.21.15.59:0/783718276 conn(0x7f6ccc0b3720 msgr2=0x7f6ccc0b3b00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:58:57.313 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:57.310+0000 7f6cd3fc7700 1 -- 172.21.15.59:0/783718276 shutdown_connections 2023-11-24T07:58:57.314 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:57.310+0000 7f6cd3fc7700 1 -- 172.21.15.59:0/783718276 wait complete. 2023-11-24T07:58:57.314 INFO:teuthology.orchestra.run.smithi059.stderr:listed 0 entries 2023-11-24T07:58:57.324 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.1... 2023-11-24T07:58:57.324 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T07:58:57.324 DEBUG:teuthology.orchestra.run.smithi076:> sudo umount /home/ubuntu/cephtest/mnt.1 2023-11-24T07:58:57.368 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi076.front.sepia.ceph.com 2023-11-24T07:58:57.368 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T07:58:57.368 DEBUG:teuthology.orchestra.run.smithi076:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2023-11-24T07:58:57.387 DEBUG:teuthology.orchestra.run.smithi059:> set -ex 2023-11-24T07:58:57.387 DEBUG:teuthology.orchestra.run.smithi059:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-24T07:58:57.405 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=16287364k,nr_inodes=4071841,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 tmpfs /sys/fs/cgroup tmpfs ro,seclabel,nosuid,nodev,noexec,mode=755 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,seclabel,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,seclabel,nosuid,nodev,noexec,relatime,hugetlb 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,seclabel,nosuid,nodev,noexec,relatime,perf_event 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,seclabel,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/rdma cgroup rw,seclabel,nosuid,nodev,noexec,relatime,rdma 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,seclabel,nosuid,nodev,noexec,relatime,freezer 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,seclabel,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,seclabel,nosuid,nodev,noexec,relatime,devices 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,blkio 0 0 none /sys/kernel/tracing tracefs rw,seclabel,relatime 0 0 configfs /sys/kernel/config configfs rw,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=42,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=16075 0 0 mqueue /dev/mqueue mqueue rw,seclabel,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,relatime 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3261172k,mode=700,uid=1000,gid=1265 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw,seclabel 0 0 2023-11-24T07:58:57.405 DEBUG:teuthology.orchestra.run.smithi076:> set -ex 2023-11-24T07:58:57.406 DEBUG:teuthology.orchestra.run.smithi076:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-24T07:58:57.446 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=16284272k,nr_inodes=4071068,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 tmpfs /sys/fs/cgroup tmpfs ro,seclabel,nosuid,nodev,noexec,mode=755 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,seclabel,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,seclabel,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,seclabel,nosuid,nodev,noexec,relatime,freezer 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,seclabel,nosuid,nodev,noexec,relatime,perf_event 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,seclabel,nosuid,nodev,noexec,relatime,hugetlb 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,seclabel,nosuid,nodev,noexec,relatime,devices 0 0 cgroup /sys/fs/cgroup/rdma cgroup rw,seclabel,nosuid,nodev,noexec,relatime,rdma 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,seclabel,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,blkio 0 0 none /sys/kernel/tracing tracefs rw,seclabel,relatime 0 0 configfs /sys/kernel/config configfs rw,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=35,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=20116 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,relatime 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3260552k,mode=700,uid=1000,gid=1265 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw,seclabel 0 0 2023-11-24T07:58:57.446 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-24T07:58:57.575 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:57.573+0000 7fd1b62f8700 1 Processor -- start 2023-11-24T07:58:57.575 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:57.573+0000 7fd1b62f8700 1 -- start start 2023-11-24T07:58:57.578 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:57.574+0000 7fd1b62f8700 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fd1b015a440 0x7fd1b00cffa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:57.578 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:57.574+0000 7fd1b62f8700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fd1b015eb80 0x7fd1b00d04e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:57.578 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:57.574+0000 7fd1b62f8700 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd1b00b05f0 con 0x7fd1b015a820 2023-11-24T07:58:57.578 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:57.574+0000 7fd1b62f8700 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd1b015f080 con 0x7fd1b015eb80 2023-11-24T07:58:57.578 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:57.574+0000 7fd1b62f8700 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd1b00afd50 con 0x7fd1b015a440 2023-11-24T07:58:57.578 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:57.574+0000 7fd1b4895700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fd1b015eb80 0x7fd1b00d04e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:57.578 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:57.574+0000 7fd1b4895700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fd1b015eb80 0x7fd1b00d04e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:54280/0 (socket says 172.21.15.59:54280) 2023-11-24T07:58:57.578 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:57.574+0000 7fd1b4895700 1 -- 172.21.15.59:0/235560047 learned_addr learned my addr 172.21.15.59:0/235560047 (peer_addr_for_me v2:172.21.15.59:0/0) 2023-11-24T07:58:57.579 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:57.574+0000 7fd1affff700 1 --2- 172.21.15.59:0/235560047 >> v2:172.21.15.76:3300/0 conn(0x7fd1b015a440 0x7fd1b00cffa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:57.579 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:57.574+0000 7fd1b4895700 1 -- 172.21.15.59:0/235560047 >> v2:172.21.15.76:3300/0 conn(0x7fd1b015a440 msgr2=0x7fd1b00cffa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T07:58:57.579 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:57.574+0000 7fd1b4895700 1 --2- 172.21.15.59:0/235560047 >> v2:172.21.15.76:3300/0 conn(0x7fd1b015a440 0x7fd1b00cffa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:57.579 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:57.574+0000 7fd1b4895700 1 -- 172.21.15.59:0/235560047 >> v1:172.21.15.76:6789/0 conn(0x7fd1b015a820 legacy=0x7fd1b00726a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:57.579 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:57.574+0000 7fd1b4895700 1 -- 172.21.15.59:0/235560047 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd1b00b0030 con 0x7fd1b015eb80 2023-11-24T07:58:57.579 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:57.575+0000 7fd1b4895700 1 --2- 172.21.15.59:0/235560047 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd1b015eb80 0x7fd1b00d04e0 secure :-1 s=READY pgs=776 cs=0 l=1 rev1=1 crypto rx=0x7fd1b0069520 tx=0x7fd1a800d430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=10a00d13b4f514d7 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:58:57.579 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:57.575+0000 7fd1aeffd700 1 -- 172.21.15.59:0/235560047 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1a8010070 con 0x7fd1b015eb80 2023-11-24T07:58:57.579 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:57.575+0000 7fd1aeffd700 1 -- 172.21.15.59:0/235560047 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd1a800dad0 con 0x7fd1b015eb80 2023-11-24T07:58:57.579 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:57.575+0000 7fd1aeffd700 1 -- 172.21.15.59:0/235560047 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1a800de80 con 0x7fd1b015eb80 2023-11-24T07:58:57.580 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:57.575+0000 7fd1b62f8700 1 -- 172.21.15.59:0/235560047 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd1b015eb80 msgr2=0x7fd1b00d04e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:57.580 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:57.575+0000 7fd1b62f8700 1 --2- 172.21.15.59:0/235560047 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd1b015eb80 0x7fd1b00d04e0 secure :-1 s=READY pgs=776 cs=0 l=1 rev1=1 crypto rx=0x7fd1b0069520 tx=0x7fd1a800d430 comp rx=0 tx=0).stop 2023-11-24T07:58:57.580 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:57.575+0000 7fd1b62f8700 1 -- 172.21.15.59:0/235560047 shutdown_connections 2023-11-24T07:58:57.580 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:57.575+0000 7fd1b62f8700 1 --2- 172.21.15.59:0/235560047 >> v2:172.21.15.76:3300/0 conn(0x7fd1b015a440 0x7fd1b00cffa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:57.580 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:57.575+0000 7fd1b62f8700 1 --2- 172.21.15.59:0/235560047 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd1b015eb80 0x7fd1b00d04e0 unknown :-1 s=CLOSED pgs=776 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:57.580 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:57.575+0000 7fd1b62f8700 1 -- 172.21.15.59:0/235560047 >> 172.21.15.59:0/235560047 conn(0x7fd1b00b5070 msgr2=0x7fd1b00b5730 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:58:57.580 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:57.575+0000 7fd1b62f8700 1 -- 172.21.15.59:0/235560047 shutdown_connections 2023-11-24T07:58:57.580 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:57.575+0000 7fd1b62f8700 1 -- 172.21.15.59:0/235560047 wait complete. 2023-11-24T07:58:57.583 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:57.577+0000 7fd1b62f8700 1 Processor -- start 2023-11-24T07:58:57.583 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:57.577+0000 7fd1b62f8700 1 -- start start 2023-11-24T07:58:57.583 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:57.578+0000 7fd1b62f8700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd1b015a440 0x7fd1b01efa50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:57.584 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:57.578+0000 7fd1b62f8700 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fd1b015eb80 0x7fd1b01f2760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:57.584 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:57.578+0000 7fd1b62f8700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd1b006b760 0x7fd1b006bc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:57.584 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:57.578+0000 7fd1b62f8700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1b006c150 con 0x7fd1b015a440 2023-11-24T07:58:57.584 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:57.578+0000 7fd1b62f8700 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1b00cdc20 con 0x7fd1b006b760 2023-11-24T07:58:57.584 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:57.578+0000 7fd1b62f8700 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd1b01e3ed0 con 0x7fd1b015eb80 2023-11-24T07:58:57.584 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:57.578+0000 7fd1affff700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd1b015a440 0x7fd1b01efa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:57.585 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:57.578+0000 7fd1affff700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd1b015a440 0x7fd1b01efa50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:54282/0 (socket says 172.21.15.59:54282) 2023-11-24T07:58:57.585 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:57.578+0000 7fd1affff700 1 -- 172.21.15.59:0/3074521333 learned_addr learned my addr 172.21.15.59:0/3074521333 (peer_addr_for_me v2:172.21.15.59:0/0) 2023-11-24T07:58:57.585 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:57.578+0000 7fd1b4895700 1 --2- 172.21.15.59:0/3074521333 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd1b006b760 0x7fd1b006bc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:57.585 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:57.578+0000 7fd1af7fe700 1 --2- 172.21.15.59:0/3074521333 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fd1b015eb80 0x7fd1b01f2760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:57.585 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:57.578+0000 7fd1affff700 1 -- 172.21.15.59:0/3074521333 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fd1b015eb80 msgr2=0x7fd1b01f2760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:57.585 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:57.578+0000 7fd1affff700 1 --2- 172.21.15.59:0/3074521333 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fd1b015eb80 0x7fd1b01f2760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:57.586 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:57.578+0000 7fd1affff700 1 -- 172.21.15.59:0/3074521333 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd1b006b760 msgr2=0x7fd1b006bc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:57.586 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:57.578+0000 7fd1affff700 1 --2- 172.21.15.59:0/3074521333 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd1b006b760 0x7fd1b006bc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:57.586 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:57.578+0000 7fd1affff700 1 -- 172.21.15.59:0/3074521333 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd1a800bd40 con 0x7fd1b015a440 2023-11-24T07:58:57.586 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:57.578+0000 7fd1b4895700 1 --2- 172.21.15.59:0/3074521333 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd1b006b760 0x7fd1b006bc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-24T07:58:57.586 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:57.579+0000 7fd1affff700 1 --2- 172.21.15.59:0/3074521333 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd1b015a440 0x7fd1b01efa50 secure :-1 s=READY pgs=777 cs=0 l=1 rev1=1 crypto rx=0x7fd1a000e9b0 tx=0x7fd1a000ed70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:58:57.586 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:57.579+0000 7fd1acff9700 1 -- 172.21.15.59:0/3074521333 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1a0009710 con 0x7fd1b015a440 2023-11-24T07:58:57.587 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:57.579+0000 7fd1acff9700 1 -- 172.21.15.59:0/3074521333 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd1a000c850 con 0x7fd1b015a440 2023-11-24T07:58:57.587 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:57.579+0000 7fd1b62f8700 1 -- 172.21.15.59:0/3074521333 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd1b01e4730 con 0x7fd1b015a440 2023-11-24T07:58:57.587 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:57.579+0000 7fd1acff9700 1 -- 172.21.15.59:0/3074521333 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1a000cbc0 con 0x7fd1b015a440 2023-11-24T07:58:57.587 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:57.579+0000 7fd1b62f8700 1 -- 172.21.15.59:0/3074521333 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd1b01e40e0 con 0x7fd1b015a440 2023-11-24T07:58:57.587 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:57.580+0000 7fd1b62f8700 1 -- 172.21.15.59:0/3074521333 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd1b0068670 con 0x7fd1b015a440 2023-11-24T07:58:57.587 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:57.581+0000 7fd1acff9700 1 -- 172.21.15.59:0/3074521333 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 10) v1 ==== 89588+0+0 (secure 0 0 0) 0x7fd1a000cd20 con 0x7fd1b015a440 2023-11-24T07:58:57.588 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:57.581+0000 7fd1acff9700 1 --2- 172.21.15.59:0/3074521333 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7fd19c06b2c0 0x7fd19c06d780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:57.588 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:57.581+0000 7fd1acff9700 1 -- 172.21.15.59:0/3074521333 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(61..61 src has 1..61) v4 ==== 5765+0+0 (secure 0 0 0) 0x7fd1a0089eb0 con 0x7fd1b015a440 2023-11-24T07:58:57.588 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:57.581+0000 7fd1af7fe700 1 --2- 172.21.15.59:0/3074521333 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7fd19c06b2c0 0x7fd19c06d780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:57.588 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:57.582+0000 7fd1af7fe700 1 --2- 172.21.15.59:0/3074521333 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7fd19c06b2c0 0x7fd19c06d780 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7fd1b015eb30 tx=0x7fd1a4009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:58:57.588 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:57.585+0000 7fd1acff9700 1 -- 172.21.15.59:0/3074521333 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fd1a0054940 con 0x7fd1b015a440 2023-11-24T07:58:57.833 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:57.831+0000 7fd1b62f8700 1 -- 172.21.15.59:0/3074521333 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fd1b00b2de0 con 0x7fd1b015a440 2023-11-24T07:58:57.834 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:57.832+0000 7fd1acff9700 1 -- 172.21.15.59:0/3074521333 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 36 v36) v1 ==== 94+0+4907 (secure 0 0 0) 0x7fd1a0021090 con 0x7fd1b015a440 2023-11-24T07:58:57.834 INFO:teuthology.orchestra.run.smithi059.stdout: 2023-11-24T07:58:57.834 INFO:teuthology.orchestra.run.smithi059.stdout:{"epoch":36,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5300,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6834/172682993","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6833","nonce":172682993},{"type":"v1","addr":"172.21.15.59:6834","nonce":172682993}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":29},{"gid":5701,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.76:6836/923821877","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6834","nonce":923821877},{"type":"v1","addr":"172.21.15.76:6836","nonce":923821877}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":30},{"gid":5704,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.76:6837/4166482716","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6835","nonce":4166482716},{"type":"v1","addr":"172.21.15.76:6837","nonce":4166482716}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":29}],"filesystems":[{"mdsmap":{"epoch":35,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-24T07:58:40.795340+0000","modified":"2023-11-24T07:58:43.812382+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":6093},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6093":{"gid":6093,"name":"a","rank":0,"incarnation":31,"state":"up:active","state_seq":3,"addr":"172.21.15.59:6835/1599845006","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6832","nonce":1599845006},{"type":"v1","addr":"172.21.15.59:6835","nonce":1599845006}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[9],"metadata_pool":8,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":4}]} 2023-11-24T07:58:57.836 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:57.834+0000 7fd1b62f8700 1 -- 172.21.15.59:0/3074521333 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7fd19c06b2c0 msgr2=0x7fd19c06d780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:57.836 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:57.834+0000 7fd1b62f8700 1 --2- 172.21.15.59:0/3074521333 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7fd19c06b2c0 0x7fd19c06d780 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7fd1b015eb30 tx=0x7fd1a4009040 comp rx=0 tx=0).stop 2023-11-24T07:58:57.836 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:57.834+0000 7fd1b62f8700 1 -- 172.21.15.59:0/3074521333 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd1b015a440 msgr2=0x7fd1b01efa50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:57.836 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:57.834+0000 7fd1b62f8700 1 --2- 172.21.15.59:0/3074521333 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd1b015a440 0x7fd1b01efa50 secure :-1 s=READY pgs=777 cs=0 l=1 rev1=1 crypto rx=0x7fd1a000e9b0 tx=0x7fd1a000ed70 comp rx=0 tx=0).stop 2023-11-24T07:58:57.836 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:57.834+0000 7fd1b62f8700 1 -- 172.21.15.59:0/3074521333 shutdown_connections 2023-11-24T07:58:57.836 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:57.834+0000 7fd1b62f8700 1 --2- 172.21.15.59:0/3074521333 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7fd19c06b2c0 0x7fd19c06d780 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:57.837 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:57.834+0000 7fd1b62f8700 1 --2- 172.21.15.59:0/3074521333 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd1b006b760 0x7fd1b006bc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:57.837 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:57.834+0000 7fd1b62f8700 1 --2- 172.21.15.59:0/3074521333 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd1b015a440 0x7fd1b01efa50 unknown :-1 s=CLOSED pgs=777 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:57.837 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:57.834+0000 7fd1b62f8700 1 --2- 172.21.15.59:0/3074521333 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fd1b015eb80 0x7fd1b01f2760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:57.837 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:57.834+0000 7fd1b62f8700 1 -- 172.21.15.59:0/3074521333 >> 172.21.15.59:0/3074521333 conn(0x7fd1b00b5070 msgr2=0x7fd1b00b1ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:58:57.837 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:57.834+0000 7fd1b62f8700 1 -- 172.21.15.59:0/3074521333 shutdown_connections 2023-11-24T07:58:57.837 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:57.834+0000 7fd1b62f8700 1 -- 172.21.15.59:0/3074521333 wait complete. 2023-11-24T07:58:57.837 INFO:teuthology.orchestra.run.smithi059.stderr:dumped fsmap epoch 36 2023-11-24T07:58:57.848 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-24T07:58:57.975 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:57.974+0000 7f4930da2700 1 Processor -- start 2023-11-24T07:58:57.976 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:57.974+0000 7f4930da2700 1 -- start start 2023-11-24T07:58:57.976 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:57.974+0000 7f4930da2700 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f492c15bd90 0x7f492c0723d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:57.976 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:57.975+0000 7f4930da2700 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f492c06b6e0 0x7f492c0cd390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:57.977 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:57.975+0000 7f4930da2700 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f492c071930 con 0x7f492c072910 2023-11-24T07:58:57.977 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:57.975+0000 7f4930da2700 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f492c0cdff0 con 0x7f492c06b6e0 2023-11-24T07:58:57.977 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:57.975+0000 7f4930da2700 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f492c071bd0 con 0x7f492c15bd90 2023-11-24T07:58:57.977 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:57.975+0000 7f492bfff700 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f492c06b6e0 0x7f492c0cd390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:57.977 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:57.975+0000 7f492b7fe700 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f492c15bd90 0x7f492c0723d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:57.978 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:57.975+0000 7f492bfff700 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f492c06b6e0 0x7f492c0cd390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.59:41492/0 (socket says 172.21.15.59:41492) 2023-11-24T07:58:57.978 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:57.975+0000 7f492bfff700 1 -- 172.21.15.59:0/2737140709 learned_addr learned my addr 172.21.15.59:0/2737140709 (peer_addr_for_me v2:172.21.15.59:0/0) 2023-11-24T07:58:57.979 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:57.975+0000 7f492b7fe700 1 -- 172.21.15.59:0/2737140709 >> v2:172.21.15.76:3300/0 conn(0x7f492c06b6e0 msgr2=0x7f492c0cd390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T07:58:57.979 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:57.975+0000 7f492b7fe700 1 --2- 172.21.15.59:0/2737140709 >> v2:172.21.15.76:3300/0 conn(0x7f492c06b6e0 0x7f492c0cd390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:57.979 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:57.975+0000 7f492b7fe700 1 -- 172.21.15.59:0/2737140709 >> v1:172.21.15.76:6789/0 conn(0x7f492c072910 legacy=0x7f492c06afd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:57.980 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:57.975+0000 7f492b7fe700 1 -- 172.21.15.59:0/2737140709 --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f492c071eb0 con 0x7f492c15bd90 2023-11-24T07:58:57.980 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:57.975+0000 7f492bfff700 1 --2- 172.21.15.59:0/2737140709 >> v2:172.21.15.76:3300/0 conn(0x7f492c06b6e0 0x7f492c0cd390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-24T07:58:57.980 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:57.975+0000 7f492b7fe700 1 --2- 172.21.15.59:0/2737140709 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f492c15bd90 0x7f492c0723d0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f491c009840 tx=0x7f491c009b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b2f12666a05e51de server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:58:57.980 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:57.976+0000 7f492a7fc700 1 -- 172.21.15.59:0/2737140709 <== mon.2 v2:172.21.15.76:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f491c011830 con 0x7f492c15bd90 2023-11-24T07:58:57.980 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:57.976+0000 7f492a7fc700 1 -- 172.21.15.59:0/2737140709 <== mon.2 v2:172.21.15.76:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f491c011990 con 0x7f492c15bd90 2023-11-24T07:58:57.980 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:57.976+0000 7f492a7fc700 1 -- 172.21.15.59:0/2737140709 <== mon.2 v2:172.21.15.76:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f491c011d20 con 0x7f492c15bd90 2023-11-24T07:58:57.980 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:57.976+0000 7f4930da2700 1 -- 172.21.15.59:0/2737140709 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f492c15bd90 msgr2=0x7f492c0723d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:57.980 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:57.976+0000 7f4930da2700 1 --2- 172.21.15.59:0/2737140709 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f492c15bd90 0x7f492c0723d0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f491c009840 tx=0x7f491c009b50 comp rx=0 tx=0).stop 2023-11-24T07:58:57.980 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:57.976+0000 7f4930da2700 1 -- 172.21.15.59:0/2737140709 shutdown_connections 2023-11-24T07:58:57.980 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:57.976+0000 7f4930da2700 1 --2- 172.21.15.59:0/2737140709 >> v2:172.21.15.76:3300/0 conn(0x7f492c06b6e0 0x7f492c0cd390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:57.980 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:57.976+0000 7f4930da2700 1 --2- 172.21.15.59:0/2737140709 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f492c15bd90 0x7f492c0723d0 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:57.980 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:57.976+0000 7f4930da2700 1 -- 172.21.15.59:0/2737140709 >> 172.21.15.59:0/2737140709 conn(0x7f492c0d09c0 msgr2=0x7f492c0d0dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:58:57.981 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:57.976+0000 7f4930da2700 1 -- 172.21.15.59:0/2737140709 shutdown_connections 2023-11-24T07:58:57.981 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:57.976+0000 7f4930da2700 1 -- 172.21.15.59:0/2737140709 wait complete. 2023-11-24T07:58:57.981 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:57.977+0000 7f4930da2700 1 Processor -- start 2023-11-24T07:58:57.981 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:57.978+0000 7f4930da2700 1 -- start start 2023-11-24T07:58:57.984 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:57.978+0000 7f4930da2700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f492c06b6e0 0x7f492c0cc3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:57.984 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:57.978+0000 7f4930da2700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f492c072910 0x7f492c15a370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:57.984 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:57.978+0000 7f4930da2700 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f492c15bd90 0x7f492c15a8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:57.984 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:57.978+0000 7f4930da2700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f492c141f50 con 0x7f492c06b6e0 2023-11-24T07:58:57.984 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:57.978+0000 7f4930da2700 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f492c142230 con 0x7f492c072910 2023-11-24T07:58:57.984 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:57.978+0000 7f4930da2700 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f492c142510 con 0x7f492c15bd90 2023-11-24T07:58:57.984 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:57.979+0000 7f492b7fe700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f492c06b6e0 0x7f492c0cc3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:57.984 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:57.979+0000 7f492b7fe700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f492c06b6e0 0x7f492c0cc3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:54296/0 (socket says 172.21.15.59:54296) 2023-11-24T07:58:57.984 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:57.979+0000 7f492b7fe700 1 -- 172.21.15.59:0/2732123417 learned_addr learned my addr 172.21.15.59:0/2732123417 (peer_addr_for_me v2:172.21.15.59:0/0) 2023-11-24T07:58:57.985 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:57.979+0000 7f492affd700 1 --2- 172.21.15.59:0/2732123417 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f492c072910 0x7f492c15a370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:57.985 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:57.979+0000 7f492bfff700 1 --2- 172.21.15.59:0/2732123417 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f492c15bd90 0x7f492c15a8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:57.985 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:57.979+0000 7f492b7fe700 1 -- 172.21.15.59:0/2732123417 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f492c15bd90 msgr2=0x7f492c15a8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:57.985 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:57.979+0000 7f492b7fe700 1 --2- 172.21.15.59:0/2732123417 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f492c15bd90 0x7f492c15a8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:57.985 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:57.979+0000 7f492b7fe700 1 -- 172.21.15.59:0/2732123417 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f492c072910 msgr2=0x7f492c15a370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:57.985 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:57.979+0000 7f492b7fe700 1 --2- 172.21.15.59:0/2732123417 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f492c072910 0x7f492c15a370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:57.985 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:57.979+0000 7f492b7fe700 1 -- 172.21.15.59:0/2732123417 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4924009b30 con 0x7f492c06b6e0 2023-11-24T07:58:57.985 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:57.979+0000 7f492b7fe700 1 --2- 172.21.15.59:0/2732123417 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f492c06b6e0 0x7f492c0cc3f0 secure :-1 s=READY pgs=778 cs=0 l=1 rev1=1 crypto rx=0x7f491c009fd0 tx=0x7f491c004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:58:57.985 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:57.979+0000 7f4913fff700 1 -- 172.21.15.59:0/2732123417 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f491c0036a0 con 0x7f492c06b6e0 2023-11-24T07:58:57.985 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:57.979+0000 7f4913fff700 1 -- 172.21.15.59:0/2732123417 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f491c01e070 con 0x7f492c06b6e0 2023-11-24T07:58:57.985 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:57.979+0000 7f4930da2700 1 -- 172.21.15.59:0/2732123417 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f491c0094a0 con 0x7f492c06b6e0 2023-11-24T07:58:57.985 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:57.979+0000 7f4913fff700 1 -- 172.21.15.59:0/2732123417 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f491c003a60 con 0x7f492c06b6e0 2023-11-24T07:58:57.986 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:57.979+0000 7f4930da2700 1 -- 172.21.15.59:0/2732123417 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f492c142ad0 con 0x7f492c06b6e0 2023-11-24T07:58:57.986 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:57.980+0000 7f4930da2700 1 -- 172.21.15.59:0/2732123417 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f492c142c60 con 0x7f492c06b6e0 2023-11-24T07:58:57.986 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:57.981+0000 7f4913fff700 1 -- 172.21.15.59:0/2732123417 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 10) v1 ==== 89588+0+0 (secure 0 0 0) 0x7f491c003bc0 con 0x7f492c06b6e0 2023-11-24T07:58:57.986 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:57.981+0000 7f4913fff700 1 --2- 172.21.15.59:0/2732123417 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f491806b2c0 0x7f491806d780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:57.986 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:57.981+0000 7f4913fff700 1 -- 172.21.15.59:0/2732123417 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(61..61 src has 1..61) v4 ==== 5765+0+0 (secure 0 0 0) 0x7f491c08abd0 con 0x7f492c06b6e0 2023-11-24T07:58:57.986 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:57.981+0000 7f492affd700 1 --2- 172.21.15.59:0/2732123417 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f491806b2c0 0x7f491806d780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:57.986 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:57.982+0000 7f492affd700 1 --2- 172.21.15.59:0/2732123417 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f491806b2c0 0x7f491806d780 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f492c0aab80 tx=0x7f4920009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:58:57.987 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:57.986+0000 7f4913fff700 1 -- 172.21.15.59:0/2732123417 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f491c055800 con 0x7f492c06b6e0 2023-11-24T07:58:58.197 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:58.195+0000 7f4930da2700 1 -- 172.21.15.59:0/2732123417 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f492c075300 con 0x7f492c06b6e0 2023-11-24T07:58:58.198 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:58.197+0000 7f4913fff700 1 -- 172.21.15.59:0/2732123417 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v61) v1 ==== 74+0+16601 (secure 0 0 0) 0x7f491c015090 con 0x7f492c06b6e0 2023-11-24T07:58:58.198 INFO:teuthology.orchestra.run.smithi059.stdout: 2023-11-24T07:58:58.199 INFO:teuthology.orchestra.run.smithi059.stdout:{"epoch":61,"fsid":"9f0e8741-b8f9-48d5-a2c3-09f45903303d","created":"2023-11-24T07:56:06.552729+0000","modified":"2023-11-24T07:58:57.458332+0000","last_up_change":"2023-11-24T07:56:17.372352+0000","last_in_change":"2023-11-24T07:56:09.073250+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-24T07:56:18.352823+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_metadata","create_time":"2023-11-24T07:58:39.262463+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"59","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_data","create_time":"2023-11-24T07:58:39.763399+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"61","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":3,"snap_epoch":61,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"6d8f9e9b-1782-4e41-928b-a39f2a8ebc06","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6800","nonce":3373878772},{"type":"v1","addr":"172.21.15.59:6801","nonce":3373878772}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6802","nonce":3373878772},{"type":"v1","addr":"172.21.15.59:6803","nonce":3373878772}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6806","nonce":3373878772},{"type":"v1","addr":"172.21.15.59:6807","nonce":3373878772}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6804","nonce":3373878772},{"type":"v1","addr":"172.21.15.59:6805","nonce":3373878772}]},"public_addr":"172.21.15.59:6801/3373878772","cluster_addr":"172.21.15.59:6803/3373878772","heartbeat_back_addr":"172.21.15.59:6807/3373878772","heartbeat_front_addr":"172.21.15.59:6805/3373878772","state":["exists","up"]},{"osd":1,"uuid":"e8d882aa-52e1-4c04-9e1e-c44ef9204a5f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6824","nonce":648258563},{"type":"v1","addr":"172.21.15.59:6825","nonce":648258563}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6826","nonce":648258563},{"type":"v1","addr":"172.21.15.59:6827","nonce":648258563}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6830","nonce":648258563},{"type":"v1","addr":"172.21.15.59:6831","nonce":648258563}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6828","nonce":648258563},{"type":"v1","addr":"172.21.15.59:6829","nonce":648258563}]},"public_addr":"172.21.15.59:6825/648258563","cluster_addr":"172.21.15.59:6827/648258563","heartbeat_back_addr":"172.21.15.59:6831/648258563","heartbeat_front_addr":"172.21.15.59:6829/648258563","state":["exists","up"]},{"osd":2,"uuid":"69eebfa0-695b-4b21-90b9-8a3537168a01","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6808","nonce":1840430654},{"type":"v1","addr":"172.21.15.59:6809","nonce":1840430654}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6810","nonce":1840430654},{"type":"v1","addr":"172.21.15.59:6811","nonce":1840430654}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6814","nonce":1840430654},{"type":"v1","addr":"172.21.15.59:6815","nonce":1840430654}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6812","nonce":1840430654},{"type":"v1","addr":"172.21.15.59:6813","nonce":1840430654}]},"public_addr":"172.21.15.59:6809/1840430654","cluster_addr":"172.21.15.59:6811/1840430654","heartbeat_back_addr":"172.21.15.59:6815/1840430654","heartbeat_front_addr":"172.21.15.59:6813/1840430654","state":["exists","up"]},{"osd":3,"uuid":"96fd77fc-f911-46ff-9445-6f0be2daa8f4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6816","nonce":2504025850},{"type":"v1","addr":"172.21.15.59:6817","nonce":2504025850}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6818","nonce":2504025850},{"type":"v1","addr":"172.21.15.59:6819","nonce":2504025850}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6822","nonce":2504025850},{"type":"v1","addr":"172.21.15.59:6823","nonce":2504025850}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6820","nonce":2504025850},{"type":"v1","addr":"172.21.15.59:6821","nonce":2504025850}]},"public_addr":"172.21.15.59:6817/2504025850","cluster_addr":"172.21.15.59:6819/2504025850","heartbeat_back_addr":"172.21.15.59:6823/2504025850","heartbeat_front_addr":"172.21.15.59:6821/2504025850","state":["exists","up"]},{"osd":4,"uuid":"7ac29fca-b5b2-4c95-9d69-2f164cbbc708","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6816","nonce":2382463431},{"type":"v1","addr":"172.21.15.76:6818","nonce":2382463431}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6820","nonce":2382463431},{"type":"v1","addr":"172.21.15.76:6822","nonce":2382463431}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6828","nonce":2382463431},{"type":"v1","addr":"172.21.15.76:6830","nonce":2382463431}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6824","nonce":2382463431},{"type":"v1","addr":"172.21.15.76:6826","nonce":2382463431}]},"public_addr":"172.21.15.76:6818/2382463431","cluster_addr":"172.21.15.76:6822/2382463431","heartbeat_back_addr":"172.21.15.76:6830/2382463431","heartbeat_front_addr":"172.21.15.76:6826/2382463431","state":["exists","up"]},{"osd":5,"uuid":"485bf711-3851-4238-a4f4-51d52e0a9c58","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6800","nonce":329334308},{"type":"v1","addr":"172.21.15.76:6802","nonce":329334308}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6805","nonce":329334308},{"type":"v1","addr":"172.21.15.76:6807","nonce":329334308}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6814","nonce":329334308},{"type":"v1","addr":"172.21.15.76:6815","nonce":329334308}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6810","nonce":329334308},{"type":"v1","addr":"172.21.15.76:6812","nonce":329334308}]},"public_addr":"172.21.15.76:6802/329334308","cluster_addr":"172.21.15.76:6807/329334308","heartbeat_back_addr":"172.21.15.76:6815/329334308","heartbeat_front_addr":"172.21.15.76:6812/329334308","state":["exists","up"]},{"osd":6,"uuid":"72b3f2fc-1e4a-4e32-968d-003647a0286f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6801","nonce":3946579410},{"type":"v1","addr":"172.21.15.76:6803","nonce":3946579410}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6804","nonce":3946579410},{"type":"v1","addr":"172.21.15.76:6806","nonce":3946579410}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6811","nonce":3946579410},{"type":"v1","addr":"172.21.15.76:6813","nonce":3946579410}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6808","nonce":3946579410},{"type":"v1","addr":"172.21.15.76:6809","nonce":3946579410}]},"public_addr":"172.21.15.76:6803/3946579410","cluster_addr":"172.21.15.76:6806/3946579410","heartbeat_back_addr":"172.21.15.76:6813/3946579410","heartbeat_front_addr":"172.21.15.76:6809/3946579410","state":["exists","up"]},{"osd":7,"uuid":"1ddceb90-6c5d-4d1a-ad56-99cb877f8c70","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6817","nonce":841761538},{"type":"v1","addr":"172.21.15.76:6819","nonce":841761538}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6821","nonce":841761538},{"type":"v1","addr":"172.21.15.76:6823","nonce":841761538}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6829","nonce":841761538},{"type":"v1","addr":"172.21.15.76:6831","nonce":841761538}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6825","nonce":841761538},{"type":"v1","addr":"172.21.15.76:6827","nonce":841761538}]},"public_addr":"172.21.15.76:6819/841761538","cluster_addr":"172.21.15.76:6823/841761538","heartbeat_back_addr":"172.21.15.76:6831/841761538","heartbeat_front_addr":"172.21.15.76:6827/841761538","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T07:56:14.294838+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T07:56:14.270779+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T07:56:14.231809+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T07:56:14.260983+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T07:56:14.302997+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T07:56:14.258600+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T07:56:14.251413+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T07:56:14.304727+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":9,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[{"pool":9,"snaps":[{"begin":2,"length":2}]}],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-24T07:58:58.201 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:58.199+0000 7f4930da2700 1 -- 172.21.15.59:0/2732123417 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f491806b2c0 msgr2=0x7f491806d780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:58.201 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:58.199+0000 7f4930da2700 1 --2- 172.21.15.59:0/2732123417 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f491806b2c0 0x7f491806d780 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f492c0aab80 tx=0x7f4920009040 comp rx=0 tx=0).stop 2023-11-24T07:58:58.201 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:58.199+0000 7f4930da2700 1 -- 172.21.15.59:0/2732123417 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f492c06b6e0 msgr2=0x7f492c0cc3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:58.201 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:58.199+0000 7f4930da2700 1 --2- 172.21.15.59:0/2732123417 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f492c06b6e0 0x7f492c0cc3f0 secure :-1 s=READY pgs=778 cs=0 l=1 rev1=1 crypto rx=0x7f491c009fd0 tx=0x7f491c004750 comp rx=0 tx=0).stop 2023-11-24T07:58:58.201 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:58.199+0000 7f4930da2700 1 -- 172.21.15.59:0/2732123417 shutdown_connections 2023-11-24T07:58:58.201 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:58.199+0000 7f4930da2700 1 --2- 172.21.15.59:0/2732123417 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f491806b2c0 0x7f491806d780 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:58.201 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:58.199+0000 7f4930da2700 1 --2- 172.21.15.59:0/2732123417 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f492c15bd90 0x7f492c15a8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:58.201 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:58.199+0000 7f4930da2700 1 --2- 172.21.15.59:0/2732123417 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f492c06b6e0 0x7f492c0cc3f0 unknown :-1 s=CLOSED pgs=778 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:58.202 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:58.199+0000 7f4930da2700 1 --2- 172.21.15.59:0/2732123417 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f492c072910 0x7f492c15a370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:58.202 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:58.199+0000 7f4930da2700 1 -- 172.21.15.59:0/2732123417 >> 172.21.15.59:0/2732123417 conn(0x7f492c0d09c0 msgr2=0x7f492c0d0da0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:58:58.202 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:58.199+0000 7f4930da2700 1 -- 172.21.15.59:0/2732123417 shutdown_connections 2023-11-24T07:58:58.202 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:58.199+0000 7f4930da2700 1 -- 172.21.15.59:0/2732123417 wait complete. 2023-11-24T07:58:58.213 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-24T07:56:18.352823+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 8, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-24T07:58:39.262463+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '59', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 9, 'pool_name': 'cephfs_data', 'create_time': '2023-11-24T07:58:39.763399+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '61', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 3, 'snap_epoch': 61, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-24T07:58:58.213 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-24T07:58:58.341 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:58.339+0000 7f39f6bee700 1 Processor -- start 2023-11-24T07:58:58.341 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:58.339+0000 7f39f6bee700 1 -- start start 2023-11-24T07:58:58.344 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:58.340+0000 7f39f6bee700 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f39f01440f0 0x7f39f00b51b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:58.344 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:58.340+0000 7f39f6bee700 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f39f0143ec0 con 0x7f39f00722c0 2023-11-24T07:58:58.344 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:58.340+0000 7f39f6bee700 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f39f00b3ae0 con 0x7f39f0143ae0 2023-11-24T07:58:58.344 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:58.340+0000 7f39f6bee700 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f39f00b40a0 con 0x7f39f01440f0 2023-11-24T07:58:58.344 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:58.340+0000 7f39effff700 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7f39f00722c0 0x7f39f006e510 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:57780/0 (socket says 172.21.15.59:57780) 2023-11-24T07:58:58.345 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:58.340+0000 7f39effff700 1 -- 172.21.15.59:0/3082809715 learned_addr learned my addr 172.21.15.59:0/3082809715 (peer_addr_for_me v1:172.21.15.59:0/0) 2023-11-24T07:58:58.345 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:58.340+0000 7f39ef7fe700 1 -- 172.21.15.59:0/3082809715 <== mon.0 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2497765711 0 0) 0x7f39f0143ec0 con 0x7f39f00722c0 2023-11-24T07:58:58.345 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:58.340+0000 7f39f518b700 1 --2- 172.21.15.59:0/3082809715 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f39f01440f0 0x7f39f00b51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:58.345 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:58.341+0000 7f39ef7fe700 1 -- 172.21.15.59:0/3082809715 --> v1:172.21.15.59:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f39e00035b0 con 0x7f39f00722c0 2023-11-24T07:58:58.345 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:58.341+0000 7f39ef7fe700 1 -- 172.21.15.59:0/3082809715 <== mon.0 v1:172.21.15.59:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 395632040 0 0) 0x7f39e00035b0 con 0x7f39f00722c0 2023-11-24T07:58:58.345 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:58.341+0000 7f39ef7fe700 1 -- 172.21.15.59:0/3082809715 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f39f01440f0 msgr2=0x7f39f00b51b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T07:58:58.345 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:58.341+0000 7f39ef7fe700 1 --2- 172.21.15.59:0/3082809715 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f39f01440f0 0x7f39f00b51b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:58.345 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:58.341+0000 7f39ef7fe700 1 -- 172.21.15.59:0/3082809715 >> v1:172.21.15.76:6789/0 conn(0x7f39f0143ae0 legacy=0x7f39f0071bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:58.345 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:58.341+0000 7f39ef7fe700 1 -- 172.21.15.59:0/3082809715 --> v1:172.21.15.59:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f39f00b3dc0 con 0x7f39f00722c0 2023-11-24T07:58:58.345 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:58.341+0000 7f39ef7fe700 1 -- 172.21.15.59:0/3082809715 <== mon.0 v1:172.21.15.59:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1312333566 0 0) 0x7f39dc001670 con 0x7f39f00722c0 2023-11-24T07:58:58.345 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:58.341+0000 7f39ef7fe700 1 -- 172.21.15.59:0/3082809715 >> v1:172.21.15.59:6789/0 conn(0x7f39f00722c0 legacy=0x7f39f006e510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:58.345 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:58.341+0000 7f39ef7fe700 1 --2- 172.21.15.59:0/3082809715 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f39f01440f0 0x7f39e0003a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:58.346 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:58.341+0000 7f39ef7fe700 1 --2- 172.21.15.59:0/3082809715 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f39e0003f70 0x7f39e0004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:58.346 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:58.341+0000 7f39ef7fe700 1 --2- 172.21.15.59:0/3082809715 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f39e0004d90 0x7f39e0005210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:58.346 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:58.341+0000 7f39ef7fe700 1 -- 172.21.15.59:0/3082809715 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39f00b40a0 con 0x7f39e0004d90 2023-11-24T07:58:58.346 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:58.341+0000 7f39ef7fe700 1 -- 172.21.15.59:0/3082809715 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39e000ba70 con 0x7f39e0003f70 2023-11-24T07:58:58.346 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:58.341+0000 7f39ef7fe700 1 -- 172.21.15.59:0/3082809715 --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f39e000bc30 con 0x7f39f01440f0 2023-11-24T07:58:58.346 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:58.341+0000 7f39f518b700 1 --2- 172.21.15.59:0/3082809715 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f39e0004d90 0x7f39e0005210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:58.346 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:58.341+0000 7f39f498a700 1 --2- 172.21.15.59:0/3082809715 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f39f01440f0 0x7f39e0003a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:58.346 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:58.341+0000 7f39effff700 1 --2- 172.21.15.59:0/3082809715 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f39e0003f70 0x7f39e0004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:58.346 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:58.341+0000 7f39f518b700 1 -- 172.21.15.59:0/3082809715 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f39f01440f0 msgr2=0x7f39e0003a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T07:58:58.346 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:58.341+0000 7f39f518b700 1 --2- 172.21.15.59:0/3082809715 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f39f01440f0 0x7f39e0003a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:58.346 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:58.341+0000 7f39f518b700 1 -- 172.21.15.59:0/3082809715 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f39e0003f70 msgr2=0x7f39e0004850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T07:58:58.347 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:58.341+0000 7f39f518b700 1 --2- 172.21.15.59:0/3082809715 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f39e0003f70 0x7f39e0004850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:58.347 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:58.341+0000 7f39f518b700 1 -- 172.21.15.59:0/3082809715 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f39e00035b0 con 0x7f39e0004d90 2023-11-24T07:58:58.347 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:58.341+0000 7f39f518b700 1 --2- 172.21.15.59:0/3082809715 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f39e0004d90 0x7f39e0005210 secure :-1 s=READY pgs=780 cs=0 l=1 rev1=1 crypto rx=0x7f39e8009fc0 tx=0x7f39e0005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=59a1c110ae8ee474 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:58:58.347 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:58.342+0000 7f39ef7fe700 1 -- 172.21.15.59:0/3082809715 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39e800ec30 con 0x7f39e0004d90 2023-11-24T07:58:58.347 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:58.342+0000 7f39ef7fe700 1 -- 172.21.15.59:0/3082809715 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f39e800f040 con 0x7f39e0004d90 2023-11-24T07:58:58.347 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:58.342+0000 7f39ef7fe700 1 -- 172.21.15.59:0/3082809715 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39e8012430 con 0x7f39e0004d90 2023-11-24T07:58:58.347 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:58.342+0000 7f39f6bee700 1 -- 172.21.15.59:0/3082809715 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f39e0004d90 msgr2=0x7f39e0005210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:58.347 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:58.342+0000 7f39f6bee700 1 --2- 172.21.15.59:0/3082809715 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f39e0004d90 0x7f39e0005210 secure :-1 s=READY pgs=780 cs=0 l=1 rev1=1 crypto rx=0x7f39e8009fc0 tx=0x7f39e0005880 comp rx=0 tx=0).stop 2023-11-24T07:58:58.347 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:58.342+0000 7f39f518b700 1 -- 172.21.15.59:0/3082809715 reap_dead start 2023-11-24T07:58:58.347 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:58.342+0000 7f39f6bee700 1 -- 172.21.15.59:0/3082809715 shutdown_connections 2023-11-24T07:58:58.347 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:58.342+0000 7f39f6bee700 1 -- 172.21.15.59:0/3082809715 >> 172.21.15.59:0/3082809715 conn(0x7f39f00b2740 msgr2=0x7f39f00b2b20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:58:58.347 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:58.342+0000 7f39f6bee700 1 -- 172.21.15.59:0/3082809715 shutdown_connections 2023-11-24T07:58:58.348 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:58.342+0000 7f39f6bee700 1 -- 172.21.15.59:0/3082809715 wait complete. 2023-11-24T07:58:58.350 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:58.343+0000 7f39f6bee700 1 Processor -- start 2023-11-24T07:58:58.350 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:58.343+0000 7f39f6bee700 1 -- start start 2023-11-24T07:58:58.350 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:58.344+0000 7f39f6bee700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f39f01ef3d0 0x7f39f015b2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:58.350 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:58.344+0000 7f39f6bee700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f39f01ef7b0 0x7f39f0150fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:58.350 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:58.344+0000 7f39f6bee700 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f39f00cf020 0x7f39f00cf4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:58.350 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:58.344+0000 7f39f6bee700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39f00cfa10 con 0x7f39f01ef3d0 2023-11-24T07:58:58.350 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:58.344+0000 7f39f6bee700 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39f01514e0 con 0x7f39f01ef7b0 2023-11-24T07:58:58.350 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:58.344+0000 7f39f6bee700 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f39f015c4e0 con 0x7f39f00cf020 2023-11-24T07:58:58.351 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:58.344+0000 7f39f498a700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f39f01ef3d0 0x7f39f015b2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:58.351 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:58.344+0000 7f39f498a700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f39f01ef3d0 0x7f39f015b2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:54316/0 (socket says 172.21.15.59:54316) 2023-11-24T07:58:58.351 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:58.344+0000 7f39f498a700 1 -- 172.21.15.59:0/1983654231 learned_addr learned my addr 172.21.15.59:0/1983654231 (peer_addr_for_me v2:172.21.15.59:0/0) 2023-11-24T07:58:58.351 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:58.345+0000 7f39f498a700 1 -- 172.21.15.59:0/1983654231 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f39f00cf020 msgr2=0x7f39f00cf4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:58.351 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:58.345+0000 7f39f498a700 1 --2- 172.21.15.59:0/1983654231 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f39f00cf020 0x7f39f00cf4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:58.351 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:58.345+0000 7f39f498a700 1 -- 172.21.15.59:0/1983654231 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f39f01ef7b0 msgr2=0x7f39f0150fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:58.351 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:58.345+0000 7f39f498a700 1 --2- 172.21.15.59:0/1983654231 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f39f01ef7b0 0x7f39f0150fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:58.351 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:58.345+0000 7f39f498a700 1 -- 172.21.15.59:0/1983654231 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f39f013f740 con 0x7f39f01ef3d0 2023-11-24T07:58:58.351 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:58.345+0000 7f39f498a700 1 --2- 172.21.15.59:0/1983654231 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f39f01ef3d0 0x7f39f015b2f0 secure :-1 s=READY pgs=781 cs=0 l=1 rev1=1 crypto rx=0x7f39e400f7d0 tx=0x7f39e400fb90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:58:58.351 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:58.345+0000 7f39ed7fa700 1 -- 172.21.15.59:0/1983654231 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39e40052e0 con 0x7f39f01ef3d0 2023-11-24T07:58:58.351 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:58.345+0000 7f39ed7fa700 1 -- 172.21.15.59:0/1983654231 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f39e4005440 con 0x7f39f01ef3d0 2023-11-24T07:58:58.351 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:58.345+0000 7f39ed7fa700 1 -- 172.21.15.59:0/1983654231 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39e40038c0 con 0x7f39f01ef3d0 2023-11-24T07:58:58.352 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:58.345+0000 7f39f6bee700 1 -- 172.21.15.59:0/1983654231 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f39f015b8d0 con 0x7f39f01ef3d0 2023-11-24T07:58:58.352 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:58.345+0000 7f39f6bee700 1 -- 172.21.15.59:0/1983654231 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f39f015be90 con 0x7f39f01ef3d0 2023-11-24T07:58:58.352 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:58.346+0000 7f39f6bee700 1 -- 172.21.15.59:0/1983654231 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f39f01447e0 con 0x7f39f01ef3d0 2023-11-24T07:58:58.352 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:58.347+0000 7f39ed7fa700 1 -- 172.21.15.59:0/1983654231 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 10) v1 ==== 89588+0+0 (secure 0 0 0) 0x7f39e4003a20 con 0x7f39f01ef3d0 2023-11-24T07:58:58.352 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:58.347+0000 7f39ed7fa700 1 --2- 172.21.15.59:0/1983654231 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f39d406b2c0 0x7f39d406d780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:58.352 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:58.347+0000 7f39ed7fa700 1 -- 172.21.15.59:0/1983654231 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(61..61 src has 1..61) v4 ==== 5765+0+0 (secure 0 0 0) 0x7f39e4089c00 con 0x7f39f01ef3d0 2023-11-24T07:58:58.352 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:58.347+0000 7f39effff700 1 --2- 172.21.15.59:0/1983654231 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f39d406b2c0 0x7f39d406d780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:58.352 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:58.348+0000 7f39effff700 1 --2- 172.21.15.59:0/1983654231 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f39d406b2c0 0x7f39d406d780 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f39dc001b60 tx=0x7f39dc001af0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:58:58.353 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:58.351+0000 7f39ed7fa700 1 -- 172.21.15.59:0/1983654231 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f39e4054690 con 0x7f39f01ef3d0 2023-11-24T07:58:58.554 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:58.553+0000 7f39f6bee700 1 -- 172.21.15.59:0/1983654231 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f39f0075480 con 0x7f39f01ef3d0 2023-11-24T07:58:58.555 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:58.553+0000 7f39ed7fa700 1 -- 172.21.15.59:0/1983654231 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v61) v1 ==== 93+0+10 (secure 0 0 0) 0x7f39e4057cb0 con 0x7f39f01ef3d0 2023-11-24T07:58:58.555 INFO:teuthology.orchestra.run.smithi059.stdout:pg_num: 1 2023-11-24T07:58:58.557 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:58.555+0000 7f39f6bee700 1 -- 172.21.15.59:0/1983654231 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f39d406b2c0 msgr2=0x7f39d406d780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:58.557 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:58.555+0000 7f39f6bee700 1 --2- 172.21.15.59:0/1983654231 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f39d406b2c0 0x7f39d406d780 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f39dc001b60 tx=0x7f39dc001af0 comp rx=0 tx=0).stop 2023-11-24T07:58:58.557 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:58.555+0000 7f39f6bee700 1 -- 172.21.15.59:0/1983654231 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f39f01ef3d0 msgr2=0x7f39f015b2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:58.557 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:58.555+0000 7f39f6bee700 1 --2- 172.21.15.59:0/1983654231 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f39f01ef3d0 0x7f39f015b2f0 secure :-1 s=READY pgs=781 cs=0 l=1 rev1=1 crypto rx=0x7f39e400f7d0 tx=0x7f39e400fb90 comp rx=0 tx=0).stop 2023-11-24T07:58:58.558 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:58.555+0000 7f39f6bee700 1 -- 172.21.15.59:0/1983654231 shutdown_connections 2023-11-24T07:58:58.558 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:58.555+0000 7f39f6bee700 1 --2- 172.21.15.59:0/1983654231 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f39d406b2c0 0x7f39d406d780 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:58.558 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:58.555+0000 7f39f6bee700 1 --2- 172.21.15.59:0/1983654231 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f39f00cf020 0x7f39f00cf4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:58.558 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:58.555+0000 7f39f6bee700 1 --2- 172.21.15.59:0/1983654231 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f39f01ef3d0 0x7f39f015b2f0 unknown :-1 s=CLOSED pgs=781 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:58.558 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:58.555+0000 7f39f6bee700 1 --2- 172.21.15.59:0/1983654231 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f39f01ef7b0 0x7f39f0150fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:58.558 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:58.555+0000 7f39f6bee700 1 -- 172.21.15.59:0/1983654231 >> 172.21.15.59:0/1983654231 conn(0x7f39f00b2740 msgr2=0x7f39f01410d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:58:58.559 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:58.555+0000 7f39f6bee700 1 -- 172.21.15.59:0/1983654231 shutdown_connections 2023-11-24T07:58:58.559 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:58.555+0000 7f39f6bee700 1 -- 172.21.15.59:0/1983654231 wait complete. 2023-11-24T07:58:58.569 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-24T07:58:58.697 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:58.695+0000 7ffa9a955700 1 Processor -- start 2023-11-24T07:58:58.697 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:58.695+0000 7ffa9a955700 1 -- start start 2023-11-24T07:58:58.699 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:58.696+0000 7ffa9a955700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7ffa94143ae0 0x7ffa940b51b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:58.699 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:58.696+0000 7ffa9a955700 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7ffa940722c0 0x7ffa9406e510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:58.699 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:58.696+0000 7ffa9a955700 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffa94143ec0 con 0x7ffa941440f0 2023-11-24T07:58:58.699 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:58.696+0000 7ffa9a955700 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffa940b40a0 con 0x7ffa94143ae0 2023-11-24T07:58:58.700 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:58.696+0000 7ffa9a955700 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffa940b3ae0 con 0x7ffa940722c0 2023-11-24T07:58:58.700 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:58.696+0000 7ffa93fff700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7ffa94143ae0 0x7ffa940b51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:58.700 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:58.696+0000 7ffa93fff700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7ffa94143ae0 0x7ffa940b51b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:54324/0 (socket says 172.21.15.59:54324) 2023-11-24T07:58:58.700 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:58.696+0000 7ffa93fff700 1 -- 172.21.15.59:0/743266572 learned_addr learned my addr 172.21.15.59:0/743266572 (peer_addr_for_me v2:172.21.15.59:0/0) 2023-11-24T07:58:58.700 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:58.696+0000 7ffa98ef2700 1 --2- 172.21.15.59:0/743266572 >> v2:172.21.15.76:3300/0 conn(0x7ffa940722c0 0x7ffa9406e510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:58.700 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:58.696+0000 7ffa93fff700 1 -- 172.21.15.59:0/743266572 >> v2:172.21.15.76:3300/0 conn(0x7ffa940722c0 msgr2=0x7ffa9406e510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T07:58:58.700 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:58.696+0000 7ffa93fff700 1 --2- 172.21.15.59:0/743266572 >> v2:172.21.15.76:3300/0 conn(0x7ffa940722c0 0x7ffa9406e510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:58.700 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:58.697+0000 7ffa93fff700 1 -- 172.21.15.59:0/743266572 >> v1:172.21.15.76:6789/0 conn(0x7ffa941440f0 legacy=0x7ffa94071bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:58.700 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:58.697+0000 7ffa93fff700 1 -- 172.21.15.59:0/743266572 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffa940b3dc0 con 0x7ffa94143ae0 2023-11-24T07:58:58.700 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:58.697+0000 7ffa93fff700 1 --2- 172.21.15.59:0/743266572 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ffa94143ae0 0x7ffa940b51b0 secure :-1 s=READY pgs=782 cs=0 l=1 rev1=1 crypto rx=0x7ffa940747c0 tx=0x7ffa8400be70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9e932b88a5d41ece server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:58:58.700 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:58.697+0000 7ffa92ffd700 1 -- 172.21.15.59:0/743266572 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa8400dc70 con 0x7ffa94143ae0 2023-11-24T07:58:58.700 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:58.697+0000 7ffa92ffd700 1 -- 172.21.15.59:0/743266572 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffa8400ddd0 con 0x7ffa94143ae0 2023-11-24T07:58:58.700 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:58.697+0000 7ffa92ffd700 1 -- 172.21.15.59:0/743266572 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa84010690 con 0x7ffa94143ae0 2023-11-24T07:58:58.701 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:58.697+0000 7ffa9a955700 1 -- 172.21.15.59:0/743266572 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ffa94143ae0 msgr2=0x7ffa940b51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:58.701 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:58.697+0000 7ffa9a955700 1 --2- 172.21.15.59:0/743266572 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ffa94143ae0 0x7ffa940b51b0 secure :-1 s=READY pgs=782 cs=0 l=1 rev1=1 crypto rx=0x7ffa940747c0 tx=0x7ffa8400be70 comp rx=0 tx=0).stop 2023-11-24T07:58:58.701 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:58.697+0000 7ffa9a955700 1 -- 172.21.15.59:0/743266572 shutdown_connections 2023-11-24T07:58:58.701 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:58.697+0000 7ffa9a955700 1 --2- 172.21.15.59:0/743266572 >> v2:172.21.15.76:3300/0 conn(0x7ffa940722c0 0x7ffa9406e510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:58.701 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:58.697+0000 7ffa9a955700 1 --2- 172.21.15.59:0/743266572 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ffa94143ae0 0x7ffa940b51b0 unknown :-1 s=CLOSED pgs=782 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:58.701 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:58.697+0000 7ffa9a955700 1 -- 172.21.15.59:0/743266572 >> 172.21.15.59:0/743266572 conn(0x7ffa940b2740 msgr2=0x7ffa940b2b20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:58:58.701 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:58.697+0000 7ffa9a955700 1 -- 172.21.15.59:0/743266572 shutdown_connections 2023-11-24T07:58:58.701 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:58.698+0000 7ffa9a955700 1 -- 172.21.15.59:0/743266572 wait complete. 2023-11-24T07:58:58.704 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:58.699+0000 7ffa9a955700 1 Processor -- start 2023-11-24T07:58:58.704 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:58.699+0000 7ffa9a955700 1 -- start start 2023-11-24T07:58:58.704 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:58.700+0000 7ffa9a955700 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7ffa940722c0 0x7ffa9413a420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:58.704 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:58.700+0000 7ffa9a955700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ffa94143ae0 0x7ffa940c9dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:58.704 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:58.700+0000 7ffa9a955700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ffa941440f0 0x7ffa940ca300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:58.704 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:58.700+0000 7ffa9a955700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa9415b700 con 0x7ffa94143ae0 2023-11-24T07:58:58.704 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:58.700+0000 7ffa9a955700 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa940cb5b0 con 0x7ffa941440f0 2023-11-24T07:58:58.704 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:58.700+0000 7ffa9a955700 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffa94141880 con 0x7ffa940722c0 2023-11-24T07:58:58.704 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:58.700+0000 7ffa937fe700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ffa94143ae0 0x7ffa940c9dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:58.704 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:58.700+0000 7ffa937fe700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ffa94143ae0 0x7ffa940c9dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:54332/0 (socket says 172.21.15.59:54332) 2023-11-24T07:58:58.704 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:58.700+0000 7ffa937fe700 1 -- 172.21.15.59:0/2746154921 learned_addr learned my addr 172.21.15.59:0/2746154921 (peer_addr_for_me v2:172.21.15.59:0/0) 2023-11-24T07:58:58.705 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:58.700+0000 7ffa93fff700 1 --2- 172.21.15.59:0/2746154921 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7ffa940722c0 0x7ffa9413a420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:58.705 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:58.700+0000 7ffa98ef2700 1 --2- 172.21.15.59:0/2746154921 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ffa941440f0 0x7ffa940ca300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:58.705 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:58.700+0000 7ffa937fe700 1 -- 172.21.15.59:0/2746154921 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7ffa940722c0 msgr2=0x7ffa9413a420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:58.705 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:58.700+0000 7ffa937fe700 1 --2- 172.21.15.59:0/2746154921 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7ffa940722c0 0x7ffa9413a420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:58.705 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:58.700+0000 7ffa937fe700 1 -- 172.21.15.59:0/2746154921 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ffa941440f0 msgr2=0x7ffa940ca300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:58.705 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:58.700+0000 7ffa937fe700 1 --2- 172.21.15.59:0/2746154921 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ffa941440f0 0x7ffa940ca300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:58.706 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:58.700+0000 7ffa937fe700 1 -- 172.21.15.59:0/2746154921 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffa8400b7f0 con 0x7ffa94143ae0 2023-11-24T07:58:58.706 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:58.700+0000 7ffa937fe700 1 --2- 172.21.15.59:0/2746154921 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ffa94143ae0 0x7ffa940c9dc0 secure :-1 s=READY pgs=783 cs=0 l=1 rev1=1 crypto rx=0x7ffa7c00c450 tx=0x7ffa7c00c760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:58:58.706 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:58.700+0000 7ffa90ff9700 1 -- 172.21.15.59:0/2746154921 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa7c0052d0 con 0x7ffa94143ae0 2023-11-24T07:58:58.706 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:58.700+0000 7ffa90ff9700 1 -- 172.21.15.59:0/2746154921 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffa7c005430 con 0x7ffa94143ae0 2023-11-24T07:58:58.706 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:58.700+0000 7ffa90ff9700 1 -- 172.21.15.59:0/2746154921 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa7c0038c0 con 0x7ffa94143ae0 2023-11-24T07:58:58.707 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:58.700+0000 7ffa9a955700 1 -- 172.21.15.59:0/2746154921 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffa940ca9a0 con 0x7ffa94143ae0 2023-11-24T07:58:58.707 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:58.700+0000 7ffa9a955700 1 -- 172.21.15.59:0/2746154921 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffa940caf60 con 0x7ffa94143ae0 2023-11-24T07:58:58.707 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:58.701+0000 7ffa9a955700 1 -- 172.21.15.59:0/2746154921 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffa94075480 con 0x7ffa94143ae0 2023-11-24T07:58:58.707 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:58.702+0000 7ffa90ff9700 1 -- 172.21.15.59:0/2746154921 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 10) v1 ==== 89588+0+0 (secure 0 0 0) 0x7ffa7c003a20 con 0x7ffa94143ae0 2023-11-24T07:58:58.707 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:58.702+0000 7ffa90ff9700 1 --2- 172.21.15.59:0/2746154921 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7ffa7406b2c0 0x7ffa7406d780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:58.707 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:58.702+0000 7ffa90ff9700 1 -- 172.21.15.59:0/2746154921 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(61..61 src has 1..61) v4 ==== 5765+0+0 (secure 0 0 0) 0x7ffa7c089080 con 0x7ffa94143ae0 2023-11-24T07:58:58.708 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:58.702+0000 7ffa93fff700 1 --2- 172.21.15.59:0/2746154921 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7ffa7406b2c0 0x7ffa7406d780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:58.708 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:58.703+0000 7ffa93fff700 1 --2- 172.21.15.59:0/2746154921 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7ffa7406b2c0 0x7ffa7406d780 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7ffa8400beb0 tx=0x7ffa8401c040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:58:58.709 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:58.707+0000 7ffa90ff9700 1 -- 172.21.15.59:0/2746154921 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7ffa7c0bbe90 con 0x7ffa94143ae0 2023-11-24T07:58:58.908 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:58.906+0000 7ffa9a955700 1 -- 172.21.15.59:0/2746154921 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7ffa94144830 con 0x7ffa94143ae0 2023-11-24T07:58:58.908 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:58.907+0000 7ffa90ff9700 1 -- 172.21.15.59:0/2746154921 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v61) v1 ==== 104+0+11 (secure 0 0 0) 0x7ffa7c017020 con 0x7ffa94143ae0 2023-11-24T07:58:58.909 INFO:teuthology.orchestra.run.smithi059.stdout:pg_num: 32 2023-11-24T07:58:58.911 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:58.909+0000 7ffa9a955700 1 -- 172.21.15.59:0/2746154921 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7ffa7406b2c0 msgr2=0x7ffa7406d780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:58.911 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:58.909+0000 7ffa9a955700 1 --2- 172.21.15.59:0/2746154921 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7ffa7406b2c0 0x7ffa7406d780 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7ffa8400beb0 tx=0x7ffa8401c040 comp rx=0 tx=0).stop 2023-11-24T07:58:58.911 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:58.909+0000 7ffa9a955700 1 -- 172.21.15.59:0/2746154921 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ffa94143ae0 msgr2=0x7ffa940c9dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:58.912 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:58.909+0000 7ffa9a955700 1 --2- 172.21.15.59:0/2746154921 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ffa94143ae0 0x7ffa940c9dc0 secure :-1 s=READY pgs=783 cs=0 l=1 rev1=1 crypto rx=0x7ffa7c00c450 tx=0x7ffa7c00c760 comp rx=0 tx=0).stop 2023-11-24T07:58:58.912 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:58.909+0000 7ffa9a955700 1 -- 172.21.15.59:0/2746154921 shutdown_connections 2023-11-24T07:58:58.912 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:58.909+0000 7ffa9a955700 1 --2- 172.21.15.59:0/2746154921 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7ffa7406b2c0 0x7ffa7406d780 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:58.912 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:58.909+0000 7ffa9a955700 1 --2- 172.21.15.59:0/2746154921 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ffa941440f0 0x7ffa940ca300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:58.913 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:58.909+0000 7ffa9a955700 1 --2- 172.21.15.59:0/2746154921 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7ffa940722c0 0x7ffa9413a420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:58.913 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:58.909+0000 7ffa9a955700 1 --2- 172.21.15.59:0/2746154921 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ffa94143ae0 0x7ffa940c9dc0 unknown :-1 s=CLOSED pgs=783 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:58.913 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:58.909+0000 7ffa9a955700 1 -- 172.21.15.59:0/2746154921 >> 172.21.15.59:0/2746154921 conn(0x7ffa940b2740 msgr2=0x7ffa940cf4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:58:58.913 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:58.909+0000 7ffa9a955700 1 -- 172.21.15.59:0/2746154921 shutdown_connections 2023-11-24T07:58:58.914 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:58.909+0000 7ffa9a955700 1 -- 172.21.15.59:0/2746154921 wait complete. 2023-11-24T07:58:58.923 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-24T07:58:59.050 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:59.048+0000 7f5dcc588700 1 Processor -- start 2023-11-24T07:58:59.050 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:59.048+0000 7f5dcc588700 1 -- start start 2023-11-24T07:58:59.052 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:59.049+0000 7f5dcc588700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f5dc4142810 0x7f5dc406afd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:59.052 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:59.049+0000 7f5dcc588700 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f5dc41459a0 0x7f5dc406b510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:59.052 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:59.049+0000 7f5dcc588700 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5dc40cf5f0 con 0x7f5dc406e580 2023-11-24T07:58:59.053 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:59.049+0000 7f5dcc588700 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5dc406ea80 con 0x7f5dc4142810 2023-11-24T07:58:59.053 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:59.049+0000 7f5dcc588700 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5dc406ed60 con 0x7f5dc41459a0 2023-11-24T07:58:59.053 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:59.049+0000 7f5dca324700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f5dc4142810 0x7f5dc406afd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:59.053 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:59.049+0000 7f5dca324700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f5dc4142810 0x7f5dc406afd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:54338/0 (socket says 172.21.15.59:54338) 2023-11-24T07:58:59.053 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:59.049+0000 7f5dca324700 1 -- 172.21.15.59:0/149469592 learned_addr learned my addr 172.21.15.59:0/149469592 (peer_addr_for_me v2:172.21.15.59:0/0) 2023-11-24T07:58:59.053 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:59.050+0000 7f5dc9b23700 1 --2- 172.21.15.59:0/149469592 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f5dc41459a0 0x7f5dc406b510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:59.053 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:59.050+0000 7f5dca324700 1 -- 172.21.15.59:0/149469592 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f5dc41459a0 msgr2=0x7f5dc406b510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T07:58:59.053 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:59.050+0000 7f5dca324700 1 --2- 172.21.15.59:0/149469592 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f5dc41459a0 0x7f5dc406b510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:59.053 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:59.050+0000 7f5dca324700 1 -- 172.21.15.59:0/149469592 >> v1:172.21.15.76:6789/0 conn(0x7f5dc406e580 legacy=0x7f5dc4071bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:59.053 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:59.050+0000 7f5dca324700 1 -- 172.21.15.59:0/149469592 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5dc406f040 con 0x7f5dc4142810 2023-11-24T07:58:59.053 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:59.050+0000 7f5dca324700 1 --2- 172.21.15.59:0/149469592 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5dc4142810 0x7f5dc406afd0 secure :-1 s=READY pgs=784 cs=0 l=1 rev1=1 crypto rx=0x7f5db40098f0 tx=0x7f5db4009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=722d117010f517bc server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:58:59.054 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:59.050+0000 7f5dc9322700 1 -- 172.21.15.59:0/149469592 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5db400b8a0 con 0x7f5dc4142810 2023-11-24T07:58:59.054 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:59.050+0000 7f5dc9322700 1 -- 172.21.15.59:0/149469592 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5db400ba00 con 0x7f5dc4142810 2023-11-24T07:58:59.054 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:59.050+0000 7f5dc9322700 1 -- 172.21.15.59:0/149469592 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5db400bd90 con 0x7f5dc4142810 2023-11-24T07:58:59.054 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:59.050+0000 7f5dcc588700 1 -- 172.21.15.59:0/149469592 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5dc4142810 msgr2=0x7f5dc406afd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:59.054 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:59.050+0000 7f5dcc588700 1 --2- 172.21.15.59:0/149469592 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5dc4142810 0x7f5dc406afd0 secure :-1 s=READY pgs=784 cs=0 l=1 rev1=1 crypto rx=0x7f5db40098f0 tx=0x7f5db4009c00 comp rx=0 tx=0).stop 2023-11-24T07:58:59.054 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:59.051+0000 7f5dcc588700 1 -- 172.21.15.59:0/149469592 shutdown_connections 2023-11-24T07:58:59.054 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:59.051+0000 7f5dcc588700 1 --2- 172.21.15.59:0/149469592 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5dc4142810 0x7f5dc406afd0 unknown :-1 s=CLOSED pgs=784 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:59.054 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:59.051+0000 7f5dcc588700 1 --2- 172.21.15.59:0/149469592 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f5dc41459a0 0x7f5dc406b510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:59.054 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:59.051+0000 7f5dcc588700 1 -- 172.21.15.59:0/149469592 >> 172.21.15.59:0/149469592 conn(0x7f5dc40b2740 msgr2=0x7f5dc40b2b20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:58:59.054 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:59.051+0000 7f5dcc588700 1 -- 172.21.15.59:0/149469592 shutdown_connections 2023-11-24T07:58:59.054 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:59.051+0000 7f5dcc588700 1 -- 172.21.15.59:0/149469592 wait complete. 2023-11-24T07:58:59.057 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:59.052+0000 7f5dcc588700 1 Processor -- start 2023-11-24T07:58:59.057 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:59.052+0000 7f5dcc588700 1 -- start start 2023-11-24T07:58:59.057 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:59.053+0000 7f5dcc588700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5dc406e580 0x7f5dc413f630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:59.057 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:59.053+0000 7f5dcc588700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5dc4142810 0x7f5dc413b500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:59.057 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:59.053+0000 7f5dcc588700 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f5dc41459a0 0x7f5dc41f4540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:59.057 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:59.053+0000 7f5dcc588700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5dc413a4f0 con 0x7f5dc4142810 2023-11-24T07:58:59.057 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:59.053+0000 7f5dcc588700 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5dc415d6d0 con 0x7f5dc406e580 2023-11-24T07:58:59.057 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:59.053+0000 7f5dcc588700 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5dc413bc00 con 0x7f5dc41459a0 2023-11-24T07:58:59.057 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:59.053+0000 7f5dc9b23700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5dc4142810 0x7f5dc413b500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:59.058 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:59.053+0000 7f5dca324700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5dc406e580 0x7f5dc413f630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:59.058 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:59.053+0000 7f5dc9b23700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5dc4142810 0x7f5dc413b500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:54346/0 (socket says 172.21.15.59:54346) 2023-11-24T07:58:59.058 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:59.053+0000 7f5dca324700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5dc406e580 0x7f5dc413f630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.59:41542/0 (socket says 172.21.15.59:41542) 2023-11-24T07:58:59.058 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:59.053+0000 7f5dc9b23700 1 -- 172.21.15.59:0/4172835363 learned_addr learned my addr 172.21.15.59:0/4172835363 (peer_addr_for_me v2:172.21.15.59:0/0) 2023-11-24T07:58:59.058 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:59.053+0000 7f5dcab25700 1 --2- 172.21.15.59:0/4172835363 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f5dc41459a0 0x7f5dc41f4540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:59.058 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:59.053+0000 7f5dc9b23700 1 -- 172.21.15.59:0/4172835363 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f5dc41459a0 msgr2=0x7f5dc41f4540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:59.058 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:59.053+0000 7f5dc9b23700 1 --2- 172.21.15.59:0/4172835363 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f5dc41459a0 0x7f5dc41f4540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:59.058 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:59.053+0000 7f5dc9b23700 1 -- 172.21.15.59:0/4172835363 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5dc406e580 msgr2=0x7f5dc413f630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:59.058 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:59.053+0000 7f5dc9b23700 1 --2- 172.21.15.59:0/4172835363 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5dc406e580 0x7f5dc413f630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:59.058 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:59.053+0000 7f5dc9b23700 1 -- 172.21.15.59:0/4172835363 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5db4009550 con 0x7f5dc4142810 2023-11-24T07:58:59.058 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:59.053+0000 7f5dc9b23700 1 --2- 172.21.15.59:0/4172835363 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5dc4142810 0x7f5dc413b500 secure :-1 s=READY pgs=785 cs=0 l=1 rev1=1 crypto rx=0x7f5dc000d800 tx=0x7f5dc000db10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:58:59.059 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:59.053+0000 7f5dbaffd700 1 -- 172.21.15.59:0/4172835363 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5dc00125f0 con 0x7f5dc4142810 2023-11-24T07:58:59.059 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:59.053+0000 7f5dbaffd700 1 -- 172.21.15.59:0/4172835363 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5dc0012750 con 0x7f5dc4142810 2023-11-24T07:58:59.059 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:59.053+0000 7f5dcc588700 1 -- 172.21.15.59:0/4172835363 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5dc413bee0 con 0x7f5dc4142810 2023-11-24T07:58:59.059 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:59.053+0000 7f5dbaffd700 1 -- 172.21.15.59:0/4172835363 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5dc0012b00 con 0x7f5dc4142810 2023-11-24T07:58:59.059 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:59.053+0000 7f5dcc588700 1 -- 172.21.15.59:0/4172835363 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5dc413c4a0 con 0x7f5dc4142810 2023-11-24T07:58:59.059 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:59.054+0000 7f5dcc588700 1 -- 172.21.15.59:0/4172835363 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5dc4075480 con 0x7f5dc4142810 2023-11-24T07:58:59.059 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:59.055+0000 7f5dbaffd700 1 -- 172.21.15.59:0/4172835363 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 10) v1 ==== 89588+0+0 (secure 0 0 0) 0x7f5dc0012c60 con 0x7f5dc4142810 2023-11-24T07:58:59.059 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:59.055+0000 7f5dbaffd700 1 --2- 172.21.15.59:0/4172835363 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f5da806b2c0 0x7f5da806d780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:59.059 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:59.055+0000 7f5dbaffd700 1 -- 172.21.15.59:0/4172835363 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(61..61 src has 1..61) v4 ==== 5765+0+0 (secure 0 0 0) 0x7f5dc0089de0 con 0x7f5dc4142810 2023-11-24T07:58:59.059 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:59.055+0000 7f5dca324700 1 --2- 172.21.15.59:0/4172835363 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f5da806b2c0 0x7f5da806d780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:59.060 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:59.056+0000 7f5dca324700 1 --2- 172.21.15.59:0/4172835363 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f5da806b2c0 0x7f5da806d780 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f5db4009c40 tx=0x7f5db4008030 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:58:59.061 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:59.060+0000 7f5dbaffd700 1 -- 172.21.15.59:0/4172835363 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f5dc0054830 con 0x7f5dc4142810 2023-11-24T07:58:59.260 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:59.259+0000 7f5dcc588700 1 -- 172.21.15.59:0/4172835363 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f5dc415d840 con 0x7f5dc4142810 2023-11-24T07:58:59.261 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:59.259+0000 7f5dbaffd700 1 -- 172.21.15.59:0/4172835363 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v61) v1 ==== 100+0+11 (secure 0 0 0) 0x7f5dc415d840 con 0x7f5dc4142810 2023-11-24T07:58:59.261 INFO:teuthology.orchestra.run.smithi059.stdout:pg_num: 64 2023-11-24T07:58:59.263 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:59.261+0000 7f5dcc588700 1 -- 172.21.15.59:0/4172835363 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f5da806b2c0 msgr2=0x7f5da806d780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:59.263 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:59.261+0000 7f5dcc588700 1 --2- 172.21.15.59:0/4172835363 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f5da806b2c0 0x7f5da806d780 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f5db4009c40 tx=0x7f5db4008030 comp rx=0 tx=0).stop 2023-11-24T07:58:59.264 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:59.261+0000 7f5dcc588700 1 -- 172.21.15.59:0/4172835363 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5dc4142810 msgr2=0x7f5dc413b500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:59.264 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:59.261+0000 7f5dcc588700 1 --2- 172.21.15.59:0/4172835363 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5dc4142810 0x7f5dc413b500 secure :-1 s=READY pgs=785 cs=0 l=1 rev1=1 crypto rx=0x7f5dc000d800 tx=0x7f5dc000db10 comp rx=0 tx=0).stop 2023-11-24T07:58:59.264 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:59.261+0000 7f5dcc588700 1 -- 172.21.15.59:0/4172835363 shutdown_connections 2023-11-24T07:58:59.264 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:59.261+0000 7f5dcc588700 1 --2- 172.21.15.59:0/4172835363 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f5da806b2c0 0x7f5da806d780 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:59.264 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:59.261+0000 7f5dcc588700 1 --2- 172.21.15.59:0/4172835363 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f5dc41459a0 0x7f5dc41f4540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:59.264 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:59.261+0000 7f5dcc588700 1 --2- 172.21.15.59:0/4172835363 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5dc406e580 0x7f5dc413f630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:59.264 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:59.261+0000 7f5dcc588700 1 --2- 172.21.15.59:0/4172835363 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5dc4142810 0x7f5dc413b500 unknown :-1 s=CLOSED pgs=785 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:59.264 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:59.261+0000 7f5dcc588700 1 -- 172.21.15.59:0/4172835363 >> 172.21.15.59:0/4172835363 conn(0x7f5dc40b2740 msgr2=0x7f5dc406afd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:58:59.264 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:59.261+0000 7f5dcc588700 1 -- 172.21.15.59:0/4172835363 shutdown_connections 2023-11-24T07:58:59.264 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:59.261+0000 7f5dcc588700 1 -- 172.21.15.59:0/4172835363 wait complete. 2023-11-24T07:58:59.275 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-24T07:58:59.400 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:59.398+0000 7fa1ab806700 1 Processor -- start 2023-11-24T07:58:59.400 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:59.398+0000 7fa1ab806700 1 -- start start 2023-11-24T07:58:59.404 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:59.399+0000 7fa1ab806700 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fa1a406b6e0 0x7fa1a406db70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:59.404 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:59.399+0000 7fa1ab806700 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fa1a4145100 0x7fa1a406e0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:59.404 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:59.399+0000 7fa1ab806700 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa1a4160720 con 0x7fa1a4144c10 2023-11-24T07:58:59.404 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:59.399+0000 7fa1ab806700 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa1a406f500 con 0x7fa1a406b6e0 2023-11-24T07:58:59.404 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:59.399+0000 7fa1ab806700 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa1a40cd610 con 0x7fa1a4145100 2023-11-24T07:58:59.404 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:59.399+0000 7fa1a95a2700 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7fa1a4144c10 0x7fa1a406afd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:57826/0 (socket says 172.21.15.59:57826) 2023-11-24T07:58:59.404 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:59.399+0000 7fa1a95a2700 1 -- 172.21.15.59:0/2301997166 learned_addr learned my addr 172.21.15.59:0/2301997166 (peer_addr_for_me v1:172.21.15.59:0/0) 2023-11-24T07:58:59.404 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:59.399+0000 7fa1a8da1700 1 --2- 172.21.15.59:0/2301997166 >> v2:172.21.15.76:3300/0 conn(0x7fa1a406b6e0 0x7fa1a406db70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:59.404 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:59.399+0000 7fa1a9da3700 1 --2- 172.21.15.59:0/2301997166 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fa1a4145100 0x7fa1a406e0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:59.404 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:59.399+0000 7fa19bfff700 1 -- 172.21.15.59:0/2301997166 <== mon.0 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3477225511 0 0) 0x7fa1a4160720 con 0x7fa1a4144c10 2023-11-24T07:58:59.405 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:59.399+0000 7fa19bfff700 1 -- 172.21.15.59:0/2301997166 --> v1:172.21.15.59:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa1900035b0 con 0x7fa1a4144c10 2023-11-24T07:58:59.405 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:59.400+0000 7fa19bfff700 1 -- 172.21.15.59:0/2301997166 <== mon.0 v1:172.21.15.59:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 899116273 0 0) 0x7fa1900035b0 con 0x7fa1a4144c10 2023-11-24T07:58:59.405 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:59.400+0000 7fa19bfff700 1 -- 172.21.15.59:0/2301997166 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fa1a4145100 msgr2=0x7fa1a406e0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T07:58:59.405 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:59.400+0000 7fa19bfff700 1 --2- 172.21.15.59:0/2301997166 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fa1a4145100 0x7fa1a406e0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:59.405 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:59.400+0000 7fa19bfff700 1 -- 172.21.15.59:0/2301997166 >> v2:172.21.15.76:3300/0 conn(0x7fa1a406b6e0 msgr2=0x7fa1a406db70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T07:58:59.405 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:59.400+0000 7fa19bfff700 1 --2- 172.21.15.59:0/2301997166 >> v2:172.21.15.76:3300/0 conn(0x7fa1a406b6e0 0x7fa1a406db70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:59.405 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:59.400+0000 7fa19bfff700 1 -- 172.21.15.59:0/2301997166 --> v1:172.21.15.59:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa1a406f710 con 0x7fa1a4144c10 2023-11-24T07:58:59.405 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:59.400+0000 7fa19bfff700 1 -- 172.21.15.59:0/2301997166 <== mon.0 v1:172.21.15.59:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1312333566 0 0) 0x7fa194001670 con 0x7fa1a4144c10 2023-11-24T07:58:59.405 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:59.400+0000 7fa1a8da1700 1 --2- 172.21.15.59:0/2301997166 >> v2:172.21.15.76:3300/0 conn(0x7fa1a406b6e0 0x7fa1a406db70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-24T07:58:59.405 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:59.400+0000 7fa19bfff700 1 -- 172.21.15.59:0/2301997166 >> v1:172.21.15.59:6789/0 conn(0x7fa1a4144c10 legacy=0x7fa1a406afd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:59.405 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:59.400+0000 7fa19bfff700 1 --2- 172.21.15.59:0/2301997166 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa190003a00 0x7fa190003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:59.406 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:59.400+0000 7fa19bfff700 1 --2- 172.21.15.59:0/2301997166 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fa1a4145100 0x7fa190004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:59.406 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:59.400+0000 7fa19bfff700 1 --2- 172.21.15.59:0/2301997166 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa190004d90 0x7fa190005210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:59.406 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:59.400+0000 7fa19bfff700 1 -- 172.21.15.59:0/2301997166 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1a406f500 con 0x7fa190003a00 2023-11-24T07:58:59.406 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:59.400+0000 7fa19bfff700 1 -- 172.21.15.59:0/2301997166 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1a40cd610 con 0x7fa190004d90 2023-11-24T07:58:59.406 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:59.400+0000 7fa19bfff700 1 -- 172.21.15.59:0/2301997166 --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa19000ba10 con 0x7fa1a4145100 2023-11-24T07:58:59.406 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:59.400+0000 7fa1a95a2700 1 --2- 172.21.15.59:0/2301997166 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa190003a00 0x7fa190003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:59.406 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:59.400+0000 7fa1a9da3700 1 --2- 172.21.15.59:0/2301997166 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa190004d90 0x7fa190005210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:59.407 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:59.400+0000 7fa1a8da1700 1 --2- 172.21.15.59:0/2301997166 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fa1a4145100 0x7fa190004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:59.407 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:59.400+0000 7fa1a95a2700 1 -- 172.21.15.59:0/2301997166 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fa1a4145100 msgr2=0x7fa190004850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T07:58:59.407 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:59.400+0000 7fa1a95a2700 1 --2- 172.21.15.59:0/2301997166 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fa1a4145100 0x7fa190004850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:59.407 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:59.400+0000 7fa1a95a2700 1 -- 172.21.15.59:0/2301997166 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa190004d90 msgr2=0x7fa190005210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T07:58:59.407 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:59.400+0000 7fa1a95a2700 1 --2- 172.21.15.59:0/2301997166 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa190004d90 0x7fa190005210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:59.407 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:59.400+0000 7fa1a95a2700 1 -- 172.21.15.59:0/2301997166 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa1900035b0 con 0x7fa190003a00 2023-11-24T07:58:59.407 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:59.400+0000 7fa1a8da1700 1 --2- 172.21.15.59:0/2301997166 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fa1a4145100 0x7fa190004850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-24T07:58:59.407 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:59.400+0000 7fa1a9da3700 1 --2- 172.21.15.59:0/2301997166 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa190004d90 0x7fa190005210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-24T07:58:59.407 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:59.400+0000 7fa1a95a2700 1 --2- 172.21.15.59:0/2301997166 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa190003a00 0x7fa190003e20 secure :-1 s=READY pgs=787 cs=0 l=1 rev1=1 crypto rx=0x7fa19400c320 tx=0x7fa190005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=405a6dde605da4c7 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:58:59.407 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:59.400+0000 7fa19bfff700 1 -- 172.21.15.59:0/2301997166 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa19400ce60 con 0x7fa190003a00 2023-11-24T07:58:59.407 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:59.400+0000 7fa19bfff700 1 -- 172.21.15.59:0/2301997166 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa194005960 con 0x7fa190003a00 2023-11-24T07:58:59.408 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:59.400+0000 7fa19bfff700 1 -- 172.21.15.59:0/2301997166 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa194005cf0 con 0x7fa190003a00 2023-11-24T07:58:59.408 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:59.401+0000 7fa1ab806700 1 -- 172.21.15.59:0/2301997166 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa190003a00 msgr2=0x7fa190003e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:59.408 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:59.401+0000 7fa1ab806700 1 --2- 172.21.15.59:0/2301997166 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa190003a00 0x7fa190003e20 secure :-1 s=READY pgs=787 cs=0 l=1 rev1=1 crypto rx=0x7fa19400c320 tx=0x7fa190005880 comp rx=0 tx=0).stop 2023-11-24T07:58:59.408 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:59.401+0000 7fa1a9da3700 1 -- 172.21.15.59:0/2301997166 reap_dead start 2023-11-24T07:58:59.408 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:59.401+0000 7fa1ab806700 1 -- 172.21.15.59:0/2301997166 shutdown_connections 2023-11-24T07:58:59.408 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:59.401+0000 7fa1ab806700 1 -- 172.21.15.59:0/2301997166 >> 172.21.15.59:0/2301997166 conn(0x7fa1a40b2740 msgr2=0x7fa1a40b2b20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:58:59.408 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:59.401+0000 7fa1ab806700 1 -- 172.21.15.59:0/2301997166 shutdown_connections 2023-11-24T07:58:59.408 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:59.401+0000 7fa1ab806700 1 -- 172.21.15.59:0/2301997166 wait complete. 2023-11-24T07:58:59.408 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:59.402+0000 7fa1ab806700 1 Processor -- start 2023-11-24T07:58:59.408 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:59.402+0000 7fa1ab806700 1 -- start start 2023-11-24T07:58:59.411 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:59.403+0000 7fa1ab806700 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fa190003a00 0x7fa1a41f19f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:59.411 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:59.403+0000 7fa1ab806700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa1a415e880 0x7fa1a40aa090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:59.411 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:59.403+0000 7fa1ab806700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa1a415ed00 0x7fa1a40aa5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:59.411 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:59.403+0000 7fa1ab806700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1a41f5f00 con 0x7fa1a415ed00 2023-11-24T07:58:59.411 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:59.403+0000 7fa1ab806700 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1a41f1f30 con 0x7fa1a415e880 2023-11-24T07:58:59.411 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:59.403+0000 7fa1ab806700 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa1a41423b0 con 0x7fa190003a00 2023-11-24T07:58:59.411 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:59.403+0000 7fa1a9da3700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa1a415ed00 0x7fa1a40aa5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:59.411 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:59.403+0000 7fa1a9da3700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa1a415ed00 0x7fa1a40aa5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:54368/0 (socket says 172.21.15.59:54368) 2023-11-24T07:58:59.412 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:59.403+0000 7fa1a9da3700 1 -- 172.21.15.59:0/87332636 learned_addr learned my addr 172.21.15.59:0/87332636 (peer_addr_for_me v2:172.21.15.59:0/0) 2023-11-24T07:58:59.412 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:59.403+0000 7fa1a8da1700 1 --2- 172.21.15.59:0/87332636 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa1a415e880 0x7fa1a40aa090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:59.412 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:59.403+0000 7fa1a95a2700 1 --2- 172.21.15.59:0/87332636 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fa190003a00 0x7fa1a41f19f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:59.412 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:59.403+0000 7fa1a9da3700 1 -- 172.21.15.59:0/87332636 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fa190003a00 msgr2=0x7fa1a41f19f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:59.412 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:59.403+0000 7fa1a9da3700 1 --2- 172.21.15.59:0/87332636 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fa190003a00 0x7fa1a41f19f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:59.412 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:59.403+0000 7fa1a9da3700 1 -- 172.21.15.59:0/87332636 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa1a415e880 msgr2=0x7fa1a40aa090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:59.412 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:59.403+0000 7fa1a9da3700 1 --2- 172.21.15.59:0/87332636 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa1a415e880 0x7fa1a40aa090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:59.412 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:59.403+0000 7fa1a9da3700 1 -- 172.21.15.59:0/87332636 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa1a4142690 con 0x7fa1a415ed00 2023-11-24T07:58:59.412 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:59.404+0000 7fa1a9da3700 1 --2- 172.21.15.59:0/87332636 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa1a415ed00 0x7fa1a40aa5d0 secure :-1 s=READY pgs=788 cs=0 l=1 rev1=1 crypto rx=0x7fa1a415c410 tx=0x7fa19c0104a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:58:59.412 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:59.404+0000 7fa199ffb700 1 -- 172.21.15.59:0/87332636 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa19c01b070 con 0x7fa1a415ed00 2023-11-24T07:58:59.412 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:59.404+0000 7fa199ffb700 1 -- 172.21.15.59:0/87332636 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa19c00f040 con 0x7fa1a415ed00 2023-11-24T07:58:59.413 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:59.404+0000 7fa1ab806700 1 -- 172.21.15.59:0/87332636 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa1a4142970 con 0x7fa1a415ed00 2023-11-24T07:58:59.413 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:59.404+0000 7fa199ffb700 1 -- 172.21.15.59:0/87332636 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa19c00e050 con 0x7fa1a415ed00 2023-11-24T07:58:59.413 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:59.404+0000 7fa1ab806700 1 -- 172.21.15.59:0/87332636 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa1a415b4c0 con 0x7fa1a415ed00 2023-11-24T07:58:59.413 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:59.405+0000 7fa1ab806700 1 -- 172.21.15.59:0/87332636 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa1a40cd350 con 0x7fa1a415ed00 2023-11-24T07:58:59.413 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:59.405+0000 7fa199ffb700 1 -- 172.21.15.59:0/87332636 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 10) v1 ==== 89588+0+0 (secure 0 0 0) 0x7fa19c010d00 con 0x7fa1a415ed00 2023-11-24T07:58:59.413 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:59.406+0000 7fa199ffb700 1 --2- 172.21.15.59:0/87332636 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7fa18806b2c0 0x7fa18806d780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:59.413 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:59.406+0000 7fa199ffb700 1 -- 172.21.15.59:0/87332636 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(61..61 src has 1..61) v4 ==== 5765+0+0 (secure 0 0 0) 0x7fa19c089e70 con 0x7fa1a415ed00 2023-11-24T07:58:59.413 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:59.406+0000 7fa1a95a2700 1 --2- 172.21.15.59:0/87332636 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7fa18806b2c0 0x7fa18806d780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:59.413 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:59.407+0000 7fa1a95a2700 1 --2- 172.21.15.59:0/87332636 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7fa18806b2c0 0x7fa18806d780 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7fa194000c00 tx=0x7fa19400a000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:58:59.414 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:59.410+0000 7fa199ffb700 1 -- 172.21.15.59:0/87332636 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fa19c054840 con 0x7fa1a415ed00 2023-11-24T07:58:59.659 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:59.658+0000 7fa1ab806700 1 -- 172.21.15.59:0/87332636 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fa1a4067a40 con 0x7fa1a415ed00 2023-11-24T07:58:59.660 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:59.658+0000 7fa199ffb700 1 -- 172.21.15.59:0/87332636 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 36 v36) v1 ==== 94+0+4907 (secure 0 0 0) 0x7fa1a4067a40 con 0x7fa1a415ed00 2023-11-24T07:58:59.660 INFO:teuthology.orchestra.run.smithi059.stdout: 2023-11-24T07:58:59.661 INFO:teuthology.orchestra.run.smithi059.stdout:{"epoch":36,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5300,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6834/172682993","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6833","nonce":172682993},{"type":"v1","addr":"172.21.15.59:6834","nonce":172682993}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":29},{"gid":5701,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.76:6836/923821877","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6834","nonce":923821877},{"type":"v1","addr":"172.21.15.76:6836","nonce":923821877}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":30},{"gid":5704,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.76:6837/4166482716","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6835","nonce":4166482716},{"type":"v1","addr":"172.21.15.76:6837","nonce":4166482716}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":29}],"filesystems":[{"mdsmap":{"epoch":35,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-24T07:58:40.795340+0000","modified":"2023-11-24T07:58:43.812382+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":6093},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6093":{"gid":6093,"name":"a","rank":0,"incarnation":31,"state":"up:active","state_seq":3,"addr":"172.21.15.59:6835/1599845006","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6832","nonce":1599845006},{"type":"v1","addr":"172.21.15.59:6835","nonce":1599845006}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[9],"metadata_pool":8,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":4}]} 2023-11-24T07:58:59.662 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:59.661+0000 7fa1ab806700 1 -- 172.21.15.59:0/87332636 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7fa18806b2c0 msgr2=0x7fa18806d780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:59.663 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:59.661+0000 7fa1ab806700 1 --2- 172.21.15.59:0/87332636 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7fa18806b2c0 0x7fa18806d780 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7fa194000c00 tx=0x7fa19400a000 comp rx=0 tx=0).stop 2023-11-24T07:58:59.663 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:59.661+0000 7fa1ab806700 1 -- 172.21.15.59:0/87332636 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa1a415ed00 msgr2=0x7fa1a40aa5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:59.663 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:59.661+0000 7fa1ab806700 1 --2- 172.21.15.59:0/87332636 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa1a415ed00 0x7fa1a40aa5d0 secure :-1 s=READY pgs=788 cs=0 l=1 rev1=1 crypto rx=0x7fa1a415c410 tx=0x7fa19c0104a0 comp rx=0 tx=0).stop 2023-11-24T07:58:59.663 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:59.661+0000 7fa1ab806700 1 -- 172.21.15.59:0/87332636 shutdown_connections 2023-11-24T07:58:59.664 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:59.661+0000 7fa1ab806700 1 --2- 172.21.15.59:0/87332636 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7fa18806b2c0 0x7fa18806d780 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:59.664 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:59.661+0000 7fa1ab806700 1 --2- 172.21.15.59:0/87332636 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa1a415ed00 0x7fa1a40aa5d0 unknown :-1 s=CLOSED pgs=788 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:59.664 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:59.661+0000 7fa1ab806700 1 --2- 172.21.15.59:0/87332636 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fa190003a00 0x7fa1a41f19f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:59.664 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:59.661+0000 7fa1ab806700 1 --2- 172.21.15.59:0/87332636 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa1a415e880 0x7fa1a40aa090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:59.664 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:59.661+0000 7fa1ab806700 1 -- 172.21.15.59:0/87332636 >> 172.21.15.59:0/87332636 conn(0x7fa1a40b2740 msgr2=0x7fa1a4072510 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:58:59.664 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:59.661+0000 7fa1ab806700 1 -- 172.21.15.59:0/87332636 shutdown_connections 2023-11-24T07:58:59.665 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:59.661+0000 7fa1ab806700 1 -- 172.21.15.59:0/87332636 wait complete. 2023-11-24T07:58:59.665 INFO:teuthology.orchestra.run.smithi059.stderr:dumped fsmap epoch 36 2023-11-24T07:58:59.675 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-24T07:58:59.799 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:59.798+0000 7f7a87d7e700 1 Processor -- start 2023-11-24T07:58:59.800 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:59.798+0000 7f7a87d7e700 1 -- start start 2023-11-24T07:58:59.801 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:59.799+0000 7f7a87d7e700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f7a8015e000 0x7f7a8006eef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:59.802 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:59.799+0000 7f7a87d7e700 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7a800cfd60 con 0x7f7a8006fe00 2023-11-24T07:58:59.802 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:59.799+0000 7f7a87d7e700 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7a8006f430 con 0x7f7a800d0ae0 2023-11-24T07:58:59.802 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:59.799+0000 7f7a87d7e700 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7a800cf4b0 con 0x7f7a8015e000 2023-11-24T07:58:59.802 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:59.799+0000 7f7a85b1a700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f7a8015e000 0x7f7a8006eef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:59.802 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:59.799+0000 7f7a85b1a700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f7a8015e000 0x7f7a8006eef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:54372/0 (socket says 172.21.15.59:54372) 2023-11-24T07:58:59.802 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:59.799+0000 7f7a85b1a700 1 -- 172.21.15.59:0/718835506 learned_addr learned my addr 172.21.15.59:0/718835506 (peer_addr_for_me v2:172.21.15.59:0/0) 2023-11-24T07:58:59.802 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:59.799+0000 7f7a85b1a700 1 -- 172.21.15.59:0/718835506 >> v1:172.21.15.76:6789/0 conn(0x7f7a800d0ae0 legacy=0x7f7a8006f6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T07:58:59.802 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:59.799+0000 7f7a85b1a700 1 -- 172.21.15.59:0/718835506 >> v1:172.21.15.59:6789/0 conn(0x7f7a8006fe00 legacy=0x7f7a800cc220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:59.802 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:59.799+0000 7f7a85b1a700 1 -- 172.21.15.59:0/718835506 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7a800cf1d0 con 0x7f7a8015e000 2023-11-24T07:58:59.802 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:59.799+0000 7f7a84b18700 1 -- 172.21.15.59:0/718835506 <== mon.0 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3063636579 0 0) 0x7f7a800cfd60 con 0x7f7a8006fe00 2023-11-24T07:58:59.803 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:59.799+0000 7f7a85b1a700 1 --2- 172.21.15.59:0/718835506 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7a8015e000 0x7f7a8006eef0 secure :-1 s=READY pgs=790 cs=0 l=1 rev1=1 crypto rx=0x7f7a7c0098f0 tx=0x7f7a7c009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c523481f0a1e10aa server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:58:59.803 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:59.799+0000 7f7a84b18700 1 -- 172.21.15.59:0/718835506 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a7c004230 con 0x7f7a8015e000 2023-11-24T07:58:59.803 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:59.799+0000 7f7a84b18700 1 -- 172.21.15.59:0/718835506 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7a7c004390 con 0x7f7a8015e000 2023-11-24T07:58:59.803 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:59.799+0000 7f7a84b18700 1 -- 172.21.15.59:0/718835506 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a7c0036a0 con 0x7f7a8015e000 2023-11-24T07:58:59.803 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:59.800+0000 7f7a87d7e700 1 -- 172.21.15.59:0/718835506 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7a8015e000 msgr2=0x7f7a8006eef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:59.803 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:59.800+0000 7f7a87d7e700 1 --2- 172.21.15.59:0/718835506 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7a8015e000 0x7f7a8006eef0 secure :-1 s=READY pgs=790 cs=0 l=1 rev1=1 crypto rx=0x7f7a7c0098f0 tx=0x7f7a7c009c00 comp rx=0 tx=0).stop 2023-11-24T07:58:59.803 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:59.800+0000 7f7a87d7e700 1 -- 172.21.15.59:0/718835506 shutdown_connections 2023-11-24T07:58:59.803 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:59.800+0000 7f7a87d7e700 1 --2- 172.21.15.59:0/718835506 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7a8015e000 0x7f7a8006eef0 unknown :-1 s=CLOSED pgs=790 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:59.803 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:59.800+0000 7f7a87d7e700 1 -- 172.21.15.59:0/718835506 >> 172.21.15.59:0/718835506 conn(0x7f7a800b3720 msgr2=0x7f7a800b3b30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:58:59.803 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:59.800+0000 7f7a87d7e700 1 -- 172.21.15.59:0/718835506 shutdown_connections 2023-11-24T07:58:59.803 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:59.800+0000 7f7a87d7e700 1 -- 172.21.15.59:0/718835506 wait complete. 2023-11-24T07:58:59.806 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:59.801+0000 7f7a87d7e700 1 Processor -- start 2023-11-24T07:58:59.806 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:59.801+0000 7f7a87d7e700 1 -- start start 2023-11-24T07:58:59.807 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:59.802+0000 7f7a87d7e700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7a8006fe00 0x7f7a80152a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:59.807 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:59.802+0000 7f7a87d7e700 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f7a800d0ae0 0x7f7a80160d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:59.807 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:59.802+0000 7f7a87d7e700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7a8015e000 0x7f7a800c7760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:59.807 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:59.802+0000 7f7a87d7e700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a80153330 con 0x7f7a8015e000 2023-11-24T07:58:59.807 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:59.802+0000 7f7a87d7e700 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a80161280 con 0x7f7a8006fe00 2023-11-24T07:58:59.807 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:59.802+0000 7f7a87d7e700 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7a800b77a0 con 0x7f7a800d0ae0 2023-11-24T07:58:59.807 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:59.802+0000 7f7a8631b700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7a8015e000 0x7f7a800c7760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:59.807 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:59.802+0000 7f7a8631b700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7a8015e000 0x7f7a800c7760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:54382/0 (socket says 172.21.15.59:54382) 2023-11-24T07:58:59.807 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:59.802+0000 7f7a8631b700 1 -- 172.21.15.59:0/1299547803 learned_addr learned my addr 172.21.15.59:0/1299547803 (peer_addr_for_me v2:172.21.15.59:0/0) 2023-11-24T07:58:59.807 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:59.802+0000 7f7a85b1a700 1 --2- 172.21.15.59:0/1299547803 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7a8006fe00 0x7f7a80152a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:59.807 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:59.802+0000 7f7a85319700 1 --2- 172.21.15.59:0/1299547803 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f7a800d0ae0 0x7f7a80160d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:59.807 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:59.803+0000 7f7a8631b700 1 -- 172.21.15.59:0/1299547803 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f7a800d0ae0 msgr2=0x7f7a80160d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:59.808 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:59.803+0000 7f7a8631b700 1 --2- 172.21.15.59:0/1299547803 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f7a800d0ae0 0x7f7a80160d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:59.808 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:59.803+0000 7f7a8631b700 1 -- 172.21.15.59:0/1299547803 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7a8006fe00 msgr2=0x7f7a80152a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:58:59.808 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:59.803+0000 7f7a8631b700 1 --2- 172.21.15.59:0/1299547803 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7a8006fe00 0x7f7a80152a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:58:59.808 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:59.803+0000 7f7a8631b700 1 -- 172.21.15.59:0/1299547803 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7a7c009550 con 0x7f7a8015e000 2023-11-24T07:58:59.808 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:59.803+0000 7f7a85319700 1 --2- 172.21.15.59:0/1299547803 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f7a800d0ae0 0x7f7a80160d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T07:58:59.808 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:59.803+0000 7f7a8631b700 1 --2- 172.21.15.59:0/1299547803 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7a8015e000 0x7f7a800c7760 secure :-1 s=READY pgs=791 cs=0 l=1 rev1=1 crypto rx=0x7f7a7800c780 tx=0x7f7a7800cb40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:58:59.808 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:59.803+0000 7f7a767fc700 1 -- 172.21.15.59:0/1299547803 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a78003e30 con 0x7f7a8015e000 2023-11-24T07:58:59.808 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:59.803+0000 7f7a767fc700 1 -- 172.21.15.59:0/1299547803 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7a78005070 con 0x7f7a8015e000 2023-11-24T07:58:59.808 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:59.803+0000 7f7a87d7e700 1 -- 172.21.15.59:0/1299547803 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7a8015ee90 con 0x7f7a8015e000 2023-11-24T07:58:59.808 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:59.803+0000 7f7a767fc700 1 -- 172.21.15.59:0/1299547803 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a78005420 con 0x7f7a8015e000 2023-11-24T07:58:59.808 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:59.803+0000 7f7a87d7e700 1 -- 172.21.15.59:0/1299547803 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7a8015f3e0 con 0x7f7a8015e000 2023-11-24T07:58:59.809 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:59.804+0000 7f7a87d7e700 1 -- 172.21.15.59:0/1299547803 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7a800cead0 con 0x7f7a8015e000 2023-11-24T07:58:59.809 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:59.804+0000 7f7a767fc700 1 -- 172.21.15.59:0/1299547803 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 10) v1 ==== 89588+0+0 (secure 0 0 0) 0x7f7a78005580 con 0x7f7a8015e000 2023-11-24T07:58:59.809 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:59.805+0000 7f7a767fc700 1 --2- 172.21.15.59:0/1299547803 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f7a6406b2c0 0x7f7a6406d780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:58:59.809 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:59.805+0000 7f7a767fc700 1 -- 172.21.15.59:0/1299547803 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(61..61 src has 1..61) v4 ==== 5765+0+0 (secure 0 0 0) 0x7f7a78089970 con 0x7f7a8015e000 2023-11-24T07:58:59.809 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:59.805+0000 7f7a85b1a700 1 --2- 172.21.15.59:0/1299547803 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f7a6406b2c0 0x7f7a6406d780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:58:59.809 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:59.805+0000 7f7a85b1a700 1 --2- 172.21.15.59:0/1299547803 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f7a6406b2c0 0x7f7a6406d780 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f7a7c009130 tx=0x7f7a7c008030 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:58:59.811 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:58:59.809+0000 7f7a767fc700 1 -- 172.21.15.59:0/1299547803 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f7a78057f20 con 0x7f7a8015e000 2023-11-24T07:59:00.013 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:00.011+0000 7f7a87d7e700 1 -- 172.21.15.59:0/1299547803 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f7a80068670 con 0x7f7a8015e000 2023-11-24T07:59:00.014 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:00.012+0000 7f7a767fc700 1 -- 172.21.15.59:0/1299547803 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v61) v1 ==== 74+0+16601 (secure 0 0 0) 0x7f7a78057ab0 con 0x7f7a8015e000 2023-11-24T07:59:00.014 INFO:teuthology.orchestra.run.smithi059.stdout: 2023-11-24T07:59:00.015 INFO:teuthology.orchestra.run.smithi059.stdout:{"epoch":61,"fsid":"9f0e8741-b8f9-48d5-a2c3-09f45903303d","created":"2023-11-24T07:56:06.552729+0000","modified":"2023-11-24T07:58:57.458332+0000","last_up_change":"2023-11-24T07:56:17.372352+0000","last_in_change":"2023-11-24T07:56:09.073250+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-24T07:56:18.352823+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_metadata","create_time":"2023-11-24T07:58:39.262463+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"59","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_data","create_time":"2023-11-24T07:58:39.763399+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"61","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":3,"snap_epoch":61,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"6d8f9e9b-1782-4e41-928b-a39f2a8ebc06","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6800","nonce":3373878772},{"type":"v1","addr":"172.21.15.59:6801","nonce":3373878772}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6802","nonce":3373878772},{"type":"v1","addr":"172.21.15.59:6803","nonce":3373878772}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6806","nonce":3373878772},{"type":"v1","addr":"172.21.15.59:6807","nonce":3373878772}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6804","nonce":3373878772},{"type":"v1","addr":"172.21.15.59:6805","nonce":3373878772}]},"public_addr":"172.21.15.59:6801/3373878772","cluster_addr":"172.21.15.59:6803/3373878772","heartbeat_back_addr":"172.21.15.59:6807/3373878772","heartbeat_front_addr":"172.21.15.59:6805/3373878772","state":["exists","up"]},{"osd":1,"uuid":"e8d882aa-52e1-4c04-9e1e-c44ef9204a5f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6824","nonce":648258563},{"type":"v1","addr":"172.21.15.59:6825","nonce":648258563}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6826","nonce":648258563},{"type":"v1","addr":"172.21.15.59:6827","nonce":648258563}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6830","nonce":648258563},{"type":"v1","addr":"172.21.15.59:6831","nonce":648258563}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6828","nonce":648258563},{"type":"v1","addr":"172.21.15.59:6829","nonce":648258563}]},"public_addr":"172.21.15.59:6825/648258563","cluster_addr":"172.21.15.59:6827/648258563","heartbeat_back_addr":"172.21.15.59:6831/648258563","heartbeat_front_addr":"172.21.15.59:6829/648258563","state":["exists","up"]},{"osd":2,"uuid":"69eebfa0-695b-4b21-90b9-8a3537168a01","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6808","nonce":1840430654},{"type":"v1","addr":"172.21.15.59:6809","nonce":1840430654}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6810","nonce":1840430654},{"type":"v1","addr":"172.21.15.59:6811","nonce":1840430654}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6814","nonce":1840430654},{"type":"v1","addr":"172.21.15.59:6815","nonce":1840430654}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6812","nonce":1840430654},{"type":"v1","addr":"172.21.15.59:6813","nonce":1840430654}]},"public_addr":"172.21.15.59:6809/1840430654","cluster_addr":"172.21.15.59:6811/1840430654","heartbeat_back_addr":"172.21.15.59:6815/1840430654","heartbeat_front_addr":"172.21.15.59:6813/1840430654","state":["exists","up"]},{"osd":3,"uuid":"96fd77fc-f911-46ff-9445-6f0be2daa8f4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6816","nonce":2504025850},{"type":"v1","addr":"172.21.15.59:6817","nonce":2504025850}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6818","nonce":2504025850},{"type":"v1","addr":"172.21.15.59:6819","nonce":2504025850}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6822","nonce":2504025850},{"type":"v1","addr":"172.21.15.59:6823","nonce":2504025850}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6820","nonce":2504025850},{"type":"v1","addr":"172.21.15.59:6821","nonce":2504025850}]},"public_addr":"172.21.15.59:6817/2504025850","cluster_addr":"172.21.15.59:6819/2504025850","heartbeat_back_addr":"172.21.15.59:6823/2504025850","heartbeat_front_addr":"172.21.15.59:6821/2504025850","state":["exists","up"]},{"osd":4,"uuid":"7ac29fca-b5b2-4c95-9d69-2f164cbbc708","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6816","nonce":2382463431},{"type":"v1","addr":"172.21.15.76:6818","nonce":2382463431}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6820","nonce":2382463431},{"type":"v1","addr":"172.21.15.76:6822","nonce":2382463431}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6828","nonce":2382463431},{"type":"v1","addr":"172.21.15.76:6830","nonce":2382463431}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6824","nonce":2382463431},{"type":"v1","addr":"172.21.15.76:6826","nonce":2382463431}]},"public_addr":"172.21.15.76:6818/2382463431","cluster_addr":"172.21.15.76:6822/2382463431","heartbeat_back_addr":"172.21.15.76:6830/2382463431","heartbeat_front_addr":"172.21.15.76:6826/2382463431","state":["exists","up"]},{"osd":5,"uuid":"485bf711-3851-4238-a4f4-51d52e0a9c58","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6800","nonce":329334308},{"type":"v1","addr":"172.21.15.76:6802","nonce":329334308}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6805","nonce":329334308},{"type":"v1","addr":"172.21.15.76:6807","nonce":329334308}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6814","nonce":329334308},{"type":"v1","addr":"172.21.15.76:6815","nonce":329334308}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6810","nonce":329334308},{"type":"v1","addr":"172.21.15.76:6812","nonce":329334308}]},"public_addr":"172.21.15.76:6802/329334308","cluster_addr":"172.21.15.76:6807/329334308","heartbeat_back_addr":"172.21.15.76:6815/329334308","heartbeat_front_addr":"172.21.15.76:6812/329334308","state":["exists","up"]},{"osd":6,"uuid":"72b3f2fc-1e4a-4e32-968d-003647a0286f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6801","nonce":3946579410},{"type":"v1","addr":"172.21.15.76:6803","nonce":3946579410}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6804","nonce":3946579410},{"type":"v1","addr":"172.21.15.76:6806","nonce":3946579410}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6811","nonce":3946579410},{"type":"v1","addr":"172.21.15.76:6813","nonce":3946579410}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6808","nonce":3946579410},{"type":"v1","addr":"172.21.15.76:6809","nonce":3946579410}]},"public_addr":"172.21.15.76:6803/3946579410","cluster_addr":"172.21.15.76:6806/3946579410","heartbeat_back_addr":"172.21.15.76:6813/3946579410","heartbeat_front_addr":"172.21.15.76:6809/3946579410","state":["exists","up"]},{"osd":7,"uuid":"1ddceb90-6c5d-4d1a-ad56-99cb877f8c70","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6817","nonce":841761538},{"type":"v1","addr":"172.21.15.76:6819","nonce":841761538}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6821","nonce":841761538},{"type":"v1","addr":"172.21.15.76:6823","nonce":841761538}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6829","nonce":841761538},{"type":"v1","addr":"172.21.15.76:6831","nonce":841761538}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6825","nonce":841761538},{"type":"v1","addr":"172.21.15.76:6827","nonce":841761538}]},"public_addr":"172.21.15.76:6819/841761538","cluster_addr":"172.21.15.76:6823/841761538","heartbeat_back_addr":"172.21.15.76:6831/841761538","heartbeat_front_addr":"172.21.15.76:6827/841761538","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T07:56:14.294838+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T07:56:14.270779+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T07:56:14.231809+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T07:56:14.260983+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T07:56:14.302997+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T07:56:14.258600+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T07:56:14.251413+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T07:56:14.304727+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":9,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[{"pool":9,"snaps":[{"begin":2,"length":2}]}],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-24T07:59:00.016 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:00.014+0000 7f7a87d7e700 1 -- 172.21.15.59:0/1299547803 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f7a6406b2c0 msgr2=0x7f7a6406d780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:00.016 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:00.014+0000 7f7a87d7e700 1 --2- 172.21.15.59:0/1299547803 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f7a6406b2c0 0x7f7a6406d780 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f7a7c009130 tx=0x7f7a7c008030 comp rx=0 tx=0).stop 2023-11-24T07:59:00.017 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:00.014+0000 7f7a87d7e700 1 -- 172.21.15.59:0/1299547803 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7a8015e000 msgr2=0x7f7a800c7760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:00.017 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:00.014+0000 7f7a87d7e700 1 --2- 172.21.15.59:0/1299547803 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7a8015e000 0x7f7a800c7760 secure :-1 s=READY pgs=791 cs=0 l=1 rev1=1 crypto rx=0x7f7a7800c780 tx=0x7f7a7800cb40 comp rx=0 tx=0).stop 2023-11-24T07:59:00.017 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:00.014+0000 7f7a87d7e700 1 -- 172.21.15.59:0/1299547803 shutdown_connections 2023-11-24T07:59:00.017 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:00.014+0000 7f7a87d7e700 1 --2- 172.21.15.59:0/1299547803 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f7a6406b2c0 0x7f7a6406d780 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:00.017 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:00.014+0000 7f7a87d7e700 1 --2- 172.21.15.59:0/1299547803 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7a8015e000 0x7f7a800c7760 unknown :-1 s=CLOSED pgs=791 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:00.018 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:00.014+0000 7f7a87d7e700 1 --2- 172.21.15.59:0/1299547803 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7a8006fe00 0x7f7a80152a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:00.018 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:00.014+0000 7f7a87d7e700 1 --2- 172.21.15.59:0/1299547803 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f7a800d0ae0 0x7f7a80160d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:00.018 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:00.014+0000 7f7a87d7e700 1 -- 172.21.15.59:0/1299547803 >> 172.21.15.59:0/1299547803 conn(0x7f7a800b3720 msgr2=0x7f7a800b0f20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:59:00.018 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:00.014+0000 7f7a87d7e700 1 -- 172.21.15.59:0/1299547803 shutdown_connections 2023-11-24T07:59:00.018 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:00.014+0000 7f7a87d7e700 1 -- 172.21.15.59:0/1299547803 wait complete. 2023-11-24T07:59:00.028 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2023-11-24T07:59:00.029 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-24T07:59:00.153 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:00.152+0000 7f8cc3082700 1 Processor -- start 2023-11-24T07:59:00.154 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:00.152+0000 7f8cc3082700 1 -- start start 2023-11-24T07:59:00.156 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:00.152+0000 7f8cc3082700 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f8cbc143ae0 0x7f8cbc0b51b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:00.156 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:00.152+0000 7f8cc3082700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f8cbc0722c0 0x7f8cbc06e510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:00.156 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:00.152+0000 7f8cc3082700 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8cbc143ec0 con 0x7f8cbc1440f0 2023-11-24T07:59:00.156 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:00.152+0000 7f8cc3082700 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8cbc0b40a0 con 0x7f8cbc0722c0 2023-11-24T07:59:00.157 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:00.153+0000 7f8cc3082700 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8cbc0b3ae0 con 0x7f8cbc143ae0 2023-11-24T07:59:00.157 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:00.153+0000 7f8cc161f700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f8cbc0722c0 0x7f8cbc06e510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:00.157 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:00.153+0000 7f8cc161f700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f8cbc0722c0 0x7f8cbc06e510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:54390/0 (socket says 172.21.15.59:54390) 2023-11-24T07:59:00.157 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:00.153+0000 7f8cc161f700 1 -- 172.21.15.59:0/2895932629 learned_addr learned my addr 172.21.15.59:0/2895932629 (peer_addr_for_me v2:172.21.15.59:0/0) 2023-11-24T07:59:00.157 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:00.153+0000 7f8cc0e1e700 1 --2- 172.21.15.59:0/2895932629 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f8cbc143ae0 0x7f8cbc0b51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:00.157 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:00.153+0000 7f8cc161f700 1 -- 172.21.15.59:0/2895932629 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f8cbc143ae0 msgr2=0x7f8cbc0b51b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T07:59:00.157 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:00.153+0000 7f8cc161f700 1 --2- 172.21.15.59:0/2895932629 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f8cbc143ae0 0x7f8cbc0b51b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:00.157 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:00.153+0000 7f8cc161f700 1 -- 172.21.15.59:0/2895932629 >> v1:172.21.15.76:6789/0 conn(0x7f8cbc1440f0 legacy=0x7f8cbc071bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:00.158 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:00.153+0000 7f8cc161f700 1 -- 172.21.15.59:0/2895932629 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8cbc0b3dc0 con 0x7f8cbc0722c0 2023-11-24T07:59:00.158 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:00.153+0000 7f8cc161f700 1 --2- 172.21.15.59:0/2895932629 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8cbc0722c0 0x7f8cbc06e510 secure :-1 s=READY pgs=792 cs=0 l=1 rev1=1 crypto rx=0x7f8cb400d150 tx=0x7f8cb400d460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a930f2a7745495a8 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:59:00.158 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:00.153+0000 7f8cbb7fe700 1 -- 172.21.15.59:0/2895932629 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8cb4010070 con 0x7f8cbc0722c0 2023-11-24T07:59:00.158 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:00.153+0000 7f8cbb7fe700 1 -- 172.21.15.59:0/2895932629 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8cb400ddc0 con 0x7f8cbc0722c0 2023-11-24T07:59:00.159 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:00.153+0000 7f8cbb7fe700 1 -- 172.21.15.59:0/2895932629 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8cb400b310 con 0x7f8cbc0722c0 2023-11-24T07:59:00.160 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:00.154+0000 7f8cc3082700 1 -- 172.21.15.59:0/2895932629 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8cbc0722c0 msgr2=0x7f8cbc06e510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:00.160 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:00.154+0000 7f8cc3082700 1 --2- 172.21.15.59:0/2895932629 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8cbc0722c0 0x7f8cbc06e510 secure :-1 s=READY pgs=792 cs=0 l=1 rev1=1 crypto rx=0x7f8cb400d150 tx=0x7f8cb400d460 comp rx=0 tx=0).stop 2023-11-24T07:59:00.160 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:00.154+0000 7f8cc3082700 1 -- 172.21.15.59:0/2895932629 shutdown_connections 2023-11-24T07:59:00.160 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:00.154+0000 7f8cc3082700 1 --2- 172.21.15.59:0/2895932629 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f8cbc143ae0 0x7f8cbc0b51b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:00.161 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:00.154+0000 7f8cc3082700 1 --2- 172.21.15.59:0/2895932629 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8cbc0722c0 0x7f8cbc06e510 unknown :-1 s=CLOSED pgs=792 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:00.161 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:00.154+0000 7f8cc3082700 1 -- 172.21.15.59:0/2895932629 >> 172.21.15.59:0/2895932629 conn(0x7f8cbc0b2740 msgr2=0x7f8cbc0b2b20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:59:00.161 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:00.154+0000 7f8cc3082700 1 -- 172.21.15.59:0/2895932629 shutdown_connections 2023-11-24T07:59:00.162 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:00.154+0000 7f8cc3082700 1 -- 172.21.15.59:0/2895932629 wait complete. 2023-11-24T07:59:00.164 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:00.155+0000 7f8cc3082700 1 Processor -- start 2023-11-24T07:59:00.164 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:00.155+0000 7f8cc3082700 1 -- start start 2023-11-24T07:59:00.164 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:00.156+0000 7f8cc3082700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8cbc0722c0 0x7f8cbc0cce70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:00.164 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:00.156+0000 7f8cc3082700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8cbc143ae0 0x7f8cbc1581b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:00.164 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:00.156+0000 7f8cc3082700 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f8cbc1440f0 0x7f8cbc1586f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:00.164 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:00.156+0000 7f8cc3082700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8cbc1f3b70 con 0x7f8cbc143ae0 2023-11-24T07:59:00.164 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:00.156+0000 7f8cc3082700 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8cbc0cd3b0 con 0x7f8cbc0722c0 2023-11-24T07:59:00.164 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:00.156+0000 7f8cc3082700 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8cbc1f3890 con 0x7f8cbc1440f0 2023-11-24T07:59:00.165 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:00.156+0000 7f8cbbfff700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8cbc143ae0 0x7f8cbc1581b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:00.165 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:00.156+0000 7f8cbbfff700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8cbc143ae0 0x7f8cbc1581b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:54394/0 (socket says 172.21.15.59:54394) 2023-11-24T07:59:00.165 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:00.156+0000 7f8cbbfff700 1 -- 172.21.15.59:0/3986964008 learned_addr learned my addr 172.21.15.59:0/3986964008 (peer_addr_for_me v2:172.21.15.59:0/0) 2023-11-24T07:59:00.165 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:00.156+0000 7f8cc0e1e700 1 --2- 172.21.15.59:0/3986964008 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8cbc0722c0 0x7f8cbc0cce70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:00.165 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:00.156+0000 7f8cc161f700 1 --2- 172.21.15.59:0/3986964008 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f8cbc1440f0 0x7f8cbc1586f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:00.165 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:00.157+0000 7f8cbbfff700 1 -- 172.21.15.59:0/3986964008 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f8cbc1440f0 msgr2=0x7f8cbc1586f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:00.165 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:00.157+0000 7f8cbbfff700 1 --2- 172.21.15.59:0/3986964008 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f8cbc1440f0 0x7f8cbc1586f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:00.165 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:00.157+0000 7f8cbbfff700 1 -- 172.21.15.59:0/3986964008 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8cbc0722c0 msgr2=0x7f8cbc0cce70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:00.165 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:00.157+0000 7f8cbbfff700 1 --2- 172.21.15.59:0/3986964008 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8cbc0722c0 0x7f8cbc0cce70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:00.165 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:00.157+0000 7f8cbbfff700 1 -- 172.21.15.59:0/3986964008 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8cb400bd40 con 0x7f8cbc143ae0 2023-11-24T07:59:00.165 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:00.157+0000 7f8cbbfff700 1 --2- 172.21.15.59:0/3986964008 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8cbc143ae0 0x7f8cbc1581b0 secure :-1 s=READY pgs=793 cs=0 l=1 rev1=1 crypto rx=0x7f8ca800c450 tx=0x7f8ca800c760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:59:00.165 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:00.157+0000 7f8cb97fa700 1 -- 172.21.15.59:0/3986964008 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ca80052d0 con 0x7f8cbc143ae0 2023-11-24T07:59:00.166 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:00.157+0000 7f8cb97fa700 1 -- 172.21.15.59:0/3986964008 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8ca8005430 con 0x7f8cbc143ae0 2023-11-24T07:59:00.166 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:00.157+0000 7f8cb97fa700 1 -- 172.21.15.59:0/3986964008 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ca80038c0 con 0x7f8cbc143ae0 2023-11-24T07:59:00.166 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:00.157+0000 7f8cc3082700 1 -- 172.21.15.59:0/3986964008 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8cbc0c5b50 con 0x7f8cbc143ae0 2023-11-24T07:59:00.166 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:00.157+0000 7f8cc3082700 1 -- 172.21.15.59:0/3986964008 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8cbc0c6110 con 0x7f8cbc143ae0 2023-11-24T07:59:00.166 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:00.158+0000 7f8cc3082700 1 -- 172.21.15.59:0/3986964008 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8cbc0c62d0 con 0x7f8cbc143ae0 2023-11-24T07:59:00.166 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:00.158+0000 7f8cb97fa700 1 -- 172.21.15.59:0/3986964008 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 10) v1 ==== 89588+0+0 (secure 0 0 0) 0x7f8ca8003a20 con 0x7f8cbc143ae0 2023-11-24T07:59:00.166 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:00.159+0000 7f8cb97fa700 1 --2- 172.21.15.59:0/3986964008 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f8ca006b2c0 0x7f8ca006d780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:00.166 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:00.159+0000 7f8cb97fa700 1 -- 172.21.15.59:0/3986964008 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(61..61 src has 1..61) v4 ==== 5765+0+0 (secure 0 0 0) 0x7f8ca8089080 con 0x7f8cbc143ae0 2023-11-24T07:59:00.166 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:00.159+0000 7f8cc0e1e700 1 --2- 172.21.15.59:0/3986964008 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f8ca006b2c0 0x7f8ca006d780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:00.167 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:00.160+0000 7f8cc0e1e700 1 --2- 172.21.15.59:0/3986964008 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f8ca006b2c0 0x7f8ca006d780 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f8cb0006b20 tx=0x7f8cb0008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:59:00.167 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:00.163+0000 7f8cb97fa700 1 -- 172.21.15.59:0/3986964008 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f8ca80bbe90 con 0x7f8cbc143ae0 2023-11-24T07:59:00.418 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:00.417+0000 7f8cc3082700 1 -- 172.21.15.59:0/3986964008 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f8cbc0c59b0 con 0x7f8cbc143ae0 2023-11-24T07:59:00.419 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:00.417+0000 7f8cb97fa700 1 -- 172.21.15.59:0/3986964008 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 36 v36) v1 ==== 94+0+4907 (secure 0 0 0) 0x7f8cbc0c59b0 con 0x7f8cbc143ae0 2023-11-24T07:59:00.419 INFO:teuthology.orchestra.run.smithi059.stdout: 2023-11-24T07:59:00.419 INFO:teuthology.orchestra.run.smithi059.stdout:{"epoch":36,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5300,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6834/172682993","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6833","nonce":172682993},{"type":"v1","addr":"172.21.15.59:6834","nonce":172682993}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":29},{"gid":5701,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.76:6836/923821877","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6834","nonce":923821877},{"type":"v1","addr":"172.21.15.76:6836","nonce":923821877}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":30},{"gid":5704,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.76:6837/4166482716","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6835","nonce":4166482716},{"type":"v1","addr":"172.21.15.76:6837","nonce":4166482716}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":29}],"filesystems":[{"mdsmap":{"epoch":35,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-24T07:58:40.795340+0000","modified":"2023-11-24T07:58:43.812382+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":6093},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6093":{"gid":6093,"name":"a","rank":0,"incarnation":31,"state":"up:active","state_seq":3,"addr":"172.21.15.59:6835/1599845006","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6832","nonce":1599845006},{"type":"v1","addr":"172.21.15.59:6835","nonce":1599845006}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[9],"metadata_pool":8,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":4}]} 2023-11-24T07:59:00.421 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:00.419+0000 7f8cc3082700 1 -- 172.21.15.59:0/3986964008 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f8ca006b2c0 msgr2=0x7f8ca006d780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:00.421 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:00.419+0000 7f8cc3082700 1 --2- 172.21.15.59:0/3986964008 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f8ca006b2c0 0x7f8ca006d780 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f8cb0006b20 tx=0x7f8cb0008040 comp rx=0 tx=0).stop 2023-11-24T07:59:00.421 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:00.419+0000 7f8cc3082700 1 -- 172.21.15.59:0/3986964008 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8cbc143ae0 msgr2=0x7f8cbc1581b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:00.422 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:00.419+0000 7f8cc3082700 1 --2- 172.21.15.59:0/3986964008 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8cbc143ae0 0x7f8cbc1581b0 secure :-1 s=READY pgs=793 cs=0 l=1 rev1=1 crypto rx=0x7f8ca800c450 tx=0x7f8ca800c760 comp rx=0 tx=0).stop 2023-11-24T07:59:00.422 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:00.419+0000 7f8cc3082700 1 -- 172.21.15.59:0/3986964008 shutdown_connections 2023-11-24T07:59:00.422 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:00.419+0000 7f8cc3082700 1 --2- 172.21.15.59:0/3986964008 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f8ca006b2c0 0x7f8ca006d780 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:00.422 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:00.419+0000 7f8cc3082700 1 --2- 172.21.15.59:0/3986964008 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f8cbc1440f0 0x7f8cbc1586f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:00.422 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:00.419+0000 7f8cc3082700 1 --2- 172.21.15.59:0/3986964008 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8cbc0722c0 0x7f8cbc0cce70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:00.422 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:00.419+0000 7f8cc3082700 1 --2- 172.21.15.59:0/3986964008 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8cbc143ae0 0x7f8cbc1581b0 unknown :-1 s=CLOSED pgs=793 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:00.422 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:00.419+0000 7f8cc3082700 1 -- 172.21.15.59:0/3986964008 >> 172.21.15.59:0/3986964008 conn(0x7f8cbc0b2740 msgr2=0x7f8cbc0cf1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:59:00.422 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:00.419+0000 7f8cc3082700 1 -- 172.21.15.59:0/3986964008 shutdown_connections 2023-11-24T07:59:00.423 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:00.419+0000 7f8cc3082700 1 -- 172.21.15.59:0/3986964008 wait complete. 2023-11-24T07:59:00.423 INFO:teuthology.orchestra.run.smithi059.stderr:dumped fsmap epoch 36 2023-11-24T07:59:00.433 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-24T07:59:00.558 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:00.557+0000 7fb2a7b5e700 1 Processor -- start 2023-11-24T07:59:00.559 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:00.557+0000 7fb2a7b5e700 1 -- start start 2023-11-24T07:59:00.560 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:00.558+0000 7fb2a7b5e700 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fb2a00b4150 0x7fb2a006de50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:00.560 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:00.558+0000 7fb2a7b5e700 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb2a0071f50 con 0x7fb2a00b5ab0 2023-11-24T07:59:00.561 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:00.558+0000 7fb2a7b5e700 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb2a0072230 con 0x7fb2a006b620 2023-11-24T07:59:00.561 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:00.558+0000 7fb2a7b5e700 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb2a0072510 con 0x7fb2a00b4150 2023-11-24T07:59:00.561 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:00.558+0000 7fb2a60fb700 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7fb2a00b5ab0 0x7fb2a006e390 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:57878/0 (socket says 172.21.15.59:57878) 2023-11-24T07:59:00.561 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:00.558+0000 7fb2a60fb700 1 -- 172.21.15.59:0/3595880801 learned_addr learned my addr 172.21.15.59:0/3595880801 (peer_addr_for_me v1:172.21.15.59:0/0) 2023-11-24T07:59:00.561 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:00.558+0000 7fb2a50f9700 1 --2- 172.21.15.59:0/3595880801 >> v2:172.21.15.76:3300/0 conn(0x7fb2a00b4150 0x7fb2a006de50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:00.561 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:00.558+0000 7fb2a48f8700 1 -- 172.21.15.59:0/3595880801 <== mon.0 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1691484306 0 0) 0x7fb2a0071f50 con 0x7fb2a00b5ab0 2023-11-24T07:59:00.561 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:00.558+0000 7fb2a48f8700 1 -- 172.21.15.59:0/3595880801 --> v1:172.21.15.59:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb28c0035b0 con 0x7fb2a00b5ab0 2023-11-24T07:59:00.561 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:00.559+0000 7fb2a50f9700 1 -- 172.21.15.59:0/3595880801 >> v1:172.21.15.76:6789/0 conn(0x7fb2a006b620 legacy=0x7fb2a00b3a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:00.561 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:00.559+0000 7fb2a50f9700 1 -- 172.21.15.59:0/3595880801 >> v1:172.21.15.59:6789/0 conn(0x7fb2a00b5ab0 legacy=0x7fb2a006e390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:00.561 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:00.559+0000 7fb2a48f8700 1 -- 172.21.15.59:0/3595880801 <== mon.0 v1:172.21.15.59:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2677033373 0 0) 0x7fb28c0035b0 con 0x7fb2a00b5ab0 2023-11-24T07:59:00.561 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:00.559+0000 7fb2a50f9700 1 -- 172.21.15.59:0/3595880801 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb2a00727f0 con 0x7fb2a00b4150 2023-11-24T07:59:00.561 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:00.559+0000 7fb2a50f9700 1 --2- 172.21.15.59:0/3595880801 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb2a00b4150 0x7fb2a006de50 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7fb290009840 tx=0x7fb290009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=47ada9e116228e29 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:59:00.562 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:00.559+0000 7fb2a48f8700 1 -- 172.21.15.59:0/3595880801 <== mon.1 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2980019f0 con 0x7fb2a00b4150 2023-11-24T07:59:00.562 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:00.559+0000 7fb2a48f8700 1 -- 172.21.15.59:0/3595880801 <== mon.1 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb290004030 con 0x7fb2a00b4150 2023-11-24T07:59:00.562 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:00.559+0000 7fb2a48f8700 1 -- 172.21.15.59:0/3595880801 <== mon.1 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2900043c0 con 0x7fb2a00b4150 2023-11-24T07:59:00.564 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:00.559+0000 7fb2a7b5e700 1 -- 172.21.15.59:0/3595880801 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb2a00b4150 msgr2=0x7fb2a006de50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:00.564 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:00.559+0000 7fb2a7b5e700 1 --2- 172.21.15.59:0/3595880801 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb2a00b4150 0x7fb2a006de50 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7fb290009840 tx=0x7fb290009b50 comp rx=0 tx=0).stop 2023-11-24T07:59:00.565 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:00.559+0000 7fb2a7b5e700 1 -- 172.21.15.59:0/3595880801 shutdown_connections 2023-11-24T07:59:00.565 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:00.559+0000 7fb2a7b5e700 1 --2- 172.21.15.59:0/3595880801 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb2a00b4150 0x7fb2a006de50 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:00.565 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:00.559+0000 7fb2a7b5e700 1 -- 172.21.15.59:0/3595880801 >> 172.21.15.59:0/3595880801 conn(0x7fb2a00b2740 msgr2=0x7fb2a00b2b20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:59:00.565 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:00.560+0000 7fb2a7b5e700 1 -- 172.21.15.59:0/3595880801 shutdown_connections 2023-11-24T07:59:00.565 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:00.560+0000 7fb2a7b5e700 1 -- 172.21.15.59:0/3595880801 wait complete. 2023-11-24T07:59:00.565 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:00.561+0000 7fb2a7b5e700 1 Processor -- start 2023-11-24T07:59:00.565 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:00.561+0000 7fb2a7b5e700 1 -- start start 2023-11-24T07:59:00.565 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:00.561+0000 7fb2a7b5e700 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fb2a006b620 0x7fb2a01f3030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:00.565 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:00.561+0000 7fb2a7b5e700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb2a00b4150 0x7fb2a0141de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:00.565 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:00.561+0000 7fb2a7b5e700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb2a00b5ab0 0x7fb2a006eb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:00.565 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:00.562+0000 7fb2a7b5e700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2a013ef50 con 0x7fb2a00b4150 2023-11-24T07:59:00.566 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:00.562+0000 7fb2a7b5e700 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2a013e960 con 0x7fb2a00b5ab0 2023-11-24T07:59:00.566 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:00.562+0000 7fb2a7b5e700 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb2a01dcd00 con 0x7fb2a006b620 2023-11-24T07:59:00.566 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:00.562+0000 7fb2a50f9700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb2a00b4150 0x7fb2a0141de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:00.566 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:00.562+0000 7fb2a50f9700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb2a00b4150 0x7fb2a0141de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:54408/0 (socket says 172.21.15.59:54408) 2023-11-24T07:59:00.566 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:00.562+0000 7fb2a50f9700 1 -- 172.21.15.59:0/3609720783 learned_addr learned my addr 172.21.15.59:0/3609720783 (peer_addr_for_me v2:172.21.15.59:0/0) 2023-11-24T07:59:00.566 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:00.562+0000 7fb2a60fb700 1 --2- 172.21.15.59:0/3609720783 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb2a00b5ab0 0x7fb2a006eb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:00.566 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:00.562+0000 7fb2a58fa700 1 --2- 172.21.15.59:0/3609720783 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fb2a006b620 0x7fb2a01f3030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:00.566 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:00.562+0000 7fb2a50f9700 1 -- 172.21.15.59:0/3609720783 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fb2a006b620 msgr2=0x7fb2a01f3030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:00.566 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:00.562+0000 7fb2a50f9700 1 --2- 172.21.15.59:0/3609720783 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fb2a006b620 0x7fb2a01f3030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:00.566 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:00.562+0000 7fb2a50f9700 1 -- 172.21.15.59:0/3609720783 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb2a00b5ab0 msgr2=0x7fb2a006eb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:00.566 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:00.562+0000 7fb2a50f9700 1 --2- 172.21.15.59:0/3609720783 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb2a00b5ab0 0x7fb2a006eb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:00.567 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:00.562+0000 7fb2a50f9700 1 -- 172.21.15.59:0/3609720783 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb2900094a0 con 0x7fb2a00b4150 2023-11-24T07:59:00.567 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:00.562+0000 7fb2a60fb700 1 --2- 172.21.15.59:0/3609720783 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb2a00b5ab0 0x7fb2a006eb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-24T07:59:00.567 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:00.562+0000 7fb2a50f9700 1 --2- 172.21.15.59:0/3609720783 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb2a00b4150 0x7fb2a0141de0 secure :-1 s=READY pgs=795 cs=0 l=1 rev1=1 crypto rx=0x7fb2900090b0 tx=0x7fb290004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:59:00.567 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:00.562+0000 7fb2967fc700 1 -- 172.21.15.59:0/3609720783 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb290011830 con 0x7fb2a00b4150 2023-11-24T07:59:00.567 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:00.562+0000 7fb2967fc700 1 -- 172.21.15.59:0/3609720783 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb29001f070 con 0x7fb2a00b4150 2023-11-24T07:59:00.567 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:00.562+0000 7fb2967fc700 1 -- 172.21.15.59:0/3609720783 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb290011bd0 con 0x7fb2a00b4150 2023-11-24T07:59:00.567 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:00.562+0000 7fb2a7b5e700 1 -- 172.21.15.59:0/3609720783 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb2a0141040 con 0x7fb2a00b4150 2023-11-24T07:59:00.567 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:00.562+0000 7fb2a7b5e700 1 -- 172.21.15.59:0/3609720783 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb2a01404e0 con 0x7fb2a00b4150 2023-11-24T07:59:00.568 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:00.563+0000 7fb2a7b5e700 1 -- 172.21.15.59:0/3609720783 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb2a01411d0 con 0x7fb2a00b4150 2023-11-24T07:59:00.568 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:00.564+0000 7fb2967fc700 1 -- 172.21.15.59:0/3609720783 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 10) v1 ==== 89588+0+0 (secure 0 0 0) 0x7fb290011d30 con 0x7fb2a00b4150 2023-11-24T07:59:00.568 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:00.564+0000 7fb2967fc700 1 --2- 172.21.15.59:0/3609720783 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7fb28406b2c0 0x7fb28406d780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:00.568 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:00.564+0000 7fb2967fc700 1 -- 172.21.15.59:0/3609720783 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(61..61 src has 1..61) v4 ==== 5765+0+0 (secure 0 0 0) 0x7fb290021070 con 0x7fb2a00b4150 2023-11-24T07:59:00.568 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:00.564+0000 7fb2a58fa700 1 --2- 172.21.15.59:0/3609720783 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7fb28406b2c0 0x7fb28406d780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:00.569 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:00.565+0000 7fb2a58fa700 1 --2- 172.21.15.59:0/3609720783 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7fb28406b2c0 0x7fb28406d780 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7fb29c000f60 tx=0x7fb29c009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:59:00.570 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:00.569+0000 7fb2967fc700 1 -- 172.21.15.59:0/3609720783 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fb2900561f0 con 0x7fb2a00b4150 2023-11-24T07:59:00.817 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:00.815+0000 7fb2a7b5e700 1 -- 172.21.15.59:0/3609720783 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fb2a0068670 con 0x7fb2a00b4150 2023-11-24T07:59:00.817 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:00.816+0000 7fb2967fc700 1 -- 172.21.15.59:0/3609720783 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 36 v36) v1 ==== 94+0+4907 (secure 0 0 0) 0x7fb290059810 con 0x7fb2a00b4150 2023-11-24T07:59:00.817 INFO:teuthology.orchestra.run.smithi059.stdout: 2023-11-24T07:59:00.817 INFO:teuthology.orchestra.run.smithi059.stdout:{"epoch":36,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5300,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6834/172682993","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6833","nonce":172682993},{"type":"v1","addr":"172.21.15.59:6834","nonce":172682993}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":29},{"gid":5701,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.76:6836/923821877","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6834","nonce":923821877},{"type":"v1","addr":"172.21.15.76:6836","nonce":923821877}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":30},{"gid":5704,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.76:6837/4166482716","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6835","nonce":4166482716},{"type":"v1","addr":"172.21.15.76:6837","nonce":4166482716}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":29}],"filesystems":[{"mdsmap":{"epoch":35,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-24T07:58:40.795340+0000","modified":"2023-11-24T07:58:43.812382+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":6093},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6093":{"gid":6093,"name":"a","rank":0,"incarnation":31,"state":"up:active","state_seq":3,"addr":"172.21.15.59:6835/1599845006","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6832","nonce":1599845006},{"type":"v1","addr":"172.21.15.59:6835","nonce":1599845006}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[9],"metadata_pool":8,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":4}]} 2023-11-24T07:59:00.819 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:00.817+0000 7fb2a7b5e700 1 -- 172.21.15.59:0/3609720783 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7fb28406b2c0 msgr2=0x7fb28406d780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:00.820 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:00.817+0000 7fb2a7b5e700 1 --2- 172.21.15.59:0/3609720783 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7fb28406b2c0 0x7fb28406d780 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7fb29c000f60 tx=0x7fb29c009040 comp rx=0 tx=0).stop 2023-11-24T07:59:00.820 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:00.817+0000 7fb2a7b5e700 1 -- 172.21.15.59:0/3609720783 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb2a00b4150 msgr2=0x7fb2a0141de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:00.820 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:00.817+0000 7fb2a7b5e700 1 --2- 172.21.15.59:0/3609720783 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb2a00b4150 0x7fb2a0141de0 secure :-1 s=READY pgs=795 cs=0 l=1 rev1=1 crypto rx=0x7fb2900090b0 tx=0x7fb290004750 comp rx=0 tx=0).stop 2023-11-24T07:59:00.820 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:00.817+0000 7fb2a7b5e700 1 -- 172.21.15.59:0/3609720783 shutdown_connections 2023-11-24T07:59:00.820 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:00.818+0000 7fb2a7b5e700 1 --2- 172.21.15.59:0/3609720783 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7fb28406b2c0 0x7fb28406d780 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:00.820 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:00.818+0000 7fb2a7b5e700 1 --2- 172.21.15.59:0/3609720783 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb2a00b5ab0 0x7fb2a006eb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:00.821 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:00.818+0000 7fb2a7b5e700 1 --2- 172.21.15.59:0/3609720783 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fb2a006b620 0x7fb2a01f3030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:00.821 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:00.818+0000 7fb2a7b5e700 1 --2- 172.21.15.59:0/3609720783 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb2a00b4150 0x7fb2a0141de0 unknown :-1 s=CLOSED pgs=795 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:00.821 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:00.818+0000 7fb2a7b5e700 1 -- 172.21.15.59:0/3609720783 >> 172.21.15.59:0/3609720783 conn(0x7fb2a00b2740 msgr2=0x7fb2a006e0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:59:00.821 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:00.818+0000 7fb2a7b5e700 1 -- 172.21.15.59:0/3609720783 shutdown_connections 2023-11-24T07:59:00.821 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:00.818+0000 7fb2a7b5e700 1 -- 172.21.15.59:0/3609720783 wait complete. 2023-11-24T07:59:00.821 INFO:teuthology.orchestra.run.smithi059.stderr:dumped fsmap epoch 36 2023-11-24T07:59:00.832 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-24T07:59:00.956 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:00.955+0000 7fd6e4c95700 1 Processor -- start 2023-11-24T07:59:00.957 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:00.955+0000 7fd6e4c95700 1 -- start start 2023-11-24T07:59:00.961 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:00.956+0000 7fd6e4c95700 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fd6e0143ae0 0x7fd6e00b51b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:00.961 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:00.956+0000 7fd6e4c95700 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fd6e00722c0 0x7fd6e006e510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:00.961 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:00.956+0000 7fd6e4c95700 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd6e0143ec0 con 0x7fd6e01440f0 2023-11-24T07:59:00.961 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:00.956+0000 7fd6e4c95700 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd6e00b40a0 con 0x7fd6e00722c0 2023-11-24T07:59:00.961 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:00.956+0000 7fd6e4c95700 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd6e00b3ae0 con 0x7fd6e0143ae0 2023-11-24T07:59:00.962 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:00.956+0000 7fd6ddd9b700 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7fd6e01440f0 0x7fd6e0071bb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:57888/0 (socket says 172.21.15.59:57888) 2023-11-24T07:59:00.962 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:00.956+0000 7fd6ddd9b700 1 -- 172.21.15.59:0/182982131 learned_addr learned my addr 172.21.15.59:0/182982131 (peer_addr_for_me v1:172.21.15.59:0/0) 2023-11-24T07:59:00.962 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:00.956+0000 7fd6ded9d700 1 --2- 172.21.15.59:0/182982131 >> v2:172.21.15.76:3300/0 conn(0x7fd6e00722c0 0x7fd6e006e510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:00.962 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:00.956+0000 7fd6de59c700 1 --2- 172.21.15.59:0/182982131 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fd6e0143ae0 0x7fd6e00b51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:00.962 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:00.956+0000 7fd6dd59a700 1 -- 172.21.15.59:0/182982131 <== mon.0 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2418734703 0 0) 0x7fd6e0143ec0 con 0x7fd6e01440f0 2023-11-24T07:59:00.963 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:00.956+0000 7fd6dd59a700 1 -- 172.21.15.59:0/182982131 --> v1:172.21.15.59:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd6c80035b0 con 0x7fd6e01440f0 2023-11-24T07:59:00.963 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:00.956+0000 7fd6dd59a700 1 -- 172.21.15.59:0/182982131 <== mon.0 v1:172.21.15.59:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3935195005 0 0) 0x7fd6c80035b0 con 0x7fd6e01440f0 2023-11-24T07:59:00.963 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:00.956+0000 7fd6dd59a700 1 -- 172.21.15.59:0/182982131 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fd6e0143ae0 msgr2=0x7fd6e00b51b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T07:59:00.963 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:00.956+0000 7fd6dd59a700 1 --2- 172.21.15.59:0/182982131 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fd6e0143ae0 0x7fd6e00b51b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:00.963 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:00.956+0000 7fd6dd59a700 1 -- 172.21.15.59:0/182982131 >> v2:172.21.15.76:3300/0 conn(0x7fd6e00722c0 msgr2=0x7fd6e006e510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T07:59:00.963 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:00.956+0000 7fd6dd59a700 1 --2- 172.21.15.59:0/182982131 >> v2:172.21.15.76:3300/0 conn(0x7fd6e00722c0 0x7fd6e006e510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:00.964 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:00.956+0000 7fd6dd59a700 1 -- 172.21.15.59:0/182982131 --> v1:172.21.15.59:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd6e00b3dc0 con 0x7fd6e01440f0 2023-11-24T07:59:00.964 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:00.956+0000 7fd6dd59a700 1 -- 172.21.15.59:0/182982131 <== mon.0 v1:172.21.15.59:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1312333566 0 0) 0x7fd6d4001670 con 0x7fd6e01440f0 2023-11-24T07:59:00.964 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:00.956+0000 7fd6dd59a700 1 -- 172.21.15.59:0/182982131 >> v1:172.21.15.59:6789/0 conn(0x7fd6e01440f0 legacy=0x7fd6e0071bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:00.964 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:00.956+0000 7fd6dd59a700 1 --2- 172.21.15.59:0/182982131 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fd6e0143ae0 0x7fd6c8003a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:00.964 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:00.956+0000 7fd6dd59a700 1 --2- 172.21.15.59:0/182982131 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd6c8003f80 0x7fd6c8004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:00.964 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:00.956+0000 7fd6dd59a700 1 --2- 172.21.15.59:0/182982131 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd6c8004d90 0x7fd6c8005210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:00.964 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:00.956+0000 7fd6dd59a700 1 -- 172.21.15.59:0/182982131 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6e00b40a0 con 0x7fd6c8004d90 2023-11-24T07:59:00.965 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:00.956+0000 7fd6dd59a700 1 -- 172.21.15.59:0/182982131 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6e00b3ae0 con 0x7fd6c8003f80 2023-11-24T07:59:00.965 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:00.957+0000 7fd6dd59a700 1 -- 172.21.15.59:0/182982131 --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd6c800ba10 con 0x7fd6e0143ae0 2023-11-24T07:59:00.965 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:00.957+0000 7fd6ded9d700 1 --2- 172.21.15.59:0/182982131 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd6c8004d90 0x7fd6c8005210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:00.965 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:00.957+0000 7fd6de59c700 1 --2- 172.21.15.59:0/182982131 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fd6e0143ae0 0x7fd6c8003a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:00.965 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:00.957+0000 7fd6ddd9b700 1 --2- 172.21.15.59:0/182982131 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd6c8003f80 0x7fd6c8004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:00.965 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:00.957+0000 7fd6ded9d700 1 -- 172.21.15.59:0/182982131 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fd6e0143ae0 msgr2=0x7fd6c8003a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T07:59:00.966 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:00.957+0000 7fd6ded9d700 1 --2- 172.21.15.59:0/182982131 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fd6e0143ae0 0x7fd6c8003a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:00.966 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:00.957+0000 7fd6ded9d700 1 -- 172.21.15.59:0/182982131 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd6c8003f80 msgr2=0x7fd6c8004850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T07:59:00.966 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:00.957+0000 7fd6ded9d700 1 --2- 172.21.15.59:0/182982131 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd6c8003f80 0x7fd6c8004850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:00.966 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:00.957+0000 7fd6ded9d700 1 -- 172.21.15.59:0/182982131 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd6c80035b0 con 0x7fd6c8004d90 2023-11-24T07:59:00.966 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:00.957+0000 7fd6ded9d700 1 --2- 172.21.15.59:0/182982131 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd6c8004d90 0x7fd6c8005210 secure :-1 s=READY pgs=797 cs=0 l=1 rev1=1 crypto rx=0x7fd6d8009f90 tx=0x7fd6c8005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6e132fd523078bb9 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:59:00.966 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:00.957+0000 7fd6dd59a700 1 -- 172.21.15.59:0/182982131 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6d800ebd0 con 0x7fd6c8004d90 2023-11-24T07:59:00.966 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:00.957+0000 7fd6dd59a700 1 -- 172.21.15.59:0/182982131 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd6d800f040 con 0x7fd6c8004d90 2023-11-24T07:59:00.967 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:00.957+0000 7fd6dd59a700 1 -- 172.21.15.59:0/182982131 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6d8014480 con 0x7fd6c8004d90 2023-11-24T07:59:00.967 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:00.958+0000 7fd6e4c95700 1 -- 172.21.15.59:0/182982131 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd6c8004d90 msgr2=0x7fd6c8005210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:00.967 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:00.958+0000 7fd6e4c95700 1 --2- 172.21.15.59:0/182982131 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd6c8004d90 0x7fd6c8005210 secure :-1 s=READY pgs=797 cs=0 l=1 rev1=1 crypto rx=0x7fd6d8009f90 tx=0x7fd6c8005880 comp rx=0 tx=0).stop 2023-11-24T07:59:00.967 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:00.958+0000 7fd6ded9d700 1 -- 172.21.15.59:0/182982131 reap_dead start 2023-11-24T07:59:00.967 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:00.958+0000 7fd6e4c95700 1 -- 172.21.15.59:0/182982131 shutdown_connections 2023-11-24T07:59:00.967 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:00.958+0000 7fd6e4c95700 1 -- 172.21.15.59:0/182982131 >> 172.21.15.59:0/182982131 conn(0x7fd6e00b2740 msgr2=0x7fd6e00b2b20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:59:00.967 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:00.958+0000 7fd6e4c95700 1 -- 172.21.15.59:0/182982131 shutdown_connections 2023-11-24T07:59:00.967 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:00.958+0000 7fd6e4c95700 1 -- 172.21.15.59:0/182982131 wait complete. 2023-11-24T07:59:00.967 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:00.959+0000 7fd6e4c95700 1 Processor -- start 2023-11-24T07:59:00.967 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:00.959+0000 7fd6e4c95700 1 -- start start 2023-11-24T07:59:00.970 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:00.960+0000 7fd6e4c95700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd6c8004d90 0x7fd6e00ccc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:00.970 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:00.960+0000 7fd6e4c95700 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fd6e0138dc0 0x7fd6e0139220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:00.970 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:00.960+0000 7fd6e4c95700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd6e013ead0 0x7fd6e013ef60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:00.970 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:00.960+0000 7fd6e4c95700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6e013f4a0 con 0x7fd6c8004d90 2023-11-24T07:59:00.970 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:00.960+0000 7fd6e4c95700 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6e015bb60 con 0x7fd6e013ead0 2023-11-24T07:59:00.970 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:00.960+0000 7fd6e4c95700 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd6e015be40 con 0x7fd6e0138dc0 2023-11-24T07:59:00.971 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:00.960+0000 7fd6de59c700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd6c8004d90 0x7fd6e00ccc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:00.971 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:00.960+0000 7fd6de59c700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd6c8004d90 0x7fd6e00ccc30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:54426/0 (socket says 172.21.15.59:54426) 2023-11-24T07:59:00.971 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:00.960+0000 7fd6de59c700 1 -- 172.21.15.59:0/1634130956 learned_addr learned my addr 172.21.15.59:0/1634130956 (peer_addr_for_me v2:172.21.15.59:0/0) 2023-11-24T07:59:00.971 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:00.960+0000 7fd6ded9d700 1 --2- 172.21.15.59:0/1634130956 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd6e013ead0 0x7fd6e013ef60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:00.971 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:00.960+0000 7fd6ddd9b700 1 --2- 172.21.15.59:0/1634130956 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fd6e0138dc0 0x7fd6e0139220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:00.971 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:00.960+0000 7fd6de59c700 1 -- 172.21.15.59:0/1634130956 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fd6e0138dc0 msgr2=0x7fd6e0139220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:00.971 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:00.960+0000 7fd6de59c700 1 --2- 172.21.15.59:0/1634130956 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fd6e0138dc0 0x7fd6e0139220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:00.971 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:00.960+0000 7fd6de59c700 1 -- 172.21.15.59:0/1634130956 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd6e013ead0 msgr2=0x7fd6e013ef60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:00.971 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:00.960+0000 7fd6de59c700 1 --2- 172.21.15.59:0/1634130956 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd6e013ead0 0x7fd6e013ef60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:00.971 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:00.960+0000 7fd6de59c700 1 -- 172.21.15.59:0/1634130956 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd6e015c120 con 0x7fd6c8004d90 2023-11-24T07:59:00.972 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:00.960+0000 7fd6ddd9b700 1 --2- 172.21.15.59:0/1634130956 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fd6e0138dc0 0x7fd6e0139220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-24T07:59:00.972 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:00.960+0000 7fd6ded9d700 1 --2- 172.21.15.59:0/1634130956 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd6e013ead0 0x7fd6e013ef60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-24T07:59:00.972 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:00.961+0000 7fd6de59c700 1 --2- 172.21.15.59:0/1634130956 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd6c8004d90 0x7fd6e00ccc30 secure :-1 s=READY pgs=798 cs=0 l=1 rev1=1 crypto rx=0x7fd6d0011aa0 tx=0x7fd6d0011e60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:59:00.972 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:00.961+0000 7fd6ceffd700 1 -- 172.21.15.59:0/1634130956 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6d000f980 con 0x7fd6c8004d90 2023-11-24T07:59:00.972 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:00.961+0000 7fd6ceffd700 1 -- 172.21.15.59:0/1634130956 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd6d000fae0 con 0x7fd6c8004d90 2023-11-24T07:59:00.972 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:00.961+0000 7fd6e4c95700 1 -- 172.21.15.59:0/1634130956 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd6e015c400 con 0x7fd6c8004d90 2023-11-24T07:59:00.972 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:00.961+0000 7fd6ceffd700 1 -- 172.21.15.59:0/1634130956 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6d000fe90 con 0x7fd6c8004d90 2023-11-24T07:59:00.972 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:00.961+0000 7fd6e4c95700 1 -- 172.21.15.59:0/1634130956 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd6e0139950 con 0x7fd6c8004d90 2023-11-24T07:59:00.972 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:00.962+0000 7fd6e4c95700 1 -- 172.21.15.59:0/1634130956 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd6e00cd620 con 0x7fd6c8004d90 2023-11-24T07:59:00.972 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:00.962+0000 7fd6ceffd700 1 -- 172.21.15.59:0/1634130956 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 10) v1 ==== 89588+0+0 (secure 0 0 0) 0x7fd6d0014440 con 0x7fd6c8004d90 2023-11-24T07:59:00.972 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:00.963+0000 7fd6ceffd700 1 --2- 172.21.15.59:0/1634130956 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7fd6c006b2c0 0x7fd6c006d780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:00.973 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:00.963+0000 7fd6ceffd700 1 -- 172.21.15.59:0/1634130956 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(61..61 src has 1..61) v4 ==== 5765+0+0 (secure 0 0 0) 0x7fd6d008a540 con 0x7fd6c8004d90 2023-11-24T07:59:00.973 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:00.963+0000 7fd6ddd9b700 1 --2- 172.21.15.59:0/1634130956 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7fd6c006b2c0 0x7fd6c006d780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:00.973 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:00.963+0000 7fd6ddd9b700 1 --2- 172.21.15.59:0/1634130956 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7fd6c006b2c0 0x7fd6c006d780 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7fd6d4001050 tx=0x7fd6d4009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:59:00.973 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:00.967+0000 7fd6ceffd700 1 -- 172.21.15.59:0/1634130956 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fd6d0055090 con 0x7fd6c8004d90 2023-11-24T07:59:01.169 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:01.167+0000 7fd6e4c95700 1 -- 172.21.15.59:0/1634130956 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fd6e0068670 con 0x7fd6c8004d90 2023-11-24T07:59:01.170 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:01.168+0000 7fd6ceffd700 1 -- 172.21.15.59:0/1634130956 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v61) v1 ==== 74+0+16601 (secure 0 0 0) 0x7fd6d00586b0 con 0x7fd6c8004d90 2023-11-24T07:59:01.170 INFO:teuthology.orchestra.run.smithi059.stdout: 2023-11-24T07:59:01.170 INFO:teuthology.orchestra.run.smithi059.stdout:{"epoch":61,"fsid":"9f0e8741-b8f9-48d5-a2c3-09f45903303d","created":"2023-11-24T07:56:06.552729+0000","modified":"2023-11-24T07:58:57.458332+0000","last_up_change":"2023-11-24T07:56:17.372352+0000","last_in_change":"2023-11-24T07:56:09.073250+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-24T07:56:18.352823+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_metadata","create_time":"2023-11-24T07:58:39.262463+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"59","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_data","create_time":"2023-11-24T07:58:39.763399+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"61","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":3,"snap_epoch":61,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"6d8f9e9b-1782-4e41-928b-a39f2a8ebc06","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6800","nonce":3373878772},{"type":"v1","addr":"172.21.15.59:6801","nonce":3373878772}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6802","nonce":3373878772},{"type":"v1","addr":"172.21.15.59:6803","nonce":3373878772}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6806","nonce":3373878772},{"type":"v1","addr":"172.21.15.59:6807","nonce":3373878772}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6804","nonce":3373878772},{"type":"v1","addr":"172.21.15.59:6805","nonce":3373878772}]},"public_addr":"172.21.15.59:6801/3373878772","cluster_addr":"172.21.15.59:6803/3373878772","heartbeat_back_addr":"172.21.15.59:6807/3373878772","heartbeat_front_addr":"172.21.15.59:6805/3373878772","state":["exists","up"]},{"osd":1,"uuid":"e8d882aa-52e1-4c04-9e1e-c44ef9204a5f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6824","nonce":648258563},{"type":"v1","addr":"172.21.15.59:6825","nonce":648258563}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6826","nonce":648258563},{"type":"v1","addr":"172.21.15.59:6827","nonce":648258563}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6830","nonce":648258563},{"type":"v1","addr":"172.21.15.59:6831","nonce":648258563}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6828","nonce":648258563},{"type":"v1","addr":"172.21.15.59:6829","nonce":648258563}]},"public_addr":"172.21.15.59:6825/648258563","cluster_addr":"172.21.15.59:6827/648258563","heartbeat_back_addr":"172.21.15.59:6831/648258563","heartbeat_front_addr":"172.21.15.59:6829/648258563","state":["exists","up"]},{"osd":2,"uuid":"69eebfa0-695b-4b21-90b9-8a3537168a01","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6808","nonce":1840430654},{"type":"v1","addr":"172.21.15.59:6809","nonce":1840430654}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6810","nonce":1840430654},{"type":"v1","addr":"172.21.15.59:6811","nonce":1840430654}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6814","nonce":1840430654},{"type":"v1","addr":"172.21.15.59:6815","nonce":1840430654}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6812","nonce":1840430654},{"type":"v1","addr":"172.21.15.59:6813","nonce":1840430654}]},"public_addr":"172.21.15.59:6809/1840430654","cluster_addr":"172.21.15.59:6811/1840430654","heartbeat_back_addr":"172.21.15.59:6815/1840430654","heartbeat_front_addr":"172.21.15.59:6813/1840430654","state":["exists","up"]},{"osd":3,"uuid":"96fd77fc-f911-46ff-9445-6f0be2daa8f4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6816","nonce":2504025850},{"type":"v1","addr":"172.21.15.59:6817","nonce":2504025850}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6818","nonce":2504025850},{"type":"v1","addr":"172.21.15.59:6819","nonce":2504025850}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6822","nonce":2504025850},{"type":"v1","addr":"172.21.15.59:6823","nonce":2504025850}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6820","nonce":2504025850},{"type":"v1","addr":"172.21.15.59:6821","nonce":2504025850}]},"public_addr":"172.21.15.59:6817/2504025850","cluster_addr":"172.21.15.59:6819/2504025850","heartbeat_back_addr":"172.21.15.59:6823/2504025850","heartbeat_front_addr":"172.21.15.59:6821/2504025850","state":["exists","up"]},{"osd":4,"uuid":"7ac29fca-b5b2-4c95-9d69-2f164cbbc708","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6816","nonce":2382463431},{"type":"v1","addr":"172.21.15.76:6818","nonce":2382463431}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6820","nonce":2382463431},{"type":"v1","addr":"172.21.15.76:6822","nonce":2382463431}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6828","nonce":2382463431},{"type":"v1","addr":"172.21.15.76:6830","nonce":2382463431}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6824","nonce":2382463431},{"type":"v1","addr":"172.21.15.76:6826","nonce":2382463431}]},"public_addr":"172.21.15.76:6818/2382463431","cluster_addr":"172.21.15.76:6822/2382463431","heartbeat_back_addr":"172.21.15.76:6830/2382463431","heartbeat_front_addr":"172.21.15.76:6826/2382463431","state":["exists","up"]},{"osd":5,"uuid":"485bf711-3851-4238-a4f4-51d52e0a9c58","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6800","nonce":329334308},{"type":"v1","addr":"172.21.15.76:6802","nonce":329334308}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6805","nonce":329334308},{"type":"v1","addr":"172.21.15.76:6807","nonce":329334308}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6814","nonce":329334308},{"type":"v1","addr":"172.21.15.76:6815","nonce":329334308}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6810","nonce":329334308},{"type":"v1","addr":"172.21.15.76:6812","nonce":329334308}]},"public_addr":"172.21.15.76:6802/329334308","cluster_addr":"172.21.15.76:6807/329334308","heartbeat_back_addr":"172.21.15.76:6815/329334308","heartbeat_front_addr":"172.21.15.76:6812/329334308","state":["exists","up"]},{"osd":6,"uuid":"72b3f2fc-1e4a-4e32-968d-003647a0286f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6801","nonce":3946579410},{"type":"v1","addr":"172.21.15.76:6803","nonce":3946579410}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6804","nonce":3946579410},{"type":"v1","addr":"172.21.15.76:6806","nonce":3946579410}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6811","nonce":3946579410},{"type":"v1","addr":"172.21.15.76:6813","nonce":3946579410}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6808","nonce":3946579410},{"type":"v1","addr":"172.21.15.76:6809","nonce":3946579410}]},"public_addr":"172.21.15.76:6803/3946579410","cluster_addr":"172.21.15.76:6806/3946579410","heartbeat_back_addr":"172.21.15.76:6813/3946579410","heartbeat_front_addr":"172.21.15.76:6809/3946579410","state":["exists","up"]},{"osd":7,"uuid":"1ddceb90-6c5d-4d1a-ad56-99cb877f8c70","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6817","nonce":841761538},{"type":"v1","addr":"172.21.15.76:6819","nonce":841761538}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6821","nonce":841761538},{"type":"v1","addr":"172.21.15.76:6823","nonce":841761538}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6829","nonce":841761538},{"type":"v1","addr":"172.21.15.76:6831","nonce":841761538}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6825","nonce":841761538},{"type":"v1","addr":"172.21.15.76:6827","nonce":841761538}]},"public_addr":"172.21.15.76:6819/841761538","cluster_addr":"172.21.15.76:6823/841761538","heartbeat_back_addr":"172.21.15.76:6831/841761538","heartbeat_front_addr":"172.21.15.76:6827/841761538","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T07:56:14.294838+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T07:56:14.270779+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T07:56:14.231809+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T07:56:14.260983+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T07:56:14.302997+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T07:56:14.258600+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T07:56:14.251413+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T07:56:14.304727+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":9,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[{"pool":9,"snaps":[{"begin":2,"length":2}]}],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-24T07:59:01.172 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:01.170+0000 7fd6e4c95700 1 -- 172.21.15.59:0/1634130956 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7fd6c006b2c0 msgr2=0x7fd6c006d780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:01.172 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:01.170+0000 7fd6e4c95700 1 --2- 172.21.15.59:0/1634130956 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7fd6c006b2c0 0x7fd6c006d780 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7fd6d4001050 tx=0x7fd6d4009040 comp rx=0 tx=0).stop 2023-11-24T07:59:01.172 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:01.170+0000 7fd6e4c95700 1 -- 172.21.15.59:0/1634130956 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd6c8004d90 msgr2=0x7fd6e00ccc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:01.172 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:01.170+0000 7fd6e4c95700 1 --2- 172.21.15.59:0/1634130956 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd6c8004d90 0x7fd6e00ccc30 secure :-1 s=READY pgs=798 cs=0 l=1 rev1=1 crypto rx=0x7fd6d0011aa0 tx=0x7fd6d0011e60 comp rx=0 tx=0).stop 2023-11-24T07:59:01.172 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:01.170+0000 7fd6e4c95700 1 -- 172.21.15.59:0/1634130956 shutdown_connections 2023-11-24T07:59:01.173 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:01.170+0000 7fd6e4c95700 1 --2- 172.21.15.59:0/1634130956 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7fd6c006b2c0 0x7fd6c006d780 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:01.173 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:01.170+0000 7fd6e4c95700 1 --2- 172.21.15.59:0/1634130956 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd6e013ead0 0x7fd6e013ef60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:01.173 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:01.170+0000 7fd6e4c95700 1 --2- 172.21.15.59:0/1634130956 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd6c8004d90 0x7fd6e00ccc30 unknown :-1 s=CLOSED pgs=798 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:01.173 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:01.170+0000 7fd6e4c95700 1 --2- 172.21.15.59:0/1634130956 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fd6e0138dc0 0x7fd6e0139220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:01.173 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:01.170+0000 7fd6e4c95700 1 -- 172.21.15.59:0/1634130956 >> 172.21.15.59:0/1634130956 conn(0x7fd6e00b2740 msgr2=0x7fd6e0159630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:59:01.173 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:01.170+0000 7fd6e4c95700 1 -- 172.21.15.59:0/1634130956 shutdown_connections 2023-11-24T07:59:01.173 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:01.170+0000 7fd6e4c95700 1 -- 172.21.15.59:0/1634130956 wait complete. 2023-11-24T07:59:01.185 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2023-11-24T07:59:01.310 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:01.309+0000 7fe7f2f47700 1 Processor -- start 2023-11-24T07:59:01.311 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:01.309+0000 7fe7f2f47700 1 -- start start 2023-11-24T07:59:01.314 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:01.310+0000 7fe7f2f47700 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fe7ec0722c0 0x7fe7ec0b51b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:01.314 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:01.310+0000 7fe7f2f47700 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe7ec143ec0 con 0x7fe7ec1440f0 2023-11-24T07:59:01.315 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:01.310+0000 7fe7f2f47700 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe7ec0b3ae0 con 0x7fe7ec143ae0 2023-11-24T07:59:01.315 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:01.310+0000 7fe7f2f47700 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe7ec0b40a0 con 0x7fe7ec0722c0 2023-11-24T07:59:01.315 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:01.310+0000 7fe7f14e4700 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7fe7ec1440f0 0x7fe7ec06e510 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:57912/0 (socket says 172.21.15.59:57912) 2023-11-24T07:59:01.315 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:01.310+0000 7fe7f14e4700 1 -- 172.21.15.59:0/2436586991 learned_addr learned my addr 172.21.15.59:0/2436586991 (peer_addr_for_me v1:172.21.15.59:0/0) 2023-11-24T07:59:01.315 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:01.310+0000 7fe7ebfff700 1 --2- 172.21.15.59:0/2436586991 >> v2:172.21.15.76:3300/0 conn(0x7fe7ec0722c0 0x7fe7ec0b51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:01.316 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:01.310+0000 7fe7eb7fe700 1 -- 172.21.15.59:0/2436586991 <== mon.0 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3339714177 0 0) 0x7fe7ec143ec0 con 0x7fe7ec1440f0 2023-11-24T07:59:01.316 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:01.310+0000 7fe7eb7fe700 1 -- 172.21.15.59:0/2436586991 --> v1:172.21.15.59:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe7dc0035b0 con 0x7fe7ec1440f0 2023-11-24T07:59:01.316 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:01.310+0000 7fe7eb7fe700 1 -- 172.21.15.59:0/2436586991 <== mon.0 v1:172.21.15.59:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1363452097 0 0) 0x7fe7dc0035b0 con 0x7fe7ec1440f0 2023-11-24T07:59:01.316 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:01.310+0000 7fe7eb7fe700 1 -- 172.21.15.59:0/2436586991 >> v2:172.21.15.76:3300/0 conn(0x7fe7ec0722c0 msgr2=0x7fe7ec0b51b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T07:59:01.316 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:01.310+0000 7fe7eb7fe700 1 --2- 172.21.15.59:0/2436586991 >> v2:172.21.15.76:3300/0 conn(0x7fe7ec0722c0 0x7fe7ec0b51b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:01.317 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:01.310+0000 7fe7eb7fe700 1 -- 172.21.15.59:0/2436586991 >> v1:172.21.15.76:6789/0 conn(0x7fe7ec143ae0 legacy=0x7fe7ec071bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:01.317 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:01.310+0000 7fe7eb7fe700 1 -- 172.21.15.59:0/2436586991 --> v1:172.21.15.59:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe7ec0b3dc0 con 0x7fe7ec1440f0 2023-11-24T07:59:01.317 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:01.310+0000 7fe7eb7fe700 1 -- 172.21.15.59:0/2436586991 <== mon.0 v1:172.21.15.59:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1312333566 0 0) 0x7fe7e40019f0 con 0x7fe7ec1440f0 2023-11-24T07:59:01.317 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:01.310+0000 7fe7eb7fe700 1 -- 172.21.15.59:0/2436586991 >> v1:172.21.15.59:6789/0 conn(0x7fe7ec1440f0 legacy=0x7fe7ec06e510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:01.317 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:01.310+0000 7fe7eb7fe700 1 --2- 172.21.15.59:0/2436586991 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe7dc003a00 0x7fe7dc003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:01.317 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:01.311+0000 7fe7eb7fe700 1 --2- 172.21.15.59:0/2436586991 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fe7dc004850 0x7fe7dc004cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:01.318 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:01.311+0000 7fe7eb7fe700 1 --2- 172.21.15.59:0/2436586991 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe7dc005210 0x7fe7dc00ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:01.318 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:01.311+0000 7fe7eb7fe700 1 -- 172.21.15.59:0/2436586991 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7ec0b40a0 con 0x7fe7dc003a00 2023-11-24T07:59:01.318 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:01.311+0000 7fe7eb7fe700 1 -- 172.21.15.59:0/2436586991 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7dc00c0b0 con 0x7fe7dc005210 2023-11-24T07:59:01.318 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:01.311+0000 7fe7eb7fe700 1 -- 172.21.15.59:0/2436586991 --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe7dc00c270 con 0x7fe7dc004850 2023-11-24T07:59:01.318 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:01.311+0000 7fe7f0ce3700 1 --2- 172.21.15.59:0/2436586991 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe7dc003a00 0x7fe7dc003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:01.319 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:01.311+0000 7fe7f14e4700 1 --2- 172.21.15.59:0/2436586991 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe7dc005210 0x7fe7dc00ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:01.319 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:01.311+0000 7fe7ebfff700 1 --2- 172.21.15.59:0/2436586991 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fe7dc004850 0x7fe7dc004cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:01.319 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:01.311+0000 7fe7f0ce3700 1 -- 172.21.15.59:0/2436586991 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fe7dc004850 msgr2=0x7fe7dc004cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T07:59:01.319 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:01.311+0000 7fe7f0ce3700 1 --2- 172.21.15.59:0/2436586991 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fe7dc004850 0x7fe7dc004cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:01.319 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:01.311+0000 7fe7f0ce3700 1 -- 172.21.15.59:0/2436586991 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe7dc005210 msgr2=0x7fe7dc00ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T07:59:01.320 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:01.311+0000 7fe7f0ce3700 1 --2- 172.21.15.59:0/2436586991 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe7dc005210 0x7fe7dc00ba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:01.320 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:01.311+0000 7fe7f0ce3700 1 -- 172.21.15.59:0/2436586991 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe7dc0035b0 con 0x7fe7dc003a00 2023-11-24T07:59:01.320 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:01.311+0000 7fe7f14e4700 1 --2- 172.21.15.59:0/2436586991 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe7dc005210 0x7fe7dc00ba10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-24T07:59:01.320 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:01.311+0000 7fe7f14e4700 1 -- 172.21.15.59:0/2436586991 reap_dead start 2023-11-24T07:59:01.320 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:01.311+0000 7fe7f0ce3700 1 --2- 172.21.15.59:0/2436586991 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe7dc003a00 0x7fe7dc003e20 secure :-1 s=READY pgs=800 cs=0 l=1 rev1=1 crypto rx=0x7fe7e00017c0 tx=0x7fe7dc005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a65dab1285a0a2b0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:59:01.321 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:01.311+0000 7fe7eb7fe700 1 -- 172.21.15.59:0/2436586991 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7e00052d0 con 0x7fe7dc003a00 2023-11-24T07:59:01.321 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:01.311+0000 7fe7eb7fe700 1 -- 172.21.15.59:0/2436586991 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe7e0005430 con 0x7fe7dc003a00 2023-11-24T07:59:01.321 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:01.311+0000 7fe7eb7fe700 1 -- 172.21.15.59:0/2436586991 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7e00038c0 con 0x7fe7dc003a00 2023-11-24T07:59:01.321 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:01.312+0000 7fe7f2f47700 1 -- 172.21.15.59:0/2436586991 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe7dc003a00 msgr2=0x7fe7dc003e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:01.321 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:01.312+0000 7fe7f2f47700 1 --2- 172.21.15.59:0/2436586991 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe7dc003a00 0x7fe7dc003e20 secure :-1 s=READY pgs=800 cs=0 l=1 rev1=1 crypto rx=0x7fe7e00017c0 tx=0x7fe7dc005880 comp rx=0 tx=0).stop 2023-11-24T07:59:01.321 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:01.312+0000 7fe7f2f47700 1 -- 172.21.15.59:0/2436586991 shutdown_connections 2023-11-24T07:59:01.322 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:01.312+0000 7fe7f2f47700 1 --2- 172.21.15.59:0/2436586991 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe7dc003a00 0x7fe7dc003e20 unknown :-1 s=CLOSED pgs=800 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:01.322 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:01.312+0000 7fe7f2f47700 1 -- 172.21.15.59:0/2436586991 >> 172.21.15.59:0/2436586991 conn(0x7fe7ec0b2740 msgr2=0x7fe7ec0b2b20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:59:01.322 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:01.312+0000 7fe7f2f47700 1 -- 172.21.15.59:0/2436586991 shutdown_connections 2023-11-24T07:59:01.322 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:01.312+0000 7fe7f2f47700 1 -- 172.21.15.59:0/2436586991 wait complete. 2023-11-24T07:59:01.326 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:01.313+0000 7fe7f2f47700 1 Processor -- start 2023-11-24T07:59:01.327 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:01.313+0000 7fe7f2f47700 1 -- start start 2023-11-24T07:59:01.327 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:01.314+0000 7fe7f2f47700 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fe7dc003a00 0x7fe7ec15a610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:01.327 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:01.314+0000 7fe7f2f47700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe7ec15d370 0x7fe7ec15d7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:01.327 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:01.314+0000 7fe7f2f47700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe7ec13d1d0 0x7fe7ec13d650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:01.328 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:01.314+0000 7fe7f2f47700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7ec15ab50 con 0x7fe7ec13d1d0 2023-11-24T07:59:01.328 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:01.314+0000 7fe7f2f47700 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7ec13db90 con 0x7fe7ec15d370 2023-11-24T07:59:01.328 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:01.314+0000 7fe7f2f47700 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe7ec1e46b0 con 0x7fe7dc003a00 2023-11-24T07:59:01.329 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:01.314+0000 7fe7f14e4700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe7ec13d1d0 0x7fe7ec13d650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:01.329 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:01.314+0000 7fe7f14e4700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe7ec13d1d0 0x7fe7ec13d650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:54450/0 (socket says 172.21.15.59:54450) 2023-11-24T07:59:01.329 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:01.314+0000 7fe7f14e4700 1 -- 172.21.15.59:0/3940049033 learned_addr learned my addr 172.21.15.59:0/3940049033 (peer_addr_for_me v2:172.21.15.59:0/0) 2023-11-24T07:59:01.329 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:01.314+0000 7fe7ebfff700 1 --2- 172.21.15.59:0/3940049033 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe7ec15d370 0x7fe7ec15d7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:01.330 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:01.314+0000 7fe7f0ce3700 1 --2- 172.21.15.59:0/3940049033 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fe7dc003a00 0x7fe7ec15a610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:01.330 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:01.314+0000 7fe7f14e4700 1 -- 172.21.15.59:0/3940049033 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fe7dc003a00 msgr2=0x7fe7ec15a610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:01.330 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:01.314+0000 7fe7f14e4700 1 --2- 172.21.15.59:0/3940049033 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fe7dc003a00 0x7fe7ec15a610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:01.330 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:01.314+0000 7fe7f14e4700 1 -- 172.21.15.59:0/3940049033 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe7ec15d370 msgr2=0x7fe7ec15d7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:01.331 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:01.314+0000 7fe7f14e4700 1 --2- 172.21.15.59:0/3940049033 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe7ec15d370 0x7fe7ec15d7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:01.331 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:01.314+0000 7fe7f14e4700 1 -- 172.21.15.59:0/3940049033 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe7ec1e4990 con 0x7fe7ec13d1d0 2023-11-24T07:59:01.331 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:01.314+0000 7fe7f0ce3700 1 --2- 172.21.15.59:0/3940049033 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fe7dc003a00 0x7fe7ec15a610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-24T07:59:01.332 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:01.315+0000 7fe7f14e4700 1 --2- 172.21.15.59:0/3940049033 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe7ec13d1d0 0x7fe7ec13d650 secure :-1 s=READY pgs=801 cs=0 l=1 rev1=1 crypto rx=0x7fe7e4000f80 tx=0x7fe7e400e440 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:59:01.332 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:01.315+0000 7fe7e97fa700 1 -- 172.21.15.59:0/3940049033 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7e4017070 con 0x7fe7ec13d1d0 2023-11-24T07:59:01.332 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:01.315+0000 7fe7e97fa700 1 -- 172.21.15.59:0/3940049033 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe7e400ebb0 con 0x7fe7ec13d1d0 2023-11-24T07:59:01.333 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:01.315+0000 7fe7f2f47700 1 -- 172.21.15.59:0/3940049033 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe7ec1e4c70 con 0x7fe7ec13d1d0 2023-11-24T07:59:01.333 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:01.315+0000 7fe7e97fa700 1 -- 172.21.15.59:0/3940049033 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7e400ed20 con 0x7fe7ec13d1d0 2023-11-24T07:59:01.333 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:01.315+0000 7fe7f2f47700 1 -- 172.21.15.59:0/3940049033 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe7ec1e5230 con 0x7fe7ec13d1d0 2023-11-24T07:59:01.333 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:01.316+0000 7fe7f2f47700 1 -- 172.21.15.59:0/3940049033 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe7ec15dd30 con 0x7fe7ec13d1d0 2023-11-24T07:59:01.334 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:01.316+0000 7fe7e97fa700 1 -- 172.21.15.59:0/3940049033 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 10) v1 ==== 89588+0+0 (secure 0 0 0) 0x7fe7e4013420 con 0x7fe7ec13d1d0 2023-11-24T07:59:01.334 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:01.317+0000 7fe7e97fa700 1 --2- 172.21.15.59:0/3940049033 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7fe7d006b2c0 0x7fe7d006d780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:01.335 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:01.317+0000 7fe7e97fa700 1 -- 172.21.15.59:0/3940049033 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(61..61 src has 1..61) v4 ==== 5765+0+0 (secure 0 0 0) 0x7fe7e4089cb0 con 0x7fe7ec13d1d0 2023-11-24T07:59:01.335 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:01.317+0000 7fe7f0ce3700 1 --2- 172.21.15.59:0/3940049033 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7fe7d006b2c0 0x7fe7d006d780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:01.335 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:01.317+0000 7fe7f0ce3700 1 --2- 172.21.15.59:0/3940049033 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7fe7d006b2c0 0x7fe7d006d780 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7fe7e0000c00 tx=0x7fe7e0009030 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:59:01.336 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:01.321+0000 7fe7e97fa700 1 -- 172.21.15.59:0/3940049033 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fe7e4058140 con 0x7fe7ec13d1d0 2023-11-24T07:59:01.583 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:01.581+0000 7fe7f2f47700 1 -- 172.21.15.59:0/3940049033 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7fe7ec1f1d50 con 0x7fe7ec13d1d0 2023-11-24T07:59:01.583 INFO:tasks.ceph.mon.a.smithi059.stderr:2023-11-24T07:59:01.582+0000 7f5560821700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-11-24T07:59:01.586 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:01.585+0000 7fe7e97fa700 1 -- 172.21.15.59:0/3940049033 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v37) v1 ==== 161+0+0 (secure 0 0 0) 0x7fe7e4057cd0 con 0x7fe7ec13d1d0 2023-11-24T07:59:01.589 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:01.587+0000 7fe7f2f47700 1 -- 172.21.15.59:0/3940049033 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7fe7d006b2c0 msgr2=0x7fe7d006d780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:01.589 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:01.587+0000 7fe7f2f47700 1 --2- 172.21.15.59:0/3940049033 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7fe7d006b2c0 0x7fe7d006d780 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7fe7e0000c00 tx=0x7fe7e0009030 comp rx=0 tx=0).stop 2023-11-24T07:59:01.589 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:01.587+0000 7fe7f2f47700 1 -- 172.21.15.59:0/3940049033 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe7ec13d1d0 msgr2=0x7fe7ec13d650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:01.589 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:01.587+0000 7fe7f2f47700 1 --2- 172.21.15.59:0/3940049033 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe7ec13d1d0 0x7fe7ec13d650 secure :-1 s=READY pgs=801 cs=0 l=1 rev1=1 crypto rx=0x7fe7e4000f80 tx=0x7fe7e400e440 comp rx=0 tx=0).stop 2023-11-24T07:59:01.589 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:01.587+0000 7fe7f2f47700 1 -- 172.21.15.59:0/3940049033 shutdown_connections 2023-11-24T07:59:01.590 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:01.587+0000 7fe7f2f47700 1 --2- 172.21.15.59:0/3940049033 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7fe7d006b2c0 0x7fe7d006d780 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:01.590 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:01.587+0000 7fe7f2f47700 1 --2- 172.21.15.59:0/3940049033 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe7ec13d1d0 0x7fe7ec13d650 unknown :-1 s=CLOSED pgs=801 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:01.590 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:01.587+0000 7fe7f2f47700 1 --2- 172.21.15.59:0/3940049033 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fe7dc003a00 0x7fe7ec15a610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:01.590 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:01.587+0000 7fe7f2f47700 1 --2- 172.21.15.59:0/3940049033 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe7ec15d370 0x7fe7ec15d7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:01.590 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:01.587+0000 7fe7f2f47700 1 -- 172.21.15.59:0/3940049033 >> 172.21.15.59:0/3940049033 conn(0x7fe7ec0b2740 msgr2=0x7fe7ec0b4470 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:59:01.590 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:01.587+0000 7fe7f2f47700 1 -- 172.21.15.59:0/3940049033 shutdown_connections 2023-11-24T07:59:01.590 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:01.587+0000 7fe7f2f47700 1 -- 172.21.15.59:0/3940049033 wait complete. 2023-11-24T07:59:01.591 INFO:teuthology.orchestra.run.smithi059.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2023-11-24T07:59:01.602 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2023-11-24T07:59:01.628 INFO:tasks.ceph.mds.a.smithi059.stdout:starting mds.a at 2023-11-24T07:59:01.730 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:01.728+0000 7fb629cff700 1 Processor -- start 2023-11-24T07:59:01.730 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:01.728+0000 7fb629cff700 1 -- start start 2023-11-24T07:59:01.732 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:01.729+0000 7fb629cff700 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fb624072a00 0x7fb62406db60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:01.733 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:01.729+0000 7fb629cff700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fb62406e0a0 0x7fb6240d1010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:01.733 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:01.729+0000 7fb629cff700 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb6240d0b80 con 0x7fb6240b4090 2023-11-24T07:59:01.733 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:01.729+0000 7fb629cff700 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb62415ed60 con 0x7fb62406e0a0 2023-11-24T07:59:01.733 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:01.729+0000 7fb629cff700 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb62415f040 con 0x7fb624072a00 2023-11-24T07:59:01.733 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:01.729+0000 7fb623fff700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fb62406e0a0 0x7fb6240d1010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:01.733 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:01.729+0000 7fb623fff700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fb62406e0a0 0x7fb6240d1010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:54474/0 (socket says 172.21.15.59:54474) 2023-11-24T07:59:01.733 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:01.729+0000 7fb623fff700 1 -- 172.21.15.59:0/3694454639 learned_addr learned my addr 172.21.15.59:0/3694454639 (peer_addr_for_me v2:172.21.15.59:0/0) 2023-11-24T07:59:01.733 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:01.730+0000 7fb622ffd700 1 --2- 172.21.15.59:0/3694454639 >> v2:172.21.15.76:3300/0 conn(0x7fb624072a00 0x7fb62406db60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:01.733 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:01.730+0000 7fb623fff700 1 -- 172.21.15.59:0/3694454639 >> v2:172.21.15.76:3300/0 conn(0x7fb624072a00 msgr2=0x7fb62406db60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T07:59:01.733 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:01.730+0000 7fb623fff700 1 --2- 172.21.15.59:0/3694454639 >> v2:172.21.15.76:3300/0 conn(0x7fb624072a00 0x7fb62406db60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:01.733 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:01.730+0000 7fb623fff700 1 -- 172.21.15.59:0/3694454639 >> v1:172.21.15.76:6789/0 conn(0x7fb6240b4090 legacy=0x7fb62415e510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T07:59:01.734 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:01.730+0000 7fb623fff700 1 -- 172.21.15.59:0/3694454639 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb62415f320 con 0x7fb62406e0a0 2023-11-24T07:59:01.734 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:01.730+0000 7fb623fff700 1 --2- 172.21.15.59:0/3694454639 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb62406e0a0 0x7fb6240d1010 secure :-1 s=READY pgs=805 cs=0 l=1 rev1=1 crypto rx=0x7fb62415f580 tx=0x7fb61c00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=773aec8e9e170f3c server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:59:01.734 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:01.730+0000 7fb6227fc700 1 -- 172.21.15.59:0/3694454639 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb61c00e070 con 0x7fb62406e0a0 2023-11-24T07:59:01.734 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:01.730+0000 7fb6227fc700 1 -- 172.21.15.59:0/3694454639 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb61c003a20 con 0x7fb62406e0a0 2023-11-24T07:59:01.734 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:01.730+0000 7fb6227fc700 1 -- 172.21.15.59:0/3694454639 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb61c003dd0 con 0x7fb62406e0a0 2023-11-24T07:59:01.734 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:01.731+0000 7fb629cff700 1 -- 172.21.15.59:0/3694454639 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb62406e0a0 msgr2=0x7fb6240d1010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:01.734 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:01.731+0000 7fb629cff700 1 --2- 172.21.15.59:0/3694454639 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb62406e0a0 0x7fb6240d1010 secure :-1 s=READY pgs=805 cs=0 l=1 rev1=1 crypto rx=0x7fb62415f580 tx=0x7fb61c00b330 comp rx=0 tx=0).stop 2023-11-24T07:59:01.734 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:01.731+0000 7fb629cff700 1 -- 172.21.15.59:0/3694454639 shutdown_connections 2023-11-24T07:59:01.734 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:01.731+0000 7fb629cff700 1 --2- 172.21.15.59:0/3694454639 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb62406e0a0 0x7fb6240d1010 unknown :-1 s=CLOSED pgs=805 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:01.734 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:01.731+0000 7fb629cff700 1 --2- 172.21.15.59:0/3694454639 >> v2:172.21.15.76:3300/0 conn(0x7fb624072a00 0x7fb62406db60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:01.735 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:01.731+0000 7fb629cff700 1 -- 172.21.15.59:0/3694454639 >> 172.21.15.59:0/3694454639 conn(0x7fb6240b2740 msgr2=0x7fb6240b2b50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:59:01.735 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:01.731+0000 7fb629cff700 1 -- 172.21.15.59:0/3694454639 shutdown_connections 2023-11-24T07:59:01.735 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:01.731+0000 7fb629cff700 1 -- 172.21.15.59:0/3694454639 wait complete. 2023-11-24T07:59:01.737 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:01.732+0000 7fb629cff700 1 Processor -- start 2023-11-24T07:59:01.737 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:01.732+0000 7fb629cff700 1 -- start start 2023-11-24T07:59:01.737 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:01.733+0000 7fb629cff700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb62406e0a0 0x7fb62415b500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:01.738 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:01.733+0000 7fb629cff700 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fb624072a00 0x7fb624144480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:01.738 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:01.733+0000 7fb629cff700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb6240b4090 0x7fb624144d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:01.738 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:01.733+0000 7fb629cff700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6240ae7d0 con 0x7fb62406e0a0 2023-11-24T07:59:01.738 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:01.733+0000 7fb629cff700 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb62415ba40 con 0x7fb6240b4090 2023-11-24T07:59:01.738 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:01.733+0000 7fb629cff700 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb6240ae4f0 con 0x7fb624072a00 2023-11-24T07:59:01.738 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:01.733+0000 7fb6237fe700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb62406e0a0 0x7fb62415b500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:01.738 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:01.733+0000 7fb6237fe700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb62406e0a0 0x7fb62415b500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:54476/0 (socket says 172.21.15.59:54476) 2023-11-24T07:59:01.738 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:01.733+0000 7fb6237fe700 1 -- 172.21.15.59:0/3274636893 learned_addr learned my addr 172.21.15.59:0/3274636893 (peer_addr_for_me v2:172.21.15.59:0/0) 2023-11-24T07:59:01.738 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:01.733+0000 7fb622ffd700 1 --2- 172.21.15.59:0/3274636893 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fb624072a00 0x7fb624144480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:01.738 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:01.733+0000 7fb623fff700 1 --2- 172.21.15.59:0/3274636893 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb6240b4090 0x7fb624144d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:01.739 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:01.733+0000 7fb6237fe700 1 -- 172.21.15.59:0/3274636893 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fb624072a00 msgr2=0x7fb624144480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:01.739 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:01.733+0000 7fb6237fe700 1 --2- 172.21.15.59:0/3274636893 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fb624072a00 0x7fb624144480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:01.739 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:01.733+0000 7fb6237fe700 1 -- 172.21.15.59:0/3274636893 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb6240b4090 msgr2=0x7fb624144d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:01.739 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:01.733+0000 7fb6237fe700 1 --2- 172.21.15.59:0/3274636893 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb6240b4090 0x7fb624144d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:01.739 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:01.733+0000 7fb6237fe700 1 -- 172.21.15.59:0/3274636893 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb61c009b30 con 0x7fb62406e0a0 2023-11-24T07:59:01.739 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:01.733+0000 7fb6237fe700 1 --2- 172.21.15.59:0/3274636893 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb62406e0a0 0x7fb62415b500 secure :-1 s=READY pgs=806 cs=0 l=1 rev1=1 crypto rx=0x7fb61800b470 tx=0x7fb61800b780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:59:01.739 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:01.734+0000 7fb60bfff700 1 -- 172.21.15.59:0/3274636893 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb618011830 con 0x7fb62406e0a0 2023-11-24T07:59:01.739 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:01.734+0000 7fb60bfff700 1 -- 172.21.15.59:0/3274636893 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb6180119b0 con 0x7fb62406e0a0 2023-11-24T07:59:01.739 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:01.734+0000 7fb629cff700 1 -- 172.21.15.59:0/3274636893 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb6240aeb20 con 0x7fb62406e0a0 2023-11-24T07:59:01.739 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:01.734+0000 7fb60bfff700 1 -- 172.21.15.59:0/3274636893 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb618011d60 con 0x7fb62406e0a0 2023-11-24T07:59:01.739 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:01.734+0000 7fb629cff700 1 -- 172.21.15.59:0/3274636893 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb6240af0e0 con 0x7fb62406e0a0 2023-11-24T07:59:01.739 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:01.735+0000 7fb629cff700 1 -- 172.21.15.59:0/3274636893 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb6240af450 con 0x7fb62406e0a0 2023-11-24T07:59:01.740 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:01.735+0000 7fb60bfff700 1 -- 172.21.15.59:0/3274636893 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 10) v1 ==== 89588+0+0 (secure 0 0 0) 0x7fb61801a400 con 0x7fb62406e0a0 2023-11-24T07:59:01.740 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:01.736+0000 7fb60bfff700 1 --2- 172.21.15.59:0/3274636893 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7fb61006b2c0 0x7fb61006d780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:01.740 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:01.736+0000 7fb60bfff700 1 -- 172.21.15.59:0/3274636893 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(63..63 src has 1..63) v4 ==== 5795+0+0 (secure 0 0 0) 0x7fb618089bd0 con 0x7fb62406e0a0 2023-11-24T07:59:01.740 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:01.736+0000 7fb622ffd700 1 --2- 172.21.15.59:0/3274636893 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7fb61006b2c0 0x7fb61006d780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:01.740 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:01.736+0000 7fb622ffd700 1 --2- 172.21.15.59:0/3274636893 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7fb61006b2c0 0x7fb61006d780 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7fb6241f7480 tx=0x7fb614008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:59:01.742 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:01.740+0000 7fb60bfff700 1 -- 172.21.15.59:0/3274636893 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fb6180546d0 con 0x7fb62406e0a0 2023-11-24T07:59:01.989 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:01.988+0000 7fb629cff700 1 -- 172.21.15.59:0/3274636893 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7fb6240ae980 con 0x7fb62406e0a0 2023-11-24T07:59:01.996 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-24T07:59:01.998 DEBUG:teuthology.orchestra.run.smithi076:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-24T07:59:02.593 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:02.591+0000 7fb60bfff700 1 -- 172.21.15.59:0/3274636893 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v38) v1 ==== 104+0+0 (secure 0 0 0) 0x7fb6240ae980 con 0x7fb62406e0a0 2023-11-24T07:59:02.596 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:02.594+0000 7fb629cff700 1 -- 172.21.15.59:0/3274636893 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7fb61006b2c0 msgr2=0x7fb61006d780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:02.596 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:02.594+0000 7fb629cff700 1 --2- 172.21.15.59:0/3274636893 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7fb61006b2c0 0x7fb61006d780 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7fb6241f7480 tx=0x7fb614008040 comp rx=0 tx=0).stop 2023-11-24T07:59:02.596 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:02.594+0000 7fb629cff700 1 -- 172.21.15.59:0/3274636893 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb62406e0a0 msgr2=0x7fb62415b500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:02.596 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:02.594+0000 7fb629cff700 1 --2- 172.21.15.59:0/3274636893 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb62406e0a0 0x7fb62415b500 secure :-1 s=READY pgs=806 cs=0 l=1 rev1=1 crypto rx=0x7fb61800b470 tx=0x7fb61800b780 comp rx=0 tx=0).stop 2023-11-24T07:59:02.596 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:02.594+0000 7fb629cff700 1 -- 172.21.15.59:0/3274636893 shutdown_connections 2023-11-24T07:59:02.596 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:02.594+0000 7fb629cff700 1 --2- 172.21.15.59:0/3274636893 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7fb61006b2c0 0x7fb61006d780 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:02.596 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:02.594+0000 7fb629cff700 1 --2- 172.21.15.59:0/3274636893 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb6240b4090 0x7fb624144d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:02.597 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:02.594+0000 7fb629cff700 1 --2- 172.21.15.59:0/3274636893 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb62406e0a0 0x7fb62415b500 unknown :-1 s=CLOSED pgs=806 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:02.597 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:02.594+0000 7fb629cff700 1 --2- 172.21.15.59:0/3274636893 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fb624072a00 0x7fb624144480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:02.597 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:02.594+0000 7fb629cff700 1 -- 172.21.15.59:0/3274636893 >> 172.21.15.59:0/3274636893 conn(0x7fb6240b2740 msgr2=0x7fb62415e510 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:59:02.597 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:02.594+0000 7fb629cff700 1 -- 172.21.15.59:0/3274636893 shutdown_connections 2023-11-24T07:59:02.597 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:02.594+0000 7fb629cff700 1 -- 172.21.15.59:0/3274636893 wait complete. 2023-11-24T07:59:02.611 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2023-11-24T07:59:02.747 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:02.745+0000 7fb5fb2b3700 1 Processor -- start 2023-11-24T07:59:02.747 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:02.746+0000 7fb5fb2b3700 1 -- start start 2023-11-24T07:59:02.750 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:02.746+0000 7fb5fb2b3700 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fb5f415be60 0x7fb5f40cc760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:02.750 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:02.746+0000 7fb5fb2b3700 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb5f40b4ae0 con 0x7fb5f40d0230 2023-11-24T07:59:02.750 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:02.746+0000 7fb5fb2b3700 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb5f40b50a0 con 0x7fb5f415c240 2023-11-24T07:59:02.750 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:02.746+0000 7fb5fb2b3700 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb5f40d0730 con 0x7fb5f415be60 2023-11-24T07:59:02.751 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:02.747+0000 7fb5f9850700 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7fb5f40d0230 0x7fb5f40b5530 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:57960/0 (socket says 172.21.15.59:57960) 2023-11-24T07:59:02.751 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:02.747+0000 7fb5f9850700 1 -- 172.21.15.59:0/522727552 learned_addr learned my addr 172.21.15.59:0/522727552 (peer_addr_for_me v1:172.21.15.59:0/0) 2023-11-24T07:59:02.751 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:02.747+0000 7fb5f904f700 1 --2- 172.21.15.59:0/522727552 >> v2:172.21.15.76:3300/0 conn(0x7fb5f415be60 0x7fb5f40cc760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:02.751 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:02.747+0000 7fb5ebfff700 1 -- 172.21.15.59:0/522727552 <== mon.0 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2266017410 0 0) 0x7fb5f40b4ae0 con 0x7fb5f40d0230 2023-11-24T07:59:02.751 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:02.747+0000 7fb5ebfff700 1 -- 172.21.15.59:0/522727552 --> v1:172.21.15.59:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb5e00035b0 con 0x7fb5f40d0230 2023-11-24T07:59:02.751 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:02.747+0000 7fb5f904f700 1 -- 172.21.15.59:0/522727552 >> v1:172.21.15.76:6789/0 conn(0x7fb5f415c240 legacy=0x7fb5f40ccca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:02.751 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:02.747+0000 7fb5f904f700 1 -- 172.21.15.59:0/522727552 >> v1:172.21.15.59:6789/0 conn(0x7fb5f40d0230 legacy=0x7fb5f40b5530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:02.751 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:02.747+0000 7fb5ebfff700 1 -- 172.21.15.59:0/522727552 <== mon.0 v1:172.21.15.59:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 574944127 0 0) 0x7fb5e00035b0 con 0x7fb5f40d0230 2023-11-24T07:59:02.751 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:02.747+0000 7fb5f904f700 1 -- 172.21.15.59:0/522727552 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb5f40d0a10 con 0x7fb5f415be60 2023-11-24T07:59:02.751 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:02.747+0000 7fb5f904f700 1 --2- 172.21.15.59:0/522727552 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb5f415be60 0x7fb5f40cc760 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7fb5e4009840 tx=0x7fb5e4009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=377ed681f26f43cd server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:59:02.751 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:02.747+0000 7fb5ebfff700 1 -- 172.21.15.59:0/522727552 <== mon.1 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5ec0019f0 con 0x7fb5f415be60 2023-11-24T07:59:02.751 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:02.747+0000 7fb5ebfff700 1 -- 172.21.15.59:0/522727552 <== mon.1 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb5e4004030 con 0x7fb5f415be60 2023-11-24T07:59:02.751 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:02.747+0000 7fb5ebfff700 1 -- 172.21.15.59:0/522727552 <== mon.1 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5e40043c0 con 0x7fb5f415be60 2023-11-24T07:59:02.752 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:02.748+0000 7fb5fb2b3700 1 -- 172.21.15.59:0/522727552 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb5f415be60 msgr2=0x7fb5f40cc760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:02.752 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:02.748+0000 7fb5fb2b3700 1 --2- 172.21.15.59:0/522727552 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb5f415be60 0x7fb5f40cc760 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7fb5e4009840 tx=0x7fb5e4009b50 comp rx=0 tx=0).stop 2023-11-24T07:59:02.752 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:02.748+0000 7fb5fb2b3700 1 -- 172.21.15.59:0/522727552 shutdown_connections 2023-11-24T07:59:02.752 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:02.748+0000 7fb5fb2b3700 1 --2- 172.21.15.59:0/522727552 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb5f415be60 0x7fb5f40cc760 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:02.752 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:02.748+0000 7fb5fb2b3700 1 -- 172.21.15.59:0/522727552 >> 172.21.15.59:0/522727552 conn(0x7fb5f40b37e0 msgr2=0x7fb5f40b3bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:59:02.752 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:02.748+0000 7fb5fb2b3700 1 -- 172.21.15.59:0/522727552 shutdown_connections 2023-11-24T07:59:02.752 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:02.748+0000 7fb5fb2b3700 1 -- 172.21.15.59:0/522727552 wait complete. 2023-11-24T07:59:02.754 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:02.749+0000 7fb5fb2b3700 1 Processor -- start 2023-11-24T07:59:02.755 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:02.749+0000 7fb5fb2b3700 1 -- start start 2023-11-24T07:59:02.755 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:02.750+0000 7fb5fb2b3700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb5f40d0230 0x7fb5f414c120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:02.755 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:02.750+0000 7fb5fb2b3700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb5f415be60 0x7fb5f40b5f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:02.755 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:02.750+0000 7fb5fb2b3700 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fb5f40b64c0 0x7fb5f40c65c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:02.755 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:02.750+0000 7fb5fb2b3700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5f414c660 con 0x7fb5f415be60 2023-11-24T07:59:02.755 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:02.750+0000 7fb5fb2b3700 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5f40b6950 con 0x7fb5f40d0230 2023-11-24T07:59:02.756 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:02.750+0000 7fb5fb2b3700 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb5f41f3190 con 0x7fb5f40b64c0 2023-11-24T07:59:02.756 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:02.750+0000 7fb5f884e700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb5f415be60 0x7fb5f40b5f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:02.756 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:02.750+0000 7fb5f884e700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb5f415be60 0x7fb5f40b5f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:54492/0 (socket says 172.21.15.59:54492) 2023-11-24T07:59:02.756 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:02.750+0000 7fb5f884e700 1 -- 172.21.15.59:0/3947011263 learned_addr learned my addr 172.21.15.59:0/3947011263 (peer_addr_for_me v2:172.21.15.59:0/0) 2023-11-24T07:59:02.756 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:02.750+0000 7fb5f904f700 1 --2- 172.21.15.59:0/3947011263 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb5f40d0230 0x7fb5f414c120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:02.756 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:02.751+0000 7fb5f884e700 1 -- 172.21.15.59:0/3947011263 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fb5f40b64c0 msgr2=0x7fb5f40c65c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:02.756 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:02.751+0000 7fb5f884e700 1 --2- 172.21.15.59:0/3947011263 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fb5f40b64c0 0x7fb5f40c65c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:02.756 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:02.751+0000 7fb5f884e700 1 -- 172.21.15.59:0/3947011263 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb5f40d0230 msgr2=0x7fb5f414c120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:02.756 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:02.751+0000 7fb5f884e700 1 --2- 172.21.15.59:0/3947011263 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb5f40d0230 0x7fb5f414c120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:02.756 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:02.751+0000 7fb5f884e700 1 -- 172.21.15.59:0/3947011263 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb5e40094a0 con 0x7fb5f415be60 2023-11-24T07:59:02.756 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:02.751+0000 7fb5f884e700 1 --2- 172.21.15.59:0/3947011263 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb5f415be60 0x7fb5f40b5f80 secure :-1 s=READY pgs=808 cs=0 l=1 rev1=1 crypto rx=0x7fb5f000c4e0 tx=0x7fb5f000c8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:59:02.757 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:02.751+0000 7fb5e9ffb700 1 -- 172.21.15.59:0/3947011263 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5f00052d0 con 0x7fb5f415be60 2023-11-24T07:59:02.757 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:02.751+0000 7fb5e9ffb700 1 -- 172.21.15.59:0/3947011263 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb5f0005430 con 0x7fb5f415be60 2023-11-24T07:59:02.757 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:02.751+0000 7fb5e9ffb700 1 -- 172.21.15.59:0/3947011263 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5f00038c0 con 0x7fb5f415be60 2023-11-24T07:59:02.757 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:02.751+0000 7fb5fb2b3700 1 -- 172.21.15.59:0/3947011263 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb5f41f23d0 con 0x7fb5f415be60 2023-11-24T07:59:02.757 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:02.751+0000 7fb5fb2b3700 1 -- 172.21.15.59:0/3947011263 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb5f41f2990 con 0x7fb5f415be60 2023-11-24T07:59:02.757 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:02.752+0000 7fb5fb2b3700 1 -- 172.21.15.59:0/3947011263 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb5f4068670 con 0x7fb5f415be60 2023-11-24T07:59:02.757 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:02.752+0000 7fb5e9ffb700 1 -- 172.21.15.59:0/3947011263 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 10) v1 ==== 89588+0+0 (secure 0 0 0) 0x7fb5f0003a20 con 0x7fb5f415be60 2023-11-24T07:59:02.757 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:02.753+0000 7fb5e9ffb700 1 --2- 172.21.15.59:0/3947011263 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7fb5d806b2c0 0x7fb5d806d780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:02.757 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:02.753+0000 7fb5e9ffb700 1 -- 172.21.15.59:0/3947011263 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(63..63 src has 1..63) v4 ==== 5795+0+0 (secure 0 0 0) 0x7fb5f0089250 con 0x7fb5f415be60 2023-11-24T07:59:02.757 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:02.753+0000 7fb5f904f700 1 --2- 172.21.15.59:0/3947011263 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7fb5d806b2c0 0x7fb5d806d780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:02.758 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:02.753+0000 7fb5f904f700 1 --2- 172.21.15.59:0/3947011263 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7fb5d806b2c0 0x7fb5d806d780 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7fb5e40090b0 tx=0x7fb5e4008030 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:59:02.759 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:02.757+0000 7fb5e9ffb700 1 -- 172.21.15.59:0/3947011263 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fb5f0053cd0 con 0x7fb5f415be60 2023-11-24T07:59:02.962 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:02.961+0000 7fb5fb2b3700 1 -- 172.21.15.59:0/3947011263 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fb5f415ad50 con 0x7fb5f415be60 2023-11-24T07:59:03.594 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:03.593+0000 7fb5e9ffb700 1 -- 172.21.15.59:0/3947011263 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v64) v1 ==== 181+0+0 (secure 0 0 0) 0x7fb5f00572f0 con 0x7fb5f415be60 2023-11-24T07:59:03.598 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:03.596+0000 7fb5fb2b3700 1 -- 172.21.15.59:0/3947011263 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7fb5d806b2c0 msgr2=0x7fb5d806d780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:03.598 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:03.596+0000 7fb5fb2b3700 1 --2- 172.21.15.59:0/3947011263 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7fb5d806b2c0 0x7fb5d806d780 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7fb5e40090b0 tx=0x7fb5e4008030 comp rx=0 tx=0).stop 2023-11-24T07:59:03.598 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:03.596+0000 7fb5fb2b3700 1 -- 172.21.15.59:0/3947011263 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb5f415be60 msgr2=0x7fb5f40b5f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:03.598 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:03.596+0000 7fb5fb2b3700 1 --2- 172.21.15.59:0/3947011263 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb5f415be60 0x7fb5f40b5f80 secure :-1 s=READY pgs=808 cs=0 l=1 rev1=1 crypto rx=0x7fb5f000c4e0 tx=0x7fb5f000c8a0 comp rx=0 tx=0).stop 2023-11-24T07:59:03.598 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:03.596+0000 7fb5fb2b3700 1 -- 172.21.15.59:0/3947011263 shutdown_connections 2023-11-24T07:59:03.598 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:03.596+0000 7fb5fb2b3700 1 --2- 172.21.15.59:0/3947011263 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7fb5d806b2c0 0x7fb5d806d780 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:03.598 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:03.596+0000 7fb5fb2b3700 1 --2- 172.21.15.59:0/3947011263 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fb5f40b64c0 0x7fb5f40c65c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:03.598 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:03.596+0000 7fb5fb2b3700 1 --2- 172.21.15.59:0/3947011263 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb5f40d0230 0x7fb5f414c120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:03.599 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:03.596+0000 7fb5fb2b3700 1 --2- 172.21.15.59:0/3947011263 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb5f415be60 0x7fb5f40b5f80 unknown :-1 s=CLOSED pgs=808 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:03.599 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:03.596+0000 7fb5fb2b3700 1 -- 172.21.15.59:0/3947011263 >> 172.21.15.59:0/3947011263 conn(0x7fb5f40b37e0 msgr2=0x7fb5f4070690 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:59:03.599 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:03.596+0000 7fb5fb2b3700 1 -- 172.21.15.59:0/3947011263 shutdown_connections 2023-11-24T07:59:03.599 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:03.596+0000 7fb5fb2b3700 1 -- 172.21.15.59:0/3947011263 wait complete. 2023-11-24T07:59:03.599 INFO:teuthology.orchestra.run.smithi059.stderr:pool 'cephfs_metadata' removed 2023-11-24T07:59:03.614 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2023-11-24T07:59:03.750 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:03.748+0000 7f4aef6e0700 1 Processor -- start 2023-11-24T07:59:03.750 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:03.749+0000 7f4aef6e0700 1 -- start start 2023-11-24T07:59:03.753 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:03.749+0000 7f4aef6e0700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f4ae81440f0 0x7f4ae80b51b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:03.753 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:03.749+0000 7f4aef6e0700 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4ae8143ec0 con 0x7f4ae80722c0 2023-11-24T07:59:03.753 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:03.749+0000 7f4aef6e0700 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4ae80b3ae0 con 0x7f4ae8143ae0 2023-11-24T07:59:03.753 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:03.749+0000 7f4aef6e0700 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4ae80b40a0 con 0x7f4ae81440f0 2023-11-24T07:59:03.753 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:03.750+0000 7f4aedc7d700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f4ae81440f0 0x7f4ae80b51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:03.753 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:03.750+0000 7f4aecc7b700 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7f4ae80722c0 0x7f4ae806e510 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:57974/0 (socket says 172.21.15.59:57974) 2023-11-24T07:59:03.753 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:03.750+0000 7f4aecc7b700 1 -- 172.21.15.59:0/75917429 learned_addr learned my addr 172.21.15.59:0/75917429 (peer_addr_for_me v1:172.21.15.59:0/0) 2023-11-24T07:59:03.753 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:03.750+0000 7f4aedc7d700 1 -- 172.21.15.59:0/75917429 >> v1:172.21.15.76:6789/0 conn(0x7f4ae8143ae0 legacy=0x7f4ae8071bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:03.753 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:03.750+0000 7f4aedc7d700 1 -- 172.21.15.59:0/75917429 >> v1:172.21.15.59:6789/0 conn(0x7f4ae80722c0 legacy=0x7f4ae806e510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:03.753 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:03.750+0000 7f4aedc7d700 1 -- 172.21.15.59:0/75917429 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4ae80b3dc0 con 0x7f4ae81440f0 2023-11-24T07:59:03.753 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:03.750+0000 7f4adbfff700 1 -- 172.21.15.59:0/75917429 <== mon.0 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4047820481 0 0) 0x7f4ae8143ec0 con 0x7f4ae80722c0 2023-11-24T07:59:03.754 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:03.750+0000 7f4aedc7d700 1 --2- 172.21.15.59:0/75917429 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4ae81440f0 0x7f4ae80b51b0 secure :-1 s=READY pgs=810 cs=0 l=1 rev1=1 crypto rx=0x7f4ae0009d90 tx=0x7f4ae000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ae5552bc0e02de5b server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:59:03.754 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:03.750+0000 7f4adbfff700 1 -- 172.21.15.59:0/75917429 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ae000e070 con 0x7f4ae81440f0 2023-11-24T07:59:03.754 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:03.750+0000 7f4adbfff700 1 -- 172.21.15.59:0/75917429 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4ae000bbd0 con 0x7f4ae81440f0 2023-11-24T07:59:03.754 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:03.750+0000 7f4adbfff700 1 -- 172.21.15.59:0/75917429 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ae0012430 con 0x7f4ae81440f0 2023-11-24T07:59:03.754 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:03.751+0000 7f4aef6e0700 1 -- 172.21.15.59:0/75917429 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4ae81440f0 msgr2=0x7f4ae80b51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:03.754 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:03.751+0000 7f4aef6e0700 1 --2- 172.21.15.59:0/75917429 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4ae81440f0 0x7f4ae80b51b0 secure :-1 s=READY pgs=810 cs=0 l=1 rev1=1 crypto rx=0x7f4ae0009d90 tx=0x7f4ae000b330 comp rx=0 tx=0).stop 2023-11-24T07:59:03.754 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:03.751+0000 7f4aef6e0700 1 -- 172.21.15.59:0/75917429 shutdown_connections 2023-11-24T07:59:03.754 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:03.751+0000 7f4aef6e0700 1 --2- 172.21.15.59:0/75917429 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4ae81440f0 0x7f4ae80b51b0 unknown :-1 s=CLOSED pgs=810 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:03.754 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:03.751+0000 7f4aef6e0700 1 -- 172.21.15.59:0/75917429 >> 172.21.15.59:0/75917429 conn(0x7f4ae80b2740 msgr2=0x7f4ae80b2b20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:59:03.754 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:03.751+0000 7f4aef6e0700 1 -- 172.21.15.59:0/75917429 shutdown_connections 2023-11-24T07:59:03.754 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:03.751+0000 7f4aef6e0700 1 -- 172.21.15.59:0/75917429 wait complete. 2023-11-24T07:59:03.757 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:03.752+0000 7f4aef6e0700 1 Processor -- start 2023-11-24T07:59:03.757 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:03.752+0000 7f4aef6e0700 1 -- start start 2023-11-24T07:59:03.757 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:03.753+0000 7f4aef6e0700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4ae80722c0 0x7f4ae813a590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:03.758 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:03.753+0000 7f4aef6e0700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4ae8143ae0 0x7f4ae813add0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:03.758 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:03.753+0000 7f4aef6e0700 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f4ae81440f0 0x7f4ae813c6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:03.758 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:03.753+0000 7f4aef6e0700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ae813aad0 con 0x7f4ae80722c0 2023-11-24T07:59:03.758 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:03.753+0000 7f4aef6e0700 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ae813b620 con 0x7f4ae8143ae0 2023-11-24T07:59:03.758 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:03.753+0000 7f4aef6e0700 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4ae813b900 con 0x7f4ae81440f0 2023-11-24T07:59:03.758 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:03.753+0000 7f4aed47c700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4ae80722c0 0x7f4ae813a590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:03.758 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:03.753+0000 7f4aed47c700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4ae80722c0 0x7f4ae813a590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:54504/0 (socket says 172.21.15.59:54504) 2023-11-24T07:59:03.758 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:03.753+0000 7f4aed47c700 1 -- 172.21.15.59:0/3418991817 learned_addr learned my addr 172.21.15.59:0/3418991817 (peer_addr_for_me v2:172.21.15.59:0/0) 2023-11-24T07:59:03.758 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:03.753+0000 7f4aecc7b700 1 --2- 172.21.15.59:0/3418991817 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4ae8143ae0 0x7f4ae813add0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:03.759 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:03.753+0000 7f4aedc7d700 1 --2- 172.21.15.59:0/3418991817 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f4ae81440f0 0x7f4ae813c6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:03.759 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:03.753+0000 7f4aed47c700 1 -- 172.21.15.59:0/3418991817 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f4ae81440f0 msgr2=0x7f4ae813c6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:03.759 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:03.753+0000 7f4aed47c700 1 --2- 172.21.15.59:0/3418991817 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f4ae81440f0 0x7f4ae813c6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:03.759 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:03.753+0000 7f4aed47c700 1 -- 172.21.15.59:0/3418991817 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4ae8143ae0 msgr2=0x7f4ae813add0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:03.759 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:03.753+0000 7f4aed47c700 1 --2- 172.21.15.59:0/3418991817 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4ae8143ae0 0x7f4ae813add0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:03.759 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:03.753+0000 7f4aed47c700 1 -- 172.21.15.59:0/3418991817 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4ae0009a70 con 0x7f4ae80722c0 2023-11-24T07:59:03.759 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:03.753+0000 7f4aedc7d700 1 --2- 172.21.15.59:0/3418991817 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f4ae81440f0 0x7f4ae813c6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-24T07:59:03.759 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:03.753+0000 7f4aed47c700 1 --2- 172.21.15.59:0/3418991817 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4ae80722c0 0x7f4ae813a590 secure :-1 s=READY pgs=811 cs=0 l=1 rev1=1 crypto rx=0x7f4ae400e940 tx=0x7f4ae400ec50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:59:03.759 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:03.753+0000 7f4ad9ffb700 1 -- 172.21.15.59:0/3418991817 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ae40052d0 con 0x7f4ae80722c0 2023-11-24T07:59:03.759 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:03.754+0000 7f4ad9ffb700 1 -- 172.21.15.59:0/3418991817 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4ae4005430 con 0x7f4ae80722c0 2023-11-24T07:59:03.759 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:03.754+0000 7f4ad9ffb700 1 -- 172.21.15.59:0/3418991817 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ae400ca20 con 0x7f4ae80722c0 2023-11-24T07:59:03.759 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:03.754+0000 7f4aef6e0700 1 -- 172.21.15.59:0/3418991817 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4ae813bbe0 con 0x7f4ae80722c0 2023-11-24T07:59:03.760 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:03.754+0000 7f4aef6e0700 1 -- 172.21.15.59:0/3418991817 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4ae813c1a0 con 0x7f4ae80722c0 2023-11-24T07:59:03.760 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:03.755+0000 7f4aef6e0700 1 -- 172.21.15.59:0/3418991817 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4ae80cd8d0 con 0x7f4ae80722c0 2023-11-24T07:59:03.760 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:03.755+0000 7f4ad9ffb700 1 -- 172.21.15.59:0/3418991817 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 10) v1 ==== 89588+0+0 (secure 0 0 0) 0x7f4ae400cb80 con 0x7f4ae80722c0 2023-11-24T07:59:03.760 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:03.755+0000 7f4ad9ffb700 1 --2- 172.21.15.59:0/3418991817 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f4acc06b2c0 0x7f4acc06d780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:03.760 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:03.755+0000 7f4ad9ffb700 1 -- 172.21.15.59:0/3418991817 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(64..64 src has 1..64) v4 ==== 5357+0+0 (secure 0 0 0) 0x7f4ae408bd80 con 0x7f4ae80722c0 2023-11-24T07:59:03.760 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:03.756+0000 7f4aecc7b700 1 --2- 172.21.15.59:0/3418991817 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f4acc06b2c0 0x7f4acc06d780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:03.760 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:03.756+0000 7f4aecc7b700 1 --2- 172.21.15.59:0/3418991817 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f4acc06b2c0 0x7f4acc06d780 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f4adc007b10 tx=0x7f4adc007a60 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:59:03.761 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:03.760+0000 7f4ad9ffb700 1 -- 172.21.15.59:0/3418991817 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f4ae40568a0 con 0x7f4ae80722c0 2023-11-24T07:59:03.964 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:03.962+0000 7f4aef6e0700 1 -- 172.21.15.59:0/3418991817 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f4ae813c3c0 con 0x7f4ae80722c0 2023-11-24T07:59:04.600 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:04.599+0000 7f4ad9ffb700 1 -- 172.21.15.59:0/3418991817 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v65) v1 ==== 169+0+0 (secure 0 0 0) 0x7f4ae401f020 con 0x7f4ae80722c0 2023-11-24T07:59:04.605 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:04.603+0000 7f4aef6e0700 1 -- 172.21.15.59:0/3418991817 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f4acc06b2c0 msgr2=0x7f4acc06d780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:04.605 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:04.603+0000 7f4aef6e0700 1 --2- 172.21.15.59:0/3418991817 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f4acc06b2c0 0x7f4acc06d780 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f4adc007b10 tx=0x7f4adc007a60 comp rx=0 tx=0).stop 2023-11-24T07:59:04.605 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:04.603+0000 7f4aef6e0700 1 -- 172.21.15.59:0/3418991817 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4ae80722c0 msgr2=0x7f4ae813a590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:04.605 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:04.603+0000 7f4aef6e0700 1 --2- 172.21.15.59:0/3418991817 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4ae80722c0 0x7f4ae813a590 secure :-1 s=READY pgs=811 cs=0 l=1 rev1=1 crypto rx=0x7f4ae400e940 tx=0x7f4ae400ec50 comp rx=0 tx=0).stop 2023-11-24T07:59:04.605 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:04.603+0000 7f4aef6e0700 1 -- 172.21.15.59:0/3418991817 shutdown_connections 2023-11-24T07:59:04.605 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:04.603+0000 7f4aef6e0700 1 --2- 172.21.15.59:0/3418991817 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f4acc06b2c0 0x7f4acc06d780 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:04.605 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:04.603+0000 7f4aef6e0700 1 --2- 172.21.15.59:0/3418991817 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f4ae81440f0 0x7f4ae813c6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:04.605 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:04.603+0000 7f4aef6e0700 1 --2- 172.21.15.59:0/3418991817 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4ae80722c0 0x7f4ae813a590 secure :-1 s=CLOSED pgs=811 cs=0 l=1 rev1=1 crypto rx=0x7f4ae400e940 tx=0x7f4ae400ec50 comp rx=0 tx=0).stop 2023-11-24T07:59:04.606 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:04.603+0000 7f4aef6e0700 1 --2- 172.21.15.59:0/3418991817 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4ae8143ae0 0x7f4ae813add0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:04.606 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:04.603+0000 7f4aef6e0700 1 -- 172.21.15.59:0/3418991817 >> 172.21.15.59:0/3418991817 conn(0x7f4ae80b2740 msgr2=0x7f4ae806e410 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:59:04.606 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:04.604+0000 7f4aef6e0700 1 -- 172.21.15.59:0/3418991817 shutdown_connections 2023-11-24T07:59:04.606 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:04.604+0000 7f4aef6e0700 1 -- 172.21.15.59:0/3418991817 wait complete. 2023-11-24T07:59:04.606 INFO:teuthology.orchestra.run.smithi059.stderr:pool 'cephfs_data' removed 2023-11-24T07:59:04.621 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2023-11-24T07:59:04.754 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:04.752+0000 7f36ec5f3700 1 Processor -- start 2023-11-24T07:59:04.754 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:04.752+0000 7f36ec5f3700 1 -- start start 2023-11-24T07:59:04.758 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:04.753+0000 7f36ec5f3700 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f36e406b620 0x7f36e40b3a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:04.758 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:04.753+0000 7f36ec5f3700 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f36e41629c0 con 0x7f36e40b5ab0 2023-11-24T07:59:04.758 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:04.753+0000 7f36ec5f3700 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f36e4071f50 con 0x7f36e406ee60 2023-11-24T07:59:04.758 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:04.753+0000 7f36ec5f3700 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f36e4072230 con 0x7f36e406b620 2023-11-24T07:59:04.758 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:04.753+0000 7f36e9b8e700 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7f36e40b5ab0 0x7f36e4161ba0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:57986/0 (socket says 172.21.15.59:57986) 2023-11-24T07:59:04.758 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:04.753+0000 7f36e9b8e700 1 -- 172.21.15.59:0/181878389 learned_addr learned my addr 172.21.15.59:0/181878389 (peer_addr_for_me v1:172.21.15.59:0/0) 2023-11-24T07:59:04.758 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:04.754+0000 7f36ea38f700 1 --2- 172.21.15.59:0/181878389 >> v2:172.21.15.76:3300/0 conn(0x7f36e406b620 0x7f36e40b3a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:04.758 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:04.754+0000 7f36e938d700 1 -- 172.21.15.59:0/181878389 <== mon.0 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1018566694 0 0) 0x7f36e41629c0 con 0x7f36e40b5ab0 2023-11-24T07:59:04.758 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:04.754+0000 7f36e938d700 1 -- 172.21.15.59:0/181878389 --> v1:172.21.15.59:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f36d00035b0 con 0x7f36e40b5ab0 2023-11-24T07:59:04.758 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:04.754+0000 7f36e938d700 1 -- 172.21.15.59:0/181878389 <== mon.0 v1:172.21.15.59:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1620519632 0 0) 0x7f36d00035b0 con 0x7f36e40b5ab0 2023-11-24T07:59:04.759 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:04.754+0000 7f36e938d700 1 -- 172.21.15.59:0/181878389 >> v2:172.21.15.76:3300/0 conn(0x7f36e406b620 msgr2=0x7f36e40b3a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T07:59:04.759 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:04.754+0000 7f36e938d700 1 --2- 172.21.15.59:0/181878389 >> v2:172.21.15.76:3300/0 conn(0x7f36e406b620 0x7f36e40b3a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:04.759 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:04.754+0000 7f36e938d700 1 -- 172.21.15.59:0/181878389 >> v1:172.21.15.76:6789/0 conn(0x7f36e406ee60 legacy=0x7f36e41622b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:04.759 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:04.754+0000 7f36e938d700 1 -- 172.21.15.59:0/181878389 --> v1:172.21.15.59:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f36e4072510 con 0x7f36e40b5ab0 2023-11-24T07:59:04.759 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:04.754+0000 7f36e938d700 1 -- 172.21.15.59:0/181878389 <== mon.0 v1:172.21.15.59:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1312333566 0 0) 0x7f36e0001670 con 0x7f36e40b5ab0 2023-11-24T07:59:04.759 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:04.754+0000 7f36e938d700 1 -- 172.21.15.59:0/181878389 >> v1:172.21.15.59:6789/0 conn(0x7f36e40b5ab0 legacy=0x7f36e4161ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:04.759 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:04.754+0000 7f36e938d700 1 --2- 172.21.15.59:0/181878389 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f36d0003a00 0x7f36d0003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:04.759 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:04.754+0000 7f36e938d700 1 --2- 172.21.15.59:0/181878389 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f36d0004850 0x7f36d0004cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:04.759 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:04.754+0000 7f36e938d700 1 --2- 172.21.15.59:0/181878389 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f36d0005210 0x7f36d000ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:04.759 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:04.754+0000 7f36e938d700 1 -- 172.21.15.59:0/181878389 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36e4072230 con 0x7f36d0003a00 2023-11-24T07:59:04.759 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:04.754+0000 7f36e938d700 1 -- 172.21.15.59:0/181878389 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36d000c0b0 con 0x7f36d0005210 2023-11-24T07:59:04.759 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:04.754+0000 7f36e938d700 1 -- 172.21.15.59:0/181878389 --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f36d000c270 con 0x7f36d0004850 2023-11-24T07:59:04.760 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:04.754+0000 7f36ea38f700 1 --2- 172.21.15.59:0/181878389 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f36d0003a00 0x7f36d0003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:04.760 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:04.754+0000 7f36eab90700 1 --2- 172.21.15.59:0/181878389 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f36d0005210 0x7f36d000ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:04.760 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:04.754+0000 7f36e9b8e700 1 --2- 172.21.15.59:0/181878389 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f36d0004850 0x7f36d0004cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:04.760 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:04.755+0000 7f36ea38f700 1 -- 172.21.15.59:0/181878389 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f36d0004850 msgr2=0x7f36d0004cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T07:59:04.760 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:04.755+0000 7f36ea38f700 1 --2- 172.21.15.59:0/181878389 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f36d0004850 0x7f36d0004cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:04.760 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:04.755+0000 7f36ea38f700 1 -- 172.21.15.59:0/181878389 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f36d0005210 msgr2=0x7f36d000ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T07:59:04.760 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:04.755+0000 7f36ea38f700 1 --2- 172.21.15.59:0/181878389 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f36d0005210 0x7f36d000ba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:04.760 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:04.755+0000 7f36ea38f700 1 -- 172.21.15.59:0/181878389 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f36d00035b0 con 0x7f36d0003a00 2023-11-24T07:59:04.760 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:04.755+0000 7f36eab90700 1 -- 172.21.15.59:0/181878389 reap_dead start 2023-11-24T07:59:04.760 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:04.755+0000 7f36ea38f700 1 --2- 172.21.15.59:0/181878389 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f36d0003a00 0x7f36d0003e20 secure :-1 s=READY pgs=813 cs=0 l=1 rev1=1 crypto rx=0x7f36d4009b60 tx=0x7f36d0005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ca2a6206a06d093d server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:59:04.760 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:04.755+0000 7f36e938d700 1 -- 172.21.15.59:0/181878389 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36d400e6c0 con 0x7f36d0003a00 2023-11-24T07:59:04.760 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:04.755+0000 7f36e938d700 1 -- 172.21.15.59:0/181878389 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f36d400e820 con 0x7f36d0003a00 2023-11-24T07:59:04.761 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:04.755+0000 7f36e938d700 1 -- 172.21.15.59:0/181878389 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36d400eb90 con 0x7f36d0003a00 2023-11-24T07:59:04.761 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:04.755+0000 7f36ec5f3700 1 -- 172.21.15.59:0/181878389 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f36d0003a00 msgr2=0x7f36d0003e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:04.761 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:04.755+0000 7f36ec5f3700 1 --2- 172.21.15.59:0/181878389 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f36d0003a00 0x7f36d0003e20 secure :-1 s=READY pgs=813 cs=0 l=1 rev1=1 crypto rx=0x7f36d4009b60 tx=0x7f36d0005880 comp rx=0 tx=0).stop 2023-11-24T07:59:04.761 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:04.755+0000 7f36ec5f3700 1 -- 172.21.15.59:0/181878389 shutdown_connections 2023-11-24T07:59:04.761 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:04.755+0000 7f36ec5f3700 1 --2- 172.21.15.59:0/181878389 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f36d0003a00 0x7f36d0003e20 unknown :-1 s=CLOSED pgs=813 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:04.761 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:04.755+0000 7f36ec5f3700 1 -- 172.21.15.59:0/181878389 >> 172.21.15.59:0/181878389 conn(0x7f36e40b2740 msgr2=0x7f36e40b2b20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:59:04.761 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:04.755+0000 7f36ec5f3700 1 -- 172.21.15.59:0/181878389 shutdown_connections 2023-11-24T07:59:04.761 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:04.756+0000 7f36ec5f3700 1 -- 172.21.15.59:0/181878389 wait complete. 2023-11-24T07:59:04.764 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:04.757+0000 7f36ec5f3700 1 Processor -- start 2023-11-24T07:59:04.764 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:04.757+0000 7f36ec5f3700 1 -- start start 2023-11-24T07:59:04.764 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:04.757+0000 7f36ec5f3700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f36d0003a00 0x7f36e4141440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:04.764 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:04.757+0000 7f36ec5f3700 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f36e413ca00 0x7f36e413f200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:04.764 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:04.757+0000 7f36ec5f3700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f36e413f740 0x7f36e4161640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:04.764 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:04.757+0000 7f36ec5f3700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36e4141020 con 0x7f36e413f740 2023-11-24T07:59:04.764 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:04.758+0000 7f36ec5f3700 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36e4140050 con 0x7f36d0003a00 2023-11-24T07:59:04.765 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:04.758+0000 7f36ec5f3700 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f36e413fd70 con 0x7f36e413ca00 2023-11-24T07:59:04.765 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:04.758+0000 7f36eab90700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f36e413f740 0x7f36e4161640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:04.765 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:04.758+0000 7f36ea38f700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f36d0003a00 0x7f36e4141440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:04.765 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:04.758+0000 7f36eab90700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f36e413f740 0x7f36e4161640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:54526/0 (socket says 172.21.15.59:54526) 2023-11-24T07:59:04.765 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:04.758+0000 7f36eab90700 1 -- 172.21.15.59:0/2977655210 learned_addr learned my addr 172.21.15.59:0/2977655210 (peer_addr_for_me v2:172.21.15.59:0/0) 2023-11-24T07:59:04.765 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:04.758+0000 7f36e9b8e700 1 --2- 172.21.15.59:0/2977655210 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f36e413ca00 0x7f36e413f200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:04.765 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:04.758+0000 7f36eab90700 1 -- 172.21.15.59:0/2977655210 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f36e413ca00 msgr2=0x7f36e413f200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:04.765 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:04.758+0000 7f36eab90700 1 --2- 172.21.15.59:0/2977655210 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f36e413ca00 0x7f36e413f200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:04.765 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:04.758+0000 7f36eab90700 1 -- 172.21.15.59:0/2977655210 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f36d0003a00 msgr2=0x7f36e4141440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:04.766 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:04.758+0000 7f36eab90700 1 --2- 172.21.15.59:0/2977655210 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f36d0003a00 0x7f36e4141440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:04.766 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:04.758+0000 7f36eab90700 1 -- 172.21.15.59:0/2977655210 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f36e4140960 con 0x7f36e413f740 2023-11-24T07:59:04.766 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:04.758+0000 7f36eab90700 1 --2- 172.21.15.59:0/2977655210 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f36e413f740 0x7f36e4161640 secure :-1 s=READY pgs=814 cs=0 l=1 rev1=1 crypto rx=0x7f36e4073020 tx=0x7f36dc00ff00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:59:04.766 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:04.758+0000 7f36daffd700 1 -- 172.21.15.59:0/2977655210 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36dc003cd0 con 0x7f36e413f740 2023-11-24T07:59:04.766 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:04.758+0000 7f36daffd700 1 -- 172.21.15.59:0/2977655210 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f36dc003e30 con 0x7f36e413f740 2023-11-24T07:59:04.766 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:04.758+0000 7f36ec5f3700 1 -- 172.21.15.59:0/2977655210 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f36e4140c40 con 0x7f36e413f740 2023-11-24T07:59:04.766 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:04.758+0000 7f36daffd700 1 -- 172.21.15.59:0/2977655210 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36dc005310 con 0x7f36e413f740 2023-11-24T07:59:04.766 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:04.758+0000 7f36ec5f3700 1 -- 172.21.15.59:0/2977655210 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f36e413ce80 con 0x7f36e413f740 2023-11-24T07:59:04.766 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:04.759+0000 7f36ec5f3700 1 -- 172.21.15.59:0/2977655210 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f36e41407c0 con 0x7f36e413f740 2023-11-24T07:59:04.766 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:04.760+0000 7f36daffd700 1 -- 172.21.15.59:0/2977655210 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 10) v1 ==== 89588+0+0 (secure 0 0 0) 0x7f36dc005470 con 0x7f36e413f740 2023-11-24T07:59:04.766 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:04.760+0000 7f36daffd700 1 --2- 172.21.15.59:0/2977655210 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f36c806b2c0 0x7f36c806d780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:04.766 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:04.760+0000 7f36daffd700 1 -- 172.21.15.59:0/2977655210 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(65..65 src has 1..65) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f36dc0899f0 con 0x7f36e413f740 2023-11-24T07:59:04.767 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:04.760+0000 7f36ea38f700 1 --2- 172.21.15.59:0/2977655210 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f36c806b2c0 0x7f36c806d780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:04.767 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:04.761+0000 7f36ea38f700 1 --2- 172.21.15.59:0/2977655210 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f36c806b2c0 0x7f36c806d780 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f36d40090b0 tx=0x7f36d401e040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:59:04.767 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:04.765+0000 7f36daffd700 1 -- 172.21.15.59:0/2977655210 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f36e41407c0 con 0x7f36e413f740 2023-11-24T07:59:04.972 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:04.971+0000 7f36ec5f3700 1 -- 172.21.15.59:0/2977655210 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7f36e41407c0 con 0x7f36e413f740 2023-11-24T07:59:04.975 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:04.974+0000 7f36daffd700 1 -- 172.21.15.59:0/2977655210 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v15) v1 ==== 104+0+0 (secure 0 0 0) 0x7f36e41407c0 con 0x7f36e413f740 2023-11-24T07:59:04.978 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:04.976+0000 7f36ec5f3700 1 -- 172.21.15.59:0/2977655210 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f36c806b2c0 msgr2=0x7f36c806d780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:04.978 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:04.976+0000 7f36ec5f3700 1 --2- 172.21.15.59:0/2977655210 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f36c806b2c0 0x7f36c806d780 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f36d40090b0 tx=0x7f36d401e040 comp rx=0 tx=0).stop 2023-11-24T07:59:04.978 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:04.976+0000 7f36ec5f3700 1 -- 172.21.15.59:0/2977655210 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f36e413f740 msgr2=0x7f36e4161640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:04.978 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:04.976+0000 7f36ec5f3700 1 --2- 172.21.15.59:0/2977655210 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f36e413f740 0x7f36e4161640 secure :-1 s=READY pgs=814 cs=0 l=1 rev1=1 crypto rx=0x7f36e4073020 tx=0x7f36dc00ff00 comp rx=0 tx=0).stop 2023-11-24T07:59:04.978 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:04.976+0000 7f36ec5f3700 1 -- 172.21.15.59:0/2977655210 shutdown_connections 2023-11-24T07:59:04.978 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:04.976+0000 7f36ec5f3700 1 --2- 172.21.15.59:0/2977655210 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f36c806b2c0 0x7f36c806d780 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:04.978 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:04.976+0000 7f36ec5f3700 1 --2- 172.21.15.59:0/2977655210 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f36e413f740 0x7f36e4161640 unknown :-1 s=CLOSED pgs=814 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:04.978 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:04.976+0000 7f36ec5f3700 1 --2- 172.21.15.59:0/2977655210 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f36d0003a00 0x7f36e4141440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:04.978 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:04.976+0000 7f36ec5f3700 1 --2- 172.21.15.59:0/2977655210 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f36e413ca00 0x7f36e413f200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:04.979 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:04.976+0000 7f36ec5f3700 1 -- 172.21.15.59:0/2977655210 >> 172.21.15.59:0/2977655210 conn(0x7f36e40b2740 msgr2=0x7f36e40b3830 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:59:04.979 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:04.976+0000 7f36ec5f3700 1 -- 172.21.15.59:0/2977655210 shutdown_connections 2023-11-24T07:59:04.979 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:04.976+0000 7f36ec5f3700 1 -- 172.21.15.59:0/2977655210 wait complete. 2023-11-24T07:59:04.990 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_volumes.TestSubvolumeSnapshots.test_nonexistent_subvolume_snapshot_rm' 2023-11-24T07:59:05.120 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:05.118+0000 7fe85de68700 1 Processor -- start 2023-11-24T07:59:05.120 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:05.118+0000 7fe85de68700 1 -- start start 2023-11-24T07:59:05.124 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:05.119+0000 7fe85de68700 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fe8581440f0 0x7fe8580b51b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:05.124 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:05.119+0000 7fe85de68700 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe858143ec0 con 0x7fe8580722c0 2023-11-24T07:59:05.125 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:05.119+0000 7fe85de68700 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe8580b3ae0 con 0x7fe858143ae0 2023-11-24T07:59:05.125 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:05.119+0000 7fe85de68700 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe8580b40a0 con 0x7fe8581440f0 2023-11-24T07:59:05.125 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:05.119+0000 7fe856ffd700 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7fe8580722c0 0x7fe85806e510 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:58004/0 (socket says 172.21.15.59:58004) 2023-11-24T07:59:05.125 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:05.119+0000 7fe856ffd700 1 -- 172.21.15.59:0/1145500391 learned_addr learned my addr 172.21.15.59:0/1145500391 (peer_addr_for_me v1:172.21.15.59:0/0) 2023-11-24T07:59:05.125 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:05.119+0000 7fe857fff700 1 --2- 172.21.15.59:0/1145500391 >> v2:172.21.15.76:3300/0 conn(0x7fe8581440f0 0x7fe8580b51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:05.125 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:05.119+0000 7fe8567fc700 1 -- 172.21.15.59:0/1145500391 <== mon.0 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3431518216 0 0) 0x7fe858143ec0 con 0x7fe8580722c0 2023-11-24T07:59:05.125 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:05.119+0000 7fe8567fc700 1 -- 172.21.15.59:0/1145500391 --> v1:172.21.15.59:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe8400035b0 con 0x7fe8580722c0 2023-11-24T07:59:05.125 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:05.119+0000 7fe8567fc700 1 -- 172.21.15.59:0/1145500391 <== mon.0 v1:172.21.15.59:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1941258866 0 0) 0x7fe8400035b0 con 0x7fe8580722c0 2023-11-24T07:59:05.125 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:05.119+0000 7fe8567fc700 1 -- 172.21.15.59:0/1145500391 >> v2:172.21.15.76:3300/0 conn(0x7fe8581440f0 msgr2=0x7fe8580b51b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T07:59:05.125 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:05.119+0000 7fe8567fc700 1 --2- 172.21.15.59:0/1145500391 >> v2:172.21.15.76:3300/0 conn(0x7fe8581440f0 0x7fe8580b51b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:05.126 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:05.119+0000 7fe8567fc700 1 -- 172.21.15.59:0/1145500391 >> v1:172.21.15.76:6789/0 conn(0x7fe858143ae0 legacy=0x7fe858071bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:05.126 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:05.119+0000 7fe8567fc700 1 -- 172.21.15.59:0/1145500391 --> v1:172.21.15.59:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe8580b3dc0 con 0x7fe8580722c0 2023-11-24T07:59:05.126 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:05.119+0000 7fe8567fc700 1 -- 172.21.15.59:0/1145500391 <== mon.0 v1:172.21.15.59:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1312333566 0 0) 0x7fe848001670 con 0x7fe8580722c0 2023-11-24T07:59:05.126 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:05.119+0000 7fe8567fc700 1 -- 172.21.15.59:0/1145500391 >> v1:172.21.15.59:6789/0 conn(0x7fe8580722c0 legacy=0x7fe85806e510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:05.126 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:05.119+0000 7fe8567fc700 1 --2- 172.21.15.59:0/1145500391 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fe840003a00 0x7fe840003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:05.126 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:05.119+0000 7fe8567fc700 1 --2- 172.21.15.59:0/1145500391 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe840004850 0x7fe840004cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:05.126 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:05.120+0000 7fe8567fc700 1 --2- 172.21.15.59:0/1145500391 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe840005210 0x7fe84000ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:05.126 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:05.120+0000 7fe8567fc700 1 -- 172.21.15.59:0/1145500391 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8580b40a0 con 0x7fe840004850 2023-11-24T07:59:05.126 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:05.120+0000 7fe8567fc700 1 -- 172.21.15.59:0/1145500391 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe84000c0b0 con 0x7fe840005210 2023-11-24T07:59:05.126 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:05.120+0000 7fe8567fc700 1 -- 172.21.15.59:0/1145500391 --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe84000c270 con 0x7fe840003a00 2023-11-24T07:59:05.127 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:05.120+0000 7fe856ffd700 1 --2- 172.21.15.59:0/1145500391 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe840004850 0x7fe840004cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:05.127 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:05.120+0000 7fe857fff700 1 --2- 172.21.15.59:0/1145500391 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe840005210 0x7fe84000ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:05.127 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:05.120+0000 7fe8577fe700 1 --2- 172.21.15.59:0/1145500391 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fe840003a00 0x7fe840003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:05.127 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:05.120+0000 7fe856ffd700 1 -- 172.21.15.59:0/1145500391 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fe840003a00 msgr2=0x7fe840003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T07:59:05.127 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:05.120+0000 7fe856ffd700 1 --2- 172.21.15.59:0/1145500391 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fe840003a00 0x7fe840003e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:05.127 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:05.120+0000 7fe856ffd700 1 -- 172.21.15.59:0/1145500391 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe840005210 msgr2=0x7fe84000ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T07:59:05.127 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:05.120+0000 7fe856ffd700 1 --2- 172.21.15.59:0/1145500391 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe840005210 0x7fe84000ba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:05.127 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:05.120+0000 7fe856ffd700 1 -- 172.21.15.59:0/1145500391 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe8400035b0 con 0x7fe840004850 2023-11-24T07:59:05.127 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:05.120+0000 7fe857fff700 1 -- 172.21.15.59:0/1145500391 reap_dead start 2023-11-24T07:59:05.127 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:05.120+0000 7fe856ffd700 1 --2- 172.21.15.59:0/1145500391 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe840004850 0x7fe840004cd0 secure :-1 s=READY pgs=816 cs=0 l=1 rev1=1 crypto rx=0x7fe84800c240 tx=0x7fe840005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2a78727a696675c6 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:59:05.127 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:05.120+0000 7fe8567fc700 1 -- 172.21.15.59:0/1145500391 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe848007c10 con 0x7fe840004850 2023-11-24T07:59:05.127 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:05.120+0000 7fe8567fc700 1 -- 172.21.15.59:0/1145500391 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe84800cd80 con 0x7fe840004850 2023-11-24T07:59:05.128 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:05.120+0000 7fe8567fc700 1 -- 172.21.15.59:0/1145500391 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe848005ba0 con 0x7fe840004850 2023-11-24T07:59:05.128 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:05.121+0000 7fe85de68700 1 -- 172.21.15.59:0/1145500391 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe840004850 msgr2=0x7fe840004cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:05.128 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:05.121+0000 7fe85de68700 1 --2- 172.21.15.59:0/1145500391 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe840004850 0x7fe840004cd0 secure :-1 s=READY pgs=816 cs=0 l=1 rev1=1 crypto rx=0x7fe84800c240 tx=0x7fe840005880 comp rx=0 tx=0).stop 2023-11-24T07:59:05.128 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:05.121+0000 7fe85de68700 1 -- 172.21.15.59:0/1145500391 shutdown_connections 2023-11-24T07:59:05.128 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:05.121+0000 7fe85de68700 1 --2- 172.21.15.59:0/1145500391 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe840004850 0x7fe840004cd0 unknown :-1 s=CLOSED pgs=816 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:05.128 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:05.121+0000 7fe85de68700 1 -- 172.21.15.59:0/1145500391 >> 172.21.15.59:0/1145500391 conn(0x7fe8580b2740 msgr2=0x7fe8580b2b20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:59:05.128 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:05.121+0000 7fe85de68700 1 -- 172.21.15.59:0/1145500391 shutdown_connections 2023-11-24T07:59:05.128 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:05.121+0000 7fe85de68700 1 -- 172.21.15.59:0/1145500391 wait complete. 2023-11-24T07:59:05.128 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:05.122+0000 7fe85de68700 1 Processor -- start 2023-11-24T07:59:05.128 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:05.123+0000 7fe85de68700 1 -- start start 2023-11-24T07:59:05.131 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:05.123+0000 7fe85de68700 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fe840004850 0x7fe8581ef700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:05.131 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:05.124+0000 7fe85de68700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe858142820 0x7fe85814f5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:05.131 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:05.124+0000 7fe85de68700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe858134c50 0x7fe8580cf660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:05.131 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:05.124+0000 7fe85de68700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8581efc40 con 0x7fe858142820 2023-11-24T07:59:05.132 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:05.124+0000 7fe85de68700 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe858150410 con 0x7fe858134c50 2023-11-24T07:59:05.132 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:05.124+0000 7fe85de68700 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe85814fe50 con 0x7fe840004850 2023-11-24T07:59:05.132 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:05.124+0000 7fe856ffd700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe858142820 0x7fe85814f5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:05.132 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:05.124+0000 7fe856ffd700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe858142820 0x7fe85814f5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:54544/0 (socket says 172.21.15.59:54544) 2023-11-24T07:59:05.132 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:05.124+0000 7fe856ffd700 1 -- 172.21.15.59:0/987924779 learned_addr learned my addr 172.21.15.59:0/987924779 (peer_addr_for_me v2:172.21.15.59:0/0) 2023-11-24T07:59:05.132 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:05.124+0000 7fe857fff700 1 --2- 172.21.15.59:0/987924779 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe858134c50 0x7fe8580cf660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:05.132 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:05.124+0000 7fe8577fe700 1 --2- 172.21.15.59:0/987924779 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fe840004850 0x7fe8581ef700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:05.132 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:05.124+0000 7fe856ffd700 1 -- 172.21.15.59:0/987924779 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fe840004850 msgr2=0x7fe8581ef700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:05.132 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:05.124+0000 7fe856ffd700 1 --2- 172.21.15.59:0/987924779 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fe840004850 0x7fe8581ef700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:05.132 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:05.124+0000 7fe856ffd700 1 -- 172.21.15.59:0/987924779 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe858134c50 msgr2=0x7fe8580cf660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:05.132 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:05.124+0000 7fe856ffd700 1 --2- 172.21.15.59:0/987924779 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe858134c50 0x7fe8580cf660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:05.132 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:05.124+0000 7fe856ffd700 1 -- 172.21.15.59:0/987924779 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe858150130 con 0x7fe858142820 2023-11-24T07:59:05.133 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:05.124+0000 7fe857fff700 1 --2- 172.21.15.59:0/987924779 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe858134c50 0x7fe8580cf660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-24T07:59:05.133 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:05.124+0000 7fe856ffd700 1 --2- 172.21.15.59:0/987924779 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe858142820 0x7fe85814f5e0 secure :-1 s=READY pgs=817 cs=0 l=1 rev1=1 crypto rx=0x7fe84800c830 tx=0x7fe848001a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:59:05.133 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:05.124+0000 7fe83ffff700 1 -- 172.21.15.59:0/987924779 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe848004d40 con 0x7fe858142820 2023-11-24T07:59:05.133 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:05.124+0000 7fe83ffff700 1 -- 172.21.15.59:0/987924779 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe8480070d0 con 0x7fe858142820 2023-11-24T07:59:05.133 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:05.124+0000 7fe85de68700 1 -- 172.21.15.59:0/987924779 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe85815cf20 con 0x7fe858142820 2023-11-24T07:59:05.133 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:05.124+0000 7fe83ffff700 1 -- 172.21.15.59:0/987924779 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe848004ea0 con 0x7fe858142820 2023-11-24T07:59:05.133 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:05.124+0000 7fe85de68700 1 -- 172.21.15.59:0/987924779 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe85814fb20 con 0x7fe858142820 2023-11-24T07:59:05.133 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:05.125+0000 7fe85de68700 1 -- 172.21.15.59:0/987924779 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe85815d570 con 0x7fe858142820 2023-11-24T07:59:05.133 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:05.126+0000 7fe83ffff700 1 -- 172.21.15.59:0/987924779 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 10) v1 ==== 89588+0+0 (secure 0 0 0) 0x7fe848007440 con 0x7fe858142820 2023-11-24T07:59:05.133 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:05.126+0000 7fe83ffff700 1 --2- 172.21.15.59:0/987924779 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7fe84406b2c0 0x7fe84406d780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:05.134 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:05.126+0000 7fe83ffff700 1 -- 172.21.15.59:0/987924779 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(65..65 src has 1..65) v4 ==== 4943+0+0 (secure 0 0 0) 0x7fe84808a4e0 con 0x7fe858142820 2023-11-24T07:59:05.134 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:05.126+0000 7fe8577fe700 1 --2- 172.21.15.59:0/987924779 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7fe84406b2c0 0x7fe84406d780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:05.134 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:05.127+0000 7fe8577fe700 1 --2- 172.21.15.59:0/987924779 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7fe84406b2c0 0x7fe84406d780 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7fe84c000c00 tx=0x7fe84c009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:59:05.134 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:05.131+0000 7fe83ffff700 1 -- 172.21.15.59:0/987924779 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fe8480552c0 con 0x7fe858142820 2023-11-24T07:59:05.418 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:05.417+0000 7fe85de68700 1 -- 172.21.15.59:0/987924779 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_volumes.TestSubvolumeSnapshots.test_nonexistent_subvolume_snapshot_rm"]} v 0) v1 -- 0x7fe858068180 con 0x7fe858142820 2023-11-24T07:59:05.602 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:05.600+0000 7fe83ffff700 1 -- 172.21.15.59:0/987924779 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_volumes.TestSubvolumeSnapshots.test_nonexistent_subvolume_snapshot_rm"]}]=0 v169) v1 ==== 166+0+0 (secure 0 0 0) 0x7fe8480588e0 con 0x7fe858142820 2023-11-24T07:59:05.604 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:05.602+0000 7fe85de68700 1 -- 172.21.15.59:0/987924779 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7fe84406b2c0 msgr2=0x7fe84406d780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:05.604 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:05.602+0000 7fe85de68700 1 --2- 172.21.15.59:0/987924779 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7fe84406b2c0 0x7fe84406d780 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7fe84c000c00 tx=0x7fe84c009040 comp rx=0 tx=0).stop 2023-11-24T07:59:05.604 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:05.602+0000 7fe85de68700 1 -- 172.21.15.59:0/987924779 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe858142820 msgr2=0x7fe85814f5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:05.605 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:05.602+0000 7fe85de68700 1 --2- 172.21.15.59:0/987924779 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe858142820 0x7fe85814f5e0 secure :-1 s=READY pgs=817 cs=0 l=1 rev1=1 crypto rx=0x7fe84800c830 tx=0x7fe848001a00 comp rx=0 tx=0).stop 2023-11-24T07:59:05.605 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:05.602+0000 7fe85de68700 1 -- 172.21.15.59:0/987924779 shutdown_connections 2023-11-24T07:59:05.605 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:05.602+0000 7fe85de68700 1 --2- 172.21.15.59:0/987924779 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7fe84406b2c0 0x7fe84406d780 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:05.605 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:05.602+0000 7fe85de68700 1 --2- 172.21.15.59:0/987924779 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe858134c50 0x7fe8580cf660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:05.605 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:05.602+0000 7fe85de68700 1 --2- 172.21.15.59:0/987924779 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fe840004850 0x7fe8581ef700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:05.605 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:05.602+0000 7fe85de68700 1 --2- 172.21.15.59:0/987924779 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe858142820 0x7fe85814f5e0 unknown :-1 s=CLOSED pgs=817 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:05.605 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:05.602+0000 7fe85de68700 1 -- 172.21.15.59:0/987924779 >> 172.21.15.59:0/987924779 conn(0x7fe8580b2740 msgr2=0x7fe8580ce8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:59:05.605 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:05.603+0000 7fe85de68700 1 -- 172.21.15.59:0/987924779 shutdown_connections 2023-11-24T07:59:05.605 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:05.603+0000 7fe85de68700 1 -- 172.21.15.59:0/987924779 wait complete. 2023-11-24T07:59:05.617 INFO:tasks.cephfs_test_runner:test_nonexistent_subvolume_snapshot_rm (tasks.cephfs.test_volumes.TestSubvolumeSnapshots) ... ok 2023-11-24T07:59:05.617 INFO:tasks.cephfs_test_runner:Starting test: test_subvolume_inherited_snapshot_info (tasks.cephfs.test_volumes.TestSubvolumeSnapshots) tests the scenario where 'fs subvolume snapshot info' command 2023-11-24T07:59:05.618 INFO:tasks.cephfs_test_runner:test_subvolume_inherited_snapshot_info (tasks.cephfs.test_volumes.TestSubvolumeSnapshots) 2023-11-24T07:59:05.618 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-24T07:59:05.744 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:05.742+0000 7f85563ae700 1 Processor -- start 2023-11-24T07:59:05.744 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:05.742+0000 7f85563ae700 1 -- start start 2023-11-24T07:59:05.747 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:05.743+0000 7f85563ae700 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f8550143ae0 0x7f85500b51b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:05.747 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:05.743+0000 7f85563ae700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f85500722c0 0x7f855006e510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:05.747 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:05.743+0000 7f85563ae700 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8550143ec0 con 0x7f85501440f0 2023-11-24T07:59:05.747 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:05.743+0000 7f85563ae700 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f85500b40a0 con 0x7f85500722c0 2023-11-24T07:59:05.747 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:05.743+0000 7f85563ae700 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f85500b3ae0 con 0x7f8550143ae0 2023-11-24T07:59:05.747 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:05.743+0000 7f854f7fe700 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7f85501440f0 0x7f8550071bb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:58024/0 (socket says 172.21.15.59:58024) 2023-11-24T07:59:05.747 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:05.743+0000 7f854f7fe700 1 -- 172.21.15.59:0/2418668511 learned_addr learned my addr 172.21.15.59:0/2418668511 (peer_addr_for_me v1:172.21.15.59:0/0) 2023-11-24T07:59:05.747 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:05.743+0000 7f855494b700 1 --2- 172.21.15.59:0/2418668511 >> v2:172.21.15.59:3300/0 conn(0x7f85500722c0 0x7f855006e510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:05.747 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:05.743+0000 7f854ffff700 1 --2- 172.21.15.59:0/2418668511 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f8550143ae0 0x7f85500b51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:05.747 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:05.743+0000 7f854effd700 1 -- 172.21.15.59:0/2418668511 <== mon.0 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2445482887 0 0) 0x7f8550143ec0 con 0x7f85501440f0 2023-11-24T07:59:05.748 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:05.743+0000 7f854effd700 1 -- 172.21.15.59:0/2418668511 --> v1:172.21.15.59:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f85380035b0 con 0x7f85501440f0 2023-11-24T07:59:05.748 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:05.743+0000 7f855494b700 1 -- 172.21.15.59:0/2418668511 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f8550143ae0 msgr2=0x7f85500b51b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T07:59:05.748 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:05.743+0000 7f855494b700 1 --2- 172.21.15.59:0/2418668511 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f8550143ae0 0x7f85500b51b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:05.748 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:05.743+0000 7f855494b700 1 -- 172.21.15.59:0/2418668511 >> v1:172.21.15.59:6789/0 conn(0x7f85501440f0 legacy=0x7f8550071bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:05.748 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:05.743+0000 7f855494b700 1 -- 172.21.15.59:0/2418668511 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f85500b3dc0 con 0x7f85500722c0 2023-11-24T07:59:05.748 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:05.743+0000 7f854ffff700 1 --2- 172.21.15.59:0/2418668511 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f8550143ae0 0x7f85500b51b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-24T07:59:05.748 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:05.743+0000 7f855494b700 1 --2- 172.21.15.59:0/2418668511 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f85500722c0 0x7f855006e510 secure :-1 s=READY pgs=819 cs=0 l=1 rev1=1 crypto rx=0x7f8548009d90 tx=0x7f854800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7fc857f974efe5f0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:59:05.748 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:05.744+0000 7f854effd700 1 -- 172.21.15.59:0/2418668511 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f854800e070 con 0x7f85500722c0 2023-11-24T07:59:05.748 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:05.744+0000 7f854effd700 1 -- 172.21.15.59:0/2418668511 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f854800bb20 con 0x7f85500722c0 2023-11-24T07:59:05.748 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:05.744+0000 7f854effd700 1 -- 172.21.15.59:0/2418668511 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8548012430 con 0x7f85500722c0 2023-11-24T07:59:05.749 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:05.744+0000 7f85563ae700 1 -- 172.21.15.59:0/2418668511 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f85500722c0 msgr2=0x7f855006e510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:05.749 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:05.744+0000 7f85563ae700 1 --2- 172.21.15.59:0/2418668511 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f85500722c0 0x7f855006e510 secure :-1 s=READY pgs=819 cs=0 l=1 rev1=1 crypto rx=0x7f8548009d90 tx=0x7f854800b330 comp rx=0 tx=0).stop 2023-11-24T07:59:05.749 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:05.744+0000 7f85563ae700 1 -- 172.21.15.59:0/2418668511 shutdown_connections 2023-11-24T07:59:05.749 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:05.744+0000 7f85563ae700 1 --2- 172.21.15.59:0/2418668511 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f85500722c0 0x7f855006e510 unknown :-1 s=CLOSED pgs=819 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:05.749 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:05.744+0000 7f85563ae700 1 --2- 172.21.15.59:0/2418668511 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f8550143ae0 0x7f85500b51b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:05.749 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:05.744+0000 7f85563ae700 1 -- 172.21.15.59:0/2418668511 >> 172.21.15.59:0/2418668511 conn(0x7f85500b2740 msgr2=0x7f85500b2b20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:59:05.749 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:05.744+0000 7f85563ae700 1 -- 172.21.15.59:0/2418668511 shutdown_connections 2023-11-24T07:59:05.749 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:05.744+0000 7f85563ae700 1 -- 172.21.15.59:0/2418668511 wait complete. 2023-11-24T07:59:05.752 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:05.746+0000 7f85563ae700 1 Processor -- start 2023-11-24T07:59:05.752 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:05.746+0000 7f85563ae700 1 -- start start 2023-11-24T07:59:05.752 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:05.747+0000 7f85563ae700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f85500722c0 0x7f85500b1f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:05.752 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:05.747+0000 7f85563ae700 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f8550143ae0 0x7f855015c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:05.752 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:05.747+0000 7f85563ae700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f85501440f0 0x7f855015ca50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:05.752 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:05.747+0000 7f85563ae700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8550137240 con 0x7f85500722c0 2023-11-24T07:59:05.753 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:05.747+0000 7f85563ae700 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8550135090 con 0x7f85501440f0 2023-11-24T07:59:05.753 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:05.747+0000 7f85563ae700 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f85501368a0 con 0x7f8550143ae0 2023-11-24T07:59:05.753 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:05.747+0000 7f854ffff700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f85500722c0 0x7f85500b1f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:05.753 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:05.747+0000 7f854ffff700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f85500722c0 0x7f85500b1f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:54554/0 (socket says 172.21.15.59:54554) 2023-11-24T07:59:05.753 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:05.747+0000 7f854ffff700 1 -- 172.21.15.59:0/197362683 learned_addr learned my addr 172.21.15.59:0/197362683 (peer_addr_for_me v2:172.21.15.59:0/0) 2023-11-24T07:59:05.753 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:05.747+0000 7f855494b700 1 --2- 172.21.15.59:0/197362683 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f85501440f0 0x7f855015ca50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:05.753 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:05.747+0000 7f854f7fe700 1 --2- 172.21.15.59:0/197362683 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f8550143ae0 0x7f855015c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:05.753 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:05.747+0000 7f854ffff700 1 -- 172.21.15.59:0/197362683 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f8550143ae0 msgr2=0x7f855015c510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:05.753 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:05.747+0000 7f854ffff700 1 --2- 172.21.15.59:0/197362683 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f8550143ae0 0x7f855015c510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:05.753 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:05.747+0000 7f854ffff700 1 -- 172.21.15.59:0/197362683 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f85501440f0 msgr2=0x7f855015ca50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:05.753 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:05.747+0000 7f854ffff700 1 --2- 172.21.15.59:0/197362683 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f85501440f0 0x7f855015ca50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:05.754 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:05.747+0000 7f854ffff700 1 -- 172.21.15.59:0/197362683 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8548009a70 con 0x7f85500722c0 2023-11-24T07:59:05.754 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:05.747+0000 7f854ffff700 1 --2- 172.21.15.59:0/197362683 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f85500722c0 0x7f85500b1f00 secure :-1 s=READY pgs=820 cs=0 l=1 rev1=1 crypto rx=0x7f854400e810 tx=0x7f854400ebd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:59:05.754 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:05.748+0000 7f854cff9700 1 -- 172.21.15.59:0/197362683 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8544013610 con 0x7f85500722c0 2023-11-24T07:59:05.754 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:05.748+0000 7f854cff9700 1 -- 172.21.15.59:0/197362683 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8544013770 con 0x7f85500722c0 2023-11-24T07:59:05.754 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:05.748+0000 7f854cff9700 1 -- 172.21.15.59:0/197362683 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8544013b20 con 0x7f85500722c0 2023-11-24T07:59:05.754 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:05.748+0000 7f85563ae700 1 -- 172.21.15.59:0/197362683 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8550136b80 con 0x7f85500722c0 2023-11-24T07:59:05.754 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:05.748+0000 7f85563ae700 1 -- 172.21.15.59:0/197362683 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f85500b2440 con 0x7f85500722c0 2023-11-24T07:59:05.754 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:05.749+0000 7f85563ae700 1 -- 172.21.15.59:0/197362683 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8550068670 con 0x7f85500722c0 2023-11-24T07:59:05.754 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:05.749+0000 7f854cff9700 1 -- 172.21.15.59:0/197362683 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 10) v1 ==== 89588+0+0 (secure 0 0 0) 0x7f8544013c80 con 0x7f85500722c0 2023-11-24T07:59:05.754 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:05.749+0000 7f854cff9700 1 --2- 172.21.15.59:0/197362683 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f853c06b2c0 0x7f853c06d780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:05.754 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:05.750+0000 7f854cff9700 1 -- 172.21.15.59:0/197362683 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(65..65 src has 1..65) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f8544089830 con 0x7f85500722c0 2023-11-24T07:59:05.754 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:05.750+0000 7f854f7fe700 1 --2- 172.21.15.59:0/197362683 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f853c06b2c0 0x7f853c06d780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:05.755 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:05.750+0000 7f854f7fe700 1 --2- 172.21.15.59:0/197362683 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f853c06b2c0 0x7f853c06d780 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f8550142d50 tx=0x7f8540009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:59:05.755 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:05.754+0000 7f854cff9700 1 -- 172.21.15.59:0/197362683 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f85440544f0 con 0x7f85500722c0 2023-11-24T07:59:05.960 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:05.958+0000 7f85563ae700 1 -- 172.21.15.59:0/197362683 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f855015def0 con 0x7f85500722c0 2023-11-24T07:59:05.960 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:05.959+0000 7f854cff9700 1 -- 172.21.15.59:0/197362683 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v65) v1 ==== 74+0+12914 (secure 0 0 0) 0x7f8544057b10 con 0x7f85500722c0 2023-11-24T07:59:05.961 INFO:teuthology.orchestra.run.smithi059.stdout: 2023-11-24T07:59:05.961 INFO:teuthology.orchestra.run.smithi059.stdout:{"epoch":65,"fsid":"9f0e8741-b8f9-48d5-a2c3-09f45903303d","created":"2023-11-24T07:56:06.552729+0000","modified":"2023-11-24T07:59:04.594581+0000","last_up_change":"2023-11-24T07:56:17.372352+0000","last_in_change":"2023-11-24T07:56:09.073250+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-24T07:56:18.352823+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"6d8f9e9b-1782-4e41-928b-a39f2a8ebc06","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6800","nonce":3373878772},{"type":"v1","addr":"172.21.15.59:6801","nonce":3373878772}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6802","nonce":3373878772},{"type":"v1","addr":"172.21.15.59:6803","nonce":3373878772}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6806","nonce":3373878772},{"type":"v1","addr":"172.21.15.59:6807","nonce":3373878772}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6804","nonce":3373878772},{"type":"v1","addr":"172.21.15.59:6805","nonce":3373878772}]},"public_addr":"172.21.15.59:6801/3373878772","cluster_addr":"172.21.15.59:6803/3373878772","heartbeat_back_addr":"172.21.15.59:6807/3373878772","heartbeat_front_addr":"172.21.15.59:6805/3373878772","state":["exists","up"]},{"osd":1,"uuid":"e8d882aa-52e1-4c04-9e1e-c44ef9204a5f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6824","nonce":648258563},{"type":"v1","addr":"172.21.15.59:6825","nonce":648258563}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6826","nonce":648258563},{"type":"v1","addr":"172.21.15.59:6827","nonce":648258563}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6830","nonce":648258563},{"type":"v1","addr":"172.21.15.59:6831","nonce":648258563}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6828","nonce":648258563},{"type":"v1","addr":"172.21.15.59:6829","nonce":648258563}]},"public_addr":"172.21.15.59:6825/648258563","cluster_addr":"172.21.15.59:6827/648258563","heartbeat_back_addr":"172.21.15.59:6831/648258563","heartbeat_front_addr":"172.21.15.59:6829/648258563","state":["exists","up"]},{"osd":2,"uuid":"69eebfa0-695b-4b21-90b9-8a3537168a01","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6808","nonce":1840430654},{"type":"v1","addr":"172.21.15.59:6809","nonce":1840430654}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6810","nonce":1840430654},{"type":"v1","addr":"172.21.15.59:6811","nonce":1840430654}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6814","nonce":1840430654},{"type":"v1","addr":"172.21.15.59:6815","nonce":1840430654}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6812","nonce":1840430654},{"type":"v1","addr":"172.21.15.59:6813","nonce":1840430654}]},"public_addr":"172.21.15.59:6809/1840430654","cluster_addr":"172.21.15.59:6811/1840430654","heartbeat_back_addr":"172.21.15.59:6815/1840430654","heartbeat_front_addr":"172.21.15.59:6813/1840430654","state":["exists","up"]},{"osd":3,"uuid":"96fd77fc-f911-46ff-9445-6f0be2daa8f4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6816","nonce":2504025850},{"type":"v1","addr":"172.21.15.59:6817","nonce":2504025850}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6818","nonce":2504025850},{"type":"v1","addr":"172.21.15.59:6819","nonce":2504025850}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6822","nonce":2504025850},{"type":"v1","addr":"172.21.15.59:6823","nonce":2504025850}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6820","nonce":2504025850},{"type":"v1","addr":"172.21.15.59:6821","nonce":2504025850}]},"public_addr":"172.21.15.59:6817/2504025850","cluster_addr":"172.21.15.59:6819/2504025850","heartbeat_back_addr":"172.21.15.59:6823/2504025850","heartbeat_front_addr":"172.21.15.59:6821/2504025850","state":["exists","up"]},{"osd":4,"uuid":"7ac29fca-b5b2-4c95-9d69-2f164cbbc708","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6816","nonce":2382463431},{"type":"v1","addr":"172.21.15.76:6818","nonce":2382463431}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6820","nonce":2382463431},{"type":"v1","addr":"172.21.15.76:6822","nonce":2382463431}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6828","nonce":2382463431},{"type":"v1","addr":"172.21.15.76:6830","nonce":2382463431}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6824","nonce":2382463431},{"type":"v1","addr":"172.21.15.76:6826","nonce":2382463431}]},"public_addr":"172.21.15.76:6818/2382463431","cluster_addr":"172.21.15.76:6822/2382463431","heartbeat_back_addr":"172.21.15.76:6830/2382463431","heartbeat_front_addr":"172.21.15.76:6826/2382463431","state":["exists","up"]},{"osd":5,"uuid":"485bf711-3851-4238-a4f4-51d52e0a9c58","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6800","nonce":329334308},{"type":"v1","addr":"172.21.15.76:6802","nonce":329334308}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6805","nonce":329334308},{"type":"v1","addr":"172.21.15.76:6807","nonce":329334308}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6814","nonce":329334308},{"type":"v1","addr":"172.21.15.76:6815","nonce":329334308}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6810","nonce":329334308},{"type":"v1","addr":"172.21.15.76:6812","nonce":329334308}]},"public_addr":"172.21.15.76:6802/329334308","cluster_addr":"172.21.15.76:6807/329334308","heartbeat_back_addr":"172.21.15.76:6815/329334308","heartbeat_front_addr":"172.21.15.76:6812/329334308","state":["exists","up"]},{"osd":6,"uuid":"72b3f2fc-1e4a-4e32-968d-003647a0286f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6801","nonce":3946579410},{"type":"v1","addr":"172.21.15.76:6803","nonce":3946579410}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6804","nonce":3946579410},{"type":"v1","addr":"172.21.15.76:6806","nonce":3946579410}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6811","nonce":3946579410},{"type":"v1","addr":"172.21.15.76:6813","nonce":3946579410}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6808","nonce":3946579410},{"type":"v1","addr":"172.21.15.76:6809","nonce":3946579410}]},"public_addr":"172.21.15.76:6803/3946579410","cluster_addr":"172.21.15.76:6806/3946579410","heartbeat_back_addr":"172.21.15.76:6813/3946579410","heartbeat_front_addr":"172.21.15.76:6809/3946579410","state":["exists","up"]},{"osd":7,"uuid":"1ddceb90-6c5d-4d1a-ad56-99cb877f8c70","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6817","nonce":841761538},{"type":"v1","addr":"172.21.15.76:6819","nonce":841761538}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6821","nonce":841761538},{"type":"v1","addr":"172.21.15.76:6823","nonce":841761538}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6829","nonce":841761538},{"type":"v1","addr":"172.21.15.76:6831","nonce":841761538}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6825","nonce":841761538},{"type":"v1","addr":"172.21.15.76:6827","nonce":841761538}]},"public_addr":"172.21.15.76:6819/841761538","cluster_addr":"172.21.15.76:6823/841761538","heartbeat_back_addr":"172.21.15.76:6831/841761538","heartbeat_front_addr":"172.21.15.76:6827/841761538","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T07:56:14.294838+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T07:56:14.270779+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T07:56:14.231809+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T07:56:14.260983+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T07:56:14.302997+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T07:56:14.258600+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T07:56:14.251413+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T07:56:14.304727+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.59:6835/1599845006":"2023-11-25T07:59:01.582827+0000","172.21.15.59:6832/1599845006":"2023-11-25T07:59:01.582827+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-24T07:59:05.963 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:05.961+0000 7f85563ae700 1 -- 172.21.15.59:0/197362683 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f853c06b2c0 msgr2=0x7f853c06d780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:05.963 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:05.961+0000 7f85563ae700 1 --2- 172.21.15.59:0/197362683 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f853c06b2c0 0x7f853c06d780 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f8550142d50 tx=0x7f8540009040 comp rx=0 tx=0).stop 2023-11-24T07:59:05.963 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:05.961+0000 7f85563ae700 1 -- 172.21.15.59:0/197362683 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f85500722c0 msgr2=0x7f85500b1f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:05.963 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:05.961+0000 7f85563ae700 1 --2- 172.21.15.59:0/197362683 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f85500722c0 0x7f85500b1f00 secure :-1 s=READY pgs=820 cs=0 l=1 rev1=1 crypto rx=0x7f854400e810 tx=0x7f854400ebd0 comp rx=0 tx=0).stop 2023-11-24T07:59:05.963 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:05.961+0000 7f85563ae700 1 -- 172.21.15.59:0/197362683 shutdown_connections 2023-11-24T07:59:05.964 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:05.961+0000 7f85563ae700 1 --2- 172.21.15.59:0/197362683 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f853c06b2c0 0x7f853c06d780 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:05.964 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:05.961+0000 7f85563ae700 1 --2- 172.21.15.59:0/197362683 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f85501440f0 0x7f855015ca50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:05.964 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:05.961+0000 7f85563ae700 1 --2- 172.21.15.59:0/197362683 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f85500722c0 0x7f85500b1f00 unknown :-1 s=CLOSED pgs=820 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:05.964 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:05.961+0000 7f85563ae700 1 --2- 172.21.15.59:0/197362683 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f8550143ae0 0x7f855015c510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:05.964 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:05.961+0000 7f85563ae700 1 -- 172.21.15.59:0/197362683 >> 172.21.15.59:0/197362683 conn(0x7f85500b2740 msgr2=0x7f85500b41e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:59:05.964 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:05.961+0000 7f85563ae700 1 -- 172.21.15.59:0/197362683 shutdown_connections 2023-11-24T07:59:05.964 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:05.961+0000 7f85563ae700 1 -- 172.21.15.59:0/197362683 wait complete. 2023-11-24T07:59:05.975 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-24T07:56:18.352823+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-24T07:59:05.976 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-24T07:59:06.104 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:06.102+0000 7fac25e58700 1 Processor -- start 2023-11-24T07:59:06.104 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:06.102+0000 7fac25e58700 1 -- start start 2023-11-24T07:59:06.108 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:06.103+0000 7fac25e58700 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fac2015e190 0x7fac200d0f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:06.109 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:06.103+0000 7fac25e58700 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fac200b7810 con 0x7fac200cfde0 2023-11-24T07:59:06.109 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:06.103+0000 7fac25e58700 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fac200b6ee0 con 0x7fac200d1490 2023-11-24T07:59:06.109 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:06.103+0000 7fac25e58700 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fac200d1930 con 0x7fac2015e190 2023-11-24T07:59:06.109 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:06.103+0000 7fac1ffff700 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7fac200cfde0 0x7fac200b4ab0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:58040/0 (socket says 172.21.15.59:58040) 2023-11-24T07:59:06.110 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:06.103+0000 7fac1ffff700 1 -- 172.21.15.59:0/3334849983 learned_addr learned my addr 172.21.15.59:0/3334849983 (peer_addr_for_me v1:172.21.15.59:0/0) 2023-11-24T07:59:06.110 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:06.103+0000 7fac1f7fe700 1 --2- 172.21.15.59:0/3334849983 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fac2015e190 0x7fac200d0f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:06.110 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:06.103+0000 7fac1e7fc700 1 -- 172.21.15.59:0/3334849983 <== mon.0 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 240457810 0 0) 0x7fac200b7810 con 0x7fac200cfde0 2023-11-24T07:59:06.111 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:06.103+0000 7fac1e7fc700 1 -- 172.21.15.59:0/3334849983 --> v1:172.21.15.59:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fac080035b0 con 0x7fac200cfde0 2023-11-24T07:59:06.111 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:06.104+0000 7fac1e7fc700 1 -- 172.21.15.59:0/3334849983 <== mon.1 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3777097059 0 0) 0x7fac200b6ee0 con 0x7fac200d1490 2023-11-24T07:59:06.111 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:06.104+0000 7fac1e7fc700 1 -- 172.21.15.59:0/3334849983 --> v1:172.21.15.76:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fac200b7810 con 0x7fac200d1490 2023-11-24T07:59:06.111 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:06.104+0000 7fac1e7fc700 1 -- 172.21.15.59:0/3334849983 <== mon.0 v1:172.21.15.59:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3524560057 0 0) 0x7fac080035b0 con 0x7fac200cfde0 2023-11-24T07:59:06.111 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:06.104+0000 7fac1e7fc700 1 -- 172.21.15.59:0/3334849983 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fac2015e190 msgr2=0x7fac200d0f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T07:59:06.111 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:06.104+0000 7fac1e7fc700 1 --2- 172.21.15.59:0/3334849983 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fac2015e190 0x7fac200d0f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:06.111 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:06.104+0000 7fac1e7fc700 1 -- 172.21.15.59:0/3334849983 >> v1:172.21.15.76:6789/0 conn(0x7fac200d1490 legacy=0x7fac200d24d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:06.112 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:06.104+0000 7fac1e7fc700 1 -- 172.21.15.59:0/3334849983 --> v1:172.21.15.59:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fac200b71c0 con 0x7fac200cfde0 2023-11-24T07:59:06.112 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:06.104+0000 7fac1e7fc700 1 -- 172.21.15.59:0/3334849983 <== mon.0 v1:172.21.15.59:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1312333566 0 0) 0x7fac180019f0 con 0x7fac200cfde0 2023-11-24T07:59:06.112 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:06.104+0000 7fac1e7fc700 1 -- 172.21.15.59:0/3334849983 >> v1:172.21.15.59:6789/0 conn(0x7fac200cfde0 legacy=0x7fac200b4ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:06.112 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:06.104+0000 7fac1e7fc700 1 --2- 172.21.15.59:0/3334849983 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fac08003bc0 0x7fac08003fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:06.112 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:06.104+0000 7fac1e7fc700 1 --2- 172.21.15.59:0/3334849983 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fac2015e190 0x7fac08004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:06.112 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:06.104+0000 7fac1e7fc700 1 --2- 172.21.15.59:0/3334849983 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fac08004d90 0x7fac08005210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:06.112 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:06.104+0000 7fac1e7fc700 1 -- 172.21.15.59:0/3334849983 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac200d1930 con 0x7fac08003bc0 2023-11-24T07:59:06.112 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:06.104+0000 7fac1e7fc700 1 -- 172.21.15.59:0/3334849983 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac0800bac0 con 0x7fac08004d90 2023-11-24T07:59:06.112 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:06.104+0000 7fac1e7fc700 1 -- 172.21.15.59:0/3334849983 --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fac0800bc30 con 0x7fac2015e190 2023-11-24T07:59:06.113 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:06.104+0000 7fac1f7fe700 1 --2- 172.21.15.59:0/3334849983 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fac08003bc0 0x7fac08003fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:06.113 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:06.104+0000 7fac1effd700 1 --2- 172.21.15.59:0/3334849983 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fac2015e190 0x7fac08004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:06.113 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:06.104+0000 7fac1ffff700 1 --2- 172.21.15.59:0/3334849983 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fac08004d90 0x7fac08005210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:06.113 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:06.104+0000 7fac1f7fe700 1 -- 172.21.15.59:0/3334849983 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fac2015e190 msgr2=0x7fac08004850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T07:59:06.113 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:06.104+0000 7fac1f7fe700 1 --2- 172.21.15.59:0/3334849983 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fac2015e190 0x7fac08004850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:06.113 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:06.104+0000 7fac1f7fe700 1 -- 172.21.15.59:0/3334849983 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fac08004d90 msgr2=0x7fac08005210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T07:59:06.113 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:06.104+0000 7fac1f7fe700 1 --2- 172.21.15.59:0/3334849983 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fac08004d90 0x7fac08005210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:06.113 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:06.104+0000 7fac1f7fe700 1 -- 172.21.15.59:0/3334849983 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fac080035b0 con 0x7fac08003bc0 2023-11-24T07:59:06.113 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:06.104+0000 7fac1ffff700 1 --2- 172.21.15.59:0/3334849983 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fac08004d90 0x7fac08005210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-24T07:59:06.114 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:06.104+0000 7fac1f7fe700 1 --2- 172.21.15.59:0/3334849983 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fac08003bc0 0x7fac08003fc0 secure :-1 s=READY pgs=822 cs=0 l=1 rev1=1 crypto rx=0x7fac10009b60 tx=0x7fac08005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4882d33be7ea1452 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:59:06.114 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:06.104+0000 7fac1e7fc700 1 -- 172.21.15.59:0/3334849983 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac1000e6c0 con 0x7fac08003bc0 2023-11-24T07:59:06.114 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:06.105+0000 7fac1e7fc700 1 -- 172.21.15.59:0/3334849983 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fac1000e820 con 0x7fac08003bc0 2023-11-24T07:59:06.114 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:06.105+0000 7fac1e7fc700 1 -- 172.21.15.59:0/3334849983 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac1000eb90 con 0x7fac08003bc0 2023-11-24T07:59:06.114 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:06.105+0000 7fac25e58700 1 -- 172.21.15.59:0/3334849983 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fac08003bc0 msgr2=0x7fac08003fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:06.114 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:06.105+0000 7fac25e58700 1 --2- 172.21.15.59:0/3334849983 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fac08003bc0 0x7fac08003fc0 secure :-1 s=READY pgs=822 cs=0 l=1 rev1=1 crypto rx=0x7fac10009b60 tx=0x7fac08005880 comp rx=0 tx=0).stop 2023-11-24T07:59:06.114 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:06.105+0000 7fac1ffff700 1 -- 172.21.15.59:0/3334849983 reap_dead start 2023-11-24T07:59:06.114 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:06.105+0000 7fac25e58700 1 -- 172.21.15.59:0/3334849983 shutdown_connections 2023-11-24T07:59:06.115 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:06.105+0000 7fac25e58700 1 -- 172.21.15.59:0/3334849983 >> 172.21.15.59:0/3334849983 conn(0x7fac2009d180 msgr2=0x7fac200d3ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:59:06.115 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:06.105+0000 7fac25e58700 1 -- 172.21.15.59:0/3334849983 shutdown_connections 2023-11-24T07:59:06.115 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:06.105+0000 7fac25e58700 1 -- 172.21.15.59:0/3334849983 wait complete. 2023-11-24T07:59:06.115 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:06.107+0000 7fac25e58700 1 Processor -- start 2023-11-24T07:59:06.115 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:06.107+0000 7fac25e58700 1 -- start start 2023-11-24T07:59:06.119 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:06.108+0000 7fac25e58700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fac08003bc0 0x7fac2014b380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:06.119 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:06.108+0000 7fac25e58700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fac201f3210 0x7fac201608c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:06.119 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:06.108+0000 7fac25e58700 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fac20160e00 0x7fac200c7680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:06.119 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:06.108+0000 7fac25e58700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac2014b8c0 con 0x7fac08003bc0 2023-11-24T07:59:06.119 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:06.108+0000 7fac25e58700 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac20160150 con 0x7fac201f3210 2023-11-24T07:59:06.119 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:06.108+0000 7fac25e58700 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fac2015aab0 con 0x7fac20160e00 2023-11-24T07:59:06.119 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:06.108+0000 7fac1f7fe700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fac08003bc0 0x7fac2014b380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:06.119 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:06.108+0000 7fac1f7fe700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fac08003bc0 0x7fac2014b380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:54574/0 (socket says 172.21.15.59:54574) 2023-11-24T07:59:06.120 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:06.108+0000 7fac1f7fe700 1 -- 172.21.15.59:0/2788167292 learned_addr learned my addr 172.21.15.59:0/2788167292 (peer_addr_for_me v2:172.21.15.59:0/0) 2023-11-24T07:59:06.120 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:06.108+0000 7fac1effd700 1 --2- 172.21.15.59:0/2788167292 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fac201f3210 0x7fac201608c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:06.120 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:06.108+0000 7fac1ffff700 1 --2- 172.21.15.59:0/2788167292 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fac20160e00 0x7fac200c7680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:06.120 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:06.108+0000 7fac1f7fe700 1 -- 172.21.15.59:0/2788167292 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fac20160e00 msgr2=0x7fac200c7680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:06.120 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:06.108+0000 7fac1f7fe700 1 --2- 172.21.15.59:0/2788167292 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fac20160e00 0x7fac200c7680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:06.120 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:06.108+0000 7fac1f7fe700 1 -- 172.21.15.59:0/2788167292 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fac201f3210 msgr2=0x7fac201608c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:06.120 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:06.108+0000 7fac1f7fe700 1 --2- 172.21.15.59:0/2788167292 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fac201f3210 0x7fac201608c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:06.121 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:06.108+0000 7fac1f7fe700 1 -- 172.21.15.59:0/2788167292 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fac2015ad90 con 0x7fac08003bc0 2023-11-24T07:59:06.121 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:06.108+0000 7fac1ffff700 1 --2- 172.21.15.59:0/2788167292 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fac20160e00 0x7fac200c7680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-24T07:59:06.121 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:06.109+0000 7fac1f7fe700 1 --2- 172.21.15.59:0/2788167292 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fac08003bc0 0x7fac2014b380 secure :-1 s=READY pgs=823 cs=0 l=1 rev1=1 crypto rx=0x7fac08005830 tx=0x7fac100047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:59:06.121 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:06.109+0000 7fac07fff700 1 -- 172.21.15.59:0/2788167292 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac1000e6c0 con 0x7fac08003bc0 2023-11-24T07:59:06.121 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:06.109+0000 7fac07fff700 1 -- 172.21.15.59:0/2788167292 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fac10003ad0 con 0x7fac08003bc0 2023-11-24T07:59:06.121 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:06.109+0000 7fac07fff700 1 -- 172.21.15.59:0/2788167292 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac10003e80 con 0x7fac08003bc0 2023-11-24T07:59:06.121 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:06.109+0000 7fac25e58700 1 -- 172.21.15.59:0/2788167292 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fac2015b070 con 0x7fac08003bc0 2023-11-24T07:59:06.121 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:06.109+0000 7fac25e58700 1 -- 172.21.15.59:0/2788167292 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fac201f3690 con 0x7fac08003bc0 2023-11-24T07:59:06.121 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:06.110+0000 7fac25e58700 1 -- 172.21.15.59:0/2788167292 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fac20075480 con 0x7fac08003bc0 2023-11-24T07:59:06.122 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:06.110+0000 7fac07fff700 1 -- 172.21.15.59:0/2788167292 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 10) v1 ==== 89588+0+0 (secure 0 0 0) 0x7fac10013470 con 0x7fac08003bc0 2023-11-24T07:59:06.122 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:06.110+0000 7fac07fff700 1 --2- 172.21.15.59:0/2788167292 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7fac0c06b2c0 0x7fac0c06d780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:06.122 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:06.111+0000 7fac07fff700 1 -- 172.21.15.59:0/2788167292 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(65..65 src has 1..65) v4 ==== 4943+0+0 (secure 0 0 0) 0x7fac1008b1a0 con 0x7fac08003bc0 2023-11-24T07:59:06.122 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:06.111+0000 7fac1effd700 1 --2- 172.21.15.59:0/2788167292 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7fac0c06b2c0 0x7fac0c06d780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:06.122 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:06.111+0000 7fac1effd700 1 --2- 172.21.15.59:0/2788167292 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7fac0c06b2c0 0x7fac0c06d780 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7fac201f6e90 tx=0x7fac14009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:59:06.122 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:06.115+0000 7fac07fff700 1 -- 172.21.15.59:0/2788167292 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fac10056120 con 0x7fac08003bc0 2023-11-24T07:59:06.319 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:06.317+0000 7fac25e58700 1 -- 172.21.15.59:0/2788167292 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fac200d27b0 con 0x7fac08003bc0 2023-11-24T07:59:06.319 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:06.317+0000 7fac07fff700 1 -- 172.21.15.59:0/2788167292 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v65) v1 ==== 93+0+10 (secure 0 0 0) 0x7fac10059740 con 0x7fac08003bc0 2023-11-24T07:59:06.320 INFO:teuthology.orchestra.run.smithi059.stdout:pg_num: 1 2023-11-24T07:59:06.321 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:06.320+0000 7fac25e58700 1 -- 172.21.15.59:0/2788167292 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7fac0c06b2c0 msgr2=0x7fac0c06d780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:06.322 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:06.320+0000 7fac25e58700 1 --2- 172.21.15.59:0/2788167292 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7fac0c06b2c0 0x7fac0c06d780 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7fac201f6e90 tx=0x7fac14009040 comp rx=0 tx=0).stop 2023-11-24T07:59:06.322 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:06.320+0000 7fac25e58700 1 -- 172.21.15.59:0/2788167292 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fac08003bc0 msgr2=0x7fac2014b380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:06.322 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:06.320+0000 7fac25e58700 1 --2- 172.21.15.59:0/2788167292 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fac08003bc0 0x7fac2014b380 secure :-1 s=READY pgs=823 cs=0 l=1 rev1=1 crypto rx=0x7fac08005830 tx=0x7fac100047c0 comp rx=0 tx=0).stop 2023-11-24T07:59:06.322 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:06.320+0000 7fac25e58700 1 -- 172.21.15.59:0/2788167292 shutdown_connections 2023-11-24T07:59:06.322 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:06.320+0000 7fac25e58700 1 --2- 172.21.15.59:0/2788167292 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7fac0c06b2c0 0x7fac0c06d780 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:06.323 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:06.320+0000 7fac25e58700 1 --2- 172.21.15.59:0/2788167292 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fac20160e00 0x7fac200c7680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:06.323 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:06.320+0000 7fac25e58700 1 --2- 172.21.15.59:0/2788167292 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fac08003bc0 0x7fac2014b380 unknown :-1 s=CLOSED pgs=823 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:06.323 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:06.320+0000 7fac25e58700 1 --2- 172.21.15.59:0/2788167292 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fac201f3210 0x7fac201608c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:06.323 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:06.320+0000 7fac25e58700 1 -- 172.21.15.59:0/2788167292 >> 172.21.15.59:0/2788167292 conn(0x7fac2009d180 msgr2=0x7fac200b62a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:59:06.323 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:06.320+0000 7fac25e58700 1 -- 172.21.15.59:0/2788167292 shutdown_connections 2023-11-24T07:59:06.323 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:06.320+0000 7fac25e58700 1 -- 172.21.15.59:0/2788167292 wait complete. 2023-11-24T07:59:06.334 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_volumes.TestSubvolumeSnapshots.test_subvolume_inherited_snapshot_info' 2023-11-24T07:59:06.460 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:06.458+0000 7f29188dd700 1 Processor -- start 2023-11-24T07:59:06.461 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:06.459+0000 7f29188dd700 1 -- start start 2023-11-24T07:59:06.463 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:06.459+0000 7f29188dd700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f291415aa10 0x7f2914072710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:06.463 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:06.459+0000 7f29188dd700 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f29140af770 con 0x7f29140b0db0 2023-11-24T07:59:06.463 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:06.459+0000 7f29188dd700 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f29140afa50 con 0x7f291415a630 2023-11-24T07:59:06.463 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:06.459+0000 7f29188dd700 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f29140ad540 con 0x7f291415aa10 2023-11-24T07:59:06.463 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:06.460+0000 7f2911d9b700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f291415aa10 0x7f2914072710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:06.463 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:06.460+0000 7f2912d9d700 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7f29140b0db0 0x7f29140ad960 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:58058/0 (socket says 172.21.15.59:58058) 2023-11-24T07:59:06.463 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:06.460+0000 7f2912d9d700 1 -- 172.21.15.59:0/2151307984 learned_addr learned my addr 172.21.15.59:0/2151307984 (peer_addr_for_me v1:172.21.15.59:0/0) 2023-11-24T07:59:06.464 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:06.460+0000 7f2911d9b700 1 -- 172.21.15.59:0/2151307984 >> v1:172.21.15.76:6789/0 conn(0x7f291415a630 legacy=0x7f29140b06a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T07:59:06.464 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:06.460+0000 7f2911d9b700 1 -- 172.21.15.59:0/2151307984 >> v1:172.21.15.59:6789/0 conn(0x7f29140b0db0 legacy=0x7f29140ad960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:06.464 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:06.460+0000 7f2911d9b700 1 -- 172.21.15.59:0/2151307984 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f29140ad160 con 0x7f291415aa10 2023-11-24T07:59:06.464 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:06.460+0000 7f291159a700 1 -- 172.21.15.59:0/2151307984 <== mon.0 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2902704824 0 0) 0x7f29140af770 con 0x7f29140b0db0 2023-11-24T07:59:06.464 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:06.460+0000 7f2911d9b700 1 --2- 172.21.15.59:0/2151307984 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f291415aa10 0x7f2914072710 secure :-1 s=READY pgs=825 cs=0 l=1 rev1=1 crypto rx=0x7f2904009840 tx=0x7f2904009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a0125d612b1a526c server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:59:06.464 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:06.460+0000 7f291159a700 1 -- 172.21.15.59:0/2151307984 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2904004230 con 0x7f291415aa10 2023-11-24T07:59:06.464 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:06.460+0000 7f291159a700 1 -- 172.21.15.59:0/2151307984 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2904004390 con 0x7f291415aa10 2023-11-24T07:59:06.464 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:06.460+0000 7f291159a700 1 -- 172.21.15.59:0/2151307984 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29040036a0 con 0x7f291415aa10 2023-11-24T07:59:06.464 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:06.461+0000 7f29188dd700 1 -- 172.21.15.59:0/2151307984 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f291415aa10 msgr2=0x7f2914072710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:06.464 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:06.461+0000 7f29188dd700 1 --2- 172.21.15.59:0/2151307984 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f291415aa10 0x7f2914072710 secure :-1 s=READY pgs=825 cs=0 l=1 rev1=1 crypto rx=0x7f2904009840 tx=0x7f2904009b50 comp rx=0 tx=0).stop 2023-11-24T07:59:06.464 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:06.461+0000 7f29188dd700 1 -- 172.21.15.59:0/2151307984 shutdown_connections 2023-11-24T07:59:06.464 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:06.461+0000 7f29188dd700 1 --2- 172.21.15.59:0/2151307984 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f291415aa10 0x7f2914072710 unknown :-1 s=CLOSED pgs=825 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:06.465 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:06.461+0000 7f29188dd700 1 -- 172.21.15.59:0/2151307984 >> 172.21.15.59:0/2151307984 conn(0x7f29140b3190 msgr2=0x7f29140b35a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:59:06.465 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:06.461+0000 7f29188dd700 1 -- 172.21.15.59:0/2151307984 shutdown_connections 2023-11-24T07:59:06.465 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:06.461+0000 7f29188dd700 1 -- 172.21.15.59:0/2151307984 wait complete. 2023-11-24T07:59:06.467 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:06.462+0000 7f29188dd700 1 Processor -- start 2023-11-24T07:59:06.467 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:06.462+0000 7f29188dd700 1 -- start start 2023-11-24T07:59:06.467 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:06.463+0000 7f29188dd700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f29140b0db0 0x7f29141f4ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:06.467 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:06.463+0000 7f29188dd700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f291415a630 0x7f291415bd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:06.468 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:06.463+0000 7f29188dd700 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f291415aa10 0x7f29140cc880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:06.468 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:06.463+0000 7f29188dd700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29140ce930 con 0x7f29140b0db0 2023-11-24T07:59:06.468 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:06.463+0000 7f29188dd700 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f291415c290 con 0x7f291415a630 2023-11-24T07:59:06.468 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:06.463+0000 7f29188dd700 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f291415cc80 con 0x7f291415aa10 2023-11-24T07:59:06.468 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:06.463+0000 7f291259c700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f29140b0db0 0x7f29141f4ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:06.468 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:06.463+0000 7f291259c700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f29140b0db0 0x7f29141f4ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:54586/0 (socket says 172.21.15.59:54586) 2023-11-24T07:59:06.468 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:06.463+0000 7f291259c700 1 -- 172.21.15.59:0/2414311555 learned_addr learned my addr 172.21.15.59:0/2414311555 (peer_addr_for_me v2:172.21.15.59:0/0) 2023-11-24T07:59:06.468 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:06.463+0000 7f2912d9d700 1 --2- 172.21.15.59:0/2414311555 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f291415aa10 0x7f29140cc880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:06.468 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:06.463+0000 7f2911d9b700 1 --2- 172.21.15.59:0/2414311555 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f291415a630 0x7f291415bd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:06.468 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:06.464+0000 7f291259c700 1 -- 172.21.15.59:0/2414311555 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f291415aa10 msgr2=0x7f29140cc880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:06.468 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:06.464+0000 7f291259c700 1 --2- 172.21.15.59:0/2414311555 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f291415aa10 0x7f29140cc880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:06.468 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:06.464+0000 7f291259c700 1 -- 172.21.15.59:0/2414311555 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f291415a630 msgr2=0x7f291415bd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:06.469 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:06.464+0000 7f291259c700 1 --2- 172.21.15.59:0/2414311555 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f291415a630 0x7f291415bd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:06.469 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:06.464+0000 7f291259c700 1 -- 172.21.15.59:0/2414311555 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f29040094a0 con 0x7f29140b0db0 2023-11-24T07:59:06.469 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:06.464+0000 7f291259c700 1 --2- 172.21.15.59:0/2414311555 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f29140b0db0 0x7f29141f4ba0 secure :-1 s=READY pgs=826 cs=0 l=1 rev1=1 crypto rx=0x7f290800c4d0 tx=0x7f290800c890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:59:06.469 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:06.464+0000 7f2902ffd700 1 -- 172.21.15.59:0/2414311555 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2908011790 con 0x7f29140b0db0 2023-11-24T07:59:06.469 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:06.464+0000 7f2902ffd700 1 -- 172.21.15.59:0/2414311555 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f29080118f0 con 0x7f29140b0db0 2023-11-24T07:59:06.469 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:06.464+0000 7f29188dd700 1 -- 172.21.15.59:0/2414311555 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f291415cf60 con 0x7f29140b0db0 2023-11-24T07:59:06.469 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:06.464+0000 7f2902ffd700 1 -- 172.21.15.59:0/2414311555 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2908011ca0 con 0x7f29140b0db0 2023-11-24T07:59:06.469 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:06.464+0000 7f29188dd700 1 -- 172.21.15.59:0/2414311555 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f291415d520 con 0x7f29140b0db0 2023-11-24T07:59:06.469 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:06.465+0000 7f29188dd700 1 -- 172.21.15.59:0/2414311555 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2914068670 con 0x7f29140b0db0 2023-11-24T07:59:06.469 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:06.465+0000 7f2902ffd700 1 -- 172.21.15.59:0/2414311555 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 10) v1 ==== 89588+0+0 (secure 0 0 0) 0x7f290801c460 con 0x7f29140b0db0 2023-11-24T07:59:06.469 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:06.466+0000 7f2902ffd700 1 --2- 172.21.15.59:0/2414311555 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f28f406b2c0 0x7f28f406d780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:06.469 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:06.466+0000 7f2902ffd700 1 -- 172.21.15.59:0/2414311555 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(65..65 src has 1..65) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f2908089600 con 0x7f29140b0db0 2023-11-24T07:59:06.470 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:06.466+0000 7f2911d9b700 1 --2- 172.21.15.59:0/2414311555 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f28f406b2c0 0x7f28f406d780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:06.470 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:06.466+0000 7f2911d9b700 1 --2- 172.21.15.59:0/2414311555 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f28f406b2c0 0x7f28f406d780 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f29040090b0 tx=0x7f290401a040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:59:06.473 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:06.470+0000 7f2902ffd700 1 -- 172.21.15.59:0/2414311555 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f29080543d0 con 0x7f29140b0db0 2023-11-24T07:59:06.757 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:06.756+0000 7f29188dd700 1 -- 172.21.15.59:0/2414311555 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_volumes.TestSubvolumeSnapshots.test_subvolume_inherited_snapshot_info"]} v 0) v1 -- 0x7f29140b3ee0 con 0x7f29140b0db0 2023-11-24T07:59:07.606 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:07.604+0000 7f2902ffd700 1 -- 172.21.15.59:0/2414311555 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_volumes.TestSubvolumeSnapshots.test_subvolume_inherited_snapshot_info"]}]=0 v171) v1 ==== 169+0+0 (secure 0 0 0) 0x7f29080579f0 con 0x7f29140b0db0 2023-11-24T07:59:07.609 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:07.607+0000 7f29188dd700 1 -- 172.21.15.59:0/2414311555 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f28f406b2c0 msgr2=0x7f28f406d780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:07.609 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:07.607+0000 7f29188dd700 1 --2- 172.21.15.59:0/2414311555 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f28f406b2c0 0x7f28f406d780 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f29040090b0 tx=0x7f290401a040 comp rx=0 tx=0).stop 2023-11-24T07:59:07.609 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:07.607+0000 7f29188dd700 1 -- 172.21.15.59:0/2414311555 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f29140b0db0 msgr2=0x7f29141f4ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:07.610 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:07.607+0000 7f29188dd700 1 --2- 172.21.15.59:0/2414311555 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f29140b0db0 0x7f29141f4ba0 secure :-1 s=READY pgs=826 cs=0 l=1 rev1=1 crypto rx=0x7f290800c4d0 tx=0x7f290800c890 comp rx=0 tx=0).stop 2023-11-24T07:59:07.610 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:07.607+0000 7f29188dd700 1 -- 172.21.15.59:0/2414311555 shutdown_connections 2023-11-24T07:59:07.610 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:07.607+0000 7f29188dd700 1 --2- 172.21.15.59:0/2414311555 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f28f406b2c0 0x7f28f406d780 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:07.610 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:07.607+0000 7f29188dd700 1 --2- 172.21.15.59:0/2414311555 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f291415aa10 0x7f29140cc880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:07.610 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:07.607+0000 7f29188dd700 1 --2- 172.21.15.59:0/2414311555 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f29140b0db0 0x7f29141f4ba0 unknown :-1 s=CLOSED pgs=826 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:07.610 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:07.607+0000 7f29188dd700 1 --2- 172.21.15.59:0/2414311555 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f291415a630 0x7f291415bd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:07.610 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:07.607+0000 7f29188dd700 1 -- 172.21.15.59:0/2414311555 >> 172.21.15.59:0/2414311555 conn(0x7f29140b3190 msgr2=0x7f29140b3570 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:59:07.610 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:07.608+0000 7f29188dd700 1 -- 172.21.15.59:0/2414311555 shutdown_connections 2023-11-24T07:59:07.610 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:07.608+0000 7f29188dd700 1 -- 172.21.15.59:0/2414311555 wait complete. 2023-11-24T07:59:07.624 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2023-11-24T07:59:07.759 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:07.757+0000 7ff1a8c4b700 1 Processor -- start 2023-11-24T07:59:07.759 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:07.757+0000 7ff1a8c4b700 1 -- start start 2023-11-24T07:59:07.761 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:07.758+0000 7ff1a8c4b700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7ff1a415ad70 0x7ff1a40b5e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:07.761 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:07.758+0000 7ff1a8c4b700 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7ff1a40adda0 0x7ff1a406b970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:07.761 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:07.758+0000 7ff1a8c4b700 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff1a415b150 con 0x7ff1a40b03b0 2023-11-24T07:59:07.762 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:07.758+0000 7ff1a8c4b700 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff1a4073610 con 0x7ff1a415ad70 2023-11-24T07:59:07.762 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:07.758+0000 7ff1a8c4b700 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff1a40b6380 con 0x7ff1a40adda0 2023-11-24T07:59:07.762 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:07.758+0000 7ff1a259c700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7ff1a415ad70 0x7ff1a40b5e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:07.762 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:07.758+0000 7ff1a259c700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7ff1a415ad70 0x7ff1a40b5e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:54594/0 (socket says 172.21.15.59:54594) 2023-11-24T07:59:07.762 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:07.758+0000 7ff1a259c700 1 -- 172.21.15.59:0/1962580758 learned_addr learned my addr 172.21.15.59:0/1962580758 (peer_addr_for_me v2:172.21.15.59:0/0) 2023-11-24T07:59:07.762 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:07.758+0000 7ff1a2d9d700 1 --2- 172.21.15.59:0/1962580758 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7ff1a40adda0 0x7ff1a406b970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:07.762 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:07.759+0000 7ff1a259c700 1 -- 172.21.15.59:0/1962580758 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7ff1a40adda0 msgr2=0x7ff1a406b970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T07:59:07.762 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:07.759+0000 7ff1a259c700 1 --2- 172.21.15.59:0/1962580758 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7ff1a40adda0 0x7ff1a406b970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:07.762 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:07.759+0000 7ff1a259c700 1 -- 172.21.15.59:0/1962580758 >> v1:172.21.15.76:6789/0 conn(0x7ff1a40b03b0 legacy=0x7ff1a40ad690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:07.762 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:07.759+0000 7ff1a259c700 1 -- 172.21.15.59:0/1962580758 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff1a40b0ec0 con 0x7ff1a415ad70 2023-11-24T07:59:07.762 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:07.759+0000 7ff1a2d9d700 1 --2- 172.21.15.59:0/1962580758 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7ff1a40adda0 0x7ff1a406b970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-24T07:59:07.763 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:07.759+0000 7ff1a259c700 1 --2- 172.21.15.59:0/1962580758 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff1a415ad70 0x7ff1a40b5e40 secure :-1 s=READY pgs=827 cs=0 l=1 rev1=1 crypto rx=0x7ff19800bbf0 tx=0x7ff19800bf00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2f046995d94efae3 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:59:07.763 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:07.759+0000 7ff1a159a700 1 -- 172.21.15.59:0/1962580758 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff19800dd60 con 0x7ff1a415ad70 2023-11-24T07:59:07.763 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:07.759+0000 7ff1a159a700 1 -- 172.21.15.59:0/1962580758 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff198010470 con 0x7ff1a415ad70 2023-11-24T07:59:07.763 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:07.759+0000 7ff1a159a700 1 -- 172.21.15.59:0/1962580758 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff198010800 con 0x7ff1a415ad70 2023-11-24T07:59:07.763 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:07.759+0000 7ff1a8c4b700 1 -- 172.21.15.59:0/1962580758 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff1a415ad70 msgr2=0x7ff1a40b5e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:07.763 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:07.759+0000 7ff1a8c4b700 1 --2- 172.21.15.59:0/1962580758 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff1a415ad70 0x7ff1a40b5e40 secure :-1 s=READY pgs=827 cs=0 l=1 rev1=1 crypto rx=0x7ff19800bbf0 tx=0x7ff19800bf00 comp rx=0 tx=0).stop 2023-11-24T07:59:07.763 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:07.760+0000 7ff1a8c4b700 1 -- 172.21.15.59:0/1962580758 shutdown_connections 2023-11-24T07:59:07.763 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:07.760+0000 7ff1a8c4b700 1 --2- 172.21.15.59:0/1962580758 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7ff1a40adda0 0x7ff1a406b970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:07.763 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:07.760+0000 7ff1a8c4b700 1 --2- 172.21.15.59:0/1962580758 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff1a415ad70 0x7ff1a40b5e40 unknown :-1 s=CLOSED pgs=827 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:07.763 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:07.760+0000 7ff1a8c4b700 1 -- 172.21.15.59:0/1962580758 >> 172.21.15.59:0/1962580758 conn(0x7ff1a40b5070 msgr2=0x7ff1a40b5730 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:59:07.763 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:07.760+0000 7ff1a8c4b700 1 -- 172.21.15.59:0/1962580758 shutdown_connections 2023-11-24T07:59:07.763 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:07.760+0000 7ff1a8c4b700 1 -- 172.21.15.59:0/1962580758 wait complete. 2023-11-24T07:59:07.766 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:07.761+0000 7ff1a8c4b700 1 Processor -- start 2023-11-24T07:59:07.766 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:07.761+0000 7ff1a8c4b700 1 -- start start 2023-11-24T07:59:07.766 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:07.762+0000 7ff1a8c4b700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff1a40adda0 0x7ff1a4156de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:07.766 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:07.762+0000 7ff1a8c4b700 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7ff1a40b03b0 0x7ff1a4152710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:07.766 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:07.762+0000 7ff1a8c4b700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff1a415ad70 0x7ff1a415d6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:07.766 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:07.762+0000 7ff1a8c4b700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1a4157320 con 0x7ff1a40adda0 2023-11-24T07:59:07.766 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:07.762+0000 7ff1a8c4b700 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1a4157550 con 0x7ff1a415ad70 2023-11-24T07:59:07.767 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:07.762+0000 7ff1a8c4b700 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff1a4157830 con 0x7ff1a40b03b0 2023-11-24T07:59:07.767 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:07.762+0000 7ff1a259c700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff1a40adda0 0x7ff1a4156de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:07.767 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:07.762+0000 7ff1a259c700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff1a40adda0 0x7ff1a4156de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:54598/0 (socket says 172.21.15.59:54598) 2023-11-24T07:59:07.767 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:07.762+0000 7ff1a259c700 1 -- 172.21.15.59:0/915492283 learned_addr learned my addr 172.21.15.59:0/915492283 (peer_addr_for_me v2:172.21.15.59:0/0) 2023-11-24T07:59:07.767 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:07.762+0000 7ff1a2d9d700 1 --2- 172.21.15.59:0/915492283 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff1a415ad70 0x7ff1a415d6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:07.767 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:07.762+0000 7ff1a1d9b700 1 --2- 172.21.15.59:0/915492283 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7ff1a40b03b0 0x7ff1a4152710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:07.767 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:07.762+0000 7ff1a259c700 1 -- 172.21.15.59:0/915492283 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7ff1a40b03b0 msgr2=0x7ff1a4152710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:07.767 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:07.762+0000 7ff1a259c700 1 --2- 172.21.15.59:0/915492283 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7ff1a40b03b0 0x7ff1a4152710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:07.768 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:07.762+0000 7ff1a259c700 1 -- 172.21.15.59:0/915492283 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff1a415ad70 msgr2=0x7ff1a415d6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:07.768 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:07.762+0000 7ff1a259c700 1 --2- 172.21.15.59:0/915492283 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff1a415ad70 0x7ff1a415d6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:07.768 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:07.762+0000 7ff1a259c700 1 -- 172.21.15.59:0/915492283 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff19800b7f0 con 0x7ff1a40adda0 2023-11-24T07:59:07.768 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:07.762+0000 7ff1a2d9d700 1 --2- 172.21.15.59:0/915492283 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff1a415ad70 0x7ff1a415d6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-24T07:59:07.768 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:07.762+0000 7ff1a259c700 1 --2- 172.21.15.59:0/915492283 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff1a40adda0 0x7ff1a4156de0 secure :-1 s=READY pgs=828 cs=0 l=1 rev1=1 crypto rx=0x7ff198000c00 tx=0x7ff198004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:59:07.768 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:07.762+0000 7ff192ffd700 1 -- 172.21.15.59:0/915492283 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff198003b70 con 0x7ff1a40adda0 2023-11-24T07:59:07.768 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:07.762+0000 7ff192ffd700 1 -- 172.21.15.59:0/915492283 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff198003cd0 con 0x7ff1a40adda0 2023-11-24T07:59:07.768 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:07.762+0000 7ff1a8c4b700 1 -- 172.21.15.59:0/915492283 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff1a4157b10 con 0x7ff1a40adda0 2023-11-24T07:59:07.768 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:07.762+0000 7ff192ffd700 1 -- 172.21.15.59:0/915492283 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff198010820 con 0x7ff1a40adda0 2023-11-24T07:59:07.768 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:07.763+0000 7ff1a8c4b700 1 -- 172.21.15.59:0/915492283 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff1a41580d0 con 0x7ff1a40adda0 2023-11-24T07:59:07.768 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:07.763+0000 7ff1a8c4b700 1 -- 172.21.15.59:0/915492283 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff1a4075480 con 0x7ff1a40adda0 2023-11-24T07:59:07.768 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:07.764+0000 7ff192ffd700 1 -- 172.21.15.59:0/915492283 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 10) v1 ==== 89588+0+0 (secure 0 0 0) 0x7ff198023440 con 0x7ff1a40adda0 2023-11-24T07:59:07.769 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:07.764+0000 7ff192ffd700 1 --2- 172.21.15.59:0/915492283 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7ff18406b2c0 0x7ff18406d780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:07.769 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:07.764+0000 7ff192ffd700 1 -- 172.21.15.59:0/915492283 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(65..65 src has 1..65) v4 ==== 4943+0+0 (secure 0 0 0) 0x7ff19808b520 con 0x7ff1a40adda0 2023-11-24T07:59:07.769 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:07.765+0000 7ff1a1d9b700 1 --2- 172.21.15.59:0/915492283 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7ff18406b2c0 0x7ff18406d780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:07.769 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:07.765+0000 7ff1a1d9b700 1 --2- 172.21.15.59:0/915492283 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7ff18406b2c0 0x7ff18406d780 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7ff1a415d200 tx=0x7ff194009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:59:07.771 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:07.769+0000 7ff192ffd700 1 -- 172.21.15.59:0/915492283 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7ff198056270 con 0x7ff1a40adda0 2023-11-24T07:59:07.972 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:07.970+0000 7ff1a8c4b700 1 -- 172.21.15.59:0/915492283 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7ff1a40aeb80 con 0x7ff1a40adda0 2023-11-24T07:59:07.975 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:07.973+0000 7ff192ffd700 1 -- 172.21.15.59:0/915492283 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v16)=0 v16) v1 ==== 122+0+0 (secure 0 0 0) 0x7ff19808b030 con 0x7ff1a40adda0 2023-11-24T07:59:07.977 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:07.975+0000 7ff1a8c4b700 1 -- 172.21.15.59:0/915492283 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7ff18406b2c0 msgr2=0x7ff18406d780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:07.977 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:07.975+0000 7ff1a8c4b700 1 --2- 172.21.15.59:0/915492283 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7ff18406b2c0 0x7ff18406d780 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7ff1a415d200 tx=0x7ff194009040 comp rx=0 tx=0).stop 2023-11-24T07:59:07.978 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:07.975+0000 7ff1a8c4b700 1 -- 172.21.15.59:0/915492283 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff1a40adda0 msgr2=0x7ff1a4156de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:07.978 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:07.975+0000 7ff1a8c4b700 1 --2- 172.21.15.59:0/915492283 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff1a40adda0 0x7ff1a4156de0 secure :-1 s=READY pgs=828 cs=0 l=1 rev1=1 crypto rx=0x7ff198000c00 tx=0x7ff198004750 comp rx=0 tx=0).stop 2023-11-24T07:59:07.978 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:07.975+0000 7ff1a8c4b700 1 -- 172.21.15.59:0/915492283 shutdown_connections 2023-11-24T07:59:07.978 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:07.975+0000 7ff1a8c4b700 1 --2- 172.21.15.59:0/915492283 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7ff18406b2c0 0x7ff18406d780 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:07.978 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:07.975+0000 7ff1a8c4b700 1 --2- 172.21.15.59:0/915492283 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff1a415ad70 0x7ff1a415d6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:07.978 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:07.975+0000 7ff1a8c4b700 1 --2- 172.21.15.59:0/915492283 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff1a40adda0 0x7ff1a4156de0 unknown :-1 s=CLOSED pgs=828 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:07.978 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:07.975+0000 7ff1a8c4b700 1 --2- 172.21.15.59:0/915492283 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7ff1a40b03b0 0x7ff1a4152710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:07.979 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:07.975+0000 7ff1a8c4b700 1 -- 172.21.15.59:0/915492283 >> 172.21.15.59:0/915492283 conn(0x7ff1a40b5070 msgr2=0x7ff1a40ae3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:59:07.979 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:07.976+0000 7ff1a8c4b700 1 -- 172.21.15.59:0/915492283 shutdown_connections 2023-11-24T07:59:07.979 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:07.976+0000 7ff1a8c4b700 1 -- 172.21.15.59:0/915492283 wait complete. 2023-11-24T07:59:07.989 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-11-24T07:59:07.990 DEBUG:teuthology.orchestra.run.smithi059:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-11-24T07:59:08.039 DEBUG:teuthology.orchestra.run.smithi076:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-11-24T07:59:08.085 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-11-24T07:59:08.086 DEBUG:teuthology.orchestra.run.smithi059:> set -ex 2023-11-24T07:59:08.086 DEBUG:teuthology.orchestra.run.smithi059:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-24T07:59:08.103 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=16287364k,nr_inodes=4071841,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 tmpfs /sys/fs/cgroup tmpfs ro,seclabel,nosuid,nodev,noexec,mode=755 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,seclabel,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,seclabel,nosuid,nodev,noexec,relatime,hugetlb 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,seclabel,nosuid,nodev,noexec,relatime,perf_event 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,seclabel,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/rdma cgroup rw,seclabel,nosuid,nodev,noexec,relatime,rdma 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,seclabel,nosuid,nodev,noexec,relatime,freezer 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,seclabel,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,seclabel,nosuid,nodev,noexec,relatime,devices 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,blkio 0 0 none /sys/kernel/tracing tracefs rw,seclabel,relatime 0 0 configfs /sys/kernel/config configfs rw,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=42,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=16075 0 0 mqueue /dev/mqueue mqueue rw,seclabel,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,relatime 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3261172k,mode=700,uid=1000,gid=1265 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw,seclabel 0 0 2023-11-24T07:59:08.104 DEBUG:teuthology.orchestra.run.smithi076:> set -ex 2023-11-24T07:59:08.104 DEBUG:teuthology.orchestra.run.smithi076:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-24T07:59:08.124 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=16284272k,nr_inodes=4071068,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 tmpfs /sys/fs/cgroup tmpfs ro,seclabel,nosuid,nodev,noexec,mode=755 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,seclabel,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,seclabel,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,seclabel,nosuid,nodev,noexec,relatime,freezer 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,seclabel,nosuid,nodev,noexec,relatime,perf_event 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,seclabel,nosuid,nodev,noexec,relatime,hugetlb 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,seclabel,nosuid,nodev,noexec,relatime,devices 0 0 cgroup /sys/fs/cgroup/rdma cgroup rw,seclabel,nosuid,nodev,noexec,relatime,rdma 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,seclabel,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,blkio 0 0 none /sys/kernel/tracing tracefs rw,seclabel,relatime 0 0 configfs /sys/kernel/config configfs rw,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=35,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=20116 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,relatime 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3260552k,mode=700,uid=1000,gid=1265 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw,seclabel 0 0 2023-11-24T07:59:08.125 DEBUG:teuthology.orchestra.run.smithi059:> set -ex 2023-11-24T07:59:08.125 DEBUG:teuthology.orchestra.run.smithi059:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-24T07:59:08.167 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=16287364k,nr_inodes=4071841,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 tmpfs /sys/fs/cgroup tmpfs ro,seclabel,nosuid,nodev,noexec,mode=755 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,seclabel,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,seclabel,nosuid,nodev,noexec,relatime,hugetlb 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,seclabel,nosuid,nodev,noexec,relatime,perf_event 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,seclabel,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/rdma cgroup rw,seclabel,nosuid,nodev,noexec,relatime,rdma 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,seclabel,nosuid,nodev,noexec,relatime,freezer 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,seclabel,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,seclabel,nosuid,nodev,noexec,relatime,devices 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,blkio 0 0 none /sys/kernel/tracing tracefs rw,seclabel,relatime 0 0 configfs /sys/kernel/config configfs rw,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=42,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=16075 0 0 mqueue /dev/mqueue mqueue rw,seclabel,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,relatime 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3261172k,mode=700,uid=1000,gid=1265 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw,seclabel 0 0 2023-11-24T07:59:08.167 DEBUG:teuthology.orchestra.run.smithi076:> set -ex 2023-11-24T07:59:08.168 DEBUG:teuthology.orchestra.run.smithi076:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-24T07:59:08.189 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=16284272k,nr_inodes=4071068,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 tmpfs /sys/fs/cgroup tmpfs ro,seclabel,nosuid,nodev,noexec,mode=755 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,seclabel,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,seclabel,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,seclabel,nosuid,nodev,noexec,relatime,freezer 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,seclabel,nosuid,nodev,noexec,relatime,perf_event 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,seclabel,nosuid,nodev,noexec,relatime,hugetlb 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,seclabel,nosuid,nodev,noexec,relatime,devices 0 0 cgroup /sys/fs/cgroup/rdma cgroup rw,seclabel,nosuid,nodev,noexec,relatime,rdma 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,seclabel,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,blkio 0 0 none /sys/kernel/tracing tracefs rw,seclabel,relatime 0 0 configfs /sys/kernel/config configfs rw,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=35,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=20116 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,relatime 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3260552k,mode=700,uid=1000,gid=1265 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw,seclabel 0 0 2023-11-24T07:59:08.189 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2023-11-24T07:59:08.190 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-24T07:59:08.333 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:08.332+0000 7f9a48c02700 1 Processor -- start 2023-11-24T07:59:08.334 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:08.332+0000 7f9a48c02700 1 -- start start 2023-11-24T07:59:08.336 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:08.333+0000 7f9a48c02700 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f9a440d16e0 0x7f9a440affb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:08.336 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:08.333+0000 7f9a48c02700 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9a440d1b40 con 0x7f9a440ada10 2023-11-24T07:59:08.337 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:08.333+0000 7f9a48c02700 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9a44070490 con 0x7f9a4406afd0 2023-11-24T07:59:08.337 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:08.333+0000 7f9a48c02700 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9a440b04f0 con 0x7f9a440d16e0 2023-11-24T07:59:08.337 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:08.333+0000 7f9a42d9d700 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7f9a440ada10 0x7f9a440ade70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:58084/0 (socket says 172.21.15.59:58084) 2023-11-24T07:59:08.337 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:08.333+0000 7f9a42d9d700 1 -- 172.21.15.59:0/3855664331 learned_addr learned my addr 172.21.15.59:0/3855664331 (peer_addr_for_me v1:172.21.15.59:0/0) 2023-11-24T07:59:08.337 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:08.333+0000 7f9a41d9b700 1 --2- 172.21.15.59:0/3855664331 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f9a440d16e0 0x7f9a440affb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:08.338 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:08.333+0000 7f9a4159a700 1 -- 172.21.15.59:0/3855664331 <== mon.0 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3537654681 0 0) 0x7f9a440d1b40 con 0x7f9a440ada10 2023-11-24T07:59:08.338 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:08.333+0000 7f9a4159a700 1 -- 172.21.15.59:0/3855664331 --> v1:172.21.15.59:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9a2c0035b0 con 0x7f9a440ada10 2023-11-24T07:59:08.338 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:08.333+0000 7f9a4159a700 1 -- 172.21.15.59:0/3855664331 <== mon.1 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 443236438 0 0) 0x7f9a44070490 con 0x7f9a4406afd0 2023-11-24T07:59:08.338 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:08.333+0000 7f9a4159a700 1 -- 172.21.15.59:0/3855664331 --> v1:172.21.15.76:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9a440d1b40 con 0x7f9a4406afd0 2023-11-24T07:59:08.338 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:08.333+0000 7f9a41d9b700 1 -- 172.21.15.59:0/3855664331 >> v1:172.21.15.76:6789/0 conn(0x7f9a4406afd0 legacy=0x7f9a4406b3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:08.339 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:08.333+0000 7f9a41d9b700 1 -- 172.21.15.59:0/3855664331 >> v1:172.21.15.59:6789/0 conn(0x7f9a440ada10 legacy=0x7f9a440ade70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:08.339 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:08.333+0000 7f9a41d9b700 1 -- 172.21.15.59:0/3855664331 --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9a440af9f0 con 0x7f9a440d16e0 2023-11-24T07:59:08.339 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:08.334+0000 7f9a41d9b700 1 --2- 172.21.15.59:0/3855664331 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f9a440d16e0 0x7f9a440affb0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f9a38009870 tx=0x7f9a38009b80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3a7717895778e984 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:59:08.339 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:08.334+0000 7f9a4159a700 1 -- 172.21.15.59:0/3855664331 <== mon.2 v2:172.21.15.76:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a38011830 con 0x7f9a440d16e0 2023-11-24T07:59:08.339 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:08.334+0000 7f9a4159a700 1 -- 172.21.15.59:0/3855664331 <== mon.2 v2:172.21.15.76:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9a38011990 con 0x7f9a440d16e0 2023-11-24T07:59:08.340 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:08.334+0000 7f9a4159a700 1 -- 172.21.15.59:0/3855664331 <== mon.2 v2:172.21.15.76:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a38011d00 con 0x7f9a440d16e0 2023-11-24T07:59:08.340 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:08.334+0000 7f9a48c02700 1 -- 172.21.15.59:0/3855664331 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f9a440d16e0 msgr2=0x7f9a440affb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:08.340 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:08.334+0000 7f9a48c02700 1 --2- 172.21.15.59:0/3855664331 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f9a440d16e0 0x7f9a440affb0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f9a38009870 tx=0x7f9a38009b80 comp rx=0 tx=0).stop 2023-11-24T07:59:08.340 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:08.334+0000 7f9a48c02700 1 -- 172.21.15.59:0/3855664331 shutdown_connections 2023-11-24T07:59:08.340 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:08.334+0000 7f9a48c02700 1 --2- 172.21.15.59:0/3855664331 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f9a440d16e0 0x7f9a440affb0 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:08.340 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:08.334+0000 7f9a48c02700 1 -- 172.21.15.59:0/3855664331 >> 172.21.15.59:0/3855664331 conn(0x7f9a440d0bc0 msgr2=0x7f9a440d0fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:59:08.341 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:08.334+0000 7f9a48c02700 1 -- 172.21.15.59:0/3855664331 shutdown_connections 2023-11-24T07:59:08.341 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:08.334+0000 7f9a48c02700 1 -- 172.21.15.59:0/3855664331 wait complete. 2023-11-24T07:59:08.343 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:08.336+0000 7f9a48c02700 1 Processor -- start 2023-11-24T07:59:08.344 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:08.336+0000 7f9a48c02700 1 -- start start 2023-11-24T07:59:08.344 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:08.337+0000 7f9a48c02700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9a4406afd0 0x7f9a44159d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:08.344 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:08.337+0000 7f9a48c02700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9a440ada10 0x7f9a4415a2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:08.344 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:08.337+0000 7f9a48c02700 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f9a440d16e0 0x7f9a440c5a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:08.344 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:08.337+0000 7f9a48c02700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a4415a7f0 con 0x7f9a440ada10 2023-11-24T07:59:08.344 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:08.337+0000 7f9a48c02700 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a4415ad60 con 0x7f9a4406afd0 2023-11-24T07:59:08.344 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:08.337+0000 7f9a48c02700 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9a4415aa80 con 0x7f9a440d16e0 2023-11-24T07:59:08.344 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:08.337+0000 7f9a41d9b700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9a440ada10 0x7f9a4415a2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:08.344 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:08.337+0000 7f9a41d9b700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9a440ada10 0x7f9a4415a2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:54612/0 (socket says 172.21.15.59:54612) 2023-11-24T07:59:08.345 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:08.337+0000 7f9a41d9b700 1 -- 172.21.15.59:0/1791723512 learned_addr learned my addr 172.21.15.59:0/1791723512 (peer_addr_for_me v2:172.21.15.59:0/0) 2023-11-24T07:59:08.345 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:08.337+0000 7f9a4259c700 1 --2- 172.21.15.59:0/1791723512 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9a4406afd0 0x7f9a44159d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:08.345 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:08.337+0000 7f9a42d9d700 1 --2- 172.21.15.59:0/1791723512 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f9a440d16e0 0x7f9a440c5a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:08.345 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:08.337+0000 7f9a41d9b700 1 -- 172.21.15.59:0/1791723512 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f9a440d16e0 msgr2=0x7f9a440c5a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:08.345 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:08.337+0000 7f9a41d9b700 1 --2- 172.21.15.59:0/1791723512 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f9a440d16e0 0x7f9a440c5a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:08.345 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:08.337+0000 7f9a41d9b700 1 -- 172.21.15.59:0/1791723512 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9a4406afd0 msgr2=0x7f9a44159d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:08.345 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:08.337+0000 7f9a41d9b700 1 --2- 172.21.15.59:0/1791723512 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9a4406afd0 0x7f9a44159d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:08.345 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:08.338+0000 7f9a41d9b700 1 -- 172.21.15.59:0/1791723512 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9a380094a0 con 0x7f9a440ada10 2023-11-24T07:59:08.346 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:08.338+0000 7f9a41d9b700 1 --2- 172.21.15.59:0/1791723512 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9a440ada10 0x7f9a4415a2b0 secure :-1 s=READY pgs=830 cs=0 l=1 rev1=1 crypto rx=0x7f9a380090b0 tx=0x7f9a38004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:59:08.346 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:08.338+0000 7f9a32ffd700 1 -- 172.21.15.59:0/1791723512 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a380044d0 con 0x7f9a440ada10 2023-11-24T07:59:08.346 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:08.338+0000 7f9a32ffd700 1 -- 172.21.15.59:0/1791723512 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9a3801f070 con 0x7f9a440ada10 2023-11-24T07:59:08.346 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:08.338+0000 7f9a48c02700 1 -- 172.21.15.59:0/1791723512 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9a4415b480 con 0x7f9a440ada10 2023-11-24T07:59:08.346 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:08.338+0000 7f9a32ffd700 1 -- 172.21.15.59:0/1791723512 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a380038e0 con 0x7f9a440ada10 2023-11-24T07:59:08.346 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:08.338+0000 7f9a48c02700 1 -- 172.21.15.59:0/1791723512 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9a4415ba40 con 0x7f9a440ada10 2023-11-24T07:59:08.346 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:08.339+0000 7f9a48c02700 1 -- 172.21.15.59:0/1791723512 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9a4415bbd0 con 0x7f9a440ada10 2023-11-24T07:59:08.346 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:08.339+0000 7f9a32ffd700 1 -- 172.21.15.59:0/1791723512 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 10) v1 ==== 89588+0+0 (secure 0 0 0) 0x7f9a38003a40 con 0x7f9a440ada10 2023-11-24T07:59:08.347 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:08.340+0000 7f9a32ffd700 1 --2- 172.21.15.59:0/1791723512 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f9a2406b2c0 0x7f9a2406d780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:08.347 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:08.340+0000 7f9a32ffd700 1 -- 172.21.15.59:0/1791723512 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(65..65 src has 1..65) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f9a38021070 con 0x7f9a440ada10 2023-11-24T07:59:08.347 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:08.340+0000 7f9a4259c700 1 --2- 172.21.15.59:0/1791723512 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f9a2406b2c0 0x7f9a2406d780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:08.347 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:08.340+0000 7f9a4259c700 1 --2- 172.21.15.59:0/1791723512 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f9a2406b2c0 0x7f9a2406d780 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f9a34001790 tx=0x7f9a34009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:59:08.348 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:08.344+0000 7f9a32ffd700 1 -- 172.21.15.59:0/1791723512 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f9a38055fd0 con 0x7f9a440ada10 2023-11-24T07:59:08.591 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:08.589+0000 7f9a48c02700 1 -- 172.21.15.59:0/1791723512 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f9a4406f0c0 con 0x7f9a440ada10 2023-11-24T07:59:08.591 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:08.590+0000 7f9a32ffd700 1 -- 172.21.15.59:0/1791723512 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 38 v38) v1 ==== 94+0+3693 (secure 0 0 0) 0x7f9a38015070 con 0x7f9a440ada10 2023-11-24T07:59:08.592 INFO:teuthology.orchestra.run.smithi059.stdout: 2023-11-24T07:59:08.592 INFO:teuthology.orchestra.run.smithi059.stdout:{"epoch":38,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5300,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6834/172682993","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6833","nonce":172682993},{"type":"v1","addr":"172.21.15.59:6834","nonce":172682993}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":29},{"gid":5701,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.76:6836/923821877","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6834","nonce":923821877},{"type":"v1","addr":"172.21.15.76:6836","nonce":923821877}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":30},{"gid":5704,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.76:6837/4166482716","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6835","nonce":4166482716},{"type":"v1","addr":"172.21.15.76:6837","nonce":4166482716}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":29},{"gid":6468,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6835/1177569889","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6832","nonce":1177569889},{"type":"v1","addr":"172.21.15.59:6835","nonce":1177569889}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":38}],"filesystems":[]} 2023-11-24T07:59:08.594 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:08.592+0000 7f9a48c02700 1 -- 172.21.15.59:0/1791723512 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f9a2406b2c0 msgr2=0x7f9a2406d780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:08.594 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:08.592+0000 7f9a48c02700 1 --2- 172.21.15.59:0/1791723512 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f9a2406b2c0 0x7f9a2406d780 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f9a34001790 tx=0x7f9a34009040 comp rx=0 tx=0).stop 2023-11-24T07:59:08.594 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:08.592+0000 7f9a48c02700 1 -- 172.21.15.59:0/1791723512 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9a440ada10 msgr2=0x7f9a4415a2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:08.594 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:08.592+0000 7f9a48c02700 1 --2- 172.21.15.59:0/1791723512 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9a440ada10 0x7f9a4415a2b0 secure :-1 s=READY pgs=830 cs=0 l=1 rev1=1 crypto rx=0x7f9a380090b0 tx=0x7f9a38004750 comp rx=0 tx=0).stop 2023-11-24T07:59:08.594 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:08.592+0000 7f9a48c02700 1 -- 172.21.15.59:0/1791723512 shutdown_connections 2023-11-24T07:59:08.594 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:08.592+0000 7f9a48c02700 1 --2- 172.21.15.59:0/1791723512 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f9a2406b2c0 0x7f9a2406d780 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:08.594 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:08.592+0000 7f9a48c02700 1 --2- 172.21.15.59:0/1791723512 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f9a440d16e0 0x7f9a440c5a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:08.594 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:08.592+0000 7f9a48c02700 1 --2- 172.21.15.59:0/1791723512 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9a4406afd0 0x7f9a44159d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:08.594 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:08.592+0000 7f9a48c02700 1 --2- 172.21.15.59:0/1791723512 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9a440ada10 0x7f9a4415a2b0 unknown :-1 s=CLOSED pgs=830 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:08.595 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:08.592+0000 7f9a48c02700 1 -- 172.21.15.59:0/1791723512 >> 172.21.15.59:0/1791723512 conn(0x7f9a440d0bc0 msgr2=0x7f9a44071ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:59:08.595 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:08.592+0000 7f9a48c02700 1 -- 172.21.15.59:0/1791723512 shutdown_connections 2023-11-24T07:59:08.595 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:08.592+0000 7f9a48c02700 1 -- 172.21.15.59:0/1791723512 wait complete. 2023-11-24T07:59:08.595 INFO:teuthology.orchestra.run.smithi059.stderr:dumped fsmap epoch 38 2023-11-24T07:59:08.606 INFO:tasks.ceph.mds.a:Restarting daemon 2023-11-24T07:59:08.606 INFO:tasks.ceph.mds.a:Stopping old one... 2023-11-24T07:59:08.607 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2023-11-24T07:59:08.607 INFO:teuthology.orchestra.run:waiting for 300 2023-11-24T07:59:08.607 INFO:tasks.ceph.mds.c:Restarting daemon 2023-11-24T07:59:08.607 INFO:tasks.ceph.mds.c:Stopping old one... 2023-11-24T07:59:08.608 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2023-11-24T07:59:08.608 INFO:teuthology.orchestra.run:waiting for 300 2023-11-24T07:59:08.608 INFO:tasks.ceph.mds.b:Restarting daemon 2023-11-24T07:59:08.608 INFO:tasks.ceph.mds.b:Stopping old one... 2023-11-24T07:59:08.608 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2023-11-24T07:59:08.608 INFO:teuthology.orchestra.run:waiting for 300 2023-11-24T07:59:08.608 INFO:tasks.ceph.mds.d:Restarting daemon 2023-11-24T07:59:08.608 INFO:tasks.ceph.mds.d:Stopping old one... 2023-11-24T07:59:08.609 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2023-11-24T07:59:08.609 INFO:teuthology.orchestra.run:waiting for 300 2023-11-24T07:59:08.617 INFO:tasks.ceph.mds.a:Stopped 2023-11-24T07:59:08.617 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2023-11-24T07:59:08.618 INFO:tasks.ceph.mds.c:Stopped 2023-11-24T07:59:08.618 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2023-11-24T07:59:08.620 INFO:tasks.ceph.mds.b:Stopped 2023-11-24T07:59:08.620 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2023-11-24T07:59:08.620 INFO:tasks.ceph.mds.d:Stopped 2023-11-24T07:59:08.620 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2023-11-24T07:59:08.660 INFO:tasks.ceph.mds.a:Started 2023-11-24T07:59:08.660 DEBUG:teuthology.parallel:result is None 2023-11-24T07:59:08.661 INFO:tasks.ceph.mds.c:Started 2023-11-24T07:59:08.661 DEBUG:teuthology.parallel:result is None 2023-11-24T07:59:08.662 INFO:tasks.ceph.mds.b:Started 2023-11-24T07:59:08.662 DEBUG:teuthology.parallel:result is None 2023-11-24T07:59:08.663 INFO:tasks.ceph.mds.d:Started 2023-11-24T07:59:08.663 DEBUG:teuthology.parallel:result is None 2023-11-24T07:59:08.663 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2023-11-24T07:59:08.742 INFO:tasks.ceph.mds.a.smithi059.stdout:starting mds.a at 2023-11-24T07:59:08.743 INFO:tasks.ceph.mds.c.smithi059.stdout:starting mds.c at 2023-11-24T07:59:08.748 INFO:tasks.ceph.mds.b.smithi076.stdout:starting mds.b at 2023-11-24T07:59:08.748 INFO:tasks.ceph.mds.d.smithi076.stdout:starting mds.d at 2023-11-24T07:59:08.794 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:08.793+0000 7fac3dd64700 1 Processor -- start 2023-11-24T07:59:08.795 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:08.793+0000 7fac3dd64700 1 -- start start 2023-11-24T07:59:08.796 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:08.794+0000 7fac3dd64700 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fac38143ae0 0x7fac380b51b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:08.796 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:08.794+0000 7fac3dd64700 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fac380722c0 0x7fac3806e510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:08.796 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:08.794+0000 7fac3dd64700 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fac38143ec0 con 0x7fac381440f0 2023-11-24T07:59:08.797 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:08.794+0000 7fac3dd64700 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fac380b40a0 con 0x7fac380722c0 2023-11-24T07:59:08.797 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:08.794+0000 7fac3dd64700 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fac380b3ae0 con 0x7fac38143ae0 2023-11-24T07:59:08.797 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:08.794+0000 7fac37fff700 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fac380722c0 0x7fac3806e510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:08.797 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:08.794+0000 7fac37fff700 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fac380722c0 0x7fac3806e510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.59:41844/0 (socket says 172.21.15.59:41844) 2023-11-24T07:59:08.797 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:08.794+0000 7fac37fff700 1 -- 172.21.15.59:0/3457230687 learned_addr learned my addr 172.21.15.59:0/3457230687 (peer_addr_for_me v2:172.21.15.59:0/0) 2023-11-24T07:59:08.797 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:08.794+0000 7fac377fe700 1 --2- 172.21.15.59:0/3457230687 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fac38143ae0 0x7fac380b51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:08.797 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:08.794+0000 7fac37fff700 1 -- 172.21.15.59:0/3457230687 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fac38143ae0 msgr2=0x7fac380b51b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T07:59:08.797 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:08.794+0000 7fac37fff700 1 --2- 172.21.15.59:0/3457230687 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fac38143ae0 0x7fac380b51b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:08.797 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:08.794+0000 7fac37fff700 1 -- 172.21.15.59:0/3457230687 >> v1:172.21.15.76:6789/0 conn(0x7fac381440f0 legacy=0x7fac38071bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:08.797 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:08.794+0000 7fac37fff700 1 -- 172.21.15.59:0/3457230687 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fac380b3dc0 con 0x7fac380722c0 2023-11-24T07:59:08.797 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:08.794+0000 7fac377fe700 1 --2- 172.21.15.59:0/3457230687 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fac38143ae0 0x7fac380b51b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-24T07:59:08.797 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:08.795+0000 7fac37fff700 1 --2- 172.21.15.59:0/3457230687 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fac380722c0 0x7fac3806e510 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7fac30009e50 tx=0x7fac3000b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=53f2f71718ad8e5b server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:59:08.798 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:08.795+0000 7fac367fc700 1 -- 172.21.15.59:0/3457230687 <== mon.1 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac3000e070 con 0x7fac380722c0 2023-11-24T07:59:08.798 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:08.795+0000 7fac367fc700 1 -- 172.21.15.59:0/3457230687 <== mon.1 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fac3000b910 con 0x7fac380722c0 2023-11-24T07:59:08.798 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:08.795+0000 7fac367fc700 1 -- 172.21.15.59:0/3457230687 <== mon.1 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac3000bcc0 con 0x7fac380722c0 2023-11-24T07:59:08.801 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:08.795+0000 7fac3dd64700 1 -- 172.21.15.59:0/3457230687 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fac380722c0 msgr2=0x7fac3806e510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:08.801 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:08.795+0000 7fac3dd64700 1 --2- 172.21.15.59:0/3457230687 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fac380722c0 0x7fac3806e510 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7fac30009e50 tx=0x7fac3000b330 comp rx=0 tx=0).stop 2023-11-24T07:59:08.801 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:08.795+0000 7fac3dd64700 1 -- 172.21.15.59:0/3457230687 shutdown_connections 2023-11-24T07:59:08.801 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:08.795+0000 7fac3dd64700 1 --2- 172.21.15.59:0/3457230687 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fac380722c0 0x7fac3806e510 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:08.802 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:08.795+0000 7fac3dd64700 1 --2- 172.21.15.59:0/3457230687 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fac38143ae0 0x7fac380b51b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:08.802 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:08.795+0000 7fac3dd64700 1 -- 172.21.15.59:0/3457230687 >> 172.21.15.59:0/3457230687 conn(0x7fac380b2740 msgr2=0x7fac380b2b20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:59:08.802 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:08.795+0000 7fac3dd64700 1 -- 172.21.15.59:0/3457230687 shutdown_connections 2023-11-24T07:59:08.802 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:08.795+0000 7fac3dd64700 1 -- 172.21.15.59:0/3457230687 wait complete. 2023-11-24T07:59:08.802 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:08.797+0000 7fac3dd64700 1 Processor -- start 2023-11-24T07:59:08.803 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:08.797+0000 7fac3dd64700 1 -- start start 2023-11-24T07:59:08.803 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:08.797+0000 7fac3dd64700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fac380722c0 0x7fac3813f000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:08.803 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:08.797+0000 7fac3dd64700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fac38143ae0 0x7fac380ce8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:08.803 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:08.797+0000 7fac3dd64700 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fac381440f0 0x7fac380cedf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:08.803 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:08.798+0000 7fac3dd64700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac3813f540 con 0x7fac380722c0 2023-11-24T07:59:08.803 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:08.798+0000 7fac3dd64700 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac380cca70 con 0x7fac38143ae0 2023-11-24T07:59:08.804 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:08.798+0000 7fac3dd64700 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fac381f55c0 con 0x7fac381440f0 2023-11-24T07:59:08.804 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:08.798+0000 7fac377fe700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fac380722c0 0x7fac3813f000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:08.804 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:08.798+0000 7fac377fe700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fac380722c0 0x7fac3813f000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:54648/0 (socket says 172.21.15.59:54648) 2023-11-24T07:59:08.804 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:08.798+0000 7fac377fe700 1 -- 172.21.15.59:0/4050959694 learned_addr learned my addr 172.21.15.59:0/4050959694 (peer_addr_for_me v2:172.21.15.59:0/0) 2023-11-24T07:59:08.804 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:08.798+0000 7fac37fff700 1 --2- 172.21.15.59:0/4050959694 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fac381440f0 0x7fac380cedf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:08.804 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:08.798+0000 7fac36ffd700 1 --2- 172.21.15.59:0/4050959694 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fac38143ae0 0x7fac380ce8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:08.804 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:08.798+0000 7fac377fe700 1 -- 172.21.15.59:0/4050959694 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fac381440f0 msgr2=0x7fac380cedf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:08.805 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:08.798+0000 7fac377fe700 1 --2- 172.21.15.59:0/4050959694 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fac381440f0 0x7fac380cedf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:08.805 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:08.798+0000 7fac377fe700 1 -- 172.21.15.59:0/4050959694 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fac38143ae0 msgr2=0x7fac380ce8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:08.805 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:08.798+0000 7fac377fe700 1 --2- 172.21.15.59:0/4050959694 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fac38143ae0 0x7fac380ce8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:08.805 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:08.798+0000 7fac377fe700 1 -- 172.21.15.59:0/4050959694 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fac280094a0 con 0x7fac380722c0 2023-11-24T07:59:08.805 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:08.798+0000 7fac377fe700 1 --2- 172.21.15.59:0/4050959694 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fac380722c0 0x7fac3813f000 secure :-1 s=READY pgs=837 cs=0 l=1 rev1=1 crypto rx=0x7fac2800ebe0 tx=0x7fac2800eef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:59:08.805 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:08.798+0000 7fac1ffff700 1 -- 172.21.15.59:0/4050959694 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac2800c850 con 0x7fac380722c0 2023-11-24T07:59:08.805 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:08.798+0000 7fac3dd64700 1 -- 172.21.15.59:0/4050959694 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fac30009b30 con 0x7fac380722c0 2023-11-24T07:59:08.805 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:08.798+0000 7fac3dd64700 1 -- 172.21.15.59:0/4050959694 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fac381f5b80 con 0x7fac380722c0 2023-11-24T07:59:08.805 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:08.798+0000 7fac1ffff700 1 -- 172.21.15.59:0/4050959694 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fac2800c9b0 con 0x7fac380722c0 2023-11-24T07:59:08.805 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:08.798+0000 7fac1ffff700 1 -- 172.21.15.59:0/4050959694 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac2800cd20 con 0x7fac380722c0 2023-11-24T07:59:08.806 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:08.799+0000 7fac3dd64700 1 -- 172.21.15.59:0/4050959694 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fac180052e0 con 0x7fac380722c0 2023-11-24T07:59:08.806 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:08.800+0000 7fac1ffff700 1 -- 172.21.15.59:0/4050959694 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 10) v1 ==== 89588+0+0 (secure 0 0 0) 0x7fac28013470 con 0x7fac380722c0 2023-11-24T07:59:08.806 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:08.800+0000 7fac1ffff700 1 --2- 172.21.15.59:0/4050959694 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7fac2406b2c0 0x7fac2406d780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:08.807 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:08.801+0000 7fac1ffff700 1 -- 172.21.15.59:0/4050959694 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(65..65 src has 1..65) v4 ==== 4943+0+0 (secure 0 0 0) 0x7fac28089dd0 con 0x7fac380722c0 2023-11-24T07:59:08.807 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:08.801+0000 7fac36ffd700 1 --2- 172.21.15.59:0/4050959694 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7fac2406b2c0 0x7fac2406d780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:08.807 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:08.801+0000 7fac36ffd700 1 --2- 172.21.15.59:0/4050959694 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7fac2406b2c0 0x7fac2406d780 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7fac38142270 tx=0x7fac2c009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:59:08.807 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:08.805+0000 7fac1ffff700 1 -- 172.21.15.59:0/4050959694 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fac28054b20 con 0x7fac380722c0 2023-11-24T07:59:09.025 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:09.023+0000 7fac3dd64700 1 -- 172.21.15.59:0/4050959694 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7fac18005590 con 0x7fac380722c0 2023-11-24T07:59:09.990 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:09.988+0000 7fac1ffff700 1 -- 172.21.15.59:0/4050959694 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v70) v1 ==== 97+0+0 (secure 0 0 0) 0x7fac28058140 con 0x7fac380722c0 2023-11-24T07:59:09.994 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:09.992+0000 7fac3dd64700 1 -- 172.21.15.59:0/4050959694 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7fac2406b2c0 msgr2=0x7fac2406d780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:09.994 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:09.992+0000 7fac3dd64700 1 --2- 172.21.15.59:0/4050959694 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7fac2406b2c0 0x7fac2406d780 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7fac38142270 tx=0x7fac2c009040 comp rx=0 tx=0).stop 2023-11-24T07:59:09.994 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:09.992+0000 7fac3dd64700 1 -- 172.21.15.59:0/4050959694 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fac380722c0 msgr2=0x7fac3813f000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:09.994 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:09.992+0000 7fac3dd64700 1 --2- 172.21.15.59:0/4050959694 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fac380722c0 0x7fac3813f000 secure :-1 s=READY pgs=837 cs=0 l=1 rev1=1 crypto rx=0x7fac2800ebe0 tx=0x7fac2800eef0 comp rx=0 tx=0).stop 2023-11-24T07:59:09.994 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:09.992+0000 7fac3dd64700 1 -- 172.21.15.59:0/4050959694 shutdown_connections 2023-11-24T07:59:09.995 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:09.992+0000 7fac3dd64700 1 --2- 172.21.15.59:0/4050959694 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7fac2406b2c0 0x7fac2406d780 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:09.995 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:09.992+0000 7fac3dd64700 1 --2- 172.21.15.59:0/4050959694 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fac381440f0 0x7fac380cedf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:09.995 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:09.992+0000 7fac3dd64700 1 --2- 172.21.15.59:0/4050959694 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fac380722c0 0x7fac3813f000 unknown :-1 s=CLOSED pgs=837 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:09.995 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:09.992+0000 7fac3dd64700 1 --2- 172.21.15.59:0/4050959694 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fac38143ae0 0x7fac380ce8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:09.995 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:09.992+0000 7fac3dd64700 1 -- 172.21.15.59:0/4050959694 >> 172.21.15.59:0/4050959694 conn(0x7fac380b2740 msgr2=0x7fac38071bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:59:09.995 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:09.992+0000 7fac3dd64700 1 -- 172.21.15.59:0/4050959694 shutdown_connections 2023-11-24T07:59:09.995 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:09.992+0000 7fac3dd64700 1 -- 172.21.15.59:0/4050959694 wait complete. 2023-11-24T07:59:09.995 INFO:teuthology.orchestra.run.smithi059.stderr: removed all blocklist entries 2023-11-24T07:59:10.009 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2023-11-24T07:59:10.141 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:10.139+0000 7f1a7255b700 1 Processor -- start 2023-11-24T07:59:10.141 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:10.139+0000 7f1a7255b700 1 -- start start 2023-11-24T07:59:10.143 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:10.140+0000 7f1a7255b700 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f1a6c143ae0 0x7f1a6c0b51b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:10.144 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:10.140+0000 7f1a7255b700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f1a6c1440f0 0x7f1a6c071bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:10.144 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:10.140+0000 7f1a7255b700 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1a6c143ec0 con 0x7f1a6c0720f0 2023-11-24T07:59:10.144 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:10.140+0000 7f1a7255b700 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1a6c0b40a0 con 0x7f1a6c1440f0 2023-11-24T07:59:10.144 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:10.140+0000 7f1a7255b700 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1a6c072580 con 0x7f1a6c143ae0 2023-11-24T07:59:10.144 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:10.140+0000 7f1a6b7fe700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f1a6c1440f0 0x7f1a6c071bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:10.144 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:10.140+0000 7f1a6b7fe700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f1a6c1440f0 0x7f1a6c071bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:54662/0 (socket says 172.21.15.59:54662) 2023-11-24T07:59:10.144 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:10.140+0000 7f1a6b7fe700 1 -- 172.21.15.59:0/4098188658 learned_addr learned my addr 172.21.15.59:0/4098188658 (peer_addr_for_me v2:172.21.15.59:0/0) 2023-11-24T07:59:10.144 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:10.140+0000 7f1a6bfff700 1 --2- 172.21.15.59:0/4098188658 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f1a6c143ae0 0x7f1a6c0b51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:10.144 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:10.141+0000 7f1a6b7fe700 1 -- 172.21.15.59:0/4098188658 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f1a6c143ae0 msgr2=0x7f1a6c0b51b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T07:59:10.144 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:10.141+0000 7f1a6b7fe700 1 --2- 172.21.15.59:0/4098188658 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f1a6c143ae0 0x7f1a6c0b51b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:10.144 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:10.141+0000 7f1a6b7fe700 1 -- 172.21.15.59:0/4098188658 >> v1:172.21.15.76:6789/0 conn(0x7f1a6c0720f0 legacy=0x7f1a6c15e8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:10.145 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:10.141+0000 7f1a6b7fe700 1 -- 172.21.15.59:0/4098188658 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1a6c0b3ae0 con 0x7f1a6c1440f0 2023-11-24T07:59:10.145 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:10.141+0000 7f1a6b7fe700 1 --2- 172.21.15.59:0/4098188658 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1a6c1440f0 0x7f1a6c071bb0 secure :-1 s=READY pgs=838 cs=0 l=1 rev1=1 crypto rx=0x7f1a5c0098f0 tx=0x7f1a5c009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=305a4ff73107374b server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:59:10.145 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:10.141+0000 7f1a6affd700 1 -- 172.21.15.59:0/4098188658 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a5c011830 con 0x7f1a6c1440f0 2023-11-24T07:59:10.145 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:10.141+0000 7f1a6affd700 1 -- 172.21.15.59:0/4098188658 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1a5c011990 con 0x7f1a6c1440f0 2023-11-24T07:59:10.145 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:10.141+0000 7f1a6affd700 1 -- 172.21.15.59:0/4098188658 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a5c011d20 con 0x7f1a6c1440f0 2023-11-24T07:59:10.145 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:10.142+0000 7f1a7255b700 1 -- 172.21.15.59:0/4098188658 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1a6c1440f0 msgr2=0x7f1a6c071bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:10.145 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:10.142+0000 7f1a7255b700 1 --2- 172.21.15.59:0/4098188658 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1a6c1440f0 0x7f1a6c071bb0 secure :-1 s=READY pgs=838 cs=0 l=1 rev1=1 crypto rx=0x7f1a5c0098f0 tx=0x7f1a5c009c00 comp rx=0 tx=0).stop 2023-11-24T07:59:10.145 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:10.142+0000 7f1a7255b700 1 -- 172.21.15.59:0/4098188658 shutdown_connections 2023-11-24T07:59:10.145 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:10.142+0000 7f1a7255b700 1 --2- 172.21.15.59:0/4098188658 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f1a6c143ae0 0x7f1a6c0b51b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:10.145 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:10.142+0000 7f1a7255b700 1 --2- 172.21.15.59:0/4098188658 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1a6c1440f0 0x7f1a6c071bb0 unknown :-1 s=CLOSED pgs=838 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:10.145 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:10.142+0000 7f1a7255b700 1 -- 172.21.15.59:0/4098188658 >> 172.21.15.59:0/4098188658 conn(0x7f1a6c0b2740 msgr2=0x7f1a6c0b2b20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:59:10.145 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:10.142+0000 7f1a7255b700 1 -- 172.21.15.59:0/4098188658 shutdown_connections 2023-11-24T07:59:10.146 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:10.142+0000 7f1a7255b700 1 -- 172.21.15.59:0/4098188658 wait complete. 2023-11-24T07:59:10.148 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:10.143+0000 7f1a7255b700 1 Processor -- start 2023-11-24T07:59:10.148 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:10.143+0000 7f1a7255b700 1 -- start start 2023-11-24T07:59:10.148 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:10.144+0000 7f1a7255b700 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f1a6c0720f0 0x7f1a6c0cc630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:10.148 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:10.144+0000 7f1a7255b700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1a6c143ae0 0x7f1a6c132670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:10.148 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:10.144+0000 7f1a7255b700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1a6c1440f0 0x7f1a6c158c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:10.148 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:10.144+0000 7f1a7255b700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a6c0cc210 con 0x7f1a6c1440f0 2023-11-24T07:59:10.148 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:10.144+0000 7f1a7255b700 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a6c132bb0 con 0x7f1a6c143ae0 2023-11-24T07:59:10.148 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:10.144+0000 7f1a7255b700 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1a6c0c9390 con 0x7f1a6c0720f0 2023-11-24T07:59:10.148 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:10.144+0000 7f1a70af8700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1a6c1440f0 0x7f1a6c158c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:10.148 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:10.144+0000 7f1a6b7fe700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1a6c143ae0 0x7f1a6c132670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:10.149 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:10.144+0000 7f1a70af8700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1a6c1440f0 0x7f1a6c158c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:54670/0 (socket says 172.21.15.59:54670) 2023-11-24T07:59:10.149 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:10.144+0000 7f1a6bfff700 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f1a6c0720f0 0x7f1a6c0cc630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:10.149 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:10.144+0000 7f1a70af8700 1 -- 172.21.15.59:0/594545827 learned_addr learned my addr 172.21.15.59:0/594545827 (peer_addr_for_me v2:172.21.15.59:0/0) 2023-11-24T07:59:10.149 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:10.144+0000 7f1a70af8700 1 -- 172.21.15.59:0/594545827 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f1a6c0720f0 msgr2=0x7f1a6c0cc630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:10.149 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:10.144+0000 7f1a70af8700 1 --2- 172.21.15.59:0/594545827 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f1a6c0720f0 0x7f1a6c0cc630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:10.149 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:10.144+0000 7f1a70af8700 1 -- 172.21.15.59:0/594545827 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1a6c143ae0 msgr2=0x7f1a6c132670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:10.149 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:10.144+0000 7f1a70af8700 1 --2- 172.21.15.59:0/594545827 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1a6c143ae0 0x7f1a6c132670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:10.149 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:10.144+0000 7f1a70af8700 1 -- 172.21.15.59:0/594545827 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1a5c009550 con 0x7f1a6c1440f0 2023-11-24T07:59:10.149 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:10.144+0000 7f1a70af8700 1 --2- 172.21.15.59:0/594545827 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1a6c1440f0 0x7f1a6c158c60 secure :-1 s=READY pgs=839 cs=0 l=1 rev1=1 crypto rx=0x7f1a6400cb50 tx=0x7f1a6400ce60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:59:10.149 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:10.144+0000 7f1a68ff9700 1 -- 172.21.15.59:0/594545827 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a64003c00 con 0x7f1a6c1440f0 2023-11-24T07:59:10.149 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:10.145+0000 7f1a68ff9700 1 -- 172.21.15.59:0/594545827 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1a64003d60 con 0x7f1a6c1440f0 2023-11-24T07:59:10.149 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:10.145+0000 7f1a7255b700 1 -- 172.21.15.59:0/594545827 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1a6c0cb590 con 0x7f1a6c1440f0 2023-11-24T07:59:10.149 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:10.145+0000 7f1a68ff9700 1 -- 172.21.15.59:0/594545827 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a6400aa20 con 0x7f1a6c1440f0 2023-11-24T07:59:10.150 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:10.145+0000 7f1a7255b700 1 -- 172.21.15.59:0/594545827 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1a6c0cbb50 con 0x7f1a6c1440f0 2023-11-24T07:59:10.150 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:10.145+0000 7f1a7255b700 1 -- 172.21.15.59:0/594545827 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1a6c0cc350 con 0x7f1a6c1440f0 2023-11-24T07:59:10.150 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:10.146+0000 7f1a68ff9700 1 -- 172.21.15.59:0/594545827 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 10) v1 ==== 89588+0+0 (secure 0 0 0) 0x7f1a6400ab80 con 0x7f1a6c1440f0 2023-11-24T07:59:10.150 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:10.146+0000 7f1a68ff9700 1 --2- 172.21.15.59:0/594545827 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f1a4c06b2c0 0x7f1a4c06d780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:10.150 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:10.146+0000 7f1a68ff9700 1 -- 172.21.15.59:0/594545827 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(70..70 src has 1..70) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f1a64089980 con 0x7f1a6c1440f0 2023-11-24T07:59:10.150 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:10.147+0000 7f1a6bfff700 1 --2- 172.21.15.59:0/594545827 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f1a4c06b2c0 0x7f1a4c06d780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:10.150 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:10.147+0000 7f1a6bfff700 1 --2- 172.21.15.59:0/594545827 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f1a4c06b2c0 0x7f1a4c06d780 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f1a54009be0 tx=0x7f1a54009210 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:59:10.152 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:10.151+0000 7f1a68ff9700 1 -- 172.21.15.59:0/594545827 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f1a6c0cc350 con 0x7f1a6c1440f0 2023-11-24T07:59:10.437 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:10.435+0000 7f1a7255b700 1 -- 172.21.15.59:0/594545827 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7f1a6c144830 con 0x7f1a6c1440f0 2023-11-24T07:59:10.437 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:10.436+0000 7f1a68ff9700 1 -- 172.21.15.59:0/594545827 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v15) v1 ==== 80+0+5304 (secure 0 0 0) 0x7f1a64058280 con 0x7f1a6c1440f0 2023-11-24T07:59:10.438 INFO:teuthology.orchestra.run.smithi059.stdout: 2023-11-24T07:59:10.438 INFO:teuthology.orchestra.run.smithi059.stdout:{ 2023-11-24T07:59:10.438 INFO:teuthology.orchestra.run.smithi059.stdout: "auth_dump": [ 2023-11-24T07:59:10.438 INFO:teuthology.orchestra.run.smithi059.stdout: { 2023-11-24T07:59:10.439 INFO:teuthology.orchestra.run.smithi059.stdout: "entity": "mds.a", 2023-11-24T07:59:10.439 INFO:teuthology.orchestra.run.smithi059.stdout: "key": "AQD0VmBlWGLCKBAASbEQYAkExr8TButtK4KHQQ==", 2023-11-24T07:59:10.439 INFO:teuthology.orchestra.run.smithi059.stdout: "caps": { 2023-11-24T07:59:10.439 INFO:teuthology.orchestra.run.smithi059.stdout: "mds": "allow", 2023-11-24T07:59:10.439 INFO:teuthology.orchestra.run.smithi059.stdout: "mgr": "allow *", 2023-11-24T07:59:10.439 INFO:teuthology.orchestra.run.smithi059.stdout: "mon": "allow *", 2023-11-24T07:59:10.439 INFO:teuthology.orchestra.run.smithi059.stdout: "osd": "allow *" 2023-11-24T07:59:10.439 INFO:teuthology.orchestra.run.smithi059.stdout: } 2023-11-24T07:59:10.439 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-11-24T07:59:10.439 INFO:teuthology.orchestra.run.smithi059.stdout: { 2023-11-24T07:59:10.439 INFO:teuthology.orchestra.run.smithi059.stdout: "entity": "mds.b", 2023-11-24T07:59:10.439 INFO:teuthology.orchestra.run.smithi059.stdout: "key": "AQD0VmBlDIS0NRAAYQ3wTunU7m/Qk1qCM6Mcvg==", 2023-11-24T07:59:10.440 INFO:teuthology.orchestra.run.smithi059.stdout: "caps": { 2023-11-24T07:59:10.440 INFO:teuthology.orchestra.run.smithi059.stdout: "mds": "allow", 2023-11-24T07:59:10.440 INFO:teuthology.orchestra.run.smithi059.stdout: "mgr": "allow *", 2023-11-24T07:59:10.440 INFO:teuthology.orchestra.run.smithi059.stdout: "mon": "allow *", 2023-11-24T07:59:10.440 INFO:teuthology.orchestra.run.smithi059.stdout: "osd": "allow *" 2023-11-24T07:59:10.440 INFO:teuthology.orchestra.run.smithi059.stdout: } 2023-11-24T07:59:10.440 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-11-24T07:59:10.440 INFO:teuthology.orchestra.run.smithi059.stdout: { 2023-11-24T07:59:10.440 INFO:teuthology.orchestra.run.smithi059.stdout: "entity": "mds.c", 2023-11-24T07:59:10.440 INFO:teuthology.orchestra.run.smithi059.stdout: "key": "AQD0VmBlvpsrMBAAr5lh7nb6B1xRfk0Umm7Xig==", 2023-11-24T07:59:10.441 INFO:teuthology.orchestra.run.smithi059.stdout: "caps": { 2023-11-24T07:59:10.441 INFO:teuthology.orchestra.run.smithi059.stdout: "mds": "allow", 2023-11-24T07:59:10.441 INFO:teuthology.orchestra.run.smithi059.stdout: "mgr": "allow *", 2023-11-24T07:59:10.441 INFO:teuthology.orchestra.run.smithi059.stdout: "mon": "allow *", 2023-11-24T07:59:10.441 INFO:teuthology.orchestra.run.smithi059.stdout: "osd": "allow *" 2023-11-24T07:59:10.441 INFO:teuthology.orchestra.run.smithi059.stdout: } 2023-11-24T07:59:10.441 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-11-24T07:59:10.441 INFO:teuthology.orchestra.run.smithi059.stdout: { 2023-11-24T07:59:10.441 INFO:teuthology.orchestra.run.smithi059.stdout: "entity": "mds.d", 2023-11-24T07:59:10.441 INFO:teuthology.orchestra.run.smithi059.stdout: "key": "AQD1VmBlPTG7ARAAvC60Rk2GarnNtRJ6MMRIMw==", 2023-11-24T07:59:10.441 INFO:teuthology.orchestra.run.smithi059.stdout: "caps": { 2023-11-24T07:59:10.441 INFO:teuthology.orchestra.run.smithi059.stdout: "mds": "allow", 2023-11-24T07:59:10.442 INFO:teuthology.orchestra.run.smithi059.stdout: "mgr": "allow *", 2023-11-24T07:59:10.442 INFO:teuthology.orchestra.run.smithi059.stdout: "mon": "allow *", 2023-11-24T07:59:10.442 INFO:teuthology.orchestra.run.smithi059.stdout: "osd": "allow *" 2023-11-24T07:59:10.442 INFO:teuthology.orchestra.run.smithi059.stdout: } 2023-11-24T07:59:10.442 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-11-24T07:59:10.442 INFO:teuthology.orchestra.run.smithi059.stdout: { 2023-11-24T07:59:10.442 INFO:teuthology.orchestra.run.smithi059.stdout: "entity": "osd.0", 2023-11-24T07:59:10.442 INFO:teuthology.orchestra.run.smithi059.stdout: "key": "AQD4VmBlXjdAHRAAjnK2Y/rQhg4a97faNqVuoA==", 2023-11-24T07:59:10.442 INFO:teuthology.orchestra.run.smithi059.stdout: "caps": { 2023-11-24T07:59:10.442 INFO:teuthology.orchestra.run.smithi059.stdout: "mgr": "allow profile osd", 2023-11-24T07:59:10.442 INFO:teuthology.orchestra.run.smithi059.stdout: "mon": "allow profile osd", 2023-11-24T07:59:10.443 INFO:teuthology.orchestra.run.smithi059.stdout: "osd": "allow *" 2023-11-24T07:59:10.443 INFO:teuthology.orchestra.run.smithi059.stdout: } 2023-11-24T07:59:10.443 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-11-24T07:59:10.443 INFO:teuthology.orchestra.run.smithi059.stdout: { 2023-11-24T07:59:10.443 INFO:teuthology.orchestra.run.smithi059.stdout: "entity": "osd.1", 2023-11-24T07:59:10.443 INFO:teuthology.orchestra.run.smithi059.stdout: "key": "AQD7VmBlBwTwHBAAbpMH9rzS95ifrq/gVHnYRA==", 2023-11-24T07:59:10.443 INFO:teuthology.orchestra.run.smithi059.stdout: "caps": { 2023-11-24T07:59:10.443 INFO:teuthology.orchestra.run.smithi059.stdout: "mgr": "allow profile osd", 2023-11-24T07:59:10.443 INFO:teuthology.orchestra.run.smithi059.stdout: "mon": "allow profile osd", 2023-11-24T07:59:10.444 INFO:teuthology.orchestra.run.smithi059.stdout: "osd": "allow *" 2023-11-24T07:59:10.444 INFO:teuthology.orchestra.run.smithi059.stdout: } 2023-11-24T07:59:10.444 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-11-24T07:59:10.444 INFO:teuthology.orchestra.run.smithi059.stdout: { 2023-11-24T07:59:10.444 INFO:teuthology.orchestra.run.smithi059.stdout: "entity": "osd.2", 2023-11-24T07:59:10.444 INFO:teuthology.orchestra.run.smithi059.stdout: "key": "AQD+VmBl0tijHBAAWF2PsV3iH5adOTvHVPFIKw==", 2023-11-24T07:59:10.444 INFO:teuthology.orchestra.run.smithi059.stdout: "caps": { 2023-11-24T07:59:10.444 INFO:teuthology.orchestra.run.smithi059.stdout: "mgr": "allow profile osd", 2023-11-24T07:59:10.444 INFO:teuthology.orchestra.run.smithi059.stdout: "mon": "allow profile osd", 2023-11-24T07:59:10.445 INFO:teuthology.orchestra.run.smithi059.stdout: "osd": "allow *" 2023-11-24T07:59:10.445 INFO:teuthology.orchestra.run.smithi059.stdout: } 2023-11-24T07:59:10.445 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-11-24T07:59:10.445 INFO:teuthology.orchestra.run.smithi059.stdout: { 2023-11-24T07:59:10.445 INFO:teuthology.orchestra.run.smithi059.stdout: "entity": "osd.3", 2023-11-24T07:59:10.446 INFO:teuthology.orchestra.run.smithi059.stdout: "key": "AQABV2Bl3i8/HBAA4waLTD/PSLuNEg1VBmhEmg==", 2023-11-24T07:59:10.446 INFO:teuthology.orchestra.run.smithi059.stdout: "caps": { 2023-11-24T07:59:10.446 INFO:teuthology.orchestra.run.smithi059.stdout: "mgr": "allow profile osd", 2023-11-24T07:59:10.446 INFO:teuthology.orchestra.run.smithi059.stdout: "mon": "allow profile osd", 2023-11-24T07:59:10.446 INFO:teuthology.orchestra.run.smithi059.stdout: "osd": "allow *" 2023-11-24T07:59:10.447 INFO:teuthology.orchestra.run.smithi059.stdout: } 2023-11-24T07:59:10.447 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-11-24T07:59:10.447 INFO:teuthology.orchestra.run.smithi059.stdout: { 2023-11-24T07:59:10.447 INFO:teuthology.orchestra.run.smithi059.stdout: "entity": "osd.4", 2023-11-24T07:59:10.447 INFO:teuthology.orchestra.run.smithi059.stdout: "key": "AQAHV2Blr2zrHxAALOr+kFGdOTr8uQ9+/G2WUg==", 2023-11-24T07:59:10.447 INFO:teuthology.orchestra.run.smithi059.stdout: "caps": { 2023-11-24T07:59:10.448 INFO:teuthology.orchestra.run.smithi059.stdout: "mgr": "allow profile osd", 2023-11-24T07:59:10.448 INFO:teuthology.orchestra.run.smithi059.stdout: "mon": "allow profile osd", 2023-11-24T07:59:10.448 INFO:teuthology.orchestra.run.smithi059.stdout: "osd": "allow *" 2023-11-24T07:59:10.448 INFO:teuthology.orchestra.run.smithi059.stdout: } 2023-11-24T07:59:10.448 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-11-24T07:59:10.448 INFO:teuthology.orchestra.run.smithi059.stdout: { 2023-11-24T07:59:10.448 INFO:teuthology.orchestra.run.smithi059.stdout: "entity": "osd.5", 2023-11-24T07:59:10.448 INFO:teuthology.orchestra.run.smithi059.stdout: "key": "AQAKV2BlDZ3bHxAA7ZPPoxi4zVN0VECYzTsAsg==", 2023-11-24T07:59:10.448 INFO:teuthology.orchestra.run.smithi059.stdout: "caps": { 2023-11-24T07:59:10.448 INFO:teuthology.orchestra.run.smithi059.stdout: "mgr": "allow profile osd", 2023-11-24T07:59:10.448 INFO:teuthology.orchestra.run.smithi059.stdout: "mon": "allow profile osd", 2023-11-24T07:59:10.449 INFO:teuthology.orchestra.run.smithi059.stdout: "osd": "allow *" 2023-11-24T07:59:10.449 INFO:teuthology.orchestra.run.smithi059.stdout: } 2023-11-24T07:59:10.449 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-11-24T07:59:10.449 INFO:teuthology.orchestra.run.smithi059.stdout: { 2023-11-24T07:59:10.449 INFO:teuthology.orchestra.run.smithi059.stdout: "entity": "osd.6", 2023-11-24T07:59:10.449 INFO:teuthology.orchestra.run.smithi059.stdout: "key": "AQANV2BlfDMrHxAAVtm8+kh5FNTd1mJOKg02gg==", 2023-11-24T07:59:10.449 INFO:teuthology.orchestra.run.smithi059.stdout: "caps": { 2023-11-24T07:59:10.449 INFO:teuthology.orchestra.run.smithi059.stdout: "mgr": "allow profile osd", 2023-11-24T07:59:10.449 INFO:teuthology.orchestra.run.smithi059.stdout: "mon": "allow profile osd", 2023-11-24T07:59:10.449 INFO:teuthology.orchestra.run.smithi059.stdout: "osd": "allow *" 2023-11-24T07:59:10.449 INFO:teuthology.orchestra.run.smithi059.stdout: } 2023-11-24T07:59:10.449 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-11-24T07:59:10.450 INFO:teuthology.orchestra.run.smithi059.stdout: { 2023-11-24T07:59:10.450 INFO:teuthology.orchestra.run.smithi059.stdout: "entity": "osd.7", 2023-11-24T07:59:10.450 INFO:teuthology.orchestra.run.smithi059.stdout: "key": "AQAQV2Bli+slHxAAHY4sa84El5NC4KVgPKHs6w==", 2023-11-24T07:59:10.450 INFO:teuthology.orchestra.run.smithi059.stdout: "caps": { 2023-11-24T07:59:10.450 INFO:teuthology.orchestra.run.smithi059.stdout: "mgr": "allow profile osd", 2023-11-24T07:59:10.450 INFO:teuthology.orchestra.run.smithi059.stdout: "mon": "allow profile osd", 2023-11-24T07:59:10.450 INFO:teuthology.orchestra.run.smithi059.stdout: "osd": "allow *" 2023-11-24T07:59:10.450 INFO:teuthology.orchestra.run.smithi059.stdout: } 2023-11-24T07:59:10.450 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-11-24T07:59:10.450 INFO:teuthology.orchestra.run.smithi059.stdout: { 2023-11-24T07:59:10.450 INFO:teuthology.orchestra.run.smithi059.stdout: "entity": "client.0", 2023-11-24T07:59:10.450 INFO:teuthology.orchestra.run.smithi059.stdout: "key": "AQD1VmBlxPkpBhAAR841hRsF6QtmzFgJIrVZ9w==", 2023-11-24T07:59:10.451 INFO:teuthology.orchestra.run.smithi059.stdout: "caps": { 2023-11-24T07:59:10.451 INFO:teuthology.orchestra.run.smithi059.stdout: "mds": "allow", 2023-11-24T07:59:10.451 INFO:teuthology.orchestra.run.smithi059.stdout: "mon": "allow r", 2023-11-24T07:59:10.451 INFO:teuthology.orchestra.run.smithi059.stdout: "osd": "allow rw tag cephfs data=cephfs" 2023-11-24T07:59:10.451 INFO:teuthology.orchestra.run.smithi059.stdout: } 2023-11-24T07:59:10.451 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-11-24T07:59:10.451 INFO:teuthology.orchestra.run.smithi059.stdout: { 2023-11-24T07:59:10.451 INFO:teuthology.orchestra.run.smithi059.stdout: "entity": "client.1", 2023-11-24T07:59:10.451 INFO:teuthology.orchestra.run.smithi059.stdout: "key": "AQD1VmBl46HpDBAAnO9MGWrJczW0epSp0eCkLg==", 2023-11-24T07:59:10.451 INFO:teuthology.orchestra.run.smithi059.stdout: "caps": { 2023-11-24T07:59:10.451 INFO:teuthology.orchestra.run.smithi059.stdout: "mds": "allow", 2023-11-24T07:59:10.451 INFO:teuthology.orchestra.run.smithi059.stdout: "mgr": "allow r", 2023-11-24T07:59:10.451 INFO:teuthology.orchestra.run.smithi059.stdout: "mon": "allow rw", 2023-11-24T07:59:10.452 INFO:teuthology.orchestra.run.smithi059.stdout: "osd": "allow rwx" 2023-11-24T07:59:10.452 INFO:teuthology.orchestra.run.smithi059.stdout: } 2023-11-24T07:59:10.452 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-11-24T07:59:10.452 INFO:teuthology.orchestra.run.smithi059.stdout: { 2023-11-24T07:59:10.452 INFO:teuthology.orchestra.run.smithi059.stdout: "entity": "client.2", 2023-11-24T07:59:10.452 INFO:teuthology.orchestra.run.smithi059.stdout: "key": "AQD1VmBlgQSRCRAAcHS1eQCYMJ3YP21jUEPJ0A==", 2023-11-24T07:59:10.452 INFO:teuthology.orchestra.run.smithi059.stdout: "caps": { 2023-11-24T07:59:10.452 INFO:teuthology.orchestra.run.smithi059.stdout: "mds": "allow", 2023-11-24T07:59:10.452 INFO:teuthology.orchestra.run.smithi059.stdout: "mgr": "allow r", 2023-11-24T07:59:10.452 INFO:teuthology.orchestra.run.smithi059.stdout: "mon": "allow rw", 2023-11-24T07:59:10.452 INFO:teuthology.orchestra.run.smithi059.stdout: "osd": "allow rwx" 2023-11-24T07:59:10.453 INFO:teuthology.orchestra.run.smithi059.stdout: } 2023-11-24T07:59:10.453 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-11-24T07:59:10.453 INFO:teuthology.orchestra.run.smithi059.stdout: { 2023-11-24T07:59:10.453 INFO:teuthology.orchestra.run.smithi059.stdout: "entity": "client.3", 2023-11-24T07:59:10.453 INFO:teuthology.orchestra.run.smithi059.stdout: "key": "AQD1VmBl7gmhEhAA6uWs7XAbOQwJItA6To549w==", 2023-11-24T07:59:10.453 INFO:teuthology.orchestra.run.smithi059.stdout: "caps": { 2023-11-24T07:59:10.453 INFO:teuthology.orchestra.run.smithi059.stdout: "mds": "allow", 2023-11-24T07:59:10.453 INFO:teuthology.orchestra.run.smithi059.stdout: "mgr": "allow r", 2023-11-24T07:59:10.453 INFO:teuthology.orchestra.run.smithi059.stdout: "mon": "allow rw", 2023-11-24T07:59:10.453 INFO:teuthology.orchestra.run.smithi059.stdout: "osd": "allow rwx" 2023-11-24T07:59:10.453 INFO:teuthology.orchestra.run.smithi059.stdout: } 2023-11-24T07:59:10.454 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-11-24T07:59:10.454 INFO:teuthology.orchestra.run.smithi059.stdout: { 2023-11-24T07:59:10.454 INFO:teuthology.orchestra.run.smithi059.stdout: "entity": "client.admin", 2023-11-24T07:59:10.454 INFO:teuthology.orchestra.run.smithi059.stdout: "key": "AQD0VmBlAECpCRAA1CkLlkyjnOovaNUGDSWhaw==", 2023-11-24T07:59:10.454 INFO:teuthology.orchestra.run.smithi059.stdout: "caps": { 2023-11-24T07:59:10.454 INFO:teuthology.orchestra.run.smithi059.stdout: "mds": "allow *", 2023-11-24T07:59:10.454 INFO:teuthology.orchestra.run.smithi059.stdout: "mgr": "allow *", 2023-11-24T07:59:10.454 INFO:teuthology.orchestra.run.smithi059.stdout: "mon": "allow *", 2023-11-24T07:59:10.454 INFO:teuthology.orchestra.run.smithi059.stdout: "osd": "allow *" 2023-11-24T07:59:10.455 INFO:teuthology.orchestra.run.smithi059.stdout: } 2023-11-24T07:59:10.455 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-11-24T07:59:10.455 INFO:teuthology.orchestra.run.smithi059.stdout: { 2023-11-24T07:59:10.455 INFO:teuthology.orchestra.run.smithi059.stdout: "entity": "mgr.x", 2023-11-24T07:59:10.455 INFO:teuthology.orchestra.run.smithi059.stdout: "key": "AQD0VmBlryM+JRAAf72PlDWEHZOwNxhW4k/8Ig==", 2023-11-24T07:59:10.455 INFO:teuthology.orchestra.run.smithi059.stdout: "caps": { 2023-11-24T07:59:10.455 INFO:teuthology.orchestra.run.smithi059.stdout: "mds": "allow *", 2023-11-24T07:59:10.455 INFO:teuthology.orchestra.run.smithi059.stdout: "mon": "allow profile mgr", 2023-11-24T07:59:10.455 INFO:teuthology.orchestra.run.smithi059.stdout: "osd": "allow *" 2023-11-24T07:59:10.455 INFO:teuthology.orchestra.run.smithi059.stdout: } 2023-11-24T07:59:10.455 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-11-24T07:59:10.456 INFO:teuthology.orchestra.run.smithi059.stdout: { 2023-11-24T07:59:10.456 INFO:teuthology.orchestra.run.smithi059.stdout: "entity": "mgr.y", 2023-11-24T07:59:10.456 INFO:teuthology.orchestra.run.smithi059.stdout: "key": "AQD0VmBlpXXaHxAAJMwwht1n+zRbRZuGOD5WTQ==", 2023-11-24T07:59:10.456 INFO:teuthology.orchestra.run.smithi059.stdout: "caps": { 2023-11-24T07:59:10.456 INFO:teuthology.orchestra.run.smithi059.stdout: "mds": "allow *", 2023-11-24T07:59:10.456 INFO:teuthology.orchestra.run.smithi059.stdout: "mon": "allow profile mgr", 2023-11-24T07:59:10.456 INFO:teuthology.orchestra.run.smithi059.stdout: "osd": "allow *" 2023-11-24T07:59:10.456 INFO:teuthology.orchestra.run.smithi059.stdout: } 2023-11-24T07:59:10.456 INFO:teuthology.orchestra.run.smithi059.stdout: } 2023-11-24T07:59:10.456 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2023-11-24T07:59:10.456 INFO:teuthology.orchestra.run.smithi059.stdout:} 2023-11-24T07:59:10.458 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:10.438+0000 7f1a7255b700 1 -- 172.21.15.59:0/594545827 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f1a4c06b2c0 msgr2=0x7f1a4c06d780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:10.458 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:10.438+0000 7f1a7255b700 1 --2- 172.21.15.59:0/594545827 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f1a4c06b2c0 0x7f1a4c06d780 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f1a54009be0 tx=0x7f1a54009210 comp rx=0 tx=0).stop 2023-11-24T07:59:10.458 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:10.438+0000 7f1a7255b700 1 -- 172.21.15.59:0/594545827 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1a6c1440f0 msgr2=0x7f1a6c158c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:10.458 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:10.438+0000 7f1a7255b700 1 --2- 172.21.15.59:0/594545827 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1a6c1440f0 0x7f1a6c158c60 secure :-1 s=READY pgs=839 cs=0 l=1 rev1=1 crypto rx=0x7f1a6400cb50 tx=0x7f1a6400ce60 comp rx=0 tx=0).stop 2023-11-24T07:59:10.458 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:10.438+0000 7f1a7255b700 1 -- 172.21.15.59:0/594545827 shutdown_connections 2023-11-24T07:59:10.458 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:10.438+0000 7f1a7255b700 1 --2- 172.21.15.59:0/594545827 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f1a4c06b2c0 0x7f1a4c06d780 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:10.458 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:10.438+0000 7f1a7255b700 1 --2- 172.21.15.59:0/594545827 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1a6c1440f0 0x7f1a6c158c60 unknown :-1 s=CLOSED pgs=839 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:10.459 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:10.438+0000 7f1a7255b700 1 --2- 172.21.15.59:0/594545827 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f1a6c0720f0 0x7f1a6c0cc630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:10.459 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:10.438+0000 7f1a7255b700 1 --2- 172.21.15.59:0/594545827 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1a6c143ae0 0x7f1a6c132670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:10.459 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:10.438+0000 7f1a7255b700 1 -- 172.21.15.59:0/594545827 >> 172.21.15.59:0/594545827 conn(0x7f1a6c0b2740 msgr2=0x7f1a6c0cf040 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:59:10.459 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:10.438+0000 7f1a7255b700 1 -- 172.21.15.59:0/594545827 shutdown_connections 2023-11-24T07:59:10.459 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:10.438+0000 7f1a7255b700 1 -- 172.21.15.59:0/594545827 wait complete. 2023-11-24T07:59:10.459 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-24T07:59:10.584 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:10.582+0000 7fcd3472f700 1 Processor -- start 2023-11-24T07:59:10.584 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:10.582+0000 7fcd3472f700 1 -- start start 2023-11-24T07:59:10.586 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:10.583+0000 7fcd3472f700 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fcd2c1456c0 0x7fcd2c06f6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:10.587 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:10.583+0000 7fcd3472f700 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fcd2c06e520 0x7fcd2c06e9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:10.587 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:10.583+0000 7fcd3472f700 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcd2c0cdb30 con 0x7fcd2c06b9e0 2023-11-24T07:59:10.587 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:10.583+0000 7fcd3472f700 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcd2c06eef0 con 0x7fcd2c1456c0 2023-11-24T07:59:10.587 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:10.583+0000 7fcd3472f700 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcd2c06e090 con 0x7fcd2c06e520 2023-11-24T07:59:10.587 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:10.584+0000 7fcd32ccc700 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fcd2c06e520 0x7fcd2c06e9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:10.587 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:10.584+0000 7fcd32ccc700 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fcd2c06e520 0x7fcd2c06e9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3301/0 says I am v2:172.21.15.59:54118/0 (socket says 172.21.15.59:54118) 2023-11-24T07:59:10.587 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:10.584+0000 7fcd324cb700 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fcd2c1456c0 0x7fcd2c06f6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:10.587 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:10.584+0000 7fcd32ccc700 1 -- 172.21.15.59:0/649099084 learned_addr learned my addr 172.21.15.59:0/649099084 (peer_addr_for_me v2:172.21.15.59:0/0) 2023-11-24T07:59:10.587 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:10.584+0000 7fcd32ccc700 1 -- 172.21.15.59:0/649099084 >> v2:172.21.15.76:3300/0 conn(0x7fcd2c1456c0 msgr2=0x7fcd2c06f6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T07:59:10.587 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:10.584+0000 7fcd32ccc700 1 --2- 172.21.15.59:0/649099084 >> v2:172.21.15.76:3300/0 conn(0x7fcd2c1456c0 0x7fcd2c06f6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:10.587 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:10.584+0000 7fcd32ccc700 1 -- 172.21.15.59:0/649099084 >> v1:172.21.15.76:6789/0 conn(0x7fcd2c06b9e0 legacy=0x7fcd2c0cf060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:10.588 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:10.584+0000 7fcd32ccc700 1 -- 172.21.15.59:0/649099084 --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcd2c145aa0 con 0x7fcd2c06e520 2023-11-24T07:59:10.588 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:10.584+0000 7fcd32ccc700 1 --2- 172.21.15.59:0/649099084 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fcd2c06e520 0x7fcd2c06e9b0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fcd240098f0 tx=0x7fcd24009c00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f056bc875b8f9e55 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:59:10.588 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:10.584+0000 7fcd314c9700 1 -- 172.21.15.59:0/649099084 <== mon.2 v2:172.21.15.76:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd24011830 con 0x7fcd2c06e520 2023-11-24T07:59:10.588 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:10.584+0000 7fcd314c9700 1 -- 172.21.15.59:0/649099084 <== mon.2 v2:172.21.15.76:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcd24011990 con 0x7fcd2c06e520 2023-11-24T07:59:10.588 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:10.585+0000 7fcd314c9700 1 -- 172.21.15.59:0/649099084 <== mon.2 v2:172.21.15.76:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd24011d20 con 0x7fcd2c06e520 2023-11-24T07:59:10.588 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:10.585+0000 7fcd3472f700 1 -- 172.21.15.59:0/649099084 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fcd2c06e520 msgr2=0x7fcd2c06e9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:10.588 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:10.585+0000 7fcd3472f700 1 --2- 172.21.15.59:0/649099084 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fcd2c06e520 0x7fcd2c06e9b0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fcd240098f0 tx=0x7fcd24009c00 comp rx=0 tx=0).stop 2023-11-24T07:59:10.588 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:10.585+0000 7fcd3472f700 1 -- 172.21.15.59:0/649099084 shutdown_connections 2023-11-24T07:59:10.588 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:10.585+0000 7fcd3472f700 1 --2- 172.21.15.59:0/649099084 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fcd2c06e520 0x7fcd2c06e9b0 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:10.588 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:10.585+0000 7fcd3472f700 1 --2- 172.21.15.59:0/649099084 >> v2:172.21.15.76:3300/0 conn(0x7fcd2c1456c0 0x7fcd2c06f6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:10.588 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:10.585+0000 7fcd3472f700 1 -- 172.21.15.59:0/649099084 >> 172.21.15.59:0/649099084 conn(0x7fcd2c0b27d0 msgr2=0x7fcd2c0b2be0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:59:10.588 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:10.585+0000 7fcd3472f700 1 -- 172.21.15.59:0/649099084 shutdown_connections 2023-11-24T07:59:10.588 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:10.585+0000 7fcd3472f700 1 -- 172.21.15.59:0/649099084 wait complete. 2023-11-24T07:59:10.590 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:10.586+0000 7fcd3472f700 1 Processor -- start 2023-11-24T07:59:10.591 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:10.586+0000 7fcd3472f700 1 -- start start 2023-11-24T07:59:10.591 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:10.587+0000 7fcd3472f700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fcd2c06b9e0 0x7fcd2c1f38e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:10.591 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:10.587+0000 7fcd3472f700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fcd2c06e520 0x7fcd2c15acb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:10.591 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:10.587+0000 7fcd3472f700 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fcd2c1456c0 0x7fcd2c141f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:10.591 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:10.587+0000 7fcd3472f700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd2c15b1f0 con 0x7fcd2c06e520 2023-11-24T07:59:10.591 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:10.587+0000 7fcd3472f700 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd2c0caaa0 con 0x7fcd2c06b9e0 2023-11-24T07:59:10.591 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:10.587+0000 7fcd3472f700 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcd2c0cad80 con 0x7fcd2c1456c0 2023-11-24T07:59:10.591 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:10.587+0000 7fcd31cca700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fcd2c06e520 0x7fcd2c15acb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:10.591 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:10.587+0000 7fcd31cca700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fcd2c06e520 0x7fcd2c15acb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:54684/0 (socket says 172.21.15.59:54684) 2023-11-24T07:59:10.591 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:10.587+0000 7fcd31cca700 1 -- 172.21.15.59:0/1603559583 learned_addr learned my addr 172.21.15.59:0/1603559583 (peer_addr_for_me v2:172.21.15.59:0/0) 2023-11-24T07:59:10.591 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:10.587+0000 7fcd324cb700 1 --2- 172.21.15.59:0/1603559583 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fcd2c06b9e0 0x7fcd2c1f38e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:10.591 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:10.587+0000 7fcd32ccc700 1 --2- 172.21.15.59:0/1603559583 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fcd2c1456c0 0x7fcd2c141f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:10.591 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:10.588+0000 7fcd31cca700 1 -- 172.21.15.59:0/1603559583 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fcd2c1456c0 msgr2=0x7fcd2c141f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:10.592 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:10.588+0000 7fcd31cca700 1 --2- 172.21.15.59:0/1603559583 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fcd2c1456c0 0x7fcd2c141f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:10.592 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:10.588+0000 7fcd31cca700 1 -- 172.21.15.59:0/1603559583 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fcd2c06b9e0 msgr2=0x7fcd2c1f38e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:10.592 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:10.588+0000 7fcd31cca700 1 --2- 172.21.15.59:0/1603559583 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fcd2c06b9e0 0x7fcd2c1f38e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:10.592 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:10.588+0000 7fcd31cca700 1 -- 172.21.15.59:0/1603559583 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcd24009550 con 0x7fcd2c06e520 2023-11-24T07:59:10.592 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:10.588+0000 7fcd31cca700 1 --2- 172.21.15.59:0/1603559583 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fcd2c06e520 0x7fcd2c15acb0 secure :-1 s=READY pgs=840 cs=0 l=1 rev1=1 crypto rx=0x7fcd2000c4e0 tx=0x7fcd2000c7f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:59:10.592 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:10.588+0000 7fcd2affd700 1 -- 172.21.15.59:0/1603559583 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd200052d0 con 0x7fcd2c06e520 2023-11-24T07:59:10.592 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:10.588+0000 7fcd2affd700 1 -- 172.21.15.59:0/1603559583 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcd20005430 con 0x7fcd2c06e520 2023-11-24T07:59:10.592 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:10.588+0000 7fcd3472f700 1 -- 172.21.15.59:0/1603559583 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcd2c0cb060 con 0x7fcd2c06e520 2023-11-24T07:59:10.593 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:10.588+0000 7fcd2affd700 1 -- 172.21.15.59:0/1603559583 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd200038c0 con 0x7fcd2c06e520 2023-11-24T07:59:10.593 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:10.588+0000 7fcd3472f700 1 -- 172.21.15.59:0/1603559583 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcd2c0cb620 con 0x7fcd2c06e520 2023-11-24T07:59:10.593 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:10.589+0000 7fcd3472f700 1 -- 172.21.15.59:0/1603559583 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcd2c068670 con 0x7fcd2c06e520 2023-11-24T07:59:10.593 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:10.589+0000 7fcd2affd700 1 -- 172.21.15.59:0/1603559583 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 10) v1 ==== 89588+0+0 (secure 0 0 0) 0x7fcd20003a20 con 0x7fcd2c06e520 2023-11-24T07:59:10.594 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:10.590+0000 7fcd2affd700 1 --2- 172.21.15.59:0/1603559583 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7fcd1806b2c0 0x7fcd1806d780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:10.594 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:10.590+0000 7fcd2affd700 1 -- 172.21.15.59:0/1603559583 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(70..70 src has 1..70) v4 ==== 4857+0+0 (secure 0 0 0) 0x7fcd20088e40 con 0x7fcd2c06e520 2023-11-24T07:59:10.594 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:10.590+0000 7fcd324cb700 1 --2- 172.21.15.59:0/1603559583 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7fcd1806b2c0 0x7fcd1806d780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:10.594 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:10.590+0000 7fcd324cb700 1 --2- 172.21.15.59:0/1603559583 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7fcd1806b2c0 0x7fcd1806d780 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7fcd1c011340 tx=0x7fcd1c00b040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:59:10.596 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:10.595+0000 7fcd2affd700 1 -- 172.21.15.59:0/1603559583 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fcd20053ba0 con 0x7fcd2c06e520 2023-11-24T07:59:10.801 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:10.800+0000 7fcd3472f700 1 -- 172.21.15.59:0/1603559583 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fcd2c0b32f0 con 0x7fcd2c06e520 2023-11-24T07:59:10.802 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:10.801+0000 7fcd2affd700 1 -- 172.21.15.59:0/1603559583 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v70) v1 ==== 74+0+12785 (secure 0 0 0) 0x7fcd200571c0 con 0x7fcd2c06e520 2023-11-24T07:59:10.803 INFO:teuthology.orchestra.run.smithi059.stdout: 2023-11-24T07:59:10.803 INFO:teuthology.orchestra.run.smithi059.stdout:{"epoch":70,"fsid":"9f0e8741-b8f9-48d5-a2c3-09f45903303d","created":"2023-11-24T07:56:06.552729+0000","modified":"2023-11-24T07:59:09.987746+0000","last_up_change":"2023-11-24T07:56:17.372352+0000","last_in_change":"2023-11-24T07:56:09.073250+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-24T07:56:18.352823+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"6d8f9e9b-1782-4e41-928b-a39f2a8ebc06","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6800","nonce":3373878772},{"type":"v1","addr":"172.21.15.59:6801","nonce":3373878772}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6802","nonce":3373878772},{"type":"v1","addr":"172.21.15.59:6803","nonce":3373878772}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6806","nonce":3373878772},{"type":"v1","addr":"172.21.15.59:6807","nonce":3373878772}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6804","nonce":3373878772},{"type":"v1","addr":"172.21.15.59:6805","nonce":3373878772}]},"public_addr":"172.21.15.59:6801/3373878772","cluster_addr":"172.21.15.59:6803/3373878772","heartbeat_back_addr":"172.21.15.59:6807/3373878772","heartbeat_front_addr":"172.21.15.59:6805/3373878772","state":["exists","up"]},{"osd":1,"uuid":"e8d882aa-52e1-4c04-9e1e-c44ef9204a5f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6824","nonce":648258563},{"type":"v1","addr":"172.21.15.59:6825","nonce":648258563}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6826","nonce":648258563},{"type":"v1","addr":"172.21.15.59:6827","nonce":648258563}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6830","nonce":648258563},{"type":"v1","addr":"172.21.15.59:6831","nonce":648258563}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6828","nonce":648258563},{"type":"v1","addr":"172.21.15.59:6829","nonce":648258563}]},"public_addr":"172.21.15.59:6825/648258563","cluster_addr":"172.21.15.59:6827/648258563","heartbeat_back_addr":"172.21.15.59:6831/648258563","heartbeat_front_addr":"172.21.15.59:6829/648258563","state":["exists","up"]},{"osd":2,"uuid":"69eebfa0-695b-4b21-90b9-8a3537168a01","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6808","nonce":1840430654},{"type":"v1","addr":"172.21.15.59:6809","nonce":1840430654}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6810","nonce":1840430654},{"type":"v1","addr":"172.21.15.59:6811","nonce":1840430654}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6814","nonce":1840430654},{"type":"v1","addr":"172.21.15.59:6815","nonce":1840430654}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6812","nonce":1840430654},{"type":"v1","addr":"172.21.15.59:6813","nonce":1840430654}]},"public_addr":"172.21.15.59:6809/1840430654","cluster_addr":"172.21.15.59:6811/1840430654","heartbeat_back_addr":"172.21.15.59:6815/1840430654","heartbeat_front_addr":"172.21.15.59:6813/1840430654","state":["exists","up"]},{"osd":3,"uuid":"96fd77fc-f911-46ff-9445-6f0be2daa8f4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6816","nonce":2504025850},{"type":"v1","addr":"172.21.15.59:6817","nonce":2504025850}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6818","nonce":2504025850},{"type":"v1","addr":"172.21.15.59:6819","nonce":2504025850}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6822","nonce":2504025850},{"type":"v1","addr":"172.21.15.59:6823","nonce":2504025850}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6820","nonce":2504025850},{"type":"v1","addr":"172.21.15.59:6821","nonce":2504025850}]},"public_addr":"172.21.15.59:6817/2504025850","cluster_addr":"172.21.15.59:6819/2504025850","heartbeat_back_addr":"172.21.15.59:6823/2504025850","heartbeat_front_addr":"172.21.15.59:6821/2504025850","state":["exists","up"]},{"osd":4,"uuid":"7ac29fca-b5b2-4c95-9d69-2f164cbbc708","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6816","nonce":2382463431},{"type":"v1","addr":"172.21.15.76:6818","nonce":2382463431}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6820","nonce":2382463431},{"type":"v1","addr":"172.21.15.76:6822","nonce":2382463431}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6828","nonce":2382463431},{"type":"v1","addr":"172.21.15.76:6830","nonce":2382463431}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6824","nonce":2382463431},{"type":"v1","addr":"172.21.15.76:6826","nonce":2382463431}]},"public_addr":"172.21.15.76:6818/2382463431","cluster_addr":"172.21.15.76:6822/2382463431","heartbeat_back_addr":"172.21.15.76:6830/2382463431","heartbeat_front_addr":"172.21.15.76:6826/2382463431","state":["exists","up"]},{"osd":5,"uuid":"485bf711-3851-4238-a4f4-51d52e0a9c58","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6800","nonce":329334308},{"type":"v1","addr":"172.21.15.76:6802","nonce":329334308}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6805","nonce":329334308},{"type":"v1","addr":"172.21.15.76:6807","nonce":329334308}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6814","nonce":329334308},{"type":"v1","addr":"172.21.15.76:6815","nonce":329334308}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6810","nonce":329334308},{"type":"v1","addr":"172.21.15.76:6812","nonce":329334308}]},"public_addr":"172.21.15.76:6802/329334308","cluster_addr":"172.21.15.76:6807/329334308","heartbeat_back_addr":"172.21.15.76:6815/329334308","heartbeat_front_addr":"172.21.15.76:6812/329334308","state":["exists","up"]},{"osd":6,"uuid":"72b3f2fc-1e4a-4e32-968d-003647a0286f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6801","nonce":3946579410},{"type":"v1","addr":"172.21.15.76:6803","nonce":3946579410}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6804","nonce":3946579410},{"type":"v1","addr":"172.21.15.76:6806","nonce":3946579410}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6811","nonce":3946579410},{"type":"v1","addr":"172.21.15.76:6813","nonce":3946579410}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6808","nonce":3946579410},{"type":"v1","addr":"172.21.15.76:6809","nonce":3946579410}]},"public_addr":"172.21.15.76:6803/3946579410","cluster_addr":"172.21.15.76:6806/3946579410","heartbeat_back_addr":"172.21.15.76:6813/3946579410","heartbeat_front_addr":"172.21.15.76:6809/3946579410","state":["exists","up"]},{"osd":7,"uuid":"1ddceb90-6c5d-4d1a-ad56-99cb877f8c70","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6817","nonce":841761538},{"type":"v1","addr":"172.21.15.76:6819","nonce":841761538}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6821","nonce":841761538},{"type":"v1","addr":"172.21.15.76:6823","nonce":841761538}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6829","nonce":841761538},{"type":"v1","addr":"172.21.15.76:6831","nonce":841761538}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6825","nonce":841761538},{"type":"v1","addr":"172.21.15.76:6827","nonce":841761538}]},"public_addr":"172.21.15.76:6819/841761538","cluster_addr":"172.21.15.76:6823/841761538","heartbeat_back_addr":"172.21.15.76:6831/841761538","heartbeat_front_addr":"172.21.15.76:6827/841761538","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T07:56:14.294838+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T07:56:14.270779+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T07:56:14.231809+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T07:56:14.260983+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T07:56:14.302997+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T07:56:14.258600+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T07:56:14.251413+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T07:56:14.304727+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-24T07:59:10.805 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:10.803+0000 7fcd3472f700 1 -- 172.21.15.59:0/1603559583 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7fcd1806b2c0 msgr2=0x7fcd1806d780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:10.805 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:10.803+0000 7fcd3472f700 1 --2- 172.21.15.59:0/1603559583 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7fcd1806b2c0 0x7fcd1806d780 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7fcd1c011340 tx=0x7fcd1c00b040 comp rx=0 tx=0).stop 2023-11-24T07:59:10.805 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:10.803+0000 7fcd3472f700 1 -- 172.21.15.59:0/1603559583 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fcd2c06e520 msgr2=0x7fcd2c15acb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:10.805 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:10.803+0000 7fcd3472f700 1 --2- 172.21.15.59:0/1603559583 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fcd2c06e520 0x7fcd2c15acb0 secure :-1 s=READY pgs=840 cs=0 l=1 rev1=1 crypto rx=0x7fcd2000c4e0 tx=0x7fcd2000c7f0 comp rx=0 tx=0).stop 2023-11-24T07:59:10.805 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:10.803+0000 7fcd3472f700 1 -- 172.21.15.59:0/1603559583 shutdown_connections 2023-11-24T07:59:10.806 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:10.803+0000 7fcd3472f700 1 --2- 172.21.15.59:0/1603559583 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7fcd1806b2c0 0x7fcd1806d780 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:10.806 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:10.803+0000 7fcd3472f700 1 --2- 172.21.15.59:0/1603559583 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fcd2c1456c0 0x7fcd2c141f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:10.806 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:10.803+0000 7fcd3472f700 1 --2- 172.21.15.59:0/1603559583 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fcd2c06b9e0 0x7fcd2c1f38e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:10.806 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:10.803+0000 7fcd3472f700 1 --2- 172.21.15.59:0/1603559583 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fcd2c06e520 0x7fcd2c15acb0 unknown :-1 s=CLOSED pgs=840 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:10.806 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:10.803+0000 7fcd3472f700 1 -- 172.21.15.59:0/1603559583 >> 172.21.15.59:0/1603559583 conn(0x7fcd2c0b27d0 msgr2=0x7fcd2c0cfbe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:59:10.806 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:10.803+0000 7fcd3472f700 1 -- 172.21.15.59:0/1603559583 shutdown_connections 2023-11-24T07:59:10.806 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:10.803+0000 7fcd3472f700 1 -- 172.21.15.59:0/1603559583 wait complete. 2023-11-24T07:59:10.817 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-24T07:56:18.352823+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-24T07:59:10.818 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-24T07:59:10.945 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:10.944+0000 7f3c386a5700 1 Processor -- start 2023-11-24T07:59:10.946 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:10.944+0000 7f3c386a5700 1 -- start start 2023-11-24T07:59:10.948 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:10.945+0000 7f3c386a5700 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f3c300722c0 0x7f3c300b51b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:10.948 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:10.945+0000 7f3c386a5700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f3c301440f0 0x7f3c3006e510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:10.948 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:10.945+0000 7f3c386a5700 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3c30143ec0 con 0x7f3c30143ae0 2023-11-24T07:59:10.948 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:10.945+0000 7f3c386a5700 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3c300b40a0 con 0x7f3c301440f0 2023-11-24T07:59:10.948 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:10.945+0000 7f3c386a5700 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3c300b3ae0 con 0x7f3c300722c0 2023-11-24T07:59:10.948 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:10.945+0000 7f3c36c42700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f3c301440f0 0x7f3c3006e510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:10.948 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:10.945+0000 7f3c36c42700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f3c301440f0 0x7f3c3006e510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:54692/0 (socket says 172.21.15.59:54692) 2023-11-24T07:59:10.949 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:10.945+0000 7f3c36c42700 1 -- 172.21.15.59:0/4083428845 learned_addr learned my addr 172.21.15.59:0/4083428845 (peer_addr_for_me v2:172.21.15.59:0/0) 2023-11-24T07:59:10.949 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:10.945+0000 7f3c35c40700 1 --2- 172.21.15.59:0/4083428845 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f3c300722c0 0x7f3c300b51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:10.949 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:10.945+0000 7f3c36c42700 1 -- 172.21.15.59:0/4083428845 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f3c300722c0 msgr2=0x7f3c300b51b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T07:59:10.949 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:10.945+0000 7f3c36c42700 1 --2- 172.21.15.59:0/4083428845 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f3c300722c0 0x7f3c300b51b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:10.949 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:10.945+0000 7f3c36c42700 1 -- 172.21.15.59:0/4083428845 >> v1:172.21.15.76:6789/0 conn(0x7f3c30143ae0 legacy=0x7f3c30071bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:10.949 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:10.945+0000 7f3c36c42700 1 -- 172.21.15.59:0/4083428845 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3c300b3dc0 con 0x7f3c301440f0 2023-11-24T07:59:10.949 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:10.945+0000 7f3c35c40700 1 --2- 172.21.15.59:0/4083428845 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f3c300722c0 0x7f3c300b51b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-24T07:59:10.949 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:10.945+0000 7f3c36c42700 1 --2- 172.21.15.59:0/4083428845 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3c301440f0 0x7f3c3006e510 secure :-1 s=READY pgs=841 cs=0 l=1 rev1=1 crypto rx=0x7f3c28009e50 tx=0x7f3c2800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=720ec1168517a86 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:59:10.949 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:10.946+0000 7f3c3543f700 1 -- 172.21.15.59:0/4083428845 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c2800e070 con 0x7f3c301440f0 2023-11-24T07:59:10.949 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:10.946+0000 7f3c3543f700 1 -- 172.21.15.59:0/4083428845 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3c2800b9c0 con 0x7f3c301440f0 2023-11-24T07:59:10.949 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:10.946+0000 7f3c3543f700 1 -- 172.21.15.59:0/4083428845 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c2800bd70 con 0x7f3c301440f0 2023-11-24T07:59:10.949 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:10.946+0000 7f3c386a5700 1 -- 172.21.15.59:0/4083428845 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3c301440f0 msgr2=0x7f3c3006e510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:10.949 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:10.946+0000 7f3c386a5700 1 --2- 172.21.15.59:0/4083428845 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3c301440f0 0x7f3c3006e510 secure :-1 s=READY pgs=841 cs=0 l=1 rev1=1 crypto rx=0x7f3c28009e50 tx=0x7f3c2800b330 comp rx=0 tx=0).stop 2023-11-24T07:59:10.950 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:10.946+0000 7f3c386a5700 1 -- 172.21.15.59:0/4083428845 shutdown_connections 2023-11-24T07:59:10.950 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:10.946+0000 7f3c386a5700 1 --2- 172.21.15.59:0/4083428845 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3c301440f0 0x7f3c3006e510 unknown :-1 s=CLOSED pgs=841 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:10.950 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:10.946+0000 7f3c386a5700 1 --2- 172.21.15.59:0/4083428845 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f3c300722c0 0x7f3c300b51b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:10.950 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:10.946+0000 7f3c386a5700 1 -- 172.21.15.59:0/4083428845 >> 172.21.15.59:0/4083428845 conn(0x7f3c300b2740 msgr2=0x7f3c300b2b20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:59:10.950 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:10.946+0000 7f3c386a5700 1 -- 172.21.15.59:0/4083428845 shutdown_connections 2023-11-24T07:59:10.950 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:10.946+0000 7f3c386a5700 1 -- 172.21.15.59:0/4083428845 wait complete. 2023-11-24T07:59:10.952 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:10.947+0000 7f3c386a5700 1 Processor -- start 2023-11-24T07:59:10.952 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:10.947+0000 7f3c386a5700 1 -- start start 2023-11-24T07:59:10.952 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:10.948+0000 7f3c386a5700 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f3c300722c0 0x7f3c30156020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:10.952 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:10.948+0000 7f3c386a5700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3c30143ae0 0x7f3c300c59b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:10.952 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:10.948+0000 7f3c386a5700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3c301440f0 0x7f3c300c5ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:10.953 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:10.948+0000 7f3c386a5700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c3013d900 con 0x7f3c30143ae0 2023-11-24T07:59:10.953 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:10.948+0000 7f3c386a5700 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c3015d020 con 0x7f3c301440f0 2023-11-24T07:59:10.953 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:10.948+0000 7f3c386a5700 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3c3015d230 con 0x7f3c300722c0 2023-11-24T07:59:10.953 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:10.948+0000 7f3c35c40700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3c30143ae0 0x7f3c300c59b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:10.953 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:10.948+0000 7f3c35c40700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3c30143ae0 0x7f3c300c59b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:54696/0 (socket says 172.21.15.59:54696) 2023-11-24T07:59:10.953 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:10.948+0000 7f3c35c40700 1 -- 172.21.15.59:0/3763760516 learned_addr learned my addr 172.21.15.59:0/3763760516 (peer_addr_for_me v2:172.21.15.59:0/0) 2023-11-24T07:59:10.953 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:10.948+0000 7f3c36c42700 1 --2- 172.21.15.59:0/3763760516 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3c301440f0 0x7f3c300c5ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:10.953 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:10.948+0000 7f3c36441700 1 --2- 172.21.15.59:0/3763760516 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f3c300722c0 0x7f3c30156020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:10.953 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:10.949+0000 7f3c35c40700 1 -- 172.21.15.59:0/3763760516 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f3c300722c0 msgr2=0x7f3c30156020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:10.953 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:10.949+0000 7f3c35c40700 1 --2- 172.21.15.59:0/3763760516 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f3c300722c0 0x7f3c30156020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:10.953 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:10.949+0000 7f3c35c40700 1 -- 172.21.15.59:0/3763760516 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3c301440f0 msgr2=0x7f3c300c5ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:10.953 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:10.949+0000 7f3c35c40700 1 --2- 172.21.15.59:0/3763760516 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3c301440f0 0x7f3c300c5ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:10.953 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:10.949+0000 7f3c35c40700 1 -- 172.21.15.59:0/3763760516 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3c28009b30 con 0x7f3c30143ae0 2023-11-24T07:59:10.954 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:10.949+0000 7f3c36c42700 1 --2- 172.21.15.59:0/3763760516 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3c301440f0 0x7f3c300c5ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-24T07:59:10.954 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:10.949+0000 7f3c35c40700 1 --2- 172.21.15.59:0/3763760516 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3c30143ae0 0x7f3c300c59b0 secure :-1 s=READY pgs=842 cs=0 l=1 rev1=1 crypto rx=0x7f3c2000e810 tx=0x7f3c2000ebd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:59:10.954 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:10.949+0000 7f3c26ffd700 1 -- 172.21.15.59:0/3763760516 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c20013610 con 0x7f3c30143ae0 2023-11-24T07:59:10.954 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:10.949+0000 7f3c26ffd700 1 -- 172.21.15.59:0/3763760516 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3c20013770 con 0x7f3c30143ae0 2023-11-24T07:59:10.954 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:10.949+0000 7f3c386a5700 1 -- 172.21.15.59:0/3763760516 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3c3015d510 con 0x7f3c30143ae0 2023-11-24T07:59:10.954 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:10.949+0000 7f3c26ffd700 1 -- 172.21.15.59:0/3763760516 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c20013b20 con 0x7f3c30143ae0 2023-11-24T07:59:10.954 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:10.949+0000 7f3c386a5700 1 -- 172.21.15.59:0/3763760516 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3c3015dad0 con 0x7f3c30143ae0 2023-11-24T07:59:10.954 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:10.950+0000 7f3c386a5700 1 -- 172.21.15.59:0/3763760516 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3c30075480 con 0x7f3c30143ae0 2023-11-24T07:59:10.955 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:10.951+0000 7f3c26ffd700 1 -- 172.21.15.59:0/3763760516 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 10) v1 ==== 89588+0+0 (secure 0 0 0) 0x7f3c20013c80 con 0x7f3c30143ae0 2023-11-24T07:59:10.955 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:10.951+0000 7f3c26ffd700 1 --2- 172.21.15.59:0/3763760516 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f3c1406b2c0 0x7f3c1406d780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:10.955 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:10.952+0000 7f3c26ffd700 1 -- 172.21.15.59:0/3763760516 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(70..70 src has 1..70) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f3c20089b60 con 0x7f3c30143ae0 2023-11-24T07:59:10.955 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:10.952+0000 7f3c36441700 1 --2- 172.21.15.59:0/3763760516 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f3c1406b2c0 0x7f3c1406d780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:10.955 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:10.952+0000 7f3c36441700 1 --2- 172.21.15.59:0/3763760516 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f3c1406b2c0 0x7f3c1406d780 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f3c2c000f60 tx=0x7f3c2c009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:59:10.957 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:10.955+0000 7f3c26ffd700 1 -- 172.21.15.59:0/3763760516 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f3c20054940 con 0x7f3c30143ae0 2023-11-24T07:59:11.156 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:11.154+0000 7f3c386a5700 1 -- 172.21.15.59:0/3763760516 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f3c300b5d20 con 0x7f3c30143ae0 2023-11-24T07:59:11.157 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:11.155+0000 7f3c26ffd700 1 -- 172.21.15.59:0/3763760516 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v70) v1 ==== 93+0+10 (secure 0 0 0) 0x7f3c20057f60 con 0x7f3c30143ae0 2023-11-24T07:59:11.157 INFO:teuthology.orchestra.run.smithi059.stdout:pg_num: 1 2023-11-24T07:59:11.159 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:11.157+0000 7f3c386a5700 1 -- 172.21.15.59:0/3763760516 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f3c1406b2c0 msgr2=0x7f3c1406d780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:11.159 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:11.157+0000 7f3c386a5700 1 --2- 172.21.15.59:0/3763760516 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f3c1406b2c0 0x7f3c1406d780 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f3c2c000f60 tx=0x7f3c2c009040 comp rx=0 tx=0).stop 2023-11-24T07:59:11.159 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:11.157+0000 7f3c386a5700 1 -- 172.21.15.59:0/3763760516 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3c30143ae0 msgr2=0x7f3c300c59b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:11.160 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:11.157+0000 7f3c386a5700 1 --2- 172.21.15.59:0/3763760516 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3c30143ae0 0x7f3c300c59b0 secure :-1 s=READY pgs=842 cs=0 l=1 rev1=1 crypto rx=0x7f3c2000e810 tx=0x7f3c2000ebd0 comp rx=0 tx=0).stop 2023-11-24T07:59:11.160 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:11.157+0000 7f3c386a5700 1 -- 172.21.15.59:0/3763760516 shutdown_connections 2023-11-24T07:59:11.160 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:11.157+0000 7f3c386a5700 1 --2- 172.21.15.59:0/3763760516 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f3c1406b2c0 0x7f3c1406d780 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:11.160 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:11.157+0000 7f3c386a5700 1 --2- 172.21.15.59:0/3763760516 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3c301440f0 0x7f3c300c5ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:11.160 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:11.157+0000 7f3c386a5700 1 --2- 172.21.15.59:0/3763760516 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f3c300722c0 0x7f3c30156020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:11.160 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:11.157+0000 7f3c386a5700 1 --2- 172.21.15.59:0/3763760516 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3c30143ae0 0x7f3c300c59b0 unknown :-1 s=CLOSED pgs=842 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:11.160 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:11.157+0000 7f3c386a5700 1 -- 172.21.15.59:0/3763760516 >> 172.21.15.59:0/3763760516 conn(0x7f3c300b2740 msgr2=0x7f3c30142e10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:59:11.160 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:11.157+0000 7f3c386a5700 1 -- 172.21.15.59:0/3763760516 shutdown_connections 2023-11-24T07:59:11.160 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:11.157+0000 7f3c386a5700 1 -- 172.21.15.59:0/3763760516 wait complete. 2023-11-24T07:59:11.171 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2023-11-24T07:59:11.296 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:11.294+0000 7febc11e8700 1 Processor -- start 2023-11-24T07:59:11.296 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:11.294+0000 7febc11e8700 1 -- start start 2023-11-24T07:59:11.298 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:11.295+0000 7febc11e8700 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7febbc143ae0 0x7febbc0b51b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:11.298 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:11.295+0000 7febc11e8700 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7febbc1440f0 0x7febbc071bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:11.298 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:11.295+0000 7febc11e8700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7febbc0720f0 0x7febbc15e8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:11.298 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:11.295+0000 7febc11e8700 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 v1 -- 0x7febbc0b40a0 con 0x7febbc0720f0 2023-11-24T07:59:11.298 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:11.295+0000 7febc11e8700 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7febbc072580 con 0x7febbc1440f0 2023-11-24T07:59:11.298 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:11.295+0000 7febc11e8700 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 v1 -- 0x7febbc143ec0 con 0x7febbc143ae0 2023-11-24T07:59:11.299 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:11.295+0000 7febbb59e700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7febbc0720f0 0x7febbc15e8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:11.299 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:11.295+0000 7febbb59e700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7febbc0720f0 0x7febbc15e8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:54706/0 (socket says 172.21.15.59:54706) 2023-11-24T07:59:11.299 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:11.295+0000 7febbb59e700 1 -- 172.21.15.59:0/1504729080 learned_addr learned my addr 172.21.15.59:0/1504729080 (peer_addr_for_me v2:172.21.15.59:0/0) 2023-11-24T07:59:11.299 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:11.295+0000 7febba59c700 1 --2- 172.21.15.59:0/1504729080 >> v2:172.21.15.76:3300/0 conn(0x7febbc1440f0 0x7febbc071bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:11.299 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:11.295+0000 7febbad9d700 1 --2- 172.21.15.59:0/1504729080 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7febbc143ae0 0x7febbc0b51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:11.299 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:11.295+0000 7febbb59e700 1 -- 172.21.15.59:0/1504729080 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7febbc143ae0 msgr2=0x7febbc0b51b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T07:59:11.299 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:11.295+0000 7febbb59e700 1 --2- 172.21.15.59:0/1504729080 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7febbc143ae0 0x7febbc0b51b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:11.299 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:11.295+0000 7febbb59e700 1 -- 172.21.15.59:0/1504729080 >> v2:172.21.15.76:3300/0 conn(0x7febbc1440f0 msgr2=0x7febbc071bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T07:59:11.299 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:11.295+0000 7febbb59e700 1 --2- 172.21.15.59:0/1504729080 >> v2:172.21.15.76:3300/0 conn(0x7febbc1440f0 0x7febbc071bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:11.299 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:11.295+0000 7febbb59e700 1 -- 172.21.15.59:0/1504729080 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7febbc0b3ae0 con 0x7febbc0720f0 2023-11-24T07:59:11.299 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:11.295+0000 7febba59c700 1 --2- 172.21.15.59:0/1504729080 >> v2:172.21.15.76:3300/0 conn(0x7febbc1440f0 0x7febbc071bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-24T07:59:11.299 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:11.295+0000 7febbb59e700 1 --2- 172.21.15.59:0/1504729080 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7febbc0720f0 0x7febbc15e8b0 secure :-1 s=READY pgs=843 cs=0 l=1 rev1=1 crypto rx=0x7febbc06d070 tx=0x7febb400d430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=538fa5e0a96930b4 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:59:11.299 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:11.296+0000 7febb9d9b700 1 -- 172.21.15.59:0/1504729080 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7febb4010070 con 0x7febbc0720f0 2023-11-24T07:59:11.300 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:11.296+0000 7febb9d9b700 1 -- 172.21.15.59:0/1504729080 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7febb400dd90 con 0x7febbc0720f0 2023-11-24T07:59:11.300 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:11.296+0000 7febb9d9b700 1 -- 172.21.15.59:0/1504729080 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7febb400b310 con 0x7febbc0720f0 2023-11-24T07:59:11.300 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:11.296+0000 7febc11e8700 1 -- 172.21.15.59:0/1504729080 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7febbc0720f0 msgr2=0x7febbc15e8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:11.300 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:11.296+0000 7febc11e8700 1 --2- 172.21.15.59:0/1504729080 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7febbc0720f0 0x7febbc15e8b0 secure :-1 s=READY pgs=843 cs=0 l=1 rev1=1 crypto rx=0x7febbc06d070 tx=0x7febb400d430 comp rx=0 tx=0).stop 2023-11-24T07:59:11.300 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:11.296+0000 7febc11e8700 1 -- 172.21.15.59:0/1504729080 shutdown_connections 2023-11-24T07:59:11.300 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:11.296+0000 7febc11e8700 1 --2- 172.21.15.59:0/1504729080 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7febbc0720f0 0x7febbc15e8b0 unknown :-1 s=CLOSED pgs=843 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:11.300 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:11.296+0000 7febc11e8700 1 --2- 172.21.15.59:0/1504729080 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7febbc143ae0 0x7febbc0b51b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:11.300 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:11.296+0000 7febc11e8700 1 --2- 172.21.15.59:0/1504729080 >> v2:172.21.15.76:3300/0 conn(0x7febbc1440f0 0x7febbc071bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:11.300 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:11.296+0000 7febc11e8700 1 -- 172.21.15.59:0/1504729080 >> 172.21.15.59:0/1504729080 conn(0x7febbc0b2740 msgr2=0x7febbc0b2b20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:59:11.300 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:11.296+0000 7febc11e8700 1 -- 172.21.15.59:0/1504729080 shutdown_connections 2023-11-24T07:59:11.300 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:11.296+0000 7febc11e8700 1 -- 172.21.15.59:0/1504729080 wait complete. 2023-11-24T07:59:11.303 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:11.298+0000 7febc11e8700 1 Processor -- start 2023-11-24T07:59:11.303 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:11.298+0000 7febc11e8700 1 -- start start 2023-11-24T07:59:11.303 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:11.298+0000 7febc11e8700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7febbc0720f0 0x7febbc141f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:11.303 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:11.298+0000 7febc11e8700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7febbc143ae0 0x7febbc157150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:11.303 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:11.298+0000 7febc11e8700 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7febbc1440f0 0x7febbc1545e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:11.303 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:11.298+0000 7febc11e8700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7febbc142470 con 0x7febbc0720f0 2023-11-24T07:59:11.303 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:11.298+0000 7febc11e8700 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7febbc157ac0 con 0x7febbc143ae0 2023-11-24T07:59:11.303 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:11.298+0000 7febc11e8700 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 v1 -- 0x7febbc157da0 con 0x7febbc1440f0 2023-11-24T07:59:11.303 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:11.299+0000 7febbad9d700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7febbc0720f0 0x7febbc141f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:11.304 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:11.299+0000 7febbad9d700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7febbc0720f0 0x7febbc141f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:54708/0 (socket says 172.21.15.59:54708) 2023-11-24T07:59:11.304 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:11.299+0000 7febbad9d700 1 -- 172.21.15.59:0/3289105966 learned_addr learned my addr 172.21.15.59:0/3289105966 (peer_addr_for_me v2:172.21.15.59:0/0) 2023-11-24T07:59:11.304 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:11.299+0000 7febba59c700 1 --2- 172.21.15.59:0/3289105966 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7febbc143ae0 0x7febbc157150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:11.304 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:11.299+0000 7febbb59e700 1 --2- 172.21.15.59:0/3289105966 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7febbc1440f0 0x7febbc1545e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:11.304 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:11.299+0000 7febbad9d700 1 -- 172.21.15.59:0/3289105966 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7febbc1440f0 msgr2=0x7febbc1545e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:11.304 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:11.299+0000 7febbad9d700 1 --2- 172.21.15.59:0/3289105966 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7febbc1440f0 0x7febbc1545e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:11.304 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:11.299+0000 7febbad9d700 1 -- 172.21.15.59:0/3289105966 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7febbc143ae0 msgr2=0x7febbc157150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:11.304 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:11.299+0000 7febbad9d700 1 --2- 172.21.15.59:0/3289105966 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7febbc143ae0 0x7febbc157150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:11.304 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:11.299+0000 7febbad9d700 1 -- 172.21.15.59:0/3289105966 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7febb400bd40 con 0x7febbc0720f0 2023-11-24T07:59:11.304 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:11.299+0000 7febbb59e700 1 --2- 172.21.15.59:0/3289105966 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7febbc1440f0 0x7febbc1545e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T07:59:11.304 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:11.299+0000 7febbad9d700 1 --2- 172.21.15.59:0/3289105966 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7febbc0720f0 0x7febbc141f30 secure :-1 s=READY pgs=844 cs=0 l=1 rev1=1 crypto rx=0x7febb000d800 tx=0x7febb000dbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:59:11.304 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:11.299+0000 7feba37fe700 1 -- 172.21.15.59:0/3289105966 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7febb00125f0 con 0x7febbc0720f0 2023-11-24T07:59:11.304 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:11.299+0000 7feba37fe700 1 -- 172.21.15.59:0/3289105966 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7febb0012790 con 0x7febbc0720f0 2023-11-24T07:59:11.305 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:11.299+0000 7febc11e8700 1 -- 172.21.15.59:0/3289105966 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7febbc158080 con 0x7febbc0720f0 2023-11-24T07:59:11.305 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:11.299+0000 7feba37fe700 1 -- 172.21.15.59:0/3289105966 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7febb0012b40 con 0x7febbc0720f0 2023-11-24T07:59:11.305 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:11.299+0000 7febc11e8700 1 -- 172.21.15.59:0/3289105966 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7febbc158640 con 0x7febbc0720f0 2023-11-24T07:59:11.305 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:11.300+0000 7febc11e8700 1 -- 172.21.15.59:0/3289105966 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7febbc158800 con 0x7febbc0720f0 2023-11-24T07:59:11.305 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:11.301+0000 7feba37fe700 1 -- 172.21.15.59:0/3289105966 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 10) v1 ==== 89588+0+0 (secure 0 0 0) 0x7febb0012ca0 con 0x7febbc0720f0 2023-11-24T07:59:11.305 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:11.301+0000 7feba37fe700 1 --2- 172.21.15.59:0/3289105966 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7feba806b2c0 0x7feba806d780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:11.305 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:11.301+0000 7feba37fe700 1 -- 172.21.15.59:0/3289105966 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(70..70 src has 1..70) v4 ==== 4857+0+0 (secure 0 0 0) 0x7febb0089d70 con 0x7febbc0720f0 2023-11-24T07:59:11.305 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:11.302+0000 7febba59c700 1 --2- 172.21.15.59:0/3289105966 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7feba806b2c0 0x7feba806d780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:11.305 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:11.302+0000 7febba59c700 1 --2- 172.21.15.59:0/3289105966 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7feba806b2c0 0x7feba806d780 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7febbc152b30 tx=0x7febac008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:59:11.307 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:11.306+0000 7feba37fe700 1 -- 172.21.15.59:0/3289105966 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7febb0054ba0 con 0x7febbc0720f0 2023-11-24T07:59:11.513 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:11.512+0000 7febc11e8700 1 -- 172.21.15.59:0/3289105966 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7febbc144830 con 0x7febbc0720f0 2023-11-24T07:59:11.514 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:11.512+0000 7feba37fe700 1 -- 172.21.15.59:0/3289105966 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v70) v1 ==== 84+0+65 (secure 0 0 0) 0x7febb00581c0 con 0x7febbc0720f0 2023-11-24T07:59:11.514 INFO:teuthology.orchestra.run.smithi059.stdout: 2023-11-24T07:59:11.514 INFO:teuthology.orchestra.run.smithi059.stdout:[ 2023-11-24T07:59:11.514 INFO:teuthology.orchestra.run.smithi059.stdout: { 2023-11-24T07:59:11.514 INFO:teuthology.orchestra.run.smithi059.stdout: "poolnum": 1, 2023-11-24T07:59:11.514 INFO:teuthology.orchestra.run.smithi059.stdout: "poolname": ".mgr" 2023-11-24T07:59:11.515 INFO:teuthology.orchestra.run.smithi059.stdout: } 2023-11-24T07:59:11.515 INFO:teuthology.orchestra.run.smithi059.stdout:] 2023-11-24T07:59:11.517 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:11.514+0000 7febc11e8700 1 -- 172.21.15.59:0/3289105966 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7feba806b2c0 msgr2=0x7feba806d780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:11.517 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:11.514+0000 7febc11e8700 1 --2- 172.21.15.59:0/3289105966 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7feba806b2c0 0x7feba806d780 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7febbc152b30 tx=0x7febac008040 comp rx=0 tx=0).stop 2023-11-24T07:59:11.517 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:11.514+0000 7febc11e8700 1 -- 172.21.15.59:0/3289105966 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7febbc0720f0 msgr2=0x7febbc141f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:11.517 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:11.514+0000 7febc11e8700 1 --2- 172.21.15.59:0/3289105966 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7febbc0720f0 0x7febbc141f30 secure :-1 s=READY pgs=844 cs=0 l=1 rev1=1 crypto rx=0x7febb000d800 tx=0x7febb000dbc0 comp rx=0 tx=0).stop 2023-11-24T07:59:11.517 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:11.515+0000 7febc11e8700 1 -- 172.21.15.59:0/3289105966 shutdown_connections 2023-11-24T07:59:11.517 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:11.515+0000 7febc11e8700 1 --2- 172.21.15.59:0/3289105966 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7feba806b2c0 0x7feba806d780 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:11.517 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:11.515+0000 7febc11e8700 1 --2- 172.21.15.59:0/3289105966 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7febbc1440f0 0x7febbc1545e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:11.517 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:11.515+0000 7febc11e8700 1 --2- 172.21.15.59:0/3289105966 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7febbc0720f0 0x7febbc141f30 unknown :-1 s=CLOSED pgs=844 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:11.517 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:11.515+0000 7febc11e8700 1 --2- 172.21.15.59:0/3289105966 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7febbc143ae0 0x7febbc157150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:11.517 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:11.515+0000 7febc11e8700 1 -- 172.21.15.59:0/3289105966 >> 172.21.15.59:0/3289105966 conn(0x7febbc0b2740 msgr2=0x7febbc0cdf90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:59:11.517 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:11.515+0000 7febc11e8700 1 -- 172.21.15.59:0/3289105966 shutdown_connections 2023-11-24T07:59:11.517 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:11.515+0000 7febc11e8700 1 -- 172.21.15.59:0/3289105966 wait complete. 2023-11-24T07:59:11.529 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2023-11-24T07:59:11.529 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2023-11-24T07:59:11.659 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:11.657+0000 7f954afe7700 1 Processor -- start 2023-11-24T07:59:11.659 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:11.657+0000 7f954afe7700 1 -- start start 2023-11-24T07:59:11.662 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:11.658+0000 7f954afe7700 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f95440cd150 0x7f95440ceb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:11.662 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:11.658+0000 7f954afe7700 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f954406f4b0 con 0x7f954406f940 2023-11-24T07:59:11.662 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:11.658+0000 7f954afe7700 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f954415c3c0 con 0x7f954415bfe0 2023-11-24T07:59:11.662 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:11.658+0000 7f954afe7700 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f95440cd760 con 0x7f95440cd150 2023-11-24T07:59:11.662 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:11.658+0000 7f9549584700 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7f954406f940 0x7f954406fd90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:58194/0 (socket says 172.21.15.59:58194) 2023-11-24T07:59:11.663 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:11.658+0000 7f9549584700 1 -- 172.21.15.59:0/991216814 learned_addr learned my addr 172.21.15.59:0/991216814 (peer_addr_for_me v1:172.21.15.59:0/0) 2023-11-24T07:59:11.663 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:11.658+0000 7f9543fff700 1 --2- 172.21.15.59:0/991216814 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f95440cd150 0x7f95440ceb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:11.663 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:11.658+0000 7f95437fe700 1 -- 172.21.15.59:0/991216814 <== mon.0 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4136481381 0 0) 0x7f954406f4b0 con 0x7f954406f940 2023-11-24T07:59:11.663 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:11.658+0000 7f95437fe700 1 -- 172.21.15.59:0/991216814 --> v1:172.21.15.59:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f95340035b0 con 0x7f954406f940 2023-11-24T07:59:11.663 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:11.658+0000 7f95437fe700 1 -- 172.21.15.59:0/991216814 <== mon.1 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3971885063 0 0) 0x7f954415c3c0 con 0x7f954415bfe0 2023-11-24T07:59:11.663 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:11.658+0000 7f95437fe700 1 -- 172.21.15.59:0/991216814 --> v1:172.21.15.76:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f954406f4b0 con 0x7f954415bfe0 2023-11-24T07:59:11.663 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:11.658+0000 7f95437fe700 1 -- 172.21.15.59:0/991216814 <== mon.0 v1:172.21.15.59:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3283546004 0 0) 0x7f95340035b0 con 0x7f954406f940 2023-11-24T07:59:11.663 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:11.658+0000 7f95437fe700 1 -- 172.21.15.59:0/991216814 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f95440cd150 msgr2=0x7f95440ceb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T07:59:11.663 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:11.658+0000 7f95437fe700 1 --2- 172.21.15.59:0/991216814 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f95440cd150 0x7f95440ceb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:11.663 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:11.658+0000 7f95437fe700 1 -- 172.21.15.59:0/991216814 >> v1:172.21.15.76:6789/0 conn(0x7f954415bfe0 legacy=0x7f95440ce430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:11.663 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:11.658+0000 7f95437fe700 1 -- 172.21.15.59:0/991216814 --> v1:172.21.15.59:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f954406ee60 con 0x7f954406f940 2023-11-24T07:59:11.663 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:11.658+0000 7f95437fe700 1 -- 172.21.15.59:0/991216814 <== mon.0 v1:172.21.15.59:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1312333566 0 0) 0x7f953c0019f0 con 0x7f954406f940 2023-11-24T07:59:11.663 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:11.659+0000 7f95437fe700 1 -- 172.21.15.59:0/991216814 >> v1:172.21.15.59:6789/0 conn(0x7f954406f940 legacy=0x7f954406fd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:11.664 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:11.659+0000 7f95437fe700 1 --2- 172.21.15.59:0/991216814 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9534003bc0 0x7f9534003fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:11.664 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:11.659+0000 7f95437fe700 1 --2- 172.21.15.59:0/991216814 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f95440cd150 0x7f9534004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:11.664 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:11.659+0000 7f95437fe700 1 --2- 172.21.15.59:0/991216814 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9534004d90 0x7f9534005210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:11.664 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:11.659+0000 7f95437fe700 1 -- 172.21.15.59:0/991216814 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95440cd760 con 0x7f9534004d90 2023-11-24T07:59:11.664 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:11.659+0000 7f95437fe700 1 -- 172.21.15.59:0/991216814 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f953400bac0 con 0x7f9534003bc0 2023-11-24T07:59:11.664 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:11.659+0000 7f95437fe700 1 -- 172.21.15.59:0/991216814 --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f953400bc30 con 0x7f95440cd150 2023-11-24T07:59:11.664 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:11.659+0000 7f9549584700 1 --2- 172.21.15.59:0/991216814 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9534004d90 0x7f9534005210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:11.664 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:11.659+0000 7f9543fff700 1 --2- 172.21.15.59:0/991216814 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f95440cd150 0x7f9534004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:11.664 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:11.659+0000 7f9548d83700 1 --2- 172.21.15.59:0/991216814 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9534003bc0 0x7f9534003fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:11.664 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:11.659+0000 7f9549584700 1 -- 172.21.15.59:0/991216814 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f95440cd150 msgr2=0x7f9534004850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T07:59:11.664 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:11.659+0000 7f9549584700 1 --2- 172.21.15.59:0/991216814 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f95440cd150 0x7f9534004850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:11.664 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:11.659+0000 7f9549584700 1 -- 172.21.15.59:0/991216814 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9534003bc0 msgr2=0x7f9534003fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T07:59:11.664 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:11.659+0000 7f9549584700 1 --2- 172.21.15.59:0/991216814 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9534003bc0 0x7f9534003fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:11.665 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:11.659+0000 7f9549584700 1 -- 172.21.15.59:0/991216814 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f95340035b0 con 0x7f9534004d90 2023-11-24T07:59:11.665 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:11.659+0000 7f9549584700 1 --2- 172.21.15.59:0/991216814 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9534004d90 0x7f9534005210 secure :-1 s=READY pgs=846 cs=0 l=1 rev1=1 crypto rx=0x7f953c00a7a0 tx=0x7f9534005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=70fc172c28bbf2db server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:59:11.665 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:11.659+0000 7f95437fe700 1 -- 172.21.15.59:0/991216814 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f953c012600 con 0x7f9534004d90 2023-11-24T07:59:11.665 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:11.659+0000 7f95437fe700 1 -- 172.21.15.59:0/991216814 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f953c012780 con 0x7f9534004d90 2023-11-24T07:59:11.665 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:11.659+0000 7f95437fe700 1 -- 172.21.15.59:0/991216814 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f953c012b30 con 0x7f9534004d90 2023-11-24T07:59:11.665 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:11.660+0000 7f954afe7700 1 -- 172.21.15.59:0/991216814 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9534004d90 msgr2=0x7f9534005210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:11.665 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:11.660+0000 7f954afe7700 1 --2- 172.21.15.59:0/991216814 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9534004d90 0x7f9534005210 secure :-1 s=READY pgs=846 cs=0 l=1 rev1=1 crypto rx=0x7f953c00a7a0 tx=0x7f9534005880 comp rx=0 tx=0).stop 2023-11-24T07:59:11.665 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:11.660+0000 7f9549584700 1 -- 172.21.15.59:0/991216814 reap_dead start 2023-11-24T07:59:11.665 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:11.660+0000 7f954afe7700 1 -- 172.21.15.59:0/991216814 shutdown_connections 2023-11-24T07:59:11.665 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:11.660+0000 7f954afe7700 1 -- 172.21.15.59:0/991216814 >> 172.21.15.59:0/991216814 conn(0x7f95440b3700 msgr2=0x7f95440b3b10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:59:11.665 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:11.660+0000 7f954afe7700 1 -- 172.21.15.59:0/991216814 shutdown_connections 2023-11-24T07:59:11.665 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:11.660+0000 7f954afe7700 1 -- 172.21.15.59:0/991216814 wait complete. 2023-11-24T07:59:11.668 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:11.661+0000 7f954afe7700 1 Processor -- start 2023-11-24T07:59:11.668 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:11.661+0000 7f954afe7700 1 -- start start 2023-11-24T07:59:11.668 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:11.662+0000 7f954afe7700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9544159130 0x7f954415cf70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:11.668 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:11.662+0000 7f954afe7700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9544159510 0x7f95440b42e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:11.669 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:11.662+0000 7f954afe7700 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f95440caa30 0x7f95440cae90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:11.669 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:11.662+0000 7f954afe7700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95440c85d0 con 0x7f9544159510 2023-11-24T07:59:11.669 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:11.662+0000 7f954afe7700 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95440cb3d0 con 0x7f9544159130 2023-11-24T07:59:11.669 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:11.662+0000 7f954afe7700 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f95440c5aa0 con 0x7f95440caa30 2023-11-24T07:59:11.669 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:11.662+0000 7f9543fff700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9544159510 0x7f95440b42e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:11.669 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:11.662+0000 7f9543fff700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9544159510 0x7f95440b42e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:54730/0 (socket says 172.21.15.59:54730) 2023-11-24T07:59:11.669 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:11.662+0000 7f9543fff700 1 -- 172.21.15.59:0/1280745684 learned_addr learned my addr 172.21.15.59:0/1280745684 (peer_addr_for_me v2:172.21.15.59:0/0) 2023-11-24T07:59:11.669 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:11.662+0000 7f9548d83700 1 --2- 172.21.15.59:0/1280745684 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9544159130 0x7f954415cf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:11.669 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:11.662+0000 7f9549584700 1 --2- 172.21.15.59:0/1280745684 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f95440caa30 0x7f95440cae90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:11.669 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:11.662+0000 7f9543fff700 1 -- 172.21.15.59:0/1280745684 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f95440caa30 msgr2=0x7f95440cae90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:11.669 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:11.662+0000 7f9543fff700 1 --2- 172.21.15.59:0/1280745684 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f95440caa30 0x7f95440cae90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:11.669 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:11.662+0000 7f9543fff700 1 -- 172.21.15.59:0/1280745684 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9544159130 msgr2=0x7f954415cf70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:11.669 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:11.662+0000 7f9543fff700 1 --2- 172.21.15.59:0/1280745684 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9544159130 0x7f954415cf70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:11.669 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:11.662+0000 7f9543fff700 1 -- 172.21.15.59:0/1280745684 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f95440c5d80 con 0x7f9544159510 2023-11-24T07:59:11.670 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:11.662+0000 7f9549584700 1 --2- 172.21.15.59:0/1280745684 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f95440caa30 0x7f95440cae90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T07:59:11.670 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:11.663+0000 7f9543fff700 1 --2- 172.21.15.59:0/1280745684 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9544159510 0x7f95440b42e0 secure :-1 s=READY pgs=847 cs=0 l=1 rev1=1 crypto rx=0x7f9530011aa0 tx=0x7f9530011e60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:59:11.670 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:11.663+0000 7f95417fa700 1 -- 172.21.15.59:0/1280745684 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f953000f980 con 0x7f9544159510 2023-11-24T07:59:11.670 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:11.663+0000 7f95417fa700 1 -- 172.21.15.59:0/1280745684 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f953000fae0 con 0x7f9544159510 2023-11-24T07:59:11.670 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:11.663+0000 7f954afe7700 1 -- 172.21.15.59:0/1280745684 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f95440c6060 con 0x7f9544159510 2023-11-24T07:59:11.670 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:11.663+0000 7f95417fa700 1 -- 172.21.15.59:0/1280745684 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f953000fe90 con 0x7f9544159510 2023-11-24T07:59:11.670 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:11.663+0000 7f954afe7700 1 -- 172.21.15.59:0/1280745684 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f95440c6620 con 0x7f9544159510 2023-11-24T07:59:11.670 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:11.664+0000 7f954afe7700 1 -- 172.21.15.59:0/1280745684 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f954406bc40 con 0x7f9544159510 2023-11-24T07:59:11.670 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:11.664+0000 7f95417fa700 1 -- 172.21.15.59:0/1280745684 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 10) v1 ==== 89588+0+0 (secure 0 0 0) 0x7f9530014440 con 0x7f9544159510 2023-11-24T07:59:11.670 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:11.665+0000 7f95417fa700 1 --2- 172.21.15.59:0/1280745684 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f952806b2c0 0x7f952806d780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:11.670 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:11.665+0000 7f95417fa700 1 -- 172.21.15.59:0/1280745684 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(70..70 src has 1..70) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f953008a150 con 0x7f9544159510 2023-11-24T07:59:11.670 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:11.665+0000 7f9548d83700 1 --2- 172.21.15.59:0/1280745684 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f952806b2c0 0x7f952806d780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:11.670 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:11.665+0000 7f9548d83700 1 --2- 172.21.15.59:0/1280745684 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f952806b2c0 0x7f952806d780 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f9538007fd0 tx=0x7f9538009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:59:11.671 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:11.669+0000 7f95417fa700 1 -- 172.21.15.59:0/1280745684 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f95300550f0 con 0x7f9544159510 2023-11-24T07:59:11.871 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:11.869+0000 7f954afe7700 1 -- 172.21.15.59:0/1280745684 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7f9544068670 con 0x7f9544159510 2023-11-24T07:59:12.620 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:12.619+0000 7f95417fa700 1 -- 172.21.15.59:0/1280745684 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v71) v1 ==== 138+0+0 (secure 0 0 0) 0x7f9530058710 con 0x7f9544159510 2023-11-24T07:59:12.625 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:12.623+0000 7f954afe7700 1 -- 172.21.15.59:0/1280745684 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f952806b2c0 msgr2=0x7f952806d780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:12.625 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:12.623+0000 7f954afe7700 1 --2- 172.21.15.59:0/1280745684 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f952806b2c0 0x7f952806d780 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f9538007fd0 tx=0x7f9538009040 comp rx=0 tx=0).stop 2023-11-24T07:59:12.625 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:12.623+0000 7f954afe7700 1 -- 172.21.15.59:0/1280745684 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9544159510 msgr2=0x7f95440b42e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:12.625 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:12.623+0000 7f954afe7700 1 --2- 172.21.15.59:0/1280745684 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9544159510 0x7f95440b42e0 secure :-1 s=READY pgs=847 cs=0 l=1 rev1=1 crypto rx=0x7f9530011aa0 tx=0x7f9530011e60 comp rx=0 tx=0).stop 2023-11-24T07:59:12.626 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:12.624+0000 7f954afe7700 1 -- 172.21.15.59:0/1280745684 shutdown_connections 2023-11-24T07:59:12.626 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:12.624+0000 7f954afe7700 1 --2- 172.21.15.59:0/1280745684 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f952806b2c0 0x7f952806d780 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:12.626 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:12.624+0000 7f954afe7700 1 --2- 172.21.15.59:0/1280745684 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f95440caa30 0x7f95440cae90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:12.627 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:12.624+0000 7f954afe7700 1 --2- 172.21.15.59:0/1280745684 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9544159130 0x7f954415cf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:12.627 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:12.624+0000 7f954afe7700 1 --2- 172.21.15.59:0/1280745684 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9544159510 0x7f95440b42e0 unknown :-1 s=CLOSED pgs=847 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:12.627 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:12.624+0000 7f954afe7700 1 -- 172.21.15.59:0/1280745684 >> 172.21.15.59:0/1280745684 conn(0x7f95440b3700 msgr2=0x7f954415a910 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:59:12.627 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:12.624+0000 7f954afe7700 1 -- 172.21.15.59:0/1280745684 shutdown_connections 2023-11-24T07:59:12.627 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:12.624+0000 7f954afe7700 1 -- 172.21.15.59:0/1280745684 wait complete. 2023-11-24T07:59:12.629 INFO:teuthology.orchestra.run.smithi059.stderr:pool 'cephfs_metadata' created 2023-11-24T07:59:12.645 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2023-11-24T07:59:12.784 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:12.782+0000 7f95dee51700 1 Processor -- start 2023-11-24T07:59:12.784 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:12.782+0000 7f95dee51700 1 -- start start 2023-11-24T07:59:12.786 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:12.783+0000 7f95dee51700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f95d8143ae0 0x7f95d80b51b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:12.786 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:12.783+0000 7f95dee51700 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f95d81440f0 0x7f95d8071bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:12.786 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:12.783+0000 7f95dee51700 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f95d8143ec0 con 0x7f95d80720f0 2023-11-24T07:59:12.786 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:12.783+0000 7f95dee51700 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f95d80b40a0 con 0x7f95d8143ae0 2023-11-24T07:59:12.786 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:12.783+0000 7f95dee51700 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f95d8072580 con 0x7f95d81440f0 2023-11-24T07:59:12.787 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:12.783+0000 7f95dcbed700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f95d8143ae0 0x7f95d80b51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:12.787 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:12.783+0000 7f95dd3ee700 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7f95d80720f0 0x7f95d815e8b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:58214/0 (socket says 172.21.15.59:58214) 2023-11-24T07:59:12.787 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:12.783+0000 7f95dcbed700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f95d8143ae0 0x7f95d80b51b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:54736/0 (socket says 172.21.15.59:54736) 2023-11-24T07:59:12.787 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:12.783+0000 7f95dd3ee700 1 -- 172.21.15.59:0/1075430724 learned_addr learned my addr 172.21.15.59:0/1075430724 (peer_addr_for_me v1:172.21.15.59:0/0) 2023-11-24T07:59:12.787 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:12.783+0000 7f95d7fff700 1 --2- 172.21.15.59:0/1075430724 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f95d81440f0 0x7f95d8071bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:12.787 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:12.783+0000 7f95dcbed700 1 -- 172.21.15.59:0/1075430724 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f95d81440f0 msgr2=0x7f95d8071bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T07:59:12.787 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:12.783+0000 7f95dcbed700 1 --2- 172.21.15.59:0/1075430724 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f95d81440f0 0x7f95d8071bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:12.787 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:12.783+0000 7f95dcbed700 1 -- 172.21.15.59:0/1075430724 >> v1:172.21.15.59:6789/0 conn(0x7f95d80720f0 legacy=0x7f95d815e8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:12.787 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:12.783+0000 7f95dcbed700 1 -- 172.21.15.59:0/1075430724 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f95d80b3ae0 con 0x7f95d8143ae0 2023-11-24T07:59:12.787 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:12.783+0000 7f95d77fe700 1 -- 172.21.15.59:0/1075430724 <== mon.0 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3749608722 0 0) 0x7f95d8143ec0 con 0x7f95d80720f0 2023-11-24T07:59:12.787 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:12.783+0000 7f95d7fff700 1 --2- 172.21.15.59:0/1075430724 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f95d81440f0 0x7f95d8071bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T07:59:12.787 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:12.784+0000 7f95dcbed700 1 --2- 172.21.15.59:0/1075430724 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f95d8143ae0 0x7f95d80b51b0 secure :-1 s=READY pgs=849 cs=0 l=1 rev1=1 crypto rx=0x7f95cc009840 tx=0x7f95cc009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=949f92959e2543b5 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:59:12.787 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:12.784+0000 7f95d77fe700 1 -- 172.21.15.59:0/1075430724 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95cc011830 con 0x7f95d8143ae0 2023-11-24T07:59:12.788 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:12.784+0000 7f95d77fe700 1 -- 172.21.15.59:0/1075430724 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f95cc011990 con 0x7f95d8143ae0 2023-11-24T07:59:12.788 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:12.784+0000 7f95d77fe700 1 -- 172.21.15.59:0/1075430724 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95cc011d20 con 0x7f95d8143ae0 2023-11-24T07:59:12.788 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:12.784+0000 7f95dee51700 1 -- 172.21.15.59:0/1075430724 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f95d8143ae0 msgr2=0x7f95d80b51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:12.788 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:12.784+0000 7f95dee51700 1 --2- 172.21.15.59:0/1075430724 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f95d8143ae0 0x7f95d80b51b0 secure :-1 s=READY pgs=849 cs=0 l=1 rev1=1 crypto rx=0x7f95cc009840 tx=0x7f95cc009b50 comp rx=0 tx=0).stop 2023-11-24T07:59:12.788 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:12.784+0000 7f95dee51700 1 -- 172.21.15.59:0/1075430724 shutdown_connections 2023-11-24T07:59:12.788 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:12.784+0000 7f95dee51700 1 --2- 172.21.15.59:0/1075430724 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f95d8143ae0 0x7f95d80b51b0 unknown :-1 s=CLOSED pgs=849 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:12.788 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:12.784+0000 7f95dee51700 1 --2- 172.21.15.59:0/1075430724 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f95d81440f0 0x7f95d8071bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:12.788 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:12.784+0000 7f95dee51700 1 -- 172.21.15.59:0/1075430724 >> 172.21.15.59:0/1075430724 conn(0x7f95d80b2740 msgr2=0x7f95d80b2b20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:59:12.788 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:12.784+0000 7f95dee51700 1 -- 172.21.15.59:0/1075430724 shutdown_connections 2023-11-24T07:59:12.788 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:12.784+0000 7f95dee51700 1 -- 172.21.15.59:0/1075430724 wait complete. 2023-11-24T07:59:12.791 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:12.786+0000 7f95dee51700 1 Processor -- start 2023-11-24T07:59:12.791 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:12.786+0000 7f95dee51700 1 -- start start 2023-11-24T07:59:12.791 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:12.786+0000 7f95dee51700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f95d80720f0 0x7f95d815de70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:12.791 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:12.786+0000 7f95dee51700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f95d8143ae0 0x7f95d81530a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:12.791 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:12.786+0000 7f95dee51700 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f95d81440f0 0x7f95d81535e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:12.791 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:12.786+0000 7f95dee51700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95d815e3b0 con 0x7f95d8143ae0 2023-11-24T07:59:12.792 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:12.787+0000 7f95dee51700 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95d815f1b0 con 0x7f95d80720f0 2023-11-24T07:59:12.792 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:12.787+0000 7f95dee51700 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f95d81430c0 con 0x7f95d81440f0 2023-11-24T07:59:12.792 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:12.787+0000 7f95d7fff700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f95d8143ae0 0x7f95d81530a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:12.792 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:12.787+0000 7f95d7fff700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f95d8143ae0 0x7f95d81530a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:54744/0 (socket says 172.21.15.59:54744) 2023-11-24T07:59:12.792 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:12.787+0000 7f95d7fff700 1 -- 172.21.15.59:0/419290202 learned_addr learned my addr 172.21.15.59:0/419290202 (peer_addr_for_me v2:172.21.15.59:0/0) 2023-11-24T07:59:12.792 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:12.787+0000 7f95dcbed700 1 --2- 172.21.15.59:0/419290202 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f95d80720f0 0x7f95d815de70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:12.792 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:12.787+0000 7f95dd3ee700 1 --2- 172.21.15.59:0/419290202 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f95d81440f0 0x7f95d81535e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:12.792 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:12.787+0000 7f95d7fff700 1 -- 172.21.15.59:0/419290202 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f95d81440f0 msgr2=0x7f95d81535e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:12.792 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:12.787+0000 7f95d7fff700 1 --2- 172.21.15.59:0/419290202 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f95d81440f0 0x7f95d81535e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:12.792 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:12.787+0000 7f95d7fff700 1 -- 172.21.15.59:0/419290202 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f95d80720f0 msgr2=0x7f95d815de70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:12.792 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:12.787+0000 7f95d7fff700 1 --2- 172.21.15.59:0/419290202 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f95d80720f0 0x7f95d815de70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:12.792 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:12.787+0000 7f95d7fff700 1 -- 172.21.15.59:0/419290202 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f95cc0094a0 con 0x7f95d8143ae0 2023-11-24T07:59:12.792 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:12.787+0000 7f95dcbed700 1 --2- 172.21.15.59:0/419290202 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f95d80720f0 0x7f95d815de70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-24T07:59:12.792 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:12.787+0000 7f95d7fff700 1 --2- 172.21.15.59:0/419290202 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f95d8143ae0 0x7f95d81530a0 secure :-1 s=READY pgs=850 cs=0 l=1 rev1=1 crypto rx=0x7f95c400c750 tx=0x7f95c400cb10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:59:12.793 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:12.787+0000 7f95d57fa700 1 -- 172.21.15.59:0/419290202 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95c4011800 con 0x7f95d8143ae0 2023-11-24T07:59:12.793 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:12.787+0000 7f95d57fa700 1 -- 172.21.15.59:0/419290202 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f95c4011960 con 0x7f95d8143ae0 2023-11-24T07:59:12.793 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:12.787+0000 7f95dee51700 1 -- 172.21.15.59:0/419290202 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f95d815e5a0 con 0x7f95d8143ae0 2023-11-24T07:59:12.793 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:12.787+0000 7f95d57fa700 1 -- 172.21.15.59:0/419290202 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95c4011cd0 con 0x7f95d8143ae0 2023-11-24T07:59:12.793 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:12.787+0000 7f95dee51700 1 -- 172.21.15.59:0/419290202 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f95d815eb60 con 0x7f95d8143ae0 2023-11-24T07:59:12.793 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:12.788+0000 7f95dee51700 1 -- 172.21.15.59:0/419290202 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f95d8075480 con 0x7f95d8143ae0 2023-11-24T07:59:12.793 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:12.789+0000 7f95d57fa700 1 -- 172.21.15.59:0/419290202 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 10) v1 ==== 89588+0+0 (secure 0 0 0) 0x7f95c4021350 con 0x7f95d8143ae0 2023-11-24T07:59:12.793 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:12.789+0000 7f95d57fa700 1 --2- 172.21.15.59:0/419290202 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f95bc06b2c0 0x7f95bc06d780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:12.793 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:12.789+0000 7f95d57fa700 1 -- 172.21.15.59:0/419290202 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(71..71 src has 1..71) v4 ==== 5227+0+0 (secure 0 0 0) 0x7f95c4089b50 con 0x7f95d8143ae0 2023-11-24T07:59:12.793 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:12.790+0000 7f95dcbed700 1 --2- 172.21.15.59:0/419290202 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f95bc06b2c0 0x7f95bc06d780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:12.794 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:12.790+0000 7f95dcbed700 1 --2- 172.21.15.59:0/419290202 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f95bc06b2c0 0x7f95bc06d780 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f95cc0090b0 tx=0x7f95cc008030 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:59:12.795 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:12.794+0000 7f95d57fa700 1 -- 172.21.15.59:0/419290202 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f95c4054780 con 0x7f95d8143ae0 2023-11-24T07:59:12.995 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:12.994+0000 7f95dee51700 1 -- 172.21.15.59:0/419290202 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7f95d80b5d20 con 0x7f95d8143ae0 2023-11-24T07:59:13.629 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:13.627+0000 7f95d57fa700 1 -- 172.21.15.59:0/419290202 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v72) v1 ==== 170+0+0 (secure 0 0 0) 0x7f95c401e0d0 con 0x7f95d8143ae0 2023-11-24T07:59:13.633 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:13.632+0000 7f95dee51700 1 -- 172.21.15.59:0/419290202 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f95bc06b2c0 msgr2=0x7f95bc06d780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:13.633 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:13.632+0000 7f95dee51700 1 --2- 172.21.15.59:0/419290202 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f95bc06b2c0 0x7f95bc06d780 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f95cc0090b0 tx=0x7f95cc008030 comp rx=0 tx=0).stop 2023-11-24T07:59:13.633 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:13.632+0000 7f95dee51700 1 -- 172.21.15.59:0/419290202 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f95d8143ae0 msgr2=0x7f95d81530a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:13.633 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:13.632+0000 7f95dee51700 1 --2- 172.21.15.59:0/419290202 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f95d8143ae0 0x7f95d81530a0 secure :-1 s=READY pgs=850 cs=0 l=1 rev1=1 crypto rx=0x7f95c400c750 tx=0x7f95c400cb10 comp rx=0 tx=0).stop 2023-11-24T07:59:13.634 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:13.632+0000 7f95dee51700 1 -- 172.21.15.59:0/419290202 shutdown_connections 2023-11-24T07:59:13.634 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:13.632+0000 7f95dee51700 1 --2- 172.21.15.59:0/419290202 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f95bc06b2c0 0x7f95bc06d780 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:13.634 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:13.632+0000 7f95dee51700 1 --2- 172.21.15.59:0/419290202 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f95d81440f0 0x7f95d81535e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:13.634 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:13.632+0000 7f95dee51700 1 --2- 172.21.15.59:0/419290202 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f95d80720f0 0x7f95d815de70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:13.634 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:13.632+0000 7f95dee51700 1 --2- 172.21.15.59:0/419290202 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f95d8143ae0 0x7f95d81530a0 unknown :-1 s=CLOSED pgs=850 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:13.634 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:13.632+0000 7f95dee51700 1 -- 172.21.15.59:0/419290202 >> 172.21.15.59:0/419290202 conn(0x7f95d80b2740 msgr2=0x7f95d80b4430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:59:13.635 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:13.632+0000 7f95dee51700 1 -- 172.21.15.59:0/419290202 shutdown_connections 2023-11-24T07:59:13.635 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:13.632+0000 7f95dee51700 1 -- 172.21.15.59:0/419290202 wait complete. 2023-11-24T07:59:13.642 INFO:teuthology.orchestra.run.smithi059.stderr:pool 'cephfs_data' created 2023-11-24T07:59:13.663 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2023-11-24T07:59:13.797 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:13.795+0000 7f9f13608700 1 Processor -- start 2023-11-24T07:59:13.797 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:13.795+0000 7f9f13608700 1 -- start start 2023-11-24T07:59:13.806 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:13.796+0000 7f9f13608700 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f9f0c0b4150 0x7f9f0c06de50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:13.806 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:13.796+0000 7f9f13608700 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9f0c071f50 con 0x7f9f0c06b620 2023-11-24T07:59:13.806 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:13.796+0000 7f9f13608700 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9f0c072230 con 0x7f9f0c0b5ab0 2023-11-24T07:59:13.806 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:13.796+0000 7f9f13608700 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9f0c072510 con 0x7f9f0c0b4150 2023-11-24T07:59:13.806 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:13.796+0000 7f9f113a4700 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7f9f0c06b620 0x7f9f0c0b3a40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:58224/0 (socket says 172.21.15.59:58224) 2023-11-24T07:59:13.806 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:13.796+0000 7f9f113a4700 1 -- 172.21.15.59:0/4000868915 learned_addr learned my addr 172.21.15.59:0/4000868915 (peer_addr_for_me v1:172.21.15.59:0/0) 2023-11-24T07:59:13.807 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:13.796+0000 7f9f10ba3700 1 --2- 172.21.15.59:0/4000868915 >> v2:172.21.15.76:3300/0 conn(0x7f9f0c0b4150 0x7f9f0c06de50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:13.807 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:13.796+0000 7f9f03fff700 1 -- 172.21.15.59:0/4000868915 <== mon.0 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1124794259 0 0) 0x7f9f0c071f50 con 0x7f9f0c06b620 2023-11-24T07:59:13.807 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:13.796+0000 7f9f03fff700 1 -- 172.21.15.59:0/4000868915 --> v1:172.21.15.59:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9ef80035b0 con 0x7f9f0c06b620 2023-11-24T07:59:13.807 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:13.796+0000 7f9f03fff700 1 -- 172.21.15.59:0/4000868915 <== mon.0 v1:172.21.15.59:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1668032809 0 0) 0x7f9ef80035b0 con 0x7f9f0c06b620 2023-11-24T07:59:13.807 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:13.796+0000 7f9f10ba3700 1 -- 172.21.15.59:0/4000868915 >> v1:172.21.15.76:6789/0 conn(0x7f9f0c0b5ab0 legacy=0x7f9f0c06e390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:13.807 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:13.796+0000 7f9f10ba3700 1 -- 172.21.15.59:0/4000868915 >> v1:172.21.15.59:6789/0 conn(0x7f9f0c06b620 legacy=0x7f9f0c0b3a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:13.808 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:13.796+0000 7f9f10ba3700 1 -- 172.21.15.59:0/4000868915 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9f0c0727f0 con 0x7f9f0c0b4150 2023-11-24T07:59:13.808 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:13.797+0000 7f9f10ba3700 1 --2- 172.21.15.59:0/4000868915 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9f0c0b4150 0x7f9f0c06de50 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f9efc009840 tx=0x7f9efc009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=98601681b74d6f6c server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:59:13.808 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:13.797+0000 7f9f03fff700 1 -- 172.21.15.59:0/4000868915 <== mon.1 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f08001670 con 0x7f9f0c0b4150 2023-11-24T07:59:13.808 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:13.797+0000 7f9f03fff700 1 -- 172.21.15.59:0/4000868915 <== mon.1 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9efc004030 con 0x7f9f0c0b4150 2023-11-24T07:59:13.808 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:13.797+0000 7f9f03fff700 1 -- 172.21.15.59:0/4000868915 <== mon.1 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9efc0043a0 con 0x7f9f0c0b4150 2023-11-24T07:59:13.808 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:13.797+0000 7f9f13608700 1 -- 172.21.15.59:0/4000868915 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9f0c0b4150 msgr2=0x7f9f0c06de50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:13.809 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:13.797+0000 7f9f13608700 1 --2- 172.21.15.59:0/4000868915 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9f0c0b4150 0x7f9f0c06de50 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f9efc009840 tx=0x7f9efc009b50 comp rx=0 tx=0).stop 2023-11-24T07:59:13.809 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:13.797+0000 7f9f13608700 1 -- 172.21.15.59:0/4000868915 shutdown_connections 2023-11-24T07:59:13.809 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:13.797+0000 7f9f13608700 1 --2- 172.21.15.59:0/4000868915 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9f0c0b4150 0x7f9f0c06de50 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:13.809 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:13.797+0000 7f9f13608700 1 -- 172.21.15.59:0/4000868915 >> 172.21.15.59:0/4000868915 conn(0x7f9f0c0b2740 msgr2=0x7f9f0c0b2b20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:59:13.809 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:13.797+0000 7f9f13608700 1 -- 172.21.15.59:0/4000868915 shutdown_connections 2023-11-24T07:59:13.809 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:13.797+0000 7f9f13608700 1 -- 172.21.15.59:0/4000868915 wait complete. 2023-11-24T07:59:13.810 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:13.799+0000 7f9f13608700 1 Processor -- start 2023-11-24T07:59:13.810 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:13.799+0000 7f9f13608700 1 -- start start 2023-11-24T07:59:13.810 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:13.799+0000 7f9f13608700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9f0c06b620 0x7f9f0c1e8f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:13.810 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:13.799+0000 7f9f13608700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9f0c0b4150 0x7f9f0c1e4e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:13.810 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:13.799+0000 7f9f13608700 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f9f0c0b5ab0 0x7f9f0c1e53d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:13.810 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:13.799+0000 7f9f13608700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f0c1f6f90 con 0x7f9f0c0b4150 2023-11-24T07:59:13.810 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:13.799+0000 7f9f13608700 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f0c1f7db0 con 0x7f9f0c06b620 2023-11-24T07:59:13.811 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:13.799+0000 7f9f13608700 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9f0c1f71a0 con 0x7f9f0c0b5ab0 2023-11-24T07:59:13.811 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:13.799+0000 7f9f10ba3700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9f0c0b4150 0x7f9f0c1e4e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:13.811 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:13.799+0000 7f9f10ba3700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9f0c0b4150 0x7f9f0c1e4e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:54758/0 (socket says 172.21.15.59:54758) 2023-11-24T07:59:13.811 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:13.799+0000 7f9f10ba3700 1 -- 172.21.15.59:0/1146381381 learned_addr learned my addr 172.21.15.59:0/1146381381 (peer_addr_for_me v2:172.21.15.59:0/0) 2023-11-24T07:59:13.811 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:13.800+0000 7f9f113a4700 1 --2- 172.21.15.59:0/1146381381 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9f0c06b620 0x7f9f0c1e8f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:13.811 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:13.800+0000 7f9f11ba5700 1 --2- 172.21.15.59:0/1146381381 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f9f0c0b5ab0 0x7f9f0c1e53d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:13.812 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:13.800+0000 7f9f10ba3700 1 -- 172.21.15.59:0/1146381381 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f9f0c0b5ab0 msgr2=0x7f9f0c1e53d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:13.812 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:13.800+0000 7f9f10ba3700 1 --2- 172.21.15.59:0/1146381381 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f9f0c0b5ab0 0x7f9f0c1e53d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:13.812 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:13.800+0000 7f9f10ba3700 1 -- 172.21.15.59:0/1146381381 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9f0c06b620 msgr2=0x7f9f0c1e8f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:13.813 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:13.800+0000 7f9f10ba3700 1 --2- 172.21.15.59:0/1146381381 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9f0c06b620 0x7f9f0c1e8f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:13.813 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:13.800+0000 7f9f10ba3700 1 -- 172.21.15.59:0/1146381381 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9efc0094a0 con 0x7f9f0c0b4150 2023-11-24T07:59:13.813 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:13.800+0000 7f9f11ba5700 1 --2- 172.21.15.59:0/1146381381 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f9f0c0b5ab0 0x7f9f0c1e53d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-24T07:59:13.813 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:13.800+0000 7f9f10ba3700 1 --2- 172.21.15.59:0/1146381381 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9f0c0b4150 0x7f9f0c1e4e90 secure :-1 s=READY pgs=852 cs=0 l=1 rev1=1 crypto rx=0x7f9efc009b90 tx=0x7f9efc004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:59:13.813 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:13.800+0000 7f9f01ffb700 1 -- 172.21.15.59:0/1146381381 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9efc011630 con 0x7f9f0c0b4150 2023-11-24T07:59:13.814 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:13.800+0000 7f9f01ffb700 1 -- 172.21.15.59:0/1146381381 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9efc01f070 con 0x7f9f0c0b4150 2023-11-24T07:59:13.814 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:13.800+0000 7f9f13608700 1 -- 172.21.15.59:0/1146381381 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9f0c1f7480 con 0x7f9f0c0b4150 2023-11-24T07:59:13.814 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:13.800+0000 7f9f01ffb700 1 -- 172.21.15.59:0/1146381381 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9efc0119f0 con 0x7f9f0c0b4150 2023-11-24T07:59:13.814 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:13.800+0000 7f9f13608700 1 -- 172.21.15.59:0/1146381381 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9f0c1f7a40 con 0x7f9f0c0b4150 2023-11-24T07:59:13.814 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:13.801+0000 7f9f13608700 1 -- 172.21.15.59:0/1146381381 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9f0c143bb0 con 0x7f9f0c0b4150 2023-11-24T07:59:13.815 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:13.802+0000 7f9f01ffb700 1 -- 172.21.15.59:0/1146381381 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 10) v1 ==== 89588+0+0 (secure 0 0 0) 0x7f9efc011b50 con 0x7f9f0c0b4150 2023-11-24T07:59:13.815 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:13.802+0000 7f9f01ffb700 1 --2- 172.21.15.59:0/1146381381 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f9ef006b2c0 0x7f9ef006d780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:13.815 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:13.802+0000 7f9f01ffb700 1 -- 172.21.15.59:0/1146381381 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(72..72 src has 1..72) v4 ==== 5609+0+0 (secure 0 0 0) 0x7f9efc021070 con 0x7f9f0c0b4150 2023-11-24T07:59:13.815 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:13.803+0000 7f9f113a4700 1 --2- 172.21.15.59:0/1146381381 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f9ef006b2c0 0x7f9ef006d780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:13.816 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:13.803+0000 7f9f113a4700 1 --2- 172.21.15.59:0/1146381381 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f9ef006b2c0 0x7f9ef006d780 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f9f0c1e8b30 tx=0x7f9f08001e40 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:59:13.816 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:13.807+0000 7f9f01ffb700 1 -- 172.21.15.59:0/1146381381 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f9efc056180 con 0x7f9f0c0b4150 2023-11-24T07:59:14.054 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:14.052+0000 7f9f13608700 1 -- 172.21.15.59:0/1146381381 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7f9f0c075480 con 0x7f9f0c0b4150 2023-11-24T07:59:14.640 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:14.638+0000 7f9f01ffb700 1 -- 172.21.15.59:0/1146381381 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 10 and data pool 11 v40) v1 ==== 174+0+0 (secure 0 0 0) 0x7f9efc089e60 con 0x7f9f0c0b4150 2023-11-24T07:59:14.647 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:14.646+0000 7f9f13608700 1 -- 172.21.15.59:0/1146381381 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f9ef006b2c0 msgr2=0x7f9ef006d780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:14.648 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:14.646+0000 7f9f13608700 1 --2- 172.21.15.59:0/1146381381 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f9ef006b2c0 0x7f9ef006d780 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f9f0c1e8b30 tx=0x7f9f08001e40 comp rx=0 tx=0).stop 2023-11-24T07:59:14.648 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:14.646+0000 7f9f13608700 1 -- 172.21.15.59:0/1146381381 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9f0c0b4150 msgr2=0x7f9f0c1e4e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:14.648 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:14.646+0000 7f9f13608700 1 --2- 172.21.15.59:0/1146381381 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9f0c0b4150 0x7f9f0c1e4e90 secure :-1 s=READY pgs=852 cs=0 l=1 rev1=1 crypto rx=0x7f9efc009b90 tx=0x7f9efc004750 comp rx=0 tx=0).stop 2023-11-24T07:59:14.649 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:14.647+0000 7f9f13608700 1 -- 172.21.15.59:0/1146381381 shutdown_connections 2023-11-24T07:59:14.649 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:14.647+0000 7f9f13608700 1 --2- 172.21.15.59:0/1146381381 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f9ef006b2c0 0x7f9ef006d780 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:14.649 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:14.647+0000 7f9f13608700 1 --2- 172.21.15.59:0/1146381381 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f9f0c0b5ab0 0x7f9f0c1e53d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:14.649 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:14.647+0000 7f9f13608700 1 --2- 172.21.15.59:0/1146381381 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9f0c06b620 0x7f9f0c1e8f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:14.649 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:14.647+0000 7f9f13608700 1 --2- 172.21.15.59:0/1146381381 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9f0c0b4150 0x7f9f0c1e4e90 unknown :-1 s=CLOSED pgs=852 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:14.649 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:14.647+0000 7f9f13608700 1 -- 172.21.15.59:0/1146381381 >> 172.21.15.59:0/1146381381 conn(0x7f9f0c0b2740 msgr2=0x7f9f0c1463d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:59:14.649 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:14.647+0000 7f9f13608700 1 -- 172.21.15.59:0/1146381381 shutdown_connections 2023-11-24T07:59:14.649 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:14.647+0000 7f9f13608700 1 -- 172.21.15.59:0/1146381381 wait complete. 2023-11-24T07:59:14.665 INFO:teuthology.orchestra.run.smithi059.stderr:new fs with metadata pool 10 and data pool 11 2023-11-24T07:59:14.676 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-24T07:59:14.806 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:14.804+0000 7f85b475d700 1 Processor -- start 2023-11-24T07:59:14.806 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:14.804+0000 7f85b475d700 1 -- start start 2023-11-24T07:59:14.807 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:14.805+0000 7f85b475d700 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f85ac143ae0 0x7f85ac0b51b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:14.809 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:14.805+0000 7f85b475d700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f85ac0722c0 0x7f85ac06e510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:14.809 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:14.805+0000 7f85b475d700 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f85ac143ec0 con 0x7f85ac1440f0 2023-11-24T07:59:14.809 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:14.805+0000 7f85b475d700 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f85ac0b40a0 con 0x7f85ac0722c0 2023-11-24T07:59:14.809 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:14.805+0000 7f85b475d700 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f85ac0b3ae0 con 0x7f85ac143ae0 2023-11-24T07:59:14.809 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:14.806+0000 7f85b2cfa700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f85ac0722c0 0x7f85ac06e510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:14.809 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:14.806+0000 7f85b2cfa700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f85ac0722c0 0x7f85ac06e510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:54780/0 (socket says 172.21.15.59:54780) 2023-11-24T07:59:14.809 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:14.806+0000 7f85b1cf8700 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7f85ac1440f0 0x7f85ac071bb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:58256/0 (socket says 172.21.15.59:58256) 2023-11-24T07:59:14.809 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:14.806+0000 7f85b2cfa700 1 -- 172.21.15.59:0/3096874667 learned_addr learned my addr 172.21.15.59:0/3096874667 (peer_addr_for_me v2:172.21.15.59:0/0) 2023-11-24T07:59:14.809 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:14.806+0000 7f85b2cfa700 1 -- 172.21.15.59:0/3096874667 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f85ac143ae0 msgr2=0x7f85ac0b51b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T07:59:14.809 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:14.806+0000 7f85b24f9700 1 --2- 172.21.15.59:0/3096874667 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f85ac143ae0 0x7f85ac0b51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:14.809 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:14.806+0000 7f85b2cfa700 1 --2- 172.21.15.59:0/3096874667 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f85ac143ae0 0x7f85ac0b51b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:14.809 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:14.806+0000 7f85b14f7700 1 -- 172.21.15.59:0/3096874667 <== mon.0 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 799142842 0 0) 0x7f85ac143ec0 con 0x7f85ac1440f0 2023-11-24T07:59:14.810 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:14.806+0000 7f85b2cfa700 1 -- 172.21.15.59:0/3096874667 >> v1:172.21.15.59:6789/0 conn(0x7f85ac1440f0 legacy=0x7f85ac071bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:14.810 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:14.806+0000 7f85b2cfa700 1 -- 172.21.15.59:0/3096874667 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f85ac0b3dc0 con 0x7f85ac0722c0 2023-11-24T07:59:14.810 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:14.806+0000 7f85b24f9700 1 --2- 172.21.15.59:0/3096874667 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f85ac143ae0 0x7f85ac0b51b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T07:59:14.810 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:14.806+0000 7f85b2cfa700 1 --2- 172.21.15.59:0/3096874667 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f85ac0722c0 0x7f85ac06e510 secure :-1 s=READY pgs=854 cs=0 l=1 rev1=1 crypto rx=0x7f85a4009e50 tx=0x7f85a400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dde7057a5c98b904 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:59:14.810 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:14.806+0000 7f85b14f7700 1 -- 172.21.15.59:0/3096874667 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85a400e070 con 0x7f85ac0722c0 2023-11-24T07:59:14.810 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:14.806+0000 7f85b14f7700 1 -- 172.21.15.59:0/3096874667 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f85a400b9c0 con 0x7f85ac0722c0 2023-11-24T07:59:14.810 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:14.806+0000 7f85b14f7700 1 -- 172.21.15.59:0/3096874667 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85a400bd70 con 0x7f85ac0722c0 2023-11-24T07:59:14.810 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:14.807+0000 7f85b475d700 1 -- 172.21.15.59:0/3096874667 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f85ac0722c0 msgr2=0x7f85ac06e510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:14.810 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:14.807+0000 7f85b475d700 1 --2- 172.21.15.59:0/3096874667 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f85ac0722c0 0x7f85ac06e510 secure :-1 s=READY pgs=854 cs=0 l=1 rev1=1 crypto rx=0x7f85a4009e50 tx=0x7f85a400b330 comp rx=0 tx=0).stop 2023-11-24T07:59:14.810 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:14.807+0000 7f85b475d700 1 -- 172.21.15.59:0/3096874667 shutdown_connections 2023-11-24T07:59:14.810 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:14.807+0000 7f85b475d700 1 --2- 172.21.15.59:0/3096874667 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f85ac0722c0 0x7f85ac06e510 unknown :-1 s=CLOSED pgs=854 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:14.810 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:14.807+0000 7f85b475d700 1 --2- 172.21.15.59:0/3096874667 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f85ac143ae0 0x7f85ac0b51b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:14.810 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:14.807+0000 7f85b475d700 1 -- 172.21.15.59:0/3096874667 >> 172.21.15.59:0/3096874667 conn(0x7f85ac0b2740 msgr2=0x7f85ac0b2b20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:59:14.811 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:14.807+0000 7f85b475d700 1 -- 172.21.15.59:0/3096874667 shutdown_connections 2023-11-24T07:59:14.811 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:14.807+0000 7f85b475d700 1 -- 172.21.15.59:0/3096874667 wait complete. 2023-11-24T07:59:14.813 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:14.808+0000 7f85b475d700 1 Processor -- start 2023-11-24T07:59:14.813 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:14.808+0000 7f85b475d700 1 -- start start 2023-11-24T07:59:14.813 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:14.809+0000 7f85b475d700 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f85ac0722c0 0x7f85ac1f35f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:14.813 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:14.809+0000 7f85b475d700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f85ac143ae0 0x7f85ac156330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:14.813 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:14.809+0000 7f85b475d700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f85ac1440f0 0x7f85ac0cc6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:14.813 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:14.809+0000 7f85b475d700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85ac0cb390 con 0x7f85ac143ae0 2023-11-24T07:59:14.813 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:14.809+0000 7f85b475d700 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85ac0cb610 con 0x7f85ac1440f0 2023-11-24T07:59:14.814 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:14.809+0000 7f85b475d700 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f85ac0cb8f0 con 0x7f85ac0722c0 2023-11-24T07:59:14.814 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:14.809+0000 7f85b1cf8700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f85ac143ae0 0x7f85ac156330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:14.814 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:14.809+0000 7f85b1cf8700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f85ac143ae0 0x7f85ac156330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:54786/0 (socket says 172.21.15.59:54786) 2023-11-24T07:59:14.814 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:14.809+0000 7f85b1cf8700 1 -- 172.21.15.59:0/621385716 learned_addr learned my addr 172.21.15.59:0/621385716 (peer_addr_for_me v2:172.21.15.59:0/0) 2023-11-24T07:59:14.814 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:14.809+0000 7f85b2cfa700 1 --2- 172.21.15.59:0/621385716 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f85ac1440f0 0x7f85ac0cc6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:14.814 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:14.809+0000 7f85b24f9700 1 --2- 172.21.15.59:0/621385716 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f85ac0722c0 0x7f85ac1f35f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:14.814 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:14.809+0000 7f85b1cf8700 1 -- 172.21.15.59:0/621385716 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f85ac0722c0 msgr2=0x7f85ac1f35f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:14.814 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:14.809+0000 7f85b1cf8700 1 --2- 172.21.15.59:0/621385716 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f85ac0722c0 0x7f85ac1f35f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:14.814 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:14.809+0000 7f85b1cf8700 1 -- 172.21.15.59:0/621385716 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f85ac1440f0 msgr2=0x7f85ac0cc6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:14.814 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:14.809+0000 7f85b1cf8700 1 --2- 172.21.15.59:0/621385716 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f85ac1440f0 0x7f85ac0cc6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:14.814 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:14.809+0000 7f85b1cf8700 1 -- 172.21.15.59:0/621385716 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f85a4009b30 con 0x7f85ac143ae0 2023-11-24T07:59:14.814 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:14.810+0000 7f85b1cf8700 1 --2- 172.21.15.59:0/621385716 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f85ac143ae0 0x7f85ac156330 secure :-1 s=READY pgs=855 cs=0 l=1 rev1=1 crypto rx=0x7f859c00c550 tx=0x7f859c00c860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:59:14.814 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:14.810+0000 7f85a2ffd700 1 -- 172.21.15.59:0/621385716 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f859c004c80 con 0x7f85ac143ae0 2023-11-24T07:59:14.814 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:14.810+0000 7f85a2ffd700 1 -- 172.21.15.59:0/621385716 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f859c004e00 con 0x7f85ac143ae0 2023-11-24T07:59:14.815 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:14.810+0000 7f85b475d700 1 -- 172.21.15.59:0/621385716 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f85ac0cbbd0 con 0x7f85ac143ae0 2023-11-24T07:59:14.815 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:14.810+0000 7f85a2ffd700 1 -- 172.21.15.59:0/621385716 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f859c007310 con 0x7f85ac143ae0 2023-11-24T07:59:14.815 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:14.810+0000 7f85b475d700 1 -- 172.21.15.59:0/621385716 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f85ac0cc190 con 0x7f85ac143ae0 2023-11-24T07:59:14.815 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:14.811+0000 7f85b475d700 1 -- 172.21.15.59:0/621385716 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f85ac1447e0 con 0x7f85ac143ae0 2023-11-24T07:59:14.815 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:14.811+0000 7f85a2ffd700 1 -- 172.21.15.59:0/621385716 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 10) v1 ==== 89588+0+0 (secure 0 0 0) 0x7f859c007470 con 0x7f85ac143ae0 2023-11-24T07:59:14.815 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:14.812+0000 7f85a2ffd700 1 --2- 172.21.15.59:0/621385716 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f859006b2c0 0x7f859006d780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:14.815 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:14.812+0000 7f85a2ffd700 1 -- 172.21.15.59:0/621385716 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(73..73 src has 1..73) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f859c0898c0 con 0x7f85ac143ae0 2023-11-24T07:59:14.815 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:14.812+0000 7f85b24f9700 1 --2- 172.21.15.59:0/621385716 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f859006b2c0 0x7f859006d780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:14.816 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:14.812+0000 7f85b24f9700 1 --2- 172.21.15.59:0/621385716 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f859006b2c0 0x7f859006d780 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f85a8006fd0 tx=0x7f85a8008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:59:14.818 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:14.816+0000 7f85a2ffd700 1 -- 172.21.15.59:0/621385716 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f859c0544a0 con 0x7f85ac143ae0 2023-11-24T07:59:15.019 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:15.018+0000 7f85b475d700 1 -- 172.21.15.59:0/621385716 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f85ac0cc3b0 con 0x7f85ac143ae0 2023-11-24T07:59:15.020 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:15.019+0000 7f85a2ffd700 1 -- 172.21.15.59:0/621385716 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v73) v1 ==== 74+0+16449 (secure 0 0 0) 0x7f859c057ac0 con 0x7f85ac143ae0 2023-11-24T07:59:15.021 INFO:teuthology.orchestra.run.smithi059.stdout: 2023-11-24T07:59:15.021 INFO:teuthology.orchestra.run.smithi059.stdout:{"epoch":73,"fsid":"9f0e8741-b8f9-48d5-a2c3-09f45903303d","created":"2023-11-24T07:56:06.552729+0000","modified":"2023-11-24T07:59:14.053980+0000","last_up_change":"2023-11-24T07:56:17.372352+0000","last_in_change":"2023-11-24T07:56:09.073250+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":11,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-24T07:56:18.352823+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_metadata","create_time":"2023-11-24T07:59:11.900882+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"73","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_data","create_time":"2023-11-24T07:59:13.025749+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"73","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"6d8f9e9b-1782-4e41-928b-a39f2a8ebc06","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6800","nonce":3373878772},{"type":"v1","addr":"172.21.15.59:6801","nonce":3373878772}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6802","nonce":3373878772},{"type":"v1","addr":"172.21.15.59:6803","nonce":3373878772}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6806","nonce":3373878772},{"type":"v1","addr":"172.21.15.59:6807","nonce":3373878772}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6804","nonce":3373878772},{"type":"v1","addr":"172.21.15.59:6805","nonce":3373878772}]},"public_addr":"172.21.15.59:6801/3373878772","cluster_addr":"172.21.15.59:6803/3373878772","heartbeat_back_addr":"172.21.15.59:6807/3373878772","heartbeat_front_addr":"172.21.15.59:6805/3373878772","state":["exists","up"]},{"osd":1,"uuid":"e8d882aa-52e1-4c04-9e1e-c44ef9204a5f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6824","nonce":648258563},{"type":"v1","addr":"172.21.15.59:6825","nonce":648258563}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6826","nonce":648258563},{"type":"v1","addr":"172.21.15.59:6827","nonce":648258563}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6830","nonce":648258563},{"type":"v1","addr":"172.21.15.59:6831","nonce":648258563}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6828","nonce":648258563},{"type":"v1","addr":"172.21.15.59:6829","nonce":648258563}]},"public_addr":"172.21.15.59:6825/648258563","cluster_addr":"172.21.15.59:6827/648258563","heartbeat_back_addr":"172.21.15.59:6831/648258563","heartbeat_front_addr":"172.21.15.59:6829/648258563","state":["exists","up"]},{"osd":2,"uuid":"69eebfa0-695b-4b21-90b9-8a3537168a01","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6808","nonce":1840430654},{"type":"v1","addr":"172.21.15.59:6809","nonce":1840430654}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6810","nonce":1840430654},{"type":"v1","addr":"172.21.15.59:6811","nonce":1840430654}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6814","nonce":1840430654},{"type":"v1","addr":"172.21.15.59:6815","nonce":1840430654}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6812","nonce":1840430654},{"type":"v1","addr":"172.21.15.59:6813","nonce":1840430654}]},"public_addr":"172.21.15.59:6809/1840430654","cluster_addr":"172.21.15.59:6811/1840430654","heartbeat_back_addr":"172.21.15.59:6815/1840430654","heartbeat_front_addr":"172.21.15.59:6813/1840430654","state":["exists","up"]},{"osd":3,"uuid":"96fd77fc-f911-46ff-9445-6f0be2daa8f4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6816","nonce":2504025850},{"type":"v1","addr":"172.21.15.59:6817","nonce":2504025850}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6818","nonce":2504025850},{"type":"v1","addr":"172.21.15.59:6819","nonce":2504025850}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6822","nonce":2504025850},{"type":"v1","addr":"172.21.15.59:6823","nonce":2504025850}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6820","nonce":2504025850},{"type":"v1","addr":"172.21.15.59:6821","nonce":2504025850}]},"public_addr":"172.21.15.59:6817/2504025850","cluster_addr":"172.21.15.59:6819/2504025850","heartbeat_back_addr":"172.21.15.59:6823/2504025850","heartbeat_front_addr":"172.21.15.59:6821/2504025850","state":["exists","up"]},{"osd":4,"uuid":"7ac29fca-b5b2-4c95-9d69-2f164cbbc708","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6816","nonce":2382463431},{"type":"v1","addr":"172.21.15.76:6818","nonce":2382463431}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6820","nonce":2382463431},{"type":"v1","addr":"172.21.15.76:6822","nonce":2382463431}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6828","nonce":2382463431},{"type":"v1","addr":"172.21.15.76:6830","nonce":2382463431}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6824","nonce":2382463431},{"type":"v1","addr":"172.21.15.76:6826","nonce":2382463431}]},"public_addr":"172.21.15.76:6818/2382463431","cluster_addr":"172.21.15.76:6822/2382463431","heartbeat_back_addr":"172.21.15.76:6830/2382463431","heartbeat_front_addr":"172.21.15.76:6826/2382463431","state":["exists","up"]},{"osd":5,"uuid":"485bf711-3851-4238-a4f4-51d52e0a9c58","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6800","nonce":329334308},{"type":"v1","addr":"172.21.15.76:6802","nonce":329334308}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6805","nonce":329334308},{"type":"v1","addr":"172.21.15.76:6807","nonce":329334308}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6814","nonce":329334308},{"type":"v1","addr":"172.21.15.76:6815","nonce":329334308}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6810","nonce":329334308},{"type":"v1","addr":"172.21.15.76:6812","nonce":329334308}]},"public_addr":"172.21.15.76:6802/329334308","cluster_addr":"172.21.15.76:6807/329334308","heartbeat_back_addr":"172.21.15.76:6815/329334308","heartbeat_front_addr":"172.21.15.76:6812/329334308","state":["exists","up"]},{"osd":6,"uuid":"72b3f2fc-1e4a-4e32-968d-003647a0286f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6801","nonce":3946579410},{"type":"v1","addr":"172.21.15.76:6803","nonce":3946579410}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6804","nonce":3946579410},{"type":"v1","addr":"172.21.15.76:6806","nonce":3946579410}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6811","nonce":3946579410},{"type":"v1","addr":"172.21.15.76:6813","nonce":3946579410}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6808","nonce":3946579410},{"type":"v1","addr":"172.21.15.76:6809","nonce":3946579410}]},"public_addr":"172.21.15.76:6803/3946579410","cluster_addr":"172.21.15.76:6806/3946579410","heartbeat_back_addr":"172.21.15.76:6813/3946579410","heartbeat_front_addr":"172.21.15.76:6809/3946579410","state":["exists","up"]},{"osd":7,"uuid":"1ddceb90-6c5d-4d1a-ad56-99cb877f8c70","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6817","nonce":841761538},{"type":"v1","addr":"172.21.15.76:6819","nonce":841761538}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6821","nonce":841761538},{"type":"v1","addr":"172.21.15.76:6823","nonce":841761538}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6829","nonce":841761538},{"type":"v1","addr":"172.21.15.76:6831","nonce":841761538}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6825","nonce":841761538},{"type":"v1","addr":"172.21.15.76:6827","nonce":841761538}]},"public_addr":"172.21.15.76:6819/841761538","cluster_addr":"172.21.15.76:6823/841761538","heartbeat_back_addr":"172.21.15.76:6831/841761538","heartbeat_front_addr":"172.21.15.76:6827/841761538","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T07:56:14.294838+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T07:56:14.270779+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T07:56:14.231809+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T07:56:14.260983+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T07:56:14.302997+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T07:56:14.258600+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T07:56:14.251413+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T07:56:14.304727+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-24T07:59:15.023 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:15.021+0000 7f85b475d700 1 -- 172.21.15.59:0/621385716 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f859006b2c0 msgr2=0x7f859006d780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:15.023 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:15.021+0000 7f85b475d700 1 --2- 172.21.15.59:0/621385716 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f859006b2c0 0x7f859006d780 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f85a8006fd0 tx=0x7f85a8008040 comp rx=0 tx=0).stop 2023-11-24T07:59:15.023 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:15.021+0000 7f85b475d700 1 -- 172.21.15.59:0/621385716 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f85ac143ae0 msgr2=0x7f85ac156330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:15.023 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:15.021+0000 7f85b475d700 1 --2- 172.21.15.59:0/621385716 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f85ac143ae0 0x7f85ac156330 secure :-1 s=READY pgs=855 cs=0 l=1 rev1=1 crypto rx=0x7f859c00c550 tx=0x7f859c00c860 comp rx=0 tx=0).stop 2023-11-24T07:59:15.023 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:15.021+0000 7f85b475d700 1 -- 172.21.15.59:0/621385716 shutdown_connections 2023-11-24T07:59:15.024 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:15.021+0000 7f85b475d700 1 --2- 172.21.15.59:0/621385716 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f859006b2c0 0x7f859006d780 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:15.024 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:15.021+0000 7f85b475d700 1 --2- 172.21.15.59:0/621385716 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f85ac1440f0 0x7f85ac0cc6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:15.024 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:15.021+0000 7f85b475d700 1 --2- 172.21.15.59:0/621385716 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f85ac0722c0 0x7f85ac1f35f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:15.024 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:15.021+0000 7f85b475d700 1 --2- 172.21.15.59:0/621385716 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f85ac143ae0 0x7f85ac156330 unknown :-1 s=CLOSED pgs=855 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:15.024 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:15.021+0000 7f85b475d700 1 -- 172.21.15.59:0/621385716 >> 172.21.15.59:0/621385716 conn(0x7f85ac0b2740 msgr2=0x7f85ac0cf530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:59:15.024 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:15.021+0000 7f85b475d700 1 -- 172.21.15.59:0/621385716 shutdown_connections 2023-11-24T07:59:15.024 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:15.021+0000 7f85b475d700 1 -- 172.21.15.59:0/621385716 wait complete. 2023-11-24T07:59:15.036 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-24T07:59:15.161 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:15.159+0000 7f576b229700 1 Processor -- start 2023-11-24T07:59:15.161 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:15.159+0000 7f576b229700 1 -- start start 2023-11-24T07:59:15.164 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:15.160+0000 7f576b229700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f576415e260 0x7f576406eef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:15.164 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:15.160+0000 7f576b229700 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f57640d09e0 0x7f576406f430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:15.164 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:15.160+0000 7f576b229700 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f57640d0f50 con 0x7f576406f970 2023-11-24T07:59:15.164 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:15.160+0000 7f576b229700 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f576406fdc0 con 0x7f576415e260 2023-11-24T07:59:15.165 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:15.160+0000 7f576b229700 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f57640d1230 con 0x7f57640d09e0 2023-11-24T07:59:15.165 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:15.160+0000 7f5768fc5700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f576415e260 0x7f576406eef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:15.165 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:15.160+0000 7f5768fc5700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f576415e260 0x7f576406eef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:54792/0 (socket says 172.21.15.59:54792) 2023-11-24T07:59:15.165 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:15.160+0000 7f5768fc5700 1 -- 172.21.15.59:0/3960859156 learned_addr learned my addr 172.21.15.59:0/3960859156 (peer_addr_for_me v2:172.21.15.59:0/0) 2023-11-24T07:59:15.165 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:15.161+0000 7f5763fff700 1 --2- 172.21.15.59:0/3960859156 >> v2:172.21.15.76:3300/0 conn(0x7f57640d09e0 0x7f576406f430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:15.165 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:15.161+0000 7f5768fc5700 1 -- 172.21.15.59:0/3960859156 >> v2:172.21.15.76:3300/0 conn(0x7f57640d09e0 msgr2=0x7f576406f430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T07:59:15.165 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:15.161+0000 7f5768fc5700 1 --2- 172.21.15.59:0/3960859156 >> v2:172.21.15.76:3300/0 conn(0x7f57640d09e0 0x7f576406f430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:15.165 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:15.161+0000 7f5768fc5700 1 -- 172.21.15.59:0/3960859156 >> v1:172.21.15.76:6789/0 conn(0x7f576406f970 legacy=0x7f57640ce480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:15.166 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:15.161+0000 7f5768fc5700 1 -- 172.21.15.59:0/3960859156 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f57640cc2f0 con 0x7f576415e260 2023-11-24T07:59:15.166 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:15.161+0000 7f5768fc5700 1 --2- 172.21.15.59:0/3960859156 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f576415e260 0x7f576406eef0 secure :-1 s=READY pgs=856 cs=0 l=1 rev1=1 crypto rx=0x7f576406bfb0 tx=0x7f5758009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=895b1a07d2a9b51a server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:59:15.166 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:15.161+0000 7f57637fe700 1 -- 172.21.15.59:0/3960859156 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5758011830 con 0x7f576415e260 2023-11-24T07:59:15.166 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:15.161+0000 7f57637fe700 1 -- 172.21.15.59:0/3960859156 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5758011990 con 0x7f576415e260 2023-11-24T07:59:15.166 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:15.161+0000 7f57637fe700 1 -- 172.21.15.59:0/3960859156 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5758011d20 con 0x7f576415e260 2023-11-24T07:59:15.166 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:15.162+0000 7f576b229700 1 -- 172.21.15.59:0/3960859156 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f576415e260 msgr2=0x7f576406eef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:15.167 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:15.162+0000 7f576b229700 1 --2- 172.21.15.59:0/3960859156 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f576415e260 0x7f576406eef0 secure :-1 s=READY pgs=856 cs=0 l=1 rev1=1 crypto rx=0x7f576406bfb0 tx=0x7f5758009bd0 comp rx=0 tx=0).stop 2023-11-24T07:59:15.167 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:15.162+0000 7f576b229700 1 -- 172.21.15.59:0/3960859156 shutdown_connections 2023-11-24T07:59:15.168 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:15.162+0000 7f576b229700 1 --2- 172.21.15.59:0/3960859156 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f576415e260 0x7f576406eef0 unknown :-1 s=CLOSED pgs=856 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:15.168 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:15.162+0000 7f576b229700 1 --2- 172.21.15.59:0/3960859156 >> v2:172.21.15.76:3300/0 conn(0x7f57640d09e0 0x7f576406f430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:15.168 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:15.162+0000 7f576b229700 1 -- 172.21.15.59:0/3960859156 >> 172.21.15.59:0/3960859156 conn(0x7f57640b3790 msgr2=0x7f57640b3ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:59:15.168 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:15.162+0000 7f576b229700 1 -- 172.21.15.59:0/3960859156 shutdown_connections 2023-11-24T07:59:15.168 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:15.162+0000 7f576b229700 1 -- 172.21.15.59:0/3960859156 wait complete. 2023-11-24T07:59:15.171 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:15.163+0000 7f576b229700 1 Processor -- start 2023-11-24T07:59:15.171 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:15.163+0000 7f576b229700 1 -- start start 2023-11-24T07:59:15.171 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:15.164+0000 7f576b229700 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f576406f970 0x7f57640cb1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:15.171 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:15.164+0000 7f576b229700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f57640d09e0 0x7f57640b46e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:15.171 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:15.164+0000 7f576b229700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f576415e260 0x7f57640b7020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:15.171 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:15.164+0000 7f576b229700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57640cada0 con 0x7f57640d09e0 2023-11-24T07:59:15.171 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:15.164+0000 7f576b229700 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57640ca120 con 0x7f576415e260 2023-11-24T07:59:15.172 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:15.164+0000 7f576b229700 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f57640ca400 con 0x7f576406f970 2023-11-24T07:59:15.172 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:15.164+0000 7f5763fff700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f57640d09e0 0x7f57640b46e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:15.172 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:15.164+0000 7f5763fff700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f57640d09e0 0x7f57640b46e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:54796/0 (socket says 172.21.15.59:54796) 2023-11-24T07:59:15.172 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:15.164+0000 7f5763fff700 1 -- 172.21.15.59:0/1012992547 learned_addr learned my addr 172.21.15.59:0/1012992547 (peer_addr_for_me v2:172.21.15.59:0/0) 2023-11-24T07:59:15.172 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:15.164+0000 7f57697c6700 1 --2- 172.21.15.59:0/1012992547 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f576415e260 0x7f57640b7020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:15.172 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:15.164+0000 7f5768fc5700 1 --2- 172.21.15.59:0/1012992547 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f576406f970 0x7f57640cb1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:15.172 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:15.164+0000 7f5763fff700 1 -- 172.21.15.59:0/1012992547 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f576406f970 msgr2=0x7f57640cb1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:15.172 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:15.164+0000 7f5763fff700 1 --2- 172.21.15.59:0/1012992547 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f576406f970 0x7f57640cb1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:15.172 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:15.164+0000 7f5763fff700 1 -- 172.21.15.59:0/1012992547 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f576415e260 msgr2=0x7f57640b7020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:15.172 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:15.164+0000 7f5763fff700 1 --2- 172.21.15.59:0/1012992547 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f576415e260 0x7f57640b7020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:15.172 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:15.164+0000 7f5763fff700 1 -- 172.21.15.59:0/1012992547 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5758009550 con 0x7f57640d09e0 2023-11-24T07:59:15.172 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:15.164+0000 7f57697c6700 1 --2- 172.21.15.59:0/1012992547 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f576415e260 0x7f57640b7020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T07:59:15.172 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:15.164+0000 7f5763fff700 1 --2- 172.21.15.59:0/1012992547 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f57640d09e0 0x7f57640b46e0 secure :-1 s=READY pgs=857 cs=0 l=1 rev1=1 crypto rx=0x7f575000e9b0 tx=0x7f575000ed70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:59:15.173 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:15.165+0000 7f57617fa700 1 -- 172.21.15.59:0/1012992547 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5750009710 con 0x7f57640d09e0 2023-11-24T07:59:15.173 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:15.165+0000 7f57617fa700 1 -- 172.21.15.59:0/1012992547 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f575000c850 con 0x7f57640d09e0 2023-11-24T07:59:15.173 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:15.165+0000 7f576b229700 1 -- 172.21.15.59:0/1012992547 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f57640ca6e0 con 0x7f57640d09e0 2023-11-24T07:59:15.173 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:15.165+0000 7f57617fa700 1 -- 172.21.15.59:0/1012992547 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f575000cbc0 con 0x7f57640d09e0 2023-11-24T07:59:15.173 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:15.165+0000 7f576b229700 1 -- 172.21.15.59:0/1012992547 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f57640c9d20 con 0x7f57640d09e0 2023-11-24T07:59:15.173 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:15.166+0000 7f576b229700 1 -- 172.21.15.59:0/1012992547 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f57640caee0 con 0x7f57640d09e0 2023-11-24T07:59:15.173 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:15.166+0000 7f57617fa700 1 -- 172.21.15.59:0/1012992547 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 10) v1 ==== 89588+0+0 (secure 0 0 0) 0x7f575000cd20 con 0x7f57640d09e0 2023-11-24T07:59:15.173 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:15.166+0000 7f57617fa700 1 --2- 172.21.15.59:0/1012992547 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f574806b2c0 0x7f574806d780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:15.173 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:15.167+0000 7f57617fa700 1 -- 172.21.15.59:0/1012992547 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(73..73 src has 1..73) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f5750089eb0 con 0x7f57640d09e0 2023-11-24T07:59:15.173 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:15.167+0000 7f5768fc5700 1 --2- 172.21.15.59:0/1012992547 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f574806b2c0 0x7f574806d780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:15.173 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:15.167+0000 7f5768fc5700 1 --2- 172.21.15.59:0/1012992547 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f574806b2c0 0x7f574806d780 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f5758014040 tx=0x7f5758008030 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:59:15.174 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:15.171+0000 7f57617fa700 1 -- 172.21.15.59:0/1012992547 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f57640caee0 con 0x7f57640d09e0 2023-11-24T07:59:15.376 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:15.374+0000 7f576b229700 1 -- 172.21.15.59:0/1012992547 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f57640caee0 con 0x7f57640d09e0 2023-11-24T07:59:15.377 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:15.375+0000 7f57617fa700 1 -- 172.21.15.59:0/1012992547 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v73) v1 ==== 74+0+16449 (secure 0 0 0) 0x7f57640caee0 con 0x7f57640d09e0 2023-11-24T07:59:15.377 INFO:teuthology.orchestra.run.smithi059.stdout: 2023-11-24T07:59:15.377 INFO:teuthology.orchestra.run.smithi059.stdout:{"epoch":73,"fsid":"9f0e8741-b8f9-48d5-a2c3-09f45903303d","created":"2023-11-24T07:56:06.552729+0000","modified":"2023-11-24T07:59:14.053980+0000","last_up_change":"2023-11-24T07:56:17.372352+0000","last_in_change":"2023-11-24T07:56:09.073250+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":11,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-24T07:56:18.352823+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_metadata","create_time":"2023-11-24T07:59:11.900882+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"73","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_data","create_time":"2023-11-24T07:59:13.025749+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"73","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"6d8f9e9b-1782-4e41-928b-a39f2a8ebc06","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6800","nonce":3373878772},{"type":"v1","addr":"172.21.15.59:6801","nonce":3373878772}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6802","nonce":3373878772},{"type":"v1","addr":"172.21.15.59:6803","nonce":3373878772}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6806","nonce":3373878772},{"type":"v1","addr":"172.21.15.59:6807","nonce":3373878772}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6804","nonce":3373878772},{"type":"v1","addr":"172.21.15.59:6805","nonce":3373878772}]},"public_addr":"172.21.15.59:6801/3373878772","cluster_addr":"172.21.15.59:6803/3373878772","heartbeat_back_addr":"172.21.15.59:6807/3373878772","heartbeat_front_addr":"172.21.15.59:6805/3373878772","state":["exists","up"]},{"osd":1,"uuid":"e8d882aa-52e1-4c04-9e1e-c44ef9204a5f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6824","nonce":648258563},{"type":"v1","addr":"172.21.15.59:6825","nonce":648258563}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6826","nonce":648258563},{"type":"v1","addr":"172.21.15.59:6827","nonce":648258563}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6830","nonce":648258563},{"type":"v1","addr":"172.21.15.59:6831","nonce":648258563}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6828","nonce":648258563},{"type":"v1","addr":"172.21.15.59:6829","nonce":648258563}]},"public_addr":"172.21.15.59:6825/648258563","cluster_addr":"172.21.15.59:6827/648258563","heartbeat_back_addr":"172.21.15.59:6831/648258563","heartbeat_front_addr":"172.21.15.59:6829/648258563","state":["exists","up"]},{"osd":2,"uuid":"69eebfa0-695b-4b21-90b9-8a3537168a01","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6808","nonce":1840430654},{"type":"v1","addr":"172.21.15.59:6809","nonce":1840430654}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6810","nonce":1840430654},{"type":"v1","addr":"172.21.15.59:6811","nonce":1840430654}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6814","nonce":1840430654},{"type":"v1","addr":"172.21.15.59:6815","nonce":1840430654}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6812","nonce":1840430654},{"type":"v1","addr":"172.21.15.59:6813","nonce":1840430654}]},"public_addr":"172.21.15.59:6809/1840430654","cluster_addr":"172.21.15.59:6811/1840430654","heartbeat_back_addr":"172.21.15.59:6815/1840430654","heartbeat_front_addr":"172.21.15.59:6813/1840430654","state":["exists","up"]},{"osd":3,"uuid":"96fd77fc-f911-46ff-9445-6f0be2daa8f4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6816","nonce":2504025850},{"type":"v1","addr":"172.21.15.59:6817","nonce":2504025850}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6818","nonce":2504025850},{"type":"v1","addr":"172.21.15.59:6819","nonce":2504025850}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6822","nonce":2504025850},{"type":"v1","addr":"172.21.15.59:6823","nonce":2504025850}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6820","nonce":2504025850},{"type":"v1","addr":"172.21.15.59:6821","nonce":2504025850}]},"public_addr":"172.21.15.59:6817/2504025850","cluster_addr":"172.21.15.59:6819/2504025850","heartbeat_back_addr":"172.21.15.59:6823/2504025850","heartbeat_front_addr":"172.21.15.59:6821/2504025850","state":["exists","up"]},{"osd":4,"uuid":"7ac29fca-b5b2-4c95-9d69-2f164cbbc708","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6816","nonce":2382463431},{"type":"v1","addr":"172.21.15.76:6818","nonce":2382463431}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6820","nonce":2382463431},{"type":"v1","addr":"172.21.15.76:6822","nonce":2382463431}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6828","nonce":2382463431},{"type":"v1","addr":"172.21.15.76:6830","nonce":2382463431}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6824","nonce":2382463431},{"type":"v1","addr":"172.21.15.76:6826","nonce":2382463431}]},"public_addr":"172.21.15.76:6818/2382463431","cluster_addr":"172.21.15.76:6822/2382463431","heartbeat_back_addr":"172.21.15.76:6830/2382463431","heartbeat_front_addr":"172.21.15.76:6826/2382463431","state":["exists","up"]},{"osd":5,"uuid":"485bf711-3851-4238-a4f4-51d52e0a9c58","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6800","nonce":329334308},{"type":"v1","addr":"172.21.15.76:6802","nonce":329334308}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6805","nonce":329334308},{"type":"v1","addr":"172.21.15.76:6807","nonce":329334308}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6814","nonce":329334308},{"type":"v1","addr":"172.21.15.76:6815","nonce":329334308}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6810","nonce":329334308},{"type":"v1","addr":"172.21.15.76:6812","nonce":329334308}]},"public_addr":"172.21.15.76:6802/329334308","cluster_addr":"172.21.15.76:6807/329334308","heartbeat_back_addr":"172.21.15.76:6815/329334308","heartbeat_front_addr":"172.21.15.76:6812/329334308","state":["exists","up"]},{"osd":6,"uuid":"72b3f2fc-1e4a-4e32-968d-003647a0286f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6801","nonce":3946579410},{"type":"v1","addr":"172.21.15.76:6803","nonce":3946579410}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6804","nonce":3946579410},{"type":"v1","addr":"172.21.15.76:6806","nonce":3946579410}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6811","nonce":3946579410},{"type":"v1","addr":"172.21.15.76:6813","nonce":3946579410}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6808","nonce":3946579410},{"type":"v1","addr":"172.21.15.76:6809","nonce":3946579410}]},"public_addr":"172.21.15.76:6803/3946579410","cluster_addr":"172.21.15.76:6806/3946579410","heartbeat_back_addr":"172.21.15.76:6813/3946579410","heartbeat_front_addr":"172.21.15.76:6809/3946579410","state":["exists","up"]},{"osd":7,"uuid":"1ddceb90-6c5d-4d1a-ad56-99cb877f8c70","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6817","nonce":841761538},{"type":"v1","addr":"172.21.15.76:6819","nonce":841761538}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6821","nonce":841761538},{"type":"v1","addr":"172.21.15.76:6823","nonce":841761538}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6829","nonce":841761538},{"type":"v1","addr":"172.21.15.76:6831","nonce":841761538}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6825","nonce":841761538},{"type":"v1","addr":"172.21.15.76:6827","nonce":841761538}]},"public_addr":"172.21.15.76:6819/841761538","cluster_addr":"172.21.15.76:6823/841761538","heartbeat_back_addr":"172.21.15.76:6831/841761538","heartbeat_front_addr":"172.21.15.76:6827/841761538","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T07:56:14.294838+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T07:56:14.270779+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T07:56:14.231809+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T07:56:14.260983+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T07:56:14.302997+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T07:56:14.258600+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T07:56:14.251413+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T07:56:14.304727+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-24T07:59:15.379 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:15.377+0000 7f576b229700 1 -- 172.21.15.59:0/1012992547 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f574806b2c0 msgr2=0x7f574806d780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:15.379 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:15.377+0000 7f576b229700 1 --2- 172.21.15.59:0/1012992547 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f574806b2c0 0x7f574806d780 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f5758014040 tx=0x7f5758008030 comp rx=0 tx=0).stop 2023-11-24T07:59:15.379 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:15.377+0000 7f576b229700 1 -- 172.21.15.59:0/1012992547 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f57640d09e0 msgr2=0x7f57640b46e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:15.380 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:15.377+0000 7f576b229700 1 --2- 172.21.15.59:0/1012992547 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f57640d09e0 0x7f57640b46e0 secure :-1 s=READY pgs=857 cs=0 l=1 rev1=1 crypto rx=0x7f575000e9b0 tx=0x7f575000ed70 comp rx=0 tx=0).stop 2023-11-24T07:59:15.380 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:15.378+0000 7f576b229700 1 -- 172.21.15.59:0/1012992547 shutdown_connections 2023-11-24T07:59:15.380 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:15.378+0000 7f576b229700 1 --2- 172.21.15.59:0/1012992547 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f574806b2c0 0x7f574806d780 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:15.380 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:15.378+0000 7f576b229700 1 --2- 172.21.15.59:0/1012992547 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f576415e260 0x7f57640b7020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:15.380 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:15.378+0000 7f576b229700 1 --2- 172.21.15.59:0/1012992547 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f576406f970 0x7f57640cb1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:15.380 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:15.378+0000 7f576b229700 1 --2- 172.21.15.59:0/1012992547 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f57640d09e0 0x7f57640b46e0 unknown :-1 s=CLOSED pgs=857 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:15.380 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:15.378+0000 7f576b229700 1 -- 172.21.15.59:0/1012992547 >> 172.21.15.59:0/1012992547 conn(0x7f57640b3790 msgr2=0x7f57640b0a80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:59:15.380 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:15.378+0000 7f576b229700 1 -- 172.21.15.59:0/1012992547 shutdown_connections 2023-11-24T07:59:15.380 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:15.378+0000 7f576b229700 1 -- 172.21.15.59:0/1012992547 wait complete. 2023-11-24T07:59:15.392 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2023-11-24T07:59:15.521 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:15.519+0000 7f4785b23700 1 Processor -- start 2023-11-24T07:59:15.521 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:15.519+0000 7f4785b23700 1 -- start start 2023-11-24T07:59:15.523 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:15.520+0000 7f4785b23700 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f478006b400 0x7f478015e5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:15.523 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:15.520+0000 7f4785b23700 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f47800b5e40 0x7f478015eb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:15.523 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:15.520+0000 7f4785b23700 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f478006bb80 con 0x7f478015e1e0 2023-11-24T07:59:15.524 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:15.520+0000 7f4785b23700 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f478006be20 con 0x7f47800b5e40 2023-11-24T07:59:15.524 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:15.520+0000 7f4785b23700 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f478006c100 con 0x7f478006b400 2023-11-24T07:59:15.524 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:15.520+0000 7f477ffff700 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f47800b5e40 0x7f478015eb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:15.524 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:15.520+0000 7f477effd700 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f478006b400 0x7f478015e5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:15.525 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:15.521+0000 7f477f7fe700 1 --1- >> v1:172.21.15.76:6789/0 conn(0x7f478015e1e0 0x7f47800b12a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.59:45556/0 (socket says 172.21.15.59:45556) 2023-11-24T07:59:15.525 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:15.521+0000 7f477effd700 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f478006b400 0x7f478015e5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3301/0 says I am v2:172.21.15.59:54246/0 (socket says 172.21.15.59:54246) 2023-11-24T07:59:15.525 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:15.521+0000 7f477ffff700 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f47800b5e40 0x7f478015eb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.59:42006/0 (socket says 172.21.15.59:42006) 2023-11-24T07:59:15.525 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:15.521+0000 7f477effd700 1 -- 172.21.15.59:0/4201283959 learned_addr learned my addr 172.21.15.59:0/4201283959 (peer_addr_for_me v2:172.21.15.59:0/0) 2023-11-24T07:59:15.526 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:15.521+0000 7f477effd700 1 -- 172.21.15.59:0/4201283959 >> v2:172.21.15.76:3300/0 conn(0x7f47800b5e40 msgr2=0x7f478015eb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T07:59:15.526 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:15.521+0000 7f477effd700 1 --2- 172.21.15.59:0/4201283959 >> v2:172.21.15.76:3300/0 conn(0x7f47800b5e40 0x7f478015eb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:15.526 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:15.521+0000 7f477effd700 1 -- 172.21.15.59:0/4201283959 >> v1:172.21.15.76:6789/0 conn(0x7f478015e1e0 legacy=0x7f47800b12a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:15.526 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:15.521+0000 7f477effd700 1 -- 172.21.15.59:0/4201283959 --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f478006c3e0 con 0x7f478006b400 2023-11-24T07:59:15.527 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:15.521+0000 7f477ffff700 1 --2- 172.21.15.59:0/4201283959 >> v2:172.21.15.76:3300/0 conn(0x7f47800b5e40 0x7f478015eb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-24T07:59:15.527 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:15.521+0000 7f477effd700 1 --2- 172.21.15.59:0/4201283959 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f478006b400 0x7f478015e5c0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f47740098f0 tx=0x7f4774009c00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=377d77613e5c22e7 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:59:15.527 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:15.521+0000 7f477e7fc700 1 -- 172.21.15.59:0/4201283959 <== mon.2 v2:172.21.15.76:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4774011830 con 0x7f478006b400 2023-11-24T07:59:15.527 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:15.521+0000 7f477e7fc700 1 -- 172.21.15.59:0/4201283959 <== mon.2 v2:172.21.15.76:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4774011990 con 0x7f478006b400 2023-11-24T07:59:15.527 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:15.521+0000 7f477e7fc700 1 -- 172.21.15.59:0/4201283959 <== mon.2 v2:172.21.15.76:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4774011d20 con 0x7f478006b400 2023-11-24T07:59:15.530 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:15.522+0000 7f4785b23700 1 -- 172.21.15.59:0/4201283959 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f478006b400 msgr2=0x7f478015e5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:15.530 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:15.522+0000 7f4785b23700 1 --2- 172.21.15.59:0/4201283959 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f478006b400 0x7f478015e5c0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f47740098f0 tx=0x7f4774009c00 comp rx=0 tx=0).stop 2023-11-24T07:59:15.530 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:15.522+0000 7f4785b23700 1 -- 172.21.15.59:0/4201283959 shutdown_connections 2023-11-24T07:59:15.531 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:15.522+0000 7f4785b23700 1 --2- 172.21.15.59:0/4201283959 >> v2:172.21.15.76:3300/0 conn(0x7f47800b5e40 0x7f478015eb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:15.531 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:15.522+0000 7f4785b23700 1 --2- 172.21.15.59:0/4201283959 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f478006b400 0x7f478015e5c0 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:15.531 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:15.522+0000 7f4785b23700 1 -- 172.21.15.59:0/4201283959 >> 172.21.15.59:0/4201283959 conn(0x7f47800b5070 msgr2=0x7f47800b5730 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:59:15.531 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:15.522+0000 7f4785b23700 1 -- 172.21.15.59:0/4201283959 shutdown_connections 2023-11-24T07:59:15.531 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:15.522+0000 7f4785b23700 1 -- 172.21.15.59:0/4201283959 wait complete. 2023-11-24T07:59:15.531 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:15.523+0000 7f4785b23700 1 Processor -- start 2023-11-24T07:59:15.531 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:15.523+0000 7f4785b23700 1 -- start start 2023-11-24T07:59:15.531 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:15.524+0000 7f4785b23700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f478006b400 0x7f47801541c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:15.531 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:15.524+0000 7f4785b23700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f47800b5e40 0x7f47800ae7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:15.531 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:15.524+0000 7f4785b23700 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f478015e1e0 0x7f47800aed20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:15.531 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:15.524+0000 7f4785b23700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4780154700 con 0x7f478006b400 2023-11-24T07:59:15.531 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:15.524+0000 7f4785b23700 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4780155e80 con 0x7f47800b5e40 2023-11-24T07:59:15.532 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:15.524+0000 7f4785b23700 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4780156160 con 0x7f478015e1e0 2023-11-24T07:59:15.532 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:15.524+0000 7f477f7fe700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f478006b400 0x7f47801541c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:15.532 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:15.524+0000 7f477f7fe700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f478006b400 0x7f47801541c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:54810/0 (socket says 172.21.15.59:54810) 2023-11-24T07:59:15.532 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:15.524+0000 7f477f7fe700 1 -- 172.21.15.59:0/1924607968 learned_addr learned my addr 172.21.15.59:0/1924607968 (peer_addr_for_me v2:172.21.15.59:0/0) 2023-11-24T07:59:15.532 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:15.524+0000 7f477effd700 1 --2- 172.21.15.59:0/1924607968 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f47800b5e40 0x7f47800ae7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:15.532 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:15.524+0000 7f477ffff700 1 --2- 172.21.15.59:0/1924607968 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f478015e1e0 0x7f47800aed20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:15.532 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:15.524+0000 7f477f7fe700 1 -- 172.21.15.59:0/1924607968 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f478015e1e0 msgr2=0x7f47800aed20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:15.532 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:15.524+0000 7f477f7fe700 1 --2- 172.21.15.59:0/1924607968 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f478015e1e0 0x7f47800aed20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:15.532 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:15.524+0000 7f477f7fe700 1 -- 172.21.15.59:0/1924607968 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f47800b5e40 msgr2=0x7f47800ae7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:15.533 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:15.524+0000 7f477f7fe700 1 --2- 172.21.15.59:0/1924607968 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f47800b5e40 0x7f47800ae7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:15.533 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:15.524+0000 7f477f7fe700 1 -- 172.21.15.59:0/1924607968 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4778009a70 con 0x7f478006b400 2023-11-24T07:59:15.533 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:15.525+0000 7f477f7fe700 1 --2- 172.21.15.59:0/1924607968 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f478006b400 0x7f47801541c0 secure :-1 s=READY pgs=858 cs=0 l=1 rev1=1 crypto rx=0x7f477000c500 tx=0x7f477000c810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:59:15.533 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:15.525+0000 7f4767fff700 1 -- 172.21.15.59:0/1924607968 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f477000cea0 con 0x7f478006b400 2023-11-24T07:59:15.533 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:15.525+0000 7f4767fff700 1 -- 172.21.15.59:0/1924607968 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4770005cb0 con 0x7f478006b400 2023-11-24T07:59:15.533 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:15.525+0000 7f4785b23700 1 -- 172.21.15.59:0/1924607968 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4774009550 con 0x7f478006b400 2023-11-24T07:59:15.533 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:15.525+0000 7f4767fff700 1 -- 172.21.15.59:0/1924607968 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47700072d0 con 0x7f478006b400 2023-11-24T07:59:15.533 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:15.525+0000 7f4785b23700 1 -- 172.21.15.59:0/1924607968 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4780156720 con 0x7f478006b400 2023-11-24T07:59:15.533 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:15.526+0000 7f4785b23700 1 -- 172.21.15.59:0/1924607968 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4780075480 con 0x7f478006b400 2023-11-24T07:59:15.533 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:15.526+0000 7f4767fff700 1 -- 172.21.15.59:0/1924607968 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 10) v1 ==== 89588+0+0 (secure 0 0 0) 0x7f4770007430 con 0x7f478006b400 2023-11-24T07:59:15.533 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:15.527+0000 7f4767fff700 1 --2- 172.21.15.59:0/1924607968 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f476c06b2c0 0x7f476c06d780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:15.533 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:15.527+0000 7f4767fff700 1 -- 172.21.15.59:0/1924607968 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(73..73 src has 1..73) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f4770088e70 con 0x7f478006b400 2023-11-24T07:59:15.534 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:15.527+0000 7f477effd700 1 --2- 172.21.15.59:0/1924607968 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f476c06b2c0 0x7f476c06d780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:15.534 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:15.527+0000 7f477effd700 1 --2- 172.21.15.59:0/1924607968 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f476c06b2c0 0x7f476c06d780 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f478006b870 tx=0x7f4774008030 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:59:15.534 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:15.531+0000 7f4767fff700 1 -- 172.21.15.59:0/1924607968 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f4770053a40 con 0x7f478006b400 2023-11-24T07:59:15.781 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:15.779+0000 7f4785b23700 1 -- 172.21.15.59:0/1924607968 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7f47800b02f0 con 0x7f478006b400 2023-11-24T07:59:16.651 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:16.649+0000 7f4767fff700 1 -- 172.21.15.59:0/1924607968 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v43) v1 ==== 118+0+0 (secure 0 0 0) 0x7f4770017020 con 0x7f478006b400 2023-11-24T07:59:16.654 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:16.652+0000 7f4785b23700 1 -- 172.21.15.59:0/1924607968 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f476c06b2c0 msgr2=0x7f476c06d780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:16.654 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:16.652+0000 7f4785b23700 1 --2- 172.21.15.59:0/1924607968 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f476c06b2c0 0x7f476c06d780 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f478006b870 tx=0x7f4774008030 comp rx=0 tx=0).stop 2023-11-24T07:59:16.655 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:16.652+0000 7f4785b23700 1 -- 172.21.15.59:0/1924607968 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f478006b400 msgr2=0x7f47801541c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:16.655 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:16.652+0000 7f4785b23700 1 --2- 172.21.15.59:0/1924607968 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f478006b400 0x7f47801541c0 secure :-1 s=READY pgs=858 cs=0 l=1 rev1=1 crypto rx=0x7f477000c500 tx=0x7f477000c810 comp rx=0 tx=0).stop 2023-11-24T07:59:16.655 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:16.652+0000 7f4785b23700 1 -- 172.21.15.59:0/1924607968 shutdown_connections 2023-11-24T07:59:16.655 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:16.652+0000 7f4785b23700 1 --2- 172.21.15.59:0/1924607968 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f476c06b2c0 0x7f476c06d780 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:16.655 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:16.652+0000 7f4785b23700 1 --2- 172.21.15.59:0/1924607968 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f478015e1e0 0x7f47800aed20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:16.655 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:16.652+0000 7f4785b23700 1 --2- 172.21.15.59:0/1924607968 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f478006b400 0x7f47801541c0 unknown :-1 s=CLOSED pgs=858 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:16.656 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:16.652+0000 7f4785b23700 1 --2- 172.21.15.59:0/1924607968 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f47800b5e40 0x7f47800ae7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:16.656 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:16.652+0000 7f4785b23700 1 -- 172.21.15.59:0/1924607968 >> 172.21.15.59:0/1924607968 conn(0x7f47800b5070 msgr2=0x7f478015ff10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:59:16.656 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:16.652+0000 7f4785b23700 1 -- 172.21.15.59:0/1924607968 shutdown_connections 2023-11-24T07:59:16.656 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:16.653+0000 7f4785b23700 1 -- 172.21.15.59:0/1924607968 wait complete. 2023-11-24T07:59:16.669 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2023-11-24T07:59:16.670 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2023-11-24T07:59:16.806 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:16.804+0000 7fd5aa228700 1 Processor -- start 2023-11-24T07:59:16.806 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:16.804+0000 7fd5aa228700 1 -- start start 2023-11-24T07:59:16.808 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:16.805+0000 7fd5aa228700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fd5a40b4150 0x7fd5a406de50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:16.808 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:16.805+0000 7fd5aa228700 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fd5a40b5ab0 0x7fd5a406e390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:16.808 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:16.805+0000 7fd5aa228700 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd5a4071f50 con 0x7fd5a406b620 2023-11-24T07:59:16.809 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:16.805+0000 7fd5aa228700 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd5a4072230 con 0x7fd5a40b4150 2023-11-24T07:59:16.809 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:16.805+0000 7fd5aa228700 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd5a4072510 con 0x7fd5a40b5ab0 2023-11-24T07:59:16.809 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:16.805+0000 7fd5a2ffd700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fd5a40b4150 0x7fd5a406de50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:16.809 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:16.805+0000 7fd5a37fe700 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7fd5a406b620 0x7fd5a40b3a40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:58292/0 (socket says 172.21.15.59:58292) 2023-11-24T07:59:16.809 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:16.805+0000 7fd5a2ffd700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fd5a40b4150 0x7fd5a406de50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:54820/0 (socket says 172.21.15.59:54820) 2023-11-24T07:59:16.809 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:16.805+0000 7fd5a37fe700 1 -- 172.21.15.59:0/1214737108 learned_addr learned my addr 172.21.15.59:0/1214737108 (peer_addr_for_me v1:172.21.15.59:0/0) 2023-11-24T07:59:16.809 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:16.806+0000 7fd5a3fff700 1 --2- 172.21.15.59:0/1214737108 >> v2:172.21.15.76:3300/0 conn(0x7fd5a40b5ab0 0x7fd5a406e390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:16.809 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:16.806+0000 7fd5a2ffd700 1 -- 172.21.15.59:0/1214737108 >> v2:172.21.15.76:3300/0 conn(0x7fd5a40b5ab0 msgr2=0x7fd5a406e390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T07:59:16.809 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:16.806+0000 7fd5a2ffd700 1 --2- 172.21.15.59:0/1214737108 >> v2:172.21.15.76:3300/0 conn(0x7fd5a40b5ab0 0x7fd5a406e390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:16.809 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:16.806+0000 7fd5a2ffd700 1 -- 172.21.15.59:0/1214737108 >> v1:172.21.15.59:6789/0 conn(0x7fd5a406b620 legacy=0x7fd5a40b3a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:16.809 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:16.806+0000 7fd5a2ffd700 1 -- 172.21.15.59:0/1214737108 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd5a40727f0 con 0x7fd5a40b4150 2023-11-24T07:59:16.810 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:16.806+0000 7fd5a3fff700 1 --2- 172.21.15.59:0/1214737108 >> v2:172.21.15.76:3300/0 conn(0x7fd5a40b5ab0 0x7fd5a406e390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T07:59:16.810 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:16.806+0000 7fd5a2ffd700 1 --2- 172.21.15.59:0/1214737108 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd5a40b4150 0x7fd5a406de50 secure :-1 s=READY pgs=860 cs=0 l=1 rev1=1 crypto rx=0x7fd5a40694e0 tx=0x7fd594009b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7906ce2ff77bee43 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:59:16.810 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:16.806+0000 7fd5a27fc700 1 -- 172.21.15.59:0/1214737108 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd594011830 con 0x7fd5a40b4150 2023-11-24T07:59:16.810 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:16.806+0000 7fd5a27fc700 1 -- 172.21.15.59:0/1214737108 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd594011990 con 0x7fd5a40b4150 2023-11-24T07:59:16.810 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:16.806+0000 7fd5a27fc700 1 -- 172.21.15.59:0/1214737108 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd594011d20 con 0x7fd5a40b4150 2023-11-24T07:59:16.810 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:16.807+0000 7fd5aa228700 1 -- 172.21.15.59:0/1214737108 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd5a40b4150 msgr2=0x7fd5a406de50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:16.810 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:16.807+0000 7fd5aa228700 1 --2- 172.21.15.59:0/1214737108 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd5a40b4150 0x7fd5a406de50 secure :-1 s=READY pgs=860 cs=0 l=1 rev1=1 crypto rx=0x7fd5a40694e0 tx=0x7fd594009b20 comp rx=0 tx=0).stop 2023-11-24T07:59:16.810 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:16.807+0000 7fd5aa228700 1 -- 172.21.15.59:0/1214737108 shutdown_connections 2023-11-24T07:59:16.810 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:16.807+0000 7fd5aa228700 1 --2- 172.21.15.59:0/1214737108 >> v2:172.21.15.76:3300/0 conn(0x7fd5a40b5ab0 0x7fd5a406e390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:16.810 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:16.807+0000 7fd5aa228700 1 --2- 172.21.15.59:0/1214737108 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd5a40b4150 0x7fd5a406de50 unknown :-1 s=CLOSED pgs=860 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:16.810 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:16.807+0000 7fd5aa228700 1 -- 172.21.15.59:0/1214737108 >> 172.21.15.59:0/1214737108 conn(0x7fd5a40b2740 msgr2=0x7fd5a40b2b20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:59:16.810 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:16.807+0000 7fd5aa228700 1 -- 172.21.15.59:0/1214737108 shutdown_connections 2023-11-24T07:59:16.810 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:16.807+0000 7fd5aa228700 1 -- 172.21.15.59:0/1214737108 wait complete. 2023-11-24T07:59:16.813 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:16.808+0000 7fd5aa228700 1 Processor -- start 2023-11-24T07:59:16.813 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:16.808+0000 7fd5aa228700 1 -- start start 2023-11-24T07:59:16.813 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:16.809+0000 7fd5aa228700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd5a406b620 0x7fd5a41453f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:16.813 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:16.809+0000 7fd5aa228700 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fd5a40b4150 0x7fd5a40c9740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:16.813 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:16.809+0000 7fd5aa228700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd5a40b5ab0 0x7fd5a406eda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:16.813 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:16.809+0000 7fd5aa228700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5a41f3920 con 0x7fd5a40b5ab0 2023-11-24T07:59:16.813 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:16.809+0000 7fd5aa228700 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5a4145930 con 0x7fd5a406b620 2023-11-24T07:59:16.813 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:16.809+0000 7fd5aa228700 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd5a40caa60 con 0x7fd5a40b4150 2023-11-24T07:59:16.813 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:16.809+0000 7fd5a3fff700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd5a40b5ab0 0x7fd5a406eda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:16.813 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:16.809+0000 7fd5a3fff700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd5a40b5ab0 0x7fd5a406eda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:54828/0 (socket says 172.21.15.59:54828) 2023-11-24T07:59:16.813 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:16.809+0000 7fd5a3fff700 1 -- 172.21.15.59:0/2875418203 learned_addr learned my addr 172.21.15.59:0/2875418203 (peer_addr_for_me v2:172.21.15.59:0/0) 2023-11-24T07:59:16.813 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:16.809+0000 7fd5a37fe700 1 --2- 172.21.15.59:0/2875418203 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd5a406b620 0x7fd5a41453f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:16.813 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:16.809+0000 7fd5a2ffd700 1 --2- 172.21.15.59:0/2875418203 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fd5a40b4150 0x7fd5a40c9740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:16.814 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:16.809+0000 7fd5a3fff700 1 -- 172.21.15.59:0/2875418203 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fd5a40b4150 msgr2=0x7fd5a40c9740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:16.814 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:16.809+0000 7fd5a3fff700 1 --2- 172.21.15.59:0/2875418203 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fd5a40b4150 0x7fd5a40c9740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:16.814 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:16.809+0000 7fd5a3fff700 1 -- 172.21.15.59:0/2875418203 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd5a406b620 msgr2=0x7fd5a41453f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:16.814 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:16.809+0000 7fd5a3fff700 1 --2- 172.21.15.59:0/2875418203 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd5a406b620 0x7fd5a41453f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:16.814 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:16.809+0000 7fd5a3fff700 1 -- 172.21.15.59:0/2875418203 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd5940094a0 con 0x7fd5a40b5ab0 2023-11-24T07:59:16.814 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:16.809+0000 7fd5a3fff700 1 --2- 172.21.15.59:0/2875418203 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd5a40b5ab0 0x7fd5a406eda0 secure :-1 s=READY pgs=861 cs=0 l=1 rev1=1 crypto rx=0x7fd59c00cd50 tx=0x7fd59c00a560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:59:16.814 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:16.810+0000 7fd59bfff700 1 -- 172.21.15.59:0/2875418203 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd59c00cea0 con 0x7fd5a40b5ab0 2023-11-24T07:59:16.814 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:16.810+0000 7fd59bfff700 1 -- 172.21.15.59:0/2875418203 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd59c00f480 con 0x7fd5a40b5ab0 2023-11-24T07:59:16.814 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:16.810+0000 7fd5aa228700 1 -- 172.21.15.59:0/2875418203 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd5a40ca1c0 con 0x7fd5a40b5ab0 2023-11-24T07:59:16.814 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:16.810+0000 7fd59bfff700 1 -- 172.21.15.59:0/2875418203 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd59c00f830 con 0x7fd5a40b5ab0 2023-11-24T07:59:16.814 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:16.810+0000 7fd5aa228700 1 -- 172.21.15.59:0/2875418203 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd5a40ca780 con 0x7fd5a40b5ab0 2023-11-24T07:59:16.814 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:16.811+0000 7fd5aa228700 1 -- 172.21.15.59:0/2875418203 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd5a4075480 con 0x7fd5a40b5ab0 2023-11-24T07:59:16.814 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:16.811+0000 7fd59bfff700 1 -- 172.21.15.59:0/2875418203 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 10) v1 ==== 89588+0+0 (secure 0 0 0) 0x7fd59c00b040 con 0x7fd5a40b5ab0 2023-11-24T07:59:16.815 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:16.812+0000 7fd59bfff700 1 --2- 172.21.15.59:0/2875418203 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7fd58406b2c0 0x7fd58406d780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:16.815 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:16.812+0000 7fd59bfff700 1 -- 172.21.15.59:0/2875418203 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fd59c08a570 con 0x7fd5a40b5ab0 2023-11-24T07:59:16.815 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:16.812+0000 7fd5a37fe700 1 --2- 172.21.15.59:0/2875418203 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7fd58406b2c0 0x7fd58406d780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:16.815 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:16.812+0000 7fd5a37fe700 1 --2- 172.21.15.59:0/2875418203 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7fd58406b2c0 0x7fd58406d780 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7fd58c005910 tx=0x7fd58c009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:59:16.818 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:16.816+0000 7fd59bfff700 1 -- 172.21.15.59:0/2875418203 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fd59c054f30 con 0x7fd5a40b5ab0 2023-11-24T07:59:17.066 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:17.064+0000 7fd5aa228700 1 -- 172.21.15.59:0/2875418203 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7fd5a4147300 con 0x7fd5a40b5ab0 2023-11-24T07:59:17.653 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:17.651+0000 7fd59bfff700 1 -- 172.21.15.59:0/2875418203 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v44) v1 ==== 122+0+0 (secure 0 0 0) 0x7fd59c058550 con 0x7fd5a40b5ab0 2023-11-24T07:59:17.656 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:17.654+0000 7fd5aa228700 1 -- 172.21.15.59:0/2875418203 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7fd58406b2c0 msgr2=0x7fd58406d780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:17.656 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:17.654+0000 7fd5aa228700 1 --2- 172.21.15.59:0/2875418203 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7fd58406b2c0 0x7fd58406d780 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7fd58c005910 tx=0x7fd58c009040 comp rx=0 tx=0).stop 2023-11-24T07:59:17.656 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:17.654+0000 7fd5aa228700 1 -- 172.21.15.59:0/2875418203 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd5a40b5ab0 msgr2=0x7fd5a406eda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:17.656 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:17.654+0000 7fd5aa228700 1 --2- 172.21.15.59:0/2875418203 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd5a40b5ab0 0x7fd5a406eda0 secure :-1 s=READY pgs=861 cs=0 l=1 rev1=1 crypto rx=0x7fd59c00cd50 tx=0x7fd59c00a560 comp rx=0 tx=0).stop 2023-11-24T07:59:17.656 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:17.654+0000 7fd5aa228700 1 -- 172.21.15.59:0/2875418203 shutdown_connections 2023-11-24T07:59:17.656 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:17.654+0000 7fd5aa228700 1 --2- 172.21.15.59:0/2875418203 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7fd58406b2c0 0x7fd58406d780 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:17.656 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:17.654+0000 7fd5aa228700 1 --2- 172.21.15.59:0/2875418203 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd5a40b5ab0 0x7fd5a406eda0 unknown :-1 s=CLOSED pgs=861 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:17.656 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:17.654+0000 7fd5aa228700 1 --2- 172.21.15.59:0/2875418203 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd5a406b620 0x7fd5a41453f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:17.657 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:17.654+0000 7fd5aa228700 1 --2- 172.21.15.59:0/2875418203 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fd5a40b4150 0x7fd5a40c9740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:17.657 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:17.654+0000 7fd5aa228700 1 -- 172.21.15.59:0/2875418203 >> 172.21.15.59:0/2875418203 conn(0x7fd5a40b2740 msgr2=0x7fd5a41464e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:59:17.657 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:17.654+0000 7fd5aa228700 1 -- 172.21.15.59:0/2875418203 shutdown_connections 2023-11-24T07:59:17.657 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:17.654+0000 7fd5aa228700 1 -- 172.21.15.59:0/2875418203 wait complete. 2023-11-24T07:59:17.672 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2023-11-24T07:59:17.673 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-24T07:59:17.814 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:17.801+0000 7f76466a4700 1 Processor -- start 2023-11-24T07:59:17.814 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:17.802+0000 7f76466a4700 1 -- start start 2023-11-24T07:59:17.814 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:17.802+0000 7f76466a4700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f76400722c0 0x7f76400b51b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:17.814 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:17.802+0000 7f76466a4700 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7640143ec0 con 0x7f7640143ae0 2023-11-24T07:59:17.814 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:17.802+0000 7f76466a4700 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f76400b3ae0 con 0x7f76401440f0 2023-11-24T07:59:17.815 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:17.802+0000 7f76466a4700 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f76400b40a0 con 0x7f76400722c0 2023-11-24T07:59:17.815 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:17.802+0000 7f763f7fe700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f76400722c0 0x7f76400b51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:17.815 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:17.803+0000 7f763f7fe700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f76400722c0 0x7f76400b51b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:54834/0 (socket says 172.21.15.59:54834) 2023-11-24T07:59:17.815 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:17.802+0000 7f763ffff700 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7f7640143ae0 0x7f7640071bb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:58306/0 (socket says 172.21.15.59:58306) 2023-11-24T07:59:17.815 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:17.803+0000 7f763f7fe700 1 -- 172.21.15.59:0/2131243770 learned_addr learned my addr 172.21.15.59:0/2131243770 (peer_addr_for_me v2:172.21.15.59:0/0) 2023-11-24T07:59:17.815 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:17.803+0000 7f763f7fe700 1 -- 172.21.15.59:0/2131243770 >> v1:172.21.15.76:6789/0 conn(0x7f76401440f0 legacy=0x7f764006e510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:17.816 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:17.803+0000 7f763f7fe700 1 -- 172.21.15.59:0/2131243770 >> v1:172.21.15.59:6789/0 conn(0x7f7640143ae0 legacy=0x7f7640071bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:17.816 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:17.803+0000 7f763effd700 1 -- 172.21.15.59:0/2131243770 <== mon.0 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 594672179 0 0) 0x7f7640143ec0 con 0x7f7640143ae0 2023-11-24T07:59:17.816 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:17.803+0000 7f763f7fe700 1 -- 172.21.15.59:0/2131243770 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f76400b3dc0 con 0x7f76400722c0 2023-11-24T07:59:17.816 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:17.803+0000 7f763f7fe700 1 --2- 172.21.15.59:0/2131243770 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f76400722c0 0x7f76400b51b0 secure :-1 s=READY pgs=863 cs=0 l=1 rev1=1 crypto rx=0x7f763400bb90 tx=0x7f763400bea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=11afd9c599455762 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:59:17.816 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:17.803+0000 7f763effd700 1 -- 172.21.15.59:0/2131243770 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f763400dd50 con 0x7f76400722c0 2023-11-24T07:59:17.817 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:17.803+0000 7f763effd700 1 -- 172.21.15.59:0/2131243770 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7634010470 con 0x7f76400722c0 2023-11-24T07:59:17.817 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:17.803+0000 7f763effd700 1 -- 172.21.15.59:0/2131243770 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7634010800 con 0x7f76400722c0 2023-11-24T07:59:17.817 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:17.804+0000 7f76466a4700 1 -- 172.21.15.59:0/2131243770 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f76400722c0 msgr2=0x7f76400b51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:17.817 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:17.804+0000 7f76466a4700 1 --2- 172.21.15.59:0/2131243770 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f76400722c0 0x7f76400b51b0 secure :-1 s=READY pgs=863 cs=0 l=1 rev1=1 crypto rx=0x7f763400bb90 tx=0x7f763400bea0 comp rx=0 tx=0).stop 2023-11-24T07:59:17.817 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:17.804+0000 7f76466a4700 1 -- 172.21.15.59:0/2131243770 shutdown_connections 2023-11-24T07:59:17.817 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:17.804+0000 7f76466a4700 1 --2- 172.21.15.59:0/2131243770 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f76400722c0 0x7f76400b51b0 unknown :-1 s=CLOSED pgs=863 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:17.818 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:17.804+0000 7f76466a4700 1 -- 172.21.15.59:0/2131243770 >> 172.21.15.59:0/2131243770 conn(0x7f76400b2740 msgr2=0x7f76400b2b20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:59:17.818 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:17.804+0000 7f76466a4700 1 -- 172.21.15.59:0/2131243770 shutdown_connections 2023-11-24T07:59:17.818 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:17.804+0000 7f76466a4700 1 -- 172.21.15.59:0/2131243770 wait complete. 2023-11-24T07:59:17.818 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:17.805+0000 7f76466a4700 1 Processor -- start 2023-11-24T07:59:17.818 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:17.805+0000 7f76466a4700 1 -- start start 2023-11-24T07:59:17.819 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:17.806+0000 7f76466a4700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f76400722c0 0x7f76400cb500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:17.819 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:17.806+0000 7f76466a4700 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f7640143ae0 0x7f76401400a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:17.819 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:17.806+0000 7f76466a4700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f76401440f0 0x7f76401f4200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:17.819 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:17.806+0000 7f76466a4700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76401405e0 con 0x7f76401440f0 2023-11-24T07:59:17.819 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:17.806+0000 7f76466a4700 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76400ca7d0 con 0x7f76400722c0 2023-11-24T07:59:17.819 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:17.806+0000 7f76466a4700 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f76400cc9b0 con 0x7f7640143ae0 2023-11-24T07:59:17.820 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:17.806+0000 7f7644c41700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f76401440f0 0x7f76401f4200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:17.820 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:17.806+0000 7f7644c41700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f76401440f0 0x7f76401f4200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:54840/0 (socket says 172.21.15.59:54840) 2023-11-24T07:59:17.820 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:17.806+0000 7f7644c41700 1 -- 172.21.15.59:0/2042158313 learned_addr learned my addr 172.21.15.59:0/2042158313 (peer_addr_for_me v2:172.21.15.59:0/0) 2023-11-24T07:59:17.820 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:17.806+0000 7f763ffff700 1 --2- 172.21.15.59:0/2042158313 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f76400722c0 0x7f76400cb500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:17.820 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:17.806+0000 7f763f7fe700 1 --2- 172.21.15.59:0/2042158313 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f7640143ae0 0x7f76401400a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:17.820 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:17.806+0000 7f7644c41700 1 -- 172.21.15.59:0/2042158313 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f7640143ae0 msgr2=0x7f76401400a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:17.821 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:17.806+0000 7f7644c41700 1 --2- 172.21.15.59:0/2042158313 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f7640143ae0 0x7f76401400a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:17.821 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:17.806+0000 7f7644c41700 1 -- 172.21.15.59:0/2042158313 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f76400722c0 msgr2=0x7f76400cb500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:17.821 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:17.806+0000 7f7644c41700 1 --2- 172.21.15.59:0/2042158313 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f76400722c0 0x7f76400cb500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:17.821 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:17.806+0000 7f7644c41700 1 -- 172.21.15.59:0/2042158313 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f763400b7f0 con 0x7f76401440f0 2023-11-24T07:59:17.821 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:17.807+0000 7f7644c41700 1 --2- 172.21.15.59:0/2042158313 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f76401440f0 0x7f76401f4200 secure :-1 s=READY pgs=864 cs=0 l=1 rev1=1 crypto rx=0x7f763800cb50 tx=0x7f763800ce60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:59:17.822 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:17.807+0000 7f763cff9700 1 -- 172.21.15.59:0/2042158313 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7638003c00 con 0x7f76401440f0 2023-11-24T07:59:17.822 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:17.807+0000 7f763cff9700 1 -- 172.21.15.59:0/2042158313 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7638003d60 con 0x7f76401440f0 2023-11-24T07:59:17.822 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:17.807+0000 7f763cff9700 1 -- 172.21.15.59:0/2042158313 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f763800a9c0 con 0x7f76401440f0 2023-11-24T07:59:17.822 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:17.807+0000 7f76466a4700 1 -- 172.21.15.59:0/2042158313 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f76400caa20 con 0x7f76401440f0 2023-11-24T07:59:17.822 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:17.807+0000 7f76466a4700 1 -- 172.21.15.59:0/2042158313 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f76400cafe0 con 0x7f76401440f0 2023-11-24T07:59:17.822 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:17.808+0000 7f76466a4700 1 -- 172.21.15.59:0/2042158313 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f76400cb170 con 0x7f76401440f0 2023-11-24T07:59:17.823 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:17.808+0000 7f763cff9700 1 -- 172.21.15.59:0/2042158313 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 10) v1 ==== 89588+0+0 (secure 0 0 0) 0x7f763800ab20 con 0x7f76401440f0 2023-11-24T07:59:17.823 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:17.809+0000 7f763cff9700 1 --2- 172.21.15.59:0/2042158313 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f762c06b2c0 0x7f762c06d780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:17.823 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:17.809+0000 7f763cff9700 1 -- 172.21.15.59:0/2042158313 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f7638089cc0 con 0x7f76401440f0 2023-11-24T07:59:17.823 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:17.809+0000 7f763ffff700 1 --2- 172.21.15.59:0/2042158313 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f762c06b2c0 0x7f762c06d780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:17.823 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:17.809+0000 7f763ffff700 1 --2- 172.21.15.59:0/2042158313 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f762c06b2c0 0x7f762c06d780 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f7630007b10 tx=0x7f7630007a60 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:59:17.824 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:17.813+0000 7f763cff9700 1 -- 172.21.15.59:0/2042158313 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f7638090080 con 0x7f76401440f0 2023-11-24T07:59:18.060 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:18.058+0000 7f76466a4700 1 -- 172.21.15.59:0/2042158313 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f76401447e0 con 0x7f76401440f0 2023-11-24T07:59:18.060 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:18.059+0000 7f763cff9700 1 -- 172.21.15.59:0/2042158313 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 44 v44) v1 ==== 94+0+4915 (secure 0 0 0) 0x7f7638054840 con 0x7f76401440f0 2023-11-24T07:59:18.061 INFO:teuthology.orchestra.run.smithi059.stdout: 2023-11-24T07:59:18.061 INFO:teuthology.orchestra.run.smithi059.stdout:{"epoch":44,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5585,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.76:6837/2282835612","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6835","nonce":2282835612},{"type":"v1","addr":"172.21.15.76:6837","nonce":2282835612}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":39},{"gid":6097,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.76:6836/3924452718","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6834","nonce":3924452718},{"type":"v1","addr":"172.21.15.76:6836","nonce":3924452718}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":39},{"gid":6549,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6834/1425120389","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6832","nonce":1425120389},{"type":"v1","addr":"172.21.15.59:6834","nonce":1425120389}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":39}],"filesystems":[{"mdsmap":{"epoch":44,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-24T07:59:14.054294+0000","modified":"2023-11-24T07:59:17.066277+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":6552},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6552":{"gid":6552,"name":"c","rank":0,"incarnation":40,"state":"up:active","state_seq":3,"addr":"172.21.15.59:6835/1228649189","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6833","nonce":1228649189},{"type":"v1","addr":"172.21.15.59:6835","nonce":1228649189}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[11],"metadata_pool":10,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":5}]} 2023-11-24T07:59:18.063 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:18.061+0000 7f76466a4700 1 -- 172.21.15.59:0/2042158313 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f762c06b2c0 msgr2=0x7f762c06d780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:18.063 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:18.061+0000 7f76466a4700 1 --2- 172.21.15.59:0/2042158313 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f762c06b2c0 0x7f762c06d780 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f7630007b10 tx=0x7f7630007a60 comp rx=0 tx=0).stop 2023-11-24T07:59:18.063 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:18.061+0000 7f76466a4700 1 -- 172.21.15.59:0/2042158313 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f76401440f0 msgr2=0x7f76401f4200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:18.063 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:18.061+0000 7f76466a4700 1 --2- 172.21.15.59:0/2042158313 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f76401440f0 0x7f76401f4200 secure :-1 s=READY pgs=864 cs=0 l=1 rev1=1 crypto rx=0x7f763800cb50 tx=0x7f763800ce60 comp rx=0 tx=0).stop 2023-11-24T07:59:18.063 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:18.061+0000 7f76466a4700 1 -- 172.21.15.59:0/2042158313 shutdown_connections 2023-11-24T07:59:18.064 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:18.061+0000 7f76466a4700 1 --2- 172.21.15.59:0/2042158313 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f762c06b2c0 0x7f762c06d780 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:18.064 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:18.061+0000 7f76466a4700 1 --2- 172.21.15.59:0/2042158313 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f76401440f0 0x7f76401f4200 unknown :-1 s=CLOSED pgs=864 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:18.064 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:18.061+0000 7f76466a4700 1 --2- 172.21.15.59:0/2042158313 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f76400722c0 0x7f76400cb500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:18.064 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:18.061+0000 7f76466a4700 1 --2- 172.21.15.59:0/2042158313 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f7640143ae0 0x7f76401400a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:18.064 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:18.061+0000 7f76466a4700 1 -- 172.21.15.59:0/2042158313 >> 172.21.15.59:0/2042158313 conn(0x7f76400b2740 msgr2=0x7f764006afd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:59:18.064 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:18.061+0000 7f76466a4700 1 -- 172.21.15.59:0/2042158313 shutdown_connections 2023-11-24T07:59:18.064 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:18.061+0000 7f76466a4700 1 -- 172.21.15.59:0/2042158313 wait complete. 2023-11-24T07:59:18.065 INFO:teuthology.orchestra.run.smithi059.stderr:dumped fsmap epoch 44 2023-11-24T07:59:18.075 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-24T07:59:18.200 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:18.199+0000 7f2ef6efa700 1 Processor -- start 2023-11-24T07:59:18.201 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:18.199+0000 7f2ef6efa700 1 -- start start 2023-11-24T07:59:18.203 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:18.199+0000 7f2ef6efa700 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f2ef00b4c40 0x7f2ef006dba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:18.203 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:18.199+0000 7f2ef6efa700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f2ef006f420 0x7f2ef00ce1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:18.203 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:18.199+0000 7f2ef6efa700 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2ef006ef90 con 0x7f2ef00b3b40 2023-11-24T07:59:18.204 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:18.199+0000 7f2ef6efa700 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2ef006e0e0 con 0x7f2ef006f420 2023-11-24T07:59:18.204 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:18.199+0000 7f2ef6efa700 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2ef006e380 con 0x7f2ef00b4c40 2023-11-24T07:59:18.204 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:18.200+0000 7f2ef5497700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f2ef006f420 0x7f2ef00ce1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:18.204 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:18.200+0000 7f2eeffff700 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7f2ef00b3b40 0x7f2ef00b5020 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:58322/0 (socket says 172.21.15.59:58322) 2023-11-24T07:59:18.205 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:18.200+0000 7f2ef5497700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f2ef006f420 0x7f2ef00ce1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:54850/0 (socket says 172.21.15.59:54850) 2023-11-24T07:59:18.205 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:18.200+0000 7f2eeffff700 1 -- 172.21.15.59:0/683368064 learned_addr learned my addr 172.21.15.59:0/683368064 (peer_addr_for_me v1:172.21.15.59:0/0) 2023-11-24T07:59:18.205 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:18.200+0000 7f2ef4c96700 1 --2- 172.21.15.59:0/683368064 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f2ef00b4c40 0x7f2ef006dba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:18.205 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:18.200+0000 7f2ef5497700 1 -- 172.21.15.59:0/683368064 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f2ef00b4c40 msgr2=0x7f2ef006dba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T07:59:18.206 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:18.200+0000 7f2ef5497700 1 --2- 172.21.15.59:0/683368064 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f2ef00b4c40 0x7f2ef006dba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:18.206 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:18.200+0000 7f2ef5497700 1 -- 172.21.15.59:0/683368064 >> v1:172.21.15.59:6789/0 conn(0x7f2ef00b3b40 legacy=0x7f2ef00b5020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:18.206 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:18.200+0000 7f2eef7fe700 1 -- 172.21.15.59:0/683368064 <== mon.0 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 460046427 0 0) 0x7f2ef006ef90 con 0x7f2ef00b3b40 2023-11-24T07:59:18.206 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:18.200+0000 7f2ef5497700 1 -- 172.21.15.59:0/683368064 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2ef006e660 con 0x7f2ef006f420 2023-11-24T07:59:18.206 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:18.200+0000 7f2ef5497700 1 --2- 172.21.15.59:0/683368064 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2ef006f420 0x7f2ef00ce1d0 secure :-1 s=READY pgs=866 cs=0 l=1 rev1=1 crypto rx=0x7f2ee800bfa0 tx=0x7f2ee800d330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d6e86ba10e45ec78 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:59:18.207 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:18.200+0000 7f2eef7fe700 1 -- 172.21.15.59:0/683368064 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ee8010070 con 0x7f2ef006f420 2023-11-24T07:59:18.207 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:18.200+0000 7f2eef7fe700 1 -- 172.21.15.59:0/683368064 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2ee800b2e0 con 0x7f2ef006f420 2023-11-24T07:59:18.207 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:18.200+0000 7f2eef7fe700 1 -- 172.21.15.59:0/683368064 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ee800dcf0 con 0x7f2ef006f420 2023-11-24T07:59:18.207 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:18.201+0000 7f2ef6efa700 1 -- 172.21.15.59:0/683368064 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2ef006f420 msgr2=0x7f2ef00ce1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:18.208 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:18.201+0000 7f2ef6efa700 1 --2- 172.21.15.59:0/683368064 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2ef006f420 0x7f2ef00ce1d0 secure :-1 s=READY pgs=866 cs=0 l=1 rev1=1 crypto rx=0x7f2ee800bfa0 tx=0x7f2ee800d330 comp rx=0 tx=0).stop 2023-11-24T07:59:18.208 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:18.201+0000 7f2ef6efa700 1 -- 172.21.15.59:0/683368064 shutdown_connections 2023-11-24T07:59:18.208 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:18.201+0000 7f2ef6efa700 1 --2- 172.21.15.59:0/683368064 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2ef006f420 0x7f2ef00ce1d0 unknown :-1 s=CLOSED pgs=866 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:18.208 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:18.201+0000 7f2ef6efa700 1 --2- 172.21.15.59:0/683368064 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f2ef00b4c40 0x7f2ef006dba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:18.208 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:18.201+0000 7f2ef6efa700 1 -- 172.21.15.59:0/683368064 >> 172.21.15.59:0/683368064 conn(0x7f2ef00b2840 msgr2=0x7f2ef00b2c20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:59:18.209 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:18.201+0000 7f2ef6efa700 1 -- 172.21.15.59:0/683368064 shutdown_connections 2023-11-24T07:59:18.209 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:18.201+0000 7f2ef6efa700 1 -- 172.21.15.59:0/683368064 wait complete. 2023-11-24T07:59:18.215 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:18.202+0000 7f2ef6efa700 1 Processor -- start 2023-11-24T07:59:18.216 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:18.202+0000 7f2ef6efa700 1 -- start start 2023-11-24T07:59:18.216 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:18.203+0000 7f2ef6efa700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2ef006f420 0x7f2ef00aff30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:18.216 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:18.203+0000 7f2ef6efa700 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f2ef00b3b40 0x7f2ef0159d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:18.216 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:18.203+0000 7f2ef6efa700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2ef00b4c40 0x7f2ef015bf30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:18.217 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:18.203+0000 7f2ef6efa700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ef00afaa0 con 0x7f2ef006f420 2023-11-24T07:59:18.217 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:18.203+0000 7f2ef6efa700 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ef00b0470 con 0x7f2ef00b4c40 2023-11-24T07:59:18.217 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:18.203+0000 7f2ef6efa700 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2ef00aee90 con 0x7f2ef00b3b40 2023-11-24T07:59:18.217 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:18.203+0000 7f2ef4c96700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2ef006f420 0x7f2ef00aff30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:18.217 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:18.203+0000 7f2ef4c96700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2ef006f420 0x7f2ef00aff30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:54852/0 (socket says 172.21.15.59:54852) 2023-11-24T07:59:18.218 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:18.203+0000 7f2ef4c96700 1 -- 172.21.15.59:0/903898986 learned_addr learned my addr 172.21.15.59:0/903898986 (peer_addr_for_me v2:172.21.15.59:0/0) 2023-11-24T07:59:18.218 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:18.203+0000 7f2ef5497700 1 --2- 172.21.15.59:0/903898986 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2ef00b4c40 0x7f2ef015bf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:18.218 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:18.203+0000 7f2eeffff700 1 --2- 172.21.15.59:0/903898986 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f2ef00b3b40 0x7f2ef0159d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:18.218 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:18.203+0000 7f2ef4c96700 1 -- 172.21.15.59:0/903898986 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f2ef00b3b40 msgr2=0x7f2ef0159d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:18.219 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:18.203+0000 7f2ef4c96700 1 --2- 172.21.15.59:0/903898986 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f2ef00b3b40 0x7f2ef0159d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:18.219 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:18.203+0000 7f2ef4c96700 1 -- 172.21.15.59:0/903898986 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2ef00b4c40 msgr2=0x7f2ef015bf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:18.219 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:18.203+0000 7f2ef4c96700 1 --2- 172.21.15.59:0/903898986 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2ef00b4c40 0x7f2ef015bf30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:18.219 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:18.203+0000 7f2ef4c96700 1 -- 172.21.15.59:0/903898986 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2ee800bc80 con 0x7f2ef006f420 2023-11-24T07:59:18.219 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:18.204+0000 7f2ef4c96700 1 --2- 172.21.15.59:0/903898986 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2ef006f420 0x7f2ef00aff30 secure :-1 s=READY pgs=867 cs=0 l=1 rev1=1 crypto rx=0x7f2ee400d800 tx=0x7f2ee400db10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:59:18.220 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:18.204+0000 7f2eed7fa700 1 -- 172.21.15.59:0/903898986 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ee40125f0 con 0x7f2ef006f420 2023-11-24T07:59:18.220 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:18.204+0000 7f2eed7fa700 1 -- 172.21.15.59:0/903898986 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2ee4012750 con 0x7f2ef006f420 2023-11-24T07:59:18.220 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:18.204+0000 7f2ef6efa700 1 -- 172.21.15.59:0/903898986 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2ef00af170 con 0x7f2ef006f420 2023-11-24T07:59:18.220 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:18.204+0000 7f2eed7fa700 1 -- 172.21.15.59:0/903898986 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ee4012b00 con 0x7f2ef006f420 2023-11-24T07:59:18.221 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:18.204+0000 7f2ef6efa700 1 -- 172.21.15.59:0/903898986 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2ef00af730 con 0x7f2ef006f420 2023-11-24T07:59:18.221 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:18.205+0000 7f2ef6efa700 1 -- 172.21.15.59:0/903898986 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2ef00afd80 con 0x7f2ef006f420 2023-11-24T07:59:18.221 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:18.205+0000 7f2eed7fa700 1 -- 172.21.15.59:0/903898986 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 10) v1 ==== 89588+0+0 (secure 0 0 0) 0x7f2ee4012c60 con 0x7f2ef006f420 2023-11-24T07:59:18.222 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:18.206+0000 7f2eed7fa700 1 --2- 172.21.15.59:0/903898986 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f2ed406b2c0 0x7f2ed406d780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:18.222 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:18.206+0000 7f2eed7fa700 1 -- 172.21.15.59:0/903898986 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f2ee4089d20 con 0x7f2ef006f420 2023-11-24T07:59:18.222 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:18.206+0000 7f2eeffff700 1 --2- 172.21.15.59:0/903898986 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f2ed406b2c0 0x7f2ed406d780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:18.222 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:18.206+0000 7f2eeffff700 1 --2- 172.21.15.59:0/903898986 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f2ed406b2c0 0x7f2ed406d780 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f2ef015e220 tx=0x7f2edc007de0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:59:18.223 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:18.210+0000 7f2eed7fa700 1 -- 172.21.15.59:0/903898986 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f2ee40547f0 con 0x7f2ef006f420 2023-11-24T07:59:18.412 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:18.411+0000 7f2ef6efa700 1 -- 172.21.15.59:0/903898986 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f2ef0067c70 con 0x7f2ef006f420 2023-11-24T07:59:18.413 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:18.412+0000 7f2eed7fa700 1 -- 172.21.15.59:0/903898986 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v74) v1 ==== 74+0+16436 (secure 0 0 0) 0x7f2ee4057e10 con 0x7f2ef006f420 2023-11-24T07:59:18.414 INFO:teuthology.orchestra.run.smithi059.stdout: 2023-11-24T07:59:18.414 INFO:teuthology.orchestra.run.smithi059.stdout:{"epoch":74,"fsid":"9f0e8741-b8f9-48d5-a2c3-09f45903303d","created":"2023-11-24T07:56:06.552729+0000","modified":"2023-11-24T07:59:15.638357+0000","last_up_change":"2023-11-24T07:56:17.372352+0000","last_in_change":"2023-11-24T07:56:09.073250+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":11,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-24T07:56:18.352823+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_metadata","create_time":"2023-11-24T07:59:11.900882+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"73","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_data","create_time":"2023-11-24T07:59:13.025749+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"74","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"6d8f9e9b-1782-4e41-928b-a39f2a8ebc06","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6800","nonce":3373878772},{"type":"v1","addr":"172.21.15.59:6801","nonce":3373878772}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6802","nonce":3373878772},{"type":"v1","addr":"172.21.15.59:6803","nonce":3373878772}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6806","nonce":3373878772},{"type":"v1","addr":"172.21.15.59:6807","nonce":3373878772}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6804","nonce":3373878772},{"type":"v1","addr":"172.21.15.59:6805","nonce":3373878772}]},"public_addr":"172.21.15.59:6801/3373878772","cluster_addr":"172.21.15.59:6803/3373878772","heartbeat_back_addr":"172.21.15.59:6807/3373878772","heartbeat_front_addr":"172.21.15.59:6805/3373878772","state":["exists","up"]},{"osd":1,"uuid":"e8d882aa-52e1-4c04-9e1e-c44ef9204a5f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6824","nonce":648258563},{"type":"v1","addr":"172.21.15.59:6825","nonce":648258563}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6826","nonce":648258563},{"type":"v1","addr":"172.21.15.59:6827","nonce":648258563}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6830","nonce":648258563},{"type":"v1","addr":"172.21.15.59:6831","nonce":648258563}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6828","nonce":648258563},{"type":"v1","addr":"172.21.15.59:6829","nonce":648258563}]},"public_addr":"172.21.15.59:6825/648258563","cluster_addr":"172.21.15.59:6827/648258563","heartbeat_back_addr":"172.21.15.59:6831/648258563","heartbeat_front_addr":"172.21.15.59:6829/648258563","state":["exists","up"]},{"osd":2,"uuid":"69eebfa0-695b-4b21-90b9-8a3537168a01","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6808","nonce":1840430654},{"type":"v1","addr":"172.21.15.59:6809","nonce":1840430654}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6810","nonce":1840430654},{"type":"v1","addr":"172.21.15.59:6811","nonce":1840430654}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6814","nonce":1840430654},{"type":"v1","addr":"172.21.15.59:6815","nonce":1840430654}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6812","nonce":1840430654},{"type":"v1","addr":"172.21.15.59:6813","nonce":1840430654}]},"public_addr":"172.21.15.59:6809/1840430654","cluster_addr":"172.21.15.59:6811/1840430654","heartbeat_back_addr":"172.21.15.59:6815/1840430654","heartbeat_front_addr":"172.21.15.59:6813/1840430654","state":["exists","up"]},{"osd":3,"uuid":"96fd77fc-f911-46ff-9445-6f0be2daa8f4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6816","nonce":2504025850},{"type":"v1","addr":"172.21.15.59:6817","nonce":2504025850}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6818","nonce":2504025850},{"type":"v1","addr":"172.21.15.59:6819","nonce":2504025850}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6822","nonce":2504025850},{"type":"v1","addr":"172.21.15.59:6823","nonce":2504025850}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6820","nonce":2504025850},{"type":"v1","addr":"172.21.15.59:6821","nonce":2504025850}]},"public_addr":"172.21.15.59:6817/2504025850","cluster_addr":"172.21.15.59:6819/2504025850","heartbeat_back_addr":"172.21.15.59:6823/2504025850","heartbeat_front_addr":"172.21.15.59:6821/2504025850","state":["exists","up"]},{"osd":4,"uuid":"7ac29fca-b5b2-4c95-9d69-2f164cbbc708","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6816","nonce":2382463431},{"type":"v1","addr":"172.21.15.76:6818","nonce":2382463431}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6820","nonce":2382463431},{"type":"v1","addr":"172.21.15.76:6822","nonce":2382463431}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6828","nonce":2382463431},{"type":"v1","addr":"172.21.15.76:6830","nonce":2382463431}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6824","nonce":2382463431},{"type":"v1","addr":"172.21.15.76:6826","nonce":2382463431}]},"public_addr":"172.21.15.76:6818/2382463431","cluster_addr":"172.21.15.76:6822/2382463431","heartbeat_back_addr":"172.21.15.76:6830/2382463431","heartbeat_front_addr":"172.21.15.76:6826/2382463431","state":["exists","up"]},{"osd":5,"uuid":"485bf711-3851-4238-a4f4-51d52e0a9c58","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6800","nonce":329334308},{"type":"v1","addr":"172.21.15.76:6802","nonce":329334308}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6805","nonce":329334308},{"type":"v1","addr":"172.21.15.76:6807","nonce":329334308}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6814","nonce":329334308},{"type":"v1","addr":"172.21.15.76:6815","nonce":329334308}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6810","nonce":329334308},{"type":"v1","addr":"172.21.15.76:6812","nonce":329334308}]},"public_addr":"172.21.15.76:6802/329334308","cluster_addr":"172.21.15.76:6807/329334308","heartbeat_back_addr":"172.21.15.76:6815/329334308","heartbeat_front_addr":"172.21.15.76:6812/329334308","state":["exists","up"]},{"osd":6,"uuid":"72b3f2fc-1e4a-4e32-968d-003647a0286f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6801","nonce":3946579410},{"type":"v1","addr":"172.21.15.76:6803","nonce":3946579410}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6804","nonce":3946579410},{"type":"v1","addr":"172.21.15.76:6806","nonce":3946579410}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6811","nonce":3946579410},{"type":"v1","addr":"172.21.15.76:6813","nonce":3946579410}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6808","nonce":3946579410},{"type":"v1","addr":"172.21.15.76:6809","nonce":3946579410}]},"public_addr":"172.21.15.76:6803/3946579410","cluster_addr":"172.21.15.76:6806/3946579410","heartbeat_back_addr":"172.21.15.76:6813/3946579410","heartbeat_front_addr":"172.21.15.76:6809/3946579410","state":["exists","up"]},{"osd":7,"uuid":"1ddceb90-6c5d-4d1a-ad56-99cb877f8c70","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6817","nonce":841761538},{"type":"v1","addr":"172.21.15.76:6819","nonce":841761538}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6821","nonce":841761538},{"type":"v1","addr":"172.21.15.76:6823","nonce":841761538}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6829","nonce":841761538},{"type":"v1","addr":"172.21.15.76:6831","nonce":841761538}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6825","nonce":841761538},{"type":"v1","addr":"172.21.15.76:6827","nonce":841761538}]},"public_addr":"172.21.15.76:6819/841761538","cluster_addr":"172.21.15.76:6823/841761538","heartbeat_back_addr":"172.21.15.76:6831/841761538","heartbeat_front_addr":"172.21.15.76:6827/841761538","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T07:56:14.294838+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T07:56:14.270779+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T07:56:14.231809+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T07:56:14.260983+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T07:56:14.302997+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T07:56:14.258600+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T07:56:14.251413+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T07:56:14.304727+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-24T07:59:18.416 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:18.414+0000 7f2ef6efa700 1 -- 172.21.15.59:0/903898986 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f2ed406b2c0 msgr2=0x7f2ed406d780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:18.416 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:18.414+0000 7f2ef6efa700 1 --2- 172.21.15.59:0/903898986 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f2ed406b2c0 0x7f2ed406d780 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f2ef015e220 tx=0x7f2edc007de0 comp rx=0 tx=0).stop 2023-11-24T07:59:18.416 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:18.414+0000 7f2ef6efa700 1 -- 172.21.15.59:0/903898986 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2ef006f420 msgr2=0x7f2ef00aff30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:18.416 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:18.414+0000 7f2ef6efa700 1 --2- 172.21.15.59:0/903898986 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2ef006f420 0x7f2ef00aff30 secure :-1 s=READY pgs=867 cs=0 l=1 rev1=1 crypto rx=0x7f2ee400d800 tx=0x7f2ee400db10 comp rx=0 tx=0).stop 2023-11-24T07:59:18.416 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:18.414+0000 7f2ef6efa700 1 -- 172.21.15.59:0/903898986 shutdown_connections 2023-11-24T07:59:18.416 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:18.414+0000 7f2ef6efa700 1 --2- 172.21.15.59:0/903898986 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f2ed406b2c0 0x7f2ed406d780 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:18.417 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:18.414+0000 7f2ef6efa700 1 --2- 172.21.15.59:0/903898986 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2ef00b4c40 0x7f2ef015bf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:18.417 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:18.414+0000 7f2ef6efa700 1 --2- 172.21.15.59:0/903898986 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2ef006f420 0x7f2ef00aff30 unknown :-1 s=CLOSED pgs=867 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:18.417 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:18.414+0000 7f2ef6efa700 1 --2- 172.21.15.59:0/903898986 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f2ef00b3b40 0x7f2ef0159d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:18.417 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:18.414+0000 7f2ef6efa700 1 -- 172.21.15.59:0/903898986 >> 172.21.15.59:0/903898986 conn(0x7f2ef00b2840 msgr2=0x7f2ef0071b40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:59:18.417 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:18.414+0000 7f2ef6efa700 1 -- 172.21.15.59:0/903898986 shutdown_connections 2023-11-24T07:59:18.417 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:18.414+0000 7f2ef6efa700 1 -- 172.21.15.59:0/903898986 wait complete. 2023-11-24T07:59:18.428 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2023-11-24T07:59:18.429 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-11-24T07:59:18.553 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:18.551+0000 7f4179efa700 1 Processor -- start 2023-11-24T07:59:18.553 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:18.552+0000 7f4179efa700 1 -- start start 2023-11-24T07:59:18.556 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:18.552+0000 7f4179efa700 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f417406b6e0 0x7f4174168e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:18.556 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:18.552+0000 7f4179efa700 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f417406dbe0 con 0x7f41740b57b0 2023-11-24T07:59:18.556 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:18.552+0000 7f4179efa700 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f41741622e0 con 0x7f41741478c0 2023-11-24T07:59:18.556 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:18.552+0000 7f4179efa700 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4174147ca0 con 0x7f417406b6e0 2023-11-24T07:59:18.556 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:18.553+0000 7f4172ffd700 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7f41740b57b0 0x7f417406afd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:58336/0 (socket says 172.21.15.59:58336) 2023-11-24T07:59:18.556 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:18.553+0000 7f4172ffd700 1 -- 172.21.15.59:0/4241894016 learned_addr learned my addr 172.21.15.59:0/4241894016 (peer_addr_for_me v1:172.21.15.59:0/0) 2023-11-24T07:59:18.556 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:18.553+0000 7f4173fff700 1 --2- 172.21.15.59:0/4241894016 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f417406b6e0 0x7f4174168e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:18.556 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:18.553+0000 7f41727fc700 1 -- 172.21.15.59:0/4241894016 <== mon.0 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4104151070 0 0) 0x7f417406dbe0 con 0x7f41740b57b0 2023-11-24T07:59:18.556 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:18.553+0000 7f41727fc700 1 -- 172.21.15.59:0/4241894016 --> v1:172.21.15.59:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f415c0035b0 con 0x7f41740b57b0 2023-11-24T07:59:18.557 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:18.553+0000 7f41727fc700 1 -- 172.21.15.59:0/4241894016 <== mon.1 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2273452550 0 0) 0x7f41741622e0 con 0x7f41741478c0 2023-11-24T07:59:18.557 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:18.553+0000 7f41727fc700 1 -- 172.21.15.59:0/4241894016 --> v1:172.21.15.76:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f417406dbe0 con 0x7f41741478c0 2023-11-24T07:59:18.557 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:18.553+0000 7f4173fff700 1 -- 172.21.15.59:0/4241894016 >> v1:172.21.15.76:6789/0 conn(0x7f41741478c0 legacy=0x7f4174161a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:18.557 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:18.553+0000 7f4173fff700 1 -- 172.21.15.59:0/4241894016 >> v1:172.21.15.59:6789/0 conn(0x7f41740b57b0 legacy=0x7f417406afd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:18.557 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:18.553+0000 7f4173fff700 1 -- 172.21.15.59:0/4241894016 --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f41741625c0 con 0x7f417406b6e0 2023-11-24T07:59:18.557 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:18.553+0000 7f4173fff700 1 --2- 172.21.15.59:0/4241894016 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f417406b6e0 0x7f4174168e20 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f416c009dc0 tx=0x7f416c00b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b7fc1abeb0924220 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:59:18.557 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:18.553+0000 7f41727fc700 1 -- 172.21.15.59:0/4241894016 <== mon.2 v2:172.21.15.76:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f416c00e070 con 0x7f417406b6e0 2023-11-24T07:59:18.557 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:18.553+0000 7f41727fc700 1 -- 172.21.15.59:0/4241894016 <== mon.2 v2:172.21.15.76:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f416c00bb20 con 0x7f417406b6e0 2023-11-24T07:59:18.557 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:18.553+0000 7f41727fc700 1 -- 172.21.15.59:0/4241894016 <== mon.2 v2:172.21.15.76:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f416c012430 con 0x7f417406b6e0 2023-11-24T07:59:18.557 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:18.554+0000 7f4179efa700 1 -- 172.21.15.59:0/4241894016 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f417406b6e0 msgr2=0x7f4174168e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:18.557 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:18.554+0000 7f4179efa700 1 --2- 172.21.15.59:0/4241894016 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f417406b6e0 0x7f4174168e20 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f416c009dc0 tx=0x7f416c00b330 comp rx=0 tx=0).stop 2023-11-24T07:59:18.557 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:18.554+0000 7f4179efa700 1 -- 172.21.15.59:0/4241894016 shutdown_connections 2023-11-24T07:59:18.557 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:18.554+0000 7f4179efa700 1 --2- 172.21.15.59:0/4241894016 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f417406b6e0 0x7f4174168e20 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:18.558 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:18.554+0000 7f4179efa700 1 -- 172.21.15.59:0/4241894016 >> 172.21.15.59:0/4241894016 conn(0x7f41740d0aa0 msgr2=0x7f41740d0eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:59:18.558 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:18.554+0000 7f4179efa700 1 -- 172.21.15.59:0/4241894016 shutdown_connections 2023-11-24T07:59:18.558 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:18.554+0000 7f4179efa700 1 -- 172.21.15.59:0/4241894016 wait complete. 2023-11-24T07:59:18.560 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:18.555+0000 7f4179efa700 1 Processor -- start 2023-11-24T07:59:18.560 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:18.555+0000 7f4179efa700 1 -- start start 2023-11-24T07:59:18.560 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:18.556+0000 7f4179efa700 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f417406b6e0 0x7f417413ac30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:18.560 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:18.556+0000 7f4179efa700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f41740b57b0 0x7f4174139490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:18.560 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:18.556+0000 7f4179efa700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f41741478c0 0x7f4174138660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:18.561 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:18.556+0000 7f4179efa700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f417413a810 con 0x7f41740b57b0 2023-11-24T07:59:18.561 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:18.556+0000 7f4179efa700 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4174138ba0 con 0x7f41741478c0 2023-11-24T07:59:18.561 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:18.556+0000 7f4179efa700 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4174139b90 con 0x7f417406b6e0 2023-11-24T07:59:18.561 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:18.556+0000 7f4172ffd700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f41740b57b0 0x7f4174139490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:18.561 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:18.556+0000 7f4172ffd700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f41740b57b0 0x7f4174139490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:54866/0 (socket says 172.21.15.59:54866) 2023-11-24T07:59:18.561 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:18.556+0000 7f4172ffd700 1 -- 172.21.15.59:0/3408439408 learned_addr learned my addr 172.21.15.59:0/3408439408 (peer_addr_for_me v2:172.21.15.59:0/0) 2023-11-24T07:59:18.561 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:18.556+0000 7f4173fff700 1 --2- 172.21.15.59:0/3408439408 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f41741478c0 0x7f4174138660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:18.561 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:18.556+0000 7f41737fe700 1 --2- 172.21.15.59:0/3408439408 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f417406b6e0 0x7f417413ac30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:18.561 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:18.556+0000 7f4172ffd700 1 -- 172.21.15.59:0/3408439408 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f417406b6e0 msgr2=0x7f417413ac30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:18.561 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:18.556+0000 7f4172ffd700 1 --2- 172.21.15.59:0/3408439408 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f417406b6e0 0x7f417413ac30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:18.561 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:18.556+0000 7f4172ffd700 1 -- 172.21.15.59:0/3408439408 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f41741478c0 msgr2=0x7f4174138660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:18.562 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:18.556+0000 7f4172ffd700 1 --2- 172.21.15.59:0/3408439408 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f41741478c0 0x7f4174138660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:18.562 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:18.556+0000 7f4172ffd700 1 -- 172.21.15.59:0/3408439408 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f416c009a70 con 0x7f41740b57b0 2023-11-24T07:59:18.562 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:18.557+0000 7f4172ffd700 1 --2- 172.21.15.59:0/3408439408 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f41740b57b0 0x7f4174139490 secure :-1 s=READY pgs=869 cs=0 l=1 rev1=1 crypto rx=0x7f416800c700 tx=0x7f416800ca10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:59:18.562 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:18.557+0000 7f415bfff700 1 -- 172.21.15.59:0/3408439408 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4168005d70 con 0x7f41740b57b0 2023-11-24T07:59:18.562 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:18.557+0000 7f415bfff700 1 -- 172.21.15.59:0/3408439408 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f41680070d0 con 0x7f41740b57b0 2023-11-24T07:59:18.562 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:18.557+0000 7f415bfff700 1 -- 172.21.15.59:0/3408439408 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4168007460 con 0x7f41740b57b0 2023-11-24T07:59:18.562 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:18.557+0000 7f4179efa700 1 -- 172.21.15.59:0/3408439408 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4174139e70 con 0x7f41740b57b0 2023-11-24T07:59:18.562 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:18.557+0000 7f4179efa700 1 -- 172.21.15.59:0/3408439408 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f417413a430 con 0x7f41740b57b0 2023-11-24T07:59:18.562 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:18.558+0000 7f4179efa700 1 -- 172.21.15.59:0/3408439408 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f417413a950 con 0x7f41740b57b0 2023-11-24T07:59:18.562 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:18.558+0000 7f415bfff700 1 -- 172.21.15.59:0/3408439408 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 10) v1 ==== 89588+0+0 (secure 0 0 0) 0x7f41680075c0 con 0x7f41740b57b0 2023-11-24T07:59:18.562 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:18.559+0000 7f415bfff700 1 --2- 172.21.15.59:0/3408439408 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f416006b2c0 0x7f416006d780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:18.563 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:18.559+0000 7f415bfff700 1 -- 172.21.15.59:0/3408439408 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f41680891a0 con 0x7f41740b57b0 2023-11-24T07:59:18.563 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:18.559+0000 7f41737fe700 1 --2- 172.21.15.59:0/3408439408 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f416006b2c0 0x7f416006d780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:18.563 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:18.559+0000 7f41737fe700 1 --2- 172.21.15.59:0/3408439408 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f416006b2c0 0x7f416006d780 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f41640011e0 tx=0x7f4164009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:59:18.564 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:18.563+0000 7f415bfff700 1 -- 172.21.15.59:0/3408439408 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f417413a950 con 0x7f41740b57b0 2023-11-24T07:59:18.760 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:18.758+0000 7f4179efa700 1 -- 172.21.15.59:0/3408439408 --> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f41740b5c60 con 0x7f416006b2c0 2023-11-24T07:59:18.768 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:18.766+0000 7f415bfff700 1 -- 172.21.15.59:0/3408439408 <== mgr.4100 v2:172.21.15.76:6832/4264046536 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+277156 (secure 0 0 0) 0x7f41740b5c60 con 0x7f416006b2c0 2023-11-24T07:59:18.771 INFO:teuthology.orchestra.run.smithi059.stdout: 2023-11-24T07:59:18.774 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:18.769+0000 7f4179efa700 1 -- 172.21.15.59:0/3408439408 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f416006b2c0 msgr2=0x7f416006d780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:18.774 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:18.769+0000 7f4179efa700 1 --2- 172.21.15.59:0/3408439408 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f416006b2c0 0x7f416006d780 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f41640011e0 tx=0x7f4164009040 comp rx=0 tx=0).stop 2023-11-24T07:59:18.774 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:18.769+0000 7f4179efa700 1 -- 172.21.15.59:0/3408439408 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f41740b57b0 msgr2=0x7f4174139490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:18.774 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:18.769+0000 7f4179efa700 1 --2- 172.21.15.59:0/3408439408 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f41740b57b0 0x7f4174139490 secure :-1 s=READY pgs=869 cs=0 l=1 rev1=1 crypto rx=0x7f416800c700 tx=0x7f416800ca10 comp rx=0 tx=0).stop 2023-11-24T07:59:18.774 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:18.769+0000 7f4179efa700 1 -- 172.21.15.59:0/3408439408 shutdown_connections 2023-11-24T07:59:18.774 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:18.769+0000 7f4179efa700 1 --2- 172.21.15.59:0/3408439408 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f416006b2c0 0x7f416006d780 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:18.774 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:18.769+0000 7f4179efa700 1 --2- 172.21.15.59:0/3408439408 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f41741478c0 0x7f4174138660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:18.774 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:18.769+0000 7f4179efa700 1 --2- 172.21.15.59:0/3408439408 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f417406b6e0 0x7f417413ac30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:18.774 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:18.769+0000 7f4179efa700 1 --2- 172.21.15.59:0/3408439408 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f41740b57b0 0x7f4174139490 unknown :-1 s=CLOSED pgs=869 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:18.774 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:18.769+0000 7f4179efa700 1 -- 172.21.15.59:0/3408439408 >> 172.21.15.59:0/3408439408 conn(0x7f41740d0aa0 msgr2=0x7f41740727a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:59:18.774 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:18.770+0000 7f4179efa700 1 -- 172.21.15.59:0/3408439408 shutdown_connections 2023-11-24T07:59:18.775 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:18.770+0000 7f4179efa700 1 -- 172.21.15.59:0/3408439408 wait complete. 2023-11-24T07:59:18.775 INFO:teuthology.orchestra.run.smithi059.stderr:dumped all 2023-11-24T07:59:18.783 INFO:teuthology.orchestra.run.smithi059.stdout:{"pg_ready":true,"pg_map":{"version":157,"stamp":"2023-11-24T07:59:18.339299+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":66,"acting":66,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":5,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":7,"kb":754974720,"kb_used":224084,"kb_used_data":7888,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754750636,"statfs":{"total":773094113280,"available":772864651264,"internally_reserved":0,"allocated":8077312,"data_stored":4087422,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12745,"internal_metadata":221236791},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":15,"apply_latency_ms":15,"commit_latency_ns":15000000,"apply_latency_ns":15000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.718179"},"pg_stats":[{"pgid":"11.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T07:59:13.620687+0000","last_change":"2023-11-24T07:59:13.620687+0000","last_active":"2023-11-24T07:59:13.620687+0000","last_peered":"2023-11-24T07:59:13.620687+0000","last_clean":"2023-11-24T07:59:13.620687+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T07:59:13.620687+0000","last_undegraded":"2023-11-24T07:59:13.620687+0000","last_fullsized":"2023-11-24T07:59:13.620687+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:59:13.620687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:59:13.620687+0000","last_clean_scrub_stamp":"2023-11-24T07:59:13.620687+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T07:59:13.620687+0000","last_change":"2023-11-24T07:59:13.620687+0000","last_active":"2023-11-24T07:59:13.620687+0000","last_peered":"2023-11-24T07:59:13.620687+0000","last_clean":"2023-11-24T07:59:13.620687+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T07:59:13.620687+0000","last_undegraded":"2023-11-24T07:59:13.620687+0000","last_fullsized":"2023-11-24T07:59:13.620687+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:59:13.620687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:59:13.620687+0000","last_clean_scrub_stamp":"2023-11-24T07:59:13.620687+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T07:59:13.620687+0000","last_change":"2023-11-24T07:59:13.620687+0000","last_active":"2023-11-24T07:59:13.620687+0000","last_peered":"2023-11-24T07:59:13.620687+0000","last_clean":"2023-11-24T07:59:13.620687+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T07:59:13.620687+0000","last_undegraded":"2023-11-24T07:59:13.620687+0000","last_fullsized":"2023-11-24T07:59:13.620687+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:59:13.620687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:59:13.620687+0000","last_clean_scrub_stamp":"2023-11-24T07:59:13.620687+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T07:59:13.620687+0000","last_change":"2023-11-24T07:59:13.620687+0000","last_active":"2023-11-24T07:59:13.620687+0000","last_peered":"2023-11-24T07:59:13.620687+0000","last_clean":"2023-11-24T07:59:13.620687+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T07:59:13.620687+0000","last_undegraded":"2023-11-24T07:59:13.620687+0000","last_fullsized":"2023-11-24T07:59:13.620687+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:59:13.620687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:59:13.620687+0000","last_clean_scrub_stamp":"2023-11-24T07:59:13.620687+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T07:59:13.620687+0000","last_change":"2023-11-24T07:59:13.620687+0000","last_active":"2023-11-24T07:59:13.620687+0000","last_peered":"2023-11-24T07:59:13.620687+0000","last_clean":"2023-11-24T07:59:13.620687+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T07:59:13.620687+0000","last_undegraded":"2023-11-24T07:59:13.620687+0000","last_fullsized":"2023-11-24T07:59:13.620687+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:59:13.620687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:59:13.620687+0000","last_clean_scrub_stamp":"2023-11-24T07:59:13.620687+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T07:59:13.620687+0000","last_change":"2023-11-24T07:59:13.620687+0000","last_active":"2023-11-24T07:59:13.620687+0000","last_peered":"2023-11-24T07:59:13.620687+0000","last_clean":"2023-11-24T07:59:13.620687+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T07:59:13.620687+0000","last_undegraded":"2023-11-24T07:59:13.620687+0000","last_fullsized":"2023-11-24T07:59:13.620687+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:59:13.620687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:59:13.620687+0000","last_clean_scrub_stamp":"2023-11-24T07:59:13.620687+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T07:59:13.620687+0000","last_change":"2023-11-24T07:59:13.620687+0000","last_active":"2023-11-24T07:59:13.620687+0000","last_peered":"2023-11-24T07:59:13.620687+0000","last_clean":"2023-11-24T07:59:13.620687+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T07:59:13.620687+0000","last_undegraded":"2023-11-24T07:59:13.620687+0000","last_fullsized":"2023-11-24T07:59:13.620687+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:59:13.620687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:59:13.620687+0000","last_clean_scrub_stamp":"2023-11-24T07:59:13.620687+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T07:59:13.620687+0000","last_change":"2023-11-24T07:59:13.620687+0000","last_active":"2023-11-24T07:59:13.620687+0000","last_peered":"2023-11-24T07:59:13.620687+0000","last_clean":"2023-11-24T07:59:13.620687+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T07:59:13.620687+0000","last_undegraded":"2023-11-24T07:59:13.620687+0000","last_fullsized":"2023-11-24T07:59:13.620687+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:59:13.620687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:59:13.620687+0000","last_clean_scrub_stamp":"2023-11-24T07:59:13.620687+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T07:59:13.620687+0000","last_change":"2023-11-24T07:59:13.620687+0000","last_active":"2023-11-24T07:59:13.620687+0000","last_peered":"2023-11-24T07:59:13.620687+0000","last_clean":"2023-11-24T07:59:13.620687+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T07:59:13.620687+0000","last_undegraded":"2023-11-24T07:59:13.620687+0000","last_fullsized":"2023-11-24T07:59:13.620687+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:59:13.620687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:59:13.620687+0000","last_clean_scrub_stamp":"2023-11-24T07:59:13.620687+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T07:59:13.620687+0000","last_change":"2023-11-24T07:59:13.620687+0000","last_active":"2023-11-24T07:59:13.620687+0000","last_peered":"2023-11-24T07:59:13.620687+0000","last_clean":"2023-11-24T07:59:13.620687+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T07:59:13.620687+0000","last_undegraded":"2023-11-24T07:59:13.620687+0000","last_fullsized":"2023-11-24T07:59:13.620687+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:59:13.620687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:59:13.620687+0000","last_clean_scrub_stamp":"2023-11-24T07:59:13.620687+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T07:59:13.620687+0000","last_change":"2023-11-24T07:59:13.620687+0000","last_active":"2023-11-24T07:59:13.620687+0000","last_peered":"2023-11-24T07:59:13.620687+0000","last_clean":"2023-11-24T07:59:13.620687+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T07:59:13.620687+0000","last_undegraded":"2023-11-24T07:59:13.620687+0000","last_fullsized":"2023-11-24T07:59:13.620687+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:59:13.620687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:59:13.620687+0000","last_clean_scrub_stamp":"2023-11-24T07:59:13.620687+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T07:59:13.620687+0000","last_change":"2023-11-24T07:59:13.620687+0000","last_active":"2023-11-24T07:59:13.620687+0000","last_peered":"2023-11-24T07:59:13.620687+0000","last_clean":"2023-11-24T07:59:13.620687+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T07:59:13.620687+0000","last_undegraded":"2023-11-24T07:59:13.620687+0000","last_fullsized":"2023-11-24T07:59:13.620687+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:59:13.620687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:59:13.620687+0000","last_clean_scrub_stamp":"2023-11-24T07:59:13.620687+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T07:59:13.620687+0000","last_change":"2023-11-24T07:59:13.620687+0000","last_active":"2023-11-24T07:59:13.620687+0000","last_peered":"2023-11-24T07:59:13.620687+0000","last_clean":"2023-11-24T07:59:13.620687+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T07:59:13.620687+0000","last_undegraded":"2023-11-24T07:59:13.620687+0000","last_fullsized":"2023-11-24T07:59:13.620687+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:59:13.620687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:59:13.620687+0000","last_clean_scrub_stamp":"2023-11-24T07:59:13.620687+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T07:59:13.620687+0000","last_change":"2023-11-24T07:59:13.620687+0000","last_active":"2023-11-24T07:59:13.620687+0000","last_peered":"2023-11-24T07:59:13.620687+0000","last_clean":"2023-11-24T07:59:13.620687+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T07:59:13.620687+0000","last_undegraded":"2023-11-24T07:59:13.620687+0000","last_fullsized":"2023-11-24T07:59:13.620687+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:59:13.620687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:59:13.620687+0000","last_clean_scrub_stamp":"2023-11-24T07:59:13.620687+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T07:59:13.620687+0000","last_change":"2023-11-24T07:59:13.620687+0000","last_active":"2023-11-24T07:59:13.620687+0000","last_peered":"2023-11-24T07:59:13.620687+0000","last_clean":"2023-11-24T07:59:13.620687+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T07:59:13.620687+0000","last_undegraded":"2023-11-24T07:59:13.620687+0000","last_fullsized":"2023-11-24T07:59:13.620687+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:59:13.620687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:59:13.620687+0000","last_clean_scrub_stamp":"2023-11-24T07:59:13.620687+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T07:59:13.620687+0000","last_change":"2023-11-24T07:59:13.620687+0000","last_active":"2023-11-24T07:59:13.620687+0000","last_peered":"2023-11-24T07:59:13.620687+0000","last_clean":"2023-11-24T07:59:13.620687+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T07:59:13.620687+0000","last_undegraded":"2023-11-24T07:59:13.620687+0000","last_fullsized":"2023-11-24T07:59:13.620687+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:59:13.620687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:59:13.620687+0000","last_clean_scrub_stamp":"2023-11-24T07:59:13.620687+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T07:59:13.620687+0000","last_change":"2023-11-24T07:59:13.620687+0000","last_active":"2023-11-24T07:59:13.620687+0000","last_peered":"2023-11-24T07:59:13.620687+0000","last_clean":"2023-11-24T07:59:13.620687+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T07:59:13.620687+0000","last_undegraded":"2023-11-24T07:59:13.620687+0000","last_fullsized":"2023-11-24T07:59:13.620687+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:59:13.620687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:59:13.620687+0000","last_clean_scrub_stamp":"2023-11-24T07:59:13.620687+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T07:59:13.620687+0000","last_change":"2023-11-24T07:59:13.620687+0000","last_active":"2023-11-24T07:59:13.620687+0000","last_peered":"2023-11-24T07:59:13.620687+0000","last_clean":"2023-11-24T07:59:13.620687+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T07:59:13.620687+0000","last_undegraded":"2023-11-24T07:59:13.620687+0000","last_fullsized":"2023-11-24T07:59:13.620687+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:59:13.620687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:59:13.620687+0000","last_clean_scrub_stamp":"2023-11-24T07:59:13.620687+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T07:59:13.620687+0000","last_change":"2023-11-24T07:59:13.620687+0000","last_active":"2023-11-24T07:59:13.620687+0000","last_peered":"2023-11-24T07:59:13.620687+0000","last_clean":"2023-11-24T07:59:13.620687+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T07:59:13.620687+0000","last_undegraded":"2023-11-24T07:59:13.620687+0000","last_fullsized":"2023-11-24T07:59:13.620687+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:59:13.620687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:59:13.620687+0000","last_clean_scrub_stamp":"2023-11-24T07:59:13.620687+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T07:59:13.620687+0000","last_change":"2023-11-24T07:59:13.620687+0000","last_active":"2023-11-24T07:59:13.620687+0000","last_peered":"2023-11-24T07:59:13.620687+0000","last_clean":"2023-11-24T07:59:13.620687+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T07:59:13.620687+0000","last_undegraded":"2023-11-24T07:59:13.620687+0000","last_fullsized":"2023-11-24T07:59:13.620687+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:59:13.620687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:59:13.620687+0000","last_clean_scrub_stamp":"2023-11-24T07:59:13.620687+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T07:59:13.620687+0000","last_change":"2023-11-24T07:59:13.620687+0000","last_active":"2023-11-24T07:59:13.620687+0000","last_peered":"2023-11-24T07:59:13.620687+0000","last_clean":"2023-11-24T07:59:13.620687+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T07:59:13.620687+0000","last_undegraded":"2023-11-24T07:59:13.620687+0000","last_fullsized":"2023-11-24T07:59:13.620687+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:59:13.620687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:59:13.620687+0000","last_clean_scrub_stamp":"2023-11-24T07:59:13.620687+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T07:59:13.620687+0000","last_change":"2023-11-24T07:59:13.620687+0000","last_active":"2023-11-24T07:59:13.620687+0000","last_peered":"2023-11-24T07:59:13.620687+0000","last_clean":"2023-11-24T07:59:13.620687+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T07:59:13.620687+0000","last_undegraded":"2023-11-24T07:59:13.620687+0000","last_fullsized":"2023-11-24T07:59:13.620687+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:59:13.620687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:59:13.620687+0000","last_clean_scrub_stamp":"2023-11-24T07:59:13.620687+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T07:59:13.620687+0000","last_change":"2023-11-24T07:59:13.620687+0000","last_active":"2023-11-24T07:59:13.620687+0000","last_peered":"2023-11-24T07:59:13.620687+0000","last_clean":"2023-11-24T07:59:13.620687+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T07:59:13.620687+0000","last_undegraded":"2023-11-24T07:59:13.620687+0000","last_fullsized":"2023-11-24T07:59:13.620687+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:59:13.620687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:59:13.620687+0000","last_clean_scrub_stamp":"2023-11-24T07:59:13.620687+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T07:59:13.620687+0000","last_change":"2023-11-24T07:59:13.620687+0000","last_active":"2023-11-24T07:59:13.620687+0000","last_peered":"2023-11-24T07:59:13.620687+0000","last_clean":"2023-11-24T07:59:13.620687+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T07:59:13.620687+0000","last_undegraded":"2023-11-24T07:59:13.620687+0000","last_fullsized":"2023-11-24T07:59:13.620687+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:59:13.620687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:59:13.620687+0000","last_clean_scrub_stamp":"2023-11-24T07:59:13.620687+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T07:59:13.620687+0000","last_change":"2023-11-24T07:59:13.620687+0000","last_active":"2023-11-24T07:59:13.620687+0000","last_peered":"2023-11-24T07:59:13.620687+0000","last_clean":"2023-11-24T07:59:13.620687+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T07:59:13.620687+0000","last_undegraded":"2023-11-24T07:59:13.620687+0000","last_fullsized":"2023-11-24T07:59:13.620687+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:59:13.620687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:59:13.620687+0000","last_clean_scrub_stamp":"2023-11-24T07:59:13.620687+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T07:59:13.620687+0000","last_change":"2023-11-24T07:59:13.620687+0000","last_active":"2023-11-24T07:59:13.620687+0000","last_peered":"2023-11-24T07:59:13.620687+0000","last_clean":"2023-11-24T07:59:13.620687+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T07:59:13.620687+0000","last_undegraded":"2023-11-24T07:59:13.620687+0000","last_fullsized":"2023-11-24T07:59:13.620687+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:59:13.620687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:59:13.620687+0000","last_clean_scrub_stamp":"2023-11-24T07:59:13.620687+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T07:59:13.620687+0000","last_change":"2023-11-24T07:59:13.620687+0000","last_active":"2023-11-24T07:59:13.620687+0000","last_peered":"2023-11-24T07:59:13.620687+0000","last_clean":"2023-11-24T07:59:13.620687+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T07:59:13.620687+0000","last_undegraded":"2023-11-24T07:59:13.620687+0000","last_fullsized":"2023-11-24T07:59:13.620687+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:59:13.620687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:59:13.620687+0000","last_clean_scrub_stamp":"2023-11-24T07:59:13.620687+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T07:59:13.620687+0000","last_change":"2023-11-24T07:59:13.620687+0000","last_active":"2023-11-24T07:59:13.620687+0000","last_peered":"2023-11-24T07:59:13.620687+0000","last_clean":"2023-11-24T07:59:13.620687+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T07:59:13.620687+0000","last_undegraded":"2023-11-24T07:59:13.620687+0000","last_fullsized":"2023-11-24T07:59:13.620687+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:59:13.620687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:59:13.620687+0000","last_clean_scrub_stamp":"2023-11-24T07:59:13.620687+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T07:59:13.620687+0000","last_change":"2023-11-24T07:59:13.620687+0000","last_active":"2023-11-24T07:59:13.620687+0000","last_peered":"2023-11-24T07:59:13.620687+0000","last_clean":"2023-11-24T07:59:13.620687+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T07:59:13.620687+0000","last_undegraded":"2023-11-24T07:59:13.620687+0000","last_fullsized":"2023-11-24T07:59:13.620687+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:59:13.620687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:59:13.620687+0000","last_clean_scrub_stamp":"2023-11-24T07:59:13.620687+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T07:59:13.620687+0000","last_change":"2023-11-24T07:59:13.620687+0000","last_active":"2023-11-24T07:59:13.620687+0000","last_peered":"2023-11-24T07:59:13.620687+0000","last_clean":"2023-11-24T07:59:13.620687+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T07:59:13.620687+0000","last_undegraded":"2023-11-24T07:59:13.620687+0000","last_fullsized":"2023-11-24T07:59:13.620687+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:59:13.620687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:59:13.620687+0000","last_clean_scrub_stamp":"2023-11-24T07:59:13.620687+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T07:59:13.620687+0000","last_change":"2023-11-24T07:59:13.620687+0000","last_active":"2023-11-24T07:59:13.620687+0000","last_peered":"2023-11-24T07:59:13.620687+0000","last_clean":"2023-11-24T07:59:13.620687+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T07:59:13.620687+0000","last_undegraded":"2023-11-24T07:59:13.620687+0000","last_fullsized":"2023-11-24T07:59:13.620687+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:59:13.620687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:59:13.620687+0000","last_clean_scrub_stamp":"2023-11-24T07:59:13.620687+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T07:59:13.620687+0000","last_change":"2023-11-24T07:59:13.620687+0000","last_active":"2023-11-24T07:59:13.620687+0000","last_peered":"2023-11-24T07:59:13.620687+0000","last_clean":"2023-11-24T07:59:13.620687+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T07:59:13.620687+0000","last_undegraded":"2023-11-24T07:59:13.620687+0000","last_fullsized":"2023-11-24T07:59:13.620687+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:59:13.620687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:59:13.620687+0000","last_clean_scrub_stamp":"2023-11-24T07:59:13.620687+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T07:59:13.620687+0000","last_change":"2023-11-24T07:59:13.620687+0000","last_active":"2023-11-24T07:59:13.620687+0000","last_peered":"2023-11-24T07:59:13.620687+0000","last_clean":"2023-11-24T07:59:13.620687+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T07:59:13.620687+0000","last_undegraded":"2023-11-24T07:59:13.620687+0000","last_fullsized":"2023-11-24T07:59:13.620687+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:59:13.620687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:59:13.620687+0000","last_clean_scrub_stamp":"2023-11-24T07:59:13.620687+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.1f","version":"0'0","reported_seq":1,"reported_epoch":71,"state":"creating+peering","last_fresh":"2023-11-24T07:59:12.627449+0000","last_change":"2023-11-24T07:59:12.627449+0000","last_active":"2023-11-24T07:59:12.615544+0000","last_peered":"2023-11-24T07:59:12.615544+0000","last_clean":"2023-11-24T07:59:12.615544+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T07:59:12.627449+0000","last_undegraded":"2023-11-24T07:59:12.627449+0000","last_fullsized":"2023-11-24T07:59:12.627449+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:59:12.615544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:59:12.615544+0000","last_clean_scrub_stamp":"2023-11-24T07:59:12.615544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T07:59:13.620687+0000","last_change":"2023-11-24T07:59:13.620687+0000","last_active":"2023-11-24T07:59:13.620687+0000","last_peered":"2023-11-24T07:59:13.620687+0000","last_clean":"2023-11-24T07:59:13.620687+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T07:59:13.620687+0000","last_undegraded":"2023-11-24T07:59:13.620687+0000","last_fullsized":"2023-11-24T07:59:13.620687+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:59:13.620687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:59:13.620687+0000","last_clean_scrub_stamp":"2023-11-24T07:59:13.620687+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.1e","version":"0'0","reported_seq":1,"reported_epoch":71,"state":"creating+peering","last_fresh":"2023-11-24T07:59:12.622955+0000","last_change":"2023-11-24T07:59:12.622955+0000","last_active":"2023-11-24T07:59:12.615544+0000","last_peered":"2023-11-24T07:59:12.615544+0000","last_clean":"2023-11-24T07:59:12.615544+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T07:59:12.622955+0000","last_undegraded":"2023-11-24T07:59:12.622955+0000","last_fullsized":"2023-11-24T07:59:12.622955+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:59:12.615544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:59:12.615544+0000","last_clean_scrub_stamp":"2023-11-24T07:59:12.615544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T07:59:13.620687+0000","last_change":"2023-11-24T07:59:13.620687+0000","last_active":"2023-11-24T07:59:13.620687+0000","last_peered":"2023-11-24T07:59:13.620687+0000","last_clean":"2023-11-24T07:59:13.620687+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T07:59:13.620687+0000","last_undegraded":"2023-11-24T07:59:13.620687+0000","last_fullsized":"2023-11-24T07:59:13.620687+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:59:13.620687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:59:13.620687+0000","last_clean_scrub_stamp":"2023-11-24T07:59:13.620687+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.1d","version":"0'0","reported_seq":1,"reported_epoch":71,"state":"creating+peering","last_fresh":"2023-11-24T07:59:12.632766+0000","last_change":"2023-11-24T07:59:12.632766+0000","last_active":"2023-11-24T07:59:12.615544+0000","last_peered":"2023-11-24T07:59:12.615544+0000","last_clean":"2023-11-24T07:59:12.615544+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T07:59:12.632766+0000","last_undegraded":"2023-11-24T07:59:12.632766+0000","last_fullsized":"2023-11-24T07:59:12.632766+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:59:12.615544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:59:12.615544+0000","last_clean_scrub_stamp":"2023-11-24T07:59:12.615544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T07:59:13.620687+0000","last_change":"2023-11-24T07:59:13.620687+0000","last_active":"2023-11-24T07:59:13.620687+0000","last_peered":"2023-11-24T07:59:13.620687+0000","last_clean":"2023-11-24T07:59:13.620687+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T07:59:13.620687+0000","last_undegraded":"2023-11-24T07:59:13.620687+0000","last_fullsized":"2023-11-24T07:59:13.620687+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:59:13.620687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:59:13.620687+0000","last_clean_scrub_stamp":"2023-11-24T07:59:13.620687+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.1c","version":"0'0","reported_seq":1,"reported_epoch":71,"state":"creating+peering","last_fresh":"2023-11-24T07:59:12.627772+0000","last_change":"2023-11-24T07:59:12.627772+0000","last_active":"2023-11-24T07:59:12.615544+0000","last_peered":"2023-11-24T07:59:12.615544+0000","last_clean":"2023-11-24T07:59:12.615544+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T07:59:12.627772+0000","last_undegraded":"2023-11-24T07:59:12.627772+0000","last_fullsized":"2023-11-24T07:59:12.627772+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:59:12.615544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:59:12.615544+0000","last_clean_scrub_stamp":"2023-11-24T07:59:12.615544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T07:59:13.620687+0000","last_change":"2023-11-24T07:59:13.620687+0000","last_active":"2023-11-24T07:59:13.620687+0000","last_peered":"2023-11-24T07:59:13.620687+0000","last_clean":"2023-11-24T07:59:13.620687+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T07:59:13.620687+0000","last_undegraded":"2023-11-24T07:59:13.620687+0000","last_fullsized":"2023-11-24T07:59:13.620687+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:59:13.620687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:59:13.620687+0000","last_clean_scrub_stamp":"2023-11-24T07:59:13.620687+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.1b","version":"0'0","reported_seq":1,"reported_epoch":71,"state":"creating+peering","last_fresh":"2023-11-24T07:59:12.626522+0000","last_change":"2023-11-24T07:59:12.626522+0000","last_active":"2023-11-24T07:59:12.615544+0000","last_peered":"2023-11-24T07:59:12.615544+0000","last_clean":"2023-11-24T07:59:12.615544+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T07:59:12.626522+0000","last_undegraded":"2023-11-24T07:59:12.626522+0000","last_fullsized":"2023-11-24T07:59:12.626522+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:59:12.615544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:59:12.615544+0000","last_clean_scrub_stamp":"2023-11-24T07:59:12.615544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T07:59:13.620687+0000","last_change":"2023-11-24T07:59:13.620687+0000","last_active":"2023-11-24T07:59:13.620687+0000","last_peered":"2023-11-24T07:59:13.620687+0000","last_clean":"2023-11-24T07:59:13.620687+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T07:59:13.620687+0000","last_undegraded":"2023-11-24T07:59:13.620687+0000","last_fullsized":"2023-11-24T07:59:13.620687+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:59:13.620687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:59:13.620687+0000","last_clean_scrub_stamp":"2023-11-24T07:59:13.620687+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.1a","version":"0'0","reported_seq":1,"reported_epoch":71,"state":"creating+peering","last_fresh":"2023-11-24T07:59:12.624736+0000","last_change":"2023-11-24T07:59:12.624736+0000","last_active":"2023-11-24T07:59:12.615544+0000","last_peered":"2023-11-24T07:59:12.615544+0000","last_clean":"2023-11-24T07:59:12.615544+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T07:59:12.624736+0000","last_undegraded":"2023-11-24T07:59:12.624736+0000","last_fullsized":"2023-11-24T07:59:12.624736+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:59:12.615544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:59:12.615544+0000","last_clean_scrub_stamp":"2023-11-24T07:59:12.615544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T07:59:13.620687+0000","last_change":"2023-11-24T07:59:13.620687+0000","last_active":"2023-11-24T07:59:13.620687+0000","last_peered":"2023-11-24T07:59:13.620687+0000","last_clean":"2023-11-24T07:59:13.620687+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T07:59:13.620687+0000","last_undegraded":"2023-11-24T07:59:13.620687+0000","last_fullsized":"2023-11-24T07:59:13.620687+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:59:13.620687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:59:13.620687+0000","last_clean_scrub_stamp":"2023-11-24T07:59:13.620687+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.19","version":"0'0","reported_seq":1,"reported_epoch":71,"state":"creating+peering","last_fresh":"2023-11-24T07:59:12.629097+0000","last_change":"2023-11-24T07:59:12.629097+0000","last_active":"2023-11-24T07:59:12.615544+0000","last_peered":"2023-11-24T07:59:12.615544+0000","last_clean":"2023-11-24T07:59:12.615544+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T07:59:12.629097+0000","last_undegraded":"2023-11-24T07:59:12.629097+0000","last_fullsized":"2023-11-24T07:59:12.629097+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:59:12.615544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:59:12.615544+0000","last_clean_scrub_stamp":"2023-11-24T07:59:12.615544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T07:59:13.620687+0000","last_change":"2023-11-24T07:59:13.620687+0000","last_active":"2023-11-24T07:59:13.620687+0000","last_peered":"2023-11-24T07:59:13.620687+0000","last_clean":"2023-11-24T07:59:13.620687+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T07:59:13.620687+0000","last_undegraded":"2023-11-24T07:59:13.620687+0000","last_fullsized":"2023-11-24T07:59:13.620687+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:59:13.620687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:59:13.620687+0000","last_clean_scrub_stamp":"2023-11-24T07:59:13.620687+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.18","version":"0'0","reported_seq":1,"reported_epoch":71,"state":"creating+peering","last_fresh":"2023-11-24T07:59:12.625986+0000","last_change":"2023-11-24T07:59:12.625986+0000","last_active":"2023-11-24T07:59:12.615544+0000","last_peered":"2023-11-24T07:59:12.615544+0000","last_clean":"2023-11-24T07:59:12.615544+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T07:59:12.625986+0000","last_undegraded":"2023-11-24T07:59:12.625986+0000","last_fullsized":"2023-11-24T07:59:12.625986+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:59:12.615544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:59:12.615544+0000","last_clean_scrub_stamp":"2023-11-24T07:59:12.615544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T07:59:13.620687+0000","last_change":"2023-11-24T07:59:13.620687+0000","last_active":"2023-11-24T07:59:13.620687+0000","last_peered":"2023-11-24T07:59:13.620687+0000","last_clean":"2023-11-24T07:59:13.620687+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T07:59:13.620687+0000","last_undegraded":"2023-11-24T07:59:13.620687+0000","last_fullsized":"2023-11-24T07:59:13.620687+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:59:13.620687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:59:13.620687+0000","last_clean_scrub_stamp":"2023-11-24T07:59:13.620687+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.17","version":"0'0","reported_seq":1,"reported_epoch":71,"state":"creating+peering","last_fresh":"2023-11-24T07:59:12.633815+0000","last_change":"2023-11-24T07:59:12.633815+0000","last_active":"2023-11-24T07:59:12.615544+0000","last_peered":"2023-11-24T07:59:12.615544+0000","last_clean":"2023-11-24T07:59:12.615544+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T07:59:12.633815+0000","last_undegraded":"2023-11-24T07:59:12.633815+0000","last_fullsized":"2023-11-24T07:59:12.633815+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:59:12.615544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:59:12.615544+0000","last_clean_scrub_stamp":"2023-11-24T07:59:12.615544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T07:59:13.620687+0000","last_change":"2023-11-24T07:59:13.620687+0000","last_active":"2023-11-24T07:59:13.620687+0000","last_peered":"2023-11-24T07:59:13.620687+0000","last_clean":"2023-11-24T07:59:13.620687+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T07:59:13.620687+0000","last_undegraded":"2023-11-24T07:59:13.620687+0000","last_fullsized":"2023-11-24T07:59:13.620687+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:59:13.620687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:59:13.620687+0000","last_clean_scrub_stamp":"2023-11-24T07:59:13.620687+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.16","version":"0'0","reported_seq":1,"reported_epoch":71,"state":"creating+peering","last_fresh":"2023-11-24T07:59:12.627824+0000","last_change":"2023-11-24T07:59:12.627824+0000","last_active":"2023-11-24T07:59:12.615544+0000","last_peered":"2023-11-24T07:59:12.615544+0000","last_clean":"2023-11-24T07:59:12.615544+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T07:59:12.627824+0000","last_undegraded":"2023-11-24T07:59:12.627824+0000","last_fullsized":"2023-11-24T07:59:12.627824+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:59:12.615544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:59:12.615544+0000","last_clean_scrub_stamp":"2023-11-24T07:59:12.615544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T07:59:13.620687+0000","last_change":"2023-11-24T07:59:13.620687+0000","last_active":"2023-11-24T07:59:13.620687+0000","last_peered":"2023-11-24T07:59:13.620687+0000","last_clean":"2023-11-24T07:59:13.620687+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T07:59:13.620687+0000","last_undegraded":"2023-11-24T07:59:13.620687+0000","last_fullsized":"2023-11-24T07:59:13.620687+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:59:13.620687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:59:13.620687+0000","last_clean_scrub_stamp":"2023-11-24T07:59:13.620687+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.15","version":"0'0","reported_seq":1,"reported_epoch":71,"state":"creating+peering","last_fresh":"2023-11-24T07:59:12.626663+0000","last_change":"2023-11-24T07:59:12.626663+0000","last_active":"2023-11-24T07:59:12.615544+0000","last_peered":"2023-11-24T07:59:12.615544+0000","last_clean":"2023-11-24T07:59:12.615544+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T07:59:12.626663+0000","last_undegraded":"2023-11-24T07:59:12.626663+0000","last_fullsized":"2023-11-24T07:59:12.626663+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:59:12.615544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:59:12.615544+0000","last_clean_scrub_stamp":"2023-11-24T07:59:12.615544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T07:59:13.620687+0000","last_change":"2023-11-24T07:59:13.620687+0000","last_active":"2023-11-24T07:59:13.620687+0000","last_peered":"2023-11-24T07:59:13.620687+0000","last_clean":"2023-11-24T07:59:13.620687+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T07:59:13.620687+0000","last_undegraded":"2023-11-24T07:59:13.620687+0000","last_fullsized":"2023-11-24T07:59:13.620687+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:59:13.620687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:59:13.620687+0000","last_clean_scrub_stamp":"2023-11-24T07:59:13.620687+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.14","version":"0'0","reported_seq":1,"reported_epoch":71,"state":"creating+peering","last_fresh":"2023-11-24T07:59:12.629104+0000","last_change":"2023-11-24T07:59:12.629104+0000","last_active":"2023-11-24T07:59:12.615544+0000","last_peered":"2023-11-24T07:59:12.615544+0000","last_clean":"2023-11-24T07:59:12.615544+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T07:59:12.629104+0000","last_undegraded":"2023-11-24T07:59:12.629104+0000","last_fullsized":"2023-11-24T07:59:12.629104+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:59:12.615544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:59:12.615544+0000","last_clean_scrub_stamp":"2023-11-24T07:59:12.615544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T07:59:13.620687+0000","last_change":"2023-11-24T07:59:13.620687+0000","last_active":"2023-11-24T07:59:13.620687+0000","last_peered":"2023-11-24T07:59:13.620687+0000","last_clean":"2023-11-24T07:59:13.620687+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T07:59:13.620687+0000","last_undegraded":"2023-11-24T07:59:13.620687+0000","last_fullsized":"2023-11-24T07:59:13.620687+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:59:13.620687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:59:13.620687+0000","last_clean_scrub_stamp":"2023-11-24T07:59:13.620687+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.13","version":"0'0","reported_seq":1,"reported_epoch":71,"state":"creating+peering","last_fresh":"2023-11-24T07:59:12.629097+0000","last_change":"2023-11-24T07:59:12.629097+0000","last_active":"2023-11-24T07:59:12.615544+0000","last_peered":"2023-11-24T07:59:12.615544+0000","last_clean":"2023-11-24T07:59:12.615544+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T07:59:12.629097+0000","last_undegraded":"2023-11-24T07:59:12.629097+0000","last_fullsized":"2023-11-24T07:59:12.629097+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:59:12.615544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:59:12.615544+0000","last_clean_scrub_stamp":"2023-11-24T07:59:12.615544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T07:59:13.620687+0000","last_change":"2023-11-24T07:59:13.620687+0000","last_active":"2023-11-24T07:59:13.620687+0000","last_peered":"2023-11-24T07:59:13.620687+0000","last_clean":"2023-11-24T07:59:13.620687+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T07:59:13.620687+0000","last_undegraded":"2023-11-24T07:59:13.620687+0000","last_fullsized":"2023-11-24T07:59:13.620687+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:59:13.620687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:59:13.620687+0000","last_clean_scrub_stamp":"2023-11-24T07:59:13.620687+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.12","version":"0'0","reported_seq":1,"reported_epoch":71,"state":"creating+peering","last_fresh":"2023-11-24T07:59:12.624085+0000","last_change":"2023-11-24T07:59:12.624085+0000","last_active":"2023-11-24T07:59:12.615544+0000","last_peered":"2023-11-24T07:59:12.615544+0000","last_clean":"2023-11-24T07:59:12.615544+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T07:59:12.624085+0000","last_undegraded":"2023-11-24T07:59:12.624085+0000","last_fullsized":"2023-11-24T07:59:12.624085+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:59:12.615544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:59:12.615544+0000","last_clean_scrub_stamp":"2023-11-24T07:59:12.615544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T07:59:13.620687+0000","last_change":"2023-11-24T07:59:13.620687+0000","last_active":"2023-11-24T07:59:13.620687+0000","last_peered":"2023-11-24T07:59:13.620687+0000","last_clean":"2023-11-24T07:59:13.620687+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T07:59:13.620687+0000","last_undegraded":"2023-11-24T07:59:13.620687+0000","last_fullsized":"2023-11-24T07:59:13.620687+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:59:13.620687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:59:13.620687+0000","last_clean_scrub_stamp":"2023-11-24T07:59:13.620687+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.11","version":"0'0","reported_seq":1,"reported_epoch":71,"state":"creating+peering","last_fresh":"2023-11-24T07:59:12.624082+0000","last_change":"2023-11-24T07:59:12.624082+0000","last_active":"2023-11-24T07:59:12.615544+0000","last_peered":"2023-11-24T07:59:12.615544+0000","last_clean":"2023-11-24T07:59:12.615544+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T07:59:12.624082+0000","last_undegraded":"2023-11-24T07:59:12.624082+0000","last_fullsized":"2023-11-24T07:59:12.624082+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:59:12.615544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:59:12.615544+0000","last_clean_scrub_stamp":"2023-11-24T07:59:12.615544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T07:59:13.620687+0000","last_change":"2023-11-24T07:59:13.620687+0000","last_active":"2023-11-24T07:59:13.620687+0000","last_peered":"2023-11-24T07:59:13.620687+0000","last_clean":"2023-11-24T07:59:13.620687+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T07:59:13.620687+0000","last_undegraded":"2023-11-24T07:59:13.620687+0000","last_fullsized":"2023-11-24T07:59:13.620687+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:59:13.620687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:59:13.620687+0000","last_clean_scrub_stamp":"2023-11-24T07:59:13.620687+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.10","version":"0'0","reported_seq":1,"reported_epoch":71,"state":"creating+peering","last_fresh":"2023-11-24T07:59:12.624170+0000","last_change":"2023-11-24T07:59:12.624170+0000","last_active":"2023-11-24T07:59:12.615544+0000","last_peered":"2023-11-24T07:59:12.615544+0000","last_clean":"2023-11-24T07:59:12.615544+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T07:59:12.624170+0000","last_undegraded":"2023-11-24T07:59:12.624170+0000","last_fullsized":"2023-11-24T07:59:12.624170+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:59:12.615544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:59:12.615544+0000","last_clean_scrub_stamp":"2023-11-24T07:59:12.615544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T07:59:13.620687+0000","last_change":"2023-11-24T07:59:13.620687+0000","last_active":"2023-11-24T07:59:13.620687+0000","last_peered":"2023-11-24T07:59:13.620687+0000","last_clean":"2023-11-24T07:59:13.620687+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T07:59:13.620687+0000","last_undegraded":"2023-11-24T07:59:13.620687+0000","last_fullsized":"2023-11-24T07:59:13.620687+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:59:13.620687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:59:13.620687+0000","last_clean_scrub_stamp":"2023-11-24T07:59:13.620687+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.f","version":"0'0","reported_seq":1,"reported_epoch":71,"state":"creating+peering","last_fresh":"2023-11-24T07:59:12.633408+0000","last_change":"2023-11-24T07:59:12.633408+0000","last_active":"2023-11-24T07:59:12.615544+0000","last_peered":"2023-11-24T07:59:12.615544+0000","last_clean":"2023-11-24T07:59:12.615544+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T07:59:12.633408+0000","last_undegraded":"2023-11-24T07:59:12.633408+0000","last_fullsized":"2023-11-24T07:59:12.633408+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:59:12.615544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:59:12.615544+0000","last_clean_scrub_stamp":"2023-11-24T07:59:12.615544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T07:59:13.620687+0000","last_change":"2023-11-24T07:59:13.620687+0000","last_active":"2023-11-24T07:59:13.620687+0000","last_peered":"2023-11-24T07:59:13.620687+0000","last_clean":"2023-11-24T07:59:13.620687+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T07:59:13.620687+0000","last_undegraded":"2023-11-24T07:59:13.620687+0000","last_fullsized":"2023-11-24T07:59:13.620687+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:59:13.620687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:59:13.620687+0000","last_clean_scrub_stamp":"2023-11-24T07:59:13.620687+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.e","version":"0'0","reported_seq":1,"reported_epoch":71,"state":"creating+peering","last_fresh":"2023-11-24T07:59:12.627298+0000","last_change":"2023-11-24T07:59:12.627298+0000","last_active":"2023-11-24T07:59:12.615544+0000","last_peered":"2023-11-24T07:59:12.615544+0000","last_clean":"2023-11-24T07:59:12.615544+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T07:59:12.627298+0000","last_undegraded":"2023-11-24T07:59:12.627298+0000","last_fullsized":"2023-11-24T07:59:12.627298+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:59:12.615544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:59:12.615544+0000","last_clean_scrub_stamp":"2023-11-24T07:59:12.615544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T07:59:13.620687+0000","last_change":"2023-11-24T07:59:13.620687+0000","last_active":"2023-11-24T07:59:13.620687+0000","last_peered":"2023-11-24T07:59:13.620687+0000","last_clean":"2023-11-24T07:59:13.620687+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T07:59:13.620687+0000","last_undegraded":"2023-11-24T07:59:13.620687+0000","last_fullsized":"2023-11-24T07:59:13.620687+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:59:13.620687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:59:13.620687+0000","last_clean_scrub_stamp":"2023-11-24T07:59:13.620687+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.d","version":"0'0","reported_seq":1,"reported_epoch":71,"state":"creating+peering","last_fresh":"2023-11-24T07:59:12.627664+0000","last_change":"2023-11-24T07:59:12.627664+0000","last_active":"2023-11-24T07:59:12.615544+0000","last_peered":"2023-11-24T07:59:12.615544+0000","last_clean":"2023-11-24T07:59:12.615544+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T07:59:12.627664+0000","last_undegraded":"2023-11-24T07:59:12.627664+0000","last_fullsized":"2023-11-24T07:59:12.627664+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:59:12.615544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:59:12.615544+0000","last_clean_scrub_stamp":"2023-11-24T07:59:12.615544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T07:59:13.620687+0000","last_change":"2023-11-24T07:59:13.620687+0000","last_active":"2023-11-24T07:59:13.620687+0000","last_peered":"2023-11-24T07:59:13.620687+0000","last_clean":"2023-11-24T07:59:13.620687+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T07:59:13.620687+0000","last_undegraded":"2023-11-24T07:59:13.620687+0000","last_fullsized":"2023-11-24T07:59:13.620687+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:59:13.620687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:59:13.620687+0000","last_clean_scrub_stamp":"2023-11-24T07:59:13.620687+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.c","version":"0'0","reported_seq":1,"reported_epoch":71,"state":"creating+peering","last_fresh":"2023-11-24T07:59:12.627260+0000","last_change":"2023-11-24T07:59:12.627260+0000","last_active":"2023-11-24T07:59:12.615544+0000","last_peered":"2023-11-24T07:59:12.615544+0000","last_clean":"2023-11-24T07:59:12.615544+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T07:59:12.627260+0000","last_undegraded":"2023-11-24T07:59:12.627260+0000","last_fullsized":"2023-11-24T07:59:12.627260+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:59:12.615544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:59:12.615544+0000","last_clean_scrub_stamp":"2023-11-24T07:59:12.615544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T07:59:13.620687+0000","last_change":"2023-11-24T07:59:13.620687+0000","last_active":"2023-11-24T07:59:13.620687+0000","last_peered":"2023-11-24T07:59:13.620687+0000","last_clean":"2023-11-24T07:59:13.620687+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T07:59:13.620687+0000","last_undegraded":"2023-11-24T07:59:13.620687+0000","last_fullsized":"2023-11-24T07:59:13.620687+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:59:13.620687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:59:13.620687+0000","last_clean_scrub_stamp":"2023-11-24T07:59:13.620687+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.a","version":"0'0","reported_seq":1,"reported_epoch":71,"state":"creating+peering","last_fresh":"2023-11-24T07:59:12.627164+0000","last_change":"2023-11-24T07:59:12.627164+0000","last_active":"2023-11-24T07:59:12.615544+0000","last_peered":"2023-11-24T07:59:12.615544+0000","last_clean":"2023-11-24T07:59:12.615544+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T07:59:12.627164+0000","last_undegraded":"2023-11-24T07:59:12.627164+0000","last_fullsized":"2023-11-24T07:59:12.627164+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:59:12.615544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:59:12.615544+0000","last_clean_scrub_stamp":"2023-11-24T07:59:12.615544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T07:59:13.620687+0000","last_change":"2023-11-24T07:59:13.620687+0000","last_active":"2023-11-24T07:59:13.620687+0000","last_peered":"2023-11-24T07:59:13.620687+0000","last_clean":"2023-11-24T07:59:13.620687+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T07:59:13.620687+0000","last_undegraded":"2023-11-24T07:59:13.620687+0000","last_fullsized":"2023-11-24T07:59:13.620687+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:59:13.620687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:59:13.620687+0000","last_clean_scrub_stamp":"2023-11-24T07:59:13.620687+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.9","version":"0'0","reported_seq":1,"reported_epoch":71,"state":"creating+peering","last_fresh":"2023-11-24T07:59:12.628577+0000","last_change":"2023-11-24T07:59:12.628577+0000","last_active":"2023-11-24T07:59:12.615544+0000","last_peered":"2023-11-24T07:59:12.615544+0000","last_clean":"2023-11-24T07:59:12.615544+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T07:59:12.628577+0000","last_undegraded":"2023-11-24T07:59:12.628577+0000","last_fullsized":"2023-11-24T07:59:12.628577+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:59:12.615544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:59:12.615544+0000","last_clean_scrub_stamp":"2023-11-24T07:59:12.615544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T07:59:13.620687+0000","last_change":"2023-11-24T07:59:13.620687+0000","last_active":"2023-11-24T07:59:13.620687+0000","last_peered":"2023-11-24T07:59:13.620687+0000","last_clean":"2023-11-24T07:59:13.620687+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T07:59:13.620687+0000","last_undegraded":"2023-11-24T07:59:13.620687+0000","last_fullsized":"2023-11-24T07:59:13.620687+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:59:13.620687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:59:13.620687+0000","last_clean_scrub_stamp":"2023-11-24T07:59:13.620687+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.8","version":"0'0","reported_seq":1,"reported_epoch":71,"state":"creating+peering","last_fresh":"2023-11-24T07:59:12.622958+0000","last_change":"2023-11-24T07:59:12.622958+0000","last_active":"2023-11-24T07:59:12.615544+0000","last_peered":"2023-11-24T07:59:12.615544+0000","last_clean":"2023-11-24T07:59:12.615544+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T07:59:12.622958+0000","last_undegraded":"2023-11-24T07:59:12.622958+0000","last_fullsized":"2023-11-24T07:59:12.622958+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:59:12.615544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:59:12.615544+0000","last_clean_scrub_stamp":"2023-11-24T07:59:12.615544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T07:59:13.620687+0000","last_change":"2023-11-24T07:59:13.620687+0000","last_active":"2023-11-24T07:59:13.620687+0000","last_peered":"2023-11-24T07:59:13.620687+0000","last_clean":"2023-11-24T07:59:13.620687+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T07:59:13.620687+0000","last_undegraded":"2023-11-24T07:59:13.620687+0000","last_fullsized":"2023-11-24T07:59:13.620687+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:59:13.620687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:59:13.620687+0000","last_clean_scrub_stamp":"2023-11-24T07:59:13.620687+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.7","version":"0'0","reported_seq":1,"reported_epoch":71,"state":"creating+peering","last_fresh":"2023-11-24T07:59:12.632816+0000","last_change":"2023-11-24T07:59:12.632816+0000","last_active":"2023-11-24T07:59:12.615544+0000","last_peered":"2023-11-24T07:59:12.615544+0000","last_clean":"2023-11-24T07:59:12.615544+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T07:59:12.632816+0000","last_undegraded":"2023-11-24T07:59:12.632816+0000","last_fullsized":"2023-11-24T07:59:12.632816+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:59:12.615544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:59:12.615544+0000","last_clean_scrub_stamp":"2023-11-24T07:59:12.615544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T07:59:13.620687+0000","last_change":"2023-11-24T07:59:13.620687+0000","last_active":"2023-11-24T07:59:13.620687+0000","last_peered":"2023-11-24T07:59:13.620687+0000","last_clean":"2023-11-24T07:59:13.620687+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T07:59:13.620687+0000","last_undegraded":"2023-11-24T07:59:13.620687+0000","last_fullsized":"2023-11-24T07:59:13.620687+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:59:13.620687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:59:13.620687+0000","last_clean_scrub_stamp":"2023-11-24T07:59:13.620687+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.6","version":"0'0","reported_seq":1,"reported_epoch":71,"state":"creating+peering","last_fresh":"2023-11-24T07:59:12.632780+0000","last_change":"2023-11-24T07:59:12.632780+0000","last_active":"2023-11-24T07:59:12.615544+0000","last_peered":"2023-11-24T07:59:12.615544+0000","last_clean":"2023-11-24T07:59:12.615544+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T07:59:12.632780+0000","last_undegraded":"2023-11-24T07:59:12.632780+0000","last_fullsized":"2023-11-24T07:59:12.632780+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:59:12.615544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:59:12.615544+0000","last_clean_scrub_stamp":"2023-11-24T07:59:12.615544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T07:59:13.620687+0000","last_change":"2023-11-24T07:59:13.620687+0000","last_active":"2023-11-24T07:59:13.620687+0000","last_peered":"2023-11-24T07:59:13.620687+0000","last_clean":"2023-11-24T07:59:13.620687+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T07:59:13.620687+0000","last_undegraded":"2023-11-24T07:59:13.620687+0000","last_fullsized":"2023-11-24T07:59:13.620687+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:59:13.620687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:59:13.620687+0000","last_clean_scrub_stamp":"2023-11-24T07:59:13.620687+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.5","version":"0'0","reported_seq":1,"reported_epoch":71,"state":"creating+peering","last_fresh":"2023-11-24T07:59:12.626089+0000","last_change":"2023-11-24T07:59:12.626089+0000","last_active":"2023-11-24T07:59:12.615544+0000","last_peered":"2023-11-24T07:59:12.615544+0000","last_clean":"2023-11-24T07:59:12.615544+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T07:59:12.626089+0000","last_undegraded":"2023-11-24T07:59:12.626089+0000","last_fullsized":"2023-11-24T07:59:12.626089+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:59:12.615544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:59:12.615544+0000","last_clean_scrub_stamp":"2023-11-24T07:59:12.615544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T07:59:13.620687+0000","last_change":"2023-11-24T07:59:13.620687+0000","last_active":"2023-11-24T07:59:13.620687+0000","last_peered":"2023-11-24T07:59:13.620687+0000","last_clean":"2023-11-24T07:59:13.620687+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T07:59:13.620687+0000","last_undegraded":"2023-11-24T07:59:13.620687+0000","last_fullsized":"2023-11-24T07:59:13.620687+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:59:13.620687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:59:13.620687+0000","last_clean_scrub_stamp":"2023-11-24T07:59:13.620687+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.4","version":"0'0","reported_seq":1,"reported_epoch":71,"state":"creating+peering","last_fresh":"2023-11-24T07:59:12.626088+0000","last_change":"2023-11-24T07:59:12.626088+0000","last_active":"2023-11-24T07:59:12.615544+0000","last_peered":"2023-11-24T07:59:12.615544+0000","last_clean":"2023-11-24T07:59:12.615544+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T07:59:12.626088+0000","last_undegraded":"2023-11-24T07:59:12.626088+0000","last_fullsized":"2023-11-24T07:59:12.626088+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:59:12.615544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:59:12.615544+0000","last_clean_scrub_stamp":"2023-11-24T07:59:12.615544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T07:59:13.620687+0000","last_change":"2023-11-24T07:59:13.620687+0000","last_active":"2023-11-24T07:59:13.620687+0000","last_peered":"2023-11-24T07:59:13.620687+0000","last_clean":"2023-11-24T07:59:13.620687+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T07:59:13.620687+0000","last_undegraded":"2023-11-24T07:59:13.620687+0000","last_fullsized":"2023-11-24T07:59:13.620687+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:59:13.620687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:59:13.620687+0000","last_clean_scrub_stamp":"2023-11-24T07:59:13.620687+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.3","version":"0'0","reported_seq":1,"reported_epoch":71,"state":"creating+peering","last_fresh":"2023-11-24T07:59:12.626018+0000","last_change":"2023-11-24T07:59:12.626018+0000","last_active":"2023-11-24T07:59:12.615544+0000","last_peered":"2023-11-24T07:59:12.615544+0000","last_clean":"2023-11-24T07:59:12.615544+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T07:59:12.626018+0000","last_undegraded":"2023-11-24T07:59:12.626018+0000","last_fullsized":"2023-11-24T07:59:12.626018+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:59:12.615544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:59:12.615544+0000","last_clean_scrub_stamp":"2023-11-24T07:59:12.615544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T07:59:13.620687+0000","last_change":"2023-11-24T07:59:13.620687+0000","last_active":"2023-11-24T07:59:13.620687+0000","last_peered":"2023-11-24T07:59:13.620687+0000","last_clean":"2023-11-24T07:59:13.620687+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T07:59:13.620687+0000","last_undegraded":"2023-11-24T07:59:13.620687+0000","last_fullsized":"2023-11-24T07:59:13.620687+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:59:13.620687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:59:13.620687+0000","last_clean_scrub_stamp":"2023-11-24T07:59:13.620687+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.2","version":"0'0","reported_seq":1,"reported_epoch":71,"state":"creating+peering","last_fresh":"2023-11-24T07:59:12.622957+0000","last_change":"2023-11-24T07:59:12.622957+0000","last_active":"2023-11-24T07:59:12.615544+0000","last_peered":"2023-11-24T07:59:12.615544+0000","last_clean":"2023-11-24T07:59:12.615544+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T07:59:12.622957+0000","last_undegraded":"2023-11-24T07:59:12.622957+0000","last_fullsized":"2023-11-24T07:59:12.622957+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:59:12.615544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:59:12.615544+0000","last_clean_scrub_stamp":"2023-11-24T07:59:12.615544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T07:59:13.620687+0000","last_change":"2023-11-24T07:59:13.620687+0000","last_active":"2023-11-24T07:59:13.620687+0000","last_peered":"2023-11-24T07:59:13.620687+0000","last_clean":"2023-11-24T07:59:13.620687+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T07:59:13.620687+0000","last_undegraded":"2023-11-24T07:59:13.620687+0000","last_fullsized":"2023-11-24T07:59:13.620687+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:59:13.620687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:59:13.620687+0000","last_clean_scrub_stamp":"2023-11-24T07:59:13.620687+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.1","version":"0'0","reported_seq":1,"reported_epoch":71,"state":"creating+peering","last_fresh":"2023-11-24T07:59:12.628166+0000","last_change":"2023-11-24T07:59:12.628166+0000","last_active":"2023-11-24T07:59:12.615544+0000","last_peered":"2023-11-24T07:59:12.615544+0000","last_clean":"2023-11-24T07:59:12.615544+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T07:59:12.628166+0000","last_undegraded":"2023-11-24T07:59:12.628166+0000","last_fullsized":"2023-11-24T07:59:12.628166+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:59:12.615544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:59:12.615544+0000","last_clean_scrub_stamp":"2023-11-24T07:59:12.615544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T07:59:13.620687+0000","last_change":"2023-11-24T07:59:13.620687+0000","last_active":"2023-11-24T07:59:13.620687+0000","last_peered":"2023-11-24T07:59:13.620687+0000","last_clean":"2023-11-24T07:59:13.620687+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T07:59:13.620687+0000","last_undegraded":"2023-11-24T07:59:13.620687+0000","last_fullsized":"2023-11-24T07:59:13.620687+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:59:13.620687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:59:13.620687+0000","last_clean_scrub_stamp":"2023-11-24T07:59:13.620687+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.0","version":"0'0","reported_seq":1,"reported_epoch":71,"state":"creating+peering","last_fresh":"2023-11-24T07:59:12.627478+0000","last_change":"2023-11-24T07:59:12.627478+0000","last_active":"2023-11-24T07:59:12.615544+0000","last_peered":"2023-11-24T07:59:12.615544+0000","last_clean":"2023-11-24T07:59:12.615544+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T07:59:12.627478+0000","last_undegraded":"2023-11-24T07:59:12.627478+0000","last_fullsized":"2023-11-24T07:59:12.627478+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:59:12.615544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:59:12.615544+0000","last_clean_scrub_stamp":"2023-11-24T07:59:12.615544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T07:59:13.620687+0000","last_change":"2023-11-24T07:59:13.620687+0000","last_active":"2023-11-24T07:59:13.620687+0000","last_peered":"2023-11-24T07:59:13.620687+0000","last_clean":"2023-11-24T07:59:13.620687+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T07:59:13.620687+0000","last_undegraded":"2023-11-24T07:59:13.620687+0000","last_fullsized":"2023-11-24T07:59:13.620687+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:59:13.620687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:59:13.620687+0000","last_clean_scrub_stamp":"2023-11-24T07:59:13.620687+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.b","version":"0'0","reported_seq":1,"reported_epoch":71,"state":"creating+peering","last_fresh":"2023-11-24T07:59:12.624115+0000","last_change":"2023-11-24T07:59:12.624115+0000","last_active":"2023-11-24T07:59:12.615544+0000","last_peered":"2023-11-24T07:59:12.615544+0000","last_clean":"2023-11-24T07:59:12.615544+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T07:59:12.624115+0000","last_undegraded":"2023-11-24T07:59:12.624115+0000","last_fullsized":"2023-11-24T07:59:12.624115+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:59:12.615544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:59:12.615544+0000","last_clean_scrub_stamp":"2023-11-24T07:59:12.615544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":208,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-11-24T07:59:09.991344+0000","last_change":"2023-11-24T07:56:19.668905+0000","last_active":"2023-11-24T07:59:09.991344+0000","last_peered":"2023-11-24T07:59:09.991344+0000","last_clean":"2023-11-24T07:59:09.991344+0000","last_became_active":"2023-11-24T07:56:19.668601+0000","last_became_peered":"2023-11-24T07:56:19.668601+0000","last_unstale":"2023-11-24T07:59:09.991344+0000","last_undegraded":"2023-11-24T07:59:09.991344+0000","last_fullsized":"2023-11-24T07:59:09.991344+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:56:18.657844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:56:18.657844+0000","last_clean_scrub_stamp":"2023-11-24T07:56:18.657844+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T11:51:53.828455+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":11,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":10,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574886,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27872,"kb_used_data":844,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343968,"statfs":{"total":96636764160,"available":96608223232,"internally_reserved":0,"allocated":864256,"data_stored":364790,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 07:58:20 2023","interfaces":[{"interface":"back","average":{"1min":0.51400000000000001,"5min":0.54800000000000004,"15min":0.55400000000000005},"min":{"1min":0.309,"5min":0.309,"15min":0.309},"max":{"1min":0.78500000000000003,"5min":1.413,"15min":1.413},"last":0.55200000000000005},{"interface":"front","average":{"1min":0.56299999999999994,"5min":0.55500000000000005,"15min":0.55400000000000005},"min":{"1min":0.436,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":0.77000000000000002,"5min":0.81000000000000005,"15min":0.81000000000000005},"last":0.56599999999999995}]},{"osd":1,"last update":"Fri Nov 24 07:58:20 2023","interfaces":[{"interface":"back","average":{"1min":0.501,"5min":0.77900000000000003,"15min":0.82499999999999996},"min":{"1min":0.32000000000000001,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.69899999999999995,"5min":6.1070000000000002,"15min":6.1070000000000002},"last":0.53100000000000003},{"interface":"front","average":{"1min":0.51300000000000001,"5min":0.56599999999999995,"15min":0.57499999999999996},"min":{"1min":0.40699999999999997,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.73599999999999999,"5min":1.5920000000000001,"15min":1.5920000000000001},"last":0.49399999999999999}]},{"osd":2,"last update":"Fri Nov 24 07:58:20 2023","interfaces":[{"interface":"back","average":{"1min":0.498,"5min":0.60099999999999998,"15min":0.61799999999999999},"min":{"1min":0.35099999999999998,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.71599999999999997,"5min":1.613,"15min":1.613},"last":0.40100000000000002},{"interface":"front","average":{"1min":0.502,"5min":0.73399999999999999,"15min":0.77200000000000002},"min":{"1min":0.34899999999999998,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.71299999999999997,"5min":4.5220000000000002,"15min":4.5220000000000002},"last":0.44700000000000001}]},{"osd":3,"last update":"Fri Nov 24 07:58:20 2023","interfaces":[{"interface":"back","average":{"1min":0.55100000000000005,"5min":0.755,"15min":0.78900000000000003},"min":{"1min":0.41699999999999998,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.754,"5min":4.6760000000000002,"15min":4.6760000000000002},"last":0.626},{"interface":"front","average":{"1min":0.60699999999999998,"5min":0.875,"15min":0.92000000000000004},"min":{"1min":0.38500000000000001,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.81599999999999995,"5min":4.6539999999999999,"15min":4.6539999999999999},"last":0.64400000000000002}]},{"osd":4,"last update":"Fri Nov 24 07:58:20 2023","interfaces":[{"interface":"back","average":{"1min":0.54900000000000004,"5min":0.56399999999999995,"15min":0.56599999999999995},"min":{"1min":0.41799999999999998,"5min":0.378,"15min":0.378},"max":{"1min":0.70999999999999996,"5min":0.84699999999999998,"15min":0.84699999999999998},"last":0.42699999999999999},{"interface":"front","average":{"1min":0.54700000000000004,"5min":0.64300000000000002,"15min":0.65900000000000003},"min":{"1min":0.35199999999999998,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.79900000000000004,"5min":1.629,"15min":1.629},"last":0.59499999999999997}]},{"osd":5,"last update":"Fri Nov 24 07:58:20 2023","interfaces":[{"interface":"back","average":{"1min":0.54200000000000004,"5min":0.54600000000000004,"15min":0.54600000000000004},"min":{"1min":0.41599999999999998,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.67900000000000005,"5min":0.875,"15min":0.875},"last":0.51600000000000001},{"interface":"front","average":{"1min":0.57799999999999996,"5min":0.61399999999999999,"15min":0.62},"min":{"1min":0.36799999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.77700000000000002,"5min":0.90900000000000003,"15min":0.90900000000000003},"last":0.57999999999999996}]},{"osd":6,"last update":"Fri Nov 24 07:58:20 2023","interfaces":[{"interface":"back","average":{"1min":0.59999999999999998,"5min":0.60799999999999998,"15min":0.60999999999999999},"min":{"1min":0.49099999999999999,"5min":0.438,"15min":0.438},"max":{"1min":0.73399999999999999,"5min":0.80800000000000005,"15min":0.80800000000000005},"last":0.61299999999999999},{"interface":"front","average":{"1min":0.63900000000000001,"5min":0.66300000000000003,"15min":0.66700000000000004},"min":{"1min":0.45600000000000002,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":0.79700000000000004,"5min":0.98399999999999999,"15min":0.98399999999999999},"last":0.66100000000000003}]}]},{"osd":6,"up_from":13,"seq":55834574887,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27872,"kb_used_data":844,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343968,"statfs":{"total":96636764160,"available":96608223232,"internally_reserved":0,"allocated":864256,"data_stored":364790,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 07:58:19 2023","interfaces":[{"interface":"back","average":{"1min":0.53300000000000003,"5min":0.54600000000000004,"15min":0.54800000000000004},"min":{"1min":0.29799999999999999,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":0.93100000000000005,"5min":0.93100000000000005,"15min":0.93100000000000005},"last":0.497},{"interface":"front","average":{"1min":0.51600000000000001,"5min":0.51600000000000001,"15min":0.51600000000000001},"min":{"1min":0.311,"5min":0.311,"15min":0.311},"max":{"1min":0.82199999999999995,"5min":0.82199999999999995,"15min":0.82199999999999995},"last":0.34899999999999998}]},{"osd":1,"last update":"Fri Nov 24 07:58:19 2023","interfaces":[{"interface":"back","average":{"1min":0.51900000000000002,"5min":0.51300000000000001,"15min":0.51200000000000001},"min":{"1min":0.35599999999999998,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.81299999999999994,"5min":0.81299999999999994,"15min":0.81299999999999994},"last":0.48399999999999999},{"interface":"front","average":{"1min":0.54000000000000004,"5min":0.59799999999999998,"15min":0.60799999999999998},"min":{"1min":0.35999999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.82799999999999996,"5min":1.0069999999999999,"15min":1.0069999999999999},"last":0.50900000000000001}]},{"osd":2,"last update":"Fri Nov 24 07:58:19 2023","interfaces":[{"interface":"back","average":{"1min":0.58399999999999996,"5min":0.57599999999999996,"15min":0.57499999999999996},"min":{"1min":0.34899999999999998,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.90600000000000003,"5min":0.90600000000000003,"15min":0.90600000000000003},"last":0.52600000000000002},{"interface":"front","average":{"1min":0.56499999999999995,"5min":0.58699999999999997,"15min":0.59099999999999997},"min":{"1min":0.34300000000000003,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.84899999999999998,"5min":0.84899999999999998,"15min":0.84899999999999998},"last":0.53700000000000003}]},{"osd":3,"last update":"Fri Nov 24 07:58:19 2023","interfaces":[{"interface":"back","average":{"1min":0.56000000000000005,"5min":0.56599999999999995,"15min":0.56699999999999995},"min":{"1min":0.38400000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.86499999999999999,"5min":0.876,"15min":0.876},"last":0.40100000000000002},{"interface":"front","average":{"1min":0.57699999999999996,"5min":0.56000000000000005,"15min":0.55700000000000005},"min":{"1min":0.39800000000000002,"5min":0.379,"15min":0.379},"max":{"1min":0.876,"5min":0.876,"15min":0.876},"last":0.41699999999999998}]},{"osd":4,"last update":"Fri Nov 24 07:58:19 2023","interfaces":[{"interface":"back","average":{"1min":0.53100000000000003,"5min":0.51900000000000002,"15min":0.51700000000000002},"min":{"1min":0.25,"5min":0.25,"15min":0.25},"max":{"1min":0.89400000000000002,"5min":0.89400000000000002,"15min":0.89400000000000002},"last":0.58299999999999996},{"interface":"front","average":{"1min":0.58899999999999997,"5min":0.59899999999999998,"15min":0.60099999999999998},"min":{"1min":0.29099999999999998,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":1.044,"5min":1.087,"15min":1.087},"last":0.30599999999999999}]},{"osd":5,"last update":"Fri Nov 24 07:58:19 2023","interfaces":[{"interface":"back","average":{"1min":0.59499999999999997,"5min":0.54300000000000004,"15min":0.53500000000000003},"min":{"1min":0.27800000000000002,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":0.97999999999999998,"5min":0.97999999999999998,"15min":0.97999999999999998},"last":0.37},{"interface":"front","average":{"1min":0.61399999999999999,"5min":0.61199999999999999,"15min":0.61199999999999999},"min":{"1min":0.31900000000000001,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":1.0009999999999999,"5min":1.0009999999999999,"15min":1.0009999999999999},"last":0.438}]},{"osd":7,"last update":"Fri Nov 24 07:58:19 2023","interfaces":[{"interface":"back","average":{"1min":0.61799999999999999,"5min":0.61799999999999999,"15min":0.61799999999999999},"min":{"1min":0.26500000000000001,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":1.0880000000000001,"5min":1.0880000000000001,"15min":1.0880000000000001},"last":0.46400000000000002},{"interface":"front","average":{"1min":0.65200000000000002,"5min":0.66600000000000004,"15min":0.66800000000000004},"min":{"1min":0.34300000000000003,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.95599999999999996,"5min":0.95599999999999996,"15min":0.95599999999999996},"last":0.32800000000000001}]}]},{"osd":1,"up_from":13,"seq":55834574888,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28432,"kb_used_data":1412,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343408,"statfs":{"total":96636764160,"available":96607649792,"internally_reserved":0,"allocated":1445888,"data_stored":949341,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 07:58:24 2023","interfaces":[{"interface":"back","average":{"1min":0.33000000000000002,"5min":0.35699999999999998,"15min":0.36099999999999999},"min":{"1min":0.249,"5min":0.20200000000000001,"15min":0.20200000000000001},"max":{"1min":0.48799999999999999,"5min":0.70899999999999996,"15min":0.70899999999999996},"last":0.249},{"interface":"front","average":{"1min":0.36399999999999999,"5min":0.39100000000000001,"15min":0.39500000000000002},"min":{"1min":0.23599999999999999,"5min":0.214,"15min":0.214},"max":{"1min":0.54700000000000004,"5min":0.82499999999999996,"15min":0.82499999999999996},"last":0.309}]},{"osd":2,"last update":"Fri Nov 24 07:58:24 2023","interfaces":[{"interface":"back","average":{"1min":0.312,"5min":0.35999999999999999,"15min":0.36799999999999999},"min":{"1min":0.253,"5min":0.22700000000000001,"15min":0.22700000000000001},"max":{"1min":0.44800000000000001,"5min":0.60999999999999999,"15min":0.60999999999999999},"last":0.40699999999999997},{"interface":"front","average":{"1min":0.36499999999999999,"5min":0.40300000000000002,"15min":0.40899999999999997},"min":{"1min":0.28100000000000003,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":0.47599999999999998,"5min":0.621,"15min":0.621},"last":0.28899999999999998}]},{"osd":3,"last update":"Fri Nov 24 07:58:24 2023","interfaces":[{"interface":"back","average":{"1min":0.40300000000000002,"5min":0.435,"15min":0.441},"min":{"1min":0.31900000000000001,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.59199999999999997,"5min":0.66500000000000004,"15min":0.66500000000000004},"last":0.34699999999999998},{"interface":"front","average":{"1min":0.38700000000000001,"5min":0.41899999999999998,"15min":0.42499999999999999},"min":{"1min":0.29199999999999998,"5min":0.28499999999999998,"15min":0.28499999999999998},"max":{"1min":0.56200000000000006,"5min":0.76300000000000001,"15min":0.76300000000000001},"last":0.36599999999999999}]},{"osd":4,"last update":"Fri Nov 24 07:58:24 2023","interfaces":[{"interface":"back","average":{"1min":0.56200000000000006,"5min":0.60099999999999998,"15min":0.60699999999999998},"min":{"1min":0.442,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":0.84099999999999997,"5min":0.84099999999999997,"15min":0.84099999999999997},"last":0.64200000000000002},{"interface":"front","average":{"1min":0.61099999999999999,"5min":0.60699999999999998,"15min":0.60599999999999998},"min":{"1min":0.499,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":0.79500000000000004,"5min":0.82199999999999995,"15min":0.82199999999999995},"last":0.67800000000000005}]},{"osd":5,"last update":"Fri Nov 24 07:58:24 2023","interfaces":[{"interface":"back","average":{"1min":0.59199999999999997,"5min":0.69799999999999995,"15min":0.71599999999999997},"min":{"1min":0.439,"5min":0.37,"15min":0.37},"max":{"1min":0.97599999999999998,"5min":2.7269999999999999,"15min":2.7269999999999999},"last":0.69399999999999995},{"interface":"front","average":{"1min":0.57699999999999996,"5min":0.63100000000000001,"15min":0.64000000000000001},"min":{"1min":0.39100000000000001,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.94499999999999995,"5min":0.94499999999999995,"15min":0.94499999999999995},"last":0.60999999999999999}]},{"osd":6,"last update":"Fri Nov 24 07:58:24 2023","interfaces":[{"interface":"back","average":{"1min":0.61199999999999999,"5min":0.64800000000000002,"15min":0.65400000000000003},"min":{"1min":0.47699999999999998,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":0.77500000000000002,"5min":0.98199999999999998,"15min":0.98199999999999998},"last":0.70299999999999996},{"interface":"front","average":{"1min":0.57599999999999996,"5min":0.60799999999999998,"15min":0.61399999999999999},"min":{"1min":0.40600000000000003,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":0.89600000000000002,"5min":0.90700000000000003,"15min":0.90700000000000003},"last":0.623}]},{"osd":7,"last update":"Fri Nov 24 07:58:24 2023","interfaces":[{"interface":"back","average":{"1min":0.63100000000000001,"5min":0.63500000000000001,"15min":0.63600000000000001},"min":{"1min":0.43099999999999999,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.79800000000000004,"5min":1.0649999999999999,"15min":1.0649999999999999},"last":0.71399999999999997},{"interface":"front","average":{"1min":0.57099999999999995,"5min":0.65000000000000002,"15min":0.66300000000000003},"min":{"1min":0.45700000000000002,"5min":0.442,"15min":0.442},"max":{"1min":0.71299999999999997,"5min":0.88800000000000001,"15min":0.88800000000000001},"last":0.66700000000000004}]}]},{"osd":0,"up_from":13,"seq":55834574888,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27856,"kb_used_data":832,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343984,"statfs":{"total":96636764160,"available":96608239616,"internally_reserved":0,"allocated":851968,"data_stored":358973,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Nov 24 07:58:20 2023","interfaces":[{"interface":"back","average":{"1min":0.38500000000000001,"5min":0.64700000000000002,"15min":0.69099999999999995},"min":{"1min":0.21299999999999999,"5min":0.21299999999999999,"15min":0.21299999999999999},"max":{"1min":0.625,"5min":6.3550000000000004,"15min":6.3550000000000004},"last":0.34000000000000002},{"interface":"front","average":{"1min":0.40799999999999997,"5min":1.464,"15min":1.6399999999999999},"min":{"1min":0.26000000000000001,"5min":0.26000000000000001,"15min":0.26000000000000001},"max":{"1min":0.60999999999999999,"5min":22.916,"15min":22.916},"last":0.29899999999999999}]},{"osd":2,"last update":"Fri Nov 24 07:58:20 2023","interfaces":[{"interface":"back","average":{"1min":0.35299999999999998,"5min":0.63,"15min":0.67600000000000005},"min":{"1min":0.22900000000000001,"5min":0.22800000000000001,"15min":0.22800000000000001},"max":{"1min":0.499,"5min":6.3899999999999997,"15min":6.3899999999999997},"last":0.27700000000000002},{"interface":"front","average":{"1min":0.39500000000000002,"5min":0.92900000000000005,"15min":1.018},"min":{"1min":0.27900000000000003,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":0.60199999999999998,"5min":11.93,"15min":11.93},"last":0.32800000000000001}]},{"osd":3,"last update":"Fri Nov 24 07:58:20 2023","interfaces":[{"interface":"back","average":{"1min":0.373,"5min":0.40999999999999998,"15min":0.41599999999999998},"min":{"1min":0.23799999999999999,"5min":0.23799999999999999,"15min":0.23799999999999999},"max":{"1min":0.79300000000000004,"5min":1.155,"15min":1.155},"last":0.247},{"interface":"front","average":{"1min":0.438,"5min":0.98899999999999999,"15min":1.081},"min":{"1min":0.30199999999999999,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.69699999999999995,"5min":12.116,"15min":12.116},"last":0.313}]},{"osd":4,"last update":"Fri Nov 24 07:58:20 2023","interfaces":[{"interface":"back","average":{"1min":0.626,"5min":0.64300000000000002,"15min":0.64600000000000002},"min":{"1min":0.40100000000000002,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":1.044,"5min":1.044,"15min":1.044},"last":0.44400000000000001},{"interface":"front","average":{"1min":0.64300000000000002,"5min":1.675,"15min":1.847},"min":{"1min":0.35699999999999998,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.93000000000000005,"5min":22.963999999999999,"15min":22.963999999999999},"last":0.377}]},{"osd":5,"last update":"Fri Nov 24 07:58:20 2023","interfaces":[{"interface":"back","average":{"1min":0.65200000000000002,"5min":1.1639999999999999,"15min":1.25},"min":{"1min":0.41999999999999998,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.97799999999999998,"5min":12.084,"15min":12.084},"last":0.436},{"interface":"front","average":{"1min":0.65800000000000003,"5min":0.67400000000000004,"15min":0.67600000000000005},"min":{"1min":0.39700000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.95499999999999996,"5min":1.073,"15min":1.073},"last":0.503}]},{"osd":6,"last update":"Fri Nov 24 07:58:20 2023","interfaces":[{"interface":"back","average":{"1min":0.65200000000000002,"5min":0.68300000000000005,"15min":0.68799999999999994},"min":{"1min":0.45000000000000001,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":0.94699999999999995,"5min":1.0580000000000001,"15min":1.0580000000000001},"last":0.36799999999999999},{"interface":"front","average":{"1min":0.67000000000000004,"5min":1.1850000000000001,"15min":1.2709999999999999},"min":{"1min":0.46200000000000002,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":0.95899999999999996,"5min":12.045,"15min":12.045},"last":0.46000000000000002}]},{"osd":7,"last update":"Fri Nov 24 07:58:20 2023","interfaces":[{"interface":"back","average":{"1min":0.67000000000000004,"5min":0.65800000000000003,"15min":0.65600000000000003},"min":{"1min":0.45200000000000001,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":0.98299999999999998,"5min":1.038,"15min":1.038},"last":0.46700000000000003},{"interface":"front","average":{"1min":0.67500000000000004,"5min":1.7050000000000001,"15min":1.877},"min":{"1min":0.41299999999999998,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":1.0069999999999999,"5min":22.995000000000001,"15min":22.995000000000001},"last":0.42299999999999999}]}]},{"osd":2,"up_from":13,"seq":55834574888,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27868,"kb_used_data":844,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343972,"statfs":{"total":96636764160,"available":96608227328,"internally_reserved":0,"allocated":864256,"data_stored":364790,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 07:58:19 2023","interfaces":[{"interface":"back","average":{"1min":0.36399999999999999,"5min":0.36899999999999999,"15min":0.37},"min":{"1min":0.219,"5min":0.219,"15min":0.219},"max":{"1min":0.55900000000000005,"5min":0.58099999999999996,"15min":0.58099999999999996},"last":0.32500000000000001},{"interface":"front","average":{"1min":0.38900000000000001,"5min":0.39900000000000002,"15min":0.40100000000000002},"min":{"1min":0.29299999999999998,"5min":0.27400000000000002,"15min":0.27400000000000002},"max":{"1min":0.54100000000000004,"5min":0.54100000000000004,"15min":0.54100000000000004},"last":0.41499999999999998}]},{"osd":1,"last update":"Fri Nov 24 07:58:19 2023","interfaces":[{"interface":"back","average":{"1min":0.34699999999999998,"5min":0.371,"15min":0.375},"min":{"1min":0.22,"5min":0.22,"15min":0.22},"max":{"1min":0.54000000000000004,"5min":0.59199999999999997,"15min":0.59199999999999997},"last":0.30599999999999999},{"interface":"front","average":{"1min":0.36599999999999999,"5min":0.40600000000000003,"15min":0.41199999999999998},"min":{"1min":0.28299999999999997,"5min":0.25600000000000001,"15min":0.25600000000000001},"max":{"1min":0.497,"5min":0.627,"15min":0.627},"last":0.34300000000000003}]},{"osd":3,"last update":"Fri Nov 24 07:58:19 2023","interfaces":[{"interface":"back","average":{"1min":0.40500000000000003,"5min":0.40899999999999997,"15min":0.40999999999999998},"min":{"1min":0.30399999999999999,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":0.64200000000000002,"5min":0.64200000000000002,"15min":0.64200000000000002},"last":0.26400000000000001},{"interface":"front","average":{"1min":0.41699999999999998,"5min":0.41399999999999998,"15min":0.41399999999999998},"min":{"1min":0.28599999999999998,"5min":0.27500000000000002,"15min":0.27500000000000002},"max":{"1min":0.59299999999999997,"5min":0.64600000000000002,"15min":0.64600000000000002},"last":0.36799999999999999}]},{"osd":4,"last update":"Fri Nov 24 07:58:19 2023","interfaces":[{"interface":"back","average":{"1min":0.54800000000000004,"5min":0.59599999999999997,"15min":0.60399999999999998},"min":{"1min":0.38700000000000001,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.75,"5min":0.81599999999999995,"15min":0.81599999999999995},"last":0.46800000000000003},{"interface":"front","average":{"1min":0.55500000000000005,"5min":0.59099999999999997,"15min":0.59699999999999998},"min":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":0.83799999999999997,"5min":0.83799999999999997,"15min":0.83799999999999997},"last":0.60299999999999998}]},{"osd":5,"last update":"Fri Nov 24 07:58:19 2023","interfaces":[{"interface":"back","average":{"1min":0.56000000000000005,"5min":0.60799999999999998,"15min":0.61599999999999999},"min":{"1min":0.434,"5min":0.434,"15min":0.434},"max":{"1min":0.73499999999999999,"5min":0.84899999999999998,"15min":0.84899999999999998},"last":0.503},{"interface":"front","average":{"1min":0.624,"5min":0.69599999999999995,"15min":0.70799999999999996},"min":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":0.82499999999999996,"5min":2.1699999999999999,"15min":2.1699999999999999},"last":0.53500000000000003}]},{"osd":6,"last update":"Fri Nov 24 07:58:19 2023","interfaces":[{"interface":"back","average":{"1min":0.60499999999999998,"5min":0.65400000000000003,"15min":0.66200000000000003},"min":{"1min":0.42499999999999999,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":0.878,"5min":0.878,"15min":0.878},"last":0.48299999999999998},{"interface":"front","average":{"1min":0.60599999999999998,"5min":0.65800000000000003,"15min":0.66600000000000004},"min":{"1min":0.41899999999999998,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":0.80300000000000005,"5min":0.88500000000000001,"15min":0.88500000000000001},"last":0.54900000000000004}]},{"osd":7,"last update":"Fri Nov 24 07:58:19 2023","interfaces":[{"interface":"back","average":{"1min":0.60399999999999998,"5min":0.65000000000000002,"15min":0.65800000000000003},"min":{"1min":0.441,"5min":0.441,"15min":0.441},"max":{"1min":0.77600000000000002,"5min":0.85999999999999999,"15min":0.85999999999999999},"last":0.56799999999999995},{"interface":"front","average":{"1min":0.623,"5min":0.66800000000000004,"15min":0.67600000000000005},"min":{"1min":0.40000000000000002,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":0.90000000000000002,"5min":0.90000000000000002,"15min":0.90000000000000002},"last":0.58399999999999996}]}]},{"osd":3,"up_from":13,"seq":55834574888,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27868,"kb_used_data":844,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343972,"statfs":{"total":96636764160,"available":96608227328,"internally_reserved":0,"allocated":864256,"data_stored":364790,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 07:58:22 2023","interfaces":[{"interface":"back","average":{"1min":0.34200000000000003,"5min":0.35299999999999998,"15min":0.35499999999999998},"min":{"1min":0.20899999999999999,"5min":0.20899999999999999,"15min":0.20899999999999999},"max":{"1min":0.439,"5min":0.84799999999999998,"15min":0.84799999999999998},"last":0.29099999999999998},{"interface":"front","average":{"1min":0.32700000000000001,"5min":0.35899999999999999,"15min":0.36399999999999999},"min":{"1min":0.253,"5min":0.23699999999999999,"15min":0.23699999999999999},"max":{"1min":0.42499999999999999,"5min":0.873,"15min":0.873},"last":0.44900000000000001}]},{"osd":1,"last update":"Fri Nov 24 07:58:22 2023","interfaces":[{"interface":"back","average":{"1min":0.35899999999999999,"5min":0.371,"15min":0.373},"min":{"1min":0.29499999999999998,"5min":0.25700000000000001,"15min":0.25700000000000001},"max":{"1min":0.443,"5min":0.92000000000000004,"15min":0.92000000000000004},"last":0.54000000000000004},{"interface":"front","average":{"1min":0.35099999999999998,"5min":0.379,"15min":0.38300000000000001},"min":{"1min":0.28299999999999997,"5min":0.248,"15min":0.248},"max":{"1min":0.46700000000000003,"5min":0.81699999999999995,"15min":0.81699999999999995},"last":0.40799999999999997}]},{"osd":2,"last update":"Fri Nov 24 07:58:22 2023","interfaces":[{"interface":"back","average":{"1min":0.33400000000000002,"5min":0.35499999999999998,"15min":0.35899999999999999},"min":{"1min":0.255,"5min":0.24399999999999999,"15min":0.24399999999999999},"max":{"1min":0.43099999999999999,"5min":0.88500000000000001,"15min":0.88500000000000001},"last":0.42999999999999999},{"interface":"front","average":{"1min":0.39500000000000002,"5min":0.45300000000000001,"15min":0.46300000000000002},"min":{"1min":0.311,"5min":0.311,"15min":0.311},"max":{"1min":0.45800000000000002,"5min":0.98099999999999998,"15min":0.98099999999999998},"last":0.47399999999999998}]},{"osd":4,"last update":"Fri Nov 24 07:58:22 2023","interfaces":[{"interface":"back","average":{"1min":0.57099999999999995,"5min":0.751,"15min":0.78100000000000003},"min":{"1min":0.42099999999999999,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.71999999999999997,"5min":5.7350000000000003,"15min":5.7350000000000003},"last":0.55500000000000005},{"interface":"front","average":{"1min":0.59299999999999997,"5min":0.58599999999999997,"15min":0.58499999999999996},"min":{"1min":0.37,"5min":0.37,"15min":0.37},"max":{"1min":0.77000000000000002,"5min":1.27,"15min":1.27},"last":0.58599999999999997}]},{"osd":5,"last update":"Fri Nov 24 07:58:22 2023","interfaces":[{"interface":"back","average":{"1min":0.56899999999999995,"5min":0.58399999999999996,"15min":0.58599999999999997},"min":{"1min":0.40799999999999997,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":0.82799999999999996,"5min":1.4430000000000001,"15min":1.4430000000000001},"last":0.65900000000000003},{"interface":"front","average":{"1min":0.60699999999999998,"5min":0.623,"15min":0.625},"min":{"1min":0.443,"5min":0.377,"15min":0.377},"max":{"1min":0.79900000000000004,"5min":1.4239999999999999,"15min":1.4239999999999999},"last":0.64000000000000001}]},{"osd":6,"last update":"Fri Nov 24 07:58:22 2023","interfaces":[{"interface":"back","average":{"1min":0.58999999999999997,"5min":0.61399999999999999,"15min":0.61799999999999999},"min":{"1min":0.46200000000000002,"5min":0.434,"15min":0.434},"max":{"1min":0.82599999999999996,"5min":1.355,"15min":1.355},"last":0.60699999999999998},{"interface":"front","average":{"1min":0.61199999999999999,"5min":0.79900000000000004,"15min":0.82999999999999996},"min":{"1min":0.47499999999999998,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.76900000000000002,"5min":5.6479999999999997,"15min":5.6479999999999997},"last":0.62}]},{"osd":7,"last update":"Fri Nov 24 07:58:22 2023","interfaces":[{"interface":"back","average":{"1min":0.59199999999999997,"5min":0.65100000000000002,"15min":0.66100000000000003},"min":{"1min":0.48099999999999998,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":0.79600000000000004,"5min":1.51,"15min":1.51},"last":0.67900000000000005},{"interface":"front","average":{"1min":0.61099999999999999,"5min":0.79500000000000004,"15min":0.82599999999999996},"min":{"1min":0.434,"5min":0.434,"15min":0.434},"max":{"1min":0.75900000000000001,"5min":5.7069999999999999,"15min":5.7069999999999999},"last":0.496}]}]},{"osd":4,"up_from":13,"seq":55834574887,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27868,"kb_used_data":844,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343972,"statfs":{"total":96636764160,"available":96608227328,"internally_reserved":0,"allocated":864256,"data_stored":364790,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 07:58:25 2023","interfaces":[{"interface":"back","average":{"1min":0.504,"5min":0.56499999999999995,"15min":0.57499999999999996},"min":{"1min":0.34899999999999998,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.70799999999999996,"5min":0.79300000000000004,"15min":0.79300000000000004},"last":0.70399999999999996},{"interface":"front","average":{"1min":0.502,"5min":0.53400000000000003,"15min":0.54000000000000004},"min":{"1min":0.33600000000000002,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.70199999999999996,"5min":1.0249999999999999,"15min":1.0249999999999999},"last":0.69299999999999995}]},{"osd":1,"last update":"Fri Nov 24 07:58:25 2023","interfaces":[{"interface":"back","average":{"1min":0.52800000000000002,"5min":0.54700000000000004,"15min":0.55000000000000004},"min":{"1min":0.39500000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.73999999999999999,"5min":0.86399999999999999,"15min":0.86399999999999999},"last":0.65700000000000003},{"interface":"front","average":{"1min":0.51900000000000002,"5min":0.54400000000000004,"15min":0.54800000000000004},"min":{"1min":0.374,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.72199999999999998,"5min":0.98699999999999999,"15min":0.98699999999999999},"last":0.60599999999999998}]},{"osd":2,"last update":"Fri Nov 24 07:58:25 2023","interfaces":[{"interface":"back","average":{"1min":0.52000000000000002,"5min":0.55200000000000005,"15min":0.55800000000000005},"min":{"1min":0.38800000000000001,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.66400000000000003,"5min":0.81100000000000005,"15min":0.81100000000000005},"last":0.63700000000000001},{"interface":"front","average":{"1min":0.56899999999999995,"5min":0.60399999999999998,"15min":0.60999999999999999},"min":{"1min":0.40000000000000002,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.75700000000000001,"5min":0.89200000000000002,"15min":0.89200000000000002},"last":0.53600000000000003}]},{"osd":3,"last update":"Fri Nov 24 07:58:25 2023","interfaces":[{"interface":"back","average":{"1min":0.54900000000000004,"5min":0.57699999999999996,"15min":0.58199999999999996},"min":{"1min":0.42899999999999999,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":0.65800000000000003,"5min":0.89700000000000002,"15min":0.89700000000000002},"last":0.52000000000000002},{"interface":"front","average":{"1min":0.57499999999999996,"5min":0.61899999999999999,"15min":0.627},"min":{"1min":0.442,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.72899999999999998,"5min":0.96199999999999997,"15min":0.96199999999999997},"last":0.67400000000000004}]},{"osd":5,"last update":"Fri Nov 24 07:58:25 2023","interfaces":[{"interface":"back","average":{"1min":0.48599999999999999,"5min":0.46100000000000002,"15min":0.45700000000000002},"min":{"1min":0.29499999999999998,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":0.67600000000000005,"5min":0.70499999999999996,"15min":0.70499999999999996},"last":0.35899999999999999},{"interface":"front","average":{"1min":0.59899999999999998,"5min":0.61499999999999999,"15min":0.61699999999999999},"min":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.746,"5min":0.86399999999999999,"15min":0.86399999999999999},"last":0.42899999999999999}]},{"osd":6,"last update":"Fri Nov 24 07:58:25 2023","interfaces":[{"interface":"back","average":{"1min":0.56799999999999995,"5min":0.58199999999999996,"15min":0.58399999999999996},"min":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.69799999999999995,"5min":0.76800000000000002,"15min":0.76800000000000002},"last":0.33600000000000002},{"interface":"front","average":{"1min":0.58999999999999997,"5min":0.60899999999999999,"15min":0.61199999999999999},"min":{"1min":0.46899999999999997,"5min":0.436,"15min":0.436},"max":{"1min":0.77700000000000002,"5min":1.095,"15min":1.095},"last":0.41199999999999998}]},{"osd":7,"last update":"Fri Nov 24 07:58:25 2023","interfaces":[{"interface":"back","average":{"1min":0.56799999999999995,"5min":0.57199999999999995,"15min":0.57199999999999995},"min":{"1min":0.39300000000000002,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.74199999999999999,"5min":0.89300000000000002,"15min":0.89300000000000002},"last":0.377},{"interface":"front","average":{"1min":0.61699999999999999,"5min":0.65300000000000002,"15min":0.65900000000000003},"min":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.79100000000000004,"5min":0.91500000000000004,"15min":0.91500000000000004},"last":0.39100000000000001}]}]},{"osd":5,"up_from":13,"seq":55834574888,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28448,"kb_used_data":1424,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343392,"statfs":{"total":96636764160,"available":96607633408,"internally_reserved":0,"allocated":1458176,"data_stored":955158,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 07:58:20 2023","interfaces":[{"interface":"back","average":{"1min":0.52300000000000002,"5min":0.55400000000000005,"15min":0.55900000000000005},"min":{"1min":0.35299999999999998,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.65800000000000003,"5min":1.0109999999999999,"15min":1.0109999999999999},"last":0.45500000000000002},{"interface":"front","average":{"1min":0.53200000000000003,"5min":0.746,"15min":0.78200000000000003},"min":{"1min":0.34399999999999997,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.72099999999999997,"5min":5.431,"15min":5.431},"last":0.34300000000000003}]},{"osd":1,"last update":"Fri Nov 24 07:58:20 2023","interfaces":[{"interface":"back","average":{"1min":0.51100000000000001,"5min":0.54900000000000004,"15min":0.55500000000000005},"min":{"1min":0.36399999999999999,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":0.67100000000000004,"5min":1.016,"15min":1.016},"last":0.41299999999999998},{"interface":"front","average":{"1min":0.54600000000000004,"5min":0.56499999999999995,"15min":0.56799999999999995},"min":{"1min":0.28599999999999998,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":0.78300000000000003,"5min":0.96499999999999997,"15min":0.96499999999999997},"last":0.47699999999999998}]},{"osd":2,"last update":"Fri Nov 24 07:58:20 2023","interfaces":[{"interface":"back","average":{"1min":0.54500000000000004,"5min":0.54900000000000004,"15min":0.55000000000000004},"min":{"1min":0.33700000000000002,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.67100000000000004,"5min":1.163,"15min":1.163},"last":0.46600000000000003},{"interface":"front","average":{"1min":0.54500000000000004,"5min":0.61299999999999999,"15min":0.624},"min":{"1min":0.41099999999999998,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.78900000000000003,"5min":1.119,"15min":1.119},"last":0.35899999999999999}]},{"osd":3,"last update":"Fri Nov 24 07:58:20 2023","interfaces":[{"interface":"back","average":{"1min":0.57199999999999995,"5min":0.83799999999999997,"15min":0.88200000000000001},"min":{"1min":0.36899999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.85899999999999999,"5min":5.609,"15min":5.609},"last":0.32400000000000001},{"interface":"front","average":{"1min":0.629,"5min":0.66600000000000004,"15min":0.67200000000000004},"min":{"1min":0.44600000000000001,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":0.76400000000000001,"5min":1.288,"15min":1.288},"last":0.39600000000000002}]},{"osd":4,"last update":"Fri Nov 24 07:58:20 2023","interfaces":[{"interface":"back","average":{"1min":0.51500000000000001,"5min":0.51100000000000001,"15min":0.51000000000000001},"min":{"1min":0.32100000000000001,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.76400000000000001,"5min":0.76400000000000001,"15min":0.76400000000000001},"last":0.30599999999999999},{"interface":"front","average":{"1min":0.58799999999999997,"5min":0.57499999999999996,"15min":0.57299999999999995},"min":{"1min":0.29599999999999999,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":0.92900000000000005,"5min":0.92900000000000005,"15min":0.92900000000000005},"last":0.378}]},{"osd":6,"last update":"Fri Nov 24 07:58:20 2023","interfaces":[{"interface":"back","average":{"1min":0.59599999999999997,"5min":0.57599999999999996,"15min":0.57199999999999995},"min":{"1min":0.35199999999999998,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.81799999999999995,"5min":0.89700000000000002,"15min":0.89700000000000002},"last":0.434},{"interface":"front","average":{"1min":0.63800000000000001,"5min":0.627,"15min":0.625},"min":{"1min":0.40300000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":1.083,"5min":1.083,"15min":1.083},"last":0.443}]},{"osd":7,"last update":"Fri Nov 24 07:58:20 2023","interfaces":[{"interface":"back","average":{"1min":0.66200000000000003,"5min":0.64900000000000002,"15min":0.64700000000000002},"min":{"1min":0.36799999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.90100000000000002,"5min":0.96599999999999997,"15min":0.96599999999999997},"last":0.42399999999999999},{"interface":"front","average":{"1min":0.66800000000000004,"5min":0.65600000000000003,"15min":0.65400000000000003},"min":{"1min":0.42699999999999999,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":0.85599999999999998,"5min":0.94199999999999995,"15min":0.94199999999999995},"last":0.49099999999999999}]}]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-11-24T07:59:18.789 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-11-24T07:59:18.949 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:18.947+0000 7f91fa1e5700 1 Processor -- start 2023-11-24T07:59:18.949 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:18.947+0000 7f91fa1e5700 1 -- start start 2023-11-24T07:59:18.951 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:18.948+0000 7f91fa1e5700 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f91f415f440 0x7f91f4072710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:18.951 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:18.948+0000 7f91fa1e5700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f91f40b3cb0 0x7f91f4072c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:18.951 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:18.948+0000 7f91fa1e5700 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f91f415f820 0x7f91f40ccaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:18.952 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:18.948+0000 7f91fa1e5700 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f91f40735e0 con 0x7f91f40b3cb0 2023-11-24T07:59:18.952 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:18.948+0000 7f91fa1e5700 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f91f406b470 con 0x7f91f415f820 2023-11-24T07:59:18.952 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:18.948+0000 7f91fa1e5700 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f91f4067370 con 0x7f91f415f440 2023-11-24T07:59:18.952 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:18.948+0000 7f91f2ffd700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f91f40b3cb0 0x7f91f4072c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:18.952 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:18.948+0000 7f91f2ffd700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f91f40b3cb0 0x7f91f4072c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:54874/0 (socket says 172.21.15.59:54874) 2023-11-24T07:59:18.952 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:18.948+0000 7f91f2ffd700 1 -- 172.21.15.59:0/2439062710 learned_addr learned my addr 172.21.15.59:0/2439062710 (peer_addr_for_me v2:172.21.15.59:0/0) 2023-11-24T07:59:18.952 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:18.948+0000 7f91f37fe700 1 --2- 172.21.15.59:0/2439062710 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f91f415f440 0x7f91f4072710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:18.952 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:18.948+0000 7f91f3fff700 1 --2- 172.21.15.59:0/2439062710 >> v2:172.21.15.76:3300/0 conn(0x7f91f415f820 0x7f91f40ccaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:18.952 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:18.948+0000 7f91f2ffd700 1 -- 172.21.15.59:0/2439062710 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f91f415f440 msgr2=0x7f91f4072710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T07:59:18.952 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:18.948+0000 7f91f2ffd700 1 --2- 172.21.15.59:0/2439062710 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f91f415f440 0x7f91f4072710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:18.952 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:18.948+0000 7f91f2ffd700 1 -- 172.21.15.59:0/2439062710 >> v2:172.21.15.76:3300/0 conn(0x7f91f415f820 msgr2=0x7f91f40ccaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T07:59:18.952 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:18.948+0000 7f91f2ffd700 1 --2- 172.21.15.59:0/2439062710 >> v2:172.21.15.76:3300/0 conn(0x7f91f415f820 0x7f91f40ccaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:18.953 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:18.948+0000 7f91f2ffd700 1 -- 172.21.15.59:0/2439062710 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f91f40ad050 con 0x7f91f40b3cb0 2023-11-24T07:59:18.953 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:18.948+0000 7f91f37fe700 1 --2- 172.21.15.59:0/2439062710 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f91f415f440 0x7f91f4072710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-24T07:59:18.953 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:18.948+0000 7f91f3fff700 1 --2- 172.21.15.59:0/2439062710 >> v2:172.21.15.76:3300/0 conn(0x7f91f415f820 0x7f91f40ccaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-24T07:59:18.953 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:18.949+0000 7f91f2ffd700 1 --2- 172.21.15.59:0/2439062710 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f91f40b3cb0 0x7f91f4072c50 secure :-1 s=READY pgs=870 cs=0 l=1 rev1=1 crypto rx=0x7f91f4070740 tx=0x7f91dc009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d6cec453663ee777 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:59:18.953 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:18.949+0000 7f91f27fc700 1 -- 172.21.15.59:0/2439062710 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91dc00b870 con 0x7f91f40b3cb0 2023-11-24T07:59:18.953 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:18.949+0000 7f91f27fc700 1 -- 172.21.15.59:0/2439062710 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f91dc00b9d0 con 0x7f91f40b3cb0 2023-11-24T07:59:18.953 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:18.949+0000 7f91f27fc700 1 -- 172.21.15.59:0/2439062710 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91dc00bd60 con 0x7f91f40b3cb0 2023-11-24T07:59:18.953 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:18.949+0000 7f91fa1e5700 1 -- 172.21.15.59:0/2439062710 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f91f40b3cb0 msgr2=0x7f91f4072c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:18.953 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:18.949+0000 7f91fa1e5700 1 --2- 172.21.15.59:0/2439062710 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f91f40b3cb0 0x7f91f4072c50 secure :-1 s=READY pgs=870 cs=0 l=1 rev1=1 crypto rx=0x7f91f4070740 tx=0x7f91dc009bd0 comp rx=0 tx=0).stop 2023-11-24T07:59:18.954 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:18.949+0000 7f91fa1e5700 1 -- 172.21.15.59:0/2439062710 shutdown_connections 2023-11-24T07:59:18.954 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:18.949+0000 7f91fa1e5700 1 --2- 172.21.15.59:0/2439062710 >> v2:172.21.15.76:3300/0 conn(0x7f91f415f820 0x7f91f40ccaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:18.954 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:18.949+0000 7f91fa1e5700 1 --2- 172.21.15.59:0/2439062710 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f91f415f440 0x7f91f4072710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:18.954 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:18.949+0000 7f91fa1e5700 1 --2- 172.21.15.59:0/2439062710 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f91f40b3cb0 0x7f91f4072c50 unknown :-1 s=CLOSED pgs=870 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:18.954 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:18.949+0000 7f91fa1e5700 1 -- 172.21.15.59:0/2439062710 >> 172.21.15.59:0/2439062710 conn(0x7f91f40b3190 msgr2=0x7f91f40b35a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:59:18.954 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:18.949+0000 7f91fa1e5700 1 -- 172.21.15.59:0/2439062710 shutdown_connections 2023-11-24T07:59:18.954 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:18.949+0000 7f91fa1e5700 1 -- 172.21.15.59:0/2439062710 wait complete. 2023-11-24T07:59:18.957 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:18.951+0000 7f91fa1e5700 1 Processor -- start 2023-11-24T07:59:18.957 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:18.951+0000 7f91fa1e5700 1 -- start start 2023-11-24T07:59:18.957 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:18.952+0000 7f91fa1e5700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f91f40b3cb0 0x7f91f40999b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:18.957 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:18.952+0000 7f91fa1e5700 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f91f415f440 0x7f91f409a160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:18.957 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:18.952+0000 7f91fa1e5700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f91f415f820 0x7f91f40cfdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:18.957 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:18.952+0000 7f91fa1e5700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91f4099ef0 con 0x7f91f40b3cb0 2023-11-24T07:59:18.957 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:18.952+0000 7f91fa1e5700 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91f40cea80 con 0x7f91f415f820 2023-11-24T07:59:18.957 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:18.952+0000 7f91fa1e5700 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f91f40ced20 con 0x7f91f415f440 2023-11-24T07:59:18.957 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:18.952+0000 7f91f37fe700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f91f40b3cb0 0x7f91f40999b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:18.957 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:18.952+0000 7f91f37fe700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f91f40b3cb0 0x7f91f40999b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:54880/0 (socket says 172.21.15.59:54880) 2023-11-24T07:59:18.957 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:18.952+0000 7f91f37fe700 1 -- 172.21.15.59:0/713459029 learned_addr learned my addr 172.21.15.59:0/713459029 (peer_addr_for_me v2:172.21.15.59:0/0) 2023-11-24T07:59:18.957 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:18.952+0000 7f91f3fff700 1 --2- 172.21.15.59:0/713459029 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f91f415f820 0x7f91f40cfdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:18.958 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:18.952+0000 7f91f2ffd700 1 --2- 172.21.15.59:0/713459029 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f91f415f440 0x7f91f409a160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:18.958 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:18.952+0000 7f91f37fe700 1 -- 172.21.15.59:0/713459029 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f91f415f440 msgr2=0x7f91f409a160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:18.958 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:18.952+0000 7f91f37fe700 1 --2- 172.21.15.59:0/713459029 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f91f415f440 0x7f91f409a160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:18.958 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:18.952+0000 7f91f37fe700 1 -- 172.21.15.59:0/713459029 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f91f415f820 msgr2=0x7f91f40cfdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:18.958 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:18.952+0000 7f91f37fe700 1 --2- 172.21.15.59:0/713459029 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f91f415f820 0x7f91f40cfdc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:18.958 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:18.952+0000 7f91f37fe700 1 -- 172.21.15.59:0/713459029 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f91dc009550 con 0x7f91f40b3cb0 2023-11-24T07:59:18.958 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:18.952+0000 7f91f3fff700 1 --2- 172.21.15.59:0/713459029 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f91f415f820 0x7f91f40cfdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T07:59:18.958 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:18.952+0000 7f91f37fe700 1 --2- 172.21.15.59:0/713459029 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f91f40b3cb0 0x7f91f40999b0 secure :-1 s=READY pgs=871 cs=0 l=1 rev1=1 crypto rx=0x7f91e400d800 tx=0x7f91e400dbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:59:18.958 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:18.952+0000 7f91ebfff700 1 -- 172.21.15.59:0/713459029 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91e4004030 con 0x7f91f40b3cb0 2023-11-24T07:59:18.958 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:18.952+0000 7f91ebfff700 1 -- 172.21.15.59:0/713459029 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f91e4004190 con 0x7f91f40b3cb0 2023-11-24T07:59:18.958 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:18.952+0000 7f91fa1e5700 1 -- 172.21.15.59:0/713459029 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f91f40cf000 con 0x7f91f40b3cb0 2023-11-24T07:59:18.958 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:18.952+0000 7f91ebfff700 1 -- 172.21.15.59:0/713459029 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91e4004520 con 0x7f91f40b3cb0 2023-11-24T07:59:18.959 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:18.952+0000 7f91fa1e5700 1 -- 172.21.15.59:0/713459029 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f91f40cf5c0 con 0x7f91f40b3cb0 2023-11-24T07:59:18.959 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:18.953+0000 7f91fa1e5700 1 -- 172.21.15.59:0/713459029 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f91f4074d40 con 0x7f91f40b3cb0 2023-11-24T07:59:18.959 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:18.954+0000 7f91ebfff700 1 -- 172.21.15.59:0/713459029 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 10) v1 ==== 89588+0+0 (secure 0 0 0) 0x7f91e40036a0 con 0x7f91f40b3cb0 2023-11-24T07:59:18.959 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:18.954+0000 7f91ebfff700 1 --2- 172.21.15.59:0/713459029 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f91d406b2c0 0x7f91d406d780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:18.959 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:18.954+0000 7f91ebfff700 1 -- 172.21.15.59:0/713459029 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f91e408a2e0 con 0x7f91f40b3cb0 2023-11-24T07:59:18.959 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:18.954+0000 7f91f2ffd700 1 --2- 172.21.15.59:0/713459029 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f91d406b2c0 0x7f91d406d780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:18.959 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:18.955+0000 7f91f2ffd700 1 --2- 172.21.15.59:0/713459029 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f91d406b2c0 0x7f91d406d780 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f91f40b40d0 tx=0x7f91dc008030 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:59:18.960 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:18.959+0000 7f91ebfff700 1 -- 172.21.15.59:0/713459029 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f91e4054db0 con 0x7f91f40b3cb0 2023-11-24T07:59:19.158 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:19.156+0000 7f91fa1e5700 1 -- 172.21.15.59:0/713459029 --> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f91f40afad0 con 0x7f91d406b2c0 2023-11-24T07:59:19.166 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:19.164+0000 7f91ebfff700 1 -- 172.21.15.59:0/713459029 <== mgr.4100 v2:172.21.15.76:6832/4264046536 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+277156 (secure 0 0 0) 0x7f91dc00bb40 con 0x7f91d406b2c0 2023-11-24T07:59:19.169 INFO:teuthology.orchestra.run.smithi059.stdout: 2023-11-24T07:59:19.171 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:19.168+0000 7f91fa1e5700 1 -- 172.21.15.59:0/713459029 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f91d406b2c0 msgr2=0x7f91d406d780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:19.171 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:19.168+0000 7f91fa1e5700 1 --2- 172.21.15.59:0/713459029 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f91d406b2c0 0x7f91d406d780 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f91f40b40d0 tx=0x7f91dc008030 comp rx=0 tx=0).stop 2023-11-24T07:59:19.171 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:19.168+0000 7f91fa1e5700 1 -- 172.21.15.59:0/713459029 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f91f40b3cb0 msgr2=0x7f91f40999b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:19.171 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:19.168+0000 7f91fa1e5700 1 --2- 172.21.15.59:0/713459029 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f91f40b3cb0 0x7f91f40999b0 secure :-1 s=READY pgs=871 cs=0 l=1 rev1=1 crypto rx=0x7f91e400d800 tx=0x7f91e400dbc0 comp rx=0 tx=0).stop 2023-11-24T07:59:19.172 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:19.168+0000 7f91fa1e5700 1 -- 172.21.15.59:0/713459029 shutdown_connections 2023-11-24T07:59:19.172 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:19.168+0000 7f91fa1e5700 1 --2- 172.21.15.59:0/713459029 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f91d406b2c0 0x7f91d406d780 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:19.172 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:19.168+0000 7f91fa1e5700 1 --2- 172.21.15.59:0/713459029 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f91f415f820 0x7f91f40cfdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:19.172 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:19.168+0000 7f91fa1e5700 1 --2- 172.21.15.59:0/713459029 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f91f40b3cb0 0x7f91f40999b0 unknown :-1 s=CLOSED pgs=871 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:19.172 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:19.168+0000 7f91fa1e5700 1 --2- 172.21.15.59:0/713459029 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f91f415f440 0x7f91f409a160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:19.172 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:19.168+0000 7f91fa1e5700 1 -- 172.21.15.59:0/713459029 >> 172.21.15.59:0/713459029 conn(0x7f91f40b3190 msgr2=0x7f91f415e220 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:59:19.172 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:19.168+0000 7f91fa1e5700 1 -- 172.21.15.59:0/713459029 shutdown_connections 2023-11-24T07:59:19.172 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:19.168+0000 7f91fa1e5700 1 -- 172.21.15.59:0/713459029 wait complete. 2023-11-24T07:59:19.172 INFO:teuthology.orchestra.run.smithi059.stderr:dumped all 2023-11-24T07:59:19.182 INFO:teuthology.orchestra.run.smithi059.stdout:{"pg_ready":true,"pg_map":{"version":157,"stamp":"2023-11-24T07:59:18.339299+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":66,"acting":66,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":5,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":7,"kb":754974720,"kb_used":224084,"kb_used_data":7888,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754750636,"statfs":{"total":773094113280,"available":772864651264,"internally_reserved":0,"allocated":8077312,"data_stored":4087422,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12745,"internal_metadata":221236791},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":15,"apply_latency_ms":15,"commit_latency_ns":15000000,"apply_latency_ns":15000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.718179"},"pg_stats":[{"pgid":"11.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T07:59:13.620687+0000","last_change":"2023-11-24T07:59:13.620687+0000","last_active":"2023-11-24T07:59:13.620687+0000","last_peered":"2023-11-24T07:59:13.620687+0000","last_clean":"2023-11-24T07:59:13.620687+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T07:59:13.620687+0000","last_undegraded":"2023-11-24T07:59:13.620687+0000","last_fullsized":"2023-11-24T07:59:13.620687+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:59:13.620687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:59:13.620687+0000","last_clean_scrub_stamp":"2023-11-24T07:59:13.620687+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T07:59:13.620687+0000","last_change":"2023-11-24T07:59:13.620687+0000","last_active":"2023-11-24T07:59:13.620687+0000","last_peered":"2023-11-24T07:59:13.620687+0000","last_clean":"2023-11-24T07:59:13.620687+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T07:59:13.620687+0000","last_undegraded":"2023-11-24T07:59:13.620687+0000","last_fullsized":"2023-11-24T07:59:13.620687+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:59:13.620687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:59:13.620687+0000","last_clean_scrub_stamp":"2023-11-24T07:59:13.620687+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T07:59:13.620687+0000","last_change":"2023-11-24T07:59:13.620687+0000","last_active":"2023-11-24T07:59:13.620687+0000","last_peered":"2023-11-24T07:59:13.620687+0000","last_clean":"2023-11-24T07:59:13.620687+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T07:59:13.620687+0000","last_undegraded":"2023-11-24T07:59:13.620687+0000","last_fullsized":"2023-11-24T07:59:13.620687+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:59:13.620687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:59:13.620687+0000","last_clean_scrub_stamp":"2023-11-24T07:59:13.620687+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T07:59:13.620687+0000","last_change":"2023-11-24T07:59:13.620687+0000","last_active":"2023-11-24T07:59:13.620687+0000","last_peered":"2023-11-24T07:59:13.620687+0000","last_clean":"2023-11-24T07:59:13.620687+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T07:59:13.620687+0000","last_undegraded":"2023-11-24T07:59:13.620687+0000","last_fullsized":"2023-11-24T07:59:13.620687+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:59:13.620687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:59:13.620687+0000","last_clean_scrub_stamp":"2023-11-24T07:59:13.620687+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T07:59:13.620687+0000","last_change":"2023-11-24T07:59:13.620687+0000","last_active":"2023-11-24T07:59:13.620687+0000","last_peered":"2023-11-24T07:59:13.620687+0000","last_clean":"2023-11-24T07:59:13.620687+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T07:59:13.620687+0000","last_undegraded":"2023-11-24T07:59:13.620687+0000","last_fullsized":"2023-11-24T07:59:13.620687+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:59:13.620687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:59:13.620687+0000","last_clean_scrub_stamp":"2023-11-24T07:59:13.620687+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T07:59:13.620687+0000","last_change":"2023-11-24T07:59:13.620687+0000","last_active":"2023-11-24T07:59:13.620687+0000","last_peered":"2023-11-24T07:59:13.620687+0000","last_clean":"2023-11-24T07:59:13.620687+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T07:59:13.620687+0000","last_undegraded":"2023-11-24T07:59:13.620687+0000","last_fullsized":"2023-11-24T07:59:13.620687+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:59:13.620687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:59:13.620687+0000","last_clean_scrub_stamp":"2023-11-24T07:59:13.620687+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T07:59:13.620687+0000","last_change":"2023-11-24T07:59:13.620687+0000","last_active":"2023-11-24T07:59:13.620687+0000","last_peered":"2023-11-24T07:59:13.620687+0000","last_clean":"2023-11-24T07:59:13.620687+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T07:59:13.620687+0000","last_undegraded":"2023-11-24T07:59:13.620687+0000","last_fullsized":"2023-11-24T07:59:13.620687+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:59:13.620687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:59:13.620687+0000","last_clean_scrub_stamp":"2023-11-24T07:59:13.620687+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T07:59:13.620687+0000","last_change":"2023-11-24T07:59:13.620687+0000","last_active":"2023-11-24T07:59:13.620687+0000","last_peered":"2023-11-24T07:59:13.620687+0000","last_clean":"2023-11-24T07:59:13.620687+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T07:59:13.620687+0000","last_undegraded":"2023-11-24T07:59:13.620687+0000","last_fullsized":"2023-11-24T07:59:13.620687+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:59:13.620687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:59:13.620687+0000","last_clean_scrub_stamp":"2023-11-24T07:59:13.620687+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T07:59:13.620687+0000","last_change":"2023-11-24T07:59:13.620687+0000","last_active":"2023-11-24T07:59:13.620687+0000","last_peered":"2023-11-24T07:59:13.620687+0000","last_clean":"2023-11-24T07:59:13.620687+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T07:59:13.620687+0000","last_undegraded":"2023-11-24T07:59:13.620687+0000","last_fullsized":"2023-11-24T07:59:13.620687+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:59:13.620687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:59:13.620687+0000","last_clean_scrub_stamp":"2023-11-24T07:59:13.620687+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T07:59:13.620687+0000","last_change":"2023-11-24T07:59:13.620687+0000","last_active":"2023-11-24T07:59:13.620687+0000","last_peered":"2023-11-24T07:59:13.620687+0000","last_clean":"2023-11-24T07:59:13.620687+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T07:59:13.620687+0000","last_undegraded":"2023-11-24T07:59:13.620687+0000","last_fullsized":"2023-11-24T07:59:13.620687+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:59:13.620687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:59:13.620687+0000","last_clean_scrub_stamp":"2023-11-24T07:59:13.620687+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T07:59:13.620687+0000","last_change":"2023-11-24T07:59:13.620687+0000","last_active":"2023-11-24T07:59:13.620687+0000","last_peered":"2023-11-24T07:59:13.620687+0000","last_clean":"2023-11-24T07:59:13.620687+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T07:59:13.620687+0000","last_undegraded":"2023-11-24T07:59:13.620687+0000","last_fullsized":"2023-11-24T07:59:13.620687+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:59:13.620687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:59:13.620687+0000","last_clean_scrub_stamp":"2023-11-24T07:59:13.620687+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T07:59:13.620687+0000","last_change":"2023-11-24T07:59:13.620687+0000","last_active":"2023-11-24T07:59:13.620687+0000","last_peered":"2023-11-24T07:59:13.620687+0000","last_clean":"2023-11-24T07:59:13.620687+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T07:59:13.620687+0000","last_undegraded":"2023-11-24T07:59:13.620687+0000","last_fullsized":"2023-11-24T07:59:13.620687+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:59:13.620687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:59:13.620687+0000","last_clean_scrub_stamp":"2023-11-24T07:59:13.620687+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T07:59:13.620687+0000","last_change":"2023-11-24T07:59:13.620687+0000","last_active":"2023-11-24T07:59:13.620687+0000","last_peered":"2023-11-24T07:59:13.620687+0000","last_clean":"2023-11-24T07:59:13.620687+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T07:59:13.620687+0000","last_undegraded":"2023-11-24T07:59:13.620687+0000","last_fullsized":"2023-11-24T07:59:13.620687+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:59:13.620687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:59:13.620687+0000","last_clean_scrub_stamp":"2023-11-24T07:59:13.620687+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T07:59:13.620687+0000","last_change":"2023-11-24T07:59:13.620687+0000","last_active":"2023-11-24T07:59:13.620687+0000","last_peered":"2023-11-24T07:59:13.620687+0000","last_clean":"2023-11-24T07:59:13.620687+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T07:59:13.620687+0000","last_undegraded":"2023-11-24T07:59:13.620687+0000","last_fullsized":"2023-11-24T07:59:13.620687+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:59:13.620687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:59:13.620687+0000","last_clean_scrub_stamp":"2023-11-24T07:59:13.620687+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T07:59:13.620687+0000","last_change":"2023-11-24T07:59:13.620687+0000","last_active":"2023-11-24T07:59:13.620687+0000","last_peered":"2023-11-24T07:59:13.620687+0000","last_clean":"2023-11-24T07:59:13.620687+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T07:59:13.620687+0000","last_undegraded":"2023-11-24T07:59:13.620687+0000","last_fullsized":"2023-11-24T07:59:13.620687+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:59:13.620687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:59:13.620687+0000","last_clean_scrub_stamp":"2023-11-24T07:59:13.620687+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T07:59:13.620687+0000","last_change":"2023-11-24T07:59:13.620687+0000","last_active":"2023-11-24T07:59:13.620687+0000","last_peered":"2023-11-24T07:59:13.620687+0000","last_clean":"2023-11-24T07:59:13.620687+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T07:59:13.620687+0000","last_undegraded":"2023-11-24T07:59:13.620687+0000","last_fullsized":"2023-11-24T07:59:13.620687+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:59:13.620687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:59:13.620687+0000","last_clean_scrub_stamp":"2023-11-24T07:59:13.620687+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T07:59:13.620687+0000","last_change":"2023-11-24T07:59:13.620687+0000","last_active":"2023-11-24T07:59:13.620687+0000","last_peered":"2023-11-24T07:59:13.620687+0000","last_clean":"2023-11-24T07:59:13.620687+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T07:59:13.620687+0000","last_undegraded":"2023-11-24T07:59:13.620687+0000","last_fullsized":"2023-11-24T07:59:13.620687+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:59:13.620687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:59:13.620687+0000","last_clean_scrub_stamp":"2023-11-24T07:59:13.620687+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T07:59:13.620687+0000","last_change":"2023-11-24T07:59:13.620687+0000","last_active":"2023-11-24T07:59:13.620687+0000","last_peered":"2023-11-24T07:59:13.620687+0000","last_clean":"2023-11-24T07:59:13.620687+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T07:59:13.620687+0000","last_undegraded":"2023-11-24T07:59:13.620687+0000","last_fullsized":"2023-11-24T07:59:13.620687+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:59:13.620687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:59:13.620687+0000","last_clean_scrub_stamp":"2023-11-24T07:59:13.620687+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T07:59:13.620687+0000","last_change":"2023-11-24T07:59:13.620687+0000","last_active":"2023-11-24T07:59:13.620687+0000","last_peered":"2023-11-24T07:59:13.620687+0000","last_clean":"2023-11-24T07:59:13.620687+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T07:59:13.620687+0000","last_undegraded":"2023-11-24T07:59:13.620687+0000","last_fullsized":"2023-11-24T07:59:13.620687+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:59:13.620687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:59:13.620687+0000","last_clean_scrub_stamp":"2023-11-24T07:59:13.620687+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T07:59:13.620687+0000","last_change":"2023-11-24T07:59:13.620687+0000","last_active":"2023-11-24T07:59:13.620687+0000","last_peered":"2023-11-24T07:59:13.620687+0000","last_clean":"2023-11-24T07:59:13.620687+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T07:59:13.620687+0000","last_undegraded":"2023-11-24T07:59:13.620687+0000","last_fullsized":"2023-11-24T07:59:13.620687+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:59:13.620687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:59:13.620687+0000","last_clean_scrub_stamp":"2023-11-24T07:59:13.620687+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T07:59:13.620687+0000","last_change":"2023-11-24T07:59:13.620687+0000","last_active":"2023-11-24T07:59:13.620687+0000","last_peered":"2023-11-24T07:59:13.620687+0000","last_clean":"2023-11-24T07:59:13.620687+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T07:59:13.620687+0000","last_undegraded":"2023-11-24T07:59:13.620687+0000","last_fullsized":"2023-11-24T07:59:13.620687+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:59:13.620687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:59:13.620687+0000","last_clean_scrub_stamp":"2023-11-24T07:59:13.620687+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T07:59:13.620687+0000","last_change":"2023-11-24T07:59:13.620687+0000","last_active":"2023-11-24T07:59:13.620687+0000","last_peered":"2023-11-24T07:59:13.620687+0000","last_clean":"2023-11-24T07:59:13.620687+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T07:59:13.620687+0000","last_undegraded":"2023-11-24T07:59:13.620687+0000","last_fullsized":"2023-11-24T07:59:13.620687+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:59:13.620687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:59:13.620687+0000","last_clean_scrub_stamp":"2023-11-24T07:59:13.620687+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T07:59:13.620687+0000","last_change":"2023-11-24T07:59:13.620687+0000","last_active":"2023-11-24T07:59:13.620687+0000","last_peered":"2023-11-24T07:59:13.620687+0000","last_clean":"2023-11-24T07:59:13.620687+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T07:59:13.620687+0000","last_undegraded":"2023-11-24T07:59:13.620687+0000","last_fullsized":"2023-11-24T07:59:13.620687+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:59:13.620687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:59:13.620687+0000","last_clean_scrub_stamp":"2023-11-24T07:59:13.620687+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T07:59:13.620687+0000","last_change":"2023-11-24T07:59:13.620687+0000","last_active":"2023-11-24T07:59:13.620687+0000","last_peered":"2023-11-24T07:59:13.620687+0000","last_clean":"2023-11-24T07:59:13.620687+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T07:59:13.620687+0000","last_undegraded":"2023-11-24T07:59:13.620687+0000","last_fullsized":"2023-11-24T07:59:13.620687+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:59:13.620687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:59:13.620687+0000","last_clean_scrub_stamp":"2023-11-24T07:59:13.620687+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T07:59:13.620687+0000","last_change":"2023-11-24T07:59:13.620687+0000","last_active":"2023-11-24T07:59:13.620687+0000","last_peered":"2023-11-24T07:59:13.620687+0000","last_clean":"2023-11-24T07:59:13.620687+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T07:59:13.620687+0000","last_undegraded":"2023-11-24T07:59:13.620687+0000","last_fullsized":"2023-11-24T07:59:13.620687+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:59:13.620687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:59:13.620687+0000","last_clean_scrub_stamp":"2023-11-24T07:59:13.620687+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T07:59:13.620687+0000","last_change":"2023-11-24T07:59:13.620687+0000","last_active":"2023-11-24T07:59:13.620687+0000","last_peered":"2023-11-24T07:59:13.620687+0000","last_clean":"2023-11-24T07:59:13.620687+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T07:59:13.620687+0000","last_undegraded":"2023-11-24T07:59:13.620687+0000","last_fullsized":"2023-11-24T07:59:13.620687+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:59:13.620687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:59:13.620687+0000","last_clean_scrub_stamp":"2023-11-24T07:59:13.620687+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T07:59:13.620687+0000","last_change":"2023-11-24T07:59:13.620687+0000","last_active":"2023-11-24T07:59:13.620687+0000","last_peered":"2023-11-24T07:59:13.620687+0000","last_clean":"2023-11-24T07:59:13.620687+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T07:59:13.620687+0000","last_undegraded":"2023-11-24T07:59:13.620687+0000","last_fullsized":"2023-11-24T07:59:13.620687+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:59:13.620687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:59:13.620687+0000","last_clean_scrub_stamp":"2023-11-24T07:59:13.620687+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T07:59:13.620687+0000","last_change":"2023-11-24T07:59:13.620687+0000","last_active":"2023-11-24T07:59:13.620687+0000","last_peered":"2023-11-24T07:59:13.620687+0000","last_clean":"2023-11-24T07:59:13.620687+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T07:59:13.620687+0000","last_undegraded":"2023-11-24T07:59:13.620687+0000","last_fullsized":"2023-11-24T07:59:13.620687+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:59:13.620687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:59:13.620687+0000","last_clean_scrub_stamp":"2023-11-24T07:59:13.620687+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T07:59:13.620687+0000","last_change":"2023-11-24T07:59:13.620687+0000","last_active":"2023-11-24T07:59:13.620687+0000","last_peered":"2023-11-24T07:59:13.620687+0000","last_clean":"2023-11-24T07:59:13.620687+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T07:59:13.620687+0000","last_undegraded":"2023-11-24T07:59:13.620687+0000","last_fullsized":"2023-11-24T07:59:13.620687+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:59:13.620687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:59:13.620687+0000","last_clean_scrub_stamp":"2023-11-24T07:59:13.620687+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T07:59:13.620687+0000","last_change":"2023-11-24T07:59:13.620687+0000","last_active":"2023-11-24T07:59:13.620687+0000","last_peered":"2023-11-24T07:59:13.620687+0000","last_clean":"2023-11-24T07:59:13.620687+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T07:59:13.620687+0000","last_undegraded":"2023-11-24T07:59:13.620687+0000","last_fullsized":"2023-11-24T07:59:13.620687+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:59:13.620687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:59:13.620687+0000","last_clean_scrub_stamp":"2023-11-24T07:59:13.620687+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T07:59:13.620687+0000","last_change":"2023-11-24T07:59:13.620687+0000","last_active":"2023-11-24T07:59:13.620687+0000","last_peered":"2023-11-24T07:59:13.620687+0000","last_clean":"2023-11-24T07:59:13.620687+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T07:59:13.620687+0000","last_undegraded":"2023-11-24T07:59:13.620687+0000","last_fullsized":"2023-11-24T07:59:13.620687+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:59:13.620687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:59:13.620687+0000","last_clean_scrub_stamp":"2023-11-24T07:59:13.620687+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T07:59:13.620687+0000","last_change":"2023-11-24T07:59:13.620687+0000","last_active":"2023-11-24T07:59:13.620687+0000","last_peered":"2023-11-24T07:59:13.620687+0000","last_clean":"2023-11-24T07:59:13.620687+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T07:59:13.620687+0000","last_undegraded":"2023-11-24T07:59:13.620687+0000","last_fullsized":"2023-11-24T07:59:13.620687+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:59:13.620687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:59:13.620687+0000","last_clean_scrub_stamp":"2023-11-24T07:59:13.620687+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T07:59:13.620687+0000","last_change":"2023-11-24T07:59:13.620687+0000","last_active":"2023-11-24T07:59:13.620687+0000","last_peered":"2023-11-24T07:59:13.620687+0000","last_clean":"2023-11-24T07:59:13.620687+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T07:59:13.620687+0000","last_undegraded":"2023-11-24T07:59:13.620687+0000","last_fullsized":"2023-11-24T07:59:13.620687+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:59:13.620687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:59:13.620687+0000","last_clean_scrub_stamp":"2023-11-24T07:59:13.620687+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.1f","version":"0'0","reported_seq":1,"reported_epoch":71,"state":"creating+peering","last_fresh":"2023-11-24T07:59:12.627449+0000","last_change":"2023-11-24T07:59:12.627449+0000","last_active":"2023-11-24T07:59:12.615544+0000","last_peered":"2023-11-24T07:59:12.615544+0000","last_clean":"2023-11-24T07:59:12.615544+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T07:59:12.627449+0000","last_undegraded":"2023-11-24T07:59:12.627449+0000","last_fullsized":"2023-11-24T07:59:12.627449+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:59:12.615544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:59:12.615544+0000","last_clean_scrub_stamp":"2023-11-24T07:59:12.615544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T07:59:13.620687+0000","last_change":"2023-11-24T07:59:13.620687+0000","last_active":"2023-11-24T07:59:13.620687+0000","last_peered":"2023-11-24T07:59:13.620687+0000","last_clean":"2023-11-24T07:59:13.620687+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T07:59:13.620687+0000","last_undegraded":"2023-11-24T07:59:13.620687+0000","last_fullsized":"2023-11-24T07:59:13.620687+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:59:13.620687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:59:13.620687+0000","last_clean_scrub_stamp":"2023-11-24T07:59:13.620687+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.1e","version":"0'0","reported_seq":1,"reported_epoch":71,"state":"creating+peering","last_fresh":"2023-11-24T07:59:12.622955+0000","last_change":"2023-11-24T07:59:12.622955+0000","last_active":"2023-11-24T07:59:12.615544+0000","last_peered":"2023-11-24T07:59:12.615544+0000","last_clean":"2023-11-24T07:59:12.615544+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T07:59:12.622955+0000","last_undegraded":"2023-11-24T07:59:12.622955+0000","last_fullsized":"2023-11-24T07:59:12.622955+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:59:12.615544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:59:12.615544+0000","last_clean_scrub_stamp":"2023-11-24T07:59:12.615544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T07:59:13.620687+0000","last_change":"2023-11-24T07:59:13.620687+0000","last_active":"2023-11-24T07:59:13.620687+0000","last_peered":"2023-11-24T07:59:13.620687+0000","last_clean":"2023-11-24T07:59:13.620687+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T07:59:13.620687+0000","last_undegraded":"2023-11-24T07:59:13.620687+0000","last_fullsized":"2023-11-24T07:59:13.620687+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:59:13.620687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:59:13.620687+0000","last_clean_scrub_stamp":"2023-11-24T07:59:13.620687+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.1d","version":"0'0","reported_seq":1,"reported_epoch":71,"state":"creating+peering","last_fresh":"2023-11-24T07:59:12.632766+0000","last_change":"2023-11-24T07:59:12.632766+0000","last_active":"2023-11-24T07:59:12.615544+0000","last_peered":"2023-11-24T07:59:12.615544+0000","last_clean":"2023-11-24T07:59:12.615544+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T07:59:12.632766+0000","last_undegraded":"2023-11-24T07:59:12.632766+0000","last_fullsized":"2023-11-24T07:59:12.632766+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:59:12.615544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:59:12.615544+0000","last_clean_scrub_stamp":"2023-11-24T07:59:12.615544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T07:59:13.620687+0000","last_change":"2023-11-24T07:59:13.620687+0000","last_active":"2023-11-24T07:59:13.620687+0000","last_peered":"2023-11-24T07:59:13.620687+0000","last_clean":"2023-11-24T07:59:13.620687+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T07:59:13.620687+0000","last_undegraded":"2023-11-24T07:59:13.620687+0000","last_fullsized":"2023-11-24T07:59:13.620687+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:59:13.620687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:59:13.620687+0000","last_clean_scrub_stamp":"2023-11-24T07:59:13.620687+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.1c","version":"0'0","reported_seq":1,"reported_epoch":71,"state":"creating+peering","last_fresh":"2023-11-24T07:59:12.627772+0000","last_change":"2023-11-24T07:59:12.627772+0000","last_active":"2023-11-24T07:59:12.615544+0000","last_peered":"2023-11-24T07:59:12.615544+0000","last_clean":"2023-11-24T07:59:12.615544+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T07:59:12.627772+0000","last_undegraded":"2023-11-24T07:59:12.627772+0000","last_fullsized":"2023-11-24T07:59:12.627772+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:59:12.615544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:59:12.615544+0000","last_clean_scrub_stamp":"2023-11-24T07:59:12.615544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T07:59:13.620687+0000","last_change":"2023-11-24T07:59:13.620687+0000","last_active":"2023-11-24T07:59:13.620687+0000","last_peered":"2023-11-24T07:59:13.620687+0000","last_clean":"2023-11-24T07:59:13.620687+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T07:59:13.620687+0000","last_undegraded":"2023-11-24T07:59:13.620687+0000","last_fullsized":"2023-11-24T07:59:13.620687+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:59:13.620687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:59:13.620687+0000","last_clean_scrub_stamp":"2023-11-24T07:59:13.620687+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.1b","version":"0'0","reported_seq":1,"reported_epoch":71,"state":"creating+peering","last_fresh":"2023-11-24T07:59:12.626522+0000","last_change":"2023-11-24T07:59:12.626522+0000","last_active":"2023-11-24T07:59:12.615544+0000","last_peered":"2023-11-24T07:59:12.615544+0000","last_clean":"2023-11-24T07:59:12.615544+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T07:59:12.626522+0000","last_undegraded":"2023-11-24T07:59:12.626522+0000","last_fullsized":"2023-11-24T07:59:12.626522+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:59:12.615544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:59:12.615544+0000","last_clean_scrub_stamp":"2023-11-24T07:59:12.615544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T07:59:13.620687+0000","last_change":"2023-11-24T07:59:13.620687+0000","last_active":"2023-11-24T07:59:13.620687+0000","last_peered":"2023-11-24T07:59:13.620687+0000","last_clean":"2023-11-24T07:59:13.620687+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T07:59:13.620687+0000","last_undegraded":"2023-11-24T07:59:13.620687+0000","last_fullsized":"2023-11-24T07:59:13.620687+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:59:13.620687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:59:13.620687+0000","last_clean_scrub_stamp":"2023-11-24T07:59:13.620687+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.1a","version":"0'0","reported_seq":1,"reported_epoch":71,"state":"creating+peering","last_fresh":"2023-11-24T07:59:12.624736+0000","last_change":"2023-11-24T07:59:12.624736+0000","last_active":"2023-11-24T07:59:12.615544+0000","last_peered":"2023-11-24T07:59:12.615544+0000","last_clean":"2023-11-24T07:59:12.615544+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T07:59:12.624736+0000","last_undegraded":"2023-11-24T07:59:12.624736+0000","last_fullsized":"2023-11-24T07:59:12.624736+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:59:12.615544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:59:12.615544+0000","last_clean_scrub_stamp":"2023-11-24T07:59:12.615544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T07:59:13.620687+0000","last_change":"2023-11-24T07:59:13.620687+0000","last_active":"2023-11-24T07:59:13.620687+0000","last_peered":"2023-11-24T07:59:13.620687+0000","last_clean":"2023-11-24T07:59:13.620687+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T07:59:13.620687+0000","last_undegraded":"2023-11-24T07:59:13.620687+0000","last_fullsized":"2023-11-24T07:59:13.620687+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:59:13.620687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:59:13.620687+0000","last_clean_scrub_stamp":"2023-11-24T07:59:13.620687+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.19","version":"0'0","reported_seq":1,"reported_epoch":71,"state":"creating+peering","last_fresh":"2023-11-24T07:59:12.629097+0000","last_change":"2023-11-24T07:59:12.629097+0000","last_active":"2023-11-24T07:59:12.615544+0000","last_peered":"2023-11-24T07:59:12.615544+0000","last_clean":"2023-11-24T07:59:12.615544+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T07:59:12.629097+0000","last_undegraded":"2023-11-24T07:59:12.629097+0000","last_fullsized":"2023-11-24T07:59:12.629097+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:59:12.615544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:59:12.615544+0000","last_clean_scrub_stamp":"2023-11-24T07:59:12.615544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T07:59:13.620687+0000","last_change":"2023-11-24T07:59:13.620687+0000","last_active":"2023-11-24T07:59:13.620687+0000","last_peered":"2023-11-24T07:59:13.620687+0000","last_clean":"2023-11-24T07:59:13.620687+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T07:59:13.620687+0000","last_undegraded":"2023-11-24T07:59:13.620687+0000","last_fullsized":"2023-11-24T07:59:13.620687+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:59:13.620687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:59:13.620687+0000","last_clean_scrub_stamp":"2023-11-24T07:59:13.620687+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.18","version":"0'0","reported_seq":1,"reported_epoch":71,"state":"creating+peering","last_fresh":"2023-11-24T07:59:12.625986+0000","last_change":"2023-11-24T07:59:12.625986+0000","last_active":"2023-11-24T07:59:12.615544+0000","last_peered":"2023-11-24T07:59:12.615544+0000","last_clean":"2023-11-24T07:59:12.615544+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T07:59:12.625986+0000","last_undegraded":"2023-11-24T07:59:12.625986+0000","last_fullsized":"2023-11-24T07:59:12.625986+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:59:12.615544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:59:12.615544+0000","last_clean_scrub_stamp":"2023-11-24T07:59:12.615544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T07:59:13.620687+0000","last_change":"2023-11-24T07:59:13.620687+0000","last_active":"2023-11-24T07:59:13.620687+0000","last_peered":"2023-11-24T07:59:13.620687+0000","last_clean":"2023-11-24T07:59:13.620687+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T07:59:13.620687+0000","last_undegraded":"2023-11-24T07:59:13.620687+0000","last_fullsized":"2023-11-24T07:59:13.620687+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:59:13.620687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:59:13.620687+0000","last_clean_scrub_stamp":"2023-11-24T07:59:13.620687+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.17","version":"0'0","reported_seq":1,"reported_epoch":71,"state":"creating+peering","last_fresh":"2023-11-24T07:59:12.633815+0000","last_change":"2023-11-24T07:59:12.633815+0000","last_active":"2023-11-24T07:59:12.615544+0000","last_peered":"2023-11-24T07:59:12.615544+0000","last_clean":"2023-11-24T07:59:12.615544+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T07:59:12.633815+0000","last_undegraded":"2023-11-24T07:59:12.633815+0000","last_fullsized":"2023-11-24T07:59:12.633815+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:59:12.615544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:59:12.615544+0000","last_clean_scrub_stamp":"2023-11-24T07:59:12.615544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T07:59:13.620687+0000","last_change":"2023-11-24T07:59:13.620687+0000","last_active":"2023-11-24T07:59:13.620687+0000","last_peered":"2023-11-24T07:59:13.620687+0000","last_clean":"2023-11-24T07:59:13.620687+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T07:59:13.620687+0000","last_undegraded":"2023-11-24T07:59:13.620687+0000","last_fullsized":"2023-11-24T07:59:13.620687+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:59:13.620687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:59:13.620687+0000","last_clean_scrub_stamp":"2023-11-24T07:59:13.620687+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.16","version":"0'0","reported_seq":1,"reported_epoch":71,"state":"creating+peering","last_fresh":"2023-11-24T07:59:12.627824+0000","last_change":"2023-11-24T07:59:12.627824+0000","last_active":"2023-11-24T07:59:12.615544+0000","last_peered":"2023-11-24T07:59:12.615544+0000","last_clean":"2023-11-24T07:59:12.615544+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T07:59:12.627824+0000","last_undegraded":"2023-11-24T07:59:12.627824+0000","last_fullsized":"2023-11-24T07:59:12.627824+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:59:12.615544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:59:12.615544+0000","last_clean_scrub_stamp":"2023-11-24T07:59:12.615544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T07:59:13.620687+0000","last_change":"2023-11-24T07:59:13.620687+0000","last_active":"2023-11-24T07:59:13.620687+0000","last_peered":"2023-11-24T07:59:13.620687+0000","last_clean":"2023-11-24T07:59:13.620687+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T07:59:13.620687+0000","last_undegraded":"2023-11-24T07:59:13.620687+0000","last_fullsized":"2023-11-24T07:59:13.620687+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:59:13.620687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:59:13.620687+0000","last_clean_scrub_stamp":"2023-11-24T07:59:13.620687+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.15","version":"0'0","reported_seq":1,"reported_epoch":71,"state":"creating+peering","last_fresh":"2023-11-24T07:59:12.626663+0000","last_change":"2023-11-24T07:59:12.626663+0000","last_active":"2023-11-24T07:59:12.615544+0000","last_peered":"2023-11-24T07:59:12.615544+0000","last_clean":"2023-11-24T07:59:12.615544+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T07:59:12.626663+0000","last_undegraded":"2023-11-24T07:59:12.626663+0000","last_fullsized":"2023-11-24T07:59:12.626663+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:59:12.615544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:59:12.615544+0000","last_clean_scrub_stamp":"2023-11-24T07:59:12.615544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T07:59:13.620687+0000","last_change":"2023-11-24T07:59:13.620687+0000","last_active":"2023-11-24T07:59:13.620687+0000","last_peered":"2023-11-24T07:59:13.620687+0000","last_clean":"2023-11-24T07:59:13.620687+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T07:59:13.620687+0000","last_undegraded":"2023-11-24T07:59:13.620687+0000","last_fullsized":"2023-11-24T07:59:13.620687+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:59:13.620687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:59:13.620687+0000","last_clean_scrub_stamp":"2023-11-24T07:59:13.620687+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.14","version":"0'0","reported_seq":1,"reported_epoch":71,"state":"creating+peering","last_fresh":"2023-11-24T07:59:12.629104+0000","last_change":"2023-11-24T07:59:12.629104+0000","last_active":"2023-11-24T07:59:12.615544+0000","last_peered":"2023-11-24T07:59:12.615544+0000","last_clean":"2023-11-24T07:59:12.615544+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T07:59:12.629104+0000","last_undegraded":"2023-11-24T07:59:12.629104+0000","last_fullsized":"2023-11-24T07:59:12.629104+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:59:12.615544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:59:12.615544+0000","last_clean_scrub_stamp":"2023-11-24T07:59:12.615544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T07:59:13.620687+0000","last_change":"2023-11-24T07:59:13.620687+0000","last_active":"2023-11-24T07:59:13.620687+0000","last_peered":"2023-11-24T07:59:13.620687+0000","last_clean":"2023-11-24T07:59:13.620687+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T07:59:13.620687+0000","last_undegraded":"2023-11-24T07:59:13.620687+0000","last_fullsized":"2023-11-24T07:59:13.620687+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:59:13.620687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:59:13.620687+0000","last_clean_scrub_stamp":"2023-11-24T07:59:13.620687+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.13","version":"0'0","reported_seq":1,"reported_epoch":71,"state":"creating+peering","last_fresh":"2023-11-24T07:59:12.629097+0000","last_change":"2023-11-24T07:59:12.629097+0000","last_active":"2023-11-24T07:59:12.615544+0000","last_peered":"2023-11-24T07:59:12.615544+0000","last_clean":"2023-11-24T07:59:12.615544+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T07:59:12.629097+0000","last_undegraded":"2023-11-24T07:59:12.629097+0000","last_fullsized":"2023-11-24T07:59:12.629097+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:59:12.615544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:59:12.615544+0000","last_clean_scrub_stamp":"2023-11-24T07:59:12.615544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T07:59:13.620687+0000","last_change":"2023-11-24T07:59:13.620687+0000","last_active":"2023-11-24T07:59:13.620687+0000","last_peered":"2023-11-24T07:59:13.620687+0000","last_clean":"2023-11-24T07:59:13.620687+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T07:59:13.620687+0000","last_undegraded":"2023-11-24T07:59:13.620687+0000","last_fullsized":"2023-11-24T07:59:13.620687+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:59:13.620687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:59:13.620687+0000","last_clean_scrub_stamp":"2023-11-24T07:59:13.620687+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.12","version":"0'0","reported_seq":1,"reported_epoch":71,"state":"creating+peering","last_fresh":"2023-11-24T07:59:12.624085+0000","last_change":"2023-11-24T07:59:12.624085+0000","last_active":"2023-11-24T07:59:12.615544+0000","last_peered":"2023-11-24T07:59:12.615544+0000","last_clean":"2023-11-24T07:59:12.615544+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T07:59:12.624085+0000","last_undegraded":"2023-11-24T07:59:12.624085+0000","last_fullsized":"2023-11-24T07:59:12.624085+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:59:12.615544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:59:12.615544+0000","last_clean_scrub_stamp":"2023-11-24T07:59:12.615544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T07:59:13.620687+0000","last_change":"2023-11-24T07:59:13.620687+0000","last_active":"2023-11-24T07:59:13.620687+0000","last_peered":"2023-11-24T07:59:13.620687+0000","last_clean":"2023-11-24T07:59:13.620687+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T07:59:13.620687+0000","last_undegraded":"2023-11-24T07:59:13.620687+0000","last_fullsized":"2023-11-24T07:59:13.620687+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:59:13.620687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:59:13.620687+0000","last_clean_scrub_stamp":"2023-11-24T07:59:13.620687+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.11","version":"0'0","reported_seq":1,"reported_epoch":71,"state":"creating+peering","last_fresh":"2023-11-24T07:59:12.624082+0000","last_change":"2023-11-24T07:59:12.624082+0000","last_active":"2023-11-24T07:59:12.615544+0000","last_peered":"2023-11-24T07:59:12.615544+0000","last_clean":"2023-11-24T07:59:12.615544+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T07:59:12.624082+0000","last_undegraded":"2023-11-24T07:59:12.624082+0000","last_fullsized":"2023-11-24T07:59:12.624082+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:59:12.615544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:59:12.615544+0000","last_clean_scrub_stamp":"2023-11-24T07:59:12.615544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T07:59:13.620687+0000","last_change":"2023-11-24T07:59:13.620687+0000","last_active":"2023-11-24T07:59:13.620687+0000","last_peered":"2023-11-24T07:59:13.620687+0000","last_clean":"2023-11-24T07:59:13.620687+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T07:59:13.620687+0000","last_undegraded":"2023-11-24T07:59:13.620687+0000","last_fullsized":"2023-11-24T07:59:13.620687+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:59:13.620687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:59:13.620687+0000","last_clean_scrub_stamp":"2023-11-24T07:59:13.620687+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.10","version":"0'0","reported_seq":1,"reported_epoch":71,"state":"creating+peering","last_fresh":"2023-11-24T07:59:12.624170+0000","last_change":"2023-11-24T07:59:12.624170+0000","last_active":"2023-11-24T07:59:12.615544+0000","last_peered":"2023-11-24T07:59:12.615544+0000","last_clean":"2023-11-24T07:59:12.615544+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T07:59:12.624170+0000","last_undegraded":"2023-11-24T07:59:12.624170+0000","last_fullsized":"2023-11-24T07:59:12.624170+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:59:12.615544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:59:12.615544+0000","last_clean_scrub_stamp":"2023-11-24T07:59:12.615544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T07:59:13.620687+0000","last_change":"2023-11-24T07:59:13.620687+0000","last_active":"2023-11-24T07:59:13.620687+0000","last_peered":"2023-11-24T07:59:13.620687+0000","last_clean":"2023-11-24T07:59:13.620687+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T07:59:13.620687+0000","last_undegraded":"2023-11-24T07:59:13.620687+0000","last_fullsized":"2023-11-24T07:59:13.620687+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:59:13.620687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:59:13.620687+0000","last_clean_scrub_stamp":"2023-11-24T07:59:13.620687+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.f","version":"0'0","reported_seq":1,"reported_epoch":71,"state":"creating+peering","last_fresh":"2023-11-24T07:59:12.633408+0000","last_change":"2023-11-24T07:59:12.633408+0000","last_active":"2023-11-24T07:59:12.615544+0000","last_peered":"2023-11-24T07:59:12.615544+0000","last_clean":"2023-11-24T07:59:12.615544+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T07:59:12.633408+0000","last_undegraded":"2023-11-24T07:59:12.633408+0000","last_fullsized":"2023-11-24T07:59:12.633408+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:59:12.615544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:59:12.615544+0000","last_clean_scrub_stamp":"2023-11-24T07:59:12.615544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T07:59:13.620687+0000","last_change":"2023-11-24T07:59:13.620687+0000","last_active":"2023-11-24T07:59:13.620687+0000","last_peered":"2023-11-24T07:59:13.620687+0000","last_clean":"2023-11-24T07:59:13.620687+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T07:59:13.620687+0000","last_undegraded":"2023-11-24T07:59:13.620687+0000","last_fullsized":"2023-11-24T07:59:13.620687+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:59:13.620687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:59:13.620687+0000","last_clean_scrub_stamp":"2023-11-24T07:59:13.620687+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.e","version":"0'0","reported_seq":1,"reported_epoch":71,"state":"creating+peering","last_fresh":"2023-11-24T07:59:12.627298+0000","last_change":"2023-11-24T07:59:12.627298+0000","last_active":"2023-11-24T07:59:12.615544+0000","last_peered":"2023-11-24T07:59:12.615544+0000","last_clean":"2023-11-24T07:59:12.615544+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T07:59:12.627298+0000","last_undegraded":"2023-11-24T07:59:12.627298+0000","last_fullsized":"2023-11-24T07:59:12.627298+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:59:12.615544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:59:12.615544+0000","last_clean_scrub_stamp":"2023-11-24T07:59:12.615544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T07:59:13.620687+0000","last_change":"2023-11-24T07:59:13.620687+0000","last_active":"2023-11-24T07:59:13.620687+0000","last_peered":"2023-11-24T07:59:13.620687+0000","last_clean":"2023-11-24T07:59:13.620687+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T07:59:13.620687+0000","last_undegraded":"2023-11-24T07:59:13.620687+0000","last_fullsized":"2023-11-24T07:59:13.620687+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:59:13.620687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:59:13.620687+0000","last_clean_scrub_stamp":"2023-11-24T07:59:13.620687+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.d","version":"0'0","reported_seq":1,"reported_epoch":71,"state":"creating+peering","last_fresh":"2023-11-24T07:59:12.627664+0000","last_change":"2023-11-24T07:59:12.627664+0000","last_active":"2023-11-24T07:59:12.615544+0000","last_peered":"2023-11-24T07:59:12.615544+0000","last_clean":"2023-11-24T07:59:12.615544+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T07:59:12.627664+0000","last_undegraded":"2023-11-24T07:59:12.627664+0000","last_fullsized":"2023-11-24T07:59:12.627664+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:59:12.615544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:59:12.615544+0000","last_clean_scrub_stamp":"2023-11-24T07:59:12.615544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T07:59:13.620687+0000","last_change":"2023-11-24T07:59:13.620687+0000","last_active":"2023-11-24T07:59:13.620687+0000","last_peered":"2023-11-24T07:59:13.620687+0000","last_clean":"2023-11-24T07:59:13.620687+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T07:59:13.620687+0000","last_undegraded":"2023-11-24T07:59:13.620687+0000","last_fullsized":"2023-11-24T07:59:13.620687+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:59:13.620687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:59:13.620687+0000","last_clean_scrub_stamp":"2023-11-24T07:59:13.620687+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.c","version":"0'0","reported_seq":1,"reported_epoch":71,"state":"creating+peering","last_fresh":"2023-11-24T07:59:12.627260+0000","last_change":"2023-11-24T07:59:12.627260+0000","last_active":"2023-11-24T07:59:12.615544+0000","last_peered":"2023-11-24T07:59:12.615544+0000","last_clean":"2023-11-24T07:59:12.615544+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T07:59:12.627260+0000","last_undegraded":"2023-11-24T07:59:12.627260+0000","last_fullsized":"2023-11-24T07:59:12.627260+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:59:12.615544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:59:12.615544+0000","last_clean_scrub_stamp":"2023-11-24T07:59:12.615544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T07:59:13.620687+0000","last_change":"2023-11-24T07:59:13.620687+0000","last_active":"2023-11-24T07:59:13.620687+0000","last_peered":"2023-11-24T07:59:13.620687+0000","last_clean":"2023-11-24T07:59:13.620687+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T07:59:13.620687+0000","last_undegraded":"2023-11-24T07:59:13.620687+0000","last_fullsized":"2023-11-24T07:59:13.620687+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:59:13.620687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:59:13.620687+0000","last_clean_scrub_stamp":"2023-11-24T07:59:13.620687+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.a","version":"0'0","reported_seq":1,"reported_epoch":71,"state":"creating+peering","last_fresh":"2023-11-24T07:59:12.627164+0000","last_change":"2023-11-24T07:59:12.627164+0000","last_active":"2023-11-24T07:59:12.615544+0000","last_peered":"2023-11-24T07:59:12.615544+0000","last_clean":"2023-11-24T07:59:12.615544+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T07:59:12.627164+0000","last_undegraded":"2023-11-24T07:59:12.627164+0000","last_fullsized":"2023-11-24T07:59:12.627164+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:59:12.615544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:59:12.615544+0000","last_clean_scrub_stamp":"2023-11-24T07:59:12.615544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T07:59:13.620687+0000","last_change":"2023-11-24T07:59:13.620687+0000","last_active":"2023-11-24T07:59:13.620687+0000","last_peered":"2023-11-24T07:59:13.620687+0000","last_clean":"2023-11-24T07:59:13.620687+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T07:59:13.620687+0000","last_undegraded":"2023-11-24T07:59:13.620687+0000","last_fullsized":"2023-11-24T07:59:13.620687+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:59:13.620687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:59:13.620687+0000","last_clean_scrub_stamp":"2023-11-24T07:59:13.620687+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.9","version":"0'0","reported_seq":1,"reported_epoch":71,"state":"creating+peering","last_fresh":"2023-11-24T07:59:12.628577+0000","last_change":"2023-11-24T07:59:12.628577+0000","last_active":"2023-11-24T07:59:12.615544+0000","last_peered":"2023-11-24T07:59:12.615544+0000","last_clean":"2023-11-24T07:59:12.615544+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T07:59:12.628577+0000","last_undegraded":"2023-11-24T07:59:12.628577+0000","last_fullsized":"2023-11-24T07:59:12.628577+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:59:12.615544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:59:12.615544+0000","last_clean_scrub_stamp":"2023-11-24T07:59:12.615544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T07:59:13.620687+0000","last_change":"2023-11-24T07:59:13.620687+0000","last_active":"2023-11-24T07:59:13.620687+0000","last_peered":"2023-11-24T07:59:13.620687+0000","last_clean":"2023-11-24T07:59:13.620687+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T07:59:13.620687+0000","last_undegraded":"2023-11-24T07:59:13.620687+0000","last_fullsized":"2023-11-24T07:59:13.620687+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:59:13.620687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:59:13.620687+0000","last_clean_scrub_stamp":"2023-11-24T07:59:13.620687+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.8","version":"0'0","reported_seq":1,"reported_epoch":71,"state":"creating+peering","last_fresh":"2023-11-24T07:59:12.622958+0000","last_change":"2023-11-24T07:59:12.622958+0000","last_active":"2023-11-24T07:59:12.615544+0000","last_peered":"2023-11-24T07:59:12.615544+0000","last_clean":"2023-11-24T07:59:12.615544+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T07:59:12.622958+0000","last_undegraded":"2023-11-24T07:59:12.622958+0000","last_fullsized":"2023-11-24T07:59:12.622958+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:59:12.615544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:59:12.615544+0000","last_clean_scrub_stamp":"2023-11-24T07:59:12.615544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T07:59:13.620687+0000","last_change":"2023-11-24T07:59:13.620687+0000","last_active":"2023-11-24T07:59:13.620687+0000","last_peered":"2023-11-24T07:59:13.620687+0000","last_clean":"2023-11-24T07:59:13.620687+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T07:59:13.620687+0000","last_undegraded":"2023-11-24T07:59:13.620687+0000","last_fullsized":"2023-11-24T07:59:13.620687+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:59:13.620687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:59:13.620687+0000","last_clean_scrub_stamp":"2023-11-24T07:59:13.620687+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.7","version":"0'0","reported_seq":1,"reported_epoch":71,"state":"creating+peering","last_fresh":"2023-11-24T07:59:12.632816+0000","last_change":"2023-11-24T07:59:12.632816+0000","last_active":"2023-11-24T07:59:12.615544+0000","last_peered":"2023-11-24T07:59:12.615544+0000","last_clean":"2023-11-24T07:59:12.615544+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T07:59:12.632816+0000","last_undegraded":"2023-11-24T07:59:12.632816+0000","last_fullsized":"2023-11-24T07:59:12.632816+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:59:12.615544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:59:12.615544+0000","last_clean_scrub_stamp":"2023-11-24T07:59:12.615544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T07:59:13.620687+0000","last_change":"2023-11-24T07:59:13.620687+0000","last_active":"2023-11-24T07:59:13.620687+0000","last_peered":"2023-11-24T07:59:13.620687+0000","last_clean":"2023-11-24T07:59:13.620687+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T07:59:13.620687+0000","last_undegraded":"2023-11-24T07:59:13.620687+0000","last_fullsized":"2023-11-24T07:59:13.620687+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:59:13.620687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:59:13.620687+0000","last_clean_scrub_stamp":"2023-11-24T07:59:13.620687+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.6","version":"0'0","reported_seq":1,"reported_epoch":71,"state":"creating+peering","last_fresh":"2023-11-24T07:59:12.632780+0000","last_change":"2023-11-24T07:59:12.632780+0000","last_active":"2023-11-24T07:59:12.615544+0000","last_peered":"2023-11-24T07:59:12.615544+0000","last_clean":"2023-11-24T07:59:12.615544+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T07:59:12.632780+0000","last_undegraded":"2023-11-24T07:59:12.632780+0000","last_fullsized":"2023-11-24T07:59:12.632780+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:59:12.615544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:59:12.615544+0000","last_clean_scrub_stamp":"2023-11-24T07:59:12.615544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T07:59:13.620687+0000","last_change":"2023-11-24T07:59:13.620687+0000","last_active":"2023-11-24T07:59:13.620687+0000","last_peered":"2023-11-24T07:59:13.620687+0000","last_clean":"2023-11-24T07:59:13.620687+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T07:59:13.620687+0000","last_undegraded":"2023-11-24T07:59:13.620687+0000","last_fullsized":"2023-11-24T07:59:13.620687+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:59:13.620687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:59:13.620687+0000","last_clean_scrub_stamp":"2023-11-24T07:59:13.620687+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.5","version":"0'0","reported_seq":1,"reported_epoch":71,"state":"creating+peering","last_fresh":"2023-11-24T07:59:12.626089+0000","last_change":"2023-11-24T07:59:12.626089+0000","last_active":"2023-11-24T07:59:12.615544+0000","last_peered":"2023-11-24T07:59:12.615544+0000","last_clean":"2023-11-24T07:59:12.615544+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T07:59:12.626089+0000","last_undegraded":"2023-11-24T07:59:12.626089+0000","last_fullsized":"2023-11-24T07:59:12.626089+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:59:12.615544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:59:12.615544+0000","last_clean_scrub_stamp":"2023-11-24T07:59:12.615544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T07:59:13.620687+0000","last_change":"2023-11-24T07:59:13.620687+0000","last_active":"2023-11-24T07:59:13.620687+0000","last_peered":"2023-11-24T07:59:13.620687+0000","last_clean":"2023-11-24T07:59:13.620687+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T07:59:13.620687+0000","last_undegraded":"2023-11-24T07:59:13.620687+0000","last_fullsized":"2023-11-24T07:59:13.620687+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:59:13.620687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:59:13.620687+0000","last_clean_scrub_stamp":"2023-11-24T07:59:13.620687+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.4","version":"0'0","reported_seq":1,"reported_epoch":71,"state":"creating+peering","last_fresh":"2023-11-24T07:59:12.626088+0000","last_change":"2023-11-24T07:59:12.626088+0000","last_active":"2023-11-24T07:59:12.615544+0000","last_peered":"2023-11-24T07:59:12.615544+0000","last_clean":"2023-11-24T07:59:12.615544+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T07:59:12.626088+0000","last_undegraded":"2023-11-24T07:59:12.626088+0000","last_fullsized":"2023-11-24T07:59:12.626088+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:59:12.615544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:59:12.615544+0000","last_clean_scrub_stamp":"2023-11-24T07:59:12.615544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T07:59:13.620687+0000","last_change":"2023-11-24T07:59:13.620687+0000","last_active":"2023-11-24T07:59:13.620687+0000","last_peered":"2023-11-24T07:59:13.620687+0000","last_clean":"2023-11-24T07:59:13.620687+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T07:59:13.620687+0000","last_undegraded":"2023-11-24T07:59:13.620687+0000","last_fullsized":"2023-11-24T07:59:13.620687+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:59:13.620687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:59:13.620687+0000","last_clean_scrub_stamp":"2023-11-24T07:59:13.620687+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.3","version":"0'0","reported_seq":1,"reported_epoch":71,"state":"creating+peering","last_fresh":"2023-11-24T07:59:12.626018+0000","last_change":"2023-11-24T07:59:12.626018+0000","last_active":"2023-11-24T07:59:12.615544+0000","last_peered":"2023-11-24T07:59:12.615544+0000","last_clean":"2023-11-24T07:59:12.615544+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T07:59:12.626018+0000","last_undegraded":"2023-11-24T07:59:12.626018+0000","last_fullsized":"2023-11-24T07:59:12.626018+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:59:12.615544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:59:12.615544+0000","last_clean_scrub_stamp":"2023-11-24T07:59:12.615544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T07:59:13.620687+0000","last_change":"2023-11-24T07:59:13.620687+0000","last_active":"2023-11-24T07:59:13.620687+0000","last_peered":"2023-11-24T07:59:13.620687+0000","last_clean":"2023-11-24T07:59:13.620687+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T07:59:13.620687+0000","last_undegraded":"2023-11-24T07:59:13.620687+0000","last_fullsized":"2023-11-24T07:59:13.620687+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:59:13.620687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:59:13.620687+0000","last_clean_scrub_stamp":"2023-11-24T07:59:13.620687+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.2","version":"0'0","reported_seq":1,"reported_epoch":71,"state":"creating+peering","last_fresh":"2023-11-24T07:59:12.622957+0000","last_change":"2023-11-24T07:59:12.622957+0000","last_active":"2023-11-24T07:59:12.615544+0000","last_peered":"2023-11-24T07:59:12.615544+0000","last_clean":"2023-11-24T07:59:12.615544+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T07:59:12.622957+0000","last_undegraded":"2023-11-24T07:59:12.622957+0000","last_fullsized":"2023-11-24T07:59:12.622957+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:59:12.615544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:59:12.615544+0000","last_clean_scrub_stamp":"2023-11-24T07:59:12.615544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T07:59:13.620687+0000","last_change":"2023-11-24T07:59:13.620687+0000","last_active":"2023-11-24T07:59:13.620687+0000","last_peered":"2023-11-24T07:59:13.620687+0000","last_clean":"2023-11-24T07:59:13.620687+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T07:59:13.620687+0000","last_undegraded":"2023-11-24T07:59:13.620687+0000","last_fullsized":"2023-11-24T07:59:13.620687+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:59:13.620687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:59:13.620687+0000","last_clean_scrub_stamp":"2023-11-24T07:59:13.620687+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.1","version":"0'0","reported_seq":1,"reported_epoch":71,"state":"creating+peering","last_fresh":"2023-11-24T07:59:12.628166+0000","last_change":"2023-11-24T07:59:12.628166+0000","last_active":"2023-11-24T07:59:12.615544+0000","last_peered":"2023-11-24T07:59:12.615544+0000","last_clean":"2023-11-24T07:59:12.615544+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T07:59:12.628166+0000","last_undegraded":"2023-11-24T07:59:12.628166+0000","last_fullsized":"2023-11-24T07:59:12.628166+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:59:12.615544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:59:12.615544+0000","last_clean_scrub_stamp":"2023-11-24T07:59:12.615544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T07:59:13.620687+0000","last_change":"2023-11-24T07:59:13.620687+0000","last_active":"2023-11-24T07:59:13.620687+0000","last_peered":"2023-11-24T07:59:13.620687+0000","last_clean":"2023-11-24T07:59:13.620687+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T07:59:13.620687+0000","last_undegraded":"2023-11-24T07:59:13.620687+0000","last_fullsized":"2023-11-24T07:59:13.620687+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:59:13.620687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:59:13.620687+0000","last_clean_scrub_stamp":"2023-11-24T07:59:13.620687+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.0","version":"0'0","reported_seq":1,"reported_epoch":71,"state":"creating+peering","last_fresh":"2023-11-24T07:59:12.627478+0000","last_change":"2023-11-24T07:59:12.627478+0000","last_active":"2023-11-24T07:59:12.615544+0000","last_peered":"2023-11-24T07:59:12.615544+0000","last_clean":"2023-11-24T07:59:12.615544+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T07:59:12.627478+0000","last_undegraded":"2023-11-24T07:59:12.627478+0000","last_fullsized":"2023-11-24T07:59:12.627478+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:59:12.615544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:59:12.615544+0000","last_clean_scrub_stamp":"2023-11-24T07:59:12.615544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T07:59:13.620687+0000","last_change":"2023-11-24T07:59:13.620687+0000","last_active":"2023-11-24T07:59:13.620687+0000","last_peered":"2023-11-24T07:59:13.620687+0000","last_clean":"2023-11-24T07:59:13.620687+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T07:59:13.620687+0000","last_undegraded":"2023-11-24T07:59:13.620687+0000","last_fullsized":"2023-11-24T07:59:13.620687+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:59:13.620687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:59:13.620687+0000","last_clean_scrub_stamp":"2023-11-24T07:59:13.620687+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.b","version":"0'0","reported_seq":1,"reported_epoch":71,"state":"creating+peering","last_fresh":"2023-11-24T07:59:12.624115+0000","last_change":"2023-11-24T07:59:12.624115+0000","last_active":"2023-11-24T07:59:12.615544+0000","last_peered":"2023-11-24T07:59:12.615544+0000","last_clean":"2023-11-24T07:59:12.615544+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T07:59:12.624115+0000","last_undegraded":"2023-11-24T07:59:12.624115+0000","last_fullsized":"2023-11-24T07:59:12.624115+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:59:12.615544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:59:12.615544+0000","last_clean_scrub_stamp":"2023-11-24T07:59:12.615544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":208,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-11-24T07:59:09.991344+0000","last_change":"2023-11-24T07:56:19.668905+0000","last_active":"2023-11-24T07:59:09.991344+0000","last_peered":"2023-11-24T07:59:09.991344+0000","last_clean":"2023-11-24T07:59:09.991344+0000","last_became_active":"2023-11-24T07:56:19.668601+0000","last_became_peered":"2023-11-24T07:56:19.668601+0000","last_unstale":"2023-11-24T07:59:09.991344+0000","last_undegraded":"2023-11-24T07:59:09.991344+0000","last_fullsized":"2023-11-24T07:59:09.991344+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:56:18.657844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:56:18.657844+0000","last_clean_scrub_stamp":"2023-11-24T07:56:18.657844+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T11:51:53.828455+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":11,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":10,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574886,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27872,"kb_used_data":844,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343968,"statfs":{"total":96636764160,"available":96608223232,"internally_reserved":0,"allocated":864256,"data_stored":364790,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 07:58:20 2023","interfaces":[{"interface":"back","average":{"1min":0.51400000000000001,"5min":0.54800000000000004,"15min":0.55400000000000005},"min":{"1min":0.309,"5min":0.309,"15min":0.309},"max":{"1min":0.78500000000000003,"5min":1.413,"15min":1.413},"last":0.55200000000000005},{"interface":"front","average":{"1min":0.56299999999999994,"5min":0.55500000000000005,"15min":0.55400000000000005},"min":{"1min":0.436,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":0.77000000000000002,"5min":0.81000000000000005,"15min":0.81000000000000005},"last":0.56599999999999995}]},{"osd":1,"last update":"Fri Nov 24 07:58:20 2023","interfaces":[{"interface":"back","average":{"1min":0.501,"5min":0.77900000000000003,"15min":0.82499999999999996},"min":{"1min":0.32000000000000001,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.69899999999999995,"5min":6.1070000000000002,"15min":6.1070000000000002},"last":0.53100000000000003},{"interface":"front","average":{"1min":0.51300000000000001,"5min":0.56599999999999995,"15min":0.57499999999999996},"min":{"1min":0.40699999999999997,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.73599999999999999,"5min":1.5920000000000001,"15min":1.5920000000000001},"last":0.49399999999999999}]},{"osd":2,"last update":"Fri Nov 24 07:58:20 2023","interfaces":[{"interface":"back","average":{"1min":0.498,"5min":0.60099999999999998,"15min":0.61799999999999999},"min":{"1min":0.35099999999999998,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.71599999999999997,"5min":1.613,"15min":1.613},"last":0.40100000000000002},{"interface":"front","average":{"1min":0.502,"5min":0.73399999999999999,"15min":0.77200000000000002},"min":{"1min":0.34899999999999998,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.71299999999999997,"5min":4.5220000000000002,"15min":4.5220000000000002},"last":0.44700000000000001}]},{"osd":3,"last update":"Fri Nov 24 07:58:20 2023","interfaces":[{"interface":"back","average":{"1min":0.55100000000000005,"5min":0.755,"15min":0.78900000000000003},"min":{"1min":0.41699999999999998,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.754,"5min":4.6760000000000002,"15min":4.6760000000000002},"last":0.626},{"interface":"front","average":{"1min":0.60699999999999998,"5min":0.875,"15min":0.92000000000000004},"min":{"1min":0.38500000000000001,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.81599999999999995,"5min":4.6539999999999999,"15min":4.6539999999999999},"last":0.64400000000000002}]},{"osd":4,"last update":"Fri Nov 24 07:58:20 2023","interfaces":[{"interface":"back","average":{"1min":0.54900000000000004,"5min":0.56399999999999995,"15min":0.56599999999999995},"min":{"1min":0.41799999999999998,"5min":0.378,"15min":0.378},"max":{"1min":0.70999999999999996,"5min":0.84699999999999998,"15min":0.84699999999999998},"last":0.42699999999999999},{"interface":"front","average":{"1min":0.54700000000000004,"5min":0.64300000000000002,"15min":0.65900000000000003},"min":{"1min":0.35199999999999998,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.79900000000000004,"5min":1.629,"15min":1.629},"last":0.59499999999999997}]},{"osd":5,"last update":"Fri Nov 24 07:58:20 2023","interfaces":[{"interface":"back","average":{"1min":0.54200000000000004,"5min":0.54600000000000004,"15min":0.54600000000000004},"min":{"1min":0.41599999999999998,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.67900000000000005,"5min":0.875,"15min":0.875},"last":0.51600000000000001},{"interface":"front","average":{"1min":0.57799999999999996,"5min":0.61399999999999999,"15min":0.62},"min":{"1min":0.36799999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.77700000000000002,"5min":0.90900000000000003,"15min":0.90900000000000003},"last":0.57999999999999996}]},{"osd":6,"last update":"Fri Nov 24 07:58:20 2023","interfaces":[{"interface":"back","average":{"1min":0.59999999999999998,"5min":0.60799999999999998,"15min":0.60999999999999999},"min":{"1min":0.49099999999999999,"5min":0.438,"15min":0.438},"max":{"1min":0.73399999999999999,"5min":0.80800000000000005,"15min":0.80800000000000005},"last":0.61299999999999999},{"interface":"front","average":{"1min":0.63900000000000001,"5min":0.66300000000000003,"15min":0.66700000000000004},"min":{"1min":0.45600000000000002,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":0.79700000000000004,"5min":0.98399999999999999,"15min":0.98399999999999999},"last":0.66100000000000003}]}]},{"osd":6,"up_from":13,"seq":55834574887,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27872,"kb_used_data":844,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343968,"statfs":{"total":96636764160,"available":96608223232,"internally_reserved":0,"allocated":864256,"data_stored":364790,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 07:58:19 2023","interfaces":[{"interface":"back","average":{"1min":0.53300000000000003,"5min":0.54600000000000004,"15min":0.54800000000000004},"min":{"1min":0.29799999999999999,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":0.93100000000000005,"5min":0.93100000000000005,"15min":0.93100000000000005},"last":0.497},{"interface":"front","average":{"1min":0.51600000000000001,"5min":0.51600000000000001,"15min":0.51600000000000001},"min":{"1min":0.311,"5min":0.311,"15min":0.311},"max":{"1min":0.82199999999999995,"5min":0.82199999999999995,"15min":0.82199999999999995},"last":0.34899999999999998}]},{"osd":1,"last update":"Fri Nov 24 07:58:19 2023","interfaces":[{"interface":"back","average":{"1min":0.51900000000000002,"5min":0.51300000000000001,"15min":0.51200000000000001},"min":{"1min":0.35599999999999998,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.81299999999999994,"5min":0.81299999999999994,"15min":0.81299999999999994},"last":0.48399999999999999},{"interface":"front","average":{"1min":0.54000000000000004,"5min":0.59799999999999998,"15min":0.60799999999999998},"min":{"1min":0.35999999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.82799999999999996,"5min":1.0069999999999999,"15min":1.0069999999999999},"last":0.50900000000000001}]},{"osd":2,"last update":"Fri Nov 24 07:58:19 2023","interfaces":[{"interface":"back","average":{"1min":0.58399999999999996,"5min":0.57599999999999996,"15min":0.57499999999999996},"min":{"1min":0.34899999999999998,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.90600000000000003,"5min":0.90600000000000003,"15min":0.90600000000000003},"last":0.52600000000000002},{"interface":"front","average":{"1min":0.56499999999999995,"5min":0.58699999999999997,"15min":0.59099999999999997},"min":{"1min":0.34300000000000003,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.84899999999999998,"5min":0.84899999999999998,"15min":0.84899999999999998},"last":0.53700000000000003}]},{"osd":3,"last update":"Fri Nov 24 07:58:19 2023","interfaces":[{"interface":"back","average":{"1min":0.56000000000000005,"5min":0.56599999999999995,"15min":0.56699999999999995},"min":{"1min":0.38400000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.86499999999999999,"5min":0.876,"15min":0.876},"last":0.40100000000000002},{"interface":"front","average":{"1min":0.57699999999999996,"5min":0.56000000000000005,"15min":0.55700000000000005},"min":{"1min":0.39800000000000002,"5min":0.379,"15min":0.379},"max":{"1min":0.876,"5min":0.876,"15min":0.876},"last":0.41699999999999998}]},{"osd":4,"last update":"Fri Nov 24 07:58:19 2023","interfaces":[{"interface":"back","average":{"1min":0.53100000000000003,"5min":0.51900000000000002,"15min":0.51700000000000002},"min":{"1min":0.25,"5min":0.25,"15min":0.25},"max":{"1min":0.89400000000000002,"5min":0.89400000000000002,"15min":0.89400000000000002},"last":0.58299999999999996},{"interface":"front","average":{"1min":0.58899999999999997,"5min":0.59899999999999998,"15min":0.60099999999999998},"min":{"1min":0.29099999999999998,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":1.044,"5min":1.087,"15min":1.087},"last":0.30599999999999999}]},{"osd":5,"last update":"Fri Nov 24 07:58:19 2023","interfaces":[{"interface":"back","average":{"1min":0.59499999999999997,"5min":0.54300000000000004,"15min":0.53500000000000003},"min":{"1min":0.27800000000000002,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":0.97999999999999998,"5min":0.97999999999999998,"15min":0.97999999999999998},"last":0.37},{"interface":"front","average":{"1min":0.61399999999999999,"5min":0.61199999999999999,"15min":0.61199999999999999},"min":{"1min":0.31900000000000001,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":1.0009999999999999,"5min":1.0009999999999999,"15min":1.0009999999999999},"last":0.438}]},{"osd":7,"last update":"Fri Nov 24 07:58:19 2023","interfaces":[{"interface":"back","average":{"1min":0.61799999999999999,"5min":0.61799999999999999,"15min":0.61799999999999999},"min":{"1min":0.26500000000000001,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":1.0880000000000001,"5min":1.0880000000000001,"15min":1.0880000000000001},"last":0.46400000000000002},{"interface":"front","average":{"1min":0.65200000000000002,"5min":0.66600000000000004,"15min":0.66800000000000004},"min":{"1min":0.34300000000000003,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.95599999999999996,"5min":0.95599999999999996,"15min":0.95599999999999996},"last":0.32800000000000001}]}]},{"osd":1,"up_from":13,"seq":55834574888,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28432,"kb_used_data":1412,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343408,"statfs":{"total":96636764160,"available":96607649792,"internally_reserved":0,"allocated":1445888,"data_stored":949341,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 07:58:24 2023","interfaces":[{"interface":"back","average":{"1min":0.33000000000000002,"5min":0.35699999999999998,"15min":0.36099999999999999},"min":{"1min":0.249,"5min":0.20200000000000001,"15min":0.20200000000000001},"max":{"1min":0.48799999999999999,"5min":0.70899999999999996,"15min":0.70899999999999996},"last":0.249},{"interface":"front","average":{"1min":0.36399999999999999,"5min":0.39100000000000001,"15min":0.39500000000000002},"min":{"1min":0.23599999999999999,"5min":0.214,"15min":0.214},"max":{"1min":0.54700000000000004,"5min":0.82499999999999996,"15min":0.82499999999999996},"last":0.309}]},{"osd":2,"last update":"Fri Nov 24 07:58:24 2023","interfaces":[{"interface":"back","average":{"1min":0.312,"5min":0.35999999999999999,"15min":0.36799999999999999},"min":{"1min":0.253,"5min":0.22700000000000001,"15min":0.22700000000000001},"max":{"1min":0.44800000000000001,"5min":0.60999999999999999,"15min":0.60999999999999999},"last":0.40699999999999997},{"interface":"front","average":{"1min":0.36499999999999999,"5min":0.40300000000000002,"15min":0.40899999999999997},"min":{"1min":0.28100000000000003,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":0.47599999999999998,"5min":0.621,"15min":0.621},"last":0.28899999999999998}]},{"osd":3,"last update":"Fri Nov 24 07:58:24 2023","interfaces":[{"interface":"back","average":{"1min":0.40300000000000002,"5min":0.435,"15min":0.441},"min":{"1min":0.31900000000000001,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.59199999999999997,"5min":0.66500000000000004,"15min":0.66500000000000004},"last":0.34699999999999998},{"interface":"front","average":{"1min":0.38700000000000001,"5min":0.41899999999999998,"15min":0.42499999999999999},"min":{"1min":0.29199999999999998,"5min":0.28499999999999998,"15min":0.28499999999999998},"max":{"1min":0.56200000000000006,"5min":0.76300000000000001,"15min":0.76300000000000001},"last":0.36599999999999999}]},{"osd":4,"last update":"Fri Nov 24 07:58:24 2023","interfaces":[{"interface":"back","average":{"1min":0.56200000000000006,"5min":0.60099999999999998,"15min":0.60699999999999998},"min":{"1min":0.442,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":0.84099999999999997,"5min":0.84099999999999997,"15min":0.84099999999999997},"last":0.64200000000000002},{"interface":"front","average":{"1min":0.61099999999999999,"5min":0.60699999999999998,"15min":0.60599999999999998},"min":{"1min":0.499,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":0.79500000000000004,"5min":0.82199999999999995,"15min":0.82199999999999995},"last":0.67800000000000005}]},{"osd":5,"last update":"Fri Nov 24 07:58:24 2023","interfaces":[{"interface":"back","average":{"1min":0.59199999999999997,"5min":0.69799999999999995,"15min":0.71599999999999997},"min":{"1min":0.439,"5min":0.37,"15min":0.37},"max":{"1min":0.97599999999999998,"5min":2.7269999999999999,"15min":2.7269999999999999},"last":0.69399999999999995},{"interface":"front","average":{"1min":0.57699999999999996,"5min":0.63100000000000001,"15min":0.64000000000000001},"min":{"1min":0.39100000000000001,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.94499999999999995,"5min":0.94499999999999995,"15min":0.94499999999999995},"last":0.60999999999999999}]},{"osd":6,"last update":"Fri Nov 24 07:58:24 2023","interfaces":[{"interface":"back","average":{"1min":0.61199999999999999,"5min":0.64800000000000002,"15min":0.65400000000000003},"min":{"1min":0.47699999999999998,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":0.77500000000000002,"5min":0.98199999999999998,"15min":0.98199999999999998},"last":0.70299999999999996},{"interface":"front","average":{"1min":0.57599999999999996,"5min":0.60799999999999998,"15min":0.61399999999999999},"min":{"1min":0.40600000000000003,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":0.89600000000000002,"5min":0.90700000000000003,"15min":0.90700000000000003},"last":0.623}]},{"osd":7,"last update":"Fri Nov 24 07:58:24 2023","interfaces":[{"interface":"back","average":{"1min":0.63100000000000001,"5min":0.63500000000000001,"15min":0.63600000000000001},"min":{"1min":0.43099999999999999,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.79800000000000004,"5min":1.0649999999999999,"15min":1.0649999999999999},"last":0.71399999999999997},{"interface":"front","average":{"1min":0.57099999999999995,"5min":0.65000000000000002,"15min":0.66300000000000003},"min":{"1min":0.45700000000000002,"5min":0.442,"15min":0.442},"max":{"1min":0.71299999999999997,"5min":0.88800000000000001,"15min":0.88800000000000001},"last":0.66700000000000004}]}]},{"osd":0,"up_from":13,"seq":55834574888,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27856,"kb_used_data":832,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343984,"statfs":{"total":96636764160,"available":96608239616,"internally_reserved":0,"allocated":851968,"data_stored":358973,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Nov 24 07:58:20 2023","interfaces":[{"interface":"back","average":{"1min":0.38500000000000001,"5min":0.64700000000000002,"15min":0.69099999999999995},"min":{"1min":0.21299999999999999,"5min":0.21299999999999999,"15min":0.21299999999999999},"max":{"1min":0.625,"5min":6.3550000000000004,"15min":6.3550000000000004},"last":0.34000000000000002},{"interface":"front","average":{"1min":0.40799999999999997,"5min":1.464,"15min":1.6399999999999999},"min":{"1min":0.26000000000000001,"5min":0.26000000000000001,"15min":0.26000000000000001},"max":{"1min":0.60999999999999999,"5min":22.916,"15min":22.916},"last":0.29899999999999999}]},{"osd":2,"last update":"Fri Nov 24 07:58:20 2023","interfaces":[{"interface":"back","average":{"1min":0.35299999999999998,"5min":0.63,"15min":0.67600000000000005},"min":{"1min":0.22900000000000001,"5min":0.22800000000000001,"15min":0.22800000000000001},"max":{"1min":0.499,"5min":6.3899999999999997,"15min":6.3899999999999997},"last":0.27700000000000002},{"interface":"front","average":{"1min":0.39500000000000002,"5min":0.92900000000000005,"15min":1.018},"min":{"1min":0.27900000000000003,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":0.60199999999999998,"5min":11.93,"15min":11.93},"last":0.32800000000000001}]},{"osd":3,"last update":"Fri Nov 24 07:58:20 2023","interfaces":[{"interface":"back","average":{"1min":0.373,"5min":0.40999999999999998,"15min":0.41599999999999998},"min":{"1min":0.23799999999999999,"5min":0.23799999999999999,"15min":0.23799999999999999},"max":{"1min":0.79300000000000004,"5min":1.155,"15min":1.155},"last":0.247},{"interface":"front","average":{"1min":0.438,"5min":0.98899999999999999,"15min":1.081},"min":{"1min":0.30199999999999999,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.69699999999999995,"5min":12.116,"15min":12.116},"last":0.313}]},{"osd":4,"last update":"Fri Nov 24 07:58:20 2023","interfaces":[{"interface":"back","average":{"1min":0.626,"5min":0.64300000000000002,"15min":0.64600000000000002},"min":{"1min":0.40100000000000002,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":1.044,"5min":1.044,"15min":1.044},"last":0.44400000000000001},{"interface":"front","average":{"1min":0.64300000000000002,"5min":1.675,"15min":1.847},"min":{"1min":0.35699999999999998,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.93000000000000005,"5min":22.963999999999999,"15min":22.963999999999999},"last":0.377}]},{"osd":5,"last update":"Fri Nov 24 07:58:20 2023","interfaces":[{"interface":"back","average":{"1min":0.65200000000000002,"5min":1.1639999999999999,"15min":1.25},"min":{"1min":0.41999999999999998,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.97799999999999998,"5min":12.084,"15min":12.084},"last":0.436},{"interface":"front","average":{"1min":0.65800000000000003,"5min":0.67400000000000004,"15min":0.67600000000000005},"min":{"1min":0.39700000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.95499999999999996,"5min":1.073,"15min":1.073},"last":0.503}]},{"osd":6,"last update":"Fri Nov 24 07:58:20 2023","interfaces":[{"interface":"back","average":{"1min":0.65200000000000002,"5min":0.68300000000000005,"15min":0.68799999999999994},"min":{"1min":0.45000000000000001,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":0.94699999999999995,"5min":1.0580000000000001,"15min":1.0580000000000001},"last":0.36799999999999999},{"interface":"front","average":{"1min":0.67000000000000004,"5min":1.1850000000000001,"15min":1.2709999999999999},"min":{"1min":0.46200000000000002,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":0.95899999999999996,"5min":12.045,"15min":12.045},"last":0.46000000000000002}]},{"osd":7,"last update":"Fri Nov 24 07:58:20 2023","interfaces":[{"interface":"back","average":{"1min":0.67000000000000004,"5min":0.65800000000000003,"15min":0.65600000000000003},"min":{"1min":0.45200000000000001,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":0.98299999999999998,"5min":1.038,"15min":1.038},"last":0.46700000000000003},{"interface":"front","average":{"1min":0.67500000000000004,"5min":1.7050000000000001,"15min":1.877},"min":{"1min":0.41299999999999998,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":1.0069999999999999,"5min":22.995000000000001,"15min":22.995000000000001},"last":0.42299999999999999}]}]},{"osd":2,"up_from":13,"seq":55834574888,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27868,"kb_used_data":844,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343972,"statfs":{"total":96636764160,"available":96608227328,"internally_reserved":0,"allocated":864256,"data_stored":364790,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 07:58:19 2023","interfaces":[{"interface":"back","average":{"1min":0.36399999999999999,"5min":0.36899999999999999,"15min":0.37},"min":{"1min":0.219,"5min":0.219,"15min":0.219},"max":{"1min":0.55900000000000005,"5min":0.58099999999999996,"15min":0.58099999999999996},"last":0.32500000000000001},{"interface":"front","average":{"1min":0.38900000000000001,"5min":0.39900000000000002,"15min":0.40100000000000002},"min":{"1min":0.29299999999999998,"5min":0.27400000000000002,"15min":0.27400000000000002},"max":{"1min":0.54100000000000004,"5min":0.54100000000000004,"15min":0.54100000000000004},"last":0.41499999999999998}]},{"osd":1,"last update":"Fri Nov 24 07:58:19 2023","interfaces":[{"interface":"back","average":{"1min":0.34699999999999998,"5min":0.371,"15min":0.375},"min":{"1min":0.22,"5min":0.22,"15min":0.22},"max":{"1min":0.54000000000000004,"5min":0.59199999999999997,"15min":0.59199999999999997},"last":0.30599999999999999},{"interface":"front","average":{"1min":0.36599999999999999,"5min":0.40600000000000003,"15min":0.41199999999999998},"min":{"1min":0.28299999999999997,"5min":0.25600000000000001,"15min":0.25600000000000001},"max":{"1min":0.497,"5min":0.627,"15min":0.627},"last":0.34300000000000003}]},{"osd":3,"last update":"Fri Nov 24 07:58:19 2023","interfaces":[{"interface":"back","average":{"1min":0.40500000000000003,"5min":0.40899999999999997,"15min":0.40999999999999998},"min":{"1min":0.30399999999999999,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":0.64200000000000002,"5min":0.64200000000000002,"15min":0.64200000000000002},"last":0.26400000000000001},{"interface":"front","average":{"1min":0.41699999999999998,"5min":0.41399999999999998,"15min":0.41399999999999998},"min":{"1min":0.28599999999999998,"5min":0.27500000000000002,"15min":0.27500000000000002},"max":{"1min":0.59299999999999997,"5min":0.64600000000000002,"15min":0.64600000000000002},"last":0.36799999999999999}]},{"osd":4,"last update":"Fri Nov 24 07:58:19 2023","interfaces":[{"interface":"back","average":{"1min":0.54800000000000004,"5min":0.59599999999999997,"15min":0.60399999999999998},"min":{"1min":0.38700000000000001,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.75,"5min":0.81599999999999995,"15min":0.81599999999999995},"last":0.46800000000000003},{"interface":"front","average":{"1min":0.55500000000000005,"5min":0.59099999999999997,"15min":0.59699999999999998},"min":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":0.83799999999999997,"5min":0.83799999999999997,"15min":0.83799999999999997},"last":0.60299999999999998}]},{"osd":5,"last update":"Fri Nov 24 07:58:19 2023","interfaces":[{"interface":"back","average":{"1min":0.56000000000000005,"5min":0.60799999999999998,"15min":0.61599999999999999},"min":{"1min":0.434,"5min":0.434,"15min":0.434},"max":{"1min":0.73499999999999999,"5min":0.84899999999999998,"15min":0.84899999999999998},"last":0.503},{"interface":"front","average":{"1min":0.624,"5min":0.69599999999999995,"15min":0.70799999999999996},"min":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":0.82499999999999996,"5min":2.1699999999999999,"15min":2.1699999999999999},"last":0.53500000000000003}]},{"osd":6,"last update":"Fri Nov 24 07:58:19 2023","interfaces":[{"interface":"back","average":{"1min":0.60499999999999998,"5min":0.65400000000000003,"15min":0.66200000000000003},"min":{"1min":0.42499999999999999,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":0.878,"5min":0.878,"15min":0.878},"last":0.48299999999999998},{"interface":"front","average":{"1min":0.60599999999999998,"5min":0.65800000000000003,"15min":0.66600000000000004},"min":{"1min":0.41899999999999998,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":0.80300000000000005,"5min":0.88500000000000001,"15min":0.88500000000000001},"last":0.54900000000000004}]},{"osd":7,"last update":"Fri Nov 24 07:58:19 2023","interfaces":[{"interface":"back","average":{"1min":0.60399999999999998,"5min":0.65000000000000002,"15min":0.65800000000000003},"min":{"1min":0.441,"5min":0.441,"15min":0.441},"max":{"1min":0.77600000000000002,"5min":0.85999999999999999,"15min":0.85999999999999999},"last":0.56799999999999995},{"interface":"front","average":{"1min":0.623,"5min":0.66800000000000004,"15min":0.67600000000000005},"min":{"1min":0.40000000000000002,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":0.90000000000000002,"5min":0.90000000000000002,"15min":0.90000000000000002},"last":0.58399999999999996}]}]},{"osd":3,"up_from":13,"seq":55834574888,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27868,"kb_used_data":844,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343972,"statfs":{"total":96636764160,"available":96608227328,"internally_reserved":0,"allocated":864256,"data_stored":364790,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 07:58:22 2023","interfaces":[{"interface":"back","average":{"1min":0.34200000000000003,"5min":0.35299999999999998,"15min":0.35499999999999998},"min":{"1min":0.20899999999999999,"5min":0.20899999999999999,"15min":0.20899999999999999},"max":{"1min":0.439,"5min":0.84799999999999998,"15min":0.84799999999999998},"last":0.29099999999999998},{"interface":"front","average":{"1min":0.32700000000000001,"5min":0.35899999999999999,"15min":0.36399999999999999},"min":{"1min":0.253,"5min":0.23699999999999999,"15min":0.23699999999999999},"max":{"1min":0.42499999999999999,"5min":0.873,"15min":0.873},"last":0.44900000000000001}]},{"osd":1,"last update":"Fri Nov 24 07:58:22 2023","interfaces":[{"interface":"back","average":{"1min":0.35899999999999999,"5min":0.371,"15min":0.373},"min":{"1min":0.29499999999999998,"5min":0.25700000000000001,"15min":0.25700000000000001},"max":{"1min":0.443,"5min":0.92000000000000004,"15min":0.92000000000000004},"last":0.54000000000000004},{"interface":"front","average":{"1min":0.35099999999999998,"5min":0.379,"15min":0.38300000000000001},"min":{"1min":0.28299999999999997,"5min":0.248,"15min":0.248},"max":{"1min":0.46700000000000003,"5min":0.81699999999999995,"15min":0.81699999999999995},"last":0.40799999999999997}]},{"osd":2,"last update":"Fri Nov 24 07:58:22 2023","interfaces":[{"interface":"back","average":{"1min":0.33400000000000002,"5min":0.35499999999999998,"15min":0.35899999999999999},"min":{"1min":0.255,"5min":0.24399999999999999,"15min":0.24399999999999999},"max":{"1min":0.43099999999999999,"5min":0.88500000000000001,"15min":0.88500000000000001},"last":0.42999999999999999},{"interface":"front","average":{"1min":0.39500000000000002,"5min":0.45300000000000001,"15min":0.46300000000000002},"min":{"1min":0.311,"5min":0.311,"15min":0.311},"max":{"1min":0.45800000000000002,"5min":0.98099999999999998,"15min":0.98099999999999998},"last":0.47399999999999998}]},{"osd":4,"last update":"Fri Nov 24 07:58:22 2023","interfaces":[{"interface":"back","average":{"1min":0.57099999999999995,"5min":0.751,"15min":0.78100000000000003},"min":{"1min":0.42099999999999999,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.71999999999999997,"5min":5.7350000000000003,"15min":5.7350000000000003},"last":0.55500000000000005},{"interface":"front","average":{"1min":0.59299999999999997,"5min":0.58599999999999997,"15min":0.58499999999999996},"min":{"1min":0.37,"5min":0.37,"15min":0.37},"max":{"1min":0.77000000000000002,"5min":1.27,"15min":1.27},"last":0.58599999999999997}]},{"osd":5,"last update":"Fri Nov 24 07:58:22 2023","interfaces":[{"interface":"back","average":{"1min":0.56899999999999995,"5min":0.58399999999999996,"15min":0.58599999999999997},"min":{"1min":0.40799999999999997,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":0.82799999999999996,"5min":1.4430000000000001,"15min":1.4430000000000001},"last":0.65900000000000003},{"interface":"front","average":{"1min":0.60699999999999998,"5min":0.623,"15min":0.625},"min":{"1min":0.443,"5min":0.377,"15min":0.377},"max":{"1min":0.79900000000000004,"5min":1.4239999999999999,"15min":1.4239999999999999},"last":0.64000000000000001}]},{"osd":6,"last update":"Fri Nov 24 07:58:22 2023","interfaces":[{"interface":"back","average":{"1min":0.58999999999999997,"5min":0.61399999999999999,"15min":0.61799999999999999},"min":{"1min":0.46200000000000002,"5min":0.434,"15min":0.434},"max":{"1min":0.82599999999999996,"5min":1.355,"15min":1.355},"last":0.60699999999999998},{"interface":"front","average":{"1min":0.61199999999999999,"5min":0.79900000000000004,"15min":0.82999999999999996},"min":{"1min":0.47499999999999998,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.76900000000000002,"5min":5.6479999999999997,"15min":5.6479999999999997},"last":0.62}]},{"osd":7,"last update":"Fri Nov 24 07:58:22 2023","interfaces":[{"interface":"back","average":{"1min":0.59199999999999997,"5min":0.65100000000000002,"15min":0.66100000000000003},"min":{"1min":0.48099999999999998,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":0.79600000000000004,"5min":1.51,"15min":1.51},"last":0.67900000000000005},{"interface":"front","average":{"1min":0.61099999999999999,"5min":0.79500000000000004,"15min":0.82599999999999996},"min":{"1min":0.434,"5min":0.434,"15min":0.434},"max":{"1min":0.75900000000000001,"5min":5.7069999999999999,"15min":5.7069999999999999},"last":0.496}]}]},{"osd":4,"up_from":13,"seq":55834574887,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27868,"kb_used_data":844,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343972,"statfs":{"total":96636764160,"available":96608227328,"internally_reserved":0,"allocated":864256,"data_stored":364790,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 07:58:25 2023","interfaces":[{"interface":"back","average":{"1min":0.504,"5min":0.56499999999999995,"15min":0.57499999999999996},"min":{"1min":0.34899999999999998,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.70799999999999996,"5min":0.79300000000000004,"15min":0.79300000000000004},"last":0.70399999999999996},{"interface":"front","average":{"1min":0.502,"5min":0.53400000000000003,"15min":0.54000000000000004},"min":{"1min":0.33600000000000002,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.70199999999999996,"5min":1.0249999999999999,"15min":1.0249999999999999},"last":0.69299999999999995}]},{"osd":1,"last update":"Fri Nov 24 07:58:25 2023","interfaces":[{"interface":"back","average":{"1min":0.52800000000000002,"5min":0.54700000000000004,"15min":0.55000000000000004},"min":{"1min":0.39500000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.73999999999999999,"5min":0.86399999999999999,"15min":0.86399999999999999},"last":0.65700000000000003},{"interface":"front","average":{"1min":0.51900000000000002,"5min":0.54400000000000004,"15min":0.54800000000000004},"min":{"1min":0.374,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.72199999999999998,"5min":0.98699999999999999,"15min":0.98699999999999999},"last":0.60599999999999998}]},{"osd":2,"last update":"Fri Nov 24 07:58:25 2023","interfaces":[{"interface":"back","average":{"1min":0.52000000000000002,"5min":0.55200000000000005,"15min":0.55800000000000005},"min":{"1min":0.38800000000000001,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.66400000000000003,"5min":0.81100000000000005,"15min":0.81100000000000005},"last":0.63700000000000001},{"interface":"front","average":{"1min":0.56899999999999995,"5min":0.60399999999999998,"15min":0.60999999999999999},"min":{"1min":0.40000000000000002,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.75700000000000001,"5min":0.89200000000000002,"15min":0.89200000000000002},"last":0.53600000000000003}]},{"osd":3,"last update":"Fri Nov 24 07:58:25 2023","interfaces":[{"interface":"back","average":{"1min":0.54900000000000004,"5min":0.57699999999999996,"15min":0.58199999999999996},"min":{"1min":0.42899999999999999,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":0.65800000000000003,"5min":0.89700000000000002,"15min":0.89700000000000002},"last":0.52000000000000002},{"interface":"front","average":{"1min":0.57499999999999996,"5min":0.61899999999999999,"15min":0.627},"min":{"1min":0.442,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.72899999999999998,"5min":0.96199999999999997,"15min":0.96199999999999997},"last":0.67400000000000004}]},{"osd":5,"last update":"Fri Nov 24 07:58:25 2023","interfaces":[{"interface":"back","average":{"1min":0.48599999999999999,"5min":0.46100000000000002,"15min":0.45700000000000002},"min":{"1min":0.29499999999999998,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":0.67600000000000005,"5min":0.70499999999999996,"15min":0.70499999999999996},"last":0.35899999999999999},{"interface":"front","average":{"1min":0.59899999999999998,"5min":0.61499999999999999,"15min":0.61699999999999999},"min":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.746,"5min":0.86399999999999999,"15min":0.86399999999999999},"last":0.42899999999999999}]},{"osd":6,"last update":"Fri Nov 24 07:58:25 2023","interfaces":[{"interface":"back","average":{"1min":0.56799999999999995,"5min":0.58199999999999996,"15min":0.58399999999999996},"min":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.69799999999999995,"5min":0.76800000000000002,"15min":0.76800000000000002},"last":0.33600000000000002},{"interface":"front","average":{"1min":0.58999999999999997,"5min":0.60899999999999999,"15min":0.61199999999999999},"min":{"1min":0.46899999999999997,"5min":0.436,"15min":0.436},"max":{"1min":0.77700000000000002,"5min":1.095,"15min":1.095},"last":0.41199999999999998}]},{"osd":7,"last update":"Fri Nov 24 07:58:25 2023","interfaces":[{"interface":"back","average":{"1min":0.56799999999999995,"5min":0.57199999999999995,"15min":0.57199999999999995},"min":{"1min":0.39300000000000002,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.74199999999999999,"5min":0.89300000000000002,"15min":0.89300000000000002},"last":0.377},{"interface":"front","average":{"1min":0.61699999999999999,"5min":0.65300000000000002,"15min":0.65900000000000003},"min":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.79100000000000004,"5min":0.91500000000000004,"15min":0.91500000000000004},"last":0.39100000000000001}]}]},{"osd":5,"up_from":13,"seq":55834574888,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28448,"kb_used_data":1424,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343392,"statfs":{"total":96636764160,"available":96607633408,"internally_reserved":0,"allocated":1458176,"data_stored":955158,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 07:58:20 2023","interfaces":[{"interface":"back","average":{"1min":0.52300000000000002,"5min":0.55400000000000005,"15min":0.55900000000000005},"min":{"1min":0.35299999999999998,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.65800000000000003,"5min":1.0109999999999999,"15min":1.0109999999999999},"last":0.45500000000000002},{"interface":"front","average":{"1min":0.53200000000000003,"5min":0.746,"15min":0.78200000000000003},"min":{"1min":0.34399999999999997,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.72099999999999997,"5min":5.431,"15min":5.431},"last":0.34300000000000003}]},{"osd":1,"last update":"Fri Nov 24 07:58:20 2023","interfaces":[{"interface":"back","average":{"1min":0.51100000000000001,"5min":0.54900000000000004,"15min":0.55500000000000005},"min":{"1min":0.36399999999999999,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":0.67100000000000004,"5min":1.016,"15min":1.016},"last":0.41299999999999998},{"interface":"front","average":{"1min":0.54600000000000004,"5min":0.56499999999999995,"15min":0.56799999999999995},"min":{"1min":0.28599999999999998,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":0.78300000000000003,"5min":0.96499999999999997,"15min":0.96499999999999997},"last":0.47699999999999998}]},{"osd":2,"last update":"Fri Nov 24 07:58:20 2023","interfaces":[{"interface":"back","average":{"1min":0.54500000000000004,"5min":0.54900000000000004,"15min":0.55000000000000004},"min":{"1min":0.33700000000000002,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.67100000000000004,"5min":1.163,"15min":1.163},"last":0.46600000000000003},{"interface":"front","average":{"1min":0.54500000000000004,"5min":0.61299999999999999,"15min":0.624},"min":{"1min":0.41099999999999998,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.78900000000000003,"5min":1.119,"15min":1.119},"last":0.35899999999999999}]},{"osd":3,"last update":"Fri Nov 24 07:58:20 2023","interfaces":[{"interface":"back","average":{"1min":0.57199999999999995,"5min":0.83799999999999997,"15min":0.88200000000000001},"min":{"1min":0.36899999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.85899999999999999,"5min":5.609,"15min":5.609},"last":0.32400000000000001},{"interface":"front","average":{"1min":0.629,"5min":0.66600000000000004,"15min":0.67200000000000004},"min":{"1min":0.44600000000000001,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":0.76400000000000001,"5min":1.288,"15min":1.288},"last":0.39600000000000002}]},{"osd":4,"last update":"Fri Nov 24 07:58:20 2023","interfaces":[{"interface":"back","average":{"1min":0.51500000000000001,"5min":0.51100000000000001,"15min":0.51000000000000001},"min":{"1min":0.32100000000000001,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.76400000000000001,"5min":0.76400000000000001,"15min":0.76400000000000001},"last":0.30599999999999999},{"interface":"front","average":{"1min":0.58799999999999997,"5min":0.57499999999999996,"15min":0.57299999999999995},"min":{"1min":0.29599999999999999,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":0.92900000000000005,"5min":0.92900000000000005,"15min":0.92900000000000005},"last":0.378}]},{"osd":6,"last update":"Fri Nov 24 07:58:20 2023","interfaces":[{"interface":"back","average":{"1min":0.59599999999999997,"5min":0.57599999999999996,"15min":0.57199999999999995},"min":{"1min":0.35199999999999998,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.81799999999999995,"5min":0.89700000000000002,"15min":0.89700000000000002},"last":0.434},{"interface":"front","average":{"1min":0.63800000000000001,"5min":0.627,"15min":0.625},"min":{"1min":0.40300000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":1.083,"5min":1.083,"15min":1.083},"last":0.443}]},{"osd":7,"last update":"Fri Nov 24 07:58:20 2023","interfaces":[{"interface":"back","average":{"1min":0.66200000000000003,"5min":0.64900000000000002,"15min":0.64700000000000002},"min":{"1min":0.36799999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.90100000000000002,"5min":0.96599999999999997,"15min":0.96599999999999997},"last":0.42399999999999999},{"interface":"front","average":{"1min":0.66800000000000004,"5min":0.65600000000000003,"15min":0.65400000000000003},"min":{"1min":0.42699999999999999,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":0.85599999999999998,"5min":0.94199999999999995,"15min":0.94199999999999995},"last":0.49099999999999999}]}]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-11-24T07:59:19.187 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-11-24T07:59:19.348 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:19.346+0000 7f70944db700 1 Processor -- start 2023-11-24T07:59:19.348 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:19.346+0000 7f70944db700 1 -- start start 2023-11-24T07:59:19.350 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:19.347+0000 7f70944db700 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f708c15e1c0 0x7f708c06eef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:19.350 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:19.347+0000 7f70944db700 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f708c06fc50 0x7f708c06ca10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:19.350 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:19.347+0000 7f70944db700 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f708c0d1230 con 0x7f708c0b0e90 2023-11-24T07:59:19.350 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:19.347+0000 7f70944db700 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f708c0d0c70 con 0x7f708c06fc50 2023-11-24T07:59:19.350 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:19.347+0000 7f70944db700 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f708c0d0f50 con 0x7f708c15e1c0 2023-11-24T07:59:19.350 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:19.347+0000 7f7092a78700 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f708c06fc50 0x7f708c06ca10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:19.350 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:19.347+0000 7f7092a78700 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f708c06fc50 0x7f708c06ca10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.59:42090/0 (socket says 172.21.15.59:42090) 2023-11-24T07:59:19.350 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:19.347+0000 7f7092a78700 1 -- 172.21.15.59:0/211397109 learned_addr learned my addr 172.21.15.59:0/211397109 (peer_addr_for_me v2:172.21.15.59:0/0) 2023-11-24T07:59:19.350 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:19.347+0000 7f7092277700 1 --2- 172.21.15.59:0/211397109 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f708c15e1c0 0x7f708c06eef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:19.350 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:19.348+0000 7f7092a78700 1 -- 172.21.15.59:0/211397109 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f708c15e1c0 msgr2=0x7f708c06eef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T07:59:19.351 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:19.348+0000 7f7092a78700 1 --2- 172.21.15.59:0/211397109 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f708c15e1c0 0x7f708c06eef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:19.351 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:19.348+0000 7f7092a78700 1 -- 172.21.15.59:0/211397109 >> v1:172.21.15.76:6789/0 conn(0x7f708c0b0e90 legacy=0x7f708c06f540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:19.351 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:19.348+0000 7f7092a78700 1 -- 172.21.15.59:0/211397109 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f708c06c210 con 0x7f708c06fc50 2023-11-24T07:59:19.351 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:19.348+0000 7f7091275700 1 -- 172.21.15.59:0/211397109 <== mon.1 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 295358531 0 0) 0x7f708c0d1230 con 0x7f708c0b0e90 2023-11-24T07:59:19.351 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:19.348+0000 7f7092277700 1 --2- 172.21.15.59:0/211397109 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f708c15e1c0 0x7f708c06eef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-24T07:59:19.351 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:19.348+0000 7f7092a78700 1 --2- 172.21.15.59:0/211397109 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f708c06fc50 0x7f708c06ca10 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f7084009e50 tx=0x7f708400b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5b56eaa5e1342a46 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:59:19.351 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:19.348+0000 7f7091275700 1 -- 172.21.15.59:0/211397109 <== mon.1 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f708400e070 con 0x7f708c06fc50 2023-11-24T07:59:19.351 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:19.348+0000 7f7091275700 1 -- 172.21.15.59:0/211397109 <== mon.1 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f708400b910 con 0x7f708c06fc50 2023-11-24T07:59:19.351 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:19.348+0000 7f7091275700 1 -- 172.21.15.59:0/211397109 <== mon.1 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f708400bcc0 con 0x7f708c06fc50 2023-11-24T07:59:19.351 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:19.348+0000 7f70944db700 1 -- 172.21.15.59:0/211397109 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f708c06fc50 msgr2=0x7f708c06ca10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:19.351 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:19.348+0000 7f70944db700 1 --2- 172.21.15.59:0/211397109 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f708c06fc50 0x7f708c06ca10 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f7084009e50 tx=0x7f708400b330 comp rx=0 tx=0).stop 2023-11-24T07:59:19.354 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:19.349+0000 7f70944db700 1 -- 172.21.15.59:0/211397109 shutdown_connections 2023-11-24T07:59:19.354 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:19.349+0000 7f70944db700 1 --2- 172.21.15.59:0/211397109 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f708c06fc50 0x7f708c06ca10 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:19.354 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:19.349+0000 7f70944db700 1 --2- 172.21.15.59:0/211397109 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f708c15e1c0 0x7f708c06eef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:19.354 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:19.349+0000 7f70944db700 1 -- 172.21.15.59:0/211397109 >> 172.21.15.59:0/211397109 conn(0x7f708c0b3720 msgr2=0x7f708c0b3b30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:59:19.354 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:19.349+0000 7f70944db700 1 -- 172.21.15.59:0/211397109 shutdown_connections 2023-11-24T07:59:19.354 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:19.349+0000 7f70944db700 1 -- 172.21.15.59:0/211397109 wait complete. 2023-11-24T07:59:19.354 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:19.350+0000 7f70944db700 1 Processor -- start 2023-11-24T07:59:19.355 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:19.350+0000 7f70944db700 1 -- start start 2023-11-24T07:59:19.355 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:19.351+0000 7f70944db700 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f708c06fc50 0x7f708c156040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:19.355 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:19.351+0000 7f70944db700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f708c0b0e90 0x7f708c0b7070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:19.355 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:19.351+0000 7f70944db700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f708c15e1c0 0x7f708c0b75b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:19.355 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:19.351+0000 7f70944db700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f708c158740 con 0x7f708c0b0e90 2023-11-24T07:59:19.355 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:19.351+0000 7f70944db700 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f708c158c30 con 0x7f708c15e1c0 2023-11-24T07:59:19.355 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:19.351+0000 7f70944db700 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f708c157ea0 con 0x7f708c06fc50 2023-11-24T07:59:19.355 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:19.351+0000 7f7091a76700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f708c0b0e90 0x7f708c0b7070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:19.356 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:19.351+0000 7f7091a76700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f708c0b0e90 0x7f708c0b7070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:54896/0 (socket says 172.21.15.59:54896) 2023-11-24T07:59:19.356 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:19.351+0000 7f7091a76700 1 -- 172.21.15.59:0/2460657455 learned_addr learned my addr 172.21.15.59:0/2460657455 (peer_addr_for_me v2:172.21.15.59:0/0) 2023-11-24T07:59:19.356 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:19.351+0000 7f7092a78700 1 --2- 172.21.15.59:0/2460657455 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f708c15e1c0 0x7f708c0b75b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:19.356 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:19.351+0000 7f7092277700 1 --2- 172.21.15.59:0/2460657455 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f708c06fc50 0x7f708c156040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:19.356 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:19.351+0000 7f7091a76700 1 -- 172.21.15.59:0/2460657455 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f708c06fc50 msgr2=0x7f708c156040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:19.356 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:19.351+0000 7f7091a76700 1 --2- 172.21.15.59:0/2460657455 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f708c06fc50 0x7f708c156040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:19.356 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:19.351+0000 7f7091a76700 1 -- 172.21.15.59:0/2460657455 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f708c15e1c0 msgr2=0x7f708c0b75b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:19.356 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:19.351+0000 7f7091a76700 1 --2- 172.21.15.59:0/2460657455 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f708c15e1c0 0x7f708c0b75b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:19.356 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:19.351+0000 7f7091a76700 1 -- 172.21.15.59:0/2460657455 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f70880094a0 con 0x7f708c0b0e90 2023-11-24T07:59:19.356 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:19.351+0000 7f7091a76700 1 --2- 172.21.15.59:0/2460657455 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f708c0b0e90 0x7f708c0b7070 secure :-1 s=READY pgs=872 cs=0 l=1 rev1=1 crypto rx=0x7f707c0011e0 tx=0x7f707c00ca30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:59:19.356 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:19.351+0000 7f7082ffd700 1 -- 172.21.15.59:0/2460657455 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f707c0054e0 con 0x7f708c0b0e90 2023-11-24T07:59:19.356 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:19.351+0000 7f7082ffd700 1 -- 172.21.15.59:0/2460657455 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f707c003680 con 0x7f708c0b0e90 2023-11-24T07:59:19.357 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:19.351+0000 7f70944db700 1 -- 172.21.15.59:0/2460657455 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7084009b30 con 0x7f708c0b0e90 2023-11-24T07:59:19.357 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:19.351+0000 7f7082ffd700 1 -- 172.21.15.59:0/2460657455 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f707c003a30 con 0x7f708c0b0e90 2023-11-24T07:59:19.357 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:19.352+0000 7f70944db700 1 -- 172.21.15.59:0/2460657455 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f708c158460 con 0x7f708c0b0e90 2023-11-24T07:59:19.357 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:19.353+0000 7f70944db700 1 -- 172.21.15.59:0/2460657455 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f708c070190 con 0x7f708c0b0e90 2023-11-24T07:59:19.357 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:19.353+0000 7f7082ffd700 1 -- 172.21.15.59:0/2460657455 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 10) v1 ==== 89588+0+0 (secure 0 0 0) 0x7f707c003b90 con 0x7f708c0b0e90 2023-11-24T07:59:19.358 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:19.353+0000 7f7082ffd700 1 --2- 172.21.15.59:0/2460657455 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f707006b2c0 0x7f707006d780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:19.358 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:19.353+0000 7f7082ffd700 1 -- 172.21.15.59:0/2460657455 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f707c089170 con 0x7f708c0b0e90 2023-11-24T07:59:19.358 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:19.354+0000 7f7092277700 1 --2- 172.21.15.59:0/2460657455 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f707006b2c0 0x7f707006d780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:19.358 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:19.354+0000 7f7092277700 1 --2- 172.21.15.59:0/2460657455 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f707006b2c0 0x7f707006d780 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f708c1f7580 tx=0x7f7088009210 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:59:19.359 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:19.358+0000 7f7082ffd700 1 -- 172.21.15.59:0/2460657455 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f707c053c00 con 0x7f708c0b0e90 2023-11-24T07:59:19.556 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:19.554+0000 7f70944db700 1 -- 172.21.15.59:0/2460657455 --> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f708c15e5a0 con 0x7f707006b2c0 2023-11-24T07:59:19.564 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:19.562+0000 7f7082ffd700 1 -- 172.21.15.59:0/2460657455 <== mgr.4100 v2:172.21.15.76:6832/4264046536 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+277156 (secure 0 0 0) 0x7f7088009e90 con 0x7f707006b2c0 2023-11-24T07:59:19.567 INFO:teuthology.orchestra.run.smithi059.stdout: 2023-11-24T07:59:19.569 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:19.565+0000 7f70944db700 1 -- 172.21.15.59:0/2460657455 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f707006b2c0 msgr2=0x7f707006d780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:19.570 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:19.565+0000 7f70944db700 1 --2- 172.21.15.59:0/2460657455 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f707006b2c0 0x7f707006d780 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f708c1f7580 tx=0x7f7088009210 comp rx=0 tx=0).stop 2023-11-24T07:59:19.570 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:19.565+0000 7f70944db700 1 -- 172.21.15.59:0/2460657455 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f708c0b0e90 msgr2=0x7f708c0b7070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:19.570 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:19.565+0000 7f70944db700 1 --2- 172.21.15.59:0/2460657455 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f708c0b0e90 0x7f708c0b7070 secure :-1 s=READY pgs=872 cs=0 l=1 rev1=1 crypto rx=0x7f707c0011e0 tx=0x7f707c00ca30 comp rx=0 tx=0).stop 2023-11-24T07:59:19.570 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:19.566+0000 7f70944db700 1 -- 172.21.15.59:0/2460657455 shutdown_connections 2023-11-24T07:59:19.570 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:19.566+0000 7f70944db700 1 --2- 172.21.15.59:0/2460657455 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f707006b2c0 0x7f707006d780 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:19.570 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:19.566+0000 7f70944db700 1 --2- 172.21.15.59:0/2460657455 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f708c15e1c0 0x7f708c0b75b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:19.570 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:19.566+0000 7f70944db700 1 --2- 172.21.15.59:0/2460657455 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f708c06fc50 0x7f708c156040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:19.570 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:19.566+0000 7f70944db700 1 --2- 172.21.15.59:0/2460657455 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f708c0b0e90 0x7f708c0b7070 unknown :-1 s=CLOSED pgs=872 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:19.570 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:19.566+0000 7f70944db700 1 -- 172.21.15.59:0/2460657455 >> 172.21.15.59:0/2460657455 conn(0x7f708c0b3720 msgr2=0x7f708c0b2b40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:59:19.571 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:19.566+0000 7f70944db700 1 -- 172.21.15.59:0/2460657455 shutdown_connections 2023-11-24T07:59:19.571 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:19.566+0000 7f70944db700 1 -- 172.21.15.59:0/2460657455 wait complete. 2023-11-24T07:59:19.571 INFO:teuthology.orchestra.run.smithi059.stderr:dumped all 2023-11-24T07:59:19.580 INFO:teuthology.orchestra.run.smithi059.stdout:{"pg_ready":true,"pg_map":{"version":157,"stamp":"2023-11-24T07:59:18.339299+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":66,"acting":66,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":5,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":7,"kb":754974720,"kb_used":224084,"kb_used_data":7888,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754750636,"statfs":{"total":773094113280,"available":772864651264,"internally_reserved":0,"allocated":8077312,"data_stored":4087422,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12745,"internal_metadata":221236791},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":15,"apply_latency_ms":15,"commit_latency_ns":15000000,"apply_latency_ns":15000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.718179"},"pg_stats":[{"pgid":"11.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T07:59:13.620687+0000","last_change":"2023-11-24T07:59:13.620687+0000","last_active":"2023-11-24T07:59:13.620687+0000","last_peered":"2023-11-24T07:59:13.620687+0000","last_clean":"2023-11-24T07:59:13.620687+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T07:59:13.620687+0000","last_undegraded":"2023-11-24T07:59:13.620687+0000","last_fullsized":"2023-11-24T07:59:13.620687+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:59:13.620687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:59:13.620687+0000","last_clean_scrub_stamp":"2023-11-24T07:59:13.620687+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T07:59:13.620687+0000","last_change":"2023-11-24T07:59:13.620687+0000","last_active":"2023-11-24T07:59:13.620687+0000","last_peered":"2023-11-24T07:59:13.620687+0000","last_clean":"2023-11-24T07:59:13.620687+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T07:59:13.620687+0000","last_undegraded":"2023-11-24T07:59:13.620687+0000","last_fullsized":"2023-11-24T07:59:13.620687+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:59:13.620687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:59:13.620687+0000","last_clean_scrub_stamp":"2023-11-24T07:59:13.620687+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T07:59:13.620687+0000","last_change":"2023-11-24T07:59:13.620687+0000","last_active":"2023-11-24T07:59:13.620687+0000","last_peered":"2023-11-24T07:59:13.620687+0000","last_clean":"2023-11-24T07:59:13.620687+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T07:59:13.620687+0000","last_undegraded":"2023-11-24T07:59:13.620687+0000","last_fullsized":"2023-11-24T07:59:13.620687+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:59:13.620687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:59:13.620687+0000","last_clean_scrub_stamp":"2023-11-24T07:59:13.620687+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T07:59:13.620687+0000","last_change":"2023-11-24T07:59:13.620687+0000","last_active":"2023-11-24T07:59:13.620687+0000","last_peered":"2023-11-24T07:59:13.620687+0000","last_clean":"2023-11-24T07:59:13.620687+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T07:59:13.620687+0000","last_undegraded":"2023-11-24T07:59:13.620687+0000","last_fullsized":"2023-11-24T07:59:13.620687+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:59:13.620687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:59:13.620687+0000","last_clean_scrub_stamp":"2023-11-24T07:59:13.620687+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T07:59:13.620687+0000","last_change":"2023-11-24T07:59:13.620687+0000","last_active":"2023-11-24T07:59:13.620687+0000","last_peered":"2023-11-24T07:59:13.620687+0000","last_clean":"2023-11-24T07:59:13.620687+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T07:59:13.620687+0000","last_undegraded":"2023-11-24T07:59:13.620687+0000","last_fullsized":"2023-11-24T07:59:13.620687+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:59:13.620687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:59:13.620687+0000","last_clean_scrub_stamp":"2023-11-24T07:59:13.620687+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T07:59:13.620687+0000","last_change":"2023-11-24T07:59:13.620687+0000","last_active":"2023-11-24T07:59:13.620687+0000","last_peered":"2023-11-24T07:59:13.620687+0000","last_clean":"2023-11-24T07:59:13.620687+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T07:59:13.620687+0000","last_undegraded":"2023-11-24T07:59:13.620687+0000","last_fullsized":"2023-11-24T07:59:13.620687+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:59:13.620687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:59:13.620687+0000","last_clean_scrub_stamp":"2023-11-24T07:59:13.620687+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T07:59:13.620687+0000","last_change":"2023-11-24T07:59:13.620687+0000","last_active":"2023-11-24T07:59:13.620687+0000","last_peered":"2023-11-24T07:59:13.620687+0000","last_clean":"2023-11-24T07:59:13.620687+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T07:59:13.620687+0000","last_undegraded":"2023-11-24T07:59:13.620687+0000","last_fullsized":"2023-11-24T07:59:13.620687+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:59:13.620687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:59:13.620687+0000","last_clean_scrub_stamp":"2023-11-24T07:59:13.620687+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T07:59:13.620687+0000","last_change":"2023-11-24T07:59:13.620687+0000","last_active":"2023-11-24T07:59:13.620687+0000","last_peered":"2023-11-24T07:59:13.620687+0000","last_clean":"2023-11-24T07:59:13.620687+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T07:59:13.620687+0000","last_undegraded":"2023-11-24T07:59:13.620687+0000","last_fullsized":"2023-11-24T07:59:13.620687+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:59:13.620687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:59:13.620687+0000","last_clean_scrub_stamp":"2023-11-24T07:59:13.620687+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T07:59:13.620687+0000","last_change":"2023-11-24T07:59:13.620687+0000","last_active":"2023-11-24T07:59:13.620687+0000","last_peered":"2023-11-24T07:59:13.620687+0000","last_clean":"2023-11-24T07:59:13.620687+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T07:59:13.620687+0000","last_undegraded":"2023-11-24T07:59:13.620687+0000","last_fullsized":"2023-11-24T07:59:13.620687+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:59:13.620687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:59:13.620687+0000","last_clean_scrub_stamp":"2023-11-24T07:59:13.620687+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T07:59:13.620687+0000","last_change":"2023-11-24T07:59:13.620687+0000","last_active":"2023-11-24T07:59:13.620687+0000","last_peered":"2023-11-24T07:59:13.620687+0000","last_clean":"2023-11-24T07:59:13.620687+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T07:59:13.620687+0000","last_undegraded":"2023-11-24T07:59:13.620687+0000","last_fullsized":"2023-11-24T07:59:13.620687+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:59:13.620687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:59:13.620687+0000","last_clean_scrub_stamp":"2023-11-24T07:59:13.620687+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T07:59:13.620687+0000","last_change":"2023-11-24T07:59:13.620687+0000","last_active":"2023-11-24T07:59:13.620687+0000","last_peered":"2023-11-24T07:59:13.620687+0000","last_clean":"2023-11-24T07:59:13.620687+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T07:59:13.620687+0000","last_undegraded":"2023-11-24T07:59:13.620687+0000","last_fullsized":"2023-11-24T07:59:13.620687+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:59:13.620687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:59:13.620687+0000","last_clean_scrub_stamp":"2023-11-24T07:59:13.620687+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T07:59:13.620687+0000","last_change":"2023-11-24T07:59:13.620687+0000","last_active":"2023-11-24T07:59:13.620687+0000","last_peered":"2023-11-24T07:59:13.620687+0000","last_clean":"2023-11-24T07:59:13.620687+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T07:59:13.620687+0000","last_undegraded":"2023-11-24T07:59:13.620687+0000","last_fullsized":"2023-11-24T07:59:13.620687+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:59:13.620687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:59:13.620687+0000","last_clean_scrub_stamp":"2023-11-24T07:59:13.620687+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T07:59:13.620687+0000","last_change":"2023-11-24T07:59:13.620687+0000","last_active":"2023-11-24T07:59:13.620687+0000","last_peered":"2023-11-24T07:59:13.620687+0000","last_clean":"2023-11-24T07:59:13.620687+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T07:59:13.620687+0000","last_undegraded":"2023-11-24T07:59:13.620687+0000","last_fullsized":"2023-11-24T07:59:13.620687+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:59:13.620687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:59:13.620687+0000","last_clean_scrub_stamp":"2023-11-24T07:59:13.620687+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T07:59:13.620687+0000","last_change":"2023-11-24T07:59:13.620687+0000","last_active":"2023-11-24T07:59:13.620687+0000","last_peered":"2023-11-24T07:59:13.620687+0000","last_clean":"2023-11-24T07:59:13.620687+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T07:59:13.620687+0000","last_undegraded":"2023-11-24T07:59:13.620687+0000","last_fullsized":"2023-11-24T07:59:13.620687+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:59:13.620687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:59:13.620687+0000","last_clean_scrub_stamp":"2023-11-24T07:59:13.620687+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T07:59:13.620687+0000","last_change":"2023-11-24T07:59:13.620687+0000","last_active":"2023-11-24T07:59:13.620687+0000","last_peered":"2023-11-24T07:59:13.620687+0000","last_clean":"2023-11-24T07:59:13.620687+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T07:59:13.620687+0000","last_undegraded":"2023-11-24T07:59:13.620687+0000","last_fullsized":"2023-11-24T07:59:13.620687+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:59:13.620687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:59:13.620687+0000","last_clean_scrub_stamp":"2023-11-24T07:59:13.620687+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T07:59:13.620687+0000","last_change":"2023-11-24T07:59:13.620687+0000","last_active":"2023-11-24T07:59:13.620687+0000","last_peered":"2023-11-24T07:59:13.620687+0000","last_clean":"2023-11-24T07:59:13.620687+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T07:59:13.620687+0000","last_undegraded":"2023-11-24T07:59:13.620687+0000","last_fullsized":"2023-11-24T07:59:13.620687+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:59:13.620687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:59:13.620687+0000","last_clean_scrub_stamp":"2023-11-24T07:59:13.620687+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T07:59:13.620687+0000","last_change":"2023-11-24T07:59:13.620687+0000","last_active":"2023-11-24T07:59:13.620687+0000","last_peered":"2023-11-24T07:59:13.620687+0000","last_clean":"2023-11-24T07:59:13.620687+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T07:59:13.620687+0000","last_undegraded":"2023-11-24T07:59:13.620687+0000","last_fullsized":"2023-11-24T07:59:13.620687+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:59:13.620687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:59:13.620687+0000","last_clean_scrub_stamp":"2023-11-24T07:59:13.620687+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T07:59:13.620687+0000","last_change":"2023-11-24T07:59:13.620687+0000","last_active":"2023-11-24T07:59:13.620687+0000","last_peered":"2023-11-24T07:59:13.620687+0000","last_clean":"2023-11-24T07:59:13.620687+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T07:59:13.620687+0000","last_undegraded":"2023-11-24T07:59:13.620687+0000","last_fullsized":"2023-11-24T07:59:13.620687+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:59:13.620687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:59:13.620687+0000","last_clean_scrub_stamp":"2023-11-24T07:59:13.620687+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T07:59:13.620687+0000","last_change":"2023-11-24T07:59:13.620687+0000","last_active":"2023-11-24T07:59:13.620687+0000","last_peered":"2023-11-24T07:59:13.620687+0000","last_clean":"2023-11-24T07:59:13.620687+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T07:59:13.620687+0000","last_undegraded":"2023-11-24T07:59:13.620687+0000","last_fullsized":"2023-11-24T07:59:13.620687+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:59:13.620687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:59:13.620687+0000","last_clean_scrub_stamp":"2023-11-24T07:59:13.620687+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T07:59:13.620687+0000","last_change":"2023-11-24T07:59:13.620687+0000","last_active":"2023-11-24T07:59:13.620687+0000","last_peered":"2023-11-24T07:59:13.620687+0000","last_clean":"2023-11-24T07:59:13.620687+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T07:59:13.620687+0000","last_undegraded":"2023-11-24T07:59:13.620687+0000","last_fullsized":"2023-11-24T07:59:13.620687+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:59:13.620687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:59:13.620687+0000","last_clean_scrub_stamp":"2023-11-24T07:59:13.620687+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T07:59:13.620687+0000","last_change":"2023-11-24T07:59:13.620687+0000","last_active":"2023-11-24T07:59:13.620687+0000","last_peered":"2023-11-24T07:59:13.620687+0000","last_clean":"2023-11-24T07:59:13.620687+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T07:59:13.620687+0000","last_undegraded":"2023-11-24T07:59:13.620687+0000","last_fullsized":"2023-11-24T07:59:13.620687+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:59:13.620687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:59:13.620687+0000","last_clean_scrub_stamp":"2023-11-24T07:59:13.620687+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T07:59:13.620687+0000","last_change":"2023-11-24T07:59:13.620687+0000","last_active":"2023-11-24T07:59:13.620687+0000","last_peered":"2023-11-24T07:59:13.620687+0000","last_clean":"2023-11-24T07:59:13.620687+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T07:59:13.620687+0000","last_undegraded":"2023-11-24T07:59:13.620687+0000","last_fullsized":"2023-11-24T07:59:13.620687+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:59:13.620687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:59:13.620687+0000","last_clean_scrub_stamp":"2023-11-24T07:59:13.620687+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T07:59:13.620687+0000","last_change":"2023-11-24T07:59:13.620687+0000","last_active":"2023-11-24T07:59:13.620687+0000","last_peered":"2023-11-24T07:59:13.620687+0000","last_clean":"2023-11-24T07:59:13.620687+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T07:59:13.620687+0000","last_undegraded":"2023-11-24T07:59:13.620687+0000","last_fullsized":"2023-11-24T07:59:13.620687+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:59:13.620687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:59:13.620687+0000","last_clean_scrub_stamp":"2023-11-24T07:59:13.620687+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T07:59:13.620687+0000","last_change":"2023-11-24T07:59:13.620687+0000","last_active":"2023-11-24T07:59:13.620687+0000","last_peered":"2023-11-24T07:59:13.620687+0000","last_clean":"2023-11-24T07:59:13.620687+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T07:59:13.620687+0000","last_undegraded":"2023-11-24T07:59:13.620687+0000","last_fullsized":"2023-11-24T07:59:13.620687+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:59:13.620687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:59:13.620687+0000","last_clean_scrub_stamp":"2023-11-24T07:59:13.620687+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T07:59:13.620687+0000","last_change":"2023-11-24T07:59:13.620687+0000","last_active":"2023-11-24T07:59:13.620687+0000","last_peered":"2023-11-24T07:59:13.620687+0000","last_clean":"2023-11-24T07:59:13.620687+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T07:59:13.620687+0000","last_undegraded":"2023-11-24T07:59:13.620687+0000","last_fullsized":"2023-11-24T07:59:13.620687+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:59:13.620687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:59:13.620687+0000","last_clean_scrub_stamp":"2023-11-24T07:59:13.620687+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T07:59:13.620687+0000","last_change":"2023-11-24T07:59:13.620687+0000","last_active":"2023-11-24T07:59:13.620687+0000","last_peered":"2023-11-24T07:59:13.620687+0000","last_clean":"2023-11-24T07:59:13.620687+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T07:59:13.620687+0000","last_undegraded":"2023-11-24T07:59:13.620687+0000","last_fullsized":"2023-11-24T07:59:13.620687+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:59:13.620687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:59:13.620687+0000","last_clean_scrub_stamp":"2023-11-24T07:59:13.620687+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T07:59:13.620687+0000","last_change":"2023-11-24T07:59:13.620687+0000","last_active":"2023-11-24T07:59:13.620687+0000","last_peered":"2023-11-24T07:59:13.620687+0000","last_clean":"2023-11-24T07:59:13.620687+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T07:59:13.620687+0000","last_undegraded":"2023-11-24T07:59:13.620687+0000","last_fullsized":"2023-11-24T07:59:13.620687+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:59:13.620687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:59:13.620687+0000","last_clean_scrub_stamp":"2023-11-24T07:59:13.620687+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T07:59:13.620687+0000","last_change":"2023-11-24T07:59:13.620687+0000","last_active":"2023-11-24T07:59:13.620687+0000","last_peered":"2023-11-24T07:59:13.620687+0000","last_clean":"2023-11-24T07:59:13.620687+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T07:59:13.620687+0000","last_undegraded":"2023-11-24T07:59:13.620687+0000","last_fullsized":"2023-11-24T07:59:13.620687+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:59:13.620687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:59:13.620687+0000","last_clean_scrub_stamp":"2023-11-24T07:59:13.620687+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T07:59:13.620687+0000","last_change":"2023-11-24T07:59:13.620687+0000","last_active":"2023-11-24T07:59:13.620687+0000","last_peered":"2023-11-24T07:59:13.620687+0000","last_clean":"2023-11-24T07:59:13.620687+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T07:59:13.620687+0000","last_undegraded":"2023-11-24T07:59:13.620687+0000","last_fullsized":"2023-11-24T07:59:13.620687+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:59:13.620687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:59:13.620687+0000","last_clean_scrub_stamp":"2023-11-24T07:59:13.620687+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T07:59:13.620687+0000","last_change":"2023-11-24T07:59:13.620687+0000","last_active":"2023-11-24T07:59:13.620687+0000","last_peered":"2023-11-24T07:59:13.620687+0000","last_clean":"2023-11-24T07:59:13.620687+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T07:59:13.620687+0000","last_undegraded":"2023-11-24T07:59:13.620687+0000","last_fullsized":"2023-11-24T07:59:13.620687+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:59:13.620687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:59:13.620687+0000","last_clean_scrub_stamp":"2023-11-24T07:59:13.620687+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T07:59:13.620687+0000","last_change":"2023-11-24T07:59:13.620687+0000","last_active":"2023-11-24T07:59:13.620687+0000","last_peered":"2023-11-24T07:59:13.620687+0000","last_clean":"2023-11-24T07:59:13.620687+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T07:59:13.620687+0000","last_undegraded":"2023-11-24T07:59:13.620687+0000","last_fullsized":"2023-11-24T07:59:13.620687+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:59:13.620687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:59:13.620687+0000","last_clean_scrub_stamp":"2023-11-24T07:59:13.620687+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T07:59:13.620687+0000","last_change":"2023-11-24T07:59:13.620687+0000","last_active":"2023-11-24T07:59:13.620687+0000","last_peered":"2023-11-24T07:59:13.620687+0000","last_clean":"2023-11-24T07:59:13.620687+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T07:59:13.620687+0000","last_undegraded":"2023-11-24T07:59:13.620687+0000","last_fullsized":"2023-11-24T07:59:13.620687+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:59:13.620687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:59:13.620687+0000","last_clean_scrub_stamp":"2023-11-24T07:59:13.620687+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T07:59:13.620687+0000","last_change":"2023-11-24T07:59:13.620687+0000","last_active":"2023-11-24T07:59:13.620687+0000","last_peered":"2023-11-24T07:59:13.620687+0000","last_clean":"2023-11-24T07:59:13.620687+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T07:59:13.620687+0000","last_undegraded":"2023-11-24T07:59:13.620687+0000","last_fullsized":"2023-11-24T07:59:13.620687+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:59:13.620687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:59:13.620687+0000","last_clean_scrub_stamp":"2023-11-24T07:59:13.620687+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.1f","version":"0'0","reported_seq":1,"reported_epoch":71,"state":"creating+peering","last_fresh":"2023-11-24T07:59:12.627449+0000","last_change":"2023-11-24T07:59:12.627449+0000","last_active":"2023-11-24T07:59:12.615544+0000","last_peered":"2023-11-24T07:59:12.615544+0000","last_clean":"2023-11-24T07:59:12.615544+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T07:59:12.627449+0000","last_undegraded":"2023-11-24T07:59:12.627449+0000","last_fullsized":"2023-11-24T07:59:12.627449+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:59:12.615544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:59:12.615544+0000","last_clean_scrub_stamp":"2023-11-24T07:59:12.615544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T07:59:13.620687+0000","last_change":"2023-11-24T07:59:13.620687+0000","last_active":"2023-11-24T07:59:13.620687+0000","last_peered":"2023-11-24T07:59:13.620687+0000","last_clean":"2023-11-24T07:59:13.620687+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T07:59:13.620687+0000","last_undegraded":"2023-11-24T07:59:13.620687+0000","last_fullsized":"2023-11-24T07:59:13.620687+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:59:13.620687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:59:13.620687+0000","last_clean_scrub_stamp":"2023-11-24T07:59:13.620687+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.1e","version":"0'0","reported_seq":1,"reported_epoch":71,"state":"creating+peering","last_fresh":"2023-11-24T07:59:12.622955+0000","last_change":"2023-11-24T07:59:12.622955+0000","last_active":"2023-11-24T07:59:12.615544+0000","last_peered":"2023-11-24T07:59:12.615544+0000","last_clean":"2023-11-24T07:59:12.615544+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T07:59:12.622955+0000","last_undegraded":"2023-11-24T07:59:12.622955+0000","last_fullsized":"2023-11-24T07:59:12.622955+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:59:12.615544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:59:12.615544+0000","last_clean_scrub_stamp":"2023-11-24T07:59:12.615544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T07:59:13.620687+0000","last_change":"2023-11-24T07:59:13.620687+0000","last_active":"2023-11-24T07:59:13.620687+0000","last_peered":"2023-11-24T07:59:13.620687+0000","last_clean":"2023-11-24T07:59:13.620687+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T07:59:13.620687+0000","last_undegraded":"2023-11-24T07:59:13.620687+0000","last_fullsized":"2023-11-24T07:59:13.620687+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:59:13.620687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:59:13.620687+0000","last_clean_scrub_stamp":"2023-11-24T07:59:13.620687+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.1d","version":"0'0","reported_seq":1,"reported_epoch":71,"state":"creating+peering","last_fresh":"2023-11-24T07:59:12.632766+0000","last_change":"2023-11-24T07:59:12.632766+0000","last_active":"2023-11-24T07:59:12.615544+0000","last_peered":"2023-11-24T07:59:12.615544+0000","last_clean":"2023-11-24T07:59:12.615544+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T07:59:12.632766+0000","last_undegraded":"2023-11-24T07:59:12.632766+0000","last_fullsized":"2023-11-24T07:59:12.632766+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:59:12.615544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:59:12.615544+0000","last_clean_scrub_stamp":"2023-11-24T07:59:12.615544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T07:59:13.620687+0000","last_change":"2023-11-24T07:59:13.620687+0000","last_active":"2023-11-24T07:59:13.620687+0000","last_peered":"2023-11-24T07:59:13.620687+0000","last_clean":"2023-11-24T07:59:13.620687+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T07:59:13.620687+0000","last_undegraded":"2023-11-24T07:59:13.620687+0000","last_fullsized":"2023-11-24T07:59:13.620687+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:59:13.620687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:59:13.620687+0000","last_clean_scrub_stamp":"2023-11-24T07:59:13.620687+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.1c","version":"0'0","reported_seq":1,"reported_epoch":71,"state":"creating+peering","last_fresh":"2023-11-24T07:59:12.627772+0000","last_change":"2023-11-24T07:59:12.627772+0000","last_active":"2023-11-24T07:59:12.615544+0000","last_peered":"2023-11-24T07:59:12.615544+0000","last_clean":"2023-11-24T07:59:12.615544+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T07:59:12.627772+0000","last_undegraded":"2023-11-24T07:59:12.627772+0000","last_fullsized":"2023-11-24T07:59:12.627772+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:59:12.615544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:59:12.615544+0000","last_clean_scrub_stamp":"2023-11-24T07:59:12.615544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T07:59:13.620687+0000","last_change":"2023-11-24T07:59:13.620687+0000","last_active":"2023-11-24T07:59:13.620687+0000","last_peered":"2023-11-24T07:59:13.620687+0000","last_clean":"2023-11-24T07:59:13.620687+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T07:59:13.620687+0000","last_undegraded":"2023-11-24T07:59:13.620687+0000","last_fullsized":"2023-11-24T07:59:13.620687+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:59:13.620687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:59:13.620687+0000","last_clean_scrub_stamp":"2023-11-24T07:59:13.620687+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.1b","version":"0'0","reported_seq":1,"reported_epoch":71,"state":"creating+peering","last_fresh":"2023-11-24T07:59:12.626522+0000","last_change":"2023-11-24T07:59:12.626522+0000","last_active":"2023-11-24T07:59:12.615544+0000","last_peered":"2023-11-24T07:59:12.615544+0000","last_clean":"2023-11-24T07:59:12.615544+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T07:59:12.626522+0000","last_undegraded":"2023-11-24T07:59:12.626522+0000","last_fullsized":"2023-11-24T07:59:12.626522+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:59:12.615544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:59:12.615544+0000","last_clean_scrub_stamp":"2023-11-24T07:59:12.615544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T07:59:13.620687+0000","last_change":"2023-11-24T07:59:13.620687+0000","last_active":"2023-11-24T07:59:13.620687+0000","last_peered":"2023-11-24T07:59:13.620687+0000","last_clean":"2023-11-24T07:59:13.620687+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T07:59:13.620687+0000","last_undegraded":"2023-11-24T07:59:13.620687+0000","last_fullsized":"2023-11-24T07:59:13.620687+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:59:13.620687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:59:13.620687+0000","last_clean_scrub_stamp":"2023-11-24T07:59:13.620687+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.1a","version":"0'0","reported_seq":1,"reported_epoch":71,"state":"creating+peering","last_fresh":"2023-11-24T07:59:12.624736+0000","last_change":"2023-11-24T07:59:12.624736+0000","last_active":"2023-11-24T07:59:12.615544+0000","last_peered":"2023-11-24T07:59:12.615544+0000","last_clean":"2023-11-24T07:59:12.615544+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T07:59:12.624736+0000","last_undegraded":"2023-11-24T07:59:12.624736+0000","last_fullsized":"2023-11-24T07:59:12.624736+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:59:12.615544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:59:12.615544+0000","last_clean_scrub_stamp":"2023-11-24T07:59:12.615544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T07:59:13.620687+0000","last_change":"2023-11-24T07:59:13.620687+0000","last_active":"2023-11-24T07:59:13.620687+0000","last_peered":"2023-11-24T07:59:13.620687+0000","last_clean":"2023-11-24T07:59:13.620687+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T07:59:13.620687+0000","last_undegraded":"2023-11-24T07:59:13.620687+0000","last_fullsized":"2023-11-24T07:59:13.620687+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:59:13.620687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:59:13.620687+0000","last_clean_scrub_stamp":"2023-11-24T07:59:13.620687+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.19","version":"0'0","reported_seq":1,"reported_epoch":71,"state":"creating+peering","last_fresh":"2023-11-24T07:59:12.629097+0000","last_change":"2023-11-24T07:59:12.629097+0000","last_active":"2023-11-24T07:59:12.615544+0000","last_peered":"2023-11-24T07:59:12.615544+0000","last_clean":"2023-11-24T07:59:12.615544+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T07:59:12.629097+0000","last_undegraded":"2023-11-24T07:59:12.629097+0000","last_fullsized":"2023-11-24T07:59:12.629097+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:59:12.615544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:59:12.615544+0000","last_clean_scrub_stamp":"2023-11-24T07:59:12.615544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T07:59:13.620687+0000","last_change":"2023-11-24T07:59:13.620687+0000","last_active":"2023-11-24T07:59:13.620687+0000","last_peered":"2023-11-24T07:59:13.620687+0000","last_clean":"2023-11-24T07:59:13.620687+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T07:59:13.620687+0000","last_undegraded":"2023-11-24T07:59:13.620687+0000","last_fullsized":"2023-11-24T07:59:13.620687+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:59:13.620687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:59:13.620687+0000","last_clean_scrub_stamp":"2023-11-24T07:59:13.620687+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.18","version":"0'0","reported_seq":1,"reported_epoch":71,"state":"creating+peering","last_fresh":"2023-11-24T07:59:12.625986+0000","last_change":"2023-11-24T07:59:12.625986+0000","last_active":"2023-11-24T07:59:12.615544+0000","last_peered":"2023-11-24T07:59:12.615544+0000","last_clean":"2023-11-24T07:59:12.615544+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T07:59:12.625986+0000","last_undegraded":"2023-11-24T07:59:12.625986+0000","last_fullsized":"2023-11-24T07:59:12.625986+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:59:12.615544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:59:12.615544+0000","last_clean_scrub_stamp":"2023-11-24T07:59:12.615544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T07:59:13.620687+0000","last_change":"2023-11-24T07:59:13.620687+0000","last_active":"2023-11-24T07:59:13.620687+0000","last_peered":"2023-11-24T07:59:13.620687+0000","last_clean":"2023-11-24T07:59:13.620687+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T07:59:13.620687+0000","last_undegraded":"2023-11-24T07:59:13.620687+0000","last_fullsized":"2023-11-24T07:59:13.620687+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:59:13.620687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:59:13.620687+0000","last_clean_scrub_stamp":"2023-11-24T07:59:13.620687+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.17","version":"0'0","reported_seq":1,"reported_epoch":71,"state":"creating+peering","last_fresh":"2023-11-24T07:59:12.633815+0000","last_change":"2023-11-24T07:59:12.633815+0000","last_active":"2023-11-24T07:59:12.615544+0000","last_peered":"2023-11-24T07:59:12.615544+0000","last_clean":"2023-11-24T07:59:12.615544+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T07:59:12.633815+0000","last_undegraded":"2023-11-24T07:59:12.633815+0000","last_fullsized":"2023-11-24T07:59:12.633815+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:59:12.615544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:59:12.615544+0000","last_clean_scrub_stamp":"2023-11-24T07:59:12.615544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T07:59:13.620687+0000","last_change":"2023-11-24T07:59:13.620687+0000","last_active":"2023-11-24T07:59:13.620687+0000","last_peered":"2023-11-24T07:59:13.620687+0000","last_clean":"2023-11-24T07:59:13.620687+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T07:59:13.620687+0000","last_undegraded":"2023-11-24T07:59:13.620687+0000","last_fullsized":"2023-11-24T07:59:13.620687+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:59:13.620687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:59:13.620687+0000","last_clean_scrub_stamp":"2023-11-24T07:59:13.620687+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.16","version":"0'0","reported_seq":1,"reported_epoch":71,"state":"creating+peering","last_fresh":"2023-11-24T07:59:12.627824+0000","last_change":"2023-11-24T07:59:12.627824+0000","last_active":"2023-11-24T07:59:12.615544+0000","last_peered":"2023-11-24T07:59:12.615544+0000","last_clean":"2023-11-24T07:59:12.615544+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T07:59:12.627824+0000","last_undegraded":"2023-11-24T07:59:12.627824+0000","last_fullsized":"2023-11-24T07:59:12.627824+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:59:12.615544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:59:12.615544+0000","last_clean_scrub_stamp":"2023-11-24T07:59:12.615544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T07:59:13.620687+0000","last_change":"2023-11-24T07:59:13.620687+0000","last_active":"2023-11-24T07:59:13.620687+0000","last_peered":"2023-11-24T07:59:13.620687+0000","last_clean":"2023-11-24T07:59:13.620687+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T07:59:13.620687+0000","last_undegraded":"2023-11-24T07:59:13.620687+0000","last_fullsized":"2023-11-24T07:59:13.620687+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:59:13.620687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:59:13.620687+0000","last_clean_scrub_stamp":"2023-11-24T07:59:13.620687+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.15","version":"0'0","reported_seq":1,"reported_epoch":71,"state":"creating+peering","last_fresh":"2023-11-24T07:59:12.626663+0000","last_change":"2023-11-24T07:59:12.626663+0000","last_active":"2023-11-24T07:59:12.615544+0000","last_peered":"2023-11-24T07:59:12.615544+0000","last_clean":"2023-11-24T07:59:12.615544+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T07:59:12.626663+0000","last_undegraded":"2023-11-24T07:59:12.626663+0000","last_fullsized":"2023-11-24T07:59:12.626663+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:59:12.615544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:59:12.615544+0000","last_clean_scrub_stamp":"2023-11-24T07:59:12.615544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T07:59:13.620687+0000","last_change":"2023-11-24T07:59:13.620687+0000","last_active":"2023-11-24T07:59:13.620687+0000","last_peered":"2023-11-24T07:59:13.620687+0000","last_clean":"2023-11-24T07:59:13.620687+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T07:59:13.620687+0000","last_undegraded":"2023-11-24T07:59:13.620687+0000","last_fullsized":"2023-11-24T07:59:13.620687+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:59:13.620687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:59:13.620687+0000","last_clean_scrub_stamp":"2023-11-24T07:59:13.620687+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.14","version":"0'0","reported_seq":1,"reported_epoch":71,"state":"creating+peering","last_fresh":"2023-11-24T07:59:12.629104+0000","last_change":"2023-11-24T07:59:12.629104+0000","last_active":"2023-11-24T07:59:12.615544+0000","last_peered":"2023-11-24T07:59:12.615544+0000","last_clean":"2023-11-24T07:59:12.615544+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T07:59:12.629104+0000","last_undegraded":"2023-11-24T07:59:12.629104+0000","last_fullsized":"2023-11-24T07:59:12.629104+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:59:12.615544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:59:12.615544+0000","last_clean_scrub_stamp":"2023-11-24T07:59:12.615544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T07:59:13.620687+0000","last_change":"2023-11-24T07:59:13.620687+0000","last_active":"2023-11-24T07:59:13.620687+0000","last_peered":"2023-11-24T07:59:13.620687+0000","last_clean":"2023-11-24T07:59:13.620687+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T07:59:13.620687+0000","last_undegraded":"2023-11-24T07:59:13.620687+0000","last_fullsized":"2023-11-24T07:59:13.620687+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:59:13.620687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:59:13.620687+0000","last_clean_scrub_stamp":"2023-11-24T07:59:13.620687+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.13","version":"0'0","reported_seq":1,"reported_epoch":71,"state":"creating+peering","last_fresh":"2023-11-24T07:59:12.629097+0000","last_change":"2023-11-24T07:59:12.629097+0000","last_active":"2023-11-24T07:59:12.615544+0000","last_peered":"2023-11-24T07:59:12.615544+0000","last_clean":"2023-11-24T07:59:12.615544+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T07:59:12.629097+0000","last_undegraded":"2023-11-24T07:59:12.629097+0000","last_fullsized":"2023-11-24T07:59:12.629097+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:59:12.615544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:59:12.615544+0000","last_clean_scrub_stamp":"2023-11-24T07:59:12.615544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T07:59:13.620687+0000","last_change":"2023-11-24T07:59:13.620687+0000","last_active":"2023-11-24T07:59:13.620687+0000","last_peered":"2023-11-24T07:59:13.620687+0000","last_clean":"2023-11-24T07:59:13.620687+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T07:59:13.620687+0000","last_undegraded":"2023-11-24T07:59:13.620687+0000","last_fullsized":"2023-11-24T07:59:13.620687+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:59:13.620687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:59:13.620687+0000","last_clean_scrub_stamp":"2023-11-24T07:59:13.620687+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.12","version":"0'0","reported_seq":1,"reported_epoch":71,"state":"creating+peering","last_fresh":"2023-11-24T07:59:12.624085+0000","last_change":"2023-11-24T07:59:12.624085+0000","last_active":"2023-11-24T07:59:12.615544+0000","last_peered":"2023-11-24T07:59:12.615544+0000","last_clean":"2023-11-24T07:59:12.615544+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T07:59:12.624085+0000","last_undegraded":"2023-11-24T07:59:12.624085+0000","last_fullsized":"2023-11-24T07:59:12.624085+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:59:12.615544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:59:12.615544+0000","last_clean_scrub_stamp":"2023-11-24T07:59:12.615544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T07:59:13.620687+0000","last_change":"2023-11-24T07:59:13.620687+0000","last_active":"2023-11-24T07:59:13.620687+0000","last_peered":"2023-11-24T07:59:13.620687+0000","last_clean":"2023-11-24T07:59:13.620687+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T07:59:13.620687+0000","last_undegraded":"2023-11-24T07:59:13.620687+0000","last_fullsized":"2023-11-24T07:59:13.620687+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:59:13.620687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:59:13.620687+0000","last_clean_scrub_stamp":"2023-11-24T07:59:13.620687+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.11","version":"0'0","reported_seq":1,"reported_epoch":71,"state":"creating+peering","last_fresh":"2023-11-24T07:59:12.624082+0000","last_change":"2023-11-24T07:59:12.624082+0000","last_active":"2023-11-24T07:59:12.615544+0000","last_peered":"2023-11-24T07:59:12.615544+0000","last_clean":"2023-11-24T07:59:12.615544+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T07:59:12.624082+0000","last_undegraded":"2023-11-24T07:59:12.624082+0000","last_fullsized":"2023-11-24T07:59:12.624082+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:59:12.615544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:59:12.615544+0000","last_clean_scrub_stamp":"2023-11-24T07:59:12.615544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T07:59:13.620687+0000","last_change":"2023-11-24T07:59:13.620687+0000","last_active":"2023-11-24T07:59:13.620687+0000","last_peered":"2023-11-24T07:59:13.620687+0000","last_clean":"2023-11-24T07:59:13.620687+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T07:59:13.620687+0000","last_undegraded":"2023-11-24T07:59:13.620687+0000","last_fullsized":"2023-11-24T07:59:13.620687+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:59:13.620687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:59:13.620687+0000","last_clean_scrub_stamp":"2023-11-24T07:59:13.620687+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.10","version":"0'0","reported_seq":1,"reported_epoch":71,"state":"creating+peering","last_fresh":"2023-11-24T07:59:12.624170+0000","last_change":"2023-11-24T07:59:12.624170+0000","last_active":"2023-11-24T07:59:12.615544+0000","last_peered":"2023-11-24T07:59:12.615544+0000","last_clean":"2023-11-24T07:59:12.615544+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T07:59:12.624170+0000","last_undegraded":"2023-11-24T07:59:12.624170+0000","last_fullsized":"2023-11-24T07:59:12.624170+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:59:12.615544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:59:12.615544+0000","last_clean_scrub_stamp":"2023-11-24T07:59:12.615544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T07:59:13.620687+0000","last_change":"2023-11-24T07:59:13.620687+0000","last_active":"2023-11-24T07:59:13.620687+0000","last_peered":"2023-11-24T07:59:13.620687+0000","last_clean":"2023-11-24T07:59:13.620687+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T07:59:13.620687+0000","last_undegraded":"2023-11-24T07:59:13.620687+0000","last_fullsized":"2023-11-24T07:59:13.620687+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:59:13.620687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:59:13.620687+0000","last_clean_scrub_stamp":"2023-11-24T07:59:13.620687+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.f","version":"0'0","reported_seq":1,"reported_epoch":71,"state":"creating+peering","last_fresh":"2023-11-24T07:59:12.633408+0000","last_change":"2023-11-24T07:59:12.633408+0000","last_active":"2023-11-24T07:59:12.615544+0000","last_peered":"2023-11-24T07:59:12.615544+0000","last_clean":"2023-11-24T07:59:12.615544+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T07:59:12.633408+0000","last_undegraded":"2023-11-24T07:59:12.633408+0000","last_fullsized":"2023-11-24T07:59:12.633408+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:59:12.615544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:59:12.615544+0000","last_clean_scrub_stamp":"2023-11-24T07:59:12.615544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T07:59:13.620687+0000","last_change":"2023-11-24T07:59:13.620687+0000","last_active":"2023-11-24T07:59:13.620687+0000","last_peered":"2023-11-24T07:59:13.620687+0000","last_clean":"2023-11-24T07:59:13.620687+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T07:59:13.620687+0000","last_undegraded":"2023-11-24T07:59:13.620687+0000","last_fullsized":"2023-11-24T07:59:13.620687+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:59:13.620687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:59:13.620687+0000","last_clean_scrub_stamp":"2023-11-24T07:59:13.620687+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.e","version":"0'0","reported_seq":1,"reported_epoch":71,"state":"creating+peering","last_fresh":"2023-11-24T07:59:12.627298+0000","last_change":"2023-11-24T07:59:12.627298+0000","last_active":"2023-11-24T07:59:12.615544+0000","last_peered":"2023-11-24T07:59:12.615544+0000","last_clean":"2023-11-24T07:59:12.615544+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T07:59:12.627298+0000","last_undegraded":"2023-11-24T07:59:12.627298+0000","last_fullsized":"2023-11-24T07:59:12.627298+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:59:12.615544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:59:12.615544+0000","last_clean_scrub_stamp":"2023-11-24T07:59:12.615544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T07:59:13.620687+0000","last_change":"2023-11-24T07:59:13.620687+0000","last_active":"2023-11-24T07:59:13.620687+0000","last_peered":"2023-11-24T07:59:13.620687+0000","last_clean":"2023-11-24T07:59:13.620687+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T07:59:13.620687+0000","last_undegraded":"2023-11-24T07:59:13.620687+0000","last_fullsized":"2023-11-24T07:59:13.620687+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:59:13.620687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:59:13.620687+0000","last_clean_scrub_stamp":"2023-11-24T07:59:13.620687+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.d","version":"0'0","reported_seq":1,"reported_epoch":71,"state":"creating+peering","last_fresh":"2023-11-24T07:59:12.627664+0000","last_change":"2023-11-24T07:59:12.627664+0000","last_active":"2023-11-24T07:59:12.615544+0000","last_peered":"2023-11-24T07:59:12.615544+0000","last_clean":"2023-11-24T07:59:12.615544+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T07:59:12.627664+0000","last_undegraded":"2023-11-24T07:59:12.627664+0000","last_fullsized":"2023-11-24T07:59:12.627664+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:59:12.615544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:59:12.615544+0000","last_clean_scrub_stamp":"2023-11-24T07:59:12.615544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T07:59:13.620687+0000","last_change":"2023-11-24T07:59:13.620687+0000","last_active":"2023-11-24T07:59:13.620687+0000","last_peered":"2023-11-24T07:59:13.620687+0000","last_clean":"2023-11-24T07:59:13.620687+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T07:59:13.620687+0000","last_undegraded":"2023-11-24T07:59:13.620687+0000","last_fullsized":"2023-11-24T07:59:13.620687+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:59:13.620687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:59:13.620687+0000","last_clean_scrub_stamp":"2023-11-24T07:59:13.620687+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.c","version":"0'0","reported_seq":1,"reported_epoch":71,"state":"creating+peering","last_fresh":"2023-11-24T07:59:12.627260+0000","last_change":"2023-11-24T07:59:12.627260+0000","last_active":"2023-11-24T07:59:12.615544+0000","last_peered":"2023-11-24T07:59:12.615544+0000","last_clean":"2023-11-24T07:59:12.615544+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T07:59:12.627260+0000","last_undegraded":"2023-11-24T07:59:12.627260+0000","last_fullsized":"2023-11-24T07:59:12.627260+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:59:12.615544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:59:12.615544+0000","last_clean_scrub_stamp":"2023-11-24T07:59:12.615544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T07:59:13.620687+0000","last_change":"2023-11-24T07:59:13.620687+0000","last_active":"2023-11-24T07:59:13.620687+0000","last_peered":"2023-11-24T07:59:13.620687+0000","last_clean":"2023-11-24T07:59:13.620687+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T07:59:13.620687+0000","last_undegraded":"2023-11-24T07:59:13.620687+0000","last_fullsized":"2023-11-24T07:59:13.620687+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:59:13.620687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:59:13.620687+0000","last_clean_scrub_stamp":"2023-11-24T07:59:13.620687+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.a","version":"0'0","reported_seq":1,"reported_epoch":71,"state":"creating+peering","last_fresh":"2023-11-24T07:59:12.627164+0000","last_change":"2023-11-24T07:59:12.627164+0000","last_active":"2023-11-24T07:59:12.615544+0000","last_peered":"2023-11-24T07:59:12.615544+0000","last_clean":"2023-11-24T07:59:12.615544+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T07:59:12.627164+0000","last_undegraded":"2023-11-24T07:59:12.627164+0000","last_fullsized":"2023-11-24T07:59:12.627164+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:59:12.615544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:59:12.615544+0000","last_clean_scrub_stamp":"2023-11-24T07:59:12.615544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T07:59:13.620687+0000","last_change":"2023-11-24T07:59:13.620687+0000","last_active":"2023-11-24T07:59:13.620687+0000","last_peered":"2023-11-24T07:59:13.620687+0000","last_clean":"2023-11-24T07:59:13.620687+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T07:59:13.620687+0000","last_undegraded":"2023-11-24T07:59:13.620687+0000","last_fullsized":"2023-11-24T07:59:13.620687+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:59:13.620687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:59:13.620687+0000","last_clean_scrub_stamp":"2023-11-24T07:59:13.620687+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.9","version":"0'0","reported_seq":1,"reported_epoch":71,"state":"creating+peering","last_fresh":"2023-11-24T07:59:12.628577+0000","last_change":"2023-11-24T07:59:12.628577+0000","last_active":"2023-11-24T07:59:12.615544+0000","last_peered":"2023-11-24T07:59:12.615544+0000","last_clean":"2023-11-24T07:59:12.615544+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T07:59:12.628577+0000","last_undegraded":"2023-11-24T07:59:12.628577+0000","last_fullsized":"2023-11-24T07:59:12.628577+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:59:12.615544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:59:12.615544+0000","last_clean_scrub_stamp":"2023-11-24T07:59:12.615544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T07:59:13.620687+0000","last_change":"2023-11-24T07:59:13.620687+0000","last_active":"2023-11-24T07:59:13.620687+0000","last_peered":"2023-11-24T07:59:13.620687+0000","last_clean":"2023-11-24T07:59:13.620687+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T07:59:13.620687+0000","last_undegraded":"2023-11-24T07:59:13.620687+0000","last_fullsized":"2023-11-24T07:59:13.620687+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:59:13.620687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:59:13.620687+0000","last_clean_scrub_stamp":"2023-11-24T07:59:13.620687+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.8","version":"0'0","reported_seq":1,"reported_epoch":71,"state":"creating+peering","last_fresh":"2023-11-24T07:59:12.622958+0000","last_change":"2023-11-24T07:59:12.622958+0000","last_active":"2023-11-24T07:59:12.615544+0000","last_peered":"2023-11-24T07:59:12.615544+0000","last_clean":"2023-11-24T07:59:12.615544+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T07:59:12.622958+0000","last_undegraded":"2023-11-24T07:59:12.622958+0000","last_fullsized":"2023-11-24T07:59:12.622958+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:59:12.615544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:59:12.615544+0000","last_clean_scrub_stamp":"2023-11-24T07:59:12.615544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T07:59:13.620687+0000","last_change":"2023-11-24T07:59:13.620687+0000","last_active":"2023-11-24T07:59:13.620687+0000","last_peered":"2023-11-24T07:59:13.620687+0000","last_clean":"2023-11-24T07:59:13.620687+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T07:59:13.620687+0000","last_undegraded":"2023-11-24T07:59:13.620687+0000","last_fullsized":"2023-11-24T07:59:13.620687+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:59:13.620687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:59:13.620687+0000","last_clean_scrub_stamp":"2023-11-24T07:59:13.620687+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.7","version":"0'0","reported_seq":1,"reported_epoch":71,"state":"creating+peering","last_fresh":"2023-11-24T07:59:12.632816+0000","last_change":"2023-11-24T07:59:12.632816+0000","last_active":"2023-11-24T07:59:12.615544+0000","last_peered":"2023-11-24T07:59:12.615544+0000","last_clean":"2023-11-24T07:59:12.615544+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T07:59:12.632816+0000","last_undegraded":"2023-11-24T07:59:12.632816+0000","last_fullsized":"2023-11-24T07:59:12.632816+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:59:12.615544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:59:12.615544+0000","last_clean_scrub_stamp":"2023-11-24T07:59:12.615544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T07:59:13.620687+0000","last_change":"2023-11-24T07:59:13.620687+0000","last_active":"2023-11-24T07:59:13.620687+0000","last_peered":"2023-11-24T07:59:13.620687+0000","last_clean":"2023-11-24T07:59:13.620687+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T07:59:13.620687+0000","last_undegraded":"2023-11-24T07:59:13.620687+0000","last_fullsized":"2023-11-24T07:59:13.620687+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:59:13.620687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:59:13.620687+0000","last_clean_scrub_stamp":"2023-11-24T07:59:13.620687+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.6","version":"0'0","reported_seq":1,"reported_epoch":71,"state":"creating+peering","last_fresh":"2023-11-24T07:59:12.632780+0000","last_change":"2023-11-24T07:59:12.632780+0000","last_active":"2023-11-24T07:59:12.615544+0000","last_peered":"2023-11-24T07:59:12.615544+0000","last_clean":"2023-11-24T07:59:12.615544+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T07:59:12.632780+0000","last_undegraded":"2023-11-24T07:59:12.632780+0000","last_fullsized":"2023-11-24T07:59:12.632780+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:59:12.615544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:59:12.615544+0000","last_clean_scrub_stamp":"2023-11-24T07:59:12.615544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T07:59:13.620687+0000","last_change":"2023-11-24T07:59:13.620687+0000","last_active":"2023-11-24T07:59:13.620687+0000","last_peered":"2023-11-24T07:59:13.620687+0000","last_clean":"2023-11-24T07:59:13.620687+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T07:59:13.620687+0000","last_undegraded":"2023-11-24T07:59:13.620687+0000","last_fullsized":"2023-11-24T07:59:13.620687+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:59:13.620687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:59:13.620687+0000","last_clean_scrub_stamp":"2023-11-24T07:59:13.620687+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.5","version":"0'0","reported_seq":1,"reported_epoch":71,"state":"creating+peering","last_fresh":"2023-11-24T07:59:12.626089+0000","last_change":"2023-11-24T07:59:12.626089+0000","last_active":"2023-11-24T07:59:12.615544+0000","last_peered":"2023-11-24T07:59:12.615544+0000","last_clean":"2023-11-24T07:59:12.615544+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T07:59:12.626089+0000","last_undegraded":"2023-11-24T07:59:12.626089+0000","last_fullsized":"2023-11-24T07:59:12.626089+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:59:12.615544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:59:12.615544+0000","last_clean_scrub_stamp":"2023-11-24T07:59:12.615544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T07:59:13.620687+0000","last_change":"2023-11-24T07:59:13.620687+0000","last_active":"2023-11-24T07:59:13.620687+0000","last_peered":"2023-11-24T07:59:13.620687+0000","last_clean":"2023-11-24T07:59:13.620687+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T07:59:13.620687+0000","last_undegraded":"2023-11-24T07:59:13.620687+0000","last_fullsized":"2023-11-24T07:59:13.620687+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:59:13.620687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:59:13.620687+0000","last_clean_scrub_stamp":"2023-11-24T07:59:13.620687+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.4","version":"0'0","reported_seq":1,"reported_epoch":71,"state":"creating+peering","last_fresh":"2023-11-24T07:59:12.626088+0000","last_change":"2023-11-24T07:59:12.626088+0000","last_active":"2023-11-24T07:59:12.615544+0000","last_peered":"2023-11-24T07:59:12.615544+0000","last_clean":"2023-11-24T07:59:12.615544+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T07:59:12.626088+0000","last_undegraded":"2023-11-24T07:59:12.626088+0000","last_fullsized":"2023-11-24T07:59:12.626088+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:59:12.615544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:59:12.615544+0000","last_clean_scrub_stamp":"2023-11-24T07:59:12.615544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T07:59:13.620687+0000","last_change":"2023-11-24T07:59:13.620687+0000","last_active":"2023-11-24T07:59:13.620687+0000","last_peered":"2023-11-24T07:59:13.620687+0000","last_clean":"2023-11-24T07:59:13.620687+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T07:59:13.620687+0000","last_undegraded":"2023-11-24T07:59:13.620687+0000","last_fullsized":"2023-11-24T07:59:13.620687+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:59:13.620687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:59:13.620687+0000","last_clean_scrub_stamp":"2023-11-24T07:59:13.620687+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.3","version":"0'0","reported_seq":1,"reported_epoch":71,"state":"creating+peering","last_fresh":"2023-11-24T07:59:12.626018+0000","last_change":"2023-11-24T07:59:12.626018+0000","last_active":"2023-11-24T07:59:12.615544+0000","last_peered":"2023-11-24T07:59:12.615544+0000","last_clean":"2023-11-24T07:59:12.615544+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T07:59:12.626018+0000","last_undegraded":"2023-11-24T07:59:12.626018+0000","last_fullsized":"2023-11-24T07:59:12.626018+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:59:12.615544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:59:12.615544+0000","last_clean_scrub_stamp":"2023-11-24T07:59:12.615544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T07:59:13.620687+0000","last_change":"2023-11-24T07:59:13.620687+0000","last_active":"2023-11-24T07:59:13.620687+0000","last_peered":"2023-11-24T07:59:13.620687+0000","last_clean":"2023-11-24T07:59:13.620687+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T07:59:13.620687+0000","last_undegraded":"2023-11-24T07:59:13.620687+0000","last_fullsized":"2023-11-24T07:59:13.620687+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:59:13.620687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:59:13.620687+0000","last_clean_scrub_stamp":"2023-11-24T07:59:13.620687+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.2","version":"0'0","reported_seq":1,"reported_epoch":71,"state":"creating+peering","last_fresh":"2023-11-24T07:59:12.622957+0000","last_change":"2023-11-24T07:59:12.622957+0000","last_active":"2023-11-24T07:59:12.615544+0000","last_peered":"2023-11-24T07:59:12.615544+0000","last_clean":"2023-11-24T07:59:12.615544+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T07:59:12.622957+0000","last_undegraded":"2023-11-24T07:59:12.622957+0000","last_fullsized":"2023-11-24T07:59:12.622957+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:59:12.615544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:59:12.615544+0000","last_clean_scrub_stamp":"2023-11-24T07:59:12.615544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T07:59:13.620687+0000","last_change":"2023-11-24T07:59:13.620687+0000","last_active":"2023-11-24T07:59:13.620687+0000","last_peered":"2023-11-24T07:59:13.620687+0000","last_clean":"2023-11-24T07:59:13.620687+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T07:59:13.620687+0000","last_undegraded":"2023-11-24T07:59:13.620687+0000","last_fullsized":"2023-11-24T07:59:13.620687+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:59:13.620687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:59:13.620687+0000","last_clean_scrub_stamp":"2023-11-24T07:59:13.620687+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.1","version":"0'0","reported_seq":1,"reported_epoch":71,"state":"creating+peering","last_fresh":"2023-11-24T07:59:12.628166+0000","last_change":"2023-11-24T07:59:12.628166+0000","last_active":"2023-11-24T07:59:12.615544+0000","last_peered":"2023-11-24T07:59:12.615544+0000","last_clean":"2023-11-24T07:59:12.615544+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T07:59:12.628166+0000","last_undegraded":"2023-11-24T07:59:12.628166+0000","last_fullsized":"2023-11-24T07:59:12.628166+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:59:12.615544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:59:12.615544+0000","last_clean_scrub_stamp":"2023-11-24T07:59:12.615544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T07:59:13.620687+0000","last_change":"2023-11-24T07:59:13.620687+0000","last_active":"2023-11-24T07:59:13.620687+0000","last_peered":"2023-11-24T07:59:13.620687+0000","last_clean":"2023-11-24T07:59:13.620687+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T07:59:13.620687+0000","last_undegraded":"2023-11-24T07:59:13.620687+0000","last_fullsized":"2023-11-24T07:59:13.620687+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:59:13.620687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:59:13.620687+0000","last_clean_scrub_stamp":"2023-11-24T07:59:13.620687+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.0","version":"0'0","reported_seq":1,"reported_epoch":71,"state":"creating+peering","last_fresh":"2023-11-24T07:59:12.627478+0000","last_change":"2023-11-24T07:59:12.627478+0000","last_active":"2023-11-24T07:59:12.615544+0000","last_peered":"2023-11-24T07:59:12.615544+0000","last_clean":"2023-11-24T07:59:12.615544+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T07:59:12.627478+0000","last_undegraded":"2023-11-24T07:59:12.627478+0000","last_fullsized":"2023-11-24T07:59:12.627478+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:59:12.615544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:59:12.615544+0000","last_clean_scrub_stamp":"2023-11-24T07:59:12.615544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T07:59:13.620687+0000","last_change":"2023-11-24T07:59:13.620687+0000","last_active":"2023-11-24T07:59:13.620687+0000","last_peered":"2023-11-24T07:59:13.620687+0000","last_clean":"2023-11-24T07:59:13.620687+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T07:59:13.620687+0000","last_undegraded":"2023-11-24T07:59:13.620687+0000","last_fullsized":"2023-11-24T07:59:13.620687+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:59:13.620687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:59:13.620687+0000","last_clean_scrub_stamp":"2023-11-24T07:59:13.620687+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.b","version":"0'0","reported_seq":1,"reported_epoch":71,"state":"creating+peering","last_fresh":"2023-11-24T07:59:12.624115+0000","last_change":"2023-11-24T07:59:12.624115+0000","last_active":"2023-11-24T07:59:12.615544+0000","last_peered":"2023-11-24T07:59:12.615544+0000","last_clean":"2023-11-24T07:59:12.615544+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T07:59:12.624115+0000","last_undegraded":"2023-11-24T07:59:12.624115+0000","last_fullsized":"2023-11-24T07:59:12.624115+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:59:12.615544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:59:12.615544+0000","last_clean_scrub_stamp":"2023-11-24T07:59:12.615544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":208,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-11-24T07:59:09.991344+0000","last_change":"2023-11-24T07:56:19.668905+0000","last_active":"2023-11-24T07:59:09.991344+0000","last_peered":"2023-11-24T07:59:09.991344+0000","last_clean":"2023-11-24T07:59:09.991344+0000","last_became_active":"2023-11-24T07:56:19.668601+0000","last_became_peered":"2023-11-24T07:56:19.668601+0000","last_unstale":"2023-11-24T07:59:09.991344+0000","last_undegraded":"2023-11-24T07:59:09.991344+0000","last_fullsized":"2023-11-24T07:59:09.991344+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:56:18.657844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:56:18.657844+0000","last_clean_scrub_stamp":"2023-11-24T07:56:18.657844+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T11:51:53.828455+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":11,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":10,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574886,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27872,"kb_used_data":844,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343968,"statfs":{"total":96636764160,"available":96608223232,"internally_reserved":0,"allocated":864256,"data_stored":364790,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 07:58:20 2023","interfaces":[{"interface":"back","average":{"1min":0.51400000000000001,"5min":0.54800000000000004,"15min":0.55400000000000005},"min":{"1min":0.309,"5min":0.309,"15min":0.309},"max":{"1min":0.78500000000000003,"5min":1.413,"15min":1.413},"last":0.55200000000000005},{"interface":"front","average":{"1min":0.56299999999999994,"5min":0.55500000000000005,"15min":0.55400000000000005},"min":{"1min":0.436,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":0.77000000000000002,"5min":0.81000000000000005,"15min":0.81000000000000005},"last":0.56599999999999995}]},{"osd":1,"last update":"Fri Nov 24 07:58:20 2023","interfaces":[{"interface":"back","average":{"1min":0.501,"5min":0.77900000000000003,"15min":0.82499999999999996},"min":{"1min":0.32000000000000001,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.69899999999999995,"5min":6.1070000000000002,"15min":6.1070000000000002},"last":0.53100000000000003},{"interface":"front","average":{"1min":0.51300000000000001,"5min":0.56599999999999995,"15min":0.57499999999999996},"min":{"1min":0.40699999999999997,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.73599999999999999,"5min":1.5920000000000001,"15min":1.5920000000000001},"last":0.49399999999999999}]},{"osd":2,"last update":"Fri Nov 24 07:58:20 2023","interfaces":[{"interface":"back","average":{"1min":0.498,"5min":0.60099999999999998,"15min":0.61799999999999999},"min":{"1min":0.35099999999999998,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.71599999999999997,"5min":1.613,"15min":1.613},"last":0.40100000000000002},{"interface":"front","average":{"1min":0.502,"5min":0.73399999999999999,"15min":0.77200000000000002},"min":{"1min":0.34899999999999998,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.71299999999999997,"5min":4.5220000000000002,"15min":4.5220000000000002},"last":0.44700000000000001}]},{"osd":3,"last update":"Fri Nov 24 07:58:20 2023","interfaces":[{"interface":"back","average":{"1min":0.55100000000000005,"5min":0.755,"15min":0.78900000000000003},"min":{"1min":0.41699999999999998,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.754,"5min":4.6760000000000002,"15min":4.6760000000000002},"last":0.626},{"interface":"front","average":{"1min":0.60699999999999998,"5min":0.875,"15min":0.92000000000000004},"min":{"1min":0.38500000000000001,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.81599999999999995,"5min":4.6539999999999999,"15min":4.6539999999999999},"last":0.64400000000000002}]},{"osd":4,"last update":"Fri Nov 24 07:58:20 2023","interfaces":[{"interface":"back","average":{"1min":0.54900000000000004,"5min":0.56399999999999995,"15min":0.56599999999999995},"min":{"1min":0.41799999999999998,"5min":0.378,"15min":0.378},"max":{"1min":0.70999999999999996,"5min":0.84699999999999998,"15min":0.84699999999999998},"last":0.42699999999999999},{"interface":"front","average":{"1min":0.54700000000000004,"5min":0.64300000000000002,"15min":0.65900000000000003},"min":{"1min":0.35199999999999998,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.79900000000000004,"5min":1.629,"15min":1.629},"last":0.59499999999999997}]},{"osd":5,"last update":"Fri Nov 24 07:58:20 2023","interfaces":[{"interface":"back","average":{"1min":0.54200000000000004,"5min":0.54600000000000004,"15min":0.54600000000000004},"min":{"1min":0.41599999999999998,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.67900000000000005,"5min":0.875,"15min":0.875},"last":0.51600000000000001},{"interface":"front","average":{"1min":0.57799999999999996,"5min":0.61399999999999999,"15min":0.62},"min":{"1min":0.36799999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.77700000000000002,"5min":0.90900000000000003,"15min":0.90900000000000003},"last":0.57999999999999996}]},{"osd":6,"last update":"Fri Nov 24 07:58:20 2023","interfaces":[{"interface":"back","average":{"1min":0.59999999999999998,"5min":0.60799999999999998,"15min":0.60999999999999999},"min":{"1min":0.49099999999999999,"5min":0.438,"15min":0.438},"max":{"1min":0.73399999999999999,"5min":0.80800000000000005,"15min":0.80800000000000005},"last":0.61299999999999999},{"interface":"front","average":{"1min":0.63900000000000001,"5min":0.66300000000000003,"15min":0.66700000000000004},"min":{"1min":0.45600000000000002,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":0.79700000000000004,"5min":0.98399999999999999,"15min":0.98399999999999999},"last":0.66100000000000003}]}]},{"osd":6,"up_from":13,"seq":55834574887,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27872,"kb_used_data":844,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343968,"statfs":{"total":96636764160,"available":96608223232,"internally_reserved":0,"allocated":864256,"data_stored":364790,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 07:58:19 2023","interfaces":[{"interface":"back","average":{"1min":0.53300000000000003,"5min":0.54600000000000004,"15min":0.54800000000000004},"min":{"1min":0.29799999999999999,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":0.93100000000000005,"5min":0.93100000000000005,"15min":0.93100000000000005},"last":0.497},{"interface":"front","average":{"1min":0.51600000000000001,"5min":0.51600000000000001,"15min":0.51600000000000001},"min":{"1min":0.311,"5min":0.311,"15min":0.311},"max":{"1min":0.82199999999999995,"5min":0.82199999999999995,"15min":0.82199999999999995},"last":0.34899999999999998}]},{"osd":1,"last update":"Fri Nov 24 07:58:19 2023","interfaces":[{"interface":"back","average":{"1min":0.51900000000000002,"5min":0.51300000000000001,"15min":0.51200000000000001},"min":{"1min":0.35599999999999998,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.81299999999999994,"5min":0.81299999999999994,"15min":0.81299999999999994},"last":0.48399999999999999},{"interface":"front","average":{"1min":0.54000000000000004,"5min":0.59799999999999998,"15min":0.60799999999999998},"min":{"1min":0.35999999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.82799999999999996,"5min":1.0069999999999999,"15min":1.0069999999999999},"last":0.50900000000000001}]},{"osd":2,"last update":"Fri Nov 24 07:58:19 2023","interfaces":[{"interface":"back","average":{"1min":0.58399999999999996,"5min":0.57599999999999996,"15min":0.57499999999999996},"min":{"1min":0.34899999999999998,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.90600000000000003,"5min":0.90600000000000003,"15min":0.90600000000000003},"last":0.52600000000000002},{"interface":"front","average":{"1min":0.56499999999999995,"5min":0.58699999999999997,"15min":0.59099999999999997},"min":{"1min":0.34300000000000003,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.84899999999999998,"5min":0.84899999999999998,"15min":0.84899999999999998},"last":0.53700000000000003}]},{"osd":3,"last update":"Fri Nov 24 07:58:19 2023","interfaces":[{"interface":"back","average":{"1min":0.56000000000000005,"5min":0.56599999999999995,"15min":0.56699999999999995},"min":{"1min":0.38400000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.86499999999999999,"5min":0.876,"15min":0.876},"last":0.40100000000000002},{"interface":"front","average":{"1min":0.57699999999999996,"5min":0.56000000000000005,"15min":0.55700000000000005},"min":{"1min":0.39800000000000002,"5min":0.379,"15min":0.379},"max":{"1min":0.876,"5min":0.876,"15min":0.876},"last":0.41699999999999998}]},{"osd":4,"last update":"Fri Nov 24 07:58:19 2023","interfaces":[{"interface":"back","average":{"1min":0.53100000000000003,"5min":0.51900000000000002,"15min":0.51700000000000002},"min":{"1min":0.25,"5min":0.25,"15min":0.25},"max":{"1min":0.89400000000000002,"5min":0.89400000000000002,"15min":0.89400000000000002},"last":0.58299999999999996},{"interface":"front","average":{"1min":0.58899999999999997,"5min":0.59899999999999998,"15min":0.60099999999999998},"min":{"1min":0.29099999999999998,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":1.044,"5min":1.087,"15min":1.087},"last":0.30599999999999999}]},{"osd":5,"last update":"Fri Nov 24 07:58:19 2023","interfaces":[{"interface":"back","average":{"1min":0.59499999999999997,"5min":0.54300000000000004,"15min":0.53500000000000003},"min":{"1min":0.27800000000000002,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":0.97999999999999998,"5min":0.97999999999999998,"15min":0.97999999999999998},"last":0.37},{"interface":"front","average":{"1min":0.61399999999999999,"5min":0.61199999999999999,"15min":0.61199999999999999},"min":{"1min":0.31900000000000001,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":1.0009999999999999,"5min":1.0009999999999999,"15min":1.0009999999999999},"last":0.438}]},{"osd":7,"last update":"Fri Nov 24 07:58:19 2023","interfaces":[{"interface":"back","average":{"1min":0.61799999999999999,"5min":0.61799999999999999,"15min":0.61799999999999999},"min":{"1min":0.26500000000000001,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":1.0880000000000001,"5min":1.0880000000000001,"15min":1.0880000000000001},"last":0.46400000000000002},{"interface":"front","average":{"1min":0.65200000000000002,"5min":0.66600000000000004,"15min":0.66800000000000004},"min":{"1min":0.34300000000000003,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.95599999999999996,"5min":0.95599999999999996,"15min":0.95599999999999996},"last":0.32800000000000001}]}]},{"osd":1,"up_from":13,"seq":55834574888,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28432,"kb_used_data":1412,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343408,"statfs":{"total":96636764160,"available":96607649792,"internally_reserved":0,"allocated":1445888,"data_stored":949341,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 07:58:24 2023","interfaces":[{"interface":"back","average":{"1min":0.33000000000000002,"5min":0.35699999999999998,"15min":0.36099999999999999},"min":{"1min":0.249,"5min":0.20200000000000001,"15min":0.20200000000000001},"max":{"1min":0.48799999999999999,"5min":0.70899999999999996,"15min":0.70899999999999996},"last":0.249},{"interface":"front","average":{"1min":0.36399999999999999,"5min":0.39100000000000001,"15min":0.39500000000000002},"min":{"1min":0.23599999999999999,"5min":0.214,"15min":0.214},"max":{"1min":0.54700000000000004,"5min":0.82499999999999996,"15min":0.82499999999999996},"last":0.309}]},{"osd":2,"last update":"Fri Nov 24 07:58:24 2023","interfaces":[{"interface":"back","average":{"1min":0.312,"5min":0.35999999999999999,"15min":0.36799999999999999},"min":{"1min":0.253,"5min":0.22700000000000001,"15min":0.22700000000000001},"max":{"1min":0.44800000000000001,"5min":0.60999999999999999,"15min":0.60999999999999999},"last":0.40699999999999997},{"interface":"front","average":{"1min":0.36499999999999999,"5min":0.40300000000000002,"15min":0.40899999999999997},"min":{"1min":0.28100000000000003,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":0.47599999999999998,"5min":0.621,"15min":0.621},"last":0.28899999999999998}]},{"osd":3,"last update":"Fri Nov 24 07:58:24 2023","interfaces":[{"interface":"back","average":{"1min":0.40300000000000002,"5min":0.435,"15min":0.441},"min":{"1min":0.31900000000000001,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.59199999999999997,"5min":0.66500000000000004,"15min":0.66500000000000004},"last":0.34699999999999998},{"interface":"front","average":{"1min":0.38700000000000001,"5min":0.41899999999999998,"15min":0.42499999999999999},"min":{"1min":0.29199999999999998,"5min":0.28499999999999998,"15min":0.28499999999999998},"max":{"1min":0.56200000000000006,"5min":0.76300000000000001,"15min":0.76300000000000001},"last":0.36599999999999999}]},{"osd":4,"last update":"Fri Nov 24 07:58:24 2023","interfaces":[{"interface":"back","average":{"1min":0.56200000000000006,"5min":0.60099999999999998,"15min":0.60699999999999998},"min":{"1min":0.442,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":0.84099999999999997,"5min":0.84099999999999997,"15min":0.84099999999999997},"last":0.64200000000000002},{"interface":"front","average":{"1min":0.61099999999999999,"5min":0.60699999999999998,"15min":0.60599999999999998},"min":{"1min":0.499,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":0.79500000000000004,"5min":0.82199999999999995,"15min":0.82199999999999995},"last":0.67800000000000005}]},{"osd":5,"last update":"Fri Nov 24 07:58:24 2023","interfaces":[{"interface":"back","average":{"1min":0.59199999999999997,"5min":0.69799999999999995,"15min":0.71599999999999997},"min":{"1min":0.439,"5min":0.37,"15min":0.37},"max":{"1min":0.97599999999999998,"5min":2.7269999999999999,"15min":2.7269999999999999},"last":0.69399999999999995},{"interface":"front","average":{"1min":0.57699999999999996,"5min":0.63100000000000001,"15min":0.64000000000000001},"min":{"1min":0.39100000000000001,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.94499999999999995,"5min":0.94499999999999995,"15min":0.94499999999999995},"last":0.60999999999999999}]},{"osd":6,"last update":"Fri Nov 24 07:58:24 2023","interfaces":[{"interface":"back","average":{"1min":0.61199999999999999,"5min":0.64800000000000002,"15min":0.65400000000000003},"min":{"1min":0.47699999999999998,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":0.77500000000000002,"5min":0.98199999999999998,"15min":0.98199999999999998},"last":0.70299999999999996},{"interface":"front","average":{"1min":0.57599999999999996,"5min":0.60799999999999998,"15min":0.61399999999999999},"min":{"1min":0.40600000000000003,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":0.89600000000000002,"5min":0.90700000000000003,"15min":0.90700000000000003},"last":0.623}]},{"osd":7,"last update":"Fri Nov 24 07:58:24 2023","interfaces":[{"interface":"back","average":{"1min":0.63100000000000001,"5min":0.63500000000000001,"15min":0.63600000000000001},"min":{"1min":0.43099999999999999,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.79800000000000004,"5min":1.0649999999999999,"15min":1.0649999999999999},"last":0.71399999999999997},{"interface":"front","average":{"1min":0.57099999999999995,"5min":0.65000000000000002,"15min":0.66300000000000003},"min":{"1min":0.45700000000000002,"5min":0.442,"15min":0.442},"max":{"1min":0.71299999999999997,"5min":0.88800000000000001,"15min":0.88800000000000001},"last":0.66700000000000004}]}]},{"osd":0,"up_from":13,"seq":55834574888,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27856,"kb_used_data":832,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343984,"statfs":{"total":96636764160,"available":96608239616,"internally_reserved":0,"allocated":851968,"data_stored":358973,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Nov 24 07:58:20 2023","interfaces":[{"interface":"back","average":{"1min":0.38500000000000001,"5min":0.64700000000000002,"15min":0.69099999999999995},"min":{"1min":0.21299999999999999,"5min":0.21299999999999999,"15min":0.21299999999999999},"max":{"1min":0.625,"5min":6.3550000000000004,"15min":6.3550000000000004},"last":0.34000000000000002},{"interface":"front","average":{"1min":0.40799999999999997,"5min":1.464,"15min":1.6399999999999999},"min":{"1min":0.26000000000000001,"5min":0.26000000000000001,"15min":0.26000000000000001},"max":{"1min":0.60999999999999999,"5min":22.916,"15min":22.916},"last":0.29899999999999999}]},{"osd":2,"last update":"Fri Nov 24 07:58:20 2023","interfaces":[{"interface":"back","average":{"1min":0.35299999999999998,"5min":0.63,"15min":0.67600000000000005},"min":{"1min":0.22900000000000001,"5min":0.22800000000000001,"15min":0.22800000000000001},"max":{"1min":0.499,"5min":6.3899999999999997,"15min":6.3899999999999997},"last":0.27700000000000002},{"interface":"front","average":{"1min":0.39500000000000002,"5min":0.92900000000000005,"15min":1.018},"min":{"1min":0.27900000000000003,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":0.60199999999999998,"5min":11.93,"15min":11.93},"last":0.32800000000000001}]},{"osd":3,"last update":"Fri Nov 24 07:58:20 2023","interfaces":[{"interface":"back","average":{"1min":0.373,"5min":0.40999999999999998,"15min":0.41599999999999998},"min":{"1min":0.23799999999999999,"5min":0.23799999999999999,"15min":0.23799999999999999},"max":{"1min":0.79300000000000004,"5min":1.155,"15min":1.155},"last":0.247},{"interface":"front","average":{"1min":0.438,"5min":0.98899999999999999,"15min":1.081},"min":{"1min":0.30199999999999999,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.69699999999999995,"5min":12.116,"15min":12.116},"last":0.313}]},{"osd":4,"last update":"Fri Nov 24 07:58:20 2023","interfaces":[{"interface":"back","average":{"1min":0.626,"5min":0.64300000000000002,"15min":0.64600000000000002},"min":{"1min":0.40100000000000002,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":1.044,"5min":1.044,"15min":1.044},"last":0.44400000000000001},{"interface":"front","average":{"1min":0.64300000000000002,"5min":1.675,"15min":1.847},"min":{"1min":0.35699999999999998,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.93000000000000005,"5min":22.963999999999999,"15min":22.963999999999999},"last":0.377}]},{"osd":5,"last update":"Fri Nov 24 07:58:20 2023","interfaces":[{"interface":"back","average":{"1min":0.65200000000000002,"5min":1.1639999999999999,"15min":1.25},"min":{"1min":0.41999999999999998,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.97799999999999998,"5min":12.084,"15min":12.084},"last":0.436},{"interface":"front","average":{"1min":0.65800000000000003,"5min":0.67400000000000004,"15min":0.67600000000000005},"min":{"1min":0.39700000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.95499999999999996,"5min":1.073,"15min":1.073},"last":0.503}]},{"osd":6,"last update":"Fri Nov 24 07:58:20 2023","interfaces":[{"interface":"back","average":{"1min":0.65200000000000002,"5min":0.68300000000000005,"15min":0.68799999999999994},"min":{"1min":0.45000000000000001,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":0.94699999999999995,"5min":1.0580000000000001,"15min":1.0580000000000001},"last":0.36799999999999999},{"interface":"front","average":{"1min":0.67000000000000004,"5min":1.1850000000000001,"15min":1.2709999999999999},"min":{"1min":0.46200000000000002,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":0.95899999999999996,"5min":12.045,"15min":12.045},"last":0.46000000000000002}]},{"osd":7,"last update":"Fri Nov 24 07:58:20 2023","interfaces":[{"interface":"back","average":{"1min":0.67000000000000004,"5min":0.65800000000000003,"15min":0.65600000000000003},"min":{"1min":0.45200000000000001,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":0.98299999999999998,"5min":1.038,"15min":1.038},"last":0.46700000000000003},{"interface":"front","average":{"1min":0.67500000000000004,"5min":1.7050000000000001,"15min":1.877},"min":{"1min":0.41299999999999998,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":1.0069999999999999,"5min":22.995000000000001,"15min":22.995000000000001},"last":0.42299999999999999}]}]},{"osd":2,"up_from":13,"seq":55834574888,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27868,"kb_used_data":844,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343972,"statfs":{"total":96636764160,"available":96608227328,"internally_reserved":0,"allocated":864256,"data_stored":364790,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 07:58:19 2023","interfaces":[{"interface":"back","average":{"1min":0.36399999999999999,"5min":0.36899999999999999,"15min":0.37},"min":{"1min":0.219,"5min":0.219,"15min":0.219},"max":{"1min":0.55900000000000005,"5min":0.58099999999999996,"15min":0.58099999999999996},"last":0.32500000000000001},{"interface":"front","average":{"1min":0.38900000000000001,"5min":0.39900000000000002,"15min":0.40100000000000002},"min":{"1min":0.29299999999999998,"5min":0.27400000000000002,"15min":0.27400000000000002},"max":{"1min":0.54100000000000004,"5min":0.54100000000000004,"15min":0.54100000000000004},"last":0.41499999999999998}]},{"osd":1,"last update":"Fri Nov 24 07:58:19 2023","interfaces":[{"interface":"back","average":{"1min":0.34699999999999998,"5min":0.371,"15min":0.375},"min":{"1min":0.22,"5min":0.22,"15min":0.22},"max":{"1min":0.54000000000000004,"5min":0.59199999999999997,"15min":0.59199999999999997},"last":0.30599999999999999},{"interface":"front","average":{"1min":0.36599999999999999,"5min":0.40600000000000003,"15min":0.41199999999999998},"min":{"1min":0.28299999999999997,"5min":0.25600000000000001,"15min":0.25600000000000001},"max":{"1min":0.497,"5min":0.627,"15min":0.627},"last":0.34300000000000003}]},{"osd":3,"last update":"Fri Nov 24 07:58:19 2023","interfaces":[{"interface":"back","average":{"1min":0.40500000000000003,"5min":0.40899999999999997,"15min":0.40999999999999998},"min":{"1min":0.30399999999999999,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":0.64200000000000002,"5min":0.64200000000000002,"15min":0.64200000000000002},"last":0.26400000000000001},{"interface":"front","average":{"1min":0.41699999999999998,"5min":0.41399999999999998,"15min":0.41399999999999998},"min":{"1min":0.28599999999999998,"5min":0.27500000000000002,"15min":0.27500000000000002},"max":{"1min":0.59299999999999997,"5min":0.64600000000000002,"15min":0.64600000000000002},"last":0.36799999999999999}]},{"osd":4,"last update":"Fri Nov 24 07:58:19 2023","interfaces":[{"interface":"back","average":{"1min":0.54800000000000004,"5min":0.59599999999999997,"15min":0.60399999999999998},"min":{"1min":0.38700000000000001,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.75,"5min":0.81599999999999995,"15min":0.81599999999999995},"last":0.46800000000000003},{"interface":"front","average":{"1min":0.55500000000000005,"5min":0.59099999999999997,"15min":0.59699999999999998},"min":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":0.83799999999999997,"5min":0.83799999999999997,"15min":0.83799999999999997},"last":0.60299999999999998}]},{"osd":5,"last update":"Fri Nov 24 07:58:19 2023","interfaces":[{"interface":"back","average":{"1min":0.56000000000000005,"5min":0.60799999999999998,"15min":0.61599999999999999},"min":{"1min":0.434,"5min":0.434,"15min":0.434},"max":{"1min":0.73499999999999999,"5min":0.84899999999999998,"15min":0.84899999999999998},"last":0.503},{"interface":"front","average":{"1min":0.624,"5min":0.69599999999999995,"15min":0.70799999999999996},"min":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":0.82499999999999996,"5min":2.1699999999999999,"15min":2.1699999999999999},"last":0.53500000000000003}]},{"osd":6,"last update":"Fri Nov 24 07:58:19 2023","interfaces":[{"interface":"back","average":{"1min":0.60499999999999998,"5min":0.65400000000000003,"15min":0.66200000000000003},"min":{"1min":0.42499999999999999,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":0.878,"5min":0.878,"15min":0.878},"last":0.48299999999999998},{"interface":"front","average":{"1min":0.60599999999999998,"5min":0.65800000000000003,"15min":0.66600000000000004},"min":{"1min":0.41899999999999998,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":0.80300000000000005,"5min":0.88500000000000001,"15min":0.88500000000000001},"last":0.54900000000000004}]},{"osd":7,"last update":"Fri Nov 24 07:58:19 2023","interfaces":[{"interface":"back","average":{"1min":0.60399999999999998,"5min":0.65000000000000002,"15min":0.65800000000000003},"min":{"1min":0.441,"5min":0.441,"15min":0.441},"max":{"1min":0.77600000000000002,"5min":0.85999999999999999,"15min":0.85999999999999999},"last":0.56799999999999995},{"interface":"front","average":{"1min":0.623,"5min":0.66800000000000004,"15min":0.67600000000000005},"min":{"1min":0.40000000000000002,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":0.90000000000000002,"5min":0.90000000000000002,"15min":0.90000000000000002},"last":0.58399999999999996}]}]},{"osd":3,"up_from":13,"seq":55834574888,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27868,"kb_used_data":844,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343972,"statfs":{"total":96636764160,"available":96608227328,"internally_reserved":0,"allocated":864256,"data_stored":364790,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 07:58:22 2023","interfaces":[{"interface":"back","average":{"1min":0.34200000000000003,"5min":0.35299999999999998,"15min":0.35499999999999998},"min":{"1min":0.20899999999999999,"5min":0.20899999999999999,"15min":0.20899999999999999},"max":{"1min":0.439,"5min":0.84799999999999998,"15min":0.84799999999999998},"last":0.29099999999999998},{"interface":"front","average":{"1min":0.32700000000000001,"5min":0.35899999999999999,"15min":0.36399999999999999},"min":{"1min":0.253,"5min":0.23699999999999999,"15min":0.23699999999999999},"max":{"1min":0.42499999999999999,"5min":0.873,"15min":0.873},"last":0.44900000000000001}]},{"osd":1,"last update":"Fri Nov 24 07:58:22 2023","interfaces":[{"interface":"back","average":{"1min":0.35899999999999999,"5min":0.371,"15min":0.373},"min":{"1min":0.29499999999999998,"5min":0.25700000000000001,"15min":0.25700000000000001},"max":{"1min":0.443,"5min":0.92000000000000004,"15min":0.92000000000000004},"last":0.54000000000000004},{"interface":"front","average":{"1min":0.35099999999999998,"5min":0.379,"15min":0.38300000000000001},"min":{"1min":0.28299999999999997,"5min":0.248,"15min":0.248},"max":{"1min":0.46700000000000003,"5min":0.81699999999999995,"15min":0.81699999999999995},"last":0.40799999999999997}]},{"osd":2,"last update":"Fri Nov 24 07:58:22 2023","interfaces":[{"interface":"back","average":{"1min":0.33400000000000002,"5min":0.35499999999999998,"15min":0.35899999999999999},"min":{"1min":0.255,"5min":0.24399999999999999,"15min":0.24399999999999999},"max":{"1min":0.43099999999999999,"5min":0.88500000000000001,"15min":0.88500000000000001},"last":0.42999999999999999},{"interface":"front","average":{"1min":0.39500000000000002,"5min":0.45300000000000001,"15min":0.46300000000000002},"min":{"1min":0.311,"5min":0.311,"15min":0.311},"max":{"1min":0.45800000000000002,"5min":0.98099999999999998,"15min":0.98099999999999998},"last":0.47399999999999998}]},{"osd":4,"last update":"Fri Nov 24 07:58:22 2023","interfaces":[{"interface":"back","average":{"1min":0.57099999999999995,"5min":0.751,"15min":0.78100000000000003},"min":{"1min":0.42099999999999999,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.71999999999999997,"5min":5.7350000000000003,"15min":5.7350000000000003},"last":0.55500000000000005},{"interface":"front","average":{"1min":0.59299999999999997,"5min":0.58599999999999997,"15min":0.58499999999999996},"min":{"1min":0.37,"5min":0.37,"15min":0.37},"max":{"1min":0.77000000000000002,"5min":1.27,"15min":1.27},"last":0.58599999999999997}]},{"osd":5,"last update":"Fri Nov 24 07:58:22 2023","interfaces":[{"interface":"back","average":{"1min":0.56899999999999995,"5min":0.58399999999999996,"15min":0.58599999999999997},"min":{"1min":0.40799999999999997,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":0.82799999999999996,"5min":1.4430000000000001,"15min":1.4430000000000001},"last":0.65900000000000003},{"interface":"front","average":{"1min":0.60699999999999998,"5min":0.623,"15min":0.625},"min":{"1min":0.443,"5min":0.377,"15min":0.377},"max":{"1min":0.79900000000000004,"5min":1.4239999999999999,"15min":1.4239999999999999},"last":0.64000000000000001}]},{"osd":6,"last update":"Fri Nov 24 07:58:22 2023","interfaces":[{"interface":"back","average":{"1min":0.58999999999999997,"5min":0.61399999999999999,"15min":0.61799999999999999},"min":{"1min":0.46200000000000002,"5min":0.434,"15min":0.434},"max":{"1min":0.82599999999999996,"5min":1.355,"15min":1.355},"last":0.60699999999999998},{"interface":"front","average":{"1min":0.61199999999999999,"5min":0.79900000000000004,"15min":0.82999999999999996},"min":{"1min":0.47499999999999998,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.76900000000000002,"5min":5.6479999999999997,"15min":5.6479999999999997},"last":0.62}]},{"osd":7,"last update":"Fri Nov 24 07:58:22 2023","interfaces":[{"interface":"back","average":{"1min":0.59199999999999997,"5min":0.65100000000000002,"15min":0.66100000000000003},"min":{"1min":0.48099999999999998,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":0.79600000000000004,"5min":1.51,"15min":1.51},"last":0.67900000000000005},{"interface":"front","average":{"1min":0.61099999999999999,"5min":0.79500000000000004,"15min":0.82599999999999996},"min":{"1min":0.434,"5min":0.434,"15min":0.434},"max":{"1min":0.75900000000000001,"5min":5.7069999999999999,"15min":5.7069999999999999},"last":0.496}]}]},{"osd":4,"up_from":13,"seq":55834574887,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27868,"kb_used_data":844,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343972,"statfs":{"total":96636764160,"available":96608227328,"internally_reserved":0,"allocated":864256,"data_stored":364790,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 07:58:25 2023","interfaces":[{"interface":"back","average":{"1min":0.504,"5min":0.56499999999999995,"15min":0.57499999999999996},"min":{"1min":0.34899999999999998,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.70799999999999996,"5min":0.79300000000000004,"15min":0.79300000000000004},"last":0.70399999999999996},{"interface":"front","average":{"1min":0.502,"5min":0.53400000000000003,"15min":0.54000000000000004},"min":{"1min":0.33600000000000002,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.70199999999999996,"5min":1.0249999999999999,"15min":1.0249999999999999},"last":0.69299999999999995}]},{"osd":1,"last update":"Fri Nov 24 07:58:25 2023","interfaces":[{"interface":"back","average":{"1min":0.52800000000000002,"5min":0.54700000000000004,"15min":0.55000000000000004},"min":{"1min":0.39500000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.73999999999999999,"5min":0.86399999999999999,"15min":0.86399999999999999},"last":0.65700000000000003},{"interface":"front","average":{"1min":0.51900000000000002,"5min":0.54400000000000004,"15min":0.54800000000000004},"min":{"1min":0.374,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.72199999999999998,"5min":0.98699999999999999,"15min":0.98699999999999999},"last":0.60599999999999998}]},{"osd":2,"last update":"Fri Nov 24 07:58:25 2023","interfaces":[{"interface":"back","average":{"1min":0.52000000000000002,"5min":0.55200000000000005,"15min":0.55800000000000005},"min":{"1min":0.38800000000000001,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.66400000000000003,"5min":0.81100000000000005,"15min":0.81100000000000005},"last":0.63700000000000001},{"interface":"front","average":{"1min":0.56899999999999995,"5min":0.60399999999999998,"15min":0.60999999999999999},"min":{"1min":0.40000000000000002,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.75700000000000001,"5min":0.89200000000000002,"15min":0.89200000000000002},"last":0.53600000000000003}]},{"osd":3,"last update":"Fri Nov 24 07:58:25 2023","interfaces":[{"interface":"back","average":{"1min":0.54900000000000004,"5min":0.57699999999999996,"15min":0.58199999999999996},"min":{"1min":0.42899999999999999,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":0.65800000000000003,"5min":0.89700000000000002,"15min":0.89700000000000002},"last":0.52000000000000002},{"interface":"front","average":{"1min":0.57499999999999996,"5min":0.61899999999999999,"15min":0.627},"min":{"1min":0.442,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.72899999999999998,"5min":0.96199999999999997,"15min":0.96199999999999997},"last":0.67400000000000004}]},{"osd":5,"last update":"Fri Nov 24 07:58:25 2023","interfaces":[{"interface":"back","average":{"1min":0.48599999999999999,"5min":0.46100000000000002,"15min":0.45700000000000002},"min":{"1min":0.29499999999999998,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":0.67600000000000005,"5min":0.70499999999999996,"15min":0.70499999999999996},"last":0.35899999999999999},{"interface":"front","average":{"1min":0.59899999999999998,"5min":0.61499999999999999,"15min":0.61699999999999999},"min":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.746,"5min":0.86399999999999999,"15min":0.86399999999999999},"last":0.42899999999999999}]},{"osd":6,"last update":"Fri Nov 24 07:58:25 2023","interfaces":[{"interface":"back","average":{"1min":0.56799999999999995,"5min":0.58199999999999996,"15min":0.58399999999999996},"min":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.69799999999999995,"5min":0.76800000000000002,"15min":0.76800000000000002},"last":0.33600000000000002},{"interface":"front","average":{"1min":0.58999999999999997,"5min":0.60899999999999999,"15min":0.61199999999999999},"min":{"1min":0.46899999999999997,"5min":0.436,"15min":0.436},"max":{"1min":0.77700000000000002,"5min":1.095,"15min":1.095},"last":0.41199999999999998}]},{"osd":7,"last update":"Fri Nov 24 07:58:25 2023","interfaces":[{"interface":"back","average":{"1min":0.56799999999999995,"5min":0.57199999999999995,"15min":0.57199999999999995},"min":{"1min":0.39300000000000002,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.74199999999999999,"5min":0.89300000000000002,"15min":0.89300000000000002},"last":0.377},{"interface":"front","average":{"1min":0.61699999999999999,"5min":0.65300000000000002,"15min":0.65900000000000003},"min":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.79100000000000004,"5min":0.91500000000000004,"15min":0.91500000000000004},"last":0.39100000000000001}]}]},{"osd":5,"up_from":13,"seq":55834574888,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28448,"kb_used_data":1424,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343392,"statfs":{"total":96636764160,"available":96607633408,"internally_reserved":0,"allocated":1458176,"data_stored":955158,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 07:58:20 2023","interfaces":[{"interface":"back","average":{"1min":0.52300000000000002,"5min":0.55400000000000005,"15min":0.55900000000000005},"min":{"1min":0.35299999999999998,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.65800000000000003,"5min":1.0109999999999999,"15min":1.0109999999999999},"last":0.45500000000000002},{"interface":"front","average":{"1min":0.53200000000000003,"5min":0.746,"15min":0.78200000000000003},"min":{"1min":0.34399999999999997,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.72099999999999997,"5min":5.431,"15min":5.431},"last":0.34300000000000003}]},{"osd":1,"last update":"Fri Nov 24 07:58:20 2023","interfaces":[{"interface":"back","average":{"1min":0.51100000000000001,"5min":0.54900000000000004,"15min":0.55500000000000005},"min":{"1min":0.36399999999999999,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":0.67100000000000004,"5min":1.016,"15min":1.016},"last":0.41299999999999998},{"interface":"front","average":{"1min":0.54600000000000004,"5min":0.56499999999999995,"15min":0.56799999999999995},"min":{"1min":0.28599999999999998,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":0.78300000000000003,"5min":0.96499999999999997,"15min":0.96499999999999997},"last":0.47699999999999998}]},{"osd":2,"last update":"Fri Nov 24 07:58:20 2023","interfaces":[{"interface":"back","average":{"1min":0.54500000000000004,"5min":0.54900000000000004,"15min":0.55000000000000004},"min":{"1min":0.33700000000000002,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.67100000000000004,"5min":1.163,"15min":1.163},"last":0.46600000000000003},{"interface":"front","average":{"1min":0.54500000000000004,"5min":0.61299999999999999,"15min":0.624},"min":{"1min":0.41099999999999998,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.78900000000000003,"5min":1.119,"15min":1.119},"last":0.35899999999999999}]},{"osd":3,"last update":"Fri Nov 24 07:58:20 2023","interfaces":[{"interface":"back","average":{"1min":0.57199999999999995,"5min":0.83799999999999997,"15min":0.88200000000000001},"min":{"1min":0.36899999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.85899999999999999,"5min":5.609,"15min":5.609},"last":0.32400000000000001},{"interface":"front","average":{"1min":0.629,"5min":0.66600000000000004,"15min":0.67200000000000004},"min":{"1min":0.44600000000000001,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":0.76400000000000001,"5min":1.288,"15min":1.288},"last":0.39600000000000002}]},{"osd":4,"last update":"Fri Nov 24 07:58:20 2023","interfaces":[{"interface":"back","average":{"1min":0.51500000000000001,"5min":0.51100000000000001,"15min":0.51000000000000001},"min":{"1min":0.32100000000000001,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.76400000000000001,"5min":0.76400000000000001,"15min":0.76400000000000001},"last":0.30599999999999999},{"interface":"front","average":{"1min":0.58799999999999997,"5min":0.57499999999999996,"15min":0.57299999999999995},"min":{"1min":0.29599999999999999,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":0.92900000000000005,"5min":0.92900000000000005,"15min":0.92900000000000005},"last":0.378}]},{"osd":6,"last update":"Fri Nov 24 07:58:20 2023","interfaces":[{"interface":"back","average":{"1min":0.59599999999999997,"5min":0.57599999999999996,"15min":0.57199999999999995},"min":{"1min":0.35199999999999998,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.81799999999999995,"5min":0.89700000000000002,"15min":0.89700000000000002},"last":0.434},{"interface":"front","average":{"1min":0.63800000000000001,"5min":0.627,"15min":0.625},"min":{"1min":0.40300000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":1.083,"5min":1.083,"15min":1.083},"last":0.443}]},{"osd":7,"last update":"Fri Nov 24 07:58:20 2023","interfaces":[{"interface":"back","average":{"1min":0.66200000000000003,"5min":0.64900000000000002,"15min":0.64700000000000002},"min":{"1min":0.36799999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.90100000000000002,"5min":0.96599999999999997,"15min":0.96599999999999997},"last":0.42399999999999999},{"interface":"front","average":{"1min":0.66800000000000004,"5min":0.65600000000000003,"15min":0.65400000000000003},"min":{"1min":0.42699999999999999,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":0.85599999999999998,"5min":0.94199999999999995,"15min":0.94199999999999995},"last":0.49099999999999999}]}]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-11-24T07:59:19.585 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.3f is not active+clean 2023-11-24T07:59:19.585 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.3f', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-24T07:59:13.620687+0000', 'last_change': '2023-11-24T07:59:13.620687+0000', 'last_active': '2023-11-24T07:59:13.620687+0000', 'last_peered': '2023-11-24T07:59:13.620687+0000', 'last_clean': '2023-11-24T07:59:13.620687+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-24T07:59:13.620687+0000', 'last_undegraded': '2023-11-24T07:59:13.620687+0000', 'last_fullsized': '2023-11-24T07:59:13.620687+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-24T07:59:13.620687+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-24T07:59:13.620687+0000', 'last_clean_scrub_stamp': '2023-11-24T07:59:13.620687+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-24T07:59:19.586 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.3e is not active+clean 2023-11-24T07:59:19.586 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.3e', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-24T07:59:13.620687+0000', 'last_change': '2023-11-24T07:59:13.620687+0000', 'last_active': '2023-11-24T07:59:13.620687+0000', 'last_peered': '2023-11-24T07:59:13.620687+0000', 'last_clean': '2023-11-24T07:59:13.620687+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-24T07:59:13.620687+0000', 'last_undegraded': '2023-11-24T07:59:13.620687+0000', 'last_fullsized': '2023-11-24T07:59:13.620687+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-24T07:59:13.620687+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-24T07:59:13.620687+0000', 'last_clean_scrub_stamp': '2023-11-24T07:59:13.620687+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-24T07:59:19.586 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.3d is not active+clean 2023-11-24T07:59:19.586 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.3d', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-24T07:59:13.620687+0000', 'last_change': '2023-11-24T07:59:13.620687+0000', 'last_active': '2023-11-24T07:59:13.620687+0000', 'last_peered': '2023-11-24T07:59:13.620687+0000', 'last_clean': '2023-11-24T07:59:13.620687+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-24T07:59:13.620687+0000', 'last_undegraded': '2023-11-24T07:59:13.620687+0000', 'last_fullsized': '2023-11-24T07:59:13.620687+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-24T07:59:13.620687+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-24T07:59:13.620687+0000', 'last_clean_scrub_stamp': '2023-11-24T07:59:13.620687+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-24T07:59:19.586 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.3c is not active+clean 2023-11-24T07:59:19.586 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.3c', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-24T07:59:13.620687+0000', 'last_change': '2023-11-24T07:59:13.620687+0000', 'last_active': '2023-11-24T07:59:13.620687+0000', 'last_peered': '2023-11-24T07:59:13.620687+0000', 'last_clean': '2023-11-24T07:59:13.620687+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-24T07:59:13.620687+0000', 'last_undegraded': '2023-11-24T07:59:13.620687+0000', 'last_fullsized': '2023-11-24T07:59:13.620687+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-24T07:59:13.620687+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-24T07:59:13.620687+0000', 'last_clean_scrub_stamp': '2023-11-24T07:59:13.620687+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-24T07:59:19.586 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.3b is not active+clean 2023-11-24T07:59:19.586 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.3b', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-24T07:59:13.620687+0000', 'last_change': '2023-11-24T07:59:13.620687+0000', 'last_active': '2023-11-24T07:59:13.620687+0000', 'last_peered': '2023-11-24T07:59:13.620687+0000', 'last_clean': '2023-11-24T07:59:13.620687+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-24T07:59:13.620687+0000', 'last_undegraded': '2023-11-24T07:59:13.620687+0000', 'last_fullsized': '2023-11-24T07:59:13.620687+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-24T07:59:13.620687+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-24T07:59:13.620687+0000', 'last_clean_scrub_stamp': '2023-11-24T07:59:13.620687+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-24T07:59:19.586 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.3a is not active+clean 2023-11-24T07:59:19.586 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.3a', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-24T07:59:13.620687+0000', 'last_change': '2023-11-24T07:59:13.620687+0000', 'last_active': '2023-11-24T07:59:13.620687+0000', 'last_peered': '2023-11-24T07:59:13.620687+0000', 'last_clean': '2023-11-24T07:59:13.620687+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-24T07:59:13.620687+0000', 'last_undegraded': '2023-11-24T07:59:13.620687+0000', 'last_fullsized': '2023-11-24T07:59:13.620687+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-24T07:59:13.620687+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-24T07:59:13.620687+0000', 'last_clean_scrub_stamp': '2023-11-24T07:59:13.620687+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-24T07:59:19.587 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.39 is not active+clean 2023-11-24T07:59:19.587 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.39', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-24T07:59:13.620687+0000', 'last_change': '2023-11-24T07:59:13.620687+0000', 'last_active': '2023-11-24T07:59:13.620687+0000', 'last_peered': '2023-11-24T07:59:13.620687+0000', 'last_clean': '2023-11-24T07:59:13.620687+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-24T07:59:13.620687+0000', 'last_undegraded': '2023-11-24T07:59:13.620687+0000', 'last_fullsized': '2023-11-24T07:59:13.620687+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-24T07:59:13.620687+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-24T07:59:13.620687+0000', 'last_clean_scrub_stamp': '2023-11-24T07:59:13.620687+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-24T07:59:19.587 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.38 is not active+clean 2023-11-24T07:59:19.587 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.38', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-24T07:59:13.620687+0000', 'last_change': '2023-11-24T07:59:13.620687+0000', 'last_active': '2023-11-24T07:59:13.620687+0000', 'last_peered': '2023-11-24T07:59:13.620687+0000', 'last_clean': '2023-11-24T07:59:13.620687+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-24T07:59:13.620687+0000', 'last_undegraded': '2023-11-24T07:59:13.620687+0000', 'last_fullsized': '2023-11-24T07:59:13.620687+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-24T07:59:13.620687+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-24T07:59:13.620687+0000', 'last_clean_scrub_stamp': '2023-11-24T07:59:13.620687+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-24T07:59:19.587 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.37 is not active+clean 2023-11-24T07:59:19.587 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.37', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-24T07:59:13.620687+0000', 'last_change': '2023-11-24T07:59:13.620687+0000', 'last_active': '2023-11-24T07:59:13.620687+0000', 'last_peered': '2023-11-24T07:59:13.620687+0000', 'last_clean': '2023-11-24T07:59:13.620687+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-24T07:59:13.620687+0000', 'last_undegraded': '2023-11-24T07:59:13.620687+0000', 'last_fullsized': '2023-11-24T07:59:13.620687+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-24T07:59:13.620687+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-24T07:59:13.620687+0000', 'last_clean_scrub_stamp': '2023-11-24T07:59:13.620687+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-24T07:59:19.587 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.36 is not active+clean 2023-11-24T07:59:19.587 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.36', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-24T07:59:13.620687+0000', 'last_change': '2023-11-24T07:59:13.620687+0000', 'last_active': '2023-11-24T07:59:13.620687+0000', 'last_peered': '2023-11-24T07:59:13.620687+0000', 'last_clean': '2023-11-24T07:59:13.620687+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-24T07:59:13.620687+0000', 'last_undegraded': '2023-11-24T07:59:13.620687+0000', 'last_fullsized': '2023-11-24T07:59:13.620687+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-24T07:59:13.620687+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-24T07:59:13.620687+0000', 'last_clean_scrub_stamp': '2023-11-24T07:59:13.620687+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-24T07:59:19.587 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.35 is not active+clean 2023-11-24T07:59:19.587 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.35', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-24T07:59:13.620687+0000', 'last_change': '2023-11-24T07:59:13.620687+0000', 'last_active': '2023-11-24T07:59:13.620687+0000', 'last_peered': '2023-11-24T07:59:13.620687+0000', 'last_clean': '2023-11-24T07:59:13.620687+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-24T07:59:13.620687+0000', 'last_undegraded': '2023-11-24T07:59:13.620687+0000', 'last_fullsized': '2023-11-24T07:59:13.620687+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-24T07:59:13.620687+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-24T07:59:13.620687+0000', 'last_clean_scrub_stamp': '2023-11-24T07:59:13.620687+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-24T07:59:19.588 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.34 is not active+clean 2023-11-24T07:59:19.588 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.34', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-24T07:59:13.620687+0000', 'last_change': '2023-11-24T07:59:13.620687+0000', 'last_active': '2023-11-24T07:59:13.620687+0000', 'last_peered': '2023-11-24T07:59:13.620687+0000', 'last_clean': '2023-11-24T07:59:13.620687+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-24T07:59:13.620687+0000', 'last_undegraded': '2023-11-24T07:59:13.620687+0000', 'last_fullsized': '2023-11-24T07:59:13.620687+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-24T07:59:13.620687+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-24T07:59:13.620687+0000', 'last_clean_scrub_stamp': '2023-11-24T07:59:13.620687+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-24T07:59:19.588 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.33 is not active+clean 2023-11-24T07:59:19.588 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.33', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-24T07:59:13.620687+0000', 'last_change': '2023-11-24T07:59:13.620687+0000', 'last_active': '2023-11-24T07:59:13.620687+0000', 'last_peered': '2023-11-24T07:59:13.620687+0000', 'last_clean': '2023-11-24T07:59:13.620687+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-24T07:59:13.620687+0000', 'last_undegraded': '2023-11-24T07:59:13.620687+0000', 'last_fullsized': '2023-11-24T07:59:13.620687+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-24T07:59:13.620687+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-24T07:59:13.620687+0000', 'last_clean_scrub_stamp': '2023-11-24T07:59:13.620687+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-24T07:59:19.588 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.32 is not active+clean 2023-11-24T07:59:19.588 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.32', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-24T07:59:13.620687+0000', 'last_change': '2023-11-24T07:59:13.620687+0000', 'last_active': '2023-11-24T07:59:13.620687+0000', 'last_peered': '2023-11-24T07:59:13.620687+0000', 'last_clean': '2023-11-24T07:59:13.620687+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-24T07:59:13.620687+0000', 'last_undegraded': '2023-11-24T07:59:13.620687+0000', 'last_fullsized': '2023-11-24T07:59:13.620687+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-24T07:59:13.620687+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-24T07:59:13.620687+0000', 'last_clean_scrub_stamp': '2023-11-24T07:59:13.620687+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-24T07:59:19.588 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.31 is not active+clean 2023-11-24T07:59:19.588 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.31', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-24T07:59:13.620687+0000', 'last_change': '2023-11-24T07:59:13.620687+0000', 'last_active': '2023-11-24T07:59:13.620687+0000', 'last_peered': '2023-11-24T07:59:13.620687+0000', 'last_clean': '2023-11-24T07:59:13.620687+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-24T07:59:13.620687+0000', 'last_undegraded': '2023-11-24T07:59:13.620687+0000', 'last_fullsized': '2023-11-24T07:59:13.620687+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-24T07:59:13.620687+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-24T07:59:13.620687+0000', 'last_clean_scrub_stamp': '2023-11-24T07:59:13.620687+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-24T07:59:19.589 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.30 is not active+clean 2023-11-24T07:59:19.589 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.30', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-24T07:59:13.620687+0000', 'last_change': '2023-11-24T07:59:13.620687+0000', 'last_active': '2023-11-24T07:59:13.620687+0000', 'last_peered': '2023-11-24T07:59:13.620687+0000', 'last_clean': '2023-11-24T07:59:13.620687+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-24T07:59:13.620687+0000', 'last_undegraded': '2023-11-24T07:59:13.620687+0000', 'last_fullsized': '2023-11-24T07:59:13.620687+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-24T07:59:13.620687+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-24T07:59:13.620687+0000', 'last_clean_scrub_stamp': '2023-11-24T07:59:13.620687+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-24T07:59:19.589 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.2f is not active+clean 2023-11-24T07:59:19.589 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.2f', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-24T07:59:13.620687+0000', 'last_change': '2023-11-24T07:59:13.620687+0000', 'last_active': '2023-11-24T07:59:13.620687+0000', 'last_peered': '2023-11-24T07:59:13.620687+0000', 'last_clean': '2023-11-24T07:59:13.620687+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-24T07:59:13.620687+0000', 'last_undegraded': '2023-11-24T07:59:13.620687+0000', 'last_fullsized': '2023-11-24T07:59:13.620687+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-24T07:59:13.620687+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-24T07:59:13.620687+0000', 'last_clean_scrub_stamp': '2023-11-24T07:59:13.620687+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-24T07:59:19.589 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.2e is not active+clean 2023-11-24T07:59:19.589 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.2e', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-24T07:59:13.620687+0000', 'last_change': '2023-11-24T07:59:13.620687+0000', 'last_active': '2023-11-24T07:59:13.620687+0000', 'last_peered': '2023-11-24T07:59:13.620687+0000', 'last_clean': '2023-11-24T07:59:13.620687+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-24T07:59:13.620687+0000', 'last_undegraded': '2023-11-24T07:59:13.620687+0000', 'last_fullsized': '2023-11-24T07:59:13.620687+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-24T07:59:13.620687+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-24T07:59:13.620687+0000', 'last_clean_scrub_stamp': '2023-11-24T07:59:13.620687+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-24T07:59:19.589 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.2d is not active+clean 2023-11-24T07:59:19.589 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.2d', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-24T07:59:13.620687+0000', 'last_change': '2023-11-24T07:59:13.620687+0000', 'last_active': '2023-11-24T07:59:13.620687+0000', 'last_peered': '2023-11-24T07:59:13.620687+0000', 'last_clean': '2023-11-24T07:59:13.620687+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-24T07:59:13.620687+0000', 'last_undegraded': '2023-11-24T07:59:13.620687+0000', 'last_fullsized': '2023-11-24T07:59:13.620687+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-24T07:59:13.620687+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-24T07:59:13.620687+0000', 'last_clean_scrub_stamp': '2023-11-24T07:59:13.620687+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-24T07:59:19.589 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.2c is not active+clean 2023-11-24T07:59:19.589 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.2c', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-24T07:59:13.620687+0000', 'last_change': '2023-11-24T07:59:13.620687+0000', 'last_active': '2023-11-24T07:59:13.620687+0000', 'last_peered': '2023-11-24T07:59:13.620687+0000', 'last_clean': '2023-11-24T07:59:13.620687+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-24T07:59:13.620687+0000', 'last_undegraded': '2023-11-24T07:59:13.620687+0000', 'last_fullsized': '2023-11-24T07:59:13.620687+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-24T07:59:13.620687+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-24T07:59:13.620687+0000', 'last_clean_scrub_stamp': '2023-11-24T07:59:13.620687+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-24T07:59:19.590 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.2b is not active+clean 2023-11-24T07:59:19.590 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.2b', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-24T07:59:13.620687+0000', 'last_change': '2023-11-24T07:59:13.620687+0000', 'last_active': '2023-11-24T07:59:13.620687+0000', 'last_peered': '2023-11-24T07:59:13.620687+0000', 'last_clean': '2023-11-24T07:59:13.620687+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-24T07:59:13.620687+0000', 'last_undegraded': '2023-11-24T07:59:13.620687+0000', 'last_fullsized': '2023-11-24T07:59:13.620687+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-24T07:59:13.620687+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-24T07:59:13.620687+0000', 'last_clean_scrub_stamp': '2023-11-24T07:59:13.620687+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-24T07:59:19.590 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.2a is not active+clean 2023-11-24T07:59:19.590 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.2a', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-24T07:59:13.620687+0000', 'last_change': '2023-11-24T07:59:13.620687+0000', 'last_active': '2023-11-24T07:59:13.620687+0000', 'last_peered': '2023-11-24T07:59:13.620687+0000', 'last_clean': '2023-11-24T07:59:13.620687+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-24T07:59:13.620687+0000', 'last_undegraded': '2023-11-24T07:59:13.620687+0000', 'last_fullsized': '2023-11-24T07:59:13.620687+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-24T07:59:13.620687+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-24T07:59:13.620687+0000', 'last_clean_scrub_stamp': '2023-11-24T07:59:13.620687+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-24T07:59:19.590 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.29 is not active+clean 2023-11-24T07:59:19.590 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.29', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-24T07:59:13.620687+0000', 'last_change': '2023-11-24T07:59:13.620687+0000', 'last_active': '2023-11-24T07:59:13.620687+0000', 'last_peered': '2023-11-24T07:59:13.620687+0000', 'last_clean': '2023-11-24T07:59:13.620687+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-24T07:59:13.620687+0000', 'last_undegraded': '2023-11-24T07:59:13.620687+0000', 'last_fullsized': '2023-11-24T07:59:13.620687+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-24T07:59:13.620687+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-24T07:59:13.620687+0000', 'last_clean_scrub_stamp': '2023-11-24T07:59:13.620687+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-24T07:59:19.590 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.28 is not active+clean 2023-11-24T07:59:19.590 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.28', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-24T07:59:13.620687+0000', 'last_change': '2023-11-24T07:59:13.620687+0000', 'last_active': '2023-11-24T07:59:13.620687+0000', 'last_peered': '2023-11-24T07:59:13.620687+0000', 'last_clean': '2023-11-24T07:59:13.620687+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-24T07:59:13.620687+0000', 'last_undegraded': '2023-11-24T07:59:13.620687+0000', 'last_fullsized': '2023-11-24T07:59:13.620687+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-24T07:59:13.620687+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-24T07:59:13.620687+0000', 'last_clean_scrub_stamp': '2023-11-24T07:59:13.620687+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-24T07:59:19.590 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.27 is not active+clean 2023-11-24T07:59:19.591 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.27', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-24T07:59:13.620687+0000', 'last_change': '2023-11-24T07:59:13.620687+0000', 'last_active': '2023-11-24T07:59:13.620687+0000', 'last_peered': '2023-11-24T07:59:13.620687+0000', 'last_clean': '2023-11-24T07:59:13.620687+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-24T07:59:13.620687+0000', 'last_undegraded': '2023-11-24T07:59:13.620687+0000', 'last_fullsized': '2023-11-24T07:59:13.620687+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-24T07:59:13.620687+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-24T07:59:13.620687+0000', 'last_clean_scrub_stamp': '2023-11-24T07:59:13.620687+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-24T07:59:19.591 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.26 is not active+clean 2023-11-24T07:59:19.591 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.26', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-24T07:59:13.620687+0000', 'last_change': '2023-11-24T07:59:13.620687+0000', 'last_active': '2023-11-24T07:59:13.620687+0000', 'last_peered': '2023-11-24T07:59:13.620687+0000', 'last_clean': '2023-11-24T07:59:13.620687+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-24T07:59:13.620687+0000', 'last_undegraded': '2023-11-24T07:59:13.620687+0000', 'last_fullsized': '2023-11-24T07:59:13.620687+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-24T07:59:13.620687+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-24T07:59:13.620687+0000', 'last_clean_scrub_stamp': '2023-11-24T07:59:13.620687+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-24T07:59:19.591 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.25 is not active+clean 2023-11-24T07:59:19.591 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.25', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-24T07:59:13.620687+0000', 'last_change': '2023-11-24T07:59:13.620687+0000', 'last_active': '2023-11-24T07:59:13.620687+0000', 'last_peered': '2023-11-24T07:59:13.620687+0000', 'last_clean': '2023-11-24T07:59:13.620687+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-24T07:59:13.620687+0000', 'last_undegraded': '2023-11-24T07:59:13.620687+0000', 'last_fullsized': '2023-11-24T07:59:13.620687+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-24T07:59:13.620687+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-24T07:59:13.620687+0000', 'last_clean_scrub_stamp': '2023-11-24T07:59:13.620687+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-24T07:59:19.591 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.24 is not active+clean 2023-11-24T07:59:19.591 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.24', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-24T07:59:13.620687+0000', 'last_change': '2023-11-24T07:59:13.620687+0000', 'last_active': '2023-11-24T07:59:13.620687+0000', 'last_peered': '2023-11-24T07:59:13.620687+0000', 'last_clean': '2023-11-24T07:59:13.620687+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-24T07:59:13.620687+0000', 'last_undegraded': '2023-11-24T07:59:13.620687+0000', 'last_fullsized': '2023-11-24T07:59:13.620687+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-24T07:59:13.620687+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-24T07:59:13.620687+0000', 'last_clean_scrub_stamp': '2023-11-24T07:59:13.620687+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-24T07:59:19.591 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.23 is not active+clean 2023-11-24T07:59:19.591 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.23', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-24T07:59:13.620687+0000', 'last_change': '2023-11-24T07:59:13.620687+0000', 'last_active': '2023-11-24T07:59:13.620687+0000', 'last_peered': '2023-11-24T07:59:13.620687+0000', 'last_clean': '2023-11-24T07:59:13.620687+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-24T07:59:13.620687+0000', 'last_undegraded': '2023-11-24T07:59:13.620687+0000', 'last_fullsized': '2023-11-24T07:59:13.620687+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-24T07:59:13.620687+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-24T07:59:13.620687+0000', 'last_clean_scrub_stamp': '2023-11-24T07:59:13.620687+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-24T07:59:19.591 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.22 is not active+clean 2023-11-24T07:59:19.591 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.22', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-24T07:59:13.620687+0000', 'last_change': '2023-11-24T07:59:13.620687+0000', 'last_active': '2023-11-24T07:59:13.620687+0000', 'last_peered': '2023-11-24T07:59:13.620687+0000', 'last_clean': '2023-11-24T07:59:13.620687+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-24T07:59:13.620687+0000', 'last_undegraded': '2023-11-24T07:59:13.620687+0000', 'last_fullsized': '2023-11-24T07:59:13.620687+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-24T07:59:13.620687+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-24T07:59:13.620687+0000', 'last_clean_scrub_stamp': '2023-11-24T07:59:13.620687+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-24T07:59:19.592 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.21 is not active+clean 2023-11-24T07:59:19.592 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.21', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-24T07:59:13.620687+0000', 'last_change': '2023-11-24T07:59:13.620687+0000', 'last_active': '2023-11-24T07:59:13.620687+0000', 'last_peered': '2023-11-24T07:59:13.620687+0000', 'last_clean': '2023-11-24T07:59:13.620687+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-24T07:59:13.620687+0000', 'last_undegraded': '2023-11-24T07:59:13.620687+0000', 'last_fullsized': '2023-11-24T07:59:13.620687+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-24T07:59:13.620687+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-24T07:59:13.620687+0000', 'last_clean_scrub_stamp': '2023-11-24T07:59:13.620687+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-24T07:59:19.592 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.20 is not active+clean 2023-11-24T07:59:19.592 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.20', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-24T07:59:13.620687+0000', 'last_change': '2023-11-24T07:59:13.620687+0000', 'last_active': '2023-11-24T07:59:13.620687+0000', 'last_peered': '2023-11-24T07:59:13.620687+0000', 'last_clean': '2023-11-24T07:59:13.620687+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-24T07:59:13.620687+0000', 'last_undegraded': '2023-11-24T07:59:13.620687+0000', 'last_fullsized': '2023-11-24T07:59:13.620687+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-24T07:59:13.620687+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-24T07:59:13.620687+0000', 'last_clean_scrub_stamp': '2023-11-24T07:59:13.620687+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-24T07:59:19.592 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.1e is not active+clean 2023-11-24T07:59:19.592 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.1e', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-24T07:59:13.620687+0000', 'last_change': '2023-11-24T07:59:13.620687+0000', 'last_active': '2023-11-24T07:59:13.620687+0000', 'last_peered': '2023-11-24T07:59:13.620687+0000', 'last_clean': '2023-11-24T07:59:13.620687+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-24T07:59:13.620687+0000', 'last_undegraded': '2023-11-24T07:59:13.620687+0000', 'last_fullsized': '2023-11-24T07:59:13.620687+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-24T07:59:13.620687+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-24T07:59:13.620687+0000', 'last_clean_scrub_stamp': '2023-11-24T07:59:13.620687+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-24T07:59:19.592 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.1f is not active+clean 2023-11-24T07:59:19.592 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.1f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 71, 'state': 'creating+peering', 'last_fresh': '2023-11-24T07:59:12.627449+0000', 'last_change': '2023-11-24T07:59:12.627449+0000', 'last_active': '2023-11-24T07:59:12.615544+0000', 'last_peered': '2023-11-24T07:59:12.615544+0000', 'last_clean': '2023-11-24T07:59:12.615544+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-24T07:59:12.627449+0000', 'last_undegraded': '2023-11-24T07:59:12.627449+0000', 'last_fullsized': '2023-11-24T07:59:12.627449+0000', 'mapping_epoch': 71, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 71, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-24T07:59:12.615544+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-24T07:59:12.615544+0000', 'last_clean_scrub_stamp': '2023-11-24T07:59:12.615544+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 4], 'acting': [2, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2023-11-24T07:59:19.592 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.1f is not active+clean 2023-11-24T07:59:19.594 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.1f', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-24T07:59:13.620687+0000', 'last_change': '2023-11-24T07:59:13.620687+0000', 'last_active': '2023-11-24T07:59:13.620687+0000', 'last_peered': '2023-11-24T07:59:13.620687+0000', 'last_clean': '2023-11-24T07:59:13.620687+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-24T07:59:13.620687+0000', 'last_undegraded': '2023-11-24T07:59:13.620687+0000', 'last_fullsized': '2023-11-24T07:59:13.620687+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-24T07:59:13.620687+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-24T07:59:13.620687+0000', 'last_clean_scrub_stamp': '2023-11-24T07:59:13.620687+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-24T07:59:19.594 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.1e is not active+clean 2023-11-24T07:59:19.594 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.1e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 71, 'state': 'creating+peering', 'last_fresh': '2023-11-24T07:59:12.622955+0000', 'last_change': '2023-11-24T07:59:12.622955+0000', 'last_active': '2023-11-24T07:59:12.615544+0000', 'last_peered': '2023-11-24T07:59:12.615544+0000', 'last_clean': '2023-11-24T07:59:12.615544+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-24T07:59:12.622955+0000', 'last_undegraded': '2023-11-24T07:59:12.622955+0000', 'last_fullsized': '2023-11-24T07:59:12.622955+0000', 'mapping_epoch': 71, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 71, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-24T07:59:12.615544+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-24T07:59:12.615544+0000', 'last_clean_scrub_stamp': '2023-11-24T07:59:12.615544+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 7], 'acting': [5, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2023-11-24T07:59:19.594 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.1c is not active+clean 2023-11-24T07:59:19.594 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.1c', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-24T07:59:13.620687+0000', 'last_change': '2023-11-24T07:59:13.620687+0000', 'last_active': '2023-11-24T07:59:13.620687+0000', 'last_peered': '2023-11-24T07:59:13.620687+0000', 'last_clean': '2023-11-24T07:59:13.620687+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-24T07:59:13.620687+0000', 'last_undegraded': '2023-11-24T07:59:13.620687+0000', 'last_fullsized': '2023-11-24T07:59:13.620687+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-24T07:59:13.620687+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-24T07:59:13.620687+0000', 'last_clean_scrub_stamp': '2023-11-24T07:59:13.620687+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-24T07:59:19.594 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.1d is not active+clean 2023-11-24T07:59:19.594 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.1d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 71, 'state': 'creating+peering', 'last_fresh': '2023-11-24T07:59:12.632766+0000', 'last_change': '2023-11-24T07:59:12.632766+0000', 'last_active': '2023-11-24T07:59:12.615544+0000', 'last_peered': '2023-11-24T07:59:12.615544+0000', 'last_clean': '2023-11-24T07:59:12.615544+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-24T07:59:12.632766+0000', 'last_undegraded': '2023-11-24T07:59:12.632766+0000', 'last_fullsized': '2023-11-24T07:59:12.632766+0000', 'mapping_epoch': 71, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 71, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-24T07:59:12.615544+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-24T07:59:12.615544+0000', 'last_clean_scrub_stamp': '2023-11-24T07:59:12.615544+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 2], 'acting': [3, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2023-11-24T07:59:19.594 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.1d is not active+clean 2023-11-24T07:59:19.594 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.1d', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-24T07:59:13.620687+0000', 'last_change': '2023-11-24T07:59:13.620687+0000', 'last_active': '2023-11-24T07:59:13.620687+0000', 'last_peered': '2023-11-24T07:59:13.620687+0000', 'last_clean': '2023-11-24T07:59:13.620687+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-24T07:59:13.620687+0000', 'last_undegraded': '2023-11-24T07:59:13.620687+0000', 'last_fullsized': '2023-11-24T07:59:13.620687+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-24T07:59:13.620687+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-24T07:59:13.620687+0000', 'last_clean_scrub_stamp': '2023-11-24T07:59:13.620687+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-24T07:59:19.595 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.1c is not active+clean 2023-11-24T07:59:19.595 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.1c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 71, 'state': 'creating+peering', 'last_fresh': '2023-11-24T07:59:12.627772+0000', 'last_change': '2023-11-24T07:59:12.627772+0000', 'last_active': '2023-11-24T07:59:12.615544+0000', 'last_peered': '2023-11-24T07:59:12.615544+0000', 'last_clean': '2023-11-24T07:59:12.615544+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-24T07:59:12.627772+0000', 'last_undegraded': '2023-11-24T07:59:12.627772+0000', 'last_fullsized': '2023-11-24T07:59:12.627772+0000', 'mapping_epoch': 71, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 71, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-24T07:59:12.615544+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-24T07:59:12.615544+0000', 'last_clean_scrub_stamp': '2023-11-24T07:59:12.615544+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 6], 'acting': [2, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2023-11-24T07:59:19.595 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.1a is not active+clean 2023-11-24T07:59:19.595 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.1a', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-24T07:59:13.620687+0000', 'last_change': '2023-11-24T07:59:13.620687+0000', 'last_active': '2023-11-24T07:59:13.620687+0000', 'last_peered': '2023-11-24T07:59:13.620687+0000', 'last_clean': '2023-11-24T07:59:13.620687+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-24T07:59:13.620687+0000', 'last_undegraded': '2023-11-24T07:59:13.620687+0000', 'last_fullsized': '2023-11-24T07:59:13.620687+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-24T07:59:13.620687+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-24T07:59:13.620687+0000', 'last_clean_scrub_stamp': '2023-11-24T07:59:13.620687+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-24T07:59:19.595 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.1b is not active+clean 2023-11-24T07:59:19.595 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.1b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 71, 'state': 'creating+peering', 'last_fresh': '2023-11-24T07:59:12.626522+0000', 'last_change': '2023-11-24T07:59:12.626522+0000', 'last_active': '2023-11-24T07:59:12.615544+0000', 'last_peered': '2023-11-24T07:59:12.615544+0000', 'last_clean': '2023-11-24T07:59:12.615544+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-24T07:59:12.626522+0000', 'last_undegraded': '2023-11-24T07:59:12.626522+0000', 'last_fullsized': '2023-11-24T07:59:12.626522+0000', 'mapping_epoch': 71, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 71, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-24T07:59:12.615544+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-24T07:59:12.615544+0000', 'last_clean_scrub_stamp': '2023-11-24T07:59:12.615544+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 2], 'acting': [4, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2023-11-24T07:59:19.595 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.1b is not active+clean 2023-11-24T07:59:19.595 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.1b', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-24T07:59:13.620687+0000', 'last_change': '2023-11-24T07:59:13.620687+0000', 'last_active': '2023-11-24T07:59:13.620687+0000', 'last_peered': '2023-11-24T07:59:13.620687+0000', 'last_clean': '2023-11-24T07:59:13.620687+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-24T07:59:13.620687+0000', 'last_undegraded': '2023-11-24T07:59:13.620687+0000', 'last_fullsized': '2023-11-24T07:59:13.620687+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-24T07:59:13.620687+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-24T07:59:13.620687+0000', 'last_clean_scrub_stamp': '2023-11-24T07:59:13.620687+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-24T07:59:19.595 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.1a is not active+clean 2023-11-24T07:59:19.595 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.1a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 71, 'state': 'creating+peering', 'last_fresh': '2023-11-24T07:59:12.624736+0000', 'last_change': '2023-11-24T07:59:12.624736+0000', 'last_active': '2023-11-24T07:59:12.615544+0000', 'last_peered': '2023-11-24T07:59:12.615544+0000', 'last_clean': '2023-11-24T07:59:12.615544+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-24T07:59:12.624736+0000', 'last_undegraded': '2023-11-24T07:59:12.624736+0000', 'last_fullsized': '2023-11-24T07:59:12.624736+0000', 'mapping_epoch': 71, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 71, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-24T07:59:12.615544+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-24T07:59:12.615544+0000', 'last_clean_scrub_stamp': '2023-11-24T07:59:12.615544+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [1, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2023-11-24T07:59:19.596 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.18 is not active+clean 2023-11-24T07:59:19.596 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.18', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-24T07:59:13.620687+0000', 'last_change': '2023-11-24T07:59:13.620687+0000', 'last_active': '2023-11-24T07:59:13.620687+0000', 'last_peered': '2023-11-24T07:59:13.620687+0000', 'last_clean': '2023-11-24T07:59:13.620687+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-24T07:59:13.620687+0000', 'last_undegraded': '2023-11-24T07:59:13.620687+0000', 'last_fullsized': '2023-11-24T07:59:13.620687+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-24T07:59:13.620687+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-24T07:59:13.620687+0000', 'last_clean_scrub_stamp': '2023-11-24T07:59:13.620687+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-24T07:59:19.596 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.19 is not active+clean 2023-11-24T07:59:19.596 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.19', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 71, 'state': 'creating+peering', 'last_fresh': '2023-11-24T07:59:12.629097+0000', 'last_change': '2023-11-24T07:59:12.629097+0000', 'last_active': '2023-11-24T07:59:12.615544+0000', 'last_peered': '2023-11-24T07:59:12.615544+0000', 'last_clean': '2023-11-24T07:59:12.615544+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-24T07:59:12.629097+0000', 'last_undegraded': '2023-11-24T07:59:12.629097+0000', 'last_fullsized': '2023-11-24T07:59:12.629097+0000', 'mapping_epoch': 71, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 71, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-24T07:59:12.615544+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-24T07:59:12.615544+0000', 'last_clean_scrub_stamp': '2023-11-24T07:59:12.615544+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 2], 'acting': [6, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2023-11-24T07:59:19.596 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.19 is not active+clean 2023-11-24T07:59:19.596 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.19', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-24T07:59:13.620687+0000', 'last_change': '2023-11-24T07:59:13.620687+0000', 'last_active': '2023-11-24T07:59:13.620687+0000', 'last_peered': '2023-11-24T07:59:13.620687+0000', 'last_clean': '2023-11-24T07:59:13.620687+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-24T07:59:13.620687+0000', 'last_undegraded': '2023-11-24T07:59:13.620687+0000', 'last_fullsized': '2023-11-24T07:59:13.620687+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-24T07:59:13.620687+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-24T07:59:13.620687+0000', 'last_clean_scrub_stamp': '2023-11-24T07:59:13.620687+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-24T07:59:19.596 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.18 is not active+clean 2023-11-24T07:59:19.596 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.18', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 71, 'state': 'creating+peering', 'last_fresh': '2023-11-24T07:59:12.625986+0000', 'last_change': '2023-11-24T07:59:12.625986+0000', 'last_active': '2023-11-24T07:59:12.615544+0000', 'last_peered': '2023-11-24T07:59:12.615544+0000', 'last_clean': '2023-11-24T07:59:12.615544+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-24T07:59:12.625986+0000', 'last_undegraded': '2023-11-24T07:59:12.625986+0000', 'last_fullsized': '2023-11-24T07:59:12.625986+0000', 'mapping_epoch': 71, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 71, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-24T07:59:12.615544+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-24T07:59:12.615544+0000', 'last_clean_scrub_stamp': '2023-11-24T07:59:12.615544+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 7], 'acting': [4, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2023-11-24T07:59:19.596 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.16 is not active+clean 2023-11-24T07:59:19.597 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.16', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-24T07:59:13.620687+0000', 'last_change': '2023-11-24T07:59:13.620687+0000', 'last_active': '2023-11-24T07:59:13.620687+0000', 'last_peered': '2023-11-24T07:59:13.620687+0000', 'last_clean': '2023-11-24T07:59:13.620687+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-24T07:59:13.620687+0000', 'last_undegraded': '2023-11-24T07:59:13.620687+0000', 'last_fullsized': '2023-11-24T07:59:13.620687+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-24T07:59:13.620687+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-24T07:59:13.620687+0000', 'last_clean_scrub_stamp': '2023-11-24T07:59:13.620687+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-24T07:59:19.597 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.17 is not active+clean 2023-11-24T07:59:19.597 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.17', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 71, 'state': 'creating+peering', 'last_fresh': '2023-11-24T07:59:12.633815+0000', 'last_change': '2023-11-24T07:59:12.633815+0000', 'last_active': '2023-11-24T07:59:12.615544+0000', 'last_peered': '2023-11-24T07:59:12.615544+0000', 'last_clean': '2023-11-24T07:59:12.615544+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-24T07:59:12.633815+0000', 'last_undegraded': '2023-11-24T07:59:12.633815+0000', 'last_fullsized': '2023-11-24T07:59:12.633815+0000', 'mapping_epoch': 71, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 71, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-24T07:59:12.615544+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-24T07:59:12.615544+0000', 'last_clean_scrub_stamp': '2023-11-24T07:59:12.615544+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 7], 'acting': [3, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2023-11-24T07:59:19.597 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.17 is not active+clean 2023-11-24T07:59:19.597 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.17', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-24T07:59:13.620687+0000', 'last_change': '2023-11-24T07:59:13.620687+0000', 'last_active': '2023-11-24T07:59:13.620687+0000', 'last_peered': '2023-11-24T07:59:13.620687+0000', 'last_clean': '2023-11-24T07:59:13.620687+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-24T07:59:13.620687+0000', 'last_undegraded': '2023-11-24T07:59:13.620687+0000', 'last_fullsized': '2023-11-24T07:59:13.620687+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-24T07:59:13.620687+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-24T07:59:13.620687+0000', 'last_clean_scrub_stamp': '2023-11-24T07:59:13.620687+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-24T07:59:19.597 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.16 is not active+clean 2023-11-24T07:59:19.597 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.16', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 71, 'state': 'creating+peering', 'last_fresh': '2023-11-24T07:59:12.627824+0000', 'last_change': '2023-11-24T07:59:12.627824+0000', 'last_active': '2023-11-24T07:59:12.615544+0000', 'last_peered': '2023-11-24T07:59:12.615544+0000', 'last_clean': '2023-11-24T07:59:12.615544+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-24T07:59:12.627824+0000', 'last_undegraded': '2023-11-24T07:59:12.627824+0000', 'last_fullsized': '2023-11-24T07:59:12.627824+0000', 'mapping_epoch': 71, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 71, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-24T07:59:12.615544+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-24T07:59:12.615544+0000', 'last_clean_scrub_stamp': '2023-11-24T07:59:12.615544+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 2], 'acting': [0, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2023-11-24T07:59:19.597 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.14 is not active+clean 2023-11-24T07:59:19.597 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.14', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-24T07:59:13.620687+0000', 'last_change': '2023-11-24T07:59:13.620687+0000', 'last_active': '2023-11-24T07:59:13.620687+0000', 'last_peered': '2023-11-24T07:59:13.620687+0000', 'last_clean': '2023-11-24T07:59:13.620687+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-24T07:59:13.620687+0000', 'last_undegraded': '2023-11-24T07:59:13.620687+0000', 'last_fullsized': '2023-11-24T07:59:13.620687+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-24T07:59:13.620687+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-24T07:59:13.620687+0000', 'last_clean_scrub_stamp': '2023-11-24T07:59:13.620687+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-24T07:59:19.597 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.15 is not active+clean 2023-11-24T07:59:19.598 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.15', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 71, 'state': 'creating+peering', 'last_fresh': '2023-11-24T07:59:12.626663+0000', 'last_change': '2023-11-24T07:59:12.626663+0000', 'last_active': '2023-11-24T07:59:12.615544+0000', 'last_peered': '2023-11-24T07:59:12.615544+0000', 'last_clean': '2023-11-24T07:59:12.615544+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-24T07:59:12.626663+0000', 'last_undegraded': '2023-11-24T07:59:12.626663+0000', 'last_fullsized': '2023-11-24T07:59:12.626663+0000', 'mapping_epoch': 71, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 71, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-24T07:59:12.615544+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-24T07:59:12.615544+0000', 'last_clean_scrub_stamp': '2023-11-24T07:59:12.615544+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 5], 'acting': [4, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2023-11-24T07:59:19.598 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.15 is not active+clean 2023-11-24T07:59:19.598 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.15', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-24T07:59:13.620687+0000', 'last_change': '2023-11-24T07:59:13.620687+0000', 'last_active': '2023-11-24T07:59:13.620687+0000', 'last_peered': '2023-11-24T07:59:13.620687+0000', 'last_clean': '2023-11-24T07:59:13.620687+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-24T07:59:13.620687+0000', 'last_undegraded': '2023-11-24T07:59:13.620687+0000', 'last_fullsized': '2023-11-24T07:59:13.620687+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-24T07:59:13.620687+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-24T07:59:13.620687+0000', 'last_clean_scrub_stamp': '2023-11-24T07:59:13.620687+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-24T07:59:19.598 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.14 is not active+clean 2023-11-24T07:59:19.598 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.14', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 71, 'state': 'creating+peering', 'last_fresh': '2023-11-24T07:59:12.629104+0000', 'last_change': '2023-11-24T07:59:12.629104+0000', 'last_active': '2023-11-24T07:59:12.615544+0000', 'last_peered': '2023-11-24T07:59:12.615544+0000', 'last_clean': '2023-11-24T07:59:12.615544+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-24T07:59:12.629104+0000', 'last_undegraded': '2023-11-24T07:59:12.629104+0000', 'last_fullsized': '2023-11-24T07:59:12.629104+0000', 'mapping_epoch': 71, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 71, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-24T07:59:12.615544+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-24T07:59:12.615544+0000', 'last_clean_scrub_stamp': '2023-11-24T07:59:12.615544+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 7], 'acting': [6, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2023-11-24T07:59:19.598 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.12 is not active+clean 2023-11-24T07:59:19.598 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.12', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-24T07:59:13.620687+0000', 'last_change': '2023-11-24T07:59:13.620687+0000', 'last_active': '2023-11-24T07:59:13.620687+0000', 'last_peered': '2023-11-24T07:59:13.620687+0000', 'last_clean': '2023-11-24T07:59:13.620687+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-24T07:59:13.620687+0000', 'last_undegraded': '2023-11-24T07:59:13.620687+0000', 'last_fullsized': '2023-11-24T07:59:13.620687+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-24T07:59:13.620687+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-24T07:59:13.620687+0000', 'last_clean_scrub_stamp': '2023-11-24T07:59:13.620687+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-24T07:59:19.599 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.13 is not active+clean 2023-11-24T07:59:19.599 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.13', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 71, 'state': 'creating+peering', 'last_fresh': '2023-11-24T07:59:12.629097+0000', 'last_change': '2023-11-24T07:59:12.629097+0000', 'last_active': '2023-11-24T07:59:12.615544+0000', 'last_peered': '2023-11-24T07:59:12.615544+0000', 'last_clean': '2023-11-24T07:59:12.615544+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-24T07:59:12.629097+0000', 'last_undegraded': '2023-11-24T07:59:12.629097+0000', 'last_fullsized': '2023-11-24T07:59:12.629097+0000', 'mapping_epoch': 71, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 71, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-24T07:59:12.615544+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-24T07:59:12.615544+0000', 'last_clean_scrub_stamp': '2023-11-24T07:59:12.615544+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 3], 'acting': [6, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2023-11-24T07:59:19.599 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.13 is not active+clean 2023-11-24T07:59:19.599 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.13', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-24T07:59:13.620687+0000', 'last_change': '2023-11-24T07:59:13.620687+0000', 'last_active': '2023-11-24T07:59:13.620687+0000', 'last_peered': '2023-11-24T07:59:13.620687+0000', 'last_clean': '2023-11-24T07:59:13.620687+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-24T07:59:13.620687+0000', 'last_undegraded': '2023-11-24T07:59:13.620687+0000', 'last_fullsized': '2023-11-24T07:59:13.620687+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-24T07:59:13.620687+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-24T07:59:13.620687+0000', 'last_clean_scrub_stamp': '2023-11-24T07:59:13.620687+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-24T07:59:19.599 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.12 is not active+clean 2023-11-24T07:59:19.599 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.12', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 71, 'state': 'creating+peering', 'last_fresh': '2023-11-24T07:59:12.624085+0000', 'last_change': '2023-11-24T07:59:12.624085+0000', 'last_active': '2023-11-24T07:59:12.615544+0000', 'last_peered': '2023-11-24T07:59:12.615544+0000', 'last_clean': '2023-11-24T07:59:12.615544+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-24T07:59:12.624085+0000', 'last_undegraded': '2023-11-24T07:59:12.624085+0000', 'last_fullsized': '2023-11-24T07:59:12.624085+0000', 'mapping_epoch': 71, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 71, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-24T07:59:12.615544+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-24T07:59:12.615544+0000', 'last_clean_scrub_stamp': '2023-11-24T07:59:12.615544+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 5], 'acting': [1, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2023-11-24T07:59:19.599 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.10 is not active+clean 2023-11-24T07:59:19.599 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.10', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-24T07:59:13.620687+0000', 'last_change': '2023-11-24T07:59:13.620687+0000', 'last_active': '2023-11-24T07:59:13.620687+0000', 'last_peered': '2023-11-24T07:59:13.620687+0000', 'last_clean': '2023-11-24T07:59:13.620687+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-24T07:59:13.620687+0000', 'last_undegraded': '2023-11-24T07:59:13.620687+0000', 'last_fullsized': '2023-11-24T07:59:13.620687+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-24T07:59:13.620687+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-24T07:59:13.620687+0000', 'last_clean_scrub_stamp': '2023-11-24T07:59:13.620687+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-24T07:59:19.599 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.11 is not active+clean 2023-11-24T07:59:19.599 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.11', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 71, 'state': 'creating+peering', 'last_fresh': '2023-11-24T07:59:12.624082+0000', 'last_change': '2023-11-24T07:59:12.624082+0000', 'last_active': '2023-11-24T07:59:12.615544+0000', 'last_peered': '2023-11-24T07:59:12.615544+0000', 'last_clean': '2023-11-24T07:59:12.615544+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-24T07:59:12.624082+0000', 'last_undegraded': '2023-11-24T07:59:12.624082+0000', 'last_fullsized': '2023-11-24T07:59:12.624082+0000', 'mapping_epoch': 71, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 71, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-24T07:59:12.615544+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-24T07:59:12.615544+0000', 'last_clean_scrub_stamp': '2023-11-24T07:59:12.615544+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 4], 'acting': [1, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2023-11-24T07:59:19.600 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.11 is not active+clean 2023-11-24T07:59:19.600 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.11', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-24T07:59:13.620687+0000', 'last_change': '2023-11-24T07:59:13.620687+0000', 'last_active': '2023-11-24T07:59:13.620687+0000', 'last_peered': '2023-11-24T07:59:13.620687+0000', 'last_clean': '2023-11-24T07:59:13.620687+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-24T07:59:13.620687+0000', 'last_undegraded': '2023-11-24T07:59:13.620687+0000', 'last_fullsized': '2023-11-24T07:59:13.620687+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-24T07:59:13.620687+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-24T07:59:13.620687+0000', 'last_clean_scrub_stamp': '2023-11-24T07:59:13.620687+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-24T07:59:19.600 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.10 is not active+clean 2023-11-24T07:59:19.600 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.10', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 71, 'state': 'creating+peering', 'last_fresh': '2023-11-24T07:59:12.624170+0000', 'last_change': '2023-11-24T07:59:12.624170+0000', 'last_active': '2023-11-24T07:59:12.615544+0000', 'last_peered': '2023-11-24T07:59:12.615544+0000', 'last_clean': '2023-11-24T07:59:12.615544+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-24T07:59:12.624170+0000', 'last_undegraded': '2023-11-24T07:59:12.624170+0000', 'last_fullsized': '2023-11-24T07:59:12.624170+0000', 'mapping_epoch': 71, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 71, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-24T07:59:12.615544+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-24T07:59:12.615544+0000', 'last_clean_scrub_stamp': '2023-11-24T07:59:12.615544+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [1, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2023-11-24T07:59:19.600 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.e is not active+clean 2023-11-24T07:59:19.600 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.e', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-24T07:59:13.620687+0000', 'last_change': '2023-11-24T07:59:13.620687+0000', 'last_active': '2023-11-24T07:59:13.620687+0000', 'last_peered': '2023-11-24T07:59:13.620687+0000', 'last_clean': '2023-11-24T07:59:13.620687+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-24T07:59:13.620687+0000', 'last_undegraded': '2023-11-24T07:59:13.620687+0000', 'last_fullsized': '2023-11-24T07:59:13.620687+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-24T07:59:13.620687+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-24T07:59:13.620687+0000', 'last_clean_scrub_stamp': '2023-11-24T07:59:13.620687+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-24T07:59:19.600 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.f is not active+clean 2023-11-24T07:59:19.600 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 71, 'state': 'creating+peering', 'last_fresh': '2023-11-24T07:59:12.633408+0000', 'last_change': '2023-11-24T07:59:12.633408+0000', 'last_active': '2023-11-24T07:59:12.615544+0000', 'last_peered': '2023-11-24T07:59:12.615544+0000', 'last_clean': '2023-11-24T07:59:12.615544+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-24T07:59:12.633408+0000', 'last_undegraded': '2023-11-24T07:59:12.633408+0000', 'last_fullsized': '2023-11-24T07:59:12.633408+0000', 'mapping_epoch': 71, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 71, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-24T07:59:12.615544+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-24T07:59:12.615544+0000', 'last_clean_scrub_stamp': '2023-11-24T07:59:12.615544+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 4], 'acting': [3, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2023-11-24T07:59:19.600 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.f is not active+clean 2023-11-24T07:59:19.600 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.f', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-24T07:59:13.620687+0000', 'last_change': '2023-11-24T07:59:13.620687+0000', 'last_active': '2023-11-24T07:59:13.620687+0000', 'last_peered': '2023-11-24T07:59:13.620687+0000', 'last_clean': '2023-11-24T07:59:13.620687+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-24T07:59:13.620687+0000', 'last_undegraded': '2023-11-24T07:59:13.620687+0000', 'last_fullsized': '2023-11-24T07:59:13.620687+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-24T07:59:13.620687+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-24T07:59:13.620687+0000', 'last_clean_scrub_stamp': '2023-11-24T07:59:13.620687+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-24T07:59:19.601 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.e is not active+clean 2023-11-24T07:59:19.601 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 71, 'state': 'creating+peering', 'last_fresh': '2023-11-24T07:59:12.627298+0000', 'last_change': '2023-11-24T07:59:12.627298+0000', 'last_active': '2023-11-24T07:59:12.615544+0000', 'last_peered': '2023-11-24T07:59:12.615544+0000', 'last_clean': '2023-11-24T07:59:12.615544+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-24T07:59:12.627298+0000', 'last_undegraded': '2023-11-24T07:59:12.627298+0000', 'last_fullsized': '2023-11-24T07:59:12.627298+0000', 'mapping_epoch': 71, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 71, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-24T07:59:12.615544+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-24T07:59:12.615544+0000', 'last_clean_scrub_stamp': '2023-11-24T07:59:12.615544+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2023-11-24T07:59:19.601 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.c is not active+clean 2023-11-24T07:59:19.601 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.c', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-24T07:59:13.620687+0000', 'last_change': '2023-11-24T07:59:13.620687+0000', 'last_active': '2023-11-24T07:59:13.620687+0000', 'last_peered': '2023-11-24T07:59:13.620687+0000', 'last_clean': '2023-11-24T07:59:13.620687+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-24T07:59:13.620687+0000', 'last_undegraded': '2023-11-24T07:59:13.620687+0000', 'last_fullsized': '2023-11-24T07:59:13.620687+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-24T07:59:13.620687+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-24T07:59:13.620687+0000', 'last_clean_scrub_stamp': '2023-11-24T07:59:13.620687+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-24T07:59:19.601 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.d is not active+clean 2023-11-24T07:59:19.601 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 71, 'state': 'creating+peering', 'last_fresh': '2023-11-24T07:59:12.627664+0000', 'last_change': '2023-11-24T07:59:12.627664+0000', 'last_active': '2023-11-24T07:59:12.615544+0000', 'last_peered': '2023-11-24T07:59:12.615544+0000', 'last_clean': '2023-11-24T07:59:12.615544+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-24T07:59:12.627664+0000', 'last_undegraded': '2023-11-24T07:59:12.627664+0000', 'last_fullsized': '2023-11-24T07:59:12.627664+0000', 'mapping_epoch': 71, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 71, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-24T07:59:12.615544+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-24T07:59:12.615544+0000', 'last_clean_scrub_stamp': '2023-11-24T07:59:12.615544+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 3], 'acting': [0, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2023-11-24T07:59:19.601 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.d is not active+clean 2023-11-24T07:59:19.601 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.d', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-24T07:59:13.620687+0000', 'last_change': '2023-11-24T07:59:13.620687+0000', 'last_active': '2023-11-24T07:59:13.620687+0000', 'last_peered': '2023-11-24T07:59:13.620687+0000', 'last_clean': '2023-11-24T07:59:13.620687+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-24T07:59:13.620687+0000', 'last_undegraded': '2023-11-24T07:59:13.620687+0000', 'last_fullsized': '2023-11-24T07:59:13.620687+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-24T07:59:13.620687+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-24T07:59:13.620687+0000', 'last_clean_scrub_stamp': '2023-11-24T07:59:13.620687+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-24T07:59:19.601 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.c is not active+clean 2023-11-24T07:59:19.601 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 71, 'state': 'creating+peering', 'last_fresh': '2023-11-24T07:59:12.627260+0000', 'last_change': '2023-11-24T07:59:12.627260+0000', 'last_active': '2023-11-24T07:59:12.615544+0000', 'last_peered': '2023-11-24T07:59:12.615544+0000', 'last_clean': '2023-11-24T07:59:12.615544+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-24T07:59:12.627260+0000', 'last_undegraded': '2023-11-24T07:59:12.627260+0000', 'last_fullsized': '2023-11-24T07:59:12.627260+0000', 'mapping_epoch': 71, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 71, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-24T07:59:12.615544+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-24T07:59:12.615544+0000', 'last_clean_scrub_stamp': '2023-11-24T07:59:12.615544+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 3], 'acting': [2, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2023-11-24T07:59:19.602 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.b is not active+clean 2023-11-24T07:59:19.602 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.b', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-24T07:59:13.620687+0000', 'last_change': '2023-11-24T07:59:13.620687+0000', 'last_active': '2023-11-24T07:59:13.620687+0000', 'last_peered': '2023-11-24T07:59:13.620687+0000', 'last_clean': '2023-11-24T07:59:13.620687+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-24T07:59:13.620687+0000', 'last_undegraded': '2023-11-24T07:59:13.620687+0000', 'last_fullsized': '2023-11-24T07:59:13.620687+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-24T07:59:13.620687+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-24T07:59:13.620687+0000', 'last_clean_scrub_stamp': '2023-11-24T07:59:13.620687+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-24T07:59:19.602 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.a is not active+clean 2023-11-24T07:59:19.602 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 71, 'state': 'creating+peering', 'last_fresh': '2023-11-24T07:59:12.627164+0000', 'last_change': '2023-11-24T07:59:12.627164+0000', 'last_active': '2023-11-24T07:59:12.615544+0000', 'last_peered': '2023-11-24T07:59:12.615544+0000', 'last_clean': '2023-11-24T07:59:12.615544+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-24T07:59:12.627164+0000', 'last_undegraded': '2023-11-24T07:59:12.627164+0000', 'last_fullsized': '2023-11-24T07:59:12.627164+0000', 'mapping_epoch': 71, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 71, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-24T07:59:12.615544+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-24T07:59:12.615544+0000', 'last_clean_scrub_stamp': '2023-11-24T07:59:12.615544+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 4], 'acting': [2, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2023-11-24T07:59:19.602 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.8 is not active+clean 2023-11-24T07:59:19.602 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.8', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-24T07:59:13.620687+0000', 'last_change': '2023-11-24T07:59:13.620687+0000', 'last_active': '2023-11-24T07:59:13.620687+0000', 'last_peered': '2023-11-24T07:59:13.620687+0000', 'last_clean': '2023-11-24T07:59:13.620687+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-24T07:59:13.620687+0000', 'last_undegraded': '2023-11-24T07:59:13.620687+0000', 'last_fullsized': '2023-11-24T07:59:13.620687+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-24T07:59:13.620687+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-24T07:59:13.620687+0000', 'last_clean_scrub_stamp': '2023-11-24T07:59:13.620687+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-24T07:59:19.602 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.9 is not active+clean 2023-11-24T07:59:19.602 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.9', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 71, 'state': 'creating+peering', 'last_fresh': '2023-11-24T07:59:12.628577+0000', 'last_change': '2023-11-24T07:59:12.628577+0000', 'last_active': '2023-11-24T07:59:12.615544+0000', 'last_peered': '2023-11-24T07:59:12.615544+0000', 'last_clean': '2023-11-24T07:59:12.615544+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-24T07:59:12.628577+0000', 'last_undegraded': '2023-11-24T07:59:12.628577+0000', 'last_fullsized': '2023-11-24T07:59:12.628577+0000', 'mapping_epoch': 71, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 71, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-24T07:59:12.615544+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-24T07:59:12.615544+0000', 'last_clean_scrub_stamp': '2023-11-24T07:59:12.615544+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 4], 'acting': [0, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2023-11-24T07:59:19.602 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.9 is not active+clean 2023-11-24T07:59:19.603 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.9', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-24T07:59:13.620687+0000', 'last_change': '2023-11-24T07:59:13.620687+0000', 'last_active': '2023-11-24T07:59:13.620687+0000', 'last_peered': '2023-11-24T07:59:13.620687+0000', 'last_clean': '2023-11-24T07:59:13.620687+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-24T07:59:13.620687+0000', 'last_undegraded': '2023-11-24T07:59:13.620687+0000', 'last_fullsized': '2023-11-24T07:59:13.620687+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-24T07:59:13.620687+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-24T07:59:13.620687+0000', 'last_clean_scrub_stamp': '2023-11-24T07:59:13.620687+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-24T07:59:19.603 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.8 is not active+clean 2023-11-24T07:59:19.603 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.8', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 71, 'state': 'creating+peering', 'last_fresh': '2023-11-24T07:59:12.622958+0000', 'last_change': '2023-11-24T07:59:12.622958+0000', 'last_active': '2023-11-24T07:59:12.615544+0000', 'last_peered': '2023-11-24T07:59:12.615544+0000', 'last_clean': '2023-11-24T07:59:12.615544+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-24T07:59:12.622958+0000', 'last_undegraded': '2023-11-24T07:59:12.622958+0000', 'last_fullsized': '2023-11-24T07:59:12.622958+0000', 'mapping_epoch': 71, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 71, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-24T07:59:12.615544+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-24T07:59:12.615544+0000', 'last_clean_scrub_stamp': '2023-11-24T07:59:12.615544+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 0], 'acting': [5, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2023-11-24T07:59:19.603 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.6 is not active+clean 2023-11-24T07:59:19.603 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.6', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-24T07:59:13.620687+0000', 'last_change': '2023-11-24T07:59:13.620687+0000', 'last_active': '2023-11-24T07:59:13.620687+0000', 'last_peered': '2023-11-24T07:59:13.620687+0000', 'last_clean': '2023-11-24T07:59:13.620687+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-24T07:59:13.620687+0000', 'last_undegraded': '2023-11-24T07:59:13.620687+0000', 'last_fullsized': '2023-11-24T07:59:13.620687+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-24T07:59:13.620687+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-24T07:59:13.620687+0000', 'last_clean_scrub_stamp': '2023-11-24T07:59:13.620687+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-24T07:59:19.603 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.7 is not active+clean 2023-11-24T07:59:19.603 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.7', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 71, 'state': 'creating+peering', 'last_fresh': '2023-11-24T07:59:12.632816+0000', 'last_change': '2023-11-24T07:59:12.632816+0000', 'last_active': '2023-11-24T07:59:12.615544+0000', 'last_peered': '2023-11-24T07:59:12.615544+0000', 'last_clean': '2023-11-24T07:59:12.615544+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-24T07:59:12.632816+0000', 'last_undegraded': '2023-11-24T07:59:12.632816+0000', 'last_fullsized': '2023-11-24T07:59:12.632816+0000', 'mapping_epoch': 71, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 71, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-24T07:59:12.615544+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-24T07:59:12.615544+0000', 'last_clean_scrub_stamp': '2023-11-24T07:59:12.615544+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 4], 'acting': [3, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2023-11-24T07:59:19.603 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.7 is not active+clean 2023-11-24T07:59:19.603 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.7', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-24T07:59:13.620687+0000', 'last_change': '2023-11-24T07:59:13.620687+0000', 'last_active': '2023-11-24T07:59:13.620687+0000', 'last_peered': '2023-11-24T07:59:13.620687+0000', 'last_clean': '2023-11-24T07:59:13.620687+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-24T07:59:13.620687+0000', 'last_undegraded': '2023-11-24T07:59:13.620687+0000', 'last_fullsized': '2023-11-24T07:59:13.620687+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-24T07:59:13.620687+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-24T07:59:13.620687+0000', 'last_clean_scrub_stamp': '2023-11-24T07:59:13.620687+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-24T07:59:19.603 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.6 is not active+clean 2023-11-24T07:59:19.604 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.6', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 71, 'state': 'creating+peering', 'last_fresh': '2023-11-24T07:59:12.632780+0000', 'last_change': '2023-11-24T07:59:12.632780+0000', 'last_active': '2023-11-24T07:59:12.615544+0000', 'last_peered': '2023-11-24T07:59:12.615544+0000', 'last_clean': '2023-11-24T07:59:12.615544+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-24T07:59:12.632780+0000', 'last_undegraded': '2023-11-24T07:59:12.632780+0000', 'last_fullsized': '2023-11-24T07:59:12.632780+0000', 'mapping_epoch': 71, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 71, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-24T07:59:12.615544+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-24T07:59:12.615544+0000', 'last_clean_scrub_stamp': '2023-11-24T07:59:12.615544+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 1], 'acting': [3, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2023-11-24T07:59:19.604 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.4 is not active+clean 2023-11-24T07:59:19.604 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.4', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-24T07:59:13.620687+0000', 'last_change': '2023-11-24T07:59:13.620687+0000', 'last_active': '2023-11-24T07:59:13.620687+0000', 'last_peered': '2023-11-24T07:59:13.620687+0000', 'last_clean': '2023-11-24T07:59:13.620687+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-24T07:59:13.620687+0000', 'last_undegraded': '2023-11-24T07:59:13.620687+0000', 'last_fullsized': '2023-11-24T07:59:13.620687+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-24T07:59:13.620687+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-24T07:59:13.620687+0000', 'last_clean_scrub_stamp': '2023-11-24T07:59:13.620687+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-24T07:59:19.604 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.5 is not active+clean 2023-11-24T07:59:19.604 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.5', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 71, 'state': 'creating+peering', 'last_fresh': '2023-11-24T07:59:12.626089+0000', 'last_change': '2023-11-24T07:59:12.626089+0000', 'last_active': '2023-11-24T07:59:12.615544+0000', 'last_peered': '2023-11-24T07:59:12.615544+0000', 'last_clean': '2023-11-24T07:59:12.615544+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-24T07:59:12.626089+0000', 'last_undegraded': '2023-11-24T07:59:12.626089+0000', 'last_fullsized': '2023-11-24T07:59:12.626089+0000', 'mapping_epoch': 71, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 71, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-24T07:59:12.615544+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-24T07:59:12.615544+0000', 'last_clean_scrub_stamp': '2023-11-24T07:59:12.615544+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 6], 'acting': [4, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2023-11-24T07:59:19.604 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.5 is not active+clean 2023-11-24T07:59:19.605 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.5', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-24T07:59:13.620687+0000', 'last_change': '2023-11-24T07:59:13.620687+0000', 'last_active': '2023-11-24T07:59:13.620687+0000', 'last_peered': '2023-11-24T07:59:13.620687+0000', 'last_clean': '2023-11-24T07:59:13.620687+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-24T07:59:13.620687+0000', 'last_undegraded': '2023-11-24T07:59:13.620687+0000', 'last_fullsized': '2023-11-24T07:59:13.620687+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-24T07:59:13.620687+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-24T07:59:13.620687+0000', 'last_clean_scrub_stamp': '2023-11-24T07:59:13.620687+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-24T07:59:19.605 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.4 is not active+clean 2023-11-24T07:59:19.605 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.4', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 71, 'state': 'creating+peering', 'last_fresh': '2023-11-24T07:59:12.626088+0000', 'last_change': '2023-11-24T07:59:12.626088+0000', 'last_active': '2023-11-24T07:59:12.615544+0000', 'last_peered': '2023-11-24T07:59:12.615544+0000', 'last_clean': '2023-11-24T07:59:12.615544+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-24T07:59:12.626088+0000', 'last_undegraded': '2023-11-24T07:59:12.626088+0000', 'last_fullsized': '2023-11-24T07:59:12.626088+0000', 'mapping_epoch': 71, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 71, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-24T07:59:12.615544+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-24T07:59:12.615544+0000', 'last_clean_scrub_stamp': '2023-11-24T07:59:12.615544+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 5], 'acting': [4, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2023-11-24T07:59:19.605 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.2 is not active+clean 2023-11-24T07:59:19.605 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.2', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-24T07:59:13.620687+0000', 'last_change': '2023-11-24T07:59:13.620687+0000', 'last_active': '2023-11-24T07:59:13.620687+0000', 'last_peered': '2023-11-24T07:59:13.620687+0000', 'last_clean': '2023-11-24T07:59:13.620687+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-24T07:59:13.620687+0000', 'last_undegraded': '2023-11-24T07:59:13.620687+0000', 'last_fullsized': '2023-11-24T07:59:13.620687+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-24T07:59:13.620687+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-24T07:59:13.620687+0000', 'last_clean_scrub_stamp': '2023-11-24T07:59:13.620687+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-24T07:59:19.605 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.3 is not active+clean 2023-11-24T07:59:19.605 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.3', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 71, 'state': 'creating+peering', 'last_fresh': '2023-11-24T07:59:12.626018+0000', 'last_change': '2023-11-24T07:59:12.626018+0000', 'last_active': '2023-11-24T07:59:12.615544+0000', 'last_peered': '2023-11-24T07:59:12.615544+0000', 'last_clean': '2023-11-24T07:59:12.615544+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-24T07:59:12.626018+0000', 'last_undegraded': '2023-11-24T07:59:12.626018+0000', 'last_fullsized': '2023-11-24T07:59:12.626018+0000', 'mapping_epoch': 71, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 71, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-24T07:59:12.615544+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-24T07:59:12.615544+0000', 'last_clean_scrub_stamp': '2023-11-24T07:59:12.615544+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 2], 'acting': [4, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2023-11-24T07:59:19.605 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.3 is not active+clean 2023-11-24T07:59:19.605 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.3', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-24T07:59:13.620687+0000', 'last_change': '2023-11-24T07:59:13.620687+0000', 'last_active': '2023-11-24T07:59:13.620687+0000', 'last_peered': '2023-11-24T07:59:13.620687+0000', 'last_clean': '2023-11-24T07:59:13.620687+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-24T07:59:13.620687+0000', 'last_undegraded': '2023-11-24T07:59:13.620687+0000', 'last_fullsized': '2023-11-24T07:59:13.620687+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-24T07:59:13.620687+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-24T07:59:13.620687+0000', 'last_clean_scrub_stamp': '2023-11-24T07:59:13.620687+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-24T07:59:19.606 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.2 is not active+clean 2023-11-24T07:59:19.606 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.2', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 71, 'state': 'creating+peering', 'last_fresh': '2023-11-24T07:59:12.622957+0000', 'last_change': '2023-11-24T07:59:12.622957+0000', 'last_active': '2023-11-24T07:59:12.615544+0000', 'last_peered': '2023-11-24T07:59:12.615544+0000', 'last_clean': '2023-11-24T07:59:12.615544+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-24T07:59:12.622957+0000', 'last_undegraded': '2023-11-24T07:59:12.622957+0000', 'last_fullsized': '2023-11-24T07:59:12.622957+0000', 'mapping_epoch': 71, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 71, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-24T07:59:12.615544+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-24T07:59:12.615544+0000', 'last_clean_scrub_stamp': '2023-11-24T07:59:12.615544+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 2], 'acting': [5, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2023-11-24T07:59:19.606 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.0 is not active+clean 2023-11-24T07:59:19.606 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.0', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-24T07:59:13.620687+0000', 'last_change': '2023-11-24T07:59:13.620687+0000', 'last_active': '2023-11-24T07:59:13.620687+0000', 'last_peered': '2023-11-24T07:59:13.620687+0000', 'last_clean': '2023-11-24T07:59:13.620687+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-24T07:59:13.620687+0000', 'last_undegraded': '2023-11-24T07:59:13.620687+0000', 'last_fullsized': '2023-11-24T07:59:13.620687+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-24T07:59:13.620687+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-24T07:59:13.620687+0000', 'last_clean_scrub_stamp': '2023-11-24T07:59:13.620687+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-24T07:59:19.606 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.1 is not active+clean 2023-11-24T07:59:19.606 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.1', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 71, 'state': 'creating+peering', 'last_fresh': '2023-11-24T07:59:12.628166+0000', 'last_change': '2023-11-24T07:59:12.628166+0000', 'last_active': '2023-11-24T07:59:12.615544+0000', 'last_peered': '2023-11-24T07:59:12.615544+0000', 'last_clean': '2023-11-24T07:59:12.615544+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-24T07:59:12.628166+0000', 'last_undegraded': '2023-11-24T07:59:12.628166+0000', 'last_fullsized': '2023-11-24T07:59:12.628166+0000', 'mapping_epoch': 71, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 71, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-24T07:59:12.615544+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-24T07:59:12.615544+0000', 'last_clean_scrub_stamp': '2023-11-24T07:59:12.615544+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 2], 'acting': [0, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2023-11-24T07:59:19.606 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.1 is not active+clean 2023-11-24T07:59:19.606 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.1', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-24T07:59:13.620687+0000', 'last_change': '2023-11-24T07:59:13.620687+0000', 'last_active': '2023-11-24T07:59:13.620687+0000', 'last_peered': '2023-11-24T07:59:13.620687+0000', 'last_clean': '2023-11-24T07:59:13.620687+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-24T07:59:13.620687+0000', 'last_undegraded': '2023-11-24T07:59:13.620687+0000', 'last_fullsized': '2023-11-24T07:59:13.620687+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-24T07:59:13.620687+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-24T07:59:13.620687+0000', 'last_clean_scrub_stamp': '2023-11-24T07:59:13.620687+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-24T07:59:19.606 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.0 is not active+clean 2023-11-24T07:59:19.606 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.0', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 71, 'state': 'creating+peering', 'last_fresh': '2023-11-24T07:59:12.627478+0000', 'last_change': '2023-11-24T07:59:12.627478+0000', 'last_active': '2023-11-24T07:59:12.615544+0000', 'last_peered': '2023-11-24T07:59:12.615544+0000', 'last_clean': '2023-11-24T07:59:12.615544+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-24T07:59:12.627478+0000', 'last_undegraded': '2023-11-24T07:59:12.627478+0000', 'last_fullsized': '2023-11-24T07:59:12.627478+0000', 'mapping_epoch': 71, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 71, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-24T07:59:12.615544+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-24T07:59:12.615544+0000', 'last_clean_scrub_stamp': '2023-11-24T07:59:12.615544+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 5], 'acting': [2, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2023-11-24T07:59:19.607 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.a is not active+clean 2023-11-24T07:59:19.607 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.a', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-24T07:59:13.620687+0000', 'last_change': '2023-11-24T07:59:13.620687+0000', 'last_active': '2023-11-24T07:59:13.620687+0000', 'last_peered': '2023-11-24T07:59:13.620687+0000', 'last_clean': '2023-11-24T07:59:13.620687+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-24T07:59:13.620687+0000', 'last_undegraded': '2023-11-24T07:59:13.620687+0000', 'last_fullsized': '2023-11-24T07:59:13.620687+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-24T07:59:13.620687+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-24T07:59:13.620687+0000', 'last_clean_scrub_stamp': '2023-11-24T07:59:13.620687+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-24T07:59:19.607 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.b is not active+clean 2023-11-24T07:59:19.607 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 71, 'state': 'creating+peering', 'last_fresh': '2023-11-24T07:59:12.624115+0000', 'last_change': '2023-11-24T07:59:12.624115+0000', 'last_active': '2023-11-24T07:59:12.615544+0000', 'last_peered': '2023-11-24T07:59:12.615544+0000', 'last_clean': '2023-11-24T07:59:12.615544+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-24T07:59:12.624115+0000', 'last_undegraded': '2023-11-24T07:59:12.624115+0000', 'last_fullsized': '2023-11-24T07:59:12.624115+0000', 'mapping_epoch': 71, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 71, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-24T07:59:12.615544+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-24T07:59:12.615544+0000', 'last_clean_scrub_stamp': '2023-11-24T07:59:12.615544+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 4], 'acting': [1, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2023-11-24T07:59:19.607 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2023-11-24T07:59:19.616 INFO:tasks.ceph.mds.c.smithi059.stderr:2023-11-24T07:59:19.615+0000 7f14bc9a5700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-11-24T07:59:19.741 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:19.738+0000 7fd890ebf700 1 Processor -- start 2023-11-24T07:59:19.741 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:19.738+0000 7fd890ebf700 1 -- start start 2023-11-24T07:59:19.743 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:19.739+0000 7fd890ebf700 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fd88c06fc50 0x7fd88c06eef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:19.744 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:19.739+0000 7fd890ebf700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fd88c0b0e90 0x7fd88c06ca10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:19.744 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:19.739+0000 7fd890ebf700 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd88c0d1230 con 0x7fd88c15e1c0 2023-11-24T07:59:19.744 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:19.739+0000 7fd890ebf700 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd88c0d0c70 con 0x7fd88c0b0e90 2023-11-24T07:59:19.744 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:19.739+0000 7fd890ebf700 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd88c0d0f50 con 0x7fd88c06fc50 2023-11-24T07:59:19.745 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:19.740+0000 7fd88ad9d700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fd88c0b0e90 0x7fd88c06ca10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:19.745 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:19.740+0000 7fd88ad9d700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fd88c0b0e90 0x7fd88c06ca10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:54906/0 (socket says 172.21.15.59:54906) 2023-11-24T07:59:19.745 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:19.740+0000 7fd88ad9d700 1 -- 172.21.15.59:0/231714788 learned_addr learned my addr 172.21.15.59:0/231714788 (peer_addr_for_me v2:172.21.15.59:0/0) 2023-11-24T07:59:19.745 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:19.740+0000 7fd889d9b700 1 --2- 172.21.15.59:0/231714788 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fd88c06fc50 0x7fd88c06eef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:19.746 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:19.740+0000 7fd88ad9d700 1 -- 172.21.15.59:0/231714788 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fd88c06fc50 msgr2=0x7fd88c06eef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T07:59:19.747 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:19.740+0000 7fd88ad9d700 1 --2- 172.21.15.59:0/231714788 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fd88c06fc50 0x7fd88c06eef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:19.747 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:19.740+0000 7fd88ad9d700 1 -- 172.21.15.59:0/231714788 >> v1:172.21.15.76:6789/0 conn(0x7fd88c15e1c0 legacy=0x7fd88c06f540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T07:59:19.748 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:19.740+0000 7fd88ad9d700 1 -- 172.21.15.59:0/231714788 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd88c06c210 con 0x7fd88c0b0e90 2023-11-24T07:59:19.748 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:19.740+0000 7fd889d9b700 1 --2- 172.21.15.59:0/231714788 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fd88c06fc50 0x7fd88c06eef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T07:59:19.748 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:19.740+0000 7fd88ad9d700 1 --2- 172.21.15.59:0/231714788 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd88c0b0e90 0x7fd88c06ca10 secure :-1 s=READY pgs=873 cs=0 l=1 rev1=1 crypto rx=0x7fd884009e50 tx=0x7fd88400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=667d9af6669ca021 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:59:19.748 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:19.740+0000 7fd88959a700 1 -- 172.21.15.59:0/231714788 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd88400e070 con 0x7fd88c0b0e90 2023-11-24T07:59:19.748 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:19.740+0000 7fd88959a700 1 -- 172.21.15.59:0/231714788 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd884003a20 con 0x7fd88c0b0e90 2023-11-24T07:59:19.748 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:19.740+0000 7fd88959a700 1 -- 172.21.15.59:0/231714788 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd884003dd0 con 0x7fd88c0b0e90 2023-11-24T07:59:19.748 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:19.741+0000 7fd890ebf700 1 -- 172.21.15.59:0/231714788 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd88c0b0e90 msgr2=0x7fd88c06ca10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:19.748 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:19.741+0000 7fd890ebf700 1 --2- 172.21.15.59:0/231714788 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd88c0b0e90 0x7fd88c06ca10 secure :-1 s=READY pgs=873 cs=0 l=1 rev1=1 crypto rx=0x7fd884009e50 tx=0x7fd88400b330 comp rx=0 tx=0).stop 2023-11-24T07:59:19.748 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:19.741+0000 7fd890ebf700 1 -- 172.21.15.59:0/231714788 shutdown_connections 2023-11-24T07:59:19.748 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:19.741+0000 7fd890ebf700 1 --2- 172.21.15.59:0/231714788 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd88c0b0e90 0x7fd88c06ca10 unknown :-1 s=CLOSED pgs=873 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:19.748 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:19.741+0000 7fd890ebf700 1 --2- 172.21.15.59:0/231714788 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fd88c06fc50 0x7fd88c06eef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:19.748 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:19.741+0000 7fd890ebf700 1 -- 172.21.15.59:0/231714788 >> 172.21.15.59:0/231714788 conn(0x7fd88c0b3720 msgr2=0x7fd88c0b3b30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:59:19.748 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:19.741+0000 7fd890ebf700 1 -- 172.21.15.59:0/231714788 shutdown_connections 2023-11-24T07:59:19.749 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:19.741+0000 7fd890ebf700 1 -- 172.21.15.59:0/231714788 wait complete. 2023-11-24T07:59:19.751 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:19.743+0000 7fd890ebf700 1 Processor -- start 2023-11-24T07:59:19.751 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:19.743+0000 7fd890ebf700 1 -- start start 2023-11-24T07:59:19.751 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:19.744+0000 7fd890ebf700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd88c06fc50 0x7fd88c0ce980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:19.751 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:19.744+0000 7fd890ebf700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd88c0b0e90 0x7fd88c1f7360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:19.752 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:19.744+0000 7fd890ebf700 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fd88c15e1c0 0x7fd88c15b070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:19.752 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:19.744+0000 7fd890ebf700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd88c0cc3c0 con 0x7fd88c06fc50 2023-11-24T07:59:19.752 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:19.744+0000 7fd890ebf700 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd88c1f78a0 con 0x7fd88c0b0e90 2023-11-24T07:59:19.752 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:19.744+0000 7fd890ebf700 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd88c1f7b40 con 0x7fd88c15e1c0 2023-11-24T07:59:19.752 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:19.744+0000 7fd88a59c700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd88c06fc50 0x7fd88c0ce980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:19.752 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:19.744+0000 7fd88a59c700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd88c06fc50 0x7fd88c0ce980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:54908/0 (socket says 172.21.15.59:54908) 2023-11-24T07:59:19.752 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:19.744+0000 7fd88a59c700 1 -- 172.21.15.59:0/2670709288 learned_addr learned my addr 172.21.15.59:0/2670709288 (peer_addr_for_me v2:172.21.15.59:0/0) 2023-11-24T07:59:19.752 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:19.744+0000 7fd889d9b700 1 --2- 172.21.15.59:0/2670709288 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd88c0b0e90 0x7fd88c1f7360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:19.752 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:19.744+0000 7fd88ad9d700 1 --2- 172.21.15.59:0/2670709288 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fd88c15e1c0 0x7fd88c15b070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:19.752 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:19.744+0000 7fd88a59c700 1 -- 172.21.15.59:0/2670709288 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fd88c15e1c0 msgr2=0x7fd88c15b070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:19.752 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:19.744+0000 7fd88a59c700 1 --2- 172.21.15.59:0/2670709288 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fd88c15e1c0 0x7fd88c15b070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:19.752 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:19.744+0000 7fd88a59c700 1 -- 172.21.15.59:0/2670709288 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd88c0b0e90 msgr2=0x7fd88c1f7360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:19.753 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:19.744+0000 7fd88a59c700 1 --2- 172.21.15.59:0/2670709288 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd88c0b0e90 0x7fd88c1f7360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:19.753 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:19.744+0000 7fd88a59c700 1 -- 172.21.15.59:0/2670709288 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd884009b30 con 0x7fd88c06fc50 2023-11-24T07:59:19.753 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:19.744+0000 7fd88ad9d700 1 --2- 172.21.15.59:0/2670709288 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fd88c15e1c0 0x7fd88c15b070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-24T07:59:19.753 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:19.744+0000 7fd889d9b700 1 --2- 172.21.15.59:0/2670709288 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd88c0b0e90 0x7fd88c1f7360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-24T07:59:19.753 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:19.744+0000 7fd88a59c700 1 --2- 172.21.15.59:0/2670709288 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd88c06fc50 0x7fd88c0ce980 secure :-1 s=READY pgs=874 cs=0 l=1 rev1=1 crypto rx=0x7fd87400b4a0 tx=0x7fd87400b860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:59:19.753 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:19.744+0000 7fd882ffd700 1 -- 172.21.15.59:0/2670709288 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd874011830 con 0x7fd88c06fc50 2023-11-24T07:59:19.753 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:19.744+0000 7fd882ffd700 1 -- 172.21.15.59:0/2670709288 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd874011990 con 0x7fd88c06fc50 2023-11-24T07:59:19.753 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:19.744+0000 7fd890ebf700 1 -- 172.21.15.59:0/2670709288 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd88c1f7e20 con 0x7fd88c06fc50 2023-11-24T07:59:19.753 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:19.744+0000 7fd882ffd700 1 -- 172.21.15.59:0/2670709288 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd874011d40 con 0x7fd88c06fc50 2023-11-24T07:59:19.753 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:19.744+0000 7fd890ebf700 1 -- 172.21.15.59:0/2670709288 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd88c0cc5d0 con 0x7fd88c06fc50 2023-11-24T07:59:19.753 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:19.745+0000 7fd890ebf700 1 -- 172.21.15.59:0/2670709288 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd88c068670 con 0x7fd88c06fc50 2023-11-24T07:59:19.753 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:19.746+0000 7fd882ffd700 1 -- 172.21.15.59:0/2670709288 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 10) v1 ==== 89588+0+0 (secure 0 0 0) 0x7fd87401a400 con 0x7fd88c06fc50 2023-11-24T07:59:19.754 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:19.746+0000 7fd882ffd700 1 --2- 172.21.15.59:0/2670709288 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7fd86c06b2c0 0x7fd86c06d780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:19.754 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:19.746+0000 7fd882ffd700 1 -- 172.21.15.59:0/2670709288 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fd8740899e0 con 0x7fd88c06fc50 2023-11-24T07:59:19.754 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:19.747+0000 7fd889d9b700 1 --2- 172.21.15.59:0/2670709288 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7fd86c06b2c0 0x7fd86c06d780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:19.754 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:19.747+0000 7fd889d9b700 1 --2- 172.21.15.59:0/2670709288 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7fd86c06b2c0 0x7fd86c06d780 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7fd88c0cbe80 tx=0x7fd87c008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:59:19.754 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:19.751+0000 7fd882ffd700 1 -- 172.21.15.59:0/2670709288 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fd874054540 con 0x7fd88c06fc50 2023-11-24T07:59:20.038 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:20.036+0000 7fd890ebf700 1 -- 172.21.15.59:0/2670709288 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) v1 -- 0x7fd88c0d1670 con 0x7fd88c06fc50 2023-11-24T07:59:20.038 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:20.037+0000 7fd882ffd700 1 -- 172.21.15.59:0/2670709288 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) v1 ==== 72+0+1527 (secure 0 0 0) 0x7fd874015610 con 0x7fd88c06fc50 2023-11-24T07:59:20.039 INFO:teuthology.orchestra.run.smithi059.stdout: 2023-11-24T07:59:20.039 INFO:teuthology.orchestra.run.smithi059.stdout:{"fsid":"9f0e8741-b8f9-48d5-a2c3-09f45903303d","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":193,"monmap":{"epoch":1,"min_mon_release_name":"reef","num_mons":3},"osdmap":{"epoch":74,"num_osds":8,"num_up_osds":8,"osd_up_since":1700812577,"num_in_osds":8,"osd_in_since":1700812569,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"unknown","count":64},{"state_name":"creating+peering","count":32},{"state_name":"active+clean","count":1}],"num_pgs":97,"num_pools":3,"num_objects":2,"data_bytes":590368,"bytes_used":229462016,"bytes_avail":772864651264,"bytes_total":773094113280,"unknown_pgs_ratio":0.65979379415512085,"inactive_pgs_ratio":0.32989689707756042},"fsmap":{"epoch":44,"id":5,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":5,"rank":0,"name":"c","status":"up:active","gid":6552}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2023-11-24T07:59:02.336121+0000","services":{"mds":{"daemons":{"summary":"","b":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"c":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"d":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2023-11-24T07:59:20.041 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:20.039+0000 7fd890ebf700 1 -- 172.21.15.59:0/2670709288 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7fd86c06b2c0 msgr2=0x7fd86c06d780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:20.041 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:20.039+0000 7fd890ebf700 1 --2- 172.21.15.59:0/2670709288 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7fd86c06b2c0 0x7fd86c06d780 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7fd88c0cbe80 tx=0x7fd87c008040 comp rx=0 tx=0).stop 2023-11-24T07:59:20.041 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:20.039+0000 7fd890ebf700 1 -- 172.21.15.59:0/2670709288 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd88c06fc50 msgr2=0x7fd88c0ce980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:20.041 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:20.039+0000 7fd890ebf700 1 --2- 172.21.15.59:0/2670709288 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd88c06fc50 0x7fd88c0ce980 secure :-1 s=READY pgs=874 cs=0 l=1 rev1=1 crypto rx=0x7fd87400b4a0 tx=0x7fd87400b860 comp rx=0 tx=0).stop 2023-11-24T07:59:20.041 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:20.039+0000 7fd890ebf700 1 -- 172.21.15.59:0/2670709288 shutdown_connections 2023-11-24T07:59:20.041 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:20.039+0000 7fd890ebf700 1 --2- 172.21.15.59:0/2670709288 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7fd86c06b2c0 0x7fd86c06d780 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:20.041 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:20.039+0000 7fd890ebf700 1 --2- 172.21.15.59:0/2670709288 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fd88c15e1c0 0x7fd88c15b070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:20.041 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:20.039+0000 7fd890ebf700 1 --2- 172.21.15.59:0/2670709288 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd88c06fc50 0x7fd88c0ce980 unknown :-1 s=CLOSED pgs=874 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:20.041 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:20.039+0000 7fd890ebf700 1 --2- 172.21.15.59:0/2670709288 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd88c0b0e90 0x7fd88c1f7360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:20.041 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:20.039+0000 7fd890ebf700 1 -- 172.21.15.59:0/2670709288 >> 172.21.15.59:0/2670709288 conn(0x7fd88c0b3720 msgr2=0x7fd88c15c2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:59:20.041 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:20.039+0000 7fd890ebf700 1 -- 172.21.15.59:0/2670709288 shutdown_connections 2023-11-24T07:59:20.041 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:20.039+0000 7fd890ebf700 1 -- 172.21.15.59:0/2670709288 wait complete. 2023-11-24T07:59:20.053 INFO:tasks.cephfs.filesystem.ceph_manager:no progress seen, keeping timeout for now 2023-11-24T07:59:20.053 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-11-24T07:59:20.178 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:20.176+0000 7fad4b1b3700 1 Processor -- start 2023-11-24T07:59:20.178 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:20.176+0000 7fad4b1b3700 1 -- start start 2023-11-24T07:59:20.182 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:20.177+0000 7fad4b1b3700 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fad440722c0 0x7fad440b51b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:20.182 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:20.177+0000 7fad4b1b3700 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fad44143ec0 con 0x7fad44143ae0 2023-11-24T07:59:20.182 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:20.177+0000 7fad4b1b3700 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fad440b3ae0 con 0x7fad441440f0 2023-11-24T07:59:20.182 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:20.177+0000 7fad4b1b3700 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fad440b40a0 con 0x7fad440722c0 2023-11-24T07:59:20.183 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:20.177+0000 7fad48f4f700 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7fad44143ae0 0x7fad44071bb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:58390/0 (socket says 172.21.15.59:58390) 2023-11-24T07:59:20.183 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:20.177+0000 7fad48f4f700 1 -- 172.21.15.59:0/246062230 learned_addr learned my addr 172.21.15.59:0/246062230 (peer_addr_for_me v1:172.21.15.59:0/0) 2023-11-24T07:59:20.183 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:20.177+0000 7fad43fff700 1 --2- 172.21.15.59:0/246062230 >> v2:172.21.15.76:3300/0 conn(0x7fad440722c0 0x7fad440b51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:20.183 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:20.177+0000 7fad437fe700 1 -- 172.21.15.59:0/246062230 <== mon.0 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2337954580 0 0) 0x7fad44143ec0 con 0x7fad44143ae0 2023-11-24T07:59:20.183 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:20.177+0000 7fad437fe700 1 -- 172.21.15.59:0/246062230 --> v1:172.21.15.59:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fad340035b0 con 0x7fad44143ae0 2023-11-24T07:59:20.183 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:20.178+0000 7fad437fe700 1 -- 172.21.15.59:0/246062230 <== mon.0 v1:172.21.15.59:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3265925193 0 0) 0x7fad340035b0 con 0x7fad44143ae0 2023-11-24T07:59:20.183 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:20.178+0000 7fad437fe700 1 -- 172.21.15.59:0/246062230 >> v2:172.21.15.76:3300/0 conn(0x7fad440722c0 msgr2=0x7fad440b51b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T07:59:20.183 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:20.178+0000 7fad437fe700 1 --2- 172.21.15.59:0/246062230 >> v2:172.21.15.76:3300/0 conn(0x7fad440722c0 0x7fad440b51b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:20.183 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:20.178+0000 7fad437fe700 1 -- 172.21.15.59:0/246062230 >> v1:172.21.15.76:6789/0 conn(0x7fad441440f0 legacy=0x7fad4406e510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:20.183 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:20.178+0000 7fad437fe700 1 -- 172.21.15.59:0/246062230 --> v1:172.21.15.59:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fad440b3dc0 con 0x7fad44143ae0 2023-11-24T07:59:20.183 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:20.178+0000 7fad437fe700 1 -- 172.21.15.59:0/246062230 <== mon.0 v1:172.21.15.59:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1312333566 0 0) 0x7fad38001670 con 0x7fad44143ae0 2023-11-24T07:59:20.183 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:20.178+0000 7fad437fe700 1 -- 172.21.15.59:0/246062230 >> v1:172.21.15.59:6789/0 conn(0x7fad44143ae0 legacy=0x7fad44071bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:20.184 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:20.178+0000 7fad437fe700 1 --2- 172.21.15.59:0/246062230 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fad34003a00 0x7fad34003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:20.184 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:20.178+0000 7fad437fe700 1 --2- 172.21.15.59:0/246062230 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fad34004850 0x7fad34004cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:20.184 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:20.178+0000 7fad437fe700 1 --2- 172.21.15.59:0/246062230 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fad34005210 0x7fad3400ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:20.184 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:20.178+0000 7fad437fe700 1 -- 172.21.15.59:0/246062230 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad440b40a0 con 0x7fad34005210 2023-11-24T07:59:20.184 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:20.178+0000 7fad437fe700 1 -- 172.21.15.59:0/246062230 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad3400c0b0 con 0x7fad34004850 2023-11-24T07:59:20.184 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:20.178+0000 7fad437fe700 1 -- 172.21.15.59:0/246062230 --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fad3400c270 con 0x7fad34003a00 2023-11-24T07:59:20.184 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:20.178+0000 7fad49750700 1 --2- 172.21.15.59:0/246062230 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fad34005210 0x7fad3400ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:20.184 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:20.178+0000 7fad43fff700 1 --2- 172.21.15.59:0/246062230 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fad34004850 0x7fad34004cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:20.184 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:20.178+0000 7fad48f4f700 1 --2- 172.21.15.59:0/246062230 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fad34003a00 0x7fad34003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:20.184 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:20.178+0000 7fad49750700 1 -- 172.21.15.59:0/246062230 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fad34003a00 msgr2=0x7fad34003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T07:59:20.184 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:20.178+0000 7fad49750700 1 --2- 172.21.15.59:0/246062230 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fad34003a00 0x7fad34003e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:20.184 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:20.178+0000 7fad49750700 1 -- 172.21.15.59:0/246062230 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fad34004850 msgr2=0x7fad34004cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T07:59:20.185 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:20.178+0000 7fad49750700 1 --2- 172.21.15.59:0/246062230 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fad34004850 0x7fad34004cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:20.185 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:20.178+0000 7fad49750700 1 -- 172.21.15.59:0/246062230 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fad340035b0 con 0x7fad34005210 2023-11-24T07:59:20.185 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:20.178+0000 7fad43fff700 1 --2- 172.21.15.59:0/246062230 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fad34004850 0x7fad34004cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-24T07:59:20.185 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:20.178+0000 7fad49750700 1 -- 172.21.15.59:0/246062230 reap_dead start 2023-11-24T07:59:20.185 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:20.178+0000 7fad49750700 1 --2- 172.21.15.59:0/246062230 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fad34005210 0x7fad3400ba10 secure :-1 s=READY pgs=876 cs=0 l=1 rev1=1 crypto rx=0x7fad3c00c190 tx=0x7fad34005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cac46d2d20ac0a5 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:59:20.185 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:20.179+0000 7fad437fe700 1 -- 172.21.15.59:0/246062230 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad3c001db0 con 0x7fad34005210 2023-11-24T07:59:20.185 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:20.179+0000 7fad437fe700 1 -- 172.21.15.59:0/246062230 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fad3c00ccd0 con 0x7fad34005210 2023-11-24T07:59:20.185 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:20.179+0000 7fad437fe700 1 -- 172.21.15.59:0/246062230 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad3c003c40 con 0x7fad34005210 2023-11-24T07:59:20.185 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:20.179+0000 7fad4b1b3700 1 -- 172.21.15.59:0/246062230 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fad34005210 msgr2=0x7fad3400ba10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:20.185 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:20.179+0000 7fad4b1b3700 1 --2- 172.21.15.59:0/246062230 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fad34005210 0x7fad3400ba10 secure :-1 s=READY pgs=876 cs=0 l=1 rev1=1 crypto rx=0x7fad3c00c190 tx=0x7fad34005880 comp rx=0 tx=0).stop 2023-11-24T07:59:20.185 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:20.179+0000 7fad4b1b3700 1 -- 172.21.15.59:0/246062230 shutdown_connections 2023-11-24T07:59:20.185 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:20.179+0000 7fad4b1b3700 1 --2- 172.21.15.59:0/246062230 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fad34005210 0x7fad3400ba10 unknown :-1 s=CLOSED pgs=876 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:20.185 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:20.179+0000 7fad4b1b3700 1 -- 172.21.15.59:0/246062230 >> 172.21.15.59:0/246062230 conn(0x7fad440b2740 msgr2=0x7fad440b2b20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:59:20.186 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:20.179+0000 7fad4b1b3700 1 -- 172.21.15.59:0/246062230 shutdown_connections 2023-11-24T07:59:20.186 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:20.179+0000 7fad4b1b3700 1 -- 172.21.15.59:0/246062230 wait complete. 2023-11-24T07:59:20.186 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:20.181+0000 7fad4b1b3700 1 Processor -- start 2023-11-24T07:59:20.186 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:20.181+0000 7fad4b1b3700 1 -- start start 2023-11-24T07:59:20.188 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:20.182+0000 7fad4b1b3700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fad34005210 0x7fad44157b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:20.188 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:20.182+0000 7fad4b1b3700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fad441f1500 0x7fad440cf600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:20.188 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:20.182+0000 7fad4b1b3700 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fad441f1980 0x7fad440cfb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:20.188 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:20.182+0000 7fad4b1b3700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad441f6280 con 0x7fad441f1500 2023-11-24T07:59:20.189 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:20.182+0000 7fad4b1b3700 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad441f5700 con 0x7fad34005210 2023-11-24T07:59:20.189 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:20.182+0000 7fad4b1b3700 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fad441f59e0 con 0x7fad441f1980 2023-11-24T07:59:20.189 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:20.182+0000 7fad43fff700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fad441f1500 0x7fad440cf600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:20.189 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:20.182+0000 7fad43fff700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fad441f1500 0x7fad440cf600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:54930/0 (socket says 172.21.15.59:54930) 2023-11-24T07:59:20.189 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:20.182+0000 7fad43fff700 1 -- 172.21.15.59:0/3398615579 learned_addr learned my addr 172.21.15.59:0/3398615579 (peer_addr_for_me v2:172.21.15.59:0/0) 2023-11-24T07:59:20.189 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:20.182+0000 7fad49750700 1 --2- 172.21.15.59:0/3398615579 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fad441f1980 0x7fad440cfb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:20.189 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:20.182+0000 7fad48f4f700 1 --2- 172.21.15.59:0/3398615579 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fad34005210 0x7fad44157b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:20.189 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:20.182+0000 7fad43fff700 1 -- 172.21.15.59:0/3398615579 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fad441f1980 msgr2=0x7fad440cfb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:20.189 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:20.182+0000 7fad43fff700 1 --2- 172.21.15.59:0/3398615579 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fad441f1980 0x7fad440cfb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:20.189 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:20.182+0000 7fad43fff700 1 -- 172.21.15.59:0/3398615579 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fad34005210 msgr2=0x7fad44157b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:20.189 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:20.182+0000 7fad43fff700 1 --2- 172.21.15.59:0/3398615579 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fad34005210 0x7fad44157b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:20.189 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:20.182+0000 7fad43fff700 1 -- 172.21.15.59:0/3398615579 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fad441f5cc0 con 0x7fad441f1500 2023-11-24T07:59:20.190 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:20.182+0000 7fad49750700 1 --2- 172.21.15.59:0/3398615579 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fad441f1980 0x7fad440cfb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-24T07:59:20.190 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:20.182+0000 7fad43fff700 1 --2- 172.21.15.59:0/3398615579 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fad441f1500 0x7fad440cf600 secure :-1 s=READY pgs=877 cs=0 l=1 rev1=1 crypto rx=0x7fad30011cd0 tx=0x7fad3000f520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:59:20.190 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:20.183+0000 7fad417fa700 1 -- 172.21.15.59:0/3398615579 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad3000fb30 con 0x7fad441f1500 2023-11-24T07:59:20.190 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:20.183+0000 7fad417fa700 1 -- 172.21.15.59:0/3398615579 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fad3000fc90 con 0x7fad441f1500 2023-11-24T07:59:20.190 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:20.183+0000 7fad4b1b3700 1 -- 172.21.15.59:0/3398615579 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fad441f5fa0 con 0x7fad441f1500 2023-11-24T07:59:20.190 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:20.183+0000 7fad417fa700 1 -- 172.21.15.59:0/3398615579 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad30004020 con 0x7fad441f1500 2023-11-24T07:59:20.190 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:20.183+0000 7fad4b1b3700 1 -- 172.21.15.59:0/3398615579 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fad440cc9a0 con 0x7fad441f1500 2023-11-24T07:59:20.190 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:20.184+0000 7fad4b1b3700 1 -- 172.21.15.59:0/3398615579 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fad4413ade0 con 0x7fad441f1500 2023-11-24T07:59:20.190 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:20.185+0000 7fad417fa700 1 -- 172.21.15.59:0/3398615579 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 10) v1 ==== 89588+0+0 (secure 0 0 0) 0x7fad30004180 con 0x7fad441f1500 2023-11-24T07:59:20.190 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:20.185+0000 7fad417fa700 1 --2- 172.21.15.59:0/3398615579 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7fad2806b2c0 0x7fad2806d780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:20.190 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:20.185+0000 7fad417fa700 1 -- 172.21.15.59:0/3398615579 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fad3008a680 con 0x7fad441f1500 2023-11-24T07:59:20.190 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:20.186+0000 7fad48f4f700 1 --2- 172.21.15.59:0/3398615579 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7fad2806b2c0 0x7fad2806d780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:20.191 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:20.186+0000 7fad48f4f700 1 --2- 172.21.15.59:0/3398615579 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7fad2806b2c0 0x7fad2806d780 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7fad38007cd0 tx=0x7fad38009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:59:20.191 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:20.189+0000 7fad417fa700 1 -- 172.21.15.59:0/3398615579 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fad30055250 con 0x7fad441f1500 2023-11-24T07:59:20.386 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:20.384+0000 7fad4b1b3700 1 -- 172.21.15.59:0/3398615579 --> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fad44144860 con 0x7fad2806b2c0 2023-11-24T07:59:20.395 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:20.393+0000 7fad417fa700 1 -- 172.21.15.59:0/3398615579 <== mgr.4100 v2:172.21.15.76:6832/4264046536 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+289397 (secure 0 0 0) 0x7fad38000ed0 con 0x7fad2806b2c0 2023-11-24T07:59:20.398 INFO:teuthology.orchestra.run.smithi059.stdout: 2023-11-24T07:59:20.400 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:20.396+0000 7fad4b1b3700 1 -- 172.21.15.59:0/3398615579 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7fad2806b2c0 msgr2=0x7fad2806d780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:20.401 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:20.396+0000 7fad4b1b3700 1 --2- 172.21.15.59:0/3398615579 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7fad2806b2c0 0x7fad2806d780 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7fad38007cd0 tx=0x7fad38009040 comp rx=0 tx=0).stop 2023-11-24T07:59:20.401 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:20.396+0000 7fad4b1b3700 1 -- 172.21.15.59:0/3398615579 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fad441f1500 msgr2=0x7fad440cf600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:20.401 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:20.396+0000 7fad4b1b3700 1 --2- 172.21.15.59:0/3398615579 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fad441f1500 0x7fad440cf600 secure :-1 s=READY pgs=877 cs=0 l=1 rev1=1 crypto rx=0x7fad30011cd0 tx=0x7fad3000f520 comp rx=0 tx=0).stop 2023-11-24T07:59:20.401 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:20.396+0000 7fad4b1b3700 1 -- 172.21.15.59:0/3398615579 shutdown_connections 2023-11-24T07:59:20.401 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:20.396+0000 7fad4b1b3700 1 --2- 172.21.15.59:0/3398615579 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7fad2806b2c0 0x7fad2806d780 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:20.401 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:20.396+0000 7fad4b1b3700 1 --2- 172.21.15.59:0/3398615579 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fad441f1980 0x7fad440cfb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:20.401 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:20.396+0000 7fad4b1b3700 1 --2- 172.21.15.59:0/3398615579 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fad34005210 0x7fad44157b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:20.401 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:20.396+0000 7fad4b1b3700 1 --2- 172.21.15.59:0/3398615579 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fad441f1500 0x7fad440cf600 unknown :-1 s=CLOSED pgs=877 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:20.401 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:20.396+0000 7fad4b1b3700 1 -- 172.21.15.59:0/3398615579 >> 172.21.15.59:0/3398615579 conn(0x7fad440b2740 msgr2=0x7fad44141ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:59:20.401 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:20.396+0000 7fad4b1b3700 1 -- 172.21.15.59:0/3398615579 shutdown_connections 2023-11-24T07:59:20.401 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:20.396+0000 7fad4b1b3700 1 -- 172.21.15.59:0/3398615579 wait complete. 2023-11-24T07:59:20.402 INFO:teuthology.orchestra.run.smithi059.stderr:dumped all 2023-11-24T07:59:20.410 INFO:teuthology.orchestra.run.smithi059.stdout:{"pg_ready":true,"pg_map":{"version":158,"stamp":"2023-11-24T07:59:20.339833+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":182,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":224448,"kb_used_data":8264,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754750272,"statfs":{"total":773094113280,"available":772864278528,"internally_reserved":0,"allocated":8462336,"data_stored":4258612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12745,"internal_metadata":221236791},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":35,"apply_latency_ms":35,"commit_latency_ns":35000000,"apply_latency_ns":35000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.710660"},"pg_stats":[{"pgid":"11.3f","version":"0'0","reported_seq":11,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-11-24T07:59:16.292941+0000","last_change":"2023-11-24T07:59:14.661661+0000","last_active":"2023-11-24T07:59:16.292941+0000","last_peered":"2023-11-24T07:59:16.292941+0000","last_clean":"2023-11-24T07:59:16.292941+0000","last_became_active":"2023-11-24T07:59:14.661350+0000","last_became_peered":"2023-11-24T07:59:14.661350+0000","last_unstale":"2023-11-24T07:59:16.292941+0000","last_undegraded":"2023-11-24T07:59:16.292941+0000","last_fullsized":"2023-11-24T07:59:16.292941+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:59:13.620687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:59:13.620687+0000","last_clean_scrub_stamp":"2023-11-24T07:59:13.620687+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T09:34:52.878616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.3e","version":"0'0","reported_seq":11,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-11-24T07:59:16.293734+0000","last_change":"2023-11-24T07:59:14.662364+0000","last_active":"2023-11-24T07:59:16.293734+0000","last_peered":"2023-11-24T07:59:16.293734+0000","last_clean":"2023-11-24T07:59:16.293734+0000","last_became_active":"2023-11-24T07:59:14.661727+0000","last_became_peered":"2023-11-24T07:59:14.661727+0000","last_unstale":"2023-11-24T07:59:16.293734+0000","last_undegraded":"2023-11-24T07:59:16.293734+0000","last_fullsized":"2023-11-24T07:59:16.293734+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:59:13.620687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:59:13.620687+0000","last_clean_scrub_stamp":"2023-11-24T07:59:13.620687+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T13:45:41.343630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.3d","version":"0'0","reported_seq":11,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-11-24T07:59:16.292014+0000","last_change":"2023-11-24T07:59:14.649579+0000","last_active":"2023-11-24T07:59:16.292014+0000","last_peered":"2023-11-24T07:59:16.292014+0000","last_clean":"2023-11-24T07:59:16.292014+0000","last_became_active":"2023-11-24T07:59:14.644685+0000","last_became_peered":"2023-11-24T07:59:14.644685+0000","last_unstale":"2023-11-24T07:59:16.292014+0000","last_undegraded":"2023-11-24T07:59:16.292014+0000","last_fullsized":"2023-11-24T07:59:16.292014+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:59:13.620687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:59:13.620687+0000","last_clean_scrub_stamp":"2023-11-24T07:59:13.620687+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:02:40.788118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.3c","version":"0'0","reported_seq":11,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-11-24T07:59:16.292011+0000","last_change":"2023-11-24T07:59:14.649105+0000","last_active":"2023-11-24T07:59:16.292011+0000","last_peered":"2023-11-24T07:59:16.292011+0000","last_clean":"2023-11-24T07:59:16.292011+0000","last_became_active":"2023-11-24T07:59:14.648733+0000","last_became_peered":"2023-11-24T07:59:14.648733+0000","last_unstale":"2023-11-24T07:59:16.292011+0000","last_undegraded":"2023-11-24T07:59:16.292011+0000","last_fullsized":"2023-11-24T07:59:16.292011+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:59:13.620687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:59:13.620687+0000","last_clean_scrub_stamp":"2023-11-24T07:59:13.620687+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T08:20:48.582684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.3b","version":"0'0","reported_seq":11,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-11-24T07:59:15.650794+0000","last_change":"2023-11-24T07:59:14.668923+0000","last_active":"2023-11-24T07:59:15.650794+0000","last_peered":"2023-11-24T07:59:15.650794+0000","last_clean":"2023-11-24T07:59:15.650794+0000","last_became_active":"2023-11-24T07:59:14.668627+0000","last_became_peered":"2023-11-24T07:59:14.668627+0000","last_unstale":"2023-11-24T07:59:15.650794+0000","last_undegraded":"2023-11-24T07:59:15.650794+0000","last_fullsized":"2023-11-24T07:59:15.650794+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:59:13.620687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:59:13.620687+0000","last_clean_scrub_stamp":"2023-11-24T07:59:13.620687+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T16:38:05.743270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.3a","version":"0'0","reported_seq":11,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-11-24T07:59:16.104059+0000","last_change":"2023-11-24T07:59:14.658517+0000","last_active":"2023-11-24T07:59:16.104059+0000","last_peered":"2023-11-24T07:59:16.104059+0000","last_clean":"2023-11-24T07:59:16.104059+0000","last_became_active":"2023-11-24T07:59:14.658227+0000","last_became_peered":"2023-11-24T07:59:14.658227+0000","last_unstale":"2023-11-24T07:59:16.104059+0000","last_undegraded":"2023-11-24T07:59:16.104059+0000","last_fullsized":"2023-11-24T07:59:16.104059+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:59:13.620687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:59:13.620687+0000","last_clean_scrub_stamp":"2023-11-24T07:59:13.620687+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T11:23:41.485071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.39","version":"0'0","reported_seq":11,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-11-24T07:59:16.292282+0000","last_change":"2023-11-24T07:59:14.650302+0000","last_active":"2023-11-24T07:59:16.292282+0000","last_peered":"2023-11-24T07:59:16.292282+0000","last_clean":"2023-11-24T07:59:16.292282+0000","last_became_active":"2023-11-24T07:59:14.648820+0000","last_became_peered":"2023-11-24T07:59:14.648820+0000","last_unstale":"2023-11-24T07:59:16.292282+0000","last_undegraded":"2023-11-24T07:59:16.292282+0000","last_fullsized":"2023-11-24T07:59:16.292282+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:59:13.620687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:59:13.620687+0000","last_clean_scrub_stamp":"2023-11-24T07:59:13.620687+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:54:57.076700+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.38","version":"0'0","reported_seq":11,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-11-24T07:59:16.291981+0000","last_change":"2023-11-24T07:59:14.660939+0000","last_active":"2023-11-24T07:59:16.291981+0000","last_peered":"2023-11-24T07:59:16.291981+0000","last_clean":"2023-11-24T07:59:16.291981+0000","last_became_active":"2023-11-24T07:59:14.660638+0000","last_became_peered":"2023-11-24T07:59:14.660638+0000","last_unstale":"2023-11-24T07:59:16.291981+0000","last_undegraded":"2023-11-24T07:59:16.291981+0000","last_fullsized":"2023-11-24T07:59:16.291981+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:59:13.620687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:59:13.620687+0000","last_clean_scrub_stamp":"2023-11-24T07:59:13.620687+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T18:41:19.164124+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.37","version":"0'0","reported_seq":11,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-11-24T07:59:16.292018+0000","last_change":"2023-11-24T07:59:14.651362+0000","last_active":"2023-11-24T07:59:16.292018+0000","last_peered":"2023-11-24T07:59:16.292018+0000","last_clean":"2023-11-24T07:59:16.292018+0000","last_became_active":"2023-11-24T07:59:14.650874+0000","last_became_peered":"2023-11-24T07:59:14.650874+0000","last_unstale":"2023-11-24T07:59:16.292018+0000","last_undegraded":"2023-11-24T07:59:16.292018+0000","last_fullsized":"2023-11-24T07:59:16.292018+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:59:13.620687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:59:13.620687+0000","last_clean_scrub_stamp":"2023-11-24T07:59:13.620687+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T19:05:59.935796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.36","version":"0'0","reported_seq":11,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-11-24T07:59:16.293188+0000","last_change":"2023-11-24T07:59:14.649665+0000","last_active":"2023-11-24T07:59:16.293188+0000","last_peered":"2023-11-24T07:59:16.293188+0000","last_clean":"2023-11-24T07:59:16.293188+0000","last_became_active":"2023-11-24T07:59:14.649305+0000","last_became_peered":"2023-11-24T07:59:14.649305+0000","last_unstale":"2023-11-24T07:59:16.293188+0000","last_undegraded":"2023-11-24T07:59:16.293188+0000","last_fullsized":"2023-11-24T07:59:16.293188+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:59:13.620687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:59:13.620687+0000","last_clean_scrub_stamp":"2023-11-24T07:59:13.620687+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T17:15:10.353695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.35","version":"0'0","reported_seq":11,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-11-24T07:59:16.292721+0000","last_change":"2023-11-24T07:59:14.663082+0000","last_active":"2023-11-24T07:59:16.292721+0000","last_peered":"2023-11-24T07:59:16.292721+0000","last_clean":"2023-11-24T07:59:16.292721+0000","last_became_active":"2023-11-24T07:59:14.662601+0000","last_became_peered":"2023-11-24T07:59:14.662601+0000","last_unstale":"2023-11-24T07:59:16.292721+0000","last_undegraded":"2023-11-24T07:59:16.292721+0000","last_fullsized":"2023-11-24T07:59:16.292721+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:59:13.620687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:59:13.620687+0000","last_clean_scrub_stamp":"2023-11-24T07:59:13.620687+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T15:51:56.807935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.34","version":"0'0","reported_seq":11,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-11-24T07:59:15.649831+0000","last_change":"2023-11-24T07:59:14.646577+0000","last_active":"2023-11-24T07:59:15.649831+0000","last_peered":"2023-11-24T07:59:15.649831+0000","last_clean":"2023-11-24T07:59:15.649831+0000","last_became_active":"2023-11-24T07:59:14.646318+0000","last_became_peered":"2023-11-24T07:59:14.646318+0000","last_unstale":"2023-11-24T07:59:15.649831+0000","last_undegraded":"2023-11-24T07:59:15.649831+0000","last_fullsized":"2023-11-24T07:59:15.649831+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:59:13.620687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:59:13.620687+0000","last_clean_scrub_stamp":"2023-11-24T07:59:13.620687+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T15:47:40.090700+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.33","version":"0'0","reported_seq":11,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-11-24T07:59:16.292052+0000","last_change":"2023-11-24T07:59:14.652220+0000","last_active":"2023-11-24T07:59:16.292052+0000","last_peered":"2023-11-24T07:59:16.292052+0000","last_clean":"2023-11-24T07:59:16.292052+0000","last_became_active":"2023-11-24T07:59:14.644814+0000","last_became_peered":"2023-11-24T07:59:14.644814+0000","last_unstale":"2023-11-24T07:59:16.292052+0000","last_undegraded":"2023-11-24T07:59:16.292052+0000","last_fullsized":"2023-11-24T07:59:16.292052+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:59:13.620687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:59:13.620687+0000","last_clean_scrub_stamp":"2023-11-24T07:59:13.620687+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T19:09:47.226456+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.32","version":"0'0","reported_seq":11,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-11-24T07:59:15.649710+0000","last_change":"2023-11-24T07:59:14.654802+0000","last_active":"2023-11-24T07:59:15.649710+0000","last_peered":"2023-11-24T07:59:15.649710+0000","last_clean":"2023-11-24T07:59:15.649710+0000","last_became_active":"2023-11-24T07:59:14.654543+0000","last_became_peered":"2023-11-24T07:59:14.654543+0000","last_unstale":"2023-11-24T07:59:15.649710+0000","last_undegraded":"2023-11-24T07:59:15.649710+0000","last_fullsized":"2023-11-24T07:59:15.649710+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:59:13.620687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:59:13.620687+0000","last_clean_scrub_stamp":"2023-11-24T07:59:13.620687+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T14:38:55.028542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.31","version":"0'0","reported_seq":11,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-11-24T07:59:15.649474+0000","last_change":"2023-11-24T07:59:14.648768+0000","last_active":"2023-11-24T07:59:15.649474+0000","last_peered":"2023-11-24T07:59:15.649474+0000","last_clean":"2023-11-24T07:59:15.649474+0000","last_became_active":"2023-11-24T07:59:14.648489+0000","last_became_peered":"2023-11-24T07:59:14.648489+0000","last_unstale":"2023-11-24T07:59:15.649474+0000","last_undegraded":"2023-11-24T07:59:15.649474+0000","last_fullsized":"2023-11-24T07:59:15.649474+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:59:13.620687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:59:13.620687+0000","last_clean_scrub_stamp":"2023-11-24T07:59:13.620687+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T15:27:01.624152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.30","version":"0'0","reported_seq":11,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-11-24T07:59:16.292844+0000","last_change":"2023-11-24T07:59:14.669482+0000","last_active":"2023-11-24T07:59:16.292844+0000","last_peered":"2023-11-24T07:59:16.292844+0000","last_clean":"2023-11-24T07:59:16.292844+0000","last_became_active":"2023-11-24T07:59:14.669140+0000","last_became_peered":"2023-11-24T07:59:14.669140+0000","last_unstale":"2023-11-24T07:59:16.292844+0000","last_undegraded":"2023-11-24T07:59:16.292844+0000","last_fullsized":"2023-11-24T07:59:16.292844+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:59:13.620687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:59:13.620687+0000","last_clean_scrub_stamp":"2023-11-24T07:59:13.620687+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T13:16:48.753168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.2f","version":"0'0","reported_seq":11,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-11-24T07:59:15.650711+0000","last_change":"2023-11-24T07:59:14.643449+0000","last_active":"2023-11-24T07:59:15.650711+0000","last_peered":"2023-11-24T07:59:15.650711+0000","last_clean":"2023-11-24T07:59:15.650711+0000","last_became_active":"2023-11-24T07:59:14.643178+0000","last_became_peered":"2023-11-24T07:59:14.643178+0000","last_unstale":"2023-11-24T07:59:15.650711+0000","last_undegraded":"2023-11-24T07:59:15.650711+0000","last_fullsized":"2023-11-24T07:59:15.650711+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:59:13.620687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:59:13.620687+0000","last_clean_scrub_stamp":"2023-11-24T07:59:13.620687+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T15:36:44.561284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.2e","version":"0'0","reported_seq":11,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-11-24T07:59:15.650810+0000","last_change":"2023-11-24T07:59:14.647261+0000","last_active":"2023-11-24T07:59:15.650810+0000","last_peered":"2023-11-24T07:59:15.650810+0000","last_clean":"2023-11-24T07:59:15.650810+0000","last_became_active":"2023-11-24T07:59:14.646984+0000","last_became_peered":"2023-11-24T07:59:14.646984+0000","last_unstale":"2023-11-24T07:59:15.650810+0000","last_undegraded":"2023-11-24T07:59:15.650810+0000","last_fullsized":"2023-11-24T07:59:15.650810+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:59:13.620687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:59:13.620687+0000","last_clean_scrub_stamp":"2023-11-24T07:59:13.620687+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T11:25:11.031296+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.2d","version":"0'0","reported_seq":11,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-11-24T07:59:16.292770+0000","last_change":"2023-11-24T07:59:14.662845+0000","last_active":"2023-11-24T07:59:16.292770+0000","last_peered":"2023-11-24T07:59:16.292770+0000","last_clean":"2023-11-24T07:59:16.292770+0000","last_became_active":"2023-11-24T07:59:14.662448+0000","last_became_peered":"2023-11-24T07:59:14.662448+0000","last_unstale":"2023-11-24T07:59:16.292770+0000","last_undegraded":"2023-11-24T07:59:16.292770+0000","last_fullsized":"2023-11-24T07:59:16.292770+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:59:13.620687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:59:13.620687+0000","last_clean_scrub_stamp":"2023-11-24T07:59:13.620687+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T17:45:18.664406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.2c","version":"0'0","reported_seq":11,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-11-24T07:59:15.650708+0000","last_change":"2023-11-24T07:59:14.642598+0000","last_active":"2023-11-24T07:59:15.650708+0000","last_peered":"2023-11-24T07:59:15.650708+0000","last_clean":"2023-11-24T07:59:15.650708+0000","last_became_active":"2023-11-24T07:59:14.642357+0000","last_became_peered":"2023-11-24T07:59:14.642357+0000","last_unstale":"2023-11-24T07:59:15.650708+0000","last_undegraded":"2023-11-24T07:59:15.650708+0000","last_fullsized":"2023-11-24T07:59:15.650708+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:59:13.620687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:59:13.620687+0000","last_clean_scrub_stamp":"2023-11-24T07:59:13.620687+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T09:58:24.335415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.2b","version":"0'0","reported_seq":11,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-11-24T07:59:16.293409+0000","last_change":"2023-11-24T07:59:14.661043+0000","last_active":"2023-11-24T07:59:16.293409+0000","last_peered":"2023-11-24T07:59:16.293409+0000","last_clean":"2023-11-24T07:59:16.293409+0000","last_became_active":"2023-11-24T07:59:14.660775+0000","last_became_peered":"2023-11-24T07:59:14.660775+0000","last_unstale":"2023-11-24T07:59:16.293409+0000","last_undegraded":"2023-11-24T07:59:16.293409+0000","last_fullsized":"2023-11-24T07:59:16.293409+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:59:13.620687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:59:13.620687+0000","last_clean_scrub_stamp":"2023-11-24T07:59:13.620687+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T15:26:05.128703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.2a","version":"0'0","reported_seq":11,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-11-24T07:59:16.103881+0000","last_change":"2023-11-24T07:59:14.647428+0000","last_active":"2023-11-24T07:59:16.103881+0000","last_peered":"2023-11-24T07:59:16.103881+0000","last_clean":"2023-11-24T07:59:16.103881+0000","last_became_active":"2023-11-24T07:59:14.647154+0000","last_became_peered":"2023-11-24T07:59:14.647154+0000","last_unstale":"2023-11-24T07:59:16.103881+0000","last_undegraded":"2023-11-24T07:59:16.103881+0000","last_fullsized":"2023-11-24T07:59:16.103881+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:59:13.620687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:59:13.620687+0000","last_clean_scrub_stamp":"2023-11-24T07:59:13.620687+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T18:49:53.391209+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.29","version":"0'0","reported_seq":11,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-11-24T07:59:16.292441+0000","last_change":"2023-11-24T07:59:14.665583+0000","last_active":"2023-11-24T07:59:16.292441+0000","last_peered":"2023-11-24T07:59:16.292441+0000","last_clean":"2023-11-24T07:59:16.292441+0000","last_became_active":"2023-11-24T07:59:14.665127+0000","last_became_peered":"2023-11-24T07:59:14.665127+0000","last_unstale":"2023-11-24T07:59:16.292441+0000","last_undegraded":"2023-11-24T07:59:16.292441+0000","last_fullsized":"2023-11-24T07:59:16.292441+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:59:13.620687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:59:13.620687+0000","last_clean_scrub_stamp":"2023-11-24T07:59:13.620687+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T12:41:31.508405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.28","version":"0'0","reported_seq":11,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-11-24T07:59:16.292171+0000","last_change":"2023-11-24T07:59:14.660934+0000","last_active":"2023-11-24T07:59:16.292171+0000","last_peered":"2023-11-24T07:59:16.292171+0000","last_clean":"2023-11-24T07:59:16.292171+0000","last_became_active":"2023-11-24T07:59:14.660353+0000","last_became_peered":"2023-11-24T07:59:14.660353+0000","last_unstale":"2023-11-24T07:59:16.292171+0000","last_undegraded":"2023-11-24T07:59:16.292171+0000","last_fullsized":"2023-11-24T07:59:16.292171+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:59:13.620687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:59:13.620687+0000","last_clean_scrub_stamp":"2023-11-24T07:59:13.620687+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T15:04:45.070729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.27","version":"0'0","reported_seq":11,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-11-24T07:59:16.292550+0000","last_change":"2023-11-24T07:59:14.668930+0000","last_active":"2023-11-24T07:59:16.292550+0000","last_peered":"2023-11-24T07:59:16.292550+0000","last_clean":"2023-11-24T07:59:16.292550+0000","last_became_active":"2023-11-24T07:59:14.668480+0000","last_became_peered":"2023-11-24T07:59:14.668480+0000","last_unstale":"2023-11-24T07:59:16.292550+0000","last_undegraded":"2023-11-24T07:59:16.292550+0000","last_fullsized":"2023-11-24T07:59:16.292550+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:59:13.620687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:59:13.620687+0000","last_clean_scrub_stamp":"2023-11-24T07:59:13.620687+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T13:43:24.163968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.26","version":"0'0","reported_seq":11,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-11-24T07:59:16.292628+0000","last_change":"2023-11-24T07:59:14.661613+0000","last_active":"2023-11-24T07:59:16.292628+0000","last_peered":"2023-11-24T07:59:16.292628+0000","last_clean":"2023-11-24T07:59:16.292628+0000","last_became_active":"2023-11-24T07:59:14.661321+0000","last_became_peered":"2023-11-24T07:59:14.661321+0000","last_unstale":"2023-11-24T07:59:16.292628+0000","last_undegraded":"2023-11-24T07:59:16.292628+0000","last_fullsized":"2023-11-24T07:59:16.292628+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:59:13.620687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:59:13.620687+0000","last_clean_scrub_stamp":"2023-11-24T07:59:13.620687+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T19:21:43.761495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.25","version":"0'0","reported_seq":11,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-11-24T07:59:15.650595+0000","last_change":"2023-11-24T07:59:14.645456+0000","last_active":"2023-11-24T07:59:15.650595+0000","last_peered":"2023-11-24T07:59:15.650595+0000","last_clean":"2023-11-24T07:59:15.650595+0000","last_became_active":"2023-11-24T07:59:14.645148+0000","last_became_peered":"2023-11-24T07:59:14.645148+0000","last_unstale":"2023-11-24T07:59:15.650595+0000","last_undegraded":"2023-11-24T07:59:15.650595+0000","last_fullsized":"2023-11-24T07:59:15.650595+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:59:13.620687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:59:13.620687+0000","last_clean_scrub_stamp":"2023-11-24T07:59:13.620687+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T15:29:23.598719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.24","version":"0'0","reported_seq":11,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-11-24T07:59:16.293908+0000","last_change":"2023-11-24T07:59:14.660458+0000","last_active":"2023-11-24T07:59:16.293908+0000","last_peered":"2023-11-24T07:59:16.293908+0000","last_clean":"2023-11-24T07:59:16.293908+0000","last_became_active":"2023-11-24T07:59:14.659937+0000","last_became_peered":"2023-11-24T07:59:14.659937+0000","last_unstale":"2023-11-24T07:59:16.293908+0000","last_undegraded":"2023-11-24T07:59:16.293908+0000","last_fullsized":"2023-11-24T07:59:16.293908+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:59:13.620687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:59:13.620687+0000","last_clean_scrub_stamp":"2023-11-24T07:59:13.620687+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T13:00:14.118611+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.23","version":"0'0","reported_seq":11,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-11-24T07:59:16.292018+0000","last_change":"2023-11-24T07:59:14.658404+0000","last_active":"2023-11-24T07:59:16.292018+0000","last_peered":"2023-11-24T07:59:16.292018+0000","last_clean":"2023-11-24T07:59:16.292018+0000","last_became_active":"2023-11-24T07:59:14.658126+0000","last_became_peered":"2023-11-24T07:59:14.658126+0000","last_unstale":"2023-11-24T07:59:16.292018+0000","last_undegraded":"2023-11-24T07:59:16.292018+0000","last_fullsized":"2023-11-24T07:59:16.292018+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:59:13.620687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:59:13.620687+0000","last_clean_scrub_stamp":"2023-11-24T07:59:13.620687+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T17:11:47.648598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.22","version":"0'0","reported_seq":11,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-11-24T07:59:16.292749+0000","last_change":"2023-11-24T07:59:14.657880+0000","last_active":"2023-11-24T07:59:16.292749+0000","last_peered":"2023-11-24T07:59:16.292749+0000","last_clean":"2023-11-24T07:59:16.292749+0000","last_became_active":"2023-11-24T07:59:14.657479+0000","last_became_peered":"2023-11-24T07:59:14.657479+0000","last_unstale":"2023-11-24T07:59:16.292749+0000","last_undegraded":"2023-11-24T07:59:16.292749+0000","last_fullsized":"2023-11-24T07:59:16.292749+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:59:13.620687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:59:13.620687+0000","last_clean_scrub_stamp":"2023-11-24T07:59:13.620687+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T13:02:40.785870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.21","version":"0'0","reported_seq":11,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-11-24T07:59:16.294293+0000","last_change":"2023-11-24T07:59:14.667786+0000","last_active":"2023-11-24T07:59:16.294293+0000","last_peered":"2023-11-24T07:59:16.294293+0000","last_clean":"2023-11-24T07:59:16.294293+0000","last_became_active":"2023-11-24T07:59:14.667479+0000","last_became_peered":"2023-11-24T07:59:14.667479+0000","last_unstale":"2023-11-24T07:59:16.294293+0000","last_undegraded":"2023-11-24T07:59:16.294293+0000","last_fullsized":"2023-11-24T07:59:16.294293+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:59:13.620687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:59:13.620687+0000","last_clean_scrub_stamp":"2023-11-24T07:59:13.620687+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T16:41:24.799664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.20","version":"0'0","reported_seq":11,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-11-24T07:59:16.293066+0000","last_change":"2023-11-24T07:59:14.667849+0000","last_active":"2023-11-24T07:59:16.293066+0000","last_peered":"2023-11-24T07:59:16.293066+0000","last_clean":"2023-11-24T07:59:16.293066+0000","last_became_active":"2023-11-24T07:59:14.667479+0000","last_became_peered":"2023-11-24T07:59:14.667479+0000","last_unstale":"2023-11-24T07:59:16.293066+0000","last_undegraded":"2023-11-24T07:59:16.293066+0000","last_fullsized":"2023-11-24T07:59:16.293066+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:59:13.620687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:59:13.620687+0000","last_clean_scrub_stamp":"2023-11-24T07:59:13.620687+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T14:28:14.357404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1e","version":"0'0","reported_seq":11,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-11-24T07:59:16.292809+0000","last_change":"2023-11-24T07:59:14.669493+0000","last_active":"2023-11-24T07:59:16.292809+0000","last_peered":"2023-11-24T07:59:16.292809+0000","last_clean":"2023-11-24T07:59:16.292809+0000","last_became_active":"2023-11-24T07:59:14.669126+0000","last_became_peered":"2023-11-24T07:59:14.669126+0000","last_unstale":"2023-11-24T07:59:16.292809+0000","last_undegraded":"2023-11-24T07:59:16.292809+0000","last_fullsized":"2023-11-24T07:59:16.292809+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:59:13.620687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:59:13.620687+0000","last_clean_scrub_stamp":"2023-11-24T07:59:13.620687+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T13:16:42.796914+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.1f","version":"73'4","reported_seq":16,"reported_epoch":73,"state":"active+clean","last_fresh":"2023-11-24T07:59:14.666712+0000","last_change":"2023-11-24T07:59:13.653337+0000","last_active":"2023-11-24T07:59:14.666712+0000","last_peered":"2023-11-24T07:59:14.666712+0000","last_clean":"2023-11-24T07:59:14.666712+0000","last_became_active":"2023-11-24T07:59:13.653064+0000","last_became_peered":"2023-11-24T07:59:13.653064+0000","last_unstale":"2023-11-24T07:59:14.666712+0000","last_undegraded":"2023-11-24T07:59:14.666712+0000","last_fullsized":"2023-11-24T07:59:14.666712+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:59:12.615544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:59:12.615544+0000","last_clean_scrub_stamp":"2023-11-24T07:59:12.615544+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T14:28:13.352261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.1f","version":"0'0","reported_seq":11,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-11-24T07:59:16.292104+0000","last_change":"2023-11-24T07:59:14.651524+0000","last_active":"2023-11-24T07:59:16.292104+0000","last_peered":"2023-11-24T07:59:16.292104+0000","last_clean":"2023-11-24T07:59:16.292104+0000","last_became_active":"2023-11-24T07:59:14.651131+0000","last_became_peered":"2023-11-24T07:59:14.651131+0000","last_unstale":"2023-11-24T07:59:16.292104+0000","last_undegraded":"2023-11-24T07:59:16.292104+0000","last_fullsized":"2023-11-24T07:59:16.292104+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:59:13.620687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:59:13.620687+0000","last_clean_scrub_stamp":"2023-11-24T07:59:13.620687+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T13:48:52.893285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.1e","version":"73'1","reported_seq":13,"reported_epoch":73,"state":"active+clean","last_fresh":"2023-11-24T07:59:14.653141+0000","last_change":"2023-11-24T07:59:13.636981+0000","last_active":"2023-11-24T07:59:14.653141+0000","last_peered":"2023-11-24T07:59:14.653141+0000","last_clean":"2023-11-24T07:59:14.653141+0000","last_became_active":"2023-11-24T07:59:13.636704+0000","last_became_peered":"2023-11-24T07:59:13.636704+0000","last_unstale":"2023-11-24T07:59:14.653141+0000","last_undegraded":"2023-11-24T07:59:14.653141+0000","last_fullsized":"2023-11-24T07:59:14.653141+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:59:12.615544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:59:12.615544+0000","last_clean_scrub_stamp":"2023-11-24T07:59:12.615544+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T11:01:49.234072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.1c","version":"0'0","reported_seq":11,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-11-24T07:59:16.104026+0000","last_change":"2023-11-24T07:59:14.658068+0000","last_active":"2023-11-24T07:59:16.104026+0000","last_peered":"2023-11-24T07:59:16.104026+0000","last_clean":"2023-11-24T07:59:16.104026+0000","last_became_active":"2023-11-24T07:59:14.657772+0000","last_became_peered":"2023-11-24T07:59:14.657772+0000","last_unstale":"2023-11-24T07:59:16.104026+0000","last_undegraded":"2023-11-24T07:59:16.104026+0000","last_fullsized":"2023-11-24T07:59:16.104026+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:59:13.620687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:59:13.620687+0000","last_clean_scrub_stamp":"2023-11-24T07:59:13.620687+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:25:35.342924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.1d","version":"73'2","reported_seq":14,"reported_epoch":73,"state":"active+clean","last_fresh":"2023-11-24T07:59:14.671494+0000","last_change":"2023-11-24T07:59:13.651205+0000","last_active":"2023-11-24T07:59:14.671494+0000","last_peered":"2023-11-24T07:59:14.671494+0000","last_clean":"2023-11-24T07:59:14.671494+0000","last_became_active":"2023-11-24T07:59:13.650625+0000","last_became_peered":"2023-11-24T07:59:13.650625+0000","last_unstale":"2023-11-24T07:59:14.671494+0000","last_undegraded":"2023-11-24T07:59:14.671494+0000","last_fullsized":"2023-11-24T07:59:14.671494+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:59:12.615544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:59:12.615544+0000","last_clean_scrub_stamp":"2023-11-24T07:59:12.615544+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T11:55:58.774666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.1d","version":"0'0","reported_seq":11,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-11-24T07:59:16.292657+0000","last_change":"2023-11-24T07:59:14.646515+0000","last_active":"2023-11-24T07:59:16.292657+0000","last_peered":"2023-11-24T07:59:16.292657+0000","last_clean":"2023-11-24T07:59:16.292657+0000","last_became_active":"2023-11-24T07:59:14.646215+0000","last_became_peered":"2023-11-24T07:59:14.646215+0000","last_unstale":"2023-11-24T07:59:16.292657+0000","last_undegraded":"2023-11-24T07:59:16.292657+0000","last_fullsized":"2023-11-24T07:59:16.292657+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:59:13.620687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:59:13.620687+0000","last_clean_scrub_stamp":"2023-11-24T07:59:13.620687+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T08:19:45.582905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.1c","version":"73'2","reported_seq":14,"reported_epoch":73,"state":"active+clean","last_fresh":"2023-11-24T07:59:14.665769+0000","last_change":"2023-11-24T07:59:13.649265+0000","last_active":"2023-11-24T07:59:14.665769+0000","last_peered":"2023-11-24T07:59:14.665769+0000","last_clean":"2023-11-24T07:59:14.665769+0000","last_became_active":"2023-11-24T07:59:13.648926+0000","last_became_peered":"2023-11-24T07:59:13.648926+0000","last_unstale":"2023-11-24T07:59:14.665769+0000","last_undegraded":"2023-11-24T07:59:14.665769+0000","last_fullsized":"2023-11-24T07:59:14.665769+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:59:12.615544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:59:12.615544+0000","last_clean_scrub_stamp":"2023-11-24T07:59:12.615544+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T17:30:27.786439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.1a","version":"0'0","reported_seq":11,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-11-24T07:59:16.293185+0000","last_change":"2023-11-24T07:59:14.657208+0000","last_active":"2023-11-24T07:59:16.293185+0000","last_peered":"2023-11-24T07:59:16.293185+0000","last_clean":"2023-11-24T07:59:16.293185+0000","last_became_active":"2023-11-24T07:59:14.656892+0000","last_became_peered":"2023-11-24T07:59:14.656892+0000","last_unstale":"2023-11-24T07:59:16.293185+0000","last_undegraded":"2023-11-24T07:59:16.293185+0000","last_fullsized":"2023-11-24T07:59:16.293185+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:59:13.620687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:59:13.620687+0000","last_clean_scrub_stamp":"2023-11-24T07:59:13.620687+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T19:33:18.678728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.1b","version":"73'1","reported_seq":13,"reported_epoch":73,"state":"active+clean","last_fresh":"2023-11-24T07:59:14.666337+0000","last_change":"2023-11-24T07:59:13.655837+0000","last_active":"2023-11-24T07:59:14.666337+0000","last_peered":"2023-11-24T07:59:14.666337+0000","last_clean":"2023-11-24T07:59:14.666337+0000","last_became_active":"2023-11-24T07:59:13.655294+0000","last_became_peered":"2023-11-24T07:59:13.655294+0000","last_unstale":"2023-11-24T07:59:14.666337+0000","last_undegraded":"2023-11-24T07:59:14.666337+0000","last_fullsized":"2023-11-24T07:59:14.666337+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:59:12.615544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:59:12.615544+0000","last_clean_scrub_stamp":"2023-11-24T07:59:12.615544+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:47:41.266774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.1b","version":"0'0","reported_seq":11,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-11-24T07:59:16.104163+0000","last_change":"2023-11-24T07:59:14.658822+0000","last_active":"2023-11-24T07:59:16.104163+0000","last_peered":"2023-11-24T07:59:16.104163+0000","last_clean":"2023-11-24T07:59:16.104163+0000","last_became_active":"2023-11-24T07:59:14.658562+0000","last_became_peered":"2023-11-24T07:59:14.658562+0000","last_unstale":"2023-11-24T07:59:16.104163+0000","last_undegraded":"2023-11-24T07:59:16.104163+0000","last_fullsized":"2023-11-24T07:59:16.104163+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:59:13.620687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:59:13.620687+0000","last_clean_scrub_stamp":"2023-11-24T07:59:13.620687+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T09:43:52.247266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.1a","version":"73'1","reported_seq":13,"reported_epoch":73,"state":"active+clean","last_fresh":"2023-11-24T07:59:14.658566+0000","last_change":"2023-11-24T07:59:13.650277+0000","last_active":"2023-11-24T07:59:14.658566+0000","last_peered":"2023-11-24T07:59:14.658566+0000","last_clean":"2023-11-24T07:59:14.658566+0000","last_became_active":"2023-11-24T07:59:13.649846+0000","last_became_peered":"2023-11-24T07:59:13.649846+0000","last_unstale":"2023-11-24T07:59:14.658566+0000","last_undegraded":"2023-11-24T07:59:14.658566+0000","last_fullsized":"2023-11-24T07:59:14.658566+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:59:12.615544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:59:12.615544+0000","last_clean_scrub_stamp":"2023-11-24T07:59:12.615544+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T12:39:08.993327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.18","version":"0'0","reported_seq":11,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-11-24T07:59:15.648205+0000","last_change":"2023-11-24T07:59:14.641027+0000","last_active":"2023-11-24T07:59:15.648205+0000","last_peered":"2023-11-24T07:59:15.648205+0000","last_clean":"2023-11-24T07:59:15.648205+0000","last_became_active":"2023-11-24T07:59:14.640764+0000","last_became_peered":"2023-11-24T07:59:14.640764+0000","last_unstale":"2023-11-24T07:59:15.648205+0000","last_undegraded":"2023-11-24T07:59:15.648205+0000","last_fullsized":"2023-11-24T07:59:15.648205+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:59:13.620687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:59:13.620687+0000","last_clean_scrub_stamp":"2023-11-24T07:59:13.620687+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T19:37:04.059622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.19","version":"0'0","reported_seq":12,"reported_epoch":73,"state":"active+clean","last_fresh":"2023-11-24T07:59:14.635053+0000","last_change":"2023-11-24T07:59:13.655628+0000","last_active":"2023-11-24T07:59:14.635053+0000","last_peered":"2023-11-24T07:59:14.635053+0000","last_clean":"2023-11-24T07:59:14.635053+0000","last_became_active":"2023-11-24T07:59:13.655373+0000","last_became_peered":"2023-11-24T07:59:13.655373+0000","last_unstale":"2023-11-24T07:59:14.635053+0000","last_undegraded":"2023-11-24T07:59:14.635053+0000","last_fullsized":"2023-11-24T07:59:14.635053+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:59:12.615544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:59:12.615544+0000","last_clean_scrub_stamp":"2023-11-24T07:59:12.615544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T12:01:57.940430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.19","version":"0'0","reported_seq":11,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-11-24T07:59:15.649327+0000","last_change":"2023-11-24T07:59:14.654812+0000","last_active":"2023-11-24T07:59:15.649327+0000","last_peered":"2023-11-24T07:59:15.649327+0000","last_clean":"2023-11-24T07:59:15.649327+0000","last_became_active":"2023-11-24T07:59:14.654554+0000","last_became_peered":"2023-11-24T07:59:14.654554+0000","last_unstale":"2023-11-24T07:59:15.649327+0000","last_undegraded":"2023-11-24T07:59:15.649327+0000","last_fullsized":"2023-11-24T07:59:15.649327+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:59:13.620687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:59:13.620687+0000","last_clean_scrub_stamp":"2023-11-24T07:59:13.620687+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T14:28:14.357404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.18","version":"0'0","reported_seq":12,"reported_epoch":73,"state":"active+clean","last_fresh":"2023-11-24T07:59:14.639330+0000","last_change":"2023-11-24T07:59:13.647547+0000","last_active":"2023-11-24T07:59:14.639330+0000","last_peered":"2023-11-24T07:59:14.639330+0000","last_clean":"2023-11-24T07:59:14.639330+0000","last_became_active":"2023-11-24T07:59:13.647276+0000","last_became_peered":"2023-11-24T07:59:13.647276+0000","last_unstale":"2023-11-24T07:59:14.639330+0000","last_undegraded":"2023-11-24T07:59:14.639330+0000","last_fullsized":"2023-11-24T07:59:14.639330+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:59:12.615544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:59:12.615544+0000","last_clean_scrub_stamp":"2023-11-24T07:59:12.615544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T16:36:46.867316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.16","version":"0'0","reported_seq":11,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-11-24T07:59:16.293403+0000","last_change":"2023-11-24T07:59:14.666084+0000","last_active":"2023-11-24T07:59:16.293403+0000","last_peered":"2023-11-24T07:59:16.293403+0000","last_clean":"2023-11-24T07:59:16.293403+0000","last_became_active":"2023-11-24T07:59:14.665449+0000","last_became_peered":"2023-11-24T07:59:14.665449+0000","last_unstale":"2023-11-24T07:59:16.293403+0000","last_undegraded":"2023-11-24T07:59:16.293403+0000","last_fullsized":"2023-11-24T07:59:16.293403+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:59:13.620687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:59:13.620687+0000","last_clean_scrub_stamp":"2023-11-24T07:59:13.620687+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T19:01:46.731335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.17","version":"0'0","reported_seq":12,"reported_epoch":73,"state":"active+clean","last_fresh":"2023-11-24T07:59:14.645021+0000","last_change":"2023-11-24T07:59:13.652504+0000","last_active":"2023-11-24T07:59:14.645021+0000","last_peered":"2023-11-24T07:59:14.645021+0000","last_clean":"2023-11-24T07:59:14.645021+0000","last_became_active":"2023-11-24T07:59:13.652187+0000","last_became_peered":"2023-11-24T07:59:13.652187+0000","last_unstale":"2023-11-24T07:59:14.645021+0000","last_undegraded":"2023-11-24T07:59:14.645021+0000","last_fullsized":"2023-11-24T07:59:14.645021+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:59:12.615544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:59:12.615544+0000","last_clean_scrub_stamp":"2023-11-24T07:59:12.615544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T17:42:54.827147+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.17","version":"0'0","reported_seq":11,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-11-24T07:59:16.293529+0000","last_change":"2023-11-24T07:59:14.669203+0000","last_active":"2023-11-24T07:59:16.293529+0000","last_peered":"2023-11-24T07:59:16.293529+0000","last_clean":"2023-11-24T07:59:16.293529+0000","last_became_active":"2023-11-24T07:59:14.668944+0000","last_became_peered":"2023-11-24T07:59:14.668944+0000","last_unstale":"2023-11-24T07:59:16.293529+0000","last_undegraded":"2023-11-24T07:59:16.293529+0000","last_fullsized":"2023-11-24T07:59:16.293529+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:59:13.620687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:59:13.620687+0000","last_clean_scrub_stamp":"2023-11-24T07:59:13.620687+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T15:47:40.090700+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.16","version":"0'0","reported_seq":12,"reported_epoch":73,"state":"active+clean","last_fresh":"2023-11-24T07:59:14.646587+0000","last_change":"2023-11-24T07:59:13.646811+0000","last_active":"2023-11-24T07:59:14.646587+0000","last_peered":"2023-11-24T07:59:14.646587+0000","last_clean":"2023-11-24T07:59:14.646587+0000","last_became_active":"2023-11-24T07:59:13.646563+0000","last_became_peered":"2023-11-24T07:59:13.646563+0000","last_unstale":"2023-11-24T07:59:14.646587+0000","last_undegraded":"2023-11-24T07:59:14.646587+0000","last_fullsized":"2023-11-24T07:59:14.646587+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:59:12.615544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:59:12.615544+0000","last_clean_scrub_stamp":"2023-11-24T07:59:12.615544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T17:15:53.221458+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.14","version":"0'0","reported_seq":11,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-11-24T07:59:16.293297+0000","last_change":"2023-11-24T07:59:14.657194+0000","last_active":"2023-11-24T07:59:16.293297+0000","last_peered":"2023-11-24T07:59:16.293297+0000","last_clean":"2023-11-24T07:59:16.293297+0000","last_became_active":"2023-11-24T07:59:14.656915+0000","last_became_peered":"2023-11-24T07:59:14.656915+0000","last_unstale":"2023-11-24T07:59:16.293297+0000","last_undegraded":"2023-11-24T07:59:16.293297+0000","last_fullsized":"2023-11-24T07:59:16.293297+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:59:13.620687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:59:13.620687+0000","last_clean_scrub_stamp":"2023-11-24T07:59:13.620687+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T16:19:25.371696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.15","version":"73'3","reported_seq":15,"reported_epoch":73,"state":"active+clean","last_fresh":"2023-11-24T07:59:14.656254+0000","last_change":"2023-11-24T07:59:13.645182+0000","last_active":"2023-11-24T07:59:14.656254+0000","last_peered":"2023-11-24T07:59:14.656254+0000","last_clean":"2023-11-24T07:59:14.656254+0000","last_became_active":"2023-11-24T07:59:13.644887+0000","last_became_peered":"2023-11-24T07:59:13.644887+0000","last_unstale":"2023-11-24T07:59:14.656254+0000","last_undegraded":"2023-11-24T07:59:14.656254+0000","last_fullsized":"2023-11-24T07:59:14.656254+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:59:12.615544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:59:12.615544+0000","last_clean_scrub_stamp":"2023-11-24T07:59:12.615544+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T19:44:36.104587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.15","version":"0'0","reported_seq":11,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-11-24T07:59:15.650622+0000","last_change":"2023-11-24T07:59:14.648158+0000","last_active":"2023-11-24T07:59:15.650622+0000","last_peered":"2023-11-24T07:59:15.650622+0000","last_clean":"2023-11-24T07:59:15.650622+0000","last_became_active":"2023-11-24T07:59:14.647759+0000","last_became_peered":"2023-11-24T07:59:14.647759+0000","last_unstale":"2023-11-24T07:59:15.650622+0000","last_undegraded":"2023-11-24T07:59:15.650622+0000","last_fullsized":"2023-11-24T07:59:15.650622+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:59:13.620687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:59:13.620687+0000","last_clean_scrub_stamp":"2023-11-24T07:59:13.620687+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:15:07.828806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.14","version":"73'6","reported_seq":18,"reported_epoch":73,"state":"active+clean","last_fresh":"2023-11-24T07:59:14.668309+0000","last_change":"2023-11-24T07:59:13.647480+0000","last_active":"2023-11-24T07:59:14.668309+0000","last_peered":"2023-11-24T07:59:14.668309+0000","last_clean":"2023-11-24T07:59:14.668309+0000","last_became_active":"2023-11-24T07:59:13.647183+0000","last_became_peered":"2023-11-24T07:59:13.647183+0000","last_unstale":"2023-11-24T07:59:14.668309+0000","last_undegraded":"2023-11-24T07:59:14.668309+0000","last_fullsized":"2023-11-24T07:59:14.668309+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:59:12.615544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:59:12.615544+0000","last_clean_scrub_stamp":"2023-11-24T07:59:12.615544+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T13:25:11.926864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.12","version":"0'0","reported_seq":11,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-11-24T07:59:16.103875+0000","last_change":"2023-11-24T07:59:14.647988+0000","last_active":"2023-11-24T07:59:16.103875+0000","last_peered":"2023-11-24T07:59:16.103875+0000","last_clean":"2023-11-24T07:59:16.103875+0000","last_became_active":"2023-11-24T07:59:14.647702+0000","last_became_peered":"2023-11-24T07:59:14.647702+0000","last_unstale":"2023-11-24T07:59:16.103875+0000","last_undegraded":"2023-11-24T07:59:16.103875+0000","last_fullsized":"2023-11-24T07:59:16.103875+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:59:13.620687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:59:13.620687+0000","last_clean_scrub_stamp":"2023-11-24T07:59:13.620687+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T09:23:20.285028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.13","version":"73'2","reported_seq":14,"reported_epoch":73,"state":"active+clean","last_fresh":"2023-11-24T07:59:14.665155+0000","last_change":"2023-11-24T07:59:13.652223+0000","last_active":"2023-11-24T07:59:14.665155+0000","last_peered":"2023-11-24T07:59:14.665155+0000","last_clean":"2023-11-24T07:59:14.665155+0000","last_became_active":"2023-11-24T07:59:13.651976+0000","last_became_peered":"2023-11-24T07:59:13.651976+0000","last_unstale":"2023-11-24T07:59:14.665155+0000","last_undegraded":"2023-11-24T07:59:14.665155+0000","last_fullsized":"2023-11-24T07:59:14.665155+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:59:12.615544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:59:12.615544+0000","last_clean_scrub_stamp":"2023-11-24T07:59:12.615544+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:47:41.266774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.13","version":"0'0","reported_seq":11,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-11-24T07:59:16.292350+0000","last_change":"2023-11-24T07:59:14.660260+0000","last_active":"2023-11-24T07:59:16.292350+0000","last_peered":"2023-11-24T07:59:16.292350+0000","last_clean":"2023-11-24T07:59:16.292350+0000","last_became_active":"2023-11-24T07:59:14.659942+0000","last_became_peered":"2023-11-24T07:59:14.659942+0000","last_unstale":"2023-11-24T07:59:16.292350+0000","last_undegraded":"2023-11-24T07:59:16.292350+0000","last_fullsized":"2023-11-24T07:59:16.292350+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:59:13.620687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:59:13.620687+0000","last_clean_scrub_stamp":"2023-11-24T07:59:13.620687+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T13:48:52.893285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.12","version":"0'0","reported_seq":12,"reported_epoch":73,"state":"active+clean","last_fresh":"2023-11-24T07:59:14.634946+0000","last_change":"2023-11-24T07:59:13.643551+0000","last_active":"2023-11-24T07:59:14.634946+0000","last_peered":"2023-11-24T07:59:14.634946+0000","last_clean":"2023-11-24T07:59:14.634946+0000","last_became_active":"2023-11-24T07:59:13.643276+0000","last_became_peered":"2023-11-24T07:59:13.643276+0000","last_unstale":"2023-11-24T07:59:14.634946+0000","last_undegraded":"2023-11-24T07:59:14.634946+0000","last_fullsized":"2023-11-24T07:59:14.634946+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:59:12.615544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:59:12.615544+0000","last_clean_scrub_stamp":"2023-11-24T07:59:12.615544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T14:50:22.913346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.10","version":"0'0","reported_seq":11,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-11-24T07:59:15.650380+0000","last_change":"2023-11-24T07:59:14.641647+0000","last_active":"2023-11-24T07:59:15.650380+0000","last_peered":"2023-11-24T07:59:15.650380+0000","last_clean":"2023-11-24T07:59:15.650380+0000","last_became_active":"2023-11-24T07:59:14.641383+0000","last_became_peered":"2023-11-24T07:59:14.641383+0000","last_unstale":"2023-11-24T07:59:15.650380+0000","last_undegraded":"2023-11-24T07:59:15.650380+0000","last_fullsized":"2023-11-24T07:59:15.650380+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:59:13.620687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:59:13.620687+0000","last_clean_scrub_stamp":"2023-11-24T07:59:13.620687+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T08:18:30.845129+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.11","version":"0'0","reported_seq":12,"reported_epoch":73,"state":"active+clean","last_fresh":"2023-11-24T07:59:14.636353+0000","last_change":"2023-11-24T07:59:13.650108+0000","last_active":"2023-11-24T07:59:14.636353+0000","last_peered":"2023-11-24T07:59:14.636353+0000","last_clean":"2023-11-24T07:59:14.636353+0000","last_became_active":"2023-11-24T07:59:13.649781+0000","last_became_peered":"2023-11-24T07:59:13.649781+0000","last_unstale":"2023-11-24T07:59:14.636353+0000","last_undegraded":"2023-11-24T07:59:14.636353+0000","last_fullsized":"2023-11-24T07:59:14.636353+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:59:12.615544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:59:12.615544+0000","last_clean_scrub_stamp":"2023-11-24T07:59:12.615544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T19:09:46.221313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.11","version":"0'0","reported_seq":11,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-11-24T07:59:16.291803+0000","last_change":"2023-11-24T07:59:14.648803+0000","last_active":"2023-11-24T07:59:16.291803+0000","last_peered":"2023-11-24T07:59:16.291803+0000","last_clean":"2023-11-24T07:59:16.291803+0000","last_became_active":"2023-11-24T07:59:14.644779+0000","last_became_peered":"2023-11-24T07:59:14.644779+0000","last_unstale":"2023-11-24T07:59:16.291803+0000","last_undegraded":"2023-11-24T07:59:16.291803+0000","last_fullsized":"2023-11-24T07:59:16.291803+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:59:13.620687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:59:13.620687+0000","last_clean_scrub_stamp":"2023-11-24T07:59:13.620687+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T13:30:24.640475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.10","version":"73'8","reported_seq":20,"reported_epoch":73,"state":"active+clean","last_fresh":"2023-11-24T07:59:14.667012+0000","last_change":"2023-11-24T07:59:13.650307+0000","last_active":"2023-11-24T07:59:14.667012+0000","last_peered":"2023-11-24T07:59:14.667012+0000","last_clean":"2023-11-24T07:59:14.667012+0000","last_became_active":"2023-11-24T07:59:13.649976+0000","last_became_peered":"2023-11-24T07:59:13.649976+0000","last_unstale":"2023-11-24T07:59:14.667012+0000","last_undegraded":"2023-11-24T07:59:14.667012+0000","last_fullsized":"2023-11-24T07:59:14.667012+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:59:12.615544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:59:12.615544+0000","last_clean_scrub_stamp":"2023-11-24T07:59:12.615544+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T15:34:17.729593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.e","version":"0'0","reported_seq":11,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-11-24T07:59:16.293506+0000","last_change":"2023-11-24T07:59:14.662540+0000","last_active":"2023-11-24T07:59:16.293506+0000","last_peered":"2023-11-24T07:59:16.293506+0000","last_clean":"2023-11-24T07:59:16.293506+0000","last_became_active":"2023-11-24T07:59:14.662109+0000","last_became_peered":"2023-11-24T07:59:14.662109+0000","last_unstale":"2023-11-24T07:59:16.293506+0000","last_undegraded":"2023-11-24T07:59:16.293506+0000","last_fullsized":"2023-11-24T07:59:16.293506+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:59:13.620687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:59:13.620687+0000","last_clean_scrub_stamp":"2023-11-24T07:59:13.620687+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T15:27:01.624152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.f","version":"73'1","reported_seq":13,"reported_epoch":73,"state":"active+clean","last_fresh":"2023-11-24T07:59:14.659241+0000","last_change":"2023-11-24T07:59:13.651768+0000","last_active":"2023-11-24T07:59:14.659241+0000","last_peered":"2023-11-24T07:59:14.659241+0000","last_clean":"2023-11-24T07:59:14.659241+0000","last_became_active":"2023-11-24T07:59:13.650680+0000","last_became_peered":"2023-11-24T07:59:13.650680+0000","last_unstale":"2023-11-24T07:59:14.659241+0000","last_undegraded":"2023-11-24T07:59:14.659241+0000","last_fullsized":"2023-11-24T07:59:14.659241+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:59:12.615544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:59:12.615544+0000","last_clean_scrub_stamp":"2023-11-24T07:59:12.615544+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T11:25:10.026153+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.f","version":"0'0","reported_seq":11,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-11-24T07:59:16.293622+0000","last_change":"2023-11-24T07:59:14.668869+0000","last_active":"2023-11-24T07:59:16.293622+0000","last_peered":"2023-11-24T07:59:16.293622+0000","last_clean":"2023-11-24T07:59:16.293622+0000","last_became_active":"2023-11-24T07:59:14.668493+0000","last_became_peered":"2023-11-24T07:59:14.668493+0000","last_unstale":"2023-11-24T07:59:16.293622+0000","last_undegraded":"2023-11-24T07:59:16.293622+0000","last_fullsized":"2023-11-24T07:59:16.293622+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:59:13.620687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:59:13.620687+0000","last_clean_scrub_stamp":"2023-11-24T07:59:13.620687+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T17:30:28.791582+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.e","version":"73'2","reported_seq":14,"reported_epoch":73,"state":"active+clean","last_fresh":"2023-11-24T07:59:14.668349+0000","last_change":"2023-11-24T07:59:13.646781+0000","last_active":"2023-11-24T07:59:14.668349+0000","last_peered":"2023-11-24T07:59:14.668349+0000","last_clean":"2023-11-24T07:59:14.668349+0000","last_became_active":"2023-11-24T07:59:13.646208+0000","last_became_peered":"2023-11-24T07:59:13.646208+0000","last_unstale":"2023-11-24T07:59:14.668349+0000","last_undegraded":"2023-11-24T07:59:14.668349+0000","last_fullsized":"2023-11-24T07:59:14.668349+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:59:12.615544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:59:12.615544+0000","last_clean_scrub_stamp":"2023-11-24T07:59:12.615544+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T19:37:03.054479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.c","version":"0'0","reported_seq":11,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-11-24T07:59:16.292086+0000","last_change":"2023-11-24T07:59:14.649728+0000","last_active":"2023-11-24T07:59:16.292086+0000","last_peered":"2023-11-24T07:59:16.292086+0000","last_clean":"2023-11-24T07:59:16.292086+0000","last_became_active":"2023-11-24T07:59:14.649405+0000","last_became_peered":"2023-11-24T07:59:14.649405+0000","last_unstale":"2023-11-24T07:59:16.292086+0000","last_undegraded":"2023-11-24T07:59:16.292086+0000","last_fullsized":"2023-11-24T07:59:16.292086+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:59:13.620687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:59:13.620687+0000","last_clean_scrub_stamp":"2023-11-24T07:59:13.620687+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:54:57.076700+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.d","version":"73'1","reported_seq":13,"reported_epoch":73,"state":"active+clean","last_fresh":"2023-11-24T07:59:14.668658+0000","last_change":"2023-11-24T07:59:13.652598+0000","last_active":"2023-11-24T07:59:14.668658+0000","last_peered":"2023-11-24T07:59:14.668658+0000","last_clean":"2023-11-24T07:59:14.668658+0000","last_became_active":"2023-11-24T07:59:13.652005+0000","last_became_peered":"2023-11-24T07:59:13.652005+0000","last_unstale":"2023-11-24T07:59:14.668658+0000","last_undegraded":"2023-11-24T07:59:14.668658+0000","last_fullsized":"2023-11-24T07:59:14.668658+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:59:12.615544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:59:12.615544+0000","last_clean_scrub_stamp":"2023-11-24T07:59:12.615544+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T14:07:52.677308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.d","version":"0'0","reported_seq":11,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-11-24T07:59:16.291967+0000","last_change":"2023-11-24T07:59:14.664027+0000","last_active":"2023-11-24T07:59:16.291967+0000","last_peered":"2023-11-24T07:59:16.291967+0000","last_clean":"2023-11-24T07:59:16.291967+0000","last_became_active":"2023-11-24T07:59:14.663669+0000","last_became_peered":"2023-11-24T07:59:14.663669+0000","last_unstale":"2023-11-24T07:59:16.291967+0000","last_undegraded":"2023-11-24T07:59:16.291967+0000","last_fullsized":"2023-11-24T07:59:16.291967+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:59:13.620687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:59:13.620687+0000","last_clean_scrub_stamp":"2023-11-24T07:59:13.620687+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T08:09:54.291825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.c","version":"0'0","reported_seq":12,"reported_epoch":73,"state":"active+clean","last_fresh":"2023-11-24T07:59:14.635643+0000","last_change":"2023-11-24T07:59:13.655857+0000","last_active":"2023-11-24T07:59:14.635643+0000","last_peered":"2023-11-24T07:59:14.635643+0000","last_clean":"2023-11-24T07:59:14.635643+0000","last_became_active":"2023-11-24T07:59:13.655496+0000","last_became_peered":"2023-11-24T07:59:13.655496+0000","last_unstale":"2023-11-24T07:59:14.635643+0000","last_undegraded":"2023-11-24T07:59:14.635643+0000","last_fullsized":"2023-11-24T07:59:14.635643+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:59:12.615544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:59:12.615544+0000","last_clean_scrub_stamp":"2023-11-24T07:59:12.615544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T13:00:13.113468+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.b","version":"0'0","reported_seq":11,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-11-24T07:59:15.648400+0000","last_change":"2023-11-24T07:59:14.640493+0000","last_active":"2023-11-24T07:59:15.648400+0000","last_peered":"2023-11-24T07:59:15.648400+0000","last_clean":"2023-11-24T07:59:15.648400+0000","last_became_active":"2023-11-24T07:59:14.640172+0000","last_became_peered":"2023-11-24T07:59:14.640172+0000","last_unstale":"2023-11-24T07:59:15.648400+0000","last_undegraded":"2023-11-24T07:59:15.648400+0000","last_fullsized":"2023-11-24T07:59:15.648400+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:59:13.620687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:59:13.620687+0000","last_clean_scrub_stamp":"2023-11-24T07:59:13.620687+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T14:28:14.357404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.a","version":"0'0","reported_seq":12,"reported_epoch":73,"state":"active+clean","last_fresh":"2023-11-24T07:59:14.635916+0000","last_change":"2023-11-24T07:59:13.653387+0000","last_active":"2023-11-24T07:59:14.635916+0000","last_peered":"2023-11-24T07:59:14.635916+0000","last_clean":"2023-11-24T07:59:14.635916+0000","last_became_active":"2023-11-24T07:59:13.653132+0000","last_became_peered":"2023-11-24T07:59:13.653132+0000","last_unstale":"2023-11-24T07:59:14.635916+0000","last_undegraded":"2023-11-24T07:59:14.635916+0000","last_fullsized":"2023-11-24T07:59:14.635916+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:59:12.615544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:59:12.615544+0000","last_clean_scrub_stamp":"2023-11-24T07:59:12.615544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T16:41:01.392188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.8","version":"0'0","reported_seq":11,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-11-24T07:59:15.648245+0000","last_change":"2023-11-24T07:59:14.644467+0000","last_active":"2023-11-24T07:59:15.648245+0000","last_peered":"2023-11-24T07:59:15.648245+0000","last_clean":"2023-11-24T07:59:15.648245+0000","last_became_active":"2023-11-24T07:59:14.644057+0000","last_became_peered":"2023-11-24T07:59:14.644057+0000","last_unstale":"2023-11-24T07:59:15.648245+0000","last_undegraded":"2023-11-24T07:59:15.648245+0000","last_fullsized":"2023-11-24T07:59:15.648245+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:59:13.620687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:59:13.620687+0000","last_clean_scrub_stamp":"2023-11-24T07:59:13.620687+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T19:44:37.109730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.9","version":"0'0","reported_seq":12,"reported_epoch":73,"state":"active+clean","last_fresh":"2023-11-24T07:59:14.646912+0000","last_change":"2023-11-24T07:59:13.649520+0000","last_active":"2023-11-24T07:59:14.646912+0000","last_peered":"2023-11-24T07:59:14.646912+0000","last_clean":"2023-11-24T07:59:14.646912+0000","last_became_active":"2023-11-24T07:59:13.649255+0000","last_became_peered":"2023-11-24T07:59:13.649255+0000","last_unstale":"2023-11-24T07:59:14.646912+0000","last_undegraded":"2023-11-24T07:59:14.646912+0000","last_fullsized":"2023-11-24T07:59:14.646912+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:59:12.615544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:59:12.615544+0000","last_clean_scrub_stamp":"2023-11-24T07:59:12.615544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T19:44:36.104587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.9","version":"0'0","reported_seq":11,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-11-24T07:59:15.649420+0000","last_change":"2023-11-24T07:59:14.653039+0000","last_active":"2023-11-24T07:59:15.649420+0000","last_peered":"2023-11-24T07:59:15.649420+0000","last_clean":"2023-11-24T07:59:15.649420+0000","last_became_active":"2023-11-24T07:59:14.652751+0000","last_became_peered":"2023-11-24T07:59:14.652751+0000","last_unstale":"2023-11-24T07:59:15.649420+0000","last_undegraded":"2023-11-24T07:59:15.649420+0000","last_fullsized":"2023-11-24T07:59:15.649420+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:59:13.620687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:59:13.620687+0000","last_clean_scrub_stamp":"2023-11-24T07:59:13.620687+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T15:26:05.128703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.8","version":"0'0","reported_seq":12,"reported_epoch":73,"state":"active+clean","last_fresh":"2023-11-24T07:59:14.635324+0000","last_change":"2023-11-24T07:59:13.641509+0000","last_active":"2023-11-24T07:59:14.635324+0000","last_peered":"2023-11-24T07:59:14.635324+0000","last_clean":"2023-11-24T07:59:14.635324+0000","last_became_active":"2023-11-24T07:59:13.641197+0000","last_became_peered":"2023-11-24T07:59:13.641197+0000","last_unstale":"2023-11-24T07:59:14.635324+0000","last_undegraded":"2023-11-24T07:59:14.635324+0000","last_fullsized":"2023-11-24T07:59:14.635324+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:59:12.615544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:59:12.615544+0000","last_clean_scrub_stamp":"2023-11-24T07:59:12.615544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T17:23:58.604523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.6","version":"0'0","reported_seq":11,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-11-24T07:59:15.650866+0000","last_change":"2023-11-24T07:59:14.641587+0000","last_active":"2023-11-24T07:59:15.650866+0000","last_peered":"2023-11-24T07:59:15.650866+0000","last_clean":"2023-11-24T07:59:15.650866+0000","last_became_active":"2023-11-24T07:59:14.641303+0000","last_became_peered":"2023-11-24T07:59:14.641303+0000","last_unstale":"2023-11-24T07:59:15.650866+0000","last_undegraded":"2023-11-24T07:59:15.650866+0000","last_fullsized":"2023-11-24T07:59:15.650866+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:59:13.620687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:59:13.620687+0000","last_clean_scrub_stamp":"2023-11-24T07:59:13.620687+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T15:14:11.003165+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.7","version":"73'2","reported_seq":14,"reported_epoch":73,"state":"active+clean","last_fresh":"2023-11-24T07:59:14.663660+0000","last_change":"2023-11-24T07:59:13.652389+0000","last_active":"2023-11-24T07:59:14.663660+0000","last_peered":"2023-11-24T07:59:14.663660+0000","last_clean":"2023-11-24T07:59:14.663660+0000","last_became_active":"2023-11-24T07:59:13.652073+0000","last_became_peered":"2023-11-24T07:59:13.652073+0000","last_unstale":"2023-11-24T07:59:14.663660+0000","last_undegraded":"2023-11-24T07:59:14.663660+0000","last_fullsized":"2023-11-24T07:59:14.663660+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:59:12.615544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:59:12.615544+0000","last_clean_scrub_stamp":"2023-11-24T07:59:12.615544+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T16:26:10.524076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.7","version":"0'0","reported_seq":11,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-11-24T07:59:16.292672+0000","last_change":"2023-11-24T07:59:14.643008+0000","last_active":"2023-11-24T07:59:16.292672+0000","last_peered":"2023-11-24T07:59:16.292672+0000","last_clean":"2023-11-24T07:59:16.292672+0000","last_became_active":"2023-11-24T07:59:14.642769+0000","last_became_peered":"2023-11-24T07:59:14.642769+0000","last_unstale":"2023-11-24T07:59:16.292672+0000","last_undegraded":"2023-11-24T07:59:16.292672+0000","last_fullsized":"2023-11-24T07:59:16.292672+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:59:13.620687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:59:13.620687+0000","last_clean_scrub_stamp":"2023-11-24T07:59:13.620687+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T18:08:53.846914+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.6","version":"0'0","reported_seq":12,"reported_epoch":73,"state":"active+clean","last_fresh":"2023-11-24T07:59:14.650495+0000","last_change":"2023-11-24T07:59:13.646769+0000","last_active":"2023-11-24T07:59:14.650495+0000","last_peered":"2023-11-24T07:59:14.650495+0000","last_clean":"2023-11-24T07:59:14.650495+0000","last_became_active":"2023-11-24T07:59:13.646438+0000","last_became_peered":"2023-11-24T07:59:13.646438+0000","last_unstale":"2023-11-24T07:59:14.650495+0000","last_undegraded":"2023-11-24T07:59:14.650495+0000","last_fullsized":"2023-11-24T07:59:14.650495+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:59:12.615544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:59:12.615544+0000","last_clean_scrub_stamp":"2023-11-24T07:59:12.615544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T09:33:05.362263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.4","version":"0'0","reported_seq":11,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-11-24T07:59:16.294066+0000","last_change":"2023-11-24T07:59:14.660249+0000","last_active":"2023-11-24T07:59:16.294066+0000","last_peered":"2023-11-24T07:59:16.294066+0000","last_clean":"2023-11-24T07:59:16.294066+0000","last_became_active":"2023-11-24T07:59:14.659550+0000","last_became_peered":"2023-11-24T07:59:14.659550+0000","last_unstale":"2023-11-24T07:59:16.294066+0000","last_undegraded":"2023-11-24T07:59:16.294066+0000","last_fullsized":"2023-11-24T07:59:16.294066+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:59:13.620687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:59:13.620687+0000","last_clean_scrub_stamp":"2023-11-24T07:59:13.620687+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T16:41:02.397331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.5","version":"0'0","reported_seq":12,"reported_epoch":73,"state":"active+clean","last_fresh":"2023-11-24T07:59:14.644674+0000","last_change":"2023-11-24T07:59:13.647782+0000","last_active":"2023-11-24T07:59:14.644674+0000","last_peered":"2023-11-24T07:59:14.644674+0000","last_clean":"2023-11-24T07:59:14.644674+0000","last_became_active":"2023-11-24T07:59:13.647473+0000","last_became_peered":"2023-11-24T07:59:13.647473+0000","last_unstale":"2023-11-24T07:59:14.644674+0000","last_undegraded":"2023-11-24T07:59:14.644674+0000","last_fullsized":"2023-11-24T07:59:14.644674+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:59:12.615544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:59:12.615544+0000","last_clean_scrub_stamp":"2023-11-24T07:59:12.615544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T09:55:49.599221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.5","version":"0'0","reported_seq":11,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-11-24T07:59:16.292638+0000","last_change":"2023-11-24T07:59:14.641861+0000","last_active":"2023-11-24T07:59:16.292638+0000","last_peered":"2023-11-24T07:59:16.292638+0000","last_clean":"2023-11-24T07:59:16.292638+0000","last_became_active":"2023-11-24T07:59:14.641558+0000","last_became_peered":"2023-11-24T07:59:14.641558+0000","last_unstale":"2023-11-24T07:59:16.292638+0000","last_undegraded":"2023-11-24T07:59:16.292638+0000","last_fullsized":"2023-11-24T07:59:16.292638+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:59:13.620687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:59:13.620687+0000","last_clean_scrub_stamp":"2023-11-24T07:59:13.620687+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T12:33:01.714569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.4","version":"0'0","reported_seq":12,"reported_epoch":73,"state":"active+clean","last_fresh":"2023-11-24T07:59:14.643365+0000","last_change":"2023-11-24T07:59:13.645153+0000","last_active":"2023-11-24T07:59:14.643365+0000","last_peered":"2023-11-24T07:59:14.643365+0000","last_clean":"2023-11-24T07:59:14.643365+0000","last_became_active":"2023-11-24T07:59:13.644868+0000","last_became_peered":"2023-11-24T07:59:13.644868+0000","last_unstale":"2023-11-24T07:59:14.643365+0000","last_undegraded":"2023-11-24T07:59:14.643365+0000","last_fullsized":"2023-11-24T07:59:14.643365+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:59:12.615544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:59:12.615544+0000","last_clean_scrub_stamp":"2023-11-24T07:59:12.615544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T19:37:03.054479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.2","version":"0'0","reported_seq":11,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-11-24T07:59:15.650594+0000","last_change":"2023-11-24T07:59:14.640912+0000","last_active":"2023-11-24T07:59:15.650594+0000","last_peered":"2023-11-24T07:59:15.650594+0000","last_clean":"2023-11-24T07:59:15.650594+0000","last_became_active":"2023-11-24T07:59:14.640633+0000","last_became_peered":"2023-11-24T07:59:14.640633+0000","last_unstale":"2023-11-24T07:59:15.650594+0000","last_undegraded":"2023-11-24T07:59:15.650594+0000","last_fullsized":"2023-11-24T07:59:15.650594+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:59:13.620687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:59:13.620687+0000","last_clean_scrub_stamp":"2023-11-24T07:59:13.620687+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T08:09:53.053268+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.3","version":"0'0","reported_seq":12,"reported_epoch":73,"state":"active+clean","last_fresh":"2023-11-24T07:59:14.642373+0000","last_change":"2023-11-24T07:59:13.655482+0000","last_active":"2023-11-24T07:59:14.642373+0000","last_peered":"2023-11-24T07:59:14.642373+0000","last_clean":"2023-11-24T07:59:14.642373+0000","last_became_active":"2023-11-24T07:59:13.655191+0000","last_became_peered":"2023-11-24T07:59:13.655191+0000","last_unstale":"2023-11-24T07:59:14.642373+0000","last_undegraded":"2023-11-24T07:59:14.642373+0000","last_fullsized":"2023-11-24T07:59:14.642373+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:59:12.615544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:59:12.615544+0000","last_clean_scrub_stamp":"2023-11-24T07:59:12.615544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T12:01:57.940430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.3","version":"0'0","reported_seq":11,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-11-24T07:59:16.292441+0000","last_change":"2023-11-24T07:59:14.669887+0000","last_active":"2023-11-24T07:59:16.292441+0000","last_peered":"2023-11-24T07:59:16.292441+0000","last_clean":"2023-11-24T07:59:16.292441+0000","last_became_active":"2023-11-24T07:59:14.669606+0000","last_became_peered":"2023-11-24T07:59:14.669606+0000","last_unstale":"2023-11-24T07:59:16.292441+0000","last_undegraded":"2023-11-24T07:59:16.292441+0000","last_fullsized":"2023-11-24T07:59:16.292441+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:59:13.620687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:59:13.620687+0000","last_clean_scrub_stamp":"2023-11-24T07:59:13.620687+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:09:08.315869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.2","version":"0'0","reported_seq":12,"reported_epoch":73,"state":"active+clean","last_fresh":"2023-11-24T07:59:14.634770+0000","last_change":"2023-11-24T07:59:13.649750+0000","last_active":"2023-11-24T07:59:14.634770+0000","last_peered":"2023-11-24T07:59:14.634770+0000","last_clean":"2023-11-24T07:59:14.634770+0000","last_became_active":"2023-11-24T07:59:13.649437+0000","last_became_peered":"2023-11-24T07:59:13.649437+0000","last_unstale":"2023-11-24T07:59:14.634770+0000","last_undegraded":"2023-11-24T07:59:14.634770+0000","last_fullsized":"2023-11-24T07:59:14.634770+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:59:12.615544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:59:12.615544+0000","last_clean_scrub_stamp":"2023-11-24T07:59:12.615544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:52:45.085545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.0","version":"0'0","reported_seq":11,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-11-24T07:59:15.648370+0000","last_change":"2023-11-24T07:59:14.667968+0000","last_active":"2023-11-24T07:59:15.648370+0000","last_peered":"2023-11-24T07:59:15.648370+0000","last_clean":"2023-11-24T07:59:15.648370+0000","last_became_active":"2023-11-24T07:59:14.667776+0000","last_became_peered":"2023-11-24T07:59:14.667776+0000","last_unstale":"2023-11-24T07:59:15.648370+0000","last_undegraded":"2023-11-24T07:59:15.648370+0000","last_fullsized":"2023-11-24T07:59:15.648370+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:59:13.620687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:59:13.620687+0000","last_clean_scrub_stamp":"2023-11-24T07:59:13.620687+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T09:55:50.604364+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.1","version":"0'0","reported_seq":12,"reported_epoch":73,"state":"active+clean","last_fresh":"2023-11-24T07:59:14.647065+0000","last_change":"2023-11-24T07:59:13.647623+0000","last_active":"2023-11-24T07:59:14.647065+0000","last_peered":"2023-11-24T07:59:14.647065+0000","last_clean":"2023-11-24T07:59:14.647065+0000","last_became_active":"2023-11-24T07:59:13.647322+0000","last_became_peered":"2023-11-24T07:59:13.647322+0000","last_unstale":"2023-11-24T07:59:14.647065+0000","last_undegraded":"2023-11-24T07:59:14.647065+0000","last_fullsized":"2023-11-24T07:59:14.647065+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:59:12.615544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:59:12.615544+0000","last_clean_scrub_stamp":"2023-11-24T07:59:12.615544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T09:55:49.599221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1","version":"0'0","reported_seq":11,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-11-24T07:59:16.293522+0000","last_change":"2023-11-24T07:59:14.659942+0000","last_active":"2023-11-24T07:59:16.293522+0000","last_peered":"2023-11-24T07:59:16.293522+0000","last_clean":"2023-11-24T07:59:16.293522+0000","last_became_active":"2023-11-24T07:59:14.659603+0000","last_became_peered":"2023-11-24T07:59:14.659603+0000","last_unstale":"2023-11-24T07:59:16.293522+0000","last_undegraded":"2023-11-24T07:59:16.293522+0000","last_fullsized":"2023-11-24T07:59:16.293522+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:59:13.620687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:59:13.620687+0000","last_clean_scrub_stamp":"2023-11-24T07:59:13.620687+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T14:38:55.028542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.0","version":"0'0","reported_seq":12,"reported_epoch":73,"state":"active+clean","last_fresh":"2023-11-24T07:59:14.635327+0000","last_change":"2023-11-24T07:59:13.647308+0000","last_active":"2023-11-24T07:59:14.635327+0000","last_peered":"2023-11-24T07:59:14.635327+0000","last_clean":"2023-11-24T07:59:14.635327+0000","last_became_active":"2023-11-24T07:59:13.646512+0000","last_became_peered":"2023-11-24T07:59:13.646512+0000","last_unstale":"2023-11-24T07:59:14.635327+0000","last_undegraded":"2023-11-24T07:59:14.635327+0000","last_fullsized":"2023-11-24T07:59:14.635327+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:59:12.615544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:59:12.615544+0000","last_clean_scrub_stamp":"2023-11-24T07:59:12.615544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T15:47:39.085557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.a","version":"0'0","reported_seq":11,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-11-24T07:59:15.650713+0000","last_change":"2023-11-24T07:59:14.661867+0000","last_active":"2023-11-24T07:59:15.650713+0000","last_peered":"2023-11-24T07:59:15.650713+0000","last_clean":"2023-11-24T07:59:15.650713+0000","last_became_active":"2023-11-24T07:59:14.661579+0000","last_became_peered":"2023-11-24T07:59:14.661579+0000","last_unstale":"2023-11-24T07:59:15.650713+0000","last_undegraded":"2023-11-24T07:59:15.650713+0000","last_fullsized":"2023-11-24T07:59:15.650713+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:59:13.620687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:59:13.620687+0000","last_clean_scrub_stamp":"2023-11-24T07:59:13.620687+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T17:42:55.832290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.b","version":"73'2","reported_seq":14,"reported_epoch":73,"state":"active+clean","last_fresh":"2023-11-24T07:59:14.656594+0000","last_change":"2023-11-24T07:59:13.650073+0000","last_active":"2023-11-24T07:59:14.656594+0000","last_peered":"2023-11-24T07:59:14.656594+0000","last_clean":"2023-11-24T07:59:14.656594+0000","last_became_active":"2023-11-24T07:59:13.649748+0000","last_became_peered":"2023-11-24T07:59:13.649748+0000","last_unstale":"2023-11-24T07:59:14.656594+0000","last_undegraded":"2023-11-24T07:59:14.656594+0000","last_fullsized":"2023-11-24T07:59:14.656594+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:59:12.615544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:59:12.615544+0000","last_clean_scrub_stamp":"2023-11-24T07:59:12.615544+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T13:35:56.042823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":211,"reported_epoch":73,"state":"active+clean","last_fresh":"2023-11-24T07:59:14.634733+0000","last_change":"2023-11-24T07:56:19.668905+0000","last_active":"2023-11-24T07:59:14.634733+0000","last_peered":"2023-11-24T07:59:14.634733+0000","last_clean":"2023-11-24T07:59:14.634733+0000","last_became_active":"2023-11-24T07:56:19.668601+0000","last_became_peered":"2023-11-24T07:56:19.668601+0000","last_unstale":"2023-11-24T07:59:14.634733+0000","last_undegraded":"2023-11-24T07:59:14.634733+0000","last_fullsized":"2023-11-24T07:59:14.634733+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:56:18.657844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:56:18.657844+0000","last_clean_scrub_stamp":"2023-11-24T07:56:18.657844+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T11:51:53.828455+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":11,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":10,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574887,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27912,"kb_used_data":892,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343928,"statfs":{"total":96636764160,"available":96608182272,"internally_reserved":0,"allocated":913408,"data_stored":385744,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 07:58:20 2023","interfaces":[{"interface":"back","average":{"1min":0.51400000000000001,"5min":0.54800000000000004,"15min":0.55400000000000005},"min":{"1min":0.309,"5min":0.309,"15min":0.309},"max":{"1min":0.78500000000000003,"5min":1.413,"15min":1.413},"last":0.27600000000000002},{"interface":"front","average":{"1min":0.56299999999999994,"5min":0.55500000000000005,"15min":0.55400000000000005},"min":{"1min":0.436,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":0.77000000000000002,"5min":0.81000000000000005,"15min":0.81000000000000005},"last":0.38300000000000001}]},{"osd":1,"last update":"Fri Nov 24 07:58:20 2023","interfaces":[{"interface":"back","average":{"1min":0.501,"5min":0.77900000000000003,"15min":0.82499999999999996},"min":{"1min":0.32000000000000001,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.69899999999999995,"5min":6.1070000000000002,"15min":6.1070000000000002},"last":0.438},{"interface":"front","average":{"1min":0.51300000000000001,"5min":0.56599999999999995,"15min":0.57499999999999996},"min":{"1min":0.40699999999999997,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.73599999999999999,"5min":1.5920000000000001,"15min":1.5920000000000001},"last":0.47999999999999998}]},{"osd":2,"last update":"Fri Nov 24 07:58:20 2023","interfaces":[{"interface":"back","average":{"1min":0.498,"5min":0.60099999999999998,"15min":0.61799999999999999},"min":{"1min":0.35099999999999998,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.71599999999999997,"5min":1.613,"15min":1.613},"last":0.41899999999999998},{"interface":"front","average":{"1min":0.502,"5min":0.73399999999999999,"15min":0.77200000000000002},"min":{"1min":0.34899999999999998,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.71299999999999997,"5min":4.5220000000000002,"15min":4.5220000000000002},"last":0.45100000000000001}]},{"osd":3,"last update":"Fri Nov 24 07:58:20 2023","interfaces":[{"interface":"back","average":{"1min":0.55100000000000005,"5min":0.755,"15min":0.78900000000000003},"min":{"1min":0.41699999999999998,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.754,"5min":4.6760000000000002,"15min":4.6760000000000002},"last":0.40699999999999997},{"interface":"front","average":{"1min":0.60699999999999998,"5min":0.875,"15min":0.92000000000000004},"min":{"1min":0.38500000000000001,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.81599999999999995,"5min":4.6539999999999999,"15min":4.6539999999999999},"last":0.53000000000000003}]},{"osd":4,"last update":"Fri Nov 24 07:58:20 2023","interfaces":[{"interface":"back","average":{"1min":0.54900000000000004,"5min":0.56399999999999995,"15min":0.56599999999999995},"min":{"1min":0.41799999999999998,"5min":0.378,"15min":0.378},"max":{"1min":0.70999999999999996,"5min":0.84699999999999998,"15min":0.84699999999999998},"last":0.36399999999999999},{"interface":"front","average":{"1min":0.54700000000000004,"5min":0.64300000000000002,"15min":0.65900000000000003},"min":{"1min":0.35199999999999998,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.79900000000000004,"5min":1.629,"15min":1.629},"last":0.57299999999999995}]},{"osd":5,"last update":"Fri Nov 24 07:58:20 2023","interfaces":[{"interface":"back","average":{"1min":0.54200000000000004,"5min":0.54600000000000004,"15min":0.54600000000000004},"min":{"1min":0.41599999999999998,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.67900000000000005,"5min":0.875,"15min":0.875},"last":0.315},{"interface":"front","average":{"1min":0.57799999999999996,"5min":0.61399999999999999,"15min":0.62},"min":{"1min":0.36799999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.77700000000000002,"5min":0.90900000000000003,"15min":0.90900000000000003},"last":0.499}]},{"osd":6,"last update":"Fri Nov 24 07:58:20 2023","interfaces":[{"interface":"back","average":{"1min":0.59999999999999998,"5min":0.60799999999999998,"15min":0.60999999999999999},"min":{"1min":0.49099999999999999,"5min":0.438,"15min":0.438},"max":{"1min":0.73399999999999999,"5min":0.80800000000000005,"15min":0.80800000000000005},"last":0.51400000000000001},{"interface":"front","average":{"1min":0.63900000000000001,"5min":0.66300000000000003,"15min":0.66700000000000004},"min":{"1min":0.45600000000000002,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":0.79700000000000004,"5min":0.98399999999999999,"15min":0.98399999999999999},"last":0.46800000000000003}]}]},{"osd":6,"up_from":13,"seq":55834574888,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27924,"kb_used_data":896,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343916,"statfs":{"total":96636764160,"available":96608169984,"internally_reserved":0,"allocated":917504,"data_stored":385778,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 07:58:19 2023","interfaces":[{"interface":"back","average":{"1min":0.53300000000000003,"5min":0.54600000000000004,"15min":0.54800000000000004},"min":{"1min":0.29799999999999999,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":0.93100000000000005,"5min":0.93100000000000005,"15min":0.93100000000000005},"last":0.60399999999999998},{"interface":"front","average":{"1min":0.51600000000000001,"5min":0.51600000000000001,"15min":0.51600000000000001},"min":{"1min":0.311,"5min":0.311,"15min":0.311},"max":{"1min":0.82199999999999995,"5min":0.82199999999999995,"15min":0.82199999999999995},"last":0.66400000000000003}]},{"osd":1,"last update":"Fri Nov 24 07:58:19 2023","interfaces":[{"interface":"back","average":{"1min":0.51900000000000002,"5min":0.51300000000000001,"15min":0.51200000000000001},"min":{"1min":0.35599999999999998,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.81299999999999994,"5min":0.81299999999999994,"15min":0.81299999999999994},"last":0.61599999999999999},{"interface":"front","average":{"1min":0.54000000000000004,"5min":0.59799999999999998,"15min":0.60799999999999998},"min":{"1min":0.35999999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.82799999999999996,"5min":1.0069999999999999,"15min":1.0069999999999999},"last":0.70799999999999996}]},{"osd":2,"last update":"Fri Nov 24 07:58:19 2023","interfaces":[{"interface":"back","average":{"1min":0.58399999999999996,"5min":0.57599999999999996,"15min":0.57499999999999996},"min":{"1min":0.34899999999999998,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.90600000000000003,"5min":0.90600000000000003,"15min":0.90600000000000003},"last":0.69599999999999995},{"interface":"front","average":{"1min":0.56499999999999995,"5min":0.58699999999999997,"15min":0.59099999999999997},"min":{"1min":0.34300000000000003,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.84899999999999998,"5min":0.84899999999999998,"15min":0.84899999999999998},"last":0.68300000000000005}]},{"osd":3,"last update":"Fri Nov 24 07:58:19 2023","interfaces":[{"interface":"back","average":{"1min":0.56000000000000005,"5min":0.56599999999999995,"15min":0.56699999999999995},"min":{"1min":0.38400000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.86499999999999999,"5min":0.876,"15min":0.876},"last":0.58899999999999997},{"interface":"front","average":{"1min":0.57699999999999996,"5min":0.56000000000000005,"15min":0.55700000000000005},"min":{"1min":0.39800000000000002,"5min":0.379,"15min":0.379},"max":{"1min":0.876,"5min":0.876,"15min":0.876},"last":0.72599999999999998}]},{"osd":4,"last update":"Fri Nov 24 07:58:19 2023","interfaces":[{"interface":"back","average":{"1min":0.53100000000000003,"5min":0.51900000000000002,"15min":0.51700000000000002},"min":{"1min":0.25,"5min":0.25,"15min":0.25},"max":{"1min":0.89400000000000002,"5min":0.89400000000000002,"15min":0.89400000000000002},"last":0.501},{"interface":"front","average":{"1min":0.58899999999999997,"5min":0.59899999999999998,"15min":0.60099999999999998},"min":{"1min":0.29099999999999998,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":1.044,"5min":1.087,"15min":1.087},"last":0.45900000000000002}]},{"osd":5,"last update":"Fri Nov 24 07:58:19 2023","interfaces":[{"interface":"back","average":{"1min":0.59499999999999997,"5min":0.54300000000000004,"15min":0.53500000000000003},"min":{"1min":0.27800000000000002,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":0.97999999999999998,"5min":0.97999999999999998,"15min":0.97999999999999998},"last":0.47899999999999998},{"interface":"front","average":{"1min":0.61399999999999999,"5min":0.61199999999999999,"15min":0.61199999999999999},"min":{"1min":0.31900000000000001,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":1.0009999999999999,"5min":1.0009999999999999,"15min":1.0009999999999999},"last":0.54700000000000004}]},{"osd":7,"last update":"Fri Nov 24 07:58:19 2023","interfaces":[{"interface":"back","average":{"1min":0.61799999999999999,"5min":0.61799999999999999,"15min":0.61799999999999999},"min":{"1min":0.26500000000000001,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":1.0880000000000001,"5min":1.0880000000000001,"15min":1.0880000000000001},"last":0.52500000000000002},{"interface":"front","average":{"1min":0.65200000000000002,"5min":0.66600000000000004,"15min":0.66800000000000004},"min":{"1min":0.34300000000000003,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.95599999999999996,"5min":0.95599999999999996,"15min":0.95599999999999996},"last":0.57099999999999995}]}]},{"osd":1,"up_from":13,"seq":55834574889,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28480,"kb_used_data":1460,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343360,"statfs":{"total":96636764160,"available":96607600640,"internally_reserved":0,"allocated":1495040,"data_stored":974500,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 07:58:24 2023","interfaces":[{"interface":"back","average":{"1min":0.33000000000000002,"5min":0.35699999999999998,"15min":0.36099999999999999},"min":{"1min":0.249,"5min":0.20200000000000001,"15min":0.20200000000000001},"max":{"1min":0.48799999999999999,"5min":0.70899999999999996,"15min":0.70899999999999996},"last":0.28699999999999998},{"interface":"front","average":{"1min":0.36399999999999999,"5min":0.39100000000000001,"15min":0.39500000000000002},"min":{"1min":0.23599999999999999,"5min":0.214,"15min":0.214},"max":{"1min":0.54700000000000004,"5min":0.82499999999999996,"15min":0.82499999999999996},"last":0.34999999999999998}]},{"osd":2,"last update":"Fri Nov 24 07:58:24 2023","interfaces":[{"interface":"back","average":{"1min":0.312,"5min":0.35999999999999999,"15min":0.36799999999999999},"min":{"1min":0.253,"5min":0.22700000000000001,"15min":0.22700000000000001},"max":{"1min":0.44800000000000001,"5min":0.60999999999999999,"15min":0.60999999999999999},"last":0.311},{"interface":"front","average":{"1min":0.36499999999999999,"5min":0.40300000000000002,"15min":0.40899999999999997},"min":{"1min":0.28100000000000003,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":0.47599999999999998,"5min":0.621,"15min":0.621},"last":0.33000000000000002}]},{"osd":3,"last update":"Fri Nov 24 07:58:24 2023","interfaces":[{"interface":"back","average":{"1min":0.40300000000000002,"5min":0.435,"15min":0.441},"min":{"1min":0.31900000000000001,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.59199999999999997,"5min":0.66500000000000004,"15min":0.66500000000000004},"last":0.373},{"interface":"front","average":{"1min":0.38700000000000001,"5min":0.41899999999999998,"15min":0.42499999999999999},"min":{"1min":0.29199999999999998,"5min":0.28499999999999998,"15min":0.28499999999999998},"max":{"1min":0.56200000000000006,"5min":0.76300000000000001,"15min":0.76300000000000001},"last":0.41199999999999998}]},{"osd":4,"last update":"Fri Nov 24 07:58:24 2023","interfaces":[{"interface":"back","average":{"1min":0.56200000000000006,"5min":0.60099999999999998,"15min":0.60699999999999998},"min":{"1min":0.442,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":0.84099999999999997,"5min":0.84099999999999997,"15min":0.84099999999999997},"last":0.52400000000000002},{"interface":"front","average":{"1min":0.61099999999999999,"5min":0.60699999999999998,"15min":0.60599999999999998},"min":{"1min":0.499,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":0.79500000000000004,"5min":0.82199999999999995,"15min":0.82199999999999995},"last":0.63800000000000001}]},{"osd":5,"last update":"Fri Nov 24 07:58:24 2023","interfaces":[{"interface":"back","average":{"1min":0.59199999999999997,"5min":0.69799999999999995,"15min":0.71599999999999997},"min":{"1min":0.439,"5min":0.37,"15min":0.37},"max":{"1min":0.97599999999999998,"5min":2.7269999999999999,"15min":2.7269999999999999},"last":0.67000000000000004},{"interface":"front","average":{"1min":0.57699999999999996,"5min":0.63100000000000001,"15min":0.64000000000000001},"min":{"1min":0.39100000000000001,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.94499999999999995,"5min":0.94499999999999995,"15min":0.94499999999999995},"last":0.56299999999999994}]},{"osd":6,"last update":"Fri Nov 24 07:58:24 2023","interfaces":[{"interface":"back","average":{"1min":0.61199999999999999,"5min":0.64800000000000002,"15min":0.65400000000000003},"min":{"1min":0.47699999999999998,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":0.77500000000000002,"5min":0.98199999999999998,"15min":0.98199999999999998},"last":0.51100000000000001},{"interface":"front","average":{"1min":0.57599999999999996,"5min":0.60799999999999998,"15min":0.61399999999999999},"min":{"1min":0.40600000000000003,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":0.89600000000000002,"5min":0.90700000000000003,"15min":0.90700000000000003},"last":0.61299999999999999}]},{"osd":7,"last update":"Fri Nov 24 07:58:24 2023","interfaces":[{"interface":"back","average":{"1min":0.63100000000000001,"5min":0.63500000000000001,"15min":0.63600000000000001},"min":{"1min":0.43099999999999999,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.79800000000000004,"5min":1.0649999999999999,"15min":1.0649999999999999},"last":0.53900000000000003},{"interface":"front","average":{"1min":0.57099999999999995,"5min":0.65000000000000002,"15min":0.66300000000000003},"min":{"1min":0.45700000000000002,"5min":0.442,"15min":0.442},"max":{"1min":0.71299999999999997,"5min":0.88800000000000001,"15min":0.88800000000000001},"last":0.65600000000000003}]}]},{"osd":0,"up_from":13,"seq":55834574889,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27904,"kb_used_data":884,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343936,"statfs":{"total":96636764160,"available":96608190464,"internally_reserved":0,"allocated":905216,"data_stored":384202,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Nov 24 07:58:20 2023","interfaces":[{"interface":"back","average":{"1min":0.38500000000000001,"5min":0.64700000000000002,"15min":0.69099999999999995},"min":{"1min":0.21299999999999999,"5min":0.21299999999999999,"15min":0.21299999999999999},"max":{"1min":0.625,"5min":6.3550000000000004,"15min":6.3550000000000004},"last":0.45500000000000002},{"interface":"front","average":{"1min":0.40799999999999997,"5min":1.464,"15min":1.6399999999999999},"min":{"1min":0.26000000000000001,"5min":0.26000000000000001,"15min":0.26000000000000001},"max":{"1min":0.60999999999999999,"5min":22.916,"15min":22.916},"last":0.44}]},{"osd":2,"last update":"Fri Nov 24 07:58:20 2023","interfaces":[{"interface":"back","average":{"1min":0.35299999999999998,"5min":0.63,"15min":0.67600000000000005},"min":{"1min":0.22900000000000001,"5min":0.22800000000000001,"15min":0.22800000000000001},"max":{"1min":0.499,"5min":6.3899999999999997,"15min":6.3899999999999997},"last":0.42199999999999999},{"interface":"front","average":{"1min":0.39500000000000002,"5min":0.92900000000000005,"15min":1.018},"min":{"1min":0.27900000000000003,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":0.60199999999999998,"5min":11.93,"15min":11.93},"last":0.36699999999999999}]},{"osd":3,"last update":"Fri Nov 24 07:58:20 2023","interfaces":[{"interface":"back","average":{"1min":0.373,"5min":0.40999999999999998,"15min":0.41599999999999998},"min":{"1min":0.23799999999999999,"5min":0.23799999999999999,"15min":0.23799999999999999},"max":{"1min":0.79300000000000004,"5min":1.155,"15min":1.155},"last":0.503},{"interface":"front","average":{"1min":0.438,"5min":0.98899999999999999,"15min":1.081},"min":{"1min":0.30199999999999999,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.69699999999999995,"5min":12.116,"15min":12.116},"last":0.38800000000000001}]},{"osd":4,"last update":"Fri Nov 24 07:58:20 2023","interfaces":[{"interface":"back","average":{"1min":0.626,"5min":0.64300000000000002,"15min":0.64600000000000002},"min":{"1min":0.40100000000000002,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":1.044,"5min":1.044,"15min":1.044},"last":0.67900000000000005},{"interface":"front","average":{"1min":0.64300000000000002,"5min":1.675,"15min":1.847},"min":{"1min":0.35699999999999998,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.93000000000000005,"5min":22.963999999999999,"15min":22.963999999999999},"last":0.54300000000000004}]},{"osd":5,"last update":"Fri Nov 24 07:58:20 2023","interfaces":[{"interface":"back","average":{"1min":0.65200000000000002,"5min":1.1639999999999999,"15min":1.25},"min":{"1min":0.41999999999999998,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.97799999999999998,"5min":12.084,"15min":12.084},"last":0.63300000000000001},{"interface":"front","average":{"1min":0.65800000000000003,"5min":0.67400000000000004,"15min":0.67600000000000005},"min":{"1min":0.39700000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.95499999999999996,"5min":1.073,"15min":1.073},"last":0.55400000000000005}]},{"osd":6,"last update":"Fri Nov 24 07:58:20 2023","interfaces":[{"interface":"back","average":{"1min":0.65200000000000002,"5min":0.68300000000000005,"15min":0.68799999999999994},"min":{"1min":0.45000000000000001,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":0.94699999999999995,"5min":1.0580000000000001,"15min":1.0580000000000001},"last":0.64500000000000002},{"interface":"front","average":{"1min":0.67000000000000004,"5min":1.1850000000000001,"15min":1.2709999999999999},"min":{"1min":0.46200000000000002,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":0.95899999999999996,"5min":12.045,"15min":12.045},"last":0.49299999999999999}]},{"osd":7,"last update":"Fri Nov 24 07:58:20 2023","interfaces":[{"interface":"back","average":{"1min":0.67000000000000004,"5min":0.65800000000000003,"15min":0.65600000000000003},"min":{"1min":0.45200000000000001,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":0.98299999999999998,"5min":1.038,"15min":1.038},"last":0.52300000000000002},{"interface":"front","average":{"1min":0.67500000000000004,"5min":1.7050000000000001,"15min":1.877},"min":{"1min":0.41299999999999998,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":1.0069999999999999,"5min":22.995000000000001,"15min":22.995000000000001},"last":0.59599999999999997}]}]},{"osd":2,"up_from":13,"seq":55834574889,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27904,"kb_used_data":884,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343936,"statfs":{"total":96636764160,"available":96608190464,"internally_reserved":0,"allocated":905216,"data_stored":384714,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 07:58:19 2023","interfaces":[{"interface":"back","average":{"1min":0.36399999999999999,"5min":0.36899999999999999,"15min":0.37},"min":{"1min":0.219,"5min":0.219,"15min":0.219},"max":{"1min":0.55900000000000005,"5min":0.58099999999999996,"15min":0.58099999999999996},"last":0.38300000000000001},{"interface":"front","average":{"1min":0.38900000000000001,"5min":0.39900000000000002,"15min":0.40100000000000002},"min":{"1min":0.29299999999999998,"5min":0.27400000000000002,"15min":0.27400000000000002},"max":{"1min":0.54100000000000004,"5min":0.54100000000000004,"15min":0.54100000000000004},"last":0.39300000000000002}]},{"osd":1,"last update":"Fri Nov 24 07:58:19 2023","interfaces":[{"interface":"back","average":{"1min":0.34699999999999998,"5min":0.371,"15min":0.375},"min":{"1min":0.22,"5min":0.22,"15min":0.22},"max":{"1min":0.54000000000000004,"5min":0.59199999999999997,"15min":0.59199999999999997},"last":0.41699999999999998},{"interface":"front","average":{"1min":0.36599999999999999,"5min":0.40600000000000003,"15min":0.41199999999999998},"min":{"1min":0.28299999999999997,"5min":0.25600000000000001,"15min":0.25600000000000001},"max":{"1min":0.497,"5min":0.627,"15min":0.627},"last":0.29199999999999998}]},{"osd":3,"last update":"Fri Nov 24 07:58:19 2023","interfaces":[{"interface":"back","average":{"1min":0.40500000000000003,"5min":0.40899999999999997,"15min":0.40999999999999998},"min":{"1min":0.30399999999999999,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":0.64200000000000002,"5min":0.64200000000000002,"15min":0.64200000000000002},"last":0.36699999999999999},{"interface":"front","average":{"1min":0.41699999999999998,"5min":0.41399999999999998,"15min":0.41399999999999998},"min":{"1min":0.28599999999999998,"5min":0.27500000000000002,"15min":0.27500000000000002},"max":{"1min":0.59299999999999997,"5min":0.64600000000000002,"15min":0.64600000000000002},"last":0.35199999999999998}]},{"osd":4,"last update":"Fri Nov 24 07:58:19 2023","interfaces":[{"interface":"back","average":{"1min":0.54800000000000004,"5min":0.59599999999999997,"15min":0.60399999999999998},"min":{"1min":0.38700000000000001,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.75,"5min":0.81599999999999995,"15min":0.81599999999999995},"last":0.55700000000000005},{"interface":"front","average":{"1min":0.55500000000000005,"5min":0.59099999999999997,"15min":0.59699999999999998},"min":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":0.83799999999999997,"5min":0.83799999999999997,"15min":0.83799999999999997},"last":0.68100000000000005}]},{"osd":5,"last update":"Fri Nov 24 07:58:19 2023","interfaces":[{"interface":"back","average":{"1min":0.56000000000000005,"5min":0.60799999999999998,"15min":0.61599999999999999},"min":{"1min":0.434,"5min":0.434,"15min":0.434},"max":{"1min":0.73499999999999999,"5min":0.84899999999999998,"15min":0.84899999999999998},"last":0.56899999999999995},{"interface":"front","average":{"1min":0.624,"5min":0.69599999999999995,"15min":0.70799999999999996},"min":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":0.82499999999999996,"5min":2.1699999999999999,"15min":2.1699999999999999},"last":0.61199999999999999}]},{"osd":6,"last update":"Fri Nov 24 07:58:19 2023","interfaces":[{"interface":"back","average":{"1min":0.60499999999999998,"5min":0.65400000000000003,"15min":0.66200000000000003},"min":{"1min":0.42499999999999999,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":0.878,"5min":0.878,"15min":0.878},"last":0.67000000000000004},{"interface":"front","average":{"1min":0.60599999999999998,"5min":0.65800000000000003,"15min":0.66600000000000004},"min":{"1min":0.41899999999999998,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":0.80300000000000005,"5min":0.88500000000000001,"15min":0.88500000000000001},"last":0.59199999999999997}]},{"osd":7,"last update":"Fri Nov 24 07:58:19 2023","interfaces":[{"interface":"back","average":{"1min":0.60399999999999998,"5min":0.65000000000000002,"15min":0.65800000000000003},"min":{"1min":0.441,"5min":0.441,"15min":0.441},"max":{"1min":0.77600000000000002,"5min":0.85999999999999999,"15min":0.85999999999999999},"last":0.501},{"interface":"front","average":{"1min":0.623,"5min":0.66800000000000004,"15min":0.67600000000000005},"min":{"1min":0.40000000000000002,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":0.90000000000000002,"5min":0.90000000000000002,"15min":0.90000000000000002},"last":0.64700000000000002}]}]},{"osd":3,"up_from":13,"seq":55834574889,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27912,"kb_used_data":892,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343928,"statfs":{"total":96636764160,"available":96608182272,"internally_reserved":0,"allocated":913408,"data_stored":384258,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 07:58:22 2023","interfaces":[{"interface":"back","average":{"1min":0.34200000000000003,"5min":0.35299999999999998,"15min":0.35499999999999998},"min":{"1min":0.20899999999999999,"5min":0.20899999999999999,"15min":0.20899999999999999},"max":{"1min":0.439,"5min":0.84799999999999998,"15min":0.84799999999999998},"last":0.53400000000000003},{"interface":"front","average":{"1min":0.32700000000000001,"5min":0.35899999999999999,"15min":0.36399999999999999},"min":{"1min":0.253,"5min":0.23699999999999999,"15min":0.23699999999999999},"max":{"1min":0.42499999999999999,"5min":0.873,"15min":0.873},"last":0.29499999999999998}]},{"osd":1,"last update":"Fri Nov 24 07:58:22 2023","interfaces":[{"interface":"back","average":{"1min":0.35899999999999999,"5min":0.371,"15min":0.373},"min":{"1min":0.29499999999999998,"5min":0.25700000000000001,"15min":0.25700000000000001},"max":{"1min":0.443,"5min":0.92000000000000004,"15min":0.92000000000000004},"last":0.36799999999999999},{"interface":"front","average":{"1min":0.35099999999999998,"5min":0.379,"15min":0.38300000000000001},"min":{"1min":0.28299999999999997,"5min":0.248,"15min":0.248},"max":{"1min":0.46700000000000003,"5min":0.81699999999999995,"15min":0.81699999999999995},"last":0.41099999999999998}]},{"osd":2,"last update":"Fri Nov 24 07:58:22 2023","interfaces":[{"interface":"back","average":{"1min":0.33400000000000002,"5min":0.35499999999999998,"15min":0.35899999999999999},"min":{"1min":0.255,"5min":0.24399999999999999,"15min":0.24399999999999999},"max":{"1min":0.43099999999999999,"5min":0.88500000000000001,"15min":0.88500000000000001},"last":0.42599999999999999},{"interface":"front","average":{"1min":0.39500000000000002,"5min":0.45300000000000001,"15min":0.46300000000000002},"min":{"1min":0.311,"5min":0.311,"15min":0.311},"max":{"1min":0.45800000000000002,"5min":0.98099999999999998,"15min":0.98099999999999998},"last":0.45000000000000001}]},{"osd":4,"last update":"Fri Nov 24 07:58:22 2023","interfaces":[{"interface":"back","average":{"1min":0.57099999999999995,"5min":0.751,"15min":0.78100000000000003},"min":{"1min":0.42099999999999999,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.71999999999999997,"5min":5.7350000000000003,"15min":5.7350000000000003},"last":0.63100000000000001},{"interface":"front","average":{"1min":0.59299999999999997,"5min":0.58599999999999997,"15min":0.58499999999999996},"min":{"1min":0.37,"5min":0.37,"15min":0.37},"max":{"1min":0.77000000000000002,"5min":1.27,"15min":1.27},"last":0.80400000000000005}]},{"osd":5,"last update":"Fri Nov 24 07:58:22 2023","interfaces":[{"interface":"back","average":{"1min":0.56899999999999995,"5min":0.58399999999999996,"15min":0.58599999999999997},"min":{"1min":0.40799999999999997,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":0.82799999999999996,"5min":1.4430000000000001,"15min":1.4430000000000001},"last":0.61099999999999999},{"interface":"front","average":{"1min":0.60699999999999998,"5min":0.623,"15min":0.625},"min":{"1min":0.443,"5min":0.377,"15min":0.377},"max":{"1min":0.79900000000000004,"5min":1.4239999999999999,"15min":1.4239999999999999},"last":0.78500000000000003}]},{"osd":6,"last update":"Fri Nov 24 07:58:22 2023","interfaces":[{"interface":"back","average":{"1min":0.58999999999999997,"5min":0.61399999999999999,"15min":0.61799999999999999},"min":{"1min":0.46200000000000002,"5min":0.434,"15min":0.434},"max":{"1min":0.82599999999999996,"5min":1.355,"15min":1.355},"last":0.84499999999999997},{"interface":"front","average":{"1min":0.61199999999999999,"5min":0.79900000000000004,"15min":0.82999999999999996},"min":{"1min":0.47499999999999998,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.76900000000000002,"5min":5.6479999999999997,"15min":5.6479999999999997},"last":0.73199999999999998}]},{"osd":7,"last update":"Fri Nov 24 07:58:22 2023","interfaces":[{"interface":"back","average":{"1min":0.59199999999999997,"5min":0.65100000000000002,"15min":0.66100000000000003},"min":{"1min":0.48099999999999998,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":0.79600000000000004,"5min":1.51,"15min":1.51},"last":0.82799999999999996},{"interface":"front","average":{"1min":0.61099999999999999,"5min":0.79500000000000004,"15min":0.82599999999999996},"min":{"1min":0.434,"5min":0.434,"15min":0.434},"max":{"1min":0.75900000000000001,"5min":5.7069999999999999,"15min":5.7069999999999999},"last":0.71199999999999997}]}]},{"osd":4,"up_from":13,"seq":55834574888,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27920,"kb_used_data":892,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343920,"statfs":{"total":96636764160,"available":96608174080,"internally_reserved":0,"allocated":913408,"data_stored":384826,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 07:58:25 2023","interfaces":[{"interface":"back","average":{"1min":0.504,"5min":0.56499999999999995,"15min":0.57499999999999996},"min":{"1min":0.34899999999999998,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.70799999999999996,"5min":0.79300000000000004,"15min":0.79300000000000004},"last":0.68000000000000005},{"interface":"front","average":{"1min":0.502,"5min":0.53400000000000003,"15min":0.54000000000000004},"min":{"1min":0.33600000000000002,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.70199999999999996,"5min":1.0249999999999999,"15min":1.0249999999999999},"last":0.44900000000000001}]},{"osd":1,"last update":"Fri Nov 24 07:58:25 2023","interfaces":[{"interface":"back","average":{"1min":0.52800000000000002,"5min":0.54700000000000004,"15min":0.55000000000000004},"min":{"1min":0.39500000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.73999999999999999,"5min":0.86399999999999999,"15min":0.86399999999999999},"last":0.63},{"interface":"front","average":{"1min":0.51900000000000002,"5min":0.54400000000000004,"15min":0.54800000000000004},"min":{"1min":0.374,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.72199999999999998,"5min":0.98699999999999999,"15min":0.98699999999999999},"last":0.40000000000000002}]},{"osd":2,"last update":"Fri Nov 24 07:58:25 2023","interfaces":[{"interface":"back","average":{"1min":0.52000000000000002,"5min":0.55200000000000005,"15min":0.55800000000000005},"min":{"1min":0.38800000000000001,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.66400000000000003,"5min":0.81100000000000005,"15min":0.81100000000000005},"last":0.60299999999999998},{"interface":"front","average":{"1min":0.56899999999999995,"5min":0.60399999999999998,"15min":0.60999999999999999},"min":{"1min":0.40000000000000002,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.75700000000000001,"5min":0.89200000000000002,"15min":0.89200000000000002},"last":0.64400000000000002}]},{"osd":3,"last update":"Fri Nov 24 07:58:25 2023","interfaces":[{"interface":"back","average":{"1min":0.54900000000000004,"5min":0.57699999999999996,"15min":0.58199999999999996},"min":{"1min":0.42899999999999999,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":0.65800000000000003,"5min":0.89700000000000002,"15min":0.89700000000000002},"last":0.49299999999999999},{"interface":"front","average":{"1min":0.57499999999999996,"5min":0.61899999999999999,"15min":0.627},"min":{"1min":0.442,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.72899999999999998,"5min":0.96199999999999997,"15min":0.96199999999999997},"last":0.58299999999999996}]},{"osd":5,"last update":"Fri Nov 24 07:58:25 2023","interfaces":[{"interface":"back","average":{"1min":0.48599999999999999,"5min":0.46100000000000002,"15min":0.45700000000000002},"min":{"1min":0.29499999999999998,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":0.67600000000000005,"5min":0.70499999999999996,"15min":0.70499999999999996},"last":0.35899999999999999},{"interface":"front","average":{"1min":0.59899999999999998,"5min":0.61499999999999999,"15min":0.61699999999999999},"min":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.746,"5min":0.86399999999999999,"15min":0.86399999999999999},"last":0.73699999999999999}]},{"osd":6,"last update":"Fri Nov 24 07:58:25 2023","interfaces":[{"interface":"back","average":{"1min":0.56799999999999995,"5min":0.58199999999999996,"15min":0.58399999999999996},"min":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.69799999999999995,"5min":0.76800000000000002,"15min":0.76800000000000002},"last":0.33000000000000002},{"interface":"front","average":{"1min":0.58999999999999997,"5min":0.60899999999999999,"15min":0.61199999999999999},"min":{"1min":0.46899999999999997,"5min":0.436,"15min":0.436},"max":{"1min":0.77700000000000002,"5min":1.095,"15min":1.095},"last":0.56000000000000005}]},{"osd":7,"last update":"Fri Nov 24 07:58:25 2023","interfaces":[{"interface":"back","average":{"1min":0.56799999999999995,"5min":0.57199999999999995,"15min":0.57199999999999995},"min":{"1min":0.39300000000000002,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.74199999999999999,"5min":0.89300000000000002,"15min":0.89300000000000002},"last":0.66000000000000003},{"interface":"front","average":{"1min":0.61699999999999999,"5min":0.65300000000000002,"15min":0.65900000000000003},"min":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.79100000000000004,"5min":0.91500000000000004,"15min":0.91500000000000004},"last":0.53600000000000003}]}]},{"osd":5,"up_from":13,"seq":55834574889,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28492,"kb_used_data":1464,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343348,"statfs":{"total":96636764160,"available":96607588352,"internally_reserved":0,"allocated":1499136,"data_stored":974590,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 07:58:20 2023","interfaces":[{"interface":"back","average":{"1min":0.52300000000000002,"5min":0.55400000000000005,"15min":0.55900000000000005},"min":{"1min":0.35299999999999998,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.65800000000000003,"5min":1.0109999999999999,"15min":1.0109999999999999},"last":0.58699999999999997},{"interface":"front","average":{"1min":0.53200000000000003,"5min":0.746,"15min":0.78200000000000003},"min":{"1min":0.34399999999999997,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.72099999999999997,"5min":5.431,"15min":5.431},"last":0.51300000000000001}]},{"osd":1,"last update":"Fri Nov 24 07:58:20 2023","interfaces":[{"interface":"back","average":{"1min":0.51100000000000001,"5min":0.54900000000000004,"15min":0.55500000000000005},"min":{"1min":0.36399999999999999,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":0.67100000000000004,"5min":1.016,"15min":1.016},"last":0.54100000000000004},{"interface":"front","average":{"1min":0.54600000000000004,"5min":0.56499999999999995,"15min":0.56799999999999995},"min":{"1min":0.28599999999999998,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":0.78300000000000003,"5min":0.96499999999999997,"15min":0.96499999999999997},"last":0.77500000000000002}]},{"osd":2,"last update":"Fri Nov 24 07:58:20 2023","interfaces":[{"interface":"back","average":{"1min":0.54500000000000004,"5min":0.54900000000000004,"15min":0.55000000000000004},"min":{"1min":0.33700000000000002,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.67100000000000004,"5min":1.163,"15min":1.163},"last":0.61399999999999999},{"interface":"front","average":{"1min":0.54500000000000004,"5min":0.61299999999999999,"15min":0.624},"min":{"1min":0.41099999999999998,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.78900000000000003,"5min":1.119,"15min":1.119},"last":0.63200000000000001}]},{"osd":3,"last update":"Fri Nov 24 07:58:20 2023","interfaces":[{"interface":"back","average":{"1min":0.57199999999999995,"5min":0.83799999999999997,"15min":0.88200000000000001},"min":{"1min":0.36899999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.85899999999999999,"5min":5.609,"15min":5.609},"last":0.71399999999999997},{"interface":"front","average":{"1min":0.629,"5min":0.66600000000000004,"15min":0.67200000000000004},"min":{"1min":0.44600000000000001,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":0.76400000000000001,"5min":1.288,"15min":1.288},"last":0.67700000000000005}]},{"osd":4,"last update":"Fri Nov 24 07:58:20 2023","interfaces":[{"interface":"back","average":{"1min":0.51500000000000001,"5min":0.51100000000000001,"15min":0.51000000000000001},"min":{"1min":0.32100000000000001,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.76400000000000001,"5min":0.76400000000000001,"15min":0.76400000000000001},"last":0.68799999999999994},{"interface":"front","average":{"1min":0.58799999999999997,"5min":0.57499999999999996,"15min":0.57299999999999995},"min":{"1min":0.29599999999999999,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":0.92900000000000005,"5min":0.92900000000000005,"15min":0.92900000000000005},"last":0.66200000000000003}]},{"osd":6,"last update":"Fri Nov 24 07:58:20 2023","interfaces":[{"interface":"back","average":{"1min":0.59599999999999997,"5min":0.57599999999999996,"15min":0.57199999999999995},"min":{"1min":0.35199999999999998,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.81799999999999995,"5min":0.89700000000000002,"15min":0.89700000000000002},"last":0.64900000000000002},{"interface":"front","average":{"1min":0.63800000000000001,"5min":0.627,"15min":0.625},"min":{"1min":0.40300000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":1.083,"5min":1.083,"15min":1.083},"last":0.72899999999999998}]},{"osd":7,"last update":"Fri Nov 24 07:58:20 2023","interfaces":[{"interface":"back","average":{"1min":0.66200000000000003,"5min":0.64900000000000002,"15min":0.64700000000000002},"min":{"1min":0.36799999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.90100000000000002,"5min":0.96599999999999997,"15min":0.96599999999999997},"last":0.75},{"interface":"front","average":{"1min":0.66800000000000004,"5min":0.65600000000000003,"15min":0.65400000000000003},"min":{"1min":0.42699999999999999,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":0.85599999999999998,"5min":0.94199999999999995,"15min":0.94199999999999995},"last":0.70199999999999996}]}]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":694,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1646,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-11-24T07:59:23.419 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-11-24T07:59:23.559 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:23.557+0000 7fec84cbd700 1 Processor -- start 2023-11-24T07:59:23.559 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:23.557+0000 7fec84cbd700 1 -- start start 2023-11-24T07:59:23.568 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:23.558+0000 7fec84cbd700 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fec80071bb0 0x7fec8006afd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:23.568 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:23.558+0000 7fec84cbd700 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fec80147bb0 con 0x7fec800d0080 2023-11-24T07:59:23.568 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:23.558+0000 7fec84cbd700 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fec800d0460 con 0x7fec8006b510 2023-11-24T07:59:23.568 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:23.558+0000 7fec84cbd700 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fec80072cd0 con 0x7fec80071bb0 2023-11-24T07:59:23.569 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:23.558+0000 7fec7e59c700 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7fec800d0080 0x7fec800b5890 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:58410/0 (socket says 172.21.15.59:58410) 2023-11-24T07:59:23.569 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:23.558+0000 7fec7e59c700 1 -- 172.21.15.59:0/1710585635 learned_addr learned my addr 172.21.15.59:0/1710585635 (peer_addr_for_me v1:172.21.15.59:0/0) 2023-11-24T07:59:23.569 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:23.558+0000 7fec7dd9b700 1 --2- 172.21.15.59:0/1710585635 >> v2:172.21.15.76:3300/0 conn(0x7fec80071bb0 0x7fec8006afd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:23.569 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:23.558+0000 7fec7d59a700 1 -- 172.21.15.59:0/1710585635 <== mon.0 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3454011842 0 0) 0x7fec80147bb0 con 0x7fec800d0080 2023-11-24T07:59:23.569 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:23.558+0000 7fec7d59a700 1 -- 172.21.15.59:0/1710585635 --> v1:172.21.15.59:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fec680035b0 con 0x7fec800d0080 2023-11-24T07:59:23.569 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:23.558+0000 7fec7dd9b700 1 -- 172.21.15.59:0/1710585635 >> v1:172.21.15.76:6789/0 conn(0x7fec8006b510 legacy=0x7fec8006dfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:23.569 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:23.558+0000 7fec7dd9b700 1 -- 172.21.15.59:0/1710585635 >> v1:172.21.15.59:6789/0 conn(0x7fec800d0080 legacy=0x7fec800b5890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:23.569 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:23.558+0000 7fec7dd9b700 1 -- 172.21.15.59:0/1710585635 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fec80072150 con 0x7fec80071bb0 2023-11-24T07:59:23.570 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:23.559+0000 7fec7dd9b700 1 --2- 172.21.15.59:0/1710585635 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fec80071bb0 0x7fec8006afd0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7fec74009840 tx=0x7fec74009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=faf1714a16c7b294 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:59:23.570 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:23.559+0000 7fec7d59a700 1 -- 172.21.15.59:0/1710585635 <== mon.1 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec74011830 con 0x7fec80071bb0 2023-11-24T07:59:23.570 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:23.559+0000 7fec7d59a700 1 -- 172.21.15.59:0/1710585635 <== mon.1 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fec74011990 con 0x7fec80071bb0 2023-11-24T07:59:23.570 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:23.559+0000 7fec7d59a700 1 -- 172.21.15.59:0/1710585635 <== mon.1 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec74011d20 con 0x7fec80071bb0 2023-11-24T07:59:23.570 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:23.559+0000 7fec84cbd700 1 -- 172.21.15.59:0/1710585635 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fec80071bb0 msgr2=0x7fec8006afd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:23.570 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:23.559+0000 7fec84cbd700 1 --2- 172.21.15.59:0/1710585635 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fec80071bb0 0x7fec8006afd0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7fec74009840 tx=0x7fec74009b50 comp rx=0 tx=0).stop 2023-11-24T07:59:23.570 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:23.559+0000 7fec84cbd700 1 -- 172.21.15.59:0/1710585635 shutdown_connections 2023-11-24T07:59:23.570 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:23.559+0000 7fec84cbd700 1 --2- 172.21.15.59:0/1710585635 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fec80071bb0 0x7fec8006afd0 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:23.571 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:23.559+0000 7fec84cbd700 1 -- 172.21.15.59:0/1710585635 >> 172.21.15.59:0/1710585635 conn(0x7fec800b27d0 msgr2=0x7fec800b2be0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:59:23.571 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:23.559+0000 7fec84cbd700 1 -- 172.21.15.59:0/1710585635 shutdown_connections 2023-11-24T07:59:23.571 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:23.559+0000 7fec84cbd700 1 -- 172.21.15.59:0/1710585635 wait complete. 2023-11-24T07:59:23.571 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:23.561+0000 7fec84cbd700 1 Processor -- start 2023-11-24T07:59:23.571 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:23.561+0000 7fec84cbd700 1 -- start start 2023-11-24T07:59:23.571 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:23.561+0000 7fec84cbd700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fec8006b510 0x7fec800c7880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:23.571 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:23.561+0000 7fec84cbd700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fec80071bb0 0x7fec8015d840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:23.572 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:23.561+0000 7fec84cbd700 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fec800d0080 0x7fec800ae960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:23.572 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:23.561+0000 7fec84cbd700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec8015dd80 con 0x7fec8006b510 2023-11-24T07:59:23.572 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:23.561+0000 7fec84cbd700 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec800c67e0 con 0x7fec80071bb0 2023-11-24T07:59:23.572 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:23.561+0000 7fec84cbd700 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fec800c6ac0 con 0x7fec800d0080 2023-11-24T07:59:23.572 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:23.562+0000 7fec7e59c700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fec8006b510 0x7fec800c7880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:23.572 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:23.562+0000 7fec7e59c700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fec8006b510 0x7fec800c7880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:54942/0 (socket says 172.21.15.59:54942) 2023-11-24T07:59:23.573 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:23.562+0000 7fec7e59c700 1 -- 172.21.15.59:0/1164560355 learned_addr learned my addr 172.21.15.59:0/1164560355 (peer_addr_for_me v2:172.21.15.59:0/0) 2023-11-24T07:59:23.573 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:23.562+0000 7fec7dd9b700 1 --2- 172.21.15.59:0/1164560355 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fec80071bb0 0x7fec8015d840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:23.573 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:23.562+0000 7fec7ed9d700 1 --2- 172.21.15.59:0/1164560355 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fec800d0080 0x7fec800ae960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:23.573 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:23.562+0000 7fec7e59c700 1 -- 172.21.15.59:0/1164560355 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fec800d0080 msgr2=0x7fec800ae960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:23.573 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:23.562+0000 7fec7e59c700 1 --2- 172.21.15.59:0/1164560355 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fec800d0080 0x7fec800ae960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:23.573 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:23.562+0000 7fec7e59c700 1 -- 172.21.15.59:0/1164560355 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fec80071bb0 msgr2=0x7fec8015d840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:23.574 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:23.562+0000 7fec7e59c700 1 --2- 172.21.15.59:0/1164560355 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fec80071bb0 0x7fec8015d840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:23.574 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:23.562+0000 7fec7e59c700 1 -- 172.21.15.59:0/1164560355 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fec740094a0 con 0x7fec8006b510 2023-11-24T07:59:23.574 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:23.562+0000 7fec7ed9d700 1 --2- 172.21.15.59:0/1164560355 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fec800d0080 0x7fec800ae960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-24T07:59:23.574 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:23.562+0000 7fec7e59c700 1 --2- 172.21.15.59:0/1164560355 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fec8006b510 0x7fec800c7880 secure :-1 s=READY pgs=879 cs=0 l=1 rev1=1 crypto rx=0x7fec70001eb0 tx=0x7fec7000c7b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:59:23.574 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:23.562+0000 7fec6effd700 1 -- 172.21.15.59:0/1164560355 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec70005d70 con 0x7fec8006b510 2023-11-24T07:59:23.575 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:23.562+0000 7fec6effd700 1 -- 172.21.15.59:0/1164560355 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fec700070d0 con 0x7fec8006b510 2023-11-24T07:59:23.575 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:23.562+0000 7fec84cbd700 1 -- 172.21.15.59:0/1164560355 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fec800c6da0 con 0x7fec8006b510 2023-11-24T07:59:23.575 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:23.562+0000 7fec6effd700 1 -- 172.21.15.59:0/1164560355 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec70007460 con 0x7fec8006b510 2023-11-24T07:59:23.575 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:23.562+0000 7fec84cbd700 1 -- 172.21.15.59:0/1164560355 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fec800c7360 con 0x7fec8006b510 2023-11-24T07:59:23.575 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:23.563+0000 7fec84cbd700 1 -- 172.21.15.59:0/1164560355 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fec800c7550 con 0x7fec8006b510 2023-11-24T07:59:23.575 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:23.565+0000 7fec6effd700 1 -- 172.21.15.59:0/1164560355 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 10) v1 ==== 89588+0+0 (secure 0 0 0) 0x7fec700075c0 con 0x7fec8006b510 2023-11-24T07:59:23.576 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:23.565+0000 7fec6effd700 1 --2- 172.21.15.59:0/1164560355 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7fec6006b2c0 0x7fec6006d780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:23.576 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:23.565+0000 7fec6effd700 1 -- 172.21.15.59:0/1164560355 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fec700895b0 con 0x7fec8006b510 2023-11-24T07:59:23.576 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:23.566+0000 7fec7dd9b700 1 --2- 172.21.15.59:0/1164560355 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7fec6006b2c0 0x7fec6006d780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:23.576 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:23.566+0000 7fec7dd9b700 1 --2- 172.21.15.59:0/1164560355 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7fec6006b2c0 0x7fec6006d780 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7fec740090b0 tx=0x7fec74008030 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:59:23.577 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:23.569+0000 7fec6effd700 1 -- 172.21.15.59:0/1164560355 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fec70054150 con 0x7fec8006b510 2023-11-24T07:59:23.771 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:23.769+0000 7fec84cbd700 1 -- 172.21.15.59:0/1164560355 --> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fec800729f0 con 0x7fec6006b2c0 2023-11-24T07:59:23.779 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:23.778+0000 7fec6effd700 1 -- 172.21.15.59:0/1164560355 <== mgr.4100 v2:172.21.15.76:6832/4264046536 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+289397 (secure 0 0 0) 0x7fec74011c00 con 0x7fec6006b2c0 2023-11-24T07:59:23.783 INFO:teuthology.orchestra.run.smithi059.stdout: 2023-11-24T07:59:23.786 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:23.781+0000 7fec84cbd700 1 -- 172.21.15.59:0/1164560355 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7fec6006b2c0 msgr2=0x7fec6006d780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:23.786 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:23.781+0000 7fec84cbd700 1 --2- 172.21.15.59:0/1164560355 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7fec6006b2c0 0x7fec6006d780 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7fec740090b0 tx=0x7fec74008030 comp rx=0 tx=0).stop 2023-11-24T07:59:23.786 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:23.781+0000 7fec84cbd700 1 -- 172.21.15.59:0/1164560355 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fec8006b510 msgr2=0x7fec800c7880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:23.786 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:23.781+0000 7fec84cbd700 1 --2- 172.21.15.59:0/1164560355 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fec8006b510 0x7fec800c7880 secure :-1 s=READY pgs=879 cs=0 l=1 rev1=1 crypto rx=0x7fec70001eb0 tx=0x7fec7000c7b0 comp rx=0 tx=0).stop 2023-11-24T07:59:23.786 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:23.782+0000 7fec84cbd700 1 -- 172.21.15.59:0/1164560355 shutdown_connections 2023-11-24T07:59:23.786 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:23.782+0000 7fec84cbd700 1 --2- 172.21.15.59:0/1164560355 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7fec6006b2c0 0x7fec6006d780 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:23.786 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:23.782+0000 7fec84cbd700 1 --2- 172.21.15.59:0/1164560355 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fec800d0080 0x7fec800ae960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:23.786 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:23.782+0000 7fec84cbd700 1 --2- 172.21.15.59:0/1164560355 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fec8006b510 0x7fec800c7880 unknown :-1 s=CLOSED pgs=879 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:23.786 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:23.782+0000 7fec84cbd700 1 --2- 172.21.15.59:0/1164560355 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fec80071bb0 0x7fec8015d840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:23.787 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:23.782+0000 7fec84cbd700 1 -- 172.21.15.59:0/1164560355 >> 172.21.15.59:0/1164560355 conn(0x7fec800b27d0 msgr2=0x7fec800b2bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:59:23.787 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:23.782+0000 7fec84cbd700 1 -- 172.21.15.59:0/1164560355 shutdown_connections 2023-11-24T07:59:23.787 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:23.782+0000 7fec84cbd700 1 -- 172.21.15.59:0/1164560355 wait complete. 2023-11-24T07:59:23.787 INFO:teuthology.orchestra.run.smithi059.stderr:dumped all 2023-11-24T07:59:23.795 INFO:teuthology.orchestra.run.smithi059.stdout:{"pg_ready":true,"pg_map":{"version":159,"stamp":"2023-11-24T07:59:22.340550+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":182,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":224448,"kb_used_data":8264,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754750272,"statfs":{"total":773094113280,"available":772864278528,"internally_reserved":0,"allocated":8462336,"data_stored":4258612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12745,"internal_metadata":221236791},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":35,"apply_latency_ms":35,"commit_latency_ns":35000000,"apply_latency_ns":35000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.002004"},"pg_stats":[{"pgid":"11.3f","version":"0'0","reported_seq":11,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-11-24T07:59:16.292941+0000","last_change":"2023-11-24T07:59:14.661661+0000","last_active":"2023-11-24T07:59:16.292941+0000","last_peered":"2023-11-24T07:59:16.292941+0000","last_clean":"2023-11-24T07:59:16.292941+0000","last_became_active":"2023-11-24T07:59:14.661350+0000","last_became_peered":"2023-11-24T07:59:14.661350+0000","last_unstale":"2023-11-24T07:59:16.292941+0000","last_undegraded":"2023-11-24T07:59:16.292941+0000","last_fullsized":"2023-11-24T07:59:16.292941+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:59:13.620687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:59:13.620687+0000","last_clean_scrub_stamp":"2023-11-24T07:59:13.620687+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T09:34:52.878616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.3e","version":"0'0","reported_seq":11,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-11-24T07:59:16.293734+0000","last_change":"2023-11-24T07:59:14.662364+0000","last_active":"2023-11-24T07:59:16.293734+0000","last_peered":"2023-11-24T07:59:16.293734+0000","last_clean":"2023-11-24T07:59:16.293734+0000","last_became_active":"2023-11-24T07:59:14.661727+0000","last_became_peered":"2023-11-24T07:59:14.661727+0000","last_unstale":"2023-11-24T07:59:16.293734+0000","last_undegraded":"2023-11-24T07:59:16.293734+0000","last_fullsized":"2023-11-24T07:59:16.293734+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:59:13.620687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:59:13.620687+0000","last_clean_scrub_stamp":"2023-11-24T07:59:13.620687+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T13:45:41.343630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.3d","version":"0'0","reported_seq":11,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-11-24T07:59:16.292014+0000","last_change":"2023-11-24T07:59:14.649579+0000","last_active":"2023-11-24T07:59:16.292014+0000","last_peered":"2023-11-24T07:59:16.292014+0000","last_clean":"2023-11-24T07:59:16.292014+0000","last_became_active":"2023-11-24T07:59:14.644685+0000","last_became_peered":"2023-11-24T07:59:14.644685+0000","last_unstale":"2023-11-24T07:59:16.292014+0000","last_undegraded":"2023-11-24T07:59:16.292014+0000","last_fullsized":"2023-11-24T07:59:16.292014+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:59:13.620687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:59:13.620687+0000","last_clean_scrub_stamp":"2023-11-24T07:59:13.620687+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:02:40.788118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.3c","version":"0'0","reported_seq":11,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-11-24T07:59:16.292011+0000","last_change":"2023-11-24T07:59:14.649105+0000","last_active":"2023-11-24T07:59:16.292011+0000","last_peered":"2023-11-24T07:59:16.292011+0000","last_clean":"2023-11-24T07:59:16.292011+0000","last_became_active":"2023-11-24T07:59:14.648733+0000","last_became_peered":"2023-11-24T07:59:14.648733+0000","last_unstale":"2023-11-24T07:59:16.292011+0000","last_undegraded":"2023-11-24T07:59:16.292011+0000","last_fullsized":"2023-11-24T07:59:16.292011+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:59:13.620687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:59:13.620687+0000","last_clean_scrub_stamp":"2023-11-24T07:59:13.620687+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T08:20:48.582684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.3b","version":"0'0","reported_seq":11,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-11-24T07:59:15.650794+0000","last_change":"2023-11-24T07:59:14.668923+0000","last_active":"2023-11-24T07:59:15.650794+0000","last_peered":"2023-11-24T07:59:15.650794+0000","last_clean":"2023-11-24T07:59:15.650794+0000","last_became_active":"2023-11-24T07:59:14.668627+0000","last_became_peered":"2023-11-24T07:59:14.668627+0000","last_unstale":"2023-11-24T07:59:15.650794+0000","last_undegraded":"2023-11-24T07:59:15.650794+0000","last_fullsized":"2023-11-24T07:59:15.650794+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:59:13.620687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:59:13.620687+0000","last_clean_scrub_stamp":"2023-11-24T07:59:13.620687+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T16:38:05.743270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.3a","version":"0'0","reported_seq":11,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-11-24T07:59:16.104059+0000","last_change":"2023-11-24T07:59:14.658517+0000","last_active":"2023-11-24T07:59:16.104059+0000","last_peered":"2023-11-24T07:59:16.104059+0000","last_clean":"2023-11-24T07:59:16.104059+0000","last_became_active":"2023-11-24T07:59:14.658227+0000","last_became_peered":"2023-11-24T07:59:14.658227+0000","last_unstale":"2023-11-24T07:59:16.104059+0000","last_undegraded":"2023-11-24T07:59:16.104059+0000","last_fullsized":"2023-11-24T07:59:16.104059+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:59:13.620687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:59:13.620687+0000","last_clean_scrub_stamp":"2023-11-24T07:59:13.620687+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T11:23:41.485071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.39","version":"0'0","reported_seq":11,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-11-24T07:59:16.292282+0000","last_change":"2023-11-24T07:59:14.650302+0000","last_active":"2023-11-24T07:59:16.292282+0000","last_peered":"2023-11-24T07:59:16.292282+0000","last_clean":"2023-11-24T07:59:16.292282+0000","last_became_active":"2023-11-24T07:59:14.648820+0000","last_became_peered":"2023-11-24T07:59:14.648820+0000","last_unstale":"2023-11-24T07:59:16.292282+0000","last_undegraded":"2023-11-24T07:59:16.292282+0000","last_fullsized":"2023-11-24T07:59:16.292282+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:59:13.620687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:59:13.620687+0000","last_clean_scrub_stamp":"2023-11-24T07:59:13.620687+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:54:57.076700+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.38","version":"0'0","reported_seq":11,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-11-24T07:59:16.291981+0000","last_change":"2023-11-24T07:59:14.660939+0000","last_active":"2023-11-24T07:59:16.291981+0000","last_peered":"2023-11-24T07:59:16.291981+0000","last_clean":"2023-11-24T07:59:16.291981+0000","last_became_active":"2023-11-24T07:59:14.660638+0000","last_became_peered":"2023-11-24T07:59:14.660638+0000","last_unstale":"2023-11-24T07:59:16.291981+0000","last_undegraded":"2023-11-24T07:59:16.291981+0000","last_fullsized":"2023-11-24T07:59:16.291981+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:59:13.620687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:59:13.620687+0000","last_clean_scrub_stamp":"2023-11-24T07:59:13.620687+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T18:41:19.164124+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.37","version":"0'0","reported_seq":11,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-11-24T07:59:16.292018+0000","last_change":"2023-11-24T07:59:14.651362+0000","last_active":"2023-11-24T07:59:16.292018+0000","last_peered":"2023-11-24T07:59:16.292018+0000","last_clean":"2023-11-24T07:59:16.292018+0000","last_became_active":"2023-11-24T07:59:14.650874+0000","last_became_peered":"2023-11-24T07:59:14.650874+0000","last_unstale":"2023-11-24T07:59:16.292018+0000","last_undegraded":"2023-11-24T07:59:16.292018+0000","last_fullsized":"2023-11-24T07:59:16.292018+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:59:13.620687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:59:13.620687+0000","last_clean_scrub_stamp":"2023-11-24T07:59:13.620687+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T19:05:59.935796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.36","version":"0'0","reported_seq":11,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-11-24T07:59:16.293188+0000","last_change":"2023-11-24T07:59:14.649665+0000","last_active":"2023-11-24T07:59:16.293188+0000","last_peered":"2023-11-24T07:59:16.293188+0000","last_clean":"2023-11-24T07:59:16.293188+0000","last_became_active":"2023-11-24T07:59:14.649305+0000","last_became_peered":"2023-11-24T07:59:14.649305+0000","last_unstale":"2023-11-24T07:59:16.293188+0000","last_undegraded":"2023-11-24T07:59:16.293188+0000","last_fullsized":"2023-11-24T07:59:16.293188+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:59:13.620687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:59:13.620687+0000","last_clean_scrub_stamp":"2023-11-24T07:59:13.620687+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T17:15:10.353695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.35","version":"0'0","reported_seq":11,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-11-24T07:59:16.292721+0000","last_change":"2023-11-24T07:59:14.663082+0000","last_active":"2023-11-24T07:59:16.292721+0000","last_peered":"2023-11-24T07:59:16.292721+0000","last_clean":"2023-11-24T07:59:16.292721+0000","last_became_active":"2023-11-24T07:59:14.662601+0000","last_became_peered":"2023-11-24T07:59:14.662601+0000","last_unstale":"2023-11-24T07:59:16.292721+0000","last_undegraded":"2023-11-24T07:59:16.292721+0000","last_fullsized":"2023-11-24T07:59:16.292721+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:59:13.620687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:59:13.620687+0000","last_clean_scrub_stamp":"2023-11-24T07:59:13.620687+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T15:51:56.807935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.34","version":"0'0","reported_seq":11,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-11-24T07:59:15.649831+0000","last_change":"2023-11-24T07:59:14.646577+0000","last_active":"2023-11-24T07:59:15.649831+0000","last_peered":"2023-11-24T07:59:15.649831+0000","last_clean":"2023-11-24T07:59:15.649831+0000","last_became_active":"2023-11-24T07:59:14.646318+0000","last_became_peered":"2023-11-24T07:59:14.646318+0000","last_unstale":"2023-11-24T07:59:15.649831+0000","last_undegraded":"2023-11-24T07:59:15.649831+0000","last_fullsized":"2023-11-24T07:59:15.649831+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:59:13.620687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:59:13.620687+0000","last_clean_scrub_stamp":"2023-11-24T07:59:13.620687+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T15:47:40.090700+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.33","version":"0'0","reported_seq":11,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-11-24T07:59:16.292052+0000","last_change":"2023-11-24T07:59:14.652220+0000","last_active":"2023-11-24T07:59:16.292052+0000","last_peered":"2023-11-24T07:59:16.292052+0000","last_clean":"2023-11-24T07:59:16.292052+0000","last_became_active":"2023-11-24T07:59:14.644814+0000","last_became_peered":"2023-11-24T07:59:14.644814+0000","last_unstale":"2023-11-24T07:59:16.292052+0000","last_undegraded":"2023-11-24T07:59:16.292052+0000","last_fullsized":"2023-11-24T07:59:16.292052+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:59:13.620687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:59:13.620687+0000","last_clean_scrub_stamp":"2023-11-24T07:59:13.620687+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T19:09:47.226456+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.32","version":"0'0","reported_seq":11,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-11-24T07:59:15.649710+0000","last_change":"2023-11-24T07:59:14.654802+0000","last_active":"2023-11-24T07:59:15.649710+0000","last_peered":"2023-11-24T07:59:15.649710+0000","last_clean":"2023-11-24T07:59:15.649710+0000","last_became_active":"2023-11-24T07:59:14.654543+0000","last_became_peered":"2023-11-24T07:59:14.654543+0000","last_unstale":"2023-11-24T07:59:15.649710+0000","last_undegraded":"2023-11-24T07:59:15.649710+0000","last_fullsized":"2023-11-24T07:59:15.649710+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:59:13.620687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:59:13.620687+0000","last_clean_scrub_stamp":"2023-11-24T07:59:13.620687+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T14:38:55.028542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.31","version":"0'0","reported_seq":11,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-11-24T07:59:15.649474+0000","last_change":"2023-11-24T07:59:14.648768+0000","last_active":"2023-11-24T07:59:15.649474+0000","last_peered":"2023-11-24T07:59:15.649474+0000","last_clean":"2023-11-24T07:59:15.649474+0000","last_became_active":"2023-11-24T07:59:14.648489+0000","last_became_peered":"2023-11-24T07:59:14.648489+0000","last_unstale":"2023-11-24T07:59:15.649474+0000","last_undegraded":"2023-11-24T07:59:15.649474+0000","last_fullsized":"2023-11-24T07:59:15.649474+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:59:13.620687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:59:13.620687+0000","last_clean_scrub_stamp":"2023-11-24T07:59:13.620687+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T15:27:01.624152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.30","version":"0'0","reported_seq":11,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-11-24T07:59:16.292844+0000","last_change":"2023-11-24T07:59:14.669482+0000","last_active":"2023-11-24T07:59:16.292844+0000","last_peered":"2023-11-24T07:59:16.292844+0000","last_clean":"2023-11-24T07:59:16.292844+0000","last_became_active":"2023-11-24T07:59:14.669140+0000","last_became_peered":"2023-11-24T07:59:14.669140+0000","last_unstale":"2023-11-24T07:59:16.292844+0000","last_undegraded":"2023-11-24T07:59:16.292844+0000","last_fullsized":"2023-11-24T07:59:16.292844+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:59:13.620687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:59:13.620687+0000","last_clean_scrub_stamp":"2023-11-24T07:59:13.620687+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T13:16:48.753168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.2f","version":"0'0","reported_seq":11,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-11-24T07:59:15.650711+0000","last_change":"2023-11-24T07:59:14.643449+0000","last_active":"2023-11-24T07:59:15.650711+0000","last_peered":"2023-11-24T07:59:15.650711+0000","last_clean":"2023-11-24T07:59:15.650711+0000","last_became_active":"2023-11-24T07:59:14.643178+0000","last_became_peered":"2023-11-24T07:59:14.643178+0000","last_unstale":"2023-11-24T07:59:15.650711+0000","last_undegraded":"2023-11-24T07:59:15.650711+0000","last_fullsized":"2023-11-24T07:59:15.650711+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:59:13.620687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:59:13.620687+0000","last_clean_scrub_stamp":"2023-11-24T07:59:13.620687+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T15:36:44.561284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.2e","version":"0'0","reported_seq":11,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-11-24T07:59:15.650810+0000","last_change":"2023-11-24T07:59:14.647261+0000","last_active":"2023-11-24T07:59:15.650810+0000","last_peered":"2023-11-24T07:59:15.650810+0000","last_clean":"2023-11-24T07:59:15.650810+0000","last_became_active":"2023-11-24T07:59:14.646984+0000","last_became_peered":"2023-11-24T07:59:14.646984+0000","last_unstale":"2023-11-24T07:59:15.650810+0000","last_undegraded":"2023-11-24T07:59:15.650810+0000","last_fullsized":"2023-11-24T07:59:15.650810+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:59:13.620687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:59:13.620687+0000","last_clean_scrub_stamp":"2023-11-24T07:59:13.620687+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T11:25:11.031296+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.2d","version":"0'0","reported_seq":11,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-11-24T07:59:16.292770+0000","last_change":"2023-11-24T07:59:14.662845+0000","last_active":"2023-11-24T07:59:16.292770+0000","last_peered":"2023-11-24T07:59:16.292770+0000","last_clean":"2023-11-24T07:59:16.292770+0000","last_became_active":"2023-11-24T07:59:14.662448+0000","last_became_peered":"2023-11-24T07:59:14.662448+0000","last_unstale":"2023-11-24T07:59:16.292770+0000","last_undegraded":"2023-11-24T07:59:16.292770+0000","last_fullsized":"2023-11-24T07:59:16.292770+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:59:13.620687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:59:13.620687+0000","last_clean_scrub_stamp":"2023-11-24T07:59:13.620687+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T17:45:18.664406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.2c","version":"0'0","reported_seq":11,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-11-24T07:59:15.650708+0000","last_change":"2023-11-24T07:59:14.642598+0000","last_active":"2023-11-24T07:59:15.650708+0000","last_peered":"2023-11-24T07:59:15.650708+0000","last_clean":"2023-11-24T07:59:15.650708+0000","last_became_active":"2023-11-24T07:59:14.642357+0000","last_became_peered":"2023-11-24T07:59:14.642357+0000","last_unstale":"2023-11-24T07:59:15.650708+0000","last_undegraded":"2023-11-24T07:59:15.650708+0000","last_fullsized":"2023-11-24T07:59:15.650708+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:59:13.620687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:59:13.620687+0000","last_clean_scrub_stamp":"2023-11-24T07:59:13.620687+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T09:58:24.335415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.2b","version":"0'0","reported_seq":11,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-11-24T07:59:16.293409+0000","last_change":"2023-11-24T07:59:14.661043+0000","last_active":"2023-11-24T07:59:16.293409+0000","last_peered":"2023-11-24T07:59:16.293409+0000","last_clean":"2023-11-24T07:59:16.293409+0000","last_became_active":"2023-11-24T07:59:14.660775+0000","last_became_peered":"2023-11-24T07:59:14.660775+0000","last_unstale":"2023-11-24T07:59:16.293409+0000","last_undegraded":"2023-11-24T07:59:16.293409+0000","last_fullsized":"2023-11-24T07:59:16.293409+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:59:13.620687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:59:13.620687+0000","last_clean_scrub_stamp":"2023-11-24T07:59:13.620687+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T15:26:05.128703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.2a","version":"0'0","reported_seq":11,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-11-24T07:59:16.103881+0000","last_change":"2023-11-24T07:59:14.647428+0000","last_active":"2023-11-24T07:59:16.103881+0000","last_peered":"2023-11-24T07:59:16.103881+0000","last_clean":"2023-11-24T07:59:16.103881+0000","last_became_active":"2023-11-24T07:59:14.647154+0000","last_became_peered":"2023-11-24T07:59:14.647154+0000","last_unstale":"2023-11-24T07:59:16.103881+0000","last_undegraded":"2023-11-24T07:59:16.103881+0000","last_fullsized":"2023-11-24T07:59:16.103881+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:59:13.620687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:59:13.620687+0000","last_clean_scrub_stamp":"2023-11-24T07:59:13.620687+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T18:49:53.391209+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.29","version":"0'0","reported_seq":11,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-11-24T07:59:16.292441+0000","last_change":"2023-11-24T07:59:14.665583+0000","last_active":"2023-11-24T07:59:16.292441+0000","last_peered":"2023-11-24T07:59:16.292441+0000","last_clean":"2023-11-24T07:59:16.292441+0000","last_became_active":"2023-11-24T07:59:14.665127+0000","last_became_peered":"2023-11-24T07:59:14.665127+0000","last_unstale":"2023-11-24T07:59:16.292441+0000","last_undegraded":"2023-11-24T07:59:16.292441+0000","last_fullsized":"2023-11-24T07:59:16.292441+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:59:13.620687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:59:13.620687+0000","last_clean_scrub_stamp":"2023-11-24T07:59:13.620687+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T12:41:31.508405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.28","version":"0'0","reported_seq":11,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-11-24T07:59:16.292171+0000","last_change":"2023-11-24T07:59:14.660934+0000","last_active":"2023-11-24T07:59:16.292171+0000","last_peered":"2023-11-24T07:59:16.292171+0000","last_clean":"2023-11-24T07:59:16.292171+0000","last_became_active":"2023-11-24T07:59:14.660353+0000","last_became_peered":"2023-11-24T07:59:14.660353+0000","last_unstale":"2023-11-24T07:59:16.292171+0000","last_undegraded":"2023-11-24T07:59:16.292171+0000","last_fullsized":"2023-11-24T07:59:16.292171+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:59:13.620687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:59:13.620687+0000","last_clean_scrub_stamp":"2023-11-24T07:59:13.620687+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T15:04:45.070729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.27","version":"0'0","reported_seq":11,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-11-24T07:59:16.292550+0000","last_change":"2023-11-24T07:59:14.668930+0000","last_active":"2023-11-24T07:59:16.292550+0000","last_peered":"2023-11-24T07:59:16.292550+0000","last_clean":"2023-11-24T07:59:16.292550+0000","last_became_active":"2023-11-24T07:59:14.668480+0000","last_became_peered":"2023-11-24T07:59:14.668480+0000","last_unstale":"2023-11-24T07:59:16.292550+0000","last_undegraded":"2023-11-24T07:59:16.292550+0000","last_fullsized":"2023-11-24T07:59:16.292550+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:59:13.620687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:59:13.620687+0000","last_clean_scrub_stamp":"2023-11-24T07:59:13.620687+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T13:43:24.163968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.26","version":"0'0","reported_seq":11,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-11-24T07:59:16.292628+0000","last_change":"2023-11-24T07:59:14.661613+0000","last_active":"2023-11-24T07:59:16.292628+0000","last_peered":"2023-11-24T07:59:16.292628+0000","last_clean":"2023-11-24T07:59:16.292628+0000","last_became_active":"2023-11-24T07:59:14.661321+0000","last_became_peered":"2023-11-24T07:59:14.661321+0000","last_unstale":"2023-11-24T07:59:16.292628+0000","last_undegraded":"2023-11-24T07:59:16.292628+0000","last_fullsized":"2023-11-24T07:59:16.292628+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:59:13.620687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:59:13.620687+0000","last_clean_scrub_stamp":"2023-11-24T07:59:13.620687+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T19:21:43.761495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.25","version":"0'0","reported_seq":11,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-11-24T07:59:15.650595+0000","last_change":"2023-11-24T07:59:14.645456+0000","last_active":"2023-11-24T07:59:15.650595+0000","last_peered":"2023-11-24T07:59:15.650595+0000","last_clean":"2023-11-24T07:59:15.650595+0000","last_became_active":"2023-11-24T07:59:14.645148+0000","last_became_peered":"2023-11-24T07:59:14.645148+0000","last_unstale":"2023-11-24T07:59:15.650595+0000","last_undegraded":"2023-11-24T07:59:15.650595+0000","last_fullsized":"2023-11-24T07:59:15.650595+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:59:13.620687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:59:13.620687+0000","last_clean_scrub_stamp":"2023-11-24T07:59:13.620687+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T15:29:23.598719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.24","version":"0'0","reported_seq":11,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-11-24T07:59:16.293908+0000","last_change":"2023-11-24T07:59:14.660458+0000","last_active":"2023-11-24T07:59:16.293908+0000","last_peered":"2023-11-24T07:59:16.293908+0000","last_clean":"2023-11-24T07:59:16.293908+0000","last_became_active":"2023-11-24T07:59:14.659937+0000","last_became_peered":"2023-11-24T07:59:14.659937+0000","last_unstale":"2023-11-24T07:59:16.293908+0000","last_undegraded":"2023-11-24T07:59:16.293908+0000","last_fullsized":"2023-11-24T07:59:16.293908+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:59:13.620687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:59:13.620687+0000","last_clean_scrub_stamp":"2023-11-24T07:59:13.620687+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T13:00:14.118611+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.23","version":"0'0","reported_seq":11,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-11-24T07:59:16.292018+0000","last_change":"2023-11-24T07:59:14.658404+0000","last_active":"2023-11-24T07:59:16.292018+0000","last_peered":"2023-11-24T07:59:16.292018+0000","last_clean":"2023-11-24T07:59:16.292018+0000","last_became_active":"2023-11-24T07:59:14.658126+0000","last_became_peered":"2023-11-24T07:59:14.658126+0000","last_unstale":"2023-11-24T07:59:16.292018+0000","last_undegraded":"2023-11-24T07:59:16.292018+0000","last_fullsized":"2023-11-24T07:59:16.292018+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:59:13.620687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:59:13.620687+0000","last_clean_scrub_stamp":"2023-11-24T07:59:13.620687+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T17:11:47.648598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.22","version":"0'0","reported_seq":11,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-11-24T07:59:16.292749+0000","last_change":"2023-11-24T07:59:14.657880+0000","last_active":"2023-11-24T07:59:16.292749+0000","last_peered":"2023-11-24T07:59:16.292749+0000","last_clean":"2023-11-24T07:59:16.292749+0000","last_became_active":"2023-11-24T07:59:14.657479+0000","last_became_peered":"2023-11-24T07:59:14.657479+0000","last_unstale":"2023-11-24T07:59:16.292749+0000","last_undegraded":"2023-11-24T07:59:16.292749+0000","last_fullsized":"2023-11-24T07:59:16.292749+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:59:13.620687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:59:13.620687+0000","last_clean_scrub_stamp":"2023-11-24T07:59:13.620687+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T13:02:40.785870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.21","version":"0'0","reported_seq":11,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-11-24T07:59:16.294293+0000","last_change":"2023-11-24T07:59:14.667786+0000","last_active":"2023-11-24T07:59:16.294293+0000","last_peered":"2023-11-24T07:59:16.294293+0000","last_clean":"2023-11-24T07:59:16.294293+0000","last_became_active":"2023-11-24T07:59:14.667479+0000","last_became_peered":"2023-11-24T07:59:14.667479+0000","last_unstale":"2023-11-24T07:59:16.294293+0000","last_undegraded":"2023-11-24T07:59:16.294293+0000","last_fullsized":"2023-11-24T07:59:16.294293+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:59:13.620687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:59:13.620687+0000","last_clean_scrub_stamp":"2023-11-24T07:59:13.620687+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T16:41:24.799664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.20","version":"0'0","reported_seq":11,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-11-24T07:59:16.293066+0000","last_change":"2023-11-24T07:59:14.667849+0000","last_active":"2023-11-24T07:59:16.293066+0000","last_peered":"2023-11-24T07:59:16.293066+0000","last_clean":"2023-11-24T07:59:16.293066+0000","last_became_active":"2023-11-24T07:59:14.667479+0000","last_became_peered":"2023-11-24T07:59:14.667479+0000","last_unstale":"2023-11-24T07:59:16.293066+0000","last_undegraded":"2023-11-24T07:59:16.293066+0000","last_fullsized":"2023-11-24T07:59:16.293066+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:59:13.620687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:59:13.620687+0000","last_clean_scrub_stamp":"2023-11-24T07:59:13.620687+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T14:28:14.357404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1e","version":"0'0","reported_seq":11,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-11-24T07:59:16.292809+0000","last_change":"2023-11-24T07:59:14.669493+0000","last_active":"2023-11-24T07:59:16.292809+0000","last_peered":"2023-11-24T07:59:16.292809+0000","last_clean":"2023-11-24T07:59:16.292809+0000","last_became_active":"2023-11-24T07:59:14.669126+0000","last_became_peered":"2023-11-24T07:59:14.669126+0000","last_unstale":"2023-11-24T07:59:16.292809+0000","last_undegraded":"2023-11-24T07:59:16.292809+0000","last_fullsized":"2023-11-24T07:59:16.292809+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:59:13.620687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:59:13.620687+0000","last_clean_scrub_stamp":"2023-11-24T07:59:13.620687+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T13:16:42.796914+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.1f","version":"73'4","reported_seq":16,"reported_epoch":73,"state":"active+clean","last_fresh":"2023-11-24T07:59:14.666712+0000","last_change":"2023-11-24T07:59:13.653337+0000","last_active":"2023-11-24T07:59:14.666712+0000","last_peered":"2023-11-24T07:59:14.666712+0000","last_clean":"2023-11-24T07:59:14.666712+0000","last_became_active":"2023-11-24T07:59:13.653064+0000","last_became_peered":"2023-11-24T07:59:13.653064+0000","last_unstale":"2023-11-24T07:59:14.666712+0000","last_undegraded":"2023-11-24T07:59:14.666712+0000","last_fullsized":"2023-11-24T07:59:14.666712+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:59:12.615544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:59:12.615544+0000","last_clean_scrub_stamp":"2023-11-24T07:59:12.615544+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T14:28:13.352261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.1f","version":"0'0","reported_seq":11,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-11-24T07:59:16.292104+0000","last_change":"2023-11-24T07:59:14.651524+0000","last_active":"2023-11-24T07:59:16.292104+0000","last_peered":"2023-11-24T07:59:16.292104+0000","last_clean":"2023-11-24T07:59:16.292104+0000","last_became_active":"2023-11-24T07:59:14.651131+0000","last_became_peered":"2023-11-24T07:59:14.651131+0000","last_unstale":"2023-11-24T07:59:16.292104+0000","last_undegraded":"2023-11-24T07:59:16.292104+0000","last_fullsized":"2023-11-24T07:59:16.292104+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:59:13.620687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:59:13.620687+0000","last_clean_scrub_stamp":"2023-11-24T07:59:13.620687+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T13:48:52.893285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.1e","version":"73'1","reported_seq":13,"reported_epoch":73,"state":"active+clean","last_fresh":"2023-11-24T07:59:14.653141+0000","last_change":"2023-11-24T07:59:13.636981+0000","last_active":"2023-11-24T07:59:14.653141+0000","last_peered":"2023-11-24T07:59:14.653141+0000","last_clean":"2023-11-24T07:59:14.653141+0000","last_became_active":"2023-11-24T07:59:13.636704+0000","last_became_peered":"2023-11-24T07:59:13.636704+0000","last_unstale":"2023-11-24T07:59:14.653141+0000","last_undegraded":"2023-11-24T07:59:14.653141+0000","last_fullsized":"2023-11-24T07:59:14.653141+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:59:12.615544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:59:12.615544+0000","last_clean_scrub_stamp":"2023-11-24T07:59:12.615544+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T11:01:49.234072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.1c","version":"0'0","reported_seq":11,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-11-24T07:59:16.104026+0000","last_change":"2023-11-24T07:59:14.658068+0000","last_active":"2023-11-24T07:59:16.104026+0000","last_peered":"2023-11-24T07:59:16.104026+0000","last_clean":"2023-11-24T07:59:16.104026+0000","last_became_active":"2023-11-24T07:59:14.657772+0000","last_became_peered":"2023-11-24T07:59:14.657772+0000","last_unstale":"2023-11-24T07:59:16.104026+0000","last_undegraded":"2023-11-24T07:59:16.104026+0000","last_fullsized":"2023-11-24T07:59:16.104026+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:59:13.620687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:59:13.620687+0000","last_clean_scrub_stamp":"2023-11-24T07:59:13.620687+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:25:35.342924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.1d","version":"73'2","reported_seq":14,"reported_epoch":73,"state":"active+clean","last_fresh":"2023-11-24T07:59:14.671494+0000","last_change":"2023-11-24T07:59:13.651205+0000","last_active":"2023-11-24T07:59:14.671494+0000","last_peered":"2023-11-24T07:59:14.671494+0000","last_clean":"2023-11-24T07:59:14.671494+0000","last_became_active":"2023-11-24T07:59:13.650625+0000","last_became_peered":"2023-11-24T07:59:13.650625+0000","last_unstale":"2023-11-24T07:59:14.671494+0000","last_undegraded":"2023-11-24T07:59:14.671494+0000","last_fullsized":"2023-11-24T07:59:14.671494+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:59:12.615544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:59:12.615544+0000","last_clean_scrub_stamp":"2023-11-24T07:59:12.615544+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T11:55:58.774666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.1d","version":"0'0","reported_seq":11,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-11-24T07:59:16.292657+0000","last_change":"2023-11-24T07:59:14.646515+0000","last_active":"2023-11-24T07:59:16.292657+0000","last_peered":"2023-11-24T07:59:16.292657+0000","last_clean":"2023-11-24T07:59:16.292657+0000","last_became_active":"2023-11-24T07:59:14.646215+0000","last_became_peered":"2023-11-24T07:59:14.646215+0000","last_unstale":"2023-11-24T07:59:16.292657+0000","last_undegraded":"2023-11-24T07:59:16.292657+0000","last_fullsized":"2023-11-24T07:59:16.292657+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:59:13.620687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:59:13.620687+0000","last_clean_scrub_stamp":"2023-11-24T07:59:13.620687+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T08:19:45.582905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.1c","version":"73'2","reported_seq":14,"reported_epoch":73,"state":"active+clean","last_fresh":"2023-11-24T07:59:14.665769+0000","last_change":"2023-11-24T07:59:13.649265+0000","last_active":"2023-11-24T07:59:14.665769+0000","last_peered":"2023-11-24T07:59:14.665769+0000","last_clean":"2023-11-24T07:59:14.665769+0000","last_became_active":"2023-11-24T07:59:13.648926+0000","last_became_peered":"2023-11-24T07:59:13.648926+0000","last_unstale":"2023-11-24T07:59:14.665769+0000","last_undegraded":"2023-11-24T07:59:14.665769+0000","last_fullsized":"2023-11-24T07:59:14.665769+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:59:12.615544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:59:12.615544+0000","last_clean_scrub_stamp":"2023-11-24T07:59:12.615544+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T17:30:27.786439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.1a","version":"0'0","reported_seq":11,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-11-24T07:59:16.293185+0000","last_change":"2023-11-24T07:59:14.657208+0000","last_active":"2023-11-24T07:59:16.293185+0000","last_peered":"2023-11-24T07:59:16.293185+0000","last_clean":"2023-11-24T07:59:16.293185+0000","last_became_active":"2023-11-24T07:59:14.656892+0000","last_became_peered":"2023-11-24T07:59:14.656892+0000","last_unstale":"2023-11-24T07:59:16.293185+0000","last_undegraded":"2023-11-24T07:59:16.293185+0000","last_fullsized":"2023-11-24T07:59:16.293185+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:59:13.620687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:59:13.620687+0000","last_clean_scrub_stamp":"2023-11-24T07:59:13.620687+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T19:33:18.678728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.1b","version":"73'1","reported_seq":13,"reported_epoch":73,"state":"active+clean","last_fresh":"2023-11-24T07:59:14.666337+0000","last_change":"2023-11-24T07:59:13.655837+0000","last_active":"2023-11-24T07:59:14.666337+0000","last_peered":"2023-11-24T07:59:14.666337+0000","last_clean":"2023-11-24T07:59:14.666337+0000","last_became_active":"2023-11-24T07:59:13.655294+0000","last_became_peered":"2023-11-24T07:59:13.655294+0000","last_unstale":"2023-11-24T07:59:14.666337+0000","last_undegraded":"2023-11-24T07:59:14.666337+0000","last_fullsized":"2023-11-24T07:59:14.666337+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:59:12.615544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:59:12.615544+0000","last_clean_scrub_stamp":"2023-11-24T07:59:12.615544+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:47:41.266774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.1b","version":"0'0","reported_seq":11,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-11-24T07:59:16.104163+0000","last_change":"2023-11-24T07:59:14.658822+0000","last_active":"2023-11-24T07:59:16.104163+0000","last_peered":"2023-11-24T07:59:16.104163+0000","last_clean":"2023-11-24T07:59:16.104163+0000","last_became_active":"2023-11-24T07:59:14.658562+0000","last_became_peered":"2023-11-24T07:59:14.658562+0000","last_unstale":"2023-11-24T07:59:16.104163+0000","last_undegraded":"2023-11-24T07:59:16.104163+0000","last_fullsized":"2023-11-24T07:59:16.104163+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:59:13.620687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:59:13.620687+0000","last_clean_scrub_stamp":"2023-11-24T07:59:13.620687+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T09:43:52.247266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.1a","version":"73'1","reported_seq":13,"reported_epoch":73,"state":"active+clean","last_fresh":"2023-11-24T07:59:14.658566+0000","last_change":"2023-11-24T07:59:13.650277+0000","last_active":"2023-11-24T07:59:14.658566+0000","last_peered":"2023-11-24T07:59:14.658566+0000","last_clean":"2023-11-24T07:59:14.658566+0000","last_became_active":"2023-11-24T07:59:13.649846+0000","last_became_peered":"2023-11-24T07:59:13.649846+0000","last_unstale":"2023-11-24T07:59:14.658566+0000","last_undegraded":"2023-11-24T07:59:14.658566+0000","last_fullsized":"2023-11-24T07:59:14.658566+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:59:12.615544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:59:12.615544+0000","last_clean_scrub_stamp":"2023-11-24T07:59:12.615544+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T12:39:08.993327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.18","version":"0'0","reported_seq":11,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-11-24T07:59:15.648205+0000","last_change":"2023-11-24T07:59:14.641027+0000","last_active":"2023-11-24T07:59:15.648205+0000","last_peered":"2023-11-24T07:59:15.648205+0000","last_clean":"2023-11-24T07:59:15.648205+0000","last_became_active":"2023-11-24T07:59:14.640764+0000","last_became_peered":"2023-11-24T07:59:14.640764+0000","last_unstale":"2023-11-24T07:59:15.648205+0000","last_undegraded":"2023-11-24T07:59:15.648205+0000","last_fullsized":"2023-11-24T07:59:15.648205+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:59:13.620687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:59:13.620687+0000","last_clean_scrub_stamp":"2023-11-24T07:59:13.620687+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T19:37:04.059622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.19","version":"0'0","reported_seq":12,"reported_epoch":73,"state":"active+clean","last_fresh":"2023-11-24T07:59:14.635053+0000","last_change":"2023-11-24T07:59:13.655628+0000","last_active":"2023-11-24T07:59:14.635053+0000","last_peered":"2023-11-24T07:59:14.635053+0000","last_clean":"2023-11-24T07:59:14.635053+0000","last_became_active":"2023-11-24T07:59:13.655373+0000","last_became_peered":"2023-11-24T07:59:13.655373+0000","last_unstale":"2023-11-24T07:59:14.635053+0000","last_undegraded":"2023-11-24T07:59:14.635053+0000","last_fullsized":"2023-11-24T07:59:14.635053+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:59:12.615544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:59:12.615544+0000","last_clean_scrub_stamp":"2023-11-24T07:59:12.615544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T12:01:57.940430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.19","version":"0'0","reported_seq":11,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-11-24T07:59:15.649327+0000","last_change":"2023-11-24T07:59:14.654812+0000","last_active":"2023-11-24T07:59:15.649327+0000","last_peered":"2023-11-24T07:59:15.649327+0000","last_clean":"2023-11-24T07:59:15.649327+0000","last_became_active":"2023-11-24T07:59:14.654554+0000","last_became_peered":"2023-11-24T07:59:14.654554+0000","last_unstale":"2023-11-24T07:59:15.649327+0000","last_undegraded":"2023-11-24T07:59:15.649327+0000","last_fullsized":"2023-11-24T07:59:15.649327+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:59:13.620687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:59:13.620687+0000","last_clean_scrub_stamp":"2023-11-24T07:59:13.620687+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T14:28:14.357404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.18","version":"0'0","reported_seq":12,"reported_epoch":73,"state":"active+clean","last_fresh":"2023-11-24T07:59:14.639330+0000","last_change":"2023-11-24T07:59:13.647547+0000","last_active":"2023-11-24T07:59:14.639330+0000","last_peered":"2023-11-24T07:59:14.639330+0000","last_clean":"2023-11-24T07:59:14.639330+0000","last_became_active":"2023-11-24T07:59:13.647276+0000","last_became_peered":"2023-11-24T07:59:13.647276+0000","last_unstale":"2023-11-24T07:59:14.639330+0000","last_undegraded":"2023-11-24T07:59:14.639330+0000","last_fullsized":"2023-11-24T07:59:14.639330+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:59:12.615544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:59:12.615544+0000","last_clean_scrub_stamp":"2023-11-24T07:59:12.615544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T16:36:46.867316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.16","version":"0'0","reported_seq":11,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-11-24T07:59:16.293403+0000","last_change":"2023-11-24T07:59:14.666084+0000","last_active":"2023-11-24T07:59:16.293403+0000","last_peered":"2023-11-24T07:59:16.293403+0000","last_clean":"2023-11-24T07:59:16.293403+0000","last_became_active":"2023-11-24T07:59:14.665449+0000","last_became_peered":"2023-11-24T07:59:14.665449+0000","last_unstale":"2023-11-24T07:59:16.293403+0000","last_undegraded":"2023-11-24T07:59:16.293403+0000","last_fullsized":"2023-11-24T07:59:16.293403+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:59:13.620687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:59:13.620687+0000","last_clean_scrub_stamp":"2023-11-24T07:59:13.620687+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T19:01:46.731335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.17","version":"0'0","reported_seq":12,"reported_epoch":73,"state":"active+clean","last_fresh":"2023-11-24T07:59:14.645021+0000","last_change":"2023-11-24T07:59:13.652504+0000","last_active":"2023-11-24T07:59:14.645021+0000","last_peered":"2023-11-24T07:59:14.645021+0000","last_clean":"2023-11-24T07:59:14.645021+0000","last_became_active":"2023-11-24T07:59:13.652187+0000","last_became_peered":"2023-11-24T07:59:13.652187+0000","last_unstale":"2023-11-24T07:59:14.645021+0000","last_undegraded":"2023-11-24T07:59:14.645021+0000","last_fullsized":"2023-11-24T07:59:14.645021+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:59:12.615544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:59:12.615544+0000","last_clean_scrub_stamp":"2023-11-24T07:59:12.615544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T17:42:54.827147+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.17","version":"0'0","reported_seq":11,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-11-24T07:59:16.293529+0000","last_change":"2023-11-24T07:59:14.669203+0000","last_active":"2023-11-24T07:59:16.293529+0000","last_peered":"2023-11-24T07:59:16.293529+0000","last_clean":"2023-11-24T07:59:16.293529+0000","last_became_active":"2023-11-24T07:59:14.668944+0000","last_became_peered":"2023-11-24T07:59:14.668944+0000","last_unstale":"2023-11-24T07:59:16.293529+0000","last_undegraded":"2023-11-24T07:59:16.293529+0000","last_fullsized":"2023-11-24T07:59:16.293529+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:59:13.620687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:59:13.620687+0000","last_clean_scrub_stamp":"2023-11-24T07:59:13.620687+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T15:47:40.090700+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.16","version":"0'0","reported_seq":12,"reported_epoch":73,"state":"active+clean","last_fresh":"2023-11-24T07:59:14.646587+0000","last_change":"2023-11-24T07:59:13.646811+0000","last_active":"2023-11-24T07:59:14.646587+0000","last_peered":"2023-11-24T07:59:14.646587+0000","last_clean":"2023-11-24T07:59:14.646587+0000","last_became_active":"2023-11-24T07:59:13.646563+0000","last_became_peered":"2023-11-24T07:59:13.646563+0000","last_unstale":"2023-11-24T07:59:14.646587+0000","last_undegraded":"2023-11-24T07:59:14.646587+0000","last_fullsized":"2023-11-24T07:59:14.646587+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:59:12.615544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:59:12.615544+0000","last_clean_scrub_stamp":"2023-11-24T07:59:12.615544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T17:15:53.221458+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.14","version":"0'0","reported_seq":11,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-11-24T07:59:16.293297+0000","last_change":"2023-11-24T07:59:14.657194+0000","last_active":"2023-11-24T07:59:16.293297+0000","last_peered":"2023-11-24T07:59:16.293297+0000","last_clean":"2023-11-24T07:59:16.293297+0000","last_became_active":"2023-11-24T07:59:14.656915+0000","last_became_peered":"2023-11-24T07:59:14.656915+0000","last_unstale":"2023-11-24T07:59:16.293297+0000","last_undegraded":"2023-11-24T07:59:16.293297+0000","last_fullsized":"2023-11-24T07:59:16.293297+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:59:13.620687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:59:13.620687+0000","last_clean_scrub_stamp":"2023-11-24T07:59:13.620687+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T16:19:25.371696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.15","version":"73'3","reported_seq":15,"reported_epoch":73,"state":"active+clean","last_fresh":"2023-11-24T07:59:14.656254+0000","last_change":"2023-11-24T07:59:13.645182+0000","last_active":"2023-11-24T07:59:14.656254+0000","last_peered":"2023-11-24T07:59:14.656254+0000","last_clean":"2023-11-24T07:59:14.656254+0000","last_became_active":"2023-11-24T07:59:13.644887+0000","last_became_peered":"2023-11-24T07:59:13.644887+0000","last_unstale":"2023-11-24T07:59:14.656254+0000","last_undegraded":"2023-11-24T07:59:14.656254+0000","last_fullsized":"2023-11-24T07:59:14.656254+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:59:12.615544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:59:12.615544+0000","last_clean_scrub_stamp":"2023-11-24T07:59:12.615544+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T19:44:36.104587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.15","version":"0'0","reported_seq":11,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-11-24T07:59:15.650622+0000","last_change":"2023-11-24T07:59:14.648158+0000","last_active":"2023-11-24T07:59:15.650622+0000","last_peered":"2023-11-24T07:59:15.650622+0000","last_clean":"2023-11-24T07:59:15.650622+0000","last_became_active":"2023-11-24T07:59:14.647759+0000","last_became_peered":"2023-11-24T07:59:14.647759+0000","last_unstale":"2023-11-24T07:59:15.650622+0000","last_undegraded":"2023-11-24T07:59:15.650622+0000","last_fullsized":"2023-11-24T07:59:15.650622+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:59:13.620687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:59:13.620687+0000","last_clean_scrub_stamp":"2023-11-24T07:59:13.620687+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:15:07.828806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.14","version":"73'6","reported_seq":18,"reported_epoch":73,"state":"active+clean","last_fresh":"2023-11-24T07:59:14.668309+0000","last_change":"2023-11-24T07:59:13.647480+0000","last_active":"2023-11-24T07:59:14.668309+0000","last_peered":"2023-11-24T07:59:14.668309+0000","last_clean":"2023-11-24T07:59:14.668309+0000","last_became_active":"2023-11-24T07:59:13.647183+0000","last_became_peered":"2023-11-24T07:59:13.647183+0000","last_unstale":"2023-11-24T07:59:14.668309+0000","last_undegraded":"2023-11-24T07:59:14.668309+0000","last_fullsized":"2023-11-24T07:59:14.668309+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:59:12.615544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:59:12.615544+0000","last_clean_scrub_stamp":"2023-11-24T07:59:12.615544+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T13:25:11.926864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.12","version":"0'0","reported_seq":11,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-11-24T07:59:16.103875+0000","last_change":"2023-11-24T07:59:14.647988+0000","last_active":"2023-11-24T07:59:16.103875+0000","last_peered":"2023-11-24T07:59:16.103875+0000","last_clean":"2023-11-24T07:59:16.103875+0000","last_became_active":"2023-11-24T07:59:14.647702+0000","last_became_peered":"2023-11-24T07:59:14.647702+0000","last_unstale":"2023-11-24T07:59:16.103875+0000","last_undegraded":"2023-11-24T07:59:16.103875+0000","last_fullsized":"2023-11-24T07:59:16.103875+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:59:13.620687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:59:13.620687+0000","last_clean_scrub_stamp":"2023-11-24T07:59:13.620687+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T09:23:20.285028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.13","version":"73'2","reported_seq":14,"reported_epoch":73,"state":"active+clean","last_fresh":"2023-11-24T07:59:14.665155+0000","last_change":"2023-11-24T07:59:13.652223+0000","last_active":"2023-11-24T07:59:14.665155+0000","last_peered":"2023-11-24T07:59:14.665155+0000","last_clean":"2023-11-24T07:59:14.665155+0000","last_became_active":"2023-11-24T07:59:13.651976+0000","last_became_peered":"2023-11-24T07:59:13.651976+0000","last_unstale":"2023-11-24T07:59:14.665155+0000","last_undegraded":"2023-11-24T07:59:14.665155+0000","last_fullsized":"2023-11-24T07:59:14.665155+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:59:12.615544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:59:12.615544+0000","last_clean_scrub_stamp":"2023-11-24T07:59:12.615544+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:47:41.266774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.13","version":"0'0","reported_seq":11,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-11-24T07:59:16.292350+0000","last_change":"2023-11-24T07:59:14.660260+0000","last_active":"2023-11-24T07:59:16.292350+0000","last_peered":"2023-11-24T07:59:16.292350+0000","last_clean":"2023-11-24T07:59:16.292350+0000","last_became_active":"2023-11-24T07:59:14.659942+0000","last_became_peered":"2023-11-24T07:59:14.659942+0000","last_unstale":"2023-11-24T07:59:16.292350+0000","last_undegraded":"2023-11-24T07:59:16.292350+0000","last_fullsized":"2023-11-24T07:59:16.292350+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:59:13.620687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:59:13.620687+0000","last_clean_scrub_stamp":"2023-11-24T07:59:13.620687+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T13:48:52.893285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.12","version":"0'0","reported_seq":12,"reported_epoch":73,"state":"active+clean","last_fresh":"2023-11-24T07:59:14.634946+0000","last_change":"2023-11-24T07:59:13.643551+0000","last_active":"2023-11-24T07:59:14.634946+0000","last_peered":"2023-11-24T07:59:14.634946+0000","last_clean":"2023-11-24T07:59:14.634946+0000","last_became_active":"2023-11-24T07:59:13.643276+0000","last_became_peered":"2023-11-24T07:59:13.643276+0000","last_unstale":"2023-11-24T07:59:14.634946+0000","last_undegraded":"2023-11-24T07:59:14.634946+0000","last_fullsized":"2023-11-24T07:59:14.634946+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:59:12.615544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:59:12.615544+0000","last_clean_scrub_stamp":"2023-11-24T07:59:12.615544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T14:50:22.913346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.10","version":"0'0","reported_seq":11,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-11-24T07:59:15.650380+0000","last_change":"2023-11-24T07:59:14.641647+0000","last_active":"2023-11-24T07:59:15.650380+0000","last_peered":"2023-11-24T07:59:15.650380+0000","last_clean":"2023-11-24T07:59:15.650380+0000","last_became_active":"2023-11-24T07:59:14.641383+0000","last_became_peered":"2023-11-24T07:59:14.641383+0000","last_unstale":"2023-11-24T07:59:15.650380+0000","last_undegraded":"2023-11-24T07:59:15.650380+0000","last_fullsized":"2023-11-24T07:59:15.650380+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:59:13.620687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:59:13.620687+0000","last_clean_scrub_stamp":"2023-11-24T07:59:13.620687+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T08:18:30.845129+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.11","version":"0'0","reported_seq":12,"reported_epoch":73,"state":"active+clean","last_fresh":"2023-11-24T07:59:14.636353+0000","last_change":"2023-11-24T07:59:13.650108+0000","last_active":"2023-11-24T07:59:14.636353+0000","last_peered":"2023-11-24T07:59:14.636353+0000","last_clean":"2023-11-24T07:59:14.636353+0000","last_became_active":"2023-11-24T07:59:13.649781+0000","last_became_peered":"2023-11-24T07:59:13.649781+0000","last_unstale":"2023-11-24T07:59:14.636353+0000","last_undegraded":"2023-11-24T07:59:14.636353+0000","last_fullsized":"2023-11-24T07:59:14.636353+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:59:12.615544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:59:12.615544+0000","last_clean_scrub_stamp":"2023-11-24T07:59:12.615544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T19:09:46.221313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.11","version":"0'0","reported_seq":11,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-11-24T07:59:16.291803+0000","last_change":"2023-11-24T07:59:14.648803+0000","last_active":"2023-11-24T07:59:16.291803+0000","last_peered":"2023-11-24T07:59:16.291803+0000","last_clean":"2023-11-24T07:59:16.291803+0000","last_became_active":"2023-11-24T07:59:14.644779+0000","last_became_peered":"2023-11-24T07:59:14.644779+0000","last_unstale":"2023-11-24T07:59:16.291803+0000","last_undegraded":"2023-11-24T07:59:16.291803+0000","last_fullsized":"2023-11-24T07:59:16.291803+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:59:13.620687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:59:13.620687+0000","last_clean_scrub_stamp":"2023-11-24T07:59:13.620687+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T13:30:24.640475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.10","version":"73'8","reported_seq":20,"reported_epoch":73,"state":"active+clean","last_fresh":"2023-11-24T07:59:14.667012+0000","last_change":"2023-11-24T07:59:13.650307+0000","last_active":"2023-11-24T07:59:14.667012+0000","last_peered":"2023-11-24T07:59:14.667012+0000","last_clean":"2023-11-24T07:59:14.667012+0000","last_became_active":"2023-11-24T07:59:13.649976+0000","last_became_peered":"2023-11-24T07:59:13.649976+0000","last_unstale":"2023-11-24T07:59:14.667012+0000","last_undegraded":"2023-11-24T07:59:14.667012+0000","last_fullsized":"2023-11-24T07:59:14.667012+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:59:12.615544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:59:12.615544+0000","last_clean_scrub_stamp":"2023-11-24T07:59:12.615544+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T15:34:17.729593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.e","version":"0'0","reported_seq":11,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-11-24T07:59:16.293506+0000","last_change":"2023-11-24T07:59:14.662540+0000","last_active":"2023-11-24T07:59:16.293506+0000","last_peered":"2023-11-24T07:59:16.293506+0000","last_clean":"2023-11-24T07:59:16.293506+0000","last_became_active":"2023-11-24T07:59:14.662109+0000","last_became_peered":"2023-11-24T07:59:14.662109+0000","last_unstale":"2023-11-24T07:59:16.293506+0000","last_undegraded":"2023-11-24T07:59:16.293506+0000","last_fullsized":"2023-11-24T07:59:16.293506+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:59:13.620687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:59:13.620687+0000","last_clean_scrub_stamp":"2023-11-24T07:59:13.620687+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T15:27:01.624152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.f","version":"73'1","reported_seq":13,"reported_epoch":73,"state":"active+clean","last_fresh":"2023-11-24T07:59:14.659241+0000","last_change":"2023-11-24T07:59:13.651768+0000","last_active":"2023-11-24T07:59:14.659241+0000","last_peered":"2023-11-24T07:59:14.659241+0000","last_clean":"2023-11-24T07:59:14.659241+0000","last_became_active":"2023-11-24T07:59:13.650680+0000","last_became_peered":"2023-11-24T07:59:13.650680+0000","last_unstale":"2023-11-24T07:59:14.659241+0000","last_undegraded":"2023-11-24T07:59:14.659241+0000","last_fullsized":"2023-11-24T07:59:14.659241+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:59:12.615544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:59:12.615544+0000","last_clean_scrub_stamp":"2023-11-24T07:59:12.615544+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T11:25:10.026153+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.f","version":"0'0","reported_seq":11,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-11-24T07:59:16.293622+0000","last_change":"2023-11-24T07:59:14.668869+0000","last_active":"2023-11-24T07:59:16.293622+0000","last_peered":"2023-11-24T07:59:16.293622+0000","last_clean":"2023-11-24T07:59:16.293622+0000","last_became_active":"2023-11-24T07:59:14.668493+0000","last_became_peered":"2023-11-24T07:59:14.668493+0000","last_unstale":"2023-11-24T07:59:16.293622+0000","last_undegraded":"2023-11-24T07:59:16.293622+0000","last_fullsized":"2023-11-24T07:59:16.293622+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:59:13.620687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:59:13.620687+0000","last_clean_scrub_stamp":"2023-11-24T07:59:13.620687+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T17:30:28.791582+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.e","version":"73'2","reported_seq":14,"reported_epoch":73,"state":"active+clean","last_fresh":"2023-11-24T07:59:14.668349+0000","last_change":"2023-11-24T07:59:13.646781+0000","last_active":"2023-11-24T07:59:14.668349+0000","last_peered":"2023-11-24T07:59:14.668349+0000","last_clean":"2023-11-24T07:59:14.668349+0000","last_became_active":"2023-11-24T07:59:13.646208+0000","last_became_peered":"2023-11-24T07:59:13.646208+0000","last_unstale":"2023-11-24T07:59:14.668349+0000","last_undegraded":"2023-11-24T07:59:14.668349+0000","last_fullsized":"2023-11-24T07:59:14.668349+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:59:12.615544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:59:12.615544+0000","last_clean_scrub_stamp":"2023-11-24T07:59:12.615544+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T19:37:03.054479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.c","version":"0'0","reported_seq":11,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-11-24T07:59:16.292086+0000","last_change":"2023-11-24T07:59:14.649728+0000","last_active":"2023-11-24T07:59:16.292086+0000","last_peered":"2023-11-24T07:59:16.292086+0000","last_clean":"2023-11-24T07:59:16.292086+0000","last_became_active":"2023-11-24T07:59:14.649405+0000","last_became_peered":"2023-11-24T07:59:14.649405+0000","last_unstale":"2023-11-24T07:59:16.292086+0000","last_undegraded":"2023-11-24T07:59:16.292086+0000","last_fullsized":"2023-11-24T07:59:16.292086+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:59:13.620687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:59:13.620687+0000","last_clean_scrub_stamp":"2023-11-24T07:59:13.620687+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:54:57.076700+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.d","version":"73'1","reported_seq":13,"reported_epoch":73,"state":"active+clean","last_fresh":"2023-11-24T07:59:14.668658+0000","last_change":"2023-11-24T07:59:13.652598+0000","last_active":"2023-11-24T07:59:14.668658+0000","last_peered":"2023-11-24T07:59:14.668658+0000","last_clean":"2023-11-24T07:59:14.668658+0000","last_became_active":"2023-11-24T07:59:13.652005+0000","last_became_peered":"2023-11-24T07:59:13.652005+0000","last_unstale":"2023-11-24T07:59:14.668658+0000","last_undegraded":"2023-11-24T07:59:14.668658+0000","last_fullsized":"2023-11-24T07:59:14.668658+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:59:12.615544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:59:12.615544+0000","last_clean_scrub_stamp":"2023-11-24T07:59:12.615544+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T14:07:52.677308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.d","version":"0'0","reported_seq":11,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-11-24T07:59:16.291967+0000","last_change":"2023-11-24T07:59:14.664027+0000","last_active":"2023-11-24T07:59:16.291967+0000","last_peered":"2023-11-24T07:59:16.291967+0000","last_clean":"2023-11-24T07:59:16.291967+0000","last_became_active":"2023-11-24T07:59:14.663669+0000","last_became_peered":"2023-11-24T07:59:14.663669+0000","last_unstale":"2023-11-24T07:59:16.291967+0000","last_undegraded":"2023-11-24T07:59:16.291967+0000","last_fullsized":"2023-11-24T07:59:16.291967+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:59:13.620687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:59:13.620687+0000","last_clean_scrub_stamp":"2023-11-24T07:59:13.620687+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T08:09:54.291825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.c","version":"0'0","reported_seq":12,"reported_epoch":73,"state":"active+clean","last_fresh":"2023-11-24T07:59:14.635643+0000","last_change":"2023-11-24T07:59:13.655857+0000","last_active":"2023-11-24T07:59:14.635643+0000","last_peered":"2023-11-24T07:59:14.635643+0000","last_clean":"2023-11-24T07:59:14.635643+0000","last_became_active":"2023-11-24T07:59:13.655496+0000","last_became_peered":"2023-11-24T07:59:13.655496+0000","last_unstale":"2023-11-24T07:59:14.635643+0000","last_undegraded":"2023-11-24T07:59:14.635643+0000","last_fullsized":"2023-11-24T07:59:14.635643+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:59:12.615544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:59:12.615544+0000","last_clean_scrub_stamp":"2023-11-24T07:59:12.615544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T13:00:13.113468+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.b","version":"0'0","reported_seq":11,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-11-24T07:59:15.648400+0000","last_change":"2023-11-24T07:59:14.640493+0000","last_active":"2023-11-24T07:59:15.648400+0000","last_peered":"2023-11-24T07:59:15.648400+0000","last_clean":"2023-11-24T07:59:15.648400+0000","last_became_active":"2023-11-24T07:59:14.640172+0000","last_became_peered":"2023-11-24T07:59:14.640172+0000","last_unstale":"2023-11-24T07:59:15.648400+0000","last_undegraded":"2023-11-24T07:59:15.648400+0000","last_fullsized":"2023-11-24T07:59:15.648400+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:59:13.620687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:59:13.620687+0000","last_clean_scrub_stamp":"2023-11-24T07:59:13.620687+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T14:28:14.357404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.a","version":"0'0","reported_seq":12,"reported_epoch":73,"state":"active+clean","last_fresh":"2023-11-24T07:59:14.635916+0000","last_change":"2023-11-24T07:59:13.653387+0000","last_active":"2023-11-24T07:59:14.635916+0000","last_peered":"2023-11-24T07:59:14.635916+0000","last_clean":"2023-11-24T07:59:14.635916+0000","last_became_active":"2023-11-24T07:59:13.653132+0000","last_became_peered":"2023-11-24T07:59:13.653132+0000","last_unstale":"2023-11-24T07:59:14.635916+0000","last_undegraded":"2023-11-24T07:59:14.635916+0000","last_fullsized":"2023-11-24T07:59:14.635916+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:59:12.615544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:59:12.615544+0000","last_clean_scrub_stamp":"2023-11-24T07:59:12.615544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T16:41:01.392188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.8","version":"0'0","reported_seq":11,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-11-24T07:59:15.648245+0000","last_change":"2023-11-24T07:59:14.644467+0000","last_active":"2023-11-24T07:59:15.648245+0000","last_peered":"2023-11-24T07:59:15.648245+0000","last_clean":"2023-11-24T07:59:15.648245+0000","last_became_active":"2023-11-24T07:59:14.644057+0000","last_became_peered":"2023-11-24T07:59:14.644057+0000","last_unstale":"2023-11-24T07:59:15.648245+0000","last_undegraded":"2023-11-24T07:59:15.648245+0000","last_fullsized":"2023-11-24T07:59:15.648245+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:59:13.620687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:59:13.620687+0000","last_clean_scrub_stamp":"2023-11-24T07:59:13.620687+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T19:44:37.109730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.9","version":"0'0","reported_seq":12,"reported_epoch":73,"state":"active+clean","last_fresh":"2023-11-24T07:59:14.646912+0000","last_change":"2023-11-24T07:59:13.649520+0000","last_active":"2023-11-24T07:59:14.646912+0000","last_peered":"2023-11-24T07:59:14.646912+0000","last_clean":"2023-11-24T07:59:14.646912+0000","last_became_active":"2023-11-24T07:59:13.649255+0000","last_became_peered":"2023-11-24T07:59:13.649255+0000","last_unstale":"2023-11-24T07:59:14.646912+0000","last_undegraded":"2023-11-24T07:59:14.646912+0000","last_fullsized":"2023-11-24T07:59:14.646912+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:59:12.615544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:59:12.615544+0000","last_clean_scrub_stamp":"2023-11-24T07:59:12.615544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T19:44:36.104587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.9","version":"0'0","reported_seq":11,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-11-24T07:59:15.649420+0000","last_change":"2023-11-24T07:59:14.653039+0000","last_active":"2023-11-24T07:59:15.649420+0000","last_peered":"2023-11-24T07:59:15.649420+0000","last_clean":"2023-11-24T07:59:15.649420+0000","last_became_active":"2023-11-24T07:59:14.652751+0000","last_became_peered":"2023-11-24T07:59:14.652751+0000","last_unstale":"2023-11-24T07:59:15.649420+0000","last_undegraded":"2023-11-24T07:59:15.649420+0000","last_fullsized":"2023-11-24T07:59:15.649420+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:59:13.620687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:59:13.620687+0000","last_clean_scrub_stamp":"2023-11-24T07:59:13.620687+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T15:26:05.128703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.8","version":"0'0","reported_seq":12,"reported_epoch":73,"state":"active+clean","last_fresh":"2023-11-24T07:59:14.635324+0000","last_change":"2023-11-24T07:59:13.641509+0000","last_active":"2023-11-24T07:59:14.635324+0000","last_peered":"2023-11-24T07:59:14.635324+0000","last_clean":"2023-11-24T07:59:14.635324+0000","last_became_active":"2023-11-24T07:59:13.641197+0000","last_became_peered":"2023-11-24T07:59:13.641197+0000","last_unstale":"2023-11-24T07:59:14.635324+0000","last_undegraded":"2023-11-24T07:59:14.635324+0000","last_fullsized":"2023-11-24T07:59:14.635324+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:59:12.615544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:59:12.615544+0000","last_clean_scrub_stamp":"2023-11-24T07:59:12.615544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T17:23:58.604523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.6","version":"0'0","reported_seq":11,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-11-24T07:59:15.650866+0000","last_change":"2023-11-24T07:59:14.641587+0000","last_active":"2023-11-24T07:59:15.650866+0000","last_peered":"2023-11-24T07:59:15.650866+0000","last_clean":"2023-11-24T07:59:15.650866+0000","last_became_active":"2023-11-24T07:59:14.641303+0000","last_became_peered":"2023-11-24T07:59:14.641303+0000","last_unstale":"2023-11-24T07:59:15.650866+0000","last_undegraded":"2023-11-24T07:59:15.650866+0000","last_fullsized":"2023-11-24T07:59:15.650866+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:59:13.620687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:59:13.620687+0000","last_clean_scrub_stamp":"2023-11-24T07:59:13.620687+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T15:14:11.003165+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.7","version":"73'2","reported_seq":14,"reported_epoch":73,"state":"active+clean","last_fresh":"2023-11-24T07:59:14.663660+0000","last_change":"2023-11-24T07:59:13.652389+0000","last_active":"2023-11-24T07:59:14.663660+0000","last_peered":"2023-11-24T07:59:14.663660+0000","last_clean":"2023-11-24T07:59:14.663660+0000","last_became_active":"2023-11-24T07:59:13.652073+0000","last_became_peered":"2023-11-24T07:59:13.652073+0000","last_unstale":"2023-11-24T07:59:14.663660+0000","last_undegraded":"2023-11-24T07:59:14.663660+0000","last_fullsized":"2023-11-24T07:59:14.663660+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:59:12.615544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:59:12.615544+0000","last_clean_scrub_stamp":"2023-11-24T07:59:12.615544+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T16:26:10.524076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.7","version":"0'0","reported_seq":11,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-11-24T07:59:16.292672+0000","last_change":"2023-11-24T07:59:14.643008+0000","last_active":"2023-11-24T07:59:16.292672+0000","last_peered":"2023-11-24T07:59:16.292672+0000","last_clean":"2023-11-24T07:59:16.292672+0000","last_became_active":"2023-11-24T07:59:14.642769+0000","last_became_peered":"2023-11-24T07:59:14.642769+0000","last_unstale":"2023-11-24T07:59:16.292672+0000","last_undegraded":"2023-11-24T07:59:16.292672+0000","last_fullsized":"2023-11-24T07:59:16.292672+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:59:13.620687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:59:13.620687+0000","last_clean_scrub_stamp":"2023-11-24T07:59:13.620687+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T18:08:53.846914+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.6","version":"0'0","reported_seq":12,"reported_epoch":73,"state":"active+clean","last_fresh":"2023-11-24T07:59:14.650495+0000","last_change":"2023-11-24T07:59:13.646769+0000","last_active":"2023-11-24T07:59:14.650495+0000","last_peered":"2023-11-24T07:59:14.650495+0000","last_clean":"2023-11-24T07:59:14.650495+0000","last_became_active":"2023-11-24T07:59:13.646438+0000","last_became_peered":"2023-11-24T07:59:13.646438+0000","last_unstale":"2023-11-24T07:59:14.650495+0000","last_undegraded":"2023-11-24T07:59:14.650495+0000","last_fullsized":"2023-11-24T07:59:14.650495+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:59:12.615544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:59:12.615544+0000","last_clean_scrub_stamp":"2023-11-24T07:59:12.615544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T09:33:05.362263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.4","version":"0'0","reported_seq":11,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-11-24T07:59:16.294066+0000","last_change":"2023-11-24T07:59:14.660249+0000","last_active":"2023-11-24T07:59:16.294066+0000","last_peered":"2023-11-24T07:59:16.294066+0000","last_clean":"2023-11-24T07:59:16.294066+0000","last_became_active":"2023-11-24T07:59:14.659550+0000","last_became_peered":"2023-11-24T07:59:14.659550+0000","last_unstale":"2023-11-24T07:59:16.294066+0000","last_undegraded":"2023-11-24T07:59:16.294066+0000","last_fullsized":"2023-11-24T07:59:16.294066+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:59:13.620687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:59:13.620687+0000","last_clean_scrub_stamp":"2023-11-24T07:59:13.620687+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T16:41:02.397331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.5","version":"0'0","reported_seq":12,"reported_epoch":73,"state":"active+clean","last_fresh":"2023-11-24T07:59:14.644674+0000","last_change":"2023-11-24T07:59:13.647782+0000","last_active":"2023-11-24T07:59:14.644674+0000","last_peered":"2023-11-24T07:59:14.644674+0000","last_clean":"2023-11-24T07:59:14.644674+0000","last_became_active":"2023-11-24T07:59:13.647473+0000","last_became_peered":"2023-11-24T07:59:13.647473+0000","last_unstale":"2023-11-24T07:59:14.644674+0000","last_undegraded":"2023-11-24T07:59:14.644674+0000","last_fullsized":"2023-11-24T07:59:14.644674+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:59:12.615544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:59:12.615544+0000","last_clean_scrub_stamp":"2023-11-24T07:59:12.615544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T09:55:49.599221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.5","version":"0'0","reported_seq":11,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-11-24T07:59:16.292638+0000","last_change":"2023-11-24T07:59:14.641861+0000","last_active":"2023-11-24T07:59:16.292638+0000","last_peered":"2023-11-24T07:59:16.292638+0000","last_clean":"2023-11-24T07:59:16.292638+0000","last_became_active":"2023-11-24T07:59:14.641558+0000","last_became_peered":"2023-11-24T07:59:14.641558+0000","last_unstale":"2023-11-24T07:59:16.292638+0000","last_undegraded":"2023-11-24T07:59:16.292638+0000","last_fullsized":"2023-11-24T07:59:16.292638+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:59:13.620687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:59:13.620687+0000","last_clean_scrub_stamp":"2023-11-24T07:59:13.620687+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T12:33:01.714569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.4","version":"0'0","reported_seq":12,"reported_epoch":73,"state":"active+clean","last_fresh":"2023-11-24T07:59:14.643365+0000","last_change":"2023-11-24T07:59:13.645153+0000","last_active":"2023-11-24T07:59:14.643365+0000","last_peered":"2023-11-24T07:59:14.643365+0000","last_clean":"2023-11-24T07:59:14.643365+0000","last_became_active":"2023-11-24T07:59:13.644868+0000","last_became_peered":"2023-11-24T07:59:13.644868+0000","last_unstale":"2023-11-24T07:59:14.643365+0000","last_undegraded":"2023-11-24T07:59:14.643365+0000","last_fullsized":"2023-11-24T07:59:14.643365+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:59:12.615544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:59:12.615544+0000","last_clean_scrub_stamp":"2023-11-24T07:59:12.615544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T19:37:03.054479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.2","version":"0'0","reported_seq":11,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-11-24T07:59:15.650594+0000","last_change":"2023-11-24T07:59:14.640912+0000","last_active":"2023-11-24T07:59:15.650594+0000","last_peered":"2023-11-24T07:59:15.650594+0000","last_clean":"2023-11-24T07:59:15.650594+0000","last_became_active":"2023-11-24T07:59:14.640633+0000","last_became_peered":"2023-11-24T07:59:14.640633+0000","last_unstale":"2023-11-24T07:59:15.650594+0000","last_undegraded":"2023-11-24T07:59:15.650594+0000","last_fullsized":"2023-11-24T07:59:15.650594+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:59:13.620687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:59:13.620687+0000","last_clean_scrub_stamp":"2023-11-24T07:59:13.620687+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T08:09:53.053268+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.3","version":"0'0","reported_seq":12,"reported_epoch":73,"state":"active+clean","last_fresh":"2023-11-24T07:59:14.642373+0000","last_change":"2023-11-24T07:59:13.655482+0000","last_active":"2023-11-24T07:59:14.642373+0000","last_peered":"2023-11-24T07:59:14.642373+0000","last_clean":"2023-11-24T07:59:14.642373+0000","last_became_active":"2023-11-24T07:59:13.655191+0000","last_became_peered":"2023-11-24T07:59:13.655191+0000","last_unstale":"2023-11-24T07:59:14.642373+0000","last_undegraded":"2023-11-24T07:59:14.642373+0000","last_fullsized":"2023-11-24T07:59:14.642373+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:59:12.615544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:59:12.615544+0000","last_clean_scrub_stamp":"2023-11-24T07:59:12.615544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T12:01:57.940430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.3","version":"0'0","reported_seq":11,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-11-24T07:59:16.292441+0000","last_change":"2023-11-24T07:59:14.669887+0000","last_active":"2023-11-24T07:59:16.292441+0000","last_peered":"2023-11-24T07:59:16.292441+0000","last_clean":"2023-11-24T07:59:16.292441+0000","last_became_active":"2023-11-24T07:59:14.669606+0000","last_became_peered":"2023-11-24T07:59:14.669606+0000","last_unstale":"2023-11-24T07:59:16.292441+0000","last_undegraded":"2023-11-24T07:59:16.292441+0000","last_fullsized":"2023-11-24T07:59:16.292441+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:59:13.620687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:59:13.620687+0000","last_clean_scrub_stamp":"2023-11-24T07:59:13.620687+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:09:08.315869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.2","version":"0'0","reported_seq":12,"reported_epoch":73,"state":"active+clean","last_fresh":"2023-11-24T07:59:14.634770+0000","last_change":"2023-11-24T07:59:13.649750+0000","last_active":"2023-11-24T07:59:14.634770+0000","last_peered":"2023-11-24T07:59:14.634770+0000","last_clean":"2023-11-24T07:59:14.634770+0000","last_became_active":"2023-11-24T07:59:13.649437+0000","last_became_peered":"2023-11-24T07:59:13.649437+0000","last_unstale":"2023-11-24T07:59:14.634770+0000","last_undegraded":"2023-11-24T07:59:14.634770+0000","last_fullsized":"2023-11-24T07:59:14.634770+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:59:12.615544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:59:12.615544+0000","last_clean_scrub_stamp":"2023-11-24T07:59:12.615544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:52:45.085545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.0","version":"0'0","reported_seq":11,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-11-24T07:59:15.648370+0000","last_change":"2023-11-24T07:59:14.667968+0000","last_active":"2023-11-24T07:59:15.648370+0000","last_peered":"2023-11-24T07:59:15.648370+0000","last_clean":"2023-11-24T07:59:15.648370+0000","last_became_active":"2023-11-24T07:59:14.667776+0000","last_became_peered":"2023-11-24T07:59:14.667776+0000","last_unstale":"2023-11-24T07:59:15.648370+0000","last_undegraded":"2023-11-24T07:59:15.648370+0000","last_fullsized":"2023-11-24T07:59:15.648370+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:59:13.620687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:59:13.620687+0000","last_clean_scrub_stamp":"2023-11-24T07:59:13.620687+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T09:55:50.604364+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.1","version":"0'0","reported_seq":12,"reported_epoch":73,"state":"active+clean","last_fresh":"2023-11-24T07:59:14.647065+0000","last_change":"2023-11-24T07:59:13.647623+0000","last_active":"2023-11-24T07:59:14.647065+0000","last_peered":"2023-11-24T07:59:14.647065+0000","last_clean":"2023-11-24T07:59:14.647065+0000","last_became_active":"2023-11-24T07:59:13.647322+0000","last_became_peered":"2023-11-24T07:59:13.647322+0000","last_unstale":"2023-11-24T07:59:14.647065+0000","last_undegraded":"2023-11-24T07:59:14.647065+0000","last_fullsized":"2023-11-24T07:59:14.647065+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:59:12.615544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:59:12.615544+0000","last_clean_scrub_stamp":"2023-11-24T07:59:12.615544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T09:55:49.599221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1","version":"0'0","reported_seq":11,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-11-24T07:59:16.293522+0000","last_change":"2023-11-24T07:59:14.659942+0000","last_active":"2023-11-24T07:59:16.293522+0000","last_peered":"2023-11-24T07:59:16.293522+0000","last_clean":"2023-11-24T07:59:16.293522+0000","last_became_active":"2023-11-24T07:59:14.659603+0000","last_became_peered":"2023-11-24T07:59:14.659603+0000","last_unstale":"2023-11-24T07:59:16.293522+0000","last_undegraded":"2023-11-24T07:59:16.293522+0000","last_fullsized":"2023-11-24T07:59:16.293522+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:59:13.620687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:59:13.620687+0000","last_clean_scrub_stamp":"2023-11-24T07:59:13.620687+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T14:38:55.028542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.0","version":"0'0","reported_seq":12,"reported_epoch":73,"state":"active+clean","last_fresh":"2023-11-24T07:59:14.635327+0000","last_change":"2023-11-24T07:59:13.647308+0000","last_active":"2023-11-24T07:59:14.635327+0000","last_peered":"2023-11-24T07:59:14.635327+0000","last_clean":"2023-11-24T07:59:14.635327+0000","last_became_active":"2023-11-24T07:59:13.646512+0000","last_became_peered":"2023-11-24T07:59:13.646512+0000","last_unstale":"2023-11-24T07:59:14.635327+0000","last_undegraded":"2023-11-24T07:59:14.635327+0000","last_fullsized":"2023-11-24T07:59:14.635327+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:59:12.615544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:59:12.615544+0000","last_clean_scrub_stamp":"2023-11-24T07:59:12.615544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T15:47:39.085557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.a","version":"0'0","reported_seq":11,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-11-24T07:59:15.650713+0000","last_change":"2023-11-24T07:59:14.661867+0000","last_active":"2023-11-24T07:59:15.650713+0000","last_peered":"2023-11-24T07:59:15.650713+0000","last_clean":"2023-11-24T07:59:15.650713+0000","last_became_active":"2023-11-24T07:59:14.661579+0000","last_became_peered":"2023-11-24T07:59:14.661579+0000","last_unstale":"2023-11-24T07:59:15.650713+0000","last_undegraded":"2023-11-24T07:59:15.650713+0000","last_fullsized":"2023-11-24T07:59:15.650713+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:59:13.620687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:59:13.620687+0000","last_clean_scrub_stamp":"2023-11-24T07:59:13.620687+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T17:42:55.832290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.b","version":"73'2","reported_seq":14,"reported_epoch":73,"state":"active+clean","last_fresh":"2023-11-24T07:59:14.656594+0000","last_change":"2023-11-24T07:59:13.650073+0000","last_active":"2023-11-24T07:59:14.656594+0000","last_peered":"2023-11-24T07:59:14.656594+0000","last_clean":"2023-11-24T07:59:14.656594+0000","last_became_active":"2023-11-24T07:59:13.649748+0000","last_became_peered":"2023-11-24T07:59:13.649748+0000","last_unstale":"2023-11-24T07:59:14.656594+0000","last_undegraded":"2023-11-24T07:59:14.656594+0000","last_fullsized":"2023-11-24T07:59:14.656594+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:59:12.615544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:59:12.615544+0000","last_clean_scrub_stamp":"2023-11-24T07:59:12.615544+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T13:35:56.042823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":211,"reported_epoch":73,"state":"active+clean","last_fresh":"2023-11-24T07:59:14.634733+0000","last_change":"2023-11-24T07:56:19.668905+0000","last_active":"2023-11-24T07:59:14.634733+0000","last_peered":"2023-11-24T07:59:14.634733+0000","last_clean":"2023-11-24T07:59:14.634733+0000","last_became_active":"2023-11-24T07:56:19.668601+0000","last_became_peered":"2023-11-24T07:56:19.668601+0000","last_unstale":"2023-11-24T07:59:14.634733+0000","last_undegraded":"2023-11-24T07:59:14.634733+0000","last_fullsized":"2023-11-24T07:59:14.634733+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:56:18.657844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:56:18.657844+0000","last_clean_scrub_stamp":"2023-11-24T07:56:18.657844+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T11:51:53.828455+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":11,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":10,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574887,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27912,"kb_used_data":892,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343928,"statfs":{"total":96636764160,"available":96608182272,"internally_reserved":0,"allocated":913408,"data_stored":385744,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 07:58:20 2023","interfaces":[{"interface":"back","average":{"1min":0.51400000000000001,"5min":0.54800000000000004,"15min":0.55400000000000005},"min":{"1min":0.309,"5min":0.309,"15min":0.309},"max":{"1min":0.78500000000000003,"5min":1.413,"15min":1.413},"last":0.27600000000000002},{"interface":"front","average":{"1min":0.56299999999999994,"5min":0.55500000000000005,"15min":0.55400000000000005},"min":{"1min":0.436,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":0.77000000000000002,"5min":0.81000000000000005,"15min":0.81000000000000005},"last":0.38300000000000001}]},{"osd":1,"last update":"Fri Nov 24 07:58:20 2023","interfaces":[{"interface":"back","average":{"1min":0.501,"5min":0.77900000000000003,"15min":0.82499999999999996},"min":{"1min":0.32000000000000001,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.69899999999999995,"5min":6.1070000000000002,"15min":6.1070000000000002},"last":0.438},{"interface":"front","average":{"1min":0.51300000000000001,"5min":0.56599999999999995,"15min":0.57499999999999996},"min":{"1min":0.40699999999999997,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.73599999999999999,"5min":1.5920000000000001,"15min":1.5920000000000001},"last":0.47999999999999998}]},{"osd":2,"last update":"Fri Nov 24 07:58:20 2023","interfaces":[{"interface":"back","average":{"1min":0.498,"5min":0.60099999999999998,"15min":0.61799999999999999},"min":{"1min":0.35099999999999998,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.71599999999999997,"5min":1.613,"15min":1.613},"last":0.41899999999999998},{"interface":"front","average":{"1min":0.502,"5min":0.73399999999999999,"15min":0.77200000000000002},"min":{"1min":0.34899999999999998,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.71299999999999997,"5min":4.5220000000000002,"15min":4.5220000000000002},"last":0.45100000000000001}]},{"osd":3,"last update":"Fri Nov 24 07:58:20 2023","interfaces":[{"interface":"back","average":{"1min":0.55100000000000005,"5min":0.755,"15min":0.78900000000000003},"min":{"1min":0.41699999999999998,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.754,"5min":4.6760000000000002,"15min":4.6760000000000002},"last":0.40699999999999997},{"interface":"front","average":{"1min":0.60699999999999998,"5min":0.875,"15min":0.92000000000000004},"min":{"1min":0.38500000000000001,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.81599999999999995,"5min":4.6539999999999999,"15min":4.6539999999999999},"last":0.53000000000000003}]},{"osd":4,"last update":"Fri Nov 24 07:58:20 2023","interfaces":[{"interface":"back","average":{"1min":0.54900000000000004,"5min":0.56399999999999995,"15min":0.56599999999999995},"min":{"1min":0.41799999999999998,"5min":0.378,"15min":0.378},"max":{"1min":0.70999999999999996,"5min":0.84699999999999998,"15min":0.84699999999999998},"last":0.36399999999999999},{"interface":"front","average":{"1min":0.54700000000000004,"5min":0.64300000000000002,"15min":0.65900000000000003},"min":{"1min":0.35199999999999998,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.79900000000000004,"5min":1.629,"15min":1.629},"last":0.57299999999999995}]},{"osd":5,"last update":"Fri Nov 24 07:58:20 2023","interfaces":[{"interface":"back","average":{"1min":0.54200000000000004,"5min":0.54600000000000004,"15min":0.54600000000000004},"min":{"1min":0.41599999999999998,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.67900000000000005,"5min":0.875,"15min":0.875},"last":0.315},{"interface":"front","average":{"1min":0.57799999999999996,"5min":0.61399999999999999,"15min":0.62},"min":{"1min":0.36799999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.77700000000000002,"5min":0.90900000000000003,"15min":0.90900000000000003},"last":0.499}]},{"osd":6,"last update":"Fri Nov 24 07:58:20 2023","interfaces":[{"interface":"back","average":{"1min":0.59999999999999998,"5min":0.60799999999999998,"15min":0.60999999999999999},"min":{"1min":0.49099999999999999,"5min":0.438,"15min":0.438},"max":{"1min":0.73399999999999999,"5min":0.80800000000000005,"15min":0.80800000000000005},"last":0.51400000000000001},{"interface":"front","average":{"1min":0.63900000000000001,"5min":0.66300000000000003,"15min":0.66700000000000004},"min":{"1min":0.45600000000000002,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":0.79700000000000004,"5min":0.98399999999999999,"15min":0.98399999999999999},"last":0.46800000000000003}]}]},{"osd":6,"up_from":13,"seq":55834574888,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27924,"kb_used_data":896,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343916,"statfs":{"total":96636764160,"available":96608169984,"internally_reserved":0,"allocated":917504,"data_stored":385778,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 07:58:19 2023","interfaces":[{"interface":"back","average":{"1min":0.53300000000000003,"5min":0.54600000000000004,"15min":0.54800000000000004},"min":{"1min":0.29799999999999999,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":0.93100000000000005,"5min":0.93100000000000005,"15min":0.93100000000000005},"last":0.60399999999999998},{"interface":"front","average":{"1min":0.51600000000000001,"5min":0.51600000000000001,"15min":0.51600000000000001},"min":{"1min":0.311,"5min":0.311,"15min":0.311},"max":{"1min":0.82199999999999995,"5min":0.82199999999999995,"15min":0.82199999999999995},"last":0.66400000000000003}]},{"osd":1,"last update":"Fri Nov 24 07:58:19 2023","interfaces":[{"interface":"back","average":{"1min":0.51900000000000002,"5min":0.51300000000000001,"15min":0.51200000000000001},"min":{"1min":0.35599999999999998,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.81299999999999994,"5min":0.81299999999999994,"15min":0.81299999999999994},"last":0.61599999999999999},{"interface":"front","average":{"1min":0.54000000000000004,"5min":0.59799999999999998,"15min":0.60799999999999998},"min":{"1min":0.35999999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.82799999999999996,"5min":1.0069999999999999,"15min":1.0069999999999999},"last":0.70799999999999996}]},{"osd":2,"last update":"Fri Nov 24 07:58:19 2023","interfaces":[{"interface":"back","average":{"1min":0.58399999999999996,"5min":0.57599999999999996,"15min":0.57499999999999996},"min":{"1min":0.34899999999999998,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.90600000000000003,"5min":0.90600000000000003,"15min":0.90600000000000003},"last":0.69599999999999995},{"interface":"front","average":{"1min":0.56499999999999995,"5min":0.58699999999999997,"15min":0.59099999999999997},"min":{"1min":0.34300000000000003,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.84899999999999998,"5min":0.84899999999999998,"15min":0.84899999999999998},"last":0.68300000000000005}]},{"osd":3,"last update":"Fri Nov 24 07:58:19 2023","interfaces":[{"interface":"back","average":{"1min":0.56000000000000005,"5min":0.56599999999999995,"15min":0.56699999999999995},"min":{"1min":0.38400000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.86499999999999999,"5min":0.876,"15min":0.876},"last":0.58899999999999997},{"interface":"front","average":{"1min":0.57699999999999996,"5min":0.56000000000000005,"15min":0.55700000000000005},"min":{"1min":0.39800000000000002,"5min":0.379,"15min":0.379},"max":{"1min":0.876,"5min":0.876,"15min":0.876},"last":0.72599999999999998}]},{"osd":4,"last update":"Fri Nov 24 07:58:19 2023","interfaces":[{"interface":"back","average":{"1min":0.53100000000000003,"5min":0.51900000000000002,"15min":0.51700000000000002},"min":{"1min":0.25,"5min":0.25,"15min":0.25},"max":{"1min":0.89400000000000002,"5min":0.89400000000000002,"15min":0.89400000000000002},"last":0.501},{"interface":"front","average":{"1min":0.58899999999999997,"5min":0.59899999999999998,"15min":0.60099999999999998},"min":{"1min":0.29099999999999998,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":1.044,"5min":1.087,"15min":1.087},"last":0.45900000000000002}]},{"osd":5,"last update":"Fri Nov 24 07:58:19 2023","interfaces":[{"interface":"back","average":{"1min":0.59499999999999997,"5min":0.54300000000000004,"15min":0.53500000000000003},"min":{"1min":0.27800000000000002,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":0.97999999999999998,"5min":0.97999999999999998,"15min":0.97999999999999998},"last":0.47899999999999998},{"interface":"front","average":{"1min":0.61399999999999999,"5min":0.61199999999999999,"15min":0.61199999999999999},"min":{"1min":0.31900000000000001,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":1.0009999999999999,"5min":1.0009999999999999,"15min":1.0009999999999999},"last":0.54700000000000004}]},{"osd":7,"last update":"Fri Nov 24 07:58:19 2023","interfaces":[{"interface":"back","average":{"1min":0.61799999999999999,"5min":0.61799999999999999,"15min":0.61799999999999999},"min":{"1min":0.26500000000000001,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":1.0880000000000001,"5min":1.0880000000000001,"15min":1.0880000000000001},"last":0.52500000000000002},{"interface":"front","average":{"1min":0.65200000000000002,"5min":0.66600000000000004,"15min":0.66800000000000004},"min":{"1min":0.34300000000000003,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.95599999999999996,"5min":0.95599999999999996,"15min":0.95599999999999996},"last":0.57099999999999995}]}]},{"osd":1,"up_from":13,"seq":55834574889,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28480,"kb_used_data":1460,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343360,"statfs":{"total":96636764160,"available":96607600640,"internally_reserved":0,"allocated":1495040,"data_stored":974500,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 07:58:24 2023","interfaces":[{"interface":"back","average":{"1min":0.33000000000000002,"5min":0.35699999999999998,"15min":0.36099999999999999},"min":{"1min":0.249,"5min":0.20200000000000001,"15min":0.20200000000000001},"max":{"1min":0.48799999999999999,"5min":0.70899999999999996,"15min":0.70899999999999996},"last":0.28699999999999998},{"interface":"front","average":{"1min":0.36399999999999999,"5min":0.39100000000000001,"15min":0.39500000000000002},"min":{"1min":0.23599999999999999,"5min":0.214,"15min":0.214},"max":{"1min":0.54700000000000004,"5min":0.82499999999999996,"15min":0.82499999999999996},"last":0.34999999999999998}]},{"osd":2,"last update":"Fri Nov 24 07:58:24 2023","interfaces":[{"interface":"back","average":{"1min":0.312,"5min":0.35999999999999999,"15min":0.36799999999999999},"min":{"1min":0.253,"5min":0.22700000000000001,"15min":0.22700000000000001},"max":{"1min":0.44800000000000001,"5min":0.60999999999999999,"15min":0.60999999999999999},"last":0.311},{"interface":"front","average":{"1min":0.36499999999999999,"5min":0.40300000000000002,"15min":0.40899999999999997},"min":{"1min":0.28100000000000003,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":0.47599999999999998,"5min":0.621,"15min":0.621},"last":0.33000000000000002}]},{"osd":3,"last update":"Fri Nov 24 07:58:24 2023","interfaces":[{"interface":"back","average":{"1min":0.40300000000000002,"5min":0.435,"15min":0.441},"min":{"1min":0.31900000000000001,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.59199999999999997,"5min":0.66500000000000004,"15min":0.66500000000000004},"last":0.373},{"interface":"front","average":{"1min":0.38700000000000001,"5min":0.41899999999999998,"15min":0.42499999999999999},"min":{"1min":0.29199999999999998,"5min":0.28499999999999998,"15min":0.28499999999999998},"max":{"1min":0.56200000000000006,"5min":0.76300000000000001,"15min":0.76300000000000001},"last":0.41199999999999998}]},{"osd":4,"last update":"Fri Nov 24 07:58:24 2023","interfaces":[{"interface":"back","average":{"1min":0.56200000000000006,"5min":0.60099999999999998,"15min":0.60699999999999998},"min":{"1min":0.442,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":0.84099999999999997,"5min":0.84099999999999997,"15min":0.84099999999999997},"last":0.52400000000000002},{"interface":"front","average":{"1min":0.61099999999999999,"5min":0.60699999999999998,"15min":0.60599999999999998},"min":{"1min":0.499,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":0.79500000000000004,"5min":0.82199999999999995,"15min":0.82199999999999995},"last":0.63800000000000001}]},{"osd":5,"last update":"Fri Nov 24 07:58:24 2023","interfaces":[{"interface":"back","average":{"1min":0.59199999999999997,"5min":0.69799999999999995,"15min":0.71599999999999997},"min":{"1min":0.439,"5min":0.37,"15min":0.37},"max":{"1min":0.97599999999999998,"5min":2.7269999999999999,"15min":2.7269999999999999},"last":0.67000000000000004},{"interface":"front","average":{"1min":0.57699999999999996,"5min":0.63100000000000001,"15min":0.64000000000000001},"min":{"1min":0.39100000000000001,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.94499999999999995,"5min":0.94499999999999995,"15min":0.94499999999999995},"last":0.56299999999999994}]},{"osd":6,"last update":"Fri Nov 24 07:58:24 2023","interfaces":[{"interface":"back","average":{"1min":0.61199999999999999,"5min":0.64800000000000002,"15min":0.65400000000000003},"min":{"1min":0.47699999999999998,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":0.77500000000000002,"5min":0.98199999999999998,"15min":0.98199999999999998},"last":0.51100000000000001},{"interface":"front","average":{"1min":0.57599999999999996,"5min":0.60799999999999998,"15min":0.61399999999999999},"min":{"1min":0.40600000000000003,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":0.89600000000000002,"5min":0.90700000000000003,"15min":0.90700000000000003},"last":0.61299999999999999}]},{"osd":7,"last update":"Fri Nov 24 07:58:24 2023","interfaces":[{"interface":"back","average":{"1min":0.63100000000000001,"5min":0.63500000000000001,"15min":0.63600000000000001},"min":{"1min":0.43099999999999999,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.79800000000000004,"5min":1.0649999999999999,"15min":1.0649999999999999},"last":0.53900000000000003},{"interface":"front","average":{"1min":0.57099999999999995,"5min":0.65000000000000002,"15min":0.66300000000000003},"min":{"1min":0.45700000000000002,"5min":0.442,"15min":0.442},"max":{"1min":0.71299999999999997,"5min":0.88800000000000001,"15min":0.88800000000000001},"last":0.65600000000000003}]}]},{"osd":0,"up_from":13,"seq":55834574889,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27904,"kb_used_data":884,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343936,"statfs":{"total":96636764160,"available":96608190464,"internally_reserved":0,"allocated":905216,"data_stored":384202,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Nov 24 07:58:20 2023","interfaces":[{"interface":"back","average":{"1min":0.38500000000000001,"5min":0.64700000000000002,"15min":0.69099999999999995},"min":{"1min":0.21299999999999999,"5min":0.21299999999999999,"15min":0.21299999999999999},"max":{"1min":0.625,"5min":6.3550000000000004,"15min":6.3550000000000004},"last":0.45500000000000002},{"interface":"front","average":{"1min":0.40799999999999997,"5min":1.464,"15min":1.6399999999999999},"min":{"1min":0.26000000000000001,"5min":0.26000000000000001,"15min":0.26000000000000001},"max":{"1min":0.60999999999999999,"5min":22.916,"15min":22.916},"last":0.44}]},{"osd":2,"last update":"Fri Nov 24 07:58:20 2023","interfaces":[{"interface":"back","average":{"1min":0.35299999999999998,"5min":0.63,"15min":0.67600000000000005},"min":{"1min":0.22900000000000001,"5min":0.22800000000000001,"15min":0.22800000000000001},"max":{"1min":0.499,"5min":6.3899999999999997,"15min":6.3899999999999997},"last":0.42199999999999999},{"interface":"front","average":{"1min":0.39500000000000002,"5min":0.92900000000000005,"15min":1.018},"min":{"1min":0.27900000000000003,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":0.60199999999999998,"5min":11.93,"15min":11.93},"last":0.36699999999999999}]},{"osd":3,"last update":"Fri Nov 24 07:58:20 2023","interfaces":[{"interface":"back","average":{"1min":0.373,"5min":0.40999999999999998,"15min":0.41599999999999998},"min":{"1min":0.23799999999999999,"5min":0.23799999999999999,"15min":0.23799999999999999},"max":{"1min":0.79300000000000004,"5min":1.155,"15min":1.155},"last":0.503},{"interface":"front","average":{"1min":0.438,"5min":0.98899999999999999,"15min":1.081},"min":{"1min":0.30199999999999999,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.69699999999999995,"5min":12.116,"15min":12.116},"last":0.38800000000000001}]},{"osd":4,"last update":"Fri Nov 24 07:58:20 2023","interfaces":[{"interface":"back","average":{"1min":0.626,"5min":0.64300000000000002,"15min":0.64600000000000002},"min":{"1min":0.40100000000000002,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":1.044,"5min":1.044,"15min":1.044},"last":0.67900000000000005},{"interface":"front","average":{"1min":0.64300000000000002,"5min":1.675,"15min":1.847},"min":{"1min":0.35699999999999998,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.93000000000000005,"5min":22.963999999999999,"15min":22.963999999999999},"last":0.54300000000000004}]},{"osd":5,"last update":"Fri Nov 24 07:58:20 2023","interfaces":[{"interface":"back","average":{"1min":0.65200000000000002,"5min":1.1639999999999999,"15min":1.25},"min":{"1min":0.41999999999999998,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.97799999999999998,"5min":12.084,"15min":12.084},"last":0.63300000000000001},{"interface":"front","average":{"1min":0.65800000000000003,"5min":0.67400000000000004,"15min":0.67600000000000005},"min":{"1min":0.39700000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.95499999999999996,"5min":1.073,"15min":1.073},"last":0.55400000000000005}]},{"osd":6,"last update":"Fri Nov 24 07:58:20 2023","interfaces":[{"interface":"back","average":{"1min":0.65200000000000002,"5min":0.68300000000000005,"15min":0.68799999999999994},"min":{"1min":0.45000000000000001,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":0.94699999999999995,"5min":1.0580000000000001,"15min":1.0580000000000001},"last":0.64500000000000002},{"interface":"front","average":{"1min":0.67000000000000004,"5min":1.1850000000000001,"15min":1.2709999999999999},"min":{"1min":0.46200000000000002,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":0.95899999999999996,"5min":12.045,"15min":12.045},"last":0.49299999999999999}]},{"osd":7,"last update":"Fri Nov 24 07:58:20 2023","interfaces":[{"interface":"back","average":{"1min":0.67000000000000004,"5min":0.65800000000000003,"15min":0.65600000000000003},"min":{"1min":0.45200000000000001,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":0.98299999999999998,"5min":1.038,"15min":1.038},"last":0.52300000000000002},{"interface":"front","average":{"1min":0.67500000000000004,"5min":1.7050000000000001,"15min":1.877},"min":{"1min":0.41299999999999998,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":1.0069999999999999,"5min":22.995000000000001,"15min":22.995000000000001},"last":0.59599999999999997}]}]},{"osd":2,"up_from":13,"seq":55834574889,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27904,"kb_used_data":884,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343936,"statfs":{"total":96636764160,"available":96608190464,"internally_reserved":0,"allocated":905216,"data_stored":384714,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 07:58:19 2023","interfaces":[{"interface":"back","average":{"1min":0.36399999999999999,"5min":0.36899999999999999,"15min":0.37},"min":{"1min":0.219,"5min":0.219,"15min":0.219},"max":{"1min":0.55900000000000005,"5min":0.58099999999999996,"15min":0.58099999999999996},"last":0.38300000000000001},{"interface":"front","average":{"1min":0.38900000000000001,"5min":0.39900000000000002,"15min":0.40100000000000002},"min":{"1min":0.29299999999999998,"5min":0.27400000000000002,"15min":0.27400000000000002},"max":{"1min":0.54100000000000004,"5min":0.54100000000000004,"15min":0.54100000000000004},"last":0.39300000000000002}]},{"osd":1,"last update":"Fri Nov 24 07:58:19 2023","interfaces":[{"interface":"back","average":{"1min":0.34699999999999998,"5min":0.371,"15min":0.375},"min":{"1min":0.22,"5min":0.22,"15min":0.22},"max":{"1min":0.54000000000000004,"5min":0.59199999999999997,"15min":0.59199999999999997},"last":0.41699999999999998},{"interface":"front","average":{"1min":0.36599999999999999,"5min":0.40600000000000003,"15min":0.41199999999999998},"min":{"1min":0.28299999999999997,"5min":0.25600000000000001,"15min":0.25600000000000001},"max":{"1min":0.497,"5min":0.627,"15min":0.627},"last":0.29199999999999998}]},{"osd":3,"last update":"Fri Nov 24 07:58:19 2023","interfaces":[{"interface":"back","average":{"1min":0.40500000000000003,"5min":0.40899999999999997,"15min":0.40999999999999998},"min":{"1min":0.30399999999999999,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":0.64200000000000002,"5min":0.64200000000000002,"15min":0.64200000000000002},"last":0.36699999999999999},{"interface":"front","average":{"1min":0.41699999999999998,"5min":0.41399999999999998,"15min":0.41399999999999998},"min":{"1min":0.28599999999999998,"5min":0.27500000000000002,"15min":0.27500000000000002},"max":{"1min":0.59299999999999997,"5min":0.64600000000000002,"15min":0.64600000000000002},"last":0.35199999999999998}]},{"osd":4,"last update":"Fri Nov 24 07:58:19 2023","interfaces":[{"interface":"back","average":{"1min":0.54800000000000004,"5min":0.59599999999999997,"15min":0.60399999999999998},"min":{"1min":0.38700000000000001,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.75,"5min":0.81599999999999995,"15min":0.81599999999999995},"last":0.55700000000000005},{"interface":"front","average":{"1min":0.55500000000000005,"5min":0.59099999999999997,"15min":0.59699999999999998},"min":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":0.83799999999999997,"5min":0.83799999999999997,"15min":0.83799999999999997},"last":0.68100000000000005}]},{"osd":5,"last update":"Fri Nov 24 07:58:19 2023","interfaces":[{"interface":"back","average":{"1min":0.56000000000000005,"5min":0.60799999999999998,"15min":0.61599999999999999},"min":{"1min":0.434,"5min":0.434,"15min":0.434},"max":{"1min":0.73499999999999999,"5min":0.84899999999999998,"15min":0.84899999999999998},"last":0.56899999999999995},{"interface":"front","average":{"1min":0.624,"5min":0.69599999999999995,"15min":0.70799999999999996},"min":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":0.82499999999999996,"5min":2.1699999999999999,"15min":2.1699999999999999},"last":0.61199999999999999}]},{"osd":6,"last update":"Fri Nov 24 07:58:19 2023","interfaces":[{"interface":"back","average":{"1min":0.60499999999999998,"5min":0.65400000000000003,"15min":0.66200000000000003},"min":{"1min":0.42499999999999999,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":0.878,"5min":0.878,"15min":0.878},"last":0.67000000000000004},{"interface":"front","average":{"1min":0.60599999999999998,"5min":0.65800000000000003,"15min":0.66600000000000004},"min":{"1min":0.41899999999999998,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":0.80300000000000005,"5min":0.88500000000000001,"15min":0.88500000000000001},"last":0.59199999999999997}]},{"osd":7,"last update":"Fri Nov 24 07:58:19 2023","interfaces":[{"interface":"back","average":{"1min":0.60399999999999998,"5min":0.65000000000000002,"15min":0.65800000000000003},"min":{"1min":0.441,"5min":0.441,"15min":0.441},"max":{"1min":0.77600000000000002,"5min":0.85999999999999999,"15min":0.85999999999999999},"last":0.501},{"interface":"front","average":{"1min":0.623,"5min":0.66800000000000004,"15min":0.67600000000000005},"min":{"1min":0.40000000000000002,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":0.90000000000000002,"5min":0.90000000000000002,"15min":0.90000000000000002},"last":0.64700000000000002}]}]},{"osd":3,"up_from":13,"seq":55834574889,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27912,"kb_used_data":892,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343928,"statfs":{"total":96636764160,"available":96608182272,"internally_reserved":0,"allocated":913408,"data_stored":384258,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 07:58:22 2023","interfaces":[{"interface":"back","average":{"1min":0.34200000000000003,"5min":0.35299999999999998,"15min":0.35499999999999998},"min":{"1min":0.20899999999999999,"5min":0.20899999999999999,"15min":0.20899999999999999},"max":{"1min":0.439,"5min":0.84799999999999998,"15min":0.84799999999999998},"last":0.53400000000000003},{"interface":"front","average":{"1min":0.32700000000000001,"5min":0.35899999999999999,"15min":0.36399999999999999},"min":{"1min":0.253,"5min":0.23699999999999999,"15min":0.23699999999999999},"max":{"1min":0.42499999999999999,"5min":0.873,"15min":0.873},"last":0.29499999999999998}]},{"osd":1,"last update":"Fri Nov 24 07:58:22 2023","interfaces":[{"interface":"back","average":{"1min":0.35899999999999999,"5min":0.371,"15min":0.373},"min":{"1min":0.29499999999999998,"5min":0.25700000000000001,"15min":0.25700000000000001},"max":{"1min":0.443,"5min":0.92000000000000004,"15min":0.92000000000000004},"last":0.36799999999999999},{"interface":"front","average":{"1min":0.35099999999999998,"5min":0.379,"15min":0.38300000000000001},"min":{"1min":0.28299999999999997,"5min":0.248,"15min":0.248},"max":{"1min":0.46700000000000003,"5min":0.81699999999999995,"15min":0.81699999999999995},"last":0.41099999999999998}]},{"osd":2,"last update":"Fri Nov 24 07:58:22 2023","interfaces":[{"interface":"back","average":{"1min":0.33400000000000002,"5min":0.35499999999999998,"15min":0.35899999999999999},"min":{"1min":0.255,"5min":0.24399999999999999,"15min":0.24399999999999999},"max":{"1min":0.43099999999999999,"5min":0.88500000000000001,"15min":0.88500000000000001},"last":0.42599999999999999},{"interface":"front","average":{"1min":0.39500000000000002,"5min":0.45300000000000001,"15min":0.46300000000000002},"min":{"1min":0.311,"5min":0.311,"15min":0.311},"max":{"1min":0.45800000000000002,"5min":0.98099999999999998,"15min":0.98099999999999998},"last":0.45000000000000001}]},{"osd":4,"last update":"Fri Nov 24 07:58:22 2023","interfaces":[{"interface":"back","average":{"1min":0.57099999999999995,"5min":0.751,"15min":0.78100000000000003},"min":{"1min":0.42099999999999999,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.71999999999999997,"5min":5.7350000000000003,"15min":5.7350000000000003},"last":0.63100000000000001},{"interface":"front","average":{"1min":0.59299999999999997,"5min":0.58599999999999997,"15min":0.58499999999999996},"min":{"1min":0.37,"5min":0.37,"15min":0.37},"max":{"1min":0.77000000000000002,"5min":1.27,"15min":1.27},"last":0.80400000000000005}]},{"osd":5,"last update":"Fri Nov 24 07:58:22 2023","interfaces":[{"interface":"back","average":{"1min":0.56899999999999995,"5min":0.58399999999999996,"15min":0.58599999999999997},"min":{"1min":0.40799999999999997,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":0.82799999999999996,"5min":1.4430000000000001,"15min":1.4430000000000001},"last":0.61099999999999999},{"interface":"front","average":{"1min":0.60699999999999998,"5min":0.623,"15min":0.625},"min":{"1min":0.443,"5min":0.377,"15min":0.377},"max":{"1min":0.79900000000000004,"5min":1.4239999999999999,"15min":1.4239999999999999},"last":0.78500000000000003}]},{"osd":6,"last update":"Fri Nov 24 07:58:22 2023","interfaces":[{"interface":"back","average":{"1min":0.58999999999999997,"5min":0.61399999999999999,"15min":0.61799999999999999},"min":{"1min":0.46200000000000002,"5min":0.434,"15min":0.434},"max":{"1min":0.82599999999999996,"5min":1.355,"15min":1.355},"last":0.84499999999999997},{"interface":"front","average":{"1min":0.61199999999999999,"5min":0.79900000000000004,"15min":0.82999999999999996},"min":{"1min":0.47499999999999998,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.76900000000000002,"5min":5.6479999999999997,"15min":5.6479999999999997},"last":0.73199999999999998}]},{"osd":7,"last update":"Fri Nov 24 07:58:22 2023","interfaces":[{"interface":"back","average":{"1min":0.59199999999999997,"5min":0.65100000000000002,"15min":0.66100000000000003},"min":{"1min":0.48099999999999998,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":0.79600000000000004,"5min":1.51,"15min":1.51},"last":0.82799999999999996},{"interface":"front","average":{"1min":0.61099999999999999,"5min":0.79500000000000004,"15min":0.82599999999999996},"min":{"1min":0.434,"5min":0.434,"15min":0.434},"max":{"1min":0.75900000000000001,"5min":5.7069999999999999,"15min":5.7069999999999999},"last":0.71199999999999997}]}]},{"osd":4,"up_from":13,"seq":55834574888,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27920,"kb_used_data":892,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343920,"statfs":{"total":96636764160,"available":96608174080,"internally_reserved":0,"allocated":913408,"data_stored":384826,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 07:58:25 2023","interfaces":[{"interface":"back","average":{"1min":0.504,"5min":0.56499999999999995,"15min":0.57499999999999996},"min":{"1min":0.34899999999999998,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.70799999999999996,"5min":0.79300000000000004,"15min":0.79300000000000004},"last":0.68000000000000005},{"interface":"front","average":{"1min":0.502,"5min":0.53400000000000003,"15min":0.54000000000000004},"min":{"1min":0.33600000000000002,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.70199999999999996,"5min":1.0249999999999999,"15min":1.0249999999999999},"last":0.44900000000000001}]},{"osd":1,"last update":"Fri Nov 24 07:58:25 2023","interfaces":[{"interface":"back","average":{"1min":0.52800000000000002,"5min":0.54700000000000004,"15min":0.55000000000000004},"min":{"1min":0.39500000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.73999999999999999,"5min":0.86399999999999999,"15min":0.86399999999999999},"last":0.63},{"interface":"front","average":{"1min":0.51900000000000002,"5min":0.54400000000000004,"15min":0.54800000000000004},"min":{"1min":0.374,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.72199999999999998,"5min":0.98699999999999999,"15min":0.98699999999999999},"last":0.40000000000000002}]},{"osd":2,"last update":"Fri Nov 24 07:58:25 2023","interfaces":[{"interface":"back","average":{"1min":0.52000000000000002,"5min":0.55200000000000005,"15min":0.55800000000000005},"min":{"1min":0.38800000000000001,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.66400000000000003,"5min":0.81100000000000005,"15min":0.81100000000000005},"last":0.60299999999999998},{"interface":"front","average":{"1min":0.56899999999999995,"5min":0.60399999999999998,"15min":0.60999999999999999},"min":{"1min":0.40000000000000002,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.75700000000000001,"5min":0.89200000000000002,"15min":0.89200000000000002},"last":0.64400000000000002}]},{"osd":3,"last update":"Fri Nov 24 07:58:25 2023","interfaces":[{"interface":"back","average":{"1min":0.54900000000000004,"5min":0.57699999999999996,"15min":0.58199999999999996},"min":{"1min":0.42899999999999999,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":0.65800000000000003,"5min":0.89700000000000002,"15min":0.89700000000000002},"last":0.49299999999999999},{"interface":"front","average":{"1min":0.57499999999999996,"5min":0.61899999999999999,"15min":0.627},"min":{"1min":0.442,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.72899999999999998,"5min":0.96199999999999997,"15min":0.96199999999999997},"last":0.58299999999999996}]},{"osd":5,"last update":"Fri Nov 24 07:58:25 2023","interfaces":[{"interface":"back","average":{"1min":0.48599999999999999,"5min":0.46100000000000002,"15min":0.45700000000000002},"min":{"1min":0.29499999999999998,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":0.67600000000000005,"5min":0.70499999999999996,"15min":0.70499999999999996},"last":0.35899999999999999},{"interface":"front","average":{"1min":0.59899999999999998,"5min":0.61499999999999999,"15min":0.61699999999999999},"min":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.746,"5min":0.86399999999999999,"15min":0.86399999999999999},"last":0.73699999999999999}]},{"osd":6,"last update":"Fri Nov 24 07:58:25 2023","interfaces":[{"interface":"back","average":{"1min":0.56799999999999995,"5min":0.58199999999999996,"15min":0.58399999999999996},"min":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.69799999999999995,"5min":0.76800000000000002,"15min":0.76800000000000002},"last":0.33000000000000002},{"interface":"front","average":{"1min":0.58999999999999997,"5min":0.60899999999999999,"15min":0.61199999999999999},"min":{"1min":0.46899999999999997,"5min":0.436,"15min":0.436},"max":{"1min":0.77700000000000002,"5min":1.095,"15min":1.095},"last":0.56000000000000005}]},{"osd":7,"last update":"Fri Nov 24 07:58:25 2023","interfaces":[{"interface":"back","average":{"1min":0.56799999999999995,"5min":0.57199999999999995,"15min":0.57199999999999995},"min":{"1min":0.39300000000000002,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.74199999999999999,"5min":0.89300000000000002,"15min":0.89300000000000002},"last":0.66000000000000003},{"interface":"front","average":{"1min":0.61699999999999999,"5min":0.65300000000000002,"15min":0.65900000000000003},"min":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.79100000000000004,"5min":0.91500000000000004,"15min":0.91500000000000004},"last":0.53600000000000003}]}]},{"osd":5,"up_from":13,"seq":55834574889,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28492,"kb_used_data":1464,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343348,"statfs":{"total":96636764160,"available":96607588352,"internally_reserved":0,"allocated":1499136,"data_stored":974590,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 07:58:20 2023","interfaces":[{"interface":"back","average":{"1min":0.52300000000000002,"5min":0.55400000000000005,"15min":0.55900000000000005},"min":{"1min":0.35299999999999998,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.65800000000000003,"5min":1.0109999999999999,"15min":1.0109999999999999},"last":0.58699999999999997},{"interface":"front","average":{"1min":0.53200000000000003,"5min":0.746,"15min":0.78200000000000003},"min":{"1min":0.34399999999999997,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.72099999999999997,"5min":5.431,"15min":5.431},"last":0.51300000000000001}]},{"osd":1,"last update":"Fri Nov 24 07:58:20 2023","interfaces":[{"interface":"back","average":{"1min":0.51100000000000001,"5min":0.54900000000000004,"15min":0.55500000000000005},"min":{"1min":0.36399999999999999,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":0.67100000000000004,"5min":1.016,"15min":1.016},"last":0.54100000000000004},{"interface":"front","average":{"1min":0.54600000000000004,"5min":0.56499999999999995,"15min":0.56799999999999995},"min":{"1min":0.28599999999999998,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":0.78300000000000003,"5min":0.96499999999999997,"15min":0.96499999999999997},"last":0.77500000000000002}]},{"osd":2,"last update":"Fri Nov 24 07:58:20 2023","interfaces":[{"interface":"back","average":{"1min":0.54500000000000004,"5min":0.54900000000000004,"15min":0.55000000000000004},"min":{"1min":0.33700000000000002,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.67100000000000004,"5min":1.163,"15min":1.163},"last":0.61399999999999999},{"interface":"front","average":{"1min":0.54500000000000004,"5min":0.61299999999999999,"15min":0.624},"min":{"1min":0.41099999999999998,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.78900000000000003,"5min":1.119,"15min":1.119},"last":0.63200000000000001}]},{"osd":3,"last update":"Fri Nov 24 07:58:20 2023","interfaces":[{"interface":"back","average":{"1min":0.57199999999999995,"5min":0.83799999999999997,"15min":0.88200000000000001},"min":{"1min":0.36899999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.85899999999999999,"5min":5.609,"15min":5.609},"last":0.71399999999999997},{"interface":"front","average":{"1min":0.629,"5min":0.66600000000000004,"15min":0.67200000000000004},"min":{"1min":0.44600000000000001,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":0.76400000000000001,"5min":1.288,"15min":1.288},"last":0.67700000000000005}]},{"osd":4,"last update":"Fri Nov 24 07:58:20 2023","interfaces":[{"interface":"back","average":{"1min":0.51500000000000001,"5min":0.51100000000000001,"15min":0.51000000000000001},"min":{"1min":0.32100000000000001,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.76400000000000001,"5min":0.76400000000000001,"15min":0.76400000000000001},"last":0.68799999999999994},{"interface":"front","average":{"1min":0.58799999999999997,"5min":0.57499999999999996,"15min":0.57299999999999995},"min":{"1min":0.29599999999999999,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":0.92900000000000005,"5min":0.92900000000000005,"15min":0.92900000000000005},"last":0.66200000000000003}]},{"osd":6,"last update":"Fri Nov 24 07:58:20 2023","interfaces":[{"interface":"back","average":{"1min":0.59599999999999997,"5min":0.57599999999999996,"15min":0.57199999999999995},"min":{"1min":0.35199999999999998,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.81799999999999995,"5min":0.89700000000000002,"15min":0.89700000000000002},"last":0.64900000000000002},{"interface":"front","average":{"1min":0.63800000000000001,"5min":0.627,"15min":0.625},"min":{"1min":0.40300000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":1.083,"5min":1.083,"15min":1.083},"last":0.72899999999999998}]},{"osd":7,"last update":"Fri Nov 24 07:58:20 2023","interfaces":[{"interface":"back","average":{"1min":0.66200000000000003,"5min":0.64900000000000002,"15min":0.64700000000000002},"min":{"1min":0.36799999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.90100000000000002,"5min":0.96599999999999997,"15min":0.96599999999999997},"last":0.75},{"interface":"front","average":{"1min":0.66800000000000004,"5min":0.65600000000000003,"15min":0.65400000000000003},"min":{"1min":0.42699999999999999,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":0.85599999999999998,"5min":0.94199999999999995,"15min":0.94199999999999995},"last":0.70199999999999996}]}]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":694,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1646,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-11-24T07:59:23.800 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2023-11-24T07:59:23.801 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2023-11-24T07:59:23.967 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:23.966+0000 7f4100c96700 1 Processor -- start 2023-11-24T07:59:23.968 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:23.966+0000 7f4100c96700 1 -- start start 2023-11-24T07:59:23.971 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:23.967+0000 7f4100c96700 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f40fc0cecd0 0x7f40fc15c1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:23.971 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:23.967+0000 7f4100c96700 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f40fc0d0a90 con 0x7f40fc15bde0 2023-11-24T07:59:23.971 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:23.967+0000 7f4100c96700 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f40fc06ee10 con 0x7f40fc0cf2b0 2023-11-24T07:59:23.971 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:23.967+0000 7f4100c96700 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f40fc06f0f0 con 0x7f40fc0cecd0 2023-11-24T07:59:23.972 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:23.967+0000 7f40fa59c700 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7f40fc15bde0 0x7f40fc0d01b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:58424/0 (socket says 172.21.15.59:58424) 2023-11-24T07:59:23.972 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:23.967+0000 7f40fa59c700 1 -- 172.21.15.59:0/1827249274 learned_addr learned my addr 172.21.15.59:0/1827249274 (peer_addr_for_me v1:172.21.15.59:0/0) 2023-11-24T07:59:23.972 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:23.967+0000 7f40f959a700 1 -- 172.21.15.59:0/1827249274 <== mon.0 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 322512128 0 0) 0x7f40fc0d0a90 con 0x7f40fc15bde0 2023-11-24T07:59:23.972 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:23.967+0000 7f40f959a700 1 -- 172.21.15.59:0/1827249274 --> v1:172.21.15.59:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f40e40035b0 con 0x7f40fc15bde0 2023-11-24T07:59:23.972 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:23.967+0000 7f40fad9d700 1 --2- 172.21.15.59:0/1827249274 >> v2:172.21.15.76:3300/0 conn(0x7f40fc0cecd0 0x7f40fc15c1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:23.972 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:23.967+0000 7f40f959a700 1 -- 172.21.15.59:0/1827249274 <== mon.0 v1:172.21.15.59:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3882595156 0 0) 0x7f40e40035b0 con 0x7f40fc15bde0 2023-11-24T07:59:23.972 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:23.967+0000 7f40f959a700 1 -- 172.21.15.59:0/1827249274 >> v2:172.21.15.76:3300/0 conn(0x7f40fc0cecd0 msgr2=0x7f40fc15c1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T07:59:23.972 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:23.967+0000 7f40f959a700 1 --2- 172.21.15.59:0/1827249274 >> v2:172.21.15.76:3300/0 conn(0x7f40fc0cecd0 0x7f40fc15c1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:23.972 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:23.967+0000 7f40f959a700 1 -- 172.21.15.59:0/1827249274 >> v1:172.21.15.76:6789/0 conn(0x7f40fc0cf2b0 legacy=0x7f40fc0ce5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:23.973 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:23.967+0000 7f40f959a700 1 -- 172.21.15.59:0/1827249274 --> v1:172.21.15.59:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f40fc06f3d0 con 0x7f40fc15bde0 2023-11-24T07:59:23.973 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:23.967+0000 7f40f959a700 1 -- 172.21.15.59:0/1827249274 <== mon.0 v1:172.21.15.59:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1312333566 0 0) 0x7f40ec001670 con 0x7f40fc15bde0 2023-11-24T07:59:23.973 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:23.967+0000 7f40f959a700 1 -- 172.21.15.59:0/1827249274 >> v1:172.21.15.59:6789/0 conn(0x7f40fc15bde0 legacy=0x7f40fc0d01b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:23.973 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:23.967+0000 7f40f959a700 1 --2- 172.21.15.59:0/1827249274 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f40e4003a00 0x7f40e4003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:23.973 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:23.967+0000 7f40f959a700 1 --2- 172.21.15.59:0/1827249274 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f40e4004850 0x7f40e4004cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:23.973 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:23.967+0000 7f40f959a700 1 --2- 172.21.15.59:0/1827249274 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f40e4005330 0x7f40e400ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:23.973 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:23.968+0000 7f40f959a700 1 -- 172.21.15.59:0/1827249274 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40fc06f0f0 con 0x7f40e4005330 2023-11-24T07:59:23.973 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:23.968+0000 7f40f959a700 1 -- 172.21.15.59:0/1827249274 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40e400c200 con 0x7f40e4003a00 2023-11-24T07:59:23.973 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:23.968+0000 7f40f959a700 1 -- 172.21.15.59:0/1827249274 --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f40e400c3c0 con 0x7f40e4004850 2023-11-24T07:59:23.973 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:23.968+0000 7f40fad9d700 1 --2- 172.21.15.59:0/1827249274 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f40e4005330 0x7f40e400ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:23.974 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:23.968+0000 7f40fa59c700 1 --2- 172.21.15.59:0/1827249274 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f40e4003a00 0x7f40e4003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:23.974 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:23.968+0000 7f40f9d9b700 1 --2- 172.21.15.59:0/1827249274 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f40e4004850 0x7f40e4004cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:23.974 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:23.968+0000 7f40fad9d700 1 -- 172.21.15.59:0/1827249274 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f40e4004850 msgr2=0x7f40e4004cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T07:59:23.974 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:23.968+0000 7f40fad9d700 1 --2- 172.21.15.59:0/1827249274 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f40e4004850 0x7f40e4004cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:23.974 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:23.968+0000 7f40fad9d700 1 -- 172.21.15.59:0/1827249274 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f40e4003a00 msgr2=0x7f40e4003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T07:59:23.974 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:23.968+0000 7f40fad9d700 1 --2- 172.21.15.59:0/1827249274 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f40e4003a00 0x7f40e4003e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:23.974 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:23.968+0000 7f40fad9d700 1 -- 172.21.15.59:0/1827249274 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f40e40035b0 con 0x7f40e4005330 2023-11-24T07:59:23.974 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:23.968+0000 7f40fad9d700 1 -- 172.21.15.59:0/1827249274 reap_dead start 2023-11-24T07:59:23.974 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:23.968+0000 7f40fad9d700 1 --2- 172.21.15.59:0/1827249274 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f40e4005330 0x7f40e400ba10 secure :-1 s=READY pgs=881 cs=0 l=1 rev1=1 crypto rx=0x7f40fc0c8c50 tx=0x7f40e4005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7b53a10771983d4 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:59:23.974 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:23.968+0000 7f40f959a700 1 -- 172.21.15.59:0/1827249274 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40f4003c20 con 0x7f40e4005330 2023-11-24T07:59:23.974 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:23.968+0000 7f40f959a700 1 -- 172.21.15.59:0/1827249274 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f40f400f040 con 0x7f40e4005330 2023-11-24T07:59:23.975 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:23.968+0000 7f40f959a700 1 -- 172.21.15.59:0/1827249274 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40f400e9d0 con 0x7f40e4005330 2023-11-24T07:59:23.975 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:23.969+0000 7f4100c96700 1 -- 172.21.15.59:0/1827249274 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f40e4005330 msgr2=0x7f40e400ba10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:23.975 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:23.969+0000 7f4100c96700 1 --2- 172.21.15.59:0/1827249274 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f40e4005330 0x7f40e400ba10 secure :-1 s=READY pgs=881 cs=0 l=1 rev1=1 crypto rx=0x7f40fc0c8c50 tx=0x7f40e4005880 comp rx=0 tx=0).stop 2023-11-24T07:59:23.975 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:23.969+0000 7f4100c96700 1 -- 172.21.15.59:0/1827249274 shutdown_connections 2023-11-24T07:59:23.975 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:23.969+0000 7f4100c96700 1 --2- 172.21.15.59:0/1827249274 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f40e4005330 0x7f40e400ba10 unknown :-1 s=CLOSED pgs=881 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:23.975 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:23.969+0000 7f4100c96700 1 -- 172.21.15.59:0/1827249274 >> 172.21.15.59:0/1827249274 conn(0x7f40fc0b3720 msgr2=0x7f40fc0b3b30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:59:23.975 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:23.969+0000 7f4100c96700 1 -- 172.21.15.59:0/1827249274 shutdown_connections 2023-11-24T07:59:23.975 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:23.969+0000 7f4100c96700 1 -- 172.21.15.59:0/1827249274 wait complete. 2023-11-24T07:59:23.978 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:23.970+0000 7f4100c96700 1 Processor -- start 2023-11-24T07:59:23.978 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:23.970+0000 7f4100c96700 1 -- start start 2023-11-24T07:59:23.978 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:23.971+0000 7f4100c96700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f40e4005330 0x7f40fc15c970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:23.978 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:23.971+0000 7f4100c96700 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f40fc15f2b0 0x7f40fc0b6f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:23.979 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:23.971+0000 7f4100c96700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f40fc0b7440 0x7f40fc0ca130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:23.979 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:23.971+0000 7f4100c96700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40fc15ceb0 con 0x7f40fc0b7440 2023-11-24T07:59:23.979 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:23.971+0000 7f4100c96700 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40fc0c9370 con 0x7f40e4005330 2023-11-24T07:59:23.979 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:23.971+0000 7f4100c96700 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f40fc0c9650 con 0x7f40fc15f2b0 2023-11-24T07:59:23.979 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:23.971+0000 7f40fad9d700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f40fc0b7440 0x7f40fc0ca130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:23.980 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:23.971+0000 7f40fad9d700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f40fc0b7440 0x7f40fc0ca130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:54966/0 (socket says 172.21.15.59:54966) 2023-11-24T07:59:23.980 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:23.971+0000 7f40fad9d700 1 -- 172.21.15.59:0/3555814662 learned_addr learned my addr 172.21.15.59:0/3555814662 (peer_addr_for_me v2:172.21.15.59:0/0) 2023-11-24T07:59:23.980 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:23.971+0000 7f40fa59c700 1 --2- 172.21.15.59:0/3555814662 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f40e4005330 0x7f40fc15c970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:23.980 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:23.971+0000 7f40f9d9b700 1 --2- 172.21.15.59:0/3555814662 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f40fc15f2b0 0x7f40fc0b6f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:23.980 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:23.971+0000 7f40fad9d700 1 -- 172.21.15.59:0/3555814662 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f40fc15f2b0 msgr2=0x7f40fc0b6f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:23.980 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:23.971+0000 7f40fad9d700 1 --2- 172.21.15.59:0/3555814662 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f40fc15f2b0 0x7f40fc0b6f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:23.980 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:23.971+0000 7f40fad9d700 1 -- 172.21.15.59:0/3555814662 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f40e4005330 msgr2=0x7f40fc15c970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:23.980 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:23.971+0000 7f40fad9d700 1 --2- 172.21.15.59:0/3555814662 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f40e4005330 0x7f40fc15c970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:23.980 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:23.971+0000 7f40fad9d700 1 -- 172.21.15.59:0/3555814662 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f40fc0c9930 con 0x7f40fc0b7440 2023-11-24T07:59:23.981 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:23.971+0000 7f40f9d9b700 1 --2- 172.21.15.59:0/3555814662 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f40fc15f2b0 0x7f40fc0b6f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-24T07:59:23.981 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:23.972+0000 7f40fad9d700 1 --2- 172.21.15.59:0/3555814662 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f40fc0b7440 0x7f40fc0ca130 secure :-1 s=READY pgs=882 cs=0 l=1 rev1=1 crypto rx=0x7f40fc0d1530 tx=0x7f40f4009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:59:23.981 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:23.972+0000 7f40eaffd700 1 -- 172.21.15.59:0/3555814662 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40f4003e10 con 0x7f40fc0b7440 2023-11-24T07:59:23.981 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:23.972+0000 7f40eaffd700 1 -- 172.21.15.59:0/3555814662 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f40f4004230 con 0x7f40fc0b7440 2023-11-24T07:59:23.981 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:23.972+0000 7f40eaffd700 1 -- 172.21.15.59:0/3555814662 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40f40045e0 con 0x7f40fc0b7440 2023-11-24T07:59:23.981 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:23.972+0000 7f4100c96700 1 -- 172.21.15.59:0/3555814662 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f40fc0c9c10 con 0x7f40fc0b7440 2023-11-24T07:59:23.981 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:23.972+0000 7f4100c96700 1 -- 172.21.15.59:0/3555814662 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f40fc0b78e0 con 0x7f40fc0b7440 2023-11-24T07:59:23.981 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:23.973+0000 7f4100c96700 1 -- 172.21.15.59:0/3555814662 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f40fc075480 con 0x7f40fc0b7440 2023-11-24T07:59:23.981 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:23.973+0000 7f40eaffd700 1 -- 172.21.15.59:0/3555814662 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 10) v1 ==== 89588+0+0 (secure 0 0 0) 0x7f40f4004740 con 0x7f40fc0b7440 2023-11-24T07:59:23.981 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:23.973+0000 7f40eaffd700 1 --2- 172.21.15.59:0/3555814662 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f40dc06b2c0 0x7f40dc06d780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:23.981 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:23.974+0000 7f40eaffd700 1 -- 172.21.15.59:0/3555814662 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f40f408b4f0 con 0x7f40fc0b7440 2023-11-24T07:59:23.982 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:23.974+0000 7f40fa59c700 1 --2- 172.21.15.59:0/3555814662 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f40dc06b2c0 0x7f40dc06d780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:23.982 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:23.974+0000 7f40fa59c700 1 --2- 172.21.15.59:0/3555814662 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f40dc06b2c0 0x7f40dc06d780 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f40fc0b61e0 tx=0x7f40ec000f80 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:59:23.982 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:23.978+0000 7f40eaffd700 1 -- 172.21.15.59:0/3555814662 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f40f4055f40 con 0x7f40fc0b7440 2023-11-24T07:59:24.264 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:24.263+0000 7f4100c96700 1 -- 172.21.15.59:0/3555814662 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7f40fc0d0d30 con 0x7f40fc0b7440 2023-11-24T07:59:24.267 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:24.265+0000 7f40eaffd700 1 -- 172.21.15.59:0/3555814662 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v16) v1 ==== 190+0+147 (secure 0 0 0) 0x7f40f40220b0 con 0x7f40fc0b7440 2023-11-24T07:59:24.267 INFO:teuthology.orchestra.run.smithi059.stdout:[client.0] 2023-11-24T07:59:24.267 INFO:teuthology.orchestra.run.smithi059.stdout: key = AQD1VmBlxPkpBhAAR841hRsF6QtmzFgJIrVZ9w== 2023-11-24T07:59:24.267 INFO:teuthology.orchestra.run.smithi059.stdout: caps mds = "allow" 2023-11-24T07:59:24.267 INFO:teuthology.orchestra.run.smithi059.stdout: caps mon = "allow r" 2023-11-24T07:59:24.268 INFO:teuthology.orchestra.run.smithi059.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2023-11-24T07:59:24.270 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:24.268+0000 7f4100c96700 1 -- 172.21.15.59:0/3555814662 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f40dc06b2c0 msgr2=0x7f40dc06d780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:24.271 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:24.268+0000 7f4100c96700 1 --2- 172.21.15.59:0/3555814662 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f40dc06b2c0 0x7f40dc06d780 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f40fc0b61e0 tx=0x7f40ec000f80 comp rx=0 tx=0).stop 2023-11-24T07:59:24.271 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:24.268+0000 7f4100c96700 1 -- 172.21.15.59:0/3555814662 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f40fc0b7440 msgr2=0x7f40fc0ca130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:24.271 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:24.268+0000 7f4100c96700 1 --2- 172.21.15.59:0/3555814662 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f40fc0b7440 0x7f40fc0ca130 secure :-1 s=READY pgs=882 cs=0 l=1 rev1=1 crypto rx=0x7f40fc0d1530 tx=0x7f40f4009750 comp rx=0 tx=0).stop 2023-11-24T07:59:24.271 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:24.268+0000 7f4100c96700 1 -- 172.21.15.59:0/3555814662 shutdown_connections 2023-11-24T07:59:24.271 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:24.268+0000 7f4100c96700 1 --2- 172.21.15.59:0/3555814662 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f40dc06b2c0 0x7f40dc06d780 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:24.271 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:24.268+0000 7f4100c96700 1 --2- 172.21.15.59:0/3555814662 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f40fc0b7440 0x7f40fc0ca130 unknown :-1 s=CLOSED pgs=882 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:24.271 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:24.268+0000 7f4100c96700 1 --2- 172.21.15.59:0/3555814662 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f40e4005330 0x7f40fc15c970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:24.271 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:24.268+0000 7f4100c96700 1 --2- 172.21.15.59:0/3555814662 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f40fc15f2b0 0x7f40fc0b6f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:24.272 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:24.268+0000 7f4100c96700 1 -- 172.21.15.59:0/3555814662 >> 172.21.15.59:0/3555814662 conn(0x7f40fc0b3720 msgr2=0x7f40fc06fa20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:59:24.272 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:24.268+0000 7f4100c96700 1 -- 172.21.15.59:0/3555814662 shutdown_connections 2023-11-24T07:59:24.272 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:24.268+0000 7f4100c96700 1 -- 172.21.15.59:0/3555814662 wait complete. 2023-11-24T07:59:24.272 INFO:teuthology.orchestra.run.smithi059.stderr:updated caps for client.0 2023-11-24T07:59:24.283 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-24T07:59:24.408 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:24.407+0000 7f162eedc700 1 Processor -- start 2023-11-24T07:59:24.409 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:24.407+0000 7f162eedc700 1 -- start start 2023-11-24T07:59:24.411 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:24.408+0000 7f162eedc700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f162815be40 0x7f16280af400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:24.411 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:24.408+0000 7f162eedc700 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f16280d0ec0 0x7f16280b1550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:24.411 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:24.408+0000 7f162eedc700 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f16280b1a90 0x7f162806b7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:24.411 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:24.408+0000 7f162eedc700 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f16280b1130 con 0x7f162815be40 2023-11-24T07:59:24.411 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:24.408+0000 7f162eedc700 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1628067420 con 0x7f16280d0ec0 2023-11-24T07:59:24.412 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:24.408+0000 7f162eedc700 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f16280af940 con 0x7f16280b1a90 2023-11-24T07:59:24.412 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:24.408+0000 7f162cc78700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f162815be40 0x7f16280af400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:24.412 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:24.408+0000 7f162cc78700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f162815be40 0x7f16280af400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:54970/0 (socket says 172.21.15.59:54970) 2023-11-24T07:59:24.412 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:24.408+0000 7f162cc78700 1 -- 172.21.15.59:0/1702018147 learned_addr learned my addr 172.21.15.59:0/1702018147 (peer_addr_for_me v2:172.21.15.59:0/0) 2023-11-24T07:59:24.412 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:24.408+0000 7f1627fff700 1 --2- 172.21.15.59:0/1702018147 >> v2:172.21.15.76:3300/0 conn(0x7f16280d0ec0 0x7f16280b1550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:24.412 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:24.408+0000 7f162d479700 1 --2- 172.21.15.59:0/1702018147 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f16280b1a90 0x7f162806b7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:24.412 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:24.408+0000 7f162cc78700 1 -- 172.21.15.59:0/1702018147 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f16280b1a90 msgr2=0x7f162806b7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T07:59:24.412 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:24.408+0000 7f162cc78700 1 --2- 172.21.15.59:0/1702018147 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f16280b1a90 0x7f162806b7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:24.412 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:24.408+0000 7f162cc78700 1 -- 172.21.15.59:0/1702018147 >> v2:172.21.15.76:3300/0 conn(0x7f16280d0ec0 msgr2=0x7f16280b1550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T07:59:24.412 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:24.408+0000 7f162cc78700 1 --2- 172.21.15.59:0/1702018147 >> v2:172.21.15.76:3300/0 conn(0x7f16280d0ec0 0x7f16280b1550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:24.412 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:24.408+0000 7f162cc78700 1 -- 172.21.15.59:0/1702018147 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f16280b0ab0 con 0x7f162815be40 2023-11-24T07:59:24.412 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:24.408+0000 7f1627fff700 1 --2- 172.21.15.59:0/1702018147 >> v2:172.21.15.76:3300/0 conn(0x7f16280d0ec0 0x7f16280b1550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-24T07:59:24.413 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:24.408+0000 7f162cc78700 1 --2- 172.21.15.59:0/1702018147 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f162815be40 0x7f16280af400 secure :-1 s=READY pgs=883 cs=0 l=1 rev1=1 crypto rx=0x7f16280709e0 tx=0x7f161c009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bd356dea08b352e9 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:59:24.413 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:24.408+0000 7f16277fe700 1 -- 172.21.15.59:0/1702018147 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f161c00b870 con 0x7f162815be40 2023-11-24T07:59:24.413 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:24.408+0000 7f16277fe700 1 -- 172.21.15.59:0/1702018147 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f161c00b9d0 con 0x7f162815be40 2023-11-24T07:59:24.413 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:24.408+0000 7f16277fe700 1 -- 172.21.15.59:0/1702018147 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f161c00bd60 con 0x7f162815be40 2023-11-24T07:59:24.413 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:24.409+0000 7f162eedc700 1 -- 172.21.15.59:0/1702018147 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f162815be40 msgr2=0x7f16280af400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:24.413 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:24.409+0000 7f162eedc700 1 --2- 172.21.15.59:0/1702018147 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f162815be40 0x7f16280af400 secure :-1 s=READY pgs=883 cs=0 l=1 rev1=1 crypto rx=0x7f16280709e0 tx=0x7f161c009bd0 comp rx=0 tx=0).stop 2023-11-24T07:59:24.413 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:24.409+0000 7f162eedc700 1 -- 172.21.15.59:0/1702018147 shutdown_connections 2023-11-24T07:59:24.413 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:24.409+0000 7f162eedc700 1 --2- 172.21.15.59:0/1702018147 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f16280b1a90 0x7f162806b7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:24.414 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:24.409+0000 7f162eedc700 1 --2- 172.21.15.59:0/1702018147 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f162815be40 0x7f16280af400 unknown :-1 s=CLOSED pgs=883 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:24.414 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:24.409+0000 7f162eedc700 1 --2- 172.21.15.59:0/1702018147 >> v2:172.21.15.76:3300/0 conn(0x7f16280d0ec0 0x7f16280b1550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:24.414 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:24.409+0000 7f162eedc700 1 -- 172.21.15.59:0/1702018147 >> 172.21.15.59:0/1702018147 conn(0x7f16280717f0 msgr2=0x7f16280b5070 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:59:24.414 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:24.409+0000 7f162eedc700 1 -- 172.21.15.59:0/1702018147 shutdown_connections 2023-11-24T07:59:24.414 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:24.409+0000 7f162eedc700 1 -- 172.21.15.59:0/1702018147 wait complete. 2023-11-24T07:59:24.416 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:24.410+0000 7f162eedc700 1 Processor -- start 2023-11-24T07:59:24.417 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:24.410+0000 7f162eedc700 1 -- start start 2023-11-24T07:59:24.417 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:24.411+0000 7f162eedc700 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f16280d0ec0 0x7f16280cb990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:24.417 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:24.411+0000 7f162eedc700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f162815be40 0x7f1628142680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:24.417 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:24.411+0000 7f162eedc700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1628142bc0 0x7f1628144710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:24.417 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:24.411+0000 7f162eedc700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16281f16d0 con 0x7f162815be40 2023-11-24T07:59:24.417 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:24.411+0000 7f162eedc700 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1628155a40 con 0x7f1628142bc0 2023-11-24T07:59:24.417 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:24.411+0000 7f162eedc700 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f16281431f0 con 0x7f16280d0ec0 2023-11-24T07:59:24.417 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:24.411+0000 7f1627fff700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f162815be40 0x7f1628142680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:24.418 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:24.411+0000 7f1627fff700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f162815be40 0x7f1628142680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:54978/0 (socket says 172.21.15.59:54978) 2023-11-24T07:59:24.418 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:24.411+0000 7f1627fff700 1 -- 172.21.15.59:0/121313879 learned_addr learned my addr 172.21.15.59:0/121313879 (peer_addr_for_me v2:172.21.15.59:0/0) 2023-11-24T07:59:24.418 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:24.411+0000 7f162d479700 1 --2- 172.21.15.59:0/121313879 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1628142bc0 0x7f1628144710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:24.418 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:24.411+0000 7f162cc78700 1 --2- 172.21.15.59:0/121313879 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f16280d0ec0 0x7f16280cb990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:24.418 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:24.411+0000 7f1627fff700 1 -- 172.21.15.59:0/121313879 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f16280d0ec0 msgr2=0x7f16280cb990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:24.418 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:24.412+0000 7f1627fff700 1 --2- 172.21.15.59:0/121313879 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f16280d0ec0 0x7f16280cb990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:24.418 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:24.412+0000 7f1627fff700 1 -- 172.21.15.59:0/121313879 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1628142bc0 msgr2=0x7f1628144710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:24.418 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:24.412+0000 7f1627fff700 1 --2- 172.21.15.59:0/121313879 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1628142bc0 0x7f1628144710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:24.418 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:24.412+0000 7f1627fff700 1 -- 172.21.15.59:0/121313879 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f161c009550 con 0x7f162815be40 2023-11-24T07:59:24.418 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:24.412+0000 7f162d479700 1 --2- 172.21.15.59:0/121313879 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1628142bc0 0x7f1628144710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-24T07:59:24.419 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:24.412+0000 7f1627fff700 1 --2- 172.21.15.59:0/121313879 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f162815be40 0x7f1628142680 secure :-1 s=READY pgs=884 cs=0 l=1 rev1=1 crypto rx=0x7f161400e810 tx=0x7f161400ebd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:59:24.419 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:24.412+0000 7f16257fa700 1 -- 172.21.15.59:0/121313879 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1614004230 con 0x7f162815be40 2023-11-24T07:59:24.419 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:24.412+0000 7f16257fa700 1 -- 172.21.15.59:0/121313879 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1614004390 con 0x7f162815be40 2023-11-24T07:59:24.419 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:24.412+0000 7f162eedc700 1 -- 172.21.15.59:0/121313879 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f16281434d0 con 0x7f162815be40 2023-11-24T07:59:24.419 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:24.412+0000 7f16257fa700 1 -- 172.21.15.59:0/121313879 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16140036a0 con 0x7f162815be40 2023-11-24T07:59:24.419 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:24.412+0000 7f162eedc700 1 -- 172.21.15.59:0/121313879 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1628143a90 con 0x7f162815be40 2023-11-24T07:59:24.419 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:24.413+0000 7f162eedc700 1 -- 172.21.15.59:0/121313879 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1628068670 con 0x7f162815be40 2023-11-24T07:59:24.419 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:24.413+0000 7f16257fa700 1 -- 172.21.15.59:0/121313879 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 10) v1 ==== 89588+0+0 (secure 0 0 0) 0x7f1614003800 con 0x7f162815be40 2023-11-24T07:59:24.419 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:24.414+0000 7f16257fa700 1 --2- 172.21.15.59:0/121313879 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f160c06b2c0 0x7f160c06d780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:24.419 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:24.414+0000 7f16257fa700 1 -- 172.21.15.59:0/121313879 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f1614089910 con 0x7f162815be40 2023-11-24T07:59:24.419 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:24.414+0000 7f162cc78700 1 --2- 172.21.15.59:0/121313879 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f160c06b2c0 0x7f160c06d780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:24.419 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:24.414+0000 7f162cc78700 1 --2- 172.21.15.59:0/121313879 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f160c06b2c0 0x7f160c06d780 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f161c014040 tx=0x7f161c008030 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:59:24.420 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:24.418+0000 7f16257fa700 1 -- 172.21.15.59:0/121313879 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f16140542d0 con 0x7f162815be40 2023-11-24T07:59:24.669 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:24.668+0000 7f162eedc700 1 -- 172.21.15.59:0/121313879 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f16280b1680 con 0x7f162815be40 2023-11-24T07:59:24.670 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:24.668+0000 7f16257fa700 1 -- 172.21.15.59:0/121313879 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 44 v44) v1 ==== 94+0+4915 (secure 0 0 0) 0x7f16140094c0 con 0x7f162815be40 2023-11-24T07:59:24.670 INFO:teuthology.orchestra.run.smithi059.stdout: 2023-11-24T07:59:24.670 INFO:teuthology.orchestra.run.smithi059.stdout:{"epoch":44,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5585,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.76:6837/2282835612","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6835","nonce":2282835612},{"type":"v1","addr":"172.21.15.76:6837","nonce":2282835612}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":39},{"gid":6097,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.76:6836/3924452718","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6834","nonce":3924452718},{"type":"v1","addr":"172.21.15.76:6836","nonce":3924452718}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":39},{"gid":6549,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6834/1425120389","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6832","nonce":1425120389},{"type":"v1","addr":"172.21.15.59:6834","nonce":1425120389}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":39}],"filesystems":[{"mdsmap":{"epoch":44,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-24T07:59:14.054294+0000","modified":"2023-11-24T07:59:17.066277+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":6552},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6552":{"gid":6552,"name":"c","rank":0,"incarnation":40,"state":"up:active","state_seq":3,"addr":"172.21.15.59:6835/1228649189","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6833","nonce":1228649189},{"type":"v1","addr":"172.21.15.59:6835","nonce":1228649189}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[11],"metadata_pool":10,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":5}]} 2023-11-24T07:59:24.672 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:24.670+0000 7f162eedc700 1 -- 172.21.15.59:0/121313879 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f160c06b2c0 msgr2=0x7f160c06d780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:24.672 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:24.670+0000 7f162eedc700 1 --2- 172.21.15.59:0/121313879 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f160c06b2c0 0x7f160c06d780 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f161c014040 tx=0x7f161c008030 comp rx=0 tx=0).stop 2023-11-24T07:59:24.672 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:24.670+0000 7f162eedc700 1 -- 172.21.15.59:0/121313879 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f162815be40 msgr2=0x7f1628142680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:24.673 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:24.670+0000 7f162eedc700 1 --2- 172.21.15.59:0/121313879 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f162815be40 0x7f1628142680 secure :-1 s=READY pgs=884 cs=0 l=1 rev1=1 crypto rx=0x7f161400e810 tx=0x7f161400ebd0 comp rx=0 tx=0).stop 2023-11-24T07:59:24.673 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:24.670+0000 7f162eedc700 1 -- 172.21.15.59:0/121313879 shutdown_connections 2023-11-24T07:59:24.673 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:24.670+0000 7f162eedc700 1 --2- 172.21.15.59:0/121313879 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f160c06b2c0 0x7f160c06d780 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:24.673 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:24.670+0000 7f162eedc700 1 --2- 172.21.15.59:0/121313879 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1628142bc0 0x7f1628144710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:24.673 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:24.670+0000 7f162eedc700 1 --2- 172.21.15.59:0/121313879 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f16280d0ec0 0x7f16280cb990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:24.673 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:24.670+0000 7f162eedc700 1 --2- 172.21.15.59:0/121313879 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f162815be40 0x7f1628142680 unknown :-1 s=CLOSED pgs=884 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:24.674 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:24.670+0000 7f162eedc700 1 -- 172.21.15.59:0/121313879 >> 172.21.15.59:0/121313879 conn(0x7f16280717f0 msgr2=0x7f162806d880 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:59:24.674 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:24.671+0000 7f162eedc700 1 -- 172.21.15.59:0/121313879 shutdown_connections 2023-11-24T07:59:24.674 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:24.671+0000 7f162eedc700 1 -- 172.21.15.59:0/121313879 wait complete. 2023-11-24T07:59:24.674 INFO:teuthology.orchestra.run.smithi059.stderr:dumped fsmap epoch 44 2023-11-24T07:59:25.087 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 44, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-11-24T07:59:14.054294+0000', 'modified': '2023-11-24T07:59:17.066277+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 6552}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_6552': {'gid': 6552, 'name': 'c', 'rank': 0, 'incarnation': 40, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.59:6835/1228649189', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.59:6833', 'nonce': 1228649189}, {'type': 'v1', 'addr': '172.21.15.59:6835', 'nonce': 1228649189}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [11], 'metadata_pool': 10, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2023-11-24T07:59:25.088 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2023-11-24T07:59:25.088 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2023-11-24T07:59:25.219 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:25.217+0000 7f538fcaa700 1 Processor -- start 2023-11-24T07:59:25.219 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:25.217+0000 7f538fcaa700 1 -- start start 2023-11-24T07:59:25.222 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:25.218+0000 7f538fcaa700 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f53880b5a70 0x7f5388072250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:25.222 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:25.218+0000 7f538fcaa700 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f53880ad860 con 0x7f538806edc0 2023-11-24T07:59:25.222 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:25.218+0000 7f538fcaa700 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f53880d0a20 con 0x7f538806afd0 2023-11-24T07:59:25.222 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:25.218+0000 7f538fcaa700 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f538806f830 con 0x7f53880b5a70 2023-11-24T07:59:25.222 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:25.218+0000 7f538d245700 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7f538806edc0 0x7f5388071b40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:58460/0 (socket says 172.21.15.59:58460) 2023-11-24T07:59:25.222 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:25.218+0000 7f538d245700 1 -- 172.21.15.59:0/771112227 learned_addr learned my addr 172.21.15.59:0/771112227 (peer_addr_for_me v1:172.21.15.59:0/0) 2023-11-24T07:59:25.223 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:25.218+0000 7f538e247700 1 --2- 172.21.15.59:0/771112227 >> v2:172.21.15.76:3300/0 conn(0x7f53880b5a70 0x7f5388072250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:25.223 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:25.218+0000 7f538ca44700 1 -- 172.21.15.59:0/771112227 <== mon.0 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 749226334 0 0) 0x7f53880ad860 con 0x7f538806edc0 2023-11-24T07:59:25.223 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:25.218+0000 7f538ca44700 1 -- 172.21.15.59:0/771112227 --> v1:172.21.15.59:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f53740035b0 con 0x7f538806edc0 2023-11-24T07:59:25.223 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:25.218+0000 7f538ca44700 1 -- 172.21.15.59:0/771112227 <== mon.0 v1:172.21.15.59:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1373862161 0 0) 0x7f53740035b0 con 0x7f538806edc0 2023-11-24T07:59:25.223 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:25.218+0000 7f538ca44700 1 -- 172.21.15.59:0/771112227 >> v2:172.21.15.76:3300/0 conn(0x7f53880b5a70 msgr2=0x7f5388072250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T07:59:25.223 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:25.218+0000 7f538ca44700 1 --2- 172.21.15.59:0/771112227 >> v2:172.21.15.76:3300/0 conn(0x7f53880b5a70 0x7f5388072250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:25.223 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:25.218+0000 7f538ca44700 1 -- 172.21.15.59:0/771112227 >> v1:172.21.15.76:6789/0 conn(0x7f538806afd0 legacy=0x7f538806b3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:25.223 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:25.218+0000 7f538ca44700 1 -- 172.21.15.59:0/771112227 --> v1:172.21.15.59:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5388162a70 con 0x7f538806edc0 2023-11-24T07:59:25.223 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:25.218+0000 7f538ca44700 1 -- 172.21.15.59:0/771112227 <== mon.0 v1:172.21.15.59:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1312333566 0 0) 0x7f5378001670 con 0x7f538806edc0 2023-11-24T07:59:25.223 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:25.218+0000 7f538ca44700 1 -- 172.21.15.59:0/771112227 >> v1:172.21.15.59:6789/0 conn(0x7f538806edc0 legacy=0x7f5388071b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:25.223 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:25.218+0000 7f538ca44700 1 --2- 172.21.15.59:0/771112227 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5374003a00 0x7f5374003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:25.223 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:25.218+0000 7f538ca44700 1 --2- 172.21.15.59:0/771112227 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f5374004850 0x7f5374004cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:25.224 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:25.218+0000 7f538ca44700 1 --2- 172.21.15.59:0/771112227 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5374005210 0x7f537400ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:25.224 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:25.219+0000 7f538ca44700 1 -- 172.21.15.59:0/771112227 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f538806f830 con 0x7f5374003a00 2023-11-24T07:59:25.224 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:25.219+0000 7f538ca44700 1 -- 172.21.15.59:0/771112227 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f537400c1b0 con 0x7f5374005210 2023-11-24T07:59:25.224 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:25.219+0000 7f538ca44700 1 -- 172.21.15.59:0/771112227 --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f537400c370 con 0x7f5374004850 2023-11-24T07:59:25.224 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:25.219+0000 7f538da46700 1 --2- 172.21.15.59:0/771112227 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5374003a00 0x7f5374003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:25.224 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:25.219+0000 7f538e247700 1 --2- 172.21.15.59:0/771112227 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5374005210 0x7f537400ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:25.224 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:25.219+0000 7f538d245700 1 --2- 172.21.15.59:0/771112227 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f5374004850 0x7f5374004cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:25.224 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:25.219+0000 7f538da46700 1 -- 172.21.15.59:0/771112227 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f5374004850 msgr2=0x7f5374004cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T07:59:25.225 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:25.219+0000 7f538da46700 1 --2- 172.21.15.59:0/771112227 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f5374004850 0x7f5374004cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:25.225 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:25.219+0000 7f538da46700 1 -- 172.21.15.59:0/771112227 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5374005210 msgr2=0x7f537400ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T07:59:25.225 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:25.219+0000 7f538da46700 1 --2- 172.21.15.59:0/771112227 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5374005210 0x7f537400ba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:25.225 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:25.219+0000 7f538e247700 1 -- 172.21.15.59:0/771112227 reap_dead start 2023-11-24T07:59:25.225 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:25.219+0000 7f538da46700 1 -- 172.21.15.59:0/771112227 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f53740035b0 con 0x7f5374003a00 2023-11-24T07:59:25.225 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:25.219+0000 7f538da46700 1 --2- 172.21.15.59:0/771112227 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5374003a00 0x7f5374003e20 secure :-1 s=READY pgs=886 cs=0 l=1 rev1=1 crypto rx=0x7f53840017c0 tx=0x7f5374005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=36ab3c426d562e19 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:59:25.225 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:25.219+0000 7f538ca44700 1 -- 172.21.15.59:0/771112227 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53840052d0 con 0x7f5374003a00 2023-11-24T07:59:25.225 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:25.219+0000 7f538ca44700 1 -- 172.21.15.59:0/771112227 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5384005450 con 0x7f5374003a00 2023-11-24T07:59:25.226 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:25.219+0000 7f538ca44700 1 -- 172.21.15.59:0/771112227 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53840038c0 con 0x7f5374003a00 2023-11-24T07:59:25.226 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:25.220+0000 7f538fcaa700 1 -- 172.21.15.59:0/771112227 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5374003a00 msgr2=0x7f5374003e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:25.226 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:25.220+0000 7f538fcaa700 1 --2- 172.21.15.59:0/771112227 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5374003a00 0x7f5374003e20 secure :-1 s=READY pgs=886 cs=0 l=1 rev1=1 crypto rx=0x7f53840017c0 tx=0x7f5374005880 comp rx=0 tx=0).stop 2023-11-24T07:59:25.226 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:25.220+0000 7f538fcaa700 1 -- 172.21.15.59:0/771112227 shutdown_connections 2023-11-24T07:59:25.226 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:25.220+0000 7f538fcaa700 1 --2- 172.21.15.59:0/771112227 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5374003a00 0x7f5374003e20 unknown :-1 s=CLOSED pgs=886 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:25.226 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:25.220+0000 7f538fcaa700 1 -- 172.21.15.59:0/771112227 >> 172.21.15.59:0/771112227 conn(0x7f53880b2840 msgr2=0x7f53880b2c20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:59:25.226 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:25.220+0000 7f538fcaa700 1 -- 172.21.15.59:0/771112227 shutdown_connections 2023-11-24T07:59:25.226 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:25.220+0000 7f538fcaa700 1 -- 172.21.15.59:0/771112227 wait complete. 2023-11-24T07:59:25.231 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:25.221+0000 7f538fcaa700 1 Processor -- start 2023-11-24T07:59:25.231 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:25.221+0000 7f538fcaa700 1 -- start start 2023-11-24T07:59:25.231 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:25.222+0000 7f538fcaa700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5374003a00 0x7f538815d570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:25.231 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:25.222+0000 7f538fcaa700 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f53880c64c0 0x7f53880c7fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:25.231 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:25.222+0000 7f538fcaa700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f53881585f0 0x7f5388158a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:25.232 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:25.222+0000 7f538fcaa700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53881f4010 con 0x7f5374003a00 2023-11-24T07:59:25.232 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:25.222+0000 7f538fcaa700 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5388159670 con 0x7f53881585f0 2023-11-24T07:59:25.232 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:25.222+0000 7f538fcaa700 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f538815dab0 con 0x7f53880c64c0 2023-11-24T07:59:25.232 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:25.222+0000 7f538da46700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5374003a00 0x7f538815d570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:25.232 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:25.222+0000 7f538da46700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5374003a00 0x7f538815d570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:54996/0 (socket says 172.21.15.59:54996) 2023-11-24T07:59:25.232 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:25.222+0000 7f538da46700 1 -- 172.21.15.59:0/976663118 learned_addr learned my addr 172.21.15.59:0/976663118 (peer_addr_for_me v2:172.21.15.59:0/0) 2023-11-24T07:59:25.232 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:25.222+0000 7f538d245700 1 --2- 172.21.15.59:0/976663118 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f53880c64c0 0x7f53880c7fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:25.232 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:25.222+0000 7f538e247700 1 --2- 172.21.15.59:0/976663118 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f53881585f0 0x7f5388158a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:25.232 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:25.222+0000 7f538da46700 1 -- 172.21.15.59:0/976663118 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f53880c64c0 msgr2=0x7f53880c7fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:25.232 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:25.222+0000 7f538da46700 1 --2- 172.21.15.59:0/976663118 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f53880c64c0 0x7f53880c7fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:25.232 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:25.222+0000 7f538da46700 1 -- 172.21.15.59:0/976663118 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f53881585f0 msgr2=0x7f5388158a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:25.233 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:25.222+0000 7f538da46700 1 --2- 172.21.15.59:0/976663118 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f53881585f0 0x7f5388158a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:25.233 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:25.222+0000 7f538da46700 1 -- 172.21.15.59:0/976663118 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f538815a7a0 con 0x7f5374003a00 2023-11-24T07:59:25.233 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:25.222+0000 7f538da46700 1 --2- 172.21.15.59:0/976663118 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5374003a00 0x7f538815d570 secure :-1 s=READY pgs=887 cs=0 l=1 rev1=1 crypto rx=0x7f5384000c00 tx=0x7f5384001120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:59:25.233 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:25.222+0000 7f537e7fc700 1 -- 172.21.15.59:0/976663118 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5384001290 con 0x7f5374003a00 2023-11-24T07:59:25.233 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:25.223+0000 7f537e7fc700 1 -- 172.21.15.59:0/976663118 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5384007040 con 0x7f5374003a00 2023-11-24T07:59:25.233 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:25.223+0000 7f538fcaa700 1 -- 172.21.15.59:0/976663118 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f53880c69c0 con 0x7f5374003a00 2023-11-24T07:59:25.233 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:25.223+0000 7f537e7fc700 1 -- 172.21.15.59:0/976663118 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53840073d0 con 0x7f5374003a00 2023-11-24T07:59:25.233 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:25.223+0000 7f538fcaa700 1 -- 172.21.15.59:0/976663118 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f538815aa40 con 0x7f5374003a00 2023-11-24T07:59:25.233 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:25.224+0000 7f537e7fc700 1 -- 172.21.15.59:0/976663118 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 10) v1 ==== 89588+0+0 (secure 0 0 0) 0x7f5384007530 con 0x7f5374003a00 2023-11-24T07:59:25.233 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:25.224+0000 7f537e7fc700 1 --2- 172.21.15.59:0/976663118 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f536c06b2c0 0x7f536c06d780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:25.233 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:25.225+0000 7f537e7fc700 1 -- 172.21.15.59:0/976663118 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f538408a5d0 con 0x7f5374003a00 2023-11-24T07:59:25.233 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:25.225+0000 7f538d245700 1 --2- 172.21.15.59:0/976663118 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f536c06b2c0 0x7f536c06d780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:25.234 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:25.225+0000 7f538d245700 1 --2- 172.21.15.59:0/976663118 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f536c06b2c0 0x7f536c06d780 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f538815d500 tx=0x7f5378001b20 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:59:25.234 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:25.227+0000 7f539bea3b80 1 Processor -- start 2023-11-24T07:59:25.234 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:25.227+0000 7f539bea3b80 1 -- start start 2023-11-24T07:59:25.234 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:25.227+0000 7f539bea3b80 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x55864ccbf080 0x55864cd05840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:25.234 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:25.227+0000 7f539bea3b80 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55864ccd4ed0 0x55864cd05d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:25.234 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:25.227+0000 7f539bea3b80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55864ccbc490 0x55864cce8e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:25.234 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:25.227+0000 7f539bea3b80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x55864cc32770 con 0x55864ccbc490 2023-11-24T07:59:25.234 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:25.227+0000 7f539bea3b80 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x55864cc19750 con 0x55864ccd4ed0 2023-11-24T07:59:25.234 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:25.227+0000 7f539bea3b80 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 v1 -- 0x55864cc17070 con 0x55864ccbf080 2023-11-24T07:59:25.234 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:25.227+0000 7f538e247700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55864ccbc490 0x55864cce8e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:25.234 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:25.228+0000 7f538e247700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55864ccbc490 0x55864cce8e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:55008/0 (socket says 172.21.15.59:55008) 2023-11-24T07:59:25.234 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:25.228+0000 7f538e247700 1 -- 172.21.15.59:0/4180975232 learned_addr learned my addr 172.21.15.59:0/4180975232 (peer_addr_for_me v2:172.21.15.59:0/0) 2023-11-24T07:59:25.235 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:25.228+0000 7f538d245700 1 --2- 172.21.15.59:0/4180975232 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55864ccd4ed0 0x55864cd05d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:25.235 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:25.228+0000 7f538da46700 1 --2- 172.21.15.59:0/4180975232 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x55864ccbf080 0x55864cd05840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:25.235 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:25.228+0000 7f538e247700 1 -- 172.21.15.59:0/4180975232 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x55864ccbf080 msgr2=0x55864cd05840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T07:59:25.235 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:25.228+0000 7f538e247700 1 --2- 172.21.15.59:0/4180975232 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x55864ccbf080 0x55864cd05840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:25.235 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:25.228+0000 7f538e247700 1 -- 172.21.15.59:0/4180975232 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55864ccd4ed0 msgr2=0x55864cd05d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T07:59:25.235 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:25.228+0000 7f538e247700 1 --2- 172.21.15.59:0/4180975232 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55864ccd4ed0 0x55864cd05d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:25.235 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:25.228+0000 7f538e247700 1 -- 172.21.15.59:0/4180975232 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55864ca8f5e0 con 0x55864ccbc490 2023-11-24T07:59:25.235 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:25.228+0000 7f538e247700 1 --2- 172.21.15.59:0/4180975232 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55864ccbc490 0x55864cce8e70 secure :-1 s=READY pgs=888 cs=0 l=1 rev1=1 crypto rx=0x7f53880b5e50 tx=0x7f5380009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fe30173f15d00dba server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:59:25.235 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:25.228+0000 7f536b7fe700 1 -- 172.21.15.59:0/4180975232 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f538000e040 con 0x55864ccbc490 2023-11-24T07:59:25.235 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:25.228+0000 7f536b7fe700 1 -- 172.21.15.59:0/4180975232 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f538000f040 con 0x55864ccbc490 2023-11-24T07:59:25.235 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:25.228+0000 7f536b7fe700 1 -- 172.21.15.59:0/4180975232 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5380003c20 con 0x55864ccbc490 2023-11-24T07:59:25.235 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:25.229+0000 7f539bea3b80 1 -- 172.21.15.59:0/4180975232 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55864ccbc490 msgr2=0x55864cce8e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:25.236 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:25.229+0000 7f539bea3b80 1 --2- 172.21.15.59:0/4180975232 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55864ccbc490 0x55864cce8e70 secure :-1 s=READY pgs=888 cs=0 l=1 rev1=1 crypto rx=0x7f53880b5e50 tx=0x7f5380009750 comp rx=0 tx=0).stop 2023-11-24T07:59:25.236 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:25.229+0000 7f539bea3b80 1 -- 172.21.15.59:0/4180975232 shutdown_connections 2023-11-24T07:59:25.236 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:25.229+0000 7f539bea3b80 1 --2- 172.21.15.59:0/4180975232 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55864ccbc490 0x55864cce8e70 unknown :-1 s=CLOSED pgs=888 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:25.236 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:25.229+0000 7f539bea3b80 1 --2- 172.21.15.59:0/4180975232 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x55864ccbf080 0x55864cd05840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:25.236 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:25.229+0000 7f539bea3b80 1 --2- 172.21.15.59:0/4180975232 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55864ccd4ed0 0x55864cd05d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:25.236 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:25.229+0000 7f539bea3b80 1 -- 172.21.15.59:0/4180975232 >> 172.21.15.59:0/4180975232 conn(0x55864cbed310 msgr2=0x55864cd03e70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:59:25.236 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:25.229+0000 7f539bea3b80 1 -- 172.21.15.59:0/4180975232 shutdown_connections 2023-11-24T07:59:25.236 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:25.229+0000 7f539bea3b80 1 -- 172.21.15.59:0/4180975232 wait complete. 2023-11-24T07:59:25.236 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:25.229+0000 7f539bea3b80 1 -- start start 2023-11-24T07:59:25.236 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:25.230+0000 7f539bea3b80 1 Processor -- start 2023-11-24T07:59:25.240 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:25.230+0000 7f539bea3b80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55864ccbc490 0x55864cce8170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:25.240 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:25.230+0000 7f539bea3b80 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55864ccbf080 0x55864cce86b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:25.240 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:25.230+0000 7f539bea3b80 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x55864ccd4ed0 0x55864ccb3330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:25.240 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:25.230+0000 7f539bea3b80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x55864cc12960 con 0x55864ccbc490 2023-11-24T07:59:25.240 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:25.230+0000 7f539bea3b80 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x55864cbcc510 con 0x55864ccbf080 2023-11-24T07:59:25.240 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:25.230+0000 7f539bea3b80 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 v1 -- 0x55864cbcc1b0 con 0x55864ccd4ed0 2023-11-24T07:59:25.240 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:25.231+0000 7f539bea3b80 10 client.0 _ll_register_callbacks cb 0x55864caf7a30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-24T07:59:25.240 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:25.231+0000 7f538da46700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55864ccbc490 0x55864cce8170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:25.240 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:25.231+0000 7f538da46700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55864ccbc490 0x55864cce8170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:55010/0 (socket says 172.21.15.59:55010) 2023-11-24T07:59:25.241 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:25.231+0000 7f538da46700 1 -- 172.21.15.59:0/2668017596 learned_addr learned my addr 172.21.15.59:0/2668017596 (peer_addr_for_me v2:172.21.15.59:0/0) 2023-11-24T07:59:25.241 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:25.231+0000 7f538d245700 1 --2- 172.21.15.59:0/2668017596 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55864ccbf080 0x55864cce86b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:25.241 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:25.231+0000 7f538e247700 1 --2- 172.21.15.59:0/2668017596 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x55864ccd4ed0 0x55864ccb3330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:25.241 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:25.231+0000 7f538da46700 1 -- 172.21.15.59:0/2668017596 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x55864ccd4ed0 msgr2=0x55864ccb3330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T07:59:25.241 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:25.231+0000 7f538da46700 1 --2- 172.21.15.59:0/2668017596 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x55864ccd4ed0 0x55864ccb3330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:25.241 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:25.231+0000 7f538da46700 1 -- 172.21.15.59:0/2668017596 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55864ccbf080 msgr2=0x55864cce86b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T07:59:25.241 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:25.231+0000 7f538da46700 1 --2- 172.21.15.59:0/2668017596 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55864ccbf080 0x55864cce86b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:25.241 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:25.231+0000 7f538da46700 1 -- 172.21.15.59:0/2668017596 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5380009cc0 con 0x55864ccbc490 2023-11-24T07:59:25.241 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:25.231+0000 7f538da46700 1 --2- 172.21.15.59:0/2668017596 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55864ccbc490 0x55864cce8170 secure :-1 s=READY pgs=889 cs=0 l=1 rev1=1 crypto rx=0x7f53840050a0 tx=0x7f5384058a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4fb80fbe3b441f4a server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:59:25.241 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:25.231+0000 7f5368ff9700 10 client.0 ms_handle_connect on v2:172.21.15.59:3300/0 2023-11-24T07:59:25.241 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:25.231+0000 7f5368ff9700 1 -- 172.21.15.59:0/2668017596 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5384059610 con 0x55864ccbc490 2023-11-24T07:59:25.241 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:25.231+0000 7f539bea3b80 1 -- 172.21.15.59:0/2668017596 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55864ccb1a40 con 0x55864ccbc490 2023-11-24T07:59:25.242 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:25.231+0000 7f5368ff9700 1 -- 172.21.15.59:0/2668017596 <== mon.0 v2:172.21.15.59:3300/0 2 ==== osd_map(74..74 src has 1..74) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f5384058f10 con 0x55864ccbc490 2023-11-24T07:59:25.242 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:25.231+0000 7f5368ff9700 1 -- 172.21.15.59:0/2668017596 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) v2 ==== 24+0+0 (secure 0 0 0) 0x7f538405d060 con 0x55864ccbc490 2023-11-24T07:59:25.242 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:25.231+0000 7f539bea3b80 10 client.6714 fetch_fsmap learned FSMap version 44 2023-11-24T07:59:25.242 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:25.231+0000 7f539bea3b80 1 -- 172.21.15.59:0/2668017596 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=2+}) v3 -- 0x55864ca3bd90 con 0x55864ccbc490 2023-11-24T07:59:25.242 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:25.232+0000 7f5368ff9700 1 -- 172.21.15.59:0/2668017596 <== mon.0 v2:172.21.15.59:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55864ccb1a40 con 0x55864ccbc490 2023-11-24T07:59:25.242 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:25.232+0000 7f5368ff9700 1 -- 172.21.15.59:0/2668017596 <== mon.0 v2:172.21.15.59:3300/0 5 ==== fsmap(e 44) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f5384088170 con 0x55864ccbc490 2023-11-24T07:59:25.242 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:25.232+0000 7f539bea3b80 10 client.6714 fetch_fsmap finished waiting for FSMap version 44 2023-11-24T07:59:25.242 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:25.232+0000 7f539bea3b80 10 client.6714 resolve_mds: resolved name 'c' to daemon mds.c 2023-11-24T07:59:25.242 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:25.232+0000 7f539bea3b80 20 client.6714 populate_metadata read hostname 'smithi059' 2023-11-24T07:59:25.242 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:25.232+0000 7f539bea3b80 1 --2- 172.21.15.59:0/2668017596 >> [v2:172.21.15.59:6833/1228649189,v1:172.21.15.59:6835/1228649189] conn(0x55864ccec9f0 0x55864ccecdd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:25.242 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:25.232+0000 7f539bea3b80 4 client.6714 mds_command: new command op to 6552 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-11-24T07:59:25.242 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:25.232+0000 7f539bea3b80 1 -- 172.21.15.59:0/2668017596 --> [v2:172.21.15.59:6833/1228649189,v1:172.21.15.59:6835/1228649189] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55864cc8d840 con 0x55864ccec9f0 2023-11-24T07:59:25.243 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:25.232+0000 7f538d245700 1 --2- 172.21.15.59:0/2668017596 >> [v2:172.21.15.59:6833/1228649189,v1:172.21.15.59:6835/1228649189] conn(0x55864ccec9f0 0x55864ccecdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:25.243 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:25.232+0000 7f5368ff9700 10 client.6714 ms_handle_connect on v2:172.21.15.59:6833/1228649189 2023-11-24T07:59:25.243 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:25.232+0000 7f538d245700 1 --2- 172.21.15.59:0/2668017596 >> [v2:172.21.15.59:6833/1228649189,v1:172.21.15.59:6835/1228649189] conn(0x55864ccec9f0 0x55864ccecdd0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c73be7e4befe5267 server_cookie=1c5df507c78c4b10 in_seq=0 out_seq=0 2023-11-24T07:59:25.243 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:25.233+0000 7f5368ff9700 1 -- 172.21.15.59:0/2668017596 <== mds.0 v2:172.21.15.59:6833/1228649189 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55864cc8d840 con 0x55864ccec9f0 2023-11-24T07:59:25.243 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:25.233+0000 7f5368ff9700 10 client.6714 handle_command_reply: tid=0 2023-11-24T07:59:25.243 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:25.233+0000 7f539bea3b80 1 client.6714 shutdown 2023-11-24T07:59:25.243 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:25.233+0000 7f539bea3b80 1 -- 172.21.15.59:0/2668017596 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55864ccbc490 msgr2=0x55864cce8170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:25.243 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:25.233+0000 7f539bea3b80 1 --2- 172.21.15.59:0/2668017596 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55864ccbc490 0x55864cce8170 secure :-1 s=READY pgs=889 cs=0 l=1 rev1=1 crypto rx=0x7f53840050a0 tx=0x7f5384058a50 comp rx=0 tx=0).stop 2023-11-24T07:59:25.243 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:25.233+0000 7f539bea3b80 1 -- 172.21.15.59:0/2668017596 shutdown_connections 2023-11-24T07:59:25.243 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:25.233+0000 7f539bea3b80 1 --2- 172.21.15.59:0/2668017596 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x55864ccd4ed0 0x55864ccb3330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:25.243 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:25.233+0000 7f539bea3b80 1 --2- 172.21.15.59:0/2668017596 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55864ccbc490 0x55864cce8170 unknown :-1 s=CLOSED pgs=889 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:25.243 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:25.233+0000 7f539bea3b80 1 --2- 172.21.15.59:0/2668017596 >> [v2:172.21.15.59:6833/1228649189,v1:172.21.15.59:6835/1228649189] conn(0x55864ccec9f0 0x55864ccecdd0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:25.244 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:25.233+0000 7f539bea3b80 1 --2- 172.21.15.59:0/2668017596 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55864ccbf080 0x55864cce86b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:25.244 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:25.233+0000 7f5368ff9700 0 client.6714 ms_handle_reset on v2:172.21.15.59:6833/1228649189 2023-11-24T07:59:25.244 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:25.233+0000 7f539bea3b80 1 -- 172.21.15.59:0/2668017596 >> 172.21.15.59:0/2668017596 conn(0x55864cbed310 msgr2=0x55864cd07c80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:59:25.244 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:25.233+0000 7f539bea3b80 1 -- 172.21.15.59:0/2668017596 shutdown_connections 2023-11-24T07:59:25.244 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:25.233+0000 7f539bea3b80 1 -- 172.21.15.59:0/2668017596 wait complete. 2023-11-24T07:59:25.244 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:25.233+0000 7f539bea3b80 20 client.6714 trim_cache size 0 max 16384 2023-11-24T07:59:25.253 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:25.251+0000 7f539bea3b80 1 Processor -- start 2023-11-24T07:59:25.253 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:25.251+0000 7f539bea3b80 1 -- start start 2023-11-24T07:59:25.255 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:25.252+0000 7f539bea3b80 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55864ccbc490 0x55864cd03f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:25.256 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:25.252+0000 7f539bea3b80 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x55864ccbf080 0x55864cd00d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:25.256 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:25.252+0000 7f539bea3b80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55864ccd4ed0 0x55864cd01280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:25.256 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:25.252+0000 7f539bea3b80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x55864ca918d0 con 0x55864ccd4ed0 2023-11-24T07:59:25.256 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:25.252+0000 7f539bea3b80 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x55864ccfd9c0 con 0x55864ccbc490 2023-11-24T07:59:25.256 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:25.252+0000 7f539bea3b80 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 v1 -- 0x55864cd06260 con 0x55864ccbf080 2023-11-24T07:59:25.256 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:25.252+0000 7f538e247700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55864ccd4ed0 0x55864cd01280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:25.256 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:25.252+0000 7f538e247700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55864ccd4ed0 0x55864cd01280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:55022/0 (socket says 172.21.15.59:55022) 2023-11-24T07:59:25.256 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:25.252+0000 7f538e247700 1 -- 172.21.15.59:0/836821277 learned_addr learned my addr 172.21.15.59:0/836821277 (peer_addr_for_me v2:172.21.15.59:0/0) 2023-11-24T07:59:25.256 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:25.252+0000 7f538da46700 1 --2- 172.21.15.59:0/836821277 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55864ccbc490 0x55864cd03f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:25.256 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:25.253+0000 7f538e247700 1 -- 172.21.15.59:0/836821277 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x55864ccbf080 msgr2=0x55864cd00d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T07:59:25.256 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:25.253+0000 7f538d245700 1 --2- 172.21.15.59:0/836821277 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x55864ccbf080 0x55864cd00d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:25.257 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:25.253+0000 7f538e247700 1 --2- 172.21.15.59:0/836821277 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x55864ccbf080 0x55864cd00d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:25.257 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:25.253+0000 7f538e247700 1 -- 172.21.15.59:0/836821277 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55864ccbc490 msgr2=0x55864cd03f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T07:59:25.257 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:25.253+0000 7f538e247700 1 --2- 172.21.15.59:0/836821277 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55864ccbc490 0x55864cd03f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:25.257 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:25.253+0000 7f538e247700 1 -- 172.21.15.59:0/836821277 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55864ca24c10 con 0x55864ccd4ed0 2023-11-24T07:59:25.257 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:25.253+0000 7f538da46700 1 --2- 172.21.15.59:0/836821277 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55864ccbc490 0x55864cd03f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T07:59:25.257 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:25.253+0000 7f538e247700 1 --2- 172.21.15.59:0/836821277 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55864ccd4ed0 0x55864cd01280 secure :-1 s=READY pgs=890 cs=0 l=1 rev1=1 crypto rx=0x7f5380004000 tx=0x7f5380019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4c05fed70b2503ba server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:59:25.257 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:25.253+0000 7f536bfff700 1 -- 172.21.15.59:0/836821277 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f538000e040 con 0x55864ccd4ed0 2023-11-24T07:59:25.257 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:25.253+0000 7f536bfff700 1 -- 172.21.15.59:0/836821277 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5380003ca0 con 0x55864ccd4ed0 2023-11-24T07:59:25.257 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:25.253+0000 7f536bfff700 1 -- 172.21.15.59:0/836821277 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5380004250 con 0x55864ccd4ed0 2023-11-24T07:59:25.257 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:25.253+0000 7f539bea3b80 1 -- 172.21.15.59:0/836821277 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55864ccd4ed0 msgr2=0x55864cd01280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:25.257 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:25.253+0000 7f539bea3b80 1 --2- 172.21.15.59:0/836821277 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55864ccd4ed0 0x55864cd01280 secure :-1 s=READY pgs=890 cs=0 l=1 rev1=1 crypto rx=0x7f5380004000 tx=0x7f5380019000 comp rx=0 tx=0).stop 2023-11-24T07:59:25.257 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:25.253+0000 7f539bea3b80 1 -- 172.21.15.59:0/836821277 shutdown_connections 2023-11-24T07:59:25.257 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:25.253+0000 7f539bea3b80 1 --2- 172.21.15.59:0/836821277 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55864ccd4ed0 0x55864cd01280 unknown :-1 s=CLOSED pgs=890 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:25.258 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:25.253+0000 7f539bea3b80 1 --2- 172.21.15.59:0/836821277 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55864ccbc490 0x55864cd03f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:25.258 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:25.253+0000 7f539bea3b80 1 --2- 172.21.15.59:0/836821277 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x55864ccbf080 0x55864cd00d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:25.258 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:25.253+0000 7f539bea3b80 1 -- 172.21.15.59:0/836821277 >> 172.21.15.59:0/836821277 conn(0x55864cbed310 msgr2=0x55864ccfcb40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:59:25.258 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:25.254+0000 7f539bea3b80 1 -- 172.21.15.59:0/836821277 shutdown_connections 2023-11-24T07:59:25.258 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:25.254+0000 7f539bea3b80 1 -- 172.21.15.59:0/836821277 wait complete. 2023-11-24T07:59:25.258 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:25.254+0000 7f539bea3b80 1 -- start start 2023-11-24T07:59:25.258 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:25.254+0000 7f539bea3b80 1 Processor -- start 2023-11-24T07:59:25.261 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:25.255+0000 7f539bea3b80 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55864ccbc490 0x55864cd04df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:25.261 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:25.255+0000 7f539bea3b80 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x55864ccbf080 0x55864cd05330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:25.261 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:25.255+0000 7f539bea3b80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55864ccd4ed0 0x55864cd07270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:25.262 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:25.255+0000 7f539bea3b80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x55864cd53400 con 0x55864ccd4ed0 2023-11-24T07:59:25.262 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:25.255+0000 7f539bea3b80 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x55864cd53860 con 0x55864ccbc490 2023-11-24T07:59:25.262 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:25.255+0000 7f539bea3b80 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 v1 -- 0x55864cd05940 con 0x55864ccbf080 2023-11-24T07:59:25.262 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:25.255+0000 7f539bea3b80 10 client.0 _ll_register_callbacks cb 0x55864caf7a30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-24T07:59:25.262 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:25.255+0000 7f538e247700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55864ccd4ed0 0x55864cd07270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:25.262 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:25.255+0000 7f538e247700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55864ccd4ed0 0x55864cd07270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:55026/0 (socket says 172.21.15.59:55026) 2023-11-24T07:59:25.262 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:25.255+0000 7f538e247700 1 -- 172.21.15.59:0/4019313774 learned_addr learned my addr 172.21.15.59:0/4019313774 (peer_addr_for_me v2:172.21.15.59:0/0) 2023-11-24T07:59:25.262 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:25.255+0000 7f538da46700 1 --2- 172.21.15.59:0/4019313774 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55864ccbc490 0x55864cd04df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:25.262 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:25.255+0000 7f538d245700 1 --2- 172.21.15.59:0/4019313774 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x55864ccbf080 0x55864cd05330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:25.262 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:25.256+0000 7f538e247700 1 -- 172.21.15.59:0/4019313774 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x55864ccbf080 msgr2=0x55864cd05330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T07:59:25.262 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:25.256+0000 7f538e247700 1 --2- 172.21.15.59:0/4019313774 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x55864ccbf080 0x55864cd05330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:25.263 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:25.256+0000 7f538e247700 1 -- 172.21.15.59:0/4019313774 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55864ccbc490 msgr2=0x55864cd04df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T07:59:25.263 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:25.256+0000 7f538e247700 1 --2- 172.21.15.59:0/4019313774 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55864ccbc490 0x55864cd04df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:25.263 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:25.256+0000 7f538e247700 1 -- 172.21.15.59:0/4019313774 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55864ca8f5e0 con 0x55864ccd4ed0 2023-11-24T07:59:25.263 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:25.256+0000 7f538d245700 1 --2- 172.21.15.59:0/4019313774 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x55864ccbf080 0x55864cd05330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T07:59:25.263 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:25.256+0000 7f538e247700 1 --2- 172.21.15.59:0/4019313774 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55864ccd4ed0 0x55864cd07270 secure :-1 s=READY pgs=891 cs=0 l=1 rev1=1 crypto rx=0x7f538000b010 tx=0x7f5380004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=35a18c2fab6fd030 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:59:25.263 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:25.256+0000 7f5368ff9700 10 client.0 ms_handle_connect on v2:172.21.15.59:3300/0 2023-11-24T07:59:25.263 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:25.256+0000 7f5368ff9700 1 -- 172.21.15.59:0/4019313774 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5380022030 con 0x55864ccd4ed0 2023-11-24T07:59:25.263 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:25.256+0000 7f5368ff9700 1 -- 172.21.15.59:0/4019313774 <== mon.0 v2:172.21.15.59:3300/0 2 ==== osd_map(74..74 src has 1..74) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f538001a070 con 0x55864ccd4ed0 2023-11-24T07:59:25.263 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:25.256+0000 7f539bea3b80 1 -- 172.21.15.59:0/4019313774 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f538000f040 con 0x55864ccd4ed0 2023-11-24T07:59:25.263 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:25.256+0000 7f5368ff9700 1 -- 172.21.15.59:0/4019313774 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) v2 ==== 24+0+0 (secure 0 0 0) 0x7f538000e040 con 0x55864ccd4ed0 2023-11-24T07:59:25.263 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:25.256+0000 7f539bea3b80 10 client.6720 fetch_fsmap learned FSMap version 44 2023-11-24T07:59:25.263 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:25.256+0000 7f539bea3b80 1 -- 172.21.15.59:0/4019313774 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=2+}) v3 -- 0x55864ca62200 con 0x55864ccd4ed0 2023-11-24T07:59:25.264 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:25.256+0000 7f5368ff9700 1 -- 172.21.15.59:0/4019313774 <== mon.0 v2:172.21.15.59:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f538000f040 con 0x55864ccd4ed0 2023-11-24T07:59:25.264 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:25.256+0000 7f5368ff9700 1 -- 172.21.15.59:0/4019313774 <== mon.0 v2:172.21.15.59:3300/0 5 ==== fsmap(e 44) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f5380032430 con 0x55864ccd4ed0 2023-11-24T07:59:25.264 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:25.256+0000 7f539bea3b80 10 client.6720 fetch_fsmap finished waiting for FSMap version 44 2023-11-24T07:59:25.264 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:25.256+0000 7f539bea3b80 10 client.6720 resolve_mds: resolved name 'c' to daemon mds.c 2023-11-24T07:59:25.264 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:25.256+0000 7f539bea3b80 20 client.6720 populate_metadata read hostname 'smithi059' 2023-11-24T07:59:25.264 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:25.256+0000 7f539bea3b80 1 --2- 172.21.15.59:0/4019313774 >> [v2:172.21.15.59:6833/1228649189,v1:172.21.15.59:6835/1228649189] conn(0x55864ccf9ae0 0x55864ccf9ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:25.264 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:25.256+0000 7f539bea3b80 4 client.6720 mds_command: new command op to 6552 tid=0 multi_id=0 [{"prefix": "status"}] 2023-11-24T07:59:25.264 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:25.256+0000 7f539bea3b80 1 -- 172.21.15.59:0/4019313774 --> [v2:172.21.15.59:6833/1228649189,v1:172.21.15.59:6835/1228649189] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55864cbd8f80 con 0x55864ccf9ae0 2023-11-24T07:59:25.264 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:25.256+0000 7f538da46700 1 --2- 172.21.15.59:0/4019313774 >> [v2:172.21.15.59:6833/1228649189,v1:172.21.15.59:6835/1228649189] conn(0x55864ccf9ae0 0x55864ccf9ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:25.264 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:25.257+0000 7f5368ff9700 10 client.6720 ms_handle_connect on v2:172.21.15.59:6833/1228649189 2023-11-24T07:59:25.264 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:25.257+0000 7f538da46700 1 --2- 172.21.15.59:0/4019313774 >> [v2:172.21.15.59:6833/1228649189,v1:172.21.15.59:6835/1228649189] conn(0x55864ccf9ae0 0x55864ccf9ec0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=91777eae3260090d server_cookie=9ac91698a4b33d8b in_seq=0 out_seq=0 2023-11-24T07:59:25.264 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:25.257+0000 7f5368ff9700 1 -- 172.21.15.59:0/4019313774 <== mds.0 v2:172.21.15.59:6833/1228649189 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+331 (crc 0 0 0) 0x55864cbd8f80 con 0x55864ccf9ae0 2023-11-24T07:59:25.264 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:25.257+0000 7f5368ff9700 10 client.6720 handle_command_reply: tid=0 2023-11-24T07:59:25.265 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:25.257+0000 7f539bea3b80 1 client.6720 shutdown 2023-11-24T07:59:25.265 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:25.257+0000 7f539bea3b80 1 -- 172.21.15.59:0/4019313774 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55864ccd4ed0 msgr2=0x55864cd07270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:25.265 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:25.257+0000 7f539bea3b80 1 --2- 172.21.15.59:0/4019313774 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55864ccd4ed0 0x55864cd07270 secure :-1 s=READY pgs=891 cs=0 l=1 rev1=1 crypto rx=0x7f538000b010 tx=0x7f5380004ad0 comp rx=0 tx=0).stop 2023-11-24T07:59:25.265 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:25.257+0000 7f539bea3b80 1 -- 172.21.15.59:0/4019313774 shutdown_connections 2023-11-24T07:59:25.265 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:25.257+0000 7f539bea3b80 1 --2- 172.21.15.59:0/4019313774 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55864ccd4ed0 0x55864cd07270 unknown :-1 s=CLOSED pgs=891 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:25.265 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:25.257+0000 7f539bea3b80 1 --2- 172.21.15.59:0/4019313774 >> [v2:172.21.15.59:6833/1228649189,v1:172.21.15.59:6835/1228649189] conn(0x55864ccf9ae0 0x55864ccf9ec0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:25.265 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:25.257+0000 7f539bea3b80 1 --2- 172.21.15.59:0/4019313774 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55864ccbc490 0x55864cd04df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:25.265 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:25.257+0000 7f539bea3b80 1 --2- 172.21.15.59:0/4019313774 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x55864ccbf080 0x55864cd05330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:25.265 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:25.257+0000 7f5368ff9700 0 client.6720 ms_handle_reset on v2:172.21.15.59:6833/1228649189 2023-11-24T07:59:25.265 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:25.257+0000 7f539bea3b80 1 -- 172.21.15.59:0/4019313774 >> 172.21.15.59:0/4019313774 conn(0x55864cbed310 msgr2=0x55864ccfcf00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:59:25.265 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:25.257+0000 7f539bea3b80 1 -- 172.21.15.59:0/4019313774 shutdown_connections 2023-11-24T07:59:25.265 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:25.257+0000 7f539bea3b80 1 -- 172.21.15.59:0/4019313774 wait complete. 2023-11-24T07:59:25.266 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:25.257+0000 7f539bea3b80 20 client.6720 trim_cache size 0 max 16384 2023-11-24T07:59:25.266 INFO:teuthology.orchestra.run.smithi059.stdout:{ 2023-11-24T07:59:25.266 INFO:teuthology.orchestra.run.smithi059.stdout: "cluster_fsid": "9f0e8741-b8f9-48d5-a2c3-09f45903303d", 2023-11-24T07:59:25.266 INFO:teuthology.orchestra.run.smithi059.stdout: "whoami": 0, 2023-11-24T07:59:25.266 INFO:teuthology.orchestra.run.smithi059.stdout: "id": 6552, 2023-11-24T07:59:25.266 INFO:teuthology.orchestra.run.smithi059.stdout: "want_state": "up:active", 2023-11-24T07:59:25.266 INFO:teuthology.orchestra.run.smithi059.stdout: "state": "up:active", 2023-11-24T07:59:25.266 INFO:teuthology.orchestra.run.smithi059.stdout: "fs_name": "cephfs", 2023-11-24T07:59:25.267 INFO:teuthology.orchestra.run.smithi059.stdout: "rank_uptime": 10.618859519000001, 2023-11-24T07:59:25.267 INFO:teuthology.orchestra.run.smithi059.stdout: "mdsmap_epoch": 44, 2023-11-24T07:59:25.267 INFO:teuthology.orchestra.run.smithi059.stdout: "osdmap_epoch": 74, 2023-11-24T07:59:25.267 INFO:teuthology.orchestra.run.smithi059.stdout: "osdmap_epoch_barrier": 74, 2023-11-24T07:59:25.267 INFO:teuthology.orchestra.run.smithi059.stdout: "uptime": 16.514279896000001 2023-11-24T07:59:25.267 INFO:teuthology.orchestra.run.smithi059.stdout:} 2023-11-24T07:59:25.268 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:25.258+0000 7f538fcaa700 1 -- 172.21.15.59:0/976663118 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f536c06b2c0 msgr2=0x7f536c06d780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:25.268 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:25.258+0000 7f538fcaa700 1 --2- 172.21.15.59:0/976663118 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f536c06b2c0 0x7f536c06d780 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f538815d500 tx=0x7f5378001b20 comp rx=0 tx=0).stop 2023-11-24T07:59:25.268 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:25.258+0000 7f538fcaa700 1 -- 172.21.15.59:0/976663118 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5374003a00 msgr2=0x7f538815d570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:25.268 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:25.258+0000 7f538fcaa700 1 --2- 172.21.15.59:0/976663118 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5374003a00 0x7f538815d570 secure :-1 s=READY pgs=887 cs=0 l=1 rev1=1 crypto rx=0x7f5384000c00 tx=0x7f5384001120 comp rx=0 tx=0).stop 2023-11-24T07:59:25.268 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:25.258+0000 7f538fcaa700 1 -- 172.21.15.59:0/976663118 shutdown_connections 2023-11-24T07:59:25.268 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:25.258+0000 7f538fcaa700 1 --2- 172.21.15.59:0/976663118 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f536c06b2c0 0x7f536c06d780 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:25.268 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:25.258+0000 7f538fcaa700 1 --2- 172.21.15.59:0/976663118 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f53881585f0 0x7f5388158a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:25.268 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:25.258+0000 7f538fcaa700 1 --2- 172.21.15.59:0/976663118 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5374003a00 0x7f538815d570 unknown :-1 s=CLOSED pgs=887 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:25.268 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:25.258+0000 7f538fcaa700 1 --2- 172.21.15.59:0/976663118 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f53880c64c0 0x7f53880c7fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:25.268 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:25.258+0000 7f538fcaa700 1 -- 172.21.15.59:0/976663118 >> 172.21.15.59:0/976663118 conn(0x7f53880b2840 msgr2=0x7f538806e430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:59:25.269 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:25.258+0000 7f538fcaa700 1 -- 172.21.15.59:0/976663118 shutdown_connections 2023-11-24T07:59:25.269 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:25.258+0000 7f538fcaa700 1 -- 172.21.15.59:0/976663118 wait complete. 2023-11-24T07:59:25.272 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-11-24T07:59:25.397 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:25.395+0000 7f15b7ec6700 1 Processor -- start 2023-11-24T07:59:25.397 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:25.395+0000 7f15b7ec6700 1 -- start start 2023-11-24T07:59:25.401 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:25.396+0000 7f15b7ec6700 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f15b006b620 0x7f15b00b3a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:25.402 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:25.396+0000 7f15b7ec6700 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f15b00b5ab0 0x7f15b00b3f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:25.402 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:25.396+0000 7f15b7ec6700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f15b01478c0 0x7f15b0161ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:25.402 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:25.396+0000 7f15b7ec6700 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f15b0071f50 con 0x7f15b01478c0 2023-11-24T07:59:25.402 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:25.396+0000 7f15b7ec6700 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f15b0072230 con 0x7f15b006b620 2023-11-24T07:59:25.402 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:25.396+0000 7f15b7ec6700 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f15b0072510 con 0x7f15b00b5ab0 2023-11-24T07:59:25.402 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:25.396+0000 7f15b6463700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f15b01478c0 0x7f15b0161ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:25.402 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:25.396+0000 7f15b5c62700 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f15b006b620 0x7f15b00b3a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:25.402 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:25.396+0000 7f15b5461700 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f15b00b5ab0 0x7f15b00b3f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:25.403 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:25.396+0000 7f15b6463700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f15b01478c0 0x7f15b0161ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:55034/0 (socket says 172.21.15.59:55034) 2023-11-24T07:59:25.403 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:25.396+0000 7f15b6463700 1 -- 172.21.15.59:0/267554036 learned_addr learned my addr 172.21.15.59:0/267554036 (peer_addr_for_me v2:172.21.15.59:0/0) 2023-11-24T07:59:25.403 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:25.397+0000 7f15b6463700 1 -- 172.21.15.59:0/267554036 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f15b00b5ab0 msgr2=0x7f15b00b3f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T07:59:25.403 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:25.397+0000 7f15b6463700 1 --2- 172.21.15.59:0/267554036 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f15b00b5ab0 0x7f15b00b3f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:25.403 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:25.397+0000 7f15b6463700 1 -- 172.21.15.59:0/267554036 >> v2:172.21.15.76:3300/0 conn(0x7f15b006b620 msgr2=0x7f15b00b3a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T07:59:25.403 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:25.397+0000 7f15b6463700 1 --2- 172.21.15.59:0/267554036 >> v2:172.21.15.76:3300/0 conn(0x7f15b006b620 0x7f15b00b3a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:25.404 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:25.397+0000 7f15b6463700 1 -- 172.21.15.59:0/267554036 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f15b00727f0 con 0x7f15b01478c0 2023-11-24T07:59:25.404 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:25.397+0000 7f15b5c62700 1 --2- 172.21.15.59:0/267554036 >> v2:172.21.15.76:3300/0 conn(0x7f15b006b620 0x7f15b00b3a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-24T07:59:25.404 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:25.397+0000 7f15b5461700 1 --2- 172.21.15.59:0/267554036 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f15b00b5ab0 0x7f15b00b3f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-24T07:59:25.404 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:25.397+0000 7f15b6463700 1 --2- 172.21.15.59:0/267554036 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f15b01478c0 0x7f15b0161ba0 secure :-1 s=READY pgs=892 cs=0 l=1 rev1=1 crypto rx=0x7f15b006d2e0 tx=0x7f15a800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3068eeffb0a2894f server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:59:25.404 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:25.398+0000 7f15b4c60700 1 -- 172.21.15.59:0/267554036 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15a800e070 con 0x7f15b01478c0 2023-11-24T07:59:25.404 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:25.398+0000 7f15b4c60700 1 -- 172.21.15.59:0/267554036 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f15a800b910 con 0x7f15b01478c0 2023-11-24T07:59:25.404 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:25.398+0000 7f15b4c60700 1 -- 172.21.15.59:0/267554036 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15a800bcc0 con 0x7f15b01478c0 2023-11-24T07:59:25.405 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:25.399+0000 7f15b7ec6700 1 -- 172.21.15.59:0/267554036 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f15b01478c0 msgr2=0x7f15b0161ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:25.405 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:25.399+0000 7f15b7ec6700 1 --2- 172.21.15.59:0/267554036 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f15b01478c0 0x7f15b0161ba0 secure :-1 s=READY pgs=892 cs=0 l=1 rev1=1 crypto rx=0x7f15b006d2e0 tx=0x7f15a800b330 comp rx=0 tx=0).stop 2023-11-24T07:59:25.405 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:25.399+0000 7f15b7ec6700 1 -- 172.21.15.59:0/267554036 shutdown_connections 2023-11-24T07:59:25.405 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:25.399+0000 7f15b7ec6700 1 --2- 172.21.15.59:0/267554036 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f15b01478c0 0x7f15b0161ba0 unknown :-1 s=CLOSED pgs=892 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:25.405 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:25.399+0000 7f15b7ec6700 1 --2- 172.21.15.59:0/267554036 >> v2:172.21.15.76:3300/0 conn(0x7f15b006b620 0x7f15b00b3a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:25.405 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:25.399+0000 7f15b7ec6700 1 --2- 172.21.15.59:0/267554036 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f15b00b5ab0 0x7f15b00b3f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:25.406 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:25.399+0000 7f15b7ec6700 1 -- 172.21.15.59:0/267554036 >> 172.21.15.59:0/267554036 conn(0x7f15b00b2740 msgr2=0x7f15b00b2b20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:59:25.406 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:25.399+0000 7f15b7ec6700 1 -- 172.21.15.59:0/267554036 shutdown_connections 2023-11-24T07:59:25.406 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:25.399+0000 7f15b7ec6700 1 -- 172.21.15.59:0/267554036 wait complete. 2023-11-24T07:59:25.406 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:25.400+0000 7f15b7ec6700 1 Processor -- start 2023-11-24T07:59:25.406 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:25.400+0000 7f15b7ec6700 1 -- start start 2023-11-24T07:59:25.410 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:25.401+0000 7f15b7ec6700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f15b006b620 0x7f15b00c7af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:25.410 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:25.401+0000 7f15b7ec6700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f15b00b5ab0 0x7f15b01444a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:25.410 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:25.401+0000 7f15b7ec6700 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f15b01478c0 0x7f15b01e5930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:25.410 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:25.401+0000 7f15b7ec6700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15b00c8320 con 0x7f15b00b5ab0 2023-11-24T07:59:25.410 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:25.401+0000 7f15b7ec6700 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15b00c8600 con 0x7f15b006b620 2023-11-24T07:59:25.410 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:25.401+0000 7f15b7ec6700 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f15b01f3940 con 0x7f15b01478c0 2023-11-24T07:59:25.410 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:25.401+0000 7f15b5461700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f15b00b5ab0 0x7f15b01444a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:25.411 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:25.401+0000 7f15b5461700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f15b00b5ab0 0x7f15b01444a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:55038/0 (socket says 172.21.15.59:55038) 2023-11-24T07:59:25.411 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:25.401+0000 7f15b5461700 1 -- 172.21.15.59:0/1714411175 learned_addr learned my addr 172.21.15.59:0/1714411175 (peer_addr_for_me v2:172.21.15.59:0/0) 2023-11-24T07:59:25.411 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:25.401+0000 7f15b5c62700 1 --2- 172.21.15.59:0/1714411175 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f15b006b620 0x7f15b00c7af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:25.411 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:25.401+0000 7f15b6463700 1 --2- 172.21.15.59:0/1714411175 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f15b01478c0 0x7f15b01e5930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:25.411 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:25.401+0000 7f15b5461700 1 -- 172.21.15.59:0/1714411175 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f15b01478c0 msgr2=0x7f15b01e5930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:25.411 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:25.401+0000 7f15b5461700 1 --2- 172.21.15.59:0/1714411175 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f15b01478c0 0x7f15b01e5930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:25.411 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:25.401+0000 7f15b5461700 1 -- 172.21.15.59:0/1714411175 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f15b006b620 msgr2=0x7f15b00c7af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:25.411 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:25.401+0000 7f15b5461700 1 --2- 172.21.15.59:0/1714411175 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f15b006b620 0x7f15b00c7af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:25.411 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:25.401+0000 7f15b5461700 1 -- 172.21.15.59:0/1714411175 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f15ac0094a0 con 0x7f15b00b5ab0 2023-11-24T07:59:25.411 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:25.401+0000 7f15b5c62700 1 --2- 172.21.15.59:0/1714411175 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f15b006b620 0x7f15b00c7af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-24T07:59:25.411 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:25.401+0000 7f15b5461700 1 --2- 172.21.15.59:0/1714411175 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f15b00b5ab0 0x7f15b01444a0 secure :-1 s=READY pgs=893 cs=0 l=1 rev1=1 crypto rx=0x7f15a000ebe0 tx=0x7f15a0009e60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:59:25.411 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:25.402+0000 7f15a67fc700 1 -- 172.21.15.59:0/1714411175 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15a000c890 con 0x7f15b00b5ab0 2023-11-24T07:59:25.412 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:25.402+0000 7f15a67fc700 1 -- 172.21.15.59:0/1714411175 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f15a000c9f0 con 0x7f15b00b5ab0 2023-11-24T07:59:25.412 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:25.402+0000 7f15b7ec6700 1 -- 172.21.15.59:0/1714411175 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f15a00094a0 con 0x7f15b00b5ab0 2023-11-24T07:59:25.412 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:25.402+0000 7f15a67fc700 1 -- 172.21.15.59:0/1714411175 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15a000cda0 con 0x7f15b00b5ab0 2023-11-24T07:59:25.412 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:25.402+0000 7f15b7ec6700 1 -- 172.21.15.59:0/1714411175 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f15a8009b30 con 0x7f15b00b5ab0 2023-11-24T07:59:25.412 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:25.403+0000 7f15b7ec6700 1 -- 172.21.15.59:0/1714411175 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f15b0068670 con 0x7f15b00b5ab0 2023-11-24T07:59:25.412 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:25.403+0000 7f15a67fc700 1 -- 172.21.15.59:0/1714411175 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 10) v1 ==== 89588+0+0 (secure 0 0 0) 0x7f15a0013470 con 0x7f15b00b5ab0 2023-11-24T07:59:25.412 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:25.404+0000 7f15a67fc700 1 --2- 172.21.15.59:0/1714411175 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f159406b2c0 0x7f159406d780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:25.412 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:25.404+0000 7f15a67fc700 1 -- 172.21.15.59:0/1714411175 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f15a0089bc0 con 0x7f15b00b5ab0 2023-11-24T07:59:25.412 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:25.404+0000 7f15b5c62700 1 --2- 172.21.15.59:0/1714411175 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f159406b2c0 0x7f159406d780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:25.412 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:25.404+0000 7f15b5c62700 1 --2- 172.21.15.59:0/1714411175 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f159406b2c0 0x7f159406d780 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f15ac009d40 tx=0x7f15ac008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:59:25.412 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:25.408+0000 7f15a67fc700 1 -- 172.21.15.59:0/1714411175 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f15a0054690 con 0x7f15b00b5ab0 2023-11-24T07:59:25.657 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:25.656+0000 7f15b7ec6700 1 -- 172.21.15.59:0/1714411175 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f15b006bd40 con 0x7f15b00b5ab0 2023-11-24T07:59:25.658 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:25.656+0000 7f15a67fc700 1 -- 172.21.15.59:0/1714411175 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v44) v1 ==== 53+0+73 (secure 0 0 0) 0x7f15a0017060 con 0x7f15b00b5ab0 2023-11-24T07:59:25.658 INFO:teuthology.orchestra.run.smithi059.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-11-24T07:59:25.660 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:25.658+0000 7f15b7ec6700 1 -- 172.21.15.59:0/1714411175 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f159406b2c0 msgr2=0x7f159406d780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:25.660 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:25.658+0000 7f15b7ec6700 1 --2- 172.21.15.59:0/1714411175 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f159406b2c0 0x7f159406d780 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f15ac009d40 tx=0x7f15ac008040 comp rx=0 tx=0).stop 2023-11-24T07:59:25.660 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:25.658+0000 7f15b7ec6700 1 -- 172.21.15.59:0/1714411175 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f15b00b5ab0 msgr2=0x7f15b01444a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:25.660 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:25.658+0000 7f15b7ec6700 1 --2- 172.21.15.59:0/1714411175 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f15b00b5ab0 0x7f15b01444a0 secure :-1 s=READY pgs=893 cs=0 l=1 rev1=1 crypto rx=0x7f15a000ebe0 tx=0x7f15a0009e60 comp rx=0 tx=0).stop 2023-11-24T07:59:25.660 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:25.658+0000 7f15b7ec6700 1 -- 172.21.15.59:0/1714411175 shutdown_connections 2023-11-24T07:59:25.660 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:25.658+0000 7f15b7ec6700 1 --2- 172.21.15.59:0/1714411175 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f159406b2c0 0x7f159406d780 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:25.660 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:25.658+0000 7f15b7ec6700 1 --2- 172.21.15.59:0/1714411175 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f15b01478c0 0x7f15b01e5930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:25.661 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:25.658+0000 7f15b7ec6700 1 --2- 172.21.15.59:0/1714411175 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f15b006b620 0x7f15b00c7af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:25.661 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:25.658+0000 7f15b7ec6700 1 --2- 172.21.15.59:0/1714411175 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f15b00b5ab0 0x7f15b01444a0 unknown :-1 s=CLOSED pgs=893 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:25.661 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:25.658+0000 7f15b7ec6700 1 -- 172.21.15.59:0/1714411175 >> 172.21.15.59:0/1714411175 conn(0x7f15b00b2740 msgr2=0x7f15b0146340 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:59:25.661 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:25.658+0000 7f15b7ec6700 1 -- 172.21.15.59:0/1714411175 shutdown_connections 2023-11-24T07:59:25.661 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:25.658+0000 7f15b7ec6700 1 -- 172.21.15.59:0/1714411175 wait complete. 2023-11-24T07:59:25.672 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2023-11-24T07:59:25.672 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-11-24T07:59:25.672 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi059.front.sepia.ceph.com 2023-11-24T07:59:25.672 INFO:tasks.cephfs.mount:self.client.name = client.0 2023-11-24T07:59:25.672 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-11-24T07:59:25.672 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2023-11-24T07:59:25.673 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2023-11-24T07:59:25.673 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-11-24T07:59:25.673 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-11-24T07:59:25.673 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T07:59:25.673 DEBUG:teuthology.orchestra.run.smithi059:> ip addr 2023-11-24T07:59:25.691 INFO:teuthology.orchestra.run.smithi059.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-11-24T07:59:25.691 INFO:teuthology.orchestra.run.smithi059.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-11-24T07:59:25.691 INFO:teuthology.orchestra.run.smithi059.stdout: inet 127.0.0.1/8 scope host lo 2023-11-24T07:59:25.692 INFO:teuthology.orchestra.run.smithi059.stdout: valid_lft forever preferred_lft forever 2023-11-24T07:59:25.692 INFO:teuthology.orchestra.run.smithi059.stdout: inet6 ::1/128 scope host 2023-11-24T07:59:25.692 INFO:teuthology.orchestra.run.smithi059.stdout: valid_lft forever preferred_lft forever 2023-11-24T07:59:25.692 INFO:teuthology.orchestra.run.smithi059.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-11-24T07:59:25.692 INFO:teuthology.orchestra.run.smithi059.stdout: link/ether 0c:c4:7a:6b:fe:ca brd ff:ff:ff:ff:ff:ff 2023-11-24T07:59:25.692 INFO:teuthology.orchestra.run.smithi059.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-11-24T07:59:25.692 INFO:teuthology.orchestra.run.smithi059.stdout: link/ether 0c:c4:7a:6b:fe:cb brd ff:ff:ff:ff:ff:ff 2023-11-24T07:59:25.693 INFO:teuthology.orchestra.run.smithi059.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2023-11-24T07:59:25.693 INFO:teuthology.orchestra.run.smithi059.stdout: link/ether 0c:c4:7a:bd:19:2a brd ff:ff:ff:ff:ff:ff 2023-11-24T07:59:25.693 INFO:teuthology.orchestra.run.smithi059.stdout: inet 172.21.15.59/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2023-11-24T07:59:25.693 INFO:teuthology.orchestra.run.smithi059.stdout: valid_lft 42202sec preferred_lft 42202sec 2023-11-24T07:59:25.693 INFO:teuthology.orchestra.run.smithi059.stdout: inet6 fe80::ec4:7aff:febd:192a/64 scope link 2023-11-24T07:59:25.694 INFO:teuthology.orchestra.run.smithi059.stdout: valid_lft forever preferred_lft forever 2023-11-24T07:59:25.694 INFO:teuthology.orchestra.run.smithi059.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-11-24T07:59:25.694 INFO:teuthology.orchestra.run.smithi059.stdout: link/ether 0c:c4:7a:bd:19:2b brd ff:ff:ff:ff:ff:ff 2023-11-24T07:59:25.695 INFO:teuthology.orchestra.run.smithi059.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-11-24T07:59:25.696 INFO:teuthology.orchestra.run.smithi059.stdout: link/ether 72:f2:ce:27:f6:18 brd ff:ff:ff:ff:ff:ff 2023-11-24T07:59:25.696 INFO:teuthology.orchestra.run.smithi059.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-11-24T07:59:25.696 INFO:teuthology.orchestra.run.smithi059.stdout: valid_lft forever preferred_lft forever 2023-11-24T07:59:25.696 INFO:teuthology.orchestra.run.smithi059.stdout: inet6 fe80::64c1:62ff:feb9:5b59/64 scope link 2023-11-24T07:59:25.696 INFO:teuthology.orchestra.run.smithi059.stdout: valid_lft forever preferred_lft forever 2023-11-24T07:59:25.696 INFO:teuthology.orchestra.run.smithi059.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-11-24T07:59:25.696 INFO:teuthology.orchestra.run.smithi059.stdout: link/ether 92:6a:7d:e0:5d:9b brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2023-11-24T07:59:25.696 INFO:teuthology.orchestra.run.smithi059.stdout: inet6 fe80::906a:7dff:fee0:5d9b/64 scope link 2023-11-24T07:59:25.696 INFO:teuthology.orchestra.run.smithi059.stdout: valid_lft forever preferred_lft forever 2023-11-24T07:59:25.696 INFO:teuthology.orchestra.run.smithi059.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-11-24T07:59:25.696 INFO:teuthology.orchestra.run.smithi059.stdout: link/ether 72:f2:ce:27:f6:18 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2023-11-24T07:59:25.697 INFO:teuthology.orchestra.run.smithi059.stdout: inet6 fe80::70f2:ceff:fe27:f618/64 scope link 2023-11-24T07:59:25.697 INFO:teuthology.orchestra.run.smithi059.stdout: valid_lft forever preferred_lft forever 2023-11-24T07:59:25.697 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T07:59:25.697 DEBUG:teuthology.orchestra.run.smithi059:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-11-24T07:59:25.757 INFO:teuthology.orchestra.run.smithi059.stdout:1 2023-11-24T07:59:25.758 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T07:59:25.759 DEBUG:teuthology.orchestra.run.smithi059:> route 2023-11-24T07:59:25.817 INFO:teuthology.orchestra.run.smithi059.stdout:Kernel IP routing table 2023-11-24T07:59:25.817 INFO:teuthology.orchestra.run.smithi059.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-11-24T07:59:25.817 INFO:teuthology.orchestra.run.smithi059.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2023-11-24T07:59:25.817 INFO:teuthology.orchestra.run.smithi059.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2023-11-24T07:59:25.818 INFO:teuthology.orchestra.run.smithi059.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-11-24T07:59:25.818 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T07:59:25.818 DEBUG:teuthology.orchestra.run.smithi059:> (cd / && exec bash -c ' 2023-11-24T07:59:25.818 DEBUG:teuthology.orchestra.run.smithi059:> set -e 2023-11-24T07:59:25.818 DEBUG:teuthology.orchestra.run.smithi059:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2023-11-24T07:59:25.818 DEBUG:teuthology.orchestra.run.smithi059:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2023-11-24T07:59:25.819 DEBUG:teuthology.orchestra.run.smithi059:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2023-11-24T07:59:25.819 DEBUG:teuthology.orchestra.run.smithi059:> ') 2023-11-24T07:59:25.924 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T07:59:25.925 DEBUG:teuthology.orchestra.run.smithi059:> ip netns list 2023-11-24T07:59:25.942 INFO:teuthology.orchestra.run.smithi059.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2023-11-24T07:59:25.942 INFO:teuthology.orchestra.run.smithi059.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-11-24T07:59:25.942 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-11-24T07:59:25.942 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T07:59:25.943 DEBUG:teuthology.orchestra.run.smithi059:> sudo ip link set brx.0 up 2023-11-24T07:59:26.009 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-11-24T07:59:26.009 DEBUG:teuthology.orchestra.run.smithi059:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2023-11-24T07:59:26.067 INFO:teuthology.orchestra.run.smithi059.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2023-11-24T07:59:26.067 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-11-24T07:59:26.067 DEBUG:teuthology.orchestra.run.smithi059:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2023-11-24T07:59:26.124 INFO:tasks.cephfs.kernel_mount:mounting using device: 0@.cephfs=/ 2023-11-24T07:59:26.124 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T07:59:26.124 DEBUG:teuthology.orchestra.run.smithi059:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 0@.cephfs=/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-11-24T07:59:26.213 INFO:teuthology.orchestra.run.smithi059.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-11-24T07:59:26.214 INFO:teuthology.orchestra.run.smithi059.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-11-24T07:59:26.242 INFO:teuthology.orchestra.run.smithi059.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-11-24T07:59:26.242 INFO:teuthology.orchestra.run.smithi059.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-11-24T07:59:26.242 INFO:teuthology.orchestra.run.smithi059.stdout:mount.ceph: resolved to: "172.21.15.59:3300,172.21.15.76:3300,172.21.15.76:3301" 2023-11-24T07:59:26.242 INFO:teuthology.orchestra.run.smithi059.stdout:mount.ceph: trying mount with new device syntax: 0@9f0e8741-b8f9-48d5-a2c3-09f45903303d.cephfs=/ 2023-11-24T07:59:26.242 INFO:teuthology.orchestra.run.smithi059.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,ms_mode=prefer-crc,key=0,mon_addr=172.21.15.59:3300/172.21.15.76:3300/172.21.15.76:3301" will pass to kernel 2023-11-24T07:59:26.243 INFO:teuthology.orchestra.run.smithi059.stdout:mount.ceph: v2 mount support check returned 2 2023-11-24T07:59:26.243 INFO:teuthology.orchestra.run.smithi059.stdout:mount.ceph: kernel does not support v2 syntax 2023-11-24T07:59:26.243 INFO:teuthology.orchestra.run.smithi059.stdout:mount.ceph: switching to using v1 address with old syntax 2023-11-24T07:59:26.315 INFO:teuthology.orchestra.run.smithi059.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-11-24T07:59:26.315 INFO:teuthology.orchestra.run.smithi059.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-11-24T07:59:26.316 INFO:teuthology.orchestra.run.smithi059.stdout:mount.ceph: resolved to: "172.21.15.59:3300,172.21.15.76:3300,172.21.15.76:3301" 2023-11-24T07:59:26.316 INFO:teuthology.orchestra.run.smithi059.stdout:mount.ceph: trying mount with new device syntax: 0@9f0e8741-b8f9-48d5-a2c3-09f45903303d.cephfs=/ 2023-11-24T07:59:26.316 INFO:teuthology.orchestra.run.smithi059.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,ms_mode=prefer-crc,key=0,mon_addr=172.21.15.59:3300/172.21.15.76:3300/172.21.15.76:3301" will pass to kernel 2023-11-24T07:59:26.316 INFO:teuthology.orchestra.run.smithi059.stdout:mount.ceph: v2 mount support check returned 2 2023-11-24T07:59:26.316 INFO:teuthology.orchestra.run.smithi059.stdout:mount.ceph: kernel does not support v2 syntax 2023-11-24T07:59:26.317 INFO:teuthology.orchestra.run.smithi059.stdout:mount.ceph: switching to using v1 address with old syntax 2023-11-24T07:59:26.317 INFO:teuthology.orchestra.run.smithi059.stdout:mount.ceph: resolved to: "172.21.15.59:6789,172.21.15.76:6789,172.21.15.76:6790" 2023-11-24T07:59:26.317 INFO:teuthology.orchestra.run.smithi059.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.59:6789,172.21.15.76:6789,172.21.15.76:6790:/ 2023-11-24T07:59:26.317 INFO:teuthology.orchestra.run.smithi059.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,key=0,mds_namespace=cephfs,fsid=9f0e8741-b8f9-48d5-a2c3-09f45903303d" will pass to kernel 2023-11-24T07:59:26.318 INFO:tasks.cephfs.kernel_mount:mount command passed 2023-11-24T07:59:26.319 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T07:59:26.319 DEBUG:teuthology.orchestra.run.smithi059:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2023-11-24T07:59:26.347 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T07:59:26.347 DEBUG:teuthology.orchestra.run.smithi059:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-11-24T07:59:26.431 INFO:teuthology.orchestra.run.smithi059.stdout:client5696 2023-11-24T07:59:26.432 DEBUG:teuthology.orchestra.run.smithi059:> set -ex 2023-11-24T07:59:26.432 DEBUG:teuthology.orchestra.run.smithi059:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-24T07:59:26.489 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-11-24T07:59:26.654 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:26.652+0000 7fa7389f2700 1 Processor -- start 2023-11-24T07:59:26.654 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:26.652+0000 7fa7389f2700 1 -- start start 2023-11-24T07:59:26.656 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:26.653+0000 7fa7389f2700 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fa73406afd0 0x7fa7340d16e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:26.657 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:26.653+0000 7fa7389f2700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fa7340ada10 0x7fa7340adea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:26.657 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:26.653+0000 7fa7389f2700 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa734070490 con 0x7fa7340affb0 2023-11-24T07:59:26.657 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:26.653+0000 7fa7389f2700 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa7340b0430 con 0x7fa7340ada10 2023-11-24T07:59:26.657 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:26.653+0000 7fa7389f2700 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa7340af9f0 con 0x7fa73406afd0 2023-11-24T07:59:26.657 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:26.653+0000 7fa732d9d700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fa7340ada10 0x7fa7340adea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:26.657 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:26.653+0000 7fa732d9d700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fa7340ada10 0x7fa7340adea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:55052/0 (socket says 172.21.15.59:55052) 2023-11-24T07:59:26.657 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:26.653+0000 7fa732d9d700 1 -- 172.21.15.59:0/3645473314 learned_addr learned my addr 172.21.15.59:0/3645473314 (peer_addr_for_me v2:172.21.15.59:0/0) 2023-11-24T07:59:26.657 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:26.654+0000 7fa73259c700 1 --2- 172.21.15.59:0/3645473314 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fa73406afd0 0x7fa7340d16e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:26.657 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:26.654+0000 7fa732d9d700 1 -- 172.21.15.59:0/3645473314 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fa73406afd0 msgr2=0x7fa7340d16e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T07:59:26.657 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:26.654+0000 7fa732d9d700 1 --2- 172.21.15.59:0/3645473314 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fa73406afd0 0x7fa7340d16e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:26.658 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:26.654+0000 7fa732d9d700 1 -- 172.21.15.59:0/3645473314 >> v1:172.21.15.76:6789/0 conn(0x7fa7340affb0 legacy=0x7fa73406b3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T07:59:26.658 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:26.654+0000 7fa732d9d700 1 -- 172.21.15.59:0/3645473314 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa7340afcd0 con 0x7fa7340ada10 2023-11-24T07:59:26.658 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:26.654+0000 7fa73259c700 1 --2- 172.21.15.59:0/3645473314 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fa73406afd0 0x7fa7340d16e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T07:59:26.658 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:26.654+0000 7fa732d9d700 1 --2- 172.21.15.59:0/3645473314 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa7340ada10 0x7fa7340adea0 secure :-1 s=READY pgs=894 cs=0 l=1 rev1=1 crypto rx=0x7fa72c009e50 tx=0x7fa72c00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aa9fe917a6832380 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:59:26.658 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:26.654+0000 7fa73159a700 1 -- 172.21.15.59:0/3645473314 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa72c00e070 con 0x7fa7340ada10 2023-11-24T07:59:26.658 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:26.654+0000 7fa73159a700 1 -- 172.21.15.59:0/3645473314 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa72c003a20 con 0x7fa7340ada10 2023-11-24T07:59:26.658 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:26.654+0000 7fa73159a700 1 -- 172.21.15.59:0/3645473314 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa72c003dd0 con 0x7fa7340ada10 2023-11-24T07:59:26.658 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:26.655+0000 7fa7389f2700 1 -- 172.21.15.59:0/3645473314 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa7340ada10 msgr2=0x7fa7340adea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:26.658 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:26.655+0000 7fa7389f2700 1 --2- 172.21.15.59:0/3645473314 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa7340ada10 0x7fa7340adea0 secure :-1 s=READY pgs=894 cs=0 l=1 rev1=1 crypto rx=0x7fa72c009e50 tx=0x7fa72c00b330 comp rx=0 tx=0).stop 2023-11-24T07:59:26.658 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:26.655+0000 7fa7389f2700 1 -- 172.21.15.59:0/3645473314 shutdown_connections 2023-11-24T07:59:26.658 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:26.655+0000 7fa7389f2700 1 --2- 172.21.15.59:0/3645473314 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fa73406afd0 0x7fa7340d16e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:26.658 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:26.655+0000 7fa7389f2700 1 --2- 172.21.15.59:0/3645473314 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa7340ada10 0x7fa7340adea0 unknown :-1 s=CLOSED pgs=894 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:26.659 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:26.655+0000 7fa7389f2700 1 -- 172.21.15.59:0/3645473314 >> 172.21.15.59:0/3645473314 conn(0x7fa7340d0bc0 msgr2=0x7fa7340d0fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:59:26.659 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:26.655+0000 7fa7389f2700 1 -- 172.21.15.59:0/3645473314 shutdown_connections 2023-11-24T07:59:26.659 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:26.655+0000 7fa7389f2700 1 -- 172.21.15.59:0/3645473314 wait complete. 2023-11-24T07:59:26.662 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:26.656+0000 7fa7389f2700 1 Processor -- start 2023-11-24T07:59:26.662 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:26.656+0000 7fa7389f2700 1 -- start start 2023-11-24T07:59:26.662 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:26.657+0000 7fa7389f2700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa73406afd0 0x7fa73415cb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:26.662 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:26.657+0000 7fa7389f2700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa7340ada10 0x7fa7340c87a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:26.662 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:26.657+0000 7fa7389f2700 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fa7340affb0 0x7fa73415b810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:26.662 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:26.657+0000 7fa7389f2700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7340c65c0 con 0x7fa73406afd0 2023-11-24T07:59:26.662 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:26.657+0000 7fa7389f2700 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7340c6840 con 0x7fa7340ada10 2023-11-24T07:59:26.662 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:26.657+0000 7fa7389f2700 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa7340c6b20 con 0x7fa7340affb0 2023-11-24T07:59:26.662 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:26.657+0000 7fa73259c700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa73406afd0 0x7fa73415cb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:26.662 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:26.657+0000 7fa73259c700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa73406afd0 0x7fa73415cb50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:55054/0 (socket says 172.21.15.59:55054) 2023-11-24T07:59:26.663 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:26.657+0000 7fa73259c700 1 -- 172.21.15.59:0/3531076668 learned_addr learned my addr 172.21.15.59:0/3531076668 (peer_addr_for_me v2:172.21.15.59:0/0) 2023-11-24T07:59:26.663 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:26.657+0000 7fa731d9b700 1 --2- 172.21.15.59:0/3531076668 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa7340ada10 0x7fa7340c87a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:26.663 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:26.657+0000 7fa732d9d700 1 --2- 172.21.15.59:0/3531076668 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fa7340affb0 0x7fa73415b810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:26.663 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:26.657+0000 7fa73259c700 1 -- 172.21.15.59:0/3531076668 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fa7340affb0 msgr2=0x7fa73415b810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:26.663 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:26.657+0000 7fa73259c700 1 --2- 172.21.15.59:0/3531076668 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fa7340affb0 0x7fa73415b810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:26.663 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:26.657+0000 7fa73259c700 1 -- 172.21.15.59:0/3531076668 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa7340ada10 msgr2=0x7fa7340c87a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:26.663 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:26.657+0000 7fa73259c700 1 --2- 172.21.15.59:0/3531076668 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa7340ada10 0x7fa7340c87a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:26.663 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:26.657+0000 7fa73259c700 1 -- 172.21.15.59:0/3531076668 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa72c009b30 con 0x7fa73406afd0 2023-11-24T07:59:26.663 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:26.658+0000 7fa73259c700 1 --2- 172.21.15.59:0/3531076668 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa73406afd0 0x7fa73415cb50 secure :-1 s=READY pgs=895 cs=0 l=1 rev1=1 crypto rx=0x7fa72800c750 tx=0x7fa72800ca60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:59:26.663 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:26.658+0000 7fa722ffd700 1 -- 172.21.15.59:0/3531076668 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa728011800 con 0x7fa73406afd0 2023-11-24T07:59:26.663 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:26.658+0000 7fa722ffd700 1 -- 172.21.15.59:0/3531076668 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa728011960 con 0x7fa73406afd0 2023-11-24T07:59:26.664 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:26.658+0000 7fa7389f2700 1 -- 172.21.15.59:0/3531076668 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa73415f470 con 0x7fa73406afd0 2023-11-24T07:59:26.664 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:26.658+0000 7fa722ffd700 1 -- 172.21.15.59:0/3531076668 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa728011cd0 con 0x7fa73406afd0 2023-11-24T07:59:26.664 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:26.658+0000 7fa7389f2700 1 -- 172.21.15.59:0/3531076668 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa7340c70e0 con 0x7fa73406afd0 2023-11-24T07:59:26.664 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:26.659+0000 7fa7389f2700 1 -- 172.21.15.59:0/3531076668 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa734068670 con 0x7fa73406afd0 2023-11-24T07:59:26.664 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:26.659+0000 7fa722ffd700 1 -- 172.21.15.59:0/3531076668 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 10) v1 ==== 89588+0+0 (secure 0 0 0) 0x7fa728021350 con 0x7fa73406afd0 2023-11-24T07:59:26.664 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:26.660+0000 7fa722ffd700 1 --2- 172.21.15.59:0/3531076668 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7fa71406b2c0 0x7fa71406d780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:26.664 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:26.660+0000 7fa722ffd700 1 -- 172.21.15.59:0/3531076668 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fa728089c30 con 0x7fa73406afd0 2023-11-24T07:59:26.664 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:26.660+0000 7fa731d9b700 1 --2- 172.21.15.59:0/3531076668 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7fa71406b2c0 0x7fa71406d780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:26.664 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:26.660+0000 7fa731d9b700 1 --2- 172.21.15.59:0/3531076668 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7fa71406b2c0 0x7fa71406d780 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7fa7340cd520 tx=0x7fa7240090e0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:59:26.666 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:26.664+0000 7fa722ffd700 1 -- 172.21.15.59:0/3531076668 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fa728054680 con 0x7fa73406afd0 2023-11-24T07:59:26.910 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:26.908+0000 7fa7389f2700 1 -- 172.21.15.59:0/3531076668 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7fa7340ce900 con 0x7fa73406afd0 2023-11-24T07:59:26.910 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:26.908+0000 7fa722ffd700 1 -- 172.21.15.59:0/3531076668 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v44) v1 ==== 53+0+73 (secure 0 0 0) 0x7fa7340ce900 con 0x7fa73406afd0 2023-11-24T07:59:26.910 INFO:teuthology.orchestra.run.smithi059.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-11-24T07:59:26.913 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:26.911+0000 7fa7389f2700 1 -- 172.21.15.59:0/3531076668 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7fa71406b2c0 msgr2=0x7fa71406d780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:26.913 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:26.911+0000 7fa7389f2700 1 --2- 172.21.15.59:0/3531076668 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7fa71406b2c0 0x7fa71406d780 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7fa7340cd520 tx=0x7fa7240090e0 comp rx=0 tx=0).stop 2023-11-24T07:59:26.913 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:26.911+0000 7fa7389f2700 1 -- 172.21.15.59:0/3531076668 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa73406afd0 msgr2=0x7fa73415cb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:26.913 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:26.911+0000 7fa7389f2700 1 --2- 172.21.15.59:0/3531076668 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa73406afd0 0x7fa73415cb50 secure :-1 s=READY pgs=895 cs=0 l=1 rev1=1 crypto rx=0x7fa72800c750 tx=0x7fa72800ca60 comp rx=0 tx=0).stop 2023-11-24T07:59:26.913 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:26.911+0000 7fa7389f2700 1 -- 172.21.15.59:0/3531076668 shutdown_connections 2023-11-24T07:59:26.913 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:26.911+0000 7fa7389f2700 1 --2- 172.21.15.59:0/3531076668 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7fa71406b2c0 0x7fa71406d780 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:26.914 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:26.911+0000 7fa7389f2700 1 --2- 172.21.15.59:0/3531076668 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fa7340affb0 0x7fa73415b810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:26.914 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:26.911+0000 7fa7389f2700 1 --2- 172.21.15.59:0/3531076668 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa73406afd0 0x7fa73415cb50 unknown :-1 s=CLOSED pgs=895 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:26.914 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:26.911+0000 7fa7389f2700 1 --2- 172.21.15.59:0/3531076668 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa7340ada10 0x7fa7340c87a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:26.914 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:26.911+0000 7fa7389f2700 1 -- 172.21.15.59:0/3531076668 >> 172.21.15.59:0/3531076668 conn(0x7fa7340d0bc0 msgr2=0x7fa7340b1b60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:59:26.914 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:26.911+0000 7fa7389f2700 1 -- 172.21.15.59:0/3531076668 shutdown_connections 2023-11-24T07:59:26.914 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:26.911+0000 7fa7389f2700 1 -- 172.21.15.59:0/3531076668 wait complete. 2023-11-24T07:59:26.925 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2023-11-24T07:59:26.925 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-11-24T07:59:26.925 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi076.front.sepia.ceph.com 2023-11-24T07:59:26.925 INFO:tasks.cephfs.mount:self.client.name = client.1 2023-11-24T07:59:26.925 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2023-11-24T07:59:26.925 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2023-11-24T07:59:26.925 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2023-11-24T07:59:26.925 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-11-24T07:59:26.925 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2023-11-24T07:59:26.925 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T07:59:26.926 DEBUG:teuthology.orchestra.run.smithi076:> ip addr 2023-11-24T07:59:26.948 INFO:teuthology.orchestra.run.smithi076.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-11-24T07:59:26.948 INFO:teuthology.orchestra.run.smithi076.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-11-24T07:59:26.948 INFO:teuthology.orchestra.run.smithi076.stdout: inet 127.0.0.1/8 scope host lo 2023-11-24T07:59:26.948 INFO:teuthology.orchestra.run.smithi076.stdout: valid_lft forever preferred_lft forever 2023-11-24T07:59:26.948 INFO:teuthology.orchestra.run.smithi076.stdout: inet6 ::1/128 scope host 2023-11-24T07:59:26.948 INFO:teuthology.orchestra.run.smithi076.stdout: valid_lft forever preferred_lft forever 2023-11-24T07:59:26.948 INFO:teuthology.orchestra.run.smithi076.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-11-24T07:59:26.948 INFO:teuthology.orchestra.run.smithi076.stdout: link/ether 0c:c4:7a:d8:da:66 brd ff:ff:ff:ff:ff:ff 2023-11-24T07:59:26.948 INFO:teuthology.orchestra.run.smithi076.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-11-24T07:59:26.949 INFO:teuthology.orchestra.run.smithi076.stdout: link/ether 0c:c4:7a:d8:da:67 brd ff:ff:ff:ff:ff:ff 2023-11-24T07:59:26.949 INFO:teuthology.orchestra.run.smithi076.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-11-24T07:59:26.949 INFO:teuthology.orchestra.run.smithi076.stdout: link/ether 0c:c4:7a:8f:47:e0 brd ff:ff:ff:ff:ff:ff 2023-11-24T07:59:26.949 INFO:teuthology.orchestra.run.smithi076.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-11-24T07:59:26.949 INFO:teuthology.orchestra.run.smithi076.stdout: link/ether 0c:c4:7a:8f:47:e1 brd ff:ff:ff:ff:ff:ff 2023-11-24T07:59:26.949 INFO:teuthology.orchestra.run.smithi076.stdout: inet 172.21.15.76/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2023-11-24T07:59:26.949 INFO:teuthology.orchestra.run.smithi076.stdout: valid_lft 42232sec preferred_lft 42232sec 2023-11-24T07:59:26.949 INFO:teuthology.orchestra.run.smithi076.stdout: inet6 fe80::a34d:4ce5:8360:8f9/64 scope link noprefixroute 2023-11-24T07:59:26.949 INFO:teuthology.orchestra.run.smithi076.stdout: valid_lft forever preferred_lft forever 2023-11-24T07:59:26.949 INFO:teuthology.orchestra.run.smithi076.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-11-24T07:59:26.950 INFO:teuthology.orchestra.run.smithi076.stdout: link/ether 2a:59:73:ff:f7:95 brd ff:ff:ff:ff:ff:ff 2023-11-24T07:59:26.950 INFO:teuthology.orchestra.run.smithi076.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-11-24T07:59:26.950 INFO:teuthology.orchestra.run.smithi076.stdout: valid_lft forever preferred_lft forever 2023-11-24T07:59:26.950 INFO:teuthology.orchestra.run.smithi076.stdout: inet6 fe80::4c23:44ff:feb9:d9d4/64 scope link 2023-11-24T07:59:26.950 INFO:teuthology.orchestra.run.smithi076.stdout: valid_lft forever preferred_lft forever 2023-11-24T07:59:26.950 INFO:teuthology.orchestra.run.smithi076.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-11-24T07:59:26.950 INFO:teuthology.orchestra.run.smithi076.stdout: link/ether 46:3e:09:af:65:64 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2023-11-24T07:59:26.950 INFO:teuthology.orchestra.run.smithi076.stdout: inet6 fe80::443e:9ff:feaf:6564/64 scope link 2023-11-24T07:59:26.950 INFO:teuthology.orchestra.run.smithi076.stdout: valid_lft forever preferred_lft forever 2023-11-24T07:59:26.950 INFO:teuthology.orchestra.run.smithi076.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-11-24T07:59:26.951 INFO:teuthology.orchestra.run.smithi076.stdout: link/ether 2a:59:73:ff:f7:95 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.3 2023-11-24T07:59:26.951 INFO:teuthology.orchestra.run.smithi076.stdout: inet6 fe80::2859:73ff:feff:f795/64 scope link 2023-11-24T07:59:26.951 INFO:teuthology.orchestra.run.smithi076.stdout: valid_lft forever preferred_lft forever 2023-11-24T07:59:26.951 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T07:59:26.951 DEBUG:teuthology.orchestra.run.smithi076:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-11-24T07:59:27.019 INFO:teuthology.orchestra.run.smithi076.stdout:1 2023-11-24T07:59:27.021 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T07:59:27.021 DEBUG:teuthology.orchestra.run.smithi076:> route 2023-11-24T07:59:27.084 INFO:teuthology.orchestra.run.smithi076.stdout:Kernel IP routing table 2023-11-24T07:59:27.084 INFO:teuthology.orchestra.run.smithi076.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-11-24T07:59:27.084 INFO:teuthology.orchestra.run.smithi076.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-11-24T07:59:27.084 INFO:teuthology.orchestra.run.smithi076.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-11-24T07:59:27.084 INFO:teuthology.orchestra.run.smithi076.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-11-24T07:59:27.085 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T07:59:27.085 DEBUG:teuthology.orchestra.run.smithi076:> (cd / && exec bash -c ' 2023-11-24T07:59:27.085 DEBUG:teuthology.orchestra.run.smithi076:> set -e 2023-11-24T07:59:27.085 DEBUG:teuthology.orchestra.run.smithi076:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-11-24T07:59:27.085 DEBUG:teuthology.orchestra.run.smithi076:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-11-24T07:59:27.085 DEBUG:teuthology.orchestra.run.smithi076:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-11-24T07:59:27.085 DEBUG:teuthology.orchestra.run.smithi076:> ') 2023-11-24T07:59:27.204 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T07:59:27.204 DEBUG:teuthology.orchestra.run.smithi076:> ip netns list 2023-11-24T07:59:27.225 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-ns--home-ubuntu-cephtest-mnt.3 (id: 1) 2023-11-24T07:59:27.225 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2023-11-24T07:59:27.226 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2023-11-24T07:59:27.226 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T07:59:27.227 DEBUG:teuthology.orchestra.run.smithi076:> sudo ip link set brx.0 up 2023-11-24T07:59:27.299 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-11-24T07:59:27.299 DEBUG:teuthology.orchestra.run.smithi076:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2023-11-24T07:59:27.359 INFO:teuthology.orchestra.run.smithi076.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2023-11-24T07:59:27.360 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-11-24T07:59:27.360 DEBUG:teuthology.orchestra.run.smithi076:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2023-11-24T07:59:27.419 INFO:tasks.cephfs.kernel_mount:mounting using device: 1@.cephfs=/ 2023-11-24T07:59:27.419 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T07:59:27.419 DEBUG:teuthology.orchestra.run.smithi076:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 1@.cephfs=/ /home/ubuntu/cephtest/mnt.1 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-11-24T07:59:27.508 INFO:teuthology.orchestra.run.smithi076.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-11-24T07:59:27.509 INFO:teuthology.orchestra.run.smithi076.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-11-24T07:59:27.532 INFO:teuthology.orchestra.run.smithi076.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-11-24T07:59:27.532 INFO:teuthology.orchestra.run.smithi076.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-11-24T07:59:27.533 INFO:teuthology.orchestra.run.smithi076.stdout:mount.ceph: resolved to: "172.21.15.59:3300,172.21.15.76:3300,172.21.15.76:3301" 2023-11-24T07:59:27.533 INFO:teuthology.orchestra.run.smithi076.stdout:mount.ceph: trying mount with new device syntax: 1@9f0e8741-b8f9-48d5-a2c3-09f45903303d.cephfs=/ 2023-11-24T07:59:27.533 INFO:teuthology.orchestra.run.smithi076.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,ms_mode=prefer-crc,key=1,mon_addr=172.21.15.59:3300/172.21.15.76:3300/172.21.15.76:3301" will pass to kernel 2023-11-24T07:59:27.533 INFO:teuthology.orchestra.run.smithi076.stdout:mount.ceph: v2 mount support check returned 2 2023-11-24T07:59:27.533 INFO:teuthology.orchestra.run.smithi076.stdout:mount.ceph: kernel does not support v2 syntax 2023-11-24T07:59:27.533 INFO:teuthology.orchestra.run.smithi076.stdout:mount.ceph: switching to using v1 address with old syntax 2023-11-24T07:59:27.561 INFO:teuthology.orchestra.run.smithi076.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-11-24T07:59:27.562 INFO:teuthology.orchestra.run.smithi076.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-11-24T07:59:27.562 INFO:teuthology.orchestra.run.smithi076.stdout:mount.ceph: resolved to: "172.21.15.59:3300,172.21.15.76:3300,172.21.15.76:3301" 2023-11-24T07:59:27.562 INFO:teuthology.orchestra.run.smithi076.stdout:mount.ceph: trying mount with new device syntax: 1@9f0e8741-b8f9-48d5-a2c3-09f45903303d.cephfs=/ 2023-11-24T07:59:27.562 INFO:teuthology.orchestra.run.smithi076.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,ms_mode=prefer-crc,key=1,mon_addr=172.21.15.59:3300/172.21.15.76:3300/172.21.15.76:3301" will pass to kernel 2023-11-24T07:59:27.562 INFO:teuthology.orchestra.run.smithi076.stdout:mount.ceph: v2 mount support check returned 2 2023-11-24T07:59:27.562 INFO:teuthology.orchestra.run.smithi076.stdout:mount.ceph: kernel does not support v2 syntax 2023-11-24T07:59:27.562 INFO:teuthology.orchestra.run.smithi076.stdout:mount.ceph: switching to using v1 address with old syntax 2023-11-24T07:59:27.562 INFO:teuthology.orchestra.run.smithi076.stdout:mount.ceph: resolved to: "172.21.15.59:6789,172.21.15.76:6789,172.21.15.76:6790" 2023-11-24T07:59:27.562 INFO:teuthology.orchestra.run.smithi076.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.59:6789,172.21.15.76:6789,172.21.15.76:6790:/ 2023-11-24T07:59:27.562 INFO:teuthology.orchestra.run.smithi076.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,key=1,mds_namespace=cephfs,fsid=9f0e8741-b8f9-48d5-a2c3-09f45903303d" will pass to kernel 2023-11-24T07:59:27.564 INFO:tasks.cephfs.kernel_mount:mount command passed 2023-11-24T07:59:27.565 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T07:59:27.565 DEBUG:teuthology.orchestra.run.smithi076:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2023-11-24T07:59:27.599 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T07:59:27.599 DEBUG:teuthology.orchestra.run.smithi076:> (cd /home/ubuntu/cephtest/mnt.1 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-11-24T07:59:27.670 INFO:teuthology.orchestra.run.smithi076.stdout:client5702 2023-11-24T07:59:27.671 DEBUG:teuthology.orchestra.run.smithi076:> set -ex 2023-11-24T07:59:27.671 DEBUG:teuthology.orchestra.run.smithi076:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-24T07:59:27.728 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs flag set enable_multiple true --yes-i-really-mean-it 2023-11-24T07:59:27.871 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:27.869+0000 7ff1f1f09700 1 Processor -- start 2023-11-24T07:59:27.871 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:27.870+0000 7ff1f1f09700 1 -- start start 2023-11-24T07:59:27.873 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:27.870+0000 7ff1f1f09700 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7ff1ec072ab0 0x7ff1ec06e770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:27.873 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:27.870+0000 7ff1f1f09700 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff1ec06dbd0 con 0x7ff1ec1447a0 2023-11-24T07:59:27.873 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:27.870+0000 7ff1f1f09700 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff1ec15b700 con 0x7ff1ec0722a0 2023-11-24T07:59:27.873 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:27.870+0000 7ff1f1f09700 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff1ec15b9e0 con 0x7ff1ec072ab0 2023-11-24T07:59:27.873 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:27.870+0000 7ff1eb7fe700 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7ff1ec1447a0 0x7ff1ec071b90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:58536/0 (socket says 172.21.15.59:58536) 2023-11-24T07:59:27.873 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:27.870+0000 7ff1eb7fe700 1 -- 172.21.15.59:0/567697437 learned_addr learned my addr 172.21.15.59:0/567697437 (peer_addr_for_me v1:172.21.15.59:0/0) 2023-11-24T07:59:27.873 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:27.871+0000 7ff1ebfff700 1 --2- 172.21.15.59:0/567697437 >> v2:172.21.15.76:3300/0 conn(0x7ff1ec072ab0 0x7ff1ec06e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:27.874 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:27.871+0000 7ff1ea7fc700 1 -- 172.21.15.59:0/567697437 <== mon.0 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3709422536 0 0) 0x7ff1ec06dbd0 con 0x7ff1ec1447a0 2023-11-24T07:59:27.874 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:27.871+0000 7ff1ea7fc700 1 -- 172.21.15.59:0/567697437 --> v1:172.21.15.59:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff1d80035b0 con 0x7ff1ec1447a0 2023-11-24T07:59:27.874 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:27.871+0000 7ff1ebfff700 1 -- 172.21.15.59:0/567697437 >> v1:172.21.15.76:6789/0 conn(0x7ff1ec0722a0 legacy=0x7ff1ec06e060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:27.874 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:27.871+0000 7ff1ebfff700 1 -- 172.21.15.59:0/567697437 >> v1:172.21.15.59:6789/0 conn(0x7ff1ec1447a0 legacy=0x7ff1ec071b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:27.874 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:27.871+0000 7ff1ebfff700 1 -- 172.21.15.59:0/567697437 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff1ec15bcc0 con 0x7ff1ec072ab0 2023-11-24T07:59:27.874 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:27.871+0000 7ff1ebfff700 1 --2- 172.21.15.59:0/567697437 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff1ec072ab0 0x7ff1ec06e770 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7ff1e4009d90 tx=0x7ff1e400b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b63225f753ef681f server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:59:27.874 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:27.871+0000 7ff1ea7fc700 1 -- 172.21.15.59:0/567697437 <== mon.1 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1e400e070 con 0x7ff1ec072ab0 2023-11-24T07:59:27.874 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:27.871+0000 7ff1ea7fc700 1 -- 172.21.15.59:0/567697437 <== mon.1 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff1e400bb20 con 0x7ff1ec072ab0 2023-11-24T07:59:27.874 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:27.871+0000 7ff1ea7fc700 1 -- 172.21.15.59:0/567697437 <== mon.1 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1e400be90 con 0x7ff1ec072ab0 2023-11-24T07:59:27.875 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:27.872+0000 7ff1f1f09700 1 -- 172.21.15.59:0/567697437 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff1ec072ab0 msgr2=0x7ff1ec06e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:27.875 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:27.872+0000 7ff1f1f09700 1 --2- 172.21.15.59:0/567697437 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff1ec072ab0 0x7ff1ec06e770 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7ff1e4009d90 tx=0x7ff1e400b330 comp rx=0 tx=0).stop 2023-11-24T07:59:27.875 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:27.872+0000 7ff1f1f09700 1 -- 172.21.15.59:0/567697437 shutdown_connections 2023-11-24T07:59:27.875 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:27.872+0000 7ff1f1f09700 1 --2- 172.21.15.59:0/567697437 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff1ec072ab0 0x7ff1ec06e770 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:27.876 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:27.872+0000 7ff1f1f09700 1 -- 172.21.15.59:0/567697437 >> 172.21.15.59:0/567697437 conn(0x7ff1ec0b2760 msgr2=0x7ff1ec0b2b70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:59:27.876 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:27.872+0000 7ff1f1f09700 1 -- 172.21.15.59:0/567697437 shutdown_connections 2023-11-24T07:59:27.876 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:27.872+0000 7ff1f1f09700 1 -- 172.21.15.59:0/567697437 wait complete. 2023-11-24T07:59:27.876 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:27.873+0000 7ff1f1f09700 1 Processor -- start 2023-11-24T07:59:27.876 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:27.874+0000 7ff1f1f09700 1 -- start start 2023-11-24T07:59:27.880 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:27.874+0000 7ff1f1f09700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff1ec0722a0 0x7ff1ec15dd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:27.880 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:27.874+0000 7ff1f1f09700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff1ec072ab0 0x7ff1ec15e290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:27.880 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:27.874+0000 7ff1f1f09700 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7ff1ec1447a0 0x7ff1ec150010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:27.880 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:27.874+0000 7ff1f1f09700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1ec13a4d0 con 0x7ff1ec0722a0 2023-11-24T07:59:27.880 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:27.874+0000 7ff1f1f09700 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1ec13aa50 con 0x7ff1ec072ab0 2023-11-24T07:59:27.880 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:27.874+0000 7ff1f1f09700 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff1ec1f0ed0 con 0x7ff1ec1447a0 2023-11-24T07:59:27.880 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:27.874+0000 7ff1eb7fe700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff1ec0722a0 0x7ff1ec15dd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:27.881 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:27.874+0000 7ff1eb7fe700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff1ec0722a0 0x7ff1ec15dd50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:55068/0 (socket says 172.21.15.59:55068) 2023-11-24T07:59:27.881 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:27.874+0000 7ff1eb7fe700 1 -- 172.21.15.59:0/2657782369 learned_addr learned my addr 172.21.15.59:0/2657782369 (peer_addr_for_me v2:172.21.15.59:0/0) 2023-11-24T07:59:27.881 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:27.875+0000 7ff1eaffd700 1 --2- 172.21.15.59:0/2657782369 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff1ec072ab0 0x7ff1ec15e290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:27.881 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:27.875+0000 7ff1ebfff700 1 --2- 172.21.15.59:0/2657782369 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7ff1ec1447a0 0x7ff1ec150010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:27.881 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:27.875+0000 7ff1eb7fe700 1 -- 172.21.15.59:0/2657782369 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7ff1ec1447a0 msgr2=0x7ff1ec150010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:27.881 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:27.875+0000 7ff1eb7fe700 1 --2- 172.21.15.59:0/2657782369 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7ff1ec1447a0 0x7ff1ec150010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:27.881 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:27.875+0000 7ff1eb7fe700 1 -- 172.21.15.59:0/2657782369 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff1ec072ab0 msgr2=0x7ff1ec15e290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:27.881 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:27.875+0000 7ff1eb7fe700 1 --2- 172.21.15.59:0/2657782369 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff1ec072ab0 0x7ff1ec15e290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:27.881 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:27.875+0000 7ff1eb7fe700 1 -- 172.21.15.59:0/2657782369 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff1e4009a70 con 0x7ff1ec0722a0 2023-11-24T07:59:27.881 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:27.875+0000 7ff1eaffd700 1 --2- 172.21.15.59:0/2657782369 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff1ec072ab0 0x7ff1ec15e290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-24T07:59:27.882 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:27.875+0000 7ff1eb7fe700 1 --2- 172.21.15.59:0/2657782369 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff1ec0722a0 0x7ff1ec15dd50 secure :-1 s=READY pgs=897 cs=0 l=1 rev1=1 crypto rx=0x7ff1dc00e910 tx=0x7ff1dc00ecd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:59:27.882 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:27.875+0000 7ff1e3fff700 1 -- 172.21.15.59:0/2657782369 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1dc005b40 con 0x7ff1ec0722a0 2023-11-24T07:59:27.882 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:27.875+0000 7ff1e3fff700 1 -- 172.21.15.59:0/2657782369 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff1dc005ca0 con 0x7ff1ec0722a0 2023-11-24T07:59:27.882 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:27.875+0000 7ff1f1f09700 1 -- 172.21.15.59:0/2657782369 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff1ec150dd0 con 0x7ff1ec0722a0 2023-11-24T07:59:27.882 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:27.875+0000 7ff1e3fff700 1 -- 172.21.15.59:0/2657782369 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1dc00ca20 con 0x7ff1ec0722a0 2023-11-24T07:59:27.882 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:27.875+0000 7ff1f1f09700 1 -- 172.21.15.59:0/2657782369 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff1ec15eba0 con 0x7ff1ec0722a0 2023-11-24T07:59:27.882 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:27.876+0000 7ff1f1f09700 1 -- 172.21.15.59:0/2657782369 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff1ec068ee0 con 0x7ff1ec0722a0 2023-11-24T07:59:27.882 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:27.877+0000 7ff1e3fff700 1 -- 172.21.15.59:0/2657782369 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 10) v1 ==== 89588+0+0 (secure 0 0 0) 0x7ff1dc00cb80 con 0x7ff1ec0722a0 2023-11-24T07:59:27.882 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:27.877+0000 7ff1e3fff700 1 --2- 172.21.15.59:0/2657782369 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7ff1cc06b2c0 0x7ff1cc06d780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:27.882 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:27.877+0000 7ff1e3fff700 1 -- 172.21.15.59:0/2657782369 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 5709+0+0 (secure 0 0 0) 0x7ff1dc08c070 con 0x7ff1ec0722a0 2023-11-24T07:59:27.882 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:27.877+0000 7ff1eaffd700 1 --2- 172.21.15.59:0/2657782369 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7ff1cc06b2c0 0x7ff1cc06d780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:27.883 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:27.877+0000 7ff1eaffd700 1 --2- 172.21.15.59:0/2657782369 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7ff1cc06b2c0 0x7ff1cc06d780 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7ff1ec143f70 tx=0x7ff1d4009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:59:27.883 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:27.881+0000 7ff1e3fff700 1 -- 172.21.15.59:0/2657782369 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7ff1dc0bee90 con 0x7ff1ec0722a0 2023-11-24T07:59:28.130 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:28.129+0000 7ff1f1f09700 1 -- 172.21.15.59:0/2657782369 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "fs flag set", "flag_name": "enable_multiple", "val": "true", "yes_i_really_mean_it": true} v 0) v1 -- 0x7ff1ec068670 con 0x7ff1ec0722a0 2023-11-24T07:59:28.405 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:28.404+0000 7ff1e3fff700 1 -- 172.21.15.59:0/2657782369 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "fs flag set", "flag_name": "enable_multiple", "val": "true", "yes_i_really_mean_it": true}]=0 v45) v1 ==== 136+0+0 (secure 0 0 0) 0x7ff1dc056ac0 con 0x7ff1ec0722a0 2023-11-24T07:59:28.408 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:28.406+0000 7ff1f1f09700 1 -- 172.21.15.59:0/2657782369 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7ff1cc06b2c0 msgr2=0x7ff1cc06d780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:28.408 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:28.406+0000 7ff1f1f09700 1 --2- 172.21.15.59:0/2657782369 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7ff1cc06b2c0 0x7ff1cc06d780 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7ff1ec143f70 tx=0x7ff1d4009040 comp rx=0 tx=0).stop 2023-11-24T07:59:28.408 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:28.406+0000 7ff1f1f09700 1 -- 172.21.15.59:0/2657782369 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff1ec0722a0 msgr2=0x7ff1ec15dd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:28.408 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:28.406+0000 7ff1f1f09700 1 --2- 172.21.15.59:0/2657782369 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff1ec0722a0 0x7ff1ec15dd50 secure :-1 s=READY pgs=897 cs=0 l=1 rev1=1 crypto rx=0x7ff1dc00e910 tx=0x7ff1dc00ecd0 comp rx=0 tx=0).stop 2023-11-24T07:59:28.408 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:28.406+0000 7ff1f1f09700 1 -- 172.21.15.59:0/2657782369 shutdown_connections 2023-11-24T07:59:28.408 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:28.406+0000 7ff1f1f09700 1 --2- 172.21.15.59:0/2657782369 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7ff1cc06b2c0 0x7ff1cc06d780 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:28.409 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:28.406+0000 7ff1f1f09700 1 --2- 172.21.15.59:0/2657782369 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7ff1ec1447a0 0x7ff1ec150010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:28.409 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:28.406+0000 7ff1f1f09700 1 --2- 172.21.15.59:0/2657782369 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff1ec0722a0 0x7ff1ec15dd50 unknown :-1 s=CLOSED pgs=897 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:28.409 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:28.406+0000 7ff1f1f09700 1 --2- 172.21.15.59:0/2657782369 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff1ec072ab0 0x7ff1ec15e290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:28.409 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:28.406+0000 7ff1f1f09700 1 -- 172.21.15.59:0/2657782369 >> 172.21.15.59:0/2657782369 conn(0x7ff1ec0b2760 msgr2=0x7ff1ec071b90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:59:28.409 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:28.406+0000 7ff1f1f09700 1 -- 172.21.15.59:0/2657782369 shutdown_connections 2023-11-24T07:59:28.409 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:28.406+0000 7ff1f1f09700 1 -- 172.21.15.59:0/2657782369 wait complete. 2023-11-24T07:59:28.421 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume ls 2023-11-24T07:59:28.548 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:28.547+0000 7f03e3c5d700 1 Processor -- start 2023-11-24T07:59:28.549 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:28.547+0000 7f03e3c5d700 1 -- start start 2023-11-24T07:59:28.550 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:28.548+0000 7f03e3c5d700 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f03dc0d1440 0x7f03dc0d4960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:28.550 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:28.548+0000 7f03e3c5d700 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f03dc0d2790 con 0x7f03dc15dfc0 2023-11-24T07:59:28.550 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:28.548+0000 7f03e3c5d700 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f03dc0d2a70 con 0x7f03dc15e3a0 2023-11-24T07:59:28.551 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:28.548+0000 7f03e3c5d700 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f03dc0c7a30 con 0x7f03dc0d1440 2023-11-24T07:59:28.551 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:28.548+0000 7f03e19f9700 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7f03dc15dfc0 0x7f03dc0d0830 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:58550/0 (socket says 172.21.15.59:58550) 2023-11-24T07:59:28.551 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:28.548+0000 7f03e19f9700 1 -- 172.21.15.59:0/155846630 learned_addr learned my addr 172.21.15.59:0/155846630 (peer_addr_for_me v1:172.21.15.59:0/0) 2023-11-24T07:59:28.551 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:28.548+0000 7f03e11f8700 1 --2- 172.21.15.59:0/155846630 >> v2:172.21.15.76:3300/0 conn(0x7f03dc0d1440 0x7f03dc0d4960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:28.551 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:28.548+0000 7f03e09f7700 1 -- 172.21.15.59:0/155846630 <== mon.0 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 366415885 0 0) 0x7f03dc0d2790 con 0x7f03dc15dfc0 2023-11-24T07:59:28.551 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:28.548+0000 7f03e09f7700 1 -- 172.21.15.59:0/155846630 --> v1:172.21.15.59:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f03c80035b0 con 0x7f03dc15dfc0 2023-11-24T07:59:28.551 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:28.548+0000 7f03e09f7700 1 -- 172.21.15.59:0/155846630 <== mon.0 v1:172.21.15.59:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1932288668 0 0) 0x7f03c80035b0 con 0x7f03dc15dfc0 2023-11-24T07:59:28.551 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:28.548+0000 7f03e11f8700 1 -- 172.21.15.59:0/155846630 >> v1:172.21.15.76:6789/0 conn(0x7f03dc15e3a0 legacy=0x7f03dc0c8230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:28.551 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:28.548+0000 7f03e11f8700 1 -- 172.21.15.59:0/155846630 >> v1:172.21.15.59:6789/0 conn(0x7f03dc15dfc0 legacy=0x7f03dc0d0830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:28.551 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:28.548+0000 7f03e11f8700 1 -- 172.21.15.59:0/155846630 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f03dc0c7d10 con 0x7f03dc0d1440 2023-11-24T07:59:28.552 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:28.549+0000 7f03e11f8700 1 --2- 172.21.15.59:0/155846630 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f03dc0d1440 0x7f03dc0d4960 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f03cc009840 tx=0x7f03cc009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f926cc3c5fbdd0b1 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:59:28.552 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:28.549+0000 7f03e09f7700 1 -- 172.21.15.59:0/155846630 <== mon.1 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03d8001670 con 0x7f03dc0d1440 2023-11-24T07:59:28.552 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:28.549+0000 7f03e09f7700 1 -- 172.21.15.59:0/155846630 <== mon.1 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f03cc004030 con 0x7f03dc0d1440 2023-11-24T07:59:28.552 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:28.549+0000 7f03e09f7700 1 -- 172.21.15.59:0/155846630 <== mon.1 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03cc0043c0 con 0x7f03dc0d1440 2023-11-24T07:59:28.554 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:28.549+0000 7f03e3c5d700 1 -- 172.21.15.59:0/155846630 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f03dc0d1440 msgr2=0x7f03dc0d4960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:28.555 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:28.549+0000 7f03e3c5d700 1 --2- 172.21.15.59:0/155846630 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f03dc0d1440 0x7f03dc0d4960 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f03cc009840 tx=0x7f03cc009b50 comp rx=0 tx=0).stop 2023-11-24T07:59:28.555 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:28.549+0000 7f03e3c5d700 1 -- 172.21.15.59:0/155846630 shutdown_connections 2023-11-24T07:59:28.555 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:28.549+0000 7f03e3c5d700 1 --2- 172.21.15.59:0/155846630 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f03dc0d1440 0x7f03dc0d4960 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:28.555 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:28.549+0000 7f03e3c5d700 1 -- 172.21.15.59:0/155846630 >> 172.21.15.59:0/155846630 conn(0x7f03dc0d3e40 msgr2=0x7f03dc0d4250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:59:28.555 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:28.549+0000 7f03e3c5d700 1 -- 172.21.15.59:0/155846630 shutdown_connections 2023-11-24T07:59:28.555 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:28.550+0000 7f03e3c5d700 1 -- 172.21.15.59:0/155846630 wait complete. 2023-11-24T07:59:28.555 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:28.551+0000 7f03e3c5d700 1 Processor -- start 2023-11-24T07:59:28.555 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:28.551+0000 7f03e3c5d700 1 -- start start 2023-11-24T07:59:28.555 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:28.551+0000 7f03e3c5d700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f03dc0d1440 0x7f03dc1502a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:28.555 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:28.551+0000 7f03e3c5d700 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f03dc15dfc0 0x7f03dc09fd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:28.556 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:28.551+0000 7f03e3c5d700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f03dc15e3a0 0x7f03dc0a0240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:28.556 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:28.551+0000 7f03e3c5d700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03dc0a0880 con 0x7f03dc0d1440 2023-11-24T07:59:28.556 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:28.551+0000 7f03e3c5d700 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03dc0a0b60 con 0x7f03dc15e3a0 2023-11-24T07:59:28.556 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:28.551+0000 7f03e3c5d700 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f03dc0a0e40 con 0x7f03dc15dfc0 2023-11-24T07:59:28.556 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:28.551+0000 7f03e19f9700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f03dc0d1440 0x7f03dc1502a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:28.556 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:28.552+0000 7f03e19f9700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f03dc0d1440 0x7f03dc1502a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:55082/0 (socket says 172.21.15.59:55082) 2023-11-24T07:59:28.556 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:28.552+0000 7f03e19f9700 1 -- 172.21.15.59:0/3211960421 learned_addr learned my addr 172.21.15.59:0/3211960421 (peer_addr_for_me v2:172.21.15.59:0/0) 2023-11-24T07:59:28.556 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:28.552+0000 7f03e11f8700 1 --2- 172.21.15.59:0/3211960421 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f03dc15dfc0 0x7f03dc09fd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:28.556 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:28.552+0000 7f03e21fa700 1 --2- 172.21.15.59:0/3211960421 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f03dc15e3a0 0x7f03dc0a0240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:28.556 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:28.552+0000 7f03e19f9700 1 -- 172.21.15.59:0/3211960421 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f03dc15dfc0 msgr2=0x7f03dc09fd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:28.556 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:28.552+0000 7f03e19f9700 1 --2- 172.21.15.59:0/3211960421 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f03dc15dfc0 0x7f03dc09fd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:28.557 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:28.552+0000 7f03e19f9700 1 -- 172.21.15.59:0/3211960421 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f03dc15e3a0 msgr2=0x7f03dc0a0240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:28.557 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:28.552+0000 7f03e19f9700 1 --2- 172.21.15.59:0/3211960421 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f03dc15e3a0 0x7f03dc0a0240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:28.557 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:28.552+0000 7f03e19f9700 1 -- 172.21.15.59:0/3211960421 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f03cc0094a0 con 0x7f03dc0d1440 2023-11-24T07:59:28.557 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:28.552+0000 7f03e19f9700 1 --2- 172.21.15.59:0/3211960421 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f03dc0d1440 0x7f03dc1502a0 secure :-1 s=READY pgs=899 cs=0 l=1 rev1=1 crypto rx=0x7f03d80011e0 tx=0x7f03d800c9e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:59:28.557 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:28.552+0000 7f03d27fc700 1 -- 172.21.15.59:0/3211960421 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03d8004c80 con 0x7f03dc0d1440 2023-11-24T07:59:28.557 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:28.552+0000 7f03d27fc700 1 -- 172.21.15.59:0/3211960421 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f03d8004e00 con 0x7f03dc0d1440 2023-11-24T07:59:28.557 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:28.552+0000 7f03e3c5d700 1 -- 172.21.15.59:0/3211960421 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f03dc0a1120 con 0x7f03dc0d1440 2023-11-24T07:59:28.557 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:28.552+0000 7f03d27fc700 1 -- 172.21.15.59:0/3211960421 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03d8007310 con 0x7f03dc0d1440 2023-11-24T07:59:28.557 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:28.552+0000 7f03e3c5d700 1 -- 172.21.15.59:0/3211960421 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f03dc1507e0 con 0x7f03dc0d1440 2023-11-24T07:59:28.557 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:28.553+0000 7f03e3c5d700 1 -- 172.21.15.59:0/3211960421 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f03dc0a1770 con 0x7f03dc0d1440 2023-11-24T07:59:28.558 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:28.554+0000 7f03d27fc700 1 -- 172.21.15.59:0/3211960421 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 10) v1 ==== 89588+0+0 (secure 0 0 0) 0x7f03d8007470 con 0x7f03dc0d1440 2023-11-24T07:59:28.558 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:28.554+0000 7f03d27fc700 1 --2- 172.21.15.59:0/3211960421 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f03c006b2c0 0x7f03c006d780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:28.558 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:28.554+0000 7f03d27fc700 1 -- 172.21.15.59:0/3211960421 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f03d8089880 con 0x7f03dc0d1440 2023-11-24T07:59:28.559 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:28.554+0000 7f03e11f8700 1 --2- 172.21.15.59:0/3211960421 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f03c006b2c0 0x7f03c006d780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:28.559 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:28.555+0000 7f03e11f8700 1 --2- 172.21.15.59:0/3211960421 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f03c006b2c0 0x7f03c006d780 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f03cc0090b0 tx=0x7f03cc008030 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:59:28.560 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:28.559+0000 7f03d27fc700 1 -- 172.21.15.59:0/3211960421 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f03d80544f0 con 0x7f03dc0d1440 2023-11-24T07:59:28.789 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:28.787+0000 7f03e3c5d700 1 -- 172.21.15.59:0/3211960421 --> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] -- mgr_command(tid 0: {"prefix": "fs volume ls", "target": ["mon-mgr", ""]}) v1 -- 0x7f03dc0a0cd0 con 0x7f03c006b2c0 2023-11-24T07:59:28.790 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:28.789+0000 7f03d27fc700 1 -- 172.21.15.59:0/3211960421 <== mgr.4100 v2:172.21.15.76:6832/4264046536 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+41 (secure 0 0 0) 0x7f03dc0a0cd0 con 0x7f03c006b2c0 2023-11-24T07:59:28.791 INFO:teuthology.orchestra.run.smithi059.stdout:[ 2023-11-24T07:59:28.791 INFO:teuthology.orchestra.run.smithi059.stdout: { 2023-11-24T07:59:28.791 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "cephfs" 2023-11-24T07:59:28.791 INFO:teuthology.orchestra.run.smithi059.stdout: } 2023-11-24T07:59:28.792 INFO:teuthology.orchestra.run.smithi059.stdout:] 2023-11-24T07:59:28.794 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:28.791+0000 7f03e3c5d700 1 -- 172.21.15.59:0/3211960421 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f03c006b2c0 msgr2=0x7f03c006d780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:28.794 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:28.791+0000 7f03e3c5d700 1 --2- 172.21.15.59:0/3211960421 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f03c006b2c0 0x7f03c006d780 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f03cc0090b0 tx=0x7f03cc008030 comp rx=0 tx=0).stop 2023-11-24T07:59:28.794 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:28.791+0000 7f03e3c5d700 1 -- 172.21.15.59:0/3211960421 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f03dc0d1440 msgr2=0x7f03dc1502a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:28.794 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:28.791+0000 7f03e3c5d700 1 --2- 172.21.15.59:0/3211960421 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f03dc0d1440 0x7f03dc1502a0 secure :-1 s=READY pgs=899 cs=0 l=1 rev1=1 crypto rx=0x7f03d80011e0 tx=0x7f03d800c9e0 comp rx=0 tx=0).stop 2023-11-24T07:59:28.795 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:28.792+0000 7f03e3c5d700 1 -- 172.21.15.59:0/3211960421 shutdown_connections 2023-11-24T07:59:28.795 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:28.792+0000 7f03e3c5d700 1 --2- 172.21.15.59:0/3211960421 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f03c006b2c0 0x7f03c006d780 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:28.795 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:28.792+0000 7f03e3c5d700 1 --2- 172.21.15.59:0/3211960421 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f03dc15e3a0 0x7f03dc0a0240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:28.795 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:28.792+0000 7f03e3c5d700 1 --2- 172.21.15.59:0/3211960421 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f03dc0d1440 0x7f03dc1502a0 unknown :-1 s=CLOSED pgs=899 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:28.795 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:28.792+0000 7f03e3c5d700 1 --2- 172.21.15.59:0/3211960421 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f03dc15dfc0 0x7f03dc09fd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:28.795 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:28.792+0000 7f03e3c5d700 1 -- 172.21.15.59:0/3211960421 >> 172.21.15.59:0/3211960421 conn(0x7f03dc0d3e40 msgr2=0x7f03dc06c7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:59:28.796 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:28.792+0000 7f03e3c5d700 1 -- 172.21.15.59:0/3211960421 shutdown_connections 2023-11-24T07:59:28.796 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:28.792+0000 7f03e3c5d700 1 -- 172.21.15.59:0/3211960421 wait complete. 2023-11-24T07:59:28.806 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2023-11-24T07:59:28.933 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:28.931+0000 7fbd98ea4700 1 Processor -- start 2023-11-24T07:59:28.933 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:28.931+0000 7fbd98ea4700 1 -- start start 2023-11-24T07:59:28.935 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:28.932+0000 7fbd98ea4700 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fbd9406fc50 0x7fbd9406eef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:28.935 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:28.932+0000 7fbd98ea4700 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbd940d1230 con 0x7fbd940b0e90 2023-11-24T07:59:28.935 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:28.932+0000 7fbd98ea4700 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbd940d0c70 con 0x7fbd9415e1c0 2023-11-24T07:59:28.935 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:28.932+0000 7fbd98ea4700 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbd940d0f50 con 0x7fbd9406fc50 2023-11-24T07:59:28.935 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:28.933+0000 7fbd92d9d700 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7fbd940b0e90 0x7fbd9406ca10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:58568/0 (socket says 172.21.15.59:58568) 2023-11-24T07:59:28.935 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:28.933+0000 7fbd92d9d700 1 -- 172.21.15.59:0/2329551613 learned_addr learned my addr 172.21.15.59:0/2329551613 (peer_addr_for_me v1:172.21.15.59:0/0) 2023-11-24T07:59:28.935 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:28.933+0000 7fbd91d9b700 1 --2- 172.21.15.59:0/2329551613 >> v2:172.21.15.76:3300/0 conn(0x7fbd9406fc50 0x7fbd9406eef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:28.935 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:28.933+0000 7fbd9159a700 1 -- 172.21.15.59:0/2329551613 <== mon.0 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3466523161 0 0) 0x7fbd940d1230 con 0x7fbd940b0e90 2023-11-24T07:59:28.936 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:28.933+0000 7fbd9159a700 1 -- 172.21.15.59:0/2329551613 --> v1:172.21.15.59:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbd800035b0 con 0x7fbd940b0e90 2023-11-24T07:59:28.936 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:28.933+0000 7fbd91d9b700 1 -- 172.21.15.59:0/2329551613 >> v1:172.21.15.76:6789/0 conn(0x7fbd9415e1c0 legacy=0x7fbd9406f540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:28.936 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:28.933+0000 7fbd91d9b700 1 -- 172.21.15.59:0/2329551613 >> v1:172.21.15.59:6789/0 conn(0x7fbd940b0e90 legacy=0x7fbd9406ca10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:28.936 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:28.933+0000 7fbd91d9b700 1 -- 172.21.15.59:0/2329551613 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbd9406c210 con 0x7fbd9406fc50 2023-11-24T07:59:28.936 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:28.933+0000 7fbd91d9b700 1 --2- 172.21.15.59:0/2329551613 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fbd9406fc50 0x7fbd9406eef0 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7fbd7c009840 tx=0x7fbd7c009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=885cb378daa4e1bf server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:59:28.936 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:28.933+0000 7fbd9159a700 1 -- 172.21.15.59:0/2329551613 <== mon.1 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd7c011830 con 0x7fbd9406fc50 2023-11-24T07:59:28.936 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:28.933+0000 7fbd9159a700 1 -- 172.21.15.59:0/2329551613 <== mon.1 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbd7c011990 con 0x7fbd9406fc50 2023-11-24T07:59:28.936 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:28.933+0000 7fbd9159a700 1 -- 172.21.15.59:0/2329551613 <== mon.1 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd7c011d00 con 0x7fbd9406fc50 2023-11-24T07:59:28.939 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:28.934+0000 7fbd98ea4700 1 -- 172.21.15.59:0/2329551613 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fbd9406fc50 msgr2=0x7fbd9406eef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:28.939 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:28.934+0000 7fbd98ea4700 1 --2- 172.21.15.59:0/2329551613 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fbd9406fc50 0x7fbd9406eef0 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7fbd7c009840 tx=0x7fbd7c009b50 comp rx=0 tx=0).stop 2023-11-24T07:59:28.939 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:28.934+0000 7fbd98ea4700 1 -- 172.21.15.59:0/2329551613 shutdown_connections 2023-11-24T07:59:28.939 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:28.934+0000 7fbd98ea4700 1 --2- 172.21.15.59:0/2329551613 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fbd9406fc50 0x7fbd9406eef0 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:28.939 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:28.934+0000 7fbd98ea4700 1 -- 172.21.15.59:0/2329551613 >> 172.21.15.59:0/2329551613 conn(0x7fbd940b3720 msgr2=0x7fbd940b3b30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:59:28.939 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:28.934+0000 7fbd98ea4700 1 -- 172.21.15.59:0/2329551613 shutdown_connections 2023-11-24T07:59:28.939 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:28.934+0000 7fbd98ea4700 1 -- 172.21.15.59:0/2329551613 wait complete. 2023-11-24T07:59:28.939 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:28.935+0000 7fbd98ea4700 1 Processor -- start 2023-11-24T07:59:28.939 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:28.935+0000 7fbd98ea4700 1 -- start start 2023-11-24T07:59:28.939 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:28.936+0000 7fbd98ea4700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbd9406fc50 0x7fbd94160460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:28.940 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:28.936+0000 7fbd98ea4700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fbd940b0e90 0x7fbd940b67c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:28.940 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:28.936+0000 7fbd98ea4700 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fbd9415e1c0 0x7fbd94161df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:28.940 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:28.936+0000 7fbd98ea4700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd940b5a00 con 0x7fbd9406fc50 2023-11-24T07:59:28.940 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:28.936+0000 7fbd98ea4700 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd940b5ce0 con 0x7fbd940b0e90 2023-11-24T07:59:28.940 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:28.936+0000 7fbd98ea4700 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbd940b5fc0 con 0x7fbd9415e1c0 2023-11-24T07:59:28.940 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:28.936+0000 7fbd9259c700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbd9406fc50 0x7fbd94160460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:28.940 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:28.936+0000 7fbd9259c700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbd9406fc50 0x7fbd94160460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:55096/0 (socket says 172.21.15.59:55096) 2023-11-24T07:59:28.940 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:28.936+0000 7fbd9259c700 1 -- 172.21.15.59:0/3815529817 learned_addr learned my addr 172.21.15.59:0/3815529817 (peer_addr_for_me v2:172.21.15.59:0/0) 2023-11-24T07:59:28.940 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:28.936+0000 7fbd91d9b700 1 --2- 172.21.15.59:0/3815529817 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fbd940b0e90 0x7fbd940b67c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:28.940 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:28.936+0000 7fbd92d9d700 1 --2- 172.21.15.59:0/3815529817 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fbd9415e1c0 0x7fbd94161df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:28.940 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:28.937+0000 7fbd9259c700 1 -- 172.21.15.59:0/3815529817 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fbd9415e1c0 msgr2=0x7fbd94161df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:28.940 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:28.937+0000 7fbd9259c700 1 --2- 172.21.15.59:0/3815529817 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fbd9415e1c0 0x7fbd94161df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:28.941 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:28.937+0000 7fbd9259c700 1 -- 172.21.15.59:0/3815529817 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fbd940b0e90 msgr2=0x7fbd940b67c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:28.941 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:28.937+0000 7fbd9259c700 1 --2- 172.21.15.59:0/3815529817 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fbd940b0e90 0x7fbd940b67c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:28.941 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:28.937+0000 7fbd9259c700 1 -- 172.21.15.59:0/3815529817 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbd7c0094a0 con 0x7fbd9406fc50 2023-11-24T07:59:28.941 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:28.937+0000 7fbd9259c700 1 --2- 172.21.15.59:0/3815529817 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbd9406fc50 0x7fbd94160460 secure :-1 s=READY pgs=901 cs=0 l=1 rev1=1 crypto rx=0x7fbd8400c450 tx=0x7fbd8400c760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:59:28.941 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:28.937+0000 7fbd8affd700 1 -- 172.21.15.59:0/3815529817 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd840052d0 con 0x7fbd9406fc50 2023-11-24T07:59:28.941 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:28.937+0000 7fbd8affd700 1 -- 172.21.15.59:0/3815529817 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbd84005430 con 0x7fbd9406fc50 2023-11-24T07:59:28.941 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:28.937+0000 7fbd98ea4700 1 -- 172.21.15.59:0/3815529817 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbd940b62a0 con 0x7fbd9406fc50 2023-11-24T07:59:28.941 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:28.937+0000 7fbd8affd700 1 -- 172.21.15.59:0/3815529817 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd840038c0 con 0x7fbd9406fc50 2023-11-24T07:59:28.941 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:28.937+0000 7fbd98ea4700 1 -- 172.21.15.59:0/3815529817 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbd94162330 con 0x7fbd9406fc50 2023-11-24T07:59:28.942 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:28.938+0000 7fbd98ea4700 1 -- 172.21.15.59:0/3815529817 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbd940b6610 con 0x7fbd9406fc50 2023-11-24T07:59:28.942 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:28.938+0000 7fbd8affd700 1 -- 172.21.15.59:0/3815529817 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 10) v1 ==== 89588+0+0 (secure 0 0 0) 0x7fbd84003a20 con 0x7fbd9406fc50 2023-11-24T07:59:28.942 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:28.939+0000 7fbd8affd700 1 --2- 172.21.15.59:0/3815529817 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7fbd7406b2c0 0x7fbd7406d780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:28.942 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:28.939+0000 7fbd8affd700 1 -- 172.21.15.59:0/3815529817 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fbd84088fb0 con 0x7fbd9406fc50 2023-11-24T07:59:28.943 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:28.939+0000 7fbd91d9b700 1 --2- 172.21.15.59:0/3815529817 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7fbd7406b2c0 0x7fbd7406d780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:28.943 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:28.940+0000 7fbd91d9b700 1 --2- 172.21.15.59:0/3815529817 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7fbd7406b2c0 0x7fbd7406d780 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7fbd7c000c00 tx=0x7fbd7c008030 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:59:28.945 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:28.944+0000 7fbd8affd700 1 -- 172.21.15.59:0/3815529817 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fbd84053b10 con 0x7fbd9406fc50 2023-11-24T07:59:29.145 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:29.143+0000 7fbd98ea4700 1 -- 172.21.15.59:0/3815529817 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7fbd940b5b40 con 0x7fbd9406fc50 2023-11-24T07:59:29.145 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:29.143+0000 7fbd8affd700 1 -- 172.21.15.59:0/3815529817 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v16)=0 v16) v1 ==== 122+0+0 (secure 0 0 0) 0x7fbd940b5b40 con 0x7fbd9406fc50 2023-11-24T07:59:29.147 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:29.145+0000 7fbd98ea4700 1 -- 172.21.15.59:0/3815529817 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7fbd7406b2c0 msgr2=0x7fbd7406d780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:29.148 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:29.145+0000 7fbd98ea4700 1 --2- 172.21.15.59:0/3815529817 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7fbd7406b2c0 0x7fbd7406d780 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7fbd7c000c00 tx=0x7fbd7c008030 comp rx=0 tx=0).stop 2023-11-24T07:59:29.148 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:29.145+0000 7fbd98ea4700 1 -- 172.21.15.59:0/3815529817 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbd9406fc50 msgr2=0x7fbd94160460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:29.148 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:29.145+0000 7fbd98ea4700 1 --2- 172.21.15.59:0/3815529817 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbd9406fc50 0x7fbd94160460 secure :-1 s=READY pgs=901 cs=0 l=1 rev1=1 crypto rx=0x7fbd8400c450 tx=0x7fbd8400c760 comp rx=0 tx=0).stop 2023-11-24T07:59:29.148 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:29.145+0000 7fbd98ea4700 1 -- 172.21.15.59:0/3815529817 shutdown_connections 2023-11-24T07:59:29.148 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:29.145+0000 7fbd98ea4700 1 --2- 172.21.15.59:0/3815529817 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7fbd7406b2c0 0x7fbd7406d780 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:29.148 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:29.145+0000 7fbd98ea4700 1 --2- 172.21.15.59:0/3815529817 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fbd9415e1c0 0x7fbd94161df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:29.148 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:29.145+0000 7fbd98ea4700 1 --2- 172.21.15.59:0/3815529817 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbd9406fc50 0x7fbd94160460 unknown :-1 s=CLOSED pgs=901 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:29.148 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:29.145+0000 7fbd98ea4700 1 --2- 172.21.15.59:0/3815529817 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fbd940b0e90 0x7fbd940b67c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:29.148 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:29.145+0000 7fbd98ea4700 1 -- 172.21.15.59:0/3815529817 >> 172.21.15.59:0/3815529817 conn(0x7fbd940b3720 msgr2=0x7fbd9415ce30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:59:29.148 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:29.146+0000 7fbd98ea4700 1 -- 172.21.15.59:0/3815529817 shutdown_connections 2023-11-24T07:59:29.148 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:29.146+0000 7fbd98ea4700 1 -- 172.21.15.59:0/3815529817 wait complete. 2023-11-24T07:59:29.159 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolumegroup create cephfs subvol_grp1853 2023-11-24T07:59:29.288 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:29.286+0000 7ff4b1b15700 1 Processor -- start 2023-11-24T07:59:29.288 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:29.286+0000 7ff4b1b15700 1 -- start start 2023-11-24T07:59:29.291 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:29.287+0000 7ff4b1b15700 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7ff4ac0cecd0 0x7ff4ac15c1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:29.291 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:29.287+0000 7ff4b1b15700 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff4ac0d0a90 con 0x7ff4ac15bde0 2023-11-24T07:59:29.291 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:29.287+0000 7ff4b1b15700 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff4ac06ee10 con 0x7ff4ac0cf2b0 2023-11-24T07:59:29.291 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:29.287+0000 7ff4b1b15700 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff4ac06f0f0 con 0x7ff4ac0cecd0 2023-11-24T07:59:29.291 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:29.287+0000 7ff4ab7fe700 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7ff4ac15bde0 0x7ff4ac0d01b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:58578/0 (socket says 172.21.15.59:58578) 2023-11-24T07:59:29.291 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:29.287+0000 7ff4ab7fe700 1 -- 172.21.15.59:0/436562482 learned_addr learned my addr 172.21.15.59:0/436562482 (peer_addr_for_me v1:172.21.15.59:0/0) 2023-11-24T07:59:29.291 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:29.287+0000 7ff4abfff700 1 --2- 172.21.15.59:0/436562482 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7ff4ac0cecd0 0x7ff4ac15c1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:29.291 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:29.288+0000 7ff4aa7fc700 1 -- 172.21.15.59:0/436562482 <== mon.0 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1929847407 0 0) 0x7ff4ac0d0a90 con 0x7ff4ac15bde0 2023-11-24T07:59:29.291 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:29.288+0000 7ff4aa7fc700 1 -- 172.21.15.59:0/436562482 --> v1:172.21.15.59:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff4980035b0 con 0x7ff4ac15bde0 2023-11-24T07:59:29.291 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:29.288+0000 7ff4aa7fc700 1 -- 172.21.15.59:0/436562482 <== mon.1 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4252793496 0 0) 0x7ff4ac06ee10 con 0x7ff4ac0cf2b0 2023-11-24T07:59:29.292 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:29.288+0000 7ff4aa7fc700 1 -- 172.21.15.59:0/436562482 --> v1:172.21.15.76:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff4ac0d0a90 con 0x7ff4ac0cf2b0 2023-11-24T07:59:29.292 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:29.288+0000 7ff4aa7fc700 1 -- 172.21.15.59:0/436562482 <== mon.0 v1:172.21.15.59:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3385567201 0 0) 0x7ff4980035b0 con 0x7ff4ac15bde0 2023-11-24T07:59:29.292 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:29.288+0000 7ff4abfff700 1 -- 172.21.15.59:0/436562482 >> v1:172.21.15.76:6789/0 conn(0x7ff4ac0cf2b0 legacy=0x7ff4ac0ce5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:29.292 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:29.288+0000 7ff4abfff700 1 -- 172.21.15.59:0/436562482 >> v1:172.21.15.59:6789/0 conn(0x7ff4ac15bde0 legacy=0x7ff4ac0d01b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:29.292 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:29.288+0000 7ff4abfff700 1 -- 172.21.15.59:0/436562482 --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff4ac06f3d0 con 0x7ff4ac0cecd0 2023-11-24T07:59:29.292 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:29.288+0000 7ff4abfff700 1 --2- 172.21.15.59:0/436562482 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7ff4ac0cecd0 0x7ff4ac15c1c0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7ff4a4009dc0 tx=0x7ff4a400b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=40e459ce1bf7a8b5 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:59:29.292 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:29.288+0000 7ff4aa7fc700 1 -- 172.21.15.59:0/436562482 <== mon.2 v2:172.21.15.76:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4a400e070 con 0x7ff4ac0cecd0 2023-11-24T07:59:29.292 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:29.288+0000 7ff4aa7fc700 1 -- 172.21.15.59:0/436562482 <== mon.2 v2:172.21.15.76:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff4a4003a20 con 0x7ff4ac0cecd0 2023-11-24T07:59:29.292 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:29.288+0000 7ff4aa7fc700 1 -- 172.21.15.59:0/436562482 <== mon.2 v2:172.21.15.76:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff49c001670 con 0x7ff4ac0cecd0 2023-11-24T07:59:29.292 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:29.289+0000 7ff4b1b15700 1 -- 172.21.15.59:0/436562482 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7ff4ac0cecd0 msgr2=0x7ff4ac15c1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:29.292 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:29.289+0000 7ff4b1b15700 1 --2- 172.21.15.59:0/436562482 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7ff4ac0cecd0 0x7ff4ac15c1c0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7ff4a4009dc0 tx=0x7ff4a400b330 comp rx=0 tx=0).stop 2023-11-24T07:59:29.292 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:29.289+0000 7ff4b1b15700 1 -- 172.21.15.59:0/436562482 shutdown_connections 2023-11-24T07:59:29.293 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:29.289+0000 7ff4b1b15700 1 --2- 172.21.15.59:0/436562482 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7ff4ac0cecd0 0x7ff4ac15c1c0 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:29.293 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:29.289+0000 7ff4b1b15700 1 -- 172.21.15.59:0/436562482 >> 172.21.15.59:0/436562482 conn(0x7ff4ac0b3720 msgr2=0x7ff4ac0b3b30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:59:29.293 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:29.289+0000 7ff4b1b15700 1 -- 172.21.15.59:0/436562482 shutdown_connections 2023-11-24T07:59:29.293 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:29.289+0000 7ff4b1b15700 1 -- 172.21.15.59:0/436562482 wait complete. 2023-11-24T07:59:29.295 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:29.290+0000 7ff4b1b15700 1 Processor -- start 2023-11-24T07:59:29.296 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:29.290+0000 7ff4b1b15700 1 -- start start 2023-11-24T07:59:29.296 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:29.291+0000 7ff4b1b15700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff4ac0cecd0 0x7ff4ac0cb450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:29.296 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:29.291+0000 7ff4b1b15700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff4ac0cf2b0 0x7ff4ac0cdeb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:29.296 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:29.291+0000 7ff4b1b15700 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7ff4ac15bde0 0x7ff4ac0c9cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:29.296 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:29.291+0000 7ff4b1b15700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4ac0b6740 con 0x7ff4ac0cecd0 2023-11-24T07:59:29.296 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:29.291+0000 7ff4b1b15700 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4ac0cbae0 con 0x7ff4ac0cf2b0 2023-11-24T07:59:29.296 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:29.291+0000 7ff4b1b15700 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff4ac0b5da0 con 0x7ff4ac15bde0 2023-11-24T07:59:29.296 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:29.291+0000 7ff4ab7fe700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff4ac0cecd0 0x7ff4ac0cb450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:29.296 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:29.291+0000 7ff4ab7fe700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff4ac0cecd0 0x7ff4ac0cb450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:55110/0 (socket says 172.21.15.59:55110) 2023-11-24T07:59:29.296 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:29.291+0000 7ff4ab7fe700 1 -- 172.21.15.59:0/4097015442 learned_addr learned my addr 172.21.15.59:0/4097015442 (peer_addr_for_me v2:172.21.15.59:0/0) 2023-11-24T07:59:29.296 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:29.291+0000 7ff4aaffd700 1 --2- 172.21.15.59:0/4097015442 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff4ac0cf2b0 0x7ff4ac0cdeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:29.296 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:29.291+0000 7ff4abfff700 1 --2- 172.21.15.59:0/4097015442 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7ff4ac15bde0 0x7ff4ac0c9cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:29.297 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:29.291+0000 7ff4ab7fe700 1 -- 172.21.15.59:0/4097015442 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7ff4ac15bde0 msgr2=0x7ff4ac0c9cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:29.297 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:29.291+0000 7ff4ab7fe700 1 --2- 172.21.15.59:0/4097015442 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7ff4ac15bde0 0x7ff4ac0c9cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:29.297 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:29.291+0000 7ff4ab7fe700 1 -- 172.21.15.59:0/4097015442 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff4ac0cf2b0 msgr2=0x7ff4ac0cdeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:29.297 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:29.291+0000 7ff4ab7fe700 1 --2- 172.21.15.59:0/4097015442 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff4ac0cf2b0 0x7ff4ac0cdeb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:29.297 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:29.291+0000 7ff4ab7fe700 1 -- 172.21.15.59:0/4097015442 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff4a4009a70 con 0x7ff4ac0cecd0 2023-11-24T07:59:29.297 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:29.291+0000 7ff4aaffd700 1 --2- 172.21.15.59:0/4097015442 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff4ac0cf2b0 0x7ff4ac0cdeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-24T07:59:29.297 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:29.291+0000 7ff4ab7fe700 1 --2- 172.21.15.59:0/4097015442 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff4ac0cecd0 0x7ff4ac0cb450 secure :-1 s=READY pgs=903 cs=0 l=1 rev1=1 crypto rx=0x7ff49c00c6b0 tx=0x7ff49c00ca70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:59:29.297 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:29.292+0000 7ff4a3fff700 1 -- 172.21.15.59:0/4097015442 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff49c004c80 con 0x7ff4ac0cecd0 2023-11-24T07:59:29.297 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:29.292+0000 7ff4a3fff700 1 -- 172.21.15.59:0/4097015442 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff49c004e00 con 0x7ff4ac0cecd0 2023-11-24T07:59:29.297 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:29.292+0000 7ff4b1b15700 1 -- 172.21.15.59:0/4097015442 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff4ac0b6080 con 0x7ff4ac0cecd0 2023-11-24T07:59:29.297 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:29.292+0000 7ff4a3fff700 1 -- 172.21.15.59:0/4097015442 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff49c007310 con 0x7ff4ac0cecd0 2023-11-24T07:59:29.297 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:29.292+0000 7ff4b1b15700 1 -- 172.21.15.59:0/4097015442 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff4ac150a30 con 0x7ff4ac0cecd0 2023-11-24T07:59:29.297 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:29.293+0000 7ff4b1b15700 1 -- 172.21.15.59:0/4097015442 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff4ac075480 con 0x7ff4ac0cecd0 2023-11-24T07:59:29.298 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:29.293+0000 7ff4a3fff700 1 -- 172.21.15.59:0/4097015442 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 10) v1 ==== 89588+0+0 (secure 0 0 0) 0x7ff49c007470 con 0x7ff4ac0cecd0 2023-11-24T07:59:29.298 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:29.293+0000 7ff4a3fff700 1 --2- 172.21.15.59:0/4097015442 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7ff48c06b2c0 0x7ff48c06d780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:29.298 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:29.294+0000 7ff4a3fff700 1 -- 172.21.15.59:0/4097015442 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 5709+0+0 (secure 0 0 0) 0x7ff49c089970 con 0x7ff4ac0cecd0 2023-11-24T07:59:29.298 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:29.294+0000 7ff4aaffd700 1 --2- 172.21.15.59:0/4097015442 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7ff48c06b2c0 0x7ff48c06d780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:29.298 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:29.294+0000 7ff4aaffd700 1 --2- 172.21.15.59:0/4097015442 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7ff48c06b2c0 0x7ff48c06d780 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7ff4940011e0 tx=0x7ff494009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:59:29.299 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:29.298+0000 7ff4a3fff700 1 -- 172.21.15.59:0/4097015442 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7ff49c0545e0 con 0x7ff4ac0cecd0 2023-11-24T07:59:29.533 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:29.531+0000 7ff4b1b15700 1 -- 172.21.15.59:0/4097015442 --> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] -- mgr_command(tid 0: {"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "subvol_grp1853", "target": ["mon-mgr", ""]}) v1 -- 0x7ff4ac166c40 con 0x7ff48c06b2c0 2023-11-24T07:59:29.540 INFO:tasks.ceph.mgr.x.smithi076.stderr:2023-11-24T07:59:29.539+0000 7ff196724700 -1 client.0 error registering admin socket command: (17) File exists 2023-11-24T07:59:29.540 INFO:tasks.ceph.mgr.x.smithi076.stderr:2023-11-24T07:59:29.539+0000 7ff196724700 -1 client.0 error registering admin socket command: (17) File exists 2023-11-24T07:59:29.540 INFO:tasks.ceph.mgr.x.smithi076.stderr:2023-11-24T07:59:29.539+0000 7ff196724700 -1 client.0 error registering admin socket command: (17) File exists 2023-11-24T07:59:29.540 INFO:tasks.ceph.mgr.x.smithi076.stderr:2023-11-24T07:59:29.539+0000 7ff196724700 -1 client.0 error registering admin socket command: (17) File exists 2023-11-24T07:59:29.540 INFO:tasks.ceph.mgr.x.smithi076.stderr:2023-11-24T07:59:29.539+0000 7ff196724700 -1 client.0 error registering admin socket command: (17) File exists 2023-11-24T07:59:29.555 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:29.553+0000 7ff4a3fff700 1 -- 172.21.15.59:0/4097015442 <== mgr.4100 v2:172.21.15.76:6832/4264046536 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7ff494001cf0 con 0x7ff48c06b2c0 2023-11-24T07:59:29.557 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:29.555+0000 7ff4b1b15700 1 -- 172.21.15.59:0/4097015442 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7ff48c06b2c0 msgr2=0x7ff48c06d780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:29.557 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:29.555+0000 7ff4b1b15700 1 --2- 172.21.15.59:0/4097015442 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7ff48c06b2c0 0x7ff48c06d780 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7ff4940011e0 tx=0x7ff494009040 comp rx=0 tx=0).stop 2023-11-24T07:59:29.558 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:29.555+0000 7ff4b1b15700 1 -- 172.21.15.59:0/4097015442 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff4ac0cecd0 msgr2=0x7ff4ac0cb450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:29.558 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:29.555+0000 7ff4b1b15700 1 --2- 172.21.15.59:0/4097015442 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff4ac0cecd0 0x7ff4ac0cb450 secure :-1 s=READY pgs=903 cs=0 l=1 rev1=1 crypto rx=0x7ff49c00c6b0 tx=0x7ff49c00ca70 comp rx=0 tx=0).stop 2023-11-24T07:59:29.558 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:29.555+0000 7ff4b1b15700 1 -- 172.21.15.59:0/4097015442 shutdown_connections 2023-11-24T07:59:29.558 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:29.555+0000 7ff4b1b15700 1 --2- 172.21.15.59:0/4097015442 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7ff48c06b2c0 0x7ff48c06d780 unknown :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:29.558 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:29.555+0000 7ff4b1b15700 1 --2- 172.21.15.59:0/4097015442 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7ff4ac15bde0 0x7ff4ac0c9cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:29.558 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:29.555+0000 7ff4b1b15700 1 --2- 172.21.15.59:0/4097015442 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff4ac0cecd0 0x7ff4ac0cb450 unknown :-1 s=CLOSED pgs=903 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:29.558 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:29.555+0000 7ff4b1b15700 1 --2- 172.21.15.59:0/4097015442 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff4ac0cf2b0 0x7ff4ac0cdeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:29.558 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:29.555+0000 7ff4b1b15700 1 -- 172.21.15.59:0/4097015442 >> 172.21.15.59:0/4097015442 conn(0x7ff4ac0b3720 msgr2=0x7ff4ac0af830 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:59:29.558 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:29.556+0000 7ff4b1b15700 1 -- 172.21.15.59:0/4097015442 shutdown_connections 2023-11-24T07:59:29.558 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:29.556+0000 7ff4b1b15700 1 -- 172.21.15.59:0/4097015442 wait complete. 2023-11-24T07:59:29.569 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume create cephfs subvol8253 --group_name subvol_grp1853 2023-11-24T07:59:29.701 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:29.699+0000 7f328b3fe700 1 Processor -- start 2023-11-24T07:59:29.701 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:29.699+0000 7f328b3fe700 1 -- start start 2023-11-24T07:59:29.703 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:29.700+0000 7f328b3fe700 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f32840722c0 0x7f328406efa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:29.705 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:29.700+0000 7f328b3fe700 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f32840b4a80 0x7f328406f4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:29.705 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:29.700+0000 7f328b3fe700 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f32840b45f0 con 0x7f3284158700 2023-11-24T07:59:29.705 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:29.700+0000 7f328b3fe700 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f328406fcd0 con 0x7f32840b4a80 2023-11-24T07:59:29.706 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:29.700+0000 7f328b3fe700 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f328406fa20 con 0x7f32840722c0 2023-11-24T07:59:29.706 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:29.700+0000 7f328919a700 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7f3284158700 0x7f3284071bb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:58592/0 (socket says 172.21.15.59:58592) 2023-11-24T07:59:29.706 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:29.700+0000 7f328919a700 1 -- 172.21.15.59:0/2065617754 learned_addr learned my addr 172.21.15.59:0/2065617754 (peer_addr_for_me v1:172.21.15.59:0/0) 2023-11-24T07:59:29.706 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:29.700+0000 7f328999b700 1 --2- 172.21.15.59:0/2065617754 >> v2:172.21.15.76:3300/0 conn(0x7f32840b4a80 0x7f328406f4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:29.707 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:29.700+0000 7f3288999700 1 --2- 172.21.15.59:0/2065617754 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f32840722c0 0x7f328406efa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:29.708 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:29.700+0000 7f327bfff700 1 -- 172.21.15.59:0/2065617754 <== mon.0 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1166209411 0 0) 0x7f32840b45f0 con 0x7f3284158700 2023-11-24T07:59:29.708 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:29.700+0000 7f327bfff700 1 -- 172.21.15.59:0/2065617754 --> v1:172.21.15.59:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f32700035b0 con 0x7f3284158700 2023-11-24T07:59:29.708 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:29.701+0000 7f3288999700 1 -- 172.21.15.59:0/2065617754 >> v2:172.21.15.76:3300/0 conn(0x7f32840b4a80 msgr2=0x7f328406f4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T07:59:29.708 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:29.701+0000 7f3288999700 1 --2- 172.21.15.59:0/2065617754 >> v2:172.21.15.76:3300/0 conn(0x7f32840b4a80 0x7f328406f4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:29.708 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:29.701+0000 7f3288999700 1 -- 172.21.15.59:0/2065617754 >> v1:172.21.15.59:6789/0 conn(0x7f3284158700 legacy=0x7f3284071bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:29.708 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:29.701+0000 7f3288999700 1 -- 172.21.15.59:0/2065617754 --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3284158ae0 con 0x7f32840722c0 2023-11-24T07:59:29.708 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:29.701+0000 7f327bfff700 1 -- 172.21.15.59:0/2065617754 <== mon.0 v1:172.21.15.59:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 438133493 0 0) 0x7f32700035b0 con 0x7f3284158700 2023-11-24T07:59:29.708 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:29.701+0000 7f3288999700 1 --2- 172.21.15.59:0/2065617754 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f32840722c0 0x7f328406efa0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f3280009840 tx=0x7f3280009b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ad9991c3c3fdec45 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:59:29.708 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:29.701+0000 7f327bfff700 1 -- 172.21.15.59:0/2065617754 <== mon.2 v2:172.21.15.76:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3274001670 con 0x7f32840722c0 2023-11-24T07:59:29.708 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:29.701+0000 7f327bfff700 1 -- 172.21.15.59:0/2065617754 <== mon.2 v2:172.21.15.76:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3280004030 con 0x7f32840722c0 2023-11-24T07:59:29.708 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:29.701+0000 7f327bfff700 1 -- 172.21.15.59:0/2065617754 <== mon.2 v2:172.21.15.76:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32800043c0 con 0x7f32840722c0 2023-11-24T07:59:29.709 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:29.702+0000 7f328b3fe700 1 -- 172.21.15.59:0/2065617754 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f32840722c0 msgr2=0x7f328406efa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:29.709 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:29.702+0000 7f328b3fe700 1 --2- 172.21.15.59:0/2065617754 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f32840722c0 0x7f328406efa0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f3280009840 tx=0x7f3280009b50 comp rx=0 tx=0).stop 2023-11-24T07:59:29.709 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:29.702+0000 7f328b3fe700 1 -- 172.21.15.59:0/2065617754 shutdown_connections 2023-11-24T07:59:29.709 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:29.702+0000 7f328b3fe700 1 --2- 172.21.15.59:0/2065617754 >> v2:172.21.15.76:3300/0 conn(0x7f32840b4a80 0x7f328406f4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:29.709 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:29.702+0000 7f328b3fe700 1 --2- 172.21.15.59:0/2065617754 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f32840722c0 0x7f328406efa0 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:29.709 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:29.702+0000 7f328b3fe700 1 -- 172.21.15.59:0/2065617754 >> 172.21.15.59:0/2065617754 conn(0x7f32840b2840 msgr2=0x7f32840b2c20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:59:29.709 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:29.702+0000 7f328b3fe700 1 -- 172.21.15.59:0/2065617754 shutdown_connections 2023-11-24T07:59:29.709 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:29.702+0000 7f328b3fe700 1 -- 172.21.15.59:0/2065617754 wait complete. 2023-11-24T07:59:29.712 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:29.703+0000 7f328b3fe700 1 Processor -- start 2023-11-24T07:59:29.712 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:29.703+0000 7f328b3fe700 1 -- start start 2023-11-24T07:59:29.712 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:29.704+0000 7f328b3fe700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f32840722c0 0x7f32841455d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:29.712 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:29.704+0000 7f328b3fe700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f32840b4a80 0x7f3284154ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:29.712 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:29.704+0000 7f328b3fe700 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f3284158700 0x7f32841e66f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:29.712 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:29.704+0000 7f328b3fe700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3284157480 con 0x7f32840b4a80 2023-11-24T07:59:29.712 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:29.704+0000 7f328b3fe700 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3284157720 con 0x7f32840722c0 2023-11-24T07:59:29.712 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:29.704+0000 7f328b3fe700 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3284157a00 con 0x7f3284158700 2023-11-24T07:59:29.712 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:29.704+0000 7f3288999700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f32840b4a80 0x7f3284154ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:29.712 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:29.704+0000 7f3288999700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f32840b4a80 0x7f3284154ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:55126/0 (socket says 172.21.15.59:55126) 2023-11-24T07:59:29.713 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:29.704+0000 7f3288999700 1 -- 172.21.15.59:0/606825801 learned_addr learned my addr 172.21.15.59:0/606825801 (peer_addr_for_me v2:172.21.15.59:0/0) 2023-11-24T07:59:29.713 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:29.704+0000 7f328999b700 1 --2- 172.21.15.59:0/606825801 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f3284158700 0x7f32841e66f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:29.713 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:29.704+0000 7f328919a700 1 --2- 172.21.15.59:0/606825801 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f32840722c0 0x7f32841455d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:29.713 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:29.704+0000 7f3288999700 1 -- 172.21.15.59:0/606825801 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f3284158700 msgr2=0x7f32841e66f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:29.713 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:29.704+0000 7f3288999700 1 --2- 172.21.15.59:0/606825801 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f3284158700 0x7f32841e66f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:29.713 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:29.704+0000 7f3288999700 1 -- 172.21.15.59:0/606825801 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f32840722c0 msgr2=0x7f32841455d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:29.713 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:29.704+0000 7f3288999700 1 --2- 172.21.15.59:0/606825801 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f32840722c0 0x7f32841455d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:29.713 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:29.704+0000 7f3288999700 1 -- 172.21.15.59:0/606825801 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f32800094a0 con 0x7f32840b4a80 2023-11-24T07:59:29.713 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:29.704+0000 7f3288999700 1 --2- 172.21.15.59:0/606825801 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f32840b4a80 0x7f3284154ca0 secure :-1 s=READY pgs=905 cs=0 l=1 rev1=1 crypto rx=0x7f32800090b0 tx=0x7f3280004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:59:29.713 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:29.704+0000 7f3279ffb700 1 -- 172.21.15.59:0/606825801 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f328001f070 con 0x7f32840b4a80 2023-11-24T07:59:29.713 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:29.704+0000 7f3279ffb700 1 -- 172.21.15.59:0/606825801 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3280003860 con 0x7f32840b4a80 2023-11-24T07:59:29.713 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:29.704+0000 7f328b3fe700 1 -- 172.21.15.59:0/606825801 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f328415c020 con 0x7f32840b4a80 2023-11-24T07:59:29.714 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:29.704+0000 7f3279ffb700 1 -- 172.21.15.59:0/606825801 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3280003bf0 con 0x7f32840b4a80 2023-11-24T07:59:29.714 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:29.705+0000 7f328b3fe700 1 -- 172.21.15.59:0/606825801 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f32841551e0 con 0x7f32840b4a80 2023-11-24T07:59:29.714 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:29.705+0000 7f328b3fe700 1 -- 172.21.15.59:0/606825801 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3284155430 con 0x7f32840b4a80 2023-11-24T07:59:29.714 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:29.706+0000 7f3279ffb700 1 -- 172.21.15.59:0/606825801 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 10) v1 ==== 89588+0+0 (secure 0 0 0) 0x7f3280023070 con 0x7f32840b4a80 2023-11-24T07:59:29.714 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:29.706+0000 7f3279ffb700 1 --2- 172.21.15.59:0/606825801 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f326806b2c0 0x7f326806d780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:29.714 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:29.707+0000 7f3279ffb700 1 -- 172.21.15.59:0/606825801 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f328008c250 con 0x7f32840b4a80 2023-11-24T07:59:29.714 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:29.707+0000 7f328919a700 1 --2- 172.21.15.59:0/606825801 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f326806b2c0 0x7f326806d780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:29.714 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:29.707+0000 7f328919a700 1 --2- 172.21.15.59:0/606825801 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f326806b2c0 0x7f326806d780 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f32740011e0 tx=0x7f3274007ba0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:59:29.714 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:29.711+0000 7f3279ffb700 1 -- 172.21.15.59:0/606825801 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f3280056d30 con 0x7f32840b4a80 2023-11-24T07:59:29.945 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:29.944+0000 7f328b3fe700 1 -- 172.21.15.59:0/606825801 --> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "subvol8253", "group_name": "subvol_grp1853", "target": ["mon-mgr", ""]}) v1 -- 0x7f32840b4280 con 0x7f326806b2c0 2023-11-24T07:59:29.973 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:29.971+0000 7f3279ffb700 1 -- 172.21.15.59:0/606825801 <== mgr.4100 v2:172.21.15.76:6832/4264046536 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7f3274004e50 con 0x7f326806b2c0 2023-11-24T07:59:29.976 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:29.974+0000 7f328b3fe700 1 -- 172.21.15.59:0/606825801 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f326806b2c0 msgr2=0x7f326806d780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:29.976 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:29.974+0000 7f328b3fe700 1 --2- 172.21.15.59:0/606825801 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f326806b2c0 0x7f326806d780 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f32740011e0 tx=0x7f3274007ba0 comp rx=0 tx=0).stop 2023-11-24T07:59:29.976 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:29.974+0000 7f328b3fe700 1 -- 172.21.15.59:0/606825801 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f32840b4a80 msgr2=0x7f3284154ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:29.976 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:29.974+0000 7f328b3fe700 1 --2- 172.21.15.59:0/606825801 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f32840b4a80 0x7f3284154ca0 secure :-1 s=READY pgs=905 cs=0 l=1 rev1=1 crypto rx=0x7f32800090b0 tx=0x7f3280004750 comp rx=0 tx=0).stop 2023-11-24T07:59:29.976 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:29.974+0000 7f328b3fe700 1 -- 172.21.15.59:0/606825801 shutdown_connections 2023-11-24T07:59:29.976 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:29.974+0000 7f328b3fe700 1 --2- 172.21.15.59:0/606825801 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f326806b2c0 0x7f326806d780 unknown :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:29.976 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:29.974+0000 7f328b3fe700 1 --2- 172.21.15.59:0/606825801 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f3284158700 0x7f32841e66f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:29.976 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:29.974+0000 7f328b3fe700 1 --2- 172.21.15.59:0/606825801 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f32840722c0 0x7f32841455d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:29.977 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:29.974+0000 7f328b3fe700 1 --2- 172.21.15.59:0/606825801 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f32840b4a80 0x7f3284154ca0 unknown :-1 s=CLOSED pgs=905 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:29.977 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:29.974+0000 7f328b3fe700 1 -- 172.21.15.59:0/606825801 >> 172.21.15.59:0/606825801 conn(0x7f32840b2840 msgr2=0x7f3284153760 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:59:29.977 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:29.974+0000 7f328b3fe700 1 -- 172.21.15.59:0/606825801 shutdown_connections 2023-11-24T07:59:29.977 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:29.974+0000 7f328b3fe700 1 -- 172.21.15.59:0/606825801 wait complete. 2023-11-24T07:59:29.988 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T07:59:29.988 DEBUG:teuthology.orchestra.run.smithi059:> (cd /home/ubuntu/cephtest/mnt.0 && exec sudo mkdir -p ./volumes/subvol_grp1853/.snap/ancestral_snap_1) 2023-11-24T07:59:30.048 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T07:59:30.048 DEBUG:teuthology.orchestra.run.smithi059:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%i ./volumes/subvol_grp1853) 2023-11-24T07:59:30.069 INFO:teuthology.orchestra.run.smithi059.stdout:1099511627777 2023-11-24T07:59:30.070 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T07:59:30.070 DEBUG:teuthology.orchestra.run.smithi059:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls ./volumes/subvol_grp1853/subvol8253/.snap/_ancestral_snap_1_1099511627777) 2023-11-24T07:59:30.129 INFO:teuthology.orchestra.run.smithi059.stdout:34a9476e-a82e-4ad6-8fe6-b140979ada4a 2023-11-24T07:59:30.129 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot info cephfs subvol8253 _ancestral_snap_1_1099511627777 subvol_grp1853 2023-11-24T07:59:30.302 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:30.301+0000 7f7869996700 1 Processor -- start 2023-11-24T07:59:30.303 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:30.301+0000 7f7869996700 1 -- start start 2023-11-24T07:59:30.306 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:30.301+0000 7f7869996700 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f786415d7f0 0x7f786406d2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:30.306 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:30.301+0000 7f7869996700 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f78640b39b0 con 0x7f786406d800 2023-11-24T07:59:30.306 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:30.302+0000 7f7869996700 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f78640cf540 con 0x7f78641512e0 2023-11-24T07:59:30.306 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:30.302+0000 7f7869996700 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f786406dc80 con 0x7f786415d7f0 2023-11-24T07:59:30.306 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:30.302+0000 7f78627fc700 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7f786406d800 0x7f7864150bd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:58606/0 (socket says 172.21.15.59:58606) 2023-11-24T07:59:30.306 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:30.302+0000 7f78627fc700 1 -- 172.21.15.59:0/2817688762 learned_addr learned my addr 172.21.15.59:0/2817688762 (peer_addr_for_me v1:172.21.15.59:0/0) 2023-11-24T07:59:30.306 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:30.302+0000 7f7861ffb700 1 -- 172.21.15.59:0/2817688762 <== mon.0 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1529943857 0 0) 0x7f78640b39b0 con 0x7f786406d800 2023-11-24T07:59:30.306 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:30.302+0000 7f7861ffb700 1 -- 172.21.15.59:0/2817688762 --> v1:172.21.15.59:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f784c0035b0 con 0x7f786406d800 2023-11-24T07:59:30.306 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:30.302+0000 7f7862ffd700 1 --2- 172.21.15.59:0/2817688762 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f786415d7f0 0x7f786406d2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:30.306 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:30.302+0000 7f7861ffb700 1 -- 172.21.15.59:0/2817688762 <== mon.0 v1:172.21.15.59:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 809317717 0 0) 0x7f784c0035b0 con 0x7f786406d800 2023-11-24T07:59:30.307 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:30.302+0000 7f7861ffb700 1 -- 172.21.15.59:0/2817688762 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f786415d7f0 msgr2=0x7f786406d2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T07:59:30.307 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:30.302+0000 7f7861ffb700 1 --2- 172.21.15.59:0/2817688762 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f786415d7f0 0x7f786406d2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:30.307 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:30.302+0000 7f7861ffb700 1 -- 172.21.15.59:0/2817688762 >> v1:172.21.15.76:6789/0 conn(0x7f78641512e0 legacy=0x7f786415b420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:30.307 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:30.302+0000 7f7861ffb700 1 -- 172.21.15.59:0/2817688762 --> v1:172.21.15.59:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f78640b2da0 con 0x7f786406d800 2023-11-24T07:59:30.307 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:30.302+0000 7f7861ffb700 1 -- 172.21.15.59:0/2817688762 <== mon.0 v1:172.21.15.59:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1312333566 0 0) 0x7f7858001670 con 0x7f786406d800 2023-11-24T07:59:30.307 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:30.302+0000 7f7861ffb700 1 -- 172.21.15.59:0/2817688762 >> v1:172.21.15.59:6789/0 conn(0x7f786406d800 legacy=0x7f7864150bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:30.307 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:30.302+0000 7f7861ffb700 1 --2- 172.21.15.59:0/2817688762 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f786415d7f0 0x7f784c003a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:30.307 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:30.302+0000 7f7861ffb700 1 --2- 172.21.15.59:0/2817688762 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f784c004060 0x7f784c004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:30.307 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:30.302+0000 7f7861ffb700 1 --2- 172.21.15.59:0/2817688762 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f784c004d90 0x7f784c005210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:30.307 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:30.302+0000 7f7861ffb700 1 -- 172.21.15.59:0/2817688762 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f786406dc80 con 0x7f784c004060 2023-11-24T07:59:30.307 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:30.302+0000 7f7861ffb700 1 -- 172.21.15.59:0/2817688762 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f784c00bc60 con 0x7f784c004d90 2023-11-24T07:59:30.307 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:30.302+0000 7f7861ffb700 1 -- 172.21.15.59:0/2817688762 --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f784c00be20 con 0x7f786415d7f0 2023-11-24T07:59:30.308 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:30.302+0000 7f78627fc700 1 --2- 172.21.15.59:0/2817688762 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f784c004060 0x7f784c004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:30.308 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:30.302+0000 7f7862ffd700 1 --2- 172.21.15.59:0/2817688762 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f786415d7f0 0x7f784c003a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:30.308 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:30.302+0000 7f78637fe700 1 --2- 172.21.15.59:0/2817688762 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f784c004d90 0x7f784c005210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:30.308 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:30.303+0000 7f78627fc700 1 -- 172.21.15.59:0/2817688762 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f786415d7f0 msgr2=0x7f784c003a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T07:59:30.308 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:30.303+0000 7f78627fc700 1 --2- 172.21.15.59:0/2817688762 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f786415d7f0 0x7f784c003a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:30.308 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:30.303+0000 7f78627fc700 1 -- 172.21.15.59:0/2817688762 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f784c004d90 msgr2=0x7f784c005210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T07:59:30.308 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:30.303+0000 7f78627fc700 1 --2- 172.21.15.59:0/2817688762 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f784c004d90 0x7f784c005210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:30.308 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:30.303+0000 7f78627fc700 1 -- 172.21.15.59:0/2817688762 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f784c0035b0 con 0x7f784c004060 2023-11-24T07:59:30.308 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:30.303+0000 7f78637fe700 1 --2- 172.21.15.59:0/2817688762 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f784c004d90 0x7f784c005210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-24T07:59:30.308 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:30.303+0000 7f78627fc700 1 --2- 172.21.15.59:0/2817688762 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f784c004060 0x7f784c004850 secure :-1 s=READY pgs=907 cs=0 l=1 rev1=1 crypto rx=0x7f785800c240 tx=0x7f784c005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ca2d2e470fc11565 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:59:30.308 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:30.303+0000 7f7861ffb700 1 -- 172.21.15.59:0/2817688762 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7858007c10 con 0x7f784c004060 2023-11-24T07:59:30.308 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:30.303+0000 7f7861ffb700 1 -- 172.21.15.59:0/2817688762 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f785800ce00 con 0x7f784c004060 2023-11-24T07:59:30.308 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:30.303+0000 7f7861ffb700 1 -- 172.21.15.59:0/2817688762 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7858005ba0 con 0x7f784c004060 2023-11-24T07:59:30.309 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:30.303+0000 7f7869996700 1 -- 172.21.15.59:0/2817688762 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f784c004060 msgr2=0x7f784c004850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:30.309 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:30.303+0000 7f7869996700 1 --2- 172.21.15.59:0/2817688762 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f784c004060 0x7f784c004850 secure :-1 s=READY pgs=907 cs=0 l=1 rev1=1 crypto rx=0x7f785800c240 tx=0x7f784c005880 comp rx=0 tx=0).stop 2023-11-24T07:59:30.309 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:30.303+0000 7f78637fe700 1 -- 172.21.15.59:0/2817688762 reap_dead start 2023-11-24T07:59:30.309 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:30.304+0000 7f7869996700 1 -- 172.21.15.59:0/2817688762 shutdown_connections 2023-11-24T07:59:30.309 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:30.304+0000 7f7869996700 1 -- 172.21.15.59:0/2817688762 >> 172.21.15.59:0/2817688762 conn(0x7f78640b1650 msgr2=0x7f78640b1a30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:59:30.309 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:30.304+0000 7f7869996700 1 -- 172.21.15.59:0/2817688762 shutdown_connections 2023-11-24T07:59:30.309 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:30.304+0000 7f7869996700 1 -- 172.21.15.59:0/2817688762 wait complete. 2023-11-24T07:59:30.312 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:30.305+0000 7f7869996700 1 Processor -- start 2023-11-24T07:59:30.312 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:30.305+0000 7f7869996700 1 -- start start 2023-11-24T07:59:30.312 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:30.306+0000 7f7869996700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f784c004060 0x7f786413f330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:30.312 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:30.306+0000 7f7869996700 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f78641491f0 0x7f7864149670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:30.312 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:30.306+0000 7f7869996700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7864151730 0x7f7864151b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:30.312 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:30.306+0000 7f7869996700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7864138370 con 0x7f7864151730 2023-11-24T07:59:30.312 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:30.306+0000 7f7869996700 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7864145b90 con 0x7f784c004060 2023-11-24T07:59:30.312 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:30.306+0000 7f7869996700 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7864149bb0 con 0x7f78641491f0 2023-11-24T07:59:30.313 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:30.306+0000 7f78637fe700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7864151730 0x7f7864151b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:30.313 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:30.306+0000 7f78637fe700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7864151730 0x7f7864151b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:55148/0 (socket says 172.21.15.59:55148) 2023-11-24T07:59:30.313 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:30.306+0000 7f7862ffd700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f784c004060 0x7f786413f330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:30.313 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:30.306+0000 7f78637fe700 1 -- 172.21.15.59:0/3890148851 learned_addr learned my addr 172.21.15.59:0/3890148851 (peer_addr_for_me v2:172.21.15.59:0/0) 2023-11-24T07:59:30.313 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:30.306+0000 7f78627fc700 1 --2- 172.21.15.59:0/3890148851 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f78641491f0 0x7f7864149670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:30.313 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:30.306+0000 7f78637fe700 1 -- 172.21.15.59:0/3890148851 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f78641491f0 msgr2=0x7f7864149670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:30.313 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:30.306+0000 7f78637fe700 1 --2- 172.21.15.59:0/3890148851 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f78641491f0 0x7f7864149670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:30.313 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:30.306+0000 7f78637fe700 1 -- 172.21.15.59:0/3890148851 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f784c004060 msgr2=0x7f786413f330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:30.313 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:30.306+0000 7f78637fe700 1 --2- 172.21.15.59:0/3890148851 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f784c004060 0x7f786413f330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:30.313 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:30.306+0000 7f78637fe700 1 -- 172.21.15.59:0/3890148851 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f786413e6e0 con 0x7f7864151730 2023-11-24T07:59:30.313 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:30.306+0000 7f78637fe700 1 --2- 172.21.15.59:0/3890148851 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7864151730 0x7f7864151b70 secure :-1 s=READY pgs=908 cs=0 l=1 rev1=1 crypto rx=0x7f786406bfb0 tx=0x7f785c00edc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:59:30.313 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:30.306+0000 7f784bfff700 1 -- 172.21.15.59:0/3890148851 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f785c003c10 con 0x7f7864151730 2023-11-24T07:59:30.314 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:30.306+0000 7f784bfff700 1 -- 172.21.15.59:0/3890148851 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f785c003d90 con 0x7f7864151730 2023-11-24T07:59:30.314 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:30.307+0000 7f784bfff700 1 -- 172.21.15.59:0/3890148851 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f785c005310 con 0x7f7864151730 2023-11-24T07:59:30.314 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:30.307+0000 7f7869996700 1 -- 172.21.15.59:0/3890148851 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f78641538b0 con 0x7f7864151730 2023-11-24T07:59:30.314 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:30.307+0000 7f7869996700 1 -- 172.21.15.59:0/3890148851 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7864153e70 con 0x7f7864151730 2023-11-24T07:59:30.314 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:30.308+0000 7f7869996700 1 -- 172.21.15.59:0/3890148851 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7864075480 con 0x7f7864151730 2023-11-24T07:59:30.314 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:30.308+0000 7f784bfff700 1 -- 172.21.15.59:0/3890148851 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 10) v1 ==== 89588+0+0 (secure 0 0 0) 0x7f785c005470 con 0x7f7864151730 2023-11-24T07:59:30.314 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:30.308+0000 7f784bfff700 1 --2- 172.21.15.59:0/3890148851 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f785006b2c0 0x7f785006d780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:30.314 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:30.309+0000 7f784bfff700 1 -- 172.21.15.59:0/3890148851 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f785c089bd0 con 0x7f7864151730 2023-11-24T07:59:30.314 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:30.309+0000 7f7862ffd700 1 --2- 172.21.15.59:0/3890148851 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f785006b2c0 0x7f785006d780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:30.314 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:30.309+0000 7f7862ffd700 1 --2- 172.21.15.59:0/3890148851 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f785006b2c0 0x7f785006d780 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f7854009a00 tx=0x7f7854008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:59:30.315 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:30.313+0000 7f784bfff700 1 -- 172.21.15.59:0/3890148851 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f785c054730 con 0x7f7864151730 2023-11-24T07:59:30.558 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:30.556+0000 7f7869996700 1 -- 172.21.15.59:0/3890148851 --> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot info", "vol_name": "cephfs", "sub_name": "subvol8253", "snap_name": "_ancestral_snap_1_1099511627777", "group_name": "subvol_grp1853", "target": ["mon-mgr", ""]}) v1 -- 0x7f78640b4210 con 0x7f785006b2c0 2023-11-24T07:59:30.565 INFO:tasks.ceph.mgr.x.smithi076.stderr:2023-11-24T07:59:30.565+0000 7ff196724700 -1 mgr.server reply reply (22) Invalid argument snapshot name '_ancestral_snap_1_1099511627777' is invalid 2023-11-24T07:59:30.566 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:30.564+0000 7f784bfff700 1 -- 172.21.15.59:0/3890148851 <== mgr.4100 v2:172.21.15.76:6832/4264046536 1 ==== mgr_command_reply(tid 0: -22 snapshot name '_ancestral_snap_1_1099511627777' is invalid) v1 ==== 66+0+0 (secure 0 0 0) 0x7f78640b4210 con 0x7f785006b2c0 2023-11-24T07:59:30.567 INFO:teuthology.orchestra.run.smithi059.stderr:Error EINVAL: snapshot name '_ancestral_snap_1_1099511627777' is invalid 2023-11-24T07:59:30.571 DEBUG:teuthology.orchestra.run:got remote process result: 22 2023-11-24T07:59:30.571 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T07:59:30.572 DEBUG:teuthology.orchestra.run.smithi059:> (cd /home/ubuntu/cephtest/mnt.0 && exec sudo rmdir ./volumes/subvol_grp1853/.snap/ancestral_snap_1) 2023-11-24T07:59:30.607 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume rm cephfs subvol8253 --group_name subvol_grp1853 2023-11-24T07:59:30.779 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:30.777+0000 7ff54378e700 1 Processor -- start 2023-11-24T07:59:30.779 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:30.777+0000 7ff54378e700 1 -- start start 2023-11-24T07:59:30.782 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:30.778+0000 7ff54378e700 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7ff53c0cecc0 0x7ff53c0cfd60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:30.783 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:30.778+0000 7ff54378e700 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff53c06f110 con 0x7ff53c0b11d0 2023-11-24T07:59:30.783 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:30.778+0000 7ff54378e700 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff53c161a10 con 0x7ff53c06d1f0 2023-11-24T07:59:30.783 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:30.778+0000 7ff54378e700 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff53c0d1010 con 0x7ff53c0cecc0 2023-11-24T07:59:30.783 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:30.778+0000 7ff54152a700 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7ff53c0b11d0 0x7ff53c06fcb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:58626/0 (socket says 172.21.15.59:58626) 2023-11-24T07:59:30.783 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:30.778+0000 7ff54152a700 1 -- 172.21.15.59:0/910104658 learned_addr learned my addr 172.21.15.59:0/910104658 (peer_addr_for_me v1:172.21.15.59:0/0) 2023-11-24T07:59:30.783 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:30.779+0000 7ff541d2b700 1 --2- 172.21.15.59:0/910104658 >> v2:172.21.15.76:3300/0 conn(0x7ff53c0cecc0 0x7ff53c0cfd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:30.783 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:30.779+0000 7ff533fff700 1 -- 172.21.15.59:0/910104658 <== mon.0 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 895680324 0 0) 0x7ff53c06f110 con 0x7ff53c0b11d0 2023-11-24T07:59:30.783 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:30.779+0000 7ff533fff700 1 -- 172.21.15.59:0/910104658 --> v1:172.21.15.59:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff5280035b0 con 0x7ff53c0b11d0 2023-11-24T07:59:30.783 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:30.779+0000 7ff533fff700 1 -- 172.21.15.59:0/910104658 <== mon.0 v1:172.21.15.59:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2842417055 0 0) 0x7ff5280035b0 con 0x7ff53c0b11d0 2023-11-24T07:59:30.783 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:30.779+0000 7ff533fff700 1 -- 172.21.15.59:0/910104658 >> v2:172.21.15.76:3300/0 conn(0x7ff53c0cecc0 msgr2=0x7ff53c0cfd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T07:59:30.783 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:30.779+0000 7ff533fff700 1 --2- 172.21.15.59:0/910104658 >> v2:172.21.15.76:3300/0 conn(0x7ff53c0cecc0 0x7ff53c0cfd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:30.784 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:30.779+0000 7ff533fff700 1 -- 172.21.15.59:0/910104658 >> v1:172.21.15.76:6789/0 conn(0x7ff53c06d1f0 legacy=0x7ff53c0ce5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:30.784 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:30.779+0000 7ff533fff700 1 -- 172.21.15.59:0/910104658 --> v1:172.21.15.59:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff53c0d0400 con 0x7ff53c0b11d0 2023-11-24T07:59:30.784 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:30.779+0000 7ff533fff700 1 -- 172.21.15.59:0/910104658 <== mon.0 v1:172.21.15.59:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1312333566 0 0) 0x7ff52c001670 con 0x7ff53c0b11d0 2023-11-24T07:59:30.784 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:30.779+0000 7ff533fff700 1 -- 172.21.15.59:0/910104658 >> v1:172.21.15.59:6789/0 conn(0x7ff53c0b11d0 legacy=0x7ff53c06fcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:30.784 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:30.779+0000 7ff533fff700 1 --2- 172.21.15.59:0/910104658 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff528003a00 0x7ff528003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:30.784 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:30.779+0000 7ff533fff700 1 --2- 172.21.15.59:0/910104658 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7ff528004850 0x7ff528004cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:30.784 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:30.779+0000 7ff533fff700 1 --2- 172.21.15.59:0/910104658 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff528005210 0x7ff52800ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:30.784 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:30.779+0000 7ff533fff700 1 -- 172.21.15.59:0/910104658 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff53c0d1010 con 0x7ff528003a00 2023-11-24T07:59:30.784 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:30.779+0000 7ff533fff700 1 -- 172.21.15.59:0/910104658 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff52800c1b0 con 0x7ff528005210 2023-11-24T07:59:30.784 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:30.779+0000 7ff533fff700 1 -- 172.21.15.59:0/910104658 --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff52800c370 con 0x7ff528004850 2023-11-24T07:59:30.784 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:30.779+0000 7ff54152a700 1 --2- 172.21.15.59:0/910104658 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff528003a00 0x7ff528003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:30.784 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:30.779+0000 7ff541d2b700 1 --2- 172.21.15.59:0/910104658 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff528005210 0x7ff52800ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:30.785 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:30.779+0000 7ff540d29700 1 --2- 172.21.15.59:0/910104658 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7ff528004850 0x7ff528004cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:30.785 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:30.779+0000 7ff54152a700 1 -- 172.21.15.59:0/910104658 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7ff528004850 msgr2=0x7ff528004cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T07:59:30.785 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:30.779+0000 7ff54152a700 1 --2- 172.21.15.59:0/910104658 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7ff528004850 0x7ff528004cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:30.785 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:30.779+0000 7ff54152a700 1 -- 172.21.15.59:0/910104658 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff528005210 msgr2=0x7ff52800ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T07:59:30.785 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:30.779+0000 7ff54152a700 1 --2- 172.21.15.59:0/910104658 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff528005210 0x7ff52800ba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:30.785 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:30.779+0000 7ff541d2b700 1 -- 172.21.15.59:0/910104658 reap_dead start 2023-11-24T07:59:30.785 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:30.779+0000 7ff54152a700 1 -- 172.21.15.59:0/910104658 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff5280035b0 con 0x7ff528003a00 2023-11-24T07:59:30.785 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:30.780+0000 7ff54152a700 1 --2- 172.21.15.59:0/910104658 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff528003a00 0x7ff528003e20 secure :-1 s=READY pgs=910 cs=0 l=1 rev1=1 crypto rx=0x7ff52c00c240 tx=0x7ff528005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cf524ed00c229d46 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:59:30.785 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:30.780+0000 7ff533fff700 1 -- 172.21.15.59:0/910104658 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff52c007c10 con 0x7ff528003a00 2023-11-24T07:59:30.785 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:30.780+0000 7ff533fff700 1 -- 172.21.15.59:0/910104658 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff52c00cd60 con 0x7ff528003a00 2023-11-24T07:59:30.785 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:30.780+0000 7ff533fff700 1 -- 172.21.15.59:0/910104658 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff52c005ba0 con 0x7ff528003a00 2023-11-24T07:59:30.785 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:30.780+0000 7ff54378e700 1 -- 172.21.15.59:0/910104658 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff528003a00 msgr2=0x7ff528003e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:30.785 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:30.780+0000 7ff54378e700 1 --2- 172.21.15.59:0/910104658 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff528003a00 0x7ff528003e20 secure :-1 s=READY pgs=910 cs=0 l=1 rev1=1 crypto rx=0x7ff52c00c240 tx=0x7ff528005880 comp rx=0 tx=0).stop 2023-11-24T07:59:30.786 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:30.780+0000 7ff54378e700 1 -- 172.21.15.59:0/910104658 shutdown_connections 2023-11-24T07:59:30.786 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:30.780+0000 7ff54378e700 1 --2- 172.21.15.59:0/910104658 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff528003a00 0x7ff528003e20 unknown :-1 s=CLOSED pgs=910 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:30.786 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:30.780+0000 7ff54378e700 1 -- 172.21.15.59:0/910104658 >> 172.21.15.59:0/910104658 conn(0x7ff53c0b3700 msgr2=0x7ff53c0b3b10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:59:30.786 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:30.780+0000 7ff54378e700 1 -- 172.21.15.59:0/910104658 shutdown_connections 2023-11-24T07:59:30.786 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:30.780+0000 7ff54378e700 1 -- 172.21.15.59:0/910104658 wait complete. 2023-11-24T07:59:30.789 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:30.782+0000 7ff54378e700 1 Processor -- start 2023-11-24T07:59:30.789 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:30.782+0000 7ff54378e700 1 -- start start 2023-11-24T07:59:30.789 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:30.783+0000 7ff54378e700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff528003a00 0x7ff53c155fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:30.789 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:30.783+0000 7ff54378e700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff53c15f730 0x7ff53c0c90e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:30.789 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:30.783+0000 7ff54378e700 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7ff53c0b4910 0x7ff53c0b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:30.789 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:30.783+0000 7ff54378e700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff53c1564f0 con 0x7ff53c15f730 2023-11-24T07:59:30.789 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:30.783+0000 7ff54378e700 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff53c0c99a0 con 0x7ff528003a00 2023-11-24T07:59:30.789 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:30.783+0000 7ff54378e700 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff53c0c9c80 con 0x7ff53c0b4910 2023-11-24T07:59:30.789 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:30.783+0000 7ff540d29700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff53c15f730 0x7ff53c0c90e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:30.789 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:30.783+0000 7ff540d29700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff53c15f730 0x7ff53c0c90e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:55166/0 (socket says 172.21.15.59:55166) 2023-11-24T07:59:30.789 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:30.783+0000 7ff540d29700 1 -- 172.21.15.59:0/3304639388 learned_addr learned my addr 172.21.15.59:0/3304639388 (peer_addr_for_me v2:172.21.15.59:0/0) 2023-11-24T07:59:30.790 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:30.783+0000 7ff54152a700 1 --2- 172.21.15.59:0/3304639388 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff528003a00 0x7ff53c155fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:30.790 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:30.783+0000 7ff541d2b700 1 --2- 172.21.15.59:0/3304639388 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7ff53c0b4910 0x7ff53c0b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:30.790 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:30.783+0000 7ff540d29700 1 -- 172.21.15.59:0/3304639388 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7ff53c0b4910 msgr2=0x7ff53c0b4d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:30.790 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:30.783+0000 7ff540d29700 1 --2- 172.21.15.59:0/3304639388 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7ff53c0b4910 0x7ff53c0b4d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:30.790 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:30.783+0000 7ff540d29700 1 -- 172.21.15.59:0/3304639388 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff528003a00 msgr2=0x7ff53c155fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:30.790 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:30.783+0000 7ff540d29700 1 --2- 172.21.15.59:0/3304639388 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff528003a00 0x7ff53c155fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:30.790 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:30.783+0000 7ff540d29700 1 -- 172.21.15.59:0/3304639388 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff53c0c9f60 con 0x7ff53c15f730 2023-11-24T07:59:30.790 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:30.783+0000 7ff54152a700 1 --2- 172.21.15.59:0/3304639388 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff528003a00 0x7ff53c155fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T07:59:30.790 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:30.784+0000 7ff540d29700 1 --2- 172.21.15.59:0/3304639388 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff53c15f730 0x7ff53c0c90e0 secure :-1 s=READY pgs=911 cs=0 l=1 rev1=1 crypto rx=0x7ff53800f610 tx=0x7ff53800f9d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:59:30.790 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:30.784+0000 7ff531ffb700 1 -- 172.21.15.59:0/3304639388 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5380052e0 con 0x7ff53c15f730 2023-11-24T07:59:30.791 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:30.784+0000 7ff531ffb700 1 -- 172.21.15.59:0/3304639388 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff538005440 con 0x7ff53c15f730 2023-11-24T07:59:30.791 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:30.784+0000 7ff54378e700 1 -- 172.21.15.59:0/3304639388 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff53c0ca240 con 0x7ff53c15f730 2023-11-24T07:59:30.791 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:30.784+0000 7ff531ffb700 1 -- 172.21.15.59:0/3304639388 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5380038a0 con 0x7ff53c15f730 2023-11-24T07:59:30.791 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:30.784+0000 7ff54378e700 1 -- 172.21.15.59:0/3304639388 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff53c0c9620 con 0x7ff53c15f730 2023-11-24T07:59:30.791 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:30.785+0000 7ff54378e700 1 -- 172.21.15.59:0/3304639388 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff53c0ca890 con 0x7ff53c15f730 2023-11-24T07:59:30.791 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:30.785+0000 7ff531ffb700 1 -- 172.21.15.59:0/3304639388 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 10) v1 ==== 89588+0+0 (secure 0 0 0) 0x7ff538003a00 con 0x7ff53c15f730 2023-11-24T07:59:30.792 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:30.786+0000 7ff531ffb700 1 --2- 172.21.15.59:0/3304639388 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7ff52006b2c0 0x7ff52006d780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:30.792 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:30.786+0000 7ff531ffb700 1 -- 172.21.15.59:0/3304639388 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 5709+0+0 (secure 0 0 0) 0x7ff538089950 con 0x7ff53c15f730 2023-11-24T07:59:30.792 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:30.786+0000 7ff54152a700 1 --2- 172.21.15.59:0/3304639388 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7ff52006b2c0 0x7ff52006d780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:30.792 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:30.787+0000 7ff54152a700 1 --2- 172.21.15.59:0/3304639388 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7ff52006b2c0 0x7ff52006d780 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7ff52c015040 tx=0x7ff52c007000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:59:30.793 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:30.791+0000 7ff531ffb700 1 -- 172.21.15.59:0/3304639388 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7ff5380543e0 con 0x7ff53c15f730 2023-11-24T07:59:31.024 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:31.023+0000 7ff54378e700 1 -- 172.21.15.59:0/3304639388 --> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] -- mgr_command(tid 0: {"prefix": "fs subvolume rm", "vol_name": "cephfs", "sub_name": "subvol8253", "group_name": "subvol_grp1853", "target": ["mon-mgr", ""]}) v1 -- 0x7ff53c0d09c0 con 0x7ff52006b2c0 2023-11-24T07:59:31.042 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:31.040+0000 7ff531ffb700 1 -- 172.21.15.59:0/3304639388 <== mgr.4100 v2:172.21.15.76:6832/4264046536 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7ff52c007b30 con 0x7ff52006b2c0 2023-11-24T07:59:31.044 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:31.042+0000 7ff54378e700 1 -- 172.21.15.59:0/3304639388 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7ff52006b2c0 msgr2=0x7ff52006d780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:31.044 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:31.042+0000 7ff54378e700 1 --2- 172.21.15.59:0/3304639388 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7ff52006b2c0 0x7ff52006d780 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7ff52c015040 tx=0x7ff52c007000 comp rx=0 tx=0).stop 2023-11-24T07:59:31.044 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:31.042+0000 7ff54378e700 1 -- 172.21.15.59:0/3304639388 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff53c15f730 msgr2=0x7ff53c0c90e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:31.044 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:31.042+0000 7ff54378e700 1 --2- 172.21.15.59:0/3304639388 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff53c15f730 0x7ff53c0c90e0 secure :-1 s=READY pgs=911 cs=0 l=1 rev1=1 crypto rx=0x7ff53800f610 tx=0x7ff53800f9d0 comp rx=0 tx=0).stop 2023-11-24T07:59:31.045 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:31.042+0000 7ff54378e700 1 -- 172.21.15.59:0/3304639388 shutdown_connections 2023-11-24T07:59:31.045 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:31.042+0000 7ff54378e700 1 --2- 172.21.15.59:0/3304639388 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7ff52006b2c0 0x7ff52006d780 unknown :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:31.045 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:31.042+0000 7ff54378e700 1 --2- 172.21.15.59:0/3304639388 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7ff53c0b4910 0x7ff53c0b4d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:31.045 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:31.042+0000 7ff54378e700 1 --2- 172.21.15.59:0/3304639388 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff528003a00 0x7ff53c155fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:31.045 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:31.042+0000 7ff54378e700 1 --2- 172.21.15.59:0/3304639388 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff53c15f730 0x7ff53c0c90e0 unknown :-1 s=CLOSED pgs=911 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:31.045 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:31.042+0000 7ff54378e700 1 -- 172.21.15.59:0/3304639388 >> 172.21.15.59:0/3304639388 conn(0x7ff53c0b3700 msgr2=0x7ff53c06d6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:59:31.045 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:31.042+0000 7ff54378e700 1 -- 172.21.15.59:0/3304639388 shutdown_connections 2023-11-24T07:59:31.045 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:31.042+0000 7ff54378e700 1 -- 172.21.15.59:0/3304639388 wait complete. 2023-11-24T07:59:31.047 INFO:tasks.ceph.mgr.x.smithi076.stderr:2023-11-24T07:59:31.047+0000 7ff19972a700 -1 client.0 error registering admin socket command: (17) File exists 2023-11-24T07:59:31.047 INFO:tasks.ceph.mgr.x.smithi076.stderr:2023-11-24T07:59:31.047+0000 7ff19972a700 -1 client.0 error registering admin socket command: (17) File exists 2023-11-24T07:59:31.047 INFO:tasks.ceph.mgr.x.smithi076.stderr:2023-11-24T07:59:31.047+0000 7ff19972a700 -1 client.0 error registering admin socket command: (17) File exists 2023-11-24T07:59:31.047 INFO:tasks.ceph.mgr.x.smithi076.stderr:2023-11-24T07:59:31.047+0000 7ff19972a700 -1 client.0 error registering admin socket command: (17) File exists 2023-11-24T07:59:31.048 INFO:tasks.ceph.mgr.x.smithi076.stderr:2023-11-24T07:59:31.047+0000 7ff19972a700 -1 client.0 error registering admin socket command: (17) File exists 2023-11-24T07:59:31.057 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T07:59:31.057 DEBUG:teuthology.orchestra.run.smithi059:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'stat -c %h /home/ubuntu/cephtest/mnt.0/./volumes/_deleting') 2023-11-24T07:59:31.063 INFO:tasks.ceph.mgr.x.smithi076.stderr:2023-11-24T07:59:31.063+0000 7ff198f29700 -1 client.0 error registering admin socket command: (17) File exists 2023-11-24T07:59:31.064 INFO:tasks.ceph.mgr.x.smithi076.stderr:2023-11-24T07:59:31.063+0000 7ff198f29700 -1 client.0 error registering admin socket command: (17) File exists 2023-11-24T07:59:31.064 INFO:tasks.ceph.mgr.x.smithi076.stderr:2023-11-24T07:59:31.063+0000 7ff198f29700 -1 client.0 error registering admin socket command: (17) File exists 2023-11-24T07:59:31.064 INFO:tasks.ceph.mgr.x.smithi076.stderr:2023-11-24T07:59:31.063+0000 7ff198f29700 -1 client.0 error registering admin socket command: (17) File exists 2023-11-24T07:59:31.064 INFO:tasks.ceph.mgr.x.smithi076.stderr:2023-11-24T07:59:31.063+0000 7ff198f29700 -1 client.0 error registering admin socket command: (17) File exists 2023-11-24T07:59:31.094 INFO:teuthology.orchestra.run.smithi059.stdout:2 2023-11-24T07:59:31.095 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolumegroup rm cephfs subvol_grp1853 2023-11-24T07:59:31.262 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:31.260+0000 7f5a7d33b700 1 Processor -- start 2023-11-24T07:59:31.262 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:31.260+0000 7f5a7d33b700 1 -- start start 2023-11-24T07:59:31.264 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:31.261+0000 7f5a7d33b700 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f5a78143ae0 0x7f5a780b51b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:31.264 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:31.261+0000 7f5a7d33b700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f5a781440f0 0x7f5a78071bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:31.265 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:31.261+0000 7f5a7d33b700 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5a78143ec0 con 0x7f5a780720f0 2023-11-24T07:59:31.265 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:31.261+0000 7f5a7d33b700 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5a780b40a0 con 0x7f5a781440f0 2023-11-24T07:59:31.265 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:31.261+0000 7f5a7d33b700 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5a78072580 con 0x7f5a78143ae0 2023-11-24T07:59:31.265 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:31.261+0000 7f5a767fc700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f5a781440f0 0x7f5a78071bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:31.265 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:31.261+0000 7f5a777fe700 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7f5a780720f0 0x7f5a7815e8b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:58650/0 (socket says 172.21.15.59:58650) 2023-11-24T07:59:31.265 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:31.261+0000 7f5a777fe700 1 -- 172.21.15.59:0/1020800714 learned_addr learned my addr 172.21.15.59:0/1020800714 (peer_addr_for_me v1:172.21.15.59:0/0) 2023-11-24T07:59:31.265 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:31.262+0000 7f5a76ffd700 1 --2- 172.21.15.59:0/1020800714 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f5a78143ae0 0x7f5a780b51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:31.266 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:31.262+0000 7f5a767fc700 1 -- 172.21.15.59:0/1020800714 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f5a78143ae0 msgr2=0x7f5a780b51b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T07:59:31.266 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:31.262+0000 7f5a767fc700 1 --2- 172.21.15.59:0/1020800714 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f5a78143ae0 0x7f5a780b51b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:31.266 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:31.262+0000 7f5a767fc700 1 -- 172.21.15.59:0/1020800714 >> v1:172.21.15.59:6789/0 conn(0x7f5a780720f0 legacy=0x7f5a7815e8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:31.266 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:31.262+0000 7f5a767fc700 1 -- 172.21.15.59:0/1020800714 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5a780b3ae0 con 0x7f5a781440f0 2023-11-24T07:59:31.266 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:31.262+0000 7f5a75ffb700 1 -- 172.21.15.59:0/1020800714 <== mon.0 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3593978453 0 0) 0x7f5a78143ec0 con 0x7f5a780720f0 2023-11-24T07:59:31.266 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:31.262+0000 7f5a767fc700 1 --2- 172.21.15.59:0/1020800714 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5a781440f0 0x7f5a78071bb0 secure :-1 s=READY pgs=914 cs=0 l=1 rev1=1 crypto rx=0x7f5a68009840 tx=0x7f5a68009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6e8aa92068a23e87 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:59:31.267 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:31.262+0000 7f5a75ffb700 1 -- 172.21.15.59:0/1020800714 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a68011830 con 0x7f5a781440f0 2023-11-24T07:59:31.267 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:31.262+0000 7f5a75ffb700 1 -- 172.21.15.59:0/1020800714 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5a68011990 con 0x7f5a781440f0 2023-11-24T07:59:31.267 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:31.262+0000 7f5a75ffb700 1 -- 172.21.15.59:0/1020800714 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a68011d20 con 0x7f5a781440f0 2023-11-24T07:59:31.267 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:31.263+0000 7f5a7d33b700 1 -- 172.21.15.59:0/1020800714 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5a781440f0 msgr2=0x7f5a78071bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:31.267 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:31.263+0000 7f5a7d33b700 1 --2- 172.21.15.59:0/1020800714 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5a781440f0 0x7f5a78071bb0 secure :-1 s=READY pgs=914 cs=0 l=1 rev1=1 crypto rx=0x7f5a68009840 tx=0x7f5a68009b50 comp rx=0 tx=0).stop 2023-11-24T07:59:31.267 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:31.263+0000 7f5a7d33b700 1 -- 172.21.15.59:0/1020800714 shutdown_connections 2023-11-24T07:59:31.268 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:31.263+0000 7f5a7d33b700 1 --2- 172.21.15.59:0/1020800714 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f5a78143ae0 0x7f5a780b51b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:31.268 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:31.263+0000 7f5a7d33b700 1 --2- 172.21.15.59:0/1020800714 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5a781440f0 0x7f5a78071bb0 unknown :-1 s=CLOSED pgs=914 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:31.268 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:31.263+0000 7f5a7d33b700 1 -- 172.21.15.59:0/1020800714 >> 172.21.15.59:0/1020800714 conn(0x7f5a780b2740 msgr2=0x7f5a780b2b20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:59:31.268 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:31.263+0000 7f5a7d33b700 1 -- 172.21.15.59:0/1020800714 shutdown_connections 2023-11-24T07:59:31.268 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:31.263+0000 7f5a7d33b700 1 -- 172.21.15.59:0/1020800714 wait complete. 2023-11-24T07:59:31.271 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:31.264+0000 7f5a7d33b700 1 Processor -- start 2023-11-24T07:59:31.271 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:31.264+0000 7f5a7d33b700 1 -- start start 2023-11-24T07:59:31.272 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:31.265+0000 7f5a7d33b700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5a780720f0 0x7f5a78142140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:31.272 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:31.265+0000 7f5a7d33b700 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f5a78143ae0 0x7f5a7806afd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:31.272 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:31.265+0000 7f5a7d33b700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5a781440f0 0x7f5a7806b510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:31.272 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:31.265+0000 7f5a7d33b700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a7815d2a0 con 0x7f5a781440f0 2023-11-24T07:59:31.272 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:31.265+0000 7f5a7d33b700 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a7815ca00 con 0x7f5a780720f0 2023-11-24T07:59:31.273 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:31.265+0000 7f5a7d33b700 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5a7815cce0 con 0x7f5a78143ae0 2023-11-24T07:59:31.273 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:31.265+0000 7f5a777fe700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5a781440f0 0x7f5a7806b510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:31.273 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:31.265+0000 7f5a777fe700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5a781440f0 0x7f5a7806b510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:55182/0 (socket says 172.21.15.59:55182) 2023-11-24T07:59:31.273 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:31.265+0000 7f5a777fe700 1 -- 172.21.15.59:0/902320593 learned_addr learned my addr 172.21.15.59:0/902320593 (peer_addr_for_me v2:172.21.15.59:0/0) 2023-11-24T07:59:31.273 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:31.265+0000 7f5a767fc700 1 --2- 172.21.15.59:0/902320593 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f5a78143ae0 0x7f5a7806afd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:31.273 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:31.265+0000 7f5a76ffd700 1 --2- 172.21.15.59:0/902320593 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5a780720f0 0x7f5a78142140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:31.273 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:31.265+0000 7f5a777fe700 1 -- 172.21.15.59:0/902320593 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f5a78143ae0 msgr2=0x7f5a7806afd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:31.273 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:31.265+0000 7f5a777fe700 1 --2- 172.21.15.59:0/902320593 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f5a78143ae0 0x7f5a7806afd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:31.274 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:31.265+0000 7f5a777fe700 1 -- 172.21.15.59:0/902320593 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5a780720f0 msgr2=0x7f5a78142140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:31.274 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:31.265+0000 7f5a777fe700 1 --2- 172.21.15.59:0/902320593 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5a780720f0 0x7f5a78142140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:31.274 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:31.265+0000 7f5a777fe700 1 -- 172.21.15.59:0/902320593 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5a680094a0 con 0x7f5a781440f0 2023-11-24T07:59:31.274 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:31.265+0000 7f5a76ffd700 1 --2- 172.21.15.59:0/902320593 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5a780720f0 0x7f5a78142140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-24T07:59:31.274 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:31.265+0000 7f5a777fe700 1 --2- 172.21.15.59:0/902320593 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5a781440f0 0x7f5a7806b510 secure :-1 s=READY pgs=915 cs=0 l=1 rev1=1 crypto rx=0x7f5a7000cb60 tx=0x7f5a7000ce70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:59:31.274 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:31.266+0000 7f5a5f7fe700 1 -- 172.21.15.59:0/902320593 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a7000aaa0 con 0x7f5a781440f0 2023-11-24T07:59:31.274 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:31.266+0000 7f5a5f7fe700 1 -- 172.21.15.59:0/902320593 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5a7000ac00 con 0x7f5a781440f0 2023-11-24T07:59:31.274 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:31.266+0000 7f5a7d33b700 1 -- 172.21.15.59:0/902320593 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5a7815cfc0 con 0x7f5a781440f0 2023-11-24T07:59:31.275 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:31.266+0000 7f5a5f7fe700 1 -- 172.21.15.59:0/902320593 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a700070d0 con 0x7f5a781440f0 2023-11-24T07:59:31.275 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:31.266+0000 7f5a7d33b700 1 -- 172.21.15.59:0/902320593 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5a78142680 con 0x7f5a781440f0 2023-11-24T07:59:31.275 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:31.267+0000 7f5a7d33b700 1 -- 172.21.15.59:0/902320593 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5a781f2d70 con 0x7f5a781440f0 2023-11-24T07:59:31.275 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:31.267+0000 7f5a5f7fe700 1 -- 172.21.15.59:0/902320593 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 10) v1 ==== 89588+0+0 (secure 0 0 0) 0x7f5a70007230 con 0x7f5a781440f0 2023-11-24T07:59:31.275 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:31.268+0000 7f5a5f7fe700 1 --2- 172.21.15.59:0/902320593 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f5a6406b2f0 0x7f5a6406d7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:31.275 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:31.268+0000 7f5a5f7fe700 1 -- 172.21.15.59:0/902320593 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f5a70089a50 con 0x7f5a781440f0 2023-11-24T07:59:31.275 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:31.268+0000 7f5a76ffd700 1 --2- 172.21.15.59:0/902320593 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f5a6406b2f0 0x7f5a6406d7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:31.275 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:31.268+0000 7f5a76ffd700 1 --2- 172.21.15.59:0/902320593 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f5a6406b2f0 0x7f5a6406d7b0 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f5a6c00aa50 tx=0x7f5a6c008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:59:31.276 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:31.272+0000 7f5a5f7fe700 1 -- 172.21.15.59:0/902320593 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f5a700545b0 con 0x7f5a781440f0 2023-11-24T07:59:31.504 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:31.503+0000 7f5a7d33b700 1 -- 172.21.15.59:0/902320593 --> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] -- mgr_command(tid 0: {"prefix": "fs subvolumegroup rm", "vol_name": "cephfs", "group_name": "subvol_grp1853", "target": ["mon-mgr", ""]}) v1 -- 0x7f5a7806dff0 con 0x7f5a6406b2f0 2023-11-24T07:59:31.510 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:31.509+0000 7f5a5f7fe700 1 -- 172.21.15.59:0/902320593 <== mgr.4100 v2:172.21.15.76:6832/4264046536 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7f5a6c00a720 con 0x7f5a6406b2f0 2023-11-24T07:59:31.513 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:31.511+0000 7f5a7d33b700 1 -- 172.21.15.59:0/902320593 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f5a6406b2f0 msgr2=0x7f5a6406d7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:31.513 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:31.511+0000 7f5a7d33b700 1 --2- 172.21.15.59:0/902320593 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f5a6406b2f0 0x7f5a6406d7b0 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f5a6c00aa50 tx=0x7f5a6c008040 comp rx=0 tx=0).stop 2023-11-24T07:59:31.513 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:31.511+0000 7f5a7d33b700 1 -- 172.21.15.59:0/902320593 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5a781440f0 msgr2=0x7f5a7806b510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:31.513 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:31.511+0000 7f5a7d33b700 1 --2- 172.21.15.59:0/902320593 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5a781440f0 0x7f5a7806b510 secure :-1 s=READY pgs=915 cs=0 l=1 rev1=1 crypto rx=0x7f5a7000cb60 tx=0x7f5a7000ce70 comp rx=0 tx=0).stop 2023-11-24T07:59:31.513 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:31.511+0000 7f5a7d33b700 1 -- 172.21.15.59:0/902320593 shutdown_connections 2023-11-24T07:59:31.513 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:31.511+0000 7f5a7d33b700 1 --2- 172.21.15.59:0/902320593 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f5a6406b2f0 0x7f5a6406d7b0 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:31.513 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:31.511+0000 7f5a7d33b700 1 --2- 172.21.15.59:0/902320593 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5a781440f0 0x7f5a7806b510 unknown :-1 s=CLOSED pgs=915 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:31.513 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:31.511+0000 7f5a7d33b700 1 --2- 172.21.15.59:0/902320593 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5a780720f0 0x7f5a78142140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:31.514 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:31.511+0000 7f5a7d33b700 1 --2- 172.21.15.59:0/902320593 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f5a78143ae0 0x7f5a7806afd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:31.514 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:31.511+0000 7f5a7d33b700 1 -- 172.21.15.59:0/902320593 >> 172.21.15.59:0/902320593 conn(0x7f5a780b2740 msgr2=0x7f5a7815e5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:59:31.514 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:31.511+0000 7f5a7d33b700 1 -- 172.21.15.59:0/902320593 shutdown_connections 2023-11-24T07:59:31.514 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:31.511+0000 7f5a7d33b700 1 -- 172.21.15.59:0/902320593 wait complete. 2023-11-24T07:59:31.524 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-11-24T07:59:31.525 DEBUG:teuthology.orchestra.run.smithi059:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-11-24T07:59:31.566 DEBUG:teuthology.orchestra.run.smithi076:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-11-24T07:59:31.613 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-11-24T07:59:31.614 DEBUG:teuthology.orchestra.run.smithi059:> set -ex 2023-11-24T07:59:31.614 DEBUG:teuthology.orchestra.run.smithi059:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-24T07:59:31.631 DEBUG:teuthology.orchestra.run.smithi059:> set -ex 2023-11-24T07:59:31.632 DEBUG:teuthology.orchestra.run.smithi059:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-24T07:59:31.690 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-24T07:59:31.855 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:31.853+0000 7fd54f043700 1 Processor -- start 2023-11-24T07:59:31.855 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:31.853+0000 7fd54f043700 1 -- start start 2023-11-24T07:59:31.857 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:31.854+0000 7fd54f043700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fd5480b57b0 0x7fd54806afd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:31.857 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:31.854+0000 7fd54f043700 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fd54806b510 0x7fd54806e370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:31.857 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:31.854+0000 7fd54f043700 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd54806dbe0 con 0x7fd5481478c0 2023-11-24T07:59:31.857 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:31.854+0000 7fd54f043700 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd54806b9a0 con 0x7fd5480b57b0 2023-11-24T07:59:31.857 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:31.854+0000 7fd54f043700 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd548147ca0 con 0x7fd54806b510 2023-11-24T07:59:31.858 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:31.854+0000 7fd547fff700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fd5480b57b0 0x7fd54806afd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:31.858 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:31.854+0000 7fd547fff700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fd5480b57b0 0x7fd54806afd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:55188/0 (socket says 172.21.15.59:55188) 2023-11-24T07:59:31.858 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:31.854+0000 7fd547fff700 1 -- 172.21.15.59:0/3415105899 learned_addr learned my addr 172.21.15.59:0/3415105899 (peer_addr_for_me v2:172.21.15.59:0/0) 2023-11-24T07:59:31.858 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:31.854+0000 7fd54d5e0700 1 --2- 172.21.15.59:0/3415105899 >> v2:172.21.15.76:3300/0 conn(0x7fd54806b510 0x7fd54806e370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:31.858 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:31.854+0000 7fd547fff700 1 -- 172.21.15.59:0/3415105899 >> v2:172.21.15.76:3300/0 conn(0x7fd54806b510 msgr2=0x7fd54806e370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T07:59:31.858 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:31.854+0000 7fd547fff700 1 --2- 172.21.15.59:0/3415105899 >> v2:172.21.15.76:3300/0 conn(0x7fd54806b510 0x7fd54806e370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:31.859 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:31.854+0000 7fd547fff700 1 -- 172.21.15.59:0/3415105899 >> v1:172.21.15.76:6789/0 conn(0x7fd5481478c0 legacy=0x7fd548161a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T07:59:31.859 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:31.854+0000 7fd547fff700 1 -- 172.21.15.59:0/3415105899 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd5481622e0 con 0x7fd5480b57b0 2023-11-24T07:59:31.859 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:31.854+0000 7fd547fff700 1 --2- 172.21.15.59:0/3415105899 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd5480b57b0 0x7fd54806afd0 secure :-1 s=READY pgs=916 cs=0 l=1 rev1=1 crypto rx=0x7fd54806c040 tx=0x7fd534009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4459d1d8b859beca server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:59:31.859 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:31.855+0000 7fd5477fe700 1 -- 172.21.15.59:0/3415105899 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd534004230 con 0x7fd5480b57b0 2023-11-24T07:59:31.859 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:31.855+0000 7fd5477fe700 1 -- 172.21.15.59:0/3415105899 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd534004390 con 0x7fd5480b57b0 2023-11-24T07:59:31.859 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:31.855+0000 7fd5477fe700 1 -- 172.21.15.59:0/3415105899 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5340036a0 con 0x7fd5480b57b0 2023-11-24T07:59:31.860 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:31.855+0000 7fd54f043700 1 -- 172.21.15.59:0/3415105899 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd5480b57b0 msgr2=0x7fd54806afd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:31.860 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:31.855+0000 7fd54f043700 1 --2- 172.21.15.59:0/3415105899 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd5480b57b0 0x7fd54806afd0 secure :-1 s=READY pgs=916 cs=0 l=1 rev1=1 crypto rx=0x7fd54806c040 tx=0x7fd534009bd0 comp rx=0 tx=0).stop 2023-11-24T07:59:31.860 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:31.855+0000 7fd54f043700 1 -- 172.21.15.59:0/3415105899 shutdown_connections 2023-11-24T07:59:31.860 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:31.855+0000 7fd54f043700 1 --2- 172.21.15.59:0/3415105899 >> v2:172.21.15.76:3300/0 conn(0x7fd54806b510 0x7fd54806e370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:31.860 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:31.855+0000 7fd54f043700 1 --2- 172.21.15.59:0/3415105899 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd5480b57b0 0x7fd54806afd0 unknown :-1 s=CLOSED pgs=916 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:31.860 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:31.855+0000 7fd54f043700 1 -- 172.21.15.59:0/3415105899 >> 172.21.15.59:0/3415105899 conn(0x7fd5480d0aa0 msgr2=0x7fd5480d0eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:59:31.861 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:31.855+0000 7fd54f043700 1 -- 172.21.15.59:0/3415105899 shutdown_connections 2023-11-24T07:59:31.861 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:31.855+0000 7fd54f043700 1 -- 172.21.15.59:0/3415105899 wait complete. 2023-11-24T07:59:31.863 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:31.856+0000 7fd54f043700 1 Processor -- start 2023-11-24T07:59:31.863 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:31.856+0000 7fd54f043700 1 -- start start 2023-11-24T07:59:31.864 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:31.857+0000 7fd54f043700 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fd54806b510 0x7fd5480c6350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:31.864 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:31.857+0000 7fd54f043700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd5480b57b0 0x7fd5480b1540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:31.864 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:31.857+0000 7fd54f043700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd5481478c0 0x7fd548071ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:31.864 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:31.857+0000 7fd54f043700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5480c7640 con 0x7fd5480b57b0 2023-11-24T07:59:31.864 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:31.857+0000 7fd54f043700 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5480b1a80 con 0x7fd5481478c0 2023-11-24T07:59:31.864 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:31.857+0000 7fd54f043700 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd5480c69c0 con 0x7fd54806b510 2023-11-24T07:59:31.865 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:31.857+0000 7fd547fff700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd5480b57b0 0x7fd5480b1540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:31.865 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:31.857+0000 7fd547fff700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd5480b57b0 0x7fd5480b1540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:55194/0 (socket says 172.21.15.59:55194) 2023-11-24T07:59:31.865 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:31.857+0000 7fd547fff700 1 -- 172.21.15.59:0/1835704134 learned_addr learned my addr 172.21.15.59:0/1835704134 (peer_addr_for_me v2:172.21.15.59:0/0) 2023-11-24T07:59:31.865 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:31.858+0000 7fd54d5e0700 1 --2- 172.21.15.59:0/1835704134 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd5481478c0 0x7fd548071ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:31.865 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:31.858+0000 7fd547fff700 1 -- 172.21.15.59:0/1835704134 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fd54806b510 msgr2=0x7fd5480c6350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:31.865 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:31.858+0000 7fd547fff700 1 --2- 172.21.15.59:0/1835704134 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fd54806b510 0x7fd5480c6350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:31.865 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:31.858+0000 7fd547fff700 1 -- 172.21.15.59:0/1835704134 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd5481478c0 msgr2=0x7fd548071ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:31.866 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:31.858+0000 7fd547fff700 1 --2- 172.21.15.59:0/1835704134 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd5481478c0 0x7fd548071ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:31.866 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:31.858+0000 7fd547fff700 1 -- 172.21.15.59:0/1835704134 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd534009550 con 0x7fd5480b57b0 2023-11-24T07:59:31.866 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:31.858+0000 7fd54d5e0700 1 --2- 172.21.15.59:0/1835704134 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd5481478c0 0x7fd548071ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T07:59:31.866 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:31.858+0000 7fd547fff700 1 --2- 172.21.15.59:0/1835704134 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd5480b57b0 0x7fd5480b1540 secure :-1 s=READY pgs=917 cs=0 l=1 rev1=1 crypto rx=0x7fd534009130 tx=0x7fd534004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:59:31.866 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:31.858+0000 7fd5457fa700 1 -- 172.21.15.59:0/1835704134 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd534003dc0 con 0x7fd5480b57b0 2023-11-24T07:59:31.867 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:31.858+0000 7fd5457fa700 1 -- 172.21.15.59:0/1835704134 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd53401f070 con 0x7fd5480b57b0 2023-11-24T07:59:31.867 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:31.858+0000 7fd54f043700 1 -- 172.21.15.59:0/1835704134 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd5480c6ca0 con 0x7fd5480b57b0 2023-11-24T07:59:31.867 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:31.858+0000 7fd5457fa700 1 -- 172.21.15.59:0/1835704134 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd534011870 con 0x7fd5480b57b0 2023-11-24T07:59:31.867 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:31.858+0000 7fd54f043700 1 -- 172.21.15.59:0/1835704134 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd5480c7260 con 0x7fd5480b57b0 2023-11-24T07:59:31.867 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:31.859+0000 7fd54f043700 1 -- 172.21.15.59:0/1835704134 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd548068670 con 0x7fd5480b57b0 2023-11-24T07:59:31.867 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:31.860+0000 7fd5457fa700 1 -- 172.21.15.59:0/1835704134 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 10) v1 ==== 89588+0+0 (secure 0 0 0) 0x7fd534021030 con 0x7fd5480b57b0 2023-11-24T07:59:31.867 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:31.860+0000 7fd5457fa700 1 --2- 172.21.15.59:0/1835704134 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7fd52c06b2c0 0x7fd52c06d780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:31.867 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:31.860+0000 7fd5457fa700 1 -- 172.21.15.59:0/1835704134 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fd53408b2a0 con 0x7fd5480b57b0 2023-11-24T07:59:31.868 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:31.860+0000 7fd54cddf700 1 --2- 172.21.15.59:0/1835704134 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7fd52c06b2c0 0x7fd52c06d780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:31.868 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:31.861+0000 7fd54cddf700 1 --2- 172.21.15.59:0/1835704134 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7fd52c06b2c0 0x7fd52c06d780 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7fd54806b480 tx=0x7fd53c006c30 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:59:31.868 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:31.865+0000 7fd5457fa700 1 -- 172.21.15.59:0/1835704134 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fd534055f10 con 0x7fd5480b57b0 2023-11-24T07:59:32.060 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-24T07:59:32.063 DEBUG:teuthology.orchestra.run.smithi076:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-24T07:59:32.067 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:32.065+0000 7fd54f043700 1 -- 172.21.15.59:0/1835704134 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fd548143af0 con 0x7fd5480b57b0 2023-11-24T07:59:32.068 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:32.067+0000 7fd5457fa700 1 -- 172.21.15.59:0/1835704134 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v74) v1 ==== 74+0+16436 (secure 0 0 0) 0x7fd534059530 con 0x7fd5480b57b0 2023-11-24T07:59:32.068 INFO:teuthology.orchestra.run.smithi059.stdout: 2023-11-24T07:59:32.069 INFO:teuthology.orchestra.run.smithi059.stdout:{"epoch":74,"fsid":"9f0e8741-b8f9-48d5-a2c3-09f45903303d","created":"2023-11-24T07:56:06.552729+0000","modified":"2023-11-24T07:59:15.638357+0000","last_up_change":"2023-11-24T07:56:17.372352+0000","last_in_change":"2023-11-24T07:56:09.073250+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":11,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-24T07:56:18.352823+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_metadata","create_time":"2023-11-24T07:59:11.900882+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"73","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_data","create_time":"2023-11-24T07:59:13.025749+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"74","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"6d8f9e9b-1782-4e41-928b-a39f2a8ebc06","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6800","nonce":3373878772},{"type":"v1","addr":"172.21.15.59:6801","nonce":3373878772}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6802","nonce":3373878772},{"type":"v1","addr":"172.21.15.59:6803","nonce":3373878772}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6806","nonce":3373878772},{"type":"v1","addr":"172.21.15.59:6807","nonce":3373878772}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6804","nonce":3373878772},{"type":"v1","addr":"172.21.15.59:6805","nonce":3373878772}]},"public_addr":"172.21.15.59:6801/3373878772","cluster_addr":"172.21.15.59:6803/3373878772","heartbeat_back_addr":"172.21.15.59:6807/3373878772","heartbeat_front_addr":"172.21.15.59:6805/3373878772","state":["exists","up"]},{"osd":1,"uuid":"e8d882aa-52e1-4c04-9e1e-c44ef9204a5f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6824","nonce":648258563},{"type":"v1","addr":"172.21.15.59:6825","nonce":648258563}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6826","nonce":648258563},{"type":"v1","addr":"172.21.15.59:6827","nonce":648258563}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6830","nonce":648258563},{"type":"v1","addr":"172.21.15.59:6831","nonce":648258563}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6828","nonce":648258563},{"type":"v1","addr":"172.21.15.59:6829","nonce":648258563}]},"public_addr":"172.21.15.59:6825/648258563","cluster_addr":"172.21.15.59:6827/648258563","heartbeat_back_addr":"172.21.15.59:6831/648258563","heartbeat_front_addr":"172.21.15.59:6829/648258563","state":["exists","up"]},{"osd":2,"uuid":"69eebfa0-695b-4b21-90b9-8a3537168a01","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6808","nonce":1840430654},{"type":"v1","addr":"172.21.15.59:6809","nonce":1840430654}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6810","nonce":1840430654},{"type":"v1","addr":"172.21.15.59:6811","nonce":1840430654}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6814","nonce":1840430654},{"type":"v1","addr":"172.21.15.59:6815","nonce":1840430654}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6812","nonce":1840430654},{"type":"v1","addr":"172.21.15.59:6813","nonce":1840430654}]},"public_addr":"172.21.15.59:6809/1840430654","cluster_addr":"172.21.15.59:6811/1840430654","heartbeat_back_addr":"172.21.15.59:6815/1840430654","heartbeat_front_addr":"172.21.15.59:6813/1840430654","state":["exists","up"]},{"osd":3,"uuid":"96fd77fc-f911-46ff-9445-6f0be2daa8f4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6816","nonce":2504025850},{"type":"v1","addr":"172.21.15.59:6817","nonce":2504025850}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6818","nonce":2504025850},{"type":"v1","addr":"172.21.15.59:6819","nonce":2504025850}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6822","nonce":2504025850},{"type":"v1","addr":"172.21.15.59:6823","nonce":2504025850}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6820","nonce":2504025850},{"type":"v1","addr":"172.21.15.59:6821","nonce":2504025850}]},"public_addr":"172.21.15.59:6817/2504025850","cluster_addr":"172.21.15.59:6819/2504025850","heartbeat_back_addr":"172.21.15.59:6823/2504025850","heartbeat_front_addr":"172.21.15.59:6821/2504025850","state":["exists","up"]},{"osd":4,"uuid":"7ac29fca-b5b2-4c95-9d69-2f164cbbc708","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6816","nonce":2382463431},{"type":"v1","addr":"172.21.15.76:6818","nonce":2382463431}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6820","nonce":2382463431},{"type":"v1","addr":"172.21.15.76:6822","nonce":2382463431}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6828","nonce":2382463431},{"type":"v1","addr":"172.21.15.76:6830","nonce":2382463431}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6824","nonce":2382463431},{"type":"v1","addr":"172.21.15.76:6826","nonce":2382463431}]},"public_addr":"172.21.15.76:6818/2382463431","cluster_addr":"172.21.15.76:6822/2382463431","heartbeat_back_addr":"172.21.15.76:6830/2382463431","heartbeat_front_addr":"172.21.15.76:6826/2382463431","state":["exists","up"]},{"osd":5,"uuid":"485bf711-3851-4238-a4f4-51d52e0a9c58","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6800","nonce":329334308},{"type":"v1","addr":"172.21.15.76:6802","nonce":329334308}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6805","nonce":329334308},{"type":"v1","addr":"172.21.15.76:6807","nonce":329334308}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6814","nonce":329334308},{"type":"v1","addr":"172.21.15.76:6815","nonce":329334308}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6810","nonce":329334308},{"type":"v1","addr":"172.21.15.76:6812","nonce":329334308}]},"public_addr":"172.21.15.76:6802/329334308","cluster_addr":"172.21.15.76:6807/329334308","heartbeat_back_addr":"172.21.15.76:6815/329334308","heartbeat_front_addr":"172.21.15.76:6812/329334308","state":["exists","up"]},{"osd":6,"uuid":"72b3f2fc-1e4a-4e32-968d-003647a0286f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6801","nonce":3946579410},{"type":"v1","addr":"172.21.15.76:6803","nonce":3946579410}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6804","nonce":3946579410},{"type":"v1","addr":"172.21.15.76:6806","nonce":3946579410}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6811","nonce":3946579410},{"type":"v1","addr":"172.21.15.76:6813","nonce":3946579410}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6808","nonce":3946579410},{"type":"v1","addr":"172.21.15.76:6809","nonce":3946579410}]},"public_addr":"172.21.15.76:6803/3946579410","cluster_addr":"172.21.15.76:6806/3946579410","heartbeat_back_addr":"172.21.15.76:6813/3946579410","heartbeat_front_addr":"172.21.15.76:6809/3946579410","state":["exists","up"]},{"osd":7,"uuid":"1ddceb90-6c5d-4d1a-ad56-99cb877f8c70","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6817","nonce":841761538},{"type":"v1","addr":"172.21.15.76:6819","nonce":841761538}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6821","nonce":841761538},{"type":"v1","addr":"172.21.15.76:6823","nonce":841761538}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6829","nonce":841761538},{"type":"v1","addr":"172.21.15.76:6831","nonce":841761538}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6825","nonce":841761538},{"type":"v1","addr":"172.21.15.76:6827","nonce":841761538}]},"public_addr":"172.21.15.76:6819/841761538","cluster_addr":"172.21.15.76:6823/841761538","heartbeat_back_addr":"172.21.15.76:6831/841761538","heartbeat_front_addr":"172.21.15.76:6827/841761538","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T07:56:14.294838+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T07:56:14.270779+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T07:56:14.231809+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T07:56:14.260983+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T07:56:14.302997+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T07:56:14.258600+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T07:56:14.251413+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T07:56:14.304727+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-24T07:59:32.071 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:32.069+0000 7fd54f043700 1 -- 172.21.15.59:0/1835704134 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7fd52c06b2c0 msgr2=0x7fd52c06d780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:32.071 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:32.069+0000 7fd54f043700 1 --2- 172.21.15.59:0/1835704134 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7fd52c06b2c0 0x7fd52c06d780 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7fd54806b480 tx=0x7fd53c006c30 comp rx=0 tx=0).stop 2023-11-24T07:59:32.071 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:32.069+0000 7fd54f043700 1 -- 172.21.15.59:0/1835704134 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd5480b57b0 msgr2=0x7fd5480b1540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:32.071 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:32.069+0000 7fd54f043700 1 --2- 172.21.15.59:0/1835704134 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd5480b57b0 0x7fd5480b1540 secure :-1 s=READY pgs=917 cs=0 l=1 rev1=1 crypto rx=0x7fd534009130 tx=0x7fd534004750 comp rx=0 tx=0).stop 2023-11-24T07:59:32.071 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:32.069+0000 7fd54f043700 1 -- 172.21.15.59:0/1835704134 shutdown_connections 2023-11-24T07:59:32.071 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:32.069+0000 7fd54f043700 1 --2- 172.21.15.59:0/1835704134 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7fd52c06b2c0 0x7fd52c06d780 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:32.071 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:32.069+0000 7fd54f043700 1 --2- 172.21.15.59:0/1835704134 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd5481478c0 0x7fd548071ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:32.071 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:32.069+0000 7fd54f043700 1 --2- 172.21.15.59:0/1835704134 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fd54806b510 0x7fd5480c6350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:32.072 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:32.069+0000 7fd54f043700 1 --2- 172.21.15.59:0/1835704134 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd5480b57b0 0x7fd5480b1540 unknown :-1 s=CLOSED pgs=917 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:32.072 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:32.069+0000 7fd54f043700 1 -- 172.21.15.59:0/1835704134 >> 172.21.15.59:0/1835704134 conn(0x7fd5480d0aa0 msgr2=0x7fd5480d1450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:59:32.072 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:32.069+0000 7fd54f043700 1 -- 172.21.15.59:0/1835704134 shutdown_connections 2023-11-24T07:59:32.072 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:32.069+0000 7fd54f043700 1 -- 172.21.15.59:0/1835704134 wait complete. 2023-11-24T07:59:32.084 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-24T07:56:18.352823+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 10, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-24T07:59:11.900882+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '73', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 11, 'pool_name': 'cephfs_data', 'create_time': '2023-11-24T07:59:13.025749+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '74', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-24T07:59:32.084 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-24T07:59:32.277 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:32.276+0000 7fafaf2c5700 1 Processor -- start 2023-11-24T07:59:32.278 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:32.276+0000 7fafaf2c5700 1 -- start start 2023-11-24T07:59:32.279 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:32.277+0000 7fafaf2c5700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fafa815bde0 0x7fafa815c1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:32.279 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:32.277+0000 7fafaf2c5700 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fafa80d0a90 con 0x7fafa80cf2b0 2023-11-24T07:59:32.279 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:32.277+0000 7fafaf2c5700 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fafa806ee10 con 0x7fafa80ce5c0 2023-11-24T07:59:32.279 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:32.277+0000 7fafaf2c5700 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fafa806f0f0 con 0x7fafa815bde0 2023-11-24T07:59:32.279 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:32.277+0000 7fafad061700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fafa815bde0 0x7fafa815c1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:32.280 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:32.277+0000 7fafad061700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fafa815bde0 0x7fafa815c1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:55200/0 (socket says 172.21.15.59:55200) 2023-11-24T07:59:32.280 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:32.277+0000 7fafac860700 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7fafa80cf2b0 0x7fafa80d01b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:58676/0 (socket says 172.21.15.59:58676) 2023-11-24T07:59:32.280 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:32.277+0000 7fafad061700 1 -- 172.21.15.59:0/3625767062 learned_addr learned my addr 172.21.15.59:0/3625767062 (peer_addr_for_me v2:172.21.15.59:0/0) 2023-11-24T07:59:32.280 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:32.277+0000 7fafad061700 1 -- 172.21.15.59:0/3625767062 >> v1:172.21.15.76:6789/0 conn(0x7fafa80ce5c0 legacy=0x7fafa806fbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T07:59:32.280 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:32.277+0000 7fafad061700 1 -- 172.21.15.59:0/3625767062 >> v1:172.21.15.59:6789/0 conn(0x7fafa80cf2b0 legacy=0x7fafa80d01b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:32.280 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:32.277+0000 7faf9ffff700 1 -- 172.21.15.59:0/3625767062 <== mon.0 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 342986367 0 0) 0x7fafa80d0a90 con 0x7fafa80cf2b0 2023-11-24T07:59:32.280 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:32.277+0000 7fafad061700 1 -- 172.21.15.59:0/3625767062 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fafa806f3d0 con 0x7fafa815bde0 2023-11-24T07:59:32.280 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:32.277+0000 7fafad061700 1 --2- 172.21.15.59:0/3625767062 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fafa815bde0 0x7fafa815c1c0 secure :-1 s=READY pgs=919 cs=0 l=1 rev1=1 crypto rx=0x7fafa40098f0 tx=0x7fafa4009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=15da1f03eeec5b39 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:59:32.280 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:32.277+0000 7faf9ffff700 1 -- 172.21.15.59:0/3625767062 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fafa4004230 con 0x7fafa815bde0 2023-11-24T07:59:32.280 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:32.277+0000 7faf9ffff700 1 -- 172.21.15.59:0/3625767062 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fafa4004390 con 0x7fafa815bde0 2023-11-24T07:59:32.281 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:32.278+0000 7faf9ffff700 1 -- 172.21.15.59:0/3625767062 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fafa40036a0 con 0x7fafa815bde0 2023-11-24T07:59:32.283 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:32.278+0000 7fafaf2c5700 1 -- 172.21.15.59:0/3625767062 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fafa815bde0 msgr2=0x7fafa815c1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:32.283 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:32.278+0000 7fafaf2c5700 1 --2- 172.21.15.59:0/3625767062 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fafa815bde0 0x7fafa815c1c0 secure :-1 s=READY pgs=919 cs=0 l=1 rev1=1 crypto rx=0x7fafa40098f0 tx=0x7fafa4009c00 comp rx=0 tx=0).stop 2023-11-24T07:59:32.284 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:32.278+0000 7fafaf2c5700 1 -- 172.21.15.59:0/3625767062 shutdown_connections 2023-11-24T07:59:32.284 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:32.278+0000 7fafaf2c5700 1 --2- 172.21.15.59:0/3625767062 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fafa815bde0 0x7fafa815c1c0 unknown :-1 s=CLOSED pgs=919 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:32.284 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:32.278+0000 7fafaf2c5700 1 -- 172.21.15.59:0/3625767062 >> 172.21.15.59:0/3625767062 conn(0x7fafa80b3720 msgr2=0x7fafa80b3b30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:59:32.284 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:32.278+0000 7fafaf2c5700 1 -- 172.21.15.59:0/3625767062 shutdown_connections 2023-11-24T07:59:32.284 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:32.278+0000 7fafaf2c5700 1 -- 172.21.15.59:0/3625767062 wait complete. 2023-11-24T07:59:32.284 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:32.279+0000 7fafaf2c5700 1 Processor -- start 2023-11-24T07:59:32.284 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:32.279+0000 7fafaf2c5700 1 -- start start 2023-11-24T07:59:32.284 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:32.280+0000 7fafaf2c5700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fafa80ce5c0 0x7fafa80b7090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:32.285 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:32.280+0000 7fafaf2c5700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fafa80cf2b0 0x7fafa8159540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:32.285 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:32.280+0000 7fafaf2c5700 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fafa815bde0 0x7fafa8158c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:32.285 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:32.280+0000 7fafaf2c5700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fafa80b5d50 con 0x7fafa80cf2b0 2023-11-24T07:59:32.285 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:32.280+0000 7fafaf2c5700 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fafa80cbf40 con 0x7fafa80ce5c0 2023-11-24T07:59:32.285 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:32.280+0000 7fafaf2c5700 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fafa80b5ff0 con 0x7fafa815bde0 2023-11-24T07:59:32.285 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:32.280+0000 7fafac860700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fafa80cf2b0 0x7fafa8159540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:32.285 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:32.280+0000 7fafac860700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fafa80cf2b0 0x7fafa8159540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:55208/0 (socket says 172.21.15.59:55208) 2023-11-24T07:59:32.286 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:32.280+0000 7fafac860700 1 -- 172.21.15.59:0/4288447245 learned_addr learned my addr 172.21.15.59:0/4288447245 (peer_addr_for_me v2:172.21.15.59:0/0) 2023-11-24T07:59:32.286 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:32.281+0000 7fafad061700 1 --2- 172.21.15.59:0/4288447245 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fafa80ce5c0 0x7fafa80b7090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:32.286 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:32.281+0000 7fafad862700 1 --2- 172.21.15.59:0/4288447245 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fafa815bde0 0x7fafa8158c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:32.286 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:32.281+0000 7fafac860700 1 -- 172.21.15.59:0/4288447245 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fafa815bde0 msgr2=0x7fafa8158c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:32.286 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:32.281+0000 7fafac860700 1 --2- 172.21.15.59:0/4288447245 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fafa815bde0 0x7fafa8158c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:32.286 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:32.281+0000 7fafac860700 1 -- 172.21.15.59:0/4288447245 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fafa80ce5c0 msgr2=0x7fafa80b7090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:32.286 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:32.281+0000 7fafac860700 1 --2- 172.21.15.59:0/4288447245 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fafa80ce5c0 0x7fafa80b7090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:32.286 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:32.281+0000 7fafac860700 1 -- 172.21.15.59:0/4288447245 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fafa4009550 con 0x7fafa80cf2b0 2023-11-24T07:59:32.286 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:32.281+0000 7fafad862700 1 --2- 172.21.15.59:0/4288447245 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fafa815bde0 0x7fafa8158c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T07:59:32.287 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:32.281+0000 7fafac860700 1 --2- 172.21.15.59:0/4288447245 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fafa80cf2b0 0x7fafa8159540 secure :-1 s=READY pgs=920 cs=0 l=1 rev1=1 crypto rx=0x7faf9800c540 tx=0x7faf9800c900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:59:32.287 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:32.281+0000 7faf9dffb700 1 -- 172.21.15.59:0/4288447245 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf98004c80 con 0x7fafa80cf2b0 2023-11-24T07:59:32.287 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:32.281+0000 7faf9dffb700 1 -- 172.21.15.59:0/4288447245 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faf98004de0 con 0x7fafa80cf2b0 2023-11-24T07:59:32.287 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:32.281+0000 7fafaf2c5700 1 -- 172.21.15.59:0/4288447245 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fafa80b62d0 con 0x7fafa80cf2b0 2023-11-24T07:59:32.287 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:32.281+0000 7faf9dffb700 1 -- 172.21.15.59:0/4288447245 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf98007310 con 0x7fafa80cf2b0 2023-11-24T07:59:32.287 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:32.281+0000 7fafaf2c5700 1 -- 172.21.15.59:0/4288447245 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fafa80b6890 con 0x7fafa80cf2b0 2023-11-24T07:59:32.288 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:32.282+0000 7fafaf2c5700 1 -- 172.21.15.59:0/4288447245 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fafa8075480 con 0x7fafa80cf2b0 2023-11-24T07:59:32.288 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:32.283+0000 7faf9dffb700 1 -- 172.21.15.59:0/4288447245 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 10) v1 ==== 89588+0+0 (secure 0 0 0) 0x7faf98007470 con 0x7fafa80cf2b0 2023-11-24T07:59:32.288 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:32.283+0000 7faf9dffb700 1 --2- 172.21.15.59:0/4288447245 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7faf8c06b2c0 0x7faf8c06d780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:32.288 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:32.283+0000 7faf9dffb700 1 -- 172.21.15.59:0/4288447245 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 5709+0+0 (secure 0 0 0) 0x7faf98089950 con 0x7fafa80cf2b0 2023-11-24T07:59:32.289 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:32.283+0000 7fafad061700 1 --2- 172.21.15.59:0/4288447245 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7faf8c06b2c0 0x7faf8c06d780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:32.289 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:32.284+0000 7fafad061700 1 --2- 172.21.15.59:0/4288447245 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7faf8c06b2c0 0x7faf8c06d780 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7fafa4009130 tx=0x7fafa4008030 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:59:32.289 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:32.288+0000 7faf9dffb700 1 -- 172.21.15.59:0/4288447245 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7faf980545c0 con 0x7fafa80cf2b0 2023-11-24T07:59:32.423 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:32.421+0000 7faf9dffb700 1 -- 172.21.15.59:0/4288447245 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mgrmap(e 11) v1 ==== 89741+0+0 (secure 0 0 0) 0x7faf98054990 con 0x7fafa80cf2b0 2023-11-24T07:59:32.492 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:32.490+0000 7fafaf2c5700 1 -- 172.21.15.59:0/4288447245 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fafa80ce180 con 0x7fafa80cf2b0 2023-11-24T07:59:32.492 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:32.490+0000 7faf9dffb700 1 -- 172.21.15.59:0/4288447245 <== mon.0 v2:172.21.15.59:3300/0 8 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v74) v1 ==== 93+0+10 (secure 0 0 0) 0x7faf98074c20 con 0x7fafa80cf2b0 2023-11-24T07:59:32.492 INFO:teuthology.orchestra.run.smithi059.stdout:pg_num: 1 2023-11-24T07:59:32.495 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:32.493+0000 7fafaf2c5700 1 -- 172.21.15.59:0/4288447245 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7faf8c06b2c0 msgr2=0x7faf8c06d780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:32.495 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:32.493+0000 7fafaf2c5700 1 --2- 172.21.15.59:0/4288447245 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7faf8c06b2c0 0x7faf8c06d780 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7fafa4009130 tx=0x7fafa4008030 comp rx=0 tx=0).stop 2023-11-24T07:59:32.495 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:32.493+0000 7fafaf2c5700 1 -- 172.21.15.59:0/4288447245 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fafa80cf2b0 msgr2=0x7fafa8159540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:32.495 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:32.493+0000 7fafaf2c5700 1 --2- 172.21.15.59:0/4288447245 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fafa80cf2b0 0x7fafa8159540 secure :-1 s=READY pgs=920 cs=0 l=1 rev1=1 crypto rx=0x7faf9800c540 tx=0x7faf9800c900 comp rx=0 tx=0).stop 2023-11-24T07:59:32.495 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:32.493+0000 7fafaf2c5700 1 -- 172.21.15.59:0/4288447245 shutdown_connections 2023-11-24T07:59:32.495 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:32.493+0000 7fafaf2c5700 1 --2- 172.21.15.59:0/4288447245 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7faf8c06b2c0 0x7faf8c06d780 unknown :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:32.495 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:32.493+0000 7fafaf2c5700 1 --2- 172.21.15.59:0/4288447245 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fafa815bde0 0x7fafa8158c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:32.495 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:32.493+0000 7fafaf2c5700 1 --2- 172.21.15.59:0/4288447245 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fafa80ce5c0 0x7fafa80b7090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:32.496 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:32.493+0000 7fafaf2c5700 1 --2- 172.21.15.59:0/4288447245 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fafa80cf2b0 0x7fafa8159540 unknown :-1 s=CLOSED pgs=920 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:32.496 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:32.493+0000 7fafaf2c5700 1 -- 172.21.15.59:0/4288447245 >> 172.21.15.59:0/4288447245 conn(0x7fafa80b3720 msgr2=0x7fafa815a2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:59:32.496 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:32.493+0000 7fafaf2c5700 1 -- 172.21.15.59:0/4288447245 shutdown_connections 2023-11-24T07:59:32.496 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:32.493+0000 7fafaf2c5700 1 -- 172.21.15.59:0/4288447245 wait complete. 2023-11-24T07:59:32.507 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-24T07:59:32.635 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:32.634+0000 7ff6073af700 1 Processor -- start 2023-11-24T07:59:32.636 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:32.634+0000 7ff6073af700 1 -- start start 2023-11-24T07:59:32.637 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:32.635+0000 7ff6073af700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7ff600145100 0x7ff60006e280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:32.637 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:32.635+0000 7ff6073af700 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff600160720 con 0x7ff600144c10 2023-11-24T07:59:32.637 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:32.635+0000 7ff6073af700 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff60006f710 con 0x7ff60006b6e0 2023-11-24T07:59:32.637 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:32.635+0000 7ff6073af700 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff60006f500 con 0x7ff600145100 2023-11-24T07:59:32.637 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:32.635+0000 7ff60594c700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7ff600145100 0x7ff60006e280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:32.637 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:32.635+0000 7ff60594c700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7ff600145100 0x7ff60006e280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:55218/0 (socket says 172.21.15.59:55218) 2023-11-24T07:59:32.638 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:32.635+0000 7ff60594c700 1 -- 172.21.15.59:0/1366029339 learned_addr learned my addr 172.21.15.59:0/1366029339 (peer_addr_for_me v2:172.21.15.59:0/0) 2023-11-24T07:59:32.638 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:32.635+0000 7ff60594c700 1 -- 172.21.15.59:0/1366029339 >> v1:172.21.15.76:6789/0 conn(0x7ff60006b6e0 legacy=0x7ff60006db70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T07:59:32.638 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:32.635+0000 7ff60594c700 1 -- 172.21.15.59:0/1366029339 >> v1:172.21.15.59:6789/0 conn(0x7ff600144c10 legacy=0x7ff60006afd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:32.638 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:32.635+0000 7ff5f7fff700 1 -- 172.21.15.59:0/1366029339 <== mon.0 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3933761040 0 0) 0x7ff600160720 con 0x7ff600144c10 2023-11-24T07:59:32.638 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:32.635+0000 7ff60594c700 1 -- 172.21.15.59:0/1366029339 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff60006f9f0 con 0x7ff600145100 2023-11-24T07:59:32.638 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:32.635+0000 7ff60594c700 1 --2- 172.21.15.59:0/1366029339 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff600145100 0x7ff60006e280 secure :-1 s=READY pgs=922 cs=0 l=1 rev1=1 crypto rx=0x7ff5f8009e50 tx=0x7ff5f800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5fa8cbdf871ff779 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:59:32.638 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:32.635+0000 7ff5f7fff700 1 -- 172.21.15.59:0/1366029339 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5f800e070 con 0x7ff600145100 2023-11-24T07:59:32.638 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:32.635+0000 7ff5f7fff700 1 -- 172.21.15.59:0/1366029339 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff5f800b9c0 con 0x7ff600145100 2023-11-24T07:59:32.638 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:32.635+0000 7ff5f7fff700 1 -- 172.21.15.59:0/1366029339 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5f800bd70 con 0x7ff600145100 2023-11-24T07:59:32.639 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:32.636+0000 7ff6073af700 1 -- 172.21.15.59:0/1366029339 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff600145100 msgr2=0x7ff60006e280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:32.639 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:32.636+0000 7ff6073af700 1 --2- 172.21.15.59:0/1366029339 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff600145100 0x7ff60006e280 secure :-1 s=READY pgs=922 cs=0 l=1 rev1=1 crypto rx=0x7ff5f8009e50 tx=0x7ff5f800b330 comp rx=0 tx=0).stop 2023-11-24T07:59:32.640 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:32.636+0000 7ff6073af700 1 -- 172.21.15.59:0/1366029339 shutdown_connections 2023-11-24T07:59:32.640 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:32.636+0000 7ff6073af700 1 --2- 172.21.15.59:0/1366029339 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff600145100 0x7ff60006e280 unknown :-1 s=CLOSED pgs=922 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:32.640 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:32.636+0000 7ff6073af700 1 -- 172.21.15.59:0/1366029339 >> 172.21.15.59:0/1366029339 conn(0x7ff6000b2740 msgr2=0x7ff6000b2b20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:59:32.640 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:32.636+0000 7ff6073af700 1 -- 172.21.15.59:0/1366029339 shutdown_connections 2023-11-24T07:59:32.640 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:32.636+0000 7ff6073af700 1 -- 172.21.15.59:0/1366029339 wait complete. 2023-11-24T07:59:32.640 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:32.638+0000 7ff6073af700 1 Processor -- start 2023-11-24T07:59:32.640 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:32.638+0000 7ff6073af700 1 -- start start 2023-11-24T07:59:32.642 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:32.639+0000 7ff6073af700 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7ff60006b6e0 0x7ff60013a8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:32.642 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:32.639+0000 7ff6073af700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff600144c10 0x7ff60013f020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:32.642 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:32.639+0000 7ff6073af700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff600145100 0x7ff6001f4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:32.643 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:32.639+0000 7ff6073af700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6001f5c70 con 0x7ff600145100 2023-11-24T07:59:32.643 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:32.639+0000 7ff6073af700 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6001f56b0 con 0x7ff600144c10 2023-11-24T07:59:32.643 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:32.639+0000 7ff6073af700 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff6001f5990 con 0x7ff60006b6e0 2023-11-24T07:59:32.643 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:32.639+0000 7ff60594c700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff600145100 0x7ff6001f4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:32.643 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:32.639+0000 7ff60594c700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff600145100 0x7ff6001f4d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:55224/0 (socket says 172.21.15.59:55224) 2023-11-24T07:59:32.643 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:32.639+0000 7ff60594c700 1 -- 172.21.15.59:0/3659871069 learned_addr learned my addr 172.21.15.59:0/3659871069 (peer_addr_for_me v2:172.21.15.59:0/0) 2023-11-24T07:59:32.643 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:32.639+0000 7ff60514b700 1 --2- 172.21.15.59:0/3659871069 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7ff60006b6e0 0x7ff60013a8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:32.643 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:32.639+0000 7ff60494a700 1 --2- 172.21.15.59:0/3659871069 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff600144c10 0x7ff60013f020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:32.644 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:32.639+0000 7ff60594c700 1 -- 172.21.15.59:0/3659871069 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7ff60006b6e0 msgr2=0x7ff60013a8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:32.644 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:32.639+0000 7ff60594c700 1 --2- 172.21.15.59:0/3659871069 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7ff60006b6e0 0x7ff60013a8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:32.644 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:32.639+0000 7ff60594c700 1 -- 172.21.15.59:0/3659871069 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff600144c10 msgr2=0x7ff60013f020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:32.644 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:32.639+0000 7ff60594c700 1 --2- 172.21.15.59:0/3659871069 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff600144c10 0x7ff60013f020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:32.644 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:32.639+0000 7ff60594c700 1 -- 172.21.15.59:0/3659871069 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff5f8009b30 con 0x7ff600145100 2023-11-24T07:59:32.644 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:32.639+0000 7ff60494a700 1 --2- 172.21.15.59:0/3659871069 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff600144c10 0x7ff60013f020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T07:59:32.644 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:32.639+0000 7ff60594c700 1 --2- 172.21.15.59:0/3659871069 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff600145100 0x7ff6001f4d40 secure :-1 s=READY pgs=923 cs=0 l=1 rev1=1 crypto rx=0x7ff6000d1bd0 tx=0x7ff5f8004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:59:32.644 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:32.639+0000 7ff5f5ffb700 1 -- 172.21.15.59:0/3659871069 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5f800bd30 con 0x7ff600145100 2023-11-24T07:59:32.644 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:32.640+0000 7ff5f5ffb700 1 -- 172.21.15.59:0/3659871069 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff5f80092e0 con 0x7ff600145100 2023-11-24T07:59:32.645 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:32.640+0000 7ff5f5ffb700 1 -- 172.21.15.59:0/3659871069 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5f801b630 con 0x7ff600145100 2023-11-24T07:59:32.645 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:32.640+0000 7ff6073af700 1 -- 172.21.15.59:0/3659871069 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff60013e820 con 0x7ff600145100 2023-11-24T07:59:32.645 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:32.640+0000 7ff6073af700 1 -- 172.21.15.59:0/3659871069 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff60013adf0 con 0x7ff600145100 2023-11-24T07:59:32.645 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:32.641+0000 7ff6073af700 1 -- 172.21.15.59:0/3659871069 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff60013e680 con 0x7ff600145100 2023-11-24T07:59:32.646 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:32.641+0000 7ff5f5ffb700 1 -- 172.21.15.59:0/3659871069 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 11) v1 ==== 89741+0+0 (secure 0 0 0) 0x7ff5f801b790 con 0x7ff600145100 2023-11-24T07:59:32.646 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:32.641+0000 7ff5f5ffb700 1 --2- 172.21.15.59:0/3659871069 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7ff5e406b470 0x7ff5e406d930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:32.646 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:32.642+0000 7ff5f5ffb700 1 -- 172.21.15.59:0/3659871069 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 5709+0+0 (secure 0 0 0) 0x7ff5f808b680 con 0x7ff600145100 2023-11-24T07:59:32.646 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:32.642+0000 7ff60514b700 1 --2- 172.21.15.59:0/3659871069 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7ff5e406b470 0x7ff5e406d930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:32.646 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:32.642+0000 7ff60514b700 1 --2- 172.21.15.59:0/3659871069 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7ff5e406b470 0x7ff5e406d930 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7ff5fc000f60 tx=0x7ff5fc009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:59:32.648 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:32.646+0000 7ff5f5ffb700 1 -- 172.21.15.59:0/3659871069 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7ff60013e680 con 0x7ff600145100 2023-11-24T07:59:32.851 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:32.849+0000 7ff6073af700 1 -- 172.21.15.59:0/3659871069 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7ff60013e680 con 0x7ff600145100 2023-11-24T07:59:32.851 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:32.849+0000 7ff5f5ffb700 1 -- 172.21.15.59:0/3659871069 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v74) v1 ==== 104+0+11 (secure 0 0 0) 0x7ff5f8021020 con 0x7ff600145100 2023-11-24T07:59:32.852 INFO:teuthology.orchestra.run.smithi059.stdout:pg_num: 32 2023-11-24T07:59:32.854 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:32.851+0000 7ff6073af700 1 -- 172.21.15.59:0/3659871069 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7ff5e406b470 msgr2=0x7ff5e406d930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:32.854 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:32.851+0000 7ff6073af700 1 --2- 172.21.15.59:0/3659871069 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7ff5e406b470 0x7ff5e406d930 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7ff5fc000f60 tx=0x7ff5fc009040 comp rx=0 tx=0).stop 2023-11-24T07:59:32.854 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:32.851+0000 7ff6073af700 1 -- 172.21.15.59:0/3659871069 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff600145100 msgr2=0x7ff6001f4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:32.854 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:32.851+0000 7ff6073af700 1 --2- 172.21.15.59:0/3659871069 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff600145100 0x7ff6001f4d40 secure :-1 s=READY pgs=923 cs=0 l=1 rev1=1 crypto rx=0x7ff6000d1bd0 tx=0x7ff5f8004ad0 comp rx=0 tx=0).stop 2023-11-24T07:59:32.854 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:32.851+0000 7ff6073af700 1 -- 172.21.15.59:0/3659871069 shutdown_connections 2023-11-24T07:59:32.854 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:32.851+0000 7ff6073af700 1 --2- 172.21.15.59:0/3659871069 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7ff5e406b470 0x7ff5e406d930 unknown :-1 s=CLOSED pgs=429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:32.854 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:32.851+0000 7ff6073af700 1 --2- 172.21.15.59:0/3659871069 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff600145100 0x7ff6001f4d40 unknown :-1 s=CLOSED pgs=923 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:32.854 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:32.851+0000 7ff6073af700 1 --2- 172.21.15.59:0/3659871069 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7ff60006b6e0 0x7ff60013a8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:32.855 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:32.851+0000 7ff6073af700 1 --2- 172.21.15.59:0/3659871069 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff600144c10 0x7ff60013f020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:32.855 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:32.851+0000 7ff6073af700 1 -- 172.21.15.59:0/3659871069 >> 172.21.15.59:0/3659871069 conn(0x7ff6000b2740 msgr2=0x7ff6000b0800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:59:32.855 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:32.851+0000 7ff6073af700 1 -- 172.21.15.59:0/3659871069 shutdown_connections 2023-11-24T07:59:32.855 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:32.852+0000 7ff6073af700 1 -- 172.21.15.59:0/3659871069 wait complete. 2023-11-24T07:59:32.865 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-24T07:59:32.993 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:32.991+0000 7f34e000a700 1 Processor -- start 2023-11-24T07:59:32.993 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:32.991+0000 7f34e000a700 1 -- start start 2023-11-24T07:59:32.996 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:32.992+0000 7f34e000a700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f34d81459c0 0x7f34d815c540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:32.996 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:32.992+0000 7f34e000a700 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f34d80cd5d0 0x7f34d806afd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:32.996 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:32.992+0000 7f34e000a700 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f34d806daf0 con 0x7f34d80cda50 2023-11-24T07:59:32.996 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:32.992+0000 7f34e000a700 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f34d80cfe60 con 0x7f34d81459c0 2023-11-24T07:59:32.996 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:32.992+0000 7f34e000a700 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f34d80d0140 con 0x7f34d80cd5d0 2023-11-24T07:59:32.996 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:32.992+0000 7f34ddda6700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f34d81459c0 0x7f34d815c540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:32.996 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:32.992+0000 7f34ddda6700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f34d81459c0 0x7f34d815c540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:55228/0 (socket says 172.21.15.59:55228) 2023-11-24T07:59:32.996 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:32.992+0000 7f34ddda6700 1 -- 172.21.15.59:0/2635329139 learned_addr learned my addr 172.21.15.59:0/2635329139 (peer_addr_for_me v2:172.21.15.59:0/0) 2023-11-24T07:59:32.997 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:32.992+0000 7f34dd5a5700 1 --2- 172.21.15.59:0/2635329139 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f34d80cd5d0 0x7f34d806afd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:32.997 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:32.993+0000 7f34ddda6700 1 -- 172.21.15.59:0/2635329139 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f34d80cd5d0 msgr2=0x7f34d806afd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T07:59:32.997 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:32.993+0000 7f34ddda6700 1 --2- 172.21.15.59:0/2635329139 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f34d80cd5d0 0x7f34d806afd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:32.997 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:32.993+0000 7f34ddda6700 1 -- 172.21.15.59:0/2635329139 >> v1:172.21.15.76:6789/0 conn(0x7f34d80cda50 legacy=0x7f34d806e260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T07:59:32.997 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:32.993+0000 7f34ddda6700 1 -- 172.21.15.59:0/2635329139 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f34d80d0420 con 0x7f34d81459c0 2023-11-24T07:59:32.997 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:32.993+0000 7f34ddda6700 1 --2- 172.21.15.59:0/2635329139 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f34d81459c0 0x7f34d815c540 secure :-1 s=READY pgs=924 cs=0 l=1 rev1=1 crypto rx=0x7f34c80098f0 tx=0x7f34c8009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3657a5480eea59da server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:59:32.997 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:32.993+0000 7f34dcda4700 1 -- 172.21.15.59:0/2635329139 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34c800b8a0 con 0x7f34d81459c0 2023-11-24T07:59:32.997 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:32.993+0000 7f34dcda4700 1 -- 172.21.15.59:0/2635329139 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f34c800ba00 con 0x7f34d81459c0 2023-11-24T07:59:32.997 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:32.993+0000 7f34dcda4700 1 -- 172.21.15.59:0/2635329139 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34c800bd90 con 0x7f34d81459c0 2023-11-24T07:59:32.998 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:32.993+0000 7f34e000a700 1 -- 172.21.15.59:0/2635329139 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f34d81459c0 msgr2=0x7f34d815c540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:32.998 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:32.993+0000 7f34e000a700 1 --2- 172.21.15.59:0/2635329139 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f34d81459c0 0x7f34d815c540 secure :-1 s=READY pgs=924 cs=0 l=1 rev1=1 crypto rx=0x7f34c80098f0 tx=0x7f34c8009c00 comp rx=0 tx=0).stop 2023-11-24T07:59:32.998 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:32.993+0000 7f34e000a700 1 -- 172.21.15.59:0/2635329139 shutdown_connections 2023-11-24T07:59:32.998 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:32.993+0000 7f34e000a700 1 --2- 172.21.15.59:0/2635329139 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f34d81459c0 0x7f34d815c540 unknown :-1 s=CLOSED pgs=924 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:32.998 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:32.993+0000 7f34e000a700 1 --2- 172.21.15.59:0/2635329139 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f34d80cd5d0 0x7f34d806afd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:32.998 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:32.993+0000 7f34e000a700 1 -- 172.21.15.59:0/2635329139 >> 172.21.15.59:0/2635329139 conn(0x7f34d80b27b0 msgr2=0x7f34d80b2b90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:59:32.998 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:32.994+0000 7f34e000a700 1 -- 172.21.15.59:0/2635329139 shutdown_connections 2023-11-24T07:59:32.999 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:32.994+0000 7f34e000a700 1 -- 172.21.15.59:0/2635329139 wait complete. 2023-11-24T07:59:33.001 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:32.995+0000 7f34e000a700 1 Processor -- start 2023-11-24T07:59:33.001 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:32.995+0000 7f34e000a700 1 -- start start 2023-11-24T07:59:33.001 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:32.995+0000 7f34e000a700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f34d80cd5d0 0x7f34d8152400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:33.001 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:32.995+0000 7f34e000a700 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f34d80cda50 0x7f34d813a080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:33.001 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:32.995+0000 7f34e000a700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f34d81459c0 0x7f34d813a5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:33.001 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:32.996+0000 7f34e000a700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f34d8152fb0 con 0x7f34d81459c0 2023-11-24T07:59:33.002 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:32.996+0000 7f34e000a700 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f34d813ac60 con 0x7f34d80cd5d0 2023-11-24T07:59:33.002 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:32.996+0000 7f34e000a700 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f34d813af40 con 0x7f34d80cda50 2023-11-24T07:59:33.002 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:32.996+0000 7f34de5a7700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f34d81459c0 0x7f34d813a5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:33.002 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:32.996+0000 7f34de5a7700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f34d81459c0 0x7f34d813a5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:55238/0 (socket says 172.21.15.59:55238) 2023-11-24T07:59:33.002 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:32.996+0000 7f34de5a7700 1 -- 172.21.15.59:0/1305672815 learned_addr learned my addr 172.21.15.59:0/1305672815 (peer_addr_for_me v2:172.21.15.59:0/0) 2023-11-24T07:59:33.002 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:32.996+0000 7f34ddda6700 1 --2- 172.21.15.59:0/1305672815 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f34d80cd5d0 0x7f34d8152400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:33.002 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:32.996+0000 7f34dd5a5700 1 --2- 172.21.15.59:0/1305672815 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f34d80cda50 0x7f34d813a080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:33.002 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:32.996+0000 7f34de5a7700 1 -- 172.21.15.59:0/1305672815 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f34d80cda50 msgr2=0x7f34d813a080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:33.002 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:32.996+0000 7f34de5a7700 1 --2- 172.21.15.59:0/1305672815 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f34d80cda50 0x7f34d813a080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:33.003 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:32.996+0000 7f34de5a7700 1 -- 172.21.15.59:0/1305672815 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f34d80cd5d0 msgr2=0x7f34d8152400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:33.003 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:32.996+0000 7f34de5a7700 1 --2- 172.21.15.59:0/1305672815 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f34d80cd5d0 0x7f34d8152400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:33.003 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:32.996+0000 7f34de5a7700 1 -- 172.21.15.59:0/1305672815 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f34c8009550 con 0x7f34d81459c0 2023-11-24T07:59:33.003 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:32.996+0000 7f34de5a7700 1 --2- 172.21.15.59:0/1305672815 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f34d81459c0 0x7f34d813a5c0 secure :-1 s=READY pgs=925 cs=0 l=1 rev1=1 crypto rx=0x7f34d000cb90 tx=0x7f34d000cea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:59:33.003 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:32.996+0000 7f34ce7fc700 1 -- 172.21.15.59:0/1305672815 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34d0003c00 con 0x7f34d81459c0 2023-11-24T07:59:33.003 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:32.996+0000 7f34ce7fc700 1 -- 172.21.15.59:0/1305672815 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f34d0003d60 con 0x7f34d81459c0 2023-11-24T07:59:33.003 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:32.996+0000 7f34e000a700 1 -- 172.21.15.59:0/1305672815 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f34d813c1b0 con 0x7f34d81459c0 2023-11-24T07:59:33.003 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:32.996+0000 7f34ce7fc700 1 -- 172.21.15.59:0/1305672815 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34d000aae0 con 0x7f34d81459c0 2023-11-24T07:59:33.003 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:32.996+0000 7f34e000a700 1 -- 172.21.15.59:0/1305672815 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f34d813c770 con 0x7f34d81459c0 2023-11-24T07:59:33.003 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:32.997+0000 7f34e000a700 1 -- 172.21.15.59:0/1305672815 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f34d813c930 con 0x7f34d81459c0 2023-11-24T07:59:33.003 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:32.998+0000 7f34ce7fc700 1 -- 172.21.15.59:0/1305672815 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 11) v1 ==== 89741+0+0 (secure 0 0 0) 0x7f34d000ac40 con 0x7f34d81459c0 2023-11-24T07:59:33.004 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:32.998+0000 7f34ce7fc700 1 --2- 172.21.15.59:0/1305672815 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f34bc06b470 0x7f34bc06d930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:33.004 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:32.998+0000 7f34ce7fc700 1 -- 172.21.15.59:0/1305672815 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f34d0089f80 con 0x7f34d81459c0 2023-11-24T07:59:33.004 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:32.999+0000 7f34ddda6700 1 --2- 172.21.15.59:0/1305672815 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f34bc06b470 0x7f34bc06d930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:33.004 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:32.999+0000 7f34ddda6700 1 --2- 172.21.15.59:0/1305672815 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f34bc06b470 0x7f34bc06d930 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f34c8014040 tx=0x7f34c8008030 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:59:33.004 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:33.003+0000 7f34ce7fc700 1 -- 172.21.15.59:0/1305672815 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f34d0054830 con 0x7f34d81459c0 2023-11-24T07:59:33.206 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:33.205+0000 7f34e000a700 1 -- 172.21.15.59:0/1305672815 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f34d80b3030 con 0x7f34d81459c0 2023-11-24T07:59:33.207 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:33.205+0000 7f34ce7fc700 1 -- 172.21.15.59:0/1305672815 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v74) v1 ==== 100+0+11 (secure 0 0 0) 0x7f34d0057e50 con 0x7f34d81459c0 2023-11-24T07:59:33.207 INFO:teuthology.orchestra.run.smithi059.stdout:pg_num: 64 2023-11-24T07:59:33.209 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:33.207+0000 7f34e000a700 1 -- 172.21.15.59:0/1305672815 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f34bc06b470 msgr2=0x7f34bc06d930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:33.209 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:33.207+0000 7f34e000a700 1 --2- 172.21.15.59:0/1305672815 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f34bc06b470 0x7f34bc06d930 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f34c8014040 tx=0x7f34c8008030 comp rx=0 tx=0).stop 2023-11-24T07:59:33.210 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:33.207+0000 7f34e000a700 1 -- 172.21.15.59:0/1305672815 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f34d81459c0 msgr2=0x7f34d813a5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:33.210 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:33.207+0000 7f34e000a700 1 --2- 172.21.15.59:0/1305672815 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f34d81459c0 0x7f34d813a5c0 secure :-1 s=READY pgs=925 cs=0 l=1 rev1=1 crypto rx=0x7f34d000cb90 tx=0x7f34d000cea0 comp rx=0 tx=0).stop 2023-11-24T07:59:33.210 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:33.207+0000 7f34e000a700 1 -- 172.21.15.59:0/1305672815 shutdown_connections 2023-11-24T07:59:33.210 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:33.207+0000 7f34e000a700 1 --2- 172.21.15.59:0/1305672815 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f34bc06b470 0x7f34bc06d930 unknown :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:33.210 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:33.207+0000 7f34e000a700 1 --2- 172.21.15.59:0/1305672815 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f34d81459c0 0x7f34d813a5c0 unknown :-1 s=CLOSED pgs=925 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:33.210 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:33.208+0000 7f34e000a700 1 --2- 172.21.15.59:0/1305672815 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f34d80cd5d0 0x7f34d8152400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:33.210 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:33.208+0000 7f34e000a700 1 --2- 172.21.15.59:0/1305672815 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f34d80cda50 0x7f34d813a080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:33.210 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:33.208+0000 7f34e000a700 1 -- 172.21.15.59:0/1305672815 >> 172.21.15.59:0/1305672815 conn(0x7f34d80b27b0 msgr2=0x7f34d80ccdb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:59:33.210 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:33.208+0000 7f34e000a700 1 -- 172.21.15.59:0/1305672815 shutdown_connections 2023-11-24T07:59:33.210 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:33.208+0000 7f34e000a700 1 -- 172.21.15.59:0/1305672815 wait complete. 2023-11-24T07:59:33.222 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-11-24T07:59:33.347 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:33.346+0000 7ff40d775700 1 Processor -- start 2023-11-24T07:59:33.348 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:33.346+0000 7ff40d775700 1 -- start start 2023-11-24T07:59:33.350 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:33.346+0000 7ff40d775700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7ff4080d0250 0x7ff4080d0630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:33.350 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:33.346+0000 7ff40d775700 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7ff4080cc780 0x7ff4080ccc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:33.350 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:33.347+0000 7ff40d775700 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff4080c5bd0 con 0x7ff4080ce680 2023-11-24T07:59:33.350 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:33.347+0000 7ff40d775700 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff4080d0b70 con 0x7ff4080d0250 2023-11-24T07:59:33.350 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:33.347+0000 7ff40d775700 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff4080cdb30 con 0x7ff4080cc780 2023-11-24T07:59:33.350 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:33.347+0000 7ff406ffd700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7ff4080d0250 0x7ff4080d0630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:33.350 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:33.347+0000 7ff406ffd700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7ff4080d0250 0x7ff4080d0630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:55242/0 (socket says 172.21.15.59:55242) 2023-11-24T07:59:33.350 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:33.347+0000 7ff406ffd700 1 -- 172.21.15.59:0/1778671776 learned_addr learned my addr 172.21.15.59:0/1778671776 (peer_addr_for_me v2:172.21.15.59:0/0) 2023-11-24T07:59:33.351 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:33.347+0000 7ff4067fc700 1 --2- 172.21.15.59:0/1778671776 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7ff4080cc780 0x7ff4080ccc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:33.351 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:33.347+0000 7ff406ffd700 1 -- 172.21.15.59:0/1778671776 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7ff4080cc780 msgr2=0x7ff4080ccc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T07:59:33.351 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:33.347+0000 7ff406ffd700 1 --2- 172.21.15.59:0/1778671776 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7ff4080cc780 0x7ff4080ccc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:33.351 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:33.347+0000 7ff406ffd700 1 -- 172.21.15.59:0/1778671776 >> v1:172.21.15.76:6789/0 conn(0x7ff4080ce680 legacy=0x7ff4080ceb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:33.351 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:33.347+0000 7ff406ffd700 1 -- 172.21.15.59:0/1778671776 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff4080cd570 con 0x7ff4080d0250 2023-11-24T07:59:33.351 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:33.347+0000 7ff406ffd700 1 --2- 172.21.15.59:0/1778671776 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff4080d0250 0x7ff4080d0630 secure :-1 s=READY pgs=926 cs=0 l=1 rev1=1 crypto rx=0x7ff3f80098f0 tx=0x7ff3f8009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=461ca2a66b838625 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:59:33.351 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:33.347+0000 7ff405ffb700 1 -- 172.21.15.59:0/1778671776 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3f8011830 con 0x7ff4080d0250 2023-11-24T07:59:33.351 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:33.347+0000 7ff405ffb700 1 -- 172.21.15.59:0/1778671776 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff3f8011990 con 0x7ff4080d0250 2023-11-24T07:59:33.351 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:33.347+0000 7ff405ffb700 1 -- 172.21.15.59:0/1778671776 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3f8011d20 con 0x7ff4080d0250 2023-11-24T07:59:33.352 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:33.348+0000 7ff40d775700 1 -- 172.21.15.59:0/1778671776 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff4080d0250 msgr2=0x7ff4080d0630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:33.352 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:33.348+0000 7ff40d775700 1 --2- 172.21.15.59:0/1778671776 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff4080d0250 0x7ff4080d0630 secure :-1 s=READY pgs=926 cs=0 l=1 rev1=1 crypto rx=0x7ff3f80098f0 tx=0x7ff3f8009c00 comp rx=0 tx=0).stop 2023-11-24T07:59:33.352 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:33.348+0000 7ff40d775700 1 -- 172.21.15.59:0/1778671776 shutdown_connections 2023-11-24T07:59:33.352 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:33.348+0000 7ff40d775700 1 --2- 172.21.15.59:0/1778671776 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff4080d0250 0x7ff4080d0630 unknown :-1 s=CLOSED pgs=926 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:33.352 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:33.348+0000 7ff40d775700 1 --2- 172.21.15.59:0/1778671776 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7ff4080cc780 0x7ff4080ccc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:33.352 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:33.348+0000 7ff40d775700 1 -- 172.21.15.59:0/1778671776 >> 172.21.15.59:0/1778671776 conn(0x7ff4080b3800 msgr2=0x7ff4080b3c10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:59:33.352 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:33.348+0000 7ff40d775700 1 -- 172.21.15.59:0/1778671776 shutdown_connections 2023-11-24T07:59:33.352 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:33.348+0000 7ff40d775700 1 -- 172.21.15.59:0/1778671776 wait complete. 2023-11-24T07:59:33.355 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:33.349+0000 7ff40d775700 1 Processor -- start 2023-11-24T07:59:33.355 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:33.349+0000 7ff40d775700 1 -- start start 2023-11-24T07:59:33.355 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:33.350+0000 7ff40d775700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff4080cc780 0x7ff4080caf50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:33.355 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:33.350+0000 7ff40d775700 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7ff4080ce680 0x7ff4081f6aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:33.355 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:33.350+0000 7ff40d775700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff4080d0250 0x7ff4080b6440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:33.356 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:33.350+0000 7ff40d775700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4080cb490 con 0x7ff4080d0250 2023-11-24T07:59:33.356 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:33.350+0000 7ff40d775700 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4080cb6c0 con 0x7ff4080cc780 2023-11-24T07:59:33.356 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:33.350+0000 7ff40d775700 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff4080ca640 con 0x7ff4080ce680 2023-11-24T07:59:33.356 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:33.350+0000 7ff4077fe700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff4080d0250 0x7ff4080b6440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:33.356 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:33.350+0000 7ff4077fe700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff4080d0250 0x7ff4080b6440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:55252/0 (socket says 172.21.15.59:55252) 2023-11-24T07:59:33.357 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:33.350+0000 7ff4077fe700 1 -- 172.21.15.59:0/2302174262 learned_addr learned my addr 172.21.15.59:0/2302174262 (peer_addr_for_me v2:172.21.15.59:0/0) 2023-11-24T07:59:33.357 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:33.350+0000 7ff4067fc700 1 --2- 172.21.15.59:0/2302174262 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7ff4080ce680 0x7ff4081f6aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:33.357 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:33.350+0000 7ff406ffd700 1 --2- 172.21.15.59:0/2302174262 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff4080cc780 0x7ff4080caf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:33.357 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:33.351+0000 7ff4077fe700 1 -- 172.21.15.59:0/2302174262 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7ff4080ce680 msgr2=0x7ff4081f6aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:33.357 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:33.351+0000 7ff4077fe700 1 --2- 172.21.15.59:0/2302174262 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7ff4080ce680 0x7ff4081f6aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:33.358 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:33.351+0000 7ff4077fe700 1 -- 172.21.15.59:0/2302174262 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff4080cc780 msgr2=0x7ff4080caf50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:33.358 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:33.351+0000 7ff4077fe700 1 --2- 172.21.15.59:0/2302174262 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff4080cc780 0x7ff4080caf50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:33.358 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:33.351+0000 7ff4077fe700 1 -- 172.21.15.59:0/2302174262 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff3f8009550 con 0x7ff4080d0250 2023-11-24T07:59:33.358 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:33.351+0000 7ff4077fe700 1 --2- 172.21.15.59:0/2302174262 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff4080d0250 0x7ff4080b6440 secure :-1 s=READY pgs=927 cs=0 l=1 rev1=1 crypto rx=0x7ff40000cc50 tx=0x7ff40000a340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:59:33.358 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:33.351+0000 7ff3ef7fe700 1 -- 172.21.15.59:0/2302174262 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff40000ad80 con 0x7ff4080d0250 2023-11-24T07:59:33.358 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:33.351+0000 7ff3ef7fe700 1 -- 172.21.15.59:0/2302174262 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff40000ce50 con 0x7ff4080d0250 2023-11-24T07:59:33.358 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:33.351+0000 7ff40d775700 1 -- 172.21.15.59:0/2302174262 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff4080c9880 con 0x7ff4080d0250 2023-11-24T07:59:33.359 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:33.351+0000 7ff3ef7fe700 1 -- 172.21.15.59:0/2302174262 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff400003c40 con 0x7ff4080d0250 2023-11-24T07:59:33.359 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:33.351+0000 7ff40d775700 1 -- 172.21.15.59:0/2302174262 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff4080c9e40 con 0x7ff4080d0250 2023-11-24T07:59:33.359 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:33.352+0000 7ff40d775700 1 -- 172.21.15.59:0/2302174262 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff4080d14b0 con 0x7ff4080d0250 2023-11-24T07:59:33.359 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:33.352+0000 7ff3ef7fe700 1 -- 172.21.15.59:0/2302174262 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 11) v1 ==== 89741+0+0 (secure 0 0 0) 0x7ff4000050d0 con 0x7ff4080d0250 2023-11-24T07:59:33.359 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:33.353+0000 7ff3ef7fe700 1 --2- 172.21.15.59:0/2302174262 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7ff3f406b470 0x7ff3f406d930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:33.359 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:33.353+0000 7ff3ef7fe700 1 -- 172.21.15.59:0/2302174262 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 5709+0+0 (secure 0 0 0) 0x7ff40008a160 con 0x7ff4080d0250 2023-11-24T07:59:33.359 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:33.353+0000 7ff406ffd700 1 --2- 172.21.15.59:0/2302174262 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7ff3f406b470 0x7ff3f406d930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:33.359 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:33.353+0000 7ff406ffd700 1 --2- 172.21.15.59:0/2302174262 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7ff3f406b470 0x7ff3f406d930 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7ff3f8000c00 tx=0x7ff3f8008030 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:59:33.360 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:33.357+0000 7ff3ef7fe700 1 -- 172.21.15.59:0/2302174262 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7ff4000549a0 con 0x7ff4080d0250 2023-11-24T07:59:33.573 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:33.572+0000 7ff40d775700 1 -- 172.21.15.59:0/2302174262 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7ff408068670 con 0x7ff4080d0250 2023-11-24T07:59:33.574 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:33.572+0000 7ff3ef7fe700 1 -- 172.21.15.59:0/2302174262 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v74) v1 ==== 80+0+0 (secure 0 0 0) 0x7ff400057fc0 con 0x7ff4080d0250 2023-11-24T07:59:33.576 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:33.574+0000 7ff40d775700 1 -- 172.21.15.59:0/2302174262 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7ff3f406b470 msgr2=0x7ff3f406d930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:33.576 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:33.574+0000 7ff40d775700 1 --2- 172.21.15.59:0/2302174262 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7ff3f406b470 0x7ff3f406d930 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7ff3f8000c00 tx=0x7ff3f8008030 comp rx=0 tx=0).stop 2023-11-24T07:59:33.576 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:33.574+0000 7ff40d775700 1 -- 172.21.15.59:0/2302174262 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff4080d0250 msgr2=0x7ff4080b6440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:33.576 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:33.574+0000 7ff40d775700 1 --2- 172.21.15.59:0/2302174262 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff4080d0250 0x7ff4080b6440 secure :-1 s=READY pgs=927 cs=0 l=1 rev1=1 crypto rx=0x7ff40000cc50 tx=0x7ff40000a340 comp rx=0 tx=0).stop 2023-11-24T07:59:33.576 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:33.574+0000 7ff40d775700 1 -- 172.21.15.59:0/2302174262 shutdown_connections 2023-11-24T07:59:33.576 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:33.574+0000 7ff40d775700 1 --2- 172.21.15.59:0/2302174262 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7ff3f406b470 0x7ff3f406d930 unknown :-1 s=CLOSED pgs=431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:33.577 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:33.574+0000 7ff40d775700 1 --2- 172.21.15.59:0/2302174262 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff4080d0250 0x7ff4080b6440 unknown :-1 s=CLOSED pgs=927 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:33.577 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:33.574+0000 7ff40d775700 1 --2- 172.21.15.59:0/2302174262 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff4080cc780 0x7ff4080caf50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:33.577 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:33.574+0000 7ff40d775700 1 --2- 172.21.15.59:0/2302174262 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7ff4080ce680 0x7ff4081f6aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:33.577 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:33.574+0000 7ff40d775700 1 -- 172.21.15.59:0/2302174262 >> 172.21.15.59:0/2302174262 conn(0x7ff4080b3800 msgr2=0x7ff408156e30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:59:33.577 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:33.574+0000 7ff40d775700 1 -- 172.21.15.59:0/2302174262 shutdown_connections 2023-11-24T07:59:33.577 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:33.574+0000 7ff40d775700 1 -- 172.21.15.59:0/2302174262 wait complete. 2023-11-24T07:59:33.577 INFO:teuthology.orchestra.run.smithi059.stderr:listed 0 entries 2023-11-24T07:59:33.588 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2023-11-24T07:59:33.589 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T07:59:33.589 DEBUG:teuthology.orchestra.run.smithi059:> sudo umount /home/ubuntu/cephtest/mnt.0 2023-11-24T07:59:33.634 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi059.front.sepia.ceph.com 2023-11-24T07:59:33.634 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T07:59:33.634 DEBUG:teuthology.orchestra.run.smithi059:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-11-24T07:59:33.651 DEBUG:teuthology.orchestra.run.smithi076:> set -ex 2023-11-24T07:59:33.651 DEBUG:teuthology.orchestra.run.smithi076:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-24T07:59:33.675 DEBUG:teuthology.orchestra.run.smithi076:> set -ex 2023-11-24T07:59:33.675 DEBUG:teuthology.orchestra.run.smithi076:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-24T07:59:33.735 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-24T07:59:33.861 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:33.859+0000 7f57d9ea9700 1 Processor -- start 2023-11-24T07:59:33.861 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:33.859+0000 7f57d9ea9700 1 -- start start 2023-11-24T07:59:33.864 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:33.860+0000 7f57d9ea9700 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f57d40d0080 0x7f57d40b5890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:33.864 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:33.860+0000 7f57d9ea9700 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f57d4147bb0 con 0x7f57d4071bb0 2023-11-24T07:59:33.864 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:33.860+0000 7f57d9ea9700 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f57d40b5dd0 con 0x7f57d406b6e0 2023-11-24T07:59:33.865 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:33.860+0000 7f57d9ea9700 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f57d4072cd0 con 0x7f57d40d0080 2023-11-24T07:59:33.865 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:33.860+0000 7f57d2ffd700 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7f57d4071bb0 0x7f57d406afd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:58732/0 (socket says 172.21.15.59:58732) 2023-11-24T07:59:33.865 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:33.860+0000 7f57d2ffd700 1 -- 172.21.15.59:0/2429459642 learned_addr learned my addr 172.21.15.59:0/2429459642 (peer_addr_for_me v1:172.21.15.59:0/0) 2023-11-24T07:59:33.865 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:33.860+0000 7f57d37fe700 1 --2- 172.21.15.59:0/2429459642 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f57d40d0080 0x7f57d40b5890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:33.865 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:33.860+0000 7f57d27fc700 1 -- 172.21.15.59:0/2429459642 <== mon.0 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1913224576 0 0) 0x7f57d4147bb0 con 0x7f57d4071bb0 2023-11-24T07:59:33.865 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:33.860+0000 7f57d27fc700 1 -- 172.21.15.59:0/2429459642 --> v1:172.21.15.59:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f57bc0035b0 con 0x7f57d4071bb0 2023-11-24T07:59:33.866 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:33.860+0000 7f57d27fc700 1 -- 172.21.15.59:0/2429459642 <== mon.1 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2794133109 0 0) 0x7f57d40b5dd0 con 0x7f57d406b6e0 2023-11-24T07:59:33.866 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:33.860+0000 7f57d27fc700 1 -- 172.21.15.59:0/2429459642 --> v1:172.21.15.76:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f57d4147bb0 con 0x7f57d406b6e0 2023-11-24T07:59:33.866 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:33.860+0000 7f57d27fc700 1 -- 172.21.15.59:0/2429459642 <== mon.0 v1:172.21.15.59:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1892984995 0 0) 0x7f57bc0035b0 con 0x7f57d4071bb0 2023-11-24T07:59:33.866 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:33.860+0000 7f57d27fc700 1 -- 172.21.15.59:0/2429459642 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f57d40d0080 msgr2=0x7f57d40b5890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T07:59:33.866 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:33.861+0000 7f57d27fc700 1 --2- 172.21.15.59:0/2429459642 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f57d40d0080 0x7f57d40b5890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:33.866 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:33.861+0000 7f57d27fc700 1 -- 172.21.15.59:0/2429459642 >> v1:172.21.15.76:6789/0 conn(0x7f57d406b6e0 legacy=0x7f57d406dfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:33.867 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:33.861+0000 7f57d27fc700 1 -- 172.21.15.59:0/2429459642 --> v1:172.21.15.59:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f57d40d0460 con 0x7f57d4071bb0 2023-11-24T07:59:33.867 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:33.861+0000 7f57d27fc700 1 -- 172.21.15.59:0/2429459642 <== mon.0 v1:172.21.15.59:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1312333566 0 0) 0x7f57c4001670 con 0x7f57d4071bb0 2023-11-24T07:59:33.867 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:33.861+0000 7f57d27fc700 1 -- 172.21.15.59:0/2429459642 >> v1:172.21.15.59:6789/0 conn(0x7f57d4071bb0 legacy=0x7f57d406afd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:33.867 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:33.861+0000 7f57d27fc700 1 --2- 172.21.15.59:0/2429459642 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f57bc003bc0 0x7f57bc003fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:33.867 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:33.861+0000 7f57d27fc700 1 --2- 172.21.15.59:0/2429459642 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f57bc004850 0x7f57bc004cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:33.867 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:33.861+0000 7f57d27fc700 1 --2- 172.21.15.59:0/2429459642 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f57d40d0080 0x7f57bc005210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:33.867 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:33.861+0000 7f57d27fc700 1 -- 172.21.15.59:0/2429459642 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57d4072cd0 con 0x7f57bc003bc0 2023-11-24T07:59:33.868 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:33.861+0000 7f57d27fc700 1 -- 172.21.15.59:0/2429459642 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57bc00bac0 con 0x7f57bc004850 2023-11-24T07:59:33.868 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:33.861+0000 7f57d27fc700 1 -- 172.21.15.59:0/2429459642 --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f57bc00bc50 con 0x7f57d40d0080 2023-11-24T07:59:33.868 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:33.861+0000 7f57d37fe700 1 --2- 172.21.15.59:0/2429459642 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f57bc003bc0 0x7f57bc003fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:33.868 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:33.861+0000 7f57d2ffd700 1 --2- 172.21.15.59:0/2429459642 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f57bc004850 0x7f57bc004cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:33.868 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:33.861+0000 7f57d3fff700 1 --2- 172.21.15.59:0/2429459642 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f57d40d0080 0x7f57bc005210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:33.868 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:33.861+0000 7f57d37fe700 1 -- 172.21.15.59:0/2429459642 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f57d40d0080 msgr2=0x7f57bc005210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T07:59:33.868 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:33.861+0000 7f57d37fe700 1 --2- 172.21.15.59:0/2429459642 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f57d40d0080 0x7f57bc005210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:33.869 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:33.861+0000 7f57d37fe700 1 -- 172.21.15.59:0/2429459642 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f57bc004850 msgr2=0x7f57bc004cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T07:59:33.869 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:33.861+0000 7f57d37fe700 1 --2- 172.21.15.59:0/2429459642 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f57bc004850 0x7f57bc004cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:33.869 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:33.861+0000 7f57d37fe700 1 -- 172.21.15.59:0/2429459642 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f57bc0035b0 con 0x7f57bc003bc0 2023-11-24T07:59:33.869 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:33.861+0000 7f57d37fe700 1 --2- 172.21.15.59:0/2429459642 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f57bc003bc0 0x7f57bc003fc0 secure :-1 s=READY pgs=929 cs=0 l=1 rev1=1 crypto rx=0x7f57c8009b60 tx=0x7f57bc005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b2335edacc664049 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:59:33.869 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:33.861+0000 7f57d27fc700 1 -- 172.21.15.59:0/2429459642 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57c800e6c0 con 0x7f57bc003bc0 2023-11-24T07:59:33.869 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:33.861+0000 7f57d27fc700 1 -- 172.21.15.59:0/2429459642 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f57c800e820 con 0x7f57bc003bc0 2023-11-24T07:59:33.870 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:33.861+0000 7f57d27fc700 1 -- 172.21.15.59:0/2429459642 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57c800eb90 con 0x7f57bc003bc0 2023-11-24T07:59:33.870 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:33.862+0000 7f57d9ea9700 1 -- 172.21.15.59:0/2429459642 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f57bc003bc0 msgr2=0x7f57bc003fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:33.870 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:33.862+0000 7f57d9ea9700 1 --2- 172.21.15.59:0/2429459642 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f57bc003bc0 0x7f57bc003fc0 secure :-1 s=READY pgs=929 cs=0 l=1 rev1=1 crypto rx=0x7f57c8009b60 tx=0x7f57bc005880 comp rx=0 tx=0).stop 2023-11-24T07:59:33.870 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:33.862+0000 7f57d3fff700 1 -- 172.21.15.59:0/2429459642 reap_dead start 2023-11-24T07:59:33.870 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:33.862+0000 7f57d9ea9700 1 -- 172.21.15.59:0/2429459642 shutdown_connections 2023-11-24T07:59:33.870 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:33.862+0000 7f57d9ea9700 1 -- 172.21.15.59:0/2429459642 >> 172.21.15.59:0/2429459642 conn(0x7f57d40b27d0 msgr2=0x7f57d40b2be0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:59:33.870 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:33.862+0000 7f57d9ea9700 1 -- 172.21.15.59:0/2429459642 shutdown_connections 2023-11-24T07:59:33.871 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:33.862+0000 7f57d9ea9700 1 -- 172.21.15.59:0/2429459642 wait complete. 2023-11-24T07:59:33.874 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:33.864+0000 7f57d9ea9700 1 Processor -- start 2023-11-24T07:59:33.874 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:33.864+0000 7f57d9ea9700 1 -- start start 2023-11-24T07:59:33.874 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:33.865+0000 7f57d9ea9700 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f57d415cec0 0x7f57d4142c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:33.874 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:33.865+0000 7f57d9ea9700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f57d41e3d00 0x7f57d40b1a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:33.875 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:33.865+0000 7f57d9ea9700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f57d40b1fb0 0x7f57d4140be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:33.875 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:33.865+0000 7f57d9ea9700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57d41e4e50 con 0x7f57d41e3d00 2023-11-24T07:59:33.875 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:33.865+0000 7f57d9ea9700 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57d415d2a0 con 0x7f57d40b1fb0 2023-11-24T07:59:33.875 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:33.865+0000 7f57d9ea9700 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f57d40b2430 con 0x7f57d415cec0 2023-11-24T07:59:33.875 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:33.865+0000 7f57d2ffd700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f57d41e3d00 0x7f57d40b1a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:33.875 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:33.865+0000 7f57d2ffd700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f57d41e3d00 0x7f57d40b1a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:55270/0 (socket says 172.21.15.59:55270) 2023-11-24T07:59:33.876 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:33.865+0000 7f57d2ffd700 1 -- 172.21.15.59:0/2189591829 learned_addr learned my addr 172.21.15.59:0/2189591829 (peer_addr_for_me v2:172.21.15.59:0/0) 2023-11-24T07:59:33.876 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:33.865+0000 7f57d37fe700 1 --2- 172.21.15.59:0/2189591829 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f57d415cec0 0x7f57d4142c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:33.876 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:33.865+0000 7f57d3fff700 1 --2- 172.21.15.59:0/2189591829 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f57d40b1fb0 0x7f57d4140be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:33.876 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:33.865+0000 7f57d2ffd700 1 -- 172.21.15.59:0/2189591829 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f57d415cec0 msgr2=0x7f57d4142c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:33.876 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:33.865+0000 7f57d2ffd700 1 --2- 172.21.15.59:0/2189591829 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f57d415cec0 0x7f57d4142c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:33.876 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:33.865+0000 7f57d2ffd700 1 -- 172.21.15.59:0/2189591829 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f57d40b1fb0 msgr2=0x7f57d4140be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:33.876 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:33.865+0000 7f57d2ffd700 1 --2- 172.21.15.59:0/2189591829 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f57d40b1fb0 0x7f57d4140be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:33.876 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:33.865+0000 7f57d2ffd700 1 -- 172.21.15.59:0/2189591829 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f57d415dcf0 con 0x7f57d41e3d00 2023-11-24T07:59:33.876 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:33.865+0000 7f57d2ffd700 1 --2- 172.21.15.59:0/2189591829 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f57d41e3d00 0x7f57d40b1a70 secure :-1 s=READY pgs=930 cs=0 l=1 rev1=1 crypto rx=0x7f57c4007340 tx=0x7f57c4007700 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:59:33.876 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:33.865+0000 7f57bbfff700 1 -- 172.21.15.59:0/2189591829 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57c4001620 con 0x7f57d41e3d00 2023-11-24T07:59:33.877 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:33.865+0000 7f57bbfff700 1 -- 172.21.15.59:0/2189591829 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f57c400f870 con 0x7f57d41e3d00 2023-11-24T07:59:33.877 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:33.865+0000 7f57d9ea9700 1 -- 172.21.15.59:0/2189591829 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f57d4143170 con 0x7f57d41e3d00 2023-11-24T07:59:33.877 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:33.865+0000 7f57bbfff700 1 -- 172.21.15.59:0/2189591829 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57c400fc20 con 0x7f57d41e3d00 2023-11-24T07:59:33.877 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:33.866+0000 7f57d9ea9700 1 -- 172.21.15.59:0/2189591829 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f57d41e44b0 con 0x7f57d41e3d00 2023-11-24T07:59:33.877 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:33.866+0000 7f57d9ea9700 1 -- 172.21.15.59:0/2189591829 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f57d4068670 con 0x7f57d41e3d00 2023-11-24T07:59:33.877 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:33.867+0000 7f57bbfff700 1 -- 172.21.15.59:0/2189591829 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 11) v1 ==== 89741+0+0 (secure 0 0 0) 0x7f57c400fd80 con 0x7f57d41e3d00 2023-11-24T07:59:33.877 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:33.867+0000 7f57bbfff700 1 --2- 172.21.15.59:0/2189591829 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f57c006b470 0x7f57c006d930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:33.877 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:33.867+0000 7f57bbfff700 1 -- 172.21.15.59:0/2189591829 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f57c408a010 con 0x7f57d41e3d00 2023-11-24T07:59:33.877 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:33.867+0000 7f57d37fe700 1 --2- 172.21.15.59:0/2189591829 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f57c006b470 0x7f57c006d930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:33.878 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:33.868+0000 7f57d37fe700 1 --2- 172.21.15.59:0/2189591829 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f57c006b470 0x7f57c006d930 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f57c80090b0 tx=0x7f57c801e040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:59:33.878 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:33.872+0000 7f57bbfff700 1 -- 172.21.15.59:0/2189591829 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f57c40548e0 con 0x7f57d41e3d00 2023-11-24T07:59:34.074 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:34.073+0000 7f57d9ea9700 1 -- 172.21.15.59:0/2189591829 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f57d40b3510 con 0x7f57d41e3d00 2023-11-24T07:59:34.075 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:34.074+0000 7f57bbfff700 1 -- 172.21.15.59:0/2189591829 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v74) v1 ==== 74+0+16436 (secure 0 0 0) 0x7f57c4057f00 con 0x7f57d41e3d00 2023-11-24T07:59:34.076 INFO:teuthology.orchestra.run.smithi059.stdout: 2023-11-24T07:59:34.076 INFO:teuthology.orchestra.run.smithi059.stdout:{"epoch":74,"fsid":"9f0e8741-b8f9-48d5-a2c3-09f45903303d","created":"2023-11-24T07:56:06.552729+0000","modified":"2023-11-24T07:59:15.638357+0000","last_up_change":"2023-11-24T07:56:17.372352+0000","last_in_change":"2023-11-24T07:56:09.073250+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":11,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-24T07:56:18.352823+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_metadata","create_time":"2023-11-24T07:59:11.900882+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"73","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_data","create_time":"2023-11-24T07:59:13.025749+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"74","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"6d8f9e9b-1782-4e41-928b-a39f2a8ebc06","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6800","nonce":3373878772},{"type":"v1","addr":"172.21.15.59:6801","nonce":3373878772}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6802","nonce":3373878772},{"type":"v1","addr":"172.21.15.59:6803","nonce":3373878772}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6806","nonce":3373878772},{"type":"v1","addr":"172.21.15.59:6807","nonce":3373878772}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6804","nonce":3373878772},{"type":"v1","addr":"172.21.15.59:6805","nonce":3373878772}]},"public_addr":"172.21.15.59:6801/3373878772","cluster_addr":"172.21.15.59:6803/3373878772","heartbeat_back_addr":"172.21.15.59:6807/3373878772","heartbeat_front_addr":"172.21.15.59:6805/3373878772","state":["exists","up"]},{"osd":1,"uuid":"e8d882aa-52e1-4c04-9e1e-c44ef9204a5f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6824","nonce":648258563},{"type":"v1","addr":"172.21.15.59:6825","nonce":648258563}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6826","nonce":648258563},{"type":"v1","addr":"172.21.15.59:6827","nonce":648258563}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6830","nonce":648258563},{"type":"v1","addr":"172.21.15.59:6831","nonce":648258563}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6828","nonce":648258563},{"type":"v1","addr":"172.21.15.59:6829","nonce":648258563}]},"public_addr":"172.21.15.59:6825/648258563","cluster_addr":"172.21.15.59:6827/648258563","heartbeat_back_addr":"172.21.15.59:6831/648258563","heartbeat_front_addr":"172.21.15.59:6829/648258563","state":["exists","up"]},{"osd":2,"uuid":"69eebfa0-695b-4b21-90b9-8a3537168a01","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6808","nonce":1840430654},{"type":"v1","addr":"172.21.15.59:6809","nonce":1840430654}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6810","nonce":1840430654},{"type":"v1","addr":"172.21.15.59:6811","nonce":1840430654}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6814","nonce":1840430654},{"type":"v1","addr":"172.21.15.59:6815","nonce":1840430654}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6812","nonce":1840430654},{"type":"v1","addr":"172.21.15.59:6813","nonce":1840430654}]},"public_addr":"172.21.15.59:6809/1840430654","cluster_addr":"172.21.15.59:6811/1840430654","heartbeat_back_addr":"172.21.15.59:6815/1840430654","heartbeat_front_addr":"172.21.15.59:6813/1840430654","state":["exists","up"]},{"osd":3,"uuid":"96fd77fc-f911-46ff-9445-6f0be2daa8f4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6816","nonce":2504025850},{"type":"v1","addr":"172.21.15.59:6817","nonce":2504025850}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6818","nonce":2504025850},{"type":"v1","addr":"172.21.15.59:6819","nonce":2504025850}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6822","nonce":2504025850},{"type":"v1","addr":"172.21.15.59:6823","nonce":2504025850}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6820","nonce":2504025850},{"type":"v1","addr":"172.21.15.59:6821","nonce":2504025850}]},"public_addr":"172.21.15.59:6817/2504025850","cluster_addr":"172.21.15.59:6819/2504025850","heartbeat_back_addr":"172.21.15.59:6823/2504025850","heartbeat_front_addr":"172.21.15.59:6821/2504025850","state":["exists","up"]},{"osd":4,"uuid":"7ac29fca-b5b2-4c95-9d69-2f164cbbc708","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6816","nonce":2382463431},{"type":"v1","addr":"172.21.15.76:6818","nonce":2382463431}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6820","nonce":2382463431},{"type":"v1","addr":"172.21.15.76:6822","nonce":2382463431}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6828","nonce":2382463431},{"type":"v1","addr":"172.21.15.76:6830","nonce":2382463431}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6824","nonce":2382463431},{"type":"v1","addr":"172.21.15.76:6826","nonce":2382463431}]},"public_addr":"172.21.15.76:6818/2382463431","cluster_addr":"172.21.15.76:6822/2382463431","heartbeat_back_addr":"172.21.15.76:6830/2382463431","heartbeat_front_addr":"172.21.15.76:6826/2382463431","state":["exists","up"]},{"osd":5,"uuid":"485bf711-3851-4238-a4f4-51d52e0a9c58","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6800","nonce":329334308},{"type":"v1","addr":"172.21.15.76:6802","nonce":329334308}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6805","nonce":329334308},{"type":"v1","addr":"172.21.15.76:6807","nonce":329334308}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6814","nonce":329334308},{"type":"v1","addr":"172.21.15.76:6815","nonce":329334308}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6810","nonce":329334308},{"type":"v1","addr":"172.21.15.76:6812","nonce":329334308}]},"public_addr":"172.21.15.76:6802/329334308","cluster_addr":"172.21.15.76:6807/329334308","heartbeat_back_addr":"172.21.15.76:6815/329334308","heartbeat_front_addr":"172.21.15.76:6812/329334308","state":["exists","up"]},{"osd":6,"uuid":"72b3f2fc-1e4a-4e32-968d-003647a0286f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6801","nonce":3946579410},{"type":"v1","addr":"172.21.15.76:6803","nonce":3946579410}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6804","nonce":3946579410},{"type":"v1","addr":"172.21.15.76:6806","nonce":3946579410}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6811","nonce":3946579410},{"type":"v1","addr":"172.21.15.76:6813","nonce":3946579410}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6808","nonce":3946579410},{"type":"v1","addr":"172.21.15.76:6809","nonce":3946579410}]},"public_addr":"172.21.15.76:6803/3946579410","cluster_addr":"172.21.15.76:6806/3946579410","heartbeat_back_addr":"172.21.15.76:6813/3946579410","heartbeat_front_addr":"172.21.15.76:6809/3946579410","state":["exists","up"]},{"osd":7,"uuid":"1ddceb90-6c5d-4d1a-ad56-99cb877f8c70","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6817","nonce":841761538},{"type":"v1","addr":"172.21.15.76:6819","nonce":841761538}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6821","nonce":841761538},{"type":"v1","addr":"172.21.15.76:6823","nonce":841761538}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6829","nonce":841761538},{"type":"v1","addr":"172.21.15.76:6831","nonce":841761538}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6825","nonce":841761538},{"type":"v1","addr":"172.21.15.76:6827","nonce":841761538}]},"public_addr":"172.21.15.76:6819/841761538","cluster_addr":"172.21.15.76:6823/841761538","heartbeat_back_addr":"172.21.15.76:6831/841761538","heartbeat_front_addr":"172.21.15.76:6827/841761538","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T07:56:14.294838+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T07:56:14.270779+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T07:56:14.231809+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T07:56:14.260983+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T07:56:14.302997+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T07:56:14.258600+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T07:56:14.251413+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T07:56:14.304727+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-24T07:59:34.078 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:34.076+0000 7f57d9ea9700 1 -- 172.21.15.59:0/2189591829 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f57c006b470 msgr2=0x7f57c006d930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:34.078 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:34.076+0000 7f57d9ea9700 1 --2- 172.21.15.59:0/2189591829 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f57c006b470 0x7f57c006d930 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f57c80090b0 tx=0x7f57c801e040 comp rx=0 tx=0).stop 2023-11-24T07:59:34.078 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:34.076+0000 7f57d9ea9700 1 -- 172.21.15.59:0/2189591829 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f57d41e3d00 msgr2=0x7f57d40b1a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:34.078 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:34.076+0000 7f57d9ea9700 1 --2- 172.21.15.59:0/2189591829 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f57d41e3d00 0x7f57d40b1a70 secure :-1 s=READY pgs=930 cs=0 l=1 rev1=1 crypto rx=0x7f57c4007340 tx=0x7f57c4007700 comp rx=0 tx=0).stop 2023-11-24T07:59:34.079 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:34.076+0000 7f57d9ea9700 1 -- 172.21.15.59:0/2189591829 shutdown_connections 2023-11-24T07:59:34.079 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:34.076+0000 7f57d9ea9700 1 --2- 172.21.15.59:0/2189591829 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f57c006b470 0x7f57c006d930 unknown :-1 s=CLOSED pgs=432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:34.079 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:34.076+0000 7f57d9ea9700 1 --2- 172.21.15.59:0/2189591829 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f57d40b1fb0 0x7f57d4140be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:34.079 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:34.076+0000 7f57d9ea9700 1 --2- 172.21.15.59:0/2189591829 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f57d415cec0 0x7f57d4142c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:34.079 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:34.076+0000 7f57d9ea9700 1 --2- 172.21.15.59:0/2189591829 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f57d41e3d00 0x7f57d40b1a70 unknown :-1 s=CLOSED pgs=930 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:34.079 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:34.076+0000 7f57d9ea9700 1 -- 172.21.15.59:0/2189591829 >> 172.21.15.59:0/2189591829 conn(0x7f57d40b27d0 msgr2=0x7f57d40b0150 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:59:34.079 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:34.076+0000 7f57d9ea9700 1 -- 172.21.15.59:0/2189591829 shutdown_connections 2023-11-24T07:59:34.079 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:34.076+0000 7f57d9ea9700 1 -- 172.21.15.59:0/2189591829 wait complete. 2023-11-24T07:59:34.091 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-24T07:56:18.352823+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 10, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-24T07:59:11.900882+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '73', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 11, 'pool_name': 'cephfs_data', 'create_time': '2023-11-24T07:59:13.025749+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '74', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-24T07:59:34.092 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-24T07:59:34.220 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:34.218+0000 7f85f4b44700 1 Processor -- start 2023-11-24T07:59:34.220 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:34.218+0000 7f85f4b44700 1 -- start start 2023-11-24T07:59:34.221 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:34.219+0000 7f85f4b44700 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f85f0143ae0 0x7f85f00b51b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:34.222 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:34.219+0000 7f85f4b44700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f85f01440f0 0x7f85f0071bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:34.222 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:34.219+0000 7f85f4b44700 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f85f00720f0 0x7f85f015e8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:34.222 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:34.219+0000 7f85f4b44700 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f85f00b40a0 con 0x7f85f01440f0 2023-11-24T07:59:34.222 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:34.219+0000 7f85f4b44700 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f85f0072580 con 0x7f85f0143ae0 2023-11-24T07:59:34.222 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:34.219+0000 7f85f4b44700 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f85f0143ec0 con 0x7f85f00720f0 2023-11-24T07:59:34.222 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:34.219+0000 7f85edd9b700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f85f01440f0 0x7f85f0071bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:34.222 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:34.219+0000 7f85edd9b700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f85f01440f0 0x7f85f0071bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:55276/0 (socket says 172.21.15.59:55276) 2023-11-24T07:59:34.223 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:34.219+0000 7f85edd9b700 1 -- 172.21.15.59:0/4140084331 learned_addr learned my addr 172.21.15.59:0/4140084331 (peer_addr_for_me v2:172.21.15.59:0/0) 2023-11-24T07:59:34.223 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:34.219+0000 7f85ee59c700 1 --2- 172.21.15.59:0/4140084331 >> v2:172.21.15.76:3300/0 conn(0x7f85f0143ae0 0x7f85f00b51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:34.223 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:34.219+0000 7f85eed9d700 1 --2- 172.21.15.59:0/4140084331 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f85f00720f0 0x7f85f015e8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:34.223 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:34.220+0000 7f85edd9b700 1 -- 172.21.15.59:0/4140084331 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f85f00720f0 msgr2=0x7f85f015e8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T07:59:34.223 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:34.220+0000 7f85edd9b700 1 --2- 172.21.15.59:0/4140084331 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f85f00720f0 0x7f85f015e8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:34.223 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:34.220+0000 7f85edd9b700 1 -- 172.21.15.59:0/4140084331 >> v2:172.21.15.76:3300/0 conn(0x7f85f0143ae0 msgr2=0x7f85f00b51b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T07:59:34.223 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:34.220+0000 7f85edd9b700 1 --2- 172.21.15.59:0/4140084331 >> v2:172.21.15.76:3300/0 conn(0x7f85f0143ae0 0x7f85f00b51b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:34.224 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:34.220+0000 7f85edd9b700 1 -- 172.21.15.59:0/4140084331 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f85f00b3ae0 con 0x7f85f01440f0 2023-11-24T07:59:34.224 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:34.220+0000 7f85edd9b700 1 --2- 172.21.15.59:0/4140084331 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f85f01440f0 0x7f85f0071bb0 secure :-1 s=READY pgs=931 cs=0 l=1 rev1=1 crypto rx=0x7f85f006d2e0 tx=0x7f85e0009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bd2f02e73e161caa server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:59:34.224 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:34.220+0000 7f85ed59a700 1 -- 172.21.15.59:0/4140084331 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85e0011830 con 0x7f85f01440f0 2023-11-24T07:59:34.224 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:34.220+0000 7f85ed59a700 1 -- 172.21.15.59:0/4140084331 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f85e0011990 con 0x7f85f01440f0 2023-11-24T07:59:34.224 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:34.220+0000 7f85ed59a700 1 -- 172.21.15.59:0/4140084331 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85e0011d20 con 0x7f85f01440f0 2023-11-24T07:59:34.230 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:34.220+0000 7f85f4b44700 1 -- 172.21.15.59:0/4140084331 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f85f01440f0 msgr2=0x7f85f0071bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:34.230 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:34.220+0000 7f85f4b44700 1 --2- 172.21.15.59:0/4140084331 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f85f01440f0 0x7f85f0071bb0 secure :-1 s=READY pgs=931 cs=0 l=1 rev1=1 crypto rx=0x7f85f006d2e0 tx=0x7f85e0009bd0 comp rx=0 tx=0).stop 2023-11-24T07:59:34.231 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:34.221+0000 7f85f4b44700 1 -- 172.21.15.59:0/4140084331 shutdown_connections 2023-11-24T07:59:34.231 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:34.221+0000 7f85f4b44700 1 --2- 172.21.15.59:0/4140084331 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f85f00720f0 0x7f85f015e8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:34.233 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:34.221+0000 7f85f4b44700 1 --2- 172.21.15.59:0/4140084331 >> v2:172.21.15.76:3300/0 conn(0x7f85f0143ae0 0x7f85f00b51b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:34.233 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:34.221+0000 7f85f4b44700 1 --2- 172.21.15.59:0/4140084331 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f85f01440f0 0x7f85f0071bb0 unknown :-1 s=CLOSED pgs=931 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:34.234 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:34.221+0000 7f85f4b44700 1 -- 172.21.15.59:0/4140084331 >> 172.21.15.59:0/4140084331 conn(0x7f85f00b2740 msgr2=0x7f85f00b2b20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:59:34.234 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:34.221+0000 7f85f4b44700 1 -- 172.21.15.59:0/4140084331 shutdown_connections 2023-11-24T07:59:34.234 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:34.221+0000 7f85f4b44700 1 -- 172.21.15.59:0/4140084331 wait complete. 2023-11-24T07:59:34.234 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:34.222+0000 7f85f4b44700 1 Processor -- start 2023-11-24T07:59:34.234 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:34.222+0000 7f85f4b44700 1 -- start start 2023-11-24T07:59:34.234 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:34.223+0000 7f85f4b44700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f85f00720f0 0x7f85f0157c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:34.234 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:34.223+0000 7f85f4b44700 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f85f0143ae0 0x7f85f01403a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:34.235 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:34.223+0000 7f85f4b44700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f85f01440f0 0x7f85f0159490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:34.235 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:34.223+0000 7f85f4b44700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85f015aeb0 con 0x7f85f00720f0 2023-11-24T07:59:34.235 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:34.223+0000 7f85f4b44700 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85f0159e10 con 0x7f85f01440f0 2023-11-24T07:59:34.235 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:34.223+0000 7f85f4b44700 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f85f015a0f0 con 0x7f85f0143ae0 2023-11-24T07:59:34.235 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:34.223+0000 7f85ee59c700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f85f00720f0 0x7f85f0157c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:34.235 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:34.223+0000 7f85ee59c700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f85f00720f0 0x7f85f0157c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:55282/0 (socket says 172.21.15.59:55282) 2023-11-24T07:59:34.236 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:34.223+0000 7f85ee59c700 1 -- 172.21.15.59:0/1659145286 learned_addr learned my addr 172.21.15.59:0/1659145286 (peer_addr_for_me v2:172.21.15.59:0/0) 2023-11-24T07:59:34.236 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:34.223+0000 7f85edd9b700 1 --2- 172.21.15.59:0/1659145286 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f85f0143ae0 0x7f85f01403a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:34.236 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:34.223+0000 7f85eed9d700 1 --2- 172.21.15.59:0/1659145286 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f85f01440f0 0x7f85f0159490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:34.236 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:34.223+0000 7f85ee59c700 1 -- 172.21.15.59:0/1659145286 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f85f0143ae0 msgr2=0x7f85f01403a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:34.236 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:34.223+0000 7f85ee59c700 1 --2- 172.21.15.59:0/1659145286 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f85f0143ae0 0x7f85f01403a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:34.236 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:34.223+0000 7f85ee59c700 1 -- 172.21.15.59:0/1659145286 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f85f01440f0 msgr2=0x7f85f0159490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:34.236 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:34.223+0000 7f85ee59c700 1 --2- 172.21.15.59:0/1659145286 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f85f01440f0 0x7f85f0159490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:34.236 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:34.223+0000 7f85ee59c700 1 -- 172.21.15.59:0/1659145286 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f85e0009550 con 0x7f85f00720f0 2023-11-24T07:59:34.237 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:34.223+0000 7f85ee59c700 1 --2- 172.21.15.59:0/1659145286 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f85f00720f0 0x7f85f0157c90 secure :-1 s=READY pgs=932 cs=0 l=1 rev1=1 crypto rx=0x7f85e400e870 tx=0x7f85e400ec30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:59:34.237 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:34.223+0000 7f85deffd700 1 -- 172.21.15.59:0/1659145286 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85e4009590 con 0x7f85f00720f0 2023-11-24T07:59:34.237 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:34.223+0000 7f85deffd700 1 -- 172.21.15.59:0/1659145286 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f85e400ee70 con 0x7f85f00720f0 2023-11-24T07:59:34.237 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:34.223+0000 7f85f4b44700 1 -- 172.21.15.59:0/1659145286 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f85f015a3d0 con 0x7f85f00720f0 2023-11-24T07:59:34.238 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:34.223+0000 7f85deffd700 1 -- 172.21.15.59:0/1659145286 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85e400ca50 con 0x7f85f00720f0 2023-11-24T07:59:34.238 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:34.224+0000 7f85f4b44700 1 -- 172.21.15.59:0/1659145286 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f85f015a990 con 0x7f85f00720f0 2023-11-24T07:59:34.238 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:34.224+0000 7f85f4b44700 1 -- 172.21.15.59:0/1659145286 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f85f0075480 con 0x7f85f00720f0 2023-11-24T07:59:34.238 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:34.225+0000 7f85deffd700 1 -- 172.21.15.59:0/1659145286 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 11) v1 ==== 89741+0+0 (secure 0 0 0) 0x7f85e400cbb0 con 0x7f85f00720f0 2023-11-24T07:59:34.238 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:34.225+0000 7f85deffd700 1 --2- 172.21.15.59:0/1659145286 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f85d006b470 0x7f85d006d930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:34.239 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:34.225+0000 7f85deffd700 1 -- 172.21.15.59:0/1659145286 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f85e4089fa0 con 0x7f85f00720f0 2023-11-24T07:59:34.239 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:34.226+0000 7f85edd9b700 1 --2- 172.21.15.59:0/1659145286 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f85d006b470 0x7f85d006d930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:34.239 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:34.226+0000 7f85edd9b700 1 --2- 172.21.15.59:0/1659145286 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f85d006b470 0x7f85d006d930 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f85e0009c10 tx=0x7f85e0008030 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:59:34.240 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:34.230+0000 7f85deffd700 1 -- 172.21.15.59:0/1659145286 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f85e40547b0 con 0x7f85f00720f0 2023-11-24T07:59:34.435 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:34.433+0000 7f85f4b44700 1 -- 172.21.15.59:0/1659145286 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f85f0144830 con 0x7f85f00720f0 2023-11-24T07:59:34.435 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:34.434+0000 7f85deffd700 1 -- 172.21.15.59:0/1659145286 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v75) v1 ==== 93+0+10 (secure 0 0 0) 0x7f85e400ce60 con 0x7f85f00720f0 2023-11-24T07:59:34.435 INFO:teuthology.orchestra.run.smithi059.stdout:pg_num: 1 2023-11-24T07:59:34.438 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:34.436+0000 7f85f4b44700 1 -- 172.21.15.59:0/1659145286 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f85d006b470 msgr2=0x7f85d006d930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:34.438 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:34.436+0000 7f85f4b44700 1 --2- 172.21.15.59:0/1659145286 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f85d006b470 0x7f85d006d930 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f85e0009c10 tx=0x7f85e0008030 comp rx=0 tx=0).stop 2023-11-24T07:59:34.438 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:34.436+0000 7f85f4b44700 1 -- 172.21.15.59:0/1659145286 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f85f00720f0 msgr2=0x7f85f0157c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:34.438 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:34.436+0000 7f85f4b44700 1 --2- 172.21.15.59:0/1659145286 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f85f00720f0 0x7f85f0157c90 secure :-1 s=READY pgs=932 cs=0 l=1 rev1=1 crypto rx=0x7f85e400e870 tx=0x7f85e400ec30 comp rx=0 tx=0).stop 2023-11-24T07:59:34.438 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:34.436+0000 7f85f4b44700 1 -- 172.21.15.59:0/1659145286 shutdown_connections 2023-11-24T07:59:34.439 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:34.436+0000 7f85f4b44700 1 --2- 172.21.15.59:0/1659145286 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f85d006b470 0x7f85d006d930 unknown :-1 s=CLOSED pgs=433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:34.439 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:34.436+0000 7f85f4b44700 1 --2- 172.21.15.59:0/1659145286 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f85f01440f0 0x7f85f0159490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:34.439 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:34.436+0000 7f85f4b44700 1 --2- 172.21.15.59:0/1659145286 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f85f00720f0 0x7f85f0157c90 unknown :-1 s=CLOSED pgs=932 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:34.439 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:34.436+0000 7f85f4b44700 1 --2- 172.21.15.59:0/1659145286 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f85f0143ae0 0x7f85f01403a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:34.439 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:34.436+0000 7f85f4b44700 1 -- 172.21.15.59:0/1659145286 >> 172.21.15.59:0/1659145286 conn(0x7f85f00b2740 msgr2=0x7f85f0141190 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:59:34.439 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:34.436+0000 7f85f4b44700 1 -- 172.21.15.59:0/1659145286 shutdown_connections 2023-11-24T07:59:34.439 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:34.436+0000 7f85f4b44700 1 -- 172.21.15.59:0/1659145286 wait complete. 2023-11-24T07:59:34.450 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-24T07:59:34.582 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:34.580+0000 7fb4181e2700 1 Processor -- start 2023-11-24T07:59:34.582 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:34.580+0000 7fb4181e2700 1 -- start start 2023-11-24T07:59:34.592 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:34.581+0000 7fb4181e2700 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fb4100b4240 0x7fb410160330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:34.592 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:34.581+0000 7fb4181e2700 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fb4100ce140 0x7fb4100ce590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:34.593 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:34.581+0000 7fb4181e2700 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb4100b14c0 con 0x7fb41015f410 2023-11-24T07:59:34.593 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:34.581+0000 7fb4181e2700 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb4100b46f0 con 0x7fb4100b4240 2023-11-24T07:59:34.593 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:34.581+0000 7fb4181e2700 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb4100b3070 con 0x7fb4100ce140 2023-11-24T07:59:34.593 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:34.581+0000 7fb41677f700 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fb4100ce140 0x7fb4100ce590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:34.594 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:34.581+0000 7fb41577d700 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fb4100b4240 0x7fb410160330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:34.594 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:34.581+0000 7fb41577d700 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fb4100b4240 0x7fb410160330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.59:42490/0 (socket says 172.21.15.59:42490) 2023-11-24T07:59:34.594 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:34.581+0000 7fb41677f700 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fb4100ce140 0x7fb4100ce590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3301/0 says I am v2:172.21.15.59:54734/0 (socket says 172.21.15.59:54734) 2023-11-24T07:59:34.594 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:34.581+0000 7fb41577d700 1 -- 172.21.15.59:0/871596964 learned_addr learned my addr 172.21.15.59:0/871596964 (peer_addr_for_me v2:172.21.15.59:0/0) 2023-11-24T07:59:34.594 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:34.582+0000 7fb414f7c700 1 -- 172.21.15.59:0/871596964 <== mon.1 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2236842413 0 0) 0x7fb4100b14c0 con 0x7fb41015f410 2023-11-24T07:59:34.595 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:34.582+0000 7fb414f7c700 1 -- 172.21.15.59:0/871596964 --> v1:172.21.15.76:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb3fc0035b0 con 0x7fb41015f410 2023-11-24T07:59:34.595 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:34.582+0000 7fb41677f700 1 -- 172.21.15.59:0/871596964 >> v2:172.21.15.76:3300/0 conn(0x7fb4100b4240 msgr2=0x7fb410160330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T07:59:34.595 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:34.582+0000 7fb41677f700 1 --2- 172.21.15.59:0/871596964 >> v2:172.21.15.76:3300/0 conn(0x7fb4100b4240 0x7fb410160330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:34.595 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:34.582+0000 7fb41677f700 1 -- 172.21.15.59:0/871596964 >> v1:172.21.15.76:6789/0 conn(0x7fb41015f410 legacy=0x7fb41015fc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:34.595 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:34.582+0000 7fb41677f700 1 -- 172.21.15.59:0/871596964 --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb4100b3270 con 0x7fb4100ce140 2023-11-24T07:59:34.595 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:34.582+0000 7fb41577d700 1 --2- 172.21.15.59:0/871596964 >> v2:172.21.15.76:3300/0 conn(0x7fb4100b4240 0x7fb410160330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-24T07:59:34.595 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:34.582+0000 7fb41677f700 1 --2- 172.21.15.59:0/871596964 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fb4100ce140 0x7fb4100ce590 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fb408009d90 tx=0x7fb40800b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=73968aa8f27a4ea9 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:59:34.596 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:34.582+0000 7fb414f7c700 1 -- 172.21.15.59:0/871596964 <== mon.2 v2:172.21.15.76:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb40800e070 con 0x7fb4100ce140 2023-11-24T07:59:34.596 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:34.582+0000 7fb414f7c700 1 -- 172.21.15.59:0/871596964 <== mon.2 v2:172.21.15.76:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb40800bb20 con 0x7fb4100ce140 2023-11-24T07:59:34.596 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:34.582+0000 7fb414f7c700 1 -- 172.21.15.59:0/871596964 <== mon.2 v2:172.21.15.76:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb40800be90 con 0x7fb4100ce140 2023-11-24T07:59:34.596 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:34.583+0000 7fb4181e2700 1 -- 172.21.15.59:0/871596964 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fb4100ce140 msgr2=0x7fb4100ce590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:34.596 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:34.583+0000 7fb4181e2700 1 --2- 172.21.15.59:0/871596964 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fb4100ce140 0x7fb4100ce590 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fb408009d90 tx=0x7fb40800b330 comp rx=0 tx=0).stop 2023-11-24T07:59:34.597 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:34.583+0000 7fb4181e2700 1 -- 172.21.15.59:0/871596964 shutdown_connections 2023-11-24T07:59:34.597 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:34.583+0000 7fb4181e2700 1 --2- 172.21.15.59:0/871596964 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fb4100ce140 0x7fb4100ce590 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:34.597 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:34.583+0000 7fb4181e2700 1 --2- 172.21.15.59:0/871596964 >> v2:172.21.15.76:3300/0 conn(0x7fb4100b4240 0x7fb410160330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:34.597 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:34.583+0000 7fb4181e2700 1 -- 172.21.15.59:0/871596964 >> 172.21.15.59:0/871596964 conn(0x7fb4100b3720 msgr2=0x7fb4100b3b30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:59:34.597 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:34.583+0000 7fb4181e2700 1 -- 172.21.15.59:0/871596964 shutdown_connections 2023-11-24T07:59:34.597 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:34.583+0000 7fb4181e2700 1 -- 172.21.15.59:0/871596964 wait complete. 2023-11-24T07:59:34.597 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:34.584+0000 7fb4181e2700 1 Processor -- start 2023-11-24T07:59:34.597 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:34.584+0000 7fb4181e2700 1 -- start start 2023-11-24T07:59:34.598 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:34.585+0000 7fb4181e2700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb4100b4240 0x7fb41006d1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:34.598 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:34.585+0000 7fb4181e2700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb4100ce140 0x7fb4100b58c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:34.598 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:34.585+0000 7fb4181e2700 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fb41015f410 0x7fb4100b5e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:34.598 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:34.585+0000 7fb4181e2700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb41006c180 con 0x7fb4100b4240 2023-11-24T07:59:34.598 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:34.585+0000 7fb4181e2700 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb41006c420 con 0x7fb4100ce140 2023-11-24T07:59:34.598 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:34.585+0000 7fb4181e2700 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb41006c700 con 0x7fb41015f410 2023-11-24T07:59:34.598 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:34.585+0000 7fb415f7e700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb4100b4240 0x7fb41006d1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:34.599 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:34.585+0000 7fb415f7e700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb4100b4240 0x7fb41006d1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:55296/0 (socket says 172.21.15.59:55296) 2023-11-24T07:59:34.599 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:34.585+0000 7fb415f7e700 1 -- 172.21.15.59:0/13887033 learned_addr learned my addr 172.21.15.59:0/13887033 (peer_addr_for_me v2:172.21.15.59:0/0) 2023-11-24T07:59:34.599 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:34.585+0000 7fb41577d700 1 --2- 172.21.15.59:0/13887033 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb4100ce140 0x7fb4100b58c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:34.599 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:34.585+0000 7fb41677f700 1 --2- 172.21.15.59:0/13887033 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fb41015f410 0x7fb4100b5e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:34.599 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:34.585+0000 7fb415f7e700 1 -- 172.21.15.59:0/13887033 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fb41015f410 msgr2=0x7fb4100b5e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:34.599 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:34.585+0000 7fb415f7e700 1 --2- 172.21.15.59:0/13887033 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fb41015f410 0x7fb4100b5e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:34.599 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:34.585+0000 7fb415f7e700 1 -- 172.21.15.59:0/13887033 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb4100ce140 msgr2=0x7fb4100b58c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:34.599 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:34.585+0000 7fb415f7e700 1 --2- 172.21.15.59:0/13887033 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb4100ce140 0x7fb4100b58c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:34.600 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:34.585+0000 7fb415f7e700 1 -- 172.21.15.59:0/13887033 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb404009550 con 0x7fb4100b4240 2023-11-24T07:59:34.600 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:34.585+0000 7fb41677f700 1 --2- 172.21.15.59:0/13887033 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fb41015f410 0x7fb4100b5e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-24T07:59:34.600 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:34.585+0000 7fb41577d700 1 --2- 172.21.15.59:0/13887033 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb4100ce140 0x7fb4100b58c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-24T07:59:34.600 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:34.586+0000 7fb415f7e700 1 --2- 172.21.15.59:0/13887033 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb4100b4240 0x7fb41006d1e0 secure :-1 s=READY pgs=933 cs=0 l=1 rev1=1 crypto rx=0x7fb40c00c500 tx=0x7fb40c00c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:59:34.600 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:34.586+0000 7fb4027fc700 1 -- 172.21.15.59:0/13887033 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb40c0052d0 con 0x7fb4100b4240 2023-11-24T07:59:34.600 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:34.586+0000 7fb4027fc700 1 -- 172.21.15.59:0/13887033 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb40c005430 con 0x7fb4100b4240 2023-11-24T07:59:34.600 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:34.586+0000 7fb4181e2700 1 -- 172.21.15.59:0/13887033 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb408009a70 con 0x7fb4100b4240 2023-11-24T07:59:34.600 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:34.586+0000 7fb4027fc700 1 -- 172.21.15.59:0/13887033 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb40c0038c0 con 0x7fb4100b4240 2023-11-24T07:59:34.601 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:34.586+0000 7fb4181e2700 1 -- 172.21.15.59:0/13887033 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb41006ccc0 con 0x7fb4100b4240 2023-11-24T07:59:34.601 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:34.587+0000 7fb4181e2700 1 -- 172.21.15.59:0/13887033 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb41006ce80 con 0x7fb4100b4240 2023-11-24T07:59:34.601 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:34.587+0000 7fb4027fc700 1 -- 172.21.15.59:0/13887033 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 11) v1 ==== 89741+0+0 (secure 0 0 0) 0x7fb40c003a20 con 0x7fb4100b4240 2023-11-24T07:59:34.601 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:34.588+0000 7fb4027fc700 1 --2- 172.21.15.59:0/13887033 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7fb3f406b470 0x7fb3f406d930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:34.601 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:34.588+0000 7fb4027fc700 1 -- 172.21.15.59:0/13887033 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(75..75 src has 1..75) v4 ==== 5765+0+0 (secure 0 0 0) 0x7fb40c089570 con 0x7fb4100b4240 2023-11-24T07:59:34.601 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:34.588+0000 7fb41577d700 1 --2- 172.21.15.59:0/13887033 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7fb3f406b470 0x7fb3f406d930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:34.601 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:34.588+0000 7fb41577d700 1 --2- 172.21.15.59:0/13887033 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7fb3f406b470 0x7fb3f406d930 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7fb41006e1d0 tx=0x7fb404008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:59:34.602 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:34.592+0000 7fb4027fc700 1 -- 172.21.15.59:0/13887033 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fb40c053d70 con 0x7fb4100b4240 2023-11-24T07:59:34.796 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:34.794+0000 7fb4181e2700 1 -- 172.21.15.59:0/13887033 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fb410075480 con 0x7fb4100b4240 2023-11-24T07:59:34.797 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:34.795+0000 7fb4027fc700 1 -- 172.21.15.59:0/13887033 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v75) v1 ==== 104+0+11 (secure 0 0 0) 0x7fb40c057390 con 0x7fb4100b4240 2023-11-24T07:59:34.797 INFO:teuthology.orchestra.run.smithi059.stdout:pg_num: 32 2023-11-24T07:59:34.799 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:34.797+0000 7fb4181e2700 1 -- 172.21.15.59:0/13887033 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7fb3f406b470 msgr2=0x7fb3f406d930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:34.799 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:34.797+0000 7fb4181e2700 1 --2- 172.21.15.59:0/13887033 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7fb3f406b470 0x7fb3f406d930 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7fb41006e1d0 tx=0x7fb404008040 comp rx=0 tx=0).stop 2023-11-24T07:59:34.799 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:34.797+0000 7fb4181e2700 1 -- 172.21.15.59:0/13887033 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb4100b4240 msgr2=0x7fb41006d1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:34.799 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:34.797+0000 7fb4181e2700 1 --2- 172.21.15.59:0/13887033 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb4100b4240 0x7fb41006d1e0 secure :-1 s=READY pgs=933 cs=0 l=1 rev1=1 crypto rx=0x7fb40c00c500 tx=0x7fb40c00c8c0 comp rx=0 tx=0).stop 2023-11-24T07:59:34.799 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:34.797+0000 7fb4181e2700 1 -- 172.21.15.59:0/13887033 shutdown_connections 2023-11-24T07:59:34.799 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:34.797+0000 7fb4181e2700 1 --2- 172.21.15.59:0/13887033 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7fb3f406b470 0x7fb3f406d930 unknown :-1 s=CLOSED pgs=434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:34.800 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:34.797+0000 7fb4181e2700 1 --2- 172.21.15.59:0/13887033 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fb41015f410 0x7fb4100b5e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:34.800 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:34.797+0000 7fb4181e2700 1 --2- 172.21.15.59:0/13887033 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb4100b4240 0x7fb41006d1e0 unknown :-1 s=CLOSED pgs=933 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:34.800 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:34.797+0000 7fb4181e2700 1 --2- 172.21.15.59:0/13887033 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb4100ce140 0x7fb4100b58c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:34.800 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:34.797+0000 7fb4181e2700 1 -- 172.21.15.59:0/13887033 >> 172.21.15.59:0/13887033 conn(0x7fb4100b3720 msgr2=0x7fb4100b1e70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:59:34.800 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:34.797+0000 7fb4181e2700 1 -- 172.21.15.59:0/13887033 shutdown_connections 2023-11-24T07:59:34.800 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:34.797+0000 7fb4181e2700 1 -- 172.21.15.59:0/13887033 wait complete. 2023-11-24T07:59:34.811 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-24T07:59:34.939 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:34.937+0000 7f191fda1700 1 Processor -- start 2023-11-24T07:59:34.939 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:34.937+0000 7f191fda1700 1 -- start start 2023-11-24T07:59:34.941 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:34.938+0000 7f191fda1700 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f19180722c0 0x7f19180b51b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:34.941 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:34.938+0000 7f191fda1700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f19181440f0 0x7f191806e510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:34.942 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:34.938+0000 7f191fda1700 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1918143ec0 con 0x7f1918143ae0 2023-11-24T07:59:34.942 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:34.938+0000 7f191fda1700 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f19180b40a0 con 0x7f19181440f0 2023-11-24T07:59:34.942 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:34.938+0000 7f191fda1700 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f19180b3ae0 con 0x7f19180722c0 2023-11-24T07:59:34.942 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:34.938+0000 7f191e33e700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f19181440f0 0x7f191806e510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:34.942 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:34.938+0000 7f191db3d700 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7f1918143ae0 0x7f1918071bb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:58778/0 (socket says 172.21.15.59:58778) 2023-11-24T07:59:34.942 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:34.938+0000 7f191e33e700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f19181440f0 0x7f191806e510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:55308/0 (socket says 172.21.15.59:55308) 2023-11-24T07:59:34.942 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:34.938+0000 7f191db3d700 1 -- 172.21.15.59:0/3342429877 learned_addr learned my addr 172.21.15.59:0/3342429877 (peer_addr_for_me v1:172.21.15.59:0/0) 2023-11-24T07:59:34.942 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:34.938+0000 7f191d33c700 1 --2- 172.21.15.59:0/3342429877 >> v2:172.21.15.76:3300/0 conn(0x7f19180722c0 0x7f19180b51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:34.942 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:34.938+0000 7f191e33e700 1 -- 172.21.15.59:0/3342429877 >> v2:172.21.15.76:3300/0 conn(0x7f19180722c0 msgr2=0x7f19180b51b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T07:59:34.943 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:34.938+0000 7f191e33e700 1 --2- 172.21.15.59:0/3342429877 >> v2:172.21.15.76:3300/0 conn(0x7f19180722c0 0x7f19180b51b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:34.943 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:34.938+0000 7f191e33e700 1 -- 172.21.15.59:0/3342429877 >> v1:172.21.15.59:6789/0 conn(0x7f1918143ae0 legacy=0x7f1918071bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:34.943 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:34.939+0000 7f191e33e700 1 -- 172.21.15.59:0/3342429877 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f19180b3dc0 con 0x7f19181440f0 2023-11-24T07:59:34.943 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:34.939+0000 7f191e33e700 1 --2- 172.21.15.59:0/3342429877 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f19181440f0 0x7f191806e510 secure :-1 s=READY pgs=935 cs=0 l=1 rev1=1 crypto rx=0x7f19180d1b00 tx=0x7f191000d330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c760f7b7c0cdde8b server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:59:34.943 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:34.939+0000 7f191cb3b700 1 -- 172.21.15.59:0/3342429877 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1910010070 con 0x7f19181440f0 2023-11-24T07:59:34.943 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:34.939+0000 7f191cb3b700 1 -- 172.21.15.59:0/3342429877 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f191000b2e0 con 0x7f19181440f0 2023-11-24T07:59:34.943 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:34.939+0000 7f191cb3b700 1 -- 172.21.15.59:0/3342429877 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f191000dc10 con 0x7f19181440f0 2023-11-24T07:59:34.943 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:34.939+0000 7f191fda1700 1 -- 172.21.15.59:0/3342429877 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f19181440f0 msgr2=0x7f191806e510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:34.943 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:34.939+0000 7f191fda1700 1 --2- 172.21.15.59:0/3342429877 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f19181440f0 0x7f191806e510 secure :-1 s=READY pgs=935 cs=0 l=1 rev1=1 crypto rx=0x7f19180d1b00 tx=0x7f191000d330 comp rx=0 tx=0).stop 2023-11-24T07:59:34.944 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:34.939+0000 7f191fda1700 1 -- 172.21.15.59:0/3342429877 shutdown_connections 2023-11-24T07:59:34.944 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:34.939+0000 7f191fda1700 1 --2- 172.21.15.59:0/3342429877 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f19181440f0 0x7f191806e510 unknown :-1 s=CLOSED pgs=935 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:34.944 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:34.939+0000 7f191fda1700 1 --2- 172.21.15.59:0/3342429877 >> v2:172.21.15.76:3300/0 conn(0x7f19180722c0 0x7f19180b51b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:34.944 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:34.939+0000 7f191fda1700 1 -- 172.21.15.59:0/3342429877 >> 172.21.15.59:0/3342429877 conn(0x7f19180b2740 msgr2=0x7f19180b2b20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:59:34.945 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:34.940+0000 7f191fda1700 1 -- 172.21.15.59:0/3342429877 shutdown_connections 2023-11-24T07:59:34.945 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:34.940+0000 7f191fda1700 1 -- 172.21.15.59:0/3342429877 wait complete. 2023-11-24T07:59:34.948 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:34.941+0000 7f191fda1700 1 Processor -- start 2023-11-24T07:59:34.948 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:34.941+0000 7f191fda1700 1 -- start start 2023-11-24T07:59:34.949 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:34.941+0000 7f191fda1700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f19180722c0 0x7f19180c8ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:34.949 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:34.941+0000 7f191fda1700 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f1918143ae0 0x7f191815c340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:34.949 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:34.941+0000 7f191fda1700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f19181440f0 0x7f191813f390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:34.949 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:34.941+0000 7f191fda1700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f191815cb90 con 0x7f19181440f0 2023-11-24T07:59:34.950 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:34.941+0000 7f191fda1700 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f191815ce70 con 0x7f19180722c0 2023-11-24T07:59:34.950 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:34.941+0000 7f191fda1700 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f191815d150 con 0x7f1918143ae0 2023-11-24T07:59:34.950 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:34.942+0000 7f191e33e700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f19181440f0 0x7f191813f390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:34.950 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:34.942+0000 7f191e33e700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f19181440f0 0x7f191813f390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:55314/0 (socket says 172.21.15.59:55314) 2023-11-24T07:59:34.950 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:34.942+0000 7f191e33e700 1 -- 172.21.15.59:0/3893885516 learned_addr learned my addr 172.21.15.59:0/3893885516 (peer_addr_for_me v2:172.21.15.59:0/0) 2023-11-24T07:59:34.951 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:34.942+0000 7f191db3d700 1 --2- 172.21.15.59:0/3893885516 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f19180722c0 0x7f19180c8ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:34.951 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:34.942+0000 7f191d33c700 1 --2- 172.21.15.59:0/3893885516 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f1918143ae0 0x7f191815c340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:34.951 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:34.942+0000 7f191e33e700 1 -- 172.21.15.59:0/3893885516 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f1918143ae0 msgr2=0x7f191815c340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:34.951 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:34.942+0000 7f191e33e700 1 --2- 172.21.15.59:0/3893885516 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f1918143ae0 0x7f191815c340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:34.951 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:34.942+0000 7f191e33e700 1 -- 172.21.15.59:0/3893885516 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f19180722c0 msgr2=0x7f19180c8ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:34.951 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:34.942+0000 7f191e33e700 1 --2- 172.21.15.59:0/3893885516 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f19180722c0 0x7f19180c8ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:34.952 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:34.942+0000 7f191e33e700 1 -- 172.21.15.59:0/3893885516 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f191000bc80 con 0x7f19181440f0 2023-11-24T07:59:34.952 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:34.942+0000 7f191db3d700 1 --2- 172.21.15.59:0/3893885516 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f19180722c0 0x7f19180c8ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-24T07:59:34.952 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:34.942+0000 7f191e33e700 1 --2- 172.21.15.59:0/3893885516 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f19181440f0 0x7f191813f390 secure :-1 s=READY pgs=936 cs=0 l=1 rev1=1 crypto rx=0x7f19180d1bd0 tx=0x7f1910004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:59:34.952 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:34.942+0000 7f190e7fc700 1 -- 172.21.15.59:0/3893885516 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1910004250 con 0x7f19181440f0 2023-11-24T07:59:34.952 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:34.942+0000 7f190e7fc700 1 -- 172.21.15.59:0/3893885516 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f19100043b0 con 0x7f19181440f0 2023-11-24T07:59:34.952 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:34.942+0000 7f191fda1700 1 -- 172.21.15.59:0/3893885516 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f191815d430 con 0x7f19181440f0 2023-11-24T07:59:34.952 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:34.942+0000 7f190e7fc700 1 -- 172.21.15.59:0/3893885516 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1910004760 con 0x7f19181440f0 2023-11-24T07:59:34.952 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:34.942+0000 7f191fda1700 1 -- 172.21.15.59:0/3893885516 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f191815dc30 con 0x7f19181440f0 2023-11-24T07:59:34.952 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:34.943+0000 7f191fda1700 1 -- 172.21.15.59:0/3893885516 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f191815da80 con 0x7f19181440f0 2023-11-24T07:59:34.953 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:34.944+0000 7f190e7fc700 1 -- 172.21.15.59:0/3893885516 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 11) v1 ==== 89741+0+0 (secure 0 0 0) 0x7f1910009460 con 0x7f19181440f0 2023-11-24T07:59:34.953 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:34.944+0000 7f190e7fc700 1 --2- 172.21.15.59:0/3893885516 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f18fc06b470 0x7f18fc06d930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:34.953 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:34.944+0000 7f190e7fc700 1 -- 172.21.15.59:0/3893885516 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(75..75 src has 1..75) v4 ==== 5765+0+0 (secure 0 0 0) 0x7f191008d6a0 con 0x7f19181440f0 2023-11-24T07:59:34.953 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:34.944+0000 7f191db3d700 1 --2- 172.21.15.59:0/3893885516 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f18fc06b470 0x7f18fc06d930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:34.953 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:34.945+0000 7f191db3d700 1 --2- 172.21.15.59:0/3893885516 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f18fc06b470 0x7f18fc06d930 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f1908007fd0 tx=0x7f1908009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:59:34.953 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:34.949+0000 7f190e7fc700 1 -- 172.21.15.59:0/3893885516 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f1910057f20 con 0x7f19181440f0 2023-11-24T07:59:35.150 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:35.148+0000 7f191fda1700 1 -- 172.21.15.59:0/3893885516 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f1918075480 con 0x7f19181440f0 2023-11-24T07:59:35.151 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:35.149+0000 7f190e7fc700 1 -- 172.21.15.59:0/3893885516 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v75) v1 ==== 100+0+11 (secure 0 0 0) 0x7f191005b540 con 0x7f19181440f0 2023-11-24T07:59:35.151 INFO:teuthology.orchestra.run.smithi059.stdout:pg_num: 64 2023-11-24T07:59:35.153 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:35.151+0000 7f191fda1700 1 -- 172.21.15.59:0/3893885516 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f18fc06b470 msgr2=0x7f18fc06d930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:35.153 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:35.151+0000 7f191fda1700 1 --2- 172.21.15.59:0/3893885516 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f18fc06b470 0x7f18fc06d930 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f1908007fd0 tx=0x7f1908009040 comp rx=0 tx=0).stop 2023-11-24T07:59:35.153 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:35.151+0000 7f191fda1700 1 -- 172.21.15.59:0/3893885516 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f19181440f0 msgr2=0x7f191813f390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:35.153 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:35.151+0000 7f191fda1700 1 --2- 172.21.15.59:0/3893885516 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f19181440f0 0x7f191813f390 secure :-1 s=READY pgs=936 cs=0 l=1 rev1=1 crypto rx=0x7f19180d1bd0 tx=0x7f1910004ad0 comp rx=0 tx=0).stop 2023-11-24T07:59:35.153 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:35.151+0000 7f191fda1700 1 -- 172.21.15.59:0/3893885516 shutdown_connections 2023-11-24T07:59:35.153 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:35.151+0000 7f191fda1700 1 --2- 172.21.15.59:0/3893885516 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f18fc06b470 0x7f18fc06d930 unknown :-1 s=CLOSED pgs=435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:35.153 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:35.151+0000 7f191fda1700 1 --2- 172.21.15.59:0/3893885516 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f19181440f0 0x7f191813f390 unknown :-1 s=CLOSED pgs=936 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:35.153 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:35.151+0000 7f191fda1700 1 --2- 172.21.15.59:0/3893885516 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f19180722c0 0x7f19180c8ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:35.153 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:35.151+0000 7f191fda1700 1 --2- 172.21.15.59:0/3893885516 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f1918143ae0 0x7f191815c340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:35.154 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:35.151+0000 7f191fda1700 1 -- 172.21.15.59:0/3893885516 >> 172.21.15.59:0/3893885516 conn(0x7f19180b2740 msgr2=0x7f1918071bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:59:35.154 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:35.151+0000 7f191fda1700 1 -- 172.21.15.59:0/3893885516 shutdown_connections 2023-11-24T07:59:35.154 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:35.151+0000 7f191fda1700 1 -- 172.21.15.59:0/3893885516 wait complete. 2023-11-24T07:59:35.165 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-11-24T07:59:35.290 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:35.288+0000 7f83b6883700 1 Processor -- start 2023-11-24T07:59:35.290 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:35.289+0000 7f83b6883700 1 -- start start 2023-11-24T07:59:35.292 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:35.289+0000 7f83b6883700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f83b00b4150 0x7f83b006de50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:35.293 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:35.289+0000 7f83b6883700 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f83b0071f50 con 0x7f83b00b5ab0 2023-11-24T07:59:35.293 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:35.289+0000 7f83b6883700 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f83b0072230 con 0x7f83b006b620 2023-11-24T07:59:35.293 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:35.289+0000 7f83b6883700 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f83b0072510 con 0x7f83b00b4150 2023-11-24T07:59:35.293 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:35.290+0000 7f83af7fe700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f83b00b4150 0x7f83b006de50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:35.293 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:35.290+0000 7f83af7fe700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f83b00b4150 0x7f83b006de50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:55320/0 (socket says 172.21.15.59:55320) 2023-11-24T07:59:35.293 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:35.290+0000 7f83b4e20700 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7f83b00b5ab0 0x7f83b006e390 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:58796/0 (socket says 172.21.15.59:58796) 2023-11-24T07:59:35.293 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:35.290+0000 7f83af7fe700 1 -- 172.21.15.59:0/3377980309 learned_addr learned my addr 172.21.15.59:0/3377980309 (peer_addr_for_me v2:172.21.15.59:0/0) 2023-11-24T07:59:35.294 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:35.290+0000 7f83af7fe700 1 -- 172.21.15.59:0/3377980309 >> v1:172.21.15.76:6789/0 conn(0x7f83b006b620 legacy=0x7f83b00b3a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:35.294 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:35.290+0000 7f83af7fe700 1 -- 172.21.15.59:0/3377980309 >> v1:172.21.15.59:6789/0 conn(0x7f83b00b5ab0 legacy=0x7f83b006e390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:35.294 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:35.290+0000 7f83af7fe700 1 -- 172.21.15.59:0/3377980309 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f83b00727f0 con 0x7f83b00b4150 2023-11-24T07:59:35.294 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:35.290+0000 7f83aeffd700 1 -- 172.21.15.59:0/3377980309 <== mon.0 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1171441497 0 0) 0x7f83b0071f50 con 0x7f83b00b5ab0 2023-11-24T07:59:35.294 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:35.290+0000 7f83af7fe700 1 --2- 172.21.15.59:0/3377980309 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f83b00b4150 0x7f83b006de50 secure :-1 s=READY pgs=938 cs=0 l=1 rev1=1 crypto rx=0x7f83a40098f0 tx=0x7f83a4009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f2d313e23434308a server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:59:35.294 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:35.290+0000 7f83aeffd700 1 -- 172.21.15.59:0/3377980309 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83a400b8a0 con 0x7f83b00b4150 2023-11-24T07:59:35.294 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:35.290+0000 7f83aeffd700 1 -- 172.21.15.59:0/3377980309 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f83a400ba00 con 0x7f83b00b4150 2023-11-24T07:59:35.294 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:35.290+0000 7f83aeffd700 1 -- 172.21.15.59:0/3377980309 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83a400bd90 con 0x7f83b00b4150 2023-11-24T07:59:35.294 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:35.291+0000 7f83b6883700 1 -- 172.21.15.59:0/3377980309 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f83b00b4150 msgr2=0x7f83b006de50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:35.294 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:35.291+0000 7f83b6883700 1 --2- 172.21.15.59:0/3377980309 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f83b00b4150 0x7f83b006de50 secure :-1 s=READY pgs=938 cs=0 l=1 rev1=1 crypto rx=0x7f83a40098f0 tx=0x7f83a4009c00 comp rx=0 tx=0).stop 2023-11-24T07:59:35.294 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:35.291+0000 7f83b6883700 1 -- 172.21.15.59:0/3377980309 shutdown_connections 2023-11-24T07:59:35.295 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:35.291+0000 7f83b6883700 1 --2- 172.21.15.59:0/3377980309 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f83b00b4150 0x7f83b006de50 unknown :-1 s=CLOSED pgs=938 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:35.295 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:35.291+0000 7f83b6883700 1 -- 172.21.15.59:0/3377980309 >> 172.21.15.59:0/3377980309 conn(0x7f83b00b2740 msgr2=0x7f83b00b2b20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:59:35.295 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:35.291+0000 7f83b6883700 1 -- 172.21.15.59:0/3377980309 shutdown_connections 2023-11-24T07:59:35.295 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:35.291+0000 7f83b6883700 1 -- 172.21.15.59:0/3377980309 wait complete. 2023-11-24T07:59:35.298 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:35.292+0000 7f83b6883700 1 Processor -- start 2023-11-24T07:59:35.298 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:35.292+0000 7f83b6883700 1 -- start start 2023-11-24T07:59:35.298 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:35.293+0000 7f83b6883700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f83b006b620 0x7f83b013b270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:35.298 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:35.293+0000 7f83b6883700 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f83b00b4150 0x7f83b006db70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:35.298 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:35.293+0000 7f83b6883700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f83b00b5ab0 0x7f83b01d5760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:35.298 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:35.293+0000 7f83b6883700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83b006e550 con 0x7f83b00b5ab0 2023-11-24T07:59:35.298 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:35.293+0000 7f83b6883700 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83b006e830 con 0x7f83b006b620 2023-11-24T07:59:35.298 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:35.293+0000 7f83b6883700 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f83b006eb10 con 0x7f83b00b4150 2023-11-24T07:59:35.298 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:35.293+0000 7f83b4e20700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f83b00b5ab0 0x7f83b01d5760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:35.299 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:35.293+0000 7f83b4e20700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f83b00b5ab0 0x7f83b01d5760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:55328/0 (socket says 172.21.15.59:55328) 2023-11-24T07:59:35.299 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:35.293+0000 7f83b4e20700 1 -- 172.21.15.59:0/2516740006 learned_addr learned my addr 172.21.15.59:0/2516740006 (peer_addr_for_me v2:172.21.15.59:0/0) 2023-11-24T07:59:35.299 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:35.293+0000 7f83affff700 1 --2- 172.21.15.59:0/2516740006 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f83b006b620 0x7f83b013b270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:35.299 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:35.293+0000 7f83af7fe700 1 --2- 172.21.15.59:0/2516740006 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f83b00b4150 0x7f83b006db70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:35.299 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:35.293+0000 7f83b4e20700 1 -- 172.21.15.59:0/2516740006 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f83b00b4150 msgr2=0x7f83b006db70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:35.299 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:35.293+0000 7f83b4e20700 1 --2- 172.21.15.59:0/2516740006 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f83b00b4150 0x7f83b006db70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:35.299 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:35.293+0000 7f83b4e20700 1 -- 172.21.15.59:0/2516740006 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f83b006b620 msgr2=0x7f83b013b270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:35.299 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:35.293+0000 7f83b4e20700 1 --2- 172.21.15.59:0/2516740006 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f83b006b620 0x7f83b013b270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:35.299 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:35.293+0000 7f83b4e20700 1 -- 172.21.15.59:0/2516740006 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f83a4009550 con 0x7f83b00b5ab0 2023-11-24T07:59:35.299 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:35.294+0000 7f83b4e20700 1 --2- 172.21.15.59:0/2516740006 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f83b00b5ab0 0x7f83b01d5760 secure :-1 s=READY pgs=939 cs=0 l=1 rev1=1 crypto rx=0x7f83a800cba0 tx=0x7f83a800ceb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:59:35.299 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:35.294+0000 7f83acff9700 1 -- 172.21.15.59:0/2516740006 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83a800aaa0 con 0x7f83b00b5ab0 2023-11-24T07:59:35.299 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:35.294+0000 7f83acff9700 1 -- 172.21.15.59:0/2516740006 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f83a800ac20 con 0x7f83b00b5ab0 2023-11-24T07:59:35.300 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:35.294+0000 7f83b6883700 1 -- 172.21.15.59:0/2516740006 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f83b006edf0 con 0x7f83b00b5ab0 2023-11-24T07:59:35.300 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:35.294+0000 7f83acff9700 1 -- 172.21.15.59:0/2516740006 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83a80070d0 con 0x7f83b00b5ab0 2023-11-24T07:59:35.300 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:35.294+0000 7f83b6883700 1 -- 172.21.15.59:0/2516740006 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f83b013b7b0 con 0x7f83b00b5ab0 2023-11-24T07:59:35.300 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:35.295+0000 7f83b6883700 1 -- 172.21.15.59:0/2516740006 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f83b006f440 con 0x7f83b00b5ab0 2023-11-24T07:59:35.300 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:35.295+0000 7f83acff9700 1 -- 172.21.15.59:0/2516740006 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 11) v1 ==== 89741+0+0 (secure 0 0 0) 0x7f83a8007230 con 0x7f83b00b5ab0 2023-11-24T07:59:35.300 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:35.296+0000 7f83acff9700 1 --2- 172.21.15.59:0/2516740006 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f839c06b470 0x7f839c06d930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:35.300 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:35.296+0000 7f83acff9700 1 -- 172.21.15.59:0/2516740006 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(75..75 src has 1..75) v4 ==== 5765+0+0 (secure 0 0 0) 0x7f83a8089da0 con 0x7f83b00b5ab0 2023-11-24T07:59:35.300 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:35.296+0000 7f83affff700 1 --2- 172.21.15.59:0/2516740006 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f839c06b470 0x7f839c06d930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:35.300 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:35.296+0000 7f83affff700 1 --2- 172.21.15.59:0/2516740006 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f839c06b470 0x7f839c06d930 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f83a0007fd0 tx=0x7f83a0009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:59:35.301 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:35.300+0000 7f83acff9700 1 -- 172.21.15.59:0/2516740006 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f83a808f020 con 0x7f83b00b5ab0 2023-11-24T07:59:35.518 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:35.517+0000 7f83b6883700 1 -- 172.21.15.59:0/2516740006 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f83b0068670 con 0x7f83b00b5ab0 2023-11-24T07:59:35.519 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:35.517+0000 7f83acff9700 1 -- 172.21.15.59:0/2516740006 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v75) v1 ==== 80+0+0 (secure 0 0 0) 0x7f83a8018020 con 0x7f83b00b5ab0 2023-11-24T07:59:35.521 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:35.519+0000 7f83b6883700 1 -- 172.21.15.59:0/2516740006 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f839c06b470 msgr2=0x7f839c06d930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:35.521 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:35.519+0000 7f83b6883700 1 --2- 172.21.15.59:0/2516740006 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f839c06b470 0x7f839c06d930 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f83a0007fd0 tx=0x7f83a0009040 comp rx=0 tx=0).stop 2023-11-24T07:59:35.522 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:35.519+0000 7f83b6883700 1 -- 172.21.15.59:0/2516740006 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f83b00b5ab0 msgr2=0x7f83b01d5760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:35.522 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:35.519+0000 7f83b6883700 1 --2- 172.21.15.59:0/2516740006 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f83b00b5ab0 0x7f83b01d5760 secure :-1 s=READY pgs=939 cs=0 l=1 rev1=1 crypto rx=0x7f83a800cba0 tx=0x7f83a800ceb0 comp rx=0 tx=0).stop 2023-11-24T07:59:35.522 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:35.519+0000 7f83b6883700 1 -- 172.21.15.59:0/2516740006 shutdown_connections 2023-11-24T07:59:35.522 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:35.519+0000 7f83b6883700 1 --2- 172.21.15.59:0/2516740006 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f839c06b470 0x7f839c06d930 unknown :-1 s=CLOSED pgs=436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:35.522 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:35.519+0000 7f83b6883700 1 --2- 172.21.15.59:0/2516740006 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f83b00b5ab0 0x7f83b01d5760 unknown :-1 s=CLOSED pgs=939 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:35.522 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:35.519+0000 7f83b6883700 1 --2- 172.21.15.59:0/2516740006 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f83b006b620 0x7f83b013b270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:35.522 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:35.519+0000 7f83b6883700 1 --2- 172.21.15.59:0/2516740006 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f83b00b4150 0x7f83b006db70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:35.522 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:35.519+0000 7f83b6883700 1 -- 172.21.15.59:0/2516740006 >> 172.21.15.59:0/2516740006 conn(0x7f83b00b2740 msgr2=0x7f83b00af070 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:59:35.522 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:35.519+0000 7f83b6883700 1 -- 172.21.15.59:0/2516740006 shutdown_connections 2023-11-24T07:59:35.522 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:35.519+0000 7f83b6883700 1 -- 172.21.15.59:0/2516740006 wait complete. 2023-11-24T07:59:35.523 INFO:teuthology.orchestra.run.smithi059.stderr:listed 0 entries 2023-11-24T07:59:35.533 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.1... 2023-11-24T07:59:35.533 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T07:59:35.533 DEBUG:teuthology.orchestra.run.smithi076:> sudo umount /home/ubuntu/cephtest/mnt.1 2023-11-24T07:59:35.582 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi076.front.sepia.ceph.com 2023-11-24T07:59:35.582 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T07:59:35.583 DEBUG:teuthology.orchestra.run.smithi076:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2023-11-24T07:59:35.603 DEBUG:teuthology.orchestra.run.smithi059:> set -ex 2023-11-24T07:59:35.603 DEBUG:teuthology.orchestra.run.smithi059:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-24T07:59:35.621 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=16287364k,nr_inodes=4071841,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 tmpfs /sys/fs/cgroup tmpfs ro,seclabel,nosuid,nodev,noexec,mode=755 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,seclabel,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,seclabel,nosuid,nodev,noexec,relatime,hugetlb 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,seclabel,nosuid,nodev,noexec,relatime,perf_event 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,seclabel,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/rdma cgroup rw,seclabel,nosuid,nodev,noexec,relatime,rdma 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,seclabel,nosuid,nodev,noexec,relatime,freezer 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,seclabel,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,seclabel,nosuid,nodev,noexec,relatime,devices 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,blkio 0 0 none /sys/kernel/tracing tracefs rw,seclabel,relatime 0 0 configfs /sys/kernel/config configfs rw,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=42,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=16075 0 0 mqueue /dev/mqueue mqueue rw,seclabel,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,relatime 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3261172k,mode=700,uid=1000,gid=1265 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw,seclabel 0 0 2023-11-24T07:59:35.621 DEBUG:teuthology.orchestra.run.smithi076:> set -ex 2023-11-24T07:59:35.621 DEBUG:teuthology.orchestra.run.smithi076:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-24T07:59:35.666 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=16284272k,nr_inodes=4071068,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 tmpfs /sys/fs/cgroup tmpfs ro,seclabel,nosuid,nodev,noexec,mode=755 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,seclabel,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,seclabel,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,seclabel,nosuid,nodev,noexec,relatime,freezer 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,seclabel,nosuid,nodev,noexec,relatime,perf_event 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,seclabel,nosuid,nodev,noexec,relatime,hugetlb 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,seclabel,nosuid,nodev,noexec,relatime,devices 0 0 cgroup /sys/fs/cgroup/rdma cgroup rw,seclabel,nosuid,nodev,noexec,relatime,rdma 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,seclabel,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,blkio 0 0 none /sys/kernel/tracing tracefs rw,seclabel,relatime 0 0 configfs /sys/kernel/config configfs rw,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=35,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=20116 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,relatime 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3260552k,mode=700,uid=1000,gid=1265 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw,seclabel 0 0 2023-11-24T07:59:35.667 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-24T07:59:35.791 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:35.790+0000 7f9543b4c700 1 Processor -- start 2023-11-24T07:59:35.792 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:35.790+0000 7f9543b4c700 1 -- start start 2023-11-24T07:59:35.798 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:35.791+0000 7f9543b4c700 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f953c0722c0 0x7f953c0b51b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:35.799 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:35.791+0000 7f9543b4c700 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f953c143ec0 con 0x7f953c1440f0 2023-11-24T07:59:35.799 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:35.791+0000 7f9543b4c700 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f953c0b3ae0 con 0x7f953c143ae0 2023-11-24T07:59:35.799 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:35.791+0000 7f9543b4c700 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f953c0b40a0 con 0x7f953c0722c0 2023-11-24T07:59:35.799 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:35.791+0000 7f95420e9700 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7f953c1440f0 0x7f953c06e510 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:58810/0 (socket says 172.21.15.59:58810) 2023-11-24T07:59:35.800 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:35.791+0000 7f95420e9700 1 -- 172.21.15.59:0/2335026188 learned_addr learned my addr 172.21.15.59:0/2335026188 (peer_addr_for_me v1:172.21.15.59:0/0) 2023-11-24T07:59:35.800 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:35.791+0000 7f95410e7700 1 --2- 172.21.15.59:0/2335026188 >> v2:172.21.15.76:3300/0 conn(0x7f953c0722c0 0x7f953c0b51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:35.800 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:35.791+0000 7f95408e6700 1 -- 172.21.15.59:0/2335026188 <== mon.0 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2958421927 0 0) 0x7f953c143ec0 con 0x7f953c1440f0 2023-11-24T07:59:35.800 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:35.791+0000 7f95408e6700 1 -- 172.21.15.59:0/2335026188 --> v1:172.21.15.59:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f95280035b0 con 0x7f953c1440f0 2023-11-24T07:59:35.800 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:35.791+0000 7f95408e6700 1 -- 172.21.15.59:0/2335026188 <== mon.1 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 800236847 0 0) 0x7f953c0b3ae0 con 0x7f953c143ae0 2023-11-24T07:59:35.800 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:35.791+0000 7f95408e6700 1 -- 172.21.15.59:0/2335026188 --> v1:172.21.15.76:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f953c143ec0 con 0x7f953c143ae0 2023-11-24T07:59:35.800 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:35.791+0000 7f95408e6700 1 -- 172.21.15.59:0/2335026188 <== mon.0 v1:172.21.15.59:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3538505628 0 0) 0x7f95280035b0 con 0x7f953c1440f0 2023-11-24T07:59:35.800 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:35.791+0000 7f95408e6700 1 -- 172.21.15.59:0/2335026188 >> v2:172.21.15.76:3300/0 conn(0x7f953c0722c0 msgr2=0x7f953c0b51b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T07:59:35.801 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:35.791+0000 7f95408e6700 1 --2- 172.21.15.59:0/2335026188 >> v2:172.21.15.76:3300/0 conn(0x7f953c0722c0 0x7f953c0b51b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:35.801 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:35.791+0000 7f95408e6700 1 -- 172.21.15.59:0/2335026188 >> v1:172.21.15.76:6789/0 conn(0x7f953c143ae0 legacy=0x7f953c071bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:35.801 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:35.791+0000 7f95408e6700 1 -- 172.21.15.59:0/2335026188 --> v1:172.21.15.59:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f953c0b3dc0 con 0x7f953c1440f0 2023-11-24T07:59:35.801 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:35.791+0000 7f95408e6700 1 -- 172.21.15.59:0/2335026188 <== mon.0 v1:172.21.15.59:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1312333566 0 0) 0x7f95340019f0 con 0x7f953c1440f0 2023-11-24T07:59:35.801 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:35.791+0000 7f95408e6700 1 -- 172.21.15.59:0/2335026188 >> v1:172.21.15.59:6789/0 conn(0x7f953c1440f0 legacy=0x7f953c06e510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:35.801 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:35.792+0000 7f95408e6700 1 --2- 172.21.15.59:0/2335026188 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f9528003b90 0x7f9528003f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:35.801 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:35.792+0000 7f95408e6700 1 --2- 172.21.15.59:0/2335026188 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9528004850 0x7f9528004cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:35.801 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:35.792+0000 7f95408e6700 1 --2- 172.21.15.59:0/2335026188 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9528005210 0x7f952800dc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:35.802 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:35.792+0000 7f95408e6700 1 -- 172.21.15.59:0/2335026188 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f953c0b40a0 con 0x7f9528005210 2023-11-24T07:59:35.802 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:35.792+0000 7f95408e6700 1 -- 172.21.15.59:0/2335026188 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f952800e2a0 con 0x7f9528004850 2023-11-24T07:59:35.802 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:35.792+0000 7f95408e6700 1 -- 172.21.15.59:0/2335026188 --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f952800e460 con 0x7f9528003b90 2023-11-24T07:59:35.802 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:35.792+0000 7f95410e7700 1 --2- 172.21.15.59:0/2335026188 >> v2:172.21.15.76:3300/0 conn(0x7f953c0722c0 0x7f953c0b51b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-24T07:59:35.802 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:35.792+0000 7f95420e9700 1 --2- 172.21.15.59:0/2335026188 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9528005210 0x7f952800dc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:35.802 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:35.792+0000 7f95418e8700 1 --2- 172.21.15.59:0/2335026188 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f9528003b90 0x7f9528003f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:35.802 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:35.792+0000 7f95410e7700 1 --2- 172.21.15.59:0/2335026188 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9528004850 0x7f9528004cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:35.802 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:35.792+0000 7f95420e9700 1 -- 172.21.15.59:0/2335026188 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f9528003b90 msgr2=0x7f9528003f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T07:59:35.803 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:35.792+0000 7f95420e9700 1 --2- 172.21.15.59:0/2335026188 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f9528003b90 0x7f9528003f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:35.803 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:35.792+0000 7f95420e9700 1 -- 172.21.15.59:0/2335026188 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9528004850 msgr2=0x7f9528004cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T07:59:35.803 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:35.792+0000 7f95420e9700 1 --2- 172.21.15.59:0/2335026188 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9528004850 0x7f9528004cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:35.803 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:35.792+0000 7f95420e9700 1 -- 172.21.15.59:0/2335026188 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f95280035b0 con 0x7f9528005210 2023-11-24T07:59:35.803 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:35.792+0000 7f95418e8700 1 --2- 172.21.15.59:0/2335026188 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f9528003b90 0x7f9528003f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-24T07:59:35.803 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:35.792+0000 7f95410e7700 1 --2- 172.21.15.59:0/2335026188 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9528004850 0x7f9528004cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-24T07:59:35.803 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:35.792+0000 7f95420e9700 1 -- 172.21.15.59:0/2335026188 reap_dead start 2023-11-24T07:59:35.803 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:35.792+0000 7f95420e9700 1 --2- 172.21.15.59:0/2335026188 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9528005210 0x7f952800dc20 secure :-1 s=READY pgs=941 cs=0 l=1 rev1=1 crypto rx=0x7f953400a7c0 tx=0x7f9528005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4483b61b3443c1ae server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:59:35.803 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:35.792+0000 7f95408e6700 1 -- 172.21.15.59:0/2335026188 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9534012600 con 0x7f9528005210 2023-11-24T07:59:35.804 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:35.792+0000 7f95408e6700 1 -- 172.21.15.59:0/2335026188 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9534012780 con 0x7f9528005210 2023-11-24T07:59:35.804 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:35.792+0000 7f95408e6700 1 -- 172.21.15.59:0/2335026188 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9534012b30 con 0x7f9528005210 2023-11-24T07:59:35.804 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:35.793+0000 7f9543b4c700 1 -- 172.21.15.59:0/2335026188 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9528005210 msgr2=0x7f952800dc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:35.804 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:35.793+0000 7f9543b4c700 1 --2- 172.21.15.59:0/2335026188 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9528005210 0x7f952800dc20 secure :-1 s=READY pgs=941 cs=0 l=1 rev1=1 crypto rx=0x7f953400a7c0 tx=0x7f9528005880 comp rx=0 tx=0).stop 2023-11-24T07:59:35.804 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:35.793+0000 7f9543b4c700 1 -- 172.21.15.59:0/2335026188 shutdown_connections 2023-11-24T07:59:35.804 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:35.793+0000 7f9543b4c700 1 --2- 172.21.15.59:0/2335026188 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9528005210 0x7f952800dc20 unknown :-1 s=CLOSED pgs=941 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:35.804 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:35.793+0000 7f9543b4c700 1 -- 172.21.15.59:0/2335026188 >> 172.21.15.59:0/2335026188 conn(0x7f953c0b2740 msgr2=0x7f953c0b2b20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:59:35.805 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:35.793+0000 7f9543b4c700 1 -- 172.21.15.59:0/2335026188 shutdown_connections 2023-11-24T07:59:35.805 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:35.793+0000 7f9543b4c700 1 -- 172.21.15.59:0/2335026188 wait complete. 2023-11-24T07:59:35.805 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:35.794+0000 7f9543b4c700 1 Processor -- start 2023-11-24T07:59:35.805 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:35.794+0000 7f9543b4c700 1 -- start start 2023-11-24T07:59:35.805 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:35.795+0000 7f9543b4c700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9528005210 0x7f953c151d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:35.805 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:35.795+0000 7f9543b4c700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f953c142640 0x7f953c150f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:35.805 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:35.795+0000 7f9543b4c700 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f953c151480 0x7f953c1f4270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:35.805 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:35.795+0000 7f9543b4c700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f953c151910 con 0x7f9528005210 2023-11-24T07:59:35.805 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:35.795+0000 7f9543b4c700 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f953c15d460 con 0x7f953c142640 2023-11-24T07:59:35.805 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:35.795+0000 7f9543b4c700 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f953c15e450 con 0x7f953c151480 2023-11-24T07:59:35.806 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:35.795+0000 7f95418e8700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9528005210 0x7f953c151d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:35.806 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:35.795+0000 7f95418e8700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9528005210 0x7f953c151d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:55344/0 (socket says 172.21.15.59:55344) 2023-11-24T07:59:35.806 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:35.795+0000 7f95418e8700 1 -- 172.21.15.59:0/1778852996 learned_addr learned my addr 172.21.15.59:0/1778852996 (peer_addr_for_me v2:172.21.15.59:0/0) 2023-11-24T07:59:35.806 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:35.795+0000 7f95420e9700 1 --2- 172.21.15.59:0/1778852996 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f953c151480 0x7f953c1f4270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:35.806 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:35.795+0000 7f95410e7700 1 --2- 172.21.15.59:0/1778852996 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f953c142640 0x7f953c150f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:35.806 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:35.795+0000 7f95418e8700 1 -- 172.21.15.59:0/1778852996 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f953c151480 msgr2=0x7f953c1f4270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:35.807 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:35.795+0000 7f95418e8700 1 --2- 172.21.15.59:0/1778852996 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f953c151480 0x7f953c1f4270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:35.807 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:35.795+0000 7f95418e8700 1 -- 172.21.15.59:0/1778852996 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f953c142640 msgr2=0x7f953c150f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:35.807 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:35.795+0000 7f95418e8700 1 --2- 172.21.15.59:0/1778852996 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f953c142640 0x7f953c150f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:35.807 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:35.795+0000 7f95418e8700 1 -- 172.21.15.59:0/1778852996 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f953c15d6b0 con 0x7f9528005210 2023-11-24T07:59:35.807 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:35.796+0000 7f95418e8700 1 --2- 172.21.15.59:0/1778852996 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9528005210 0x7f953c151d10 secure :-1 s=READY pgs=942 cs=0 l=1 rev1=1 crypto rx=0x7f953800e570 tx=0x7f953800e930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:59:35.807 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:35.796+0000 7f95327fc700 1 -- 172.21.15.59:0/1778852996 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95380052e0 con 0x7f9528005210 2023-11-24T07:59:35.807 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:35.796+0000 7f95327fc700 1 -- 172.21.15.59:0/1778852996 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9538005440 con 0x7f9528005210 2023-11-24T07:59:35.807 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:35.796+0000 7f9543b4c700 1 -- 172.21.15.59:0/1778852996 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f953c15d990 con 0x7f9528005210 2023-11-24T07:59:35.807 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:35.796+0000 7f95327fc700 1 -- 172.21.15.59:0/1778852996 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95380038a0 con 0x7f9528005210 2023-11-24T07:59:35.807 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:35.796+0000 7f9543b4c700 1 -- 172.21.15.59:0/1778852996 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f953c15df50 con 0x7f9528005210 2023-11-24T07:59:35.807 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:35.797+0000 7f9543b4c700 1 -- 172.21.15.59:0/1778852996 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f953c15e110 con 0x7f9528005210 2023-11-24T07:59:35.808 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:35.797+0000 7f95327fc700 1 -- 172.21.15.59:0/1778852996 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 11) v1 ==== 89741+0+0 (secure 0 0 0) 0x7f9538003a00 con 0x7f9528005210 2023-11-24T07:59:35.809 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:35.798+0000 7f95327fc700 1 --2- 172.21.15.59:0/1778852996 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f952006b470 0x7f952006d930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:35.809 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:35.798+0000 7f95327fc700 1 -- 172.21.15.59:0/1778852996 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(75..75 src has 1..75) v4 ==== 5765+0+0 (secure 0 0 0) 0x7f95380896c0 con 0x7f9528005210 2023-11-24T07:59:35.809 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:35.798+0000 7f95410e7700 1 --2- 172.21.15.59:0/1778852996 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f952006b470 0x7f952006d930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:35.809 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:35.798+0000 7f95410e7700 1 --2- 172.21.15.59:0/1778852996 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f952006b470 0x7f952006d930 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f952c004510 tx=0x7f952c008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:59:35.809 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:35.802+0000 7f95327fc700 1 -- 172.21.15.59:0/1778852996 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f9538054100 con 0x7f9528005210 2023-11-24T07:59:36.051 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:36.049+0000 7f9543b4c700 1 -- 172.21.15.59:0/1778852996 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f953c0681b0 con 0x7f9528005210 2023-11-24T07:59:36.052 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:36.050+0000 7f95327fc700 1 -- 172.21.15.59:0/1778852996 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 45 v45) v1 ==== 94+0+4915 (secure 0 0 0) 0x7f9538019090 con 0x7f9528005210 2023-11-24T07:59:36.052 INFO:teuthology.orchestra.run.smithi059.stdout: 2023-11-24T07:59:36.052 INFO:teuthology.orchestra.run.smithi059.stdout:{"epoch":45,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5585,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.76:6837/2282835612","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6835","nonce":2282835612},{"type":"v1","addr":"172.21.15.76:6837","nonce":2282835612}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":39},{"gid":6097,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.76:6836/3924452718","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6834","nonce":3924452718},{"type":"v1","addr":"172.21.15.76:6836","nonce":3924452718}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":39},{"gid":6549,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6834/1425120389","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6832","nonce":1425120389},{"type":"v1","addr":"172.21.15.59:6834","nonce":1425120389}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":39}],"filesystems":[{"mdsmap":{"epoch":44,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-24T07:59:14.054294+0000","modified":"2023-11-24T07:59:17.066277+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":6552},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6552":{"gid":6552,"name":"c","rank":0,"incarnation":40,"state":"up:active","state_seq":3,"addr":"172.21.15.59:6835/1228649189","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6833","nonce":1228649189},{"type":"v1","addr":"172.21.15.59:6835","nonce":1228649189}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[11],"metadata_pool":10,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":5}]} 2023-11-24T07:59:36.054 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:36.052+0000 7f9543b4c700 1 -- 172.21.15.59:0/1778852996 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f952006b470 msgr2=0x7f952006d930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:36.054 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:36.052+0000 7f9543b4c700 1 --2- 172.21.15.59:0/1778852996 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f952006b470 0x7f952006d930 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f952c004510 tx=0x7f952c008040 comp rx=0 tx=0).stop 2023-11-24T07:59:36.054 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:36.052+0000 7f9543b4c700 1 -- 172.21.15.59:0/1778852996 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9528005210 msgr2=0x7f953c151d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:36.054 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:36.052+0000 7f9543b4c700 1 --2- 172.21.15.59:0/1778852996 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9528005210 0x7f953c151d10 secure :-1 s=READY pgs=942 cs=0 l=1 rev1=1 crypto rx=0x7f953800e570 tx=0x7f953800e930 comp rx=0 tx=0).stop 2023-11-24T07:59:36.054 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:36.052+0000 7f9543b4c700 1 -- 172.21.15.59:0/1778852996 shutdown_connections 2023-11-24T07:59:36.055 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:36.052+0000 7f9543b4c700 1 --2- 172.21.15.59:0/1778852996 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f952006b470 0x7f952006d930 unknown :-1 s=CLOSED pgs=437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:36.055 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:36.052+0000 7f9543b4c700 1 --2- 172.21.15.59:0/1778852996 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f953c151480 0x7f953c1f4270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:36.055 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:36.052+0000 7f9543b4c700 1 --2- 172.21.15.59:0/1778852996 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9528005210 0x7f953c151d10 unknown :-1 s=CLOSED pgs=942 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:36.055 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:36.052+0000 7f9543b4c700 1 --2- 172.21.15.59:0/1778852996 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f953c142640 0x7f953c150f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:36.055 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:36.052+0000 7f9543b4c700 1 -- 172.21.15.59:0/1778852996 >> 172.21.15.59:0/1778852996 conn(0x7f953c0b2740 msgr2=0x7f953c0cf7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:59:36.055 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:36.052+0000 7f9543b4c700 1 -- 172.21.15.59:0/1778852996 shutdown_connections 2023-11-24T07:59:36.055 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:36.052+0000 7f9543b4c700 1 -- 172.21.15.59:0/1778852996 wait complete. 2023-11-24T07:59:36.055 INFO:teuthology.orchestra.run.smithi059.stderr:dumped fsmap epoch 45 2023-11-24T07:59:36.066 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-24T07:59:36.191 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:36.189+0000 7fd73d791700 1 Processor -- start 2023-11-24T07:59:36.191 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:36.189+0000 7fd73d791700 1 -- start start 2023-11-24T07:59:36.193 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:36.190+0000 7fd73d791700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fd7380d49d0 0x7fd7380d2490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:36.194 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:36.190+0000 7fd73d791700 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fd7380d29d0 0x7fd7380b40a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:36.194 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:36.190+0000 7fd73d791700 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd7380d4e50 con 0x7fd73815e460 2023-11-24T07:59:36.194 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:36.190+0000 7fd73d791700 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd7380d2e60 con 0x7fd7380d49d0 2023-11-24T07:59:36.194 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:36.190+0000 7fd73d791700 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd7380b3370 con 0x7fd7380d29d0 2023-11-24T07:59:36.194 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:36.190+0000 7fd7367fc700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fd7380d49d0 0x7fd7380d2490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:36.194 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:36.190+0000 7fd736ffd700 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7fd73815e460 0x7fd7380cf480 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:58826/0 (socket says 172.21.15.59:58826) 2023-11-24T07:59:36.194 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:36.190+0000 7fd736ffd700 1 -- 172.21.15.59:0/2851961996 learned_addr learned my addr 172.21.15.59:0/2851961996 (peer_addr_for_me v1:172.21.15.59:0/0) 2023-11-24T07:59:36.195 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:36.190+0000 7fd7377fe700 1 --2- 172.21.15.59:0/2851961996 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fd7380d29d0 0x7fd7380b40a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:36.195 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:36.190+0000 7fd7367fc700 1 -- 172.21.15.59:0/2851961996 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fd7380d29d0 msgr2=0x7fd7380b40a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T07:59:36.195 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:36.190+0000 7fd7367fc700 1 --2- 172.21.15.59:0/2851961996 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fd7380d29d0 0x7fd7380b40a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:36.195 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:36.190+0000 7fd7367fc700 1 -- 172.21.15.59:0/2851961996 >> v1:172.21.15.59:6789/0 conn(0x7fd73815e460 legacy=0x7fd7380cf480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:36.195 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:36.190+0000 7fd735ffb700 1 -- 172.21.15.59:0/2851961996 <== mon.0 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3261622225 0 0) 0x7fd7380d4e50 con 0x7fd73815e460 2023-11-24T07:59:36.195 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:36.190+0000 7fd7367fc700 1 -- 172.21.15.59:0/2851961996 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd7380cfc40 con 0x7fd7380d49d0 2023-11-24T07:59:36.195 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:36.190+0000 7fd7367fc700 1 --2- 172.21.15.59:0/2851961996 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd7380d49d0 0x7fd7380d2490 secure :-1 s=READY pgs=944 cs=0 l=1 rev1=1 crypto rx=0x7fd728009840 tx=0x7fd728009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5ba5ccd555bfdad2 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:59:36.195 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:36.191+0000 7fd735ffb700 1 -- 172.21.15.59:0/2851961996 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd728011830 con 0x7fd7380d49d0 2023-11-24T07:59:36.195 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:36.191+0000 7fd735ffb700 1 -- 172.21.15.59:0/2851961996 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd728011990 con 0x7fd7380d49d0 2023-11-24T07:59:36.196 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:36.191+0000 7fd735ffb700 1 -- 172.21.15.59:0/2851961996 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd728011d20 con 0x7fd7380d49d0 2023-11-24T07:59:36.196 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:36.191+0000 7fd73d791700 1 -- 172.21.15.59:0/2851961996 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd7380d49d0 msgr2=0x7fd7380d2490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:36.196 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:36.191+0000 7fd73d791700 1 --2- 172.21.15.59:0/2851961996 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd7380d49d0 0x7fd7380d2490 secure :-1 s=READY pgs=944 cs=0 l=1 rev1=1 crypto rx=0x7fd728009840 tx=0x7fd728009b50 comp rx=0 tx=0).stop 2023-11-24T07:59:36.196 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:36.191+0000 7fd73d791700 1 -- 172.21.15.59:0/2851961996 shutdown_connections 2023-11-24T07:59:36.196 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:36.191+0000 7fd73d791700 1 --2- 172.21.15.59:0/2851961996 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fd7380d29d0 0x7fd7380b40a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:36.196 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:36.191+0000 7fd73d791700 1 --2- 172.21.15.59:0/2851961996 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd7380d49d0 0x7fd7380d2490 unknown :-1 s=CLOSED pgs=944 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:36.196 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:36.191+0000 7fd73d791700 1 -- 172.21.15.59:0/2851961996 >> 172.21.15.59:0/2851961996 conn(0x7fd7380d3eb0 msgr2=0x7fd7380d42c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:59:36.196 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:36.191+0000 7fd73d791700 1 -- 172.21.15.59:0/2851961996 shutdown_connections 2023-11-24T07:59:36.196 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:36.191+0000 7fd73d791700 1 -- 172.21.15.59:0/2851961996 wait complete. 2023-11-24T07:59:36.199 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:36.193+0000 7fd73d791700 1 Processor -- start 2023-11-24T07:59:36.199 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:36.193+0000 7fd73d791700 1 -- start start 2023-11-24T07:59:36.199 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:36.193+0000 7fd73d791700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd7380d29d0 0x7fd73815f2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:36.200 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:36.193+0000 7fd73d791700 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fd7380d49d0 0x7fd738161050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:36.200 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:36.193+0000 7fd73d791700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd73815e460 0x7fd73815b150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:36.200 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:36.193+0000 7fd73d791700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7381f71e0 con 0x7fd73815e460 2023-11-24T07:59:36.200 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:36.193+0000 7fd73d791700 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7381f74c0 con 0x7fd7380d29d0 2023-11-24T07:59:36.200 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:36.193+0000 7fd73d791700 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd7381f77a0 con 0x7fd7380d49d0 2023-11-24T07:59:36.200 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:36.193+0000 7fd7377fe700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd73815e460 0x7fd73815b150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:36.201 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:36.193+0000 7fd7377fe700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd73815e460 0x7fd73815b150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:55362/0 (socket says 172.21.15.59:55362) 2023-11-24T07:59:36.201 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:36.193+0000 7fd7377fe700 1 -- 172.21.15.59:0/3406623772 learned_addr learned my addr 172.21.15.59:0/3406623772 (peer_addr_for_me v2:172.21.15.59:0/0) 2023-11-24T07:59:36.201 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:36.194+0000 7fd736ffd700 1 --2- 172.21.15.59:0/3406623772 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd7380d29d0 0x7fd73815f2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:36.201 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:36.194+0000 7fd7367fc700 1 --2- 172.21.15.59:0/3406623772 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fd7380d49d0 0x7fd738161050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:36.201 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:36.194+0000 7fd7377fe700 1 -- 172.21.15.59:0/3406623772 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fd7380d49d0 msgr2=0x7fd738161050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:36.201 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:36.194+0000 7fd7377fe700 1 --2- 172.21.15.59:0/3406623772 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fd7380d49d0 0x7fd738161050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:36.201 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:36.194+0000 7fd7377fe700 1 -- 172.21.15.59:0/3406623772 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd7380d29d0 msgr2=0x7fd73815f2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:36.201 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:36.194+0000 7fd7377fe700 1 --2- 172.21.15.59:0/3406623772 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd7380d29d0 0x7fd73815f2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:36.201 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:36.194+0000 7fd7377fe700 1 -- 172.21.15.59:0/3406623772 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd7280094a0 con 0x7fd73815e460 2023-11-24T07:59:36.201 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:36.194+0000 7fd7367fc700 1 --2- 172.21.15.59:0/3406623772 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fd7380d49d0 0x7fd738161050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-24T07:59:36.202 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:36.194+0000 7fd7377fe700 1 --2- 172.21.15.59:0/3406623772 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd73815e460 0x7fd73815b150 secure :-1 s=READY pgs=945 cs=0 l=1 rev1=1 crypto rx=0x7fd730007e40 tx=0x7fd73000e3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:59:36.202 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:36.194+0000 7fd71f7fe700 1 -- 172.21.15.59:0/3406623772 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd73000ede0 con 0x7fd73815e460 2023-11-24T07:59:36.202 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:36.194+0000 7fd71f7fe700 1 -- 172.21.15.59:0/3406623772 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd730014420 con 0x7fd73815e460 2023-11-24T07:59:36.202 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:36.194+0000 7fd73d791700 1 -- 172.21.15.59:0/3406623772 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd7381f7a80 con 0x7fd73815e460 2023-11-24T07:59:36.202 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:36.194+0000 7fd71f7fe700 1 -- 172.21.15.59:0/3406623772 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7300147d0 con 0x7fd73815e460 2023-11-24T07:59:36.202 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:36.194+0000 7fd73d791700 1 -- 172.21.15.59:0/3406623772 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd7380cc1d0 con 0x7fd73815e460 2023-11-24T07:59:36.202 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:36.195+0000 7fd73d791700 1 -- 172.21.15.59:0/3406623772 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd738067f60 con 0x7fd73815e460 2023-11-24T07:59:36.202 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:36.196+0000 7fd71f7fe700 1 -- 172.21.15.59:0/3406623772 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 11) v1 ==== 89741+0+0 (secure 0 0 0) 0x7fd730014930 con 0x7fd73815e460 2023-11-24T07:59:36.202 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:36.196+0000 7fd71f7fe700 1 --2- 172.21.15.59:0/3406623772 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7fd72406b470 0x7fd72406d930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:36.203 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:36.196+0000 7fd71f7fe700 1 -- 172.21.15.59:0/3406623772 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(75..75 src has 1..75) v4 ==== 5765+0+0 (secure 0 0 0) 0x7fd73008a6c0 con 0x7fd73815e460 2023-11-24T07:59:36.203 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:36.196+0000 7fd736ffd700 1 --2- 172.21.15.59:0/3406623772 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7fd72406b470 0x7fd72406d930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:36.203 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:36.197+0000 7fd736ffd700 1 --2- 172.21.15.59:0/3406623772 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7fd72406b470 0x7fd72406d930 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7fd72c000f60 tx=0x7fd72c007e70 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:59:36.203 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:36.201+0000 7fd71f7fe700 1 -- 172.21.15.59:0/3406623772 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fd730054fc0 con 0x7fd73815e460 2023-11-24T07:59:36.406 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:36.404+0000 7fd73d791700 1 -- 172.21.15.59:0/3406623772 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fd738161b20 con 0x7fd73815e460 2023-11-24T07:59:36.407 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:36.405+0000 7fd71f7fe700 1 -- 172.21.15.59:0/3406623772 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v75) v1 ==== 74+0+16546 (secure 0 0 0) 0x7fd7300585e0 con 0x7fd73815e460 2023-11-24T07:59:36.407 INFO:teuthology.orchestra.run.smithi059.stdout: 2023-11-24T07:59:36.408 INFO:teuthology.orchestra.run.smithi059.stdout:{"epoch":75,"fsid":"9f0e8741-b8f9-48d5-a2c3-09f45903303d","created":"2023-11-24T07:56:06.552729+0000","modified":"2023-11-24T07:59:34.419418+0000","last_up_change":"2023-11-24T07:56:17.372352+0000","last_in_change":"2023-11-24T07:56:09.073250+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":11,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-24T07:56:18.352823+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_metadata","create_time":"2023-11-24T07:59:11.900882+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"73","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_data","create_time":"2023-11-24T07:59:13.025749+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"75","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":3,"snap_epoch":75,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"6d8f9e9b-1782-4e41-928b-a39f2a8ebc06","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6800","nonce":3373878772},{"type":"v1","addr":"172.21.15.59:6801","nonce":3373878772}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6802","nonce":3373878772},{"type":"v1","addr":"172.21.15.59:6803","nonce":3373878772}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6806","nonce":3373878772},{"type":"v1","addr":"172.21.15.59:6807","nonce":3373878772}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6804","nonce":3373878772},{"type":"v1","addr":"172.21.15.59:6805","nonce":3373878772}]},"public_addr":"172.21.15.59:6801/3373878772","cluster_addr":"172.21.15.59:6803/3373878772","heartbeat_back_addr":"172.21.15.59:6807/3373878772","heartbeat_front_addr":"172.21.15.59:6805/3373878772","state":["exists","up"]},{"osd":1,"uuid":"e8d882aa-52e1-4c04-9e1e-c44ef9204a5f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6824","nonce":648258563},{"type":"v1","addr":"172.21.15.59:6825","nonce":648258563}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6826","nonce":648258563},{"type":"v1","addr":"172.21.15.59:6827","nonce":648258563}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6830","nonce":648258563},{"type":"v1","addr":"172.21.15.59:6831","nonce":648258563}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6828","nonce":648258563},{"type":"v1","addr":"172.21.15.59:6829","nonce":648258563}]},"public_addr":"172.21.15.59:6825/648258563","cluster_addr":"172.21.15.59:6827/648258563","heartbeat_back_addr":"172.21.15.59:6831/648258563","heartbeat_front_addr":"172.21.15.59:6829/648258563","state":["exists","up"]},{"osd":2,"uuid":"69eebfa0-695b-4b21-90b9-8a3537168a01","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6808","nonce":1840430654},{"type":"v1","addr":"172.21.15.59:6809","nonce":1840430654}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6810","nonce":1840430654},{"type":"v1","addr":"172.21.15.59:6811","nonce":1840430654}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6814","nonce":1840430654},{"type":"v1","addr":"172.21.15.59:6815","nonce":1840430654}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6812","nonce":1840430654},{"type":"v1","addr":"172.21.15.59:6813","nonce":1840430654}]},"public_addr":"172.21.15.59:6809/1840430654","cluster_addr":"172.21.15.59:6811/1840430654","heartbeat_back_addr":"172.21.15.59:6815/1840430654","heartbeat_front_addr":"172.21.15.59:6813/1840430654","state":["exists","up"]},{"osd":3,"uuid":"96fd77fc-f911-46ff-9445-6f0be2daa8f4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6816","nonce":2504025850},{"type":"v1","addr":"172.21.15.59:6817","nonce":2504025850}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6818","nonce":2504025850},{"type":"v1","addr":"172.21.15.59:6819","nonce":2504025850}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6822","nonce":2504025850},{"type":"v1","addr":"172.21.15.59:6823","nonce":2504025850}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6820","nonce":2504025850},{"type":"v1","addr":"172.21.15.59:6821","nonce":2504025850}]},"public_addr":"172.21.15.59:6817/2504025850","cluster_addr":"172.21.15.59:6819/2504025850","heartbeat_back_addr":"172.21.15.59:6823/2504025850","heartbeat_front_addr":"172.21.15.59:6821/2504025850","state":["exists","up"]},{"osd":4,"uuid":"7ac29fca-b5b2-4c95-9d69-2f164cbbc708","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6816","nonce":2382463431},{"type":"v1","addr":"172.21.15.76:6818","nonce":2382463431}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6820","nonce":2382463431},{"type":"v1","addr":"172.21.15.76:6822","nonce":2382463431}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6828","nonce":2382463431},{"type":"v1","addr":"172.21.15.76:6830","nonce":2382463431}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6824","nonce":2382463431},{"type":"v1","addr":"172.21.15.76:6826","nonce":2382463431}]},"public_addr":"172.21.15.76:6818/2382463431","cluster_addr":"172.21.15.76:6822/2382463431","heartbeat_back_addr":"172.21.15.76:6830/2382463431","heartbeat_front_addr":"172.21.15.76:6826/2382463431","state":["exists","up"]},{"osd":5,"uuid":"485bf711-3851-4238-a4f4-51d52e0a9c58","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6800","nonce":329334308},{"type":"v1","addr":"172.21.15.76:6802","nonce":329334308}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6805","nonce":329334308},{"type":"v1","addr":"172.21.15.76:6807","nonce":329334308}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6814","nonce":329334308},{"type":"v1","addr":"172.21.15.76:6815","nonce":329334308}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6810","nonce":329334308},{"type":"v1","addr":"172.21.15.76:6812","nonce":329334308}]},"public_addr":"172.21.15.76:6802/329334308","cluster_addr":"172.21.15.76:6807/329334308","heartbeat_back_addr":"172.21.15.76:6815/329334308","heartbeat_front_addr":"172.21.15.76:6812/329334308","state":["exists","up"]},{"osd":6,"uuid":"72b3f2fc-1e4a-4e32-968d-003647a0286f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6801","nonce":3946579410},{"type":"v1","addr":"172.21.15.76:6803","nonce":3946579410}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6804","nonce":3946579410},{"type":"v1","addr":"172.21.15.76:6806","nonce":3946579410}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6811","nonce":3946579410},{"type":"v1","addr":"172.21.15.76:6813","nonce":3946579410}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6808","nonce":3946579410},{"type":"v1","addr":"172.21.15.76:6809","nonce":3946579410}]},"public_addr":"172.21.15.76:6803/3946579410","cluster_addr":"172.21.15.76:6806/3946579410","heartbeat_back_addr":"172.21.15.76:6813/3946579410","heartbeat_front_addr":"172.21.15.76:6809/3946579410","state":["exists","up"]},{"osd":7,"uuid":"1ddceb90-6c5d-4d1a-ad56-99cb877f8c70","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6817","nonce":841761538},{"type":"v1","addr":"172.21.15.76:6819","nonce":841761538}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6821","nonce":841761538},{"type":"v1","addr":"172.21.15.76:6823","nonce":841761538}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6829","nonce":841761538},{"type":"v1","addr":"172.21.15.76:6831","nonce":841761538}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6825","nonce":841761538},{"type":"v1","addr":"172.21.15.76:6827","nonce":841761538}]},"public_addr":"172.21.15.76:6819/841761538","cluster_addr":"172.21.15.76:6823/841761538","heartbeat_back_addr":"172.21.15.76:6831/841761538","heartbeat_front_addr":"172.21.15.76:6827/841761538","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T07:56:14.294838+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T07:56:14.270779+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T07:56:14.231809+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T07:56:14.260983+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T07:56:14.302997+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T07:56:14.258600+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T07:56:14.251413+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T07:56:14.304727+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":11,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[{"pool":11,"snaps":[{"begin":2,"length":2}]}],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-24T07:59:36.409 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:36.407+0000 7fd73d791700 1 -- 172.21.15.59:0/3406623772 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7fd72406b470 msgr2=0x7fd72406d930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:36.410 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:36.407+0000 7fd73d791700 1 --2- 172.21.15.59:0/3406623772 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7fd72406b470 0x7fd72406d930 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7fd72c000f60 tx=0x7fd72c007e70 comp rx=0 tx=0).stop 2023-11-24T07:59:36.410 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:36.407+0000 7fd73d791700 1 -- 172.21.15.59:0/3406623772 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd73815e460 msgr2=0x7fd73815b150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:36.410 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:36.407+0000 7fd73d791700 1 --2- 172.21.15.59:0/3406623772 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd73815e460 0x7fd73815b150 secure :-1 s=READY pgs=945 cs=0 l=1 rev1=1 crypto rx=0x7fd730007e40 tx=0x7fd73000e3c0 comp rx=0 tx=0).stop 2023-11-24T07:59:36.410 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:36.407+0000 7fd73d791700 1 -- 172.21.15.59:0/3406623772 shutdown_connections 2023-11-24T07:59:36.410 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:36.407+0000 7fd73d791700 1 --2- 172.21.15.59:0/3406623772 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7fd72406b470 0x7fd72406d930 unknown :-1 s=CLOSED pgs=438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:36.410 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:36.407+0000 7fd73d791700 1 --2- 172.21.15.59:0/3406623772 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd73815e460 0x7fd73815b150 unknown :-1 s=CLOSED pgs=945 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:36.410 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:36.407+0000 7fd73d791700 1 --2- 172.21.15.59:0/3406623772 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd7380d29d0 0x7fd73815f2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:36.410 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:36.407+0000 7fd73d791700 1 --2- 172.21.15.59:0/3406623772 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fd7380d49d0 0x7fd738161050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:36.410 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:36.408+0000 7fd73d791700 1 -- 172.21.15.59:0/3406623772 >> 172.21.15.59:0/3406623772 conn(0x7fd7380d3eb0 msgr2=0x7fd73806cdf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:59:36.410 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:36.408+0000 7fd73d791700 1 -- 172.21.15.59:0/3406623772 shutdown_connections 2023-11-24T07:59:36.411 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:36.408+0000 7fd73d791700 1 -- 172.21.15.59:0/3406623772 wait complete. 2023-11-24T07:59:36.422 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-24T07:56:18.352823+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 10, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-24T07:59:11.900882+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '73', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 11, 'pool_name': 'cephfs_data', 'create_time': '2023-11-24T07:59:13.025749+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '75', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 3, 'snap_epoch': 75, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-24T07:59:36.422 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-24T07:59:36.549 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:36.548+0000 7f9d0e3cc700 1 Processor -- start 2023-11-24T07:59:36.550 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:36.548+0000 7f9d0e3cc700 1 -- start start 2023-11-24T07:59:36.552 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:36.549+0000 7f9d0e3cc700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f9d08072a00 0x7f9d0806db60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:36.552 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:36.549+0000 7f9d0e3cc700 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f9d0806e0a0 0x7f9d080d1010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:36.552 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:36.549+0000 7f9d0e3cc700 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9d080d0b80 con 0x7f9d080b4090 2023-11-24T07:59:36.552 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:36.549+0000 7f9d0e3cc700 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9d0815ed60 con 0x7f9d08072a00 2023-11-24T07:59:36.552 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:36.549+0000 7f9d0e3cc700 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9d0815f040 con 0x7f9d0806e0a0 2023-11-24T07:59:36.552 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:36.549+0000 7f9d077fe700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f9d08072a00 0x7f9d0806db60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:36.552 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:36.549+0000 7f9d07fff700 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7f9d080b4090 0x7f9d0815e510 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:58840/0 (socket says 172.21.15.59:58840) 2023-11-24T07:59:36.552 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:36.549+0000 7f9d077fe700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f9d08072a00 0x7f9d0806db60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:55368/0 (socket says 172.21.15.59:55368) 2023-11-24T07:59:36.552 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:36.549+0000 7f9d07fff700 1 -- 172.21.15.59:0/2806643105 learned_addr learned my addr 172.21.15.59:0/2806643105 (peer_addr_for_me v1:172.21.15.59:0/0) 2023-11-24T07:59:36.553 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:36.549+0000 7f9d0c969700 1 --2- 172.21.15.59:0/2806643105 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f9d0806e0a0 0x7f9d080d1010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:36.553 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:36.549+0000 7f9d077fe700 1 -- 172.21.15.59:0/2806643105 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f9d0806e0a0 msgr2=0x7f9d080d1010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T07:59:36.553 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:36.549+0000 7f9d077fe700 1 --2- 172.21.15.59:0/2806643105 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f9d0806e0a0 0x7f9d080d1010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:36.553 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:36.549+0000 7f9d077fe700 1 -- 172.21.15.59:0/2806643105 >> v1:172.21.15.59:6789/0 conn(0x7f9d080b4090 legacy=0x7f9d0815e510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:36.553 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:36.549+0000 7f9d077fe700 1 -- 172.21.15.59:0/2806643105 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9d0815f320 con 0x7f9d08072a00 2023-11-24T07:59:36.553 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:36.549+0000 7f9d06ffd700 1 -- 172.21.15.59:0/2806643105 <== mon.0 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2965112848 0 0) 0x7f9d080d0b80 con 0x7f9d080b4090 2023-11-24T07:59:36.553 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:36.549+0000 7f9d077fe700 1 --2- 172.21.15.59:0/2806643105 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9d08072a00 0x7f9d0806db60 secure :-1 s=READY pgs=947 cs=0 l=1 rev1=1 crypto rx=0x7f9cfc009840 tx=0x7f9cfc009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ad5545893d3df396 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:59:36.553 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:36.549+0000 7f9d06ffd700 1 -- 172.21.15.59:0/2806643105 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9cfc011830 con 0x7f9d08072a00 2023-11-24T07:59:36.553 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:36.549+0000 7f9d06ffd700 1 -- 172.21.15.59:0/2806643105 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9cfc011990 con 0x7f9d08072a00 2023-11-24T07:59:36.553 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:36.550+0000 7f9d06ffd700 1 -- 172.21.15.59:0/2806643105 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9cfc011d20 con 0x7f9d08072a00 2023-11-24T07:59:36.553 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:36.550+0000 7f9d0e3cc700 1 -- 172.21.15.59:0/2806643105 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9d08072a00 msgr2=0x7f9d0806db60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:36.553 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:36.550+0000 7f9d0e3cc700 1 --2- 172.21.15.59:0/2806643105 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9d08072a00 0x7f9d0806db60 secure :-1 s=READY pgs=947 cs=0 l=1 rev1=1 crypto rx=0x7f9cfc009840 tx=0x7f9cfc009b50 comp rx=0 tx=0).stop 2023-11-24T07:59:36.554 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:36.550+0000 7f9d0e3cc700 1 -- 172.21.15.59:0/2806643105 shutdown_connections 2023-11-24T07:59:36.554 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:36.550+0000 7f9d0e3cc700 1 --2- 172.21.15.59:0/2806643105 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f9d0806e0a0 0x7f9d080d1010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:36.554 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:36.550+0000 7f9d0e3cc700 1 --2- 172.21.15.59:0/2806643105 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9d08072a00 0x7f9d0806db60 unknown :-1 s=CLOSED pgs=947 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:36.554 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:36.550+0000 7f9d0e3cc700 1 -- 172.21.15.59:0/2806643105 >> 172.21.15.59:0/2806643105 conn(0x7f9d080b2740 msgr2=0x7f9d080b2b50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:59:36.554 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:36.550+0000 7f9d0e3cc700 1 -- 172.21.15.59:0/2806643105 shutdown_connections 2023-11-24T07:59:36.554 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:36.550+0000 7f9d0e3cc700 1 -- 172.21.15.59:0/2806643105 wait complete. 2023-11-24T07:59:36.557 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:36.551+0000 7f9d0e3cc700 1 Processor -- start 2023-11-24T07:59:36.557 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:36.551+0000 7f9d0e3cc700 1 -- start start 2023-11-24T07:59:36.557 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:36.552+0000 7f9d0e3cc700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9d0806e0a0 0x7f9d08143600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:36.557 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:36.552+0000 7f9d0e3cc700 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f9d08072a00 0x7f9d08160db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:36.557 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:36.552+0000 7f9d0e3cc700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9d080b4090 0x7f9d080b1720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:36.558 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:36.552+0000 7f9d0e3cc700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d0815b9e0 con 0x7f9d080b4090 2023-11-24T07:59:36.558 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:36.552+0000 7f9d0e3cc700 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d0815b420 con 0x7f9d0806e0a0 2023-11-24T07:59:36.558 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:36.552+0000 7f9d0e3cc700 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9d0815b700 con 0x7f9d08072a00 2023-11-24T07:59:36.558 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:36.552+0000 7f9d0c969700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9d080b4090 0x7f9d080b1720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:36.558 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:36.552+0000 7f9d0c969700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9d080b4090 0x7f9d080b1720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:55376/0 (socket says 172.21.15.59:55376) 2023-11-24T07:59:36.559 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:36.552+0000 7f9d0c969700 1 -- 172.21.15.59:0/1278076816 learned_addr learned my addr 172.21.15.59:0/1278076816 (peer_addr_for_me v2:172.21.15.59:0/0) 2023-11-24T07:59:36.559 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:36.553+0000 7f9d0c969700 1 -- 172.21.15.59:0/1278076816 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f9d08072a00 msgr2=0x7f9d08160db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:36.559 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:36.553+0000 7f9d077fe700 1 --2- 172.21.15.59:0/1278076816 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f9d08072a00 0x7f9d08160db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:36.559 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:36.553+0000 7f9d07fff700 1 --2- 172.21.15.59:0/1278076816 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9d0806e0a0 0x7f9d08143600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:36.559 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:36.553+0000 7f9d0c969700 1 --2- 172.21.15.59:0/1278076816 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f9d08072a00 0x7f9d08160db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:36.559 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:36.553+0000 7f9d0c969700 1 -- 172.21.15.59:0/1278076816 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9d0806e0a0 msgr2=0x7f9d08143600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:36.560 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:36.553+0000 7f9d0c969700 1 --2- 172.21.15.59:0/1278076816 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9d0806e0a0 0x7f9d08143600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:36.560 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:36.553+0000 7f9d0c969700 1 -- 172.21.15.59:0/1278076816 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9cfc0094a0 con 0x7f9d080b4090 2023-11-24T07:59:36.560 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:36.553+0000 7f9d0c969700 1 --2- 172.21.15.59:0/1278076816 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9d080b4090 0x7f9d080b1720 secure :-1 s=READY pgs=948 cs=0 l=1 rev1=1 crypto rx=0x7f9d00007e40 tx=0x7f9d0000e3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:59:36.560 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:36.553+0000 7f9d04ff9700 1 -- 172.21.15.59:0/1278076816 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d0000eea0 con 0x7f9d080b4090 2023-11-24T07:59:36.560 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:36.553+0000 7f9d04ff9700 1 -- 172.21.15.59:0/1278076816 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9d00014420 con 0x7f9d080b4090 2023-11-24T07:59:36.561 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:36.553+0000 7f9d04ff9700 1 -- 172.21.15.59:0/1278076816 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d000147b0 con 0x7f9d080b4090 2023-11-24T07:59:36.561 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:36.553+0000 7f9d0e3cc700 1 -- 172.21.15.59:0/1278076816 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9d0815bed0 con 0x7f9d080b4090 2023-11-24T07:59:36.561 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:36.553+0000 7f9d0e3cc700 1 -- 172.21.15.59:0/1278076816 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9d0815b040 con 0x7f9d080b4090 2023-11-24T07:59:36.561 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:36.554+0000 7f9d0e3cc700 1 -- 172.21.15.59:0/1278076816 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9d08075480 con 0x7f9d080b4090 2023-11-24T07:59:36.561 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:36.554+0000 7f9d04ff9700 1 -- 172.21.15.59:0/1278076816 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 11) v1 ==== 89741+0+0 (secure 0 0 0) 0x7f9d00014910 con 0x7f9d080b4090 2023-11-24T07:59:36.562 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:36.555+0000 7f9d04ff9700 1 --2- 172.21.15.59:0/1278076816 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f9cf406b470 0x7f9cf406d930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:36.562 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:36.555+0000 7f9d04ff9700 1 -- 172.21.15.59:0/1278076816 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(75..75 src has 1..75) v4 ==== 5765+0+0 (secure 0 0 0) 0x7f9d0008a6e0 con 0x7f9d080b4090 2023-11-24T07:59:36.562 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:36.555+0000 7f9d07fff700 1 --2- 172.21.15.59:0/1278076816 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f9cf406b470 0x7f9cf406d930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:36.563 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:36.556+0000 7f9d07fff700 1 --2- 172.21.15.59:0/1278076816 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f9cf406b470 0x7f9cf406d930 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f9cf8000f60 tx=0x7f9cf8009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:59:36.563 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:36.559+0000 7f9d04ff9700 1 -- 172.21.15.59:0/1278076816 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f9d00055070 con 0x7f9d080b4090 2023-11-24T07:59:36.760 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:36.758+0000 7f9d0e3cc700 1 -- 172.21.15.59:0/1278076816 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f9d08072650 con 0x7f9d080b4090 2023-11-24T07:59:36.760 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:36.759+0000 7f9d04ff9700 1 -- 172.21.15.59:0/1278076816 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v75) v1 ==== 93+0+10 (secure 0 0 0) 0x7f9d00058690 con 0x7f9d080b4090 2023-11-24T07:59:36.761 INFO:teuthology.orchestra.run.smithi059.stdout:pg_num: 1 2023-11-24T07:59:36.763 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:36.761+0000 7f9d0e3cc700 1 -- 172.21.15.59:0/1278076816 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f9cf406b470 msgr2=0x7f9cf406d930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:36.763 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:36.761+0000 7f9d0e3cc700 1 --2- 172.21.15.59:0/1278076816 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f9cf406b470 0x7f9cf406d930 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f9cf8000f60 tx=0x7f9cf8009040 comp rx=0 tx=0).stop 2023-11-24T07:59:36.763 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:36.761+0000 7f9d0e3cc700 1 -- 172.21.15.59:0/1278076816 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9d080b4090 msgr2=0x7f9d080b1720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:36.763 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:36.761+0000 7f9d0e3cc700 1 --2- 172.21.15.59:0/1278076816 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9d080b4090 0x7f9d080b1720 secure :-1 s=READY pgs=948 cs=0 l=1 rev1=1 crypto rx=0x7f9d00007e40 tx=0x7f9d0000e3c0 comp rx=0 tx=0).stop 2023-11-24T07:59:36.763 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:36.761+0000 7f9d0e3cc700 1 -- 172.21.15.59:0/1278076816 shutdown_connections 2023-11-24T07:59:36.763 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:36.761+0000 7f9d0e3cc700 1 --2- 172.21.15.59:0/1278076816 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f9cf406b470 0x7f9cf406d930 unknown :-1 s=CLOSED pgs=439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:36.763 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:36.761+0000 7f9d0e3cc700 1 --2- 172.21.15.59:0/1278076816 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9d080b4090 0x7f9d080b1720 unknown :-1 s=CLOSED pgs=948 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:36.763 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:36.761+0000 7f9d0e3cc700 1 --2- 172.21.15.59:0/1278076816 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9d0806e0a0 0x7f9d08143600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:36.763 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:36.761+0000 7f9d0e3cc700 1 --2- 172.21.15.59:0/1278076816 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f9d08072a00 0x7f9d08160db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:36.763 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:36.761+0000 7f9d0e3cc700 1 -- 172.21.15.59:0/1278076816 >> 172.21.15.59:0/1278076816 conn(0x7f9d080b2740 msgr2=0x7f9d081583b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:59:36.764 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:36.761+0000 7f9d0e3cc700 1 -- 172.21.15.59:0/1278076816 shutdown_connections 2023-11-24T07:59:36.764 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:36.761+0000 7f9d0e3cc700 1 -- 172.21.15.59:0/1278076816 wait complete. 2023-11-24T07:59:36.775 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-24T07:59:36.902 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:36.900+0000 7f38d1b06700 1 Processor -- start 2023-11-24T07:59:36.902 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:36.900+0000 7f38d1b06700 1 -- start start 2023-11-24T07:59:36.904 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:36.901+0000 7f38d1b06700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f38cc143ae0 0x7f38cc0b51b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:36.904 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:36.901+0000 7f38d1b06700 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f38cc1440f0 0x7f38cc071bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:36.904 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:36.901+0000 7f38d1b06700 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f38cc143ec0 con 0x7f38cc0720f0 2023-11-24T07:59:36.905 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:36.901+0000 7f38d1b06700 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f38cc0b40a0 con 0x7f38cc143ae0 2023-11-24T07:59:36.905 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:36.901+0000 7f38d1b06700 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f38cc072560 con 0x7f38cc1440f0 2023-11-24T07:59:36.905 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:36.901+0000 7f38cb7fe700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f38cc143ae0 0x7f38cc0b51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:36.905 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:36.901+0000 7f38cb7fe700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f38cc143ae0 0x7f38cc0b51b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:55380/0 (socket says 172.21.15.59:55380) 2023-11-24T07:59:36.905 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:36.901+0000 7f38cb7fe700 1 -- 172.21.15.59:0/3361729393 learned_addr learned my addr 172.21.15.59:0/3361729393 (peer_addr_for_me v2:172.21.15.59:0/0) 2023-11-24T07:59:36.905 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:36.901+0000 7f38caffd700 1 --2- 172.21.15.59:0/3361729393 >> v2:172.21.15.76:3300/0 conn(0x7f38cc1440f0 0x7f38cc071bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:36.905 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:36.902+0000 7f38cb7fe700 1 -- 172.21.15.59:0/3361729393 >> v2:172.21.15.76:3300/0 conn(0x7f38cc1440f0 msgr2=0x7f38cc071bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T07:59:36.905 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:36.902+0000 7f38cb7fe700 1 --2- 172.21.15.59:0/3361729393 >> v2:172.21.15.76:3300/0 conn(0x7f38cc1440f0 0x7f38cc071bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:36.905 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:36.902+0000 7f38cb7fe700 1 -- 172.21.15.59:0/3361729393 >> v1:172.21.15.76:6789/0 conn(0x7f38cc0720f0 legacy=0x7f38cc15e8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:36.905 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:36.902+0000 7f38cb7fe700 1 -- 172.21.15.59:0/3361729393 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f38cc0b3ae0 con 0x7f38cc143ae0 2023-11-24T07:59:36.905 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:36.902+0000 7f38cb7fe700 1 --2- 172.21.15.59:0/3361729393 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f38cc143ae0 0x7f38cc0b51b0 secure :-1 s=READY pgs=949 cs=0 l=1 rev1=1 crypto rx=0x7f38cc0d1b00 tx=0x7f38bc009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=df2fcd2ef9d3d202 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:59:36.905 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:36.902+0000 7f38ca7fc700 1 -- 172.21.15.59:0/3361729393 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38bc011830 con 0x7f38cc143ae0 2023-11-24T07:59:36.906 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:36.902+0000 7f38ca7fc700 1 -- 172.21.15.59:0/3361729393 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f38bc011990 con 0x7f38cc143ae0 2023-11-24T07:59:36.906 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:36.902+0000 7f38ca7fc700 1 -- 172.21.15.59:0/3361729393 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38bc011d20 con 0x7f38cc143ae0 2023-11-24T07:59:36.906 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:36.902+0000 7f38d1b06700 1 -- 172.21.15.59:0/3361729393 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f38cc143ae0 msgr2=0x7f38cc0b51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:36.906 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:36.902+0000 7f38d1b06700 1 --2- 172.21.15.59:0/3361729393 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f38cc143ae0 0x7f38cc0b51b0 secure :-1 s=READY pgs=949 cs=0 l=1 rev1=1 crypto rx=0x7f38cc0d1b00 tx=0x7f38bc009c00 comp rx=0 tx=0).stop 2023-11-24T07:59:36.906 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:36.903+0000 7f38d1b06700 1 -- 172.21.15.59:0/3361729393 shutdown_connections 2023-11-24T07:59:36.906 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:36.903+0000 7f38d1b06700 1 --2- 172.21.15.59:0/3361729393 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f38cc143ae0 0x7f38cc0b51b0 unknown :-1 s=CLOSED pgs=949 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:36.906 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:36.903+0000 7f38d1b06700 1 --2- 172.21.15.59:0/3361729393 >> v2:172.21.15.76:3300/0 conn(0x7f38cc1440f0 0x7f38cc071bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:36.906 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:36.903+0000 7f38d1b06700 1 -- 172.21.15.59:0/3361729393 >> 172.21.15.59:0/3361729393 conn(0x7f38cc0b2740 msgr2=0x7f38cc0b2b20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:59:36.906 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:36.903+0000 7f38d1b06700 1 -- 172.21.15.59:0/3361729393 shutdown_connections 2023-11-24T07:59:36.906 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:36.903+0000 7f38d1b06700 1 -- 172.21.15.59:0/3361729393 wait complete. 2023-11-24T07:59:36.909 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:36.904+0000 7f38d1b06700 1 Processor -- start 2023-11-24T07:59:36.910 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:36.904+0000 7f38d1b06700 1 -- start start 2023-11-24T07:59:36.910 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:36.905+0000 7f38d1b06700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f38cc0720f0 0x7f38cc13f4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:36.910 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:36.905+0000 7f38d1b06700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f38cc143ae0 0x7f38cc1f5e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:36.910 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:36.905+0000 7f38d1b06700 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f38cc1440f0 0x7f38cc1f6340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:36.910 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:36.905+0000 7f38d1b06700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38cc136240 con 0x7f38cc0720f0 2023-11-24T07:59:36.910 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:36.905+0000 7f38d1b06700 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38cc136b40 con 0x7f38cc143ae0 2023-11-24T07:59:36.910 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:36.905+0000 7f38d1b06700 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f38cc13b6f0 con 0x7f38cc1440f0 2023-11-24T07:59:36.910 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:36.905+0000 7f38cb7fe700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f38cc0720f0 0x7f38cc13f4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:36.910 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:36.905+0000 7f38cb7fe700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f38cc0720f0 0x7f38cc13f4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:55386/0 (socket says 172.21.15.59:55386) 2023-11-24T07:59:36.910 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:36.905+0000 7f38cb7fe700 1 -- 172.21.15.59:0/3476921713 learned_addr learned my addr 172.21.15.59:0/3476921713 (peer_addr_for_me v2:172.21.15.59:0/0) 2023-11-24T07:59:36.911 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:36.905+0000 7f38cbfff700 1 --2- 172.21.15.59:0/3476921713 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f38cc1440f0 0x7f38cc1f6340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:36.911 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:36.905+0000 7f38caffd700 1 --2- 172.21.15.59:0/3476921713 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f38cc143ae0 0x7f38cc1f5e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:36.911 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:36.905+0000 7f38cb7fe700 1 -- 172.21.15.59:0/3476921713 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f38cc1440f0 msgr2=0x7f38cc1f6340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:36.911 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:36.905+0000 7f38cb7fe700 1 --2- 172.21.15.59:0/3476921713 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f38cc1440f0 0x7f38cc1f6340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:36.911 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:36.905+0000 7f38cb7fe700 1 -- 172.21.15.59:0/3476921713 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f38cc143ae0 msgr2=0x7f38cc1f5e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:36.911 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:36.905+0000 7f38cb7fe700 1 --2- 172.21.15.59:0/3476921713 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f38cc143ae0 0x7f38cc1f5e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:36.911 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:36.905+0000 7f38cb7fe700 1 -- 172.21.15.59:0/3476921713 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f38bc009550 con 0x7f38cc0720f0 2023-11-24T07:59:36.911 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:36.906+0000 7f38cb7fe700 1 --2- 172.21.15.59:0/3476921713 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f38cc0720f0 0x7f38cc13f4b0 secure :-1 s=READY pgs=950 cs=0 l=1 rev1=1 crypto rx=0x7f38bc009c40 tx=0x7f38bc004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:59:36.912 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:36.906+0000 7f38c3fff700 1 -- 172.21.15.59:0/3476921713 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38bc0036a0 con 0x7f38cc0720f0 2023-11-24T07:59:36.912 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:36.906+0000 7f38c3fff700 1 -- 172.21.15.59:0/3476921713 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f38bc01e070 con 0x7f38cc0720f0 2023-11-24T07:59:36.912 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:36.906+0000 7f38d1b06700 1 -- 172.21.15.59:0/3476921713 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f38cc13b9d0 con 0x7f38cc0720f0 2023-11-24T07:59:36.912 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:36.906+0000 7f38c3fff700 1 -- 172.21.15.59:0/3476921713 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38bc003a00 con 0x7f38cc0720f0 2023-11-24T07:59:36.912 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:36.906+0000 7f38d1b06700 1 -- 172.21.15.59:0/3476921713 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f38cc13bf90 con 0x7f38cc0720f0 2023-11-24T07:59:36.912 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:36.907+0000 7f38d1b06700 1 -- 172.21.15.59:0/3476921713 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f38cc13c120 con 0x7f38cc0720f0 2023-11-24T07:59:36.912 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:36.907+0000 7f38c3fff700 1 -- 172.21.15.59:0/3476921713 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 11) v1 ==== 89741+0+0 (secure 0 0 0) 0x7f38bc003b60 con 0x7f38cc0720f0 2023-11-24T07:59:36.912 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:36.908+0000 7f38c3fff700 1 --2- 172.21.15.59:0/3476921713 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f38ac06b470 0x7f38ac06d930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:36.912 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:36.908+0000 7f38c3fff700 1 -- 172.21.15.59:0/3476921713 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(75..75 src has 1..75) v4 ==== 5765+0+0 (secure 0 0 0) 0x7f38bc08af70 con 0x7f38cc0720f0 2023-11-24T07:59:36.912 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:36.908+0000 7f38caffd700 1 --2- 172.21.15.59:0/3476921713 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f38ac06b470 0x7f38ac06d930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:36.913 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:36.908+0000 7f38caffd700 1 --2- 172.21.15.59:0/3476921713 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f38ac06b470 0x7f38ac06d930 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f38cc151e60 tx=0x7f38b4008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:59:36.914 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:36.912+0000 7f38c3fff700 1 -- 172.21.15.59:0/3476921713 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f38bc055800 con 0x7f38cc0720f0 2023-11-24T07:59:37.116 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:37.114+0000 7f38d1b06700 1 -- 172.21.15.59:0/3476921713 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f38cc075480 con 0x7f38cc0720f0 2023-11-24T07:59:37.116 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:37.114+0000 7f38c3fff700 1 -- 172.21.15.59:0/3476921713 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v75) v1 ==== 104+0+11 (secure 0 0 0) 0x7f38bc020020 con 0x7f38cc0720f0 2023-11-24T07:59:37.116 INFO:teuthology.orchestra.run.smithi059.stdout:pg_num: 32 2023-11-24T07:59:37.118 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:37.116+0000 7f38d1b06700 1 -- 172.21.15.59:0/3476921713 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f38ac06b470 msgr2=0x7f38ac06d930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:37.119 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:37.116+0000 7f38d1b06700 1 --2- 172.21.15.59:0/3476921713 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f38ac06b470 0x7f38ac06d930 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f38cc151e60 tx=0x7f38b4008040 comp rx=0 tx=0).stop 2023-11-24T07:59:37.119 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:37.116+0000 7f38d1b06700 1 -- 172.21.15.59:0/3476921713 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f38cc0720f0 msgr2=0x7f38cc13f4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:37.119 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:37.116+0000 7f38d1b06700 1 --2- 172.21.15.59:0/3476921713 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f38cc0720f0 0x7f38cc13f4b0 secure :-1 s=READY pgs=950 cs=0 l=1 rev1=1 crypto rx=0x7f38bc009c40 tx=0x7f38bc004750 comp rx=0 tx=0).stop 2023-11-24T07:59:37.119 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:37.117+0000 7f38d1b06700 1 -- 172.21.15.59:0/3476921713 shutdown_connections 2023-11-24T07:59:37.119 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:37.117+0000 7f38d1b06700 1 --2- 172.21.15.59:0/3476921713 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f38ac06b470 0x7f38ac06d930 unknown :-1 s=CLOSED pgs=440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:37.119 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:37.117+0000 7f38d1b06700 1 --2- 172.21.15.59:0/3476921713 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f38cc1440f0 0x7f38cc1f6340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:37.119 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:37.117+0000 7f38d1b06700 1 --2- 172.21.15.59:0/3476921713 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f38cc0720f0 0x7f38cc13f4b0 unknown :-1 s=CLOSED pgs=950 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:37.119 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:37.117+0000 7f38d1b06700 1 --2- 172.21.15.59:0/3476921713 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f38cc143ae0 0x7f38cc1f5e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:37.120 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:37.117+0000 7f38d1b06700 1 -- 172.21.15.59:0/3476921713 >> 172.21.15.59:0/3476921713 conn(0x7f38cc0b2740 msgr2=0x7f38cc0cf2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:59:37.120 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:37.117+0000 7f38d1b06700 1 -- 172.21.15.59:0/3476921713 shutdown_connections 2023-11-24T07:59:37.120 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:37.117+0000 7f38d1b06700 1 -- 172.21.15.59:0/3476921713 wait complete. 2023-11-24T07:59:37.130 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-24T07:59:37.259 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:37.257+0000 7f8fa8ebe700 1 Processor -- start 2023-11-24T07:59:37.259 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:37.257+0000 7f8fa8ebe700 1 -- start start 2023-11-24T07:59:37.261 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:37.258+0000 7f8fa8ebe700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f8fa40ceb00 0x7f8fa40cfb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:37.261 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:37.258+0000 7f8fa8ebe700 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f8fa40aefc0 0x7f8fa40af450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:37.261 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:37.258+0000 7f8fa8ebe700 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8fa40aeb30 con 0x7f8fa40b0c40 2023-11-24T07:59:37.261 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:37.258+0000 7f8fa8ebe700 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8fa406d9e0 con 0x7f8fa40ceb00 2023-11-24T07:59:37.261 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:37.258+0000 7f8fa8ebe700 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8fa406d420 con 0x7f8fa40aefc0 2023-11-24T07:59:37.261 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:37.258+0000 7f8fa259c700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f8fa40ceb00 0x7f8fa40cfb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:37.261 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:37.258+0000 7f8fa1d9b700 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7f8fa40b0c40 0x7f8fa40d0050 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:58868/0 (socket says 172.21.15.59:58868) 2023-11-24T07:59:37.261 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:37.258+0000 7f8fa259c700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f8fa40ceb00 0x7f8fa40cfb10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:55394/0 (socket says 172.21.15.59:55394) 2023-11-24T07:59:37.261 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:37.258+0000 7f8fa1d9b700 1 -- 172.21.15.59:0/4213972263 learned_addr learned my addr 172.21.15.59:0/4213972263 (peer_addr_for_me v1:172.21.15.59:0/0) 2023-11-24T07:59:37.262 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:37.258+0000 7f8fa2d9d700 1 --2- 172.21.15.59:0/4213972263 >> v2:172.21.15.76:3300/0 conn(0x7f8fa40aefc0 0x7f8fa40af450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:37.262 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:37.258+0000 7f8fa259c700 1 -- 172.21.15.59:0/4213972263 >> v2:172.21.15.76:3300/0 conn(0x7f8fa40aefc0 msgr2=0x7f8fa40af450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T07:59:37.262 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:37.258+0000 7f8fa259c700 1 --2- 172.21.15.59:0/4213972263 >> v2:172.21.15.76:3300/0 conn(0x7f8fa40aefc0 0x7f8fa40af450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:37.262 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:37.258+0000 7f8fa259c700 1 -- 172.21.15.59:0/4213972263 >> v1:172.21.15.59:6789/0 conn(0x7f8fa40b0c40 legacy=0x7f8fa40d0050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:37.262 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:37.258+0000 7f8fa259c700 1 -- 172.21.15.59:0/4213972263 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8fa406d700 con 0x7f8fa40ceb00 2023-11-24T07:59:37.262 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:37.258+0000 7f8fa159a700 1 -- 172.21.15.59:0/4213972263 <== mon.0 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3119569287 0 0) 0x7f8fa40aeb30 con 0x7f8fa40b0c40 2023-11-24T07:59:37.262 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:37.258+0000 7f8fa2d9d700 1 --2- 172.21.15.59:0/4213972263 >> v2:172.21.15.76:3300/0 conn(0x7f8fa40aefc0 0x7f8fa40af450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T07:59:37.262 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:37.259+0000 7f8fa259c700 1 --2- 172.21.15.59:0/4213972263 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8fa40ceb00 0x7f8fa40cfb10 secure :-1 s=READY pgs=952 cs=0 l=1 rev1=1 crypto rx=0x7f8fa4071640 tx=0x7f8f98009b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f2b2b27298fbf7ae server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:59:37.262 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:37.259+0000 7f8fa159a700 1 -- 172.21.15.59:0/4213972263 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f98011830 con 0x7f8fa40ceb00 2023-11-24T07:59:37.262 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:37.259+0000 7f8fa159a700 1 -- 172.21.15.59:0/4213972263 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8f98011990 con 0x7f8fa40ceb00 2023-11-24T07:59:37.262 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:37.259+0000 7f8fa159a700 1 -- 172.21.15.59:0/4213972263 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f98011d20 con 0x7f8fa40ceb00 2023-11-24T07:59:37.263 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:37.259+0000 7f8fa8ebe700 1 -- 172.21.15.59:0/4213972263 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8fa40ceb00 msgr2=0x7f8fa40cfb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:37.263 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:37.259+0000 7f8fa8ebe700 1 --2- 172.21.15.59:0/4213972263 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8fa40ceb00 0x7f8fa40cfb10 secure :-1 s=READY pgs=952 cs=0 l=1 rev1=1 crypto rx=0x7f8fa4071640 tx=0x7f8f98009b20 comp rx=0 tx=0).stop 2023-11-24T07:59:37.263 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:37.259+0000 7f8fa8ebe700 1 -- 172.21.15.59:0/4213972263 shutdown_connections 2023-11-24T07:59:37.263 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:37.259+0000 7f8fa8ebe700 1 --2- 172.21.15.59:0/4213972263 >> v2:172.21.15.76:3300/0 conn(0x7f8fa40aefc0 0x7f8fa40af450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:37.263 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:37.259+0000 7f8fa8ebe700 1 --2- 172.21.15.59:0/4213972263 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8fa40ceb00 0x7f8fa40cfb10 unknown :-1 s=CLOSED pgs=952 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:37.263 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:37.259+0000 7f8fa8ebe700 1 -- 172.21.15.59:0/4213972263 >> 172.21.15.59:0/4213972263 conn(0x7f8fa40b3720 msgr2=0x7f8fa40b3b30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:59:37.263 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:37.259+0000 7f8fa8ebe700 1 -- 172.21.15.59:0/4213972263 shutdown_connections 2023-11-24T07:59:37.263 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:37.259+0000 7f8fa8ebe700 1 -- 172.21.15.59:0/4213972263 wait complete. 2023-11-24T07:59:37.266 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:37.261+0000 7f8fa8ebe700 1 Processor -- start 2023-11-24T07:59:37.266 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:37.261+0000 7f8fa8ebe700 1 -- start start 2023-11-24T07:59:37.266 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:37.261+0000 7f8fa8ebe700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8fa40aefc0 0x7f8fa415a2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:37.266 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:37.261+0000 7f8fa8ebe700 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f8fa40b0c40 0x7f8fa40b6c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:37.266 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:37.261+0000 7f8fa8ebe700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8fa40ceb00 0x7f8fa40c6350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:37.266 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:37.261+0000 7f8fa8ebe700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8fa40b79b0 con 0x7f8fa40ceb00 2023-11-24T07:59:37.266 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:37.261+0000 7f8fa8ebe700 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8fa40c5ec0 con 0x7f8fa40aefc0 2023-11-24T07:59:37.266 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:37.261+0000 7f8fa8ebe700 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8fa40b76d0 con 0x7f8fa40b0c40 2023-11-24T07:59:37.266 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:37.262+0000 7f8fa2d9d700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8fa40ceb00 0x7f8fa40c6350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:37.266 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:37.262+0000 7f8fa2d9d700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8fa40ceb00 0x7f8fa40c6350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:55404/0 (socket says 172.21.15.59:55404) 2023-11-24T07:59:37.267 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:37.262+0000 7f8fa2d9d700 1 -- 172.21.15.59:0/940134909 learned_addr learned my addr 172.21.15.59:0/940134909 (peer_addr_for_me v2:172.21.15.59:0/0) 2023-11-24T07:59:37.267 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:37.262+0000 7f8fa259c700 1 --2- 172.21.15.59:0/940134909 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8fa40aefc0 0x7f8fa415a2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:37.267 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:37.262+0000 7f8fa1d9b700 1 --2- 172.21.15.59:0/940134909 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f8fa40b0c40 0x7f8fa40b6c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:37.267 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:37.262+0000 7f8fa2d9d700 1 -- 172.21.15.59:0/940134909 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f8fa40b0c40 msgr2=0x7f8fa40b6c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:37.267 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:37.262+0000 7f8fa2d9d700 1 --2- 172.21.15.59:0/940134909 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f8fa40b0c40 0x7f8fa40b6c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:37.267 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:37.262+0000 7f8fa2d9d700 1 -- 172.21.15.59:0/940134909 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8fa40aefc0 msgr2=0x7f8fa415a2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:37.267 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:37.262+0000 7f8fa2d9d700 1 --2- 172.21.15.59:0/940134909 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8fa40aefc0 0x7f8fa415a2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:37.267 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:37.262+0000 7f8fa2d9d700 1 -- 172.21.15.59:0/940134909 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8f980094a0 con 0x7f8fa40ceb00 2023-11-24T07:59:37.267 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:37.262+0000 7f8fa259c700 1 --2- 172.21.15.59:0/940134909 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8fa40aefc0 0x7f8fa415a2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T07:59:37.267 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:37.262+0000 7f8fa2d9d700 1 --2- 172.21.15.59:0/940134909 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8fa40ceb00 0x7f8fa40c6350 secure :-1 s=READY pgs=953 cs=0 l=1 rev1=1 crypto rx=0x7f8f9c00cef0 tx=0x7f8f9c00a340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:59:37.267 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:37.262+0000 7f8f92ffd700 1 -- 172.21.15.59:0/940134909 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f9c00ae50 con 0x7f8fa40ceb00 2023-11-24T07:59:37.267 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:37.262+0000 7f8f92ffd700 1 -- 172.21.15.59:0/940134909 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8f9c015440 con 0x7f8fa40ceb00 2023-11-24T07:59:37.268 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:37.262+0000 7f8fa8ebe700 1 -- 172.21.15.59:0/940134909 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8fa40c5870 con 0x7f8fa40ceb00 2023-11-24T07:59:37.268 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:37.262+0000 7f8f92ffd700 1 -- 172.21.15.59:0/940134909 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f9c0157f0 con 0x7f8fa40ceb00 2023-11-24T07:59:37.268 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:37.262+0000 7f8fa8ebe700 1 -- 172.21.15.59:0/940134909 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8fa415a800 con 0x7f8fa40ceb00 2023-11-24T07:59:37.268 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:37.263+0000 7f8fa8ebe700 1 -- 172.21.15.59:0/940134909 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8fa40d11f0 con 0x7f8fa40ceb00 2023-11-24T07:59:37.268 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:37.264+0000 7f8f92ffd700 1 -- 172.21.15.59:0/940134909 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 11) v1 ==== 89741+0+0 (secure 0 0 0) 0x7f8f9c00d430 con 0x7f8fa40ceb00 2023-11-24T07:59:37.268 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:37.264+0000 7f8f92ffd700 1 --2- 172.21.15.59:0/940134909 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f8f8406b470 0x7f8f8406d930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:37.268 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:37.264+0000 7f8f92ffd700 1 -- 172.21.15.59:0/940134909 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(75..75 src has 1..75) v4 ==== 5765+0+0 (secure 0 0 0) 0x7f8f9c08a610 con 0x7f8fa40ceb00 2023-11-24T07:59:37.268 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:37.264+0000 7f8fa259c700 1 --2- 172.21.15.59:0/940134909 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f8f8406b470 0x7f8f8406d930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:37.268 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:37.265+0000 7f8fa259c700 1 --2- 172.21.15.59:0/940134909 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f8f8406b470 0x7f8f8406d930 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f8fa406dee0 tx=0x7f8f98008030 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:59:37.270 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:37.269+0000 7f8f92ffd700 1 -- 172.21.15.59:0/940134909 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f8f9c054ea0 con 0x7f8fa40ceb00 2023-11-24T07:59:37.472 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:37.470+0000 7f8fa8ebe700 1 -- 172.21.15.59:0/940134909 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f8fa4075480 con 0x7f8fa40ceb00 2023-11-24T07:59:37.472 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:37.471+0000 7f8f92ffd700 1 -- 172.21.15.59:0/940134909 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v75) v1 ==== 100+0+11 (secure 0 0 0) 0x7f8f9c01a020 con 0x7f8fa40ceb00 2023-11-24T07:59:37.473 INFO:teuthology.orchestra.run.smithi059.stdout:pg_num: 64 2023-11-24T07:59:37.475 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:37.473+0000 7f8fa8ebe700 1 -- 172.21.15.59:0/940134909 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f8f8406b470 msgr2=0x7f8f8406d930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:37.475 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:37.473+0000 7f8fa8ebe700 1 --2- 172.21.15.59:0/940134909 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f8f8406b470 0x7f8f8406d930 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f8fa406dee0 tx=0x7f8f98008030 comp rx=0 tx=0).stop 2023-11-24T07:59:37.475 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:37.473+0000 7f8fa8ebe700 1 -- 172.21.15.59:0/940134909 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8fa40ceb00 msgr2=0x7f8fa40c6350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:37.475 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:37.473+0000 7f8fa8ebe700 1 --2- 172.21.15.59:0/940134909 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8fa40ceb00 0x7f8fa40c6350 secure :-1 s=READY pgs=953 cs=0 l=1 rev1=1 crypto rx=0x7f8f9c00cef0 tx=0x7f8f9c00a340 comp rx=0 tx=0).stop 2023-11-24T07:59:37.475 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:37.473+0000 7f8fa8ebe700 1 -- 172.21.15.59:0/940134909 shutdown_connections 2023-11-24T07:59:37.475 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:37.473+0000 7f8fa8ebe700 1 --2- 172.21.15.59:0/940134909 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f8f8406b470 0x7f8f8406d930 unknown :-1 s=CLOSED pgs=441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:37.475 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:37.473+0000 7f8fa8ebe700 1 --2- 172.21.15.59:0/940134909 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8fa40ceb00 0x7f8fa40c6350 unknown :-1 s=CLOSED pgs=953 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:37.476 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:37.473+0000 7f8fa8ebe700 1 --2- 172.21.15.59:0/940134909 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8fa40aefc0 0x7f8fa415a2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:37.476 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:37.473+0000 7f8fa8ebe700 1 --2- 172.21.15.59:0/940134909 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f8fa40b0c40 0x7f8fa40b6c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:37.476 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:37.473+0000 7f8fa8ebe700 1 -- 172.21.15.59:0/940134909 >> 172.21.15.59:0/940134909 conn(0x7f8fa40b3720 msgr2=0x7f8fa4070390 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:59:37.476 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:37.473+0000 7f8fa8ebe700 1 -- 172.21.15.59:0/940134909 shutdown_connections 2023-11-24T07:59:37.476 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:37.473+0000 7f8fa8ebe700 1 -- 172.21.15.59:0/940134909 wait complete. 2023-11-24T07:59:37.487 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-24T07:59:37.613 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:37.611+0000 7f6a39f57700 1 Processor -- start 2023-11-24T07:59:37.613 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:37.611+0000 7f6a39f57700 1 -- start start 2023-11-24T07:59:37.615 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:37.612+0000 7f6a39f57700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f6a340d49d0 0x7f6a340d2490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:37.615 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:37.612+0000 7f6a39f57700 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f6a340d29d0 0x7f6a340b40a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:37.615 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:37.612+0000 7f6a39f57700 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6a340d4e50 con 0x7f6a3415e460 2023-11-24T07:59:37.616 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:37.612+0000 7f6a39f57700 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6a340d2e40 con 0x7f6a340d49d0 2023-11-24T07:59:37.616 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:37.612+0000 7f6a39f57700 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6a340b3370 con 0x7f6a340d29d0 2023-11-24T07:59:37.616 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:37.612+0000 7f6a32ffd700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f6a340d49d0 0x7f6a340d2490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:37.616 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:37.612+0000 7f6a32ffd700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f6a340d49d0 0x7f6a340d2490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:55410/0 (socket says 172.21.15.59:55410) 2023-11-24T07:59:37.616 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:37.612+0000 7f6a32ffd700 1 -- 172.21.15.59:0/3559915438 learned_addr learned my addr 172.21.15.59:0/3559915438 (peer_addr_for_me v2:172.21.15.59:0/0) 2023-11-24T07:59:37.616 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:37.613+0000 7f6a33fff700 1 --2- 172.21.15.59:0/3559915438 >> v2:172.21.15.76:3300/0 conn(0x7f6a340d29d0 0x7f6a340b40a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:37.616 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:37.613+0000 7f6a32ffd700 1 -- 172.21.15.59:0/3559915438 >> v2:172.21.15.76:3300/0 conn(0x7f6a340d29d0 msgr2=0x7f6a340b40a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T07:59:37.616 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:37.613+0000 7f6a32ffd700 1 --2- 172.21.15.59:0/3559915438 >> v2:172.21.15.76:3300/0 conn(0x7f6a340d29d0 0x7f6a340b40a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:37.616 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:37.613+0000 7f6a32ffd700 1 -- 172.21.15.59:0/3559915438 >> v1:172.21.15.76:6789/0 conn(0x7f6a3415e460 legacy=0x7f6a340cf480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T07:59:37.616 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:37.613+0000 7f6a32ffd700 1 -- 172.21.15.59:0/3559915438 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6a340cfc40 con 0x7f6a340d49d0 2023-11-24T07:59:37.616 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:37.613+0000 7f6a33fff700 1 --2- 172.21.15.59:0/3559915438 >> v2:172.21.15.76:3300/0 conn(0x7f6a340d29d0 0x7f6a340b40a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T07:59:37.617 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:37.613+0000 7f6a32ffd700 1 --2- 172.21.15.59:0/3559915438 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6a340d49d0 0x7f6a340d2490 secure :-1 s=READY pgs=954 cs=0 l=1 rev1=1 crypto rx=0x7f6a3408d240 tx=0x7f6a28009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=df6881388e58fb21 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:59:37.617 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:37.613+0000 7f6a327fc700 1 -- 172.21.15.59:0/3559915438 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a28004230 con 0x7f6a340d49d0 2023-11-24T07:59:37.617 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:37.613+0000 7f6a327fc700 1 -- 172.21.15.59:0/3559915438 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6a28004390 con 0x7f6a340d49d0 2023-11-24T07:59:37.617 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:37.613+0000 7f6a327fc700 1 -- 172.21.15.59:0/3559915438 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a280036a0 con 0x7f6a340d49d0 2023-11-24T07:59:37.617 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:37.613+0000 7f6a39f57700 1 -- 172.21.15.59:0/3559915438 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6a340d49d0 msgr2=0x7f6a340d2490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:37.617 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:37.613+0000 7f6a39f57700 1 --2- 172.21.15.59:0/3559915438 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6a340d49d0 0x7f6a340d2490 secure :-1 s=READY pgs=954 cs=0 l=1 rev1=1 crypto rx=0x7f6a3408d240 tx=0x7f6a28009bd0 comp rx=0 tx=0).stop 2023-11-24T07:59:37.617 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:37.614+0000 7f6a39f57700 1 -- 172.21.15.59:0/3559915438 shutdown_connections 2023-11-24T07:59:37.617 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:37.614+0000 7f6a39f57700 1 --2- 172.21.15.59:0/3559915438 >> v2:172.21.15.76:3300/0 conn(0x7f6a340d29d0 0x7f6a340b40a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:37.617 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:37.614+0000 7f6a39f57700 1 --2- 172.21.15.59:0/3559915438 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6a340d49d0 0x7f6a340d2490 unknown :-1 s=CLOSED pgs=954 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:37.618 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:37.614+0000 7f6a39f57700 1 -- 172.21.15.59:0/3559915438 >> 172.21.15.59:0/3559915438 conn(0x7f6a340d3eb0 msgr2=0x7f6a340d42c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:59:37.618 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:37.614+0000 7f6a39f57700 1 -- 172.21.15.59:0/3559915438 shutdown_connections 2023-11-24T07:59:37.618 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:37.614+0000 7f6a39f57700 1 -- 172.21.15.59:0/3559915438 wait complete. 2023-11-24T07:59:37.620 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:37.615+0000 7f6a39f57700 1 Processor -- start 2023-11-24T07:59:37.620 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:37.615+0000 7f6a39f57700 1 -- start start 2023-11-24T07:59:37.621 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:37.616+0000 7f6a39f57700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6a340d29d0 0x7f6a341f44c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:37.621 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:37.616+0000 7f6a39f57700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6a340d49d0 0x7f6a3415fcf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:37.621 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:37.616+0000 7f6a39f57700 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f6a3415e460 0x7f6a340c9990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:37.621 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:37.616+0000 7f6a39f57700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a341f4a00 con 0x7f6a340d49d0 2023-11-24T07:59:37.621 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:37.616+0000 7f6a39f57700 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a340c8a80 con 0x7f6a340d29d0 2023-11-24T07:59:37.621 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:37.616+0000 7f6a39f57700 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6a341f1440 con 0x7f6a3415e460 2023-11-24T07:59:37.621 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:37.616+0000 7f6a32ffd700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6a340d49d0 0x7f6a3415fcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:37.621 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:37.616+0000 7f6a32ffd700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6a340d49d0 0x7f6a3415fcf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:55416/0 (socket says 172.21.15.59:55416) 2023-11-24T07:59:37.621 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:37.616+0000 7f6a32ffd700 1 -- 172.21.15.59:0/995901490 learned_addr learned my addr 172.21.15.59:0/995901490 (peer_addr_for_me v2:172.21.15.59:0/0) 2023-11-24T07:59:37.621 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:37.617+0000 7f6a33fff700 1 --2- 172.21.15.59:0/995901490 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f6a3415e460 0x7f6a340c9990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:37.622 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:37.617+0000 7f6a337fe700 1 --2- 172.21.15.59:0/995901490 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6a340d29d0 0x7f6a341f44c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:37.622 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:37.617+0000 7f6a32ffd700 1 -- 172.21.15.59:0/995901490 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f6a3415e460 msgr2=0x7f6a340c9990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:37.622 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:37.617+0000 7f6a32ffd700 1 --2- 172.21.15.59:0/995901490 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f6a3415e460 0x7f6a340c9990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:37.622 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:37.617+0000 7f6a32ffd700 1 -- 172.21.15.59:0/995901490 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6a340d29d0 msgr2=0x7f6a341f44c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:37.622 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:37.617+0000 7f6a32ffd700 1 --2- 172.21.15.59:0/995901490 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6a340d29d0 0x7f6a341f44c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:37.622 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:37.617+0000 7f6a32ffd700 1 -- 172.21.15.59:0/995901490 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6a28009550 con 0x7f6a340d49d0 2023-11-24T07:59:37.622 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:37.617+0000 7f6a32ffd700 1 --2- 172.21.15.59:0/995901490 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6a340d49d0 0x7f6a3415fcf0 secure :-1 s=READY pgs=955 cs=0 l=1 rev1=1 crypto rx=0x7f6a3408c300 tx=0x7f6a28004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:59:37.622 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:37.617+0000 7f6a1bfff700 1 -- 172.21.15.59:0/995901490 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a28003c90 con 0x7f6a340d49d0 2023-11-24T07:59:37.622 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:37.617+0000 7f6a1bfff700 1 -- 172.21.15.59:0/995901490 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6a2801f070 con 0x7f6a340d49d0 2023-11-24T07:59:37.622 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:37.617+0000 7f6a39f57700 1 -- 172.21.15.59:0/995901490 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6a34160230 con 0x7f6a340d49d0 2023-11-24T07:59:37.622 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:37.617+0000 7f6a1bfff700 1 -- 172.21.15.59:0/995901490 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a28011870 con 0x7f6a340d49d0 2023-11-24T07:59:37.623 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:37.617+0000 7f6a39f57700 1 -- 172.21.15.59:0/995901490 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6a340c8150 con 0x7f6a340d49d0 2023-11-24T07:59:37.623 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:37.618+0000 7f6a39f57700 1 -- 172.21.15.59:0/995901490 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6a34067f60 con 0x7f6a340d49d0 2023-11-24T07:59:37.623 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:37.619+0000 7f6a1bfff700 1 -- 172.21.15.59:0/995901490 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 11) v1 ==== 89741+0+0 (secure 0 0 0) 0x7f6a28021030 con 0x7f6a340d49d0 2023-11-24T07:59:37.623 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:37.619+0000 7f6a1bfff700 1 --2- 172.21.15.59:0/995901490 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f6a2006b470 0x7f6a2006d930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:37.623 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:37.619+0000 7f6a1bfff700 1 -- 172.21.15.59:0/995901490 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(75..75 src has 1..75) v4 ==== 5765+0+0 (secure 0 0 0) 0x7f6a2808b670 con 0x7f6a340d49d0 2023-11-24T07:59:37.623 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:37.619+0000 7f6a337fe700 1 --2- 172.21.15.59:0/995901490 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f6a2006b470 0x7f6a2006d930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:37.624 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:37.620+0000 7f6a337fe700 1 --2- 172.21.15.59:0/995901490 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f6a2006b470 0x7f6a2006d930 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f6a24009960 tx=0x7f6a240090e0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:59:37.625 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:37.623+0000 7f6a1bfff700 1 -- 172.21.15.59:0/995901490 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f6a28056020 con 0x7f6a340d49d0 2023-11-24T07:59:37.872 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:37.870+0000 7f6a39f57700 1 -- 172.21.15.59:0/995901490 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f6a340c8bc0 con 0x7f6a340d49d0 2023-11-24T07:59:37.872 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:37.871+0000 7f6a1bfff700 1 -- 172.21.15.59:0/995901490 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 45 v45) v1 ==== 94+0+4915 (secure 0 0 0) 0x7f6a28059640 con 0x7f6a340d49d0 2023-11-24T07:59:37.872 INFO:teuthology.orchestra.run.smithi059.stdout: 2023-11-24T07:59:37.873 INFO:teuthology.orchestra.run.smithi059.stdout:{"epoch":45,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5585,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.76:6837/2282835612","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6835","nonce":2282835612},{"type":"v1","addr":"172.21.15.76:6837","nonce":2282835612}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":39},{"gid":6097,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.76:6836/3924452718","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6834","nonce":3924452718},{"type":"v1","addr":"172.21.15.76:6836","nonce":3924452718}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":39},{"gid":6549,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6834/1425120389","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6832","nonce":1425120389},{"type":"v1","addr":"172.21.15.59:6834","nonce":1425120389}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":39}],"filesystems":[{"mdsmap":{"epoch":44,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-24T07:59:14.054294+0000","modified":"2023-11-24T07:59:17.066277+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":6552},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6552":{"gid":6552,"name":"c","rank":0,"incarnation":40,"state":"up:active","state_seq":3,"addr":"172.21.15.59:6835/1228649189","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6833","nonce":1228649189},{"type":"v1","addr":"172.21.15.59:6835","nonce":1228649189}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[11],"metadata_pool":10,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":5}]} 2023-11-24T07:59:37.875 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:37.873+0000 7f6a39f57700 1 -- 172.21.15.59:0/995901490 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f6a2006b470 msgr2=0x7f6a2006d930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:37.875 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:37.873+0000 7f6a39f57700 1 --2- 172.21.15.59:0/995901490 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f6a2006b470 0x7f6a2006d930 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f6a24009960 tx=0x7f6a240090e0 comp rx=0 tx=0).stop 2023-11-24T07:59:37.875 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:37.873+0000 7f6a39f57700 1 -- 172.21.15.59:0/995901490 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6a340d49d0 msgr2=0x7f6a3415fcf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:37.875 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:37.873+0000 7f6a39f57700 1 --2- 172.21.15.59:0/995901490 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6a340d49d0 0x7f6a3415fcf0 secure :-1 s=READY pgs=955 cs=0 l=1 rev1=1 crypto rx=0x7f6a3408c300 tx=0x7f6a28004750 comp rx=0 tx=0).stop 2023-11-24T07:59:37.876 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:37.873+0000 7f6a39f57700 1 -- 172.21.15.59:0/995901490 shutdown_connections 2023-11-24T07:59:37.876 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:37.873+0000 7f6a39f57700 1 --2- 172.21.15.59:0/995901490 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f6a2006b470 0x7f6a2006d930 unknown :-1 s=CLOSED pgs=442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:37.876 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:37.873+0000 7f6a39f57700 1 --2- 172.21.15.59:0/995901490 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f6a3415e460 0x7f6a340c9990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:37.876 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:37.873+0000 7f6a39f57700 1 --2- 172.21.15.59:0/995901490 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6a340d29d0 0x7f6a341f44c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:37.876 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:37.873+0000 7f6a39f57700 1 --2- 172.21.15.59:0/995901490 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6a340d49d0 0x7f6a3415fcf0 unknown :-1 s=CLOSED pgs=955 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:37.876 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:37.873+0000 7f6a39f57700 1 -- 172.21.15.59:0/995901490 >> 172.21.15.59:0/995901490 conn(0x7f6a340d3eb0 msgr2=0x7f6a340b49d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:59:37.876 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:37.873+0000 7f6a39f57700 1 -- 172.21.15.59:0/995901490 shutdown_connections 2023-11-24T07:59:37.877 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:37.873+0000 7f6a39f57700 1 -- 172.21.15.59:0/995901490 wait complete. 2023-11-24T07:59:37.877 INFO:teuthology.orchestra.run.smithi059.stderr:dumped fsmap epoch 45 2023-11-24T07:59:37.887 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-24T07:59:38.011 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:38.010+0000 7fbd9759e700 1 Processor -- start 2023-11-24T07:59:38.012 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:38.010+0000 7fbd9759e700 1 -- start start 2023-11-24T07:59:38.014 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:38.011+0000 7fbd9759e700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fbd9815ac40 0x7fbd980af6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:38.014 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:38.011+0000 7fbd9759e700 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fbd980cda90 0x7fbd980cf890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:38.014 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:38.011+0000 7fbd9759e700 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbd9806f7c0 con 0x7fbd980d1670 2023-11-24T07:59:38.014 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:38.011+0000 7fbd9759e700 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbd980afc00 con 0x7fbd9815ac40 2023-11-24T07:59:38.015 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:38.011+0000 7fbd9759e700 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbd9806dc20 con 0x7fbd980cda90 2023-11-24T07:59:38.015 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:38.011+0000 7fbd9659c700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fbd9815ac40 0x7fbd980af6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:38.015 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:38.011+0000 7fbd9659c700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fbd9815ac40 0x7fbd980af6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:55422/0 (socket says 172.21.15.59:55422) 2023-11-24T07:59:38.015 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:38.011+0000 7fbd9659c700 1 -- 172.21.15.59:0/1409728271 learned_addr learned my addr 172.21.15.59:0/1409728271 (peer_addr_for_me v2:172.21.15.59:0/0) 2023-11-24T07:59:38.015 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:38.011+0000 7fbd96d9d700 1 --2- 172.21.15.59:0/1409728271 >> v2:172.21.15.76:3300/0 conn(0x7fbd980cda90 0x7fbd980cf890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:38.015 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:38.011+0000 7fbd9659c700 1 -- 172.21.15.59:0/1409728271 >> v2:172.21.15.76:3300/0 conn(0x7fbd980cda90 msgr2=0x7fbd980cf890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T07:59:38.015 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:38.011+0000 7fbd9659c700 1 --2- 172.21.15.59:0/1409728271 >> v2:172.21.15.76:3300/0 conn(0x7fbd980cda90 0x7fbd980cf890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:38.015 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:38.011+0000 7fbd9659c700 1 -- 172.21.15.59:0/1409728271 >> v1:172.21.15.76:6789/0 conn(0x7fbd980d1670 legacy=0x7fbd980cd380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T07:59:38.015 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:38.011+0000 7fbd9659c700 1 -- 172.21.15.59:0/1409728271 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbd9815b020 con 0x7fbd9815ac40 2023-11-24T07:59:38.015 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:38.011+0000 7fbd9659c700 1 --2- 172.21.15.59:0/1409728271 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbd9815ac40 0x7fbd980af6c0 secure :-1 s=READY pgs=956 cs=0 l=1 rev1=1 crypto rx=0x7fbd980693c0 tx=0x7fbd88009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9699399370c38528 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:59:38.015 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:38.011+0000 7fbd9559a700 1 -- 172.21.15.59:0/1409728271 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd88004230 con 0x7fbd9815ac40 2023-11-24T07:59:38.015 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:38.012+0000 7fbd9559a700 1 -- 172.21.15.59:0/1409728271 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbd88004390 con 0x7fbd9815ac40 2023-11-24T07:59:38.016 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:38.012+0000 7fbd9559a700 1 -- 172.21.15.59:0/1409728271 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd880036a0 con 0x7fbd9815ac40 2023-11-24T07:59:38.016 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:38.012+0000 7fbd9759e700 1 -- 172.21.15.59:0/1409728271 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbd9815ac40 msgr2=0x7fbd980af6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:38.016 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:38.012+0000 7fbd9759e700 1 --2- 172.21.15.59:0/1409728271 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbd9815ac40 0x7fbd980af6c0 secure :-1 s=READY pgs=956 cs=0 l=1 rev1=1 crypto rx=0x7fbd980693c0 tx=0x7fbd88009bd0 comp rx=0 tx=0).stop 2023-11-24T07:59:38.016 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:38.012+0000 7fbd9759e700 1 -- 172.21.15.59:0/1409728271 shutdown_connections 2023-11-24T07:59:38.016 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:38.012+0000 7fbd9759e700 1 --2- 172.21.15.59:0/1409728271 >> v2:172.21.15.76:3300/0 conn(0x7fbd980cda90 0x7fbd980cf890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:38.016 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:38.012+0000 7fbd9759e700 1 --2- 172.21.15.59:0/1409728271 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbd9815ac40 0x7fbd980af6c0 unknown :-1 s=CLOSED pgs=956 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:38.016 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:38.012+0000 7fbd9759e700 1 -- 172.21.15.59:0/1409728271 >> 172.21.15.59:0/1409728271 conn(0x7fbd980d09c0 msgr2=0x7fbd980d0dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:59:38.016 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:38.012+0000 7fbd9759e700 1 -- 172.21.15.59:0/1409728271 shutdown_connections 2023-11-24T07:59:38.016 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:38.012+0000 7fbd9759e700 1 -- 172.21.15.59:0/1409728271 wait complete. 2023-11-24T07:59:38.019 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:38.014+0000 7fbd9759e700 1 Processor -- start 2023-11-24T07:59:38.019 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:38.014+0000 7fbd9759e700 1 -- start start 2023-11-24T07:59:38.019 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:38.014+0000 7fbd9759e700 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fbd980cda90 0x7fbd980af030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:38.019 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:38.014+0000 7fbd9759e700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbd980d1670 0x7fbd9814c830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:38.019 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:38.014+0000 7fbd9759e700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fbd9815ac40 0x7fbd980b51a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:38.019 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:38.014+0000 7fbd9759e700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd981f4670 con 0x7fbd980d1670 2023-11-24T07:59:38.019 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:38.014+0000 7fbd9759e700 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd980ae330 con 0x7fbd9815ac40 2023-11-24T07:59:38.019 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:38.014+0000 7fbd9759e700 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbd981f4390 con 0x7fbd980cda90 2023-11-24T07:59:38.019 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:38.014+0000 7fbd95d9b700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbd980d1670 0x7fbd9814c830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:38.020 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:38.014+0000 7fbd95d9b700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbd980d1670 0x7fbd9814c830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:55428/0 (socket says 172.21.15.59:55428) 2023-11-24T07:59:38.020 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:38.014+0000 7fbd95d9b700 1 -- 172.21.15.59:0/3995068839 learned_addr learned my addr 172.21.15.59:0/3995068839 (peer_addr_for_me v2:172.21.15.59:0/0) 2023-11-24T07:59:38.020 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:38.015+0000 7fbd9659c700 1 --2- 172.21.15.59:0/3995068839 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fbd980cda90 0x7fbd980af030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:38.020 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:38.015+0000 7fbd96d9d700 1 --2- 172.21.15.59:0/3995068839 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fbd9815ac40 0x7fbd980b51a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:38.020 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:38.015+0000 7fbd95d9b700 1 -- 172.21.15.59:0/3995068839 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fbd980cda90 msgr2=0x7fbd980af030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:38.020 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:38.015+0000 7fbd95d9b700 1 --2- 172.21.15.59:0/3995068839 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fbd980cda90 0x7fbd980af030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:38.020 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:38.015+0000 7fbd95d9b700 1 -- 172.21.15.59:0/3995068839 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fbd9815ac40 msgr2=0x7fbd980b51a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:38.020 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:38.015+0000 7fbd95d9b700 1 --2- 172.21.15.59:0/3995068839 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fbd9815ac40 0x7fbd980b51a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:38.020 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:38.015+0000 7fbd95d9b700 1 -- 172.21.15.59:0/3995068839 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbd88009550 con 0x7fbd980d1670 2023-11-24T07:59:38.020 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:38.015+0000 7fbd96d9d700 1 --2- 172.21.15.59:0/3995068839 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fbd9815ac40 0x7fbd980b51a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-24T07:59:38.020 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:38.015+0000 7fbd9659c700 1 --2- 172.21.15.59:0/3995068839 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fbd980cda90 0x7fbd980af030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-24T07:59:38.020 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:38.015+0000 7fbd95d9b700 1 --2- 172.21.15.59:0/3995068839 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbd980d1670 0x7fbd9814c830 secure :-1 s=READY pgs=957 cs=0 l=1 rev1=1 crypto rx=0x7fbd8c00c4a0 tx=0x7fbd8c00c860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:59:38.021 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:38.015+0000 7fbd86ffd700 1 -- 172.21.15.59:0/3995068839 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd8c0054e0 con 0x7fbd980d1670 2023-11-24T07:59:38.021 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:38.015+0000 7fbd86ffd700 1 -- 172.21.15.59:0/3995068839 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbd8c003680 con 0x7fbd980d1670 2023-11-24T07:59:38.021 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:38.015+0000 7fbd86ffd700 1 -- 172.21.15.59:0/3995068839 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd8c003a10 con 0x7fbd980d1670 2023-11-24T07:59:38.021 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:38.015+0000 7fbd9759e700 1 -- 172.21.15.59:0/3995068839 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbd980ae550 con 0x7fbd980d1670 2023-11-24T07:59:38.021 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:38.015+0000 7fbd9759e700 1 -- 172.21.15.59:0/3995068839 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbd980aeb10 con 0x7fbd980d1670 2023-11-24T07:59:38.021 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:38.016+0000 7fbd9759e700 1 -- 172.21.15.59:0/3995068839 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbd980aecd0 con 0x7fbd980d1670 2023-11-24T07:59:38.021 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:38.017+0000 7fbd86ffd700 1 -- 172.21.15.59:0/3995068839 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 11) v1 ==== 89741+0+0 (secure 0 0 0) 0x7fbd8c003b70 con 0x7fbd980d1670 2023-11-24T07:59:38.021 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:38.017+0000 7fbd86ffd700 1 --2- 172.21.15.59:0/3995068839 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7fbd7806b470 0x7fbd7806d930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:38.021 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:38.017+0000 7fbd86ffd700 1 -- 172.21.15.59:0/3995068839 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(75..75 src has 1..75) v4 ==== 5765+0+0 (secure 0 0 0) 0x7fbd8c0891f0 con 0x7fbd980d1670 2023-11-24T07:59:38.021 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:38.017+0000 7fbd9659c700 1 --2- 172.21.15.59:0/3995068839 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7fbd7806b470 0x7fbd7806d930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:38.022 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:38.018+0000 7fbd9659c700 1 --2- 172.21.15.59:0/3995068839 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7fbd7806b470 0x7fbd7806d930 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7fbd88009130 tx=0x7fbd88008030 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:59:38.023 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:38.021+0000 7fbd86ffd700 1 -- 172.21.15.59:0/3995068839 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fbd8c053a80 con 0x7fbd980d1670 2023-11-24T07:59:38.224 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:38.223+0000 7fbd9759e700 1 -- 172.21.15.59:0/3995068839 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fbd980af570 con 0x7fbd980d1670 2023-11-24T07:59:38.225 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:38.224+0000 7fbd86ffd700 1 -- 172.21.15.59:0/3995068839 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v75) v1 ==== 74+0+16546 (secure 0 0 0) 0x7fbd8c0570a0 con 0x7fbd980d1670 2023-11-24T07:59:38.226 INFO:teuthology.orchestra.run.smithi059.stdout: 2023-11-24T07:59:38.226 INFO:teuthology.orchestra.run.smithi059.stdout:{"epoch":75,"fsid":"9f0e8741-b8f9-48d5-a2c3-09f45903303d","created":"2023-11-24T07:56:06.552729+0000","modified":"2023-11-24T07:59:34.419418+0000","last_up_change":"2023-11-24T07:56:17.372352+0000","last_in_change":"2023-11-24T07:56:09.073250+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":11,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-24T07:56:18.352823+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_metadata","create_time":"2023-11-24T07:59:11.900882+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"73","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_data","create_time":"2023-11-24T07:59:13.025749+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"75","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":3,"snap_epoch":75,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"6d8f9e9b-1782-4e41-928b-a39f2a8ebc06","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6800","nonce":3373878772},{"type":"v1","addr":"172.21.15.59:6801","nonce":3373878772}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6802","nonce":3373878772},{"type":"v1","addr":"172.21.15.59:6803","nonce":3373878772}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6806","nonce":3373878772},{"type":"v1","addr":"172.21.15.59:6807","nonce":3373878772}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6804","nonce":3373878772},{"type":"v1","addr":"172.21.15.59:6805","nonce":3373878772}]},"public_addr":"172.21.15.59:6801/3373878772","cluster_addr":"172.21.15.59:6803/3373878772","heartbeat_back_addr":"172.21.15.59:6807/3373878772","heartbeat_front_addr":"172.21.15.59:6805/3373878772","state":["exists","up"]},{"osd":1,"uuid":"e8d882aa-52e1-4c04-9e1e-c44ef9204a5f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6824","nonce":648258563},{"type":"v1","addr":"172.21.15.59:6825","nonce":648258563}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6826","nonce":648258563},{"type":"v1","addr":"172.21.15.59:6827","nonce":648258563}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6830","nonce":648258563},{"type":"v1","addr":"172.21.15.59:6831","nonce":648258563}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6828","nonce":648258563},{"type":"v1","addr":"172.21.15.59:6829","nonce":648258563}]},"public_addr":"172.21.15.59:6825/648258563","cluster_addr":"172.21.15.59:6827/648258563","heartbeat_back_addr":"172.21.15.59:6831/648258563","heartbeat_front_addr":"172.21.15.59:6829/648258563","state":["exists","up"]},{"osd":2,"uuid":"69eebfa0-695b-4b21-90b9-8a3537168a01","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6808","nonce":1840430654},{"type":"v1","addr":"172.21.15.59:6809","nonce":1840430654}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6810","nonce":1840430654},{"type":"v1","addr":"172.21.15.59:6811","nonce":1840430654}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6814","nonce":1840430654},{"type":"v1","addr":"172.21.15.59:6815","nonce":1840430654}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6812","nonce":1840430654},{"type":"v1","addr":"172.21.15.59:6813","nonce":1840430654}]},"public_addr":"172.21.15.59:6809/1840430654","cluster_addr":"172.21.15.59:6811/1840430654","heartbeat_back_addr":"172.21.15.59:6815/1840430654","heartbeat_front_addr":"172.21.15.59:6813/1840430654","state":["exists","up"]},{"osd":3,"uuid":"96fd77fc-f911-46ff-9445-6f0be2daa8f4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6816","nonce":2504025850},{"type":"v1","addr":"172.21.15.59:6817","nonce":2504025850}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6818","nonce":2504025850},{"type":"v1","addr":"172.21.15.59:6819","nonce":2504025850}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6822","nonce":2504025850},{"type":"v1","addr":"172.21.15.59:6823","nonce":2504025850}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6820","nonce":2504025850},{"type":"v1","addr":"172.21.15.59:6821","nonce":2504025850}]},"public_addr":"172.21.15.59:6817/2504025850","cluster_addr":"172.21.15.59:6819/2504025850","heartbeat_back_addr":"172.21.15.59:6823/2504025850","heartbeat_front_addr":"172.21.15.59:6821/2504025850","state":["exists","up"]},{"osd":4,"uuid":"7ac29fca-b5b2-4c95-9d69-2f164cbbc708","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6816","nonce":2382463431},{"type":"v1","addr":"172.21.15.76:6818","nonce":2382463431}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6820","nonce":2382463431},{"type":"v1","addr":"172.21.15.76:6822","nonce":2382463431}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6828","nonce":2382463431},{"type":"v1","addr":"172.21.15.76:6830","nonce":2382463431}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6824","nonce":2382463431},{"type":"v1","addr":"172.21.15.76:6826","nonce":2382463431}]},"public_addr":"172.21.15.76:6818/2382463431","cluster_addr":"172.21.15.76:6822/2382463431","heartbeat_back_addr":"172.21.15.76:6830/2382463431","heartbeat_front_addr":"172.21.15.76:6826/2382463431","state":["exists","up"]},{"osd":5,"uuid":"485bf711-3851-4238-a4f4-51d52e0a9c58","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6800","nonce":329334308},{"type":"v1","addr":"172.21.15.76:6802","nonce":329334308}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6805","nonce":329334308},{"type":"v1","addr":"172.21.15.76:6807","nonce":329334308}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6814","nonce":329334308},{"type":"v1","addr":"172.21.15.76:6815","nonce":329334308}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6810","nonce":329334308},{"type":"v1","addr":"172.21.15.76:6812","nonce":329334308}]},"public_addr":"172.21.15.76:6802/329334308","cluster_addr":"172.21.15.76:6807/329334308","heartbeat_back_addr":"172.21.15.76:6815/329334308","heartbeat_front_addr":"172.21.15.76:6812/329334308","state":["exists","up"]},{"osd":6,"uuid":"72b3f2fc-1e4a-4e32-968d-003647a0286f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6801","nonce":3946579410},{"type":"v1","addr":"172.21.15.76:6803","nonce":3946579410}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6804","nonce":3946579410},{"type":"v1","addr":"172.21.15.76:6806","nonce":3946579410}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6811","nonce":3946579410},{"type":"v1","addr":"172.21.15.76:6813","nonce":3946579410}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6808","nonce":3946579410},{"type":"v1","addr":"172.21.15.76:6809","nonce":3946579410}]},"public_addr":"172.21.15.76:6803/3946579410","cluster_addr":"172.21.15.76:6806/3946579410","heartbeat_back_addr":"172.21.15.76:6813/3946579410","heartbeat_front_addr":"172.21.15.76:6809/3946579410","state":["exists","up"]},{"osd":7,"uuid":"1ddceb90-6c5d-4d1a-ad56-99cb877f8c70","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6817","nonce":841761538},{"type":"v1","addr":"172.21.15.76:6819","nonce":841761538}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6821","nonce":841761538},{"type":"v1","addr":"172.21.15.76:6823","nonce":841761538}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6829","nonce":841761538},{"type":"v1","addr":"172.21.15.76:6831","nonce":841761538}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6825","nonce":841761538},{"type":"v1","addr":"172.21.15.76:6827","nonce":841761538}]},"public_addr":"172.21.15.76:6819/841761538","cluster_addr":"172.21.15.76:6823/841761538","heartbeat_back_addr":"172.21.15.76:6831/841761538","heartbeat_front_addr":"172.21.15.76:6827/841761538","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T07:56:14.294838+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T07:56:14.270779+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T07:56:14.231809+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T07:56:14.260983+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T07:56:14.302997+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T07:56:14.258600+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T07:56:14.251413+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T07:56:14.304727+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":11,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[{"pool":11,"snaps":[{"begin":2,"length":2}]}],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-24T07:59:38.228 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:38.226+0000 7fbd9759e700 1 -- 172.21.15.59:0/3995068839 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7fbd7806b470 msgr2=0x7fbd7806d930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:38.228 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:38.226+0000 7fbd9759e700 1 --2- 172.21.15.59:0/3995068839 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7fbd7806b470 0x7fbd7806d930 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7fbd88009130 tx=0x7fbd88008030 comp rx=0 tx=0).stop 2023-11-24T07:59:38.228 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:38.226+0000 7fbd9759e700 1 -- 172.21.15.59:0/3995068839 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbd980d1670 msgr2=0x7fbd9814c830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:38.228 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:38.226+0000 7fbd9759e700 1 --2- 172.21.15.59:0/3995068839 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbd980d1670 0x7fbd9814c830 secure :-1 s=READY pgs=957 cs=0 l=1 rev1=1 crypto rx=0x7fbd8c00c4a0 tx=0x7fbd8c00c860 comp rx=0 tx=0).stop 2023-11-24T07:59:38.228 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:38.226+0000 7fbd9759e700 1 -- 172.21.15.59:0/3995068839 shutdown_connections 2023-11-24T07:59:38.229 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:38.226+0000 7fbd9759e700 1 --2- 172.21.15.59:0/3995068839 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7fbd7806b470 0x7fbd7806d930 unknown :-1 s=CLOSED pgs=443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:38.229 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:38.226+0000 7fbd9759e700 1 --2- 172.21.15.59:0/3995068839 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fbd9815ac40 0x7fbd980b51a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:38.229 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:38.226+0000 7fbd9759e700 1 --2- 172.21.15.59:0/3995068839 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fbd980cda90 0x7fbd980af030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:38.229 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:38.226+0000 7fbd9759e700 1 --2- 172.21.15.59:0/3995068839 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbd980d1670 0x7fbd9814c830 unknown :-1 s=CLOSED pgs=957 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:38.230 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:38.226+0000 7fbd9759e700 1 -- 172.21.15.59:0/3995068839 >> 172.21.15.59:0/3995068839 conn(0x7fbd980d09c0 msgr2=0x7fbd980b1cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:59:38.230 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:38.226+0000 7fbd9759e700 1 -- 172.21.15.59:0/3995068839 shutdown_connections 2023-11-24T07:59:38.230 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:38.226+0000 7fbd9759e700 1 -- 172.21.15.59:0/3995068839 wait complete. 2023-11-24T07:59:38.240 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2023-11-24T07:59:38.241 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-24T07:59:38.365 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:38.364+0000 7fed20c80700 1 Processor -- start 2023-11-24T07:59:38.365 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:38.364+0000 7fed20c80700 1 -- start start 2023-11-24T07:59:38.368 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:38.365+0000 7fed20c80700 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fed1c143ae0 0x7fed1c0b51b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:38.368 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:38.365+0000 7fed20c80700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fed1c1440f0 0x7fed1c071bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:38.368 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:38.365+0000 7fed20c80700 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fed1c143ec0 con 0x7fed1c0720f0 2023-11-24T07:59:38.368 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:38.365+0000 7fed20c80700 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fed1c0b40a0 con 0x7fed1c1440f0 2023-11-24T07:59:38.368 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:38.365+0000 7fed20c80700 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fed1c072560 con 0x7fed1c143ae0 2023-11-24T07:59:38.368 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:38.365+0000 7fed19d9b700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fed1c1440f0 0x7fed1c071bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:38.368 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:38.365+0000 7fed19d9b700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fed1c1440f0 0x7fed1c071bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:55434/0 (socket says 172.21.15.59:55434) 2023-11-24T07:59:38.368 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:38.365+0000 7fed19d9b700 1 -- 172.21.15.59:0/655597556 learned_addr learned my addr 172.21.15.59:0/655597556 (peer_addr_for_me v2:172.21.15.59:0/0) 2023-11-24T07:59:38.369 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:38.365+0000 7fed1a59c700 1 --2- 172.21.15.59:0/655597556 >> v2:172.21.15.76:3300/0 conn(0x7fed1c143ae0 0x7fed1c0b51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:38.369 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:38.365+0000 7fed19d9b700 1 -- 172.21.15.59:0/655597556 >> v2:172.21.15.76:3300/0 conn(0x7fed1c143ae0 msgr2=0x7fed1c0b51b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T07:59:38.369 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:38.365+0000 7fed19d9b700 1 --2- 172.21.15.59:0/655597556 >> v2:172.21.15.76:3300/0 conn(0x7fed1c143ae0 0x7fed1c0b51b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:38.369 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:38.365+0000 7fed19d9b700 1 -- 172.21.15.59:0/655597556 >> v1:172.21.15.76:6789/0 conn(0x7fed1c0720f0 legacy=0x7fed1c15e8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:38.369 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:38.365+0000 7fed19d9b700 1 -- 172.21.15.59:0/655597556 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fed1c0b3ae0 con 0x7fed1c1440f0 2023-11-24T07:59:38.369 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:38.365+0000 7fed19d9b700 1 --2- 172.21.15.59:0/655597556 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fed1c1440f0 0x7fed1c071bb0 secure :-1 s=READY pgs=958 cs=0 l=1 rev1=1 crypto rx=0x7fed1c06d2e0 tx=0x7fed04009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bbfe7a5090299c71 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:59:38.369 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:38.366+0000 7fed1959a700 1 -- 172.21.15.59:0/655597556 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed04011830 con 0x7fed1c1440f0 2023-11-24T07:59:38.369 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:38.366+0000 7fed1959a700 1 -- 172.21.15.59:0/655597556 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fed04011990 con 0x7fed1c1440f0 2023-11-24T07:59:38.369 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:38.366+0000 7fed1959a700 1 -- 172.21.15.59:0/655597556 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed04011d20 con 0x7fed1c1440f0 2023-11-24T07:59:38.369 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:38.366+0000 7fed20c80700 1 -- 172.21.15.59:0/655597556 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fed1c1440f0 msgr2=0x7fed1c071bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:38.369 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:38.366+0000 7fed20c80700 1 --2- 172.21.15.59:0/655597556 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fed1c1440f0 0x7fed1c071bb0 secure :-1 s=READY pgs=958 cs=0 l=1 rev1=1 crypto rx=0x7fed1c06d2e0 tx=0x7fed04009c00 comp rx=0 tx=0).stop 2023-11-24T07:59:38.369 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:38.366+0000 7fed20c80700 1 -- 172.21.15.59:0/655597556 shutdown_connections 2023-11-24T07:59:38.370 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:38.366+0000 7fed20c80700 1 --2- 172.21.15.59:0/655597556 >> v2:172.21.15.76:3300/0 conn(0x7fed1c143ae0 0x7fed1c0b51b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:38.370 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:38.366+0000 7fed20c80700 1 --2- 172.21.15.59:0/655597556 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fed1c1440f0 0x7fed1c071bb0 unknown :-1 s=CLOSED pgs=958 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:38.370 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:38.366+0000 7fed20c80700 1 -- 172.21.15.59:0/655597556 >> 172.21.15.59:0/655597556 conn(0x7fed1c0b2740 msgr2=0x7fed1c0b2b20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:59:38.370 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:38.366+0000 7fed20c80700 1 -- 172.21.15.59:0/655597556 shutdown_connections 2023-11-24T07:59:38.370 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:38.366+0000 7fed20c80700 1 -- 172.21.15.59:0/655597556 wait complete. 2023-11-24T07:59:38.372 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:38.368+0000 7fed20c80700 1 Processor -- start 2023-11-24T07:59:38.372 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:38.368+0000 7fed20c80700 1 -- start start 2023-11-24T07:59:38.372 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:38.368+0000 7fed20c80700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fed1c0720f0 0x7fed1c156dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:38.372 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:38.368+0000 7fed20c80700 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fed1c143ae0 0x7fed1c13cac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:38.372 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:38.368+0000 7fed20c80700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fed1c1440f0 0x7fed1c1f04f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:38.372 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:38.368+0000 7fed20c80700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed1c157310 con 0x7fed1c1440f0 2023-11-24T07:59:38.373 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:38.368+0000 7fed20c80700 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed1c158790 con 0x7fed1c0720f0 2023-11-24T07:59:38.373 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:38.368+0000 7fed20c80700 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fed1c158a70 con 0x7fed1c143ae0 2023-11-24T07:59:38.373 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:38.368+0000 7fed1ad9d700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fed1c1440f0 0x7fed1c1f04f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:38.373 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:38.369+0000 7fed1ad9d700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fed1c1440f0 0x7fed1c1f04f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:55442/0 (socket says 172.21.15.59:55442) 2023-11-24T07:59:38.373 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:38.369+0000 7fed1ad9d700 1 -- 172.21.15.59:0/2046942404 learned_addr learned my addr 172.21.15.59:0/2046942404 (peer_addr_for_me v2:172.21.15.59:0/0) 2023-11-24T07:59:38.373 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:38.369+0000 7fed19d9b700 1 --2- 172.21.15.59:0/2046942404 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fed1c143ae0 0x7fed1c13cac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:38.373 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:38.369+0000 7fed1a59c700 1 --2- 172.21.15.59:0/2046942404 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fed1c0720f0 0x7fed1c156dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:38.373 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:38.369+0000 7fed1ad9d700 1 -- 172.21.15.59:0/2046942404 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fed1c143ae0 msgr2=0x7fed1c13cac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:38.373 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:38.369+0000 7fed1ad9d700 1 --2- 172.21.15.59:0/2046942404 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fed1c143ae0 0x7fed1c13cac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:38.373 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:38.369+0000 7fed1ad9d700 1 -- 172.21.15.59:0/2046942404 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fed1c0720f0 msgr2=0x7fed1c156dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:38.373 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:38.369+0000 7fed1ad9d700 1 --2- 172.21.15.59:0/2046942404 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fed1c0720f0 0x7fed1c156dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:38.374 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:38.369+0000 7fed1ad9d700 1 -- 172.21.15.59:0/2046942404 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fed04009550 con 0x7fed1c1440f0 2023-11-24T07:59:38.374 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:38.369+0000 7fed1ad9d700 1 --2- 172.21.15.59:0/2046942404 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fed1c1440f0 0x7fed1c1f04f0 secure :-1 s=READY pgs=959 cs=0 l=1 rev1=1 crypto rx=0x7fed1400edf0 tx=0x7fed1400c550 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:59:38.374 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:38.369+0000 7fed12ffd700 1 -- 172.21.15.59:0/2046942404 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed14003a00 con 0x7fed1c1440f0 2023-11-24T07:59:38.374 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:38.369+0000 7fed12ffd700 1 -- 172.21.15.59:0/2046942404 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fed14003b60 con 0x7fed1c1440f0 2023-11-24T07:59:38.374 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:38.369+0000 7fed20c80700 1 -- 172.21.15.59:0/2046942404 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fed1c158d50 con 0x7fed1c1440f0 2023-11-24T07:59:38.374 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:38.369+0000 7fed12ffd700 1 -- 172.21.15.59:0/2046942404 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed140050d0 con 0x7fed1c1440f0 2023-11-24T07:59:38.374 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:38.369+0000 7fed20c80700 1 -- 172.21.15.59:0/2046942404 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fed1c138bd0 con 0x7fed1c1440f0 2023-11-24T07:59:38.374 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:38.370+0000 7fed20c80700 1 -- 172.21.15.59:0/2046942404 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fed1c138d90 con 0x7fed1c1440f0 2023-11-24T07:59:38.374 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:38.371+0000 7fed12ffd700 1 -- 172.21.15.59:0/2046942404 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 11) v1 ==== 89741+0+0 (secure 0 0 0) 0x7fed14005230 con 0x7fed1c1440f0 2023-11-24T07:59:38.375 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:38.371+0000 7fed12ffd700 1 --2- 172.21.15.59:0/2046942404 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7fecfc06b470 0x7fecfc06d930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:38.375 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:38.371+0000 7fed12ffd700 1 -- 172.21.15.59:0/2046942404 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(75..75 src has 1..75) v4 ==== 5765+0+0 (secure 0 0 0) 0x7fed1408c620 con 0x7fed1c1440f0 2023-11-24T07:59:38.375 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:38.371+0000 7fed1a59c700 1 --2- 172.21.15.59:0/2046942404 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7fecfc06b470 0x7fecfc06d930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:38.375 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:38.372+0000 7fed1a59c700 1 --2- 172.21.15.59:0/2046942404 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7fecfc06b470 0x7fecfc06d930 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7fed0c009b40 tx=0x7fed0c009210 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:59:38.377 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:38.376+0000 7fed12ffd700 1 -- 172.21.15.59:0/2046942404 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fed1c138d90 con 0x7fed1c1440f0 2023-11-24T07:59:38.624 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:38.622+0000 7fed20c80700 1 -- 172.21.15.59:0/2046942404 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fed1c144830 con 0x7fed1c1440f0 2023-11-24T07:59:38.624 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:38.623+0000 7fed12ffd700 1 -- 172.21.15.59:0/2046942404 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 45 v45) v1 ==== 94+0+4915 (secure 0 0 0) 0x7fed14056f60 con 0x7fed1c1440f0 2023-11-24T07:59:38.625 INFO:teuthology.orchestra.run.smithi059.stdout: 2023-11-24T07:59:38.625 INFO:teuthology.orchestra.run.smithi059.stdout:{"epoch":45,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5585,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.76:6837/2282835612","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6835","nonce":2282835612},{"type":"v1","addr":"172.21.15.76:6837","nonce":2282835612}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":39},{"gid":6097,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.76:6836/3924452718","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6834","nonce":3924452718},{"type":"v1","addr":"172.21.15.76:6836","nonce":3924452718}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":39},{"gid":6549,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6834/1425120389","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6832","nonce":1425120389},{"type":"v1","addr":"172.21.15.59:6834","nonce":1425120389}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":39}],"filesystems":[{"mdsmap":{"epoch":44,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-24T07:59:14.054294+0000","modified":"2023-11-24T07:59:17.066277+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":6552},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6552":{"gid":6552,"name":"c","rank":0,"incarnation":40,"state":"up:active","state_seq":3,"addr":"172.21.15.59:6835/1228649189","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6833","nonce":1228649189},{"type":"v1","addr":"172.21.15.59:6835","nonce":1228649189}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[11],"metadata_pool":10,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":5}]} 2023-11-24T07:59:38.626 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:38.624+0000 7fed20c80700 1 -- 172.21.15.59:0/2046942404 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7fecfc06b470 msgr2=0x7fecfc06d930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:38.627 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:38.625+0000 7fed20c80700 1 --2- 172.21.15.59:0/2046942404 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7fecfc06b470 0x7fecfc06d930 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7fed0c009b40 tx=0x7fed0c009210 comp rx=0 tx=0).stop 2023-11-24T07:59:38.627 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:38.625+0000 7fed20c80700 1 -- 172.21.15.59:0/2046942404 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fed1c1440f0 msgr2=0x7fed1c1f04f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:38.627 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:38.625+0000 7fed20c80700 1 --2- 172.21.15.59:0/2046942404 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fed1c1440f0 0x7fed1c1f04f0 secure :-1 s=READY pgs=959 cs=0 l=1 rev1=1 crypto rx=0x7fed1400edf0 tx=0x7fed1400c550 comp rx=0 tx=0).stop 2023-11-24T07:59:38.627 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:38.625+0000 7fed20c80700 1 -- 172.21.15.59:0/2046942404 shutdown_connections 2023-11-24T07:59:38.627 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:38.625+0000 7fed20c80700 1 --2- 172.21.15.59:0/2046942404 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7fecfc06b470 0x7fecfc06d930 unknown :-1 s=CLOSED pgs=444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:38.627 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:38.625+0000 7fed20c80700 1 --2- 172.21.15.59:0/2046942404 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fed1c1440f0 0x7fed1c1f04f0 unknown :-1 s=CLOSED pgs=959 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:38.627 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:38.625+0000 7fed20c80700 1 --2- 172.21.15.59:0/2046942404 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fed1c0720f0 0x7fed1c156dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:38.627 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:38.625+0000 7fed20c80700 1 --2- 172.21.15.59:0/2046942404 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fed1c143ae0 0x7fed1c13cac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:38.627 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:38.625+0000 7fed20c80700 1 -- 172.21.15.59:0/2046942404 >> 172.21.15.59:0/2046942404 conn(0x7fed1c0b2740 msgr2=0x7fed1c13f2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:59:38.627 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:38.625+0000 7fed20c80700 1 -- 172.21.15.59:0/2046942404 shutdown_connections 2023-11-24T07:59:38.627 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:38.625+0000 7fed20c80700 1 -- 172.21.15.59:0/2046942404 wait complete. 2023-11-24T07:59:38.628 INFO:teuthology.orchestra.run.smithi059.stderr:dumped fsmap epoch 45 2023-11-24T07:59:38.639 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-24T07:59:38.766 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:38.765+0000 7f2573c99700 1 Processor -- start 2023-11-24T07:59:38.767 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:38.765+0000 7f2573c99700 1 -- start start 2023-11-24T07:59:38.769 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:38.766+0000 7f2573c99700 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f256c15c010 0x7f256c0cd2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:38.769 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:38.766+0000 7f2573c99700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f256c0ce560 0x7f256c0ce9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:38.769 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:38.766+0000 7f2573c99700 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f256c0cef20 0x7f256c0cfc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:38.769 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:38.766+0000 7f2573c99700 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f256c06f610 con 0x7f256c0ce560 2023-11-24T07:59:38.769 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:38.766+0000 7f2573c99700 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f256c0cf370 con 0x7f256c0cef20 2023-11-24T07:59:38.769 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:38.766+0000 7f2573c99700 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f256c0cd7f0 con 0x7f256c15c010 2023-11-24T07:59:38.769 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:38.766+0000 7f2571234700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f256c0ce560 0x7f256c0ce9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:38.769 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:38.766+0000 7f2571234700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f256c0ce560 0x7f256c0ce9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:55446/0 (socket says 172.21.15.59:55446) 2023-11-24T07:59:38.769 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:38.766+0000 7f2571234700 1 -- 172.21.15.59:0/1517596933 learned_addr learned my addr 172.21.15.59:0/1517596933 (peer_addr_for_me v2:172.21.15.59:0/0) 2023-11-24T07:59:38.769 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:38.766+0000 7f2571a35700 1 --2- 172.21.15.59:0/1517596933 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f256c15c010 0x7f256c0cd2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:38.770 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:38.766+0000 7f2572236700 1 --2- 172.21.15.59:0/1517596933 >> v2:172.21.15.76:3300/0 conn(0x7f256c0cef20 0x7f256c0cfc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:38.770 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:38.766+0000 7f2571234700 1 -- 172.21.15.59:0/1517596933 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f256c15c010 msgr2=0x7f256c0cd2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T07:59:38.770 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:38.766+0000 7f2571234700 1 --2- 172.21.15.59:0/1517596933 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f256c15c010 0x7f256c0cd2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:38.770 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:38.766+0000 7f2571234700 1 -- 172.21.15.59:0/1517596933 >> v2:172.21.15.76:3300/0 conn(0x7f256c0cef20 msgr2=0x7f256c0cfc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T07:59:38.770 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:38.766+0000 7f2571234700 1 --2- 172.21.15.59:0/1517596933 >> v2:172.21.15.76:3300/0 conn(0x7f256c0cef20 0x7f256c0cfc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:38.770 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:38.766+0000 7f2571234700 1 -- 172.21.15.59:0/1517596933 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f256c0cdac0 con 0x7f256c0ce560 2023-11-24T07:59:38.770 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:38.766+0000 7f2571234700 1 --2- 172.21.15.59:0/1517596933 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f256c0ce560 0x7f256c0ce9e0 secure :-1 s=READY pgs=960 cs=0 l=1 rev1=1 crypto rx=0x7f256c071690 tx=0x7f255c009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7377c53daa8f3af6 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:59:38.770 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:38.766+0000 7f2570a33700 1 -- 172.21.15.59:0/1517596933 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f255c011830 con 0x7f256c0ce560 2023-11-24T07:59:38.770 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:38.767+0000 7f2570a33700 1 -- 172.21.15.59:0/1517596933 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f255c011990 con 0x7f256c0ce560 2023-11-24T07:59:38.770 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:38.767+0000 7f2570a33700 1 -- 172.21.15.59:0/1517596933 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f255c011d20 con 0x7f256c0ce560 2023-11-24T07:59:38.770 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:38.767+0000 7f2573c99700 1 -- 172.21.15.59:0/1517596933 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f256c0ce560 msgr2=0x7f256c0ce9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:38.770 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:38.767+0000 7f2573c99700 1 --2- 172.21.15.59:0/1517596933 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f256c0ce560 0x7f256c0ce9e0 secure :-1 s=READY pgs=960 cs=0 l=1 rev1=1 crypto rx=0x7f256c071690 tx=0x7f255c009bd0 comp rx=0 tx=0).stop 2023-11-24T07:59:38.771 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:38.767+0000 7f2573c99700 1 -- 172.21.15.59:0/1517596933 shutdown_connections 2023-11-24T07:59:38.771 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:38.767+0000 7f2573c99700 1 --2- 172.21.15.59:0/1517596933 >> v2:172.21.15.76:3300/0 conn(0x7f256c0cef20 0x7f256c0cfc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:38.771 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:38.767+0000 7f2573c99700 1 --2- 172.21.15.59:0/1517596933 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f256c15c010 0x7f256c0cd2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:38.771 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:38.767+0000 7f2573c99700 1 --2- 172.21.15.59:0/1517596933 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f256c0ce560 0x7f256c0ce9e0 unknown :-1 s=CLOSED pgs=960 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:38.771 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:38.767+0000 7f2573c99700 1 -- 172.21.15.59:0/1517596933 >> 172.21.15.59:0/1517596933 conn(0x7f256c0b3790 msgr2=0x7f256c0b3ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:59:38.771 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:38.767+0000 7f2573c99700 1 -- 172.21.15.59:0/1517596933 shutdown_connections 2023-11-24T07:59:38.771 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:38.767+0000 7f2573c99700 1 -- 172.21.15.59:0/1517596933 wait complete. 2023-11-24T07:59:38.774 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:38.768+0000 7f2573c99700 1 Processor -- start 2023-11-24T07:59:38.774 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:38.768+0000 7f2573c99700 1 -- start start 2023-11-24T07:59:38.774 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:38.769+0000 7f2573c99700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f256c0ce560 0x7f256c1f0fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:38.774 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:38.769+0000 7f2573c99700 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f256c0cef20 0x7f256c0c96b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:38.774 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:38.769+0000 7f2573c99700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f256c15c010 0x7f256c15eab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:38.774 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:38.769+0000 7f2573c99700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f256c09bd20 con 0x7f256c0ce560 2023-11-24T07:59:38.774 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:38.769+0000 7f2573c99700 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f256c0cad40 con 0x7f256c15c010 2023-11-24T07:59:38.775 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:38.769+0000 7f2573c99700 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f256c1f1500 con 0x7f256c0cef20 2023-11-24T07:59:38.775 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:38.769+0000 7f2571a35700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f256c0ce560 0x7f256c1f0fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:38.775 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:38.769+0000 7f2571a35700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f256c0ce560 0x7f256c1f0fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:55452/0 (socket says 172.21.15.59:55452) 2023-11-24T07:59:38.775 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:38.769+0000 7f2571a35700 1 -- 172.21.15.59:0/2025473825 learned_addr learned my addr 172.21.15.59:0/2025473825 (peer_addr_for_me v2:172.21.15.59:0/0) 2023-11-24T07:59:38.775 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:38.769+0000 7f2572236700 1 --2- 172.21.15.59:0/2025473825 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f256c15c010 0x7f256c15eab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:38.775 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:38.769+0000 7f2571234700 1 --2- 172.21.15.59:0/2025473825 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f256c0cef20 0x7f256c0c96b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:38.775 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:38.769+0000 7f2571a35700 1 -- 172.21.15.59:0/2025473825 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f256c0cef20 msgr2=0x7f256c0c96b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:38.775 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:38.769+0000 7f2571a35700 1 --2- 172.21.15.59:0/2025473825 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f256c0cef20 0x7f256c0c96b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:38.775 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:38.769+0000 7f2571a35700 1 -- 172.21.15.59:0/2025473825 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f256c15c010 msgr2=0x7f256c15eab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:38.775 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:38.769+0000 7f2571a35700 1 --2- 172.21.15.59:0/2025473825 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f256c15c010 0x7f256c15eab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:38.775 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:38.769+0000 7f2571a35700 1 -- 172.21.15.59:0/2025473825 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f255c009550 con 0x7f256c0ce560 2023-11-24T07:59:38.775 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:38.770+0000 7f2571a35700 1 --2- 172.21.15.59:0/2025473825 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f256c0ce560 0x7f256c1f0fc0 secure :-1 s=READY pgs=961 cs=0 l=1 rev1=1 crypto rx=0x7f256800e870 tx=0x7f256800ec30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:59:38.776 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:38.770+0000 7f25627fc700 1 -- 172.21.15.59:0/2025473825 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2568009590 con 0x7f256c0ce560 2023-11-24T07:59:38.776 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:38.770+0000 7f25627fc700 1 -- 172.21.15.59:0/2025473825 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f256800ee70 con 0x7f256c0ce560 2023-11-24T07:59:38.776 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:38.770+0000 7f2573c99700 1 -- 172.21.15.59:0/2025473825 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f256c0ca130 con 0x7f256c0ce560 2023-11-24T07:59:38.776 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:38.770+0000 7f25627fc700 1 -- 172.21.15.59:0/2025473825 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f256800ca50 con 0x7f256c0ce560 2023-11-24T07:59:38.776 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:38.770+0000 7f2573c99700 1 -- 172.21.15.59:0/2025473825 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f256c0ca6f0 con 0x7f256c0ce560 2023-11-24T07:59:38.776 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:38.771+0000 7f2573c99700 1 -- 172.21.15.59:0/2025473825 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f256c068670 con 0x7f256c0ce560 2023-11-24T07:59:38.776 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:38.771+0000 7f25627fc700 1 -- 172.21.15.59:0/2025473825 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 11) v1 ==== 89741+0+0 (secure 0 0 0) 0x7f256800cbb0 con 0x7f256c0ce560 2023-11-24T07:59:38.776 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:38.772+0000 7f25627fc700 1 --2- 172.21.15.59:0/2025473825 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f255006b470 0x7f255006d930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:38.776 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:38.772+0000 7f25627fc700 1 -- 172.21.15.59:0/2025473825 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(75..75 src has 1..75) v4 ==== 5765+0+0 (secure 0 0 0) 0x7f2568089fa0 con 0x7f256c0ce560 2023-11-24T07:59:38.776 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:38.772+0000 7f2571234700 1 --2- 172.21.15.59:0/2025473825 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f255006b470 0x7f255006d930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:38.776 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:38.772+0000 7f2571234700 1 --2- 172.21.15.59:0/2025473825 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f255006b470 0x7f255006d930 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f255c009c10 tx=0x7f255c008030 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:59:38.778 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:38.776+0000 7f25627fc700 1 -- 172.21.15.59:0/2025473825 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f25680547a0 con 0x7f256c0ce560 2023-11-24T07:59:39.022 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:39.019+0000 7f2573c99700 1 -- 172.21.15.59:0/2025473825 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f256c15ca10 con 0x7f256c0ce560 2023-11-24T07:59:39.022 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:39.020+0000 7f25627fc700 1 -- 172.21.15.59:0/2025473825 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 45 v45) v1 ==== 94+0+4915 (secure 0 0 0) 0x7f2568021090 con 0x7f256c0ce560 2023-11-24T07:59:39.023 INFO:teuthology.orchestra.run.smithi059.stdout: 2023-11-24T07:59:39.023 INFO:teuthology.orchestra.run.smithi059.stdout:{"epoch":45,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5585,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.76:6837/2282835612","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6835","nonce":2282835612},{"type":"v1","addr":"172.21.15.76:6837","nonce":2282835612}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":39},{"gid":6097,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.76:6836/3924452718","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6834","nonce":3924452718},{"type":"v1","addr":"172.21.15.76:6836","nonce":3924452718}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":39},{"gid":6549,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6834/1425120389","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6832","nonce":1425120389},{"type":"v1","addr":"172.21.15.59:6834","nonce":1425120389}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":39}],"filesystems":[{"mdsmap":{"epoch":44,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-24T07:59:14.054294+0000","modified":"2023-11-24T07:59:17.066277+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":6552},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6552":{"gid":6552,"name":"c","rank":0,"incarnation":40,"state":"up:active","state_seq":3,"addr":"172.21.15.59:6835/1228649189","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6833","nonce":1228649189},{"type":"v1","addr":"172.21.15.59:6835","nonce":1228649189}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[11],"metadata_pool":10,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":5}]} 2023-11-24T07:59:39.025 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:39.022+0000 7f2573c99700 1 -- 172.21.15.59:0/2025473825 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f255006b470 msgr2=0x7f255006d930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:39.025 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:39.022+0000 7f2573c99700 1 --2- 172.21.15.59:0/2025473825 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f255006b470 0x7f255006d930 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f255c009c10 tx=0x7f255c008030 comp rx=0 tx=0).stop 2023-11-24T07:59:39.025 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:39.022+0000 7f2573c99700 1 -- 172.21.15.59:0/2025473825 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f256c0ce560 msgr2=0x7f256c1f0fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:39.026 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:39.022+0000 7f2573c99700 1 --2- 172.21.15.59:0/2025473825 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f256c0ce560 0x7f256c1f0fc0 secure :-1 s=READY pgs=961 cs=0 l=1 rev1=1 crypto rx=0x7f256800e870 tx=0x7f256800ec30 comp rx=0 tx=0).stop 2023-11-24T07:59:39.026 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:39.022+0000 7f2573c99700 1 -- 172.21.15.59:0/2025473825 shutdown_connections 2023-11-24T07:59:39.026 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:39.022+0000 7f2573c99700 1 --2- 172.21.15.59:0/2025473825 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f255006b470 0x7f255006d930 unknown :-1 s=CLOSED pgs=445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:39.026 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:39.022+0000 7f2573c99700 1 --2- 172.21.15.59:0/2025473825 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f256c15c010 0x7f256c15eab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:39.027 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:39.022+0000 7f2573c99700 1 --2- 172.21.15.59:0/2025473825 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f256c0ce560 0x7f256c1f0fc0 unknown :-1 s=CLOSED pgs=961 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:39.027 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:39.022+0000 7f2573c99700 1 --2- 172.21.15.59:0/2025473825 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f256c0cef20 0x7f256c0c96b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:39.027 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:39.022+0000 7f2573c99700 1 -- 172.21.15.59:0/2025473825 >> 172.21.15.59:0/2025473825 conn(0x7f256c0b3790 msgr2=0x7f256c15d670 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:59:39.027 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:39.022+0000 7f2573c99700 1 -- 172.21.15.59:0/2025473825 shutdown_connections 2023-11-24T07:59:39.027 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:39.022+0000 7f2573c99700 1 -- 172.21.15.59:0/2025473825 wait complete. 2023-11-24T07:59:39.027 INFO:teuthology.orchestra.run.smithi059.stderr:dumped fsmap epoch 45 2023-11-24T07:59:39.036 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-24T07:59:39.166 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:39.164+0000 7f3e900b8700 1 Processor -- start 2023-11-24T07:59:39.166 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:39.164+0000 7f3e900b8700 1 -- start start 2023-11-24T07:59:39.169 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:39.165+0000 7f3e900b8700 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f3e8815e1c0 0x7f3e8806eef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:39.169 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:39.165+0000 7f3e900b8700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f3e8806fc50 0x7f3e8806ca10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:39.169 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:39.165+0000 7f3e900b8700 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3e880d1230 con 0x7f3e880b0e90 2023-11-24T07:59:39.169 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:39.165+0000 7f3e900b8700 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3e880d0c70 con 0x7f3e8806fc50 2023-11-24T07:59:39.169 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:39.165+0000 7f3e900b8700 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3e880d0f50 con 0x7f3e8815e1c0 2023-11-24T07:59:39.169 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:39.165+0000 7f3e8e655700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f3e8806fc50 0x7f3e8806ca10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:39.170 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:39.165+0000 7f3e8e655700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f3e8806fc50 0x7f3e8806ca10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:55464/0 (socket says 172.21.15.59:55464) 2023-11-24T07:59:39.170 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:39.165+0000 7f3e8e655700 1 -- 172.21.15.59:0/2680346630 learned_addr learned my addr 172.21.15.59:0/2680346630 (peer_addr_for_me v2:172.21.15.59:0/0) 2023-11-24T07:59:39.170 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:39.166+0000 7f3e8de54700 1 --2- 172.21.15.59:0/2680346630 >> v2:172.21.15.76:3300/0 conn(0x7f3e8815e1c0 0x7f3e8806eef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:39.170 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:39.166+0000 7f3e8e655700 1 -- 172.21.15.59:0/2680346630 >> v2:172.21.15.76:3300/0 conn(0x7f3e8815e1c0 msgr2=0x7f3e8806eef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T07:59:39.170 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:39.166+0000 7f3e8e655700 1 --2- 172.21.15.59:0/2680346630 >> v2:172.21.15.76:3300/0 conn(0x7f3e8815e1c0 0x7f3e8806eef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:39.171 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:39.166+0000 7f3e8e655700 1 -- 172.21.15.59:0/2680346630 >> v1:172.21.15.76:6789/0 conn(0x7f3e880b0e90 legacy=0x7f3e8806f540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T07:59:39.171 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:39.166+0000 7f3e8e655700 1 -- 172.21.15.59:0/2680346630 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3e8806c210 con 0x7f3e8806fc50 2023-11-24T07:59:39.171 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:39.166+0000 7f3e8de54700 1 --2- 172.21.15.59:0/2680346630 >> v2:172.21.15.76:3300/0 conn(0x7f3e8815e1c0 0x7f3e8806eef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T07:59:39.171 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:39.166+0000 7f3e8e655700 1 --2- 172.21.15.59:0/2680346630 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3e8806fc50 0x7f3e8806ca10 secure :-1 s=READY pgs=962 cs=0 l=1 rev1=1 crypto rx=0x7f3e88071990 tx=0x7f3e8000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aff6da402f0c982f server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:59:39.171 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:39.166+0000 7f3e8ce52700 1 -- 172.21.15.59:0/2680346630 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e8000e070 con 0x7f3e8806fc50 2023-11-24T07:59:39.172 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:39.166+0000 7f3e8ce52700 1 -- 172.21.15.59:0/2680346630 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3e80003a20 con 0x7f3e8806fc50 2023-11-24T07:59:39.172 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:39.166+0000 7f3e8ce52700 1 -- 172.21.15.59:0/2680346630 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e80003dd0 con 0x7f3e8806fc50 2023-11-24T07:59:39.172 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:39.167+0000 7f3e900b8700 1 -- 172.21.15.59:0/2680346630 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3e8806fc50 msgr2=0x7f3e8806ca10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:39.172 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:39.167+0000 7f3e900b8700 1 --2- 172.21.15.59:0/2680346630 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3e8806fc50 0x7f3e8806ca10 secure :-1 s=READY pgs=962 cs=0 l=1 rev1=1 crypto rx=0x7f3e88071990 tx=0x7f3e8000b330 comp rx=0 tx=0).stop 2023-11-24T07:59:39.172 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:39.167+0000 7f3e900b8700 1 -- 172.21.15.59:0/2680346630 shutdown_connections 2023-11-24T07:59:39.172 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:39.167+0000 7f3e900b8700 1 --2- 172.21.15.59:0/2680346630 >> v2:172.21.15.76:3300/0 conn(0x7f3e8815e1c0 0x7f3e8806eef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:39.173 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:39.167+0000 7f3e900b8700 1 --2- 172.21.15.59:0/2680346630 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3e8806fc50 0x7f3e8806ca10 unknown :-1 s=CLOSED pgs=962 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:39.173 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:39.167+0000 7f3e900b8700 1 -- 172.21.15.59:0/2680346630 >> 172.21.15.59:0/2680346630 conn(0x7f3e880b3720 msgr2=0x7f3e880b3b30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:59:39.173 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:39.167+0000 7f3e900b8700 1 -- 172.21.15.59:0/2680346630 shutdown_connections 2023-11-24T07:59:39.173 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:39.167+0000 7f3e900b8700 1 -- 172.21.15.59:0/2680346630 wait complete. 2023-11-24T07:59:39.176 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:39.168+0000 7f3e900b8700 1 Processor -- start 2023-11-24T07:59:39.177 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:39.168+0000 7f3e900b8700 1 -- start start 2023-11-24T07:59:39.177 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:39.169+0000 7f3e900b8700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3e8806fc50 0x7f3e8815ff60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:39.177 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:39.169+0000 7f3e900b8700 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f3e880b0e90 0x7f3e881f3270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:39.177 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:39.169+0000 7f3e900b8700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3e8815e1c0 0x7f3e8806d310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:39.177 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:39.169+0000 7f3e900b8700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e881604a0 con 0x7f3e8806fc50 2023-11-24T07:59:39.178 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:39.169+0000 7f3e900b8700 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e880b6580 con 0x7f3e8815e1c0 2023-11-24T07:59:39.178 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:39.169+0000 7f3e900b8700 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3e880b5970 con 0x7f3e880b0e90 2023-11-24T07:59:39.178 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:39.169+0000 7f3e8de54700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3e8806fc50 0x7f3e8815ff60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:39.179 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:39.169+0000 7f3e8de54700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3e8806fc50 0x7f3e8815ff60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:55466/0 (socket says 172.21.15.59:55466) 2023-11-24T07:59:39.179 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:39.169+0000 7f3e8de54700 1 -- 172.21.15.59:0/1430834555 learned_addr learned my addr 172.21.15.59:0/1430834555 (peer_addr_for_me v2:172.21.15.59:0/0) 2023-11-24T07:59:39.179 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:39.169+0000 7f3e8e655700 1 --2- 172.21.15.59:0/1430834555 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3e8815e1c0 0x7f3e8806d310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:39.179 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:39.169+0000 7f3e8d653700 1 --2- 172.21.15.59:0/1430834555 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f3e880b0e90 0x7f3e881f3270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:39.180 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:39.169+0000 7f3e8de54700 1 -- 172.21.15.59:0/1430834555 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f3e880b0e90 msgr2=0x7f3e881f3270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:39.180 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:39.169+0000 7f3e8de54700 1 --2- 172.21.15.59:0/1430834555 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f3e880b0e90 0x7f3e881f3270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:39.180 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:39.169+0000 7f3e8de54700 1 -- 172.21.15.59:0/1430834555 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3e8815e1c0 msgr2=0x7f3e8806d310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:39.180 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:39.169+0000 7f3e8de54700 1 --2- 172.21.15.59:0/1430834555 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3e8815e1c0 0x7f3e8806d310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:39.180 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:39.169+0000 7f3e8de54700 1 -- 172.21.15.59:0/1430834555 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3e80009b30 con 0x7f3e8806fc50 2023-11-24T07:59:39.180 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:39.169+0000 7f3e8e655700 1 --2- 172.21.15.59:0/1430834555 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3e8815e1c0 0x7f3e8806d310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-24T07:59:39.181 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:39.169+0000 7f3e8de54700 1 --2- 172.21.15.59:0/1430834555 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3e8806fc50 0x7f3e8815ff60 secure :-1 s=READY pgs=963 cs=0 l=1 rev1=1 crypto rx=0x7f3e7800c7f0 tx=0x7f3e7800cbb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:59:39.181 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:39.169+0000 7f3e7e7fc700 1 -- 172.21.15.59:0/1430834555 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e78011800 con 0x7f3e8806fc50 2023-11-24T07:59:39.181 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:39.169+0000 7f3e7e7fc700 1 -- 172.21.15.59:0/1430834555 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3e78011960 con 0x7f3e8806fc50 2023-11-24T07:59:39.181 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:39.169+0000 7f3e900b8700 1 -- 172.21.15.59:0/1430834555 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3e880b5c50 con 0x7f3e8806fc50 2023-11-24T07:59:39.181 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:39.169+0000 7f3e7e7fc700 1 -- 172.21.15.59:0/1430834555 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e78011d10 con 0x7f3e8806fc50 2023-11-24T07:59:39.182 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:39.170+0000 7f3e900b8700 1 -- 172.21.15.59:0/1430834555 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3e880b6210 con 0x7f3e8806fc50 2023-11-24T07:59:39.182 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:39.170+0000 7f3e900b8700 1 -- 172.21.15.59:0/1430834555 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3e88068670 con 0x7f3e8806fc50 2023-11-24T07:59:39.182 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:39.171+0000 7f3e7e7fc700 1 -- 172.21.15.59:0/1430834555 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 11) v1 ==== 89741+0+0 (secure 0 0 0) 0x7f3e78018360 con 0x7f3e8806fc50 2023-11-24T07:59:39.182 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:39.171+0000 7f3e7e7fc700 1 --2- 172.21.15.59:0/1430834555 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f3e6c06b470 0x7f3e6c06d930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:39.182 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:39.171+0000 7f3e7e7fc700 1 -- 172.21.15.59:0/1430834555 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(75..75 src has 1..75) v4 ==== 5765+0+0 (secure 0 0 0) 0x7f3e7808a0a0 con 0x7f3e8806fc50 2023-11-24T07:59:39.182 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:39.172+0000 7f3e8d653700 1 --2- 172.21.15.59:0/1430834555 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f3e6c06b470 0x7f3e6c06d930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:39.183 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:39.172+0000 7f3e8d653700 1 --2- 172.21.15.59:0/1430834555 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f3e6c06b470 0x7f3e6c06d930 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f3e8809fcd0 tx=0x7f3e84009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:59:39.183 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:39.176+0000 7f3e7e7fc700 1 -- 172.21.15.59:0/1430834555 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f3e78054820 con 0x7f3e8806fc50 2023-11-24T07:59:39.377 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:39.376+0000 7f3e900b8700 1 -- 172.21.15.59:0/1430834555 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f3e880b68f0 con 0x7f3e8806fc50 2023-11-24T07:59:39.379 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:39.377+0000 7f3e7e7fc700 1 -- 172.21.15.59:0/1430834555 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v75) v1 ==== 74+0+16546 (secure 0 0 0) 0x7f3e7801f090 con 0x7f3e8806fc50 2023-11-24T07:59:39.379 INFO:teuthology.orchestra.run.smithi059.stdout: 2023-11-24T07:59:39.379 INFO:teuthology.orchestra.run.smithi059.stdout:{"epoch":75,"fsid":"9f0e8741-b8f9-48d5-a2c3-09f45903303d","created":"2023-11-24T07:56:06.552729+0000","modified":"2023-11-24T07:59:34.419418+0000","last_up_change":"2023-11-24T07:56:17.372352+0000","last_in_change":"2023-11-24T07:56:09.073250+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":11,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-24T07:56:18.352823+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_metadata","create_time":"2023-11-24T07:59:11.900882+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"73","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_data","create_time":"2023-11-24T07:59:13.025749+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"75","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":3,"snap_epoch":75,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"6d8f9e9b-1782-4e41-928b-a39f2a8ebc06","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6800","nonce":3373878772},{"type":"v1","addr":"172.21.15.59:6801","nonce":3373878772}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6802","nonce":3373878772},{"type":"v1","addr":"172.21.15.59:6803","nonce":3373878772}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6806","nonce":3373878772},{"type":"v1","addr":"172.21.15.59:6807","nonce":3373878772}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6804","nonce":3373878772},{"type":"v1","addr":"172.21.15.59:6805","nonce":3373878772}]},"public_addr":"172.21.15.59:6801/3373878772","cluster_addr":"172.21.15.59:6803/3373878772","heartbeat_back_addr":"172.21.15.59:6807/3373878772","heartbeat_front_addr":"172.21.15.59:6805/3373878772","state":["exists","up"]},{"osd":1,"uuid":"e8d882aa-52e1-4c04-9e1e-c44ef9204a5f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6824","nonce":648258563},{"type":"v1","addr":"172.21.15.59:6825","nonce":648258563}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6826","nonce":648258563},{"type":"v1","addr":"172.21.15.59:6827","nonce":648258563}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6830","nonce":648258563},{"type":"v1","addr":"172.21.15.59:6831","nonce":648258563}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6828","nonce":648258563},{"type":"v1","addr":"172.21.15.59:6829","nonce":648258563}]},"public_addr":"172.21.15.59:6825/648258563","cluster_addr":"172.21.15.59:6827/648258563","heartbeat_back_addr":"172.21.15.59:6831/648258563","heartbeat_front_addr":"172.21.15.59:6829/648258563","state":["exists","up"]},{"osd":2,"uuid":"69eebfa0-695b-4b21-90b9-8a3537168a01","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6808","nonce":1840430654},{"type":"v1","addr":"172.21.15.59:6809","nonce":1840430654}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6810","nonce":1840430654},{"type":"v1","addr":"172.21.15.59:6811","nonce":1840430654}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6814","nonce":1840430654},{"type":"v1","addr":"172.21.15.59:6815","nonce":1840430654}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6812","nonce":1840430654},{"type":"v1","addr":"172.21.15.59:6813","nonce":1840430654}]},"public_addr":"172.21.15.59:6809/1840430654","cluster_addr":"172.21.15.59:6811/1840430654","heartbeat_back_addr":"172.21.15.59:6815/1840430654","heartbeat_front_addr":"172.21.15.59:6813/1840430654","state":["exists","up"]},{"osd":3,"uuid":"96fd77fc-f911-46ff-9445-6f0be2daa8f4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6816","nonce":2504025850},{"type":"v1","addr":"172.21.15.59:6817","nonce":2504025850}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6818","nonce":2504025850},{"type":"v1","addr":"172.21.15.59:6819","nonce":2504025850}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6822","nonce":2504025850},{"type":"v1","addr":"172.21.15.59:6823","nonce":2504025850}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6820","nonce":2504025850},{"type":"v1","addr":"172.21.15.59:6821","nonce":2504025850}]},"public_addr":"172.21.15.59:6817/2504025850","cluster_addr":"172.21.15.59:6819/2504025850","heartbeat_back_addr":"172.21.15.59:6823/2504025850","heartbeat_front_addr":"172.21.15.59:6821/2504025850","state":["exists","up"]},{"osd":4,"uuid":"7ac29fca-b5b2-4c95-9d69-2f164cbbc708","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6816","nonce":2382463431},{"type":"v1","addr":"172.21.15.76:6818","nonce":2382463431}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6820","nonce":2382463431},{"type":"v1","addr":"172.21.15.76:6822","nonce":2382463431}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6828","nonce":2382463431},{"type":"v1","addr":"172.21.15.76:6830","nonce":2382463431}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6824","nonce":2382463431},{"type":"v1","addr":"172.21.15.76:6826","nonce":2382463431}]},"public_addr":"172.21.15.76:6818/2382463431","cluster_addr":"172.21.15.76:6822/2382463431","heartbeat_back_addr":"172.21.15.76:6830/2382463431","heartbeat_front_addr":"172.21.15.76:6826/2382463431","state":["exists","up"]},{"osd":5,"uuid":"485bf711-3851-4238-a4f4-51d52e0a9c58","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6800","nonce":329334308},{"type":"v1","addr":"172.21.15.76:6802","nonce":329334308}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6805","nonce":329334308},{"type":"v1","addr":"172.21.15.76:6807","nonce":329334308}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6814","nonce":329334308},{"type":"v1","addr":"172.21.15.76:6815","nonce":329334308}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6810","nonce":329334308},{"type":"v1","addr":"172.21.15.76:6812","nonce":329334308}]},"public_addr":"172.21.15.76:6802/329334308","cluster_addr":"172.21.15.76:6807/329334308","heartbeat_back_addr":"172.21.15.76:6815/329334308","heartbeat_front_addr":"172.21.15.76:6812/329334308","state":["exists","up"]},{"osd":6,"uuid":"72b3f2fc-1e4a-4e32-968d-003647a0286f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6801","nonce":3946579410},{"type":"v1","addr":"172.21.15.76:6803","nonce":3946579410}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6804","nonce":3946579410},{"type":"v1","addr":"172.21.15.76:6806","nonce":3946579410}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6811","nonce":3946579410},{"type":"v1","addr":"172.21.15.76:6813","nonce":3946579410}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6808","nonce":3946579410},{"type":"v1","addr":"172.21.15.76:6809","nonce":3946579410}]},"public_addr":"172.21.15.76:6803/3946579410","cluster_addr":"172.21.15.76:6806/3946579410","heartbeat_back_addr":"172.21.15.76:6813/3946579410","heartbeat_front_addr":"172.21.15.76:6809/3946579410","state":["exists","up"]},{"osd":7,"uuid":"1ddceb90-6c5d-4d1a-ad56-99cb877f8c70","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6817","nonce":841761538},{"type":"v1","addr":"172.21.15.76:6819","nonce":841761538}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6821","nonce":841761538},{"type":"v1","addr":"172.21.15.76:6823","nonce":841761538}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6829","nonce":841761538},{"type":"v1","addr":"172.21.15.76:6831","nonce":841761538}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6825","nonce":841761538},{"type":"v1","addr":"172.21.15.76:6827","nonce":841761538}]},"public_addr":"172.21.15.76:6819/841761538","cluster_addr":"172.21.15.76:6823/841761538","heartbeat_back_addr":"172.21.15.76:6831/841761538","heartbeat_front_addr":"172.21.15.76:6827/841761538","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T07:56:14.294838+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T07:56:14.270779+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T07:56:14.231809+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T07:56:14.260983+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T07:56:14.302997+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T07:56:14.258600+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T07:56:14.251413+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T07:56:14.304727+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":11,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[{"pool":11,"snaps":[{"begin":2,"length":2}]}],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-24T07:59:39.381 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:39.379+0000 7f3e900b8700 1 -- 172.21.15.59:0/1430834555 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f3e6c06b470 msgr2=0x7f3e6c06d930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:39.382 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:39.379+0000 7f3e900b8700 1 --2- 172.21.15.59:0/1430834555 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f3e6c06b470 0x7f3e6c06d930 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f3e8809fcd0 tx=0x7f3e84009040 comp rx=0 tx=0).stop 2023-11-24T07:59:39.382 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:39.379+0000 7f3e900b8700 1 -- 172.21.15.59:0/1430834555 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3e8806fc50 msgr2=0x7f3e8815ff60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:39.382 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:39.379+0000 7f3e900b8700 1 --2- 172.21.15.59:0/1430834555 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3e8806fc50 0x7f3e8815ff60 secure :-1 s=READY pgs=963 cs=0 l=1 rev1=1 crypto rx=0x7f3e7800c7f0 tx=0x7f3e7800cbb0 comp rx=0 tx=0).stop 2023-11-24T07:59:39.383 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:39.379+0000 7f3e900b8700 1 -- 172.21.15.59:0/1430834555 shutdown_connections 2023-11-24T07:59:39.383 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:39.379+0000 7f3e900b8700 1 --2- 172.21.15.59:0/1430834555 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f3e6c06b470 0x7f3e6c06d930 unknown :-1 s=CLOSED pgs=446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:39.383 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:39.379+0000 7f3e900b8700 1 --2- 172.21.15.59:0/1430834555 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3e8815e1c0 0x7f3e8806d310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:39.383 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:39.379+0000 7f3e900b8700 1 --2- 172.21.15.59:0/1430834555 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3e8806fc50 0x7f3e8815ff60 unknown :-1 s=CLOSED pgs=963 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:39.384 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:39.379+0000 7f3e900b8700 1 --2- 172.21.15.59:0/1430834555 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f3e880b0e90 0x7f3e881f3270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:39.384 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:39.379+0000 7f3e900b8700 1 -- 172.21.15.59:0/1430834555 >> 172.21.15.59:0/1430834555 conn(0x7f3e880b3720 msgr2=0x7f3e8815b220 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:59:39.384 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:39.379+0000 7f3e900b8700 1 -- 172.21.15.59:0/1430834555 shutdown_connections 2023-11-24T07:59:39.384 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:39.379+0000 7f3e900b8700 1 -- 172.21.15.59:0/1430834555 wait complete. 2023-11-24T07:59:39.395 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2023-11-24T07:59:39.527 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:39.524+0000 7f5d03ca3700 1 Processor -- start 2023-11-24T07:59:39.527 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:39.525+0000 7f5d03ca3700 1 -- start start 2023-11-24T07:59:39.530 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:39.525+0000 7f5d03ca3700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f5cfc0726a0 0x7f5cfc072a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:39.530 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:39.525+0000 7f5d03ca3700 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5cfc15f3e0 con 0x7f5cfc0b3d90 2023-11-24T07:59:39.530 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:39.525+0000 7f5d03ca3700 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5cfc06c8e0 con 0x7f5cfc072fc0 2023-11-24T07:59:39.530 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:39.525+0000 7f5d03ca3700 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5cfc0affd0 con 0x7f5cfc0726a0 2023-11-24T07:59:39.531 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:39.526+0000 7f5d01a3f700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f5cfc0726a0 0x7f5cfc072a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:39.531 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:39.526+0000 7f5d0123e700 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7f5cfc0b3d90 0x7f5cfc06bac0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:58948/0 (socket says 172.21.15.59:58948) 2023-11-24T07:59:39.531 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:39.526+0000 7f5d01a3f700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f5cfc0726a0 0x7f5cfc072a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:55474/0 (socket says 172.21.15.59:55474) 2023-11-24T07:59:39.531 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:39.526+0000 7f5d0123e700 1 -- 172.21.15.59:0/510512355 learned_addr learned my addr 172.21.15.59:0/510512355 (peer_addr_for_me v1:172.21.15.59:0/0) 2023-11-24T07:59:39.531 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:39.526+0000 7f5d01a3f700 1 -- 172.21.15.59:0/510512355 >> v1:172.21.15.76:6789/0 conn(0x7f5cfc072fc0 legacy=0x7f5cfc06c1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T07:59:39.532 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:39.526+0000 7f5d01a3f700 1 -- 172.21.15.59:0/510512355 >> v1:172.21.15.59:6789/0 conn(0x7f5cfc0b3d90 legacy=0x7f5cfc06bac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:39.532 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:39.526+0000 7f5d01a3f700 1 -- 172.21.15.59:0/510512355 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5cfc15e7d0 con 0x7f5cfc0726a0 2023-11-24T07:59:39.532 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:39.526+0000 7f5d00a3d700 1 -- 172.21.15.59:0/510512355 <== mon.0 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2674522565 0 0) 0x7f5cfc15f3e0 con 0x7f5cfc0b3d90 2023-11-24T07:59:39.533 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:39.526+0000 7f5d01a3f700 1 --2- 172.21.15.59:0/510512355 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5cfc0726a0 0x7f5cfc072a80 secure :-1 s=READY pgs=965 cs=0 l=1 rev1=1 crypto rx=0x7f5cec009870 tx=0x7f5cec009b80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4a0fcadefd621407 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:59:39.533 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:39.526+0000 7f5d00a3d700 1 -- 172.21.15.59:0/510512355 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5cec004230 con 0x7f5cfc0726a0 2023-11-24T07:59:39.533 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:39.526+0000 7f5d00a3d700 1 -- 172.21.15.59:0/510512355 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5cec004390 con 0x7f5cfc0726a0 2023-11-24T07:59:39.533 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:39.526+0000 7f5d00a3d700 1 -- 172.21.15.59:0/510512355 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5cec0036a0 con 0x7f5cfc0726a0 2023-11-24T07:59:39.534 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:39.527+0000 7f5d03ca3700 1 -- 172.21.15.59:0/510512355 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5cfc0726a0 msgr2=0x7f5cfc072a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:39.534 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:39.527+0000 7f5d03ca3700 1 --2- 172.21.15.59:0/510512355 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5cfc0726a0 0x7f5cfc072a80 secure :-1 s=READY pgs=965 cs=0 l=1 rev1=1 crypto rx=0x7f5cec009870 tx=0x7f5cec009b80 comp rx=0 tx=0).stop 2023-11-24T07:59:39.535 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:39.527+0000 7f5d03ca3700 1 -- 172.21.15.59:0/510512355 shutdown_connections 2023-11-24T07:59:39.535 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:39.527+0000 7f5d03ca3700 1 --2- 172.21.15.59:0/510512355 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5cfc0726a0 0x7f5cfc072a80 unknown :-1 s=CLOSED pgs=965 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:39.535 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:39.527+0000 7f5d03ca3700 1 -- 172.21.15.59:0/510512355 >> 172.21.15.59:0/510512355 conn(0x7f5cfc0717f0 msgr2=0x7f5cfc0b5070 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:59:39.535 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:39.527+0000 7f5d03ca3700 1 -- 172.21.15.59:0/510512355 shutdown_connections 2023-11-24T07:59:39.535 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:39.527+0000 7f5d03ca3700 1 -- 172.21.15.59:0/510512355 wait complete. 2023-11-24T07:59:39.536 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:39.528+0000 7f5d03ca3700 1 Processor -- start 2023-11-24T07:59:39.536 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:39.528+0000 7f5d03ca3700 1 -- start start 2023-11-24T07:59:39.541 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:39.529+0000 7f5d03ca3700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5cfc0726a0 0x7f5cfc158cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:39.541 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:39.529+0000 7f5d03ca3700 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f5cfc072fc0 0x7f5cfc159900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:39.541 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:39.529+0000 7f5d03ca3700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5cfc0b3d90 0x7f5cfc1f4e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:39.541 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:39.529+0000 7f5d03ca3700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5cfc158830 con 0x7f5cfc0726a0 2023-11-24T07:59:39.542 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:39.529+0000 7f5d03ca3700 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5cfc157c20 con 0x7f5cfc0b3d90 2023-11-24T07:59:39.542 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:39.529+0000 7f5d03ca3700 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5cfc157f00 con 0x7f5cfc072fc0 2023-11-24T07:59:39.542 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:39.529+0000 7f5d01a3f700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5cfc0726a0 0x7f5cfc158cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:39.542 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:39.529+0000 7f5d01a3f700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5cfc0726a0 0x7f5cfc158cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:55478/0 (socket says 172.21.15.59:55478) 2023-11-24T07:59:39.543 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:39.529+0000 7f5d01a3f700 1 -- 172.21.15.59:0/2518271706 learned_addr learned my addr 172.21.15.59:0/2518271706 (peer_addr_for_me v2:172.21.15.59:0/0) 2023-11-24T07:59:39.543 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:39.529+0000 7f5d0123e700 1 --2- 172.21.15.59:0/2518271706 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f5cfc072fc0 0x7f5cfc159900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:39.543 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:39.529+0000 7f5d01a3f700 1 -- 172.21.15.59:0/2518271706 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f5cfc072fc0 msgr2=0x7f5cfc159900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:39.543 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:39.529+0000 7f5d02240700 1 --2- 172.21.15.59:0/2518271706 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5cfc0b3d90 0x7f5cfc1f4e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:39.544 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:39.529+0000 7f5d01a3f700 1 --2- 172.21.15.59:0/2518271706 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f5cfc072fc0 0x7f5cfc159900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:39.544 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:39.529+0000 7f5d01a3f700 1 -- 172.21.15.59:0/2518271706 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5cfc0b3d90 msgr2=0x7f5cfc1f4e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:39.544 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:39.529+0000 7f5d01a3f700 1 --2- 172.21.15.59:0/2518271706 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5cfc0b3d90 0x7f5cfc1f4e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:39.544 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:39.529+0000 7f5d01a3f700 1 -- 172.21.15.59:0/2518271706 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5cec0094a0 con 0x7f5cfc0726a0 2023-11-24T07:59:39.545 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:39.529+0000 7f5d02240700 1 --2- 172.21.15.59:0/2518271706 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5cfc0b3d90 0x7f5cfc1f4e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T07:59:39.545 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:39.529+0000 7f5d01a3f700 1 --2- 172.21.15.59:0/2518271706 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5cfc0726a0 0x7f5cfc158cc0 secure :-1 s=READY pgs=966 cs=0 l=1 rev1=1 crypto rx=0x7f5cec0090b0 tx=0x7f5cec004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:59:39.545 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:39.529+0000 7f5cf27fc700 1 -- 172.21.15.59:0/2518271706 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5cec011830 con 0x7f5cfc0726a0 2023-11-24T07:59:39.545 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:39.530+0000 7f5cf27fc700 1 -- 172.21.15.59:0/2518271706 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5cec01e070 con 0x7f5cfc0726a0 2023-11-24T07:59:39.545 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:39.530+0000 7f5d03ca3700 1 -- 172.21.15.59:0/2518271706 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5cfc1581e0 con 0x7f5cfc0726a0 2023-11-24T07:59:39.546 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:39.530+0000 7f5cf27fc700 1 -- 172.21.15.59:0/2518271706 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5cec011bf0 con 0x7f5cfc0726a0 2023-11-24T07:59:39.546 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:39.530+0000 7f5d03ca3700 1 -- 172.21.15.59:0/2518271706 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5cfc157880 con 0x7f5cfc0726a0 2023-11-24T07:59:39.546 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:39.531+0000 7f5d03ca3700 1 -- 172.21.15.59:0/2518271706 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5cfc068670 con 0x7f5cfc0726a0 2023-11-24T07:59:39.547 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:39.531+0000 7f5cf27fc700 1 -- 172.21.15.59:0/2518271706 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 11) v1 ==== 89741+0+0 (secure 0 0 0) 0x7f5cec011d50 con 0x7f5cfc0726a0 2023-11-24T07:59:39.547 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:39.531+0000 7f5cf27fc700 1 --2- 172.21.15.59:0/2518271706 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f5ce006b470 0x7f5ce006d930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:39.547 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:39.532+0000 7f5cf27fc700 1 -- 172.21.15.59:0/2518271706 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(75..75 src has 1..75) v4 ==== 5765+0+0 (secure 0 0 0) 0x7f5cec08b430 con 0x7f5cfc0726a0 2023-11-24T07:59:39.547 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:39.532+0000 7f5d0123e700 1 --2- 172.21.15.59:0/2518271706 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f5ce006b470 0x7f5ce006d930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:39.548 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:39.532+0000 7f5d0123e700 1 --2- 172.21.15.59:0/2518271706 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f5ce006b470 0x7f5ce006d930 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f5cfc156f70 tx=0x7f5cf8009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:59:39.548 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:39.536+0000 7f5cf27fc700 1 -- 172.21.15.59:0/2518271706 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f5cec055cf0 con 0x7f5cfc0726a0 2023-11-24T07:59:39.786 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:39.784+0000 7f5d03ca3700 1 -- 172.21.15.59:0/2518271706 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f5cfc0d0bd0 con 0x7f5cfc0726a0 2023-11-24T07:59:39.787 INFO:tasks.ceph.mon.a.smithi059.stderr:2023-11-24T07:59:39.784+0000 7f5560821700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-11-24T07:59:39.789 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:39.788+0000 7f5cf27fc700 1 -- 172.21.15.59:0/2518271706 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v46) v1 ==== 161+0+0 (secure 0 0 0) 0x7f5cec076e60 con 0x7f5cfc0726a0 2023-11-24T07:59:39.792 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:39.791+0000 7f5d03ca3700 1 -- 172.21.15.59:0/2518271706 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f5ce006b470 msgr2=0x7f5ce006d930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:39.793 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:39.791+0000 7f5d03ca3700 1 --2- 172.21.15.59:0/2518271706 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f5ce006b470 0x7f5ce006d930 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f5cfc156f70 tx=0x7f5cf8009040 comp rx=0 tx=0).stop 2023-11-24T07:59:39.793 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:39.791+0000 7f5d03ca3700 1 -- 172.21.15.59:0/2518271706 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5cfc0726a0 msgr2=0x7f5cfc158cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:39.793 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:39.791+0000 7f5d03ca3700 1 --2- 172.21.15.59:0/2518271706 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5cfc0726a0 0x7f5cfc158cc0 secure :-1 s=READY pgs=966 cs=0 l=1 rev1=1 crypto rx=0x7f5cec0090b0 tx=0x7f5cec004750 comp rx=0 tx=0).stop 2023-11-24T07:59:39.793 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:39.791+0000 7f5d03ca3700 1 -- 172.21.15.59:0/2518271706 shutdown_connections 2023-11-24T07:59:39.793 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:39.791+0000 7f5d03ca3700 1 --2- 172.21.15.59:0/2518271706 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f5ce006b470 0x7f5ce006d930 unknown :-1 s=CLOSED pgs=447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:39.793 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:39.791+0000 7f5d03ca3700 1 --2- 172.21.15.59:0/2518271706 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5cfc0b3d90 0x7f5cfc1f4e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:39.793 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:39.791+0000 7f5d03ca3700 1 --2- 172.21.15.59:0/2518271706 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5cfc0726a0 0x7f5cfc158cc0 unknown :-1 s=CLOSED pgs=966 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:39.794 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:39.791+0000 7f5d03ca3700 1 --2- 172.21.15.59:0/2518271706 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f5cfc072fc0 0x7f5cfc159900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:39.794 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:39.791+0000 7f5d03ca3700 1 -- 172.21.15.59:0/2518271706 >> 172.21.15.59:0/2518271706 conn(0x7f5cfc0717f0 msgr2=0x7f5cfc15d8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:59:39.795 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:39.791+0000 7f5d03ca3700 1 -- 172.21.15.59:0/2518271706 shutdown_connections 2023-11-24T07:59:39.795 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:39.791+0000 7f5d03ca3700 1 -- 172.21.15.59:0/2518271706 wait complete. 2023-11-24T07:59:39.795 INFO:teuthology.orchestra.run.smithi059.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2023-11-24T07:59:39.806 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2023-11-24T07:59:39.831 INFO:tasks.ceph.mds.c.smithi059.stdout:starting mds.c at 2023-11-24T07:59:39.937 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:39.935+0000 7f62ad92f700 1 Processor -- start 2023-11-24T07:59:39.937 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:39.935+0000 7f62ad92f700 1 -- start start 2023-11-24T07:59:39.939 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:39.936+0000 7f62ad92f700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f62a8072250 0x7f62a80cc1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:39.939 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:39.936+0000 7f62ad92f700 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f62a815f5b0 con 0x7f62a813dd00 2023-11-24T07:59:39.940 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:39.936+0000 7f62ad92f700 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f62a813e0e0 con 0x7f62a80cf970 2023-11-24T07:59:39.940 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:39.936+0000 7f62ad92f700 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f62a80d0690 con 0x7f62a8072250 2023-11-24T07:59:39.940 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:39.936+0000 7f62a77fe700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f62a8072250 0x7f62a80cc1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:39.940 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:39.936+0000 7f62a77fe700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f62a8072250 0x7f62a80cc1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:55502/0 (socket says 172.21.15.59:55502) 2023-11-24T07:59:39.940 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:39.936+0000 7f62a77fe700 1 -- 172.21.15.59:0/2846295835 learned_addr learned my addr 172.21.15.59:0/2846295835 (peer_addr_for_me v2:172.21.15.59:0/0) 2023-11-24T07:59:39.940 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:39.936+0000 7f62a6ffd700 1 --1- 172.21.15.59:0/2846295835 >> v1:172.21.15.59:6789/0 conn(0x7f62a813dd00 0x7f62a80cff80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:58972/0 (socket says 172.21.15.59:58972) 2023-11-24T07:59:39.940 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:39.937+0000 7f62a77fe700 1 -- 172.21.15.59:0/2846295835 >> v1:172.21.15.76:6789/0 conn(0x7f62a80cf970 legacy=0x7f62a8071b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:39.940 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:39.937+0000 7f62a77fe700 1 -- 172.21.15.59:0/2846295835 >> v1:172.21.15.59:6789/0 conn(0x7f62a813dd00 legacy=0x7f62a80cff80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:39.940 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:39.937+0000 7f62a77fe700 1 -- 172.21.15.59:0/2846295835 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f62a815e9a0 con 0x7f62a8072250 2023-11-24T07:59:39.940 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:39.937+0000 7f62a5ffb700 1 -- 172.21.15.59:0/2846295835 <== mon.0 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3876504100 0 0) 0x7f62a815f5b0 con 0x7f62a813dd00 2023-11-24T07:59:39.940 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:39.937+0000 7f62a77fe700 1 --2- 172.21.15.59:0/2846295835 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f62a8072250 0x7f62a80cc1a0 secure :-1 s=READY pgs=971 cs=0 l=1 rev1=1 crypto rx=0x7f62a000d150 tx=0x7f62a000d460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6369f2faf3f85bc6 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:59:39.940 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:39.937+0000 7f62a5ffb700 1 -- 172.21.15.59:0/2846295835 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62a0010070 con 0x7f62a8072250 2023-11-24T07:59:39.941 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:39.937+0000 7f62a5ffb700 1 -- 172.21.15.59:0/2846295835 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f62a000ddc0 con 0x7f62a8072250 2023-11-24T07:59:39.941 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:39.937+0000 7f62a5ffb700 1 -- 172.21.15.59:0/2846295835 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62a000b310 con 0x7f62a8072250 2023-11-24T07:59:39.941 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:39.938+0000 7f62ad92f700 1 -- 172.21.15.59:0/2846295835 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f62a8072250 msgr2=0x7f62a80cc1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:39.941 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:39.938+0000 7f62ad92f700 1 --2- 172.21.15.59:0/2846295835 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f62a8072250 0x7f62a80cc1a0 secure :-1 s=READY pgs=971 cs=0 l=1 rev1=1 crypto rx=0x7f62a000d150 tx=0x7f62a000d460 comp rx=0 tx=0).stop 2023-11-24T07:59:39.941 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:39.938+0000 7f62ad92f700 1 -- 172.21.15.59:0/2846295835 shutdown_connections 2023-11-24T07:59:39.941 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:39.938+0000 7f62ad92f700 1 --2- 172.21.15.59:0/2846295835 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f62a8072250 0x7f62a80cc1a0 unknown :-1 s=CLOSED pgs=971 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:39.941 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:39.938+0000 7f62ad92f700 1 -- 172.21.15.59:0/2846295835 >> 172.21.15.59:0/2846295835 conn(0x7f62a80d0b10 msgr2=0x7f62a80d0f20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:59:39.941 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:39.938+0000 7f62ad92f700 1 -- 172.21.15.59:0/2846295835 shutdown_connections 2023-11-24T07:59:39.941 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:39.938+0000 7f62ad92f700 1 -- 172.21.15.59:0/2846295835 wait complete. 2023-11-24T07:59:39.943 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:39.939+0000 7f62ad92f700 1 Processor -- start 2023-11-24T07:59:39.944 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:39.939+0000 7f62ad92f700 1 -- start start 2023-11-24T07:59:39.944 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:39.940+0000 7f62ad92f700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f62a8072250 0x7f62a813b090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:39.944 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:39.940+0000 7f62ad92f700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f62a80cf970 0x7f62a80ccdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:39.944 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:39.940+0000 7f62ad92f700 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f62a813dd00 0x7f62a806db60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:39.944 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:39.940+0000 7f62ad92f700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62a80c9600 con 0x7f62a8072250 2023-11-24T07:59:39.945 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:39.940+0000 7f62ad92f700 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62a813bb70 con 0x7f62a80cf970 2023-11-24T07:59:39.945 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:39.940+0000 7f62ad92f700 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f62a80ca420 con 0x7f62a813dd00 2023-11-24T07:59:39.945 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:39.940+0000 7f62a6ffd700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f62a8072250 0x7f62a813b090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:39.945 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:39.940+0000 7f62a6ffd700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f62a8072250 0x7f62a813b090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:55504/0 (socket says 172.21.15.59:55504) 2023-11-24T07:59:39.945 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:39.940+0000 7f62a6ffd700 1 -- 172.21.15.59:0/382936885 learned_addr learned my addr 172.21.15.59:0/382936885 (peer_addr_for_me v2:172.21.15.59:0/0) 2023-11-24T07:59:39.945 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:39.940+0000 7f62a67fc700 1 --2- 172.21.15.59:0/382936885 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f62a80cf970 0x7f62a80ccdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:39.945 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:39.940+0000 7f62a77fe700 1 --2- 172.21.15.59:0/382936885 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f62a813dd00 0x7f62a806db60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:39.945 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:39.940+0000 7f62a6ffd700 1 -- 172.21.15.59:0/382936885 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f62a813dd00 msgr2=0x7f62a806db60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:39.945 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:39.940+0000 7f62a6ffd700 1 --2- 172.21.15.59:0/382936885 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f62a813dd00 0x7f62a806db60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:39.945 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:39.940+0000 7f62a6ffd700 1 -- 172.21.15.59:0/382936885 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f62a80cf970 msgr2=0x7f62a80ccdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:39.946 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:39.940+0000 7f62a6ffd700 1 --2- 172.21.15.59:0/382936885 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f62a80cf970 0x7f62a80ccdc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:39.946 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:39.940+0000 7f62a6ffd700 1 -- 172.21.15.59:0/382936885 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f62a000bd40 con 0x7f62a8072250 2023-11-24T07:59:39.946 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:39.940+0000 7f62a67fc700 1 --2- 172.21.15.59:0/382936885 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f62a80cf970 0x7f62a80ccdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-24T07:59:39.946 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:39.940+0000 7f62a6ffd700 1 --2- 172.21.15.59:0/382936885 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f62a8072250 0x7f62a813b090 secure :-1 s=READY pgs=972 cs=0 l=1 rev1=1 crypto rx=0x7f629c00c550 tx=0x7f629c00c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:59:39.946 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:39.940+0000 7f628ffff700 1 -- 172.21.15.59:0/382936885 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f629c004c80 con 0x7f62a8072250 2023-11-24T07:59:39.946 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:39.941+0000 7f628ffff700 1 -- 172.21.15.59:0/382936885 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f629c004de0 con 0x7f62a8072250 2023-11-24T07:59:39.946 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:39.941+0000 7f62ad92f700 1 -- 172.21.15.59:0/382936885 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f62a813b5d0 con 0x7f62a8072250 2023-11-24T07:59:39.946 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:39.941+0000 7f628ffff700 1 -- 172.21.15.59:0/382936885 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f629c007310 con 0x7f62a8072250 2023-11-24T07:59:39.946 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:39.941+0000 7f62ad92f700 1 -- 172.21.15.59:0/382936885 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f62a80c9af0 con 0x7f62a8072250 2023-11-24T07:59:39.946 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:39.942+0000 7f62ad92f700 1 -- 172.21.15.59:0/382936885 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f62a8075480 con 0x7f62a8072250 2023-11-24T07:59:39.946 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:39.942+0000 7f628ffff700 1 -- 172.21.15.59:0/382936885 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 11) v1 ==== 89741+0+0 (secure 0 0 0) 0x7f629c007470 con 0x7f62a8072250 2023-11-24T07:59:39.947 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:39.943+0000 7f628ffff700 1 --2- 172.21.15.59:0/382936885 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f629406b470 0x7f629406d930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:39.947 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:39.943+0000 7f628ffff700 1 -- 172.21.15.59:0/382936885 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(76..76 src has 1..76) v4 ==== 5823+0+0 (secure 0 0 0) 0x7f629c089e40 con 0x7f62a8072250 2023-11-24T07:59:39.947 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:39.943+0000 7f62a67fc700 1 --2- 172.21.15.59:0/382936885 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f629406b470 0x7f629406d930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:39.947 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:39.943+0000 7f62a67fc700 1 --2- 172.21.15.59:0/382936885 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f629406b470 0x7f629406d930 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f62a813d340 tx=0x7f6298009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:59:39.950 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:39.947+0000 7f628ffff700 1 -- 172.21.15.59:0/382936885 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f629c054580 con 0x7f62a8072250 2023-11-24T07:59:40.198 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:40.196+0000 7f62ad92f700 1 -- 172.21.15.59:0/382936885 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f62a813d520 con 0x7f62a8072250 2023-11-24T07:59:40.796 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:40.794+0000 7f628ffff700 1 -- 172.21.15.59:0/382936885 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v47) v1 ==== 104+0+0 (secure 0 0 0) 0x7f629c057ba0 con 0x7f62a8072250 2023-11-24T07:59:40.799 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:40.797+0000 7f62ad92f700 1 -- 172.21.15.59:0/382936885 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f629406b470 msgr2=0x7f629406d930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:40.799 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:40.797+0000 7f62ad92f700 1 --2- 172.21.15.59:0/382936885 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f629406b470 0x7f629406d930 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f62a813d340 tx=0x7f6298009040 comp rx=0 tx=0).stop 2023-11-24T07:59:40.799 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:40.797+0000 7f62ad92f700 1 -- 172.21.15.59:0/382936885 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f62a8072250 msgr2=0x7f62a813b090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:40.799 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:40.797+0000 7f62ad92f700 1 --2- 172.21.15.59:0/382936885 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f62a8072250 0x7f62a813b090 secure :-1 s=READY pgs=972 cs=0 l=1 rev1=1 crypto rx=0x7f629c00c550 tx=0x7f629c00c910 comp rx=0 tx=0).stop 2023-11-24T07:59:40.799 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:40.797+0000 7f62ad92f700 1 -- 172.21.15.59:0/382936885 shutdown_connections 2023-11-24T07:59:40.799 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:40.797+0000 7f62ad92f700 1 --2- 172.21.15.59:0/382936885 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f629406b470 0x7f629406d930 unknown :-1 s=CLOSED pgs=448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:40.799 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:40.797+0000 7f62ad92f700 1 --2- 172.21.15.59:0/382936885 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f62a813dd00 0x7f62a806db60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:40.799 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:40.797+0000 7f62ad92f700 1 --2- 172.21.15.59:0/382936885 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f62a8072250 0x7f62a813b090 unknown :-1 s=CLOSED pgs=972 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:40.800 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:40.797+0000 7f62ad92f700 1 --2- 172.21.15.59:0/382936885 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f62a80cf970 0x7f62a80ccdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:40.800 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:40.797+0000 7f62ad92f700 1 -- 172.21.15.59:0/382936885 >> 172.21.15.59:0/382936885 conn(0x7f62a80d0b10 msgr2=0x7f62a80c7420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:59:40.800 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:40.797+0000 7f62ad92f700 1 -- 172.21.15.59:0/382936885 shutdown_connections 2023-11-24T07:59:40.800 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:40.797+0000 7f62ad92f700 1 -- 172.21.15.59:0/382936885 wait complete. 2023-11-24T07:59:40.813 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2023-11-24T07:59:40.987 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:40.985+0000 7fae65f7d700 1 Processor -- start 2023-11-24T07:59:40.987 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:40.985+0000 7fae65f7d700 1 -- start start 2023-11-24T07:59:40.989 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:40.986+0000 7fae65f7d700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fae6006afd0 0x7fae6006b3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:40.989 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:40.986+0000 7fae65f7d700 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fae60071c10 0x7fae60072040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:40.989 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:40.986+0000 7fae65f7d700 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fae6006b8f0 con 0x7fae600add30 2023-11-24T07:59:40.989 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:40.986+0000 7fae65f7d700 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fae6015e5c0 con 0x7fae6006afd0 2023-11-24T07:59:40.989 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:40.986+0000 7fae65f7d700 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fae6015e8a0 con 0x7fae60071c10 2023-11-24T07:59:40.989 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:40.986+0000 7fae5f7fe700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fae6006afd0 0x7fae6006b3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:40.990 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:40.986+0000 7fae5f7fe700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fae6006afd0 0x7fae6006b3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:55514/0 (socket says 172.21.15.59:55514) 2023-11-24T07:59:40.990 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:40.986+0000 7fae5f7fe700 1 -- 172.21.15.59:0/3269471280 learned_addr learned my addr 172.21.15.59:0/3269471280 (peer_addr_for_me v2:172.21.15.59:0/0) 2023-11-24T07:59:40.990 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:40.986+0000 7fae5ffff700 1 --2- 172.21.15.59:0/3269471280 >> v2:172.21.15.76:3300/0 conn(0x7fae60071c10 0x7fae60072040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:40.990 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:40.986+0000 7fae5f7fe700 1 -- 172.21.15.59:0/3269471280 >> v2:172.21.15.76:3300/0 conn(0x7fae60071c10 msgr2=0x7fae60072040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T07:59:40.990 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:40.986+0000 7fae5f7fe700 1 --2- 172.21.15.59:0/3269471280 >> v2:172.21.15.76:3300/0 conn(0x7fae60071c10 0x7fae60072040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:40.990 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:40.986+0000 7fae5f7fe700 1 -- 172.21.15.59:0/3269471280 >> v1:172.21.15.76:6789/0 conn(0x7fae600add30 legacy=0x7fae600ae1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T07:59:40.990 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:40.986+0000 7fae5f7fe700 1 -- 172.21.15.59:0/3269471280 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fae6015eb80 con 0x7fae6006afd0 2023-11-24T07:59:40.990 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:40.987+0000 7fae5f7fe700 1 --2- 172.21.15.59:0/3269471280 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fae6006afd0 0x7fae6006b3b0 secure :-1 s=READY pgs=973 cs=0 l=1 rev1=1 crypto rx=0x7fae6015bff0 tx=0x7fae50009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3d15809f29098e90 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:59:40.990 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:40.987+0000 7fae5e7fc700 1 -- 172.21.15.59:0/3269471280 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae50004230 con 0x7fae6006afd0 2023-11-24T07:59:40.990 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:40.987+0000 7fae5e7fc700 1 -- 172.21.15.59:0/3269471280 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fae50004390 con 0x7fae6006afd0 2023-11-24T07:59:40.990 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:40.987+0000 7fae5e7fc700 1 -- 172.21.15.59:0/3269471280 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae500036a0 con 0x7fae6006afd0 2023-11-24T07:59:40.991 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:40.987+0000 7fae65f7d700 1 -- 172.21.15.59:0/3269471280 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fae6006afd0 msgr2=0x7fae6006b3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:40.991 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:40.987+0000 7fae65f7d700 1 --2- 172.21.15.59:0/3269471280 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fae6006afd0 0x7fae6006b3b0 secure :-1 s=READY pgs=973 cs=0 l=1 rev1=1 crypto rx=0x7fae6015bff0 tx=0x7fae50009bd0 comp rx=0 tx=0).stop 2023-11-24T07:59:40.991 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:40.987+0000 7fae65f7d700 1 -- 172.21.15.59:0/3269471280 shutdown_connections 2023-11-24T07:59:40.991 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:40.987+0000 7fae65f7d700 1 --2- 172.21.15.59:0/3269471280 >> v2:172.21.15.76:3300/0 conn(0x7fae60071c10 0x7fae60072040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:40.991 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:40.987+0000 7fae65f7d700 1 --2- 172.21.15.59:0/3269471280 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fae6006afd0 0x7fae6006b3b0 unknown :-1 s=CLOSED pgs=973 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:40.991 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:40.987+0000 7fae65f7d700 1 -- 172.21.15.59:0/3269471280 >> 172.21.15.59:0/3269471280 conn(0x7fae600b2870 msgr2=0x7fae600b2c80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:59:40.991 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:40.987+0000 7fae65f7d700 1 -- 172.21.15.59:0/3269471280 shutdown_connections 2023-11-24T07:59:40.991 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:40.987+0000 7fae65f7d700 1 -- 172.21.15.59:0/3269471280 wait complete. 2023-11-24T07:59:40.994 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:40.988+0000 7fae65f7d700 1 Processor -- start 2023-11-24T07:59:40.994 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:40.989+0000 7fae65f7d700 1 -- start start 2023-11-24T07:59:40.994 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:40.989+0000 7fae65f7d700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fae6006afd0 0x7fae6015ae40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:40.994 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:40.989+0000 7fae65f7d700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fae60071c10 0x7fae60159800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:40.994 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:40.989+0000 7fae65f7d700 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fae600add30 0x7fae60159d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:40.994 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:40.989+0000 7fae65f7d700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae6015b380 con 0x7fae60071c10 2023-11-24T07:59:40.994 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:40.989+0000 7fae65f7d700 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae60153730 con 0x7fae6006afd0 2023-11-24T07:59:40.994 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:40.989+0000 7fae65f7d700 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fae60153d60 con 0x7fae600add30 2023-11-24T07:59:40.994 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:40.989+0000 7fae5effd700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fae60071c10 0x7fae60159800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:40.994 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:40.990+0000 7fae5effd700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fae60071c10 0x7fae60159800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:55524/0 (socket says 172.21.15.59:55524) 2023-11-24T07:59:40.994 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:40.990+0000 7fae5effd700 1 -- 172.21.15.59:0/1428058869 learned_addr learned my addr 172.21.15.59:0/1428058869 (peer_addr_for_me v2:172.21.15.59:0/0) 2023-11-24T07:59:40.995 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:40.990+0000 7fae5f7fe700 1 --2- 172.21.15.59:0/1428058869 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fae6006afd0 0x7fae6015ae40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:40.995 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:40.990+0000 7fae5ffff700 1 --2- 172.21.15.59:0/1428058869 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fae600add30 0x7fae60159d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:40.995 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:40.990+0000 7fae5effd700 1 -- 172.21.15.59:0/1428058869 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fae600add30 msgr2=0x7fae60159d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:40.995 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:40.990+0000 7fae5effd700 1 --2- 172.21.15.59:0/1428058869 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fae600add30 0x7fae60159d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:40.995 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:40.990+0000 7fae5effd700 1 -- 172.21.15.59:0/1428058869 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fae6006afd0 msgr2=0x7fae6015ae40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:40.995 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:40.990+0000 7fae5effd700 1 --2- 172.21.15.59:0/1428058869 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fae6006afd0 0x7fae6015ae40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:40.995 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:40.990+0000 7fae5effd700 1 -- 172.21.15.59:0/1428058869 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fae50009550 con 0x7fae60071c10 2023-11-24T07:59:40.995 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:40.990+0000 7fae5ffff700 1 --2- 172.21.15.59:0/1428058869 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fae600add30 0x7fae60159d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T07:59:40.995 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:40.990+0000 7fae5f7fe700 1 --2- 172.21.15.59:0/1428058869 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fae6006afd0 0x7fae6015ae40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T07:59:40.995 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:40.990+0000 7fae5effd700 1 --2- 172.21.15.59:0/1428058869 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fae60071c10 0x7fae60159800 secure :-1 s=READY pgs=974 cs=0 l=1 rev1=1 crypto rx=0x7fae4800b4a0 tx=0x7fae4800b860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:59:40.995 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:40.990+0000 7fae57fff700 1 -- 172.21.15.59:0/1428058869 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae48011830 con 0x7fae60071c10 2023-11-24T07:59:40.995 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:40.990+0000 7fae57fff700 1 -- 172.21.15.59:0/1428058869 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fae48011990 con 0x7fae60071c10 2023-11-24T07:59:40.995 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:40.990+0000 7fae65f7d700 1 -- 172.21.15.59:0/1428058869 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fae60154040 con 0x7fae60071c10 2023-11-24T07:59:40.996 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:40.990+0000 7fae57fff700 1 -- 172.21.15.59:0/1428058869 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae48011d40 con 0x7fae60071c10 2023-11-24T07:59:40.996 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:40.990+0000 7fae65f7d700 1 -- 172.21.15.59:0/1428058869 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fae60153a10 con 0x7fae60071c10 2023-11-24T07:59:40.996 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:40.991+0000 7fae65f7d700 1 -- 172.21.15.59:0/1428058869 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fae60068670 con 0x7fae60071c10 2023-11-24T07:59:40.996 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:40.992+0000 7fae57fff700 1 -- 172.21.15.59:0/1428058869 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 11) v1 ==== 89741+0+0 (secure 0 0 0) 0x7fae4801a400 con 0x7fae60071c10 2023-11-24T07:59:40.996 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:40.992+0000 7fae57fff700 1 --2- 172.21.15.59:0/1428058869 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7fae4006b470 0x7fae4006d930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:40.996 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:40.992+0000 7fae57fff700 1 -- 172.21.15.59:0/1428058869 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(76..76 src has 1..76) v4 ==== 5823+0+0 (secure 0 0 0) 0x7fae4808a290 con 0x7fae60071c10 2023-11-24T07:59:40.996 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:40.992+0000 7fae5f7fe700 1 --2- 172.21.15.59:0/1428058869 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7fae4006b470 0x7fae4006d930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:40.996 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:40.993+0000 7fae5f7fe700 1 --2- 172.21.15.59:0/1428058869 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7fae4006b470 0x7fae4006d930 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7fae50009c10 tx=0x7fae50008030 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:59:40.998 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:40.996+0000 7fae57fff700 1 -- 172.21.15.59:0/1428058869 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fae480549d0 con 0x7fae60071c10 2023-11-24T07:59:41.197 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:41.195+0000 7fae65f7d700 1 -- 172.21.15.59:0/1428058869 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fae600b50a0 con 0x7fae60071c10 2023-11-24T07:59:41.339 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:41.337+0000 7fae57fff700 1 -- 172.21.15.59:0/1428058869 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v77) v1 ==== 181+0+0 (secure 0 0 0) 0x7fae480150d0 con 0x7fae60071c10 2023-11-24T07:59:41.344 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:41.341+0000 7fae65f7d700 1 -- 172.21.15.59:0/1428058869 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7fae4006b470 msgr2=0x7fae4006d930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:41.344 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:41.341+0000 7fae65f7d700 1 --2- 172.21.15.59:0/1428058869 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7fae4006b470 0x7fae4006d930 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7fae50009c10 tx=0x7fae50008030 comp rx=0 tx=0).stop 2023-11-24T07:59:41.344 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:41.341+0000 7fae65f7d700 1 -- 172.21.15.59:0/1428058869 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fae60071c10 msgr2=0x7fae60159800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:41.344 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:41.341+0000 7fae65f7d700 1 --2- 172.21.15.59:0/1428058869 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fae60071c10 0x7fae60159800 secure :-1 s=READY pgs=974 cs=0 l=1 rev1=1 crypto rx=0x7fae4800b4a0 tx=0x7fae4800b860 comp rx=0 tx=0).stop 2023-11-24T07:59:41.344 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:41.342+0000 7fae65f7d700 1 -- 172.21.15.59:0/1428058869 shutdown_connections 2023-11-24T07:59:41.344 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:41.342+0000 7fae65f7d700 1 --2- 172.21.15.59:0/1428058869 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7fae4006b470 0x7fae4006d930 unknown :-1 s=CLOSED pgs=450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:41.345 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:41.342+0000 7fae65f7d700 1 --2- 172.21.15.59:0/1428058869 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fae600add30 0x7fae60159d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:41.345 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:41.342+0000 7fae65f7d700 1 --2- 172.21.15.59:0/1428058869 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fae6006afd0 0x7fae6015ae40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:41.345 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:41.342+0000 7fae65f7d700 1 --2- 172.21.15.59:0/1428058869 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fae60071c10 0x7fae60159800 unknown :-1 s=CLOSED pgs=974 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:41.345 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:41.342+0000 7fae65f7d700 1 -- 172.21.15.59:0/1428058869 >> 172.21.15.59:0/1428058869 conn(0x7fae600b2870 msgr2=0x7fae60157790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:59:41.345 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:41.343+0000 7fae65f7d700 1 -- 172.21.15.59:0/1428058869 shutdown_connections 2023-11-24T07:59:41.345 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:41.344+0000 7fae65f7d700 1 -- 172.21.15.59:0/1428058869 wait complete. 2023-11-24T07:59:41.347 INFO:teuthology.orchestra.run.smithi059.stderr:pool 'cephfs_metadata' removed 2023-11-24T07:59:41.361 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2023-11-24T07:59:41.493 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:41.491+0000 7ff67a9a4700 1 Processor -- start 2023-11-24T07:59:41.494 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:41.491+0000 7ff67a9a4700 1 -- start start 2023-11-24T07:59:41.500 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:41.492+0000 7ff67a9a4700 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7ff6740cecd0 0x7ff67415c1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:41.501 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:41.492+0000 7ff67a9a4700 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff6740d0a90 con 0x7ff67415bde0 2023-11-24T07:59:41.501 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:41.492+0000 7ff67a9a4700 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff67406ee10 con 0x7ff6740cf2b0 2023-11-24T07:59:41.501 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:41.492+0000 7ff67a9a4700 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff67406f0f0 con 0x7ff6740cecd0 2023-11-24T07:59:41.501 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:41.492+0000 7ff673fff700 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7ff67415bde0 0x7ff6740d01b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:59002/0 (socket says 172.21.15.59:59002) 2023-11-24T07:59:41.501 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:41.492+0000 7ff673fff700 1 -- 172.21.15.59:0/2572652878 learned_addr learned my addr 172.21.15.59:0/2572652878 (peer_addr_for_me v1:172.21.15.59:0/0) 2023-11-24T07:59:41.501 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:41.492+0000 7ff678f41700 1 --2- 172.21.15.59:0/2572652878 >> v2:172.21.15.76:3300/0 conn(0x7ff6740cecd0 0x7ff67415c1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:41.502 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:41.492+0000 7ff672ffd700 1 -- 172.21.15.59:0/2572652878 <== mon.0 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 593673550 0 0) 0x7ff6740d0a90 con 0x7ff67415bde0 2023-11-24T07:59:41.502 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:41.492+0000 7ff672ffd700 1 -- 172.21.15.59:0/2572652878 --> v1:172.21.15.59:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff65c0035b0 con 0x7ff67415bde0 2023-11-24T07:59:41.502 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:41.493+0000 7ff678f41700 1 -- 172.21.15.59:0/2572652878 >> v1:172.21.15.76:6789/0 conn(0x7ff6740cf2b0 legacy=0x7ff6740ce5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:41.502 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:41.493+0000 7ff672ffd700 1 -- 172.21.15.59:0/2572652878 <== mon.1 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2420635223 0 0) 0x7ff67406ee10 con 0x7ff6740cf2b0 2023-11-24T07:59:41.502 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:41.493+0000 7ff678f41700 1 -- 172.21.15.59:0/2572652878 >> v1:172.21.15.59:6789/0 conn(0x7ff67415bde0 legacy=0x7ff6740d01b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:41.503 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:41.493+0000 7ff678f41700 1 -- 172.21.15.59:0/2572652878 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff67406f3d0 con 0x7ff6740cecd0 2023-11-24T07:59:41.503 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:41.493+0000 7ff678f41700 1 --2- 172.21.15.59:0/2572652878 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff6740cecd0 0x7ff67415c1c0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7ff66c009d90 tx=0x7ff66c00b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a4b751af6bccdfbb server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:59:41.504 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:41.493+0000 7ff672ffd700 1 -- 172.21.15.59:0/2572652878 <== mon.1 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff66c00e070 con 0x7ff6740cecd0 2023-11-24T07:59:41.504 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:41.493+0000 7ff672ffd700 1 -- 172.21.15.59:0/2572652878 <== mon.1 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff66c003a20 con 0x7ff6740cecd0 2023-11-24T07:59:41.504 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:41.493+0000 7ff672ffd700 1 -- 172.21.15.59:0/2572652878 <== mon.1 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff66c003db0 con 0x7ff6740cecd0 2023-11-24T07:59:41.504 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:41.493+0000 7ff67a9a4700 1 -- 172.21.15.59:0/2572652878 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff6740cecd0 msgr2=0x7ff67415c1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:41.504 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:41.493+0000 7ff67a9a4700 1 --2- 172.21.15.59:0/2572652878 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff6740cecd0 0x7ff67415c1c0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7ff66c009d90 tx=0x7ff66c00b330 comp rx=0 tx=0).stop 2023-11-24T07:59:41.504 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:41.494+0000 7ff67a9a4700 1 -- 172.21.15.59:0/2572652878 shutdown_connections 2023-11-24T07:59:41.504 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:41.494+0000 7ff67a9a4700 1 --2- 172.21.15.59:0/2572652878 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff6740cecd0 0x7ff67415c1c0 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:41.505 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:41.494+0000 7ff67a9a4700 1 -- 172.21.15.59:0/2572652878 >> 172.21.15.59:0/2572652878 conn(0x7ff6740b3720 msgr2=0x7ff6740b3b30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:59:41.505 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:41.494+0000 7ff67a9a4700 1 -- 172.21.15.59:0/2572652878 shutdown_connections 2023-11-24T07:59:41.505 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:41.494+0000 7ff67a9a4700 1 -- 172.21.15.59:0/2572652878 wait complete. 2023-11-24T07:59:41.505 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:41.495+0000 7ff67a9a4700 1 Processor -- start 2023-11-24T07:59:41.505 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:41.495+0000 7ff67a9a4700 1 -- start start 2023-11-24T07:59:41.505 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:41.496+0000 7ff67a9a4700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff6740cecd0 0x7ff67414c9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:41.505 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:41.496+0000 7ff67a9a4700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff6740cf2b0 0x7ff6740b4810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:41.505 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:41.496+0000 7ff67a9a4700 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7ff67415bde0 0x7ff6740b1ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:41.506 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:41.496+0000 7ff67a9a4700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6740b3440 con 0x7ff6740cecd0 2023-11-24T07:59:41.506 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:41.496+0000 7ff67a9a4700 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6740b2830 con 0x7ff6740cf2b0 2023-11-24T07:59:41.506 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:41.496+0000 7ff67a9a4700 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff6740b2b10 con 0x7ff67415bde0 2023-11-24T07:59:41.506 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:41.496+0000 7ff673fff700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff6740cecd0 0x7ff67414c9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:41.506 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:41.496+0000 7ff673fff700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff6740cecd0 0x7ff67414c9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:55534/0 (socket says 172.21.15.59:55534) 2023-11-24T07:59:41.506 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:41.496+0000 7ff673fff700 1 -- 172.21.15.59:0/999677190 learned_addr learned my addr 172.21.15.59:0/999677190 (peer_addr_for_me v2:172.21.15.59:0/0) 2023-11-24T07:59:41.506 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:41.496+0000 7ff6737fe700 1 --2- 172.21.15.59:0/999677190 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff6740cf2b0 0x7ff6740b4810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:41.506 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:41.496+0000 7ff678f41700 1 --2- 172.21.15.59:0/999677190 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7ff67415bde0 0x7ff6740b1ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:41.507 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:41.496+0000 7ff673fff700 1 -- 172.21.15.59:0/999677190 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7ff67415bde0 msgr2=0x7ff6740b1ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:41.507 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:41.496+0000 7ff673fff700 1 --2- 172.21.15.59:0/999677190 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7ff67415bde0 0x7ff6740b1ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:41.507 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:41.496+0000 7ff673fff700 1 -- 172.21.15.59:0/999677190 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff6740cf2b0 msgr2=0x7ff6740b4810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:41.507 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:41.496+0000 7ff673fff700 1 --2- 172.21.15.59:0/999677190 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff6740cf2b0 0x7ff6740b4810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:41.507 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:41.496+0000 7ff673fff700 1 -- 172.21.15.59:0/999677190 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff66c009a70 con 0x7ff6740cecd0 2023-11-24T07:59:41.507 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:41.496+0000 7ff6737fe700 1 --2- 172.21.15.59:0/999677190 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff6740cf2b0 0x7ff6740b4810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-24T07:59:41.507 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:41.496+0000 7ff678f41700 1 --2- 172.21.15.59:0/999677190 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7ff67415bde0 0x7ff6740b1ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-24T07:59:41.507 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:41.496+0000 7ff673fff700 1 --2- 172.21.15.59:0/999677190 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff6740cecd0 0x7ff67414c9a0 secure :-1 s=READY pgs=976 cs=0 l=1 rev1=1 crypto rx=0x7ff66800c780 tx=0x7ff66800cb40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:59:41.507 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:41.496+0000 7ff670ff9700 1 -- 172.21.15.59:0/999677190 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff668005e30 con 0x7ff6740cecd0 2023-11-24T07:59:41.507 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:41.496+0000 7ff670ff9700 1 -- 172.21.15.59:0/999677190 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff668007070 con 0x7ff6740cecd0 2023-11-24T07:59:41.507 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:41.496+0000 7ff67a9a4700 1 -- 172.21.15.59:0/999677190 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff6740b2df0 con 0x7ff6740cecd0 2023-11-24T07:59:41.507 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:41.496+0000 7ff670ff9700 1 -- 172.21.15.59:0/999677190 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff668007400 con 0x7ff6740cecd0 2023-11-24T07:59:41.507 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:41.496+0000 7ff67a9a4700 1 -- 172.21.15.59:0/999677190 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff67414d410 con 0x7ff6740cecd0 2023-11-24T07:59:41.508 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:41.497+0000 7ff67a9a4700 1 -- 172.21.15.59:0/999677190 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff674068670 con 0x7ff6740cecd0 2023-11-24T07:59:41.508 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:41.498+0000 7ff670ff9700 1 -- 172.21.15.59:0/999677190 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 11) v1 ==== 89741+0+0 (secure 0 0 0) 0x7ff668007560 con 0x7ff6740cecd0 2023-11-24T07:59:41.508 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:41.498+0000 7ff670ff9700 1 --2- 172.21.15.59:0/999677190 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7ff66006b470 0x7ff66006d930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:41.508 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:41.499+0000 7ff670ff9700 1 -- 172.21.15.59:0/999677190 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(77..77 src has 1..77) v4 ==== 5385+0+0 (secure 0 0 0) 0x7ff668089480 con 0x7ff6740cecd0 2023-11-24T07:59:41.508 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:41.499+0000 7ff6737fe700 1 --2- 172.21.15.59:0/999677190 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7ff66006b470 0x7ff66006d930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:41.509 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:41.499+0000 7ff6737fe700 1 --2- 172.21.15.59:0/999677190 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7ff66006b470 0x7ff66006d930 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7ff664007fd0 tx=0x7ff664009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:59:41.509 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:41.503+0000 7ff670ff9700 1 -- 172.21.15.59:0/999677190 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7ff668053f10 con 0x7ff6740cecd0 2023-11-24T07:59:41.708 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:41.707+0000 7ff67a9a4700 1 -- 172.21.15.59:0/999677190 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7ff6740af640 con 0x7ff6740cecd0 2023-11-24T07:59:42.451 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:42.449+0000 7ff670ff9700 1 -- 172.21.15.59:0/999677190 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v78) v1 ==== 169+0+0 (secure 0 0 0) 0x7ff668057530 con 0x7ff6740cecd0 2023-11-24T07:59:42.454 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:42.452+0000 7ff67a9a4700 1 -- 172.21.15.59:0/999677190 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7ff66006b470 msgr2=0x7ff66006d930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:42.454 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:42.452+0000 7ff67a9a4700 1 --2- 172.21.15.59:0/999677190 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7ff66006b470 0x7ff66006d930 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7ff664007fd0 tx=0x7ff664009040 comp rx=0 tx=0).stop 2023-11-24T07:59:42.454 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:42.452+0000 7ff67a9a4700 1 -- 172.21.15.59:0/999677190 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff6740cecd0 msgr2=0x7ff67414c9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:42.454 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:42.452+0000 7ff67a9a4700 1 --2- 172.21.15.59:0/999677190 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff6740cecd0 0x7ff67414c9a0 secure :-1 s=READY pgs=976 cs=0 l=1 rev1=1 crypto rx=0x7ff66800c780 tx=0x7ff66800cb40 comp rx=0 tx=0).stop 2023-11-24T07:59:42.454 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:42.452+0000 7ff67a9a4700 1 -- 172.21.15.59:0/999677190 shutdown_connections 2023-11-24T07:59:42.454 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:42.452+0000 7ff67a9a4700 1 --2- 172.21.15.59:0/999677190 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7ff66006b470 0x7ff66006d930 unknown :-1 s=CLOSED pgs=451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:42.454 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:42.452+0000 7ff67a9a4700 1 --2- 172.21.15.59:0/999677190 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7ff67415bde0 0x7ff6740b1ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:42.454 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:42.452+0000 7ff67a9a4700 1 --2- 172.21.15.59:0/999677190 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff6740cecd0 0x7ff67414c9a0 unknown :-1 s=CLOSED pgs=976 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:42.454 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:42.452+0000 7ff67a9a4700 1 --2- 172.21.15.59:0/999677190 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff6740cf2b0 0x7ff6740b4810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:42.455 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:42.452+0000 7ff67a9a4700 1 -- 172.21.15.59:0/999677190 >> 172.21.15.59:0/999677190 conn(0x7ff6740b3720 msgr2=0x7ff67415b090 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:59:42.455 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:42.452+0000 7ff67a9a4700 1 -- 172.21.15.59:0/999677190 shutdown_connections 2023-11-24T07:59:42.455 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:42.452+0000 7ff67a9a4700 1 -- 172.21.15.59:0/999677190 wait complete. 2023-11-24T07:59:42.455 INFO:teuthology.orchestra.run.smithi059.stderr:pool 'cephfs_data' removed 2023-11-24T07:59:42.472 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2023-11-24T07:59:42.606 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:42.604+0000 7f4d42a28700 1 Processor -- start 2023-11-24T07:59:42.606 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:42.604+0000 7f4d42a28700 1 -- start start 2023-11-24T07:59:42.610 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:42.605+0000 7f4d42a28700 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f4d3c072a00 0x7f4d3c06db60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:42.610 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:42.605+0000 7f4d42a28700 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f4d3c06e0a0 0x7f4d3c0d1010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:42.610 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:42.605+0000 7f4d42a28700 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4d3c0d0b80 con 0x7f4d3c0b4090 2023-11-24T07:59:42.610 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:42.605+0000 7f4d42a28700 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4d3c15ed60 con 0x7f4d3c06e0a0 2023-11-24T07:59:42.610 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:42.605+0000 7f4d42a28700 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4d3c15f040 con 0x7f4d3c072a00 2023-11-24T07:59:42.610 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:42.605+0000 7f4d3bfff700 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7f4d3c0b4090 0x7f4d3c15e510 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:59016/0 (socket says 172.21.15.59:59016) 2023-11-24T07:59:42.610 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:42.605+0000 7f4d3bfff700 1 -- 172.21.15.59:0/2104786699 learned_addr learned my addr 172.21.15.59:0/2104786699 (peer_addr_for_me v1:172.21.15.59:0/0) 2023-11-24T07:59:42.610 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:42.606+0000 7f4d3affd700 1 -- 172.21.15.59:0/2104786699 <== mon.0 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2803778145 0 0) 0x7f4d3c0d0b80 con 0x7f4d3c0b4090 2023-11-24T07:59:42.611 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:42.606+0000 7f4d3affd700 1 -- 172.21.15.59:0/2104786699 --> v1:172.21.15.59:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4d280035b0 con 0x7f4d3c0b4090 2023-11-24T07:59:42.611 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:42.606+0000 7f4d3b7fe700 1 --2- 172.21.15.59:0/2104786699 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f4d3c072a00 0x7f4d3c06db60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:42.611 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:42.606+0000 7f4d40fc5700 1 --2- 172.21.15.59:0/2104786699 >> v2:172.21.15.76:3300/0 conn(0x7f4d3c06e0a0 0x7f4d3c0d1010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:42.611 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:42.606+0000 7f4d3affd700 1 -- 172.21.15.59:0/2104786699 <== mon.0 v1:172.21.15.59:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4108258649 0 0) 0x7f4d280035b0 con 0x7f4d3c0b4090 2023-11-24T07:59:42.611 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:42.606+0000 7f4d3affd700 1 -- 172.21.15.59:0/2104786699 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f4d3c072a00 msgr2=0x7f4d3c06db60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T07:59:42.611 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:42.606+0000 7f4d3affd700 1 --2- 172.21.15.59:0/2104786699 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f4d3c072a00 0x7f4d3c06db60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:42.611 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:42.606+0000 7f4d3affd700 1 -- 172.21.15.59:0/2104786699 >> v2:172.21.15.76:3300/0 conn(0x7f4d3c06e0a0 msgr2=0x7f4d3c0d1010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T07:59:42.611 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:42.606+0000 7f4d3affd700 1 --2- 172.21.15.59:0/2104786699 >> v2:172.21.15.76:3300/0 conn(0x7f4d3c06e0a0 0x7f4d3c0d1010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:42.611 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:42.606+0000 7f4d3affd700 1 -- 172.21.15.59:0/2104786699 --> v1:172.21.15.59:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4d3c15f320 con 0x7f4d3c0b4090 2023-11-24T07:59:42.611 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:42.606+0000 7f4d3affd700 1 -- 172.21.15.59:0/2104786699 <== mon.0 v1:172.21.15.59:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1312333566 0 0) 0x7f4d2c001670 con 0x7f4d3c0b4090 2023-11-24T07:59:42.611 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:42.606+0000 7f4d3affd700 1 -- 172.21.15.59:0/2104786699 >> v1:172.21.15.59:6789/0 conn(0x7f4d3c0b4090 legacy=0x7f4d3c15e510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:42.611 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:42.606+0000 7f4d3affd700 1 --2- 172.21.15.59:0/2104786699 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4d28003a00 0x7f4d28003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:42.611 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:42.606+0000 7f4d3affd700 1 --2- 172.21.15.59:0/2104786699 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4d28004850 0x7f4d28004cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:42.612 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:42.606+0000 7f4d3affd700 1 --2- 172.21.15.59:0/2104786699 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f4d3c072a00 0x7f4d28005210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:42.612 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:42.606+0000 7f4d3affd700 1 -- 172.21.15.59:0/2104786699 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d3c15ed60 con 0x7f4d28003a00 2023-11-24T07:59:42.612 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:42.606+0000 7f4d3affd700 1 -- 172.21.15.59:0/2104786699 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d3c15f040 con 0x7f4d28004850 2023-11-24T07:59:42.612 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:42.606+0000 7f4d3affd700 1 -- 172.21.15.59:0/2104786699 --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4d2800baf0 con 0x7f4d3c072a00 2023-11-24T07:59:42.612 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:42.606+0000 7f4d3bfff700 1 --2- 172.21.15.59:0/2104786699 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4d28003a00 0x7f4d28003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:42.612 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:42.606+0000 7f4d3b7fe700 1 --2- 172.21.15.59:0/2104786699 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4d28004850 0x7f4d28004cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:42.612 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:42.606+0000 7f4d40fc5700 1 --2- 172.21.15.59:0/2104786699 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f4d3c072a00 0x7f4d28005210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:42.612 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:42.607+0000 7f4d3bfff700 1 -- 172.21.15.59:0/2104786699 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f4d3c072a00 msgr2=0x7f4d28005210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T07:59:42.612 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:42.607+0000 7f4d3bfff700 1 --2- 172.21.15.59:0/2104786699 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f4d3c072a00 0x7f4d28005210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:42.612 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:42.607+0000 7f4d3bfff700 1 -- 172.21.15.59:0/2104786699 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4d28004850 msgr2=0x7f4d28004cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T07:59:42.612 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:42.607+0000 7f4d3bfff700 1 --2- 172.21.15.59:0/2104786699 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4d28004850 0x7f4d28004cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:42.612 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:42.607+0000 7f4d3bfff700 1 -- 172.21.15.59:0/2104786699 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4d280035b0 con 0x7f4d28003a00 2023-11-24T07:59:42.612 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:42.607+0000 7f4d40fc5700 1 --2- 172.21.15.59:0/2104786699 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f4d3c072a00 0x7f4d28005210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-24T07:59:42.613 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:42.607+0000 7f4d3bfff700 1 --2- 172.21.15.59:0/2104786699 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4d28003a00 0x7f4d28003e20 secure :-1 s=READY pgs=978 cs=0 l=1 rev1=1 crypto rx=0x7f4d2c001050 tx=0x7f4d28005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d3ff7b3b03ec3a00 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:59:42.613 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:42.607+0000 7f4d3affd700 1 -- 172.21.15.59:0/2104786699 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d2c007c40 con 0x7f4d28003a00 2023-11-24T07:59:42.613 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:42.607+0000 7f4d3affd700 1 -- 172.21.15.59:0/2104786699 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4d2c00cd40 con 0x7f4d28003a00 2023-11-24T07:59:42.613 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:42.607+0000 7f4d3affd700 1 -- 172.21.15.59:0/2104786699 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d2c005ba0 con 0x7f4d28003a00 2023-11-24T07:59:42.613 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:42.607+0000 7f4d42a28700 1 -- 172.21.15.59:0/2104786699 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4d28003a00 msgr2=0x7f4d28003e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:42.613 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:42.607+0000 7f4d42a28700 1 --2- 172.21.15.59:0/2104786699 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4d28003a00 0x7f4d28003e20 secure :-1 s=READY pgs=978 cs=0 l=1 rev1=1 crypto rx=0x7f4d2c001050 tx=0x7f4d28005880 comp rx=0 tx=0).stop 2023-11-24T07:59:42.613 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:42.607+0000 7f4d40fc5700 1 -- 172.21.15.59:0/2104786699 reap_dead start 2023-11-24T07:59:42.613 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:42.607+0000 7f4d42a28700 1 -- 172.21.15.59:0/2104786699 shutdown_connections 2023-11-24T07:59:42.613 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:42.607+0000 7f4d42a28700 1 -- 172.21.15.59:0/2104786699 >> 172.21.15.59:0/2104786699 conn(0x7f4d3c0b2740 msgr2=0x7f4d3c0b2b50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:59:42.613 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:42.607+0000 7f4d42a28700 1 -- 172.21.15.59:0/2104786699 shutdown_connections 2023-11-24T07:59:42.613 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:42.607+0000 7f4d42a28700 1 -- 172.21.15.59:0/2104786699 wait complete. 2023-11-24T07:59:42.613 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:42.609+0000 7f4d42a28700 1 Processor -- start 2023-11-24T07:59:42.616 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:42.609+0000 7f4d42a28700 1 -- start start 2023-11-24T07:59:42.616 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:42.610+0000 7f4d42a28700 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f4d28003a00 0x7f4d3c1474d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:42.616 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:42.610+0000 7f4d42a28700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4d3c15ab80 0x7f4d3c1f3d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:42.616 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:42.610+0000 7f4d42a28700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4d3c158b10 0x7f4d3c158f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:42.616 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:42.610+0000 7f4d42a28700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d3c147a10 con 0x7f4d3c15ab80 2023-11-24T07:59:42.616 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:42.610+0000 7f4d42a28700 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d3c1594d0 con 0x7f4d3c158b10 2023-11-24T07:59:42.616 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:42.610+0000 7f4d42a28700 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4d3c1567a0 con 0x7f4d28003a00 2023-11-24T07:59:42.616 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:42.610+0000 7f4d3b7fe700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4d3c15ab80 0x7f4d3c1f3d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:42.616 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:42.610+0000 7f4d3b7fe700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4d3c15ab80 0x7f4d3c1f3d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:55556/0 (socket says 172.21.15.59:55556) 2023-11-24T07:59:42.617 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:42.610+0000 7f4d3b7fe700 1 -- 172.21.15.59:0/1251767050 learned_addr learned my addr 172.21.15.59:0/1251767050 (peer_addr_for_me v2:172.21.15.59:0/0) 2023-11-24T07:59:42.617 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:42.610+0000 7f4d3bfff700 1 --2- 172.21.15.59:0/1251767050 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f4d28003a00 0x7f4d3c1474d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:42.617 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:42.610+0000 7f4d40fc5700 1 --2- 172.21.15.59:0/1251767050 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4d3c158b10 0x7f4d3c158f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:42.617 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:42.610+0000 7f4d3b7fe700 1 -- 172.21.15.59:0/1251767050 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f4d28003a00 msgr2=0x7f4d3c1474d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:42.617 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:42.610+0000 7f4d3b7fe700 1 --2- 172.21.15.59:0/1251767050 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f4d28003a00 0x7f4d3c1474d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:42.617 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:42.610+0000 7f4d3b7fe700 1 -- 172.21.15.59:0/1251767050 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4d3c158b10 msgr2=0x7f4d3c158f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:42.617 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:42.610+0000 7f4d3b7fe700 1 --2- 172.21.15.59:0/1251767050 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4d3c158b10 0x7f4d3c158f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:42.617 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:42.610+0000 7f4d3b7fe700 1 -- 172.21.15.59:0/1251767050 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4d3c155b90 con 0x7f4d3c15ab80 2023-11-24T07:59:42.617 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:42.610+0000 7f4d3b7fe700 1 --2- 172.21.15.59:0/1251767050 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4d3c15ab80 0x7f4d3c1f3d30 secure :-1 s=READY pgs=979 cs=0 l=1 rev1=1 crypto rx=0x7f4d24011aa0 tx=0x7f4d24011e60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:59:42.617 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:42.610+0000 7f4d38ff9700 1 -- 172.21.15.59:0/1251767050 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d2400f980 con 0x7f4d3c15ab80 2023-11-24T07:59:42.617 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:42.610+0000 7f4d38ff9700 1 -- 172.21.15.59:0/1251767050 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4d2400fae0 con 0x7f4d3c15ab80 2023-11-24T07:59:42.617 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:42.610+0000 7f4d42a28700 1 -- 172.21.15.59:0/1251767050 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4d3c155e70 con 0x7f4d3c15ab80 2023-11-24T07:59:42.617 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:42.610+0000 7f4d38ff9700 1 -- 172.21.15.59:0/1251767050 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d2400fe90 con 0x7f4d3c15ab80 2023-11-24T07:59:42.617 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:42.610+0000 7f4d42a28700 1 -- 172.21.15.59:0/1251767050 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4d3c156430 con 0x7f4d3c15ab80 2023-11-24T07:59:42.618 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:42.611+0000 7f4d42a28700 1 -- 172.21.15.59:0/1251767050 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4d3c075480 con 0x7f4d3c15ab80 2023-11-24T07:59:42.618 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:42.612+0000 7f4d38ff9700 1 -- 172.21.15.59:0/1251767050 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 11) v1 ==== 89741+0+0 (secure 0 0 0) 0x7f4d24014440 con 0x7f4d3c15ab80 2023-11-24T07:59:42.618 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:42.612+0000 7f4d38ff9700 1 --2- 172.21.15.59:0/1251767050 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f4d1c06b470 0x7f4d1c06d930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:42.618 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:42.612+0000 7f4d38ff9700 1 -- 172.21.15.59:0/1251767050 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(78..78 src has 1..78) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f4d2408a610 con 0x7f4d3c15ab80 2023-11-24T07:59:42.618 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:42.613+0000 7f4d3bfff700 1 --2- 172.21.15.59:0/1251767050 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f4d1c06b470 0x7f4d1c06d930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:42.618 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:42.613+0000 7f4d3bfff700 1 --2- 172.21.15.59:0/1251767050 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f4d1c06b470 0x7f4d1c06d930 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f4d2c008fd0 tx=0x7f4d2c00a000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:59:42.618 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:42.617+0000 7f4d38ff9700 1 -- 172.21.15.59:0/1251767050 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f4d24055180 con 0x7f4d3c15ab80 2023-11-24T07:59:42.823 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:42.822+0000 7f4d42a28700 1 -- 172.21.15.59:0/1251767050 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7f4d3c072650 con 0x7f4d3c15ab80 2023-11-24T07:59:42.826 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:42.825+0000 7f4d38ff9700 1 -- 172.21.15.59:0/1251767050 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v17) v1 ==== 104+0+0 (secure 0 0 0) 0x7f4d240587a0 con 0x7f4d3c15ab80 2023-11-24T07:59:42.829 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:42.827+0000 7f4d42a28700 1 -- 172.21.15.59:0/1251767050 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f4d1c06b470 msgr2=0x7f4d1c06d930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:42.829 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:42.827+0000 7f4d42a28700 1 --2- 172.21.15.59:0/1251767050 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f4d1c06b470 0x7f4d1c06d930 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f4d2c008fd0 tx=0x7f4d2c00a000 comp rx=0 tx=0).stop 2023-11-24T07:59:42.829 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:42.827+0000 7f4d42a28700 1 -- 172.21.15.59:0/1251767050 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4d3c15ab80 msgr2=0x7f4d3c1f3d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:42.829 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:42.827+0000 7f4d42a28700 1 --2- 172.21.15.59:0/1251767050 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4d3c15ab80 0x7f4d3c1f3d30 secure :-1 s=READY pgs=979 cs=0 l=1 rev1=1 crypto rx=0x7f4d24011aa0 tx=0x7f4d24011e60 comp rx=0 tx=0).stop 2023-11-24T07:59:42.829 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:42.827+0000 7f4d42a28700 1 -- 172.21.15.59:0/1251767050 shutdown_connections 2023-11-24T07:59:42.829 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:42.827+0000 7f4d42a28700 1 --2- 172.21.15.59:0/1251767050 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f4d1c06b470 0x7f4d1c06d930 unknown :-1 s=CLOSED pgs=452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:42.829 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:42.827+0000 7f4d42a28700 1 --2- 172.21.15.59:0/1251767050 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4d3c158b10 0x7f4d3c158f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:42.830 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:42.827+0000 7f4d42a28700 1 --2- 172.21.15.59:0/1251767050 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f4d28003a00 0x7f4d3c1474d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:42.830 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:42.827+0000 7f4d42a28700 1 --2- 172.21.15.59:0/1251767050 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4d3c15ab80 0x7f4d3c1f3d30 unknown :-1 s=CLOSED pgs=979 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:42.830 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:42.827+0000 7f4d42a28700 1 -- 172.21.15.59:0/1251767050 >> 172.21.15.59:0/1251767050 conn(0x7f4d3c0b2740 msgr2=0x7f4d3c0b2b50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:59:42.830 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:42.827+0000 7f4d42a28700 1 -- 172.21.15.59:0/1251767050 shutdown_connections 2023-11-24T07:59:42.830 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:42.827+0000 7f4d42a28700 1 -- 172.21.15.59:0/1251767050 wait complete. 2023-11-24T07:59:42.841 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_volumes.TestSubvolumeSnapshots.test_subvolume_inherited_snapshot_info' 2023-11-24T07:59:42.966 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:42.964+0000 7f86a836c700 1 Processor -- start 2023-11-24T07:59:42.966 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:42.964+0000 7f86a836c700 1 -- start start 2023-11-24T07:59:42.968 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:42.965+0000 7f86a836c700 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f86a00722c0 0x7f86a00b51b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:42.968 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:42.965+0000 7f86a836c700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f86a01440f0 0x7f86a006e510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:42.969 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:42.965+0000 7f86a836c700 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f86a0143ec0 con 0x7f86a0143ae0 2023-11-24T07:59:42.969 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:42.965+0000 7f86a836c700 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f86a00b40a0 con 0x7f86a01440f0 2023-11-24T07:59:42.969 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:42.965+0000 7f86a836c700 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f86a00b3ae0 con 0x7f86a00722c0 2023-11-24T07:59:42.969 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:42.965+0000 7f86a6909700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f86a01440f0 0x7f86a006e510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:42.969 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:42.965+0000 7f86a6108700 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7f86a0143ae0 0x7f86a0071bb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:59036/0 (socket says 172.21.15.59:59036) 2023-11-24T07:59:42.969 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:42.965+0000 7f86a6909700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f86a01440f0 0x7f86a006e510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:55566/0 (socket says 172.21.15.59:55566) 2023-11-24T07:59:42.969 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:42.965+0000 7f86a6108700 1 -- 172.21.15.59:0/1956643995 learned_addr learned my addr 172.21.15.59:0/1956643995 (peer_addr_for_me v1:172.21.15.59:0/0) 2023-11-24T07:59:42.969 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:42.966+0000 7f86a5907700 1 --2- 172.21.15.59:0/1956643995 >> v2:172.21.15.76:3300/0 conn(0x7f86a00722c0 0x7f86a00b51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:42.969 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:42.966+0000 7f86a6909700 1 -- 172.21.15.59:0/1956643995 >> v2:172.21.15.76:3300/0 conn(0x7f86a00722c0 msgr2=0x7f86a00b51b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T07:59:42.969 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:42.966+0000 7f86a6909700 1 --2- 172.21.15.59:0/1956643995 >> v2:172.21.15.76:3300/0 conn(0x7f86a00722c0 0x7f86a00b51b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:42.969 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:42.966+0000 7f86a6909700 1 -- 172.21.15.59:0/1956643995 >> v1:172.21.15.59:6789/0 conn(0x7f86a0143ae0 legacy=0x7f86a0071bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:42.969 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:42.966+0000 7f86a5106700 1 -- 172.21.15.59:0/1956643995 <== mon.0 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 874024328 0 0) 0x7f86a0143ec0 con 0x7f86a0143ae0 2023-11-24T07:59:42.969 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:42.966+0000 7f86a6909700 1 -- 172.21.15.59:0/1956643995 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f86a00b3dc0 con 0x7f86a01440f0 2023-11-24T07:59:42.970 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:42.966+0000 7f86a5907700 1 --2- 172.21.15.59:0/1956643995 >> v2:172.21.15.76:3300/0 conn(0x7f86a00722c0 0x7f86a00b51b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T07:59:42.970 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:42.966+0000 7f86a6909700 1 --2- 172.21.15.59:0/1956643995 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f86a01440f0 0x7f86a006e510 secure :-1 s=READY pgs=981 cs=0 l=1 rev1=1 crypto rx=0x7f86a00d1b00 tx=0x7f869800d330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6c8b9d91d1fef9ca server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:59:42.970 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:42.966+0000 7f86a5106700 1 -- 172.21.15.59:0/1956643995 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8698010070 con 0x7f86a01440f0 2023-11-24T07:59:42.970 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:42.966+0000 7f86a5106700 1 -- 172.21.15.59:0/1956643995 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f869800b2e0 con 0x7f86a01440f0 2023-11-24T07:59:42.970 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:42.966+0000 7f86a5106700 1 -- 172.21.15.59:0/1956643995 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f869800dcc0 con 0x7f86a01440f0 2023-11-24T07:59:42.970 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:42.967+0000 7f86a836c700 1 -- 172.21.15.59:0/1956643995 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f86a01440f0 msgr2=0x7f86a006e510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:42.970 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:42.967+0000 7f86a836c700 1 --2- 172.21.15.59:0/1956643995 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f86a01440f0 0x7f86a006e510 secure :-1 s=READY pgs=981 cs=0 l=1 rev1=1 crypto rx=0x7f86a00d1b00 tx=0x7f869800d330 comp rx=0 tx=0).stop 2023-11-24T07:59:42.970 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:42.967+0000 7f86a836c700 1 -- 172.21.15.59:0/1956643995 shutdown_connections 2023-11-24T07:59:42.970 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:42.967+0000 7f86a836c700 1 --2- 172.21.15.59:0/1956643995 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f86a01440f0 0x7f86a006e510 unknown :-1 s=CLOSED pgs=981 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:42.971 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:42.967+0000 7f86a836c700 1 --2- 172.21.15.59:0/1956643995 >> v2:172.21.15.76:3300/0 conn(0x7f86a00722c0 0x7f86a00b51b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:42.971 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:42.967+0000 7f86a836c700 1 -- 172.21.15.59:0/1956643995 >> 172.21.15.59:0/1956643995 conn(0x7f86a00b2740 msgr2=0x7f86a00b2b20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:59:42.971 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:42.967+0000 7f86a836c700 1 -- 172.21.15.59:0/1956643995 shutdown_connections 2023-11-24T07:59:42.971 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:42.967+0000 7f86a836c700 1 -- 172.21.15.59:0/1956643995 wait complete. 2023-11-24T07:59:42.974 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:42.968+0000 7f86a836c700 1 Processor -- start 2023-11-24T07:59:42.974 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:42.968+0000 7f86a836c700 1 -- start start 2023-11-24T07:59:42.974 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:42.969+0000 7f86a836c700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f86a00722c0 0x7f86a01f4920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:42.974 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:42.969+0000 7f86a836c700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f86a0143ae0 0x7f86a013f550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:42.974 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:42.969+0000 7f86a836c700 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f86a01440f0 0x7f86a00ce880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:42.974 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:42.969+0000 7f86a836c700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86a013fa90 con 0x7f86a0143ae0 2023-11-24T07:59:42.974 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:42.969+0000 7f86a836c700 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86a00cb070 con 0x7f86a00722c0 2023-11-24T07:59:42.974 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:42.969+0000 7f86a836c700 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f86a00cb350 con 0x7f86a01440f0 2023-11-24T07:59:42.975 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:42.969+0000 7f86a5907700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f86a0143ae0 0x7f86a013f550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:42.975 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:42.969+0000 7f86a5907700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f86a0143ae0 0x7f86a013f550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:55570/0 (socket says 172.21.15.59:55570) 2023-11-24T07:59:42.975 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:42.969+0000 7f86a5907700 1 -- 172.21.15.59:0/1566202890 learned_addr learned my addr 172.21.15.59:0/1566202890 (peer_addr_for_me v2:172.21.15.59:0/0) 2023-11-24T07:59:42.975 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:42.969+0000 7f86a6108700 1 --2- 172.21.15.59:0/1566202890 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f86a00722c0 0x7f86a01f4920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:42.975 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:42.969+0000 7f86a5907700 1 -- 172.21.15.59:0/1566202890 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f86a01440f0 msgr2=0x7f86a00ce880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:42.976 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:42.969+0000 7f86a6909700 1 --2- 172.21.15.59:0/1566202890 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f86a01440f0 0x7f86a00ce880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:42.976 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:42.969+0000 7f86a5907700 1 --2- 172.21.15.59:0/1566202890 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f86a01440f0 0x7f86a00ce880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:42.976 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:42.969+0000 7f86a5907700 1 -- 172.21.15.59:0/1566202890 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f86a00722c0 msgr2=0x7f86a01f4920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:42.976 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:42.969+0000 7f86a5907700 1 --2- 172.21.15.59:0/1566202890 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f86a00722c0 0x7f86a01f4920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:42.976 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:42.969+0000 7f86a5907700 1 -- 172.21.15.59:0/1566202890 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f869800bc80 con 0x7f86a0143ae0 2023-11-24T07:59:42.976 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:42.969+0000 7f86a6108700 1 --2- 172.21.15.59:0/1566202890 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f86a00722c0 0x7f86a01f4920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T07:59:42.976 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:42.969+0000 7f86a5907700 1 --2- 172.21.15.59:0/1566202890 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f86a0143ae0 0x7f86a013f550 secure :-1 s=READY pgs=982 cs=0 l=1 rev1=1 crypto rx=0x7f869c00c750 tx=0x7f869c00cb10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:59:42.977 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:42.969+0000 7f8692ffd700 1 -- 172.21.15.59:0/1566202890 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f869c011800 con 0x7f86a0143ae0 2023-11-24T07:59:42.977 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:42.970+0000 7f8692ffd700 1 -- 172.21.15.59:0/1566202890 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f869c011960 con 0x7f86a0143ae0 2023-11-24T07:59:42.977 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:42.970+0000 7f86a836c700 1 -- 172.21.15.59:0/1566202890 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f86a00cb630 con 0x7f86a0143ae0 2023-11-24T07:59:42.977 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:42.970+0000 7f8692ffd700 1 -- 172.21.15.59:0/1566202890 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f869c011cd0 con 0x7f86a0143ae0 2023-11-24T07:59:42.977 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:42.970+0000 7f86a836c700 1 -- 172.21.15.59:0/1566202890 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f86a015b860 con 0x7f86a0143ae0 2023-11-24T07:59:42.977 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:42.970+0000 7f86a836c700 1 -- 172.21.15.59:0/1566202890 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f86a0068670 con 0x7f86a0143ae0 2023-11-24T07:59:42.977 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:42.971+0000 7f8692ffd700 1 -- 172.21.15.59:0/1566202890 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 11) v1 ==== 89741+0+0 (secure 0 0 0) 0x7f869c021350 con 0x7f86a0143ae0 2023-11-24T07:59:42.978 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:42.971+0000 7f8692ffd700 1 --2- 172.21.15.59:0/1566202890 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f868406b4a0 0x7f868406d960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:42.978 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:42.972+0000 7f8692ffd700 1 -- 172.21.15.59:0/1566202890 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(78..78 src has 1..78) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f869c089e50 con 0x7f86a0143ae0 2023-11-24T07:59:42.978 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:42.972+0000 7f86a6108700 1 --2- 172.21.15.59:0/1566202890 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f868406b4a0 0x7f868406d960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:42.978 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:42.972+0000 7f86a6108700 1 --2- 172.21.15.59:0/1566202890 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f868406b4a0 0x7f868406d960 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f8694000f60 tx=0x7f8694009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:59:42.979 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:42.976+0000 7f8692ffd700 1 -- 172.21.15.59:0/1566202890 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f869c054840 con 0x7f86a0143ae0 2023-11-24T07:59:43.268 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:43.267+0000 7f86a836c700 1 -- 172.21.15.59:0/1566202890 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_volumes.TestSubvolumeSnapshots.test_subvolume_inherited_snapshot_info"]} v 0) v1 -- 0x7f86a00b5d20 con 0x7f86a0143ae0 2023-11-24T07:59:43.452 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:43.450+0000 7f8692ffd700 1 -- 172.21.15.59:0/1566202890 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_volumes.TestSubvolumeSnapshots.test_subvolume_inherited_snapshot_info"]}]=0 v205) v1 ==== 166+0+0 (secure 0 0 0) 0x7f869c01e0e0 con 0x7f86a0143ae0 2023-11-24T07:59:43.454 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:43.452+0000 7f86a836c700 1 -- 172.21.15.59:0/1566202890 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f868406b4a0 msgr2=0x7f868406d960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:43.454 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:43.452+0000 7f86a836c700 1 --2- 172.21.15.59:0/1566202890 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f868406b4a0 0x7f868406d960 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f8694000f60 tx=0x7f8694009040 comp rx=0 tx=0).stop 2023-11-24T07:59:43.455 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:43.452+0000 7f86a836c700 1 -- 172.21.15.59:0/1566202890 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f86a0143ae0 msgr2=0x7f86a013f550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:43.455 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:43.452+0000 7f86a836c700 1 --2- 172.21.15.59:0/1566202890 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f86a0143ae0 0x7f86a013f550 secure :-1 s=READY pgs=982 cs=0 l=1 rev1=1 crypto rx=0x7f869c00c750 tx=0x7f869c00cb10 comp rx=0 tx=0).stop 2023-11-24T07:59:43.455 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:43.453+0000 7f86a836c700 1 -- 172.21.15.59:0/1566202890 shutdown_connections 2023-11-24T07:59:43.455 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:43.453+0000 7f86a836c700 1 --2- 172.21.15.59:0/1566202890 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f868406b4a0 0x7f868406d960 unknown :-1 s=CLOSED pgs=453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:43.455 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:43.453+0000 7f86a836c700 1 --2- 172.21.15.59:0/1566202890 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f86a01440f0 0x7f86a00ce880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:43.455 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:43.453+0000 7f86a836c700 1 --2- 172.21.15.59:0/1566202890 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f86a00722c0 0x7f86a01f4920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:43.455 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:43.453+0000 7f86a836c700 1 --2- 172.21.15.59:0/1566202890 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f86a0143ae0 0x7f86a013f550 unknown :-1 s=CLOSED pgs=982 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:43.455 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:43.453+0000 7f86a836c700 1 -- 172.21.15.59:0/1566202890 >> 172.21.15.59:0/1566202890 conn(0x7f86a00b2740 msgr2=0x7f86a01410d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:59:43.455 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:43.453+0000 7f86a836c700 1 -- 172.21.15.59:0/1566202890 shutdown_connections 2023-11-24T07:59:43.455 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:43.453+0000 7f86a836c700 1 -- 172.21.15.59:0/1566202890 wait complete. 2023-11-24T07:59:43.467 INFO:tasks.cephfs_test_runner:tests the scenario where 'fs subvolume snapshot info' command ... ok 2023-11-24T07:59:43.467 INFO:tasks.cephfs_test_runner:Starting test: test_subvolume_inherited_snapshot_ls (tasks.cephfs.test_volumes.TestSubvolumeSnapshots) 2023-11-24T07:59:43.468 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-24T07:59:43.592 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:43.591+0000 7f700488e700 1 Processor -- start 2023-11-24T07:59:43.593 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:43.591+0000 7f700488e700 1 -- start start 2023-11-24T07:59:43.594 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:43.592+0000 7f700488e700 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f700013edd0 0x7f70000cffb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:43.595 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:43.592+0000 7f700488e700 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f70000d04f0 0x7f70001414b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:43.595 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:43.592+0000 7f700488e700 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f700013f1b0 con 0x7f70001419f0 2023-11-24T07:59:43.595 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:43.592+0000 7f700488e700 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7000141e50 con 0x7f700013edd0 2023-11-24T07:59:43.595 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:43.592+0000 7f700488e700 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7000072b50 con 0x7f70000d04f0 2023-11-24T07:59:43.595 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:43.592+0000 7f6ffe59c700 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f700013edd0 0x7f70000cffb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:43.595 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:43.592+0000 7f6ffdd9b700 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f70000d04f0 0x7f70001414b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:43.595 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:43.592+0000 7f6ffed9d700 1 --1- >> v1:172.21.15.76:6789/0 conn(0x7f70001419f0 0x7f700013f460 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.59:46332/0 (socket says 172.21.15.59:46332) 2023-11-24T07:59:43.595 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:43.592+0000 7f6ffed9d700 1 -- 172.21.15.59:0/3217819587 learned_addr learned my addr 172.21.15.59:0/3217819587 (peer_addr_for_me v1:172.21.15.59:0/0) 2023-11-24T07:59:43.595 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:43.592+0000 7f6ffd59a700 1 -- 172.21.15.59:0/3217819587 <== mon.1 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3305143997 0 0) 0x7f700013f1b0 con 0x7f70001419f0 2023-11-24T07:59:43.595 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:43.592+0000 7f6ffd59a700 1 -- 172.21.15.59:0/3217819587 --> v1:172.21.15.76:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6fe80035b0 con 0x7f70001419f0 2023-11-24T07:59:43.595 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:43.592+0000 7f6ffe59c700 1 -- 172.21.15.59:0/3217819587 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f70000d04f0 msgr2=0x7f70001414b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T07:59:43.596 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:43.592+0000 7f6ffe59c700 1 --2- 172.21.15.59:0/3217819587 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f70000d04f0 0x7f70001414b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:43.596 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:43.592+0000 7f6ffe59c700 1 -- 172.21.15.59:0/3217819587 >> v1:172.21.15.76:6789/0 conn(0x7f70001419f0 legacy=0x7f700013f460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:43.596 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:43.592+0000 7f6ffe59c700 1 -- 172.21.15.59:0/3217819587 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7000072870 con 0x7f700013edd0 2023-11-24T07:59:43.596 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:43.592+0000 7f6ffdd9b700 1 --2- 172.21.15.59:0/3217819587 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f70000d04f0 0x7f70001414b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-24T07:59:43.596 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:43.593+0000 7f6ffe59c700 1 --2- 172.21.15.59:0/3217819587 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f700013edd0 0x7f70000cffb0 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f6ff0009840 tx=0x7f6ff0009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b4630f9855a051ce server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:59:43.596 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:43.593+0000 7f6ffd59a700 1 -- 172.21.15.59:0/3217819587 <== mon.1 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ff0011830 con 0x7f700013edd0 2023-11-24T07:59:43.596 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:43.593+0000 7f6ffd59a700 1 -- 172.21.15.59:0/3217819587 <== mon.1 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6ff0011990 con 0x7f700013edd0 2023-11-24T07:59:43.596 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:43.593+0000 7f6ffd59a700 1 -- 172.21.15.59:0/3217819587 <== mon.1 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ff0011d00 con 0x7f700013edd0 2023-11-24T07:59:43.600 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:43.593+0000 7f700488e700 1 -- 172.21.15.59:0/3217819587 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f700013edd0 msgr2=0x7f70000cffb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:43.600 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:43.593+0000 7f700488e700 1 --2- 172.21.15.59:0/3217819587 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f700013edd0 0x7f70000cffb0 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f6ff0009840 tx=0x7f6ff0009b50 comp rx=0 tx=0).stop 2023-11-24T07:59:43.600 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:43.593+0000 7f700488e700 1 -- 172.21.15.59:0/3217819587 shutdown_connections 2023-11-24T07:59:43.600 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:43.593+0000 7f700488e700 1 --2- 172.21.15.59:0/3217819587 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f700013edd0 0x7f70000cffb0 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:43.600 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:43.593+0000 7f700488e700 1 --2- 172.21.15.59:0/3217819587 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f70000d04f0 0x7f70001414b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:43.600 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:43.593+0000 7f700488e700 1 -- 172.21.15.59:0/3217819587 >> 172.21.15.59:0/3217819587 conn(0x7f70000b2840 msgr2=0x7f70000b2c20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:59:43.600 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:43.594+0000 7f700488e700 1 -- 172.21.15.59:0/3217819587 shutdown_connections 2023-11-24T07:59:43.600 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:43.594+0000 7f700488e700 1 -- 172.21.15.59:0/3217819587 wait complete. 2023-11-24T07:59:43.600 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:43.595+0000 7f700488e700 1 Processor -- start 2023-11-24T07:59:43.601 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:43.595+0000 7f700488e700 1 -- start start 2023-11-24T07:59:43.601 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:43.596+0000 7f700488e700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f70000d04f0 0x7f700013af20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:43.601 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:43.596+0000 7f700488e700 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f700013edd0 0x7f7000158e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:43.601 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:43.596+0000 7f700488e700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f70001419f0 0x7f7000140af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:43.601 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:43.596+0000 7f700488e700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7000142cc0 con 0x7f70000d04f0 2023-11-24T07:59:43.602 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:43.596+0000 7f700488e700 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f700013b460 con 0x7f70001419f0 2023-11-24T07:59:43.602 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:43.596+0000 7f700488e700 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7000157db0 con 0x7f700013edd0 2023-11-24T07:59:43.602 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:43.596+0000 7f6ffe59c700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f70000d04f0 0x7f700013af20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:43.603 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:43.596+0000 7f6ffe59c700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f70000d04f0 0x7f700013af20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:55582/0 (socket says 172.21.15.59:55582) 2023-11-24T07:59:43.603 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:43.596+0000 7f6ffe59c700 1 -- 172.21.15.59:0/809037416 learned_addr learned my addr 172.21.15.59:0/809037416 (peer_addr_for_me v2:172.21.15.59:0/0) 2023-11-24T07:59:43.603 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:43.596+0000 7f6ffed9d700 1 --2- 172.21.15.59:0/809037416 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f70001419f0 0x7f7000140af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:43.603 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:43.596+0000 7f6ffdd9b700 1 --2- 172.21.15.59:0/809037416 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f700013edd0 0x7f7000158e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:43.603 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:43.596+0000 7f6ffe59c700 1 -- 172.21.15.59:0/809037416 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f700013edd0 msgr2=0x7f7000158e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:43.604 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:43.596+0000 7f6ffe59c700 1 --2- 172.21.15.59:0/809037416 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f700013edd0 0x7f7000158e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:43.604 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:43.596+0000 7f6ffe59c700 1 -- 172.21.15.59:0/809037416 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f70001419f0 msgr2=0x7f7000140af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:43.604 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:43.596+0000 7f6ffe59c700 1 --2- 172.21.15.59:0/809037416 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f70001419f0 0x7f7000140af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:43.605 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:43.596+0000 7f6ffe59c700 1 -- 172.21.15.59:0/809037416 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6ff40094a0 con 0x7f70000d04f0 2023-11-24T07:59:43.605 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:43.596+0000 7f6ffed9d700 1 --2- 172.21.15.59:0/809037416 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f70001419f0 0x7f7000140af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-24T07:59:43.605 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:43.596+0000 7f6ffe59c700 1 --2- 172.21.15.59:0/809037416 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f70000d04f0 0x7f700013af20 secure :-1 s=READY pgs=983 cs=0 l=1 rev1=1 crypto rx=0x7f6ff00090b0 tx=0x7f6ff0004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:59:43.605 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:43.596+0000 7f6feeffd700 1 -- 172.21.15.59:0/809037416 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ff0004550 con 0x7f70000d04f0 2023-11-24T07:59:43.606 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:43.596+0000 7f6feeffd700 1 -- 172.21.15.59:0/809037416 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6ff001f070 con 0x7f70000d04f0 2023-11-24T07:59:43.606 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:43.596+0000 7f6feeffd700 1 -- 172.21.15.59:0/809037416 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ff00038a0 con 0x7f70000d04f0 2023-11-24T07:59:43.606 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:43.597+0000 7f700488e700 1 -- 172.21.15.59:0/809037416 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6ff00094a0 con 0x7f70000d04f0 2023-11-24T07:59:43.606 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:43.597+0000 7f700488e700 1 -- 172.21.15.59:0/809037416 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7000158370 con 0x7f70000d04f0 2023-11-24T07:59:43.607 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:43.597+0000 7f700488e700 1 -- 172.21.15.59:0/809037416 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7000158ca0 con 0x7f70000d04f0 2023-11-24T07:59:43.607 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:43.598+0000 7f6feeffd700 1 -- 172.21.15.59:0/809037416 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 11) v1 ==== 89741+0+0 (secure 0 0 0) 0x7f6ff0003a00 con 0x7f70000d04f0 2023-11-24T07:59:43.607 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:43.598+0000 7f6feeffd700 1 --2- 172.21.15.59:0/809037416 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f6fe006b470 0x7f6fe006d930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:43.607 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:43.599+0000 7f6feeffd700 1 -- 172.21.15.59:0/809037416 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(78..78 src has 1..78) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f6ff0021070 con 0x7f70000d04f0 2023-11-24T07:59:43.608 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:43.599+0000 7f6ffdd9b700 1 --2- 172.21.15.59:0/809037416 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f6fe006b470 0x7f6fe006d930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:43.609 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:43.599+0000 7f6ffdd9b700 1 --2- 172.21.15.59:0/809037416 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f6fe006b470 0x7f6fe006d930 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f70000b0570 tx=0x7f6ff4009210 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:59:43.609 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:43.603+0000 7f6feeffd700 1 -- 172.21.15.59:0/809037416 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f6ff0056130 con 0x7f70000d04f0 2023-11-24T07:59:43.807 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:43.806+0000 7f700488e700 1 -- 172.21.15.59:0/809037416 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f7000068670 con 0x7f70000d04f0 2023-11-24T07:59:43.808 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:43.807+0000 7f6feeffd700 1 -- 172.21.15.59:0/809037416 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v78) v1 ==== 74+0+12915 (secure 0 0 0) 0x7f6ff0006cc0 con 0x7f70000d04f0 2023-11-24T07:59:43.808 INFO:teuthology.orchestra.run.smithi059.stdout: 2023-11-24T07:59:43.809 INFO:teuthology.orchestra.run.smithi059.stdout:{"epoch":78,"fsid":"9f0e8741-b8f9-48d5-a2c3-09f45903303d","created":"2023-11-24T07:56:06.552729+0000","modified":"2023-11-24T07:59:42.439944+0000","last_up_change":"2023-11-24T07:56:17.372352+0000","last_in_change":"2023-11-24T07:56:09.073250+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":11,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-24T07:56:18.352823+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"6d8f9e9b-1782-4e41-928b-a39f2a8ebc06","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6800","nonce":3373878772},{"type":"v1","addr":"172.21.15.59:6801","nonce":3373878772}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6802","nonce":3373878772},{"type":"v1","addr":"172.21.15.59:6803","nonce":3373878772}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6806","nonce":3373878772},{"type":"v1","addr":"172.21.15.59:6807","nonce":3373878772}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6804","nonce":3373878772},{"type":"v1","addr":"172.21.15.59:6805","nonce":3373878772}]},"public_addr":"172.21.15.59:6801/3373878772","cluster_addr":"172.21.15.59:6803/3373878772","heartbeat_back_addr":"172.21.15.59:6807/3373878772","heartbeat_front_addr":"172.21.15.59:6805/3373878772","state":["exists","up"]},{"osd":1,"uuid":"e8d882aa-52e1-4c04-9e1e-c44ef9204a5f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6824","nonce":648258563},{"type":"v1","addr":"172.21.15.59:6825","nonce":648258563}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6826","nonce":648258563},{"type":"v1","addr":"172.21.15.59:6827","nonce":648258563}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6830","nonce":648258563},{"type":"v1","addr":"172.21.15.59:6831","nonce":648258563}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6828","nonce":648258563},{"type":"v1","addr":"172.21.15.59:6829","nonce":648258563}]},"public_addr":"172.21.15.59:6825/648258563","cluster_addr":"172.21.15.59:6827/648258563","heartbeat_back_addr":"172.21.15.59:6831/648258563","heartbeat_front_addr":"172.21.15.59:6829/648258563","state":["exists","up"]},{"osd":2,"uuid":"69eebfa0-695b-4b21-90b9-8a3537168a01","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6808","nonce":1840430654},{"type":"v1","addr":"172.21.15.59:6809","nonce":1840430654}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6810","nonce":1840430654},{"type":"v1","addr":"172.21.15.59:6811","nonce":1840430654}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6814","nonce":1840430654},{"type":"v1","addr":"172.21.15.59:6815","nonce":1840430654}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6812","nonce":1840430654},{"type":"v1","addr":"172.21.15.59:6813","nonce":1840430654}]},"public_addr":"172.21.15.59:6809/1840430654","cluster_addr":"172.21.15.59:6811/1840430654","heartbeat_back_addr":"172.21.15.59:6815/1840430654","heartbeat_front_addr":"172.21.15.59:6813/1840430654","state":["exists","up"]},{"osd":3,"uuid":"96fd77fc-f911-46ff-9445-6f0be2daa8f4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6816","nonce":2504025850},{"type":"v1","addr":"172.21.15.59:6817","nonce":2504025850}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6818","nonce":2504025850},{"type":"v1","addr":"172.21.15.59:6819","nonce":2504025850}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6822","nonce":2504025850},{"type":"v1","addr":"172.21.15.59:6823","nonce":2504025850}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6820","nonce":2504025850},{"type":"v1","addr":"172.21.15.59:6821","nonce":2504025850}]},"public_addr":"172.21.15.59:6817/2504025850","cluster_addr":"172.21.15.59:6819/2504025850","heartbeat_back_addr":"172.21.15.59:6823/2504025850","heartbeat_front_addr":"172.21.15.59:6821/2504025850","state":["exists","up"]},{"osd":4,"uuid":"7ac29fca-b5b2-4c95-9d69-2f164cbbc708","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6816","nonce":2382463431},{"type":"v1","addr":"172.21.15.76:6818","nonce":2382463431}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6820","nonce":2382463431},{"type":"v1","addr":"172.21.15.76:6822","nonce":2382463431}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6828","nonce":2382463431},{"type":"v1","addr":"172.21.15.76:6830","nonce":2382463431}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6824","nonce":2382463431},{"type":"v1","addr":"172.21.15.76:6826","nonce":2382463431}]},"public_addr":"172.21.15.76:6818/2382463431","cluster_addr":"172.21.15.76:6822/2382463431","heartbeat_back_addr":"172.21.15.76:6830/2382463431","heartbeat_front_addr":"172.21.15.76:6826/2382463431","state":["exists","up"]},{"osd":5,"uuid":"485bf711-3851-4238-a4f4-51d52e0a9c58","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6800","nonce":329334308},{"type":"v1","addr":"172.21.15.76:6802","nonce":329334308}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6805","nonce":329334308},{"type":"v1","addr":"172.21.15.76:6807","nonce":329334308}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6814","nonce":329334308},{"type":"v1","addr":"172.21.15.76:6815","nonce":329334308}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6810","nonce":329334308},{"type":"v1","addr":"172.21.15.76:6812","nonce":329334308}]},"public_addr":"172.21.15.76:6802/329334308","cluster_addr":"172.21.15.76:6807/329334308","heartbeat_back_addr":"172.21.15.76:6815/329334308","heartbeat_front_addr":"172.21.15.76:6812/329334308","state":["exists","up"]},{"osd":6,"uuid":"72b3f2fc-1e4a-4e32-968d-003647a0286f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6801","nonce":3946579410},{"type":"v1","addr":"172.21.15.76:6803","nonce":3946579410}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6804","nonce":3946579410},{"type":"v1","addr":"172.21.15.76:6806","nonce":3946579410}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6811","nonce":3946579410},{"type":"v1","addr":"172.21.15.76:6813","nonce":3946579410}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6808","nonce":3946579410},{"type":"v1","addr":"172.21.15.76:6809","nonce":3946579410}]},"public_addr":"172.21.15.76:6803/3946579410","cluster_addr":"172.21.15.76:6806/3946579410","heartbeat_back_addr":"172.21.15.76:6813/3946579410","heartbeat_front_addr":"172.21.15.76:6809/3946579410","state":["exists","up"]},{"osd":7,"uuid":"1ddceb90-6c5d-4d1a-ad56-99cb877f8c70","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6817","nonce":841761538},{"type":"v1","addr":"172.21.15.76:6819","nonce":841761538}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6821","nonce":841761538},{"type":"v1","addr":"172.21.15.76:6823","nonce":841761538}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6829","nonce":841761538},{"type":"v1","addr":"172.21.15.76:6831","nonce":841761538}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6825","nonce":841761538},{"type":"v1","addr":"172.21.15.76:6827","nonce":841761538}]},"public_addr":"172.21.15.76:6819/841761538","cluster_addr":"172.21.15.76:6823/841761538","heartbeat_back_addr":"172.21.15.76:6831/841761538","heartbeat_front_addr":"172.21.15.76:6827/841761538","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T07:56:14.294838+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T07:56:14.270779+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T07:56:14.231809+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T07:56:14.260983+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T07:56:14.302997+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T07:56:14.258600+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T07:56:14.251413+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T07:56:14.304727+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.59:6835/1228649189":"2023-11-25T07:59:39.785740+0000","172.21.15.59:6833/1228649189":"2023-11-25T07:59:39.785740+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-24T07:59:43.811 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:43.809+0000 7f700488e700 1 -- 172.21.15.59:0/809037416 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f6fe006b470 msgr2=0x7f6fe006d930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:43.811 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:43.809+0000 7f700488e700 1 --2- 172.21.15.59:0/809037416 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f6fe006b470 0x7f6fe006d930 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f70000b0570 tx=0x7f6ff4009210 comp rx=0 tx=0).stop 2023-11-24T07:59:43.811 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:43.809+0000 7f700488e700 1 -- 172.21.15.59:0/809037416 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f70000d04f0 msgr2=0x7f700013af20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:43.811 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:43.809+0000 7f700488e700 1 --2- 172.21.15.59:0/809037416 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f70000d04f0 0x7f700013af20 secure :-1 s=READY pgs=983 cs=0 l=1 rev1=1 crypto rx=0x7f6ff00090b0 tx=0x7f6ff0004750 comp rx=0 tx=0).stop 2023-11-24T07:59:43.811 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:43.809+0000 7f700488e700 1 -- 172.21.15.59:0/809037416 shutdown_connections 2023-11-24T07:59:43.812 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:43.809+0000 7f700488e700 1 --2- 172.21.15.59:0/809037416 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f6fe006b470 0x7f6fe006d930 unknown :-1 s=CLOSED pgs=454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:43.812 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:43.809+0000 7f700488e700 1 --2- 172.21.15.59:0/809037416 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f70001419f0 0x7f7000140af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:43.812 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:43.809+0000 7f700488e700 1 --2- 172.21.15.59:0/809037416 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f70000d04f0 0x7f700013af20 unknown :-1 s=CLOSED pgs=983 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:43.812 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:43.809+0000 7f700488e700 1 --2- 172.21.15.59:0/809037416 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f700013edd0 0x7f7000158e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:43.813 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:43.809+0000 7f700488e700 1 -- 172.21.15.59:0/809037416 >> 172.21.15.59:0/809037416 conn(0x7f70000b2840 msgr2=0x7f7000071b40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:59:43.813 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:43.809+0000 7f700488e700 1 -- 172.21.15.59:0/809037416 shutdown_connections 2023-11-24T07:59:43.813 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:43.809+0000 7f700488e700 1 -- 172.21.15.59:0/809037416 wait complete. 2023-11-24T07:59:43.823 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-24T07:56:18.352823+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-24T07:59:43.823 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-24T07:59:43.951 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:43.949+0000 7fb506712700 1 Processor -- start 2023-11-24T07:59:43.951 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:43.949+0000 7fb506712700 1 -- start start 2023-11-24T07:59:43.954 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:43.950+0000 7fb506712700 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fb50006afd0 0x7fb50006b430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:43.954 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:43.950+0000 7fb506712700 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fb5000cebf0 0x7fb5000cf080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:43.954 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:43.950+0000 7fb506712700 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb50006b970 con 0x7fb5001598d0 2023-11-24T07:59:43.954 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:43.950+0000 7fb506712700 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb500160240 con 0x7fb50006afd0 2023-11-24T07:59:43.954 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:43.950+0000 7fb506712700 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb50015f8e0 con 0x7fb5000cebf0 2023-11-24T07:59:43.954 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:43.950+0000 7fb4fffff700 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7fb5001598d0 0x7fb5000cb160 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:59064/0 (socket says 172.21.15.59:59064) 2023-11-24T07:59:43.955 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:43.950+0000 7fb4fffff700 1 -- 172.21.15.59:0/151879091 learned_addr learned my addr 172.21.15.59:0/151879091 (peer_addr_for_me v1:172.21.15.59:0/0) 2023-11-24T07:59:43.955 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:43.951+0000 7fb4ff7fe700 1 --2- 172.21.15.59:0/151879091 >> v2:172.21.15.76:3300/0 conn(0x7fb50006afd0 0x7fb50006b430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:43.955 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:43.951+0000 7fb504caf700 1 --2- 172.21.15.59:0/151879091 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fb5000cebf0 0x7fb5000cf080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:43.955 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:43.951+0000 7fb4feffd700 1 -- 172.21.15.59:0/151879091 <== mon.0 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2915962270 0 0) 0x7fb50006b970 con 0x7fb5001598d0 2023-11-24T07:59:43.955 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:43.951+0000 7fb4feffd700 1 -- 172.21.15.59:0/151879091 --> v1:172.21.15.59:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb4ec0035b0 con 0x7fb5001598d0 2023-11-24T07:59:43.955 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:43.951+0000 7fb4ff7fe700 1 -- 172.21.15.59:0/151879091 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fb5000cebf0 msgr2=0x7fb5000cf080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T07:59:43.955 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:43.951+0000 7fb4ff7fe700 1 --2- 172.21.15.59:0/151879091 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fb5000cebf0 0x7fb5000cf080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:43.955 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:43.951+0000 7fb4ff7fe700 1 -- 172.21.15.59:0/151879091 >> v1:172.21.15.59:6789/0 conn(0x7fb5001598d0 legacy=0x7fb5000cb160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:43.956 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:43.951+0000 7fb4ff7fe700 1 -- 172.21.15.59:0/151879091 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb50015d250 con 0x7fb50006afd0 2023-11-24T07:59:43.956 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:43.951+0000 7fb504caf700 1 --2- 172.21.15.59:0/151879091 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fb5000cebf0 0x7fb5000cf080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-24T07:59:43.956 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:43.951+0000 7fb4ff7fe700 1 --2- 172.21.15.59:0/151879091 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb50006afd0 0x7fb50006b430 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7fb4f0009840 tx=0x7fb4f0009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c477e41da466630b server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:59:43.956 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:43.951+0000 7fb4feffd700 1 -- 172.21.15.59:0/151879091 <== mon.1 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4f0011830 con 0x7fb50006afd0 2023-11-24T07:59:43.956 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:43.951+0000 7fb4feffd700 1 -- 172.21.15.59:0/151879091 <== mon.1 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb4f0011990 con 0x7fb50006afd0 2023-11-24T07:59:43.956 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:43.951+0000 7fb4feffd700 1 -- 172.21.15.59:0/151879091 <== mon.1 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4f0011d20 con 0x7fb50006afd0 2023-11-24T07:59:43.956 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:43.952+0000 7fb506712700 1 -- 172.21.15.59:0/151879091 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb50006afd0 msgr2=0x7fb50006b430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:43.956 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:43.952+0000 7fb506712700 1 --2- 172.21.15.59:0/151879091 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb50006afd0 0x7fb50006b430 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7fb4f0009840 tx=0x7fb4f0009b50 comp rx=0 tx=0).stop 2023-11-24T07:59:43.956 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:43.952+0000 7fb506712700 1 -- 172.21.15.59:0/151879091 shutdown_connections 2023-11-24T07:59:43.956 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:43.952+0000 7fb506712700 1 --2- 172.21.15.59:0/151879091 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fb5000cebf0 0x7fb5000cf080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:43.956 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:43.952+0000 7fb506712700 1 --2- 172.21.15.59:0/151879091 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb50006afd0 0x7fb50006b430 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:43.956 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:43.952+0000 7fb506712700 1 -- 172.21.15.59:0/151879091 >> 172.21.15.59:0/151879091 conn(0x7fb5000d0b10 msgr2=0x7fb5000d0f20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:59:43.957 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:43.952+0000 7fb506712700 1 -- 172.21.15.59:0/151879091 shutdown_connections 2023-11-24T07:59:43.957 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:43.952+0000 7fb506712700 1 -- 172.21.15.59:0/151879091 wait complete. 2023-11-24T07:59:43.959 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:43.953+0000 7fb506712700 1 Processor -- start 2023-11-24T07:59:43.959 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:43.953+0000 7fb506712700 1 -- start start 2023-11-24T07:59:43.960 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:43.954+0000 7fb506712700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb50006afd0 0x7fb500153fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:43.960 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:43.954+0000 7fb506712700 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fb5000cebf0 0x7fb5000b5e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:43.960 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:43.954+0000 7fb506712700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb5001598d0 0x7fb50006ef80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:43.960 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:43.954+0000 7fb506712700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5000b6760 con 0x7fb50006afd0 2023-11-24T07:59:43.960 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:43.954+0000 7fb506712700 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5001e6e20 con 0x7fb5001598d0 2023-11-24T07:59:43.960 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:43.954+0000 7fb506712700 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb5001e7100 con 0x7fb5000cebf0 2023-11-24T07:59:43.961 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:43.954+0000 7fb4fffff700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb50006afd0 0x7fb500153fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:43.961 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:43.954+0000 7fb4fffff700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb50006afd0 0x7fb500153fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:55594/0 (socket says 172.21.15.59:55594) 2023-11-24T07:59:43.961 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:43.954+0000 7fb4fffff700 1 -- 172.21.15.59:0/3627735270 learned_addr learned my addr 172.21.15.59:0/3627735270 (peer_addr_for_me v2:172.21.15.59:0/0) 2023-11-24T07:59:43.961 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:43.954+0000 7fb4ff7fe700 1 --2- 172.21.15.59:0/3627735270 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fb5000cebf0 0x7fb5000b5e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:43.961 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:43.954+0000 7fb504caf700 1 --2- 172.21.15.59:0/3627735270 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb5001598d0 0x7fb50006ef80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:43.961 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:43.954+0000 7fb4fffff700 1 -- 172.21.15.59:0/3627735270 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fb5000cebf0 msgr2=0x7fb5000b5e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:43.961 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:43.954+0000 7fb4fffff700 1 --2- 172.21.15.59:0/3627735270 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fb5000cebf0 0x7fb5000b5e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:43.961 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:43.954+0000 7fb4fffff700 1 -- 172.21.15.59:0/3627735270 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb5001598d0 msgr2=0x7fb50006ef80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:43.962 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:43.954+0000 7fb4fffff700 1 --2- 172.21.15.59:0/3627735270 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb5001598d0 0x7fb50006ef80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:43.962 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:43.954+0000 7fb4fffff700 1 -- 172.21.15.59:0/3627735270 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb4f8009a70 con 0x7fb50006afd0 2023-11-24T07:59:43.962 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:43.955+0000 7fb4fffff700 1 --2- 172.21.15.59:0/3627735270 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb50006afd0 0x7fb500153fd0 secure :-1 s=READY pgs=985 cs=0 l=1 rev1=1 crypto rx=0x7fb4e8001eb0 tx=0x7fb4e800c7b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:59:43.962 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:43.955+0000 7fb4fcff9700 1 -- 172.21.15.59:0/3627735270 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4e800cea0 con 0x7fb50006afd0 2023-11-24T07:59:43.962 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:43.955+0000 7fb4fcff9700 1 -- 172.21.15.59:0/3627735270 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb4e8005d70 con 0x7fb50006afd0 2023-11-24T07:59:43.962 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:43.955+0000 7fb506712700 1 -- 172.21.15.59:0/3627735270 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb4f00094a0 con 0x7fb50006afd0 2023-11-24T07:59:43.962 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:43.955+0000 7fb4fcff9700 1 -- 172.21.15.59:0/3627735270 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4e80072f0 con 0x7fb50006afd0 2023-11-24T07:59:43.962 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:43.955+0000 7fb506712700 1 -- 172.21.15.59:0/3627735270 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb5000b6350 con 0x7fb50006afd0 2023-11-24T07:59:43.963 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:43.956+0000 7fb506712700 1 -- 172.21.15.59:0/3627735270 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb5001e6c80 con 0x7fb50006afd0 2023-11-24T07:59:43.963 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:43.956+0000 7fb4fcff9700 1 -- 172.21.15.59:0/3627735270 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 11) v1 ==== 89741+0+0 (secure 0 0 0) 0x7fb4e8007450 con 0x7fb50006afd0 2023-11-24T07:59:43.963 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:43.957+0000 7fb4fcff9700 1 --2- 172.21.15.59:0/3627735270 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7fb4e006b470 0x7fb4e006d930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:43.963 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:43.957+0000 7fb4fcff9700 1 -- 172.21.15.59:0/3627735270 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(78..78 src has 1..78) v4 ==== 4943+0+0 (secure 0 0 0) 0x7fb4e80891b0 con 0x7fb50006afd0 2023-11-24T07:59:43.963 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:43.957+0000 7fb4ff7fe700 1 --2- 172.21.15.59:0/3627735270 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7fb4e006b470 0x7fb4e006d930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:43.963 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:43.957+0000 7fb4ff7fe700 1 --2- 172.21.15.59:0/3627735270 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7fb4e006b470 0x7fb4e006d930 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7fb4f0000c00 tx=0x7fb4f0008030 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:59:43.964 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:43.961+0000 7fb4fcff9700 1 -- 172.21.15.59:0/3627735270 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fb5001e6c80 con 0x7fb50006afd0 2023-11-24T07:59:44.163 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:44.161+0000 7fb506712700 1 -- 172.21.15.59:0/3627735270 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fb5001e6c80 con 0x7fb50006afd0 2023-11-24T07:59:44.163 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:44.162+0000 7fb4fcff9700 1 -- 172.21.15.59:0/3627735270 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v78) v1 ==== 93+0+10 (secure 0 0 0) 0x7fb5001e6c80 con 0x7fb50006afd0 2023-11-24T07:59:44.164 INFO:teuthology.orchestra.run.smithi059.stdout:pg_num: 1 2023-11-24T07:59:44.166 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:44.164+0000 7fb506712700 1 -- 172.21.15.59:0/3627735270 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7fb4e006b470 msgr2=0x7fb4e006d930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:44.166 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:44.164+0000 7fb506712700 1 --2- 172.21.15.59:0/3627735270 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7fb4e006b470 0x7fb4e006d930 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7fb4f0000c00 tx=0x7fb4f0008030 comp rx=0 tx=0).stop 2023-11-24T07:59:44.166 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:44.164+0000 7fb506712700 1 -- 172.21.15.59:0/3627735270 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb50006afd0 msgr2=0x7fb500153fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:44.166 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:44.164+0000 7fb506712700 1 --2- 172.21.15.59:0/3627735270 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb50006afd0 0x7fb500153fd0 secure :-1 s=READY pgs=985 cs=0 l=1 rev1=1 crypto rx=0x7fb4e8001eb0 tx=0x7fb4e800c7b0 comp rx=0 tx=0).stop 2023-11-24T07:59:44.166 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:44.164+0000 7fb506712700 1 -- 172.21.15.59:0/3627735270 shutdown_connections 2023-11-24T07:59:44.167 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:44.164+0000 7fb506712700 1 --2- 172.21.15.59:0/3627735270 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7fb4e006b470 0x7fb4e006d930 unknown :-1 s=CLOSED pgs=455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:44.167 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:44.164+0000 7fb506712700 1 --2- 172.21.15.59:0/3627735270 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb5001598d0 0x7fb50006ef80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:44.167 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:44.164+0000 7fb506712700 1 --2- 172.21.15.59:0/3627735270 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb50006afd0 0x7fb500153fd0 unknown :-1 s=CLOSED pgs=985 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:44.167 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:44.164+0000 7fb506712700 1 --2- 172.21.15.59:0/3627735270 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fb5000cebf0 0x7fb5000b5e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:44.167 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:44.164+0000 7fb506712700 1 -- 172.21.15.59:0/3627735270 >> 172.21.15.59:0/3627735270 conn(0x7fb5000d0b10 msgr2=0x7fb500157500 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:59:44.167 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:44.164+0000 7fb506712700 1 -- 172.21.15.59:0/3627735270 shutdown_connections 2023-11-24T07:59:44.167 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:44.164+0000 7fb506712700 1 -- 172.21.15.59:0/3627735270 wait complete. 2023-11-24T07:59:44.178 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_volumes.TestSubvolumeSnapshots.test_subvolume_inherited_snapshot_ls' 2023-11-24T07:59:44.310 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:44.302+0000 7f487d4b7700 1 Processor -- start 2023-11-24T07:59:44.311 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:44.302+0000 7f487d4b7700 1 -- start start 2023-11-24T07:59:44.311 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:44.303+0000 7f487d4b7700 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f487806afd0 0x7f48780d16e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:44.311 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:44.303+0000 7f487d4b7700 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4878070490 con 0x7f48780affb0 2023-11-24T07:59:44.311 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:44.303+0000 7f487d4b7700 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f48780b0430 con 0x7f48780ada10 2023-11-24T07:59:44.311 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:44.303+0000 7f487d4b7700 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f48780af9f0 con 0x7f487806afd0 2023-11-24T07:59:44.311 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:44.303+0000 7f48767fc700 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7f48780affb0 0x7f487806b3b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:59076/0 (socket says 172.21.15.59:59076) 2023-11-24T07:59:44.312 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:44.303+0000 7f48767fc700 1 -- 172.21.15.59:0/3909039251 learned_addr learned my addr 172.21.15.59:0/3909039251 (peer_addr_for_me v1:172.21.15.59:0/0) 2023-11-24T07:59:44.312 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:44.303+0000 7f4876ffd700 1 --2- 172.21.15.59:0/3909039251 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f487806afd0 0x7f48780d16e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:44.312 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:44.303+0000 7f4875ffb700 1 -- 172.21.15.59:0/3909039251 <== mon.0 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3588337248 0 0) 0x7f4878070490 con 0x7f48780affb0 2023-11-24T07:59:44.312 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:44.303+0000 7f4875ffb700 1 -- 172.21.15.59:0/3909039251 --> v1:172.21.15.59:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f48600035b0 con 0x7f48780affb0 2023-11-24T07:59:44.312 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:44.303+0000 7f4875ffb700 1 -- 172.21.15.59:0/3909039251 <== mon.1 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2279933180 0 0) 0x7f48780b0430 con 0x7f48780ada10 2023-11-24T07:59:44.312 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:44.303+0000 7f4875ffb700 1 -- 172.21.15.59:0/3909039251 --> v1:172.21.15.76:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4878070490 con 0x7f48780ada10 2023-11-24T07:59:44.312 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:44.303+0000 7f4875ffb700 1 -- 172.21.15.59:0/3909039251 <== mon.0 v1:172.21.15.59:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1369948381 0 0) 0x7f48600035b0 con 0x7f48780affb0 2023-11-24T07:59:44.313 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:44.303+0000 7f4875ffb700 1 -- 172.21.15.59:0/3909039251 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f487806afd0 msgr2=0x7f48780d16e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T07:59:44.313 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:44.303+0000 7f4875ffb700 1 --2- 172.21.15.59:0/3909039251 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f487806afd0 0x7f48780d16e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:44.313 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:44.303+0000 7f4875ffb700 1 -- 172.21.15.59:0/3909039251 >> v1:172.21.15.76:6789/0 conn(0x7f48780ada10 legacy=0x7f48780adea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:44.313 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:44.303+0000 7f4875ffb700 1 -- 172.21.15.59:0/3909039251 --> v1:172.21.15.59:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f48780afcd0 con 0x7f48780affb0 2023-11-24T07:59:44.313 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:44.303+0000 7f4875ffb700 1 -- 172.21.15.59:0/3909039251 <== mon.0 v1:172.21.15.59:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1312333566 0 0) 0x7f486c001670 con 0x7f48780affb0 2023-11-24T07:59:44.313 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:44.303+0000 7f4876ffd700 1 --2- 172.21.15.59:0/3909039251 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f487806afd0 0x7f48780d16e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-24T07:59:44.313 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:44.304+0000 7f4875ffb700 1 -- 172.21.15.59:0/3909039251 >> v1:172.21.15.59:6789/0 conn(0x7f48780affb0 legacy=0x7f487806b3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:44.313 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:44.304+0000 7f4875ffb700 1 --2- 172.21.15.59:0/3909039251 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4860003bc0 0x7f4860003fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:44.314 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:44.304+0000 7f4875ffb700 1 --2- 172.21.15.59:0/3909039251 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4860004850 0x7f4860004cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:44.314 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:44.304+0000 7f4875ffb700 1 --2- 172.21.15.59:0/3909039251 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f487806afd0 0x7f4860005210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:44.314 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:44.304+0000 7f4875ffb700 1 -- 172.21.15.59:0/3909039251 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48780af9f0 con 0x7f4860003bc0 2023-11-24T07:59:44.314 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:44.304+0000 7f4875ffb700 1 -- 172.21.15.59:0/3909039251 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f486000bac0 con 0x7f4860004850 2023-11-24T07:59:44.314 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:44.304+0000 7f4875ffb700 1 -- 172.21.15.59:0/3909039251 --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f486000bc50 con 0x7f487806afd0 2023-11-24T07:59:44.314 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:44.304+0000 7f4876ffd700 1 --2- 172.21.15.59:0/3909039251 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4860003bc0 0x7f4860003fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:44.314 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:44.304+0000 7f48767fc700 1 --2- 172.21.15.59:0/3909039251 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4860004850 0x7f4860004cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:44.315 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:44.304+0000 7f48777fe700 1 --2- 172.21.15.59:0/3909039251 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f487806afd0 0x7f4860005210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:44.315 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:44.304+0000 7f4876ffd700 1 -- 172.21.15.59:0/3909039251 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f487806afd0 msgr2=0x7f4860005210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T07:59:44.315 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:44.304+0000 7f4876ffd700 1 --2- 172.21.15.59:0/3909039251 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f487806afd0 0x7f4860005210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:44.315 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:44.304+0000 7f4876ffd700 1 -- 172.21.15.59:0/3909039251 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4860004850 msgr2=0x7f4860004cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T07:59:44.315 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:44.304+0000 7f4876ffd700 1 --2- 172.21.15.59:0/3909039251 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4860004850 0x7f4860004cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:44.315 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:44.304+0000 7f4876ffd700 1 -- 172.21.15.59:0/3909039251 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f48680094a0 con 0x7f4860003bc0 2023-11-24T07:59:44.315 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:44.304+0000 7f4876ffd700 1 --2- 172.21.15.59:0/3909039251 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4860003bc0 0x7f4860003fc0 secure :-1 s=READY pgs=987 cs=0 l=1 rev1=1 crypto rx=0x7f4868009d00 tx=0x7f4860005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d56f8c7b4725399f server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:59:44.315 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:44.304+0000 7f4875ffb700 1 -- 172.21.15.59:0/3909039251 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4868009e50 con 0x7f4860003bc0 2023-11-24T07:59:44.316 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:44.304+0000 7f4875ffb700 1 -- 172.21.15.59:0/3909039251 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f486800e9a0 con 0x7f4860003bc0 2023-11-24T07:59:44.317 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:44.304+0000 7f4875ffb700 1 -- 172.21.15.59:0/3909039251 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f486800ed30 con 0x7f4860003bc0 2023-11-24T07:59:44.317 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:44.305+0000 7f487d4b7700 1 -- 172.21.15.59:0/3909039251 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4860003bc0 msgr2=0x7f4860003fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:44.317 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:44.305+0000 7f487d4b7700 1 --2- 172.21.15.59:0/3909039251 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4860003bc0 0x7f4860003fc0 secure :-1 s=READY pgs=987 cs=0 l=1 rev1=1 crypto rx=0x7f4868009d00 tx=0x7f4860005880 comp rx=0 tx=0).stop 2023-11-24T07:59:44.317 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:44.305+0000 7f48777fe700 1 -- 172.21.15.59:0/3909039251 reap_dead start 2023-11-24T07:59:44.318 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:44.305+0000 7f487d4b7700 1 -- 172.21.15.59:0/3909039251 shutdown_connections 2023-11-24T07:59:44.318 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:44.305+0000 7f487d4b7700 1 -- 172.21.15.59:0/3909039251 >> 172.21.15.59:0/3909039251 conn(0x7f48780d0bc0 msgr2=0x7f48780d0fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:59:44.318 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:44.305+0000 7f487d4b7700 1 -- 172.21.15.59:0/3909039251 shutdown_connections 2023-11-24T07:59:44.318 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:44.305+0000 7f487d4b7700 1 -- 172.21.15.59:0/3909039251 wait complete. 2023-11-24T07:59:44.319 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:44.306+0000 7f487d4b7700 1 Processor -- start 2023-11-24T07:59:44.319 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:44.306+0000 7f487d4b7700 1 -- start start 2023-11-24T07:59:44.319 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:44.307+0000 7f487d4b7700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4860003bc0 0x7f4878156490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:44.319 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:44.307+0000 7f487d4b7700 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f48780cd840 0x7f487813fcf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:44.320 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:44.307+0000 7f487d4b7700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f48780cdcc0 0x7f4878140230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:44.320 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:44.307+0000 7f487d4b7700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4878156070 con 0x7f4860003bc0 2023-11-24T07:59:44.320 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:44.307+0000 7f487d4b7700 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48781569d0 con 0x7f48780cdcc0 2023-11-24T07:59:44.320 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:44.307+0000 7f487d4b7700 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f48781553f0 con 0x7f48780cd840 2023-11-24T07:59:44.321 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:44.307+0000 7f4876ffd700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4860003bc0 0x7f4878156490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:44.321 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:44.307+0000 7f4876ffd700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4860003bc0 0x7f4878156490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:55614/0 (socket says 172.21.15.59:55614) 2023-11-24T07:59:44.321 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:44.307+0000 7f4876ffd700 1 -- 172.21.15.59:0/887399992 learned_addr learned my addr 172.21.15.59:0/887399992 (peer_addr_for_me v2:172.21.15.59:0/0) 2023-11-24T07:59:44.321 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:44.307+0000 7f48767fc700 1 --2- 172.21.15.59:0/887399992 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f48780cd840 0x7f487813fcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:44.322 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:44.307+0000 7f48777fe700 1 --2- 172.21.15.59:0/887399992 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f48780cdcc0 0x7f4878140230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:44.322 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:44.307+0000 7f4876ffd700 1 -- 172.21.15.59:0/887399992 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f48780cd840 msgr2=0x7f487813fcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:44.322 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:44.307+0000 7f4876ffd700 1 --2- 172.21.15.59:0/887399992 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f48780cd840 0x7f487813fcf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:44.322 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:44.307+0000 7f4876ffd700 1 -- 172.21.15.59:0/887399992 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f48780cdcc0 msgr2=0x7f4878140230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:44.322 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:44.307+0000 7f4876ffd700 1 --2- 172.21.15.59:0/887399992 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f48780cdcc0 0x7f4878140230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:44.322 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:44.308+0000 7f4876ffd700 1 -- 172.21.15.59:0/887399992 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f487815f470 con 0x7f4860003bc0 2023-11-24T07:59:44.323 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:44.308+0000 7f4876ffd700 1 --2- 172.21.15.59:0/887399992 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4860003bc0 0x7f4878156490 secure :-1 s=READY pgs=988 cs=0 l=1 rev1=1 crypto rx=0x7f48600047a0 tx=0x7f48680047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:59:44.323 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:44.308+0000 7f485f7fe700 1 -- 172.21.15.59:0/887399992 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4868021070 con 0x7f4860003bc0 2023-11-24T07:59:44.323 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:44.308+0000 7f485f7fe700 1 -- 172.21.15.59:0/887399992 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4868009e50 con 0x7f4860003bc0 2023-11-24T07:59:44.323 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:44.308+0000 7f487d4b7700 1 -- 172.21.15.59:0/887399992 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f48781556d0 con 0x7f4860003bc0 2023-11-24T07:59:44.323 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:44.308+0000 7f485f7fe700 1 -- 172.21.15.59:0/887399992 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48680038e0 con 0x7f4860003bc0 2023-11-24T07:59:44.323 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:44.308+0000 7f487d4b7700 1 -- 172.21.15.59:0/887399992 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4878155c90 con 0x7f4860003bc0 2023-11-24T07:59:44.323 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:44.309+0000 7f487d4b7700 1 -- 172.21.15.59:0/887399992 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4878068670 con 0x7f4860003bc0 2023-11-24T07:59:44.324 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:44.309+0000 7f485f7fe700 1 -- 172.21.15.59:0/887399992 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 11) v1 ==== 89741+0+0 (secure 0 0 0) 0x7f4868025070 con 0x7f4860003bc0 2023-11-24T07:59:44.325 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:44.310+0000 7f485f7fe700 1 --2- 172.21.15.59:0/887399992 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f486406b470 0x7f486406d930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:44.325 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:44.310+0000 7f485f7fe700 1 -- 172.21.15.59:0/887399992 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(78..78 src has 1..78) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f486808ae60 con 0x7f4860003bc0 2023-11-24T07:59:44.325 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:44.310+0000 7f48767fc700 1 --2- 172.21.15.59:0/887399992 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f486406b470 0x7f486406d930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:44.325 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:44.310+0000 7f48767fc700 1 --2- 172.21.15.59:0/887399992 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f486406b470 0x7f486406d930 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f486c001b80 tx=0x7f486c001a30 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:59:44.325 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:44.314+0000 7f485f7fe700 1 -- 172.21.15.59:0/887399992 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f4868055a30 con 0x7f4860003bc0 2023-11-24T07:59:44.603 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:44.601+0000 7f487d4b7700 1 -- 172.21.15.59:0/887399992 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_volumes.TestSubvolumeSnapshots.test_subvolume_inherited_snapshot_ls"]} v 0) v1 -- 0x7f48780670b0 con 0x7f4860003bc0 2023-11-24T07:59:45.459 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:45.457+0000 7f485f7fe700 1 -- 172.21.15.59:0/887399992 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_volumes.TestSubvolumeSnapshots.test_subvolume_inherited_snapshot_ls"]}]=0 v207) v1 ==== 167+0+0 (secure 0 0 0) 0x7f486802c0e0 con 0x7f4860003bc0 2023-11-24T07:59:45.462 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:45.460+0000 7f487d4b7700 1 -- 172.21.15.59:0/887399992 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f486406b470 msgr2=0x7f486406d930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:45.462 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:45.460+0000 7f487d4b7700 1 --2- 172.21.15.59:0/887399992 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f486406b470 0x7f486406d930 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f486c001b80 tx=0x7f486c001a30 comp rx=0 tx=0).stop 2023-11-24T07:59:45.463 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:45.460+0000 7f487d4b7700 1 -- 172.21.15.59:0/887399992 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4860003bc0 msgr2=0x7f4878156490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:45.463 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:45.460+0000 7f487d4b7700 1 --2- 172.21.15.59:0/887399992 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4860003bc0 0x7f4878156490 secure :-1 s=READY pgs=988 cs=0 l=1 rev1=1 crypto rx=0x7f48600047a0 tx=0x7f48680047c0 comp rx=0 tx=0).stop 2023-11-24T07:59:45.463 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:45.460+0000 7f487d4b7700 1 -- 172.21.15.59:0/887399992 shutdown_connections 2023-11-24T07:59:45.463 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:45.460+0000 7f487d4b7700 1 --2- 172.21.15.59:0/887399992 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f486406b470 0x7f486406d930 unknown :-1 s=CLOSED pgs=456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:45.463 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:45.460+0000 7f487d4b7700 1 --2- 172.21.15.59:0/887399992 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f48780cdcc0 0x7f4878140230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:45.463 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:45.460+0000 7f487d4b7700 1 --2- 172.21.15.59:0/887399992 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4860003bc0 0x7f4878156490 unknown :-1 s=CLOSED pgs=988 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:45.463 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:45.460+0000 7f487d4b7700 1 --2- 172.21.15.59:0/887399992 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f48780cd840 0x7f487813fcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:45.463 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:45.460+0000 7f487d4b7700 1 -- 172.21.15.59:0/887399992 >> 172.21.15.59:0/887399992 conn(0x7f48780d0bc0 msgr2=0x7f487806dd20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:59:45.463 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:45.460+0000 7f487d4b7700 1 -- 172.21.15.59:0/887399992 shutdown_connections 2023-11-24T07:59:45.463 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:45.460+0000 7f487d4b7700 1 -- 172.21.15.59:0/887399992 wait complete. 2023-11-24T07:59:45.477 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2023-11-24T07:59:45.612 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:45.611+0000 7f138be74700 1 Processor -- start 2023-11-24T07:59:45.612 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:45.611+0000 7f138be74700 1 -- start start 2023-11-24T07:59:45.614 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:45.612+0000 7f138be74700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f138406b620 0x7f13840b3a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:45.614 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:45.612+0000 7f138be74700 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f13841629c0 con 0x7f13840b5ab0 2023-11-24T07:59:45.614 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:45.612+0000 7f138be74700 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1384071f50 con 0x7f138406ee60 2023-11-24T07:59:45.614 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:45.612+0000 7f138be74700 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1384072230 con 0x7f138406b620 2023-11-24T07:59:45.614 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:45.612+0000 7f1389c10700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f138406b620 0x7f13840b3a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:45.615 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:45.612+0000 7f138940f700 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7f13840b5ab0 0x7f1384161ba0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:59098/0 (socket says 172.21.15.59:59098) 2023-11-24T07:59:45.615 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:45.612+0000 7f1389c10700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f138406b620 0x7f13840b3a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:55622/0 (socket says 172.21.15.59:55622) 2023-11-24T07:59:45.615 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:45.612+0000 7f138940f700 1 -- 172.21.15.59:0/4285809222 learned_addr learned my addr 172.21.15.59:0/4285809222 (peer_addr_for_me v1:172.21.15.59:0/0) 2023-11-24T07:59:45.615 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:45.612+0000 7f1389c10700 1 -- 172.21.15.59:0/4285809222 >> v1:172.21.15.76:6789/0 conn(0x7f138406ee60 legacy=0x7f13841622b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:45.615 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:45.612+0000 7f1389c10700 1 -- 172.21.15.59:0/4285809222 >> v1:172.21.15.59:6789/0 conn(0x7f13840b5ab0 legacy=0x7f1384161ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:45.615 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:45.612+0000 7f1388c0e700 1 -- 172.21.15.59:0/4285809222 <== mon.0 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 343926006 0 0) 0x7f13841629c0 con 0x7f13840b5ab0 2023-11-24T07:59:45.615 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:45.612+0000 7f1389c10700 1 -- 172.21.15.59:0/4285809222 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1384072510 con 0x7f138406b620 2023-11-24T07:59:45.615 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:45.612+0000 7f1389c10700 1 --2- 172.21.15.59:0/4285809222 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f138406b620 0x7f13840b3a40 secure :-1 s=READY pgs=990 cs=0 l=1 rev1=1 crypto rx=0x7f1380009840 tx=0x7f1380009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=173ed1d9dd8b9083 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:59:45.615 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:45.613+0000 7f1388c0e700 1 -- 172.21.15.59:0/4285809222 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1380011830 con 0x7f138406b620 2023-11-24T07:59:45.615 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:45.613+0000 7f1388c0e700 1 -- 172.21.15.59:0/4285809222 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1380011990 con 0x7f138406b620 2023-11-24T07:59:45.615 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:45.613+0000 7f1388c0e700 1 -- 172.21.15.59:0/4285809222 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1380011d20 con 0x7f138406b620 2023-11-24T07:59:45.617 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:45.613+0000 7f138be74700 1 -- 172.21.15.59:0/4285809222 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f138406b620 msgr2=0x7f13840b3a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:45.617 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:45.613+0000 7f138be74700 1 --2- 172.21.15.59:0/4285809222 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f138406b620 0x7f13840b3a40 secure :-1 s=READY pgs=990 cs=0 l=1 rev1=1 crypto rx=0x7f1380009840 tx=0x7f1380009b50 comp rx=0 tx=0).stop 2023-11-24T07:59:45.617 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:45.613+0000 7f138be74700 1 -- 172.21.15.59:0/4285809222 shutdown_connections 2023-11-24T07:59:45.617 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:45.613+0000 7f138be74700 1 --2- 172.21.15.59:0/4285809222 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f138406b620 0x7f13840b3a40 unknown :-1 s=CLOSED pgs=990 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:45.617 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:45.613+0000 7f138be74700 1 -- 172.21.15.59:0/4285809222 >> 172.21.15.59:0/4285809222 conn(0x7f13840b2740 msgr2=0x7f13840b2b20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:59:45.617 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:45.613+0000 7f138be74700 1 -- 172.21.15.59:0/4285809222 shutdown_connections 2023-11-24T07:59:45.617 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:45.613+0000 7f138be74700 1 -- 172.21.15.59:0/4285809222 wait complete. 2023-11-24T07:59:45.617 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:45.615+0000 7f138be74700 1 Processor -- start 2023-11-24T07:59:45.617 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:45.615+0000 7f138be74700 1 -- start start 2023-11-24T07:59:45.619 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:45.616+0000 7f138be74700 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f138406b620 0x7f138413f670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:45.619 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:45.616+0000 7f138be74700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f138406ee60 0x7f13841f39f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:45.620 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:45.616+0000 7f138be74700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f13840b5ab0 0x7f13841f6f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:45.620 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:45.616+0000 7f138be74700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13841f7e40 con 0x7f138406ee60 2023-11-24T07:59:45.620 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:45.616+0000 7f138be74700 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1384140ee0 con 0x7f13840b5ab0 2023-11-24T07:59:45.620 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:45.616+0000 7f138be74700 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f138413fbb0 con 0x7f138406b620 2023-11-24T07:59:45.620 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:45.616+0000 7f138940f700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f138406ee60 0x7f13841f39f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:45.620 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:45.616+0000 7f138940f700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f138406ee60 0x7f13841f39f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:55628/0 (socket says 172.21.15.59:55628) 2023-11-24T07:59:45.620 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:45.616+0000 7f138940f700 1 -- 172.21.15.59:0/4270251536 learned_addr learned my addr 172.21.15.59:0/4270251536 (peer_addr_for_me v2:172.21.15.59:0/0) 2023-11-24T07:59:45.620 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:45.616+0000 7f1389c10700 1 --2- 172.21.15.59:0/4270251536 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f138406b620 0x7f138413f670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:45.620 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:45.616+0000 7f138a411700 1 --2- 172.21.15.59:0/4270251536 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f13840b5ab0 0x7f13841f6f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:45.620 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:45.616+0000 7f138940f700 1 -- 172.21.15.59:0/4270251536 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f138406b620 msgr2=0x7f138413f670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:45.620 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:45.616+0000 7f138940f700 1 --2- 172.21.15.59:0/4270251536 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f138406b620 0x7f138413f670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:45.621 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:45.616+0000 7f138940f700 1 -- 172.21.15.59:0/4270251536 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f13840b5ab0 msgr2=0x7f13841f6f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:45.621 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:45.616+0000 7f138940f700 1 --2- 172.21.15.59:0/4270251536 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f13840b5ab0 0x7f13841f6f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:45.621 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:45.616+0000 7f138940f700 1 -- 172.21.15.59:0/4270251536 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f13800094a0 con 0x7f138406ee60 2023-11-24T07:59:45.621 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:45.616+0000 7f138940f700 1 --2- 172.21.15.59:0/4270251536 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f138406ee60 0x7f13841f39f0 secure :-1 s=READY pgs=991 cs=0 l=1 rev1=1 crypto rx=0x7f1374001eb0 tx=0x7f137400c800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:59:45.621 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:45.617+0000 7f137a7fc700 1 -- 172.21.15.59:0/4270251536 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1374004c80 con 0x7f138406ee60 2023-11-24T07:59:45.621 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:45.617+0000 7f137a7fc700 1 -- 172.21.15.59:0/4270251536 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1374004de0 con 0x7f138406ee60 2023-11-24T07:59:45.621 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:45.617+0000 7f138be74700 1 -- 172.21.15.59:0/4270251536 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f13841402d0 con 0x7f138406ee60 2023-11-24T07:59:45.621 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:45.617+0000 7f137a7fc700 1 -- 172.21.15.59:0/4270251536 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1374007310 con 0x7f138406ee60 2023-11-24T07:59:45.621 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:45.617+0000 7f138be74700 1 -- 172.21.15.59:0/4270251536 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1384140890 con 0x7f138406ee60 2023-11-24T07:59:45.621 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:45.617+0000 7f138be74700 1 -- 172.21.15.59:0/4270251536 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1384141090 con 0x7f138406ee60 2023-11-24T07:59:45.621 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:45.618+0000 7f137a7fc700 1 -- 172.21.15.59:0/4270251536 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 11) v1 ==== 89741+0+0 (secure 0 0 0) 0x7f1374007470 con 0x7f138406ee60 2023-11-24T07:59:45.622 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:45.618+0000 7f137a7fc700 1 --2- 172.21.15.59:0/4270251536 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f136806b470 0x7f136806d930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:45.622 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:45.618+0000 7f137a7fc700 1 -- 172.21.15.59:0/4270251536 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(78..78 src has 1..78) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f1374089850 con 0x7f138406ee60 2023-11-24T07:59:45.622 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:45.619+0000 7f1389c10700 1 --2- 172.21.15.59:0/4270251536 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f136806b470 0x7f136806d930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:45.622 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:45.619+0000 7f1389c10700 1 --2- 172.21.15.59:0/4270251536 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f136806b470 0x7f136806d930 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f13800090b0 tx=0x7f138001c040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:59:45.624 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:45.623+0000 7f137a7fc700 1 -- 172.21.15.59:0/4270251536 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f1384141090 con 0x7f138406ee60 2023-11-24T07:59:45.827 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:45.826+0000 7f138be74700 1 -- 172.21.15.59:0/4270251536 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7f1384141050 con 0x7f138406ee60 2023-11-24T07:59:45.833 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:45.831+0000 7f137a7fc700 1 -- 172.21.15.59:0/4270251536 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v18)=0 v18) v1 ==== 122+0+0 (secure 0 0 0) 0x7f1374054420 con 0x7f138406ee60 2023-11-24T07:59:45.835 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:45.833+0000 7f138be74700 1 -- 172.21.15.59:0/4270251536 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f136806b470 msgr2=0x7f136806d930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:45.835 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:45.833+0000 7f138be74700 1 --2- 172.21.15.59:0/4270251536 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f136806b470 0x7f136806d930 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f13800090b0 tx=0x7f138001c040 comp rx=0 tx=0).stop 2023-11-24T07:59:45.836 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:45.833+0000 7f138be74700 1 -- 172.21.15.59:0/4270251536 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f138406ee60 msgr2=0x7f13841f39f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:45.836 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:45.833+0000 7f138be74700 1 --2- 172.21.15.59:0/4270251536 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f138406ee60 0x7f13841f39f0 secure :-1 s=READY pgs=991 cs=0 l=1 rev1=1 crypto rx=0x7f1374001eb0 tx=0x7f137400c800 comp rx=0 tx=0).stop 2023-11-24T07:59:45.836 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:45.833+0000 7f138be74700 1 -- 172.21.15.59:0/4270251536 shutdown_connections 2023-11-24T07:59:45.836 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:45.833+0000 7f138be74700 1 --2- 172.21.15.59:0/4270251536 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f136806b470 0x7f136806d930 unknown :-1 s=CLOSED pgs=457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:45.836 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:45.833+0000 7f138be74700 1 --2- 172.21.15.59:0/4270251536 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f13840b5ab0 0x7f13841f6f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:45.836 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:45.833+0000 7f138be74700 1 --2- 172.21.15.59:0/4270251536 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f138406b620 0x7f138413f670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:45.836 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:45.833+0000 7f138be74700 1 --2- 172.21.15.59:0/4270251536 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f138406ee60 0x7f13841f39f0 unknown :-1 s=CLOSED pgs=991 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:45.836 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:45.833+0000 7f138be74700 1 -- 172.21.15.59:0/4270251536 >> 172.21.15.59:0/4270251536 conn(0x7f13840b2740 msgr2=0x7f13840b2d30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:59:45.836 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:45.833+0000 7f138be74700 1 -- 172.21.15.59:0/4270251536 shutdown_connections 2023-11-24T07:59:45.836 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:45.833+0000 7f138be74700 1 -- 172.21.15.59:0/4270251536 wait complete. 2023-11-24T07:59:45.847 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-11-24T07:59:45.847 DEBUG:teuthology.orchestra.run.smithi059:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-11-24T07:59:45.893 DEBUG:teuthology.orchestra.run.smithi076:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-11-24T07:59:45.944 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-11-24T07:59:45.944 DEBUG:teuthology.orchestra.run.smithi059:> set -ex 2023-11-24T07:59:45.944 DEBUG:teuthology.orchestra.run.smithi059:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-24T07:59:45.962 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=16287364k,nr_inodes=4071841,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 tmpfs /sys/fs/cgroup tmpfs ro,seclabel,nosuid,nodev,noexec,mode=755 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,seclabel,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,seclabel,nosuid,nodev,noexec,relatime,hugetlb 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,seclabel,nosuid,nodev,noexec,relatime,perf_event 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,seclabel,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/rdma cgroup rw,seclabel,nosuid,nodev,noexec,relatime,rdma 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,seclabel,nosuid,nodev,noexec,relatime,freezer 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,seclabel,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,seclabel,nosuid,nodev,noexec,relatime,devices 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,blkio 0 0 none /sys/kernel/tracing tracefs rw,seclabel,relatime 0 0 configfs /sys/kernel/config configfs rw,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=42,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=16075 0 0 mqueue /dev/mqueue mqueue rw,seclabel,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,relatime 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3261172k,mode=700,uid=1000,gid=1265 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw,seclabel 0 0 2023-11-24T07:59:45.962 DEBUG:teuthology.orchestra.run.smithi076:> set -ex 2023-11-24T07:59:45.962 DEBUG:teuthology.orchestra.run.smithi076:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-24T07:59:45.982 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=16284272k,nr_inodes=4071068,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 tmpfs /sys/fs/cgroup tmpfs ro,seclabel,nosuid,nodev,noexec,mode=755 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,seclabel,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,seclabel,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,seclabel,nosuid,nodev,noexec,relatime,freezer 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,seclabel,nosuid,nodev,noexec,relatime,perf_event 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,seclabel,nosuid,nodev,noexec,relatime,hugetlb 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,seclabel,nosuid,nodev,noexec,relatime,devices 0 0 cgroup /sys/fs/cgroup/rdma cgroup rw,seclabel,nosuid,nodev,noexec,relatime,rdma 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,seclabel,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,blkio 0 0 none /sys/kernel/tracing tracefs rw,seclabel,relatime 0 0 configfs /sys/kernel/config configfs rw,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=35,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=20116 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,relatime 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3260552k,mode=700,uid=1000,gid=1265 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw,seclabel 0 0 2023-11-24T07:59:45.982 DEBUG:teuthology.orchestra.run.smithi059:> set -ex 2023-11-24T07:59:45.982 DEBUG:teuthology.orchestra.run.smithi059:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-24T07:59:46.019 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=16287364k,nr_inodes=4071841,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 tmpfs /sys/fs/cgroup tmpfs ro,seclabel,nosuid,nodev,noexec,mode=755 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,seclabel,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,seclabel,nosuid,nodev,noexec,relatime,hugetlb 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,seclabel,nosuid,nodev,noexec,relatime,perf_event 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,seclabel,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/rdma cgroup rw,seclabel,nosuid,nodev,noexec,relatime,rdma 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,seclabel,nosuid,nodev,noexec,relatime,freezer 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,seclabel,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,seclabel,nosuid,nodev,noexec,relatime,devices 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,blkio 0 0 none /sys/kernel/tracing tracefs rw,seclabel,relatime 0 0 configfs /sys/kernel/config configfs rw,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=42,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=16075 0 0 mqueue /dev/mqueue mqueue rw,seclabel,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,relatime 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3261172k,mode=700,uid=1000,gid=1265 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw,seclabel 0 0 2023-11-24T07:59:46.020 DEBUG:teuthology.orchestra.run.smithi076:> set -ex 2023-11-24T07:59:46.020 DEBUG:teuthology.orchestra.run.smithi076:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-24T07:59:46.043 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=16284272k,nr_inodes=4071068,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 tmpfs /sys/fs/cgroup tmpfs ro,seclabel,nosuid,nodev,noexec,mode=755 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,seclabel,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,seclabel,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,seclabel,nosuid,nodev,noexec,relatime,freezer 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,seclabel,nosuid,nodev,noexec,relatime,perf_event 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,seclabel,nosuid,nodev,noexec,relatime,hugetlb 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,seclabel,nosuid,nodev,noexec,relatime,devices 0 0 cgroup /sys/fs/cgroup/rdma cgroup rw,seclabel,nosuid,nodev,noexec,relatime,rdma 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,seclabel,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,blkio 0 0 none /sys/kernel/tracing tracefs rw,seclabel,relatime 0 0 configfs /sys/kernel/config configfs rw,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=35,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=20116 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,relatime 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3260552k,mode=700,uid=1000,gid=1265 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw,seclabel 0 0 2023-11-24T07:59:46.043 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2023-11-24T07:59:46.043 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-24T07:59:46.184 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:46.183+0000 7f6d9142d700 1 Processor -- start 2023-11-24T07:59:46.185 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:46.183+0000 7f6d9142d700 1 -- start start 2023-11-24T07:59:46.187 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:46.183+0000 7f6d9142d700 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f6d8c15bde0 0x7f6d8c06ece0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:46.187 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:46.183+0000 7f6d9142d700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f6d8c15c1c0 0x7f6d8c0cd0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:46.187 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:46.184+0000 7f6d9142d700 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6d8c0cc670 con 0x7f6d8c0b4240 2023-11-24T07:59:46.187 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:46.184+0000 7f6d9142d700 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6d8c0cffb0 con 0x7f6d8c15c1c0 2023-11-24T07:59:46.187 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:46.184+0000 7f6d9142d700 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6d8c0cf7a0 con 0x7f6d8c15bde0 2023-11-24T07:59:46.188 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:46.184+0000 7f6d8a7fc700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f6d8c15c1c0 0x7f6d8c0cd0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:46.188 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:46.184+0000 7f6d8b7fe700 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7f6d8c0b4240 0x7f6d8c0ce430 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:59112/0 (socket says 172.21.15.59:59112) 2023-11-24T07:59:46.188 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:46.184+0000 7f6d8b7fe700 1 -- 172.21.15.59:0/3289035920 learned_addr learned my addr 172.21.15.59:0/3289035920 (peer_addr_for_me v1:172.21.15.59:0/0) 2023-11-24T07:59:46.188 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:46.184+0000 7f6d8affd700 1 --2- 172.21.15.59:0/3289035920 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f6d8c15bde0 0x7f6d8c06ece0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:46.188 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:46.184+0000 7f6d8a7fc700 1 -- 172.21.15.59:0/3289035920 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f6d8c15bde0 msgr2=0x7f6d8c06ece0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T07:59:46.188 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:46.184+0000 7f6d8a7fc700 1 --2- 172.21.15.59:0/3289035920 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f6d8c15bde0 0x7f6d8c06ece0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:46.188 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:46.184+0000 7f6d8a7fc700 1 -- 172.21.15.59:0/3289035920 >> v1:172.21.15.59:6789/0 conn(0x7f6d8c0b4240 legacy=0x7f6d8c0ce430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:46.188 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:46.184+0000 7f6d89ffb700 1 -- 172.21.15.59:0/3289035920 <== mon.0 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1044915071 0 0) 0x7f6d8c0cc670 con 0x7f6d8c0b4240 2023-11-24T07:59:46.188 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:46.184+0000 7f6d8a7fc700 1 -- 172.21.15.59:0/3289035920 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6d8c0cf9f0 con 0x7f6d8c15c1c0 2023-11-24T07:59:46.188 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:46.184+0000 7f6d8a7fc700 1 --2- 172.21.15.59:0/3289035920 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6d8c15c1c0 0x7f6d8c0cd0a0 secure :-1 s=READY pgs=993 cs=0 l=1 rev1=1 crypto rx=0x7f6d80009840 tx=0x7f6d80009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=916168eadedaf599 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:59:46.189 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:46.184+0000 7f6d89ffb700 1 -- 172.21.15.59:0/3289035920 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d80011830 con 0x7f6d8c15c1c0 2023-11-24T07:59:46.189 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:46.184+0000 7f6d89ffb700 1 -- 172.21.15.59:0/3289035920 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6d80011990 con 0x7f6d8c15c1c0 2023-11-24T07:59:46.189 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:46.184+0000 7f6d89ffb700 1 -- 172.21.15.59:0/3289035920 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d80011d20 con 0x7f6d8c15c1c0 2023-11-24T07:59:46.189 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:46.185+0000 7f6d9142d700 1 -- 172.21.15.59:0/3289035920 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6d8c15c1c0 msgr2=0x7f6d8c0cd0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:46.189 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:46.185+0000 7f6d9142d700 1 --2- 172.21.15.59:0/3289035920 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6d8c15c1c0 0x7f6d8c0cd0a0 secure :-1 s=READY pgs=993 cs=0 l=1 rev1=1 crypto rx=0x7f6d80009840 tx=0x7f6d80009b50 comp rx=0 tx=0).stop 2023-11-24T07:59:46.189 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:46.185+0000 7f6d9142d700 1 -- 172.21.15.59:0/3289035920 shutdown_connections 2023-11-24T07:59:46.189 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:46.185+0000 7f6d9142d700 1 --2- 172.21.15.59:0/3289035920 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f6d8c15bde0 0x7f6d8c06ece0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:46.189 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:46.185+0000 7f6d9142d700 1 --2- 172.21.15.59:0/3289035920 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6d8c15c1c0 0x7f6d8c0cd0a0 unknown :-1 s=CLOSED pgs=993 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:46.189 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:46.185+0000 7f6d9142d700 1 -- 172.21.15.59:0/3289035920 >> 172.21.15.59:0/3289035920 conn(0x7f6d8c0b3720 msgr2=0x7f6d8c0b3b30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:59:46.189 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:46.185+0000 7f6d9142d700 1 -- 172.21.15.59:0/3289035920 shutdown_connections 2023-11-24T07:59:46.190 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:46.185+0000 7f6d9142d700 1 -- 172.21.15.59:0/3289035920 wait complete. 2023-11-24T07:59:46.192 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:46.186+0000 7f6d9142d700 1 Processor -- start 2023-11-24T07:59:46.192 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:46.186+0000 7f6d9142d700 1 -- start start 2023-11-24T07:59:46.192 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:46.187+0000 7f6d9142d700 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f6d8c0b4240 0x7f6d8c1f3340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:46.193 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:46.187+0000 7f6d9142d700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6d8c15bde0 0x7f6d8c1f6e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:46.193 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:46.187+0000 7f6d9142d700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6d8c06cc20 0x7f6d8c0b21f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:46.193 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:46.187+0000 7f6d9142d700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d8c15c7e0 con 0x7f6d8c15bde0 2023-11-24T07:59:46.193 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:46.187+0000 7f6d9142d700 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d8c1f7dc0 con 0x7f6d8c06cc20 2023-11-24T07:59:46.193 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:46.187+0000 7f6d9142d700 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6d8c1f7520 con 0x7f6d8c0b4240 2023-11-24T07:59:46.193 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:46.187+0000 7f6d8a7fc700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6d8c15bde0 0x7f6d8c1f6e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:46.193 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:46.187+0000 7f6d8a7fc700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6d8c15bde0 0x7f6d8c1f6e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:55642/0 (socket says 172.21.15.59:55642) 2023-11-24T07:59:46.193 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:46.187+0000 7f6d8a7fc700 1 -- 172.21.15.59:0/3669099721 learned_addr learned my addr 172.21.15.59:0/3669099721 (peer_addr_for_me v2:172.21.15.59:0/0) 2023-11-24T07:59:46.193 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:46.187+0000 7f6d8affd700 1 --2- 172.21.15.59:0/3669099721 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f6d8c0b4240 0x7f6d8c1f3340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:46.193 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:46.187+0000 7f6d8b7fe700 1 --2- 172.21.15.59:0/3669099721 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6d8c06cc20 0x7f6d8c0b21f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:46.194 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:46.187+0000 7f6d8a7fc700 1 -- 172.21.15.59:0/3669099721 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f6d8c0b4240 msgr2=0x7f6d8c1f3340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:46.194 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:46.187+0000 7f6d8a7fc700 1 --2- 172.21.15.59:0/3669099721 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f6d8c0b4240 0x7f6d8c1f3340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:46.194 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:46.187+0000 7f6d8a7fc700 1 -- 172.21.15.59:0/3669099721 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6d8c06cc20 msgr2=0x7f6d8c0b21f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:46.194 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:46.187+0000 7f6d8a7fc700 1 --2- 172.21.15.59:0/3669099721 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6d8c06cc20 0x7f6d8c0b21f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:46.194 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:46.187+0000 7f6d8a7fc700 1 -- 172.21.15.59:0/3669099721 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6d800094a0 con 0x7f6d8c15bde0 2023-11-24T07:59:46.194 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:46.188+0000 7f6d8a7fc700 1 --2- 172.21.15.59:0/3669099721 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6d8c15bde0 0x7f6d8c1f6e90 secure :-1 s=READY pgs=994 cs=0 l=1 rev1=1 crypto rx=0x7f6d80009b90 tx=0x7f6d80004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:59:46.194 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:46.188+0000 7f6d737fe700 1 -- 172.21.15.59:0/3669099721 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d800036a0 con 0x7f6d8c15bde0 2023-11-24T07:59:46.194 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:46.188+0000 7f6d737fe700 1 -- 172.21.15.59:0/3669099721 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6d8001f070 con 0x7f6d8c15bde0 2023-11-24T07:59:46.194 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:46.188+0000 7f6d9142d700 1 -- 172.21.15.59:0/3669099721 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6d8c1f7800 con 0x7f6d8c15bde0 2023-11-24T07:59:46.195 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:46.188+0000 7f6d737fe700 1 -- 172.21.15.59:0/3669099721 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d80003a40 con 0x7f6d8c15bde0 2023-11-24T07:59:46.195 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:46.188+0000 7f6d9142d700 1 -- 172.21.15.59:0/3669099721 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6d8c06d0b0 con 0x7f6d8c15bde0 2023-11-24T07:59:46.195 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:46.189+0000 7f6d9142d700 1 -- 172.21.15.59:0/3669099721 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6d8c068670 con 0x7f6d8c15bde0 2023-11-24T07:59:46.195 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:46.189+0000 7f6d737fe700 1 -- 172.21.15.59:0/3669099721 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 11) v1 ==== 89741+0+0 (secure 0 0 0) 0x7f6d80003ba0 con 0x7f6d8c15bde0 2023-11-24T07:59:46.195 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:46.190+0000 7f6d737fe700 1 --2- 172.21.15.59:0/3669099721 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f6d7806b470 0x7f6d7806d930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:46.195 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:46.190+0000 7f6d737fe700 1 -- 172.21.15.59:0/3669099721 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(78..78 src has 1..78) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f6d80021070 con 0x7f6d8c15bde0 2023-11-24T07:59:46.195 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:46.190+0000 7f6d8affd700 1 --2- 172.21.15.59:0/3669099721 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f6d7806b470 0x7f6d7806d930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:46.195 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:46.191+0000 7f6d8affd700 1 --2- 172.21.15.59:0/3669099721 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f6d7806b470 0x7f6d7806d930 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f6d7c009950 tx=0x7f6d7c008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:59:46.196 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:46.195+0000 7f6d737fe700 1 -- 172.21.15.59:0/3669099721 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f6d800562d0 con 0x7f6d8c15bde0 2023-11-24T07:59:46.444 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:46.443+0000 7f6d9142d700 1 -- 172.21.15.59:0/3669099721 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f6d8c0b7b50 con 0x7f6d8c15bde0 2023-11-24T07:59:46.445 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:46.443+0000 7f6d737fe700 1 -- 172.21.15.59:0/3669099721 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 47 v47) v1 ==== 94+0+3696 (secure 0 0 0) 0x7f6d800598f0 con 0x7f6d8c15bde0 2023-11-24T07:59:46.445 INFO:teuthology.orchestra.run.smithi059.stdout: 2023-11-24T07:59:46.445 INFO:teuthology.orchestra.run.smithi059.stdout:{"epoch":47,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5585,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.76:6837/2282835612","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6835","nonce":2282835612},{"type":"v1","addr":"172.21.15.76:6837","nonce":2282835612}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":39},{"gid":6097,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.76:6836/3924452718","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6834","nonce":3924452718},{"type":"v1","addr":"172.21.15.76:6836","nonce":3924452718}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":39},{"gid":6549,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6834/1425120389","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6832","nonce":1425120389},{"type":"v1","addr":"172.21.15.59:6834","nonce":1425120389}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":39},{"gid":6957,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6835/403612308","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6833","nonce":403612308},{"type":"v1","addr":"172.21.15.59:6835","nonce":403612308}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":47}],"filesystems":[]} 2023-11-24T07:59:46.447 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:46.445+0000 7f6d9142d700 1 -- 172.21.15.59:0/3669099721 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f6d7806b470 msgr2=0x7f6d7806d930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:46.447 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:46.445+0000 7f6d9142d700 1 --2- 172.21.15.59:0/3669099721 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f6d7806b470 0x7f6d7806d930 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f6d7c009950 tx=0x7f6d7c008040 comp rx=0 tx=0).stop 2023-11-24T07:59:46.447 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:46.445+0000 7f6d9142d700 1 -- 172.21.15.59:0/3669099721 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6d8c15bde0 msgr2=0x7f6d8c1f6e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:46.447 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:46.445+0000 7f6d9142d700 1 --2- 172.21.15.59:0/3669099721 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6d8c15bde0 0x7f6d8c1f6e90 secure :-1 s=READY pgs=994 cs=0 l=1 rev1=1 crypto rx=0x7f6d80009b90 tx=0x7f6d80004750 comp rx=0 tx=0).stop 2023-11-24T07:59:46.447 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:46.445+0000 7f6d9142d700 1 -- 172.21.15.59:0/3669099721 shutdown_connections 2023-11-24T07:59:46.447 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:46.445+0000 7f6d9142d700 1 --2- 172.21.15.59:0/3669099721 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f6d7806b470 0x7f6d7806d930 unknown :-1 s=CLOSED pgs=458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:46.447 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:46.445+0000 7f6d9142d700 1 --2- 172.21.15.59:0/3669099721 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6d8c06cc20 0x7f6d8c0b21f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:46.448 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:46.445+0000 7f6d9142d700 1 --2- 172.21.15.59:0/3669099721 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f6d8c0b4240 0x7f6d8c1f3340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:46.448 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:46.445+0000 7f6d9142d700 1 --2- 172.21.15.59:0/3669099721 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6d8c15bde0 0x7f6d8c1f6e90 unknown :-1 s=CLOSED pgs=994 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:46.448 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:46.445+0000 7f6d9142d700 1 -- 172.21.15.59:0/3669099721 >> 172.21.15.59:0/3669099721 conn(0x7f6d8c0b3720 msgr2=0x7f6d8c0d2560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:59:46.448 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:46.445+0000 7f6d9142d700 1 -- 172.21.15.59:0/3669099721 shutdown_connections 2023-11-24T07:59:46.448 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:46.445+0000 7f6d9142d700 1 -- 172.21.15.59:0/3669099721 wait complete. 2023-11-24T07:59:46.448 INFO:teuthology.orchestra.run.smithi059.stderr:dumped fsmap epoch 47 2023-11-24T07:59:46.459 INFO:tasks.ceph.mds.a:Restarting daemon 2023-11-24T07:59:46.459 INFO:tasks.ceph.mds.a:Stopping old one... 2023-11-24T07:59:46.459 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2023-11-24T07:59:46.460 INFO:teuthology.orchestra.run:waiting for 300 2023-11-24T07:59:46.460 INFO:tasks.ceph.mds.c:Restarting daemon 2023-11-24T07:59:46.460 INFO:tasks.ceph.mds.c:Stopping old one... 2023-11-24T07:59:46.460 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2023-11-24T07:59:46.460 INFO:teuthology.orchestra.run:waiting for 300 2023-11-24T07:59:46.460 INFO:tasks.ceph.mds.b:Restarting daemon 2023-11-24T07:59:46.461 INFO:tasks.ceph.mds.b:Stopping old one... 2023-11-24T07:59:46.461 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2023-11-24T07:59:46.461 INFO:teuthology.orchestra.run:waiting for 300 2023-11-24T07:59:46.461 INFO:tasks.ceph.mds.d:Restarting daemon 2023-11-24T07:59:46.461 INFO:tasks.ceph.mds.d:Stopping old one... 2023-11-24T07:59:46.461 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2023-11-24T07:59:46.461 INFO:teuthology.orchestra.run:waiting for 300 2023-11-24T07:59:46.471 INFO:tasks.ceph.mds.a:Stopped 2023-11-24T07:59:46.471 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2023-11-24T07:59:46.472 INFO:tasks.ceph.mds.c:Stopped 2023-11-24T07:59:46.472 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2023-11-24T07:59:46.473 INFO:tasks.ceph.mds.b:Stopped 2023-11-24T07:59:46.473 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2023-11-24T07:59:46.474 INFO:tasks.ceph.mds.d:Stopped 2023-11-24T07:59:46.474 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2023-11-24T07:59:46.514 INFO:tasks.ceph.mds.a:Started 2023-11-24T07:59:46.514 DEBUG:teuthology.parallel:result is None 2023-11-24T07:59:46.515 INFO:tasks.ceph.mds.c:Started 2023-11-24T07:59:46.515 DEBUG:teuthology.parallel:result is None 2023-11-24T07:59:46.516 INFO:tasks.ceph.mds.b:Started 2023-11-24T07:59:46.516 DEBUG:teuthology.parallel:result is None 2023-11-24T07:59:46.517 INFO:tasks.ceph.mds.d:Started 2023-11-24T07:59:46.517 DEBUG:teuthology.parallel:result is None 2023-11-24T07:59:46.518 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2023-11-24T07:59:46.596 INFO:tasks.ceph.mds.a.smithi059.stdout:starting mds.a at 2023-11-24T07:59:46.596 INFO:tasks.ceph.mds.c.smithi059.stdout:starting mds.c at 2023-11-24T07:59:46.601 INFO:tasks.ceph.mds.d.smithi076.stdout:starting mds.d at 2023-11-24T07:59:46.602 INFO:tasks.ceph.mds.b.smithi076.stdout:starting mds.b at 2023-11-24T07:59:46.651 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:46.649+0000 7f9388ebc700 1 Processor -- start 2023-11-24T07:59:46.651 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:46.649+0000 7f9388ebc700 1 -- start start 2023-11-24T07:59:46.654 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:46.650+0000 7f9388ebc700 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f93841448b0 0x7f938406ec70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:46.654 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:46.650+0000 7f9388ebc700 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f938406f990 0x7f93840ce2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:46.654 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:46.650+0000 7f9388ebc700 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9384144c90 con 0x7f93840b44c0 2023-11-24T07:59:46.654 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:46.650+0000 7f9388ebc700 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f93840b4c80 con 0x7f938406f990 2023-11-24T07:59:46.655 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:46.650+0000 7f9388ebc700 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f93840b4f60 con 0x7f93841448b0 2023-11-24T07:59:46.655 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:46.650+0000 7f9381d9b700 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7f93840b44c0 0x7f938406f280 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:59148/0 (socket says 172.21.15.59:59148) 2023-11-24T07:59:46.655 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:46.650+0000 7f9381d9b700 1 -- 172.21.15.59:0/2403492728 learned_addr learned my addr 172.21.15.59:0/2403492728 (peer_addr_for_me v1:172.21.15.59:0/0) 2023-11-24T07:59:46.655 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:46.650+0000 7f9382d9d700 1 --2- 172.21.15.59:0/2403492728 >> v2:172.21.15.76:3300/0 conn(0x7f938406f990 0x7f93840ce2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:46.656 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:46.650+0000 7f938259c700 1 --2- 172.21.15.59:0/2403492728 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f93841448b0 0x7f938406ec70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:46.656 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:46.650+0000 7f938159a700 1 -- 172.21.15.59:0/2403492728 <== mon.0 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4163277991 0 0) 0x7f9384144c90 con 0x7f93840b44c0 2023-11-24T07:59:46.656 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:46.650+0000 7f9382d9d700 1 -- 172.21.15.59:0/2403492728 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f93841448b0 msgr2=0x7f938406ec70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T07:59:46.656 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:46.650+0000 7f9382d9d700 1 --2- 172.21.15.59:0/2403492728 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f93841448b0 0x7f938406ec70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:46.657 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:46.650+0000 7f9382d9d700 1 -- 172.21.15.59:0/2403492728 >> v1:172.21.15.59:6789/0 conn(0x7f93840b44c0 legacy=0x7f938406f280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:46.657 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:46.650+0000 7f9382d9d700 1 -- 172.21.15.59:0/2403492728 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f93840b5240 con 0x7f938406f990 2023-11-24T07:59:46.657 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:46.650+0000 7f938259c700 1 --2- 172.21.15.59:0/2403492728 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f93841448b0 0x7f938406ec70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-24T07:59:46.657 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:46.651+0000 7f9382d9d700 1 --2- 172.21.15.59:0/2403492728 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f938406f990 0x7f93840ce2a0 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f9374009840 tx=0x7f9374009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e1478893b994ae5a server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:59:46.658 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:46.651+0000 7f938159a700 1 -- 172.21.15.59:0/2403492728 <== mon.1 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9374011830 con 0x7f938406f990 2023-11-24T07:59:46.658 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:46.651+0000 7f938159a700 1 -- 172.21.15.59:0/2403492728 <== mon.1 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9374011990 con 0x7f938406f990 2023-11-24T07:59:46.658 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:46.651+0000 7f938159a700 1 -- 172.21.15.59:0/2403492728 <== mon.1 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9374011d20 con 0x7f938406f990 2023-11-24T07:59:46.658 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:46.651+0000 7f9388ebc700 1 -- 172.21.15.59:0/2403492728 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f938406f990 msgr2=0x7f93840ce2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:46.659 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:46.651+0000 7f9388ebc700 1 --2- 172.21.15.59:0/2403492728 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f938406f990 0x7f93840ce2a0 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f9374009840 tx=0x7f9374009b50 comp rx=0 tx=0).stop 2023-11-24T07:59:46.659 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:46.651+0000 7f9388ebc700 1 -- 172.21.15.59:0/2403492728 shutdown_connections 2023-11-24T07:59:46.659 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:46.651+0000 7f9388ebc700 1 --2- 172.21.15.59:0/2403492728 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f938406f990 0x7f93840ce2a0 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:46.659 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:46.651+0000 7f9388ebc700 1 --2- 172.21.15.59:0/2403492728 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f93841448b0 0x7f938406ec70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:46.660 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:46.651+0000 7f9388ebc700 1 -- 172.21.15.59:0/2403492728 >> 172.21.15.59:0/2403492728 conn(0x7f93840d0bc0 msgr2=0x7f93840d0fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:59:46.660 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:46.651+0000 7f9388ebc700 1 -- 172.21.15.59:0/2403492728 shutdown_connections 2023-11-24T07:59:46.660 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:46.651+0000 7f9388ebc700 1 -- 172.21.15.59:0/2403492728 wait complete. 2023-11-24T07:59:46.663 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:46.653+0000 7f9388ebc700 1 Processor -- start 2023-11-24T07:59:46.664 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:46.653+0000 7f9388ebc700 1 -- start start 2023-11-24T07:59:46.664 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:46.653+0000 7f9388ebc700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f938406f990 0x7f938415a5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:46.664 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:46.654+0000 7f9388ebc700 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f93840b44c0 0x7f938406db10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:46.664 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:46.654+0000 7f9388ebc700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f93841448b0 0x7f938406e050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:46.665 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:46.654+0000 7f9388ebc700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9384144000 con 0x7f938406f990 2023-11-24T07:59:46.665 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:46.654+0000 7f9388ebc700 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93840ccff0 con 0x7f93841448b0 2023-11-24T07:59:46.665 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:46.654+0000 7f9388ebc700 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f93840cca30 con 0x7f93840b44c0 2023-11-24T07:59:46.665 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:46.654+0000 7f938259c700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f938406f990 0x7f938415a5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:46.666 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:46.654+0000 7f938259c700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f938406f990 0x7f938415a5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:55678/0 (socket says 172.21.15.59:55678) 2023-11-24T07:59:46.666 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:46.654+0000 7f938259c700 1 -- 172.21.15.59:0/3388312893 learned_addr learned my addr 172.21.15.59:0/3388312893 (peer_addr_for_me v2:172.21.15.59:0/0) 2023-11-24T07:59:46.666 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:46.654+0000 7f9382d9d700 1 --2- 172.21.15.59:0/3388312893 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f93841448b0 0x7f938406e050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:46.666 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:46.654+0000 7f9381d9b700 1 --2- 172.21.15.59:0/3388312893 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f93840b44c0 0x7f938406db10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:46.667 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:46.654+0000 7f938259c700 1 -- 172.21.15.59:0/3388312893 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f93840b44c0 msgr2=0x7f938406db10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:46.667 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:46.654+0000 7f938259c700 1 --2- 172.21.15.59:0/3388312893 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f93840b44c0 0x7f938406db10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:46.667 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:46.654+0000 7f938259c700 1 -- 172.21.15.59:0/3388312893 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f93841448b0 msgr2=0x7f938406e050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:46.667 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:46.654+0000 7f938259c700 1 --2- 172.21.15.59:0/3388312893 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f93841448b0 0x7f938406e050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:46.668 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:46.654+0000 7f938259c700 1 -- 172.21.15.59:0/3388312893 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f937c009a70 con 0x7f938406f990 2023-11-24T07:59:46.668 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:46.654+0000 7f938259c700 1 --2- 172.21.15.59:0/3388312893 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f938406f990 0x7f938415a5a0 secure :-1 s=READY pgs=1000 cs=0 l=1 rev1=1 crypto rx=0x7f937c00efc0 tx=0x7f937c00c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:59:46.668 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:46.654+0000 7f9372ffd700 1 -- 172.21.15.59:0/3388312893 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f937c00cd10 con 0x7f938406f990 2023-11-24T07:59:46.669 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:46.654+0000 7f9372ffd700 1 -- 172.21.15.59:0/3388312893 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f937c00f040 con 0x7f938406f990 2023-11-24T07:59:46.669 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:46.654+0000 7f9388ebc700 1 -- 172.21.15.59:0/3388312893 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f93740094a0 con 0x7f938406f990 2023-11-24T07:59:46.669 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:46.654+0000 7f9372ffd700 1 -- 172.21.15.59:0/3388312893 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f937c0146a0 con 0x7f938406f990 2023-11-24T07:59:46.670 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:46.655+0000 7f9388ebc700 1 -- 172.21.15.59:0/3388312893 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f93840c6cd0 con 0x7f938406f990 2023-11-24T07:59:46.670 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:46.655+0000 7f9388ebc700 1 -- 172.21.15.59:0/3388312893 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f93840c7320 con 0x7f938406f990 2023-11-24T07:59:46.670 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:46.656+0000 7f9372ffd700 1 -- 172.21.15.59:0/3388312893 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 11) v1 ==== 89741+0+0 (secure 0 0 0) 0x7f937c014800 con 0x7f938406f990 2023-11-24T07:59:46.670 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:46.656+0000 7f9372ffd700 1 --2- 172.21.15.59:0/3388312893 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f936406b470 0x7f936406d930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:46.671 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:46.656+0000 7f9372ffd700 1 -- 172.21.15.59:0/3388312893 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(78..78 src has 1..78) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f937c08a950 con 0x7f938406f990 2023-11-24T07:59:46.671 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:46.657+0000 7f9381d9b700 1 --2- 172.21.15.59:0/3388312893 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f936406b470 0x7f936406d930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:46.671 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:46.657+0000 7f9381d9b700 1 --2- 172.21.15.59:0/3388312893 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f936406b470 0x7f936406d930 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f9384131f60 tx=0x7f9378007de0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:59:46.671 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:46.661+0000 7f9372ffd700 1 -- 172.21.15.59:0/3388312893 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f937c055410 con 0x7f938406f990 2023-11-24T07:59:46.877 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:46.875+0000 7f9388ebc700 1 -- 172.21.15.59:0/3388312893 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7f9384068670 con 0x7f938406f990 2023-11-24T07:59:48.368 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:48.366+0000 7f9372ffd700 1 -- 172.21.15.59:0/3388312893 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v83) v1 ==== 97+0+0 (secure 0 0 0) 0x7f937c058a30 con 0x7f938406f990 2023-11-24T07:59:48.371 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:48.369+0000 7f9388ebc700 1 -- 172.21.15.59:0/3388312893 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f936406b470 msgr2=0x7f936406d930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:48.371 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:48.369+0000 7f9388ebc700 1 --2- 172.21.15.59:0/3388312893 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f936406b470 0x7f936406d930 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f9384131f60 tx=0x7f9378007de0 comp rx=0 tx=0).stop 2023-11-24T07:59:48.371 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:48.369+0000 7f9388ebc700 1 -- 172.21.15.59:0/3388312893 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f938406f990 msgr2=0x7f938415a5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:48.371 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:48.369+0000 7f9388ebc700 1 --2- 172.21.15.59:0/3388312893 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f938406f990 0x7f938415a5a0 secure :-1 s=READY pgs=1000 cs=0 l=1 rev1=1 crypto rx=0x7f937c00efc0 tx=0x7f937c00c3a0 comp rx=0 tx=0).stop 2023-11-24T07:59:48.371 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:48.369+0000 7f9388ebc700 1 -- 172.21.15.59:0/3388312893 shutdown_connections 2023-11-24T07:59:48.371 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:48.369+0000 7f9388ebc700 1 --2- 172.21.15.59:0/3388312893 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f936406b470 0x7f936406d930 unknown :-1 s=CLOSED pgs=459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:48.371 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:48.369+0000 7f9388ebc700 1 --2- 172.21.15.59:0/3388312893 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f93841448b0 0x7f938406e050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:48.371 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:48.369+0000 7f9388ebc700 1 --2- 172.21.15.59:0/3388312893 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f938406f990 0x7f938415a5a0 unknown :-1 s=CLOSED pgs=1000 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:48.371 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:48.369+0000 7f9388ebc700 1 --2- 172.21.15.59:0/3388312893 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f93840b44c0 0x7f938406db10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:48.373 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:48.369+0000 7f9388ebc700 1 -- 172.21.15.59:0/3388312893 >> 172.21.15.59:0/3388312893 conn(0x7f93840d0bc0 msgr2=0x7f93840ceb70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:59:48.374 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:48.369+0000 7f9388ebc700 1 -- 172.21.15.59:0/3388312893 shutdown_connections 2023-11-24T07:59:48.374 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:48.369+0000 7f9388ebc700 1 -- 172.21.15.59:0/3388312893 wait complete. 2023-11-24T07:59:48.374 INFO:teuthology.orchestra.run.smithi059.stderr: removed all blocklist entries 2023-11-24T07:59:48.386 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2023-11-24T07:59:48.517 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:48.515+0000 7f3340276700 1 Processor -- start 2023-11-24T07:59:48.517 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:48.516+0000 7f3340276700 1 -- start start 2023-11-24T07:59:48.519 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:48.516+0000 7f3340276700 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f33380b0e90 0x7f333806eef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:48.519 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:48.516+0000 7f3340276700 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f33380d1230 con 0x7f333806fc50 2023-11-24T07:59:48.519 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:48.516+0000 7f3340276700 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f33380d0c70 con 0x7f333815e1c0 2023-11-24T07:59:48.519 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:48.516+0000 7f3340276700 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f33380d0f50 con 0x7f33380b0e90 2023-11-24T07:59:48.520 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:48.517+0000 7f333d811700 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7f333806fc50 0x7f333806ca10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:59166/0 (socket says 172.21.15.59:59166) 2023-11-24T07:59:48.520 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:48.517+0000 7f333d811700 1 -- 172.21.15.59:0/3610339359 learned_addr learned my addr 172.21.15.59:0/3610339359 (peer_addr_for_me v1:172.21.15.59:0/0) 2023-11-24T07:59:48.520 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:48.517+0000 7f333e813700 1 --2- 172.21.15.59:0/3610339359 >> v2:172.21.15.76:3300/0 conn(0x7f33380b0e90 0x7f333806eef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:48.520 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:48.517+0000 7f333d010700 1 -- 172.21.15.59:0/3610339359 <== mon.0 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2808433873 0 0) 0x7f33380d1230 con 0x7f333806fc50 2023-11-24T07:59:48.520 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:48.517+0000 7f333d010700 1 -- 172.21.15.59:0/3610339359 --> v1:172.21.15.59:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f33240035b0 con 0x7f333806fc50 2023-11-24T07:59:48.520 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:48.517+0000 7f333d010700 1 -- 172.21.15.59:0/3610339359 <== mon.0 v1:172.21.15.59:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1636147979 0 0) 0x7f33240035b0 con 0x7f333806fc50 2023-11-24T07:59:48.520 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:48.517+0000 7f333e813700 1 -- 172.21.15.59:0/3610339359 >> v1:172.21.15.76:6789/0 conn(0x7f333815e1c0 legacy=0x7f333806f540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:48.520 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:48.517+0000 7f333e813700 1 -- 172.21.15.59:0/3610339359 >> v1:172.21.15.59:6789/0 conn(0x7f333806fc50 legacy=0x7f333806ca10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:48.520 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:48.517+0000 7f333e813700 1 -- 172.21.15.59:0/3610339359 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f333806c210 con 0x7f33380b0e90 2023-11-24T07:59:48.520 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:48.517+0000 7f333e813700 1 --2- 172.21.15.59:0/3610339359 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f33380b0e90 0x7f333806eef0 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f3330009d90 tx=0x7f333000b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c9b50f9bf556b67c server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:59:48.520 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:48.518+0000 7f333d010700 1 -- 172.21.15.59:0/3610339359 <== mon.1 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f333000e070 con 0x7f33380b0e90 2023-11-24T07:59:48.520 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:48.518+0000 7f333d010700 1 -- 172.21.15.59:0/3610339359 <== mon.1 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3330003a20 con 0x7f33380b0e90 2023-11-24T07:59:48.520 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:48.518+0000 7f333d010700 1 -- 172.21.15.59:0/3610339359 <== mon.1 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3334001670 con 0x7f33380b0e90 2023-11-24T07:59:48.523 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:48.518+0000 7f3340276700 1 -- 172.21.15.59:0/3610339359 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f33380b0e90 msgr2=0x7f333806eef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:48.523 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:48.518+0000 7f3340276700 1 --2- 172.21.15.59:0/3610339359 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f33380b0e90 0x7f333806eef0 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f3330009d90 tx=0x7f333000b330 comp rx=0 tx=0).stop 2023-11-24T07:59:48.524 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:48.518+0000 7f3340276700 1 -- 172.21.15.59:0/3610339359 shutdown_connections 2023-11-24T07:59:48.524 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:48.518+0000 7f3340276700 1 --2- 172.21.15.59:0/3610339359 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f33380b0e90 0x7f333806eef0 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:48.524 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:48.518+0000 7f3340276700 1 -- 172.21.15.59:0/3610339359 >> 172.21.15.59:0/3610339359 conn(0x7f33380b3720 msgr2=0x7f33380b3b30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:59:48.524 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:48.518+0000 7f3340276700 1 -- 172.21.15.59:0/3610339359 shutdown_connections 2023-11-24T07:59:48.524 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:48.518+0000 7f3340276700 1 -- 172.21.15.59:0/3610339359 wait complete. 2023-11-24T07:59:48.524 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:48.519+0000 7f3340276700 1 Processor -- start 2023-11-24T07:59:48.524 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:48.520+0000 7f3340276700 1 -- start start 2023-11-24T07:59:48.524 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:48.520+0000 7f3340276700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f333806fc50 0x7f3338149160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:48.524 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:48.520+0000 7f3340276700 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f33380b0e90 0x7f33380b1790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:48.524 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:48.520+0000 7f3340276700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f333815e1c0 0x7f33380b1cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:48.524 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:48.520+0000 7f3340276700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f333814a210 con 0x7f333815e1c0 2023-11-24T07:59:48.524 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:48.520+0000 7f3340276700 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f333806f620 con 0x7f333806fc50 2023-11-24T07:59:48.525 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:48.520+0000 7f3340276700 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f33381614a0 con 0x7f33380b0e90 2023-11-24T07:59:48.525 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:48.520+0000 7f333e813700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f333815e1c0 0x7f33380b1cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:48.525 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:48.520+0000 7f333e813700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f333815e1c0 0x7f33380b1cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:55700/0 (socket says 172.21.15.59:55700) 2023-11-24T07:59:48.525 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:48.520+0000 7f333e813700 1 -- 172.21.15.59:0/1912497493 learned_addr learned my addr 172.21.15.59:0/1912497493 (peer_addr_for_me v2:172.21.15.59:0/0) 2023-11-24T07:59:48.525 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:48.521+0000 7f333e012700 1 --2- 172.21.15.59:0/1912497493 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f333806fc50 0x7f3338149160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:48.525 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:48.521+0000 7f333d811700 1 --2- 172.21.15.59:0/1912497493 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f33380b0e90 0x7f33380b1790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:48.525 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:48.521+0000 7f333e813700 1 -- 172.21.15.59:0/1912497493 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f33380b0e90 msgr2=0x7f33380b1790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:48.525 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:48.521+0000 7f333e813700 1 --2- 172.21.15.59:0/1912497493 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f33380b0e90 0x7f33380b1790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:48.525 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:48.521+0000 7f333e813700 1 -- 172.21.15.59:0/1912497493 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f333806fc50 msgr2=0x7f3338149160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:48.525 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:48.521+0000 7f333e813700 1 --2- 172.21.15.59:0/1912497493 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f333806fc50 0x7f3338149160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:48.525 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:48.521+0000 7f333e813700 1 -- 172.21.15.59:0/1912497493 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3330009a70 con 0x7f333815e1c0 2023-11-24T07:59:48.525 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:48.521+0000 7f333e813700 1 --2- 172.21.15.59:0/1912497493 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f333815e1c0 0x7f33380b1cd0 secure :-1 s=READY pgs=1002 cs=0 l=1 rev1=1 crypto rx=0x7f3338072340 tx=0x7f3330004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:59:48.526 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:48.521+0000 7f332effd700 1 -- 172.21.15.59:0/1912497493 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33300046b0 con 0x7f333815e1c0 2023-11-24T07:59:48.526 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:48.521+0000 7f332effd700 1 -- 172.21.15.59:0/1912497493 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3330004830 con 0x7f333815e1c0 2023-11-24T07:59:48.526 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:48.521+0000 7f3340276700 1 -- 172.21.15.59:0/1912497493 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3338149b50 con 0x7f333815e1c0 2023-11-24T07:59:48.526 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:48.521+0000 7f332effd700 1 -- 172.21.15.59:0/1912497493 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f333001d6b0 con 0x7f333815e1c0 2023-11-24T07:59:48.526 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:48.521+0000 7f3340276700 1 -- 172.21.15.59:0/1912497493 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f333814a630 con 0x7f333815e1c0 2023-11-24T07:59:48.526 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:48.522+0000 7f3340276700 1 -- 172.21.15.59:0/1912497493 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3338068670 con 0x7f333815e1c0 2023-11-24T07:59:48.527 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:48.523+0000 7f332effd700 1 -- 172.21.15.59:0/1912497493 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 11) v1 ==== 89741+0+0 (secure 0 0 0) 0x7f333001d810 con 0x7f333815e1c0 2023-11-24T07:59:48.527 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:48.523+0000 7f332effd700 1 --2- 172.21.15.59:0/1912497493 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f331c06b470 0x7f331c06d930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:48.527 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:48.523+0000 7f332effd700 1 -- 172.21.15.59:0/1912497493 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f333008bb70 con 0x7f333815e1c0 2023-11-24T07:59:48.527 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:48.523+0000 7f333e012700 1 --2- 172.21.15.59:0/1912497493 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f331c06b470 0x7f331c06d930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:48.527 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:48.524+0000 7f333e012700 1 --2- 172.21.15.59:0/1912497493 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f331c06b470 0x7f331c06d930 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f3328000f60 tx=0x7f3328009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:59:48.529 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:48.527+0000 7f332effd700 1 -- 172.21.15.59:0/1912497493 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f3330056680 con 0x7f333815e1c0 2023-11-24T07:59:48.812 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:48.810+0000 7f3340276700 1 -- 172.21.15.59:0/1912497493 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7f333814a350 con 0x7f333815e1c0 2023-11-24T07:59:48.812 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:48.811+0000 7f332effd700 1 -- 172.21.15.59:0/1912497493 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v16) v1 ==== 80+0+5304 (secure 0 0 0) 0x7f333814a350 con 0x7f333815e1c0 2023-11-24T07:59:48.813 INFO:teuthology.orchestra.run.smithi059.stdout: 2023-11-24T07:59:48.813 INFO:teuthology.orchestra.run.smithi059.stdout:{ 2023-11-24T07:59:48.813 INFO:teuthology.orchestra.run.smithi059.stdout: "auth_dump": [ 2023-11-24T07:59:48.813 INFO:teuthology.orchestra.run.smithi059.stdout: { 2023-11-24T07:59:48.813 INFO:teuthology.orchestra.run.smithi059.stdout: "entity": "mds.a", 2023-11-24T07:59:48.813 INFO:teuthology.orchestra.run.smithi059.stdout: "key": "AQD0VmBlWGLCKBAASbEQYAkExr8TButtK4KHQQ==", 2023-11-24T07:59:48.813 INFO:teuthology.orchestra.run.smithi059.stdout: "caps": { 2023-11-24T07:59:48.813 INFO:teuthology.orchestra.run.smithi059.stdout: "mds": "allow", 2023-11-24T07:59:48.814 INFO:teuthology.orchestra.run.smithi059.stdout: "mgr": "allow *", 2023-11-24T07:59:48.814 INFO:teuthology.orchestra.run.smithi059.stdout: "mon": "allow *", 2023-11-24T07:59:48.814 INFO:teuthology.orchestra.run.smithi059.stdout: "osd": "allow *" 2023-11-24T07:59:48.814 INFO:teuthology.orchestra.run.smithi059.stdout: } 2023-11-24T07:59:48.814 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-11-24T07:59:48.814 INFO:teuthology.orchestra.run.smithi059.stdout: { 2023-11-24T07:59:48.814 INFO:teuthology.orchestra.run.smithi059.stdout: "entity": "mds.b", 2023-11-24T07:59:48.815 INFO:teuthology.orchestra.run.smithi059.stdout: "key": "AQD0VmBlDIS0NRAAYQ3wTunU7m/Qk1qCM6Mcvg==", 2023-11-24T07:59:48.815 INFO:teuthology.orchestra.run.smithi059.stdout: "caps": { 2023-11-24T07:59:48.815 INFO:teuthology.orchestra.run.smithi059.stdout: "mds": "allow", 2023-11-24T07:59:48.815 INFO:teuthology.orchestra.run.smithi059.stdout: "mgr": "allow *", 2023-11-24T07:59:48.815 INFO:teuthology.orchestra.run.smithi059.stdout: "mon": "allow *", 2023-11-24T07:59:48.815 INFO:teuthology.orchestra.run.smithi059.stdout: "osd": "allow *" 2023-11-24T07:59:48.815 INFO:teuthology.orchestra.run.smithi059.stdout: } 2023-11-24T07:59:48.816 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-11-24T07:59:48.816 INFO:teuthology.orchestra.run.smithi059.stdout: { 2023-11-24T07:59:48.816 INFO:teuthology.orchestra.run.smithi059.stdout: "entity": "mds.c", 2023-11-24T07:59:48.816 INFO:teuthology.orchestra.run.smithi059.stdout: "key": "AQD0VmBlvpsrMBAAr5lh7nb6B1xRfk0Umm7Xig==", 2023-11-24T07:59:48.816 INFO:teuthology.orchestra.run.smithi059.stdout: "caps": { 2023-11-24T07:59:48.816 INFO:teuthology.orchestra.run.smithi059.stdout: "mds": "allow", 2023-11-24T07:59:48.816 INFO:teuthology.orchestra.run.smithi059.stdout: "mgr": "allow *", 2023-11-24T07:59:48.816 INFO:teuthology.orchestra.run.smithi059.stdout: "mon": "allow *", 2023-11-24T07:59:48.816 INFO:teuthology.orchestra.run.smithi059.stdout: "osd": "allow *" 2023-11-24T07:59:48.816 INFO:teuthology.orchestra.run.smithi059.stdout: } 2023-11-24T07:59:48.816 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-11-24T07:59:48.816 INFO:teuthology.orchestra.run.smithi059.stdout: { 2023-11-24T07:59:48.817 INFO:teuthology.orchestra.run.smithi059.stdout: "entity": "mds.d", 2023-11-24T07:59:48.817 INFO:teuthology.orchestra.run.smithi059.stdout: "key": "AQD1VmBlPTG7ARAAvC60Rk2GarnNtRJ6MMRIMw==", 2023-11-24T07:59:48.817 INFO:teuthology.orchestra.run.smithi059.stdout: "caps": { 2023-11-24T07:59:48.817 INFO:teuthology.orchestra.run.smithi059.stdout: "mds": "allow", 2023-11-24T07:59:48.817 INFO:teuthology.orchestra.run.smithi059.stdout: "mgr": "allow *", 2023-11-24T07:59:48.817 INFO:teuthology.orchestra.run.smithi059.stdout: "mon": "allow *", 2023-11-24T07:59:48.817 INFO:teuthology.orchestra.run.smithi059.stdout: "osd": "allow *" 2023-11-24T07:59:48.817 INFO:teuthology.orchestra.run.smithi059.stdout: } 2023-11-24T07:59:48.817 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-11-24T07:59:48.817 INFO:teuthology.orchestra.run.smithi059.stdout: { 2023-11-24T07:59:48.817 INFO:teuthology.orchestra.run.smithi059.stdout: "entity": "osd.0", 2023-11-24T07:59:48.817 INFO:teuthology.orchestra.run.smithi059.stdout: "key": "AQD4VmBlXjdAHRAAjnK2Y/rQhg4a97faNqVuoA==", 2023-11-24T07:59:48.817 INFO:teuthology.orchestra.run.smithi059.stdout: "caps": { 2023-11-24T07:59:48.818 INFO:teuthology.orchestra.run.smithi059.stdout: "mgr": "allow profile osd", 2023-11-24T07:59:48.818 INFO:teuthology.orchestra.run.smithi059.stdout: "mon": "allow profile osd", 2023-11-24T07:59:48.818 INFO:teuthology.orchestra.run.smithi059.stdout: "osd": "allow *" 2023-11-24T07:59:48.818 INFO:teuthology.orchestra.run.smithi059.stdout: } 2023-11-24T07:59:48.818 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-11-24T07:59:48.818 INFO:teuthology.orchestra.run.smithi059.stdout: { 2023-11-24T07:59:48.818 INFO:teuthology.orchestra.run.smithi059.stdout: "entity": "osd.1", 2023-11-24T07:59:48.818 INFO:teuthology.orchestra.run.smithi059.stdout: "key": "AQD7VmBlBwTwHBAAbpMH9rzS95ifrq/gVHnYRA==", 2023-11-24T07:59:48.818 INFO:teuthology.orchestra.run.smithi059.stdout: "caps": { 2023-11-24T07:59:48.818 INFO:teuthology.orchestra.run.smithi059.stdout: "mgr": "allow profile osd", 2023-11-24T07:59:48.818 INFO:teuthology.orchestra.run.smithi059.stdout: "mon": "allow profile osd", 2023-11-24T07:59:48.818 INFO:teuthology.orchestra.run.smithi059.stdout: "osd": "allow *" 2023-11-24T07:59:48.819 INFO:teuthology.orchestra.run.smithi059.stdout: } 2023-11-24T07:59:48.819 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-11-24T07:59:48.819 INFO:teuthology.orchestra.run.smithi059.stdout: { 2023-11-24T07:59:48.819 INFO:teuthology.orchestra.run.smithi059.stdout: "entity": "osd.2", 2023-11-24T07:59:48.819 INFO:teuthology.orchestra.run.smithi059.stdout: "key": "AQD+VmBl0tijHBAAWF2PsV3iH5adOTvHVPFIKw==", 2023-11-24T07:59:48.819 INFO:teuthology.orchestra.run.smithi059.stdout: "caps": { 2023-11-24T07:59:48.819 INFO:teuthology.orchestra.run.smithi059.stdout: "mgr": "allow profile osd", 2023-11-24T07:59:48.819 INFO:teuthology.orchestra.run.smithi059.stdout: "mon": "allow profile osd", 2023-11-24T07:59:48.819 INFO:teuthology.orchestra.run.smithi059.stdout: "osd": "allow *" 2023-11-24T07:59:48.819 INFO:teuthology.orchestra.run.smithi059.stdout: } 2023-11-24T07:59:48.819 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-11-24T07:59:48.819 INFO:teuthology.orchestra.run.smithi059.stdout: { 2023-11-24T07:59:48.820 INFO:teuthology.orchestra.run.smithi059.stdout: "entity": "osd.3", 2023-11-24T07:59:48.820 INFO:teuthology.orchestra.run.smithi059.stdout: "key": "AQABV2Bl3i8/HBAA4waLTD/PSLuNEg1VBmhEmg==", 2023-11-24T07:59:48.820 INFO:teuthology.orchestra.run.smithi059.stdout: "caps": { 2023-11-24T07:59:48.820 INFO:teuthology.orchestra.run.smithi059.stdout: "mgr": "allow profile osd", 2023-11-24T07:59:48.820 INFO:teuthology.orchestra.run.smithi059.stdout: "mon": "allow profile osd", 2023-11-24T07:59:48.820 INFO:teuthology.orchestra.run.smithi059.stdout: "osd": "allow *" 2023-11-24T07:59:48.820 INFO:teuthology.orchestra.run.smithi059.stdout: } 2023-11-24T07:59:48.820 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-11-24T07:59:48.820 INFO:teuthology.orchestra.run.smithi059.stdout: { 2023-11-24T07:59:48.820 INFO:teuthology.orchestra.run.smithi059.stdout: "entity": "osd.4", 2023-11-24T07:59:48.820 INFO:teuthology.orchestra.run.smithi059.stdout: "key": "AQAHV2Blr2zrHxAALOr+kFGdOTr8uQ9+/G2WUg==", 2023-11-24T07:59:48.821 INFO:teuthology.orchestra.run.smithi059.stdout: "caps": { 2023-11-24T07:59:48.821 INFO:teuthology.orchestra.run.smithi059.stdout: "mgr": "allow profile osd", 2023-11-24T07:59:48.821 INFO:teuthology.orchestra.run.smithi059.stdout: "mon": "allow profile osd", 2023-11-24T07:59:48.821 INFO:teuthology.orchestra.run.smithi059.stdout: "osd": "allow *" 2023-11-24T07:59:48.821 INFO:teuthology.orchestra.run.smithi059.stdout: } 2023-11-24T07:59:48.821 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-11-24T07:59:48.821 INFO:teuthology.orchestra.run.smithi059.stdout: { 2023-11-24T07:59:48.821 INFO:teuthology.orchestra.run.smithi059.stdout: "entity": "osd.5", 2023-11-24T07:59:48.821 INFO:teuthology.orchestra.run.smithi059.stdout: "key": "AQAKV2BlDZ3bHxAA7ZPPoxi4zVN0VECYzTsAsg==", 2023-11-24T07:59:48.821 INFO:teuthology.orchestra.run.smithi059.stdout: "caps": { 2023-11-24T07:59:48.821 INFO:teuthology.orchestra.run.smithi059.stdout: "mgr": "allow profile osd", 2023-11-24T07:59:48.822 INFO:teuthology.orchestra.run.smithi059.stdout: "mon": "allow profile osd", 2023-11-24T07:59:48.822 INFO:teuthology.orchestra.run.smithi059.stdout: "osd": "allow *" 2023-11-24T07:59:48.822 INFO:teuthology.orchestra.run.smithi059.stdout: } 2023-11-24T07:59:48.822 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-11-24T07:59:48.822 INFO:teuthology.orchestra.run.smithi059.stdout: { 2023-11-24T07:59:48.822 INFO:teuthology.orchestra.run.smithi059.stdout: "entity": "osd.6", 2023-11-24T07:59:48.822 INFO:teuthology.orchestra.run.smithi059.stdout: "key": "AQANV2BlfDMrHxAAVtm8+kh5FNTd1mJOKg02gg==", 2023-11-24T07:59:48.822 INFO:teuthology.orchestra.run.smithi059.stdout: "caps": { 2023-11-24T07:59:48.822 INFO:teuthology.orchestra.run.smithi059.stdout: "mgr": "allow profile osd", 2023-11-24T07:59:48.822 INFO:teuthology.orchestra.run.smithi059.stdout: "mon": "allow profile osd", 2023-11-24T07:59:48.822 INFO:teuthology.orchestra.run.smithi059.stdout: "osd": "allow *" 2023-11-24T07:59:48.822 INFO:teuthology.orchestra.run.smithi059.stdout: } 2023-11-24T07:59:48.823 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-11-24T07:59:48.823 INFO:teuthology.orchestra.run.smithi059.stdout: { 2023-11-24T07:59:48.823 INFO:teuthology.orchestra.run.smithi059.stdout: "entity": "osd.7", 2023-11-24T07:59:48.823 INFO:teuthology.orchestra.run.smithi059.stdout: "key": "AQAQV2Bli+slHxAAHY4sa84El5NC4KVgPKHs6w==", 2023-11-24T07:59:48.823 INFO:teuthology.orchestra.run.smithi059.stdout: "caps": { 2023-11-24T07:59:48.823 INFO:teuthology.orchestra.run.smithi059.stdout: "mgr": "allow profile osd", 2023-11-24T07:59:48.823 INFO:teuthology.orchestra.run.smithi059.stdout: "mon": "allow profile osd", 2023-11-24T07:59:48.823 INFO:teuthology.orchestra.run.smithi059.stdout: "osd": "allow *" 2023-11-24T07:59:48.823 INFO:teuthology.orchestra.run.smithi059.stdout: } 2023-11-24T07:59:48.823 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-11-24T07:59:48.823 INFO:teuthology.orchestra.run.smithi059.stdout: { 2023-11-24T07:59:48.824 INFO:teuthology.orchestra.run.smithi059.stdout: "entity": "client.0", 2023-11-24T07:59:48.824 INFO:teuthology.orchestra.run.smithi059.stdout: "key": "AQD1VmBlxPkpBhAAR841hRsF6QtmzFgJIrVZ9w==", 2023-11-24T07:59:48.824 INFO:teuthology.orchestra.run.smithi059.stdout: "caps": { 2023-11-24T07:59:48.824 INFO:teuthology.orchestra.run.smithi059.stdout: "mds": "allow", 2023-11-24T07:59:48.824 INFO:teuthology.orchestra.run.smithi059.stdout: "mon": "allow r", 2023-11-24T07:59:48.824 INFO:teuthology.orchestra.run.smithi059.stdout: "osd": "allow rw tag cephfs data=cephfs" 2023-11-24T07:59:48.824 INFO:teuthology.orchestra.run.smithi059.stdout: } 2023-11-24T07:59:48.824 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-11-24T07:59:48.824 INFO:teuthology.orchestra.run.smithi059.stdout: { 2023-11-24T07:59:48.824 INFO:teuthology.orchestra.run.smithi059.stdout: "entity": "client.1", 2023-11-24T07:59:48.824 INFO:teuthology.orchestra.run.smithi059.stdout: "key": "AQD1VmBl46HpDBAAnO9MGWrJczW0epSp0eCkLg==", 2023-11-24T07:59:48.825 INFO:teuthology.orchestra.run.smithi059.stdout: "caps": { 2023-11-24T07:59:48.825 INFO:teuthology.orchestra.run.smithi059.stdout: "mds": "allow", 2023-11-24T07:59:48.825 INFO:teuthology.orchestra.run.smithi059.stdout: "mgr": "allow r", 2023-11-24T07:59:48.825 INFO:teuthology.orchestra.run.smithi059.stdout: "mon": "allow rw", 2023-11-24T07:59:48.825 INFO:teuthology.orchestra.run.smithi059.stdout: "osd": "allow rwx" 2023-11-24T07:59:48.825 INFO:teuthology.orchestra.run.smithi059.stdout: } 2023-11-24T07:59:48.825 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-11-24T07:59:48.825 INFO:teuthology.orchestra.run.smithi059.stdout: { 2023-11-24T07:59:48.825 INFO:teuthology.orchestra.run.smithi059.stdout: "entity": "client.2", 2023-11-24T07:59:48.825 INFO:teuthology.orchestra.run.smithi059.stdout: "key": "AQD1VmBlgQSRCRAAcHS1eQCYMJ3YP21jUEPJ0A==", 2023-11-24T07:59:48.825 INFO:teuthology.orchestra.run.smithi059.stdout: "caps": { 2023-11-24T07:59:48.825 INFO:teuthology.orchestra.run.smithi059.stdout: "mds": "allow", 2023-11-24T07:59:48.826 INFO:teuthology.orchestra.run.smithi059.stdout: "mgr": "allow r", 2023-11-24T07:59:48.826 INFO:teuthology.orchestra.run.smithi059.stdout: "mon": "allow rw", 2023-11-24T07:59:48.826 INFO:teuthology.orchestra.run.smithi059.stdout: "osd": "allow rwx" 2023-11-24T07:59:48.826 INFO:teuthology.orchestra.run.smithi059.stdout: } 2023-11-24T07:59:48.826 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-11-24T07:59:48.826 INFO:teuthology.orchestra.run.smithi059.stdout: { 2023-11-24T07:59:48.826 INFO:teuthology.orchestra.run.smithi059.stdout: "entity": "client.3", 2023-11-24T07:59:48.826 INFO:teuthology.orchestra.run.smithi059.stdout: "key": "AQD1VmBl7gmhEhAA6uWs7XAbOQwJItA6To549w==", 2023-11-24T07:59:48.826 INFO:teuthology.orchestra.run.smithi059.stdout: "caps": { 2023-11-24T07:59:48.826 INFO:teuthology.orchestra.run.smithi059.stdout: "mds": "allow", 2023-11-24T07:59:48.827 INFO:teuthology.orchestra.run.smithi059.stdout: "mgr": "allow r", 2023-11-24T07:59:48.827 INFO:teuthology.orchestra.run.smithi059.stdout: "mon": "allow rw", 2023-11-24T07:59:48.827 INFO:teuthology.orchestra.run.smithi059.stdout: "osd": "allow rwx" 2023-11-24T07:59:48.827 INFO:teuthology.orchestra.run.smithi059.stdout: } 2023-11-24T07:59:48.827 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-11-24T07:59:48.827 INFO:teuthology.orchestra.run.smithi059.stdout: { 2023-11-24T07:59:48.827 INFO:teuthology.orchestra.run.smithi059.stdout: "entity": "client.admin", 2023-11-24T07:59:48.827 INFO:teuthology.orchestra.run.smithi059.stdout: "key": "AQD0VmBlAECpCRAA1CkLlkyjnOovaNUGDSWhaw==", 2023-11-24T07:59:48.827 INFO:teuthology.orchestra.run.smithi059.stdout: "caps": { 2023-11-24T07:59:48.827 INFO:teuthology.orchestra.run.smithi059.stdout: "mds": "allow *", 2023-11-24T07:59:48.827 INFO:teuthology.orchestra.run.smithi059.stdout: "mgr": "allow *", 2023-11-24T07:59:48.828 INFO:teuthology.orchestra.run.smithi059.stdout: "mon": "allow *", 2023-11-24T07:59:48.828 INFO:teuthology.orchestra.run.smithi059.stdout: "osd": "allow *" 2023-11-24T07:59:48.828 INFO:teuthology.orchestra.run.smithi059.stdout: } 2023-11-24T07:59:48.828 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-11-24T07:59:48.828 INFO:teuthology.orchestra.run.smithi059.stdout: { 2023-11-24T07:59:48.828 INFO:teuthology.orchestra.run.smithi059.stdout: "entity": "mgr.x", 2023-11-24T07:59:48.828 INFO:teuthology.orchestra.run.smithi059.stdout: "key": "AQD0VmBlryM+JRAAf72PlDWEHZOwNxhW4k/8Ig==", 2023-11-24T07:59:48.828 INFO:teuthology.orchestra.run.smithi059.stdout: "caps": { 2023-11-24T07:59:48.828 INFO:teuthology.orchestra.run.smithi059.stdout: "mds": "allow *", 2023-11-24T07:59:48.828 INFO:teuthology.orchestra.run.smithi059.stdout: "mon": "allow profile mgr", 2023-11-24T07:59:48.828 INFO:teuthology.orchestra.run.smithi059.stdout: "osd": "allow *" 2023-11-24T07:59:48.829 INFO:teuthology.orchestra.run.smithi059.stdout: } 2023-11-24T07:59:48.829 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-11-24T07:59:48.829 INFO:teuthology.orchestra.run.smithi059.stdout: { 2023-11-24T07:59:48.829 INFO:teuthology.orchestra.run.smithi059.stdout: "entity": "mgr.y", 2023-11-24T07:59:48.829 INFO:teuthology.orchestra.run.smithi059.stdout: "key": "AQD0VmBlpXXaHxAAJMwwht1n+zRbRZuGOD5WTQ==", 2023-11-24T07:59:48.829 INFO:teuthology.orchestra.run.smithi059.stdout: "caps": { 2023-11-24T07:59:48.829 INFO:teuthology.orchestra.run.smithi059.stdout: "mds": "allow *", 2023-11-24T07:59:48.829 INFO:teuthology.orchestra.run.smithi059.stdout: "mon": "allow profile mgr", 2023-11-24T07:59:48.829 INFO:teuthology.orchestra.run.smithi059.stdout: "osd": "allow *" 2023-11-24T07:59:48.829 INFO:teuthology.orchestra.run.smithi059.stdout: } 2023-11-24T07:59:48.829 INFO:teuthology.orchestra.run.smithi059.stdout: } 2023-11-24T07:59:48.829 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2023-11-24T07:59:48.830 INFO:teuthology.orchestra.run.smithi059.stdout:} 2023-11-24T07:59:48.831 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:48.813+0000 7f3340276700 1 -- 172.21.15.59:0/1912497493 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f331c06b470 msgr2=0x7f331c06d930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:48.831 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:48.813+0000 7f3340276700 1 --2- 172.21.15.59:0/1912497493 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f331c06b470 0x7f331c06d930 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f3328000f60 tx=0x7f3328009040 comp rx=0 tx=0).stop 2023-11-24T07:59:48.831 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:48.813+0000 7f3340276700 1 -- 172.21.15.59:0/1912497493 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f333815e1c0 msgr2=0x7f33380b1cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:48.832 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:48.813+0000 7f3340276700 1 --2- 172.21.15.59:0/1912497493 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f333815e1c0 0x7f33380b1cd0 secure :-1 s=READY pgs=1002 cs=0 l=1 rev1=1 crypto rx=0x7f3338072340 tx=0x7f3330004ad0 comp rx=0 tx=0).stop 2023-11-24T07:59:48.832 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:48.813+0000 7f3340276700 1 -- 172.21.15.59:0/1912497493 shutdown_connections 2023-11-24T07:59:48.832 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:48.813+0000 7f3340276700 1 --2- 172.21.15.59:0/1912497493 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f331c06b470 0x7f331c06d930 unknown :-1 s=CLOSED pgs=464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:48.832 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:48.813+0000 7f3340276700 1 --2- 172.21.15.59:0/1912497493 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f333815e1c0 0x7f33380b1cd0 unknown :-1 s=CLOSED pgs=1002 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:48.832 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:48.813+0000 7f3340276700 1 --2- 172.21.15.59:0/1912497493 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f333806fc50 0x7f3338149160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:48.832 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:48.813+0000 7f3340276700 1 --2- 172.21.15.59:0/1912497493 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f33380b0e90 0x7f33380b1790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:48.832 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:48.813+0000 7f3340276700 1 -- 172.21.15.59:0/1912497493 >> 172.21.15.59:0/1912497493 conn(0x7f33380b3720 msgr2=0x7f33380b3b00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:59:48.832 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:48.813+0000 7f3340276700 1 -- 172.21.15.59:0/1912497493 shutdown_connections 2023-11-24T07:59:48.832 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:48.813+0000 7f3340276700 1 -- 172.21.15.59:0/1912497493 wait complete. 2023-11-24T07:59:48.833 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-24T07:59:48.957 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:48.955+0000 7fca305b9700 1 Processor -- start 2023-11-24T07:59:48.957 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:48.955+0000 7fca305b9700 1 -- start start 2023-11-24T07:59:48.959 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:48.956+0000 7fca305b9700 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fca280722c0 0x7fca280b51b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:48.959 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:48.956+0000 7fca305b9700 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fca281440f0 0x7fca2806e510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:48.959 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:48.956+0000 7fca305b9700 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fca28143ec0 con 0x7fca28143ae0 2023-11-24T07:59:48.959 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:48.956+0000 7fca305b9700 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fca280b40a0 con 0x7fca280722c0 2023-11-24T07:59:48.959 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:48.956+0000 7fca305b9700 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fca280b3ae0 con 0x7fca281440f0 2023-11-24T07:59:48.959 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:48.956+0000 7fca2eb56700 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fca281440f0 0x7fca2806e510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:48.959 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:48.956+0000 7fca2db54700 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fca280722c0 0x7fca280b51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:48.959 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:48.957+0000 7fca2eb56700 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fca281440f0 0x7fca2806e510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3301/0 says I am v2:172.21.15.59:55148/0 (socket says 172.21.15.59:55148) 2023-11-24T07:59:48.959 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:48.957+0000 7fca2eb56700 1 -- 172.21.15.59:0/4126403460 learned_addr learned my addr 172.21.15.59:0/4126403460 (peer_addr_for_me v2:172.21.15.59:0/0) 2023-11-24T07:59:48.959 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:48.957+0000 7fca2eb56700 1 -- 172.21.15.59:0/4126403460 >> v2:172.21.15.76:3300/0 conn(0x7fca280722c0 msgr2=0x7fca280b51b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T07:59:48.959 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:48.957+0000 7fca2eb56700 1 --2- 172.21.15.59:0/4126403460 >> v2:172.21.15.76:3300/0 conn(0x7fca280722c0 0x7fca280b51b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:48.960 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:48.957+0000 7fca2eb56700 1 -- 172.21.15.59:0/4126403460 >> v1:172.21.15.76:6789/0 conn(0x7fca28143ae0 legacy=0x7fca28071bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:48.960 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:48.957+0000 7fca2eb56700 1 -- 172.21.15.59:0/4126403460 --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fca280b3dc0 con 0x7fca281440f0 2023-11-24T07:59:48.960 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:48.957+0000 7fca2db54700 1 --2- 172.21.15.59:0/4126403460 >> v2:172.21.15.76:3300/0 conn(0x7fca280722c0 0x7fca280b51b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-24T07:59:48.960 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:48.957+0000 7fca2eb56700 1 --2- 172.21.15.59:0/4126403460 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fca281440f0 0x7fca2806e510 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fca20009e50 tx=0x7fca2000b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b8cae3b9bd30cc82 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:59:48.960 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:48.957+0000 7fca2d353700 1 -- 172.21.15.59:0/4126403460 <== mon.2 v2:172.21.15.76:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca2000e070 con 0x7fca281440f0 2023-11-24T07:59:48.960 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:48.957+0000 7fca2d353700 1 -- 172.21.15.59:0/4126403460 <== mon.2 v2:172.21.15.76:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fca2000b910 con 0x7fca281440f0 2023-11-24T07:59:48.960 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:48.957+0000 7fca2d353700 1 -- 172.21.15.59:0/4126403460 <== mon.2 v2:172.21.15.76:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca2000bcc0 con 0x7fca281440f0 2023-11-24T07:59:48.961 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:48.958+0000 7fca305b9700 1 -- 172.21.15.59:0/4126403460 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fca281440f0 msgr2=0x7fca2806e510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:48.961 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:48.958+0000 7fca305b9700 1 --2- 172.21.15.59:0/4126403460 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fca281440f0 0x7fca2806e510 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fca20009e50 tx=0x7fca2000b330 comp rx=0 tx=0).stop 2023-11-24T07:59:48.961 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:48.958+0000 7fca305b9700 1 -- 172.21.15.59:0/4126403460 shutdown_connections 2023-11-24T07:59:48.961 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:48.958+0000 7fca305b9700 1 --2- 172.21.15.59:0/4126403460 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fca281440f0 0x7fca2806e510 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:48.961 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:48.958+0000 7fca305b9700 1 --2- 172.21.15.59:0/4126403460 >> v2:172.21.15.76:3300/0 conn(0x7fca280722c0 0x7fca280b51b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:48.961 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:48.958+0000 7fca305b9700 1 -- 172.21.15.59:0/4126403460 >> 172.21.15.59:0/4126403460 conn(0x7fca280b2740 msgr2=0x7fca280b2b20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:59:48.961 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:48.958+0000 7fca305b9700 1 -- 172.21.15.59:0/4126403460 shutdown_connections 2023-11-24T07:59:48.962 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:48.958+0000 7fca305b9700 1 -- 172.21.15.59:0/4126403460 wait complete. 2023-11-24T07:59:48.962 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:48.959+0000 7fca305b9700 1 Processor -- start 2023-11-24T07:59:48.962 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:48.959+0000 7fca305b9700 1 -- start start 2023-11-24T07:59:48.964 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:48.960+0000 7fca305b9700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fca280722c0 0x7fca2813a710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:48.964 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:48.960+0000 7fca305b9700 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fca28143ae0 0x7fca28156700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:48.964 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:48.960+0000 7fca305b9700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fca281440f0 0x7fca28153b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:48.964 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:48.960+0000 7fca305b9700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca2813ac50 con 0x7fca281440f0 2023-11-24T07:59:48.964 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:48.960+0000 7fca305b9700 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca28151470 con 0x7fca280722c0 2023-11-24T07:59:48.965 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:48.960+0000 7fca305b9700 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fca281598f0 con 0x7fca28143ae0 2023-11-24T07:59:48.965 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:48.960+0000 7fca2eb56700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fca281440f0 0x7fca28153b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:48.965 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:48.960+0000 7fca2eb56700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fca281440f0 0x7fca28153b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:55714/0 (socket says 172.21.15.59:55714) 2023-11-24T07:59:48.965 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:48.960+0000 7fca2eb56700 1 -- 172.21.15.59:0/2886103535 learned_addr learned my addr 172.21.15.59:0/2886103535 (peer_addr_for_me v2:172.21.15.59:0/0) 2023-11-24T07:59:48.965 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:48.960+0000 7fca2e355700 1 --2- 172.21.15.59:0/2886103535 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fca280722c0 0x7fca2813a710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:48.965 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:48.960+0000 7fca2db54700 1 --2- 172.21.15.59:0/2886103535 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fca28143ae0 0x7fca28156700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:48.965 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:48.961+0000 7fca2eb56700 1 -- 172.21.15.59:0/2886103535 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fca28143ae0 msgr2=0x7fca28156700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:48.965 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:48.961+0000 7fca2eb56700 1 --2- 172.21.15.59:0/2886103535 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fca28143ae0 0x7fca28156700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:48.965 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:48.961+0000 7fca2eb56700 1 -- 172.21.15.59:0/2886103535 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fca280722c0 msgr2=0x7fca2813a710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:48.965 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:48.961+0000 7fca2eb56700 1 --2- 172.21.15.59:0/2886103535 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fca280722c0 0x7fca2813a710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:48.965 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:48.961+0000 7fca2eb56700 1 -- 172.21.15.59:0/2886103535 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fca240094a0 con 0x7fca281440f0 2023-11-24T07:59:48.965 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:48.961+0000 7fca2e355700 1 --2- 172.21.15.59:0/2886103535 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fca280722c0 0x7fca2813a710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-24T07:59:48.965 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:48.961+0000 7fca2eb56700 1 --2- 172.21.15.59:0/2886103535 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fca281440f0 0x7fca28153b60 secure :-1 s=READY pgs=1003 cs=0 l=1 rev1=1 crypto rx=0x7fca280b3c20 tx=0x7fca20004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:59:48.966 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:48.961+0000 7fca1effd700 1 -- 172.21.15.59:0/2886103535 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca20004960 con 0x7fca281440f0 2023-11-24T07:59:48.966 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:48.961+0000 7fca1effd700 1 -- 172.21.15.59:0/2886103535 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fca200092e0 con 0x7fca281440f0 2023-11-24T07:59:48.966 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:48.961+0000 7fca305b9700 1 -- 172.21.15.59:0/2886103535 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fca20009b30 con 0x7fca281440f0 2023-11-24T07:59:48.966 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:48.961+0000 7fca1effd700 1 -- 172.21.15.59:0/2886103535 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca2001b650 con 0x7fca281440f0 2023-11-24T07:59:48.966 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:48.961+0000 7fca305b9700 1 -- 172.21.15.59:0/2886103535 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fca281516e0 con 0x7fca281440f0 2023-11-24T07:59:48.966 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:48.962+0000 7fca305b9700 1 -- 172.21.15.59:0/2886103535 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fca281518a0 con 0x7fca281440f0 2023-11-24T07:59:48.966 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:48.962+0000 7fca1effd700 1 -- 172.21.15.59:0/2886103535 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 11) v1 ==== 89741+0+0 (secure 0 0 0) 0x7fca2001b7b0 con 0x7fca281440f0 2023-11-24T07:59:48.966 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:48.963+0000 7fca1effd700 1 --2- 172.21.15.59:0/2886103535 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7fca0c06b470 0x7fca0c06d930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:48.966 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:48.963+0000 7fca1effd700 1 -- 172.21.15.59:0/2886103535 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 4857+0+0 (secure 0 0 0) 0x7fca2008b750 con 0x7fca281440f0 2023-11-24T07:59:48.967 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:48.963+0000 7fca2e355700 1 --2- 172.21.15.59:0/2886103535 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7fca0c06b470 0x7fca0c06d930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:48.967 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:48.964+0000 7fca2e355700 1 --2- 172.21.15.59:0/2886103535 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7fca0c06b470 0x7fca0c06d930 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7fca18000f60 tx=0x7fca18009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:59:48.969 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:48.967+0000 7fca1effd700 1 -- 172.21.15.59:0/2886103535 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fca20056260 con 0x7fca281440f0 2023-11-24T07:59:49.170 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:49.169+0000 7fca305b9700 1 -- 172.21.15.59:0/2886103535 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fca280cd8d0 con 0x7fca281440f0 2023-11-24T07:59:49.171 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:49.170+0000 7fca1effd700 1 -- 172.21.15.59:0/2886103535 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v83) v1 ==== 74+0+12786 (secure 0 0 0) 0x7fca20059880 con 0x7fca281440f0 2023-11-24T07:59:49.171 INFO:teuthology.orchestra.run.smithi059.stdout: 2023-11-24T07:59:49.172 INFO:teuthology.orchestra.run.smithi059.stdout:{"epoch":83,"fsid":"9f0e8741-b8f9-48d5-a2c3-09f45903303d","created":"2023-11-24T07:56:06.552729+0000","modified":"2023-11-24T07:59:48.365725+0000","last_up_change":"2023-11-24T07:56:17.372352+0000","last_in_change":"2023-11-24T07:56:09.073250+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":11,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-24T07:56:18.352823+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"6d8f9e9b-1782-4e41-928b-a39f2a8ebc06","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6800","nonce":3373878772},{"type":"v1","addr":"172.21.15.59:6801","nonce":3373878772}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6802","nonce":3373878772},{"type":"v1","addr":"172.21.15.59:6803","nonce":3373878772}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6806","nonce":3373878772},{"type":"v1","addr":"172.21.15.59:6807","nonce":3373878772}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6804","nonce":3373878772},{"type":"v1","addr":"172.21.15.59:6805","nonce":3373878772}]},"public_addr":"172.21.15.59:6801/3373878772","cluster_addr":"172.21.15.59:6803/3373878772","heartbeat_back_addr":"172.21.15.59:6807/3373878772","heartbeat_front_addr":"172.21.15.59:6805/3373878772","state":["exists","up"]},{"osd":1,"uuid":"e8d882aa-52e1-4c04-9e1e-c44ef9204a5f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6824","nonce":648258563},{"type":"v1","addr":"172.21.15.59:6825","nonce":648258563}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6826","nonce":648258563},{"type":"v1","addr":"172.21.15.59:6827","nonce":648258563}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6830","nonce":648258563},{"type":"v1","addr":"172.21.15.59:6831","nonce":648258563}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6828","nonce":648258563},{"type":"v1","addr":"172.21.15.59:6829","nonce":648258563}]},"public_addr":"172.21.15.59:6825/648258563","cluster_addr":"172.21.15.59:6827/648258563","heartbeat_back_addr":"172.21.15.59:6831/648258563","heartbeat_front_addr":"172.21.15.59:6829/648258563","state":["exists","up"]},{"osd":2,"uuid":"69eebfa0-695b-4b21-90b9-8a3537168a01","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6808","nonce":1840430654},{"type":"v1","addr":"172.21.15.59:6809","nonce":1840430654}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6810","nonce":1840430654},{"type":"v1","addr":"172.21.15.59:6811","nonce":1840430654}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6814","nonce":1840430654},{"type":"v1","addr":"172.21.15.59:6815","nonce":1840430654}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6812","nonce":1840430654},{"type":"v1","addr":"172.21.15.59:6813","nonce":1840430654}]},"public_addr":"172.21.15.59:6809/1840430654","cluster_addr":"172.21.15.59:6811/1840430654","heartbeat_back_addr":"172.21.15.59:6815/1840430654","heartbeat_front_addr":"172.21.15.59:6813/1840430654","state":["exists","up"]},{"osd":3,"uuid":"96fd77fc-f911-46ff-9445-6f0be2daa8f4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6816","nonce":2504025850},{"type":"v1","addr":"172.21.15.59:6817","nonce":2504025850}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6818","nonce":2504025850},{"type":"v1","addr":"172.21.15.59:6819","nonce":2504025850}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6822","nonce":2504025850},{"type":"v1","addr":"172.21.15.59:6823","nonce":2504025850}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6820","nonce":2504025850},{"type":"v1","addr":"172.21.15.59:6821","nonce":2504025850}]},"public_addr":"172.21.15.59:6817/2504025850","cluster_addr":"172.21.15.59:6819/2504025850","heartbeat_back_addr":"172.21.15.59:6823/2504025850","heartbeat_front_addr":"172.21.15.59:6821/2504025850","state":["exists","up"]},{"osd":4,"uuid":"7ac29fca-b5b2-4c95-9d69-2f164cbbc708","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6816","nonce":2382463431},{"type":"v1","addr":"172.21.15.76:6818","nonce":2382463431}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6820","nonce":2382463431},{"type":"v1","addr":"172.21.15.76:6822","nonce":2382463431}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6828","nonce":2382463431},{"type":"v1","addr":"172.21.15.76:6830","nonce":2382463431}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6824","nonce":2382463431},{"type":"v1","addr":"172.21.15.76:6826","nonce":2382463431}]},"public_addr":"172.21.15.76:6818/2382463431","cluster_addr":"172.21.15.76:6822/2382463431","heartbeat_back_addr":"172.21.15.76:6830/2382463431","heartbeat_front_addr":"172.21.15.76:6826/2382463431","state":["exists","up"]},{"osd":5,"uuid":"485bf711-3851-4238-a4f4-51d52e0a9c58","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6800","nonce":329334308},{"type":"v1","addr":"172.21.15.76:6802","nonce":329334308}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6805","nonce":329334308},{"type":"v1","addr":"172.21.15.76:6807","nonce":329334308}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6814","nonce":329334308},{"type":"v1","addr":"172.21.15.76:6815","nonce":329334308}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6810","nonce":329334308},{"type":"v1","addr":"172.21.15.76:6812","nonce":329334308}]},"public_addr":"172.21.15.76:6802/329334308","cluster_addr":"172.21.15.76:6807/329334308","heartbeat_back_addr":"172.21.15.76:6815/329334308","heartbeat_front_addr":"172.21.15.76:6812/329334308","state":["exists","up"]},{"osd":6,"uuid":"72b3f2fc-1e4a-4e32-968d-003647a0286f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6801","nonce":3946579410},{"type":"v1","addr":"172.21.15.76:6803","nonce":3946579410}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6804","nonce":3946579410},{"type":"v1","addr":"172.21.15.76:6806","nonce":3946579410}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6811","nonce":3946579410},{"type":"v1","addr":"172.21.15.76:6813","nonce":3946579410}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6808","nonce":3946579410},{"type":"v1","addr":"172.21.15.76:6809","nonce":3946579410}]},"public_addr":"172.21.15.76:6803/3946579410","cluster_addr":"172.21.15.76:6806/3946579410","heartbeat_back_addr":"172.21.15.76:6813/3946579410","heartbeat_front_addr":"172.21.15.76:6809/3946579410","state":["exists","up"]},{"osd":7,"uuid":"1ddceb90-6c5d-4d1a-ad56-99cb877f8c70","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6817","nonce":841761538},{"type":"v1","addr":"172.21.15.76:6819","nonce":841761538}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6821","nonce":841761538},{"type":"v1","addr":"172.21.15.76:6823","nonce":841761538}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6829","nonce":841761538},{"type":"v1","addr":"172.21.15.76:6831","nonce":841761538}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6825","nonce":841761538},{"type":"v1","addr":"172.21.15.76:6827","nonce":841761538}]},"public_addr":"172.21.15.76:6819/841761538","cluster_addr":"172.21.15.76:6823/841761538","heartbeat_back_addr":"172.21.15.76:6831/841761538","heartbeat_front_addr":"172.21.15.76:6827/841761538","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T07:56:14.294838+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T07:56:14.270779+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T07:56:14.231809+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T07:56:14.260983+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T07:56:14.302997+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T07:56:14.258600+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T07:56:14.251413+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T07:56:14.304727+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-24T07:59:49.174 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:49.172+0000 7fca305b9700 1 -- 172.21.15.59:0/2886103535 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7fca0c06b470 msgr2=0x7fca0c06d930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:49.174 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:49.172+0000 7fca305b9700 1 --2- 172.21.15.59:0/2886103535 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7fca0c06b470 0x7fca0c06d930 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7fca18000f60 tx=0x7fca18009040 comp rx=0 tx=0).stop 2023-11-24T07:59:49.174 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:49.172+0000 7fca305b9700 1 -- 172.21.15.59:0/2886103535 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fca281440f0 msgr2=0x7fca28153b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:49.174 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:49.172+0000 7fca305b9700 1 --2- 172.21.15.59:0/2886103535 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fca281440f0 0x7fca28153b60 secure :-1 s=READY pgs=1003 cs=0 l=1 rev1=1 crypto rx=0x7fca280b3c20 tx=0x7fca20004ad0 comp rx=0 tx=0).stop 2023-11-24T07:59:49.174 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:49.172+0000 7fca305b9700 1 -- 172.21.15.59:0/2886103535 shutdown_connections 2023-11-24T07:59:49.174 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:49.172+0000 7fca305b9700 1 --2- 172.21.15.59:0/2886103535 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7fca0c06b470 0x7fca0c06d930 unknown :-1 s=CLOSED pgs=465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:49.174 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:49.172+0000 7fca305b9700 1 --2- 172.21.15.59:0/2886103535 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fca281440f0 0x7fca28153b60 unknown :-1 s=CLOSED pgs=1003 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:49.174 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:49.172+0000 7fca305b9700 1 --2- 172.21.15.59:0/2886103535 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fca280722c0 0x7fca2813a710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:49.174 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:49.172+0000 7fca305b9700 1 --2- 172.21.15.59:0/2886103535 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fca28143ae0 0x7fca28156700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:49.175 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:49.172+0000 7fca305b9700 1 -- 172.21.15.59:0/2886103535 >> 172.21.15.59:0/2886103535 conn(0x7fca280b2740 msgr2=0x7fca280b41e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:59:49.175 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:49.172+0000 7fca305b9700 1 -- 172.21.15.59:0/2886103535 shutdown_connections 2023-11-24T07:59:49.175 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:49.172+0000 7fca305b9700 1 -- 172.21.15.59:0/2886103535 wait complete. 2023-11-24T07:59:49.187 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-24T07:56:18.352823+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-24T07:59:49.187 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-24T07:59:49.316 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:49.314+0000 7f08beab5700 1 Processor -- start 2023-11-24T07:59:49.316 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:49.314+0000 7f08beab5700 1 -- start start 2023-11-24T07:59:49.318 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:49.315+0000 7f08beab5700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f08b80b4150 0x7f08b806de50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:49.318 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:49.315+0000 7f08beab5700 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f08b80b5ab0 0x7f08b806e390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:49.318 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:49.315+0000 7f08beab5700 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f08b8071f50 con 0x7f08b806b620 2023-11-24T07:59:49.318 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:49.315+0000 7f08beab5700 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f08b8072230 con 0x7f08b80b4150 2023-11-24T07:59:49.318 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:49.315+0000 7f08beab5700 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f08b8072510 con 0x7f08b80b5ab0 2023-11-24T07:59:49.318 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:49.315+0000 7f08b7fff700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f08b80b4150 0x7f08b806de50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:49.318 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:49.315+0000 7f08b7fff700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f08b80b4150 0x7f08b806de50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:55720/0 (socket says 172.21.15.59:55720) 2023-11-24T07:59:49.319 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:49.315+0000 7f08b7fff700 1 -- 172.21.15.59:0/2795108511 learned_addr learned my addr 172.21.15.59:0/2795108511 (peer_addr_for_me v2:172.21.15.59:0/0) 2023-11-24T07:59:49.319 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:49.315+0000 7f08bd052700 1 --2- 172.21.15.59:0/2795108511 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f08b80b5ab0 0x7f08b806e390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:49.319 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:49.315+0000 7f08b7fff700 1 -- 172.21.15.59:0/2795108511 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f08b80b5ab0 msgr2=0x7f08b806e390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T07:59:49.319 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:49.315+0000 7f08b7fff700 1 --2- 172.21.15.59:0/2795108511 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f08b80b5ab0 0x7f08b806e390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:49.319 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:49.315+0000 7f08b7fff700 1 -- 172.21.15.59:0/2795108511 >> v1:172.21.15.76:6789/0 conn(0x7f08b806b620 legacy=0x7f08b80b3a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:49.319 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:49.315+0000 7f08b7fff700 1 -- 172.21.15.59:0/2795108511 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f08b80727f0 con 0x7f08b80b4150 2023-11-24T07:59:49.319 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:49.315+0000 7f08b7fff700 1 --2- 172.21.15.59:0/2795108511 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f08b80b4150 0x7f08b806de50 secure :-1 s=READY pgs=1004 cs=0 l=1 rev1=1 crypto rx=0x7f08a40098f0 tx=0x7f08a4009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=43526d38a3b8c77c server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:59:49.319 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:49.316+0000 7f08b77fe700 1 -- 172.21.15.59:0/2795108511 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08a400b8a0 con 0x7f08b80b4150 2023-11-24T07:59:49.319 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:49.316+0000 7f08b77fe700 1 -- 172.21.15.59:0/2795108511 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f08a400ba00 con 0x7f08b80b4150 2023-11-24T07:59:49.319 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:49.316+0000 7f08b77fe700 1 -- 172.21.15.59:0/2795108511 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08a400bd90 con 0x7f08b80b4150 2023-11-24T07:59:49.319 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:49.316+0000 7f08beab5700 1 -- 172.21.15.59:0/2795108511 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f08b80b4150 msgr2=0x7f08b806de50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:49.319 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:49.316+0000 7f08beab5700 1 --2- 172.21.15.59:0/2795108511 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f08b80b4150 0x7f08b806de50 secure :-1 s=READY pgs=1004 cs=0 l=1 rev1=1 crypto rx=0x7f08a40098f0 tx=0x7f08a4009c00 comp rx=0 tx=0).stop 2023-11-24T07:59:49.319 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:49.316+0000 7f08beab5700 1 -- 172.21.15.59:0/2795108511 shutdown_connections 2023-11-24T07:59:49.319 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:49.316+0000 7f08beab5700 1 --2- 172.21.15.59:0/2795108511 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f08b80b5ab0 0x7f08b806e390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:49.320 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:49.316+0000 7f08beab5700 1 --2- 172.21.15.59:0/2795108511 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f08b80b4150 0x7f08b806de50 unknown :-1 s=CLOSED pgs=1004 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:49.320 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:49.316+0000 7f08beab5700 1 -- 172.21.15.59:0/2795108511 >> 172.21.15.59:0/2795108511 conn(0x7f08b80b2740 msgr2=0x7f08b80b2b20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:59:49.320 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:49.316+0000 7f08beab5700 1 -- 172.21.15.59:0/2795108511 shutdown_connections 2023-11-24T07:59:49.320 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:49.316+0000 7f08beab5700 1 -- 172.21.15.59:0/2795108511 wait complete. 2023-11-24T07:59:49.322 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:49.318+0000 7f08beab5700 1 Processor -- start 2023-11-24T07:59:49.322 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:49.318+0000 7f08beab5700 1 -- start start 2023-11-24T07:59:49.322 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:49.319+0000 7f08beab5700 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f08b806b620 0x7f08b80cf260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:49.322 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:49.319+0000 7f08beab5700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f08b80b4150 0x7f08b81f3820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:49.322 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:49.319+0000 7f08beab5700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f08b80b5ab0 0x7f08b80c7350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:49.322 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:49.319+0000 7f08beab5700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08b8144a10 con 0x7f08b80b4150 2023-11-24T07:59:49.322 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:49.319+0000 7f08beab5700 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08b80c7f60 con 0x7f08b80b5ab0 2023-11-24T07:59:49.322 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:49.319+0000 7f08beab5700 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f08b80c8240 con 0x7f08b806b620 2023-11-24T07:59:49.322 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:49.319+0000 7f08b7fff700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f08b80b4150 0x7f08b81f3820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:49.323 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:49.319+0000 7f08b7fff700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f08b80b4150 0x7f08b81f3820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:55726/0 (socket says 172.21.15.59:55726) 2023-11-24T07:59:49.323 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:49.319+0000 7f08b7fff700 1 -- 172.21.15.59:0/3029184395 learned_addr learned my addr 172.21.15.59:0/3029184395 (peer_addr_for_me v2:172.21.15.59:0/0) 2023-11-24T07:59:49.323 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:49.319+0000 7f08bd052700 1 --2- 172.21.15.59:0/3029184395 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f08b80b5ab0 0x7f08b80c7350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:49.323 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:49.319+0000 7f08bc851700 1 --2- 172.21.15.59:0/3029184395 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f08b806b620 0x7f08b80cf260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:49.323 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:49.319+0000 7f08b7fff700 1 -- 172.21.15.59:0/3029184395 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f08b806b620 msgr2=0x7f08b80cf260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:49.323 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:49.319+0000 7f08b7fff700 1 --2- 172.21.15.59:0/3029184395 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f08b806b620 0x7f08b80cf260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:49.323 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:49.319+0000 7f08b7fff700 1 -- 172.21.15.59:0/3029184395 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f08b80b5ab0 msgr2=0x7f08b80c7350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:49.323 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:49.319+0000 7f08b7fff700 1 --2- 172.21.15.59:0/3029184395 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f08b80b5ab0 0x7f08b80c7350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:49.323 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:49.319+0000 7f08b7fff700 1 -- 172.21.15.59:0/3029184395 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f08a4009550 con 0x7f08b80b4150 2023-11-24T07:59:49.323 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:49.320+0000 7f08b7fff700 1 --2- 172.21.15.59:0/3029184395 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f08b80b4150 0x7f08b81f3820 secure :-1 s=READY pgs=1005 cs=0 l=1 rev1=1 crypto rx=0x7f08a4000c00 tx=0x7f08a4004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:59:49.323 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:49.320+0000 7f08b57fa700 1 -- 172.21.15.59:0/3029184395 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08a4004300 con 0x7f08b80b4150 2023-11-24T07:59:49.323 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:49.320+0000 7f08b57fa700 1 -- 172.21.15.59:0/3029184395 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f08a401f070 con 0x7f08b80b4150 2023-11-24T07:59:49.323 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:49.320+0000 7f08beab5700 1 -- 172.21.15.59:0/3029184395 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f08b80c8520 con 0x7f08b80b4150 2023-11-24T07:59:49.324 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:49.320+0000 7f08b57fa700 1 -- 172.21.15.59:0/3029184395 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08a40036a0 con 0x7f08b80b4150 2023-11-24T07:59:49.324 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:49.320+0000 7f08beab5700 1 -- 172.21.15.59:0/3029184395 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f08b80c8ae0 con 0x7f08b80b4150 2023-11-24T07:59:49.324 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:49.321+0000 7f08beab5700 1 -- 172.21.15.59:0/3029184395 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f08b80c8c70 con 0x7f08b80b4150 2023-11-24T07:59:49.325 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:49.321+0000 7f08b57fa700 1 -- 172.21.15.59:0/3029184395 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 11) v1 ==== 89741+0+0 (secure 0 0 0) 0x7f08a4021030 con 0x7f08b80b4150 2023-11-24T07:59:49.325 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:49.322+0000 7f08b57fa700 1 --2- 172.21.15.59:0/3029184395 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f089c06b470 0x7f089c06d930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:49.325 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:49.322+0000 7f08b57fa700 1 -- 172.21.15.59:0/3029184395 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f08a408b020 con 0x7f08b80b4150 2023-11-24T07:59:49.325 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:49.322+0000 7f08bc851700 1 --2- 172.21.15.59:0/3029184395 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f089c06b470 0x7f089c06d930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:49.325 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:49.323+0000 7f08bc851700 1 --2- 172.21.15.59:0/3029184395 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f089c06b470 0x7f089c06d930 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f08ac007fd0 tx=0x7f08ac009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:59:49.328 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:49.326+0000 7f08b57fa700 1 -- 172.21.15.59:0/3029184395 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f08a4055bc0 con 0x7f08b80b4150 2023-11-24T07:59:49.530 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:49.529+0000 7f08beab5700 1 -- 172.21.15.59:0/3029184395 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f08b8075480 con 0x7f08b80b4150 2023-11-24T07:59:49.530 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:49.529+0000 7f08b57fa700 1 -- 172.21.15.59:0/3029184395 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v83) v1 ==== 93+0+10 (secure 0 0 0) 0x7f08a40591e0 con 0x7f08b80b4150 2023-11-24T07:59:49.531 INFO:teuthology.orchestra.run.smithi059.stdout:pg_num: 1 2023-11-24T07:59:49.533 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:49.531+0000 7f08beab5700 1 -- 172.21.15.59:0/3029184395 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f089c06b470 msgr2=0x7f089c06d930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:49.533 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:49.531+0000 7f08beab5700 1 --2- 172.21.15.59:0/3029184395 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f089c06b470 0x7f089c06d930 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f08ac007fd0 tx=0x7f08ac009040 comp rx=0 tx=0).stop 2023-11-24T07:59:49.533 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:49.531+0000 7f08beab5700 1 -- 172.21.15.59:0/3029184395 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f08b80b4150 msgr2=0x7f08b81f3820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:49.533 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:49.531+0000 7f08beab5700 1 --2- 172.21.15.59:0/3029184395 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f08b80b4150 0x7f08b81f3820 secure :-1 s=READY pgs=1005 cs=0 l=1 rev1=1 crypto rx=0x7f08a4000c00 tx=0x7f08a4004750 comp rx=0 tx=0).stop 2023-11-24T07:59:49.533 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:49.531+0000 7f08beab5700 1 -- 172.21.15.59:0/3029184395 shutdown_connections 2023-11-24T07:59:49.533 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:49.531+0000 7f08beab5700 1 --2- 172.21.15.59:0/3029184395 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f089c06b470 0x7f089c06d930 unknown :-1 s=CLOSED pgs=466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:49.533 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:49.531+0000 7f08beab5700 1 --2- 172.21.15.59:0/3029184395 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f08b80b5ab0 0x7f08b80c7350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:49.533 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:49.531+0000 7f08beab5700 1 --2- 172.21.15.59:0/3029184395 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f08b806b620 0x7f08b80cf260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:49.534 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:49.531+0000 7f08beab5700 1 --2- 172.21.15.59:0/3029184395 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f08b80b4150 0x7f08b81f3820 unknown :-1 s=CLOSED pgs=1005 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:49.534 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:49.531+0000 7f08beab5700 1 -- 172.21.15.59:0/3029184395 >> 172.21.15.59:0/3029184395 conn(0x7f08b80b2740 msgr2=0x7f08b81476e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:59:49.534 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:49.531+0000 7f08beab5700 1 -- 172.21.15.59:0/3029184395 shutdown_connections 2023-11-24T07:59:49.534 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:49.531+0000 7f08beab5700 1 -- 172.21.15.59:0/3029184395 wait complete. 2023-11-24T07:59:49.545 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2023-11-24T07:59:49.670 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:49.669+0000 7f0c93ce1700 1 Processor -- start 2023-11-24T07:59:49.671 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:49.669+0000 7f0c93ce1700 1 -- start start 2023-11-24T07:59:49.673 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:49.670+0000 7f0c93ce1700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f0c8c0b4150 0x7f0c8c06de50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:49.673 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:49.670+0000 7f0c93ce1700 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f0c8c0b5ab0 0x7f0c8c06e390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:49.673 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:49.670+0000 7f0c93ce1700 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0c8c071f50 con 0x7f0c8c06b620 2023-11-24T07:59:49.673 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:49.670+0000 7f0c93ce1700 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0c8c072230 con 0x7f0c8c0b4150 2023-11-24T07:59:49.673 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:49.670+0000 7f0c93ce1700 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0c8c072510 con 0x7f0c8c0b5ab0 2023-11-24T07:59:49.673 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:49.670+0000 7f0c9127c700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f0c8c0b4150 0x7f0c8c06de50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:49.673 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:49.670+0000 7f0c9127c700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f0c8c0b4150 0x7f0c8c06de50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:55734/0 (socket says 172.21.15.59:55734) 2023-11-24T07:59:49.673 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:49.670+0000 7f0c9127c700 1 -- 172.21.15.59:0/3708743887 learned_addr learned my addr 172.21.15.59:0/3708743887 (peer_addr_for_me v2:172.21.15.59:0/0) 2023-11-24T07:59:49.673 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:49.670+0000 7f0c9227e700 1 --2- 172.21.15.59:0/3708743887 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f0c8c0b5ab0 0x7f0c8c06e390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:49.674 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:49.670+0000 7f0c9127c700 1 -- 172.21.15.59:0/3708743887 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f0c8c0b5ab0 msgr2=0x7f0c8c06e390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T07:59:49.674 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:49.670+0000 7f0c9127c700 1 --2- 172.21.15.59:0/3708743887 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f0c8c0b5ab0 0x7f0c8c06e390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:49.674 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:49.670+0000 7f0c9127c700 1 -- 172.21.15.59:0/3708743887 >> v1:172.21.15.76:6789/0 conn(0x7f0c8c06b620 legacy=0x7f0c8c0b3a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:49.674 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:49.670+0000 7f0c9127c700 1 -- 172.21.15.59:0/3708743887 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0c8c0727f0 con 0x7f0c8c0b4150 2023-11-24T07:59:49.674 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:49.670+0000 7f0c9227e700 1 --2- 172.21.15.59:0/3708743887 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f0c8c0b5ab0 0x7f0c8c06e390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-24T07:59:49.674 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:49.670+0000 7f0c9127c700 1 --2- 172.21.15.59:0/3708743887 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0c8c0b4150 0x7f0c8c06de50 secure :-1 s=READY pgs=1006 cs=0 l=1 rev1=1 crypto rx=0x7f0c7c0098f0 tx=0x7f0c7c009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3d4d9cab6f9bce41 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:59:49.674 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:49.670+0000 7f0c90a7b700 1 -- 172.21.15.59:0/3708743887 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c7c00b8a0 con 0x7f0c8c0b4150 2023-11-24T07:59:49.674 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:49.670+0000 7f0c90a7b700 1 -- 172.21.15.59:0/3708743887 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0c7c00ba00 con 0x7f0c8c0b4150 2023-11-24T07:59:49.674 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:49.670+0000 7f0c90a7b700 1 -- 172.21.15.59:0/3708743887 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c7c00bd90 con 0x7f0c8c0b4150 2023-11-24T07:59:49.674 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:49.671+0000 7f0c93ce1700 1 -- 172.21.15.59:0/3708743887 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0c8c0b4150 msgr2=0x7f0c8c06de50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:49.674 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:49.671+0000 7f0c93ce1700 1 --2- 172.21.15.59:0/3708743887 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0c8c0b4150 0x7f0c8c06de50 secure :-1 s=READY pgs=1006 cs=0 l=1 rev1=1 crypto rx=0x7f0c7c0098f0 tx=0x7f0c7c009c00 comp rx=0 tx=0).stop 2023-11-24T07:59:49.674 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:49.671+0000 7f0c93ce1700 1 -- 172.21.15.59:0/3708743887 shutdown_connections 2023-11-24T07:59:49.674 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:49.671+0000 7f0c93ce1700 1 --2- 172.21.15.59:0/3708743887 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f0c8c0b5ab0 0x7f0c8c06e390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:49.675 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:49.671+0000 7f0c93ce1700 1 --2- 172.21.15.59:0/3708743887 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0c8c0b4150 0x7f0c8c06de50 unknown :-1 s=CLOSED pgs=1006 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:49.675 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:49.671+0000 7f0c93ce1700 1 -- 172.21.15.59:0/3708743887 >> 172.21.15.59:0/3708743887 conn(0x7f0c8c0b2740 msgr2=0x7f0c8c0b2b20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:59:49.675 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:49.671+0000 7f0c93ce1700 1 -- 172.21.15.59:0/3708743887 shutdown_connections 2023-11-24T07:59:49.675 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:49.671+0000 7f0c93ce1700 1 -- 172.21.15.59:0/3708743887 wait complete. 2023-11-24T07:59:49.678 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:49.672+0000 7f0c93ce1700 1 Processor -- start 2023-11-24T07:59:49.678 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:49.672+0000 7f0c93ce1700 1 -- start start 2023-11-24T07:59:49.678 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:49.673+0000 7f0c93ce1700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0c8c06b620 0x7f0c8c0c63c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:49.678 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:49.673+0000 7f0c93ce1700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0c8c0b4150 0x7f0c8c141640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:49.678 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:49.673+0000 7f0c93ce1700 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f0c8c0b5ab0 0x7f0c8c1e6c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:49.678 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:49.673+0000 7f0c93ce1700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c8c141b80 con 0x7f0c8c06b620 2023-11-24T07:59:49.678 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:49.673+0000 7f0c93ce1700 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c8c141db0 con 0x7f0c8c0b4150 2023-11-24T07:59:49.678 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:49.673+0000 7f0c93ce1700 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0c8c142090 con 0x7f0c8c0b5ab0 2023-11-24T07:59:49.678 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:49.673+0000 7f0c91a7d700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0c8c06b620 0x7f0c8c0c63c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:49.678 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:49.673+0000 7f0c91a7d700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0c8c06b620 0x7f0c8c0c63c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:55738/0 (socket says 172.21.15.59:55738) 2023-11-24T07:59:49.678 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:49.673+0000 7f0c91a7d700 1 -- 172.21.15.59:0/781332640 learned_addr learned my addr 172.21.15.59:0/781332640 (peer_addr_for_me v2:172.21.15.59:0/0) 2023-11-24T07:59:49.679 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:49.673+0000 7f0c9127c700 1 --2- 172.21.15.59:0/781332640 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0c8c0b4150 0x7f0c8c141640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:49.679 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:49.673+0000 7f0c9227e700 1 --2- 172.21.15.59:0/781332640 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f0c8c0b5ab0 0x7f0c8c1e6c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:49.679 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:49.673+0000 7f0c91a7d700 1 -- 172.21.15.59:0/781332640 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f0c8c0b5ab0 msgr2=0x7f0c8c1e6c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:49.679 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:49.673+0000 7f0c91a7d700 1 --2- 172.21.15.59:0/781332640 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f0c8c0b5ab0 0x7f0c8c1e6c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:49.679 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:49.673+0000 7f0c91a7d700 1 -- 172.21.15.59:0/781332640 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0c8c0b4150 msgr2=0x7f0c8c141640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:49.679 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:49.673+0000 7f0c91a7d700 1 --2- 172.21.15.59:0/781332640 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0c8c0b4150 0x7f0c8c141640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:49.679 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:49.673+0000 7f0c91a7d700 1 -- 172.21.15.59:0/781332640 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0c7c009550 con 0x7f0c8c06b620 2023-11-24T07:59:49.679 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:49.674+0000 7f0c91a7d700 1 --2- 172.21.15.59:0/781332640 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0c8c06b620 0x7f0c8c0c63c0 secure :-1 s=READY pgs=1007 cs=0 l=1 rev1=1 crypto rx=0x7f0c8800c450 tx=0x7f0c8800c810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:59:49.679 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:49.674+0000 7f0c827fc700 1 -- 172.21.15.59:0/781332640 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c880052d0 con 0x7f0c8c06b620 2023-11-24T07:59:49.679 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:49.674+0000 7f0c827fc700 1 -- 172.21.15.59:0/781332640 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0c88005430 con 0x7f0c8c06b620 2023-11-24T07:59:49.680 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:49.674+0000 7f0c93ce1700 1 -- 172.21.15.59:0/781332640 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0c8c142370 con 0x7f0c8c06b620 2023-11-24T07:59:49.680 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:49.674+0000 7f0c827fc700 1 -- 172.21.15.59:0/781332640 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c880038c0 con 0x7f0c8c06b620 2023-11-24T07:59:49.680 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:49.674+0000 7f0c93ce1700 1 -- 172.21.15.59:0/781332640 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0c8c142930 con 0x7f0c8c06b620 2023-11-24T07:59:49.680 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:49.675+0000 7f0c93ce1700 1 -- 172.21.15.59:0/781332640 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0c8c142af0 con 0x7f0c8c06b620 2023-11-24T07:59:49.680 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:49.675+0000 7f0c827fc700 1 -- 172.21.15.59:0/781332640 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 11) v1 ==== 89741+0+0 (secure 0 0 0) 0x7f0c88003a20 con 0x7f0c8c06b620 2023-11-24T07:59:49.680 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:49.675+0000 7f0c827fc700 1 --2- 172.21.15.59:0/781332640 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f0c7006b470 0x7f0c7006d930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:49.680 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:49.676+0000 7f0c827fc700 1 -- 172.21.15.59:0/781332640 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f0c88088fc0 con 0x7f0c8c06b620 2023-11-24T07:59:49.680 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:49.676+0000 7f0c9127c700 1 --2- 172.21.15.59:0/781332640 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f0c7006b470 0x7f0c7006d930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:49.680 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:49.676+0000 7f0c9127c700 1 --2- 172.21.15.59:0/781332640 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f0c7006b470 0x7f0c7006d930 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f0c7c009c40 tx=0x7f0c7c008030 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:59:49.681 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:49.680+0000 7f0c827fc700 1 -- 172.21.15.59:0/781332640 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f0c88053be0 con 0x7f0c8c06b620 2023-11-24T07:59:49.889 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:49.888+0000 7f0c93ce1700 1 -- 172.21.15.59:0/781332640 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f0c8c068670 con 0x7f0c8c06b620 2023-11-24T07:59:49.890 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:49.888+0000 7f0c827fc700 1 -- 172.21.15.59:0/781332640 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v83) v1 ==== 84+0+65 (secure 0 0 0) 0x7f0c88057200 con 0x7f0c8c06b620 2023-11-24T07:59:49.890 INFO:teuthology.orchestra.run.smithi059.stdout: 2023-11-24T07:59:49.890 INFO:teuthology.orchestra.run.smithi059.stdout:[ 2023-11-24T07:59:49.890 INFO:teuthology.orchestra.run.smithi059.stdout: { 2023-11-24T07:59:49.891 INFO:teuthology.orchestra.run.smithi059.stdout: "poolnum": 1, 2023-11-24T07:59:49.891 INFO:teuthology.orchestra.run.smithi059.stdout: "poolname": ".mgr" 2023-11-24T07:59:49.891 INFO:teuthology.orchestra.run.smithi059.stdout: } 2023-11-24T07:59:49.891 INFO:teuthology.orchestra.run.smithi059.stdout:] 2023-11-24T07:59:49.893 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:49.891+0000 7f0c93ce1700 1 -- 172.21.15.59:0/781332640 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f0c7006b470 msgr2=0x7f0c7006d930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:49.893 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:49.891+0000 7f0c93ce1700 1 --2- 172.21.15.59:0/781332640 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f0c7006b470 0x7f0c7006d930 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f0c7c009c40 tx=0x7f0c7c008030 comp rx=0 tx=0).stop 2023-11-24T07:59:49.893 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:49.891+0000 7f0c93ce1700 1 -- 172.21.15.59:0/781332640 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0c8c06b620 msgr2=0x7f0c8c0c63c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:49.893 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:49.891+0000 7f0c93ce1700 1 --2- 172.21.15.59:0/781332640 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0c8c06b620 0x7f0c8c0c63c0 secure :-1 s=READY pgs=1007 cs=0 l=1 rev1=1 crypto rx=0x7f0c8800c450 tx=0x7f0c8800c810 comp rx=0 tx=0).stop 2023-11-24T07:59:49.893 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:49.891+0000 7f0c93ce1700 1 -- 172.21.15.59:0/781332640 shutdown_connections 2023-11-24T07:59:49.893 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:49.891+0000 7f0c93ce1700 1 --2- 172.21.15.59:0/781332640 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f0c7006b470 0x7f0c7006d930 unknown :-1 s=CLOSED pgs=467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:49.893 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:49.891+0000 7f0c93ce1700 1 --2- 172.21.15.59:0/781332640 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f0c8c0b5ab0 0x7f0c8c1e6c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:49.893 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:49.891+0000 7f0c93ce1700 1 --2- 172.21.15.59:0/781332640 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0c8c06b620 0x7f0c8c0c63c0 unknown :-1 s=CLOSED pgs=1007 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:49.893 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:49.891+0000 7f0c93ce1700 1 --2- 172.21.15.59:0/781332640 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0c8c0b4150 0x7f0c8c141640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:49.893 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:49.891+0000 7f0c93ce1700 1 -- 172.21.15.59:0/781332640 >> 172.21.15.59:0/781332640 conn(0x7f0c8c0b2740 msgr2=0x7f0c8c1475e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:59:49.894 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:49.891+0000 7f0c93ce1700 1 -- 172.21.15.59:0/781332640 shutdown_connections 2023-11-24T07:59:49.894 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:49.891+0000 7f0c93ce1700 1 -- 172.21.15.59:0/781332640 wait complete. 2023-11-24T07:59:49.905 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2023-11-24T07:59:49.905 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2023-11-24T07:59:50.034 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:50.032+0000 7f13b6642700 1 Processor -- start 2023-11-24T07:59:50.034 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:50.032+0000 7f13b6642700 1 -- start start 2023-11-24T07:59:50.036 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:50.033+0000 7f13b6642700 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f13b00722c0 0x7f13b00b51b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:50.036 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:50.033+0000 7f13b6642700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f13b01440f0 0x7f13b006e510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:50.036 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:50.033+0000 7f13b6642700 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f13b0143ec0 con 0x7f13b0143ae0 2023-11-24T07:59:50.037 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:50.033+0000 7f13b6642700 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f13b00b40a0 con 0x7f13b01440f0 2023-11-24T07:59:50.037 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:50.033+0000 7f13b6642700 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f13b00b3ae0 con 0x7f13b00722c0 2023-11-24T07:59:50.037 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:50.033+0000 7f13b4bdf700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f13b01440f0 0x7f13b006e510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:50.037 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:50.033+0000 7f13b4bdf700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f13b01440f0 0x7f13b006e510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:55750/0 (socket says 172.21.15.59:55750) 2023-11-24T07:59:50.037 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:50.033+0000 7f13b4bdf700 1 -- 172.21.15.59:0/2996134110 learned_addr learned my addr 172.21.15.59:0/2996134110 (peer_addr_for_me v2:172.21.15.59:0/0) 2023-11-24T07:59:50.037 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:50.033+0000 7f13af7fe700 1 --2- 172.21.15.59:0/2996134110 >> v2:172.21.15.76:3300/0 conn(0x7f13b00722c0 0x7f13b00b51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:50.037 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:50.033+0000 7f13b4bdf700 1 -- 172.21.15.59:0/2996134110 >> v2:172.21.15.76:3300/0 conn(0x7f13b00722c0 msgr2=0x7f13b00b51b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T07:59:50.038 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:50.033+0000 7f13b4bdf700 1 --2- 172.21.15.59:0/2996134110 >> v2:172.21.15.76:3300/0 conn(0x7f13b00722c0 0x7f13b00b51b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:50.038 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:50.033+0000 7f13b4bdf700 1 -- 172.21.15.59:0/2996134110 >> v1:172.21.15.76:6789/0 conn(0x7f13b0143ae0 legacy=0x7f13b0071bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T07:59:50.038 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:50.033+0000 7f13b4bdf700 1 -- 172.21.15.59:0/2996134110 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f13b00b3dc0 con 0x7f13b01440f0 2023-11-24T07:59:50.038 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:50.033+0000 7f13b4bdf700 1 --2- 172.21.15.59:0/2996134110 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f13b01440f0 0x7f13b006e510 secure :-1 s=READY pgs=1008 cs=0 l=1 rev1=1 crypto rx=0x7f13b00d1b00 tx=0x7f13a800d430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d50430b2b82517bc server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:59:50.038 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:50.034+0000 7f13aeffd700 1 -- 172.21.15.59:0/2996134110 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13a8010070 con 0x7f13b01440f0 2023-11-24T07:59:50.038 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:50.034+0000 7f13aeffd700 1 -- 172.21.15.59:0/2996134110 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f13a800dd50 con 0x7f13b01440f0 2023-11-24T07:59:50.038 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:50.034+0000 7f13aeffd700 1 -- 172.21.15.59:0/2996134110 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13a800b310 con 0x7f13b01440f0 2023-11-24T07:59:50.038 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:50.034+0000 7f13b6642700 1 -- 172.21.15.59:0/2996134110 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f13b01440f0 msgr2=0x7f13b006e510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:50.039 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:50.034+0000 7f13b6642700 1 --2- 172.21.15.59:0/2996134110 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f13b01440f0 0x7f13b006e510 secure :-1 s=READY pgs=1008 cs=0 l=1 rev1=1 crypto rx=0x7f13b00d1b00 tx=0x7f13a800d430 comp rx=0 tx=0).stop 2023-11-24T07:59:50.039 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:50.034+0000 7f13b6642700 1 -- 172.21.15.59:0/2996134110 shutdown_connections 2023-11-24T07:59:50.039 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:50.034+0000 7f13b6642700 1 --2- 172.21.15.59:0/2996134110 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f13b01440f0 0x7f13b006e510 unknown :-1 s=CLOSED pgs=1008 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:50.039 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:50.034+0000 7f13b6642700 1 --2- 172.21.15.59:0/2996134110 >> v2:172.21.15.76:3300/0 conn(0x7f13b00722c0 0x7f13b00b51b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:50.039 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:50.034+0000 7f13b6642700 1 -- 172.21.15.59:0/2996134110 >> 172.21.15.59:0/2996134110 conn(0x7f13b00b2740 msgr2=0x7f13b00b2b20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:59:50.039 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:50.034+0000 7f13b6642700 1 -- 172.21.15.59:0/2996134110 shutdown_connections 2023-11-24T07:59:50.039 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:50.034+0000 7f13b6642700 1 -- 172.21.15.59:0/2996134110 wait complete. 2023-11-24T07:59:50.043 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:50.036+0000 7f13b6642700 1 Processor -- start 2023-11-24T07:59:50.043 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:50.036+0000 7f13b6642700 1 -- start start 2023-11-24T07:59:50.043 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:50.036+0000 7f13b6642700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f13b00722c0 0x7f13b00cbb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:50.043 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:50.036+0000 7f13b6642700 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f13b0143ae0 0x7f13b00d0f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:50.043 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:50.036+0000 7f13b6642700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f13b01440f0 0x7f13b00d14b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:50.043 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:50.036+0000 7f13b6642700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13b01f6fd0 con 0x7f13b00722c0 2023-11-24T07:59:50.043 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:50.037+0000 7f13b6642700 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13b01f7870 con 0x7f13b01440f0 2023-11-24T07:59:50.044 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:50.037+0000 7f13b6642700 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f13b00cb4c0 con 0x7f13b0143ae0 2023-11-24T07:59:50.044 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:50.037+0000 7f13affff700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f13b00722c0 0x7f13b00cbb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:50.044 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:50.037+0000 7f13affff700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f13b00722c0 0x7f13b00cbb40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:55752/0 (socket says 172.21.15.59:55752) 2023-11-24T07:59:50.044 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:50.037+0000 7f13affff700 1 -- 172.21.15.59:0/2179413475 learned_addr learned my addr 172.21.15.59:0/2179413475 (peer_addr_for_me v2:172.21.15.59:0/0) 2023-11-24T07:59:50.044 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:50.037+0000 7f13b4bdf700 1 --2- 172.21.15.59:0/2179413475 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f13b01440f0 0x7f13b00d14b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:50.044 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:50.037+0000 7f13af7fe700 1 --2- 172.21.15.59:0/2179413475 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f13b0143ae0 0x7f13b00d0f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:50.044 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:50.037+0000 7f13affff700 1 -- 172.21.15.59:0/2179413475 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f13b0143ae0 msgr2=0x7f13b00d0f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:50.045 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:50.037+0000 7f13affff700 1 --2- 172.21.15.59:0/2179413475 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f13b0143ae0 0x7f13b00d0f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:50.045 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:50.037+0000 7f13affff700 1 -- 172.21.15.59:0/2179413475 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f13b01440f0 msgr2=0x7f13b00d14b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:50.045 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:50.037+0000 7f13affff700 1 --2- 172.21.15.59:0/2179413475 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f13b01440f0 0x7f13b00d14b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:50.045 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:50.037+0000 7f13affff700 1 -- 172.21.15.59:0/2179413475 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f13a800bd40 con 0x7f13b00722c0 2023-11-24T07:59:50.045 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:50.037+0000 7f13b4bdf700 1 --2- 172.21.15.59:0/2179413475 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f13b01440f0 0x7f13b00d14b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-24T07:59:50.045 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:50.037+0000 7f13affff700 1 --2- 172.21.15.59:0/2179413475 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f13b00722c0 0x7f13b00cbb40 secure :-1 s=READY pgs=1009 cs=0 l=1 rev1=1 crypto rx=0x7f139800c4d0 tx=0x7f139800c890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:59:50.045 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:50.037+0000 7f13acff9700 1 -- 172.21.15.59:0/2179413475 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1398011790 con 0x7f13b00722c0 2023-11-24T07:59:50.046 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:50.037+0000 7f13acff9700 1 -- 172.21.15.59:0/2179413475 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f13980118f0 con 0x7f13b00722c0 2023-11-24T07:59:50.046 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:50.037+0000 7f13b6642700 1 -- 172.21.15.59:0/2179413475 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f13b015e490 con 0x7f13b00722c0 2023-11-24T07:59:50.046 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:50.037+0000 7f13acff9700 1 -- 172.21.15.59:0/2179413475 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1398011ca0 con 0x7f13b00722c0 2023-11-24T07:59:50.046 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:50.037+0000 7f13b6642700 1 -- 172.21.15.59:0/2179413475 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f13b01f7500 con 0x7f13b00722c0 2023-11-24T07:59:50.046 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:50.038+0000 7f13b6642700 1 -- 172.21.15.59:0/2179413475 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f13b0068670 con 0x7f13b00722c0 2023-11-24T07:59:50.046 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:50.039+0000 7f13acff9700 1 -- 172.21.15.59:0/2179413475 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 11) v1 ==== 89741+0+0 (secure 0 0 0) 0x7f139801c460 con 0x7f13b00722c0 2023-11-24T07:59:50.046 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:50.039+0000 7f13acff9700 1 --2- 172.21.15.59:0/2179413475 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f139006b470 0x7f139006d930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:50.046 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:50.039+0000 7f13acff9700 1 -- 172.21.15.59:0/2179413475 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f1398089ad0 con 0x7f13b00722c0 2023-11-24T07:59:50.047 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:50.039+0000 7f13af7fe700 1 --2- 172.21.15.59:0/2179413475 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f139006b470 0x7f139006d930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:50.047 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:50.040+0000 7f13af7fe700 1 --2- 172.21.15.59:0/2179413475 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f139006b470 0x7f139006d930 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f13b00b4dc0 tx=0x7f13a0008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:59:50.047 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:50.044+0000 7f13acff9700 1 -- 172.21.15.59:0/2179413475 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f13980546f0 con 0x7f13b00722c0 2023-11-24T07:59:50.245 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:50.243+0000 7f13b6642700 1 -- 172.21.15.59:0/2179413475 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7f13b00b5d20 con 0x7f13b00722c0 2023-11-24T07:59:50.478 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:50.477+0000 7f13acff9700 1 -- 172.21.15.59:0/2179413475 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v84) v1 ==== 138+0+0 (secure 0 0 0) 0x7f1398057d10 con 0x7f13b00722c0 2023-11-24T07:59:50.482 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:50.480+0000 7f13b6642700 1 -- 172.21.15.59:0/2179413475 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f139006b470 msgr2=0x7f139006d930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:50.482 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:50.480+0000 7f13b6642700 1 --2- 172.21.15.59:0/2179413475 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f139006b470 0x7f139006d930 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f13b00b4dc0 tx=0x7f13a0008040 comp rx=0 tx=0).stop 2023-11-24T07:59:50.482 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:50.480+0000 7f13b6642700 1 -- 172.21.15.59:0/2179413475 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f13b00722c0 msgr2=0x7f13b00cbb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:50.482 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:50.480+0000 7f13b6642700 1 --2- 172.21.15.59:0/2179413475 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f13b00722c0 0x7f13b00cbb40 secure :-1 s=READY pgs=1009 cs=0 l=1 rev1=1 crypto rx=0x7f139800c4d0 tx=0x7f139800c890 comp rx=0 tx=0).stop 2023-11-24T07:59:50.482 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:50.480+0000 7f13b6642700 1 -- 172.21.15.59:0/2179413475 shutdown_connections 2023-11-24T07:59:50.483 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:50.480+0000 7f13b6642700 1 --2- 172.21.15.59:0/2179413475 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f139006b470 0x7f139006d930 secure :-1 s=CLOSED pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f13b00b4dc0 tx=0x7f13a0008040 comp rx=0 tx=0).stop 2023-11-24T07:59:50.483 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:50.480+0000 7f13b6642700 1 --2- 172.21.15.59:0/2179413475 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f13b01440f0 0x7f13b00d14b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:50.483 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:50.480+0000 7f13b6642700 1 --2- 172.21.15.59:0/2179413475 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f13b00722c0 0x7f13b00cbb40 unknown :-1 s=CLOSED pgs=1009 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:50.483 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:50.480+0000 7f13b6642700 1 --2- 172.21.15.59:0/2179413475 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f13b0143ae0 0x7f13b00d0f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:50.483 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:50.480+0000 7f13b6642700 1 -- 172.21.15.59:0/2179413475 >> 172.21.15.59:0/2179413475 conn(0x7f13b00b2740 msgr2=0x7f13b01410d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:59:50.483 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:50.481+0000 7f13b6642700 1 -- 172.21.15.59:0/2179413475 shutdown_connections 2023-11-24T07:59:50.483 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:50.481+0000 7f13b6642700 1 -- 172.21.15.59:0/2179413475 wait complete. 2023-11-24T07:59:50.484 INFO:teuthology.orchestra.run.smithi059.stderr:pool 'cephfs_metadata' created 2023-11-24T07:59:50.498 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2023-11-24T07:59:50.631 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:50.629+0000 7f23a270e700 1 Processor -- start 2023-11-24T07:59:50.631 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:50.629+0000 7f23a270e700 1 -- start start 2023-11-24T07:59:50.633 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:50.630+0000 7f23a270e700 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f239c06b620 0x7f239c0b3a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:50.633 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:50.630+0000 7f23a270e700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f239c0b5ab0 0x7f239c0b3f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:50.633 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:50.630+0000 7f23a270e700 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f239c071f50 con 0x7f239c1478c0 2023-11-24T07:59:50.633 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:50.630+0000 7f23a270e700 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f239c072230 con 0x7f239c0b5ab0 2023-11-24T07:59:50.633 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:50.630+0000 7f23a270e700 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f239c072510 con 0x7f239c06b620 2023-11-24T07:59:50.634 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:50.630+0000 7f239b7fe700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f239c0b5ab0 0x7f239c0b3f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:50.634 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:50.630+0000 7f239b7fe700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f239c0b5ab0 0x7f239c0b3f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:55760/0 (socket says 172.21.15.59:55760) 2023-11-24T07:59:50.634 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:50.630+0000 7f23a0cab700 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7f239c1478c0 0x7f239c161ba0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:59234/0 (socket says 172.21.15.59:59234) 2023-11-24T07:59:50.634 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:50.630+0000 7f239b7fe700 1 -- 172.21.15.59:0/2996572164 learned_addr learned my addr 172.21.15.59:0/2996572164 (peer_addr_for_me v2:172.21.15.59:0/0) 2023-11-24T07:59:50.634 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:50.630+0000 7f239bfff700 1 --2- 172.21.15.59:0/2996572164 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f239c06b620 0x7f239c0b3a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:50.634 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:50.631+0000 7f239b7fe700 1 -- 172.21.15.59:0/2996572164 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f239c06b620 msgr2=0x7f239c0b3a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T07:59:50.634 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:50.631+0000 7f239b7fe700 1 --2- 172.21.15.59:0/2996572164 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f239c06b620 0x7f239c0b3a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:50.634 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:50.631+0000 7f239b7fe700 1 -- 172.21.15.59:0/2996572164 >> v1:172.21.15.59:6789/0 conn(0x7f239c1478c0 legacy=0x7f239c161ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:50.634 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:50.631+0000 7f239b7fe700 1 -- 172.21.15.59:0/2996572164 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f239c0727f0 con 0x7f239c0b5ab0 2023-11-24T07:59:50.634 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:50.631+0000 7f239affd700 1 -- 172.21.15.59:0/2996572164 <== mon.0 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 932872869 0 0) 0x7f239c071f50 con 0x7f239c1478c0 2023-11-24T07:59:50.634 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:50.631+0000 7f239b7fe700 1 --2- 172.21.15.59:0/2996572164 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f239c0b5ab0 0x7f239c0b3f80 secure :-1 s=READY pgs=1011 cs=0 l=1 rev1=1 crypto rx=0x7f238c0098f0 tx=0x7f238c009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e138ecf63bba1a44 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:59:50.634 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:50.631+0000 7f239affd700 1 -- 172.21.15.59:0/2996572164 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f238c00b8a0 con 0x7f239c0b5ab0 2023-11-24T07:59:50.635 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:50.631+0000 7f239affd700 1 -- 172.21.15.59:0/2996572164 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f238c00ba00 con 0x7f239c0b5ab0 2023-11-24T07:59:50.635 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:50.631+0000 7f239affd700 1 -- 172.21.15.59:0/2996572164 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f238c00bd90 con 0x7f239c0b5ab0 2023-11-24T07:59:50.635 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:50.631+0000 7f23a270e700 1 -- 172.21.15.59:0/2996572164 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f239c0b5ab0 msgr2=0x7f239c0b3f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:50.635 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:50.631+0000 7f23a270e700 1 --2- 172.21.15.59:0/2996572164 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f239c0b5ab0 0x7f239c0b3f80 secure :-1 s=READY pgs=1011 cs=0 l=1 rev1=1 crypto rx=0x7f238c0098f0 tx=0x7f238c009c00 comp rx=0 tx=0).stop 2023-11-24T07:59:50.635 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:50.631+0000 7f23a270e700 1 -- 172.21.15.59:0/2996572164 shutdown_connections 2023-11-24T07:59:50.635 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:50.631+0000 7f23a270e700 1 --2- 172.21.15.59:0/2996572164 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f239c06b620 0x7f239c0b3a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:50.635 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:50.631+0000 7f23a270e700 1 --2- 172.21.15.59:0/2996572164 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f239c0b5ab0 0x7f239c0b3f80 unknown :-1 s=CLOSED pgs=1011 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:50.635 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:50.631+0000 7f23a270e700 1 -- 172.21.15.59:0/2996572164 >> 172.21.15.59:0/2996572164 conn(0x7f239c0b2740 msgr2=0x7f239c0b2b20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:59:50.635 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:50.632+0000 7f23a270e700 1 -- 172.21.15.59:0/2996572164 shutdown_connections 2023-11-24T07:59:50.635 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:50.632+0000 7f23a270e700 1 -- 172.21.15.59:0/2996572164 wait complete. 2023-11-24T07:59:50.637 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:50.633+0000 7f23a270e700 1 Processor -- start 2023-11-24T07:59:50.637 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:50.633+0000 7f23a270e700 1 -- start start 2023-11-24T07:59:50.638 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:50.634+0000 7f23a270e700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f239c06b620 0x7f239c0cbd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:50.638 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:50.634+0000 7f23a270e700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f239c0b5ab0 0x7f239c1343e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:50.638 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:50.634+0000 7f23a270e700 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f239c1478c0 0x7f239c136470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:50.638 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:50.634+0000 7f23a270e700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f239c0c81d0 con 0x7f239c06b620 2023-11-24T07:59:50.638 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:50.634+0000 7f23a270e700 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f239c145cc0 con 0x7f239c0b5ab0 2023-11-24T07:59:50.638 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:50.634+0000 7f23a270e700 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f239c136fe0 con 0x7f239c1478c0 2023-11-24T07:59:50.638 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:50.634+0000 7f239bfff700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f239c06b620 0x7f239c0cbd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:50.638 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:50.634+0000 7f239bfff700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f239c06b620 0x7f239c0cbd60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:55766/0 (socket says 172.21.15.59:55766) 2023-11-24T07:59:50.638 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:50.634+0000 7f239bfff700 1 -- 172.21.15.59:0/2942488101 learned_addr learned my addr 172.21.15.59:0/2942488101 (peer_addr_for_me v2:172.21.15.59:0/0) 2023-11-24T07:59:50.639 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:50.634+0000 7f239b7fe700 1 --2- 172.21.15.59:0/2942488101 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f239c0b5ab0 0x7f239c1343e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:50.639 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:50.634+0000 7f23a0cab700 1 --2- 172.21.15.59:0/2942488101 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f239c1478c0 0x7f239c136470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:50.639 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:50.634+0000 7f239bfff700 1 -- 172.21.15.59:0/2942488101 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f239c1478c0 msgr2=0x7f239c136470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:50.639 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:50.634+0000 7f239bfff700 1 --2- 172.21.15.59:0/2942488101 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f239c1478c0 0x7f239c136470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:50.639 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:50.634+0000 7f239bfff700 1 -- 172.21.15.59:0/2942488101 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f239c0b5ab0 msgr2=0x7f239c1343e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:50.639 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:50.634+0000 7f239bfff700 1 --2- 172.21.15.59:0/2942488101 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f239c0b5ab0 0x7f239c1343e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:50.640 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:50.634+0000 7f239bfff700 1 -- 172.21.15.59:0/2942488101 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f238c009550 con 0x7f239c06b620 2023-11-24T07:59:50.640 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:50.634+0000 7f239bfff700 1 --2- 172.21.15.59:0/2942488101 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f239c06b620 0x7f239c0cbd60 secure :-1 s=READY pgs=1012 cs=0 l=1 rev1=1 crypto rx=0x7f239000e810 tx=0x7f239000eb20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:59:50.640 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:50.635+0000 7f2398ff9700 1 -- 172.21.15.59:0/2942488101 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f239000c070 con 0x7f239c06b620 2023-11-24T07:59:50.640 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:50.635+0000 7f2398ff9700 1 -- 172.21.15.59:0/2942488101 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2390013610 con 0x7f239c06b620 2023-11-24T07:59:50.640 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:50.635+0000 7f23a270e700 1 -- 172.21.15.59:0/2942488101 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f239c1372c0 con 0x7f239c06b620 2023-11-24T07:59:50.640 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:50.635+0000 7f2398ff9700 1 -- 172.21.15.59:0/2942488101 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23900139c0 con 0x7f239c06b620 2023-11-24T07:59:50.640 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:50.635+0000 7f23a270e700 1 -- 172.21.15.59:0/2942488101 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f239c0c7e60 con 0x7f239c06b620 2023-11-24T07:59:50.640 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:50.636+0000 7f23a270e700 1 -- 172.21.15.59:0/2942488101 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f239c0b3a40 con 0x7f239c06b620 2023-11-24T07:59:50.641 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:50.636+0000 7f2398ff9700 1 -- 172.21.15.59:0/2942488101 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 11) v1 ==== 89741+0+0 (secure 0 0 0) 0x7f2390013b20 con 0x7f239c06b620 2023-11-24T07:59:50.641 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:50.637+0000 7f2398ff9700 1 --2- 172.21.15.59:0/2942488101 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f238806b470 0x7f238806d930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:50.641 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:50.637+0000 7f2398ff9700 1 -- 172.21.15.59:0/2942488101 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 5227+0+0 (secure 0 0 0) 0x7f2390089d10 con 0x7f239c06b620 2023-11-24T07:59:50.641 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:50.637+0000 7f239b7fe700 1 --2- 172.21.15.59:0/2942488101 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f238806b470 0x7f238806d930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:50.641 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:50.638+0000 7f239b7fe700 1 --2- 172.21.15.59:0/2942488101 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f238806b470 0x7f238806d930 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7f239c0b5f10 tx=0x7f238c008030 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:59:50.643 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:50.642+0000 7f2398ff9700 1 -- 172.21.15.59:0/2942488101 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f23900545f0 con 0x7f239c06b620 2023-11-24T07:59:50.844 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:50.842+0000 7f23a270e700 1 -- 172.21.15.59:0/2942488101 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7f239c075480 con 0x7f239c06b620 2023-11-24T07:59:51.480 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:51.479+0000 7f2398ff9700 1 -- 172.21.15.59:0/2942488101 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v85) v1 ==== 170+0+0 (secure 0 0 0) 0x7f2390057c10 con 0x7f239c06b620 2023-11-24T07:59:51.485 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:51.483+0000 7f23a270e700 1 -- 172.21.15.59:0/2942488101 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f238806b470 msgr2=0x7f238806d930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:51.485 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:51.483+0000 7f23a270e700 1 --2- 172.21.15.59:0/2942488101 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f238806b470 0x7f238806d930 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7f239c0b5f10 tx=0x7f238c008030 comp rx=0 tx=0).stop 2023-11-24T07:59:51.485 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:51.483+0000 7f23a270e700 1 -- 172.21.15.59:0/2942488101 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f239c06b620 msgr2=0x7f239c0cbd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:51.485 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:51.483+0000 7f23a270e700 1 --2- 172.21.15.59:0/2942488101 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f239c06b620 0x7f239c0cbd60 secure :-1 s=READY pgs=1012 cs=0 l=1 rev1=1 crypto rx=0x7f239000e810 tx=0x7f239000eb20 comp rx=0 tx=0).stop 2023-11-24T07:59:51.485 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:51.483+0000 7f23a270e700 1 -- 172.21.15.59:0/2942488101 shutdown_connections 2023-11-24T07:59:51.485 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:51.483+0000 7f23a270e700 1 --2- 172.21.15.59:0/2942488101 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f238806b470 0x7f238806d930 unknown :-1 s=CLOSED pgs=469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:51.485 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:51.483+0000 7f23a270e700 1 --2- 172.21.15.59:0/2942488101 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f239c1478c0 0x7f239c136470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:51.486 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:51.483+0000 7f23a270e700 1 --2- 172.21.15.59:0/2942488101 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f239c06b620 0x7f239c0cbd60 unknown :-1 s=CLOSED pgs=1012 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:51.486 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:51.483+0000 7f23a270e700 1 --2- 172.21.15.59:0/2942488101 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f239c0b5ab0 0x7f239c1343e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:51.486 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:51.483+0000 7f23a270e700 1 -- 172.21.15.59:0/2942488101 >> 172.21.15.59:0/2942488101 conn(0x7f239c0b2740 msgr2=0x7f239c147130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:59:51.486 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:51.483+0000 7f23a270e700 1 -- 172.21.15.59:0/2942488101 shutdown_connections 2023-11-24T07:59:51.486 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:51.483+0000 7f23a270e700 1 -- 172.21.15.59:0/2942488101 wait complete. 2023-11-24T07:59:51.492 INFO:teuthology.orchestra.run.smithi059.stderr:pool 'cephfs_data' created 2023-11-24T07:59:51.515 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2023-11-24T07:59:51.648 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:51.646+0000 7f0f83599700 1 Processor -- start 2023-11-24T07:59:51.648 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:51.647+0000 7f0f83599700 1 -- start start 2023-11-24T07:59:51.652 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:51.648+0000 7f0f83599700 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f0f7c0cf970 0x7f0f7c0cc1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:51.652 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:51.648+0000 7f0f83599700 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0f7c15f5b0 con 0x7f0f7c13dd00 2023-11-24T07:59:51.653 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:51.648+0000 7f0f83599700 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0f7c13e0e0 con 0x7f0f7c071b40 2023-11-24T07:59:51.653 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:51.648+0000 7f0f83599700 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0f7c0d0690 con 0x7f0f7c0cf970 2023-11-24T07:59:51.653 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:51.648+0000 7f0f81335700 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7f0f7c13dd00 0x7f0f7c0cff80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:59248/0 (socket says 172.21.15.59:59248) 2023-11-24T07:59:51.653 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:51.648+0000 7f0f81335700 1 -- 172.21.15.59:0/3903920858 learned_addr learned my addr 172.21.15.59:0/3903920858 (peer_addr_for_me v1:172.21.15.59:0/0) 2023-11-24T07:59:51.653 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:51.648+0000 7f0f80b34700 1 --2- 172.21.15.59:0/3903920858 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f0f7c0cf970 0x7f0f7c0cc1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:51.653 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:51.648+0000 7f0f6ffff700 1 -- 172.21.15.59:0/3903920858 <== mon.0 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3489060712 0 0) 0x7f0f7c15f5b0 con 0x7f0f7c13dd00 2023-11-24T07:59:51.653 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:51.648+0000 7f0f6ffff700 1 -- 172.21.15.59:0/3903920858 --> v1:172.21.15.59:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0f680035b0 con 0x7f0f7c13dd00 2023-11-24T07:59:51.653 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:51.648+0000 7f0f6ffff700 1 -- 172.21.15.59:0/3903920858 <== mon.0 v1:172.21.15.59:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4050867508 0 0) 0x7f0f680035b0 con 0x7f0f7c13dd00 2023-11-24T07:59:51.653 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:51.648+0000 7f0f6ffff700 1 -- 172.21.15.59:0/3903920858 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f0f7c0cf970 msgr2=0x7f0f7c0cc1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T07:59:51.653 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:51.648+0000 7f0f6ffff700 1 --2- 172.21.15.59:0/3903920858 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f0f7c0cf970 0x7f0f7c0cc1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:51.653 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:51.648+0000 7f0f6ffff700 1 -- 172.21.15.59:0/3903920858 >> v1:172.21.15.76:6789/0 conn(0x7f0f7c071b40 legacy=0x7f0f7c0ce7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:51.653 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:51.648+0000 7f0f6ffff700 1 -- 172.21.15.59:0/3903920858 --> v1:172.21.15.59:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0f7c15e9a0 con 0x7f0f7c13dd00 2023-11-24T07:59:51.653 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:51.648+0000 7f0f6ffff700 1 -- 172.21.15.59:0/3903920858 <== mon.0 v1:172.21.15.59:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1312333566 0 0) 0x7f0f78001670 con 0x7f0f7c13dd00 2023-11-24T07:59:51.654 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:51.648+0000 7f0f6ffff700 1 -- 172.21.15.59:0/3903920858 >> v1:172.21.15.59:6789/0 conn(0x7f0f7c13dd00 legacy=0x7f0f7c0cff80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:51.654 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:51.648+0000 7f0f6ffff700 1 --2- 172.21.15.59:0/3903920858 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0f68003a30 0x7f0f68003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:51.654 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:51.648+0000 7f0f6ffff700 1 --2- 172.21.15.59:0/3903920858 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0f680043e0 0x7f0f680048b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:51.654 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:51.648+0000 7f0f6ffff700 1 --2- 172.21.15.59:0/3903920858 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f0f7c0cf970 0x7f0f68005110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:51.654 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:51.648+0000 7f0f6ffff700 1 -- 172.21.15.59:0/3903920858 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f7c0d0690 con 0x7f0f68003a30 2023-11-24T07:59:51.654 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:51.648+0000 7f0f6ffff700 1 -- 172.21.15.59:0/3903920858 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f6800ba10 con 0x7f0f680043e0 2023-11-24T07:59:51.654 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:51.648+0000 7f0f6ffff700 1 -- 172.21.15.59:0/3903920858 --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0f6800bbd0 con 0x7f0f7c0cf970 2023-11-24T07:59:51.654 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:51.648+0000 7f0f81335700 1 --2- 172.21.15.59:0/3903920858 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0f68003a30 0x7f0f68003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:51.654 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:51.649+0000 7f0f80b34700 1 --2- 172.21.15.59:0/3903920858 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0f680043e0 0x7f0f680048b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:51.654 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:51.649+0000 7f0f81b36700 1 --2- 172.21.15.59:0/3903920858 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f0f7c0cf970 0x7f0f68005110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:51.654 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:51.649+0000 7f0f81335700 1 -- 172.21.15.59:0/3903920858 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f0f7c0cf970 msgr2=0x7f0f68005110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T07:59:51.654 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:51.649+0000 7f0f81335700 1 --2- 172.21.15.59:0/3903920858 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f0f7c0cf970 0x7f0f68005110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:51.654 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:51.649+0000 7f0f81335700 1 -- 172.21.15.59:0/3903920858 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0f680043e0 msgr2=0x7f0f680048b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T07:59:51.655 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:51.649+0000 7f0f81335700 1 --2- 172.21.15.59:0/3903920858 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0f680043e0 0x7f0f680048b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:51.655 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:51.649+0000 7f0f81335700 1 -- 172.21.15.59:0/3903920858 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0f680035b0 con 0x7f0f68003a30 2023-11-24T07:59:51.655 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:51.649+0000 7f0f81335700 1 --2- 172.21.15.59:0/3903920858 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0f68003a30 0x7f0f68003e10 secure :-1 s=READY pgs=1014 cs=0 l=1 rev1=1 crypto rx=0x7f0f78007f40 tx=0x7f0f68005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=65944972be748309 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:59:51.655 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:51.649+0000 7f0f6ffff700 1 -- 172.21.15.59:0/3903920858 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f7800ce00 con 0x7f0f68003a30 2023-11-24T07:59:51.655 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:51.649+0000 7f0f6ffff700 1 -- 172.21.15.59:0/3903920858 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0f78005960 con 0x7f0f68003a30 2023-11-24T07:59:51.655 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:51.649+0000 7f0f6ffff700 1 -- 172.21.15.59:0/3903920858 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f78005cf0 con 0x7f0f68003a30 2023-11-24T07:59:51.655 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:51.650+0000 7f0f83599700 1 -- 172.21.15.59:0/3903920858 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0f68003a30 msgr2=0x7f0f68003e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:51.655 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:51.650+0000 7f0f83599700 1 --2- 172.21.15.59:0/3903920858 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0f68003a30 0x7f0f68003e10 secure :-1 s=READY pgs=1014 cs=0 l=1 rev1=1 crypto rx=0x7f0f78007f40 tx=0x7f0f68005880 comp rx=0 tx=0).stop 2023-11-24T07:59:51.655 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:51.650+0000 7f0f81b36700 1 -- 172.21.15.59:0/3903920858 reap_dead start 2023-11-24T07:59:51.655 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:51.650+0000 7f0f83599700 1 -- 172.21.15.59:0/3903920858 shutdown_connections 2023-11-24T07:59:51.655 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:51.650+0000 7f0f83599700 1 -- 172.21.15.59:0/3903920858 >> 172.21.15.59:0/3903920858 conn(0x7f0f7c0d0b10 msgr2=0x7f0f7c0d0f20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:59:51.655 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:51.650+0000 7f0f83599700 1 -- 172.21.15.59:0/3903920858 shutdown_connections 2023-11-24T07:59:51.655 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:51.650+0000 7f0f83599700 1 -- 172.21.15.59:0/3903920858 wait complete. 2023-11-24T07:59:51.656 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:51.651+0000 7f0f83599700 1 Processor -- start 2023-11-24T07:59:51.656 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:51.651+0000 7f0f83599700 1 -- start start 2023-11-24T07:59:51.658 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:51.652+0000 7f0f83599700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0f68003a30 0x7f0f7c0ca580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:51.658 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:51.652+0000 7f0f83599700 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f0f7c13c130 0x7f0f7c13ae60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:51.658 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:51.652+0000 7f0f83599700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0f7c06db60 0x7f0f7c06dfe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:51.658 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:51.652+0000 7f0f83599700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f7c06e520 con 0x7f0f68003a30 2023-11-24T07:59:51.658 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:51.652+0000 7f0f83599700 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f7c153e50 con 0x7f0f7c06db60 2023-11-24T07:59:51.658 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:51.652+0000 7f0f83599700 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0f7c153820 con 0x7f0f7c13c130 2023-11-24T07:59:51.659 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:51.652+0000 7f0f81335700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0f68003a30 0x7f0f7c0ca580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:51.659 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:51.652+0000 7f0f81335700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0f68003a30 0x7f0f7c0ca580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:55786/0 (socket says 172.21.15.59:55786) 2023-11-24T07:59:51.659 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:51.652+0000 7f0f81335700 1 -- 172.21.15.59:0/3052308750 learned_addr learned my addr 172.21.15.59:0/3052308750 (peer_addr_for_me v2:172.21.15.59:0/0) 2023-11-24T07:59:51.659 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:51.652+0000 7f0f80b34700 1 --2- 172.21.15.59:0/3052308750 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f0f7c13c130 0x7f0f7c13ae60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:51.659 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:51.652+0000 7f0f81b36700 1 --2- 172.21.15.59:0/3052308750 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0f7c06db60 0x7f0f7c06dfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:51.659 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:51.652+0000 7f0f81335700 1 -- 172.21.15.59:0/3052308750 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f0f7c13c130 msgr2=0x7f0f7c13ae60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:51.659 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:51.652+0000 7f0f81335700 1 --2- 172.21.15.59:0/3052308750 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f0f7c13c130 0x7f0f7c13ae60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:51.659 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:51.652+0000 7f0f81335700 1 -- 172.21.15.59:0/3052308750 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0f7c06db60 msgr2=0x7f0f7c06dfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:51.659 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:51.652+0000 7f0f81335700 1 --2- 172.21.15.59:0/3052308750 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0f7c06db60 0x7f0f7c06dfe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:51.659 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:51.652+0000 7f0f81335700 1 -- 172.21.15.59:0/3052308750 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0f7c13c6a0 con 0x7f0f68003a30 2023-11-24T07:59:51.659 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:51.652+0000 7f0f81335700 1 --2- 172.21.15.59:0/3052308750 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0f68003a30 0x7f0f7c0ca580 secure :-1 s=READY pgs=1015 cs=0 l=1 rev1=1 crypto rx=0x7f0f7800c8b0 tx=0x7f0f78001a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:59:51.659 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:51.652+0000 7f0f6dffb700 1 -- 172.21.15.59:0/3052308750 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f78004870 con 0x7f0f68003a30 2023-11-24T07:59:51.659 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:51.652+0000 7f0f6dffb700 1 -- 172.21.15.59:0/3052308750 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0f780049d0 con 0x7f0f68003a30 2023-11-24T07:59:51.660 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:51.652+0000 7f0f83599700 1 -- 172.21.15.59:0/3052308750 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0f7c13c980 con 0x7f0f68003a30 2023-11-24T07:59:51.660 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:51.653+0000 7f0f6dffb700 1 -- 172.21.15.59:0/3052308750 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f78004d80 con 0x7f0f68003a30 2023-11-24T07:59:51.660 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:51.653+0000 7f0f83599700 1 -- 172.21.15.59:0/3052308750 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0f7c0ce3a0 con 0x7f0f68003a30 2023-11-24T07:59:51.660 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:51.654+0000 7f0f83599700 1 -- 172.21.15.59:0/3052308750 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0f7c154130 con 0x7f0f68003a30 2023-11-24T07:59:51.660 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:51.655+0000 7f0f6dffb700 1 -- 172.21.15.59:0/3052308750 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 11) v1 ==== 89741+0+0 (secure 0 0 0) 0x7f0f78007070 con 0x7f0f68003a30 2023-11-24T07:59:51.660 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:51.655+0000 7f0f6dffb700 1 --2- 172.21.15.59:0/3052308750 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f0f6006b4a0 0x7f0f6006d960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:51.660 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:51.655+0000 7f0f6dffb700 1 -- 172.21.15.59:0/3052308750 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 5609+0+0 (secure 0 0 0) 0x7f0f7808dad0 con 0x7f0f68003a30 2023-11-24T07:59:51.660 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:51.656+0000 7f0f80b34700 1 --2- 172.21.15.59:0/3052308750 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f0f6006b4a0 0x7f0f6006d960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:51.660 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:51.656+0000 7f0f80b34700 1 --2- 172.21.15.59:0/3052308750 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f0f6006b4a0 0x7f0f6006d960 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7f0f7c0ce630 tx=0x7f0f70009210 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:59:51.661 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:51.659+0000 7f0f6dffb700 1 -- 172.21.15.59:0/3052308750 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f0f78058620 con 0x7f0f68003a30 2023-11-24T07:59:51.912 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:51.907+0000 7f0f83599700 1 -- 172.21.15.59:0/3052308750 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7f0f7c075480 con 0x7f0f68003a30 2023-11-24T07:59:52.501 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:52.499+0000 7f0f6dffb700 1 -- 172.21.15.59:0/3052308750 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 12 and data pool 13 v49) v1 ==== 174+0+0 (secure 0 0 0) 0x7f0f78062070 con 0x7f0f68003a30 2023-11-24T07:59:52.505 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:52.503+0000 7f0f83599700 1 -- 172.21.15.59:0/3052308750 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f0f6006b4a0 msgr2=0x7f0f6006d960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:52.505 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:52.503+0000 7f0f83599700 1 --2- 172.21.15.59:0/3052308750 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f0f6006b4a0 0x7f0f6006d960 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7f0f7c0ce630 tx=0x7f0f70009210 comp rx=0 tx=0).stop 2023-11-24T07:59:52.505 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:52.503+0000 7f0f83599700 1 -- 172.21.15.59:0/3052308750 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0f68003a30 msgr2=0x7f0f7c0ca580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:52.505 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:52.503+0000 7f0f83599700 1 --2- 172.21.15.59:0/3052308750 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0f68003a30 0x7f0f7c0ca580 secure :-1 s=READY pgs=1015 cs=0 l=1 rev1=1 crypto rx=0x7f0f7800c8b0 tx=0x7f0f78001a00 comp rx=0 tx=0).stop 2023-11-24T07:59:52.505 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:52.503+0000 7f0f83599700 1 -- 172.21.15.59:0/3052308750 shutdown_connections 2023-11-24T07:59:52.505 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:52.503+0000 7f0f83599700 1 --2- 172.21.15.59:0/3052308750 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f0f6006b4a0 0x7f0f6006d960 secure :-1 s=CLOSED pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7f0f7c0ce630 tx=0x7f0f70009210 comp rx=0 tx=0).stop 2023-11-24T07:59:52.505 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:52.503+0000 7f0f83599700 1 --2- 172.21.15.59:0/3052308750 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0f7c06db60 0x7f0f7c06dfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:52.505 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:52.503+0000 7f0f83599700 1 --2- 172.21.15.59:0/3052308750 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0f68003a30 0x7f0f7c0ca580 secure :-1 s=CLOSED pgs=1015 cs=0 l=1 rev1=1 crypto rx=0x7f0f7800c8b0 tx=0x7f0f78001a00 comp rx=0 tx=0).stop 2023-11-24T07:59:52.505 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:52.503+0000 7f0f83599700 1 --2- 172.21.15.59:0/3052308750 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f0f7c13c130 0x7f0f7c13ae60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:52.505 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:52.503+0000 7f0f83599700 1 -- 172.21.15.59:0/3052308750 >> 172.21.15.59:0/3052308750 conn(0x7f0f7c0d0b10 msgr2=0x7f0f7c1563e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:59:52.506 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:52.504+0000 7f0f83599700 1 -- 172.21.15.59:0/3052308750 shutdown_connections 2023-11-24T07:59:52.506 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:52.504+0000 7f0f83599700 1 -- 172.21.15.59:0/3052308750 wait complete. 2023-11-24T07:59:52.520 INFO:teuthology.orchestra.run.smithi059.stderr:new fs with metadata pool 12 and data pool 13 2023-11-24T07:59:52.545 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-24T07:59:52.674 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:52.672+0000 7f2affec9700 1 Processor -- start 2023-11-24T07:59:52.674 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:52.673+0000 7f2affec9700 1 -- start start 2023-11-24T07:59:52.676 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:52.673+0000 7f2affec9700 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f2af815e460 0x7f2af80d49d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:52.676 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:52.673+0000 7f2affec9700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f2af80d2ba0 0x7f2af80b40a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:52.676 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:52.673+0000 7f2affec9700 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2af80b3370 con 0x7f2af80cf480 2023-11-24T07:59:52.677 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:52.673+0000 7f2affec9700 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2af80cfc40 con 0x7f2af80d2ba0 2023-11-24T07:59:52.677 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:52.673+0000 7f2affec9700 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2af80b35c0 con 0x7f2af815e460 2023-11-24T07:59:52.677 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:52.673+0000 7f2afe466700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f2af80d2ba0 0x7f2af80b40a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:52.677 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:52.674+0000 7f2afe466700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f2af80d2ba0 0x7f2af80b40a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:55810/0 (socket says 172.21.15.59:55810) 2023-11-24T07:59:52.677 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:52.674+0000 7f2afe466700 1 -- 172.21.15.59:0/3065545577 learned_addr learned my addr 172.21.15.59:0/3065545577 (peer_addr_for_me v2:172.21.15.59:0/0) 2023-11-24T07:59:52.677 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:52.674+0000 7f2afdc65700 1 --2- 172.21.15.59:0/3065545577 >> v2:172.21.15.76:3300/0 conn(0x7f2af815e460 0x7f2af80d49d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:52.677 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:52.674+0000 7f2afe466700 1 -- 172.21.15.59:0/3065545577 >> v2:172.21.15.76:3300/0 conn(0x7f2af815e460 msgr2=0x7f2af80d49d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T07:59:52.677 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:52.674+0000 7f2afe466700 1 --2- 172.21.15.59:0/3065545577 >> v2:172.21.15.76:3300/0 conn(0x7f2af815e460 0x7f2af80d49d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:52.677 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:52.674+0000 7f2afe466700 1 -- 172.21.15.59:0/3065545577 >> v1:172.21.15.76:6789/0 conn(0x7f2af80cf480 legacy=0x7f2af80d2490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:52.677 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:52.674+0000 7f2afe466700 1 -- 172.21.15.59:0/3065545577 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2af80b38a0 con 0x7f2af80d2ba0 2023-11-24T07:59:52.677 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:52.674+0000 7f2afe466700 1 --2- 172.21.15.59:0/3065545577 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2af80d2ba0 0x7f2af80b40a0 secure :-1 s=READY pgs=1016 cs=0 l=1 rev1=1 crypto rx=0x7f2af80b6a30 tx=0x7f2af000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ef47f5397d4d16b3 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:59:52.677 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:52.674+0000 7f2afcc63700 1 -- 172.21.15.59:0/3065545577 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2af000e070 con 0x7f2af80d2ba0 2023-11-24T07:59:52.677 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:52.674+0000 7f2afcc63700 1 -- 172.21.15.59:0/3065545577 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2af000b9c0 con 0x7f2af80d2ba0 2023-11-24T07:59:52.677 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:52.674+0000 7f2afcc63700 1 -- 172.21.15.59:0/3065545577 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2af000bd70 con 0x7f2af80d2ba0 2023-11-24T07:59:52.678 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:52.675+0000 7f2affec9700 1 -- 172.21.15.59:0/3065545577 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2af80d2ba0 msgr2=0x7f2af80b40a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:52.678 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:52.675+0000 7f2affec9700 1 --2- 172.21.15.59:0/3065545577 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2af80d2ba0 0x7f2af80b40a0 secure :-1 s=READY pgs=1016 cs=0 l=1 rev1=1 crypto rx=0x7f2af80b6a30 tx=0x7f2af000b330 comp rx=0 tx=0).stop 2023-11-24T07:59:52.678 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:52.675+0000 7f2affec9700 1 -- 172.21.15.59:0/3065545577 shutdown_connections 2023-11-24T07:59:52.678 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:52.675+0000 7f2affec9700 1 --2- 172.21.15.59:0/3065545577 >> v2:172.21.15.76:3300/0 conn(0x7f2af815e460 0x7f2af80d49d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:52.678 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:52.675+0000 7f2affec9700 1 --2- 172.21.15.59:0/3065545577 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2af80d2ba0 0x7f2af80b40a0 unknown :-1 s=CLOSED pgs=1016 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:52.678 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:52.675+0000 7f2affec9700 1 -- 172.21.15.59:0/3065545577 >> 172.21.15.59:0/3065545577 conn(0x7f2af80d3eb0 msgr2=0x7f2af80d42c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:59:52.678 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:52.675+0000 7f2affec9700 1 -- 172.21.15.59:0/3065545577 shutdown_connections 2023-11-24T07:59:52.678 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:52.675+0000 7f2affec9700 1 -- 172.21.15.59:0/3065545577 wait complete. 2023-11-24T07:59:52.680 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:52.676+0000 7f2affec9700 1 Processor -- start 2023-11-24T07:59:52.680 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:52.676+0000 7f2affec9700 1 -- start start 2023-11-24T07:59:52.681 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:52.677+0000 7f2affec9700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2af80cf480 0x7f2af81e3550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:52.681 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:52.677+0000 7f2affec9700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2af80d2ba0 0x7f2af81e5580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:52.681 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:52.677+0000 7f2affec9700 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f2af815e460 0x7f2af814fde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:52.681 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:52.677+0000 7f2affec9700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2af81e5f70 con 0x7f2af80cf480 2023-11-24T07:59:52.681 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:52.677+0000 7f2affec9700 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2af81e6250 con 0x7f2af80d2ba0 2023-11-24T07:59:52.681 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:52.677+0000 7f2affec9700 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2af81e6530 con 0x7f2af815e460 2023-11-24T07:59:52.681 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:52.677+0000 7f2afdc65700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2af80cf480 0x7f2af81e3550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:52.681 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:52.677+0000 7f2afdc65700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2af80cf480 0x7f2af81e3550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:55812/0 (socket says 172.21.15.59:55812) 2023-11-24T07:59:52.682 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:52.677+0000 7f2afdc65700 1 -- 172.21.15.59:0/3199779881 learned_addr learned my addr 172.21.15.59:0/3199779881 (peer_addr_for_me v2:172.21.15.59:0/0) 2023-11-24T07:59:52.682 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:52.677+0000 7f2afd464700 1 --2- 172.21.15.59:0/3199779881 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2af80d2ba0 0x7f2af81e5580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:52.682 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:52.677+0000 7f2afe466700 1 --2- 172.21.15.59:0/3199779881 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f2af815e460 0x7f2af814fde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:52.682 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:52.677+0000 7f2afdc65700 1 -- 172.21.15.59:0/3199779881 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f2af815e460 msgr2=0x7f2af814fde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:52.682 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:52.677+0000 7f2afdc65700 1 --2- 172.21.15.59:0/3199779881 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f2af815e460 0x7f2af814fde0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:52.682 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:52.677+0000 7f2afdc65700 1 -- 172.21.15.59:0/3199779881 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2af80d2ba0 msgr2=0x7f2af81e5580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:52.682 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:52.677+0000 7f2afdc65700 1 --2- 172.21.15.59:0/3199779881 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2af80d2ba0 0x7f2af81e5580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:52.682 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:52.677+0000 7f2afdc65700 1 -- 172.21.15.59:0/3199779881 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2af0009b30 con 0x7f2af80cf480 2023-11-24T07:59:52.682 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:52.677+0000 7f2afdc65700 1 --2- 172.21.15.59:0/3199779881 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2af80cf480 0x7f2af81e3550 secure :-1 s=READY pgs=1017 cs=0 l=1 rev1=1 crypto rx=0x7f2ae800d800 tx=0x7f2ae800dbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:59:52.682 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:52.678+0000 7f2aee7fc700 1 -- 172.21.15.59:0/3199779881 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ae80125f0 con 0x7f2af80cf480 2023-11-24T07:59:52.682 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:52.678+0000 7f2aee7fc700 1 -- 172.21.15.59:0/3199779881 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2ae8012750 con 0x7f2af80cf480 2023-11-24T07:59:52.683 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:52.678+0000 7f2affec9700 1 -- 172.21.15.59:0/3199779881 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2af81e6810 con 0x7f2af80cf480 2023-11-24T07:59:52.683 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:52.678+0000 7f2aee7fc700 1 -- 172.21.15.59:0/3199779881 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ae8012b00 con 0x7f2af80cf480 2023-11-24T07:59:52.683 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:52.678+0000 7f2affec9700 1 -- 172.21.15.59:0/3199779881 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2af815d390 con 0x7f2af80cf480 2023-11-24T07:59:52.683 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:52.679+0000 7f2affec9700 1 -- 172.21.15.59:0/3199779881 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2af81f40f0 con 0x7f2af80cf480 2023-11-24T07:59:52.683 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:52.679+0000 7f2aee7fc700 1 -- 172.21.15.59:0/3199779881 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 11) v1 ==== 89741+0+0 (secure 0 0 0) 0x7f2ae8012c60 con 0x7f2af80cf480 2023-11-24T07:59:52.684 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:52.680+0000 7f2aee7fc700 1 --2- 172.21.15.59:0/3199779881 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f2adc06b470 0x7f2adc06d930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:52.684 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:52.680+0000 7f2aee7fc700 1 -- 172.21.15.59:0/3199779881 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f2ae808a3f0 con 0x7f2af80cf480 2023-11-24T07:59:52.684 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:52.680+0000 7f2afd464700 1 --2- 172.21.15.59:0/3199779881 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f2adc06b470 0x7f2adc06d930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:52.684 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:52.680+0000 7f2afd464700 1 --2- 172.21.15.59:0/3199779881 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f2adc06b470 0x7f2adc06d930 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7f2af815dc40 tx=0x7f2af4009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:59:52.686 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:52.684+0000 7f2aee7fc700 1 -- 172.21.15.59:0/3199779881 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f2af81f40f0 con 0x7f2af80cf480 2023-11-24T07:59:52.888 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:52.887+0000 7f2affec9700 1 -- 172.21.15.59:0/3199779881 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f2af81f40f0 con 0x7f2af80cf480 2023-11-24T07:59:52.889 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:52.888+0000 7f2aee7fc700 1 -- 172.21.15.59:0/3199779881 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v86) v1 ==== 74+0+16513 (secure 0 0 0) 0x7f2af81f40f0 con 0x7f2af80cf480 2023-11-24T07:59:52.890 INFO:teuthology.orchestra.run.smithi059.stdout: 2023-11-24T07:59:52.890 INFO:teuthology.orchestra.run.smithi059.stdout:{"epoch":86,"fsid":"9f0e8741-b8f9-48d5-a2c3-09f45903303d","created":"2023-11-24T07:56:06.552729+0000","modified":"2023-11-24T07:59:51.909277+0000","last_up_change":"2023-11-24T07:56:17.372352+0000","last_in_change":"2023-11-24T07:56:09.073250+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":13,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-24T07:56:18.352823+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_metadata","create_time":"2023-11-24T07:59:50.274996+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_data","create_time":"2023-11-24T07:59:50.873919+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"6d8f9e9b-1782-4e41-928b-a39f2a8ebc06","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6800","nonce":3373878772},{"type":"v1","addr":"172.21.15.59:6801","nonce":3373878772}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6802","nonce":3373878772},{"type":"v1","addr":"172.21.15.59:6803","nonce":3373878772}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6806","nonce":3373878772},{"type":"v1","addr":"172.21.15.59:6807","nonce":3373878772}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6804","nonce":3373878772},{"type":"v1","addr":"172.21.15.59:6805","nonce":3373878772}]},"public_addr":"172.21.15.59:6801/3373878772","cluster_addr":"172.21.15.59:6803/3373878772","heartbeat_back_addr":"172.21.15.59:6807/3373878772","heartbeat_front_addr":"172.21.15.59:6805/3373878772","state":["exists","up"]},{"osd":1,"uuid":"e8d882aa-52e1-4c04-9e1e-c44ef9204a5f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6824","nonce":648258563},{"type":"v1","addr":"172.21.15.59:6825","nonce":648258563}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6826","nonce":648258563},{"type":"v1","addr":"172.21.15.59:6827","nonce":648258563}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6830","nonce":648258563},{"type":"v1","addr":"172.21.15.59:6831","nonce":648258563}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6828","nonce":648258563},{"type":"v1","addr":"172.21.15.59:6829","nonce":648258563}]},"public_addr":"172.21.15.59:6825/648258563","cluster_addr":"172.21.15.59:6827/648258563","heartbeat_back_addr":"172.21.15.59:6831/648258563","heartbeat_front_addr":"172.21.15.59:6829/648258563","state":["exists","up"]},{"osd":2,"uuid":"69eebfa0-695b-4b21-90b9-8a3537168a01","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6808","nonce":1840430654},{"type":"v1","addr":"172.21.15.59:6809","nonce":1840430654}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6810","nonce":1840430654},{"type":"v1","addr":"172.21.15.59:6811","nonce":1840430654}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6814","nonce":1840430654},{"type":"v1","addr":"172.21.15.59:6815","nonce":1840430654}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6812","nonce":1840430654},{"type":"v1","addr":"172.21.15.59:6813","nonce":1840430654}]},"public_addr":"172.21.15.59:6809/1840430654","cluster_addr":"172.21.15.59:6811/1840430654","heartbeat_back_addr":"172.21.15.59:6815/1840430654","heartbeat_front_addr":"172.21.15.59:6813/1840430654","state":["exists","up"]},{"osd":3,"uuid":"96fd77fc-f911-46ff-9445-6f0be2daa8f4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6816","nonce":2504025850},{"type":"v1","addr":"172.21.15.59:6817","nonce":2504025850}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6818","nonce":2504025850},{"type":"v1","addr":"172.21.15.59:6819","nonce":2504025850}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6822","nonce":2504025850},{"type":"v1","addr":"172.21.15.59:6823","nonce":2504025850}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6820","nonce":2504025850},{"type":"v1","addr":"172.21.15.59:6821","nonce":2504025850}]},"public_addr":"172.21.15.59:6817/2504025850","cluster_addr":"172.21.15.59:6819/2504025850","heartbeat_back_addr":"172.21.15.59:6823/2504025850","heartbeat_front_addr":"172.21.15.59:6821/2504025850","state":["exists","up"]},{"osd":4,"uuid":"7ac29fca-b5b2-4c95-9d69-2f164cbbc708","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6816","nonce":2382463431},{"type":"v1","addr":"172.21.15.76:6818","nonce":2382463431}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6820","nonce":2382463431},{"type":"v1","addr":"172.21.15.76:6822","nonce":2382463431}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6828","nonce":2382463431},{"type":"v1","addr":"172.21.15.76:6830","nonce":2382463431}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6824","nonce":2382463431},{"type":"v1","addr":"172.21.15.76:6826","nonce":2382463431}]},"public_addr":"172.21.15.76:6818/2382463431","cluster_addr":"172.21.15.76:6822/2382463431","heartbeat_back_addr":"172.21.15.76:6830/2382463431","heartbeat_front_addr":"172.21.15.76:6826/2382463431","state":["exists","up"]},{"osd":5,"uuid":"485bf711-3851-4238-a4f4-51d52e0a9c58","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6800","nonce":329334308},{"type":"v1","addr":"172.21.15.76:6802","nonce":329334308}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6805","nonce":329334308},{"type":"v1","addr":"172.21.15.76:6807","nonce":329334308}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6814","nonce":329334308},{"type":"v1","addr":"172.21.15.76:6815","nonce":329334308}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6810","nonce":329334308},{"type":"v1","addr":"172.21.15.76:6812","nonce":329334308}]},"public_addr":"172.21.15.76:6802/329334308","cluster_addr":"172.21.15.76:6807/329334308","heartbeat_back_addr":"172.21.15.76:6815/329334308","heartbeat_front_addr":"172.21.15.76:6812/329334308","state":["exists","up"]},{"osd":6,"uuid":"72b3f2fc-1e4a-4e32-968d-003647a0286f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6801","nonce":3946579410},{"type":"v1","addr":"172.21.15.76:6803","nonce":3946579410}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6804","nonce":3946579410},{"type":"v1","addr":"172.21.15.76:6806","nonce":3946579410}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6811","nonce":3946579410},{"type":"v1","addr":"172.21.15.76:6813","nonce":3946579410}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6808","nonce":3946579410},{"type":"v1","addr":"172.21.15.76:6809","nonce":3946579410}]},"public_addr":"172.21.15.76:6803/3946579410","cluster_addr":"172.21.15.76:6806/3946579410","heartbeat_back_addr":"172.21.15.76:6813/3946579410","heartbeat_front_addr":"172.21.15.76:6809/3946579410","state":["exists","up"]},{"osd":7,"uuid":"1ddceb90-6c5d-4d1a-ad56-99cb877f8c70","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6817","nonce":841761538},{"type":"v1","addr":"172.21.15.76:6819","nonce":841761538}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6821","nonce":841761538},{"type":"v1","addr":"172.21.15.76:6823","nonce":841761538}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6829","nonce":841761538},{"type":"v1","addr":"172.21.15.76:6831","nonce":841761538}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6825","nonce":841761538},{"type":"v1","addr":"172.21.15.76:6827","nonce":841761538}]},"public_addr":"172.21.15.76:6819/841761538","cluster_addr":"172.21.15.76:6823/841761538","heartbeat_back_addr":"172.21.15.76:6831/841761538","heartbeat_front_addr":"172.21.15.76:6827/841761538","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T07:56:14.294838+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T07:56:14.270779+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T07:56:14.231809+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T07:56:14.260983+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T07:56:14.302997+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T07:56:14.258600+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T07:56:14.251413+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T07:56:14.304727+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-24T07:59:52.892 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:52.890+0000 7f2affec9700 1 -- 172.21.15.59:0/3199779881 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f2adc06b470 msgr2=0x7f2adc06d930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:52.892 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:52.890+0000 7f2affec9700 1 --2- 172.21.15.59:0/3199779881 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f2adc06b470 0x7f2adc06d930 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7f2af815dc40 tx=0x7f2af4009040 comp rx=0 tx=0).stop 2023-11-24T07:59:52.892 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:52.890+0000 7f2affec9700 1 -- 172.21.15.59:0/3199779881 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2af80cf480 msgr2=0x7f2af81e3550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:52.892 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:52.890+0000 7f2affec9700 1 --2- 172.21.15.59:0/3199779881 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2af80cf480 0x7f2af81e3550 secure :-1 s=READY pgs=1017 cs=0 l=1 rev1=1 crypto rx=0x7f2ae800d800 tx=0x7f2ae800dbc0 comp rx=0 tx=0).stop 2023-11-24T07:59:52.892 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:52.890+0000 7f2affec9700 1 -- 172.21.15.59:0/3199779881 shutdown_connections 2023-11-24T07:59:52.892 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:52.890+0000 7f2affec9700 1 --2- 172.21.15.59:0/3199779881 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f2adc06b470 0x7f2adc06d930 unknown :-1 s=CLOSED pgs=471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:52.892 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:52.890+0000 7f2affec9700 1 --2- 172.21.15.59:0/3199779881 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f2af815e460 0x7f2af814fde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:52.892 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:52.890+0000 7f2affec9700 1 --2- 172.21.15.59:0/3199779881 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2af80cf480 0x7f2af81e3550 unknown :-1 s=CLOSED pgs=1017 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:52.893 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:52.890+0000 7f2affec9700 1 --2- 172.21.15.59:0/3199779881 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2af80d2ba0 0x7f2af81e5580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:52.893 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:52.890+0000 7f2affec9700 1 -- 172.21.15.59:0/3199779881 >> 172.21.15.59:0/3199779881 conn(0x7f2af80d3eb0 msgr2=0x7f2af80b8ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:59:52.893 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:52.890+0000 7f2affec9700 1 -- 172.21.15.59:0/3199779881 shutdown_connections 2023-11-24T07:59:52.893 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:52.890+0000 7f2affec9700 1 -- 172.21.15.59:0/3199779881 wait complete. 2023-11-24T07:59:52.904 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-24T07:59:53.029 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:53.027+0000 7fba1a254700 1 Processor -- start 2023-11-24T07:59:53.030 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:53.027+0000 7fba1a254700 1 -- start start 2023-11-24T07:59:53.032 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:53.028+0000 7fba1a254700 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fba140b4c40 0x7fba1406dba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:53.032 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:53.028+0000 7fba1a254700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fba1406f420 0x7fba140ce1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:53.032 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:53.028+0000 7fba1a254700 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fba1406ef90 con 0x7fba140b3b40 2023-11-24T07:59:53.032 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:53.028+0000 7fba1a254700 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fba1406e0e0 con 0x7fba1406f420 2023-11-24T07:59:53.032 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:53.028+0000 7fba1a254700 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fba1406e380 con 0x7fba140b4c40 2023-11-24T07:59:53.032 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:53.028+0000 7fba13fff700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fba1406f420 0x7fba140ce1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:53.032 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:53.028+0000 7fba13fff700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fba1406f420 0x7fba140ce1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:55824/0 (socket says 172.21.15.59:55824) 2023-11-24T07:59:53.032 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:53.028+0000 7fba13fff700 1 -- 172.21.15.59:0/4139248426 learned_addr learned my addr 172.21.15.59:0/4139248426 (peer_addr_for_me v2:172.21.15.59:0/0) 2023-11-24T07:59:53.032 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:53.028+0000 7fba137fe700 1 --2- 172.21.15.59:0/4139248426 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fba140b4c40 0x7fba1406dba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:53.032 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:53.029+0000 7fba13fff700 1 -- 172.21.15.59:0/4139248426 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fba140b4c40 msgr2=0x7fba1406dba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T07:59:53.032 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:53.029+0000 7fba13fff700 1 --2- 172.21.15.59:0/4139248426 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fba140b4c40 0x7fba1406dba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:53.033 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:53.029+0000 7fba13fff700 1 -- 172.21.15.59:0/4139248426 >> v1:172.21.15.76:6789/0 conn(0x7fba140b3b40 legacy=0x7fba140b5020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:53.033 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:53.029+0000 7fba13fff700 1 -- 172.21.15.59:0/4139248426 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fba1406e660 con 0x7fba1406f420 2023-11-24T07:59:53.033 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:53.029+0000 7fba13fff700 1 --2- 172.21.15.59:0/4139248426 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fba1406f420 0x7fba140ce1d0 secure :-1 s=READY pgs=1018 cs=0 l=1 rev1=1 crypto rx=0x7fba0c009e50 tx=0x7fba0c00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fa640e1b1427b7c7 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:59:53.033 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:53.029+0000 7fba127fc700 1 -- 172.21.15.59:0/4139248426 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba0c00e070 con 0x7fba1406f420 2023-11-24T07:59:53.033 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:53.029+0000 7fba127fc700 1 -- 172.21.15.59:0/4139248426 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fba0c00b910 con 0x7fba1406f420 2023-11-24T07:59:53.033 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:53.029+0000 7fba127fc700 1 -- 172.21.15.59:0/4139248426 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba0c00bcc0 con 0x7fba1406f420 2023-11-24T07:59:53.033 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:53.030+0000 7fba1a254700 1 -- 172.21.15.59:0/4139248426 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fba1406f420 msgr2=0x7fba140ce1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:53.033 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:53.030+0000 7fba1a254700 1 --2- 172.21.15.59:0/4139248426 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fba1406f420 0x7fba140ce1d0 secure :-1 s=READY pgs=1018 cs=0 l=1 rev1=1 crypto rx=0x7fba0c009e50 tx=0x7fba0c00b330 comp rx=0 tx=0).stop 2023-11-24T07:59:53.033 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:53.030+0000 7fba1a254700 1 -- 172.21.15.59:0/4139248426 shutdown_connections 2023-11-24T07:59:53.033 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:53.030+0000 7fba1a254700 1 --2- 172.21.15.59:0/4139248426 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fba140b4c40 0x7fba1406dba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:53.033 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:53.030+0000 7fba1a254700 1 --2- 172.21.15.59:0/4139248426 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fba1406f420 0x7fba140ce1d0 unknown :-1 s=CLOSED pgs=1018 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:53.033 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:53.030+0000 7fba1a254700 1 -- 172.21.15.59:0/4139248426 >> 172.21.15.59:0/4139248426 conn(0x7fba140b2840 msgr2=0x7fba140b2c20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:59:53.034 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:53.030+0000 7fba1a254700 1 -- 172.21.15.59:0/4139248426 shutdown_connections 2023-11-24T07:59:53.034 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:53.030+0000 7fba1a254700 1 -- 172.21.15.59:0/4139248426 wait complete. 2023-11-24T07:59:53.035 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:53.031+0000 7fba1a254700 1 Processor -- start 2023-11-24T07:59:53.036 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:53.031+0000 7fba1a254700 1 -- start start 2023-11-24T07:59:53.036 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:53.032+0000 7fba1a254700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fba1406f420 0x7fba14143ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:53.036 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:53.032+0000 7fba1a254700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fba140b3b40 0x7fba140af360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:53.036 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:53.032+0000 7fba1a254700 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fba140b4c40 0x7fba1406afd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:53.036 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:53.032+0000 7fba1a254700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba140af8a0 con 0x7fba140b3b40 2023-11-24T07:59:53.036 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:53.032+0000 7fba1a254700 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba14143a50 con 0x7fba1406f420 2023-11-24T07:59:53.036 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:53.032+0000 7fba1a254700 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fba141f5460 con 0x7fba140b4c40 2023-11-24T07:59:53.036 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:53.032+0000 7fba12ffd700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fba140b3b40 0x7fba140af360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:53.036 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:53.032+0000 7fba12ffd700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fba140b3b40 0x7fba140af360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:55828/0 (socket says 172.21.15.59:55828) 2023-11-24T07:59:53.036 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:53.032+0000 7fba12ffd700 1 -- 172.21.15.59:0/3329872829 learned_addr learned my addr 172.21.15.59:0/3329872829 (peer_addr_for_me v2:172.21.15.59:0/0) 2023-11-24T07:59:53.036 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:53.032+0000 7fba137fe700 1 --2- 172.21.15.59:0/3329872829 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fba1406f420 0x7fba14143ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:53.036 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:53.032+0000 7fba13fff700 1 --2- 172.21.15.59:0/3329872829 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fba140b4c40 0x7fba1406afd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:53.037 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:53.032+0000 7fba12ffd700 1 -- 172.21.15.59:0/3329872829 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fba140b4c40 msgr2=0x7fba1406afd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:53.037 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:53.032+0000 7fba12ffd700 1 --2- 172.21.15.59:0/3329872829 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fba140b4c40 0x7fba1406afd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:53.037 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:53.032+0000 7fba12ffd700 1 -- 172.21.15.59:0/3329872829 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fba1406f420 msgr2=0x7fba14143ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:53.037 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:53.032+0000 7fba12ffd700 1 --2- 172.21.15.59:0/3329872829 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fba1406f420 0x7fba14143ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:53.037 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:53.032+0000 7fba12ffd700 1 -- 172.21.15.59:0/3329872829 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fba0c009b30 con 0x7fba140b3b40 2023-11-24T07:59:53.037 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:53.032+0000 7fba12ffd700 1 --2- 172.21.15.59:0/3329872829 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fba140b3b40 0x7fba140af360 secure :-1 s=READY pgs=1019 cs=0 l=1 rev1=1 crypto rx=0x7fb9fc00c4e0 tx=0x7fb9fc00c8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:59:53.037 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:53.033+0000 7fba0bfff700 1 -- 172.21.15.59:0/3329872829 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9fc0052d0 con 0x7fba140b3b40 2023-11-24T07:59:53.037 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:53.033+0000 7fba0bfff700 1 -- 172.21.15.59:0/3329872829 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb9fc005430 con 0x7fba140b3b40 2023-11-24T07:59:53.037 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:53.033+0000 7fba1a254700 1 -- 172.21.15.59:0/3329872829 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fba14143400 con 0x7fba140b3b40 2023-11-24T07:59:53.037 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:53.033+0000 7fba0bfff700 1 -- 172.21.15.59:0/3329872829 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9fc0038c0 con 0x7fba140b3b40 2023-11-24T07:59:53.037 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:53.033+0000 7fba1a254700 1 -- 172.21.15.59:0/3329872829 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fba14143060 con 0x7fba140b3b40 2023-11-24T07:59:53.037 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:53.034+0000 7fba1a254700 1 -- 172.21.15.59:0/3329872829 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fba14143d30 con 0x7fba140b3b40 2023-11-24T07:59:53.037 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:53.034+0000 7fba0bfff700 1 -- 172.21.15.59:0/3329872829 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 11) v1 ==== 89741+0+0 (secure 0 0 0) 0x7fb9fc003a20 con 0x7fba140b3b40 2023-11-24T07:59:53.038 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:53.035+0000 7fba0bfff700 1 --2- 172.21.15.59:0/3329872829 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7fb9f406b470 0x7fb9f406d930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:53.038 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:53.035+0000 7fba0bfff700 1 -- 172.21.15.59:0/3329872829 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fb9fc089530 con 0x7fba140b3b40 2023-11-24T07:59:53.038 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:53.035+0000 7fba137fe700 1 --2- 172.21.15.59:0/3329872829 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7fb9f406b470 0x7fb9f406d930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:53.038 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:53.035+0000 7fba137fe700 1 --2- 172.21.15.59:0/3329872829 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7fb9f406b470 0x7fb9f406d930 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7fba1415d9a0 tx=0x7fba04008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:59:53.041 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:53.039+0000 7fba0bfff700 1 -- 172.21.15.59:0/3329872829 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fb9fc053d70 con 0x7fba140b3b40 2023-11-24T07:59:53.242 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:53.241+0000 7fba1a254700 1 -- 172.21.15.59:0/3329872829 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fba14067c70 con 0x7fba140b3b40 2023-11-24T07:59:53.243 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:53.242+0000 7fba0bfff700 1 -- 172.21.15.59:0/3329872829 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v86) v1 ==== 74+0+16513 (secure 0 0 0) 0x7fb9fc057390 con 0x7fba140b3b40 2023-11-24T07:59:53.244 INFO:teuthology.orchestra.run.smithi059.stdout: 2023-11-24T07:59:53.244 INFO:teuthology.orchestra.run.smithi059.stdout:{"epoch":86,"fsid":"9f0e8741-b8f9-48d5-a2c3-09f45903303d","created":"2023-11-24T07:56:06.552729+0000","modified":"2023-11-24T07:59:51.909277+0000","last_up_change":"2023-11-24T07:56:17.372352+0000","last_in_change":"2023-11-24T07:56:09.073250+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":13,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-24T07:56:18.352823+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_metadata","create_time":"2023-11-24T07:59:50.274996+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_data","create_time":"2023-11-24T07:59:50.873919+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"6d8f9e9b-1782-4e41-928b-a39f2a8ebc06","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6800","nonce":3373878772},{"type":"v1","addr":"172.21.15.59:6801","nonce":3373878772}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6802","nonce":3373878772},{"type":"v1","addr":"172.21.15.59:6803","nonce":3373878772}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6806","nonce":3373878772},{"type":"v1","addr":"172.21.15.59:6807","nonce":3373878772}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6804","nonce":3373878772},{"type":"v1","addr":"172.21.15.59:6805","nonce":3373878772}]},"public_addr":"172.21.15.59:6801/3373878772","cluster_addr":"172.21.15.59:6803/3373878772","heartbeat_back_addr":"172.21.15.59:6807/3373878772","heartbeat_front_addr":"172.21.15.59:6805/3373878772","state":["exists","up"]},{"osd":1,"uuid":"e8d882aa-52e1-4c04-9e1e-c44ef9204a5f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6824","nonce":648258563},{"type":"v1","addr":"172.21.15.59:6825","nonce":648258563}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6826","nonce":648258563},{"type":"v1","addr":"172.21.15.59:6827","nonce":648258563}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6830","nonce":648258563},{"type":"v1","addr":"172.21.15.59:6831","nonce":648258563}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6828","nonce":648258563},{"type":"v1","addr":"172.21.15.59:6829","nonce":648258563}]},"public_addr":"172.21.15.59:6825/648258563","cluster_addr":"172.21.15.59:6827/648258563","heartbeat_back_addr":"172.21.15.59:6831/648258563","heartbeat_front_addr":"172.21.15.59:6829/648258563","state":["exists","up"]},{"osd":2,"uuid":"69eebfa0-695b-4b21-90b9-8a3537168a01","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6808","nonce":1840430654},{"type":"v1","addr":"172.21.15.59:6809","nonce":1840430654}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6810","nonce":1840430654},{"type":"v1","addr":"172.21.15.59:6811","nonce":1840430654}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6814","nonce":1840430654},{"type":"v1","addr":"172.21.15.59:6815","nonce":1840430654}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6812","nonce":1840430654},{"type":"v1","addr":"172.21.15.59:6813","nonce":1840430654}]},"public_addr":"172.21.15.59:6809/1840430654","cluster_addr":"172.21.15.59:6811/1840430654","heartbeat_back_addr":"172.21.15.59:6815/1840430654","heartbeat_front_addr":"172.21.15.59:6813/1840430654","state":["exists","up"]},{"osd":3,"uuid":"96fd77fc-f911-46ff-9445-6f0be2daa8f4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6816","nonce":2504025850},{"type":"v1","addr":"172.21.15.59:6817","nonce":2504025850}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6818","nonce":2504025850},{"type":"v1","addr":"172.21.15.59:6819","nonce":2504025850}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6822","nonce":2504025850},{"type":"v1","addr":"172.21.15.59:6823","nonce":2504025850}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6820","nonce":2504025850},{"type":"v1","addr":"172.21.15.59:6821","nonce":2504025850}]},"public_addr":"172.21.15.59:6817/2504025850","cluster_addr":"172.21.15.59:6819/2504025850","heartbeat_back_addr":"172.21.15.59:6823/2504025850","heartbeat_front_addr":"172.21.15.59:6821/2504025850","state":["exists","up"]},{"osd":4,"uuid":"7ac29fca-b5b2-4c95-9d69-2f164cbbc708","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6816","nonce":2382463431},{"type":"v1","addr":"172.21.15.76:6818","nonce":2382463431}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6820","nonce":2382463431},{"type":"v1","addr":"172.21.15.76:6822","nonce":2382463431}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6828","nonce":2382463431},{"type":"v1","addr":"172.21.15.76:6830","nonce":2382463431}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6824","nonce":2382463431},{"type":"v1","addr":"172.21.15.76:6826","nonce":2382463431}]},"public_addr":"172.21.15.76:6818/2382463431","cluster_addr":"172.21.15.76:6822/2382463431","heartbeat_back_addr":"172.21.15.76:6830/2382463431","heartbeat_front_addr":"172.21.15.76:6826/2382463431","state":["exists","up"]},{"osd":5,"uuid":"485bf711-3851-4238-a4f4-51d52e0a9c58","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6800","nonce":329334308},{"type":"v1","addr":"172.21.15.76:6802","nonce":329334308}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6805","nonce":329334308},{"type":"v1","addr":"172.21.15.76:6807","nonce":329334308}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6814","nonce":329334308},{"type":"v1","addr":"172.21.15.76:6815","nonce":329334308}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6810","nonce":329334308},{"type":"v1","addr":"172.21.15.76:6812","nonce":329334308}]},"public_addr":"172.21.15.76:6802/329334308","cluster_addr":"172.21.15.76:6807/329334308","heartbeat_back_addr":"172.21.15.76:6815/329334308","heartbeat_front_addr":"172.21.15.76:6812/329334308","state":["exists","up"]},{"osd":6,"uuid":"72b3f2fc-1e4a-4e32-968d-003647a0286f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6801","nonce":3946579410},{"type":"v1","addr":"172.21.15.76:6803","nonce":3946579410}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6804","nonce":3946579410},{"type":"v1","addr":"172.21.15.76:6806","nonce":3946579410}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6811","nonce":3946579410},{"type":"v1","addr":"172.21.15.76:6813","nonce":3946579410}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6808","nonce":3946579410},{"type":"v1","addr":"172.21.15.76:6809","nonce":3946579410}]},"public_addr":"172.21.15.76:6803/3946579410","cluster_addr":"172.21.15.76:6806/3946579410","heartbeat_back_addr":"172.21.15.76:6813/3946579410","heartbeat_front_addr":"172.21.15.76:6809/3946579410","state":["exists","up"]},{"osd":7,"uuid":"1ddceb90-6c5d-4d1a-ad56-99cb877f8c70","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6817","nonce":841761538},{"type":"v1","addr":"172.21.15.76:6819","nonce":841761538}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6821","nonce":841761538},{"type":"v1","addr":"172.21.15.76:6823","nonce":841761538}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6829","nonce":841761538},{"type":"v1","addr":"172.21.15.76:6831","nonce":841761538}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6825","nonce":841761538},{"type":"v1","addr":"172.21.15.76:6827","nonce":841761538}]},"public_addr":"172.21.15.76:6819/841761538","cluster_addr":"172.21.15.76:6823/841761538","heartbeat_back_addr":"172.21.15.76:6831/841761538","heartbeat_front_addr":"172.21.15.76:6827/841761538","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T07:56:14.294838+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T07:56:14.270779+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T07:56:14.231809+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T07:56:14.260983+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T07:56:14.302997+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T07:56:14.258600+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T07:56:14.251413+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T07:56:14.304727+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-24T07:59:53.246 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:53.244+0000 7fba1a254700 1 -- 172.21.15.59:0/3329872829 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7fb9f406b470 msgr2=0x7fb9f406d930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:53.246 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:53.244+0000 7fba1a254700 1 --2- 172.21.15.59:0/3329872829 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7fb9f406b470 0x7fb9f406d930 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7fba1415d9a0 tx=0x7fba04008040 comp rx=0 tx=0).stop 2023-11-24T07:59:53.246 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:53.244+0000 7fba1a254700 1 -- 172.21.15.59:0/3329872829 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fba140b3b40 msgr2=0x7fba140af360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:53.246 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:53.244+0000 7fba1a254700 1 --2- 172.21.15.59:0/3329872829 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fba140b3b40 0x7fba140af360 secure :-1 s=READY pgs=1019 cs=0 l=1 rev1=1 crypto rx=0x7fb9fc00c4e0 tx=0x7fb9fc00c8a0 comp rx=0 tx=0).stop 2023-11-24T07:59:53.246 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:53.244+0000 7fba1a254700 1 -- 172.21.15.59:0/3329872829 shutdown_connections 2023-11-24T07:59:53.246 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:53.244+0000 7fba1a254700 1 --2- 172.21.15.59:0/3329872829 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7fb9f406b470 0x7fb9f406d930 unknown :-1 s=CLOSED pgs=472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:53.246 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:53.244+0000 7fba1a254700 1 --2- 172.21.15.59:0/3329872829 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fba140b4c40 0x7fba1406afd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:53.246 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:53.244+0000 7fba1a254700 1 --2- 172.21.15.59:0/3329872829 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fba1406f420 0x7fba14143ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:53.246 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:53.244+0000 7fba1a254700 1 --2- 172.21.15.59:0/3329872829 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fba140b3b40 0x7fba140af360 unknown :-1 s=CLOSED pgs=1019 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:53.247 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:53.244+0000 7fba1a254700 1 -- 172.21.15.59:0/3329872829 >> 172.21.15.59:0/3329872829 conn(0x7fba140b2840 msgr2=0x7fba1415e4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:59:53.247 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:53.244+0000 7fba1a254700 1 -- 172.21.15.59:0/3329872829 shutdown_connections 2023-11-24T07:59:53.247 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:53.244+0000 7fba1a254700 1 -- 172.21.15.59:0/3329872829 wait complete. 2023-11-24T07:59:53.259 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2023-11-24T07:59:53.389 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:53.385+0000 7f146ecbb700 1 Processor -- start 2023-11-24T07:59:53.389 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:53.385+0000 7f146ecbb700 1 -- start start 2023-11-24T07:59:53.390 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:53.386+0000 7f146ecbb700 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f14680b4090 0x7f146815e510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:53.390 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:53.386+0000 7f146ecbb700 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f14680d1010 0x7f14680d1440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:53.390 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:53.386+0000 7f146ecbb700 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f14680d0b80 con 0x7f1468072a00 2023-11-24T07:59:53.390 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:53.386+0000 7f146ecbb700 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f14680d1980 con 0x7f14680d1010 2023-11-24T07:59:53.390 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:53.386+0000 7f146ecbb700 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f146815ed60 con 0x7f14680b4090 2023-11-24T07:59:53.390 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:53.386+0000 7f146ca57700 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f14680b4090 0x7f146815e510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:53.390 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:53.386+0000 7f146d258700 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f14680d1010 0x7f14680d1440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:53.390 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:53.386+0000 7f146ca57700 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f14680b4090 0x7f146815e510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3301/0 says I am v2:172.21.15.59:55274/0 (socket says 172.21.15.59:55274) 2023-11-24T07:59:53.390 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:53.386+0000 7f146d258700 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f14680d1010 0x7f14680d1440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.59:43036/0 (socket says 172.21.15.59:43036) 2023-11-24T07:59:53.390 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:53.386+0000 7f146ca57700 1 -- 172.21.15.59:0/2955977509 learned_addr learned my addr 172.21.15.59:0/2955977509 (peer_addr_for_me v2:172.21.15.59:0/0) 2023-11-24T07:59:53.390 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:53.387+0000 7f14677fe700 1 -- 172.21.15.59:0/2955977509 <== mon.1 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2037828107 0 0) 0x7f14680d0b80 con 0x7f1468072a00 2023-11-24T07:59:53.390 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:53.387+0000 7f146ca57700 1 -- 172.21.15.59:0/2955977509 >> v2:172.21.15.76:3300/0 conn(0x7f14680d1010 msgr2=0x7f14680d1440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T07:59:53.391 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:53.387+0000 7f146ca57700 1 --2- 172.21.15.59:0/2955977509 >> v2:172.21.15.76:3300/0 conn(0x7f14680d1010 0x7f14680d1440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:53.391 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:53.387+0000 7f146ca57700 1 -- 172.21.15.59:0/2955977509 >> v1:172.21.15.76:6789/0 conn(0x7f1468072a00 legacy=0x7f146806db60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:53.391 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:53.387+0000 7f146ca57700 1 -- 172.21.15.59:0/2955977509 --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f146815f040 con 0x7f14680b4090 2023-11-24T07:59:53.391 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:53.387+0000 7f146ca57700 1 --2- 172.21.15.59:0/2955977509 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f14680b4090 0x7f146815e510 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f145c0098f0 tx=0x7f145c009c00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f559761bb35045e6 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:59:53.391 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:53.387+0000 7f14677fe700 1 -- 172.21.15.59:0/2955977509 <== mon.2 v2:172.21.15.76:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f145c011830 con 0x7f14680b4090 2023-11-24T07:59:53.391 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:53.387+0000 7f14677fe700 1 -- 172.21.15.59:0/2955977509 <== mon.2 v2:172.21.15.76:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f145c011990 con 0x7f14680b4090 2023-11-24T07:59:53.391 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:53.387+0000 7f14677fe700 1 -- 172.21.15.59:0/2955977509 <== mon.2 v2:172.21.15.76:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f145c011d20 con 0x7f14680b4090 2023-11-24T07:59:53.391 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:53.388+0000 7f146ecbb700 1 -- 172.21.15.59:0/2955977509 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f14680b4090 msgr2=0x7f146815e510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:53.391 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:53.388+0000 7f146ecbb700 1 --2- 172.21.15.59:0/2955977509 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f14680b4090 0x7f146815e510 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f145c0098f0 tx=0x7f145c009c00 comp rx=0 tx=0).stop 2023-11-24T07:59:53.391 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:53.388+0000 7f146ecbb700 1 -- 172.21.15.59:0/2955977509 shutdown_connections 2023-11-24T07:59:53.391 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:53.388+0000 7f146ecbb700 1 --2- 172.21.15.59:0/2955977509 >> v2:172.21.15.76:3300/0 conn(0x7f14680d1010 0x7f14680d1440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:53.391 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:53.388+0000 7f146ecbb700 1 --2- 172.21.15.59:0/2955977509 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f14680b4090 0x7f146815e510 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:53.392 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:53.388+0000 7f146ecbb700 1 -- 172.21.15.59:0/2955977509 >> 172.21.15.59:0/2955977509 conn(0x7f14680b2740 msgr2=0x7f14680b2b50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:59:53.392 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:53.388+0000 7f146ecbb700 1 -- 172.21.15.59:0/2955977509 shutdown_connections 2023-11-24T07:59:53.392 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:53.388+0000 7f146ecbb700 1 -- 172.21.15.59:0/2955977509 wait complete. 2023-11-24T07:59:53.394 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:53.389+0000 7f146ecbb700 1 Processor -- start 2023-11-24T07:59:53.394 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:53.389+0000 7f146ecbb700 1 -- start start 2023-11-24T07:59:53.394 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:53.390+0000 7f146ecbb700 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f1468072a00 0x7f14680c6620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:53.394 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:53.390+0000 7f146ecbb700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f14680b4090 0x7f14681f3130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:53.395 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:53.390+0000 7f146ecbb700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f14680d1010 0x7f146815a530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:53.395 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:53.390+0000 7f146ecbb700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14681f3670 con 0x7f14680d1010 2023-11-24T07:59:53.395 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:53.390+0000 7f146ecbb700 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f146815ad60 con 0x7f14680b4090 2023-11-24T07:59:53.395 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:53.390+0000 7f146ecbb700 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f146815b040 con 0x7f1468072a00 2023-11-24T07:59:53.395 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:53.390+0000 7f146d258700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f14680d1010 0x7f146815a530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:53.395 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:53.390+0000 7f146ca57700 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f1468072a00 0x7f14680c6620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:53.395 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:53.390+0000 7f146d258700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f14680d1010 0x7f146815a530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:55844/0 (socket says 172.21.15.59:55844) 2023-11-24T07:59:53.395 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:53.390+0000 7f146ca57700 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f1468072a00 0x7f14680c6620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3301/0 says I am v2:172.21.15.59:55280/0 (socket says 172.21.15.59:55280) 2023-11-24T07:59:53.395 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:53.390+0000 7f146d258700 1 -- 172.21.15.59:0/1669793494 learned_addr learned my addr 172.21.15.59:0/1669793494 (peer_addr_for_me v2:172.21.15.59:0/0) 2023-11-24T07:59:53.395 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:53.390+0000 7f1467fff700 1 --2- 172.21.15.59:0/1669793494 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f14680b4090 0x7f14681f3130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:53.395 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:53.390+0000 7f146d258700 1 -- 172.21.15.59:0/1669793494 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f1468072a00 msgr2=0x7f14680c6620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:53.395 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:53.390+0000 7f146d258700 1 --2- 172.21.15.59:0/1669793494 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f1468072a00 0x7f14680c6620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:53.395 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:53.390+0000 7f146d258700 1 -- 172.21.15.59:0/1669793494 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f14680b4090 msgr2=0x7f14681f3130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:53.396 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:53.390+0000 7f146d258700 1 --2- 172.21.15.59:0/1669793494 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f14680b4090 0x7f14681f3130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:53.396 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:53.390+0000 7f146d258700 1 -- 172.21.15.59:0/1669793494 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f145c009550 con 0x7f14680d1010 2023-11-24T07:59:53.396 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:53.390+0000 7f146d258700 1 --2- 172.21.15.59:0/1669793494 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f14680d1010 0x7f146815a530 secure :-1 s=READY pgs=1020 cs=0 l=1 rev1=1 crypto rx=0x7f146000efc0 tx=0x7f146000c0c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:59:53.396 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:53.391+0000 7f14657fa700 1 -- 172.21.15.59:0/1669793494 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f146000cad0 con 0x7f14680d1010 2023-11-24T07:59:53.396 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:53.391+0000 7f14657fa700 1 -- 172.21.15.59:0/1669793494 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f146000f040 con 0x7f14680d1010 2023-11-24T07:59:53.396 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:53.391+0000 7f14657fa700 1 -- 172.21.15.59:0/1669793494 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f146000ce70 con 0x7f14680d1010 2023-11-24T07:59:53.396 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:53.391+0000 7f146ecbb700 1 -- 172.21.15.59:0/1669793494 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f146815b320 con 0x7f14680d1010 2023-11-24T07:59:53.396 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:53.391+0000 7f146ecbb700 1 -- 172.21.15.59:0/1669793494 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f146815b8e0 con 0x7f14680d1010 2023-11-24T07:59:53.396 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:53.392+0000 7f146ecbb700 1 -- 172.21.15.59:0/1669793494 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1468075480 con 0x7f14680d1010 2023-11-24T07:59:53.396 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:53.392+0000 7f14657fa700 1 -- 172.21.15.59:0/1669793494 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 11) v1 ==== 89741+0+0 (secure 0 0 0) 0x7f1460014480 con 0x7f14680d1010 2023-11-24T07:59:53.396 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:53.392+0000 7f14657fa700 1 --2- 172.21.15.59:0/1669793494 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f144c06b470 0x7f144c06d930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:53.396 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:53.393+0000 7f14657fa700 1 -- 172.21.15.59:0/1669793494 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f146008aaf0 con 0x7f14680d1010 2023-11-24T07:59:53.396 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:53.393+0000 7f146ca57700 1 --2- 172.21.15.59:0/1669793494 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f144c06b470 0x7f144c06d930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:53.397 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:53.393+0000 7f146ca57700 1 --2- 172.21.15.59:0/1669793494 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f144c06b470 0x7f144c06d930 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f145c009c40 tx=0x7f145c008030 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:59:53.398 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:53.397+0000 7f14657fa700 1 -- 172.21.15.59:0/1669793494 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f14600552b0 con 0x7f14680d1010 2023-11-24T07:59:53.650 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:53.648+0000 7f146ecbb700 1 -- 172.21.15.59:0/1669793494 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7f1468072650 con 0x7f14680d1010 2023-11-24T07:59:54.536 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:54.534+0000 7f14657fa700 1 -- 172.21.15.59:0/1669793494 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v52) v1 ==== 118+0+0 (secure 0 0 0) 0x7f14600588d0 con 0x7f14680d1010 2023-11-24T07:59:54.539 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:54.537+0000 7f146ecbb700 1 -- 172.21.15.59:0/1669793494 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f144c06b470 msgr2=0x7f144c06d930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:54.539 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:54.537+0000 7f146ecbb700 1 --2- 172.21.15.59:0/1669793494 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f144c06b470 0x7f144c06d930 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f145c009c40 tx=0x7f145c008030 comp rx=0 tx=0).stop 2023-11-24T07:59:54.539 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:54.537+0000 7f146ecbb700 1 -- 172.21.15.59:0/1669793494 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f14680d1010 msgr2=0x7f146815a530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:54.540 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:54.537+0000 7f146ecbb700 1 --2- 172.21.15.59:0/1669793494 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f14680d1010 0x7f146815a530 secure :-1 s=READY pgs=1020 cs=0 l=1 rev1=1 crypto rx=0x7f146000efc0 tx=0x7f146000c0c0 comp rx=0 tx=0).stop 2023-11-24T07:59:54.540 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:54.537+0000 7f146ecbb700 1 -- 172.21.15.59:0/1669793494 shutdown_connections 2023-11-24T07:59:54.540 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:54.537+0000 7f146ecbb700 1 --2- 172.21.15.59:0/1669793494 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f144c06b470 0x7f144c06d930 unknown :-1 s=CLOSED pgs=473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:54.540 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:54.537+0000 7f146ecbb700 1 --2- 172.21.15.59:0/1669793494 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f14680d1010 0x7f146815a530 unknown :-1 s=CLOSED pgs=1020 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:54.540 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:54.537+0000 7f146ecbb700 1 --2- 172.21.15.59:0/1669793494 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f1468072a00 0x7f14680c6620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:54.540 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:54.537+0000 7f146ecbb700 1 --2- 172.21.15.59:0/1669793494 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f14680b4090 0x7f14681f3130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:54.540 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:54.537+0000 7f146ecbb700 1 -- 172.21.15.59:0/1669793494 >> 172.21.15.59:0/1669793494 conn(0x7f14680b2740 msgr2=0x7f146806db60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:59:54.540 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:54.537+0000 7f146ecbb700 1 -- 172.21.15.59:0/1669793494 shutdown_connections 2023-11-24T07:59:54.540 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:54.537+0000 7f146ecbb700 1 -- 172.21.15.59:0/1669793494 wait complete. 2023-11-24T07:59:54.554 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2023-11-24T07:59:54.554 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2023-11-24T07:59:54.689 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:54.687+0000 7f44c1444700 1 Processor -- start 2023-11-24T07:59:54.689 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:54.687+0000 7f44c1444700 1 -- start start 2023-11-24T07:59:54.692 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:54.688+0000 7f44c1444700 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f44bc1440f0 0x7f44bc0b51b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:54.692 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:54.688+0000 7f44c1444700 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f44bc143ec0 con 0x7f44bc143ae0 2023-11-24T07:59:54.692 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:54.688+0000 7f44c1444700 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f44bc0b3ae0 con 0x7f44bc0722c0 2023-11-24T07:59:54.692 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:54.688+0000 7f44c1444700 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f44bc0b40a0 con 0x7f44bc1440f0 2023-11-24T07:59:54.692 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:54.688+0000 7f44baffd700 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7f44bc143ae0 0x7f44bc071bb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:59322/0 (socket says 172.21.15.59:59322) 2023-11-24T07:59:54.693 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:54.688+0000 7f44baffd700 1 -- 172.21.15.59:0/279495030 learned_addr learned my addr 172.21.15.59:0/279495030 (peer_addr_for_me v1:172.21.15.59:0/0) 2023-11-24T07:59:54.693 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:54.689+0000 7f44bb7fe700 1 --2- 172.21.15.59:0/279495030 >> v2:172.21.15.76:3300/0 conn(0x7f44bc1440f0 0x7f44bc0b51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:54.693 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:54.689+0000 7f44b9ffb700 1 -- 172.21.15.59:0/279495030 <== mon.0 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1722305705 0 0) 0x7f44bc143ec0 con 0x7f44bc143ae0 2023-11-24T07:59:54.693 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:54.689+0000 7f44b9ffb700 1 -- 172.21.15.59:0/279495030 --> v1:172.21.15.59:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f44a40035b0 con 0x7f44bc143ae0 2023-11-24T07:59:54.693 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:54.689+0000 7f44b9ffb700 1 -- 172.21.15.59:0/279495030 <== mon.0 v1:172.21.15.59:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3578392516 0 0) 0x7f44a40035b0 con 0x7f44bc143ae0 2023-11-24T07:59:54.693 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:54.689+0000 7f44b9ffb700 1 -- 172.21.15.59:0/279495030 >> v2:172.21.15.76:3300/0 conn(0x7f44bc1440f0 msgr2=0x7f44bc0b51b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T07:59:54.693 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:54.689+0000 7f44b9ffb700 1 --2- 172.21.15.59:0/279495030 >> v2:172.21.15.76:3300/0 conn(0x7f44bc1440f0 0x7f44bc0b51b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:54.693 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:54.689+0000 7f44b9ffb700 1 -- 172.21.15.59:0/279495030 >> v1:172.21.15.76:6789/0 conn(0x7f44bc0722c0 legacy=0x7f44bc06e510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:54.693 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:54.689+0000 7f44b9ffb700 1 -- 172.21.15.59:0/279495030 --> v1:172.21.15.59:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f44bc0b3dc0 con 0x7f44bc143ae0 2023-11-24T07:59:54.693 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:54.689+0000 7f44b9ffb700 1 -- 172.21.15.59:0/279495030 <== mon.0 v1:172.21.15.59:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1312333566 0 0) 0x7f44b0001670 con 0x7f44bc143ae0 2023-11-24T07:59:54.693 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:54.689+0000 7f44b9ffb700 1 -- 172.21.15.59:0/279495030 >> v1:172.21.15.59:6789/0 conn(0x7f44bc143ae0 legacy=0x7f44bc071bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:54.693 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:54.689+0000 7f44b9ffb700 1 --2- 172.21.15.59:0/279495030 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f44a4003a00 0x7f44a4003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:54.693 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:54.689+0000 7f44b9ffb700 1 --2- 172.21.15.59:0/279495030 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f44a4004850 0x7f44a4004cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:54.693 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:54.689+0000 7f44b9ffb700 1 --2- 172.21.15.59:0/279495030 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f44a4005210 0x7f44a400ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:54.694 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:54.689+0000 7f44b9ffb700 1 -- 172.21.15.59:0/279495030 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f44bc0b40a0 con 0x7f44a4003a00 2023-11-24T07:59:54.694 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:54.689+0000 7f44b9ffb700 1 -- 172.21.15.59:0/279495030 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f44a400c0b0 con 0x7f44a4004850 2023-11-24T07:59:54.694 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:54.689+0000 7f44b9ffb700 1 -- 172.21.15.59:0/279495030 --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f44a400c270 con 0x7f44a4005210 2023-11-24T07:59:54.694 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:54.689+0000 7f44baffd700 1 --2- 172.21.15.59:0/279495030 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f44a4003a00 0x7f44a4003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:54.694 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:54.689+0000 7f44bb7fe700 1 --2- 172.21.15.59:0/279495030 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f44a4005210 0x7f44a400ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:54.694 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:54.689+0000 7f44ba7fc700 1 --2- 172.21.15.59:0/279495030 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f44a4004850 0x7f44a4004cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:54.694 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:54.689+0000 7f44baffd700 1 -- 172.21.15.59:0/279495030 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f44a4005210 msgr2=0x7f44a400ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T07:59:54.694 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:54.689+0000 7f44baffd700 1 --2- 172.21.15.59:0/279495030 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f44a4005210 0x7f44a400ba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:54.694 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:54.689+0000 7f44baffd700 1 -- 172.21.15.59:0/279495030 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f44a4004850 msgr2=0x7f44a4004cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T07:59:54.694 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:54.689+0000 7f44baffd700 1 --2- 172.21.15.59:0/279495030 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f44a4004850 0x7f44a4004cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:54.694 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:54.689+0000 7f44bb7fe700 1 -- 172.21.15.59:0/279495030 reap_dead start 2023-11-24T07:59:54.694 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:54.689+0000 7f44baffd700 1 -- 172.21.15.59:0/279495030 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f44a40035b0 con 0x7f44a4003a00 2023-11-24T07:59:54.694 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:54.689+0000 7f44ba7fc700 1 --2- 172.21.15.59:0/279495030 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f44a4004850 0x7f44a4004cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-24T07:59:54.695 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:54.690+0000 7f44baffd700 1 --2- 172.21.15.59:0/279495030 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f44a4003a00 0x7f44a4003e20 secure :-1 s=READY pgs=1022 cs=0 l=1 rev1=1 crypto rx=0x7f44b000c240 tx=0x7f44a4005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7dfc4bc405b4fc21 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:59:54.695 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:54.690+0000 7f44b9ffb700 1 -- 172.21.15.59:0/279495030 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44b0007c10 con 0x7f44a4003a00 2023-11-24T07:59:54.695 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:54.690+0000 7f44b9ffb700 1 -- 172.21.15.59:0/279495030 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f44b000cd60 con 0x7f44a4003a00 2023-11-24T07:59:54.695 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:54.690+0000 7f44b9ffb700 1 -- 172.21.15.59:0/279495030 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44b0005ba0 con 0x7f44a4003a00 2023-11-24T07:59:54.695 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:54.690+0000 7f44c1444700 1 -- 172.21.15.59:0/279495030 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f44a4003a00 msgr2=0x7f44a4003e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:54.695 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:54.690+0000 7f44c1444700 1 --2- 172.21.15.59:0/279495030 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f44a4003a00 0x7f44a4003e20 secure :-1 s=READY pgs=1022 cs=0 l=1 rev1=1 crypto rx=0x7f44b000c240 tx=0x7f44a4005880 comp rx=0 tx=0).stop 2023-11-24T07:59:54.695 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:54.690+0000 7f44c1444700 1 -- 172.21.15.59:0/279495030 shutdown_connections 2023-11-24T07:59:54.695 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:54.690+0000 7f44c1444700 1 --2- 172.21.15.59:0/279495030 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f44a4003a00 0x7f44a4003e20 unknown :-1 s=CLOSED pgs=1022 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:54.695 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:54.690+0000 7f44c1444700 1 -- 172.21.15.59:0/279495030 >> 172.21.15.59:0/279495030 conn(0x7f44bc0b2740 msgr2=0x7f44bc0b2b20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:59:54.695 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:54.690+0000 7f44c1444700 1 -- 172.21.15.59:0/279495030 shutdown_connections 2023-11-24T07:59:54.696 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:54.690+0000 7f44c1444700 1 -- 172.21.15.59:0/279495030 wait complete. 2023-11-24T07:59:54.698 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:54.692+0000 7f44c1444700 1 Processor -- start 2023-11-24T07:59:54.698 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:54.692+0000 7f44c1444700 1 -- start start 2023-11-24T07:59:54.698 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:54.692+0000 7f44c1444700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f44a4003a00 0x7f44bc1f19b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:54.699 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:54.692+0000 7f44c1444700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f44bc154d00 0x7f44bc13f1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:54.699 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:54.692+0000 7f44c1444700 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f44bc1571b0 0x7f44bc157660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:54.699 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:54.692+0000 7f44c1444700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f44bc157ba0 con 0x7f44bc154d00 2023-11-24T07:59:54.699 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:54.692+0000 7f44c1444700 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f44bc1562a0 con 0x7f44a4003a00 2023-11-24T07:59:54.699 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:54.692+0000 7f44c1444700 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f44bc155e10 con 0x7f44bc1571b0 2023-11-24T07:59:54.699 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:54.693+0000 7f44ba7fc700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f44bc154d00 0x7f44bc13f1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:54.699 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:54.693+0000 7f44ba7fc700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f44bc154d00 0x7f44bc13f1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:55862/0 (socket says 172.21.15.59:55862) 2023-11-24T07:59:54.699 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:54.693+0000 7f44ba7fc700 1 -- 172.21.15.59:0/943722761 learned_addr learned my addr 172.21.15.59:0/943722761 (peer_addr_for_me v2:172.21.15.59:0/0) 2023-11-24T07:59:54.700 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:54.693+0000 7f44bb7fe700 1 --2- 172.21.15.59:0/943722761 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f44bc1571b0 0x7f44bc157660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:54.700 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:54.693+0000 7f44baffd700 1 --2- 172.21.15.59:0/943722761 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f44a4003a00 0x7f44bc1f19b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:54.700 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:54.693+0000 7f44ba7fc700 1 -- 172.21.15.59:0/943722761 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f44bc1571b0 msgr2=0x7f44bc157660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:54.700 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:54.693+0000 7f44ba7fc700 1 --2- 172.21.15.59:0/943722761 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f44bc1571b0 0x7f44bc157660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:54.700 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:54.693+0000 7f44ba7fc700 1 -- 172.21.15.59:0/943722761 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f44a4003a00 msgr2=0x7f44bc1f19b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:54.700 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:54.693+0000 7f44ba7fc700 1 --2- 172.21.15.59:0/943722761 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f44a4003a00 0x7f44bc1f19b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:54.700 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:54.693+0000 7f44ba7fc700 1 -- 172.21.15.59:0/943722761 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f44bc155200 con 0x7f44bc154d00 2023-11-24T07:59:54.700 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:54.693+0000 7f44baffd700 1 --2- 172.21.15.59:0/943722761 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f44a4003a00 0x7f44bc1f19b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T07:59:54.700 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:54.693+0000 7f44ba7fc700 1 --2- 172.21.15.59:0/943722761 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f44bc154d00 0x7f44bc13f1b0 secure :-1 s=READY pgs=1023 cs=0 l=1 rev1=1 crypto rx=0x7f44ac00e580 tx=0x7f44ac00e940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:59:54.700 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:54.693+0000 7f44a37fe700 1 -- 172.21.15.59:0/943722761 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44ac0052e0 con 0x7f44bc154d00 2023-11-24T07:59:54.701 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:54.693+0000 7f44a37fe700 1 -- 172.21.15.59:0/943722761 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f44ac005460 con 0x7f44bc154d00 2023-11-24T07:59:54.701 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:54.693+0000 7f44c1444700 1 -- 172.21.15.59:0/943722761 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f44bc1554e0 con 0x7f44bc154d00 2023-11-24T07:59:54.701 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:54.693+0000 7f44a37fe700 1 -- 172.21.15.59:0/943722761 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44ac0038c0 con 0x7f44bc154d00 2023-11-24T07:59:54.701 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:54.693+0000 7f44c1444700 1 -- 172.21.15.59:0/943722761 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f44bc155aa0 con 0x7f44bc154d00 2023-11-24T07:59:54.701 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:54.694+0000 7f44c1444700 1 -- 172.21.15.59:0/943722761 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f44bc075480 con 0x7f44bc154d00 2023-11-24T07:59:54.701 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:54.695+0000 7f44a37fe700 1 -- 172.21.15.59:0/943722761 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 11) v1 ==== 89741+0+0 (secure 0 0 0) 0x7f44ac003a20 con 0x7f44bc154d00 2023-11-24T07:59:54.701 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:54.695+0000 7f44a37fe700 1 --2- 172.21.15.59:0/943722761 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f44a806b470 0x7f44a806d930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:54.701 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:54.695+0000 7f44a37fe700 1 -- 172.21.15.59:0/943722761 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f44ac089690 con 0x7f44bc154d00 2023-11-24T07:59:54.701 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:54.696+0000 7f44baffd700 1 --2- 172.21.15.59:0/943722761 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f44a806b470 0x7f44a806d930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:54.701 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:54.696+0000 7f44baffd700 1 --2- 172.21.15.59:0/943722761 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f44a806b470 0x7f44a806d930 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f44b0015040 tx=0x7f44b0007000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:59:54.702 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:54.700+0000 7f44a37fe700 1 -- 172.21.15.59:0/943722761 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f44ac053f80 con 0x7f44bc154d00 2023-11-24T07:59:54.948 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:54.946+0000 7f44c1444700 1 -- 172.21.15.59:0/943722761 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7f44bc155c30 con 0x7f44bc154d00 2023-11-24T07:59:55.537 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:55.535+0000 7f44a37fe700 1 -- 172.21.15.59:0/943722761 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v53) v1 ==== 122+0+0 (secure 0 0 0) 0x7f44ac0575a0 con 0x7f44bc154d00 2023-11-24T07:59:55.540 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:55.538+0000 7f44c1444700 1 -- 172.21.15.59:0/943722761 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f44a806b470 msgr2=0x7f44a806d930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:55.540 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:55.538+0000 7f44c1444700 1 --2- 172.21.15.59:0/943722761 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f44a806b470 0x7f44a806d930 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f44b0015040 tx=0x7f44b0007000 comp rx=0 tx=0).stop 2023-11-24T07:59:55.540 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:55.538+0000 7f44c1444700 1 -- 172.21.15.59:0/943722761 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f44bc154d00 msgr2=0x7f44bc13f1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:55.540 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:55.538+0000 7f44c1444700 1 --2- 172.21.15.59:0/943722761 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f44bc154d00 0x7f44bc13f1b0 secure :-1 s=READY pgs=1023 cs=0 l=1 rev1=1 crypto rx=0x7f44ac00e580 tx=0x7f44ac00e940 comp rx=0 tx=0).stop 2023-11-24T07:59:55.540 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:55.538+0000 7f44c1444700 1 -- 172.21.15.59:0/943722761 shutdown_connections 2023-11-24T07:59:55.540 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:55.538+0000 7f44c1444700 1 --2- 172.21.15.59:0/943722761 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f44a806b470 0x7f44a806d930 unknown :-1 s=CLOSED pgs=474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:55.541 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:55.538+0000 7f44c1444700 1 --2- 172.21.15.59:0/943722761 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f44bc1571b0 0x7f44bc157660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:55.541 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:55.538+0000 7f44c1444700 1 --2- 172.21.15.59:0/943722761 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f44a4003a00 0x7f44bc1f19b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:55.541 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:55.538+0000 7f44c1444700 1 --2- 172.21.15.59:0/943722761 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f44bc154d00 0x7f44bc13f1b0 unknown :-1 s=CLOSED pgs=1023 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:55.541 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:55.538+0000 7f44c1444700 1 -- 172.21.15.59:0/943722761 >> 172.21.15.59:0/943722761 conn(0x7f44bc0b2740 msgr2=0x7f44bc140200 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:59:55.541 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:55.538+0000 7f44c1444700 1 -- 172.21.15.59:0/943722761 shutdown_connections 2023-11-24T07:59:55.541 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:55.538+0000 7f44c1444700 1 -- 172.21.15.59:0/943722761 wait complete. 2023-11-24T07:59:55.555 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2023-11-24T07:59:55.555 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-24T07:59:55.687 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:55.686+0000 7f0b37bdf700 1 Processor -- start 2023-11-24T07:59:55.687 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:55.686+0000 7f0b37bdf700 1 -- start start 2023-11-24T07:59:55.692 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:55.687+0000 7f0b37bdf700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f0b3015fa40 0x7f0b3015fec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:55.692 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:55.687+0000 7f0b37bdf700 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f0b30160400 0x7f0b3006dfa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:55.692 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:55.687+0000 7f0b37bdf700 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0b3006db10 con 0x7f0b30145830 2023-11-24T07:59:55.692 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:55.687+0000 7f0b37bdf700 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0b30145c10 con 0x7f0b3015fa40 2023-11-24T07:59:55.692 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:55.687+0000 7f0b37bdf700 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0b30160890 con 0x7f0b30160400 2023-11-24T07:59:55.692 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:55.687+0000 7f0b3517a700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f0b3015fa40 0x7f0b3015fec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:55.693 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:55.687+0000 7f0b3517a700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f0b3015fa40 0x7f0b3015fec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:55870/0 (socket says 172.21.15.59:55870) 2023-11-24T07:59:55.693 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:55.687+0000 7f0b3517a700 1 -- 172.21.15.59:0/1692092009 learned_addr learned my addr 172.21.15.59:0/1692092009 (peer_addr_for_me v2:172.21.15.59:0/0) 2023-11-24T07:59:55.693 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:55.687+0000 7f0b3617c700 1 --2- 172.21.15.59:0/1692092009 >> v2:172.21.15.76:3300/0 conn(0x7f0b30160400 0x7f0b3006dfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:55.693 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:55.687+0000 7f0b3517a700 1 -- 172.21.15.59:0/1692092009 >> v2:172.21.15.76:3300/0 conn(0x7f0b30160400 msgr2=0x7f0b3006dfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T07:59:55.693 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:55.687+0000 7f0b3517a700 1 --2- 172.21.15.59:0/1692092009 >> v2:172.21.15.76:3300/0 conn(0x7f0b30160400 0x7f0b3006dfa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:55.693 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:55.687+0000 7f0b3517a700 1 -- 172.21.15.59:0/1692092009 >> v1:172.21.15.59:6789/0 conn(0x7f0b30145830 legacy=0x7f0b300cd5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:55.694 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:55.687+0000 7f0b3517a700 1 -- 172.21.15.59:0/1692092009 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0b300cddb0 con 0x7f0b3015fa40 2023-11-24T07:59:55.694 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:55.687+0000 7f0b34979700 1 -- 172.21.15.59:0/1692092009 <== mon.0 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1713806270 0 0) 0x7f0b3006db10 con 0x7f0b30145830 2023-11-24T07:59:55.694 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:55.687+0000 7f0b3517a700 1 --2- 172.21.15.59:0/1692092009 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0b3015fa40 0x7f0b3015fec0 secure :-1 s=READY pgs=1025 cs=0 l=1 rev1=1 crypto rx=0x7f0b3006c090 tx=0x7f0b20009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=93601501c4887b4b server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:59:55.694 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:55.687+0000 7f0b34979700 1 -- 172.21.15.59:0/1692092009 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b2000b870 con 0x7f0b3015fa40 2023-11-24T07:59:55.694 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:55.687+0000 7f0b34979700 1 -- 172.21.15.59:0/1692092009 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0b2000b9d0 con 0x7f0b3015fa40 2023-11-24T07:59:55.695 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:55.687+0000 7f0b34979700 1 -- 172.21.15.59:0/1692092009 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b2000bd60 con 0x7f0b3015fa40 2023-11-24T07:59:55.695 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:55.688+0000 7f0b37bdf700 1 -- 172.21.15.59:0/1692092009 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0b3015fa40 msgr2=0x7f0b3015fec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:55.695 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:55.688+0000 7f0b37bdf700 1 --2- 172.21.15.59:0/1692092009 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0b3015fa40 0x7f0b3015fec0 secure :-1 s=READY pgs=1025 cs=0 l=1 rev1=1 crypto rx=0x7f0b3006c090 tx=0x7f0b20009bd0 comp rx=0 tx=0).stop 2023-11-24T07:59:55.695 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:55.688+0000 7f0b37bdf700 1 -- 172.21.15.59:0/1692092009 shutdown_connections 2023-11-24T07:59:55.695 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:55.688+0000 7f0b37bdf700 1 --2- 172.21.15.59:0/1692092009 >> v2:172.21.15.76:3300/0 conn(0x7f0b30160400 0x7f0b3006dfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:55.695 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:55.688+0000 7f0b37bdf700 1 --2- 172.21.15.59:0/1692092009 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0b3015fa40 0x7f0b3015fec0 unknown :-1 s=CLOSED pgs=1025 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:55.695 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:55.688+0000 7f0b37bdf700 1 -- 172.21.15.59:0/1692092009 >> 172.21.15.59:0/1692092009 conn(0x7f0b300d0bc0 msgr2=0x7f0b300d0fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T07:59:55.695 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:55.688+0000 7f0b37bdf700 1 -- 172.21.15.59:0/1692092009 shutdown_connections 2023-11-24T07:59:55.696 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:55.688+0000 7f0b37bdf700 1 -- 172.21.15.59:0/1692092009 wait complete. 2023-11-24T07:59:55.696 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:55.690+0000 7f0b37bdf700 1 Processor -- start 2023-11-24T07:59:55.696 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:55.690+0000 7f0b37bdf700 1 -- start start 2023-11-24T07:59:55.699 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:55.691+0000 7f0b37bdf700 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f0b30145830 0x7f0b301447d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:55.699 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:55.691+0000 7f0b37bdf700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0b3015fa40 0x7f0b300cc440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:55.700 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:55.691+0000 7f0b37bdf700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0b30160400 0x7f0b301f5760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:55.700 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:55.691+0000 7f0b37bdf700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b30144340 con 0x7f0b3015fa40 2023-11-24T07:59:55.700 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:55.691+0000 7f0b37bdf700 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b30136410 con 0x7f0b30160400 2023-11-24T07:59:55.700 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:55.691+0000 7f0b37bdf700 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0b301f5ca0 con 0x7f0b30145830 2023-11-24T07:59:55.701 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:55.691+0000 7f0b3517a700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0b3015fa40 0x7f0b300cc440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:55.701 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:55.691+0000 7f0b3517a700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0b3015fa40 0x7f0b300cc440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:55876/0 (socket says 172.21.15.59:55876) 2023-11-24T07:59:55.701 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:55.691+0000 7f0b3517a700 1 -- 172.21.15.59:0/762032026 learned_addr learned my addr 172.21.15.59:0/762032026 (peer_addr_for_me v2:172.21.15.59:0/0) 2023-11-24T07:59:55.701 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:55.691+0000 7f0b3597b700 1 --2- 172.21.15.59:0/762032026 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f0b30145830 0x7f0b301447d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:55.701 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:55.691+0000 7f0b3617c700 1 --2- 172.21.15.59:0/762032026 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0b30160400 0x7f0b301f5760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:55.701 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:55.691+0000 7f0b3517a700 1 -- 172.21.15.59:0/762032026 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f0b30145830 msgr2=0x7f0b301447d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:55.701 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:55.691+0000 7f0b3517a700 1 --2- 172.21.15.59:0/762032026 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f0b30145830 0x7f0b301447d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:55.701 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:55.691+0000 7f0b3517a700 1 -- 172.21.15.59:0/762032026 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0b30160400 msgr2=0x7f0b301f5760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T07:59:55.702 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:55.691+0000 7f0b3517a700 1 --2- 172.21.15.59:0/762032026 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0b30160400 0x7f0b301f5760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T07:59:55.702 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:55.691+0000 7f0b3517a700 1 -- 172.21.15.59:0/762032026 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0b20009550 con 0x7f0b3015fa40 2023-11-24T07:59:55.702 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:55.691+0000 7f0b3617c700 1 --2- 172.21.15.59:0/762032026 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0b30160400 0x7f0b301f5760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T07:59:55.702 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:55.691+0000 7f0b3517a700 1 --2- 172.21.15.59:0/762032026 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0b3015fa40 0x7f0b300cc440 secure :-1 s=READY pgs=1026 cs=0 l=1 rev1=1 crypto rx=0x7f0b300676b0 tx=0x7f0b20004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:59:55.702 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:55.691+0000 7f0b267fc700 1 -- 172.21.15.59:0/762032026 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b20004310 con 0x7f0b3015fa40 2023-11-24T07:59:55.702 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:55.692+0000 7f0b267fc700 1 -- 172.21.15.59:0/762032026 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0b2001f070 con 0x7f0b3015fa40 2023-11-24T07:59:55.703 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:55.692+0000 7f0b37bdf700 1 -- 172.21.15.59:0/762032026 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0b30143730 con 0x7f0b3015fa40 2023-11-24T07:59:55.703 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:55.692+0000 7f0b267fc700 1 -- 172.21.15.59:0/762032026 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b200036a0 con 0x7f0b3015fa40 2023-11-24T07:59:55.703 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:55.692+0000 7f0b37bdf700 1 -- 172.21.15.59:0/762032026 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0b30143cf0 con 0x7f0b3015fa40 2023-11-24T07:59:55.703 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:55.692+0000 7f0b37bdf700 1 -- 172.21.15.59:0/762032026 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0b301444f0 con 0x7f0b3015fa40 2023-11-24T07:59:55.703 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:55.693+0000 7f0b267fc700 1 -- 172.21.15.59:0/762032026 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 11) v1 ==== 89741+0+0 (secure 0 0 0) 0x7f0b20021030 con 0x7f0b3015fa40 2023-11-24T07:59:55.703 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:55.693+0000 7f0b267fc700 1 --2- 172.21.15.59:0/762032026 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f0b1406b470 0x7f0b1406d930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T07:59:55.704 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:55.693+0000 7f0b267fc700 1 -- 172.21.15.59:0/762032026 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f0b2008b520 con 0x7f0b3015fa40 2023-11-24T07:59:55.704 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:55.694+0000 7f0b3597b700 1 --2- 172.21.15.59:0/762032026 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f0b1406b470 0x7f0b1406d930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T07:59:55.704 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:55.694+0000 7f0b3597b700 1 --2- 172.21.15.59:0/762032026 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f0b1406b470 0x7f0b1406d930 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f0b2c000f60 tx=0x7f0b2c009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T07:59:55.704 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:55.698+0000 7f0b267fc700 1 -- 172.21.15.59:0/762032026 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f0b20055f10 con 0x7f0b3015fa40 2023-11-24T07:59:55.950 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:55.948+0000 7f0b37bdf700 1 -- 172.21.15.59:0/762032026 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f0b30068670 con 0x7f0b3015fa40 2023-11-24T08:00:04.993 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-24T08:00:05.001 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:55.948+0000 7f0b267fc700 1 -- 172.21.15.59:0/762032026 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 53 v53) v1 ==== 94+0+4912 (secure 0 0 0) 0x7f0b20059530 con 0x7f0b3015fa40 2023-11-24T08:00:05.001 INFO:teuthology.orchestra.run.smithi059.stdout: 2023-11-24T08:00:05.001 INFO:teuthology.orchestra.run.smithi059.stdout:{"epoch":53,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5876,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.76:6837/1132179319","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6835","nonce":1132179319},{"type":"v1","addr":"172.21.15.76:6837","nonce":1132179319}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":48},{"gid":6487,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.76:6836/453627414","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6834","nonce":453627414},{"type":"v1","addr":"172.21.15.76:6836","nonce":453627414}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":48},{"gid":7035,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6834/4160100816","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6832","nonce":4160100816},{"type":"v1","addr":"172.21.15.59:6834","nonce":4160100816}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":48}],"filesystems":[{"mdsmap":{"epoch":53,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-24T07:59:51.909592+0000","modified":"2023-11-24T07:59:54.948256+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":7038},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7038":{"gid":7038,"name":"c","rank":0,"incarnation":49,"state":"up:active","state_seq":3,"addr":"172.21.15.59:6835/2337335405","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6833","nonce":2337335405},{"type":"v1","addr":"172.21.15.59:6835","nonce":2337335405}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":6}]} 2023-11-24T08:00:05.003 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:55.951+0000 7f0b37bdf700 1 -- 172.21.15.59:0/762032026 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f0b1406b470 msgr2=0x7f0b1406d930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:00:05.003 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:55.951+0000 7f0b37bdf700 1 --2- 172.21.15.59:0/762032026 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f0b1406b470 0x7f0b1406d930 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f0b2c000f60 tx=0x7f0b2c009040 comp rx=0 tx=0).stop 2023-11-24T08:00:05.003 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:55.951+0000 7f0b37bdf700 1 -- 172.21.15.59:0/762032026 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0b3015fa40 msgr2=0x7f0b300cc440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:00:05.003 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:55.951+0000 7f0b37bdf700 1 --2- 172.21.15.59:0/762032026 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0b3015fa40 0x7f0b300cc440 secure :-1 s=READY pgs=1026 cs=0 l=1 rev1=1 crypto rx=0x7f0b300676b0 tx=0x7f0b20004750 comp rx=0 tx=0).stop 2023-11-24T08:00:05.003 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:55.951+0000 7f0b37bdf700 1 -- 172.21.15.59:0/762032026 shutdown_connections 2023-11-24T08:00:05.003 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:55.951+0000 7f0b37bdf700 1 --2- 172.21.15.59:0/762032026 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f0b1406b470 0x7f0b1406d930 unknown :-1 s=CLOSED pgs=475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:00:05.003 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:55.951+0000 7f0b37bdf700 1 --2- 172.21.15.59:0/762032026 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0b30160400 0x7f0b301f5760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:00:05.004 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:55.951+0000 7f0b37bdf700 1 --2- 172.21.15.59:0/762032026 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f0b30145830 0x7f0b301447d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:00:05.004 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:55.951+0000 7f0b37bdf700 1 --2- 172.21.15.59:0/762032026 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0b3015fa40 0x7f0b300cc440 unknown :-1 s=CLOSED pgs=1026 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:00:05.004 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:55.951+0000 7f0b37bdf700 1 -- 172.21.15.59:0/762032026 >> 172.21.15.59:0/762032026 conn(0x7f0b300d0bc0 msgr2=0x7f0b300d0fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:00:05.004 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:55.951+0000 7f0b37bdf700 1 -- 172.21.15.59:0/762032026 shutdown_connections 2023-11-24T08:00:05.004 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T07:59:55.951+0000 7f0b37bdf700 1 -- 172.21.15.59:0/762032026 wait complete. 2023-11-24T08:00:05.004 INFO:teuthology.orchestra.run.smithi059.stderr:dumped fsmap epoch 53 2023-11-24T08:00:05.005 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-24T08:00:05.005 INFO:tasks.ceph.mds.c.smithi059.stderr:2023-11-24T07:59:57.472+0000 7f3ea5e3b700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-11-24T08:00:05.046 DEBUG:teuthology.orchestra.run.smithi076:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-24T08:00:05.185 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:05.183+0000 7f97c6f52700 1 Processor -- start 2023-11-24T08:00:05.185 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:05.183+0000 7f97c6f52700 1 -- start start 2023-11-24T08:00:05.187 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:05.184+0000 7f97c6f52700 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f97c015e460 0x7f97c00d49d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:00:05.187 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:05.184+0000 7f97c6f52700 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f97c00b3370 con 0x7f97c00d2ba0 2023-11-24T08:00:05.187 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:05.184+0000 7f97c6f52700 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f97c00cfc40 con 0x7f97c00cf480 2023-11-24T08:00:05.187 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:05.184+0000 7f97c6f52700 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f97c00b35c0 con 0x7f97c015e460 2023-11-24T08:00:05.187 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:05.185+0000 7f97c54ef700 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7f97c00d2ba0 0x7f97c00b40a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:59360/0 (socket says 172.21.15.59:59360) 2023-11-24T08:00:05.188 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:05.185+0000 7f97c54ef700 1 -- 172.21.15.59:0/583544489 learned_addr learned my addr 172.21.15.59:0/583544489 (peer_addr_for_me v1:172.21.15.59:0/0) 2023-11-24T08:00:05.188 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:05.185+0000 7f97c4cee700 1 --2- 172.21.15.59:0/583544489 >> v2:172.21.15.76:3300/0 conn(0x7f97c015e460 0x7f97c00d49d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:00:05.188 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:05.185+0000 7f97bf7fe700 1 -- 172.21.15.59:0/583544489 <== mon.0 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2526297138 0 0) 0x7f97c00b3370 con 0x7f97c00d2ba0 2023-11-24T08:00:05.188 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:05.185+0000 7f97bf7fe700 1 -- 172.21.15.59:0/583544489 --> v1:172.21.15.59:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f97b00035b0 con 0x7f97c00d2ba0 2023-11-24T08:00:05.188 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:05.185+0000 7f97bf7fe700 1 -- 172.21.15.59:0/583544489 <== mon.1 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2112806156 0 0) 0x7f97c00cfc40 con 0x7f97c00cf480 2023-11-24T08:00:05.188 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:05.185+0000 7f97bf7fe700 1 -- 172.21.15.59:0/583544489 --> v1:172.21.15.76:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f97c00b3370 con 0x7f97c00cf480 2023-11-24T08:00:05.188 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:05.185+0000 7f97bf7fe700 1 -- 172.21.15.59:0/583544489 <== mon.0 v1:172.21.15.59:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1307780209 0 0) 0x7f97b00035b0 con 0x7f97c00d2ba0 2023-11-24T08:00:05.188 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:05.185+0000 7f97c4cee700 1 -- 172.21.15.59:0/583544489 >> v1:172.21.15.76:6789/0 conn(0x7f97c00cf480 legacy=0x7f97c00d2490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:00:05.188 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:05.185+0000 7f97c4cee700 1 -- 172.21.15.59:0/583544489 >> v1:172.21.15.59:6789/0 conn(0x7f97c00d2ba0 legacy=0x7f97c00b40a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:00:05.188 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:05.185+0000 7f97c4cee700 1 -- 172.21.15.59:0/583544489 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f97c00b38a0 con 0x7f97c015e460 2023-11-24T08:00:05.188 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:05.185+0000 7f97c4cee700 1 --2- 172.21.15.59:0/583544489 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f97c015e460 0x7f97c00d49d0 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f97b4009840 tx=0x7f97b4009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c33c4be94a49851a server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:00:05.189 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:05.186+0000 7f97bf7fe700 1 -- 172.21.15.59:0/583544489 <== mon.1 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97b80019f0 con 0x7f97c015e460 2023-11-24T08:00:05.189 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:05.186+0000 7f97bf7fe700 1 -- 172.21.15.59:0/583544489 <== mon.1 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f97b4004030 con 0x7f97c015e460 2023-11-24T08:00:05.189 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:05.186+0000 7f97bf7fe700 1 -- 172.21.15.59:0/583544489 <== mon.1 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97b40043a0 con 0x7f97c015e460 2023-11-24T08:00:05.190 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:05.186+0000 7f97c6f52700 1 -- 172.21.15.59:0/583544489 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f97c015e460 msgr2=0x7f97c00d49d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:00:05.190 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:05.186+0000 7f97c6f52700 1 --2- 172.21.15.59:0/583544489 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f97c015e460 0x7f97c00d49d0 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f97b4009840 tx=0x7f97b4009b50 comp rx=0 tx=0).stop 2023-11-24T08:00:05.190 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:05.186+0000 7f97c6f52700 1 -- 172.21.15.59:0/583544489 shutdown_connections 2023-11-24T08:00:05.190 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:05.186+0000 7f97c6f52700 1 --2- 172.21.15.59:0/583544489 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f97c015e460 0x7f97c00d49d0 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:00:05.190 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:05.186+0000 7f97c6f52700 1 -- 172.21.15.59:0/583544489 >> 172.21.15.59:0/583544489 conn(0x7f97c00d3eb0 msgr2=0x7f97c00d42c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:00:05.190 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:05.186+0000 7f97c6f52700 1 -- 172.21.15.59:0/583544489 shutdown_connections 2023-11-24T08:00:05.190 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:05.186+0000 7f97c6f52700 1 -- 172.21.15.59:0/583544489 wait complete. 2023-11-24T08:00:05.190 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:05.188+0000 7f97c6f52700 1 Processor -- start 2023-11-24T08:00:05.191 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:05.188+0000 7f97c6f52700 1 -- start start 2023-11-24T08:00:05.193 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:05.189+0000 7f97c6f52700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f97c00cf480 0x7f97c015f890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:00:05.193 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:05.189+0000 7f97c6f52700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f97c00d2ba0 0x7f97c00b8ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:00:05.193 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:05.189+0000 7f97c6f52700 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f97c015e460 0x7f97c00b9000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:00:05.193 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:05.189+0000 7f97c6f52700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97c00cd6c0 con 0x7f97c00d2ba0 2023-11-24T08:00:05.193 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:05.189+0000 7f97c6f52700 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97c015fdd0 con 0x7f97c00cf480 2023-11-24T08:00:05.193 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:05.189+0000 7f97c6f52700 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f97c0160030 con 0x7f97c015e460 2023-11-24T08:00:05.193 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:05.189+0000 7f97bffff700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f97c00d2ba0 0x7f97c00b8ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:00:05.193 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:05.189+0000 7f97bffff700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f97c00d2ba0 0x7f97c00b8ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:55890/0 (socket says 172.21.15.59:55890) 2023-11-24T08:00:05.193 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:05.189+0000 7f97bffff700 1 -- 172.21.15.59:0/1820968244 learned_addr learned my addr 172.21.15.59:0/1820968244 (peer_addr_for_me v2:172.21.15.59:0/0) 2023-11-24T08:00:05.193 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:05.189+0000 7f97c4cee700 1 --2- 172.21.15.59:0/1820968244 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f97c00cf480 0x7f97c015f890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:00:05.193 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:05.190+0000 7f97c54ef700 1 --2- 172.21.15.59:0/1820968244 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f97c015e460 0x7f97c00b9000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:00:05.194 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:05.190+0000 7f97bffff700 1 -- 172.21.15.59:0/1820968244 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f97c015e460 msgr2=0x7f97c00b9000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:00:05.194 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:05.190+0000 7f97bffff700 1 --2- 172.21.15.59:0/1820968244 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f97c015e460 0x7f97c00b9000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:00:05.194 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:05.190+0000 7f97bffff700 1 -- 172.21.15.59:0/1820968244 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f97c00cf480 msgr2=0x7f97c015f890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:00:05.194 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:05.190+0000 7f97bffff700 1 --2- 172.21.15.59:0/1820968244 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f97c00cf480 0x7f97c015f890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:00:05.194 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:05.190+0000 7f97bffff700 1 -- 172.21.15.59:0/1820968244 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f97b40094a0 con 0x7f97c00d2ba0 2023-11-24T08:00:05.194 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:05.190+0000 7f97c4cee700 1 --2- 172.21.15.59:0/1820968244 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f97c00cf480 0x7f97c015f890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T08:00:05.194 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:05.190+0000 7f97bffff700 1 --2- 172.21.15.59:0/1820968244 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f97c00d2ba0 0x7f97c00b8ac0 secure :-1 s=READY pgs=1028 cs=0 l=1 rev1=1 crypto rx=0x7f97ac00c500 tx=0x7f97ac00c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:00:05.194 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:05.190+0000 7f97bd7fa700 1 -- 172.21.15.59:0/1820968244 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97ac0052d0 con 0x7f97c00d2ba0 2023-11-24T08:00:05.194 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:05.190+0000 7f97bd7fa700 1 -- 172.21.15.59:0/1820968244 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f97ac005430 con 0x7f97c00d2ba0 2023-11-24T08:00:05.194 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:05.190+0000 7f97c6f52700 1 -- 172.21.15.59:0/1820968244 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f97c0160310 con 0x7f97c00d2ba0 2023-11-24T08:00:05.194 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:05.190+0000 7f97bd7fa700 1 -- 172.21.15.59:0/1820968244 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97ac0038c0 con 0x7f97c00d2ba0 2023-11-24T08:00:05.194 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:05.190+0000 7f97c6f52700 1 -- 172.21.15.59:0/1820968244 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f97c01608d0 con 0x7f97c00d2ba0 2023-11-24T08:00:05.194 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:05.191+0000 7f97c6f52700 1 -- 172.21.15.59:0/1820968244 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f97c0067f60 con 0x7f97c00d2ba0 2023-11-24T08:00:05.195 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:05.192+0000 7f97bd7fa700 1 -- 172.21.15.59:0/1820968244 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 11) v1 ==== 89741+0+0 (secure 0 0 0) 0x7f97ac003a20 con 0x7f97c00d2ba0 2023-11-24T08:00:05.195 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:05.192+0000 7f97bd7fa700 1 --2- 172.21.15.59:0/1820968244 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f97a406b470 0x7f97a406d930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:00:05.195 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:05.192+0000 7f97bd7fa700 1 -- 172.21.15.59:0/1820968244 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f97ac089590 con 0x7f97c00d2ba0 2023-11-24T08:00:05.196 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:05.192+0000 7f97c4cee700 1 --2- 172.21.15.59:0/1820968244 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f97a406b470 0x7f97a406d930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:00:05.196 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:05.193+0000 7f97c4cee700 1 --2- 172.21.15.59:0/1820968244 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f97a406b470 0x7f97a406d930 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f97b4000c00 tx=0x7f97b4013040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:00:05.198 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:05.197+0000 7f97bd7fa700 1 -- 172.21.15.59:0/1820968244 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f97ac053de0 con 0x7f97c00d2ba0 2023-11-24T08:00:05.399 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:05.398+0000 7f97c6f52700 1 -- 172.21.15.59:0/1820968244 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f97c00d2270 con 0x7f97c00d2ba0 2023-11-24T08:00:05.400 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:05.399+0000 7f97bd7fa700 1 -- 172.21.15.59:0/1820968244 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v87) v1 ==== 74+0+16500 (secure 0 0 0) 0x7f97ac057400 con 0x7f97c00d2ba0 2023-11-24T08:00:05.401 INFO:teuthology.orchestra.run.smithi059.stdout: 2023-11-24T08:00:05.401 INFO:teuthology.orchestra.run.smithi059.stdout:{"epoch":87,"fsid":"9f0e8741-b8f9-48d5-a2c3-09f45903303d","created":"2023-11-24T07:56:06.552729+0000","modified":"2023-11-24T07:59:53.500645+0000","last_up_change":"2023-11-24T07:56:17.372352+0000","last_in_change":"2023-11-24T07:56:09.073250+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":13,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-24T07:56:18.352823+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_metadata","create_time":"2023-11-24T07:59:50.274996+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_data","create_time":"2023-11-24T07:59:50.873919+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"87","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"6d8f9e9b-1782-4e41-928b-a39f2a8ebc06","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6800","nonce":3373878772},{"type":"v1","addr":"172.21.15.59:6801","nonce":3373878772}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6802","nonce":3373878772},{"type":"v1","addr":"172.21.15.59:6803","nonce":3373878772}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6806","nonce":3373878772},{"type":"v1","addr":"172.21.15.59:6807","nonce":3373878772}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6804","nonce":3373878772},{"type":"v1","addr":"172.21.15.59:6805","nonce":3373878772}]},"public_addr":"172.21.15.59:6801/3373878772","cluster_addr":"172.21.15.59:6803/3373878772","heartbeat_back_addr":"172.21.15.59:6807/3373878772","heartbeat_front_addr":"172.21.15.59:6805/3373878772","state":["exists","up"]},{"osd":1,"uuid":"e8d882aa-52e1-4c04-9e1e-c44ef9204a5f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6824","nonce":648258563},{"type":"v1","addr":"172.21.15.59:6825","nonce":648258563}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6826","nonce":648258563},{"type":"v1","addr":"172.21.15.59:6827","nonce":648258563}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6830","nonce":648258563},{"type":"v1","addr":"172.21.15.59:6831","nonce":648258563}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6828","nonce":648258563},{"type":"v1","addr":"172.21.15.59:6829","nonce":648258563}]},"public_addr":"172.21.15.59:6825/648258563","cluster_addr":"172.21.15.59:6827/648258563","heartbeat_back_addr":"172.21.15.59:6831/648258563","heartbeat_front_addr":"172.21.15.59:6829/648258563","state":["exists","up"]},{"osd":2,"uuid":"69eebfa0-695b-4b21-90b9-8a3537168a01","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6808","nonce":1840430654},{"type":"v1","addr":"172.21.15.59:6809","nonce":1840430654}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6810","nonce":1840430654},{"type":"v1","addr":"172.21.15.59:6811","nonce":1840430654}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6814","nonce":1840430654},{"type":"v1","addr":"172.21.15.59:6815","nonce":1840430654}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6812","nonce":1840430654},{"type":"v1","addr":"172.21.15.59:6813","nonce":1840430654}]},"public_addr":"172.21.15.59:6809/1840430654","cluster_addr":"172.21.15.59:6811/1840430654","heartbeat_back_addr":"172.21.15.59:6815/1840430654","heartbeat_front_addr":"172.21.15.59:6813/1840430654","state":["exists","up"]},{"osd":3,"uuid":"96fd77fc-f911-46ff-9445-6f0be2daa8f4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6816","nonce":2504025850},{"type":"v1","addr":"172.21.15.59:6817","nonce":2504025850}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6818","nonce":2504025850},{"type":"v1","addr":"172.21.15.59:6819","nonce":2504025850}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6822","nonce":2504025850},{"type":"v1","addr":"172.21.15.59:6823","nonce":2504025850}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6820","nonce":2504025850},{"type":"v1","addr":"172.21.15.59:6821","nonce":2504025850}]},"public_addr":"172.21.15.59:6817/2504025850","cluster_addr":"172.21.15.59:6819/2504025850","heartbeat_back_addr":"172.21.15.59:6823/2504025850","heartbeat_front_addr":"172.21.15.59:6821/2504025850","state":["exists","up"]},{"osd":4,"uuid":"7ac29fca-b5b2-4c95-9d69-2f164cbbc708","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6816","nonce":2382463431},{"type":"v1","addr":"172.21.15.76:6818","nonce":2382463431}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6820","nonce":2382463431},{"type":"v1","addr":"172.21.15.76:6822","nonce":2382463431}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6828","nonce":2382463431},{"type":"v1","addr":"172.21.15.76:6830","nonce":2382463431}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6824","nonce":2382463431},{"type":"v1","addr":"172.21.15.76:6826","nonce":2382463431}]},"public_addr":"172.21.15.76:6818/2382463431","cluster_addr":"172.21.15.76:6822/2382463431","heartbeat_back_addr":"172.21.15.76:6830/2382463431","heartbeat_front_addr":"172.21.15.76:6826/2382463431","state":["exists","up"]},{"osd":5,"uuid":"485bf711-3851-4238-a4f4-51d52e0a9c58","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6800","nonce":329334308},{"type":"v1","addr":"172.21.15.76:6802","nonce":329334308}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6805","nonce":329334308},{"type":"v1","addr":"172.21.15.76:6807","nonce":329334308}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6814","nonce":329334308},{"type":"v1","addr":"172.21.15.76:6815","nonce":329334308}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6810","nonce":329334308},{"type":"v1","addr":"172.21.15.76:6812","nonce":329334308}]},"public_addr":"172.21.15.76:6802/329334308","cluster_addr":"172.21.15.76:6807/329334308","heartbeat_back_addr":"172.21.15.76:6815/329334308","heartbeat_front_addr":"172.21.15.76:6812/329334308","state":["exists","up"]},{"osd":6,"uuid":"72b3f2fc-1e4a-4e32-968d-003647a0286f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6801","nonce":3946579410},{"type":"v1","addr":"172.21.15.76:6803","nonce":3946579410}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6804","nonce":3946579410},{"type":"v1","addr":"172.21.15.76:6806","nonce":3946579410}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6811","nonce":3946579410},{"type":"v1","addr":"172.21.15.76:6813","nonce":3946579410}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6808","nonce":3946579410},{"type":"v1","addr":"172.21.15.76:6809","nonce":3946579410}]},"public_addr":"172.21.15.76:6803/3946579410","cluster_addr":"172.21.15.76:6806/3946579410","heartbeat_back_addr":"172.21.15.76:6813/3946579410","heartbeat_front_addr":"172.21.15.76:6809/3946579410","state":["exists","up"]},{"osd":7,"uuid":"1ddceb90-6c5d-4d1a-ad56-99cb877f8c70","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6817","nonce":841761538},{"type":"v1","addr":"172.21.15.76:6819","nonce":841761538}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6821","nonce":841761538},{"type":"v1","addr":"172.21.15.76:6823","nonce":841761538}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6829","nonce":841761538},{"type":"v1","addr":"172.21.15.76:6831","nonce":841761538}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6825","nonce":841761538},{"type":"v1","addr":"172.21.15.76:6827","nonce":841761538}]},"public_addr":"172.21.15.76:6819/841761538","cluster_addr":"172.21.15.76:6823/841761538","heartbeat_back_addr":"172.21.15.76:6831/841761538","heartbeat_front_addr":"172.21.15.76:6827/841761538","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T07:56:14.294838+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T07:56:14.270779+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T07:56:14.231809+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T07:56:14.260983+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T07:56:14.302997+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T07:56:14.258600+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T07:56:14.251413+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T07:56:14.304727+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-24T08:00:05.403 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:05.401+0000 7f97c6f52700 1 -- 172.21.15.59:0/1820968244 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f97a406b470 msgr2=0x7f97a406d930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:00:05.404 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:05.401+0000 7f97c6f52700 1 --2- 172.21.15.59:0/1820968244 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f97a406b470 0x7f97a406d930 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f97b4000c00 tx=0x7f97b4013040 comp rx=0 tx=0).stop 2023-11-24T08:00:05.404 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:05.401+0000 7f97c6f52700 1 -- 172.21.15.59:0/1820968244 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f97c00d2ba0 msgr2=0x7f97c00b8ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:00:05.404 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:05.401+0000 7f97c6f52700 1 --2- 172.21.15.59:0/1820968244 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f97c00d2ba0 0x7f97c00b8ac0 secure :-1 s=READY pgs=1028 cs=0 l=1 rev1=1 crypto rx=0x7f97ac00c500 tx=0x7f97ac00c8c0 comp rx=0 tx=0).stop 2023-11-24T08:00:05.404 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:05.401+0000 7f97c6f52700 1 -- 172.21.15.59:0/1820968244 shutdown_connections 2023-11-24T08:00:05.404 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:05.401+0000 7f97c6f52700 1 --2- 172.21.15.59:0/1820968244 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f97a406b470 0x7f97a406d930 unknown :-1 s=CLOSED pgs=476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:00:05.404 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:05.401+0000 7f97c6f52700 1 --2- 172.21.15.59:0/1820968244 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f97c015e460 0x7f97c00b9000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:00:05.404 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:05.401+0000 7f97c6f52700 1 --2- 172.21.15.59:0/1820968244 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f97c00cf480 0x7f97c015f890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:00:05.404 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:05.401+0000 7f97c6f52700 1 --2- 172.21.15.59:0/1820968244 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f97c00d2ba0 0x7f97c00b8ac0 unknown :-1 s=CLOSED pgs=1028 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:00:05.404 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:05.401+0000 7f97c6f52700 1 -- 172.21.15.59:0/1820968244 >> 172.21.15.59:0/1820968244 conn(0x7f97c00d3eb0 msgr2=0x7f97c00b4cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:00:05.404 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:05.402+0000 7f97c6f52700 1 -- 172.21.15.59:0/1820968244 shutdown_connections 2023-11-24T08:00:05.405 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:05.402+0000 7f97c6f52700 1 -- 172.21.15.59:0/1820968244 wait complete. 2023-11-24T08:00:05.416 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2023-11-24T08:00:05.416 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-11-24T08:00:05.541 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:05.540+0000 7fbe3f8f5700 1 Processor -- start 2023-11-24T08:00:05.541 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:05.540+0000 7fbe3f8f5700 1 -- start start 2023-11-24T08:00:05.543 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:05.541+0000 7fbe3f8f5700 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fbe381478c0 0x7fbe380b57b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:00:05.543 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:05.541+0000 7fbe3f8f5700 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fbe3806b6e0 0x7fbe3806e370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:00:05.544 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:05.541+0000 7fbe3f8f5700 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbe3806dbe0 con 0x7fbe38161a90 2023-11-24T08:00:05.544 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:05.541+0000 7fbe3f8f5700 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbe38147ca0 con 0x7fbe3806b6e0 2023-11-24T08:00:05.544 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:05.541+0000 7fbe3f8f5700 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbe381622e0 con 0x7fbe381478c0 2023-11-24T08:00:05.544 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:05.541+0000 7fbe3ce90700 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7fbe38161a90 0x7fbe3806afd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:59370/0 (socket says 172.21.15.59:59370) 2023-11-24T08:00:05.544 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:05.541+0000 7fbe3ce90700 1 -- 172.21.15.59:0/1944690755 learned_addr learned my addr 172.21.15.59:0/1944690755 (peer_addr_for_me v1:172.21.15.59:0/0) 2023-11-24T08:00:05.544 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:05.541+0000 7fbe3d691700 1 --2- 172.21.15.59:0/1944690755 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fbe381478c0 0x7fbe380b57b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:00:05.544 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:05.541+0000 7fbe3de92700 1 --2- 172.21.15.59:0/1944690755 >> v2:172.21.15.76:3300/0 conn(0x7fbe3806b6e0 0x7fbe3806e370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:00:05.544 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:05.541+0000 7fbe2ffff700 1 -- 172.21.15.59:0/1944690755 <== mon.0 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1077141747 0 0) 0x7fbe3806dbe0 con 0x7fbe38161a90 2023-11-24T08:00:05.544 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:05.541+0000 7fbe2ffff700 1 -- 172.21.15.59:0/1944690755 --> v1:172.21.15.59:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbe240035b0 con 0x7fbe38161a90 2023-11-24T08:00:05.544 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:05.541+0000 7fbe2ffff700 1 -- 172.21.15.59:0/1944690755 <== mon.0 v1:172.21.15.59:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3115169702 0 0) 0x7fbe240035b0 con 0x7fbe38161a90 2023-11-24T08:00:05.544 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:05.541+0000 7fbe3de92700 1 -- 172.21.15.59:0/1944690755 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fbe381478c0 msgr2=0x7fbe380b57b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:00:05.544 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:05.541+0000 7fbe3de92700 1 --2- 172.21.15.59:0/1944690755 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fbe381478c0 0x7fbe380b57b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:00:05.545 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:05.541+0000 7fbe3de92700 1 -- 172.21.15.59:0/1944690755 >> v1:172.21.15.59:6789/0 conn(0x7fbe38161a90 legacy=0x7fbe3806afd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:00:05.545 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:05.541+0000 7fbe3de92700 1 -- 172.21.15.59:0/1944690755 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbe381625c0 con 0x7fbe3806b6e0 2023-11-24T08:00:05.545 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:05.542+0000 7fbe3de92700 1 --2- 172.21.15.59:0/1944690755 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fbe3806b6e0 0x7fbe3806e370 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7fbe30009d90 tx=0x7fbe3000b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ca76f553e5f04673 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:00:05.545 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:05.542+0000 7fbe2ffff700 1 -- 172.21.15.59:0/1944690755 <== mon.1 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe3000e070 con 0x7fbe3806b6e0 2023-11-24T08:00:05.545 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:05.542+0000 7fbe2ffff700 1 -- 172.21.15.59:0/1944690755 <== mon.1 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbe30003a20 con 0x7fbe3806b6e0 2023-11-24T08:00:05.545 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:05.542+0000 7fbe2ffff700 1 -- 172.21.15.59:0/1944690755 <== mon.1 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe28001670 con 0x7fbe3806b6e0 2023-11-24T08:00:05.548 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:05.542+0000 7fbe3f8f5700 1 -- 172.21.15.59:0/1944690755 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fbe3806b6e0 msgr2=0x7fbe3806e370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:00:05.548 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:05.542+0000 7fbe3f8f5700 1 --2- 172.21.15.59:0/1944690755 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fbe3806b6e0 0x7fbe3806e370 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7fbe30009d90 tx=0x7fbe3000b330 comp rx=0 tx=0).stop 2023-11-24T08:00:05.548 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:05.542+0000 7fbe3f8f5700 1 -- 172.21.15.59:0/1944690755 shutdown_connections 2023-11-24T08:00:05.548 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:05.542+0000 7fbe3f8f5700 1 --2- 172.21.15.59:0/1944690755 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fbe3806b6e0 0x7fbe3806e370 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:00:05.548 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:05.542+0000 7fbe3f8f5700 1 --2- 172.21.15.59:0/1944690755 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fbe381478c0 0x7fbe380b57b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:00:05.548 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:05.542+0000 7fbe3f8f5700 1 -- 172.21.15.59:0/1944690755 >> 172.21.15.59:0/1944690755 conn(0x7fbe380d0aa0 msgr2=0x7fbe380d0eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:00:05.548 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:05.542+0000 7fbe3f8f5700 1 -- 172.21.15.59:0/1944690755 shutdown_connections 2023-11-24T08:00:05.548 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:05.542+0000 7fbe3f8f5700 1 -- 172.21.15.59:0/1944690755 wait complete. 2023-11-24T08:00:05.548 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:05.544+0000 7fbe3f8f5700 1 Processor -- start 2023-11-24T08:00:05.548 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:05.544+0000 7fbe3f8f5700 1 -- start start 2023-11-24T08:00:05.548 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:05.544+0000 7fbe3f8f5700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fbe3806b6e0 0x7fbe38141070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:00:05.548 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:05.544+0000 7fbe3f8f5700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbe381478c0 0x7fbe380b1e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:00:05.549 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:05.544+0000 7fbe3f8f5700 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fbe38161a90 0x7fbe38140a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:00:05.549 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:05.544+0000 7fbe3f8f5700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe38141970 con 0x7fbe381478c0 2023-11-24T08:00:05.549 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:05.544+0000 7fbe3f8f5700 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe380b2510 con 0x7fbe3806b6e0 2023-11-24T08:00:05.549 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:05.544+0000 7fbe3f8f5700 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbe3815e660 con 0x7fbe38161a90 2023-11-24T08:00:05.549 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:05.544+0000 7fbe3ce90700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbe381478c0 0x7fbe380b1e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:00:05.549 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:05.544+0000 7fbe3ce90700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbe381478c0 0x7fbe380b1e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:55904/0 (socket says 172.21.15.59:55904) 2023-11-24T08:00:05.549 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:05.544+0000 7fbe3ce90700 1 -- 172.21.15.59:0/2506967247 learned_addr learned my addr 172.21.15.59:0/2506967247 (peer_addr_for_me v2:172.21.15.59:0/0) 2023-11-24T08:00:05.549 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:05.545+0000 7fbe3d691700 1 --2- 172.21.15.59:0/2506967247 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fbe3806b6e0 0x7fbe38141070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:00:05.549 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:05.545+0000 7fbe3de92700 1 --2- 172.21.15.59:0/2506967247 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fbe38161a90 0x7fbe38140a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:00:05.549 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:05.545+0000 7fbe3ce90700 1 -- 172.21.15.59:0/2506967247 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fbe38161a90 msgr2=0x7fbe38140a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:00:05.549 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:05.545+0000 7fbe3ce90700 1 --2- 172.21.15.59:0/2506967247 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fbe38161a90 0x7fbe38140a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:00:05.549 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:05.545+0000 7fbe3ce90700 1 -- 172.21.15.59:0/2506967247 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fbe3806b6e0 msgr2=0x7fbe38141070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:00:05.550 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:05.545+0000 7fbe3ce90700 1 --2- 172.21.15.59:0/2506967247 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fbe3806b6e0 0x7fbe38141070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:00:05.550 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:05.545+0000 7fbe3ce90700 1 -- 172.21.15.59:0/2506967247 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbe30009a70 con 0x7fbe381478c0 2023-11-24T08:00:05.550 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:05.545+0000 7fbe3ce90700 1 --2- 172.21.15.59:0/2506967247 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbe381478c0 0x7fbe380b1e30 secure :-1 s=READY pgs=1030 cs=0 l=1 rev1=1 crypto rx=0x7fbe2800c520 tx=0x7fbe2800c830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:00:05.550 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:05.545+0000 7fbe2dffb700 1 -- 172.21.15.59:0/2506967247 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe28004c80 con 0x7fbe381478c0 2023-11-24T08:00:05.550 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:05.545+0000 7fbe2dffb700 1 -- 172.21.15.59:0/2506967247 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbe28004de0 con 0x7fbe381478c0 2023-11-24T08:00:05.550 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:05.545+0000 7fbe3f8f5700 1 -- 172.21.15.59:0/2506967247 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbe3815e940 con 0x7fbe381478c0 2023-11-24T08:00:05.550 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:05.545+0000 7fbe2dffb700 1 -- 172.21.15.59:0/2506967247 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe28007310 con 0x7fbe381478c0 2023-11-24T08:00:05.550 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:05.545+0000 7fbe3f8f5700 1 -- 172.21.15.59:0/2506967247 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbe3815ef00 con 0x7fbe381478c0 2023-11-24T08:00:05.550 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:05.546+0000 7fbe3f8f5700 1 -- 172.21.15.59:0/2506967247 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbe3815f120 con 0x7fbe381478c0 2023-11-24T08:00:05.551 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:05.547+0000 7fbe2dffb700 1 -- 172.21.15.59:0/2506967247 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 11) v1 ==== 89741+0+0 (secure 0 0 0) 0x7fbe28007470 con 0x7fbe381478c0 2023-11-24T08:00:05.551 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:05.547+0000 7fbe2dffb700 1 --2- 172.21.15.59:0/2506967247 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7fbe1c06b470 0x7fbe1c06d930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:00:05.551 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:05.547+0000 7fbe2dffb700 1 -- 172.21.15.59:0/2506967247 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fbe28089cc0 con 0x7fbe381478c0 2023-11-24T08:00:05.551 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:05.547+0000 7fbe3d691700 1 --2- 172.21.15.59:0/2506967247 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7fbe1c06b470 0x7fbe1c06d930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:00:05.552 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:05.548+0000 7fbe3d691700 1 --2- 172.21.15.59:0/2506967247 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7fbe1c06b470 0x7fbe1c06d930 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7fbe380cf0b0 tx=0x7fbe34009210 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:00:05.553 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:05.552+0000 7fbe2dffb700 1 -- 172.21.15.59:0/2506967247 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fbe28054510 con 0x7fbe381478c0 2023-11-24T08:00:05.750 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:05.748+0000 7fbe3f8f5700 1 -- 172.21.15.59:0/2506967247 --> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fbe3815e7d0 con 0x7fbe1c06b470 2023-11-24T08:00:05.758 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:05.757+0000 7fbe2dffb700 1 -- 172.21.15.59:0/2506967247 <== mgr.4100 v2:172.21.15.76:6832/4264046536 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+289446 (secure 0 0 0) 0x7fbe3815e7d0 con 0x7fbe1c06b470 2023-11-24T08:00:05.763 INFO:teuthology.orchestra.run.smithi059.stdout: 2023-11-24T08:00:05.766 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:05.760+0000 7fbe3f8f5700 1 -- 172.21.15.59:0/2506967247 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7fbe1c06b470 msgr2=0x7fbe1c06d930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:00:05.766 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:05.760+0000 7fbe3f8f5700 1 --2- 172.21.15.59:0/2506967247 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7fbe1c06b470 0x7fbe1c06d930 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7fbe380cf0b0 tx=0x7fbe34009210 comp rx=0 tx=0).stop 2023-11-24T08:00:05.766 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:05.761+0000 7fbe3f8f5700 1 -- 172.21.15.59:0/2506967247 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbe381478c0 msgr2=0x7fbe380b1e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:00:05.766 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:05.761+0000 7fbe3f8f5700 1 --2- 172.21.15.59:0/2506967247 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbe381478c0 0x7fbe380b1e30 secure :-1 s=READY pgs=1030 cs=0 l=1 rev1=1 crypto rx=0x7fbe2800c520 tx=0x7fbe2800c830 comp rx=0 tx=0).stop 2023-11-24T08:00:05.766 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:05.761+0000 7fbe3f8f5700 1 -- 172.21.15.59:0/2506967247 shutdown_connections 2023-11-24T08:00:05.766 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:05.761+0000 7fbe3f8f5700 1 --2- 172.21.15.59:0/2506967247 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7fbe1c06b470 0x7fbe1c06d930 unknown :-1 s=CLOSED pgs=477 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:00:05.766 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:05.761+0000 7fbe3f8f5700 1 --2- 172.21.15.59:0/2506967247 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fbe38161a90 0x7fbe38140a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:00:05.767 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:05.761+0000 7fbe3f8f5700 1 --2- 172.21.15.59:0/2506967247 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fbe3806b6e0 0x7fbe38141070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:00:05.767 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:05.761+0000 7fbe3f8f5700 1 --2- 172.21.15.59:0/2506967247 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbe381478c0 0x7fbe380b1e30 unknown :-1 s=CLOSED pgs=1030 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:00:05.767 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:05.761+0000 7fbe3f8f5700 1 -- 172.21.15.59:0/2506967247 >> 172.21.15.59:0/2506967247 conn(0x7fbe380d0aa0 msgr2=0x7fbe3806e270 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:00:05.767 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:05.761+0000 7fbe3f8f5700 1 -- 172.21.15.59:0/2506967247 shutdown_connections 2023-11-24T08:00:05.767 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:05.761+0000 7fbe3f8f5700 1 -- 172.21.15.59:0/2506967247 wait complete. 2023-11-24T08:00:05.767 INFO:teuthology.orchestra.run.smithi059.stderr:dumped all 2023-11-24T08:00:05.775 INFO:teuthology.orchestra.run.smithi059.stdout:{"pg_ready":true,"pg_map":{"version":193,"stamp":"2023-11-24T08:00:04.350377+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":229928,"kb_used_data":9576,"kb_used_omap":12,"kb_used_meta":220211,"kb_avail":754744792,"statfs":{"total":773094113280,"available":772858667008,"internally_reserved":0,"allocated":9805824,"data_stored":4858476,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12745,"internal_metadata":225496631},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"10.845551"},"pg_stats":[{"pgid":"13.3f","version":"0'0","reported_seq":11,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-11-24T07:59:53.608729+0000","last_change":"2023-11-24T07:59:52.533349+0000","last_active":"2023-11-24T07:59:53.608729+0000","last_peered":"2023-11-24T07:59:53.608729+0000","last_clean":"2023-11-24T07:59:53.608729+0000","last_became_active":"2023-11-24T07:59:52.532951+0000","last_became_peered":"2023-11-24T07:59:52.532951+0000","last_unstale":"2023-11-24T07:59:53.608729+0000","last_undegraded":"2023-11-24T07:59:53.608729+0000","last_fullsized":"2023-11-24T07:59:53.608729+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:59:51.477944+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:59:51.477944+0000","last_clean_scrub_stamp":"2023-11-24T07:59:51.477944+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T16:38:43.600527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.3e","version":"0'0","reported_seq":11,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-11-24T07:59:53.608999+0000","last_change":"2023-11-24T07:59:52.531701+0000","last_active":"2023-11-24T07:59:53.608999+0000","last_peered":"2023-11-24T07:59:53.608999+0000","last_clean":"2023-11-24T07:59:53.608999+0000","last_became_active":"2023-11-24T07:59:52.531393+0000","last_became_peered":"2023-11-24T07:59:52.531393+0000","last_unstale":"2023-11-24T07:59:53.608999+0000","last_undegraded":"2023-11-24T07:59:53.608999+0000","last_fullsized":"2023-11-24T07:59:53.608999+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:59:51.477944+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:59:51.477944+0000","last_clean_scrub_stamp":"2023-11-24T07:59:51.477944+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T11:25:48.888553+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.3d","version":"0'0","reported_seq":11,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-11-24T07:59:53.608830+0000","last_change":"2023-11-24T07:59:52.532095+0000","last_active":"2023-11-24T07:59:53.608830+0000","last_peered":"2023-11-24T07:59:53.608830+0000","last_clean":"2023-11-24T07:59:53.608830+0000","last_became_active":"2023-11-24T07:59:52.531717+0000","last_became_peered":"2023-11-24T07:59:52.531717+0000","last_unstale":"2023-11-24T07:59:53.608830+0000","last_undegraded":"2023-11-24T07:59:53.608830+0000","last_fullsized":"2023-11-24T07:59:53.608830+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:59:51.477944+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:59:51.477944+0000","last_clean_scrub_stamp":"2023-11-24T07:59:51.477944+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T12:58:00.253001+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.3c","version":"0'0","reported_seq":11,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-11-24T07:59:53.506857+0000","last_change":"2023-11-24T07:59:52.531686+0000","last_active":"2023-11-24T07:59:53.506857+0000","last_peered":"2023-11-24T07:59:53.506857+0000","last_clean":"2023-11-24T07:59:53.506857+0000","last_became_active":"2023-11-24T07:59:52.531434+0000","last_became_peered":"2023-11-24T07:59:52.531434+0000","last_unstale":"2023-11-24T07:59:53.506857+0000","last_undegraded":"2023-11-24T07:59:53.506857+0000","last_fullsized":"2023-11-24T07:59:53.506857+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:59:51.477944+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:59:51.477944+0000","last_clean_scrub_stamp":"2023-11-24T07:59:51.477944+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T13:08:53.960278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.3b","version":"0'0","reported_seq":11,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-11-24T07:59:53.505961+0000","last_change":"2023-11-24T07:59:52.520726+0000","last_active":"2023-11-24T07:59:53.505961+0000","last_peered":"2023-11-24T07:59:53.505961+0000","last_clean":"2023-11-24T07:59:53.505961+0000","last_became_active":"2023-11-24T07:59:52.520430+0000","last_became_peered":"2023-11-24T07:59:52.520430+0000","last_unstale":"2023-11-24T07:59:53.505961+0000","last_undegraded":"2023-11-24T07:59:53.505961+0000","last_fullsized":"2023-11-24T07:59:53.505961+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:59:51.477944+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:59:51.477944+0000","last_clean_scrub_stamp":"2023-11-24T07:59:51.477944+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T15:47:53.005002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.3a","version":"0'0","reported_seq":11,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-11-24T07:59:53.608618+0000","last_change":"2023-11-24T07:59:52.531758+0000","last_active":"2023-11-24T07:59:53.608618+0000","last_peered":"2023-11-24T07:59:53.608618+0000","last_clean":"2023-11-24T07:59:53.608618+0000","last_became_active":"2023-11-24T07:59:52.531393+0000","last_became_peered":"2023-11-24T07:59:52.531393+0000","last_unstale":"2023-11-24T07:59:53.608618+0000","last_undegraded":"2023-11-24T07:59:53.608618+0000","last_fullsized":"2023-11-24T07:59:53.608618+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:59:51.477944+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:59:51.477944+0000","last_clean_scrub_stamp":"2023-11-24T07:59:51.477944+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T19:59:12.479564+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.39","version":"0'0","reported_seq":11,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-11-24T07:59:53.608610+0000","last_change":"2023-11-24T07:59:52.516615+0000","last_active":"2023-11-24T07:59:53.608610+0000","last_peered":"2023-11-24T07:59:53.608610+0000","last_clean":"2023-11-24T07:59:53.608610+0000","last_became_active":"2023-11-24T07:59:52.516230+0000","last_became_peered":"2023-11-24T07:59:52.516230+0000","last_unstale":"2023-11-24T07:59:53.608610+0000","last_undegraded":"2023-11-24T07:59:53.608610+0000","last_fullsized":"2023-11-24T07:59:53.608610+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:59:51.477944+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:59:51.477944+0000","last_clean_scrub_stamp":"2023-11-24T07:59:51.477944+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T17:43:33.689547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.38","version":"0'0","reported_seq":11,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-11-24T07:59:53.607999+0000","last_change":"2023-11-24T07:59:52.518308+0000","last_active":"2023-11-24T07:59:53.607999+0000","last_peered":"2023-11-24T07:59:53.607999+0000","last_clean":"2023-11-24T07:59:53.607999+0000","last_became_active":"2023-11-24T07:59:52.517481+0000","last_became_peered":"2023-11-24T07:59:52.517481+0000","last_unstale":"2023-11-24T07:59:53.607999+0000","last_undegraded":"2023-11-24T07:59:53.607999+0000","last_fullsized":"2023-11-24T07:59:53.607999+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:59:51.477944+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:59:51.477944+0000","last_clean_scrub_stamp":"2023-11-24T07:59:51.477944+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T13:03:18.643127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.37","version":"0'0","reported_seq":11,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-11-24T07:59:53.608855+0000","last_change":"2023-11-24T07:59:52.520229+0000","last_active":"2023-11-24T07:59:53.608855+0000","last_peered":"2023-11-24T07:59:53.608855+0000","last_clean":"2023-11-24T07:59:53.608855+0000","last_became_active":"2023-11-24T07:59:52.519949+0000","last_became_peered":"2023-11-24T07:59:52.519949+0000","last_unstale":"2023-11-24T07:59:53.608855+0000","last_undegraded":"2023-11-24T07:59:53.608855+0000","last_fullsized":"2023-11-24T07:59:53.608855+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:59:51.477944+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:59:51.477944+0000","last_clean_scrub_stamp":"2023-11-24T07:59:51.477944+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T11:25:48.888553+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.36","version":"0'0","reported_seq":11,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-11-24T07:59:53.609124+0000","last_change":"2023-11-24T07:59:52.530667+0000","last_active":"2023-11-24T07:59:53.609124+0000","last_peered":"2023-11-24T07:59:53.609124+0000","last_clean":"2023-11-24T07:59:53.609124+0000","last_became_active":"2023-11-24T07:59:52.530370+0000","last_became_peered":"2023-11-24T07:59:52.530370+0000","last_unstale":"2023-11-24T07:59:53.609124+0000","last_undegraded":"2023-11-24T07:59:53.609124+0000","last_fullsized":"2023-11-24T07:59:53.609124+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:59:51.477944+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:59:51.477944+0000","last_clean_scrub_stamp":"2023-11-24T07:59:51.477944+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T18:55:14.661533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.35","version":"0'0","reported_seq":11,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-11-24T07:59:53.608580+0000","last_change":"2023-11-24T07:59:52.532327+0000","last_active":"2023-11-24T07:59:53.608580+0000","last_peered":"2023-11-24T07:59:53.608580+0000","last_clean":"2023-11-24T07:59:53.608580+0000","last_became_active":"2023-11-24T07:59:52.532053+0000","last_became_peered":"2023-11-24T07:59:52.532053+0000","last_unstale":"2023-11-24T07:59:53.608580+0000","last_undegraded":"2023-11-24T07:59:53.608580+0000","last_fullsized":"2023-11-24T07:59:53.608580+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:59:51.477944+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:59:51.477944+0000","last_clean_scrub_stamp":"2023-11-24T07:59:51.477944+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:25:11.323450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.34","version":"0'0","reported_seq":11,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-11-24T07:59:53.609558+0000","last_change":"2023-11-24T07:59:52.520219+0000","last_active":"2023-11-24T07:59:53.609558+0000","last_peered":"2023-11-24T07:59:53.609558+0000","last_clean":"2023-11-24T07:59:53.609558+0000","last_became_active":"2023-11-24T07:59:52.519942+0000","last_became_peered":"2023-11-24T07:59:52.519942+0000","last_unstale":"2023-11-24T07:59:53.609558+0000","last_undegraded":"2023-11-24T07:59:53.609558+0000","last_fullsized":"2023-11-24T07:59:53.609558+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:59:51.477944+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:59:51.477944+0000","last_clean_scrub_stamp":"2023-11-24T07:59:51.477944+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T09:35:30.735874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.33","version":"0'0","reported_seq":11,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-11-24T07:59:53.505979+0000","last_change":"2023-11-24T07:59:52.535905+0000","last_active":"2023-11-24T07:59:53.505979+0000","last_peered":"2023-11-24T07:59:53.505979+0000","last_clean":"2023-11-24T07:59:53.505979+0000","last_became_active":"2023-11-24T07:59:52.535592+0000","last_became_peered":"2023-11-24T07:59:52.535592+0000","last_unstale":"2023-11-24T07:59:53.505979+0000","last_undegraded":"2023-11-24T07:59:53.505979+0000","last_fullsized":"2023-11-24T07:59:53.505979+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:59:51.477944+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:59:51.477944+0000","last_clean_scrub_stamp":"2023-11-24T07:59:51.477944+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T19:41:55.782652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.32","version":"0'0","reported_seq":11,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-11-24T07:59:53.505391+0000","last_change":"2023-11-24T07:59:52.518344+0000","last_active":"2023-11-24T07:59:53.505391+0000","last_peered":"2023-11-24T07:59:53.505391+0000","last_clean":"2023-11-24T07:59:53.505391+0000","last_became_active":"2023-11-24T07:59:52.518067+0000","last_became_peered":"2023-11-24T07:59:52.518067+0000","last_unstale":"2023-11-24T07:59:53.505391+0000","last_undegraded":"2023-11-24T07:59:53.505391+0000","last_fullsized":"2023-11-24T07:59:53.505391+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:59:51.477944+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:59:51.477944+0000","last_clean_scrub_stamp":"2023-11-24T07:59:51.477944+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T11:49:25.692175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.31","version":"0'0","reported_seq":11,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-11-24T07:59:53.608788+0000","last_change":"2023-11-24T07:59:52.509975+0000","last_active":"2023-11-24T07:59:53.608788+0000","last_peered":"2023-11-24T07:59:53.608788+0000","last_clean":"2023-11-24T07:59:53.608788+0000","last_became_active":"2023-11-24T07:59:52.509707+0000","last_became_peered":"2023-11-24T07:59:52.509707+0000","last_unstale":"2023-11-24T07:59:53.608788+0000","last_undegraded":"2023-11-24T07:59:53.608788+0000","last_fullsized":"2023-11-24T07:59:53.608788+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:59:51.477944+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:59:51.477944+0000","last_clean_scrub_stamp":"2023-11-24T07:59:51.477944+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T16:38:43.600527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.30","version":"0'0","reported_seq":11,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-11-24T07:59:53.608396+0000","last_change":"2023-11-24T07:59:52.530519+0000","last_active":"2023-11-24T07:59:53.608396+0000","last_peered":"2023-11-24T07:59:53.608396+0000","last_clean":"2023-11-24T07:59:53.608396+0000","last_became_active":"2023-11-24T07:59:52.530206+0000","last_became_peered":"2023-11-24T07:59:52.530206+0000","last_unstale":"2023-11-24T07:59:53.608396+0000","last_undegraded":"2023-11-24T07:59:53.608396+0000","last_fullsized":"2023-11-24T07:59:53.608396+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:59:51.477944+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:59:51.477944+0000","last_clean_scrub_stamp":"2023-11-24T07:59:51.477944+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T19:33:56.535985+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.2f","version":"0'0","reported_seq":11,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-11-24T07:59:53.610055+0000","last_change":"2023-11-24T07:59:52.520834+0000","last_active":"2023-11-24T07:59:53.610055+0000","last_peered":"2023-11-24T07:59:53.610055+0000","last_clean":"2023-11-24T07:59:53.610055+0000","last_became_active":"2023-11-24T07:59:52.520529+0000","last_became_peered":"2023-11-24T07:59:52.520529+0000","last_unstale":"2023-11-24T07:59:53.610055+0000","last_undegraded":"2023-11-24T07:59:53.610055+0000","last_fullsized":"2023-11-24T07:59:53.610055+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:59:51.477944+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:59:51.477944+0000","last_clean_scrub_stamp":"2023-11-24T07:59:51.477944+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T19:59:12.479564+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.2e","version":"0'0","reported_seq":11,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-11-24T07:59:53.609062+0000","last_change":"2023-11-24T07:59:52.523045+0000","last_active":"2023-11-24T07:59:53.609062+0000","last_peered":"2023-11-24T07:59:53.609062+0000","last_clean":"2023-11-24T07:59:53.609062+0000","last_became_active":"2023-11-24T07:59:52.522759+0000","last_became_peered":"2023-11-24T07:59:52.522759+0000","last_unstale":"2023-11-24T07:59:53.609062+0000","last_undegraded":"2023-11-24T07:59:53.609062+0000","last_fullsized":"2023-11-24T07:59:53.609062+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:59:51.477944+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:59:51.477944+0000","last_clean_scrub_stamp":"2023-11-24T07:59:51.477944+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T13:20:44.301407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.2d","version":"0'0","reported_seq":11,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-11-24T07:59:53.505727+0000","last_change":"2023-11-24T07:59:52.519290+0000","last_active":"2023-11-24T07:59:53.505727+0000","last_peered":"2023-11-24T07:59:53.505727+0000","last_clean":"2023-11-24T07:59:53.505727+0000","last_became_active":"2023-11-24T07:59:52.519007+0000","last_became_peered":"2023-11-24T07:59:52.519007+0000","last_unstale":"2023-11-24T07:59:53.505727+0000","last_undegraded":"2023-11-24T07:59:53.505727+0000","last_fullsized":"2023-11-24T07:59:53.505727+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:59:51.477944+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:59:51.477944+0000","last_clean_scrub_stamp":"2023-11-24T07:59:51.477944+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T15:52:11.017061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.2c","version":"0'0","reported_seq":11,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-11-24T07:59:53.505856+0000","last_change":"2023-11-24T07:59:52.518769+0000","last_active":"2023-11-24T07:59:53.505856+0000","last_peered":"2023-11-24T07:59:53.505856+0000","last_clean":"2023-11-24T07:59:53.505856+0000","last_became_active":"2023-11-24T07:59:52.518377+0000","last_became_peered":"2023-11-24T07:59:52.518377+0000","last_unstale":"2023-11-24T07:59:53.505856+0000","last_undegraded":"2023-11-24T07:59:53.505856+0000","last_fullsized":"2023-11-24T07:59:53.505856+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:59:51.477944+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:59:51.477944+0000","last_clean_scrub_stamp":"2023-11-24T07:59:51.477944+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T08:19:08.702386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.2b","version":"0'0","reported_seq":11,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-11-24T07:59:53.506840+0000","last_change":"2023-11-24T07:59:52.509772+0000","last_active":"2023-11-24T07:59:53.506840+0000","last_peered":"2023-11-24T07:59:53.506840+0000","last_clean":"2023-11-24T07:59:53.506840+0000","last_became_active":"2023-11-24T07:59:52.509458+0000","last_became_peered":"2023-11-24T07:59:52.509458+0000","last_unstale":"2023-11-24T07:59:53.506840+0000","last_undegraded":"2023-11-24T07:59:53.506840+0000","last_fullsized":"2023-11-24T07:59:53.506840+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:59:51.477944+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:59:51.477944+0000","last_clean_scrub_stamp":"2023-11-24T07:59:51.477944+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T15:50:40.414696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.2a","version":"0'0","reported_seq":11,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-11-24T07:59:53.505572+0000","last_change":"2023-11-24T07:59:52.518858+0000","last_active":"2023-11-24T07:59:53.505572+0000","last_peered":"2023-11-24T07:59:53.505572+0000","last_clean":"2023-11-24T07:59:53.505572+0000","last_became_active":"2023-11-24T07:59:52.518579+0000","last_became_peered":"2023-11-24T07:59:52.518579+0000","last_unstale":"2023-11-24T07:59:53.505572+0000","last_undegraded":"2023-11-24T07:59:53.505572+0000","last_fullsized":"2023-11-24T07:59:53.505572+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:59:51.477944+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:59:51.477944+0000","last_clean_scrub_stamp":"2023-11-24T07:59:51.477944+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T14:16:27.359759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.29","version":"0'0","reported_seq":11,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-11-24T07:59:53.609416+0000","last_change":"2023-11-24T07:59:52.522726+0000","last_active":"2023-11-24T07:59:53.609416+0000","last_peered":"2023-11-24T07:59:53.609416+0000","last_clean":"2023-11-24T07:59:53.609416+0000","last_became_active":"2023-11-24T07:59:52.522310+0000","last_became_peered":"2023-11-24T07:59:52.522310+0000","last_unstale":"2023-11-24T07:59:53.609416+0000","last_undegraded":"2023-11-24T07:59:53.609416+0000","last_fullsized":"2023-11-24T07:59:53.609416+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:59:51.477944+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:59:51.477944+0000","last_clean_scrub_stamp":"2023-11-24T07:59:51.477944+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T17:45:56.521663+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.28","version":"0'0","reported_seq":11,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-11-24T07:59:53.608491+0000","last_change":"2023-11-24T07:59:52.515597+0000","last_active":"2023-11-24T07:59:53.608491+0000","last_peered":"2023-11-24T07:59:53.608491+0000","last_clean":"2023-11-24T07:59:53.608491+0000","last_became_active":"2023-11-24T07:59:52.515247+0000","last_became_peered":"2023-11-24T07:59:52.515247+0000","last_unstale":"2023-11-24T07:59:53.608491+0000","last_undegraded":"2023-11-24T07:59:53.608491+0000","last_fullsized":"2023-11-24T07:59:53.608491+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:59:51.477944+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:59:51.477944+0000","last_clean_scrub_stamp":"2023-11-24T07:59:51.477944+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T15:14:48.860422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.27","version":"0'0","reported_seq":11,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-11-24T07:59:53.608968+0000","last_change":"2023-11-24T07:59:52.536629+0000","last_active":"2023-11-24T07:59:53.608968+0000","last_peered":"2023-11-24T07:59:53.608968+0000","last_clean":"2023-11-24T07:59:53.608968+0000","last_became_active":"2023-11-24T07:59:52.535822+0000","last_became_peered":"2023-11-24T07:59:52.535822+0000","last_unstale":"2023-11-24T07:59:53.608968+0000","last_undegraded":"2023-11-24T07:59:53.608968+0000","last_fullsized":"2023-11-24T07:59:53.608968+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:59:51.477944+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:59:51.477944+0000","last_clean_scrub_stamp":"2023-11-24T07:59:51.477944+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T19:07:22.502426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.26","version":"0'0","reported_seq":11,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-11-24T07:59:53.506206+0000","last_change":"2023-11-24T07:59:52.521004+0000","last_active":"2023-11-24T07:59:53.506206+0000","last_peered":"2023-11-24T07:59:53.506206+0000","last_clean":"2023-11-24T07:59:53.506206+0000","last_became_active":"2023-11-24T07:59:52.520546+0000","last_became_peered":"2023-11-24T07:59:52.520546+0000","last_unstale":"2023-11-24T07:59:53.506206+0000","last_undegraded":"2023-11-24T07:59:53.506206+0000","last_fullsized":"2023-11-24T07:59:53.506206+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:59:51.477944+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:59:51.477944+0000","last_clean_scrub_stamp":"2023-11-24T07:59:51.477944+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T15:25:25.968856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.25","version":"0'0","reported_seq":11,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-11-24T07:59:53.506158+0000","last_change":"2023-11-24T07:59:52.513690+0000","last_active":"2023-11-24T07:59:53.506158+0000","last_peered":"2023-11-24T07:59:53.506158+0000","last_clean":"2023-11-24T07:59:53.506158+0000","last_became_active":"2023-11-24T07:59:52.513378+0000","last_became_peered":"2023-11-24T07:59:52.513378+0000","last_unstale":"2023-11-24T07:59:53.506158+0000","last_undegraded":"2023-11-24T07:59:53.506158+0000","last_fullsized":"2023-11-24T07:59:53.506158+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:59:51.477944+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:59:51.477944+0000","last_clean_scrub_stamp":"2023-11-24T07:59:51.477944+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T12:11:43.705658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.24","version":"0'0","reported_seq":11,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-11-24T07:59:53.608136+0000","last_change":"2023-11-24T07:59:52.525922+0000","last_active":"2023-11-24T07:59:53.608136+0000","last_peered":"2023-11-24T07:59:53.608136+0000","last_clean":"2023-11-24T07:59:53.608136+0000","last_became_active":"2023-11-24T07:59:52.525654+0000","last_became_peered":"2023-11-24T07:59:52.525654+0000","last_unstale":"2023-11-24T07:59:53.608136+0000","last_undegraded":"2023-11-24T07:59:53.608136+0000","last_fullsized":"2023-11-24T07:59:53.608136+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:59:51.477944+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:59:51.477944+0000","last_clean_scrub_stamp":"2023-11-24T07:59:51.477944+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T08:10:30.910525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.23","version":"0'0","reported_seq":11,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-11-24T07:59:53.506690+0000","last_change":"2023-11-24T07:59:52.510412+0000","last_active":"2023-11-24T07:59:53.506690+0000","last_peered":"2023-11-24T07:59:53.506690+0000","last_clean":"2023-11-24T07:59:53.506690+0000","last_became_active":"2023-11-24T07:59:52.510126+0000","last_became_peered":"2023-11-24T07:59:52.510126+0000","last_unstale":"2023-11-24T07:59:53.506690+0000","last_undegraded":"2023-11-24T07:59:53.506690+0000","last_fullsized":"2023-11-24T07:59:53.506690+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:59:51.477944+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:59:51.477944+0000","last_clean_scrub_stamp":"2023-11-24T07:59:51.477944+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T15:40:18.171862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.22","version":"0'0","reported_seq":11,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-11-24T07:59:53.505766+0000","last_change":"2023-11-24T07:59:52.518764+0000","last_active":"2023-11-24T07:59:53.505766+0000","last_peered":"2023-11-24T07:59:53.505766+0000","last_clean":"2023-11-24T07:59:53.505766+0000","last_became_active":"2023-11-24T07:59:52.518431+0000","last_became_peered":"2023-11-24T07:59:52.518431+0000","last_unstale":"2023-11-24T07:59:53.505766+0000","last_undegraded":"2023-11-24T07:59:53.505766+0000","last_fullsized":"2023-11-24T07:59:53.505766+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:59:51.477944+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:59:51.477944+0000","last_clean_scrub_stamp":"2023-11-24T07:59:51.477944+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T08:25:07.421503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.21","version":"0'0","reported_seq":11,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-11-24T07:59:53.608233+0000","last_change":"2023-11-24T07:59:52.531404+0000","last_active":"2023-11-24T07:59:53.608233+0000","last_peered":"2023-11-24T07:59:53.608233+0000","last_clean":"2023-11-24T07:59:53.608233+0000","last_became_active":"2023-11-24T07:59:52.531032+0000","last_became_peered":"2023-11-24T07:59:52.531032+0000","last_unstale":"2023-11-24T07:59:53.608233+0000","last_undegraded":"2023-11-24T07:59:53.608233+0000","last_fullsized":"2023-11-24T07:59:53.608233+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:59:51.477944+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:59:51.477944+0000","last_clean_scrub_stamp":"2023-11-24T07:59:51.477944+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T09:35:30.735874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.20","version":"0'0","reported_seq":11,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-11-24T07:59:53.608451+0000","last_change":"2023-11-24T07:59:52.530888+0000","last_active":"2023-11-24T07:59:53.608451+0000","last_peered":"2023-11-24T07:59:53.608451+0000","last_clean":"2023-11-24T07:59:53.608451+0000","last_became_active":"2023-11-24T07:59:52.530528+0000","last_became_peered":"2023-11-24T07:59:52.530528+0000","last_unstale":"2023-11-24T07:59:53.608451+0000","last_undegraded":"2023-11-24T07:59:53.608451+0000","last_fullsized":"2023-11-24T07:59:53.608451+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:59:51.477944+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:59:51.477944+0000","last_clean_scrub_stamp":"2023-11-24T07:59:51.477944+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T16:20:03.228953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.1e","version":"0'0","reported_seq":11,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-11-24T07:59:53.609869+0000","last_change":"2023-11-24T07:59:52.525254+0000","last_active":"2023-11-24T07:59:53.609869+0000","last_peered":"2023-11-24T07:59:53.609869+0000","last_clean":"2023-11-24T07:59:53.609869+0000","last_became_active":"2023-11-24T07:59:52.524955+0000","last_became_peered":"2023-11-24T07:59:52.524955+0000","last_unstale":"2023-11-24T07:59:53.609869+0000","last_undegraded":"2023-11-24T07:59:53.609869+0000","last_fullsized":"2023-11-24T07:59:53.609869+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:59:51.477944+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:59:51.477944+0000","last_clean_scrub_stamp":"2023-11-24T07:59:51.477944+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T17:27:50.274990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.1f","version":"86'4","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-11-24T07:59:53.373372+0000","last_change":"2023-11-24T07:59:51.501301+0000","last_active":"2023-11-24T07:59:53.373372+0000","last_peered":"2023-11-24T07:59:53.373372+0000","last_clean":"2023-11-24T07:59:53.373372+0000","last_became_active":"2023-11-24T07:59:51.500984+0000","last_became_peered":"2023-11-24T07:59:51.500984+0000","last_unstale":"2023-11-24T07:59:53.373372+0000","last_undegraded":"2023-11-24T07:59:53.373372+0000","last_fullsized":"2023-11-24T07:59:53.373372+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:59:50.473699+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:59:50.473699+0000","last_clean_scrub_stamp":"2023-11-24T07:59:50.473699+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T19:08:46.605605+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.1f","version":"0'0","reported_seq":11,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-11-24T07:59:53.608973+0000","last_change":"2023-11-24T07:59:52.533153+0000","last_active":"2023-11-24T07:59:53.608973+0000","last_peered":"2023-11-24T07:59:53.608973+0000","last_clean":"2023-11-24T07:59:53.608973+0000","last_became_active":"2023-11-24T07:59:52.532666+0000","last_became_peered":"2023-11-24T07:59:52.532666+0000","last_unstale":"2023-11-24T07:59:53.608973+0000","last_undegraded":"2023-11-24T07:59:53.608973+0000","last_fullsized":"2023-11-24T07:59:53.608973+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:59:51.477944+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:59:51.477944+0000","last_clean_scrub_stamp":"2023-11-24T07:59:51.477944+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T15:14:48.860422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.1e","version":"86'1","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-11-24T07:59:53.367164+0000","last_change":"2023-11-24T07:59:51.500301+0000","last_active":"2023-11-24T07:59:53.367164+0000","last_peered":"2023-11-24T07:59:53.367164+0000","last_clean":"2023-11-24T07:59:53.367164+0000","last_became_active":"2023-11-24T07:59:51.499985+0000","last_became_peered":"2023-11-24T07:59:51.499985+0000","last_unstale":"2023-11-24T07:59:53.367164+0000","last_undegraded":"2023-11-24T07:59:53.367164+0000","last_fullsized":"2023-11-24T07:59:53.367164+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:59:50.473699+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:59:50.473699+0000","last_clean_scrub_stamp":"2023-11-24T07:59:50.473699+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T17:45:55.517418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.1c","version":"0'0","reported_seq":11,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-11-24T07:59:53.609568+0000","last_change":"2023-11-24T07:59:52.517822+0000","last_active":"2023-11-24T07:59:53.609568+0000","last_peered":"2023-11-24T07:59:53.609568+0000","last_clean":"2023-11-24T07:59:53.609568+0000","last_became_active":"2023-11-24T07:59:52.517510+0000","last_became_peered":"2023-11-24T07:59:52.517510+0000","last_unstale":"2023-11-24T07:59:53.609568+0000","last_undegraded":"2023-11-24T07:59:53.609568+0000","last_fullsized":"2023-11-24T07:59:53.609568+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:59:51.477944+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:59:51.477944+0000","last_clean_scrub_stamp":"2023-11-24T07:59:51.477944+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T13:17:26.610425+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.1d","version":"86'2","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-11-24T07:59:53.373128+0000","last_change":"2023-11-24T07:59:51.500384+0000","last_active":"2023-11-24T07:59:53.373128+0000","last_peered":"2023-11-24T07:59:53.373128+0000","last_clean":"2023-11-24T07:59:53.373128+0000","last_became_active":"2023-11-24T07:59:51.500044+0000","last_became_peered":"2023-11-24T07:59:51.500044+0000","last_unstale":"2023-11-24T07:59:53.373128+0000","last_undegraded":"2023-11-24T07:59:53.373128+0000","last_fullsized":"2023-11-24T07:59:53.373128+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:59:50.473699+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:59:50.473699+0000","last_clean_scrub_stamp":"2023-11-24T07:59:50.473699+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T08:10:29.906280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.1d","version":"0'0","reported_seq":11,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-11-24T07:59:53.608370+0000","last_change":"2023-11-24T07:59:52.527299+0000","last_active":"2023-11-24T07:59:53.608370+0000","last_peered":"2023-11-24T07:59:53.608370+0000","last_clean":"2023-11-24T07:59:53.608370+0000","last_became_active":"2023-11-24T07:59:52.523777+0000","last_became_peered":"2023-11-24T07:59:52.523777+0000","last_unstale":"2023-11-24T07:59:53.608370+0000","last_undegraded":"2023-11-24T07:59:53.608370+0000","last_fullsized":"2023-11-24T07:59:53.608370+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:59:51.477944+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:59:51.477944+0000","last_clean_scrub_stamp":"2023-11-24T07:59:51.477944+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T17:43:33.689547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.1c","version":"86'2","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-11-24T07:59:53.374704+0000","last_change":"2023-11-24T07:59:51.504319+0000","last_active":"2023-11-24T07:59:53.374704+0000","last_peered":"2023-11-24T07:59:53.374704+0000","last_clean":"2023-11-24T07:59:53.374704+0000","last_became_active":"2023-11-24T07:59:51.503943+0000","last_became_peered":"2023-11-24T07:59:51.503943+0000","last_unstale":"2023-11-24T07:59:53.374704+0000","last_undegraded":"2023-11-24T07:59:53.374704+0000","last_fullsized":"2023-11-24T07:59:53.374704+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:59:50.473699+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:59:50.473699+0000","last_clean_scrub_stamp":"2023-11-24T07:59:50.473699+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T11:53:53.251329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.1a","version":"0'0","reported_seq":11,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-11-24T07:59:53.507108+0000","last_change":"2023-11-24T07:59:52.515179+0000","last_active":"2023-11-24T07:59:53.507108+0000","last_peered":"2023-11-24T07:59:53.507108+0000","last_clean":"2023-11-24T07:59:53.507108+0000","last_became_active":"2023-11-24T07:59:52.514892+0000","last_became_peered":"2023-11-24T07:59:52.514892+0000","last_unstale":"2023-11-24T07:59:53.507108+0000","last_undegraded":"2023-11-24T07:59:53.507108+0000","last_fullsized":"2023-11-24T07:59:53.507108+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:59:51.477944+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:59:51.477944+0000","last_clean_scrub_stamp":"2023-11-24T07:59:51.477944+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:57:55.850876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.1b","version":"86'1","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-11-24T07:59:53.366988+0000","last_change":"2023-11-24T07:59:51.500410+0000","last_active":"2023-11-24T07:59:53.366988+0000","last_peered":"2023-11-24T07:59:53.366988+0000","last_clean":"2023-11-24T07:59:53.366988+0000","last_became_active":"2023-11-24T07:59:51.500029+0000","last_became_peered":"2023-11-24T07:59:51.500029+0000","last_unstale":"2023-11-24T07:59:53.366988+0000","last_undegraded":"2023-11-24T07:59:53.366988+0000","last_fullsized":"2023-11-24T07:59:53.366988+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:59:50.473699+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:59:50.473699+0000","last_clean_scrub_stamp":"2023-11-24T07:59:50.473699+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T15:52:33.660947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.1b","version":"0'0","reported_seq":11,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-11-24T07:59:53.608579+0000","last_change":"2023-11-24T07:59:52.530658+0000","last_active":"2023-11-24T07:59:53.608579+0000","last_peered":"2023-11-24T07:59:53.608579+0000","last_clean":"2023-11-24T07:59:53.608579+0000","last_became_active":"2023-11-24T07:59:52.530357+0000","last_became_peered":"2023-11-24T07:59:52.530357+0000","last_unstale":"2023-11-24T07:59:53.608579+0000","last_undegraded":"2023-11-24T07:59:53.608579+0000","last_fullsized":"2023-11-24T07:59:53.608579+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:59:51.477944+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:59:51.477944+0000","last_clean_scrub_stamp":"2023-11-24T07:59:51.477944+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T17:27:50.274990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.1a","version":"86'1","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-11-24T07:59:53.367044+0000","last_change":"2023-11-24T07:59:51.496684+0000","last_active":"2023-11-24T07:59:53.367044+0000","last_peered":"2023-11-24T07:59:53.367044+0000","last_clean":"2023-11-24T07:59:53.367044+0000","last_became_active":"2023-11-24T07:59:51.496233+0000","last_became_peered":"2023-11-24T07:59:51.496233+0000","last_unstale":"2023-11-24T07:59:53.367044+0000","last_undegraded":"2023-11-24T07:59:53.367044+0000","last_fullsized":"2023-11-24T07:59:53.367044+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:59:50.473699+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:59:50.473699+0000","last_clean_scrub_stamp":"2023-11-24T07:59:50.473699+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T13:17:25.606179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.18","version":"0'0","reported_seq":11,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-11-24T07:59:53.607889+0000","last_change":"2023-11-24T07:59:52.521426+0000","last_active":"2023-11-24T07:59:53.607889+0000","last_peered":"2023-11-24T07:59:53.607889+0000","last_clean":"2023-11-24T07:59:53.607889+0000","last_became_active":"2023-11-24T07:59:52.521064+0000","last_became_peered":"2023-11-24T07:59:52.521064+0000","last_unstale":"2023-11-24T07:59:53.607889+0000","last_undegraded":"2023-11-24T07:59:53.607889+0000","last_fullsized":"2023-11-24T07:59:53.607889+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:59:51.477944+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:59:51.477944+0000","last_clean_scrub_stamp":"2023-11-24T07:59:51.477944+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T08:19:08.702386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.19","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-11-24T07:59:53.370563+0000","last_change":"2023-11-24T07:59:51.502385+0000","last_active":"2023-11-24T07:59:53.370563+0000","last_peered":"2023-11-24T07:59:53.370563+0000","last_clean":"2023-11-24T07:59:53.370563+0000","last_became_active":"2023-11-24T07:59:51.502112+0000","last_became_peered":"2023-11-24T07:59:51.502112+0000","last_unstale":"2023-11-24T07:59:53.370563+0000","last_undegraded":"2023-11-24T07:59:53.370563+0000","last_fullsized":"2023-11-24T07:59:53.370563+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:59:50.473699+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:59:50.473699+0000","last_clean_scrub_stamp":"2023-11-24T07:59:50.473699+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T18:23:25.376630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.19","version":"0'0","reported_seq":11,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-11-24T07:59:53.507468+0000","last_change":"2023-11-24T07:59:52.524304+0000","last_active":"2023-11-24T07:59:53.507468+0000","last_peered":"2023-11-24T07:59:53.507468+0000","last_clean":"2023-11-24T07:59:53.507468+0000","last_became_active":"2023-11-24T07:59:52.524057+0000","last_became_peered":"2023-11-24T07:59:52.524057+0000","last_unstale":"2023-11-24T07:59:53.507468+0000","last_undegraded":"2023-11-24T07:59:53.507468+0000","last_fullsized":"2023-11-24T07:59:53.507468+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:59:51.477944+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:59:51.477944+0000","last_clean_scrub_stamp":"2023-11-24T07:59:51.477944+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T15:30:01.455976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.18","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-11-24T07:59:53.370527+0000","last_change":"2023-11-24T07:59:51.495208+0000","last_active":"2023-11-24T07:59:53.370527+0000","last_peered":"2023-11-24T07:59:53.370527+0000","last_clean":"2023-11-24T07:59:53.370527+0000","last_became_active":"2023-11-24T07:59:51.494958+0000","last_became_peered":"2023-11-24T07:59:51.494958+0000","last_unstale":"2023-11-24T07:59:53.370527+0000","last_undegraded":"2023-11-24T07:59:53.370527+0000","last_fullsized":"2023-11-24T07:59:53.370527+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:59:50.473699+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:59:50.473699+0000","last_clean_scrub_stamp":"2023-11-24T07:59:50.473699+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T13:00:50.971622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.16","version":"0'0","reported_seq":11,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-11-24T07:59:53.505638+0000","last_change":"2023-11-24T07:59:52.527538+0000","last_active":"2023-11-24T07:59:53.505638+0000","last_peered":"2023-11-24T07:59:53.505638+0000","last_clean":"2023-11-24T07:59:53.505638+0000","last_became_active":"2023-11-24T07:59:52.527111+0000","last_became_peered":"2023-11-24T07:59:52.527111+0000","last_unstale":"2023-11-24T07:59:53.505638+0000","last_undegraded":"2023-11-24T07:59:53.505638+0000","last_fullsized":"2023-11-24T07:59:53.505638+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:59:51.477944+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:59:51.477944+0000","last_clean_scrub_stamp":"2023-11-24T07:59:51.477944+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T11:02:28.096472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.17","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-11-24T07:59:53.377035+0000","last_change":"2023-11-24T07:59:51.500327+0000","last_active":"2023-11-24T07:59:53.377035+0000","last_peered":"2023-11-24T07:59:53.377035+0000","last_clean":"2023-11-24T07:59:53.377035+0000","last_became_active":"2023-11-24T07:59:51.500042+0000","last_became_peered":"2023-11-24T07:59:51.500042+0000","last_unstale":"2023-11-24T07:59:53.377035+0000","last_undegraded":"2023-11-24T07:59:53.377035+0000","last_fullsized":"2023-11-24T07:59:53.377035+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:59:50.473699+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:59:50.473699+0000","last_clean_scrub_stamp":"2023-11-24T07:59:50.473699+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T15:52:10.012815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.17","version":"0'0","reported_seq":11,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-11-24T07:59:53.609101+0000","last_change":"2023-11-24T07:59:52.519325+0000","last_active":"2023-11-24T07:59:53.609101+0000","last_peered":"2023-11-24T07:59:53.609101+0000","last_clean":"2023-11-24T07:59:53.609101+0000","last_became_active":"2023-11-24T07:59:52.519012+0000","last_became_peered":"2023-11-24T07:59:52.519012+0000","last_unstale":"2023-11-24T07:59:53.609101+0000","last_undegraded":"2023-11-24T07:59:53.609101+0000","last_fullsized":"2023-11-24T07:59:53.609101+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:59:51.477944+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:59:51.477944+0000","last_clean_scrub_stamp":"2023-11-24T07:59:51.477944+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:25:11.323450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.16","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-11-24T07:59:53.367176+0000","last_change":"2023-11-24T07:59:51.492433+0000","last_active":"2023-11-24T07:59:53.367176+0000","last_peered":"2023-11-24T07:59:53.367176+0000","last_clean":"2023-11-24T07:59:53.367176+0000","last_became_active":"2023-11-24T07:59:51.492145+0000","last_became_peered":"2023-11-24T07:59:51.492145+0000","last_unstale":"2023-11-24T07:59:53.367176+0000","last_undegraded":"2023-11-24T07:59:53.367176+0000","last_fullsized":"2023-11-24T07:59:53.367176+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:59:50.473699+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:59:50.473699+0000","last_clean_scrub_stamp":"2023-11-24T07:59:50.473699+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T13:17:19.649925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.14","version":"0'0","reported_seq":11,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-11-24T07:59:53.608453+0000","last_change":"2023-11-24T07:59:52.526145+0000","last_active":"2023-11-24T07:59:53.608453+0000","last_peered":"2023-11-24T07:59:53.608453+0000","last_clean":"2023-11-24T07:59:53.608453+0000","last_became_active":"2023-11-24T07:59:52.525892+0000","last_became_peered":"2023-11-24T07:59:52.525892+0000","last_unstale":"2023-11-24T07:59:53.608453+0000","last_undegraded":"2023-11-24T07:59:53.608453+0000","last_fullsized":"2023-11-24T07:59:53.608453+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:59:51.477944+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:59:51.477944+0000","last_clean_scrub_stamp":"2023-11-24T07:59:51.477944+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T16:26:49.386477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.15","version":"86'3","reported_seq":15,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-11-24T07:59:53.376936+0000","last_change":"2023-11-24T07:59:51.502200+0000","last_active":"2023-11-24T07:59:53.376936+0000","last_peered":"2023-11-24T07:59:53.376936+0000","last_clean":"2023-11-24T07:59:53.376936+0000","last_became_active":"2023-11-24T07:59:51.501924+0000","last_became_peered":"2023-11-24T07:59:51.501924+0000","last_unstale":"2023-11-24T07:59:53.376936+0000","last_undegraded":"2023-11-24T07:59:53.376936+0000","last_fullsized":"2023-11-24T07:59:53.376936+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:59:50.473699+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:59:50.473699+0000","last_clean_scrub_stamp":"2023-11-24T07:59:50.473699+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T11:02:27.092227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.15","version":"0'0","reported_seq":11,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-11-24T07:59:53.506323+0000","last_change":"2023-11-24T07:59:52.514787+0000","last_active":"2023-11-24T07:59:53.506323+0000","last_peered":"2023-11-24T07:59:53.506323+0000","last_clean":"2023-11-24T07:59:53.506323+0000","last_became_active":"2023-11-24T07:59:52.514493+0000","last_became_peered":"2023-11-24T07:59:52.514493+0000","last_unstale":"2023-11-24T07:59:53.506323+0000","last_undegraded":"2023-11-24T07:59:53.506323+0000","last_fullsized":"2023-11-24T07:59:53.506323+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:59:51.477944+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:59:51.477944+0000","last_clean_scrub_stamp":"2023-11-24T07:59:51.477944+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T08:34:28.451170+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.14","version":"86'6","reported_seq":18,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-11-24T07:59:53.367026+0000","last_change":"2023-11-24T07:59:51.496195+0000","last_active":"2023-11-24T07:59:53.367026+0000","last_peered":"2023-11-24T07:59:53.367026+0000","last_clean":"2023-11-24T07:59:53.367026+0000","last_became_active":"2023-11-24T07:59:51.495930+0000","last_became_peered":"2023-11-24T07:59:51.495930+0000","last_unstale":"2023-11-24T07:59:53.367026+0000","last_undegraded":"2023-11-24T07:59:53.367026+0000","last_fullsized":"2023-11-24T07:59:53.367026+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:59:50.473699+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:59:50.473699+0000","last_clean_scrub_stamp":"2023-11-24T07:59:50.473699+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:09:45.168881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.12","version":"0'0","reported_seq":11,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-11-24T07:59:53.608994+0000","last_change":"2023-11-24T07:59:52.522452+0000","last_active":"2023-11-24T07:59:53.608994+0000","last_peered":"2023-11-24T07:59:53.608994+0000","last_clean":"2023-11-24T07:59:53.608994+0000","last_became_active":"2023-11-24T07:59:52.522173+0000","last_became_peered":"2023-11-24T07:59:52.522173+0000","last_unstale":"2023-11-24T07:59:53.608994+0000","last_undegraded":"2023-11-24T07:59:53.608994+0000","last_fullsized":"2023-11-24T07:59:53.608994+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:59:51.477944+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:59:51.477944+0000","last_clean_scrub_stamp":"2023-11-24T07:59:51.477944+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T15:17:29.156978+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.13","version":"86'2","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-11-24T07:59:53.374661+0000","last_change":"2023-11-24T07:59:51.499029+0000","last_active":"2023-11-24T07:59:53.374661+0000","last_peered":"2023-11-24T07:59:53.374661+0000","last_clean":"2023-11-24T07:59:53.374661+0000","last_became_active":"2023-11-24T07:59:51.498746+0000","last_became_peered":"2023-11-24T07:59:51.498746+0000","last_unstale":"2023-11-24T07:59:53.374661+0000","last_undegraded":"2023-11-24T07:59:53.374661+0000","last_fullsized":"2023-11-24T07:59:53.374661+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:59:50.473699+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:59:50.473699+0000","last_clean_scrub_stamp":"2023-11-24T07:59:50.473699+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T11:36:02.086046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.13","version":"0'0","reported_seq":11,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-11-24T07:59:53.608312+0000","last_change":"2023-11-24T07:59:52.519427+0000","last_active":"2023-11-24T07:59:53.608312+0000","last_peered":"2023-11-24T07:59:53.608312+0000","last_clean":"2023-11-24T07:59:53.608312+0000","last_became_active":"2023-11-24T07:59:52.518944+0000","last_became_peered":"2023-11-24T07:59:52.518944+0000","last_unstale":"2023-11-24T07:59:53.608312+0000","last_undegraded":"2023-11-24T07:59:53.608312+0000","last_fullsized":"2023-11-24T07:59:53.608312+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:59:51.477944+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:59:51.477944+0000","last_clean_scrub_stamp":"2023-11-24T07:59:51.477944+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T15:37:22.418541+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.12","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-11-24T07:59:53.373001+0000","last_change":"2023-11-24T07:59:51.503076+0000","last_active":"2023-11-24T07:59:53.373001+0000","last_peered":"2023-11-24T07:59:53.373001+0000","last_clean":"2023-11-24T07:59:53.373001+0000","last_became_active":"2023-11-24T07:59:51.502784+0000","last_became_peered":"2023-11-24T07:59:51.502784+0000","last_unstale":"2023-11-24T07:59:53.373001+0000","last_undegraded":"2023-11-24T07:59:53.373001+0000","last_fullsized":"2023-11-24T07:59:53.373001+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:59:50.473699+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:59:50.473699+0000","last_clean_scrub_stamp":"2023-11-24T07:59:50.473699+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T08:19:07.698141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.10","version":"0'0","reported_seq":11,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-11-24T07:59:53.505332+0000","last_change":"2023-11-24T07:59:52.518212+0000","last_active":"2023-11-24T07:59:53.505332+0000","last_peered":"2023-11-24T07:59:53.505332+0000","last_clean":"2023-11-24T07:59:53.505332+0000","last_became_active":"2023-11-24T07:59:52.517869+0000","last_became_peered":"2023-11-24T07:59:52.517869+0000","last_unstale":"2023-11-24T07:59:53.505332+0000","last_undegraded":"2023-11-24T07:59:53.505332+0000","last_fullsized":"2023-11-24T07:59:53.505332+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:59:51.477944+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:59:51.477944+0000","last_clean_scrub_stamp":"2023-11-24T07:59:51.477944+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T08:13:59.445242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.11","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-11-24T07:59:53.366997+0000","last_change":"2023-11-24T07:59:51.501760+0000","last_active":"2023-11-24T07:59:53.366997+0000","last_peered":"2023-11-24T07:59:53.366997+0000","last_clean":"2023-11-24T07:59:53.366997+0000","last_became_active":"2023-11-24T07:59:51.501505+0000","last_became_peered":"2023-11-24T07:59:51.501505+0000","last_unstale":"2023-11-24T07:59:53.366997+0000","last_undegraded":"2023-11-24T07:59:53.366997+0000","last_fullsized":"2023-11-24T07:59:53.366997+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:59:50.473699+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:59:50.473699+0000","last_clean_scrub_stamp":"2023-11-24T07:59:50.473699+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T15:17:28.152732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.11","version":"0'0","reported_seq":11,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-11-24T07:59:53.608423+0000","last_change":"2023-11-24T07:59:52.515933+0000","last_active":"2023-11-24T07:59:53.608423+0000","last_peered":"2023-11-24T07:59:53.608423+0000","last_clean":"2023-11-24T07:59:53.608423+0000","last_became_active":"2023-11-24T07:59:52.513820+0000","last_became_peered":"2023-11-24T07:59:52.513820+0000","last_unstale":"2023-11-24T07:59:53.608423+0000","last_undegraded":"2023-11-24T07:59:53.608423+0000","last_fullsized":"2023-11-24T07:59:53.608423+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:59:51.477944+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:59:51.477944+0000","last_clean_scrub_stamp":"2023-11-24T07:59:51.477944+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T11:25:48.888553+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.10","version":"86'8","reported_seq":20,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-11-24T07:59:53.372959+0000","last_change":"2023-11-24T07:59:51.503492+0000","last_active":"2023-11-24T07:59:53.372959+0000","last_peered":"2023-11-24T07:59:53.372959+0000","last_clean":"2023-11-24T07:59:53.372959+0000","last_became_active":"2023-11-24T07:59:51.502903+0000","last_became_peered":"2023-11-24T07:59:51.502903+0000","last_unstale":"2023-11-24T07:59:53.372959+0000","last_undegraded":"2023-11-24T07:59:53.372959+0000","last_fullsized":"2023-11-24T07:59:53.372959+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:59:50.473699+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:59:50.473699+0000","last_clean_scrub_stamp":"2023-11-24T07:59:50.473699+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T16:38:42.596281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.e","version":"0'0","reported_seq":11,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-11-24T07:59:53.609794+0000","last_change":"2023-11-24T07:59:52.521956+0000","last_active":"2023-11-24T07:59:53.609794+0000","last_peered":"2023-11-24T07:59:53.609794+0000","last_clean":"2023-11-24T07:59:53.609794+0000","last_became_active":"2023-11-24T07:59:52.521664+0000","last_became_peered":"2023-11-24T07:59:52.521664+0000","last_unstale":"2023-11-24T07:59:53.609794+0000","last_undegraded":"2023-11-24T07:59:53.609794+0000","last_fullsized":"2023-11-24T07:59:53.609794+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:59:51.477944+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:59:51.477944+0000","last_clean_scrub_stamp":"2023-11-24T07:59:51.477944+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T16:20:03.228953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.f","version":"86'1","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-11-24T07:59:53.373414+0000","last_change":"2023-11-24T07:59:51.503493+0000","last_active":"2023-11-24T07:59:53.373414+0000","last_peered":"2023-11-24T07:59:53.373414+0000","last_clean":"2023-11-24T07:59:53.373414+0000","last_became_active":"2023-11-24T07:59:51.502675+0000","last_became_peered":"2023-11-24T07:59:51.502675+0000","last_unstale":"2023-11-24T07:59:53.373414+0000","last_undegraded":"2023-11-24T07:59:53.373414+0000","last_fullsized":"2023-11-24T07:59:53.373414+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:59:50.473699+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:59:50.473699+0000","last_clean_scrub_stamp":"2023-11-24T07:59:50.473699+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T17:47:41.894391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.f","version":"0'0","reported_seq":11,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-11-24T07:59:53.608741+0000","last_change":"2023-11-24T07:59:52.532100+0000","last_active":"2023-11-24T07:59:53.608741+0000","last_peered":"2023-11-24T07:59:53.608741+0000","last_clean":"2023-11-24T07:59:53.608741+0000","last_became_active":"2023-11-24T07:59:52.531613+0000","last_became_peered":"2023-11-24T07:59:52.531613+0000","last_unstale":"2023-11-24T07:59:53.608741+0000","last_undegraded":"2023-11-24T07:59:53.608741+0000","last_fullsized":"2023-11-24T07:59:53.608741+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:59:51.477944+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:59:51.477944+0000","last_clean_scrub_stamp":"2023-11-24T07:59:51.477944+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T16:39:20.453395+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.e","version":"86'2","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-11-24T07:59:53.370697+0000","last_change":"2023-11-24T07:59:51.505119+0000","last_active":"2023-11-24T07:59:53.370697+0000","last_peered":"2023-11-24T07:59:53.370697+0000","last_clean":"2023-11-24T07:59:53.370697+0000","last_became_active":"2023-11-24T07:59:51.504803+0000","last_became_peered":"2023-11-24T07:59:51.504803+0000","last_unstale":"2023-11-24T07:59:53.370697+0000","last_undegraded":"2023-11-24T07:59:53.370697+0000","last_fullsized":"2023-11-24T07:59:53.370697+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:59:50.473699+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:59:50.473699+0000","last_clean_scrub_stamp":"2023-11-24T07:59:50.473699+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T08:47:02.210906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.d","version":"0'0","reported_seq":11,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-11-24T07:59:53.609661+0000","last_change":"2023-11-24T07:59:52.525088+0000","last_active":"2023-11-24T07:59:53.609661+0000","last_peered":"2023-11-24T07:59:53.609661+0000","last_clean":"2023-11-24T07:59:53.609661+0000","last_became_active":"2023-11-24T07:59:52.524659+0000","last_became_peered":"2023-11-24T07:59:52.524659+0000","last_unstale":"2023-11-24T07:59:53.609661+0000","last_undegraded":"2023-11-24T07:59:53.609661+0000","last_fullsized":"2023-11-24T07:59:53.609661+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:59:51.477944+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:59:51.477944+0000","last_clean_scrub_stamp":"2023-11-24T07:59:51.477944+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T19:33:56.535985+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.c","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-11-24T07:59:53.370659+0000","last_change":"2023-11-24T07:59:51.498787+0000","last_active":"2023-11-24T07:59:53.370659+0000","last_peered":"2023-11-24T07:59:53.370659+0000","last_clean":"2023-11-24T07:59:53.370659+0000","last_became_active":"2023-11-24T07:59:51.498491+0000","last_became_peered":"2023-11-24T07:59:51.498491+0000","last_unstale":"2023-11-24T07:59:53.370659+0000","last_undegraded":"2023-11-24T07:59:53.370659+0000","last_fullsized":"2023-11-24T07:59:53.370659+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:59:50.473699+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:59:50.473699+0000","last_clean_scrub_stamp":"2023-11-24T07:59:50.473699+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T09:44:29.100278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.a","version":"0'0","reported_seq":11,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-11-24T07:59:53.609246+0000","last_change":"2023-11-24T07:59:52.514611+0000","last_active":"2023-11-24T07:59:53.609246+0000","last_peered":"2023-11-24T07:59:53.609246+0000","last_clean":"2023-11-24T07:59:53.609246+0000","last_became_active":"2023-11-24T07:59:52.514320+0000","last_became_peered":"2023-11-24T07:59:52.514320+0000","last_unstale":"2023-11-24T07:59:53.609246+0000","last_undegraded":"2023-11-24T07:59:53.609246+0000","last_fullsized":"2023-11-24T07:59:53.609246+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:59:51.477944+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:59:51.477944+0000","last_clean_scrub_stamp":"2023-11-24T07:59:51.477944+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T15:14:48.860422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.b","version":"86'2","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-11-24T07:59:53.370611+0000","last_change":"2023-11-24T07:59:51.497956+0000","last_active":"2023-11-24T07:59:53.370611+0000","last_peered":"2023-11-24T07:59:53.370611+0000","last_clean":"2023-11-24T07:59:53.370611+0000","last_became_active":"2023-11-24T07:59:51.497650+0000","last_became_peered":"2023-11-24T07:59:51.497650+0000","last_unstale":"2023-11-24T07:59:53.370611+0000","last_undegraded":"2023-11-24T07:59:53.370611+0000","last_fullsized":"2023-11-24T07:59:53.370611+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:59:50.473699+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:59:50.473699+0000","last_clean_scrub_stamp":"2023-11-24T07:59:50.473699+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T16:41:39.250342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.b","version":"0'0","reported_seq":11,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-11-24T07:59:53.506139+0000","last_change":"2023-11-24T07:59:52.517321+0000","last_active":"2023-11-24T07:59:53.506139+0000","last_peered":"2023-11-24T07:59:53.506139+0000","last_clean":"2023-11-24T07:59:53.506139+0000","last_became_active":"2023-11-24T07:59:52.517019+0000","last_became_peered":"2023-11-24T07:59:52.517019+0000","last_unstale":"2023-11-24T07:59:53.506139+0000","last_undegraded":"2023-11-24T07:59:53.506139+0000","last_fullsized":"2023-11-24T07:59:53.506139+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:59:51.477944+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:59:51.477944+0000","last_clean_scrub_stamp":"2023-11-24T07:59:51.477944+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T09:23:54.510983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.a","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-11-24T07:59:53.373019+0000","last_change":"2023-11-24T07:59:51.498910+0000","last_active":"2023-11-24T07:59:53.373019+0000","last_peered":"2023-11-24T07:59:53.373019+0000","last_clean":"2023-11-24T07:59:53.373019+0000","last_became_active":"2023-11-24T07:59:51.498576+0000","last_became_peered":"2023-11-24T07:59:51.498576+0000","last_unstale":"2023-11-24T07:59:53.373019+0000","last_undegraded":"2023-11-24T07:59:53.373019+0000","last_fullsized":"2023-11-24T07:59:53.373019+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:59:50.473699+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:59:50.473699+0000","last_clean_scrub_stamp":"2023-11-24T07:59:50.473699+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T15:30:00.451730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.8","version":"0'0","reported_seq":11,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-11-24T07:59:53.506366+0000","last_change":"2023-11-24T07:59:52.518390+0000","last_active":"2023-11-24T07:59:53.506366+0000","last_peered":"2023-11-24T07:59:53.506366+0000","last_clean":"2023-11-24T07:59:53.506366+0000","last_became_active":"2023-11-24T07:59:52.518123+0000","last_became_peered":"2023-11-24T07:59:52.518123+0000","last_unstale":"2023-11-24T07:59:53.506366+0000","last_undegraded":"2023-11-24T07:59:53.506366+0000","last_fullsized":"2023-11-24T07:59:53.506366+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:59:51.477944+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:59:51.477944+0000","last_clean_scrub_stamp":"2023-11-24T07:59:51.477944+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T14:01:22.935817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.9","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-11-24T07:59:53.373001+0000","last_change":"2023-11-24T07:59:51.493885+0000","last_active":"2023-11-24T07:59:53.373001+0000","last_peered":"2023-11-24T07:59:53.373001+0000","last_clean":"2023-11-24T07:59:53.373001+0000","last_became_active":"2023-11-24T07:59:51.493587+0000","last_became_peered":"2023-11-24T07:59:51.493587+0000","last_unstale":"2023-11-24T07:59:53.373001+0000","last_undegraded":"2023-11-24T07:59:53.373001+0000","last_fullsized":"2023-11-24T07:59:53.373001+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:59:50.473699+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:59:50.473699+0000","last_clean_scrub_stamp":"2023-11-24T07:59:50.473699+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T09:59:01.188427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.9","version":"0'0","reported_seq":11,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-11-24T07:59:53.609478+0000","last_change":"2023-11-24T07:59:52.517894+0000","last_active":"2023-11-24T07:59:53.609478+0000","last_peered":"2023-11-24T07:59:53.609478+0000","last_clean":"2023-11-24T07:59:53.609478+0000","last_became_active":"2023-11-24T07:59:52.517590+0000","last_became_peered":"2023-11-24T07:59:52.517590+0000","last_unstale":"2023-11-24T07:59:53.609478+0000","last_undegraded":"2023-11-24T07:59:53.609478+0000","last_fullsized":"2023-11-24T07:59:53.609478+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:59:51.477944+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:59:51.477944+0000","last_clean_scrub_stamp":"2023-11-24T07:59:51.477944+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T13:17:20.654171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.8","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-11-24T07:59:53.370542+0000","last_change":"2023-11-24T07:59:51.501348+0000","last_active":"2023-11-24T07:59:53.370542+0000","last_peered":"2023-11-24T07:59:53.370542+0000","last_clean":"2023-11-24T07:59:53.370542+0000","last_became_active":"2023-11-24T07:59:51.501049+0000","last_became_peered":"2023-11-24T07:59:51.501049+0000","last_unstale":"2023-11-24T07:59:53.370542+0000","last_undegraded":"2023-11-24T07:59:53.370542+0000","last_fullsized":"2023-11-24T07:59:53.370542+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:59:50.473699+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:59:50.473699+0000","last_clean_scrub_stamp":"2023-11-24T07:59:50.473699+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T18:50:30.244220+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.6","version":"0'0","reported_seq":11,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-11-24T07:59:53.506232+0000","last_change":"2023-11-24T07:59:52.520831+0000","last_active":"2023-11-24T07:59:53.506232+0000","last_peered":"2023-11-24T07:59:53.506232+0000","last_clean":"2023-11-24T07:59:53.506232+0000","last_became_active":"2023-11-24T07:59:52.520157+0000","last_became_peered":"2023-11-24T07:59:52.520157+0000","last_unstale":"2023-11-24T07:59:53.506232+0000","last_undegraded":"2023-11-24T07:59:53.506232+0000","last_fullsized":"2023-11-24T07:59:53.506232+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:59:51.477944+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:59:51.477944+0000","last_clean_scrub_stamp":"2023-11-24T07:59:51.477944+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T15:32:44.719383+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.7","version":"86'2","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-11-24T07:59:53.370712+0000","last_change":"2023-11-24T07:59:51.492490+0000","last_active":"2023-11-24T07:59:53.370712+0000","last_peered":"2023-11-24T07:59:53.370712+0000","last_clean":"2023-11-24T07:59:53.370712+0000","last_became_active":"2023-11-24T07:59:51.491865+0000","last_became_peered":"2023-11-24T07:59:51.491865+0000","last_unstale":"2023-11-24T07:59:53.370712+0000","last_undegraded":"2023-11-24T07:59:53.370712+0000","last_fullsized":"2023-11-24T07:59:53.370712+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:59:50.473699+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:59:50.473699+0000","last_clean_scrub_stamp":"2023-11-24T07:59:50.473699+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T08:40:15.071070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.7","version":"0'0","reported_seq":11,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-11-24T07:59:53.506478+0000","last_change":"2023-11-24T07:59:52.529281+0000","last_active":"2023-11-24T07:59:53.506478+0000","last_peered":"2023-11-24T07:59:53.506478+0000","last_clean":"2023-11-24T07:59:53.506478+0000","last_became_active":"2023-11-24T07:59:52.529002+0000","last_became_peered":"2023-11-24T07:59:52.529002+0000","last_unstale":"2023-11-24T07:59:53.506478+0000","last_undegraded":"2023-11-24T07:59:53.506478+0000","last_fullsized":"2023-11-24T07:59:53.506478+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:59:51.477944+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:59:51.477944+0000","last_clean_scrub_stamp":"2023-11-24T07:59:51.477944+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:50:14.772487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.6","version":"87'1","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-11-24T07:59:53.609109+0000","last_change":"2023-11-24T07:59:51.497940+0000","last_active":"2023-11-24T07:59:53.609109+0000","last_peered":"2023-11-24T07:59:53.609109+0000","last_clean":"2023-11-24T07:59:53.609109+0000","last_became_active":"2023-11-24T07:59:51.497679+0000","last_became_peered":"2023-11-24T07:59:51.497679+0000","last_unstale":"2023-11-24T07:59:53.609109+0000","last_undegraded":"2023-11-24T07:59:53.609109+0000","last_fullsized":"2023-11-24T07:59:53.609109+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:59:50.473699+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:59:50.473699+0000","last_clean_scrub_stamp":"2023-11-24T07:59:50.473699+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T08:25:06.417257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.4","version":"0'0","reported_seq":11,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-11-24T07:59:53.506233+0000","last_change":"2023-11-24T07:59:52.518391+0000","last_active":"2023-11-24T07:59:53.506233+0000","last_peered":"2023-11-24T07:59:53.506233+0000","last_clean":"2023-11-24T07:59:53.506233+0000","last_became_active":"2023-11-24T07:59:52.518101+0000","last_became_peered":"2023-11-24T07:59:52.518101+0000","last_unstale":"2023-11-24T07:59:53.506233+0000","last_undegraded":"2023-11-24T07:59:53.506233+0000","last_fullsized":"2023-11-24T07:59:53.506233+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:59:51.477944+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:59:51.477944+0000","last_clean_scrub_stamp":"2023-11-24T07:59:51.477944+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T19:52:10.399576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.5","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-11-24T07:59:53.367055+0000","last_change":"2023-11-24T07:59:51.494987+0000","last_active":"2023-11-24T07:59:53.367055+0000","last_peered":"2023-11-24T07:59:53.367055+0000","last_clean":"2023-11-24T07:59:53.367055+0000","last_became_active":"2023-11-24T07:59:51.494704+0000","last_became_peered":"2023-11-24T07:59:51.494704+0000","last_unstale":"2023-11-24T07:59:53.367055+0000","last_undegraded":"2023-11-24T07:59:53.367055+0000","last_fullsized":"2023-11-24T07:59:53.367055+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:59:50.473699+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:59:50.473699+0000","last_clean_scrub_stamp":"2023-11-24T07:59:50.473699+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T19:22:20.614507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.5","version":"0'0","reported_seq":11,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-11-24T07:59:53.608903+0000","last_change":"2023-11-24T07:59:52.532040+0000","last_active":"2023-11-24T07:59:53.608903+0000","last_peered":"2023-11-24T07:59:53.608903+0000","last_clean":"2023-11-24T07:59:53.608903+0000","last_became_active":"2023-11-24T07:59:52.531715+0000","last_became_peered":"2023-11-24T07:59:52.531715+0000","last_unstale":"2023-11-24T07:59:53.608903+0000","last_undegraded":"2023-11-24T07:59:53.608903+0000","last_fullsized":"2023-11-24T07:59:53.608903+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:59:51.477944+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:59:51.477944+0000","last_clean_scrub_stamp":"2023-11-24T07:59:51.477944+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T08:09:51.912125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.4","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-11-24T07:59:53.367112+0000","last_change":"2023-11-24T07:59:51.498705+0000","last_active":"2023-11-24T07:59:53.367112+0000","last_peered":"2023-11-24T07:59:53.367112+0000","last_clean":"2023-11-24T07:59:53.367112+0000","last_became_active":"2023-11-24T07:59:51.498398+0000","last_became_peered":"2023-11-24T07:59:51.498398+0000","last_unstale":"2023-11-24T07:59:53.367112+0000","last_undegraded":"2023-11-24T07:59:53.367112+0000","last_fullsized":"2023-11-24T07:59:53.367112+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:59:50.473699+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:59:50.473699+0000","last_clean_scrub_stamp":"2023-11-24T07:59:50.473699+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T19:22:20.614507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.2","version":"0'0","reported_seq":11,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-11-24T07:59:53.608160+0000","last_change":"2023-11-24T07:59:52.532307+0000","last_active":"2023-11-24T07:59:53.608160+0000","last_peered":"2023-11-24T07:59:53.608160+0000","last_clean":"2023-11-24T07:59:53.608160+0000","last_became_active":"2023-11-24T07:59:52.531978+0000","last_became_peered":"2023-11-24T07:59:52.531978+0000","last_unstale":"2023-11-24T07:59:53.608160+0000","last_undegraded":"2023-11-24T07:59:53.608160+0000","last_fullsized":"2023-11-24T07:59:53.608160+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:59:51.477944+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:59:51.477944+0000","last_clean_scrub_stamp":"2023-11-24T07:59:51.477944+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:15:45.686063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.3","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-11-24T07:59:53.373039+0000","last_change":"2023-11-24T07:59:51.502368+0000","last_active":"2023-11-24T07:59:53.373039+0000","last_peered":"2023-11-24T07:59:53.373039+0000","last_clean":"2023-11-24T07:59:53.373039+0000","last_became_active":"2023-11-24T07:59:51.501719+0000","last_became_peered":"2023-11-24T07:59:51.501719+0000","last_unstale":"2023-11-24T07:59:53.373039+0000","last_undegraded":"2023-11-24T07:59:53.373039+0000","last_fullsized":"2023-11-24T07:59:53.373039+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:59:50.473699+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:59:50.473699+0000","last_clean_scrub_stamp":"2023-11-24T07:59:50.473699+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:15:44.681817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.3","version":"0'0","reported_seq":11,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-11-24T07:59:53.608764+0000","last_change":"2023-11-24T07:59:52.531520+0000","last_active":"2023-11-24T07:59:53.608764+0000","last_peered":"2023-11-24T07:59:53.608764+0000","last_clean":"2023-11-24T07:59:53.608764+0000","last_became_active":"2023-11-24T07:59:52.531120+0000","last_became_peered":"2023-11-24T07:59:52.531120+0000","last_unstale":"2023-11-24T07:59:53.608764+0000","last_undegraded":"2023-11-24T07:59:53.608764+0000","last_fullsized":"2023-11-24T07:59:53.608764+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:59:51.477944+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:59:51.477944+0000","last_clean_scrub_stamp":"2023-11-24T07:59:51.477944+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T11:56:37.637066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.2","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-11-24T07:59:53.367071+0000","last_change":"2023-11-24T07:59:51.496507+0000","last_active":"2023-11-24T07:59:53.367071+0000","last_peered":"2023-11-24T07:59:53.367071+0000","last_clean":"2023-11-24T07:59:53.367071+0000","last_became_active":"2023-11-24T07:59:51.495858+0000","last_became_peered":"2023-11-24T07:59:51.495858+0000","last_unstale":"2023-11-24T07:59:53.367071+0000","last_undegraded":"2023-11-24T07:59:53.367071+0000","last_fullsized":"2023-11-24T07:59:53.367071+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:59:50.473699+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:59:50.473699+0000","last_clean_scrub_stamp":"2023-11-24T07:59:50.473699+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T15:05:21.923740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.0","version":"0'0","reported_seq":11,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-11-24T07:59:53.608044+0000","last_change":"2023-11-24T07:59:52.538042+0000","last_active":"2023-11-24T07:59:53.608044+0000","last_peered":"2023-11-24T07:59:53.608044+0000","last_clean":"2023-11-24T07:59:53.608044+0000","last_became_active":"2023-11-24T07:59:52.537692+0000","last_became_peered":"2023-11-24T07:59:52.537692+0000","last_unstale":"2023-11-24T07:59:53.608044+0000","last_undegraded":"2023-11-24T07:59:53.608044+0000","last_fullsized":"2023-11-24T07:59:53.608044+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:59:51.477944+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:59:51.477944+0000","last_clean_scrub_stamp":"2023-11-24T07:59:51.477944+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T15:30:01.455976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.1","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-11-24T07:59:53.376829+0000","last_change":"2023-11-24T07:59:51.501509+0000","last_active":"2023-11-24T07:59:53.376829+0000","last_peered":"2023-11-24T07:59:53.376829+0000","last_clean":"2023-11-24T07:59:53.376829+0000","last_became_active":"2023-11-24T07:59:51.501201+0000","last_became_peered":"2023-11-24T07:59:51.501201+0000","last_unstale":"2023-11-24T07:59:53.376829+0000","last_undegraded":"2023-11-24T07:59:53.376829+0000","last_fullsized":"2023-11-24T07:59:53.376829+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:59:50.473699+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:59:50.473699+0000","last_clean_scrub_stamp":"2023-11-24T07:59:50.473699+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T14:16:26.355513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.1","version":"0'0","reported_seq":11,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-11-24T07:59:53.608183+0000","last_change":"2023-11-24T07:59:52.521402+0000","last_active":"2023-11-24T07:59:53.608183+0000","last_peered":"2023-11-24T07:59:53.608183+0000","last_clean":"2023-11-24T07:59:53.608183+0000","last_became_active":"2023-11-24T07:59:52.520843+0000","last_became_peered":"2023-11-24T07:59:52.520843+0000","last_unstale":"2023-11-24T07:59:53.608183+0000","last_undegraded":"2023-11-24T07:59:53.608183+0000","last_fullsized":"2023-11-24T07:59:53.608183+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:59:51.477944+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:59:51.477944+0000","last_clean_scrub_stamp":"2023-11-24T07:59:51.477944+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T09:59:02.192672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.0","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-11-24T07:59:53.366951+0000","last_change":"2023-11-24T07:59:51.500716+0000","last_active":"2023-11-24T07:59:53.366951+0000","last_peered":"2023-11-24T07:59:53.366951+0000","last_clean":"2023-11-24T07:59:53.366951+0000","last_became_active":"2023-11-24T07:59:51.500429+0000","last_became_peered":"2023-11-24T07:59:51.500429+0000","last_unstale":"2023-11-24T07:59:53.366951+0000","last_undegraded":"2023-11-24T07:59:53.366951+0000","last_fullsized":"2023-11-24T07:59:53.366951+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:59:50.473699+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:59:50.473699+0000","last_clean_scrub_stamp":"2023-11-24T07:59:50.473699+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T15:05:21.923740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.c","version":"0'0","reported_seq":11,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-11-24T07:59:53.609248+0000","last_change":"2023-11-24T07:59:52.519423+0000","last_active":"2023-11-24T07:59:53.609248+0000","last_peered":"2023-11-24T07:59:53.609248+0000","last_clean":"2023-11-24T07:59:53.609248+0000","last_became_active":"2023-11-24T07:59:52.517567+0000","last_became_peered":"2023-11-24T07:59:52.517567+0000","last_unstale":"2023-11-24T07:59:53.609248+0000","last_undegraded":"2023-11-24T07:59:53.609248+0000","last_fullsized":"2023-11-24T07:59:53.609248+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:59:51.477944+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:59:51.477944+0000","last_clean_scrub_stamp":"2023-11-24T07:59:51.477944+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T09:33:44.224663+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.d","version":"86'1","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-11-24T07:59:53.373305+0000","last_change":"2023-11-24T07:59:51.502027+0000","last_active":"2023-11-24T07:59:53.373305+0000","last_peered":"2023-11-24T07:59:53.373305+0000","last_clean":"2023-11-24T07:59:53.373305+0000","last_became_active":"2023-11-24T07:59:51.501705+0000","last_became_peered":"2023-11-24T07:59:51.501705+0000","last_unstale":"2023-11-24T07:59:53.373305+0000","last_undegraded":"2023-11-24T07:59:53.373305+0000","last_fullsized":"2023-11-24T07:59:53.373305+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:59:50.473699+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:59:50.473699+0000","last_clean_scrub_stamp":"2023-11-24T07:59:50.473699+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T17:10:42.541250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":222,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-11-24T07:59:52.514156+0000","last_change":"2023-11-24T07:56:19.668905+0000","last_active":"2023-11-24T07:59:52.514156+0000","last_peered":"2023-11-24T07:59:52.514156+0000","last_clean":"2023-11-24T07:59:52.514156+0000","last_became_active":"2023-11-24T07:56:19.668601+0000","last_became_peered":"2023-11-24T07:56:19.668601+0000","last_unstale":"2023-11-24T07:59:52.514156+0000","last_undegraded":"2023-11-24T07:59:52.514156+0000","last_fullsized":"2023-11-24T07:59:52.514156+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:56:18.657844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:56:18.657844+0000","last_clean_scrub_stamp":"2023-11-24T07:56:18.657844+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T11:51:53.828455+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":13,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":12,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574896,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28072,"kb_used_data":1048,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343768,"statfs":{"total":96636764160,"available":96608018432,"internally_reserved":0,"allocated":1073152,"data_stored":459185,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 07:59:20 2023","interfaces":[{"interface":"back","average":{"1min":0.51800000000000002,"5min":0.54000000000000004,"15min":0.55100000000000005},"min":{"1min":0.27600000000000002,"5min":0.27600000000000002,"15min":0.27600000000000002},"max":{"1min":0.83699999999999997,"5min":1.413,"15min":1.413},"last":0.40400000000000003},{"interface":"front","average":{"1min":0.53500000000000003,"5min":0.55200000000000005,"15min":0.55300000000000005},"min":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.72999999999999998,"5min":0.81000000000000005,"15min":0.81000000000000005},"last":0.54300000000000004}]},{"osd":1,"last update":"Fri Nov 24 07:59:20 2023","interfaces":[{"interface":"back","average":{"1min":0.51500000000000001,"5min":0.71199999999999997,"15min":0.80300000000000005},"min":{"1min":0.35499999999999998,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.76400000000000001,"5min":6.1070000000000002,"15min":6.1070000000000002},"last":0.51400000000000001},{"interface":"front","average":{"1min":0.48699999999999999,"5min":0.54800000000000004,"15min":0.56899999999999995},"min":{"1min":0.35199999999999998,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.77200000000000002,"5min":1.5920000000000001,"15min":1.5920000000000001},"last":0.56200000000000006}]},{"osd":2,"last update":"Fri Nov 24 07:59:20 2023","interfaces":[{"interface":"back","average":{"1min":0.52700000000000002,"5min":0.58099999999999996,"15min":0.61099999999999999},"min":{"1min":0.308,"5min":0.308,"15min":0.308},"max":{"1min":0.96899999999999997,"5min":1.613,"15min":1.613},"last":0.503},{"interface":"front","average":{"1min":0.48099999999999998,"5min":0.67100000000000004,"15min":0.751},"min":{"1min":0.33400000000000002,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.78000000000000003,"5min":4.5220000000000002,"15min":4.5220000000000002},"last":0.48899999999999999}]},{"osd":3,"last update":"Fri Nov 24 07:59:20 2023","interfaces":[{"interface":"back","average":{"1min":0.56599999999999995,"5min":0.70699999999999996,"15min":0.77300000000000002},"min":{"1min":0.40699999999999997,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.82699999999999996,"5min":4.6760000000000002,"15min":4.6760000000000002},"last":0.60399999999999998},{"interface":"front","average":{"1min":0.60499999999999998,"5min":0.80800000000000005,"15min":0.89800000000000002},"min":{"1min":0.41699999999999998,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.92200000000000004,"5min":4.6539999999999999,"15min":4.6539999999999999},"last":0.59399999999999997}]},{"osd":4,"last update":"Fri Nov 24 07:59:20 2023","interfaces":[{"interface":"back","average":{"1min":0.47199999999999998,"5min":0.54500000000000004,"15min":0.56000000000000005},"min":{"1min":0.32300000000000001,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.95099999999999996,"5min":0.95099999999999996,"15min":0.95099999999999996},"last":0.41999999999999998},{"interface":"front","average":{"1min":0.55500000000000005,"5min":0.62,"15min":0.65100000000000002},"min":{"1min":0.38,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":1.0069999999999999,"5min":1.629,"15min":1.629},"last":0.44800000000000001}]},{"osd":5,"last update":"Fri Nov 24 07:59:20 2023","interfaces":[{"interface":"back","average":{"1min":0.501,"5min":0.53600000000000003,"15min":0.54300000000000004},"min":{"1min":0.315,"5min":0.315,"15min":0.315},"max":{"1min":0.748,"5min":0.875,"15min":0.875},"last":0.38700000000000001},{"interface":"front","average":{"1min":0.56799999999999995,"5min":0.60299999999999998,"15min":0.61599999999999999},"min":{"1min":0.40500000000000003,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.89800000000000002,"5min":0.90900000000000003,"15min":0.90900000000000003},"last":0.47499999999999998}]},{"osd":6,"last update":"Fri Nov 24 07:59:20 2023","interfaces":[{"interface":"back","average":{"1min":0.55200000000000005,"5min":0.59699999999999998,"15min":0.60599999999999998},"min":{"1min":0.33100000000000002,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.85799999999999998,"5min":0.85799999999999998,"15min":0.85799999999999998},"last":0.435},{"interface":"front","average":{"1min":0.61199999999999999,"5min":0.65200000000000002,"15min":0.66400000000000003},"min":{"1min":0.443,"5min":0.443,"15min":0.443},"max":{"1min":0.99099999999999999,"5min":0.99099999999999999,"15min":0.99099999999999999},"last":0.57599999999999996}]}]},{"osd":6,"up_from":13,"seq":55834574897,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28076,"kb_used_data":1052,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343764,"statfs":{"total":96636764160,"available":96608014336,"internally_reserved":0,"allocated":1077248,"data_stored":459787,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 07:59:19 2023","interfaces":[{"interface":"back","average":{"1min":0.51000000000000001,"5min":0.53800000000000003,"15min":0.54600000000000004},"min":{"1min":0.33500000000000002,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":0.82999999999999996,"5min":0.93100000000000005,"15min":0.93100000000000005},"last":0.65300000000000002},{"interface":"front","average":{"1min":0.53100000000000003,"5min":0.51900000000000002,"15min":0.51700000000000002},"min":{"1min":0.34000000000000002,"5min":0.311,"15min":0.311},"max":{"1min":0.72299999999999998,"5min":0.82199999999999995,"15min":0.82199999999999995},"last":0.73799999999999999}]},{"osd":1,"last update":"Fri Nov 24 07:59:19 2023","interfaces":[{"interface":"back","average":{"1min":0.499,"5min":0.51000000000000001,"15min":0.51100000000000001},"min":{"1min":0.312,"5min":0.312,"15min":0.312},"max":{"1min":0.746,"5min":0.81299999999999994,"15min":0.81299999999999994},"last":0.67700000000000005},{"interface":"front","average":{"1min":0.55900000000000005,"5min":0.58699999999999997,"15min":0.60399999999999998},"min":{"1min":0.29999999999999999,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":0.70799999999999996,"5min":1.0069999999999999,"15min":1.0069999999999999},"last":0.68999999999999995}]},{"osd":2,"last update":"Fri Nov 24 07:59:19 2023","interfaces":[{"interface":"back","average":{"1min":0.57699999999999996,"5min":0.57699999999999996,"15min":0.57499999999999996},"min":{"1min":0.41399999999999998,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.71099999999999997,"5min":0.90600000000000003,"15min":0.90600000000000003},"last":0.70499999999999996},{"interface":"front","average":{"1min":0.59899999999999998,"5min":0.58799999999999997,"15min":0.59099999999999997},"min":{"1min":0.52500000000000002,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.78800000000000003,"5min":0.84899999999999998,"15min":0.84899999999999998},"last":0.71699999999999997}]},{"osd":3,"last update":"Fri Nov 24 07:59:19 2023","interfaces":[{"interface":"back","average":{"1min":0.56200000000000006,"5min":0.56499999999999995,"15min":0.56699999999999995},"min":{"1min":0.38700000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.75700000000000001,"5min":0.876,"15min":0.876},"last":0.76900000000000002},{"interface":"front","average":{"1min":0.55700000000000005,"5min":0.56000000000000005,"15min":0.55700000000000005},"min":{"1min":0.40500000000000003,"5min":0.379,"15min":0.379},"max":{"1min":0.72599999999999998,"5min":0.876,"15min":0.876},"last":0.753}]},{"osd":4,"last update":"Fri Nov 24 07:59:19 2023","interfaces":[{"interface":"back","average":{"1min":0.55300000000000005,"5min":0.52700000000000002,"15min":0.52000000000000002},"min":{"1min":0.36299999999999999,"5min":0.25,"15min":0.25},"max":{"1min":0.80700000000000005,"5min":0.89400000000000002,"15min":0.89400000000000002},"last":0.501},{"interface":"front","average":{"1min":0.53100000000000003,"5min":0.58499999999999996,"15min":0.59599999999999997},"min":{"1min":0.30599999999999999,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":0.81899999999999995,"5min":1.087,"15min":1.087},"last":0.59299999999999997}]},{"osd":5,"last update":"Fri Nov 24 07:59:19 2023","interfaces":[{"interface":"back","average":{"1min":0.57099999999999995,"5min":0.55100000000000005,"15min":0.53700000000000003},"min":{"1min":0.33300000000000002,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":0.76200000000000001,"5min":0.97999999999999998,"15min":0.97999999999999998},"last":0.47999999999999998},{"interface":"front","average":{"1min":0.62,"5min":0.61399999999999999,"15min":0.61199999999999999},"min":{"1min":0.438,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.997,"5min":1.0009999999999999,"15min":1.0009999999999999},"last":0.56699999999999995}]},{"osd":7,"last update":"Fri Nov 24 07:59:19 2023","interfaces":[{"interface":"back","average":{"1min":0.59899999999999998,"5min":0.61399999999999999,"15min":0.61599999999999999},"min":{"1min":0.44800000000000001,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":0.85499999999999998,"5min":1.0880000000000001,"15min":1.0880000000000001},"last":0.51900000000000002},{"interface":"front","average":{"1min":0.628,"5min":0.65800000000000003,"15min":0.66600000000000004},"min":{"1min":0.32800000000000001,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.79500000000000004,"5min":0.95599999999999996,"15min":0.95599999999999996},"last":0.61099999999999999}]}]},{"osd":1,"up_from":13,"seq":55834574898,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28652,"kb_used_data":1628,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343188,"statfs":{"total":96636764160,"available":96607424512,"internally_reserved":0,"allocated":1667072,"data_stored":1049573,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 07:59:24 2023","interfaces":[{"interface":"back","average":{"1min":0.35499999999999998,"5min":0.35499999999999998,"15min":0.36099999999999999},"min":{"1min":0.248,"5min":0.20200000000000001,"15min":0.20200000000000001},"max":{"1min":0.52600000000000002,"5min":0.70899999999999996,"15min":0.70899999999999996},"last":0.49299999999999999},{"interface":"front","average":{"1min":0.36299999999999999,"5min":0.38400000000000001,"15min":0.39300000000000002},"min":{"1min":0.23300000000000001,"5min":0.214,"15min":0.214},"max":{"1min":0.65600000000000003,"5min":0.82499999999999996,"15min":0.82499999999999996},"last":0.43099999999999999}]},{"osd":2,"last update":"Fri Nov 24 07:59:24 2023","interfaces":[{"interface":"back","average":{"1min":0.36899999999999999,"5min":0.35999999999999999,"15min":0.36799999999999999},"min":{"1min":0.27100000000000002,"5min":0.22700000000000001,"15min":0.22700000000000001},"max":{"1min":0.53000000000000003,"5min":0.60999999999999999,"15min":0.60999999999999999},"last":0.47099999999999997},{"interface":"front","average":{"1min":0.42299999999999999,"5min":0.40500000000000003,"15min":0.40999999999999998},"min":{"1min":0.28899999999999998,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":0.63700000000000001,"5min":0.63700000000000001,"15min":0.63700000000000001},"last":0.55600000000000005}]},{"osd":3,"last update":"Fri Nov 24 07:59:24 2023","interfaces":[{"interface":"back","average":{"1min":0.43099999999999999,"5min":0.433,"15min":0.44},"min":{"1min":0.317,"5min":0.317,"15min":0.317},"max":{"1min":0.62,"5min":0.66500000000000004,"15min":0.66500000000000004},"last":0.58299999999999996},{"interface":"front","average":{"1min":0.41799999999999998,"5min":0.41699999999999998,"15min":0.42399999999999999},"min":{"1min":0.32900000000000001,"5min":0.28499999999999998,"15min":0.28499999999999998},"max":{"1min":0.59999999999999998,"5min":0.76300000000000001,"15min":0.76300000000000001},"last":0.60899999999999999}]},{"osd":4,"last update":"Fri Nov 24 07:59:24 2023","interfaces":[{"interface":"back","average":{"1min":0.625,"5min":0.60399999999999998,"15min":0.60799999999999998},"min":{"1min":0.434,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":1.5580000000000001,"5min":1.5580000000000001,"15min":1.5580000000000001},"last":0.90500000000000003},{"interface":"front","average":{"1min":0.65200000000000002,"5min":0.61599999999999999,"15min":0.60899999999999999},"min":{"1min":0.48599999999999999,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":0.82399999999999995,"5min":0.82399999999999995,"15min":0.82399999999999995},"last":0.77600000000000002}]},{"osd":5,"last update":"Fri Nov 24 07:59:24 2023","interfaces":[{"interface":"back","average":{"1min":0.625,"5min":0.67800000000000005,"15min":0.70899999999999996},"min":{"1min":0.44400000000000001,"5min":0.37,"15min":0.37},"max":{"1min":0.755,"5min":2.7269999999999999,"15min":2.7269999999999999},"last":0.98699999999999999},{"interface":"front","average":{"1min":0.68500000000000005,"5min":0.63900000000000001,"15min":0.64300000000000002},"min":{"1min":0.434,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":1.6459999999999999,"5min":1.6459999999999999,"15min":1.6459999999999999},"last":0.84999999999999998}]},{"osd":6,"last update":"Fri Nov 24 07:59:24 2023","interfaces":[{"interface":"back","average":{"1min":0.63200000000000001,"5min":0.64300000000000002,"15min":0.65300000000000002},"min":{"1min":0.46000000000000002,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":0.93799999999999994,"5min":0.98199999999999998,"15min":0.98199999999999998},"last":0.81999999999999995},{"interface":"front","average":{"1min":0.67900000000000005,"5min":0.621,"15min":0.61799999999999999},"min":{"1min":0.46100000000000002,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":0.91600000000000004,"5min":0.91600000000000004,"15min":0.91600000000000004},"last":0.93600000000000005}]},{"osd":7,"last update":"Fri Nov 24 07:59:24 2023","interfaces":[{"interface":"back","average":{"1min":0.69699999999999995,"5min":0.64700000000000002,"15min":0.64000000000000001},"min":{"1min":0.53900000000000003,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.95599999999999996,"5min":1.0649999999999999,"15min":1.0649999999999999},"last":0.87},{"interface":"front","average":{"1min":0.64300000000000002,"5min":0.64400000000000002,"15min":0.66100000000000003},"min":{"1min":0.45900000000000002,"5min":0.442,"15min":0.442},"max":{"1min":0.93300000000000005,"5min":0.93300000000000005,"15min":0.93300000000000005},"last":0.96799999999999997}]}]},{"osd":0,"up_from":13,"seq":55834574898,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28084,"kb_used_data":1060,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343756,"statfs":{"total":96636764160,"available":96608006144,"internally_reserved":0,"allocated":1085440,"data_stored":460749,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Nov 24 07:59:22 2023","interfaces":[{"interface":"back","average":{"1min":0.36899999999999999,"5min":0.57799999999999996,"15min":0.66800000000000004},"min":{"1min":0.247,"5min":0.21299999999999999,"15min":0.21299999999999999},"max":{"1min":0.56100000000000005,"5min":6.3550000000000004,"15min":6.3550000000000004},"last":0.45700000000000002},{"interface":"front","average":{"1min":0.39300000000000002,"5min":1.1970000000000001,"15min":1.5509999999999999},"min":{"1min":0.29399999999999998,"5min":0.26000000000000001,"15min":0.26000000000000001},"max":{"1min":0.59099999999999997,"5min":22.916,"15min":22.916},"last":0.53500000000000003}]},{"osd":2,"last update":"Fri Nov 24 07:59:22 2023","interfaces":[{"interface":"back","average":{"1min":0.35999999999999999,"5min":0.56200000000000006,"15min":0.65400000000000003},"min":{"1min":0.23400000000000001,"5min":0.22800000000000001,"15min":0.22800000000000001},"max":{"1min":0.54100000000000004,"5min":6.3899999999999997,"15min":6.3899999999999997},"last":0.39700000000000002},{"interface":"front","average":{"1min":0.32300000000000001,"5min":0.78100000000000003,"15min":0.96899999999999997},"min":{"1min":0.253,"5min":0.253,"15min":0.253},"max":{"1min":0.497,"5min":11.93,"15min":11.93},"last":0.30499999999999999}]},{"osd":3,"last update":"Fri Nov 24 07:59:22 2023","interfaces":[{"interface":"back","average":{"1min":0.373,"5min":0.40100000000000002,"15min":0.41299999999999998},"min":{"1min":0.247,"5min":0.23799999999999999,"15min":0.23799999999999999},"max":{"1min":0.52500000000000002,"5min":1.155,"15min":1.155},"last":0.36499999999999999},{"interface":"front","average":{"1min":0.40200000000000002,"5min":0.84399999999999997,"15min":1.032},"min":{"1min":0.27400000000000002,"5min":0.27400000000000002,"15min":0.27400000000000002},"max":{"1min":0.63400000000000001,"5min":12.116,"15min":12.116},"last":0.33800000000000002}]},{"osd":4,"last update":"Fri Nov 24 07:59:22 2023","interfaces":[{"interface":"back","average":{"1min":0.59799999999999998,"5min":0.63300000000000001,"15min":0.64300000000000002},"min":{"1min":0.44400000000000001,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.93500000000000005,"5min":1.044,"15min":1.044},"last":0.46800000000000003},{"interface":"front","average":{"1min":0.58499999999999996,"5min":1.4059999999999999,"15min":1.758},"min":{"1min":0.377,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.84999999999999998,"5min":22.963999999999999,"15min":22.963999999999999},"last":0.622}]},{"osd":5,"last update":"Fri Nov 24 07:59:22 2023","interfaces":[{"interface":"back","average":{"1min":0.59999999999999998,"5min":1.026,"15min":1.204},"min":{"1min":0.41199999999999998,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.86299999999999999,"5min":12.084,"15min":12.084},"last":0.56599999999999995},{"interface":"front","average":{"1min":0.61899999999999999,"5min":0.66200000000000003,"15min":0.67200000000000004},"min":{"1min":0.46600000000000003,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.89100000000000001,"5min":1.073,"15min":1.073},"last":0.59999999999999998}]},{"osd":6,"last update":"Fri Nov 24 07:59:22 2023","interfaces":[{"interface":"back","average":{"1min":0.61599999999999999,"5min":0.66800000000000004,"15min":0.68300000000000005},"min":{"1min":0.36799999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.92400000000000004,"5min":1.0580000000000001,"15min":1.0580000000000001},"last":0.65600000000000003},{"interface":"front","average":{"1min":0.60099999999999998,"5min":1.042,"15min":1.2230000000000001},"min":{"1min":0.46000000000000002,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":0.81999999999999995,"5min":12.045,"15min":12.045},"last":0.48599999999999999}]},{"osd":7,"last update":"Fri Nov 24 07:59:22 2023","interfaces":[{"interface":"back","average":{"1min":0.57699999999999996,"5min":0.64300000000000002,"15min":0.65100000000000002},"min":{"1min":0.45700000000000002,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":0.82999999999999996,"5min":1.038,"15min":1.038},"last":0.51900000000000002},{"interface":"front","average":{"1min":0.59499999999999997,"5min":1.431,"15min":1.7849999999999999},"min":{"1min":0.42299999999999999,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":0.80300000000000005,"5min":22.995000000000001,"15min":22.995000000000001},"last":0.67000000000000004}]}]},{"osd":2,"up_from":13,"seq":55834574898,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28068,"kb_used_data":1044,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343772,"statfs":{"total":96636764160,"available":96608022528,"internally_reserved":0,"allocated":1069056,"data_stored":459115,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 07:59:20 2023","interfaces":[{"interface":"back","average":{"1min":0.31900000000000001,"5min":0.35899999999999999,"15min":0.36699999999999999},"min":{"1min":0.24399999999999999,"5min":0.219,"15min":0.219},"max":{"1min":0.52600000000000002,"5min":0.58099999999999996,"15min":0.58099999999999996},"last":0.49399999999999999},{"interface":"front","average":{"1min":0.376,"5min":0.39400000000000002,"15min":0.39900000000000002},"min":{"1min":0.29699999999999999,"5min":0.27400000000000002,"15min":0.27400000000000002},"max":{"1min":0.52700000000000002,"5min":0.54100000000000004,"15min":0.54100000000000004},"last":0.39900000000000002}]},{"osd":1,"last update":"Fri Nov 24 07:59:20 2023","interfaces":[{"interface":"back","average":{"1min":0.32400000000000001,"5min":0.36099999999999999,"15min":0.372},"min":{"1min":0.21099999999999999,"5min":0.21099999999999999,"15min":0.21099999999999999},"max":{"1min":0.44600000000000001,"5min":0.59199999999999997,"15min":0.59199999999999997},"last":0.375},{"interface":"front","average":{"1min":0.36599999999999999,"5min":0.39600000000000002,"15min":0.40899999999999997},"min":{"1min":0.26900000000000002,"5min":0.25600000000000001,"15min":0.25600000000000001},"max":{"1min":0.51100000000000001,"5min":0.627,"15min":0.627},"last":0.45000000000000001}]},{"osd":3,"last update":"Fri Nov 24 07:59:20 2023","interfaces":[{"interface":"back","average":{"1min":0.36799999999999999,"5min":0.40100000000000002,"15min":0.40699999999999997},"min":{"1min":0.26400000000000001,"5min":0.26400000000000001,"15min":0.26400000000000001},"max":{"1min":0.59899999999999998,"5min":0.64200000000000002,"15min":0.64200000000000002},"last":0.46999999999999997},{"interface":"front","average":{"1min":0.36499999999999999,"5min":0.40400000000000003,"15min":0.40999999999999998},"min":{"1min":0.28100000000000003,"5min":0.27500000000000002,"15min":0.27500000000000002},"max":{"1min":0.58199999999999996,"5min":0.64600000000000002,"15min":0.64600000000000002},"last":0.42299999999999999}]},{"osd":4,"last update":"Fri Nov 24 07:59:20 2023","interfaces":[{"interface":"back","average":{"1min":0.50700000000000001,"5min":0.57499999999999996,"15min":0.59699999999999998},"min":{"1min":0.35799999999999998,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.73099999999999998,"5min":0.81599999999999995,"15min":0.81599999999999995},"last":0.80000000000000004},{"interface":"front","average":{"1min":0.56000000000000005,"5min":0.58299999999999996,"15min":0.59399999999999997},"min":{"1min":0.38700000000000001,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.68100000000000005,"5min":0.83799999999999997,"15min":0.83799999999999997},"last":0.61699999999999999}]},{"osd":5,"last update":"Fri Nov 24 07:59:20 2023","interfaces":[{"interface":"back","average":{"1min":0.57199999999999995,"5min":0.59899999999999998,"15min":0.61299999999999999},"min":{"1min":0.41599999999999998,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":0.745,"5min":0.84899999999999998,"15min":0.84899999999999998},"last":0.69999999999999996},{"interface":"front","average":{"1min":0.53400000000000003,"5min":0.66000000000000003,"15min":0.69599999999999995},"min":{"1min":0.41599999999999998,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":0.75800000000000001,"5min":2.1699999999999999,"15min":2.1699999999999999},"last":0.63100000000000001}]},{"osd":6,"last update":"Fri Nov 24 07:59:20 2023","interfaces":[{"interface":"back","average":{"1min":0.55000000000000004,"5min":0.63100000000000001,"15min":0.65500000000000003},"min":{"1min":0.42099999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":0.75,"5min":0.878,"15min":0.878},"last":0.83799999999999997},{"interface":"front","average":{"1min":0.54300000000000004,"5min":0.63200000000000001,"15min":0.65800000000000003},"min":{"1min":0.436,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":0.81699999999999995,"5min":0.88500000000000001,"15min":0.88500000000000001},"last":0.76000000000000001}]},{"osd":7,"last update":"Fri Nov 24 07:59:20 2023","interfaces":[{"interface":"back","average":{"1min":0.57499999999999996,"5min":0.63300000000000001,"15min":0.65200000000000002},"min":{"1min":0.42799999999999999,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":0.74199999999999999,"5min":0.85999999999999999,"15min":0.85999999999999999},"last":0.89300000000000002},{"interface":"front","average":{"1min":0.57399999999999995,"5min":0.64700000000000002,"15min":0.66900000000000004},"min":{"1min":0.39700000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.77900000000000003,"5min":0.90000000000000002,"15min":0.90000000000000002},"last":0.74099999999999999}]}]},{"osd":3,"up_from":13,"seq":55834574898,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28080,"kb_used_data":1056,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343760,"statfs":{"total":96636764160,"available":96608010240,"internally_reserved":0,"allocated":1081344,"data_stored":459789,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 07:59:27 2023","interfaces":[{"interface":"back","average":{"1min":0.33600000000000002,"5min":0.34899999999999998,"15min":0.35299999999999998},"min":{"1min":0.183,"5min":0.183,"15min":0.183},"max":{"1min":0.53400000000000003,"5min":0.84799999999999998,"15min":0.84799999999999998},"last":0.35199999999999998},{"interface":"front","average":{"1min":0.34599999999999997,"5min":0.35399999999999998,"15min":0.36199999999999999},"min":{"1min":0.24299999999999999,"5min":0.23699999999999999,"15min":0.23699999999999999},"max":{"1min":0.51000000000000001,"5min":0.873,"15min":0.873},"last":0.45900000000000002}]},{"osd":1,"last update":"Fri Nov 24 07:59:27 2023","interfaces":[{"interface":"back","average":{"1min":0.38400000000000001,"5min":0.373,"15min":0.374},"min":{"1min":0.217,"5min":0.217,"15min":0.217},"max":{"1min":0.66300000000000003,"5min":0.92000000000000004,"15min":0.92000000000000004},"last":0.48799999999999999},{"interface":"front","average":{"1min":0.38,"5min":0.377,"15min":0.38300000000000001},"min":{"1min":0.28599999999999998,"5min":0.248,"15min":0.248},"max":{"1min":0.626,"5min":0.81699999999999995,"15min":0.81699999999999995},"last":0.38}]},{"osd":2,"last update":"Fri Nov 24 07:59:27 2023","interfaces":[{"interface":"back","average":{"1min":0.35599999999999998,"5min":0.35399999999999998,"15min":0.35799999999999998},"min":{"1min":0.253,"5min":0.24399999999999999,"15min":0.24399999999999999},"max":{"1min":0.496,"5min":0.88500000000000001,"15min":0.88500000000000001},"last":0.40600000000000003},{"interface":"front","average":{"1min":0.40999999999999998,"5min":0.441,"15min":0.45900000000000002},"min":{"1min":0.315,"5min":0.311,"15min":0.311},"max":{"1min":0.59799999999999998,"5min":0.98099999999999998,"15min":0.98099999999999998},"last":0.42999999999999999}]},{"osd":4,"last update":"Fri Nov 24 07:59:27 2023","interfaces":[{"interface":"back","average":{"1min":0.56299999999999994,"5min":0.70499999999999996,"15min":0.76600000000000001},"min":{"1min":0.38900000000000001,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.73599999999999999,"5min":5.7350000000000003,"15min":5.7350000000000003},"last":0.67300000000000004},{"interface":"front","average":{"1min":0.55700000000000005,"5min":0.58099999999999996,"15min":0.58299999999999996},"min":{"1min":0.34399999999999997,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.80400000000000005,"5min":1.27,"15min":1.27},"last":0.65800000000000003}]},{"osd":5,"last update":"Fri Nov 24 07:59:27 2023","interfaces":[{"interface":"back","average":{"1min":0.56100000000000005,"5min":0.57799999999999996,"15min":0.58399999999999996},"min":{"1min":0.42899999999999999,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":0.67900000000000005,"5min":1.4430000000000001,"15min":1.4430000000000001},"last":0.73499999999999999},{"interface":"front","average":{"1min":0.58799999999999997,"5min":0.61499999999999999,"15min":0.623},"min":{"1min":0.40999999999999998,"5min":0.377,"15min":0.377},"max":{"1min":0.78500000000000003,"5min":1.4239999999999999,"15min":1.4239999999999999},"last":0.63900000000000001}]},{"osd":6,"last update":"Fri Nov 24 07:59:27 2023","interfaces":[{"interface":"back","average":{"1min":0.58799999999999997,"5min":0.60699999999999998,"15min":0.61499999999999999},"min":{"1min":0.42599999999999999,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":0.84499999999999997,"5min":1.355,"15min":1.355},"last":0.59899999999999998},{"interface":"front","average":{"1min":0.57899999999999996,"5min":0.745,"15min":0.81200000000000006},"min":{"1min":0.44800000000000001,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.77100000000000002,"5min":5.6479999999999997,"15min":5.6479999999999997},"last":0.70599999999999996}]},{"osd":7,"last update":"Fri Nov 24 07:59:27 2023","interfaces":[{"interface":"back","average":{"1min":0.61799999999999999,"5min":0.64100000000000001,"15min":0.65700000000000003},"min":{"1min":0.499,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":0.82799999999999996,"5min":1.51,"15min":1.51},"last":0.61599999999999999},{"interface":"front","average":{"1min":0.57699999999999996,"5min":0.74199999999999999,"15min":0.80800000000000005},"min":{"1min":0.40100000000000002,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.79500000000000004,"5min":5.7069999999999999,"15min":5.7069999999999999},"last":0.77600000000000002}]}]},{"osd":4,"up_from":13,"seq":55834574897,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28072,"kb_used_data":1048,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343768,"statfs":{"total":96636764160,"available":96608018432,"internally_reserved":0,"allocated":1073152,"data_stored":459149,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 07:59:28 2023","interfaces":[{"interface":"back","average":{"1min":0.55900000000000005,"5min":0.56100000000000005,"15min":0.57399999999999995},"min":{"1min":0.32400000000000001,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.93300000000000005,"5min":0.93300000000000005,"15min":0.93300000000000005},"last":0.64200000000000002},{"interface":"front","average":{"1min":0.51100000000000001,"5min":0.52800000000000002,"15min":0.53800000000000003},"min":{"1min":0.36199999999999999,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.69299999999999995,"5min":1.0249999999999999,"15min":1.0249999999999999},"last":0.55300000000000005}]},{"osd":1,"last update":"Fri Nov 24 07:59:28 2023","interfaces":[{"interface":"back","average":{"1min":0.54800000000000004,"5min":0.54600000000000004,"15min":0.55000000000000004},"min":{"1min":0.35899999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.71599999999999997,"5min":0.86399999999999999,"15min":0.86399999999999999},"last":0.43099999999999999},{"interface":"front","average":{"1min":0.51600000000000001,"5min":0.53700000000000003,"15min":0.54600000000000004},"min":{"1min":0.36699999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.72099999999999997,"5min":0.98699999999999999,"15min":0.98699999999999999},"last":0.61899999999999999}]},{"osd":2,"last update":"Fri Nov 24 07:59:28 2023","interfaces":[{"interface":"back","average":{"1min":0.54200000000000004,"5min":0.54900000000000004,"15min":0.55700000000000005},"min":{"1min":0.34499999999999997,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.86399999999999999,"5min":0.86399999999999999,"15min":0.86399999999999999},"last":0.45600000000000002},{"interface":"front","average":{"1min":0.56200000000000006,"5min":0.59399999999999997,"15min":0.60599999999999998},"min":{"1min":0.34000000000000002,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.81899999999999995,"5min":0.89200000000000002,"15min":0.89200000000000002},"last":0.46899999999999997}]},{"osd":3,"last update":"Fri Nov 24 07:59:28 2023","interfaces":[{"interface":"back","average":{"1min":0.53600000000000003,"5min":0.56799999999999995,"15min":0.57899999999999996},"min":{"1min":0.38800000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.78100000000000003,"5min":0.89700000000000002,"15min":0.89700000000000002},"last":0.375},{"interface":"front","average":{"1min":0.58899999999999997,"5min":0.61099999999999999,"15min":0.624},"min":{"1min":0.42699999999999999,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.73999999999999999,"5min":0.96199999999999997,"15min":0.96199999999999997},"last":0.52000000000000002}]},{"osd":5,"last update":"Fri Nov 24 07:59:28 2023","interfaces":[{"interface":"back","average":{"1min":0.46800000000000003,"5min":0.46300000000000002,"15min":0.45700000000000002},"min":{"1min":0.34999999999999998,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":0.60499999999999998,"5min":0.70499999999999996,"15min":0.70499999999999996},"last":0.35999999999999999},{"interface":"front","average":{"1min":0.56599999999999995,"5min":0.60399999999999998,"15min":0.61399999999999999},"min":{"1min":0.41599999999999998,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.73699999999999999,"5min":0.86399999999999999,"15min":0.86399999999999999},"last":0.505}]},{"osd":6,"last update":"Fri Nov 24 07:59:28 2023","interfaces":[{"interface":"back","average":{"1min":0.53900000000000003,"5min":0.57299999999999995,"15min":0.58099999999999996},"min":{"1min":0.33000000000000002,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.88600000000000001,"5min":0.88600000000000001,"15min":0.88600000000000001},"last":0.59499999999999997},{"interface":"front","average":{"1min":0.56899999999999995,"5min":0.59999999999999998,"15min":0.60899999999999999},"min":{"1min":0.32100000000000001,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.80300000000000005,"5min":1.095,"15min":1.095},"last":0.49299999999999999}]},{"osd":7,"last update":"Fri Nov 24 07:59:28 2023","interfaces":[{"interface":"back","average":{"1min":0.52000000000000002,"5min":0.56100000000000005,"15min":0.56899999999999995},"min":{"1min":0.36299999999999999,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.69999999999999996,"5min":0.89300000000000002,"15min":0.89300000000000002},"last":0.41399999999999998},{"interface":"front","average":{"1min":0.57799999999999996,"5min":0.63600000000000001,"15min":0.65400000000000003},"min":{"1min":0.39000000000000001,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.74399999999999999,"5min":0.91500000000000004,"15min":0.91500000000000004},"last":0.57099999999999995}]}]},{"osd":5,"up_from":13,"seq":55834574898,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32824,"kb_used_data":1640,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339016,"statfs":{"total":96636764160,"available":96603152384,"internally_reserved":0,"allocated":1679360,"data_stored":1051129,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 07:59:21 2023","interfaces":[{"interface":"back","average":{"1min":0.52500000000000002,"5min":0.54600000000000004,"15min":0.55600000000000005},"min":{"1min":0.311,"5min":0.311,"15min":0.311},"max":{"1min":0.74399999999999999,"5min":1.0109999999999999,"15min":1.0109999999999999},"last":0.59299999999999997},{"interface":"front","average":{"1min":0.53600000000000003,"5min":0.69299999999999995,"15min":0.76400000000000001},"min":{"1min":0.32400000000000001,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.81899999999999995,"5min":5.431,"15min":5.431},"last":0.61299999999999999}]},{"osd":1,"last update":"Fri Nov 24 07:59:21 2023","interfaces":[{"interface":"back","average":{"1min":0.52600000000000002,"5min":0.54200000000000004,"15min":0.55300000000000005},"min":{"1min":0.35399999999999998,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":0.69099999999999995,"5min":1.016,"15min":1.016},"last":0.54000000000000004},{"interface":"front","average":{"1min":0.56399999999999995,"5min":0.56399999999999995,"15min":0.56799999999999995},"min":{"1min":0.34000000000000002,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":0.80100000000000005,"5min":0.96499999999999997,"15min":0.96499999999999997},"last":0.67900000000000005}]},{"osd":2,"last update":"Fri Nov 24 07:59:21 2023","interfaces":[{"interface":"back","average":{"1min":0.58099999999999996,"5min":0.55500000000000005,"15min":0.55200000000000005},"min":{"1min":0.44800000000000001,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.876,"5min":1.163,"15min":1.163},"last":0.50900000000000001},{"interface":"front","average":{"1min":0.55900000000000005,"5min":0.59799999999999998,"15min":0.61899999999999999},"min":{"1min":0.32600000000000001,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.72399999999999998,"5min":1.119,"15min":1.119},"last":0.57599999999999996}]},{"osd":3,"last update":"Fri Nov 24 07:59:21 2023","interfaces":[{"interface":"back","average":{"1min":0.59799999999999998,"5min":0.77700000000000002,"15min":0.86199999999999999},"min":{"1min":0.32400000000000001,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.77100000000000002,"5min":5.609,"15min":5.609},"last":0.64000000000000001},{"interface":"front","average":{"1min":0.64300000000000002,"5min":0.66000000000000003,"15min":0.67000000000000004},"min":{"1min":0.38900000000000001,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.84999999999999998,"5min":1.288,"15min":1.288},"last":0.56100000000000005}]},{"osd":4,"last update":"Fri Nov 24 07:59:21 2023","interfaces":[{"interface":"back","average":{"1min":0.53800000000000003,"5min":0.51600000000000001,"15min":0.51200000000000001},"min":{"1min":0.30599999999999999,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":0.79100000000000004,"5min":0.79100000000000004,"15min":0.79100000000000004},"last":0.32800000000000001},{"interface":"front","average":{"1min":0.58899999999999997,"5min":0.57799999999999996,"15min":0.57399999999999995},"min":{"1min":0.35399999999999998,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":0.82599999999999996,"5min":0.92900000000000005,"15min":0.92900000000000005},"last":0.48199999999999998}]},{"osd":6,"last update":"Fri Nov 24 07:59:21 2023","interfaces":[{"interface":"back","average":{"1min":0.63,"5min":0.58699999999999997,"15min":0.57599999999999996},"min":{"1min":0.434,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.85399999999999998,"5min":0.89700000000000002,"15min":0.89700000000000002},"last":0.36799999999999999},{"interface":"front","average":{"1min":0.59299999999999997,"5min":0.621,"15min":0.623},"min":{"1min":0.376,"5min":0.376,"15min":0.376},"max":{"1min":0.74299999999999999,"5min":1.083,"15min":1.083},"last":0.38900000000000001}]},{"osd":7,"last update":"Fri Nov 24 07:59:21 2023","interfaces":[{"interface":"back","average":{"1min":0.65400000000000003,"5min":0.65000000000000002,"15min":0.64700000000000002},"min":{"1min":0.40699999999999997,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.86099999999999999,"5min":0.96599999999999997,"15min":0.96599999999999997},"last":0.42899999999999999},{"interface":"front","average":{"1min":0.68000000000000005,"5min":0.66100000000000003,"15min":0.65500000000000003},"min":{"1min":0.49099999999999999,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":0.78500000000000003,"5min":0.94199999999999995,"15min":0.94199999999999995},"last":0.40799999999999997}]}]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":674,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1646,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-11-24T08:00:05.784 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-11-24T08:00:05.940 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:05.939+0000 7f52fdc71700 1 Processor -- start 2023-11-24T08:00:05.941 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:05.939+0000 7f52fdc71700 1 -- start start 2023-11-24T08:00:05.945 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:05.939+0000 7f52fdc71700 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f52f806afd0 0x7f52f80d16e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:00:05.945 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:05.939+0000 7f52fdc71700 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f52f80ada10 0x7f52f80adea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:00:05.945 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:05.939+0000 7f52fdc71700 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f52f8070490 con 0x7f52f80affb0 2023-11-24T08:00:05.945 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:05.939+0000 7f52fdc71700 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f52f80b0410 con 0x7f52f806afd0 2023-11-24T08:00:05.945 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:05.939+0000 7f52fdc71700 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f52f80af9f0 con 0x7f52f80ada10 2023-11-24T08:00:05.945 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:05.940+0000 7f52f6ffd700 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7f52f80affb0 0x7f52f806b3b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:59386/0 (socket says 172.21.15.59:59386) 2023-11-24T08:00:05.946 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:05.940+0000 7f52f6ffd700 1 -- 172.21.15.59:0/4229227844 learned_addr learned my addr 172.21.15.59:0/4229227844 (peer_addr_for_me v1:172.21.15.59:0/0) 2023-11-24T08:00:05.946 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:05.940+0000 7f52f77fe700 1 --2- 172.21.15.59:0/4229227844 >> v2:172.21.15.76:3300/0 conn(0x7f52f806afd0 0x7f52f80d16e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:00:05.946 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:05.940+0000 7f52f7fff700 1 --2- 172.21.15.59:0/4229227844 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f52f80ada10 0x7f52f80adea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:00:05.946 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:05.940+0000 7f52f67fc700 1 -- 172.21.15.59:0/4229227844 <== mon.0 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 984835208 0 0) 0x7f52f8070490 con 0x7f52f80affb0 2023-11-24T08:00:05.946 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:05.940+0000 7f52f67fc700 1 -- 172.21.15.59:0/4229227844 --> v1:172.21.15.59:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f52e40035b0 con 0x7f52f80affb0 2023-11-24T08:00:05.946 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:05.940+0000 7f52f67fc700 1 -- 172.21.15.59:0/4229227844 <== mon.0 v1:172.21.15.59:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2169237 0 0) 0x7f52e40035b0 con 0x7f52f80affb0 2023-11-24T08:00:05.946 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:05.940+0000 7f52f67fc700 1 -- 172.21.15.59:0/4229227844 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f52f80ada10 msgr2=0x7f52f80adea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:00:05.947 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:05.940+0000 7f52f67fc700 1 --2- 172.21.15.59:0/4229227844 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f52f80ada10 0x7f52f80adea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:00:05.947 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:05.940+0000 7f52f67fc700 1 -- 172.21.15.59:0/4229227844 >> v2:172.21.15.76:3300/0 conn(0x7f52f806afd0 msgr2=0x7f52f80d16e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:00:05.947 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:05.940+0000 7f52f67fc700 1 --2- 172.21.15.59:0/4229227844 >> v2:172.21.15.76:3300/0 conn(0x7f52f806afd0 0x7f52f80d16e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:00:05.947 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:05.940+0000 7f52f67fc700 1 -- 172.21.15.59:0/4229227844 --> v1:172.21.15.59:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f52f80afcd0 con 0x7f52f80affb0 2023-11-24T08:00:05.947 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:05.940+0000 7f52f67fc700 1 -- 172.21.15.59:0/4229227844 <== mon.0 v1:172.21.15.59:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1312333566 0 0) 0x7f52e0001670 con 0x7f52f80affb0 2023-11-24T08:00:05.947 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:05.940+0000 7f52f67fc700 1 -- 172.21.15.59:0/4229227844 >> v1:172.21.15.59:6789/0 conn(0x7f52f80affb0 legacy=0x7f52f806b3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:00:05.948 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:05.940+0000 7f52f67fc700 1 --2- 172.21.15.59:0/4229227844 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f52f80ada10 0x7f52e4003a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:00:05.948 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:05.940+0000 7f52f67fc700 1 --2- 172.21.15.59:0/4229227844 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f52e4003f80 0x7f52e4004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:00:05.948 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:05.940+0000 7f52f67fc700 1 --2- 172.21.15.59:0/4229227844 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f52e4004d90 0x7f52e4005210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:00:05.948 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:05.940+0000 7f52f67fc700 1 -- 172.21.15.59:0/4229227844 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52f80b0410 con 0x7f52e4003f80 2023-11-24T08:00:05.948 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:05.940+0000 7f52f67fc700 1 -- 172.21.15.59:0/4229227844 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52f80af9f0 con 0x7f52e4004d90 2023-11-24T08:00:05.948 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:05.940+0000 7f52f67fc700 1 -- 172.21.15.59:0/4229227844 --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f52e400ba10 con 0x7f52f80ada10 2023-11-24T08:00:05.949 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:05.940+0000 7f52f6ffd700 1 --2- 172.21.15.59:0/4229227844 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f52e4003f80 0x7f52e4004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:00:05.949 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:05.941+0000 7f52f77fe700 1 --2- 172.21.15.59:0/4229227844 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f52f80ada10 0x7f52e4003a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:00:05.949 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:05.941+0000 7f52f7fff700 1 --2- 172.21.15.59:0/4229227844 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f52e4004d90 0x7f52e4005210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:00:05.949 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:05.941+0000 7f52f6ffd700 1 -- 172.21.15.59:0/4229227844 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f52f80ada10 msgr2=0x7f52e4003a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:00:05.949 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:05.941+0000 7f52f6ffd700 1 --2- 172.21.15.59:0/4229227844 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f52f80ada10 0x7f52e4003a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:00:05.949 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:05.941+0000 7f52f6ffd700 1 -- 172.21.15.59:0/4229227844 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f52e4004d90 msgr2=0x7f52e4005210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:00:05.949 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:05.941+0000 7f52f6ffd700 1 --2- 172.21.15.59:0/4229227844 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f52e4004d90 0x7f52e4005210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:00:05.949 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:05.941+0000 7f52f6ffd700 1 -- 172.21.15.59:0/4229227844 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f52e40035b0 con 0x7f52e4003f80 2023-11-24T08:00:05.949 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:05.941+0000 7f52f7fff700 1 --2- 172.21.15.59:0/4229227844 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f52e4004d90 0x7f52e4005210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-24T08:00:05.950 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:05.941+0000 7f52f6ffd700 1 --2- 172.21.15.59:0/4229227844 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f52e4003f80 0x7f52e4004850 secure :-1 s=READY pgs=1032 cs=0 l=1 rev1=1 crypto rx=0x7f52e000c370 tx=0x7f52e4005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7fbeae3100433871 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:00:05.950 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:05.941+0000 7f52f67fc700 1 -- 172.21.15.59:0/4229227844 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52e0007c60 con 0x7f52e4003f80 2023-11-24T08:00:05.950 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:05.941+0000 7f52f67fc700 1 -- 172.21.15.59:0/4229227844 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f52e000ce50 con 0x7f52e4003f80 2023-11-24T08:00:05.950 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:05.941+0000 7f52f67fc700 1 -- 172.21.15.59:0/4229227844 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52e0005ba0 con 0x7f52e4003f80 2023-11-24T08:00:05.950 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:05.941+0000 7f52fdc71700 1 -- 172.21.15.59:0/4229227844 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f52e4003f80 msgr2=0x7f52e4004850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:00:05.950 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:05.941+0000 7f52fdc71700 1 --2- 172.21.15.59:0/4229227844 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f52e4003f80 0x7f52e4004850 secure :-1 s=READY pgs=1032 cs=0 l=1 rev1=1 crypto rx=0x7f52e000c370 tx=0x7f52e4005880 comp rx=0 tx=0).stop 2023-11-24T08:00:05.950 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:05.941+0000 7f52f7fff700 1 -- 172.21.15.59:0/4229227844 reap_dead start 2023-11-24T08:00:05.950 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:05.942+0000 7f52fdc71700 1 -- 172.21.15.59:0/4229227844 shutdown_connections 2023-11-24T08:00:05.950 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:05.942+0000 7f52fdc71700 1 -- 172.21.15.59:0/4229227844 >> 172.21.15.59:0/4229227844 conn(0x7f52f80d0bc0 msgr2=0x7f52f80d0fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:00:05.950 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:05.942+0000 7f52fdc71700 1 -- 172.21.15.59:0/4229227844 shutdown_connections 2023-11-24T08:00:05.951 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:05.942+0000 7f52fdc71700 1 -- 172.21.15.59:0/4229227844 wait complete. 2023-11-24T08:00:05.951 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:05.943+0000 7f52fdc71700 1 Processor -- start 2023-11-24T08:00:05.951 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:05.943+0000 7f52fdc71700 1 -- start start 2023-11-24T08:00:05.953 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:05.944+0000 7f52fdc71700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f52e4003f80 0x7f52f8157440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:00:05.953 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:05.944+0000 7f52fdc71700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f52f813ff60 0x7f52f8140720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:00:05.954 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:05.944+0000 7f52fdc71700 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f52f8140c60 0x7f52f80cf6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:00:05.954 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:05.944+0000 7f52fdc71700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52f815a2b0 con 0x7f52e4003f80 2023-11-24T08:00:05.954 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:05.944+0000 7f52fdc71700 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52f815a590 con 0x7f52f813ff60 2023-11-24T08:00:05.954 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:05.944+0000 7f52fdc71700 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f52f80cf1b0 con 0x7f52f8140c60 2023-11-24T08:00:05.954 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:05.944+0000 7f52f77fe700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f52e4003f80 0x7f52f8157440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:00:05.954 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:05.944+0000 7f52f77fe700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f52e4003f80 0x7f52f8157440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:55922/0 (socket says 172.21.15.59:55922) 2023-11-24T08:00:05.954 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:05.944+0000 7f52f77fe700 1 -- 172.21.15.59:0/3558344218 learned_addr learned my addr 172.21.15.59:0/3558344218 (peer_addr_for_me v2:172.21.15.59:0/0) 2023-11-24T08:00:05.954 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:05.944+0000 7f52f6ffd700 1 --2- 172.21.15.59:0/3558344218 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f52f813ff60 0x7f52f8140720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:00:05.954 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:05.944+0000 7f52f7fff700 1 --2- 172.21.15.59:0/3558344218 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f52f8140c60 0x7f52f80cf6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:00:05.955 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:05.944+0000 7f52f77fe700 1 -- 172.21.15.59:0/3558344218 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f52f8140c60 msgr2=0x7f52f80cf6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:00:05.955 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:05.944+0000 7f52f77fe700 1 --2- 172.21.15.59:0/3558344218 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f52f8140c60 0x7f52f80cf6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:00:05.955 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:05.944+0000 7f52f77fe700 1 -- 172.21.15.59:0/3558344218 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f52f813ff60 msgr2=0x7f52f8140720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:00:05.955 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:05.944+0000 7f52f77fe700 1 --2- 172.21.15.59:0/3558344218 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f52f813ff60 0x7f52f8140720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:00:05.955 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:05.944+0000 7f52f77fe700 1 -- 172.21.15.59:0/3558344218 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f52f815f470 con 0x7f52e4003f80 2023-11-24T08:00:05.956 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:05.944+0000 7f52f77fe700 1 --2- 172.21.15.59:0/3558344218 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f52e4003f80 0x7f52f8157440 secure :-1 s=READY pgs=1033 cs=0 l=1 rev1=1 crypto rx=0x7f52e8011aa0 tx=0x7f52e8011db0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:00:05.956 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:05.944+0000 7f52effff700 1 -- 172.21.15.59:0/3558344218 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52e800f940 con 0x7f52e4003f80 2023-11-24T08:00:05.956 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:05.944+0000 7f52effff700 1 -- 172.21.15.59:0/3558344218 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f52e800faa0 con 0x7f52e4003f80 2023-11-24T08:00:05.956 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:05.944+0000 7f52fdc71700 1 -- 172.21.15.59:0/3558344218 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f52f81403e0 con 0x7f52e4003f80 2023-11-24T08:00:05.956 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:05.944+0000 7f52effff700 1 -- 172.21.15.59:0/3558344218 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52e800fe50 con 0x7f52e4003f80 2023-11-24T08:00:05.956 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:05.945+0000 7f52fdc71700 1 -- 172.21.15.59:0/3558344218 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f52f80cfdf0 con 0x7f52e4003f80 2023-11-24T08:00:05.956 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:05.945+0000 7f52fdc71700 1 -- 172.21.15.59:0/3558344218 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f52f80cf520 con 0x7f52e4003f80 2023-11-24T08:00:05.956 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:05.946+0000 7f52effff700 1 -- 172.21.15.59:0/3558344218 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 11) v1 ==== 89741+0+0 (secure 0 0 0) 0x7f52e8014440 con 0x7f52e4003f80 2023-11-24T08:00:05.956 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:05.946+0000 7f52effff700 1 --2- 172.21.15.59:0/3558344218 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f52d806b470 0x7f52d806d930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:00:05.956 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:05.947+0000 7f52effff700 1 -- 172.21.15.59:0/3558344218 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f52e808a5e0 con 0x7f52e4003f80 2023-11-24T08:00:05.957 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:05.947+0000 7f52f6ffd700 1 --2- 172.21.15.59:0/3558344218 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f52d806b470 0x7f52d806d930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:00:05.957 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:05.947+0000 7f52f6ffd700 1 --2- 172.21.15.59:0/3558344218 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f52d806b470 0x7f52d806d930 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7f52e0015040 tx=0x7f52e000a000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:00:05.957 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:05.951+0000 7f52effff700 1 -- 172.21.15.59:0/3558344218 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f52e8054fb0 con 0x7f52e4003f80 2023-11-24T08:00:06.148 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:06.147+0000 7f52fdc71700 1 -- 172.21.15.59:0/3558344218 --> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f52f80d0010 con 0x7f52d806b470 2023-11-24T08:00:06.157 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:06.156+0000 7f52effff700 1 -- 172.21.15.59:0/3558344218 <== mgr.4100 v2:172.21.15.76:6832/4264046536 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+289446 (secure 0 0 0) 0x7f52f80d0010 con 0x7f52d806b470 2023-11-24T08:00:06.160 INFO:teuthology.orchestra.run.smithi059.stdout: 2023-11-24T08:00:06.163 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:06.158+0000 7f52fdc71700 1 -- 172.21.15.59:0/3558344218 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f52d806b470 msgr2=0x7f52d806d930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:00:06.163 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:06.158+0000 7f52fdc71700 1 --2- 172.21.15.59:0/3558344218 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f52d806b470 0x7f52d806d930 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7f52e0015040 tx=0x7f52e000a000 comp rx=0 tx=0).stop 2023-11-24T08:00:06.163 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:06.158+0000 7f52fdc71700 1 -- 172.21.15.59:0/3558344218 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f52e4003f80 msgr2=0x7f52f8157440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:00:06.163 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:06.158+0000 7f52fdc71700 1 --2- 172.21.15.59:0/3558344218 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f52e4003f80 0x7f52f8157440 secure :-1 s=READY pgs=1033 cs=0 l=1 rev1=1 crypto rx=0x7f52e8011aa0 tx=0x7f52e8011db0 comp rx=0 tx=0).stop 2023-11-24T08:00:06.163 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:06.158+0000 7f52fdc71700 1 -- 172.21.15.59:0/3558344218 shutdown_connections 2023-11-24T08:00:06.164 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:06.158+0000 7f52fdc71700 1 --2- 172.21.15.59:0/3558344218 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f52d806b470 0x7f52d806d930 unknown :-1 s=CLOSED pgs=478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:00:06.164 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:06.158+0000 7f52fdc71700 1 --2- 172.21.15.59:0/3558344218 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f52f8140c60 0x7f52f80cf6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:00:06.164 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:06.158+0000 7f52fdc71700 1 --2- 172.21.15.59:0/3558344218 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f52e4003f80 0x7f52f8157440 unknown :-1 s=CLOSED pgs=1033 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:00:06.164 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:06.158+0000 7f52fdc71700 1 --2- 172.21.15.59:0/3558344218 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f52f813ff60 0x7f52f8140720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:00:06.164 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:06.159+0000 7f52fdc71700 1 -- 172.21.15.59:0/3558344218 >> 172.21.15.59:0/3558344218 conn(0x7f52f80d0bc0 msgr2=0x7f52f80b1390 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:00:06.164 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:06.159+0000 7f52fdc71700 1 -- 172.21.15.59:0/3558344218 shutdown_connections 2023-11-24T08:00:06.164 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:06.159+0000 7f52fdc71700 1 -- 172.21.15.59:0/3558344218 wait complete. 2023-11-24T08:00:06.164 INFO:teuthology.orchestra.run.smithi059.stderr:dumped all 2023-11-24T08:00:06.172 INFO:teuthology.orchestra.run.smithi059.stdout:{"pg_ready":true,"pg_map":{"version":193,"stamp":"2023-11-24T08:00:04.350377+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":229928,"kb_used_data":9576,"kb_used_omap":12,"kb_used_meta":220211,"kb_avail":754744792,"statfs":{"total":773094113280,"available":772858667008,"internally_reserved":0,"allocated":9805824,"data_stored":4858476,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12745,"internal_metadata":225496631},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"10.845551"},"pg_stats":[{"pgid":"13.3f","version":"0'0","reported_seq":11,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-11-24T07:59:53.608729+0000","last_change":"2023-11-24T07:59:52.533349+0000","last_active":"2023-11-24T07:59:53.608729+0000","last_peered":"2023-11-24T07:59:53.608729+0000","last_clean":"2023-11-24T07:59:53.608729+0000","last_became_active":"2023-11-24T07:59:52.532951+0000","last_became_peered":"2023-11-24T07:59:52.532951+0000","last_unstale":"2023-11-24T07:59:53.608729+0000","last_undegraded":"2023-11-24T07:59:53.608729+0000","last_fullsized":"2023-11-24T07:59:53.608729+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:59:51.477944+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:59:51.477944+0000","last_clean_scrub_stamp":"2023-11-24T07:59:51.477944+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T16:38:43.600527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.3e","version":"0'0","reported_seq":11,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-11-24T07:59:53.608999+0000","last_change":"2023-11-24T07:59:52.531701+0000","last_active":"2023-11-24T07:59:53.608999+0000","last_peered":"2023-11-24T07:59:53.608999+0000","last_clean":"2023-11-24T07:59:53.608999+0000","last_became_active":"2023-11-24T07:59:52.531393+0000","last_became_peered":"2023-11-24T07:59:52.531393+0000","last_unstale":"2023-11-24T07:59:53.608999+0000","last_undegraded":"2023-11-24T07:59:53.608999+0000","last_fullsized":"2023-11-24T07:59:53.608999+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:59:51.477944+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:59:51.477944+0000","last_clean_scrub_stamp":"2023-11-24T07:59:51.477944+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T11:25:48.888553+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.3d","version":"0'0","reported_seq":11,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-11-24T07:59:53.608830+0000","last_change":"2023-11-24T07:59:52.532095+0000","last_active":"2023-11-24T07:59:53.608830+0000","last_peered":"2023-11-24T07:59:53.608830+0000","last_clean":"2023-11-24T07:59:53.608830+0000","last_became_active":"2023-11-24T07:59:52.531717+0000","last_became_peered":"2023-11-24T07:59:52.531717+0000","last_unstale":"2023-11-24T07:59:53.608830+0000","last_undegraded":"2023-11-24T07:59:53.608830+0000","last_fullsized":"2023-11-24T07:59:53.608830+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:59:51.477944+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:59:51.477944+0000","last_clean_scrub_stamp":"2023-11-24T07:59:51.477944+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T12:58:00.253001+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.3c","version":"0'0","reported_seq":11,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-11-24T07:59:53.506857+0000","last_change":"2023-11-24T07:59:52.531686+0000","last_active":"2023-11-24T07:59:53.506857+0000","last_peered":"2023-11-24T07:59:53.506857+0000","last_clean":"2023-11-24T07:59:53.506857+0000","last_became_active":"2023-11-24T07:59:52.531434+0000","last_became_peered":"2023-11-24T07:59:52.531434+0000","last_unstale":"2023-11-24T07:59:53.506857+0000","last_undegraded":"2023-11-24T07:59:53.506857+0000","last_fullsized":"2023-11-24T07:59:53.506857+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:59:51.477944+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:59:51.477944+0000","last_clean_scrub_stamp":"2023-11-24T07:59:51.477944+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T13:08:53.960278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.3b","version":"0'0","reported_seq":11,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-11-24T07:59:53.505961+0000","last_change":"2023-11-24T07:59:52.520726+0000","last_active":"2023-11-24T07:59:53.505961+0000","last_peered":"2023-11-24T07:59:53.505961+0000","last_clean":"2023-11-24T07:59:53.505961+0000","last_became_active":"2023-11-24T07:59:52.520430+0000","last_became_peered":"2023-11-24T07:59:52.520430+0000","last_unstale":"2023-11-24T07:59:53.505961+0000","last_undegraded":"2023-11-24T07:59:53.505961+0000","last_fullsized":"2023-11-24T07:59:53.505961+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:59:51.477944+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:59:51.477944+0000","last_clean_scrub_stamp":"2023-11-24T07:59:51.477944+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T15:47:53.005002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.3a","version":"0'0","reported_seq":11,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-11-24T07:59:53.608618+0000","last_change":"2023-11-24T07:59:52.531758+0000","last_active":"2023-11-24T07:59:53.608618+0000","last_peered":"2023-11-24T07:59:53.608618+0000","last_clean":"2023-11-24T07:59:53.608618+0000","last_became_active":"2023-11-24T07:59:52.531393+0000","last_became_peered":"2023-11-24T07:59:52.531393+0000","last_unstale":"2023-11-24T07:59:53.608618+0000","last_undegraded":"2023-11-24T07:59:53.608618+0000","last_fullsized":"2023-11-24T07:59:53.608618+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:59:51.477944+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:59:51.477944+0000","last_clean_scrub_stamp":"2023-11-24T07:59:51.477944+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T19:59:12.479564+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.39","version":"0'0","reported_seq":11,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-11-24T07:59:53.608610+0000","last_change":"2023-11-24T07:59:52.516615+0000","last_active":"2023-11-24T07:59:53.608610+0000","last_peered":"2023-11-24T07:59:53.608610+0000","last_clean":"2023-11-24T07:59:53.608610+0000","last_became_active":"2023-11-24T07:59:52.516230+0000","last_became_peered":"2023-11-24T07:59:52.516230+0000","last_unstale":"2023-11-24T07:59:53.608610+0000","last_undegraded":"2023-11-24T07:59:53.608610+0000","last_fullsized":"2023-11-24T07:59:53.608610+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:59:51.477944+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:59:51.477944+0000","last_clean_scrub_stamp":"2023-11-24T07:59:51.477944+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T17:43:33.689547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.38","version":"0'0","reported_seq":11,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-11-24T07:59:53.607999+0000","last_change":"2023-11-24T07:59:52.518308+0000","last_active":"2023-11-24T07:59:53.607999+0000","last_peered":"2023-11-24T07:59:53.607999+0000","last_clean":"2023-11-24T07:59:53.607999+0000","last_became_active":"2023-11-24T07:59:52.517481+0000","last_became_peered":"2023-11-24T07:59:52.517481+0000","last_unstale":"2023-11-24T07:59:53.607999+0000","last_undegraded":"2023-11-24T07:59:53.607999+0000","last_fullsized":"2023-11-24T07:59:53.607999+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:59:51.477944+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:59:51.477944+0000","last_clean_scrub_stamp":"2023-11-24T07:59:51.477944+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T13:03:18.643127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.37","version":"0'0","reported_seq":11,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-11-24T07:59:53.608855+0000","last_change":"2023-11-24T07:59:52.520229+0000","last_active":"2023-11-24T07:59:53.608855+0000","last_peered":"2023-11-24T07:59:53.608855+0000","last_clean":"2023-11-24T07:59:53.608855+0000","last_became_active":"2023-11-24T07:59:52.519949+0000","last_became_peered":"2023-11-24T07:59:52.519949+0000","last_unstale":"2023-11-24T07:59:53.608855+0000","last_undegraded":"2023-11-24T07:59:53.608855+0000","last_fullsized":"2023-11-24T07:59:53.608855+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:59:51.477944+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:59:51.477944+0000","last_clean_scrub_stamp":"2023-11-24T07:59:51.477944+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T11:25:48.888553+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.36","version":"0'0","reported_seq":11,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-11-24T07:59:53.609124+0000","last_change":"2023-11-24T07:59:52.530667+0000","last_active":"2023-11-24T07:59:53.609124+0000","last_peered":"2023-11-24T07:59:53.609124+0000","last_clean":"2023-11-24T07:59:53.609124+0000","last_became_active":"2023-11-24T07:59:52.530370+0000","last_became_peered":"2023-11-24T07:59:52.530370+0000","last_unstale":"2023-11-24T07:59:53.609124+0000","last_undegraded":"2023-11-24T07:59:53.609124+0000","last_fullsized":"2023-11-24T07:59:53.609124+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:59:51.477944+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:59:51.477944+0000","last_clean_scrub_stamp":"2023-11-24T07:59:51.477944+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T18:55:14.661533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.35","version":"0'0","reported_seq":11,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-11-24T07:59:53.608580+0000","last_change":"2023-11-24T07:59:52.532327+0000","last_active":"2023-11-24T07:59:53.608580+0000","last_peered":"2023-11-24T07:59:53.608580+0000","last_clean":"2023-11-24T07:59:53.608580+0000","last_became_active":"2023-11-24T07:59:52.532053+0000","last_became_peered":"2023-11-24T07:59:52.532053+0000","last_unstale":"2023-11-24T07:59:53.608580+0000","last_undegraded":"2023-11-24T07:59:53.608580+0000","last_fullsized":"2023-11-24T07:59:53.608580+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:59:51.477944+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:59:51.477944+0000","last_clean_scrub_stamp":"2023-11-24T07:59:51.477944+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:25:11.323450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.34","version":"0'0","reported_seq":11,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-11-24T07:59:53.609558+0000","last_change":"2023-11-24T07:59:52.520219+0000","last_active":"2023-11-24T07:59:53.609558+0000","last_peered":"2023-11-24T07:59:53.609558+0000","last_clean":"2023-11-24T07:59:53.609558+0000","last_became_active":"2023-11-24T07:59:52.519942+0000","last_became_peered":"2023-11-24T07:59:52.519942+0000","last_unstale":"2023-11-24T07:59:53.609558+0000","last_undegraded":"2023-11-24T07:59:53.609558+0000","last_fullsized":"2023-11-24T07:59:53.609558+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:59:51.477944+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:59:51.477944+0000","last_clean_scrub_stamp":"2023-11-24T07:59:51.477944+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T09:35:30.735874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.33","version":"0'0","reported_seq":11,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-11-24T07:59:53.505979+0000","last_change":"2023-11-24T07:59:52.535905+0000","last_active":"2023-11-24T07:59:53.505979+0000","last_peered":"2023-11-24T07:59:53.505979+0000","last_clean":"2023-11-24T07:59:53.505979+0000","last_became_active":"2023-11-24T07:59:52.535592+0000","last_became_peered":"2023-11-24T07:59:52.535592+0000","last_unstale":"2023-11-24T07:59:53.505979+0000","last_undegraded":"2023-11-24T07:59:53.505979+0000","last_fullsized":"2023-11-24T07:59:53.505979+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:59:51.477944+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:59:51.477944+0000","last_clean_scrub_stamp":"2023-11-24T07:59:51.477944+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T19:41:55.782652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.32","version":"0'0","reported_seq":11,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-11-24T07:59:53.505391+0000","last_change":"2023-11-24T07:59:52.518344+0000","last_active":"2023-11-24T07:59:53.505391+0000","last_peered":"2023-11-24T07:59:53.505391+0000","last_clean":"2023-11-24T07:59:53.505391+0000","last_became_active":"2023-11-24T07:59:52.518067+0000","last_became_peered":"2023-11-24T07:59:52.518067+0000","last_unstale":"2023-11-24T07:59:53.505391+0000","last_undegraded":"2023-11-24T07:59:53.505391+0000","last_fullsized":"2023-11-24T07:59:53.505391+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:59:51.477944+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:59:51.477944+0000","last_clean_scrub_stamp":"2023-11-24T07:59:51.477944+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T11:49:25.692175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.31","version":"0'0","reported_seq":11,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-11-24T07:59:53.608788+0000","last_change":"2023-11-24T07:59:52.509975+0000","last_active":"2023-11-24T07:59:53.608788+0000","last_peered":"2023-11-24T07:59:53.608788+0000","last_clean":"2023-11-24T07:59:53.608788+0000","last_became_active":"2023-11-24T07:59:52.509707+0000","last_became_peered":"2023-11-24T07:59:52.509707+0000","last_unstale":"2023-11-24T07:59:53.608788+0000","last_undegraded":"2023-11-24T07:59:53.608788+0000","last_fullsized":"2023-11-24T07:59:53.608788+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:59:51.477944+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:59:51.477944+0000","last_clean_scrub_stamp":"2023-11-24T07:59:51.477944+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T16:38:43.600527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.30","version":"0'0","reported_seq":11,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-11-24T07:59:53.608396+0000","last_change":"2023-11-24T07:59:52.530519+0000","last_active":"2023-11-24T07:59:53.608396+0000","last_peered":"2023-11-24T07:59:53.608396+0000","last_clean":"2023-11-24T07:59:53.608396+0000","last_became_active":"2023-11-24T07:59:52.530206+0000","last_became_peered":"2023-11-24T07:59:52.530206+0000","last_unstale":"2023-11-24T07:59:53.608396+0000","last_undegraded":"2023-11-24T07:59:53.608396+0000","last_fullsized":"2023-11-24T07:59:53.608396+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:59:51.477944+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:59:51.477944+0000","last_clean_scrub_stamp":"2023-11-24T07:59:51.477944+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T19:33:56.535985+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.2f","version":"0'0","reported_seq":11,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-11-24T07:59:53.610055+0000","last_change":"2023-11-24T07:59:52.520834+0000","last_active":"2023-11-24T07:59:53.610055+0000","last_peered":"2023-11-24T07:59:53.610055+0000","last_clean":"2023-11-24T07:59:53.610055+0000","last_became_active":"2023-11-24T07:59:52.520529+0000","last_became_peered":"2023-11-24T07:59:52.520529+0000","last_unstale":"2023-11-24T07:59:53.610055+0000","last_undegraded":"2023-11-24T07:59:53.610055+0000","last_fullsized":"2023-11-24T07:59:53.610055+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:59:51.477944+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:59:51.477944+0000","last_clean_scrub_stamp":"2023-11-24T07:59:51.477944+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T19:59:12.479564+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.2e","version":"0'0","reported_seq":11,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-11-24T07:59:53.609062+0000","last_change":"2023-11-24T07:59:52.523045+0000","last_active":"2023-11-24T07:59:53.609062+0000","last_peered":"2023-11-24T07:59:53.609062+0000","last_clean":"2023-11-24T07:59:53.609062+0000","last_became_active":"2023-11-24T07:59:52.522759+0000","last_became_peered":"2023-11-24T07:59:52.522759+0000","last_unstale":"2023-11-24T07:59:53.609062+0000","last_undegraded":"2023-11-24T07:59:53.609062+0000","last_fullsized":"2023-11-24T07:59:53.609062+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:59:51.477944+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:59:51.477944+0000","last_clean_scrub_stamp":"2023-11-24T07:59:51.477944+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T13:20:44.301407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.2d","version":"0'0","reported_seq":11,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-11-24T07:59:53.505727+0000","last_change":"2023-11-24T07:59:52.519290+0000","last_active":"2023-11-24T07:59:53.505727+0000","last_peered":"2023-11-24T07:59:53.505727+0000","last_clean":"2023-11-24T07:59:53.505727+0000","last_became_active":"2023-11-24T07:59:52.519007+0000","last_became_peered":"2023-11-24T07:59:52.519007+0000","last_unstale":"2023-11-24T07:59:53.505727+0000","last_undegraded":"2023-11-24T07:59:53.505727+0000","last_fullsized":"2023-11-24T07:59:53.505727+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:59:51.477944+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:59:51.477944+0000","last_clean_scrub_stamp":"2023-11-24T07:59:51.477944+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T15:52:11.017061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.2c","version":"0'0","reported_seq":11,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-11-24T07:59:53.505856+0000","last_change":"2023-11-24T07:59:52.518769+0000","last_active":"2023-11-24T07:59:53.505856+0000","last_peered":"2023-11-24T07:59:53.505856+0000","last_clean":"2023-11-24T07:59:53.505856+0000","last_became_active":"2023-11-24T07:59:52.518377+0000","last_became_peered":"2023-11-24T07:59:52.518377+0000","last_unstale":"2023-11-24T07:59:53.505856+0000","last_undegraded":"2023-11-24T07:59:53.505856+0000","last_fullsized":"2023-11-24T07:59:53.505856+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:59:51.477944+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:59:51.477944+0000","last_clean_scrub_stamp":"2023-11-24T07:59:51.477944+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T08:19:08.702386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.2b","version":"0'0","reported_seq":11,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-11-24T07:59:53.506840+0000","last_change":"2023-11-24T07:59:52.509772+0000","last_active":"2023-11-24T07:59:53.506840+0000","last_peered":"2023-11-24T07:59:53.506840+0000","last_clean":"2023-11-24T07:59:53.506840+0000","last_became_active":"2023-11-24T07:59:52.509458+0000","last_became_peered":"2023-11-24T07:59:52.509458+0000","last_unstale":"2023-11-24T07:59:53.506840+0000","last_undegraded":"2023-11-24T07:59:53.506840+0000","last_fullsized":"2023-11-24T07:59:53.506840+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:59:51.477944+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:59:51.477944+0000","last_clean_scrub_stamp":"2023-11-24T07:59:51.477944+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T15:50:40.414696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.2a","version":"0'0","reported_seq":11,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-11-24T07:59:53.505572+0000","last_change":"2023-11-24T07:59:52.518858+0000","last_active":"2023-11-24T07:59:53.505572+0000","last_peered":"2023-11-24T07:59:53.505572+0000","last_clean":"2023-11-24T07:59:53.505572+0000","last_became_active":"2023-11-24T07:59:52.518579+0000","last_became_peered":"2023-11-24T07:59:52.518579+0000","last_unstale":"2023-11-24T07:59:53.505572+0000","last_undegraded":"2023-11-24T07:59:53.505572+0000","last_fullsized":"2023-11-24T07:59:53.505572+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:59:51.477944+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:59:51.477944+0000","last_clean_scrub_stamp":"2023-11-24T07:59:51.477944+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T14:16:27.359759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.29","version":"0'0","reported_seq":11,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-11-24T07:59:53.609416+0000","last_change":"2023-11-24T07:59:52.522726+0000","last_active":"2023-11-24T07:59:53.609416+0000","last_peered":"2023-11-24T07:59:53.609416+0000","last_clean":"2023-11-24T07:59:53.609416+0000","last_became_active":"2023-11-24T07:59:52.522310+0000","last_became_peered":"2023-11-24T07:59:52.522310+0000","last_unstale":"2023-11-24T07:59:53.609416+0000","last_undegraded":"2023-11-24T07:59:53.609416+0000","last_fullsized":"2023-11-24T07:59:53.609416+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:59:51.477944+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:59:51.477944+0000","last_clean_scrub_stamp":"2023-11-24T07:59:51.477944+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T17:45:56.521663+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.28","version":"0'0","reported_seq":11,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-11-24T07:59:53.608491+0000","last_change":"2023-11-24T07:59:52.515597+0000","last_active":"2023-11-24T07:59:53.608491+0000","last_peered":"2023-11-24T07:59:53.608491+0000","last_clean":"2023-11-24T07:59:53.608491+0000","last_became_active":"2023-11-24T07:59:52.515247+0000","last_became_peered":"2023-11-24T07:59:52.515247+0000","last_unstale":"2023-11-24T07:59:53.608491+0000","last_undegraded":"2023-11-24T07:59:53.608491+0000","last_fullsized":"2023-11-24T07:59:53.608491+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:59:51.477944+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:59:51.477944+0000","last_clean_scrub_stamp":"2023-11-24T07:59:51.477944+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T15:14:48.860422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.27","version":"0'0","reported_seq":11,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-11-24T07:59:53.608968+0000","last_change":"2023-11-24T07:59:52.536629+0000","last_active":"2023-11-24T07:59:53.608968+0000","last_peered":"2023-11-24T07:59:53.608968+0000","last_clean":"2023-11-24T07:59:53.608968+0000","last_became_active":"2023-11-24T07:59:52.535822+0000","last_became_peered":"2023-11-24T07:59:52.535822+0000","last_unstale":"2023-11-24T07:59:53.608968+0000","last_undegraded":"2023-11-24T07:59:53.608968+0000","last_fullsized":"2023-11-24T07:59:53.608968+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:59:51.477944+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:59:51.477944+0000","last_clean_scrub_stamp":"2023-11-24T07:59:51.477944+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T19:07:22.502426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.26","version":"0'0","reported_seq":11,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-11-24T07:59:53.506206+0000","last_change":"2023-11-24T07:59:52.521004+0000","last_active":"2023-11-24T07:59:53.506206+0000","last_peered":"2023-11-24T07:59:53.506206+0000","last_clean":"2023-11-24T07:59:53.506206+0000","last_became_active":"2023-11-24T07:59:52.520546+0000","last_became_peered":"2023-11-24T07:59:52.520546+0000","last_unstale":"2023-11-24T07:59:53.506206+0000","last_undegraded":"2023-11-24T07:59:53.506206+0000","last_fullsized":"2023-11-24T07:59:53.506206+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:59:51.477944+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:59:51.477944+0000","last_clean_scrub_stamp":"2023-11-24T07:59:51.477944+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T15:25:25.968856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.25","version":"0'0","reported_seq":11,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-11-24T07:59:53.506158+0000","last_change":"2023-11-24T07:59:52.513690+0000","last_active":"2023-11-24T07:59:53.506158+0000","last_peered":"2023-11-24T07:59:53.506158+0000","last_clean":"2023-11-24T07:59:53.506158+0000","last_became_active":"2023-11-24T07:59:52.513378+0000","last_became_peered":"2023-11-24T07:59:52.513378+0000","last_unstale":"2023-11-24T07:59:53.506158+0000","last_undegraded":"2023-11-24T07:59:53.506158+0000","last_fullsized":"2023-11-24T07:59:53.506158+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:59:51.477944+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:59:51.477944+0000","last_clean_scrub_stamp":"2023-11-24T07:59:51.477944+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T12:11:43.705658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.24","version":"0'0","reported_seq":11,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-11-24T07:59:53.608136+0000","last_change":"2023-11-24T07:59:52.525922+0000","last_active":"2023-11-24T07:59:53.608136+0000","last_peered":"2023-11-24T07:59:53.608136+0000","last_clean":"2023-11-24T07:59:53.608136+0000","last_became_active":"2023-11-24T07:59:52.525654+0000","last_became_peered":"2023-11-24T07:59:52.525654+0000","last_unstale":"2023-11-24T07:59:53.608136+0000","last_undegraded":"2023-11-24T07:59:53.608136+0000","last_fullsized":"2023-11-24T07:59:53.608136+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:59:51.477944+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:59:51.477944+0000","last_clean_scrub_stamp":"2023-11-24T07:59:51.477944+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T08:10:30.910525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.23","version":"0'0","reported_seq":11,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-11-24T07:59:53.506690+0000","last_change":"2023-11-24T07:59:52.510412+0000","last_active":"2023-11-24T07:59:53.506690+0000","last_peered":"2023-11-24T07:59:53.506690+0000","last_clean":"2023-11-24T07:59:53.506690+0000","last_became_active":"2023-11-24T07:59:52.510126+0000","last_became_peered":"2023-11-24T07:59:52.510126+0000","last_unstale":"2023-11-24T07:59:53.506690+0000","last_undegraded":"2023-11-24T07:59:53.506690+0000","last_fullsized":"2023-11-24T07:59:53.506690+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:59:51.477944+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:59:51.477944+0000","last_clean_scrub_stamp":"2023-11-24T07:59:51.477944+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T15:40:18.171862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.22","version":"0'0","reported_seq":11,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-11-24T07:59:53.505766+0000","last_change":"2023-11-24T07:59:52.518764+0000","last_active":"2023-11-24T07:59:53.505766+0000","last_peered":"2023-11-24T07:59:53.505766+0000","last_clean":"2023-11-24T07:59:53.505766+0000","last_became_active":"2023-11-24T07:59:52.518431+0000","last_became_peered":"2023-11-24T07:59:52.518431+0000","last_unstale":"2023-11-24T07:59:53.505766+0000","last_undegraded":"2023-11-24T07:59:53.505766+0000","last_fullsized":"2023-11-24T07:59:53.505766+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:59:51.477944+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:59:51.477944+0000","last_clean_scrub_stamp":"2023-11-24T07:59:51.477944+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T08:25:07.421503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.21","version":"0'0","reported_seq":11,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-11-24T07:59:53.608233+0000","last_change":"2023-11-24T07:59:52.531404+0000","last_active":"2023-11-24T07:59:53.608233+0000","last_peered":"2023-11-24T07:59:53.608233+0000","last_clean":"2023-11-24T07:59:53.608233+0000","last_became_active":"2023-11-24T07:59:52.531032+0000","last_became_peered":"2023-11-24T07:59:52.531032+0000","last_unstale":"2023-11-24T07:59:53.608233+0000","last_undegraded":"2023-11-24T07:59:53.608233+0000","last_fullsized":"2023-11-24T07:59:53.608233+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:59:51.477944+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:59:51.477944+0000","last_clean_scrub_stamp":"2023-11-24T07:59:51.477944+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T09:35:30.735874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.20","version":"0'0","reported_seq":11,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-11-24T07:59:53.608451+0000","last_change":"2023-11-24T07:59:52.530888+0000","last_active":"2023-11-24T07:59:53.608451+0000","last_peered":"2023-11-24T07:59:53.608451+0000","last_clean":"2023-11-24T07:59:53.608451+0000","last_became_active":"2023-11-24T07:59:52.530528+0000","last_became_peered":"2023-11-24T07:59:52.530528+0000","last_unstale":"2023-11-24T07:59:53.608451+0000","last_undegraded":"2023-11-24T07:59:53.608451+0000","last_fullsized":"2023-11-24T07:59:53.608451+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:59:51.477944+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:59:51.477944+0000","last_clean_scrub_stamp":"2023-11-24T07:59:51.477944+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T16:20:03.228953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.1e","version":"0'0","reported_seq":11,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-11-24T07:59:53.609869+0000","last_change":"2023-11-24T07:59:52.525254+0000","last_active":"2023-11-24T07:59:53.609869+0000","last_peered":"2023-11-24T07:59:53.609869+0000","last_clean":"2023-11-24T07:59:53.609869+0000","last_became_active":"2023-11-24T07:59:52.524955+0000","last_became_peered":"2023-11-24T07:59:52.524955+0000","last_unstale":"2023-11-24T07:59:53.609869+0000","last_undegraded":"2023-11-24T07:59:53.609869+0000","last_fullsized":"2023-11-24T07:59:53.609869+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:59:51.477944+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:59:51.477944+0000","last_clean_scrub_stamp":"2023-11-24T07:59:51.477944+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T17:27:50.274990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.1f","version":"86'4","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-11-24T07:59:53.373372+0000","last_change":"2023-11-24T07:59:51.501301+0000","last_active":"2023-11-24T07:59:53.373372+0000","last_peered":"2023-11-24T07:59:53.373372+0000","last_clean":"2023-11-24T07:59:53.373372+0000","last_became_active":"2023-11-24T07:59:51.500984+0000","last_became_peered":"2023-11-24T07:59:51.500984+0000","last_unstale":"2023-11-24T07:59:53.373372+0000","last_undegraded":"2023-11-24T07:59:53.373372+0000","last_fullsized":"2023-11-24T07:59:53.373372+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:59:50.473699+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:59:50.473699+0000","last_clean_scrub_stamp":"2023-11-24T07:59:50.473699+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T19:08:46.605605+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.1f","version":"0'0","reported_seq":11,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-11-24T07:59:53.608973+0000","last_change":"2023-11-24T07:59:52.533153+0000","last_active":"2023-11-24T07:59:53.608973+0000","last_peered":"2023-11-24T07:59:53.608973+0000","last_clean":"2023-11-24T07:59:53.608973+0000","last_became_active":"2023-11-24T07:59:52.532666+0000","last_became_peered":"2023-11-24T07:59:52.532666+0000","last_unstale":"2023-11-24T07:59:53.608973+0000","last_undegraded":"2023-11-24T07:59:53.608973+0000","last_fullsized":"2023-11-24T07:59:53.608973+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:59:51.477944+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:59:51.477944+0000","last_clean_scrub_stamp":"2023-11-24T07:59:51.477944+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T15:14:48.860422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.1e","version":"86'1","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-11-24T07:59:53.367164+0000","last_change":"2023-11-24T07:59:51.500301+0000","last_active":"2023-11-24T07:59:53.367164+0000","last_peered":"2023-11-24T07:59:53.367164+0000","last_clean":"2023-11-24T07:59:53.367164+0000","last_became_active":"2023-11-24T07:59:51.499985+0000","last_became_peered":"2023-11-24T07:59:51.499985+0000","last_unstale":"2023-11-24T07:59:53.367164+0000","last_undegraded":"2023-11-24T07:59:53.367164+0000","last_fullsized":"2023-11-24T07:59:53.367164+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:59:50.473699+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:59:50.473699+0000","last_clean_scrub_stamp":"2023-11-24T07:59:50.473699+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T17:45:55.517418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.1c","version":"0'0","reported_seq":11,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-11-24T07:59:53.609568+0000","last_change":"2023-11-24T07:59:52.517822+0000","last_active":"2023-11-24T07:59:53.609568+0000","last_peered":"2023-11-24T07:59:53.609568+0000","last_clean":"2023-11-24T07:59:53.609568+0000","last_became_active":"2023-11-24T07:59:52.517510+0000","last_became_peered":"2023-11-24T07:59:52.517510+0000","last_unstale":"2023-11-24T07:59:53.609568+0000","last_undegraded":"2023-11-24T07:59:53.609568+0000","last_fullsized":"2023-11-24T07:59:53.609568+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:59:51.477944+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:59:51.477944+0000","last_clean_scrub_stamp":"2023-11-24T07:59:51.477944+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T13:17:26.610425+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.1d","version":"86'2","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-11-24T07:59:53.373128+0000","last_change":"2023-11-24T07:59:51.500384+0000","last_active":"2023-11-24T07:59:53.373128+0000","last_peered":"2023-11-24T07:59:53.373128+0000","last_clean":"2023-11-24T07:59:53.373128+0000","last_became_active":"2023-11-24T07:59:51.500044+0000","last_became_peered":"2023-11-24T07:59:51.500044+0000","last_unstale":"2023-11-24T07:59:53.373128+0000","last_undegraded":"2023-11-24T07:59:53.373128+0000","last_fullsized":"2023-11-24T07:59:53.373128+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:59:50.473699+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:59:50.473699+0000","last_clean_scrub_stamp":"2023-11-24T07:59:50.473699+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T08:10:29.906280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.1d","version":"0'0","reported_seq":11,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-11-24T07:59:53.608370+0000","last_change":"2023-11-24T07:59:52.527299+0000","last_active":"2023-11-24T07:59:53.608370+0000","last_peered":"2023-11-24T07:59:53.608370+0000","last_clean":"2023-11-24T07:59:53.608370+0000","last_became_active":"2023-11-24T07:59:52.523777+0000","last_became_peered":"2023-11-24T07:59:52.523777+0000","last_unstale":"2023-11-24T07:59:53.608370+0000","last_undegraded":"2023-11-24T07:59:53.608370+0000","last_fullsized":"2023-11-24T07:59:53.608370+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:59:51.477944+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:59:51.477944+0000","last_clean_scrub_stamp":"2023-11-24T07:59:51.477944+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T17:43:33.689547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.1c","version":"86'2","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-11-24T07:59:53.374704+0000","last_change":"2023-11-24T07:59:51.504319+0000","last_active":"2023-11-24T07:59:53.374704+0000","last_peered":"2023-11-24T07:59:53.374704+0000","last_clean":"2023-11-24T07:59:53.374704+0000","last_became_active":"2023-11-24T07:59:51.503943+0000","last_became_peered":"2023-11-24T07:59:51.503943+0000","last_unstale":"2023-11-24T07:59:53.374704+0000","last_undegraded":"2023-11-24T07:59:53.374704+0000","last_fullsized":"2023-11-24T07:59:53.374704+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:59:50.473699+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:59:50.473699+0000","last_clean_scrub_stamp":"2023-11-24T07:59:50.473699+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T11:53:53.251329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.1a","version":"0'0","reported_seq":11,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-11-24T07:59:53.507108+0000","last_change":"2023-11-24T07:59:52.515179+0000","last_active":"2023-11-24T07:59:53.507108+0000","last_peered":"2023-11-24T07:59:53.507108+0000","last_clean":"2023-11-24T07:59:53.507108+0000","last_became_active":"2023-11-24T07:59:52.514892+0000","last_became_peered":"2023-11-24T07:59:52.514892+0000","last_unstale":"2023-11-24T07:59:53.507108+0000","last_undegraded":"2023-11-24T07:59:53.507108+0000","last_fullsized":"2023-11-24T07:59:53.507108+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:59:51.477944+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:59:51.477944+0000","last_clean_scrub_stamp":"2023-11-24T07:59:51.477944+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:57:55.850876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.1b","version":"86'1","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-11-24T07:59:53.366988+0000","last_change":"2023-11-24T07:59:51.500410+0000","last_active":"2023-11-24T07:59:53.366988+0000","last_peered":"2023-11-24T07:59:53.366988+0000","last_clean":"2023-11-24T07:59:53.366988+0000","last_became_active":"2023-11-24T07:59:51.500029+0000","last_became_peered":"2023-11-24T07:59:51.500029+0000","last_unstale":"2023-11-24T07:59:53.366988+0000","last_undegraded":"2023-11-24T07:59:53.366988+0000","last_fullsized":"2023-11-24T07:59:53.366988+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:59:50.473699+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:59:50.473699+0000","last_clean_scrub_stamp":"2023-11-24T07:59:50.473699+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T15:52:33.660947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.1b","version":"0'0","reported_seq":11,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-11-24T07:59:53.608579+0000","last_change":"2023-11-24T07:59:52.530658+0000","last_active":"2023-11-24T07:59:53.608579+0000","last_peered":"2023-11-24T07:59:53.608579+0000","last_clean":"2023-11-24T07:59:53.608579+0000","last_became_active":"2023-11-24T07:59:52.530357+0000","last_became_peered":"2023-11-24T07:59:52.530357+0000","last_unstale":"2023-11-24T07:59:53.608579+0000","last_undegraded":"2023-11-24T07:59:53.608579+0000","last_fullsized":"2023-11-24T07:59:53.608579+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:59:51.477944+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:59:51.477944+0000","last_clean_scrub_stamp":"2023-11-24T07:59:51.477944+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T17:27:50.274990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.1a","version":"86'1","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-11-24T07:59:53.367044+0000","last_change":"2023-11-24T07:59:51.496684+0000","last_active":"2023-11-24T07:59:53.367044+0000","last_peered":"2023-11-24T07:59:53.367044+0000","last_clean":"2023-11-24T07:59:53.367044+0000","last_became_active":"2023-11-24T07:59:51.496233+0000","last_became_peered":"2023-11-24T07:59:51.496233+0000","last_unstale":"2023-11-24T07:59:53.367044+0000","last_undegraded":"2023-11-24T07:59:53.367044+0000","last_fullsized":"2023-11-24T07:59:53.367044+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:59:50.473699+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:59:50.473699+0000","last_clean_scrub_stamp":"2023-11-24T07:59:50.473699+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T13:17:25.606179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.18","version":"0'0","reported_seq":11,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-11-24T07:59:53.607889+0000","last_change":"2023-11-24T07:59:52.521426+0000","last_active":"2023-11-24T07:59:53.607889+0000","last_peered":"2023-11-24T07:59:53.607889+0000","last_clean":"2023-11-24T07:59:53.607889+0000","last_became_active":"2023-11-24T07:59:52.521064+0000","last_became_peered":"2023-11-24T07:59:52.521064+0000","last_unstale":"2023-11-24T07:59:53.607889+0000","last_undegraded":"2023-11-24T07:59:53.607889+0000","last_fullsized":"2023-11-24T07:59:53.607889+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:59:51.477944+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:59:51.477944+0000","last_clean_scrub_stamp":"2023-11-24T07:59:51.477944+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T08:19:08.702386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.19","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-11-24T07:59:53.370563+0000","last_change":"2023-11-24T07:59:51.502385+0000","last_active":"2023-11-24T07:59:53.370563+0000","last_peered":"2023-11-24T07:59:53.370563+0000","last_clean":"2023-11-24T07:59:53.370563+0000","last_became_active":"2023-11-24T07:59:51.502112+0000","last_became_peered":"2023-11-24T07:59:51.502112+0000","last_unstale":"2023-11-24T07:59:53.370563+0000","last_undegraded":"2023-11-24T07:59:53.370563+0000","last_fullsized":"2023-11-24T07:59:53.370563+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:59:50.473699+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:59:50.473699+0000","last_clean_scrub_stamp":"2023-11-24T07:59:50.473699+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T18:23:25.376630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.19","version":"0'0","reported_seq":11,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-11-24T07:59:53.507468+0000","last_change":"2023-11-24T07:59:52.524304+0000","last_active":"2023-11-24T07:59:53.507468+0000","last_peered":"2023-11-24T07:59:53.507468+0000","last_clean":"2023-11-24T07:59:53.507468+0000","last_became_active":"2023-11-24T07:59:52.524057+0000","last_became_peered":"2023-11-24T07:59:52.524057+0000","last_unstale":"2023-11-24T07:59:53.507468+0000","last_undegraded":"2023-11-24T07:59:53.507468+0000","last_fullsized":"2023-11-24T07:59:53.507468+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:59:51.477944+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:59:51.477944+0000","last_clean_scrub_stamp":"2023-11-24T07:59:51.477944+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T15:30:01.455976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.18","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-11-24T07:59:53.370527+0000","last_change":"2023-11-24T07:59:51.495208+0000","last_active":"2023-11-24T07:59:53.370527+0000","last_peered":"2023-11-24T07:59:53.370527+0000","last_clean":"2023-11-24T07:59:53.370527+0000","last_became_active":"2023-11-24T07:59:51.494958+0000","last_became_peered":"2023-11-24T07:59:51.494958+0000","last_unstale":"2023-11-24T07:59:53.370527+0000","last_undegraded":"2023-11-24T07:59:53.370527+0000","last_fullsized":"2023-11-24T07:59:53.370527+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:59:50.473699+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:59:50.473699+0000","last_clean_scrub_stamp":"2023-11-24T07:59:50.473699+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T13:00:50.971622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.16","version":"0'0","reported_seq":11,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-11-24T07:59:53.505638+0000","last_change":"2023-11-24T07:59:52.527538+0000","last_active":"2023-11-24T07:59:53.505638+0000","last_peered":"2023-11-24T07:59:53.505638+0000","last_clean":"2023-11-24T07:59:53.505638+0000","last_became_active":"2023-11-24T07:59:52.527111+0000","last_became_peered":"2023-11-24T07:59:52.527111+0000","last_unstale":"2023-11-24T07:59:53.505638+0000","last_undegraded":"2023-11-24T07:59:53.505638+0000","last_fullsized":"2023-11-24T07:59:53.505638+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:59:51.477944+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:59:51.477944+0000","last_clean_scrub_stamp":"2023-11-24T07:59:51.477944+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T11:02:28.096472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.17","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-11-24T07:59:53.377035+0000","last_change":"2023-11-24T07:59:51.500327+0000","last_active":"2023-11-24T07:59:53.377035+0000","last_peered":"2023-11-24T07:59:53.377035+0000","last_clean":"2023-11-24T07:59:53.377035+0000","last_became_active":"2023-11-24T07:59:51.500042+0000","last_became_peered":"2023-11-24T07:59:51.500042+0000","last_unstale":"2023-11-24T07:59:53.377035+0000","last_undegraded":"2023-11-24T07:59:53.377035+0000","last_fullsized":"2023-11-24T07:59:53.377035+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:59:50.473699+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:59:50.473699+0000","last_clean_scrub_stamp":"2023-11-24T07:59:50.473699+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T15:52:10.012815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.17","version":"0'0","reported_seq":11,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-11-24T07:59:53.609101+0000","last_change":"2023-11-24T07:59:52.519325+0000","last_active":"2023-11-24T07:59:53.609101+0000","last_peered":"2023-11-24T07:59:53.609101+0000","last_clean":"2023-11-24T07:59:53.609101+0000","last_became_active":"2023-11-24T07:59:52.519012+0000","last_became_peered":"2023-11-24T07:59:52.519012+0000","last_unstale":"2023-11-24T07:59:53.609101+0000","last_undegraded":"2023-11-24T07:59:53.609101+0000","last_fullsized":"2023-11-24T07:59:53.609101+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:59:51.477944+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:59:51.477944+0000","last_clean_scrub_stamp":"2023-11-24T07:59:51.477944+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:25:11.323450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.16","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-11-24T07:59:53.367176+0000","last_change":"2023-11-24T07:59:51.492433+0000","last_active":"2023-11-24T07:59:53.367176+0000","last_peered":"2023-11-24T07:59:53.367176+0000","last_clean":"2023-11-24T07:59:53.367176+0000","last_became_active":"2023-11-24T07:59:51.492145+0000","last_became_peered":"2023-11-24T07:59:51.492145+0000","last_unstale":"2023-11-24T07:59:53.367176+0000","last_undegraded":"2023-11-24T07:59:53.367176+0000","last_fullsized":"2023-11-24T07:59:53.367176+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:59:50.473699+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:59:50.473699+0000","last_clean_scrub_stamp":"2023-11-24T07:59:50.473699+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T13:17:19.649925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.14","version":"0'0","reported_seq":11,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-11-24T07:59:53.608453+0000","last_change":"2023-11-24T07:59:52.526145+0000","last_active":"2023-11-24T07:59:53.608453+0000","last_peered":"2023-11-24T07:59:53.608453+0000","last_clean":"2023-11-24T07:59:53.608453+0000","last_became_active":"2023-11-24T07:59:52.525892+0000","last_became_peered":"2023-11-24T07:59:52.525892+0000","last_unstale":"2023-11-24T07:59:53.608453+0000","last_undegraded":"2023-11-24T07:59:53.608453+0000","last_fullsized":"2023-11-24T07:59:53.608453+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:59:51.477944+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:59:51.477944+0000","last_clean_scrub_stamp":"2023-11-24T07:59:51.477944+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T16:26:49.386477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.15","version":"86'3","reported_seq":15,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-11-24T07:59:53.376936+0000","last_change":"2023-11-24T07:59:51.502200+0000","last_active":"2023-11-24T07:59:53.376936+0000","last_peered":"2023-11-24T07:59:53.376936+0000","last_clean":"2023-11-24T07:59:53.376936+0000","last_became_active":"2023-11-24T07:59:51.501924+0000","last_became_peered":"2023-11-24T07:59:51.501924+0000","last_unstale":"2023-11-24T07:59:53.376936+0000","last_undegraded":"2023-11-24T07:59:53.376936+0000","last_fullsized":"2023-11-24T07:59:53.376936+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:59:50.473699+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:59:50.473699+0000","last_clean_scrub_stamp":"2023-11-24T07:59:50.473699+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T11:02:27.092227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.15","version":"0'0","reported_seq":11,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-11-24T07:59:53.506323+0000","last_change":"2023-11-24T07:59:52.514787+0000","last_active":"2023-11-24T07:59:53.506323+0000","last_peered":"2023-11-24T07:59:53.506323+0000","last_clean":"2023-11-24T07:59:53.506323+0000","last_became_active":"2023-11-24T07:59:52.514493+0000","last_became_peered":"2023-11-24T07:59:52.514493+0000","last_unstale":"2023-11-24T07:59:53.506323+0000","last_undegraded":"2023-11-24T07:59:53.506323+0000","last_fullsized":"2023-11-24T07:59:53.506323+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:59:51.477944+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:59:51.477944+0000","last_clean_scrub_stamp":"2023-11-24T07:59:51.477944+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T08:34:28.451170+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.14","version":"86'6","reported_seq":18,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-11-24T07:59:53.367026+0000","last_change":"2023-11-24T07:59:51.496195+0000","last_active":"2023-11-24T07:59:53.367026+0000","last_peered":"2023-11-24T07:59:53.367026+0000","last_clean":"2023-11-24T07:59:53.367026+0000","last_became_active":"2023-11-24T07:59:51.495930+0000","last_became_peered":"2023-11-24T07:59:51.495930+0000","last_unstale":"2023-11-24T07:59:53.367026+0000","last_undegraded":"2023-11-24T07:59:53.367026+0000","last_fullsized":"2023-11-24T07:59:53.367026+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:59:50.473699+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:59:50.473699+0000","last_clean_scrub_stamp":"2023-11-24T07:59:50.473699+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:09:45.168881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.12","version":"0'0","reported_seq":11,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-11-24T07:59:53.608994+0000","last_change":"2023-11-24T07:59:52.522452+0000","last_active":"2023-11-24T07:59:53.608994+0000","last_peered":"2023-11-24T07:59:53.608994+0000","last_clean":"2023-11-24T07:59:53.608994+0000","last_became_active":"2023-11-24T07:59:52.522173+0000","last_became_peered":"2023-11-24T07:59:52.522173+0000","last_unstale":"2023-11-24T07:59:53.608994+0000","last_undegraded":"2023-11-24T07:59:53.608994+0000","last_fullsized":"2023-11-24T07:59:53.608994+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:59:51.477944+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:59:51.477944+0000","last_clean_scrub_stamp":"2023-11-24T07:59:51.477944+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T15:17:29.156978+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.13","version":"86'2","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-11-24T07:59:53.374661+0000","last_change":"2023-11-24T07:59:51.499029+0000","last_active":"2023-11-24T07:59:53.374661+0000","last_peered":"2023-11-24T07:59:53.374661+0000","last_clean":"2023-11-24T07:59:53.374661+0000","last_became_active":"2023-11-24T07:59:51.498746+0000","last_became_peered":"2023-11-24T07:59:51.498746+0000","last_unstale":"2023-11-24T07:59:53.374661+0000","last_undegraded":"2023-11-24T07:59:53.374661+0000","last_fullsized":"2023-11-24T07:59:53.374661+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:59:50.473699+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:59:50.473699+0000","last_clean_scrub_stamp":"2023-11-24T07:59:50.473699+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T11:36:02.086046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.13","version":"0'0","reported_seq":11,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-11-24T07:59:53.608312+0000","last_change":"2023-11-24T07:59:52.519427+0000","last_active":"2023-11-24T07:59:53.608312+0000","last_peered":"2023-11-24T07:59:53.608312+0000","last_clean":"2023-11-24T07:59:53.608312+0000","last_became_active":"2023-11-24T07:59:52.518944+0000","last_became_peered":"2023-11-24T07:59:52.518944+0000","last_unstale":"2023-11-24T07:59:53.608312+0000","last_undegraded":"2023-11-24T07:59:53.608312+0000","last_fullsized":"2023-11-24T07:59:53.608312+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:59:51.477944+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:59:51.477944+0000","last_clean_scrub_stamp":"2023-11-24T07:59:51.477944+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T15:37:22.418541+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.12","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-11-24T07:59:53.373001+0000","last_change":"2023-11-24T07:59:51.503076+0000","last_active":"2023-11-24T07:59:53.373001+0000","last_peered":"2023-11-24T07:59:53.373001+0000","last_clean":"2023-11-24T07:59:53.373001+0000","last_became_active":"2023-11-24T07:59:51.502784+0000","last_became_peered":"2023-11-24T07:59:51.502784+0000","last_unstale":"2023-11-24T07:59:53.373001+0000","last_undegraded":"2023-11-24T07:59:53.373001+0000","last_fullsized":"2023-11-24T07:59:53.373001+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:59:50.473699+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:59:50.473699+0000","last_clean_scrub_stamp":"2023-11-24T07:59:50.473699+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T08:19:07.698141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.10","version":"0'0","reported_seq":11,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-11-24T07:59:53.505332+0000","last_change":"2023-11-24T07:59:52.518212+0000","last_active":"2023-11-24T07:59:53.505332+0000","last_peered":"2023-11-24T07:59:53.505332+0000","last_clean":"2023-11-24T07:59:53.505332+0000","last_became_active":"2023-11-24T07:59:52.517869+0000","last_became_peered":"2023-11-24T07:59:52.517869+0000","last_unstale":"2023-11-24T07:59:53.505332+0000","last_undegraded":"2023-11-24T07:59:53.505332+0000","last_fullsized":"2023-11-24T07:59:53.505332+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:59:51.477944+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:59:51.477944+0000","last_clean_scrub_stamp":"2023-11-24T07:59:51.477944+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T08:13:59.445242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.11","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-11-24T07:59:53.366997+0000","last_change":"2023-11-24T07:59:51.501760+0000","last_active":"2023-11-24T07:59:53.366997+0000","last_peered":"2023-11-24T07:59:53.366997+0000","last_clean":"2023-11-24T07:59:53.366997+0000","last_became_active":"2023-11-24T07:59:51.501505+0000","last_became_peered":"2023-11-24T07:59:51.501505+0000","last_unstale":"2023-11-24T07:59:53.366997+0000","last_undegraded":"2023-11-24T07:59:53.366997+0000","last_fullsized":"2023-11-24T07:59:53.366997+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:59:50.473699+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:59:50.473699+0000","last_clean_scrub_stamp":"2023-11-24T07:59:50.473699+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T15:17:28.152732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.11","version":"0'0","reported_seq":11,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-11-24T07:59:53.608423+0000","last_change":"2023-11-24T07:59:52.515933+0000","last_active":"2023-11-24T07:59:53.608423+0000","last_peered":"2023-11-24T07:59:53.608423+0000","last_clean":"2023-11-24T07:59:53.608423+0000","last_became_active":"2023-11-24T07:59:52.513820+0000","last_became_peered":"2023-11-24T07:59:52.513820+0000","last_unstale":"2023-11-24T07:59:53.608423+0000","last_undegraded":"2023-11-24T07:59:53.608423+0000","last_fullsized":"2023-11-24T07:59:53.608423+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:59:51.477944+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:59:51.477944+0000","last_clean_scrub_stamp":"2023-11-24T07:59:51.477944+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T11:25:48.888553+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.10","version":"86'8","reported_seq":20,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-11-24T07:59:53.372959+0000","last_change":"2023-11-24T07:59:51.503492+0000","last_active":"2023-11-24T07:59:53.372959+0000","last_peered":"2023-11-24T07:59:53.372959+0000","last_clean":"2023-11-24T07:59:53.372959+0000","last_became_active":"2023-11-24T07:59:51.502903+0000","last_became_peered":"2023-11-24T07:59:51.502903+0000","last_unstale":"2023-11-24T07:59:53.372959+0000","last_undegraded":"2023-11-24T07:59:53.372959+0000","last_fullsized":"2023-11-24T07:59:53.372959+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:59:50.473699+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:59:50.473699+0000","last_clean_scrub_stamp":"2023-11-24T07:59:50.473699+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T16:38:42.596281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.e","version":"0'0","reported_seq":11,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-11-24T07:59:53.609794+0000","last_change":"2023-11-24T07:59:52.521956+0000","last_active":"2023-11-24T07:59:53.609794+0000","last_peered":"2023-11-24T07:59:53.609794+0000","last_clean":"2023-11-24T07:59:53.609794+0000","last_became_active":"2023-11-24T07:59:52.521664+0000","last_became_peered":"2023-11-24T07:59:52.521664+0000","last_unstale":"2023-11-24T07:59:53.609794+0000","last_undegraded":"2023-11-24T07:59:53.609794+0000","last_fullsized":"2023-11-24T07:59:53.609794+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:59:51.477944+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:59:51.477944+0000","last_clean_scrub_stamp":"2023-11-24T07:59:51.477944+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T16:20:03.228953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.f","version":"86'1","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-11-24T07:59:53.373414+0000","last_change":"2023-11-24T07:59:51.503493+0000","last_active":"2023-11-24T07:59:53.373414+0000","last_peered":"2023-11-24T07:59:53.373414+0000","last_clean":"2023-11-24T07:59:53.373414+0000","last_became_active":"2023-11-24T07:59:51.502675+0000","last_became_peered":"2023-11-24T07:59:51.502675+0000","last_unstale":"2023-11-24T07:59:53.373414+0000","last_undegraded":"2023-11-24T07:59:53.373414+0000","last_fullsized":"2023-11-24T07:59:53.373414+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:59:50.473699+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:59:50.473699+0000","last_clean_scrub_stamp":"2023-11-24T07:59:50.473699+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T17:47:41.894391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.f","version":"0'0","reported_seq":11,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-11-24T07:59:53.608741+0000","last_change":"2023-11-24T07:59:52.532100+0000","last_active":"2023-11-24T07:59:53.608741+0000","last_peered":"2023-11-24T07:59:53.608741+0000","last_clean":"2023-11-24T07:59:53.608741+0000","last_became_active":"2023-11-24T07:59:52.531613+0000","last_became_peered":"2023-11-24T07:59:52.531613+0000","last_unstale":"2023-11-24T07:59:53.608741+0000","last_undegraded":"2023-11-24T07:59:53.608741+0000","last_fullsized":"2023-11-24T07:59:53.608741+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:59:51.477944+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:59:51.477944+0000","last_clean_scrub_stamp":"2023-11-24T07:59:51.477944+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T16:39:20.453395+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.e","version":"86'2","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-11-24T07:59:53.370697+0000","last_change":"2023-11-24T07:59:51.505119+0000","last_active":"2023-11-24T07:59:53.370697+0000","last_peered":"2023-11-24T07:59:53.370697+0000","last_clean":"2023-11-24T07:59:53.370697+0000","last_became_active":"2023-11-24T07:59:51.504803+0000","last_became_peered":"2023-11-24T07:59:51.504803+0000","last_unstale":"2023-11-24T07:59:53.370697+0000","last_undegraded":"2023-11-24T07:59:53.370697+0000","last_fullsized":"2023-11-24T07:59:53.370697+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:59:50.473699+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:59:50.473699+0000","last_clean_scrub_stamp":"2023-11-24T07:59:50.473699+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T08:47:02.210906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.d","version":"0'0","reported_seq":11,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-11-24T07:59:53.609661+0000","last_change":"2023-11-24T07:59:52.525088+0000","last_active":"2023-11-24T07:59:53.609661+0000","last_peered":"2023-11-24T07:59:53.609661+0000","last_clean":"2023-11-24T07:59:53.609661+0000","last_became_active":"2023-11-24T07:59:52.524659+0000","last_became_peered":"2023-11-24T07:59:52.524659+0000","last_unstale":"2023-11-24T07:59:53.609661+0000","last_undegraded":"2023-11-24T07:59:53.609661+0000","last_fullsized":"2023-11-24T07:59:53.609661+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:59:51.477944+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:59:51.477944+0000","last_clean_scrub_stamp":"2023-11-24T07:59:51.477944+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T19:33:56.535985+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.c","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-11-24T07:59:53.370659+0000","last_change":"2023-11-24T07:59:51.498787+0000","last_active":"2023-11-24T07:59:53.370659+0000","last_peered":"2023-11-24T07:59:53.370659+0000","last_clean":"2023-11-24T07:59:53.370659+0000","last_became_active":"2023-11-24T07:59:51.498491+0000","last_became_peered":"2023-11-24T07:59:51.498491+0000","last_unstale":"2023-11-24T07:59:53.370659+0000","last_undegraded":"2023-11-24T07:59:53.370659+0000","last_fullsized":"2023-11-24T07:59:53.370659+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:59:50.473699+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:59:50.473699+0000","last_clean_scrub_stamp":"2023-11-24T07:59:50.473699+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T09:44:29.100278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.a","version":"0'0","reported_seq":11,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-11-24T07:59:53.609246+0000","last_change":"2023-11-24T07:59:52.514611+0000","last_active":"2023-11-24T07:59:53.609246+0000","last_peered":"2023-11-24T07:59:53.609246+0000","last_clean":"2023-11-24T07:59:53.609246+0000","last_became_active":"2023-11-24T07:59:52.514320+0000","last_became_peered":"2023-11-24T07:59:52.514320+0000","last_unstale":"2023-11-24T07:59:53.609246+0000","last_undegraded":"2023-11-24T07:59:53.609246+0000","last_fullsized":"2023-11-24T07:59:53.609246+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:59:51.477944+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:59:51.477944+0000","last_clean_scrub_stamp":"2023-11-24T07:59:51.477944+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T15:14:48.860422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.b","version":"86'2","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-11-24T07:59:53.370611+0000","last_change":"2023-11-24T07:59:51.497956+0000","last_active":"2023-11-24T07:59:53.370611+0000","last_peered":"2023-11-24T07:59:53.370611+0000","last_clean":"2023-11-24T07:59:53.370611+0000","last_became_active":"2023-11-24T07:59:51.497650+0000","last_became_peered":"2023-11-24T07:59:51.497650+0000","last_unstale":"2023-11-24T07:59:53.370611+0000","last_undegraded":"2023-11-24T07:59:53.370611+0000","last_fullsized":"2023-11-24T07:59:53.370611+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:59:50.473699+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:59:50.473699+0000","last_clean_scrub_stamp":"2023-11-24T07:59:50.473699+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T16:41:39.250342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.b","version":"0'0","reported_seq":11,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-11-24T07:59:53.506139+0000","last_change":"2023-11-24T07:59:52.517321+0000","last_active":"2023-11-24T07:59:53.506139+0000","last_peered":"2023-11-24T07:59:53.506139+0000","last_clean":"2023-11-24T07:59:53.506139+0000","last_became_active":"2023-11-24T07:59:52.517019+0000","last_became_peered":"2023-11-24T07:59:52.517019+0000","last_unstale":"2023-11-24T07:59:53.506139+0000","last_undegraded":"2023-11-24T07:59:53.506139+0000","last_fullsized":"2023-11-24T07:59:53.506139+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:59:51.477944+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:59:51.477944+0000","last_clean_scrub_stamp":"2023-11-24T07:59:51.477944+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T09:23:54.510983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.a","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-11-24T07:59:53.373019+0000","last_change":"2023-11-24T07:59:51.498910+0000","last_active":"2023-11-24T07:59:53.373019+0000","last_peered":"2023-11-24T07:59:53.373019+0000","last_clean":"2023-11-24T07:59:53.373019+0000","last_became_active":"2023-11-24T07:59:51.498576+0000","last_became_peered":"2023-11-24T07:59:51.498576+0000","last_unstale":"2023-11-24T07:59:53.373019+0000","last_undegraded":"2023-11-24T07:59:53.373019+0000","last_fullsized":"2023-11-24T07:59:53.373019+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:59:50.473699+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:59:50.473699+0000","last_clean_scrub_stamp":"2023-11-24T07:59:50.473699+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T15:30:00.451730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.8","version":"0'0","reported_seq":11,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-11-24T07:59:53.506366+0000","last_change":"2023-11-24T07:59:52.518390+0000","last_active":"2023-11-24T07:59:53.506366+0000","last_peered":"2023-11-24T07:59:53.506366+0000","last_clean":"2023-11-24T07:59:53.506366+0000","last_became_active":"2023-11-24T07:59:52.518123+0000","last_became_peered":"2023-11-24T07:59:52.518123+0000","last_unstale":"2023-11-24T07:59:53.506366+0000","last_undegraded":"2023-11-24T07:59:53.506366+0000","last_fullsized":"2023-11-24T07:59:53.506366+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:59:51.477944+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:59:51.477944+0000","last_clean_scrub_stamp":"2023-11-24T07:59:51.477944+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T14:01:22.935817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.9","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-11-24T07:59:53.373001+0000","last_change":"2023-11-24T07:59:51.493885+0000","last_active":"2023-11-24T07:59:53.373001+0000","last_peered":"2023-11-24T07:59:53.373001+0000","last_clean":"2023-11-24T07:59:53.373001+0000","last_became_active":"2023-11-24T07:59:51.493587+0000","last_became_peered":"2023-11-24T07:59:51.493587+0000","last_unstale":"2023-11-24T07:59:53.373001+0000","last_undegraded":"2023-11-24T07:59:53.373001+0000","last_fullsized":"2023-11-24T07:59:53.373001+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:59:50.473699+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:59:50.473699+0000","last_clean_scrub_stamp":"2023-11-24T07:59:50.473699+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T09:59:01.188427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.9","version":"0'0","reported_seq":11,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-11-24T07:59:53.609478+0000","last_change":"2023-11-24T07:59:52.517894+0000","last_active":"2023-11-24T07:59:53.609478+0000","last_peered":"2023-11-24T07:59:53.609478+0000","last_clean":"2023-11-24T07:59:53.609478+0000","last_became_active":"2023-11-24T07:59:52.517590+0000","last_became_peered":"2023-11-24T07:59:52.517590+0000","last_unstale":"2023-11-24T07:59:53.609478+0000","last_undegraded":"2023-11-24T07:59:53.609478+0000","last_fullsized":"2023-11-24T07:59:53.609478+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:59:51.477944+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:59:51.477944+0000","last_clean_scrub_stamp":"2023-11-24T07:59:51.477944+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T13:17:20.654171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.8","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-11-24T07:59:53.370542+0000","last_change":"2023-11-24T07:59:51.501348+0000","last_active":"2023-11-24T07:59:53.370542+0000","last_peered":"2023-11-24T07:59:53.370542+0000","last_clean":"2023-11-24T07:59:53.370542+0000","last_became_active":"2023-11-24T07:59:51.501049+0000","last_became_peered":"2023-11-24T07:59:51.501049+0000","last_unstale":"2023-11-24T07:59:53.370542+0000","last_undegraded":"2023-11-24T07:59:53.370542+0000","last_fullsized":"2023-11-24T07:59:53.370542+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:59:50.473699+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:59:50.473699+0000","last_clean_scrub_stamp":"2023-11-24T07:59:50.473699+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T18:50:30.244220+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.6","version":"0'0","reported_seq":11,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-11-24T07:59:53.506232+0000","last_change":"2023-11-24T07:59:52.520831+0000","last_active":"2023-11-24T07:59:53.506232+0000","last_peered":"2023-11-24T07:59:53.506232+0000","last_clean":"2023-11-24T07:59:53.506232+0000","last_became_active":"2023-11-24T07:59:52.520157+0000","last_became_peered":"2023-11-24T07:59:52.520157+0000","last_unstale":"2023-11-24T07:59:53.506232+0000","last_undegraded":"2023-11-24T07:59:53.506232+0000","last_fullsized":"2023-11-24T07:59:53.506232+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:59:51.477944+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:59:51.477944+0000","last_clean_scrub_stamp":"2023-11-24T07:59:51.477944+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T15:32:44.719383+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.7","version":"86'2","reported_seq":14,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-11-24T07:59:53.370712+0000","last_change":"2023-11-24T07:59:51.492490+0000","last_active":"2023-11-24T07:59:53.370712+0000","last_peered":"2023-11-24T07:59:53.370712+0000","last_clean":"2023-11-24T07:59:53.370712+0000","last_became_active":"2023-11-24T07:59:51.491865+0000","last_became_peered":"2023-11-24T07:59:51.491865+0000","last_unstale":"2023-11-24T07:59:53.370712+0000","last_undegraded":"2023-11-24T07:59:53.370712+0000","last_fullsized":"2023-11-24T07:59:53.370712+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:59:50.473699+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:59:50.473699+0000","last_clean_scrub_stamp":"2023-11-24T07:59:50.473699+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T08:40:15.071070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.7","version":"0'0","reported_seq":11,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-11-24T07:59:53.506478+0000","last_change":"2023-11-24T07:59:52.529281+0000","last_active":"2023-11-24T07:59:53.506478+0000","last_peered":"2023-11-24T07:59:53.506478+0000","last_clean":"2023-11-24T07:59:53.506478+0000","last_became_active":"2023-11-24T07:59:52.529002+0000","last_became_peered":"2023-11-24T07:59:52.529002+0000","last_unstale":"2023-11-24T07:59:53.506478+0000","last_undegraded":"2023-11-24T07:59:53.506478+0000","last_fullsized":"2023-11-24T07:59:53.506478+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:59:51.477944+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:59:51.477944+0000","last_clean_scrub_stamp":"2023-11-24T07:59:51.477944+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:50:14.772487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.6","version":"87'1","reported_seq":13,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-11-24T07:59:53.609109+0000","last_change":"2023-11-24T07:59:51.497940+0000","last_active":"2023-11-24T07:59:53.609109+0000","last_peered":"2023-11-24T07:59:53.609109+0000","last_clean":"2023-11-24T07:59:53.609109+0000","last_became_active":"2023-11-24T07:59:51.497679+0000","last_became_peered":"2023-11-24T07:59:51.497679+0000","last_unstale":"2023-11-24T07:59:53.609109+0000","last_undegraded":"2023-11-24T07:59:53.609109+0000","last_fullsized":"2023-11-24T07:59:53.609109+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:59:50.473699+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:59:50.473699+0000","last_clean_scrub_stamp":"2023-11-24T07:59:50.473699+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T08:25:06.417257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.4","version":"0'0","reported_seq":11,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-11-24T07:59:53.506233+0000","last_change":"2023-11-24T07:59:52.518391+0000","last_active":"2023-11-24T07:59:53.506233+0000","last_peered":"2023-11-24T07:59:53.506233+0000","last_clean":"2023-11-24T07:59:53.506233+0000","last_became_active":"2023-11-24T07:59:52.518101+0000","last_became_peered":"2023-11-24T07:59:52.518101+0000","last_unstale":"2023-11-24T07:59:53.506233+0000","last_undegraded":"2023-11-24T07:59:53.506233+0000","last_fullsized":"2023-11-24T07:59:53.506233+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:59:51.477944+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:59:51.477944+0000","last_clean_scrub_stamp":"2023-11-24T07:59:51.477944+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T19:52:10.399576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.5","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-11-24T07:59:53.367055+0000","last_change":"2023-11-24T07:59:51.494987+0000","last_active":"2023-11-24T07:59:53.367055+0000","last_peered":"2023-11-24T07:59:53.367055+0000","last_clean":"2023-11-24T07:59:53.367055+0000","last_became_active":"2023-11-24T07:59:51.494704+0000","last_became_peered":"2023-11-24T07:59:51.494704+0000","last_unstale":"2023-11-24T07:59:53.367055+0000","last_undegraded":"2023-11-24T07:59:53.367055+0000","last_fullsized":"2023-11-24T07:59:53.367055+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:59:50.473699+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:59:50.473699+0000","last_clean_scrub_stamp":"2023-11-24T07:59:50.473699+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T19:22:20.614507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.5","version":"0'0","reported_seq":11,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-11-24T07:59:53.608903+0000","last_change":"2023-11-24T07:59:52.532040+0000","last_active":"2023-11-24T07:59:53.608903+0000","last_peered":"2023-11-24T07:59:53.608903+0000","last_clean":"2023-11-24T07:59:53.608903+0000","last_became_active":"2023-11-24T07:59:52.531715+0000","last_became_peered":"2023-11-24T07:59:52.531715+0000","last_unstale":"2023-11-24T07:59:53.608903+0000","last_undegraded":"2023-11-24T07:59:53.608903+0000","last_fullsized":"2023-11-24T07:59:53.608903+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:59:51.477944+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:59:51.477944+0000","last_clean_scrub_stamp":"2023-11-24T07:59:51.477944+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T08:09:51.912125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.4","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-11-24T07:59:53.367112+0000","last_change":"2023-11-24T07:59:51.498705+0000","last_active":"2023-11-24T07:59:53.367112+0000","last_peered":"2023-11-24T07:59:53.367112+0000","last_clean":"2023-11-24T07:59:53.367112+0000","last_became_active":"2023-11-24T07:59:51.498398+0000","last_became_peered":"2023-11-24T07:59:51.498398+0000","last_unstale":"2023-11-24T07:59:53.367112+0000","last_undegraded":"2023-11-24T07:59:53.367112+0000","last_fullsized":"2023-11-24T07:59:53.367112+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:59:50.473699+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:59:50.473699+0000","last_clean_scrub_stamp":"2023-11-24T07:59:50.473699+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T19:22:20.614507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.2","version":"0'0","reported_seq":11,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-11-24T07:59:53.608160+0000","last_change":"2023-11-24T07:59:52.532307+0000","last_active":"2023-11-24T07:59:53.608160+0000","last_peered":"2023-11-24T07:59:53.608160+0000","last_clean":"2023-11-24T07:59:53.608160+0000","last_became_active":"2023-11-24T07:59:52.531978+0000","last_became_peered":"2023-11-24T07:59:52.531978+0000","last_unstale":"2023-11-24T07:59:53.608160+0000","last_undegraded":"2023-11-24T07:59:53.608160+0000","last_fullsized":"2023-11-24T07:59:53.608160+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:59:51.477944+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:59:51.477944+0000","last_clean_scrub_stamp":"2023-11-24T07:59:51.477944+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:15:45.686063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.3","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-11-24T07:59:53.373039+0000","last_change":"2023-11-24T07:59:51.502368+0000","last_active":"2023-11-24T07:59:53.373039+0000","last_peered":"2023-11-24T07:59:53.373039+0000","last_clean":"2023-11-24T07:59:53.373039+0000","last_became_active":"2023-11-24T07:59:51.501719+0000","last_became_peered":"2023-11-24T07:59:51.501719+0000","last_unstale":"2023-11-24T07:59:53.373039+0000","last_undegraded":"2023-11-24T07:59:53.373039+0000","last_fullsized":"2023-11-24T07:59:53.373039+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:59:50.473699+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:59:50.473699+0000","last_clean_scrub_stamp":"2023-11-24T07:59:50.473699+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:15:44.681817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.3","version":"0'0","reported_seq":11,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-11-24T07:59:53.608764+0000","last_change":"2023-11-24T07:59:52.531520+0000","last_active":"2023-11-24T07:59:53.608764+0000","last_peered":"2023-11-24T07:59:53.608764+0000","last_clean":"2023-11-24T07:59:53.608764+0000","last_became_active":"2023-11-24T07:59:52.531120+0000","last_became_peered":"2023-11-24T07:59:52.531120+0000","last_unstale":"2023-11-24T07:59:53.608764+0000","last_undegraded":"2023-11-24T07:59:53.608764+0000","last_fullsized":"2023-11-24T07:59:53.608764+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:59:51.477944+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:59:51.477944+0000","last_clean_scrub_stamp":"2023-11-24T07:59:51.477944+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T11:56:37.637066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.2","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-11-24T07:59:53.367071+0000","last_change":"2023-11-24T07:59:51.496507+0000","last_active":"2023-11-24T07:59:53.367071+0000","last_peered":"2023-11-24T07:59:53.367071+0000","last_clean":"2023-11-24T07:59:53.367071+0000","last_became_active":"2023-11-24T07:59:51.495858+0000","last_became_peered":"2023-11-24T07:59:51.495858+0000","last_unstale":"2023-11-24T07:59:53.367071+0000","last_undegraded":"2023-11-24T07:59:53.367071+0000","last_fullsized":"2023-11-24T07:59:53.367071+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:59:50.473699+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:59:50.473699+0000","last_clean_scrub_stamp":"2023-11-24T07:59:50.473699+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T15:05:21.923740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.0","version":"0'0","reported_seq":11,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-11-24T07:59:53.608044+0000","last_change":"2023-11-24T07:59:52.538042+0000","last_active":"2023-11-24T07:59:53.608044+0000","last_peered":"2023-11-24T07:59:53.608044+0000","last_clean":"2023-11-24T07:59:53.608044+0000","last_became_active":"2023-11-24T07:59:52.537692+0000","last_became_peered":"2023-11-24T07:59:52.537692+0000","last_unstale":"2023-11-24T07:59:53.608044+0000","last_undegraded":"2023-11-24T07:59:53.608044+0000","last_fullsized":"2023-11-24T07:59:53.608044+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:59:51.477944+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:59:51.477944+0000","last_clean_scrub_stamp":"2023-11-24T07:59:51.477944+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T15:30:01.455976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.1","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-11-24T07:59:53.376829+0000","last_change":"2023-11-24T07:59:51.501509+0000","last_active":"2023-11-24T07:59:53.376829+0000","last_peered":"2023-11-24T07:59:53.376829+0000","last_clean":"2023-11-24T07:59:53.376829+0000","last_became_active":"2023-11-24T07:59:51.501201+0000","last_became_peered":"2023-11-24T07:59:51.501201+0000","last_unstale":"2023-11-24T07:59:53.376829+0000","last_undegraded":"2023-11-24T07:59:53.376829+0000","last_fullsized":"2023-11-24T07:59:53.376829+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:59:50.473699+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:59:50.473699+0000","last_clean_scrub_stamp":"2023-11-24T07:59:50.473699+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T14:16:26.355513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.1","version":"0'0","reported_seq":11,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-11-24T07:59:53.608183+0000","last_change":"2023-11-24T07:59:52.521402+0000","last_active":"2023-11-24T07:59:53.608183+0000","last_peered":"2023-11-24T07:59:53.608183+0000","last_clean":"2023-11-24T07:59:53.608183+0000","last_became_active":"2023-11-24T07:59:52.520843+0000","last_became_peered":"2023-11-24T07:59:52.520843+0000","last_unstale":"2023-11-24T07:59:53.608183+0000","last_undegraded":"2023-11-24T07:59:53.608183+0000","last_fullsized":"2023-11-24T07:59:53.608183+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:59:51.477944+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:59:51.477944+0000","last_clean_scrub_stamp":"2023-11-24T07:59:51.477944+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T09:59:02.192672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.0","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-11-24T07:59:53.366951+0000","last_change":"2023-11-24T07:59:51.500716+0000","last_active":"2023-11-24T07:59:53.366951+0000","last_peered":"2023-11-24T07:59:53.366951+0000","last_clean":"2023-11-24T07:59:53.366951+0000","last_became_active":"2023-11-24T07:59:51.500429+0000","last_became_peered":"2023-11-24T07:59:51.500429+0000","last_unstale":"2023-11-24T07:59:53.366951+0000","last_undegraded":"2023-11-24T07:59:53.366951+0000","last_fullsized":"2023-11-24T07:59:53.366951+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:59:50.473699+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:59:50.473699+0000","last_clean_scrub_stamp":"2023-11-24T07:59:50.473699+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T15:05:21.923740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.c","version":"0'0","reported_seq":11,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-11-24T07:59:53.609248+0000","last_change":"2023-11-24T07:59:52.519423+0000","last_active":"2023-11-24T07:59:53.609248+0000","last_peered":"2023-11-24T07:59:53.609248+0000","last_clean":"2023-11-24T07:59:53.609248+0000","last_became_active":"2023-11-24T07:59:52.517567+0000","last_became_peered":"2023-11-24T07:59:52.517567+0000","last_unstale":"2023-11-24T07:59:53.609248+0000","last_undegraded":"2023-11-24T07:59:53.609248+0000","last_fullsized":"2023-11-24T07:59:53.609248+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:59:51.477944+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:59:51.477944+0000","last_clean_scrub_stamp":"2023-11-24T07:59:51.477944+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T09:33:44.224663+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.d","version":"86'1","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-11-24T07:59:53.373305+0000","last_change":"2023-11-24T07:59:51.502027+0000","last_active":"2023-11-24T07:59:53.373305+0000","last_peered":"2023-11-24T07:59:53.373305+0000","last_clean":"2023-11-24T07:59:53.373305+0000","last_became_active":"2023-11-24T07:59:51.501705+0000","last_became_peered":"2023-11-24T07:59:51.501705+0000","last_unstale":"2023-11-24T07:59:53.373305+0000","last_undegraded":"2023-11-24T07:59:53.373305+0000","last_fullsized":"2023-11-24T07:59:53.373305+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:59:50.473699+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:59:50.473699+0000","last_clean_scrub_stamp":"2023-11-24T07:59:50.473699+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T17:10:42.541250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":222,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-11-24T07:59:52.514156+0000","last_change":"2023-11-24T07:56:19.668905+0000","last_active":"2023-11-24T07:59:52.514156+0000","last_peered":"2023-11-24T07:59:52.514156+0000","last_clean":"2023-11-24T07:59:52.514156+0000","last_became_active":"2023-11-24T07:56:19.668601+0000","last_became_peered":"2023-11-24T07:56:19.668601+0000","last_unstale":"2023-11-24T07:59:52.514156+0000","last_undegraded":"2023-11-24T07:59:52.514156+0000","last_fullsized":"2023-11-24T07:59:52.514156+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:56:18.657844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:56:18.657844+0000","last_clean_scrub_stamp":"2023-11-24T07:56:18.657844+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T11:51:53.828455+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":13,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":12,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574896,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28072,"kb_used_data":1048,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343768,"statfs":{"total":96636764160,"available":96608018432,"internally_reserved":0,"allocated":1073152,"data_stored":459185,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 07:59:20 2023","interfaces":[{"interface":"back","average":{"1min":0.51800000000000002,"5min":0.54000000000000004,"15min":0.55100000000000005},"min":{"1min":0.27600000000000002,"5min":0.27600000000000002,"15min":0.27600000000000002},"max":{"1min":0.83699999999999997,"5min":1.413,"15min":1.413},"last":0.40400000000000003},{"interface":"front","average":{"1min":0.53500000000000003,"5min":0.55200000000000005,"15min":0.55300000000000005},"min":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.72999999999999998,"5min":0.81000000000000005,"15min":0.81000000000000005},"last":0.54300000000000004}]},{"osd":1,"last update":"Fri Nov 24 07:59:20 2023","interfaces":[{"interface":"back","average":{"1min":0.51500000000000001,"5min":0.71199999999999997,"15min":0.80300000000000005},"min":{"1min":0.35499999999999998,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.76400000000000001,"5min":6.1070000000000002,"15min":6.1070000000000002},"last":0.51400000000000001},{"interface":"front","average":{"1min":0.48699999999999999,"5min":0.54800000000000004,"15min":0.56899999999999995},"min":{"1min":0.35199999999999998,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.77200000000000002,"5min":1.5920000000000001,"15min":1.5920000000000001},"last":0.56200000000000006}]},{"osd":2,"last update":"Fri Nov 24 07:59:20 2023","interfaces":[{"interface":"back","average":{"1min":0.52700000000000002,"5min":0.58099999999999996,"15min":0.61099999999999999},"min":{"1min":0.308,"5min":0.308,"15min":0.308},"max":{"1min":0.96899999999999997,"5min":1.613,"15min":1.613},"last":0.503},{"interface":"front","average":{"1min":0.48099999999999998,"5min":0.67100000000000004,"15min":0.751},"min":{"1min":0.33400000000000002,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.78000000000000003,"5min":4.5220000000000002,"15min":4.5220000000000002},"last":0.48899999999999999}]},{"osd":3,"last update":"Fri Nov 24 07:59:20 2023","interfaces":[{"interface":"back","average":{"1min":0.56599999999999995,"5min":0.70699999999999996,"15min":0.77300000000000002},"min":{"1min":0.40699999999999997,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.82699999999999996,"5min":4.6760000000000002,"15min":4.6760000000000002},"last":0.60399999999999998},{"interface":"front","average":{"1min":0.60499999999999998,"5min":0.80800000000000005,"15min":0.89800000000000002},"min":{"1min":0.41699999999999998,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.92200000000000004,"5min":4.6539999999999999,"15min":4.6539999999999999},"last":0.59399999999999997}]},{"osd":4,"last update":"Fri Nov 24 07:59:20 2023","interfaces":[{"interface":"back","average":{"1min":0.47199999999999998,"5min":0.54500000000000004,"15min":0.56000000000000005},"min":{"1min":0.32300000000000001,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.95099999999999996,"5min":0.95099999999999996,"15min":0.95099999999999996},"last":0.41999999999999998},{"interface":"front","average":{"1min":0.55500000000000005,"5min":0.62,"15min":0.65100000000000002},"min":{"1min":0.38,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":1.0069999999999999,"5min":1.629,"15min":1.629},"last":0.44800000000000001}]},{"osd":5,"last update":"Fri Nov 24 07:59:20 2023","interfaces":[{"interface":"back","average":{"1min":0.501,"5min":0.53600000000000003,"15min":0.54300000000000004},"min":{"1min":0.315,"5min":0.315,"15min":0.315},"max":{"1min":0.748,"5min":0.875,"15min":0.875},"last":0.38700000000000001},{"interface":"front","average":{"1min":0.56799999999999995,"5min":0.60299999999999998,"15min":0.61599999999999999},"min":{"1min":0.40500000000000003,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.89800000000000002,"5min":0.90900000000000003,"15min":0.90900000000000003},"last":0.47499999999999998}]},{"osd":6,"last update":"Fri Nov 24 07:59:20 2023","interfaces":[{"interface":"back","average":{"1min":0.55200000000000005,"5min":0.59699999999999998,"15min":0.60599999999999998},"min":{"1min":0.33100000000000002,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.85799999999999998,"5min":0.85799999999999998,"15min":0.85799999999999998},"last":0.435},{"interface":"front","average":{"1min":0.61199999999999999,"5min":0.65200000000000002,"15min":0.66400000000000003},"min":{"1min":0.443,"5min":0.443,"15min":0.443},"max":{"1min":0.99099999999999999,"5min":0.99099999999999999,"15min":0.99099999999999999},"last":0.57599999999999996}]}]},{"osd":6,"up_from":13,"seq":55834574897,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28076,"kb_used_data":1052,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343764,"statfs":{"total":96636764160,"available":96608014336,"internally_reserved":0,"allocated":1077248,"data_stored":459787,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 07:59:19 2023","interfaces":[{"interface":"back","average":{"1min":0.51000000000000001,"5min":0.53800000000000003,"15min":0.54600000000000004},"min":{"1min":0.33500000000000002,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":0.82999999999999996,"5min":0.93100000000000005,"15min":0.93100000000000005},"last":0.65300000000000002},{"interface":"front","average":{"1min":0.53100000000000003,"5min":0.51900000000000002,"15min":0.51700000000000002},"min":{"1min":0.34000000000000002,"5min":0.311,"15min":0.311},"max":{"1min":0.72299999999999998,"5min":0.82199999999999995,"15min":0.82199999999999995},"last":0.73799999999999999}]},{"osd":1,"last update":"Fri Nov 24 07:59:19 2023","interfaces":[{"interface":"back","average":{"1min":0.499,"5min":0.51000000000000001,"15min":0.51100000000000001},"min":{"1min":0.312,"5min":0.312,"15min":0.312},"max":{"1min":0.746,"5min":0.81299999999999994,"15min":0.81299999999999994},"last":0.67700000000000005},{"interface":"front","average":{"1min":0.55900000000000005,"5min":0.58699999999999997,"15min":0.60399999999999998},"min":{"1min":0.29999999999999999,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":0.70799999999999996,"5min":1.0069999999999999,"15min":1.0069999999999999},"last":0.68999999999999995}]},{"osd":2,"last update":"Fri Nov 24 07:59:19 2023","interfaces":[{"interface":"back","average":{"1min":0.57699999999999996,"5min":0.57699999999999996,"15min":0.57499999999999996},"min":{"1min":0.41399999999999998,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.71099999999999997,"5min":0.90600000000000003,"15min":0.90600000000000003},"last":0.70499999999999996},{"interface":"front","average":{"1min":0.59899999999999998,"5min":0.58799999999999997,"15min":0.59099999999999997},"min":{"1min":0.52500000000000002,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.78800000000000003,"5min":0.84899999999999998,"15min":0.84899999999999998},"last":0.71699999999999997}]},{"osd":3,"last update":"Fri Nov 24 07:59:19 2023","interfaces":[{"interface":"back","average":{"1min":0.56200000000000006,"5min":0.56499999999999995,"15min":0.56699999999999995},"min":{"1min":0.38700000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.75700000000000001,"5min":0.876,"15min":0.876},"last":0.76900000000000002},{"interface":"front","average":{"1min":0.55700000000000005,"5min":0.56000000000000005,"15min":0.55700000000000005},"min":{"1min":0.40500000000000003,"5min":0.379,"15min":0.379},"max":{"1min":0.72599999999999998,"5min":0.876,"15min":0.876},"last":0.753}]},{"osd":4,"last update":"Fri Nov 24 07:59:19 2023","interfaces":[{"interface":"back","average":{"1min":0.55300000000000005,"5min":0.52700000000000002,"15min":0.52000000000000002},"min":{"1min":0.36299999999999999,"5min":0.25,"15min":0.25},"max":{"1min":0.80700000000000005,"5min":0.89400000000000002,"15min":0.89400000000000002},"last":0.501},{"interface":"front","average":{"1min":0.53100000000000003,"5min":0.58499999999999996,"15min":0.59599999999999997},"min":{"1min":0.30599999999999999,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":0.81899999999999995,"5min":1.087,"15min":1.087},"last":0.59299999999999997}]},{"osd":5,"last update":"Fri Nov 24 07:59:19 2023","interfaces":[{"interface":"back","average":{"1min":0.57099999999999995,"5min":0.55100000000000005,"15min":0.53700000000000003},"min":{"1min":0.33300000000000002,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":0.76200000000000001,"5min":0.97999999999999998,"15min":0.97999999999999998},"last":0.47999999999999998},{"interface":"front","average":{"1min":0.62,"5min":0.61399999999999999,"15min":0.61199999999999999},"min":{"1min":0.438,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.997,"5min":1.0009999999999999,"15min":1.0009999999999999},"last":0.56699999999999995}]},{"osd":7,"last update":"Fri Nov 24 07:59:19 2023","interfaces":[{"interface":"back","average":{"1min":0.59899999999999998,"5min":0.61399999999999999,"15min":0.61599999999999999},"min":{"1min":0.44800000000000001,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":0.85499999999999998,"5min":1.0880000000000001,"15min":1.0880000000000001},"last":0.51900000000000002},{"interface":"front","average":{"1min":0.628,"5min":0.65800000000000003,"15min":0.66600000000000004},"min":{"1min":0.32800000000000001,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.79500000000000004,"5min":0.95599999999999996,"15min":0.95599999999999996},"last":0.61099999999999999}]}]},{"osd":1,"up_from":13,"seq":55834574898,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28652,"kb_used_data":1628,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343188,"statfs":{"total":96636764160,"available":96607424512,"internally_reserved":0,"allocated":1667072,"data_stored":1049573,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 07:59:24 2023","interfaces":[{"interface":"back","average":{"1min":0.35499999999999998,"5min":0.35499999999999998,"15min":0.36099999999999999},"min":{"1min":0.248,"5min":0.20200000000000001,"15min":0.20200000000000001},"max":{"1min":0.52600000000000002,"5min":0.70899999999999996,"15min":0.70899999999999996},"last":0.49299999999999999},{"interface":"front","average":{"1min":0.36299999999999999,"5min":0.38400000000000001,"15min":0.39300000000000002},"min":{"1min":0.23300000000000001,"5min":0.214,"15min":0.214},"max":{"1min":0.65600000000000003,"5min":0.82499999999999996,"15min":0.82499999999999996},"last":0.43099999999999999}]},{"osd":2,"last update":"Fri Nov 24 07:59:24 2023","interfaces":[{"interface":"back","average":{"1min":0.36899999999999999,"5min":0.35999999999999999,"15min":0.36799999999999999},"min":{"1min":0.27100000000000002,"5min":0.22700000000000001,"15min":0.22700000000000001},"max":{"1min":0.53000000000000003,"5min":0.60999999999999999,"15min":0.60999999999999999},"last":0.47099999999999997},{"interface":"front","average":{"1min":0.42299999999999999,"5min":0.40500000000000003,"15min":0.40999999999999998},"min":{"1min":0.28899999999999998,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":0.63700000000000001,"5min":0.63700000000000001,"15min":0.63700000000000001},"last":0.55600000000000005}]},{"osd":3,"last update":"Fri Nov 24 07:59:24 2023","interfaces":[{"interface":"back","average":{"1min":0.43099999999999999,"5min":0.433,"15min":0.44},"min":{"1min":0.317,"5min":0.317,"15min":0.317},"max":{"1min":0.62,"5min":0.66500000000000004,"15min":0.66500000000000004},"last":0.58299999999999996},{"interface":"front","average":{"1min":0.41799999999999998,"5min":0.41699999999999998,"15min":0.42399999999999999},"min":{"1min":0.32900000000000001,"5min":0.28499999999999998,"15min":0.28499999999999998},"max":{"1min":0.59999999999999998,"5min":0.76300000000000001,"15min":0.76300000000000001},"last":0.60899999999999999}]},{"osd":4,"last update":"Fri Nov 24 07:59:24 2023","interfaces":[{"interface":"back","average":{"1min":0.625,"5min":0.60399999999999998,"15min":0.60799999999999998},"min":{"1min":0.434,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":1.5580000000000001,"5min":1.5580000000000001,"15min":1.5580000000000001},"last":0.90500000000000003},{"interface":"front","average":{"1min":0.65200000000000002,"5min":0.61599999999999999,"15min":0.60899999999999999},"min":{"1min":0.48599999999999999,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":0.82399999999999995,"5min":0.82399999999999995,"15min":0.82399999999999995},"last":0.77600000000000002}]},{"osd":5,"last update":"Fri Nov 24 07:59:24 2023","interfaces":[{"interface":"back","average":{"1min":0.625,"5min":0.67800000000000005,"15min":0.70899999999999996},"min":{"1min":0.44400000000000001,"5min":0.37,"15min":0.37},"max":{"1min":0.755,"5min":2.7269999999999999,"15min":2.7269999999999999},"last":0.98699999999999999},{"interface":"front","average":{"1min":0.68500000000000005,"5min":0.63900000000000001,"15min":0.64300000000000002},"min":{"1min":0.434,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":1.6459999999999999,"5min":1.6459999999999999,"15min":1.6459999999999999},"last":0.84999999999999998}]},{"osd":6,"last update":"Fri Nov 24 07:59:24 2023","interfaces":[{"interface":"back","average":{"1min":0.63200000000000001,"5min":0.64300000000000002,"15min":0.65300000000000002},"min":{"1min":0.46000000000000002,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":0.93799999999999994,"5min":0.98199999999999998,"15min":0.98199999999999998},"last":0.81999999999999995},{"interface":"front","average":{"1min":0.67900000000000005,"5min":0.621,"15min":0.61799999999999999},"min":{"1min":0.46100000000000002,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":0.91600000000000004,"5min":0.91600000000000004,"15min":0.91600000000000004},"last":0.93600000000000005}]},{"osd":7,"last update":"Fri Nov 24 07:59:24 2023","interfaces":[{"interface":"back","average":{"1min":0.69699999999999995,"5min":0.64700000000000002,"15min":0.64000000000000001},"min":{"1min":0.53900000000000003,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.95599999999999996,"5min":1.0649999999999999,"15min":1.0649999999999999},"last":0.87},{"interface":"front","average":{"1min":0.64300000000000002,"5min":0.64400000000000002,"15min":0.66100000000000003},"min":{"1min":0.45900000000000002,"5min":0.442,"15min":0.442},"max":{"1min":0.93300000000000005,"5min":0.93300000000000005,"15min":0.93300000000000005},"last":0.96799999999999997}]}]},{"osd":0,"up_from":13,"seq":55834574898,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28084,"kb_used_data":1060,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343756,"statfs":{"total":96636764160,"available":96608006144,"internally_reserved":0,"allocated":1085440,"data_stored":460749,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Nov 24 07:59:22 2023","interfaces":[{"interface":"back","average":{"1min":0.36899999999999999,"5min":0.57799999999999996,"15min":0.66800000000000004},"min":{"1min":0.247,"5min":0.21299999999999999,"15min":0.21299999999999999},"max":{"1min":0.56100000000000005,"5min":6.3550000000000004,"15min":6.3550000000000004},"last":0.45700000000000002},{"interface":"front","average":{"1min":0.39300000000000002,"5min":1.1970000000000001,"15min":1.5509999999999999},"min":{"1min":0.29399999999999998,"5min":0.26000000000000001,"15min":0.26000000000000001},"max":{"1min":0.59099999999999997,"5min":22.916,"15min":22.916},"last":0.53500000000000003}]},{"osd":2,"last update":"Fri Nov 24 07:59:22 2023","interfaces":[{"interface":"back","average":{"1min":0.35999999999999999,"5min":0.56200000000000006,"15min":0.65400000000000003},"min":{"1min":0.23400000000000001,"5min":0.22800000000000001,"15min":0.22800000000000001},"max":{"1min":0.54100000000000004,"5min":6.3899999999999997,"15min":6.3899999999999997},"last":0.39700000000000002},{"interface":"front","average":{"1min":0.32300000000000001,"5min":0.78100000000000003,"15min":0.96899999999999997},"min":{"1min":0.253,"5min":0.253,"15min":0.253},"max":{"1min":0.497,"5min":11.93,"15min":11.93},"last":0.30499999999999999}]},{"osd":3,"last update":"Fri Nov 24 07:59:22 2023","interfaces":[{"interface":"back","average":{"1min":0.373,"5min":0.40100000000000002,"15min":0.41299999999999998},"min":{"1min":0.247,"5min":0.23799999999999999,"15min":0.23799999999999999},"max":{"1min":0.52500000000000002,"5min":1.155,"15min":1.155},"last":0.36499999999999999},{"interface":"front","average":{"1min":0.40200000000000002,"5min":0.84399999999999997,"15min":1.032},"min":{"1min":0.27400000000000002,"5min":0.27400000000000002,"15min":0.27400000000000002},"max":{"1min":0.63400000000000001,"5min":12.116,"15min":12.116},"last":0.33800000000000002}]},{"osd":4,"last update":"Fri Nov 24 07:59:22 2023","interfaces":[{"interface":"back","average":{"1min":0.59799999999999998,"5min":0.63300000000000001,"15min":0.64300000000000002},"min":{"1min":0.44400000000000001,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.93500000000000005,"5min":1.044,"15min":1.044},"last":0.46800000000000003},{"interface":"front","average":{"1min":0.58499999999999996,"5min":1.4059999999999999,"15min":1.758},"min":{"1min":0.377,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.84999999999999998,"5min":22.963999999999999,"15min":22.963999999999999},"last":0.622}]},{"osd":5,"last update":"Fri Nov 24 07:59:22 2023","interfaces":[{"interface":"back","average":{"1min":0.59999999999999998,"5min":1.026,"15min":1.204},"min":{"1min":0.41199999999999998,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.86299999999999999,"5min":12.084,"15min":12.084},"last":0.56599999999999995},{"interface":"front","average":{"1min":0.61899999999999999,"5min":0.66200000000000003,"15min":0.67200000000000004},"min":{"1min":0.46600000000000003,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.89100000000000001,"5min":1.073,"15min":1.073},"last":0.59999999999999998}]},{"osd":6,"last update":"Fri Nov 24 07:59:22 2023","interfaces":[{"interface":"back","average":{"1min":0.61599999999999999,"5min":0.66800000000000004,"15min":0.68300000000000005},"min":{"1min":0.36799999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.92400000000000004,"5min":1.0580000000000001,"15min":1.0580000000000001},"last":0.65600000000000003},{"interface":"front","average":{"1min":0.60099999999999998,"5min":1.042,"15min":1.2230000000000001},"min":{"1min":0.46000000000000002,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":0.81999999999999995,"5min":12.045,"15min":12.045},"last":0.48599999999999999}]},{"osd":7,"last update":"Fri Nov 24 07:59:22 2023","interfaces":[{"interface":"back","average":{"1min":0.57699999999999996,"5min":0.64300000000000002,"15min":0.65100000000000002},"min":{"1min":0.45700000000000002,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":0.82999999999999996,"5min":1.038,"15min":1.038},"last":0.51900000000000002},{"interface":"front","average":{"1min":0.59499999999999997,"5min":1.431,"15min":1.7849999999999999},"min":{"1min":0.42299999999999999,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":0.80300000000000005,"5min":22.995000000000001,"15min":22.995000000000001},"last":0.67000000000000004}]}]},{"osd":2,"up_from":13,"seq":55834574898,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28068,"kb_used_data":1044,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343772,"statfs":{"total":96636764160,"available":96608022528,"internally_reserved":0,"allocated":1069056,"data_stored":459115,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 07:59:20 2023","interfaces":[{"interface":"back","average":{"1min":0.31900000000000001,"5min":0.35899999999999999,"15min":0.36699999999999999},"min":{"1min":0.24399999999999999,"5min":0.219,"15min":0.219},"max":{"1min":0.52600000000000002,"5min":0.58099999999999996,"15min":0.58099999999999996},"last":0.49399999999999999},{"interface":"front","average":{"1min":0.376,"5min":0.39400000000000002,"15min":0.39900000000000002},"min":{"1min":0.29699999999999999,"5min":0.27400000000000002,"15min":0.27400000000000002},"max":{"1min":0.52700000000000002,"5min":0.54100000000000004,"15min":0.54100000000000004},"last":0.39900000000000002}]},{"osd":1,"last update":"Fri Nov 24 07:59:20 2023","interfaces":[{"interface":"back","average":{"1min":0.32400000000000001,"5min":0.36099999999999999,"15min":0.372},"min":{"1min":0.21099999999999999,"5min":0.21099999999999999,"15min":0.21099999999999999},"max":{"1min":0.44600000000000001,"5min":0.59199999999999997,"15min":0.59199999999999997},"last":0.375},{"interface":"front","average":{"1min":0.36599999999999999,"5min":0.39600000000000002,"15min":0.40899999999999997},"min":{"1min":0.26900000000000002,"5min":0.25600000000000001,"15min":0.25600000000000001},"max":{"1min":0.51100000000000001,"5min":0.627,"15min":0.627},"last":0.45000000000000001}]},{"osd":3,"last update":"Fri Nov 24 07:59:20 2023","interfaces":[{"interface":"back","average":{"1min":0.36799999999999999,"5min":0.40100000000000002,"15min":0.40699999999999997},"min":{"1min":0.26400000000000001,"5min":0.26400000000000001,"15min":0.26400000000000001},"max":{"1min":0.59899999999999998,"5min":0.64200000000000002,"15min":0.64200000000000002},"last":0.46999999999999997},{"interface":"front","average":{"1min":0.36499999999999999,"5min":0.40400000000000003,"15min":0.40999999999999998},"min":{"1min":0.28100000000000003,"5min":0.27500000000000002,"15min":0.27500000000000002},"max":{"1min":0.58199999999999996,"5min":0.64600000000000002,"15min":0.64600000000000002},"last":0.42299999999999999}]},{"osd":4,"last update":"Fri Nov 24 07:59:20 2023","interfaces":[{"interface":"back","average":{"1min":0.50700000000000001,"5min":0.57499999999999996,"15min":0.59699999999999998},"min":{"1min":0.35799999999999998,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.73099999999999998,"5min":0.81599999999999995,"15min":0.81599999999999995},"last":0.80000000000000004},{"interface":"front","average":{"1min":0.56000000000000005,"5min":0.58299999999999996,"15min":0.59399999999999997},"min":{"1min":0.38700000000000001,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.68100000000000005,"5min":0.83799999999999997,"15min":0.83799999999999997},"last":0.61699999999999999}]},{"osd":5,"last update":"Fri Nov 24 07:59:20 2023","interfaces":[{"interface":"back","average":{"1min":0.57199999999999995,"5min":0.59899999999999998,"15min":0.61299999999999999},"min":{"1min":0.41599999999999998,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":0.745,"5min":0.84899999999999998,"15min":0.84899999999999998},"last":0.69999999999999996},{"interface":"front","average":{"1min":0.53400000000000003,"5min":0.66000000000000003,"15min":0.69599999999999995},"min":{"1min":0.41599999999999998,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":0.75800000000000001,"5min":2.1699999999999999,"15min":2.1699999999999999},"last":0.63100000000000001}]},{"osd":6,"last update":"Fri Nov 24 07:59:20 2023","interfaces":[{"interface":"back","average":{"1min":0.55000000000000004,"5min":0.63100000000000001,"15min":0.65500000000000003},"min":{"1min":0.42099999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":0.75,"5min":0.878,"15min":0.878},"last":0.83799999999999997},{"interface":"front","average":{"1min":0.54300000000000004,"5min":0.63200000000000001,"15min":0.65800000000000003},"min":{"1min":0.436,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":0.81699999999999995,"5min":0.88500000000000001,"15min":0.88500000000000001},"last":0.76000000000000001}]},{"osd":7,"last update":"Fri Nov 24 07:59:20 2023","interfaces":[{"interface":"back","average":{"1min":0.57499999999999996,"5min":0.63300000000000001,"15min":0.65200000000000002},"min":{"1min":0.42799999999999999,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":0.74199999999999999,"5min":0.85999999999999999,"15min":0.85999999999999999},"last":0.89300000000000002},{"interface":"front","average":{"1min":0.57399999999999995,"5min":0.64700000000000002,"15min":0.66900000000000004},"min":{"1min":0.39700000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.77900000000000003,"5min":0.90000000000000002,"15min":0.90000000000000002},"last":0.74099999999999999}]}]},{"osd":3,"up_from":13,"seq":55834574898,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28080,"kb_used_data":1056,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343760,"statfs":{"total":96636764160,"available":96608010240,"internally_reserved":0,"allocated":1081344,"data_stored":459789,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 07:59:27 2023","interfaces":[{"interface":"back","average":{"1min":0.33600000000000002,"5min":0.34899999999999998,"15min":0.35299999999999998},"min":{"1min":0.183,"5min":0.183,"15min":0.183},"max":{"1min":0.53400000000000003,"5min":0.84799999999999998,"15min":0.84799999999999998},"last":0.35199999999999998},{"interface":"front","average":{"1min":0.34599999999999997,"5min":0.35399999999999998,"15min":0.36199999999999999},"min":{"1min":0.24299999999999999,"5min":0.23699999999999999,"15min":0.23699999999999999},"max":{"1min":0.51000000000000001,"5min":0.873,"15min":0.873},"last":0.45900000000000002}]},{"osd":1,"last update":"Fri Nov 24 07:59:27 2023","interfaces":[{"interface":"back","average":{"1min":0.38400000000000001,"5min":0.373,"15min":0.374},"min":{"1min":0.217,"5min":0.217,"15min":0.217},"max":{"1min":0.66300000000000003,"5min":0.92000000000000004,"15min":0.92000000000000004},"last":0.48799999999999999},{"interface":"front","average":{"1min":0.38,"5min":0.377,"15min":0.38300000000000001},"min":{"1min":0.28599999999999998,"5min":0.248,"15min":0.248},"max":{"1min":0.626,"5min":0.81699999999999995,"15min":0.81699999999999995},"last":0.38}]},{"osd":2,"last update":"Fri Nov 24 07:59:27 2023","interfaces":[{"interface":"back","average":{"1min":0.35599999999999998,"5min":0.35399999999999998,"15min":0.35799999999999998},"min":{"1min":0.253,"5min":0.24399999999999999,"15min":0.24399999999999999},"max":{"1min":0.496,"5min":0.88500000000000001,"15min":0.88500000000000001},"last":0.40600000000000003},{"interface":"front","average":{"1min":0.40999999999999998,"5min":0.441,"15min":0.45900000000000002},"min":{"1min":0.315,"5min":0.311,"15min":0.311},"max":{"1min":0.59799999999999998,"5min":0.98099999999999998,"15min":0.98099999999999998},"last":0.42999999999999999}]},{"osd":4,"last update":"Fri Nov 24 07:59:27 2023","interfaces":[{"interface":"back","average":{"1min":0.56299999999999994,"5min":0.70499999999999996,"15min":0.76600000000000001},"min":{"1min":0.38900000000000001,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.73599999999999999,"5min":5.7350000000000003,"15min":5.7350000000000003},"last":0.67300000000000004},{"interface":"front","average":{"1min":0.55700000000000005,"5min":0.58099999999999996,"15min":0.58299999999999996},"min":{"1min":0.34399999999999997,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.80400000000000005,"5min":1.27,"15min":1.27},"last":0.65800000000000003}]},{"osd":5,"last update":"Fri Nov 24 07:59:27 2023","interfaces":[{"interface":"back","average":{"1min":0.56100000000000005,"5min":0.57799999999999996,"15min":0.58399999999999996},"min":{"1min":0.42899999999999999,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":0.67900000000000005,"5min":1.4430000000000001,"15min":1.4430000000000001},"last":0.73499999999999999},{"interface":"front","average":{"1min":0.58799999999999997,"5min":0.61499999999999999,"15min":0.623},"min":{"1min":0.40999999999999998,"5min":0.377,"15min":0.377},"max":{"1min":0.78500000000000003,"5min":1.4239999999999999,"15min":1.4239999999999999},"last":0.63900000000000001}]},{"osd":6,"last update":"Fri Nov 24 07:59:27 2023","interfaces":[{"interface":"back","average":{"1min":0.58799999999999997,"5min":0.60699999999999998,"15min":0.61499999999999999},"min":{"1min":0.42599999999999999,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":0.84499999999999997,"5min":1.355,"15min":1.355},"last":0.59899999999999998},{"interface":"front","average":{"1min":0.57899999999999996,"5min":0.745,"15min":0.81200000000000006},"min":{"1min":0.44800000000000001,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.77100000000000002,"5min":5.6479999999999997,"15min":5.6479999999999997},"last":0.70599999999999996}]},{"osd":7,"last update":"Fri Nov 24 07:59:27 2023","interfaces":[{"interface":"back","average":{"1min":0.61799999999999999,"5min":0.64100000000000001,"15min":0.65700000000000003},"min":{"1min":0.499,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":0.82799999999999996,"5min":1.51,"15min":1.51},"last":0.61599999999999999},{"interface":"front","average":{"1min":0.57699999999999996,"5min":0.74199999999999999,"15min":0.80800000000000005},"min":{"1min":0.40100000000000002,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.79500000000000004,"5min":5.7069999999999999,"15min":5.7069999999999999},"last":0.77600000000000002}]}]},{"osd":4,"up_from":13,"seq":55834574897,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28072,"kb_used_data":1048,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343768,"statfs":{"total":96636764160,"available":96608018432,"internally_reserved":0,"allocated":1073152,"data_stored":459149,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 07:59:28 2023","interfaces":[{"interface":"back","average":{"1min":0.55900000000000005,"5min":0.56100000000000005,"15min":0.57399999999999995},"min":{"1min":0.32400000000000001,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.93300000000000005,"5min":0.93300000000000005,"15min":0.93300000000000005},"last":0.64200000000000002},{"interface":"front","average":{"1min":0.51100000000000001,"5min":0.52800000000000002,"15min":0.53800000000000003},"min":{"1min":0.36199999999999999,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.69299999999999995,"5min":1.0249999999999999,"15min":1.0249999999999999},"last":0.55300000000000005}]},{"osd":1,"last update":"Fri Nov 24 07:59:28 2023","interfaces":[{"interface":"back","average":{"1min":0.54800000000000004,"5min":0.54600000000000004,"15min":0.55000000000000004},"min":{"1min":0.35899999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.71599999999999997,"5min":0.86399999999999999,"15min":0.86399999999999999},"last":0.43099999999999999},{"interface":"front","average":{"1min":0.51600000000000001,"5min":0.53700000000000003,"15min":0.54600000000000004},"min":{"1min":0.36699999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.72099999999999997,"5min":0.98699999999999999,"15min":0.98699999999999999},"last":0.61899999999999999}]},{"osd":2,"last update":"Fri Nov 24 07:59:28 2023","interfaces":[{"interface":"back","average":{"1min":0.54200000000000004,"5min":0.54900000000000004,"15min":0.55700000000000005},"min":{"1min":0.34499999999999997,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.86399999999999999,"5min":0.86399999999999999,"15min":0.86399999999999999},"last":0.45600000000000002},{"interface":"front","average":{"1min":0.56200000000000006,"5min":0.59399999999999997,"15min":0.60599999999999998},"min":{"1min":0.34000000000000002,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.81899999999999995,"5min":0.89200000000000002,"15min":0.89200000000000002},"last":0.46899999999999997}]},{"osd":3,"last update":"Fri Nov 24 07:59:28 2023","interfaces":[{"interface":"back","average":{"1min":0.53600000000000003,"5min":0.56799999999999995,"15min":0.57899999999999996},"min":{"1min":0.38800000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.78100000000000003,"5min":0.89700000000000002,"15min":0.89700000000000002},"last":0.375},{"interface":"front","average":{"1min":0.58899999999999997,"5min":0.61099999999999999,"15min":0.624},"min":{"1min":0.42699999999999999,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.73999999999999999,"5min":0.96199999999999997,"15min":0.96199999999999997},"last":0.52000000000000002}]},{"osd":5,"last update":"Fri Nov 24 07:59:28 2023","interfaces":[{"interface":"back","average":{"1min":0.46800000000000003,"5min":0.46300000000000002,"15min":0.45700000000000002},"min":{"1min":0.34999999999999998,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":0.60499999999999998,"5min":0.70499999999999996,"15min":0.70499999999999996},"last":0.35999999999999999},{"interface":"front","average":{"1min":0.56599999999999995,"5min":0.60399999999999998,"15min":0.61399999999999999},"min":{"1min":0.41599999999999998,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.73699999999999999,"5min":0.86399999999999999,"15min":0.86399999999999999},"last":0.505}]},{"osd":6,"last update":"Fri Nov 24 07:59:28 2023","interfaces":[{"interface":"back","average":{"1min":0.53900000000000003,"5min":0.57299999999999995,"15min":0.58099999999999996},"min":{"1min":0.33000000000000002,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.88600000000000001,"5min":0.88600000000000001,"15min":0.88600000000000001},"last":0.59499999999999997},{"interface":"front","average":{"1min":0.56899999999999995,"5min":0.59999999999999998,"15min":0.60899999999999999},"min":{"1min":0.32100000000000001,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.80300000000000005,"5min":1.095,"15min":1.095},"last":0.49299999999999999}]},{"osd":7,"last update":"Fri Nov 24 07:59:28 2023","interfaces":[{"interface":"back","average":{"1min":0.52000000000000002,"5min":0.56100000000000005,"15min":0.56899999999999995},"min":{"1min":0.36299999999999999,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.69999999999999996,"5min":0.89300000000000002,"15min":0.89300000000000002},"last":0.41399999999999998},{"interface":"front","average":{"1min":0.57799999999999996,"5min":0.63600000000000001,"15min":0.65400000000000003},"min":{"1min":0.39000000000000001,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.74399999999999999,"5min":0.91500000000000004,"15min":0.91500000000000004},"last":0.57099999999999995}]}]},{"osd":5,"up_from":13,"seq":55834574898,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32824,"kb_used_data":1640,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339016,"statfs":{"total":96636764160,"available":96603152384,"internally_reserved":0,"allocated":1679360,"data_stored":1051129,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 07:59:21 2023","interfaces":[{"interface":"back","average":{"1min":0.52500000000000002,"5min":0.54600000000000004,"15min":0.55600000000000005},"min":{"1min":0.311,"5min":0.311,"15min":0.311},"max":{"1min":0.74399999999999999,"5min":1.0109999999999999,"15min":1.0109999999999999},"last":0.59299999999999997},{"interface":"front","average":{"1min":0.53600000000000003,"5min":0.69299999999999995,"15min":0.76400000000000001},"min":{"1min":0.32400000000000001,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.81899999999999995,"5min":5.431,"15min":5.431},"last":0.61299999999999999}]},{"osd":1,"last update":"Fri Nov 24 07:59:21 2023","interfaces":[{"interface":"back","average":{"1min":0.52600000000000002,"5min":0.54200000000000004,"15min":0.55300000000000005},"min":{"1min":0.35399999999999998,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":0.69099999999999995,"5min":1.016,"15min":1.016},"last":0.54000000000000004},{"interface":"front","average":{"1min":0.56399999999999995,"5min":0.56399999999999995,"15min":0.56799999999999995},"min":{"1min":0.34000000000000002,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":0.80100000000000005,"5min":0.96499999999999997,"15min":0.96499999999999997},"last":0.67900000000000005}]},{"osd":2,"last update":"Fri Nov 24 07:59:21 2023","interfaces":[{"interface":"back","average":{"1min":0.58099999999999996,"5min":0.55500000000000005,"15min":0.55200000000000005},"min":{"1min":0.44800000000000001,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.876,"5min":1.163,"15min":1.163},"last":0.50900000000000001},{"interface":"front","average":{"1min":0.55900000000000005,"5min":0.59799999999999998,"15min":0.61899999999999999},"min":{"1min":0.32600000000000001,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.72399999999999998,"5min":1.119,"15min":1.119},"last":0.57599999999999996}]},{"osd":3,"last update":"Fri Nov 24 07:59:21 2023","interfaces":[{"interface":"back","average":{"1min":0.59799999999999998,"5min":0.77700000000000002,"15min":0.86199999999999999},"min":{"1min":0.32400000000000001,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.77100000000000002,"5min":5.609,"15min":5.609},"last":0.64000000000000001},{"interface":"front","average":{"1min":0.64300000000000002,"5min":0.66000000000000003,"15min":0.67000000000000004},"min":{"1min":0.38900000000000001,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.84999999999999998,"5min":1.288,"15min":1.288},"last":0.56100000000000005}]},{"osd":4,"last update":"Fri Nov 24 07:59:21 2023","interfaces":[{"interface":"back","average":{"1min":0.53800000000000003,"5min":0.51600000000000001,"15min":0.51200000000000001},"min":{"1min":0.30599999999999999,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":0.79100000000000004,"5min":0.79100000000000004,"15min":0.79100000000000004},"last":0.32800000000000001},{"interface":"front","average":{"1min":0.58899999999999997,"5min":0.57799999999999996,"15min":0.57399999999999995},"min":{"1min":0.35399999999999998,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":0.82599999999999996,"5min":0.92900000000000005,"15min":0.92900000000000005},"last":0.48199999999999998}]},{"osd":6,"last update":"Fri Nov 24 07:59:21 2023","interfaces":[{"interface":"back","average":{"1min":0.63,"5min":0.58699999999999997,"15min":0.57599999999999996},"min":{"1min":0.434,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.85399999999999998,"5min":0.89700000000000002,"15min":0.89700000000000002},"last":0.36799999999999999},{"interface":"front","average":{"1min":0.59299999999999997,"5min":0.621,"15min":0.623},"min":{"1min":0.376,"5min":0.376,"15min":0.376},"max":{"1min":0.74299999999999999,"5min":1.083,"15min":1.083},"last":0.38900000000000001}]},{"osd":7,"last update":"Fri Nov 24 07:59:21 2023","interfaces":[{"interface":"back","average":{"1min":0.65400000000000003,"5min":0.65000000000000002,"15min":0.64700000000000002},"min":{"1min":0.40699999999999997,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.86099999999999999,"5min":0.96599999999999997,"15min":0.96599999999999997},"last":0.42899999999999999},{"interface":"front","average":{"1min":0.68000000000000005,"5min":0.66100000000000003,"15min":0.65500000000000003},"min":{"1min":0.49099999999999999,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":0.78500000000000003,"5min":0.94199999999999995,"15min":0.94199999999999995},"last":0.40799999999999997}]}]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":674,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1646,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-11-24T08:00:06.178 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2023-11-24T08:00:06.179 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2023-11-24T08:00:06.345 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:06.343+0000 7fd58a463700 1 Processor -- start 2023-11-24T08:00:06.346 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:06.344+0000 7fd58a463700 1 -- start start 2023-11-24T08:00:06.354 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:06.345+0000 7fd58a463700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fd5840b2060 0x7fd58406c0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:00:06.354 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:06.345+0000 7fd58a463700 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fd5840b2440 0x7fd5840b4a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:00:06.354 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:06.345+0000 7fd58a463700 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd5840b3310 con 0x7fd5840b2e10 2023-11-24T08:00:06.354 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:06.345+0000 7fd58a463700 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd5840d0970 con 0x7fd5840b2060 2023-11-24T08:00:06.354 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:06.345+0000 7fd58a463700 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd5840d0c50 con 0x7fd5840b2440 2023-11-24T08:00:06.354 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:06.345+0000 7fd583fff700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fd5840b2060 0x7fd58406c0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:00:06.354 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:06.345+0000 7fd5837fe700 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7fd5840b2e10 0x7fd58406c620 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:59406/0 (socket says 172.21.15.59:59406) 2023-11-24T08:00:06.355 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:06.345+0000 7fd583fff700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fd5840b2060 0x7fd58406c0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:55930/0 (socket says 172.21.15.59:55930) 2023-11-24T08:00:06.355 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:06.345+0000 7fd5837fe700 1 -- 172.21.15.59:0/1773988345 learned_addr learned my addr 172.21.15.59:0/1773988345 (peer_addr_for_me v1:172.21.15.59:0/0) 2023-11-24T08:00:06.355 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:06.345+0000 7fd588a00700 1 --2- 172.21.15.59:0/1773988345 >> v2:172.21.15.76:3300/0 conn(0x7fd5840b2440 0x7fd5840b4a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:00:06.355 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:06.345+0000 7fd583fff700 1 -- 172.21.15.59:0/1773988345 >> v2:172.21.15.76:3300/0 conn(0x7fd5840b2440 msgr2=0x7fd5840b4a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:00:06.355 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:06.345+0000 7fd583fff700 1 --2- 172.21.15.59:0/1773988345 >> v2:172.21.15.76:3300/0 conn(0x7fd5840b2440 0x7fd5840b4a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:00:06.355 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:06.345+0000 7fd583fff700 1 -- 172.21.15.59:0/1773988345 >> v1:172.21.15.59:6789/0 conn(0x7fd5840b2e10 legacy=0x7fd58406c620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:00:06.356 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:06.345+0000 7fd582ffd700 1 -- 172.21.15.59:0/1773988345 <== mon.0 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1149642209 0 0) 0x7fd5840b3310 con 0x7fd5840b2e10 2023-11-24T08:00:06.356 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:06.345+0000 7fd583fff700 1 -- 172.21.15.59:0/1773988345 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd5840d0f30 con 0x7fd5840b2060 2023-11-24T08:00:06.356 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:06.345+0000 7fd583fff700 1 --2- 172.21.15.59:0/1773988345 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd5840b2060 0x7fd58406c0e0 secure :-1 s=READY pgs=1035 cs=0 l=1 rev1=1 crypto rx=0x7fd5840cf700 tx=0x7fd574009b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ce0db5ea9410f922 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:00:06.356 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:06.345+0000 7fd582ffd700 1 -- 172.21.15.59:0/1773988345 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd574011830 con 0x7fd5840b2060 2023-11-24T08:00:06.356 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:06.345+0000 7fd582ffd700 1 -- 172.21.15.59:0/1773988345 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd574011990 con 0x7fd5840b2060 2023-11-24T08:00:06.356 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:06.345+0000 7fd582ffd700 1 -- 172.21.15.59:0/1773988345 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd574011d20 con 0x7fd5840b2060 2023-11-24T08:00:06.356 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:06.346+0000 7fd58a463700 1 -- 172.21.15.59:0/1773988345 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd5840b2060 msgr2=0x7fd58406c0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:00:06.357 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:06.346+0000 7fd58a463700 1 --2- 172.21.15.59:0/1773988345 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd5840b2060 0x7fd58406c0e0 secure :-1 s=READY pgs=1035 cs=0 l=1 rev1=1 crypto rx=0x7fd5840cf700 tx=0x7fd574009b20 comp rx=0 tx=0).stop 2023-11-24T08:00:06.357 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:06.346+0000 7fd58a463700 1 -- 172.21.15.59:0/1773988345 shutdown_connections 2023-11-24T08:00:06.357 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:06.346+0000 7fd58a463700 1 --2- 172.21.15.59:0/1773988345 >> v2:172.21.15.76:3300/0 conn(0x7fd5840b2440 0x7fd5840b4a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:00:06.357 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:06.346+0000 7fd58a463700 1 --2- 172.21.15.59:0/1773988345 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd5840b2060 0x7fd58406c0e0 unknown :-1 s=CLOSED pgs=1035 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:00:06.357 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:06.346+0000 7fd58a463700 1 -- 172.21.15.59:0/1773988345 >> 172.21.15.59:0/1773988345 conn(0x7fd5840d1c30 msgr2=0x7fd5840d2040 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:00:06.357 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:06.346+0000 7fd58a463700 1 -- 172.21.15.59:0/1773988345 shutdown_connections 2023-11-24T08:00:06.357 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:06.346+0000 7fd58a463700 1 -- 172.21.15.59:0/1773988345 wait complete. 2023-11-24T08:00:06.357 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:06.347+0000 7fd58a463700 1 Processor -- start 2023-11-24T08:00:06.358 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:06.348+0000 7fd58a463700 1 -- start start 2023-11-24T08:00:06.358 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:06.348+0000 7fd58a463700 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fd5840b2060 0x7fd5841f3e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:00:06.358 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:06.348+0000 7fd58a463700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd5840b2440 0x7fd5840b1070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:00:06.358 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:06.348+0000 7fd58a463700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd5840b2e10 0x7fd5840b6940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:00:06.358 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:06.348+0000 7fd58a463700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5841f7af0 con 0x7fd5840b2e10 2023-11-24T08:00:06.358 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:06.348+0000 7fd58a463700 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5841f6e70 con 0x7fd5840b2440 2023-11-24T08:00:06.358 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:06.348+0000 7fd58a463700 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd5841f7150 con 0x7fd5840b2060 2023-11-24T08:00:06.359 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:06.348+0000 7fd588a00700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd5840b2e10 0x7fd5840b6940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:00:06.359 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:06.349+0000 7fd588a00700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd5840b2e10 0x7fd5840b6940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:55940/0 (socket says 172.21.15.59:55940) 2023-11-24T08:00:06.359 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:06.349+0000 7fd588a00700 1 -- 172.21.15.59:0/988544347 learned_addr learned my addr 172.21.15.59:0/988544347 (peer_addr_for_me v2:172.21.15.59:0/0) 2023-11-24T08:00:06.359 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:06.349+0000 7fd5837fe700 1 --2- 172.21.15.59:0/988544347 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd5840b2440 0x7fd5840b1070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:00:06.359 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:06.349+0000 7fd583fff700 1 --2- 172.21.15.59:0/988544347 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fd5840b2060 0x7fd5841f3e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:00:06.359 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:06.349+0000 7fd588a00700 1 -- 172.21.15.59:0/988544347 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fd5840b2060 msgr2=0x7fd5841f3e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:00:06.359 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:06.349+0000 7fd588a00700 1 --2- 172.21.15.59:0/988544347 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fd5840b2060 0x7fd5841f3e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:00:06.359 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:06.349+0000 7fd588a00700 1 -- 172.21.15.59:0/988544347 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd5840b2440 msgr2=0x7fd5840b1070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:00:06.359 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:06.349+0000 7fd588a00700 1 --2- 172.21.15.59:0/988544347 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd5840b2440 0x7fd5840b1070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:00:06.359 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:06.349+0000 7fd588a00700 1 -- 172.21.15.59:0/988544347 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd5740094a0 con 0x7fd5840b2e10 2023-11-24T08:00:06.360 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:06.349+0000 7fd5837fe700 1 --2- 172.21.15.59:0/988544347 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd5840b2440 0x7fd5840b1070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-24T08:00:06.360 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:06.349+0000 7fd588a00700 1 --2- 172.21.15.59:0/988544347 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd5840b2e10 0x7fd5840b6940 secure :-1 s=READY pgs=1036 cs=0 l=1 rev1=1 crypto rx=0x7fd57c00ef90 tx=0x7fd57c00c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:00:06.360 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:06.349+0000 7fd580ff9700 1 -- 172.21.15.59:0/988544347 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd57c009ca0 con 0x7fd5840b2e10 2023-11-24T08:00:06.360 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:06.349+0000 7fd580ff9700 1 -- 172.21.15.59:0/988544347 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd57c00f040 con 0x7fd5840b2e10 2023-11-24T08:00:06.360 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:06.349+0000 7fd58a463700 1 -- 172.21.15.59:0/988544347 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd5841f7430 con 0x7fd5840b2e10 2023-11-24T08:00:06.360 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:06.349+0000 7fd580ff9700 1 -- 172.21.15.59:0/988544347 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd57c014480 con 0x7fd5840b2e10 2023-11-24T08:00:06.360 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:06.349+0000 7fd58a463700 1 -- 172.21.15.59:0/988544347 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd5841f43c0 con 0x7fd5840b2e10 2023-11-24T08:00:06.360 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:06.350+0000 7fd58a463700 1 -- 172.21.15.59:0/988544347 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd5841f7c30 con 0x7fd5840b2e10 2023-11-24T08:00:06.360 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:06.351+0000 7fd580ff9700 1 -- 172.21.15.59:0/988544347 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 11) v1 ==== 89741+0+0 (secure 0 0 0) 0x7fd57c0145e0 con 0x7fd5840b2e10 2023-11-24T08:00:06.360 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:06.351+0000 7fd580ff9700 1 --2- 172.21.15.59:0/988544347 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7fd57006b470 0x7fd57006d930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:00:06.360 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:06.351+0000 7fd580ff9700 1 -- 172.21.15.59:0/988544347 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fd57c08ad70 con 0x7fd5840b2e10 2023-11-24T08:00:06.361 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:06.351+0000 7fd583fff700 1 --2- 172.21.15.59:0/988544347 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7fd57006b470 0x7fd57006d930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:00:06.361 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:06.352+0000 7fd583fff700 1 --2- 172.21.15.59:0/988544347 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7fd57006b470 0x7fd57006d930 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7fd5740090b0 tx=0x7fd574008030 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:00:06.361 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:06.356+0000 7fd580ff9700 1 -- 172.21.15.59:0/988544347 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fd5841f7c30 con 0x7fd5840b2e10 2023-11-24T08:00:06.642 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:06.641+0000 7fd58a463700 1 -- 172.21.15.59:0/988544347 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7fd5841f7c30 con 0x7fd5840b2e10 2023-11-24T08:00:06.646 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:06.645+0000 7fd580ff9700 1 -- 172.21.15.59:0/988544347 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v17) v1 ==== 190+0+147 (secure 0 0 0) 0x7fd5841f7c30 con 0x7fd5840b2e10 2023-11-24T08:00:06.646 INFO:teuthology.orchestra.run.smithi059.stdout:[client.0] 2023-11-24T08:00:06.646 INFO:teuthology.orchestra.run.smithi059.stdout: key = AQD1VmBlxPkpBhAAR841hRsF6QtmzFgJIrVZ9w== 2023-11-24T08:00:06.647 INFO:teuthology.orchestra.run.smithi059.stdout: caps mds = "allow" 2023-11-24T08:00:06.647 INFO:teuthology.orchestra.run.smithi059.stdout: caps mon = "allow r" 2023-11-24T08:00:06.647 INFO:teuthology.orchestra.run.smithi059.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2023-11-24T08:00:06.649 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:06.647+0000 7fd58a463700 1 -- 172.21.15.59:0/988544347 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7fd57006b470 msgr2=0x7fd57006d930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:00:06.649 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:06.647+0000 7fd58a463700 1 --2- 172.21.15.59:0/988544347 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7fd57006b470 0x7fd57006d930 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7fd5740090b0 tx=0x7fd574008030 comp rx=0 tx=0).stop 2023-11-24T08:00:06.649 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:06.647+0000 7fd58a463700 1 -- 172.21.15.59:0/988544347 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd5840b2e10 msgr2=0x7fd5840b6940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:00:06.649 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:06.647+0000 7fd58a463700 1 --2- 172.21.15.59:0/988544347 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd5840b2e10 0x7fd5840b6940 secure :-1 s=READY pgs=1036 cs=0 l=1 rev1=1 crypto rx=0x7fd57c00ef90 tx=0x7fd57c00c3a0 comp rx=0 tx=0).stop 2023-11-24T08:00:06.650 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:06.647+0000 7fd58a463700 1 -- 172.21.15.59:0/988544347 shutdown_connections 2023-11-24T08:00:06.650 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:06.647+0000 7fd58a463700 1 --2- 172.21.15.59:0/988544347 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7fd57006b470 0x7fd57006d930 unknown :-1 s=CLOSED pgs=479 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:00:06.650 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:06.647+0000 7fd58a463700 1 --2- 172.21.15.59:0/988544347 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd5840b2e10 0x7fd5840b6940 unknown :-1 s=CLOSED pgs=1036 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:00:06.651 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:06.647+0000 7fd58a463700 1 --2- 172.21.15.59:0/988544347 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fd5840b2060 0x7fd5841f3e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:00:06.651 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:06.647+0000 7fd58a463700 1 --2- 172.21.15.59:0/988544347 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd5840b2440 0x7fd5840b1070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:00:06.651 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:06.647+0000 7fd58a463700 1 -- 172.21.15.59:0/988544347 >> 172.21.15.59:0/988544347 conn(0x7fd5840d1c30 msgr2=0x7fd5840701b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:00:06.652 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:06.647+0000 7fd58a463700 1 -- 172.21.15.59:0/988544347 shutdown_connections 2023-11-24T08:00:06.652 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:06.647+0000 7fd58a463700 1 -- 172.21.15.59:0/988544347 wait complete. 2023-11-24T08:00:06.653 INFO:teuthology.orchestra.run.smithi059.stderr:updated caps for client.0 2023-11-24T08:00:06.662 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-24T08:00:06.786 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:06.785+0000 7f810b25d700 1 Processor -- start 2023-11-24T08:00:06.787 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:06.785+0000 7f810b25d700 1 -- start start 2023-11-24T08:00:06.790 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:06.786+0000 7f810b25d700 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f81040d1440 0x7f81040d4960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:00:06.790 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:06.786+0000 7f810b25d700 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f81040d2790 con 0x7f810415dfc0 2023-11-24T08:00:06.791 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:06.786+0000 7f810b25d700 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f81040d2a70 con 0x7f810415e3a0 2023-11-24T08:00:06.791 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:06.786+0000 7f810b25d700 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f81040c7a30 con 0x7f81040d1440 2023-11-24T08:00:06.791 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:06.786+0000 7f8108ff9700 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7f810415dfc0 0x7f81040d0830 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:59418/0 (socket says 172.21.15.59:59418) 2023-11-24T08:00:06.791 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:06.786+0000 7f8108ff9700 1 -- 172.21.15.59:0/2271504009 learned_addr learned my addr 172.21.15.59:0/2271504009 (peer_addr_for_me v1:172.21.15.59:0/0) 2023-11-24T08:00:06.791 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:06.786+0000 7f8103fff700 1 --2- 172.21.15.59:0/2271504009 >> v2:172.21.15.76:3300/0 conn(0x7f81040d1440 0x7f81040d4960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:00:06.791 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:06.786+0000 7f81037fe700 1 -- 172.21.15.59:0/2271504009 <== mon.0 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2170182408 0 0) 0x7f81040d2790 con 0x7f810415dfc0 2023-11-24T08:00:06.791 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:06.786+0000 7f81037fe700 1 -- 172.21.15.59:0/2271504009 --> v1:172.21.15.59:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f80f40035b0 con 0x7f810415dfc0 2023-11-24T08:00:06.791 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:06.786+0000 7f81037fe700 1 -- 172.21.15.59:0/2271504009 <== mon.0 v1:172.21.15.59:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1988731340 0 0) 0x7f80f40035b0 con 0x7f810415dfc0 2023-11-24T08:00:06.792 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:06.786+0000 7f81037fe700 1 -- 172.21.15.59:0/2271504009 >> v2:172.21.15.76:3300/0 conn(0x7f81040d1440 msgr2=0x7f81040d4960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:00:06.792 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:06.786+0000 7f81037fe700 1 --2- 172.21.15.59:0/2271504009 >> v2:172.21.15.76:3300/0 conn(0x7f81040d1440 0x7f81040d4960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:00:06.792 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:06.786+0000 7f81037fe700 1 -- 172.21.15.59:0/2271504009 >> v1:172.21.15.76:6789/0 conn(0x7f810415e3a0 legacy=0x7f81040c8230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:00:06.792 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:06.786+0000 7f81037fe700 1 -- 172.21.15.59:0/2271504009 --> v1:172.21.15.59:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f81040c7d10 con 0x7f810415dfc0 2023-11-24T08:00:06.792 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:06.786+0000 7f81037fe700 1 -- 172.21.15.59:0/2271504009 <== mon.0 v1:172.21.15.59:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1312333566 0 0) 0x7f80f8001670 con 0x7f810415dfc0 2023-11-24T08:00:06.792 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:06.786+0000 7f81037fe700 1 -- 172.21.15.59:0/2271504009 >> v1:172.21.15.59:6789/0 conn(0x7f810415dfc0 legacy=0x7f81040d0830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:00:06.793 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:06.786+0000 7f81037fe700 1 --2- 172.21.15.59:0/2271504009 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f80f4003a00 0x7f80f4003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:00:06.793 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:06.787+0000 7f81037fe700 1 --2- 172.21.15.59:0/2271504009 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f80f4004400 0x7f80f40048d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:00:06.793 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:06.787+0000 7f81037fe700 1 --2- 172.21.15.59:0/2271504009 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f80f40050d0 0x7f80f400ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:00:06.793 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:06.787+0000 7f81037fe700 1 -- 172.21.15.59:0/2271504009 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f81040c7a30 con 0x7f80f4004400 2023-11-24T08:00:06.793 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:06.787+0000 7f81037fe700 1 -- 172.21.15.59:0/2271504009 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80f400c030 con 0x7f80f40050d0 2023-11-24T08:00:06.794 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:06.787+0000 7f81037fe700 1 -- 172.21.15.59:0/2271504009 --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f80f400c1f0 con 0x7f80f4003a00 2023-11-24T08:00:06.794 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:06.787+0000 7f8103fff700 1 --2- 172.21.15.59:0/2271504009 >> v2:172.21.15.76:3300/0 conn(0x7f81040d1440 0x7f81040d4960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-24T08:00:06.794 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:06.787+0000 7f8103fff700 1 --2- 172.21.15.59:0/2271504009 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f80f4004400 0x7f80f40048d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:00:06.794 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:06.787+0000 7f81097fa700 1 --2- 172.21.15.59:0/2271504009 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f80f40050d0 0x7f80f400ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:00:06.794 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:06.787+0000 7f8108ff9700 1 --2- 172.21.15.59:0/2271504009 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f80f4003a00 0x7f80f4003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:00:06.794 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:06.787+0000 7f8103fff700 1 -- 172.21.15.59:0/2271504009 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f80f4003a00 msgr2=0x7f80f4003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:00:06.795 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:06.787+0000 7f8103fff700 1 --2- 172.21.15.59:0/2271504009 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f80f4003a00 0x7f80f4003de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:00:06.795 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:06.787+0000 7f8103fff700 1 -- 172.21.15.59:0/2271504009 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f80f40050d0 msgr2=0x7f80f400ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:00:06.795 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:06.787+0000 7f8103fff700 1 --2- 172.21.15.59:0/2271504009 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f80f40050d0 0x7f80f400ba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:00:06.795 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:06.787+0000 7f8103fff700 1 -- 172.21.15.59:0/2271504009 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f80f40035b0 con 0x7f80f4004400 2023-11-24T08:00:06.795 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:06.787+0000 7f81097fa700 1 -- 172.21.15.59:0/2271504009 reap_dead start 2023-11-24T08:00:06.795 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:06.787+0000 7f8103fff700 1 --2- 172.21.15.59:0/2271504009 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f80f4004400 0x7f80f40048d0 secure :-1 s=READY pgs=1038 cs=0 l=1 rev1=1 crypto rx=0x7f80f0009d10 tx=0x7f80f4005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2b7f77bd4d1f038c server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:00:06.795 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:06.787+0000 7f81037fe700 1 -- 172.21.15.59:0/2271504009 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80f0009e60 con 0x7f80f4004400 2023-11-24T08:00:06.796 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:06.787+0000 7f81037fe700 1 -- 172.21.15.59:0/2271504009 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f80f000e9a0 con 0x7f80f4004400 2023-11-24T08:00:06.796 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:06.787+0000 7f81037fe700 1 -- 172.21.15.59:0/2271504009 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80f000ed30 con 0x7f80f4004400 2023-11-24T08:00:06.796 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:06.788+0000 7f810b25d700 1 -- 172.21.15.59:0/2271504009 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f80f4004400 msgr2=0x7f80f40048d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:00:06.796 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:06.788+0000 7f810b25d700 1 --2- 172.21.15.59:0/2271504009 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f80f4004400 0x7f80f40048d0 secure :-1 s=READY pgs=1038 cs=0 l=1 rev1=1 crypto rx=0x7f80f0009d10 tx=0x7f80f4005880 comp rx=0 tx=0).stop 2023-11-24T08:00:06.796 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:06.788+0000 7f810b25d700 1 -- 172.21.15.59:0/2271504009 shutdown_connections 2023-11-24T08:00:06.796 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:06.788+0000 7f810b25d700 1 --2- 172.21.15.59:0/2271504009 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f80f4004400 0x7f80f40048d0 unknown :-1 s=CLOSED pgs=1038 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:00:06.796 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:06.788+0000 7f810b25d700 1 -- 172.21.15.59:0/2271504009 >> 172.21.15.59:0/2271504009 conn(0x7f81040d3e40 msgr2=0x7f81040d4250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:00:06.796 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:06.788+0000 7f810b25d700 1 -- 172.21.15.59:0/2271504009 shutdown_connections 2023-11-24T08:00:06.796 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:06.788+0000 7f810b25d700 1 -- 172.21.15.59:0/2271504009 wait complete. 2023-11-24T08:00:06.799 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:06.789+0000 7f810b25d700 1 Processor -- start 2023-11-24T08:00:06.799 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:06.789+0000 7f810b25d700 1 -- start start 2023-11-24T08:00:06.799 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:06.790+0000 7f810b25d700 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f80f4004400 0x7f810414e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:00:06.799 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:06.790+0000 7f810b25d700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f810414eaa0 0x7f81040b9250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:00:06.799 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:06.790+0000 7f810b25d700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f810415a5e0 0x7f810415b6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:00:06.799 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:06.790+0000 7f810b25d700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f81040b9ba0 con 0x7f810415a5e0 2023-11-24T08:00:06.800 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:06.790+0000 7f810b25d700 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f810414e830 con 0x7f810414eaa0 2023-11-24T08:00:06.800 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:06.790+0000 7f810b25d700 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f81040cc0c0 con 0x7f80f4004400 2023-11-24T08:00:06.800 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:06.790+0000 7f81097fa700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f810415a5e0 0x7f810415b6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:00:06.800 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:06.790+0000 7f81097fa700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f810415a5e0 0x7f810415b6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:55960/0 (socket says 172.21.15.59:55960) 2023-11-24T08:00:06.800 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:06.790+0000 7f81097fa700 1 -- 172.21.15.59:0/19928961 learned_addr learned my addr 172.21.15.59:0/19928961 (peer_addr_for_me v2:172.21.15.59:0/0) 2023-11-24T08:00:06.800 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:06.790+0000 7f8103fff700 1 --2- 172.21.15.59:0/19928961 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f810414eaa0 0x7f81040b9250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:00:06.800 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:06.790+0000 7f8108ff9700 1 --2- 172.21.15.59:0/19928961 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f80f4004400 0x7f810414e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:00:06.800 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:06.791+0000 7f81097fa700 1 -- 172.21.15.59:0/19928961 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f80f4004400 msgr2=0x7f810414e2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:00:06.800 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:06.791+0000 7f81097fa700 1 --2- 172.21.15.59:0/19928961 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f80f4004400 0x7f810414e2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:00:06.800 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:06.791+0000 7f81097fa700 1 -- 172.21.15.59:0/19928961 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f810414eaa0 msgr2=0x7f81040b9250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:00:06.800 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:06.791+0000 7f81097fa700 1 --2- 172.21.15.59:0/19928961 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f810414eaa0 0x7f81040b9250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:00:06.801 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:06.791+0000 7f81097fa700 1 -- 172.21.15.59:0/19928961 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f81040cc3a0 con 0x7f810415a5e0 2023-11-24T08:00:06.801 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:06.791+0000 7f8108ff9700 1 --2- 172.21.15.59:0/19928961 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f80f4004400 0x7f810414e2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-24T08:00:06.801 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:06.791+0000 7f81097fa700 1 --2- 172.21.15.59:0/19928961 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f810415a5e0 0x7f810415b6f0 secure :-1 s=READY pgs=1039 cs=0 l=1 rev1=1 crypto rx=0x7f81040b6a30 tx=0x7f80fc00d430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:00:06.801 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:06.791+0000 7f81017fa700 1 -- 172.21.15.59:0/19928961 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80fc003c10 con 0x7f810415a5e0 2023-11-24T08:00:06.801 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:06.791+0000 7f81017fa700 1 -- 172.21.15.59:0/19928961 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f80fc003d70 con 0x7f810415a5e0 2023-11-24T08:00:06.801 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:06.791+0000 7f810b25d700 1 -- 172.21.15.59:0/19928961 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f81040cc680 con 0x7f810415a5e0 2023-11-24T08:00:06.801 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:06.791+0000 7f81017fa700 1 -- 172.21.15.59:0/19928961 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80fc00db30 con 0x7f810415a5e0 2023-11-24T08:00:06.801 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:06.791+0000 7f810b25d700 1 -- 172.21.15.59:0/19928961 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f81040b9790 con 0x7f810415a5e0 2023-11-24T08:00:06.801 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:06.792+0000 7f810b25d700 1 -- 172.21.15.59:0/19928961 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f81040b9920 con 0x7f810415a5e0 2023-11-24T08:00:06.801 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:06.792+0000 7f81017fa700 1 -- 172.21.15.59:0/19928961 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 11) v1 ==== 89741+0+0 (secure 0 0 0) 0x7f80fc00dc90 con 0x7f810415a5e0 2023-11-24T08:00:06.801 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:06.793+0000 7f81017fa700 1 --2- 172.21.15.59:0/19928961 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f80e806b470 0x7f80e806d930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:00:06.802 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:06.793+0000 7f81017fa700 1 -- 172.21.15.59:0/19928961 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f80fc08a2d0 con 0x7f810415a5e0 2023-11-24T08:00:06.802 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:06.793+0000 7f8108ff9700 1 --2- 172.21.15.59:0/19928961 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f80e806b470 0x7f80e806d930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:00:06.802 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:06.794+0000 7f8108ff9700 1 --2- 172.21.15.59:0/19928961 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f80e806b470 0x7f80e806d930 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7f80f8000c00 tx=0x7f80f8009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:00:06.802 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:06.798+0000 7f81017fa700 1 -- 172.21.15.59:0/19928961 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f80fc054ba0 con 0x7f810415a5e0 2023-11-24T08:00:07.044 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:07.042+0000 7f810b25d700 1 -- 172.21.15.59:0/19928961 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f81040cc200 con 0x7f810415a5e0 2023-11-24T08:00:07.045 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:07.043+0000 7f81017fa700 1 -- 172.21.15.59:0/19928961 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 53 v53) v1 ==== 94+0+4912 (secure 0 0 0) 0x7f81040cc200 con 0x7f810415a5e0 2023-11-24T08:00:07.046 INFO:teuthology.orchestra.run.smithi059.stdout: 2023-11-24T08:00:07.046 INFO:teuthology.orchestra.run.smithi059.stdout:{"epoch":53,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5876,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.76:6837/1132179319","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6835","nonce":1132179319},{"type":"v1","addr":"172.21.15.76:6837","nonce":1132179319}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":48},{"gid":6487,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.76:6836/453627414","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6834","nonce":453627414},{"type":"v1","addr":"172.21.15.76:6836","nonce":453627414}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":48},{"gid":7035,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6834/4160100816","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6832","nonce":4160100816},{"type":"v1","addr":"172.21.15.59:6834","nonce":4160100816}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":48}],"filesystems":[{"mdsmap":{"epoch":53,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-24T07:59:51.909592+0000","modified":"2023-11-24T07:59:54.948256+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":7038},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7038":{"gid":7038,"name":"c","rank":0,"incarnation":49,"state":"up:active","state_seq":3,"addr":"172.21.15.59:6835/2337335405","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6833","nonce":2337335405},{"type":"v1","addr":"172.21.15.59:6835","nonce":2337335405}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":6}]} 2023-11-24T08:00:07.048 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:07.046+0000 7f810b25d700 1 -- 172.21.15.59:0/19928961 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f80e806b470 msgr2=0x7f80e806d930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:00:07.049 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:07.046+0000 7f810b25d700 1 --2- 172.21.15.59:0/19928961 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f80e806b470 0x7f80e806d930 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7f80f8000c00 tx=0x7f80f8009040 comp rx=0 tx=0).stop 2023-11-24T08:00:07.049 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:07.046+0000 7f810b25d700 1 -- 172.21.15.59:0/19928961 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f810415a5e0 msgr2=0x7f810415b6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:00:07.049 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:07.046+0000 7f810b25d700 1 --2- 172.21.15.59:0/19928961 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f810415a5e0 0x7f810415b6f0 secure :-1 s=READY pgs=1039 cs=0 l=1 rev1=1 crypto rx=0x7f81040b6a30 tx=0x7f80fc00d430 comp rx=0 tx=0).stop 2023-11-24T08:00:07.049 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:07.046+0000 7f810b25d700 1 -- 172.21.15.59:0/19928961 shutdown_connections 2023-11-24T08:00:07.050 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:07.046+0000 7f810b25d700 1 --2- 172.21.15.59:0/19928961 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f80e806b470 0x7f80e806d930 unknown :-1 s=CLOSED pgs=480 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:00:07.050 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:07.046+0000 7f810b25d700 1 --2- 172.21.15.59:0/19928961 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f810415a5e0 0x7f810415b6f0 unknown :-1 s=CLOSED pgs=1039 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:00:07.050 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:07.046+0000 7f810b25d700 1 --2- 172.21.15.59:0/19928961 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f80f4004400 0x7f810414e2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:00:07.050 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:07.046+0000 7f810b25d700 1 --2- 172.21.15.59:0/19928961 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f810414eaa0 0x7f81040b9250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:00:07.051 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:07.046+0000 7f810b25d700 1 -- 172.21.15.59:0/19928961 >> 172.21.15.59:0/19928961 conn(0x7f81040d3e40 msgr2=0x7f81040d09e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:00:07.051 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:07.046+0000 7f810b25d700 1 -- 172.21.15.59:0/19928961 shutdown_connections 2023-11-24T08:00:07.051 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:07.046+0000 7f810b25d700 1 -- 172.21.15.59:0/19928961 wait complete. 2023-11-24T08:00:07.051 INFO:teuthology.orchestra.run.smithi059.stderr:dumped fsmap epoch 53 2023-11-24T08:00:07.462 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 53, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-11-24T07:59:51.909592+0000', 'modified': '2023-11-24T07:59:54.948256+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 7038}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_7038': {'gid': 7038, 'name': 'c', 'rank': 0, 'incarnation': 49, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.59:6835/2337335405', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.59:6833', 'nonce': 2337335405}, {'type': 'v1', 'addr': '172.21.15.59:6835', 'nonce': 2337335405}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [13], 'metadata_pool': 12, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2023-11-24T08:00:07.462 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2023-11-24T08:00:07.463 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2023-11-24T08:00:07.596 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:07.594+0000 7f2444824700 1 Processor -- start 2023-11-24T08:00:07.596 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:07.594+0000 7f2444824700 1 -- start start 2023-11-24T08:00:07.597 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:07.596+0000 7f2444824700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f244006b6e0 0x7f2440168e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:00:07.597 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:07.596+0000 7f2444824700 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f244006dbe0 con 0x7f24400b57b0 2023-11-24T08:00:07.597 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:07.596+0000 7f2444824700 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f24401622e0 con 0x7f24401478c0 2023-11-24T08:00:07.597 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:07.596+0000 7f2444824700 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2440147ca0 con 0x7f244006b6e0 2023-11-24T08:00:07.597 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:07.596+0000 7f243dd9b700 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7f24400b57b0 0x7f244006afd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:59438/0 (socket says 172.21.15.59:59438) 2023-11-24T08:00:07.597 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:07.596+0000 7f243ed9d700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f244006b6e0 0x7f2440168e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:00:07.597 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:07.596+0000 7f243dd9b700 1 -- 172.21.15.59:0/3802218480 learned_addr learned my addr 172.21.15.59:0/3802218480 (peer_addr_for_me v1:172.21.15.59:0/0) 2023-11-24T08:00:07.598 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:07.596+0000 7f243ed9d700 1 -- 172.21.15.59:0/3802218480 >> v1:172.21.15.76:6789/0 conn(0x7f24401478c0 legacy=0x7f2440161a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:00:07.598 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:07.596+0000 7f243ed9d700 1 -- 172.21.15.59:0/3802218480 >> v1:172.21.15.59:6789/0 conn(0x7f24400b57b0 legacy=0x7f244006afd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:00:07.598 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:07.596+0000 7f243ed9d700 1 -- 172.21.15.59:0/3802218480 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f24401625c0 con 0x7f244006b6e0 2023-11-24T08:00:07.598 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:07.596+0000 7f243d59a700 1 -- 172.21.15.59:0/3802218480 <== mon.0 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2755484758 0 0) 0x7f244006dbe0 con 0x7f24400b57b0 2023-11-24T08:00:07.598 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:07.596+0000 7f243ed9d700 1 --2- 172.21.15.59:0/3802218480 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f244006b6e0 0x7f2440168e20 secure :-1 s=READY pgs=1041 cs=0 l=1 rev1=1 crypto rx=0x7f2438009d90 tx=0x7f243800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a3aaa686a28dab73 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:00:07.599 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:07.596+0000 7f243d59a700 1 -- 172.21.15.59:0/3802218480 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f243800e070 con 0x7f244006b6e0 2023-11-24T08:00:07.599 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:07.596+0000 7f243d59a700 1 -- 172.21.15.59:0/3802218480 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f243800bbd0 con 0x7f244006b6e0 2023-11-24T08:00:07.599 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:07.596+0000 7f243d59a700 1 -- 172.21.15.59:0/3802218480 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2438012430 con 0x7f244006b6e0 2023-11-24T08:00:07.599 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:07.596+0000 7f2444824700 1 -- 172.21.15.59:0/3802218480 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f244006b6e0 msgr2=0x7f2440168e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:00:07.599 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:07.596+0000 7f2444824700 1 --2- 172.21.15.59:0/3802218480 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f244006b6e0 0x7f2440168e20 secure :-1 s=READY pgs=1041 cs=0 l=1 rev1=1 crypto rx=0x7f2438009d90 tx=0x7f243800b330 comp rx=0 tx=0).stop 2023-11-24T08:00:07.599 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:07.597+0000 7f2444824700 1 -- 172.21.15.59:0/3802218480 shutdown_connections 2023-11-24T08:00:07.599 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:07.597+0000 7f2444824700 1 --2- 172.21.15.59:0/3802218480 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f244006b6e0 0x7f2440168e20 unknown :-1 s=CLOSED pgs=1041 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:00:07.600 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:07.597+0000 7f2444824700 1 -- 172.21.15.59:0/3802218480 >> 172.21.15.59:0/3802218480 conn(0x7f24400d0aa0 msgr2=0x7f24400d0eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:00:07.600 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:07.597+0000 7f2444824700 1 -- 172.21.15.59:0/3802218480 shutdown_connections 2023-11-24T08:00:07.600 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:07.597+0000 7f2444824700 1 -- 172.21.15.59:0/3802218480 wait complete. 2023-11-24T08:00:07.602 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:07.598+0000 7f2444824700 1 Processor -- start 2023-11-24T08:00:07.602 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:07.598+0000 7f2444824700 1 -- start start 2023-11-24T08:00:07.602 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:07.599+0000 7f2444824700 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f244006b6e0 0x7f244013f420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:00:07.602 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:07.599+0000 7f2444824700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f24400b57b0 0x7f24401402b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:00:07.602 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:07.599+0000 7f2444824700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f24401478c0 0x7f24401407f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:00:07.602 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:07.599+0000 7f2444824700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24401f41d0 con 0x7f24401478c0 2023-11-24T08:00:07.602 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:07.599+0000 7f2444824700 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24400afa20 con 0x7f24400b57b0 2023-11-24T08:00:07.602 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:07.599+0000 7f2444824700 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f24400af180 con 0x7f244006b6e0 2023-11-24T08:00:07.602 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:07.599+0000 7f243ed9d700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f24401478c0 0x7f24401407f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:00:07.602 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:07.599+0000 7f243ed9d700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f24401478c0 0x7f24401407f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:55972/0 (socket says 172.21.15.59:55972) 2023-11-24T08:00:07.603 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:07.599+0000 7f243ed9d700 1 -- 172.21.15.59:0/1318314911 learned_addr learned my addr 172.21.15.59:0/1318314911 (peer_addr_for_me v2:172.21.15.59:0/0) 2023-11-24T08:00:07.603 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:07.599+0000 7f243dd9b700 1 --2- 172.21.15.59:0/1318314911 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f24400b57b0 0x7f24401402b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:00:07.603 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:07.599+0000 7f243e59c700 1 --2- 172.21.15.59:0/1318314911 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f244006b6e0 0x7f244013f420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:00:07.603 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:07.599+0000 7f243ed9d700 1 -- 172.21.15.59:0/1318314911 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f244006b6e0 msgr2=0x7f244013f420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:00:07.603 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:07.599+0000 7f243ed9d700 1 --2- 172.21.15.59:0/1318314911 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f244006b6e0 0x7f244013f420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:00:07.603 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:07.599+0000 7f243ed9d700 1 -- 172.21.15.59:0/1318314911 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f24400b57b0 msgr2=0x7f24401402b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:00:07.603 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:07.599+0000 7f243ed9d700 1 --2- 172.21.15.59:0/1318314911 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f24400b57b0 0x7f24401402b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:00:07.603 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:07.599+0000 7f243ed9d700 1 -- 172.21.15.59:0/1318314911 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2438009a70 con 0x7f24401478c0 2023-11-24T08:00:07.604 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:07.599+0000 7f243ed9d700 1 --2- 172.21.15.59:0/1318314911 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f24401478c0 0x7f24401407f0 secure :-1 s=READY pgs=1042 cs=0 l=1 rev1=1 crypto rx=0x7f2438014040 tx=0x7f2438004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:00:07.604 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:07.599+0000 7f242effd700 1 -- 172.21.15.59:0/1318314911 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f243801b810 con 0x7f24401478c0 2023-11-24T08:00:07.604 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:07.599+0000 7f242effd700 1 -- 172.21.15.59:0/1318314911 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2438020070 con 0x7f24401478c0 2023-11-24T08:00:07.604 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:07.600+0000 7f242effd700 1 -- 172.21.15.59:0/1318314911 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f243801bbb0 con 0x7f24401478c0 2023-11-24T08:00:07.604 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:07.600+0000 7f2444824700 1 -- 172.21.15.59:0/1318314911 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f24400af460 con 0x7f24401478c0 2023-11-24T08:00:07.604 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:07.600+0000 7f2444824700 1 -- 172.21.15.59:0/1318314911 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f24401f4440 con 0x7f24401478c0 2023-11-24T08:00:07.605 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:07.601+0000 7f242effd700 1 -- 172.21.15.59:0/1318314911 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 11) v1 ==== 89741+0+0 (secure 0 0 0) 0x7f243801bd10 con 0x7f24401478c0 2023-11-24T08:00:07.605 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:07.601+0000 7f242effd700 1 --2- 172.21.15.59:0/1318314911 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f242006b470 0x7f242006d930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:00:07.605 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:07.602+0000 7f242effd700 1 -- 172.21.15.59:0/1318314911 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f2438022070 con 0x7f24401478c0 2023-11-24T08:00:07.605 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:07.602+0000 7f243e59c700 1 --2- 172.21.15.59:0/1318314911 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f242006b470 0x7f242006d930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:00:07.605 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:07.602+0000 7f243e59c700 1 --2- 172.21.15.59:0/1318314911 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f242006b470 0x7f242006d930 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7f2434000f60 tx=0x7f2434009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:00:07.605 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:07.603+0000 7f2450a1db80 1 Processor -- start 2023-11-24T08:00:07.606 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:07.604+0000 7f2450a1db80 1 -- start start 2023-11-24T08:00:07.607 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:07.605+0000 7f2450a1db80 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x561eedf5b080 0x561eedfa1840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:00:07.607 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:07.605+0000 7f2450a1db80 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x561eedf70ed0 0x561eedfa1d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:00:07.607 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:07.605+0000 7f2450a1db80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561eedf58490 0x561eedf84e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:00:07.607 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:07.605+0000 7f2450a1db80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x561eedece770 con 0x561eedf58490 2023-11-24T08:00:07.607 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:07.605+0000 7f2450a1db80 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x561eedeb5750 con 0x561eedf70ed0 2023-11-24T08:00:07.607 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:07.605+0000 7f2450a1db80 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 v1 -- 0x561eedeb3070 con 0x561eedf5b080 2023-11-24T08:00:07.608 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:07.605+0000 7f243ed9d700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561eedf58490 0x561eedf84e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:00:07.608 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:07.605+0000 7f243ed9d700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561eedf58490 0x561eedf84e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:55980/0 (socket says 172.21.15.59:55980) 2023-11-24T08:00:07.608 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:07.605+0000 7f243ed9d700 1 -- 172.21.15.59:0/3490457682 learned_addr learned my addr 172.21.15.59:0/3490457682 (peer_addr_for_me v2:172.21.15.59:0/0) 2023-11-24T08:00:07.608 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:07.605+0000 7f243dd9b700 1 --2- 172.21.15.59:0/3490457682 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x561eedf70ed0 0x561eedfa1d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:00:07.608 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:07.605+0000 7f243e59c700 1 --2- 172.21.15.59:0/3490457682 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x561eedf5b080 0x561eedfa1840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:00:07.608 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:07.605+0000 7f243ed9d700 1 -- 172.21.15.59:0/3490457682 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x561eedf5b080 msgr2=0x561eedfa1840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:00:07.608 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:07.605+0000 7f243ed9d700 1 --2- 172.21.15.59:0/3490457682 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x561eedf5b080 0x561eedfa1840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:00:07.608 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:07.605+0000 7f243ed9d700 1 -- 172.21.15.59:0/3490457682 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x561eedf70ed0 msgr2=0x561eedfa1d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:00:07.608 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:07.605+0000 7f243ed9d700 1 --2- 172.21.15.59:0/3490457682 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x561eedf70ed0 0x561eedfa1d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:00:07.608 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:07.605+0000 7f243ed9d700 1 -- 172.21.15.59:0/3490457682 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561eedd2b5e0 con 0x561eedf58490 2023-11-24T08:00:07.608 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:07.605+0000 7f243ed9d700 1 --2- 172.21.15.59:0/3490457682 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561eedf58490 0x561eedf84e70 secure :-1 s=READY pgs=1043 cs=0 l=1 rev1=1 crypto rx=0x7f2438062430 tx=0x7f243805ae90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=75e21e0848bcc91f server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:00:07.608 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:07.605+0000 7f241ffff700 1 -- 172.21.15.59:0/3490457682 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2438089cf0 con 0x561eedf58490 2023-11-24T08:00:07.608 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:07.605+0000 7f241ffff700 1 -- 172.21.15.59:0/3490457682 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2438062960 con 0x561eedf58490 2023-11-24T08:00:07.609 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:07.605+0000 7f241ffff700 1 -- 172.21.15.59:0/3490457682 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2438059d50 con 0x561eedf58490 2023-11-24T08:00:07.613 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:07.606+0000 7f2450a1db80 1 -- 172.21.15.59:0/3490457682 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561eedf58490 msgr2=0x561eedf84e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:00:07.613 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:07.606+0000 7f2450a1db80 1 --2- 172.21.15.59:0/3490457682 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561eedf58490 0x561eedf84e70 secure :-1 s=READY pgs=1043 cs=0 l=1 rev1=1 crypto rx=0x7f2438062430 tx=0x7f243805ae90 comp rx=0 tx=0).stop 2023-11-24T08:00:07.613 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:07.606+0000 7f2450a1db80 1 -- 172.21.15.59:0/3490457682 shutdown_connections 2023-11-24T08:00:07.613 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:07.606+0000 7f2450a1db80 1 --2- 172.21.15.59:0/3490457682 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561eedf58490 0x561eedf84e70 unknown :-1 s=CLOSED pgs=1043 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:00:07.613 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:07.606+0000 7f2450a1db80 1 --2- 172.21.15.59:0/3490457682 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x561eedf5b080 0x561eedfa1840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:00:07.613 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:07.606+0000 7f2450a1db80 1 --2- 172.21.15.59:0/3490457682 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x561eedf70ed0 0x561eedfa1d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:00:07.613 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:07.606+0000 7f2450a1db80 1 -- 172.21.15.59:0/3490457682 >> 172.21.15.59:0/3490457682 conn(0x561eede89310 msgr2=0x561eedf9fe70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:00:07.613 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:07.606+0000 7f2450a1db80 1 -- 172.21.15.59:0/3490457682 shutdown_connections 2023-11-24T08:00:07.613 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:07.606+0000 7f2450a1db80 1 -- 172.21.15.59:0/3490457682 wait complete. 2023-11-24T08:00:07.613 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:07.607+0000 7f2450a1db80 1 -- start start 2023-11-24T08:00:07.613 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:07.607+0000 7f2450a1db80 1 Processor -- start 2023-11-24T08:00:07.613 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:07.608+0000 7f2450a1db80 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x561eedf58490 0x561eedf9a450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:00:07.614 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:07.608+0000 7f2450a1db80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561eedf5b080 0x561eedfb29c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:00:07.614 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:07.608+0000 7f2450a1db80 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x561eedf70ed0 0x561eedfb2f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:00:07.614 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:07.608+0000 7f2450a1db80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x561eedeae960 con 0x561eedf5b080 2023-11-24T08:00:07.614 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:07.608+0000 7f2450a1db80 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x561eede68510 con 0x561eedf70ed0 2023-11-24T08:00:07.614 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:07.608+0000 7f2450a1db80 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 v1 -- 0x561eede681b0 con 0x561eedf58490 2023-11-24T08:00:07.614 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:07.608+0000 7f2450a1db80 10 client.0 _ll_register_callbacks cb 0x561eedd93a30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-24T08:00:07.614 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:07.608+0000 7f243dd9b700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561eedf5b080 0x561eedfb29c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:00:07.614 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:07.608+0000 7f243dd9b700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561eedf5b080 0x561eedfb29c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:55984/0 (socket says 172.21.15.59:55984) 2023-11-24T08:00:07.614 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:07.608+0000 7f243dd9b700 1 -- 172.21.15.59:0/2467293064 learned_addr learned my addr 172.21.15.59:0/2467293064 (peer_addr_for_me v2:172.21.15.59:0/0) 2023-11-24T08:00:07.614 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:07.608+0000 7f243e59c700 1 --2- 172.21.15.59:0/2467293064 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x561eedf58490 0x561eedf9a450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:00:07.614 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:07.608+0000 7f243ed9d700 1 --2- 172.21.15.59:0/2467293064 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x561eedf70ed0 0x561eedfb2f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:00:07.615 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:07.608+0000 7f243dd9b700 1 -- 172.21.15.59:0/2467293064 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x561eedf58490 msgr2=0x561eedf9a450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:00:07.615 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:07.608+0000 7f243dd9b700 1 --2- 172.21.15.59:0/2467293064 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x561eedf58490 0x561eedf9a450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:00:07.615 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:07.608+0000 7f243dd9b700 1 -- 172.21.15.59:0/2467293064 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x561eedf70ed0 msgr2=0x561eedfb2f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:00:07.615 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:07.608+0000 7f243dd9b700 1 --2- 172.21.15.59:0/2467293064 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x561eedf70ed0 0x561eedfb2f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:00:07.615 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:07.608+0000 7f243dd9b700 1 -- 172.21.15.59:0/2467293064 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f24401f4440 con 0x561eedf5b080 2023-11-24T08:00:07.615 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:07.608+0000 7f243dd9b700 1 --2- 172.21.15.59:0/2467293064 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561eedf5b080 0x561eedfb29c0 secure :-1 s=READY pgs=1044 cs=0 l=1 rev1=1 crypto rx=0x7f2430000f60 tx=0x7f2430001610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9e5660d9f889faf9 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:00:07.615 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:07.608+0000 7f241d7fa700 10 client.0 ms_handle_connect on v2:172.21.15.59:3300/0 2023-11-24T08:00:07.615 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:07.608+0000 7f241d7fa700 1 -- 172.21.15.59:0/2467293064 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2430001cd0 con 0x561eedf5b080 2023-11-24T08:00:07.615 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:07.609+0000 7f241d7fa700 1 -- 172.21.15.59:0/2467293064 <== mon.0 v2:172.21.15.59:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f243000c780 con 0x561eedf5b080 2023-11-24T08:00:07.615 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:07.609+0000 7f2450a1db80 1 -- 172.21.15.59:0/2467293064 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x561eedf4da40 con 0x561eedf5b080 2023-11-24T08:00:07.615 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:07.609+0000 7f241d7fa700 1 -- 172.21.15.59:0/2467293064 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_get_version_reply(handle=1 version=53) v2 ==== 24+0+0 (secure 0 0 0) 0x7f243000c940 con 0x561eedf5b080 2023-11-24T08:00:07.615 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:07.609+0000 7f2450a1db80 10 client.7170 fetch_fsmap learned FSMap version 53 2023-11-24T08:00:07.617 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:07.609+0000 7f2450a1db80 1 -- 172.21.15.59:0/2467293064 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,fsmap=53,monmap=2+}) v3 -- 0x561eedcd7d90 con 0x561eedf5b080 2023-11-24T08:00:07.617 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:07.609+0000 7f241d7fa700 1 -- 172.21.15.59:0/2467293064 <== mon.0 v2:172.21.15.59:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2430024020 con 0x561eedf5b080 2023-11-24T08:00:07.617 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:07.609+0000 7f241d7fa700 1 -- 172.21.15.59:0/2467293064 <== mon.0 v2:172.21.15.59:3300/0 5 ==== fsmap(e 53) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f2430012050 con 0x561eedf5b080 2023-11-24T08:00:07.617 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:07.609+0000 7f2450a1db80 10 client.7170 fetch_fsmap finished waiting for FSMap version 53 2023-11-24T08:00:07.617 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:07.609+0000 7f2450a1db80 10 client.7170 resolve_mds: resolved name 'c' to daemon mds.c 2023-11-24T08:00:07.617 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:07.609+0000 7f2450a1db80 20 client.7170 populate_metadata read hostname 'smithi059' 2023-11-24T08:00:07.618 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:07.609+0000 7f2450a1db80 1 --2- 172.21.15.59:0/2467293064 >> [v2:172.21.15.59:6833/2337335405,v1:172.21.15.59:6835/2337335405] conn(0x561eedf988a0 0x561eedf9bdb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:00:07.618 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:07.609+0000 7f2450a1db80 4 client.7170 mds_command: new command op to 7038 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-11-24T08:00:07.618 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:07.609+0000 7f2450a1db80 1 -- 172.21.15.59:0/2467293064 --> [v2:172.21.15.59:6833/2337335405,v1:172.21.15.59:6835/2337335405] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x561eedf29840 con 0x561eedf988a0 2023-11-24T08:00:07.618 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:07.609+0000 7f243e59c700 1 --2- 172.21.15.59:0/2467293064 >> [v2:172.21.15.59:6833/2337335405,v1:172.21.15.59:6835/2337335405] conn(0x561eedf988a0 0x561eedf9bdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:00:07.618 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:07.609+0000 7f243e59c700 1 --2- 172.21.15.59:0/2467293064 >> [v2:172.21.15.59:6833/2337335405,v1:172.21.15.59:6835/2337335405] conn(0x561eedf988a0 0x561eedf9bdb0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c30afda880fde4c3 server_cookie=7072258d6a37fefd in_seq=0 out_seq=0 2023-11-24T08:00:07.618 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:07.609+0000 7f241d7fa700 10 client.7170 ms_handle_connect on v2:172.21.15.59:6833/2337335405 2023-11-24T08:00:07.618 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:07.610+0000 7f241d7fa700 1 -- 172.21.15.59:0/2467293064 <== mds.0 v2:172.21.15.59:6833/2337335405 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x561eedf29840 con 0x561eedf988a0 2023-11-24T08:00:07.618 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:07.610+0000 7f241d7fa700 10 client.7170 handle_command_reply: tid=0 2023-11-24T08:00:07.619 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:07.610+0000 7f2450a1db80 1 client.7170 shutdown 2023-11-24T08:00:07.619 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:07.610+0000 7f2450a1db80 1 -- 172.21.15.59:0/2467293064 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561eedf5b080 msgr2=0x561eedfb29c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:00:07.619 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:07.610+0000 7f2450a1db80 1 --2- 172.21.15.59:0/2467293064 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561eedf5b080 0x561eedfb29c0 secure :-1 s=READY pgs=1044 cs=0 l=1 rev1=1 crypto rx=0x7f2430000f60 tx=0x7f2430001610 comp rx=0 tx=0).stop 2023-11-24T08:00:07.619 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:07.610+0000 7f2450a1db80 1 -- 172.21.15.59:0/2467293064 shutdown_connections 2023-11-24T08:00:07.619 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:07.610+0000 7f2450a1db80 1 --2- 172.21.15.59:0/2467293064 >> [v2:172.21.15.59:6833/2337335405,v1:172.21.15.59:6835/2337335405] conn(0x561eedf988a0 0x561eedf9bdb0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:00:07.619 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:07.610+0000 7f2450a1db80 1 --2- 172.21.15.59:0/2467293064 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x561eedf70ed0 0x561eedfb2f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:00:07.619 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:07.610+0000 7f2450a1db80 1 --2- 172.21.15.59:0/2467293064 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x561eedf58490 0x561eedf9a450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:00:07.619 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:07.610+0000 7f241d7fa700 0 client.7170 ms_handle_reset on v2:172.21.15.59:6833/2337335405 2023-11-24T08:00:07.619 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:07.610+0000 7f2450a1db80 1 --2- 172.21.15.59:0/2467293064 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561eedf5b080 0x561eedfb29c0 unknown :-1 s=CLOSED pgs=1044 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:00:07.619 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:07.610+0000 7f2450a1db80 1 -- 172.21.15.59:0/2467293064 >> 172.21.15.59:0/2467293064 conn(0x561eede89310 msgr2=0x561eedfa3c80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:00:07.620 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:07.610+0000 7f2450a1db80 1 -- 172.21.15.59:0/2467293064 shutdown_connections 2023-11-24T08:00:07.620 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:07.610+0000 7f2450a1db80 1 -- 172.21.15.59:0/2467293064 wait complete. 2023-11-24T08:00:07.620 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:07.611+0000 7f2450a1db80 20 client.7170 trim_cache size 0 max 16384 2023-11-24T08:00:07.630 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:07.629+0000 7f2450a1db80 1 Processor -- start 2023-11-24T08:00:07.630 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:07.629+0000 7f2450a1db80 1 -- start start 2023-11-24T08:00:07.633 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:07.629+0000 7f2450a1db80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561eedf58490 0x561eedf9ab30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:00:07.633 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:07.629+0000 7f2450a1db80 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x561eedf5b080 0x561eedf9b070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:00:07.633 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:07.629+0000 7f2450a1db80 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x561eedf70ed0 0x561eedf874b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:00:07.633 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:07.629+0000 7f2450a1db80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x561eedd2d8d0 con 0x561eedf58490 2023-11-24T08:00:07.633 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:07.629+0000 7f2450a1db80 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x561eedf301b0 con 0x561eedf5b080 2023-11-24T08:00:07.633 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:07.629+0000 7f2450a1db80 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 v1 -- 0x561eedf58280 con 0x561eedf70ed0 2023-11-24T08:00:07.633 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:07.630+0000 7f243e59c700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561eedf58490 0x561eedf9ab30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:00:07.633 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:07.630+0000 7f243e59c700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561eedf58490 0x561eedf9ab30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:55990/0 (socket says 172.21.15.59:55990) 2023-11-24T08:00:07.634 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:07.630+0000 7f243e59c700 1 -- 172.21.15.59:0/658149558 learned_addr learned my addr 172.21.15.59:0/658149558 (peer_addr_for_me v2:172.21.15.59:0/0) 2023-11-24T08:00:07.634 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:07.630+0000 7f243ed9d700 1 --2- 172.21.15.59:0/658149558 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x561eedf70ed0 0x561eedf874b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:00:07.634 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:07.630+0000 7f243e59c700 1 -- 172.21.15.59:0/658149558 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x561eedf70ed0 msgr2=0x561eedf874b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:00:07.634 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:07.630+0000 7f243e59c700 1 --2- 172.21.15.59:0/658149558 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x561eedf70ed0 0x561eedf874b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:00:07.634 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:07.630+0000 7f243e59c700 1 -- 172.21.15.59:0/658149558 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x561eedf5b080 msgr2=0x561eedf9b070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:00:07.634 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:07.630+0000 7f243dd9b700 1 --2- 172.21.15.59:0/658149558 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x561eedf5b080 0x561eedf9b070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:00:07.634 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:07.630+0000 7f243e59c700 1 --2- 172.21.15.59:0/658149558 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x561eedf5b080 0x561eedf9b070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:00:07.634 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:07.630+0000 7f243e59c700 1 -- 172.21.15.59:0/658149558 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561eedcc0c10 con 0x561eedf58490 2023-11-24T08:00:07.634 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:07.630+0000 7f243ed9d700 1 --2- 172.21.15.59:0/658149558 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x561eedf70ed0 0x561eedf874b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T08:00:07.634 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:07.630+0000 7f243dd9b700 1 --2- 172.21.15.59:0/658149558 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x561eedf5b080 0x561eedf9b070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T08:00:07.634 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:07.630+0000 7f243e59c700 1 --2- 172.21.15.59:0/658149558 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561eedf58490 0x561eedf9ab30 secure :-1 s=READY pgs=1045 cs=0 l=1 rev1=1 crypto rx=0x7f2434007f70 tx=0x7f2434012000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d361c7e723926716 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:00:07.634 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:07.630+0000 7f242cff9700 1 -- 172.21.15.59:0/658149558 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24340260e0 con 0x561eedf58490 2023-11-24T08:00:07.635 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:07.630+0000 7f242cff9700 1 -- 172.21.15.59:0/658149558 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2434026240 con 0x561eedf58490 2023-11-24T08:00:07.635 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:07.630+0000 7f242cff9700 1 -- 172.21.15.59:0/658149558 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24340263b0 con 0x561eedf58490 2023-11-24T08:00:07.635 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:07.631+0000 7f2450a1db80 1 -- 172.21.15.59:0/658149558 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561eedf58490 msgr2=0x561eedf9ab30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:00:07.635 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:07.631+0000 7f2450a1db80 1 --2- 172.21.15.59:0/658149558 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561eedf58490 0x561eedf9ab30 secure :-1 s=READY pgs=1045 cs=0 l=1 rev1=1 crypto rx=0x7f2434007f70 tx=0x7f2434012000 comp rx=0 tx=0).stop 2023-11-24T08:00:07.635 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:07.631+0000 7f2450a1db80 1 -- 172.21.15.59:0/658149558 shutdown_connections 2023-11-24T08:00:07.635 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:07.631+0000 7f2450a1db80 1 --2- 172.21.15.59:0/658149558 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x561eedf70ed0 0x561eedf874b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:00:07.635 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:07.631+0000 7f2450a1db80 1 --2- 172.21.15.59:0/658149558 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561eedf58490 0x561eedf9ab30 unknown :-1 s=CLOSED pgs=1045 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:00:07.635 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:07.631+0000 7f2450a1db80 1 --2- 172.21.15.59:0/658149558 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x561eedf5b080 0x561eedf9b070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:00:07.636 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:07.631+0000 7f2450a1db80 1 -- 172.21.15.59:0/658149558 >> 172.21.15.59:0/658149558 conn(0x561eede89310 msgr2=0x561eedf4ecb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:00:07.636 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:07.631+0000 7f2450a1db80 1 -- 172.21.15.59:0/658149558 shutdown_connections 2023-11-24T08:00:07.636 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:07.631+0000 7f2450a1db80 1 -- 172.21.15.59:0/658149558 wait complete. 2023-11-24T08:00:07.636 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:07.631+0000 7f2450a1db80 1 -- start start 2023-11-24T08:00:07.636 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:07.631+0000 7f2450a1db80 1 Processor -- start 2023-11-24T08:00:07.641 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:07.632+0000 7f2450a1db80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561eedf58490 0x561eedfaeaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:00:07.641 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:07.632+0000 7f2450a1db80 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x561eedf5b080 0x561eee00c550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:00:07.641 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:07.632+0000 7f2450a1db80 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x561eedf70ed0 0x561eedf87270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:00:07.641 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:07.632+0000 7f2450a1db80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x561eedfb3250 con 0x561eedf58490 2023-11-24T08:00:07.641 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:07.632+0000 7f2450a1db80 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x561eedf87d60 con 0x561eedf5b080 2023-11-24T08:00:07.641 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:07.632+0000 7f2450a1db80 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 v1 -- 0x561eedfa2fd0 con 0x561eedf70ed0 2023-11-24T08:00:07.641 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:07.632+0000 7f2450a1db80 10 client.0 _ll_register_callbacks cb 0x561eedd93a30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-24T08:00:07.641 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:07.632+0000 7f243e59c700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561eedf58490 0x561eedfaeaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:00:07.641 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:07.632+0000 7f243e59c700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561eedf58490 0x561eedfaeaa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:55996/0 (socket says 172.21.15.59:55996) 2023-11-24T08:00:07.641 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:07.632+0000 7f243e59c700 1 -- 172.21.15.59:0/2296741047 learned_addr learned my addr 172.21.15.59:0/2296741047 (peer_addr_for_me v2:172.21.15.59:0/0) 2023-11-24T08:00:07.642 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:07.633+0000 7f243ed9d700 1 --2- 172.21.15.59:0/2296741047 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x561eedf70ed0 0x561eedf87270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:00:07.642 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:07.633+0000 7f243dd9b700 1 --2- 172.21.15.59:0/2296741047 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x561eedf5b080 0x561eee00c550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:00:07.642 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:07.633+0000 7f243e59c700 1 -- 172.21.15.59:0/2296741047 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x561eedf70ed0 msgr2=0x561eedf87270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:00:07.642 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:07.633+0000 7f243e59c700 1 --2- 172.21.15.59:0/2296741047 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x561eedf70ed0 0x561eedf87270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:00:07.642 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:07.633+0000 7f243e59c700 1 -- 172.21.15.59:0/2296741047 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x561eedf5b080 msgr2=0x561eee00c550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:00:07.642 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:07.633+0000 7f243e59c700 1 --2- 172.21.15.59:0/2296741047 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x561eedf5b080 0x561eee00c550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:00:07.642 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:07.633+0000 7f243e59c700 1 -- 172.21.15.59:0/2296741047 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f243400f040 con 0x561eedf58490 2023-11-24T08:00:07.642 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:07.633+0000 7f243dd9b700 1 --2- 172.21.15.59:0/2296741047 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x561eedf5b080 0x561eee00c550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-24T08:00:07.642 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:07.633+0000 7f243e59c700 1 --2- 172.21.15.59:0/2296741047 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561eedf58490 0x561eedfaeaa0 secure :-1 s=READY pgs=1046 cs=0 l=1 rev1=1 crypto rx=0x561eedf88930 tx=0x7f2434000ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9673ddf948173c7e server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:00:07.642 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:07.633+0000 7f241d7fa700 10 client.0 ms_handle_connect on v2:172.21.15.59:3300/0 2023-11-24T08:00:07.642 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:07.633+0000 7f241d7fa700 1 -- 172.21.15.59:0/2296741047 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24340265c0 con 0x561eedf58490 2023-11-24T08:00:07.642 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:07.633+0000 7f241d7fa700 1 -- 172.21.15.59:0/2296741047 <== mon.0 v2:172.21.15.59:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f2434026720 con 0x561eedf58490 2023-11-24T08:00:07.642 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:07.633+0000 7f2450a1db80 1 -- 172.21.15.59:0/2296741047 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f2438062960 con 0x561eedf58490 2023-11-24T08:00:07.643 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:07.633+0000 7f241d7fa700 1 -- 172.21.15.59:0/2296741047 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_get_version_reply(handle=1 version=53) v2 ==== 24+0+0 (secure 0 0 0) 0x7f24340268e0 con 0x561eedf58490 2023-11-24T08:00:07.643 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:07.633+0000 7f2450a1db80 10 client.7176 fetch_fsmap learned FSMap version 53 2023-11-24T08:00:07.643 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:07.633+0000 7f2450a1db80 1 -- 172.21.15.59:0/2296741047 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,fsmap=53,monmap=2+}) v3 -- 0x561eedcfe200 con 0x561eedf58490 2023-11-24T08:00:07.643 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:07.633+0000 7f241d7fa700 1 -- 172.21.15.59:0/2296741047 <== mon.0 v2:172.21.15.59:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2438062960 con 0x561eedf58490 2023-11-24T08:00:07.643 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:07.633+0000 7f241d7fa700 1 -- 172.21.15.59:0/2296741047 <== mon.0 v2:172.21.15.59:3300/0 5 ==== fsmap(e 53) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f243402d070 con 0x561eedf58490 2023-11-24T08:00:07.643 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:07.633+0000 7f2450a1db80 10 client.7176 fetch_fsmap finished waiting for FSMap version 53 2023-11-24T08:00:07.643 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:07.633+0000 7f2450a1db80 10 client.7176 resolve_mds: resolved name 'c' to daemon mds.c 2023-11-24T08:00:07.643 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:07.633+0000 7f2450a1db80 20 client.7176 populate_metadata read hostname 'smithi059' 2023-11-24T08:00:07.643 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:07.633+0000 7f2450a1db80 1 --2- 172.21.15.59:0/2296741047 >> [v2:172.21.15.59:6833/2337335405,v1:172.21.15.59:6835/2337335405] conn(0x561eedfb1a10 0x561eedfb1df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:00:07.643 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:07.633+0000 7f2450a1db80 4 client.7176 mds_command: new command op to 7038 tid=0 multi_id=0 [{"prefix": "status"}] 2023-11-24T08:00:07.643 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:07.633+0000 7f2450a1db80 1 -- 172.21.15.59:0/2296741047 --> [v2:172.21.15.59:6833/2337335405,v1:172.21.15.59:6835/2337335405] -- command(tid 0: {"prefix": "status"}) v1 -- 0x561eede74f80 con 0x561eedfb1a10 2023-11-24T08:00:07.643 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:07.634+0000 7f243dd9b700 1 --2- 172.21.15.59:0/2296741047 >> [v2:172.21.15.59:6833/2337335405,v1:172.21.15.59:6835/2337335405] conn(0x561eedfb1a10 0x561eedfb1df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:00:07.643 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:07.634+0000 7f241d7fa700 10 client.7176 ms_handle_connect on v2:172.21.15.59:6833/2337335405 2023-11-24T08:00:07.644 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:07.634+0000 7f243dd9b700 1 --2- 172.21.15.59:0/2296741047 >> [v2:172.21.15.59:6833/2337335405,v1:172.21.15.59:6835/2337335405] conn(0x561eedfb1a10 0x561eedfb1df0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6476bb75cc40ffd server_cookie=f0e58e3f1a292472 in_seq=0 out_seq=0 2023-11-24T08:00:07.644 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:07.634+0000 7f241d7fa700 1 -- 172.21.15.59:0/2296741047 <== mds.0 v2:172.21.15.59:6833/2337335405 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+325 (crc 0 0 0) 0x561eede74f80 con 0x561eedfb1a10 2023-11-24T08:00:07.644 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:07.634+0000 7f241d7fa700 10 client.7176 handle_command_reply: tid=0 2023-11-24T08:00:07.644 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:07.634+0000 7f2450a1db80 1 client.7176 shutdown 2023-11-24T08:00:07.644 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:07.634+0000 7f2450a1db80 1 -- 172.21.15.59:0/2296741047 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561eedf58490 msgr2=0x561eedfaeaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:00:07.644 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:07.634+0000 7f2450a1db80 1 --2- 172.21.15.59:0/2296741047 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561eedf58490 0x561eedfaeaa0 secure :-1 s=READY pgs=1046 cs=0 l=1 rev1=1 crypto rx=0x561eedf88930 tx=0x7f2434000ed0 comp rx=0 tx=0).stop 2023-11-24T08:00:07.644 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:07.634+0000 7f2450a1db80 1 -- 172.21.15.59:0/2296741047 shutdown_connections 2023-11-24T08:00:07.644 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:07.634+0000 7f2450a1db80 1 --2- 172.21.15.59:0/2296741047 >> [v2:172.21.15.59:6833/2337335405,v1:172.21.15.59:6835/2337335405] conn(0x561eedfb1a10 0x561eedfb1df0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:00:07.644 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:07.634+0000 7f2450a1db80 1 --2- 172.21.15.59:0/2296741047 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x561eedf70ed0 0x561eedf87270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:00:07.644 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:07.634+0000 7f2450a1db80 1 --2- 172.21.15.59:0/2296741047 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561eedf58490 0x561eedfaeaa0 unknown :-1 s=CLOSED pgs=1046 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:00:07.644 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:07.634+0000 7f2450a1db80 1 --2- 172.21.15.59:0/2296741047 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x561eedf5b080 0x561eee00c550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:00:07.644 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:07.634+0000 7f241d7fa700 0 client.7176 ms_handle_reset on v2:172.21.15.59:6833/2337335405 2023-11-24T08:00:07.645 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:07.634+0000 7f2450a1db80 1 -- 172.21.15.59:0/2296741047 >> 172.21.15.59:0/2296741047 conn(0x561eede89310 msgr2=0x561eedf4ecb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:00:07.645 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:07.634+0000 7f2450a1db80 1 -- 172.21.15.59:0/2296741047 shutdown_connections 2023-11-24T08:00:07.645 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:07.634+0000 7f2450a1db80 1 -- 172.21.15.59:0/2296741047 wait complete. 2023-11-24T08:00:07.645 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:07.635+0000 7f2450a1db80 20 client.7176 trim_cache size 0 max 16384 2023-11-24T08:00:07.645 INFO:teuthology.orchestra.run.smithi059.stdout:{ 2023-11-24T08:00:07.645 INFO:teuthology.orchestra.run.smithi059.stdout: "cluster_fsid": "9f0e8741-b8f9-48d5-a2c3-09f45903303d", 2023-11-24T08:00:07.645 INFO:teuthology.orchestra.run.smithi059.stdout: "whoami": 0, 2023-11-24T08:00:07.645 INFO:teuthology.orchestra.run.smithi059.stdout: "id": 7038, 2023-11-24T08:00:07.645 INFO:teuthology.orchestra.run.smithi059.stdout: "want_state": "up:active", 2023-11-24T08:00:07.646 INFO:teuthology.orchestra.run.smithi059.stdout: "state": "up:active", 2023-11-24T08:00:07.646 INFO:teuthology.orchestra.run.smithi059.stdout: "fs_name": "cephfs", 2023-11-24T08:00:07.646 INFO:teuthology.orchestra.run.smithi059.stdout: "rank_uptime": 15.134148543, 2023-11-24T08:00:07.646 INFO:teuthology.orchestra.run.smithi059.stdout: "mdsmap_epoch": 53, 2023-11-24T08:00:07.646 INFO:teuthology.orchestra.run.smithi059.stdout: "osdmap_epoch": 87, 2023-11-24T08:00:07.646 INFO:teuthology.orchestra.run.smithi059.stdout: "osdmap_epoch_barrier": 87, 2023-11-24T08:00:07.646 INFO:teuthology.orchestra.run.smithi059.stdout: "uptime": 21.038240292000001 2023-11-24T08:00:07.646 INFO:teuthology.orchestra.run.smithi059.stdout:} 2023-11-24T08:00:07.647 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:07.635+0000 7f2444824700 1 -- 172.21.15.59:0/1318314911 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f242006b470 msgr2=0x7f242006d930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:00:07.647 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:07.635+0000 7f2444824700 1 --2- 172.21.15.59:0/1318314911 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f242006b470 0x7f242006d930 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7f2434000f60 tx=0x7f2434009040 comp rx=0 tx=0).stop 2023-11-24T08:00:07.647 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:07.635+0000 7f2444824700 1 -- 172.21.15.59:0/1318314911 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f24401478c0 msgr2=0x7f24401407f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:00:07.647 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:07.635+0000 7f2444824700 1 --2- 172.21.15.59:0/1318314911 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f24401478c0 0x7f24401407f0 secure :-1 s=READY pgs=1042 cs=0 l=1 rev1=1 crypto rx=0x7f2438014040 tx=0x7f2438004ad0 comp rx=0 tx=0).stop 2023-11-24T08:00:07.648 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:07.635+0000 7f2444824700 1 -- 172.21.15.59:0/1318314911 shutdown_connections 2023-11-24T08:00:07.648 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:07.635+0000 7f2444824700 1 --2- 172.21.15.59:0/1318314911 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f242006b470 0x7f242006d930 unknown :-1 s=CLOSED pgs=481 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:00:07.648 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:07.635+0000 7f2444824700 1 --2- 172.21.15.59:0/1318314911 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f24401478c0 0x7f24401407f0 unknown :-1 s=CLOSED pgs=1042 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:00:07.648 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:07.635+0000 7f2444824700 1 --2- 172.21.15.59:0/1318314911 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f244006b6e0 0x7f244013f420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:00:07.648 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:07.635+0000 7f2444824700 1 --2- 172.21.15.59:0/1318314911 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f24400b57b0 0x7f24401402b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:00:07.648 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:07.635+0000 7f2444824700 1 -- 172.21.15.59:0/1318314911 >> 172.21.15.59:0/1318314911 conn(0x7f24400d0aa0 msgr2=0x7f2440072830 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:00:07.648 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:07.635+0000 7f2444824700 1 -- 172.21.15.59:0/1318314911 shutdown_connections 2023-11-24T08:00:07.648 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:07.635+0000 7f2444824700 1 -- 172.21.15.59:0/1318314911 wait complete. 2023-11-24T08:00:07.649 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-11-24T08:00:07.775 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:07.773+0000 7f117d0ca700 1 Processor -- start 2023-11-24T08:00:07.775 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:07.773+0000 7f117d0ca700 1 -- start start 2023-11-24T08:00:07.778 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:07.774+0000 7f117d0ca700 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f11780d0080 0x7f11780b5890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:00:07.778 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:07.774+0000 7f117d0ca700 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f1178071bb0 0x7f117806afd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:00:07.778 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:07.774+0000 7f117d0ca700 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1178147bb0 con 0x7f117806b510 2023-11-24T08:00:07.778 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:07.774+0000 7f117d0ca700 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1178072cd0 con 0x7f11780d0080 2023-11-24T08:00:07.778 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:07.774+0000 7f117d0ca700 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f117806b9a0 con 0x7f1178071bb0 2023-11-24T08:00:07.778 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:07.774+0000 7f117659c700 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f1178071bb0 0x7f117806afd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:00:07.779 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:07.774+0000 7f1176d9d700 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f11780d0080 0x7f11780b5890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:00:07.779 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:07.774+0000 7f117659c700 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f1178071bb0 0x7f117806afd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3301/0 says I am v2:172.21.15.59:55444/0 (socket says 172.21.15.59:55444) 2023-11-24T08:00:07.779 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:07.774+0000 7f117659c700 1 -- 172.21.15.59:0/832841807 learned_addr learned my addr 172.21.15.59:0/832841807 (peer_addr_for_me v2:172.21.15.59:0/0) 2023-11-24T08:00:07.779 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:07.775+0000 7f1176d9d700 1 -- 172.21.15.59:0/832841807 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f1178071bb0 msgr2=0x7f117806afd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:00:07.779 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:07.775+0000 7f1176d9d700 1 --2- 172.21.15.59:0/832841807 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f1178071bb0 0x7f117806afd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:00:07.779 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:07.775+0000 7f1176d9d700 1 -- 172.21.15.59:0/832841807 >> v1:172.21.15.76:6789/0 conn(0x7f117806b510 legacy=0x7f1178161b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:00:07.779 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:07.775+0000 7f1175d9b700 1 -- 172.21.15.59:0/832841807 <== mon.1 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3568300709 0 0) 0x7f1178147bb0 con 0x7f117806b510 2023-11-24T08:00:07.779 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:07.775+0000 7f1176d9d700 1 -- 172.21.15.59:0/832841807 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f11780b5dd0 con 0x7f11780d0080 2023-11-24T08:00:07.779 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:07.775+0000 7f1176d9d700 1 --2- 172.21.15.59:0/832841807 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f11780d0080 0x7f11780b5890 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f116c009840 tx=0x7f116c009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d3c912151c6362cc server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:00:07.779 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:07.775+0000 7f1175d9b700 1 -- 172.21.15.59:0/832841807 <== mon.1 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f116c011830 con 0x7f11780d0080 2023-11-24T08:00:07.779 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:07.775+0000 7f1175d9b700 1 -- 172.21.15.59:0/832841807 <== mon.1 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f116c011990 con 0x7f11780d0080 2023-11-24T08:00:07.780 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:07.775+0000 7f1175d9b700 1 -- 172.21.15.59:0/832841807 <== mon.1 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f116c011d20 con 0x7f11780d0080 2023-11-24T08:00:07.780 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:07.776+0000 7f117d0ca700 1 -- 172.21.15.59:0/832841807 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f11780d0080 msgr2=0x7f11780b5890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:00:07.780 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:07.776+0000 7f117d0ca700 1 --2- 172.21.15.59:0/832841807 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f11780d0080 0x7f11780b5890 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f116c009840 tx=0x7f116c009b50 comp rx=0 tx=0).stop 2023-11-24T08:00:07.780 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:07.776+0000 7f117d0ca700 1 -- 172.21.15.59:0/832841807 shutdown_connections 2023-11-24T08:00:07.780 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:07.776+0000 7f117d0ca700 1 --2- 172.21.15.59:0/832841807 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f11780d0080 0x7f11780b5890 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:00:07.780 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:07.776+0000 7f117d0ca700 1 --2- 172.21.15.59:0/832841807 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f1178071bb0 0x7f117806afd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:00:07.780 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:07.776+0000 7f117d0ca700 1 -- 172.21.15.59:0/832841807 >> 172.21.15.59:0/832841807 conn(0x7f11780b27d0 msgr2=0x7f11780b2be0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:00:07.780 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:07.776+0000 7f117d0ca700 1 -- 172.21.15.59:0/832841807 shutdown_connections 2023-11-24T08:00:07.780 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:07.776+0000 7f117d0ca700 1 -- 172.21.15.59:0/832841807 wait complete. 2023-11-24T08:00:07.786 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:07.777+0000 7f117d0ca700 1 Processor -- start 2023-11-24T08:00:07.786 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:07.777+0000 7f117d0ca700 1 -- start start 2023-11-24T08:00:07.786 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:07.778+0000 7f117d0ca700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f117806b510 0x7f1178160c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:00:07.786 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:07.778+0000 7f117d0ca700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1178071bb0 0x7f1178140e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:00:07.786 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:07.778+0000 7f117d0ca700 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f11780d0080 0x7f117815fec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:00:07.786 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:07.778+0000 7f117d0ca700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1178143d30 con 0x7f1178071bb0 2023-11-24T08:00:07.786 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:07.778+0000 7f117d0ca700 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11781611b0 con 0x7f117806b510 2023-11-24T08:00:07.786 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:07.778+0000 7f117d0ca700 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f11781413c0 con 0x7f11780d0080 2023-11-24T08:00:07.786 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:07.778+0000 7f117659c700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1178071bb0 0x7f1178140e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:00:07.787 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:07.778+0000 7f117659c700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1178071bb0 0x7f1178140e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:56010/0 (socket says 172.21.15.59:56010) 2023-11-24T08:00:07.787 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:07.778+0000 7f117659c700 1 -- 172.21.15.59:0/2346897895 learned_addr learned my addr 172.21.15.59:0/2346897895 (peer_addr_for_me v2:172.21.15.59:0/0) 2023-11-24T08:00:07.787 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:07.778+0000 7f117759e700 1 --2- 172.21.15.59:0/2346897895 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f11780d0080 0x7f117815fec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:00:07.787 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:07.778+0000 7f1176d9d700 1 --2- 172.21.15.59:0/2346897895 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f117806b510 0x7f1178160c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:00:07.787 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:07.778+0000 7f117659c700 1 -- 172.21.15.59:0/2346897895 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f11780d0080 msgr2=0x7f117815fec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:00:07.787 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:07.778+0000 7f117659c700 1 --2- 172.21.15.59:0/2346897895 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f11780d0080 0x7f117815fec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:00:07.787 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:07.778+0000 7f117659c700 1 -- 172.21.15.59:0/2346897895 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f117806b510 msgr2=0x7f1178160c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:00:07.787 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:07.778+0000 7f117659c700 1 --2- 172.21.15.59:0/2346897895 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f117806b510 0x7f1178160c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:00:07.787 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:07.778+0000 7f117659c700 1 -- 172.21.15.59:0/2346897895 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f116c0094a0 con 0x7f1178071bb0 2023-11-24T08:00:07.787 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:07.778+0000 7f117759e700 1 --2- 172.21.15.59:0/2346897895 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f11780d0080 0x7f117815fec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-24T08:00:07.787 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:07.779+0000 7f117659c700 1 --2- 172.21.15.59:0/2346897895 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1178071bb0 0x7f1178140e80 secure :-1 s=READY pgs=1047 cs=0 l=1 rev1=1 crypto rx=0x7f116800e910 tx=0x7f116800ec20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:00:07.787 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:07.779+0000 7f115f7fe700 1 -- 172.21.15.59:0/2346897895 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11680096a0 con 0x7f1178071bb0 2023-11-24T08:00:07.787 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:07.779+0000 7f115f7fe700 1 -- 172.21.15.59:0/2346897895 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f116800ee60 con 0x7f1178071bb0 2023-11-24T08:00:07.788 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:07.779+0000 7f117d0ca700 1 -- 172.21.15.59:0/2346897895 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1178143390 con 0x7f1178071bb0 2023-11-24T08:00:07.788 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:07.779+0000 7f115f7fe700 1 -- 172.21.15.59:0/2346897895 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f116800ca30 con 0x7f1178071bb0 2023-11-24T08:00:07.788 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:07.779+0000 7f117d0ca700 1 -- 172.21.15.59:0/2346897895 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f11781613a0 con 0x7f1178071bb0 2023-11-24T08:00:07.788 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:07.780+0000 7f117d0ca700 1 -- 172.21.15.59:0/2346897895 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1178068670 con 0x7f1178071bb0 2023-11-24T08:00:07.788 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:07.780+0000 7f115f7fe700 1 -- 172.21.15.59:0/2346897895 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 11) v1 ==== 89741+0+0 (secure 0 0 0) 0x7f116800cb90 con 0x7f1178071bb0 2023-11-24T08:00:07.788 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:07.781+0000 7f115f7fe700 1 --2- 172.21.15.59:0/2346897895 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f116406b470 0x7f116406d930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:00:07.788 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:07.781+0000 7f115f7fe700 1 -- 172.21.15.59:0/2346897895 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f1168089fc0 con 0x7f1178071bb0 2023-11-24T08:00:07.788 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:07.781+0000 7f1176d9d700 1 --2- 172.21.15.59:0/2346897895 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f116406b470 0x7f116406d930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:00:07.788 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:07.781+0000 7f1176d9d700 1 --2- 172.21.15.59:0/2346897895 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f116406b470 0x7f116406d930 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f116c0090b0 tx=0x7f116c01c040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:00:07.788 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:07.785+0000 7f115f7fe700 1 -- 172.21.15.59:0/2346897895 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f1168054780 con 0x7f1178071bb0 2023-11-24T08:00:08.035 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:08.033+0000 7f117d0ca700 1 -- 172.21.15.59:0/2346897895 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f11780b3510 con 0x7f1178071bb0 2023-11-24T08:00:08.035 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:08.033+0000 7f115f7fe700 1 -- 172.21.15.59:0/2346897895 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v53) v1 ==== 53+0+73 (secure 0 0 0) 0x7f1168057da0 con 0x7f1178071bb0 2023-11-24T08:00:08.035 INFO:teuthology.orchestra.run.smithi059.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-11-24T08:00:08.037 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:08.035+0000 7f117d0ca700 1 -- 172.21.15.59:0/2346897895 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f116406b470 msgr2=0x7f116406d930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:00:08.037 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:08.035+0000 7f117d0ca700 1 --2- 172.21.15.59:0/2346897895 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f116406b470 0x7f116406d930 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f116c0090b0 tx=0x7f116c01c040 comp rx=0 tx=0).stop 2023-11-24T08:00:08.037 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:08.035+0000 7f117d0ca700 1 -- 172.21.15.59:0/2346897895 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1178071bb0 msgr2=0x7f1178140e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:00:08.037 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:08.035+0000 7f117d0ca700 1 --2- 172.21.15.59:0/2346897895 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1178071bb0 0x7f1178140e80 secure :-1 s=READY pgs=1047 cs=0 l=1 rev1=1 crypto rx=0x7f116800e910 tx=0x7f116800ec20 comp rx=0 tx=0).stop 2023-11-24T08:00:08.037 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:08.035+0000 7f117d0ca700 1 -- 172.21.15.59:0/2346897895 shutdown_connections 2023-11-24T08:00:08.037 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:08.035+0000 7f117d0ca700 1 --2- 172.21.15.59:0/2346897895 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f116406b470 0x7f116406d930 unknown :-1 s=CLOSED pgs=482 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:00:08.037 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:08.035+0000 7f117d0ca700 1 --2- 172.21.15.59:0/2346897895 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f11780d0080 0x7f117815fec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:00:08.037 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:08.035+0000 7f117d0ca700 1 --2- 172.21.15.59:0/2346897895 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f117806b510 0x7f1178160c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:00:08.037 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:08.035+0000 7f117d0ca700 1 --2- 172.21.15.59:0/2346897895 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1178071bb0 0x7f1178140e80 unknown :-1 s=CLOSED pgs=1047 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:00:08.037 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:08.035+0000 7f117d0ca700 1 -- 172.21.15.59:0/2346897895 >> 172.21.15.59:0/2346897895 conn(0x7f11780b27d0 msgr2=0x7f11780ad860 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:00:08.038 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:08.035+0000 7f117d0ca700 1 -- 172.21.15.59:0/2346897895 shutdown_connections 2023-11-24T08:00:08.038 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:08.035+0000 7f117d0ca700 1 -- 172.21.15.59:0/2346897895 wait complete. 2023-11-24T08:00:08.049 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2023-11-24T08:00:08.049 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-11-24T08:00:08.049 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi059.front.sepia.ceph.com 2023-11-24T08:00:08.049 INFO:tasks.cephfs.mount:self.client.name = client.0 2023-11-24T08:00:08.049 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-11-24T08:00:08.050 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2023-11-24T08:00:08.050 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2023-11-24T08:00:08.050 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-11-24T08:00:08.050 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-11-24T08:00:08.050 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T08:00:08.050 DEBUG:teuthology.orchestra.run.smithi059:> ip addr 2023-11-24T08:00:08.068 INFO:teuthology.orchestra.run.smithi059.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-11-24T08:00:08.068 INFO:teuthology.orchestra.run.smithi059.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-11-24T08:00:08.068 INFO:teuthology.orchestra.run.smithi059.stdout: inet 127.0.0.1/8 scope host lo 2023-11-24T08:00:08.068 INFO:teuthology.orchestra.run.smithi059.stdout: valid_lft forever preferred_lft forever 2023-11-24T08:00:08.068 INFO:teuthology.orchestra.run.smithi059.stdout: inet6 ::1/128 scope host 2023-11-24T08:00:08.068 INFO:teuthology.orchestra.run.smithi059.stdout: valid_lft forever preferred_lft forever 2023-11-24T08:00:08.068 INFO:teuthology.orchestra.run.smithi059.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-11-24T08:00:08.068 INFO:teuthology.orchestra.run.smithi059.stdout: link/ether 0c:c4:7a:6b:fe:ca brd ff:ff:ff:ff:ff:ff 2023-11-24T08:00:08.068 INFO:teuthology.orchestra.run.smithi059.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-11-24T08:00:08.068 INFO:teuthology.orchestra.run.smithi059.stdout: link/ether 0c:c4:7a:6b:fe:cb brd ff:ff:ff:ff:ff:ff 2023-11-24T08:00:08.069 INFO:teuthology.orchestra.run.smithi059.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2023-11-24T08:00:08.069 INFO:teuthology.orchestra.run.smithi059.stdout: link/ether 0c:c4:7a:bd:19:2a brd ff:ff:ff:ff:ff:ff 2023-11-24T08:00:08.069 INFO:teuthology.orchestra.run.smithi059.stdout: inet 172.21.15.59/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2023-11-24T08:00:08.069 INFO:teuthology.orchestra.run.smithi059.stdout: valid_lft 42159sec preferred_lft 42159sec 2023-11-24T08:00:08.069 INFO:teuthology.orchestra.run.smithi059.stdout: inet6 fe80::ec4:7aff:febd:192a/64 scope link 2023-11-24T08:00:08.069 INFO:teuthology.orchestra.run.smithi059.stdout: valid_lft forever preferred_lft forever 2023-11-24T08:00:08.069 INFO:teuthology.orchestra.run.smithi059.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-11-24T08:00:08.069 INFO:teuthology.orchestra.run.smithi059.stdout: link/ether 0c:c4:7a:bd:19:2b brd ff:ff:ff:ff:ff:ff 2023-11-24T08:00:08.069 INFO:teuthology.orchestra.run.smithi059.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-11-24T08:00:08.069 INFO:teuthology.orchestra.run.smithi059.stdout: link/ether 72:f2:ce:27:f6:18 brd ff:ff:ff:ff:ff:ff 2023-11-24T08:00:08.069 INFO:teuthology.orchestra.run.smithi059.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-11-24T08:00:08.069 INFO:teuthology.orchestra.run.smithi059.stdout: valid_lft forever preferred_lft forever 2023-11-24T08:00:08.069 INFO:teuthology.orchestra.run.smithi059.stdout: inet6 fe80::64c1:62ff:feb9:5b59/64 scope link 2023-11-24T08:00:08.070 INFO:teuthology.orchestra.run.smithi059.stdout: valid_lft forever preferred_lft forever 2023-11-24T08:00:08.070 INFO:teuthology.orchestra.run.smithi059.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-11-24T08:00:08.070 INFO:teuthology.orchestra.run.smithi059.stdout: link/ether 92:6a:7d:e0:5d:9b brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2023-11-24T08:00:08.070 INFO:teuthology.orchestra.run.smithi059.stdout: inet6 fe80::906a:7dff:fee0:5d9b/64 scope link 2023-11-24T08:00:08.070 INFO:teuthology.orchestra.run.smithi059.stdout: valid_lft forever preferred_lft forever 2023-11-24T08:00:08.070 INFO:teuthology.orchestra.run.smithi059.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-11-24T08:00:08.070 INFO:teuthology.orchestra.run.smithi059.stdout: link/ether 72:f2:ce:27:f6:18 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2023-11-24T08:00:08.070 INFO:teuthology.orchestra.run.smithi059.stdout: inet6 fe80::70f2:ceff:fe27:f618/64 scope link 2023-11-24T08:00:08.070 INFO:teuthology.orchestra.run.smithi059.stdout: valid_lft forever preferred_lft forever 2023-11-24T08:00:08.070 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T08:00:08.071 DEBUG:teuthology.orchestra.run.smithi059:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-11-24T08:00:08.135 INFO:teuthology.orchestra.run.smithi059.stdout:1 2023-11-24T08:00:08.137 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T08:00:08.137 DEBUG:teuthology.orchestra.run.smithi059:> route 2023-11-24T08:00:08.197 INFO:teuthology.orchestra.run.smithi059.stdout:Kernel IP routing table 2023-11-24T08:00:08.197 INFO:teuthology.orchestra.run.smithi059.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-11-24T08:00:08.197 INFO:teuthology.orchestra.run.smithi059.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2023-11-24T08:00:08.197 INFO:teuthology.orchestra.run.smithi059.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2023-11-24T08:00:08.197 INFO:teuthology.orchestra.run.smithi059.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-11-24T08:00:08.198 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T08:00:08.198 DEBUG:teuthology.orchestra.run.smithi059:> (cd / && exec bash -c ' 2023-11-24T08:00:08.198 DEBUG:teuthology.orchestra.run.smithi059:> set -e 2023-11-24T08:00:08.198 DEBUG:teuthology.orchestra.run.smithi059:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2023-11-24T08:00:08.199 DEBUG:teuthology.orchestra.run.smithi059:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2023-11-24T08:00:08.199 DEBUG:teuthology.orchestra.run.smithi059:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2023-11-24T08:00:08.199 DEBUG:teuthology.orchestra.run.smithi059:> ') 2023-11-24T08:00:08.305 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T08:00:08.305 DEBUG:teuthology.orchestra.run.smithi059:> ip netns list 2023-11-24T08:00:08.323 INFO:teuthology.orchestra.run.smithi059.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2023-11-24T08:00:08.323 INFO:teuthology.orchestra.run.smithi059.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-11-24T08:00:08.324 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-11-24T08:00:08.324 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T08:00:08.324 DEBUG:teuthology.orchestra.run.smithi059:> sudo ip link set brx.0 up 2023-11-24T08:00:08.391 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-11-24T08:00:08.391 DEBUG:teuthology.orchestra.run.smithi059:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2023-11-24T08:00:08.448 INFO:teuthology.orchestra.run.smithi059.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2023-11-24T08:00:08.449 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-11-24T08:00:08.449 DEBUG:teuthology.orchestra.run.smithi059:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2023-11-24T08:00:08.505 INFO:tasks.cephfs.kernel_mount:mounting using device: 0@.cephfs=/ 2023-11-24T08:00:08.505 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T08:00:08.505 DEBUG:teuthology.orchestra.run.smithi059:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 0@.cephfs=/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-11-24T08:00:08.592 INFO:teuthology.orchestra.run.smithi059.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-11-24T08:00:08.592 INFO:teuthology.orchestra.run.smithi059.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-11-24T08:00:08.624 INFO:teuthology.orchestra.run.smithi059.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-11-24T08:00:08.624 INFO:teuthology.orchestra.run.smithi059.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-11-24T08:00:08.624 INFO:teuthology.orchestra.run.smithi059.stdout:mount.ceph: resolved to: "172.21.15.59:3300,172.21.15.76:3300,172.21.15.76:3301" 2023-11-24T08:00:08.624 INFO:teuthology.orchestra.run.smithi059.stdout:mount.ceph: trying mount with new device syntax: 0@9f0e8741-b8f9-48d5-a2c3-09f45903303d.cephfs=/ 2023-11-24T08:00:08.624 INFO:teuthology.orchestra.run.smithi059.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,ms_mode=prefer-crc,key=0,mon_addr=172.21.15.59:3300/172.21.15.76:3300/172.21.15.76:3301" will pass to kernel 2023-11-24T08:00:08.624 INFO:teuthology.orchestra.run.smithi059.stdout:mount.ceph: v2 mount support check returned 2 2023-11-24T08:00:08.624 INFO:teuthology.orchestra.run.smithi059.stdout:mount.ceph: kernel does not support v2 syntax 2023-11-24T08:00:08.624 INFO:teuthology.orchestra.run.smithi059.stdout:mount.ceph: switching to using v1 address with old syntax 2023-11-24T08:00:08.666 INFO:teuthology.orchestra.run.smithi059.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-11-24T08:00:08.667 INFO:teuthology.orchestra.run.smithi059.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-11-24T08:00:08.667 INFO:teuthology.orchestra.run.smithi059.stdout:mount.ceph: resolved to: "172.21.15.59:3300,172.21.15.76:3300,172.21.15.76:3301" 2023-11-24T08:00:08.667 INFO:teuthology.orchestra.run.smithi059.stdout:mount.ceph: trying mount with new device syntax: 0@9f0e8741-b8f9-48d5-a2c3-09f45903303d.cephfs=/ 2023-11-24T08:00:08.667 INFO:teuthology.orchestra.run.smithi059.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,ms_mode=prefer-crc,key=0,mon_addr=172.21.15.59:3300/172.21.15.76:3300/172.21.15.76:3301" will pass to kernel 2023-11-24T08:00:08.667 INFO:teuthology.orchestra.run.smithi059.stdout:mount.ceph: v2 mount support check returned 2 2023-11-24T08:00:08.667 INFO:teuthology.orchestra.run.smithi059.stdout:mount.ceph: kernel does not support v2 syntax 2023-11-24T08:00:08.667 INFO:teuthology.orchestra.run.smithi059.stdout:mount.ceph: switching to using v1 address with old syntax 2023-11-24T08:00:08.667 INFO:teuthology.orchestra.run.smithi059.stdout:mount.ceph: resolved to: "172.21.15.59:6789,172.21.15.76:6789,172.21.15.76:6790" 2023-11-24T08:00:08.667 INFO:teuthology.orchestra.run.smithi059.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.59:6789,172.21.15.76:6789,172.21.15.76:6790:/ 2023-11-24T08:00:08.667 INFO:teuthology.orchestra.run.smithi059.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,key=0,mds_namespace=cephfs,fsid=9f0e8741-b8f9-48d5-a2c3-09f45903303d" will pass to kernel 2023-11-24T08:00:08.683 INFO:tasks.cephfs.kernel_mount:mount command passed 2023-11-24T08:00:08.683 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T08:00:08.683 DEBUG:teuthology.orchestra.run.smithi059:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2023-11-24T08:00:08.711 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T08:00:08.712 DEBUG:teuthology.orchestra.run.smithi059:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-11-24T08:00:08.795 INFO:teuthology.orchestra.run.smithi059.stdout:client7182 2023-11-24T08:00:08.796 DEBUG:teuthology.orchestra.run.smithi059:> set -ex 2023-11-24T08:00:08.796 DEBUG:teuthology.orchestra.run.smithi059:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-24T08:00:08.857 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-11-24T08:00:09.022 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:09.022+0000 7f065abcd700 1 Processor -- start 2023-11-24T08:00:09.022 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:09.022+0000 7f065abcd700 1 -- start start 2023-11-24T08:00:09.026 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:09.022+0000 7f065abcd700 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f06540ca6e0 0x7f065415b4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:00:09.026 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:09.022+0000 7f065abcd700 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0654072d20 con 0x7f06540cf1e0 2023-11-24T08:00:09.026 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:09.023+0000 7f065abcd700 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0654071020 con 0x7f0654070a80 2023-11-24T08:00:09.026 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:09.023+0000 7f065abcd700 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f065415ba10 con 0x7f06540ca6e0 2023-11-24T08:00:09.026 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:09.023+0000 7f0658969700 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7f06540cf1e0 0x7f06540c9fd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:59494/0 (socket says 172.21.15.59:59494) 2023-11-24T08:00:09.026 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:09.023+0000 7f0658969700 1 -- 172.21.15.59:0/4012370103 learned_addr learned my addr 172.21.15.59:0/4012370103 (peer_addr_for_me v1:172.21.15.59:0/0) 2023-11-24T08:00:09.027 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:09.023+0000 7f06537fe700 1 -- 172.21.15.59:0/4012370103 <== mon.0 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3955937930 0 0) 0x7f0654072d20 con 0x7f06540cf1e0 2023-11-24T08:00:09.027 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:09.023+0000 7f0653fff700 1 --2- 172.21.15.59:0/4012370103 >> v2:172.21.15.76:3300/0 conn(0x7f06540ca6e0 0x7f065415b4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:00:09.027 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:09.023+0000 7f06537fe700 1 -- 172.21.15.59:0/4012370103 --> v1:172.21.15.59:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f06440035b0 con 0x7f06540cf1e0 2023-11-24T08:00:09.027 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:09.023+0000 7f06537fe700 1 -- 172.21.15.59:0/4012370103 <== mon.0 v1:172.21.15.59:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2023087689 0 0) 0x7f06440035b0 con 0x7f06540cf1e0 2023-11-24T08:00:09.027 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:09.023+0000 7f06537fe700 1 -- 172.21.15.59:0/4012370103 >> v2:172.21.15.76:3300/0 conn(0x7f06540ca6e0 msgr2=0x7f065415b4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:00:09.027 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:09.023+0000 7f06537fe700 1 --2- 172.21.15.59:0/4012370103 >> v2:172.21.15.76:3300/0 conn(0x7f06540ca6e0 0x7f065415b4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:00:09.027 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:09.023+0000 7f06537fe700 1 -- 172.21.15.59:0/4012370103 >> v1:172.21.15.76:6789/0 conn(0x7f0654070a80 legacy=0x7f065406d450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:00:09.028 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:09.023+0000 7f06537fe700 1 -- 172.21.15.59:0/4012370103 --> v1:172.21.15.59:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0654071300 con 0x7f06540cf1e0 2023-11-24T08:00:09.028 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:09.023+0000 7f06537fe700 1 -- 172.21.15.59:0/4012370103 <== mon.0 v1:172.21.15.59:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1312333566 0 0) 0x7f0648001670 con 0x7f06540cf1e0 2023-11-24T08:00:09.028 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:09.023+0000 7f06537fe700 1 -- 172.21.15.59:0/4012370103 >> v1:172.21.15.59:6789/0 conn(0x7f06540cf1e0 legacy=0x7f06540c9fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:00:09.028 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:09.023+0000 7f06537fe700 1 --2- 172.21.15.59:0/4012370103 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0644003a00 0x7f0644003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:00:09.028 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:09.023+0000 7f06537fe700 1 --2- 172.21.15.59:0/4012370103 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f0644004850 0x7f0644004cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:00:09.028 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:09.023+0000 7f06537fe700 1 --2- 172.21.15.59:0/4012370103 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0644005210 0x7f064400cb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:00:09.028 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:09.023+0000 7f06537fe700 1 -- 172.21.15.59:0/4012370103 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f065415ba10 con 0x7f0644003a00 2023-11-24T08:00:09.028 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:09.023+0000 7f06537fe700 1 -- 172.21.15.59:0/4012370103 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f064400d1c0 con 0x7f0644005210 2023-11-24T08:00:09.028 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:09.023+0000 7f06537fe700 1 -- 172.21.15.59:0/4012370103 --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f064400d380 con 0x7f0644004850 2023-11-24T08:00:09.028 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:09.023+0000 7f0658969700 1 --2- 172.21.15.59:0/4012370103 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0644003a00 0x7f0644003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:00:09.028 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:09.023+0000 7f065916a700 1 --2- 172.21.15.59:0/4012370103 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0644005210 0x7f064400cb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:00:09.029 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:09.024+0000 7f0653fff700 1 --2- 172.21.15.59:0/4012370103 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f0644004850 0x7f0644004cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:00:09.029 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:09.024+0000 7f0658969700 1 -- 172.21.15.59:0/4012370103 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f0644004850 msgr2=0x7f0644004cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:00:09.029 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:09.024+0000 7f0658969700 1 --2- 172.21.15.59:0/4012370103 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f0644004850 0x7f0644004cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:00:09.029 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:09.024+0000 7f0658969700 1 -- 172.21.15.59:0/4012370103 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0644005210 msgr2=0x7f064400cb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:00:09.029 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:09.024+0000 7f0658969700 1 --2- 172.21.15.59:0/4012370103 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0644005210 0x7f064400cb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:00:09.029 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:09.024+0000 7f065916a700 1 -- 172.21.15.59:0/4012370103 reap_dead start 2023-11-24T08:00:09.029 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:09.024+0000 7f0658969700 1 -- 172.21.15.59:0/4012370103 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f06440035b0 con 0x7f0644003a00 2023-11-24T08:00:09.029 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:09.024+0000 7f0658969700 1 --2- 172.21.15.59:0/4012370103 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0644003a00 0x7f0644003e20 secure :-1 s=READY pgs=1050 cs=0 l=1 rev1=1 crypto rx=0x7f064800c240 tx=0x7f0644005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cbad48e08c6553ce server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:00:09.029 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:09.024+0000 7f06537fe700 1 -- 172.21.15.59:0/4012370103 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0648007c10 con 0x7f0644003a00 2023-11-24T08:00:09.029 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:09.024+0000 7f06537fe700 1 -- 172.21.15.59:0/4012370103 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f064800ce00 con 0x7f0644003a00 2023-11-24T08:00:09.029 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:09.024+0000 7f06537fe700 1 -- 172.21.15.59:0/4012370103 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0648005ba0 con 0x7f0644003a00 2023-11-24T08:00:09.030 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:09.024+0000 7f065abcd700 1 -- 172.21.15.59:0/4012370103 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0644003a00 msgr2=0x7f0644003e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:00:09.030 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:09.024+0000 7f065abcd700 1 --2- 172.21.15.59:0/4012370103 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0644003a00 0x7f0644003e20 secure :-1 s=READY pgs=1050 cs=0 l=1 rev1=1 crypto rx=0x7f064800c240 tx=0x7f0644005880 comp rx=0 tx=0).stop 2023-11-24T08:00:09.030 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:09.025+0000 7f065abcd700 1 -- 172.21.15.59:0/4012370103 shutdown_connections 2023-11-24T08:00:09.030 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:09.025+0000 7f065abcd700 1 --2- 172.21.15.59:0/4012370103 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0644003a00 0x7f0644003e20 unknown :-1 s=CLOSED pgs=1050 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:00:09.030 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:09.025+0000 7f065abcd700 1 -- 172.21.15.59:0/4012370103 >> 172.21.15.59:0/4012370103 conn(0x7f06540cf980 msgr2=0x7f06540cfd60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:00:09.030 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:09.025+0000 7f065abcd700 1 -- 172.21.15.59:0/4012370103 shutdown_connections 2023-11-24T08:00:09.030 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:09.025+0000 7f065abcd700 1 -- 172.21.15.59:0/4012370103 wait complete. 2023-11-24T08:00:09.033 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:09.026+0000 7f065abcd700 1 Processor -- start 2023-11-24T08:00:09.033 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:09.026+0000 7f065abcd700 1 -- start start 2023-11-24T08:00:09.033 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:09.027+0000 7f065abcd700 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f0644003a00 0x7f06540b50a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:00:09.033 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:09.027+0000 7f065abcd700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f06540b3b70 0x7f065406d670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:00:09.033 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:09.027+0000 7f065abcd700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f065414eb00 0x7f06541f1500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:00:09.033 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:09.027+0000 7f065abcd700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06540b55e0 con 0x7f065414eb00 2023-11-24T08:00:09.033 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:09.027+0000 7f065abcd700 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06540b4000 con 0x7f06540b3b70 2023-11-24T08:00:09.033 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:09.027+0000 7f065abcd700 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f06540b42e0 con 0x7f0644003a00 2023-11-24T08:00:09.033 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:09.027+0000 7f065916a700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f065414eb00 0x7f06541f1500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:00:09.033 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:09.027+0000 7f065916a700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f065414eb00 0x7f06541f1500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:56036/0 (socket says 172.21.15.59:56036) 2023-11-24T08:00:09.034 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:09.027+0000 7f065916a700 1 -- 172.21.15.59:0/3707018650 learned_addr learned my addr 172.21.15.59:0/3707018650 (peer_addr_for_me v2:172.21.15.59:0/0) 2023-11-24T08:00:09.034 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:09.027+0000 7f0658969700 1 --2- 172.21.15.59:0/3707018650 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f0644003a00 0x7f06540b50a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:00:09.034 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:09.027+0000 7f0653fff700 1 --2- 172.21.15.59:0/3707018650 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f06540b3b70 0x7f065406d670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:00:09.034 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:09.027+0000 7f065916a700 1 -- 172.21.15.59:0/3707018650 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f0644003a00 msgr2=0x7f06540b50a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:00:09.034 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:09.027+0000 7f065916a700 1 --2- 172.21.15.59:0/3707018650 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f0644003a00 0x7f06540b50a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:00:09.034 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:09.027+0000 7f065916a700 1 -- 172.21.15.59:0/3707018650 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f06540b3b70 msgr2=0x7f065406d670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:00:09.034 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:09.027+0000 7f065916a700 1 --2- 172.21.15.59:0/3707018650 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f06540b3b70 0x7f065406d670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:00:09.034 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:09.027+0000 7f065916a700 1 -- 172.21.15.59:0/3707018650 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f06540b45c0 con 0x7f065414eb00 2023-11-24T08:00:09.035 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:09.027+0000 7f065916a700 1 --2- 172.21.15.59:0/3707018650 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f065414eb00 0x7f06541f1500 secure :-1 s=READY pgs=1051 cs=0 l=1 rev1=1 crypto rx=0x7f06540d0590 tx=0x7f064c00fdc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:00:09.035 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:09.028+0000 7f06517fa700 1 -- 172.21.15.59:0/3707018650 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f064c003c10 con 0x7f065414eb00 2023-11-24T08:00:09.035 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:09.028+0000 7f06517fa700 1 -- 172.21.15.59:0/3707018650 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f064c003d70 con 0x7f065414eb00 2023-11-24T08:00:09.035 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:09.028+0000 7f065abcd700 1 -- 172.21.15.59:0/3707018650 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f06540b48a0 con 0x7f065414eb00 2023-11-24T08:00:09.035 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:09.028+0000 7f06517fa700 1 -- 172.21.15.59:0/3707018650 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f064c005310 con 0x7f065414eb00 2023-11-24T08:00:09.035 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:09.028+0000 7f065abcd700 1 -- 172.21.15.59:0/3707018650 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f065406dbb0 con 0x7f065414eb00 2023-11-24T08:00:09.035 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:09.029+0000 7f065abcd700 1 -- 172.21.15.59:0/3707018650 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f065406dd70 con 0x7f065414eb00 2023-11-24T08:00:09.035 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:09.029+0000 7f06517fa700 1 -- 172.21.15.59:0/3707018650 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 11) v1 ==== 89741+0+0 (secure 0 0 0) 0x7f064c005470 con 0x7f065414eb00 2023-11-24T08:00:09.035 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:09.029+0000 7f06517fa700 1 --2- 172.21.15.59:0/3707018650 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f063806b470 0x7f063806d930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:00:09.035 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:09.029+0000 7f06517fa700 1 -- 172.21.15.59:0/3707018650 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f064c089f00 con 0x7f065414eb00 2023-11-24T08:00:09.035 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:09.030+0000 7f0658969700 1 --2- 172.21.15.59:0/3707018650 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f063806b470 0x7f063806d930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:00:09.036 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:09.030+0000 7f0658969700 1 --2- 172.21.15.59:0/3707018650 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f063806b470 0x7f063806d930 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7f0648000c00 tx=0x7f0648007000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:00:09.036 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:09.034+0000 7f06517fa700 1 -- 172.21.15.59:0/3707018650 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f064c0547d0 con 0x7f065414eb00 2023-11-24T08:00:09.286 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:09.285+0000 7f065abcd700 1 -- 172.21.15.59:0/3707018650 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f0654067f60 con 0x7f065414eb00 2023-11-24T08:00:09.286 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:09.285+0000 7f06517fa700 1 -- 172.21.15.59:0/3707018650 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v53) v1 ==== 53+0+73 (secure 0 0 0) 0x7f064c057df0 con 0x7f065414eb00 2023-11-24T08:00:09.286 INFO:teuthology.orchestra.run.smithi059.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-11-24T08:00:09.290 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:09.288+0000 7f065abcd700 1 -- 172.21.15.59:0/3707018650 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f063806b470 msgr2=0x7f063806d930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:00:09.290 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:09.288+0000 7f065abcd700 1 --2- 172.21.15.59:0/3707018650 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f063806b470 0x7f063806d930 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7f0648000c00 tx=0x7f0648007000 comp rx=0 tx=0).stop 2023-11-24T08:00:09.290 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:09.288+0000 7f065abcd700 1 -- 172.21.15.59:0/3707018650 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f065414eb00 msgr2=0x7f06541f1500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:00:09.291 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:09.288+0000 7f065abcd700 1 --2- 172.21.15.59:0/3707018650 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f065414eb00 0x7f06541f1500 secure :-1 s=READY pgs=1051 cs=0 l=1 rev1=1 crypto rx=0x7f06540d0590 tx=0x7f064c00fdc0 comp rx=0 tx=0).stop 2023-11-24T08:00:09.291 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:09.288+0000 7f065abcd700 1 -- 172.21.15.59:0/3707018650 shutdown_connections 2023-11-24T08:00:09.291 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:09.288+0000 7f065abcd700 1 --2- 172.21.15.59:0/3707018650 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f063806b470 0x7f063806d930 unknown :-1 s=CLOSED pgs=483 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:00:09.291 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:09.288+0000 7f065abcd700 1 --2- 172.21.15.59:0/3707018650 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f065414eb00 0x7f06541f1500 unknown :-1 s=CLOSED pgs=1051 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:00:09.291 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:09.288+0000 7f065abcd700 1 --2- 172.21.15.59:0/3707018650 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f0644003a00 0x7f06540b50a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:00:09.292 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:09.288+0000 7f065abcd700 1 --2- 172.21.15.59:0/3707018650 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f06540b3b70 0x7f065406d670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:00:09.292 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:09.288+0000 7f065abcd700 1 -- 172.21.15.59:0/3707018650 >> 172.21.15.59:0/3707018650 conn(0x7f06540cf980 msgr2=0x7f06540cb2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:00:09.292 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:09.288+0000 7f065abcd700 1 -- 172.21.15.59:0/3707018650 shutdown_connections 2023-11-24T08:00:09.292 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:09.288+0000 7f065abcd700 1 -- 172.21.15.59:0/3707018650 wait complete. 2023-11-24T08:00:09.301 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2023-11-24T08:00:09.301 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-11-24T08:00:09.301 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi076.front.sepia.ceph.com 2023-11-24T08:00:09.301 INFO:tasks.cephfs.mount:self.client.name = client.1 2023-11-24T08:00:09.301 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2023-11-24T08:00:09.301 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2023-11-24T08:00:09.301 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2023-11-24T08:00:09.301 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-11-24T08:00:09.301 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2023-11-24T08:00:09.302 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T08:00:09.302 DEBUG:teuthology.orchestra.run.smithi076:> ip addr 2023-11-24T08:00:09.323 INFO:teuthology.orchestra.run.smithi076.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-11-24T08:00:09.324 INFO:teuthology.orchestra.run.smithi076.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-11-24T08:00:09.324 INFO:teuthology.orchestra.run.smithi076.stdout: inet 127.0.0.1/8 scope host lo 2023-11-24T08:00:09.324 INFO:teuthology.orchestra.run.smithi076.stdout: valid_lft forever preferred_lft forever 2023-11-24T08:00:09.324 INFO:teuthology.orchestra.run.smithi076.stdout: inet6 ::1/128 scope host 2023-11-24T08:00:09.324 INFO:teuthology.orchestra.run.smithi076.stdout: valid_lft forever preferred_lft forever 2023-11-24T08:00:09.324 INFO:teuthology.orchestra.run.smithi076.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-11-24T08:00:09.324 INFO:teuthology.orchestra.run.smithi076.stdout: link/ether 0c:c4:7a:d8:da:66 brd ff:ff:ff:ff:ff:ff 2023-11-24T08:00:09.324 INFO:teuthology.orchestra.run.smithi076.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-11-24T08:00:09.324 INFO:teuthology.orchestra.run.smithi076.stdout: link/ether 0c:c4:7a:d8:da:67 brd ff:ff:ff:ff:ff:ff 2023-11-24T08:00:09.324 INFO:teuthology.orchestra.run.smithi076.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-11-24T08:00:09.324 INFO:teuthology.orchestra.run.smithi076.stdout: link/ether 0c:c4:7a:8f:47:e0 brd ff:ff:ff:ff:ff:ff 2023-11-24T08:00:09.325 INFO:teuthology.orchestra.run.smithi076.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-11-24T08:00:09.325 INFO:teuthology.orchestra.run.smithi076.stdout: link/ether 0c:c4:7a:8f:47:e1 brd ff:ff:ff:ff:ff:ff 2023-11-24T08:00:09.325 INFO:teuthology.orchestra.run.smithi076.stdout: inet 172.21.15.76/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2023-11-24T08:00:09.325 INFO:teuthology.orchestra.run.smithi076.stdout: valid_lft 42190sec preferred_lft 42190sec 2023-11-24T08:00:09.325 INFO:teuthology.orchestra.run.smithi076.stdout: inet6 fe80::a34d:4ce5:8360:8f9/64 scope link noprefixroute 2023-11-24T08:00:09.325 INFO:teuthology.orchestra.run.smithi076.stdout: valid_lft forever preferred_lft forever 2023-11-24T08:00:09.325 INFO:teuthology.orchestra.run.smithi076.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-11-24T08:00:09.325 INFO:teuthology.orchestra.run.smithi076.stdout: link/ether 2a:59:73:ff:f7:95 brd ff:ff:ff:ff:ff:ff 2023-11-24T08:00:09.325 INFO:teuthology.orchestra.run.smithi076.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-11-24T08:00:09.325 INFO:teuthology.orchestra.run.smithi076.stdout: valid_lft forever preferred_lft forever 2023-11-24T08:00:09.325 INFO:teuthology.orchestra.run.smithi076.stdout: inet6 fe80::4c23:44ff:feb9:d9d4/64 scope link 2023-11-24T08:00:09.325 INFO:teuthology.orchestra.run.smithi076.stdout: valid_lft forever preferred_lft forever 2023-11-24T08:00:09.326 INFO:teuthology.orchestra.run.smithi076.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-11-24T08:00:09.326 INFO:teuthology.orchestra.run.smithi076.stdout: link/ether 46:3e:09:af:65:64 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2023-11-24T08:00:09.326 INFO:teuthology.orchestra.run.smithi076.stdout: inet6 fe80::443e:9ff:feaf:6564/64 scope link 2023-11-24T08:00:09.326 INFO:teuthology.orchestra.run.smithi076.stdout: valid_lft forever preferred_lft forever 2023-11-24T08:00:09.326 INFO:teuthology.orchestra.run.smithi076.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-11-24T08:00:09.326 INFO:teuthology.orchestra.run.smithi076.stdout: link/ether 2a:59:73:ff:f7:95 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.3 2023-11-24T08:00:09.326 INFO:teuthology.orchestra.run.smithi076.stdout: inet6 fe80::2859:73ff:feff:f795/64 scope link 2023-11-24T08:00:09.326 INFO:teuthology.orchestra.run.smithi076.stdout: valid_lft forever preferred_lft forever 2023-11-24T08:00:09.326 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T08:00:09.327 DEBUG:teuthology.orchestra.run.smithi076:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-11-24T08:00:09.396 INFO:teuthology.orchestra.run.smithi076.stdout:1 2023-11-24T08:00:09.397 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T08:00:09.398 DEBUG:teuthology.orchestra.run.smithi076:> route 2023-11-24T08:00:09.461 INFO:teuthology.orchestra.run.smithi076.stdout:Kernel IP routing table 2023-11-24T08:00:09.461 INFO:teuthology.orchestra.run.smithi076.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-11-24T08:00:09.461 INFO:teuthology.orchestra.run.smithi076.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-11-24T08:00:09.461 INFO:teuthology.orchestra.run.smithi076.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-11-24T08:00:09.461 INFO:teuthology.orchestra.run.smithi076.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-11-24T08:00:09.461 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T08:00:09.462 DEBUG:teuthology.orchestra.run.smithi076:> (cd / && exec bash -c ' 2023-11-24T08:00:09.462 DEBUG:teuthology.orchestra.run.smithi076:> set -e 2023-11-24T08:00:09.462 DEBUG:teuthology.orchestra.run.smithi076:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-11-24T08:00:09.462 DEBUG:teuthology.orchestra.run.smithi076:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-11-24T08:00:09.462 DEBUG:teuthology.orchestra.run.smithi076:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-11-24T08:00:09.462 DEBUG:teuthology.orchestra.run.smithi076:> ') 2023-11-24T08:00:09.570 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T08:00:09.571 DEBUG:teuthology.orchestra.run.smithi076:> ip netns list 2023-11-24T08:00:09.588 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-ns--home-ubuntu-cephtest-mnt.3 (id: 1) 2023-11-24T08:00:09.589 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2023-11-24T08:00:09.589 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2023-11-24T08:00:09.589 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T08:00:09.590 DEBUG:teuthology.orchestra.run.smithi076:> sudo ip link set brx.0 up 2023-11-24T08:00:09.656 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-11-24T08:00:09.656 DEBUG:teuthology.orchestra.run.smithi076:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2023-11-24T08:00:09.713 INFO:teuthology.orchestra.run.smithi076.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2023-11-24T08:00:09.714 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-11-24T08:00:09.714 DEBUG:teuthology.orchestra.run.smithi076:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2023-11-24T08:00:09.770 INFO:tasks.cephfs.kernel_mount:mounting using device: 1@.cephfs=/ 2023-11-24T08:00:09.771 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T08:00:09.771 DEBUG:teuthology.orchestra.run.smithi076:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 1@.cephfs=/ /home/ubuntu/cephtest/mnt.1 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-11-24T08:00:09.857 INFO:teuthology.orchestra.run.smithi076.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-11-24T08:00:09.857 INFO:teuthology.orchestra.run.smithi076.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-11-24T08:00:09.885 INFO:teuthology.orchestra.run.smithi076.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-11-24T08:00:09.885 INFO:teuthology.orchestra.run.smithi076.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-11-24T08:00:09.886 INFO:teuthology.orchestra.run.smithi076.stdout:mount.ceph: resolved to: "172.21.15.59:3300,172.21.15.76:3300,172.21.15.76:3301" 2023-11-24T08:00:09.886 INFO:teuthology.orchestra.run.smithi076.stdout:mount.ceph: trying mount with new device syntax: 1@9f0e8741-b8f9-48d5-a2c3-09f45903303d.cephfs=/ 2023-11-24T08:00:09.886 INFO:teuthology.orchestra.run.smithi076.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,ms_mode=prefer-crc,key=1,mon_addr=172.21.15.59:3300/172.21.15.76:3300/172.21.15.76:3301" will pass to kernel 2023-11-24T08:00:09.886 INFO:teuthology.orchestra.run.smithi076.stdout:mount.ceph: v2 mount support check returned 2 2023-11-24T08:00:09.886 INFO:teuthology.orchestra.run.smithi076.stdout:mount.ceph: kernel does not support v2 syntax 2023-11-24T08:00:09.887 INFO:teuthology.orchestra.run.smithi076.stdout:mount.ceph: switching to using v1 address with old syntax 2023-11-24T08:00:09.959 INFO:teuthology.orchestra.run.smithi076.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-11-24T08:00:09.960 INFO:teuthology.orchestra.run.smithi076.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-11-24T08:00:09.960 INFO:teuthology.orchestra.run.smithi076.stdout:mount.ceph: resolved to: "172.21.15.59:3300,172.21.15.76:3300,172.21.15.76:3301" 2023-11-24T08:00:09.960 INFO:teuthology.orchestra.run.smithi076.stdout:mount.ceph: trying mount with new device syntax: 1@9f0e8741-b8f9-48d5-a2c3-09f45903303d.cephfs=/ 2023-11-24T08:00:09.960 INFO:teuthology.orchestra.run.smithi076.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,ms_mode=prefer-crc,key=1,mon_addr=172.21.15.59:3300/172.21.15.76:3300/172.21.15.76:3301" will pass to kernel 2023-11-24T08:00:09.960 INFO:teuthology.orchestra.run.smithi076.stdout:mount.ceph: v2 mount support check returned 2 2023-11-24T08:00:09.960 INFO:teuthology.orchestra.run.smithi076.stdout:mount.ceph: kernel does not support v2 syntax 2023-11-24T08:00:09.960 INFO:teuthology.orchestra.run.smithi076.stdout:mount.ceph: switching to using v1 address with old syntax 2023-11-24T08:00:09.960 INFO:teuthology.orchestra.run.smithi076.stdout:mount.ceph: resolved to: "172.21.15.59:6789,172.21.15.76:6789,172.21.15.76:6790" 2023-11-24T08:00:09.960 INFO:teuthology.orchestra.run.smithi076.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.59:6789,172.21.15.76:6789,172.21.15.76:6790:/ 2023-11-24T08:00:09.960 INFO:teuthology.orchestra.run.smithi076.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,key=1,mds_namespace=cephfs,fsid=9f0e8741-b8f9-48d5-a2c3-09f45903303d" will pass to kernel 2023-11-24T08:00:09.963 INFO:tasks.cephfs.kernel_mount:mount command passed 2023-11-24T08:00:09.963 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T08:00:09.963 DEBUG:teuthology.orchestra.run.smithi076:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2023-11-24T08:00:09.994 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T08:00:09.995 DEBUG:teuthology.orchestra.run.smithi076:> (cd /home/ubuntu/cephtest/mnt.1 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-11-24T08:00:10.084 INFO:teuthology.orchestra.run.smithi076.stdout:client7191 2023-11-24T08:00:10.085 DEBUG:teuthology.orchestra.run.smithi076:> set -ex 2023-11-24T08:00:10.085 DEBUG:teuthology.orchestra.run.smithi076:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-24T08:00:10.154 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs flag set enable_multiple true --yes-i-really-mean-it 2023-11-24T08:00:10.301 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:10.300+0000 7f2ea4d09700 1 Processor -- start 2023-11-24T08:00:10.301 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:10.300+0000 7f2ea4d09700 1 -- start start 2023-11-24T08:00:10.308 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:10.301+0000 7f2ea4d09700 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f2ea015bde0 0x7f2ea015c1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:00:10.308 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:10.301+0000 7f2ea4d09700 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f2ea00ce5c0 0x7f2ea00cea70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:00:10.308 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:10.301+0000 7f2ea4d09700 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2ea00d0a90 con 0x7f2ea00cf2b0 2023-11-24T08:00:10.308 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:10.301+0000 7f2ea4d09700 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2ea00cefb0 con 0x7f2ea015bde0 2023-11-24T08:00:10.308 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:10.301+0000 7f2ea4d09700 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2ea006ee10 con 0x7f2ea00ce5c0 2023-11-24T08:00:10.308 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:10.302+0000 7f2e9e59c700 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f2ea015bde0 0x7f2ea015c1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:00:10.308 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:10.302+0000 7f2e9ed9d700 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f2ea00ce5c0 0x7f2ea00cea70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:00:10.308 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:10.302+0000 7f2e9e59c700 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f2ea015bde0 0x7f2ea015c1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.59:43238/0 (socket says 172.21.15.59:43238) 2023-11-24T08:00:10.308 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:10.302+0000 7f2e9ed9d700 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f2ea00ce5c0 0x7f2ea00cea70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3301/0 says I am v2:172.21.15.59:55482/0 (socket says 172.21.15.59:55482) 2023-11-24T08:00:10.308 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:10.302+0000 7f2e9e59c700 1 -- 172.21.15.59:0/412300662 learned_addr learned my addr 172.21.15.59:0/412300662 (peer_addr_for_me v2:172.21.15.59:0/0) 2023-11-24T08:00:10.308 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:10.302+0000 7f2e9dd9b700 1 --1- 172.21.15.59:0/412300662 >> v1:172.21.15.76:6789/0 conn(0x7f2ea00cf2b0 0x7f2ea00d01b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.59:46792/0 (socket says 172.21.15.59:46792) 2023-11-24T08:00:10.309 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:10.302+0000 7f2e9e59c700 1 -- 172.21.15.59:0/412300662 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f2ea00ce5c0 msgr2=0x7f2ea00cea70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:00:10.309 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:10.302+0000 7f2e9e59c700 1 --2- 172.21.15.59:0/412300662 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f2ea00ce5c0 0x7f2ea00cea70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:00:10.309 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:10.302+0000 7f2e9e59c700 1 -- 172.21.15.59:0/412300662 >> v1:172.21.15.76:6789/0 conn(0x7f2ea00cf2b0 legacy=0x7f2ea00d01b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:00:10.309 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:10.302+0000 7f2e9e59c700 1 -- 172.21.15.59:0/412300662 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2ea006f0f0 con 0x7f2ea015bde0 2023-11-24T08:00:10.309 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:10.302+0000 7f2e9d59a700 1 -- 172.21.15.59:0/412300662 <== mon.1 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3833673289 0 0) 0x7f2ea00d0a90 con 0x7f2ea00cf2b0 2023-11-24T08:00:10.309 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:10.302+0000 7f2e9ed9d700 1 --2- 172.21.15.59:0/412300662 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f2ea00ce5c0 0x7f2ea00cea70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-24T08:00:10.309 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:10.302+0000 7f2e9e59c700 1 --2- 172.21.15.59:0/412300662 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2ea015bde0 0x7f2ea015c1c0 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f2e940098f0 tx=0x7f2e94009c00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=949878e479f24384 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:00:10.309 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:10.302+0000 7f2e9d59a700 1 -- 172.21.15.59:0/412300662 <== mon.1 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e94011830 con 0x7f2ea015bde0 2023-11-24T08:00:10.309 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:10.302+0000 7f2e9d59a700 1 -- 172.21.15.59:0/412300662 <== mon.1 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2e94011990 con 0x7f2ea015bde0 2023-11-24T08:00:10.309 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:10.302+0000 7f2e9d59a700 1 -- 172.21.15.59:0/412300662 <== mon.1 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e94011d20 con 0x7f2ea015bde0 2023-11-24T08:00:10.310 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:10.303+0000 7f2ea4d09700 1 -- 172.21.15.59:0/412300662 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2ea015bde0 msgr2=0x7f2ea015c1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:00:10.310 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:10.303+0000 7f2ea4d09700 1 --2- 172.21.15.59:0/412300662 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2ea015bde0 0x7f2ea015c1c0 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f2e940098f0 tx=0x7f2e94009c00 comp rx=0 tx=0).stop 2023-11-24T08:00:10.310 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:10.303+0000 7f2ea4d09700 1 -- 172.21.15.59:0/412300662 shutdown_connections 2023-11-24T08:00:10.310 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:10.303+0000 7f2ea4d09700 1 --2- 172.21.15.59:0/412300662 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f2ea00ce5c0 0x7f2ea00cea70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:00:10.310 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:10.303+0000 7f2ea4d09700 1 --2- 172.21.15.59:0/412300662 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2ea015bde0 0x7f2ea015c1c0 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:00:10.310 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:10.303+0000 7f2ea4d09700 1 -- 172.21.15.59:0/412300662 >> 172.21.15.59:0/412300662 conn(0x7f2ea00b3720 msgr2=0x7f2ea00b3b30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:00:10.310 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:10.303+0000 7f2ea4d09700 1 -- 172.21.15.59:0/412300662 shutdown_connections 2023-11-24T08:00:10.310 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:10.303+0000 7f2ea4d09700 1 -- 172.21.15.59:0/412300662 wait complete. 2023-11-24T08:00:10.310 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:10.304+0000 7f2ea4d09700 1 Processor -- start 2023-11-24T08:00:10.310 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:10.304+0000 7f2ea4d09700 1 -- start start 2023-11-24T08:00:10.310 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:10.305+0000 7f2ea4d09700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2ea00ce5c0 0x7f2ea0158f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:00:10.311 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:10.305+0000 7f2ea4d09700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2ea00cf2b0 0x7f2ea00b6730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:00:10.311 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:10.305+0000 7f2ea4d09700 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f2ea015bde0 0x7f2ea015c830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:00:10.311 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:10.305+0000 7f2ea4d09700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ea0159440 con 0x7f2ea00ce5c0 2023-11-24T08:00:10.311 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:10.305+0000 7f2ea4d09700 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ea00ca080 con 0x7f2ea00cf2b0 2023-11-24T08:00:10.314 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:10.305+0000 7f2ea4d09700 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2ea00b6c70 con 0x7f2ea015bde0 2023-11-24T08:00:10.314 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:10.305+0000 7f2e9e59c700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2ea00ce5c0 0x7f2ea0158f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:00:10.315 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:10.305+0000 7f2e9e59c700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2ea00ce5c0 0x7f2ea0158f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:56044/0 (socket says 172.21.15.59:56044) 2023-11-24T08:00:10.315 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:10.305+0000 7f2e9e59c700 1 -- 172.21.15.59:0/1661527178 learned_addr learned my addr 172.21.15.59:0/1661527178 (peer_addr_for_me v2:172.21.15.59:0/0) 2023-11-24T08:00:10.315 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:10.305+0000 7f2e9ed9d700 1 --2- 172.21.15.59:0/1661527178 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f2ea015bde0 0x7f2ea015c830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:00:10.315 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:10.305+0000 7f2e9dd9b700 1 --2- 172.21.15.59:0/1661527178 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2ea00cf2b0 0x7f2ea00b6730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:00:10.315 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:10.306+0000 7f2e9e59c700 1 -- 172.21.15.59:0/1661527178 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f2ea015bde0 msgr2=0x7f2ea015c830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:00:10.316 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:10.306+0000 7f2e9e59c700 1 --2- 172.21.15.59:0/1661527178 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f2ea015bde0 0x7f2ea015c830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:00:10.316 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:10.306+0000 7f2e9e59c700 1 -- 172.21.15.59:0/1661527178 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2ea00cf2b0 msgr2=0x7f2ea00b6730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:00:10.316 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:10.306+0000 7f2e9e59c700 1 --2- 172.21.15.59:0/1661527178 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2ea00cf2b0 0x7f2ea00b6730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:00:10.316 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:10.306+0000 7f2e9e59c700 1 -- 172.21.15.59:0/1661527178 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2e98009a70 con 0x7f2ea00ce5c0 2023-11-24T08:00:10.316 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:10.306+0000 7f2e9ed9d700 1 --2- 172.21.15.59:0/1661527178 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f2ea015bde0 0x7f2ea015c830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-24T08:00:10.316 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:10.306+0000 7f2e9dd9b700 1 --2- 172.21.15.59:0/1661527178 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2ea00cf2b0 0x7f2ea00b6730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-24T08:00:10.316 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:10.306+0000 7f2e9e59c700 1 --2- 172.21.15.59:0/1661527178 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2ea00ce5c0 0x7f2ea0158f00 secure :-1 s=READY pgs=1053 cs=0 l=1 rev1=1 crypto rx=0x7f2e94009c40 tx=0x7f2e94004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:00:10.317 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:10.306+0000 7f2e8effd700 1 -- 172.21.15.59:0/1661527178 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e940044d0 con 0x7f2ea00ce5c0 2023-11-24T08:00:10.317 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:10.306+0000 7f2e8effd700 1 -- 172.21.15.59:0/1661527178 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2e9401e070 con 0x7f2ea00ce5c0 2023-11-24T08:00:10.317 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:10.306+0000 7f2e8effd700 1 -- 172.21.15.59:0/1661527178 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e940038c0 con 0x7f2ea00ce5c0 2023-11-24T08:00:10.317 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:10.306+0000 7f2ea4d09700 1 -- 172.21.15.59:0/1661527178 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2e94009550 con 0x7f2ea00ce5c0 2023-11-24T08:00:10.317 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:10.306+0000 7f2ea4d09700 1 -- 172.21.15.59:0/1661527178 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2ea00c9750 con 0x7f2ea00ce5c0 2023-11-24T08:00:10.318 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:10.307+0000 7f2ea4d09700 1 -- 172.21.15.59:0/1661527178 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2ea00ca360 con 0x7f2ea00ce5c0 2023-11-24T08:00:10.318 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:10.308+0000 7f2e8effd700 1 -- 172.21.15.59:0/1661527178 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 11) v1 ==== 89741+0+0 (secure 0 0 0) 0x7f2e94003a20 con 0x7f2ea00ce5c0 2023-11-24T08:00:10.318 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:10.308+0000 7f2e8effd700 1 --2- 172.21.15.59:0/1661527178 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f2e8006b470 0x7f2e8006d930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:00:10.318 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:10.308+0000 7f2e8effd700 1 -- 172.21.15.59:0/1661527178 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f2e9408ae70 con 0x7f2ea00ce5c0 2023-11-24T08:00:10.318 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:10.308+0000 7f2e9dd9b700 1 --2- 172.21.15.59:0/1661527178 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f2e8006b470 0x7f2e8006d930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:00:10.318 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:10.309+0000 7f2e9dd9b700 1 --2- 172.21.15.59:0/1661527178 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f2e8006b470 0x7f2e8006d930 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7f2ea015abd0 tx=0x7f2e90007de0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:00:10.319 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:10.313+0000 7f2e8effd700 1 -- 172.21.15.59:0/1661527178 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f2e94055740 con 0x7f2ea00ce5c0 2023-11-24T08:00:10.561 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:10.560+0000 7f2ea4d09700 1 -- 172.21.15.59:0/1661527178 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "fs flag set", "flag_name": "enable_multiple", "val": "true", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f2ea00d0d30 con 0x7f2ea00ce5c0 2023-11-24T08:00:11.364 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:11.363+0000 7f2e8effd700 1 -- 172.21.15.59:0/1661527178 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "fs flag set", "flag_name": "enable_multiple", "val": "true", "yes_i_really_mean_it": true}]=0 v54) v1 ==== 136+0+0 (secure 0 0 0) 0x7f2e94058d60 con 0x7f2ea00ce5c0 2023-11-24T08:00:11.367 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:11.366+0000 7f2ea4d09700 1 -- 172.21.15.59:0/1661527178 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f2e8006b470 msgr2=0x7f2e8006d930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:00:11.367 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:11.366+0000 7f2ea4d09700 1 --2- 172.21.15.59:0/1661527178 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f2e8006b470 0x7f2e8006d930 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7f2ea015abd0 tx=0x7f2e90007de0 comp rx=0 tx=0).stop 2023-11-24T08:00:11.367 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:11.366+0000 7f2ea4d09700 1 -- 172.21.15.59:0/1661527178 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2ea00ce5c0 msgr2=0x7f2ea0158f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:00:11.367 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:11.366+0000 7f2ea4d09700 1 --2- 172.21.15.59:0/1661527178 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2ea00ce5c0 0x7f2ea0158f00 secure :-1 s=READY pgs=1053 cs=0 l=1 rev1=1 crypto rx=0x7f2e94009c40 tx=0x7f2e94004750 comp rx=0 tx=0).stop 2023-11-24T08:00:11.367 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:11.366+0000 7f2ea4d09700 1 -- 172.21.15.59:0/1661527178 shutdown_connections 2023-11-24T08:00:11.367 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:11.366+0000 7f2ea4d09700 1 --2- 172.21.15.59:0/1661527178 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f2e8006b470 0x7f2e8006d930 unknown :-1 s=CLOSED pgs=484 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:00:11.368 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:11.366+0000 7f2ea4d09700 1 --2- 172.21.15.59:0/1661527178 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f2ea015bde0 0x7f2ea015c830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:00:11.368 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:11.366+0000 7f2ea4d09700 1 --2- 172.21.15.59:0/1661527178 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2ea00ce5c0 0x7f2ea0158f00 unknown :-1 s=CLOSED pgs=1053 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:00:11.368 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:11.366+0000 7f2ea4d09700 1 --2- 172.21.15.59:0/1661527178 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2ea00cf2b0 0x7f2ea00b6730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:00:11.368 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:11.366+0000 7f2ea4d09700 1 -- 172.21.15.59:0/1661527178 >> 172.21.15.59:0/1661527178 conn(0x7f2ea00b3720 msgr2=0x7f2ea015b120 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:00:11.368 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:11.366+0000 7f2ea4d09700 1 -- 172.21.15.59:0/1661527178 shutdown_connections 2023-11-24T08:00:11.368 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:11.366+0000 7f2ea4d09700 1 -- 172.21.15.59:0/1661527178 wait complete. 2023-11-24T08:00:11.382 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume ls 2023-11-24T08:00:11.522 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:11.514+0000 7ffaae00f700 1 Processor -- start 2023-11-24T08:00:11.522 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:11.514+0000 7ffaae00f700 1 -- start start 2023-11-24T08:00:11.522 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:11.515+0000 7ffaae00f700 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7ffaa806afd0 0x7ffaa80d16e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:00:11.522 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:11.515+0000 7ffaae00f700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7ffaa80ada10 0x7ffaa80adea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:00:11.522 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:11.515+0000 7ffaae00f700 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffaa8070490 con 0x7ffaa80affb0 2023-11-24T08:00:11.522 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:11.515+0000 7ffaae00f700 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffaa80b0430 con 0x7ffaa80ada10 2023-11-24T08:00:11.523 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:11.515+0000 7ffaae00f700 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffaa80af9f0 con 0x7ffaa806afd0 2023-11-24T08:00:11.523 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:11.515+0000 7ffaa7fff700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7ffaa80ada10 0x7ffaa80adea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:00:11.523 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:11.515+0000 7ffaa6ffd700 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7ffaa80affb0 0x7ffaa806b3b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:59526/0 (socket says 172.21.15.59:59526) 2023-11-24T08:00:11.523 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:11.515+0000 7ffaa6ffd700 1 -- 172.21.15.59:0/1957058927 learned_addr learned my addr 172.21.15.59:0/1957058927 (peer_addr_for_me v1:172.21.15.59:0/0) 2023-11-24T08:00:11.523 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:11.515+0000 7ffaa77fe700 1 --2- 172.21.15.59:0/1957058927 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7ffaa806afd0 0x7ffaa80d16e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:00:11.523 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:11.515+0000 7ffaa7fff700 1 -- 172.21.15.59:0/1957058927 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7ffaa806afd0 msgr2=0x7ffaa80d16e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:00:11.523 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:11.515+0000 7ffaa7fff700 1 --2- 172.21.15.59:0/1957058927 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7ffaa806afd0 0x7ffaa80d16e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:00:11.523 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:11.515+0000 7ffaa7fff700 1 -- 172.21.15.59:0/1957058927 >> v1:172.21.15.59:6789/0 conn(0x7ffaa80affb0 legacy=0x7ffaa806b3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:00:11.523 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:11.515+0000 7ffaa7fff700 1 -- 172.21.15.59:0/1957058927 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffaa80afcd0 con 0x7ffaa80ada10 2023-11-24T08:00:11.523 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:11.515+0000 7ffaa67fc700 1 -- 172.21.15.59:0/1957058927 <== mon.0 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3714540065 0 0) 0x7ffaa8070490 con 0x7ffaa80affb0 2023-11-24T08:00:11.523 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:11.515+0000 7ffaa7fff700 1 --2- 172.21.15.59:0/1957058927 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ffaa80ada10 0x7ffaa80adea0 secure :-1 s=READY pgs=1055 cs=0 l=1 rev1=1 crypto rx=0x7ffaa0009d90 tx=0x7ffaa000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f75cb0fbef5f27d server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:00:11.523 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:11.515+0000 7ffaa67fc700 1 -- 172.21.15.59:0/1957058927 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffaa000e070 con 0x7ffaa80ada10 2023-11-24T08:00:11.524 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:11.516+0000 7ffaa67fc700 1 -- 172.21.15.59:0/1957058927 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffaa000bbd0 con 0x7ffaa80ada10 2023-11-24T08:00:11.524 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:11.516+0000 7ffaa67fc700 1 -- 172.21.15.59:0/1957058927 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffaa0012430 con 0x7ffaa80ada10 2023-11-24T08:00:11.524 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:11.516+0000 7ffaae00f700 1 -- 172.21.15.59:0/1957058927 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ffaa80ada10 msgr2=0x7ffaa80adea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:00:11.524 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:11.516+0000 7ffaae00f700 1 --2- 172.21.15.59:0/1957058927 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ffaa80ada10 0x7ffaa80adea0 secure :-1 s=READY pgs=1055 cs=0 l=1 rev1=1 crypto rx=0x7ffaa0009d90 tx=0x7ffaa000b330 comp rx=0 tx=0).stop 2023-11-24T08:00:11.524 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:11.516+0000 7ffaae00f700 1 -- 172.21.15.59:0/1957058927 shutdown_connections 2023-11-24T08:00:11.524 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:11.516+0000 7ffaae00f700 1 --2- 172.21.15.59:0/1957058927 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ffaa80ada10 0x7ffaa80adea0 unknown :-1 s=CLOSED pgs=1055 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:00:11.524 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:11.516+0000 7ffaae00f700 1 --2- 172.21.15.59:0/1957058927 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7ffaa806afd0 0x7ffaa80d16e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:00:11.524 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:11.516+0000 7ffaae00f700 1 -- 172.21.15.59:0/1957058927 >> 172.21.15.59:0/1957058927 conn(0x7ffaa80d0bc0 msgr2=0x7ffaa80d0fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:00:11.524 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:11.516+0000 7ffaae00f700 1 -- 172.21.15.59:0/1957058927 shutdown_connections 2023-11-24T08:00:11.524 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:11.516+0000 7ffaae00f700 1 -- 172.21.15.59:0/1957058927 wait complete. 2023-11-24T08:00:11.524 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:11.517+0000 7ffaae00f700 1 Processor -- start 2023-11-24T08:00:11.524 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:11.518+0000 7ffaae00f700 1 -- start start 2023-11-24T08:00:11.525 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:11.518+0000 7ffaae00f700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ffaa806afd0 0x7ffaa80cd5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:00:11.525 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:11.518+0000 7ffaae00f700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ffaa80ada10 0x7ffaa815be50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:00:11.525 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:11.518+0000 7ffaae00f700 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7ffaa80affb0 0x7ffaa8147f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:00:11.525 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:11.518+0000 7ffaae00f700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffaa81563b0 con 0x7ffaa80ada10 2023-11-24T08:00:11.525 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:11.518+0000 7ffaae00f700 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffaa8156600 con 0x7ffaa806afd0 2023-11-24T08:00:11.525 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:11.518+0000 7ffaae00f700 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffaa81568e0 con 0x7ffaa80affb0 2023-11-24T08:00:11.525 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:11.518+0000 7ffaa6ffd700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ffaa80ada10 0x7ffaa815be50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:00:11.525 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:11.518+0000 7ffaa6ffd700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ffaa80ada10 0x7ffaa815be50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:56060/0 (socket says 172.21.15.59:56060) 2023-11-24T08:00:11.525 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:11.518+0000 7ffaa6ffd700 1 -- 172.21.15.59:0/1768850168 learned_addr learned my addr 172.21.15.59:0/1768850168 (peer_addr_for_me v2:172.21.15.59:0/0) 2023-11-24T08:00:11.525 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:11.519+0000 7ffaa7fff700 1 --2- 172.21.15.59:0/1768850168 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7ffaa80affb0 0x7ffaa8147f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:00:11.525 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:11.519+0000 7ffaa77fe700 1 --2- 172.21.15.59:0/1768850168 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ffaa806afd0 0x7ffaa80cd5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:00:11.525 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:11.519+0000 7ffaa6ffd700 1 -- 172.21.15.59:0/1768850168 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7ffaa80affb0 msgr2=0x7ffaa8147f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:00:11.526 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:11.519+0000 7ffaa6ffd700 1 --2- 172.21.15.59:0/1768850168 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7ffaa80affb0 0x7ffaa8147f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:00:11.526 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:11.519+0000 7ffaa6ffd700 1 -- 172.21.15.59:0/1768850168 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ffaa806afd0 msgr2=0x7ffaa80cd5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:00:11.526 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:11.519+0000 7ffaa6ffd700 1 --2- 172.21.15.59:0/1768850168 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ffaa806afd0 0x7ffaa80cd5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:00:11.526 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:11.519+0000 7ffaa6ffd700 1 -- 172.21.15.59:0/1768850168 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffaa0009a70 con 0x7ffaa80ada10 2023-11-24T08:00:11.526 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:11.519+0000 7ffaa7fff700 1 --2- 172.21.15.59:0/1768850168 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7ffaa80affb0 0x7ffaa8147f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T08:00:11.526 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:11.519+0000 7ffaa77fe700 1 --2- 172.21.15.59:0/1768850168 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ffaa806afd0 0x7ffaa80cd5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T08:00:11.526 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:11.519+0000 7ffaa6ffd700 1 --2- 172.21.15.59:0/1768850168 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ffaa80ada10 0x7ffaa815be50 secure :-1 s=READY pgs=1056 cs=0 l=1 rev1=1 crypto rx=0x7ffa9800c550 tx=0x7ffa9800c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:00:11.526 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:11.519+0000 7ffa8ffff700 1 -- 172.21.15.59:0/1768850168 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa98004c80 con 0x7ffaa80ada10 2023-11-24T08:00:11.526 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:11.519+0000 7ffa8ffff700 1 -- 172.21.15.59:0/1768850168 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffa98004e00 con 0x7ffaa80ada10 2023-11-24T08:00:11.526 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:11.519+0000 7ffaae00f700 1 -- 172.21.15.59:0/1768850168 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffaa815f470 con 0x7ffaa80ada10 2023-11-24T08:00:11.526 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:11.519+0000 7ffa8ffff700 1 -- 172.21.15.59:0/1768850168 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa98007310 con 0x7ffaa80ada10 2023-11-24T08:00:11.526 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:11.519+0000 7ffaae00f700 1 -- 172.21.15.59:0/1768850168 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffaa8156ea0 con 0x7ffaa80ada10 2023-11-24T08:00:11.527 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:11.520+0000 7ffaae00f700 1 -- 172.21.15.59:0/1768850168 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffaa8068670 con 0x7ffaa80ada10 2023-11-24T08:00:11.527 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:11.521+0000 7ffa8ffff700 1 -- 172.21.15.59:0/1768850168 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 11) v1 ==== 89741+0+0 (secure 0 0 0) 0x7ffa98007470 con 0x7ffaa80ada10 2023-11-24T08:00:11.527 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:11.521+0000 7ffa8ffff700 1 --2- 172.21.15.59:0/1768850168 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7ffa9406b470 0x7ffa9406d930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:00:11.527 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:11.521+0000 7ffa8ffff700 1 -- 172.21.15.59:0/1768850168 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 5709+0+0 (secure 0 0 0) 0x7ffa98089c90 con 0x7ffaa80ada10 2023-11-24T08:00:11.527 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:11.521+0000 7ffaa77fe700 1 --2- 172.21.15.59:0/1768850168 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7ffa9406b470 0x7ffa9406d930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:00:11.527 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:11.522+0000 7ffaa77fe700 1 --2- 172.21.15.59:0/1768850168 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7ffa9406b470 0x7ffa9406d930 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7ffa9c009a00 tx=0x7ffa9c008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:00:11.528 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:11.525+0000 7ffa8ffff700 1 -- 172.21.15.59:0/1768850168 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7ffa98054560 con 0x7ffaa80ada10 2023-11-24T08:00:11.757 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:11.756+0000 7ffaae00f700 1 -- 172.21.15.59:0/1768850168 --> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] -- mgr_command(tid 0: {"prefix": "fs volume ls", "target": ["mon-mgr", ""]}) v1 -- 0x7ffaa80b2640 con 0x7ffa9406b470 2023-11-24T08:00:11.759 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:11.758+0000 7ffa8ffff700 1 -- 172.21.15.59:0/1768850168 <== mgr.4100 v2:172.21.15.76:6832/4264046536 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+41 (secure 0 0 0) 0x7ffaa80b2640 con 0x7ffa9406b470 2023-11-24T08:00:11.759 INFO:teuthology.orchestra.run.smithi059.stdout:[ 2023-11-24T08:00:11.759 INFO:teuthology.orchestra.run.smithi059.stdout: { 2023-11-24T08:00:11.759 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "cephfs" 2023-11-24T08:00:11.760 INFO:teuthology.orchestra.run.smithi059.stdout: } 2023-11-24T08:00:11.760 INFO:teuthology.orchestra.run.smithi059.stdout:] 2023-11-24T08:00:11.762 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:11.761+0000 7ffaae00f700 1 -- 172.21.15.59:0/1768850168 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7ffa9406b470 msgr2=0x7ffa9406d930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:00:11.762 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:11.761+0000 7ffaae00f700 1 --2- 172.21.15.59:0/1768850168 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7ffa9406b470 0x7ffa9406d930 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7ffa9c009a00 tx=0x7ffa9c008040 comp rx=0 tx=0).stop 2023-11-24T08:00:11.762 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:11.761+0000 7ffaae00f700 1 -- 172.21.15.59:0/1768850168 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ffaa80ada10 msgr2=0x7ffaa815be50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:00:11.762 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:11.761+0000 7ffaae00f700 1 --2- 172.21.15.59:0/1768850168 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ffaa80ada10 0x7ffaa815be50 secure :-1 s=READY pgs=1056 cs=0 l=1 rev1=1 crypto rx=0x7ffa9800c550 tx=0x7ffa9800c910 comp rx=0 tx=0).stop 2023-11-24T08:00:11.762 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:11.761+0000 7ffaae00f700 1 -- 172.21.15.59:0/1768850168 shutdown_connections 2023-11-24T08:00:11.762 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:11.761+0000 7ffaae00f700 1 --2- 172.21.15.59:0/1768850168 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7ffa9406b470 0x7ffa9406d930 unknown :-1 s=CLOSED pgs=485 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:00:11.762 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:11.761+0000 7ffaae00f700 1 --2- 172.21.15.59:0/1768850168 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7ffaa80affb0 0x7ffaa8147f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:00:11.762 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:11.761+0000 7ffaae00f700 1 --2- 172.21.15.59:0/1768850168 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ffaa806afd0 0x7ffaa80cd5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:00:11.763 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:11.761+0000 7ffaae00f700 1 --2- 172.21.15.59:0/1768850168 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ffaa80ada10 0x7ffaa815be50 unknown :-1 s=CLOSED pgs=1056 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:00:11.763 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:11.761+0000 7ffaae00f700 1 -- 172.21.15.59:0/1768850168 >> 172.21.15.59:0/1768850168 conn(0x7ffaa80d0bc0 msgr2=0x7ffaa80addf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:00:11.763 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:11.761+0000 7ffaae00f700 1 -- 172.21.15.59:0/1768850168 shutdown_connections 2023-11-24T08:00:11.763 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:11.761+0000 7ffaae00f700 1 -- 172.21.15.59:0/1768850168 wait complete. 2023-11-24T08:00:11.774 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2023-11-24T08:00:11.903 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:11.901+0000 7f03170c7700 1 Processor -- start 2023-11-24T08:00:11.903 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:11.901+0000 7f03170c7700 1 -- start start 2023-11-24T08:00:11.905 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:11.902+0000 7f03170c7700 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f03100d0c50 0x7f03100d26c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:00:11.905 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:11.902+0000 7f03170c7700 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f03100d4e20 con 0x7f03100d0870 2023-11-24T08:00:11.905 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:11.902+0000 7f03170c7700 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f03100b5370 con 0x7f03100d49f0 2023-11-24T08:00:11.905 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:11.902+0000 7f03170c7700 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f03100d2f80 con 0x7f03100d0c50 2023-11-24T08:00:11.905 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:11.902+0000 7f0314e63700 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7f03100d0870 0x7f03100cf860 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:59540/0 (socket says 172.21.15.59:59540) 2023-11-24T08:00:11.906 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:11.902+0000 7f0314e63700 1 -- 172.21.15.59:0/416911843 learned_addr learned my addr 172.21.15.59:0/416911843 (peer_addr_for_me v1:172.21.15.59:0/0) 2023-11-24T08:00:11.906 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:11.902+0000 7f030ffff700 1 --2- 172.21.15.59:0/416911843 >> v2:172.21.15.76:3300/0 conn(0x7f03100d0c50 0x7f03100d26c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:00:11.906 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:11.903+0000 7f030f7fe700 1 -- 172.21.15.59:0/416911843 <== mon.0 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 234977468 0 0) 0x7f03100d4e20 con 0x7f03100d0870 2023-11-24T08:00:11.906 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:11.903+0000 7f030f7fe700 1 -- 172.21.15.59:0/416911843 --> v1:172.21.15.59:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f02fc0035b0 con 0x7f03100d0870 2023-11-24T08:00:11.906 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:11.903+0000 7f030ffff700 1 -- 172.21.15.59:0/416911843 >> v1:172.21.15.76:6789/0 conn(0x7f03100d49f0 legacy=0x7f03100b6d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:00:11.906 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:11.903+0000 7f030ffff700 1 -- 172.21.15.59:0/416911843 >> v1:172.21.15.59:6789/0 conn(0x7f03100d0870 legacy=0x7f03100cf860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:00:11.906 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:11.903+0000 7f030ffff700 1 -- 172.21.15.59:0/416911843 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f03100ceae0 con 0x7f03100d0c50 2023-11-24T08:00:11.906 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:11.903+0000 7f030ffff700 1 --2- 172.21.15.59:0/416911843 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f03100d0c50 0x7f03100d26c0 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f02f8009840 tx=0x7f02f8009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c071ecc16d1888ad server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:00:11.907 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:11.903+0000 7f030f7fe700 1 -- 172.21.15.59:0/416911843 <== mon.1 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02f8011830 con 0x7f03100d0c50 2023-11-24T08:00:11.907 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:11.903+0000 7f030f7fe700 1 -- 172.21.15.59:0/416911843 <== mon.1 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f02f8011990 con 0x7f03100d0c50 2023-11-24T08:00:11.907 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:11.903+0000 7f030f7fe700 1 -- 172.21.15.59:0/416911843 <== mon.1 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02f8011d20 con 0x7f03100d0c50 2023-11-24T08:00:11.911 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:11.904+0000 7f03170c7700 1 -- 172.21.15.59:0/416911843 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f03100d0c50 msgr2=0x7f03100d26c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:00:11.911 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:11.904+0000 7f03170c7700 1 --2- 172.21.15.59:0/416911843 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f03100d0c50 0x7f03100d26c0 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f02f8009840 tx=0x7f02f8009b50 comp rx=0 tx=0).stop 2023-11-24T08:00:11.911 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:11.904+0000 7f03170c7700 1 -- 172.21.15.59:0/416911843 shutdown_connections 2023-11-24T08:00:11.911 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:11.904+0000 7f03170c7700 1 --2- 172.21.15.59:0/416911843 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f03100d0c50 0x7f03100d26c0 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:00:11.911 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:11.904+0000 7f03170c7700 1 -- 172.21.15.59:0/416911843 >> 172.21.15.59:0/416911843 conn(0x7f03100d3ed0 msgr2=0x7f03100d42e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:00:11.911 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:11.904+0000 7f03170c7700 1 -- 172.21.15.59:0/416911843 shutdown_connections 2023-11-24T08:00:11.911 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:11.904+0000 7f03170c7700 1 -- 172.21.15.59:0/416911843 wait complete. 2023-11-24T08:00:11.911 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:11.905+0000 7f03170c7700 1 Processor -- start 2023-11-24T08:00:11.911 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:11.905+0000 7f03170c7700 1 -- start start 2023-11-24T08:00:11.912 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:11.906+0000 7f03170c7700 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f03100d0870 0x7f03100c9ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:00:11.912 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:11.906+0000 7f03170c7700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f03100d0c50 0x7f0310149850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:00:11.912 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:11.906+0000 7f03170c7700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f03100d49f0 0x7f031014b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:00:11.912 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:11.906+0000 7f03170c7700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03100cff70 con 0x7f03100d0c50 2023-11-24T08:00:11.912 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:11.906+0000 7f03170c7700 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03100ca020 con 0x7f03100d49f0 2023-11-24T08:00:11.912 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:11.906+0000 7f03170c7700 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f03100c72d0 con 0x7f03100d0870 2023-11-24T08:00:11.913 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:11.906+0000 7f030ffff700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f03100d0c50 0x7f0310149850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:00:11.913 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:11.906+0000 7f030ffff700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f03100d0c50 0x7f0310149850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:56074/0 (socket says 172.21.15.59:56074) 2023-11-24T08:00:11.913 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:11.906+0000 7f030ffff700 1 -- 172.21.15.59:0/4033974159 learned_addr learned my addr 172.21.15.59:0/4033974159 (peer_addr_for_me v2:172.21.15.59:0/0) 2023-11-24T08:00:11.913 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:11.906+0000 7f0315664700 1 --2- 172.21.15.59:0/4033974159 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f03100d49f0 0x7f031014b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:00:11.913 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:11.906+0000 7f0314e63700 1 --2- 172.21.15.59:0/4033974159 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f03100d0870 0x7f03100c9ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:00:11.913 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:11.906+0000 7f030ffff700 1 -- 172.21.15.59:0/4033974159 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f03100d0870 msgr2=0x7f03100c9ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:00:11.913 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:11.906+0000 7f030ffff700 1 --2- 172.21.15.59:0/4033974159 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f03100d0870 0x7f03100c9ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:00:11.913 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:11.906+0000 7f030ffff700 1 -- 172.21.15.59:0/4033974159 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f03100d49f0 msgr2=0x7f031014b750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:00:11.913 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:11.906+0000 7f030ffff700 1 --2- 172.21.15.59:0/4033974159 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f03100d49f0 0x7f031014b750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:00:11.914 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:11.906+0000 7f030ffff700 1 -- 172.21.15.59:0/4033974159 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f02f80094a0 con 0x7f03100d0c50 2023-11-24T08:00:11.914 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:11.906+0000 7f0315664700 1 --2- 172.21.15.59:0/4033974159 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f03100d49f0 0x7f031014b750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T08:00:11.914 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:11.906+0000 7f0314e63700 1 --2- 172.21.15.59:0/4033974159 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f03100d0870 0x7f03100c9ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T08:00:11.914 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:11.907+0000 7f030ffff700 1 --2- 172.21.15.59:0/4033974159 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f03100d0c50 0x7f0310149850 secure :-1 s=READY pgs=1058 cs=0 l=1 rev1=1 crypto rx=0x7f02f80090b0 tx=0x7f02f8004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:00:11.914 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:11.907+0000 7f030d7fa700 1 -- 172.21.15.59:0/4033974159 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02f80038a0 con 0x7f03100d0c50 2023-11-24T08:00:11.914 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:11.907+0000 7f030d7fa700 1 -- 172.21.15.59:0/4033974159 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f02f801f070 con 0x7f03100d0c50 2023-11-24T08:00:11.915 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:11.907+0000 7f03170c7700 1 -- 172.21.15.59:0/4033974159 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f03101493c0 con 0x7f03100d0c50 2023-11-24T08:00:11.915 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:11.907+0000 7f030d7fa700 1 -- 172.21.15.59:0/4033974159 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02f8003c40 con 0x7f03100d0c50 2023-11-24T08:00:11.915 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:11.907+0000 7f03170c7700 1 -- 172.21.15.59:0/4033974159 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f03101487b0 con 0x7f03100d0c50 2023-11-24T08:00:11.915 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:11.908+0000 7f03170c7700 1 -- 172.21.15.59:0/4033974159 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f03100b57c0 con 0x7f03100d0c50 2023-11-24T08:00:11.916 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:11.908+0000 7f030d7fa700 1 -- 172.21.15.59:0/4033974159 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 11) v1 ==== 89741+0+0 (secure 0 0 0) 0x7f02f8026480 con 0x7f03100d0c50 2023-11-24T08:00:11.916 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:11.908+0000 7f030d7fa700 1 --2- 172.21.15.59:0/4033974159 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f02f006b4a0 0x7f02f006d960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:00:11.916 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:11.909+0000 7f030d7fa700 1 -- 172.21.15.59:0/4033974159 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f02f8021070 con 0x7f03100d0c50 2023-11-24T08:00:11.916 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:11.909+0000 7f0314e63700 1 --2- 172.21.15.59:0/4033974159 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f02f006b4a0 0x7f02f006d960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:00:11.916 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:11.909+0000 7f0314e63700 1 --2- 172.21.15.59:0/4033974159 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f02f006b4a0 0x7f02f006d960 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7f0308001560 tx=0x7f0308009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:00:11.916 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:11.913+0000 7f030d7fa700 1 -- 172.21.15.59:0/4033974159 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f02f80561b0 con 0x7f03100d0c50 2023-11-24T08:00:12.118 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:12.116+0000 7f03170c7700 1 -- 172.21.15.59:0/4033974159 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7f0310075480 con 0x7f03100d0c50 2023-11-24T08:00:12.118 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:12.116+0000 7f030d7fa700 1 -- 172.21.15.59:0/4033974159 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v18)=0 v18) v1 ==== 122+0+0 (secure 0 0 0) 0x7f02f802fe60 con 0x7f03100d0c50 2023-11-24T08:00:12.121 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:12.119+0000 7f03170c7700 1 -- 172.21.15.59:0/4033974159 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f02f006b4a0 msgr2=0x7f02f006d960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:00:12.121 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:12.119+0000 7f03170c7700 1 --2- 172.21.15.59:0/4033974159 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f02f006b4a0 0x7f02f006d960 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7f0308001560 tx=0x7f0308009040 comp rx=0 tx=0).stop 2023-11-24T08:00:12.121 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:12.119+0000 7f03170c7700 1 -- 172.21.15.59:0/4033974159 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f03100d0c50 msgr2=0x7f0310149850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:00:12.121 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:12.119+0000 7f03170c7700 1 --2- 172.21.15.59:0/4033974159 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f03100d0c50 0x7f0310149850 secure :-1 s=READY pgs=1058 cs=0 l=1 rev1=1 crypto rx=0x7f02f80090b0 tx=0x7f02f8004750 comp rx=0 tx=0).stop 2023-11-24T08:00:12.121 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:12.119+0000 7f03170c7700 1 -- 172.21.15.59:0/4033974159 shutdown_connections 2023-11-24T08:00:12.121 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:12.119+0000 7f03170c7700 1 --2- 172.21.15.59:0/4033974159 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f02f006b4a0 0x7f02f006d960 unknown :-1 s=CLOSED pgs=486 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:00:12.121 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:12.119+0000 7f03170c7700 1 --2- 172.21.15.59:0/4033974159 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f03100d49f0 0x7f031014b750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:00:12.121 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:12.119+0000 7f03170c7700 1 --2- 172.21.15.59:0/4033974159 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f03100d0870 0x7f03100c9ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:00:12.121 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:12.119+0000 7f03170c7700 1 --2- 172.21.15.59:0/4033974159 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f03100d0c50 0x7f0310149850 unknown :-1 s=CLOSED pgs=1058 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:00:12.121 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:12.119+0000 7f03170c7700 1 -- 172.21.15.59:0/4033974159 >> 172.21.15.59:0/4033974159 conn(0x7f03100d3ed0 msgr2=0x7f03101627f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:00:12.121 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:12.119+0000 7f03170c7700 1 -- 172.21.15.59:0/4033974159 shutdown_connections 2023-11-24T08:00:12.121 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:12.119+0000 7f03170c7700 1 -- 172.21.15.59:0/4033974159 wait complete. 2023-11-24T08:00:12.134 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolumegroup create cephfs subvol_grp1997 2023-11-24T08:00:12.262 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:12.260+0000 7fbd6a4fa700 1 Processor -- start 2023-11-24T08:00:12.262 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:12.260+0000 7fbd6a4fa700 1 -- start start 2023-11-24T08:00:12.264 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:12.261+0000 7fbd6a4fa700 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fbd640722c0 0x7fbd640b51b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:00:12.264 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:12.261+0000 7fbd6a4fa700 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbd64143ec0 con 0x7fbd641440f0 2023-11-24T08:00:12.264 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:12.261+0000 7fbd6a4fa700 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbd640b3ae0 con 0x7fbd64143ae0 2023-11-24T08:00:12.264 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:12.261+0000 7fbd6a4fa700 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbd640b40a0 con 0x7fbd640722c0 2023-11-24T08:00:12.264 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:12.261+0000 7fbd68a97700 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7fbd641440f0 0x7fbd6406e510 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:59558/0 (socket says 172.21.15.59:59558) 2023-11-24T08:00:12.264 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:12.261+0000 7fbd68a97700 1 -- 172.21.15.59:0/2597337370 learned_addr learned my addr 172.21.15.59:0/2597337370 (peer_addr_for_me v1:172.21.15.59:0/0) 2023-11-24T08:00:12.264 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:12.262+0000 7fbd637fe700 1 --2- 172.21.15.59:0/2597337370 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fbd640722c0 0x7fbd640b51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:00:12.265 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:12.262+0000 7fbd62ffd700 1 -- 172.21.15.59:0/2597337370 <== mon.0 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2620984188 0 0) 0x7fbd64143ec0 con 0x7fbd641440f0 2023-11-24T08:00:12.265 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:12.262+0000 7fbd62ffd700 1 -- 172.21.15.59:0/2597337370 --> v1:172.21.15.59:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbd500035b0 con 0x7fbd641440f0 2023-11-24T08:00:12.265 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:12.262+0000 7fbd62ffd700 1 -- 172.21.15.59:0/2597337370 <== mon.1 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3172142873 0 0) 0x7fbd640b3ae0 con 0x7fbd64143ae0 2023-11-24T08:00:12.265 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:12.262+0000 7fbd62ffd700 1 -- 172.21.15.59:0/2597337370 --> v1:172.21.15.76:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbd64143ec0 con 0x7fbd64143ae0 2023-11-24T08:00:12.265 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:12.262+0000 7fbd637fe700 1 -- 172.21.15.59:0/2597337370 >> v1:172.21.15.76:6789/0 conn(0x7fbd64143ae0 legacy=0x7fbd64071bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:00:12.266 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:12.262+0000 7fbd62ffd700 1 -- 172.21.15.59:0/2597337370 <== mon.0 v1:172.21.15.59:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3235685337 0 0) 0x7fbd500035b0 con 0x7fbd641440f0 2023-11-24T08:00:12.266 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:12.262+0000 7fbd637fe700 1 -- 172.21.15.59:0/2597337370 >> v1:172.21.15.59:6789/0 conn(0x7fbd641440f0 legacy=0x7fbd6406e510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:00:12.266 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:12.262+0000 7fbd637fe700 1 -- 172.21.15.59:0/2597337370 --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbd640b3dc0 con 0x7fbd640722c0 2023-11-24T08:00:12.266 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:12.262+0000 7fbd637fe700 1 --2- 172.21.15.59:0/2597337370 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fbd640722c0 0x7fbd640b51b0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fbd4c009870 tx=0x7fbd4c009b80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=dda3f6fe78cd14e7 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:00:12.266 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:12.262+0000 7fbd62ffd700 1 -- 172.21.15.59:0/2597337370 <== mon.2 v2:172.21.15.76:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd5c0019f0 con 0x7fbd640722c0 2023-11-24T08:00:12.267 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:12.262+0000 7fbd62ffd700 1 -- 172.21.15.59:0/2597337370 <== mon.2 v2:172.21.15.76:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbd4c004030 con 0x7fbd640722c0 2023-11-24T08:00:12.267 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:12.262+0000 7fbd62ffd700 1 -- 172.21.15.59:0/2597337370 <== mon.2 v2:172.21.15.76:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd4c0043c0 con 0x7fbd640722c0 2023-11-24T08:00:12.270 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:12.263+0000 7fbd6a4fa700 1 -- 172.21.15.59:0/2597337370 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fbd640722c0 msgr2=0x7fbd640b51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:00:12.271 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:12.263+0000 7fbd6a4fa700 1 --2- 172.21.15.59:0/2597337370 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fbd640722c0 0x7fbd640b51b0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fbd4c009870 tx=0x7fbd4c009b80 comp rx=0 tx=0).stop 2023-11-24T08:00:12.271 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:12.263+0000 7fbd6a4fa700 1 -- 172.21.15.59:0/2597337370 shutdown_connections 2023-11-24T08:00:12.271 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:12.263+0000 7fbd6a4fa700 1 --2- 172.21.15.59:0/2597337370 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fbd640722c0 0x7fbd640b51b0 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:00:12.271 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:12.263+0000 7fbd6a4fa700 1 -- 172.21.15.59:0/2597337370 >> 172.21.15.59:0/2597337370 conn(0x7fbd640b2740 msgr2=0x7fbd640b2b20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:00:12.272 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:12.263+0000 7fbd6a4fa700 1 -- 172.21.15.59:0/2597337370 shutdown_connections 2023-11-24T08:00:12.272 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:12.263+0000 7fbd6a4fa700 1 -- 172.21.15.59:0/2597337370 wait complete. 2023-11-24T08:00:12.272 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:12.264+0000 7fbd6a4fa700 1 Processor -- start 2023-11-24T08:00:12.272 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:12.264+0000 7fbd6a4fa700 1 -- start start 2023-11-24T08:00:12.272 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:12.265+0000 7fbd6a4fa700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbd640722c0 0x7fbd6413edd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:00:12.273 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:12.265+0000 7fbd6a4fa700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fbd64143ae0 0x7fbd640c8280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:00:12.273 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:12.265+0000 7fbd6a4fa700 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fbd641440f0 0x7fbd641f3c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:00:12.273 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:12.265+0000 7fbd6a4fa700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd641f4b60 con 0x7fbd640722c0 2023-11-24T08:00:12.273 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:12.265+0000 7fbd6a4fa700 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd640c71e0 con 0x7fbd64143ae0 2023-11-24T08:00:12.274 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:12.265+0000 7fbd6a4fa700 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbd640c74c0 con 0x7fbd641440f0 2023-11-24T08:00:12.274 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:12.265+0000 7fbd63fff700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbd640722c0 0x7fbd6413edd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:00:12.274 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:12.265+0000 7fbd63fff700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbd640722c0 0x7fbd6413edd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:56086/0 (socket says 172.21.15.59:56086) 2023-11-24T08:00:12.274 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:12.265+0000 7fbd63fff700 1 -- 172.21.15.59:0/88549919 learned_addr learned my addr 172.21.15.59:0/88549919 (peer_addr_for_me v2:172.21.15.59:0/0) 2023-11-24T08:00:12.275 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:12.265+0000 7fbd637fe700 1 --2- 172.21.15.59:0/88549919 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fbd64143ae0 0x7fbd640c8280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:00:12.275 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:12.265+0000 7fbd68a97700 1 --2- 172.21.15.59:0/88549919 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fbd641440f0 0x7fbd641f3c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:00:12.275 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:12.266+0000 7fbd63fff700 1 -- 172.21.15.59:0/88549919 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fbd641440f0 msgr2=0x7fbd641f3c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:00:12.275 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:12.266+0000 7fbd63fff700 1 --2- 172.21.15.59:0/88549919 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fbd641440f0 0x7fbd641f3c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:00:12.275 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:12.266+0000 7fbd63fff700 1 -- 172.21.15.59:0/88549919 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fbd64143ae0 msgr2=0x7fbd640c8280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:00:12.276 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:12.266+0000 7fbd63fff700 1 --2- 172.21.15.59:0/88549919 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fbd64143ae0 0x7fbd640c8280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:00:12.276 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:12.266+0000 7fbd63fff700 1 -- 172.21.15.59:0/88549919 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbd4c0094a0 con 0x7fbd640722c0 2023-11-24T08:00:12.276 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:12.266+0000 7fbd637fe700 1 --2- 172.21.15.59:0/88549919 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fbd64143ae0 0x7fbd640c8280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-24T08:00:12.276 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:12.266+0000 7fbd63fff700 1 --2- 172.21.15.59:0/88549919 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbd640722c0 0x7fbd6413edd0 secure :-1 s=READY pgs=1060 cs=0 l=1 rev1=1 crypto rx=0x7fbd5400c520 tx=0x7fbd5400c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:00:12.277 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:12.266+0000 7fbd60ff9700 1 -- 172.21.15.59:0/88549919 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd540052d0 con 0x7fbd640722c0 2023-11-24T08:00:12.277 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:12.266+0000 7fbd60ff9700 1 -- 172.21.15.59:0/88549919 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbd54005430 con 0x7fbd640722c0 2023-11-24T08:00:12.277 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:12.266+0000 7fbd6a4fa700 1 -- 172.21.15.59:0/88549919 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbd640c77a0 con 0x7fbd640722c0 2023-11-24T08:00:12.277 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:12.266+0000 7fbd60ff9700 1 -- 172.21.15.59:0/88549919 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd540038c0 con 0x7fbd640722c0 2023-11-24T08:00:12.277 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:12.266+0000 7fbd6a4fa700 1 -- 172.21.15.59:0/88549919 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbd640c7d60 con 0x7fbd640722c0 2023-11-24T08:00:12.278 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:12.267+0000 7fbd6a4fa700 1 -- 172.21.15.59:0/88549919 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbd640c7ef0 con 0x7fbd640722c0 2023-11-24T08:00:12.278 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:12.267+0000 7fbd60ff9700 1 -- 172.21.15.59:0/88549919 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 11) v1 ==== 89741+0+0 (secure 0 0 0) 0x7fbd54003a20 con 0x7fbd640722c0 2023-11-24T08:00:12.278 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:12.268+0000 7fbd60ff9700 1 --2- 172.21.15.59:0/88549919 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7fbd4406b470 0x7fbd4406d930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:00:12.278 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:12.268+0000 7fbd60ff9700 1 -- 172.21.15.59:0/88549919 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fbd54089610 con 0x7fbd640722c0 2023-11-24T08:00:12.278 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:12.268+0000 7fbd637fe700 1 --2- 172.21.15.59:0/88549919 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7fbd4406b470 0x7fbd4406d930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:00:12.279 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:12.269+0000 7fbd637fe700 1 --2- 172.21.15.59:0/88549919 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7fbd4406b470 0x7fbd4406d930 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7fbd4c0090b0 tx=0x7fbd4c008030 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:00:12.280 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:12.272+0000 7fbd60ff9700 1 -- 172.21.15.59:0/88549919 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fbd54053e50 con 0x7fbd640722c0 2023-11-24T08:00:12.505 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:12.503+0000 7fbd6a4fa700 1 -- 172.21.15.59:0/88549919 --> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] -- mgr_command(tid 0: {"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "subvol_grp1997", "target": ["mon-mgr", ""]}) v1 -- 0x7fbd6406dff0 con 0x7fbd4406b470 2023-11-24T08:00:12.511 INFO:tasks.ceph.mgr.x.smithi076.stderr:2023-11-24T08:00:12.510+0000 7ff196724700 -1 client.0 error registering admin socket command: (17) File exists 2023-11-24T08:00:12.511 INFO:tasks.ceph.mgr.x.smithi076.stderr:2023-11-24T08:00:12.510+0000 7ff196724700 -1 client.0 error registering admin socket command: (17) File exists 2023-11-24T08:00:12.512 INFO:tasks.ceph.mgr.x.smithi076.stderr:2023-11-24T08:00:12.510+0000 7ff196724700 -1 client.0 error registering admin socket command: (17) File exists 2023-11-24T08:00:12.512 INFO:tasks.ceph.mgr.x.smithi076.stderr:2023-11-24T08:00:12.510+0000 7ff196724700 -1 client.0 error registering admin socket command: (17) File exists 2023-11-24T08:00:12.512 INFO:tasks.ceph.mgr.x.smithi076.stderr:2023-11-24T08:00:12.510+0000 7ff196724700 -1 client.0 error registering admin socket command: (17) File exists 2023-11-24T08:00:12.528 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:12.526+0000 7fbd60ff9700 1 -- 172.21.15.59:0/88549919 <== mgr.4100 v2:172.21.15.76:6832/4264046536 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7fbd4c004320 con 0x7fbd4406b470 2023-11-24T08:00:12.530 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:12.528+0000 7fbd6a4fa700 1 -- 172.21.15.59:0/88549919 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7fbd4406b470 msgr2=0x7fbd4406d930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:00:12.531 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:12.528+0000 7fbd6a4fa700 1 --2- 172.21.15.59:0/88549919 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7fbd4406b470 0x7fbd4406d930 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7fbd4c0090b0 tx=0x7fbd4c008030 comp rx=0 tx=0).stop 2023-11-24T08:00:12.531 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:12.528+0000 7fbd6a4fa700 1 -- 172.21.15.59:0/88549919 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbd640722c0 msgr2=0x7fbd6413edd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:00:12.531 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:12.528+0000 7fbd6a4fa700 1 --2- 172.21.15.59:0/88549919 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbd640722c0 0x7fbd6413edd0 secure :-1 s=READY pgs=1060 cs=0 l=1 rev1=1 crypto rx=0x7fbd5400c520 tx=0x7fbd5400c8e0 comp rx=0 tx=0).stop 2023-11-24T08:00:12.531 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:12.529+0000 7fbd6a4fa700 1 -- 172.21.15.59:0/88549919 shutdown_connections 2023-11-24T08:00:12.531 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:12.529+0000 7fbd6a4fa700 1 --2- 172.21.15.59:0/88549919 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7fbd4406b470 0x7fbd4406d930 unknown :-1 s=CLOSED pgs=487 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:00:12.531 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:12.529+0000 7fbd6a4fa700 1 --2- 172.21.15.59:0/88549919 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fbd641440f0 0x7fbd641f3c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:00:12.531 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:12.529+0000 7fbd6a4fa700 1 --2- 172.21.15.59:0/88549919 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbd640722c0 0x7fbd6413edd0 unknown :-1 s=CLOSED pgs=1060 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:00:12.532 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:12.529+0000 7fbd6a4fa700 1 --2- 172.21.15.59:0/88549919 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fbd64143ae0 0x7fbd640c8280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:00:12.532 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:12.529+0000 7fbd6a4fa700 1 -- 172.21.15.59:0/88549919 >> 172.21.15.59:0/88549919 conn(0x7fbd640b2740 msgr2=0x7fbd64071bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:00:12.532 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:12.529+0000 7fbd6a4fa700 1 -- 172.21.15.59:0/88549919 shutdown_connections 2023-11-24T08:00:12.532 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:12.529+0000 7fbd6a4fa700 1 -- 172.21.15.59:0/88549919 wait complete. 2023-11-24T08:00:12.542 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume create cephfs subvol773 --group_name subvol_grp1997 2023-11-24T08:00:12.672 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:12.670+0000 7fb9a835f700 1 Processor -- start 2023-11-24T08:00:12.672 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:12.670+0000 7fb9a835f700 1 -- start start 2023-11-24T08:00:12.674 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:12.671+0000 7fb9a835f700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fb9a0072260 0x7fb9a015bc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:00:12.674 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:12.671+0000 7fb9a835f700 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb9a0140ae0 con 0x7fb9a0140340 2023-11-24T08:00:12.674 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:12.671+0000 7fb9a835f700 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb9a0140dc0 con 0x7fb9a0071e80 2023-11-24T08:00:12.674 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:12.671+0000 7fb9a835f700 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb9a006dc30 con 0x7fb9a0072260 2023-11-24T08:00:12.674 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:12.672+0000 7fb9a58fa700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fb9a0072260 0x7fb9a015bc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:00:12.674 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:12.672+0000 7fb9a68fc700 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7fb9a0140340 0x7fb9a00b5110 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:59572/0 (socket says 172.21.15.59:59572) 2023-11-24T08:00:12.674 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:12.672+0000 7fb9a68fc700 1 -- 172.21.15.59:0/4269439067 learned_addr learned my addr 172.21.15.59:0/4269439067 (peer_addr_for_me v1:172.21.15.59:0/0) 2023-11-24T08:00:12.674 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:12.672+0000 7fb9a58fa700 1 -- 172.21.15.59:0/4269439067 >> v1:172.21.15.76:6789/0 conn(0x7fb9a0071e80 legacy=0x7fb9a006e360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:00:12.675 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:12.672+0000 7fb9a58fa700 1 -- 172.21.15.59:0/4269439067 >> v1:172.21.15.59:6789/0 conn(0x7fb9a0140340 legacy=0x7fb9a00b5110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:00:12.675 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:12.672+0000 7fb9a58fa700 1 -- 172.21.15.59:0/4269439067 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb9a01410a0 con 0x7fb9a0072260 2023-11-24T08:00:12.675 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:12.672+0000 7fb9a50f9700 1 -- 172.21.15.59:0/4269439067 <== mon.0 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3722230286 0 0) 0x7fb9a0140ae0 con 0x7fb9a0140340 2023-11-24T08:00:12.675 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:12.672+0000 7fb9a58fa700 1 --2- 172.21.15.59:0/4269439067 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb9a0072260 0x7fb9a015bc10 secure :-1 s=READY pgs=1062 cs=0 l=1 rev1=1 crypto rx=0x7fb99c009840 tx=0x7fb99c009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=381bd5f9f68b851e server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:00:12.675 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:12.672+0000 7fb9a50f9700 1 -- 172.21.15.59:0/4269439067 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb99c004230 con 0x7fb9a0072260 2023-11-24T08:00:12.675 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:12.672+0000 7fb9a50f9700 1 -- 172.21.15.59:0/4269439067 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb99c004390 con 0x7fb9a0072260 2023-11-24T08:00:12.675 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:12.672+0000 7fb9a50f9700 1 -- 172.21.15.59:0/4269439067 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb99c0036a0 con 0x7fb9a0072260 2023-11-24T08:00:12.678 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:12.673+0000 7fb9a835f700 1 -- 172.21.15.59:0/4269439067 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb9a0072260 msgr2=0x7fb9a015bc10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:00:12.678 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:12.673+0000 7fb9a835f700 1 --2- 172.21.15.59:0/4269439067 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb9a0072260 0x7fb9a015bc10 secure :-1 s=READY pgs=1062 cs=0 l=1 rev1=1 crypto rx=0x7fb99c009840 tx=0x7fb99c009b50 comp rx=0 tx=0).stop 2023-11-24T08:00:12.678 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:12.673+0000 7fb9a835f700 1 -- 172.21.15.59:0/4269439067 shutdown_connections 2023-11-24T08:00:12.678 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:12.673+0000 7fb9a835f700 1 --2- 172.21.15.59:0/4269439067 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb9a0072260 0x7fb9a015bc10 unknown :-1 s=CLOSED pgs=1062 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:00:12.678 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:12.673+0000 7fb9a835f700 1 -- 172.21.15.59:0/4269439067 >> 172.21.15.59:0/4269439067 conn(0x7fb9a00b2840 msgr2=0x7fb9a00b2c20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:00:12.678 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:12.673+0000 7fb9a835f700 1 -- 172.21.15.59:0/4269439067 shutdown_connections 2023-11-24T08:00:12.679 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:12.673+0000 7fb9a835f700 1 -- 172.21.15.59:0/4269439067 wait complete. 2023-11-24T08:00:12.679 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:12.674+0000 7fb9a835f700 1 Processor -- start 2023-11-24T08:00:12.679 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:12.674+0000 7fb9a835f700 1 -- start start 2023-11-24T08:00:12.679 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:12.675+0000 7fb9a835f700 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fb9a0071e80 0x7fb9a00cea80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:00:12.679 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:12.675+0000 7fb9a835f700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb9a0072260 0x7fb9a0154240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:00:12.679 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:12.675+0000 7fb9a835f700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb9a0140340 0x7fb9a01581a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:00:12.679 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:12.675+0000 7fb9a835f700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9a01f1cc0 con 0x7fb9a0072260 2023-11-24T08:00:12.679 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:12.675+0000 7fb9a835f700 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9a01555e0 con 0x7fb9a0140340 2023-11-24T08:00:12.680 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:12.675+0000 7fb9a835f700 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb9a00cf180 con 0x7fb9a0071e80 2023-11-24T08:00:12.680 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:12.675+0000 7fb9a58fa700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb9a0072260 0x7fb9a0154240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:00:12.680 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:12.675+0000 7fb9a58fa700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb9a0072260 0x7fb9a0154240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:56102/0 (socket says 172.21.15.59:56102) 2023-11-24T08:00:12.680 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:12.675+0000 7fb9a58fa700 1 -- 172.21.15.59:0/3175091559 learned_addr learned my addr 172.21.15.59:0/3175091559 (peer_addr_for_me v2:172.21.15.59:0/0) 2023-11-24T08:00:12.680 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:12.675+0000 7fb9a60fb700 1 --2- 172.21.15.59:0/3175091559 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fb9a0071e80 0x7fb9a00cea80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:00:12.680 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:12.675+0000 7fb9a68fc700 1 --2- 172.21.15.59:0/3175091559 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb9a0140340 0x7fb9a01581a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:00:12.680 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:12.675+0000 7fb9a58fa700 1 -- 172.21.15.59:0/3175091559 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fb9a0071e80 msgr2=0x7fb9a00cea80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:00:12.680 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:12.675+0000 7fb9a58fa700 1 --2- 172.21.15.59:0/3175091559 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fb9a0071e80 0x7fb9a00cea80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:00:12.680 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:12.675+0000 7fb9a58fa700 1 -- 172.21.15.59:0/3175091559 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb9a0140340 msgr2=0x7fb9a01581a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:00:12.680 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:12.675+0000 7fb9a58fa700 1 --2- 172.21.15.59:0/3175091559 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb9a0140340 0x7fb9a01581a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:00:12.680 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:12.675+0000 7fb9a58fa700 1 -- 172.21.15.59:0/3175091559 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb99c0094a0 con 0x7fb9a0072260 2023-11-24T08:00:12.680 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:12.675+0000 7fb9a60fb700 1 --2- 172.21.15.59:0/3175091559 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fb9a0071e80 0x7fb9a00cea80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-24T08:00:12.680 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:12.675+0000 7fb9a68fc700 1 --2- 172.21.15.59:0/3175091559 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb9a0140340 0x7fb9a01581a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-24T08:00:12.681 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:12.676+0000 7fb9a58fa700 1 --2- 172.21.15.59:0/3175091559 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb9a0072260 0x7fb9a0154240 secure :-1 s=READY pgs=1063 cs=0 l=1 rev1=1 crypto rx=0x7fb99c0090b0 tx=0x7fb99c004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:00:12.681 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:12.676+0000 7fb996ffd700 1 -- 172.21.15.59:0/3175091559 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb99c011830 con 0x7fb9a0072260 2023-11-24T08:00:12.681 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:12.676+0000 7fb996ffd700 1 -- 172.21.15.59:0/3175091559 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb99c01e070 con 0x7fb9a0072260 2023-11-24T08:00:12.681 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:12.676+0000 7fb9a835f700 1 -- 172.21.15.59:0/3175091559 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb9a00cf460 con 0x7fb9a0072260 2023-11-24T08:00:12.681 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:12.676+0000 7fb996ffd700 1 -- 172.21.15.59:0/3175091559 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb99c011bb0 con 0x7fb9a0072260 2023-11-24T08:00:12.681 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:12.676+0000 7fb9a835f700 1 -- 172.21.15.59:0/3175091559 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb9a0155830 con 0x7fb9a0072260 2023-11-24T08:00:12.682 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:12.677+0000 7fb9a835f700 1 -- 172.21.15.59:0/3175091559 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb9a0155ba0 con 0x7fb9a0072260 2023-11-24T08:00:12.682 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:12.677+0000 7fb996ffd700 1 -- 172.21.15.59:0/3175091559 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 11) v1 ==== 89741+0+0 (secure 0 0 0) 0x7fb99c011d10 con 0x7fb9a0072260 2023-11-24T08:00:12.682 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:12.678+0000 7fb996ffd700 1 --2- 172.21.15.59:0/3175091559 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7fb98406b470 0x7fb98406d930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:00:12.682 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:12.678+0000 7fb996ffd700 1 -- 172.21.15.59:0/3175091559 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fb99c08b3f0 con 0x7fb9a0072260 2023-11-24T08:00:12.682 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:12.678+0000 7fb9a60fb700 1 --2- 172.21.15.59:0/3175091559 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7fb98406b470 0x7fb98406d930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:00:12.683 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:12.678+0000 7fb9a60fb700 1 --2- 172.21.15.59:0/3175091559 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7fb98406b470 0x7fb98406d930 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7fb990003fc0 tx=0x7fb990009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:00:12.683 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:12.682+0000 7fb996ffd700 1 -- 172.21.15.59:0/3175091559 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fb99c055c80 con 0x7fb9a0072260 2023-11-24T08:00:12.918 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:12.916+0000 7fb9a835f700 1 -- 172.21.15.59:0/3175091559 --> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "subvol773", "group_name": "subvol_grp1997", "target": ["mon-mgr", ""]}) v1 -- 0x7fb9a006ea70 con 0x7fb98406b470 2023-11-24T08:00:12.944 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:12.943+0000 7fb996ffd700 1 -- 172.21.15.59:0/3175091559 <== mgr.4100 v2:172.21.15.76:6832/4264046536 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7fb990003bb0 con 0x7fb98406b470 2023-11-24T08:00:12.947 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:12.945+0000 7fb9a835f700 1 -- 172.21.15.59:0/3175091559 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7fb98406b470 msgr2=0x7fb98406d930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:00:12.947 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:12.945+0000 7fb9a835f700 1 --2- 172.21.15.59:0/3175091559 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7fb98406b470 0x7fb98406d930 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7fb990003fc0 tx=0x7fb990009040 comp rx=0 tx=0).stop 2023-11-24T08:00:12.947 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:12.945+0000 7fb9a835f700 1 -- 172.21.15.59:0/3175091559 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb9a0072260 msgr2=0x7fb9a0154240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:00:12.947 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:12.945+0000 7fb9a835f700 1 --2- 172.21.15.59:0/3175091559 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb9a0072260 0x7fb9a0154240 secure :-1 s=READY pgs=1063 cs=0 l=1 rev1=1 crypto rx=0x7fb99c0090b0 tx=0x7fb99c004750 comp rx=0 tx=0).stop 2023-11-24T08:00:12.947 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:12.945+0000 7fb9a835f700 1 -- 172.21.15.59:0/3175091559 shutdown_connections 2023-11-24T08:00:12.947 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:12.945+0000 7fb9a835f700 1 --2- 172.21.15.59:0/3175091559 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7fb98406b470 0x7fb98406d930 unknown :-1 s=CLOSED pgs=488 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:00:12.949 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:12.945+0000 7fb9a835f700 1 --2- 172.21.15.59:0/3175091559 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb9a0140340 0x7fb9a01581a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:00:12.949 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:12.945+0000 7fb9a835f700 1 --2- 172.21.15.59:0/3175091559 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fb9a0071e80 0x7fb9a00cea80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:00:12.949 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:12.945+0000 7fb9a835f700 1 --2- 172.21.15.59:0/3175091559 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb9a0072260 0x7fb9a0154240 unknown :-1 s=CLOSED pgs=1063 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:00:12.949 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:12.945+0000 7fb9a835f700 1 -- 172.21.15.59:0/3175091559 >> 172.21.15.59:0/3175091559 conn(0x7fb9a00b2840 msgr2=0x7fb9a015d560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:00:12.949 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:12.945+0000 7fb9a835f700 1 -- 172.21.15.59:0/3175091559 shutdown_connections 2023-11-24T08:00:12.949 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:12.945+0000 7fb9a835f700 1 -- 172.21.15.59:0/3175091559 wait complete. 2023-11-24T08:00:12.959 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-11-24T08:00:12.959 DEBUG:teuthology.orchestra.run.smithi059:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-11-24T08:00:13.014 DEBUG:teuthology.orchestra.run.smithi076:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-11-24T08:00:13.056 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-11-24T08:00:13.056 DEBUG:teuthology.orchestra.run.smithi059:> set -ex 2023-11-24T08:00:13.057 DEBUG:teuthology.orchestra.run.smithi059:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-24T08:00:13.074 DEBUG:teuthology.orchestra.run.smithi059:> set -ex 2023-11-24T08:00:13.074 DEBUG:teuthology.orchestra.run.smithi059:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-24T08:00:13.132 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-24T08:00:13.297 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:13.296+0000 7fbddf59e700 1 Processor -- start 2023-11-24T08:00:13.298 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:13.296+0000 7fbddf59e700 1 -- start start 2023-11-24T08:00:13.301 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:13.297+0000 7fbddf59e700 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fbde00cda90 0x7fbde00af6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:00:13.301 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:13.297+0000 7fbddf59e700 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbde006f7c0 con 0x7fbde00d1670 2023-11-24T08:00:13.302 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:13.297+0000 7fbddf59e700 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbde006dc20 con 0x7fbde015ac40 2023-11-24T08:00:13.302 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:13.297+0000 7fbddf59e700 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbde00afc00 con 0x7fbde00cda90 2023-11-24T08:00:13.302 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:13.297+0000 7fbdded9d700 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7fbde00d1670 0x7fbde00cf890 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:59586/0 (socket says 172.21.15.59:59586) 2023-11-24T08:00:13.302 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:13.297+0000 7fbdded9d700 1 -- 172.21.15.59:0/4224352719 learned_addr learned my addr 172.21.15.59:0/4224352719 (peer_addr_for_me v1:172.21.15.59:0/0) 2023-11-24T08:00:13.302 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:13.297+0000 7fbdddd9b700 1 --2- 172.21.15.59:0/4224352719 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fbde00cda90 0x7fbde00af6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:00:13.302 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:13.297+0000 7fbddd59a700 1 -- 172.21.15.59:0/4224352719 <== mon.0 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2305899098 0 0) 0x7fbde006f7c0 con 0x7fbde00d1670 2023-11-24T08:00:13.302 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:13.297+0000 7fbddd59a700 1 -- 172.21.15.59:0/4224352719 --> v1:172.21.15.59:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbdc80035b0 con 0x7fbde00d1670 2023-11-24T08:00:13.302 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:13.297+0000 7fbddd59a700 1 -- 172.21.15.59:0/4224352719 <== mon.1 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 355337610 0 0) 0x7fbde006dc20 con 0x7fbde015ac40 2023-11-24T08:00:13.302 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:13.297+0000 7fbddd59a700 1 -- 172.21.15.59:0/4224352719 --> v1:172.21.15.76:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbde006f7c0 con 0x7fbde015ac40 2023-11-24T08:00:13.302 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:13.297+0000 7fbddd59a700 1 -- 172.21.15.59:0/4224352719 <== mon.0 v1:172.21.15.59:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3052987530 0 0) 0x7fbdc80035b0 con 0x7fbde00d1670 2023-11-24T08:00:13.303 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:13.297+0000 7fbddd59a700 1 -- 172.21.15.59:0/4224352719 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fbde00cda90 msgr2=0x7fbde00af6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:00:13.303 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:13.297+0000 7fbddd59a700 1 --2- 172.21.15.59:0/4224352719 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fbde00cda90 0x7fbde00af6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:00:13.303 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:13.297+0000 7fbddd59a700 1 -- 172.21.15.59:0/4224352719 >> v1:172.21.15.76:6789/0 conn(0x7fbde015ac40 legacy=0x7fbde00cd380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:00:13.303 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:13.297+0000 7fbddd59a700 1 -- 172.21.15.59:0/4224352719 --> v1:172.21.15.59:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbde015b020 con 0x7fbde00d1670 2023-11-24T08:00:13.303 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:13.297+0000 7fbddd59a700 1 -- 172.21.15.59:0/4224352719 <== mon.0 v1:172.21.15.59:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1312333566 0 0) 0x7fbdd80019f0 con 0x7fbde00d1670 2023-11-24T08:00:13.303 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:13.298+0000 7fbddd59a700 1 -- 172.21.15.59:0/4224352719 >> v1:172.21.15.59:6789/0 conn(0x7fbde00d1670 legacy=0x7fbde00cf890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:00:13.303 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:13.298+0000 7fbddd59a700 1 --2- 172.21.15.59:0/4224352719 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbdc8003bc0 0x7fbdc8003fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:00:13.303 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:13.298+0000 7fbddd59a700 1 --2- 172.21.15.59:0/4224352719 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fbdc8004850 0x7fbdc8004cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:00:13.303 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:13.298+0000 7fbddd59a700 1 --2- 172.21.15.59:0/4224352719 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fbde00cda90 0x7fbdc8005210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:00:13.303 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:13.298+0000 7fbddd59a700 1 -- 172.21.15.59:0/4224352719 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbdc800bac0 con 0x7fbdc8003bc0 2023-11-24T08:00:13.303 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:13.298+0000 7fbddd59a700 1 -- 172.21.15.59:0/4224352719 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbdc800bc00 con 0x7fbdc8004850 2023-11-24T08:00:13.304 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:13.298+0000 7fbddd59a700 1 -- 172.21.15.59:0/4224352719 --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbdc800bd90 con 0x7fbde00cda90 2023-11-24T08:00:13.304 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:13.298+0000 7fbdde59c700 1 --2- 172.21.15.59:0/4224352719 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbdc8003bc0 0x7fbdc8003fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:00:13.304 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:13.298+0000 7fbdded9d700 1 --2- 172.21.15.59:0/4224352719 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fbde00cda90 0x7fbdc8005210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:00:13.304 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:13.298+0000 7fbdddd9b700 1 --2- 172.21.15.59:0/4224352719 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fbdc8004850 0x7fbdc8004cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:00:13.304 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:13.298+0000 7fbdde59c700 1 -- 172.21.15.59:0/4224352719 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fbde00cda90 msgr2=0x7fbdc8005210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:00:13.304 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:13.298+0000 7fbdde59c700 1 --2- 172.21.15.59:0/4224352719 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fbde00cda90 0x7fbdc8005210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:00:13.304 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:13.298+0000 7fbdde59c700 1 -- 172.21.15.59:0/4224352719 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fbdc8004850 msgr2=0x7fbdc8004cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:00:13.304 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:13.298+0000 7fbdde59c700 1 --2- 172.21.15.59:0/4224352719 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fbdc8004850 0x7fbdc8004cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:00:13.304 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:13.298+0000 7fbdde59c700 1 -- 172.21.15.59:0/4224352719 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbdc80035b0 con 0x7fbdc8003bc0 2023-11-24T08:00:13.304 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:13.298+0000 7fbdde59c700 1 --2- 172.21.15.59:0/4224352719 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbdc8003bc0 0x7fbdc8003fc0 secure :-1 s=READY pgs=1065 cs=0 l=1 rev1=1 crypto rx=0x7fbdd0001980 tx=0x7fbdc8005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7ed4733ee064578d server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:00:13.304 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:13.298+0000 7fbddd59a700 1 -- 172.21.15.59:0/4224352719 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbdd0001470 con 0x7fbdc8003bc0 2023-11-24T08:00:13.304 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:13.298+0000 7fbddd59a700 1 -- 172.21.15.59:0/4224352719 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbdd00052d0 con 0x7fbdc8003bc0 2023-11-24T08:00:13.305 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:13.298+0000 7fbddd59a700 1 -- 172.21.15.59:0/4224352719 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbdd0003680 con 0x7fbdc8003bc0 2023-11-24T08:00:13.305 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:13.299+0000 7fbddf59e700 1 -- 172.21.15.59:0/4224352719 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbdc8003bc0 msgr2=0x7fbdc8003fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:00:13.305 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:13.299+0000 7fbddf59e700 1 --2- 172.21.15.59:0/4224352719 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbdc8003bc0 0x7fbdc8003fc0 secure :-1 s=READY pgs=1065 cs=0 l=1 rev1=1 crypto rx=0x7fbdd0001980 tx=0x7fbdc8005880 comp rx=0 tx=0).stop 2023-11-24T08:00:13.305 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:13.299+0000 7fbdded9d700 1 -- 172.21.15.59:0/4224352719 reap_dead start 2023-11-24T08:00:13.305 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:13.299+0000 7fbddf59e700 1 -- 172.21.15.59:0/4224352719 shutdown_connections 2023-11-24T08:00:13.305 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:13.299+0000 7fbddf59e700 1 -- 172.21.15.59:0/4224352719 >> 172.21.15.59:0/4224352719 conn(0x7fbde00d09c0 msgr2=0x7fbde00d0dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:00:13.305 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:13.299+0000 7fbddf59e700 1 -- 172.21.15.59:0/4224352719 shutdown_connections 2023-11-24T08:00:13.305 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:13.299+0000 7fbddf59e700 1 -- 172.21.15.59:0/4224352719 wait complete. 2023-11-24T08:00:13.308 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:13.300+0000 7fbddf59e700 1 Processor -- start 2023-11-24T08:00:13.308 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:13.300+0000 7fbddf59e700 1 -- start start 2023-11-24T08:00:13.308 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:13.301+0000 7fbddf59e700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fbdc8003bc0 0x7fbde014b0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:00:13.308 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:13.301+0000 7fbddf59e700 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fbde0147320 0x7fbde015bc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:00:13.308 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:13.301+0000 7fbddf59e700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbde015c160 0x7fbde006afd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:00:13.308 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:13.301+0000 7fbddf59e700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbde01589a0 con 0x7fbde015c160 2023-11-24T08:00:13.309 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:13.301+0000 7fbddf59e700 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbde0158c40 con 0x7fbdc8003bc0 2023-11-24T08:00:13.309 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:13.301+0000 7fbddf59e700 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbde0158f20 con 0x7fbde0147320 2023-11-24T08:00:13.309 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:13.301+0000 7fbdded9d700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbde015c160 0x7fbde006afd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:00:13.309 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:13.301+0000 7fbdded9d700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbde015c160 0x7fbde006afd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:56124/0 (socket says 172.21.15.59:56124) 2023-11-24T08:00:13.309 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:13.301+0000 7fbdded9d700 1 -- 172.21.15.59:0/927583612 learned_addr learned my addr 172.21.15.59:0/927583612 (peer_addr_for_me v2:172.21.15.59:0/0) 2023-11-24T08:00:13.309 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:13.301+0000 7fbdddd9b700 1 --2- 172.21.15.59:0/927583612 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fbde0147320 0x7fbde015bc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:00:13.309 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:13.301+0000 7fbdde59c700 1 --2- 172.21.15.59:0/927583612 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fbdc8003bc0 0x7fbde014b0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:00:13.309 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:13.302+0000 7fbdded9d700 1 -- 172.21.15.59:0/927583612 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fbde0147320 msgr2=0x7fbde015bc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:00:13.309 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:13.302+0000 7fbdded9d700 1 --2- 172.21.15.59:0/927583612 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fbde0147320 0x7fbde015bc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:00:13.309 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:13.302+0000 7fbdded9d700 1 -- 172.21.15.59:0/927583612 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fbdc8003bc0 msgr2=0x7fbde014b0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:00:13.309 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:13.302+0000 7fbdded9d700 1 --2- 172.21.15.59:0/927583612 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fbdc8003bc0 0x7fbde014b0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:00:13.309 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:13.302+0000 7fbdded9d700 1 -- 172.21.15.59:0/927583612 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbde0159200 con 0x7fbde015c160 2023-11-24T08:00:13.310 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:13.302+0000 7fbdddd9b700 1 --2- 172.21.15.59:0/927583612 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fbde0147320 0x7fbde015bc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T08:00:13.310 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:13.302+0000 7fbdded9d700 1 --2- 172.21.15.59:0/927583612 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbde015c160 0x7fbde006afd0 secure :-1 s=READY pgs=1066 cs=0 l=1 rev1=1 crypto rx=0x7fbde0070390 tx=0x7fbdd800a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:00:13.310 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:13.302+0000 7fbdceffd700 1 -- 172.21.15.59:0/927583612 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbdd80133f0 con 0x7fbde015c160 2023-11-24T08:00:13.310 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:13.302+0000 7fbdceffd700 1 -- 172.21.15.59:0/927583612 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbdd8013590 con 0x7fbde015c160 2023-11-24T08:00:13.310 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:13.302+0000 7fbddf59e700 1 -- 172.21.15.59:0/927583612 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbde015c670 con 0x7fbde015c160 2023-11-24T08:00:13.310 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:13.302+0000 7fbdceffd700 1 -- 172.21.15.59:0/927583612 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbdd8013940 con 0x7fbde015c160 2023-11-24T08:00:13.310 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:13.302+0000 7fbddf59e700 1 -- 172.21.15.59:0/927583612 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbde0154000 con 0x7fbde015c160 2023-11-24T08:00:13.310 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:13.303+0000 7fbddf59e700 1 -- 172.21.15.59:0/927583612 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbde0075300 con 0x7fbde015c160 2023-11-24T08:00:13.310 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:13.303+0000 7fbdceffd700 1 -- 172.21.15.59:0/927583612 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 11) v1 ==== 89741+0+0 (secure 0 0 0) 0x7fbdd8013aa0 con 0x7fbde015c160 2023-11-24T08:00:13.310 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:13.304+0000 7fbdceffd700 1 --2- 172.21.15.59:0/927583612 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7fbdc006b470 0x7fbdc006d930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:00:13.310 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:13.304+0000 7fbdceffd700 1 -- 172.21.15.59:0/927583612 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fbdd808aae0 con 0x7fbde015c160 2023-11-24T08:00:13.311 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:13.304+0000 7fbdde59c700 1 --2- 172.21.15.59:0/927583612 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7fbdc006b470 0x7fbdc006d930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:00:13.311 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:13.305+0000 7fbdde59c700 1 --2- 172.21.15.59:0/927583612 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7fbdc006b470 0x7fbdc006d930 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7fbdd0015040 tx=0x7fbdd000a000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:00:13.311 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:13.308+0000 7fbdceffd700 1 -- 172.21.15.59:0/927583612 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fbdd80553b0 con 0x7fbde015c160 2023-11-24T08:00:13.514 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:13.512+0000 7fbddf59e700 1 -- 172.21.15.59:0/927583612 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fbde0154000 con 0x7fbde015c160 2023-11-24T08:00:13.515 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:13.514+0000 7fbdceffd700 1 -- 172.21.15.59:0/927583612 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v87) v1 ==== 74+0+16500 (secure 0 0 0) 0x7fbde0154000 con 0x7fbde015c160 2023-11-24T08:00:13.516 INFO:teuthology.orchestra.run.smithi059.stdout: 2023-11-24T08:00:13.516 INFO:teuthology.orchestra.run.smithi059.stdout:{"epoch":87,"fsid":"9f0e8741-b8f9-48d5-a2c3-09f45903303d","created":"2023-11-24T07:56:06.552729+0000","modified":"2023-11-24T07:59:53.500645+0000","last_up_change":"2023-11-24T07:56:17.372352+0000","last_in_change":"2023-11-24T07:56:09.073250+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":13,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-24T07:56:18.352823+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_metadata","create_time":"2023-11-24T07:59:50.274996+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_data","create_time":"2023-11-24T07:59:50.873919+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"87","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"6d8f9e9b-1782-4e41-928b-a39f2a8ebc06","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6800","nonce":3373878772},{"type":"v1","addr":"172.21.15.59:6801","nonce":3373878772}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6802","nonce":3373878772},{"type":"v1","addr":"172.21.15.59:6803","nonce":3373878772}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6806","nonce":3373878772},{"type":"v1","addr":"172.21.15.59:6807","nonce":3373878772}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6804","nonce":3373878772},{"type":"v1","addr":"172.21.15.59:6805","nonce":3373878772}]},"public_addr":"172.21.15.59:6801/3373878772","cluster_addr":"172.21.15.59:6803/3373878772","heartbeat_back_addr":"172.21.15.59:6807/3373878772","heartbeat_front_addr":"172.21.15.59:6805/3373878772","state":["exists","up"]},{"osd":1,"uuid":"e8d882aa-52e1-4c04-9e1e-c44ef9204a5f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6824","nonce":648258563},{"type":"v1","addr":"172.21.15.59:6825","nonce":648258563}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6826","nonce":648258563},{"type":"v1","addr":"172.21.15.59:6827","nonce":648258563}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6830","nonce":648258563},{"type":"v1","addr":"172.21.15.59:6831","nonce":648258563}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6828","nonce":648258563},{"type":"v1","addr":"172.21.15.59:6829","nonce":648258563}]},"public_addr":"172.21.15.59:6825/648258563","cluster_addr":"172.21.15.59:6827/648258563","heartbeat_back_addr":"172.21.15.59:6831/648258563","heartbeat_front_addr":"172.21.15.59:6829/648258563","state":["exists","up"]},{"osd":2,"uuid":"69eebfa0-695b-4b21-90b9-8a3537168a01","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6808","nonce":1840430654},{"type":"v1","addr":"172.21.15.59:6809","nonce":1840430654}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6810","nonce":1840430654},{"type":"v1","addr":"172.21.15.59:6811","nonce":1840430654}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6814","nonce":1840430654},{"type":"v1","addr":"172.21.15.59:6815","nonce":1840430654}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6812","nonce":1840430654},{"type":"v1","addr":"172.21.15.59:6813","nonce":1840430654}]},"public_addr":"172.21.15.59:6809/1840430654","cluster_addr":"172.21.15.59:6811/1840430654","heartbeat_back_addr":"172.21.15.59:6815/1840430654","heartbeat_front_addr":"172.21.15.59:6813/1840430654","state":["exists","up"]},{"osd":3,"uuid":"96fd77fc-f911-46ff-9445-6f0be2daa8f4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6816","nonce":2504025850},{"type":"v1","addr":"172.21.15.59:6817","nonce":2504025850}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6818","nonce":2504025850},{"type":"v1","addr":"172.21.15.59:6819","nonce":2504025850}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6822","nonce":2504025850},{"type":"v1","addr":"172.21.15.59:6823","nonce":2504025850}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6820","nonce":2504025850},{"type":"v1","addr":"172.21.15.59:6821","nonce":2504025850}]},"public_addr":"172.21.15.59:6817/2504025850","cluster_addr":"172.21.15.59:6819/2504025850","heartbeat_back_addr":"172.21.15.59:6823/2504025850","heartbeat_front_addr":"172.21.15.59:6821/2504025850","state":["exists","up"]},{"osd":4,"uuid":"7ac29fca-b5b2-4c95-9d69-2f164cbbc708","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6816","nonce":2382463431},{"type":"v1","addr":"172.21.15.76:6818","nonce":2382463431}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6820","nonce":2382463431},{"type":"v1","addr":"172.21.15.76:6822","nonce":2382463431}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6828","nonce":2382463431},{"type":"v1","addr":"172.21.15.76:6830","nonce":2382463431}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6824","nonce":2382463431},{"type":"v1","addr":"172.21.15.76:6826","nonce":2382463431}]},"public_addr":"172.21.15.76:6818/2382463431","cluster_addr":"172.21.15.76:6822/2382463431","heartbeat_back_addr":"172.21.15.76:6830/2382463431","heartbeat_front_addr":"172.21.15.76:6826/2382463431","state":["exists","up"]},{"osd":5,"uuid":"485bf711-3851-4238-a4f4-51d52e0a9c58","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6800","nonce":329334308},{"type":"v1","addr":"172.21.15.76:6802","nonce":329334308}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6805","nonce":329334308},{"type":"v1","addr":"172.21.15.76:6807","nonce":329334308}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6814","nonce":329334308},{"type":"v1","addr":"172.21.15.76:6815","nonce":329334308}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6810","nonce":329334308},{"type":"v1","addr":"172.21.15.76:6812","nonce":329334308}]},"public_addr":"172.21.15.76:6802/329334308","cluster_addr":"172.21.15.76:6807/329334308","heartbeat_back_addr":"172.21.15.76:6815/329334308","heartbeat_front_addr":"172.21.15.76:6812/329334308","state":["exists","up"]},{"osd":6,"uuid":"72b3f2fc-1e4a-4e32-968d-003647a0286f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6801","nonce":3946579410},{"type":"v1","addr":"172.21.15.76:6803","nonce":3946579410}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6804","nonce":3946579410},{"type":"v1","addr":"172.21.15.76:6806","nonce":3946579410}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6811","nonce":3946579410},{"type":"v1","addr":"172.21.15.76:6813","nonce":3946579410}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6808","nonce":3946579410},{"type":"v1","addr":"172.21.15.76:6809","nonce":3946579410}]},"public_addr":"172.21.15.76:6803/3946579410","cluster_addr":"172.21.15.76:6806/3946579410","heartbeat_back_addr":"172.21.15.76:6813/3946579410","heartbeat_front_addr":"172.21.15.76:6809/3946579410","state":["exists","up"]},{"osd":7,"uuid":"1ddceb90-6c5d-4d1a-ad56-99cb877f8c70","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6817","nonce":841761538},{"type":"v1","addr":"172.21.15.76:6819","nonce":841761538}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6821","nonce":841761538},{"type":"v1","addr":"172.21.15.76:6823","nonce":841761538}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6829","nonce":841761538},{"type":"v1","addr":"172.21.15.76:6831","nonce":841761538}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6825","nonce":841761538},{"type":"v1","addr":"172.21.15.76:6827","nonce":841761538}]},"public_addr":"172.21.15.76:6819/841761538","cluster_addr":"172.21.15.76:6823/841761538","heartbeat_back_addr":"172.21.15.76:6831/841761538","heartbeat_front_addr":"172.21.15.76:6827/841761538","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T07:56:14.294838+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T07:56:14.270779+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T07:56:14.231809+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T07:56:14.260983+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T07:56:14.302997+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T07:56:14.258600+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T07:56:14.251413+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T07:56:14.304727+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-24T08:00:13.518 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:13.516+0000 7fbddf59e700 1 -- 172.21.15.59:0/927583612 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7fbdc006b470 msgr2=0x7fbdc006d930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:00:13.518 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:13.516+0000 7fbddf59e700 1 --2- 172.21.15.59:0/927583612 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7fbdc006b470 0x7fbdc006d930 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7fbdd0015040 tx=0x7fbdd000a000 comp rx=0 tx=0).stop 2023-11-24T08:00:13.518 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:13.516+0000 7fbddf59e700 1 -- 172.21.15.59:0/927583612 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbde015c160 msgr2=0x7fbde006afd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:00:13.518 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:13.516+0000 7fbddf59e700 1 --2- 172.21.15.59:0/927583612 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbde015c160 0x7fbde006afd0 secure :-1 s=READY pgs=1066 cs=0 l=1 rev1=1 crypto rx=0x7fbde0070390 tx=0x7fbdd800a660 comp rx=0 tx=0).stop 2023-11-24T08:00:13.518 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:13.516+0000 7fbddf59e700 1 -- 172.21.15.59:0/927583612 shutdown_connections 2023-11-24T08:00:13.519 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:13.516+0000 7fbddf59e700 1 --2- 172.21.15.59:0/927583612 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7fbdc006b470 0x7fbdc006d930 unknown :-1 s=CLOSED pgs=489 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:00:13.519 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:13.516+0000 7fbddf59e700 1 --2- 172.21.15.59:0/927583612 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbde015c160 0x7fbde006afd0 unknown :-1 s=CLOSED pgs=1066 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:00:13.519 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:13.516+0000 7fbddf59e700 1 --2- 172.21.15.59:0/927583612 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fbdc8003bc0 0x7fbde014b0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:00:13.519 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:13.516+0000 7fbddf59e700 1 --2- 172.21.15.59:0/927583612 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fbde0147320 0x7fbde015bc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:00:13.519 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:13.516+0000 7fbddf59e700 1 -- 172.21.15.59:0/927583612 >> 172.21.15.59:0/927583612 conn(0x7fbde00d09c0 msgr2=0x7fbde00b1bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:00:13.519 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:13.516+0000 7fbddf59e700 1 -- 172.21.15.59:0/927583612 shutdown_connections 2023-11-24T08:00:13.519 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:13.516+0000 7fbddf59e700 1 -- 172.21.15.59:0/927583612 wait complete. 2023-11-24T08:00:13.530 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-24T07:56:18.352823+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 12, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-24T07:59:50.274996+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '86', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 13, 'pool_name': 'cephfs_data', 'create_time': '2023-11-24T07:59:50.873919+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '87', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-24T08:00:13.531 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-24T08:00:13.659 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:13.657+0000 7f5675fcd700 1 Processor -- start 2023-11-24T08:00:13.659 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:13.657+0000 7f5675fcd700 1 -- start start 2023-11-24T08:00:13.661 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:13.658+0000 7f5675fcd700 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f56700b4150 0x7f567006de50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:00:13.661 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:13.658+0000 7f5675fcd700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f56700b5ab0 0x7f567006e390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:00:13.661 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:13.658+0000 7f5675fcd700 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5670071f50 con 0x7f567006b620 2023-11-24T08:00:13.661 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:13.658+0000 7f5675fcd700 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5670072230 con 0x7f56700b5ab0 2023-11-24T08:00:13.661 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:13.658+0000 7f5675fcd700 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5670072510 con 0x7f56700b4150 2023-11-24T08:00:13.661 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:13.658+0000 7f566ffff700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f56700b5ab0 0x7f567006e390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:00:13.661 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:13.658+0000 7f566ffff700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f56700b5ab0 0x7f567006e390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:56132/0 (socket says 172.21.15.59:56132) 2023-11-24T08:00:13.662 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:13.658+0000 7f566ffff700 1 -- 172.21.15.59:0/1505020971 learned_addr learned my addr 172.21.15.59:0/1505020971 (peer_addr_for_me v2:172.21.15.59:0/0) 2023-11-24T08:00:13.662 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:13.658+0000 7f566effd700 1 --2- 172.21.15.59:0/1505020971 >> v2:172.21.15.76:3300/0 conn(0x7f56700b4150 0x7f567006de50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:00:13.662 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:13.658+0000 7f566ffff700 1 -- 172.21.15.59:0/1505020971 >> v2:172.21.15.76:3300/0 conn(0x7f56700b4150 msgr2=0x7f567006de50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:00:13.662 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:13.658+0000 7f566ffff700 1 --2- 172.21.15.59:0/1505020971 >> v2:172.21.15.76:3300/0 conn(0x7f56700b4150 0x7f567006de50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:00:13.662 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:13.658+0000 7f566ffff700 1 -- 172.21.15.59:0/1505020971 >> v1:172.21.15.76:6789/0 conn(0x7f567006b620 legacy=0x7f56700b3a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:00:13.662 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:13.658+0000 7f566ffff700 1 -- 172.21.15.59:0/1505020971 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f56700727f0 con 0x7f56700b5ab0 2023-11-24T08:00:13.662 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:13.658+0000 7f566ffff700 1 --2- 172.21.15.59:0/1505020971 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f56700b5ab0 0x7f567006e390 secure :-1 s=READY pgs=1067 cs=0 l=1 rev1=1 crypto rx=0x7f567006d6f0 tx=0x7f566800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8fe5b1f1a2766374 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:00:13.662 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:13.659+0000 7f566e7fc700 1 -- 172.21.15.59:0/1505020971 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f566800e070 con 0x7f56700b5ab0 2023-11-24T08:00:13.662 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:13.659+0000 7f566e7fc700 1 -- 172.21.15.59:0/1505020971 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5668003a20 con 0x7f56700b5ab0 2023-11-24T08:00:13.662 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:13.659+0000 7f566e7fc700 1 -- 172.21.15.59:0/1505020971 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5668003dd0 con 0x7f56700b5ab0 2023-11-24T08:00:13.662 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:13.659+0000 7f5675fcd700 1 -- 172.21.15.59:0/1505020971 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f56700b5ab0 msgr2=0x7f567006e390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:00:13.662 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:13.659+0000 7f5675fcd700 1 --2- 172.21.15.59:0/1505020971 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f56700b5ab0 0x7f567006e390 secure :-1 s=READY pgs=1067 cs=0 l=1 rev1=1 crypto rx=0x7f567006d6f0 tx=0x7f566800b330 comp rx=0 tx=0).stop 2023-11-24T08:00:13.663 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:13.659+0000 7f5675fcd700 1 -- 172.21.15.59:0/1505020971 shutdown_connections 2023-11-24T08:00:13.663 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:13.659+0000 7f5675fcd700 1 --2- 172.21.15.59:0/1505020971 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f56700b5ab0 0x7f567006e390 unknown :-1 s=CLOSED pgs=1067 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:00:13.663 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:13.659+0000 7f5675fcd700 1 --2- 172.21.15.59:0/1505020971 >> v2:172.21.15.76:3300/0 conn(0x7f56700b4150 0x7f567006de50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:00:13.663 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:13.659+0000 7f5675fcd700 1 -- 172.21.15.59:0/1505020971 >> 172.21.15.59:0/1505020971 conn(0x7f56700b2740 msgr2=0x7f56700b2b20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:00:13.663 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:13.659+0000 7f5675fcd700 1 -- 172.21.15.59:0/1505020971 shutdown_connections 2023-11-24T08:00:13.663 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:13.659+0000 7f5675fcd700 1 -- 172.21.15.59:0/1505020971 wait complete. 2023-11-24T08:00:13.666 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:13.661+0000 7f5675fcd700 1 Processor -- start 2023-11-24T08:00:13.666 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:13.661+0000 7f5675fcd700 1 -- start start 2023-11-24T08:00:13.666 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:13.662+0000 7f5675fcd700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f567006b620 0x7f56701e5260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:00:13.666 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:13.662+0000 7f5675fcd700 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f56700b4150 0x7f56700c59b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:00:13.666 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:13.662+0000 7f5675fcd700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f56700b5ab0 0x7f56700ca490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:00:13.666 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:13.662+0000 7f5675fcd700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56700c6fa0 con 0x7f567006b620 2023-11-24T08:00:13.666 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:13.662+0000 7f5675fcd700 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56701e57a0 con 0x7f56700b5ab0 2023-11-24T08:00:13.666 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:13.662+0000 7f5675fcd700 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f56700c73c0 con 0x7f56700b4150 2023-11-24T08:00:13.666 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:13.662+0000 7f566f7fe700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f567006b620 0x7f56701e5260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:00:13.666 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:13.662+0000 7f566f7fe700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f567006b620 0x7f56701e5260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:56136/0 (socket says 172.21.15.59:56136) 2023-11-24T08:00:13.667 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:13.662+0000 7f566f7fe700 1 -- 172.21.15.59:0/791684726 learned_addr learned my addr 172.21.15.59:0/791684726 (peer_addr_for_me v2:172.21.15.59:0/0) 2023-11-24T08:00:13.667 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:13.662+0000 7f566effd700 1 --2- 172.21.15.59:0/791684726 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f56700b4150 0x7f56700c59b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:00:13.667 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:13.662+0000 7f566ffff700 1 --2- 172.21.15.59:0/791684726 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f56700b5ab0 0x7f56700ca490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:00:13.667 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:13.662+0000 7f566f7fe700 1 -- 172.21.15.59:0/791684726 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f56700b4150 msgr2=0x7f56700c59b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:00:13.667 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:13.662+0000 7f566f7fe700 1 --2- 172.21.15.59:0/791684726 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f56700b4150 0x7f56700c59b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:00:13.667 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:13.662+0000 7f566f7fe700 1 -- 172.21.15.59:0/791684726 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f56700b5ab0 msgr2=0x7f56700ca490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:00:13.667 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:13.662+0000 7f566f7fe700 1 --2- 172.21.15.59:0/791684726 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f56700b5ab0 0x7f56700ca490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:00:13.667 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:13.662+0000 7f566f7fe700 1 -- 172.21.15.59:0/791684726 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5668009b30 con 0x7f567006b620 2023-11-24T08:00:13.667 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:13.662+0000 7f566effd700 1 --2- 172.21.15.59:0/791684726 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f56700b4150 0x7f56700c59b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T08:00:13.667 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:13.662+0000 7f566ffff700 1 --2- 172.21.15.59:0/791684726 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f56700b5ab0 0x7f56700ca490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T08:00:13.667 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:13.663+0000 7f566f7fe700 1 --2- 172.21.15.59:0/791684726 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f567006b620 0x7f56701e5260 secure :-1 s=READY pgs=1068 cs=0 l=1 rev1=1 crypto rx=0x7f566000c4a0 tx=0x7f566000c860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:00:13.667 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:13.663+0000 7f5667fff700 1 -- 172.21.15.59:0/791684726 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56600054e0 con 0x7f567006b620 2023-11-24T08:00:13.667 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:13.663+0000 7f5667fff700 1 -- 172.21.15.59:0/791684726 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5660003680 con 0x7f567006b620 2023-11-24T08:00:13.668 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:13.663+0000 7f5675fcd700 1 -- 172.21.15.59:0/791684726 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f56700c6320 con 0x7f567006b620 2023-11-24T08:00:13.668 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:13.663+0000 7f5667fff700 1 -- 172.21.15.59:0/791684726 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5660003a30 con 0x7f567006b620 2023-11-24T08:00:13.668 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:13.663+0000 7f5675fcd700 1 -- 172.21.15.59:0/791684726 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f56700c68e0 con 0x7f567006b620 2023-11-24T08:00:13.668 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:13.664+0000 7f5675fcd700 1 -- 172.21.15.59:0/791684726 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5670075480 con 0x7f567006b620 2023-11-24T08:00:13.668 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:13.664+0000 7f5667fff700 1 -- 172.21.15.59:0/791684726 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 11) v1 ==== 89741+0+0 (secure 0 0 0) 0x7f5660003b90 con 0x7f567006b620 2023-11-24T08:00:13.668 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:13.665+0000 7f5667fff700 1 --2- 172.21.15.59:0/791684726 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f565006b470 0x7f565006d930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:00:13.669 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:13.665+0000 7f5667fff700 1 -- 172.21.15.59:0/791684726 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f5660089100 con 0x7f567006b620 2023-11-24T08:00:13.669 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:13.665+0000 7f566effd700 1 --2- 172.21.15.59:0/791684726 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f565006b470 0x7f565006d930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:00:13.669 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:13.665+0000 7f566effd700 1 --2- 172.21.15.59:0/791684726 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f565006b470 0x7f565006d930 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7f56700b3700 tx=0x7f5658008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:00:13.671 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:13.669+0000 7f5667fff700 1 -- 172.21.15.59:0/791684726 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f56600bbe90 con 0x7f567006b620 2023-11-24T08:00:13.870 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:13.868+0000 7f5675fcd700 1 -- 172.21.15.59:0/791684726 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f56700b2520 con 0x7f567006b620 2023-11-24T08:00:13.870 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:13.869+0000 7f5667fff700 1 -- 172.21.15.59:0/791684726 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v87) v1 ==== 93+0+10 (secure 0 0 0) 0x7f5660053940 con 0x7f567006b620 2023-11-24T08:00:13.871 INFO:teuthology.orchestra.run.smithi059.stdout:pg_num: 1 2023-11-24T08:00:13.873 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:13.871+0000 7f5675fcd700 1 -- 172.21.15.59:0/791684726 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f565006b470 msgr2=0x7f565006d930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:00:13.873 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:13.871+0000 7f5675fcd700 1 --2- 172.21.15.59:0/791684726 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f565006b470 0x7f565006d930 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7f56700b3700 tx=0x7f5658008040 comp rx=0 tx=0).stop 2023-11-24T08:00:13.873 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:13.871+0000 7f5675fcd700 1 -- 172.21.15.59:0/791684726 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f567006b620 msgr2=0x7f56701e5260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:00:13.873 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:13.871+0000 7f5675fcd700 1 --2- 172.21.15.59:0/791684726 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f567006b620 0x7f56701e5260 secure :-1 s=READY pgs=1068 cs=0 l=1 rev1=1 crypto rx=0x7f566000c4a0 tx=0x7f566000c860 comp rx=0 tx=0).stop 2023-11-24T08:00:13.873 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:13.871+0000 7f5675fcd700 1 -- 172.21.15.59:0/791684726 shutdown_connections 2023-11-24T08:00:13.873 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:13.871+0000 7f5675fcd700 1 --2- 172.21.15.59:0/791684726 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f565006b470 0x7f565006d930 unknown :-1 s=CLOSED pgs=490 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:00:13.873 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:13.871+0000 7f5675fcd700 1 --2- 172.21.15.59:0/791684726 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f56700b5ab0 0x7f56700ca490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:00:13.873 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:13.871+0000 7f5675fcd700 1 --2- 172.21.15.59:0/791684726 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f567006b620 0x7f56701e5260 unknown :-1 s=CLOSED pgs=1068 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:00:13.874 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:13.871+0000 7f5675fcd700 1 --2- 172.21.15.59:0/791684726 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f56700b4150 0x7f56700c59b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:00:13.874 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:13.871+0000 7f5675fcd700 1 -- 172.21.15.59:0/791684726 >> 172.21.15.59:0/791684726 conn(0x7f56700b2740 msgr2=0x7f567006ea40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:00:13.874 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:13.871+0000 7f5675fcd700 1 -- 172.21.15.59:0/791684726 shutdown_connections 2023-11-24T08:00:13.874 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:13.871+0000 7f5675fcd700 1 -- 172.21.15.59:0/791684726 wait complete. 2023-11-24T08:00:13.885 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-24T08:00:14.012 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:14.010+0000 7f5c27e72700 1 Processor -- start 2023-11-24T08:00:14.012 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:14.010+0000 7f5c27e72700 1 -- start start 2023-11-24T08:00:14.015 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:14.011+0000 7f5c27e72700 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f5c20143ae0 0x7f5c200b51b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:00:14.015 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:14.011+0000 7f5c27e72700 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f5c200722c0 0x7f5c2006e510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:00:14.015 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:14.011+0000 7f5c27e72700 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5c20143ec0 con 0x7f5c201440f0 2023-11-24T08:00:14.015 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:14.011+0000 7f5c27e72700 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5c200b40a0 con 0x7f5c20143ae0 2023-11-24T08:00:14.015 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:14.011+0000 7f5c27e72700 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5c200b3ae0 con 0x7f5c200722c0 2023-11-24T08:00:14.015 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:14.011+0000 7f5c2540d700 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7f5c201440f0 0x7f5c20071bb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:59616/0 (socket says 172.21.15.59:59616) 2023-11-24T08:00:14.015 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:14.011+0000 7f5c2540d700 1 -- 172.21.15.59:0/3768348978 learned_addr learned my addr 172.21.15.59:0/3768348978 (peer_addr_for_me v1:172.21.15.59:0/0) 2023-11-24T08:00:14.015 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:14.011+0000 7f5c25c0e700 1 --2- 172.21.15.59:0/3768348978 >> v2:172.21.15.76:3300/0 conn(0x7f5c20143ae0 0x7f5c200b51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:00:14.015 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:14.012+0000 7f5c2640f700 1 --2- 172.21.15.59:0/3768348978 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f5c200722c0 0x7f5c2006e510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:00:14.016 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:14.012+0000 7f5c24c0c700 1 -- 172.21.15.59:0/3768348978 <== mon.0 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3409828310 0 0) 0x7f5c20143ec0 con 0x7f5c201440f0 2023-11-24T08:00:14.016 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:14.012+0000 7f5c24c0c700 1 -- 172.21.15.59:0/3768348978 --> v1:172.21.15.59:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5c0c0035b0 con 0x7f5c201440f0 2023-11-24T08:00:14.016 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:14.012+0000 7f5c25c0e700 1 -- 172.21.15.59:0/3768348978 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f5c200722c0 msgr2=0x7f5c2006e510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:00:14.016 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:14.012+0000 7f5c25c0e700 1 --2- 172.21.15.59:0/3768348978 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f5c200722c0 0x7f5c2006e510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:00:14.016 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:14.012+0000 7f5c25c0e700 1 -- 172.21.15.59:0/3768348978 >> v1:172.21.15.59:6789/0 conn(0x7f5c201440f0 legacy=0x7f5c20071bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:00:14.016 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:14.012+0000 7f5c25c0e700 1 -- 172.21.15.59:0/3768348978 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5c200b3dc0 con 0x7f5c20143ae0 2023-11-24T08:00:14.016 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:14.012+0000 7f5c25c0e700 1 --2- 172.21.15.59:0/3768348978 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5c20143ae0 0x7f5c200b51b0 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f5c10009840 tx=0x7f5c10009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fb9c1c571da4cc05 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:00:14.016 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:14.012+0000 7f5c24c0c700 1 -- 172.21.15.59:0/3768348978 <== mon.1 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c10011830 con 0x7f5c20143ae0 2023-11-24T08:00:14.016 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:14.012+0000 7f5c24c0c700 1 -- 172.21.15.59:0/3768348978 <== mon.1 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5c10011990 con 0x7f5c20143ae0 2023-11-24T08:00:14.016 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:14.012+0000 7f5c24c0c700 1 -- 172.21.15.59:0/3768348978 <== mon.1 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c10011d00 con 0x7f5c20143ae0 2023-11-24T08:00:14.016 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:14.013+0000 7f5c27e72700 1 -- 172.21.15.59:0/3768348978 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5c20143ae0 msgr2=0x7f5c200b51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:00:14.016 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:14.013+0000 7f5c27e72700 1 --2- 172.21.15.59:0/3768348978 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5c20143ae0 0x7f5c200b51b0 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f5c10009840 tx=0x7f5c10009b50 comp rx=0 tx=0).stop 2023-11-24T08:00:14.017 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:14.013+0000 7f5c27e72700 1 -- 172.21.15.59:0/3768348978 shutdown_connections 2023-11-24T08:00:14.017 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:14.013+0000 7f5c27e72700 1 --2- 172.21.15.59:0/3768348978 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f5c200722c0 0x7f5c2006e510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:00:14.017 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:14.013+0000 7f5c27e72700 1 --2- 172.21.15.59:0/3768348978 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5c20143ae0 0x7f5c200b51b0 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:00:14.017 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:14.013+0000 7f5c27e72700 1 -- 172.21.15.59:0/3768348978 >> 172.21.15.59:0/3768348978 conn(0x7f5c200b2740 msgr2=0x7f5c200b2b20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:00:14.017 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:14.013+0000 7f5c27e72700 1 -- 172.21.15.59:0/3768348978 shutdown_connections 2023-11-24T08:00:14.017 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:14.013+0000 7f5c27e72700 1 -- 172.21.15.59:0/3768348978 wait complete. 2023-11-24T08:00:14.020 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:14.014+0000 7f5c27e72700 1 Processor -- start 2023-11-24T08:00:14.020 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:14.014+0000 7f5c27e72700 1 -- start start 2023-11-24T08:00:14.020 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:14.015+0000 7f5c27e72700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5c200722c0 0x7f5c200cdf90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:00:14.020 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:14.015+0000 7f5c27e72700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5c20143ae0 0x7f5c200cfa50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:00:14.020 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:14.015+0000 7f5c27e72700 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f5c201440f0 0x7f5c20140940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:00:14.020 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:14.015+0000 7f5c27e72700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c200cff90 con 0x7f5c200722c0 2023-11-24T08:00:14.020 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:14.015+0000 7f5c27e72700 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c200ce5e0 con 0x7f5c20143ae0 2023-11-24T08:00:14.020 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:14.015+0000 7f5c27e72700 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5c200ce8c0 con 0x7f5c201440f0 2023-11-24T08:00:14.020 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:14.015+0000 7f5c25c0e700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5c200722c0 0x7f5c200cdf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:00:14.020 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:14.015+0000 7f5c25c0e700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5c200722c0 0x7f5c200cdf90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:56148/0 (socket says 172.21.15.59:56148) 2023-11-24T08:00:14.020 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:14.015+0000 7f5c25c0e700 1 -- 172.21.15.59:0/1235509679 learned_addr learned my addr 172.21.15.59:0/1235509679 (peer_addr_for_me v2:172.21.15.59:0/0) 2023-11-24T08:00:14.021 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:14.015+0000 7f5c2640f700 1 --2- 172.21.15.59:0/1235509679 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f5c201440f0 0x7f5c20140940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:00:14.021 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:14.015+0000 7f5c2540d700 1 --2- 172.21.15.59:0/1235509679 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5c20143ae0 0x7f5c200cfa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:00:14.021 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:14.016+0000 7f5c25c0e700 1 -- 172.21.15.59:0/1235509679 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f5c201440f0 msgr2=0x7f5c20140940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:00:14.021 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:14.016+0000 7f5c25c0e700 1 --2- 172.21.15.59:0/1235509679 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f5c201440f0 0x7f5c20140940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:00:14.021 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:14.016+0000 7f5c25c0e700 1 -- 172.21.15.59:0/1235509679 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5c20143ae0 msgr2=0x7f5c200cfa50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:00:14.021 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:14.016+0000 7f5c25c0e700 1 --2- 172.21.15.59:0/1235509679 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5c20143ae0 0x7f5c200cfa50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:00:14.021 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:14.016+0000 7f5c25c0e700 1 -- 172.21.15.59:0/1235509679 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5c100094a0 con 0x7f5c200722c0 2023-11-24T08:00:14.021 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:14.016+0000 7f5c25c0e700 1 --2- 172.21.15.59:0/1235509679 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5c200722c0 0x7f5c200cdf90 secure :-1 s=READY pgs=1070 cs=0 l=1 rev1=1 crypto rx=0x7f5c10009b90 tx=0x7f5c10004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:00:14.021 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:14.016+0000 7f5c167fc700 1 -- 172.21.15.59:0/1235509679 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c10004550 con 0x7f5c200722c0 2023-11-24T08:00:14.021 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:14.016+0000 7f5c167fc700 1 -- 172.21.15.59:0/1235509679 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5c1001f070 con 0x7f5c200722c0 2023-11-24T08:00:14.021 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:14.016+0000 7f5c27e72700 1 -- 172.21.15.59:0/1235509679 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5c200ceba0 con 0x7f5c200722c0 2023-11-24T08:00:14.021 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:14.016+0000 7f5c167fc700 1 -- 172.21.15.59:0/1235509679 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c100038e0 con 0x7f5c200722c0 2023-11-24T08:00:14.022 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:14.016+0000 7f5c27e72700 1 -- 172.21.15.59:0/1235509679 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5c200cf160 con 0x7f5c200722c0 2023-11-24T08:00:14.022 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:14.017+0000 7f5c27e72700 1 -- 172.21.15.59:0/1235509679 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5c20075480 con 0x7f5c200722c0 2023-11-24T08:00:14.022 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:14.018+0000 7f5c167fc700 1 -- 172.21.15.59:0/1235509679 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 11) v1 ==== 89741+0+0 (secure 0 0 0) 0x7f5c10003a40 con 0x7f5c200722c0 2023-11-24T08:00:14.022 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:14.018+0000 7f5c167fc700 1 --2- 172.21.15.59:0/1235509679 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f5c0406b470 0x7f5c0406d930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:00:14.022 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:14.018+0000 7f5c167fc700 1 -- 172.21.15.59:0/1235509679 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f5c10021070 con 0x7f5c200722c0 2023-11-24T08:00:14.022 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:14.018+0000 7f5c2540d700 1 --2- 172.21.15.59:0/1235509679 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f5c0406b470 0x7f5c0406d930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:00:14.022 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:14.019+0000 7f5c2540d700 1 --2- 172.21.15.59:0/1235509679 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f5c0406b470 0x7f5c0406d930 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7f5c1c007fd0 tx=0x7f5c1c009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:00:14.024 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:14.022+0000 7f5c167fc700 1 -- 172.21.15.59:0/1235509679 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f5c10055fc0 con 0x7f5c200722c0 2023-11-24T08:00:14.223 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:14.221+0000 7f5c27e72700 1 -- 172.21.15.59:0/1235509679 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f5c20144830 con 0x7f5c200722c0 2023-11-24T08:00:14.224 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:14.221+0000 7f5c167fc700 1 -- 172.21.15.59:0/1235509679 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v87) v1 ==== 104+0+11 (secure 0 0 0) 0x7f5c100595e0 con 0x7f5c200722c0 2023-11-24T08:00:14.224 INFO:teuthology.orchestra.run.smithi059.stdout:pg_num: 32 2023-11-24T08:00:14.226 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:14.223+0000 7f5c27e72700 1 -- 172.21.15.59:0/1235509679 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f5c0406b470 msgr2=0x7f5c0406d930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:00:14.226 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:14.223+0000 7f5c27e72700 1 --2- 172.21.15.59:0/1235509679 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f5c0406b470 0x7f5c0406d930 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7f5c1c007fd0 tx=0x7f5c1c009040 comp rx=0 tx=0).stop 2023-11-24T08:00:14.226 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:14.223+0000 7f5c27e72700 1 -- 172.21.15.59:0/1235509679 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5c200722c0 msgr2=0x7f5c200cdf90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:00:14.226 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:14.223+0000 7f5c27e72700 1 --2- 172.21.15.59:0/1235509679 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5c200722c0 0x7f5c200cdf90 secure :-1 s=READY pgs=1070 cs=0 l=1 rev1=1 crypto rx=0x7f5c10009b90 tx=0x7f5c10004750 comp rx=0 tx=0).stop 2023-11-24T08:00:14.226 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:14.223+0000 7f5c27e72700 1 -- 172.21.15.59:0/1235509679 shutdown_connections 2023-11-24T08:00:14.226 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:14.223+0000 7f5c27e72700 1 --2- 172.21.15.59:0/1235509679 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f5c0406b470 0x7f5c0406d930 unknown :-1 s=CLOSED pgs=491 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:00:14.227 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:14.223+0000 7f5c27e72700 1 --2- 172.21.15.59:0/1235509679 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f5c201440f0 0x7f5c20140940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:00:14.227 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:14.223+0000 7f5c27e72700 1 --2- 172.21.15.59:0/1235509679 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5c200722c0 0x7f5c200cdf90 unknown :-1 s=CLOSED pgs=1070 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:00:14.227 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:14.223+0000 7f5c27e72700 1 --2- 172.21.15.59:0/1235509679 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5c20143ae0 0x7f5c200cfa50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:00:14.227 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:14.223+0000 7f5c27e72700 1 -- 172.21.15.59:0/1235509679 >> 172.21.15.59:0/1235509679 conn(0x7f5c200b2740 msgr2=0x7f5c20141260 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:00:14.227 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:14.224+0000 7f5c27e72700 1 -- 172.21.15.59:0/1235509679 shutdown_connections 2023-11-24T08:00:14.227 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:14.224+0000 7f5c27e72700 1 -- 172.21.15.59:0/1235509679 wait complete. 2023-11-24T08:00:14.238 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-24T08:00:14.366 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:14.364+0000 7fb2deb31700 1 Processor -- start 2023-11-24T08:00:14.366 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:14.364+0000 7fb2deb31700 1 -- start start 2023-11-24T08:00:14.368 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:14.365+0000 7fb2deb31700 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fb2d80b03b0 0x7fb2d80b5e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:00:14.368 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:14.365+0000 7fb2deb31700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fb2d80adda0 0x7fb2d806b970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:00:14.369 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:14.365+0000 7fb2deb31700 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb2d80b0ec0 con 0x7fb2d815ae00 2023-11-24T08:00:14.369 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:14.365+0000 7fb2deb31700 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb2d8073610 con 0x7fb2d80adda0 2023-11-24T08:00:14.369 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:14.365+0000 7fb2deb31700 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb2d815b1e0 con 0x7fb2d80b03b0 2023-11-24T08:00:14.369 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:14.365+0000 7fb2dd0ce700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fb2d80adda0 0x7fb2d806b970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:00:14.369 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:14.365+0000 7fb2dd0ce700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fb2d80adda0 0x7fb2d806b970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:56160/0 (socket says 172.21.15.59:56160) 2023-11-24T08:00:14.369 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:14.365+0000 7fb2dd0ce700 1 -- 172.21.15.59:0/768954737 learned_addr learned my addr 172.21.15.59:0/768954737 (peer_addr_for_me v2:172.21.15.59:0/0) 2023-11-24T08:00:14.370 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:14.365+0000 7fb2d7fff700 1 --2- 172.21.15.59:0/768954737 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fb2d80b03b0 0x7fb2d80b5e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:00:14.370 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:14.365+0000 7fb2dd0ce700 1 -- 172.21.15.59:0/768954737 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fb2d80b03b0 msgr2=0x7fb2d80b5e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:00:14.370 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:14.365+0000 7fb2dd0ce700 1 --2- 172.21.15.59:0/768954737 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fb2d80b03b0 0x7fb2d80b5e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:00:14.370 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:14.365+0000 7fb2dd0ce700 1 -- 172.21.15.59:0/768954737 >> v1:172.21.15.76:6789/0 conn(0x7fb2d815ae00 legacy=0x7fb2d80ad690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:00:14.370 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:14.365+0000 7fb2dd0ce700 1 -- 172.21.15.59:0/768954737 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb2d80b0900 con 0x7fb2d80adda0 2023-11-24T08:00:14.370 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:14.366+0000 7fb2dd0ce700 1 --2- 172.21.15.59:0/768954737 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb2d80adda0 0x7fb2d806b970 secure :-1 s=READY pgs=1071 cs=0 l=1 rev1=1 crypto rx=0x7fb2d000d180 tx=0x7fb2d000d490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cc4bc739d4dda6d0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:00:14.371 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:14.366+0000 7fb2d77fe700 1 -- 172.21.15.59:0/768954737 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2d0010070 con 0x7fb2d80adda0 2023-11-24T08:00:14.371 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:14.366+0000 7fb2d77fe700 1 -- 172.21.15.59:0/768954737 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb2d000dd70 con 0x7fb2d80adda0 2023-11-24T08:00:14.371 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:14.366+0000 7fb2d77fe700 1 -- 172.21.15.59:0/768954737 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2d000b310 con 0x7fb2d80adda0 2023-11-24T08:00:14.371 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:14.366+0000 7fb2deb31700 1 -- 172.21.15.59:0/768954737 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb2d80adda0 msgr2=0x7fb2d806b970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:00:14.371 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:14.366+0000 7fb2deb31700 1 --2- 172.21.15.59:0/768954737 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb2d80adda0 0x7fb2d806b970 secure :-1 s=READY pgs=1071 cs=0 l=1 rev1=1 crypto rx=0x7fb2d000d180 tx=0x7fb2d000d490 comp rx=0 tx=0).stop 2023-11-24T08:00:14.372 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:14.366+0000 7fb2deb31700 1 -- 172.21.15.59:0/768954737 shutdown_connections 2023-11-24T08:00:14.372 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:14.366+0000 7fb2deb31700 1 --2- 172.21.15.59:0/768954737 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb2d80adda0 0x7fb2d806b970 unknown :-1 s=CLOSED pgs=1071 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:00:14.372 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:14.366+0000 7fb2deb31700 1 --2- 172.21.15.59:0/768954737 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fb2d80b03b0 0x7fb2d80b5e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:00:14.372 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:14.366+0000 7fb2deb31700 1 -- 172.21.15.59:0/768954737 >> 172.21.15.59:0/768954737 conn(0x7fb2d80b5070 msgr2=0x7fb2d80b5730 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:00:14.372 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:14.366+0000 7fb2deb31700 1 -- 172.21.15.59:0/768954737 shutdown_connections 2023-11-24T08:00:14.372 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:14.366+0000 7fb2deb31700 1 -- 172.21.15.59:0/768954737 wait complete. 2023-11-24T08:00:14.376 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:14.368+0000 7fb2deb31700 1 Processor -- start 2023-11-24T08:00:14.376 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:14.368+0000 7fb2deb31700 1 -- start start 2023-11-24T08:00:14.376 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:14.369+0000 7fb2deb31700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb2d80adda0 0x7fb2d81f28d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:00:14.377 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:14.369+0000 7fb2deb31700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb2d80b03b0 0x7fb2d81f2e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:00:14.377 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:14.369+0000 7fb2deb31700 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fb2d815ae00 0x7fb2d81583f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:00:14.377 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:14.369+0000 7fb2deb31700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2d8159be0 con 0x7fb2d80b03b0 2023-11-24T08:00:14.377 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:14.369+0000 7fb2deb31700 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2d81f3720 con 0x7fb2d80adda0 2023-11-24T08:00:14.378 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:14.369+0000 7fb2deb31700 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb2d8158fd0 con 0x7fb2d815ae00 2023-11-24T08:00:14.378 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:14.369+0000 7fb2d7fff700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb2d80b03b0 0x7fb2d81f2e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:00:14.378 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:14.369+0000 7fb2d7fff700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb2d80b03b0 0x7fb2d81f2e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:56164/0 (socket says 172.21.15.59:56164) 2023-11-24T08:00:14.378 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:14.369+0000 7fb2d7fff700 1 -- 172.21.15.59:0/2728700324 learned_addr learned my addr 172.21.15.59:0/2728700324 (peer_addr_for_me v2:172.21.15.59:0/0) 2023-11-24T08:00:14.378 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:14.369+0000 7fb2dd0ce700 1 --2- 172.21.15.59:0/2728700324 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fb2d815ae00 0x7fb2d81583f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:00:14.379 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:14.369+0000 7fb2dc8cd700 1 --2- 172.21.15.59:0/2728700324 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb2d80adda0 0x7fb2d81f28d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:00:14.379 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:14.369+0000 7fb2d7fff700 1 -- 172.21.15.59:0/2728700324 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fb2d815ae00 msgr2=0x7fb2d81583f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:00:14.379 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:14.369+0000 7fb2d7fff700 1 --2- 172.21.15.59:0/2728700324 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fb2d815ae00 0x7fb2d81583f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:00:14.379 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:14.370+0000 7fb2d7fff700 1 -- 172.21.15.59:0/2728700324 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb2d80adda0 msgr2=0x7fb2d81f28d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:00:14.380 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:14.370+0000 7fb2d7fff700 1 --2- 172.21.15.59:0/2728700324 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb2d80adda0 0x7fb2d81f28d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:00:14.380 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:14.370+0000 7fb2d7fff700 1 -- 172.21.15.59:0/2728700324 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb2d000bd40 con 0x7fb2d80b03b0 2023-11-24T08:00:14.380 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:14.370+0000 7fb2dc8cd700 1 --2- 172.21.15.59:0/2728700324 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb2d80adda0 0x7fb2d81f28d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-24T08:00:14.380 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:14.370+0000 7fb2d7fff700 1 --2- 172.21.15.59:0/2728700324 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb2d80b03b0 0x7fb2d81f2e10 secure :-1 s=READY pgs=1072 cs=0 l=1 rev1=1 crypto rx=0x7fb2c400e7e0 tx=0x7fb2c400eaf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:00:14.380 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:14.370+0000 7fb2d57fa700 1 -- 172.21.15.59:0/2728700324 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2c4013610 con 0x7fb2d80b03b0 2023-11-24T08:00:14.380 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:14.370+0000 7fb2d57fa700 1 -- 172.21.15.59:0/2728700324 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb2c4013770 con 0x7fb2d80b03b0 2023-11-24T08:00:14.380 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:14.370+0000 7fb2deb31700 1 -- 172.21.15.59:0/2728700324 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb2d81592b0 con 0x7fb2d80b03b0 2023-11-24T08:00:14.381 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:14.370+0000 7fb2d57fa700 1 -- 172.21.15.59:0/2728700324 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2c4013ae0 con 0x7fb2d80b03b0 2023-11-24T08:00:14.381 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:14.370+0000 7fb2deb31700 1 -- 172.21.15.59:0/2728700324 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb2d8159870 con 0x7fb2d80b03b0 2023-11-24T08:00:14.381 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:14.372+0000 7fb2deb31700 1 -- 172.21.15.59:0/2728700324 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb2d8159ec0 con 0x7fb2d80b03b0 2023-11-24T08:00:14.381 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:14.372+0000 7fb2d57fa700 1 -- 172.21.15.59:0/2728700324 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 11) v1 ==== 89741+0+0 (secure 0 0 0) 0x7fb2c4013c40 con 0x7fb2d80b03b0 2023-11-24T08:00:14.381 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:14.372+0000 7fb2d57fa700 1 --2- 172.21.15.59:0/2728700324 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7fb2bc06b470 0x7fb2bc06d930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:00:14.381 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:14.372+0000 7fb2dc8cd700 1 --2- 172.21.15.59:0/2728700324 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7fb2bc06b470 0x7fb2bc06d930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:00:14.381 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:14.373+0000 7fb2dc8cd700 1 --2- 172.21.15.59:0/2728700324 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7fb2bc06b470 0x7fb2bc06d930 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7fb2cc009930 tx=0x7fb2cc008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:00:14.381 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:14.373+0000 7fb2d57fa700 1 -- 172.21.15.59:0/2728700324 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fb2c40588a0 con 0x7fb2d80b03b0 2023-11-24T08:00:14.382 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:14.378+0000 7fb2d57fa700 1 -- 172.21.15.59:0/2728700324 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fb2c40544e0 con 0x7fb2d80b03b0 2023-11-24T08:00:14.384 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:14.383+0000 7fb2d57fa700 1 -- 172.21.15.59:0/2728700324 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mgrmap(e 12) v1 ==== 89792+0+0 (secure 0 0 0) 0x7fb2c40548b0 con 0x7fb2d80b03b0 2023-11-24T08:00:14.581 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:14.580+0000 7fb2deb31700 1 -- 172.21.15.59:0/2728700324 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fb2d8075480 con 0x7fb2d80b03b0 2023-11-24T08:00:14.582 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:14.580+0000 7fb2d57fa700 1 -- 172.21.15.59:0/2728700324 <== mon.0 v2:172.21.15.59:3300/0 8 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v87) v1 ==== 100+0+11 (secure 0 0 0) 0x7fb2c4085770 con 0x7fb2d80b03b0 2023-11-24T08:00:14.582 INFO:teuthology.orchestra.run.smithi059.stdout:pg_num: 64 2023-11-24T08:00:14.584 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:14.582+0000 7fb2deb31700 1 -- 172.21.15.59:0/2728700324 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7fb2bc06b470 msgr2=0x7fb2bc06d930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:00:14.584 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:14.582+0000 7fb2deb31700 1 --2- 172.21.15.59:0/2728700324 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7fb2bc06b470 0x7fb2bc06d930 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7fb2cc009930 tx=0x7fb2cc008040 comp rx=0 tx=0).stop 2023-11-24T08:00:14.584 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:14.582+0000 7fb2deb31700 1 -- 172.21.15.59:0/2728700324 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb2d80b03b0 msgr2=0x7fb2d81f2e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:00:14.584 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:14.582+0000 7fb2deb31700 1 --2- 172.21.15.59:0/2728700324 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb2d80b03b0 0x7fb2d81f2e10 secure :-1 s=READY pgs=1072 cs=0 l=1 rev1=1 crypto rx=0x7fb2c400e7e0 tx=0x7fb2c400eaf0 comp rx=0 tx=0).stop 2023-11-24T08:00:14.584 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:14.582+0000 7fb2deb31700 1 -- 172.21.15.59:0/2728700324 shutdown_connections 2023-11-24T08:00:14.584 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:14.582+0000 7fb2deb31700 1 --2- 172.21.15.59:0/2728700324 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7fb2bc06b470 0x7fb2bc06d930 unknown :-1 s=CLOSED pgs=492 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:00:14.584 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:14.582+0000 7fb2deb31700 1 --2- 172.21.15.59:0/2728700324 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fb2d815ae00 0x7fb2d81583f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:00:14.584 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:14.582+0000 7fb2deb31700 1 --2- 172.21.15.59:0/2728700324 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb2d80adda0 0x7fb2d81f28d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:00:14.584 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:14.582+0000 7fb2deb31700 1 --2- 172.21.15.59:0/2728700324 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb2d80b03b0 0x7fb2d81f2e10 unknown :-1 s=CLOSED pgs=1072 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:00:14.585 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:14.582+0000 7fb2deb31700 1 -- 172.21.15.59:0/2728700324 >> 172.21.15.59:0/2728700324 conn(0x7fb2d80b5070 msgr2=0x7fb2d80ad650 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:00:14.585 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:14.582+0000 7fb2deb31700 1 -- 172.21.15.59:0/2728700324 shutdown_connections 2023-11-24T08:00:14.585 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:14.582+0000 7fb2deb31700 1 -- 172.21.15.59:0/2728700324 wait complete. 2023-11-24T08:00:14.596 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-11-24T08:00:14.722 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:14.720+0000 7f1c9f0ad700 1 Processor -- start 2023-11-24T08:00:14.722 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:14.720+0000 7f1c9f0ad700 1 -- start start 2023-11-24T08:00:14.725 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:14.721+0000 7f1c9f0ad700 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f1c981478c0 0x7f1c980b57b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:00:14.725 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:14.721+0000 7f1c9f0ad700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f1c9806b6e0 0x7f1c9806e370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:00:14.725 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:14.721+0000 7f1c9f0ad700 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1c9806dbe0 con 0x7f1c98161a90 2023-11-24T08:00:14.725 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:14.721+0000 7f1c9f0ad700 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1c98147ca0 con 0x7f1c9806b6e0 2023-11-24T08:00:14.725 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:14.721+0000 7f1c9f0ad700 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1c981622e0 con 0x7f1c981478c0 2023-11-24T08:00:14.725 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:14.721+0000 7f1c9d64a700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f1c9806b6e0 0x7f1c9806e370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:00:14.725 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:14.721+0000 7f1c9d64a700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f1c9806b6e0 0x7f1c9806e370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:56174/0 (socket says 172.21.15.59:56174) 2023-11-24T08:00:14.725 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:14.721+0000 7f1c9d64a700 1 -- 172.21.15.59:0/1686743609 learned_addr learned my addr 172.21.15.59:0/1686743609 (peer_addr_for_me v2:172.21.15.59:0/0) 2023-11-24T08:00:14.726 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:14.722+0000 7f1c9ce49700 1 --2- 172.21.15.59:0/1686743609 >> v2:172.21.15.76:3300/0 conn(0x7f1c981478c0 0x7f1c980b57b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:00:14.726 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:14.722+0000 7f1c9d64a700 1 -- 172.21.15.59:0/1686743609 >> v2:172.21.15.76:3300/0 conn(0x7f1c981478c0 msgr2=0x7f1c980b57b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:00:14.726 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:14.722+0000 7f1c9d64a700 1 --2- 172.21.15.59:0/1686743609 >> v2:172.21.15.76:3300/0 conn(0x7f1c981478c0 0x7f1c980b57b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:00:14.726 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:14.722+0000 7f1c9d64a700 1 -- 172.21.15.59:0/1686743609 >> v1:172.21.15.76:6789/0 conn(0x7f1c98161a90 legacy=0x7f1c9806afd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:00:14.726 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:14.722+0000 7f1c9d64a700 1 -- 172.21.15.59:0/1686743609 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1c981625c0 con 0x7f1c9806b6e0 2023-11-24T08:00:14.726 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:14.722+0000 7f1c9d64a700 1 --2- 172.21.15.59:0/1686743609 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1c9806b6e0 0x7f1c9806e370 secure :-1 s=READY pgs=1073 cs=0 l=1 rev1=1 crypto rx=0x7f1c98072fa0 tx=0x7f1c9000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b1a19e7d8cd4cc2c server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:00:14.726 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:14.722+0000 7f1c977fe700 1 -- 172.21.15.59:0/1686743609 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c9000e070 con 0x7f1c9806b6e0 2023-11-24T08:00:14.726 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:14.722+0000 7f1c977fe700 1 -- 172.21.15.59:0/1686743609 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1c9000b9c0 con 0x7f1c9806b6e0 2023-11-24T08:00:14.726 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:14.722+0000 7f1c977fe700 1 -- 172.21.15.59:0/1686743609 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c9000bd70 con 0x7f1c9806b6e0 2023-11-24T08:00:14.726 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:14.723+0000 7f1c9f0ad700 1 -- 172.21.15.59:0/1686743609 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1c9806b6e0 msgr2=0x7f1c9806e370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:00:14.726 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:14.723+0000 7f1c9f0ad700 1 --2- 172.21.15.59:0/1686743609 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1c9806b6e0 0x7f1c9806e370 secure :-1 s=READY pgs=1073 cs=0 l=1 rev1=1 crypto rx=0x7f1c98072fa0 tx=0x7f1c9000b330 comp rx=0 tx=0).stop 2023-11-24T08:00:14.727 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:14.723+0000 7f1c9f0ad700 1 -- 172.21.15.59:0/1686743609 shutdown_connections 2023-11-24T08:00:14.727 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:14.723+0000 7f1c9f0ad700 1 --2- 172.21.15.59:0/1686743609 >> v2:172.21.15.76:3300/0 conn(0x7f1c981478c0 0x7f1c980b57b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:00:14.727 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:14.723+0000 7f1c9f0ad700 1 --2- 172.21.15.59:0/1686743609 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1c9806b6e0 0x7f1c9806e370 unknown :-1 s=CLOSED pgs=1073 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:00:14.727 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:14.723+0000 7f1c9f0ad700 1 -- 172.21.15.59:0/1686743609 >> 172.21.15.59:0/1686743609 conn(0x7f1c980d0aa0 msgr2=0x7f1c980d0eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:00:14.727 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:14.723+0000 7f1c9f0ad700 1 -- 172.21.15.59:0/1686743609 shutdown_connections 2023-11-24T08:00:14.727 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:14.723+0000 7f1c9f0ad700 1 -- 172.21.15.59:0/1686743609 wait complete. 2023-11-24T08:00:14.730 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:14.724+0000 7f1c9f0ad700 1 Processor -- start 2023-11-24T08:00:14.730 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:14.724+0000 7f1c9f0ad700 1 -- start start 2023-11-24T08:00:14.730 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:14.725+0000 7f1c9f0ad700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1c9806b6e0 0x7f1c980b1d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:00:14.730 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:14.725+0000 7f1c9f0ad700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1c981478c0 0x7f1c98140030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:00:14.731 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:14.725+0000 7f1c9f0ad700 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f1c98161a90 0x7f1c98140570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:00:14.731 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:14.725+0000 7f1c9f0ad700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c980b13e0 con 0x7f1c981478c0 2023-11-24T08:00:14.731 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:14.725+0000 7f1c9f0ad700 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c98140c70 con 0x7f1c9806b6e0 2023-11-24T08:00:14.731 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:14.725+0000 7f1c9f0ad700 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1c98140f50 con 0x7f1c98161a90 2023-11-24T08:00:14.731 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:14.726+0000 7f1c97fff700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1c981478c0 0x7f1c98140030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:00:14.731 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:14.726+0000 7f1c97fff700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1c981478c0 0x7f1c98140030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:56178/0 (socket says 172.21.15.59:56178) 2023-11-24T08:00:14.731 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:14.726+0000 7f1c97fff700 1 -- 172.21.15.59:0/1299713411 learned_addr learned my addr 172.21.15.59:0/1299713411 (peer_addr_for_me v2:172.21.15.59:0/0) 2023-11-24T08:00:14.732 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:14.726+0000 7f1c9d64a700 1 --2- 172.21.15.59:0/1299713411 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f1c98161a90 0x7f1c98140570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:00:14.732 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:14.726+0000 7f1c9ce49700 1 --2- 172.21.15.59:0/1299713411 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1c9806b6e0 0x7f1c980b1d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:00:14.732 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:14.726+0000 7f1c97fff700 1 -- 172.21.15.59:0/1299713411 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f1c98161a90 msgr2=0x7f1c98140570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:00:14.732 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:14.726+0000 7f1c97fff700 1 --2- 172.21.15.59:0/1299713411 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f1c98161a90 0x7f1c98140570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:00:14.732 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:14.726+0000 7f1c97fff700 1 -- 172.21.15.59:0/1299713411 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1c9806b6e0 msgr2=0x7f1c980b1d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:00:14.732 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:14.726+0000 7f1c97fff700 1 --2- 172.21.15.59:0/1299713411 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1c9806b6e0 0x7f1c980b1d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:00:14.733 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:14.726+0000 7f1c97fff700 1 -- 172.21.15.59:0/1299713411 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1c90009b30 con 0x7f1c981478c0 2023-11-24T08:00:14.733 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:14.726+0000 7f1c9ce49700 1 --2- 172.21.15.59:0/1299713411 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1c9806b6e0 0x7f1c980b1d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-24T08:00:14.733 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:14.726+0000 7f1c97fff700 1 --2- 172.21.15.59:0/1299713411 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1c981478c0 0x7f1c98140030 secure :-1 s=READY pgs=1074 cs=0 l=1 rev1=1 crypto rx=0x7f1c8400c450 tx=0x7f1c8400c810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:00:14.733 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:14.726+0000 7f1c957fa700 1 -- 172.21.15.59:0/1299713411 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c840052d0 con 0x7f1c981478c0 2023-11-24T08:00:14.733 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:14.726+0000 7f1c957fa700 1 -- 172.21.15.59:0/1299713411 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1c84005430 con 0x7f1c981478c0 2023-11-24T08:00:14.733 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:14.726+0000 7f1c9f0ad700 1 -- 172.21.15.59:0/1299713411 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1c98141230 con 0x7f1c981478c0 2023-11-24T08:00:14.733 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:14.726+0000 7f1c957fa700 1 -- 172.21.15.59:0/1299713411 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c840038c0 con 0x7f1c981478c0 2023-11-24T08:00:14.734 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:14.726+0000 7f1c9f0ad700 1 -- 172.21.15.59:0/1299713411 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1c981417f0 con 0x7f1c981478c0 2023-11-24T08:00:14.734 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:14.728+0000 7f1c9f0ad700 1 -- 172.21.15.59:0/1299713411 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1c981419b0 con 0x7f1c981478c0 2023-11-24T08:00:14.734 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:14.728+0000 7f1c957fa700 1 -- 172.21.15.59:0/1299713411 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 12) v1 ==== 89792+0+0 (secure 0 0 0) 0x7f1c84003a20 con 0x7f1c981478c0 2023-11-24T08:00:14.734 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:14.728+0000 7f1c957fa700 1 --2- 172.21.15.59:0/1299713411 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f1c7c06b500 0x7f1c7c06d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:00:14.734 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:14.728+0000 7f1c957fa700 1 -- 172.21.15.59:0/1299713411 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f1c840893d0 con 0x7f1c981478c0 2023-11-24T08:00:14.734 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:14.728+0000 7f1c9ce49700 1 --2- 172.21.15.59:0/1299713411 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f1c7c06b500 0x7f1c7c06d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:00:14.735 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:14.729+0000 7f1c9ce49700 1 --2- 172.21.15.59:0/1299713411 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f1c7c06b500 0x7f1c7c06d9c0 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7f1c8c0099f0 tx=0x7f1c8c008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:00:14.735 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:14.733+0000 7f1c957fa700 1 -- 172.21.15.59:0/1299713411 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f1c84053bd0 con 0x7f1c981478c0 2023-11-24T08:00:14.950 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:14.948+0000 7f1c9f0ad700 1 -- 172.21.15.59:0/1299713411 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f1c980b5090 con 0x7f1c981478c0 2023-11-24T08:00:14.951 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:14.949+0000 7f1c957fa700 1 -- 172.21.15.59:0/1299713411 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v87) v1 ==== 80+0+0 (secure 0 0 0) 0x7f1c840571f0 con 0x7f1c981478c0 2023-11-24T08:00:14.953 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:14.951+0000 7f1c9f0ad700 1 -- 172.21.15.59:0/1299713411 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f1c7c06b500 msgr2=0x7f1c7c06d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:00:14.953 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:14.951+0000 7f1c9f0ad700 1 --2- 172.21.15.59:0/1299713411 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f1c7c06b500 0x7f1c7c06d9c0 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7f1c8c0099f0 tx=0x7f1c8c008040 comp rx=0 tx=0).stop 2023-11-24T08:00:14.953 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:14.951+0000 7f1c9f0ad700 1 -- 172.21.15.59:0/1299713411 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1c981478c0 msgr2=0x7f1c98140030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:00:14.953 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:14.951+0000 7f1c9f0ad700 1 --2- 172.21.15.59:0/1299713411 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1c981478c0 0x7f1c98140030 secure :-1 s=READY pgs=1074 cs=0 l=1 rev1=1 crypto rx=0x7f1c8400c450 tx=0x7f1c8400c810 comp rx=0 tx=0).stop 2023-11-24T08:00:14.954 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:14.951+0000 7f1c9f0ad700 1 -- 172.21.15.59:0/1299713411 shutdown_connections 2023-11-24T08:00:14.954 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:14.951+0000 7f1c9f0ad700 1 --2- 172.21.15.59:0/1299713411 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f1c7c06b500 0x7f1c7c06d9c0 unknown :-1 s=CLOSED pgs=493 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:00:14.954 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:14.951+0000 7f1c9f0ad700 1 --2- 172.21.15.59:0/1299713411 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f1c98161a90 0x7f1c98140570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:00:14.954 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:14.951+0000 7f1c9f0ad700 1 --2- 172.21.15.59:0/1299713411 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1c9806b6e0 0x7f1c980b1d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:00:14.954 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:14.951+0000 7f1c9f0ad700 1 --2- 172.21.15.59:0/1299713411 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1c981478c0 0x7f1c98140030 unknown :-1 s=CLOSED pgs=1074 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:00:14.954 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:14.951+0000 7f1c9f0ad700 1 -- 172.21.15.59:0/1299713411 >> 172.21.15.59:0/1299713411 conn(0x7f1c980d0aa0 msgr2=0x7f1c9806afd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:00:14.954 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:14.951+0000 7f1c9f0ad700 1 -- 172.21.15.59:0/1299713411 shutdown_connections 2023-11-24T08:00:14.954 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:14.951+0000 7f1c9f0ad700 1 -- 172.21.15.59:0/1299713411 wait complete. 2023-11-24T08:00:14.955 INFO:teuthology.orchestra.run.smithi059.stderr:listed 0 entries 2023-11-24T08:00:14.965 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2023-11-24T08:00:14.965 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T08:00:14.966 DEBUG:teuthology.orchestra.run.smithi059:> sudo umount /home/ubuntu/cephtest/mnt.0 2023-11-24T08:00:15.005 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi059.front.sepia.ceph.com 2023-11-24T08:00:15.005 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T08:00:15.005 DEBUG:teuthology.orchestra.run.smithi059:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-11-24T08:00:15.063 DEBUG:teuthology.orchestra.run.smithi076:> set -ex 2023-11-24T08:00:15.064 DEBUG:teuthology.orchestra.run.smithi076:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-24T08:00:15.085 DEBUG:teuthology.orchestra.run.smithi076:> set -ex 2023-11-24T08:00:15.085 DEBUG:teuthology.orchestra.run.smithi076:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-24T08:00:15.146 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-24T08:00:15.271 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:15.269+0000 7f7cff0cd700 1 Processor -- start 2023-11-24T08:00:15.271 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:15.269+0000 7f7cff0cd700 1 -- start start 2023-11-24T08:00:15.273 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:15.270+0000 7f7cff0cd700 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f7cf81457a0 0x7f7cf80cff70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:00:15.273 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:15.270+0000 7f7cff0cd700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f7cf80cd470 0x7f7cf8160330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:00:15.273 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:15.270+0000 7f7cff0cd700 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7cf80d04b0 con 0x7f7cf806afd0 2023-11-24T08:00:15.273 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:15.270+0000 7f7cff0cd700 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7cf8160870 con 0x7f7cf80cd470 2023-11-24T08:00:15.274 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:15.270+0000 7f7cff0cd700 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7cf80cf1b0 con 0x7f7cf81457a0 2023-11-24T08:00:15.274 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:15.270+0000 7f7cf7fff700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f7cf80cd470 0x7f7cf8160330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:00:15.274 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:15.270+0000 7f7cfd66a700 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7f7cf806afd0 0x7f7cf806db40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:59662/0 (socket says 172.21.15.59:59662) 2023-11-24T08:00:15.274 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:15.270+0000 7f7cfd66a700 1 -- 172.21.15.59:0/1972400504 learned_addr learned my addr 172.21.15.59:0/1972400504 (peer_addr_for_me v1:172.21.15.59:0/0) 2023-11-24T08:00:15.274 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:15.270+0000 7f7cfce69700 1 --2- 172.21.15.59:0/1972400504 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f7cf81457a0 0x7f7cf80cff70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:00:15.274 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:15.271+0000 7f7cf7fff700 1 -- 172.21.15.59:0/1972400504 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f7cf81457a0 msgr2=0x7f7cf80cff70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:00:15.274 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:15.271+0000 7f7cf7fff700 1 --2- 172.21.15.59:0/1972400504 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f7cf81457a0 0x7f7cf80cff70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:00:15.274 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:15.271+0000 7f7cf7fff700 1 -- 172.21.15.59:0/1972400504 >> v1:172.21.15.59:6789/0 conn(0x7f7cf806afd0 legacy=0x7f7cf806db40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:00:15.274 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:15.271+0000 7f7cf7fff700 1 -- 172.21.15.59:0/1972400504 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7cf80cf490 con 0x7f7cf80cd470 2023-11-24T08:00:15.274 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:15.271+0000 7f7cf77fe700 1 -- 172.21.15.59:0/1972400504 <== mon.0 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 350759063 0 0) 0x7f7cf80d04b0 con 0x7f7cf806afd0 2023-11-24T08:00:15.274 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:15.271+0000 7f7cf7fff700 1 --2- 172.21.15.59:0/1972400504 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7cf80cd470 0x7f7cf8160330 secure :-1 s=READY pgs=1076 cs=0 l=1 rev1=1 crypto rx=0x7f7ce4009840 tx=0x7f7ce4009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=858162f24d0ac470 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:00:15.274 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:15.271+0000 7f7cf77fe700 1 -- 172.21.15.59:0/1972400504 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ce4011830 con 0x7f7cf80cd470 2023-11-24T08:00:15.275 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:15.271+0000 7f7cf77fe700 1 -- 172.21.15.59:0/1972400504 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7ce4011990 con 0x7f7cf80cd470 2023-11-24T08:00:15.275 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:15.271+0000 7f7cf77fe700 1 -- 172.21.15.59:0/1972400504 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ce4011d20 con 0x7f7cf80cd470 2023-11-24T08:00:15.275 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:15.271+0000 7f7cff0cd700 1 -- 172.21.15.59:0/1972400504 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7cf80cd470 msgr2=0x7f7cf8160330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:00:15.275 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:15.271+0000 7f7cff0cd700 1 --2- 172.21.15.59:0/1972400504 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7cf80cd470 0x7f7cf8160330 secure :-1 s=READY pgs=1076 cs=0 l=1 rev1=1 crypto rx=0x7f7ce4009840 tx=0x7f7ce4009b50 comp rx=0 tx=0).stop 2023-11-24T08:00:15.275 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:15.272+0000 7f7cff0cd700 1 -- 172.21.15.59:0/1972400504 shutdown_connections 2023-11-24T08:00:15.275 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:15.272+0000 7f7cff0cd700 1 --2- 172.21.15.59:0/1972400504 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f7cf81457a0 0x7f7cf80cff70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:00:15.275 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:15.272+0000 7f7cff0cd700 1 --2- 172.21.15.59:0/1972400504 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7cf80cd470 0x7f7cf8160330 unknown :-1 s=CLOSED pgs=1076 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:00:15.275 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:15.272+0000 7f7cff0cd700 1 -- 172.21.15.59:0/1972400504 >> 172.21.15.59:0/1972400504 conn(0x7f7cf80d0aa0 msgr2=0x7f7cf80d0eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:00:15.275 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:15.272+0000 7f7cff0cd700 1 -- 172.21.15.59:0/1972400504 shutdown_connections 2023-11-24T08:00:15.275 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:15.272+0000 7f7cff0cd700 1 -- 172.21.15.59:0/1972400504 wait complete. 2023-11-24T08:00:15.277 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:15.273+0000 7f7cff0cd700 1 Processor -- start 2023-11-24T08:00:15.278 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:15.273+0000 7f7cff0cd700 1 -- start start 2023-11-24T08:00:15.278 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:15.274+0000 7f7cff0cd700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7cf806afd0 0x7f7cf815a9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:00:15.278 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:15.274+0000 7f7cff0cd700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7cf80cd470 0x7f7cf815c870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:00:15.278 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:15.274+0000 7f7cff0cd700 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f7cf81457a0 0x7f7cf815cdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:00:15.278 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:15.274+0000 7f7cff0cd700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7cf81417a0 con 0x7f7cf806afd0 2023-11-24T08:00:15.278 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:15.274+0000 7f7cff0cd700 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7cf8158670 con 0x7f7cf80cd470 2023-11-24T08:00:15.278 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:15.274+0000 7f7cff0cd700 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7cf8158060 con 0x7f7cf81457a0 2023-11-24T08:00:15.278 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:15.274+0000 7f7cfce69700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7cf806afd0 0x7f7cf815a9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:00:15.278 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:15.274+0000 7f7cfce69700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7cf806afd0 0x7f7cf815a9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:56190/0 (socket says 172.21.15.59:56190) 2023-11-24T08:00:15.278 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:15.274+0000 7f7cfce69700 1 -- 172.21.15.59:0/1277627512 learned_addr learned my addr 172.21.15.59:0/1277627512 (peer_addr_for_me v2:172.21.15.59:0/0) 2023-11-24T08:00:15.278 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:15.274+0000 7f7cfd66a700 1 --2- 172.21.15.59:0/1277627512 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f7cf81457a0 0x7f7cf815cdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:00:15.278 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:15.274+0000 7f7cf7fff700 1 --2- 172.21.15.59:0/1277627512 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7cf80cd470 0x7f7cf815c870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:00:15.279 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:15.274+0000 7f7cfce69700 1 -- 172.21.15.59:0/1277627512 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f7cf81457a0 msgr2=0x7f7cf815cdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:00:15.279 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:15.274+0000 7f7cfce69700 1 --2- 172.21.15.59:0/1277627512 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f7cf81457a0 0x7f7cf815cdb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:00:15.279 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:15.274+0000 7f7cfce69700 1 -- 172.21.15.59:0/1277627512 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7cf80cd470 msgr2=0x7f7cf815c870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:00:15.279 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:15.274+0000 7f7cfce69700 1 --2- 172.21.15.59:0/1277627512 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7cf80cd470 0x7f7cf815c870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:00:15.279 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:15.274+0000 7f7cfce69700 1 -- 172.21.15.59:0/1277627512 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7ce40094a0 con 0x7f7cf806afd0 2023-11-24T08:00:15.279 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:15.274+0000 7f7cfce69700 1 --2- 172.21.15.59:0/1277627512 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7cf806afd0 0x7f7cf815a9c0 secure :-1 s=READY pgs=1077 cs=0 l=1 rev1=1 crypto rx=0x7f7cec00e810 tx=0x7f7cec00ebd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:00:15.279 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:15.275+0000 7f7cf57fa700 1 -- 172.21.15.59:0/1277627512 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7cec013610 con 0x7f7cf806afd0 2023-11-24T08:00:15.279 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:15.275+0000 7f7cf57fa700 1 -- 172.21.15.59:0/1277627512 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7cec013770 con 0x7f7cf806afd0 2023-11-24T08:00:15.279 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:15.275+0000 7f7cff0cd700 1 -- 172.21.15.59:0/1277627512 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7cf815f300 con 0x7f7cf806afd0 2023-11-24T08:00:15.279 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:15.275+0000 7f7cf57fa700 1 -- 172.21.15.59:0/1277627512 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7cec013b20 con 0x7f7cf806afd0 2023-11-24T08:00:15.279 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:15.275+0000 7f7cff0cd700 1 -- 172.21.15.59:0/1277627512 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7cf815f5e0 con 0x7f7cf806afd0 2023-11-24T08:00:15.279 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:15.276+0000 7f7cff0cd700 1 -- 172.21.15.59:0/1277627512 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7cf8068670 con 0x7f7cf806afd0 2023-11-24T08:00:15.279 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:15.276+0000 7f7cf57fa700 1 -- 172.21.15.59:0/1277627512 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 12) v1 ==== 89792+0+0 (secure 0 0 0) 0x7f7cec013c80 con 0x7f7cf806afd0 2023-11-24T08:00:15.280 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:15.277+0000 7f7cf57fa700 1 --2- 172.21.15.59:0/1277627512 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f7cdc06b500 0x7f7cdc06d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:00:15.280 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:15.277+0000 7f7cf57fa700 1 -- 172.21.15.59:0/1277627512 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f7cec089ef0 con 0x7f7cf806afd0 2023-11-24T08:00:15.280 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:15.277+0000 7f7cf7fff700 1 --2- 172.21.15.59:0/1277627512 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f7cdc06b500 0x7f7cdc06d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:00:15.280 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:15.277+0000 7f7cf7fff700 1 --2- 172.21.15.59:0/1277627512 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f7cdc06b500 0x7f7cdc06d9c0 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7f7ce40090b0 tx=0x7f7ce4008030 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:00:15.283 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:15.281+0000 7f7cf57fa700 1 -- 172.21.15.59:0/1277627512 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f7cec054560 con 0x7f7cf806afd0 2023-11-24T08:00:15.487 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:15.486+0000 7f7cff0cd700 1 -- 172.21.15.59:0/1277627512 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f7cf80d0880 con 0x7f7cf806afd0 2023-11-24T08:00:15.488 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:15.487+0000 7f7cf57fa700 1 -- 172.21.15.59:0/1277627512 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v87) v1 ==== 74+0+16500 (secure 0 0 0) 0x7f7cec057b80 con 0x7f7cf806afd0 2023-11-24T08:00:15.489 INFO:teuthology.orchestra.run.smithi059.stdout: 2023-11-24T08:00:15.489 INFO:teuthology.orchestra.run.smithi059.stdout:{"epoch":87,"fsid":"9f0e8741-b8f9-48d5-a2c3-09f45903303d","created":"2023-11-24T07:56:06.552729+0000","modified":"2023-11-24T07:59:53.500645+0000","last_up_change":"2023-11-24T07:56:17.372352+0000","last_in_change":"2023-11-24T07:56:09.073250+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":13,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-24T07:56:18.352823+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_metadata","create_time":"2023-11-24T07:59:50.274996+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_data","create_time":"2023-11-24T07:59:50.873919+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"87","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"6d8f9e9b-1782-4e41-928b-a39f2a8ebc06","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6800","nonce":3373878772},{"type":"v1","addr":"172.21.15.59:6801","nonce":3373878772}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6802","nonce":3373878772},{"type":"v1","addr":"172.21.15.59:6803","nonce":3373878772}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6806","nonce":3373878772},{"type":"v1","addr":"172.21.15.59:6807","nonce":3373878772}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6804","nonce":3373878772},{"type":"v1","addr":"172.21.15.59:6805","nonce":3373878772}]},"public_addr":"172.21.15.59:6801/3373878772","cluster_addr":"172.21.15.59:6803/3373878772","heartbeat_back_addr":"172.21.15.59:6807/3373878772","heartbeat_front_addr":"172.21.15.59:6805/3373878772","state":["exists","up"]},{"osd":1,"uuid":"e8d882aa-52e1-4c04-9e1e-c44ef9204a5f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6824","nonce":648258563},{"type":"v1","addr":"172.21.15.59:6825","nonce":648258563}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6826","nonce":648258563},{"type":"v1","addr":"172.21.15.59:6827","nonce":648258563}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6830","nonce":648258563},{"type":"v1","addr":"172.21.15.59:6831","nonce":648258563}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6828","nonce":648258563},{"type":"v1","addr":"172.21.15.59:6829","nonce":648258563}]},"public_addr":"172.21.15.59:6825/648258563","cluster_addr":"172.21.15.59:6827/648258563","heartbeat_back_addr":"172.21.15.59:6831/648258563","heartbeat_front_addr":"172.21.15.59:6829/648258563","state":["exists","up"]},{"osd":2,"uuid":"69eebfa0-695b-4b21-90b9-8a3537168a01","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6808","nonce":1840430654},{"type":"v1","addr":"172.21.15.59:6809","nonce":1840430654}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6810","nonce":1840430654},{"type":"v1","addr":"172.21.15.59:6811","nonce":1840430654}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6814","nonce":1840430654},{"type":"v1","addr":"172.21.15.59:6815","nonce":1840430654}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6812","nonce":1840430654},{"type":"v1","addr":"172.21.15.59:6813","nonce":1840430654}]},"public_addr":"172.21.15.59:6809/1840430654","cluster_addr":"172.21.15.59:6811/1840430654","heartbeat_back_addr":"172.21.15.59:6815/1840430654","heartbeat_front_addr":"172.21.15.59:6813/1840430654","state":["exists","up"]},{"osd":3,"uuid":"96fd77fc-f911-46ff-9445-6f0be2daa8f4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6816","nonce":2504025850},{"type":"v1","addr":"172.21.15.59:6817","nonce":2504025850}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6818","nonce":2504025850},{"type":"v1","addr":"172.21.15.59:6819","nonce":2504025850}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6822","nonce":2504025850},{"type":"v1","addr":"172.21.15.59:6823","nonce":2504025850}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6820","nonce":2504025850},{"type":"v1","addr":"172.21.15.59:6821","nonce":2504025850}]},"public_addr":"172.21.15.59:6817/2504025850","cluster_addr":"172.21.15.59:6819/2504025850","heartbeat_back_addr":"172.21.15.59:6823/2504025850","heartbeat_front_addr":"172.21.15.59:6821/2504025850","state":["exists","up"]},{"osd":4,"uuid":"7ac29fca-b5b2-4c95-9d69-2f164cbbc708","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6816","nonce":2382463431},{"type":"v1","addr":"172.21.15.76:6818","nonce":2382463431}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6820","nonce":2382463431},{"type":"v1","addr":"172.21.15.76:6822","nonce":2382463431}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6828","nonce":2382463431},{"type":"v1","addr":"172.21.15.76:6830","nonce":2382463431}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6824","nonce":2382463431},{"type":"v1","addr":"172.21.15.76:6826","nonce":2382463431}]},"public_addr":"172.21.15.76:6818/2382463431","cluster_addr":"172.21.15.76:6822/2382463431","heartbeat_back_addr":"172.21.15.76:6830/2382463431","heartbeat_front_addr":"172.21.15.76:6826/2382463431","state":["exists","up"]},{"osd":5,"uuid":"485bf711-3851-4238-a4f4-51d52e0a9c58","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6800","nonce":329334308},{"type":"v1","addr":"172.21.15.76:6802","nonce":329334308}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6805","nonce":329334308},{"type":"v1","addr":"172.21.15.76:6807","nonce":329334308}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6814","nonce":329334308},{"type":"v1","addr":"172.21.15.76:6815","nonce":329334308}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6810","nonce":329334308},{"type":"v1","addr":"172.21.15.76:6812","nonce":329334308}]},"public_addr":"172.21.15.76:6802/329334308","cluster_addr":"172.21.15.76:6807/329334308","heartbeat_back_addr":"172.21.15.76:6815/329334308","heartbeat_front_addr":"172.21.15.76:6812/329334308","state":["exists","up"]},{"osd":6,"uuid":"72b3f2fc-1e4a-4e32-968d-003647a0286f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6801","nonce":3946579410},{"type":"v1","addr":"172.21.15.76:6803","nonce":3946579410}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6804","nonce":3946579410},{"type":"v1","addr":"172.21.15.76:6806","nonce":3946579410}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6811","nonce":3946579410},{"type":"v1","addr":"172.21.15.76:6813","nonce":3946579410}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6808","nonce":3946579410},{"type":"v1","addr":"172.21.15.76:6809","nonce":3946579410}]},"public_addr":"172.21.15.76:6803/3946579410","cluster_addr":"172.21.15.76:6806/3946579410","heartbeat_back_addr":"172.21.15.76:6813/3946579410","heartbeat_front_addr":"172.21.15.76:6809/3946579410","state":["exists","up"]},{"osd":7,"uuid":"1ddceb90-6c5d-4d1a-ad56-99cb877f8c70","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6817","nonce":841761538},{"type":"v1","addr":"172.21.15.76:6819","nonce":841761538}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6821","nonce":841761538},{"type":"v1","addr":"172.21.15.76:6823","nonce":841761538}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6829","nonce":841761538},{"type":"v1","addr":"172.21.15.76:6831","nonce":841761538}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6825","nonce":841761538},{"type":"v1","addr":"172.21.15.76:6827","nonce":841761538}]},"public_addr":"172.21.15.76:6819/841761538","cluster_addr":"172.21.15.76:6823/841761538","heartbeat_back_addr":"172.21.15.76:6831/841761538","heartbeat_front_addr":"172.21.15.76:6827/841761538","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T07:56:14.294838+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T07:56:14.270779+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T07:56:14.231809+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T07:56:14.260983+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T07:56:14.302997+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T07:56:14.258600+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T07:56:14.251413+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T07:56:14.304727+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-24T08:00:15.491 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:15.489+0000 7f7cff0cd700 1 -- 172.21.15.59:0/1277627512 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f7cdc06b500 msgr2=0x7f7cdc06d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:00:15.491 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:15.489+0000 7f7cff0cd700 1 --2- 172.21.15.59:0/1277627512 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f7cdc06b500 0x7f7cdc06d9c0 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7f7ce40090b0 tx=0x7f7ce4008030 comp rx=0 tx=0).stop 2023-11-24T08:00:15.492 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:15.489+0000 7f7cff0cd700 1 -- 172.21.15.59:0/1277627512 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7cf806afd0 msgr2=0x7f7cf815a9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:00:15.492 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:15.489+0000 7f7cff0cd700 1 --2- 172.21.15.59:0/1277627512 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7cf806afd0 0x7f7cf815a9c0 secure :-1 s=READY pgs=1077 cs=0 l=1 rev1=1 crypto rx=0x7f7cec00e810 tx=0x7f7cec00ebd0 comp rx=0 tx=0).stop 2023-11-24T08:00:15.492 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:15.489+0000 7f7cff0cd700 1 -- 172.21.15.59:0/1277627512 shutdown_connections 2023-11-24T08:00:15.492 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:15.489+0000 7f7cff0cd700 1 --2- 172.21.15.59:0/1277627512 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f7cdc06b500 0x7f7cdc06d9c0 unknown :-1 s=CLOSED pgs=494 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:00:15.492 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:15.489+0000 7f7cff0cd700 1 --2- 172.21.15.59:0/1277627512 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f7cf81457a0 0x7f7cf815cdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:00:15.492 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:15.489+0000 7f7cff0cd700 1 --2- 172.21.15.59:0/1277627512 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7cf806afd0 0x7f7cf815a9c0 unknown :-1 s=CLOSED pgs=1077 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:00:15.493 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:15.489+0000 7f7cff0cd700 1 --2- 172.21.15.59:0/1277627512 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7cf80cd470 0x7f7cf815c870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:00:15.493 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:15.489+0000 7f7cff0cd700 1 -- 172.21.15.59:0/1277627512 >> 172.21.15.59:0/1277627512 conn(0x7f7cf80d0aa0 msgr2=0x7f7cf806db20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:00:15.493 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:15.489+0000 7f7cff0cd700 1 -- 172.21.15.59:0/1277627512 shutdown_connections 2023-11-24T08:00:15.493 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:15.489+0000 7f7cff0cd700 1 -- 172.21.15.59:0/1277627512 wait complete. 2023-11-24T08:00:15.504 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-24T07:56:18.352823+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 12, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-24T07:59:50.274996+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '86', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 13, 'pool_name': 'cephfs_data', 'create_time': '2023-11-24T07:59:50.873919+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '87', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-24T08:00:15.505 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-24T08:00:15.632 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:15.631+0000 7f8d582ed700 1 Processor -- start 2023-11-24T08:00:15.633 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:15.631+0000 7f8d582ed700 1 -- start start 2023-11-24T08:00:15.639 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:15.632+0000 7f8d582ed700 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f8d500d2be0 0x7f8d500b4ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:00:15.639 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:15.632+0000 7f8d582ed700 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8d500b7810 con 0x7f8d500cfde0 2023-11-24T08:00:15.639 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:15.632+0000 7f8d582ed700 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8d500d1660 con 0x7f8d5015e190 2023-11-24T08:00:15.639 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:15.632+0000 7f8d582ed700 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8d500d1930 con 0x7f8d500d2be0 2023-11-24T08:00:15.639 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:15.632+0000 7f8d55888700 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7f8d500cfde0 0x7f8d500d24d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:59674/0 (socket says 172.21.15.59:59674) 2023-11-24T08:00:15.639 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:15.632+0000 7f8d55888700 1 -- 172.21.15.59:0/773854841 learned_addr learned my addr 172.21.15.59:0/773854841 (peer_addr_for_me v1:172.21.15.59:0/0) 2023-11-24T08:00:15.639 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:15.632+0000 7f8d5688a700 1 --2- 172.21.15.59:0/773854841 >> v2:172.21.15.76:3300/0 conn(0x7f8d500d2be0 0x7f8d500b4ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:00:15.639 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:15.632+0000 7f8d55087700 1 -- 172.21.15.59:0/773854841 <== mon.0 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2958370778 0 0) 0x7f8d500b7810 con 0x7f8d500cfde0 2023-11-24T08:00:15.639 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:15.632+0000 7f8d55087700 1 -- 172.21.15.59:0/773854841 --> v1:172.21.15.59:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8d380035b0 con 0x7f8d500cfde0 2023-11-24T08:00:15.640 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:15.632+0000 7f8d55087700 1 -- 172.21.15.59:0/773854841 <== mon.0 v1:172.21.15.59:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 301031914 0 0) 0x7f8d380035b0 con 0x7f8d500cfde0 2023-11-24T08:00:15.640 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:15.632+0000 7f8d55087700 1 -- 172.21.15.59:0/773854841 >> v2:172.21.15.76:3300/0 conn(0x7f8d500d2be0 msgr2=0x7f8d500b4ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:00:15.640 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:15.632+0000 7f8d55087700 1 --2- 172.21.15.59:0/773854841 >> v2:172.21.15.76:3300/0 conn(0x7f8d500d2be0 0x7f8d500b4ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:00:15.640 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:15.632+0000 7f8d55087700 1 -- 172.21.15.59:0/773854841 >> v1:172.21.15.76:6789/0 conn(0x7f8d5015e190 legacy=0x7f8d500d0f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:00:15.640 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:15.632+0000 7f8d55087700 1 -- 172.21.15.59:0/773854841 --> v1:172.21.15.59:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8d500b6ee0 con 0x7f8d500cfde0 2023-11-24T08:00:15.640 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:15.632+0000 7f8d55087700 1 -- 172.21.15.59:0/773854841 <== mon.0 v1:172.21.15.59:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1312333566 0 0) 0x7f8d44001670 con 0x7f8d500cfde0 2023-11-24T08:00:15.641 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:15.632+0000 7f8d55087700 1 -- 172.21.15.59:0/773854841 >> v1:172.21.15.59:6789/0 conn(0x7f8d500cfde0 legacy=0x7f8d500d24d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:00:15.641 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:15.632+0000 7f8d55087700 1 --2- 172.21.15.59:0/773854841 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8d38003a00 0x7f8d38003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:00:15.641 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:15.632+0000 7f8d55087700 1 --2- 172.21.15.59:0/773854841 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f8d38004850 0x7f8d38004cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:00:15.641 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:15.632+0000 7f8d55087700 1 --2- 172.21.15.59:0/773854841 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8d38005210 0x7f8d3800ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:00:15.641 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:15.632+0000 7f8d55087700 1 -- 172.21.15.59:0/773854841 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d500d1930 con 0x7f8d38005210 2023-11-24T08:00:15.641 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:15.632+0000 7f8d55087700 1 -- 172.21.15.59:0/773854841 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d3800c0b0 con 0x7f8d38003a00 2023-11-24T08:00:15.641 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:15.632+0000 7f8d55087700 1 -- 172.21.15.59:0/773854841 --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8d3800c270 con 0x7f8d38004850 2023-11-24T08:00:15.641 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:15.633+0000 7f8d5688a700 1 --2- 172.21.15.59:0/773854841 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8d38005210 0x7f8d3800ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:00:15.642 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:15.633+0000 7f8d56089700 1 --2- 172.21.15.59:0/773854841 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8d38003a00 0x7f8d38003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:00:15.642 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:15.633+0000 7f8d55888700 1 --2- 172.21.15.59:0/773854841 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f8d38004850 0x7f8d38004cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:00:15.642 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:15.633+0000 7f8d5688a700 1 -- 172.21.15.59:0/773854841 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f8d38004850 msgr2=0x7f8d38004cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:00:15.642 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:15.633+0000 7f8d5688a700 1 --2- 172.21.15.59:0/773854841 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f8d38004850 0x7f8d38004cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:00:15.642 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:15.633+0000 7f8d5688a700 1 -- 172.21.15.59:0/773854841 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8d38003a00 msgr2=0x7f8d38003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:00:15.642 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:15.633+0000 7f8d5688a700 1 --2- 172.21.15.59:0/773854841 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8d38003a00 0x7f8d38003e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:00:15.643 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:15.633+0000 7f8d5688a700 1 -- 172.21.15.59:0/773854841 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8d380035b0 con 0x7f8d38005210 2023-11-24T08:00:15.643 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:15.633+0000 7f8d5688a700 1 -- 172.21.15.59:0/773854841 reap_dead start 2023-11-24T08:00:15.643 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:15.633+0000 7f8d5688a700 1 --2- 172.21.15.59:0/773854841 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8d38005210 0x7f8d3800ba10 secure :-1 s=READY pgs=1079 cs=0 l=1 rev1=1 crypto rx=0x7f8d48009b30 tx=0x7f8d38005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a9b8d67ea5337759 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:00:15.643 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:15.633+0000 7f8d55087700 1 -- 172.21.15.59:0/773854841 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d48004020 con 0x7f8d38005210 2023-11-24T08:00:15.643 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:15.633+0000 7f8d55087700 1 -- 172.21.15.59:0/773854841 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8d48004180 con 0x7f8d38005210 2023-11-24T08:00:15.643 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:15.633+0000 7f8d55087700 1 -- 172.21.15.59:0/773854841 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d48004530 con 0x7f8d38005210 2023-11-24T08:00:15.643 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:15.634+0000 7f8d582ed700 1 -- 172.21.15.59:0/773854841 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8d38005210 msgr2=0x7f8d3800ba10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:00:15.643 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:15.634+0000 7f8d582ed700 1 --2- 172.21.15.59:0/773854841 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8d38005210 0x7f8d3800ba10 secure :-1 s=READY pgs=1079 cs=0 l=1 rev1=1 crypto rx=0x7f8d48009b30 tx=0x7f8d38005880 comp rx=0 tx=0).stop 2023-11-24T08:00:15.643 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:15.634+0000 7f8d582ed700 1 -- 172.21.15.59:0/773854841 shutdown_connections 2023-11-24T08:00:15.643 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:15.634+0000 7f8d582ed700 1 --2- 172.21.15.59:0/773854841 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8d38005210 0x7f8d3800ba10 unknown :-1 s=CLOSED pgs=1079 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:00:15.644 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:15.634+0000 7f8d582ed700 1 -- 172.21.15.59:0/773854841 >> 172.21.15.59:0/773854841 conn(0x7f8d5009d180 msgr2=0x7f8d500d3ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:00:15.644 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:15.634+0000 7f8d582ed700 1 -- 172.21.15.59:0/773854841 shutdown_connections 2023-11-24T08:00:15.644 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:15.634+0000 7f8d582ed700 1 -- 172.21.15.59:0/773854841 wait complete. 2023-11-24T08:00:15.644 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:15.635+0000 7f8d582ed700 1 Processor -- start 2023-11-24T08:00:15.644 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:15.635+0000 7f8d582ed700 1 -- start start 2023-11-24T08:00:15.644 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:15.636+0000 7f8d582ed700 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f8d38005210 0x7f8d5009ff60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:00:15.644 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:15.636+0000 7f8d582ed700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8d5015c410 0x7f8d5015c890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:00:15.644 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:15.636+0000 7f8d582ed700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8d500c9810 0x7f8d500c9cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:00:15.644 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:15.636+0000 7f8d582ed700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d500a13a0 con 0x7f8d5015c410 2023-11-24T08:00:15.644 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:15.636+0000 7f8d582ed700 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d500ca200 con 0x7f8d500c9810 2023-11-24T08:00:15.644 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:15.636+0000 7f8d582ed700 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8d5009eba0 con 0x7f8d38005210 2023-11-24T08:00:15.644 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:15.636+0000 7f8d55888700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8d5015c410 0x7f8d5015c890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:00:15.645 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:15.636+0000 7f8d55888700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8d5015c410 0x7f8d5015c890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:56212/0 (socket says 172.21.15.59:56212) 2023-11-24T08:00:15.645 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:15.636+0000 7f8d55888700 1 -- 172.21.15.59:0/1429243384 learned_addr learned my addr 172.21.15.59:0/1429243384 (peer_addr_for_me v2:172.21.15.59:0/0) 2023-11-24T08:00:15.645 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:15.636+0000 7f8d56089700 1 --2- 172.21.15.59:0/1429243384 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f8d38005210 0x7f8d5009ff60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:00:15.645 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:15.636+0000 7f8d5688a700 1 --2- 172.21.15.59:0/1429243384 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8d500c9810 0x7f8d500c9cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:00:15.645 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:15.636+0000 7f8d55888700 1 -- 172.21.15.59:0/1429243384 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f8d38005210 msgr2=0x7f8d5009ff60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:00:15.645 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:15.636+0000 7f8d55888700 1 --2- 172.21.15.59:0/1429243384 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f8d38005210 0x7f8d5009ff60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:00:15.645 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:15.636+0000 7f8d55888700 1 -- 172.21.15.59:0/1429243384 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8d500c9810 msgr2=0x7f8d500c9cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:00:15.645 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:15.636+0000 7f8d55888700 1 --2- 172.21.15.59:0/1429243384 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8d500c9810 0x7f8d500c9cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:00:15.645 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:15.636+0000 7f8d55888700 1 -- 172.21.15.59:0/1429243384 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8d5009ddb0 con 0x7f8d5015c410 2023-11-24T08:00:15.645 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:15.636+0000 7f8d55888700 1 --2- 172.21.15.59:0/1429243384 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8d5015c410 0x7f8d5015c890 secure :-1 s=READY pgs=1080 cs=0 l=1 rev1=1 crypto rx=0x7f8d440072c0 tx=0x7f8d440075d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:00:15.646 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:15.637+0000 7f8d4effd700 1 -- 172.21.15.59:0/1429243384 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d44011580 con 0x7f8d5015c410 2023-11-24T08:00:15.646 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:15.637+0000 7f8d4effd700 1 -- 172.21.15.59:0/1429243384 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8d440116e0 con 0x7f8d5015c410 2023-11-24T08:00:15.646 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:15.637+0000 7f8d582ed700 1 -- 172.21.15.59:0/1429243384 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8d500a04a0 con 0x7f8d5015c410 2023-11-24T08:00:15.646 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:15.637+0000 7f8d4effd700 1 -- 172.21.15.59:0/1429243384 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d44011a90 con 0x7f8d5015c410 2023-11-24T08:00:15.646 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:15.637+0000 7f8d582ed700 1 -- 172.21.15.59:0/1429243384 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8d500a0a00 con 0x7f8d5015c410 2023-11-24T08:00:15.648 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:15.638+0000 7f8d582ed700 1 -- 172.21.15.59:0/1429243384 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8d500a14e0 con 0x7f8d5015c410 2023-11-24T08:00:15.649 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:15.638+0000 7f8d4effd700 1 -- 172.21.15.59:0/1429243384 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 12) v1 ==== 89792+0+0 (secure 0 0 0) 0x7f8d44011bf0 con 0x7f8d5015c410 2023-11-24T08:00:15.649 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:15.639+0000 7f8d4effd700 1 --2- 172.21.15.59:0/1429243384 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f8d3c06b500 0x7f8d3c06d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:00:15.649 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:15.639+0000 7f8d4effd700 1 -- 172.21.15.59:0/1429243384 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f8d44089e10 con 0x7f8d5015c410 2023-11-24T08:00:15.649 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:15.639+0000 7f8d56089700 1 --2- 172.21.15.59:0/1429243384 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f8d3c06b500 0x7f8d3c06d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:00:15.649 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:15.639+0000 7f8d56089700 1 --2- 172.21.15.59:0/1429243384 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f8d3c06b500 0x7f8d3c06d9c0 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7f8d500cd050 tx=0x7f8d40009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:00:15.649 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:15.643+0000 7f8d4effd700 1 -- 172.21.15.59:0/1429243384 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f8d500a14e0 con 0x7f8d5015c410 2023-11-24T08:00:15.848 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:15.846+0000 7f8d582ed700 1 -- 172.21.15.59:0/1429243384 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f8d5009ddb0 con 0x7f8d5015c410 2023-11-24T08:00:15.848 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:15.846+0000 7f8d4effd700 1 -- 172.21.15.59:0/1429243384 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v87) v1 ==== 93+0+10 (secure 0 0 0) 0x7f8d5009ddb0 con 0x7f8d5015c410 2023-11-24T08:00:15.848 INFO:teuthology.orchestra.run.smithi059.stdout:pg_num: 1 2023-11-24T08:00:15.851 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:15.849+0000 7f8d582ed700 1 -- 172.21.15.59:0/1429243384 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f8d3c06b500 msgr2=0x7f8d3c06d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:00:15.851 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:15.849+0000 7f8d582ed700 1 --2- 172.21.15.59:0/1429243384 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f8d3c06b500 0x7f8d3c06d9c0 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7f8d500cd050 tx=0x7f8d40009040 comp rx=0 tx=0).stop 2023-11-24T08:00:15.851 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:15.849+0000 7f8d582ed700 1 -- 172.21.15.59:0/1429243384 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8d5015c410 msgr2=0x7f8d5015c890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:00:15.851 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:15.849+0000 7f8d582ed700 1 --2- 172.21.15.59:0/1429243384 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8d5015c410 0x7f8d5015c890 secure :-1 s=READY pgs=1080 cs=0 l=1 rev1=1 crypto rx=0x7f8d440072c0 tx=0x7f8d440075d0 comp rx=0 tx=0).stop 2023-11-24T08:00:15.851 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:15.849+0000 7f8d582ed700 1 -- 172.21.15.59:0/1429243384 shutdown_connections 2023-11-24T08:00:15.851 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:15.849+0000 7f8d582ed700 1 --2- 172.21.15.59:0/1429243384 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f8d3c06b500 0x7f8d3c06d9c0 unknown :-1 s=CLOSED pgs=495 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:00:15.851 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:15.849+0000 7f8d582ed700 1 --2- 172.21.15.59:0/1429243384 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8d500c9810 0x7f8d500c9cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:00:15.851 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:15.849+0000 7f8d582ed700 1 --2- 172.21.15.59:0/1429243384 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f8d38005210 0x7f8d5009ff60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:00:15.851 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:15.849+0000 7f8d582ed700 1 --2- 172.21.15.59:0/1429243384 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8d5015c410 0x7f8d5015c890 unknown :-1 s=CLOSED pgs=1080 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:00:15.851 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:15.849+0000 7f8d582ed700 1 -- 172.21.15.59:0/1429243384 >> 172.21.15.59:0/1429243384 conn(0x7f8d5009d180 msgr2=0x7f8d5006be60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:00:15.851 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:15.849+0000 7f8d582ed700 1 -- 172.21.15.59:0/1429243384 shutdown_connections 2023-11-24T08:00:15.851 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:15.849+0000 7f8d582ed700 1 -- 172.21.15.59:0/1429243384 wait complete. 2023-11-24T08:00:15.862 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-24T08:00:15.990 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:15.989+0000 7fd00cad9700 1 Processor -- start 2023-11-24T08:00:15.991 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:15.989+0000 7fd00cad9700 1 -- start start 2023-11-24T08:00:15.993 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:15.990+0000 7fd00cad9700 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fd00815bde0 0x7fd00815c1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:00:15.993 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:15.990+0000 7fd00cad9700 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fd0080cf2b0 0x7fd0080d01b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:00:15.993 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:15.990+0000 7fd00cad9700 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd0080d0a90 con 0x7fd0080ce5c0 2023-11-24T08:00:15.993 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:15.990+0000 7fd00cad9700 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd00806ee10 con 0x7fd00815bde0 2023-11-24T08:00:15.993 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:15.990+0000 7fd00cad9700 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd00806f0f0 con 0x7fd0080cf2b0 2023-11-24T08:00:15.993 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:15.990+0000 7fd006d9d700 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7fd0080ce5c0 0x7fd0080cc670 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:59696/0 (socket says 172.21.15.59:59696) 2023-11-24T08:00:15.993 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:15.990+0000 7fd006d9d700 1 -- 172.21.15.59:0/1403450812 learned_addr learned my addr 172.21.15.59:0/1403450812 (peer_addr_for_me v1:172.21.15.59:0/0) 2023-11-24T08:00:15.993 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:15.990+0000 7fd00659c700 1 --2- 172.21.15.59:0/1403450812 >> v2:172.21.15.76:3300/0 conn(0x7fd00815bde0 0x7fd00815c1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:00:15.994 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:15.990+0000 7fd005d9b700 1 --2- 172.21.15.59:0/1403450812 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fd0080cf2b0 0x7fd0080d01b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:00:15.994 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:15.990+0000 7fd00559a700 1 -- 172.21.15.59:0/1403450812 <== mon.0 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 93564766 0 0) 0x7fd0080d0a90 con 0x7fd0080ce5c0 2023-11-24T08:00:15.994 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:15.990+0000 7fd00559a700 1 -- 172.21.15.59:0/1403450812 --> v1:172.21.15.59:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcff00035b0 con 0x7fd0080ce5c0 2023-11-24T08:00:15.994 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:15.990+0000 7fd00659c700 1 -- 172.21.15.59:0/1403450812 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fd0080cf2b0 msgr2=0x7fd0080d01b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:00:15.994 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:15.990+0000 7fd00659c700 1 --2- 172.21.15.59:0/1403450812 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fd0080cf2b0 0x7fd0080d01b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:00:15.994 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:15.990+0000 7fd00659c700 1 -- 172.21.15.59:0/1403450812 >> v1:172.21.15.59:6789/0 conn(0x7fd0080ce5c0 legacy=0x7fd0080cc670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:00:15.994 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:15.990+0000 7fd00659c700 1 -- 172.21.15.59:0/1403450812 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd00806f3d0 con 0x7fd00815bde0 2023-11-24T08:00:15.994 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:15.990+0000 7fd00659c700 1 --2- 172.21.15.59:0/1403450812 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd00815bde0 0x7fd00815c1c0 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7fcff8009840 tx=0x7fcff8009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e22d997eb6fd6e server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:00:15.994 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:15.991+0000 7fd00559a700 1 -- 172.21.15.59:0/1403450812 <== mon.1 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcff8011830 con 0x7fd00815bde0 2023-11-24T08:00:15.994 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:15.991+0000 7fd00559a700 1 -- 172.21.15.59:0/1403450812 <== mon.1 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcff8011990 con 0x7fd00815bde0 2023-11-24T08:00:15.994 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:15.991+0000 7fd00559a700 1 -- 172.21.15.59:0/1403450812 <== mon.1 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcff8011d00 con 0x7fd00815bde0 2023-11-24T08:00:15.994 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:15.991+0000 7fd00cad9700 1 -- 172.21.15.59:0/1403450812 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd00815bde0 msgr2=0x7fd00815c1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:00:15.994 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:15.991+0000 7fd00cad9700 1 --2- 172.21.15.59:0/1403450812 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd00815bde0 0x7fd00815c1c0 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7fcff8009840 tx=0x7fcff8009b50 comp rx=0 tx=0).stop 2023-11-24T08:00:15.995 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:15.991+0000 7fd00cad9700 1 -- 172.21.15.59:0/1403450812 shutdown_connections 2023-11-24T08:00:15.995 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:15.991+0000 7fd00cad9700 1 --2- 172.21.15.59:0/1403450812 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd00815bde0 0x7fd00815c1c0 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:00:15.995 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:15.991+0000 7fd00cad9700 1 --2- 172.21.15.59:0/1403450812 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fd0080cf2b0 0x7fd0080d01b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:00:15.995 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:15.991+0000 7fd00cad9700 1 -- 172.21.15.59:0/1403450812 >> 172.21.15.59:0/1403450812 conn(0x7fd0080b3720 msgr2=0x7fd0080b3b30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:00:15.995 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:15.991+0000 7fd00cad9700 1 -- 172.21.15.59:0/1403450812 shutdown_connections 2023-11-24T08:00:15.995 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:15.991+0000 7fd00cad9700 1 -- 172.21.15.59:0/1403450812 wait complete. 2023-11-24T08:00:15.997 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:15.993+0000 7fd00cad9700 1 Processor -- start 2023-11-24T08:00:15.997 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:15.993+0000 7fd00cad9700 1 -- start start 2023-11-24T08:00:15.997 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:15.994+0000 7fd00cad9700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd0080ce5c0 0x7fd00814adb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:00:15.998 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:15.994+0000 7fd00cad9700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd0080cf2b0 0x7fd0081f0bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:00:15.998 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:15.994+0000 7fd00cad9700 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fd00815bde0 0x7fd0081f1100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:00:15.998 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:15.994+0000 7fd00cad9700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd008149410 con 0x7fd0080ce5c0 2023-11-24T08:00:15.998 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:15.994+0000 7fd00cad9700 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd0081f1640 con 0x7fd0080cf2b0 2023-11-24T08:00:15.998 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:15.994+0000 7fd00cad9700 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd008148ab0 con 0x7fd00815bde0 2023-11-24T08:00:15.998 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:15.994+0000 7fd00659c700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd0080ce5c0 0x7fd00814adb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:00:15.998 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:15.994+0000 7fd00659c700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd0080ce5c0 0x7fd00814adb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:56224/0 (socket says 172.21.15.59:56224) 2023-11-24T08:00:15.998 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:15.994+0000 7fd00659c700 1 -- 172.21.15.59:0/3119239486 learned_addr learned my addr 172.21.15.59:0/3119239486 (peer_addr_for_me v2:172.21.15.59:0/0) 2023-11-24T08:00:15.998 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:15.994+0000 7fd005d9b700 1 --2- 172.21.15.59:0/3119239486 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd0080cf2b0 0x7fd0081f0bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:00:15.998 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:15.994+0000 7fd006d9d700 1 --2- 172.21.15.59:0/3119239486 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fd00815bde0 0x7fd0081f1100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:00:15.998 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:15.994+0000 7fd00659c700 1 -- 172.21.15.59:0/3119239486 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fd00815bde0 msgr2=0x7fd0081f1100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:00:15.999 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:15.994+0000 7fd00659c700 1 --2- 172.21.15.59:0/3119239486 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fd00815bde0 0x7fd0081f1100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:00:15.999 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:15.994+0000 7fd00659c700 1 -- 172.21.15.59:0/3119239486 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd0080cf2b0 msgr2=0x7fd0081f0bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:00:15.999 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:15.994+0000 7fd00659c700 1 --2- 172.21.15.59:0/3119239486 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd0080cf2b0 0x7fd0081f0bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:00:15.999 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:15.994+0000 7fd00659c700 1 -- 172.21.15.59:0/3119239486 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcff80094a0 con 0x7fd0080ce5c0 2023-11-24T08:00:15.999 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:15.994+0000 7fd005d9b700 1 --2- 172.21.15.59:0/3119239486 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd0080cf2b0 0x7fd0081f0bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-24T08:00:15.999 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:15.995+0000 7fd00659c700 1 --2- 172.21.15.59:0/3119239486 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd0080ce5c0 0x7fd00814adb0 secure :-1 s=READY pgs=1082 cs=0 l=1 rev1=1 crypto rx=0x7fcff80090b0 tx=0x7fcff8004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:00:15.999 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:15.995+0000 7fcff6ffd700 1 -- 172.21.15.59:0/3119239486 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcff80036a0 con 0x7fd0080ce5c0 2023-11-24T08:00:15.999 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:15.995+0000 7fcff6ffd700 1 -- 172.21.15.59:0/3119239486 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcff801e070 con 0x7fd0080ce5c0 2023-11-24T08:00:15.999 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:15.995+0000 7fd00cad9700 1 -- 172.21.15.59:0/3119239486 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd00815d9e0 con 0x7fd0080ce5c0 2023-11-24T08:00:16.000 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:15.995+0000 7fcff6ffd700 1 -- 172.21.15.59:0/3119239486 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcff8003a20 con 0x7fd0080ce5c0 2023-11-24T08:00:16.000 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:15.995+0000 7fd00cad9700 1 -- 172.21.15.59:0/3119239486 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd008149030 con 0x7fd0080ce5c0 2023-11-24T08:00:16.000 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:15.996+0000 7fd00cad9700 1 -- 172.21.15.59:0/3119239486 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd008075480 con 0x7fd0080ce5c0 2023-11-24T08:00:16.001 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:15.996+0000 7fcff6ffd700 1 -- 172.21.15.59:0/3119239486 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 12) v1 ==== 89792+0+0 (secure 0 0 0) 0x7fcff8003b80 con 0x7fd0080ce5c0 2023-11-24T08:00:16.001 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:15.997+0000 7fcff6ffd700 1 --2- 172.21.15.59:0/3119239486 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7fcfe806b500 0x7fcfe806d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:00:16.001 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:15.997+0000 7fcff6ffd700 1 -- 172.21.15.59:0/3119239486 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fcff808b060 con 0x7fd0080ce5c0 2023-11-24T08:00:16.001 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:15.997+0000 7fd005d9b700 1 --2- 172.21.15.59:0/3119239486 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7fcfe806b500 0x7fcfe806d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:00:16.001 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:15.997+0000 7fd005d9b700 1 --2- 172.21.15.59:0/3119239486 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7fcfe806b500 0x7fcfe806d9c0 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7fd00814ab10 tx=0x7fcffc008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:00:16.003 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:16.001+0000 7fcff6ffd700 1 -- 172.21.15.59:0/3119239486 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fcff8055860 con 0x7fd0080ce5c0 2023-11-24T08:00:16.203 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:16.202+0000 7fd00cad9700 1 -- 172.21.15.59:0/3119239486 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fd0081e4d90 con 0x7fd0080ce5c0 2023-11-24T08:00:16.204 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:16.202+0000 7fcff6ffd700 1 -- 172.21.15.59:0/3119239486 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v87) v1 ==== 104+0+11 (secure 0 0 0) 0x7fcff80200a0 con 0x7fd0080ce5c0 2023-11-24T08:00:16.204 INFO:teuthology.orchestra.run.smithi059.stdout:pg_num: 32 2023-11-24T08:00:16.206 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:16.204+0000 7fd00cad9700 1 -- 172.21.15.59:0/3119239486 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7fcfe806b500 msgr2=0x7fcfe806d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:00:16.206 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:16.204+0000 7fd00cad9700 1 --2- 172.21.15.59:0/3119239486 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7fcfe806b500 0x7fcfe806d9c0 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7fd00814ab10 tx=0x7fcffc008040 comp rx=0 tx=0).stop 2023-11-24T08:00:16.206 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:16.204+0000 7fd00cad9700 1 -- 172.21.15.59:0/3119239486 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd0080ce5c0 msgr2=0x7fd00814adb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:00:16.206 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:16.204+0000 7fd00cad9700 1 --2- 172.21.15.59:0/3119239486 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd0080ce5c0 0x7fd00814adb0 secure :-1 s=READY pgs=1082 cs=0 l=1 rev1=1 crypto rx=0x7fcff80090b0 tx=0x7fcff8004750 comp rx=0 tx=0).stop 2023-11-24T08:00:16.207 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:16.204+0000 7fd00cad9700 1 -- 172.21.15.59:0/3119239486 shutdown_connections 2023-11-24T08:00:16.207 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:16.204+0000 7fd00cad9700 1 --2- 172.21.15.59:0/3119239486 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7fcfe806b500 0x7fcfe806d9c0 unknown :-1 s=CLOSED pgs=496 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:00:16.207 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:16.204+0000 7fd00cad9700 1 --2- 172.21.15.59:0/3119239486 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fd00815bde0 0x7fd0081f1100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:00:16.207 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:16.204+0000 7fd00cad9700 1 --2- 172.21.15.59:0/3119239486 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd0080ce5c0 0x7fd00814adb0 unknown :-1 s=CLOSED pgs=1082 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:00:16.207 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:16.204+0000 7fd00cad9700 1 --2- 172.21.15.59:0/3119239486 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd0080cf2b0 0x7fd0081f0bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:00:16.207 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:16.204+0000 7fd00cad9700 1 -- 172.21.15.59:0/3119239486 >> 172.21.15.59:0/3119239486 conn(0x7fd0080b3720 msgr2=0x7fd0080b3b00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:00:16.207 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:16.204+0000 7fd00cad9700 1 -- 172.21.15.59:0/3119239486 shutdown_connections 2023-11-24T08:00:16.207 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:16.204+0000 7fd00cad9700 1 -- 172.21.15.59:0/3119239486 wait complete. 2023-11-24T08:00:16.218 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-24T08:00:16.347 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:16.345+0000 7f20e879d700 1 Processor -- start 2023-11-24T08:00:16.347 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:16.345+0000 7f20e879d700 1 -- start start 2023-11-24T08:00:16.348 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:16.346+0000 7f20e879d700 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f20e00b0e90 0x7f20e006eef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:00:16.348 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:16.346+0000 7f20e879d700 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f20e00d1230 con 0x7f20e006fc50 2023-11-24T08:00:16.348 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:16.346+0000 7f20e879d700 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f20e00d0c70 con 0x7f20e015e1c0 2023-11-24T08:00:16.348 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:16.346+0000 7f20e879d700 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f20e00d0f50 con 0x7f20e00b0e90 2023-11-24T08:00:16.348 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:16.346+0000 7f20e5d38700 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7f20e006fc50 0x7f20e006ca10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:59708/0 (socket says 172.21.15.59:59708) 2023-11-24T08:00:16.349 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:16.346+0000 7f20e5d38700 1 -- 172.21.15.59:0/1854590006 learned_addr learned my addr 172.21.15.59:0/1854590006 (peer_addr_for_me v1:172.21.15.59:0/0) 2023-11-24T08:00:16.349 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:16.346+0000 7f20e6d3a700 1 --2- 172.21.15.59:0/1854590006 >> v2:172.21.15.76:3300/0 conn(0x7f20e00b0e90 0x7f20e006eef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:00:16.349 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:16.346+0000 7f20e5537700 1 -- 172.21.15.59:0/1854590006 <== mon.0 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 740595042 0 0) 0x7f20e00d1230 con 0x7f20e006fc50 2023-11-24T08:00:16.349 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:16.346+0000 7f20e5537700 1 -- 172.21.15.59:0/1854590006 --> v1:172.21.15.59:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f20cc0035b0 con 0x7f20e006fc50 2023-11-24T08:00:16.349 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:16.347+0000 7f20e6d3a700 1 -- 172.21.15.59:0/1854590006 >> v1:172.21.15.76:6789/0 conn(0x7f20e015e1c0 legacy=0x7f20e006f540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:00:16.349 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:16.347+0000 7f20e5537700 1 -- 172.21.15.59:0/1854590006 <== mon.0 v1:172.21.15.59:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 816253172 0 0) 0x7f20cc0035b0 con 0x7f20e006fc50 2023-11-24T08:00:16.349 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:16.347+0000 7f20e6d3a700 1 -- 172.21.15.59:0/1854590006 >> v1:172.21.15.59:6789/0 conn(0x7f20e006fc50 legacy=0x7f20e006ca10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:00:16.349 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:16.347+0000 7f20e6d3a700 1 -- 172.21.15.59:0/1854590006 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f20e006c210 con 0x7f20e00b0e90 2023-11-24T08:00:16.349 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:16.347+0000 7f20e6d3a700 1 --2- 172.21.15.59:0/1854590006 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f20e00b0e90 0x7f20e006eef0 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f20d8009d90 tx=0x7f20d800b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=138e4287732ae761 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:00:16.350 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:16.347+0000 7f20e5537700 1 -- 172.21.15.59:0/1854590006 <== mon.1 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20d800e070 con 0x7f20e00b0e90 2023-11-24T08:00:16.350 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:16.347+0000 7f20e5537700 1 -- 172.21.15.59:0/1854590006 <== mon.1 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f20d8003a20 con 0x7f20e00b0e90 2023-11-24T08:00:16.351 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:16.347+0000 7f20e5537700 1 -- 172.21.15.59:0/1854590006 <== mon.1 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20d0001670 con 0x7f20e00b0e90 2023-11-24T08:00:16.351 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:16.348+0000 7f20e879d700 1 -- 172.21.15.59:0/1854590006 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f20e00b0e90 msgr2=0x7f20e006eef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:00:16.351 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:16.348+0000 7f20e879d700 1 --2- 172.21.15.59:0/1854590006 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f20e00b0e90 0x7f20e006eef0 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f20d8009d90 tx=0x7f20d800b330 comp rx=0 tx=0).stop 2023-11-24T08:00:16.351 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:16.348+0000 7f20e879d700 1 -- 172.21.15.59:0/1854590006 shutdown_connections 2023-11-24T08:00:16.351 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:16.348+0000 7f20e879d700 1 --2- 172.21.15.59:0/1854590006 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f20e00b0e90 0x7f20e006eef0 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:00:16.351 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:16.348+0000 7f20e879d700 1 -- 172.21.15.59:0/1854590006 >> 172.21.15.59:0/1854590006 conn(0x7f20e00b3720 msgr2=0x7f20e00b3b30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:00:16.351 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:16.348+0000 7f20e879d700 1 -- 172.21.15.59:0/1854590006 shutdown_connections 2023-11-24T08:00:16.351 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:16.348+0000 7f20e879d700 1 -- 172.21.15.59:0/1854590006 wait complete. 2023-11-24T08:00:16.352 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:16.350+0000 7f20e879d700 1 Processor -- start 2023-11-24T08:00:16.352 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:16.350+0000 7f20e879d700 1 -- start start 2023-11-24T08:00:16.354 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:16.350+0000 7f20e879d700 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f20e006fc50 0x7f20e0149de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:00:16.354 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:16.351+0000 7f20e879d700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f20e00b0e90 0x7f20e014b660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:00:16.354 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:16.351+0000 7f20e879d700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f20e015e1c0 0x7f20e014c050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:00:16.354 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:16.351+0000 7f20e879d700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20e014b1d0 con 0x7f20e015e1c0 2023-11-24T08:00:16.354 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:16.351+0000 7f20e879d700 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20e014a320 con 0x7f20e00b0e90 2023-11-24T08:00:16.354 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:16.351+0000 7f20e879d700 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f20e014a5c0 con 0x7f20e006fc50 2023-11-24T08:00:16.354 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:16.351+0000 7f20e6d3a700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f20e015e1c0 0x7f20e014c050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:00:16.354 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:16.351+0000 7f20e6d3a700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f20e015e1c0 0x7f20e014c050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:56242/0 (socket says 172.21.15.59:56242) 2023-11-24T08:00:16.354 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:16.351+0000 7f20e6d3a700 1 -- 172.21.15.59:0/1576427706 learned_addr learned my addr 172.21.15.59:0/1576427706 (peer_addr_for_me v2:172.21.15.59:0/0) 2023-11-24T08:00:16.354 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:16.351+0000 7f20e5d38700 1 --2- 172.21.15.59:0/1576427706 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f20e00b0e90 0x7f20e014b660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:00:16.355 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:16.351+0000 7f20e6539700 1 --2- 172.21.15.59:0/1576427706 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f20e006fc50 0x7f20e0149de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:00:16.355 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:16.351+0000 7f20e6d3a700 1 -- 172.21.15.59:0/1576427706 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f20e006fc50 msgr2=0x7f20e0149de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:00:16.355 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:16.351+0000 7f20e6d3a700 1 --2- 172.21.15.59:0/1576427706 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f20e006fc50 0x7f20e0149de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:00:16.355 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:16.351+0000 7f20e6d3a700 1 -- 172.21.15.59:0/1576427706 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f20e00b0e90 msgr2=0x7f20e014b660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:00:16.355 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:16.351+0000 7f20e6d3a700 1 --2- 172.21.15.59:0/1576427706 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f20e00b0e90 0x7f20e014b660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:00:16.355 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:16.351+0000 7f20e6d3a700 1 -- 172.21.15.59:0/1576427706 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f20d8009a70 con 0x7f20e015e1c0 2023-11-24T08:00:16.355 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:16.351+0000 7f20e5d38700 1 --2- 172.21.15.59:0/1576427706 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f20e00b0e90 0x7f20e014b660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-24T08:00:16.355 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:16.351+0000 7f20e6539700 1 --2- 172.21.15.59:0/1576427706 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f20e006fc50 0x7f20e0149de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-24T08:00:16.355 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:16.351+0000 7f20e6d3a700 1 --2- 172.21.15.59:0/1576427706 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f20e015e1c0 0x7f20e014c050 secure :-1 s=READY pgs=1084 cs=0 l=1 rev1=1 crypto rx=0x7f20d8004000 tx=0x7f20d8004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:00:16.355 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:16.351+0000 7f20d6ffd700 1 -- 172.21.15.59:0/1576427706 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20d80046f0 con 0x7f20e015e1c0 2023-11-24T08:00:16.355 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:16.351+0000 7f20d6ffd700 1 -- 172.21.15.59:0/1576427706 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f20d8004870 con 0x7f20e015e1c0 2023-11-24T08:00:16.356 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:16.351+0000 7f20e879d700 1 -- 172.21.15.59:0/1576427706 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f20e014a8a0 con 0x7f20e015e1c0 2023-11-24T08:00:16.356 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:16.351+0000 7f20d6ffd700 1 -- 172.21.15.59:0/1576427706 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20d801d6b0 con 0x7f20e015e1c0 2023-11-24T08:00:16.356 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:16.351+0000 7f20e879d700 1 -- 172.21.15.59:0/1576427706 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f20e014ae60 con 0x7f20e015e1c0 2023-11-24T08:00:16.356 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:16.352+0000 7f20e879d700 1 -- 172.21.15.59:0/1576427706 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f20e0067ac0 con 0x7f20e015e1c0 2023-11-24T08:00:16.357 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:16.353+0000 7f20d6ffd700 1 -- 172.21.15.59:0/1576427706 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 12) v1 ==== 89792+0+0 (secure 0 0 0) 0x7f20d801d810 con 0x7f20e015e1c0 2023-11-24T08:00:16.357 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:16.353+0000 7f20d6ffd700 1 --2- 172.21.15.59:0/1576427706 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f20c406b500 0x7f20c406d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:00:16.357 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:16.353+0000 7f20d6ffd700 1 -- 172.21.15.59:0/1576427706 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f20d808bff0 con 0x7f20e015e1c0 2023-11-24T08:00:16.357 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:16.353+0000 7f20e6539700 1 --2- 172.21.15.59:0/1576427706 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f20c406b500 0x7f20c406d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:00:16.357 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:16.354+0000 7f20e6539700 1 --2- 172.21.15.59:0/1576427706 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f20c406b500 0x7f20c406d9c0 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7f20dc000f60 tx=0x7f20dc009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:00:16.359 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:16.358+0000 7f20d6ffd700 1 -- 172.21.15.59:0/1576427706 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f20d80566e0 con 0x7f20e015e1c0 2023-11-24T08:00:16.561 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:16.559+0000 7f20e879d700 1 -- 172.21.15.59:0/1576427706 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f20e0075480 con 0x7f20e015e1c0 2023-11-24T08:00:16.562 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:16.560+0000 7f20d6ffd700 1 -- 172.21.15.59:0/1576427706 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v87) v1 ==== 100+0+11 (secure 0 0 0) 0x7f20d8059d00 con 0x7f20e015e1c0 2023-11-24T08:00:16.562 INFO:teuthology.orchestra.run.smithi059.stdout:pg_num: 64 2023-11-24T08:00:16.564 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:16.562+0000 7f20e879d700 1 -- 172.21.15.59:0/1576427706 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f20c406b500 msgr2=0x7f20c406d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:00:16.564 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:16.562+0000 7f20e879d700 1 --2- 172.21.15.59:0/1576427706 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f20c406b500 0x7f20c406d9c0 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7f20dc000f60 tx=0x7f20dc009040 comp rx=0 tx=0).stop 2023-11-24T08:00:16.564 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:16.562+0000 7f20e879d700 1 -- 172.21.15.59:0/1576427706 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f20e015e1c0 msgr2=0x7f20e014c050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:00:16.564 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:16.562+0000 7f20e879d700 1 --2- 172.21.15.59:0/1576427706 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f20e015e1c0 0x7f20e014c050 secure :-1 s=READY pgs=1084 cs=0 l=1 rev1=1 crypto rx=0x7f20d8004000 tx=0x7f20d8004ad0 comp rx=0 tx=0).stop 2023-11-24T08:00:16.564 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:16.562+0000 7f20e879d700 1 -- 172.21.15.59:0/1576427706 shutdown_connections 2023-11-24T08:00:16.564 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:16.562+0000 7f20e879d700 1 --2- 172.21.15.59:0/1576427706 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f20c406b500 0x7f20c406d9c0 unknown :-1 s=CLOSED pgs=497 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:00:16.564 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:16.562+0000 7f20e879d700 1 --2- 172.21.15.59:0/1576427706 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f20e015e1c0 0x7f20e014c050 unknown :-1 s=CLOSED pgs=1084 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:00:16.565 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:16.562+0000 7f20e879d700 1 --2- 172.21.15.59:0/1576427706 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f20e006fc50 0x7f20e0149de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:00:16.565 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:16.562+0000 7f20e879d700 1 --2- 172.21.15.59:0/1576427706 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f20e00b0e90 0x7f20e014b660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:00:16.565 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:16.562+0000 7f20e879d700 1 -- 172.21.15.59:0/1576427706 >> 172.21.15.59:0/1576427706 conn(0x7f20e00b3720 msgr2=0x7f20e006ca10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:00:16.565 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:16.562+0000 7f20e879d700 1 -- 172.21.15.59:0/1576427706 shutdown_connections 2023-11-24T08:00:16.565 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:16.562+0000 7f20e879d700 1 -- 172.21.15.59:0/1576427706 wait complete. 2023-11-24T08:00:16.576 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-11-24T08:00:16.702 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:16.700+0000 7f4deefa2700 1 Processor -- start 2023-11-24T08:00:16.702 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:16.700+0000 7f4deefa2700 1 -- start start 2023-11-24T08:00:16.704 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:16.701+0000 7f4deefa2700 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f4de80af5e0 0x7f4de80ceda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:00:16.704 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:16.701+0000 7f4deefa2700 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4de80d0000 con 0x7f4de80b5d00 2023-11-24T08:00:16.705 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:16.701+0000 7f4deefa2700 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4de80af9c0 con 0x7f4de806b9e0 2023-11-24T08:00:16.705 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:16.701+0000 7f4deefa2700 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4de80b6180 con 0x7f4de80af5e0 2023-11-24T08:00:16.705 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:16.701+0000 7f4ded79f700 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7f4de80b5d00 0x7f4de80d0490 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:59720/0 (socket says 172.21.15.59:59720) 2023-11-24T08:00:16.705 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:16.701+0000 7f4ded79f700 1 -- 172.21.15.59:0/3033446148 learned_addr learned my addr 172.21.15.59:0/3033446148 (peer_addr_for_me v1:172.21.15.59:0/0) 2023-11-24T08:00:16.705 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:16.701+0000 7f4dedfa0700 1 --2- 172.21.15.59:0/3033446148 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f4de80af5e0 0x7f4de80ceda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:00:16.705 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:16.701+0000 7f4decf9e700 1 -- 172.21.15.59:0/3033446148 <== mon.0 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1645419428 0 0) 0x7f4de80d0000 con 0x7f4de80b5d00 2023-11-24T08:00:16.705 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:16.702+0000 7f4decf9e700 1 -- 172.21.15.59:0/3033446148 --> v1:172.21.15.59:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4dd40035b0 con 0x7f4de80b5d00 2023-11-24T08:00:16.705 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:16.702+0000 7f4decf9e700 1 -- 172.21.15.59:0/3033446148 <== mon.1 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3842412626 0 0) 0x7f4de80af9c0 con 0x7f4de806b9e0 2023-11-24T08:00:16.705 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:16.702+0000 7f4decf9e700 1 -- 172.21.15.59:0/3033446148 --> v1:172.21.15.76:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4de80d0000 con 0x7f4de806b9e0 2023-11-24T08:00:16.705 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:16.702+0000 7f4decf9e700 1 -- 172.21.15.59:0/3033446148 <== mon.0 v1:172.21.15.59:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3665274555 0 0) 0x7f4dd40035b0 con 0x7f4de80b5d00 2023-11-24T08:00:16.705 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:16.702+0000 7f4dedfa0700 1 -- 172.21.15.59:0/3033446148 >> v1:172.21.15.76:6789/0 conn(0x7f4de806b9e0 legacy=0x7f4de806d060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:00:16.705 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:16.702+0000 7f4dedfa0700 1 -- 172.21.15.59:0/3033446148 >> v1:172.21.15.59:6789/0 conn(0x7f4de80b5d00 legacy=0x7f4de80d0490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:00:16.706 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:16.702+0000 7f4dedfa0700 1 -- 172.21.15.59:0/3033446148 --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4de80cf3f0 con 0x7f4de80af5e0 2023-11-24T08:00:16.706 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:16.702+0000 7f4dedfa0700 1 --2- 172.21.15.59:0/3033446148 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f4de80af5e0 0x7f4de80ceda0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f4dd8009870 tx=0x7f4dd8009b80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8c6a35b0f02f73bd server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:00:16.706 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:16.702+0000 7f4decf9e700 1 -- 172.21.15.59:0/3033446148 <== mon.2 v2:172.21.15.76:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4de4001670 con 0x7f4de80af5e0 2023-11-24T08:00:16.706 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:16.702+0000 7f4decf9e700 1 -- 172.21.15.59:0/3033446148 <== mon.2 v2:172.21.15.76:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4dd8004030 con 0x7f4de80af5e0 2023-11-24T08:00:16.706 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:16.702+0000 7f4decf9e700 1 -- 172.21.15.59:0/3033446148 <== mon.2 v2:172.21.15.76:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4dd80043c0 con 0x7f4de80af5e0 2023-11-24T08:00:16.706 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:16.703+0000 7f4deefa2700 1 -- 172.21.15.59:0/3033446148 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f4de80af5e0 msgr2=0x7f4de80ceda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:00:16.706 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:16.703+0000 7f4deefa2700 1 --2- 172.21.15.59:0/3033446148 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f4de80af5e0 0x7f4de80ceda0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f4dd8009870 tx=0x7f4dd8009b80 comp rx=0 tx=0).stop 2023-11-24T08:00:16.706 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:16.703+0000 7f4deefa2700 1 -- 172.21.15.59:0/3033446148 shutdown_connections 2023-11-24T08:00:16.706 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:16.703+0000 7f4deefa2700 1 --2- 172.21.15.59:0/3033446148 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f4de80af5e0 0x7f4de80ceda0 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:00:16.706 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:16.703+0000 7f4deefa2700 1 -- 172.21.15.59:0/3033446148 >> 172.21.15.59:0/3033446148 conn(0x7f4de8072410 msgr2=0x7f4de8072820 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:00:16.707 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:16.703+0000 7f4deefa2700 1 -- 172.21.15.59:0/3033446148 shutdown_connections 2023-11-24T08:00:16.707 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:16.703+0000 7f4deefa2700 1 -- 172.21.15.59:0/3033446148 wait complete. 2023-11-24T08:00:16.709 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:16.704+0000 7f4deefa2700 1 Processor -- start 2023-11-24T08:00:16.710 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:16.704+0000 7f4deefa2700 1 -- start start 2023-11-24T08:00:16.710 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:16.705+0000 7f4deefa2700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4de806b9e0 0x7f4de8151400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:00:16.710 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:16.705+0000 7f4deefa2700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4de80af5e0 0x7f4de814ff50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:00:16.710 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:16.705+0000 7f4deefa2700 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f4de80b5d00 0x7f4de8150490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:00:16.710 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:16.705+0000 7f4deefa2700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4de8099720 con 0x7f4de80af5e0 2023-11-24T08:00:16.710 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:16.705+0000 7f4deefa2700 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4de80c4050 con 0x7f4de806b9e0 2023-11-24T08:00:16.710 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:16.705+0000 7f4deefa2700 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4de80c4bd0 con 0x7f4de80b5d00 2023-11-24T08:00:16.710 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:16.705+0000 7f4ded79f700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4de80af5e0 0x7f4de814ff50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:00:16.710 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:16.705+0000 7f4ded79f700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4de80af5e0 0x7f4de814ff50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:56252/0 (socket says 172.21.15.59:56252) 2023-11-24T08:00:16.710 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:16.705+0000 7f4ded79f700 1 -- 172.21.15.59:0/1619216594 learned_addr learned my addr 172.21.15.59:0/1619216594 (peer_addr_for_me v2:172.21.15.59:0/0) 2023-11-24T08:00:16.711 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:16.705+0000 7f4dedfa0700 1 --2- 172.21.15.59:0/1619216594 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4de806b9e0 0x7f4de8151400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:00:16.711 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:16.705+0000 7f4dee7a1700 1 --2- 172.21.15.59:0/1619216594 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f4de80b5d00 0x7f4de8150490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:00:16.711 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:16.705+0000 7f4ded79f700 1 -- 172.21.15.59:0/1619216594 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f4de80b5d00 msgr2=0x7f4de8150490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:00:16.711 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:16.705+0000 7f4ded79f700 1 --2- 172.21.15.59:0/1619216594 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f4de80b5d00 0x7f4de8150490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:00:16.711 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:16.705+0000 7f4ded79f700 1 -- 172.21.15.59:0/1619216594 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4de806b9e0 msgr2=0x7f4de8151400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:00:16.711 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:16.705+0000 7f4ded79f700 1 --2- 172.21.15.59:0/1619216594 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4de806b9e0 0x7f4de8151400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:00:16.711 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:16.705+0000 7f4ded79f700 1 -- 172.21.15.59:0/1619216594 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4dd80094a0 con 0x7f4de80af5e0 2023-11-24T08:00:16.711 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:16.705+0000 7f4ded79f700 1 --2- 172.21.15.59:0/1619216594 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4de80af5e0 0x7f4de814ff50 secure :-1 s=READY pgs=1086 cs=0 l=1 rev1=1 crypto rx=0x7f4de400c550 tx=0x7f4de400c860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:00:16.711 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:16.705+0000 7f4dde7fc700 1 -- 172.21.15.59:0/1619216594 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4de4004c80 con 0x7f4de80af5e0 2023-11-24T08:00:16.711 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:16.706+0000 7f4dde7fc700 1 -- 172.21.15.59:0/1619216594 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4de4004de0 con 0x7f4de80af5e0 2023-11-24T08:00:16.711 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:16.706+0000 7f4deefa2700 1 -- 172.21.15.59:0/1619216594 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4de80c42a0 con 0x7f4de80af5e0 2023-11-24T08:00:16.711 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:16.706+0000 7f4dde7fc700 1 -- 172.21.15.59:0/1619216594 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4de4007310 con 0x7f4de80af5e0 2023-11-24T08:00:16.712 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:16.706+0000 7f4deefa2700 1 -- 172.21.15.59:0/1619216594 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4de80c4860 con 0x7f4de80af5e0 2023-11-24T08:00:16.712 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:16.706+0000 7f4deefa2700 1 -- 172.21.15.59:0/1619216594 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4de8075300 con 0x7f4de80af5e0 2023-11-24T08:00:16.712 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:16.707+0000 7f4dde7fc700 1 -- 172.21.15.59:0/1619216594 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 12) v1 ==== 89792+0+0 (secure 0 0 0) 0x7f4de4007470 con 0x7f4de80af5e0 2023-11-24T08:00:16.712 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:16.707+0000 7f4dde7fc700 1 --2- 172.21.15.59:0/1619216594 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f4dcc06b500 0x7f4dcc06d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:00:16.712 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:16.708+0000 7f4dde7fc700 1 -- 172.21.15.59:0/1619216594 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f4de4089db0 con 0x7f4de80af5e0 2023-11-24T08:00:16.712 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:16.708+0000 7f4dedfa0700 1 --2- 172.21.15.59:0/1619216594 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f4dcc06b500 0x7f4dcc06d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:00:16.712 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:16.708+0000 7f4dedfa0700 1 --2- 172.21.15.59:0/1619216594 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f4dcc06b500 0x7f4dcc06d9c0 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7f4dd80090b0 tx=0x7f4dd8008030 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:00:16.714 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:16.712+0000 7f4dde7fc700 1 -- 172.21.15.59:0/1619216594 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f4de4054520 con 0x7f4de80af5e0 2023-11-24T08:00:16.933 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:16.931+0000 7f4deefa2700 1 -- 172.21.15.59:0/1619216594 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f4de80c4d10 con 0x7f4de80af5e0 2023-11-24T08:00:16.933 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:16.931+0000 7f4dde7fc700 1 -- 172.21.15.59:0/1619216594 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v87) v1 ==== 80+0+0 (secure 0 0 0) 0x7f4de4057b40 con 0x7f4de80af5e0 2023-11-24T08:00:16.936 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:16.933+0000 7f4deefa2700 1 -- 172.21.15.59:0/1619216594 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f4dcc06b500 msgr2=0x7f4dcc06d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:00:16.936 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:16.933+0000 7f4deefa2700 1 --2- 172.21.15.59:0/1619216594 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f4dcc06b500 0x7f4dcc06d9c0 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7f4dd80090b0 tx=0x7f4dd8008030 comp rx=0 tx=0).stop 2023-11-24T08:00:16.936 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:16.933+0000 7f4deefa2700 1 -- 172.21.15.59:0/1619216594 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4de80af5e0 msgr2=0x7f4de814ff50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:00:16.936 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:16.933+0000 7f4deefa2700 1 --2- 172.21.15.59:0/1619216594 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4de80af5e0 0x7f4de814ff50 secure :-1 s=READY pgs=1086 cs=0 l=1 rev1=1 crypto rx=0x7f4de400c550 tx=0x7f4de400c860 comp rx=0 tx=0).stop 2023-11-24T08:00:16.936 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:16.934+0000 7f4deefa2700 1 -- 172.21.15.59:0/1619216594 shutdown_connections 2023-11-24T08:00:16.936 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:16.934+0000 7f4deefa2700 1 --2- 172.21.15.59:0/1619216594 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f4dcc06b500 0x7f4dcc06d9c0 unknown :-1 s=CLOSED pgs=498 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:00:16.936 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:16.934+0000 7f4deefa2700 1 --2- 172.21.15.59:0/1619216594 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f4de80b5d00 0x7f4de8150490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:00:16.937 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:16.934+0000 7f4deefa2700 1 --2- 172.21.15.59:0/1619216594 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4de806b9e0 0x7f4de8151400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:00:16.937 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:16.934+0000 7f4deefa2700 1 --2- 172.21.15.59:0/1619216594 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4de80af5e0 0x7f4de814ff50 unknown :-1 s=CLOSED pgs=1086 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:00:16.937 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:16.934+0000 7f4deefa2700 1 -- 172.21.15.59:0/1619216594 >> 172.21.15.59:0/1619216594 conn(0x7f4de8072410 msgr2=0x7f4de80b1190 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:00:16.937 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:16.934+0000 7f4deefa2700 1 -- 172.21.15.59:0/1619216594 shutdown_connections 2023-11-24T08:00:16.937 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:16.934+0000 7f4deefa2700 1 -- 172.21.15.59:0/1619216594 wait complete. 2023-11-24T08:00:16.937 INFO:teuthology.orchestra.run.smithi059.stderr:listed 0 entries 2023-11-24T08:00:16.947 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.1... 2023-11-24T08:00:16.948 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T08:00:16.948 DEBUG:teuthology.orchestra.run.smithi076:> sudo umount /home/ubuntu/cephtest/mnt.1 2023-11-24T08:00:16.996 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi076.front.sepia.ceph.com 2023-11-24T08:00:16.996 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T08:00:16.996 DEBUG:teuthology.orchestra.run.smithi076:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2023-11-24T08:00:17.016 DEBUG:teuthology.orchestra.run.smithi059:> set -ex 2023-11-24T08:00:17.017 DEBUG:teuthology.orchestra.run.smithi059:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-24T08:00:17.035 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=16287364k,nr_inodes=4071841,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 tmpfs /sys/fs/cgroup tmpfs ro,seclabel,nosuid,nodev,noexec,mode=755 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,seclabel,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,seclabel,nosuid,nodev,noexec,relatime,hugetlb 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,seclabel,nosuid,nodev,noexec,relatime,perf_event 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,seclabel,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/rdma cgroup rw,seclabel,nosuid,nodev,noexec,relatime,rdma 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,seclabel,nosuid,nodev,noexec,relatime,freezer 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,seclabel,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,seclabel,nosuid,nodev,noexec,relatime,devices 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,blkio 0 0 none /sys/kernel/tracing tracefs rw,seclabel,relatime 0 0 configfs /sys/kernel/config configfs rw,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=42,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=16075 0 0 mqueue /dev/mqueue mqueue rw,seclabel,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,relatime 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3261172k,mode=700,uid=1000,gid=1265 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw,seclabel 0 0 2023-11-24T08:00:17.035 DEBUG:teuthology.orchestra.run.smithi076:> set -ex 2023-11-24T08:00:17.035 DEBUG:teuthology.orchestra.run.smithi076:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-24T08:00:17.077 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=16284272k,nr_inodes=4071068,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 tmpfs /sys/fs/cgroup tmpfs ro,seclabel,nosuid,nodev,noexec,mode=755 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,seclabel,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,seclabel,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,seclabel,nosuid,nodev,noexec,relatime,freezer 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,seclabel,nosuid,nodev,noexec,relatime,perf_event 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,seclabel,nosuid,nodev,noexec,relatime,hugetlb 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,seclabel,nosuid,nodev,noexec,relatime,devices 0 0 cgroup /sys/fs/cgroup/rdma cgroup rw,seclabel,nosuid,nodev,noexec,relatime,rdma 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,seclabel,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,blkio 0 0 none /sys/kernel/tracing tracefs rw,seclabel,relatime 0 0 configfs /sys/kernel/config configfs rw,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=35,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=20116 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,relatime 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3260552k,mode=700,uid=1000,gid=1265 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw,seclabel 0 0 2023-11-24T08:00:17.078 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-24T08:00:17.202 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:17.201+0000 7fd09b1d2700 1 Processor -- start 2023-11-24T08:00:17.203 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:17.201+0000 7fd09b1d2700 1 -- start start 2023-11-24T08:00:17.210 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:17.202+0000 7fd09b1d2700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fd09415bde0 0x7fd09415c1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:00:17.210 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:17.202+0000 7fd09b1d2700 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fd0940cf2b0 0x7fd0940d01b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:00:17.210 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:17.202+0000 7fd09b1d2700 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd0940d0a90 con 0x7fd0940ce5c0 2023-11-24T08:00:17.210 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:17.202+0000 7fd09b1d2700 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd09406ee10 con 0x7fd09415bde0 2023-11-24T08:00:17.210 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:17.202+0000 7fd09b1d2700 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd09406f0f0 con 0x7fd0940cf2b0 2023-11-24T08:00:17.210 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:17.202+0000 7fd098f6e700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fd09415bde0 0x7fd09415c1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:00:17.210 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:17.202+0000 7fd09976f700 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7fd0940ce5c0 0x7fd0940cc670 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:59736/0 (socket says 172.21.15.59:59736) 2023-11-24T08:00:17.210 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:17.202+0000 7fd09976f700 1 -- 172.21.15.59:0/2380856627 learned_addr learned my addr 172.21.15.59:0/2380856627 (peer_addr_for_me v1:172.21.15.59:0/0) 2023-11-24T08:00:17.210 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:17.202+0000 7fd093fff700 1 --2- 172.21.15.59:0/2380856627 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fd0940cf2b0 0x7fd0940d01b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:00:17.211 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:17.202+0000 7fd098f6e700 1 -- 172.21.15.59:0/2380856627 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fd0940cf2b0 msgr2=0x7fd0940d01b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:00:17.211 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:17.202+0000 7fd098f6e700 1 --2- 172.21.15.59:0/2380856627 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fd0940cf2b0 0x7fd0940d01b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:00:17.211 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:17.202+0000 7fd098f6e700 1 -- 172.21.15.59:0/2380856627 >> v1:172.21.15.59:6789/0 conn(0x7fd0940ce5c0 legacy=0x7fd0940cc670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:00:17.211 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:17.202+0000 7fd098f6e700 1 -- 172.21.15.59:0/2380856627 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd09406f3d0 con 0x7fd09415bde0 2023-11-24T08:00:17.211 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:17.202+0000 7fd0937fe700 1 -- 172.21.15.59:0/2380856627 <== mon.0 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1966736826 0 0) 0x7fd0940d0a90 con 0x7fd0940ce5c0 2023-11-24T08:00:17.211 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:17.202+0000 7fd093fff700 1 --2- 172.21.15.59:0/2380856627 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fd0940cf2b0 0x7fd0940d01b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-24T08:00:17.211 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:17.202+0000 7fd098f6e700 1 --2- 172.21.15.59:0/2380856627 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd09415bde0 0x7fd09415c1c0 secure :-1 s=READY pgs=1088 cs=0 l=1 rev1=1 crypto rx=0x7fd088009840 tx=0x7fd088009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4f17beaa036f72de server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:00:17.211 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:17.203+0000 7fd0937fe700 1 -- 172.21.15.59:0/2380856627 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd088011830 con 0x7fd09415bde0 2023-11-24T08:00:17.211 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:17.203+0000 7fd0937fe700 1 -- 172.21.15.59:0/2380856627 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd088011990 con 0x7fd09415bde0 2023-11-24T08:00:17.211 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:17.203+0000 7fd0937fe700 1 -- 172.21.15.59:0/2380856627 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd088011d20 con 0x7fd09415bde0 2023-11-24T08:00:17.211 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:17.203+0000 7fd09b1d2700 1 -- 172.21.15.59:0/2380856627 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd09415bde0 msgr2=0x7fd09415c1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:00:17.211 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:17.203+0000 7fd09b1d2700 1 --2- 172.21.15.59:0/2380856627 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd09415bde0 0x7fd09415c1c0 secure :-1 s=READY pgs=1088 cs=0 l=1 rev1=1 crypto rx=0x7fd088009840 tx=0x7fd088009b50 comp rx=0 tx=0).stop 2023-11-24T08:00:17.212 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:17.203+0000 7fd09b1d2700 1 -- 172.21.15.59:0/2380856627 shutdown_connections 2023-11-24T08:00:17.212 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:17.203+0000 7fd09b1d2700 1 --2- 172.21.15.59:0/2380856627 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd09415bde0 0x7fd09415c1c0 unknown :-1 s=CLOSED pgs=1088 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:00:17.212 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:17.203+0000 7fd09b1d2700 1 --2- 172.21.15.59:0/2380856627 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fd0940cf2b0 0x7fd0940d01b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:00:17.212 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:17.203+0000 7fd09b1d2700 1 -- 172.21.15.59:0/2380856627 >> 172.21.15.59:0/2380856627 conn(0x7fd0940b3720 msgr2=0x7fd0940b3b30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:00:17.212 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:17.203+0000 7fd09b1d2700 1 -- 172.21.15.59:0/2380856627 shutdown_connections 2023-11-24T08:00:17.212 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:17.203+0000 7fd09b1d2700 1 -- 172.21.15.59:0/2380856627 wait complete. 2023-11-24T08:00:17.212 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:17.204+0000 7fd09b1d2700 1 Processor -- start 2023-11-24T08:00:17.212 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:17.204+0000 7fd09b1d2700 1 -- start start 2023-11-24T08:00:17.212 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:17.205+0000 7fd09b1d2700 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fd0940ce5c0 0x7fd094158a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:00:17.212 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:17.205+0000 7fd09b1d2700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd0940cf2b0 0x7fd09409f540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:00:17.212 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:17.205+0000 7fd09b1d2700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd09415bde0 0x7fd0940c8ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:00:17.212 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:17.205+0000 7fd09b1d2700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd09415df00 con 0x7fd0940cf2b0 2023-11-24T08:00:17.213 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:17.205+0000 7fd09b1d2700 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd09415e1e0 con 0x7fd09415bde0 2023-11-24T08:00:17.213 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:17.205+0000 7fd09b1d2700 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd09415e4c0 con 0x7fd0940ce5c0 2023-11-24T08:00:17.213 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:17.205+0000 7fd093fff700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd0940cf2b0 0x7fd09409f540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:00:17.213 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:17.205+0000 7fd093fff700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd0940cf2b0 0x7fd09409f540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:56266/0 (socket says 172.21.15.59:56266) 2023-11-24T08:00:17.213 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:17.205+0000 7fd093fff700 1 -- 172.21.15.59:0/881402912 learned_addr learned my addr 172.21.15.59:0/881402912 (peer_addr_for_me v2:172.21.15.59:0/0) 2023-11-24T08:00:17.213 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:17.206+0000 7fd098f6e700 1 --2- 172.21.15.59:0/881402912 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fd0940ce5c0 0x7fd094158a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:00:17.213 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:17.206+0000 7fd09976f700 1 --2- 172.21.15.59:0/881402912 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd09415bde0 0x7fd0940c8ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:00:17.213 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:17.206+0000 7fd093fff700 1 -- 172.21.15.59:0/881402912 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fd0940ce5c0 msgr2=0x7fd094158a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:00:17.213 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:17.206+0000 7fd093fff700 1 --2- 172.21.15.59:0/881402912 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fd0940ce5c0 0x7fd094158a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:00:17.213 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:17.206+0000 7fd093fff700 1 -- 172.21.15.59:0/881402912 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd09415bde0 msgr2=0x7fd0940c8ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:00:17.213 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:17.206+0000 7fd093fff700 1 --2- 172.21.15.59:0/881402912 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd09415bde0 0x7fd0940c8ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:00:17.213 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:17.206+0000 7fd093fff700 1 -- 172.21.15.59:0/881402912 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd0880094a0 con 0x7fd0940cf2b0 2023-11-24T08:00:17.213 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:17.206+0000 7fd09976f700 1 --2- 172.21.15.59:0/881402912 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd09415bde0 0x7fd0940c8ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T08:00:17.214 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:17.206+0000 7fd093fff700 1 --2- 172.21.15.59:0/881402912 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd0940cf2b0 0x7fd09409f540 secure :-1 s=READY pgs=1089 cs=0 l=1 rev1=1 crypto rx=0x7fd080009fd0 tx=0x7fd08000eb70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:00:17.214 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:17.206+0000 7fd0917fa700 1 -- 172.21.15.59:0/881402912 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd080013610 con 0x7fd0940cf2b0 2023-11-24T08:00:17.214 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:17.206+0000 7fd0917fa700 1 -- 172.21.15.59:0/881402912 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd080013770 con 0x7fd0940cf2b0 2023-11-24T08:00:17.214 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:17.206+0000 7fd0917fa700 1 -- 172.21.15.59:0/881402912 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd080013b20 con 0x7fd0940cf2b0 2023-11-24T08:00:17.214 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:17.206+0000 7fd09b1d2700 1 -- 172.21.15.59:0/881402912 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd09415e7a0 con 0x7fd0940cf2b0 2023-11-24T08:00:17.214 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:17.206+0000 7fd09b1d2700 1 -- 172.21.15.59:0/881402912 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd094158fa0 con 0x7fd0940cf2b0 2023-11-24T08:00:17.214 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:17.207+0000 7fd09b1d2700 1 -- 172.21.15.59:0/881402912 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd094068670 con 0x7fd0940cf2b0 2023-11-24T08:00:17.214 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:17.208+0000 7fd0917fa700 1 -- 172.21.15.59:0/881402912 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 12) v1 ==== 89792+0+0 (secure 0 0 0) 0x7fd080013c80 con 0x7fd0940cf2b0 2023-11-24T08:00:17.214 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:17.208+0000 7fd0917fa700 1 --2- 172.21.15.59:0/881402912 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7fd07806b500 0x7fd07806d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:00:17.214 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:17.208+0000 7fd0917fa700 1 -- 172.21.15.59:0/881402912 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fd080089e00 con 0x7fd0940cf2b0 2023-11-24T08:00:17.214 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:17.209+0000 7fd098f6e700 1 --2- 172.21.15.59:0/881402912 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7fd07806b500 0x7fd07806d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:00:17.215 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:17.209+0000 7fd098f6e700 1 --2- 172.21.15.59:0/881402912 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7fd07806b500 0x7fd07806d9c0 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7fd0880090b0 tx=0x7fd08801c040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:00:17.215 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:17.213+0000 7fd0917fa700 1 -- 172.21.15.59:0/881402912 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fd0800544f0 con 0x7fd0940cf2b0 2023-11-24T08:00:17.463 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:17.459+0000 7fd09b1d2700 1 -- 172.21.15.59:0/881402912 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fd09415b390 con 0x7fd0940cf2b0 2023-11-24T08:00:17.464 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:17.460+0000 7fd0917fa700 1 -- 172.21.15.59:0/881402912 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 54 v54) v1 ==== 94+0+4912 (secure 0 0 0) 0x7fd080057b10 con 0x7fd0940cf2b0 2023-11-24T08:00:17.464 INFO:teuthology.orchestra.run.smithi059.stdout: 2023-11-24T08:00:17.464 INFO:teuthology.orchestra.run.smithi059.stdout:{"epoch":54,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5876,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.76:6837/1132179319","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6835","nonce":1132179319},{"type":"v1","addr":"172.21.15.76:6837","nonce":1132179319}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":48},{"gid":6487,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.76:6836/453627414","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6834","nonce":453627414},{"type":"v1","addr":"172.21.15.76:6836","nonce":453627414}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":48},{"gid":7035,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6834/4160100816","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6832","nonce":4160100816},{"type":"v1","addr":"172.21.15.59:6834","nonce":4160100816}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":48}],"filesystems":[{"mdsmap":{"epoch":53,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-24T07:59:51.909592+0000","modified":"2023-11-24T07:59:54.948256+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":7038},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7038":{"gid":7038,"name":"c","rank":0,"incarnation":49,"state":"up:active","state_seq":3,"addr":"172.21.15.59:6835/2337335405","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6833","nonce":2337335405},{"type":"v1","addr":"172.21.15.59:6835","nonce":2337335405}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":6}]} 2023-11-24T08:00:17.465 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:17.462+0000 7fd09b1d2700 1 -- 172.21.15.59:0/881402912 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7fd07806b500 msgr2=0x7fd07806d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:00:17.465 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:17.462+0000 7fd09b1d2700 1 --2- 172.21.15.59:0/881402912 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7fd07806b500 0x7fd07806d9c0 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7fd0880090b0 tx=0x7fd08801c040 comp rx=0 tx=0).stop 2023-11-24T08:00:17.465 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:17.462+0000 7fd09b1d2700 1 -- 172.21.15.59:0/881402912 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd0940cf2b0 msgr2=0x7fd09409f540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:00:17.465 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:17.462+0000 7fd09b1d2700 1 --2- 172.21.15.59:0/881402912 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd0940cf2b0 0x7fd09409f540 secure :-1 s=READY pgs=1089 cs=0 l=1 rev1=1 crypto rx=0x7fd080009fd0 tx=0x7fd08000eb70 comp rx=0 tx=0).stop 2023-11-24T08:00:17.465 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:17.462+0000 7fd09b1d2700 1 -- 172.21.15.59:0/881402912 shutdown_connections 2023-11-24T08:00:17.465 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:17.462+0000 7fd09b1d2700 1 --2- 172.21.15.59:0/881402912 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7fd07806b500 0x7fd07806d9c0 unknown :-1 s=CLOSED pgs=499 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:00:17.466 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:17.462+0000 7fd09b1d2700 1 --2- 172.21.15.59:0/881402912 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd09415bde0 0x7fd0940c8ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:00:17.466 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:17.462+0000 7fd09b1d2700 1 --2- 172.21.15.59:0/881402912 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fd0940ce5c0 0x7fd094158a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:00:17.466 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:17.462+0000 7fd09b1d2700 1 --2- 172.21.15.59:0/881402912 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd0940cf2b0 0x7fd09409f540 unknown :-1 s=CLOSED pgs=1089 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:00:17.466 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:17.462+0000 7fd09b1d2700 1 -- 172.21.15.59:0/881402912 >> 172.21.15.59:0/881402912 conn(0x7fd0940b3720 msgr2=0x7fd09415a110 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:00:17.466 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:17.462+0000 7fd09b1d2700 1 -- 172.21.15.59:0/881402912 shutdown_connections 2023-11-24T08:00:17.466 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:17.462+0000 7fd09b1d2700 1 -- 172.21.15.59:0/881402912 wait complete. 2023-11-24T08:00:17.466 INFO:teuthology.orchestra.run.smithi059.stderr:dumped fsmap epoch 54 2023-11-24T08:00:17.477 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-24T08:00:17.602 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:17.601+0000 7f8a6901d700 1 Processor -- start 2023-11-24T08:00:17.603 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:17.601+0000 7f8a6901d700 1 -- start start 2023-11-24T08:00:17.609 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:17.602+0000 7f8a6901d700 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f8a6406b620 0x7f8a640b3a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:00:17.610 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:17.602+0000 7f8a6901d700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f8a640b5ab0 0x7f8a640b3f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:00:17.610 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:17.602+0000 7f8a6901d700 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8a64071f50 con 0x7f8a641478c0 2023-11-24T08:00:17.610 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:17.602+0000 7f8a6901d700 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8a64072230 con 0x7f8a640b5ab0 2023-11-24T08:00:17.610 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:17.602+0000 7f8a6901d700 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8a64072510 con 0x7f8a6406b620 2023-11-24T08:00:17.610 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:17.602+0000 7f8a6259c700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f8a640b5ab0 0x7f8a640b3f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:00:17.610 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:17.602+0000 7f8a6359e700 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7f8a641478c0 0x7f8a64161ba0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:59750/0 (socket says 172.21.15.59:59750) 2023-11-24T08:00:17.610 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:17.602+0000 7f8a6359e700 1 -- 172.21.15.59:0/3120743326 learned_addr learned my addr 172.21.15.59:0/3120743326 (peer_addr_for_me v1:172.21.15.59:0/0) 2023-11-24T08:00:17.610 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:17.602+0000 7f8a62d9d700 1 --2- 172.21.15.59:0/3120743326 >> v2:172.21.15.76:3300/0 conn(0x7f8a6406b620 0x7f8a640b3a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:00:17.610 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:17.602+0000 7f8a6259c700 1 -- 172.21.15.59:0/3120743326 >> v2:172.21.15.76:3300/0 conn(0x7f8a6406b620 msgr2=0x7f8a640b3a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:00:17.610 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:17.602+0000 7f8a6259c700 1 --2- 172.21.15.59:0/3120743326 >> v2:172.21.15.76:3300/0 conn(0x7f8a6406b620 0x7f8a640b3a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:00:17.610 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:17.602+0000 7f8a6259c700 1 -- 172.21.15.59:0/3120743326 >> v1:172.21.15.59:6789/0 conn(0x7f8a641478c0 legacy=0x7f8a64161ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:00:17.611 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:17.602+0000 7f8a61d9b700 1 -- 172.21.15.59:0/3120743326 <== mon.0 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2595821221 0 0) 0x7f8a64071f50 con 0x7f8a641478c0 2023-11-24T08:00:17.611 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:17.602+0000 7f8a6259c700 1 -- 172.21.15.59:0/3120743326 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8a640727f0 con 0x7f8a640b5ab0 2023-11-24T08:00:17.611 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:17.602+0000 7f8a6259c700 1 --2- 172.21.15.59:0/3120743326 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8a640b5ab0 0x7f8a640b3f80 secure :-1 s=READY pgs=1091 cs=0 l=1 rev1=1 crypto rx=0x7f8a6406d2e0 tx=0x7f8a58009b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fa41acb472601c83 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:00:17.611 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:17.602+0000 7f8a61d9b700 1 -- 172.21.15.59:0/3120743326 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a58011830 con 0x7f8a640b5ab0 2023-11-24T08:00:17.611 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:17.602+0000 7f8a61d9b700 1 -- 172.21.15.59:0/3120743326 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8a58011990 con 0x7f8a640b5ab0 2023-11-24T08:00:17.611 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:17.602+0000 7f8a61d9b700 1 -- 172.21.15.59:0/3120743326 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a58011d20 con 0x7f8a640b5ab0 2023-11-24T08:00:17.611 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:17.603+0000 7f8a6901d700 1 -- 172.21.15.59:0/3120743326 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8a640b5ab0 msgr2=0x7f8a640b3f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:00:17.611 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:17.603+0000 7f8a6901d700 1 --2- 172.21.15.59:0/3120743326 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8a640b5ab0 0x7f8a640b3f80 secure :-1 s=READY pgs=1091 cs=0 l=1 rev1=1 crypto rx=0x7f8a6406d2e0 tx=0x7f8a58009b20 comp rx=0 tx=0).stop 2023-11-24T08:00:17.611 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:17.603+0000 7f8a6901d700 1 -- 172.21.15.59:0/3120743326 shutdown_connections 2023-11-24T08:00:17.612 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:17.603+0000 7f8a6901d700 1 --2- 172.21.15.59:0/3120743326 >> v2:172.21.15.76:3300/0 conn(0x7f8a6406b620 0x7f8a640b3a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:00:17.612 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:17.603+0000 7f8a6901d700 1 --2- 172.21.15.59:0/3120743326 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8a640b5ab0 0x7f8a640b3f80 unknown :-1 s=CLOSED pgs=1091 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:00:17.612 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:17.603+0000 7f8a6901d700 1 -- 172.21.15.59:0/3120743326 >> 172.21.15.59:0/3120743326 conn(0x7f8a640b2740 msgr2=0x7f8a640b2b20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:00:17.612 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:17.603+0000 7f8a6901d700 1 -- 172.21.15.59:0/3120743326 shutdown_connections 2023-11-24T08:00:17.612 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:17.603+0000 7f8a6901d700 1 -- 172.21.15.59:0/3120743326 wait complete. 2023-11-24T08:00:17.612 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:17.605+0000 7f8a6901d700 1 Processor -- start 2023-11-24T08:00:17.612 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:17.605+0000 7f8a6901d700 1 -- start start 2023-11-24T08:00:17.612 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:17.605+0000 7f8a6901d700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8a6406b620 0x7f8a641e5260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:00:17.612 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:17.605+0000 7f8a6901d700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8a640b5ab0 0x7f8a641f7330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:00:17.612 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:17.605+0000 7f8a6901d700 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f8a641478c0 0x7f8a641f7870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:00:17.613 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:17.605+0000 7f8a6901d700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a640c72d0 con 0x7f8a6406b620 2023-11-24T08:00:17.613 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:17.605+0000 7f8a6901d700 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a640c66c0 con 0x7f8a640b5ab0 2023-11-24T08:00:17.613 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:17.605+0000 7f8a6901d700 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8a640c69a0 con 0x7f8a641478c0 2023-11-24T08:00:17.613 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:17.605+0000 7f8a62d9d700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8a6406b620 0x7f8a641e5260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:00:17.613 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:17.605+0000 7f8a62d9d700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8a6406b620 0x7f8a641e5260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:56278/0 (socket says 172.21.15.59:56278) 2023-11-24T08:00:17.613 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:17.605+0000 7f8a62d9d700 1 -- 172.21.15.59:0/3858164142 learned_addr learned my addr 172.21.15.59:0/3858164142 (peer_addr_for_me v2:172.21.15.59:0/0) 2023-11-24T08:00:17.613 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:17.605+0000 7f8a6259c700 1 --2- 172.21.15.59:0/3858164142 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8a640b5ab0 0x7f8a641f7330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:00:17.613 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:17.605+0000 7f8a6359e700 1 --2- 172.21.15.59:0/3858164142 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f8a641478c0 0x7f8a641f7870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:00:17.613 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:17.606+0000 7f8a62d9d700 1 -- 172.21.15.59:0/3858164142 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f8a641478c0 msgr2=0x7f8a641f7870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:00:17.613 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:17.606+0000 7f8a62d9d700 1 --2- 172.21.15.59:0/3858164142 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f8a641478c0 0x7f8a641f7870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:00:17.613 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:17.606+0000 7f8a62d9d700 1 -- 172.21.15.59:0/3858164142 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8a640b5ab0 msgr2=0x7f8a641f7330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:00:17.614 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:17.606+0000 7f8a62d9d700 1 --2- 172.21.15.59:0/3858164142 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8a640b5ab0 0x7f8a641f7330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:00:17.614 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:17.606+0000 7f8a62d9d700 1 -- 172.21.15.59:0/3858164142 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8a580094a0 con 0x7f8a6406b620 2023-11-24T08:00:17.614 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:17.606+0000 7f8a62d9d700 1 --2- 172.21.15.59:0/3858164142 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8a6406b620 0x7f8a641e5260 secure :-1 s=READY pgs=1092 cs=0 l=1 rev1=1 crypto rx=0x7f8a5400e810 tx=0x7f8a5400eb20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:00:17.614 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:17.606+0000 7f8a4b7fe700 1 -- 172.21.15.59:0/3858164142 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a5400c070 con 0x7f8a6406b620 2023-11-24T08:00:17.614 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:17.606+0000 7f8a4b7fe700 1 -- 172.21.15.59:0/3858164142 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8a54013610 con 0x7f8a6406b620 2023-11-24T08:00:17.614 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:17.606+0000 7f8a6901d700 1 -- 172.21.15.59:0/3858164142 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8a640c6c80 con 0x7f8a6406b620 2023-11-24T08:00:17.615 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:17.606+0000 7f8a4b7fe700 1 -- 172.21.15.59:0/3858164142 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a540139c0 con 0x7f8a6406b620 2023-11-24T08:00:17.615 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:17.606+0000 7f8a6901d700 1 -- 172.21.15.59:0/3858164142 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8a641e57a0 con 0x7f8a6406b620 2023-11-24T08:00:17.615 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:17.607+0000 7f8a6901d700 1 -- 172.21.15.59:0/3858164142 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8a64068670 con 0x7f8a6406b620 2023-11-24T08:00:17.615 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:17.608+0000 7f8a4b7fe700 1 -- 172.21.15.59:0/3858164142 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 12) v1 ==== 89792+0+0 (secure 0 0 0) 0x7f8a54013b20 con 0x7f8a6406b620 2023-11-24T08:00:17.615 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:17.608+0000 7f8a4b7fe700 1 --2- 172.21.15.59:0/3858164142 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f8a5006b500 0x7f8a5006d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:00:17.615 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:17.608+0000 7f8a4b7fe700 1 -- 172.21.15.59:0/3858164142 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f8a54089f80 con 0x7f8a6406b620 2023-11-24T08:00:17.615 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:17.608+0000 7f8a6259c700 1 --2- 172.21.15.59:0/3858164142 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f8a5006b500 0x7f8a5006d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:00:17.616 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:17.609+0000 7f8a6259c700 1 --2- 172.21.15.59:0/3858164142 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f8a5006b500 0x7f8a5006d9c0 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7f8a640b5f10 tx=0x7f8a58008030 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:00:17.616 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:17.612+0000 7f8a4b7fe700 1 -- 172.21.15.59:0/3858164142 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f8a540545f0 con 0x7f8a6406b620 2023-11-24T08:00:17.814 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:17.812+0000 7f8a6901d700 1 -- 172.21.15.59:0/3858164142 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f8a640b42a0 con 0x7f8a6406b620 2023-11-24T08:00:17.815 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:17.813+0000 7f8a4b7fe700 1 -- 172.21.15.59:0/3858164142 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v87) v1 ==== 74+0+16500 (secure 0 0 0) 0x7f8a54057c10 con 0x7f8a6406b620 2023-11-24T08:00:17.815 INFO:teuthology.orchestra.run.smithi059.stdout: 2023-11-24T08:00:17.815 INFO:teuthology.orchestra.run.smithi059.stdout:{"epoch":87,"fsid":"9f0e8741-b8f9-48d5-a2c3-09f45903303d","created":"2023-11-24T07:56:06.552729+0000","modified":"2023-11-24T07:59:53.500645+0000","last_up_change":"2023-11-24T07:56:17.372352+0000","last_in_change":"2023-11-24T07:56:09.073250+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":13,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-24T07:56:18.352823+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_metadata","create_time":"2023-11-24T07:59:50.274996+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_data","create_time":"2023-11-24T07:59:50.873919+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"87","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"6d8f9e9b-1782-4e41-928b-a39f2a8ebc06","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6800","nonce":3373878772},{"type":"v1","addr":"172.21.15.59:6801","nonce":3373878772}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6802","nonce":3373878772},{"type":"v1","addr":"172.21.15.59:6803","nonce":3373878772}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6806","nonce":3373878772},{"type":"v1","addr":"172.21.15.59:6807","nonce":3373878772}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6804","nonce":3373878772},{"type":"v1","addr":"172.21.15.59:6805","nonce":3373878772}]},"public_addr":"172.21.15.59:6801/3373878772","cluster_addr":"172.21.15.59:6803/3373878772","heartbeat_back_addr":"172.21.15.59:6807/3373878772","heartbeat_front_addr":"172.21.15.59:6805/3373878772","state":["exists","up"]},{"osd":1,"uuid":"e8d882aa-52e1-4c04-9e1e-c44ef9204a5f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6824","nonce":648258563},{"type":"v1","addr":"172.21.15.59:6825","nonce":648258563}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6826","nonce":648258563},{"type":"v1","addr":"172.21.15.59:6827","nonce":648258563}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6830","nonce":648258563},{"type":"v1","addr":"172.21.15.59:6831","nonce":648258563}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6828","nonce":648258563},{"type":"v1","addr":"172.21.15.59:6829","nonce":648258563}]},"public_addr":"172.21.15.59:6825/648258563","cluster_addr":"172.21.15.59:6827/648258563","heartbeat_back_addr":"172.21.15.59:6831/648258563","heartbeat_front_addr":"172.21.15.59:6829/648258563","state":["exists","up"]},{"osd":2,"uuid":"69eebfa0-695b-4b21-90b9-8a3537168a01","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6808","nonce":1840430654},{"type":"v1","addr":"172.21.15.59:6809","nonce":1840430654}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6810","nonce":1840430654},{"type":"v1","addr":"172.21.15.59:6811","nonce":1840430654}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6814","nonce":1840430654},{"type":"v1","addr":"172.21.15.59:6815","nonce":1840430654}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6812","nonce":1840430654},{"type":"v1","addr":"172.21.15.59:6813","nonce":1840430654}]},"public_addr":"172.21.15.59:6809/1840430654","cluster_addr":"172.21.15.59:6811/1840430654","heartbeat_back_addr":"172.21.15.59:6815/1840430654","heartbeat_front_addr":"172.21.15.59:6813/1840430654","state":["exists","up"]},{"osd":3,"uuid":"96fd77fc-f911-46ff-9445-6f0be2daa8f4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6816","nonce":2504025850},{"type":"v1","addr":"172.21.15.59:6817","nonce":2504025850}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6818","nonce":2504025850},{"type":"v1","addr":"172.21.15.59:6819","nonce":2504025850}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6822","nonce":2504025850},{"type":"v1","addr":"172.21.15.59:6823","nonce":2504025850}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6820","nonce":2504025850},{"type":"v1","addr":"172.21.15.59:6821","nonce":2504025850}]},"public_addr":"172.21.15.59:6817/2504025850","cluster_addr":"172.21.15.59:6819/2504025850","heartbeat_back_addr":"172.21.15.59:6823/2504025850","heartbeat_front_addr":"172.21.15.59:6821/2504025850","state":["exists","up"]},{"osd":4,"uuid":"7ac29fca-b5b2-4c95-9d69-2f164cbbc708","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6816","nonce":2382463431},{"type":"v1","addr":"172.21.15.76:6818","nonce":2382463431}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6820","nonce":2382463431},{"type":"v1","addr":"172.21.15.76:6822","nonce":2382463431}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6828","nonce":2382463431},{"type":"v1","addr":"172.21.15.76:6830","nonce":2382463431}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6824","nonce":2382463431},{"type":"v1","addr":"172.21.15.76:6826","nonce":2382463431}]},"public_addr":"172.21.15.76:6818/2382463431","cluster_addr":"172.21.15.76:6822/2382463431","heartbeat_back_addr":"172.21.15.76:6830/2382463431","heartbeat_front_addr":"172.21.15.76:6826/2382463431","state":["exists","up"]},{"osd":5,"uuid":"485bf711-3851-4238-a4f4-51d52e0a9c58","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6800","nonce":329334308},{"type":"v1","addr":"172.21.15.76:6802","nonce":329334308}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6805","nonce":329334308},{"type":"v1","addr":"172.21.15.76:6807","nonce":329334308}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6814","nonce":329334308},{"type":"v1","addr":"172.21.15.76:6815","nonce":329334308}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6810","nonce":329334308},{"type":"v1","addr":"172.21.15.76:6812","nonce":329334308}]},"public_addr":"172.21.15.76:6802/329334308","cluster_addr":"172.21.15.76:6807/329334308","heartbeat_back_addr":"172.21.15.76:6815/329334308","heartbeat_front_addr":"172.21.15.76:6812/329334308","state":["exists","up"]},{"osd":6,"uuid":"72b3f2fc-1e4a-4e32-968d-003647a0286f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6801","nonce":3946579410},{"type":"v1","addr":"172.21.15.76:6803","nonce":3946579410}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6804","nonce":3946579410},{"type":"v1","addr":"172.21.15.76:6806","nonce":3946579410}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6811","nonce":3946579410},{"type":"v1","addr":"172.21.15.76:6813","nonce":3946579410}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6808","nonce":3946579410},{"type":"v1","addr":"172.21.15.76:6809","nonce":3946579410}]},"public_addr":"172.21.15.76:6803/3946579410","cluster_addr":"172.21.15.76:6806/3946579410","heartbeat_back_addr":"172.21.15.76:6813/3946579410","heartbeat_front_addr":"172.21.15.76:6809/3946579410","state":["exists","up"]},{"osd":7,"uuid":"1ddceb90-6c5d-4d1a-ad56-99cb877f8c70","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6817","nonce":841761538},{"type":"v1","addr":"172.21.15.76:6819","nonce":841761538}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6821","nonce":841761538},{"type":"v1","addr":"172.21.15.76:6823","nonce":841761538}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6829","nonce":841761538},{"type":"v1","addr":"172.21.15.76:6831","nonce":841761538}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6825","nonce":841761538},{"type":"v1","addr":"172.21.15.76:6827","nonce":841761538}]},"public_addr":"172.21.15.76:6819/841761538","cluster_addr":"172.21.15.76:6823/841761538","heartbeat_back_addr":"172.21.15.76:6831/841761538","heartbeat_front_addr":"172.21.15.76:6827/841761538","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T07:56:14.294838+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T07:56:14.270779+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T07:56:14.231809+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T07:56:14.260983+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T07:56:14.302997+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T07:56:14.258600+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T07:56:14.251413+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T07:56:14.304727+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-24T08:00:17.817 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:17.815+0000 7f8a6901d700 1 -- 172.21.15.59:0/3858164142 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f8a5006b500 msgr2=0x7f8a5006d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:00:17.817 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:17.815+0000 7f8a6901d700 1 --2- 172.21.15.59:0/3858164142 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f8a5006b500 0x7f8a5006d9c0 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7f8a640b5f10 tx=0x7f8a58008030 comp rx=0 tx=0).stop 2023-11-24T08:00:17.818 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:17.815+0000 7f8a6901d700 1 -- 172.21.15.59:0/3858164142 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8a6406b620 msgr2=0x7f8a641e5260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:00:17.818 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:17.815+0000 7f8a6901d700 1 --2- 172.21.15.59:0/3858164142 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8a6406b620 0x7f8a641e5260 secure :-1 s=READY pgs=1092 cs=0 l=1 rev1=1 crypto rx=0x7f8a5400e810 tx=0x7f8a5400eb20 comp rx=0 tx=0).stop 2023-11-24T08:00:17.818 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:17.816+0000 7f8a6901d700 1 -- 172.21.15.59:0/3858164142 shutdown_connections 2023-11-24T08:00:17.818 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:17.816+0000 7f8a6901d700 1 --2- 172.21.15.59:0/3858164142 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f8a5006b500 0x7f8a5006d9c0 unknown :-1 s=CLOSED pgs=500 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:00:17.818 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:17.816+0000 7f8a6901d700 1 --2- 172.21.15.59:0/3858164142 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f8a641478c0 0x7f8a641f7870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:00:17.818 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:17.816+0000 7f8a6901d700 1 --2- 172.21.15.59:0/3858164142 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8a6406b620 0x7f8a641e5260 unknown :-1 s=CLOSED pgs=1092 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:00:17.818 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:17.816+0000 7f8a6901d700 1 --2- 172.21.15.59:0/3858164142 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8a640b5ab0 0x7f8a641f7330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:00:17.818 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:17.816+0000 7f8a6901d700 1 -- 172.21.15.59:0/3858164142 >> 172.21.15.59:0/3858164142 conn(0x7f8a640b2740 msgr2=0x7f8a641464d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:00:17.818 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:17.816+0000 7f8a6901d700 1 -- 172.21.15.59:0/3858164142 shutdown_connections 2023-11-24T08:00:17.818 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:17.816+0000 7f8a6901d700 1 -- 172.21.15.59:0/3858164142 wait complete. 2023-11-24T08:00:17.830 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-24T07:56:18.352823+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 12, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-24T07:59:50.274996+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '86', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 13, 'pool_name': 'cephfs_data', 'create_time': '2023-11-24T07:59:50.873919+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '87', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-24T08:00:17.830 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-24T08:00:17.958 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:17.956+0000 7fdba4ca1700 1 Processor -- start 2023-11-24T08:00:17.959 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:17.956+0000 7fdba4ca1700 1 -- start start 2023-11-24T08:00:17.961 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:17.957+0000 7fdba4ca1700 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fdba015f410 0x7fdba00b4240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:00:17.961 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:17.957+0000 7fdba4ca1700 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fdba01607b0 0x7fdba00ce140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:00:17.961 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:17.957+0000 7fdba4ca1700 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdba00b14c0 con 0x7fdba015fc20 2023-11-24T08:00:17.961 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:17.957+0000 7fdba4ca1700 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdba00b3070 con 0x7fdba01607b0 2023-11-24T08:00:17.962 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:17.957+0000 7fdba4ca1700 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdba00b3270 con 0x7fdba015f410 2023-11-24T08:00:17.962 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:17.958+0000 7fdb9ed9d700 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fdba01607b0 0x7fdba00ce140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:00:17.962 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:17.958+0000 7fdb9e59c700 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fdba015f410 0x7fdba00b4240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:00:17.962 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:17.958+0000 7fdb9e59c700 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fdba015f410 0x7fdba00b4240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3301/0 says I am v2:172.21.15.59:55726/0 (socket says 172.21.15.59:55726) 2023-11-24T08:00:17.962 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:17.958+0000 7fdb9ed9d700 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fdba01607b0 0x7fdba00ce140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.59:43488/0 (socket says 172.21.15.59:43488) 2023-11-24T08:00:17.962 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:17.958+0000 7fdb9e59c700 1 -- 172.21.15.59:0/3840505372 learned_addr learned my addr 172.21.15.59:0/3840505372 (peer_addr_for_me v2:172.21.15.59:0/0) 2023-11-24T08:00:17.962 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:17.958+0000 7fdb9dd9b700 1 --1- 172.21.15.59:0/3840505372 >> v1:172.21.15.76:6789/0 conn(0x7fdba015fc20 0x7fdba01600a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.59:47038/0 (socket says 172.21.15.59:47038) 2023-11-24T08:00:17.962 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:17.958+0000 7fdb9d59a700 1 -- 172.21.15.59:0/3840505372 <== mon.1 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1769683224 0 0) 0x7fdba00b14c0 con 0x7fdba015fc20 2023-11-24T08:00:17.962 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:17.958+0000 7fdb9d59a700 1 -- 172.21.15.59:0/3840505372 --> v1:172.21.15.76:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdb880035b0 con 0x7fdba015fc20 2023-11-24T08:00:17.962 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:17.958+0000 7fdb9e59c700 1 -- 172.21.15.59:0/3840505372 >> v2:172.21.15.76:3300/0 conn(0x7fdba01607b0 msgr2=0x7fdba00ce140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:00:17.962 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:17.958+0000 7fdb9e59c700 1 --2- 172.21.15.59:0/3840505372 >> v2:172.21.15.76:3300/0 conn(0x7fdba01607b0 0x7fdba00ce140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:00:17.963 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:17.958+0000 7fdb9e59c700 1 -- 172.21.15.59:0/3840505372 >> v1:172.21.15.76:6789/0 conn(0x7fdba015fc20 legacy=0x7fdba01600a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:00:17.963 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:17.958+0000 7fdb9e59c700 1 -- 172.21.15.59:0/3840505372 --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdba00b3510 con 0x7fdba015f410 2023-11-24T08:00:17.963 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:17.958+0000 7fdb9ed9d700 1 --2- 172.21.15.59:0/3840505372 >> v2:172.21.15.76:3300/0 conn(0x7fdba01607b0 0x7fdba00ce140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-24T08:00:17.963 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:17.958+0000 7fdb9e59c700 1 --2- 172.21.15.59:0/3840505372 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fdba015f410 0x7fdba00b4240 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fdb940098f0 tx=0x7fdb94009c00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cb51ab0c1025f683 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:00:17.963 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:17.958+0000 7fdb9d59a700 1 -- 172.21.15.59:0/3840505372 <== mon.2 v2:172.21.15.76:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb94011830 con 0x7fdba015f410 2023-11-24T08:00:17.963 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:17.959+0000 7fdb9d59a700 1 -- 172.21.15.59:0/3840505372 <== mon.2 v2:172.21.15.76:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdb94011990 con 0x7fdba015f410 2023-11-24T08:00:17.963 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:17.959+0000 7fdb9d59a700 1 -- 172.21.15.59:0/3840505372 <== mon.2 v2:172.21.15.76:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb94011d20 con 0x7fdba015f410 2023-11-24T08:00:17.963 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:17.959+0000 7fdba4ca1700 1 -- 172.21.15.59:0/3840505372 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fdba015f410 msgr2=0x7fdba00b4240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:00:17.963 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:17.959+0000 7fdba4ca1700 1 --2- 172.21.15.59:0/3840505372 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fdba015f410 0x7fdba00b4240 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fdb940098f0 tx=0x7fdb94009c00 comp rx=0 tx=0).stop 2023-11-24T08:00:17.963 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:17.959+0000 7fdba4ca1700 1 -- 172.21.15.59:0/3840505372 shutdown_connections 2023-11-24T08:00:17.963 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:17.959+0000 7fdba4ca1700 1 --2- 172.21.15.59:0/3840505372 >> v2:172.21.15.76:3300/0 conn(0x7fdba01607b0 0x7fdba00ce140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:00:17.963 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:17.959+0000 7fdba4ca1700 1 --2- 172.21.15.59:0/3840505372 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fdba015f410 0x7fdba00b4240 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:00:17.964 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:17.959+0000 7fdba4ca1700 1 -- 172.21.15.59:0/3840505372 >> 172.21.15.59:0/3840505372 conn(0x7fdba00b3720 msgr2=0x7fdba00b3b30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:00:17.964 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:17.959+0000 7fdba4ca1700 1 -- 172.21.15.59:0/3840505372 shutdown_connections 2023-11-24T08:00:17.964 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:17.959+0000 7fdba4ca1700 1 -- 172.21.15.59:0/3840505372 wait complete. 2023-11-24T08:00:17.966 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:17.961+0000 7fdba4ca1700 1 Processor -- start 2023-11-24T08:00:17.966 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:17.961+0000 7fdba4ca1700 1 -- start start 2023-11-24T08:00:17.967 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:17.961+0000 7fdba4ca1700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fdba015f410 0x7fdba01f5b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:00:17.967 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:17.961+0000 7fdba4ca1700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fdba015fc20 0x7fdba0070110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:00:17.967 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:17.961+0000 7fdba4ca1700 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fdba01607b0 0x7fdba014fcf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:00:17.967 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:17.961+0000 7fdba4ca1700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdba00b5320 con 0x7fdba015fc20 2023-11-24T08:00:17.967 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:17.961+0000 7fdba4ca1700 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdba01f60c0 con 0x7fdba015f410 2023-11-24T08:00:17.967 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:17.961+0000 7fdba4ca1700 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdba00b6570 con 0x7fdba01607b0 2023-11-24T08:00:17.967 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:17.961+0000 7fdb9dd9b700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fdba015fc20 0x7fdba0070110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:00:17.967 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:17.962+0000 7fdb9dd9b700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fdba015fc20 0x7fdba0070110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:56294/0 (socket says 172.21.15.59:56294) 2023-11-24T08:00:17.967 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:17.962+0000 7fdb9dd9b700 1 -- 172.21.15.59:0/4250717658 learned_addr learned my addr 172.21.15.59:0/4250717658 (peer_addr_for_me v2:172.21.15.59:0/0) 2023-11-24T08:00:17.967 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:17.962+0000 7fdb9e59c700 1 --2- 172.21.15.59:0/4250717658 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fdba015f410 0x7fdba01f5b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:00:17.967 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:17.962+0000 7fdb9ed9d700 1 --2- 172.21.15.59:0/4250717658 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fdba01607b0 0x7fdba014fcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:00:17.967 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:17.962+0000 7fdb9dd9b700 1 -- 172.21.15.59:0/4250717658 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fdba01607b0 msgr2=0x7fdba014fcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:00:17.967 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:17.962+0000 7fdb9dd9b700 1 --2- 172.21.15.59:0/4250717658 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fdba01607b0 0x7fdba014fcf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:00:17.968 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:17.962+0000 7fdb9dd9b700 1 -- 172.21.15.59:0/4250717658 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fdba015f410 msgr2=0x7fdba01f5b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:00:17.968 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:17.962+0000 7fdb9dd9b700 1 --2- 172.21.15.59:0/4250717658 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fdba015f410 0x7fdba01f5b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:00:17.968 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:17.962+0000 7fdb9dd9b700 1 -- 172.21.15.59:0/4250717658 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdb98009a70 con 0x7fdba015fc20 2023-11-24T08:00:17.968 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:17.962+0000 7fdb9dd9b700 1 --2- 172.21.15.59:0/4250717658 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fdba015fc20 0x7fdba0070110 secure :-1 s=READY pgs=1093 cs=0 l=1 rev1=1 crypto rx=0x7fdb9000c700 tx=0x7fdb9000ca10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:00:17.968 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:17.962+0000 7fdb8effd700 1 -- 172.21.15.59:0/4250717658 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb90005cb0 con 0x7fdba015fc20 2023-11-24T08:00:17.968 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:17.962+0000 7fdb8effd700 1 -- 172.21.15.59:0/4250717658 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdb90005e10 con 0x7fdba015fc20 2023-11-24T08:00:17.968 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:17.962+0000 7fdba4ca1700 1 -- 172.21.15.59:0/4250717658 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdb94009550 con 0x7fdba015fc20 2023-11-24T08:00:17.968 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:17.962+0000 7fdb8effd700 1 -- 172.21.15.59:0/4250717658 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb900072d0 con 0x7fdba015fc20 2023-11-24T08:00:17.968 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:17.962+0000 7fdba4ca1700 1 -- 172.21.15.59:0/4250717658 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdba00b5cd0 con 0x7fdba015fc20 2023-11-24T08:00:17.968 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:17.963+0000 7fdba4ca1700 1 -- 172.21.15.59:0/4250717658 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdba00ca2c0 con 0x7fdba015fc20 2023-11-24T08:00:17.968 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:17.964+0000 7fdb8effd700 1 -- 172.21.15.59:0/4250717658 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 12) v1 ==== 89792+0+0 (secure 0 0 0) 0x7fdb90007430 con 0x7fdba015fc20 2023-11-24T08:00:17.968 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:17.964+0000 7fdb8effd700 1 --2- 172.21.15.59:0/4250717658 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7fdb8006b530 0x7fdb8006d9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:00:17.968 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:17.964+0000 7fdb8effd700 1 -- 172.21.15.59:0/4250717658 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fdb90089730 con 0x7fdba015fc20 2023-11-24T08:00:17.969 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:17.965+0000 7fdb9e59c700 1 --2- 172.21.15.59:0/4250717658 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7fdb8006b530 0x7fdb8006d9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:00:17.969 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:17.965+0000 7fdb9e59c700 1 --2- 172.21.15.59:0/4250717658 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7fdb8006b530 0x7fdb8006d9f0 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7fdb94009c40 tx=0x7fdb94008030 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:00:17.969 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:17.968+0000 7fdb8effd700 1 -- 172.21.15.59:0/4250717658 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fdb900540e0 con 0x7fdba015fc20 2023-11-24T08:00:18.171 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:18.170+0000 7fdba4ca1700 1 -- 172.21.15.59:0/4250717658 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fdba0075480 con 0x7fdba015fc20 2023-11-24T08:00:18.172 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:18.170+0000 7fdb8effd700 1 -- 172.21.15.59:0/4250717658 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v87) v1 ==== 93+0+10 (secure 0 0 0) 0x7fdb900170a0 con 0x7fdba015fc20 2023-11-24T08:00:18.172 INFO:teuthology.orchestra.run.smithi059.stdout:pg_num: 1 2023-11-24T08:00:18.175 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:18.172+0000 7fdba4ca1700 1 -- 172.21.15.59:0/4250717658 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7fdb8006b530 msgr2=0x7fdb8006d9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:00:18.175 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:18.172+0000 7fdba4ca1700 1 --2- 172.21.15.59:0/4250717658 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7fdb8006b530 0x7fdb8006d9f0 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7fdb94009c40 tx=0x7fdb94008030 comp rx=0 tx=0).stop 2023-11-24T08:00:18.175 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:18.172+0000 7fdba4ca1700 1 -- 172.21.15.59:0/4250717658 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fdba015fc20 msgr2=0x7fdba0070110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:00:18.176 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:18.172+0000 7fdba4ca1700 1 --2- 172.21.15.59:0/4250717658 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fdba015fc20 0x7fdba0070110 secure :-1 s=READY pgs=1093 cs=0 l=1 rev1=1 crypto rx=0x7fdb9000c700 tx=0x7fdb9000ca10 comp rx=0 tx=0).stop 2023-11-24T08:00:18.176 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:18.172+0000 7fdba4ca1700 1 -- 172.21.15.59:0/4250717658 shutdown_connections 2023-11-24T08:00:18.176 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:18.172+0000 7fdba4ca1700 1 --2- 172.21.15.59:0/4250717658 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7fdb8006b530 0x7fdb8006d9f0 unknown :-1 s=CLOSED pgs=501 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:00:18.176 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:18.172+0000 7fdba4ca1700 1 --2- 172.21.15.59:0/4250717658 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fdba01607b0 0x7fdba014fcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:00:18.176 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:18.172+0000 7fdba4ca1700 1 --2- 172.21.15.59:0/4250717658 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fdba015f410 0x7fdba01f5b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:00:18.176 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:18.172+0000 7fdba4ca1700 1 --2- 172.21.15.59:0/4250717658 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fdba015fc20 0x7fdba0070110 unknown :-1 s=CLOSED pgs=1093 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:00:18.177 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:18.172+0000 7fdba4ca1700 1 -- 172.21.15.59:0/4250717658 >> 172.21.15.59:0/4250717658 conn(0x7fdba00b3720 msgr2=0x7fdba00cee90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:00:18.177 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:18.172+0000 7fdba4ca1700 1 -- 172.21.15.59:0/4250717658 shutdown_connections 2023-11-24T08:00:18.177 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:18.173+0000 7fdba4ca1700 1 -- 172.21.15.59:0/4250717658 wait complete. 2023-11-24T08:00:18.186 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-24T08:00:18.315 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:18.313+0000 7f2890a3d700 1 Processor -- start 2023-11-24T08:00:18.315 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:18.313+0000 7f2890a3d700 1 -- start start 2023-11-24T08:00:18.317 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:18.314+0000 7f2890a3d700 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f288c143ae0 0x7f288c0b51b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:00:18.317 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:18.314+0000 7f2890a3d700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f288c0722c0 0x7f288c06e510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:00:18.317 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:18.314+0000 7f2890a3d700 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f288c143ec0 con 0x7f288c1440f0 2023-11-24T08:00:18.318 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:18.314+0000 7f2890a3d700 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f288c0b40a0 con 0x7f288c0722c0 2023-11-24T08:00:18.318 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:18.314+0000 7f2890a3d700 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f288c0b3ae0 con 0x7f288c143ae0 2023-11-24T08:00:18.318 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:18.314+0000 7f288ad9d700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f288c0722c0 0x7f288c06e510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:00:18.318 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:18.314+0000 7f288ad9d700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f288c0722c0 0x7f288c06e510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:56304/0 (socket says 172.21.15.59:56304) 2023-11-24T08:00:18.318 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:18.314+0000 7f288ad9d700 1 -- 172.21.15.59:0/877677186 learned_addr learned my addr 172.21.15.59:0/877677186 (peer_addr_for_me v2:172.21.15.59:0/0) 2023-11-24T08:00:18.318 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:18.314+0000 7f288a59c700 1 --2- 172.21.15.59:0/877677186 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f288c143ae0 0x7f288c0b51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:00:18.318 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:18.314+0000 7f288ad9d700 1 -- 172.21.15.59:0/877677186 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f288c143ae0 msgr2=0x7f288c0b51b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:00:18.319 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:18.314+0000 7f288ad9d700 1 --2- 172.21.15.59:0/877677186 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f288c143ae0 0x7f288c0b51b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:00:18.319 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:18.314+0000 7f288ad9d700 1 -- 172.21.15.59:0/877677186 >> v1:172.21.15.76:6789/0 conn(0x7f288c1440f0 legacy=0x7f288c071bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:00:18.319 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:18.314+0000 7f288ad9d700 1 -- 172.21.15.59:0/877677186 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f288c0b3dc0 con 0x7f288c0722c0 2023-11-24T08:00:18.319 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:18.315+0000 7f288ad9d700 1 --2- 172.21.15.59:0/877677186 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f288c0722c0 0x7f288c06e510 secure :-1 s=READY pgs=1094 cs=0 l=1 rev1=1 crypto rx=0x7f2884009e50 tx=0x7f288400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5df7aa7c65d66479 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:00:18.319 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:18.315+0000 7f288959a700 1 -- 172.21.15.59:0/877677186 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f288400e070 con 0x7f288c0722c0 2023-11-24T08:00:18.319 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:18.315+0000 7f288959a700 1 -- 172.21.15.59:0/877677186 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f288400b9c0 con 0x7f288c0722c0 2023-11-24T08:00:18.319 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:18.315+0000 7f288959a700 1 -- 172.21.15.59:0/877677186 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f288400bd70 con 0x7f288c0722c0 2023-11-24T08:00:18.320 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:18.315+0000 7f2890a3d700 1 -- 172.21.15.59:0/877677186 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f288c0722c0 msgr2=0x7f288c06e510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:00:18.320 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:18.315+0000 7f2890a3d700 1 --2- 172.21.15.59:0/877677186 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f288c0722c0 0x7f288c06e510 secure :-1 s=READY pgs=1094 cs=0 l=1 rev1=1 crypto rx=0x7f2884009e50 tx=0x7f288400b330 comp rx=0 tx=0).stop 2023-11-24T08:00:18.320 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:18.315+0000 7f2890a3d700 1 -- 172.21.15.59:0/877677186 shutdown_connections 2023-11-24T08:00:18.320 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:18.315+0000 7f2890a3d700 1 --2- 172.21.15.59:0/877677186 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f288c143ae0 0x7f288c0b51b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:00:18.320 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:18.315+0000 7f2890a3d700 1 --2- 172.21.15.59:0/877677186 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f288c0722c0 0x7f288c06e510 unknown :-1 s=CLOSED pgs=1094 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:00:18.320 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:18.315+0000 7f2890a3d700 1 -- 172.21.15.59:0/877677186 >> 172.21.15.59:0/877677186 conn(0x7f288c0b2740 msgr2=0x7f288c0b2b20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:00:18.320 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:18.315+0000 7f2890a3d700 1 -- 172.21.15.59:0/877677186 shutdown_connections 2023-11-24T08:00:18.321 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:18.316+0000 7f2890a3d700 1 -- 172.21.15.59:0/877677186 wait complete. 2023-11-24T08:00:18.324 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:18.317+0000 7f2890a3d700 1 Processor -- start 2023-11-24T08:00:18.324 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:18.317+0000 7f2890a3d700 1 -- start start 2023-11-24T08:00:18.324 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:18.318+0000 7f2890a3d700 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f288c0722c0 0x7f288c13b130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:00:18.324 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:18.318+0000 7f2890a3d700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f288c143ae0 0x7f288c141f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:00:18.324 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:18.318+0000 7f2890a3d700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f288c1440f0 0x7f288c142480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:00:18.325 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:18.318+0000 7f2890a3d700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f288c1f5ba0 con 0x7f288c1440f0 2023-11-24T08:00:18.325 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:18.318+0000 7f2890a3d700 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f288c158fe0 con 0x7f288c143ae0 2023-11-24T08:00:18.325 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:18.318+0000 7f2890a3d700 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f288c1f54e0 con 0x7f288c0722c0 2023-11-24T08:00:18.325 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:18.318+0000 7f288ad9d700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f288c1440f0 0x7f288c142480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:00:18.325 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:18.318+0000 7f288ad9d700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f288c1440f0 0x7f288c142480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:56310/0 (socket says 172.21.15.59:56310) 2023-11-24T08:00:18.326 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:18.318+0000 7f288ad9d700 1 -- 172.21.15.59:0/761727235 learned_addr learned my addr 172.21.15.59:0/761727235 (peer_addr_for_me v2:172.21.15.59:0/0) 2023-11-24T08:00:18.326 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:18.318+0000 7f2889d9b700 1 --2- 172.21.15.59:0/761727235 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f288c143ae0 0x7f288c141f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:00:18.326 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:18.318+0000 7f288a59c700 1 --2- 172.21.15.59:0/761727235 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f288c0722c0 0x7f288c13b130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:00:18.326 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:18.318+0000 7f288ad9d700 1 -- 172.21.15.59:0/761727235 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f288c0722c0 msgr2=0x7f288c13b130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:00:18.326 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:18.318+0000 7f288ad9d700 1 --2- 172.21.15.59:0/761727235 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f288c0722c0 0x7f288c13b130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:00:18.326 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:18.318+0000 7f288ad9d700 1 -- 172.21.15.59:0/761727235 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f288c143ae0 msgr2=0x7f288c141f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:00:18.327 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:18.318+0000 7f288ad9d700 1 --2- 172.21.15.59:0/761727235 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f288c143ae0 0x7f288c141f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:00:18.327 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:18.318+0000 7f288ad9d700 1 -- 172.21.15.59:0/761727235 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2884009b30 con 0x7f288c1440f0 2023-11-24T08:00:18.327 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:18.318+0000 7f288ad9d700 1 --2- 172.21.15.59:0/761727235 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f288c1440f0 0x7f288c142480 secure :-1 s=READY pgs=1095 cs=0 l=1 rev1=1 crypto rx=0x7f2884012000 tx=0x7f2884004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:00:18.327 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:18.318+0000 7f287affd700 1 -- 172.21.15.59:0/761727235 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2884004840 con 0x7f288c1440f0 2023-11-24T08:00:18.327 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:18.318+0000 7f287affd700 1 -- 172.21.15.59:0/761727235 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f28840092e0 con 0x7f288c1440f0 2023-11-24T08:00:18.328 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:18.318+0000 7f2890a3d700 1 -- 172.21.15.59:0/761727235 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f288c1f57c0 con 0x7f288c1440f0 2023-11-24T08:00:18.328 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:18.318+0000 7f287affd700 1 -- 172.21.15.59:0/761727235 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f288401c660 con 0x7f288c1440f0 2023-11-24T08:00:18.328 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:18.319+0000 7f2890a3d700 1 -- 172.21.15.59:0/761727235 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f288c1f5fc0 con 0x7f288c1440f0 2023-11-24T08:00:18.328 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:18.319+0000 7f2890a3d700 1 -- 172.21.15.59:0/761727235 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f288c075480 con 0x7f288c1440f0 2023-11-24T08:00:18.328 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:18.320+0000 7f287affd700 1 -- 172.21.15.59:0/761727235 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 12) v1 ==== 89792+0+0 (secure 0 0 0) 0x7f288401c7c0 con 0x7f288c1440f0 2023-11-24T08:00:18.329 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:18.320+0000 7f287affd700 1 --2- 172.21.15.59:0/761727235 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f286c06b500 0x7f286c06d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:00:18.329 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:18.320+0000 7f287affd700 1 -- 172.21.15.59:0/761727235 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f288408bb30 con 0x7f288c1440f0 2023-11-24T08:00:18.329 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:18.320+0000 7f288a59c700 1 --2- 172.21.15.59:0/761727235 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f286c06b500 0x7f286c06d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:00:18.329 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:18.321+0000 7f288a59c700 1 --2- 172.21.15.59:0/761727235 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f286c06b500 0x7f286c06d9c0 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7f2880006b20 tx=0x7f2880008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:00:18.330 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:18.325+0000 7f287affd700 1 -- 172.21.15.59:0/761727235 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f28840561e0 con 0x7f288c1440f0 2023-11-24T08:00:18.530 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:18.528+0000 7f2890a3d700 1 -- 172.21.15.59:0/761727235 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f288c1f5ce0 con 0x7f288c1440f0 2023-11-24T08:00:18.531 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:18.529+0000 7f287affd700 1 -- 172.21.15.59:0/761727235 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v87) v1 ==== 104+0+11 (secure 0 0 0) 0x7f288c1f5ce0 con 0x7f288c1440f0 2023-11-24T08:00:18.531 INFO:teuthology.orchestra.run.smithi059.stdout:pg_num: 32 2023-11-24T08:00:18.533 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:18.531+0000 7f2890a3d700 1 -- 172.21.15.59:0/761727235 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f286c06b500 msgr2=0x7f286c06d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:00:18.533 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:18.531+0000 7f2890a3d700 1 --2- 172.21.15.59:0/761727235 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f286c06b500 0x7f286c06d9c0 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7f2880006b20 tx=0x7f2880008040 comp rx=0 tx=0).stop 2023-11-24T08:00:18.534 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:18.531+0000 7f2890a3d700 1 -- 172.21.15.59:0/761727235 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f288c1440f0 msgr2=0x7f288c142480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:00:18.534 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:18.531+0000 7f2890a3d700 1 --2- 172.21.15.59:0/761727235 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f288c1440f0 0x7f288c142480 secure :-1 s=READY pgs=1095 cs=0 l=1 rev1=1 crypto rx=0x7f2884012000 tx=0x7f2884004ad0 comp rx=0 tx=0).stop 2023-11-24T08:00:18.534 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:18.531+0000 7f2890a3d700 1 -- 172.21.15.59:0/761727235 shutdown_connections 2023-11-24T08:00:18.534 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:18.531+0000 7f2890a3d700 1 --2- 172.21.15.59:0/761727235 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f286c06b500 0x7f286c06d9c0 unknown :-1 s=CLOSED pgs=502 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:00:18.534 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:18.531+0000 7f2890a3d700 1 --2- 172.21.15.59:0/761727235 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f288c1440f0 0x7f288c142480 unknown :-1 s=CLOSED pgs=1095 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:00:18.534 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:18.531+0000 7f2890a3d700 1 --2- 172.21.15.59:0/761727235 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f288c0722c0 0x7f288c13b130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:00:18.534 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:18.531+0000 7f2890a3d700 1 --2- 172.21.15.59:0/761727235 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f288c143ae0 0x7f288c141f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:00:18.535 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:18.531+0000 7f2890a3d700 1 -- 172.21.15.59:0/761727235 >> 172.21.15.59:0/761727235 conn(0x7f288c0b2740 msgr2=0x7f288c15bf90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:00:18.535 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:18.531+0000 7f2890a3d700 1 -- 172.21.15.59:0/761727235 shutdown_connections 2023-11-24T08:00:18.535 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:18.531+0000 7f2890a3d700 1 -- 172.21.15.59:0/761727235 wait complete. 2023-11-24T08:00:18.546 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-24T08:00:18.675 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:18.673+0000 7f59d28a1700 1 Processor -- start 2023-11-24T08:00:18.675 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:18.673+0000 7f59d28a1700 1 -- start start 2023-11-24T08:00:18.685 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:18.674+0000 7f59d28a1700 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f59cc072a00 0x7f59cc06db60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:00:18.685 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:18.674+0000 7f59d28a1700 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f59cc0d0b80 con 0x7f59cc0b4090 2023-11-24T08:00:18.685 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:18.674+0000 7f59d28a1700 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f59cc15ed60 con 0x7f59cc06e0a0 2023-11-24T08:00:18.685 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:18.674+0000 7f59d28a1700 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f59cc15f040 con 0x7f59cc072a00 2023-11-24T08:00:18.685 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:18.674+0000 7f59cbfff700 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7f59cc0b4090 0x7f59cc15e510 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:59788/0 (socket says 172.21.15.59:59788) 2023-11-24T08:00:18.685 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:18.674+0000 7f59cbfff700 1 -- 172.21.15.59:0/200551370 learned_addr learned my addr 172.21.15.59:0/200551370 (peer_addr_for_me v1:172.21.15.59:0/0) 2023-11-24T08:00:18.685 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:18.674+0000 7f59cb7fe700 1 --2- 172.21.15.59:0/200551370 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f59cc072a00 0x7f59cc06db60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:00:18.685 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:18.674+0000 7f59caffd700 1 -- 172.21.15.59:0/200551370 <== mon.0 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3773076280 0 0) 0x7f59cc0d0b80 con 0x7f59cc0b4090 2023-11-24T08:00:18.686 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:18.674+0000 7f59caffd700 1 -- 172.21.15.59:0/200551370 --> v1:172.21.15.59:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f59b40035b0 con 0x7f59cc0b4090 2023-11-24T08:00:18.686 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:18.675+0000 7f59caffd700 1 -- 172.21.15.59:0/200551370 <== mon.0 v1:172.21.15.59:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2179971750 0 0) 0x7f59b40035b0 con 0x7f59cc0b4090 2023-11-24T08:00:18.686 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:18.675+0000 7f59caffd700 1 -- 172.21.15.59:0/200551370 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f59cc072a00 msgr2=0x7f59cc06db60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:00:18.686 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:18.675+0000 7f59caffd700 1 --2- 172.21.15.59:0/200551370 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f59cc072a00 0x7f59cc06db60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:00:18.686 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:18.675+0000 7f59caffd700 1 -- 172.21.15.59:0/200551370 >> v1:172.21.15.76:6789/0 conn(0x7f59cc06e0a0 legacy=0x7f59cc0d1010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:00:18.686 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:18.675+0000 7f59caffd700 1 -- 172.21.15.59:0/200551370 --> v1:172.21.15.59:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f59cc15f320 con 0x7f59cc0b4090 2023-11-24T08:00:18.686 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:18.675+0000 7f59caffd700 1 -- 172.21.15.59:0/200551370 <== mon.0 v1:172.21.15.59:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1312333566 0 0) 0x7f59c0001670 con 0x7f59cc0b4090 2023-11-24T08:00:18.686 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:18.675+0000 7f59caffd700 1 -- 172.21.15.59:0/200551370 >> v1:172.21.15.59:6789/0 conn(0x7f59cc0b4090 legacy=0x7f59cc15e510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:00:18.686 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:18.675+0000 7f59caffd700 1 --2- 172.21.15.59:0/200551370 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f59b4003a30 0x7f59b4003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:00:18.686 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:18.675+0000 7f59caffd700 1 --2- 172.21.15.59:0/200551370 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f59cc072a00 0x7f59b4004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:00:18.686 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:18.675+0000 7f59caffd700 1 --2- 172.21.15.59:0/200551370 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f59b4004d90 0x7f59b4005210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:00:18.686 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:18.675+0000 7f59caffd700 1 -- 172.21.15.59:0/200551370 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59cc15f040 con 0x7f59b4004d90 2023-11-24T08:00:18.687 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:18.675+0000 7f59caffd700 1 -- 172.21.15.59:0/200551370 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59b400baf0 con 0x7f59b4003a30 2023-11-24T08:00:18.687 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:18.675+0000 7f59caffd700 1 -- 172.21.15.59:0/200551370 --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f59b400bcb0 con 0x7f59cc072a00 2023-11-24T08:00:18.687 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:18.675+0000 7f59cb7fe700 1 --2- 172.21.15.59:0/200551370 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f59cc072a00 0x7f59b4004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:00:18.687 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:18.675+0000 7f59cbfff700 1 --2- 172.21.15.59:0/200551370 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f59b4003a30 0x7f59b4003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:00:18.687 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:18.675+0000 7f59d0e3e700 1 --2- 172.21.15.59:0/200551370 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f59b4004d90 0x7f59b4005210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:00:18.687 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:18.675+0000 7f59cb7fe700 1 -- 172.21.15.59:0/200551370 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f59b4003a30 msgr2=0x7f59b4003e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:00:18.687 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:18.675+0000 7f59cb7fe700 1 --2- 172.21.15.59:0/200551370 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f59b4003a30 0x7f59b4003e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:00:18.687 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:18.675+0000 7f59cb7fe700 1 -- 172.21.15.59:0/200551370 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f59b4004d90 msgr2=0x7f59b4005210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:00:18.687 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:18.675+0000 7f59cb7fe700 1 --2- 172.21.15.59:0/200551370 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f59b4004d90 0x7f59b4005210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:00:18.687 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:18.675+0000 7f59cb7fe700 1 -- 172.21.15.59:0/200551370 --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f59b40035b0 con 0x7f59cc072a00 2023-11-24T08:00:18.687 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:18.676+0000 7f59cb7fe700 1 --2- 172.21.15.59:0/200551370 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f59cc072a00 0x7f59b4004850 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f59bc009b60 tx=0x7f59b4005880 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fd35fb02d110edbc server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:00:18.688 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:18.676+0000 7f59caffd700 1 -- 172.21.15.59:0/200551370 <== mon.2 v2:172.21.15.76:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59bc00e6c0 con 0x7f59cc072a00 2023-11-24T08:00:18.688 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:18.676+0000 7f59caffd700 1 -- 172.21.15.59:0/200551370 <== mon.2 v2:172.21.15.76:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f59bc00e820 con 0x7f59cc072a00 2023-11-24T08:00:18.688 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:18.676+0000 7f59caffd700 1 -- 172.21.15.59:0/200551370 <== mon.2 v2:172.21.15.76:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59bc00eb90 con 0x7f59cc072a00 2023-11-24T08:00:18.688 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:18.676+0000 7f59d28a1700 1 -- 172.21.15.59:0/200551370 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f59cc072a00 msgr2=0x7f59b4004850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:00:18.688 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:18.676+0000 7f59d28a1700 1 --2- 172.21.15.59:0/200551370 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f59cc072a00 0x7f59b4004850 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f59bc009b60 tx=0x7f59b4005880 comp rx=0 tx=0).stop 2023-11-24T08:00:18.688 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:18.676+0000 7f59d0e3e700 1 -- 172.21.15.59:0/200551370 reap_dead start 2023-11-24T08:00:18.688 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:18.676+0000 7f59d28a1700 1 -- 172.21.15.59:0/200551370 shutdown_connections 2023-11-24T08:00:18.688 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:18.676+0000 7f59d28a1700 1 -- 172.21.15.59:0/200551370 >> 172.21.15.59:0/200551370 conn(0x7f59cc0b2740 msgr2=0x7f59cc0b2b50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:00:18.688 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:18.676+0000 7f59d28a1700 1 -- 172.21.15.59:0/200551370 shutdown_connections 2023-11-24T08:00:18.688 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:18.676+0000 7f59d28a1700 1 -- 172.21.15.59:0/200551370 wait complete. 2023-11-24T08:00:18.688 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:18.678+0000 7f59d28a1700 1 Processor -- start 2023-11-24T08:00:18.688 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:18.678+0000 7f59d28a1700 1 -- start start 2023-11-24T08:00:18.688 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:18.679+0000 7f59d28a1700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f59cc072a00 0x7f59cc15a1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:00:18.689 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:18.679+0000 7f59d28a1700 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f59cc0cf200 0x7f59cc159620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:00:18.689 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:18.679+0000 7f59d28a1700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f59cc0aeb20 0x7f59cc0c8b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:00:18.689 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:18.679+0000 7f59d28a1700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59cc159220 con 0x7f59cc0aeb20 2023-11-24T08:00:18.689 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:18.679+0000 7f59d28a1700 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59cc159b60 con 0x7f59cc072a00 2023-11-24T08:00:18.689 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:18.679+0000 7f59d28a1700 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f59cc0cf680 con 0x7f59cc0cf200 2023-11-24T08:00:18.689 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:18.679+0000 7f59d0e3e700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f59cc0aeb20 0x7f59cc0c8b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:00:18.689 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:18.679+0000 7f59d0e3e700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f59cc0aeb20 0x7f59cc0c8b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:56330/0 (socket says 172.21.15.59:56330) 2023-11-24T08:00:18.689 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:18.679+0000 7f59d0e3e700 1 -- 172.21.15.59:0/2076682857 learned_addr learned my addr 172.21.15.59:0/2076682857 (peer_addr_for_me v2:172.21.15.59:0/0) 2023-11-24T08:00:18.689 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:18.679+0000 7f59cbfff700 1 --2- 172.21.15.59:0/2076682857 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f59cc072a00 0x7f59cc15a1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:00:18.690 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:18.679+0000 7f59cb7fe700 1 --2- 172.21.15.59:0/2076682857 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f59cc0cf200 0x7f59cc159620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:00:18.690 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:18.679+0000 7f59d0e3e700 1 -- 172.21.15.59:0/2076682857 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f59cc0cf200 msgr2=0x7f59cc159620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:00:18.690 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:18.679+0000 7f59d0e3e700 1 --2- 172.21.15.59:0/2076682857 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f59cc0cf200 0x7f59cc159620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:00:18.690 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:18.679+0000 7f59d0e3e700 1 -- 172.21.15.59:0/2076682857 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f59cc072a00 msgr2=0x7f59cc15a1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:00:18.691 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:18.679+0000 7f59d0e3e700 1 --2- 172.21.15.59:0/2076682857 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f59cc072a00 0x7f59cc15a1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:00:18.691 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:18.679+0000 7f59d0e3e700 1 -- 172.21.15.59:0/2076682857 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f59cc0cf8e0 con 0x7f59cc0aeb20 2023-11-24T08:00:18.691 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:18.679+0000 7f59d0e3e700 1 --2- 172.21.15.59:0/2076682857 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f59cc0aeb20 0x7f59cc0c8b30 secure :-1 s=READY pgs=1097 cs=0 l=1 rev1=1 crypto rx=0x7f59cc0d1af0 tx=0x7f59c400d430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:00:18.691 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:18.679+0000 7f59c8ff9700 1 -- 172.21.15.59:0/2076682857 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59c4003c10 con 0x7f59cc0aeb20 2023-11-24T08:00:18.692 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:18.679+0000 7f59c8ff9700 1 -- 172.21.15.59:0/2076682857 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f59c4003d90 con 0x7f59cc0aeb20 2023-11-24T08:00:18.692 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:18.679+0000 7f59d28a1700 1 -- 172.21.15.59:0/2076682857 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f59cc0cfbc0 con 0x7f59cc0aeb20 2023-11-24T08:00:18.692 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:18.679+0000 7f59c8ff9700 1 -- 172.21.15.59:0/2076682857 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59c400dac0 con 0x7f59cc0aeb20 2023-11-24T08:00:18.692 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:18.679+0000 7f59d28a1700 1 -- 172.21.15.59:0/2076682857 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f59cc158e40 con 0x7f59cc0aeb20 2023-11-24T08:00:18.692 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:18.681+0000 7f59d28a1700 1 -- 172.21.15.59:0/2076682857 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f59cc06fa70 con 0x7f59cc0aeb20 2023-11-24T08:00:18.692 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:18.681+0000 7f59c8ff9700 1 -- 172.21.15.59:0/2076682857 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 12) v1 ==== 89792+0+0 (secure 0 0 0) 0x7f59c400dc20 con 0x7f59cc0aeb20 2023-11-24T08:00:18.693 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:18.681+0000 7f59c8ff9700 1 --2- 172.21.15.59:0/2076682857 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f59b806b500 0x7f59b806d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:00:18.693 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:18.681+0000 7f59c8ff9700 1 -- 172.21.15.59:0/2076682857 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f59c408a2a0 con 0x7f59cc0aeb20 2023-11-24T08:00:18.693 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:18.682+0000 7f59cbfff700 1 --2- 172.21.15.59:0/2076682857 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f59b806b500 0x7f59b806d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:00:18.693 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:18.682+0000 7f59cbfff700 1 --2- 172.21.15.59:0/2076682857 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f59b806b500 0x7f59b806d9c0 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7f59c0000c00 tx=0x7f59c0001090 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:00:18.694 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:18.686+0000 7f59c8ff9700 1 -- 172.21.15.59:0/2076682857 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f59c4054aa0 con 0x7f59cc0aeb20 2023-11-24T08:00:18.890 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:18.888+0000 7f59d28a1700 1 -- 172.21.15.59:0/2076682857 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f59cc075480 con 0x7f59cc0aeb20 2023-11-24T08:00:18.890 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:18.888+0000 7f59c8ff9700 1 -- 172.21.15.59:0/2076682857 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v87) v1 ==== 100+0+11 (secure 0 0 0) 0x7f59c40580c0 con 0x7f59cc0aeb20 2023-11-24T08:00:18.890 INFO:teuthology.orchestra.run.smithi059.stdout:pg_num: 64 2023-11-24T08:00:18.892 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:18.890+0000 7f59d28a1700 1 -- 172.21.15.59:0/2076682857 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f59b806b500 msgr2=0x7f59b806d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:00:18.892 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:18.890+0000 7f59d28a1700 1 --2- 172.21.15.59:0/2076682857 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f59b806b500 0x7f59b806d9c0 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7f59c0000c00 tx=0x7f59c0001090 comp rx=0 tx=0).stop 2023-11-24T08:00:18.893 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:18.890+0000 7f59d28a1700 1 -- 172.21.15.59:0/2076682857 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f59cc0aeb20 msgr2=0x7f59cc0c8b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:00:18.893 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:18.890+0000 7f59d28a1700 1 --2- 172.21.15.59:0/2076682857 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f59cc0aeb20 0x7f59cc0c8b30 secure :-1 s=READY pgs=1097 cs=0 l=1 rev1=1 crypto rx=0x7f59cc0d1af0 tx=0x7f59c400d430 comp rx=0 tx=0).stop 2023-11-24T08:00:18.893 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:18.890+0000 7f59d28a1700 1 -- 172.21.15.59:0/2076682857 shutdown_connections 2023-11-24T08:00:18.893 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:18.890+0000 7f59d28a1700 1 --2- 172.21.15.59:0/2076682857 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f59b806b500 0x7f59b806d9c0 unknown :-1 s=CLOSED pgs=503 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:00:18.893 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:18.890+0000 7f59d28a1700 1 --2- 172.21.15.59:0/2076682857 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f59cc0aeb20 0x7f59cc0c8b30 unknown :-1 s=CLOSED pgs=1097 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:00:18.893 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:18.890+0000 7f59d28a1700 1 --2- 172.21.15.59:0/2076682857 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f59cc072a00 0x7f59cc15a1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:00:18.893 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:18.890+0000 7f59d28a1700 1 --2- 172.21.15.59:0/2076682857 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f59cc0cf200 0x7f59cc159620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:00:18.894 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:18.890+0000 7f59d28a1700 1 -- 172.21.15.59:0/2076682857 >> 172.21.15.59:0/2076682857 conn(0x7f59cc0b2740 msgr2=0x7f59cc15e510 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:00:18.894 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:18.890+0000 7f59d28a1700 1 -- 172.21.15.59:0/2076682857 shutdown_connections 2023-11-24T08:00:18.894 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:18.890+0000 7f59d28a1700 1 -- 172.21.15.59:0/2076682857 wait complete. 2023-11-24T08:00:18.905 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-24T08:00:19.030 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:19.028+0000 7f9920ac5700 1 Processor -- start 2023-11-24T08:00:19.030 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:19.028+0000 7f9920ac5700 1 -- start start 2023-11-24T08:00:19.039 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:19.029+0000 7f9920ac5700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f991c0b57b0 0x7f991c06afd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:00:19.039 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:19.029+0000 7f9920ac5700 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f991c06b510 0x7f991c06e370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:00:19.039 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:19.029+0000 7f9920ac5700 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f991c06b970 con 0x7f991c1478c0 2023-11-24T08:00:19.039 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:19.029+0000 7f9920ac5700 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f991c147ca0 con 0x7f991c0b57b0 2023-11-24T08:00:19.040 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:19.029+0000 7f9920ac5700 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f991c06dbe0 con 0x7f991c06b510 2023-11-24T08:00:19.040 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:19.029+0000 7f9919d9b700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f991c0b57b0 0x7f991c06afd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:00:19.040 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:19.029+0000 7f9919d9b700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f991c0b57b0 0x7f991c06afd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:56336/0 (socket says 172.21.15.59:56336) 2023-11-24T08:00:19.040 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:19.029+0000 7f9919d9b700 1 -- 172.21.15.59:0/1970904597 learned_addr learned my addr 172.21.15.59:0/1970904597 (peer_addr_for_me v2:172.21.15.59:0/0) 2023-11-24T08:00:19.040 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:19.029+0000 7f991a59c700 1 --1- 172.21.15.59:0/1970904597 >> v1:172.21.15.59:6789/0 conn(0x7f991c1478c0 0x7f991c161a90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:59808/0 (socket says 172.21.15.59:59808) 2023-11-24T08:00:19.040 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:19.029+0000 7f991ad9d700 1 --2- 172.21.15.59:0/1970904597 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f991c06b510 0x7f991c06e370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:00:19.040 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:19.029+0000 7f9919d9b700 1 -- 172.21.15.59:0/1970904597 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f991c06b510 msgr2=0x7f991c06e370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:00:19.040 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:19.029+0000 7f9919d9b700 1 --2- 172.21.15.59:0/1970904597 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f991c06b510 0x7f991c06e370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:00:19.040 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:19.029+0000 7f9919d9b700 1 -- 172.21.15.59:0/1970904597 >> v1:172.21.15.59:6789/0 conn(0x7f991c1478c0 legacy=0x7f991c161a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:00:19.041 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:19.029+0000 7f9919d9b700 1 -- 172.21.15.59:0/1970904597 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f991c1622e0 con 0x7f991c0b57b0 2023-11-24T08:00:19.041 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:19.029+0000 7f991959a700 1 -- 172.21.15.59:0/1970904597 <== mon.0 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2037027209 0 0) 0x7f991c06b970 con 0x7f991c1478c0 2023-11-24T08:00:19.041 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:19.029+0000 7f991ad9d700 1 --2- 172.21.15.59:0/1970904597 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f991c06b510 0x7f991c06e370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-24T08:00:19.041 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:19.030+0000 7f9919d9b700 1 --2- 172.21.15.59:0/1970904597 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f991c0b57b0 0x7f991c06afd0 secure :-1 s=READY pgs=1099 cs=0 l=1 rev1=1 crypto rx=0x7f990c0098f0 tx=0x7f990c009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6fe399894f5d7943 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:00:19.041 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:19.030+0000 7f991959a700 1 -- 172.21.15.59:0/1970904597 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f990c00b8a0 con 0x7f991c0b57b0 2023-11-24T08:00:19.041 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:19.030+0000 7f991959a700 1 -- 172.21.15.59:0/1970904597 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f990c00ba00 con 0x7f991c0b57b0 2023-11-24T08:00:19.041 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:19.030+0000 7f991959a700 1 -- 172.21.15.59:0/1970904597 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f990c00bd90 con 0x7f991c0b57b0 2023-11-24T08:00:19.041 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:19.030+0000 7f9920ac5700 1 -- 172.21.15.59:0/1970904597 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f991c0b57b0 msgr2=0x7f991c06afd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:00:19.041 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:19.030+0000 7f9920ac5700 1 --2- 172.21.15.59:0/1970904597 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f991c0b57b0 0x7f991c06afd0 secure :-1 s=READY pgs=1099 cs=0 l=1 rev1=1 crypto rx=0x7f990c0098f0 tx=0x7f990c009c00 comp rx=0 tx=0).stop 2023-11-24T08:00:19.041 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:19.030+0000 7f9920ac5700 1 -- 172.21.15.59:0/1970904597 shutdown_connections 2023-11-24T08:00:19.041 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:19.030+0000 7f9920ac5700 1 --2- 172.21.15.59:0/1970904597 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f991c06b510 0x7f991c06e370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:00:19.042 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:19.030+0000 7f9920ac5700 1 --2- 172.21.15.59:0/1970904597 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f991c0b57b0 0x7f991c06afd0 unknown :-1 s=CLOSED pgs=1099 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:00:19.042 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:19.030+0000 7f9920ac5700 1 -- 172.21.15.59:0/1970904597 >> 172.21.15.59:0/1970904597 conn(0x7f991c0d0aa0 msgr2=0x7f991c0d0eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:00:19.042 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:19.030+0000 7f9920ac5700 1 -- 172.21.15.59:0/1970904597 shutdown_connections 2023-11-24T08:00:19.042 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:19.030+0000 7f9920ac5700 1 -- 172.21.15.59:0/1970904597 wait complete. 2023-11-24T08:00:19.042 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:19.032+0000 7f9920ac5700 1 Processor -- start 2023-11-24T08:00:19.042 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:19.032+0000 7f9920ac5700 1 -- start start 2023-11-24T08:00:19.042 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:19.033+0000 7f9920ac5700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f991c06b510 0x7f991c13f6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:00:19.042 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:19.033+0000 7f9920ac5700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f991c0b57b0 0x7f991c0ae320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:00:19.042 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:19.033+0000 7f9920ac5700 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f991c1478c0 0x7f991c0b0f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:00:19.042 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:19.033+0000 7f9920ac5700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f991c140e70 con 0x7f991c0b57b0 2023-11-24T08:00:19.042 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:19.033+0000 7f9920ac5700 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f991c141290 con 0x7f991c06b510 2023-11-24T08:00:19.043 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:19.033+0000 7f9920ac5700 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f991c1401f0 con 0x7f991c1478c0 2023-11-24T08:00:19.043 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:19.033+0000 7f9919d9b700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f991c0b57b0 0x7f991c0ae320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:00:19.043 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:19.033+0000 7f9919d9b700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f991c0b57b0 0x7f991c0ae320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:56342/0 (socket says 172.21.15.59:56342) 2023-11-24T08:00:19.043 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:19.033+0000 7f9919d9b700 1 -- 172.21.15.59:0/1498465316 learned_addr learned my addr 172.21.15.59:0/1498465316 (peer_addr_for_me v2:172.21.15.59:0/0) 2023-11-24T08:00:19.043 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:19.033+0000 7f9919d9b700 1 -- 172.21.15.59:0/1498465316 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f991c1478c0 msgr2=0x7f991c0b0f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:00:19.043 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:19.033+0000 7f9919d9b700 1 --2- 172.21.15.59:0/1498465316 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f991c1478c0 0x7f991c0b0f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:00:19.043 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:19.033+0000 7f9919d9b700 1 -- 172.21.15.59:0/1498465316 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f991c06b510 msgr2=0x7f991c13f6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:00:19.043 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:19.033+0000 7f9919d9b700 1 --2- 172.21.15.59:0/1498465316 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f991c06b510 0x7f991c13f6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:00:19.043 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:19.033+0000 7f9919d9b700 1 -- 172.21.15.59:0/1498465316 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f990c009550 con 0x7f991c0b57b0 2023-11-24T08:00:19.043 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:19.033+0000 7f9919d9b700 1 --2- 172.21.15.59:0/1498465316 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f991c0b57b0 0x7f991c0ae320 secure :-1 s=READY pgs=1100 cs=0 l=1 rev1=1 crypto rx=0x7f990c009c40 tx=0x7f990c004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:00:19.043 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:19.033+0000 7f990affd700 1 -- 172.21.15.59:0/1498465316 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f990c004380 con 0x7f991c0b57b0 2023-11-24T08:00:19.044 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:19.033+0000 7f990affd700 1 -- 172.21.15.59:0/1498465316 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f990c01f070 con 0x7f991c0b57b0 2023-11-24T08:00:19.044 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:19.033+0000 7f990affd700 1 -- 172.21.15.59:0/1498465316 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f990c0036a0 con 0x7f991c0b57b0 2023-11-24T08:00:19.044 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:19.033+0000 7f9920ac5700 1 -- 172.21.15.59:0/1498465316 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f991c1404d0 con 0x7f991c0b57b0 2023-11-24T08:00:19.044 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:19.033+0000 7f9920ac5700 1 -- 172.21.15.59:0/1498465316 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f991c140a90 con 0x7f991c0b57b0 2023-11-24T08:00:19.044 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:19.034+0000 7f9920ac5700 1 -- 172.21.15.59:0/1498465316 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f991c068670 con 0x7f991c0b57b0 2023-11-24T08:00:19.044 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:19.035+0000 7f990affd700 1 -- 172.21.15.59:0/1498465316 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 12) v1 ==== 89792+0+0 (secure 0 0 0) 0x7f990c021030 con 0x7f991c0b57b0 2023-11-24T08:00:19.044 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:19.035+0000 7f990affd700 1 --2- 172.21.15.59:0/1498465316 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f98fc06b500 0x7f98fc06d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:00:19.045 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:19.035+0000 7f990affd700 1 -- 172.21.15.59:0/1498465316 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f990c08b540 con 0x7f991c0b57b0 2023-11-24T08:00:19.045 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:19.036+0000 7f991a59c700 1 --2- 172.21.15.59:0/1498465316 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f98fc06b500 0x7f98fc06d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:00:19.045 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:19.036+0000 7f991a59c700 1 --2- 172.21.15.59:0/1498465316 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f98fc06b500 0x7f98fc06d9c0 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7f9910001610 tx=0x7f99100079f0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:00:19.046 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:19.040+0000 7f990affd700 1 -- 172.21.15.59:0/1498465316 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f990c055d40 con 0x7f991c0b57b0 2023-11-24T08:00:19.285 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:19.284+0000 7f9920ac5700 1 -- 172.21.15.59:0/1498465316 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f991c140fb0 con 0x7f991c0b57b0 2023-11-24T08:00:19.286 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:19.284+0000 7f990affd700 1 -- 172.21.15.59:0/1498465316 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 54 v54) v1 ==== 94+0+4912 (secure 0 0 0) 0x7f991c140fb0 con 0x7f991c0b57b0 2023-11-24T08:00:19.286 INFO:teuthology.orchestra.run.smithi059.stdout: 2023-11-24T08:00:19.287 INFO:teuthology.orchestra.run.smithi059.stdout:{"epoch":54,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5876,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.76:6837/1132179319","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6835","nonce":1132179319},{"type":"v1","addr":"172.21.15.76:6837","nonce":1132179319}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":48},{"gid":6487,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.76:6836/453627414","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6834","nonce":453627414},{"type":"v1","addr":"172.21.15.76:6836","nonce":453627414}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":48},{"gid":7035,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6834/4160100816","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6832","nonce":4160100816},{"type":"v1","addr":"172.21.15.59:6834","nonce":4160100816}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":48}],"filesystems":[{"mdsmap":{"epoch":53,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-24T07:59:51.909592+0000","modified":"2023-11-24T07:59:54.948256+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":7038},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7038":{"gid":7038,"name":"c","rank":0,"incarnation":49,"state":"up:active","state_seq":3,"addr":"172.21.15.59:6835/2337335405","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6833","nonce":2337335405},{"type":"v1","addr":"172.21.15.59:6835","nonce":2337335405}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":6}]} 2023-11-24T08:00:19.288 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:19.286+0000 7f9920ac5700 1 -- 172.21.15.59:0/1498465316 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f98fc06b500 msgr2=0x7f98fc06d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:00:19.289 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:19.286+0000 7f9920ac5700 1 --2- 172.21.15.59:0/1498465316 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f98fc06b500 0x7f98fc06d9c0 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7f9910001610 tx=0x7f99100079f0 comp rx=0 tx=0).stop 2023-11-24T08:00:19.289 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:19.286+0000 7f9920ac5700 1 -- 172.21.15.59:0/1498465316 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f991c0b57b0 msgr2=0x7f991c0ae320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:00:19.289 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:19.286+0000 7f9920ac5700 1 --2- 172.21.15.59:0/1498465316 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f991c0b57b0 0x7f991c0ae320 secure :-1 s=READY pgs=1100 cs=0 l=1 rev1=1 crypto rx=0x7f990c009c40 tx=0x7f990c004750 comp rx=0 tx=0).stop 2023-11-24T08:00:19.289 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:19.286+0000 7f9920ac5700 1 -- 172.21.15.59:0/1498465316 shutdown_connections 2023-11-24T08:00:19.289 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:19.287+0000 7f9920ac5700 1 --2- 172.21.15.59:0/1498465316 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f98fc06b500 0x7f98fc06d9c0 unknown :-1 s=CLOSED pgs=504 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:00:19.289 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:19.287+0000 7f9920ac5700 1 --2- 172.21.15.59:0/1498465316 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f991c1478c0 0x7f991c0b0f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:00:19.289 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:19.287+0000 7f9920ac5700 1 --2- 172.21.15.59:0/1498465316 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f991c06b510 0x7f991c13f6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:00:19.290 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:19.287+0000 7f9920ac5700 1 --2- 172.21.15.59:0/1498465316 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f991c0b57b0 0x7f991c0ae320 unknown :-1 s=CLOSED pgs=1100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:00:19.290 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:19.287+0000 7f9920ac5700 1 -- 172.21.15.59:0/1498465316 >> 172.21.15.59:0/1498465316 conn(0x7f991c0d0aa0 msgr2=0x7f991c072730 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:00:19.290 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:19.287+0000 7f9920ac5700 1 -- 172.21.15.59:0/1498465316 shutdown_connections 2023-11-24T08:00:19.290 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:19.287+0000 7f9920ac5700 1 -- 172.21.15.59:0/1498465316 wait complete. 2023-11-24T08:00:19.290 INFO:teuthology.orchestra.run.smithi059.stderr:dumped fsmap epoch 54 2023-11-24T08:00:19.300 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-24T08:00:19.426 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:19.424+0000 7f93e4a19700 1 Processor -- start 2023-11-24T08:00:19.426 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:19.424+0000 7f93e4a19700 1 -- start start 2023-11-24T08:00:19.428 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:19.425+0000 7f93e4a19700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f93e015be40 0x7f93e00af400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:00:19.429 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:19.425+0000 7f93e4a19700 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f93e00d0ec0 0x7f93e00b1550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:00:19.429 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:19.425+0000 7f93e4a19700 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f93e00b1a90 0x7f93e006b7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:00:19.429 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:19.425+0000 7f93e4a19700 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f93e00b1130 con 0x7f93e015be40 2023-11-24T08:00:19.429 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:19.425+0000 7f93e4a19700 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f93e0067420 con 0x7f93e00d0ec0 2023-11-24T08:00:19.429 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:19.425+0000 7f93e4a19700 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f93e00af940 con 0x7f93e00b1a90 2023-11-24T08:00:19.429 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:19.425+0000 7f93de59c700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f93e015be40 0x7f93e00af400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:00:19.429 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:19.426+0000 7f93de59c700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f93e015be40 0x7f93e00af400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:56348/0 (socket says 172.21.15.59:56348) 2023-11-24T08:00:19.429 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:19.426+0000 7f93de59c700 1 -- 172.21.15.59:0/3646330888 learned_addr learned my addr 172.21.15.59:0/3646330888 (peer_addr_for_me v2:172.21.15.59:0/0) 2023-11-24T08:00:19.429 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:19.426+0000 7f93ded9d700 1 --2- 172.21.15.59:0/3646330888 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f93e00b1a90 0x7f93e006b7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:00:19.429 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:19.426+0000 7f93ddd9b700 1 --2- 172.21.15.59:0/3646330888 >> v2:172.21.15.76:3300/0 conn(0x7f93e00d0ec0 0x7f93e00b1550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:00:19.429 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:19.426+0000 7f93de59c700 1 -- 172.21.15.59:0/3646330888 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f93e00b1a90 msgr2=0x7f93e006b7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:00:19.429 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:19.426+0000 7f93de59c700 1 --2- 172.21.15.59:0/3646330888 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f93e00b1a90 0x7f93e006b7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:00:19.429 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:19.426+0000 7f93de59c700 1 -- 172.21.15.59:0/3646330888 >> v2:172.21.15.76:3300/0 conn(0x7f93e00d0ec0 msgr2=0x7f93e00b1550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:00:19.430 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:19.426+0000 7f93de59c700 1 --2- 172.21.15.59:0/3646330888 >> v2:172.21.15.76:3300/0 conn(0x7f93e00d0ec0 0x7f93e00b1550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:00:19.430 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:19.426+0000 7f93de59c700 1 -- 172.21.15.59:0/3646330888 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f93e00b0ab0 con 0x7f93e015be40 2023-11-24T08:00:19.430 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:19.426+0000 7f93ddd9b700 1 --2- 172.21.15.59:0/3646330888 >> v2:172.21.15.76:3300/0 conn(0x7f93e00d0ec0 0x7f93e00b1550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-24T08:00:19.430 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:19.426+0000 7f93de59c700 1 --2- 172.21.15.59:0/3646330888 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f93e015be40 0x7f93e00af400 secure :-1 s=READY pgs=1101 cs=0 l=1 rev1=1 crypto rx=0x7f93e00709e0 tx=0x7f93d4009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c0dd4cf8fe6007f server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:00:19.430 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:19.426+0000 7f93dd59a700 1 -- 172.21.15.59:0/3646330888 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93d4011830 con 0x7f93e015be40 2023-11-24T08:00:19.430 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:19.426+0000 7f93dd59a700 1 -- 172.21.15.59:0/3646330888 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f93d4011990 con 0x7f93e015be40 2023-11-24T08:00:19.430 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:19.426+0000 7f93dd59a700 1 -- 172.21.15.59:0/3646330888 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93d4011d20 con 0x7f93e015be40 2023-11-24T08:00:19.430 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:19.427+0000 7f93e4a19700 1 -- 172.21.15.59:0/3646330888 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f93e015be40 msgr2=0x7f93e00af400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:00:19.430 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:19.427+0000 7f93e4a19700 1 --2- 172.21.15.59:0/3646330888 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f93e015be40 0x7f93e00af400 secure :-1 s=READY pgs=1101 cs=0 l=1 rev1=1 crypto rx=0x7f93e00709e0 tx=0x7f93d4009bd0 comp rx=0 tx=0).stop 2023-11-24T08:00:19.430 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:19.427+0000 7f93e4a19700 1 -- 172.21.15.59:0/3646330888 shutdown_connections 2023-11-24T08:00:19.430 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:19.427+0000 7f93e4a19700 1 --2- 172.21.15.59:0/3646330888 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f93e00b1a90 0x7f93e006b7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:00:19.430 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:19.427+0000 7f93e4a19700 1 --2- 172.21.15.59:0/3646330888 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f93e015be40 0x7f93e00af400 unknown :-1 s=CLOSED pgs=1101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:00:19.430 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:19.427+0000 7f93e4a19700 1 --2- 172.21.15.59:0/3646330888 >> v2:172.21.15.76:3300/0 conn(0x7f93e00d0ec0 0x7f93e00b1550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:00:19.431 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:19.427+0000 7f93e4a19700 1 -- 172.21.15.59:0/3646330888 >> 172.21.15.59:0/3646330888 conn(0x7f93e00717f0 msgr2=0x7f93e00b5070 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:00:19.431 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:19.427+0000 7f93e4a19700 1 -- 172.21.15.59:0/3646330888 shutdown_connections 2023-11-24T08:00:19.431 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:19.427+0000 7f93e4a19700 1 -- 172.21.15.59:0/3646330888 wait complete. 2023-11-24T08:00:19.433 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:19.428+0000 7f93e4a19700 1 Processor -- start 2023-11-24T08:00:19.433 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:19.428+0000 7f93e4a19700 1 -- start start 2023-11-24T08:00:19.433 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:19.429+0000 7f93e4a19700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f93e00d0ec0 0x7f93e01f33b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:00:19.434 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:19.429+0000 7f93e4a19700 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f93e015be40 0x7f93e00c9590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:00:19.434 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:19.429+0000 7f93e4a19700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f93e00c9ad0 0x7f93e00adca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:00:19.434 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:19.429+0000 7f93e4a19700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93e00ca050 con 0x7f93e00d0ec0 2023-11-24T08:00:19.434 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:19.429+0000 7f93e4a19700 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93e00ca330 con 0x7f93e00c9ad0 2023-11-24T08:00:19.434 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:19.429+0000 7f93e4a19700 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f93e00ca610 con 0x7f93e015be40 2023-11-24T08:00:19.434 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:19.429+0000 7f93de59c700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f93e00d0ec0 0x7f93e01f33b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:00:19.434 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:19.429+0000 7f93de59c700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f93e00d0ec0 0x7f93e01f33b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:56354/0 (socket says 172.21.15.59:56354) 2023-11-24T08:00:19.434 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:19.429+0000 7f93de59c700 1 -- 172.21.15.59:0/4077885177 learned_addr learned my addr 172.21.15.59:0/4077885177 (peer_addr_for_me v2:172.21.15.59:0/0) 2023-11-24T08:00:19.434 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:19.429+0000 7f93ded9d700 1 --2- 172.21.15.59:0/4077885177 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f93e00c9ad0 0x7f93e00adca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:00:19.434 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:19.429+0000 7f93ddd9b700 1 --2- 172.21.15.59:0/4077885177 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f93e015be40 0x7f93e00c9590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:00:19.434 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:19.429+0000 7f93de59c700 1 -- 172.21.15.59:0/4077885177 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f93e015be40 msgr2=0x7f93e00c9590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:00:19.434 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:19.429+0000 7f93de59c700 1 --2- 172.21.15.59:0/4077885177 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f93e015be40 0x7f93e00c9590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:00:19.434 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:19.429+0000 7f93de59c700 1 -- 172.21.15.59:0/4077885177 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f93e00c9ad0 msgr2=0x7f93e00adca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:00:19.435 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:19.429+0000 7f93de59c700 1 --2- 172.21.15.59:0/4077885177 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f93e00c9ad0 0x7f93e00adca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:00:19.435 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:19.429+0000 7f93de59c700 1 -- 172.21.15.59:0/4077885177 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f93d4009550 con 0x7f93e00d0ec0 2023-11-24T08:00:19.435 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:19.430+0000 7f93de59c700 1 --2- 172.21.15.59:0/4077885177 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f93e00d0ec0 0x7f93e01f33b0 secure :-1 s=READY pgs=1102 cs=0 l=1 rev1=1 crypto rx=0x7f93d4009c10 tx=0x7f93d4004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:00:19.435 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:19.430+0000 7f93ceffd700 1 -- 172.21.15.59:0/4077885177 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93d4004510 con 0x7f93e00d0ec0 2023-11-24T08:00:19.435 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:19.430+0000 7f93ceffd700 1 -- 172.21.15.59:0/4077885177 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f93d401e070 con 0x7f93e00d0ec0 2023-11-24T08:00:19.435 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:19.430+0000 7f93ceffd700 1 -- 172.21.15.59:0/4077885177 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93d40038e0 con 0x7f93e00d0ec0 2023-11-24T08:00:19.435 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:19.430+0000 7f93e4a19700 1 -- 172.21.15.59:0/4077885177 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f93e00ca8f0 con 0x7f93e00d0ec0 2023-11-24T08:00:19.435 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:19.430+0000 7f93e4a19700 1 -- 172.21.15.59:0/4077885177 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f93e0155cc0 con 0x7f93e00d0ec0 2023-11-24T08:00:19.435 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:19.431+0000 7f93e4a19700 1 -- 172.21.15.59:0/4077885177 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f93e00caf40 con 0x7f93e00d0ec0 2023-11-24T08:00:19.435 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:19.431+0000 7f93ceffd700 1 -- 172.21.15.59:0/4077885177 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 12) v1 ==== 89792+0+0 (secure 0 0 0) 0x7f93d4003a40 con 0x7f93e00d0ec0 2023-11-24T08:00:19.435 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:19.432+0000 7f93ceffd700 1 --2- 172.21.15.59:0/4077885177 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f93c006b500 0x7f93c006d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:00:19.435 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:19.432+0000 7f93ceffd700 1 -- 172.21.15.59:0/4077885177 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f93d408af60 con 0x7f93e00d0ec0 2023-11-24T08:00:19.435 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:19.432+0000 7f93ddd9b700 1 --2- 172.21.15.59:0/4077885177 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f93c006b500 0x7f93c006d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:00:19.436 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:19.432+0000 7f93ddd9b700 1 --2- 172.21.15.59:0/4077885177 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f93c006b500 0x7f93c006d9c0 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7f93e00af0c0 tx=0x7f93d0008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:00:19.438 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:19.436+0000 7f93ceffd700 1 -- 172.21.15.59:0/4077885177 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f93d40556e0 con 0x7f93e00d0ec0 2023-11-24T08:00:19.642 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:19.641+0000 7f93e4a19700 1 -- 172.21.15.59:0/4077885177 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f93e0068670 con 0x7f93e00d0ec0 2023-11-24T08:00:19.644 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:19.642+0000 7f93ceffd700 1 -- 172.21.15.59:0/4077885177 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v87) v1 ==== 74+0+16500 (secure 0 0 0) 0x7f93d4020090 con 0x7f93e00d0ec0 2023-11-24T08:00:19.644 INFO:teuthology.orchestra.run.smithi059.stdout: 2023-11-24T08:00:19.644 INFO:teuthology.orchestra.run.smithi059.stdout:{"epoch":87,"fsid":"9f0e8741-b8f9-48d5-a2c3-09f45903303d","created":"2023-11-24T07:56:06.552729+0000","modified":"2023-11-24T07:59:53.500645+0000","last_up_change":"2023-11-24T07:56:17.372352+0000","last_in_change":"2023-11-24T07:56:09.073250+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":13,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-24T07:56:18.352823+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_metadata","create_time":"2023-11-24T07:59:50.274996+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_data","create_time":"2023-11-24T07:59:50.873919+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"87","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"6d8f9e9b-1782-4e41-928b-a39f2a8ebc06","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6800","nonce":3373878772},{"type":"v1","addr":"172.21.15.59:6801","nonce":3373878772}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6802","nonce":3373878772},{"type":"v1","addr":"172.21.15.59:6803","nonce":3373878772}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6806","nonce":3373878772},{"type":"v1","addr":"172.21.15.59:6807","nonce":3373878772}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6804","nonce":3373878772},{"type":"v1","addr":"172.21.15.59:6805","nonce":3373878772}]},"public_addr":"172.21.15.59:6801/3373878772","cluster_addr":"172.21.15.59:6803/3373878772","heartbeat_back_addr":"172.21.15.59:6807/3373878772","heartbeat_front_addr":"172.21.15.59:6805/3373878772","state":["exists","up"]},{"osd":1,"uuid":"e8d882aa-52e1-4c04-9e1e-c44ef9204a5f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6824","nonce":648258563},{"type":"v1","addr":"172.21.15.59:6825","nonce":648258563}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6826","nonce":648258563},{"type":"v1","addr":"172.21.15.59:6827","nonce":648258563}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6830","nonce":648258563},{"type":"v1","addr":"172.21.15.59:6831","nonce":648258563}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6828","nonce":648258563},{"type":"v1","addr":"172.21.15.59:6829","nonce":648258563}]},"public_addr":"172.21.15.59:6825/648258563","cluster_addr":"172.21.15.59:6827/648258563","heartbeat_back_addr":"172.21.15.59:6831/648258563","heartbeat_front_addr":"172.21.15.59:6829/648258563","state":["exists","up"]},{"osd":2,"uuid":"69eebfa0-695b-4b21-90b9-8a3537168a01","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6808","nonce":1840430654},{"type":"v1","addr":"172.21.15.59:6809","nonce":1840430654}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6810","nonce":1840430654},{"type":"v1","addr":"172.21.15.59:6811","nonce":1840430654}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6814","nonce":1840430654},{"type":"v1","addr":"172.21.15.59:6815","nonce":1840430654}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6812","nonce":1840430654},{"type":"v1","addr":"172.21.15.59:6813","nonce":1840430654}]},"public_addr":"172.21.15.59:6809/1840430654","cluster_addr":"172.21.15.59:6811/1840430654","heartbeat_back_addr":"172.21.15.59:6815/1840430654","heartbeat_front_addr":"172.21.15.59:6813/1840430654","state":["exists","up"]},{"osd":3,"uuid":"96fd77fc-f911-46ff-9445-6f0be2daa8f4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6816","nonce":2504025850},{"type":"v1","addr":"172.21.15.59:6817","nonce":2504025850}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6818","nonce":2504025850},{"type":"v1","addr":"172.21.15.59:6819","nonce":2504025850}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6822","nonce":2504025850},{"type":"v1","addr":"172.21.15.59:6823","nonce":2504025850}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6820","nonce":2504025850},{"type":"v1","addr":"172.21.15.59:6821","nonce":2504025850}]},"public_addr":"172.21.15.59:6817/2504025850","cluster_addr":"172.21.15.59:6819/2504025850","heartbeat_back_addr":"172.21.15.59:6823/2504025850","heartbeat_front_addr":"172.21.15.59:6821/2504025850","state":["exists","up"]},{"osd":4,"uuid":"7ac29fca-b5b2-4c95-9d69-2f164cbbc708","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6816","nonce":2382463431},{"type":"v1","addr":"172.21.15.76:6818","nonce":2382463431}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6820","nonce":2382463431},{"type":"v1","addr":"172.21.15.76:6822","nonce":2382463431}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6828","nonce":2382463431},{"type":"v1","addr":"172.21.15.76:6830","nonce":2382463431}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6824","nonce":2382463431},{"type":"v1","addr":"172.21.15.76:6826","nonce":2382463431}]},"public_addr":"172.21.15.76:6818/2382463431","cluster_addr":"172.21.15.76:6822/2382463431","heartbeat_back_addr":"172.21.15.76:6830/2382463431","heartbeat_front_addr":"172.21.15.76:6826/2382463431","state":["exists","up"]},{"osd":5,"uuid":"485bf711-3851-4238-a4f4-51d52e0a9c58","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6800","nonce":329334308},{"type":"v1","addr":"172.21.15.76:6802","nonce":329334308}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6805","nonce":329334308},{"type":"v1","addr":"172.21.15.76:6807","nonce":329334308}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6814","nonce":329334308},{"type":"v1","addr":"172.21.15.76:6815","nonce":329334308}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6810","nonce":329334308},{"type":"v1","addr":"172.21.15.76:6812","nonce":329334308}]},"public_addr":"172.21.15.76:6802/329334308","cluster_addr":"172.21.15.76:6807/329334308","heartbeat_back_addr":"172.21.15.76:6815/329334308","heartbeat_front_addr":"172.21.15.76:6812/329334308","state":["exists","up"]},{"osd":6,"uuid":"72b3f2fc-1e4a-4e32-968d-003647a0286f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6801","nonce":3946579410},{"type":"v1","addr":"172.21.15.76:6803","nonce":3946579410}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6804","nonce":3946579410},{"type":"v1","addr":"172.21.15.76:6806","nonce":3946579410}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6811","nonce":3946579410},{"type":"v1","addr":"172.21.15.76:6813","nonce":3946579410}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6808","nonce":3946579410},{"type":"v1","addr":"172.21.15.76:6809","nonce":3946579410}]},"public_addr":"172.21.15.76:6803/3946579410","cluster_addr":"172.21.15.76:6806/3946579410","heartbeat_back_addr":"172.21.15.76:6813/3946579410","heartbeat_front_addr":"172.21.15.76:6809/3946579410","state":["exists","up"]},{"osd":7,"uuid":"1ddceb90-6c5d-4d1a-ad56-99cb877f8c70","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6817","nonce":841761538},{"type":"v1","addr":"172.21.15.76:6819","nonce":841761538}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6821","nonce":841761538},{"type":"v1","addr":"172.21.15.76:6823","nonce":841761538}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6829","nonce":841761538},{"type":"v1","addr":"172.21.15.76:6831","nonce":841761538}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6825","nonce":841761538},{"type":"v1","addr":"172.21.15.76:6827","nonce":841761538}]},"public_addr":"172.21.15.76:6819/841761538","cluster_addr":"172.21.15.76:6823/841761538","heartbeat_back_addr":"172.21.15.76:6831/841761538","heartbeat_front_addr":"172.21.15.76:6827/841761538","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T07:56:14.294838+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T07:56:14.270779+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T07:56:14.231809+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T07:56:14.260983+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T07:56:14.302997+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T07:56:14.258600+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T07:56:14.251413+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T07:56:14.304727+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-24T08:00:19.646 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:19.644+0000 7f93e4a19700 1 -- 172.21.15.59:0/4077885177 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f93c006b500 msgr2=0x7f93c006d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:00:19.646 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:19.644+0000 7f93e4a19700 1 --2- 172.21.15.59:0/4077885177 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f93c006b500 0x7f93c006d9c0 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7f93e00af0c0 tx=0x7f93d0008040 comp rx=0 tx=0).stop 2023-11-24T08:00:19.646 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:19.644+0000 7f93e4a19700 1 -- 172.21.15.59:0/4077885177 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f93e00d0ec0 msgr2=0x7f93e01f33b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:00:19.647 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:19.644+0000 7f93e4a19700 1 --2- 172.21.15.59:0/4077885177 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f93e00d0ec0 0x7f93e01f33b0 secure :-1 s=READY pgs=1102 cs=0 l=1 rev1=1 crypto rx=0x7f93d4009c10 tx=0x7f93d4004750 comp rx=0 tx=0).stop 2023-11-24T08:00:19.647 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:19.644+0000 7f93e4a19700 1 -- 172.21.15.59:0/4077885177 shutdown_connections 2023-11-24T08:00:19.647 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:19.644+0000 7f93e4a19700 1 --2- 172.21.15.59:0/4077885177 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f93c006b500 0x7f93c006d9c0 unknown :-1 s=CLOSED pgs=505 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:00:19.647 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:19.644+0000 7f93e4a19700 1 --2- 172.21.15.59:0/4077885177 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f93e00c9ad0 0x7f93e00adca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:00:19.647 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:19.644+0000 7f93e4a19700 1 --2- 172.21.15.59:0/4077885177 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f93e00d0ec0 0x7f93e01f33b0 unknown :-1 s=CLOSED pgs=1102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:00:19.647 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:19.644+0000 7f93e4a19700 1 --2- 172.21.15.59:0/4077885177 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f93e015be40 0x7f93e00c9590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:00:19.647 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:19.644+0000 7f93e4a19700 1 -- 172.21.15.59:0/4077885177 >> 172.21.15.59:0/4077885177 conn(0x7f93e00717f0 msgr2=0x7f93e006d880 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:00:19.647 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:19.644+0000 7f93e4a19700 1 -- 172.21.15.59:0/4077885177 shutdown_connections 2023-11-24T08:00:19.647 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:19.644+0000 7f93e4a19700 1 -- 172.21.15.59:0/4077885177 wait complete. 2023-11-24T08:00:19.658 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2023-11-24T08:00:19.659 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-24T08:00:19.783 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:19.781+0000 7f63dde94700 1 Processor -- start 2023-11-24T08:00:19.783 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:19.781+0000 7f63dde94700 1 -- start start 2023-11-24T08:00:19.785 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:19.782+0000 7f63dde94700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f63d806b400 0x7f63d815e5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:00:19.785 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:19.782+0000 7f63dde94700 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f63d80b5e40 0x7f63d815eb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:00:19.785 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:19.782+0000 7f63dde94700 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f63d806bb80 con 0x7f63d815e1e0 2023-11-24T08:00:19.785 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:19.782+0000 7f63dde94700 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f63d806be20 con 0x7f63d806b400 2023-11-24T08:00:19.785 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:19.782+0000 7f63dde94700 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f63d806c100 con 0x7f63d80b5e40 2023-11-24T08:00:19.785 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:19.782+0000 7f63d6ffd700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f63d806b400 0x7f63d815e5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:00:19.786 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:19.782+0000 7f63d77fe700 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7f63d815e1e0 0x7f63d80b12a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:59836/0 (socket says 172.21.15.59:59836) 2023-11-24T08:00:19.786 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:19.782+0000 7f63d77fe700 1 -- 172.21.15.59:0/2807520940 learned_addr learned my addr 172.21.15.59:0/2807520940 (peer_addr_for_me v1:172.21.15.59:0/0) 2023-11-24T08:00:19.786 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:19.782+0000 7f63d7fff700 1 --2- 172.21.15.59:0/2807520940 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f63d80b5e40 0x7f63d815eb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:00:19.786 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:19.782+0000 7f63d6ffd700 1 -- 172.21.15.59:0/2807520940 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f63d80b5e40 msgr2=0x7f63d815eb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:00:19.786 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:19.782+0000 7f63d6ffd700 1 --2- 172.21.15.59:0/2807520940 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f63d80b5e40 0x7f63d815eb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:00:19.786 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:19.782+0000 7f63d6ffd700 1 -- 172.21.15.59:0/2807520940 >> v1:172.21.15.59:6789/0 conn(0x7f63d815e1e0 legacy=0x7f63d80b12a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:00:19.786 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:19.782+0000 7f63d6ffd700 1 -- 172.21.15.59:0/2807520940 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f63d806c3e0 con 0x7f63d806b400 2023-11-24T08:00:19.786 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:19.782+0000 7f63d67fc700 1 -- 172.21.15.59:0/2807520940 <== mon.0 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1874590092 0 0) 0x7f63d806bb80 con 0x7f63d815e1e0 2023-11-24T08:00:19.786 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:19.783+0000 7f63d6ffd700 1 --2- 172.21.15.59:0/2807520940 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f63d806b400 0x7f63d815e5c0 secure :-1 s=READY pgs=1104 cs=0 l=1 rev1=1 crypto rx=0x7f63c0009840 tx=0x7f63c0009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=87259c9250004e5f server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:00:19.786 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:19.783+0000 7f63d67fc700 1 -- 172.21.15.59:0/2807520940 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63c0011830 con 0x7f63d806b400 2023-11-24T08:00:19.786 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:19.783+0000 7f63d67fc700 1 -- 172.21.15.59:0/2807520940 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f63c0011990 con 0x7f63d806b400 2023-11-24T08:00:19.787 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:19.783+0000 7f63d67fc700 1 -- 172.21.15.59:0/2807520940 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63c0011d20 con 0x7f63d806b400 2023-11-24T08:00:19.787 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:19.783+0000 7f63dde94700 1 -- 172.21.15.59:0/2807520940 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f63d806b400 msgr2=0x7f63d815e5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:00:19.787 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:19.783+0000 7f63dde94700 1 --2- 172.21.15.59:0/2807520940 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f63d806b400 0x7f63d815e5c0 secure :-1 s=READY pgs=1104 cs=0 l=1 rev1=1 crypto rx=0x7f63c0009840 tx=0x7f63c0009b50 comp rx=0 tx=0).stop 2023-11-24T08:00:19.787 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:19.783+0000 7f63dde94700 1 -- 172.21.15.59:0/2807520940 shutdown_connections 2023-11-24T08:00:19.787 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:19.783+0000 7f63dde94700 1 --2- 172.21.15.59:0/2807520940 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f63d80b5e40 0x7f63d815eb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:00:19.787 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:19.783+0000 7f63dde94700 1 --2- 172.21.15.59:0/2807520940 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f63d806b400 0x7f63d815e5c0 unknown :-1 s=CLOSED pgs=1104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:00:19.787 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:19.783+0000 7f63dde94700 1 -- 172.21.15.59:0/2807520940 >> 172.21.15.59:0/2807520940 conn(0x7f63d80b5070 msgr2=0x7f63d80b5730 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:00:19.787 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:19.783+0000 7f63dde94700 1 -- 172.21.15.59:0/2807520940 shutdown_connections 2023-11-24T08:00:19.787 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:19.783+0000 7f63dde94700 1 -- 172.21.15.59:0/2807520940 wait complete. 2023-11-24T08:00:19.789 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:19.785+0000 7f63dde94700 1 Processor -- start 2023-11-24T08:00:19.790 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:19.785+0000 7f63dde94700 1 -- start start 2023-11-24T08:00:19.790 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:19.786+0000 7f63dde94700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f63d806b400 0x7f63d8159bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:00:19.790 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:19.786+0000 7f63dde94700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f63d80b5e40 0x7f63d8157020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:00:19.790 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:19.786+0000 7f63dde94700 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f63d815e1e0 0x7f63d8145980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:00:19.790 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:19.786+0000 7f63dde94700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63d815a1c0 con 0x7f63d806b400 2023-11-24T08:00:19.790 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:19.786+0000 7f63dde94700 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63d815a4a0 con 0x7f63d80b5e40 2023-11-24T08:00:19.790 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:19.786+0000 7f63dde94700 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f63d815a780 con 0x7f63d815e1e0 2023-11-24T08:00:19.790 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:19.786+0000 7f63d77fe700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f63d806b400 0x7f63d8159bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:00:19.790 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:19.786+0000 7f63d77fe700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f63d806b400 0x7f63d8159bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:56368/0 (socket says 172.21.15.59:56368) 2023-11-24T08:00:19.790 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:19.786+0000 7f63d77fe700 1 -- 172.21.15.59:0/2515104044 learned_addr learned my addr 172.21.15.59:0/2515104044 (peer_addr_for_me v2:172.21.15.59:0/0) 2023-11-24T08:00:19.791 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:19.786+0000 7f63d7fff700 1 --2- 172.21.15.59:0/2515104044 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f63d815e1e0 0x7f63d8145980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:00:19.791 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:19.786+0000 7f63d6ffd700 1 --2- 172.21.15.59:0/2515104044 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f63d80b5e40 0x7f63d8157020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:00:19.791 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:19.786+0000 7f63d77fe700 1 -- 172.21.15.59:0/2515104044 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f63d815e1e0 msgr2=0x7f63d8145980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:00:19.791 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:19.786+0000 7f63d77fe700 1 --2- 172.21.15.59:0/2515104044 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f63d815e1e0 0x7f63d8145980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:00:19.791 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:19.786+0000 7f63d77fe700 1 -- 172.21.15.59:0/2515104044 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f63d80b5e40 msgr2=0x7f63d8157020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:00:19.791 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:19.786+0000 7f63d77fe700 1 --2- 172.21.15.59:0/2515104044 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f63d80b5e40 0x7f63d8157020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:00:19.791 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:19.786+0000 7f63d77fe700 1 -- 172.21.15.59:0/2515104044 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f63c00094a0 con 0x7f63d806b400 2023-11-24T08:00:19.791 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:19.786+0000 7f63d77fe700 1 --2- 172.21.15.59:0/2515104044 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f63d806b400 0x7f63d8159bc0 secure :-1 s=READY pgs=1105 cs=0 l=1 rev1=1 crypto rx=0x7f63c800c550 tx=0x7f63c800c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:00:19.791 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:19.786+0000 7f63cffff700 1 -- 172.21.15.59:0/2515104044 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63c8004c80 con 0x7f63d806b400 2023-11-24T08:00:19.791 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:19.786+0000 7f63dde94700 1 -- 172.21.15.59:0/2515104044 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f63d815aa60 con 0x7f63d806b400 2023-11-24T08:00:19.791 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:19.786+0000 7f63cffff700 1 -- 172.21.15.59:0/2515104044 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f63c8004e00 con 0x7f63d806b400 2023-11-24T08:00:19.792 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:19.786+0000 7f63cffff700 1 -- 172.21.15.59:0/2515104044 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63c8007310 con 0x7f63d806b400 2023-11-24T08:00:19.792 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:19.786+0000 7f63dde94700 1 -- 172.21.15.59:0/2515104044 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f63d8157560 con 0x7f63d806b400 2023-11-24T08:00:19.792 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:19.787+0000 7f63dde94700 1 -- 172.21.15.59:0/2515104044 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f63d8068670 con 0x7f63d806b400 2023-11-24T08:00:19.792 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:19.788+0000 7f63cffff700 1 -- 172.21.15.59:0/2515104044 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 12) v1 ==== 89792+0+0 (secure 0 0 0) 0x7f63c8007470 con 0x7f63d806b400 2023-11-24T08:00:19.792 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:19.788+0000 7f63cffff700 1 --2- 172.21.15.59:0/2515104044 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f63b806b500 0x7f63b806d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:00:19.793 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:19.788+0000 7f63cffff700 1 -- 172.21.15.59:0/2515104044 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f63c8089ee0 con 0x7f63d806b400 2023-11-24T08:00:19.793 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:19.788+0000 7f63d6ffd700 1 --2- 172.21.15.59:0/2515104044 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f63b806b500 0x7f63b806d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:00:19.793 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:19.789+0000 7f63d6ffd700 1 --2- 172.21.15.59:0/2515104044 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f63b806b500 0x7f63b806d9c0 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7f63d806b870 tx=0x7f63c0008030 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:00:19.794 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:19.793+0000 7f63cffff700 1 -- 172.21.15.59:0/2515104044 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f63c80545d0 con 0x7f63d806b400 2023-11-24T08:00:20.039 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:20.037+0000 7f63dde94700 1 -- 172.21.15.59:0/2515104044 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f63d8067460 con 0x7f63d806b400 2023-11-24T08:00:20.039 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:20.038+0000 7f63cffff700 1 -- 172.21.15.59:0/2515104044 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 54 v54) v1 ==== 94+0+4912 (secure 0 0 0) 0x7f63c8057bf0 con 0x7f63d806b400 2023-11-24T08:00:20.040 INFO:teuthology.orchestra.run.smithi059.stdout: 2023-11-24T08:00:20.040 INFO:teuthology.orchestra.run.smithi059.stdout:{"epoch":54,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5876,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.76:6837/1132179319","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6835","nonce":1132179319},{"type":"v1","addr":"172.21.15.76:6837","nonce":1132179319}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":48},{"gid":6487,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.76:6836/453627414","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6834","nonce":453627414},{"type":"v1","addr":"172.21.15.76:6836","nonce":453627414}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":48},{"gid":7035,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6834/4160100816","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6832","nonce":4160100816},{"type":"v1","addr":"172.21.15.59:6834","nonce":4160100816}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":48}],"filesystems":[{"mdsmap":{"epoch":53,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-24T07:59:51.909592+0000","modified":"2023-11-24T07:59:54.948256+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":7038},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7038":{"gid":7038,"name":"c","rank":0,"incarnation":49,"state":"up:active","state_seq":3,"addr":"172.21.15.59:6835/2337335405","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6833","nonce":2337335405},{"type":"v1","addr":"172.21.15.59:6835","nonce":2337335405}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":6}]} 2023-11-24T08:00:20.042 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:20.040+0000 7f63dde94700 1 -- 172.21.15.59:0/2515104044 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f63b806b500 msgr2=0x7f63b806d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:00:20.042 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:20.040+0000 7f63dde94700 1 --2- 172.21.15.59:0/2515104044 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f63b806b500 0x7f63b806d9c0 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7f63d806b870 tx=0x7f63c0008030 comp rx=0 tx=0).stop 2023-11-24T08:00:20.042 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:20.040+0000 7f63dde94700 1 -- 172.21.15.59:0/2515104044 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f63d806b400 msgr2=0x7f63d8159bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:00:20.042 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:20.040+0000 7f63dde94700 1 --2- 172.21.15.59:0/2515104044 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f63d806b400 0x7f63d8159bc0 secure :-1 s=READY pgs=1105 cs=0 l=1 rev1=1 crypto rx=0x7f63c800c550 tx=0x7f63c800c910 comp rx=0 tx=0).stop 2023-11-24T08:00:20.042 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:20.040+0000 7f63dde94700 1 -- 172.21.15.59:0/2515104044 shutdown_connections 2023-11-24T08:00:20.043 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:20.040+0000 7f63dde94700 1 --2- 172.21.15.59:0/2515104044 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f63b806b500 0x7f63b806d9c0 unknown :-1 s=CLOSED pgs=506 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:00:20.043 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:20.040+0000 7f63dde94700 1 --2- 172.21.15.59:0/2515104044 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f63d815e1e0 0x7f63d8145980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:00:20.043 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:20.040+0000 7f63dde94700 1 --2- 172.21.15.59:0/2515104044 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f63d806b400 0x7f63d8159bc0 unknown :-1 s=CLOSED pgs=1105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:00:20.043 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:20.040+0000 7f63dde94700 1 --2- 172.21.15.59:0/2515104044 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f63d80b5e40 0x7f63d8157020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:00:20.043 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:20.040+0000 7f63dde94700 1 -- 172.21.15.59:0/2515104044 >> 172.21.15.59:0/2515104044 conn(0x7f63d80b5070 msgr2=0x7f63d815f420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:00:20.043 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:20.040+0000 7f63dde94700 1 -- 172.21.15.59:0/2515104044 shutdown_connections 2023-11-24T08:00:20.044 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:20.040+0000 7f63dde94700 1 -- 172.21.15.59:0/2515104044 wait complete. 2023-11-24T08:00:20.044 INFO:teuthology.orchestra.run.smithi059.stderr:dumped fsmap epoch 54 2023-11-24T08:00:20.054 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-24T08:00:20.179 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:20.177+0000 7ff1a2373700 1 Processor -- start 2023-11-24T08:00:20.179 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:20.177+0000 7ff1a2373700 1 -- start start 2023-11-24T08:00:20.181 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:20.178+0000 7ff1a2373700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7ff19c0722c0 0x7ff19c0b51b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:00:20.181 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:20.178+0000 7ff1a2373700 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff19c143ec0 con 0x7ff19c1440f0 2023-11-24T08:00:20.181 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:20.178+0000 7ff1a2373700 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff19c0b3ae0 con 0x7ff19c143ae0 2023-11-24T08:00:20.182 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:20.178+0000 7ff1a2373700 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff19c0b40a0 con 0x7ff19c0722c0 2023-11-24T08:00:20.182 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:20.178+0000 7ff19b7fe700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7ff19c0722c0 0x7ff19c0b51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:00:20.182 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:20.178+0000 7ff19b7fe700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7ff19c0722c0 0x7ff19c0b51b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:56378/0 (socket says 172.21.15.59:56378) 2023-11-24T08:00:20.182 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:20.178+0000 7ff1a0910700 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7ff19c1440f0 0x7ff19c06e510 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:59854/0 (socket says 172.21.15.59:59854) 2023-11-24T08:00:20.182 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:20.178+0000 7ff19b7fe700 1 -- 172.21.15.59:0/2605101234 learned_addr learned my addr 172.21.15.59:0/2605101234 (peer_addr_for_me v2:172.21.15.59:0/0) 2023-11-24T08:00:20.182 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:20.178+0000 7ff19b7fe700 1 -- 172.21.15.59:0/2605101234 >> v1:172.21.15.76:6789/0 conn(0x7ff19c143ae0 legacy=0x7ff19c071bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:00:20.183 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:20.178+0000 7ff19b7fe700 1 -- 172.21.15.59:0/2605101234 >> v1:172.21.15.59:6789/0 conn(0x7ff19c1440f0 legacy=0x7ff19c06e510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:00:20.183 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:20.178+0000 7ff19b7fe700 1 -- 172.21.15.59:0/2605101234 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff19c0b3dc0 con 0x7ff19c0722c0 2023-11-24T08:00:20.183 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:20.179+0000 7ff19b7fe700 1 --2- 172.21.15.59:0/2605101234 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff19c0722c0 0x7ff19c0b51b0 secure :-1 s=READY pgs=1107 cs=0 l=1 rev1=1 crypto rx=0x7ff1900098f0 tx=0x7ff190009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2e13548035d49f8f server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:00:20.183 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:20.179+0000 7ff19affd700 1 -- 172.21.15.59:0/2605101234 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff19000f8a0 con 0x7ff19c0722c0 2023-11-24T08:00:20.183 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:20.179+0000 7ff19affd700 1 -- 172.21.15.59:0/2605101234 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff19000fa00 con 0x7ff19c0722c0 2023-11-24T08:00:20.184 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:20.179+0000 7ff19affd700 1 -- 172.21.15.59:0/2605101234 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff19000fd90 con 0x7ff19c0722c0 2023-11-24T08:00:20.184 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:20.179+0000 7ff1a2373700 1 -- 172.21.15.59:0/2605101234 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff19c0722c0 msgr2=0x7ff19c0b51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:00:20.184 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:20.179+0000 7ff1a2373700 1 --2- 172.21.15.59:0/2605101234 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff19c0722c0 0x7ff19c0b51b0 secure :-1 s=READY pgs=1107 cs=0 l=1 rev1=1 crypto rx=0x7ff1900098f0 tx=0x7ff190009c00 comp rx=0 tx=0).stop 2023-11-24T08:00:20.184 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:20.179+0000 7ff1a2373700 1 -- 172.21.15.59:0/2605101234 shutdown_connections 2023-11-24T08:00:20.184 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:20.179+0000 7ff1a2373700 1 --2- 172.21.15.59:0/2605101234 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff19c0722c0 0x7ff19c0b51b0 unknown :-1 s=CLOSED pgs=1107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:00:20.184 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:20.179+0000 7ff1a2373700 1 -- 172.21.15.59:0/2605101234 >> 172.21.15.59:0/2605101234 conn(0x7ff19c0b2740 msgr2=0x7ff19c0b2b20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:00:20.184 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:20.179+0000 7ff1a2373700 1 -- 172.21.15.59:0/2605101234 shutdown_connections 2023-11-24T08:00:20.184 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:20.179+0000 7ff1a2373700 1 -- 172.21.15.59:0/2605101234 wait complete. 2023-11-24T08:00:20.187 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:20.181+0000 7ff1a2373700 1 Processor -- start 2023-11-24T08:00:20.187 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:20.181+0000 7ff1a2373700 1 -- start start 2023-11-24T08:00:20.187 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:20.181+0000 7ff1a2373700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff19c0722c0 0x7ff19c1410b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:00:20.187 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:20.181+0000 7ff1a2373700 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7ff19c143ae0 0x7ff19c1417b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:00:20.187 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:20.181+0000 7ff1a2373700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff19c1440f0 0x7ff19c1426a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:00:20.188 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:20.181+0000 7ff1a2373700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff19c1415f0 con 0x7ff19c0722c0 2023-11-24T08:00:20.188 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:20.181+0000 7ff1a2373700 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff19c0c9dc0 con 0x7ff19c1440f0 2023-11-24T08:00:20.188 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:20.181+0000 7ff1a2373700 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff19c0ca010 con 0x7ff19c143ae0 2023-11-24T08:00:20.188 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:20.182+0000 7ff19bfff700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff19c0722c0 0x7ff19c1410b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:00:20.188 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:20.182+0000 7ff19bfff700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff19c0722c0 0x7ff19c1410b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:56382/0 (socket says 172.21.15.59:56382) 2023-11-24T08:00:20.188 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:20.182+0000 7ff19bfff700 1 -- 172.21.15.59:0/1575429910 learned_addr learned my addr 172.21.15.59:0/1575429910 (peer_addr_for_me v2:172.21.15.59:0/0) 2023-11-24T08:00:20.188 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:20.182+0000 7ff1a0910700 1 --2- 172.21.15.59:0/1575429910 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff19c1440f0 0x7ff19c1426a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:00:20.188 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:20.182+0000 7ff19b7fe700 1 --2- 172.21.15.59:0/1575429910 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7ff19c143ae0 0x7ff19c1417b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:00:20.188 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:20.182+0000 7ff19bfff700 1 -- 172.21.15.59:0/1575429910 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7ff19c143ae0 msgr2=0x7ff19c1417b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:00:20.188 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:20.182+0000 7ff19bfff700 1 --2- 172.21.15.59:0/1575429910 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7ff19c143ae0 0x7ff19c1417b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:00:20.188 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:20.182+0000 7ff19bfff700 1 -- 172.21.15.59:0/1575429910 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff19c1440f0 msgr2=0x7ff19c1426a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:00:20.189 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:20.182+0000 7ff19bfff700 1 --2- 172.21.15.59:0/1575429910 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff19c1440f0 0x7ff19c1426a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:00:20.189 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:20.182+0000 7ff19bfff700 1 -- 172.21.15.59:0/1575429910 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff190009550 con 0x7ff19c0722c0 2023-11-24T08:00:20.189 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:20.182+0000 7ff1a0910700 1 --2- 172.21.15.59:0/1575429910 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff19c1440f0 0x7ff19c1426a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-24T08:00:20.189 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:20.182+0000 7ff19bfff700 1 --2- 172.21.15.59:0/1575429910 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff19c0722c0 0x7ff19c1410b0 secure :-1 s=READY pgs=1108 cs=0 l=1 rev1=1 crypto rx=0x7ff18c00c4e0 tx=0x7ff18c00c8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:00:20.189 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:20.182+0000 7ff198ff9700 1 -- 172.21.15.59:0/1575429910 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff18c0052d0 con 0x7ff19c0722c0 2023-11-24T08:00:20.189 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:20.182+0000 7ff198ff9700 1 -- 172.21.15.59:0/1575429910 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff18c005430 con 0x7ff19c0722c0 2023-11-24T08:00:20.189 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:20.182+0000 7ff1a2373700 1 -- 172.21.15.59:0/1575429910 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff19c0ca2f0 con 0x7ff19c0722c0 2023-11-24T08:00:20.189 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:20.182+0000 7ff198ff9700 1 -- 172.21.15.59:0/1575429910 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff18c0038c0 con 0x7ff19c0722c0 2023-11-24T08:00:20.189 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:20.182+0000 7ff1a2373700 1 -- 172.21.15.59:0/1575429910 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff19c0ca8b0 con 0x7ff19c0722c0 2023-11-24T08:00:20.189 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:20.183+0000 7ff1a2373700 1 -- 172.21.15.59:0/1575429910 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff19c06eba0 con 0x7ff19c0722c0 2023-11-24T08:00:20.190 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:20.184+0000 7ff198ff9700 1 -- 172.21.15.59:0/1575429910 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 12) v1 ==== 89792+0+0 (secure 0 0 0) 0x7ff18c003a20 con 0x7ff19c0722c0 2023-11-24T08:00:20.190 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:20.184+0000 7ff198ff9700 1 --2- 172.21.15.59:0/1575429910 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7ff18806b500 0x7ff18806d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:00:20.190 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:20.184+0000 7ff198ff9700 1 -- 172.21.15.59:0/1575429910 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 5709+0+0 (secure 0 0 0) 0x7ff18c0895e0 con 0x7ff19c0722c0 2023-11-24T08:00:20.190 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:20.184+0000 7ff19b7fe700 1 --2- 172.21.15.59:0/1575429910 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7ff18806b500 0x7ff18806d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:00:20.190 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:20.185+0000 7ff19b7fe700 1 --2- 172.21.15.59:0/1575429910 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7ff18806b500 0x7ff18806d9c0 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7ff190000c00 tx=0x7ff190008030 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:00:20.190 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:20.188+0000 7ff198ff9700 1 -- 172.21.15.59:0/1575429910 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7ff18c053d50 con 0x7ff19c0722c0 2023-11-24T08:00:20.438 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:20.436+0000 7ff1a2373700 1 -- 172.21.15.59:0/1575429910 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ff19c068670 con 0x7ff19c0722c0 2023-11-24T08:00:20.439 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:20.437+0000 7ff198ff9700 1 -- 172.21.15.59:0/1575429910 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 54 v54) v1 ==== 94+0+4912 (secure 0 0 0) 0x7ff18c057370 con 0x7ff19c0722c0 2023-11-24T08:00:20.439 INFO:teuthology.orchestra.run.smithi059.stdout: 2023-11-24T08:00:20.440 INFO:teuthology.orchestra.run.smithi059.stdout:{"epoch":54,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5876,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.76:6837/1132179319","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6835","nonce":1132179319},{"type":"v1","addr":"172.21.15.76:6837","nonce":1132179319}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":48},{"gid":6487,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.76:6836/453627414","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6834","nonce":453627414},{"type":"v1","addr":"172.21.15.76:6836","nonce":453627414}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":48},{"gid":7035,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6834/4160100816","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6832","nonce":4160100816},{"type":"v1","addr":"172.21.15.59:6834","nonce":4160100816}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":48}],"filesystems":[{"mdsmap":{"epoch":53,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-24T07:59:51.909592+0000","modified":"2023-11-24T07:59:54.948256+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":7038},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7038":{"gid":7038,"name":"c","rank":0,"incarnation":49,"state":"up:active","state_seq":3,"addr":"172.21.15.59:6835/2337335405","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6833","nonce":2337335405},{"type":"v1","addr":"172.21.15.59:6835","nonce":2337335405}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":6}]} 2023-11-24T08:00:20.442 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:20.439+0000 7ff1a2373700 1 -- 172.21.15.59:0/1575429910 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7ff18806b500 msgr2=0x7ff18806d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:00:20.442 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:20.439+0000 7ff1a2373700 1 --2- 172.21.15.59:0/1575429910 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7ff18806b500 0x7ff18806d9c0 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7ff190000c00 tx=0x7ff190008030 comp rx=0 tx=0).stop 2023-11-24T08:00:20.442 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:20.439+0000 7ff1a2373700 1 -- 172.21.15.59:0/1575429910 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff19c0722c0 msgr2=0x7ff19c1410b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:00:20.442 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:20.439+0000 7ff1a2373700 1 --2- 172.21.15.59:0/1575429910 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff19c0722c0 0x7ff19c1410b0 secure :-1 s=READY pgs=1108 cs=0 l=1 rev1=1 crypto rx=0x7ff18c00c4e0 tx=0x7ff18c00c8a0 comp rx=0 tx=0).stop 2023-11-24T08:00:20.442 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:20.439+0000 7ff1a2373700 1 -- 172.21.15.59:0/1575429910 shutdown_connections 2023-11-24T08:00:20.442 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:20.439+0000 7ff1a2373700 1 --2- 172.21.15.59:0/1575429910 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7ff18806b500 0x7ff18806d9c0 unknown :-1 s=CLOSED pgs=507 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:00:20.442 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:20.439+0000 7ff1a2373700 1 --2- 172.21.15.59:0/1575429910 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff19c1440f0 0x7ff19c1426a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:00:20.442 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:20.439+0000 7ff1a2373700 1 --2- 172.21.15.59:0/1575429910 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff19c0722c0 0x7ff19c1410b0 unknown :-1 s=CLOSED pgs=1108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:00:20.443 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:20.439+0000 7ff1a2373700 1 --2- 172.21.15.59:0/1575429910 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7ff19c143ae0 0x7ff19c1417b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:00:20.443 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:20.439+0000 7ff1a2373700 1 -- 172.21.15.59:0/1575429910 >> 172.21.15.59:0/1575429910 conn(0x7ff19c0b2740 msgr2=0x7ff19c06afd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:00:20.443 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:20.439+0000 7ff1a2373700 1 -- 172.21.15.59:0/1575429910 shutdown_connections 2023-11-24T08:00:20.443 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:20.439+0000 7ff1a2373700 1 -- 172.21.15.59:0/1575429910 wait complete. 2023-11-24T08:00:20.443 INFO:teuthology.orchestra.run.smithi059.stderr:dumped fsmap epoch 54 2023-11-24T08:00:20.454 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-24T08:00:20.579 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:20.578+0000 7f606a65a700 1 Processor -- start 2023-11-24T08:00:20.580 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:20.578+0000 7f606a65a700 1 -- start start 2023-11-24T08:00:20.583 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:20.578+0000 7f606a65a700 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f606415dfc0 0x7f60640d4960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:00:20.583 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:20.578+0000 7f606a65a700 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f60640d2790 con 0x7f60640d1440 2023-11-24T08:00:20.583 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:20.578+0000 7f606a65a700 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f60640d2a70 con 0x7f606415e3a0 2023-11-24T08:00:20.583 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:20.578+0000 7f606a65a700 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f60640c7a30 con 0x7f606415dfc0 2023-11-24T08:00:20.583 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:20.579+0000 7f60637fe700 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7f60640d1440 0x7f60640d0830 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:59864/0 (socket says 172.21.15.59:59864) 2023-11-24T08:00:20.583 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:20.579+0000 7f60637fe700 1 -- 172.21.15.59:0/2369754484 learned_addr learned my addr 172.21.15.59:0/2369754484 (peer_addr_for_me v1:172.21.15.59:0/0) 2023-11-24T08:00:20.583 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:20.579+0000 7f6063fff700 1 --2- 172.21.15.59:0/2369754484 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f606415dfc0 0x7f60640d4960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:00:20.583 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:20.579+0000 7f6062ffd700 1 -- 172.21.15.59:0/2369754484 <== mon.0 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3337224873 0 0) 0x7f60640d2790 con 0x7f60640d1440 2023-11-24T08:00:20.584 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:20.579+0000 7f6062ffd700 1 -- 172.21.15.59:0/2369754484 --> v1:172.21.15.59:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f604c0035b0 con 0x7f60640d1440 2023-11-24T08:00:20.584 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:20.579+0000 7f6062ffd700 1 -- 172.21.15.59:0/2369754484 <== mon.0 v1:172.21.15.59:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 764802505 0 0) 0x7f604c0035b0 con 0x7f60640d1440 2023-11-24T08:00:20.584 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:20.579+0000 7f6062ffd700 1 -- 172.21.15.59:0/2369754484 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f606415dfc0 msgr2=0x7f60640d4960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:00:20.584 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:20.579+0000 7f6062ffd700 1 --2- 172.21.15.59:0/2369754484 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f606415dfc0 0x7f60640d4960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:00:20.584 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:20.579+0000 7f6062ffd700 1 -- 172.21.15.59:0/2369754484 >> v1:172.21.15.76:6789/0 conn(0x7f606415e3a0 legacy=0x7f60640c8230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:00:20.584 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:20.579+0000 7f6062ffd700 1 -- 172.21.15.59:0/2369754484 --> v1:172.21.15.59:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f60640c7d10 con 0x7f60640d1440 2023-11-24T08:00:20.584 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:20.579+0000 7f6062ffd700 1 -- 172.21.15.59:0/2369754484 <== mon.0 v1:172.21.15.59:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1312333566 0 0) 0x7f6058001670 con 0x7f60640d1440 2023-11-24T08:00:20.584 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:20.579+0000 7f6062ffd700 1 -- 172.21.15.59:0/2369754484 >> v1:172.21.15.59:6789/0 conn(0x7f60640d1440 legacy=0x7f60640d0830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:00:20.584 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:20.579+0000 7f6062ffd700 1 --2- 172.21.15.59:0/2369754484 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f604c003a30 0x7f604c003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:00:20.584 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:20.579+0000 7f6062ffd700 1 --2- 172.21.15.59:0/2369754484 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f606415dfc0 0x7f604c004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:00:20.584 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:20.579+0000 7f6062ffd700 1 --2- 172.21.15.59:0/2369754484 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f604c004d90 0x7f604c005210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:00:20.584 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:20.579+0000 7f6062ffd700 1 -- 172.21.15.59:0/2369754484 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60640c7a30 con 0x7f604c003a30 2023-11-24T08:00:20.585 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:20.579+0000 7f6062ffd700 1 -- 172.21.15.59:0/2369754484 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f604c00ba10 con 0x7f604c004d90 2023-11-24T08:00:20.585 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:20.579+0000 7f6062ffd700 1 -- 172.21.15.59:0/2369754484 --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f604c00bbd0 con 0x7f606415dfc0 2023-11-24T08:00:20.585 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:20.579+0000 7f6063fff700 1 --2- 172.21.15.59:0/2369754484 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f604c003a30 0x7f604c003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:00:20.585 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:20.580+0000 7f60637fe700 1 --2- 172.21.15.59:0/2369754484 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f606415dfc0 0x7f604c004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:00:20.585 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:20.580+0000 7f6068bf7700 1 --2- 172.21.15.59:0/2369754484 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f604c004d90 0x7f604c005210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:00:20.585 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:20.580+0000 7f6063fff700 1 -- 172.21.15.59:0/2369754484 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f606415dfc0 msgr2=0x7f604c004850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:00:20.585 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:20.580+0000 7f6063fff700 1 --2- 172.21.15.59:0/2369754484 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f606415dfc0 0x7f604c004850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:00:20.585 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:20.580+0000 7f6063fff700 1 -- 172.21.15.59:0/2369754484 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f604c004d90 msgr2=0x7f604c005210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:00:20.586 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:20.580+0000 7f6063fff700 1 --2- 172.21.15.59:0/2369754484 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f604c004d90 0x7f604c005210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:00:20.586 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:20.580+0000 7f6063fff700 1 -- 172.21.15.59:0/2369754484 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f604c0035b0 con 0x7f604c003a30 2023-11-24T08:00:20.586 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:20.580+0000 7f6068bf7700 1 --2- 172.21.15.59:0/2369754484 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f604c004d90 0x7f604c005210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-24T08:00:20.586 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:20.580+0000 7f60637fe700 1 --2- 172.21.15.59:0/2369754484 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f606415dfc0 0x7f604c004850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-24T08:00:20.586 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:20.580+0000 7f6063fff700 1 --2- 172.21.15.59:0/2369754484 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f604c003a30 0x7f604c003e10 secure :-1 s=READY pgs=1110 cs=0 l=1 rev1=1 crypto rx=0x7f6054009b60 tx=0x7f604c005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bfa68b4c1f6f3b64 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:00:20.586 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:20.580+0000 7f6062ffd700 1 -- 172.21.15.59:0/2369754484 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f605400e6c0 con 0x7f604c003a30 2023-11-24T08:00:20.586 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:20.580+0000 7f6062ffd700 1 -- 172.21.15.59:0/2369754484 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f605400e820 con 0x7f604c003a30 2023-11-24T08:00:20.586 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:20.580+0000 7f6062ffd700 1 -- 172.21.15.59:0/2369754484 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f605400eb90 con 0x7f604c003a30 2023-11-24T08:00:20.586 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:20.581+0000 7f606a65a700 1 -- 172.21.15.59:0/2369754484 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f604c003a30 msgr2=0x7f604c003e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:00:20.586 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:20.581+0000 7f606a65a700 1 --2- 172.21.15.59:0/2369754484 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f604c003a30 0x7f604c003e10 secure :-1 s=READY pgs=1110 cs=0 l=1 rev1=1 crypto rx=0x7f6054009b60 tx=0x7f604c005880 comp rx=0 tx=0).stop 2023-11-24T08:00:20.587 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:20.581+0000 7f6068bf7700 1 -- 172.21.15.59:0/2369754484 reap_dead start 2023-11-24T08:00:20.587 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:20.581+0000 7f606a65a700 1 -- 172.21.15.59:0/2369754484 shutdown_connections 2023-11-24T08:00:20.587 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:20.581+0000 7f606a65a700 1 -- 172.21.15.59:0/2369754484 >> 172.21.15.59:0/2369754484 conn(0x7f60640d3e40 msgr2=0x7f60640d4250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:00:20.587 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:20.581+0000 7f606a65a700 1 -- 172.21.15.59:0/2369754484 shutdown_connections 2023-11-24T08:00:20.587 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:20.581+0000 7f606a65a700 1 -- 172.21.15.59:0/2369754484 wait complete. 2023-11-24T08:00:20.590 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:20.582+0000 7f606a65a700 1 Processor -- start 2023-11-24T08:00:20.590 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:20.582+0000 7f606a65a700 1 -- start start 2023-11-24T08:00:20.590 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:20.583+0000 7f606a65a700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f604c003a30 0x7f606414e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:00:20.590 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:20.583+0000 7f606a65a700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f60640caf10 0x7f60640cd3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:00:20.590 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:20.583+0000 7f606a65a700 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f60640cb390 0x7f60640b24b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:00:20.590 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:20.583+0000 7f606a65a700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f606414e830 con 0x7f604c003a30 2023-11-24T08:00:20.590 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:20.583+0000 7f606a65a700 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60640b0a70 con 0x7f60640caf10 2023-11-24T08:00:20.590 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:20.583+0000 7f606a65a700 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f60640cde90 con 0x7f60640cb390 2023-11-24T08:00:20.590 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:20.583+0000 7f6063fff700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f604c003a30 0x7f606414e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:00:20.591 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:20.583+0000 7f6063fff700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f604c003a30 0x7f606414e2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:56402/0 (socket says 172.21.15.59:56402) 2023-11-24T08:00:20.591 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:20.583+0000 7f6063fff700 1 -- 172.21.15.59:0/2398111010 learned_addr learned my addr 172.21.15.59:0/2398111010 (peer_addr_for_me v2:172.21.15.59:0/0) 2023-11-24T08:00:20.591 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:20.583+0000 7f6068bf7700 1 --2- 172.21.15.59:0/2398111010 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f60640cb390 0x7f60640b24b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:00:20.591 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:20.583+0000 7f60637fe700 1 --2- 172.21.15.59:0/2398111010 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f60640caf10 0x7f60640cd3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:00:20.591 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:20.583+0000 7f6063fff700 1 -- 172.21.15.59:0/2398111010 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f60640cb390 msgr2=0x7f60640b24b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:00:20.591 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:20.583+0000 7f6063fff700 1 --2- 172.21.15.59:0/2398111010 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f60640cb390 0x7f60640b24b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:00:20.591 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:20.583+0000 7f6063fff700 1 -- 172.21.15.59:0/2398111010 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f60640caf10 msgr2=0x7f60640cd3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:00:20.591 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:20.583+0000 7f6063fff700 1 --2- 172.21.15.59:0/2398111010 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f60640caf10 0x7f60640cd3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:00:20.591 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:20.583+0000 7f6063fff700 1 -- 172.21.15.59:0/2398111010 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f60640b0cc0 con 0x7f604c003a30 2023-11-24T08:00:20.591 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:20.584+0000 7f6063fff700 1 --2- 172.21.15.59:0/2398111010 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f604c003a30 0x7f606414e2f0 secure :-1 s=READY pgs=1111 cs=0 l=1 rev1=1 crypto rx=0x7f604c005830 tx=0x7f60540047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:00:20.591 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:20.584+0000 7f6060ff9700 1 -- 172.21.15.59:0/2398111010 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6054021070 con 0x7f604c003a30 2023-11-24T08:00:20.591 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:20.584+0000 7f6060ff9700 1 -- 172.21.15.59:0/2398111010 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6054009ed0 con 0x7f604c003a30 2023-11-24T08:00:20.592 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:20.584+0000 7f6060ff9700 1 -- 172.21.15.59:0/2398111010 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f605400e6c0 con 0x7f604c003a30 2023-11-24T08:00:20.592 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:20.584+0000 7f606a65a700 1 -- 172.21.15.59:0/2398111010 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f60640b0fa0 con 0x7f604c003a30 2023-11-24T08:00:20.592 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:20.584+0000 7f606a65a700 1 -- 172.21.15.59:0/2398111010 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f60641f5160 con 0x7f604c003a30 2023-11-24T08:00:20.592 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:20.585+0000 7f606a65a700 1 -- 172.21.15.59:0/2398111010 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f60640b15f0 con 0x7f604c003a30 2023-11-24T08:00:20.592 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:20.585+0000 7f6060ff9700 1 -- 172.21.15.59:0/2398111010 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 12) v1 ==== 89792+0+0 (secure 0 0 0) 0x7f6054025070 con 0x7f604c003a30 2023-11-24T08:00:20.592 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:20.585+0000 7f6060ff9700 1 --2- 172.21.15.59:0/2398111010 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f605006b500 0x7f605006d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:00:20.592 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:20.586+0000 7f6060ff9700 1 -- 172.21.15.59:0/2398111010 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f605408b230 con 0x7f604c003a30 2023-11-24T08:00:20.593 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:20.586+0000 7f60637fe700 1 --2- 172.21.15.59:0/2398111010 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f605006b500 0x7f605006d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:00:20.593 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:20.586+0000 7f60637fe700 1 --2- 172.21.15.59:0/2398111010 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f605006b500 0x7f605006d9c0 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7f6058001cb0 tx=0x7f6058001150 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:00:20.593 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:20.590+0000 7f6060ff9700 1 -- 172.21.15.59:0/2398111010 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f6054055a30 con 0x7f604c003a30 2023-11-24T08:00:20.792 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:20.790+0000 7f606a65a700 1 -- 172.21.15.59:0/2398111010 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f6064068670 con 0x7f604c003a30 2023-11-24T08:00:20.793 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:20.791+0000 7f6060ff9700 1 -- 172.21.15.59:0/2398111010 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v87) v1 ==== 74+0+16500 (secure 0 0 0) 0x7f6054059050 con 0x7f604c003a30 2023-11-24T08:00:20.793 INFO:teuthology.orchestra.run.smithi059.stdout: 2023-11-24T08:00:20.794 INFO:teuthology.orchestra.run.smithi059.stdout:{"epoch":87,"fsid":"9f0e8741-b8f9-48d5-a2c3-09f45903303d","created":"2023-11-24T07:56:06.552729+0000","modified":"2023-11-24T07:59:53.500645+0000","last_up_change":"2023-11-24T07:56:17.372352+0000","last_in_change":"2023-11-24T07:56:09.073250+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":13,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-24T07:56:18.352823+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_metadata","create_time":"2023-11-24T07:59:50.274996+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_data","create_time":"2023-11-24T07:59:50.873919+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"87","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"6d8f9e9b-1782-4e41-928b-a39f2a8ebc06","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6800","nonce":3373878772},{"type":"v1","addr":"172.21.15.59:6801","nonce":3373878772}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6802","nonce":3373878772},{"type":"v1","addr":"172.21.15.59:6803","nonce":3373878772}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6806","nonce":3373878772},{"type":"v1","addr":"172.21.15.59:6807","nonce":3373878772}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6804","nonce":3373878772},{"type":"v1","addr":"172.21.15.59:6805","nonce":3373878772}]},"public_addr":"172.21.15.59:6801/3373878772","cluster_addr":"172.21.15.59:6803/3373878772","heartbeat_back_addr":"172.21.15.59:6807/3373878772","heartbeat_front_addr":"172.21.15.59:6805/3373878772","state":["exists","up"]},{"osd":1,"uuid":"e8d882aa-52e1-4c04-9e1e-c44ef9204a5f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6824","nonce":648258563},{"type":"v1","addr":"172.21.15.59:6825","nonce":648258563}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6826","nonce":648258563},{"type":"v1","addr":"172.21.15.59:6827","nonce":648258563}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6830","nonce":648258563},{"type":"v1","addr":"172.21.15.59:6831","nonce":648258563}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6828","nonce":648258563},{"type":"v1","addr":"172.21.15.59:6829","nonce":648258563}]},"public_addr":"172.21.15.59:6825/648258563","cluster_addr":"172.21.15.59:6827/648258563","heartbeat_back_addr":"172.21.15.59:6831/648258563","heartbeat_front_addr":"172.21.15.59:6829/648258563","state":["exists","up"]},{"osd":2,"uuid":"69eebfa0-695b-4b21-90b9-8a3537168a01","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6808","nonce":1840430654},{"type":"v1","addr":"172.21.15.59:6809","nonce":1840430654}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6810","nonce":1840430654},{"type":"v1","addr":"172.21.15.59:6811","nonce":1840430654}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6814","nonce":1840430654},{"type":"v1","addr":"172.21.15.59:6815","nonce":1840430654}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6812","nonce":1840430654},{"type":"v1","addr":"172.21.15.59:6813","nonce":1840430654}]},"public_addr":"172.21.15.59:6809/1840430654","cluster_addr":"172.21.15.59:6811/1840430654","heartbeat_back_addr":"172.21.15.59:6815/1840430654","heartbeat_front_addr":"172.21.15.59:6813/1840430654","state":["exists","up"]},{"osd":3,"uuid":"96fd77fc-f911-46ff-9445-6f0be2daa8f4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6816","nonce":2504025850},{"type":"v1","addr":"172.21.15.59:6817","nonce":2504025850}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6818","nonce":2504025850},{"type":"v1","addr":"172.21.15.59:6819","nonce":2504025850}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6822","nonce":2504025850},{"type":"v1","addr":"172.21.15.59:6823","nonce":2504025850}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6820","nonce":2504025850},{"type":"v1","addr":"172.21.15.59:6821","nonce":2504025850}]},"public_addr":"172.21.15.59:6817/2504025850","cluster_addr":"172.21.15.59:6819/2504025850","heartbeat_back_addr":"172.21.15.59:6823/2504025850","heartbeat_front_addr":"172.21.15.59:6821/2504025850","state":["exists","up"]},{"osd":4,"uuid":"7ac29fca-b5b2-4c95-9d69-2f164cbbc708","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6816","nonce":2382463431},{"type":"v1","addr":"172.21.15.76:6818","nonce":2382463431}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6820","nonce":2382463431},{"type":"v1","addr":"172.21.15.76:6822","nonce":2382463431}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6828","nonce":2382463431},{"type":"v1","addr":"172.21.15.76:6830","nonce":2382463431}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6824","nonce":2382463431},{"type":"v1","addr":"172.21.15.76:6826","nonce":2382463431}]},"public_addr":"172.21.15.76:6818/2382463431","cluster_addr":"172.21.15.76:6822/2382463431","heartbeat_back_addr":"172.21.15.76:6830/2382463431","heartbeat_front_addr":"172.21.15.76:6826/2382463431","state":["exists","up"]},{"osd":5,"uuid":"485bf711-3851-4238-a4f4-51d52e0a9c58","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6800","nonce":329334308},{"type":"v1","addr":"172.21.15.76:6802","nonce":329334308}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6805","nonce":329334308},{"type":"v1","addr":"172.21.15.76:6807","nonce":329334308}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6814","nonce":329334308},{"type":"v1","addr":"172.21.15.76:6815","nonce":329334308}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6810","nonce":329334308},{"type":"v1","addr":"172.21.15.76:6812","nonce":329334308}]},"public_addr":"172.21.15.76:6802/329334308","cluster_addr":"172.21.15.76:6807/329334308","heartbeat_back_addr":"172.21.15.76:6815/329334308","heartbeat_front_addr":"172.21.15.76:6812/329334308","state":["exists","up"]},{"osd":6,"uuid":"72b3f2fc-1e4a-4e32-968d-003647a0286f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6801","nonce":3946579410},{"type":"v1","addr":"172.21.15.76:6803","nonce":3946579410}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6804","nonce":3946579410},{"type":"v1","addr":"172.21.15.76:6806","nonce":3946579410}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6811","nonce":3946579410},{"type":"v1","addr":"172.21.15.76:6813","nonce":3946579410}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6808","nonce":3946579410},{"type":"v1","addr":"172.21.15.76:6809","nonce":3946579410}]},"public_addr":"172.21.15.76:6803/3946579410","cluster_addr":"172.21.15.76:6806/3946579410","heartbeat_back_addr":"172.21.15.76:6813/3946579410","heartbeat_front_addr":"172.21.15.76:6809/3946579410","state":["exists","up"]},{"osd":7,"uuid":"1ddceb90-6c5d-4d1a-ad56-99cb877f8c70","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6817","nonce":841761538},{"type":"v1","addr":"172.21.15.76:6819","nonce":841761538}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6821","nonce":841761538},{"type":"v1","addr":"172.21.15.76:6823","nonce":841761538}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6829","nonce":841761538},{"type":"v1","addr":"172.21.15.76:6831","nonce":841761538}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6825","nonce":841761538},{"type":"v1","addr":"172.21.15.76:6827","nonce":841761538}]},"public_addr":"172.21.15.76:6819/841761538","cluster_addr":"172.21.15.76:6823/841761538","heartbeat_back_addr":"172.21.15.76:6831/841761538","heartbeat_front_addr":"172.21.15.76:6827/841761538","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T07:56:14.294838+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T07:56:14.270779+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T07:56:14.231809+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T07:56:14.260983+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T07:56:14.302997+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T07:56:14.258600+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T07:56:14.251413+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T07:56:14.304727+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-24T08:00:20.795 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:20.793+0000 7f606a65a700 1 -- 172.21.15.59:0/2398111010 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f605006b500 msgr2=0x7f605006d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:00:20.795 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:20.793+0000 7f606a65a700 1 --2- 172.21.15.59:0/2398111010 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f605006b500 0x7f605006d9c0 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7f6058001cb0 tx=0x7f6058001150 comp rx=0 tx=0).stop 2023-11-24T08:00:20.795 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:20.793+0000 7f606a65a700 1 -- 172.21.15.59:0/2398111010 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f604c003a30 msgr2=0x7f606414e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:00:20.796 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:20.793+0000 7f606a65a700 1 --2- 172.21.15.59:0/2398111010 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f604c003a30 0x7f606414e2f0 secure :-1 s=READY pgs=1111 cs=0 l=1 rev1=1 crypto rx=0x7f604c005830 tx=0x7f60540047c0 comp rx=0 tx=0).stop 2023-11-24T08:00:20.796 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:20.793+0000 7f606a65a700 1 -- 172.21.15.59:0/2398111010 shutdown_connections 2023-11-24T08:00:20.796 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:20.793+0000 7f606a65a700 1 --2- 172.21.15.59:0/2398111010 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f605006b500 0x7f605006d9c0 unknown :-1 s=CLOSED pgs=508 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:00:20.796 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:20.793+0000 7f606a65a700 1 --2- 172.21.15.59:0/2398111010 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f60640cb390 0x7f60640b24b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:00:20.796 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:20.793+0000 7f606a65a700 1 --2- 172.21.15.59:0/2398111010 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f604c003a30 0x7f606414e2f0 unknown :-1 s=CLOSED pgs=1111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:00:20.796 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:20.793+0000 7f606a65a700 1 --2- 172.21.15.59:0/2398111010 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f60640caf10 0x7f60640cd3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:00:20.796 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:20.793+0000 7f606a65a700 1 -- 172.21.15.59:0/2398111010 >> 172.21.15.59:0/2398111010 conn(0x7f60640d3e40 msgr2=0x7f606415c7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:00:20.797 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:20.793+0000 7f606a65a700 1 -- 172.21.15.59:0/2398111010 shutdown_connections 2023-11-24T08:00:20.797 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:20.793+0000 7f606a65a700 1 -- 172.21.15.59:0/2398111010 wait complete. 2023-11-24T08:00:20.808 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2023-11-24T08:00:20.934 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:20.932+0000 7fe3ba4d4700 1 Processor -- start 2023-11-24T08:00:20.934 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:20.932+0000 7fe3ba4d4700 1 -- start start 2023-11-24T08:00:20.937 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:20.933+0000 7fe3ba4d4700 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fe3b40b5ab0 0x7fe3b406e560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:00:20.937 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:20.933+0000 7fe3ba4d4700 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe3b4071f50 con 0x7fe3b40b4150 2023-11-24T08:00:20.937 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:20.933+0000 7fe3ba4d4700 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe3b4072230 con 0x7fe3b406b620 2023-11-24T08:00:20.937 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:20.933+0000 7fe3ba4d4700 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe3b4072510 con 0x7fe3b40b5ab0 2023-11-24T08:00:20.937 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:20.933+0000 7fe3b37fe700 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7fe3b40b4150 0x7fe3b406de50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:59886/0 (socket says 172.21.15.59:59886) 2023-11-24T08:00:20.938 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:20.933+0000 7fe3b37fe700 1 -- 172.21.15.59:0/2975097791 learned_addr learned my addr 172.21.15.59:0/2975097791 (peer_addr_for_me v1:172.21.15.59:0/0) 2023-11-24T08:00:20.938 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:20.933+0000 7fe3b8a71700 1 --2- 172.21.15.59:0/2975097791 >> v2:172.21.15.76:3300/0 conn(0x7fe3b40b5ab0 0x7fe3b406e560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:00:20.938 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:20.933+0000 7fe3b2ffd700 1 -- 172.21.15.59:0/2975097791 <== mon.0 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 718543654 0 0) 0x7fe3b4071f50 con 0x7fe3b40b4150 2023-11-24T08:00:20.938 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:20.933+0000 7fe3b2ffd700 1 -- 172.21.15.59:0/2975097791 --> v1:172.21.15.59:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe39c0035b0 con 0x7fe3b40b4150 2023-11-24T08:00:20.938 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:20.933+0000 7fe3b2ffd700 1 -- 172.21.15.59:0/2975097791 <== mon.0 v1:172.21.15.59:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1805216732 0 0) 0x7fe39c0035b0 con 0x7fe3b40b4150 2023-11-24T08:00:20.938 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:20.933+0000 7fe3b2ffd700 1 -- 172.21.15.59:0/2975097791 >> v2:172.21.15.76:3300/0 conn(0x7fe3b40b5ab0 msgr2=0x7fe3b406e560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:00:20.938 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:20.933+0000 7fe3b2ffd700 1 --2- 172.21.15.59:0/2975097791 >> v2:172.21.15.76:3300/0 conn(0x7fe3b40b5ab0 0x7fe3b406e560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:00:20.938 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:20.933+0000 7fe3b2ffd700 1 -- 172.21.15.59:0/2975097791 >> v1:172.21.15.76:6789/0 conn(0x7fe3b406b620 legacy=0x7fe3b40b3a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:00:20.939 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:20.933+0000 7fe3b2ffd700 1 -- 172.21.15.59:0/2975097791 --> v1:172.21.15.59:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe3b40727f0 con 0x7fe3b40b4150 2023-11-24T08:00:20.939 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:20.933+0000 7fe3b2ffd700 1 -- 172.21.15.59:0/2975097791 <== mon.0 v1:172.21.15.59:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1312333566 0 0) 0x7fe3a8001670 con 0x7fe3b40b4150 2023-11-24T08:00:20.939 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:20.933+0000 7fe3b2ffd700 1 -- 172.21.15.59:0/2975097791 >> v1:172.21.15.59:6789/0 conn(0x7fe3b40b4150 legacy=0x7fe3b406de50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:00:20.939 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:20.933+0000 7fe3b2ffd700 1 --2- 172.21.15.59:0/2975097791 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe39c003a00 0x7fe39c003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:00:20.939 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:20.933+0000 7fe3b2ffd700 1 --2- 172.21.15.59:0/2975097791 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe39c004850 0x7fe39c004cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:00:20.939 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:20.933+0000 7fe3b2ffd700 1 --2- 172.21.15.59:0/2975097791 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fe39c005210 0x7fe39c00ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:00:20.940 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:20.933+0000 7fe3b2ffd700 1 -- 172.21.15.59:0/2975097791 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3b4072510 con 0x7fe39c004850 2023-11-24T08:00:20.940 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:20.933+0000 7fe3b2ffd700 1 -- 172.21.15.59:0/2975097791 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe39c00c0b0 con 0x7fe39c003a00 2023-11-24T08:00:20.940 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:20.934+0000 7fe3b2ffd700 1 -- 172.21.15.59:0/2975097791 --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe39c00c270 con 0x7fe39c005210 2023-11-24T08:00:20.940 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:20.934+0000 7fe3b37fe700 1 --2- 172.21.15.59:0/2975097791 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe39c004850 0x7fe39c004cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:00:20.940 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:20.934+0000 7fe3b8a71700 1 --2- 172.21.15.59:0/2975097791 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fe39c005210 0x7fe39c00ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:00:20.940 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:20.934+0000 7fe3b3fff700 1 --2- 172.21.15.59:0/2975097791 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe39c003a00 0x7fe39c003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:00:20.940 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:20.934+0000 7fe3b37fe700 1 -- 172.21.15.59:0/2975097791 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fe39c005210 msgr2=0x7fe39c00ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:00:20.940 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:20.934+0000 7fe3b37fe700 1 --2- 172.21.15.59:0/2975097791 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fe39c005210 0x7fe39c00ba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:00:20.940 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:20.934+0000 7fe3b37fe700 1 -- 172.21.15.59:0/2975097791 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe39c003a00 msgr2=0x7fe39c003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:00:20.940 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:20.934+0000 7fe3b37fe700 1 --2- 172.21.15.59:0/2975097791 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe39c003a00 0x7fe39c003e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:00:20.940 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:20.934+0000 7fe3b37fe700 1 -- 172.21.15.59:0/2975097791 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe39c0035b0 con 0x7fe39c004850 2023-11-24T08:00:20.941 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:20.934+0000 7fe3b8a71700 1 -- 172.21.15.59:0/2975097791 reap_dead start 2023-11-24T08:00:20.941 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:20.934+0000 7fe3b37fe700 1 --2- 172.21.15.59:0/2975097791 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe39c004850 0x7fe39c004cd0 secure :-1 s=READY pgs=1113 cs=0 l=1 rev1=1 crypto rx=0x7fe3a800c240 tx=0x7fe39c005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6da9caa115c6ae45 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:00:20.941 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:20.934+0000 7fe3b2ffd700 1 -- 172.21.15.59:0/2975097791 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3a8007c10 con 0x7fe39c004850 2023-11-24T08:00:20.941 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:20.934+0000 7fe3b2ffd700 1 -- 172.21.15.59:0/2975097791 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe3a800cd60 con 0x7fe39c004850 2023-11-24T08:00:20.941 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:20.934+0000 7fe3b2ffd700 1 -- 172.21.15.59:0/2975097791 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3a8005ba0 con 0x7fe39c004850 2023-11-24T08:00:20.941 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:20.935+0000 7fe3ba4d4700 1 -- 172.21.15.59:0/2975097791 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe39c004850 msgr2=0x7fe39c004cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:00:20.941 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:20.935+0000 7fe3ba4d4700 1 --2- 172.21.15.59:0/2975097791 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe39c004850 0x7fe39c004cd0 secure :-1 s=READY pgs=1113 cs=0 l=1 rev1=1 crypto rx=0x7fe3a800c240 tx=0x7fe39c005880 comp rx=0 tx=0).stop 2023-11-24T08:00:20.941 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:20.935+0000 7fe3ba4d4700 1 -- 172.21.15.59:0/2975097791 shutdown_connections 2023-11-24T08:00:20.941 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:20.935+0000 7fe3ba4d4700 1 --2- 172.21.15.59:0/2975097791 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe39c004850 0x7fe39c004cd0 unknown :-1 s=CLOSED pgs=1113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:00:20.941 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:20.935+0000 7fe3ba4d4700 1 -- 172.21.15.59:0/2975097791 >> 172.21.15.59:0/2975097791 conn(0x7fe3b40b2740 msgr2=0x7fe3b40b2b20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:00:20.941 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:20.935+0000 7fe3ba4d4700 1 -- 172.21.15.59:0/2975097791 shutdown_connections 2023-11-24T08:00:20.942 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:20.935+0000 7fe3ba4d4700 1 -- 172.21.15.59:0/2975097791 wait complete. 2023-11-24T08:00:20.944 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:20.936+0000 7fe3ba4d4700 1 Processor -- start 2023-11-24T08:00:20.945 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:20.936+0000 7fe3ba4d4700 1 -- start start 2023-11-24T08:00:20.945 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:20.937+0000 7fe3ba4d4700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe39c004850 0x7fe3b41392a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:00:20.945 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:20.937+0000 7fe3ba4d4700 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fe3b41f46c0 0x7fe3b4137250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:00:20.945 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:20.937+0000 7fe3ba4d4700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe3b406ecb0 0x7fe3b406f160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:00:20.945 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:20.937+0000 7fe3ba4d4700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3b406f6a0 con 0x7fe3b406ecb0 2023-11-24T08:00:20.945 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:20.937+0000 7fe3ba4d4700 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3b41ccd30 con 0x7fe39c004850 2023-11-24T08:00:20.945 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:20.937+0000 7fe3ba4d4700 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe3b40c9cc0 con 0x7fe3b41f46c0 2023-11-24T08:00:20.945 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:20.937+0000 7fe3b8a71700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe3b406ecb0 0x7fe3b406f160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:00:20.945 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:20.937+0000 7fe3b8a71700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe3b406ecb0 0x7fe3b406f160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:56426/0 (socket says 172.21.15.59:56426) 2023-11-24T08:00:20.946 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:20.937+0000 7fe3b8a71700 1 -- 172.21.15.59:0/803660119 learned_addr learned my addr 172.21.15.59:0/803660119 (peer_addr_for_me v2:172.21.15.59:0/0) 2023-11-24T08:00:20.946 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:20.937+0000 7fe3b37fe700 1 --2- 172.21.15.59:0/803660119 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fe3b41f46c0 0x7fe3b4137250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:00:20.946 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:20.937+0000 7fe3b3fff700 1 --2- 172.21.15.59:0/803660119 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe39c004850 0x7fe3b41392a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:00:20.946 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:20.937+0000 7fe3b8a71700 1 -- 172.21.15.59:0/803660119 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fe3b41f46c0 msgr2=0x7fe3b4137250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:00:20.946 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:20.937+0000 7fe3b8a71700 1 --2- 172.21.15.59:0/803660119 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fe3b41f46c0 0x7fe3b4137250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:00:20.946 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:20.937+0000 7fe3b8a71700 1 -- 172.21.15.59:0/803660119 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe39c004850 msgr2=0x7fe3b41392a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:00:20.946 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:20.937+0000 7fe3b8a71700 1 --2- 172.21.15.59:0/803660119 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe39c004850 0x7fe3b41392a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:00:20.946 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:20.937+0000 7fe3b8a71700 1 -- 172.21.15.59:0/803660119 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe3b4139930 con 0x7fe3b406ecb0 2023-11-24T08:00:20.946 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:20.937+0000 7fe3b37fe700 1 --2- 172.21.15.59:0/803660119 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fe3b41f46c0 0x7fe3b4137250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-24T08:00:20.946 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:20.937+0000 7fe3b3fff700 1 --2- 172.21.15.59:0/803660119 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe39c004850 0x7fe3b41392a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-24T08:00:20.947 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:20.938+0000 7fe3b8a71700 1 --2- 172.21.15.59:0/803660119 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe3b406ecb0 0x7fe3b406f160 secure :-1 s=READY pgs=1114 cs=0 l=1 rev1=1 crypto rx=0x7fe3b406bff0 tx=0x7fe3ac011ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:00:20.947 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:20.938+0000 7fe3b0ff9700 1 -- 172.21.15.59:0/803660119 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3ac00e040 con 0x7fe3b406ecb0 2023-11-24T08:00:20.947 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:20.938+0000 7fe3b0ff9700 1 -- 172.21.15.59:0/803660119 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe3ac00f040 con 0x7fe3b406ecb0 2023-11-24T08:00:20.947 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:20.938+0000 7fe3ba4d4700 1 -- 172.21.15.59:0/803660119 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe3b4139c10 con 0x7fe3b406ecb0 2023-11-24T08:00:20.947 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:20.938+0000 7fe3b0ff9700 1 -- 172.21.15.59:0/803660119 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3ac00fd00 con 0x7fe3b406ecb0 2023-11-24T08:00:20.947 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:20.938+0000 7fe3ba4d4700 1 -- 172.21.15.59:0/803660119 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe3b41f4b40 con 0x7fe3b406ecb0 2023-11-24T08:00:20.947 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:20.939+0000 7fe3ba4d4700 1 -- 172.21.15.59:0/803660119 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe3b4068670 con 0x7fe3b406ecb0 2023-11-24T08:00:20.947 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:20.939+0000 7fe3b0ff9700 1 -- 172.21.15.59:0/803660119 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 12) v1 ==== 89792+0+0 (secure 0 0 0) 0x7fe3ac022460 con 0x7fe3b406ecb0 2023-11-24T08:00:20.947 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:20.940+0000 7fe3b0ff9700 1 --2- 172.21.15.59:0/803660119 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7fe3a006b500 0x7fe3a006d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:00:20.947 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:20.940+0000 7fe3b0ff9700 1 -- 172.21.15.59:0/803660119 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fe3ac08a680 con 0x7fe3b406ecb0 2023-11-24T08:00:20.947 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:20.940+0000 7fe3b3fff700 1 --2- 172.21.15.59:0/803660119 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7fe3a006b500 0x7fe3a006d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:00:20.947 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:20.940+0000 7fe3b3fff700 1 --2- 172.21.15.59:0/803660119 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7fe3a006b500 0x7fe3a006d9c0 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7fe3a4000c00 tx=0x7fe3a4009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:00:20.948 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:20.944+0000 7fe3b0ff9700 1 -- 172.21.15.59:0/803660119 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fe3ac054de0 con 0x7fe3b406ecb0 2023-11-24T08:00:21.193 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:21.192+0000 7fe3ba4d4700 1 -- 172.21.15.59:0/803660119 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7fe3b40b3aa0 con 0x7fe3b406ecb0 2023-11-24T08:00:21.194 INFO:tasks.ceph.mon.a.smithi059.stderr:2023-11-24T08:00:21.192+0000 7f5560821700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-11-24T08:00:21.195 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:21.194+0000 7fe3b0ff9700 1 -- 172.21.15.59:0/803660119 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v55) v1 ==== 161+0+0 (secure 0 0 0) 0x7fe3ac058400 con 0x7fe3b406ecb0 2023-11-24T08:00:21.198 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:21.196+0000 7fe3ba4d4700 1 -- 172.21.15.59:0/803660119 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7fe3a006b500 msgr2=0x7fe3a006d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:00:21.198 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:21.196+0000 7fe3ba4d4700 1 --2- 172.21.15.59:0/803660119 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7fe3a006b500 0x7fe3a006d9c0 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7fe3a4000c00 tx=0x7fe3a4009040 comp rx=0 tx=0).stop 2023-11-24T08:00:21.198 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:21.196+0000 7fe3ba4d4700 1 -- 172.21.15.59:0/803660119 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe3b406ecb0 msgr2=0x7fe3b406f160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:00:21.199 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:21.196+0000 7fe3ba4d4700 1 --2- 172.21.15.59:0/803660119 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe3b406ecb0 0x7fe3b406f160 secure :-1 s=READY pgs=1114 cs=0 l=1 rev1=1 crypto rx=0x7fe3b406bff0 tx=0x7fe3ac011ed0 comp rx=0 tx=0).stop 2023-11-24T08:00:21.199 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:21.196+0000 7fe3ba4d4700 1 -- 172.21.15.59:0/803660119 shutdown_connections 2023-11-24T08:00:21.199 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:21.196+0000 7fe3ba4d4700 1 --2- 172.21.15.59:0/803660119 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7fe3a006b500 0x7fe3a006d9c0 secure :-1 s=CLOSED pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7fe3a4000c00 tx=0x7fe3a4009040 comp rx=0 tx=0).stop 2023-11-24T08:00:21.199 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:21.196+0000 7fe3ba4d4700 1 --2- 172.21.15.59:0/803660119 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe3b406ecb0 0x7fe3b406f160 unknown :-1 s=CLOSED pgs=1114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:00:21.199 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:21.196+0000 7fe3ba4d4700 1 --2- 172.21.15.59:0/803660119 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe39c004850 0x7fe3b41392a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:00:21.200 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:21.196+0000 7fe3ba4d4700 1 --2- 172.21.15.59:0/803660119 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fe3b41f46c0 0x7fe3b4137250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:00:21.200 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:21.196+0000 7fe3ba4d4700 1 -- 172.21.15.59:0/803660119 >> 172.21.15.59:0/803660119 conn(0x7fe3b40b2740 msgr2=0x7fe3b40b0a50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:00:21.200 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:21.197+0000 7fe3ba4d4700 1 -- 172.21.15.59:0/803660119 shutdown_connections 2023-11-24T08:00:21.201 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:21.197+0000 7fe3ba4d4700 1 -- 172.21.15.59:0/803660119 wait complete. 2023-11-24T08:00:21.201 INFO:teuthology.orchestra.run.smithi059.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2023-11-24T08:00:21.201 INFO:tasks.ceph.mds.c.smithi059.stderr: -6841> 2023-11-24T07:59:57.472+0000 7f3ea5e3b700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-11-24T08:00:21.212 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2023-11-24T08:00:21.233 INFO:tasks.ceph.mds.c.smithi059.stdout:starting mds.c at 2023-11-24T08:00:21.341 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:21.340+0000 7fbc24f11700 1 Processor -- start 2023-11-24T08:00:21.342 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:21.340+0000 7fbc24f11700 1 -- start start 2023-11-24T08:00:21.343 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:21.341+0000 7fbc24f11700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fbc20143ae0 0x7fbc200b51b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:00:21.344 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:21.341+0000 7fbc24f11700 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fbc200722c0 0x7fbc2006e510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:00:21.344 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:21.341+0000 7fbc24f11700 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbc20143ec0 con 0x7fbc201440f0 2023-11-24T08:00:21.344 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:21.341+0000 7fbc24f11700 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbc200b40a0 con 0x7fbc20143ae0 2023-11-24T08:00:21.344 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:21.341+0000 7fbc24f11700 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbc200b3ae0 con 0x7fbc200722c0 2023-11-24T08:00:21.344 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:21.341+0000 7fbc1e59c700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fbc20143ae0 0x7fbc200b51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:00:21.344 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:21.341+0000 7fbc1e59c700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fbc20143ae0 0x7fbc200b51b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:56442/0 (socket says 172.21.15.59:56442) 2023-11-24T08:00:21.344 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:21.341+0000 7fbc1e59c700 1 -- 172.21.15.59:0/3253417449 learned_addr learned my addr 172.21.15.59:0/3253417449 (peer_addr_for_me v2:172.21.15.59:0/0) 2023-11-24T08:00:21.344 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:21.341+0000 7fbc1ed9d700 1 --2- 172.21.15.59:0/3253417449 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fbc200722c0 0x7fbc2006e510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:00:21.344 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:21.341+0000 7fbc1e59c700 1 -- 172.21.15.59:0/3253417449 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fbc200722c0 msgr2=0x7fbc2006e510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:00:21.344 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:21.341+0000 7fbc1e59c700 1 --2- 172.21.15.59:0/3253417449 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fbc200722c0 0x7fbc2006e510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:00:21.344 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:21.341+0000 7fbc1e59c700 1 -- 172.21.15.59:0/3253417449 >> v1:172.21.15.76:6789/0 conn(0x7fbc201440f0 legacy=0x7fbc20071bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:00:21.345 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:21.341+0000 7fbc1e59c700 1 -- 172.21.15.59:0/3253417449 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbc200b3dc0 con 0x7fbc20143ae0 2023-11-24T08:00:21.345 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:21.342+0000 7fbc1e59c700 1 --2- 172.21.15.59:0/3253417449 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbc20143ae0 0x7fbc200b51b0 secure :-1 s=READY pgs=1117 cs=0 l=1 rev1=1 crypto rx=0x7fbc140098f0 tx=0x7fbc14009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fb68eacf5c816c7 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:00:21.345 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:21.342+0000 7fbc1d59a700 1 -- 172.21.15.59:0/3253417449 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc1400f8a0 con 0x7fbc20143ae0 2023-11-24T08:00:21.345 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:21.342+0000 7fbc1d59a700 1 -- 172.21.15.59:0/3253417449 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbc1400fa00 con 0x7fbc20143ae0 2023-11-24T08:00:21.345 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:21.342+0000 7fbc1d59a700 1 -- 172.21.15.59:0/3253417449 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc1400fd90 con 0x7fbc20143ae0 2023-11-24T08:00:21.347 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:21.342+0000 7fbc24f11700 1 -- 172.21.15.59:0/3253417449 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbc20143ae0 msgr2=0x7fbc200b51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:00:21.347 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:21.342+0000 7fbc24f11700 1 --2- 172.21.15.59:0/3253417449 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbc20143ae0 0x7fbc200b51b0 secure :-1 s=READY pgs=1117 cs=0 l=1 rev1=1 crypto rx=0x7fbc140098f0 tx=0x7fbc14009c00 comp rx=0 tx=0).stop 2023-11-24T08:00:21.348 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:21.343+0000 7fbc24f11700 1 -- 172.21.15.59:0/3253417449 shutdown_connections 2023-11-24T08:00:21.348 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:21.343+0000 7fbc24f11700 1 --2- 172.21.15.59:0/3253417449 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fbc200722c0 0x7fbc2006e510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:00:21.348 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:21.343+0000 7fbc24f11700 1 --2- 172.21.15.59:0/3253417449 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbc20143ae0 0x7fbc200b51b0 unknown :-1 s=CLOSED pgs=1117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:00:21.348 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:21.343+0000 7fbc24f11700 1 -- 172.21.15.59:0/3253417449 >> 172.21.15.59:0/3253417449 conn(0x7fbc200b2740 msgr2=0x7fbc200b2b20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:00:21.348 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:21.343+0000 7fbc24f11700 1 -- 172.21.15.59:0/3253417449 shutdown_connections 2023-11-24T08:00:21.348 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:21.343+0000 7fbc24f11700 1 -- 172.21.15.59:0/3253417449 wait complete. 2023-11-24T08:00:21.348 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:21.344+0000 7fbc24f11700 1 Processor -- start 2023-11-24T08:00:21.348 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:21.344+0000 7fbc24f11700 1 -- start start 2023-11-24T08:00:21.348 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:21.345+0000 7fbc24f11700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fbc200722c0 0x7fbc200cace0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:00:21.348 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:21.345+0000 7fbc24f11700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbc20143ae0 0x7fbc200a96c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:00:21.348 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:21.345+0000 7fbc24f11700 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fbc201440f0 0x7fbc20140ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:00:21.349 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:21.345+0000 7fbc24f11700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc20141ca0 con 0x7fbc20143ae0 2023-11-24T08:00:21.349 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:21.345+0000 7fbc24f11700 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc201f40e0 con 0x7fbc200722c0 2023-11-24T08:00:21.349 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:21.345+0000 7fbc24f11700 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbc200cb3e0 con 0x7fbc201440f0 2023-11-24T08:00:21.349 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:21.345+0000 7fbc1e59c700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fbc200722c0 0x7fbc200cace0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:00:21.349 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:21.345+0000 7fbc1dd9b700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbc20143ae0 0x7fbc200a96c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:00:21.349 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:21.345+0000 7fbc1e59c700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fbc200722c0 0x7fbc200cace0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.59:43646/0 (socket says 172.21.15.59:43646) 2023-11-24T08:00:21.349 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:21.345+0000 7fbc1e59c700 1 -- 172.21.15.59:0/971754080 learned_addr learned my addr 172.21.15.59:0/971754080 (peer_addr_for_me v2:172.21.15.59:0/0) 2023-11-24T08:00:21.349 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:21.345+0000 7fbc1ed9d700 1 --2- 172.21.15.59:0/971754080 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fbc201440f0 0x7fbc20140ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:00:21.349 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:21.345+0000 7fbc1dd9b700 1 -- 172.21.15.59:0/971754080 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fbc201440f0 msgr2=0x7fbc20140ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:00:21.349 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:21.345+0000 7fbc1dd9b700 1 --2- 172.21.15.59:0/971754080 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fbc201440f0 0x7fbc20140ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:00:21.349 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:21.345+0000 7fbc1dd9b700 1 -- 172.21.15.59:0/971754080 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fbc200722c0 msgr2=0x7fbc200cace0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:00:21.349 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:21.345+0000 7fbc1dd9b700 1 --2- 172.21.15.59:0/971754080 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fbc200722c0 0x7fbc200cace0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:00:21.350 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:21.345+0000 7fbc1dd9b700 1 -- 172.21.15.59:0/971754080 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbc14009550 con 0x7fbc20143ae0 2023-11-24T08:00:21.350 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:21.345+0000 7fbc1e59c700 1 --2- 172.21.15.59:0/971754080 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fbc200722c0 0x7fbc200cace0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-24T08:00:21.350 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:21.346+0000 7fbc1dd9b700 1 --2- 172.21.15.59:0/971754080 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbc20143ae0 0x7fbc200a96c0 secure :-1 s=READY pgs=1118 cs=0 l=1 rev1=1 crypto rx=0x7fbc10000f60 tx=0x7fbc1000c6e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:00:21.351 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:21.347+0000 7fbc0effd700 1 -- 172.21.15.59:0/971754080 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc100052d0 con 0x7fbc20143ae0 2023-11-24T08:00:21.351 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:21.347+0000 7fbc0effd700 1 -- 172.21.15.59:0/971754080 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbc10005430 con 0x7fbc20143ae0 2023-11-24T08:00:21.351 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:21.347+0000 7fbc24f11700 1 -- 172.21.15.59:0/971754080 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbc200cb6c0 con 0x7fbc20143ae0 2023-11-24T08:00:21.351 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:21.347+0000 7fbc0effd700 1 -- 172.21.15.59:0/971754080 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc100038c0 con 0x7fbc20143ae0 2023-11-24T08:00:21.351 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:21.347+0000 7fbc24f11700 1 -- 172.21.15.59:0/971754080 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbc2015d300 con 0x7fbc20143ae0 2023-11-24T08:00:21.351 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:21.348+0000 7fbc24f11700 1 -- 172.21.15.59:0/971754080 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbc200ccde0 con 0x7fbc20143ae0 2023-11-24T08:00:21.352 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:21.348+0000 7fbc0effd700 1 -- 172.21.15.59:0/971754080 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 12) v1 ==== 89792+0+0 (secure 0 0 0) 0x7fbc10003c30 con 0x7fbc20143ae0 2023-11-24T08:00:21.352 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:21.349+0000 7fbc0effd700 1 --2- 172.21.15.59:0/971754080 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7fbc0006b500 0x7fbc0006d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:00:21.352 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:21.349+0000 7fbc0effd700 1 -- 172.21.15.59:0/971754080 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 5795+0+0 (secure 0 0 0) 0x7fbc10089580 con 0x7fbc20143ae0 2023-11-24T08:00:21.352 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:21.349+0000 7fbc1e59c700 1 --2- 172.21.15.59:0/971754080 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7fbc0006b500 0x7fbc0006d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:00:21.352 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:21.350+0000 7fbc1e59c700 1 --2- 172.21.15.59:0/971754080 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7fbc0006b500 0x7fbc0006d9c0 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7fbc14009130 tx=0x7fbc1401b040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:00:21.355 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:21.353+0000 7fbc0effd700 1 -- 172.21.15.59:0/971754080 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fbc10053c90 con 0x7fbc20143ae0 2023-11-24T08:00:21.605 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:21.604+0000 7fbc24f11700 1 -- 172.21.15.59:0/971754080 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7fbc20144830 con 0x7fbc20143ae0 2023-11-24T08:00:22.197 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:22.195+0000 7fbc0effd700 1 -- 172.21.15.59:0/971754080 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v56) v1 ==== 104+0+0 (secure 0 0 0) 0x7fbc100572b0 con 0x7fbc20143ae0 2023-11-24T08:00:22.200 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:22.198+0000 7fbc24f11700 1 -- 172.21.15.59:0/971754080 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7fbc0006b500 msgr2=0x7fbc0006d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:00:22.201 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:22.198+0000 7fbc24f11700 1 --2- 172.21.15.59:0/971754080 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7fbc0006b500 0x7fbc0006d9c0 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7fbc14009130 tx=0x7fbc1401b040 comp rx=0 tx=0).stop 2023-11-24T08:00:22.201 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:22.198+0000 7fbc24f11700 1 -- 172.21.15.59:0/971754080 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbc20143ae0 msgr2=0x7fbc200a96c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:00:22.201 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:22.198+0000 7fbc24f11700 1 --2- 172.21.15.59:0/971754080 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbc20143ae0 0x7fbc200a96c0 secure :-1 s=READY pgs=1118 cs=0 l=1 rev1=1 crypto rx=0x7fbc10000f60 tx=0x7fbc1000c6e0 comp rx=0 tx=0).stop 2023-11-24T08:00:22.201 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:22.198+0000 7fbc24f11700 1 -- 172.21.15.59:0/971754080 shutdown_connections 2023-11-24T08:00:22.201 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:22.198+0000 7fbc24f11700 1 --2- 172.21.15.59:0/971754080 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7fbc0006b500 0x7fbc0006d9c0 unknown :-1 s=CLOSED pgs=510 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:00:22.201 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:22.198+0000 7fbc24f11700 1 --2- 172.21.15.59:0/971754080 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fbc201440f0 0x7fbc20140ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:00:22.202 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:22.198+0000 7fbc24f11700 1 --2- 172.21.15.59:0/971754080 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fbc200722c0 0x7fbc200cace0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:00:22.202 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:22.198+0000 7fbc24f11700 1 --2- 172.21.15.59:0/971754080 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbc20143ae0 0x7fbc200a96c0 unknown :-1 s=CLOSED pgs=1118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:00:22.202 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:22.199+0000 7fbc24f11700 1 -- 172.21.15.59:0/971754080 >> 172.21.15.59:0/971754080 conn(0x7fbc200b2740 msgr2=0x7fbc200b52f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:00:22.202 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:22.199+0000 7fbc24f11700 1 -- 172.21.15.59:0/971754080 shutdown_connections 2023-11-24T08:00:22.202 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:22.199+0000 7fbc24f11700 1 -- 172.21.15.59:0/971754080 wait complete. 2023-11-24T08:00:22.216 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2023-11-24T08:00:22.394 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:22.392+0000 7f17e10d6700 1 Processor -- start 2023-11-24T08:00:22.394 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:22.392+0000 7f17e10d6700 1 -- start start 2023-11-24T08:00:22.396 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:22.393+0000 7f17e10d6700 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f17dc0722a0 0x7f17dc06e060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:00:22.396 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:22.393+0000 7f17e10d6700 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f17dc072ab0 0x7f17dc06e5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:00:22.396 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:22.393+0000 7f17e10d6700 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f17dc06dbd0 con 0x7f17dc1447a0 2023-11-24T08:00:22.396 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:22.393+0000 7f17e10d6700 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f17dc15b700 con 0x7f17dc0722a0 2023-11-24T08:00:22.397 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:22.393+0000 7f17e10d6700 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f17dc15b9e0 con 0x7f17dc072ab0 2023-11-24T08:00:22.397 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:22.393+0000 7f17da59c700 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f17dc0722a0 0x7f17dc06e060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:00:22.397 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:22.393+0000 7f17db59e700 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f17dc072ab0 0x7f17dc06e5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:00:22.397 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:22.393+0000 7f17dad9d700 1 --1- >> v1:172.21.15.76:6789/0 conn(0x7f17dc1447a0 0x7f17dc071b90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.59:47210/0 (socket says 172.21.15.59:47210) 2023-11-24T08:00:22.397 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:22.393+0000 7f17da59c700 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f17dc0722a0 0x7f17dc06e060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.59:43658/0 (socket says 172.21.15.59:43658) 2023-11-24T08:00:22.398 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:22.393+0000 7f17dad9d700 1 -- 172.21.15.59:0/4293229790 learned_addr learned my addr 172.21.15.59:0/4293229790 (peer_addr_for_me v1:172.21.15.59:0/0) 2023-11-24T08:00:22.398 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:22.393+0000 7f17da59c700 1 -- 172.21.15.59:0/4293229790 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f17dc072ab0 msgr2=0x7f17dc06e5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:00:22.398 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:22.393+0000 7f17da59c700 1 --2- 172.21.15.59:0/4293229790 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f17dc072ab0 0x7f17dc06e5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:00:22.398 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:22.393+0000 7f17da59c700 1 -- 172.21.15.59:0/4293229790 >> v1:172.21.15.76:6789/0 conn(0x7f17dc1447a0 legacy=0x7f17dc071b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:00:22.398 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:22.393+0000 7f17d9d9b700 1 -- 172.21.15.59:0/4293229790 <== mon.1 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4014880680 0 0) 0x7f17dc06dbd0 con 0x7f17dc1447a0 2023-11-24T08:00:22.399 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:22.393+0000 7f17da59c700 1 -- 172.21.15.59:0/4293229790 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f17dc15bcc0 con 0x7f17dc0722a0 2023-11-24T08:00:22.399 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:22.393+0000 7f17db59e700 1 --2- 172.21.15.59:0/4293229790 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f17dc072ab0 0x7f17dc06e5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-24T08:00:22.399 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:22.394+0000 7f17da59c700 1 --2- 172.21.15.59:0/4293229790 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f17dc0722a0 0x7f17dc06e060 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f17d000bae0 tx=0x7f17d000bdf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7fe2108625cf65cd server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:00:22.399 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:22.394+0000 7f17d9d9b700 1 -- 172.21.15.59:0/4293229790 <== mon.1 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17d000dc70 con 0x7f17dc0722a0 2023-11-24T08:00:22.399 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:22.394+0000 7f17d9d9b700 1 -- 172.21.15.59:0/4293229790 <== mon.1 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f17d000ddd0 con 0x7f17dc0722a0 2023-11-24T08:00:22.399 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:22.394+0000 7f17d9d9b700 1 -- 172.21.15.59:0/4293229790 <== mon.1 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17d0010690 con 0x7f17dc0722a0 2023-11-24T08:00:22.399 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:22.394+0000 7f17e10d6700 1 -- 172.21.15.59:0/4293229790 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f17dc0722a0 msgr2=0x7f17dc06e060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:00:22.399 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:22.394+0000 7f17e10d6700 1 --2- 172.21.15.59:0/4293229790 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f17dc0722a0 0x7f17dc06e060 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f17d000bae0 tx=0x7f17d000bdf0 comp rx=0 tx=0).stop 2023-11-24T08:00:22.400 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:22.394+0000 7f17e10d6700 1 -- 172.21.15.59:0/4293229790 shutdown_connections 2023-11-24T08:00:22.400 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:22.394+0000 7f17e10d6700 1 --2- 172.21.15.59:0/4293229790 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f17dc072ab0 0x7f17dc06e5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:00:22.400 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:22.394+0000 7f17e10d6700 1 --2- 172.21.15.59:0/4293229790 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f17dc0722a0 0x7f17dc06e060 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:00:22.400 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:22.394+0000 7f17e10d6700 1 -- 172.21.15.59:0/4293229790 >> 172.21.15.59:0/4293229790 conn(0x7f17dc0b2760 msgr2=0x7f17dc0b2b70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:00:22.400 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:22.394+0000 7f17e10d6700 1 -- 172.21.15.59:0/4293229790 shutdown_connections 2023-11-24T08:00:22.400 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:22.394+0000 7f17e10d6700 1 -- 172.21.15.59:0/4293229790 wait complete. 2023-11-24T08:00:22.404 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:22.396+0000 7f17e10d6700 1 Processor -- start 2023-11-24T08:00:22.404 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:22.396+0000 7f17e10d6700 1 -- start start 2023-11-24T08:00:22.404 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:22.397+0000 7f17e10d6700 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f17dc0722a0 0x7f17dc14d770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:00:22.404 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:22.397+0000 7f17e10d6700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f17dc072ab0 0x7f17dc150210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:00:22.404 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:22.397+0000 7f17e10d6700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f17dc1447a0 0x7f17dc13d510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:00:22.404 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:22.397+0000 7f17e10d6700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17dc15dcd0 con 0x7f17dc1447a0 2023-11-24T08:00:22.405 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:22.397+0000 7f17e10d6700 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17dc14dcb0 con 0x7f17dc072ab0 2023-11-24T08:00:22.405 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:22.397+0000 7f17e10d6700 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f17dc150750 con 0x7f17dc0722a0 2023-11-24T08:00:22.405 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:22.397+0000 7f17db59e700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f17dc1447a0 0x7f17dc13d510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:00:22.405 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:22.397+0000 7f17db59e700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f17dc1447a0 0x7f17dc13d510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:56468/0 (socket says 172.21.15.59:56468) 2023-11-24T08:00:22.405 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:22.397+0000 7f17db59e700 1 -- 172.21.15.59:0/3667566115 learned_addr learned my addr 172.21.15.59:0/3667566115 (peer_addr_for_me v2:172.21.15.59:0/0) 2023-11-24T08:00:22.405 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:22.397+0000 7f17dad9d700 1 --2- 172.21.15.59:0/3667566115 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f17dc0722a0 0x7f17dc14d770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:00:22.405 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:22.397+0000 7f17da59c700 1 --2- 172.21.15.59:0/3667566115 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f17dc072ab0 0x7f17dc150210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:00:22.406 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:22.397+0000 7f17db59e700 1 -- 172.21.15.59:0/3667566115 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f17dc0722a0 msgr2=0x7f17dc14d770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:00:22.406 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:22.397+0000 7f17db59e700 1 --2- 172.21.15.59:0/3667566115 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f17dc0722a0 0x7f17dc14d770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:00:22.406 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:22.397+0000 7f17db59e700 1 -- 172.21.15.59:0/3667566115 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f17dc072ab0 msgr2=0x7f17dc150210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:00:22.406 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:22.397+0000 7f17db59e700 1 --2- 172.21.15.59:0/3667566115 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f17dc072ab0 0x7f17dc150210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:00:22.406 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:22.397+0000 7f17db59e700 1 -- 172.21.15.59:0/3667566115 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f17d4009a70 con 0x7f17dc1447a0 2023-11-24T08:00:22.406 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:22.397+0000 7f17db59e700 1 --2- 172.21.15.59:0/3667566115 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f17dc1447a0 0x7f17dc13d510 secure :-1 s=READY pgs=1119 cs=0 l=1 rev1=1 crypto rx=0x7f17d4009f00 tx=0x7f17d400c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:00:22.406 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:22.397+0000 7f17c37fe700 1 -- 172.21.15.59:0/3667566115 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17d4009d70 con 0x7f17dc1447a0 2023-11-24T08:00:22.407 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:22.397+0000 7f17c37fe700 1 -- 172.21.15.59:0/3667566115 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f17d400f040 con 0x7f17dc1447a0 2023-11-24T08:00:22.407 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:22.397+0000 7f17e10d6700 1 -- 172.21.15.59:0/3667566115 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f17d000b740 con 0x7f17dc1447a0 2023-11-24T08:00:22.407 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:22.397+0000 7f17c37fe700 1 -- 172.21.15.59:0/3667566115 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17d400cd70 con 0x7f17dc1447a0 2023-11-24T08:00:22.407 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:22.398+0000 7f17e10d6700 1 -- 172.21.15.59:0/3667566115 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f17dc15c5c0 con 0x7f17dc1447a0 2023-11-24T08:00:22.407 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:22.398+0000 7f17e10d6700 1 -- 172.21.15.59:0/3667566115 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f17dc068670 con 0x7f17dc1447a0 2023-11-24T08:00:22.407 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:22.399+0000 7f17c37fe700 1 -- 172.21.15.59:0/3667566115 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 12) v1 ==== 89792+0+0 (secure 0 0 0) 0x7f17d4014480 con 0x7f17dc1447a0 2023-11-24T08:00:22.407 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:22.399+0000 7f17c37fe700 1 --2- 172.21.15.59:0/3667566115 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f17c806b530 0x7f17c806d9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:00:22.407 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:22.399+0000 7f17c37fe700 1 -- 172.21.15.59:0/3667566115 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 5795+0+0 (secure 0 0 0) 0x7f17d408a6d0 con 0x7f17dc1447a0 2023-11-24T08:00:22.408 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:22.400+0000 7f17dad9d700 1 --2- 172.21.15.59:0/3667566115 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f17c806b530 0x7f17c806d9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:00:22.408 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:22.400+0000 7f17dad9d700 1 --2- 172.21.15.59:0/3667566115 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f17c806b530 0x7f17c806d9f0 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7f17cc000f60 tx=0x7f17cc007e70 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:00:22.408 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:22.404+0000 7f17c37fe700 1 -- 172.21.15.59:0/3667566115 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f17d4054e60 con 0x7f17dc1447a0 2023-11-24T08:00:22.607 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:22.606+0000 7f17e10d6700 1 -- 172.21.15.59:0/3667566115 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f17dc06f9f0 con 0x7f17dc1447a0 2023-11-24T08:00:23.411 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:23.410+0000 7f17c37fe700 1 -- 172.21.15.59:0/3667566115 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v89) v1 ==== 181+0+0 (secure 0 0 0) 0x7f17d4058480 con 0x7f17dc1447a0 2023-11-24T08:00:23.415 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:23.413+0000 7f17e10d6700 1 -- 172.21.15.59:0/3667566115 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f17c806b530 msgr2=0x7f17c806d9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:00:23.415 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:23.413+0000 7f17e10d6700 1 --2- 172.21.15.59:0/3667566115 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f17c806b530 0x7f17c806d9f0 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7f17cc000f60 tx=0x7f17cc007e70 comp rx=0 tx=0).stop 2023-11-24T08:00:23.415 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:23.413+0000 7f17e10d6700 1 -- 172.21.15.59:0/3667566115 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f17dc1447a0 msgr2=0x7f17dc13d510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:00:23.415 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:23.413+0000 7f17e10d6700 1 --2- 172.21.15.59:0/3667566115 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f17dc1447a0 0x7f17dc13d510 secure :-1 s=READY pgs=1119 cs=0 l=1 rev1=1 crypto rx=0x7f17d4009f00 tx=0x7f17d400c3a0 comp rx=0 tx=0).stop 2023-11-24T08:00:23.417 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:23.413+0000 7f17e10d6700 1 -- 172.21.15.59:0/3667566115 shutdown_connections 2023-11-24T08:00:23.417 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:23.413+0000 7f17e10d6700 1 --2- 172.21.15.59:0/3667566115 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f17c806b530 0x7f17c806d9f0 unknown :-1 s=CLOSED pgs=512 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:00:23.417 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:23.413+0000 7f17e10d6700 1 --2- 172.21.15.59:0/3667566115 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f17dc1447a0 0x7f17dc13d510 unknown :-1 s=CLOSED pgs=1119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:00:23.417 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:23.413+0000 7f17e10d6700 1 --2- 172.21.15.59:0/3667566115 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f17dc0722a0 0x7f17dc14d770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:00:23.417 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:23.413+0000 7f17e10d6700 1 --2- 172.21.15.59:0/3667566115 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f17dc072ab0 0x7f17dc150210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:00:23.418 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:23.413+0000 7f17e10d6700 1 -- 172.21.15.59:0/3667566115 >> 172.21.15.59:0/3667566115 conn(0x7f17dc0b2760 msgr2=0x7f17dc13fb70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:00:23.418 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:23.413+0000 7f17e10d6700 1 -- 172.21.15.59:0/3667566115 shutdown_connections 2023-11-24T08:00:23.418 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:23.413+0000 7f17e10d6700 1 -- 172.21.15.59:0/3667566115 wait complete. 2023-11-24T08:00:23.418 INFO:teuthology.orchestra.run.smithi059.stderr:pool 'cephfs_metadata' removed 2023-11-24T08:00:23.430 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2023-11-24T08:00:23.567 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:23.565+0000 7fd592e17700 1 Processor -- start 2023-11-24T08:00:23.567 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:23.565+0000 7fd592e17700 1 -- start start 2023-11-24T08:00:23.569 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:23.566+0000 7fd592e17700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fd58c06afd0 0x7fd58c0b59d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:00:23.569 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:23.566+0000 7fd592e17700 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fd58c072350 0x7fd58c072760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:00:23.569 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:23.566+0000 7fd592e17700 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd58c071ec0 con 0x7fd58c06b3b0 2023-11-24T08:00:23.569 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:23.566+0000 7fd592e17700 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd58c071b90 con 0x7fd58c06afd0 2023-11-24T08:00:23.569 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:23.566+0000 7fd592e17700 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd58c15ed30 con 0x7fd58c072350 2023-11-24T08:00:23.570 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:23.566+0000 7fd590bb3700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fd58c06afd0 0x7fd58c0b59d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:00:23.570 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:23.566+0000 7fd58bfff700 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7fd58c06b3b0 0x7fd58c15e470 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:59946/0 (socket says 172.21.15.59:59946) 2023-11-24T08:00:23.570 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:23.566+0000 7fd58bfff700 1 -- 172.21.15.59:0/3940302779 learned_addr learned my addr 172.21.15.59:0/3940302779 (peer_addr_for_me v1:172.21.15.59:0/0) 2023-11-24T08:00:23.570 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:23.566+0000 7fd5913b4700 1 --2- 172.21.15.59:0/3940302779 >> v2:172.21.15.76:3300/0 conn(0x7fd58c072350 0x7fd58c072760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:00:23.570 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:23.566+0000 7fd590bb3700 1 -- 172.21.15.59:0/3940302779 >> v2:172.21.15.76:3300/0 conn(0x7fd58c072350 msgr2=0x7fd58c072760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:00:23.570 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:23.566+0000 7fd590bb3700 1 --2- 172.21.15.59:0/3940302779 >> v2:172.21.15.76:3300/0 conn(0x7fd58c072350 0x7fd58c072760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:00:23.570 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:23.566+0000 7fd590bb3700 1 -- 172.21.15.59:0/3940302779 >> v1:172.21.15.59:6789/0 conn(0x7fd58c06b3b0 legacy=0x7fd58c15e470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:00:23.570 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:23.566+0000 7fd58b7fe700 1 -- 172.21.15.59:0/3940302779 <== mon.0 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1308194564 0 0) 0x7fd58c071ec0 con 0x7fd58c06b3b0 2023-11-24T08:00:23.570 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:23.566+0000 7fd590bb3700 1 -- 172.21.15.59:0/3940302779 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd58c15f010 con 0x7fd58c06afd0 2023-11-24T08:00:23.570 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:23.566+0000 7fd590bb3700 1 --2- 172.21.15.59:0/3940302779 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd58c06afd0 0x7fd58c0b59d0 secure :-1 s=READY pgs=1121 cs=0 l=1 rev1=1 crypto rx=0x7fd58c0d0510 tx=0x7fd580009b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=154f7dabd649af3b server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:00:23.571 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:23.567+0000 7fd58b7fe700 1 -- 172.21.15.59:0/3940302779 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd580011830 con 0x7fd58c06afd0 2023-11-24T08:00:23.571 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:23.567+0000 7fd58b7fe700 1 -- 172.21.15.59:0/3940302779 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd580011990 con 0x7fd58c06afd0 2023-11-24T08:00:23.571 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:23.567+0000 7fd58b7fe700 1 -- 172.21.15.59:0/3940302779 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd580011d20 con 0x7fd58c06afd0 2023-11-24T08:00:23.571 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:23.567+0000 7fd592e17700 1 -- 172.21.15.59:0/3940302779 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd58c06afd0 msgr2=0x7fd58c0b59d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:00:23.571 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:23.567+0000 7fd592e17700 1 --2- 172.21.15.59:0/3940302779 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd58c06afd0 0x7fd58c0b59d0 secure :-1 s=READY pgs=1121 cs=0 l=1 rev1=1 crypto rx=0x7fd58c0d0510 tx=0x7fd580009b20 comp rx=0 tx=0).stop 2023-11-24T08:00:23.571 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:23.567+0000 7fd592e17700 1 -- 172.21.15.59:0/3940302779 shutdown_connections 2023-11-24T08:00:23.571 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:23.567+0000 7fd592e17700 1 --2- 172.21.15.59:0/3940302779 >> v2:172.21.15.76:3300/0 conn(0x7fd58c072350 0x7fd58c072760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:00:23.571 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:23.567+0000 7fd592e17700 1 --2- 172.21.15.59:0/3940302779 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd58c06afd0 0x7fd58c0b59d0 unknown :-1 s=CLOSED pgs=1121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:00:23.571 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:23.567+0000 7fd592e17700 1 -- 172.21.15.59:0/3940302779 >> 172.21.15.59:0/3940302779 conn(0x7fd58c0b2760 msgr2=0x7fd58c0b2b70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:00:23.571 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:23.567+0000 7fd592e17700 1 -- 172.21.15.59:0/3940302779 shutdown_connections 2023-11-24T08:00:23.571 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:23.567+0000 7fd592e17700 1 -- 172.21.15.59:0/3940302779 wait complete. 2023-11-24T08:00:23.574 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:23.569+0000 7fd592e17700 1 Processor -- start 2023-11-24T08:00:23.574 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:23.569+0000 7fd592e17700 1 -- start start 2023-11-24T08:00:23.575 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:23.570+0000 7fd592e17700 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fd58c06afd0 0x7fd58c153940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:00:23.575 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:23.570+0000 7fd592e17700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd58c06b3b0 0x7fd58c1f7790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:00:23.575 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:23.570+0000 7fd592e17700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd58c072350 0x7fd58c0ad860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:00:23.575 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:23.570+0000 7fd592e17700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd58c13e8b0 con 0x7fd58c06b3b0 2023-11-24T08:00:23.575 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:23.570+0000 7fd592e17700 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd58c13fcb0 con 0x7fd58c072350 2023-11-24T08:00:23.575 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:23.570+0000 7fd592e17700 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd58c1f86a0 con 0x7fd58c06afd0 2023-11-24T08:00:23.575 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:23.570+0000 7fd58bfff700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd58c06b3b0 0x7fd58c1f7790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:00:23.575 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:23.570+0000 7fd58bfff700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd58c06b3b0 0x7fd58c1f7790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:56478/0 (socket says 172.21.15.59:56478) 2023-11-24T08:00:23.575 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:23.570+0000 7fd58bfff700 1 -- 172.21.15.59:0/3924391577 learned_addr learned my addr 172.21.15.59:0/3924391577 (peer_addr_for_me v2:172.21.15.59:0/0) 2023-11-24T08:00:23.576 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:23.570+0000 7fd5913b4700 1 --2- 172.21.15.59:0/3924391577 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd58c072350 0x7fd58c0ad860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:00:23.576 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:23.570+0000 7fd590bb3700 1 --2- 172.21.15.59:0/3924391577 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fd58c06afd0 0x7fd58c153940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:00:23.576 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:23.570+0000 7fd58bfff700 1 -- 172.21.15.59:0/3924391577 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fd58c06afd0 msgr2=0x7fd58c153940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:00:23.576 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:23.570+0000 7fd58bfff700 1 --2- 172.21.15.59:0/3924391577 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fd58c06afd0 0x7fd58c153940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:00:23.576 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:23.570+0000 7fd58bfff700 1 -- 172.21.15.59:0/3924391577 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd58c072350 msgr2=0x7fd58c0ad860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:00:23.576 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:23.570+0000 7fd58bfff700 1 --2- 172.21.15.59:0/3924391577 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd58c072350 0x7fd58c0ad860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:00:23.576 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:23.570+0000 7fd58bfff700 1 -- 172.21.15.59:0/3924391577 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd5800094a0 con 0x7fd58c06b3b0 2023-11-24T08:00:23.576 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:23.570+0000 7fd590bb3700 1 --2- 172.21.15.59:0/3924391577 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fd58c06afd0 0x7fd58c153940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T08:00:23.577 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:23.571+0000 7fd58bfff700 1 --2- 172.21.15.59:0/3924391577 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd58c06b3b0 0x7fd58c1f7790 secure :-1 s=READY pgs=1122 cs=0 l=1 rev1=1 crypto rx=0x7fd57800c750 tx=0x7fd57800cb10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:00:23.577 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:23.571+0000 7fd5897fa700 1 -- 172.21.15.59:0/3924391577 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd578004c80 con 0x7fd58c06b3b0 2023-11-24T08:00:23.577 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:23.571+0000 7fd5897fa700 1 -- 172.21.15.59:0/3924391577 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd578004e00 con 0x7fd58c06b3b0 2023-11-24T08:00:23.577 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:23.571+0000 7fd592e17700 1 -- 172.21.15.59:0/3924391577 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd58c0adda0 con 0x7fd58c06b3b0 2023-11-24T08:00:23.577 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:23.571+0000 7fd5897fa700 1 -- 172.21.15.59:0/3924391577 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd578007310 con 0x7fd58c06b3b0 2023-11-24T08:00:23.577 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:23.571+0000 7fd592e17700 1 -- 172.21.15.59:0/3924391577 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd58c1f83c0 con 0x7fd58c06b3b0 2023-11-24T08:00:23.578 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:23.572+0000 7fd592e17700 1 -- 172.21.15.59:0/3924391577 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd58c068670 con 0x7fd58c06b3b0 2023-11-24T08:00:23.578 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:23.572+0000 7fd5897fa700 1 -- 172.21.15.59:0/3924391577 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 12) v1 ==== 89792+0+0 (secure 0 0 0) 0x7fd578007470 con 0x7fd58c06b3b0 2023-11-24T08:00:23.578 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:23.573+0000 7fd5897fa700 1 --2- 172.21.15.59:0/3924391577 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7fd57006b500 0x7fd57006d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:00:23.578 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:23.573+0000 7fd5897fa700 1 -- 172.21.15.59:0/3924391577 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 5357+0+0 (secure 0 0 0) 0x7fd578089ce0 con 0x7fd58c06b3b0 2023-11-24T08:00:23.578 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:23.573+0000 7fd590bb3700 1 --2- 172.21.15.59:0/3924391577 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7fd57006b500 0x7fd57006d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:00:23.579 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:23.573+0000 7fd590bb3700 1 --2- 172.21.15.59:0/3924391577 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7fd57006b500 0x7fd57006d9c0 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7fd5800090b0 tx=0x7fd580008030 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:00:23.579 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:23.577+0000 7fd5897fa700 1 -- 172.21.15.59:0/3924391577 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fd578054640 con 0x7fd58c06b3b0 2023-11-24T08:00:23.783 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:23.781+0000 7fd592e17700 1 -- 172.21.15.59:0/3924391577 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fd58c0b33f0 con 0x7fd58c06b3b0 2023-11-24T08:00:24.417 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:24.415+0000 7fd5897fa700 1 -- 172.21.15.59:0/3924391577 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v90) v1 ==== 169+0+0 (secure 0 0 0) 0x7fd578057c60 con 0x7fd58c06b3b0 2023-11-24T08:00:24.421 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:24.419+0000 7fd592e17700 1 -- 172.21.15.59:0/3924391577 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7fd57006b500 msgr2=0x7fd57006d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:00:24.422 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:24.419+0000 7fd592e17700 1 --2- 172.21.15.59:0/3924391577 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7fd57006b500 0x7fd57006d9c0 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7fd5800090b0 tx=0x7fd580008030 comp rx=0 tx=0).stop 2023-11-24T08:00:24.422 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:24.419+0000 7fd592e17700 1 -- 172.21.15.59:0/3924391577 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd58c06b3b0 msgr2=0x7fd58c1f7790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:00:24.422 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:24.419+0000 7fd592e17700 1 --2- 172.21.15.59:0/3924391577 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd58c06b3b0 0x7fd58c1f7790 secure :-1 s=READY pgs=1122 cs=0 l=1 rev1=1 crypto rx=0x7fd57800c750 tx=0x7fd57800cb10 comp rx=0 tx=0).stop 2023-11-24T08:00:24.422 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:24.420+0000 7fd592e17700 1 -- 172.21.15.59:0/3924391577 shutdown_connections 2023-11-24T08:00:24.423 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:24.420+0000 7fd592e17700 1 --2- 172.21.15.59:0/3924391577 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7fd57006b500 0x7fd57006d9c0 unknown :-1 s=CLOSED pgs=513 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:00:24.423 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:24.420+0000 7fd592e17700 1 --2- 172.21.15.59:0/3924391577 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd58c072350 0x7fd58c0ad860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:00:24.423 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:24.420+0000 7fd592e17700 1 --2- 172.21.15.59:0/3924391577 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fd58c06afd0 0x7fd58c153940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:00:24.423 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:24.420+0000 7fd592e17700 1 --2- 172.21.15.59:0/3924391577 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd58c06b3b0 0x7fd58c1f7790 unknown :-1 s=CLOSED pgs=1122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:00:24.423 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:24.420+0000 7fd592e17700 1 -- 172.21.15.59:0/3924391577 >> 172.21.15.59:0/3924391577 conn(0x7fd58c0b2760 msgr2=0x7fd58c1447d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:00:24.424 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:24.420+0000 7fd592e17700 1 -- 172.21.15.59:0/3924391577 shutdown_connections 2023-11-24T08:00:24.424 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:24.420+0000 7fd592e17700 1 -- 172.21.15.59:0/3924391577 wait complete. 2023-11-24T08:00:24.424 INFO:teuthology.orchestra.run.smithi059.stderr:pool 'cephfs_data' removed 2023-11-24T08:00:24.437 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2023-11-24T08:00:24.569 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:24.567+0000 7f21e378e700 1 Processor -- start 2023-11-24T08:00:24.569 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:24.567+0000 7f21e378e700 1 -- start start 2023-11-24T08:00:24.571 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:24.568+0000 7f21e378e700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f21dc1589a0 0x7f21dc072710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:00:24.571 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:24.568+0000 7f21e378e700 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f21dc0b5e40 0x7f21dc072c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:00:24.571 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:24.568+0000 7f21e378e700 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f21dc158d80 con 0x7f21dc06e300 2023-11-24T08:00:24.571 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:24.568+0000 7f21e378e700 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f21dc15b300 con 0x7f21dc1589a0 2023-11-24T08:00:24.571 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:24.568+0000 7f21e378e700 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f21dc15c120 con 0x7f21dc0b5e40 2023-11-24T08:00:24.571 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:24.568+0000 7f21e152a700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f21dc1589a0 0x7f21dc072710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:00:24.572 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:24.568+0000 7f21e152a700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f21dc1589a0 0x7f21dc072710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:56484/0 (socket says 172.21.15.59:56484) 2023-11-24T08:00:24.572 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:24.568+0000 7f21e152a700 1 -- 172.21.15.59:0/4003682743 learned_addr learned my addr 172.21.15.59:0/4003682743 (peer_addr_for_me v2:172.21.15.59:0/0) 2023-11-24T08:00:24.572 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:24.568+0000 7f21e0d29700 1 --2- 172.21.15.59:0/4003682743 >> v2:172.21.15.76:3300/0 conn(0x7f21dc0b5e40 0x7f21dc072c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:00:24.572 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:24.568+0000 7f21e152a700 1 -- 172.21.15.59:0/4003682743 >> v2:172.21.15.76:3300/0 conn(0x7f21dc0b5e40 msgr2=0x7f21dc072c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:00:24.572 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:24.568+0000 7f21e152a700 1 --2- 172.21.15.59:0/4003682743 >> v2:172.21.15.76:3300/0 conn(0x7f21dc0b5e40 0x7f21dc072c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:00:24.572 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:24.568+0000 7f21e152a700 1 -- 172.21.15.59:0/4003682743 >> v1:172.21.15.76:6789/0 conn(0x7f21dc06e300 legacy=0x7f21dc15e8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:00:24.572 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:24.568+0000 7f21e152a700 1 -- 172.21.15.59:0/4003682743 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f21dc15b510 con 0x7f21dc1589a0 2023-11-24T08:00:24.572 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:24.569+0000 7f21e152a700 1 --2- 172.21.15.59:0/4003682743 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f21dc1589a0 0x7f21dc072710 secure :-1 s=READY pgs=1123 cs=0 l=1 rev1=1 crypto rx=0x7f21cc0098f0 tx=0x7f21cc009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dda9dc887ef70cd1 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:00:24.572 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:24.569+0000 7f21d3fff700 1 -- 172.21.15.59:0/4003682743 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21cc004230 con 0x7f21dc1589a0 2023-11-24T08:00:24.572 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:24.569+0000 7f21d3fff700 1 -- 172.21.15.59:0/4003682743 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f21cc004390 con 0x7f21dc1589a0 2023-11-24T08:00:24.572 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:24.569+0000 7f21d3fff700 1 -- 172.21.15.59:0/4003682743 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21cc0036a0 con 0x7f21dc1589a0 2023-11-24T08:00:24.572 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:24.569+0000 7f21e378e700 1 -- 172.21.15.59:0/4003682743 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f21dc1589a0 msgr2=0x7f21dc072710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:00:24.573 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:24.569+0000 7f21e378e700 1 --2- 172.21.15.59:0/4003682743 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f21dc1589a0 0x7f21dc072710 secure :-1 s=READY pgs=1123 cs=0 l=1 rev1=1 crypto rx=0x7f21cc0098f0 tx=0x7f21cc009c00 comp rx=0 tx=0).stop 2023-11-24T08:00:24.573 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:24.569+0000 7f21e378e700 1 -- 172.21.15.59:0/4003682743 shutdown_connections 2023-11-24T08:00:24.573 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:24.569+0000 7f21e378e700 1 --2- 172.21.15.59:0/4003682743 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f21dc1589a0 0x7f21dc072710 unknown :-1 s=CLOSED pgs=1123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:00:24.573 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:24.569+0000 7f21e378e700 1 --2- 172.21.15.59:0/4003682743 >> v2:172.21.15.76:3300/0 conn(0x7f21dc0b5e40 0x7f21dc072c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:00:24.573 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:24.569+0000 7f21e378e700 1 -- 172.21.15.59:0/4003682743 >> 172.21.15.59:0/4003682743 conn(0x7f21dc0b5070 msgr2=0x7f21dc0b5730 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:00:24.573 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:24.569+0000 7f21e378e700 1 -- 172.21.15.59:0/4003682743 shutdown_connections 2023-11-24T08:00:24.573 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:24.569+0000 7f21e378e700 1 -- 172.21.15.59:0/4003682743 wait complete. 2023-11-24T08:00:24.576 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:24.571+0000 7f21e378e700 1 Processor -- start 2023-11-24T08:00:24.576 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:24.571+0000 7f21e378e700 1 -- start start 2023-11-24T08:00:24.576 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:24.571+0000 7f21e378e700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f21dc06e300 0x7f21dc159aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:00:24.576 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:24.571+0000 7f21e378e700 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f21dc0b5e40 0x7f21dc09d7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:00:24.576 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:24.571+0000 7f21e378e700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f21dc1589a0 0x7f21dc1518a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:00:24.576 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:24.571+0000 7f21e378e700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f21dc159fe0 con 0x7f21dc06e300 2023-11-24T08:00:24.576 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:24.571+0000 7f21e378e700 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f21dc15af30 con 0x7f21dc1589a0 2023-11-24T08:00:24.576 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:24.571+0000 7f21e378e700 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f21dc09dd30 con 0x7f21dc0b5e40 2023-11-24T08:00:24.576 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:24.572+0000 7f21e152a700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f21dc06e300 0x7f21dc159aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:00:24.576 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:24.572+0000 7f21e152a700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f21dc06e300 0x7f21dc159aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:56490/0 (socket says 172.21.15.59:56490) 2023-11-24T08:00:24.577 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:24.572+0000 7f21e152a700 1 -- 172.21.15.59:0/1984157953 learned_addr learned my addr 172.21.15.59:0/1984157953 (peer_addr_for_me v2:172.21.15.59:0/0) 2023-11-24T08:00:24.577 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:24.572+0000 7f21e0d29700 1 --2- 172.21.15.59:0/1984157953 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f21dc0b5e40 0x7f21dc09d7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:00:24.577 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:24.572+0000 7f21e1d2b700 1 --2- 172.21.15.59:0/1984157953 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f21dc1589a0 0x7f21dc1518a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:00:24.577 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:24.572+0000 7f21e152a700 1 -- 172.21.15.59:0/1984157953 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f21dc0b5e40 msgr2=0x7f21dc09d7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:00:24.577 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:24.572+0000 7f21e152a700 1 --2- 172.21.15.59:0/1984157953 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f21dc0b5e40 0x7f21dc09d7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:00:24.577 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:24.572+0000 7f21e152a700 1 -- 172.21.15.59:0/1984157953 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f21dc1589a0 msgr2=0x7f21dc1518a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:00:24.577 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:24.572+0000 7f21e152a700 1 --2- 172.21.15.59:0/1984157953 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f21dc1589a0 0x7f21dc1518a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:00:24.577 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:24.572+0000 7f21e152a700 1 -- 172.21.15.59:0/1984157953 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f21cc009550 con 0x7f21dc06e300 2023-11-24T08:00:24.577 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:24.572+0000 7f21e0d29700 1 --2- 172.21.15.59:0/1984157953 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f21dc0b5e40 0x7f21dc09d7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T08:00:24.577 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:24.572+0000 7f21e1d2b700 1 --2- 172.21.15.59:0/1984157953 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f21dc1589a0 0x7f21dc1518a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T08:00:24.577 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:24.572+0000 7f21e152a700 1 --2- 172.21.15.59:0/1984157953 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f21dc06e300 0x7f21dc159aa0 secure :-1 s=READY pgs=1124 cs=0 l=1 rev1=1 crypto rx=0x7f21cc000c00 tx=0x7f21cc004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:00:24.577 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:24.572+0000 7f21d1ffb700 1 -- 172.21.15.59:0/1984157953 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21cc011830 con 0x7f21dc06e300 2023-11-24T08:00:24.578 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:24.572+0000 7f21d1ffb700 1 -- 172.21.15.59:0/1984157953 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f21cc011990 con 0x7f21dc06e300 2023-11-24T08:00:24.578 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:24.572+0000 7f21e378e700 1 -- 172.21.15.59:0/1984157953 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f21dc15ac50 con 0x7f21dc06e300 2023-11-24T08:00:24.578 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:24.572+0000 7f21d1ffb700 1 -- 172.21.15.59:0/1984157953 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21cc011d40 con 0x7f21dc06e300 2023-11-24T08:00:24.578 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:24.572+0000 7f21e378e700 1 -- 172.21.15.59:0/1984157953 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f21dc1592a0 con 0x7f21dc06e300 2023-11-24T08:00:24.578 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:24.573+0000 7f21e378e700 1 -- 172.21.15.59:0/1984157953 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f21dc075480 con 0x7f21dc06e300 2023-11-24T08:00:24.578 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:24.574+0000 7f21d1ffb700 1 -- 172.21.15.59:0/1984157953 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 12) v1 ==== 89792+0+0 (secure 0 0 0) 0x7f21cc01b440 con 0x7f21dc06e300 2023-11-24T08:00:24.578 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:24.574+0000 7f21d1ffb700 1 --2- 172.21.15.59:0/1984157953 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f21c006b500 0x7f21c006d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:00:24.578 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:24.574+0000 7f21d1ffb700 1 -- 172.21.15.59:0/1984157953 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f21cc08b700 con 0x7f21dc06e300 2023-11-24T08:00:24.578 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:24.574+0000 7f21e0d29700 1 --2- 172.21.15.59:0/1984157953 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f21c006b500 0x7f21c006d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:00:24.579 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:24.575+0000 7f21e0d29700 1 --2- 172.21.15.59:0/1984157953 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f21c006b500 0x7f21c006d9c0 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7f21d8006fd0 tx=0x7f21d8008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:00:24.580 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:24.579+0000 7f21d1ffb700 1 -- 172.21.15.59:0/1984157953 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f21cc0561c0 con 0x7f21dc06e300 2023-11-24T08:00:24.783 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:24.782+0000 7f21e378e700 1 -- 172.21.15.59:0/1984157953 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7f21dc157040 con 0x7f21dc06e300 2023-11-24T08:00:24.786 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:24.785+0000 7f21d1ffb700 1 -- 172.21.15.59:0/1984157953 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v19) v1 ==== 104+0+0 (secure 0 0 0) 0x7f21cc0597e0 con 0x7f21dc06e300 2023-11-24T08:00:24.789 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:24.787+0000 7f21e378e700 1 -- 172.21.15.59:0/1984157953 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f21c006b500 msgr2=0x7f21c006d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:00:24.789 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:24.787+0000 7f21e378e700 1 --2- 172.21.15.59:0/1984157953 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f21c006b500 0x7f21c006d9c0 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7f21d8006fd0 tx=0x7f21d8008040 comp rx=0 tx=0).stop 2023-11-24T08:00:24.790 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:24.787+0000 7f21e378e700 1 -- 172.21.15.59:0/1984157953 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f21dc06e300 msgr2=0x7f21dc159aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:00:24.790 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:24.787+0000 7f21e378e700 1 --2- 172.21.15.59:0/1984157953 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f21dc06e300 0x7f21dc159aa0 secure :-1 s=READY pgs=1124 cs=0 l=1 rev1=1 crypto rx=0x7f21cc000c00 tx=0x7f21cc004750 comp rx=0 tx=0).stop 2023-11-24T08:00:24.790 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:24.787+0000 7f21e378e700 1 -- 172.21.15.59:0/1984157953 shutdown_connections 2023-11-24T08:00:24.790 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:24.787+0000 7f21e378e700 1 --2- 172.21.15.59:0/1984157953 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f21c006b500 0x7f21c006d9c0 unknown :-1 s=CLOSED pgs=514 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:00:24.790 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:24.787+0000 7f21e378e700 1 --2- 172.21.15.59:0/1984157953 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f21dc1589a0 0x7f21dc1518a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:00:24.790 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:24.787+0000 7f21e378e700 1 --2- 172.21.15.59:0/1984157953 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f21dc06e300 0x7f21dc159aa0 unknown :-1 s=CLOSED pgs=1124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:00:24.790 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:24.787+0000 7f21e378e700 1 --2- 172.21.15.59:0/1984157953 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f21dc0b5e40 0x7f21dc09d7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:00:24.790 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:24.787+0000 7f21e378e700 1 -- 172.21.15.59:0/1984157953 >> 172.21.15.59:0/1984157953 conn(0x7f21dc0b5070 msgr2=0x7f21dc0ad670 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:00:24.790 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:24.787+0000 7f21e378e700 1 -- 172.21.15.59:0/1984157953 shutdown_connections 2023-11-24T08:00:24.790 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:24.788+0000 7f21e378e700 1 -- 172.21.15.59:0/1984157953 wait complete. 2023-11-24T08:00:24.801 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_volumes.TestSubvolumeSnapshots.test_subvolume_inherited_snapshot_ls' 2023-11-24T08:00:24.929 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:24.927+0000 7f00ab579700 1 Processor -- start 2023-11-24T08:00:24.929 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:24.927+0000 7f00ab579700 1 -- start start 2023-11-24T08:00:24.937 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:24.928+0000 7f00ab579700 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f00a4143ae0 0x7f00a40b51b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:00:24.937 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:24.928+0000 7f00ab579700 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f00a41440f0 0x7f00a4071bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:00:24.937 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:24.928+0000 7f00ab579700 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f00a4143ec0 con 0x7f00a40720f0 2023-11-24T08:00:24.937 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:24.928+0000 7f00ab579700 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f00a40b40a0 con 0x7f00a41440f0 2023-11-24T08:00:24.938 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:24.928+0000 7f00ab579700 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f00a4072580 con 0x7f00a4143ae0 2023-11-24T08:00:24.938 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:24.928+0000 7f00a9b16700 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7f00a40720f0 0x7f00a415e8b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:59976/0 (socket says 172.21.15.59:59976) 2023-11-24T08:00:24.938 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:24.928+0000 7f00a9b16700 1 -- 172.21.15.59:0/1372375004 learned_addr learned my addr 172.21.15.59:0/1372375004 (peer_addr_for_me v1:172.21.15.59:0/0) 2023-11-24T08:00:24.938 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:24.928+0000 7f00a8b14700 1 --2- 172.21.15.59:0/1372375004 >> v2:172.21.15.76:3300/0 conn(0x7f00a41440f0 0x7f00a4071bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:00:24.938 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:24.928+0000 7f00a9315700 1 --2- 172.21.15.59:0/1372375004 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f00a4143ae0 0x7f00a40b51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:00:24.938 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:24.928+0000 7f009bfff700 1 -- 172.21.15.59:0/1372375004 <== mon.0 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2124377746 0 0) 0x7f00a4143ec0 con 0x7f00a40720f0 2023-11-24T08:00:24.938 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:24.929+0000 7f009bfff700 1 -- 172.21.15.59:0/1372375004 --> v1:172.21.15.59:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f00900035b0 con 0x7f00a40720f0 2023-11-24T08:00:24.938 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:24.929+0000 7f009bfff700 1 -- 172.21.15.59:0/1372375004 <== mon.0 v1:172.21.15.59:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2455043855 0 0) 0x7f00900035b0 con 0x7f00a40720f0 2023-11-24T08:00:24.939 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:24.929+0000 7f009bfff700 1 -- 172.21.15.59:0/1372375004 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f00a4143ae0 msgr2=0x7f00a40b51b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:00:24.939 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:24.929+0000 7f009bfff700 1 --2- 172.21.15.59:0/1372375004 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f00a4143ae0 0x7f00a40b51b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:00:24.939 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:24.929+0000 7f009bfff700 1 -- 172.21.15.59:0/1372375004 >> v2:172.21.15.76:3300/0 conn(0x7f00a41440f0 msgr2=0x7f00a4071bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:00:24.939 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:24.929+0000 7f009bfff700 1 --2- 172.21.15.59:0/1372375004 >> v2:172.21.15.76:3300/0 conn(0x7f00a41440f0 0x7f00a4071bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:00:24.939 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:24.929+0000 7f009bfff700 1 -- 172.21.15.59:0/1372375004 --> v1:172.21.15.59:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f00a40b3ae0 con 0x7f00a40720f0 2023-11-24T08:00:24.939 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:24.929+0000 7f009bfff700 1 -- 172.21.15.59:0/1372375004 <== mon.0 v1:172.21.15.59:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1312333566 0 0) 0x7f009c0019f0 con 0x7f00a40720f0 2023-11-24T08:00:24.939 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:24.929+0000 7f009bfff700 1 -- 172.21.15.59:0/1372375004 >> v1:172.21.15.59:6789/0 conn(0x7f00a40720f0 legacy=0x7f00a415e8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:00:24.939 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:24.929+0000 7f009bfff700 1 --2- 172.21.15.59:0/1372375004 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0090003a00 0x7f0090003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:00:24.940 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:24.929+0000 7f009bfff700 1 --2- 172.21.15.59:0/1372375004 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f00a4143ae0 0x7f0090004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:00:24.940 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:24.929+0000 7f009bfff700 1 --2- 172.21.15.59:0/1372375004 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0090004d90 0x7f0090005210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:00:24.940 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:24.929+0000 7f009bfff700 1 -- 172.21.15.59:0/1372375004 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00a40b40a0 con 0x7f0090003a00 2023-11-24T08:00:24.940 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:24.929+0000 7f009bfff700 1 -- 172.21.15.59:0/1372375004 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00a4072580 con 0x7f0090004d90 2023-11-24T08:00:24.940 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:24.929+0000 7f009bfff700 1 -- 172.21.15.59:0/1372375004 --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f009000ba10 con 0x7f00a4143ae0 2023-11-24T08:00:24.940 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:24.929+0000 7f00a9315700 1 --2- 172.21.15.59:0/1372375004 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0090003a00 0x7f0090003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:00:24.941 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:24.929+0000 7f00a9b16700 1 --2- 172.21.15.59:0/1372375004 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0090004d90 0x7f0090005210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:00:24.941 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:24.929+0000 7f00a8b14700 1 --2- 172.21.15.59:0/1372375004 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f00a4143ae0 0x7f0090004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:00:24.941 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:24.929+0000 7f00a9315700 1 -- 172.21.15.59:0/1372375004 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f00a4143ae0 msgr2=0x7f0090004850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:00:24.941 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:24.929+0000 7f00a9315700 1 --2- 172.21.15.59:0/1372375004 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f00a4143ae0 0x7f0090004850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:00:24.941 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:24.929+0000 7f00a9315700 1 -- 172.21.15.59:0/1372375004 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0090004d90 msgr2=0x7f0090005210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:00:24.941 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:24.929+0000 7f00a9315700 1 --2- 172.21.15.59:0/1372375004 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0090004d90 0x7f0090005210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:00:24.941 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:24.929+0000 7f00a9315700 1 -- 172.21.15.59:0/1372375004 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f00900035b0 con 0x7f0090003a00 2023-11-24T08:00:24.941 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:24.929+0000 7f00a9315700 1 --2- 172.21.15.59:0/1372375004 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0090003a00 0x7f0090003e20 secure :-1 s=READY pgs=1126 cs=0 l=1 rev1=1 crypto rx=0x7f0094009b60 tx=0x7f0090005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8887936d57a3bc18 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:00:24.942 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:24.930+0000 7f009bfff700 1 -- 172.21.15.59:0/1372375004 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f009400e6c0 con 0x7f0090003a00 2023-11-24T08:00:24.942 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:24.930+0000 7f009bfff700 1 -- 172.21.15.59:0/1372375004 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f009400e820 con 0x7f0090003a00 2023-11-24T08:00:24.942 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:24.930+0000 7f009bfff700 1 -- 172.21.15.59:0/1372375004 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f009400eb90 con 0x7f0090003a00 2023-11-24T08:00:24.942 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:24.930+0000 7f00ab579700 1 -- 172.21.15.59:0/1372375004 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0090003a00 msgr2=0x7f0090003e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:00:24.942 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:24.930+0000 7f00ab579700 1 --2- 172.21.15.59:0/1372375004 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0090003a00 0x7f0090003e20 secure :-1 s=READY pgs=1126 cs=0 l=1 rev1=1 crypto rx=0x7f0094009b60 tx=0x7f0090005880 comp rx=0 tx=0).stop 2023-11-24T08:00:24.942 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:24.930+0000 7f00a9b16700 1 -- 172.21.15.59:0/1372375004 reap_dead start 2023-11-24T08:00:24.943 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:24.930+0000 7f00ab579700 1 -- 172.21.15.59:0/1372375004 shutdown_connections 2023-11-24T08:00:24.943 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:24.930+0000 7f00ab579700 1 -- 172.21.15.59:0/1372375004 >> 172.21.15.59:0/1372375004 conn(0x7f00a40b2740 msgr2=0x7f00a40b2b20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:00:24.943 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:24.930+0000 7f00ab579700 1 -- 172.21.15.59:0/1372375004 shutdown_connections 2023-11-24T08:00:24.943 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:24.930+0000 7f00ab579700 1 -- 172.21.15.59:0/1372375004 wait complete. 2023-11-24T08:00:24.943 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:24.931+0000 7f00ab579700 1 Processor -- start 2023-11-24T08:00:24.943 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:24.931+0000 7f00ab579700 1 -- start start 2023-11-24T08:00:24.943 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:24.932+0000 7f00ab579700 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f0090003a00 0x7f00a4142740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:00:24.944 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:24.932+0000 7f00ab579700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f00a41f7100 0x7f00a4141b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:00:24.944 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:24.932+0000 7f00ab579700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f00a40ca9e0 0x7f00a40cae70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:00:24.944 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:24.932+0000 7f00ab579700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00a415c810 con 0x7f00a41f7100 2023-11-24T08:00:24.944 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:24.932+0000 7f00ab579700 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00a4142090 con 0x7f00a40ca9e0 2023-11-24T08:00:24.944 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:24.932+0000 7f00ab579700 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f00a40cb3b0 con 0x7f0090003a00 2023-11-24T08:00:24.944 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:24.932+0000 7f00a8b14700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f00a41f7100 0x7f00a4141b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:00:24.945 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:24.932+0000 7f00a8b14700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f00a41f7100 0x7f00a4141b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:56512/0 (socket says 172.21.15.59:56512) 2023-11-24T08:00:24.945 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:24.932+0000 7f00a8b14700 1 -- 172.21.15.59:0/3775644495 learned_addr learned my addr 172.21.15.59:0/3775644495 (peer_addr_for_me v2:172.21.15.59:0/0) 2023-11-24T08:00:24.945 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:24.932+0000 7f00a9b16700 1 --2- 172.21.15.59:0/3775644495 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f00a40ca9e0 0x7f00a40cae70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:00:24.945 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:24.932+0000 7f00a9315700 1 --2- 172.21.15.59:0/3775644495 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f0090003a00 0x7f00a4142740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:00:24.946 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:24.933+0000 7f00a8b14700 1 -- 172.21.15.59:0/3775644495 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f0090003a00 msgr2=0x7f00a4142740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:00:24.946 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:24.933+0000 7f00a8b14700 1 --2- 172.21.15.59:0/3775644495 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f0090003a00 0x7f00a4142740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:00:24.946 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:24.933+0000 7f00a8b14700 1 -- 172.21.15.59:0/3775644495 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f00a40ca9e0 msgr2=0x7f00a40cae70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:00:24.946 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:24.933+0000 7f00a8b14700 1 --2- 172.21.15.59:0/3775644495 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f00a40ca9e0 0x7f00a40cae70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:00:24.946 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:24.933+0000 7f00a8b14700 1 -- 172.21.15.59:0/3775644495 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f00a41f7600 con 0x7f00a41f7100 2023-11-24T08:00:24.946 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:24.933+0000 7f00a9315700 1 --2- 172.21.15.59:0/3775644495 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f0090003a00 0x7f00a4142740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-24T08:00:24.947 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:24.933+0000 7f00a8b14700 1 --2- 172.21.15.59:0/3775644495 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f00a41f7100 0x7f00a4141b50 secure :-1 s=READY pgs=1127 cs=0 l=1 rev1=1 crypto rx=0x7f00a0011b00 tx=0x7f00a0011e10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:00:24.947 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:24.933+0000 7f0099ffb700 1 -- 172.21.15.59:0/3775644495 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00a000f9a0 con 0x7f00a41f7100 2023-11-24T08:00:24.947 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:24.933+0000 7f0099ffb700 1 -- 172.21.15.59:0/3775644495 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f00a000fb00 con 0x7f00a41f7100 2023-11-24T08:00:24.947 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:24.933+0000 7f00ab579700 1 -- 172.21.15.59:0/3775644495 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f00a41f78e0 con 0x7f00a41f7100 2023-11-24T08:00:24.947 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:24.933+0000 7f0099ffb700 1 -- 172.21.15.59:0/3775644495 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00a000feb0 con 0x7f00a41f7100 2023-11-24T08:00:24.948 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:24.933+0000 7f00ab579700 1 -- 172.21.15.59:0/3775644495 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f00a41f7ea0 con 0x7f00a41f7100 2023-11-24T08:00:24.948 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:24.934+0000 7f00ab579700 1 -- 172.21.15.59:0/3775644495 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f00a415cab0 con 0x7f00a41f7100 2023-11-24T08:00:24.948 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:24.935+0000 7f0099ffb700 1 -- 172.21.15.59:0/3775644495 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 12) v1 ==== 89792+0+0 (secure 0 0 0) 0x7f00a0014440 con 0x7f00a41f7100 2023-11-24T08:00:24.948 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:24.935+0000 7f0099ffb700 1 --2- 172.21.15.59:0/3775644495 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f008806b4f0 0x7f008806d9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:00:24.948 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:24.935+0000 7f0099ffb700 1 -- 172.21.15.59:0/3775644495 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f00a008a360 con 0x7f00a41f7100 2023-11-24T08:00:24.948 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:24.936+0000 7f00a9315700 1 --2- 172.21.15.59:0/3775644495 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f008806b4f0 0x7f008806d9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:00:24.949 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:24.936+0000 7f00a9315700 1 --2- 172.21.15.59:0/3775644495 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f008806b4f0 0x7f008806d9b0 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7f00940090b0 tx=0x7f0094008030 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:00:24.950 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:24.940+0000 7f0099ffb700 1 -- 172.21.15.59:0/3775644495 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f00a0054ed0 con 0x7f00a41f7100 2023-11-24T08:00:25.229 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:25.228+0000 7f00ab579700 1 -- 172.21.15.59:0/3775644495 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_volumes.TestSubvolumeSnapshots.test_subvolume_inherited_snapshot_ls"]} v 0) v1 -- 0x7f00a4068670 con 0x7f00a41f7100 2023-11-24T08:00:25.418 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:25.417+0000 7f0099ffb700 1 -- 172.21.15.59:0/3775644495 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_volumes.TestSubvolumeSnapshots.test_subvolume_inherited_snapshot_ls"]}]=0 v242) v1 ==== 164+0+0 (secure 0 0 0) 0x7f00a00584f0 con 0x7f00a41f7100 2023-11-24T08:00:25.421 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:25.419+0000 7f00ab579700 1 -- 172.21.15.59:0/3775644495 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f008806b4f0 msgr2=0x7f008806d9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:00:25.421 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:25.419+0000 7f00ab579700 1 --2- 172.21.15.59:0/3775644495 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f008806b4f0 0x7f008806d9b0 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7f00940090b0 tx=0x7f0094008030 comp rx=0 tx=0).stop 2023-11-24T08:00:25.421 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:25.419+0000 7f00ab579700 1 -- 172.21.15.59:0/3775644495 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f00a41f7100 msgr2=0x7f00a4141b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:00:25.421 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:25.419+0000 7f00ab579700 1 --2- 172.21.15.59:0/3775644495 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f00a41f7100 0x7f00a4141b50 secure :-1 s=READY pgs=1127 cs=0 l=1 rev1=1 crypto rx=0x7f00a0011b00 tx=0x7f00a0011e10 comp rx=0 tx=0).stop 2023-11-24T08:00:25.421 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:25.419+0000 7f00ab579700 1 -- 172.21.15.59:0/3775644495 shutdown_connections 2023-11-24T08:00:25.421 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:25.419+0000 7f00ab579700 1 --2- 172.21.15.59:0/3775644495 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f008806b4f0 0x7f008806d9b0 unknown :-1 s=CLOSED pgs=515 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:00:25.422 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:25.419+0000 7f00ab579700 1 --2- 172.21.15.59:0/3775644495 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f00a40ca9e0 0x7f00a40cae70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:00:25.422 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:25.419+0000 7f00ab579700 1 --2- 172.21.15.59:0/3775644495 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f0090003a00 0x7f00a4142740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:00:25.422 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:25.419+0000 7f00ab579700 1 --2- 172.21.15.59:0/3775644495 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f00a41f7100 0x7f00a4141b50 unknown :-1 s=CLOSED pgs=1127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:00:25.422 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:25.419+0000 7f00ab579700 1 -- 172.21.15.59:0/3775644495 >> 172.21.15.59:0/3775644495 conn(0x7f00a40b2740 msgr2=0x7f00a415f0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:00:25.422 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:25.419+0000 7f00ab579700 1 -- 172.21.15.59:0/3775644495 shutdown_connections 2023-11-24T08:00:25.422 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:25.419+0000 7f00ab579700 1 -- 172.21.15.59:0/3775644495 wait complete. 2023-11-24T08:00:25.434 INFO:tasks.cephfs_test_runner:test_subvolume_inherited_snapshot_ls (tasks.cephfs.test_volumes.TestSubvolumeSnapshots) ... ERROR 2023-11-24T08:00:25.435 INFO:tasks.cephfs_test_runner: 2023-11-24T08:00:25.435 INFO:tasks.cephfs_test_runner:====================================================================== 2023-11-24T08:00:25.435 INFO:tasks.cephfs_test_runner:ERROR: test_subvolume_inherited_snapshot_ls (tasks.cephfs.test_volumes.TestSubvolumeSnapshots) 2023-11-24T08:00:25.435 INFO:tasks.cephfs_test_runner:---------------------------------------------------------------------- 2023-11-24T08:00:25.435 INFO:tasks.cephfs_test_runner:Traceback (most recent call last): 2023-11-24T08:00:25.435 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/github.com_rishabh-d-dave_ceph_73843baf45f8a92d8e3fcbea3f024c97e73036da/qa/tasks/cephfs/test_volumes.py", line 4530, in test_subvolume_inherited_snapshot_ls 2023-11-24T08:00:25.435 INFO:tasks.cephfs_test_runner: snapshots = self._generate_random_snapshot_name(snap_count) 2023-11-24T08:00:25.435 INFO:tasks.cephfs_test_runner:AttributeError: 'TestSubvolumeSnapshots' object has no attribute '_generate_random_snapshot_name' 2023-11-24T08:00:25.435 INFO:tasks.cephfs_test_runner: 2023-11-24T08:00:25.435 INFO:tasks.cephfs_test_runner:---------------------------------------------------------------------- 2023-11-24T08:00:25.435 INFO:tasks.cephfs_test_runner:Ran 10 tests in 206.744s 2023-11-24T08:00:25.436 INFO:tasks.cephfs_test_runner: 2023-11-24T08:00:25.436 INFO:tasks.cephfs_test_runner:FAILED (errors=1, skipped=5) 2023-11-24T08:00:25.436 INFO:tasks.cephfs_test_runner: 2023-11-24T08:00:25.436 INFO:tasks.cephfs_test_runner:====================================================================== 2023-11-24T08:00:25.436 INFO:tasks.cephfs_test_runner:ERROR: test_subvolume_inherited_snapshot_ls (tasks.cephfs.test_volumes.TestSubvolumeSnapshots) 2023-11-24T08:00:25.436 INFO:tasks.cephfs_test_runner:---------------------------------------------------------------------- 2023-11-24T08:00:25.436 INFO:tasks.cephfs_test_runner:Traceback (most recent call last): 2023-11-24T08:00:25.436 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/github.com_rishabh-d-dave_ceph_73843baf45f8a92d8e3fcbea3f024c97e73036da/qa/tasks/cephfs/test_volumes.py", line 4530, in test_subvolume_inherited_snapshot_ls 2023-11-24T08:00:25.436 INFO:tasks.cephfs_test_runner: snapshots = self._generate_random_snapshot_name(snap_count) 2023-11-24T08:00:25.436 INFO:tasks.cephfs_test_runner:AttributeError: 'TestSubvolumeSnapshots' object has no attribute '_generate_random_snapshot_name' 2023-11-24T08:00:25.436 INFO:tasks.cephfs_test_runner: 2023-11-24T08:00:25.437 ERROR:teuthology.run_tasks:Saw exception from tasks. Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_2442239d2653456406c25ae0c71b689c8f2657b6/teuthology/run_tasks.py", line 109, in run_tasks manager.__enter__() File "/usr/lib/python3.8/contextlib.py", line 113, in __enter__ return next(self.gen) File "/home/teuthworker/src/github.com_rishabh-d-dave_ceph_73843baf45f8a92d8e3fcbea3f024c97e73036da/qa/tasks/cephfs_test_runner.py", line 211, in task raise RuntimeError("Test failure: {0}".format(", ".join(bad_tests))) RuntimeError: Test failure: test_subvolume_inherited_snapshot_ls (tasks.cephfs.test_volumes.TestSubvolumeSnapshots) 2023-11-24T08:00:25.539 ERROR:teuthology.util.sentry: Sentry event: https://sentry.ceph.com/organizations/ceph/?query=309312f7e36d402987032f090b30c5f4 Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_2442239d2653456406c25ae0c71b689c8f2657b6/teuthology/run_tasks.py", line 109, in run_tasks manager.__enter__() File "/usr/lib/python3.8/contextlib.py", line 113, in __enter__ return next(self.gen) File "/home/teuthworker/src/github.com_rishabh-d-dave_ceph_73843baf45f8a92d8e3fcbea3f024c97e73036da/qa/tasks/cephfs_test_runner.py", line 211, in task raise RuntimeError("Test failure: {0}".format(", ".join(bad_tests))) RuntimeError: Test failure: test_subvolume_inherited_snapshot_ls (tasks.cephfs.test_volumes.TestSubvolumeSnapshots) 2023-11-24T08:00:25.540 DEBUG:teuthology.run_tasks:Unwinding manager cephfs_test_runner 2023-11-24T08:00:25.549 DEBUG:teuthology.run_tasks:Unwinding manager kclient 2023-11-24T08:00:25.582 INFO:tasks.kclient:Unmounting kernel clients... 2023-11-24T08:00:25.582 DEBUG:teuthology.orchestra.run.smithi059:> set -ex 2023-11-24T08:00:25.582 DEBUG:teuthology.orchestra.run.smithi059:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-24T08:00:25.605 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=16287364k,nr_inodes=4071841,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 tmpfs /sys/fs/cgroup tmpfs ro,seclabel,nosuid,nodev,noexec,mode=755 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,seclabel,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,seclabel,nosuid,nodev,noexec,relatime,hugetlb 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,seclabel,nosuid,nodev,noexec,relatime,perf_event 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,seclabel,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/rdma cgroup rw,seclabel,nosuid,nodev,noexec,relatime,rdma 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,seclabel,nosuid,nodev,noexec,relatime,freezer 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,seclabel,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,seclabel,nosuid,nodev,noexec,relatime,devices 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,blkio 0 0 none /sys/kernel/tracing tracefs rw,seclabel,relatime 0 0 configfs /sys/kernel/config configfs rw,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=42,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=16075 0 0 mqueue /dev/mqueue mqueue rw,seclabel,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,relatime 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3261172k,mode=700,uid=1000,gid=1265 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw,seclabel 0 0 2023-11-24T08:00:25.605 DEBUG:teuthology.orchestra.run.smithi059:> set -ex 2023-11-24T08:00:25.605 DEBUG:teuthology.orchestra.run.smithi059:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-24T08:00:25.662 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=16287364k,nr_inodes=4071841,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 tmpfs /sys/fs/cgroup tmpfs ro,seclabel,nosuid,nodev,noexec,mode=755 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,seclabel,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,seclabel,nosuid,nodev,noexec,relatime,hugetlb 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,seclabel,nosuid,nodev,noexec,relatime,perf_event 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,seclabel,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/rdma cgroup rw,seclabel,nosuid,nodev,noexec,relatime,rdma 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,seclabel,nosuid,nodev,noexec,relatime,freezer 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,seclabel,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,seclabel,nosuid,nodev,noexec,relatime,devices 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,blkio 0 0 none /sys/kernel/tracing tracefs rw,seclabel,relatime 0 0 configfs /sys/kernel/config configfs rw,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=42,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=16075 0 0 mqueue /dev/mqueue mqueue rw,seclabel,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,relatime 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3261172k,mode=700,uid=1000,gid=1265 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw,seclabel 0 0 2023-11-24T08:00:25.663 DEBUG:teuthology.orchestra.run.smithi076:> set -ex 2023-11-24T08:00:25.663 DEBUG:teuthology.orchestra.run.smithi076:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-24T08:00:25.684 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=16284272k,nr_inodes=4071068,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 tmpfs /sys/fs/cgroup tmpfs ro,seclabel,nosuid,nodev,noexec,mode=755 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,seclabel,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,seclabel,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,seclabel,nosuid,nodev,noexec,relatime,freezer 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,seclabel,nosuid,nodev,noexec,relatime,perf_event 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,seclabel,nosuid,nodev,noexec,relatime,hugetlb 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,seclabel,nosuid,nodev,noexec,relatime,devices 0 0 cgroup /sys/fs/cgroup/rdma cgroup rw,seclabel,nosuid,nodev,noexec,relatime,rdma 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,seclabel,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,blkio 0 0 none /sys/kernel/tracing tracefs rw,seclabel,relatime 0 0 configfs /sys/kernel/config configfs rw,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=35,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=20116 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,relatime 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3260552k,mode=700,uid=1000,gid=1265 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw,seclabel 0 0 2023-11-24T08:00:25.684 DEBUG:teuthology.orchestra.run.smithi076:> set -ex 2023-11-24T08:00:25.684 DEBUG:teuthology.orchestra.run.smithi076:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-24T08:00:25.745 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=16284272k,nr_inodes=4071068,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 tmpfs /sys/fs/cgroup tmpfs ro,seclabel,nosuid,nodev,noexec,mode=755 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,seclabel,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,seclabel,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,seclabel,nosuid,nodev,noexec,relatime,freezer 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,seclabel,nosuid,nodev,noexec,relatime,perf_event 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,seclabel,nosuid,nodev,noexec,relatime,hugetlb 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,seclabel,nosuid,nodev,noexec,relatime,devices 0 0 cgroup /sys/fs/cgroup/rdma cgroup rw,seclabel,nosuid,nodev,noexec,relatime,rdma 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,seclabel,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,blkio 0 0 none /sys/kernel/tracing tracefs rw,seclabel,relatime 0 0 configfs /sys/kernel/config configfs rw,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=35,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=20116 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,relatime 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3260552k,mode=700,uid=1000,gid=1265 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw,seclabel 0 0 2023-11-24T08:00:25.745 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T08:00:25.746 DEBUG:teuthology.orchestra.run.smithi059:> ip netns list 2023-11-24T08:00:25.763 INFO:teuthology.orchestra.run.smithi059.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2023-11-24T08:00:25.763 INFO:teuthology.orchestra.run.smithi059.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-11-24T08:00:25.764 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T08:00:25.764 DEBUG:teuthology.orchestra.run.smithi059:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.2 2023-11-24T08:00:25.840 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T08:00:25.840 DEBUG:teuthology.orchestra.run.smithi059:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.0 2023-11-24T08:00:25.932 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T08:00:25.933 DEBUG:teuthology.orchestra.run.smithi059:> sudo ip link delete ceph-brx 2023-11-24T08:00:26.029 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T08:00:26.029 DEBUG:teuthology.orchestra.run.smithi059:> ip netns list 2023-11-24T08:00:26.046 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T08:00:26.047 DEBUG:teuthology.orchestra.run.smithi059:> sudo ip link delete ceph-brx 2023-11-24T08:00:26.117 INFO:teuthology.orchestra.run.smithi059.stderr:Cannot find device "ceph-brx" 2023-11-24T08:00:26.118 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-11-24T08:00:26.119 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T08:00:26.119 DEBUG:teuthology.orchestra.run.smithi076:> ip netns list 2023-11-24T08:00:26.139 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-ns--home-ubuntu-cephtest-mnt.3 (id: 1) 2023-11-24T08:00:26.140 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2023-11-24T08:00:26.140 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T08:00:26.140 DEBUG:teuthology.orchestra.run.smithi076:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.3 2023-11-24T08:00:26.225 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T08:00:26.225 DEBUG:teuthology.orchestra.run.smithi076:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.1 2023-11-24T08:00:26.316 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T08:00:26.316 DEBUG:teuthology.orchestra.run.smithi076:> sudo ip link delete ceph-brx 2023-11-24T08:00:26.411 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T08:00:26.412 DEBUG:teuthology.orchestra.run.smithi076:> ip netns list 2023-11-24T08:00:26.431 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T08:00:26.431 DEBUG:teuthology.orchestra.run.smithi076:> sudo ip link delete ceph-brx 2023-11-24T08:00:26.502 INFO:teuthology.orchestra.run.smithi076.stderr:Cannot find device "ceph-brx" 2023-11-24T08:00:26.503 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-11-24T08:00:26.503 DEBUG:teuthology.run_tasks:Unwinding manager ceph 2023-11-24T08:00:26.513 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2023-11-24T08:00:26.514 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-11-24T08:00:26.645 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:26.643+0000 7f8c96440700 1 Processor -- start 2023-11-24T08:00:26.646 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:26.643+0000 7f8c96440700 1 -- start start 2023-11-24T08:00:26.651 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:26.644+0000 7f8c96440700 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f8c9006cae0 0x7f8c900cb2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:00:26.652 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:26.644+0000 7f8c96440700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f8c9006e480 0x7f8c9006e910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:00:26.652 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:26.644+0000 7f8c96440700 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8c901449d0 con 0x7f8c901445f0 2023-11-24T08:00:26.652 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:26.644+0000 7f8c96440700 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8c90072e00 con 0x7f8c9006e480 2023-11-24T08:00:26.652 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:26.644+0000 7f8c96440700 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8c9006ee50 con 0x7f8c9006cae0 2023-11-24T08:00:26.652 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:26.644+0000 7f8c949dd700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f8c9006e480 0x7f8c9006e910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:00:26.652 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:26.644+0000 7f8c949dd700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f8c9006e480 0x7f8c9006e910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:56522/0 (socket says 172.21.15.59:56522) 2023-11-24T08:00:26.652 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:26.644+0000 7f8c949dd700 1 -- 172.21.15.59:0/1792934260 learned_addr learned my addr 172.21.15.59:0/1792934260 (peer_addr_for_me v2:172.21.15.59:0/0) 2023-11-24T08:00:26.653 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:26.644+0000 7f8c8f7fe700 1 --2- 172.21.15.59:0/1792934260 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f8c9006cae0 0x7f8c900cb2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:00:26.653 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:26.644+0000 7f8c949dd700 1 -- 172.21.15.59:0/1792934260 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f8c9006cae0 msgr2=0x7f8c900cb2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:00:26.653 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:26.644+0000 7f8c949dd700 1 --2- 172.21.15.59:0/1792934260 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f8c9006cae0 0x7f8c900cb2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:00:26.653 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:26.644+0000 7f8c949dd700 1 -- 172.21.15.59:0/1792934260 >> v1:172.21.15.76:6789/0 conn(0x7f8c901445f0 legacy=0x7f8c900726f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:00:26.653 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:26.644+0000 7f8c8effd700 1 -- 172.21.15.59:0/1792934260 <== mon.1 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 542817346 0 0) 0x7f8c901449d0 con 0x7f8c901445f0 2023-11-24T08:00:26.653 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:26.644+0000 7f8c949dd700 1 -- 172.21.15.59:0/1792934260 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8c90071650 con 0x7f8c9006e480 2023-11-24T08:00:26.653 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:26.644+0000 7f8c949dd700 1 --2- 172.21.15.59:0/1792934260 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8c9006e480 0x7f8c9006e910 secure :-1 s=READY pgs=1128 cs=0 l=1 rev1=1 crypto rx=0x7f8c88009e50 tx=0x7f8c8800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=849abc258303e6e7 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:00:26.653 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:26.644+0000 7f8c8effd700 1 -- 172.21.15.59:0/1792934260 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c8800e070 con 0x7f8c9006e480 2023-11-24T08:00:26.654 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:26.645+0000 7f8c8effd700 1 -- 172.21.15.59:0/1792934260 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8c8800b910 con 0x7f8c9006e480 2023-11-24T08:00:26.654 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:26.645+0000 7f8c8effd700 1 -- 172.21.15.59:0/1792934260 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c8800bcc0 con 0x7f8c9006e480 2023-11-24T08:00:26.654 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:26.645+0000 7f8c96440700 1 -- 172.21.15.59:0/1792934260 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8c9006e480 msgr2=0x7f8c9006e910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:00:26.654 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:26.645+0000 7f8c96440700 1 --2- 172.21.15.59:0/1792934260 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8c9006e480 0x7f8c9006e910 secure :-1 s=READY pgs=1128 cs=0 l=1 rev1=1 crypto rx=0x7f8c88009e50 tx=0x7f8c8800b330 comp rx=0 tx=0).stop 2023-11-24T08:00:26.654 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:26.645+0000 7f8c96440700 1 -- 172.21.15.59:0/1792934260 shutdown_connections 2023-11-24T08:00:26.654 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:26.645+0000 7f8c96440700 1 --2- 172.21.15.59:0/1792934260 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8c9006e480 0x7f8c9006e910 unknown :-1 s=CLOSED pgs=1128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:00:26.654 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:26.645+0000 7f8c96440700 1 --2- 172.21.15.59:0/1792934260 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f8c9006cae0 0x7f8c900cb2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:00:26.655 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:26.645+0000 7f8c96440700 1 -- 172.21.15.59:0/1792934260 >> 172.21.15.59:0/1792934260 conn(0x7f8c900cf980 msgr2=0x7f8c900cfd60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:00:26.655 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:26.645+0000 7f8c96440700 1 -- 172.21.15.59:0/1792934260 shutdown_connections 2023-11-24T08:00:26.655 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:26.645+0000 7f8c96440700 1 -- 172.21.15.59:0/1792934260 wait complete. 2023-11-24T08:00:26.655 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:26.646+0000 7f8c96440700 1 Processor -- start 2023-11-24T08:00:26.655 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:26.647+0000 7f8c96440700 1 -- start start 2023-11-24T08:00:26.655 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:26.647+0000 7f8c96440700 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f8c9006cae0 0x7f8c900b50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:00:26.655 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:26.647+0000 7f8c96440700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8c9006e480 0x7f8c90157cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:00:26.656 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:26.647+0000 7f8c96440700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8c901445f0 0x7f8c90153e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:00:26.656 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:26.647+0000 7f8c96440700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c900b5620 con 0x7f8c9006e480 2023-11-24T08:00:26.656 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:26.647+0000 7f8c96440700 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c90158720 con 0x7f8c901445f0 2023-11-24T08:00:26.656 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:26.647+0000 7f8c96440700 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8c90158a00 con 0x7f8c9006cae0 2023-11-24T08:00:26.656 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:26.647+0000 7f8c8f7fe700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8c9006e480 0x7f8c90157cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:00:26.656 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:26.647+0000 7f8c8f7fe700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8c9006e480 0x7f8c90157cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:56526/0 (socket says 172.21.15.59:56526) 2023-11-24T08:00:26.656 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:26.647+0000 7f8c8f7fe700 1 -- 172.21.15.59:0/4010927465 learned_addr learned my addr 172.21.15.59:0/4010927465 (peer_addr_for_me v2:172.21.15.59:0/0) 2023-11-24T08:00:26.657 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:26.647+0000 7f8c949dd700 1 --2- 172.21.15.59:0/4010927465 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8c901445f0 0x7f8c90153e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:00:26.657 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:26.647+0000 7f8c8ffff700 1 --2- 172.21.15.59:0/4010927465 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f8c9006cae0 0x7f8c900b50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:00:26.657 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:26.648+0000 7f8c8f7fe700 1 -- 172.21.15.59:0/4010927465 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f8c9006cae0 msgr2=0x7f8c900b50e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:00:26.657 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:26.648+0000 7f8c8f7fe700 1 --2- 172.21.15.59:0/4010927465 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f8c9006cae0 0x7f8c900b50e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:00:26.657 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:26.648+0000 7f8c8f7fe700 1 -- 172.21.15.59:0/4010927465 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8c901445f0 msgr2=0x7f8c90153e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:00:26.657 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:26.648+0000 7f8c8f7fe700 1 --2- 172.21.15.59:0/4010927465 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8c901445f0 0x7f8c90153e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:00:26.657 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:26.648+0000 7f8c8f7fe700 1 -- 172.21.15.59:0/4010927465 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8c88009b30 con 0x7f8c9006e480 2023-11-24T08:00:26.657 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:26.648+0000 7f8c8f7fe700 1 --2- 172.21.15.59:0/4010927465 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8c9006e480 0x7f8c90157cb0 secure :-1 s=READY pgs=1129 cs=0 l=1 rev1=1 crypto rx=0x7f8c7800e870 tx=0x7f8c7800eb80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:00:26.658 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:26.648+0000 7f8c8cff9700 1 -- 172.21.15.59:0/4010927465 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c78009590 con 0x7f8c9006e480 2023-11-24T08:00:26.658 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:26.648+0000 7f8c8cff9700 1 -- 172.21.15.59:0/4010927465 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8c78013680 con 0x7f8c9006e480 2023-11-24T08:00:26.658 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:26.648+0000 7f8c8cff9700 1 -- 172.21.15.59:0/4010927465 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c780139f0 con 0x7f8c9006e480 2023-11-24T08:00:26.658 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:26.648+0000 7f8c96440700 1 -- 172.21.15.59:0/4010927465 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8c90158ce0 con 0x7f8c9006e480 2023-11-24T08:00:26.658 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:26.648+0000 7f8c96440700 1 -- 172.21.15.59:0/4010927465 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8c900b3250 con 0x7f8c9006e480 2023-11-24T08:00:26.658 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:26.649+0000 7f8c96440700 1 -- 172.21.15.59:0/4010927465 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8c90067f60 con 0x7f8c9006e480 2023-11-24T08:00:26.659 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:26.649+0000 7f8c8cff9700 1 -- 172.21.15.59:0/4010927465 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 12) v1 ==== 89792+0+0 (secure 0 0 0) 0x7f8c78013b50 con 0x7f8c9006e480 2023-11-24T08:00:26.659 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:26.650+0000 7f8c8cff9700 1 --2- 172.21.15.59:0/4010927465 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f8c7006b500 0x7f8c7006d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:00:26.659 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:26.650+0000 7f8c8cff9700 1 -- 172.21.15.59:0/4010927465 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f8c78089c70 con 0x7f8c9006e480 2023-11-24T08:00:26.659 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:26.650+0000 7f8c8ffff700 1 --2- 172.21.15.59:0/4010927465 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f8c7006b500 0x7f8c7006d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:00:26.660 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:26.650+0000 7f8c8ffff700 1 --2- 172.21.15.59:0/4010927465 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f8c7006b500 0x7f8c7006d9c0 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7f8c80007fd0 tx=0x7f8c80009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:00:26.660 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:26.654+0000 7f8c8cff9700 1 -- 172.21.15.59:0/4010927465 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f8c780546e0 con 0x7f8c9006e480 2023-11-24T08:00:26.853 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:26.851+0000 7f8c96440700 1 -- 172.21.15.59:0/4010927465 --> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f8c90071ef0 con 0x7f8c7006b500 2023-11-24T08:00:26.855 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:26.854+0000 7f8c8cff9700 1 -- 172.21.15.59:0/4010927465 <== mgr.4100 v2:172.21.15.76:6832/4264046536 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+50856 (secure 0 0 0) 0x7f8c800015b0 con 0x7f8c7006b500 2023-11-24T08:00:26.856 INFO:teuthology.orchestra.run.smithi059.stdout: 2023-11-24T08:00:26.858 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:26.856+0000 7f8c96440700 1 -- 172.21.15.59:0/4010927465 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f8c7006b500 msgr2=0x7f8c7006d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:00:26.858 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:26.856+0000 7f8c96440700 1 --2- 172.21.15.59:0/4010927465 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f8c7006b500 0x7f8c7006d9c0 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7f8c80007fd0 tx=0x7f8c80009040 comp rx=0 tx=0).stop 2023-11-24T08:00:26.858 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:26.856+0000 7f8c96440700 1 -- 172.21.15.59:0/4010927465 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8c9006e480 msgr2=0x7f8c90157cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:00:26.858 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:26.856+0000 7f8c96440700 1 --2- 172.21.15.59:0/4010927465 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8c9006e480 0x7f8c90157cb0 secure :-1 s=READY pgs=1129 cs=0 l=1 rev1=1 crypto rx=0x7f8c7800e870 tx=0x7f8c7800eb80 comp rx=0 tx=0).stop 2023-11-24T08:00:26.858 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:26.856+0000 7f8c96440700 1 -- 172.21.15.59:0/4010927465 shutdown_connections 2023-11-24T08:00:26.858 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:26.856+0000 7f8c96440700 1 --2- 172.21.15.59:0/4010927465 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f8c7006b500 0x7f8c7006d9c0 unknown :-1 s=CLOSED pgs=516 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:00:26.859 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:26.856+0000 7f8c96440700 1 --2- 172.21.15.59:0/4010927465 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8c901445f0 0x7f8c90153e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:00:26.859 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:26.856+0000 7f8c96440700 1 --2- 172.21.15.59:0/4010927465 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f8c9006cae0 0x7f8c900b50e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:00:26.859 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:26.856+0000 7f8c96440700 1 --2- 172.21.15.59:0/4010927465 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8c9006e480 0x7f8c90157cb0 unknown :-1 s=CLOSED pgs=1129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:00:26.859 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:26.856+0000 7f8c96440700 1 -- 172.21.15.59:0/4010927465 >> 172.21.15.59:0/4010927465 conn(0x7f8c900cf980 msgr2=0x7f8c900cb080 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:00:26.859 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:26.856+0000 7f8c96440700 1 -- 172.21.15.59:0/4010927465 shutdown_connections 2023-11-24T08:00:26.859 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:26.856+0000 7f8c96440700 1 -- 172.21.15.59:0/4010927465 wait complete. 2023-11-24T08:00:26.860 INFO:teuthology.orchestra.run.smithi059.stderr:dumped all 2023-11-24T08:00:26.870 INFO:teuthology.orchestra.run.smithi059.stdout:{"pg_ready":true,"pg_map":{"version":207,"stamp":"2023-11-24T08:00:26.355476+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":230048,"kb_used_data":9728,"kb_used_omap":12,"kb_used_meta":220211,"kb_avail":754744672,"statfs":{"total":773094113280,"available":772858544128,"internally_reserved":0,"allocated":9961472,"data_stored":4958984,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12745,"internal_metadata":225496631},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-28340,"num_objects":-23,"num_object_clones":0,"num_object_copies":-46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-23,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":-61,"num_write_kb":-49,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001376"},"pg_stats":[{"pgid":"1.0","version":"16'117","reported_seq":224,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-11-24T08:00:21.312012+0000","last_change":"2023-11-24T07:56:19.668905+0000","last_active":"2023-11-24T08:00:21.312012+0000","last_peered":"2023-11-24T08:00:21.312012+0000","last_clean":"2023-11-24T08:00:21.312012+0000","last_became_active":"2023-11-24T07:56:19.668601+0000","last_became_peered":"2023-11-24T07:56:19.668601+0000","last_unstale":"2023-11-24T08:00:21.312012+0000","last_undegraded":"2023-11-24T08:00:21.312012+0000","last_fullsized":"2023-11-24T08:00:21.312012+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:56:18.657844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:56:18.657844+0000","last_clean_scrub_stamp":"2023-11-24T07:56:18.657844+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T11:51:53.828455+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574900,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28084,"kb_used_data":1064,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343756,"statfs":{"total":96636764160,"available":96608006144,"internally_reserved":0,"allocated":1089536,"data_stored":465400,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 08:00:21 2023","interfaces":[{"interface":"back","average":{"1min":0.66700000000000004,"5min":0.56200000000000006,"15min":0.55800000000000005},"min":{"1min":0.27700000000000002,"5min":0.27600000000000002,"15min":0.27600000000000002},"max":{"1min":3.1640000000000001,"5min":3.1640000000000001,"15min":3.1640000000000001},"last":0.51300000000000001},{"interface":"front","average":{"1min":0.51800000000000002,"5min":0.54400000000000004,"15min":0.55000000000000004},"min":{"1min":0.35499999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.67200000000000004,"5min":0.81000000000000005,"15min":0.81000000000000005},"last":0.40200000000000002}]},{"osd":1,"last update":"Fri Nov 24 08:00:21 2023","interfaces":[{"interface":"back","average":{"1min":0.50700000000000001,"5min":0.64400000000000002,"15min":0.78000000000000003},"min":{"1min":0.28799999999999998,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":0.83499999999999996,"5min":6.1070000000000002,"15min":6.1070000000000002},"last":0.58999999999999997},{"interface":"front","average":{"1min":0.68899999999999995,"5min":0.56899999999999995,"15min":0.57599999999999996},"min":{"1min":0.38400000000000001,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":3.1970000000000001,"5min":3.1970000000000001,"15min":3.1970000000000001},"last":0.55800000000000005}]},{"osd":2,"last update":"Fri Nov 24 08:00:21 2023","interfaces":[{"interface":"back","average":{"1min":0.53500000000000003,"5min":0.56200000000000006,"15min":0.60499999999999998},"min":{"1min":0.35599999999999998,"5min":0.308,"15min":0.308},"max":{"1min":0.94099999999999995,"5min":1.613,"15min":1.613},"last":0.61399999999999999},{"interface":"front","average":{"1min":0.51800000000000002,"5min":0.61699999999999999,"15min":0.73299999999999998},"min":{"1min":0.35199999999999998,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.65400000000000003,"5min":4.5220000000000002,"15min":4.5220000000000002},"last":0.45700000000000002}]},{"osd":3,"last update":"Fri Nov 24 08:00:21 2023","interfaces":[{"interface":"back","average":{"1min":0.56399999999999995,"5min":0.65800000000000003,"15min":0.75600000000000001},"min":{"1min":0.42599999999999999,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.88500000000000001,"5min":4.6760000000000002,"15min":4.6760000000000002},"last":0.54000000000000004},{"interface":"front","average":{"1min":0.64500000000000002,"5min":0.748,"15min":0.878},"min":{"1min":0.51200000000000001,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":1.1299999999999999,"5min":4.6539999999999999,"15min":4.6539999999999999},"last":0.68700000000000006}]},{"osd":4,"last update":"Fri Nov 24 08:00:21 2023","interfaces":[{"interface":"back","average":{"1min":0.53300000000000003,"5min":0.53800000000000003,"15min":0.55800000000000005},"min":{"1min":0.33600000000000002,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":1.0760000000000001,"5min":1.0760000000000001,"15min":1.0760000000000001},"last":0.42599999999999999},{"interface":"front","average":{"1min":0.59299999999999997,"5min":0.60499999999999998,"15min":0.64600000000000002},"min":{"1min":0.39900000000000002,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":1.284,"5min":1.629,"15min":1.629},"last":0.71999999999999997}]},{"osd":5,"last update":"Fri Nov 24 08:00:21 2023","interfaces":[{"interface":"back","average":{"1min":0.51100000000000001,"5min":0.52900000000000003,"15min":0.54100000000000004},"min":{"1min":0.36399999999999999,"5min":0.315,"15min":0.315},"max":{"1min":0.77200000000000002,"5min":0.875,"15min":0.875},"last":0.49299999999999999},{"interface":"front","average":{"1min":0.626,"5min":0.60299999999999998,"15min":0.61599999999999999},"min":{"1min":0.47499999999999998,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":1.0429999999999999,"5min":1.0429999999999999,"15min":1.0429999999999999},"last":0.66700000000000004}]},{"osd":6,"last update":"Fri Nov 24 08:00:21 2023","interfaces":[{"interface":"back","average":{"1min":0.55000000000000004,"5min":0.58399999999999996,"15min":0.60199999999999998},"min":{"1min":0.38600000000000001,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.745,"5min":0.85799999999999998,"15min":0.85799999999999998},"last":0.64600000000000002},{"interface":"front","average":{"1min":0.61199999999999999,"5min":0.64000000000000001,"15min":0.66000000000000003},"min":{"1min":0.438,"5min":0.438,"15min":0.438},"max":{"1min":0.91700000000000004,"5min":0.99099999999999999,"15min":0.99099999999999999},"last":0.47499999999999998}]}]},{"osd":6,"up_from":13,"seq":55834574901,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28084,"kb_used_data":1064,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343756,"statfs":{"total":96636764160,"available":96608006144,"internally_reserved":0,"allocated":1089536,"data_stored":465868,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 07:59:19 2023","interfaces":[{"interface":"back","average":{"1min":0.51000000000000001,"5min":0.53800000000000003,"15min":0.54600000000000004},"min":{"1min":0.33500000000000002,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":0.82999999999999996,"5min":0.93100000000000005,"15min":0.93100000000000005},"last":0.55900000000000005},{"interface":"front","average":{"1min":0.53100000000000003,"5min":0.51900000000000002,"15min":0.51700000000000002},"min":{"1min":0.34000000000000002,"5min":0.311,"15min":0.311},"max":{"1min":0.72299999999999998,"5min":0.82199999999999995,"15min":0.82199999999999995},"last":0.45800000000000002}]},{"osd":1,"last update":"Fri Nov 24 07:59:19 2023","interfaces":[{"interface":"back","average":{"1min":0.499,"5min":0.51000000000000001,"15min":0.51100000000000001},"min":{"1min":0.312,"5min":0.312,"15min":0.312},"max":{"1min":0.746,"5min":0.81299999999999994,"15min":0.81299999999999994},"last":0.47099999999999997},{"interface":"front","average":{"1min":0.55900000000000005,"5min":0.58699999999999997,"15min":0.60399999999999998},"min":{"1min":0.29999999999999999,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":0.70799999999999996,"5min":1.0069999999999999,"15min":1.0069999999999999},"last":0.66200000000000003}]},{"osd":2,"last update":"Fri Nov 24 07:59:19 2023","interfaces":[{"interface":"back","average":{"1min":0.57699999999999996,"5min":0.57699999999999996,"15min":0.57499999999999996},"min":{"1min":0.41399999999999998,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.71099999999999997,"5min":0.90600000000000003,"15min":0.90600000000000003},"last":0.52000000000000002},{"interface":"front","average":{"1min":0.59899999999999998,"5min":0.58799999999999997,"15min":0.59099999999999997},"min":{"1min":0.52500000000000002,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.78800000000000003,"5min":0.84899999999999998,"15min":0.84899999999999998},"last":0.63400000000000001}]},{"osd":3,"last update":"Fri Nov 24 07:59:19 2023","interfaces":[{"interface":"back","average":{"1min":0.56200000000000006,"5min":0.56499999999999995,"15min":0.56699999999999995},"min":{"1min":0.38700000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.75700000000000001,"5min":0.876,"15min":0.876},"last":0.40999999999999998},{"interface":"front","average":{"1min":0.55700000000000005,"5min":0.56000000000000005,"15min":0.55700000000000005},"min":{"1min":0.40500000000000003,"5min":0.379,"15min":0.379},"max":{"1min":0.72599999999999998,"5min":0.876,"15min":0.876},"last":0.622}]},{"osd":4,"last update":"Fri Nov 24 07:59:19 2023","interfaces":[{"interface":"back","average":{"1min":0.55300000000000005,"5min":0.52700000000000002,"15min":0.52000000000000002},"min":{"1min":0.36299999999999999,"5min":0.25,"15min":0.25},"max":{"1min":0.80700000000000005,"5min":0.89400000000000002,"15min":0.89400000000000002},"last":0.505},{"interface":"front","average":{"1min":0.53100000000000003,"5min":0.58499999999999996,"15min":0.59599999999999997},"min":{"1min":0.30599999999999999,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":0.81899999999999995,"5min":1.087,"15min":1.087},"last":0.60399999999999998}]},{"osd":5,"last update":"Fri Nov 24 07:59:19 2023","interfaces":[{"interface":"back","average":{"1min":0.57099999999999995,"5min":0.55100000000000005,"15min":0.53700000000000003},"min":{"1min":0.33300000000000002,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":0.76200000000000001,"5min":0.97999999999999998,"15min":0.97999999999999998},"last":0.58299999999999996},{"interface":"front","average":{"1min":0.62,"5min":0.61399999999999999,"15min":0.61199999999999999},"min":{"1min":0.438,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.997,"5min":1.0009999999999999,"15min":1.0009999999999999},"last":0.434}]},{"osd":7,"last update":"Fri Nov 24 07:59:19 2023","interfaces":[{"interface":"back","average":{"1min":0.59899999999999998,"5min":0.61399999999999999,"15min":0.61599999999999999},"min":{"1min":0.44800000000000001,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":0.85499999999999998,"5min":1.0880000000000001,"15min":1.0880000000000001},"last":0.53600000000000003},{"interface":"front","average":{"1min":0.628,"5min":0.65800000000000003,"15min":0.66600000000000004},"min":{"1min":0.32800000000000001,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.79500000000000004,"5min":0.95599999999999996,"15min":0.95599999999999996},"last":0.65100000000000002}]}]},{"osd":1,"up_from":13,"seq":55834574902,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28660,"kb_used_data":1640,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343180,"statfs":{"total":96636764160,"available":96607416320,"internally_reserved":0,"allocated":1679360,"data_stored":1055654,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 07:59:24 2023","interfaces":[{"interface":"back","average":{"1min":0.35499999999999998,"5min":0.35499999999999998,"15min":0.36099999999999999},"min":{"1min":0.248,"5min":0.20200000000000001,"15min":0.20200000000000001},"max":{"1min":0.52600000000000002,"5min":0.70899999999999996,"15min":0.70899999999999996},"last":0.313},{"interface":"front","average":{"1min":0.36299999999999999,"5min":0.38400000000000001,"15min":0.39300000000000002},"min":{"1min":0.23300000000000001,"5min":0.214,"15min":0.214},"max":{"1min":0.65600000000000003,"5min":0.82499999999999996,"15min":0.82499999999999996},"last":0.33600000000000002}]},{"osd":2,"last update":"Fri Nov 24 07:59:24 2023","interfaces":[{"interface":"back","average":{"1min":0.36899999999999999,"5min":0.35999999999999999,"15min":0.36799999999999999},"min":{"1min":0.27100000000000002,"5min":0.22700000000000001,"15min":0.22700000000000001},"max":{"1min":0.53000000000000003,"5min":0.60999999999999999,"15min":0.60999999999999999},"last":0.26600000000000001},{"interface":"front","average":{"1min":0.42299999999999999,"5min":0.40500000000000003,"15min":0.40999999999999998},"min":{"1min":0.28899999999999998,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":0.63700000000000001,"5min":0.63700000000000001,"15min":0.63700000000000001},"last":0.376}]},{"osd":3,"last update":"Fri Nov 24 07:59:24 2023","interfaces":[{"interface":"back","average":{"1min":0.43099999999999999,"5min":0.433,"15min":0.44},"min":{"1min":0.317,"5min":0.317,"15min":0.317},"max":{"1min":0.62,"5min":0.66500000000000004,"15min":0.66500000000000004},"last":0.35199999999999998},{"interface":"front","average":{"1min":0.41799999999999998,"5min":0.41699999999999998,"15min":0.42399999999999999},"min":{"1min":0.32900000000000001,"5min":0.28499999999999998,"15min":0.28499999999999998},"max":{"1min":0.59999999999999998,"5min":0.76300000000000001,"15min":0.76300000000000001},"last":0.29499999999999998}]},{"osd":4,"last update":"Fri Nov 24 07:59:24 2023","interfaces":[{"interface":"back","average":{"1min":0.625,"5min":0.60399999999999998,"15min":0.60799999999999998},"min":{"1min":0.434,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":1.5580000000000001,"5min":1.5580000000000001,"15min":1.5580000000000001},"last":0.54100000000000004},{"interface":"front","average":{"1min":0.65200000000000002,"5min":0.61599999999999999,"15min":0.60899999999999999},"min":{"1min":0.48599999999999999,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":0.82399999999999995,"5min":0.82399999999999995,"15min":0.82399999999999995},"last":0.66200000000000003}]},{"osd":5,"last update":"Fri Nov 24 07:59:24 2023","interfaces":[{"interface":"back","average":{"1min":0.625,"5min":0.67800000000000005,"15min":0.70899999999999996},"min":{"1min":0.44400000000000001,"5min":0.37,"15min":0.37},"max":{"1min":0.755,"5min":2.7269999999999999,"15min":2.7269999999999999},"last":0.60699999999999998},{"interface":"front","average":{"1min":0.68500000000000005,"5min":0.63900000000000001,"15min":0.64300000000000002},"min":{"1min":0.434,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":1.6459999999999999,"5min":1.6459999999999999,"15min":1.6459999999999999},"last":0.65100000000000002}]},{"osd":6,"last update":"Fri Nov 24 07:59:24 2023","interfaces":[{"interface":"back","average":{"1min":0.63200000000000001,"5min":0.64300000000000002,"15min":0.65300000000000002},"min":{"1min":0.46000000000000002,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":0.93799999999999994,"5min":0.98199999999999998,"15min":0.98199999999999998},"last":0.56000000000000005},{"interface":"front","average":{"1min":0.67900000000000005,"5min":0.621,"15min":0.61799999999999999},"min":{"1min":0.46100000000000002,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":0.91600000000000004,"5min":0.91600000000000004,"15min":0.91600000000000004},"last":0.67200000000000004}]},{"osd":7,"last update":"Fri Nov 24 07:59:24 2023","interfaces":[{"interface":"back","average":{"1min":0.69699999999999995,"5min":0.64700000000000002,"15min":0.64000000000000001},"min":{"1min":0.53900000000000003,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.95599999999999996,"5min":1.0649999999999999,"15min":1.0649999999999999},"last":0.57699999999999996},{"interface":"front","average":{"1min":0.64300000000000002,"5min":0.64400000000000002,"15min":0.66100000000000003},"min":{"1min":0.45900000000000002,"5min":0.442,"15min":0.442},"max":{"1min":0.93300000000000005,"5min":0.93300000000000005,"15min":0.93300000000000005},"last":0.68600000000000005}]}]},{"osd":0,"up_from":13,"seq":55834574902,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28116,"kb_used_data":1096,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343724,"statfs":{"total":96636764160,"available":96607973376,"internally_reserved":0,"allocated":1122304,"data_stored":492626,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Nov 24 07:59:22 2023","interfaces":[{"interface":"back","average":{"1min":0.36899999999999999,"5min":0.57799999999999996,"15min":0.66800000000000004},"min":{"1min":0.247,"5min":0.21299999999999999,"15min":0.21299999999999999},"max":{"1min":0.56100000000000005,"5min":6.3550000000000004,"15min":6.3550000000000004},"last":0.377},{"interface":"front","average":{"1min":0.39300000000000002,"5min":1.1970000000000001,"15min":1.5509999999999999},"min":{"1min":0.29399999999999998,"5min":0.26000000000000001,"15min":0.26000000000000001},"max":{"1min":0.59099999999999997,"5min":22.916,"15min":22.916},"last":0.40100000000000002}]},{"osd":2,"last update":"Fri Nov 24 07:59:22 2023","interfaces":[{"interface":"back","average":{"1min":0.35999999999999999,"5min":0.56200000000000006,"15min":0.65400000000000003},"min":{"1min":0.23400000000000001,"5min":0.22800000000000001,"15min":0.22800000000000001},"max":{"1min":0.54100000000000004,"5min":6.3899999999999997,"15min":6.3899999999999997},"last":0.29599999999999999},{"interface":"front","average":{"1min":0.32300000000000001,"5min":0.78100000000000003,"15min":0.96899999999999997},"min":{"1min":0.253,"5min":0.253,"15min":0.253},"max":{"1min":0.497,"5min":11.93,"15min":11.93},"last":0.35599999999999998}]},{"osd":3,"last update":"Fri Nov 24 07:59:22 2023","interfaces":[{"interface":"back","average":{"1min":0.373,"5min":0.40100000000000002,"15min":0.41299999999999998},"min":{"1min":0.247,"5min":0.23799999999999999,"15min":0.23799999999999999},"max":{"1min":0.52500000000000002,"5min":1.155,"15min":1.155},"last":0.317},{"interface":"front","average":{"1min":0.40200000000000002,"5min":0.84399999999999997,"15min":1.032},"min":{"1min":0.27400000000000002,"5min":0.27400000000000002,"15min":0.27400000000000002},"max":{"1min":0.63400000000000001,"5min":12.116,"15min":12.116},"last":0.41599999999999998}]},{"osd":4,"last update":"Fri Nov 24 07:59:22 2023","interfaces":[{"interface":"back","average":{"1min":0.59799999999999998,"5min":0.63300000000000001,"15min":0.64300000000000002},"min":{"1min":0.44400000000000001,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.93500000000000005,"5min":1.044,"15min":1.044},"last":0.63700000000000001},{"interface":"front","average":{"1min":0.58499999999999996,"5min":1.4059999999999999,"15min":1.758},"min":{"1min":0.377,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.84999999999999998,"5min":22.963999999999999,"15min":22.963999999999999},"last":0.55200000000000005}]},{"osd":5,"last update":"Fri Nov 24 07:59:22 2023","interfaces":[{"interface":"back","average":{"1min":0.59999999999999998,"5min":1.026,"15min":1.204},"min":{"1min":0.41199999999999998,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.86299999999999999,"5min":12.084,"15min":12.084},"last":0.66100000000000003},{"interface":"front","average":{"1min":0.61899999999999999,"5min":0.66200000000000003,"15min":0.67200000000000004},"min":{"1min":0.46600000000000003,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.89100000000000001,"5min":1.073,"15min":1.073},"last":0.56699999999999995}]},{"osd":6,"last update":"Fri Nov 24 07:59:22 2023","interfaces":[{"interface":"back","average":{"1min":0.61599999999999999,"5min":0.66800000000000004,"15min":0.68300000000000005},"min":{"1min":0.36799999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.92400000000000004,"5min":1.0580000000000001,"15min":1.0580000000000001},"last":0.53900000000000003},{"interface":"front","average":{"1min":0.60099999999999998,"5min":1.042,"15min":1.2230000000000001},"min":{"1min":0.46000000000000002,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":0.81999999999999995,"5min":12.045,"15min":12.045},"last":0.62}]},{"osd":7,"last update":"Fri Nov 24 07:59:22 2023","interfaces":[{"interface":"back","average":{"1min":0.57699999999999996,"5min":0.64300000000000002,"15min":0.65100000000000002},"min":{"1min":0.45700000000000002,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":0.82999999999999996,"5min":1.038,"15min":1.038},"last":0.60899999999999999},{"interface":"front","average":{"1min":0.59499999999999997,"5min":1.431,"15min":1.7849999999999999},"min":{"1min":0.42299999999999999,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":0.80300000000000005,"5min":22.995000000000001,"15min":22.995000000000001},"last":0.67300000000000004}]}]},{"osd":2,"up_from":13,"seq":55834574902,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28080,"kb_used_data":1060,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343760,"statfs":{"total":96636764160,"available":96608010240,"internally_reserved":0,"allocated":1085440,"data_stored":465330,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 08:00:21 2023","interfaces":[{"interface":"back","average":{"1min":0.34499999999999997,"5min":0.35399999999999998,"15min":0.36499999999999999},"min":{"1min":0.22500000000000001,"5min":0.219,"15min":0.219},"max":{"1min":0.499,"5min":0.58099999999999996,"15min":0.58099999999999996},"last":0.47199999999999998},{"interface":"front","average":{"1min":0.41299999999999998,"5min":0.39600000000000002,"15min":0.40000000000000002},"min":{"1min":0.28899999999999998,"5min":0.27400000000000002,"15min":0.27400000000000002},"max":{"1min":0.69099999999999995,"5min":0.69099999999999995,"15min":0.69099999999999995},"last":0.442}]},{"osd":1,"last update":"Fri Nov 24 08:00:21 2023","interfaces":[{"interface":"back","average":{"1min":0.34999999999999998,"5min":0.35499999999999998,"15min":0.37},"min":{"1min":0.249,"5min":0.21099999999999999,"15min":0.21099999999999999},"max":{"1min":0.55700000000000005,"5min":0.59199999999999997,"15min":0.59199999999999997},"last":0.40100000000000002},{"interface":"front","average":{"1min":0.379,"5min":0.38800000000000001,"15min":0.40600000000000003},"min":{"1min":0.24399999999999999,"5min":0.24399999999999999,"15min":0.24399999999999999},"max":{"1min":0.55100000000000005,"5min":0.627,"15min":0.627},"last":0.54700000000000004}]},{"osd":3,"last update":"Fri Nov 24 08:00:21 2023","interfaces":[{"interface":"back","average":{"1min":0.42499999999999999,"5min":0.40400000000000003,"15min":0.40799999999999997},"min":{"1min":0.28699999999999998,"5min":0.26400000000000001,"15min":0.26400000000000001},"max":{"1min":0.63500000000000001,"5min":0.64200000000000002,"15min":0.64200000000000002},"last":0.499},{"interface":"front","average":{"1min":0.39000000000000001,"5min":0.40000000000000002,"15min":0.40899999999999997},"min":{"1min":0.315,"5min":0.27500000000000002,"15min":0.27500000000000002},"max":{"1min":0.58299999999999996,"5min":0.64600000000000002,"15min":0.64600000000000002},"last":0.33100000000000002}]},{"osd":4,"last update":"Fri Nov 24 08:00:21 2023","interfaces":[{"interface":"back","average":{"1min":0.58699999999999997,"5min":0.57099999999999995,"15min":0.59499999999999997},"min":{"1min":0.32900000000000001,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.91000000000000003,"5min":0.91000000000000003,"15min":0.91000000000000003},"last":0.91000000000000003},{"interface":"front","average":{"1min":0.57099999999999995,"5min":0.57699999999999996,"15min":0.59199999999999997},"min":{"1min":0.35299999999999998,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.75600000000000001,"5min":0.83799999999999997,"15min":0.83799999999999997},"last":0.65200000000000002}]},{"osd":5,"last update":"Fri Nov 24 08:00:21 2023","interfaces":[{"interface":"back","average":{"1min":0.57899999999999996,"5min":0.58999999999999997,"15min":0.60999999999999999},"min":{"1min":0.39100000000000001,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.84099999999999997,"5min":0.84899999999999998,"15min":0.84899999999999998},"last":0.68100000000000005},{"interface":"front","average":{"1min":0.64500000000000002,"5min":0.64600000000000002,"15min":0.69199999999999995},"min":{"1min":0.45100000000000001,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":1.119,"5min":2.1699999999999999,"15min":2.1699999999999999},"last":0.83699999999999997}]},{"osd":6,"last update":"Fri Nov 24 08:00:21 2023","interfaces":[{"interface":"back","average":{"1min":0.64000000000000001,"5min":0.625,"15min":0.65300000000000002},"min":{"1min":0.41999999999999998,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":0.88300000000000001,"5min":0.88300000000000001,"15min":0.88300000000000001},"last":0.88300000000000001},{"interface":"front","average":{"1min":0.65400000000000003,"5min":0.629,"15min":0.65700000000000003},"min":{"1min":0.44500000000000001,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":1.105,"5min":1.105,"15min":1.105},"last":0.80600000000000005}]},{"osd":7,"last update":"Fri Nov 24 08:00:21 2023","interfaces":[{"interface":"back","average":{"1min":0.628,"5min":0.626,"15min":0.65000000000000002},"min":{"1min":0.42199999999999999,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":0.93000000000000005,"5min":0.93000000000000005,"15min":0.93000000000000005},"last":0.70899999999999996},{"interface":"front","average":{"1min":0.627,"5min":0.63600000000000001,"15min":0.66500000000000004},"min":{"1min":0.46200000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.80000000000000004,"5min":0.90000000000000002,"15min":0.90000000000000002},"last":0.77400000000000002}]}]},{"osd":3,"up_from":13,"seq":55834574902,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28088,"kb_used_data":1068,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343752,"statfs":{"total":96636764160,"available":96608002048,"internally_reserved":0,"allocated":1093632,"data_stored":465870,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 07:59:27 2023","interfaces":[{"interface":"back","average":{"1min":0.33600000000000002,"5min":0.34899999999999998,"15min":0.35299999999999998},"min":{"1min":0.183,"5min":0.183,"15min":0.183},"max":{"1min":0.53400000000000003,"5min":0.84799999999999998,"15min":0.84799999999999998},"last":0.30299999999999999},{"interface":"front","average":{"1min":0.34599999999999997,"5min":0.35399999999999998,"15min":0.36199999999999999},"min":{"1min":0.24299999999999999,"5min":0.23699999999999999,"15min":0.23699999999999999},"max":{"1min":0.51000000000000001,"5min":0.873,"15min":0.873},"last":0.32000000000000001}]},{"osd":1,"last update":"Fri Nov 24 07:59:27 2023","interfaces":[{"interface":"back","average":{"1min":0.38400000000000001,"5min":0.373,"15min":0.374},"min":{"1min":0.217,"5min":0.217,"15min":0.217},"max":{"1min":0.66300000000000003,"5min":0.92000000000000004,"15min":0.92000000000000004},"last":0.26600000000000001},{"interface":"front","average":{"1min":0.38,"5min":0.377,"15min":0.38300000000000001},"min":{"1min":0.28599999999999998,"5min":0.248,"15min":0.248},"max":{"1min":0.626,"5min":0.81699999999999995,"15min":0.81699999999999995},"last":0.34899999999999998}]},{"osd":2,"last update":"Fri Nov 24 07:59:27 2023","interfaces":[{"interface":"back","average":{"1min":0.35599999999999998,"5min":0.35399999999999998,"15min":0.35799999999999998},"min":{"1min":0.253,"5min":0.24399999999999999,"15min":0.24399999999999999},"max":{"1min":0.496,"5min":0.88500000000000001,"15min":0.88500000000000001},"last":0.36599999999999999},{"interface":"front","average":{"1min":0.40999999999999998,"5min":0.441,"15min":0.45900000000000002},"min":{"1min":0.315,"5min":0.311,"15min":0.311},"max":{"1min":0.59799999999999998,"5min":0.98099999999999998,"15min":0.98099999999999998},"last":0.33600000000000002}]},{"osd":4,"last update":"Fri Nov 24 07:59:27 2023","interfaces":[{"interface":"back","average":{"1min":0.56299999999999994,"5min":0.70499999999999996,"15min":0.76600000000000001},"min":{"1min":0.38900000000000001,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.73599999999999999,"5min":5.7350000000000003,"15min":5.7350000000000003},"last":0.51000000000000001},{"interface":"front","average":{"1min":0.55700000000000005,"5min":0.58099999999999996,"15min":0.58299999999999996},"min":{"1min":0.34399999999999997,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.80400000000000005,"5min":1.27,"15min":1.27},"last":0.41299999999999998}]},{"osd":5,"last update":"Fri Nov 24 07:59:27 2023","interfaces":[{"interface":"back","average":{"1min":0.56100000000000005,"5min":0.57799999999999996,"15min":0.58399999999999996},"min":{"1min":0.42899999999999999,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":0.67900000000000005,"5min":1.4430000000000001,"15min":1.4430000000000001},"last":0.47999999999999998},{"interface":"front","average":{"1min":0.58799999999999997,"5min":0.61499999999999999,"15min":0.623},"min":{"1min":0.40999999999999998,"5min":0.377,"15min":0.377},"max":{"1min":0.78500000000000003,"5min":1.4239999999999999,"15min":1.4239999999999999},"last":0.434}]},{"osd":6,"last update":"Fri Nov 24 07:59:27 2023","interfaces":[{"interface":"back","average":{"1min":0.58799999999999997,"5min":0.60699999999999998,"15min":0.61499999999999999},"min":{"1min":0.42599999999999999,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":0.84499999999999997,"5min":1.355,"15min":1.355},"last":0.49399999999999999},{"interface":"front","average":{"1min":0.57899999999999996,"5min":0.745,"15min":0.81200000000000006},"min":{"1min":0.44800000000000001,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.77100000000000002,"5min":5.6479999999999997,"15min":5.6479999999999997},"last":0.42399999999999999}]},{"osd":7,"last update":"Fri Nov 24 07:59:27 2023","interfaces":[{"interface":"back","average":{"1min":0.61799999999999999,"5min":0.64100000000000001,"15min":0.65700000000000003},"min":{"1min":0.499,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":0.82799999999999996,"5min":1.51,"15min":1.51},"last":0.53400000000000003},{"interface":"front","average":{"1min":0.57699999999999996,"5min":0.74199999999999999,"15min":0.80800000000000005},"min":{"1min":0.40100000000000002,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.79500000000000004,"5min":5.7069999999999999,"15min":5.7069999999999999},"last":0.44700000000000001}]}]},{"osd":4,"up_from":13,"seq":55834574901,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28080,"kb_used_data":1060,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343760,"statfs":{"total":96636764160,"available":96608010240,"internally_reserved":0,"allocated":1085440,"data_stored":465230,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 07:59:28 2023","interfaces":[{"interface":"back","average":{"1min":0.55900000000000005,"5min":0.56100000000000005,"15min":0.57399999999999995},"min":{"1min":0.32400000000000001,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.93300000000000005,"5min":0.93300000000000005,"15min":0.93300000000000005},"last":0.63700000000000001},{"interface":"front","average":{"1min":0.51100000000000001,"5min":0.52800000000000002,"15min":0.53800000000000003},"min":{"1min":0.36199999999999999,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.69299999999999995,"5min":1.0249999999999999,"15min":1.0249999999999999},"last":0.72099999999999997}]},{"osd":1,"last update":"Fri Nov 24 07:59:28 2023","interfaces":[{"interface":"back","average":{"1min":0.54800000000000004,"5min":0.54600000000000004,"15min":0.55000000000000004},"min":{"1min":0.35899999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.71599999999999997,"5min":0.86399999999999999,"15min":0.86399999999999999},"last":0.68500000000000005},{"interface":"front","average":{"1min":0.51600000000000001,"5min":0.53700000000000003,"15min":0.54600000000000004},"min":{"1min":0.36699999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.72099999999999997,"5min":0.98699999999999999,"15min":0.98699999999999999},"last":0.70299999999999996}]},{"osd":2,"last update":"Fri Nov 24 07:59:28 2023","interfaces":[{"interface":"back","average":{"1min":0.54200000000000004,"5min":0.54900000000000004,"15min":0.55700000000000005},"min":{"1min":0.34499999999999997,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.86399999999999999,"5min":0.86399999999999999,"15min":0.86399999999999999},"last":0.59799999999999998},{"interface":"front","average":{"1min":0.56200000000000006,"5min":0.59399999999999997,"15min":0.60599999999999998},"min":{"1min":0.34000000000000002,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.81899999999999995,"5min":0.89200000000000002,"15min":0.89200000000000002},"last":0.80700000000000005}]},{"osd":3,"last update":"Fri Nov 24 07:59:28 2023","interfaces":[{"interface":"back","average":{"1min":0.53600000000000003,"5min":0.56799999999999995,"15min":0.57899999999999996},"min":{"1min":0.38800000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.78100000000000003,"5min":0.89700000000000002,"15min":0.89700000000000002},"last":0.73799999999999999},{"interface":"front","average":{"1min":0.58899999999999997,"5min":0.61099999999999999,"15min":0.624},"min":{"1min":0.42699999999999999,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.73999999999999999,"5min":0.96199999999999997,"15min":0.96199999999999997},"last":0.83899999999999997}]},{"osd":5,"last update":"Fri Nov 24 07:59:28 2023","interfaces":[{"interface":"back","average":{"1min":0.46800000000000003,"5min":0.46300000000000002,"15min":0.45700000000000002},"min":{"1min":0.34999999999999998,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":0.60499999999999998,"5min":0.70499999999999996,"15min":0.70499999999999996},"last":0.61899999999999999},{"interface":"front","average":{"1min":0.56599999999999995,"5min":0.60399999999999998,"15min":0.61399999999999999},"min":{"1min":0.41599999999999998,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.73699999999999999,"5min":0.86399999999999999,"15min":0.86399999999999999},"last":0.79100000000000004}]},{"osd":6,"last update":"Fri Nov 24 07:59:28 2023","interfaces":[{"interface":"back","average":{"1min":0.53900000000000003,"5min":0.57299999999999995,"15min":0.58099999999999996},"min":{"1min":0.33000000000000002,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.88600000000000001,"5min":0.88600000000000001,"15min":0.88600000000000001},"last":0.56100000000000005},{"interface":"front","average":{"1min":0.56899999999999995,"5min":0.59999999999999998,"15min":0.60899999999999999},"min":{"1min":0.32100000000000001,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.80300000000000005,"5min":1.095,"15min":1.095},"last":0.77200000000000002}]},{"osd":7,"last update":"Fri Nov 24 07:59:28 2023","interfaces":[{"interface":"back","average":{"1min":0.52000000000000002,"5min":0.56100000000000005,"15min":0.56899999999999995},"min":{"1min":0.36299999999999999,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.69999999999999996,"5min":0.89300000000000002,"15min":0.89300000000000002},"last":0.67000000000000004},{"interface":"front","average":{"1min":0.57799999999999996,"5min":0.63600000000000001,"15min":0.65400000000000003},"min":{"1min":0.39000000000000001,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.74399999999999999,"5min":0.91500000000000004,"15min":0.91500000000000004},"last":0.75600000000000001}]}]},{"osd":5,"up_from":13,"seq":55834574902,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32856,"kb_used_data":1676,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338984,"statfs":{"total":96636764160,"available":96603119616,"internally_reserved":0,"allocated":1716224,"data_stored":1083006,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 07:59:21 2023","interfaces":[{"interface":"back","average":{"1min":0.52500000000000002,"5min":0.54600000000000004,"15min":0.55600000000000005},"min":{"1min":0.311,"5min":0.311,"15min":0.311},"max":{"1min":0.74399999999999999,"5min":1.0109999999999999,"15min":1.0109999999999999},"last":0.40200000000000002},{"interface":"front","average":{"1min":0.53600000000000003,"5min":0.69299999999999995,"15min":0.76400000000000001},"min":{"1min":0.32400000000000001,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.81899999999999995,"5min":5.431,"15min":5.431},"last":0.504}]},{"osd":1,"last update":"Fri Nov 24 07:59:21 2023","interfaces":[{"interface":"back","average":{"1min":0.52600000000000002,"5min":0.54200000000000004,"15min":0.55300000000000005},"min":{"1min":0.35399999999999998,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":0.69099999999999995,"5min":1.016,"15min":1.016},"last":0.32400000000000001},{"interface":"front","average":{"1min":0.56399999999999995,"5min":0.56399999999999995,"15min":0.56799999999999995},"min":{"1min":0.34000000000000002,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":0.80100000000000005,"5min":0.96499999999999997,"15min":0.96499999999999997},"last":0.47899999999999998}]},{"osd":2,"last update":"Fri Nov 24 07:59:21 2023","interfaces":[{"interface":"back","average":{"1min":0.58099999999999996,"5min":0.55500000000000005,"15min":0.55200000000000005},"min":{"1min":0.44800000000000001,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.876,"5min":1.163,"15min":1.163},"last":0.35599999999999998},{"interface":"front","average":{"1min":0.55900000000000005,"5min":0.59799999999999998,"15min":0.61899999999999999},"min":{"1min":0.32600000000000001,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.72399999999999998,"5min":1.119,"15min":1.119},"last":0.53900000000000003}]},{"osd":3,"last update":"Fri Nov 24 07:59:21 2023","interfaces":[{"interface":"back","average":{"1min":0.59799999999999998,"5min":0.77700000000000002,"15min":0.86199999999999999},"min":{"1min":0.32400000000000001,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.77100000000000002,"5min":5.609,"15min":5.609},"last":0.52700000000000002},{"interface":"front","average":{"1min":0.64300000000000002,"5min":0.66000000000000003,"15min":0.67000000000000004},"min":{"1min":0.38900000000000001,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.84999999999999998,"5min":1.288,"15min":1.288},"last":0.56699999999999995}]},{"osd":4,"last update":"Fri Nov 24 07:59:21 2023","interfaces":[{"interface":"back","average":{"1min":0.53800000000000003,"5min":0.51600000000000001,"15min":0.51200000000000001},"min":{"1min":0.30599999999999999,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":0.79100000000000004,"5min":0.79100000000000004,"15min":0.79100000000000004},"last":0.45900000000000002},{"interface":"front","average":{"1min":0.58899999999999997,"5min":0.57799999999999996,"15min":0.57399999999999995},"min":{"1min":0.35399999999999998,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":0.82599999999999996,"5min":0.92900000000000005,"15min":0.92900000000000005},"last":0.55400000000000005}]},{"osd":6,"last update":"Fri Nov 24 07:59:21 2023","interfaces":[{"interface":"back","average":{"1min":0.63,"5min":0.58699999999999997,"15min":0.57599999999999996},"min":{"1min":0.434,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.85399999999999998,"5min":0.89700000000000002,"15min":0.89700000000000002},"last":0.627},{"interface":"front","average":{"1min":0.59299999999999997,"5min":0.621,"15min":0.623},"min":{"1min":0.376,"5min":0.376,"15min":0.376},"max":{"1min":0.74299999999999999,"5min":1.083,"15min":1.083},"last":0.59199999999999997}]},{"osd":7,"last update":"Fri Nov 24 07:59:21 2023","interfaces":[{"interface":"back","average":{"1min":0.65400000000000003,"5min":0.65000000000000002,"15min":0.64700000000000002},"min":{"1min":0.40699999999999997,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.86099999999999999,"5min":0.96599999999999997,"15min":0.96599999999999997},"last":0.58199999999999996},{"interface":"front","average":{"1min":0.68000000000000005,"5min":0.66100000000000003,"15min":0.65500000000000003},"min":{"1min":0.49099999999999999,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":0.78500000000000003,"5min":0.94199999999999995,"15min":0.94199999999999995},"last":0.60399999999999998}]}]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-11-24T08:00:26.872 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-11-24T08:00:26.996 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:26.994+0000 7f954c8be700 1 Processor -- start 2023-11-24T08:00:26.996 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:26.994+0000 7f954c8be700 1 -- start start 2023-11-24T08:00:26.998 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:26.995+0000 7f954c8be700 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f954815e1c0 0x7f954806eef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:00:26.999 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:26.995+0000 7f954c8be700 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f95480d1230 con 0x7f95480b0e90 2023-11-24T08:00:26.999 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:26.995+0000 7f954c8be700 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f95480d0c70 con 0x7f954806fc50 2023-11-24T08:00:26.999 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:26.995+0000 7f954c8be700 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f95480d0f50 con 0x7f954815e1c0 2023-11-24T08:00:26.999 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:26.995+0000 7f9545d9b700 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7f95480b0e90 0x7f954806f540 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:60008/0 (socket says 172.21.15.59:60008) 2023-11-24T08:00:26.999 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:26.995+0000 7f9545d9b700 1 -- 172.21.15.59:0/3763368248 learned_addr learned my addr 172.21.15.59:0/3763368248 (peer_addr_for_me v1:172.21.15.59:0/0) 2023-11-24T08:00:26.999 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:26.996+0000 7f954659c700 1 --2- 172.21.15.59:0/3763368248 >> v2:172.21.15.76:3300/0 conn(0x7f954815e1c0 0x7f954806eef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:00:26.999 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:26.996+0000 7f954559a700 1 -- 172.21.15.59:0/3763368248 <== mon.0 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2938950163 0 0) 0x7f95480d1230 con 0x7f95480b0e90 2023-11-24T08:00:27.000 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:26.996+0000 7f954559a700 1 -- 172.21.15.59:0/3763368248 --> v1:172.21.15.59:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f95300035b0 con 0x7f95480b0e90 2023-11-24T08:00:27.000 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:26.996+0000 7f954559a700 1 -- 172.21.15.59:0/3763368248 <== mon.1 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4114407393 0 0) 0x7f95480d0c70 con 0x7f954806fc50 2023-11-24T08:00:27.000 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:26.996+0000 7f954559a700 1 -- 172.21.15.59:0/3763368248 --> v1:172.21.15.76:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f95480d1230 con 0x7f954806fc50 2023-11-24T08:00:27.000 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:26.996+0000 7f954659c700 1 -- 172.21.15.59:0/3763368248 >> v1:172.21.15.76:6789/0 conn(0x7f954806fc50 legacy=0x7f954806ca10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:00:27.000 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:26.996+0000 7f954659c700 1 -- 172.21.15.59:0/3763368248 >> v1:172.21.15.59:6789/0 conn(0x7f95480b0e90 legacy=0x7f954806f540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:00:27.000 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:26.996+0000 7f954659c700 1 -- 172.21.15.59:0/3763368248 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f954806c210 con 0x7f954815e1c0 2023-11-24T08:00:27.000 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:26.996+0000 7f954659c700 1 --2- 172.21.15.59:0/3763368248 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f954815e1c0 0x7f954806eef0 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f9538009d90 tx=0x7f953800b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d44b22d7508fc30b server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:00:27.000 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:26.996+0000 7f954559a700 1 -- 172.21.15.59:0/3763368248 <== mon.1 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f953800e070 con 0x7f954815e1c0 2023-11-24T08:00:27.001 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:26.996+0000 7f954559a700 1 -- 172.21.15.59:0/3763368248 <== mon.1 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f953800bb20 con 0x7f954815e1c0 2023-11-24T08:00:27.001 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:26.996+0000 7f954559a700 1 -- 172.21.15.59:0/3763368248 <== mon.1 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9538012430 con 0x7f954815e1c0 2023-11-24T08:00:27.001 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:26.997+0000 7f954c8be700 1 -- 172.21.15.59:0/3763368248 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f954815e1c0 msgr2=0x7f954806eef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:00:27.001 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:26.997+0000 7f954c8be700 1 --2- 172.21.15.59:0/3763368248 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f954815e1c0 0x7f954806eef0 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f9538009d90 tx=0x7f953800b330 comp rx=0 tx=0).stop 2023-11-24T08:00:27.001 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:26.997+0000 7f954c8be700 1 -- 172.21.15.59:0/3763368248 shutdown_connections 2023-11-24T08:00:27.001 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:26.997+0000 7f954c8be700 1 --2- 172.21.15.59:0/3763368248 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f954815e1c0 0x7f954806eef0 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:00:27.001 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:26.997+0000 7f954c8be700 1 -- 172.21.15.59:0/3763368248 >> 172.21.15.59:0/3763368248 conn(0x7f95480b3720 msgr2=0x7f95480b3b30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:00:27.001 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:26.997+0000 7f954c8be700 1 -- 172.21.15.59:0/3763368248 shutdown_connections 2023-11-24T08:00:27.002 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:26.997+0000 7f954c8be700 1 -- 172.21.15.59:0/3763368248 wait complete. 2023-11-24T08:00:27.004 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:26.998+0000 7f954c8be700 1 Processor -- start 2023-11-24T08:00:27.005 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:26.998+0000 7f954c8be700 1 -- start start 2023-11-24T08:00:27.005 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:26.999+0000 7f954c8be700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f954806fc50 0x7f95481e58d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:00:27.005 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:26.999+0000 7f954c8be700 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f95480b0e90 0x7f954809f540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:00:27.005 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:26.999+0000 7f954c8be700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f954815e1c0 0x7f95481e7420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:00:27.005 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:26.999+0000 7f954c8be700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f954809ba80 con 0x7f954806fc50 2023-11-24T08:00:27.005 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:26.999+0000 7f954c8be700 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9548158e20 con 0x7f954815e1c0 2023-11-24T08:00:27.005 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:26.999+0000 7f954c8be700 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f95481e5eb0 con 0x7f95480b0e90 2023-11-24T08:00:27.006 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:26.999+0000 7f954659c700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f954806fc50 0x7f95481e58d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:00:27.006 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:26.999+0000 7f954659c700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f954806fc50 0x7f95481e58d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:56538/0 (socket says 172.21.15.59:56538) 2023-11-24T08:00:27.006 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:26.999+0000 7f954659c700 1 -- 172.21.15.59:0/262561392 learned_addr learned my addr 172.21.15.59:0/262561392 (peer_addr_for_me v2:172.21.15.59:0/0) 2023-11-24T08:00:27.006 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:26.999+0000 7f9545d9b700 1 --2- 172.21.15.59:0/262561392 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f95480b0e90 0x7f954809f540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:00:27.006 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:26.999+0000 7f9546d9d700 1 --2- 172.21.15.59:0/262561392 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f954815e1c0 0x7f95481e7420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:00:27.006 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:26.999+0000 7f954659c700 1 -- 172.21.15.59:0/262561392 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f95480b0e90 msgr2=0x7f954809f540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:00:27.006 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:26.999+0000 7f954659c700 1 --2- 172.21.15.59:0/262561392 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f95480b0e90 0x7f954809f540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:00:27.007 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:26.999+0000 7f954659c700 1 -- 172.21.15.59:0/262561392 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f954815e1c0 msgr2=0x7f95481e7420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:00:27.007 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:26.999+0000 7f954659c700 1 --2- 172.21.15.59:0/262561392 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f954815e1c0 0x7f95481e7420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:00:27.007 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:26.999+0000 7f954659c700 1 -- 172.21.15.59:0/262561392 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9538009a70 con 0x7f954806fc50 2023-11-24T08:00:27.007 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:26.999+0000 7f9545d9b700 1 --2- 172.21.15.59:0/262561392 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f95480b0e90 0x7f954809f540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-24T08:00:27.007 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:27.000+0000 7f954659c700 1 --2- 172.21.15.59:0/262561392 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f954806fc50 0x7f95481e58d0 secure :-1 s=READY pgs=1131 cs=0 l=1 rev1=1 crypto rx=0x7f9548072340 tx=0x7f9538004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:00:27.008 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:27.000+0000 7f9536ffd700 1 -- 172.21.15.59:0/262561392 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f953801b810 con 0x7f954806fc50 2023-11-24T08:00:27.008 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:27.000+0000 7f9536ffd700 1 -- 172.21.15.59:0/262561392 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9538020070 con 0x7f954806fc50 2023-11-24T08:00:27.008 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:27.000+0000 7f954c8be700 1 -- 172.21.15.59:0/262561392 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f95481e6190 con 0x7f954806fc50 2023-11-24T08:00:27.008 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:27.000+0000 7f9536ffd700 1 -- 172.21.15.59:0/262561392 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f953801bbf0 con 0x7f954806fc50 2023-11-24T08:00:27.008 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:27.000+0000 7f954c8be700 1 -- 172.21.15.59:0/262561392 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f95481e6a30 con 0x7f954806fc50 2023-11-24T08:00:27.008 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:27.001+0000 7f954c8be700 1 -- 172.21.15.59:0/262561392 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9548068670 con 0x7f954806fc50 2023-11-24T08:00:27.008 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:27.001+0000 7f9536ffd700 1 -- 172.21.15.59:0/262561392 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 12) v1 ==== 89792+0+0 (secure 0 0 0) 0x7f953801bd50 con 0x7f954806fc50 2023-11-24T08:00:27.008 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:27.002+0000 7f9536ffd700 1 --2- 172.21.15.59:0/262561392 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f952806b500 0x7f952806d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:00:27.008 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:27.002+0000 7f9536ffd700 1 -- 172.21.15.59:0/262561392 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f9538022070 con 0x7f954806fc50 2023-11-24T08:00:27.009 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:27.002+0000 7f9545d9b700 1 --2- 172.21.15.59:0/262561392 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f952806b500 0x7f952806d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:00:27.009 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:27.002+0000 7f9545d9b700 1 --2- 172.21.15.59:0/262561392 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f952806b500 0x7f952806d9c0 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7f95481f6f10 tx=0x7f953c009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:00:27.009 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:27.006+0000 7f9536ffd700 1 -- 172.21.15.59:0/262561392 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f9538056eb0 con 0x7f954806fc50 2023-11-24T08:00:27.203 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:27.201+0000 7f954c8be700 1 -- 172.21.15.59:0/262561392 --> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f954815e5a0 con 0x7f952806b500 2023-11-24T08:00:27.205 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:27.204+0000 7f9536ffd700 1 -- 172.21.15.59:0/262561392 <== mgr.4100 v2:172.21.15.76:6832/4264046536 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+50856 (secure 0 0 0) 0x7f953c0014f0 con 0x7f952806b500 2023-11-24T08:00:27.206 INFO:teuthology.orchestra.run.smithi059.stdout: 2023-11-24T08:00:27.208 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:27.206+0000 7f954c8be700 1 -- 172.21.15.59:0/262561392 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f952806b500 msgr2=0x7f952806d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:00:27.208 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:27.206+0000 7f954c8be700 1 --2- 172.21.15.59:0/262561392 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f952806b500 0x7f952806d9c0 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7f95481f6f10 tx=0x7f953c009040 comp rx=0 tx=0).stop 2023-11-24T08:00:27.208 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:27.206+0000 7f954c8be700 1 -- 172.21.15.59:0/262561392 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f954806fc50 msgr2=0x7f95481e58d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:00:27.208 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:27.206+0000 7f954c8be700 1 --2- 172.21.15.59:0/262561392 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f954806fc50 0x7f95481e58d0 secure :-1 s=READY pgs=1131 cs=0 l=1 rev1=1 crypto rx=0x7f9548072340 tx=0x7f9538004ad0 comp rx=0 tx=0).stop 2023-11-24T08:00:27.208 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:27.206+0000 7f954c8be700 1 -- 172.21.15.59:0/262561392 shutdown_connections 2023-11-24T08:00:27.209 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:27.206+0000 7f954c8be700 1 --2- 172.21.15.59:0/262561392 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f952806b500 0x7f952806d9c0 unknown :-1 s=CLOSED pgs=517 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:00:27.209 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:27.206+0000 7f954c8be700 1 --2- 172.21.15.59:0/262561392 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f954815e1c0 0x7f95481e7420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:00:27.209 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:27.206+0000 7f954c8be700 1 --2- 172.21.15.59:0/262561392 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f954806fc50 0x7f95481e58d0 unknown :-1 s=CLOSED pgs=1131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:00:27.209 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:27.206+0000 7f954c8be700 1 --2- 172.21.15.59:0/262561392 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f95480b0e90 0x7f954809f540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:00:27.209 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:27.206+0000 7f954c8be700 1 -- 172.21.15.59:0/262561392 >> 172.21.15.59:0/262561392 conn(0x7f95480b3720 msgr2=0x7f95480b1690 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:00:27.209 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:27.206+0000 7f954c8be700 1 -- 172.21.15.59:0/262561392 shutdown_connections 2023-11-24T08:00:27.209 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:27.206+0000 7f954c8be700 1 -- 172.21.15.59:0/262561392 wait complete. 2023-11-24T08:00:27.210 INFO:teuthology.orchestra.run.smithi059.stderr:dumped all 2023-11-24T08:00:27.220 INFO:teuthology.orchestra.run.smithi059.stdout:{"pg_ready":true,"pg_map":{"version":207,"stamp":"2023-11-24T08:00:26.355476+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":230048,"kb_used_data":9728,"kb_used_omap":12,"kb_used_meta":220211,"kb_avail":754744672,"statfs":{"total":773094113280,"available":772858544128,"internally_reserved":0,"allocated":9961472,"data_stored":4958984,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12745,"internal_metadata":225496631},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-28340,"num_objects":-23,"num_object_clones":0,"num_object_copies":-46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-23,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":-61,"num_write_kb":-49,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001376"},"pg_stats":[{"pgid":"1.0","version":"16'117","reported_seq":224,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-11-24T08:00:21.312012+0000","last_change":"2023-11-24T07:56:19.668905+0000","last_active":"2023-11-24T08:00:21.312012+0000","last_peered":"2023-11-24T08:00:21.312012+0000","last_clean":"2023-11-24T08:00:21.312012+0000","last_became_active":"2023-11-24T07:56:19.668601+0000","last_became_peered":"2023-11-24T07:56:19.668601+0000","last_unstale":"2023-11-24T08:00:21.312012+0000","last_undegraded":"2023-11-24T08:00:21.312012+0000","last_fullsized":"2023-11-24T08:00:21.312012+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:56:18.657844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:56:18.657844+0000","last_clean_scrub_stamp":"2023-11-24T07:56:18.657844+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T11:51:53.828455+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574900,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28084,"kb_used_data":1064,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343756,"statfs":{"total":96636764160,"available":96608006144,"internally_reserved":0,"allocated":1089536,"data_stored":465400,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 08:00:21 2023","interfaces":[{"interface":"back","average":{"1min":0.66700000000000004,"5min":0.56200000000000006,"15min":0.55800000000000005},"min":{"1min":0.27700000000000002,"5min":0.27600000000000002,"15min":0.27600000000000002},"max":{"1min":3.1640000000000001,"5min":3.1640000000000001,"15min":3.1640000000000001},"last":0.51300000000000001},{"interface":"front","average":{"1min":0.51800000000000002,"5min":0.54400000000000004,"15min":0.55000000000000004},"min":{"1min":0.35499999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.67200000000000004,"5min":0.81000000000000005,"15min":0.81000000000000005},"last":0.40200000000000002}]},{"osd":1,"last update":"Fri Nov 24 08:00:21 2023","interfaces":[{"interface":"back","average":{"1min":0.50700000000000001,"5min":0.64400000000000002,"15min":0.78000000000000003},"min":{"1min":0.28799999999999998,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":0.83499999999999996,"5min":6.1070000000000002,"15min":6.1070000000000002},"last":0.58999999999999997},{"interface":"front","average":{"1min":0.68899999999999995,"5min":0.56899999999999995,"15min":0.57599999999999996},"min":{"1min":0.38400000000000001,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":3.1970000000000001,"5min":3.1970000000000001,"15min":3.1970000000000001},"last":0.55800000000000005}]},{"osd":2,"last update":"Fri Nov 24 08:00:21 2023","interfaces":[{"interface":"back","average":{"1min":0.53500000000000003,"5min":0.56200000000000006,"15min":0.60499999999999998},"min":{"1min":0.35599999999999998,"5min":0.308,"15min":0.308},"max":{"1min":0.94099999999999995,"5min":1.613,"15min":1.613},"last":0.61399999999999999},{"interface":"front","average":{"1min":0.51800000000000002,"5min":0.61699999999999999,"15min":0.73299999999999998},"min":{"1min":0.35199999999999998,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.65400000000000003,"5min":4.5220000000000002,"15min":4.5220000000000002},"last":0.45700000000000002}]},{"osd":3,"last update":"Fri Nov 24 08:00:21 2023","interfaces":[{"interface":"back","average":{"1min":0.56399999999999995,"5min":0.65800000000000003,"15min":0.75600000000000001},"min":{"1min":0.42599999999999999,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.88500000000000001,"5min":4.6760000000000002,"15min":4.6760000000000002},"last":0.54000000000000004},{"interface":"front","average":{"1min":0.64500000000000002,"5min":0.748,"15min":0.878},"min":{"1min":0.51200000000000001,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":1.1299999999999999,"5min":4.6539999999999999,"15min":4.6539999999999999},"last":0.68700000000000006}]},{"osd":4,"last update":"Fri Nov 24 08:00:21 2023","interfaces":[{"interface":"back","average":{"1min":0.53300000000000003,"5min":0.53800000000000003,"15min":0.55800000000000005},"min":{"1min":0.33600000000000002,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":1.0760000000000001,"5min":1.0760000000000001,"15min":1.0760000000000001},"last":0.42599999999999999},{"interface":"front","average":{"1min":0.59299999999999997,"5min":0.60499999999999998,"15min":0.64600000000000002},"min":{"1min":0.39900000000000002,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":1.284,"5min":1.629,"15min":1.629},"last":0.71999999999999997}]},{"osd":5,"last update":"Fri Nov 24 08:00:21 2023","interfaces":[{"interface":"back","average":{"1min":0.51100000000000001,"5min":0.52900000000000003,"15min":0.54100000000000004},"min":{"1min":0.36399999999999999,"5min":0.315,"15min":0.315},"max":{"1min":0.77200000000000002,"5min":0.875,"15min":0.875},"last":0.49299999999999999},{"interface":"front","average":{"1min":0.626,"5min":0.60299999999999998,"15min":0.61599999999999999},"min":{"1min":0.47499999999999998,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":1.0429999999999999,"5min":1.0429999999999999,"15min":1.0429999999999999},"last":0.66700000000000004}]},{"osd":6,"last update":"Fri Nov 24 08:00:21 2023","interfaces":[{"interface":"back","average":{"1min":0.55000000000000004,"5min":0.58399999999999996,"15min":0.60199999999999998},"min":{"1min":0.38600000000000001,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.745,"5min":0.85799999999999998,"15min":0.85799999999999998},"last":0.64600000000000002},{"interface":"front","average":{"1min":0.61199999999999999,"5min":0.64000000000000001,"15min":0.66000000000000003},"min":{"1min":0.438,"5min":0.438,"15min":0.438},"max":{"1min":0.91700000000000004,"5min":0.99099999999999999,"15min":0.99099999999999999},"last":0.47499999999999998}]}]},{"osd":6,"up_from":13,"seq":55834574901,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28084,"kb_used_data":1064,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343756,"statfs":{"total":96636764160,"available":96608006144,"internally_reserved":0,"allocated":1089536,"data_stored":465868,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 07:59:19 2023","interfaces":[{"interface":"back","average":{"1min":0.51000000000000001,"5min":0.53800000000000003,"15min":0.54600000000000004},"min":{"1min":0.33500000000000002,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":0.82999999999999996,"5min":0.93100000000000005,"15min":0.93100000000000005},"last":0.55900000000000005},{"interface":"front","average":{"1min":0.53100000000000003,"5min":0.51900000000000002,"15min":0.51700000000000002},"min":{"1min":0.34000000000000002,"5min":0.311,"15min":0.311},"max":{"1min":0.72299999999999998,"5min":0.82199999999999995,"15min":0.82199999999999995},"last":0.45800000000000002}]},{"osd":1,"last update":"Fri Nov 24 07:59:19 2023","interfaces":[{"interface":"back","average":{"1min":0.499,"5min":0.51000000000000001,"15min":0.51100000000000001},"min":{"1min":0.312,"5min":0.312,"15min":0.312},"max":{"1min":0.746,"5min":0.81299999999999994,"15min":0.81299999999999994},"last":0.47099999999999997},{"interface":"front","average":{"1min":0.55900000000000005,"5min":0.58699999999999997,"15min":0.60399999999999998},"min":{"1min":0.29999999999999999,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":0.70799999999999996,"5min":1.0069999999999999,"15min":1.0069999999999999},"last":0.66200000000000003}]},{"osd":2,"last update":"Fri Nov 24 07:59:19 2023","interfaces":[{"interface":"back","average":{"1min":0.57699999999999996,"5min":0.57699999999999996,"15min":0.57499999999999996},"min":{"1min":0.41399999999999998,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.71099999999999997,"5min":0.90600000000000003,"15min":0.90600000000000003},"last":0.52000000000000002},{"interface":"front","average":{"1min":0.59899999999999998,"5min":0.58799999999999997,"15min":0.59099999999999997},"min":{"1min":0.52500000000000002,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.78800000000000003,"5min":0.84899999999999998,"15min":0.84899999999999998},"last":0.63400000000000001}]},{"osd":3,"last update":"Fri Nov 24 07:59:19 2023","interfaces":[{"interface":"back","average":{"1min":0.56200000000000006,"5min":0.56499999999999995,"15min":0.56699999999999995},"min":{"1min":0.38700000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.75700000000000001,"5min":0.876,"15min":0.876},"last":0.40999999999999998},{"interface":"front","average":{"1min":0.55700000000000005,"5min":0.56000000000000005,"15min":0.55700000000000005},"min":{"1min":0.40500000000000003,"5min":0.379,"15min":0.379},"max":{"1min":0.72599999999999998,"5min":0.876,"15min":0.876},"last":0.622}]},{"osd":4,"last update":"Fri Nov 24 07:59:19 2023","interfaces":[{"interface":"back","average":{"1min":0.55300000000000005,"5min":0.52700000000000002,"15min":0.52000000000000002},"min":{"1min":0.36299999999999999,"5min":0.25,"15min":0.25},"max":{"1min":0.80700000000000005,"5min":0.89400000000000002,"15min":0.89400000000000002},"last":0.505},{"interface":"front","average":{"1min":0.53100000000000003,"5min":0.58499999999999996,"15min":0.59599999999999997},"min":{"1min":0.30599999999999999,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":0.81899999999999995,"5min":1.087,"15min":1.087},"last":0.60399999999999998}]},{"osd":5,"last update":"Fri Nov 24 07:59:19 2023","interfaces":[{"interface":"back","average":{"1min":0.57099999999999995,"5min":0.55100000000000005,"15min":0.53700000000000003},"min":{"1min":0.33300000000000002,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":0.76200000000000001,"5min":0.97999999999999998,"15min":0.97999999999999998},"last":0.58299999999999996},{"interface":"front","average":{"1min":0.62,"5min":0.61399999999999999,"15min":0.61199999999999999},"min":{"1min":0.438,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.997,"5min":1.0009999999999999,"15min":1.0009999999999999},"last":0.434}]},{"osd":7,"last update":"Fri Nov 24 07:59:19 2023","interfaces":[{"interface":"back","average":{"1min":0.59899999999999998,"5min":0.61399999999999999,"15min":0.61599999999999999},"min":{"1min":0.44800000000000001,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":0.85499999999999998,"5min":1.0880000000000001,"15min":1.0880000000000001},"last":0.53600000000000003},{"interface":"front","average":{"1min":0.628,"5min":0.65800000000000003,"15min":0.66600000000000004},"min":{"1min":0.32800000000000001,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.79500000000000004,"5min":0.95599999999999996,"15min":0.95599999999999996},"last":0.65100000000000002}]}]},{"osd":1,"up_from":13,"seq":55834574902,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28660,"kb_used_data":1640,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343180,"statfs":{"total":96636764160,"available":96607416320,"internally_reserved":0,"allocated":1679360,"data_stored":1055654,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 07:59:24 2023","interfaces":[{"interface":"back","average":{"1min":0.35499999999999998,"5min":0.35499999999999998,"15min":0.36099999999999999},"min":{"1min":0.248,"5min":0.20200000000000001,"15min":0.20200000000000001},"max":{"1min":0.52600000000000002,"5min":0.70899999999999996,"15min":0.70899999999999996},"last":0.313},{"interface":"front","average":{"1min":0.36299999999999999,"5min":0.38400000000000001,"15min":0.39300000000000002},"min":{"1min":0.23300000000000001,"5min":0.214,"15min":0.214},"max":{"1min":0.65600000000000003,"5min":0.82499999999999996,"15min":0.82499999999999996},"last":0.33600000000000002}]},{"osd":2,"last update":"Fri Nov 24 07:59:24 2023","interfaces":[{"interface":"back","average":{"1min":0.36899999999999999,"5min":0.35999999999999999,"15min":0.36799999999999999},"min":{"1min":0.27100000000000002,"5min":0.22700000000000001,"15min":0.22700000000000001},"max":{"1min":0.53000000000000003,"5min":0.60999999999999999,"15min":0.60999999999999999},"last":0.26600000000000001},{"interface":"front","average":{"1min":0.42299999999999999,"5min":0.40500000000000003,"15min":0.40999999999999998},"min":{"1min":0.28899999999999998,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":0.63700000000000001,"5min":0.63700000000000001,"15min":0.63700000000000001},"last":0.376}]},{"osd":3,"last update":"Fri Nov 24 07:59:24 2023","interfaces":[{"interface":"back","average":{"1min":0.43099999999999999,"5min":0.433,"15min":0.44},"min":{"1min":0.317,"5min":0.317,"15min":0.317},"max":{"1min":0.62,"5min":0.66500000000000004,"15min":0.66500000000000004},"last":0.35199999999999998},{"interface":"front","average":{"1min":0.41799999999999998,"5min":0.41699999999999998,"15min":0.42399999999999999},"min":{"1min":0.32900000000000001,"5min":0.28499999999999998,"15min":0.28499999999999998},"max":{"1min":0.59999999999999998,"5min":0.76300000000000001,"15min":0.76300000000000001},"last":0.29499999999999998}]},{"osd":4,"last update":"Fri Nov 24 07:59:24 2023","interfaces":[{"interface":"back","average":{"1min":0.625,"5min":0.60399999999999998,"15min":0.60799999999999998},"min":{"1min":0.434,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":1.5580000000000001,"5min":1.5580000000000001,"15min":1.5580000000000001},"last":0.54100000000000004},{"interface":"front","average":{"1min":0.65200000000000002,"5min":0.61599999999999999,"15min":0.60899999999999999},"min":{"1min":0.48599999999999999,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":0.82399999999999995,"5min":0.82399999999999995,"15min":0.82399999999999995},"last":0.66200000000000003}]},{"osd":5,"last update":"Fri Nov 24 07:59:24 2023","interfaces":[{"interface":"back","average":{"1min":0.625,"5min":0.67800000000000005,"15min":0.70899999999999996},"min":{"1min":0.44400000000000001,"5min":0.37,"15min":0.37},"max":{"1min":0.755,"5min":2.7269999999999999,"15min":2.7269999999999999},"last":0.60699999999999998},{"interface":"front","average":{"1min":0.68500000000000005,"5min":0.63900000000000001,"15min":0.64300000000000002},"min":{"1min":0.434,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":1.6459999999999999,"5min":1.6459999999999999,"15min":1.6459999999999999},"last":0.65100000000000002}]},{"osd":6,"last update":"Fri Nov 24 07:59:24 2023","interfaces":[{"interface":"back","average":{"1min":0.63200000000000001,"5min":0.64300000000000002,"15min":0.65300000000000002},"min":{"1min":0.46000000000000002,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":0.93799999999999994,"5min":0.98199999999999998,"15min":0.98199999999999998},"last":0.56000000000000005},{"interface":"front","average":{"1min":0.67900000000000005,"5min":0.621,"15min":0.61799999999999999},"min":{"1min":0.46100000000000002,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":0.91600000000000004,"5min":0.91600000000000004,"15min":0.91600000000000004},"last":0.67200000000000004}]},{"osd":7,"last update":"Fri Nov 24 07:59:24 2023","interfaces":[{"interface":"back","average":{"1min":0.69699999999999995,"5min":0.64700000000000002,"15min":0.64000000000000001},"min":{"1min":0.53900000000000003,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.95599999999999996,"5min":1.0649999999999999,"15min":1.0649999999999999},"last":0.57699999999999996},{"interface":"front","average":{"1min":0.64300000000000002,"5min":0.64400000000000002,"15min":0.66100000000000003},"min":{"1min":0.45900000000000002,"5min":0.442,"15min":0.442},"max":{"1min":0.93300000000000005,"5min":0.93300000000000005,"15min":0.93300000000000005},"last":0.68600000000000005}]}]},{"osd":0,"up_from":13,"seq":55834574902,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28116,"kb_used_data":1096,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343724,"statfs":{"total":96636764160,"available":96607973376,"internally_reserved":0,"allocated":1122304,"data_stored":492626,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Nov 24 07:59:22 2023","interfaces":[{"interface":"back","average":{"1min":0.36899999999999999,"5min":0.57799999999999996,"15min":0.66800000000000004},"min":{"1min":0.247,"5min":0.21299999999999999,"15min":0.21299999999999999},"max":{"1min":0.56100000000000005,"5min":6.3550000000000004,"15min":6.3550000000000004},"last":0.377},{"interface":"front","average":{"1min":0.39300000000000002,"5min":1.1970000000000001,"15min":1.5509999999999999},"min":{"1min":0.29399999999999998,"5min":0.26000000000000001,"15min":0.26000000000000001},"max":{"1min":0.59099999999999997,"5min":22.916,"15min":22.916},"last":0.40100000000000002}]},{"osd":2,"last update":"Fri Nov 24 07:59:22 2023","interfaces":[{"interface":"back","average":{"1min":0.35999999999999999,"5min":0.56200000000000006,"15min":0.65400000000000003},"min":{"1min":0.23400000000000001,"5min":0.22800000000000001,"15min":0.22800000000000001},"max":{"1min":0.54100000000000004,"5min":6.3899999999999997,"15min":6.3899999999999997},"last":0.29599999999999999},{"interface":"front","average":{"1min":0.32300000000000001,"5min":0.78100000000000003,"15min":0.96899999999999997},"min":{"1min":0.253,"5min":0.253,"15min":0.253},"max":{"1min":0.497,"5min":11.93,"15min":11.93},"last":0.35599999999999998}]},{"osd":3,"last update":"Fri Nov 24 07:59:22 2023","interfaces":[{"interface":"back","average":{"1min":0.373,"5min":0.40100000000000002,"15min":0.41299999999999998},"min":{"1min":0.247,"5min":0.23799999999999999,"15min":0.23799999999999999},"max":{"1min":0.52500000000000002,"5min":1.155,"15min":1.155},"last":0.317},{"interface":"front","average":{"1min":0.40200000000000002,"5min":0.84399999999999997,"15min":1.032},"min":{"1min":0.27400000000000002,"5min":0.27400000000000002,"15min":0.27400000000000002},"max":{"1min":0.63400000000000001,"5min":12.116,"15min":12.116},"last":0.41599999999999998}]},{"osd":4,"last update":"Fri Nov 24 07:59:22 2023","interfaces":[{"interface":"back","average":{"1min":0.59799999999999998,"5min":0.63300000000000001,"15min":0.64300000000000002},"min":{"1min":0.44400000000000001,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.93500000000000005,"5min":1.044,"15min":1.044},"last":0.63700000000000001},{"interface":"front","average":{"1min":0.58499999999999996,"5min":1.4059999999999999,"15min":1.758},"min":{"1min":0.377,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.84999999999999998,"5min":22.963999999999999,"15min":22.963999999999999},"last":0.55200000000000005}]},{"osd":5,"last update":"Fri Nov 24 07:59:22 2023","interfaces":[{"interface":"back","average":{"1min":0.59999999999999998,"5min":1.026,"15min":1.204},"min":{"1min":0.41199999999999998,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.86299999999999999,"5min":12.084,"15min":12.084},"last":0.66100000000000003},{"interface":"front","average":{"1min":0.61899999999999999,"5min":0.66200000000000003,"15min":0.67200000000000004},"min":{"1min":0.46600000000000003,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.89100000000000001,"5min":1.073,"15min":1.073},"last":0.56699999999999995}]},{"osd":6,"last update":"Fri Nov 24 07:59:22 2023","interfaces":[{"interface":"back","average":{"1min":0.61599999999999999,"5min":0.66800000000000004,"15min":0.68300000000000005},"min":{"1min":0.36799999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.92400000000000004,"5min":1.0580000000000001,"15min":1.0580000000000001},"last":0.53900000000000003},{"interface":"front","average":{"1min":0.60099999999999998,"5min":1.042,"15min":1.2230000000000001},"min":{"1min":0.46000000000000002,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":0.81999999999999995,"5min":12.045,"15min":12.045},"last":0.62}]},{"osd":7,"last update":"Fri Nov 24 07:59:22 2023","interfaces":[{"interface":"back","average":{"1min":0.57699999999999996,"5min":0.64300000000000002,"15min":0.65100000000000002},"min":{"1min":0.45700000000000002,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":0.82999999999999996,"5min":1.038,"15min":1.038},"last":0.60899999999999999},{"interface":"front","average":{"1min":0.59499999999999997,"5min":1.431,"15min":1.7849999999999999},"min":{"1min":0.42299999999999999,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":0.80300000000000005,"5min":22.995000000000001,"15min":22.995000000000001},"last":0.67300000000000004}]}]},{"osd":2,"up_from":13,"seq":55834574902,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28080,"kb_used_data":1060,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343760,"statfs":{"total":96636764160,"available":96608010240,"internally_reserved":0,"allocated":1085440,"data_stored":465330,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 08:00:21 2023","interfaces":[{"interface":"back","average":{"1min":0.34499999999999997,"5min":0.35399999999999998,"15min":0.36499999999999999},"min":{"1min":0.22500000000000001,"5min":0.219,"15min":0.219},"max":{"1min":0.499,"5min":0.58099999999999996,"15min":0.58099999999999996},"last":0.47199999999999998},{"interface":"front","average":{"1min":0.41299999999999998,"5min":0.39600000000000002,"15min":0.40000000000000002},"min":{"1min":0.28899999999999998,"5min":0.27400000000000002,"15min":0.27400000000000002},"max":{"1min":0.69099999999999995,"5min":0.69099999999999995,"15min":0.69099999999999995},"last":0.442}]},{"osd":1,"last update":"Fri Nov 24 08:00:21 2023","interfaces":[{"interface":"back","average":{"1min":0.34999999999999998,"5min":0.35499999999999998,"15min":0.37},"min":{"1min":0.249,"5min":0.21099999999999999,"15min":0.21099999999999999},"max":{"1min":0.55700000000000005,"5min":0.59199999999999997,"15min":0.59199999999999997},"last":0.40100000000000002},{"interface":"front","average":{"1min":0.379,"5min":0.38800000000000001,"15min":0.40600000000000003},"min":{"1min":0.24399999999999999,"5min":0.24399999999999999,"15min":0.24399999999999999},"max":{"1min":0.55100000000000005,"5min":0.627,"15min":0.627},"last":0.54700000000000004}]},{"osd":3,"last update":"Fri Nov 24 08:00:21 2023","interfaces":[{"interface":"back","average":{"1min":0.42499999999999999,"5min":0.40400000000000003,"15min":0.40799999999999997},"min":{"1min":0.28699999999999998,"5min":0.26400000000000001,"15min":0.26400000000000001},"max":{"1min":0.63500000000000001,"5min":0.64200000000000002,"15min":0.64200000000000002},"last":0.499},{"interface":"front","average":{"1min":0.39000000000000001,"5min":0.40000000000000002,"15min":0.40899999999999997},"min":{"1min":0.315,"5min":0.27500000000000002,"15min":0.27500000000000002},"max":{"1min":0.58299999999999996,"5min":0.64600000000000002,"15min":0.64600000000000002},"last":0.33100000000000002}]},{"osd":4,"last update":"Fri Nov 24 08:00:21 2023","interfaces":[{"interface":"back","average":{"1min":0.58699999999999997,"5min":0.57099999999999995,"15min":0.59499999999999997},"min":{"1min":0.32900000000000001,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.91000000000000003,"5min":0.91000000000000003,"15min":0.91000000000000003},"last":0.91000000000000003},{"interface":"front","average":{"1min":0.57099999999999995,"5min":0.57699999999999996,"15min":0.59199999999999997},"min":{"1min":0.35299999999999998,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.75600000000000001,"5min":0.83799999999999997,"15min":0.83799999999999997},"last":0.65200000000000002}]},{"osd":5,"last update":"Fri Nov 24 08:00:21 2023","interfaces":[{"interface":"back","average":{"1min":0.57899999999999996,"5min":0.58999999999999997,"15min":0.60999999999999999},"min":{"1min":0.39100000000000001,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.84099999999999997,"5min":0.84899999999999998,"15min":0.84899999999999998},"last":0.68100000000000005},{"interface":"front","average":{"1min":0.64500000000000002,"5min":0.64600000000000002,"15min":0.69199999999999995},"min":{"1min":0.45100000000000001,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":1.119,"5min":2.1699999999999999,"15min":2.1699999999999999},"last":0.83699999999999997}]},{"osd":6,"last update":"Fri Nov 24 08:00:21 2023","interfaces":[{"interface":"back","average":{"1min":0.64000000000000001,"5min":0.625,"15min":0.65300000000000002},"min":{"1min":0.41999999999999998,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":0.88300000000000001,"5min":0.88300000000000001,"15min":0.88300000000000001},"last":0.88300000000000001},{"interface":"front","average":{"1min":0.65400000000000003,"5min":0.629,"15min":0.65700000000000003},"min":{"1min":0.44500000000000001,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":1.105,"5min":1.105,"15min":1.105},"last":0.80600000000000005}]},{"osd":7,"last update":"Fri Nov 24 08:00:21 2023","interfaces":[{"interface":"back","average":{"1min":0.628,"5min":0.626,"15min":0.65000000000000002},"min":{"1min":0.42199999999999999,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":0.93000000000000005,"5min":0.93000000000000005,"15min":0.93000000000000005},"last":0.70899999999999996},{"interface":"front","average":{"1min":0.627,"5min":0.63600000000000001,"15min":0.66500000000000004},"min":{"1min":0.46200000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.80000000000000004,"5min":0.90000000000000002,"15min":0.90000000000000002},"last":0.77400000000000002}]}]},{"osd":3,"up_from":13,"seq":55834574902,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28088,"kb_used_data":1068,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343752,"statfs":{"total":96636764160,"available":96608002048,"internally_reserved":0,"allocated":1093632,"data_stored":465870,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 07:59:27 2023","interfaces":[{"interface":"back","average":{"1min":0.33600000000000002,"5min":0.34899999999999998,"15min":0.35299999999999998},"min":{"1min":0.183,"5min":0.183,"15min":0.183},"max":{"1min":0.53400000000000003,"5min":0.84799999999999998,"15min":0.84799999999999998},"last":0.30299999999999999},{"interface":"front","average":{"1min":0.34599999999999997,"5min":0.35399999999999998,"15min":0.36199999999999999},"min":{"1min":0.24299999999999999,"5min":0.23699999999999999,"15min":0.23699999999999999},"max":{"1min":0.51000000000000001,"5min":0.873,"15min":0.873},"last":0.32000000000000001}]},{"osd":1,"last update":"Fri Nov 24 07:59:27 2023","interfaces":[{"interface":"back","average":{"1min":0.38400000000000001,"5min":0.373,"15min":0.374},"min":{"1min":0.217,"5min":0.217,"15min":0.217},"max":{"1min":0.66300000000000003,"5min":0.92000000000000004,"15min":0.92000000000000004},"last":0.26600000000000001},{"interface":"front","average":{"1min":0.38,"5min":0.377,"15min":0.38300000000000001},"min":{"1min":0.28599999999999998,"5min":0.248,"15min":0.248},"max":{"1min":0.626,"5min":0.81699999999999995,"15min":0.81699999999999995},"last":0.34899999999999998}]},{"osd":2,"last update":"Fri Nov 24 07:59:27 2023","interfaces":[{"interface":"back","average":{"1min":0.35599999999999998,"5min":0.35399999999999998,"15min":0.35799999999999998},"min":{"1min":0.253,"5min":0.24399999999999999,"15min":0.24399999999999999},"max":{"1min":0.496,"5min":0.88500000000000001,"15min":0.88500000000000001},"last":0.36599999999999999},{"interface":"front","average":{"1min":0.40999999999999998,"5min":0.441,"15min":0.45900000000000002},"min":{"1min":0.315,"5min":0.311,"15min":0.311},"max":{"1min":0.59799999999999998,"5min":0.98099999999999998,"15min":0.98099999999999998},"last":0.33600000000000002}]},{"osd":4,"last update":"Fri Nov 24 07:59:27 2023","interfaces":[{"interface":"back","average":{"1min":0.56299999999999994,"5min":0.70499999999999996,"15min":0.76600000000000001},"min":{"1min":0.38900000000000001,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.73599999999999999,"5min":5.7350000000000003,"15min":5.7350000000000003},"last":0.51000000000000001},{"interface":"front","average":{"1min":0.55700000000000005,"5min":0.58099999999999996,"15min":0.58299999999999996},"min":{"1min":0.34399999999999997,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.80400000000000005,"5min":1.27,"15min":1.27},"last":0.41299999999999998}]},{"osd":5,"last update":"Fri Nov 24 07:59:27 2023","interfaces":[{"interface":"back","average":{"1min":0.56100000000000005,"5min":0.57799999999999996,"15min":0.58399999999999996},"min":{"1min":0.42899999999999999,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":0.67900000000000005,"5min":1.4430000000000001,"15min":1.4430000000000001},"last":0.47999999999999998},{"interface":"front","average":{"1min":0.58799999999999997,"5min":0.61499999999999999,"15min":0.623},"min":{"1min":0.40999999999999998,"5min":0.377,"15min":0.377},"max":{"1min":0.78500000000000003,"5min":1.4239999999999999,"15min":1.4239999999999999},"last":0.434}]},{"osd":6,"last update":"Fri Nov 24 07:59:27 2023","interfaces":[{"interface":"back","average":{"1min":0.58799999999999997,"5min":0.60699999999999998,"15min":0.61499999999999999},"min":{"1min":0.42599999999999999,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":0.84499999999999997,"5min":1.355,"15min":1.355},"last":0.49399999999999999},{"interface":"front","average":{"1min":0.57899999999999996,"5min":0.745,"15min":0.81200000000000006},"min":{"1min":0.44800000000000001,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.77100000000000002,"5min":5.6479999999999997,"15min":5.6479999999999997},"last":0.42399999999999999}]},{"osd":7,"last update":"Fri Nov 24 07:59:27 2023","interfaces":[{"interface":"back","average":{"1min":0.61799999999999999,"5min":0.64100000000000001,"15min":0.65700000000000003},"min":{"1min":0.499,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":0.82799999999999996,"5min":1.51,"15min":1.51},"last":0.53400000000000003},{"interface":"front","average":{"1min":0.57699999999999996,"5min":0.74199999999999999,"15min":0.80800000000000005},"min":{"1min":0.40100000000000002,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.79500000000000004,"5min":5.7069999999999999,"15min":5.7069999999999999},"last":0.44700000000000001}]}]},{"osd":4,"up_from":13,"seq":55834574901,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28080,"kb_used_data":1060,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343760,"statfs":{"total":96636764160,"available":96608010240,"internally_reserved":0,"allocated":1085440,"data_stored":465230,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 07:59:28 2023","interfaces":[{"interface":"back","average":{"1min":0.55900000000000005,"5min":0.56100000000000005,"15min":0.57399999999999995},"min":{"1min":0.32400000000000001,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.93300000000000005,"5min":0.93300000000000005,"15min":0.93300000000000005},"last":0.63700000000000001},{"interface":"front","average":{"1min":0.51100000000000001,"5min":0.52800000000000002,"15min":0.53800000000000003},"min":{"1min":0.36199999999999999,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.69299999999999995,"5min":1.0249999999999999,"15min":1.0249999999999999},"last":0.72099999999999997}]},{"osd":1,"last update":"Fri Nov 24 07:59:28 2023","interfaces":[{"interface":"back","average":{"1min":0.54800000000000004,"5min":0.54600000000000004,"15min":0.55000000000000004},"min":{"1min":0.35899999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.71599999999999997,"5min":0.86399999999999999,"15min":0.86399999999999999},"last":0.68500000000000005},{"interface":"front","average":{"1min":0.51600000000000001,"5min":0.53700000000000003,"15min":0.54600000000000004},"min":{"1min":0.36699999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.72099999999999997,"5min":0.98699999999999999,"15min":0.98699999999999999},"last":0.70299999999999996}]},{"osd":2,"last update":"Fri Nov 24 07:59:28 2023","interfaces":[{"interface":"back","average":{"1min":0.54200000000000004,"5min":0.54900000000000004,"15min":0.55700000000000005},"min":{"1min":0.34499999999999997,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.86399999999999999,"5min":0.86399999999999999,"15min":0.86399999999999999},"last":0.59799999999999998},{"interface":"front","average":{"1min":0.56200000000000006,"5min":0.59399999999999997,"15min":0.60599999999999998},"min":{"1min":0.34000000000000002,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.81899999999999995,"5min":0.89200000000000002,"15min":0.89200000000000002},"last":0.80700000000000005}]},{"osd":3,"last update":"Fri Nov 24 07:59:28 2023","interfaces":[{"interface":"back","average":{"1min":0.53600000000000003,"5min":0.56799999999999995,"15min":0.57899999999999996},"min":{"1min":0.38800000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.78100000000000003,"5min":0.89700000000000002,"15min":0.89700000000000002},"last":0.73799999999999999},{"interface":"front","average":{"1min":0.58899999999999997,"5min":0.61099999999999999,"15min":0.624},"min":{"1min":0.42699999999999999,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.73999999999999999,"5min":0.96199999999999997,"15min":0.96199999999999997},"last":0.83899999999999997}]},{"osd":5,"last update":"Fri Nov 24 07:59:28 2023","interfaces":[{"interface":"back","average":{"1min":0.46800000000000003,"5min":0.46300000000000002,"15min":0.45700000000000002},"min":{"1min":0.34999999999999998,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":0.60499999999999998,"5min":0.70499999999999996,"15min":0.70499999999999996},"last":0.61899999999999999},{"interface":"front","average":{"1min":0.56599999999999995,"5min":0.60399999999999998,"15min":0.61399999999999999},"min":{"1min":0.41599999999999998,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.73699999999999999,"5min":0.86399999999999999,"15min":0.86399999999999999},"last":0.79100000000000004}]},{"osd":6,"last update":"Fri Nov 24 07:59:28 2023","interfaces":[{"interface":"back","average":{"1min":0.53900000000000003,"5min":0.57299999999999995,"15min":0.58099999999999996},"min":{"1min":0.33000000000000002,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.88600000000000001,"5min":0.88600000000000001,"15min":0.88600000000000001},"last":0.56100000000000005},{"interface":"front","average":{"1min":0.56899999999999995,"5min":0.59999999999999998,"15min":0.60899999999999999},"min":{"1min":0.32100000000000001,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.80300000000000005,"5min":1.095,"15min":1.095},"last":0.77200000000000002}]},{"osd":7,"last update":"Fri Nov 24 07:59:28 2023","interfaces":[{"interface":"back","average":{"1min":0.52000000000000002,"5min":0.56100000000000005,"15min":0.56899999999999995},"min":{"1min":0.36299999999999999,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.69999999999999996,"5min":0.89300000000000002,"15min":0.89300000000000002},"last":0.67000000000000004},{"interface":"front","average":{"1min":0.57799999999999996,"5min":0.63600000000000001,"15min":0.65400000000000003},"min":{"1min":0.39000000000000001,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.74399999999999999,"5min":0.91500000000000004,"15min":0.91500000000000004},"last":0.75600000000000001}]}]},{"osd":5,"up_from":13,"seq":55834574902,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32856,"kb_used_data":1676,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338984,"statfs":{"total":96636764160,"available":96603119616,"internally_reserved":0,"allocated":1716224,"data_stored":1083006,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 07:59:21 2023","interfaces":[{"interface":"back","average":{"1min":0.52500000000000002,"5min":0.54600000000000004,"15min":0.55600000000000005},"min":{"1min":0.311,"5min":0.311,"15min":0.311},"max":{"1min":0.74399999999999999,"5min":1.0109999999999999,"15min":1.0109999999999999},"last":0.40200000000000002},{"interface":"front","average":{"1min":0.53600000000000003,"5min":0.69299999999999995,"15min":0.76400000000000001},"min":{"1min":0.32400000000000001,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.81899999999999995,"5min":5.431,"15min":5.431},"last":0.504}]},{"osd":1,"last update":"Fri Nov 24 07:59:21 2023","interfaces":[{"interface":"back","average":{"1min":0.52600000000000002,"5min":0.54200000000000004,"15min":0.55300000000000005},"min":{"1min":0.35399999999999998,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":0.69099999999999995,"5min":1.016,"15min":1.016},"last":0.32400000000000001},{"interface":"front","average":{"1min":0.56399999999999995,"5min":0.56399999999999995,"15min":0.56799999999999995},"min":{"1min":0.34000000000000002,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":0.80100000000000005,"5min":0.96499999999999997,"15min":0.96499999999999997},"last":0.47899999999999998}]},{"osd":2,"last update":"Fri Nov 24 07:59:21 2023","interfaces":[{"interface":"back","average":{"1min":0.58099999999999996,"5min":0.55500000000000005,"15min":0.55200000000000005},"min":{"1min":0.44800000000000001,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.876,"5min":1.163,"15min":1.163},"last":0.35599999999999998},{"interface":"front","average":{"1min":0.55900000000000005,"5min":0.59799999999999998,"15min":0.61899999999999999},"min":{"1min":0.32600000000000001,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.72399999999999998,"5min":1.119,"15min":1.119},"last":0.53900000000000003}]},{"osd":3,"last update":"Fri Nov 24 07:59:21 2023","interfaces":[{"interface":"back","average":{"1min":0.59799999999999998,"5min":0.77700000000000002,"15min":0.86199999999999999},"min":{"1min":0.32400000000000001,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.77100000000000002,"5min":5.609,"15min":5.609},"last":0.52700000000000002},{"interface":"front","average":{"1min":0.64300000000000002,"5min":0.66000000000000003,"15min":0.67000000000000004},"min":{"1min":0.38900000000000001,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.84999999999999998,"5min":1.288,"15min":1.288},"last":0.56699999999999995}]},{"osd":4,"last update":"Fri Nov 24 07:59:21 2023","interfaces":[{"interface":"back","average":{"1min":0.53800000000000003,"5min":0.51600000000000001,"15min":0.51200000000000001},"min":{"1min":0.30599999999999999,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":0.79100000000000004,"5min":0.79100000000000004,"15min":0.79100000000000004},"last":0.45900000000000002},{"interface":"front","average":{"1min":0.58899999999999997,"5min":0.57799999999999996,"15min":0.57399999999999995},"min":{"1min":0.35399999999999998,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":0.82599999999999996,"5min":0.92900000000000005,"15min":0.92900000000000005},"last":0.55400000000000005}]},{"osd":6,"last update":"Fri Nov 24 07:59:21 2023","interfaces":[{"interface":"back","average":{"1min":0.63,"5min":0.58699999999999997,"15min":0.57599999999999996},"min":{"1min":0.434,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.85399999999999998,"5min":0.89700000000000002,"15min":0.89700000000000002},"last":0.627},{"interface":"front","average":{"1min":0.59299999999999997,"5min":0.621,"15min":0.623},"min":{"1min":0.376,"5min":0.376,"15min":0.376},"max":{"1min":0.74299999999999999,"5min":1.083,"15min":1.083},"last":0.59199999999999997}]},{"osd":7,"last update":"Fri Nov 24 07:59:21 2023","interfaces":[{"interface":"back","average":{"1min":0.65400000000000003,"5min":0.65000000000000002,"15min":0.64700000000000002},"min":{"1min":0.40699999999999997,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.86099999999999999,"5min":0.96599999999999997,"15min":0.96599999999999997},"last":0.58199999999999996},{"interface":"front","average":{"1min":0.68000000000000005,"5min":0.66100000000000003,"15min":0.65500000000000003},"min":{"1min":0.49099999999999999,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":0.78500000000000003,"5min":0.94199999999999995,"15min":0.94199999999999995},"last":0.60399999999999998}]}]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-11-24T08:00:27.221 INFO:tasks.ceph.ceph_manager.ceph:clean! 2023-11-24T08:00:27.222 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-11-24T08:00:27.347 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:27.345+0000 7f48a6af1700 1 Processor -- start 2023-11-24T08:00:27.347 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:27.345+0000 7f48a6af1700 1 -- start start 2023-11-24T08:00:27.349 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:27.346+0000 7f48a6af1700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f48a006b6e0 0x7f48a006db70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:00:27.349 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:27.346+0000 7f48a6af1700 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f48a0145100 0x7f48a006e0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:00:27.350 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:27.346+0000 7f48a6af1700 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f48a0160720 con 0x7f48a0144c10 2023-11-24T08:00:27.350 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:27.346+0000 7f48a6af1700 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f48a006f500 con 0x7f48a006b6e0 2023-11-24T08:00:27.350 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:27.346+0000 7f48a6af1700 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f48a00cd610 con 0x7f48a0145100 2023-11-24T08:00:27.350 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:27.346+0000 7f489ffff700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f48a006b6e0 0x7f48a006db70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:00:27.351 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:27.346+0000 7f489ffff700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f48a006b6e0 0x7f48a006db70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:56546/0 (socket says 172.21.15.59:56546) 2023-11-24T08:00:27.351 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:27.346+0000 7f489ffff700 1 -- 172.21.15.59:0/2996823370 learned_addr learned my addr 172.21.15.59:0/2996823370 (peer_addr_for_me v2:172.21.15.59:0/0) 2023-11-24T08:00:27.351 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:27.346+0000 7f48a508e700 1 --2- 172.21.15.59:0/2996823370 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f48a0145100 0x7f48a006e0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:00:27.351 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:27.346+0000 7f489ffff700 1 -- 172.21.15.59:0/2996823370 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f48a0145100 msgr2=0x7f48a006e0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:00:27.351 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:27.346+0000 7f489ffff700 1 --2- 172.21.15.59:0/2996823370 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f48a0145100 0x7f48a006e0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:00:27.352 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:27.346+0000 7f489ffff700 1 -- 172.21.15.59:0/2996823370 >> v1:172.21.15.76:6789/0 conn(0x7f48a0144c10 legacy=0x7f48a006afd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:00:27.352 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:27.346+0000 7f489ffff700 1 -- 172.21.15.59:0/2996823370 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f48a006f710 con 0x7f48a006b6e0 2023-11-24T08:00:27.352 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:27.347+0000 7f489ffff700 1 --2- 172.21.15.59:0/2996823370 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f48a006b6e0 0x7f48a006db70 secure :-1 s=READY pgs=1132 cs=0 l=1 rev1=1 crypto rx=0x7f488c0098f0 tx=0x7f488c009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=82cf2225ce5addd server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:00:27.352 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:27.347+0000 7f489f7fe700 1 -- 172.21.15.59:0/2996823370 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f488c011830 con 0x7f48a006b6e0 2023-11-24T08:00:27.352 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:27.347+0000 7f489f7fe700 1 -- 172.21.15.59:0/2996823370 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f488c011990 con 0x7f48a006b6e0 2023-11-24T08:00:27.353 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:27.347+0000 7f489f7fe700 1 -- 172.21.15.59:0/2996823370 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f488c011d20 con 0x7f48a006b6e0 2023-11-24T08:00:27.353 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:27.347+0000 7f48a6af1700 1 -- 172.21.15.59:0/2996823370 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f48a006b6e0 msgr2=0x7f48a006db70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:00:27.353 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:27.347+0000 7f48a6af1700 1 --2- 172.21.15.59:0/2996823370 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f48a006b6e0 0x7f48a006db70 secure :-1 s=READY pgs=1132 cs=0 l=1 rev1=1 crypto rx=0x7f488c0098f0 tx=0x7f488c009c00 comp rx=0 tx=0).stop 2023-11-24T08:00:27.353 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:27.347+0000 7f48a6af1700 1 -- 172.21.15.59:0/2996823370 shutdown_connections 2023-11-24T08:00:27.353 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:27.347+0000 7f48a6af1700 1 --2- 172.21.15.59:0/2996823370 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f48a0145100 0x7f48a006e0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:00:27.354 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:27.347+0000 7f48a6af1700 1 --2- 172.21.15.59:0/2996823370 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f48a006b6e0 0x7f48a006db70 unknown :-1 s=CLOSED pgs=1132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:00:27.354 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:27.347+0000 7f48a6af1700 1 -- 172.21.15.59:0/2996823370 >> 172.21.15.59:0/2996823370 conn(0x7f48a00b2740 msgr2=0x7f48a00b2b20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:00:27.354 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:27.347+0000 7f48a6af1700 1 -- 172.21.15.59:0/2996823370 shutdown_connections 2023-11-24T08:00:27.354 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:27.347+0000 7f48a6af1700 1 -- 172.21.15.59:0/2996823370 wait complete. 2023-11-24T08:00:27.359 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:27.349+0000 7f48a6af1700 1 Processor -- start 2023-11-24T08:00:27.359 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:27.349+0000 7f48a6af1700 1 -- start start 2023-11-24T08:00:27.360 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:27.350+0000 7f48a6af1700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f48a006b6e0 0x7f48a00b50a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:00:27.360 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:27.350+0000 7f48a6af1700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f48a0144c10 0x7f48a00ae240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:00:27.360 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:27.350+0000 7f48a6af1700 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f48a0145100 0x7f48a013d9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:00:27.360 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:27.350+0000 7f48a6af1700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48a00ae780 con 0x7f48a0144c10 2023-11-24T08:00:27.361 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:27.350+0000 7f48a6af1700 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48a013dee0 con 0x7f48a006b6e0 2023-11-24T08:00:27.361 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:27.350+0000 7f48a6af1700 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f48a00afa60 con 0x7f48a0145100 2023-11-24T08:00:27.361 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:27.350+0000 7f489ffff700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f48a0144c10 0x7f48a00ae240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:00:27.361 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:27.350+0000 7f489ffff700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f48a0144c10 0x7f48a00ae240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:56554/0 (socket says 172.21.15.59:56554) 2023-11-24T08:00:27.362 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:27.350+0000 7f489ffff700 1 -- 172.21.15.59:0/1682852611 learned_addr learned my addr 172.21.15.59:0/1682852611 (peer_addr_for_me v2:172.21.15.59:0/0) 2023-11-24T08:00:27.362 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:27.350+0000 7f48a488d700 1 --2- 172.21.15.59:0/1682852611 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f48a006b6e0 0x7f48a00b50a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:00:27.362 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:27.350+0000 7f48a508e700 1 --2- 172.21.15.59:0/1682852611 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f48a0145100 0x7f48a013d9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:00:27.362 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:27.350+0000 7f489ffff700 1 -- 172.21.15.59:0/1682852611 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f48a0145100 msgr2=0x7f48a013d9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:00:27.363 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:27.350+0000 7f489ffff700 1 --2- 172.21.15.59:0/1682852611 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f48a0145100 0x7f48a013d9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:00:27.363 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:27.350+0000 7f489ffff700 1 -- 172.21.15.59:0/1682852611 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f48a006b6e0 msgr2=0x7f48a00b50a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:00:27.363 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:27.350+0000 7f489ffff700 1 --2- 172.21.15.59:0/1682852611 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f48a006b6e0 0x7f48a00b50a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:00:27.363 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:27.350+0000 7f489ffff700 1 -- 172.21.15.59:0/1682852611 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f488c009550 con 0x7f48a0144c10 2023-11-24T08:00:27.363 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:27.350+0000 7f489ffff700 1 --2- 172.21.15.59:0/1682852611 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f48a0144c10 0x7f48a00ae240 secure :-1 s=READY pgs=1133 cs=0 l=1 rev1=1 crypto rx=0x7f488c009c40 tx=0x7f488c004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:00:27.364 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:27.351+0000 7f489d7fa700 1 -- 172.21.15.59:0/1682852611 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f488c004340 con 0x7f48a0144c10 2023-11-24T08:00:27.364 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:27.351+0000 7f489d7fa700 1 -- 172.21.15.59:0/1682852611 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f488c01f070 con 0x7f48a0144c10 2023-11-24T08:00:27.364 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:27.351+0000 7f48a6af1700 1 -- 172.21.15.59:0/1682852611 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f48a00ae9c0 con 0x7f48a0144c10 2023-11-24T08:00:27.364 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:27.351+0000 7f489d7fa700 1 -- 172.21.15.59:0/1682852611 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f488c0036a0 con 0x7f48a0144c10 2023-11-24T08:00:27.364 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:27.351+0000 7f48a6af1700 1 -- 172.21.15.59:0/1682852611 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f48a00aef80 con 0x7f48a0144c10 2023-11-24T08:00:27.365 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:27.352+0000 7f48a6af1700 1 -- 172.21.15.59:0/1682852611 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f48a0068670 con 0x7f48a0144c10 2023-11-24T08:00:27.365 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:27.353+0000 7f489d7fa700 1 -- 172.21.15.59:0/1682852611 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 12) v1 ==== 89792+0+0 (secure 0 0 0) 0x7f488c021030 con 0x7f48a0144c10 2023-11-24T08:00:27.365 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:27.353+0000 7f489d7fa700 1 --2- 172.21.15.59:0/1682852611 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f488406b500 0x7f488406d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:00:27.365 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:27.353+0000 7f489d7fa700 1 -- 172.21.15.59:0/1682852611 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f488c08b1c0 con 0x7f48a0144c10 2023-11-24T08:00:27.365 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:27.353+0000 7f48a488d700 1 --2- 172.21.15.59:0/1682852611 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f488406b500 0x7f488406d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:00:27.366 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:27.354+0000 7f48a488d700 1 --2- 172.21.15.59:0/1682852611 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f488406b500 0x7f488406d9c0 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7f48940018a0 tx=0x7f4894009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:00:27.366 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:27.358+0000 7f489d7fa700 1 -- 172.21.15.59:0/1682852611 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f488c055c40 con 0x7f48a0144c10 2023-11-24T08:00:27.556 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:27.554+0000 7f48a6af1700 1 -- 172.21.15.59:0/1682852611 --> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f48a01603b0 con 0x7f488406b500 2023-11-24T08:00:27.558 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:27.557+0000 7f489d7fa700 1 -- 172.21.15.59:0/1682852611 <== mgr.4100 v2:172.21.15.76:6832/4264046536 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+50856 (secure 0 0 0) 0x7f4894001540 con 0x7f488406b500 2023-11-24T08:00:27.559 INFO:teuthology.orchestra.run.smithi059.stdout: 2023-11-24T08:00:27.561 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:27.559+0000 7f48a6af1700 1 -- 172.21.15.59:0/1682852611 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f488406b500 msgr2=0x7f488406d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:00:27.561 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:27.559+0000 7f48a6af1700 1 --2- 172.21.15.59:0/1682852611 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f488406b500 0x7f488406d9c0 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7f48940018a0 tx=0x7f4894009040 comp rx=0 tx=0).stop 2023-11-24T08:00:27.561 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:27.559+0000 7f48a6af1700 1 -- 172.21.15.59:0/1682852611 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f48a0144c10 msgr2=0x7f48a00ae240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:00:27.561 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:27.559+0000 7f48a6af1700 1 --2- 172.21.15.59:0/1682852611 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f48a0144c10 0x7f48a00ae240 secure :-1 s=READY pgs=1133 cs=0 l=1 rev1=1 crypto rx=0x7f488c009c40 tx=0x7f488c004750 comp rx=0 tx=0).stop 2023-11-24T08:00:27.561 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:27.559+0000 7f48a6af1700 1 -- 172.21.15.59:0/1682852611 shutdown_connections 2023-11-24T08:00:27.562 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:27.559+0000 7f48a6af1700 1 --2- 172.21.15.59:0/1682852611 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f488406b500 0x7f488406d9c0 unknown :-1 s=CLOSED pgs=518 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:00:27.562 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:27.559+0000 7f48a6af1700 1 --2- 172.21.15.59:0/1682852611 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f48a0145100 0x7f48a013d9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:00:27.562 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:27.559+0000 7f48a6af1700 1 --2- 172.21.15.59:0/1682852611 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f48a006b6e0 0x7f48a00b50a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:00:27.562 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:27.559+0000 7f48a6af1700 1 --2- 172.21.15.59:0/1682852611 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f48a0144c10 0x7f48a00ae240 unknown :-1 s=CLOSED pgs=1133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:00:27.562 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:27.559+0000 7f48a6af1700 1 -- 172.21.15.59:0/1682852611 >> 172.21.15.59:0/1682852611 conn(0x7f48a00b2740 msgr2=0x7f48a0072300 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:00:27.562 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:27.559+0000 7f48a6af1700 1 -- 172.21.15.59:0/1682852611 shutdown_connections 2023-11-24T08:00:27.562 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:27.559+0000 7f48a6af1700 1 -- 172.21.15.59:0/1682852611 wait complete. 2023-11-24T08:00:27.563 INFO:teuthology.orchestra.run.smithi059.stderr:dumped all 2023-11-24T08:00:27.573 INFO:teuthology.orchestra.run.smithi059.stdout:{"pg_ready":true,"pg_map":{"version":207,"stamp":"2023-11-24T08:00:26.355476+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":230048,"kb_used_data":9728,"kb_used_omap":12,"kb_used_meta":220211,"kb_avail":754744672,"statfs":{"total":773094113280,"available":772858544128,"internally_reserved":0,"allocated":9961472,"data_stored":4958984,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12745,"internal_metadata":225496631},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-28340,"num_objects":-23,"num_object_clones":0,"num_object_copies":-46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-23,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":-61,"num_write_kb":-49,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001376"},"pg_stats":[{"pgid":"1.0","version":"16'117","reported_seq":224,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-11-24T08:00:21.312012+0000","last_change":"2023-11-24T07:56:19.668905+0000","last_active":"2023-11-24T08:00:21.312012+0000","last_peered":"2023-11-24T08:00:21.312012+0000","last_clean":"2023-11-24T08:00:21.312012+0000","last_became_active":"2023-11-24T07:56:19.668601+0000","last_became_peered":"2023-11-24T07:56:19.668601+0000","last_unstale":"2023-11-24T08:00:21.312012+0000","last_undegraded":"2023-11-24T08:00:21.312012+0000","last_fullsized":"2023-11-24T08:00:21.312012+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:56:18.657844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:56:18.657844+0000","last_clean_scrub_stamp":"2023-11-24T07:56:18.657844+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T11:51:53.828455+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574900,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28084,"kb_used_data":1064,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343756,"statfs":{"total":96636764160,"available":96608006144,"internally_reserved":0,"allocated":1089536,"data_stored":465400,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 08:00:21 2023","interfaces":[{"interface":"back","average":{"1min":0.66700000000000004,"5min":0.56200000000000006,"15min":0.55800000000000005},"min":{"1min":0.27700000000000002,"5min":0.27600000000000002,"15min":0.27600000000000002},"max":{"1min":3.1640000000000001,"5min":3.1640000000000001,"15min":3.1640000000000001},"last":0.51300000000000001},{"interface":"front","average":{"1min":0.51800000000000002,"5min":0.54400000000000004,"15min":0.55000000000000004},"min":{"1min":0.35499999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.67200000000000004,"5min":0.81000000000000005,"15min":0.81000000000000005},"last":0.40200000000000002}]},{"osd":1,"last update":"Fri Nov 24 08:00:21 2023","interfaces":[{"interface":"back","average":{"1min":0.50700000000000001,"5min":0.64400000000000002,"15min":0.78000000000000003},"min":{"1min":0.28799999999999998,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":0.83499999999999996,"5min":6.1070000000000002,"15min":6.1070000000000002},"last":0.58999999999999997},{"interface":"front","average":{"1min":0.68899999999999995,"5min":0.56899999999999995,"15min":0.57599999999999996},"min":{"1min":0.38400000000000001,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":3.1970000000000001,"5min":3.1970000000000001,"15min":3.1970000000000001},"last":0.55800000000000005}]},{"osd":2,"last update":"Fri Nov 24 08:00:21 2023","interfaces":[{"interface":"back","average":{"1min":0.53500000000000003,"5min":0.56200000000000006,"15min":0.60499999999999998},"min":{"1min":0.35599999999999998,"5min":0.308,"15min":0.308},"max":{"1min":0.94099999999999995,"5min":1.613,"15min":1.613},"last":0.61399999999999999},{"interface":"front","average":{"1min":0.51800000000000002,"5min":0.61699999999999999,"15min":0.73299999999999998},"min":{"1min":0.35199999999999998,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.65400000000000003,"5min":4.5220000000000002,"15min":4.5220000000000002},"last":0.45700000000000002}]},{"osd":3,"last update":"Fri Nov 24 08:00:21 2023","interfaces":[{"interface":"back","average":{"1min":0.56399999999999995,"5min":0.65800000000000003,"15min":0.75600000000000001},"min":{"1min":0.42599999999999999,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.88500000000000001,"5min":4.6760000000000002,"15min":4.6760000000000002},"last":0.54000000000000004},{"interface":"front","average":{"1min":0.64500000000000002,"5min":0.748,"15min":0.878},"min":{"1min":0.51200000000000001,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":1.1299999999999999,"5min":4.6539999999999999,"15min":4.6539999999999999},"last":0.68700000000000006}]},{"osd":4,"last update":"Fri Nov 24 08:00:21 2023","interfaces":[{"interface":"back","average":{"1min":0.53300000000000003,"5min":0.53800000000000003,"15min":0.55800000000000005},"min":{"1min":0.33600000000000002,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":1.0760000000000001,"5min":1.0760000000000001,"15min":1.0760000000000001},"last":0.42599999999999999},{"interface":"front","average":{"1min":0.59299999999999997,"5min":0.60499999999999998,"15min":0.64600000000000002},"min":{"1min":0.39900000000000002,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":1.284,"5min":1.629,"15min":1.629},"last":0.71999999999999997}]},{"osd":5,"last update":"Fri Nov 24 08:00:21 2023","interfaces":[{"interface":"back","average":{"1min":0.51100000000000001,"5min":0.52900000000000003,"15min":0.54100000000000004},"min":{"1min":0.36399999999999999,"5min":0.315,"15min":0.315},"max":{"1min":0.77200000000000002,"5min":0.875,"15min":0.875},"last":0.49299999999999999},{"interface":"front","average":{"1min":0.626,"5min":0.60299999999999998,"15min":0.61599999999999999},"min":{"1min":0.47499999999999998,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":1.0429999999999999,"5min":1.0429999999999999,"15min":1.0429999999999999},"last":0.66700000000000004}]},{"osd":6,"last update":"Fri Nov 24 08:00:21 2023","interfaces":[{"interface":"back","average":{"1min":0.55000000000000004,"5min":0.58399999999999996,"15min":0.60199999999999998},"min":{"1min":0.38600000000000001,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.745,"5min":0.85799999999999998,"15min":0.85799999999999998},"last":0.64600000000000002},{"interface":"front","average":{"1min":0.61199999999999999,"5min":0.64000000000000001,"15min":0.66000000000000003},"min":{"1min":0.438,"5min":0.438,"15min":0.438},"max":{"1min":0.91700000000000004,"5min":0.99099999999999999,"15min":0.99099999999999999},"last":0.47499999999999998}]}]},{"osd":6,"up_from":13,"seq":55834574901,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28084,"kb_used_data":1064,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343756,"statfs":{"total":96636764160,"available":96608006144,"internally_reserved":0,"allocated":1089536,"data_stored":465868,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 07:59:19 2023","interfaces":[{"interface":"back","average":{"1min":0.51000000000000001,"5min":0.53800000000000003,"15min":0.54600000000000004},"min":{"1min":0.33500000000000002,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":0.82999999999999996,"5min":0.93100000000000005,"15min":0.93100000000000005},"last":0.55900000000000005},{"interface":"front","average":{"1min":0.53100000000000003,"5min":0.51900000000000002,"15min":0.51700000000000002},"min":{"1min":0.34000000000000002,"5min":0.311,"15min":0.311},"max":{"1min":0.72299999999999998,"5min":0.82199999999999995,"15min":0.82199999999999995},"last":0.45800000000000002}]},{"osd":1,"last update":"Fri Nov 24 07:59:19 2023","interfaces":[{"interface":"back","average":{"1min":0.499,"5min":0.51000000000000001,"15min":0.51100000000000001},"min":{"1min":0.312,"5min":0.312,"15min":0.312},"max":{"1min":0.746,"5min":0.81299999999999994,"15min":0.81299999999999994},"last":0.47099999999999997},{"interface":"front","average":{"1min":0.55900000000000005,"5min":0.58699999999999997,"15min":0.60399999999999998},"min":{"1min":0.29999999999999999,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":0.70799999999999996,"5min":1.0069999999999999,"15min":1.0069999999999999},"last":0.66200000000000003}]},{"osd":2,"last update":"Fri Nov 24 07:59:19 2023","interfaces":[{"interface":"back","average":{"1min":0.57699999999999996,"5min":0.57699999999999996,"15min":0.57499999999999996},"min":{"1min":0.41399999999999998,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.71099999999999997,"5min":0.90600000000000003,"15min":0.90600000000000003},"last":0.52000000000000002},{"interface":"front","average":{"1min":0.59899999999999998,"5min":0.58799999999999997,"15min":0.59099999999999997},"min":{"1min":0.52500000000000002,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.78800000000000003,"5min":0.84899999999999998,"15min":0.84899999999999998},"last":0.63400000000000001}]},{"osd":3,"last update":"Fri Nov 24 07:59:19 2023","interfaces":[{"interface":"back","average":{"1min":0.56200000000000006,"5min":0.56499999999999995,"15min":0.56699999999999995},"min":{"1min":0.38700000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.75700000000000001,"5min":0.876,"15min":0.876},"last":0.40999999999999998},{"interface":"front","average":{"1min":0.55700000000000005,"5min":0.56000000000000005,"15min":0.55700000000000005},"min":{"1min":0.40500000000000003,"5min":0.379,"15min":0.379},"max":{"1min":0.72599999999999998,"5min":0.876,"15min":0.876},"last":0.622}]},{"osd":4,"last update":"Fri Nov 24 07:59:19 2023","interfaces":[{"interface":"back","average":{"1min":0.55300000000000005,"5min":0.52700000000000002,"15min":0.52000000000000002},"min":{"1min":0.36299999999999999,"5min":0.25,"15min":0.25},"max":{"1min":0.80700000000000005,"5min":0.89400000000000002,"15min":0.89400000000000002},"last":0.505},{"interface":"front","average":{"1min":0.53100000000000003,"5min":0.58499999999999996,"15min":0.59599999999999997},"min":{"1min":0.30599999999999999,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":0.81899999999999995,"5min":1.087,"15min":1.087},"last":0.60399999999999998}]},{"osd":5,"last update":"Fri Nov 24 07:59:19 2023","interfaces":[{"interface":"back","average":{"1min":0.57099999999999995,"5min":0.55100000000000005,"15min":0.53700000000000003},"min":{"1min":0.33300000000000002,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":0.76200000000000001,"5min":0.97999999999999998,"15min":0.97999999999999998},"last":0.58299999999999996},{"interface":"front","average":{"1min":0.62,"5min":0.61399999999999999,"15min":0.61199999999999999},"min":{"1min":0.438,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.997,"5min":1.0009999999999999,"15min":1.0009999999999999},"last":0.434}]},{"osd":7,"last update":"Fri Nov 24 07:59:19 2023","interfaces":[{"interface":"back","average":{"1min":0.59899999999999998,"5min":0.61399999999999999,"15min":0.61599999999999999},"min":{"1min":0.44800000000000001,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":0.85499999999999998,"5min":1.0880000000000001,"15min":1.0880000000000001},"last":0.53600000000000003},{"interface":"front","average":{"1min":0.628,"5min":0.65800000000000003,"15min":0.66600000000000004},"min":{"1min":0.32800000000000001,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.79500000000000004,"5min":0.95599999999999996,"15min":0.95599999999999996},"last":0.65100000000000002}]}]},{"osd":1,"up_from":13,"seq":55834574902,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28660,"kb_used_data":1640,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343180,"statfs":{"total":96636764160,"available":96607416320,"internally_reserved":0,"allocated":1679360,"data_stored":1055654,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 07:59:24 2023","interfaces":[{"interface":"back","average":{"1min":0.35499999999999998,"5min":0.35499999999999998,"15min":0.36099999999999999},"min":{"1min":0.248,"5min":0.20200000000000001,"15min":0.20200000000000001},"max":{"1min":0.52600000000000002,"5min":0.70899999999999996,"15min":0.70899999999999996},"last":0.313},{"interface":"front","average":{"1min":0.36299999999999999,"5min":0.38400000000000001,"15min":0.39300000000000002},"min":{"1min":0.23300000000000001,"5min":0.214,"15min":0.214},"max":{"1min":0.65600000000000003,"5min":0.82499999999999996,"15min":0.82499999999999996},"last":0.33600000000000002}]},{"osd":2,"last update":"Fri Nov 24 07:59:24 2023","interfaces":[{"interface":"back","average":{"1min":0.36899999999999999,"5min":0.35999999999999999,"15min":0.36799999999999999},"min":{"1min":0.27100000000000002,"5min":0.22700000000000001,"15min":0.22700000000000001},"max":{"1min":0.53000000000000003,"5min":0.60999999999999999,"15min":0.60999999999999999},"last":0.26600000000000001},{"interface":"front","average":{"1min":0.42299999999999999,"5min":0.40500000000000003,"15min":0.40999999999999998},"min":{"1min":0.28899999999999998,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":0.63700000000000001,"5min":0.63700000000000001,"15min":0.63700000000000001},"last":0.376}]},{"osd":3,"last update":"Fri Nov 24 07:59:24 2023","interfaces":[{"interface":"back","average":{"1min":0.43099999999999999,"5min":0.433,"15min":0.44},"min":{"1min":0.317,"5min":0.317,"15min":0.317},"max":{"1min":0.62,"5min":0.66500000000000004,"15min":0.66500000000000004},"last":0.35199999999999998},{"interface":"front","average":{"1min":0.41799999999999998,"5min":0.41699999999999998,"15min":0.42399999999999999},"min":{"1min":0.32900000000000001,"5min":0.28499999999999998,"15min":0.28499999999999998},"max":{"1min":0.59999999999999998,"5min":0.76300000000000001,"15min":0.76300000000000001},"last":0.29499999999999998}]},{"osd":4,"last update":"Fri Nov 24 07:59:24 2023","interfaces":[{"interface":"back","average":{"1min":0.625,"5min":0.60399999999999998,"15min":0.60799999999999998},"min":{"1min":0.434,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":1.5580000000000001,"5min":1.5580000000000001,"15min":1.5580000000000001},"last":0.54100000000000004},{"interface":"front","average":{"1min":0.65200000000000002,"5min":0.61599999999999999,"15min":0.60899999999999999},"min":{"1min":0.48599999999999999,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":0.82399999999999995,"5min":0.82399999999999995,"15min":0.82399999999999995},"last":0.66200000000000003}]},{"osd":5,"last update":"Fri Nov 24 07:59:24 2023","interfaces":[{"interface":"back","average":{"1min":0.625,"5min":0.67800000000000005,"15min":0.70899999999999996},"min":{"1min":0.44400000000000001,"5min":0.37,"15min":0.37},"max":{"1min":0.755,"5min":2.7269999999999999,"15min":2.7269999999999999},"last":0.60699999999999998},{"interface":"front","average":{"1min":0.68500000000000005,"5min":0.63900000000000001,"15min":0.64300000000000002},"min":{"1min":0.434,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":1.6459999999999999,"5min":1.6459999999999999,"15min":1.6459999999999999},"last":0.65100000000000002}]},{"osd":6,"last update":"Fri Nov 24 07:59:24 2023","interfaces":[{"interface":"back","average":{"1min":0.63200000000000001,"5min":0.64300000000000002,"15min":0.65300000000000002},"min":{"1min":0.46000000000000002,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":0.93799999999999994,"5min":0.98199999999999998,"15min":0.98199999999999998},"last":0.56000000000000005},{"interface":"front","average":{"1min":0.67900000000000005,"5min":0.621,"15min":0.61799999999999999},"min":{"1min":0.46100000000000002,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":0.91600000000000004,"5min":0.91600000000000004,"15min":0.91600000000000004},"last":0.67200000000000004}]},{"osd":7,"last update":"Fri Nov 24 07:59:24 2023","interfaces":[{"interface":"back","average":{"1min":0.69699999999999995,"5min":0.64700000000000002,"15min":0.64000000000000001},"min":{"1min":0.53900000000000003,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.95599999999999996,"5min":1.0649999999999999,"15min":1.0649999999999999},"last":0.57699999999999996},{"interface":"front","average":{"1min":0.64300000000000002,"5min":0.64400000000000002,"15min":0.66100000000000003},"min":{"1min":0.45900000000000002,"5min":0.442,"15min":0.442},"max":{"1min":0.93300000000000005,"5min":0.93300000000000005,"15min":0.93300000000000005},"last":0.68600000000000005}]}]},{"osd":0,"up_from":13,"seq":55834574902,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28116,"kb_used_data":1096,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343724,"statfs":{"total":96636764160,"available":96607973376,"internally_reserved":0,"allocated":1122304,"data_stored":492626,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Nov 24 07:59:22 2023","interfaces":[{"interface":"back","average":{"1min":0.36899999999999999,"5min":0.57799999999999996,"15min":0.66800000000000004},"min":{"1min":0.247,"5min":0.21299999999999999,"15min":0.21299999999999999},"max":{"1min":0.56100000000000005,"5min":6.3550000000000004,"15min":6.3550000000000004},"last":0.377},{"interface":"front","average":{"1min":0.39300000000000002,"5min":1.1970000000000001,"15min":1.5509999999999999},"min":{"1min":0.29399999999999998,"5min":0.26000000000000001,"15min":0.26000000000000001},"max":{"1min":0.59099999999999997,"5min":22.916,"15min":22.916},"last":0.40100000000000002}]},{"osd":2,"last update":"Fri Nov 24 07:59:22 2023","interfaces":[{"interface":"back","average":{"1min":0.35999999999999999,"5min":0.56200000000000006,"15min":0.65400000000000003},"min":{"1min":0.23400000000000001,"5min":0.22800000000000001,"15min":0.22800000000000001},"max":{"1min":0.54100000000000004,"5min":6.3899999999999997,"15min":6.3899999999999997},"last":0.29599999999999999},{"interface":"front","average":{"1min":0.32300000000000001,"5min":0.78100000000000003,"15min":0.96899999999999997},"min":{"1min":0.253,"5min":0.253,"15min":0.253},"max":{"1min":0.497,"5min":11.93,"15min":11.93},"last":0.35599999999999998}]},{"osd":3,"last update":"Fri Nov 24 07:59:22 2023","interfaces":[{"interface":"back","average":{"1min":0.373,"5min":0.40100000000000002,"15min":0.41299999999999998},"min":{"1min":0.247,"5min":0.23799999999999999,"15min":0.23799999999999999},"max":{"1min":0.52500000000000002,"5min":1.155,"15min":1.155},"last":0.317},{"interface":"front","average":{"1min":0.40200000000000002,"5min":0.84399999999999997,"15min":1.032},"min":{"1min":0.27400000000000002,"5min":0.27400000000000002,"15min":0.27400000000000002},"max":{"1min":0.63400000000000001,"5min":12.116,"15min":12.116},"last":0.41599999999999998}]},{"osd":4,"last update":"Fri Nov 24 07:59:22 2023","interfaces":[{"interface":"back","average":{"1min":0.59799999999999998,"5min":0.63300000000000001,"15min":0.64300000000000002},"min":{"1min":0.44400000000000001,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.93500000000000005,"5min":1.044,"15min":1.044},"last":0.63700000000000001},{"interface":"front","average":{"1min":0.58499999999999996,"5min":1.4059999999999999,"15min":1.758},"min":{"1min":0.377,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.84999999999999998,"5min":22.963999999999999,"15min":22.963999999999999},"last":0.55200000000000005}]},{"osd":5,"last update":"Fri Nov 24 07:59:22 2023","interfaces":[{"interface":"back","average":{"1min":0.59999999999999998,"5min":1.026,"15min":1.204},"min":{"1min":0.41199999999999998,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.86299999999999999,"5min":12.084,"15min":12.084},"last":0.66100000000000003},{"interface":"front","average":{"1min":0.61899999999999999,"5min":0.66200000000000003,"15min":0.67200000000000004},"min":{"1min":0.46600000000000003,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.89100000000000001,"5min":1.073,"15min":1.073},"last":0.56699999999999995}]},{"osd":6,"last update":"Fri Nov 24 07:59:22 2023","interfaces":[{"interface":"back","average":{"1min":0.61599999999999999,"5min":0.66800000000000004,"15min":0.68300000000000005},"min":{"1min":0.36799999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.92400000000000004,"5min":1.0580000000000001,"15min":1.0580000000000001},"last":0.53900000000000003},{"interface":"front","average":{"1min":0.60099999999999998,"5min":1.042,"15min":1.2230000000000001},"min":{"1min":0.46000000000000002,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":0.81999999999999995,"5min":12.045,"15min":12.045},"last":0.62}]},{"osd":7,"last update":"Fri Nov 24 07:59:22 2023","interfaces":[{"interface":"back","average":{"1min":0.57699999999999996,"5min":0.64300000000000002,"15min":0.65100000000000002},"min":{"1min":0.45700000000000002,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":0.82999999999999996,"5min":1.038,"15min":1.038},"last":0.60899999999999999},{"interface":"front","average":{"1min":0.59499999999999997,"5min":1.431,"15min":1.7849999999999999},"min":{"1min":0.42299999999999999,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":0.80300000000000005,"5min":22.995000000000001,"15min":22.995000000000001},"last":0.67300000000000004}]}]},{"osd":2,"up_from":13,"seq":55834574902,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28080,"kb_used_data":1060,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343760,"statfs":{"total":96636764160,"available":96608010240,"internally_reserved":0,"allocated":1085440,"data_stored":465330,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 08:00:21 2023","interfaces":[{"interface":"back","average":{"1min":0.34499999999999997,"5min":0.35399999999999998,"15min":0.36499999999999999},"min":{"1min":0.22500000000000001,"5min":0.219,"15min":0.219},"max":{"1min":0.499,"5min":0.58099999999999996,"15min":0.58099999999999996},"last":0.47199999999999998},{"interface":"front","average":{"1min":0.41299999999999998,"5min":0.39600000000000002,"15min":0.40000000000000002},"min":{"1min":0.28899999999999998,"5min":0.27400000000000002,"15min":0.27400000000000002},"max":{"1min":0.69099999999999995,"5min":0.69099999999999995,"15min":0.69099999999999995},"last":0.442}]},{"osd":1,"last update":"Fri Nov 24 08:00:21 2023","interfaces":[{"interface":"back","average":{"1min":0.34999999999999998,"5min":0.35499999999999998,"15min":0.37},"min":{"1min":0.249,"5min":0.21099999999999999,"15min":0.21099999999999999},"max":{"1min":0.55700000000000005,"5min":0.59199999999999997,"15min":0.59199999999999997},"last":0.40100000000000002},{"interface":"front","average":{"1min":0.379,"5min":0.38800000000000001,"15min":0.40600000000000003},"min":{"1min":0.24399999999999999,"5min":0.24399999999999999,"15min":0.24399999999999999},"max":{"1min":0.55100000000000005,"5min":0.627,"15min":0.627},"last":0.54700000000000004}]},{"osd":3,"last update":"Fri Nov 24 08:00:21 2023","interfaces":[{"interface":"back","average":{"1min":0.42499999999999999,"5min":0.40400000000000003,"15min":0.40799999999999997},"min":{"1min":0.28699999999999998,"5min":0.26400000000000001,"15min":0.26400000000000001},"max":{"1min":0.63500000000000001,"5min":0.64200000000000002,"15min":0.64200000000000002},"last":0.499},{"interface":"front","average":{"1min":0.39000000000000001,"5min":0.40000000000000002,"15min":0.40899999999999997},"min":{"1min":0.315,"5min":0.27500000000000002,"15min":0.27500000000000002},"max":{"1min":0.58299999999999996,"5min":0.64600000000000002,"15min":0.64600000000000002},"last":0.33100000000000002}]},{"osd":4,"last update":"Fri Nov 24 08:00:21 2023","interfaces":[{"interface":"back","average":{"1min":0.58699999999999997,"5min":0.57099999999999995,"15min":0.59499999999999997},"min":{"1min":0.32900000000000001,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.91000000000000003,"5min":0.91000000000000003,"15min":0.91000000000000003},"last":0.91000000000000003},{"interface":"front","average":{"1min":0.57099999999999995,"5min":0.57699999999999996,"15min":0.59199999999999997},"min":{"1min":0.35299999999999998,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.75600000000000001,"5min":0.83799999999999997,"15min":0.83799999999999997},"last":0.65200000000000002}]},{"osd":5,"last update":"Fri Nov 24 08:00:21 2023","interfaces":[{"interface":"back","average":{"1min":0.57899999999999996,"5min":0.58999999999999997,"15min":0.60999999999999999},"min":{"1min":0.39100000000000001,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.84099999999999997,"5min":0.84899999999999998,"15min":0.84899999999999998},"last":0.68100000000000005},{"interface":"front","average":{"1min":0.64500000000000002,"5min":0.64600000000000002,"15min":0.69199999999999995},"min":{"1min":0.45100000000000001,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":1.119,"5min":2.1699999999999999,"15min":2.1699999999999999},"last":0.83699999999999997}]},{"osd":6,"last update":"Fri Nov 24 08:00:21 2023","interfaces":[{"interface":"back","average":{"1min":0.64000000000000001,"5min":0.625,"15min":0.65300000000000002},"min":{"1min":0.41999999999999998,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":0.88300000000000001,"5min":0.88300000000000001,"15min":0.88300000000000001},"last":0.88300000000000001},{"interface":"front","average":{"1min":0.65400000000000003,"5min":0.629,"15min":0.65700000000000003},"min":{"1min":0.44500000000000001,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":1.105,"5min":1.105,"15min":1.105},"last":0.80600000000000005}]},{"osd":7,"last update":"Fri Nov 24 08:00:21 2023","interfaces":[{"interface":"back","average":{"1min":0.628,"5min":0.626,"15min":0.65000000000000002},"min":{"1min":0.42199999999999999,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":0.93000000000000005,"5min":0.93000000000000005,"15min":0.93000000000000005},"last":0.70899999999999996},{"interface":"front","average":{"1min":0.627,"5min":0.63600000000000001,"15min":0.66500000000000004},"min":{"1min":0.46200000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.80000000000000004,"5min":0.90000000000000002,"15min":0.90000000000000002},"last":0.77400000000000002}]}]},{"osd":3,"up_from":13,"seq":55834574902,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28088,"kb_used_data":1068,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343752,"statfs":{"total":96636764160,"available":96608002048,"internally_reserved":0,"allocated":1093632,"data_stored":465870,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 07:59:27 2023","interfaces":[{"interface":"back","average":{"1min":0.33600000000000002,"5min":0.34899999999999998,"15min":0.35299999999999998},"min":{"1min":0.183,"5min":0.183,"15min":0.183},"max":{"1min":0.53400000000000003,"5min":0.84799999999999998,"15min":0.84799999999999998},"last":0.30299999999999999},{"interface":"front","average":{"1min":0.34599999999999997,"5min":0.35399999999999998,"15min":0.36199999999999999},"min":{"1min":0.24299999999999999,"5min":0.23699999999999999,"15min":0.23699999999999999},"max":{"1min":0.51000000000000001,"5min":0.873,"15min":0.873},"last":0.32000000000000001}]},{"osd":1,"last update":"Fri Nov 24 07:59:27 2023","interfaces":[{"interface":"back","average":{"1min":0.38400000000000001,"5min":0.373,"15min":0.374},"min":{"1min":0.217,"5min":0.217,"15min":0.217},"max":{"1min":0.66300000000000003,"5min":0.92000000000000004,"15min":0.92000000000000004},"last":0.26600000000000001},{"interface":"front","average":{"1min":0.38,"5min":0.377,"15min":0.38300000000000001},"min":{"1min":0.28599999999999998,"5min":0.248,"15min":0.248},"max":{"1min":0.626,"5min":0.81699999999999995,"15min":0.81699999999999995},"last":0.34899999999999998}]},{"osd":2,"last update":"Fri Nov 24 07:59:27 2023","interfaces":[{"interface":"back","average":{"1min":0.35599999999999998,"5min":0.35399999999999998,"15min":0.35799999999999998},"min":{"1min":0.253,"5min":0.24399999999999999,"15min":0.24399999999999999},"max":{"1min":0.496,"5min":0.88500000000000001,"15min":0.88500000000000001},"last":0.36599999999999999},{"interface":"front","average":{"1min":0.40999999999999998,"5min":0.441,"15min":0.45900000000000002},"min":{"1min":0.315,"5min":0.311,"15min":0.311},"max":{"1min":0.59799999999999998,"5min":0.98099999999999998,"15min":0.98099999999999998},"last":0.33600000000000002}]},{"osd":4,"last update":"Fri Nov 24 07:59:27 2023","interfaces":[{"interface":"back","average":{"1min":0.56299999999999994,"5min":0.70499999999999996,"15min":0.76600000000000001},"min":{"1min":0.38900000000000001,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.73599999999999999,"5min":5.7350000000000003,"15min":5.7350000000000003},"last":0.51000000000000001},{"interface":"front","average":{"1min":0.55700000000000005,"5min":0.58099999999999996,"15min":0.58299999999999996},"min":{"1min":0.34399999999999997,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.80400000000000005,"5min":1.27,"15min":1.27},"last":0.41299999999999998}]},{"osd":5,"last update":"Fri Nov 24 07:59:27 2023","interfaces":[{"interface":"back","average":{"1min":0.56100000000000005,"5min":0.57799999999999996,"15min":0.58399999999999996},"min":{"1min":0.42899999999999999,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":0.67900000000000005,"5min":1.4430000000000001,"15min":1.4430000000000001},"last":0.47999999999999998},{"interface":"front","average":{"1min":0.58799999999999997,"5min":0.61499999999999999,"15min":0.623},"min":{"1min":0.40999999999999998,"5min":0.377,"15min":0.377},"max":{"1min":0.78500000000000003,"5min":1.4239999999999999,"15min":1.4239999999999999},"last":0.434}]},{"osd":6,"last update":"Fri Nov 24 07:59:27 2023","interfaces":[{"interface":"back","average":{"1min":0.58799999999999997,"5min":0.60699999999999998,"15min":0.61499999999999999},"min":{"1min":0.42599999999999999,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":0.84499999999999997,"5min":1.355,"15min":1.355},"last":0.49399999999999999},{"interface":"front","average":{"1min":0.57899999999999996,"5min":0.745,"15min":0.81200000000000006},"min":{"1min":0.44800000000000001,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.77100000000000002,"5min":5.6479999999999997,"15min":5.6479999999999997},"last":0.42399999999999999}]},{"osd":7,"last update":"Fri Nov 24 07:59:27 2023","interfaces":[{"interface":"back","average":{"1min":0.61799999999999999,"5min":0.64100000000000001,"15min":0.65700000000000003},"min":{"1min":0.499,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":0.82799999999999996,"5min":1.51,"15min":1.51},"last":0.53400000000000003},{"interface":"front","average":{"1min":0.57699999999999996,"5min":0.74199999999999999,"15min":0.80800000000000005},"min":{"1min":0.40100000000000002,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.79500000000000004,"5min":5.7069999999999999,"15min":5.7069999999999999},"last":0.44700000000000001}]}]},{"osd":4,"up_from":13,"seq":55834574901,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28080,"kb_used_data":1060,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343760,"statfs":{"total":96636764160,"available":96608010240,"internally_reserved":0,"allocated":1085440,"data_stored":465230,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 07:59:28 2023","interfaces":[{"interface":"back","average":{"1min":0.55900000000000005,"5min":0.56100000000000005,"15min":0.57399999999999995},"min":{"1min":0.32400000000000001,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.93300000000000005,"5min":0.93300000000000005,"15min":0.93300000000000005},"last":0.63700000000000001},{"interface":"front","average":{"1min":0.51100000000000001,"5min":0.52800000000000002,"15min":0.53800000000000003},"min":{"1min":0.36199999999999999,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.69299999999999995,"5min":1.0249999999999999,"15min":1.0249999999999999},"last":0.72099999999999997}]},{"osd":1,"last update":"Fri Nov 24 07:59:28 2023","interfaces":[{"interface":"back","average":{"1min":0.54800000000000004,"5min":0.54600000000000004,"15min":0.55000000000000004},"min":{"1min":0.35899999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.71599999999999997,"5min":0.86399999999999999,"15min":0.86399999999999999},"last":0.68500000000000005},{"interface":"front","average":{"1min":0.51600000000000001,"5min":0.53700000000000003,"15min":0.54600000000000004},"min":{"1min":0.36699999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.72099999999999997,"5min":0.98699999999999999,"15min":0.98699999999999999},"last":0.70299999999999996}]},{"osd":2,"last update":"Fri Nov 24 07:59:28 2023","interfaces":[{"interface":"back","average":{"1min":0.54200000000000004,"5min":0.54900000000000004,"15min":0.55700000000000005},"min":{"1min":0.34499999999999997,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.86399999999999999,"5min":0.86399999999999999,"15min":0.86399999999999999},"last":0.59799999999999998},{"interface":"front","average":{"1min":0.56200000000000006,"5min":0.59399999999999997,"15min":0.60599999999999998},"min":{"1min":0.34000000000000002,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.81899999999999995,"5min":0.89200000000000002,"15min":0.89200000000000002},"last":0.80700000000000005}]},{"osd":3,"last update":"Fri Nov 24 07:59:28 2023","interfaces":[{"interface":"back","average":{"1min":0.53600000000000003,"5min":0.56799999999999995,"15min":0.57899999999999996},"min":{"1min":0.38800000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.78100000000000003,"5min":0.89700000000000002,"15min":0.89700000000000002},"last":0.73799999999999999},{"interface":"front","average":{"1min":0.58899999999999997,"5min":0.61099999999999999,"15min":0.624},"min":{"1min":0.42699999999999999,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.73999999999999999,"5min":0.96199999999999997,"15min":0.96199999999999997},"last":0.83899999999999997}]},{"osd":5,"last update":"Fri Nov 24 07:59:28 2023","interfaces":[{"interface":"back","average":{"1min":0.46800000000000003,"5min":0.46300000000000002,"15min":0.45700000000000002},"min":{"1min":0.34999999999999998,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":0.60499999999999998,"5min":0.70499999999999996,"15min":0.70499999999999996},"last":0.61899999999999999},{"interface":"front","average":{"1min":0.56599999999999995,"5min":0.60399999999999998,"15min":0.61399999999999999},"min":{"1min":0.41599999999999998,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.73699999999999999,"5min":0.86399999999999999,"15min":0.86399999999999999},"last":0.79100000000000004}]},{"osd":6,"last update":"Fri Nov 24 07:59:28 2023","interfaces":[{"interface":"back","average":{"1min":0.53900000000000003,"5min":0.57299999999999995,"15min":0.58099999999999996},"min":{"1min":0.33000000000000002,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.88600000000000001,"5min":0.88600000000000001,"15min":0.88600000000000001},"last":0.56100000000000005},{"interface":"front","average":{"1min":0.56899999999999995,"5min":0.59999999999999998,"15min":0.60899999999999999},"min":{"1min":0.32100000000000001,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.80300000000000005,"5min":1.095,"15min":1.095},"last":0.77200000000000002}]},{"osd":7,"last update":"Fri Nov 24 07:59:28 2023","interfaces":[{"interface":"back","average":{"1min":0.52000000000000002,"5min":0.56100000000000005,"15min":0.56899999999999995},"min":{"1min":0.36299999999999999,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.69999999999999996,"5min":0.89300000000000002,"15min":0.89300000000000002},"last":0.67000000000000004},{"interface":"front","average":{"1min":0.57799999999999996,"5min":0.63600000000000001,"15min":0.65400000000000003},"min":{"1min":0.39000000000000001,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.74399999999999999,"5min":0.91500000000000004,"15min":0.91500000000000004},"last":0.75600000000000001}]}]},{"osd":5,"up_from":13,"seq":55834574902,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32856,"kb_used_data":1676,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338984,"statfs":{"total":96636764160,"available":96603119616,"internally_reserved":0,"allocated":1716224,"data_stored":1083006,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 07:59:21 2023","interfaces":[{"interface":"back","average":{"1min":0.52500000000000002,"5min":0.54600000000000004,"15min":0.55600000000000005},"min":{"1min":0.311,"5min":0.311,"15min":0.311},"max":{"1min":0.74399999999999999,"5min":1.0109999999999999,"15min":1.0109999999999999},"last":0.40200000000000002},{"interface":"front","average":{"1min":0.53600000000000003,"5min":0.69299999999999995,"15min":0.76400000000000001},"min":{"1min":0.32400000000000001,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.81899999999999995,"5min":5.431,"15min":5.431},"last":0.504}]},{"osd":1,"last update":"Fri Nov 24 07:59:21 2023","interfaces":[{"interface":"back","average":{"1min":0.52600000000000002,"5min":0.54200000000000004,"15min":0.55300000000000005},"min":{"1min":0.35399999999999998,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":0.69099999999999995,"5min":1.016,"15min":1.016},"last":0.32400000000000001},{"interface":"front","average":{"1min":0.56399999999999995,"5min":0.56399999999999995,"15min":0.56799999999999995},"min":{"1min":0.34000000000000002,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":0.80100000000000005,"5min":0.96499999999999997,"15min":0.96499999999999997},"last":0.47899999999999998}]},{"osd":2,"last update":"Fri Nov 24 07:59:21 2023","interfaces":[{"interface":"back","average":{"1min":0.58099999999999996,"5min":0.55500000000000005,"15min":0.55200000000000005},"min":{"1min":0.44800000000000001,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.876,"5min":1.163,"15min":1.163},"last":0.35599999999999998},{"interface":"front","average":{"1min":0.55900000000000005,"5min":0.59799999999999998,"15min":0.61899999999999999},"min":{"1min":0.32600000000000001,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.72399999999999998,"5min":1.119,"15min":1.119},"last":0.53900000000000003}]},{"osd":3,"last update":"Fri Nov 24 07:59:21 2023","interfaces":[{"interface":"back","average":{"1min":0.59799999999999998,"5min":0.77700000000000002,"15min":0.86199999999999999},"min":{"1min":0.32400000000000001,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.77100000000000002,"5min":5.609,"15min":5.609},"last":0.52700000000000002},{"interface":"front","average":{"1min":0.64300000000000002,"5min":0.66000000000000003,"15min":0.67000000000000004},"min":{"1min":0.38900000000000001,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.84999999999999998,"5min":1.288,"15min":1.288},"last":0.56699999999999995}]},{"osd":4,"last update":"Fri Nov 24 07:59:21 2023","interfaces":[{"interface":"back","average":{"1min":0.53800000000000003,"5min":0.51600000000000001,"15min":0.51200000000000001},"min":{"1min":0.30599999999999999,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":0.79100000000000004,"5min":0.79100000000000004,"15min":0.79100000000000004},"last":0.45900000000000002},{"interface":"front","average":{"1min":0.58899999999999997,"5min":0.57799999999999996,"15min":0.57399999999999995},"min":{"1min":0.35399999999999998,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":0.82599999999999996,"5min":0.92900000000000005,"15min":0.92900000000000005},"last":0.55400000000000005}]},{"osd":6,"last update":"Fri Nov 24 07:59:21 2023","interfaces":[{"interface":"back","average":{"1min":0.63,"5min":0.58699999999999997,"15min":0.57599999999999996},"min":{"1min":0.434,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.85399999999999998,"5min":0.89700000000000002,"15min":0.89700000000000002},"last":0.627},{"interface":"front","average":{"1min":0.59299999999999997,"5min":0.621,"15min":0.623},"min":{"1min":0.376,"5min":0.376,"15min":0.376},"max":{"1min":0.74299999999999999,"5min":1.083,"15min":1.083},"last":0.59199999999999997}]},{"osd":7,"last update":"Fri Nov 24 07:59:21 2023","interfaces":[{"interface":"back","average":{"1min":0.65400000000000003,"5min":0.65000000000000002,"15min":0.64700000000000002},"min":{"1min":0.40699999999999997,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.86099999999999999,"5min":0.96599999999999997,"15min":0.96599999999999997},"last":0.58199999999999996},{"interface":"front","average":{"1min":0.68000000000000005,"5min":0.66100000000000003,"15min":0.65500000000000003},"min":{"1min":0.49099999999999999,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":0.78500000000000003,"5min":0.94199999999999995,"15min":0.94199999999999995},"last":0.60399999999999998}]}]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-11-24T08:00:27.574 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-24T08:00:27.699 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:27.698+0000 7f20c831a700 1 Processor -- start 2023-11-24T08:00:27.700 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:27.698+0000 7f20c831a700 1 -- start start 2023-11-24T08:00:27.704 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:27.699+0000 7f20c831a700 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f20c0143ae0 0x7f20c00b51b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:00:27.704 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:27.699+0000 7f20c831a700 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f20c00722c0 0x7f20c006e510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:00:27.704 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:27.699+0000 7f20c831a700 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f20c0143ec0 con 0x7f20c01440f0 2023-11-24T08:00:27.704 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:27.699+0000 7f20c831a700 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f20c00b40a0 con 0x7f20c00722c0 2023-11-24T08:00:27.704 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:27.699+0000 7f20c831a700 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f20c00b3ae0 con 0x7f20c0143ae0 2023-11-24T08:00:27.704 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:27.699+0000 7f20c58b5700 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7f20c01440f0 0x7f20c0071bb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:60036/0 (socket says 172.21.15.59:60036) 2023-11-24T08:00:27.704 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:27.699+0000 7f20c58b5700 1 -- 172.21.15.59:0/1446556131 learned_addr learned my addr 172.21.15.59:0/1446556131 (peer_addr_for_me v1:172.21.15.59:0/0) 2023-11-24T08:00:27.705 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:27.699+0000 7f20c60b6700 1 --2- 172.21.15.59:0/1446556131 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f20c0143ae0 0x7f20c00b51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:00:27.705 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:27.699+0000 7f20c68b7700 1 --2- 172.21.15.59:0/1446556131 >> v2:172.21.15.76:3300/0 conn(0x7f20c00722c0 0x7f20c006e510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:00:27.705 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:27.699+0000 7f20c50b4700 1 -- 172.21.15.59:0/1446556131 <== mon.0 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2047890175 0 0) 0x7f20c0143ec0 con 0x7f20c01440f0 2023-11-24T08:00:27.705 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:27.699+0000 7f20c50b4700 1 -- 172.21.15.59:0/1446556131 --> v1:172.21.15.59:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f20ac0035b0 con 0x7f20c01440f0 2023-11-24T08:00:27.705 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:27.699+0000 7f20c50b4700 1 -- 172.21.15.59:0/1446556131 <== mon.0 v1:172.21.15.59:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2634209727 0 0) 0x7f20ac0035b0 con 0x7f20c01440f0 2023-11-24T08:00:27.705 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:27.699+0000 7f20c50b4700 1 -- 172.21.15.59:0/1446556131 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f20c0143ae0 msgr2=0x7f20c00b51b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:00:27.705 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:27.699+0000 7f20c50b4700 1 --2- 172.21.15.59:0/1446556131 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f20c0143ae0 0x7f20c00b51b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:00:27.705 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:27.699+0000 7f20c50b4700 1 -- 172.21.15.59:0/1446556131 >> v2:172.21.15.76:3300/0 conn(0x7f20c00722c0 msgr2=0x7f20c006e510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:00:27.706 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:27.699+0000 7f20c50b4700 1 --2- 172.21.15.59:0/1446556131 >> v2:172.21.15.76:3300/0 conn(0x7f20c00722c0 0x7f20c006e510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:00:27.706 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:27.699+0000 7f20c50b4700 1 -- 172.21.15.59:0/1446556131 --> v1:172.21.15.59:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f20c00b3dc0 con 0x7f20c01440f0 2023-11-24T08:00:27.706 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:27.699+0000 7f20c50b4700 1 -- 172.21.15.59:0/1446556131 <== mon.0 v1:172.21.15.59:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1312333566 0 0) 0x7f20b0001670 con 0x7f20c01440f0 2023-11-24T08:00:27.706 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:27.699+0000 7f20c50b4700 1 -- 172.21.15.59:0/1446556131 >> v1:172.21.15.59:6789/0 conn(0x7f20c01440f0 legacy=0x7f20c0071bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:00:27.706 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:27.700+0000 7f20c50b4700 1 --2- 172.21.15.59:0/1446556131 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f20c0143ae0 0x7f20ac003a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:00:27.706 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:27.700+0000 7f20c50b4700 1 --2- 172.21.15.59:0/1446556131 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f20ac003f80 0x7f20ac004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:00:27.706 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:27.700+0000 7f20c50b4700 1 --2- 172.21.15.59:0/1446556131 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f20ac004d90 0x7f20ac005210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:00:27.706 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:27.700+0000 7f20c50b4700 1 -- 172.21.15.59:0/1446556131 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20c00b40a0 con 0x7f20ac003f80 2023-11-24T08:00:27.707 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:27.700+0000 7f20c50b4700 1 -- 172.21.15.59:0/1446556131 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20c00b3ae0 con 0x7f20ac004d90 2023-11-24T08:00:27.707 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:27.700+0000 7f20c50b4700 1 -- 172.21.15.59:0/1446556131 --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f20ac00ba10 con 0x7f20c0143ae0 2023-11-24T08:00:27.707 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:27.700+0000 7f20c58b5700 1 --2- 172.21.15.59:0/1446556131 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f20ac003f80 0x7f20ac004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:00:27.707 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:27.700+0000 7f20c60b6700 1 --2- 172.21.15.59:0/1446556131 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f20c0143ae0 0x7f20ac003a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:00:27.707 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:27.700+0000 7f20c68b7700 1 --2- 172.21.15.59:0/1446556131 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f20ac004d90 0x7f20ac005210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:00:27.707 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:27.700+0000 7f20c58b5700 1 -- 172.21.15.59:0/1446556131 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f20c0143ae0 msgr2=0x7f20ac003a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:00:27.707 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:27.700+0000 7f20c58b5700 1 --2- 172.21.15.59:0/1446556131 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f20c0143ae0 0x7f20ac003a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:00:27.707 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:27.700+0000 7f20c58b5700 1 -- 172.21.15.59:0/1446556131 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f20ac004d90 msgr2=0x7f20ac005210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:00:27.708 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:27.700+0000 7f20c58b5700 1 --2- 172.21.15.59:0/1446556131 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f20ac004d90 0x7f20ac005210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:00:27.708 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:27.700+0000 7f20c58b5700 1 -- 172.21.15.59:0/1446556131 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f20ac0035b0 con 0x7f20ac003f80 2023-11-24T08:00:27.708 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:27.700+0000 7f20c58b5700 1 --2- 172.21.15.59:0/1446556131 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f20ac003f80 0x7f20ac004850 secure :-1 s=READY pgs=1135 cs=0 l=1 rev1=1 crypto rx=0x7f20b000c2f0 tx=0x7f20ac005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8ccf69ffd734fffb server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:00:27.708 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:27.700+0000 7f20c50b4700 1 -- 172.21.15.59:0/1446556131 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20b000cdd0 con 0x7f20ac003f80 2023-11-24T08:00:27.708 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:27.700+0000 7f20c50b4700 1 -- 172.21.15.59:0/1446556131 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f20b0005960 con 0x7f20ac003f80 2023-11-24T08:00:27.708 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:27.700+0000 7f20c50b4700 1 -- 172.21.15.59:0/1446556131 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20b0005d10 con 0x7f20ac003f80 2023-11-24T08:00:27.708 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:27.701+0000 7f20c831a700 1 -- 172.21.15.59:0/1446556131 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f20ac003f80 msgr2=0x7f20ac004850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:00:27.708 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:27.701+0000 7f20c831a700 1 --2- 172.21.15.59:0/1446556131 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f20ac003f80 0x7f20ac004850 secure :-1 s=READY pgs=1135 cs=0 l=1 rev1=1 crypto rx=0x7f20b000c2f0 tx=0x7f20ac005880 comp rx=0 tx=0).stop 2023-11-24T08:00:27.709 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:27.701+0000 7f20c68b7700 1 -- 172.21.15.59:0/1446556131 reap_dead start 2023-11-24T08:00:27.709 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:27.701+0000 7f20c831a700 1 -- 172.21.15.59:0/1446556131 shutdown_connections 2023-11-24T08:00:27.709 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:27.701+0000 7f20c831a700 1 -- 172.21.15.59:0/1446556131 >> 172.21.15.59:0/1446556131 conn(0x7f20c00b2740 msgr2=0x7f20c00b2b20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:00:27.709 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:27.701+0000 7f20c831a700 1 -- 172.21.15.59:0/1446556131 shutdown_connections 2023-11-24T08:00:27.709 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:27.701+0000 7f20c831a700 1 -- 172.21.15.59:0/1446556131 wait complete. 2023-11-24T08:00:27.709 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:27.702+0000 7f20c831a700 1 Processor -- start 2023-11-24T08:00:27.709 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:27.702+0000 7f20c831a700 1 -- start start 2023-11-24T08:00:27.714 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:27.703+0000 7f20c831a700 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f20ac003f80 0x7f20c01f3b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:00:27.714 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:27.703+0000 7f20c831a700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f20c006afd0 0x7f20c006b450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:00:27.714 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:27.703+0000 7f20c831a700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f20c0140e60 0x7f20c0141310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:00:27.714 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:27.703+0000 7f20c831a700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20c006b990 con 0x7f20c0140e60 2023-11-24T08:00:27.714 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:27.703+0000 7f20c831a700 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20c0141b90 con 0x7f20c006afd0 2023-11-24T08:00:27.714 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:27.703+0000 7f20c831a700 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f20c0141e70 con 0x7f20ac003f80 2023-11-24T08:00:27.715 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:27.703+0000 7f20c68b7700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f20c0140e60 0x7f20c0141310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:00:27.715 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:27.703+0000 7f20c68b7700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f20c0140e60 0x7f20c0141310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:56576/0 (socket says 172.21.15.59:56576) 2023-11-24T08:00:27.715 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:27.703+0000 7f20c68b7700 1 -- 172.21.15.59:0/4177146260 learned_addr learned my addr 172.21.15.59:0/4177146260 (peer_addr_for_me v2:172.21.15.59:0/0) 2023-11-24T08:00:27.715 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:27.703+0000 7f20c60b6700 1 --2- 172.21.15.59:0/4177146260 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f20ac003f80 0x7f20c01f3b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:00:27.715 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:27.703+0000 7f20c58b5700 1 --2- 172.21.15.59:0/4177146260 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f20c006afd0 0x7f20c006b450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:00:27.715 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:27.703+0000 7f20c68b7700 1 -- 172.21.15.59:0/4177146260 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f20ac003f80 msgr2=0x7f20c01f3b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:00:27.715 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:27.703+0000 7f20c68b7700 1 --2- 172.21.15.59:0/4177146260 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f20ac003f80 0x7f20c01f3b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:00:27.715 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:27.703+0000 7f20c68b7700 1 -- 172.21.15.59:0/4177146260 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f20c006afd0 msgr2=0x7f20c006b450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:00:27.716 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:27.703+0000 7f20c68b7700 1 --2- 172.21.15.59:0/4177146260 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f20c006afd0 0x7f20c006b450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:00:27.716 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:27.703+0000 7f20c68b7700 1 -- 172.21.15.59:0/4177146260 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f20c0154e30 con 0x7f20c0140e60 2023-11-24T08:00:27.716 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:27.703+0000 7f20c58b5700 1 --2- 172.21.15.59:0/4177146260 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f20c006afd0 0x7f20c006b450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-24T08:00:27.716 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:27.704+0000 7f20c68b7700 1 --2- 172.21.15.59:0/4177146260 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f20c0140e60 0x7f20c0141310 secure :-1 s=READY pgs=1136 cs=0 l=1 rev1=1 crypto rx=0x7f20c00706c0 tx=0x7f20b8011f80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:00:27.716 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:27.704+0000 7f20b6ffd700 1 -- 172.21.15.59:0/4177146260 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20b800e040 con 0x7f20c0140e60 2023-11-24T08:00:27.716 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:27.704+0000 7f20b6ffd700 1 -- 172.21.15.59:0/4177146260 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f20b800f040 con 0x7f20c0140e60 2023-11-24T08:00:27.716 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:27.704+0000 7f20c831a700 1 -- 172.21.15.59:0/4177146260 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f20c0155110 con 0x7f20c0140e60 2023-11-24T08:00:27.716 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:27.704+0000 7f20b6ffd700 1 -- 172.21.15.59:0/4177146260 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20b800fcf0 con 0x7f20c0140e60 2023-11-24T08:00:27.717 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:27.704+0000 7f20c831a700 1 -- 172.21.15.59:0/4177146260 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f20c0141850 con 0x7f20c0140e60 2023-11-24T08:00:27.717 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:27.705+0000 7f20c831a700 1 -- 172.21.15.59:0/4177146260 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f20c00b34c0 con 0x7f20c0140e60 2023-11-24T08:00:27.717 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:27.705+0000 7f20b6ffd700 1 -- 172.21.15.59:0/4177146260 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 12) v1 ==== 89792+0+0 (secure 0 0 0) 0x7f20b8022460 con 0x7f20c0140e60 2023-11-24T08:00:27.717 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:27.706+0000 7f20b6ffd700 1 --2- 172.21.15.59:0/4177146260 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f20a406b500 0x7f20a406d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:00:27.717 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:27.706+0000 7f20b6ffd700 1 -- 172.21.15.59:0/4177146260 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f20b808a330 con 0x7f20c0140e60 2023-11-24T08:00:27.717 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:27.706+0000 7f20c60b6700 1 --2- 172.21.15.59:0/4177146260 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f20a406b500 0x7f20a406d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:00:27.717 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:27.706+0000 7f20c60b6700 1 --2- 172.21.15.59:0/4177146260 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f20a406b500 0x7f20a406d9c0 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7f20bc009b80 tx=0x7f20bc009210 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:00:27.717 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:27.710+0000 7f20b6ffd700 1 -- 172.21.15.59:0/4177146260 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f20b8054d90 con 0x7f20c0140e60 2023-11-24T08:00:27.913 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:27.911+0000 7f20c831a700 1 -- 172.21.15.59:0/4177146260 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f20c0067a40 con 0x7f20c0140e60 2023-11-24T08:00:27.913 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:27.912+0000 7f20b6ffd700 1 -- 172.21.15.59:0/4177146260 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v90) v1 ==== 74+0+12915 (secure 0 0 0) 0x7f20c0067a40 con 0x7f20c0140e60 2023-11-24T08:00:27.914 INFO:teuthology.orchestra.run.smithi059.stdout: 2023-11-24T08:00:27.914 INFO:teuthology.orchestra.run.smithi059.stdout:{"epoch":90,"fsid":"9f0e8741-b8f9-48d5-a2c3-09f45903303d","created":"2023-11-24T07:56:06.552729+0000","modified":"2023-11-24T08:00:24.411637+0000","last_up_change":"2023-11-24T07:56:17.372352+0000","last_in_change":"2023-11-24T07:56:09.073250+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":13,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-24T07:56:18.352823+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"6d8f9e9b-1782-4e41-928b-a39f2a8ebc06","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6800","nonce":3373878772},{"type":"v1","addr":"172.21.15.59:6801","nonce":3373878772}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6802","nonce":3373878772},{"type":"v1","addr":"172.21.15.59:6803","nonce":3373878772}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6806","nonce":3373878772},{"type":"v1","addr":"172.21.15.59:6807","nonce":3373878772}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6804","nonce":3373878772},{"type":"v1","addr":"172.21.15.59:6805","nonce":3373878772}]},"public_addr":"172.21.15.59:6801/3373878772","cluster_addr":"172.21.15.59:6803/3373878772","heartbeat_back_addr":"172.21.15.59:6807/3373878772","heartbeat_front_addr":"172.21.15.59:6805/3373878772","state":["exists","up"]},{"osd":1,"uuid":"e8d882aa-52e1-4c04-9e1e-c44ef9204a5f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6824","nonce":648258563},{"type":"v1","addr":"172.21.15.59:6825","nonce":648258563}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6826","nonce":648258563},{"type":"v1","addr":"172.21.15.59:6827","nonce":648258563}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6830","nonce":648258563},{"type":"v1","addr":"172.21.15.59:6831","nonce":648258563}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6828","nonce":648258563},{"type":"v1","addr":"172.21.15.59:6829","nonce":648258563}]},"public_addr":"172.21.15.59:6825/648258563","cluster_addr":"172.21.15.59:6827/648258563","heartbeat_back_addr":"172.21.15.59:6831/648258563","heartbeat_front_addr":"172.21.15.59:6829/648258563","state":["exists","up"]},{"osd":2,"uuid":"69eebfa0-695b-4b21-90b9-8a3537168a01","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6808","nonce":1840430654},{"type":"v1","addr":"172.21.15.59:6809","nonce":1840430654}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6810","nonce":1840430654},{"type":"v1","addr":"172.21.15.59:6811","nonce":1840430654}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6814","nonce":1840430654},{"type":"v1","addr":"172.21.15.59:6815","nonce":1840430654}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6812","nonce":1840430654},{"type":"v1","addr":"172.21.15.59:6813","nonce":1840430654}]},"public_addr":"172.21.15.59:6809/1840430654","cluster_addr":"172.21.15.59:6811/1840430654","heartbeat_back_addr":"172.21.15.59:6815/1840430654","heartbeat_front_addr":"172.21.15.59:6813/1840430654","state":["exists","up"]},{"osd":3,"uuid":"96fd77fc-f911-46ff-9445-6f0be2daa8f4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6816","nonce":2504025850},{"type":"v1","addr":"172.21.15.59:6817","nonce":2504025850}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6818","nonce":2504025850},{"type":"v1","addr":"172.21.15.59:6819","nonce":2504025850}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6822","nonce":2504025850},{"type":"v1","addr":"172.21.15.59:6823","nonce":2504025850}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6820","nonce":2504025850},{"type":"v1","addr":"172.21.15.59:6821","nonce":2504025850}]},"public_addr":"172.21.15.59:6817/2504025850","cluster_addr":"172.21.15.59:6819/2504025850","heartbeat_back_addr":"172.21.15.59:6823/2504025850","heartbeat_front_addr":"172.21.15.59:6821/2504025850","state":["exists","up"]},{"osd":4,"uuid":"7ac29fca-b5b2-4c95-9d69-2f164cbbc708","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6816","nonce":2382463431},{"type":"v1","addr":"172.21.15.76:6818","nonce":2382463431}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6820","nonce":2382463431},{"type":"v1","addr":"172.21.15.76:6822","nonce":2382463431}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6828","nonce":2382463431},{"type":"v1","addr":"172.21.15.76:6830","nonce":2382463431}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6824","nonce":2382463431},{"type":"v1","addr":"172.21.15.76:6826","nonce":2382463431}]},"public_addr":"172.21.15.76:6818/2382463431","cluster_addr":"172.21.15.76:6822/2382463431","heartbeat_back_addr":"172.21.15.76:6830/2382463431","heartbeat_front_addr":"172.21.15.76:6826/2382463431","state":["exists","up"]},{"osd":5,"uuid":"485bf711-3851-4238-a4f4-51d52e0a9c58","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6800","nonce":329334308},{"type":"v1","addr":"172.21.15.76:6802","nonce":329334308}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6805","nonce":329334308},{"type":"v1","addr":"172.21.15.76:6807","nonce":329334308}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6814","nonce":329334308},{"type":"v1","addr":"172.21.15.76:6815","nonce":329334308}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6810","nonce":329334308},{"type":"v1","addr":"172.21.15.76:6812","nonce":329334308}]},"public_addr":"172.21.15.76:6802/329334308","cluster_addr":"172.21.15.76:6807/329334308","heartbeat_back_addr":"172.21.15.76:6815/329334308","heartbeat_front_addr":"172.21.15.76:6812/329334308","state":["exists","up"]},{"osd":6,"uuid":"72b3f2fc-1e4a-4e32-968d-003647a0286f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6801","nonce":3946579410},{"type":"v1","addr":"172.21.15.76:6803","nonce":3946579410}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6804","nonce":3946579410},{"type":"v1","addr":"172.21.15.76:6806","nonce":3946579410}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6811","nonce":3946579410},{"type":"v1","addr":"172.21.15.76:6813","nonce":3946579410}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6808","nonce":3946579410},{"type":"v1","addr":"172.21.15.76:6809","nonce":3946579410}]},"public_addr":"172.21.15.76:6803/3946579410","cluster_addr":"172.21.15.76:6806/3946579410","heartbeat_back_addr":"172.21.15.76:6813/3946579410","heartbeat_front_addr":"172.21.15.76:6809/3946579410","state":["exists","up"]},{"osd":7,"uuid":"1ddceb90-6c5d-4d1a-ad56-99cb877f8c70","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6817","nonce":841761538},{"type":"v1","addr":"172.21.15.76:6819","nonce":841761538}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6821","nonce":841761538},{"type":"v1","addr":"172.21.15.76:6823","nonce":841761538}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6829","nonce":841761538},{"type":"v1","addr":"172.21.15.76:6831","nonce":841761538}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6825","nonce":841761538},{"type":"v1","addr":"172.21.15.76:6827","nonce":841761538}]},"public_addr":"172.21.15.76:6819/841761538","cluster_addr":"172.21.15.76:6823/841761538","heartbeat_back_addr":"172.21.15.76:6831/841761538","heartbeat_front_addr":"172.21.15.76:6827/841761538","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T07:56:14.294838+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T07:56:14.270779+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T07:56:14.231809+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T07:56:14.260983+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T07:56:14.302997+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T07:56:14.258600+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T07:56:14.251413+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T07:56:14.304727+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.59:6835/2337335405":"2023-11-25T08:00:21.193380+0000","172.21.15.59:6833/2337335405":"2023-11-25T08:00:21.193380+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-24T08:00:27.916 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:27.914+0000 7f20c831a700 1 -- 172.21.15.59:0/4177146260 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f20a406b500 msgr2=0x7f20a406d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:00:27.916 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:27.914+0000 7f20c831a700 1 --2- 172.21.15.59:0/4177146260 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f20a406b500 0x7f20a406d9c0 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7f20bc009b80 tx=0x7f20bc009210 comp rx=0 tx=0).stop 2023-11-24T08:00:27.916 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:27.914+0000 7f20c831a700 1 -- 172.21.15.59:0/4177146260 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f20c0140e60 msgr2=0x7f20c0141310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:00:27.916 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:27.914+0000 7f20c831a700 1 --2- 172.21.15.59:0/4177146260 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f20c0140e60 0x7f20c0141310 secure :-1 s=READY pgs=1136 cs=0 l=1 rev1=1 crypto rx=0x7f20c00706c0 tx=0x7f20b8011f80 comp rx=0 tx=0).stop 2023-11-24T08:00:27.917 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:27.914+0000 7f20c831a700 1 -- 172.21.15.59:0/4177146260 shutdown_connections 2023-11-24T08:00:27.917 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:27.914+0000 7f20c831a700 1 --2- 172.21.15.59:0/4177146260 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f20a406b500 0x7f20a406d9c0 unknown :-1 s=CLOSED pgs=519 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:00:27.917 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:27.914+0000 7f20c831a700 1 --2- 172.21.15.59:0/4177146260 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f20c0140e60 0x7f20c0141310 unknown :-1 s=CLOSED pgs=1136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:00:27.917 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:27.914+0000 7f20c831a700 1 --2- 172.21.15.59:0/4177146260 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f20ac003f80 0x7f20c01f3b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:00:27.917 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:27.914+0000 7f20c831a700 1 --2- 172.21.15.59:0/4177146260 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f20c006afd0 0x7f20c006b450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:00:27.917 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:27.914+0000 7f20c831a700 1 -- 172.21.15.59:0/4177146260 >> 172.21.15.59:0/4177146260 conn(0x7f20c00b2740 msgr2=0x7f20c015b430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:00:27.917 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:27.914+0000 7f20c831a700 1 -- 172.21.15.59:0/4177146260 shutdown_connections 2023-11-24T08:00:27.918 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:27.914+0000 7f20c831a700 1 -- 172.21.15.59:0/4177146260 wait complete. 2023-11-24T08:00:28.929 INFO:tasks.ceph:Scrubbing osd.0 2023-11-24T08:00:28.930 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 config set osd_debug_deep_scrub_sleep 0 2023-11-24T08:00:29.074 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:29.072+0000 7f2c421cb700 1 Processor -- start 2023-11-24T08:00:29.074 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:29.072+0000 7f2c421cb700 1 -- start start 2023-11-24T08:00:29.076 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:29.073+0000 7f2c421cb700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f2c3c1457b0 0x7f2c3c0725a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:00:29.077 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:29.073+0000 7f2c421cb700 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f2c3c0cf020 0x7f2c3c0cf4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:00:29.077 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:29.073+0000 7f2c421cb700 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2c3c0cd490 con 0x7f2c3c06f580 2023-11-24T08:00:29.077 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:29.073+0000 7f2c421cb700 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2c3c06fa00 con 0x7f2c3c1457b0 2023-11-24T08:00:29.077 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:29.073+0000 7f2c421cb700 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2c3c0cd6e0 con 0x7f2c3c0cf020 2023-11-24T08:00:29.077 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:29.073+0000 7f2c3b7fe700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f2c3c1457b0 0x7f2c3c0725a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:00:29.077 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:29.073+0000 7f2c3affd700 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7f2c3c06f580 0x7f2c3c06e3d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:60056/0 (socket says 172.21.15.59:60056) 2023-11-24T08:00:29.077 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:29.073+0000 7f2c3affd700 1 -- 172.21.15.59:0/1174667097 learned_addr learned my addr 172.21.15.59:0/1174667097 (peer_addr_for_me v1:172.21.15.59:0/0) 2023-11-24T08:00:29.077 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:29.073+0000 7f2c3bfff700 1 --2- 172.21.15.59:0/1174667097 >> v2:172.21.15.76:3300/0 conn(0x7f2c3c0cf020 0x7f2c3c0cf4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:00:29.078 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:29.073+0000 7f2c3b7fe700 1 -- 172.21.15.59:0/1174667097 >> v2:172.21.15.76:3300/0 conn(0x7f2c3c0cf020 msgr2=0x7f2c3c0cf4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:00:29.078 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:29.073+0000 7f2c3b7fe700 1 --2- 172.21.15.59:0/1174667097 >> v2:172.21.15.76:3300/0 conn(0x7f2c3c0cf020 0x7f2c3c0cf4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:00:29.078 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:29.073+0000 7f2c3b7fe700 1 -- 172.21.15.59:0/1174667097 >> v1:172.21.15.59:6789/0 conn(0x7f2c3c06f580 legacy=0x7f2c3c06e3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:00:29.078 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:29.073+0000 7f2c3b7fe700 1 -- 172.21.15.59:0/1174667097 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2c3c0cd9c0 con 0x7f2c3c1457b0 2023-11-24T08:00:29.078 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:29.073+0000 7f2c3a7fc700 1 -- 172.21.15.59:0/1174667097 <== mon.0 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1895390259 0 0) 0x7f2c3c0cd490 con 0x7f2c3c06f580 2023-11-24T08:00:29.078 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:29.073+0000 7f2c3b7fe700 1 --2- 172.21.15.59:0/1174667097 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2c3c1457b0 0x7f2c3c0725a0 secure :-1 s=READY pgs=1138 cs=0 l=1 rev1=1 crypto rx=0x7f2c3c0b0670 tx=0x7f2c2c00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=275271a20dfb6a76 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:00:29.078 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:29.074+0000 7f2c3a7fc700 1 -- 172.21.15.59:0/1174667097 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c2c00e070 con 0x7f2c3c1457b0 2023-11-24T08:00:29.079 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:29.074+0000 7f2c3a7fc700 1 -- 172.21.15.59:0/1174667097 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2c2c00bbd0 con 0x7f2c3c1457b0 2023-11-24T08:00:29.079 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:29.074+0000 7f2c3a7fc700 1 -- 172.21.15.59:0/1174667097 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c2c012430 con 0x7f2c3c1457b0 2023-11-24T08:00:29.079 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:29.074+0000 7f2c421cb700 1 -- 172.21.15.59:0/1174667097 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2c3c1457b0 msgr2=0x7f2c3c0725a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:00:29.079 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:29.074+0000 7f2c421cb700 1 --2- 172.21.15.59:0/1174667097 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2c3c1457b0 0x7f2c3c0725a0 secure :-1 s=READY pgs=1138 cs=0 l=1 rev1=1 crypto rx=0x7f2c3c0b0670 tx=0x7f2c2c00b330 comp rx=0 tx=0).stop 2023-11-24T08:00:29.079 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:29.074+0000 7f2c421cb700 1 -- 172.21.15.59:0/1174667097 shutdown_connections 2023-11-24T08:00:29.079 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:29.074+0000 7f2c421cb700 1 --2- 172.21.15.59:0/1174667097 >> v2:172.21.15.76:3300/0 conn(0x7f2c3c0cf020 0x7f2c3c0cf4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:00:29.079 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:29.074+0000 7f2c421cb700 1 --2- 172.21.15.59:0/1174667097 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2c3c1457b0 0x7f2c3c0725a0 unknown :-1 s=CLOSED pgs=1138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:00:29.079 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:29.074+0000 7f2c421cb700 1 -- 172.21.15.59:0/1174667097 >> 172.21.15.59:0/1174667097 conn(0x7f2c3c0b2760 msgr2=0x7f2c3c0b2b70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:00:29.080 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:29.074+0000 7f2c421cb700 1 -- 172.21.15.59:0/1174667097 shutdown_connections 2023-11-24T08:00:29.080 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:29.074+0000 7f2c421cb700 1 -- 172.21.15.59:0/1174667097 wait complete. 2023-11-24T08:00:29.083 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:29.075+0000 7f2c421cb700 1 Processor -- start 2023-11-24T08:00:29.083 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:29.076+0000 7f2c421cb700 1 -- start start 2023-11-24T08:00:29.083 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:29.076+0000 7f2c421cb700 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f2c3c06f580 0x7f2c3c13f750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:00:29.084 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:29.076+0000 7f2c421cb700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2c3c0cf020 0x7f2c3c15eb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:00:29.084 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:29.076+0000 7f2c421cb700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2c3c1457b0 0x7f2c3c0cfcb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:00:29.084 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:29.076+0000 7f2c421cb700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c3c0aab10 con 0x7f2c3c0cf020 2023-11-24T08:00:29.084 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:29.076+0000 7f2c421cb700 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c3c15d160 con 0x7f2c3c1457b0 2023-11-24T08:00:29.084 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:29.076+0000 7f2c421cb700 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2c3c15d440 con 0x7f2c3c06f580 2023-11-24T08:00:29.085 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:29.077+0000 7f2c3affd700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2c3c0cf020 0x7f2c3c15eb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:00:29.085 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:29.077+0000 7f2c3affd700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2c3c0cf020 0x7f2c3c15eb60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:56588/0 (socket says 172.21.15.59:56588) 2023-11-24T08:00:29.085 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:29.077+0000 7f2c3affd700 1 -- 172.21.15.59:0/3794512157 learned_addr learned my addr 172.21.15.59:0/3794512157 (peer_addr_for_me v2:172.21.15.59:0/0) 2023-11-24T08:00:29.085 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:29.077+0000 7f2c3bfff700 1 --2- 172.21.15.59:0/3794512157 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2c3c1457b0 0x7f2c3c0cfcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:00:29.085 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:29.077+0000 7f2c3b7fe700 1 --2- 172.21.15.59:0/3794512157 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f2c3c06f580 0x7f2c3c13f750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:00:29.085 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:29.077+0000 7f2c3affd700 1 -- 172.21.15.59:0/3794512157 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f2c3c06f580 msgr2=0x7f2c3c13f750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:00:29.085 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:29.077+0000 7f2c3affd700 1 --2- 172.21.15.59:0/3794512157 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f2c3c06f580 0x7f2c3c13f750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:00:29.085 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:29.077+0000 7f2c3affd700 1 -- 172.21.15.59:0/3794512157 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2c3c1457b0 msgr2=0x7f2c3c0cfcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:00:29.086 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:29.077+0000 7f2c3affd700 1 --2- 172.21.15.59:0/3794512157 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2c3c1457b0 0x7f2c3c0cfcb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:00:29.086 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:29.077+0000 7f2c3affd700 1 -- 172.21.15.59:0/3794512157 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2c2c009a70 con 0x7f2c3c0cf020 2023-11-24T08:00:29.086 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:29.077+0000 7f2c3bfff700 1 --2- 172.21.15.59:0/3794512157 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2c3c1457b0 0x7f2c3c0cfcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-24T08:00:29.086 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:29.077+0000 7f2c3affd700 1 --2- 172.21.15.59:0/3794512157 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2c3c0cf020 0x7f2c3c15eb60 secure :-1 s=READY pgs=1139 cs=0 l=1 rev1=1 crypto rx=0x7f2c300011e0 tx=0x7f2c3000cae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:00:29.086 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:29.077+0000 7f2c23fff700 1 -- 172.21.15.59:0/3794512157 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c30004c80 con 0x7f2c3c0cf020 2023-11-24T08:00:29.087 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:29.077+0000 7f2c23fff700 1 -- 172.21.15.59:0/3794512157 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2c30004e00 con 0x7f2c3c0cf020 2023-11-24T08:00:29.087 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:29.077+0000 7f2c421cb700 1 -- 172.21.15.59:0/3794512157 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2c3c15d720 con 0x7f2c3c0cf020 2023-11-24T08:00:29.087 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:29.077+0000 7f2c23fff700 1 -- 172.21.15.59:0/3794512157 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c30007310 con 0x7f2c3c0cf020 2023-11-24T08:00:29.087 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:29.077+0000 7f2c421cb700 1 -- 172.21.15.59:0/3794512157 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2c3c15cd80 con 0x7f2c3c0cf020 2023-11-24T08:00:29.087 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:29.078+0000 7f2c421cb700 1 -- 172.21.15.59:0/3794512157 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f2c3c06deb0 con 0x7f2c3c0cf020 2023-11-24T08:00:29.087 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:29.079+0000 7f2c23fff700 1 -- 172.21.15.59:0/3794512157 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 12) v1 ==== 89792+0+0 (secure 0 0 0) 0x7f2c30007470 con 0x7f2c3c0cf020 2023-11-24T08:00:29.087 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:29.079+0000 7f2c23fff700 1 --2- 172.21.15.59:0/3794512157 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f2c2806b500 0x7f2c2806d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:00:29.087 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:29.079+0000 7f2c23fff700 1 -- 172.21.15.59:0/3794512157 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f2c30089bb0 con 0x7f2c3c0cf020 2023-11-24T08:00:29.088 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:29.079+0000 7f2c23fff700 1 --2- 172.21.15.59:0/3794512157 >> [v2:172.21.15.59:6800/3373878772,v1:172.21.15.59:6801/3373878772] conn(0x7f2c28071360 0x7f2c28073780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:00:29.088 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:29.079+0000 7f2c3b7fe700 1 --2- 172.21.15.59:0/3794512157 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f2c2806b500 0x7f2c2806d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:00:29.088 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:29.079+0000 7f2c23fff700 1 -- 172.21.15.59:0/3794512157 --> [v2:172.21.15.59:6800/3373878772,v1:172.21.15.59:6801/3373878772] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f2c28073e50 con 0x7f2c28071360 2023-11-24T08:00:29.088 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:29.079+0000 7f2c23fff700 1 -- 172.21.15.59:0/3794512157 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_get_version_reply(handle=1 version=90) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2c3008e2a0 con 0x7f2c3c0cf020 2023-11-24T08:00:29.088 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:29.080+0000 7f2c3bfff700 1 --2- 172.21.15.59:0/3794512157 >> [v2:172.21.15.59:6800/3373878772,v1:172.21.15.59:6801/3373878772] conn(0x7f2c28071360 0x7f2c28073780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:00:29.088 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:29.080+0000 7f2c3b7fe700 1 --2- 172.21.15.59:0/3794512157 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f2c2806b500 0x7f2c2806d9c0 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7f2c2c014040 tx=0x7f2c2c011000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:00:29.089 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:29.080+0000 7f2c3bfff700 1 --2- 172.21.15.59:0/3794512157 >> [v2:172.21.15.59:6800/3373878772,v1:172.21.15.59:6801/3373878772] conn(0x7f2c28071360 0x7f2c28073780 crc :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:00:29.089 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:29.081+0000 7f2c23fff700 1 -- 172.21.15.59:0/3794512157 <== osd.0 v2:172.21.15.59:6800/3373878772 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+26965 (crc 0 0 0) 0x7f2c28073e50 con 0x7f2c28071360 2023-11-24T08:00:29.097 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:29.096+0000 7f2c421cb700 1 -- 172.21.15.59:0/3794512157 --> [v2:172.21.15.59:6800/3373878772,v1:172.21.15.59:6801/3373878772] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f2c3c15d5b0 con 0x7f2c28071360 2023-11-24T08:00:29.098 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:29.096+0000 7f2c23fff700 1 -- 172.21.15.59:0/3794512157 <== osd.0 v2:172.21.15.59:6800/3373878772 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+97 (crc 0 0 0) 0x7f2c3c15d5b0 con 0x7f2c28071360 2023-11-24T08:00:29.098 INFO:teuthology.orchestra.run.smithi059.stdout:{ 2023-11-24T08:00:29.098 INFO:teuthology.orchestra.run.smithi059.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2023-11-24T08:00:29.098 INFO:teuthology.orchestra.run.smithi059.stdout:} 2023-11-24T08:00:29.100 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:29.096+0000 7f2c421cb700 1 -- 172.21.15.59:0/3794512157 >> [v2:172.21.15.59:6800/3373878772,v1:172.21.15.59:6801/3373878772] conn(0x7f2c28071360 msgr2=0x7f2c28073780 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:00:29.100 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:29.096+0000 7f2c421cb700 1 --2- 172.21.15.59:0/3794512157 >> [v2:172.21.15.59:6800/3373878772,v1:172.21.15.59:6801/3373878772] conn(0x7f2c28071360 0x7f2c28073780 crc :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:00:29.100 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:29.096+0000 7f2c421cb700 1 -- 172.21.15.59:0/3794512157 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f2c2806b500 msgr2=0x7f2c2806d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:00:29.100 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:29.097+0000 7f2c421cb700 1 --2- 172.21.15.59:0/3794512157 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f2c2806b500 0x7f2c2806d9c0 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7f2c2c014040 tx=0x7f2c2c011000 comp rx=0 tx=0).stop 2023-11-24T08:00:29.100 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:29.097+0000 7f2c421cb700 1 -- 172.21.15.59:0/3794512157 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2c3c0cf020 msgr2=0x7f2c3c15eb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:00:29.100 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:29.097+0000 7f2c421cb700 1 --2- 172.21.15.59:0/3794512157 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2c3c0cf020 0x7f2c3c15eb60 secure :-1 s=READY pgs=1139 cs=0 l=1 rev1=1 crypto rx=0x7f2c300011e0 tx=0x7f2c3000cae0 comp rx=0 tx=0).stop 2023-11-24T08:00:29.100 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:29.097+0000 7f2c3bfff700 1 -- 172.21.15.59:0/3794512157 reap_dead start 2023-11-24T08:00:29.101 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:29.097+0000 7f2c421cb700 1 -- 172.21.15.59:0/3794512157 shutdown_connections 2023-11-24T08:00:29.101 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:29.097+0000 7f2c421cb700 1 -- 172.21.15.59:0/3794512157 >> 172.21.15.59:0/3794512157 conn(0x7f2c3c0b2760 msgr2=0x7f2c3c141d90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:00:29.101 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:29.097+0000 7f2c421cb700 1 -- 172.21.15.59:0/3794512157 shutdown_connections 2023-11-24T08:00:29.101 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:29.097+0000 7f2c421cb700 1 -- 172.21.15.59:0/3794512157 wait complete. 2023-11-24T08:00:29.110 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 0 2023-11-24T08:00:29.248 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:29.247+0000 7f42491d3700 1 Processor -- start 2023-11-24T08:00:29.249 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:29.247+0000 7f42491d3700 1 -- start start 2023-11-24T08:00:29.250 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:29.247+0000 7f42491d3700 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f42440b4150 0x7f424406de50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:00:29.250 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:29.247+0000 7f42491d3700 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f42440b5ab0 0x7f424406e390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:00:29.251 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:29.248+0000 7f42491d3700 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4244071f50 con 0x7f424406b620 2023-11-24T08:00:29.251 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:29.248+0000 7f42491d3700 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4244072230 con 0x7f42440b5ab0 2023-11-24T08:00:29.251 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:29.248+0000 7f42491d3700 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4244072510 con 0x7f42440b4150 2023-11-24T08:00:29.251 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:29.248+0000 7f424359e700 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f42440b5ab0 0x7f424406e390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:00:29.251 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:29.248+0000 7f424259c700 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f42440b4150 0x7f424406de50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:00:29.251 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:29.248+0000 7f424259c700 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f42440b4150 0x7f424406de50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3301/0 says I am v2:172.21.15.59:56038/0 (socket says 172.21.15.59:56038) 2023-11-24T08:00:29.252 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:29.248+0000 7f424359e700 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f42440b5ab0 0x7f424406e390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.59:43798/0 (socket says 172.21.15.59:43798) 2023-11-24T08:00:29.252 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:29.248+0000 7f424259c700 1 -- 172.21.15.59:0/1727163578 learned_addr learned my addr 172.21.15.59:0/1727163578 (peer_addr_for_me v2:172.21.15.59:0/0) 2023-11-24T08:00:29.252 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:29.248+0000 7f424259c700 1 -- 172.21.15.59:0/1727163578 >> v2:172.21.15.76:3300/0 conn(0x7f42440b5ab0 msgr2=0x7f424406e390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:00:29.252 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:29.248+0000 7f424259c700 1 --2- 172.21.15.59:0/1727163578 >> v2:172.21.15.76:3300/0 conn(0x7f42440b5ab0 0x7f424406e390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:00:29.252 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:29.248+0000 7f424259c700 1 -- 172.21.15.59:0/1727163578 >> v1:172.21.15.76:6789/0 conn(0x7f424406b620 legacy=0x7f42440b3a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:00:29.252 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:29.248+0000 7f424259c700 1 -- 172.21.15.59:0/1727163578 --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f42440727f0 con 0x7f42440b4150 2023-11-24T08:00:29.252 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:29.249+0000 7f424259c700 1 --2- 172.21.15.59:0/1727163578 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f42440b4150 0x7f424406de50 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f42380098f0 tx=0x7f4238009c00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ed6e9fdfbdcd5ac9 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:00:29.252 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:29.249+0000 7f4241d9b700 1 -- 172.21.15.59:0/1727163578 <== mon.2 v2:172.21.15.76:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4238011830 con 0x7f42440b4150 2023-11-24T08:00:29.253 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:29.249+0000 7f4241d9b700 1 -- 172.21.15.59:0/1727163578 <== mon.2 v2:172.21.15.76:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4238011990 con 0x7f42440b4150 2023-11-24T08:00:29.253 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:29.249+0000 7f4241d9b700 1 -- 172.21.15.59:0/1727163578 <== mon.2 v2:172.21.15.76:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4238011d20 con 0x7f42440b4150 2023-11-24T08:00:29.256 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:29.249+0000 7f42491d3700 1 -- 172.21.15.59:0/1727163578 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f42440b4150 msgr2=0x7f424406de50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:00:29.256 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:29.249+0000 7f42491d3700 1 --2- 172.21.15.59:0/1727163578 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f42440b4150 0x7f424406de50 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f42380098f0 tx=0x7f4238009c00 comp rx=0 tx=0).stop 2023-11-24T08:00:29.256 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:29.249+0000 7f42491d3700 1 -- 172.21.15.59:0/1727163578 shutdown_connections 2023-11-24T08:00:29.256 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:29.249+0000 7f42491d3700 1 --2- 172.21.15.59:0/1727163578 >> v2:172.21.15.76:3300/0 conn(0x7f42440b5ab0 0x7f424406e390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:00:29.257 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:29.249+0000 7f42491d3700 1 --2- 172.21.15.59:0/1727163578 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f42440b4150 0x7f424406de50 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:00:29.257 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:29.249+0000 7f42491d3700 1 -- 172.21.15.59:0/1727163578 >> 172.21.15.59:0/1727163578 conn(0x7f42440b2740 msgr2=0x7f42440b2b20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:00:29.257 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:29.249+0000 7f42491d3700 1 -- 172.21.15.59:0/1727163578 shutdown_connections 2023-11-24T08:00:29.257 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:29.249+0000 7f42491d3700 1 -- 172.21.15.59:0/1727163578 wait complete. 2023-11-24T08:00:29.257 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:29.251+0000 7f42491d3700 1 Processor -- start 2023-11-24T08:00:29.258 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:29.251+0000 7f42491d3700 1 -- start start 2023-11-24T08:00:29.258 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:29.252+0000 7f42491d3700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f424406b620 0x7f42441e6b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:00:29.258 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:29.252+0000 7f42491d3700 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f42440b4150 0x7f42441df6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:00:29.258 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:29.252+0000 7f42491d3700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f42440b5ab0 0x7f42441e1240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:00:29.258 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:29.252+0000 7f42491d3700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42441dfc00 con 0x7f42440b5ab0 2023-11-24T08:00:29.258 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:29.252+0000 7f42491d3700 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42441e1780 con 0x7f424406b620 2023-11-24T08:00:29.259 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:29.252+0000 7f42491d3700 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f42441e7e40 con 0x7f42440b4150 2023-11-24T08:00:29.259 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:29.252+0000 7f424359e700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f42440b5ab0 0x7f42441e1240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:00:29.259 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:29.252+0000 7f424259c700 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f42440b4150 0x7f42441df6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:00:29.259 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:29.252+0000 7f424359e700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f42440b5ab0 0x7f42441e1240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:56606/0 (socket says 172.21.15.59:56606) 2023-11-24T08:00:29.259 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:29.252+0000 7f424359e700 1 -- 172.21.15.59:0/2849723179 learned_addr learned my addr 172.21.15.59:0/2849723179 (peer_addr_for_me v2:172.21.15.59:0/0) 2023-11-24T08:00:29.259 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:29.252+0000 7f4242d9d700 1 --2- 172.21.15.59:0/2849723179 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f424406b620 0x7f42441e6b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:00:29.259 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:29.252+0000 7f424359e700 1 -- 172.21.15.59:0/2849723179 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f42440b4150 msgr2=0x7f42441df6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:00:29.259 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:29.252+0000 7f424359e700 1 --2- 172.21.15.59:0/2849723179 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f42440b4150 0x7f42441df6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:00:29.260 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:29.252+0000 7f424359e700 1 -- 172.21.15.59:0/2849723179 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f424406b620 msgr2=0x7f42441e6b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:00:29.260 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:29.252+0000 7f424359e700 1 --2- 172.21.15.59:0/2849723179 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f424406b620 0x7f42441e6b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:00:29.260 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:29.252+0000 7f424359e700 1 -- 172.21.15.59:0/2849723179 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4238009550 con 0x7f42440b5ab0 2023-11-24T08:00:29.260 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:29.252+0000 7f424359e700 1 --2- 172.21.15.59:0/2849723179 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f42440b5ab0 0x7f42441e1240 secure :-1 s=READY pgs=1140 cs=0 l=1 rev1=1 crypto rx=0x7f423c00efc0 tx=0x7f423c00c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:00:29.260 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:29.252+0000 7f422b7fe700 1 -- 172.21.15.59:0/2849723179 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f423c00cc90 con 0x7f42440b5ab0 2023-11-24T08:00:29.260 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:29.253+0000 7f422b7fe700 1 -- 172.21.15.59:0/2849723179 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f423c00f040 con 0x7f42440b5ab0 2023-11-24T08:00:29.260 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:29.253+0000 7f42491d3700 1 -- 172.21.15.59:0/2849723179 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f42441e7230 con 0x7f42440b5ab0 2023-11-24T08:00:29.260 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:29.253+0000 7f422b7fe700 1 -- 172.21.15.59:0/2849723179 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f423c0146c0 con 0x7f42440b5ab0 2023-11-24T08:00:29.261 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:29.253+0000 7f42491d3700 1 -- 172.21.15.59:0/2849723179 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f42441e77f0 con 0x7f42440b5ab0 2023-11-24T08:00:29.261 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:29.253+0000 7f42491d3700 1 -- 172.21.15.59:0/2849723179 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4244068670 con 0x7f42440b5ab0 2023-11-24T08:00:29.261 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:29.254+0000 7f422b7fe700 1 -- 172.21.15.59:0/2849723179 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 12) v1 ==== 89792+0+0 (secure 0 0 0) 0x7f423c014820 con 0x7f42440b5ab0 2023-11-24T08:00:29.261 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:29.254+0000 7f422b7fe700 1 --2- 172.21.15.59:0/2849723179 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f423006b500 0x7f423006d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:00:29.261 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:29.254+0000 7f422b7fe700 1 -- 172.21.15.59:0/2849723179 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f423c08aab0 con 0x7f42440b5ab0 2023-11-24T08:00:29.261 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:29.255+0000 7f4242d9d700 1 --2- 172.21.15.59:0/2849723179 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f423006b500 0x7f423006d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:00:29.262 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:29.255+0000 7f4242d9d700 1 --2- 172.21.15.59:0/2849723179 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f423006b500 0x7f423006d9c0 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7f42441ea060 tx=0x7f4234009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:00:29.262 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:29.259+0000 7f422b7fe700 1 -- 172.21.15.59:0/2849723179 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f423c055410 con 0x7f42440b5ab0 2023-11-24T08:00:29.462 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:29.461+0000 7f42491d3700 1 -- 172.21.15.59:0/2849723179 --> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "0", "target": ["mon-mgr", ""]}) v1 -- 0x7f424406ba00 con 0x7f423006b500 2023-11-24T08:00:29.463 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:29.462+0000 7f422b7fe700 1 -- 172.21.15.59:0/2849723179 <== mgr.4100 v2:172.21.15.76:6832/4264046536 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 0 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f4234001cc0 con 0x7f423006b500 2023-11-24T08:00:29.466 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:29.464+0000 7f42491d3700 1 -- 172.21.15.59:0/2849723179 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f423006b500 msgr2=0x7f423006d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:00:29.466 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:29.464+0000 7f42491d3700 1 --2- 172.21.15.59:0/2849723179 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f423006b500 0x7f423006d9c0 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7f42441ea060 tx=0x7f4234009040 comp rx=0 tx=0).stop 2023-11-24T08:00:29.467 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:29.464+0000 7f42491d3700 1 -- 172.21.15.59:0/2849723179 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f42440b5ab0 msgr2=0x7f42441e1240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:00:29.467 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:29.464+0000 7f42491d3700 1 --2- 172.21.15.59:0/2849723179 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f42440b5ab0 0x7f42441e1240 secure :-1 s=READY pgs=1140 cs=0 l=1 rev1=1 crypto rx=0x7f423c00efc0 tx=0x7f423c00c3a0 comp rx=0 tx=0).stop 2023-11-24T08:00:29.467 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:29.464+0000 7f42491d3700 1 -- 172.21.15.59:0/2849723179 shutdown_connections 2023-11-24T08:00:29.467 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:29.464+0000 7f42491d3700 1 --2- 172.21.15.59:0/2849723179 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f423006b500 0x7f423006d9c0 unknown :-1 s=CLOSED pgs=521 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:00:29.467 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:29.464+0000 7f42491d3700 1 --2- 172.21.15.59:0/2849723179 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f42440b5ab0 0x7f42441e1240 unknown :-1 s=CLOSED pgs=1140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:00:29.467 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:29.464+0000 7f42491d3700 1 --2- 172.21.15.59:0/2849723179 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f424406b620 0x7f42441e6b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:00:29.468 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:29.464+0000 7f42491d3700 1 --2- 172.21.15.59:0/2849723179 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f42440b4150 0x7f42441df6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:00:29.468 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:29.464+0000 7f42491d3700 1 -- 172.21.15.59:0/2849723179 >> 172.21.15.59:0/2849723179 conn(0x7f42440b2740 msgr2=0x7f42441461b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:00:29.468 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:29.464+0000 7f42491d3700 1 -- 172.21.15.59:0/2849723179 shutdown_connections 2023-11-24T08:00:29.468 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:29.464+0000 7f42491d3700 1 -- 172.21.15.59:0/2849723179 wait complete. 2023-11-24T08:00:29.468 INFO:teuthology.orchestra.run.smithi059.stderr:instructed osd(s) 0 to deep-scrub 2023-11-24T08:00:29.478 INFO:tasks.ceph:Scrubbing osd.1 2023-11-24T08:00:29.478 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 config set osd_debug_deep_scrub_sleep 0 2023-11-24T08:00:29.608 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:29.606+0000 7f14216fa700 1 Processor -- start 2023-11-24T08:00:29.608 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:29.606+0000 7f14216fa700 1 -- start start 2023-11-24T08:00:29.610 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:29.607+0000 7f14216fa700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f141c0b49b0 0x7f141c0b3e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:00:29.610 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:29.607+0000 7f14216fa700 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f141c0cf7b0 con 0x7f141c162210 2023-11-24T08:00:29.610 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:29.607+0000 7f14216fa700 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f141c0d2f80 con 0x7f141c0b4370 2023-11-24T08:00:29.610 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:29.607+0000 7f14216fa700 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f141c0b4820 con 0x7f141c0b49b0 2023-11-24T08:00:29.610 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:29.607+0000 7f141a7fc700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f141c0b49b0 0x7f141c0b3e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:00:29.611 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:29.607+0000 7f141a7fc700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f141c0b49b0 0x7f141c0b3e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:56612/0 (socket says 172.21.15.59:56612) 2023-11-24T08:00:29.611 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:29.607+0000 7f141a7fc700 1 -- 172.21.15.59:0/42560346 learned_addr learned my addr 172.21.15.59:0/42560346 (peer_addr_for_me v2:172.21.15.59:0/0) 2023-11-24T08:00:29.611 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:29.607+0000 7f141affd700 1 --1- 172.21.15.59:0/42560346 >> v1:172.21.15.59:6789/0 conn(0x7f141c162210 0x7f141c1625f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:60084/0 (socket says 172.21.15.59:60084) 2023-11-24T08:00:29.611 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:29.607+0000 7f141a7fc700 1 -- 172.21.15.59:0/42560346 >> v1:172.21.15.76:6789/0 conn(0x7f141c0b4370 legacy=0x7f141c0d0200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:00:29.611 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:29.607+0000 7f141a7fc700 1 -- 172.21.15.59:0/42560346 >> v1:172.21.15.59:6789/0 conn(0x7f141c162210 legacy=0x7f141c1625f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:00:29.611 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:29.607+0000 7f141a7fc700 1 -- 172.21.15.59:0/42560346 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f141c0cfa00 con 0x7f141c0b49b0 2023-11-24T08:00:29.611 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:29.607+0000 7f1419ffb700 1 -- 172.21.15.59:0/42560346 <== mon.0 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2412252806 0 0) 0x7f141c0cf7b0 con 0x7f141c162210 2023-11-24T08:00:29.611 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:29.607+0000 7f141a7fc700 1 --2- 172.21.15.59:0/42560346 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f141c0b49b0 0x7f141c0b3e30 secure :-1 s=READY pgs=1142 cs=0 l=1 rev1=1 crypto rx=0x7f14040098f0 tx=0x7f1404009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=653ff0987c9b45af server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:00:29.612 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:29.607+0000 7f1419ffb700 1 -- 172.21.15.59:0/42560346 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1404004230 con 0x7f141c0b49b0 2023-11-24T08:00:29.612 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:29.608+0000 7f1419ffb700 1 -- 172.21.15.59:0/42560346 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1404004390 con 0x7f141c0b49b0 2023-11-24T08:00:29.612 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:29.608+0000 7f1419ffb700 1 -- 172.21.15.59:0/42560346 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14040036a0 con 0x7f141c0b49b0 2023-11-24T08:00:29.612 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:29.608+0000 7f14216fa700 1 -- 172.21.15.59:0/42560346 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f141c0b49b0 msgr2=0x7f141c0b3e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:00:29.612 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:29.608+0000 7f14216fa700 1 --2- 172.21.15.59:0/42560346 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f141c0b49b0 0x7f141c0b3e30 secure :-1 s=READY pgs=1142 cs=0 l=1 rev1=1 crypto rx=0x7f14040098f0 tx=0x7f1404009c00 comp rx=0 tx=0).stop 2023-11-24T08:00:29.612 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:29.608+0000 7f14216fa700 1 -- 172.21.15.59:0/42560346 shutdown_connections 2023-11-24T08:00:29.612 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:29.608+0000 7f14216fa700 1 --2- 172.21.15.59:0/42560346 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f141c0b49b0 0x7f141c0b3e30 unknown :-1 s=CLOSED pgs=1142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:00:29.612 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:29.608+0000 7f14216fa700 1 -- 172.21.15.59:0/42560346 >> 172.21.15.59:0/42560346 conn(0x7f141c09d0c0 msgr2=0x7f141c0d3eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:00:29.613 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:29.608+0000 7f14216fa700 1 -- 172.21.15.59:0/42560346 shutdown_connections 2023-11-24T08:00:29.613 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:29.608+0000 7f14216fa700 1 -- 172.21.15.59:0/42560346 wait complete. 2023-11-24T08:00:29.616 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:29.609+0000 7f14216fa700 1 Processor -- start 2023-11-24T08:00:29.616 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:29.609+0000 7f14216fa700 1 -- start start 2023-11-24T08:00:29.616 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:29.610+0000 7f14216fa700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f141c0b4370 0x7f141c0b7490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:00:29.616 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:29.610+0000 7f14216fa700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f141c0b49b0 0x7f141c159f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:00:29.616 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:29.610+0000 7f14216fa700 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f141c162210 0x7f141c0d1a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:00:29.616 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:29.610+0000 7f14216fa700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f141c0b8780 con 0x7f141c0b49b0 2023-11-24T08:00:29.617 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:29.610+0000 7f14216fa700 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f141c15a4a0 con 0x7f141c0b4370 2023-11-24T08:00:29.617 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:29.610+0000 7f14216fa700 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f141c0b7b70 con 0x7f141c162210 2023-11-24T08:00:29.617 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:29.610+0000 7f141a7fc700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f141c0b49b0 0x7f141c159f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:00:29.617 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:29.610+0000 7f141a7fc700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f141c0b49b0 0x7f141c159f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:56618/0 (socket says 172.21.15.59:56618) 2023-11-24T08:00:29.617 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:29.610+0000 7f141a7fc700 1 -- 172.21.15.59:0/3046055581 learned_addr learned my addr 172.21.15.59:0/3046055581 (peer_addr_for_me v2:172.21.15.59:0/0) 2023-11-24T08:00:29.617 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:29.610+0000 7f141affd700 1 --2- 172.21.15.59:0/3046055581 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f141c0b4370 0x7f141c0b7490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:00:29.617 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:29.610+0000 7f141b7fe700 1 --2- 172.21.15.59:0/3046055581 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f141c162210 0x7f141c0d1a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:00:29.617 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:29.611+0000 7f141a7fc700 1 -- 172.21.15.59:0/3046055581 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f141c162210 msgr2=0x7f141c0d1a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:00:29.618 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:29.611+0000 7f141a7fc700 1 --2- 172.21.15.59:0/3046055581 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f141c162210 0x7f141c0d1a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:00:29.618 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:29.611+0000 7f141a7fc700 1 -- 172.21.15.59:0/3046055581 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f141c0b4370 msgr2=0x7f141c0b7490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:00:29.618 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:29.611+0000 7f141a7fc700 1 --2- 172.21.15.59:0/3046055581 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f141c0b4370 0x7f141c0b7490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:00:29.618 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:29.611+0000 7f141a7fc700 1 -- 172.21.15.59:0/3046055581 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1404009550 con 0x7f141c0b49b0 2023-11-24T08:00:29.618 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:29.611+0000 7f141affd700 1 --2- 172.21.15.59:0/3046055581 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f141c0b4370 0x7f141c0b7490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-24T08:00:29.618 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:29.611+0000 7f141a7fc700 1 --2- 172.21.15.59:0/3046055581 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f141c0b49b0 0x7f141c159f60 secure :-1 s=READY pgs=1143 cs=0 l=1 rev1=1 crypto rx=0x7f1404009130 tx=0x7f1404004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:00:29.618 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:29.611+0000 7f14137fe700 1 -- 172.21.15.59:0/3046055581 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f140400b660 con 0x7f141c0b49b0 2023-11-24T08:00:29.619 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:29.611+0000 7f14137fe700 1 -- 172.21.15.59:0/3046055581 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f140400b7c0 con 0x7f141c0b49b0 2023-11-24T08:00:29.619 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:29.611+0000 7f14216fa700 1 -- 172.21.15.59:0/3046055581 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f141c0b7e50 con 0x7f141c0b49b0 2023-11-24T08:00:29.619 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:29.611+0000 7f14137fe700 1 -- 172.21.15.59:0/3046055581 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f140400bb70 con 0x7f141c0b49b0 2023-11-24T08:00:29.619 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:29.611+0000 7f14216fa700 1 -- 172.21.15.59:0/3046055581 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f141c0b8410 con 0x7f141c0b49b0 2023-11-24T08:00:29.619 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:29.612+0000 7f14216fa700 1 -- 172.21.15.59:0/3046055581 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f141c161010 con 0x7f141c0b49b0 2023-11-24T08:00:29.619 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:29.612+0000 7f14137fe700 1 -- 172.21.15.59:0/3046055581 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 12) v1 ==== 89792+0+0 (secure 0 0 0) 0x7f140400bcd0 con 0x7f141c0b49b0 2023-11-24T08:00:29.619 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:29.613+0000 7f14137fe700 1 --2- 172.21.15.59:0/3046055581 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f13fc06b500 0x7f13fc06d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:00:29.619 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:29.613+0000 7f14137fe700 1 -- 172.21.15.59:0/3046055581 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f140408b590 con 0x7f141c0b49b0 2023-11-24T08:00:29.620 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:29.613+0000 7f14137fe700 1 --2- 172.21.15.59:0/3046055581 >> [v2:172.21.15.59:6824/648258563,v1:172.21.15.59:6825/648258563] conn(0x7f13fc071360 0x7f13fc073780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:00:29.620 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:29.613+0000 7f14137fe700 1 -- 172.21.15.59:0/3046055581 --> [v2:172.21.15.59:6824/648258563,v1:172.21.15.59:6825/648258563] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f13fc073e50 con 0x7f13fc071360 2023-11-24T08:00:29.620 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:29.613+0000 7f14137fe700 1 -- 172.21.15.59:0/3046055581 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_get_version_reply(handle=1 version=90) v2 ==== 24+0+0 (secure 0 0 0) 0x7f140408dac0 con 0x7f141c0b49b0 2023-11-24T08:00:29.620 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:29.613+0000 7f141affd700 1 --2- 172.21.15.59:0/3046055581 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f13fc06b500 0x7f13fc06d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:00:29.620 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:29.613+0000 7f141b7fe700 1 --2- 172.21.15.59:0/3046055581 >> [v2:172.21.15.59:6824/648258563,v1:172.21.15.59:6825/648258563] conn(0x7f13fc071360 0x7f13fc073780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:00:29.622 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:29.613+0000 7f141b7fe700 1 --2- 172.21.15.59:0/3046055581 >> [v2:172.21.15.59:6824/648258563,v1:172.21.15.59:6825/648258563] conn(0x7f13fc071360 0x7f13fc073780 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:00:29.623 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:29.613+0000 7f141affd700 1 --2- 172.21.15.59:0/3046055581 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f13fc06b500 0x7f13fc06d9c0 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7f140c000f60 tx=0x7f140c007ce0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:00:29.623 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:29.614+0000 7f14137fe700 1 -- 172.21.15.59:0/3046055581 <== osd.1 v2:172.21.15.59:6824/648258563 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+26965 (crc 0 0 0) 0x7f13fc073e50 con 0x7f13fc071360 2023-11-24T08:00:29.631 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:29.629+0000 7f14216fa700 1 -- 172.21.15.59:0/3046055581 --> [v2:172.21.15.59:6824/648258563,v1:172.21.15.59:6825/648258563] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f141c0b5170 con 0x7f13fc071360 2023-11-24T08:00:29.631 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:29.629+0000 7f14137fe700 1 -- 172.21.15.59:0/3046055581 <== osd.1 v2:172.21.15.59:6824/648258563 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+97 (crc 0 0 0) 0x7f1414009a70 con 0x7f13fc071360 2023-11-24T08:00:29.632 INFO:teuthology.orchestra.run.smithi059.stdout:{ 2023-11-24T08:00:29.632 INFO:teuthology.orchestra.run.smithi059.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2023-11-24T08:00:29.632 INFO:teuthology.orchestra.run.smithi059.stdout:} 2023-11-24T08:00:29.633 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:29.630+0000 7f14216fa700 1 -- 172.21.15.59:0/3046055581 >> [v2:172.21.15.59:6824/648258563,v1:172.21.15.59:6825/648258563] conn(0x7f13fc071360 msgr2=0x7f13fc073780 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:00:29.633 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:29.630+0000 7f14216fa700 1 --2- 172.21.15.59:0/3046055581 >> [v2:172.21.15.59:6824/648258563,v1:172.21.15.59:6825/648258563] conn(0x7f13fc071360 0x7f13fc073780 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:00:29.633 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:29.630+0000 7f14216fa700 1 -- 172.21.15.59:0/3046055581 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f13fc06b500 msgr2=0x7f13fc06d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:00:29.633 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:29.630+0000 7f14216fa700 1 --2- 172.21.15.59:0/3046055581 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f13fc06b500 0x7f13fc06d9c0 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7f140c000f60 tx=0x7f140c007ce0 comp rx=0 tx=0).stop 2023-11-24T08:00:29.634 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:29.630+0000 7f14216fa700 1 -- 172.21.15.59:0/3046055581 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f141c0b49b0 msgr2=0x7f141c159f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:00:29.634 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:29.630+0000 7f14216fa700 1 --2- 172.21.15.59:0/3046055581 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f141c0b49b0 0x7f141c159f60 secure :-1 s=READY pgs=1143 cs=0 l=1 rev1=1 crypto rx=0x7f1404009130 tx=0x7f1404004750 comp rx=0 tx=0).stop 2023-11-24T08:00:29.634 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:29.630+0000 7f141b7fe700 1 -- 172.21.15.59:0/3046055581 reap_dead start 2023-11-24T08:00:29.634 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:29.630+0000 7f14216fa700 1 -- 172.21.15.59:0/3046055581 shutdown_connections 2023-11-24T08:00:29.634 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:29.630+0000 7f14216fa700 1 -- 172.21.15.59:0/3046055581 >> 172.21.15.59:0/3046055581 conn(0x7f141c09d0c0 msgr2=0x7f141c1611a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:00:29.634 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:29.630+0000 7f14216fa700 1 -- 172.21.15.59:0/3046055581 shutdown_connections 2023-11-24T08:00:29.634 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:29.630+0000 7f14216fa700 1 -- 172.21.15.59:0/3046055581 wait complete. 2023-11-24T08:00:29.644 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 1 2023-11-24T08:00:29.769 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:29.767+0000 7efed0ce8700 1 Processor -- start 2023-11-24T08:00:29.770 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:29.767+0000 7efed0ce8700 1 -- start start 2023-11-24T08:00:29.772 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:29.768+0000 7efed0ce8700 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7efecc0d16e0 0x7efecc0affb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:00:29.772 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:29.768+0000 7efed0ce8700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7efecc0ada10 0x7efecc0adea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:00:29.772 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:29.768+0000 7efed0ce8700 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efecc0d1b40 con 0x7efecc06afd0 2023-11-24T08:00:29.772 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:29.768+0000 7efed0ce8700 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efecc0b04f0 con 0x7efecc0ada10 2023-11-24T08:00:29.773 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:29.768+0000 7efed0ce8700 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efecc070490 con 0x7efecc0d16e0 2023-11-24T08:00:29.773 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:29.769+0000 7efecad9d700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7efecc0ada10 0x7efecc0adea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:00:29.773 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:29.769+0000 7efecad9d700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7efecc0ada10 0x7efecc0adea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:56630/0 (socket says 172.21.15.59:56630) 2023-11-24T08:00:29.773 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:29.769+0000 7efeca59c700 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7efecc06afd0 0x7efecc06b3b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:60100/0 (socket says 172.21.15.59:60100) 2023-11-24T08:00:29.773 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:29.769+0000 7efecad9d700 1 -- 172.21.15.59:0/2075108189 learned_addr learned my addr 172.21.15.59:0/2075108189 (peer_addr_for_me v2:172.21.15.59:0/0) 2023-11-24T08:00:29.773 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:29.769+0000 7efec9d9b700 1 --2- 172.21.15.59:0/2075108189 >> v2:172.21.15.76:3300/0 conn(0x7efecc0d16e0 0x7efecc0affb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:00:29.773 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:29.769+0000 7efecad9d700 1 -- 172.21.15.59:0/2075108189 >> v2:172.21.15.76:3300/0 conn(0x7efecc0d16e0 msgr2=0x7efecc0affb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:00:29.773 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:29.769+0000 7efecad9d700 1 --2- 172.21.15.59:0/2075108189 >> v2:172.21.15.76:3300/0 conn(0x7efecc0d16e0 0x7efecc0affb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:00:29.774 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:29.769+0000 7efecad9d700 1 -- 172.21.15.59:0/2075108189 >> v1:172.21.15.59:6789/0 conn(0x7efecc06afd0 legacy=0x7efecc06b3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:00:29.774 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:29.769+0000 7efecad9d700 1 -- 172.21.15.59:0/2075108189 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efecc0af9f0 con 0x7efecc0ada10 2023-11-24T08:00:29.774 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:29.769+0000 7efec9d9b700 1 --2- 172.21.15.59:0/2075108189 >> v2:172.21.15.76:3300/0 conn(0x7efecc0d16e0 0x7efecc0affb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-24T08:00:29.774 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:29.769+0000 7efecad9d700 1 --2- 172.21.15.59:0/2075108189 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7efecc0ada10 0x7efecc0adea0 secure :-1 s=READY pgs=1145 cs=0 l=1 rev1=1 crypto rx=0x7efecc06fdd0 tx=0x7efec400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d24da66b869e85c8 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:00:29.774 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:29.769+0000 7efec959a700 1 -- 172.21.15.59:0/2075108189 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efec400e070 con 0x7efecc0ada10 2023-11-24T08:00:29.774 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:29.769+0000 7efec959a700 1 -- 172.21.15.59:0/2075108189 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efec400b9c0 con 0x7efecc0ada10 2023-11-24T08:00:29.774 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:29.769+0000 7efec959a700 1 -- 172.21.15.59:0/2075108189 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efec400bd70 con 0x7efecc0ada10 2023-11-24T08:00:29.774 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:29.770+0000 7efed0ce8700 1 -- 172.21.15.59:0/2075108189 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7efecc0ada10 msgr2=0x7efecc0adea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:00:29.775 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:29.770+0000 7efed0ce8700 1 --2- 172.21.15.59:0/2075108189 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7efecc0ada10 0x7efecc0adea0 secure :-1 s=READY pgs=1145 cs=0 l=1 rev1=1 crypto rx=0x7efecc06fdd0 tx=0x7efec400b330 comp rx=0 tx=0).stop 2023-11-24T08:00:29.775 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:29.770+0000 7efed0ce8700 1 -- 172.21.15.59:0/2075108189 shutdown_connections 2023-11-24T08:00:29.775 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:29.770+0000 7efed0ce8700 1 --2- 172.21.15.59:0/2075108189 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7efecc0ada10 0x7efecc0adea0 unknown :-1 s=CLOSED pgs=1145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:00:29.775 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:29.770+0000 7efed0ce8700 1 --2- 172.21.15.59:0/2075108189 >> v2:172.21.15.76:3300/0 conn(0x7efecc0d16e0 0x7efecc0affb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:00:29.775 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:29.770+0000 7efed0ce8700 1 -- 172.21.15.59:0/2075108189 >> 172.21.15.59:0/2075108189 conn(0x7efecc0d0bc0 msgr2=0x7efecc0d0fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:00:29.775 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:29.770+0000 7efed0ce8700 1 -- 172.21.15.59:0/2075108189 shutdown_connections 2023-11-24T08:00:29.775 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:29.770+0000 7efed0ce8700 1 -- 172.21.15.59:0/2075108189 wait complete. 2023-11-24T08:00:29.778 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:29.771+0000 7efed0ce8700 1 Processor -- start 2023-11-24T08:00:29.778 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:29.771+0000 7efed0ce8700 1 -- start start 2023-11-24T08:00:29.778 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:29.772+0000 7efed0ce8700 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7efecc06afd0 0x7efecc149090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:00:29.778 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:29.772+0000 7efed0ce8700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7efecc0ada10 0x7efecc145d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:00:29.779 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:29.772+0000 7efed0ce8700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7efecc0d16e0 0x7efecc0cd5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:00:29.779 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:29.772+0000 7efed0ce8700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efecc0d0920 con 0x7efecc0d16e0 2023-11-24T08:00:29.779 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:29.772+0000 7efed0ce8700 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efecc1495d0 con 0x7efecc0ada10 2023-11-24T08:00:29.779 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:29.772+0000 7efed0ce8700 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efecc1539b0 con 0x7efecc06afd0 2023-11-24T08:00:29.779 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:29.772+0000 7efecad9d700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7efecc0d16e0 0x7efecc0cd5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:00:29.779 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:29.772+0000 7efecad9d700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7efecc0d16e0 0x7efecc0cd5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:56636/0 (socket says 172.21.15.59:56636) 2023-11-24T08:00:29.779 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:29.772+0000 7efecad9d700 1 -- 172.21.15.59:0/1013786106 learned_addr learned my addr 172.21.15.59:0/1013786106 (peer_addr_for_me v2:172.21.15.59:0/0) 2023-11-24T08:00:29.779 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:29.772+0000 7efec9d9b700 1 --2- 172.21.15.59:0/1013786106 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7efecc0ada10 0x7efecc145d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:00:29.780 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:29.772+0000 7efeca59c700 1 --2- 172.21.15.59:0/1013786106 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7efecc06afd0 0x7efecc149090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:00:29.780 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:29.772+0000 7efecad9d700 1 -- 172.21.15.59:0/1013786106 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7efecc06afd0 msgr2=0x7efecc149090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:00:29.780 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:29.772+0000 7efecad9d700 1 --2- 172.21.15.59:0/1013786106 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7efecc06afd0 0x7efecc149090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:00:29.780 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:29.772+0000 7efecad9d700 1 -- 172.21.15.59:0/1013786106 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7efecc0ada10 msgr2=0x7efecc145d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:00:29.780 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:29.772+0000 7efecad9d700 1 --2- 172.21.15.59:0/1013786106 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7efecc0ada10 0x7efecc145d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:00:29.780 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:29.772+0000 7efecad9d700 1 -- 172.21.15.59:0/1013786106 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efec4009b30 con 0x7efecc0d16e0 2023-11-24T08:00:29.780 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:29.772+0000 7efec9d9b700 1 --2- 172.21.15.59:0/1013786106 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7efecc0ada10 0x7efecc145d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-24T08:00:29.781 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:29.773+0000 7efecad9d700 1 --2- 172.21.15.59:0/1013786106 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7efecc0d16e0 0x7efecc0cd5d0 secure :-1 s=READY pgs=1146 cs=0 l=1 rev1=1 crypto rx=0x7efec400b370 tx=0x7efec4004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:00:29.781 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:29.773+0000 7efec2ffd700 1 -- 172.21.15.59:0/1013786106 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efec40099b0 con 0x7efecc0d16e0 2023-11-24T08:00:29.781 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:29.773+0000 7efec2ffd700 1 -- 172.21.15.59:0/1013786106 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efec40092e0 con 0x7efecc0d16e0 2023-11-24T08:00:29.781 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:29.773+0000 7efec2ffd700 1 -- 172.21.15.59:0/1013786106 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efec401b650 con 0x7efecc0d16e0 2023-11-24T08:00:29.781 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:29.773+0000 7efed0ce8700 1 -- 172.21.15.59:0/1013786106 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efecc0cfd10 con 0x7efecc0d16e0 2023-11-24T08:00:29.781 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:29.773+0000 7efed0ce8700 1 -- 172.21.15.59:0/1013786106 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efecc0d02d0 con 0x7efecc0d16e0 2023-11-24T08:00:29.781 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:29.774+0000 7efed0ce8700 1 -- 172.21.15.59:0/1013786106 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efecc068670 con 0x7efecc0d16e0 2023-11-24T08:00:29.781 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:29.774+0000 7efec2ffd700 1 -- 172.21.15.59:0/1013786106 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 12) v1 ==== 89792+0+0 (secure 0 0 0) 0x7efec401b7b0 con 0x7efecc0d16e0 2023-11-24T08:00:29.782 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:29.774+0000 7efec2ffd700 1 --2- 172.21.15.59:0/1013786106 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7efeac06b500 0x7efeac06d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:00:29.782 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:29.775+0000 7efec2ffd700 1 -- 172.21.15.59:0/1013786106 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 4943+0+0 (secure 0 0 0) 0x7efec408b530 con 0x7efecc0d16e0 2023-11-24T08:00:29.782 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:29.775+0000 7efeca59c700 1 --2- 172.21.15.59:0/1013786106 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7efeac06b500 0x7efeac06d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:00:29.782 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:29.775+0000 7efeca59c700 1 --2- 172.21.15.59:0/1013786106 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7efeac06b500 0x7efeac06d9c0 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7efebc000f60 tx=0x7efebc007d10 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:00:29.782 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:29.779+0000 7efec2ffd700 1 -- 172.21.15.59:0/1013786106 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7efec4055f20 con 0x7efecc0d16e0 2023-11-24T08:00:29.980 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:29.979+0000 7efed0ce8700 1 -- 172.21.15.59:0/1013786106 --> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "1", "target": ["mon-mgr", ""]}) v1 -- 0x7efecc157440 con 0x7efeac06b500 2023-11-24T08:00:29.981 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:29.979+0000 7efec2ffd700 1 -- 172.21.15.59:0/1013786106 <== mgr.4100 v2:172.21.15.76:6832/4264046536 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 1 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7efecc157440 con 0x7efeac06b500 2023-11-24T08:00:29.983 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:29.981+0000 7efed0ce8700 1 -- 172.21.15.59:0/1013786106 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7efeac06b500 msgr2=0x7efeac06d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:00:29.983 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:29.981+0000 7efed0ce8700 1 --2- 172.21.15.59:0/1013786106 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7efeac06b500 0x7efeac06d9c0 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7efebc000f60 tx=0x7efebc007d10 comp rx=0 tx=0).stop 2023-11-24T08:00:29.983 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:29.981+0000 7efed0ce8700 1 -- 172.21.15.59:0/1013786106 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7efecc0d16e0 msgr2=0x7efecc0cd5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:00:29.984 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:29.981+0000 7efed0ce8700 1 --2- 172.21.15.59:0/1013786106 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7efecc0d16e0 0x7efecc0cd5d0 secure :-1 s=READY pgs=1146 cs=0 l=1 rev1=1 crypto rx=0x7efec400b370 tx=0x7efec4004ad0 comp rx=0 tx=0).stop 2023-11-24T08:00:29.984 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:29.981+0000 7efed0ce8700 1 -- 172.21.15.59:0/1013786106 shutdown_connections 2023-11-24T08:00:29.984 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:29.981+0000 7efed0ce8700 1 --2- 172.21.15.59:0/1013786106 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7efeac06b500 0x7efeac06d9c0 unknown :-1 s=CLOSED pgs=523 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:00:29.984 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:29.981+0000 7efed0ce8700 1 --2- 172.21.15.59:0/1013786106 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7efecc0d16e0 0x7efecc0cd5d0 unknown :-1 s=CLOSED pgs=1146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:00:29.984 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:29.981+0000 7efed0ce8700 1 --2- 172.21.15.59:0/1013786106 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7efecc06afd0 0x7efecc149090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:00:29.984 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:29.981+0000 7efed0ce8700 1 --2- 172.21.15.59:0/1013786106 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7efecc0ada10 0x7efecc145d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:00:29.984 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:29.981+0000 7efed0ce8700 1 -- 172.21.15.59:0/1013786106 >> 172.21.15.59:0/1013786106 conn(0x7efecc0d0bc0 msgr2=0x7efecc15d600 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:00:29.985 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:29.981+0000 7efed0ce8700 1 -- 172.21.15.59:0/1013786106 shutdown_connections 2023-11-24T08:00:29.985 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:29.981+0000 7efed0ce8700 1 -- 172.21.15.59:0/1013786106 wait complete. 2023-11-24T08:00:29.985 INFO:teuthology.orchestra.run.smithi059.stderr:instructed osd(s) 1 to deep-scrub 2023-11-24T08:00:29.995 INFO:tasks.ceph:Scrubbing osd.2 2023-11-24T08:00:29.996 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 config set osd_debug_deep_scrub_sleep 0 2023-11-24T08:00:30.124 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:30.122+0000 7f8de1c1b700 1 Processor -- start 2023-11-24T08:00:30.124 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:30.122+0000 7f8de1c1b700 1 -- start start 2023-11-24T08:00:30.126 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:30.123+0000 7f8de1c1b700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f8ddc1447a0 0x7f8ddc071b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:00:30.126 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:30.123+0000 7f8de1c1b700 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8ddc06dbd0 con 0x7f8ddc072ab0 2023-11-24T08:00:30.127 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:30.123+0000 7f8de1c1b700 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8ddc15f570 con 0x7f8ddc06f070 2023-11-24T08:00:30.127 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:30.123+0000 7f8de1c1b700 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8ddc15b700 con 0x7f8ddc1447a0 2023-11-24T08:00:30.127 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:30.123+0000 7f8ddb7fe700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f8ddc1447a0 0x7f8ddc071b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:00:30.127 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:30.123+0000 7f8ddaffd700 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7f8ddc072ab0 0x7f8ddc15e750 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:60114/0 (socket says 172.21.15.59:60114) 2023-11-24T08:00:30.127 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:30.123+0000 7f8ddaffd700 1 -- 172.21.15.59:0/1142216352 learned_addr learned my addr 172.21.15.59:0/1142216352 (peer_addr_for_me v1:172.21.15.59:0/0) 2023-11-24T08:00:30.127 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:30.124+0000 7f8ddb7fe700 1 -- 172.21.15.59:0/1142216352 >> v1:172.21.15.76:6789/0 conn(0x7f8ddc06f070 legacy=0x7f8ddc15ee60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:00:30.127 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:30.124+0000 7f8ddb7fe700 1 -- 172.21.15.59:0/1142216352 >> v1:172.21.15.59:6789/0 conn(0x7f8ddc072ab0 legacy=0x7f8ddc15e750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:00:30.128 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:30.124+0000 7f8dda7fc700 1 -- 172.21.15.59:0/1142216352 <== mon.0 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2102369431 0 0) 0x7f8ddc06dbd0 con 0x7f8ddc072ab0 2023-11-24T08:00:30.128 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:30.124+0000 7f8ddb7fe700 1 -- 172.21.15.59:0/1142216352 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8ddc15b9e0 con 0x7f8ddc1447a0 2023-11-24T08:00:30.128 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:30.124+0000 7f8ddb7fe700 1 --2- 172.21.15.59:0/1142216352 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8ddc1447a0 0x7f8ddc071b90 secure :-1 s=READY pgs=1148 cs=0 l=1 rev1=1 crypto rx=0x7f8dcc00bae0 tx=0x7f8dcc00bdf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2d807ef4681c8226 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:00:30.128 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:30.124+0000 7f8dda7fc700 1 -- 172.21.15.59:0/1142216352 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8dcc00dd30 con 0x7f8ddc1447a0 2023-11-24T08:00:30.128 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:30.124+0000 7f8dda7fc700 1 -- 172.21.15.59:0/1142216352 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8dcc00de90 con 0x7f8ddc1447a0 2023-11-24T08:00:30.128 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:30.124+0000 7f8dda7fc700 1 -- 172.21.15.59:0/1142216352 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8dcc010690 con 0x7f8ddc1447a0 2023-11-24T08:00:30.128 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:30.124+0000 7f8de1c1b700 1 -- 172.21.15.59:0/1142216352 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8ddc1447a0 msgr2=0x7f8ddc071b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:00:30.129 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:30.124+0000 7f8de1c1b700 1 --2- 172.21.15.59:0/1142216352 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8ddc1447a0 0x7f8ddc071b90 secure :-1 s=READY pgs=1148 cs=0 l=1 rev1=1 crypto rx=0x7f8dcc00bae0 tx=0x7f8dcc00bdf0 comp rx=0 tx=0).stop 2023-11-24T08:00:30.129 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:30.125+0000 7f8de1c1b700 1 -- 172.21.15.59:0/1142216352 shutdown_connections 2023-11-24T08:00:30.129 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:30.125+0000 7f8de1c1b700 1 --2- 172.21.15.59:0/1142216352 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8ddc1447a0 0x7f8ddc071b90 unknown :-1 s=CLOSED pgs=1148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:00:30.129 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:30.125+0000 7f8de1c1b700 1 -- 172.21.15.59:0/1142216352 >> 172.21.15.59:0/1142216352 conn(0x7f8ddc0b2760 msgr2=0x7f8ddc0b2b70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:00:30.129 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:30.125+0000 7f8de1c1b700 1 -- 172.21.15.59:0/1142216352 shutdown_connections 2023-11-24T08:00:30.130 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:30.125+0000 7f8de1c1b700 1 -- 172.21.15.59:0/1142216352 wait complete. 2023-11-24T08:00:30.133 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:30.126+0000 7f8de1c1b700 1 Processor -- start 2023-11-24T08:00:30.133 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:30.126+0000 7f8de1c1b700 1 -- start start 2023-11-24T08:00:30.133 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:30.127+0000 7f8de1c1b700 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f8ddc06f070 0x7f8ddc15a700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:00:30.133 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:30.127+0000 7f8de1c1b700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8ddc072ab0 0x7f8ddc136ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:00:30.134 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:30.127+0000 7f8de1c1b700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8ddc1447a0 0x7f8ddc0cf6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:00:30.134 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:30.127+0000 7f8de1c1b700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ddc15ac40 con 0x7f8ddc072ab0 2023-11-24T08:00:30.134 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:30.127+0000 7f8de1c1b700 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ddc134de0 con 0x7f8ddc1447a0 2023-11-24T08:00:30.134 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:30.127+0000 7f8de1c1b700 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8ddc1350c0 con 0x7f8ddc06f070 2023-11-24T08:00:30.134 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:30.127+0000 7f8ddaffd700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8ddc072ab0 0x7f8ddc136ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:00:30.134 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:30.127+0000 7f8ddaffd700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8ddc072ab0 0x7f8ddc136ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:56648/0 (socket says 172.21.15.59:56648) 2023-11-24T08:00:30.134 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:30.127+0000 7f8ddaffd700 1 -- 172.21.15.59:0/2390884311 learned_addr learned my addr 172.21.15.59:0/2390884311 (peer_addr_for_me v2:172.21.15.59:0/0) 2023-11-24T08:00:30.135 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:30.127+0000 7f8ddbfff700 1 --2- 172.21.15.59:0/2390884311 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8ddc1447a0 0x7f8ddc0cf6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:00:30.135 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:30.127+0000 7f8ddb7fe700 1 --2- 172.21.15.59:0/2390884311 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f8ddc06f070 0x7f8ddc15a700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:00:30.135 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:30.127+0000 7f8ddaffd700 1 -- 172.21.15.59:0/2390884311 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f8ddc06f070 msgr2=0x7f8ddc15a700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:00:30.135 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:30.127+0000 7f8ddaffd700 1 --2- 172.21.15.59:0/2390884311 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f8ddc06f070 0x7f8ddc15a700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:00:30.135 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:30.127+0000 7f8ddaffd700 1 -- 172.21.15.59:0/2390884311 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8ddc1447a0 msgr2=0x7f8ddc0cf6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:00:30.135 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:30.127+0000 7f8ddaffd700 1 --2- 172.21.15.59:0/2390884311 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8ddc1447a0 0x7f8ddc0cf6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:00:30.135 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:30.127+0000 7f8ddaffd700 1 -- 172.21.15.59:0/2390884311 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8dcc00b740 con 0x7f8ddc072ab0 2023-11-24T08:00:30.136 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:30.127+0000 7f8ddaffd700 1 --2- 172.21.15.59:0/2390884311 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8ddc072ab0 0x7f8ddc136ce0 secure :-1 s=READY pgs=1149 cs=0 l=1 rev1=1 crypto rx=0x7f8dc400c520 tx=0x7f8dc400c830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:00:30.136 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:30.127+0000 7f8dd3fff700 1 -- 172.21.15.59:0/2390884311 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8dc4004c80 con 0x7f8ddc072ab0 2023-11-24T08:00:30.136 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:30.127+0000 7f8dd3fff700 1 -- 172.21.15.59:0/2390884311 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8dc4004e00 con 0x7f8ddc072ab0 2023-11-24T08:00:30.136 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:30.127+0000 7f8de1c1b700 1 -- 172.21.15.59:0/2390884311 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8ddc1353a0 con 0x7f8ddc072ab0 2023-11-24T08:00:30.136 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:30.127+0000 7f8dd3fff700 1 -- 172.21.15.59:0/2390884311 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8dc4007310 con 0x7f8ddc072ab0 2023-11-24T08:00:30.136 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:30.128+0000 7f8de1c1b700 1 -- 172.21.15.59:0/2390884311 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8ddc135960 con 0x7f8ddc072ab0 2023-11-24T08:00:30.136 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:30.128+0000 7f8de1c1b700 1 -- 172.21.15.59:0/2390884311 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f8ddc15bfa0 con 0x7f8ddc072ab0 2023-11-24T08:00:30.137 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:30.129+0000 7f8dd3fff700 1 -- 172.21.15.59:0/2390884311 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 12) v1 ==== 89792+0+0 (secure 0 0 0) 0x7f8dc4007470 con 0x7f8ddc072ab0 2023-11-24T08:00:30.137 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:30.129+0000 7f8dd3fff700 1 --2- 172.21.15.59:0/2390884311 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f8dbc06b500 0x7f8dbc06d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:00:30.137 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:30.129+0000 7f8dd3fff700 1 -- 172.21.15.59:0/2390884311 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f8dc4089aa0 con 0x7f8ddc072ab0 2023-11-24T08:00:30.137 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:30.129+0000 7f8dd3fff700 1 --2- 172.21.15.59:0/2390884311 >> [v2:172.21.15.59:6808/1840430654,v1:172.21.15.59:6809/1840430654] conn(0x7f8dbc071360 0x7f8dbc073780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:00:30.137 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:30.129+0000 7f8dd3fff700 1 -- 172.21.15.59:0/2390884311 --> [v2:172.21.15.59:6808/1840430654,v1:172.21.15.59:6809/1840430654] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f8dbc073e50 con 0x7f8dbc071360 2023-11-24T08:00:30.137 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:30.130+0000 7f8dd3fff700 1 -- 172.21.15.59:0/2390884311 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_get_version_reply(handle=1 version=90) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8dc408e2a0 con 0x7f8ddc072ab0 2023-11-24T08:00:30.137 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:30.130+0000 7f8ddb7fe700 1 --2- 172.21.15.59:0/2390884311 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f8dbc06b500 0x7f8dbc06d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:00:30.137 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:30.130+0000 7f8ddbfff700 1 --2- 172.21.15.59:0/2390884311 >> [v2:172.21.15.59:6808/1840430654,v1:172.21.15.59:6809/1840430654] conn(0x7f8dbc071360 0x7f8dbc073780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:00:30.138 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:30.130+0000 7f8ddbfff700 1 --2- 172.21.15.59:0/2390884311 >> [v2:172.21.15.59:6808/1840430654,v1:172.21.15.59:6809/1840430654] conn(0x7f8dbc071360 0x7f8dbc073780 crc :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:00:30.138 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:30.130+0000 7f8ddb7fe700 1 --2- 172.21.15.59:0/2390884311 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f8dbc06b500 0x7f8dbc06d9c0 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7f8dcc00b2c0 tx=0x7f8dcc013000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:00:30.138 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:30.131+0000 7f8dd3fff700 1 -- 172.21.15.59:0/2390884311 <== osd.2 v2:172.21.15.59:6808/1840430654 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+26965 (crc 0 0 0) 0x7f8dbc073e50 con 0x7f8dbc071360 2023-11-24T08:00:30.148 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:30.146+0000 7f8de1c1b700 1 -- 172.21.15.59:0/2390884311 --> [v2:172.21.15.59:6808/1840430654,v1:172.21.15.59:6809/1840430654] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f8ddc068670 con 0x7f8dbc071360 2023-11-24T08:00:30.148 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:30.146+0000 7f8dd3fff700 1 -- 172.21.15.59:0/2390884311 <== osd.2 v2:172.21.15.59:6808/1840430654 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+97 (crc 0 0 0) 0x7f8ddc068670 con 0x7f8dbc071360 2023-11-24T08:00:30.148 INFO:teuthology.orchestra.run.smithi059.stdout:{ 2023-11-24T08:00:30.149 INFO:teuthology.orchestra.run.smithi059.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2023-11-24T08:00:30.149 INFO:teuthology.orchestra.run.smithi059.stdout:} 2023-11-24T08:00:30.149 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:30.146+0000 7f8de1c1b700 1 -- 172.21.15.59:0/2390884311 >> [v2:172.21.15.59:6808/1840430654,v1:172.21.15.59:6809/1840430654] conn(0x7f8dbc071360 msgr2=0x7f8dbc073780 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:00:30.150 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:30.146+0000 7f8de1c1b700 1 --2- 172.21.15.59:0/2390884311 >> [v2:172.21.15.59:6808/1840430654,v1:172.21.15.59:6809/1840430654] conn(0x7f8dbc071360 0x7f8dbc073780 crc :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:00:30.150 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:30.146+0000 7f8de1c1b700 1 -- 172.21.15.59:0/2390884311 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f8dbc06b500 msgr2=0x7f8dbc06d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:00:30.150 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:30.146+0000 7f8de1c1b700 1 --2- 172.21.15.59:0/2390884311 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f8dbc06b500 0x7f8dbc06d9c0 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7f8dcc00b2c0 tx=0x7f8dcc013000 comp rx=0 tx=0).stop 2023-11-24T08:00:30.150 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:30.146+0000 7f8de1c1b700 1 -- 172.21.15.59:0/2390884311 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8ddc072ab0 msgr2=0x7f8ddc136ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:00:30.150 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:30.146+0000 7f8de1c1b700 1 --2- 172.21.15.59:0/2390884311 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8ddc072ab0 0x7f8ddc136ce0 secure :-1 s=READY pgs=1149 cs=0 l=1 rev1=1 crypto rx=0x7f8dc400c520 tx=0x7f8dc400c830 comp rx=0 tx=0).stop 2023-11-24T08:00:30.150 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:30.146+0000 7f8ddbfff700 1 -- 172.21.15.59:0/2390884311 reap_dead start 2023-11-24T08:00:30.150 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:30.146+0000 7f8de1c1b700 1 -- 172.21.15.59:0/2390884311 shutdown_connections 2023-11-24T08:00:30.151 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:30.146+0000 7f8de1c1b700 1 -- 172.21.15.59:0/2390884311 >> 172.21.15.59:0/2390884311 conn(0x7f8ddc0b2760 msgr2=0x7f8ddc0b1630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:00:30.151 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:30.146+0000 7f8de1c1b700 1 -- 172.21.15.59:0/2390884311 shutdown_connections 2023-11-24T08:00:30.151 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:30.146+0000 7f8de1c1b700 1 -- 172.21.15.59:0/2390884311 wait complete. 2023-11-24T08:00:30.160 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 2 2023-11-24T08:00:30.286 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:30.284+0000 7f3fcb477700 1 Processor -- start 2023-11-24T08:00:30.286 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:30.284+0000 7f3fcb477700 1 -- start start 2023-11-24T08:00:30.294 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:30.285+0000 7f3fcb477700 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f3fc4143ae0 0x7f3fc40b51b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:00:30.295 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:30.285+0000 7f3fcb477700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f3fc41440f0 0x7f3fc4071bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:00:30.295 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:30.285+0000 7f3fcb477700 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3fc4143ec0 con 0x7f3fc40720f0 2023-11-24T08:00:30.295 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:30.285+0000 7f3fcb477700 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3fc40b40a0 con 0x7f3fc41440f0 2023-11-24T08:00:30.295 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:30.285+0000 7f3fcb477700 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3fc4072580 con 0x7f3fc4143ae0 2023-11-24T08:00:30.295 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:30.285+0000 7f3fc8a12700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f3fc41440f0 0x7f3fc4071bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:00:30.295 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:30.285+0000 7f3fc9a14700 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7f3fc40720f0 0x7f3fc415e8b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:60134/0 (socket says 172.21.15.59:60134) 2023-11-24T08:00:30.295 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:30.285+0000 7f3fc9a14700 1 -- 172.21.15.59:0/1690790821 learned_addr learned my addr 172.21.15.59:0/1690790821 (peer_addr_for_me v1:172.21.15.59:0/0) 2023-11-24T08:00:30.296 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:30.285+0000 7f3fc9213700 1 --2- 172.21.15.59:0/1690790821 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f3fc4143ae0 0x7f3fc40b51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:00:30.296 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:30.285+0000 7f3fc8a12700 1 -- 172.21.15.59:0/1690790821 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f3fc4143ae0 msgr2=0x7f3fc40b51b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:00:30.296 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:30.285+0000 7f3fc8a12700 1 --2- 172.21.15.59:0/1690790821 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f3fc4143ae0 0x7f3fc40b51b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:00:30.296 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:30.285+0000 7f3fc8a12700 1 -- 172.21.15.59:0/1690790821 >> v1:172.21.15.59:6789/0 conn(0x7f3fc40720f0 legacy=0x7f3fc415e8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:00:30.296 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:30.285+0000 7f3fbbfff700 1 -- 172.21.15.59:0/1690790821 <== mon.0 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3309614666 0 0) 0x7f3fc4143ec0 con 0x7f3fc40720f0 2023-11-24T08:00:30.296 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:30.285+0000 7f3fc8a12700 1 -- 172.21.15.59:0/1690790821 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3fc40b3ae0 con 0x7f3fc41440f0 2023-11-24T08:00:30.296 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:30.286+0000 7f3fc8a12700 1 --2- 172.21.15.59:0/1690790821 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3fc41440f0 0x7f3fc4071bb0 secure :-1 s=READY pgs=1151 cs=0 l=1 rev1=1 crypto rx=0x7f3fb4009840 tx=0x7f3fb4009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=840d93341e427c07 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:00:30.297 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:30.286+0000 7f3fbbfff700 1 -- 172.21.15.59:0/1690790821 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3fb4011830 con 0x7f3fc41440f0 2023-11-24T08:00:30.297 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:30.286+0000 7f3fbbfff700 1 -- 172.21.15.59:0/1690790821 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3fb4011990 con 0x7f3fc41440f0 2023-11-24T08:00:30.297 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:30.286+0000 7f3fbbfff700 1 -- 172.21.15.59:0/1690790821 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3fb4011d20 con 0x7f3fc41440f0 2023-11-24T08:00:30.297 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:30.286+0000 7f3fcb477700 1 -- 172.21.15.59:0/1690790821 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3fc41440f0 msgr2=0x7f3fc4071bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:00:30.297 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:30.286+0000 7f3fcb477700 1 --2- 172.21.15.59:0/1690790821 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3fc41440f0 0x7f3fc4071bb0 secure :-1 s=READY pgs=1151 cs=0 l=1 rev1=1 crypto rx=0x7f3fb4009840 tx=0x7f3fb4009b50 comp rx=0 tx=0).stop 2023-11-24T08:00:30.297 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:30.286+0000 7f3fcb477700 1 -- 172.21.15.59:0/1690790821 shutdown_connections 2023-11-24T08:00:30.297 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:30.286+0000 7f3fcb477700 1 --2- 172.21.15.59:0/1690790821 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f3fc4143ae0 0x7f3fc40b51b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:00:30.298 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:30.286+0000 7f3fcb477700 1 --2- 172.21.15.59:0/1690790821 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3fc41440f0 0x7f3fc4071bb0 unknown :-1 s=CLOSED pgs=1151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:00:30.298 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:30.286+0000 7f3fcb477700 1 -- 172.21.15.59:0/1690790821 >> 172.21.15.59:0/1690790821 conn(0x7f3fc40b2740 msgr2=0x7f3fc40b2b20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:00:30.298 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:30.287+0000 7f3fcb477700 1 -- 172.21.15.59:0/1690790821 shutdown_connections 2023-11-24T08:00:30.298 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:30.287+0000 7f3fcb477700 1 -- 172.21.15.59:0/1690790821 wait complete. 2023-11-24T08:00:30.298 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:30.288+0000 7f3fcb477700 1 Processor -- start 2023-11-24T08:00:30.298 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:30.288+0000 7f3fcb477700 1 -- start start 2023-11-24T08:00:30.298 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:30.289+0000 7f3fcb477700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3fc40720f0 0x7f3fc40cc1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:00:30.298 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:30.289+0000 7f3fcb477700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3fc4143ae0 0x7f3fc4142580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:00:30.299 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:30.289+0000 7f3fcb477700 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f3fc41440f0 0x7f3fc4142ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:00:30.299 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:30.289+0000 7f3fcb477700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3fc415c2e0 con 0x7f3fc40720f0 2023-11-24T08:00:30.299 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:30.289+0000 7f3fcb477700 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3fc40cc6f0 con 0x7f3fc4143ae0 2023-11-24T08:00:30.299 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:30.289+0000 7f3fcb477700 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3fc415c000 con 0x7f3fc41440f0 2023-11-24T08:00:30.299 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:30.289+0000 7f3fc9213700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3fc40720f0 0x7f3fc40cc1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:00:30.299 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:30.289+0000 7f3fc9213700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3fc40720f0 0x7f3fc40cc1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:56662/0 (socket says 172.21.15.59:56662) 2023-11-24T08:00:30.299 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:30.289+0000 7f3fc9213700 1 -- 172.21.15.59:0/1095338063 learned_addr learned my addr 172.21.15.59:0/1095338063 (peer_addr_for_me v2:172.21.15.59:0/0) 2023-11-24T08:00:30.299 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:30.289+0000 7f3fc8a12700 1 --2- 172.21.15.59:0/1095338063 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3fc4143ae0 0x7f3fc4142580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:00:30.300 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:30.289+0000 7f3fc9a14700 1 --2- 172.21.15.59:0/1095338063 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f3fc41440f0 0x7f3fc4142ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:00:30.300 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:30.289+0000 7f3fc9213700 1 -- 172.21.15.59:0/1095338063 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f3fc41440f0 msgr2=0x7f3fc4142ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:00:30.300 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:30.289+0000 7f3fc9213700 1 --2- 172.21.15.59:0/1095338063 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f3fc41440f0 0x7f3fc4142ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:00:30.300 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:30.289+0000 7f3fc9213700 1 -- 172.21.15.59:0/1095338063 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3fc4143ae0 msgr2=0x7f3fc4142580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:00:30.300 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:30.289+0000 7f3fc9213700 1 --2- 172.21.15.59:0/1095338063 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3fc4143ae0 0x7f3fc4142580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:00:30.300 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:30.289+0000 7f3fc9213700 1 -- 172.21.15.59:0/1095338063 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3fb40094a0 con 0x7f3fc40720f0 2023-11-24T08:00:30.300 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:30.289+0000 7f3fc8a12700 1 --2- 172.21.15.59:0/1095338063 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3fc4143ae0 0x7f3fc4142580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-24T08:00:30.300 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:30.289+0000 7f3fc9213700 1 --2- 172.21.15.59:0/1095338063 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3fc40720f0 0x7f3fc40cc1b0 secure :-1 s=READY pgs=1152 cs=0 l=1 rev1=1 crypto rx=0x7f3fc000e810 tx=0x7f3fc000ebd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:00:30.301 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:30.289+0000 7f3fb9ffb700 1 -- 172.21.15.59:0/1095338063 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3fc0013610 con 0x7f3fc40720f0 2023-11-24T08:00:30.301 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:30.289+0000 7f3fb9ffb700 1 -- 172.21.15.59:0/1095338063 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3fc0013770 con 0x7f3fc40720f0 2023-11-24T08:00:30.301 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:30.289+0000 7f3fcb477700 1 -- 172.21.15.59:0/1095338063 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3fc414c060 con 0x7f3fc40720f0 2023-11-24T08:00:30.301 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:30.289+0000 7f3fb9ffb700 1 -- 172.21.15.59:0/1095338063 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3fc0013b20 con 0x7f3fc40720f0 2023-11-24T08:00:30.301 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:30.290+0000 7f3fcb477700 1 -- 172.21.15.59:0/1095338063 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3fc414c620 con 0x7f3fc40720f0 2023-11-24T08:00:30.301 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:30.290+0000 7f3fcb477700 1 -- 172.21.15.59:0/1095338063 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3fc414bec0 con 0x7f3fc40720f0 2023-11-24T08:00:30.301 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:30.291+0000 7f3fb9ffb700 1 -- 172.21.15.59:0/1095338063 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 12) v1 ==== 89792+0+0 (secure 0 0 0) 0x7f3fc0013c80 con 0x7f3fc40720f0 2023-11-24T08:00:30.301 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:30.291+0000 7f3fb9ffb700 1 --2- 172.21.15.59:0/1095338063 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f3fa806b500 0x7f3fa806d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:00:30.302 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:30.291+0000 7f3fb9ffb700 1 -- 172.21.15.59:0/1095338063 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f3fc0089cf0 con 0x7f3fc40720f0 2023-11-24T08:00:30.302 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:30.292+0000 7f3fc8a12700 1 --2- 172.21.15.59:0/1095338063 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f3fa806b500 0x7f3fa806d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:00:30.302 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:30.292+0000 7f3fc8a12700 1 --2- 172.21.15.59:0/1095338063 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f3fa806b500 0x7f3fa806d9c0 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7f3fb40090b0 tx=0x7f3fb4008030 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:00:30.302 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:30.296+0000 7f3fb9ffb700 1 -- 172.21.15.59:0/1095338063 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f3fc414bec0 con 0x7f3fc40720f0 2023-11-24T08:00:30.499 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:30.497+0000 7f3fcb477700 1 -- 172.21.15.59:0/1095338063 --> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "2", "target": ["mon-mgr", ""]}) v1 -- 0x7f3fc406dff0 con 0x7f3fa806b500 2023-11-24T08:00:30.499 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:30.498+0000 7f3fb9ffb700 1 -- 172.21.15.59:0/1095338063 <== mgr.4100 v2:172.21.15.76:6832/4264046536 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 2 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f3fb4011cb0 con 0x7f3fa806b500 2023-11-24T08:00:30.503 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:30.500+0000 7f3fcb477700 1 -- 172.21.15.59:0/1095338063 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f3fa806b500 msgr2=0x7f3fa806d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:00:30.503 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:30.500+0000 7f3fcb477700 1 --2- 172.21.15.59:0/1095338063 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f3fa806b500 0x7f3fa806d9c0 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7f3fb40090b0 tx=0x7f3fb4008030 comp rx=0 tx=0).stop 2023-11-24T08:00:30.503 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:30.500+0000 7f3fcb477700 1 -- 172.21.15.59:0/1095338063 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3fc40720f0 msgr2=0x7f3fc40cc1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:00:30.503 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:30.500+0000 7f3fcb477700 1 --2- 172.21.15.59:0/1095338063 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3fc40720f0 0x7f3fc40cc1b0 secure :-1 s=READY pgs=1152 cs=0 l=1 rev1=1 crypto rx=0x7f3fc000e810 tx=0x7f3fc000ebd0 comp rx=0 tx=0).stop 2023-11-24T08:00:30.503 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:30.500+0000 7f3fcb477700 1 -- 172.21.15.59:0/1095338063 shutdown_connections 2023-11-24T08:00:30.503 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:30.500+0000 7f3fcb477700 1 --2- 172.21.15.59:0/1095338063 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f3fa806b500 0x7f3fa806d9c0 unknown :-1 s=CLOSED pgs=525 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:00:30.504 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:30.500+0000 7f3fcb477700 1 --2- 172.21.15.59:0/1095338063 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f3fc41440f0 0x7f3fc4142ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:00:30.504 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:30.500+0000 7f3fcb477700 1 --2- 172.21.15.59:0/1095338063 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3fc40720f0 0x7f3fc40cc1b0 unknown :-1 s=CLOSED pgs=1152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:00:30.504 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:30.500+0000 7f3fcb477700 1 --2- 172.21.15.59:0/1095338063 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3fc4143ae0 0x7f3fc4142580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:00:30.504 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:30.500+0000 7f3fcb477700 1 -- 172.21.15.59:0/1095338063 >> 172.21.15.59:0/1095338063 conn(0x7f3fc40b2740 msgr2=0x7f3fc40b4430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:00:30.504 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:30.500+0000 7f3fcb477700 1 -- 172.21.15.59:0/1095338063 shutdown_connections 2023-11-24T08:00:30.504 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:30.500+0000 7f3fcb477700 1 -- 172.21.15.59:0/1095338063 wait complete. 2023-11-24T08:00:30.504 INFO:teuthology.orchestra.run.smithi059.stderr:instructed osd(s) 2 to deep-scrub 2023-11-24T08:00:30.514 INFO:tasks.ceph:Scrubbing osd.3 2023-11-24T08:00:30.514 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 config set osd_debug_deep_scrub_sleep 0 2023-11-24T08:00:30.644 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:30.642+0000 7fd24b8da700 1 Processor -- start 2023-11-24T08:00:30.644 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:30.642+0000 7fd24b8da700 1 -- start start 2023-11-24T08:00:30.646 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:30.643+0000 7fd24b8da700 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fd2441447a0 0x7fd244071b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:00:30.647 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:30.643+0000 7fd24b8da700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fd24406f070 0x7fd2440720d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:00:30.648 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:30.643+0000 7fd24b8da700 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd24406dbd0 con 0x7fd244072ab0 2023-11-24T08:00:30.648 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:30.643+0000 7fd24b8da700 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd24415b700 con 0x7fd24406f070 2023-11-24T08:00:30.648 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:30.643+0000 7fd24b8da700 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd24415b9e0 con 0x7fd2441447a0 2023-11-24T08:00:30.649 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:30.643+0000 7fd249e77700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fd24406f070 0x7fd2440720d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:00:30.649 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:30.643+0000 7fd249e77700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fd24406f070 0x7fd2440720d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:56674/0 (socket says 172.21.15.59:56674) 2023-11-24T08:00:30.649 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:30.643+0000 7fd249e77700 1 -- 172.21.15.59:0/301289431 learned_addr learned my addr 172.21.15.59:0/301289431 (peer_addr_for_me v2:172.21.15.59:0/0) 2023-11-24T08:00:30.649 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:30.643+0000 7fd249676700 1 --2- 172.21.15.59:0/301289431 >> v2:172.21.15.76:3300/0 conn(0x7fd2441447a0 0x7fd244071b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:00:30.649 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:30.643+0000 7fd249e77700 1 -- 172.21.15.59:0/301289431 >> v2:172.21.15.76:3300/0 conn(0x7fd2441447a0 msgr2=0x7fd244071b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:00:30.649 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:30.643+0000 7fd249e77700 1 --2- 172.21.15.59:0/301289431 >> v2:172.21.15.76:3300/0 conn(0x7fd2441447a0 0x7fd244071b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:00:30.649 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:30.643+0000 7fd249e77700 1 -- 172.21.15.59:0/301289431 >> v1:172.21.15.76:6789/0 conn(0x7fd244072ab0 legacy=0x7fd24415e750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:00:30.650 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:30.643+0000 7fd249e77700 1 -- 172.21.15.59:0/301289431 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd24415bcc0 con 0x7fd24406f070 2023-11-24T08:00:30.650 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:30.643+0000 7fd249676700 1 --2- 172.21.15.59:0/301289431 >> v2:172.21.15.76:3300/0 conn(0x7fd2441447a0 0x7fd244071b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T08:00:30.650 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:30.644+0000 7fd249e77700 1 --2- 172.21.15.59:0/301289431 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd24406f070 0x7fd2440720d0 secure :-1 s=READY pgs=1153 cs=0 l=1 rev1=1 crypto rx=0x7fd2440b05a0 tx=0x7fd23c00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=44cfe5f8c6c7f083 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:00:30.650 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:30.644+0000 7fd23bfff700 1 -- 172.21.15.59:0/301289431 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd23c00e070 con 0x7fd24406f070 2023-11-24T08:00:30.650 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:30.644+0000 7fd23bfff700 1 -- 172.21.15.59:0/301289431 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd23c003a20 con 0x7fd24406f070 2023-11-24T08:00:30.650 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:30.644+0000 7fd23bfff700 1 -- 172.21.15.59:0/301289431 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd23c003dd0 con 0x7fd24406f070 2023-11-24T08:00:30.650 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:30.644+0000 7fd24b8da700 1 -- 172.21.15.59:0/301289431 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd24406f070 msgr2=0x7fd2440720d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:00:30.650 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:30.644+0000 7fd24b8da700 1 --2- 172.21.15.59:0/301289431 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd24406f070 0x7fd2440720d0 secure :-1 s=READY pgs=1153 cs=0 l=1 rev1=1 crypto rx=0x7fd2440b05a0 tx=0x7fd23c00b330 comp rx=0 tx=0).stop 2023-11-24T08:00:30.651 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:30.644+0000 7fd24b8da700 1 -- 172.21.15.59:0/301289431 shutdown_connections 2023-11-24T08:00:30.651 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:30.644+0000 7fd24b8da700 1 --2- 172.21.15.59:0/301289431 >> v2:172.21.15.76:3300/0 conn(0x7fd2441447a0 0x7fd244071b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:00:30.651 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:30.644+0000 7fd24b8da700 1 --2- 172.21.15.59:0/301289431 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd24406f070 0x7fd2440720d0 unknown :-1 s=CLOSED pgs=1153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:00:30.651 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:30.644+0000 7fd24b8da700 1 -- 172.21.15.59:0/301289431 >> 172.21.15.59:0/301289431 conn(0x7fd2440b2760 msgr2=0x7fd2440b2b70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:00:30.651 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:30.644+0000 7fd24b8da700 1 -- 172.21.15.59:0/301289431 shutdown_connections 2023-11-24T08:00:30.651 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:30.644+0000 7fd24b8da700 1 -- 172.21.15.59:0/301289431 wait complete. 2023-11-24T08:00:30.655 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:30.646+0000 7fd24b8da700 1 Processor -- start 2023-11-24T08:00:30.655 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:30.646+0000 7fd24b8da700 1 -- start start 2023-11-24T08:00:30.655 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:30.646+0000 7fd24b8da700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd24406f070 0x7fd24415d110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:00:30.655 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:30.646+0000 7fd24b8da700 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fd244072ab0 0x7fd24413d8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:00:30.655 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:30.646+0000 7fd24b8da700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd2441447a0 0x7fd24413ff40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:00:30.655 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:30.646+0000 7fd24b8da700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd24415d810 con 0x7fd2441447a0 2023-11-24T08:00:30.656 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:30.646+0000 7fd24b8da700 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd244140830 con 0x7fd24406f070 2023-11-24T08:00:30.656 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:30.646+0000 7fd24b8da700 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd244140b10 con 0x7fd244072ab0 2023-11-24T08:00:30.656 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:30.647+0000 7fd249e77700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd2441447a0 0x7fd24413ff40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:00:30.656 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:30.647+0000 7fd249e77700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd2441447a0 0x7fd24413ff40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:56680/0 (socket says 172.21.15.59:56680) 2023-11-24T08:00:30.656 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:30.647+0000 7fd249e77700 1 -- 172.21.15.59:0/1328757567 learned_addr learned my addr 172.21.15.59:0/1328757567 (peer_addr_for_me v2:172.21.15.59:0/0) 2023-11-24T08:00:30.656 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:30.647+0000 7fd248e75700 1 --2- 172.21.15.59:0/1328757567 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fd244072ab0 0x7fd24413d8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:00:30.656 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:30.647+0000 7fd249676700 1 --2- 172.21.15.59:0/1328757567 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd24406f070 0x7fd24415d110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:00:30.657 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:30.647+0000 7fd249e77700 1 -- 172.21.15.59:0/1328757567 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fd244072ab0 msgr2=0x7fd24413d8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:00:30.657 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:30.647+0000 7fd249e77700 1 --2- 172.21.15.59:0/1328757567 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fd244072ab0 0x7fd24413d8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:00:30.657 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:30.647+0000 7fd249e77700 1 -- 172.21.15.59:0/1328757567 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd24406f070 msgr2=0x7fd24415d110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:00:30.657 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:30.647+0000 7fd249e77700 1 --2- 172.21.15.59:0/1328757567 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd24406f070 0x7fd24415d110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:00:30.657 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:30.647+0000 7fd249e77700 1 -- 172.21.15.59:0/1328757567 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd23c009b30 con 0x7fd2441447a0 2023-11-24T08:00:30.657 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:30.647+0000 7fd249e77700 1 --2- 172.21.15.59:0/1328757567 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd2441447a0 0x7fd24413ff40 secure :-1 s=READY pgs=1154 cs=0 l=1 rev1=1 crypto rx=0x7fd23c015040 tx=0x7fd23c004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:00:30.658 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:30.647+0000 7fd239ffb700 1 -- 172.21.15.59:0/1328757567 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd23c004440 con 0x7fd2441447a0 2023-11-24T08:00:30.658 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:30.647+0000 7fd239ffb700 1 -- 172.21.15.59:0/1328757567 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd23c0045c0 con 0x7fd2441447a0 2023-11-24T08:00:30.658 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:30.647+0000 7fd24b8da700 1 -- 172.21.15.59:0/1328757567 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd244140df0 con 0x7fd2441447a0 2023-11-24T08:00:30.658 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:30.647+0000 7fd239ffb700 1 -- 172.21.15.59:0/1328757567 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd23c004970 con 0x7fd2441447a0 2023-11-24T08:00:30.658 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:30.647+0000 7fd24b8da700 1 -- 172.21.15.59:0/1328757567 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd244140480 con 0x7fd2441447a0 2023-11-24T08:00:30.658 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:30.648+0000 7fd24b8da700 1 -- 172.21.15.59:0/1328757567 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fd24415b130 con 0x7fd2441447a0 2023-11-24T08:00:30.658 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:30.649+0000 7fd239ffb700 1 -- 172.21.15.59:0/1328757567 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 12) v1 ==== 89792+0+0 (secure 0 0 0) 0x7fd23c012430 con 0x7fd2441447a0 2023-11-24T08:00:30.659 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:30.649+0000 7fd239ffb700 1 --2- 172.21.15.59:0/1328757567 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7fd22806b500 0x7fd22806d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:00:30.659 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:30.650+0000 7fd239ffb700 1 -- 172.21.15.59:0/1328757567 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 4943+0+0 (secure 0 0 0) 0x7fd23c08b9c0 con 0x7fd2441447a0 2023-11-24T08:00:30.659 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:30.650+0000 7fd249676700 1 --2- 172.21.15.59:0/1328757567 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7fd22806b500 0x7fd22806d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:00:30.659 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:30.650+0000 7fd239ffb700 1 --2- 172.21.15.59:0/1328757567 >> [v2:172.21.15.59:6816/2504025850,v1:172.21.15.59:6817/2504025850] conn(0x7fd228071360 0x7fd228073780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:00:30.659 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:30.650+0000 7fd239ffb700 1 -- 172.21.15.59:0/1328757567 --> [v2:172.21.15.59:6816/2504025850,v1:172.21.15.59:6817/2504025850] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fd228073e50 con 0x7fd228071360 2023-11-24T08:00:30.659 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:30.650+0000 7fd239ffb700 1 -- 172.21.15.59:0/1328757567 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_get_version_reply(handle=1 version=90) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd23c08bc20 con 0x7fd2441447a0 2023-11-24T08:00:30.659 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:30.650+0000 7fd248e75700 1 --2- 172.21.15.59:0/1328757567 >> [v2:172.21.15.59:6816/2504025850,v1:172.21.15.59:6817/2504025850] conn(0x7fd228071360 0x7fd228073780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:00:30.660 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:30.650+0000 7fd248e75700 1 --2- 172.21.15.59:0/1328757567 >> [v2:172.21.15.59:6816/2504025850,v1:172.21.15.59:6817/2504025850] conn(0x7fd228071360 0x7fd228073780 crc :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:00:30.660 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:30.650+0000 7fd249676700 1 --2- 172.21.15.59:0/1328757567 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7fd22806b500 0x7fd22806d9c0 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7fd234006fd0 tx=0x7fd234008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:00:30.660 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:30.651+0000 7fd239ffb700 1 -- 172.21.15.59:0/1328757567 <== osd.3 v2:172.21.15.59:6816/2504025850 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+26965 (crc 0 0 0) 0x7fd228073e50 con 0x7fd228071360 2023-11-24T08:00:30.667 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:30.666+0000 7fd24b8da700 1 -- 172.21.15.59:0/1328757567 --> [v2:172.21.15.59:6816/2504025850,v1:172.21.15.59:6817/2504025850] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7fd244140610 con 0x7fd228071360 2023-11-24T08:00:30.667 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:30.666+0000 7fd239ffb700 1 -- 172.21.15.59:0/1328757567 <== osd.3 v2:172.21.15.59:6816/2504025850 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+97 (crc 0 0 0) 0x7fd240004fc0 con 0x7fd228071360 2023-11-24T08:00:30.668 INFO:teuthology.orchestra.run.smithi059.stdout:{ 2023-11-24T08:00:30.668 INFO:teuthology.orchestra.run.smithi059.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2023-11-24T08:00:30.668 INFO:teuthology.orchestra.run.smithi059.stdout:} 2023-11-24T08:00:30.669 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:30.666+0000 7fd24b8da700 1 -- 172.21.15.59:0/1328757567 >> [v2:172.21.15.59:6816/2504025850,v1:172.21.15.59:6817/2504025850] conn(0x7fd228071360 msgr2=0x7fd228073780 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:00:30.669 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:30.666+0000 7fd24b8da700 1 --2- 172.21.15.59:0/1328757567 >> [v2:172.21.15.59:6816/2504025850,v1:172.21.15.59:6817/2504025850] conn(0x7fd228071360 0x7fd228073780 crc :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:00:30.669 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:30.666+0000 7fd24b8da700 1 -- 172.21.15.59:0/1328757567 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7fd22806b500 msgr2=0x7fd22806d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:00:30.670 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:30.666+0000 7fd24b8da700 1 --2- 172.21.15.59:0/1328757567 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7fd22806b500 0x7fd22806d9c0 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7fd234006fd0 tx=0x7fd234008040 comp rx=0 tx=0).stop 2023-11-24T08:00:30.670 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:30.666+0000 7fd24b8da700 1 -- 172.21.15.59:0/1328757567 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd2441447a0 msgr2=0x7fd24413ff40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:00:30.670 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:30.666+0000 7fd24b8da700 1 --2- 172.21.15.59:0/1328757567 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd2441447a0 0x7fd24413ff40 secure :-1 s=READY pgs=1154 cs=0 l=1 rev1=1 crypto rx=0x7fd23c015040 tx=0x7fd23c004ad0 comp rx=0 tx=0).stop 2023-11-24T08:00:30.670 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:30.666+0000 7fd249e77700 1 -- 172.21.15.59:0/1328757567 reap_dead start 2023-11-24T08:00:30.670 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:30.666+0000 7fd24b8da700 1 -- 172.21.15.59:0/1328757567 shutdown_connections 2023-11-24T08:00:30.670 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:30.666+0000 7fd24b8da700 1 -- 172.21.15.59:0/1328757567 >> 172.21.15.59:0/1328757567 conn(0x7fd2440b2760 msgr2=0x7fd244071b90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:00:30.670 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:30.667+0000 7fd24b8da700 1 -- 172.21.15.59:0/1328757567 shutdown_connections 2023-11-24T08:00:30.671 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:30.667+0000 7fd24b8da700 1 -- 172.21.15.59:0/1328757567 wait complete. 2023-11-24T08:00:30.680 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 3 2023-11-24T08:00:30.805 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:30.803+0000 7fce35b15700 1 Processor -- start 2023-11-24T08:00:30.805 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:30.803+0000 7fce35b15700 1 -- start start 2023-11-24T08:00:30.809 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:30.804+0000 7fce35b15700 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fce30143ae0 0x7fce300b51b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:00:30.809 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:30.804+0000 7fce35b15700 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fce30143ec0 con 0x7fce301440f0 2023-11-24T08:00:30.809 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:30.804+0000 7fce35b15700 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fce300b3ae0 con 0x7fce300722c0 2023-11-24T08:00:30.809 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:30.804+0000 7fce35b15700 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fce300b40a0 con 0x7fce30143ae0 2023-11-24T08:00:30.809 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:30.804+0000 7fce2effd700 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7fce301440f0 0x7fce30071bb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:60160/0 (socket says 172.21.15.59:60160) 2023-11-24T08:00:30.810 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:30.804+0000 7fce2effd700 1 -- 172.21.15.59:0/1521510863 learned_addr learned my addr 172.21.15.59:0/1521510863 (peer_addr_for_me v1:172.21.15.59:0/0) 2023-11-24T08:00:30.810 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:30.804+0000 7fce2f7fe700 1 --2- 172.21.15.59:0/1521510863 >> v2:172.21.15.76:3300/0 conn(0x7fce30143ae0 0x7fce300b51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:00:30.810 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:30.805+0000 7fce2e7fc700 1 -- 172.21.15.59:0/1521510863 <== mon.0 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3341148157 0 0) 0x7fce30143ec0 con 0x7fce301440f0 2023-11-24T08:00:30.810 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:30.805+0000 7fce2e7fc700 1 -- 172.21.15.59:0/1521510863 --> v1:172.21.15.59:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fce1c0035b0 con 0x7fce301440f0 2023-11-24T08:00:30.810 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:30.805+0000 7fce2e7fc700 1 -- 172.21.15.59:0/1521510863 <== mon.0 v1:172.21.15.59:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3461727256 0 0) 0x7fce1c0035b0 con 0x7fce301440f0 2023-11-24T08:00:30.810 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:30.805+0000 7fce2e7fc700 1 -- 172.21.15.59:0/1521510863 >> v2:172.21.15.76:3300/0 conn(0x7fce30143ae0 msgr2=0x7fce300b51b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:00:30.811 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:30.805+0000 7fce2e7fc700 1 --2- 172.21.15.59:0/1521510863 >> v2:172.21.15.76:3300/0 conn(0x7fce30143ae0 0x7fce300b51b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:00:30.811 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:30.805+0000 7fce2e7fc700 1 -- 172.21.15.59:0/1521510863 >> v1:172.21.15.76:6789/0 conn(0x7fce300722c0 legacy=0x7fce3006e510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:00:30.811 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:30.805+0000 7fce2e7fc700 1 -- 172.21.15.59:0/1521510863 --> v1:172.21.15.59:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fce300b3dc0 con 0x7fce301440f0 2023-11-24T08:00:30.812 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:30.805+0000 7fce2e7fc700 1 -- 172.21.15.59:0/1521510863 <== mon.0 v1:172.21.15.59:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1312333566 0 0) 0x7fce18001670 con 0x7fce301440f0 2023-11-24T08:00:30.812 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:30.805+0000 7fce2e7fc700 1 -- 172.21.15.59:0/1521510863 >> v1:172.21.15.59:6789/0 conn(0x7fce301440f0 legacy=0x7fce30071bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:00:30.812 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:30.805+0000 7fce2e7fc700 1 --2- 172.21.15.59:0/1521510863 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fce1c003a00 0x7fce1c003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:00:30.812 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:30.805+0000 7fce2e7fc700 1 --2- 172.21.15.59:0/1521510863 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fce1c004850 0x7fce1c004cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:00:30.812 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:30.805+0000 7fce2e7fc700 1 --2- 172.21.15.59:0/1521510863 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fce1c005210 0x7fce1c00ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:00:30.813 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:30.805+0000 7fce2e7fc700 1 -- 172.21.15.59:0/1521510863 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce300b40a0 con 0x7fce1c004850 2023-11-24T08:00:30.813 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:30.805+0000 7fce2e7fc700 1 -- 172.21.15.59:0/1521510863 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce1c00c0b0 con 0x7fce1c005210 2023-11-24T08:00:30.813 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:30.805+0000 7fce2e7fc700 1 -- 172.21.15.59:0/1521510863 --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fce1c00c270 con 0x7fce1c003a00 2023-11-24T08:00:30.813 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:30.805+0000 7fce2effd700 1 --2- 172.21.15.59:0/1521510863 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fce1c004850 0x7fce1c004cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:00:30.813 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:30.805+0000 7fce2f7fe700 1 --2- 172.21.15.59:0/1521510863 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fce1c003a00 0x7fce1c003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:00:30.814 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:30.805+0000 7fce2ffff700 1 --2- 172.21.15.59:0/1521510863 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fce1c005210 0x7fce1c00ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:00:30.814 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:30.805+0000 7fce2effd700 1 -- 172.21.15.59:0/1521510863 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fce1c003a00 msgr2=0x7fce1c003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:00:30.814 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:30.805+0000 7fce2effd700 1 --2- 172.21.15.59:0/1521510863 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fce1c003a00 0x7fce1c003e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:00:30.814 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:30.805+0000 7fce2effd700 1 -- 172.21.15.59:0/1521510863 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fce1c005210 msgr2=0x7fce1c00ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:00:30.814 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:30.805+0000 7fce2effd700 1 --2- 172.21.15.59:0/1521510863 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fce1c005210 0x7fce1c00ba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:00:30.815 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:30.805+0000 7fce2effd700 1 -- 172.21.15.59:0/1521510863 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fce1c0035b0 con 0x7fce1c004850 2023-11-24T08:00:30.815 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:30.805+0000 7fce2ffff700 1 -- 172.21.15.59:0/1521510863 reap_dead start 2023-11-24T08:00:30.815 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:30.806+0000 7fce2effd700 1 --2- 172.21.15.59:0/1521510863 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fce1c004850 0x7fce1c004cd0 secure :-1 s=READY pgs=1156 cs=0 l=1 rev1=1 crypto rx=0x7fce1800c240 tx=0x7fce1c005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=442c64e6ec786aa7 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:00:30.815 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:30.806+0000 7fce2e7fc700 1 -- 172.21.15.59:0/1521510863 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce18007c10 con 0x7fce1c004850 2023-11-24T08:00:30.815 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:30.806+0000 7fce2e7fc700 1 -- 172.21.15.59:0/1521510863 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fce1800cde0 con 0x7fce1c004850 2023-11-24T08:00:30.815 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:30.806+0000 7fce2e7fc700 1 -- 172.21.15.59:0/1521510863 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce18005ba0 con 0x7fce1c004850 2023-11-24T08:00:30.816 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:30.806+0000 7fce35b15700 1 -- 172.21.15.59:0/1521510863 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fce1c004850 msgr2=0x7fce1c004cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:00:30.816 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:30.806+0000 7fce35b15700 1 --2- 172.21.15.59:0/1521510863 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fce1c004850 0x7fce1c004cd0 secure :-1 s=READY pgs=1156 cs=0 l=1 rev1=1 crypto rx=0x7fce1800c240 tx=0x7fce1c005880 comp rx=0 tx=0).stop 2023-11-24T08:00:30.816 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:30.806+0000 7fce35b15700 1 -- 172.21.15.59:0/1521510863 shutdown_connections 2023-11-24T08:00:30.816 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:30.806+0000 7fce35b15700 1 --2- 172.21.15.59:0/1521510863 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fce1c004850 0x7fce1c004cd0 unknown :-1 s=CLOSED pgs=1156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:00:30.816 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:30.806+0000 7fce35b15700 1 -- 172.21.15.59:0/1521510863 >> 172.21.15.59:0/1521510863 conn(0x7fce300b2740 msgr2=0x7fce300b2b20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:00:30.816 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:30.806+0000 7fce35b15700 1 -- 172.21.15.59:0/1521510863 shutdown_connections 2023-11-24T08:00:30.816 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:30.806+0000 7fce35b15700 1 -- 172.21.15.59:0/1521510863 wait complete. 2023-11-24T08:00:30.819 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:30.807+0000 7fce35b15700 1 Processor -- start 2023-11-24T08:00:30.819 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:30.808+0000 7fce35b15700 1 -- start start 2023-11-24T08:00:30.819 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:30.808+0000 7fce35b15700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fce1c004850 0x7fce300cdfd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:00:30.820 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:30.808+0000 7fce35b15700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fce300cb180 0x7fce3006e680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:00:30.820 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:30.808+0000 7fce35b15700 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fce300cf8c0 0x7fce301411e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:00:30.820 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:30.808+0000 7fce35b15700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce300cf430 con 0x7fce1c004850 2023-11-24T08:00:30.820 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:30.808+0000 7fce35b15700 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce300ce820 con 0x7fce300cb180 2023-11-24T08:00:30.820 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:30.808+0000 7fce35b15700 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fce300ceb00 con 0x7fce300cf8c0 2023-11-24T08:00:30.820 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:30.808+0000 7fce2f7fe700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fce1c004850 0x7fce300cdfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:00:30.820 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:30.809+0000 7fce2f7fe700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fce1c004850 0x7fce300cdfd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:56698/0 (socket says 172.21.15.59:56698) 2023-11-24T08:00:30.820 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:30.809+0000 7fce2f7fe700 1 -- 172.21.15.59:0/701639027 learned_addr learned my addr 172.21.15.59:0/701639027 (peer_addr_for_me v2:172.21.15.59:0/0) 2023-11-24T08:00:30.821 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:30.809+0000 7fce2effd700 1 --2- 172.21.15.59:0/701639027 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fce300cb180 0x7fce3006e680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:00:30.821 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:30.809+0000 7fce2ffff700 1 --2- 172.21.15.59:0/701639027 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fce300cf8c0 0x7fce301411e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:00:30.821 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:30.809+0000 7fce2f7fe700 1 -- 172.21.15.59:0/701639027 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fce300cf8c0 msgr2=0x7fce301411e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:00:30.821 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:30.809+0000 7fce2f7fe700 1 --2- 172.21.15.59:0/701639027 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fce300cf8c0 0x7fce301411e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:00:30.822 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:30.809+0000 7fce2f7fe700 1 -- 172.21.15.59:0/701639027 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fce300cb180 msgr2=0x7fce3006e680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:00:30.822 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:30.809+0000 7fce2f7fe700 1 --2- 172.21.15.59:0/701639027 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fce300cb180 0x7fce3006e680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:00:30.822 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:30.809+0000 7fce2f7fe700 1 -- 172.21.15.59:0/701639027 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fce300cede0 con 0x7fce1c004850 2023-11-24T08:00:30.822 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:30.809+0000 7fce2effd700 1 --2- 172.21.15.59:0/701639027 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fce300cb180 0x7fce3006e680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-24T08:00:30.822 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:30.809+0000 7fce2f7fe700 1 --2- 172.21.15.59:0/701639027 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fce1c004850 0x7fce300cdfd0 secure :-1 s=READY pgs=1157 cs=0 l=1 rev1=1 crypto rx=0x7fce20011aa0 tx=0x7fce20011e60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:00:30.822 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:30.809+0000 7fce27fff700 1 -- 172.21.15.59:0/701639027 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce2000f9e0 con 0x7fce1c004850 2023-11-24T08:00:30.822 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:30.809+0000 7fce27fff700 1 -- 172.21.15.59:0/701639027 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fce2000fb40 con 0x7fce1c004850 2023-11-24T08:00:30.822 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:30.809+0000 7fce35b15700 1 -- 172.21.15.59:0/701639027 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fce300cf0c0 con 0x7fce1c004850 2023-11-24T08:00:30.823 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:30.809+0000 7fce27fff700 1 -- 172.21.15.59:0/701639027 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce20014440 con 0x7fce1c004850 2023-11-24T08:00:30.823 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:30.809+0000 7fce35b15700 1 -- 172.21.15.59:0/701639027 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fce3006ebc0 con 0x7fce1c004850 2023-11-24T08:00:30.823 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:30.810+0000 7fce35b15700 1 -- 172.21.15.59:0/701639027 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fce30068670 con 0x7fce1c004850 2023-11-24T08:00:30.823 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:30.811+0000 7fce27fff700 1 -- 172.21.15.59:0/701639027 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 12) v1 ==== 89792+0+0 (secure 0 0 0) 0x7fce2001d400 con 0x7fce1c004850 2023-11-24T08:00:30.823 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:30.812+0000 7fce27fff700 1 --2- 172.21.15.59:0/701639027 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7fce1006b500 0x7fce1006d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:00:30.823 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:30.812+0000 7fce27fff700 1 -- 172.21.15.59:0/701639027 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 4943+0+0 (secure 0 0 0) 0x7fce2008a9c0 con 0x7fce1c004850 2023-11-24T08:00:30.823 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:30.812+0000 7fce2effd700 1 --2- 172.21.15.59:0/701639027 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7fce1006b500 0x7fce1006d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:00:30.824 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:30.812+0000 7fce2effd700 1 --2- 172.21.15.59:0/701639027 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7fce1006b500 0x7fce1006d9c0 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7fce18001930 tx=0x7fce18007000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:00:30.824 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:30.815+0000 7fce27fff700 1 -- 172.21.15.59:0/701639027 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fce200555c0 con 0x7fce1c004850 2023-11-24T08:00:31.015 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:31.013+0000 7fce35b15700 1 -- 172.21.15.59:0/701639027 --> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "3", "target": ["mon-mgr", ""]}) v1 -- 0x7fce30144860 con 0x7fce1006b500 2023-11-24T08:00:31.016 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:31.014+0000 7fce27fff700 1 -- 172.21.15.59:0/701639027 <== mgr.4100 v2:172.21.15.76:6832/4264046536 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 3 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7fce18007b30 con 0x7fce1006b500 2023-11-24T08:00:31.018 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:31.016+0000 7fce35b15700 1 -- 172.21.15.59:0/701639027 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7fce1006b500 msgr2=0x7fce1006d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:00:31.019 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:31.016+0000 7fce35b15700 1 --2- 172.21.15.59:0/701639027 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7fce1006b500 0x7fce1006d9c0 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7fce18001930 tx=0x7fce18007000 comp rx=0 tx=0).stop 2023-11-24T08:00:31.019 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:31.016+0000 7fce35b15700 1 -- 172.21.15.59:0/701639027 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fce1c004850 msgr2=0x7fce300cdfd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:00:31.019 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:31.016+0000 7fce35b15700 1 --2- 172.21.15.59:0/701639027 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fce1c004850 0x7fce300cdfd0 secure :-1 s=READY pgs=1157 cs=0 l=1 rev1=1 crypto rx=0x7fce20011aa0 tx=0x7fce20011e60 comp rx=0 tx=0).stop 2023-11-24T08:00:31.019 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:31.016+0000 7fce35b15700 1 -- 172.21.15.59:0/701639027 shutdown_connections 2023-11-24T08:00:31.019 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:31.016+0000 7fce35b15700 1 --2- 172.21.15.59:0/701639027 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7fce1006b500 0x7fce1006d9c0 unknown :-1 s=CLOSED pgs=527 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:00:31.019 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:31.016+0000 7fce35b15700 1 --2- 172.21.15.59:0/701639027 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fce300cf8c0 0x7fce301411e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:00:31.019 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:31.016+0000 7fce35b15700 1 --2- 172.21.15.59:0/701639027 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fce1c004850 0x7fce300cdfd0 unknown :-1 s=CLOSED pgs=1157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:00:31.020 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:31.016+0000 7fce35b15700 1 --2- 172.21.15.59:0/701639027 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fce300cb180 0x7fce3006e680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:00:31.020 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:31.017+0000 7fce35b15700 1 -- 172.21.15.59:0/701639027 >> 172.21.15.59:0/701639027 conn(0x7fce300b2740 msgr2=0x7fce3015ec50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:00:31.020 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:31.017+0000 7fce35b15700 1 -- 172.21.15.59:0/701639027 shutdown_connections 2023-11-24T08:00:31.020 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:31.017+0000 7fce35b15700 1 -- 172.21.15.59:0/701639027 wait complete. 2023-11-24T08:00:31.020 INFO:teuthology.orchestra.run.smithi059.stderr:instructed osd(s) 3 to deep-scrub 2023-11-24T08:00:31.030 INFO:tasks.ceph:Scrubbing osd.4 2023-11-24T08:00:31.031 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 config set osd_debug_deep_scrub_sleep 0 2023-11-24T08:00:31.160 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:31.158+0000 7f1706254700 1 Processor -- start 2023-11-24T08:00:31.160 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:31.158+0000 7f1706254700 1 -- start start 2023-11-24T08:00:31.168 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:31.159+0000 7f1706254700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f17000cd150 0x7f17000ceb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:00:31.169 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:31.159+0000 7f1706254700 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f170006f940 0x7f170006fd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:00:31.169 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:31.159+0000 7f1706254700 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f170006f4b0 con 0x7f170015bfe0 2023-11-24T08:00:31.170 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:31.159+0000 7f1706254700 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f170015c3c0 con 0x7f17000cd150 2023-11-24T08:00:31.170 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:31.159+0000 7f1706254700 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f17000cd760 con 0x7f170006f940 2023-11-24T08:00:31.170 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:31.159+0000 7f16feffd700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f17000cd150 0x7f17000ceb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:00:31.170 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:31.159+0000 7f16ff7fe700 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7f170015bfe0 0x7f17000ce430 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:60180/0 (socket says 172.21.15.59:60180) 2023-11-24T08:00:31.171 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:31.159+0000 7f16ff7fe700 1 -- 172.21.15.59:0/2129087022 learned_addr learned my addr 172.21.15.59:0/2129087022 (peer_addr_for_me v1:172.21.15.59:0/0) 2023-11-24T08:00:31.171 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:31.159+0000 7f16fffff700 1 --2- 172.21.15.59:0/2129087022 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f170006f940 0x7f170006fd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:00:31.171 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:31.159+0000 7f16feffd700 1 -- 172.21.15.59:0/2129087022 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f170006f940 msgr2=0x7f170006fd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:00:31.171 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:31.159+0000 7f16feffd700 1 --2- 172.21.15.59:0/2129087022 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f170006f940 0x7f170006fd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:00:31.171 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:31.159+0000 7f16feffd700 1 -- 172.21.15.59:0/2129087022 >> v1:172.21.15.59:6789/0 conn(0x7f170015bfe0 legacy=0x7f17000ce430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:00:31.171 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:31.159+0000 7f16fe7fc700 1 -- 172.21.15.59:0/2129087022 <== mon.0 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2684779106 0 0) 0x7f170006f4b0 con 0x7f170015bfe0 2023-11-24T08:00:31.171 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:31.159+0000 7f16feffd700 1 -- 172.21.15.59:0/2129087022 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f170006ee60 con 0x7f17000cd150 2023-11-24T08:00:31.172 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:31.159+0000 7f16feffd700 1 --2- 172.21.15.59:0/2129087022 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f17000cd150 0x7f17000ceb40 secure :-1 s=READY pgs=1159 cs=0 l=1 rev1=1 crypto rx=0x7f16f4009840 tx=0x7f16f4009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=500c014e4824e4e6 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:00:31.172 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:31.159+0000 7f16fe7fc700 1 -- 172.21.15.59:0/2129087022 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16f4011830 con 0x7f17000cd150 2023-11-24T08:00:31.172 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:31.160+0000 7f16fe7fc700 1 -- 172.21.15.59:0/2129087022 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f16f4011990 con 0x7f17000cd150 2023-11-24T08:00:31.172 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:31.160+0000 7f16fe7fc700 1 -- 172.21.15.59:0/2129087022 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16f4011d20 con 0x7f17000cd150 2023-11-24T08:00:31.172 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:31.160+0000 7f1706254700 1 -- 172.21.15.59:0/2129087022 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f17000cd150 msgr2=0x7f17000ceb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:00:31.173 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:31.160+0000 7f1706254700 1 --2- 172.21.15.59:0/2129087022 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f17000cd150 0x7f17000ceb40 secure :-1 s=READY pgs=1159 cs=0 l=1 rev1=1 crypto rx=0x7f16f4009840 tx=0x7f16f4009b50 comp rx=0 tx=0).stop 2023-11-24T08:00:31.173 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:31.160+0000 7f1706254700 1 -- 172.21.15.59:0/2129087022 shutdown_connections 2023-11-24T08:00:31.173 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:31.160+0000 7f1706254700 1 --2- 172.21.15.59:0/2129087022 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f170006f940 0x7f170006fd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:00:31.173 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:31.160+0000 7f1706254700 1 --2- 172.21.15.59:0/2129087022 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f17000cd150 0x7f17000ceb40 unknown :-1 s=CLOSED pgs=1159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:00:31.173 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:31.160+0000 7f1706254700 1 -- 172.21.15.59:0/2129087022 >> 172.21.15.59:0/2129087022 conn(0x7f17000b3700 msgr2=0x7f17000b3b10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:00:31.173 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:31.160+0000 7f1706254700 1 -- 172.21.15.59:0/2129087022 shutdown_connections 2023-11-24T08:00:31.174 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:31.160+0000 7f1706254700 1 -- 172.21.15.59:0/2129087022 wait complete. 2023-11-24T08:00:31.174 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:31.161+0000 7f1706254700 1 Processor -- start 2023-11-24T08:00:31.174 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:31.162+0000 7f1706254700 1 -- start start 2023-11-24T08:00:31.174 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:31.162+0000 7f1706254700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f170006f940 0x7f17000b0740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:00:31.174 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:31.162+0000 7f1706254700 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f17000cd150 0x7f17000b0c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:00:31.175 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:31.162+0000 7f1706254700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f170015bfe0 0x7f170015e010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:00:31.177 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:31.162+0000 7f1706254700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1700153bf0 con 0x7f170015bfe0 2023-11-24T08:00:31.177 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:31.162+0000 7f1706254700 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17000c62b0 con 0x7f170006f940 2023-11-24T08:00:31.178 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:31.162+0000 7f1706254700 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f17000c6590 con 0x7f17000cd150 2023-11-24T08:00:31.178 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:31.162+0000 7f16fffff700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f170015bfe0 0x7f170015e010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:00:31.178 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:31.163+0000 7f16fffff700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f170015bfe0 0x7f170015e010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:56716/0 (socket says 172.21.15.59:56716) 2023-11-24T08:00:31.178 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:31.163+0000 7f16fffff700 1 -- 172.21.15.59:0/3663482262 learned_addr learned my addr 172.21.15.59:0/3663482262 (peer_addr_for_me v2:172.21.15.59:0/0) 2023-11-24T08:00:31.178 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:31.163+0000 7f16ff7fe700 1 --2- 172.21.15.59:0/3663482262 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f170006f940 0x7f17000b0740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:00:31.179 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:31.163+0000 7f16feffd700 1 --2- 172.21.15.59:0/3663482262 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f17000cd150 0x7f17000b0c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:00:31.179 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:31.163+0000 7f16fffff700 1 -- 172.21.15.59:0/3663482262 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f17000cd150 msgr2=0x7f17000b0c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:00:31.179 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:31.163+0000 7f16fffff700 1 --2- 172.21.15.59:0/3663482262 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f17000cd150 0x7f17000b0c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:00:31.179 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:31.163+0000 7f16fffff700 1 -- 172.21.15.59:0/3663482262 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f170006f940 msgr2=0x7f17000b0740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:00:31.180 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:31.163+0000 7f16fffff700 1 --2- 172.21.15.59:0/3663482262 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f170006f940 0x7f17000b0740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:00:31.180 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:31.163+0000 7f16fffff700 1 -- 172.21.15.59:0/3663482262 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f16f40094a0 con 0x7f170015bfe0 2023-11-24T08:00:31.180 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:31.163+0000 7f16fffff700 1 --2- 172.21.15.59:0/3663482262 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f170015bfe0 0x7f170015e010 secure :-1 s=READY pgs=1160 cs=0 l=1 rev1=1 crypto rx=0x7f16f800efc0 tx=0x7f16f800c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:00:31.180 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:31.163+0000 7f16e7fff700 1 -- 172.21.15.59:0/3663482262 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16f800cd40 con 0x7f170015bfe0 2023-11-24T08:00:31.180 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:31.163+0000 7f16e7fff700 1 -- 172.21.15.59:0/3663482262 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f16f800f040 con 0x7f170015bfe0 2023-11-24T08:00:31.180 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:31.163+0000 7f1706254700 1 -- 172.21.15.59:0/3663482262 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f17000c6870 con 0x7f170015bfe0 2023-11-24T08:00:31.180 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:31.163+0000 7f16e7fff700 1 -- 172.21.15.59:0/3663482262 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16f80146c0 con 0x7f170015bfe0 2023-11-24T08:00:31.181 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:31.163+0000 7f1706254700 1 -- 172.21.15.59:0/3663482262 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f17000c6e30 con 0x7f170015bfe0 2023-11-24T08:00:31.181 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:31.164+0000 7f1706254700 1 -- 172.21.15.59:0/3663482262 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f17000cda40 con 0x7f170015bfe0 2023-11-24T08:00:31.181 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:31.165+0000 7f16e7fff700 1 -- 172.21.15.59:0/3663482262 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 12) v1 ==== 89792+0+0 (secure 0 0 0) 0x7f16f8014820 con 0x7f170015bfe0 2023-11-24T08:00:31.181 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:31.165+0000 7f16e7fff700 1 --2- 172.21.15.59:0/3663482262 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f16ec06b500 0x7f16ec06d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:00:31.181 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:31.165+0000 7f16e7fff700 1 -- 172.21.15.59:0/3663482262 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f16f808a7c0 con 0x7f170015bfe0 2023-11-24T08:00:31.181 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:31.165+0000 7f16ff7fe700 1 --2- 172.21.15.59:0/3663482262 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f16ec06b500 0x7f16ec06d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:00:31.182 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:31.165+0000 7f16e7fff700 1 --2- 172.21.15.59:0/3663482262 >> [v2:172.21.15.76:6816/2382463431,v1:172.21.15.76:6818/2382463431] conn(0x7f16ec071360 0x7f16ec073780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:00:31.182 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:31.165+0000 7f16e7fff700 1 -- 172.21.15.59:0/3663482262 --> [v2:172.21.15.76:6816/2382463431,v1:172.21.15.76:6818/2382463431] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f16ec073e50 con 0x7f16ec071360 2023-11-24T08:00:31.182 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:31.165+0000 7f16e7fff700 1 -- 172.21.15.59:0/3663482262 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_get_version_reply(handle=1 version=90) v2 ==== 24+0+0 (secure 0 0 0) 0x7f16f808ccf0 con 0x7f170015bfe0 2023-11-24T08:00:31.182 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:31.165+0000 7f16feffd700 1 --2- 172.21.15.59:0/3663482262 >> [v2:172.21.15.76:6816/2382463431,v1:172.21.15.76:6818/2382463431] conn(0x7f16ec071360 0x7f16ec073780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:00:31.182 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:31.166+0000 7f16ff7fe700 1 --2- 172.21.15.59:0/3663482262 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f16ec06b500 0x7f16ec06d9c0 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7f16f0000f60 tx=0x7f16f0007c90 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:00:31.183 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:31.166+0000 7f16feffd700 1 --2- 172.21.15.59:0/3663482262 >> [v2:172.21.15.76:6816/2382463431,v1:172.21.15.76:6818/2382463431] conn(0x7f16ec071360 0x7f16ec073780 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:00:31.183 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:31.166+0000 7f16e7fff700 1 -- 172.21.15.59:0/3663482262 <== osd.4 v2:172.21.15.76:6816/2382463431 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+26965 (crc 0 0 0) 0x7f16ec073e50 con 0x7f16ec071360 2023-11-24T08:00:31.183 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:31.182+0000 7f1706254700 1 -- 172.21.15.59:0/3663482262 --> [v2:172.21.15.76:6816/2382463431,v1:172.21.15.76:6818/2382463431] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f17000c6700 con 0x7f16ec071360 2023-11-24T08:00:31.184 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:31.182+0000 7f16e7fff700 1 -- 172.21.15.59:0/3663482262 <== osd.4 v2:172.21.15.76:6816/2382463431 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+97 (crc 0 0 0) 0x7f17000c6700 con 0x7f16ec071360 2023-11-24T08:00:31.184 INFO:teuthology.orchestra.run.smithi059.stdout:{ 2023-11-24T08:00:31.184 INFO:teuthology.orchestra.run.smithi059.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2023-11-24T08:00:31.184 INFO:teuthology.orchestra.run.smithi059.stdout:} 2023-11-24T08:00:31.185 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:31.182+0000 7f1706254700 1 -- 172.21.15.59:0/3663482262 >> [v2:172.21.15.76:6816/2382463431,v1:172.21.15.76:6818/2382463431] conn(0x7f16ec071360 msgr2=0x7f16ec073780 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:00:31.186 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:31.182+0000 7f1706254700 1 --2- 172.21.15.59:0/3663482262 >> [v2:172.21.15.76:6816/2382463431,v1:172.21.15.76:6818/2382463431] conn(0x7f16ec071360 0x7f16ec073780 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:00:31.186 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:31.182+0000 7f1706254700 1 -- 172.21.15.59:0/3663482262 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f16ec06b500 msgr2=0x7f16ec06d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:00:31.186 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:31.182+0000 7f1706254700 1 --2- 172.21.15.59:0/3663482262 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f16ec06b500 0x7f16ec06d9c0 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7f16f0000f60 tx=0x7f16f0007c90 comp rx=0 tx=0).stop 2023-11-24T08:00:31.186 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:31.182+0000 7f1706254700 1 -- 172.21.15.59:0/3663482262 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f170015bfe0 msgr2=0x7f170015e010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:00:31.186 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:31.182+0000 7f1706254700 1 --2- 172.21.15.59:0/3663482262 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f170015bfe0 0x7f170015e010 secure :-1 s=READY pgs=1160 cs=0 l=1 rev1=1 crypto rx=0x7f16f800efc0 tx=0x7f16f800c3a0 comp rx=0 tx=0).stop 2023-11-24T08:00:31.186 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:31.182+0000 7f16fffff700 1 -- 172.21.15.59:0/3663482262 reap_dead start 2023-11-24T08:00:31.187 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:31.182+0000 7f1706254700 1 -- 172.21.15.59:0/3663482262 shutdown_connections 2023-11-24T08:00:31.187 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:31.182+0000 7f1706254700 1 -- 172.21.15.59:0/3663482262 >> 172.21.15.59:0/3663482262 conn(0x7f17000b3700 msgr2=0x7f17000b1ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:00:31.187 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:31.183+0000 7f1706254700 1 -- 172.21.15.59:0/3663482262 shutdown_connections 2023-11-24T08:00:31.187 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:31.183+0000 7f1706254700 1 -- 172.21.15.59:0/3663482262 wait complete. 2023-11-24T08:00:31.196 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 4 2023-11-24T08:00:31.322 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:31.320+0000 7f7f23378700 1 Processor -- start 2023-11-24T08:00:31.322 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:31.321+0000 7f7f23378700 1 -- start start 2023-11-24T08:00:31.326 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:31.321+0000 7f7f23378700 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f7f1c0b4150 0x7f7f1c06de50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:00:31.326 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:31.321+0000 7f7f23378700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f7f1c0b5ab0 0x7f7f1c06e390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:00:31.326 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:31.321+0000 7f7f23378700 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7f1c071f50 con 0x7f7f1c06b620 2023-11-24T08:00:31.326 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:31.321+0000 7f7f23378700 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7f1c072230 con 0x7f7f1c0b5ab0 2023-11-24T08:00:31.327 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:31.321+0000 7f7f23378700 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7f1c072510 con 0x7f7f1c0b4150 2023-11-24T08:00:31.327 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:31.321+0000 7f7f21915700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f7f1c0b5ab0 0x7f7f1c06e390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:00:31.327 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:31.322+0000 7f7f21915700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f7f1c0b5ab0 0x7f7f1c06e390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:56726/0 (socket says 172.21.15.59:56726) 2023-11-24T08:00:31.327 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:31.322+0000 7f7f21114700 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7f7f1c06b620 0x7f7f1c0b3a40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:60196/0 (socket says 172.21.15.59:60196) 2023-11-24T08:00:31.327 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:31.322+0000 7f7f21915700 1 -- 172.21.15.59:0/799999995 learned_addr learned my addr 172.21.15.59:0/799999995 (peer_addr_for_me v2:172.21.15.59:0/0) 2023-11-24T08:00:31.328 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:31.322+0000 7f7f20913700 1 --2- 172.21.15.59:0/799999995 >> v2:172.21.15.76:3300/0 conn(0x7f7f1c0b4150 0x7f7f1c06de50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:00:31.328 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:31.322+0000 7f7f21915700 1 -- 172.21.15.59:0/799999995 >> v2:172.21.15.76:3300/0 conn(0x7f7f1c0b4150 msgr2=0x7f7f1c06de50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:00:31.328 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:31.322+0000 7f7f21915700 1 --2- 172.21.15.59:0/799999995 >> v2:172.21.15.76:3300/0 conn(0x7f7f1c0b4150 0x7f7f1c06de50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:00:31.328 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:31.322+0000 7f7f21915700 1 -- 172.21.15.59:0/799999995 >> v1:172.21.15.59:6789/0 conn(0x7f7f1c06b620 legacy=0x7f7f1c0b3a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:00:31.328 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:31.322+0000 7f7f21915700 1 -- 172.21.15.59:0/799999995 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7f1c0727f0 con 0x7f7f1c0b5ab0 2023-11-24T08:00:31.329 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:31.322+0000 7f7f0ffff700 1 -- 172.21.15.59:0/799999995 <== mon.0 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2749088996 0 0) 0x7f7f1c071f50 con 0x7f7f1c06b620 2023-11-24T08:00:31.329 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:31.322+0000 7f7f21915700 1 --2- 172.21.15.59:0/799999995 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7f1c0b5ab0 0x7f7f1c06e390 secure :-1 s=READY pgs=1162 cs=0 l=1 rev1=1 crypto rx=0x7f7f1c06d6f0 tx=0x7f7f1400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=607cc8d3bed7f2da server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:00:31.329 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:31.322+0000 7f7f0ffff700 1 -- 172.21.15.59:0/799999995 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f1400e070 con 0x7f7f1c0b5ab0 2023-11-24T08:00:31.329 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:31.322+0000 7f7f0ffff700 1 -- 172.21.15.59:0/799999995 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7f1400b9c0 con 0x7f7f1c0b5ab0 2023-11-24T08:00:31.329 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:31.322+0000 7f7f0ffff700 1 -- 172.21.15.59:0/799999995 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f1400bd70 con 0x7f7f1c0b5ab0 2023-11-24T08:00:31.330 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:31.323+0000 7f7f23378700 1 -- 172.21.15.59:0/799999995 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7f1c0b5ab0 msgr2=0x7f7f1c06e390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:00:31.330 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:31.323+0000 7f7f23378700 1 --2- 172.21.15.59:0/799999995 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7f1c0b5ab0 0x7f7f1c06e390 secure :-1 s=READY pgs=1162 cs=0 l=1 rev1=1 crypto rx=0x7f7f1c06d6f0 tx=0x7f7f1400b330 comp rx=0 tx=0).stop 2023-11-24T08:00:31.330 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:31.323+0000 7f7f23378700 1 -- 172.21.15.59:0/799999995 shutdown_connections 2023-11-24T08:00:31.330 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:31.323+0000 7f7f23378700 1 --2- 172.21.15.59:0/799999995 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7f1c0b5ab0 0x7f7f1c06e390 unknown :-1 s=CLOSED pgs=1162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:00:31.330 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:31.323+0000 7f7f23378700 1 --2- 172.21.15.59:0/799999995 >> v2:172.21.15.76:3300/0 conn(0x7f7f1c0b4150 0x7f7f1c06de50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:00:31.331 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:31.323+0000 7f7f23378700 1 -- 172.21.15.59:0/799999995 >> 172.21.15.59:0/799999995 conn(0x7f7f1c0b2740 msgr2=0x7f7f1c0b2b20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:00:31.331 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:31.323+0000 7f7f23378700 1 -- 172.21.15.59:0/799999995 shutdown_connections 2023-11-24T08:00:31.331 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:31.323+0000 7f7f23378700 1 -- 172.21.15.59:0/799999995 wait complete. 2023-11-24T08:00:31.331 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:31.324+0000 7f7f23378700 1 Processor -- start 2023-11-24T08:00:31.331 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:31.324+0000 7f7f23378700 1 -- start start 2023-11-24T08:00:31.334 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:31.325+0000 7f7f23378700 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f7f1c06b620 0x7f7f1c141830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:00:31.334 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:31.325+0000 7f7f23378700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7f1c0b4150 0x7f7f1c0c78c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:00:31.335 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:31.325+0000 7f7f23378700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7f1c0b5ab0 0x7f7f1c1f2dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:00:31.335 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:31.325+0000 7f7f23378700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f1c0c68a0 con 0x7f7f1c0b5ab0 2023-11-24T08:00:31.335 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:31.325+0000 7f7f23378700 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f1c0c6b80 con 0x7f7f1c0b4150 2023-11-24T08:00:31.335 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:31.325+0000 7f7f23378700 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7f1c0c6e60 con 0x7f7f1c06b620 2023-11-24T08:00:31.335 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:31.325+0000 7f7f21915700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7f1c0b5ab0 0x7f7f1c1f2dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:00:31.335 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:31.325+0000 7f7f21915700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7f1c0b5ab0 0x7f7f1c1f2dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:56732/0 (socket says 172.21.15.59:56732) 2023-11-24T08:00:31.335 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:31.325+0000 7f7f21915700 1 -- 172.21.15.59:0/906123749 learned_addr learned my addr 172.21.15.59:0/906123749 (peer_addr_for_me v2:172.21.15.59:0/0) 2023-11-24T08:00:31.335 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:31.325+0000 7f7f20913700 1 --2- 172.21.15.59:0/906123749 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7f1c0b4150 0x7f7f1c0c78c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:00:31.336 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:31.325+0000 7f7f21114700 1 --2- 172.21.15.59:0/906123749 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f7f1c06b620 0x7f7f1c141830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:00:31.336 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:31.325+0000 7f7f21915700 1 -- 172.21.15.59:0/906123749 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f7f1c06b620 msgr2=0x7f7f1c141830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:00:31.336 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:31.325+0000 7f7f21915700 1 --2- 172.21.15.59:0/906123749 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f7f1c06b620 0x7f7f1c141830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:00:31.336 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:31.325+0000 7f7f21915700 1 -- 172.21.15.59:0/906123749 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7f1c0b4150 msgr2=0x7f7f1c0c78c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:00:31.336 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:31.325+0000 7f7f21915700 1 --2- 172.21.15.59:0/906123749 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7f1c0b4150 0x7f7f1c0c78c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:00:31.336 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:31.325+0000 7f7f21915700 1 -- 172.21.15.59:0/906123749 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7f14009b30 con 0x7f7f1c0b5ab0 2023-11-24T08:00:31.336 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:31.325+0000 7f7f20913700 1 --2- 172.21.15.59:0/906123749 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7f1c0b4150 0x7f7f1c0c78c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T08:00:31.336 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:31.325+0000 7f7f21915700 1 --2- 172.21.15.59:0/906123749 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7f1c0b5ab0 0x7f7f1c1f2dc0 secure :-1 s=READY pgs=1163 cs=0 l=1 rev1=1 crypto rx=0x7f7f1c0d1bd0 tx=0x7f7f14004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:00:31.337 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:31.326+0000 7f7f0dffb700 1 -- 172.21.15.59:0/906123749 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f140099b0 con 0x7f7f1c0b5ab0 2023-11-24T08:00:31.337 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:31.326+0000 7f7f0dffb700 1 -- 172.21.15.59:0/906123749 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7f140092e0 con 0x7f7f1c0b5ab0 2023-11-24T08:00:31.337 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:31.326+0000 7f7f23378700 1 -- 172.21.15.59:0/906123749 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7f1c0c7140 con 0x7f7f1c0b5ab0 2023-11-24T08:00:31.337 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:31.326+0000 7f7f0dffb700 1 -- 172.21.15.59:0/906123749 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f1401b650 con 0x7f7f1c0b5ab0 2023-11-24T08:00:31.337 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:31.326+0000 7f7f23378700 1 -- 172.21.15.59:0/906123749 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7f1c141d70 con 0x7f7f1c0b5ab0 2023-11-24T08:00:31.337 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:31.327+0000 7f7f23378700 1 -- 172.21.15.59:0/906123749 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7f1c068670 con 0x7f7f1c0b5ab0 2023-11-24T08:00:31.337 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:31.327+0000 7f7f0dffb700 1 -- 172.21.15.59:0/906123749 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 12) v1 ==== 89792+0+0 (secure 0 0 0) 0x7f7f1401b7b0 con 0x7f7f1c0b5ab0 2023-11-24T08:00:31.337 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:31.328+0000 7f7f0dffb700 1 --2- 172.21.15.59:0/906123749 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f7f0006b500 0x7f7f0006d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:00:31.338 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:31.328+0000 7f7f0dffb700 1 -- 172.21.15.59:0/906123749 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f7f1408b5a0 con 0x7f7f1c0b5ab0 2023-11-24T08:00:31.338 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:31.328+0000 7f7f21114700 1 --2- 172.21.15.59:0/906123749 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f7f0006b500 0x7f7f0006d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:00:31.338 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:31.329+0000 7f7f21114700 1 --2- 172.21.15.59:0/906123749 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f7f0006b500 0x7f7f0006d9c0 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7f7f18000f60 tx=0x7f7f18009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:00:31.338 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:31.332+0000 7f7f0dffb700 1 -- 172.21.15.59:0/906123749 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f7f14055ed0 con 0x7f7f1c0b5ab0 2023-11-24T08:00:31.536 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:31.534+0000 7f7f23378700 1 -- 172.21.15.59:0/906123749 --> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "4", "target": ["mon-mgr", ""]}) v1 -- 0x7f7f1c06ba00 con 0x7f7f0006b500 2023-11-24T08:00:31.537 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:31.535+0000 7f7f0dffb700 1 -- 172.21.15.59:0/906123749 <== mgr.4100 v2:172.21.15.76:6832/4264046536 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 4 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f7f180013d0 con 0x7f7f0006b500 2023-11-24T08:00:31.539 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:31.537+0000 7f7f23378700 1 -- 172.21.15.59:0/906123749 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f7f0006b500 msgr2=0x7f7f0006d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:00:31.539 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:31.537+0000 7f7f23378700 1 --2- 172.21.15.59:0/906123749 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f7f0006b500 0x7f7f0006d9c0 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7f7f18000f60 tx=0x7f7f18009040 comp rx=0 tx=0).stop 2023-11-24T08:00:31.540 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:31.537+0000 7f7f23378700 1 -- 172.21.15.59:0/906123749 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7f1c0b5ab0 msgr2=0x7f7f1c1f2dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:00:31.540 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:31.537+0000 7f7f23378700 1 --2- 172.21.15.59:0/906123749 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7f1c0b5ab0 0x7f7f1c1f2dc0 secure :-1 s=READY pgs=1163 cs=0 l=1 rev1=1 crypto rx=0x7f7f1c0d1bd0 tx=0x7f7f14004ad0 comp rx=0 tx=0).stop 2023-11-24T08:00:31.540 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:31.537+0000 7f7f23378700 1 -- 172.21.15.59:0/906123749 shutdown_connections 2023-11-24T08:00:31.540 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:31.537+0000 7f7f23378700 1 --2- 172.21.15.59:0/906123749 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f7f0006b500 0x7f7f0006d9c0 unknown :-1 s=CLOSED pgs=529 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:00:31.540 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:31.537+0000 7f7f23378700 1 --2- 172.21.15.59:0/906123749 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7f1c0b5ab0 0x7f7f1c1f2dc0 unknown :-1 s=CLOSED pgs=1163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:00:31.540 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:31.537+0000 7f7f23378700 1 --2- 172.21.15.59:0/906123749 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f7f1c06b620 0x7f7f1c141830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:00:31.541 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:31.537+0000 7f7f23378700 1 --2- 172.21.15.59:0/906123749 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7f1c0b4150 0x7f7f1c0c78c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:00:31.541 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:31.537+0000 7f7f23378700 1 -- 172.21.15.59:0/906123749 >> 172.21.15.59:0/906123749 conn(0x7f7f1c0b2740 msgr2=0x7f7f1c06dd50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:00:31.541 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:31.537+0000 7f7f23378700 1 -- 172.21.15.59:0/906123749 shutdown_connections 2023-11-24T08:00:31.541 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:31.537+0000 7f7f23378700 1 -- 172.21.15.59:0/906123749 wait complete. 2023-11-24T08:00:31.541 INFO:teuthology.orchestra.run.smithi059.stderr:instructed osd(s) 4 to deep-scrub 2023-11-24T08:00:31.551 INFO:tasks.ceph:Scrubbing osd.5 2023-11-24T08:00:31.552 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 config set osd_debug_deep_scrub_sleep 0 2023-11-24T08:00:31.681 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:31.679+0000 7fc69d6db700 1 Processor -- start 2023-11-24T08:00:31.681 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:31.679+0000 7fc69d6db700 1 -- start start 2023-11-24T08:00:31.685 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:31.680+0000 7fc69d6db700 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fc6980cf730 0x7fc69806f580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:00:31.685 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:31.680+0000 7fc69d6db700 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc6980cd490 con 0x7fc6981457b0 2023-11-24T08:00:31.685 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:31.680+0000 7fc69d6db700 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc6980cd6e0 con 0x7fc6980725a0 2023-11-24T08:00:31.685 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:31.680+0000 7fc69d6db700 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc69806fac0 con 0x7fc6980cf730 2023-11-24T08:00:31.685 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:31.680+0000 7fc696ffd700 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7fc6981457b0 0x7fc69806e3d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:60210/0 (socket says 172.21.15.59:60210) 2023-11-24T08:00:31.685 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:31.680+0000 7fc696ffd700 1 -- 172.21.15.59:0/4137469115 learned_addr learned my addr 172.21.15.59:0/4137469115 (peer_addr_for_me v1:172.21.15.59:0/0) 2023-11-24T08:00:31.686 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:31.680+0000 7fc6977fe700 1 --2- 172.21.15.59:0/4137469115 >> v2:172.21.15.76:3300/0 conn(0x7fc6980cf730 0x7fc69806f580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:00:31.686 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:31.681+0000 7fc695ffb700 1 -- 172.21.15.59:0/4137469115 <== mon.0 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1964029471 0 0) 0x7fc6980cd490 con 0x7fc6981457b0 2023-11-24T08:00:31.686 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:31.681+0000 7fc695ffb700 1 -- 172.21.15.59:0/4137469115 --> v1:172.21.15.59:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc6800035b0 con 0x7fc6981457b0 2023-11-24T08:00:31.686 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:31.681+0000 7fc695ffb700 1 -- 172.21.15.59:0/4137469115 <== mon.0 v1:172.21.15.59:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3091757110 0 0) 0x7fc6800035b0 con 0x7fc6981457b0 2023-11-24T08:00:31.686 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:31.681+0000 7fc695ffb700 1 -- 172.21.15.59:0/4137469115 >> v2:172.21.15.76:3300/0 conn(0x7fc6980cf730 msgr2=0x7fc69806f580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:00:31.686 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:31.681+0000 7fc695ffb700 1 --2- 172.21.15.59:0/4137469115 >> v2:172.21.15.76:3300/0 conn(0x7fc6980cf730 0x7fc69806f580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:00:31.686 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:31.681+0000 7fc695ffb700 1 -- 172.21.15.59:0/4137469115 >> v1:172.21.15.76:6789/0 conn(0x7fc6980725a0 legacy=0x7fc6980cf020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:00:31.686 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:31.681+0000 7fc695ffb700 1 -- 172.21.15.59:0/4137469115 --> v1:172.21.15.59:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc6980cd9c0 con 0x7fc6981457b0 2023-11-24T08:00:31.687 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:31.681+0000 7fc695ffb700 1 -- 172.21.15.59:0/4137469115 <== mon.0 v1:172.21.15.59:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1312333566 0 0) 0x7fc68c001670 con 0x7fc6981457b0 2023-11-24T08:00:31.687 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:31.681+0000 7fc695ffb700 1 -- 172.21.15.59:0/4137469115 >> v1:172.21.15.59:6789/0 conn(0x7fc6981457b0 legacy=0x7fc69806e3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:00:31.687 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:31.681+0000 7fc695ffb700 1 --2- 172.21.15.59:0/4137469115 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc680003a00 0x7fc680003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:00:31.687 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:31.681+0000 7fc695ffb700 1 --2- 172.21.15.59:0/4137469115 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fc680004850 0x7fc680004cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:00:31.687 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:31.681+0000 7fc695ffb700 1 --2- 172.21.15.59:0/4137469115 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc680005210 0x7fc68000ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:00:31.687 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:31.681+0000 7fc695ffb700 1 -- 172.21.15.59:0/4137469115 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc69806fac0 con 0x7fc680003a00 2023-11-24T08:00:31.687 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:31.681+0000 7fc695ffb700 1 -- 172.21.15.59:0/4137469115 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc68000c1b0 con 0x7fc680005210 2023-11-24T08:00:31.687 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:31.681+0000 7fc695ffb700 1 -- 172.21.15.59:0/4137469115 --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc68000c370 con 0x7fc680004850 2023-11-24T08:00:31.688 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:31.681+0000 7fc696ffd700 1 --2- 172.21.15.59:0/4137469115 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc680003a00 0x7fc680003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:00:31.688 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:31.681+0000 7fc6977fe700 1 --2- 172.21.15.59:0/4137469115 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc680005210 0x7fc68000ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:00:31.688 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:31.681+0000 7fc6967fc700 1 --2- 172.21.15.59:0/4137469115 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fc680004850 0x7fc680004cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:00:31.688 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:31.681+0000 7fc696ffd700 1 -- 172.21.15.59:0/4137469115 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fc680004850 msgr2=0x7fc680004cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:00:31.688 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:31.681+0000 7fc696ffd700 1 --2- 172.21.15.59:0/4137469115 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fc680004850 0x7fc680004cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:00:31.688 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:31.681+0000 7fc696ffd700 1 -- 172.21.15.59:0/4137469115 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc680005210 msgr2=0x7fc68000ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:00:31.688 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:31.681+0000 7fc696ffd700 1 --2- 172.21.15.59:0/4137469115 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc680005210 0x7fc68000ba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:00:31.689 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:31.681+0000 7fc696ffd700 1 -- 172.21.15.59:0/4137469115 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc6800035b0 con 0x7fc680003a00 2023-11-24T08:00:31.689 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:31.681+0000 7fc6977fe700 1 --2- 172.21.15.59:0/4137469115 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc680005210 0x7fc68000ba10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-24T08:00:31.689 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:31.681+0000 7fc6977fe700 1 -- 172.21.15.59:0/4137469115 reap_dead start 2023-11-24T08:00:31.689 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:31.682+0000 7fc696ffd700 1 --2- 172.21.15.59:0/4137469115 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc680003a00 0x7fc680003e20 secure :-1 s=READY pgs=1165 cs=0 l=1 rev1=1 crypto rx=0x7fc68c00c240 tx=0x7fc680005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f799f9cedd5af2f3 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:00:31.689 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:31.682+0000 7fc695ffb700 1 -- 172.21.15.59:0/4137469115 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc68c007c10 con 0x7fc680003a00 2023-11-24T08:00:31.690 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:31.682+0000 7fc695ffb700 1 -- 172.21.15.59:0/4137469115 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc68c00cda0 con 0x7fc680003a00 2023-11-24T08:00:31.690 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:31.682+0000 7fc695ffb700 1 -- 172.21.15.59:0/4137469115 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc68c005ba0 con 0x7fc680003a00 2023-11-24T08:00:31.690 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:31.682+0000 7fc69d6db700 1 -- 172.21.15.59:0/4137469115 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc680003a00 msgr2=0x7fc680003e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:00:31.690 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:31.682+0000 7fc69d6db700 1 --2- 172.21.15.59:0/4137469115 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc680003a00 0x7fc680003e20 secure :-1 s=READY pgs=1165 cs=0 l=1 rev1=1 crypto rx=0x7fc68c00c240 tx=0x7fc680005880 comp rx=0 tx=0).stop 2023-11-24T08:00:31.690 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:31.682+0000 7fc69d6db700 1 -- 172.21.15.59:0/4137469115 shutdown_connections 2023-11-24T08:00:31.690 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:31.682+0000 7fc69d6db700 1 --2- 172.21.15.59:0/4137469115 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc680003a00 0x7fc680003e20 unknown :-1 s=CLOSED pgs=1165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:00:31.690 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:31.682+0000 7fc69d6db700 1 -- 172.21.15.59:0/4137469115 >> 172.21.15.59:0/4137469115 conn(0x7fc6980b2760 msgr2=0x7fc6980b2b70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:00:31.691 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:31.682+0000 7fc69d6db700 1 -- 172.21.15.59:0/4137469115 shutdown_connections 2023-11-24T08:00:31.691 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:31.682+0000 7fc69d6db700 1 -- 172.21.15.59:0/4137469115 wait complete. 2023-11-24T08:00:31.693 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:31.684+0000 7fc69d6db700 1 Processor -- start 2023-11-24T08:00:31.694 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:31.684+0000 7fc69d6db700 1 -- start start 2023-11-24T08:00:31.694 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:31.684+0000 7fc69d6db700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc680003a00 0x7fc6980b3b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:00:31.694 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:31.684+0000 7fc69d6db700 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fc6981f77f0 0x7fc69815d1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:00:31.694 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:31.684+0000 7fc69d6db700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc6981425d0 0x7fc6980ca430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:00:31.694 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:31.684+0000 7fc69d6db700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6980b40a0 con 0x7fc680003a00 2023-11-24T08:00:31.694 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:31.684+0000 7fc69d6db700 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6980b4b30 con 0x7fc6981425d0 2023-11-24T08:00:31.695 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:31.684+0000 7fc69d6db700 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc6980c9fa0 con 0x7fc6981f77f0 2023-11-24T08:00:31.695 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:31.685+0000 7fc696ffd700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc680003a00 0x7fc6980b3b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:00:31.695 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:31.685+0000 7fc696ffd700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc680003a00 0x7fc6980b3b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:56748/0 (socket says 172.21.15.59:56748) 2023-11-24T08:00:31.695 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:31.685+0000 7fc696ffd700 1 -- 172.21.15.59:0/3531129091 learned_addr learned my addr 172.21.15.59:0/3531129091 (peer_addr_for_me v2:172.21.15.59:0/0) 2023-11-24T08:00:31.695 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:31.685+0000 7fc6977fe700 1 --2- 172.21.15.59:0/3531129091 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc6981425d0 0x7fc6980ca430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:00:31.695 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:31.685+0000 7fc6967fc700 1 --2- 172.21.15.59:0/3531129091 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fc6981f77f0 0x7fc69815d1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:00:31.695 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:31.685+0000 7fc696ffd700 1 -- 172.21.15.59:0/3531129091 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fc6981f77f0 msgr2=0x7fc69815d1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:00:31.695 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:31.685+0000 7fc696ffd700 1 --2- 172.21.15.59:0/3531129091 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fc6981f77f0 0x7fc69815d1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:00:31.696 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:31.685+0000 7fc696ffd700 1 -- 172.21.15.59:0/3531129091 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc6981425d0 msgr2=0x7fc6980ca430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:00:31.696 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:31.685+0000 7fc696ffd700 1 --2- 172.21.15.59:0/3531129091 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc6981425d0 0x7fc6980ca430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:00:31.696 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:31.685+0000 7fc696ffd700 1 -- 172.21.15.59:0/3531129091 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc6981f7c70 con 0x7fc680003a00 2023-11-24T08:00:31.696 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:31.685+0000 7fc6977fe700 1 --2- 172.21.15.59:0/3531129091 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc6981425d0 0x7fc6980ca430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-24T08:00:31.696 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:31.685+0000 7fc696ffd700 1 --2- 172.21.15.59:0/3531129091 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc680003a00 0x7fc6980b3b60 secure :-1 s=READY pgs=1166 cs=0 l=1 rev1=1 crypto rx=0x7fc68c00c830 tx=0x7fc68c001a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:00:31.696 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:31.685+0000 7fc67f7fe700 1 -- 172.21.15.59:0/3531129091 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc68c001640 con 0x7fc680003a00 2023-11-24T08:00:31.696 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:31.685+0000 7fc67f7fe700 1 -- 172.21.15.59:0/3531129091 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc68c0070d0 con 0x7fc680003a00 2023-11-24T08:00:31.697 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:31.685+0000 7fc69d6db700 1 -- 172.21.15.59:0/3531129091 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc6980b4290 con 0x7fc680003a00 2023-11-24T08:00:31.697 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:31.685+0000 7fc67f7fe700 1 -- 172.21.15.59:0/3531129091 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc68c0074a0 con 0x7fc680003a00 2023-11-24T08:00:31.697 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:31.685+0000 7fc69d6db700 1 -- 172.21.15.59:0/3531129091 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc6980b4850 con 0x7fc680003a00 2023-11-24T08:00:31.697 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:31.686+0000 7fc69d6db700 1 -- 172.21.15.59:0/3531129091 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fc69806deb0 con 0x7fc680003a00 2023-11-24T08:00:31.697 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:31.687+0000 7fc67f7fe700 1 -- 172.21.15.59:0/3531129091 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 12) v1 ==== 89792+0+0 (secure 0 0 0) 0x7fc68c007600 con 0x7fc680003a00 2023-11-24T08:00:31.697 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:31.687+0000 7fc67f7fe700 1 --2- 172.21.15.59:0/3531129091 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7fc68406b500 0x7fc68406d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:00:31.697 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:31.687+0000 7fc67f7fe700 1 -- 172.21.15.59:0/3531129091 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 4943+0+0 (secure 0 0 0) 0x7fc68c08b360 con 0x7fc680003a00 2023-11-24T08:00:31.697 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:31.687+0000 7fc67f7fe700 1 --2- 172.21.15.59:0/3531129091 >> [v2:172.21.15.76:6800/329334308,v1:172.21.15.76:6802/329334308] conn(0x7fc684071360 0x7fc684073780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:00:31.698 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:31.687+0000 7fc67f7fe700 1 -- 172.21.15.59:0/3531129091 --> [v2:172.21.15.76:6800/329334308,v1:172.21.15.76:6802/329334308] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fc684073e50 con 0x7fc684071360 2023-11-24T08:00:31.698 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:31.687+0000 7fc67f7fe700 1 -- 172.21.15.59:0/3531129091 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_get_version_reply(handle=1 version=90) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc68c08d890 con 0x7fc680003a00 2023-11-24T08:00:31.698 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:31.687+0000 7fc6967fc700 1 --2- 172.21.15.59:0/3531129091 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7fc68406b500 0x7fc68406d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:00:31.698 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:31.687+0000 7fc6977fe700 1 --2- 172.21.15.59:0/3531129091 >> [v2:172.21.15.76:6800/329334308,v1:172.21.15.76:6802/329334308] conn(0x7fc684071360 0x7fc684073780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:00:31.698 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:31.688+0000 7fc6967fc700 1 --2- 172.21.15.59:0/3531129091 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7fc68406b500 0x7fc68406d9c0 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7fc6980b5d10 tx=0x7fc688009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:00:31.698 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:31.688+0000 7fc6977fe700 1 --2- 172.21.15.59:0/3531129091 >> [v2:172.21.15.76:6800/329334308,v1:172.21.15.76:6802/329334308] conn(0x7fc684071360 0x7fc684073780 crc :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:00:31.698 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:31.689+0000 7fc67f7fe700 1 -- 172.21.15.59:0/3531129091 <== osd.5 v2:172.21.15.76:6800/329334308 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+26965 (crc 0 0 0) 0x7fc684073e50 con 0x7fc684071360 2023-11-24T08:00:31.705 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:31.704+0000 7fc69d6db700 1 -- 172.21.15.59:0/3531129091 --> [v2:172.21.15.76:6800/329334308,v1:172.21.15.76:6802/329334308] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7fc6980ca0e0 con 0x7fc684071360 2023-11-24T08:00:31.706 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:31.705+0000 7fc67f7fe700 1 -- 172.21.15.59:0/3531129091 <== osd.5 v2:172.21.15.76:6800/329334308 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+97 (crc 0 0 0) 0x7fc690017410 con 0x7fc684071360 2023-11-24T08:00:31.706 INFO:teuthology.orchestra.run.smithi059.stdout:{ 2023-11-24T08:00:31.707 INFO:teuthology.orchestra.run.smithi059.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2023-11-24T08:00:31.707 INFO:teuthology.orchestra.run.smithi059.stdout:} 2023-11-24T08:00:31.708 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:31.705+0000 7fc69d6db700 1 -- 172.21.15.59:0/3531129091 >> [v2:172.21.15.76:6800/329334308,v1:172.21.15.76:6802/329334308] conn(0x7fc684071360 msgr2=0x7fc684073780 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:00:31.708 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:31.705+0000 7fc69d6db700 1 --2- 172.21.15.59:0/3531129091 >> [v2:172.21.15.76:6800/329334308,v1:172.21.15.76:6802/329334308] conn(0x7fc684071360 0x7fc684073780 crc :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:00:31.708 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:31.705+0000 7fc69d6db700 1 -- 172.21.15.59:0/3531129091 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7fc68406b500 msgr2=0x7fc68406d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:00:31.708 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:31.705+0000 7fc69d6db700 1 --2- 172.21.15.59:0/3531129091 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7fc68406b500 0x7fc68406d9c0 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7fc6980b5d10 tx=0x7fc688009040 comp rx=0 tx=0).stop 2023-11-24T08:00:31.708 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:31.705+0000 7fc69d6db700 1 -- 172.21.15.59:0/3531129091 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc680003a00 msgr2=0x7fc6980b3b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:00:31.709 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:31.705+0000 7fc69d6db700 1 --2- 172.21.15.59:0/3531129091 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc680003a00 0x7fc6980b3b60 secure :-1 s=READY pgs=1166 cs=0 l=1 rev1=1 crypto rx=0x7fc68c00c830 tx=0x7fc68c001a00 comp rx=0 tx=0).stop 2023-11-24T08:00:31.709 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:31.705+0000 7fc6977fe700 1 -- 172.21.15.59:0/3531129091 reap_dead start 2023-11-24T08:00:31.709 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:31.705+0000 7fc69d6db700 1 -- 172.21.15.59:0/3531129091 shutdown_connections 2023-11-24T08:00:31.709 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:31.705+0000 7fc69d6db700 1 -- 172.21.15.59:0/3531129091 >> 172.21.15.59:0/3531129091 conn(0x7fc6980b2760 msgr2=0x7fc698071b90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:00:31.709 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:31.705+0000 7fc69d6db700 1 -- 172.21.15.59:0/3531129091 shutdown_connections 2023-11-24T08:00:31.709 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:31.705+0000 7fc69d6db700 1 -- 172.21.15.59:0/3531129091 wait complete. 2023-11-24T08:00:31.718 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 5 2023-11-24T08:00:31.844 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:31.842+0000 7fdc81cd6700 1 Processor -- start 2023-11-24T08:00:31.844 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:31.842+0000 7fdc81cd6700 1 -- start start 2023-11-24T08:00:31.847 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:31.843+0000 7fdc81cd6700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fdc7c15bf10 0x7fdc7c066ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:00:31.848 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:31.843+0000 7fdc81cd6700 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fdc7c06b7c0 0x7fdc7c06bc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:00:31.848 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:31.843+0000 7fdc81cd6700 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdc7c0b10c0 con 0x7fdc7c0af980 2023-11-24T08:00:31.848 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:31.843+0000 7fdc81cd6700 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdc7c06c170 con 0x7fdc7c15bf10 2023-11-24T08:00:31.848 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:31.843+0000 7fdc81cd6700 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdc7c0af730 con 0x7fdc7c06b7c0 2023-11-24T08:00:31.848 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:31.843+0000 7fdc7b7fe700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fdc7c15bf10 0x7fdc7c066ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:00:31.849 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:31.843+0000 7fdc7b7fe700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fdc7c15bf10 0x7fdc7c066ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:56758/0 (socket says 172.21.15.59:56758) 2023-11-24T08:00:31.849 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:31.843+0000 7fdc7b7fe700 1 -- 172.21.15.59:0/67899692 learned_addr learned my addr 172.21.15.59:0/67899692 (peer_addr_for_me v2:172.21.15.59:0/0) 2023-11-24T08:00:31.849 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:31.844+0000 7fdc7bfff700 1 --2- 172.21.15.59:0/67899692 >> v2:172.21.15.76:3300/0 conn(0x7fdc7c06b7c0 0x7fdc7c06bc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:00:31.849 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:31.844+0000 7fdc7b7fe700 1 -- 172.21.15.59:0/67899692 >> v2:172.21.15.76:3300/0 conn(0x7fdc7c06b7c0 msgr2=0x7fdc7c06bc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:00:31.849 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:31.844+0000 7fdc7b7fe700 1 --2- 172.21.15.59:0/67899692 >> v2:172.21.15.76:3300/0 conn(0x7fdc7c06b7c0 0x7fdc7c06bc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:00:31.850 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:31.844+0000 7fdc7b7fe700 1 -- 172.21.15.59:0/67899692 >> v1:172.21.15.76:6789/0 conn(0x7fdc7c0af980 legacy=0x7fdc7c0b1550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:00:31.850 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:31.844+0000 7fdc7b7fe700 1 -- 172.21.15.59:0/67899692 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdc7c0b1c60 con 0x7fdc7c15bf10 2023-11-24T08:00:31.850 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:31.844+0000 7fdc7b7fe700 1 --2- 172.21.15.59:0/67899692 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fdc7c15bf10 0x7fdc7c066ce0 secure :-1 s=READY pgs=1167 cs=0 l=1 rev1=1 crypto rx=0x7fdc7c071720 tx=0x7fdc6c009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=baaa811cb55df60c server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:00:31.850 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:31.844+0000 7fdc7a7fc700 1 -- 172.21.15.59:0/67899692 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc6c00b870 con 0x7fdc7c15bf10 2023-11-24T08:00:31.851 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:31.844+0000 7fdc7a7fc700 1 -- 172.21.15.59:0/67899692 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdc6c00b9d0 con 0x7fdc7c15bf10 2023-11-24T08:00:31.851 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:31.844+0000 7fdc7a7fc700 1 -- 172.21.15.59:0/67899692 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc6c00bd60 con 0x7fdc7c15bf10 2023-11-24T08:00:31.851 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:31.845+0000 7fdc81cd6700 1 -- 172.21.15.59:0/67899692 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fdc7c15bf10 msgr2=0x7fdc7c066ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:00:31.851 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:31.845+0000 7fdc81cd6700 1 --2- 172.21.15.59:0/67899692 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fdc7c15bf10 0x7fdc7c066ce0 secure :-1 s=READY pgs=1167 cs=0 l=1 rev1=1 crypto rx=0x7fdc7c071720 tx=0x7fdc6c009bd0 comp rx=0 tx=0).stop 2023-11-24T08:00:31.852 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:31.845+0000 7fdc81cd6700 1 -- 172.21.15.59:0/67899692 shutdown_connections 2023-11-24T08:00:31.852 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:31.845+0000 7fdc81cd6700 1 --2- 172.21.15.59:0/67899692 >> v2:172.21.15.76:3300/0 conn(0x7fdc7c06b7c0 0x7fdc7c06bc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:00:31.852 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:31.845+0000 7fdc81cd6700 1 --2- 172.21.15.59:0/67899692 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fdc7c15bf10 0x7fdc7c066ce0 unknown :-1 s=CLOSED pgs=1167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:00:31.852 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:31.845+0000 7fdc81cd6700 1 -- 172.21.15.59:0/67899692 >> 172.21.15.59:0/67899692 conn(0x7fdc7c0b5070 msgr2=0x7fdc7c0b5730 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:00:31.853 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:31.845+0000 7fdc81cd6700 1 -- 172.21.15.59:0/67899692 shutdown_connections 2023-11-24T08:00:31.853 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:31.845+0000 7fdc81cd6700 1 -- 172.21.15.59:0/67899692 wait complete. 2023-11-24T08:00:31.853 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:31.846+0000 7fdc81cd6700 1 Processor -- start 2023-11-24T08:00:31.853 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:31.846+0000 7fdc81cd6700 1 -- start start 2023-11-24T08:00:31.857 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:31.847+0000 7fdc81cd6700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fdc7c06b7c0 0x7fdc7c151a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:00:31.857 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:31.847+0000 7fdc81cd6700 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fdc7c0af980 0x7fdc7c14e7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:00:31.858 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:31.847+0000 7fdc81cd6700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fdc7c15bf10 0x7fdc7c09a370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:00:31.858 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:31.847+0000 7fdc81cd6700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc7c153dd0 con 0x7fdc7c15bf10 2023-11-24T08:00:31.858 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:31.847+0000 7fdc81cd6700 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc7c151fb0 con 0x7fdc7c06b7c0 2023-11-24T08:00:31.858 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:31.847+0000 7fdc81cd6700 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdc7c153150 con 0x7fdc7c0af980 2023-11-24T08:00:31.858 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:31.847+0000 7fdc7bfff700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fdc7c15bf10 0x7fdc7c09a370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:00:31.859 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:31.847+0000 7fdc7bfff700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fdc7c15bf10 0x7fdc7c09a370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:56768/0 (socket says 172.21.15.59:56768) 2023-11-24T08:00:31.859 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:31.847+0000 7fdc7bfff700 1 -- 172.21.15.59:0/2302379518 learned_addr learned my addr 172.21.15.59:0/2302379518 (peer_addr_for_me v2:172.21.15.59:0/0) 2023-11-24T08:00:31.859 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:31.847+0000 7fdc7affd700 1 --2- 172.21.15.59:0/2302379518 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fdc7c0af980 0x7fdc7c14e7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:00:31.859 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:31.847+0000 7fdc7b7fe700 1 --2- 172.21.15.59:0/2302379518 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fdc7c06b7c0 0x7fdc7c151a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:00:31.859 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:31.847+0000 7fdc7bfff700 1 -- 172.21.15.59:0/2302379518 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fdc7c0af980 msgr2=0x7fdc7c14e7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:00:31.860 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:31.847+0000 7fdc7bfff700 1 --2- 172.21.15.59:0/2302379518 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fdc7c0af980 0x7fdc7c14e7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:00:31.860 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:31.847+0000 7fdc7bfff700 1 -- 172.21.15.59:0/2302379518 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fdc7c06b7c0 msgr2=0x7fdc7c151a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:00:31.860 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:31.847+0000 7fdc7bfff700 1 --2- 172.21.15.59:0/2302379518 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fdc7c06b7c0 0x7fdc7c151a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:00:31.860 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:31.847+0000 7fdc7bfff700 1 -- 172.21.15.59:0/2302379518 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdc6c009550 con 0x7fdc7c15bf10 2023-11-24T08:00:31.861 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:31.848+0000 7fdc7bfff700 1 --2- 172.21.15.59:0/2302379518 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fdc7c15bf10 0x7fdc7c09a370 secure :-1 s=READY pgs=1168 cs=0 l=1 rev1=1 crypto rx=0x7fdc74007e40 tx=0x7fdc7400e3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:00:31.861 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:31.848+0000 7fdc73fff700 1 -- 172.21.15.59:0/2302379518 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc7400ed30 con 0x7fdc7c15bf10 2023-11-24T08:00:31.861 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:31.848+0000 7fdc73fff700 1 -- 172.21.15.59:0/2302379518 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdc74014420 con 0x7fdc7c15bf10 2023-11-24T08:00:31.861 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:31.848+0000 7fdc81cd6700 1 -- 172.21.15.59:0/2302379518 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdc7c153430 con 0x7fdc7c15bf10 2023-11-24T08:00:31.862 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:31.848+0000 7fdc73fff700 1 -- 172.21.15.59:0/2302379518 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc740147d0 con 0x7fdc7c15bf10 2023-11-24T08:00:31.862 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:31.848+0000 7fdc81cd6700 1 -- 172.21.15.59:0/2302379518 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdc7c1539f0 con 0x7fdc7c15bf10 2023-11-24T08:00:31.862 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:31.849+0000 7fdc81cd6700 1 -- 172.21.15.59:0/2302379518 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdc7c068670 con 0x7fdc7c15bf10 2023-11-24T08:00:31.862 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:31.849+0000 7fdc73fff700 1 -- 172.21.15.59:0/2302379518 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 12) v1 ==== 89792+0+0 (secure 0 0 0) 0x7fdc74014930 con 0x7fdc7c15bf10 2023-11-24T08:00:31.863 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:31.850+0000 7fdc73fff700 1 --2- 172.21.15.59:0/2302379518 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7fdc5c06b500 0x7fdc5c06d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:00:31.863 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:31.850+0000 7fdc73fff700 1 -- 172.21.15.59:0/2302379518 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 4943+0+0 (secure 0 0 0) 0x7fdc7408a390 con 0x7fdc7c15bf10 2023-11-24T08:00:31.863 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:31.850+0000 7fdc7b7fe700 1 --2- 172.21.15.59:0/2302379518 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7fdc5c06b500 0x7fdc5c06d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:00:31.863 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:31.850+0000 7fdc7b7fe700 1 --2- 172.21.15.59:0/2302379518 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7fdc5c06b500 0x7fdc5c06d9c0 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7fdc6c014040 tx=0x7fdc6c008030 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:00:31.864 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:31.854+0000 7fdc73fff700 1 -- 172.21.15.59:0/2302379518 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fdc74054f10 con 0x7fdc7c15bf10 2023-11-24T08:00:32.057 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:32.055+0000 7fdc81cd6700 1 -- 172.21.15.59:0/2302379518 --> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "5", "target": ["mon-mgr", ""]}) v1 -- 0x7fdc7c15ac10 con 0x7fdc5c06b500 2023-11-24T08:00:32.057 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:32.056+0000 7fdc73fff700 1 -- 172.21.15.59:0/2302379518 <== mgr.4100 v2:172.21.15.76:6832/4264046536 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 5 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7fdc7c15ac10 con 0x7fdc5c06b500 2023-11-24T08:00:32.060 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:32.057+0000 7fdc81cd6700 1 -- 172.21.15.59:0/2302379518 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7fdc5c06b500 msgr2=0x7fdc5c06d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:00:32.060 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:32.057+0000 7fdc81cd6700 1 --2- 172.21.15.59:0/2302379518 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7fdc5c06b500 0x7fdc5c06d9c0 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7fdc6c014040 tx=0x7fdc6c008030 comp rx=0 tx=0).stop 2023-11-24T08:00:32.060 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:32.057+0000 7fdc81cd6700 1 -- 172.21.15.59:0/2302379518 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fdc7c15bf10 msgr2=0x7fdc7c09a370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:00:32.060 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:32.057+0000 7fdc81cd6700 1 --2- 172.21.15.59:0/2302379518 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fdc7c15bf10 0x7fdc7c09a370 secure :-1 s=READY pgs=1168 cs=0 l=1 rev1=1 crypto rx=0x7fdc74007e40 tx=0x7fdc7400e3c0 comp rx=0 tx=0).stop 2023-11-24T08:00:32.060 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:32.058+0000 7fdc81cd6700 1 -- 172.21.15.59:0/2302379518 shutdown_connections 2023-11-24T08:00:32.060 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:32.058+0000 7fdc81cd6700 1 --2- 172.21.15.59:0/2302379518 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7fdc5c06b500 0x7fdc5c06d9c0 unknown :-1 s=CLOSED pgs=531 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:00:32.061 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:32.058+0000 7fdc81cd6700 1 --2- 172.21.15.59:0/2302379518 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fdc7c15bf10 0x7fdc7c09a370 unknown :-1 s=CLOSED pgs=1168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:00:32.061 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:32.058+0000 7fdc81cd6700 1 --2- 172.21.15.59:0/2302379518 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fdc7c06b7c0 0x7fdc7c151a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:00:32.061 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:32.058+0000 7fdc81cd6700 1 --2- 172.21.15.59:0/2302379518 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fdc7c0af980 0x7fdc7c14e7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:00:32.061 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:32.058+0000 7fdc81cd6700 1 -- 172.21.15.59:0/2302379518 >> 172.21.15.59:0/2302379518 conn(0x7fdc7c0b5070 msgr2=0x7fdc7c06ebd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:00:32.061 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:32.058+0000 7fdc81cd6700 1 -- 172.21.15.59:0/2302379518 shutdown_connections 2023-11-24T08:00:32.061 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:32.058+0000 7fdc81cd6700 1 -- 172.21.15.59:0/2302379518 wait complete. 2023-11-24T08:00:32.062 INFO:teuthology.orchestra.run.smithi059.stderr:instructed osd(s) 5 to deep-scrub 2023-11-24T08:00:32.071 INFO:tasks.ceph:Scrubbing osd.6 2023-11-24T08:00:32.072 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 config set osd_debug_deep_scrub_sleep 0 2023-11-24T08:00:32.201 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:32.199+0000 7f1a6e646700 1 Processor -- start 2023-11-24T08:00:32.201 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:32.199+0000 7f1a6e646700 1 -- start start 2023-11-24T08:00:32.205 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:32.200+0000 7f1a6e646700 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f1a6815bfe0 0x7f1a680cd150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:00:32.205 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:32.200+0000 7f1a6e646700 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f1a680cefc0 0x7f1a6806f940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:00:32.205 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:32.200+0000 7f1a6e646700 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1a6806f4b0 con 0x7f1a680ce430 2023-11-24T08:00:32.205 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:32.200+0000 7f1a6e646700 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1a6815c3c0 con 0x7f1a6815bfe0 2023-11-24T08:00:32.205 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:32.200+0000 7f1a6e646700 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1a680cd760 con 0x7f1a680cefc0 2023-11-24T08:00:32.206 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:32.200+0000 7f1a677fe700 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7f1a680ce430 0x7f1a680ce8b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:60248/0 (socket says 172.21.15.59:60248) 2023-11-24T08:00:32.206 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:32.200+0000 7f1a677fe700 1 -- 172.21.15.59:0/2436185011 learned_addr learned my addr 172.21.15.59:0/2436185011 (peer_addr_for_me v1:172.21.15.59:0/0) 2023-11-24T08:00:32.206 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:32.200+0000 7f1a67fff700 1 --2- 172.21.15.59:0/2436185011 >> v2:172.21.15.76:3300/0 conn(0x7f1a6815bfe0 0x7f1a680cd150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:00:32.206 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:32.200+0000 7f1a66ffd700 1 -- 172.21.15.59:0/2436185011 <== mon.0 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2163725935 0 0) 0x7f1a6806f4b0 con 0x7f1a680ce430 2023-11-24T08:00:32.207 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:32.200+0000 7f1a6cbe3700 1 --2- 172.21.15.59:0/2436185011 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f1a680cefc0 0x7f1a6806f940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:00:32.207 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:32.200+0000 7f1a66ffd700 1 -- 172.21.15.59:0/2436185011 --> v1:172.21.15.59:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1a540035b0 con 0x7f1a680ce430 2023-11-24T08:00:32.207 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:32.201+0000 7f1a66ffd700 1 -- 172.21.15.59:0/2436185011 <== mon.0 v1:172.21.15.59:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2738098342 0 0) 0x7f1a540035b0 con 0x7f1a680ce430 2023-11-24T08:00:32.207 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:32.201+0000 7f1a66ffd700 1 -- 172.21.15.59:0/2436185011 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f1a680cefc0 msgr2=0x7f1a6806f940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:00:32.207 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:32.201+0000 7f1a66ffd700 1 --2- 172.21.15.59:0/2436185011 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f1a680cefc0 0x7f1a6806f940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:00:32.207 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:32.201+0000 7f1a66ffd700 1 -- 172.21.15.59:0/2436185011 >> v2:172.21.15.76:3300/0 conn(0x7f1a6815bfe0 msgr2=0x7f1a680cd150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:00:32.207 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:32.201+0000 7f1a66ffd700 1 --2- 172.21.15.59:0/2436185011 >> v2:172.21.15.76:3300/0 conn(0x7f1a6815bfe0 0x7f1a680cd150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:00:32.207 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:32.201+0000 7f1a66ffd700 1 -- 172.21.15.59:0/2436185011 --> v1:172.21.15.59:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1a6806ee60 con 0x7f1a680ce430 2023-11-24T08:00:32.208 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:32.201+0000 7f1a66ffd700 1 -- 172.21.15.59:0/2436185011 <== mon.0 v1:172.21.15.59:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1312333566 0 0) 0x7f1a58001670 con 0x7f1a680ce430 2023-11-24T08:00:32.208 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:32.201+0000 7f1a66ffd700 1 -- 172.21.15.59:0/2436185011 >> v1:172.21.15.59:6789/0 conn(0x7f1a680ce430 legacy=0x7f1a680ce8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:00:32.208 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:32.201+0000 7f1a66ffd700 1 --2- 172.21.15.59:0/2436185011 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1a54003a00 0x7f1a54003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:00:32.208 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:32.201+0000 7f1a66ffd700 1 --2- 172.21.15.59:0/2436185011 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f1a680cefc0 0x7f1a54004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:00:32.208 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:32.201+0000 7f1a66ffd700 1 --2- 172.21.15.59:0/2436185011 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1a54004d90 0x7f1a54005210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:00:32.208 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:32.201+0000 7f1a66ffd700 1 -- 172.21.15.59:0/2436185011 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a6815c3c0 con 0x7f1a54004d90 2023-11-24T08:00:32.208 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:32.201+0000 7f1a66ffd700 1 -- 172.21.15.59:0/2436185011 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a680cd760 con 0x7f1a54003a00 2023-11-24T08:00:32.208 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:32.201+0000 7f1a66ffd700 1 -- 172.21.15.59:0/2436185011 --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1a5400ba10 con 0x7f1a680cefc0 2023-11-24T08:00:32.209 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:32.201+0000 7f1a6cbe3700 1 --2- 172.21.15.59:0/2436185011 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1a54004d90 0x7f1a54005210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:00:32.209 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:32.201+0000 7f1a67fff700 1 --2- 172.21.15.59:0/2436185011 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1a54003a00 0x7f1a54003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:00:32.209 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:32.201+0000 7f1a677fe700 1 --2- 172.21.15.59:0/2436185011 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f1a680cefc0 0x7f1a54004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:00:32.209 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:32.201+0000 7f1a6cbe3700 1 -- 172.21.15.59:0/2436185011 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f1a680cefc0 msgr2=0x7f1a54004850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:00:32.209 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:32.201+0000 7f1a6cbe3700 1 --2- 172.21.15.59:0/2436185011 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f1a680cefc0 0x7f1a54004850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:00:32.209 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:32.201+0000 7f1a6cbe3700 1 -- 172.21.15.59:0/2436185011 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1a54003a00 msgr2=0x7f1a54003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:00:32.209 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:32.201+0000 7f1a6cbe3700 1 --2- 172.21.15.59:0/2436185011 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1a54003a00 0x7f1a54003e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:00:32.210 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:32.201+0000 7f1a6cbe3700 1 -- 172.21.15.59:0/2436185011 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1a540035b0 con 0x7f1a54004d90 2023-11-24T08:00:32.210 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:32.201+0000 7f1a677fe700 1 --2- 172.21.15.59:0/2436185011 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f1a680cefc0 0x7f1a54004850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-24T08:00:32.210 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:32.201+0000 7f1a6cbe3700 1 --2- 172.21.15.59:0/2436185011 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1a54004d90 0x7f1a54005210 secure :-1 s=READY pgs=1170 cs=0 l=1 rev1=1 crypto rx=0x7f1a60009f90 tx=0x7f1a54005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e4753552842e5dd3 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:00:32.210 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:32.201+0000 7f1a66ffd700 1 -- 172.21.15.59:0/2436185011 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a6000ec30 con 0x7f1a54004d90 2023-11-24T08:00:32.210 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:32.201+0000 7f1a66ffd700 1 -- 172.21.15.59:0/2436185011 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1a6000f040 con 0x7f1a54004d90 2023-11-24T08:00:32.210 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:32.201+0000 7f1a66ffd700 1 -- 172.21.15.59:0/2436185011 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a60014480 con 0x7f1a54004d90 2023-11-24T08:00:32.210 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:32.202+0000 7f1a6e646700 1 -- 172.21.15.59:0/2436185011 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1a54004d90 msgr2=0x7f1a54005210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:00:32.210 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:32.202+0000 7f1a6e646700 1 --2- 172.21.15.59:0/2436185011 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1a54004d90 0x7f1a54005210 secure :-1 s=READY pgs=1170 cs=0 l=1 rev1=1 crypto rx=0x7f1a60009f90 tx=0x7f1a54005880 comp rx=0 tx=0).stop 2023-11-24T08:00:32.211 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:32.202+0000 7f1a6cbe3700 1 -- 172.21.15.59:0/2436185011 reap_dead start 2023-11-24T08:00:32.211 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:32.202+0000 7f1a6e646700 1 -- 172.21.15.59:0/2436185011 shutdown_connections 2023-11-24T08:00:32.211 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:32.202+0000 7f1a6e646700 1 -- 172.21.15.59:0/2436185011 >> 172.21.15.59:0/2436185011 conn(0x7f1a680b3700 msgr2=0x7f1a680b3b10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:00:32.211 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:32.202+0000 7f1a6e646700 1 -- 172.21.15.59:0/2436185011 shutdown_connections 2023-11-24T08:00:32.211 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:32.202+0000 7f1a6e646700 1 -- 172.21.15.59:0/2436185011 wait complete. 2023-11-24T08:00:32.211 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:32.203+0000 7f1a6e646700 1 Processor -- start 2023-11-24T08:00:32.211 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:32.203+0000 7f1a6e646700 1 -- start start 2023-11-24T08:00:32.214 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:32.204+0000 7f1a6e646700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1a6814eef0 0x7f1a6815ccf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:00:32.215 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:32.204+0000 7f1a6e646700 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f1a681580e0 0x7f1a6815ea00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:00:32.215 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:32.204+0000 7f1a6e646700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1a6815ef40 0x7f1a680c5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:00:32.215 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:32.204+0000 7f1a6e646700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a680cb970 con 0x7f1a6814eef0 2023-11-24T08:00:32.215 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:32.204+0000 7f1a6e646700 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a6815d230 con 0x7f1a6815ef40 2023-11-24T08:00:32.215 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:32.204+0000 7f1a6e646700 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1a680c67b0 con 0x7f1a681580e0 2023-11-24T08:00:32.215 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:32.204+0000 7f1a67fff700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1a6814eef0 0x7f1a6815ccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:00:32.217 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:32.204+0000 7f1a67fff700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1a6814eef0 0x7f1a6815ccf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:56784/0 (socket says 172.21.15.59:56784) 2023-11-24T08:00:32.217 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:32.204+0000 7f1a67fff700 1 -- 172.21.15.59:0/1304193342 learned_addr learned my addr 172.21.15.59:0/1304193342 (peer_addr_for_me v2:172.21.15.59:0/0) 2023-11-24T08:00:32.217 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:32.204+0000 7f1a6cbe3700 1 --2- 172.21.15.59:0/1304193342 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1a6815ef40 0x7f1a680c5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:00:32.217 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:32.204+0000 7f1a677fe700 1 --2- 172.21.15.59:0/1304193342 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f1a681580e0 0x7f1a6815ea00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:00:32.217 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:32.205+0000 7f1a67fff700 1 -- 172.21.15.59:0/1304193342 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f1a681580e0 msgr2=0x7f1a6815ea00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:00:32.218 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:32.205+0000 7f1a67fff700 1 --2- 172.21.15.59:0/1304193342 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f1a681580e0 0x7f1a6815ea00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:00:32.218 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:32.205+0000 7f1a67fff700 1 -- 172.21.15.59:0/1304193342 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1a6815ef40 msgr2=0x7f1a680c5700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:00:32.218 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:32.205+0000 7f1a67fff700 1 --2- 172.21.15.59:0/1304193342 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1a6815ef40 0x7f1a680c5700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:00:32.218 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:32.205+0000 7f1a67fff700 1 -- 172.21.15.59:0/1304193342 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1a680c6a50 con 0x7f1a6814eef0 2023-11-24T08:00:32.218 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:32.205+0000 7f1a6cbe3700 1 --2- 172.21.15.59:0/1304193342 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1a6815ef40 0x7f1a680c5700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-24T08:00:32.218 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:32.205+0000 7f1a677fe700 1 --2- 172.21.15.59:0/1304193342 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f1a681580e0 0x7f1a6815ea00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-24T08:00:32.218 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:32.205+0000 7f1a67fff700 1 --2- 172.21.15.59:0/1304193342 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1a6814eef0 0x7f1a6815ccf0 secure :-1 s=READY pgs=1171 cs=0 l=1 rev1=1 crypto rx=0x7f1a50011ad0 tx=0x7f1a50011e90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:00:32.219 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:32.205+0000 7f1a64ff9700 1 -- 172.21.15.59:0/1304193342 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a5000fa00 con 0x7f1a6814eef0 2023-11-24T08:00:32.219 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:32.205+0000 7f1a64ff9700 1 -- 172.21.15.59:0/1304193342 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1a5000fb80 con 0x7f1a6814eef0 2023-11-24T08:00:32.219 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:32.205+0000 7f1a6e646700 1 -- 172.21.15.59:0/1304193342 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1a680c6d30 con 0x7f1a6814eef0 2023-11-24T08:00:32.219 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:32.205+0000 7f1a64ff9700 1 -- 172.21.15.59:0/1304193342 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a50014440 con 0x7f1a6814eef0 2023-11-24T08:00:32.219 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:32.205+0000 7f1a6e646700 1 -- 172.21.15.59:0/1304193342 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1a6815f3f0 con 0x7f1a6814eef0 2023-11-24T08:00:32.219 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:32.206+0000 7f1a6e646700 1 -- 172.21.15.59:0/1304193342 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f1a680cda40 con 0x7f1a6814eef0 2023-11-24T08:00:32.219 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:32.206+0000 7f1a64ff9700 1 -- 172.21.15.59:0/1304193342 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 12) v1 ==== 89792+0+0 (secure 0 0 0) 0x7f1a500263f0 con 0x7f1a6814eef0 2023-11-24T08:00:32.220 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:32.207+0000 7f1a64ff9700 1 --2- 172.21.15.59:0/1304193342 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f1a4806b500 0x7f1a4806d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:00:32.220 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:32.207+0000 7f1a64ff9700 1 -- 172.21.15.59:0/1304193342 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f1a5008ab60 con 0x7f1a6814eef0 2023-11-24T08:00:32.220 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:32.207+0000 7f1a677fe700 1 --2- 172.21.15.59:0/1304193342 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f1a4806b500 0x7f1a4806d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:00:32.220 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:32.207+0000 7f1a64ff9700 1 --2- 172.21.15.59:0/1304193342 >> [v2:172.21.15.76:6801/3946579410,v1:172.21.15.76:6803/3946579410] conn(0x7f1a48071360 0x7f1a48073780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:00:32.220 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:32.207+0000 7f1a64ff9700 1 -- 172.21.15.59:0/1304193342 --> [v2:172.21.15.76:6801/3946579410,v1:172.21.15.76:6803/3946579410] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f1a48073e50 con 0x7f1a48071360 2023-11-24T08:00:32.220 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:32.207+0000 7f1a64ff9700 1 -- 172.21.15.59:0/1304193342 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_get_version_reply(handle=1 version=90) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1a5008adc0 con 0x7f1a6814eef0 2023-11-24T08:00:32.220 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:32.207+0000 7f1a6cbe3700 1 --2- 172.21.15.59:0/1304193342 >> [v2:172.21.15.76:6801/3946579410,v1:172.21.15.76:6803/3946579410] conn(0x7f1a48071360 0x7f1a48073780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:00:32.220 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:32.207+0000 7f1a677fe700 1 --2- 172.21.15.59:0/1304193342 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f1a4806b500 0x7f1a4806d9c0 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7f1a58000c00 tx=0x7f1a58009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:00:32.221 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:32.208+0000 7f1a6cbe3700 1 --2- 172.21.15.59:0/1304193342 >> [v2:172.21.15.76:6801/3946579410,v1:172.21.15.76:6803/3946579410] conn(0x7f1a48071360 0x7f1a48073780 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:00:32.221 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:32.208+0000 7f1a64ff9700 1 -- 172.21.15.59:0/1304193342 <== osd.6 v2:172.21.15.76:6801/3946579410 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+26965 (crc 0 0 0) 0x7f1a48073e50 con 0x7f1a48071360 2023-11-24T08:00:32.225 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:32.224+0000 7f1a6e646700 1 -- 172.21.15.59:0/1304193342 --> [v2:172.21.15.76:6801/3946579410,v1:172.21.15.76:6803/3946579410] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f1a680d09d0 con 0x7f1a48071360 2023-11-24T08:00:32.225 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:32.224+0000 7f1a64ff9700 1 -- 172.21.15.59:0/1304193342 <== osd.6 v2:172.21.15.76:6801/3946579410 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+97 (crc 0 0 0) 0x7f1a6000ebd0 con 0x7f1a48071360 2023-11-24T08:00:32.226 INFO:teuthology.orchestra.run.smithi059.stdout:{ 2023-11-24T08:00:32.226 INFO:teuthology.orchestra.run.smithi059.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2023-11-24T08:00:32.226 INFO:teuthology.orchestra.run.smithi059.stdout:} 2023-11-24T08:00:32.227 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:32.224+0000 7f1a6e646700 1 -- 172.21.15.59:0/1304193342 >> [v2:172.21.15.76:6801/3946579410,v1:172.21.15.76:6803/3946579410] conn(0x7f1a48071360 msgr2=0x7f1a48073780 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:00:32.227 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:32.224+0000 7f1a6e646700 1 --2- 172.21.15.59:0/1304193342 >> [v2:172.21.15.76:6801/3946579410,v1:172.21.15.76:6803/3946579410] conn(0x7f1a48071360 0x7f1a48073780 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:00:32.228 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:32.224+0000 7f1a6e646700 1 -- 172.21.15.59:0/1304193342 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f1a4806b500 msgr2=0x7f1a4806d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:00:32.228 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:32.224+0000 7f1a6e646700 1 --2- 172.21.15.59:0/1304193342 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f1a4806b500 0x7f1a4806d9c0 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7f1a58000c00 tx=0x7f1a58009040 comp rx=0 tx=0).stop 2023-11-24T08:00:32.228 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:32.224+0000 7f1a6e646700 1 -- 172.21.15.59:0/1304193342 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1a6814eef0 msgr2=0x7f1a6815ccf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:00:32.228 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:32.224+0000 7f1a6e646700 1 --2- 172.21.15.59:0/1304193342 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1a6814eef0 0x7f1a6815ccf0 secure :-1 s=READY pgs=1171 cs=0 l=1 rev1=1 crypto rx=0x7f1a50011ad0 tx=0x7f1a50011e90 comp rx=0 tx=0).stop 2023-11-24T08:00:32.228 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:32.224+0000 7f1a6cbe3700 1 -- 172.21.15.59:0/1304193342 reap_dead start 2023-11-24T08:00:32.228 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:32.225+0000 7f1a6e646700 1 -- 172.21.15.59:0/1304193342 shutdown_connections 2023-11-24T08:00:32.228 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:32.225+0000 7f1a6e646700 1 -- 172.21.15.59:0/1304193342 >> 172.21.15.59:0/1304193342 conn(0x7f1a680b3700 msgr2=0x7f1a680b0990 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:00:32.228 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:32.225+0000 7f1a6e646700 1 -- 172.21.15.59:0/1304193342 shutdown_connections 2023-11-24T08:00:32.229 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:32.225+0000 7f1a6e646700 1 -- 172.21.15.59:0/1304193342 wait complete. 2023-11-24T08:00:32.238 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 6 2023-11-24T08:00:32.363 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:32.362+0000 7f716fb00700 1 Processor -- start 2023-11-24T08:00:32.364 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:32.362+0000 7f716fb00700 1 -- start start 2023-11-24T08:00:32.366 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:32.363+0000 7f716fb00700 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f716815a440 0x7f71680cffa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:00:32.366 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:32.363+0000 7f716fb00700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f716815eb80 0x7f71680d04e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:00:32.366 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:32.363+0000 7f716fb00700 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f71680b05f0 con 0x7f716815a820 2023-11-24T08:00:32.367 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:32.363+0000 7f716fb00700 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f716815f080 con 0x7f716815eb80 2023-11-24T08:00:32.367 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:32.363+0000 7f716fb00700 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f71680afd50 con 0x7f716815a440 2023-11-24T08:00:32.367 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:32.363+0000 7f716e09d700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f716815eb80 0x7f71680d04e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:00:32.367 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:32.363+0000 7f716d09b700 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7f716815a820 0x7f71680726a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:60268/0 (socket says 172.21.15.59:60268) 2023-11-24T08:00:32.367 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:32.363+0000 7f716e09d700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f716815eb80 0x7f71680d04e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:56798/0 (socket says 172.21.15.59:56798) 2023-11-24T08:00:32.367 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:32.363+0000 7f716d09b700 1 -- 172.21.15.59:0/2638665456 learned_addr learned my addr 172.21.15.59:0/2638665456 (peer_addr_for_me v1:172.21.15.59:0/0) 2023-11-24T08:00:32.367 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:32.363+0000 7f716d89c700 1 --2- 172.21.15.59:0/2638665456 >> v2:172.21.15.76:3300/0 conn(0x7f716815a440 0x7f71680cffa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:00:32.367 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:32.363+0000 7f716e09d700 1 -- 172.21.15.59:0/2638665456 >> v2:172.21.15.76:3300/0 conn(0x7f716815a440 msgr2=0x7f71680cffa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:00:32.368 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:32.363+0000 7f716e09d700 1 --2- 172.21.15.59:0/2638665456 >> v2:172.21.15.76:3300/0 conn(0x7f716815a440 0x7f71680cffa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:00:32.368 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:32.363+0000 7f716e09d700 1 -- 172.21.15.59:0/2638665456 >> v1:172.21.15.59:6789/0 conn(0x7f716815a820 legacy=0x7f71680726a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:00:32.368 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:32.363+0000 7f716c89a700 1 -- 172.21.15.59:0/2638665456 <== mon.0 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 790685600 0 0) 0x7f71680b05f0 con 0x7f716815a820 2023-11-24T08:00:32.368 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:32.363+0000 7f716e09d700 1 -- 172.21.15.59:0/2638665456 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f71680b0030 con 0x7f716815eb80 2023-11-24T08:00:32.368 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:32.364+0000 7f716e09d700 1 --2- 172.21.15.59:0/2638665456 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f716815eb80 0x7f71680d04e0 secure :-1 s=READY pgs=1173 cs=0 l=1 rev1=1 crypto rx=0x7f7168069520 tx=0x7f716000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dced031f251f78b6 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:00:32.368 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:32.364+0000 7f716c89a700 1 -- 172.21.15.59:0/2638665456 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f716000e070 con 0x7f716815eb80 2023-11-24T08:00:32.368 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:32.364+0000 7f716c89a700 1 -- 172.21.15.59:0/2638665456 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f716000bbd0 con 0x7f716815eb80 2023-11-24T08:00:32.368 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:32.364+0000 7f716c89a700 1 -- 172.21.15.59:0/2638665456 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7160012430 con 0x7f716815eb80 2023-11-24T08:00:32.369 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:32.364+0000 7f716fb00700 1 -- 172.21.15.59:0/2638665456 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f716815eb80 msgr2=0x7f71680d04e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:00:32.369 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:32.364+0000 7f716fb00700 1 --2- 172.21.15.59:0/2638665456 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f716815eb80 0x7f71680d04e0 secure :-1 s=READY pgs=1173 cs=0 l=1 rev1=1 crypto rx=0x7f7168069520 tx=0x7f716000b330 comp rx=0 tx=0).stop 2023-11-24T08:00:32.369 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:32.364+0000 7f716fb00700 1 -- 172.21.15.59:0/2638665456 shutdown_connections 2023-11-24T08:00:32.369 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:32.364+0000 7f716fb00700 1 --2- 172.21.15.59:0/2638665456 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f716815eb80 0x7f71680d04e0 unknown :-1 s=CLOSED pgs=1173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:00:32.369 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:32.364+0000 7f716fb00700 1 --2- 172.21.15.59:0/2638665456 >> v2:172.21.15.76:3300/0 conn(0x7f716815a440 0x7f71680cffa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:00:32.369 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:32.364+0000 7f716fb00700 1 -- 172.21.15.59:0/2638665456 >> 172.21.15.59:0/2638665456 conn(0x7f71680b5070 msgr2=0x7f71680b5730 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:00:32.369 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:32.364+0000 7f716fb00700 1 -- 172.21.15.59:0/2638665456 shutdown_connections 2023-11-24T08:00:32.369 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:32.364+0000 7f716fb00700 1 -- 172.21.15.59:0/2638665456 wait complete. 2023-11-24T08:00:32.372 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:32.365+0000 7f716fb00700 1 Processor -- start 2023-11-24T08:00:32.372 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:32.366+0000 7f716fb00700 1 -- start start 2023-11-24T08:00:32.373 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:32.366+0000 7f716fb00700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f716815a440 0x7f7168099810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:00:32.373 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:32.366+0000 7f716fb00700 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f716815eb80 0x7f71680ad1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:00:32.373 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:32.366+0000 7f716fb00700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f716806eea0 0x7f71680ad730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:00:32.373 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:32.366+0000 7f716fb00700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71680cfb60 con 0x7f716806eea0 2023-11-24T08:00:32.373 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:32.366+0000 7f716fb00700 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7168167d40 con 0x7f716815a440 2023-11-24T08:00:32.373 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:32.366+0000 7f716fb00700 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7168167a60 con 0x7f716815eb80 2023-11-24T08:00:32.373 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:32.366+0000 7f716e09d700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f716806eea0 0x7f71680ad730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:00:32.373 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:32.366+0000 7f716e09d700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f716806eea0 0x7f71680ad730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:56804/0 (socket says 172.21.15.59:56804) 2023-11-24T08:00:32.374 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:32.366+0000 7f716e09d700 1 -- 172.21.15.59:0/4248840226 learned_addr learned my addr 172.21.15.59:0/4248840226 (peer_addr_for_me v2:172.21.15.59:0/0) 2023-11-24T08:00:32.374 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:32.367+0000 7f716d89c700 1 --2- 172.21.15.59:0/4248840226 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f716815a440 0x7f7168099810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:00:32.374 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:32.367+0000 7f716d09b700 1 --2- 172.21.15.59:0/4248840226 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f716815eb80 0x7f71680ad1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:00:32.374 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:32.367+0000 7f716e09d700 1 -- 172.21.15.59:0/4248840226 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f716815eb80 msgr2=0x7f71680ad1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:00:32.374 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:32.367+0000 7f716e09d700 1 --2- 172.21.15.59:0/4248840226 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f716815eb80 0x7f71680ad1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:00:32.374 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:32.367+0000 7f716e09d700 1 -- 172.21.15.59:0/4248840226 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f716815a440 msgr2=0x7f7168099810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:00:32.374 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:32.367+0000 7f716e09d700 1 --2- 172.21.15.59:0/4248840226 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f716815a440 0x7f7168099810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:00:32.374 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:32.367+0000 7f716e09d700 1 -- 172.21.15.59:0/4248840226 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7160009a70 con 0x7f716806eea0 2023-11-24T08:00:32.375 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:32.367+0000 7f716d89c700 1 --2- 172.21.15.59:0/4248840226 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f716815a440 0x7f7168099810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T08:00:32.375 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:32.367+0000 7f716e09d700 1 --2- 172.21.15.59:0/4248840226 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f716806eea0 0x7f71680ad730 secure :-1 s=READY pgs=1174 cs=0 l=1 rev1=1 crypto rx=0x7f7160013000 tx=0x7f7160004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:00:32.375 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:32.367+0000 7f715e7fc700 1 -- 172.21.15.59:0/4248840226 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7160012430 con 0x7f716806eea0 2023-11-24T08:00:32.375 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:32.367+0000 7f715e7fc700 1 -- 172.21.15.59:0/4248840226 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7160020070 con 0x7f716806eea0 2023-11-24T08:00:32.375 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:32.367+0000 7f716fb00700 1 -- 172.21.15.59:0/4248840226 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f71680cf080 con 0x7f716806eea0 2023-11-24T08:00:32.375 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:32.367+0000 7f715e7fc700 1 -- 172.21.15.59:0/4248840226 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7160004280 con 0x7f716806eea0 2023-11-24T08:00:32.375 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:32.367+0000 7f716fb00700 1 -- 172.21.15.59:0/4248840226 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f71680cf640 con 0x7f716806eea0 2023-11-24T08:00:32.375 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:32.368+0000 7f714bfff700 1 -- 172.21.15.59:0/4248840226 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f71440052e0 con 0x7f716806eea0 2023-11-24T08:00:32.376 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:32.369+0000 7f715e7fc700 1 -- 172.21.15.59:0/4248840226 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 12) v1 ==== 89792+0+0 (secure 0 0 0) 0x7f71600043e0 con 0x7f716806eea0 2023-11-24T08:00:32.376 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:32.369+0000 7f715e7fc700 1 --2- 172.21.15.59:0/4248840226 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f714c06b500 0x7f714c06d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:00:32.376 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:32.369+0000 7f715e7fc700 1 -- 172.21.15.59:0/4248840226 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f7160022070 con 0x7f716806eea0 2023-11-24T08:00:32.376 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:32.369+0000 7f716d89c700 1 --2- 172.21.15.59:0/4248840226 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f714c06b500 0x7f714c06d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:00:32.376 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:32.370+0000 7f716d89c700 1 --2- 172.21.15.59:0/4248840226 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f714c06b500 0x7f714c06d9c0 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7f7158009a00 tx=0x7f7158008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:00:32.376 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:32.373+0000 7f715e7fc700 1 -- 172.21.15.59:0/4248840226 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f7160056830 con 0x7f716806eea0 2023-11-24T08:00:32.579 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:32.577+0000 7f714bfff700 1 -- 172.21.15.59:0/4248840226 --> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "6", "target": ["mon-mgr", ""]}) v1 -- 0x7f7144000bb0 con 0x7f714c06b500 2023-11-24T08:00:32.579 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:32.578+0000 7f715e7fc700 1 -- 172.21.15.59:0/4248840226 <== mgr.4100 v2:172.21.15.76:6832/4264046536 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 6 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f7144000bb0 con 0x7f714c06b500 2023-11-24T08:00:32.582 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:32.580+0000 7f714bfff700 1 -- 172.21.15.59:0/4248840226 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f714c06b500 msgr2=0x7f714c06d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:00:32.582 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:32.580+0000 7f714bfff700 1 --2- 172.21.15.59:0/4248840226 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f714c06b500 0x7f714c06d9c0 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7f7158009a00 tx=0x7f7158008040 comp rx=0 tx=0).stop 2023-11-24T08:00:32.582 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:32.580+0000 7f714bfff700 1 -- 172.21.15.59:0/4248840226 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f716806eea0 msgr2=0x7f71680ad730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:00:32.582 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:32.580+0000 7f714bfff700 1 --2- 172.21.15.59:0/4248840226 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f716806eea0 0x7f71680ad730 secure :-1 s=READY pgs=1174 cs=0 l=1 rev1=1 crypto rx=0x7f7160013000 tx=0x7f7160004ad0 comp rx=0 tx=0).stop 2023-11-24T08:00:32.582 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:32.580+0000 7f714bfff700 1 -- 172.21.15.59:0/4248840226 shutdown_connections 2023-11-24T08:00:32.583 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:32.580+0000 7f714bfff700 1 --2- 172.21.15.59:0/4248840226 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f714c06b500 0x7f714c06d9c0 unknown :-1 s=CLOSED pgs=533 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:00:32.583 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:32.580+0000 7f714bfff700 1 --2- 172.21.15.59:0/4248840226 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f716806eea0 0x7f71680ad730 unknown :-1 s=CLOSED pgs=1174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:00:32.583 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:32.580+0000 7f714bfff700 1 --2- 172.21.15.59:0/4248840226 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f716815a440 0x7f7168099810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:00:32.583 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:32.580+0000 7f714bfff700 1 --2- 172.21.15.59:0/4248840226 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f716815eb80 0x7f71680ad1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:00:32.584 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:32.580+0000 7f714bfff700 1 -- 172.21.15.59:0/4248840226 >> 172.21.15.59:0/4248840226 conn(0x7f71680b5070 msgr2=0x7f716806dbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:00:32.584 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:32.580+0000 7f714bfff700 1 -- 172.21.15.59:0/4248840226 shutdown_connections 2023-11-24T08:00:32.584 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:32.580+0000 7f714bfff700 1 -- 172.21.15.59:0/4248840226 wait complete. 2023-11-24T08:00:32.585 INFO:teuthology.orchestra.run.smithi059.stderr:instructed osd(s) 6 to deep-scrub 2023-11-24T08:00:32.594 INFO:tasks.ceph:Scrubbing osd.7 2023-11-24T08:00:32.595 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 config set osd_debug_deep_scrub_sleep 0 2023-11-24T08:00:32.725 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:32.723+0000 7f47b2a1b700 1 Processor -- start 2023-11-24T08:00:32.725 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:32.723+0000 7f47b2a1b700 1 -- start start 2023-11-24T08:00:32.727 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:32.724+0000 7f47b2a1b700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f47ac070b50 0x7f47ac0ce640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:00:32.727 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:32.724+0000 7f47b2a1b700 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f47ac0d2bd0 con 0x7f47ac15f6f0 2023-11-24T08:00:32.728 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:32.724+0000 7f47b2a1b700 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f47ac06f4d0 con 0x7f47ac0700b0 2023-11-24T08:00:32.728 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:32.724+0000 7f47b2a1b700 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f47ac15fad0 con 0x7f47ac070b50 2023-11-24T08:00:32.728 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:32.724+0000 7f47ab7fe700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f47ac070b50 0x7f47ac0ce640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:00:32.728 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:32.724+0000 7f47abfff700 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7f47ac15f6f0 0x7f47ac06f960 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:60282/0 (socket says 172.21.15.59:60282) 2023-11-24T08:00:32.728 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:32.724+0000 7f47ab7fe700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f47ac070b50 0x7f47ac0ce640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:56810/0 (socket says 172.21.15.59:56810) 2023-11-24T08:00:32.728 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:32.724+0000 7f47abfff700 1 -- 172.21.15.59:0/1244036379 learned_addr learned my addr 172.21.15.59:0/1244036379 (peer_addr_for_me v1:172.21.15.59:0/0) 2023-11-24T08:00:32.728 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:32.724+0000 7f47ab7fe700 1 -- 172.21.15.59:0/1244036379 >> v1:172.21.15.76:6789/0 conn(0x7f47ac0700b0 legacy=0x7f47ac0cc660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:00:32.728 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:32.724+0000 7f47ab7fe700 1 -- 172.21.15.59:0/1244036379 >> v1:172.21.15.59:6789/0 conn(0x7f47ac15f6f0 legacy=0x7f47ac06f960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:00:32.729 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:32.724+0000 7f47ab7fe700 1 -- 172.21.15.59:0/1244036379 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f47ac0d28f0 con 0x7f47ac070b50 2023-11-24T08:00:32.729 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:32.724+0000 7f47aaffd700 1 -- 172.21.15.59:0/1244036379 <== mon.0 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 991525352 0 0) 0x7f47ac0d2bd0 con 0x7f47ac15f6f0 2023-11-24T08:00:32.729 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:32.724+0000 7f47ab7fe700 1 --2- 172.21.15.59:0/1244036379 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f47ac070b50 0x7f47ac0ce640 secure :-1 s=READY pgs=1176 cs=0 l=1 rev1=1 crypto rx=0x7f479c009840 tx=0x7f479c009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4b546f657b6a77b5 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:00:32.729 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:32.725+0000 7f47aaffd700 1 -- 172.21.15.59:0/1244036379 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f479c004230 con 0x7f47ac070b50 2023-11-24T08:00:32.729 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:32.725+0000 7f47aaffd700 1 -- 172.21.15.59:0/1244036379 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f479c004390 con 0x7f47ac070b50 2023-11-24T08:00:32.729 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:32.725+0000 7f47aaffd700 1 -- 172.21.15.59:0/1244036379 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f479c0036a0 con 0x7f47ac070b50 2023-11-24T08:00:32.729 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:32.725+0000 7f47b2a1b700 1 -- 172.21.15.59:0/1244036379 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f47ac070b50 msgr2=0x7f47ac0ce640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:00:32.729 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:32.725+0000 7f47b2a1b700 1 --2- 172.21.15.59:0/1244036379 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f47ac070b50 0x7f47ac0ce640 secure :-1 s=READY pgs=1176 cs=0 l=1 rev1=1 crypto rx=0x7f479c009840 tx=0x7f479c009b50 comp rx=0 tx=0).stop 2023-11-24T08:00:32.730 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:32.725+0000 7f47b2a1b700 1 -- 172.21.15.59:0/1244036379 shutdown_connections 2023-11-24T08:00:32.730 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:32.725+0000 7f47b2a1b700 1 --2- 172.21.15.59:0/1244036379 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f47ac070b50 0x7f47ac0ce640 unknown :-1 s=CLOSED pgs=1176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:00:32.730 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:32.725+0000 7f47b2a1b700 1 -- 172.21.15.59:0/1244036379 >> 172.21.15.59:0/1244036379 conn(0x7f47ac0d1c30 msgr2=0x7f47ac0d2040 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:00:32.730 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:32.725+0000 7f47b2a1b700 1 -- 172.21.15.59:0/1244036379 shutdown_connections 2023-11-24T08:00:32.730 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:32.725+0000 7f47b2a1b700 1 -- 172.21.15.59:0/1244036379 wait complete. 2023-11-24T08:00:32.734 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:32.726+0000 7f47b2a1b700 1 Processor -- start 2023-11-24T08:00:32.734 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:32.727+0000 7f47b2a1b700 1 -- start start 2023-11-24T08:00:32.735 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:32.727+0000 7f47b2a1b700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f47ac0700b0 0x7f47ac1531e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:00:32.735 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:32.727+0000 7f47b2a1b700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f47ac070b50 0x7f47ac06dbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:00:32.735 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:32.727+0000 7f47b2a1b700 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f47ac15f6f0 0x7f47ac15a0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:00:32.735 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:32.727+0000 7f47b2a1b700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47ac153720 con 0x7f47ac070b50 2023-11-24T08:00:32.735 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:32.727+0000 7f47b2a1b700 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47ac06e100 con 0x7f47ac0700b0 2023-11-24T08:00:32.735 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:32.727+0000 7f47b2a1b700 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f47ac0b6620 con 0x7f47ac15f6f0 2023-11-24T08:00:32.735 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:32.727+0000 7f47ab7fe700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f47ac070b50 0x7f47ac06dbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:00:32.735 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:32.727+0000 7f47ab7fe700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f47ac070b50 0x7f47ac06dbc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:56816/0 (socket says 172.21.15.59:56816) 2023-11-24T08:00:32.736 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:32.727+0000 7f47ab7fe700 1 -- 172.21.15.59:0/951347147 learned_addr learned my addr 172.21.15.59:0/951347147 (peer_addr_for_me v2:172.21.15.59:0/0) 2023-11-24T08:00:32.736 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:32.728+0000 7f47abfff700 1 --2- 172.21.15.59:0/951347147 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f47ac0700b0 0x7f47ac1531e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:00:32.736 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:32.728+0000 7f47ab7fe700 1 -- 172.21.15.59:0/951347147 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f47ac15f6f0 msgr2=0x7f47ac15a0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:00:32.736 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:32.728+0000 7f47ab7fe700 1 --2- 172.21.15.59:0/951347147 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f47ac15f6f0 0x7f47ac15a0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:00:32.736 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:32.728+0000 7f47ab7fe700 1 -- 172.21.15.59:0/951347147 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f47ac0700b0 msgr2=0x7f47ac1531e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:00:32.736 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:32.728+0000 7f47ab7fe700 1 --2- 172.21.15.59:0/951347147 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f47ac0700b0 0x7f47ac1531e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:00:32.736 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:32.728+0000 7f47ab7fe700 1 -- 172.21.15.59:0/951347147 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f479c0094a0 con 0x7f47ac070b50 2023-11-24T08:00:32.736 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:32.728+0000 7f47ab7fe700 1 --2- 172.21.15.59:0/951347147 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f47ac070b50 0x7f47ac06dbc0 secure :-1 s=READY pgs=1177 cs=0 l=1 rev1=1 crypto rx=0x7f479c0090b0 tx=0x7f479c004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:00:32.737 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:32.728+0000 7f47a8ff9700 1 -- 172.21.15.59:0/951347147 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f479c011630 con 0x7f47ac070b50 2023-11-24T08:00:32.737 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:32.728+0000 7f47a8ff9700 1 -- 172.21.15.59:0/951347147 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f479c01e070 con 0x7f47ac070b50 2023-11-24T08:00:32.737 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:32.728+0000 7f47b2a1b700 1 -- 172.21.15.59:0/951347147 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f47ac06ceb0 con 0x7f47ac070b50 2023-11-24T08:00:32.737 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:32.728+0000 7f47a8ff9700 1 -- 172.21.15.59:0/951347147 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f479c011990 con 0x7f47ac070b50 2023-11-24T08:00:32.737 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:32.728+0000 7f47b2a1b700 1 -- 172.21.15.59:0/951347147 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f47ac0b62b0 con 0x7f47ac070b50 2023-11-24T08:00:32.737 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:32.729+0000 7f47b2a1b700 1 -- 172.21.15.59:0/951347147 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f47ac0706e0 con 0x7f47ac070b50 2023-11-24T08:00:32.737 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:32.729+0000 7f47a8ff9700 1 -- 172.21.15.59:0/951347147 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 12) v1 ==== 89792+0+0 (secure 0 0 0) 0x7f479c011af0 con 0x7f47ac070b50 2023-11-24T08:00:32.737 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:32.730+0000 7f47a8ff9700 1 --2- 172.21.15.59:0/951347147 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f478c06b500 0x7f478c06d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:00:32.738 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:32.730+0000 7f47a8ff9700 1 -- 172.21.15.59:0/951347147 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f479c08b250 con 0x7f47ac070b50 2023-11-24T08:00:32.738 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:32.730+0000 7f47a8ff9700 1 --2- 172.21.15.59:0/951347147 >> [v2:172.21.15.76:6817/841761538,v1:172.21.15.76:6819/841761538] conn(0x7f478c071360 0x7f478c073780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:00:32.738 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:32.730+0000 7f47a8ff9700 1 -- 172.21.15.59:0/951347147 --> [v2:172.21.15.76:6817/841761538,v1:172.21.15.76:6819/841761538] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f478c073e50 con 0x7f478c071360 2023-11-24T08:00:32.738 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:32.730+0000 7f47a8ff9700 1 -- 172.21.15.59:0/951347147 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_get_version_reply(handle=1 version=90) v2 ==== 24+0+0 (secure 0 0 0) 0x7f479c08d780 con 0x7f47ac070b50 2023-11-24T08:00:32.738 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:32.730+0000 7f47abfff700 1 --2- 172.21.15.59:0/951347147 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f478c06b500 0x7f478c06d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:00:32.738 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:32.730+0000 7f47b0fb8700 1 --2- 172.21.15.59:0/951347147 >> [v2:172.21.15.76:6817/841761538,v1:172.21.15.76:6819/841761538] conn(0x7f478c071360 0x7f478c073780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:00:32.738 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:32.731+0000 7f47abfff700 1 --2- 172.21.15.59:0/951347147 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f478c06b500 0x7f478c06d9c0 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7f4794000f60 tx=0x7f4794009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:00:32.738 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:32.731+0000 7f47b0fb8700 1 --2- 172.21.15.59:0/951347147 >> [v2:172.21.15.76:6817/841761538,v1:172.21.15.76:6819/841761538] conn(0x7f478c071360 0x7f478c073780 crc :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:00:32.739 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:32.731+0000 7f47a8ff9700 1 -- 172.21.15.59:0/951347147 <== osd.7 v2:172.21.15.76:6817/841761538 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+26965 (crc 0 0 0) 0x7f478c073e50 con 0x7f478c071360 2023-11-24T08:00:32.748 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:32.746+0000 7f47b2a1b700 1 -- 172.21.15.59:0/951347147 --> [v2:172.21.15.76:6817/841761538,v1:172.21.15.76:6819/841761538] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f47ac06f160 con 0x7f478c071360 2023-11-24T08:00:32.749 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:32.747+0000 7f47a8ff9700 1 -- 172.21.15.59:0/951347147 <== osd.7 v2:172.21.15.76:6817/841761538 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+97 (crc 0 0 0) 0x7f47a40049b0 con 0x7f478c071360 2023-11-24T08:00:32.749 INFO:teuthology.orchestra.run.smithi059.stdout:{ 2023-11-24T08:00:32.749 INFO:teuthology.orchestra.run.smithi059.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2023-11-24T08:00:32.750 INFO:teuthology.orchestra.run.smithi059.stdout:} 2023-11-24T08:00:32.751 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:32.747+0000 7f47b2a1b700 1 -- 172.21.15.59:0/951347147 >> [v2:172.21.15.76:6817/841761538,v1:172.21.15.76:6819/841761538] conn(0x7f478c071360 msgr2=0x7f478c073780 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:00:32.752 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:32.747+0000 7f47b2a1b700 1 --2- 172.21.15.59:0/951347147 >> [v2:172.21.15.76:6817/841761538,v1:172.21.15.76:6819/841761538] conn(0x7f478c071360 0x7f478c073780 crc :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:00:32.752 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:32.747+0000 7f47b2a1b700 1 -- 172.21.15.59:0/951347147 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f478c06b500 msgr2=0x7f478c06d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:00:32.752 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:32.747+0000 7f47b2a1b700 1 --2- 172.21.15.59:0/951347147 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f478c06b500 0x7f478c06d9c0 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7f4794000f60 tx=0x7f4794009040 comp rx=0 tx=0).stop 2023-11-24T08:00:32.753 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:32.747+0000 7f47b2a1b700 1 -- 172.21.15.59:0/951347147 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f47ac070b50 msgr2=0x7f47ac06dbc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:00:32.753 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:32.747+0000 7f47b2a1b700 1 --2- 172.21.15.59:0/951347147 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f47ac070b50 0x7f47ac06dbc0 secure :-1 s=READY pgs=1177 cs=0 l=1 rev1=1 crypto rx=0x7f479c0090b0 tx=0x7f479c004750 comp rx=0 tx=0).stop 2023-11-24T08:00:32.753 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:32.747+0000 7f47b0fb8700 1 -- 172.21.15.59:0/951347147 reap_dead start 2023-11-24T08:00:32.754 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:32.748+0000 7f47b2a1b700 1 -- 172.21.15.59:0/951347147 shutdown_connections 2023-11-24T08:00:32.754 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:32.748+0000 7f47b2a1b700 1 -- 172.21.15.59:0/951347147 >> 172.21.15.59:0/951347147 conn(0x7f47ac0d1c30 msgr2=0x7f47ac0b1690 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:00:32.754 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:32.748+0000 7f47b2a1b700 1 -- 172.21.15.59:0/951347147 shutdown_connections 2023-11-24T08:00:32.754 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:32.748+0000 7f47b2a1b700 1 -- 172.21.15.59:0/951347147 wait complete. 2023-11-24T08:00:32.761 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 7 2023-11-24T08:00:32.887 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:32.885+0000 7fea0082d700 1 Processor -- start 2023-11-24T08:00:32.887 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:32.885+0000 7fea0082d700 1 -- start start 2023-11-24T08:00:32.892 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:32.886+0000 7fea0082d700 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fe9fc143ae0 0x7fe9fc0b51b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:00:32.892 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:32.886+0000 7fea0082d700 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe9fc143ec0 con 0x7fe9fc1440f0 2023-11-24T08:00:32.892 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:32.886+0000 7fea0082d700 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe9fc0b3ae0 con 0x7fe9fc0722c0 2023-11-24T08:00:32.892 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:32.886+0000 7fea0082d700 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe9fc0b40a0 con 0x7fe9fc143ae0 2023-11-24T08:00:32.892 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:32.886+0000 7fe9f9d9b700 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7fe9fc1440f0 0x7fe9fc071bb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:60300/0 (socket says 172.21.15.59:60300) 2023-11-24T08:00:32.892 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:32.886+0000 7fe9f9d9b700 1 -- 172.21.15.59:0/921780868 learned_addr learned my addr 172.21.15.59:0/921780868 (peer_addr_for_me v1:172.21.15.59:0/0) 2023-11-24T08:00:32.892 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:32.887+0000 7fe9fa59c700 1 --2- 172.21.15.59:0/921780868 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fe9fc143ae0 0x7fe9fc0b51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:00:32.893 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:32.887+0000 7fe9f959a700 1 -- 172.21.15.59:0/921780868 <== mon.0 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 623838249 0 0) 0x7fe9fc143ec0 con 0x7fe9fc1440f0 2023-11-24T08:00:32.893 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:32.887+0000 7fe9f959a700 1 -- 172.21.15.59:0/921780868 --> v1:172.21.15.59:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe9e80035b0 con 0x7fe9fc1440f0 2023-11-24T08:00:32.893 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:32.887+0000 7fe9f959a700 1 -- 172.21.15.59:0/921780868 <== mon.0 v1:172.21.15.59:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 729228687 0 0) 0x7fe9e80035b0 con 0x7fe9fc1440f0 2023-11-24T08:00:32.893 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:32.887+0000 7fe9f959a700 1 -- 172.21.15.59:0/921780868 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fe9fc143ae0 msgr2=0x7fe9fc0b51b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:00:32.893 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:32.887+0000 7fe9f959a700 1 --2- 172.21.15.59:0/921780868 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fe9fc143ae0 0x7fe9fc0b51b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:00:32.893 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:32.887+0000 7fe9f959a700 1 -- 172.21.15.59:0/921780868 >> v1:172.21.15.76:6789/0 conn(0x7fe9fc0722c0 legacy=0x7fe9fc06e510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:00:32.893 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:32.887+0000 7fe9f959a700 1 -- 172.21.15.59:0/921780868 --> v1:172.21.15.59:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe9fc0b3dc0 con 0x7fe9fc1440f0 2023-11-24T08:00:32.894 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:32.887+0000 7fe9f959a700 1 -- 172.21.15.59:0/921780868 <== mon.0 v1:172.21.15.59:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1312333566 0 0) 0x7fe9ec001670 con 0x7fe9fc1440f0 2023-11-24T08:00:32.894 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:32.887+0000 7fe9f959a700 1 -- 172.21.15.59:0/921780868 >> v1:172.21.15.59:6789/0 conn(0x7fe9fc1440f0 legacy=0x7fe9fc071bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:00:32.894 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:32.887+0000 7fe9f959a700 1 --2- 172.21.15.59:0/921780868 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe9e8003a30 0x7fe9e8003e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:00:32.894 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:32.887+0000 7fe9f959a700 1 --2- 172.21.15.59:0/921780868 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe9e8004850 0x7fe9e8004cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:00:32.894 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:32.887+0000 7fe9f959a700 1 --2- 172.21.15.59:0/921780868 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fe9fc143ae0 0x7fe9e8005210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:00:32.894 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:32.887+0000 7fe9f959a700 1 -- 172.21.15.59:0/921780868 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9fc0b40a0 con 0x7fe9e8004850 2023-11-24T08:00:32.894 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:32.887+0000 7fe9f959a700 1 -- 172.21.15.59:0/921780868 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9e800bac0 con 0x7fe9e8003a30 2023-11-24T08:00:32.895 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:32.887+0000 7fe9f959a700 1 -- 172.21.15.59:0/921780868 --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe9e800bc80 con 0x7fe9fc143ae0 2023-11-24T08:00:32.895 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:32.887+0000 7fe9f9d9b700 1 --2- 172.21.15.59:0/921780868 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe9e8004850 0x7fe9e8004cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:00:32.895 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:32.887+0000 7fe9fa59c700 1 --2- 172.21.15.59:0/921780868 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe9e8003a30 0x7fe9e8003e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:00:32.895 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:32.887+0000 7fe9fad9d700 1 --2- 172.21.15.59:0/921780868 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fe9fc143ae0 0x7fe9e8005210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:00:32.895 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:32.888+0000 7fe9f9d9b700 1 -- 172.21.15.59:0/921780868 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fe9fc143ae0 msgr2=0x7fe9e8005210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:00:32.895 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:32.888+0000 7fe9f9d9b700 1 --2- 172.21.15.59:0/921780868 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fe9fc143ae0 0x7fe9e8005210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:00:32.895 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:32.888+0000 7fe9f9d9b700 1 -- 172.21.15.59:0/921780868 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe9e8003a30 msgr2=0x7fe9e8003e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:00:32.896 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:32.888+0000 7fe9f9d9b700 1 --2- 172.21.15.59:0/921780868 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe9e8003a30 0x7fe9e8003e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:00:32.896 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:32.888+0000 7fe9f9d9b700 1 -- 172.21.15.59:0/921780868 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe9e80035b0 con 0x7fe9e8004850 2023-11-24T08:00:32.896 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:32.888+0000 7fe9f9d9b700 1 --2- 172.21.15.59:0/921780868 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe9e8004850 0x7fe9e8004cd0 secure :-1 s=READY pgs=1179 cs=0 l=1 rev1=1 crypto rx=0x7fe9ec00c270 tx=0x7fe9e8005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8b4a30c23a67b87f server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:00:32.896 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:32.888+0000 7fe9f959a700 1 -- 172.21.15.59:0/921780868 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9ec007c10 con 0x7fe9e8004850 2023-11-24T08:00:32.896 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:32.888+0000 7fe9f959a700 1 -- 172.21.15.59:0/921780868 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe9ec00cdb0 con 0x7fe9e8004850 2023-11-24T08:00:32.896 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:32.888+0000 7fe9f959a700 1 -- 172.21.15.59:0/921780868 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9ec005ba0 con 0x7fe9e8004850 2023-11-24T08:00:32.896 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:32.888+0000 7fea0082d700 1 -- 172.21.15.59:0/921780868 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe9e8004850 msgr2=0x7fe9e8004cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:00:32.896 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:32.888+0000 7fea0082d700 1 --2- 172.21.15.59:0/921780868 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe9e8004850 0x7fe9e8004cd0 secure :-1 s=READY pgs=1179 cs=0 l=1 rev1=1 crypto rx=0x7fe9ec00c270 tx=0x7fe9e8005880 comp rx=0 tx=0).stop 2023-11-24T08:00:32.897 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:32.888+0000 7fe9fad9d700 1 -- 172.21.15.59:0/921780868 reap_dead start 2023-11-24T08:00:32.897 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:32.888+0000 7fea0082d700 1 -- 172.21.15.59:0/921780868 shutdown_connections 2023-11-24T08:00:32.897 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:32.888+0000 7fea0082d700 1 -- 172.21.15.59:0/921780868 >> 172.21.15.59:0/921780868 conn(0x7fe9fc0b2740 msgr2=0x7fe9fc0b2b20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:00:32.897 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:32.889+0000 7fea0082d700 1 -- 172.21.15.59:0/921780868 shutdown_connections 2023-11-24T08:00:32.897 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:32.889+0000 7fea0082d700 1 -- 172.21.15.59:0/921780868 wait complete. 2023-11-24T08:00:32.897 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:32.890+0000 7fea0082d700 1 Processor -- start 2023-11-24T08:00:32.897 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:32.890+0000 7fea0082d700 1 -- start start 2023-11-24T08:00:32.900 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:32.891+0000 7fea0082d700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe9e8004850 0x7fe9fc1f3c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:00:32.900 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:32.891+0000 7fea0082d700 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fe9fc0ca230 0x7fe9fc0cbbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:00:32.901 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:32.891+0000 7fea0082d700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe9fc0ce4f0 0x7fe9fc0ce970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:00:32.901 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:32.891+0000 7fea0082d700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9fc0ceeb0 con 0x7fe9e8004850 2023-11-24T08:00:32.901 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:32.891+0000 7fea0082d700 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9fc06f790 con 0x7fe9fc0ce4f0 2023-11-24T08:00:32.901 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:32.891+0000 7fea0082d700 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe9fc154cb0 con 0x7fe9fc0ca230 2023-11-24T08:00:32.901 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:32.891+0000 7fe9fa59c700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe9e8004850 0x7fe9fc1f3c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:00:32.902 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:32.891+0000 7fe9fa59c700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe9e8004850 0x7fe9fc1f3c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:56836/0 (socket says 172.21.15.59:56836) 2023-11-24T08:00:32.902 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:32.891+0000 7fe9fa59c700 1 -- 172.21.15.59:0/1007427830 learned_addr learned my addr 172.21.15.59:0/1007427830 (peer_addr_for_me v2:172.21.15.59:0/0) 2023-11-24T08:00:32.903 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:32.891+0000 7fe9f9d9b700 1 --2- 172.21.15.59:0/1007427830 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fe9fc0ca230 0x7fe9fc0cbbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:00:32.903 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:32.891+0000 7fe9fad9d700 1 --2- 172.21.15.59:0/1007427830 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe9fc0ce4f0 0x7fe9fc0ce970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:00:32.903 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:32.891+0000 7fe9fa59c700 1 -- 172.21.15.59:0/1007427830 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fe9fc0ca230 msgr2=0x7fe9fc0cbbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:00:32.903 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:32.891+0000 7fe9fa59c700 1 --2- 172.21.15.59:0/1007427830 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fe9fc0ca230 0x7fe9fc0cbbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:00:32.903 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:32.891+0000 7fe9fa59c700 1 -- 172.21.15.59:0/1007427830 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe9fc0ce4f0 msgr2=0x7fe9fc0ce970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:00:32.903 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:32.891+0000 7fe9fa59c700 1 --2- 172.21.15.59:0/1007427830 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe9fc0ce4f0 0x7fe9fc0ce970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:00:32.903 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:32.891+0000 7fe9fa59c700 1 -- 172.21.15.59:0/1007427830 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe9fc0cc100 con 0x7fe9e8004850 2023-11-24T08:00:32.904 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:32.891+0000 7fe9fa59c700 1 --2- 172.21.15.59:0/1007427830 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe9e8004850 0x7fe9fc1f3c40 secure :-1 s=READY pgs=1180 cs=0 l=1 rev1=1 crypto rx=0x7fe9e4011ad0 tx=0x7fe9e4011e90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:00:32.904 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:32.891+0000 7fe9f2ffd700 1 -- 172.21.15.59:0/1007427830 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9e400f9a0 con 0x7fe9e8004850 2023-11-24T08:00:32.904 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:32.892+0000 7fe9f2ffd700 1 -- 172.21.15.59:0/1007427830 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe9e400fb00 con 0x7fe9e8004850 2023-11-24T08:00:32.904 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:32.892+0000 7fea0082d700 1 -- 172.21.15.59:0/1007427830 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe9fc0cc3c0 con 0x7fe9e8004850 2023-11-24T08:00:32.904 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:32.892+0000 7fe9f2ffd700 1 -- 172.21.15.59:0/1007427830 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9e400feb0 con 0x7fe9e8004850 2023-11-24T08:00:32.904 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:32.892+0000 7fea0082d700 1 -- 172.21.15.59:0/1007427830 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe9fc154f00 con 0x7fe9e8004850 2023-11-24T08:00:32.904 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:32.892+0000 7fea0082d700 1 -- 172.21.15.59:0/1007427830 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe9fc068670 con 0x7fe9e8004850 2023-11-24T08:00:32.905 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:32.893+0000 7fe9f2ffd700 1 -- 172.21.15.59:0/1007427830 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 12) v1 ==== 89792+0+0 (secure 0 0 0) 0x7fe9e4014440 con 0x7fe9e8004850 2023-11-24T08:00:32.905 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:32.894+0000 7fe9f2ffd700 1 --2- 172.21.15.59:0/1007427830 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7fe9dc06b500 0x7fe9dc06d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:00:32.905 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:32.894+0000 7fe9f2ffd700 1 -- 172.21.15.59:0/1007427830 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 4943+0+0 (secure 0 0 0) 0x7fe9e408a580 con 0x7fe9e8004850 2023-11-24T08:00:32.905 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:32.894+0000 7fe9f9d9b700 1 --2- 172.21.15.59:0/1007427830 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7fe9dc06b500 0x7fe9dc06d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:00:32.905 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:32.894+0000 7fe9f9d9b700 1 --2- 172.21.15.59:0/1007427830 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7fe9dc06b500 0x7fe9dc06d9c0 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7fe9ec001930 tx=0x7fe9ec007000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:00:32.905 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:32.898+0000 7fe9f2ffd700 1 -- 172.21.15.59:0/1007427830 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fe9e4055070 con 0x7fe9e8004850 2023-11-24T08:00:33.100 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:33.098+0000 7fea0082d700 1 -- 172.21.15.59:0/1007427830 --> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "7", "target": ["mon-mgr", ""]}) v1 -- 0x7fe9fc144860 con 0x7fe9dc06b500 2023-11-24T08:00:33.100 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:33.099+0000 7fe9f2ffd700 1 -- 172.21.15.59:0/1007427830 <== mgr.4100 v2:172.21.15.76:6832/4264046536 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 7 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7fe9ec007b30 con 0x7fe9dc06b500 2023-11-24T08:00:33.104 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:33.101+0000 7fea0082d700 1 -- 172.21.15.59:0/1007427830 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7fe9dc06b500 msgr2=0x7fe9dc06d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:00:33.104 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:33.101+0000 7fea0082d700 1 --2- 172.21.15.59:0/1007427830 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7fe9dc06b500 0x7fe9dc06d9c0 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7fe9ec001930 tx=0x7fe9ec007000 comp rx=0 tx=0).stop 2023-11-24T08:00:33.104 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:33.101+0000 7fea0082d700 1 -- 172.21.15.59:0/1007427830 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe9e8004850 msgr2=0x7fe9fc1f3c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:00:33.104 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:33.101+0000 7fea0082d700 1 --2- 172.21.15.59:0/1007427830 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe9e8004850 0x7fe9fc1f3c40 secure :-1 s=READY pgs=1180 cs=0 l=1 rev1=1 crypto rx=0x7fe9e4011ad0 tx=0x7fe9e4011e90 comp rx=0 tx=0).stop 2023-11-24T08:00:33.104 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:33.101+0000 7fea0082d700 1 -- 172.21.15.59:0/1007427830 shutdown_connections 2023-11-24T08:00:33.105 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:33.101+0000 7fea0082d700 1 --2- 172.21.15.59:0/1007427830 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7fe9dc06b500 0x7fe9dc06d9c0 unknown :-1 s=CLOSED pgs=535 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:00:33.105 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:33.101+0000 7fea0082d700 1 --2- 172.21.15.59:0/1007427830 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe9fc0ce4f0 0x7fe9fc0ce970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:00:33.105 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:33.101+0000 7fea0082d700 1 --2- 172.21.15.59:0/1007427830 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe9e8004850 0x7fe9fc1f3c40 unknown :-1 s=CLOSED pgs=1180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:00:33.105 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:33.101+0000 7fea0082d700 1 --2- 172.21.15.59:0/1007427830 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fe9fc0ca230 0x7fe9fc0cbbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:00:33.105 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:33.101+0000 7fea0082d700 1 -- 172.21.15.59:0/1007427830 >> 172.21.15.59:0/1007427830 conn(0x7fe9fc0b2740 msgr2=0x7fe9fc0cfa80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:00:33.105 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:33.101+0000 7fea0082d700 1 -- 172.21.15.59:0/1007427830 shutdown_connections 2023-11-24T08:00:33.106 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:33.101+0000 7fea0082d700 1 -- 172.21.15.59:0/1007427830 wait complete. 2023-11-24T08:00:33.106 INFO:teuthology.orchestra.run.smithi059.stderr:instructed osd(s) 7 to deep-scrub 2023-11-24T08:00:33.115 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-11-24T08:00:33.240 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:33.238+0000 7fa1b2e33700 1 Processor -- start 2023-11-24T08:00:33.240 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:33.238+0000 7fa1b2e33700 1 -- start start 2023-11-24T08:00:33.249 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:33.239+0000 7fa1b2e33700 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fa1ac06b620 0x7fa1ac0b3a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:00:33.249 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:33.239+0000 7fa1b2e33700 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa1ac1629c0 con 0x7fa1ac0b5ab0 2023-11-24T08:00:33.249 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:33.239+0000 7fa1b2e33700 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa1ac071f50 con 0x7fa1ac06ee60 2023-11-24T08:00:33.249 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:33.239+0000 7fa1b2e33700 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa1ac072230 con 0x7fa1ac06b620 2023-11-24T08:00:33.250 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:33.239+0000 7fa1abfff700 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7fa1ac0b5ab0 0x7fa1ac161ba0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:60320/0 (socket says 172.21.15.59:60320) 2023-11-24T08:00:33.250 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:33.239+0000 7fa1abfff700 1 -- 172.21.15.59:0/1827772203 learned_addr learned my addr 172.21.15.59:0/1827772203 (peer_addr_for_me v1:172.21.15.59:0/0) 2023-11-24T08:00:33.250 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:33.239+0000 7fa1ab7fe700 1 -- 172.21.15.59:0/1827772203 <== mon.0 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1422150888 0 0) 0x7fa1ac1629c0 con 0x7fa1ac0b5ab0 2023-11-24T08:00:33.251 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:33.239+0000 7fa1b0bcf700 1 --2- 172.21.15.59:0/1827772203 >> v2:172.21.15.76:3300/0 conn(0x7fa1ac06b620 0x7fa1ac0b3a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:00:33.251 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:33.239+0000 7fa1ab7fe700 1 -- 172.21.15.59:0/1827772203 --> v1:172.21.15.59:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa19c0035b0 con 0x7fa1ac0b5ab0 2023-11-24T08:00:33.251 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:33.239+0000 7fa1ab7fe700 1 -- 172.21.15.59:0/1827772203 <== mon.0 v1:172.21.15.59:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2915227865 0 0) 0x7fa19c0035b0 con 0x7fa1ac0b5ab0 2023-11-24T08:00:33.251 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:33.239+0000 7fa1ab7fe700 1 -- 172.21.15.59:0/1827772203 >> v2:172.21.15.76:3300/0 conn(0x7fa1ac06b620 msgr2=0x7fa1ac0b3a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:00:33.251 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:33.239+0000 7fa1ab7fe700 1 --2- 172.21.15.59:0/1827772203 >> v2:172.21.15.76:3300/0 conn(0x7fa1ac06b620 0x7fa1ac0b3a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:00:33.252 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:33.239+0000 7fa1ab7fe700 1 -- 172.21.15.59:0/1827772203 >> v1:172.21.15.76:6789/0 conn(0x7fa1ac06ee60 legacy=0x7fa1ac1622b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:00:33.252 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:33.239+0000 7fa1ab7fe700 1 -- 172.21.15.59:0/1827772203 --> v1:172.21.15.59:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa1ac072510 con 0x7fa1ac0b5ab0 2023-11-24T08:00:33.252 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:33.239+0000 7fa1ab7fe700 1 -- 172.21.15.59:0/1827772203 <== mon.0 v1:172.21.15.59:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1312333566 0 0) 0x7fa198001670 con 0x7fa1ac0b5ab0 2023-11-24T08:00:33.252 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:33.239+0000 7fa1ab7fe700 1 -- 172.21.15.59:0/1827772203 >> v1:172.21.15.59:6789/0 conn(0x7fa1ac0b5ab0 legacy=0x7fa1ac161ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:00:33.252 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:33.239+0000 7fa1ab7fe700 1 --2- 172.21.15.59:0/1827772203 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa19c003a50 0x7fa19c003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:00:33.253 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:33.239+0000 7fa1ab7fe700 1 --2- 172.21.15.59:0/1827772203 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa19c004850 0x7fa19c004cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:00:33.253 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:33.239+0000 7fa1ab7fe700 1 --2- 172.21.15.59:0/1827772203 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fa19c005210 0x7fa19c00ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:00:33.253 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:33.240+0000 7fa1ab7fe700 1 -- 172.21.15.59:0/1827772203 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1ac072230 con 0x7fa19c004850 2023-11-24T08:00:33.253 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:33.240+0000 7fa1ab7fe700 1 -- 172.21.15.59:0/1827772203 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa19c00c110 con 0x7fa19c003a50 2023-11-24T08:00:33.253 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:33.240+0000 7fa1ab7fe700 1 -- 172.21.15.59:0/1827772203 --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa19c00c2d0 con 0x7fa19c005210 2023-11-24T08:00:33.253 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:33.240+0000 7fa1abfff700 1 --2- 172.21.15.59:0/1827772203 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa19c004850 0x7fa19c004cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:00:33.253 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:33.240+0000 7fa1b0bcf700 1 --2- 172.21.15.59:0/1827772203 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa19c003a50 0x7fa19c003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:00:33.254 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:33.240+0000 7fa1b13d0700 1 --2- 172.21.15.59:0/1827772203 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fa19c005210 0x7fa19c00ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:00:33.254 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:33.240+0000 7fa1abfff700 1 -- 172.21.15.59:0/1827772203 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fa19c005210 msgr2=0x7fa19c00ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:00:33.254 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:33.240+0000 7fa1abfff700 1 --2- 172.21.15.59:0/1827772203 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fa19c005210 0x7fa19c00ba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:00:33.254 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:33.240+0000 7fa1abfff700 1 -- 172.21.15.59:0/1827772203 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa19c003a50 msgr2=0x7fa19c003e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:00:33.254 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:33.240+0000 7fa1abfff700 1 --2- 172.21.15.59:0/1827772203 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa19c003a50 0x7fa19c003e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:00:33.254 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:33.240+0000 7fa1abfff700 1 -- 172.21.15.59:0/1827772203 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa19c0035b0 con 0x7fa19c004850 2023-11-24T08:00:33.254 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:33.240+0000 7fa1b13d0700 1 -- 172.21.15.59:0/1827772203 reap_dead start 2023-11-24T08:00:33.255 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:33.240+0000 7fa1abfff700 1 --2- 172.21.15.59:0/1827772203 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa19c004850 0x7fa19c004cd0 secure :-1 s=READY pgs=1182 cs=0 l=1 rev1=1 crypto rx=0x7fa19800c160 tx=0x7fa19c005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f22eb0bde5f41b7a server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:00:33.255 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:33.240+0000 7fa1ab7fe700 1 -- 172.21.15.59:0/1827772203 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa198007b30 con 0x7fa19c004850 2023-11-24T08:00:33.255 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:33.240+0000 7fa1ab7fe700 1 -- 172.21.15.59:0/1827772203 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa19800ccc0 con 0x7fa19c004850 2023-11-24T08:00:33.255 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:33.240+0000 7fa1ab7fe700 1 -- 172.21.15.59:0/1827772203 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa198005b80 con 0x7fa19c004850 2023-11-24T08:00:33.255 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:33.241+0000 7fa1b2e33700 1 -- 172.21.15.59:0/1827772203 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa19c004850 msgr2=0x7fa19c004cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:00:33.255 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:33.241+0000 7fa1b2e33700 1 --2- 172.21.15.59:0/1827772203 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa19c004850 0x7fa19c004cd0 secure :-1 s=READY pgs=1182 cs=0 l=1 rev1=1 crypto rx=0x7fa19800c160 tx=0x7fa19c005880 comp rx=0 tx=0).stop 2023-11-24T08:00:33.255 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:33.241+0000 7fa1b2e33700 1 -- 172.21.15.59:0/1827772203 shutdown_connections 2023-11-24T08:00:33.255 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:33.241+0000 7fa1b2e33700 1 --2- 172.21.15.59:0/1827772203 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa19c004850 0x7fa19c004cd0 unknown :-1 s=CLOSED pgs=1182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:00:33.256 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:33.241+0000 7fa1b2e33700 1 -- 172.21.15.59:0/1827772203 >> 172.21.15.59:0/1827772203 conn(0x7fa1ac0b2740 msgr2=0x7fa1ac0b2b20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:00:33.256 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:33.241+0000 7fa1b2e33700 1 -- 172.21.15.59:0/1827772203 shutdown_connections 2023-11-24T08:00:33.256 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:33.241+0000 7fa1b2e33700 1 -- 172.21.15.59:0/1827772203 wait complete. 2023-11-24T08:00:33.256 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:33.242+0000 7fa1b2e33700 1 Processor -- start 2023-11-24T08:00:33.256 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:33.242+0000 7fa1b2e33700 1 -- start start 2023-11-24T08:00:33.256 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:33.243+0000 7fa1b2e33700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa19c004850 0x7fa1ac13f5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:00:33.256 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:33.243+0000 7fa1b2e33700 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fa1ac0ce430 0x7fa1ac15b4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:00:33.257 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:33.243+0000 7fa1b2e33700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa1ac0ce8b0 0x7fa1ac15ba00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:00:33.257 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:33.243+0000 7fa1b2e33700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1ac13e280 con 0x7fa1ac0ce8b0 2023-11-24T08:00:33.257 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:33.243+0000 7fa1b2e33700 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1ac13eae0 con 0x7fa19c004850 2023-11-24T08:00:33.257 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:33.243+0000 7fa1b2e33700 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa1ac13edc0 con 0x7fa1ac0ce430 2023-11-24T08:00:33.257 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:33.243+0000 7fa1b13d0700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa1ac0ce8b0 0x7fa1ac15ba00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:00:33.257 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:33.243+0000 7fa1b13d0700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa1ac0ce8b0 0x7fa1ac15ba00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:56860/0 (socket says 172.21.15.59:56860) 2023-11-24T08:00:33.258 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:33.243+0000 7fa1b13d0700 1 -- 172.21.15.59:0/1698417892 learned_addr learned my addr 172.21.15.59:0/1698417892 (peer_addr_for_me v2:172.21.15.59:0/0) 2023-11-24T08:00:33.258 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:33.243+0000 7fa1b0bcf700 1 --2- 172.21.15.59:0/1698417892 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa19c004850 0x7fa1ac13f5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:00:33.258 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:33.243+0000 7fa1abfff700 1 --2- 172.21.15.59:0/1698417892 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fa1ac0ce430 0x7fa1ac15b4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:00:33.258 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:33.243+0000 7fa1b13d0700 1 -- 172.21.15.59:0/1698417892 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fa1ac0ce430 msgr2=0x7fa1ac15b4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:00:33.258 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:33.243+0000 7fa1b13d0700 1 --2- 172.21.15.59:0/1698417892 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fa1ac0ce430 0x7fa1ac15b4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:00:33.258 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:33.243+0000 7fa1b13d0700 1 -- 172.21.15.59:0/1698417892 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa19c004850 msgr2=0x7fa1ac13f5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:00:33.258 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:33.243+0000 7fa1b13d0700 1 --2- 172.21.15.59:0/1698417892 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa19c004850 0x7fa1ac13f5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:00:33.258 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:33.243+0000 7fa1b13d0700 1 -- 172.21.15.59:0/1698417892 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa1ac13f0a0 con 0x7fa1ac0ce8b0 2023-11-24T08:00:33.259 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:33.244+0000 7fa1b13d0700 1 --2- 172.21.15.59:0/1698417892 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa1ac0ce8b0 0x7fa1ac15ba00 secure :-1 s=READY pgs=1183 cs=0 l=1 rev1=1 crypto rx=0x7fa1ac073020 tx=0x7fa1a400d150 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:00:33.259 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:33.244+0000 7fa1a97fa700 1 -- 172.21.15.59:0/1698417892 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1a4003c10 con 0x7fa1ac0ce8b0 2023-11-24T08:00:33.259 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:33.244+0000 7fa1a97fa700 1 -- 172.21.15.59:0/1698417892 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa1a4003d90 con 0x7fa1ac0ce8b0 2023-11-24T08:00:33.259 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:33.244+0000 7fa1b2e33700 1 -- 172.21.15.59:0/1698417892 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa1ac13fb00 con 0x7fa1ac0ce8b0 2023-11-24T08:00:33.259 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:33.244+0000 7fa1a97fa700 1 -- 172.21.15.59:0/1698417892 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1a4005310 con 0x7fa1ac0ce8b0 2023-11-24T08:00:33.260 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:33.244+0000 7fa1b2e33700 1 -- 172.21.15.59:0/1698417892 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa1ac13d620 con 0x7fa1ac0ce8b0 2023-11-24T08:00:33.260 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:33.245+0000 7fa1b2e33700 1 -- 172.21.15.59:0/1698417892 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa1ac161e80 con 0x7fa1ac0ce8b0 2023-11-24T08:00:33.260 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:33.246+0000 7fa1a97fa700 1 -- 172.21.15.59:0/1698417892 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 12) v1 ==== 89792+0+0 (secure 0 0 0) 0x7fa1a4005470 con 0x7fa1ac0ce8b0 2023-11-24T08:00:33.260 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:33.246+0000 7fa1a97fa700 1 --2- 172.21.15.59:0/1698417892 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7fa19006b500 0x7fa19006d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:00:33.260 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:33.246+0000 7fa1a97fa700 1 -- 172.21.15.59:0/1698417892 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 4943+0+0 (secure 0 0 0) 0x7fa1a408a7e0 con 0x7fa1ac0ce8b0 2023-11-24T08:00:33.260 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:33.246+0000 7fa1b0bcf700 1 --2- 172.21.15.59:0/1698417892 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7fa19006b500 0x7fa19006d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:00:33.261 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:33.247+0000 7fa1b0bcf700 1 --2- 172.21.15.59:0/1698417892 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7fa19006b500 0x7fa19006d9c0 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7fa1ac069c90 tx=0x7fa1a0009210 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:00:33.261 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:33.250+0000 7fa1a97fa700 1 -- 172.21.15.59:0/1698417892 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fa1a4055350 con 0x7fa1ac0ce8b0 2023-11-24T08:00:33.449 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:33.447+0000 7fa1b2e33700 1 -- 172.21.15.59:0/1698417892 --> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fa1ac072ad0 con 0x7fa19006b500 2023-11-24T08:00:33.451 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:33.450+0000 7fa1a97fa700 1 -- 172.21.15.59:0/1698417892 <== mgr.4100 v2:172.21.15.76:6832/4264046536 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+50763 (secure 0 0 0) 0x7fa1a0019070 con 0x7fa19006b500 2023-11-24T08:00:33.452 INFO:teuthology.orchestra.run.smithi059.stdout: 2023-11-24T08:00:33.454 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:33.452+0000 7fa1b2e33700 1 -- 172.21.15.59:0/1698417892 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7fa19006b500 msgr2=0x7fa19006d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:00:33.454 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:33.452+0000 7fa1b2e33700 1 --2- 172.21.15.59:0/1698417892 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7fa19006b500 0x7fa19006d9c0 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7fa1ac069c90 tx=0x7fa1a0009210 comp rx=0 tx=0).stop 2023-11-24T08:00:33.454 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:33.452+0000 7fa1b2e33700 1 -- 172.21.15.59:0/1698417892 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa1ac0ce8b0 msgr2=0x7fa1ac15ba00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:00:33.454 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:33.452+0000 7fa1b2e33700 1 --2- 172.21.15.59:0/1698417892 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa1ac0ce8b0 0x7fa1ac15ba00 secure :-1 s=READY pgs=1183 cs=0 l=1 rev1=1 crypto rx=0x7fa1ac073020 tx=0x7fa1a400d150 comp rx=0 tx=0).stop 2023-11-24T08:00:33.454 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:33.452+0000 7fa1b2e33700 1 -- 172.21.15.59:0/1698417892 shutdown_connections 2023-11-24T08:00:33.455 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:33.452+0000 7fa1b2e33700 1 --2- 172.21.15.59:0/1698417892 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7fa19006b500 0x7fa19006d9c0 unknown :-1 s=CLOSED pgs=536 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:00:33.455 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:33.452+0000 7fa1b2e33700 1 --2- 172.21.15.59:0/1698417892 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa1ac0ce8b0 0x7fa1ac15ba00 unknown :-1 s=CLOSED pgs=1183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:00:33.455 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:33.452+0000 7fa1b2e33700 1 --2- 172.21.15.59:0/1698417892 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa19c004850 0x7fa1ac13f5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:00:33.455 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:33.452+0000 7fa1b2e33700 1 --2- 172.21.15.59:0/1698417892 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fa1ac0ce430 0x7fa1ac15b4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:00:33.455 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:33.452+0000 7fa1b2e33700 1 -- 172.21.15.59:0/1698417892 >> 172.21.15.59:0/1698417892 conn(0x7fa1ac0b2740 msgr2=0x7fa1ac0afe70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:00:33.455 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:33.452+0000 7fa1b2e33700 1 -- 172.21.15.59:0/1698417892 shutdown_connections 2023-11-24T08:00:33.455 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:33.452+0000 7fa1b2e33700 1 -- 172.21.15.59:0/1698417892 wait complete. 2023-11-24T08:00:33.456 INFO:teuthology.orchestra.run.smithi059.stderr:dumped all 2023-11-24T08:00:33.465 INFO:teuthology.orchestra.run.smithi059.stdout:{"pg_ready":true,"pg_map":{"version":210,"stamp":"2023-11-24T08:00:32.356793+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":40,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":2,"kb":754974720,"kb_used":230196,"kb_used_data":9864,"kb_used_omap":12,"kb_used_meta":220211,"kb_avail":754744524,"statfs":{"total":773094113280,"available":772858392576,"internally_reserved":0,"allocated":10100736,"data_stored":4994848,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12745,"internal_metadata":225496631},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":18,"apply_latency_ms":18,"commit_latency_ns":18000000,"apply_latency_ns":18000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-134,"num_objects":-1,"num_object_clones":0,"num_object_copies":-2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":-1,"num_write_kb":-1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.944590"},"pg_stats":[{"pgid":"1.0","version":"16'117","reported_seq":226,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-11-24T08:00:24.016774+0000","last_change":"2023-11-24T07:56:19.668905+0000","last_active":"2023-11-24T08:00:24.016774+0000","last_peered":"2023-11-24T08:00:24.016774+0000","last_clean":"2023-11-24T08:00:24.016774+0000","last_became_active":"2023-11-24T07:56:19.668601+0000","last_became_peered":"2023-11-24T07:56:19.668601+0000","last_unstale":"2023-11-24T08:00:24.016774+0000","last_undegraded":"2023-11-24T08:00:24.016774+0000","last_fullsized":"2023-11-24T08:00:24.016774+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T07:56:18.657844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T07:56:18.657844+0000","last_clean_scrub_stamp":"2023-11-24T07:56:18.657844+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T11:51:53.828455+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574901,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":28108,"kb_used_data":1088,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343732,"statfs":{"total":96636764160,"available":96607981568,"internally_reserved":0,"allocated":1114112,"data_stored":476764,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 08:00:21 2023","interfaces":[{"interface":"back","average":{"1min":0.66700000000000004,"5min":0.56200000000000006,"15min":0.55800000000000005},"min":{"1min":0.27700000000000002,"5min":0.27600000000000002,"15min":0.27600000000000002},"max":{"1min":3.1640000000000001,"5min":3.1640000000000001,"15min":3.1640000000000001},"last":0.43099999999999999},{"interface":"front","average":{"1min":0.51800000000000002,"5min":0.54400000000000004,"15min":0.55000000000000004},"min":{"1min":0.35499999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.67200000000000004,"5min":0.81000000000000005,"15min":0.81000000000000005},"last":0.51500000000000001}]},{"osd":1,"last update":"Fri Nov 24 08:00:21 2023","interfaces":[{"interface":"back","average":{"1min":0.50700000000000001,"5min":0.64400000000000002,"15min":0.78000000000000003},"min":{"1min":0.28799999999999998,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":0.83499999999999996,"5min":6.1070000000000002,"15min":6.1070000000000002},"last":0.45100000000000001},{"interface":"front","average":{"1min":0.68899999999999995,"5min":0.56899999999999995,"15min":0.57599999999999996},"min":{"1min":0.38400000000000001,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":3.1970000000000001,"5min":3.1970000000000001,"15min":3.1970000000000001},"last":0.35999999999999999}]},{"osd":2,"last update":"Fri Nov 24 08:00:21 2023","interfaces":[{"interface":"back","average":{"1min":0.53500000000000003,"5min":0.56200000000000006,"15min":0.60499999999999998},"min":{"1min":0.35599999999999998,"5min":0.308,"15min":0.308},"max":{"1min":0.94099999999999995,"5min":1.613,"15min":1.613},"last":0.60199999999999998},{"interface":"front","average":{"1min":0.51800000000000002,"5min":0.61699999999999999,"15min":0.73299999999999998},"min":{"1min":0.35199999999999998,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.65400000000000003,"5min":4.5220000000000002,"15min":4.5220000000000002},"last":0.48299999999999998}]},{"osd":3,"last update":"Fri Nov 24 08:00:21 2023","interfaces":[{"interface":"back","average":{"1min":0.56399999999999995,"5min":0.65800000000000003,"15min":0.75600000000000001},"min":{"1min":0.42599999999999999,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.88500000000000001,"5min":4.6760000000000002,"15min":4.6760000000000002},"last":0.53400000000000003},{"interface":"front","average":{"1min":0.64500000000000002,"5min":0.748,"15min":0.878},"min":{"1min":0.51200000000000001,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":1.1299999999999999,"5min":4.6539999999999999,"15min":4.6539999999999999},"last":0.55200000000000005}]},{"osd":4,"last update":"Fri Nov 24 08:00:21 2023","interfaces":[{"interface":"back","average":{"1min":0.53300000000000003,"5min":0.53800000000000003,"15min":0.55800000000000005},"min":{"1min":0.33600000000000002,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":1.0760000000000001,"5min":1.0760000000000001,"15min":1.0760000000000001},"last":0.57799999999999996},{"interface":"front","average":{"1min":0.59299999999999997,"5min":0.60499999999999998,"15min":0.64600000000000002},"min":{"1min":0.39900000000000002,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":1.284,"5min":1.629,"15min":1.629},"last":0.497}]},{"osd":5,"last update":"Fri Nov 24 08:00:21 2023","interfaces":[{"interface":"back","average":{"1min":0.51100000000000001,"5min":0.52900000000000003,"15min":0.54100000000000004},"min":{"1min":0.36399999999999999,"5min":0.315,"15min":0.315},"max":{"1min":0.77200000000000002,"5min":0.875,"15min":0.875},"last":0.40999999999999998},{"interface":"front","average":{"1min":0.626,"5min":0.60299999999999998,"15min":0.61599999999999999},"min":{"1min":0.47499999999999998,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":1.0429999999999999,"5min":1.0429999999999999,"15min":1.0429999999999999},"last":0.625}]},{"osd":6,"last update":"Fri Nov 24 08:00:21 2023","interfaces":[{"interface":"back","average":{"1min":0.55000000000000004,"5min":0.58399999999999996,"15min":0.60199999999999998},"min":{"1min":0.38600000000000001,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.745,"5min":0.85799999999999998,"15min":0.85799999999999998},"last":0.56899999999999995},{"interface":"front","average":{"1min":0.61199999999999999,"5min":0.64000000000000001,"15min":0.66000000000000003},"min":{"1min":0.438,"5min":0.438,"15min":0.438},"max":{"1min":0.91700000000000004,"5min":0.99099999999999999,"15min":0.99099999999999999},"last":0.66200000000000003}]}]},{"osd":6,"up_from":13,"seq":55834574902,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":28112,"kb_used_data":1088,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343728,"statfs":{"total":96636764160,"available":96607977472,"internally_reserved":0,"allocated":1114112,"data_stored":476764,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 08:00:24 2023","interfaces":[{"interface":"back","average":{"1min":0.48899999999999999,"5min":0.52600000000000002,"15min":0.54200000000000004},"min":{"1min":0.24099999999999999,"5min":0.24099999999999999,"15min":0.24099999999999999},"max":{"1min":0.65300000000000002,"5min":0.93100000000000005,"15min":0.93100000000000005},"last":0.51200000000000001},{"interface":"front","average":{"1min":0.52200000000000002,"5min":0.52000000000000002,"15min":0.51700000000000002},"min":{"1min":0.40000000000000002,"5min":0.311,"15min":0.311},"max":{"1min":0.73799999999999999,"5min":0.82199999999999995,"15min":0.82199999999999995},"last":0.48499999999999999}]},{"osd":1,"last update":"Fri Nov 24 08:00:24 2023","interfaces":[{"interface":"back","average":{"1min":0.52500000000000002,"5min":0.51300000000000001,"15min":0.51200000000000001},"min":{"1min":0.311,"5min":0.311,"15min":0.311},"max":{"1min":0.68700000000000006,"5min":0.81299999999999994,"15min":0.81299999999999994},"last":0.496},{"interface":"front","average":{"1min":0.58699999999999997,"5min":0.58199999999999996,"15min":0.60199999999999998},"min":{"1min":0.308,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":0.75,"5min":1.0069999999999999,"15min":1.0069999999999999},"last":0.317}]},{"osd":2,"last update":"Fri Nov 24 08:00:24 2023","interfaces":[{"interface":"back","average":{"1min":0.56399999999999995,"5min":0.57499999999999996,"15min":0.57499999999999996},"min":{"1min":0.32400000000000001,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.90500000000000003,"5min":0.90600000000000003,"15min":0.90600000000000003},"last":0.438},{"interface":"front","average":{"1min":0.56299999999999994,"5min":0.58199999999999996,"15min":0.58899999999999997},"min":{"1min":0.34699999999999998,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.878,"5min":0.878,"15min":0.878},"last":0.36899999999999999}]},{"osd":3,"last update":"Fri Nov 24 08:00:24 2023","interfaces":[{"interface":"back","average":{"1min":0.57099999999999995,"5min":0.56499999999999995,"15min":0.56699999999999995},"min":{"1min":0.32600000000000001,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.76900000000000002,"5min":0.876,"15min":0.876},"last":0.45800000000000002},{"interface":"front","average":{"1min":0.56200000000000006,"5min":0.56100000000000005,"15min":0.55700000000000005},"min":{"1min":0.42799999999999999,"5min":0.379,"15min":0.379},"max":{"1min":0.753,"5min":0.876,"15min":0.876},"last":0.39000000000000001}]},{"osd":4,"last update":"Fri Nov 24 08:00:24 2023","interfaces":[{"interface":"back","average":{"1min":0.49299999999999999,"5min":0.52200000000000002,"15min":0.51800000000000002},"min":{"1min":0.32900000000000001,"5min":0.25,"15min":0.25},"max":{"1min":0.70699999999999996,"5min":0.89400000000000002,"15min":0.89400000000000002},"last":0.47199999999999998},{"interface":"front","average":{"1min":0.59799999999999998,"5min":0.58399999999999996,"15min":0.59599999999999997},"min":{"1min":0.34000000000000002,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":0.78600000000000003,"5min":1.087,"15min":1.087},"last":0.40899999999999997}]},{"osd":5,"last update":"Fri Nov 24 08:00:24 2023","interfaces":[{"interface":"back","average":{"1min":0.52900000000000003,"5min":0.55100000000000005,"15min":0.53700000000000003},"min":{"1min":0.35599999999999998,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":0.753,"5min":0.97999999999999998,"15min":0.97999999999999998},"last":0.53400000000000003},{"interface":"front","average":{"1min":0.58299999999999996,"5min":0.60799999999999998,"15min":0.60999999999999999},"min":{"1min":0.42899999999999999,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.78800000000000003,"5min":1.0009999999999999,"15min":1.0009999999999999},"last":0.42199999999999999}]},{"osd":7,"last update":"Fri Nov 24 08:00:24 2023","interfaces":[{"interface":"back","average":{"1min":0.624,"5min":0.61499999999999999,"15min":0.61699999999999999},"min":{"1min":0.38800000000000001,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":0.81799999999999995,"5min":1.0880000000000001,"15min":1.0880000000000001},"last":0.52300000000000002},{"interface":"front","average":{"1min":0.65700000000000003,"5min":0.65500000000000003,"15min":0.66500000000000004},"min":{"1min":0.41699999999999998,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.81899999999999995,"5min":0.95599999999999996,"15min":0.95599999999999996},"last":0.56399999999999995}]}]},{"osd":1,"up_from":13,"seq":55834574903,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28688,"kb_used_data":1668,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343152,"statfs":{"total":96636764160,"available":96607387648,"internally_reserved":0,"allocated":1708032,"data_stored":1067132,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 08:00:24 2023","interfaces":[{"interface":"back","average":{"1min":0.33600000000000002,"5min":0.34899999999999998,"15min":0.35899999999999999},"min":{"1min":0.251,"5min":0.20200000000000001,"15min":0.20200000000000001},"max":{"1min":0.60499999999999998,"5min":0.70899999999999996,"15min":0.70899999999999996},"last":0.25600000000000001},{"interface":"front","average":{"1min":0.36699999999999999,"5min":0.378,"15min":0.39100000000000001},"min":{"1min":0.23300000000000001,"5min":0.214,"15min":0.214},"max":{"1min":0.55400000000000005,"5min":0.82499999999999996,"15min":0.82499999999999996},"last":0.29399999999999998}]},{"osd":2,"last update":"Fri Nov 24 08:00:24 2023","interfaces":[{"interface":"back","average":{"1min":0.34599999999999997,"5min":0.35399999999999998,"15min":0.36599999999999999},"min":{"1min":0.26500000000000001,"5min":0.22700000000000001,"15min":0.22700000000000001},"max":{"1min":0.47099999999999997,"5min":0.60999999999999999,"15min":0.60999999999999999},"last":0.33000000000000002},{"interface":"front","average":{"1min":0.40799999999999997,"5min":0.40400000000000003,"15min":0.40999999999999998},"min":{"1min":0.318,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":0.55600000000000005,"5min":0.63700000000000001,"15min":0.63700000000000001},"last":0.311}]},{"osd":3,"last update":"Fri Nov 24 08:00:24 2023","interfaces":[{"interface":"back","average":{"1min":0.41199999999999998,"5min":0.42599999999999999,"15min":0.438},"min":{"1min":0.28399999999999997,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":0.58299999999999996,"5min":0.66500000000000004,"15min":0.66500000000000004},"last":0.36199999999999999},{"interface":"front","average":{"1min":0.40899999999999997,"5min":0.41399999999999998,"15min":0.42299999999999999},"min":{"1min":0.29499999999999998,"5min":0.28499999999999998,"15min":0.28499999999999998},"max":{"1min":0.60899999999999999,"5min":0.76300000000000001,"15min":0.76300000000000001},"last":0.34899999999999998}]},{"osd":4,"last update":"Fri Nov 24 08:00:24 2023","interfaces":[{"interface":"back","average":{"1min":0.64200000000000002,"5min":0.60999999999999999,"15min":0.60999999999999999},"min":{"1min":0.42499999999999999,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":0.90800000000000003,"5min":1.5580000000000001,"15min":1.5580000000000001},"last":0.47099999999999997},{"interface":"front","average":{"1min":0.60599999999999998,"5min":0.61599999999999999,"15min":0.60899999999999999},"min":{"1min":0.39700000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.84999999999999998,"5min":0.84999999999999998,"15min":0.84999999999999998},"last":0.52900000000000003}]},{"osd":5,"last update":"Fri Nov 24 08:00:24 2023","interfaces":[{"interface":"back","average":{"1min":0.64300000000000002,"5min":0.66200000000000003,"15min":0.70399999999999996},"min":{"1min":0.44,"5min":0.37,"15min":0.37},"max":{"1min":0.98699999999999999,"5min":2.7269999999999999,"15min":2.7269999999999999},"last":0.44600000000000001},{"interface":"front","average":{"1min":0.629,"5min":0.63600000000000001,"15min":0.64200000000000002},"min":{"1min":0.40000000000000002,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.96199999999999997,"5min":1.6459999999999999,"15min":1.6459999999999999},"last":0.54200000000000004}]},{"osd":6,"last update":"Fri Nov 24 08:00:24 2023","interfaces":[{"interface":"back","average":{"1min":0.65200000000000002,"5min":0.64200000000000002,"15min":0.65200000000000002},"min":{"1min":0.40999999999999998,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.84599999999999997,"5min":0.98199999999999998,"15min":0.98199999999999998},"last":0.49099999999999999},{"interface":"front","average":{"1min":0.65400000000000003,"5min":0.628,"15min":0.62},"min":{"1min":0.48899999999999999,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":0.93600000000000005,"5min":0.93600000000000005,"15min":0.93600000000000005},"last":0.57699999999999996}]},{"osd":7,"last update":"Fri Nov 24 08:00:24 2023","interfaces":[{"interface":"back","average":{"1min":0.64800000000000002,"5min":0.65000000000000002,"15min":0.64100000000000001},"min":{"1min":0.495,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.91100000000000003,"5min":1.0649999999999999,"15min":1.0649999999999999},"last":0.502},{"interface":"front","average":{"1min":0.67700000000000005,"5min":0.64600000000000002,"15min":0.66200000000000003},"min":{"1min":0.45000000000000001,"5min":0.442,"15min":0.442},"max":{"1min":1.004,"5min":1.004,"15min":1.004},"last":0.56499999999999995}]}]},{"osd":0,"up_from":13,"seq":55834574903,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":28108,"kb_used_data":1088,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343732,"statfs":{"total":96636764160,"available":96607981568,"internally_reserved":0,"allocated":1114112,"data_stored":476764,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Nov 24 08:00:26 2023","interfaces":[{"interface":"back","average":{"1min":0.36499999999999999,"5min":0.50900000000000001,"15min":0.64500000000000002},"min":{"1min":0.22700000000000001,"5min":0.21299999999999999,"15min":0.21299999999999999},"max":{"1min":0.60499999999999998,"5min":6.3550000000000004,"15min":6.3550000000000004},"last":0.436},{"interface":"front","average":{"1min":0.41799999999999998,"5min":0.93500000000000005,"15min":1.464},"min":{"1min":0.318,"5min":0.26000000000000001,"15min":0.26000000000000001},"max":{"1min":0.58499999999999996,"5min":22.916,"15min":22.916},"last":0.41799999999999998}]},{"osd":2,"last update":"Fri Nov 24 08:00:26 2023","interfaces":[{"interface":"back","average":{"1min":0.379,"5min":0.498,"15min":0.63200000000000001},"min":{"1min":0.21299999999999999,"5min":0.21299999999999999,"15min":0.21299999999999999},"max":{"1min":0.57399999999999995,"5min":6.3899999999999997,"15min":6.3899999999999997},"last":0.39900000000000002},{"interface":"front","average":{"1min":0.34599999999999997,"5min":0.63800000000000001,"15min":0.92100000000000004},"min":{"1min":0.25,"5min":0.25,"15min":0.25},"max":{"1min":0.51200000000000001,"5min":11.93,"15min":11.93},"last":0.38}]},{"osd":3,"last update":"Fri Nov 24 08:00:26 2023","interfaces":[{"interface":"back","average":{"1min":0.35699999999999998,"5min":0.38800000000000001,"15min":0.40899999999999997},"min":{"1min":0.25,"5min":0.23799999999999999,"15min":0.23799999999999999},"max":{"1min":0.48799999999999999,"5min":1.155,"15min":1.155},"last":0.35499999999999998},{"interface":"front","average":{"1min":0.41499999999999998,"5min":0.70099999999999996,"15min":0.98499999999999999},"min":{"1min":0.29199999999999998,"5min":0.27400000000000002,"15min":0.27400000000000002},"max":{"1min":0.64800000000000002,"5min":12.116,"15min":12.116},"last":0.46200000000000002}]},{"osd":4,"last update":"Fri Nov 24 08:00:26 2023","interfaces":[{"interface":"back","average":{"1min":0.61299999999999999,"5min":0.626,"15min":0.64000000000000001},"min":{"1min":0.432,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.97399999999999998,"5min":1.044,"15min":1.044},"last":0.623},{"interface":"front","average":{"1min":0.57399999999999995,"5min":1.1339999999999999,"15min":1.667},"min":{"1min":0.42499999999999999,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.82999999999999996,"5min":22.963999999999999,"15min":22.963999999999999},"last":0.55500000000000005}]},{"osd":5,"last update":"Fri Nov 24 08:00:26 2023","interfaces":[{"interface":"back","average":{"1min":0.62,"5min":0.89100000000000001,"15min":1.159},"min":{"1min":0.501,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.91900000000000004,"5min":12.084,"15min":12.084},"last":0.58799999999999997},{"interface":"front","average":{"1min":0.60399999999999998,"5min":0.64700000000000002,"15min":0.66700000000000004},"min":{"1min":0.41299999999999998,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.80500000000000005,"5min":1.073,"15min":1.073},"last":0.57299999999999995}]},{"osd":6,"last update":"Fri Nov 24 08:00:26 2023","interfaces":[{"interface":"back","average":{"1min":0.63,"5min":0.65600000000000003,"15min":0.67900000000000005},"min":{"1min":0.45400000000000001,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.93999999999999995,"5min":1.0580000000000001,"15min":1.0580000000000001},"last":0.60099999999999998},{"interface":"front","average":{"1min":0.59799999999999998,"5min":0.89900000000000002,"15min":1.175},"min":{"1min":0.437,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":0.88100000000000001,"5min":12.045,"15min":12.045},"last":0.61499999999999999}]},{"osd":7,"last update":"Fri Nov 24 08:00:26 2023","interfaces":[{"interface":"back","average":{"1min":0.60599999999999998,"5min":0.63300000000000001,"15min":0.64800000000000002},"min":{"1min":0.48399999999999999,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":0.90000000000000002,"5min":1.038,"15min":1.038},"last":0.53900000000000003},{"interface":"front","average":{"1min":0.65100000000000002,"5min":1.169,"15min":1.698},"min":{"1min":0.45300000000000001,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":0.95799999999999996,"5min":22.995000000000001,"15min":22.995000000000001},"last":0.65600000000000003}]}]},{"osd":2,"up_from":13,"seq":55834574903,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":28108,"kb_used_data":1088,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343732,"statfs":{"total":96636764160,"available":96607981568,"internally_reserved":0,"allocated":1114112,"data_stored":476764,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 08:00:21 2023","interfaces":[{"interface":"back","average":{"1min":0.34499999999999997,"5min":0.35399999999999998,"15min":0.36499999999999999},"min":{"1min":0.22500000000000001,"5min":0.219,"15min":0.219},"max":{"1min":0.499,"5min":0.58099999999999996,"15min":0.58099999999999996},"last":0.437},{"interface":"front","average":{"1min":0.41299999999999998,"5min":0.39600000000000002,"15min":0.40000000000000002},"min":{"1min":0.28899999999999998,"5min":0.27400000000000002,"15min":0.27400000000000002},"max":{"1min":0.69099999999999995,"5min":0.69099999999999995,"15min":0.69099999999999995},"last":0.54500000000000004}]},{"osd":1,"last update":"Fri Nov 24 08:00:21 2023","interfaces":[{"interface":"back","average":{"1min":0.34999999999999998,"5min":0.35499999999999998,"15min":0.37},"min":{"1min":0.249,"5min":0.21099999999999999,"15min":0.21099999999999999},"max":{"1min":0.55700000000000005,"5min":0.59199999999999997,"15min":0.59199999999999997},"last":0.49099999999999999},{"interface":"front","average":{"1min":0.379,"5min":0.38800000000000001,"15min":0.40600000000000003},"min":{"1min":0.24399999999999999,"5min":0.24399999999999999,"15min":0.24399999999999999},"max":{"1min":0.55100000000000005,"5min":0.627,"15min":0.627},"last":0.755}]},{"osd":3,"last update":"Fri Nov 24 08:00:21 2023","interfaces":[{"interface":"back","average":{"1min":0.42499999999999999,"5min":0.40400000000000003,"15min":0.40799999999999997},"min":{"1min":0.28699999999999998,"5min":0.26400000000000001,"15min":0.26400000000000001},"max":{"1min":0.63500000000000001,"5min":0.64200000000000002,"15min":0.64200000000000002},"last":0.60699999999999998},{"interface":"front","average":{"1min":0.39000000000000001,"5min":0.40000000000000002,"15min":0.40899999999999997},"min":{"1min":0.315,"5min":0.27500000000000002,"15min":0.27500000000000002},"max":{"1min":0.58299999999999996,"5min":0.64600000000000002,"15min":0.64600000000000002},"last":0.53100000000000003}]},{"osd":4,"last update":"Fri Nov 24 08:00:21 2023","interfaces":[{"interface":"back","average":{"1min":0.58699999999999997,"5min":0.57099999999999995,"15min":0.59499999999999997},"min":{"1min":0.32900000000000001,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.91000000000000003,"5min":0.91000000000000003,"15min":0.91000000000000003},"last":0.56399999999999995},{"interface":"front","average":{"1min":0.57099999999999995,"5min":0.57699999999999996,"15min":0.59199999999999997},"min":{"1min":0.35299999999999998,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.75600000000000001,"5min":0.83799999999999997,"15min":0.83799999999999997},"last":0.68799999999999994}]},{"osd":5,"last update":"Fri Nov 24 08:00:21 2023","interfaces":[{"interface":"back","average":{"1min":0.57899999999999996,"5min":0.58999999999999997,"15min":0.60999999999999999},"min":{"1min":0.39100000000000001,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.84099999999999997,"5min":0.84899999999999998,"15min":0.84899999999999998},"last":0.58699999999999997},{"interface":"front","average":{"1min":0.64500000000000002,"5min":0.64600000000000002,"15min":0.69199999999999995},"min":{"1min":0.45100000000000001,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":1.119,"5min":2.1699999999999999,"15min":2.1699999999999999},"last":0.67100000000000004}]},{"osd":6,"last update":"Fri Nov 24 08:00:21 2023","interfaces":[{"interface":"back","average":{"1min":0.64000000000000001,"5min":0.625,"15min":0.65300000000000002},"min":{"1min":0.41999999999999998,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":0.88300000000000001,"5min":0.88300000000000001,"15min":0.88300000000000001},"last":0.72599999999999998},{"interface":"front","average":{"1min":0.65400000000000003,"5min":0.629,"15min":0.65700000000000003},"min":{"1min":0.44500000000000001,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":1.105,"5min":1.105,"15min":1.105},"last":0.74299999999999999}]},{"osd":7,"last update":"Fri Nov 24 08:00:21 2023","interfaces":[{"interface":"back","average":{"1min":0.628,"5min":0.626,"15min":0.65000000000000002},"min":{"1min":0.42199999999999999,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":0.93000000000000005,"5min":0.93000000000000005,"15min":0.93000000000000005},"last":0.78900000000000003},{"interface":"front","average":{"1min":0.627,"5min":0.63600000000000001,"15min":0.66500000000000004},"min":{"1min":0.46200000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.80000000000000004,"5min":0.90000000000000002,"15min":0.90000000000000002},"last":0.76700000000000002}]}]},{"osd":3,"up_from":13,"seq":55834574903,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":28112,"kb_used_data":1088,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343728,"statfs":{"total":96636764160,"available":96607977472,"internally_reserved":0,"allocated":1114112,"data_stored":476764,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 07:59:27 2023","interfaces":[{"interface":"back","average":{"1min":0.33600000000000002,"5min":0.34899999999999998,"15min":0.35299999999999998},"min":{"1min":0.183,"5min":0.183,"15min":0.183},"max":{"1min":0.53400000000000003,"5min":0.84799999999999998,"15min":0.84799999999999998},"last":0.65600000000000003},{"interface":"front","average":{"1min":0.34599999999999997,"5min":0.35399999999999998,"15min":0.36199999999999999},"min":{"1min":0.24299999999999999,"5min":0.23699999999999999,"15min":0.23699999999999999},"max":{"1min":0.51000000000000001,"5min":0.873,"15min":0.873},"last":0.58499999999999996}]},{"osd":1,"last update":"Fri Nov 24 07:59:27 2023","interfaces":[{"interface":"back","average":{"1min":0.38400000000000001,"5min":0.373,"15min":0.374},"min":{"1min":0.217,"5min":0.217,"15min":0.217},"max":{"1min":0.66300000000000003,"5min":0.92000000000000004,"15min":0.92000000000000004},"last":0.56200000000000006},{"interface":"front","average":{"1min":0.38,"5min":0.377,"15min":0.38300000000000001},"min":{"1min":0.28599999999999998,"5min":0.248,"15min":0.248},"max":{"1min":0.626,"5min":0.81699999999999995,"15min":0.81699999999999995},"last":0.51900000000000002}]},{"osd":2,"last update":"Fri Nov 24 07:59:27 2023","interfaces":[{"interface":"back","average":{"1min":0.35599999999999998,"5min":0.35399999999999998,"15min":0.35799999999999998},"min":{"1min":0.253,"5min":0.24399999999999999,"15min":0.24399999999999999},"max":{"1min":0.496,"5min":0.88500000000000001,"15min":0.88500000000000001},"last":0.54600000000000004},{"interface":"front","average":{"1min":0.40999999999999998,"5min":0.441,"15min":0.45900000000000002},"min":{"1min":0.315,"5min":0.311,"15min":0.311},"max":{"1min":0.59799999999999998,"5min":0.98099999999999998,"15min":0.98099999999999998},"last":0.60799999999999998}]},{"osd":4,"last update":"Fri Nov 24 07:59:27 2023","interfaces":[{"interface":"back","average":{"1min":0.56299999999999994,"5min":0.70499999999999996,"15min":0.76600000000000001},"min":{"1min":0.38900000000000001,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.73599999999999999,"5min":5.7350000000000003,"15min":5.7350000000000003},"last":0.67000000000000004},{"interface":"front","average":{"1min":0.55700000000000005,"5min":0.58099999999999996,"15min":0.58299999999999996},"min":{"1min":0.34399999999999997,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.80400000000000005,"5min":1.27,"15min":1.27},"last":0.625}]},{"osd":5,"last update":"Fri Nov 24 07:59:27 2023","interfaces":[{"interface":"back","average":{"1min":0.56100000000000005,"5min":0.57799999999999996,"15min":0.58399999999999996},"min":{"1min":0.42899999999999999,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":0.67900000000000005,"5min":1.4430000000000001,"15min":1.4430000000000001},"last":0.64600000000000002},{"interface":"front","average":{"1min":0.58799999999999997,"5min":0.61499999999999999,"15min":0.623},"min":{"1min":0.40999999999999998,"5min":0.377,"15min":0.377},"max":{"1min":0.78500000000000003,"5min":1.4239999999999999,"15min":1.4239999999999999},"last":0.71799999999999997}]},{"osd":6,"last update":"Fri Nov 24 07:59:27 2023","interfaces":[{"interface":"back","average":{"1min":0.58799999999999997,"5min":0.60699999999999998,"15min":0.61499999999999999},"min":{"1min":0.42599999999999999,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":0.84499999999999997,"5min":1.355,"15min":1.355},"last":0.63400000000000001},{"interface":"front","average":{"1min":0.57899999999999996,"5min":0.745,"15min":0.81200000000000006},"min":{"1min":0.44800000000000001,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.77100000000000002,"5min":5.6479999999999997,"15min":5.6479999999999997},"last":0.74099999999999999}]},{"osd":7,"last update":"Fri Nov 24 07:59:27 2023","interfaces":[{"interface":"back","average":{"1min":0.61799999999999999,"5min":0.64100000000000001,"15min":0.65700000000000003},"min":{"1min":0.499,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":0.82799999999999996,"5min":1.51,"15min":1.51},"last":0.70599999999999996},{"interface":"front","average":{"1min":0.57699999999999996,"5min":0.74199999999999999,"15min":0.80800000000000005},"min":{"1min":0.40100000000000002,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.79500000000000004,"5min":5.7069999999999999,"15min":5.7069999999999999},"last":0.82799999999999996}]}]},{"osd":4,"up_from":13,"seq":55834574902,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":28108,"kb_used_data":1088,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343732,"statfs":{"total":96636764160,"available":96607981568,"internally_reserved":0,"allocated":1114112,"data_stored":476764,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 07:59:28 2023","interfaces":[{"interface":"back","average":{"1min":0.55900000000000005,"5min":0.56100000000000005,"15min":0.57399999999999995},"min":{"1min":0.32400000000000001,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.93300000000000005,"5min":0.93300000000000005,"15min":0.93300000000000005},"last":0.63700000000000001},{"interface":"front","average":{"1min":0.51100000000000001,"5min":0.52800000000000002,"15min":0.53800000000000003},"min":{"1min":0.36199999999999999,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.69299999999999995,"5min":1.0249999999999999,"15min":1.0249999999999999},"last":0.72099999999999997}]},{"osd":1,"last update":"Fri Nov 24 07:59:28 2023","interfaces":[{"interface":"back","average":{"1min":0.54800000000000004,"5min":0.54600000000000004,"15min":0.55000000000000004},"min":{"1min":0.35899999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.71599999999999997,"5min":0.86399999999999999,"15min":0.86399999999999999},"last":0.68500000000000005},{"interface":"front","average":{"1min":0.51600000000000001,"5min":0.53700000000000003,"15min":0.54600000000000004},"min":{"1min":0.36699999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.72099999999999997,"5min":0.98699999999999999,"15min":0.98699999999999999},"last":0.70299999999999996}]},{"osd":2,"last update":"Fri Nov 24 07:59:28 2023","interfaces":[{"interface":"back","average":{"1min":0.54200000000000004,"5min":0.54900000000000004,"15min":0.55700000000000005},"min":{"1min":0.34499999999999997,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.86399999999999999,"5min":0.86399999999999999,"15min":0.86399999999999999},"last":0.59799999999999998},{"interface":"front","average":{"1min":0.56200000000000006,"5min":0.59399999999999997,"15min":0.60599999999999998},"min":{"1min":0.34000000000000002,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.81899999999999995,"5min":0.89200000000000002,"15min":0.89200000000000002},"last":0.80700000000000005}]},{"osd":3,"last update":"Fri Nov 24 07:59:28 2023","interfaces":[{"interface":"back","average":{"1min":0.53600000000000003,"5min":0.56799999999999995,"15min":0.57899999999999996},"min":{"1min":0.38800000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.78100000000000003,"5min":0.89700000000000002,"15min":0.89700000000000002},"last":0.73799999999999999},{"interface":"front","average":{"1min":0.58899999999999997,"5min":0.61099999999999999,"15min":0.624},"min":{"1min":0.42699999999999999,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.73999999999999999,"5min":0.96199999999999997,"15min":0.96199999999999997},"last":0.83899999999999997}]},{"osd":5,"last update":"Fri Nov 24 07:59:28 2023","interfaces":[{"interface":"back","average":{"1min":0.46800000000000003,"5min":0.46300000000000002,"15min":0.45700000000000002},"min":{"1min":0.34999999999999998,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":0.60499999999999998,"5min":0.70499999999999996,"15min":0.70499999999999996},"last":0.61899999999999999},{"interface":"front","average":{"1min":0.56599999999999995,"5min":0.60399999999999998,"15min":0.61399999999999999},"min":{"1min":0.41599999999999998,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.73699999999999999,"5min":0.86399999999999999,"15min":0.86399999999999999},"last":0.79100000000000004}]},{"osd":6,"last update":"Fri Nov 24 07:59:28 2023","interfaces":[{"interface":"back","average":{"1min":0.53900000000000003,"5min":0.57299999999999995,"15min":0.58099999999999996},"min":{"1min":0.33000000000000002,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.88600000000000001,"5min":0.88600000000000001,"15min":0.88600000000000001},"last":0.56100000000000005},{"interface":"front","average":{"1min":0.56899999999999995,"5min":0.59999999999999998,"15min":0.60899999999999999},"min":{"1min":0.32100000000000001,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.80300000000000005,"5min":1.095,"15min":1.095},"last":0.77200000000000002}]},{"osd":7,"last update":"Fri Nov 24 07:59:28 2023","interfaces":[{"interface":"back","average":{"1min":0.52000000000000002,"5min":0.56100000000000005,"15min":0.56899999999999995},"min":{"1min":0.36299999999999999,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.69999999999999996,"5min":0.89300000000000002,"15min":0.89300000000000002},"last":0.67000000000000004},{"interface":"front","average":{"1min":0.57799999999999996,"5min":0.63600000000000001,"15min":0.65400000000000003},"min":{"1min":0.39000000000000001,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.74399999999999999,"5min":0.91500000000000004,"15min":0.91500000000000004},"last":0.75600000000000001}]}]},{"osd":5,"up_from":13,"seq":55834574903,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32852,"kb_used_data":1668,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338988,"statfs":{"total":96636764160,"available":96603123712,"internally_reserved":0,"allocated":1708032,"data_stored":1067132,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 08:00:25 2023","interfaces":[{"interface":"back","average":{"1min":0.54700000000000004,"5min":0.54300000000000004,"15min":0.55500000000000005},"min":{"1min":0.29399999999999998,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":0.747,"5min":1.0109999999999999,"15min":1.0109999999999999},"last":0.72099999999999997},{"interface":"front","average":{"1min":0.54100000000000004,"5min":0.64100000000000001,"15min":0.747},"min":{"1min":0.35999999999999999,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.78300000000000003,"5min":5.431,"15min":5.431},"last":0.58199999999999996}]},{"osd":1,"last update":"Fri Nov 24 08:00:25 2023","interfaces":[{"interface":"back","average":{"1min":0.53900000000000003,"5min":0.53800000000000003,"15min":0.55200000000000005},"min":{"1min":0.32400000000000001,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":0.73399999999999999,"5min":1.016,"15min":1.016},"last":0.70899999999999996},{"interface":"front","average":{"1min":0.54500000000000004,"5min":0.55900000000000005,"15min":0.56599999999999995},"min":{"1min":0.36799999999999999,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":0.747,"5min":0.96499999999999997,"15min":0.96499999999999997},"last":0.63300000000000001}]},{"osd":2,"last update":"Fri Nov 24 08:00:25 2023","interfaces":[{"interface":"back","average":{"1min":0.81899999999999995,"5min":0.60899999999999999,"15min":0.56999999999999995},"min":{"1min":0.29699999999999999,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":6.024,"5min":6.024,"15min":6.024},"last":0.61899999999999999},{"interface":"front","average":{"1min":0.59399999999999997,"5min":0.59099999999999997,"15min":0.61699999999999999},"min":{"1min":0.34200000000000003,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.82199999999999995,"5min":1.119,"15min":1.119},"last":0.66300000000000003}]},{"osd":3,"last update":"Fri Nov 24 08:00:25 2023","interfaces":[{"interface":"back","average":{"1min":0.56999999999999995,"5min":0.70999999999999996,"15min":0.83999999999999997},"min":{"1min":0.33700000000000002,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.753,"5min":5.609,"15min":5.609},"last":0.68300000000000005},{"interface":"front","average":{"1min":0.79800000000000004,"5min":0.68400000000000005,"15min":0.67800000000000005},"min":{"1min":0.35999999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":4.2439999999999998,"5min":4.2439999999999998,"15min":4.2439999999999998},"last":0.65100000000000002}]},{"osd":4,"last update":"Fri Nov 24 08:00:25 2023","interfaces":[{"interface":"back","average":{"1min":0.53000000000000003,"5min":0.52000000000000002,"15min":0.51300000000000001},"min":{"1min":0.31900000000000001,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":1.145,"5min":1.145,"15min":1.145},"last":0.47599999999999998},{"interface":"front","average":{"1min":0.54900000000000004,"5min":0.57399999999999995,"15min":0.57199999999999995},"min":{"1min":0.25900000000000001,"5min":0.25900000000000001,"15min":0.25900000000000001},"max":{"1min":0.93700000000000006,"5min":0.93700000000000006,"15min":0.93700000000000006},"last":0.45700000000000002}]},{"osd":6,"last update":"Fri Nov 24 08:00:25 2023","interfaces":[{"interface":"back","average":{"1min":0.628,"5min":0.59899999999999998,"15min":0.57999999999999996},"min":{"1min":0.27900000000000003,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":2.2109999999999999,"5min":2.2109999999999999,"15min":2.2109999999999999},"last":0.42899999999999999},{"interface":"front","average":{"1min":0.69899999999999995,"5min":0.63600000000000001,"15min":0.628},"min":{"1min":0.38,"5min":0.376,"15min":0.376},"max":{"1min":3.391,"5min":3.391,"15min":3.391},"last":0.59299999999999997}]},{"osd":7,"last update":"Fri Nov 24 08:00:25 2023","interfaces":[{"interface":"back","average":{"1min":0.58399999999999996,"5min":0.63800000000000001,"15min":0.64300000000000002},"min":{"1min":0.39200000000000002,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.83999999999999997,"5min":0.96599999999999997,"15min":0.96599999999999997},"last":0.56200000000000006},{"interface":"front","average":{"1min":0.59899999999999998,"5min":0.65000000000000002,"15min":0.65200000000000002},"min":{"1min":0.38500000000000001,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.874,"5min":0.94199999999999995,"15min":0.94199999999999995},"last":0.54100000000000004}]}]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-11-24T08:00:33.469 INFO:tasks.ceph:pgid 1.0 last_scrub_stamp 2023-11-24T07:56:18.657844+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=24, tm_hour=7, tm_min=56, tm_sec=18, tm_wday=4, tm_yday=328, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=24, tm_hour=8, tm_min=0, tm_sec=27, tm_wday=4, tm_yday=328, tm_isdst=0) 2023-11-24T08:00:33.469 INFO:tasks.ceph:Still waiting for all pgs to be scrubbed. 2023-11-24T08:00:35.107 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-24T08:00:35.110 DEBUG:teuthology.orchestra.run.smithi076:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-24T08:00:53.470 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-11-24T08:00:53.611 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:53.609+0000 7f95f15a4700 1 Processor -- start 2023-11-24T08:00:53.612 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:53.610+0000 7f95f15a4700 1 -- start start 2023-11-24T08:00:53.614 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:53.610+0000 7f95f15a4700 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f95ec0ca6e0 0x7f95ec15b4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:00:53.614 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:53.610+0000 7f95f15a4700 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f95ec072d20 con 0x7f95ec0cf1e0 2023-11-24T08:00:53.614 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:53.610+0000 7f95f15a4700 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f95ec071020 con 0x7f95ec070a80 2023-11-24T08:00:53.615 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:53.610+0000 7f95f15a4700 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f95ec15ba10 con 0x7f95ec0ca6e0 2023-11-24T08:00:53.615 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:53.611+0000 7f95eaffd700 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7f95ec0cf1e0 0x7f95ec0c9fd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:60338/0 (socket says 172.21.15.59:60338) 2023-11-24T08:00:53.615 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:53.611+0000 7f95eaffd700 1 -- 172.21.15.59:0/412800276 learned_addr learned my addr 172.21.15.59:0/412800276 (peer_addr_for_me v1:172.21.15.59:0/0) 2023-11-24T08:00:53.615 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:53.611+0000 7f95ea7fc700 1 --2- 172.21.15.59:0/412800276 >> v2:172.21.15.76:3300/0 conn(0x7f95ec0ca6e0 0x7f95ec15b4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:00:53.615 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:53.611+0000 7f95e9ffb700 1 -- 172.21.15.59:0/412800276 <== mon.0 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 475326114 0 0) 0x7f95ec072d20 con 0x7f95ec0cf1e0 2023-11-24T08:00:53.615 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:53.611+0000 7f95e9ffb700 1 -- 172.21.15.59:0/412800276 --> v1:172.21.15.59:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f95d80035b0 con 0x7f95ec0cf1e0 2023-11-24T08:00:53.615 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:53.611+0000 7f95ea7fc700 1 -- 172.21.15.59:0/412800276 >> v1:172.21.15.76:6789/0 conn(0x7f95ec070a80 legacy=0x7f95ec06d450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:00:53.616 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:53.611+0000 7f95ea7fc700 1 -- 172.21.15.59:0/412800276 >> v1:172.21.15.59:6789/0 conn(0x7f95ec0cf1e0 legacy=0x7f95ec0c9fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:00:53.616 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:53.611+0000 7f95e9ffb700 1 -- 172.21.15.59:0/412800276 <== mon.0 v1:172.21.15.59:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3148778039 0 0) 0x7f95d80035b0 con 0x7f95ec0cf1e0 2023-11-24T08:00:53.616 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:53.611+0000 7f95ea7fc700 1 -- 172.21.15.59:0/412800276 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f95ec071300 con 0x7f95ec0ca6e0 2023-11-24T08:00:53.616 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:53.611+0000 7f95ea7fc700 1 --2- 172.21.15.59:0/412800276 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f95ec0ca6e0 0x7f95ec15b4d0 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f95d4009840 tx=0x7f95d4009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=50883e7fff5eea34 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:00:53.616 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:53.611+0000 7f95e9ffb700 1 -- 172.21.15.59:0/412800276 <== mon.1 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95dc001670 con 0x7f95ec0ca6e0 2023-11-24T08:00:53.616 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:53.612+0000 7f95e9ffb700 1 -- 172.21.15.59:0/412800276 <== mon.1 v2:172.21.15.76:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f95d4004030 con 0x7f95ec0ca6e0 2023-11-24T08:00:53.616 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:53.612+0000 7f95e9ffb700 1 -- 172.21.15.59:0/412800276 <== mon.1 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95d40043c0 con 0x7f95ec0ca6e0 2023-11-24T08:00:53.616 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:53.612+0000 7f95f15a4700 1 -- 172.21.15.59:0/412800276 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f95ec0ca6e0 msgr2=0x7f95ec15b4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:00:53.617 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:53.612+0000 7f95f15a4700 1 --2- 172.21.15.59:0/412800276 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f95ec0ca6e0 0x7f95ec15b4d0 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f95d4009840 tx=0x7f95d4009b50 comp rx=0 tx=0).stop 2023-11-24T08:00:53.617 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:53.612+0000 7f95f15a4700 1 -- 172.21.15.59:0/412800276 shutdown_connections 2023-11-24T08:00:53.617 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:53.612+0000 7f95f15a4700 1 --2- 172.21.15.59:0/412800276 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f95ec0ca6e0 0x7f95ec15b4d0 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:00:53.617 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:53.612+0000 7f95f15a4700 1 -- 172.21.15.59:0/412800276 >> 172.21.15.59:0/412800276 conn(0x7f95ec0cf980 msgr2=0x7f95ec0cfd60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:00:53.617 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:53.612+0000 7f95f15a4700 1 -- 172.21.15.59:0/412800276 shutdown_connections 2023-11-24T08:00:53.617 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:53.612+0000 7f95f15a4700 1 -- 172.21.15.59:0/412800276 wait complete. 2023-11-24T08:00:53.620 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:53.614+0000 7f95f15a4700 1 Processor -- start 2023-11-24T08:00:53.620 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:53.614+0000 7f95f15a4700 1 -- start start 2023-11-24T08:00:53.620 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:53.614+0000 7f95f15a4700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f95ec070a80 0x7f95ec14dd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:00:53.621 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:53.614+0000 7f95f15a4700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f95ec0ca6e0 0x7f95ec14e2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:00:53.621 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:53.614+0000 7f95f15a4700 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f95ec0cf1e0 0x7f95ec0b2670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:00:53.621 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:53.614+0000 7f95f15a4700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95ec1482e0 con 0x7f95ec070a80 2023-11-24T08:00:53.621 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:53.614+0000 7f95f15a4700 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95ec159f60 con 0x7f95ec0ca6e0 2023-11-24T08:00:53.621 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:53.614+0000 7f95f15a4700 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f95ec159c80 con 0x7f95ec0cf1e0 2023-11-24T08:00:53.621 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:53.614+0000 7f95eaffd700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f95ec070a80 0x7f95ec14dd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:00:53.621 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:53.614+0000 7f95eaffd700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f95ec070a80 0x7f95ec14dd90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:56870/0 (socket says 172.21.15.59:56870) 2023-11-24T08:00:53.622 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:53.614+0000 7f95eaffd700 1 -- 172.21.15.59:0/81535668 learned_addr learned my addr 172.21.15.59:0/81535668 (peer_addr_for_me v2:172.21.15.59:0/0) 2023-11-24T08:00:53.622 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:53.615+0000 7f95ea7fc700 1 --2- 172.21.15.59:0/81535668 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f95ec0ca6e0 0x7f95ec14e2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:00:53.622 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:53.615+0000 7f95eb7fe700 1 --2- 172.21.15.59:0/81535668 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f95ec0cf1e0 0x7f95ec0b2670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:00:53.622 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:53.615+0000 7f95eaffd700 1 -- 172.21.15.59:0/81535668 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f95ec0cf1e0 msgr2=0x7f95ec0b2670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:00:53.622 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:53.615+0000 7f95eaffd700 1 --2- 172.21.15.59:0/81535668 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f95ec0cf1e0 0x7f95ec0b2670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:00:53.622 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:53.615+0000 7f95eaffd700 1 -- 172.21.15.59:0/81535668 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f95ec0ca6e0 msgr2=0x7f95ec14e2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:00:53.622 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:53.615+0000 7f95eaffd700 1 --2- 172.21.15.59:0/81535668 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f95ec0ca6e0 0x7f95ec14e2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:00:53.622 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:53.615+0000 7f95eaffd700 1 -- 172.21.15.59:0/81535668 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f95d40094a0 con 0x7f95ec070a80 2023-11-24T08:00:53.623 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:53.615+0000 7f95eb7fe700 1 --2- 172.21.15.59:0/81535668 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f95ec0cf1e0 0x7f95ec0b2670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T08:00:53.623 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:53.615+0000 7f95eaffd700 1 --2- 172.21.15.59:0/81535668 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f95ec070a80 0x7f95ec14dd90 secure :-1 s=READY pgs=1185 cs=0 l=1 rev1=1 crypto rx=0x7f95dc00c520 tx=0x7f95dc00c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:00:53.623 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:53.615+0000 7f95e37fe700 1 -- 172.21.15.59:0/81535668 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95dc004c80 con 0x7f95ec070a80 2023-11-24T08:00:53.623 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:53.615+0000 7f95e37fe700 1 -- 172.21.15.59:0/81535668 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f95dc004e20 con 0x7f95ec070a80 2023-11-24T08:00:53.624 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:53.615+0000 7f95f15a4700 1 -- 172.21.15.59:0/81535668 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f95ec1459a0 con 0x7f95ec070a80 2023-11-24T08:00:53.624 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:53.615+0000 7f95e37fe700 1 -- 172.21.15.59:0/81535668 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95dc007310 con 0x7f95ec070a80 2023-11-24T08:00:53.624 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:53.615+0000 7f95f15a4700 1 -- 172.21.15.59:0/81535668 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f95ec147e90 con 0x7f95ec070a80 2023-11-24T08:00:53.624 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:53.616+0000 7f95f15a4700 1 -- 172.21.15.59:0/81535668 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f95ec0b5ab0 con 0x7f95ec070a80 2023-11-24T08:00:53.624 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:53.617+0000 7f95e37fe700 1 -- 172.21.15.59:0/81535668 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 12) v1 ==== 89792+0+0 (secure 0 0 0) 0x7f95dc007470 con 0x7f95ec070a80 2023-11-24T08:00:53.625 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:53.617+0000 7f95e37fe700 1 --2- 172.21.15.59:0/81535668 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f95cc06b500 0x7f95cc06d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:00:53.625 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:53.617+0000 7f95e37fe700 1 -- 172.21.15.59:0/81535668 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f95dc089c20 con 0x7f95ec070a80 2023-11-24T08:00:53.625 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:53.617+0000 7f95ea7fc700 1 --2- 172.21.15.59:0/81535668 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f95cc06b500 0x7f95cc06d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:00:53.625 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:53.618+0000 7f95ea7fc700 1 --2- 172.21.15.59:0/81535668 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f95cc06b500 0x7f95cc06d9c0 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7f95d40090b0 tx=0x7f95d4008030 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:00:53.625 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:53.622+0000 7f95e37fe700 1 -- 172.21.15.59:0/81535668 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f95dc054720 con 0x7f95ec070a80 2023-11-24T08:00:53.820 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:53.818+0000 7f95f15a4700 1 -- 172.21.15.59:0/81535668 --> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f95ec071ba0 con 0x7f95cc06b500 2023-11-24T08:00:53.822 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:53.821+0000 7f95e37fe700 1 -- 172.21.15.59:0/81535668 <== mgr.4100 v2:172.21.15.76:6832/4264046536 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+50566 (secure 0 0 0) 0x7f95d4004570 con 0x7f95cc06b500 2023-11-24T08:00:53.824 INFO:teuthology.orchestra.run.smithi059.stdout: 2023-11-24T08:00:53.825 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:53.823+0000 7f95f15a4700 1 -- 172.21.15.59:0/81535668 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f95cc06b500 msgr2=0x7f95cc06d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:00:53.825 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:53.823+0000 7f95f15a4700 1 --2- 172.21.15.59:0/81535668 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f95cc06b500 0x7f95cc06d9c0 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7f95d40090b0 tx=0x7f95d4008030 comp rx=0 tx=0).stop 2023-11-24T08:00:53.826 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:53.823+0000 7f95f15a4700 1 -- 172.21.15.59:0/81535668 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f95ec070a80 msgr2=0x7f95ec14dd90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:00:53.826 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:53.823+0000 7f95f15a4700 1 --2- 172.21.15.59:0/81535668 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f95ec070a80 0x7f95ec14dd90 secure :-1 s=READY pgs=1185 cs=0 l=1 rev1=1 crypto rx=0x7f95dc00c520 tx=0x7f95dc00c8e0 comp rx=0 tx=0).stop 2023-11-24T08:00:53.826 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:53.823+0000 7f95f15a4700 1 -- 172.21.15.59:0/81535668 shutdown_connections 2023-11-24T08:00:53.826 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:53.823+0000 7f95f15a4700 1 --2- 172.21.15.59:0/81535668 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f95cc06b500 0x7f95cc06d9c0 unknown :-1 s=CLOSED pgs=537 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:00:53.826 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:53.823+0000 7f95f15a4700 1 --2- 172.21.15.59:0/81535668 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f95ec0cf1e0 0x7f95ec0b2670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:00:53.826 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:53.823+0000 7f95f15a4700 1 --2- 172.21.15.59:0/81535668 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f95ec070a80 0x7f95ec14dd90 unknown :-1 s=CLOSED pgs=1185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:00:53.826 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:53.823+0000 7f95f15a4700 1 --2- 172.21.15.59:0/81535668 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f95ec0ca6e0 0x7f95ec14e2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:00:53.827 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:53.823+0000 7f95f15a4700 1 -- 172.21.15.59:0/81535668 >> 172.21.15.59:0/81535668 conn(0x7f95ec0cf980 msgr2=0x7f95ec074030 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:00:53.827 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:53.823+0000 7f95f15a4700 1 -- 172.21.15.59:0/81535668 shutdown_connections 2023-11-24T08:00:53.827 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:53.823+0000 7f95f15a4700 1 -- 172.21.15.59:0/81535668 wait complete. 2023-11-24T08:00:53.827 INFO:teuthology.orchestra.run.smithi059.stderr:dumped all 2023-11-24T08:00:53.837 INFO:teuthology.orchestra.run.smithi059.stdout:{"pg_ready":true,"pg_map":{"version":220,"stamp":"2023-11-24T08:00:52.361064+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":119,"ondisk_log_size":119,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":2,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":2,"kb":754974720,"kb_used":230184,"kb_used_data":9864,"kb_used_omap":12,"kb_used_meta":220211,"kb_avail":754744536,"statfs":{"total":773094113280,"available":772858404864,"internally_reserved":0,"allocated":10100736,"data_stored":4994848,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12745,"internal_metadata":225496631},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002682"},"pg_stats":[{"pgid":"1.0","version":"90'119","reported_seq":237,"reported_epoch":90,"state":"active+clean","last_fresh":"2023-11-24T08:00:30.583502+0000","last_change":"2023-11-24T08:00:30.583532+0000","last_active":"2023-11-24T08:00:30.583502+0000","last_peered":"2023-11-24T08:00:30.583502+0000","last_clean":"2023-11-24T08:00:30.583502+0000","last_became_active":"2023-11-24T07:56:19.668601+0000","last_became_peered":"2023-11-24T07:56:19.668601+0000","last_unstale":"2023-11-24T08:00:30.583502+0000","last_undegraded":"2023-11-24T08:00:30.583502+0000","last_fullsized":"2023-11-24T08:00:30.583502+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"90'119","last_scrub_stamp":"2023-11-24T08:00:30.583424+0000","last_deep_scrub":"90'119","last_deep_scrub_stamp":"2023-11-24T08:00:30.583424+0000","last_clean_scrub_stamp":"2023-11-24T08:00:30.583424+0000","objects_scrubbed":2,"log_size":119,"log_dups_size":0,"ondisk_log_size":119,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T13:18:05.715905+0000","scrub_duration":0.0056663989999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":119,"ondisk_log_size":119,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574905,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":28108,"kb_used_data":1088,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343732,"statfs":{"total":96636764160,"available":96607981568,"internally_reserved":0,"allocated":1114112,"data_stored":476764,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 08:00:21 2023","interfaces":[{"interface":"back","average":{"1min":0.66700000000000004,"5min":0.56200000000000006,"15min":0.55800000000000005},"min":{"1min":0.27700000000000002,"5min":0.27600000000000002,"15min":0.27600000000000002},"max":{"1min":3.1640000000000001,"5min":3.1640000000000001,"15min":3.1640000000000001},"last":0.58799999999999997},{"interface":"front","average":{"1min":0.51800000000000002,"5min":0.54400000000000004,"15min":0.55000000000000004},"min":{"1min":0.35499999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.67200000000000004,"5min":0.81000000000000005,"15min":0.81000000000000005},"last":0.60099999999999998}]},{"osd":1,"last update":"Fri Nov 24 08:00:21 2023","interfaces":[{"interface":"back","average":{"1min":0.50700000000000001,"5min":0.64400000000000002,"15min":0.78000000000000003},"min":{"1min":0.28799999999999998,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":0.83499999999999996,"5min":6.1070000000000002,"15min":6.1070000000000002},"last":0.56999999999999995},{"interface":"front","average":{"1min":0.68899999999999995,"5min":0.56899999999999995,"15min":0.57599999999999996},"min":{"1min":0.38400000000000001,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":3.1970000000000001,"5min":3.1970000000000001,"15min":3.1970000000000001},"last":0.40000000000000002}]},{"osd":2,"last update":"Fri Nov 24 08:00:21 2023","interfaces":[{"interface":"back","average":{"1min":0.53500000000000003,"5min":0.56200000000000006,"15min":0.60499999999999998},"min":{"1min":0.35599999999999998,"5min":0.308,"15min":0.308},"max":{"1min":0.94099999999999995,"5min":1.613,"15min":1.613},"last":0.45200000000000001},{"interface":"front","average":{"1min":0.51800000000000002,"5min":0.61699999999999999,"15min":0.73299999999999998},"min":{"1min":0.35199999999999998,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.65400000000000003,"5min":4.5220000000000002,"15min":4.5220000000000002},"last":0.55100000000000005}]},{"osd":3,"last update":"Fri Nov 24 08:00:21 2023","interfaces":[{"interface":"back","average":{"1min":0.56399999999999995,"5min":0.65800000000000003,"15min":0.75600000000000001},"min":{"1min":0.42599999999999999,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.88500000000000001,"5min":4.6760000000000002,"15min":4.6760000000000002},"last":0.42399999999999999},{"interface":"front","average":{"1min":0.64500000000000002,"5min":0.748,"15min":0.878},"min":{"1min":0.51200000000000001,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":1.1299999999999999,"5min":4.6539999999999999,"15min":4.6539999999999999},"last":0.63}]},{"osd":4,"last update":"Fri Nov 24 08:00:21 2023","interfaces":[{"interface":"back","average":{"1min":0.53300000000000003,"5min":0.53800000000000003,"15min":0.55800000000000005},"min":{"1min":0.33600000000000002,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":1.0760000000000001,"5min":1.0760000000000001,"15min":1.0760000000000001},"last":0.504},{"interface":"front","average":{"1min":0.59299999999999997,"5min":0.60499999999999998,"15min":0.64600000000000002},"min":{"1min":0.39900000000000002,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":1.284,"5min":1.629,"15min":1.629},"last":0.52300000000000002}]},{"osd":5,"last update":"Fri Nov 24 08:00:21 2023","interfaces":[{"interface":"back","average":{"1min":0.51100000000000001,"5min":0.52900000000000003,"15min":0.54100000000000004},"min":{"1min":0.36399999999999999,"5min":0.315,"15min":0.315},"max":{"1min":0.77200000000000002,"5min":0.875,"15min":0.875},"last":0.48899999999999999},{"interface":"front","average":{"1min":0.626,"5min":0.60299999999999998,"15min":0.61599999999999999},"min":{"1min":0.47499999999999998,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":1.0429999999999999,"5min":1.0429999999999999,"15min":1.0429999999999999},"last":0.61499999999999999}]},{"osd":6,"last update":"Fri Nov 24 08:00:21 2023","interfaces":[{"interface":"back","average":{"1min":0.55000000000000004,"5min":0.58399999999999996,"15min":0.60199999999999998},"min":{"1min":0.38600000000000001,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.745,"5min":0.85799999999999998,"15min":0.85799999999999998},"last":0.64500000000000002},{"interface":"front","average":{"1min":0.61199999999999999,"5min":0.64000000000000001,"15min":0.66000000000000003},"min":{"1min":0.438,"5min":0.438,"15min":0.438},"max":{"1min":0.91700000000000004,"5min":0.99099999999999999,"15min":0.99099999999999999},"last":0.65600000000000003}]}]},{"osd":6,"up_from":13,"seq":55834574906,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":28112,"kb_used_data":1088,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343728,"statfs":{"total":96636764160,"available":96607977472,"internally_reserved":0,"allocated":1114112,"data_stored":476764,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 08:00:24 2023","interfaces":[{"interface":"back","average":{"1min":0.48899999999999999,"5min":0.52600000000000002,"15min":0.54200000000000004},"min":{"1min":0.24099999999999999,"5min":0.24099999999999999,"15min":0.24099999999999999},"max":{"1min":0.65300000000000002,"5min":0.93100000000000005,"15min":0.93100000000000005},"last":0.59199999999999997},{"interface":"front","average":{"1min":0.52200000000000002,"5min":0.52000000000000002,"15min":0.51700000000000002},"min":{"1min":0.40000000000000002,"5min":0.311,"15min":0.311},"max":{"1min":0.73799999999999999,"5min":0.82199999999999995,"15min":0.82199999999999995},"last":0.53300000000000003}]},{"osd":1,"last update":"Fri Nov 24 08:00:24 2023","interfaces":[{"interface":"back","average":{"1min":0.52500000000000002,"5min":0.51300000000000001,"15min":0.51200000000000001},"min":{"1min":0.311,"5min":0.311,"15min":0.311},"max":{"1min":0.68700000000000006,"5min":0.81299999999999994,"15min":0.81299999999999994},"last":0.54700000000000004},{"interface":"front","average":{"1min":0.58699999999999997,"5min":0.58199999999999996,"15min":0.60199999999999998},"min":{"1min":0.308,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":0.75,"5min":1.0069999999999999,"15min":1.0069999999999999},"last":0.56699999999999995}]},{"osd":2,"last update":"Fri Nov 24 08:00:24 2023","interfaces":[{"interface":"back","average":{"1min":0.56399999999999995,"5min":0.57499999999999996,"15min":0.57499999999999996},"min":{"1min":0.32400000000000001,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.90500000000000003,"5min":0.90600000000000003,"15min":0.90600000000000003},"last":0.68999999999999995},{"interface":"front","average":{"1min":0.56299999999999994,"5min":0.58199999999999996,"15min":0.58899999999999997},"min":{"1min":0.34699999999999998,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.878,"5min":0.878,"15min":0.878},"last":0.64300000000000002}]},{"osd":3,"last update":"Fri Nov 24 08:00:24 2023","interfaces":[{"interface":"back","average":{"1min":0.57099999999999995,"5min":0.56499999999999995,"15min":0.56699999999999995},"min":{"1min":0.32600000000000001,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.76900000000000002,"5min":0.876,"15min":0.876},"last":0.629},{"interface":"front","average":{"1min":0.56200000000000006,"5min":0.56100000000000005,"15min":0.55700000000000005},"min":{"1min":0.42799999999999999,"5min":0.379,"15min":0.379},"max":{"1min":0.753,"5min":0.876,"15min":0.876},"last":0.61099999999999999}]},{"osd":4,"last update":"Fri Nov 24 08:00:24 2023","interfaces":[{"interface":"back","average":{"1min":0.49299999999999999,"5min":0.52200000000000002,"15min":0.51800000000000002},"min":{"1min":0.32900000000000001,"5min":0.25,"15min":0.25},"max":{"1min":0.70699999999999996,"5min":0.89400000000000002,"15min":0.89400000000000002},"last":0.435},{"interface":"front","average":{"1min":0.59799999999999998,"5min":0.58399999999999996,"15min":0.59599999999999997},"min":{"1min":0.34000000000000002,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":0.78600000000000003,"5min":1.087,"15min":1.087},"last":0.46600000000000003}]},{"osd":5,"last update":"Fri Nov 24 08:00:24 2023","interfaces":[{"interface":"back","average":{"1min":0.52900000000000003,"5min":0.55100000000000005,"15min":0.53700000000000003},"min":{"1min":0.35599999999999998,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":0.753,"5min":0.97999999999999998,"15min":0.97999999999999998},"last":0.41199999999999998},{"interface":"front","average":{"1min":0.58299999999999996,"5min":0.60799999999999998,"15min":0.60999999999999999},"min":{"1min":0.42899999999999999,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.78800000000000003,"5min":1.0009999999999999,"15min":1.0009999999999999},"last":0.48899999999999999}]},{"osd":7,"last update":"Fri Nov 24 08:00:24 2023","interfaces":[{"interface":"back","average":{"1min":0.624,"5min":0.61499999999999999,"15min":0.61699999999999999},"min":{"1min":0.38800000000000001,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":0.81799999999999995,"5min":1.0880000000000001,"15min":1.0880000000000001},"last":0.51200000000000001},{"interface":"front","average":{"1min":0.65700000000000003,"5min":0.65500000000000003,"15min":0.66500000000000004},"min":{"1min":0.41699999999999998,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.81899999999999995,"5min":0.95599999999999996,"15min":0.95599999999999996},"last":0.65200000000000002}]}]},{"osd":1,"up_from":13,"seq":55834574907,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28684,"kb_used_data":1668,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343156,"statfs":{"total":96636764160,"available":96607391744,"internally_reserved":0,"allocated":1708032,"data_stored":1067132,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 08:00:24 2023","interfaces":[{"interface":"back","average":{"1min":0.33600000000000002,"5min":0.34899999999999998,"15min":0.35899999999999999},"min":{"1min":0.251,"5min":0.20200000000000001,"15min":0.20200000000000001},"max":{"1min":0.60499999999999998,"5min":0.70899999999999996,"15min":0.70899999999999996},"last":0.47999999999999998},{"interface":"front","average":{"1min":0.36699999999999999,"5min":0.378,"15min":0.39100000000000001},"min":{"1min":0.23300000000000001,"5min":0.214,"15min":0.214},"max":{"1min":0.55400000000000005,"5min":0.82499999999999996,"15min":0.82499999999999996},"last":0.45700000000000002}]},{"osd":2,"last update":"Fri Nov 24 08:00:24 2023","interfaces":[{"interface":"back","average":{"1min":0.34599999999999997,"5min":0.35399999999999998,"15min":0.36599999999999999},"min":{"1min":0.26500000000000001,"5min":0.22700000000000001,"15min":0.22700000000000001},"max":{"1min":0.47099999999999997,"5min":0.60999999999999999,"15min":0.60999999999999999},"last":0.39100000000000001},{"interface":"front","average":{"1min":0.40799999999999997,"5min":0.40400000000000003,"15min":0.40999999999999998},"min":{"1min":0.318,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":0.55600000000000005,"5min":0.63700000000000001,"15min":0.63700000000000001},"last":0.56200000000000006}]},{"osd":3,"last update":"Fri Nov 24 08:00:24 2023","interfaces":[{"interface":"back","average":{"1min":0.41199999999999998,"5min":0.42599999999999999,"15min":0.438},"min":{"1min":0.28399999999999997,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":0.58299999999999996,"5min":0.66500000000000004,"15min":0.66500000000000004},"last":0.52900000000000003},{"interface":"front","average":{"1min":0.40899999999999997,"5min":0.41399999999999998,"15min":0.42299999999999999},"min":{"1min":0.29499999999999998,"5min":0.28499999999999998,"15min":0.28499999999999998},"max":{"1min":0.60899999999999999,"5min":0.76300000000000001,"15min":0.76300000000000001},"last":0.50600000000000001}]},{"osd":4,"last update":"Fri Nov 24 08:00:24 2023","interfaces":[{"interface":"back","average":{"1min":0.64200000000000002,"5min":0.60999999999999999,"15min":0.60999999999999999},"min":{"1min":0.42499999999999999,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":0.90800000000000003,"5min":1.5580000000000001,"15min":1.5580000000000001},"last":0.65100000000000002},{"interface":"front","average":{"1min":0.60599999999999998,"5min":0.61599999999999999,"15min":0.60899999999999999},"min":{"1min":0.39700000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.84999999999999998,"5min":0.84999999999999998,"15min":0.84999999999999998},"last":0.58899999999999997}]},{"osd":5,"last update":"Fri Nov 24 08:00:24 2023","interfaces":[{"interface":"back","average":{"1min":0.64300000000000002,"5min":0.66200000000000003,"15min":0.70399999999999996},"min":{"1min":0.44,"5min":0.37,"15min":0.37},"max":{"1min":0.98699999999999999,"5min":2.7269999999999999,"15min":2.7269999999999999},"last":0.73199999999999998},{"interface":"front","average":{"1min":0.629,"5min":0.63600000000000001,"15min":0.64200000000000002},"min":{"1min":0.40000000000000002,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.96199999999999997,"5min":1.6459999999999999,"15min":1.6459999999999999},"last":0.752}]},{"osd":6,"last update":"Fri Nov 24 08:00:24 2023","interfaces":[{"interface":"back","average":{"1min":0.65200000000000002,"5min":0.64200000000000002,"15min":0.65200000000000002},"min":{"1min":0.40999999999999998,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.84599999999999997,"5min":0.98199999999999998,"15min":0.98199999999999998},"last":0.71299999999999997},{"interface":"front","average":{"1min":0.65400000000000003,"5min":0.628,"15min":0.62},"min":{"1min":0.48899999999999999,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":0.93600000000000005,"5min":0.93600000000000005,"15min":0.93600000000000005},"last":0.60899999999999999}]},{"osd":7,"last update":"Fri Nov 24 08:00:24 2023","interfaces":[{"interface":"back","average":{"1min":0.64800000000000002,"5min":0.65000000000000002,"15min":0.64100000000000001},"min":{"1min":0.495,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.91100000000000003,"5min":1.0649999999999999,"15min":1.0649999999999999},"last":0.63700000000000001},{"interface":"front","average":{"1min":0.67700000000000005,"5min":0.64600000000000002,"15min":0.66200000000000003},"min":{"1min":0.45000000000000001,"5min":0.442,"15min":0.442},"max":{"1min":1.004,"5min":1.004,"15min":1.004},"last":0.79500000000000004}]}]},{"osd":0,"up_from":13,"seq":55834574907,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":28108,"kb_used_data":1088,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343732,"statfs":{"total":96636764160,"available":96607981568,"internally_reserved":0,"allocated":1114112,"data_stored":476764,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Nov 24 08:00:26 2023","interfaces":[{"interface":"back","average":{"1min":0.36499999999999999,"5min":0.50900000000000001,"15min":0.64500000000000002},"min":{"1min":0.22700000000000001,"5min":0.21299999999999999,"15min":0.21299999999999999},"max":{"1min":0.60499999999999998,"5min":6.3550000000000004,"15min":6.3550000000000004},"last":0.51000000000000001},{"interface":"front","average":{"1min":0.41799999999999998,"5min":0.93500000000000005,"15min":1.464},"min":{"1min":0.318,"5min":0.26000000000000001,"15min":0.26000000000000001},"max":{"1min":0.58499999999999996,"5min":22.916,"15min":22.916},"last":0.44500000000000001}]},{"osd":2,"last update":"Fri Nov 24 08:00:26 2023","interfaces":[{"interface":"back","average":{"1min":0.379,"5min":0.498,"15min":0.63200000000000001},"min":{"1min":0.21299999999999999,"5min":0.21299999999999999,"15min":0.21299999999999999},"max":{"1min":0.57399999999999995,"5min":6.3899999999999997,"15min":6.3899999999999997},"last":0.40999999999999998},{"interface":"front","average":{"1min":0.34599999999999997,"5min":0.63800000000000001,"15min":0.92100000000000004},"min":{"1min":0.25,"5min":0.25,"15min":0.25},"max":{"1min":0.51200000000000001,"5min":11.93,"15min":11.93},"last":0.35099999999999998}]},{"osd":3,"last update":"Fri Nov 24 08:00:26 2023","interfaces":[{"interface":"back","average":{"1min":0.35699999999999998,"5min":0.38800000000000001,"15min":0.40899999999999997},"min":{"1min":0.25,"5min":0.23799999999999999,"15min":0.23799999999999999},"max":{"1min":0.48799999999999999,"5min":1.155,"15min":1.155},"last":0.38600000000000001},{"interface":"front","average":{"1min":0.41499999999999998,"5min":0.70099999999999996,"15min":0.98499999999999999},"min":{"1min":0.29199999999999998,"5min":0.27400000000000002,"15min":0.27400000000000002},"max":{"1min":0.64800000000000002,"5min":12.116,"15min":12.116},"last":0.47399999999999998}]},{"osd":4,"last update":"Fri Nov 24 08:00:26 2023","interfaces":[{"interface":"back","average":{"1min":0.61299999999999999,"5min":0.626,"15min":0.64000000000000001},"min":{"1min":0.432,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.97399999999999998,"5min":1.044,"15min":1.044},"last":0.63600000000000001},{"interface":"front","average":{"1min":0.57399999999999995,"5min":1.1339999999999999,"15min":1.667},"min":{"1min":0.42499999999999999,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.82999999999999996,"5min":22.963999999999999,"15min":22.963999999999999},"last":0.72499999999999998}]},{"osd":5,"last update":"Fri Nov 24 08:00:26 2023","interfaces":[{"interface":"back","average":{"1min":0.62,"5min":0.89100000000000001,"15min":1.159},"min":{"1min":0.501,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.91900000000000004,"5min":12.084,"15min":12.084},"last":0.621},{"interface":"front","average":{"1min":0.60399999999999998,"5min":0.64700000000000002,"15min":0.66700000000000004},"min":{"1min":0.41299999999999998,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.80500000000000005,"5min":1.073,"15min":1.073},"last":0.76700000000000002}]},{"osd":6,"last update":"Fri Nov 24 08:00:26 2023","interfaces":[{"interface":"back","average":{"1min":0.63,"5min":0.65600000000000003,"15min":0.67900000000000005},"min":{"1min":0.45400000000000001,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.93999999999999995,"5min":1.0580000000000001,"15min":1.0580000000000001},"last":0.67200000000000004},{"interface":"front","average":{"1min":0.59799999999999998,"5min":0.89900000000000002,"15min":1.175},"min":{"1min":0.437,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":0.88100000000000001,"5min":12.045,"15min":12.045},"last":0.746}]},{"osd":7,"last update":"Fri Nov 24 08:00:26 2023","interfaces":[{"interface":"back","average":{"1min":0.60599999999999998,"5min":0.63300000000000001,"15min":0.64800000000000002},"min":{"1min":0.48399999999999999,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":0.90000000000000002,"5min":1.038,"15min":1.038},"last":0.70899999999999996},{"interface":"front","average":{"1min":0.65100000000000002,"5min":1.169,"15min":1.698},"min":{"1min":0.45300000000000001,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":0.95799999999999996,"5min":22.995000000000001,"15min":22.995000000000001},"last":0.59999999999999998}]}]},{"osd":2,"up_from":13,"seq":55834574907,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":28108,"kb_used_data":1088,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343732,"statfs":{"total":96636764160,"available":96607981568,"internally_reserved":0,"allocated":1114112,"data_stored":476764,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 08:00:21 2023","interfaces":[{"interface":"back","average":{"1min":0.34499999999999997,"5min":0.35399999999999998,"15min":0.36499999999999999},"min":{"1min":0.22500000000000001,"5min":0.219,"15min":0.219},"max":{"1min":0.499,"5min":0.58099999999999996,"15min":0.58099999999999996},"last":0.37},{"interface":"front","average":{"1min":0.41299999999999998,"5min":0.39600000000000002,"15min":0.40000000000000002},"min":{"1min":0.28899999999999998,"5min":0.27400000000000002,"15min":0.27400000000000002},"max":{"1min":0.69099999999999995,"5min":0.69099999999999995,"15min":0.69099999999999995},"last":0.438}]},{"osd":1,"last update":"Fri Nov 24 08:00:21 2023","interfaces":[{"interface":"back","average":{"1min":0.34999999999999998,"5min":0.35499999999999998,"15min":0.37},"min":{"1min":0.249,"5min":0.21099999999999999,"15min":0.21099999999999999},"max":{"1min":0.55700000000000005,"5min":0.59199999999999997,"15min":0.59199999999999997},"last":0.39200000000000002},{"interface":"front","average":{"1min":0.379,"5min":0.38800000000000001,"15min":0.40600000000000003},"min":{"1min":0.24399999999999999,"5min":0.24399999999999999,"15min":0.24399999999999999},"max":{"1min":0.55100000000000005,"5min":0.627,"15min":0.627},"last":0.501}]},{"osd":3,"last update":"Fri Nov 24 08:00:21 2023","interfaces":[{"interface":"back","average":{"1min":0.42499999999999999,"5min":0.40400000000000003,"15min":0.40799999999999997},"min":{"1min":0.28699999999999998,"5min":0.26400000000000001,"15min":0.26400000000000001},"max":{"1min":0.63500000000000001,"5min":0.64200000000000002,"15min":0.64200000000000002},"last":0.46800000000000003},{"interface":"front","average":{"1min":0.39000000000000001,"5min":0.40000000000000002,"15min":0.40899999999999997},"min":{"1min":0.315,"5min":0.27500000000000002,"15min":0.27500000000000002},"max":{"1min":0.58299999999999996,"5min":0.64600000000000002,"15min":0.64600000000000002},"last":0.42499999999999999}]},{"osd":4,"last update":"Fri Nov 24 08:00:21 2023","interfaces":[{"interface":"back","average":{"1min":0.58699999999999997,"5min":0.57099999999999995,"15min":0.59499999999999997},"min":{"1min":0.32900000000000001,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.91000000000000003,"5min":0.91000000000000003,"15min":0.91000000000000003},"last":0.82099999999999995},{"interface":"front","average":{"1min":0.57099999999999995,"5min":0.57699999999999996,"15min":0.59199999999999997},"min":{"1min":0.35299999999999998,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.75600000000000001,"5min":0.83799999999999997,"15min":0.83799999999999997},"last":0.73399999999999999}]},{"osd":5,"last update":"Fri Nov 24 08:00:21 2023","interfaces":[{"interface":"back","average":{"1min":0.57899999999999996,"5min":0.58999999999999997,"15min":0.60999999999999999},"min":{"1min":0.39100000000000001,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.84099999999999997,"5min":0.84899999999999998,"15min":0.84899999999999998},"last":0.78100000000000003},{"interface":"front","average":{"1min":0.64500000000000002,"5min":0.64600000000000002,"15min":0.69199999999999995},"min":{"1min":0.45100000000000001,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":1.119,"5min":2.1699999999999999,"15min":2.1699999999999999},"last":0.84899999999999998}]},{"osd":6,"last update":"Fri Nov 24 08:00:21 2023","interfaces":[{"interface":"back","average":{"1min":0.64000000000000001,"5min":0.625,"15min":0.65300000000000002},"min":{"1min":0.41999999999999998,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":0.88300000000000001,"5min":0.88300000000000001,"15min":0.88300000000000001},"last":0.88500000000000001},{"interface":"front","average":{"1min":0.65400000000000003,"5min":0.629,"15min":0.65700000000000003},"min":{"1min":0.44500000000000001,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":1.105,"5min":1.105,"15min":1.105},"last":0.70399999999999996}]},{"osd":7,"last update":"Fri Nov 24 08:00:21 2023","interfaces":[{"interface":"back","average":{"1min":0.628,"5min":0.626,"15min":0.65000000000000002},"min":{"1min":0.42199999999999999,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":0.93000000000000005,"5min":0.93000000000000005,"15min":0.93000000000000005},"last":0.68600000000000005},{"interface":"front","average":{"1min":0.627,"5min":0.63600000000000001,"15min":0.66500000000000004},"min":{"1min":0.46200000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.80000000000000004,"5min":0.90000000000000002,"15min":0.90000000000000002},"last":0.80000000000000004}]}]},{"osd":3,"up_from":13,"seq":55834574907,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":28112,"kb_used_data":1088,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343728,"statfs":{"total":96636764160,"available":96607977472,"internally_reserved":0,"allocated":1114112,"data_stored":476764,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 08:00:29 2023","interfaces":[{"interface":"back","average":{"1min":0.37,"5min":0.35199999999999998,"15min":0.35399999999999998},"min":{"1min":0.23000000000000001,"5min":0.183,"15min":0.183},"max":{"1min":0.65600000000000003,"5min":0.84799999999999998,"15min":0.84799999999999998},"last":0.60999999999999999},{"interface":"front","average":{"1min":0.38,"5min":0.35699999999999998,"15min":0.36299999999999999},"min":{"1min":0.24399999999999999,"5min":0.23699999999999999,"15min":0.23699999999999999},"max":{"1min":0.58499999999999996,"5min":0.873,"15min":0.873},"last":0.53900000000000003}]},{"osd":1,"last update":"Fri Nov 24 08:00:29 2023","interfaces":[{"interface":"back","average":{"1min":0.41799999999999998,"5min":0.38200000000000001,"15min":0.377},"min":{"1min":0.24399999999999999,"5min":0.217,"15min":0.217},"max":{"1min":0.71499999999999997,"5min":0.92000000000000004,"15min":0.92000000000000004},"last":0.49399999999999999},{"interface":"front","average":{"1min":0.38600000000000001,"5min":0.377,"15min":0.38300000000000001},"min":{"1min":0.26600000000000001,"5min":0.248,"15min":0.248},"max":{"1min":0.54500000000000004,"5min":0.81699999999999995,"15min":0.81699999999999995},"last":0.65500000000000003}]},{"osd":2,"last update":"Fri Nov 24 08:00:29 2023","interfaces":[{"interface":"back","average":{"1min":0.38600000000000001,"5min":0.35899999999999999,"15min":0.35999999999999999},"min":{"1min":0.24299999999999999,"5min":0.24299999999999999,"15min":0.24299999999999999},"max":{"1min":0.54600000000000004,"5min":0.88500000000000001,"15min":0.88500000000000001},"last":0.58499999999999996},{"interface":"front","average":{"1min":0.433,"5min":0.434,"15min":0.45600000000000002},"min":{"1min":0.312,"5min":0.311,"15min":0.311},"max":{"1min":0.65100000000000002,"5min":0.98099999999999998,"15min":0.98099999999999998},"last":0.70699999999999996}]},{"osd":4,"last update":"Fri Nov 24 08:00:29 2023","interfaces":[{"interface":"back","average":{"1min":0.62,"5min":0.66900000000000004,"15min":0.754},"min":{"1min":0.47099999999999997,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.78600000000000003,"5min":5.7350000000000003,"15min":5.7350000000000003},"last":0.82199999999999995},{"interface":"front","average":{"1min":0.56599999999999995,"5min":0.57699999999999996,"15min":0.58199999999999996},"min":{"1min":0.41299999999999998,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.83099999999999996,"5min":1.27,"15min":1.27},"last":0.73299999999999998}]},{"osd":5,"last update":"Fri Nov 24 08:00:29 2023","interfaces":[{"interface":"back","average":{"1min":0.59399999999999997,"5min":0.57999999999999996,"15min":0.58499999999999996},"min":{"1min":0.45700000000000002,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":0.80300000000000005,"5min":1.4430000000000001,"15min":1.4430000000000001},"last":0.80800000000000005},{"interface":"front","average":{"1min":0.59899999999999998,"5min":0.60899999999999999,"15min":0.621},"min":{"1min":0.42899999999999999,"5min":0.377,"15min":0.377},"max":{"1min":0.874,"5min":1.4239999999999999,"15min":1.4239999999999999},"last":0.80000000000000004}]},{"osd":6,"last update":"Fri Nov 24 08:00:29 2023","interfaces":[{"interface":"back","average":{"1min":0.60599999999999998,"5min":0.60399999999999998,"15min":0.61399999999999999},"min":{"1min":0.46400000000000002,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":0.92100000000000004,"5min":1.355,"15min":1.355},"last":0.76800000000000002},{"interface":"front","average":{"1min":0.59999999999999998,"5min":0.69599999999999995,"15min":0.79600000000000004},"min":{"1min":0.42099999999999999,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.78200000000000003,"5min":5.6479999999999997,"15min":5.6479999999999997},"last":0.88300000000000001}]},{"osd":7,"last update":"Fri Nov 24 08:00:29 2023","interfaces":[{"interface":"back","average":{"1min":0.58999999999999997,"5min":0.626,"15min":0.65200000000000002},"min":{"1min":0.437,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":0.753,"5min":1.51,"15min":1.51},"last":0.85799999999999998},{"interface":"front","average":{"1min":0.61399999999999999,"5min":0.69699999999999995,"15min":0.79300000000000004},"min":{"1min":0.42199999999999999,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.82799999999999996,"5min":5.7069999999999999,"15min":5.7069999999999999},"last":0.83599999999999997}]}]},{"osd":4,"up_from":13,"seq":55834574906,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":28108,"kb_used_data":1088,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343732,"statfs":{"total":96636764160,"available":96607981568,"internally_reserved":0,"allocated":1114112,"data_stored":476764,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 08:00:29 2023","interfaces":[{"interface":"back","average":{"1min":0.55900000000000005,"5min":0.55600000000000005,"15min":0.57199999999999995},"min":{"1min":0.38600000000000001,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.79500000000000004,"5min":0.93300000000000005,"15min":0.93300000000000005},"last":0.70099999999999996},{"interface":"front","average":{"1min":0.54700000000000004,"5min":0.52900000000000003,"15min":0.53800000000000003},"min":{"1min":0.378,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.72099999999999997,"5min":1.0249999999999999,"15min":1.0249999999999999},"last":0.53800000000000003}]},{"osd":1,"last update":"Fri Nov 24 08:00:29 2023","interfaces":[{"interface":"back","average":{"1min":0.53900000000000003,"5min":0.54300000000000004,"15min":0.54900000000000004},"min":{"1min":0.42299999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.68500000000000005,"5min":0.86399999999999999,"15min":0.86399999999999999},"last":0.59499999999999997},{"interface":"front","average":{"1min":0.59399999999999997,"5min":0.54600000000000004,"15min":0.54900000000000004},"min":{"1min":0.31900000000000001,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.83599999999999997,"5min":0.98699999999999999,"15min":0.98699999999999999},"last":0.76200000000000001}]},{"osd":2,"last update":"Fri Nov 24 08:00:29 2023","interfaces":[{"interface":"back","average":{"1min":0.59399999999999997,"5min":0.55500000000000005,"15min":0.55900000000000005},"min":{"1min":0.41399999999999998,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.84999999999999998,"5min":0.86399999999999999,"15min":0.86399999999999999},"last":0.629},{"interface":"front","average":{"1min":0.56599999999999995,"5min":0.58399999999999996,"15min":0.60299999999999998},"min":{"1min":0.34399999999999997,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.80700000000000005,"5min":0.89200000000000002,"15min":0.89200000000000002},"last":0.72899999999999998}]},{"osd":3,"last update":"Fri Nov 24 08:00:29 2023","interfaces":[{"interface":"back","average":{"1min":0.55000000000000004,"5min":0.56100000000000005,"15min":0.57699999999999996},"min":{"1min":0.34599999999999997,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.81799999999999995,"5min":0.89700000000000002,"15min":0.89700000000000002},"last":0.74199999999999999},{"interface":"front","average":{"1min":0.61599999999999999,"5min":0.60799999999999998,"15min":0.623},"min":{"1min":0.47699999999999998,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.86399999999999999,"5min":0.96199999999999997,"15min":0.96199999999999997},"last":0.61199999999999999}]},{"osd":5,"last update":"Fri Nov 24 08:00:29 2023","interfaces":[{"interface":"back","average":{"1min":0.496,"5min":0.47199999999999998,"15min":0.46000000000000002},"min":{"1min":0.33100000000000002,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":0.66100000000000003,"5min":0.70499999999999996,"15min":0.70499999999999996},"last":0.48399999999999999},{"interface":"front","average":{"1min":0.59299999999999997,"5min":0.59899999999999998,"15min":0.61199999999999999},"min":{"1min":0.42599999999999999,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.79100000000000004,"5min":0.86399999999999999,"15min":0.86399999999999999},"last":0.68100000000000005}]},{"osd":6,"last update":"Fri Nov 24 08:00:29 2023","interfaces":[{"interface":"back","average":{"1min":0.56699999999999995,"5min":0.56899999999999995,"15min":0.57999999999999996},"min":{"1min":0.318,"5min":0.318,"15min":0.318},"max":{"1min":0.71299999999999997,"5min":0.88600000000000001,"15min":0.88600000000000001},"last":0.50800000000000001},{"interface":"front","average":{"1min":0.58999999999999997,"5min":0.59499999999999997,"15min":0.60699999999999998},"min":{"1min":0.41099999999999998,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.77200000000000002,"5min":1.095,"15min":1.095},"last":0.64600000000000002}]},{"osd":7,"last update":"Fri Nov 24 08:00:29 2023","interfaces":[{"interface":"back","average":{"1min":0.54100000000000004,"5min":0.55500000000000005,"15min":0.56699999999999995},"min":{"1min":0.38800000000000001,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.73099999999999998,"5min":0.89300000000000002,"15min":0.89300000000000002},"last":0.57499999999999996},{"interface":"front","average":{"1min":0.59499999999999997,"5min":0.623,"15min":0.64900000000000002},"min":{"1min":0.371,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.75600000000000001,"5min":0.91500000000000004,"15min":0.91500000000000004},"last":0.66500000000000004}]}]},{"osd":5,"up_from":13,"seq":55834574907,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32844,"kb_used_data":1668,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338996,"statfs":{"total":96636764160,"available":96603131904,"internally_reserved":0,"allocated":1708032,"data_stored":1067132,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 08:00:25 2023","interfaces":[{"interface":"back","average":{"1min":0.54700000000000004,"5min":0.54300000000000004,"15min":0.55500000000000005},"min":{"1min":0.29399999999999998,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":0.747,"5min":1.0109999999999999,"15min":1.0109999999999999},"last":0.64000000000000001},{"interface":"front","average":{"1min":0.54100000000000004,"5min":0.64100000000000001,"15min":0.747},"min":{"1min":0.35999999999999999,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.78300000000000003,"5min":5.431,"15min":5.431},"last":0.53000000000000003}]},{"osd":1,"last update":"Fri Nov 24 08:00:25 2023","interfaces":[{"interface":"back","average":{"1min":0.53900000000000003,"5min":0.53800000000000003,"15min":0.55200000000000005},"min":{"1min":0.32400000000000001,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":0.73399999999999999,"5min":1.016,"15min":1.016},"last":0.55600000000000005},{"interface":"front","average":{"1min":0.54500000000000004,"5min":0.55900000000000005,"15min":0.56599999999999995},"min":{"1min":0.36799999999999999,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":0.747,"5min":0.96499999999999997,"15min":0.96499999999999997},"last":0.58999999999999997}]},{"osd":2,"last update":"Fri Nov 24 08:00:25 2023","interfaces":[{"interface":"back","average":{"1min":0.81899999999999995,"5min":0.60899999999999999,"15min":0.56999999999999995},"min":{"1min":0.29699999999999999,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":6.024,"5min":6.024,"15min":6.024},"last":0.67400000000000004},{"interface":"front","average":{"1min":0.59399999999999997,"5min":0.59099999999999997,"15min":0.61699999999999999},"min":{"1min":0.34200000000000003,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.82199999999999995,"5min":1.119,"15min":1.119},"last":0.61499999999999999}]},{"osd":3,"last update":"Fri Nov 24 08:00:25 2023","interfaces":[{"interface":"back","average":{"1min":0.56999999999999995,"5min":0.70999999999999996,"15min":0.83999999999999997},"min":{"1min":0.33700000000000002,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.753,"5min":5.609,"15min":5.609},"last":0.68999999999999995},{"interface":"front","average":{"1min":0.79800000000000004,"5min":0.68400000000000005,"15min":0.67800000000000005},"min":{"1min":0.35999999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":4.2439999999999998,"5min":4.2439999999999998,"15min":4.2439999999999998},"last":0.47999999999999998}]},{"osd":4,"last update":"Fri Nov 24 08:00:25 2023","interfaces":[{"interface":"back","average":{"1min":0.53000000000000003,"5min":0.52000000000000002,"15min":0.51300000000000001},"min":{"1min":0.31900000000000001,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":1.145,"5min":1.145,"15min":1.145},"last":0.45600000000000002},{"interface":"front","average":{"1min":0.54900000000000004,"5min":0.57399999999999995,"15min":0.57199999999999995},"min":{"1min":0.25900000000000001,"5min":0.25900000000000001,"15min":0.25900000000000001},"max":{"1min":0.93700000000000006,"5min":0.93700000000000006,"15min":0.93700000000000006},"last":0.503}]},{"osd":6,"last update":"Fri Nov 24 08:00:25 2023","interfaces":[{"interface":"back","average":{"1min":0.628,"5min":0.59899999999999998,"15min":0.57999999999999996},"min":{"1min":0.27900000000000003,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":2.2109999999999999,"5min":2.2109999999999999,"15min":2.2109999999999999},"last":0.57499999999999996},{"interface":"front","average":{"1min":0.69899999999999995,"5min":0.63600000000000001,"15min":0.628},"min":{"1min":0.38,"5min":0.376,"15min":0.376},"max":{"1min":3.391,"5min":3.391,"15min":3.391},"last":0.65800000000000003}]},{"osd":7,"last update":"Fri Nov 24 08:00:25 2023","interfaces":[{"interface":"back","average":{"1min":0.58399999999999996,"5min":0.63800000000000001,"15min":0.64300000000000002},"min":{"1min":0.39200000000000002,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.83999999999999997,"5min":0.96599999999999997,"15min":0.96599999999999997},"last":0.73299999999999998},{"interface":"front","average":{"1min":0.59899999999999998,"5min":0.65000000000000002,"15min":0.65200000000000002},"min":{"1min":0.38500000000000001,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.874,"5min":0.94199999999999995,"15min":0.94199999999999995},"last":0.628}]}]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-11-24T08:00:53.840 DEBUG:teuthology.orchestra.run.smithi059:> sudo ceph --cluster ceph config set global mon_health_to_clog false 2023-11-24T08:00:53.965 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:53.963+0000 7f6a7250f700 1 Processor -- start 2023-11-24T08:00:53.965 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:53.963+0000 7f6a7250f700 1 -- start start 2023-11-24T08:00:53.967 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:53.964+0000 7f6a7250f700 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f6a6c0d0870 0x7f6a6c0d26c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:00:53.967 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:53.964+0000 7f6a7250f700 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f6a6c0d49f0 0x7f6a6c0b6d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:00:53.967 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:53.964+0000 7f6a7250f700 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6a6c0b5370 con 0x7f6a6c0d0c50 2023-11-24T08:00:53.967 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:53.964+0000 7f6a7250f700 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6a6c0d2f80 con 0x7f6a6c0d0870 2023-11-24T08:00:53.968 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:53.964+0000 7f6a7250f700 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6a6c0cff70 con 0x7f6a6c0d49f0 2023-11-24T08:00:53.968 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:53.964+0000 7f6a6bfff700 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f6a6c0d0870 0x7f6a6c0d26c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:00:53.968 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:53.964+0000 7f6a70aac700 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f6a6c0d49f0 0x7f6a6c0b6d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:00:53.968 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:53.964+0000 7f6a6b7fe700 1 --1- >> v1:172.21.15.76:6789/0 conn(0x7f6a6c0d0c50 0x7f6a6c0cf860 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.59:47630/0 (socket says 172.21.15.59:47630) 2023-11-24T08:00:53.968 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:53.964+0000 7f6a6b7fe700 1 -- 172.21.15.59:0/3224248771 learned_addr learned my addr 172.21.15.59:0/3224248771 (peer_addr_for_me v1:172.21.15.59:0/0) 2023-11-24T08:00:53.968 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:53.964+0000 7f6a70aac700 1 -- 172.21.15.59:0/3224248771 >> v2:172.21.15.76:3300/0 conn(0x7f6a6c0d0870 msgr2=0x7f6a6c0d26c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:00:53.968 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:53.964+0000 7f6a70aac700 1 --2- 172.21.15.59:0/3224248771 >> v2:172.21.15.76:3300/0 conn(0x7f6a6c0d0870 0x7f6a6c0d26c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:00:53.969 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:53.964+0000 7f6a70aac700 1 -- 172.21.15.59:0/3224248771 >> v1:172.21.15.76:6789/0 conn(0x7f6a6c0d0c50 legacy=0x7f6a6c0cf860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:00:53.969 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:53.965+0000 7f6a70aac700 1 -- 172.21.15.59:0/3224248771 --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6a6c0ceae0 con 0x7f6a6c0d49f0 2023-11-24T08:00:53.969 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:53.965+0000 7f6a6bfff700 1 --2- 172.21.15.59:0/3224248771 >> v2:172.21.15.76:3300/0 conn(0x7f6a6c0d0870 0x7f6a6c0d26c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-24T08:00:53.969 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:53.965+0000 7f6a6affd700 1 -- 172.21.15.59:0/3224248771 <== mon.1 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3817087052 0 0) 0x7f6a6c0b5370 con 0x7f6a6c0d0c50 2023-11-24T08:00:53.969 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:53.965+0000 7f6a70aac700 1 --2- 172.21.15.59:0/3224248771 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f6a6c0d49f0 0x7f6a6c0b6d20 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f6a64009d90 tx=0x7f6a6400b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=12211f85b0b4c156 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:00:53.969 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:53.965+0000 7f6a6affd700 1 -- 172.21.15.59:0/3224248771 <== mon.2 v2:172.21.15.76:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a6400e070 con 0x7f6a6c0d49f0 2023-11-24T08:00:53.969 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:53.965+0000 7f6a6affd700 1 -- 172.21.15.59:0/3224248771 <== mon.2 v2:172.21.15.76:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6a6400bb20 con 0x7f6a6c0d49f0 2023-11-24T08:00:53.969 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:53.965+0000 7f6a6affd700 1 -- 172.21.15.59:0/3224248771 <== mon.2 v2:172.21.15.76:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a64012430 con 0x7f6a6c0d49f0 2023-11-24T08:00:53.970 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:53.965+0000 7f6a7250f700 1 -- 172.21.15.59:0/3224248771 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f6a6c0d49f0 msgr2=0x7f6a6c0b6d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:00:53.970 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:53.965+0000 7f6a7250f700 1 --2- 172.21.15.59:0/3224248771 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f6a6c0d49f0 0x7f6a6c0b6d20 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f6a64009d90 tx=0x7f6a6400b330 comp rx=0 tx=0).stop 2023-11-24T08:00:53.970 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:53.966+0000 7f6a7250f700 1 -- 172.21.15.59:0/3224248771 shutdown_connections 2023-11-24T08:00:53.970 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:53.966+0000 7f6a7250f700 1 --2- 172.21.15.59:0/3224248771 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f6a6c0d49f0 0x7f6a6c0b6d20 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:00:53.970 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:53.966+0000 7f6a7250f700 1 --2- 172.21.15.59:0/3224248771 >> v2:172.21.15.76:3300/0 conn(0x7f6a6c0d0870 0x7f6a6c0d26c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:00:53.970 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:53.966+0000 7f6a7250f700 1 -- 172.21.15.59:0/3224248771 >> 172.21.15.59:0/3224248771 conn(0x7f6a6c0d3ed0 msgr2=0x7f6a6c0d42e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:00:53.970 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:53.966+0000 7f6a7250f700 1 -- 172.21.15.59:0/3224248771 shutdown_connections 2023-11-24T08:00:53.970 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:53.966+0000 7f6a7250f700 1 -- 172.21.15.59:0/3224248771 wait complete. 2023-11-24T08:00:53.973 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:53.967+0000 7f6a7250f700 1 Processor -- start 2023-11-24T08:00:53.973 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:53.967+0000 7f6a7250f700 1 -- start start 2023-11-24T08:00:53.973 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:53.968+0000 7f6a7250f700 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f6a6c0d0870 0x7f6a6c0b8fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:00:53.974 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:53.968+0000 7f6a7250f700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6a6c0d0c50 0x7f6a6c0b3120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:00:53.974 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:53.968+0000 7f6a7250f700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6a6c0d49f0 0x7f6a6c0b3c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:00:53.974 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:53.968+0000 7f6a7250f700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a6c0b2c90 con 0x7f6a6c0d0c50 2023-11-24T08:00:53.974 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:53.968+0000 7f6a7250f700 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a6c0b2080 con 0x7f6a6c0d49f0 2023-11-24T08:00:53.974 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:53.968+0000 7f6a7250f700 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6a6c0b2360 con 0x7f6a6c0d0870 2023-11-24T08:00:53.974 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:53.968+0000 7f6a6b7fe700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6a6c0d0c50 0x7f6a6c0b3120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:00:53.974 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:53.968+0000 7f6a6b7fe700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6a6c0d0c50 0x7f6a6c0b3120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:56886/0 (socket says 172.21.15.59:56886) 2023-11-24T08:00:53.975 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:53.968+0000 7f6a6b7fe700 1 -- 172.21.15.59:0/4063772835 learned_addr learned my addr 172.21.15.59:0/4063772835 (peer_addr_for_me v2:172.21.15.59:0/0) 2023-11-24T08:00:53.975 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:53.968+0000 7f6a6bfff700 1 --2- 172.21.15.59:0/4063772835 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f6a6c0d0870 0x7f6a6c0b8fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:00:53.975 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:53.968+0000 7f6a70aac700 1 --2- 172.21.15.59:0/4063772835 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6a6c0d49f0 0x7f6a6c0b3c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:00:53.975 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:53.968+0000 7f6a6b7fe700 1 -- 172.21.15.59:0/4063772835 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f6a6c0d0870 msgr2=0x7f6a6c0b8fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:00:53.975 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:53.968+0000 7f6a6b7fe700 1 --2- 172.21.15.59:0/4063772835 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f6a6c0d0870 0x7f6a6c0b8fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:00:53.975 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:53.968+0000 7f6a6b7fe700 1 -- 172.21.15.59:0/4063772835 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6a6c0d49f0 msgr2=0x7f6a6c0b3c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:00:53.975 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:53.968+0000 7f6a6b7fe700 1 --2- 172.21.15.59:0/4063772835 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6a6c0d49f0 0x7f6a6c0b3c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:00:53.975 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:53.968+0000 7f6a6b7fe700 1 -- 172.21.15.59:0/4063772835 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6a5c0094a0 con 0x7f6a6c0d0c50 2023-11-24T08:00:53.976 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:53.968+0000 7f6a6bfff700 1 --2- 172.21.15.59:0/4063772835 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f6a6c0d0870 0x7f6a6c0b8fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-24T08:00:53.976 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:53.968+0000 7f6a6b7fe700 1 --2- 172.21.15.59:0/4063772835 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6a6c0d0c50 0x7f6a6c0b3120 secure :-1 s=READY pgs=1186 cs=0 l=1 rev1=1 crypto rx=0x7f6a6000c540 tx=0x7f6a6000c850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:00:53.976 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:53.968+0000 7f6a68ff9700 1 -- 172.21.15.59:0/4063772835 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a60004c80 con 0x7f6a6c0d0c50 2023-11-24T08:00:53.976 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:53.968+0000 7f6a68ff9700 1 -- 172.21.15.59:0/4063772835 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6a60004de0 con 0x7f6a6c0d0c50 2023-11-24T08:00:53.976 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:53.968+0000 7f6a7250f700 1 -- 172.21.15.59:0/4063772835 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6a64009a70 con 0x7f6a6c0d0c50 2023-11-24T08:00:53.976 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:53.968+0000 7f6a68ff9700 1 -- 172.21.15.59:0/4063772835 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a60007310 con 0x7f6a6c0d0c50 2023-11-24T08:00:53.976 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:53.969+0000 7f6a7250f700 1 -- 172.21.15.59:0/4063772835 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6a6c0b2920 con 0x7f6a6c0d0c50 2023-11-24T08:00:53.976 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:53.969+0000 7f6a7250f700 1 -- 172.21.15.59:0/4063772835 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6a6c0b57c0 con 0x7f6a6c0d0c50 2023-11-24T08:00:53.977 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:53.970+0000 7f6a68ff9700 1 -- 172.21.15.59:0/4063772835 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 12) v1 ==== 89792+0+0 (secure 0 0 0) 0x7f6a60007470 con 0x7f6a6c0d0c50 2023-11-24T08:00:53.977 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:53.970+0000 7f6a68ff9700 1 --2- 172.21.15.59:0/4063772835 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f6a5806b500 0x7f6a5806d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:00:53.977 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:53.971+0000 7f6a68ff9700 1 -- 172.21.15.59:0/4063772835 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f6a60089ae0 con 0x7f6a6c0d0c50 2023-11-24T08:00:53.977 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:53.971+0000 7f6a6bfff700 1 --2- 172.21.15.59:0/4063772835 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f6a5806b500 0x7f6a5806d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:00:53.977 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:53.971+0000 7f6a6bfff700 1 --2- 172.21.15.59:0/4063772835 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f6a5806b500 0x7f6a5806d9c0 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7f6a5c0114b0 tx=0x7f6a5c009210 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:00:53.978 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:53.975+0000 7f6a68ff9700 1 -- 172.21.15.59:0/4063772835 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f6a60054550 con 0x7f6a6c0d0c50 2023-11-24T08:00:54.180 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:54.179+0000 7f6a7250f700 1 -- 172.21.15.59:0/4063772835 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command([{prefix=config set, name=mon_health_to_clog}] v 0) v1 -- 0x7f6a6c067a20 con 0x7f6a6c0d0c50 2023-11-24T08:00:54.183 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:54.182+0000 7f6a68ff9700 1 -- 172.21.15.59:0/4063772835 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_health_to_clog}]=0 v20)=0 v20) v1 ==== 123+0+0 (secure 0 0 0) 0x7f6a60057b70 con 0x7f6a6c0d0c50 2023-11-24T08:00:54.185 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:54.183+0000 7f6a68ff9700 1 -- 172.21.15.59:0/4063772835 <== mon.0 v2:172.21.15.59:3300/0 8 ==== config(1 keys) v1 ==== 35+0+0 (secure 0 0 0) 0x7f6a600878a0 con 0x7f6a6c0d0c50 2023-11-24T08:00:54.187 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:54.185+0000 7f6a7250f700 1 -- 172.21.15.59:0/4063772835 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f6a5806b500 msgr2=0x7f6a5806d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:00:54.187 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:54.185+0000 7f6a7250f700 1 --2- 172.21.15.59:0/4063772835 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f6a5806b500 0x7f6a5806d9c0 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7f6a5c0114b0 tx=0x7f6a5c009210 comp rx=0 tx=0).stop 2023-11-24T08:00:54.187 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:54.185+0000 7f6a7250f700 1 -- 172.21.15.59:0/4063772835 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6a6c0d0c50 msgr2=0x7f6a6c0b3120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:00:54.187 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:54.185+0000 7f6a7250f700 1 --2- 172.21.15.59:0/4063772835 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6a6c0d0c50 0x7f6a6c0b3120 secure :-1 s=READY pgs=1186 cs=0 l=1 rev1=1 crypto rx=0x7f6a6000c540 tx=0x7f6a6000c850 comp rx=0 tx=0).stop 2023-11-24T08:00:54.187 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:54.185+0000 7f6a7250f700 1 -- 172.21.15.59:0/4063772835 shutdown_connections 2023-11-24T08:00:54.187 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:54.185+0000 7f6a7250f700 1 --2- 172.21.15.59:0/4063772835 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7f6a5806b500 0x7f6a5806d9c0 unknown :-1 s=CLOSED pgs=538 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:00:54.187 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:54.185+0000 7f6a7250f700 1 --2- 172.21.15.59:0/4063772835 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6a6c0d49f0 0x7f6a6c0b3c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:00:54.188 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:54.185+0000 7f6a7250f700 1 --2- 172.21.15.59:0/4063772835 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7f6a6c0d0870 0x7f6a6c0b8fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:00:54.188 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:54.185+0000 7f6a7250f700 1 --2- 172.21.15.59:0/4063772835 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6a6c0d0c50 0x7f6a6c0b3120 unknown :-1 s=CLOSED pgs=1186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:00:54.188 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:54.185+0000 7f6a7250f700 1 -- 172.21.15.59:0/4063772835 >> 172.21.15.59:0/4063772835 conn(0x7f6a6c0d3ed0 msgr2=0x7f6a6c0ccd10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:00:54.188 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:54.185+0000 7f6a7250f700 1 -- 172.21.15.59:0/4063772835 shutdown_connections 2023-11-24T08:00:54.188 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:54.185+0000 7f6a7250f700 1 -- 172.21.15.59:0/4063772835 wait complete. 2023-11-24T08:00:54.198 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2023-11-24T08:00:54.199 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-24T08:00:54.323 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:54.321+0000 7fd919a4a700 1 Processor -- start 2023-11-24T08:00:54.323 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:54.321+0000 7fd919a4a700 1 -- start start 2023-11-24T08:00:54.326 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:54.322+0000 7fd919a4a700 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fd91415e460 0x7fd9140d49d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:00:54.326 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:54.322+0000 7fd919a4a700 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fd9140d2ba0 0x7fd9140b40a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:00:54.326 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:54.322+0000 7fd919a4a700 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd9140b3370 con 0x7fd9140cf480 2023-11-24T08:00:54.326 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:54.322+0000 7fd919a4a700 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd9140cfc40 con 0x7fd91415e460 2023-11-24T08:00:54.326 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:54.322+0000 7fd919a4a700 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd9140b35c0 con 0x7fd9140d2ba0 2023-11-24T08:00:54.326 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:54.322+0000 7fd912ffd700 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fd91415e460 0x7fd9140d49d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:00:54.326 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:54.322+0000 7fd9137fe700 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fd9140d2ba0 0x7fd9140b40a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:00:54.327 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:54.322+0000 7fd9127fc700 1 --1- >> v1:172.21.15.76:6789/0 conn(0x7fd9140cf480 0x7fd9140d2490 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.59:47644/0 (socket says 172.21.15.59:47644) 2023-11-24T08:00:54.327 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:54.322+0000 7fd912ffd700 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fd91415e460 0x7fd9140d49d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.59:44090/0 (socket says 172.21.15.59:44090) 2023-11-24T08:00:54.327 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:54.322+0000 7fd9127fc700 1 -- 172.21.15.59:0/2800867672 learned_addr learned my addr 172.21.15.59:0/2800867672 (peer_addr_for_me v1:172.21.15.59:0/0) 2023-11-24T08:00:54.327 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:54.323+0000 7fd912ffd700 1 -- 172.21.15.59:0/2800867672 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fd9140d2ba0 msgr2=0x7fd9140b40a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:00:54.327 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:54.323+0000 7fd912ffd700 1 --2- 172.21.15.59:0/2800867672 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fd9140d2ba0 0x7fd9140b40a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:00:54.327 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:54.323+0000 7fd912ffd700 1 -- 172.21.15.59:0/2800867672 >> v1:172.21.15.76:6789/0 conn(0x7fd9140cf480 legacy=0x7fd9140d2490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:00:54.327 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:54.323+0000 7fd912ffd700 1 -- 172.21.15.59:0/2800867672 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd9140b38a0 con 0x7fd91415e460 2023-11-24T08:00:54.328 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:54.323+0000 7fd911ffb700 1 -- 172.21.15.59:0/2800867672 <== mon.1 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3339741002 0 0) 0x7fd9140b3370 con 0x7fd9140cf480 2023-11-24T08:00:54.328 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:54.323+0000 7fd9137fe700 1 --2- 172.21.15.59:0/2800867672 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fd9140d2ba0 0x7fd9140b40a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-24T08:00:54.328 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:54.323+0000 7fd912ffd700 1 --2- 172.21.15.59:0/2800867672 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd91415e460 0x7fd9140d49d0 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7fd908009840 tx=0x7fd908009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=796948bff8ab0c48 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:00:54.328 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:54.323+0000 7fd911ffb700 1 -- 172.21.15.59:0/2800867672 <== mon.1 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd908011830 con 0x7fd91415e460 2023-11-24T08:00:54.328 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:54.323+0000 7fd911ffb700 1 -- 172.21.15.59:0/2800867672 <== mon.1 v2:172.21.15.76:3300/0 2 ==== config(1 keys) v1 ==== 35+0+0 (secure 0 0 0) 0x7fd908011990 con 0x7fd91415e460 2023-11-24T08:00:54.328 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:54.323+0000 7fd911ffb700 1 -- 172.21.15.59:0/2800867672 <== mon.1 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd908011d40 con 0x7fd91415e460 2023-11-24T08:00:54.328 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:54.324+0000 7fd919a4a700 1 -- 172.21.15.59:0/2800867672 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd91415e460 msgr2=0x7fd9140d49d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:00:54.328 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:54.324+0000 7fd919a4a700 1 --2- 172.21.15.59:0/2800867672 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd91415e460 0x7fd9140d49d0 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7fd908009840 tx=0x7fd908009b50 comp rx=0 tx=0).stop 2023-11-24T08:00:54.328 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:54.324+0000 7fd919a4a700 1 -- 172.21.15.59:0/2800867672 shutdown_connections 2023-11-24T08:00:54.329 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:54.324+0000 7fd919a4a700 1 --2- 172.21.15.59:0/2800867672 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fd9140d2ba0 0x7fd9140b40a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:00:54.329 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:54.324+0000 7fd919a4a700 1 --2- 172.21.15.59:0/2800867672 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd91415e460 0x7fd9140d49d0 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:00:54.329 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:54.324+0000 7fd919a4a700 1 -- 172.21.15.59:0/2800867672 >> 172.21.15.59:0/2800867672 conn(0x7fd9140d3eb0 msgr2=0x7fd9140d42c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:00:54.329 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:54.324+0000 7fd919a4a700 1 -- 172.21.15.59:0/2800867672 shutdown_connections 2023-11-24T08:00:54.329 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:54.324+0000 7fd919a4a700 1 -- 172.21.15.59:0/2800867672 wait complete. 2023-11-24T08:00:54.332 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:54.325+0000 7fd919a4a700 1 Processor -- start 2023-11-24T08:00:54.332 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:54.326+0000 7fd919a4a700 1 -- start start 2023-11-24T08:00:54.332 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:54.326+0000 7fd919a4a700 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd9140cf480 0x7fd9140ca620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:00:54.332 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:54.326+0000 7fd919a4a700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd9140d2ba0 0x7fd9140cdb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:00:54.332 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:54.326+0000 7fd919a4a700 1 --2- >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fd91415e460 0x7fd9140cc2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:00:54.332 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:54.326+0000 7fd919a4a700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9140cd6f0 con 0x7fd9140d2ba0 2023-11-24T08:00:54.333 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:54.326+0000 7fd919a4a700 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9140cc810 con 0x7fd9140cf480 2023-11-24T08:00:54.333 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:54.326+0000 7fd919a4a700 1 -- --> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd9140cca70 con 0x7fd91415e460 2023-11-24T08:00:54.333 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:54.327+0000 7fd9127fc700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd9140d2ba0 0x7fd9140cdb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:00:54.333 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:54.327+0000 7fd9127fc700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd9140d2ba0 0x7fd9140cdb10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:56898/0 (socket says 172.21.15.59:56898) 2023-11-24T08:00:54.333 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:54.327+0000 7fd9127fc700 1 -- 172.21.15.59:0/3088028642 learned_addr learned my addr 172.21.15.59:0/3088028642 (peer_addr_for_me v2:172.21.15.59:0/0) 2023-11-24T08:00:54.333 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:54.327+0000 7fd9137fe700 1 --2- 172.21.15.59:0/3088028642 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fd91415e460 0x7fd9140cc2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:00:54.333 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:54.327+0000 7fd912ffd700 1 --2- 172.21.15.59:0/3088028642 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd9140cf480 0x7fd9140ca620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:00:54.333 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:54.327+0000 7fd9127fc700 1 -- 172.21.15.59:0/3088028642 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fd91415e460 msgr2=0x7fd9140cc2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:00:54.334 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:54.327+0000 7fd9127fc700 1 --2- 172.21.15.59:0/3088028642 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fd91415e460 0x7fd9140cc2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:00:54.334 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:54.327+0000 7fd9127fc700 1 -- 172.21.15.59:0/3088028642 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd9140cf480 msgr2=0x7fd9140ca620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:00:54.334 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:54.327+0000 7fd9127fc700 1 --2- 172.21.15.59:0/3088028642 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd9140cf480 0x7fd9140ca620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:00:54.334 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:54.327+0000 7fd9127fc700 1 -- 172.21.15.59:0/3088028642 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd90c009a70 con 0x7fd9140d2ba0 2023-11-24T08:00:54.334 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:54.327+0000 7fd9127fc700 1 --2- 172.21.15.59:0/3088028642 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd9140d2ba0 0x7fd9140cdb10 secure :-1 s=READY pgs=1187 cs=0 l=1 rev1=1 crypto rx=0x7fd9040011e0 tx=0x7fd90400ca30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:00:54.334 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:54.327+0000 7fd8fbfff700 1 -- 172.21.15.59:0/3088028642 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd904004c80 con 0x7fd9140d2ba0 2023-11-24T08:00:54.334 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:54.327+0000 7fd8fbfff700 1 -- 172.21.15.59:0/3088028642 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(1 keys) v1 ==== 35+0+0 (secure 0 0 0) 0x7fd904004de0 con 0x7fd9140d2ba0 2023-11-24T08:00:54.335 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:54.327+0000 7fd919a4a700 1 -- 172.21.15.59:0/3088028642 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd9080094a0 con 0x7fd9140d2ba0 2023-11-24T08:00:54.335 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:54.327+0000 7fd8fbfff700 1 -- 172.21.15.59:0/3088028642 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd904007330 con 0x7fd9140d2ba0 2023-11-24T08:00:54.335 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:54.327+0000 7fd919a4a700 1 -- 172.21.15.59:0/3088028642 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd9140cd030 con 0x7fd9140d2ba0 2023-11-24T08:00:54.335 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:54.328+0000 7fd919a4a700 1 -- 172.21.15.59:0/3088028642 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd9140cd830 con 0x7fd9140d2ba0 2023-11-24T08:00:54.335 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:54.329+0000 7fd8fbfff700 1 -- 172.21.15.59:0/3088028642 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 12) v1 ==== 89792+0+0 (secure 0 0 0) 0x7fd904007490 con 0x7fd9140d2ba0 2023-11-24T08:00:54.335 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:54.329+0000 7fd8fbfff700 1 --2- 172.21.15.59:0/3088028642 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7fd90006b500 0x7fd90006d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:00:54.335 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:54.329+0000 7fd8fbfff700 1 -- 172.21.15.59:0/3088028642 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 4943+0+0 (secure 0 0 0) 0x7fd904089b90 con 0x7fd9140d2ba0 2023-11-24T08:00:54.336 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:54.329+0000 7fd912ffd700 1 --2- 172.21.15.59:0/3088028642 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7fd90006b500 0x7fd90006d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:00:54.336 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:54.330+0000 7fd912ffd700 1 --2- 172.21.15.59:0/3088028642 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7fd90006b500 0x7fd90006d9c0 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7fd9080090b0 tx=0x7fd90801c040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:00:54.336 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:54.334+0000 7fd8fbfff700 1 -- 172.21.15.59:0/3088028642 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fd9140cd830 con 0x7fd9140d2ba0 2023-11-24T08:00:54.584 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:54.582+0000 7fd919a4a700 1 -- 172.21.15.59:0/3088028642 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fd9140cd830 con 0x7fd9140d2ba0 2023-11-24T08:00:54.585 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:54.583+0000 7fd8fbfff700 1 -- 172.21.15.59:0/3088028642 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 56 v56) v1 ==== 94+0+3696 (secure 0 0 0) 0x7fd9140cd830 con 0x7fd9140d2ba0 2023-11-24T08:00:54.585 INFO:teuthology.orchestra.run.smithi059.stdout: 2023-11-24T08:00:54.585 INFO:teuthology.orchestra.run.smithi059.stdout:{"epoch":56,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5876,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.76:6837/1132179319","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6835","nonce":1132179319},{"type":"v1","addr":"172.21.15.76:6837","nonce":1132179319}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":48},{"gid":6487,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.76:6836/453627414","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6834","nonce":453627414},{"type":"v1","addr":"172.21.15.76:6836","nonce":453627414}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":48},{"gid":7035,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6834/4160100816","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6832","nonce":4160100816},{"type":"v1","addr":"172.21.15.59:6834","nonce":4160100816}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":48},{"gid":7377,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6835/1552435133","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6833","nonce":1552435133},{"type":"v1","addr":"172.21.15.59:6835","nonce":1552435133}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":56}],"filesystems":[]} 2023-11-24T08:00:54.587 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:54.585+0000 7fd919a4a700 1 -- 172.21.15.59:0/3088028642 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7fd90006b500 msgr2=0x7fd90006d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:00:54.587 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:54.585+0000 7fd919a4a700 1 --2- 172.21.15.59:0/3088028642 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7fd90006b500 0x7fd90006d9c0 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7fd9080090b0 tx=0x7fd90801c040 comp rx=0 tx=0).stop 2023-11-24T08:00:54.587 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:54.585+0000 7fd919a4a700 1 -- 172.21.15.59:0/3088028642 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd9140d2ba0 msgr2=0x7fd9140cdb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:00:54.587 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:54.585+0000 7fd919a4a700 1 --2- 172.21.15.59:0/3088028642 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd9140d2ba0 0x7fd9140cdb10 secure :-1 s=READY pgs=1187 cs=0 l=1 rev1=1 crypto rx=0x7fd9040011e0 tx=0x7fd90400ca30 comp rx=0 tx=0).stop 2023-11-24T08:00:54.587 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:54.585+0000 7fd919a4a700 1 -- 172.21.15.59:0/3088028642 shutdown_connections 2023-11-24T08:00:54.587 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:54.585+0000 7fd919a4a700 1 --2- 172.21.15.59:0/3088028642 >> [v2:172.21.15.76:6832/4264046536,v1:172.21.15.76:6833/4264046536] conn(0x7fd90006b500 0x7fd90006d9c0 unknown :-1 s=CLOSED pgs=539 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:00:54.588 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:54.585+0000 7fd919a4a700 1 --2- 172.21.15.59:0/3088028642 >> [v2:172.21.15.76:3301/0,v1:172.21.15.76:6790/0] conn(0x7fd91415e460 0x7fd9140cc2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:00:54.588 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:54.585+0000 7fd919a4a700 1 --2- 172.21.15.59:0/3088028642 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd9140cf480 0x7fd9140ca620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:00:54.588 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:54.585+0000 7fd919a4a700 1 --2- 172.21.15.59:0/3088028642 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd9140d2ba0 0x7fd9140cdb10 unknown :-1 s=CLOSED pgs=1187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:00:54.588 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:54.585+0000 7fd919a4a700 1 -- 172.21.15.59:0/3088028642 >> 172.21.15.59:0/3088028642 conn(0x7fd9140d3eb0 msgr2=0x7fd91406bba0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:00:54.588 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:54.585+0000 7fd919a4a700 1 -- 172.21.15.59:0/3088028642 shutdown_connections 2023-11-24T08:00:54.588 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-24T08:00:54.585+0000 7fd919a4a700 1 -- 172.21.15.59:0/3088028642 wait complete. 2023-11-24T08:00:54.589 INFO:teuthology.orchestra.run.smithi059.stderr:dumped fsmap epoch 56 2023-11-24T08:00:54.599 DEBUG:tasks.cephfs.filesystem:already dead... 2023-11-24T08:00:54.600 INFO:teuthology.misc:Shutting down mds daemons... 2023-11-24T08:00:54.600 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2023-11-24T08:00:54.600 INFO:teuthology.orchestra.run:waiting for 300 2023-11-24T08:00:54.609 INFO:tasks.ceph.mds.a:Stopped 2023-11-24T08:00:54.609 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2023-11-24T08:00:54.609 INFO:teuthology.orchestra.run:waiting for 300 2023-11-24T08:00:54.658 INFO:tasks.ceph.mds.c:Stopped 2023-11-24T08:00:54.659 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2023-11-24T08:00:54.659 INFO:teuthology.orchestra.run:waiting for 300 2023-11-24T08:00:54.670 INFO:tasks.ceph.mds.b:Stopped 2023-11-24T08:00:54.670 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2023-11-24T08:00:54.670 INFO:teuthology.orchestra.run:waiting for 300 2023-11-24T08:00:54.721 INFO:tasks.ceph.mds.d:Stopped 2023-11-24T08:00:54.722 INFO:teuthology.misc:Shutting down osd daemons... 2023-11-24T08:00:54.722 DEBUG:tasks.ceph.osd.0:waiting for process to exit 2023-11-24T08:00:54.722 INFO:teuthology.orchestra.run:waiting for 300 2023-11-24T08:00:54.763 INFO:tasks.ceph.osd.0:Stopped 2023-11-24T08:00:54.764 DEBUG:tasks.ceph.osd.1:waiting for process to exit 2023-11-24T08:00:54.764 INFO:teuthology.orchestra.run:waiting for 300 2023-11-24T08:00:54.807 INFO:tasks.ceph.osd.1:Stopped 2023-11-24T08:00:54.807 DEBUG:tasks.ceph.osd.2:waiting for process to exit 2023-11-24T08:00:54.807 INFO:teuthology.orchestra.run:waiting for 300 2023-11-24T08:00:54.844 INFO:tasks.ceph.osd.2:Stopped 2023-11-24T08:00:54.845 DEBUG:tasks.ceph.osd.3:waiting for process to exit 2023-11-24T08:00:54.845 INFO:teuthology.orchestra.run:waiting for 300 2023-11-24T08:00:54.916 INFO:tasks.ceph.osd.3:Stopped 2023-11-24T08:00:54.917 DEBUG:tasks.ceph.osd.4:waiting for process to exit 2023-11-24T08:00:54.917 INFO:teuthology.orchestra.run:waiting for 300 2023-11-24T08:00:54.956 INFO:tasks.ceph.osd.4:Stopped 2023-11-24T08:00:54.956 DEBUG:tasks.ceph.osd.5:waiting for process to exit 2023-11-24T08:00:54.956 INFO:teuthology.orchestra.run:waiting for 300 2023-11-24T08:00:55.028 INFO:tasks.ceph.osd.5:Stopped 2023-11-24T08:00:55.029 DEBUG:tasks.ceph.osd.6:waiting for process to exit 2023-11-24T08:00:55.029 INFO:teuthology.orchestra.run:waiting for 300 2023-11-24T08:00:55.102 INFO:tasks.ceph.osd.6:Stopped 2023-11-24T08:00:55.102 DEBUG:tasks.ceph.osd.7:waiting for process to exit 2023-11-24T08:00:55.102 INFO:teuthology.orchestra.run:waiting for 300 2023-11-24T08:00:55.181 INFO:tasks.ceph.osd.7:Stopped 2023-11-24T08:00:55.182 INFO:teuthology.misc:Shutting down mgr daemons... 2023-11-24T08:00:55.182 DEBUG:tasks.ceph.mgr.y:waiting for process to exit 2023-11-24T08:00:55.182 INFO:teuthology.orchestra.run:waiting for 300 2023-11-24T08:00:55.207 INFO:tasks.ceph.mgr.y:Stopped 2023-11-24T08:00:55.207 DEBUG:tasks.ceph.mgr.x:waiting for process to exit 2023-11-24T08:00:55.207 INFO:teuthology.orchestra.run:waiting for 300 2023-11-24T08:00:55.249 INFO:tasks.ceph.mgr.x:Stopped 2023-11-24T08:00:55.250 INFO:teuthology.misc:Shutting down mon daemons... 2023-11-24T08:00:55.250 DEBUG:tasks.ceph.mon.a:waiting for process to exit 2023-11-24T08:00:55.250 INFO:teuthology.orchestra.run:waiting for 300 2023-11-24T08:00:55.262 INFO:tasks.ceph.mon.a:Stopped 2023-11-24T08:00:55.262 DEBUG:tasks.ceph.mon.b:waiting for process to exit 2023-11-24T08:00:55.263 INFO:teuthology.orchestra.run:waiting for 300 2023-11-24T08:00:55.301 INFO:tasks.ceph.mon.b:Stopped 2023-11-24T08:00:55.302 DEBUG:tasks.ceph.mon.c:waiting for process to exit 2023-11-24T08:00:55.302 INFO:teuthology.orchestra.run:waiting for 300 2023-11-24T08:00:55.353 INFO:tasks.ceph.mon.c:Stopped 2023-11-24T08:00:55.353 INFO:tasks.ceph:Checking cluster log for badness... 2023-11-24T08:00:55.354 DEBUG:teuthology.orchestra.run.smithi059:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v 'overall HEALTH_' | egrep -v '\(FS_DEGRADED\)' | egrep -v '\(MDS_FAILED\)' | egrep -v '\(MDS_DEGRADED\)' | egrep -v '\(FS_WITH_FAILED_MDS\)' | egrep -v '\(MDS_DAMAGE\)' | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v '\(FS_INLINE_DATA_DEPRECATED\)' | egrep -v '\(POOL_APP_NOT_ENABLED\)' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v 'OSD full dropping all updates' | egrep -v 'OSD near full' | egrep -v 'pausewr flag' | egrep -v 'failsafe engaged, dropping updates' | egrep -v 'failsafe disengaged, no longer dropping' | egrep -v 'is full \(reached quota' | egrep -v POOL_FULL | egrep -v POOL_BACKFILLFULL | head -n 1 2023-11-24T08:00:55.391 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-0 on ubuntu@smithi059.front.sepia.ceph.com 2023-11-24T08:00:55.391 DEBUG:teuthology.orchestra.run.smithi059:> sync && sudo umount -f /var/lib/ceph/osd/ceph-0 2023-11-24T08:00:55.665 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-1 on ubuntu@smithi059.front.sepia.ceph.com 2023-11-24T08:00:55.665 DEBUG:teuthology.orchestra.run.smithi059:> sync && sudo umount -f /var/lib/ceph/osd/ceph-1 2023-11-24T08:00:55.791 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-2 on ubuntu@smithi059.front.sepia.ceph.com 2023-11-24T08:00:55.791 DEBUG:teuthology.orchestra.run.smithi059:> sync && sudo umount -f /var/lib/ceph/osd/ceph-2 2023-11-24T08:00:55.909 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-3 on ubuntu@smithi059.front.sepia.ceph.com 2023-11-24T08:00:55.910 DEBUG:teuthology.orchestra.run.smithi059:> sync && sudo umount -f /var/lib/ceph/osd/ceph-3 2023-11-24T08:00:56.017 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-4 on ubuntu@smithi076.front.sepia.ceph.com 2023-11-24T08:00:56.017 DEBUG:teuthology.orchestra.run.smithi076:> sync && sudo umount -f /var/lib/ceph/osd/ceph-4 2023-11-24T08:00:56.272 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-5 on ubuntu@smithi076.front.sepia.ceph.com 2023-11-24T08:00:56.272 DEBUG:teuthology.orchestra.run.smithi076:> sync && sudo umount -f /var/lib/ceph/osd/ceph-5 2023-11-24T08:00:56.363 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-6 on ubuntu@smithi076.front.sepia.ceph.com 2023-11-24T08:00:56.363 DEBUG:teuthology.orchestra.run.smithi076:> sync && sudo umount -f /var/lib/ceph/osd/ceph-6 2023-11-24T08:00:56.446 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-7 on ubuntu@smithi076.front.sepia.ceph.com 2023-11-24T08:00:56.447 DEBUG:teuthology.orchestra.run.smithi076:> sync && sudo umount -f /var/lib/ceph/osd/ceph-7 2023-11-24T08:00:56.550 INFO:tasks.ceph:Archiving mon data... 2023-11-24T08:00:56.552 DEBUG:teuthology.misc:Transferring archived files from smithi059:/var/lib/ceph/mon/ceph-a to /home/teuthworker/archive/rishabh-2023-11-24_07:31:48-fs:volumes-rishabh-mgr-vol-misc-testing-default-smithi/7465575/data/mon.a.tgz 2023-11-24T08:00:56.552 DEBUG:teuthology.orchestra.run.smithi059:> mktemp 2023-11-24T08:00:56.571 INFO:teuthology.orchestra.run.smithi059.stdout:/tmp/tmp.RWRHuiuA8r 2023-11-24T08:00:56.571 DEBUG:teuthology.orchestra.run.smithi059:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-a -- . > /tmp/tmp.RWRHuiuA8r 2023-11-24T08:00:56.775 DEBUG:teuthology.orchestra.run.smithi059:> sudo chmod 0666 /tmp/tmp.RWRHuiuA8r 2023-11-24T08:00:56.862 DEBUG:teuthology.orchestra.remote:smithi059:/tmp/tmp.RWRHuiuA8r is 685KB 2023-11-24T08:00:56.964 DEBUG:teuthology.orchestra.run.smithi059:> rm -fr /tmp/tmp.RWRHuiuA8r 2023-11-24T08:00:56.981 DEBUG:teuthology.misc:Transferring archived files from smithi076:/var/lib/ceph/mon/ceph-b to /home/teuthworker/archive/rishabh-2023-11-24_07:31:48-fs:volumes-rishabh-mgr-vol-misc-testing-default-smithi/7465575/data/mon.b.tgz 2023-11-24T08:00:56.982 DEBUG:teuthology.orchestra.run.smithi076:> mktemp 2023-11-24T08:00:57.001 INFO:teuthology.orchestra.run.smithi076.stdout:/tmp/tmp.nGau19Ys1l 2023-11-24T08:00:57.001 DEBUG:teuthology.orchestra.run.smithi076:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-b -- . > /tmp/tmp.nGau19Ys1l 2023-11-24T08:00:57.235 DEBUG:teuthology.orchestra.run.smithi076:> sudo chmod 0666 /tmp/tmp.nGau19Ys1l 2023-11-24T08:00:57.320 DEBUG:teuthology.orchestra.remote:smithi076:/tmp/tmp.nGau19Ys1l is 935KB 2023-11-24T08:00:57.423 DEBUG:teuthology.orchestra.run.smithi076:> rm -fr /tmp/tmp.nGau19Ys1l 2023-11-24T08:00:57.439 DEBUG:teuthology.misc:Transferring archived files from smithi076:/var/lib/ceph/mon/ceph-c to /home/teuthworker/archive/rishabh-2023-11-24_07:31:48-fs:volumes-rishabh-mgr-vol-misc-testing-default-smithi/7465575/data/mon.c.tgz 2023-11-24T08:00:57.440 DEBUG:teuthology.orchestra.run.smithi076:> mktemp 2023-11-24T08:00:57.495 INFO:teuthology.orchestra.run.smithi076.stdout:/tmp/tmp.8cyrQ1l3DG 2023-11-24T08:00:57.496 DEBUG:teuthology.orchestra.run.smithi076:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-c -- . > /tmp/tmp.8cyrQ1l3DG 2023-11-24T08:00:57.718 DEBUG:teuthology.orchestra.run.smithi076:> sudo chmod 0666 /tmp/tmp.8cyrQ1l3DG 2023-11-24T08:00:57.804 DEBUG:teuthology.orchestra.remote:smithi076:/tmp/tmp.8cyrQ1l3DG is 935KB 2023-11-24T08:00:57.910 DEBUG:teuthology.orchestra.run.smithi076:> rm -fr /tmp/tmp.8cyrQ1l3DG 2023-11-24T08:00:57.927 INFO:tasks.ceph:Cleaning ceph cluster... 2023-11-24T08:00:57.927 DEBUG:teuthology.orchestra.run.smithi059:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2023-11-24T08:00:57.930 DEBUG:teuthology.orchestra.run.smithi076:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2023-11-24T08:00:57.993 INFO:tasks.ceph:Checking for errors in any valgrind logs... 2023-11-24T08:00:57.993 DEBUG:teuthology.orchestra.run.smithi059:> sudo zgrep '' /var/log/ceph/valgrind/* /dev/null | sort | uniq 2023-11-24T08:00:58.004 DEBUG:teuthology.orchestra.run.smithi076:> sudo zgrep '' /var/log/ceph/valgrind/* /dev/null | sort | uniq 2023-11-24T08:00:58.054 INFO:teuthology.orchestra.run.smithi059.stderr:gzip: /var/log/ceph/valgrind/*.gz: No such file or directory 2023-11-24T08:00:58.068 INFO:teuthology.orchestra.run.smithi076.stderr:gzip: /var/log/ceph/valgrind/*.gz: No such file or directory 2023-11-24T08:00:58.074 INFO:tasks.ceph:Archiving crash dumps... 2023-11-24T08:00:58.076 DEBUG:teuthology.misc:Transferring archived files from smithi059:/var/lib/ceph/crash to /home/teuthworker/archive/rishabh-2023-11-24_07:31:48-fs:volumes-rishabh-mgr-vol-misc-testing-default-smithi/7465575/remote/smithi059/crash 2023-11-24T08:00:58.076 DEBUG:teuthology.orchestra.run.smithi059:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2023-11-24T08:00:58.111 DEBUG:teuthology.misc:Transferring archived files from smithi076:/var/lib/ceph/crash to /home/teuthworker/archive/rishabh-2023-11-24_07:31:48-fs:volumes-rishabh-mgr-vol-misc-testing-default-smithi/7465575/remote/smithi076/crash 2023-11-24T08:00:58.112 DEBUG:teuthology.orchestra.run.smithi076:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2023-11-24T08:00:58.139 INFO:tasks.ceph:Shutting down logrotate 2023-11-24T08:00:58.139 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-24T08:00:58.156 DEBUG:teuthology.orchestra.run.smithi076:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-24T08:00:58.239 DEBUG:teuthology.orchestra.run.smithi059:> sudo rm /etc/logrotate.d/ceph-test.conf 2023-11-24T08:00:58.270 DEBUG:teuthology.orchestra.run.smithi076:> sudo rm /etc/logrotate.d/ceph-test.conf 2023-11-24T08:00:58.296 INFO:tasks.ceph:Compressing logs... 2023-11-24T08:00:58.297 DEBUG:teuthology.orchestra.run.smithi059:> time sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2023-11-24T08:00:58.313 DEBUG:teuthology.orchestra.run.smithi076:> time sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2023-11-24T08:00:58.340 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.119680.log 2023-11-24T08:00:58.389 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.103483.log 2023-11-24T08:00:58.390 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.1.113434.log 2023-11-24T08:00:58.390 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.103483.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.5.log 2023-11-24T08:00:58.390 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.1.113434.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.103483.log.gz 2023-11-24T08:00:58.390 INFO:teuthology.orchestra.run.smithi076.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.1.113434.log.gz 2023-11-24T08:00:58.391 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.1.112197.log 2023-11-24T08:00:58.391 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph.log 2023-11-24T08:00:58.391 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.103618.log 2023-11-24T08:00:58.391 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.1.114711.log 2023-11-24T08:00:58.392 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.1.112808.log 2023-11-24T08:00:58.392 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-osd.5.log: /var/log/ceph/ceph-client.1.112197.log: /var/log/ceph/ceph.log: /var/log/ceph/ceph-client.admin.103618.log: 0.0% -- replaced with /var/log/ceph/ceph-client.1.112197.log.gz 2023-11-24T08:00:58.392 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.1.114711.log: 0.0% -- replaced with /var/log/ceph/ceph-client.1.114711.log.gz 2023-11-24T08:00:58.392 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.6.log 2023-11-24T08:00:58.392 INFO:teuthology.orchestra.run.smithi076.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.103618.log.gz 2023-11-24T08:00:58.393 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.1.112808.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.1.110789.log 2023-11-24T08:00:58.393 INFO:teuthology.orchestra.run.smithi076.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.1.112808.log.gz 2023-11-24T08:00:58.393 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-osd.6.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.1.114065.log 2023-11-24T08:00:58.393 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.1.110789.log: 0.0% -- replaced with /var/log/ceph/ceph-client.1.110789.log.gz 2023-11-24T08:00:58.394 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.1.112804.log 2023-11-24T08:00:58.394 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.1.114707.log 2023-11-24T08:00:58.394 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.1.114065.log: 0.0% -- replaced with /var/log/ceph/ceph-client.1.114065.log.gz 2023-11-24T08:00:58.394 INFO:teuthology.orchestra.run.smithi076.stderr: 92.2% -- replaced with /var/log/ceph/ceph.log.gz 2023-11-24T08:00:58.395 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.1.112804.log: gzip -5 0.0%--verbose -- -- replaced with /var/log/ceph/ceph-client.1.112804.log.gz/var/log/ceph/ceph-client.admin.103528.log 2023-11-24T08:00:58.395 INFO:teuthology.orchestra.run.smithi076.stderr: 2023-11-24T08:00:58.395 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.1.114707.log: gzip -5 --verbose -- /var/log/ceph/ceph-mds.d.log 2023-11-24T08:00:58.395 INFO:teuthology.orchestra.run.smithi076.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.1.114707.log.gz 2023-11-24T08:00:58.396 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.103528.log: gzip -5 --verbose -- /var/log/ceph/ceph.audit.log 2023-11-24T08:00:58.396 INFO:teuthology.orchestra.run.smithi076.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.103528.log.gz 2023-11-24T08:00:58.396 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-mds.d.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.4.log 2023-11-24T08:00:58.396 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mgr.x.log 2023-11-24T08:00:58.396 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph.audit.log: /var/log/ceph/ceph-osd.4.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.103436.log 2023-11-24T08:00:58.397 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-mgr.x.log: /var/log/ceph/ceph-client.admin.103436.log: 86.9% -- replaced with /var/log/ceph/ceph-client.admin.103436.log.gz 2023-11-24T08:00:58.397 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mon.c.log 2023-11-24T08:00:58.397 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-mon.c.log: 92.6% -- replaced with /var/log/ceph/ceph-mds.d.log.gz 2023-11-24T08:00:58.397 INFO:teuthology.orchestra.run.smithi076.stderr: 93.6% -- replaced with /var/log/ceph/ceph.audit.log.gz 2023-11-24T08:00:58.398 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.103573.log 2023-11-24T08:00:58.398 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.1.112193.log 2023-11-24T08:00:58.398 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.103573.log: /var/log/ceph/ceph-client.1.112193.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.103573.log.gz 2023-11-24T08:00:58.398 INFO:teuthology.orchestra.run.smithi076.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.1.112193.log.gz 2023-11-24T08:00:58.399 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.103391.log 2023-11-24T08:00:58.399 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.7.log 2023-11-24T08:00:58.399 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.103391.log: /var/log/ceph/ceph-osd.7.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.103391.log.gz 2023-11-24T08:00:58.399 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.3.111404.log 2023-11-24T08:00:58.400 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.3.111404.log: 0.0% -- replaced with /var/log/ceph/ceph-client.3.111404.log.gz 2023-11-24T08:00:58.400 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mon.b.log 2023-11-24T08:00:58.403 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-mon.b.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.103671.log 2023-11-24T08:00:58.404 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.103671.log: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.103671.log.gz 2023-11-24T08:00:58.404 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.103344.log 2023-11-24T08:00:58.404 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.103344.log: 86.9% -- replaced with /var/log/ceph/ceph-client.admin.103344.log.gz 2023-11-24T08:00:58.405 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.3.111408.log 2023-11-24T08:00:58.405 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.3.111408.log: 0.0% -- replaced with /var/log/ceph/ceph-client.3.111408.log.gz 2023-11-24T08:00:58.405 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mds.b.log 2023-11-24T08:00:58.405 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-mds.b.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.1.113430.log 2023-11-24T08:00:58.421 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.1.113430.log: 0.0% -- replaced with /var/log/ceph/ceph-client.1.113430.log.gz 2023-11-24T08:00:58.422 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.1.114061.log 2023-11-24T08:00:58.427 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.1.114061.log: 0.0% -- replaced with /var/log/ceph/ceph-client.1.114061.log.gz 2023-11-24T08:00:58.441 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.1.110797.log 2023-11-24T08:00:58.454 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.117594.log 2023-11-24T08:00:58.454 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose /var/log/ceph/ceph-client.admin.119680.log: -- /var/log/ceph/ceph-client.admin.134257.log 2023-11-24T08:00:58.454 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.117594.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.119680.log.gz 2023-11-24T08:00:58.454 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.115708.log 2023-11-24T08:00:58.455 INFO:teuthology.orchestra.run.smithi059.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.117594.log.gz 2023-11-24T08:00:58.455 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.134257.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.114863.log 2023-11-24T08:00:58.455 INFO:teuthology.orchestra.run.smithi059.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.134257.log.gz 2023-11-24T08:00:58.455 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.123029.log 2023-11-24T08:00:58.456 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.115708.log: /var/log/ceph/ceph-client.admin.114863.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.126819.log 2023-11-24T08:00:58.456 INFO:teuthology.orchestra.run.smithi059.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.115708.log.gz 2023-11-24T08:00:58.456 INFO:teuthology.orchestra.run.smithi059.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.114863.log.gz 2023-11-24T08:00:58.456 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.123029.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.118959.log 2023-11-24T08:00:58.456 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.126819.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.113285.log 2023-11-24T08:00:58.457 INFO:teuthology.orchestra.run.smithi059.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.123029.log.gz 2023-11-24T08:00:58.457 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.118959.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.126819.log.gz 2023-11-24T08:00:58.457 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.120343.log 2023-11-24T08:00:58.457 INFO:teuthology.orchestra.run.smithi059.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.118959.log.gz 2023-11-24T08:00:58.458 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.113285.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.122803.log 2023-11-24T08:00:58.458 INFO:teuthology.orchestra.run.smithi059.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.113285.log.gz 2023-11-24T08:00:58.458 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.120343.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.132369.log 2023-11-24T08:00:58.458 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.122803.log: 89.3%gzip -- replaced with /var/log/ceph/ceph-client.admin.120343.log.gz-5 2023-11-24T08:00:58.458 INFO:teuthology.orchestra.run.smithi059.stderr:--verbose -- /var/log/ceph/ceph-client.admin.117306.log 2023-11-24T08:00:58.459 INFO:teuthology.orchestra.run.smithi059.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.122803.log.gz 2023-11-24T08:00:58.459 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.132369.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.125142.log 2023-11-24T08:00:58.459 INFO:teuthology.orchestra.run.smithi059.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.132369.log.gz 2023-11-24T08:00:58.459 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.117306.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.116624.log 2023-11-24T08:00:58.459 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.125142.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.117306.log.gz 2023-11-24T08:00:58.460 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.114633.log 2023-11-24T08:00:58.460 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.116624.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.125142.log.gz 2023-11-24T08:00:58.460 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.111997.log 2023-11-24T08:00:58.460 INFO:teuthology.orchestra.run.smithi059.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.116624.log.gz 2023-11-24T08:00:58.460 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.114633.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.125050.log 2023-11-24T08:00:58.461 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.111997.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.114633.log.gz 2023-11-24T08:00:58.461 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.125096.log 2023-11-24T08:00:58.461 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.125050.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.111997.log.gz 2023-11-24T08:00:58.461 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.133733.log 2023-11-24T08:00:58.462 INFO:teuthology.orchestra.run.smithi059.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.125050.log.gz 2023-11-24T08:00:58.462 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.112573.log 2023-11-24T08:00:58.462 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.125096.log: /var/log/ceph/ceph-client.admin.133733.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.111721.log 2023-11-24T08:00:58.462 INFO:teuthology.orchestra.run.smithi059.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.125096.log.gz 2023-11-24T08:00:58.463 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.112573.log: 88.3%gzip -5 -- replaced with /var/log/ceph/ceph-client.admin.133733.log.gz--verbose 2023-11-24T08:00:58.463 INFO:teuthology.orchestra.run.smithi059.stderr:-- /var/log/ceph/ceph-client.admin.114034.log 2023-11-24T08:00:58.463 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.111721.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.112573.log.gz 2023-11-24T08:00:58.463 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.130549.log 2023-11-24T08:00:58.463 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.114034.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.111721.log.gz 2023-11-24T08:00:58.464 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.119726.log 2023-11-24T08:00:58.464 INFO:teuthology.orchestra.run.smithi059.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.114034.log.gz 2023-11-24T08:00:58.464 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.130549.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.103504.log 2023-11-24T08:00:58.464 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.119726.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.130549.log.gz 2023-11-24T08:00:58.465 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.120389.log 2023-11-24T08:00:58.465 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.103504.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.119726.log.gz 2023-11-24T08:00:58.465 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.135445.log 2023-11-24T08:00:58.465 INFO:teuthology.orchestra.run.smithi059.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.103504.log.gz 2023-11-24T08:00:58.465 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.120389.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.111140.log 2023-11-24T08:00:58.466 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.135445.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.120389.log.gz 2023-11-24T08:00:58.466 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.110778.log 2023-11-24T08:00:58.466 INFO:teuthology.orchestra.run.smithi059.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.135445.log.gz 2023-11-24T08:00:58.466 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.111140.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.113942.log 2023-11-24T08:00:58.466 INFO:teuthology.orchestra.run.smithi059.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.111140.log.gz 2023-11-24T08:00:58.467 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.132022.log 2023-11-24T08:00:58.467 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.110778.log: /var/log/ceph/ceph-client.admin.113942.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.123213.log 2023-11-24T08:00:58.467 INFO:teuthology.orchestra.run.smithi059.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.110778.log.gz 2023-11-24T08:00:58.467 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.132022.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.113942.log.gz 2023-11-24T08:00:58.467 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.114449.log 2023-11-24T08:00:58.468 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.123213.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.2.log 2023-11-24T08:00:58.468 INFO:teuthology.orchestra.run.smithi059.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.132022.log.gz 2023-11-24T08:00:58.468 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.114449.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.114817.log 2023-11-24T08:00:58.468 INFO:teuthology.orchestra.run.smithi059.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.123213.log.gz 2023-11-24T08:00:58.468 INFO:teuthology.orchestra.run.smithi059.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.114449.log.gz 2023-11-24T08:00:58.469 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-osd.2.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.131358.log 2023-11-24T08:00:58.469 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.114817.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.111813.log 2023-11-24T08:00:58.469 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.131358.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.114817.log.gz 2023-11-24T08:00:58.469 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.117492.log 2023-11-24T08:00:58.469 INFO:teuthology.orchestra.run.smithi059.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.131358.log.gz 2023-11-24T08:00:58.470 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.120903.log 2023-11-24T08:00:58.470 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.111813.log: /var/log/ceph/ceph-client.admin.117492.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.123462.log 2023-11-24T08:00:58.470 INFO:teuthology.orchestra.run.smithi059.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.111813.log.gz 2023-11-24T08:00:58.470 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.120903.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.117492.log.gz 2023-11-24T08:00:58.471 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.116486.log 2023-11-24T08:00:58.471 INFO:teuthology.orchestra.run.smithi059.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.120903.log.gz 2023-11-24T08:00:58.471 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 /var/log/ceph/ceph-client.admin.123462.log: --verbose -- /var/log/ceph/ceph-client.admin.130871.log 2023-11-24T08:00:58.471 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.116486.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.128629.log 2023-11-24T08:00:58.472 INFO:teuthology.orchestra.run.smithi059.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.123462.log.gz 2023-11-24T08:00:58.472 INFO:teuthology.orchestra.run.smithi059.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.116486.log.gz 2023-11-24T08:00:58.472 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.130871.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.127187.log 2023-11-24T08:00:58.472 INFO:teuthology.orchestra.run.smithi059.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.130871.log.gz 2023-11-24T08:00:58.472 INFO:teuthology.orchestra.run.smithi059.stderr:gzip /var/log/ceph/ceph-client.admin.128629.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.127365.log 2023-11-24T08:00:58.473 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.127187.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.127549.log 2023-11-24T08:00:58.473 INFO:teuthology.orchestra.run.smithi059.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.128629.log.gz 2023-11-24T08:00:58.473 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.127365.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.127187.log.gz 2023-11-24T08:00:58.473 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.126865.log 2023-11-24T08:00:58.474 INFO:teuthology.orchestra.run.smithi059.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.127365.log.gz 2023-11-24T08:00:58.474 INFO:teuthology.orchestra.run.smithi059.stderr:gzip /var/log/ceph/ceph-client.admin.127549.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.130120.log 2023-11-24T08:00:58.474 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.126865.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.116716.log 2023-11-24T08:00:58.474 INFO:teuthology.orchestra.run.smithi059.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.127549.log.gz 2023-11-24T08:00:58.475 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.130120.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.126865.log.gz 2023-11-24T08:00:58.475 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.121995.log 2023-11-24T08:00:58.475 INFO:teuthology.orchestra.run.smithi059.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.130120.log.gz 2023-11-24T08:00:58.475 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph.log 2023-11-24T08:00:58.477 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.116716.log: /var/log/ceph/ceph-client.admin.121995.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.117782.log 2023-11-24T08:00:58.477 INFO:teuthology.orchestra.run.smithi059.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.116716.log.gz 2023-11-24T08:00:58.477 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.121995.log.gz 2023-11-24T08:00:58.477 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.119772.log 2023-11-24T08:00:58.478 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.117782.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.117074.log 2023-11-24T08:00:58.478 INFO:teuthology.orchestra.run.smithi059.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.117782.log.gz 2023-11-24T08:00:58.478 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.119772.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.116210.log 2023-11-24T08:00:58.478 INFO:teuthology.orchestra.run.smithi059.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.119772.log.gz 2023-11-24T08:00:58.479 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.118263.log 2023-11-24T08:00:58.479 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.117074.log: 86.0% -- replaced with /var/log/ceph/ceph.log.gz 2023-11-24T08:00:58.479 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.116210.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.117074.log.gzgzip 2023-11-24T08:00:58.479 INFO:teuthology.orchestra.run.smithi059.stderr:-5 --verbose -- /var/log/ceph/ceph-client.0.132764.log 2023-11-24T08:00:58.480 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.117920.log 2023-11-24T08:00:58.480 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.118263.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.116210.log.gz 2023-11-24T08:00:58.480 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.0.132764.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.121212.log 2023-11-24T08:00:58.480 INFO:teuthology.orchestra.run.smithi059.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.0.132764.log.gz 2023-11-24T08:00:58.481 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.117920.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.130779.log 2023-11-24T08:00:58.481 INFO:teuthology.orchestra.run.smithi059.stderr: 91.1% -- replaced with /var/log/ceph/ceph-client.admin.118263.log.gz 2023-11-24T08:00:58.481 INFO:teuthology.orchestra.run.smithi059.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.117920.log.gz 2023-11-24T08:00:58.481 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.121212.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.112097.log 2023-11-24T08:00:58.482 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.130779.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.121212.log.gz 2023-11-24T08:00:58.482 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.126911.log 2023-11-24T08:00:58.482 INFO:teuthology.orchestra.run.smithi059.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.130779.log.gz 2023-11-24T08:00:58.482 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.112097.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.126773.log 2023-11-24T08:00:58.482 INFO:teuthology.orchestra.run.smithi059.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.112097.log.gz 2023-11-24T08:00:58.483 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.126911.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.123895.log 2023-11-24T08:00:58.483 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.126773.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.126911.log.gz 2023-11-24T08:00:58.483 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.128859.log 2023-11-24T08:00:58.483 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.123895.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.126773.log.gz 2023-11-24T08:00:58.484 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.110320.log 2023-11-24T08:00:58.484 INFO:teuthology.orchestra.run.smithi059.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.123895.log.gz 2023-11-24T08:00:58.484 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.128859.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.131128.log 2023-11-24T08:00:58.484 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.110320.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.128859.log.gz 2023-11-24T08:00:58.485 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.109490.log 2023-11-24T08:00:58.485 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.131128.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.123646.log 2023-11-24T08:00:58.485 INFO:teuthology.orchestra.run.smithi059.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.110320.log.gz 2023-11-24T08:00:58.485 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.109490.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.131128.log.gz 2023-11-24T08:00:58.485 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.117398.log 2023-11-24T08:00:58.485 INFO:teuthology.orchestra.run.smithi059.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.109490.log.gz 2023-11-24T08:00:58.486 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.123646.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.109763.log 2023-11-24T08:00:58.486 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.117398.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.115980.log 2023-11-24T08:00:58.486 INFO:teuthology.orchestra.run.smithi059.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.123646.log.gz 2023-11-24T08:00:58.486 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.109763.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.117398.log.gz 2023-11-24T08:00:58.486 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.116118.log 2023-11-24T08:00:58.486 INFO:teuthology.orchestra.run.smithi059.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.109763.log.gz 2023-11-24T08:00:58.486 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.115980.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.120811.log 2023-11-24T08:00:58.486 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.116118.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.115980.log.gz 2023-11-24T08:00:58.487 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.116302.log 2023-11-24T08:00:58.487 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.120811.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.116118.log.gz 2023-11-24T08:00:58.487 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.111507.log 2023-11-24T08:00:58.487 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.116302.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.120811.log.gz 2023-11-24T08:00:58.487 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.124192.log 2023-11-24T08:00:58.487 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.111507.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.116302.log.gz 2023-11-24T08:00:58.487 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.131466.log 2023-11-24T08:00:58.487 INFO:teuthology.orchestra.run.smithi059.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.111507.log.gz 2023-11-24T08:00:58.488 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.124192.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.114587.log 2023-11-24T08:00:58.488 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.131466.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.124192.log.gz 2023-11-24T08:00:58.488 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.125693.log 2023-11-24T08:00:58.488 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.114587.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.131466.log.gz 2023-11-24T08:00:58.488 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.115605.log 2023-11-24T08:00:58.488 INFO:teuthology.orchestra.run.smithi059.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.114587.log.gz 2023-11-24T08:00:58.488 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.125693.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.130346.log 2023-11-24T08:00:58.488 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.115605.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.125693.log.gzgzip 2023-11-24T08:00:58.488 INFO:teuthology.orchestra.run.smithi059.stderr:-5 --verbose -- /var/log/ceph/ceph-client.admin.117260.log 2023-11-24T08:00:58.489 INFO:teuthology.orchestra.run.smithi059.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.115605.log.gz 2023-11-24T08:00:58.489 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.130346.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.135583.log 2023-11-24T08:00:58.489 INFO:teuthology.orchestra.run.smithi059.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.130346.log.gz 2023-11-24T08:00:58.489 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.117260.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.131556.log 2023-11-24T08:00:58.489 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.135583.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.117260.log.gz 2023-11-24T08:00:58.489 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.122041.log 2023-11-24T08:00:58.489 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.131556.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.135583.log.gz 2023-11-24T08:00:58.489 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.129364.log 2023-11-24T08:00:58.490 INFO:teuthology.orchestra.run.smithi059.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.131556.log.gz 2023-11-24T08:00:58.490 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.122041.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.126616.log 2023-11-24T08:00:58.490 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.129364.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.122757.log 2023-11-24T08:00:58.490 INFO:teuthology.orchestra.run.smithi059.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.122041.log.gz 2023-11-24T08:00:58.490 INFO:teuthology.orchestra.run.smithi059.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.129364.log.gz 2023-11-24T08:00:58.490 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.126616.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.117828.log 2023-11-24T08:00:58.490 INFO:teuthology.orchestra.run.smithi059.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.126616.log.gz 2023-11-24T08:00:58.490 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.122757.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.0.log 2023-11-24T08:00:58.491 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.117828.log: 88.7%gzip -5 --verbose -- replaced with /var/log/ceph/ceph-client.admin.122757.log.gz-- 2023-11-24T08:00:58.491 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.118814.log 2023-11-24T08:00:58.491 INFO:teuthology.orchestra.run.smithi059.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.117828.log.gz 2023-11-24T08:00:58.491 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-osd.0.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.131266.log 2023-11-24T08:00:58.491 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.118814.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.127971.log 2023-11-24T08:00:58.491 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.131266.log: 88.0%gzip -- replaced with /var/log/ceph/ceph-client.admin.118814.log.gz-5 2023-11-24T08:00:58.491 INFO:teuthology.orchestra.run.smithi059.stderr:--verbose -- /var/log/ceph/ceph-client.admin.134009.log 2023-11-24T08:00:58.491 INFO:teuthology.orchestra.run.smithi059.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.131266.log.gz 2023-11-24T08:00:58.492 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.127971.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.111859.log 2023-11-24T08:00:58.492 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.134009.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.127971.log.gz 2023-11-24T08:00:58.492 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.134442.log 2023-11-24T08:00:58.492 INFO:teuthology.orchestra.run.smithi059.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.134009.log.gz 2023-11-24T08:00:58.492 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.111859.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.129318.log 2023-11-24T08:00:58.492 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.116808.log 2023-11-24T08:00:58.492 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.134442.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.111859.log.gz 2023-11-24T08:00:58.492 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.129318.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.111951.log 2023-11-24T08:00:58.493 INFO:teuthology.orchestra.run.smithi059.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.134442.log.gz 2023-11-24T08:00:58.493 INFO:teuthology.orchestra.run.smithi059.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.129318.log.gz 2023-11-24T08:00:58.493 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.116808.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.131312.log 2023-11-24T08:00:58.493 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.111951.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.121536.log 2023-11-24T08:00:58.493 INFO:teuthology.orchestra.run.smithi059.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.116808.log.gz 2023-11-24T08:00:58.493 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.131312.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.110479.log 2023-11-24T08:00:58.493 INFO:teuthology.orchestra.run.smithi059.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.111951.log.gz 2023-11-24T08:00:58.493 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.121536.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.131312.log.gz 2023-11-24T08:00:58.494 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.116670.log 2023-11-24T08:00:58.494 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.135399.log 89.2% 2023-11-24T08:00:58.494 INFO:teuthology.orchestra.run.smithi059.stderr: -- replaced with /var/log/ceph/ceph-client.admin.121536.log.gz 2023-11-24T08:00:58.494 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.110479.log: /var/log/ceph/ceph-client.admin.116670.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.115754.log 2023-11-24T08:00:58.494 INFO:teuthology.orchestra.run.smithi059.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.110479.log.gz 2023-11-24T08:00:58.494 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.135399.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.126570.log 2023-11-24T08:00:58.494 INFO:teuthology.orchestra.run.smithi059.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.116670.log.gz 2023-11-24T08:00:58.495 INFO:teuthology.orchestra.run.smithi059.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.135399.log.gz 2023-11-24T08:00:58.495 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.115754.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.123416.log 2023-11-24T08:00:58.495 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.126570.log: gzip -5 --verbose -- /var/log/ceph/ceph-mds.a.log 2023-11-24T08:00:58.495 INFO:teuthology.orchestra.run.smithi059.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.115754.log.gz 2023-11-24T08:00:58.495 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.123416.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.128721.log 2023-11-24T08:00:58.495 INFO:teuthology.orchestra.run.smithi059.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.126570.log.gz 2023-11-24T08:00:58.495 INFO:teuthology.orchestra.run.smithi059.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.123416.log.gz 2023-11-24T08:00:58.495 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-mds.a.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.121444.log 2023-11-24T08:00:58.496 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.117874.log 2023-11-24T08:00:58.496 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.128721.log: /var/log/ceph/ceph-client.admin.121444.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.127095.log 2023-11-24T08:00:58.496 INFO:teuthology.orchestra.run.smithi059.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.128721.log.gz 2023-11-24T08:00:58.496 INFO:teuthology.orchestra.run.smithi059.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.121444.log.gz/var/log/ceph/ceph-client.admin.117874.log: 2023-11-24T08:00:58.496 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.120048.log 2023-11-24T08:00:58.496 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.127095.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.117874.log.gz 2023-11-24T08:00:58.496 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.128063.log 2023-11-24T08:00:58.496 INFO:teuthology.orchestra.run.smithi059.stderr: 87.6%/var/log/ceph/ceph-client.admin.120048.log: -- replaced with /var/log/ceph/ceph-client.admin.127095.log.gz 2023-11-24T08:00:58.497 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.129208.log 2023-11-24T08:00:58.497 INFO:teuthology.orchestra.run.smithi059.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.120048.log.gz 2023-11-24T08:00:58.497 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.128063.log: /var/log/ceph/ceph-client.0.129208.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.128063.log.gz 2023-11-24T08:00:58.497 INFO:teuthology.orchestra.run.smithi059.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.0.129208.log.gz 2023-11-24T08:00:58.497 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.126206.log 2023-11-24T08:00:58.497 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.118125.log 2023-11-24T08:00:58.497 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.126206.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.114126.log 2023-11-24T08:00:58.497 INFO:teuthology.orchestra.run.smithi059.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.126206.log.gz 2023-11-24T08:00:58.498 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.118125.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.116026.log 2023-11-24T08:00:58.498 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.114126.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.124818.log 2023-11-24T08:00:58.498 INFO:teuthology.orchestra.run.smithi059.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.118125.log.gz 2023-11-24T08:00:58.498 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.116026.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.128491.log 2023-11-24T08:00:58.498 INFO:teuthology.orchestra.run.smithi059.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.114126.log.gz 2023-11-24T08:00:58.498 INFO:teuthology.orchestra.run.smithi059.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.116026.log.gz 2023-11-24T08:00:58.498 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.124818.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.133622.log 2023-11-24T08:00:58.498 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.119910.log 2023-11-24T08:00:58.499 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.128491.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.124818.log.gz 2023-11-24T08:00:58.499 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.133622.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.128491.log.gz 2023-11-24T08:00:58.499 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.119910.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.133622.log.gz 2023-11-24T08:00:58.499 INFO:teuthology.orchestra.run.smithi059.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.119910.log.gz 2023-11-24T08:00:58.499 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.119159.log 2023-11-24T08:00:58.499 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.119588.log 2023-11-24T08:00:58.499 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.119159.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.118079.log 2023-11-24T08:00:58.499 INFO:teuthology.orchestra.run.smithi059.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.119159.log.gz 2023-11-24T08:00:58.499 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.119588.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.125587.log 2023-11-24T08:00:58.500 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.118079.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.124708.log 2023-11-24T08:00:58.500 INFO:teuthology.orchestra.run.smithi059.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.119588.log.gz 2023-11-24T08:00:58.500 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.0.125587.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.118217.log 2023-11-24T08:00:58.500 INFO:teuthology.orchestra.run.smithi059.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.118079.log.gz 2023-11-24T08:00:58.500 INFO:teuthology.orchestra.run.smithi059.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.0.125587.log.gz 2023-11-24T08:00:58.500 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.124708.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.117352.log 2023-11-24T08:00:58.500 INFO:teuthology.orchestra.run.smithi059.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.124708.log.gz 2023-11-24T08:00:58.500 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.118217.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.129656.log 2023-11-24T08:00:58.501 INFO:teuthology.orchestra.run.smithi059.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.118217.log.gz 2023-11-24T08:00:58.501 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.117352.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.131174.log 2023-11-24T08:00:58.501 INFO:teuthology.orchestra.run.smithi059.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.117352.log.gz 2023-11-24T08:00:58.501 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.129656.log: /var/log/ceph/ceph-client.admin.131174.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.129656.log.gz 2023-11-24T08:00:58.501 INFO:teuthology.orchestra.run.smithi059.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.131174.log.gz 2023-11-24T08:00:58.501 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.111353.log 2023-11-24T08:00:58.501 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.134801.log 2023-11-24T08:00:58.501 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.111353.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.122589.log 2023-11-24T08:00:58.502 INFO:teuthology.orchestra.run.smithi059.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.111353.log.gz 2023-11-24T08:00:58.502 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.134801.log: /var/log/ceph/ceph-client.admin.122589.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.134801.log.gz 2023-11-24T08:00:58.502 INFO:teuthology.orchestra.run.smithi059.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.122589.log.gz 2023-11-24T08:00:58.502 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.123849.log 2023-11-24T08:00:58.502 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.128813.log 2023-11-24T08:00:58.502 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.123849.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.133963.log 2023-11-24T08:00:58.502 INFO:teuthology.orchestra.run.smithi059.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.123849.log.gz 2023-11-24T08:00:58.502 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.128813.log: /var/log/ceph/ceph-client.admin.133963.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.128813.log.gz 2023-11-24T08:00:58.503 INFO:teuthology.orchestra.run.smithi059.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.133963.log.gz 2023-11-24T08:00:58.503 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.135169.log 2023-11-24T08:00:58.503 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.123370.log 2023-11-24T08:00:58.503 INFO:teuthology.orchestra.run.smithi059.stderr:gzip /var/log/ceph/ceph-client.admin.135169.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.109906.log 2023-11-24T08:00:58.503 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.123370.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.135169.log.gz 2023-11-24T08:00:58.503 INFO:teuthology.orchestra.run.smithi059.stderr: 87.6%/var/log/ceph/ceph-client.admin.109906.log: -- replaced with /var/log/ceph/ceph-client.admin.123370.log.gz 2023-11-24T08:00:58.503 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.128351.log 2023-11-24T08:00:58.503 INFO:teuthology.orchestra.run.smithi059.stderr: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.109906.log.gz 2023-11-24T08:00:58.504 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.125923.log 2023-11-24T08:00:58.504 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.128351.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.135123.log 2023-11-24T08:00:58.504 INFO:teuthology.orchestra.run.smithi059.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.128351.log.gz 2023-11-24T08:00:58.504 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.125923.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.3.log 2023-11-24T08:00:58.504 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.130503.log 2023-11-24T08:00:58.504 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.135123.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.125923.log.gz 2023-11-24T08:00:58.504 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-osd.3.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.110772.log 2023-11-24T08:00:58.504 INFO:teuthology.orchestra.run.smithi059.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.135123.log.gz 2023-11-24T08:00:58.504 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.130503.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.130503.log.gz 2023-11-24T08:00:58.505 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.115559.log 2023-11-24T08:00:58.505 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.110772.log: /var/log/ceph/ceph-client.admin.115559.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.110772.log.gz 2023-11-24T08:00:58.505 INFO:teuthology.orchestra.run.smithi059.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.115559.log.gz 2023-11-24T08:00:58.505 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.135077.log 2023-11-24T08:00:58.505 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.125831.log 2023-11-24T08:00:58.505 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.135077.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.111145.log 2023-11-24T08:00:58.505 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.125831.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.135077.log.gz 2023-11-24T08:00:58.505 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.111145.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.125831.log.gz 2023-11-24T08:00:58.506 INFO:teuthology.orchestra.run.smithi059.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.111145.log.gz 2023-11-24T08:00:58.506 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.125234.log 2023-11-24T08:00:58.506 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.130208.log 2023-11-24T08:00:58.506 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.125234.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.126662.log 2023-11-24T08:00:58.506 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.130208.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.125234.log.gz 2023-11-24T08:00:58.506 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.126662.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.130208.log.gz 2023-11-24T08:00:58.506 INFO:teuthology.orchestra.run.smithi059.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.126662.log.gz 2023-11-24T08:00:58.506 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.110090.log 2023-11-24T08:00:58.507 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.119818.log 2023-11-24T08:00:58.507 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.110090.log: /var/log/ceph/ceph-client.admin.119818.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.110090.log.gz 2023-11-24T08:00:58.507 INFO:teuthology.orchestra.run.smithi059.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.119818.log.gz 2023-11-24T08:00:58.507 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.129410.log 2023-11-24T08:00:58.507 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.133668.log 2023-11-24T08:00:58.507 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.129410.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.117168.log 2023-11-24T08:00:58.507 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.133668.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.129410.log.gz 2023-11-24T08:00:58.507 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.117168.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.133668.log.gz 2023-11-24T08:00:58.508 INFO:teuthology.orchestra.run.smithi059.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.117168.log.gz 2023-11-24T08:00:58.508 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.135659.log 2023-11-24T08:00:58.508 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.115421.log 2023-11-24T08:00:58.508 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.135659.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.130392.log 2023-11-24T08:00:58.508 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.115421.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.135659.log.gz 2023-11-24T08:00:58.508 INFO:teuthology.orchestra.run.smithi059.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.115421.log.gz 2023-11-24T08:00:58.508 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.130392.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.124764.log 2023-11-24T08:00:58.508 INFO:teuthology.orchestra.run.smithi059.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.130392.log.gz 2023-11-24T08:00:58.509 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.124764.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.124764.log.gz 2023-11-24T08:00:58.509 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.119205.log 2023-11-24T08:00:58.509 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.114403.log 2023-11-24T08:00:58.509 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.119205.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.124146.log 2023-11-24T08:00:58.509 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.114403.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.119205.log.gz 2023-11-24T08:00:58.509 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.124146.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.114403.log.gz 2023-11-24T08:00:58.509 INFO:teuthology.orchestra.run.smithi059.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.124146.log.gz 2023-11-24T08:00:58.509 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.119339.log 2023-11-24T08:00:58.509 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.122941.log 2023-11-24T08:00:58.510 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.119339.log: /var/log/ceph/ceph-client.admin.122941.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.119339.log.gz 2023-11-24T08:00:58.510 INFO:teuthology.orchestra.run.smithi059.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.122941.log.gz 2023-11-24T08:00:58.510 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.128017.log 2023-11-24T08:00:58.510 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.112143.log 2023-11-24T08:00:58.510 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.128017.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.118676.log 2023-11-24T08:00:58.510 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.112143.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.128017.log.gz 2023-11-24T08:00:58.510 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.118676.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.112143.log.gz 2023-11-24T08:00:58.510 INFO:teuthology.orchestra.run.smithi059.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.118676.log.gz 2023-11-24T08:00:58.511 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.118333.log 2023-11-24T08:00:58.511 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.109952.log 2023-11-24T08:00:58.511 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.118333.log: /var/log/ceph/ceph-client.admin.109952.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.118333.log.gz 2023-11-24T08:00:58.511 INFO:teuthology.orchestra.run.smithi059.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.109952.log.gz 2023-11-24T08:00:58.511 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph.tmp-client.admin.96903.log 2023-11-24T08:00:58.511 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.119067.log 2023-11-24T08:00:58.511 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph.tmp-client.admin.96903.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.132323.log 2023-11-24T08:00:58.511 INFO:teuthology.orchestra.run.smithi059.stderr: 0.0% -- replaced with /var/log/ceph/ceph.tmp-client.admin.96903.log.gz 2023-11-24T08:00:58.512 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.119067.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.130300.log 2023-11-24T08:00:58.512 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.132323.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.119067.log.gz 2023-11-24T08:00:58.512 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.130300.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.132323.log.gz 2023-11-24T08:00:58.512 INFO:teuthology.orchestra.run.smithi059.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.130300.log.gz 2023-11-24T08:00:58.512 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.126727.log 2023-11-24T08:00:58.512 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.129915.log 2023-11-24T08:00:58.512 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.126727.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.119113.log 2023-11-24T08:00:58.512 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.129915.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.126727.log.gz 2023-11-24T08:00:58.513 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.119113.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.129915.log.gz 2023-11-24T08:00:58.513 INFO:teuthology.orchestra.run.smithi059.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.119113.log.gz 2023-11-24T08:00:58.513 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.125583.log 2023-11-24T08:00:58.513 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.132068.log 2023-11-24T08:00:58.513 INFO:teuthology.orchestra.run.smithi059.stderr:gzip /var/log/ceph/ceph-client.0.125583.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.134709.log 2023-11-24T08:00:58.513 INFO:teuthology.orchestra.run.smithi059.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.0.125583.log.gz 2023-11-24T08:00:58.513 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.132068.log: /var/log/ceph/ceph-client.admin.134709.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.132068.log.gz 2023-11-24T08:00:58.513 INFO:teuthology.orchestra.run.smithi059.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.134709.log.gz 2023-11-24T08:00:58.514 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.121490.log 2023-11-24T08:00:58.514 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.128767.log 2023-11-24T08:00:58.514 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.121490.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.118768.log 2023-11-24T08:00:58.514 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.128767.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.121490.log.gz 2023-11-24T08:00:58.514 INFO:teuthology.orchestra.run.smithi059.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.128767.log.gz 2023-11-24T08:00:58.514 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.118768.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.118768.log.gz 2023-11-24T08:00:58.514 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.121066.log 2023-11-24T08:00:58.514 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.116164.log 2023-11-24T08:00:58.515 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.121066.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.128905.log 2023-11-24T08:00:58.515 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.116164.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.121066.log.gz 2023-11-24T08:00:58.515 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.128905.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.116164.log.gz 2023-11-24T08:00:58.515 INFO:teuthology.orchestra.run.smithi059.stderr: 91.1% -- replaced with /var/log/ceph/ceph-client.admin.128905.log.gz 2023-11-24T08:00:58.515 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.122475.log 2023-11-24T08:00:58.515 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.131036.log 2023-11-24T08:00:58.515 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.122475.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.125739.log 2023-11-24T08:00:58.515 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.131036.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.122475.log.gz 2023-11-24T08:00:58.516 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.125739.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.131036.log.gz 2023-11-24T08:00:58.516 INFO:teuthology.orchestra.run.smithi059.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.125739.log.gz 2023-11-24T08:00:58.516 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.125785.log 2023-11-24T08:00:58.516 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.110776.log 2023-11-24T08:00:58.516 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.125785.log: /var/log/ceph/ceph-client.admin.110776.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.125785.log.gz 2023-11-24T08:00:58.516 INFO:teuthology.orchestra.run.smithi059.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.110776.log.gz 2023-11-24T08:00:58.516 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.134939.log 2023-11-24T08:00:58.516 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.111510.log 2023-11-24T08:00:58.517 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.134939.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.118570.log 2023-11-24T08:00:58.517 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.111510.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.134939.log.gz 2023-11-24T08:00:58.517 INFO:teuthology.orchestra.run.smithi059.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.111510.log.gz 2023-11-24T08:00:58.517 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.0.118570.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.118570.log.gz 2023-11-24T08:00:58.517 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mds.c.log 2023-11-24T08:00:58.517 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.135215.log 2023-11-24T08:00:58.517 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-mds.c.log: /var/log/ceph/ceph-client.admin.135215.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.135215.log.gz 2023-11-24T08:00:58.517 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.115022.log 2023-11-24T08:00:58.518 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.119385.log 2023-11-24T08:00:58.518 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.115022.log: /var/log/ceph/ceph-client.admin.119385.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.115022.log.gz 2023-11-24T08:00:58.518 INFO:teuthology.orchestra.run.smithi059.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.119385.log.gz 2023-11-24T08:00:58.518 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.103196.log 2023-11-24T08:00:58.518 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.124300.log 2023-11-24T08:00:58.518 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.103196.log: /var/log/ceph/ceph-client.admin.124300.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.103196.log.gz 2023-11-24T08:00:58.518 INFO:teuthology.orchestra.run.smithi059.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.124300.log.gz 2023-11-24T08:00:58.518 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.133917.log 2023-11-24T08:00:58.519 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.128399.log 2023-11-24T08:00:58.519 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.133917.log: /var/log/ceph/ceph-client.admin.128399.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.133917.log.gz 2023-11-24T08:00:58.519 INFO:teuthology.orchestra.run.smithi059.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.128399.log.gz 2023-11-24T08:00:58.519 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.126478.log 2023-11-24T08:00:58.519 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.112373.log 2023-11-24T08:00:58.519 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.126478.log: 88.8%/var/log/ceph/ceph-client.admin.112373.log: -- replaced with /var/log/ceph/ceph-client.admin.126478.log.gz 2023-11-24T08:00:58.519 INFO:teuthology.orchestra.run.smithi059.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.112373.log.gz 2023-11-24T08:00:58.519 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.133012.log 2023-11-24T08:00:58.520 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.122225.log 2023-11-24T08:00:58.520 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.133012.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.133012.log.gz 2023-11-24T08:00:58.520 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.122225.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.122225.log.gz 2023-11-24T08:00:58.520 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.124390.log 2023-11-24T08:00:58.520 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.133484.log 2023-11-24T08:00:58.520 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.124390.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.116394.log 2023-11-24T08:00:58.520 INFO:teuthology.orchestra.run.smithi059.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.124390.log.gz 2023-11-24T08:00:58.520 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.133484.log: /var/log/ceph/ceph-client.admin.116394.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.133484.log.gz 2023-11-24T08:00:58.520 INFO:teuthology.orchestra.run.smithi059.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.116394.log.gz 2023-11-24T08:00:58.521 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.123738.log 2023-11-24T08:00:58.521 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.119634.log 2023-11-24T08:00:58.521 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.123738.log: /var/log/ceph/ceph-client.admin.119634.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.123738.log.gz 2023-11-24T08:00:58.521 INFO:teuthology.orchestra.run.smithi059.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.119634.log.gz 2023-11-24T08:00:58.521 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.117735.log 2023-11-24T08:00:58.521 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.117028.log 2023-11-24T08:00:58.521 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.117735.log: /var/log/ceph/ceph-client.admin.117028.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.117735.log.gz 2023-11-24T08:00:58.521 INFO:teuthology.orchestra.run.smithi059.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.117028.log.gz 2023-11-24T08:00:58.522 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.133576.log 2023-11-24T08:00:58.522 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.123167.log 2023-11-24T08:00:58.522 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.133576.log: /var/log/ceph/ceph-client.admin.123167.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.133576.log.gz 2023-11-24T08:00:58.522 INFO:teuthology.orchestra.run.smithi059.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.123167.log.gz 2023-11-24T08:00:58.522 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.132162.log 2023-11-24T08:00:58.522 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.116898.log 2023-11-24T08:00:58.522 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.132162.log: /var/log/ceph/ceph-client.admin.116898.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.132162.log.gz 2023-11-24T08:00:58.522 INFO:teuthology.orchestra.run.smithi059.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.116898.log.gz 2023-11-24T08:00:58.523 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.122849.log 2023-11-24T08:00:58.523 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.121885.log 2023-11-24T08:00:58.523 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.122849.log: /var/log/ceph/ceph-client.0.121885.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.122849.log.gz 2023-11-24T08:00:58.523 INFO:teuthology.orchestra.run.smithi059.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.0.121885.log.gz 2023-11-24T08:00:58.523 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.131734.log 2023-11-24T08:00:58.523 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.116532.log 2023-11-24T08:00:58.523 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.131734.log: /var/log/ceph/ceph-client.admin.116532.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.131734.log.gz 2023-11-24T08:00:58.523 INFO:teuthology.orchestra.run.smithi059.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.116532.log.gz 2023-11-24T08:00:58.524 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.124958.log 2023-11-24T08:00:58.524 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.122543.log 2023-11-24T08:00:58.524 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.124958.log: /var/log/ceph/ceph-client.admin.122543.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.124958.log.gz 2023-11-24T08:00:58.524 INFO:teuthology.orchestra.run.smithi059.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.122543.log.gz 2023-11-24T08:00:58.524 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.109717.log 2023-11-24T08:00:58.524 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.127141.log 2023-11-24T08:00:58.524 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.109717.log: /var/log/ceph/ceph-client.admin.127141.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.109717.log.gz 2023-11-24T08:00:58.524 INFO:teuthology.orchestra.run.smithi059.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.127141.log.gz 2023-11-24T08:00:58.525 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.114172.log 2023-11-24T08:00:58.525 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.129212.log 2023-11-24T08:00:58.525 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.114172.log: /var/log/ceph/ceph-client.0.129212.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.114172.log.gz 2023-11-24T08:00:58.525 INFO:teuthology.orchestra.run.smithi059.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.0.129212.log.gz 2023-11-24T08:00:58.525 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.120297.log 2023-11-24T08:00:58.525 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.134147.log 2023-11-24T08:00:58.525 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.120297.log: /var/log/ceph/ceph-client.admin.134147.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.120297.log.gz 2023-11-24T08:00:58.525 INFO:teuthology.orchestra.run.smithi059.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.134147.log.gz 2023-11-24T08:00:58.526 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.133779.log 2023-11-24T08:00:58.526 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.133212.log 2023-11-24T08:00:58.526 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.133779.log: /var/log/ceph/ceph-client.admin.133212.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.133779.log.gz 2023-11-24T08:00:58.526 INFO:teuthology.orchestra.run.smithi059.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.133212.log.gz 2023-11-24T08:00:58.526 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.111143.log 2023-11-24T08:00:58.526 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.124054.log 2023-11-24T08:00:58.526 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.111143.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.111143.log.gz 2023-11-24T08:00:58.526 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.114725.log 2023-11-24T08:00:58.527 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.124054.log: /var/log/ceph/ceph-client.admin.114725.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.124054.log.gz 2023-11-24T08:00:58.527 INFO:teuthology.orchestra.run.smithi059.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.114725.log.gz 2023-11-24T08:00:58.527 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.133871.log 2023-11-24T08:00:58.527 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.115287.log 2023-11-24T08:00:58.527 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.133871.log: /var/log/ceph/ceph-client.admin.115287.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.133871.log.gz 2023-11-24T08:00:58.527 INFO:teuthology.orchestra.run.smithi059.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.115287.log.gz 2023-11-24T08:00:58.527 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.116982.log 2023-11-24T08:00:58.527 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.110433.log 2023-11-24T08:00:58.528 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.116982.log: /var/log/ceph/ceph-client.admin.110433.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.116982.log.gz 2023-11-24T08:00:58.528 INFO:teuthology.orchestra.run.smithi059.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.110433.log.gz 2023-11-24T08:00:58.528 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.130028.log 2023-11-24T08:00:58.528 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.120949.log 2023-11-24T08:00:58.528 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.130028.log: /var/log/ceph/ceph-client.admin.120949.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.130028.log.gz 2023-11-24T08:00:58.528 INFO:teuthology.orchestra.run.smithi059.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.120949.log.gz 2023-11-24T08:00:58.528 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.131780.log 2023-11-24T08:00:58.528 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.123508.log 2023-11-24T08:00:58.529 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.131780.log: /var/log/ceph/ceph-client.admin.123508.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.131780.log.gz 2023-11-24T08:00:58.529 INFO:teuthology.orchestra.run.smithi059.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.123508.log.gz 2023-11-24T08:00:58.529 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.120481.log 2023-11-24T08:00:58.530 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.1.110797.log: 0.0% -- replaced with /var/log/ceph/ceph-client.1.110797.log.gz 2023-11-24T08:00:58.530 INFO:teuthology.orchestra.run.smithi076.stderr: 95.5% -- replaced with /var/log/ceph/ceph-mds.b.log.gz 2023-11-24T08:00:58.555 INFO:teuthology.orchestra.run.smithi076.stderr: 91.6% -- replaced with /var/log/ceph/ceph-mon.c.log.gz 2023-11-24T08:00:58.556 INFO:teuthology.orchestra.run.smithi076.stderr: 92.3% -- replaced with /var/log/ceph/ceph-mgr.x.log.gz 2023-11-24T08:00:58.565 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph.audit.log 2023-11-24T08:00:58.565 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.120481.log: /var/log/ceph/ceph.audit.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.120481.log.gz 2023-11-24T08:00:58.565 INFO:teuthology.orchestra.run.smithi059.stderr: 89.4% -- replaced with /var/log/ceph/ceph.audit.log.gz 2023-11-24T08:00:58.565 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.121582.log 2023-11-24T08:00:58.565 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.135749.log 2023-11-24T08:00:58.565 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.121582.log: /var/log/ceph/ceph-client.admin.135749.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.135749.log.gz 2023-11-24T08:00:58.565 INFO:teuthology.orchestra.run.smithi059.stderr: 91.2% -- replaced with /var/log/ceph/ceph-client.admin.121582.log.gz 2023-11-24T08:00:58.566 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.117122.log 2023-11-24T08:00:58.566 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.115241.log 2023-11-24T08:00:58.566 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.117122.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.117122.log.gz 2023-11-24T08:00:58.566 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.115241.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.115241.log.gz 2023-11-24T08:00:58.566 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.132920.log 2023-11-24T08:00:58.566 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.126390.log 2023-11-24T08:00:58.566 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.132920.log: /var/log/ceph/ceph-client.admin.126390.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.132920.log.gz 2023-11-24T08:00:58.566 INFO:teuthology.orchestra.run.smithi059.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.126390.log.gz 2023-11-24T08:00:58.567 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.112235.log 2023-11-24T08:00:58.567 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.114541.log 2023-11-24T08:00:58.567 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.112235.log: /var/log/ceph/ceph-client.admin.114541.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.112235.log.gz 2023-11-24T08:00:58.567 INFO:teuthology.orchestra.run.smithi059.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.114541.log.gz 2023-11-24T08:00:58.567 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.130254.log 2023-11-24T08:00:58.567 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.114771.log 2023-11-24T08:00:58.567 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.130254.log: /var/log/ceph/ceph-client.admin.114771.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.130254.log.gz 2023-11-24T08:00:58.567 INFO:teuthology.orchestra.run.smithi059.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.114771.log.gz 2023-11-24T08:00:58.568 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.117214.log 2023-11-24T08:00:58.568 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.135353.log 2023-11-24T08:00:58.568 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.117214.log: 88.3%/var/log/ceph/ceph-client.admin.135353.log: -- replaced with /var/log/ceph/ceph-client.admin.117214.log.gz 2023-11-24T08:00:58.568 INFO:teuthology.orchestra.run.smithi059.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.135353.log.gz 2023-11-24T08:00:58.568 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.122317.log 2023-11-24T08:00:58.568 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.135537.log 2023-11-24T08:00:58.568 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.122317.log: /var/log/ceph/ceph-client.admin.135537.log: 86.9% -- replaced with /var/log/ceph/ceph-client.admin.122317.log.gz 2023-11-24T08:00:58.568 INFO:teuthology.orchestra.run.smithi059.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.135537.log.gz 2023-11-24T08:00:58.569 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.127319.log 2023-11-24T08:00:58.569 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.124100.log 2023-11-24T08:00:58.569 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.127319.log: /var/log/ceph/ceph-client.admin.124100.log: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.127319.log.gz 2023-11-24T08:00:58.569 INFO:teuthology.orchestra.run.smithi059.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.124100.log.gz 2023-11-24T08:00:58.569 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.125877.log 2023-11-24T08:00:58.569 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.122179.log 2023-11-24T08:00:58.569 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.125877.log: /var/log/ceph/ceph-client.admin.122179.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.125877.log.gz 2023-11-24T08:00:58.569 INFO:teuthology.orchestra.run.smithi059.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.122179.log.gz 2023-11-24T08:00:58.570 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.124864.log 2023-11-24T08:00:58.570 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.116348.log 2023-11-24T08:00:58.570 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.124864.log: /var/log/ceph/ceph-client.admin.116348.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.124864.log.gz 2023-11-24T08:00:58.570 INFO:teuthology.orchestra.run.smithi059.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.116348.log.gz 2023-11-24T08:00:58.570 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.132531.log 2023-11-24T08:00:58.570 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.124008.log 2023-11-24T08:00:58.570 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.132531.log: /var/log/ceph/ceph-client.admin.124008.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.132531.log.gz 2023-11-24T08:00:58.571 INFO:teuthology.orchestra.run.smithi059.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.124008.log.gz 2023-11-24T08:00:58.571 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.116578.log 2023-11-24T08:00:58.571 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.121352.log 2023-11-24T08:00:58.571 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.116578.log: /var/log/ceph/ceph-client.admin.121352.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.116578.log.gz 2023-11-24T08:00:58.571 INFO:teuthology.orchestra.run.smithi059.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.121352.log.gz 2023-11-24T08:00:58.571 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.109671.log 2023-11-24T08:00:58.571 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.130595.log 2023-11-24T08:00:58.571 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.109671.log: /var/log/ceph/ceph-client.admin.130595.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.109671.log.gz 2023-11-24T08:00:58.572 INFO:teuthology.orchestra.run.smithi059.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.130595.log.gz 2023-11-24T08:00:58.572 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.128205.log 2023-11-24T08:00:58.572 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.119956.log 2023-11-24T08:00:58.572 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.128205.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.128205.log.gz 2023-11-24T08:00:58.572 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.122087.log 2023-11-24T08:00:58.572 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.119956.log: /var/log/ceph/ceph-client.admin.122087.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.119956.log.gz 2023-11-24T08:00:58.572 INFO:teuthology.orchestra.run.smithi059.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.122087.log.gz 2023-11-24T08:00:58.572 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.111146.log 2023-11-24T08:00:58.573 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.133825.log 2023-11-24T08:00:58.573 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.111146.log: /var/log/ceph/ceph-client.admin.133825.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.111146.log.gz 2023-11-24T08:00:58.573 INFO:teuthology.orchestra.run.smithi059.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.133825.log.gz 2023-11-24T08:00:58.573 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.131642.log 2023-11-24T08:00:58.573 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.121018.log 2023-11-24T08:00:58.573 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.131642.log: /var/log/ceph/ceph-client.admin.121018.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.131642.log.gz 2023-11-24T08:00:58.573 INFO:teuthology.orchestra.run.smithi059.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.121018.log.gz 2023-11-24T08:00:58.573 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.121260.log 2023-11-24T08:00:58.574 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.111512.log 2023-11-24T08:00:58.574 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.121260.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.121260.log.gz 2023-11-24T08:00:58.574 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.111512.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.111512.log.gz 2023-11-24T08:00:58.574 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.110775.log 2023-11-24T08:00:58.574 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.122271.log 2023-11-24T08:00:58.574 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.110775.log: /var/log/ceph/ceph-client.admin.122271.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.110775.log.gz 2023-11-24T08:00:58.574 INFO:teuthology.orchestra.run.smithi059.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.122271.log.gz 2023-11-24T08:00:58.575 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.2.113720.log 2023-11-24T08:00:58.575 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.114679.log 2023-11-24T08:00:58.575 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.2.113720.log: 0.0% -- replaced with /var/log/ceph/ceph-client.2.113720.log.gz 2023-11-24T08:00:58.575 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.114679.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.114679.log.gz 2023-11-24T08:00:58.575 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.115892.log 2023-11-24T08:00:58.575 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.132231.log 2023-11-24T08:00:58.575 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.115892.log: /var/log/ceph/ceph-client.admin.132231.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.115892.log.gz 2023-11-24T08:00:58.575 INFO:teuthology.orchestra.run.smithi059.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.132231.log.gz 2023-11-24T08:00:58.576 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.114495.log 2023-11-24T08:00:58.576 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.111905.log 2023-11-24T08:00:58.576 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.114495.log: /var/log/ceph/ceph-client.admin.111905.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.114495.log.gz 2023-11-24T08:00:58.576 INFO:teuthology.orchestra.run.smithi059.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.111905.log.gz 2023-11-24T08:00:58.576 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.126298.log 2023-11-24T08:00:58.576 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.109860.log 2023-11-24T08:00:58.576 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.126298.log: /var/log/ceph/ceph-client.admin.109860.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.126298.log.gz 2023-11-24T08:00:58.576 INFO:teuthology.orchestra.run.smithi059.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.109860.log.gz 2023-11-24T08:00:58.577 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.126060.log 2023-11-24T08:00:58.577 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.111767.log 2023-11-24T08:00:58.577 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.126060.log: /var/log/ceph/ceph-client.admin.111767.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.126060.log.gz 2023-11-24T08:00:58.577 INFO:teuthology.orchestra.run.smithi059.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.111767.log.gz 2023-11-24T08:00:58.577 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.110763.log 2023-11-24T08:00:58.577 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.127503.log 2023-11-24T08:00:58.577 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.110763.log: /var/log/ceph/ceph-client.admin.127503.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.110763.log.gz 2023-11-24T08:00:58.577 INFO:teuthology.orchestra.run.smithi059.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.127503.log.gz 2023-11-24T08:00:58.578 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.109998.log 2023-11-24T08:00:58.578 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.133258.log 2023-11-24T08:00:58.578 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.109998.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.109998.log.gz 2023-11-24T08:00:58.578 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.115800.log 2023-11-24T08:00:58.578 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.133258.log: 88.7%/var/log/ceph/ceph-client.admin.115800.log: -- replaced with /var/log/ceph/ceph-client.admin.133258.log.gz 2023-11-24T08:00:58.578 INFO:teuthology.orchestra.run.smithi059.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.115800.log.gz 2023-11-24T08:00:58.578 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.134304.log 2023-11-24T08:00:58.579 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.120251.log 2023-11-24T08:00:58.579 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.134304.log: /var/log/ceph/ceph-client.admin.120251.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.134304.log.gz 2023-11-24T08:00:58.579 INFO:teuthology.orchestra.run.smithi059.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.120251.log.gz 2023-11-24T08:00:58.579 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.111141.log 2023-11-24T08:00:58.579 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.111514.log 2023-11-24T08:00:58.579 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.111141.log: /var/log/ceph/ceph-client.admin.111514.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.111141.log.gz 2023-11-24T08:00:58.579 INFO:teuthology.orchestra.run.smithi059.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.111514.log.gz 2023-11-24T08:00:58.579 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.132415.log 2023-11-24T08:00:58.580 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.123802.log 2023-11-24T08:00:58.580 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.132415.log: /var/log/ceph/ceph-client.admin.123802.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.132415.log.gz 2023-11-24T08:00:58.580 INFO:teuthology.orchestra.run.smithi059.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.123802.log.gz 2023-11-24T08:00:58.580 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.114310.log 2023-11-24T08:00:58.580 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.127925.log 2023-11-24T08:00:58.580 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.114310.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.114310.log.gz 2023-11-24T08:00:58.580 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.127925.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.127925.log.gz 2023-11-24T08:00:58.580 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.124660.log 2023-11-24T08:00:58.581 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.1.log 2023-11-24T08:00:58.581 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.124660.log: /var/log/ceph/ceph-osd.1.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.124660.log.gz 2023-11-24T08:00:58.581 INFO:teuthology.orchestra.run.smithi059.stderr: 93.2% -- replaced with /var/log/ceph/ceph-mds.a.log.gz 2023-11-24T08:00:58.581 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.113988.log 2023-11-24T08:00:58.581 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.114357.log 2023-11-24T08:00:58.581 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.113988.log: /var/log/ceph/ceph-client.admin.114357.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.113988.log.gz 2023-11-24T08:00:58.581 INFO:teuthology.orchestra.run.smithi059.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.114357.log.gz 2023-11-24T08:00:58.581 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.110274.log 2023-11-24T08:00:58.582 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.109296.log 2023-11-24T08:00:58.582 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.110274.log: /var/log/ceph/ceph-client.admin.109296.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.110274.log.gz 2023-11-24T08:00:58.582 INFO:teuthology.orchestra.run.smithi059.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.109296.log.gz 2023-11-24T08:00:58.582 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.110525.log 2023-11-24T08:00:58.582 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.112051.log 2023-11-24T08:00:58.582 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.110525.log: /var/log/ceph/ceph-client.admin.112051.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.110525.log.gz 2023-11-24T08:00:58.582 INFO:teuthology.orchestra.run.smithi059.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.112051.log.gz 2023-11-24T08:00:58.583 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.121652.log 2023-11-24T08:00:58.583 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.132461.log 2023-11-24T08:00:58.583 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.121652.log: /var/log/ceph/ceph-client.admin.132461.log: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.121652.log.gz 2023-11-24T08:00:58.583 INFO:teuthology.orchestra.run.smithi059.stderr: 90.8% -- replaced with /var/log/ceph/ceph-client.admin.132461.log.gz 2023-11-24T08:00:58.583 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.134985.log 2023-11-24T08:00:58.583 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.134985.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.134985.log.gz 2023-11-24T08:00:58.583 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.119523.log 2023-11-24T08:00:58.583 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.123554.log 2023-11-24T08:00:58.584 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.119523.log: /var/log/ceph/ceph-client.admin.123554.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.119523.log.gz 2023-11-24T08:00:58.584 INFO:teuthology.orchestra.run.smithi059.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.123554.log.gz 2023-11-24T08:00:58.584 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.114080.log 2023-11-24T08:00:58.584 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.127749.log 2023-11-24T08:00:58.584 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.114080.log: /var/log/ceph/ceph-client.admin.127749.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.114080.log.gz 2023-11-24T08:00:58.584 INFO:teuthology.orchestra.run.smithi059.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.127749.log.gz 2023-11-24T08:00:58.584 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.121120.log 2023-11-24T08:00:58.584 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.133304.log 2023-11-24T08:00:58.585 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.121120.log: /var/log/ceph/ceph-client.admin.133304.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.121120.log.gz 2023-11-24T08:00:58.585 INFO:teuthology.orchestra.run.smithi059.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.133304.log.gz 2023-11-24T08:00:58.585 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.129723.log 2023-11-24T08:00:58.585 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.126957.log 2023-11-24T08:00:58.585 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.129723.log: /var/log/ceph/ceph-client.admin.126957.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.129723.log.gz 2023-11-24T08:00:58.585 INFO:teuthology.orchestra.run.smithi059.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.126957.log.gz 2023-11-24T08:00:58.585 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.125280.log 2023-11-24T08:00:58.585 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.117540.log 2023-11-24T08:00:58.586 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.125280.log: /var/log/ceph/ceph-client.admin.117540.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.117540.log.gz 2023-11-24T08:00:58.586 INFO:teuthology.orchestra.run.smithi059.stderr: 90.8% -- replaced with /var/log/ceph/ceph-client.admin.125280.log.gzgzip 2023-11-24T08:00:58.586 INFO:teuthology.orchestra.run.smithi059.stderr:-5 --verbose -- /var/log/ceph/ceph-client.admin.123075.log 2023-11-24T08:00:58.586 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.133530.log 2023-11-24T08:00:58.586 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.123075.log: /var/log/ceph/ceph-client.admin.133530.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.123075.log.gz 2023-11-24T08:00:58.586 INFO:teuthology.orchestra.run.smithi059.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.133530.log.gz 2023-11-24T08:00:58.586 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.120002.log 2023-11-24T08:00:58.586 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.123278.log 2023-11-24T08:00:58.587 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.120002.log: /var/log/ceph/ceph-client.admin.123278.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.120002.log.gz 2023-11-24T08:00:58.587 INFO:teuthology.orchestra.run.smithi059.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.123278.log.gz 2023-11-24T08:00:58.587 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.110770.log 2023-11-24T08:00:58.587 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.115846.log 2023-11-24T08:00:58.587 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.110770.log: /var/log/ceph/ceph-client.admin.115846.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.110770.log.gz 2023-11-24T08:00:58.587 INFO:teuthology.orchestra.run.smithi059.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.115846.log.gz 2023-11-24T08:00:58.587 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.130733.log 2023-11-24T08:00:58.587 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.124910.log 2023-11-24T08:00:58.588 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.130733.log: /var/log/ceph/ceph-client.admin.124910.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.130733.log.gz 2023-11-24T08:00:58.588 INFO:teuthology.orchestra.run.smithi059.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.124910.log.gz 2023-11-24T08:00:58.588 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.133058.log 2023-11-24T08:00:58.588 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.127049.log 2023-11-24T08:00:58.588 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.133058.log: /var/log/ceph/ceph-client.admin.127049.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.133058.log.gz 2023-11-24T08:00:58.588 INFO:teuthology.orchestra.run.smithi059.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.127049.log.gz 2023-11-24T08:00:58.588 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.122385.log 2023-11-24T08:00:58.589 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.125350.log 2023-11-24T08:00:58.589 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.122385.log: /var/log/ceph/ceph-client.admin.125350.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.122385.log.gz 2023-11-24T08:00:58.589 INFO:teuthology.orchestra.run.smithi059.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.125350.log.gz 2023-11-24T08:00:58.589 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.129456.log 2023-11-24T08:00:58.589 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.124614.log 2023-11-24T08:00:58.589 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.129456.log: /var/log/ceph/ceph-client.admin.124614.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.129456.log.gz 2023-11-24T08:00:58.589 INFO:teuthology.orchestra.run.smithi059.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.124614.log.gz 2023-11-24T08:00:58.589 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.134101.log 2023-11-24T08:00:58.590 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.127251.log 2023-11-24T08:00:58.590 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.134101.log: /var/log/ceph/ceph-client.admin.127251.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.134101.log.gz 2023-11-24T08:00:58.590 INFO:teuthology.orchestra.run.smithi059.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.127251.log.gz 2023-11-24T08:00:58.590 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.109405.log 2023-11-24T08:00:58.590 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.111508.log 2023-11-24T08:00:58.590 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.109405.log: /var/log/ceph/ceph-client.admin.111508.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.109405.log.gz 2023-11-24T08:00:58.590 INFO:teuthology.orchestra.run.smithi059.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.111508.log.gz 2023-11-24T08:00:58.590 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.127839.log 2023-11-24T08:00:58.591 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.131220.log 2023-11-24T08:00:58.591 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.127839.log: /var/log/ceph/ceph-client.admin.131220.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.127839.log.gz 2023-11-24T08:00:58.591 INFO:teuthology.orchestra.run.smithi059.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.131220.log.gz 2023-11-24T08:00:58.591 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.109625.log 2023-11-24T08:00:58.591 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.110228.log 2023-11-24T08:00:58.591 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.109625.log: /var/log/ceph/ceph-client.admin.110228.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.109625.log.gz 2023-11-24T08:00:58.591 INFO:teuthology.orchestra.run.smithi059.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.110228.log.gz 2023-11-24T08:00:58.592 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.128157.log 2023-11-24T08:00:58.592 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.120589.log 2023-11-24T08:00:58.592 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.128157.log: /var/log/ceph/ceph-client.admin.120589.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.128157.log.gz 2023-11-24T08:00:58.592 INFO:teuthology.orchestra.run.smithi059.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.120589.log.gz 2023-11-24T08:00:58.592 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.119251.log 2023-11-24T08:00:58.592 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.126344.log 2023-11-24T08:00:58.592 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.119251.log: /var/log/ceph/ceph-client.admin.126344.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.119251.log.gz 2023-11-24T08:00:58.593 INFO:teuthology.orchestra.run.smithi059.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.126344.log.gz 2023-11-24T08:00:58.593 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.118033.log 2023-11-24T08:00:58.593 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.120205.log 2023-11-24T08:00:58.593 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.118033.log: /var/log/ceph/ceph-client.admin.120205.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.118033.log.gz 2023-11-24T08:00:58.593 INFO:teuthology.orchestra.run.smithi059.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.120205.log.gz 2023-11-24T08:00:58.593 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.118722.log 2023-11-24T08:00:58.593 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.134755.log 2023-11-24T08:00:58.593 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.118722.log: /var/log/ceph/ceph-client.admin.134755.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.118722.log.gz 2023-11-24T08:00:58.594 INFO:teuthology.orchestra.run.smithi059.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.134755.log.gz 2023-11-24T08:00:58.594 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.115333.log 2023-11-24T08:00:58.594 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.120679.log 2023-11-24T08:00:58.594 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.115333.log: /var/log/ceph/ceph-client.admin.120679.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.115333.log.gz 2023-11-24T08:00:58.594 INFO:teuthology.orchestra.run.smithi059.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.120679.log.gz 2023-11-24T08:00:58.594 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.135261.log 2023-11-24T08:00:58.594 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.134893.log 2023-11-24T08:00:58.594 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.135261.log: /var/log/ceph/ceph-client.admin.134893.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.135261.log.gz 2023-11-24T08:00:58.595 INFO:teuthology.orchestra.run.smithi059.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.134893.log.gz 2023-11-24T08:00:58.595 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.114909.log 2023-11-24T08:00:58.595 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.132768.log 2023-11-24T08:00:58.595 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.114909.log: /var/log/ceph/ceph-client.0.132768.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.114909.log.gz 2023-11-24T08:00:58.595 INFO:teuthology.orchestra.run.smithi059.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.0.132768.log.gz 2023-11-24T08:00:58.595 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.134350.log 2023-11-24T08:00:58.595 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.130641.log 2023-11-24T08:00:58.595 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.134350.log: /var/log/ceph/ceph-client.admin.130641.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.134350.log.gz 2023-11-24T08:00:58.596 INFO:teuthology.orchestra.run.smithi059.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.130641.log.gz 2023-11-24T08:00:58.596 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.113055.log 2023-11-24T08:00:58.596 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.129502.log 2023-11-24T08:00:58.596 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.0.113055.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.113055.log.gz 2023-11-24T08:00:58.596 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.129502.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.129502.log.gz 2023-11-24T08:00:58.596 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.134193.log 2023-11-24T08:00:58.596 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.130981.log 2023-11-24T08:00:58.596 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.134193.log: /var/log/ceph/ceph-client.admin.130981.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.134193.log.gz 2023-11-24T08:00:58.597 INFO:teuthology.orchestra.run.smithi059.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.130981.log.gz 2023-11-24T08:00:58.597 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.112327.log 2023-11-24T08:00:58.597 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.120159.log 2023-11-24T08:00:58.597 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.112327.log: /var/log/ceph/ceph-client.admin.120159.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.112327.log.gz 2023-11-24T08:00:58.597 INFO:teuthology.orchestra.run.smithi059.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.120159.log.gz 2023-11-24T08:00:58.597 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.2.113716.log 2023-11-24T08:00:58.597 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.114264.log 2023-11-24T08:00:58.598 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.2.113716.log: 0.0% -- replaced with /var/log/ceph/ceph-client.2.113716.log.gz 2023-11-24T08:00:58.598 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.114264.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.114264.log.gz 2023-11-24T08:00:58.598 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.135491.log 2023-11-24T08:00:58.598 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.127457.log 2023-11-24T08:00:58.598 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.135491.log: 87.3%/var/log/ceph/ceph-client.admin.127457.log: -- replaced with /var/log/ceph/ceph-client.admin.135491.log.gz 2023-11-24T08:00:58.598 INFO:teuthology.orchestra.run.smithi059.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.127457.log.gz 2023-11-24T08:00:58.598 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.123324.log 2023-11-24T08:00:58.598 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.111511.log 2023-11-24T08:00:58.599 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.123324.log: /var/log/ceph/ceph-client.admin.111511.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.123324.log.gz 2023-11-24T08:00:58.599 INFO:teuthology.orchestra.run.smithi059.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.111511.log.gz 2023-11-24T08:00:58.599 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.132966.log 2023-11-24T08:00:58.599 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.116440.log 2023-11-24T08:00:58.599 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.132966.log: /var/log/ceph/ceph-client.admin.116440.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.132966.log.gz 2023-11-24T08:00:58.599 INFO:teuthology.orchestra.run.smithi059.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.116440.log.gz 2023-11-24T08:00:58.599 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.116256.log 2023-11-24T08:00:58.599 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.119477.log 2023-11-24T08:00:58.600 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.116256.log: /var/log/ceph/ceph-client.admin.119477.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.116256.log.gz 2023-11-24T08:00:58.600 INFO:teuthology.orchestra.run.smithi059.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.119477.log.gz 2023-11-24T08:00:58.600 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.125188.log 2023-11-24T08:00:58.600 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.131826.log 2023-11-24T08:00:58.600 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.125188.log: /var/log/ceph/ceph-client.admin.131826.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.125188.log.gz 2023-11-24T08:00:58.600 INFO:teuthology.orchestra.run.smithi059.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.131826.log.gz 2023-11-24T08:00:58.600 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.133396.log 2023-11-24T08:00:58.600 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.135307.log 2023-11-24T08:00:58.601 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.133396.log: /var/log/ceph/ceph-client.admin.135307.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.133396.log.gz 2023-11-24T08:00:58.601 INFO:teuthology.orchestra.run.smithi059.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.135307.log.gz 2023-11-24T08:00:58.601 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.126015.log 2023-11-24T08:00:58.601 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.120435.log 2023-11-24T08:00:58.601 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.126015.log: /var/log/ceph/ceph-client.admin.120435.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.126015.log.gz 2023-11-24T08:00:58.601 INFO:teuthology.orchestra.run.smithi059.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.120435.log.gz 2023-11-24T08:00:58.601 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.134055.log 2023-11-24T08:00:58.601 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.134847.log 2023-11-24T08:00:58.602 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.134055.log: /var/log/ceph/ceph-client.admin.134847.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.134055.log.gz 2023-11-24T08:00:58.602 INFO:teuthology.orchestra.run.smithi059.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.134847.log.gz 2023-11-24T08:00:58.602 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.128583.log 2023-11-24T08:00:58.602 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.128583.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.128583.log.gz 2023-11-24T08:00:58.602 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.110044.log 2023-11-24T08:00:58.602 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.123962.log 2023-11-24T08:00:58.609 INFO:teuthology.orchestra.run.smithi076.stderr: 93.6% -- replaced with /var/log/ceph/ceph-osd.7.log.gz 2023-11-24T08:00:58.617 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.110044.log: /var/log/ceph/ceph-client.admin.123962.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.110044.log.gz 2023-11-24T08:00:58.617 INFO:teuthology.orchestra.run.smithi059.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.123962.log.gz 2023-11-24T08:00:58.617 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.113063.log 2023-11-24T08:00:58.617 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.121889.log 2023-11-24T08:00:58.617 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.0.113063.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.113063.log.gz 2023-11-24T08:00:58.617 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.0.121889.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.121889.log.gz 2023-11-24T08:00:58.618 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.115467.log 2023-11-24T08:00:58.618 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.131082.log 2023-11-24T08:00:58.618 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.115467.log: /var/log/ceph/ceph-client.admin.131082.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.115467.log.gz 2023-11-24T08:00:58.618 INFO:teuthology.orchestra.run.smithi059.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.131082.log.gz 2023-11-24T08:00:58.618 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.130074.log 2023-11-24T08:00:58.618 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.118566.log 2023-11-24T08:00:58.618 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.130074.log: /var/log/ceph/ceph-client.0.118566.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.130074.log.gz 2023-11-24T08:00:58.618 INFO:teuthology.orchestra.run.smithi059.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.0.118566.log.gz 2023-11-24T08:00:58.619 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.116072.log 2023-11-24T08:00:58.619 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.117640.log 2023-11-24T08:00:58.619 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.116072.log: /var/log/ceph/ceph-client.admin.117640.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.116072.log.gz 2023-11-24T08:00:58.619 INFO:teuthology.orchestra.run.smithi059.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.117640.log.gz 2023-11-24T08:00:58.619 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.133104.log 2023-11-24T08:00:58.619 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.128975.log 2023-11-24T08:00:58.619 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.133104.log: /var/log/ceph/ceph-client.admin.128975.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.133104.log.gz 2023-11-24T08:00:58.620 INFO:teuthology.orchestra.run.smithi059.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.128975.log.gz 2023-11-24T08:00:58.620 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.125004.log 2023-11-24T08:00:58.620 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.115195.log 2023-11-24T08:00:58.620 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.125004.log: /var/log/ceph/ceph-client.admin.115195.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.125004.log.gz 2023-11-24T08:00:58.620 INFO:teuthology.orchestra.run.smithi059.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.115195.log.gz 2023-11-24T08:00:58.620 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.128675.log 2023-11-24T08:00:58.620 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.119431.log 2023-11-24T08:00:58.620 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.128675.log: /var/log/ceph/ceph-client.admin.119431.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.128675.log.gz 2023-11-24T08:00:58.621 INFO:teuthology.orchestra.run.smithi059.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.119431.log.gz 2023-11-24T08:00:58.621 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.120765.log 2023-11-24T08:00:58.621 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.121398.log 2023-11-24T08:00:58.621 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.120765.log: /var/log/ceph/ceph-client.admin.121398.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.120765.log.gz 2023-11-24T08:00:58.621 INFO:teuthology.orchestra.run.smithi059.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.121398.log.gz 2023-11-24T08:00:58.621 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.132874.log 2023-11-24T08:00:58.621 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.114976.log 2023-11-24T08:00:58.621 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.132874.log: /var/log/ceph/ceph-client.admin.114976.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.132874.log.gz 2023-11-24T08:00:58.622 INFO:teuthology.orchestra.run.smithi059.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.114976.log.gz 2023-11-24T08:00:58.622 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.115149.log 2023-11-24T08:00:58.622 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.118171.log 2023-11-24T08:00:58.622 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.115149.log: /var/log/ceph/ceph-client.admin.118171.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.115149.log.gz 2023-11-24T08:00:58.622 INFO:teuthology.orchestra.run.smithi059.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.118171.log.gz 2023-11-24T08:00:58.622 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.112281.log 2023-11-24T08:00:58.622 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.103635.log 2023-11-24T08:00:58.623 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.112281.log: /var/log/ceph/ceph-client.admin.103635.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.112281.log.gz 2023-11-24T08:00:58.623 INFO:teuthology.orchestra.run.smithi059.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.103635.log.gz 2023-11-24T08:00:58.623 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.111506.log 2023-11-24T08:00:58.623 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.131874.log 2023-11-24T08:00:58.623 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.111506.log: /var/log/ceph/ceph-client.admin.131874.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.111506.log.gz 2023-11-24T08:00:58.623 INFO:teuthology.orchestra.run.smithi059.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.131874.log.gz 2023-11-24T08:00:58.623 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.123692.log 2023-11-24T08:00:58.623 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.128109.log 2023-11-24T08:00:58.624 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.123692.log: /var/log/ceph/ceph-client.admin.128109.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.123692.log.gz 2023-11-24T08:00:58.624 INFO:teuthology.orchestra.run.smithi059.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.128109.log.gz 2023-11-24T08:00:58.624 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.124568.log 2023-11-24T08:00:58.624 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.127641.log 2023-11-24T08:00:58.624 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.124568.log: /var/log/ceph/ceph-client.admin.127641.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.124568.log.gz 2023-11-24T08:00:58.624 INFO:teuthology.orchestra.run.smithi059.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.127641.log.gz 2023-11-24T08:00:58.624 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.134396.log 2023-11-24T08:00:58.624 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.128537.log 2023-11-24T08:00:58.625 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.134396.log: /var/log/ceph/ceph-client.admin.128537.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.134396.log.gz 2023-11-24T08:00:58.625 INFO:teuthology.orchestra.run.smithi059.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.128537.log.gz 2023-11-24T08:00:58.625 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.110773.log 2023-11-24T08:00:58.625 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.112189.log 2023-11-24T08:00:58.625 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.110773.log: /var/log/ceph/ceph-client.admin.112189.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.110773.log.gz 2023-11-24T08:00:58.625 INFO:teuthology.orchestra.run.smithi059.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.112189.log.gz 2023-11-24T08:00:58.625 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.112419.log 2023-11-24T08:00:58.625 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.128305.log 2023-11-24T08:00:58.626 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.112419.log: /var/log/ceph/ceph-client.admin.128305.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.128305.log.gz 2023-11-24T08:00:58.626 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.110571.log 2023-11-24T08:00:58.626 INFO:teuthology.orchestra.run.smithi059.stderr: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.112419.log.gz 2023-11-24T08:00:58.626 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.130825.log 2023-11-24T08:00:58.626 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.110571.log: /var/log/ceph/ceph-client.admin.130825.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.110571.log.gz 2023-11-24T08:00:58.626 INFO:teuthology.orchestra.run.smithi059.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.130825.log.gz 2023-11-24T08:00:58.626 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.111135.log 2023-11-24T08:00:58.627 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.127595.log 2023-11-24T08:00:58.627 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.111135.log: /var/log/ceph/ceph-client.admin.127595.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.111135.log.gz 2023-11-24T08:00:58.627 INFO:teuthology.orchestra.run.smithi059.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.127595.log.gz 2023-11-24T08:00:58.627 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.130687.log 2023-11-24T08:00:58.627 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.122895.log 2023-11-24T08:00:58.627 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.130687.log: /var/log/ceph/ceph-client.admin.122895.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.130687.log.gz 2023-11-24T08:00:58.627 INFO:teuthology.orchestra.run.smithi059.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.122895.log.gz 2023-11-24T08:00:58.627 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.127411.log 2023-11-24T08:00:58.628 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.128445.log 2023-11-24T08:00:58.628 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.127411.log: /var/log/ceph/ceph-client.admin.128445.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.127411.log.gz 2023-11-24T08:00:58.628 INFO:teuthology.orchestra.run.smithi059.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.128445.log.gz 2023-11-24T08:00:58.628 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.118860.log 2023-11-24T08:00:58.628 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.126524.log 2023-11-24T08:00:58.628 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.118860.log: /var/log/ceph/ceph-client.admin.126524.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.118860.log.gz 2023-11-24T08:00:58.628 INFO:teuthology.orchestra.run.smithi059.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.126524.log.gz 2023-11-24T08:00:58.628 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.129807.log 2023-11-24T08:00:58.629 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.131688.log 2023-11-24T08:00:58.629 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.129807.log: /var/log/ceph/ceph-client.admin.131688.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.129807.log.gz 2023-11-24T08:00:58.629 INFO:teuthology.orchestra.run.smithi059.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.131688.log.gz 2023-11-24T08:00:58.629 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.126252.log 2023-11-24T08:00:58.629 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.133350.log 2023-11-24T08:00:58.629 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.126252.log: /var/log/ceph/ceph-client.admin.133350.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.126252.log.gz 2023-11-24T08:00:58.630 INFO:teuthology.orchestra.run.smithi059.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.133350.log.gz 2023-11-24T08:00:58.630 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.110136.log 2023-11-24T08:00:58.630 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.124476.log 2023-11-24T08:00:58.630 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.110136.log: /var/log/ceph/ceph-client.admin.124476.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.110136.log.gz 2023-11-24T08:00:58.630 INFO:teuthology.orchestra.run.smithi059.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.124476.log.gz 2023-11-24T08:00:58.630 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.123121.log 2023-11-24T08:00:58.630 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.121306.log 2023-11-24T08:00:58.630 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.123121.log: /var/log/ceph/ceph-client.admin.121306.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.123121.log.gz 2023-11-24T08:00:58.631 INFO:teuthology.orchestra.run.smithi059.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.121306.log.gz 2023-11-24T08:00:58.631 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.125969.log 2023-11-24T08:00:58.631 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.130457.log 2023-11-24T08:00:58.631 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.125969.log: /var/log/ceph/ceph-client.admin.130457.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.125969.log.gz 2023-11-24T08:00:58.631 INFO:teuthology.orchestra.run.smithi059.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.130457.log.gz 2023-11-24T08:00:58.631 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mon.a.log 2023-11-24T08:00:58.631 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.117444.log 2023-11-24T08:00:58.631 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-mon.a.log: /var/log/ceph/ceph-client.admin.117444.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.117444.log.gz 2023-11-24T08:00:58.632 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.117688.log 2023-11-24T08:00:58.632 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.117688.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.117688.log.gz 2023-11-24T08:00:58.632 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.117987.log 2023-11-24T08:00:58.632 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.117987.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.117987.log.gz 2023-11-24T08:00:58.632 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.110387.log 2023-11-24T08:00:58.632 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.110387.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.110387.log.gz 2023-11-24T08:00:58.632 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.120112.log 2023-11-24T08:00:58.632 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.120112.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.120112.log.gz 2023-11-24T08:00:58.633 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.122133.log 2023-11-24T08:00:58.633 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.122133.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.122133.log.gz 2023-11-24T08:00:58.633 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.110182.log 2023-11-24T08:00:58.633 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.110182.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.110182.log.gz 2023-11-24T08:00:58.633 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.114218.log 2023-11-24T08:00:58.633 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.114218.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.114218.log.gz 2023-11-24T08:00:58.633 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.115513.log 2023-11-24T08:00:58.634 INFO:teuthology.orchestra.run.smithi076.stderr: 90.9% -- replaced with /var/log/ceph/ceph-mon.b.log.gz 2023-11-24T08:00:58.638 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.115513.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.111137.log 2023-11-24T08:00:58.639 INFO:teuthology.orchestra.run.smithi059.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.115513.log.gz 2023-11-24T08:00:58.639 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.111137.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.111137.log.gz 2023-11-24T08:00:58.639 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.109811.log 2023-11-24T08:00:58.639 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.109811.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.109811.log.gz 2023-11-24T08:00:58.639 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.116762.log 2023-11-24T08:00:58.639 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.116762.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.116762.log.gz 2023-11-24T08:00:58.639 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.129548.log 2023-11-24T08:00:58.640 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.129548.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.129548.log.gz 2023-11-24T08:00:58.640 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.110771.log 2023-11-24T08:00:58.640 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.110771.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.110771.log.gz 2023-11-24T08:00:58.640 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.135703.log 2023-11-24T08:00:58.640 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.135703.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.135703.log.gz 2023-11-24T08:00:58.640 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.134663.log 2023-11-24T08:00:58.640 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.134663.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.134663.log.gz 2023-11-24T08:00:58.640 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.123600.log 2023-11-24T08:00:58.641 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.123600.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.123600.log.gz 2023-11-24T08:00:58.641 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.135031.log 2023-11-24T08:00:58.641 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.135031.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.135031.log.gz 2023-11-24T08:00:58.641 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.132116.log 2023-11-24T08:00:58.641 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.132116.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.132116.log.gz 2023-11-24T08:00:58.641 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mgr.y.log 2023-11-24T08:00:58.641 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-mgr.y.log: 93.4% -- replaced with /var/log/ceph/ceph-mgr.y.log.gz 2023-11-24T08:00:58.641 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.130917.log 2023-11-24T08:00:58.642 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.130917.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.130917.log.gz 2023-11-24T08:00:58.642 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.129982.log 2023-11-24T08:00:58.642 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.129982.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.129982.log.gz 2023-11-24T08:00:58.642 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.121166.log 2023-11-24T08:00:58.642 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.121166.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.121166.log.gz 2023-11-24T08:00:58.642 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.127003.log 2023-11-24T08:00:58.642 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.127003.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.127003.log.gz 2023-11-24T08:00:58.643 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.124522.log 2023-11-24T08:00:58.643 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.124522.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.124522.log.gz 2023-11-24T08:00:58.643 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.120857.log 2023-11-24T08:00:58.643 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.131976.log 2023-11-24T08:00:58.644 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.120857.log: /var/log/ceph/ceph-client.admin.131976.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.120857.log.gz 2023-11-24T08:00:58.644 INFO:teuthology.orchestra.run.smithi059.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.131976.log.gz 2023-11-24T08:00:58.644 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.132277.log 2023-11-24T08:00:58.644 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.131922.log 2023-11-24T08:00:58.645 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.132277.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.132277.log.gz 2023-11-24T08:00:58.645 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.131922.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.131922.log.gz 2023-11-24T08:00:58.645 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.119864.log 2023-11-24T08:00:58.646 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.119864.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.119864.log.gz 2023-11-24T08:00:58.646 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.128259.log 2023-11-24T08:00:58.647 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.128259.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.128259.log.gz 2023-11-24T08:00:58.647 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.111138.log 2023-11-24T08:00:58.648 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.111138.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.111138.log.gz 2023-11-24T08:00:58.648 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.109571.log 2023-11-24T08:00:58.649 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.109571.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.109571.log.gz 2023-11-24T08:00:58.649 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.118914.log 2023-11-24T08:00:58.650 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.118914.log: 86.4% -- replaced with /var/log/ceph/ceph-client.admin.118914.log.gz 2023-11-24T08:00:58.657 INFO:teuthology.orchestra.run.smithi076.stderr: 93.7% -- replaced with /var/log/ceph/ceph-osd.6.log.gz 2023-11-24T08:00:58.661 INFO:teuthology.orchestra.run.smithi059.stderr: 93.3% -- replaced with /var/log/ceph/ceph-mds.c.log.gz 2023-11-24T08:00:58.667 INFO:teuthology.orchestra.run.smithi076.stderr: 93.6% -- replaced with /var/log/ceph/ceph-osd.4.log.gz 2023-11-24T08:00:58.670 INFO:teuthology.orchestra.run.smithi059.stderr: 93.7% -- replaced with /var/log/ceph/ceph-osd.2.log.gz 2023-11-24T08:00:58.686 INFO:teuthology.orchestra.run.smithi076.stderr: 93.7% -- replaced with /var/log/ceph/ceph-osd.5.log.gz 2023-11-24T08:00:58.688 INFO:teuthology.orchestra.run.smithi076.stderr: 2023-11-24T08:00:58.688 INFO:teuthology.orchestra.run.smithi076.stderr:real 0m0.335s 2023-11-24T08:00:58.688 INFO:teuthology.orchestra.run.smithi076.stderr:user 0m1.770s 2023-11-24T08:00:58.688 INFO:teuthology.orchestra.run.smithi076.stderr:sys 0m0.084s 2023-11-24T08:00:58.710 INFO:teuthology.orchestra.run.smithi059.stderr: 93.7% -- replaced with /var/log/ceph/ceph-osd.0.log.gz 2023-11-24T08:00:58.747 INFO:teuthology.orchestra.run.smithi059.stderr: 93.7% -- replaced with /var/log/ceph/ceph-osd.3.log.gz 2023-11-24T08:00:58.787 INFO:teuthology.orchestra.run.smithi059.stderr: 93.7% -- replaced with /var/log/ceph/ceph-osd.1.log.gz 2023-11-24T08:00:58.984 INFO:teuthology.orchestra.run.smithi059.stderr: 90.7% -- replaced with /var/log/ceph/ceph-mon.a.log.gz 2023-11-24T08:00:58.986 INFO:teuthology.orchestra.run.smithi059.stderr: 2023-11-24T08:00:58.987 INFO:teuthology.orchestra.run.smithi059.stderr:real 0m0.659s 2023-11-24T08:00:58.987 INFO:teuthology.orchestra.run.smithi059.stderr:user 0m2.220s 2023-11-24T08:00:58.987 INFO:teuthology.orchestra.run.smithi059.stderr:sys 0m0.347s 2023-11-24T08:00:58.988 INFO:tasks.ceph:Archiving logs... 2023-11-24T08:00:58.988 DEBUG:teuthology.misc:Transferring archived files from smithi059:/var/log/ceph to /home/teuthworker/archive/rishabh-2023-11-24_07:31:48-fs:volumes-rishabh-mgr-vol-misc-testing-default-smithi/7465575/remote/smithi059/log 2023-11-24T08:00:58.989 DEBUG:teuthology.orchestra.run.smithi059:> sudo tar c -f - -C /var/log/ceph -- . 2023-11-24T08:00:59.688 DEBUG:teuthology.misc:Transferring archived files from smithi076:/var/log/ceph to /home/teuthworker/archive/rishabh-2023-11-24_07:31:48-fs:volumes-rishabh-mgr-vol-misc-testing-default-smithi/7465575/remote/smithi076/log 2023-11-24T08:00:59.689 DEBUG:teuthology.orchestra.run.smithi076:> sudo tar c -f - -C /var/log/ceph -- . 2023-11-24T08:00:59.961 DEBUG:teuthology.run_tasks:Unwinding manager install 2023-11-24T08:00:59.980 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2023-11-24T08:00:59.981 DEBUG:teuthology.orchestra.run.smithi059:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2023-11-24T08:00:59.983 DEBUG:teuthology.orchestra.run.smithi076:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2023-11-24T08:01:00.066 DEBUG:teuthology.run_tasks:Unwinding manager clock 2023-11-24T08:01:00.075 INFO:teuthology.task.clock:Checking final clock skew... 2023-11-24T08:01:00.075 DEBUG:teuthology.orchestra.run.smithi059:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2023-11-24T08:01:00.078 DEBUG:teuthology.orchestra.run.smithi076:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2023-11-24T08:01:00.095 INFO:teuthology.orchestra.run.smithi059.stderr:bash: ntpq: command not found 2023-11-24T08:01:00.097 INFO:teuthology.orchestra.run.smithi076.stderr:bash: ntpq: command not found 2023-11-24T08:01:00.100 INFO:teuthology.orchestra.run.smithi059.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2023-11-24T08:01:00.100 INFO:teuthology.orchestra.run.smithi059.stdout:=============================================================================== 2023-11-24T08:01:00.100 INFO:teuthology.orchestra.run.smithi059.stdout:^+ hv01.front.sepia.ceph.com 2 6 377 15 +2753us[+2752us] +/- 110ms 2023-11-24T08:01:00.100 INFO:teuthology.orchestra.run.smithi059.stdout:^* hv02.front.sepia.ceph.com 3 6 377 13 -707us[ -708us] +/- 77ms 2023-11-24T08:01:00.100 INFO:teuthology.orchestra.run.smithi059.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 13 -1153us[-1154us] +/- 77ms 2023-11-24T08:01:00.100 INFO:teuthology.orchestra.run.smithi059.stdout:^? hv04.front.sepia.ceph.com 0 8 0 - +0ns[ +0ns] +/- 0ns 2023-11-24T08:01:00.102 INFO:teuthology.orchestra.run.smithi076.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2023-11-24T08:01:00.102 INFO:teuthology.orchestra.run.smithi076.stdout:=============================================================================== 2023-11-24T08:01:00.102 INFO:teuthology.orchestra.run.smithi076.stdout:^+ hv01.front.sepia.ceph.com 2 6 377 11 +2743us[+2743us] +/- 110ms 2023-11-24T08:01:00.102 INFO:teuthology.orchestra.run.smithi076.stdout:^* hv02.front.sepia.ceph.com 3 6 377 12 -693us[ -654us] +/- 77ms 2023-11-24T08:01:00.102 INFO:teuthology.orchestra.run.smithi076.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 13 -1142us[-1102us] +/- 77ms 2023-11-24T08:01:00.102 INFO:teuthology.orchestra.run.smithi076.stdout:^? hv04.front.sepia.ceph.com 0 8 0 - +0ns[ +0ns] +/- 0ns 2023-11-24T08:01:00.103 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2023-11-24T08:01:00.113 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2023-11-24T08:01:00.114 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2023-11-24T08:01:00.123 DEBUG:teuthology.orchestra.run.smithi059:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2023-11-24T08:01:00.259 DEBUG:teuthology.orchestra.run.smithi076:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2023-11-24T08:01:00.361 DEBUG:teuthology.orchestra.run.smithi059:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2023-11-24T08:01:00.396 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-11-24T08:01:00.397 DEBUG:teuthology.orchestra.run.smithi076:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2023-11-24T08:01:00.430 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-11-24T08:01:00.430 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2023-11-24T08:01:00.440 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2023-11-24T08:01:00.449 INFO:teuthology.task.internal:Duration was 1019.666626 seconds 2023-11-24T08:01:00.450 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2023-11-24T08:01:00.458 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2023-11-24T08:01:00.459 DEBUG:teuthology.orchestra.run.smithi059:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2023-11-24T08:01:00.461 DEBUG:teuthology.orchestra.run.smithi076:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2023-11-24T08:01:00.507 INFO:teuthology.orchestra.run.smithi059.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2023-11-24T08:01:00.517 INFO:teuthology.orchestra.run.smithi076.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2023-11-24T08:01:00.878 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2023-11-24T08:01:00.878 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi059.front.sepia.ceph.com 2023-11-24T08:01:00.878 DEBUG:teuthology.orchestra.run.smithi059:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2023-11-24T08:01:00.902 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi076.front.sepia.ceph.com 2023-11-24T08:01:00.902 DEBUG:teuthology.orchestra.run.smithi076:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2023-11-24T08:01:00.924 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2023-11-24T08:01:00.925 DEBUG:teuthology.orchestra.run.smithi059:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2023-11-24T08:01:00.945 DEBUG:teuthology.orchestra.run.smithi076:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2023-11-24T08:01:01.007 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2023-11-24T08:01:01.007 DEBUG:teuthology.orchestra.run.smithi059:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2023-11-24T08:01:01.009 DEBUG:teuthology.orchestra.run.smithi076:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2023-11-24T08:01:01.123 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2023-11-24T08:01:01.134 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2023-11-24T08:01:01.134 DEBUG:teuthology.orchestra.run.smithi059:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2023-11-24T08:01:01.162 DEBUG:teuthology.orchestra.run.smithi076:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2023-11-24T08:01:01.189 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2023-11-24T08:01:01.203 DEBUG:teuthology.orchestra.run.smithi059:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2023-11-24T08:01:01.207 DEBUG:teuthology.orchestra.run.smithi076:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2023-11-24T08:01:01.231 INFO:teuthology.orchestra.run.smithi059.stdout:kernel.core_pattern = core 2023-11-24T08:01:01.256 INFO:teuthology.orchestra.run.smithi076.stdout:kernel.core_pattern = core 2023-11-24T08:01:01.269 DEBUG:teuthology.orchestra.run.smithi059:> test -e /home/ubuntu/cephtest/archive/coredump 2023-11-24T08:01:01.301 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-11-24T08:01:01.302 DEBUG:teuthology.orchestra.run.smithi076:> test -e /home/ubuntu/cephtest/archive/coredump 2023-11-24T08:01:01.325 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-11-24T08:01:01.326 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2023-11-24T08:01:01.340 INFO:teuthology.task.internal:Transferring archived files... 2023-11-24T08:01:01.341 DEBUG:teuthology.misc:Transferring archived files from smithi059:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/rishabh-2023-11-24_07:31:48-fs:volumes-rishabh-mgr-vol-misc-testing-default-smithi/7465575/remote/smithi059 2023-11-24T08:01:01.341 DEBUG:teuthology.orchestra.run.smithi059:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2023-11-24T08:01:01.382 DEBUG:teuthology.misc:Transferring archived files from smithi076:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/rishabh-2023-11-24_07:31:48-fs:volumes-rishabh-mgr-vol-misc-testing-default-smithi/7465575/remote/smithi076 2023-11-24T08:01:01.383 DEBUG:teuthology.orchestra.run.smithi076:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2023-11-24T08:01:01.422 INFO:teuthology.task.internal:Removing archive directory... 2023-11-24T08:01:01.423 DEBUG:teuthology.orchestra.run.smithi059:> rm -rf -- /home/ubuntu/cephtest/archive 2023-11-24T08:01:01.425 DEBUG:teuthology.orchestra.run.smithi076:> rm -rf -- /home/ubuntu/cephtest/archive 2023-11-24T08:01:01.475 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2023-11-24T08:01:01.487 INFO:teuthology.task.internal:Not uploading archives. 2023-11-24T08:01:01.487 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2023-11-24T08:01:01.498 INFO:teuthology.task.internal:Tidying up after the test... 2023-11-24T08:01:01.498 DEBUG:teuthology.orchestra.run.smithi059:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2023-11-24T08:01:01.501 DEBUG:teuthology.orchestra.run.smithi076:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2023-11-24T08:01:01.515 INFO:teuthology.orchestra.run.smithi059.stdout: 262307 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Nov 24 08:01 /home/ubuntu/cephtest 2023-11-24T08:01:01.531 INFO:teuthology.orchestra.run.smithi076.stdout: 262320 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Nov 24 08:01 /home/ubuntu/cephtest 2023-11-24T08:01:01.532 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2023-11-24T08:01:01.544 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2023-11-24T08:01:01.617 INFO:teuthology.nuke:Checking targets against current locks 2023-11-24T08:01:01.644 DEBUG:teuthology.nuke:shortname: smithi059 2023-11-24T08:01:01.645 INFO:teuthology.task.internal.check_lock:Checking locks... 2023-11-24T08:01:01.671 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi059.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/rishabh-2023-11-24_07:31:48-fs:volumes-rishabh-mgr-vol-misc-testing-default-smithi/7465575', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'rhel', 'os_version': '8.6', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-11-24 07:38:24.068042', 'locked_by': 'scheduled_rishabh@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLVvN3+LmzPZ4ixiwyCkS3KZLZ9R6MOPm+kPtaJbl8fExpzbk8TudXCcJWHBL7DGPWFvye2jT6sei9eiX0EUQYs='} 2023-11-24T08:01:01.675 DEBUG:teuthology.nuke:shortname: smithi076 2023-11-24T08:01:01.675 INFO:teuthology.task.internal.check_lock:Checking locks... 2023-11-24T08:01:01.695 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi076.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/rishabh-2023-11-24_07:31:48-fs:volumes-rishabh-mgr-vol-misc-testing-default-smithi/7465575', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'rhel', 'os_version': '8.6', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-11-24 07:38:24.069710', 'locked_by': 'scheduled_rishabh@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLVvN3+LmzPZ4ixiwyCkS3KZLZ9R6MOPm+kPtaJbl8fExpzbk8TudXCcJWHBL7DGPWFvye2jT6sei9eiX0EUQYs='} 2023-11-24T08:01:01.727 INFO:teuthology.orchestra.console.smithi059:Power off 2023-11-24T08:01:01.728 DEBUG:teuthology.orchestra.console.smithi059:pexpect command: ipmitool -H smithi059.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-11-24T08:01:01.751 DEBUG:teuthology.orchestra.console.smithi059:power off output: Chassis Power Control: Down/Off 2023-11-24T08:01:01.751 DEBUG:teuthology.orchestra.console.smithi059:pexpect command: ipmitool -H smithi059.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-11-24T08:01:01.768 INFO:teuthology.orchestra.console.smithi076:Power off 2023-11-24T08:01:01.768 DEBUG:teuthology.orchestra.console.smithi076:pexpect command: ipmitool -H smithi076.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-11-24T08:01:01.778 DEBUG:teuthology.orchestra.console.smithi059:check power output: Chassis Power is on 2023-11-24T08:01:01.792 DEBUG:teuthology.orchestra.console.smithi076:power off output: Chassis Power Control: Down/Off 2023-11-24T08:01:01.792 DEBUG:teuthology.orchestra.console.smithi076:pexpect command: ipmitool -H smithi076.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-11-24T08:01:01.825 DEBUG:teuthology.orchestra.console.smithi076:check power output: Chassis Power is on 2023-11-24T08:01:05.779 DEBUG:teuthology.orchestra.console.smithi059:pexpect command: ipmitool -H smithi059.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-11-24T08:01:05.827 DEBUG:teuthology.orchestra.console.smithi076:pexpect command: ipmitool -H smithi076.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-11-24T08:01:05.889 DEBUG:teuthology.orchestra.console.smithi059:check power output: Chassis Power is on 2023-11-24T08:01:05.937 DEBUG:teuthology.orchestra.console.smithi076:check power output: Chassis Power is on 2023-11-24T08:01:09.892 DEBUG:teuthology.orchestra.console.smithi059:pexpect command: ipmitool -H smithi059.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-11-24T08:01:09.937 DEBUG:teuthology.orchestra.console.smithi076:pexpect command: ipmitool -H smithi076.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-11-24T08:01:10.003 DEBUG:teuthology.orchestra.console.smithi059:check power output: Chassis Power is on 2023-11-24T08:01:10.052 DEBUG:teuthology.orchestra.console.smithi076:check power output: Chassis Power is on 2023-11-24T08:01:14.004 DEBUG:teuthology.orchestra.console.smithi059:pexpect command: ipmitool -H smithi059.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-11-24T08:01:14.053 DEBUG:teuthology.orchestra.console.smithi076:pexpect command: ipmitool -H smithi076.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-11-24T08:01:14.115 DEBUG:teuthology.orchestra.console.smithi059:check power output: Chassis Power is off 2023-11-24T08:01:14.164 DEBUG:teuthology.orchestra.console.smithi076:check power output: Chassis Power is off 2023-11-24T08:01:14.217 INFO:teuthology.orchestra.console.smithi059:Power off completed 2023-11-24T08:01:14.265 INFO:teuthology.orchestra.console.smithi076:Power off completed 2023-11-24T08:01:14.371 INFO:teuthology.run:Summary data: description: fs:volumes/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{centos_latest} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/volumes/{overrides test/snapshot}} duration: 1019.6666262149811 failure_reason: 'Test failure: test_subvolume_inherited_snapshot_ls (tasks.cephfs.test_volumes.TestSubvolumeSnapshots)' flavor: default owner: scheduled_rishabh@teuthology sentry_event: https://sentry.ceph.com/organizations/ceph/?query=309312f7e36d402987032f090b30c5f4 status: fail success: false 2023-11-24T08:01:14.371 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-11-24T08:01:14.504 INFO:teuthology.run:FAIL